2023-12-20T04:10:36.735 INFO:root:teuthology version: 0.0.1.dev216+g5f3731e 2023-12-20T04:10:36.735 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --archive /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395 --description fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} --name vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi --owner scheduled_vshankar@teuthology -- /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395/orig.config.yaml 2023-12-20T04:10:36.786 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-20T04:10:36.872 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395 branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} email: vshankar@redhat.com first_in_suite: false job_id: '7497395' kernel: client: branch: testing kdb: true sha1: distro ktype: upstream last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi name: vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: ubuntu os_version: '22.04' overrides: admin_socket: branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 ceph: conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 pid file: '' rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: debug client: 20 debug finisher: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon op complaint time: 120 osd: bdev async discard: true bdev enable discard: true bluestore allocator: bitmap bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug ms: 1 debug osd: 20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - overall HEALTH_ - \(FS_DEGRADED\) - \(MDS_FAILED\) - \(MDS_DEGRADED\) - \(FS_WITH_FAILED_MDS\) - \(MDS_DAMAGE\) - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(FS_INLINE_DATA_DEPRECATED\) - \(POOL_APP_NOT_ENABLED\) - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding log-whitelist: - OSD full dropping all updates - OSD near full - pausewr flag - failsafe engaged, dropping updates - failsafe disengaged, no longer dropping - is full \(reached quota - POOL_FULL - POOL_BACKFILLFULL sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: bdev async discard: true bdev enable discard: true bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 1/20 debug bluestore: 1/20 debug rocksdb: 4/10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs install: ceph: flavor: default sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e kclient: syntax: v2 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e owner: scheduled_vshankar@teuthology priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.y - mds.a - mds.c - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - client.2 - - mon.b - mon.c - mgr.x - mds.b - mds.d - osd.4 - osd.5 - osd.6 - osd.7 - client.1 - client.3 seed: 430 sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e sleep_before_teardown: 0 subset: 301/1024 suite: fs suite_branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi032.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMVliqZJMAR+8i1UIgvLMOW2HwRq2iziIYQuIUOiaOB3RW74bqK8M0Px+7m7U1vi9gSVfien5K+r+Pq5a8Qk7kI= smithi039.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLNmc+OQbgX+2td9QV+1fnzDuUIn09Fw7zDPWqNzdyN1TBzSxdTCN0yAAQXLDFQO9E0yC2E05BGvJuuIhqwybGY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - ceph: create_rbd_pool: false - kclient: null - cephfs_test_runner: modules: - tasks.cephfs.test_snap_schedules teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73 timestamp: 2023-12-20_03:59:06 tube: smithi use_shaman: true user: vshankar verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.1485078 2023-12-20T04:10:36.872 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa; will attempt to use it 2023-12-20T04:10:36.873 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa/tasks 2023-12-20T04:10:36.873 INFO:teuthology.run_tasks:Running task internal.check_packages... 2023-12-20T04:10:36.875 INFO:teuthology.task.internal:Checking packages... 2023-12-20T04:10:36.900 INFO:teuthology.task.internal:Checking packages for os_type 'ubuntu', flavor 'default' and ceph hash '568df92b7e50b2108bce4c77c393ff294d6a5e5e' 2023-12-20T04:10:36.900 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2023-12-20T04:10:36.900 INFO:teuthology.packaging:ref: None 2023-12-20T04:10:36.900 INFO:teuthology.packaging:tag: None 2023-12-20T04:10:36.900 INFO:teuthology.packaging:branch: wip-vshankar-testing1-2023-12-18-1207-reef-2 2023-12-20T04:10:36.900 INFO:teuthology.packaging:sha1: 568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-20T04:10:36.900 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=wip-vshankar-testing1-2023-12-18-1207-reef-2 2023-12-20T04:10:37.119 INFO:teuthology.task.internal:Found packages for ceph version 18.2.1-132-g568df92b-1jammy 2023-12-20T04:10:37.120 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2023-12-20T04:10:37.128 INFO:teuthology.task.internal:no buildpackages task found 2023-12-20T04:10:37.128 INFO:teuthology.run_tasks:Running task internal.save_config... 2023-12-20T04:10:37.147 INFO:teuthology.task.internal:Saving configuration 2023-12-20T04:10:37.166 INFO:teuthology.run_tasks:Running task internal.check_lock... 2023-12-20T04:10:37.174 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-20T04:10:37.199 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-20 03:59:44.362898', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIPaJ99kPudzMyOtL8cTTPvg+QN3rxCwoJQin4yf866uc'} 2023-12-20T04:10:37.222 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-20 03:59:44.361480', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIC/MKdDppYp/Aq3o2srxnu927wvDTRut5jxVxxb2wnpa'} 2023-12-20T04:10:37.222 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2023-12-20T04:10:37.230 INFO:teuthology.task.internal:roles: ubuntu@smithi032.front.sepia.ceph.com - ['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'] 2023-12-20T04:10:37.231 INFO:teuthology.task.internal:roles: ubuntu@smithi039.front.sepia.ceph.com - ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3'] 2023-12-20T04:10:37.231 INFO:teuthology.run_tasks:Running task console_log... 2023-12-20T04:10:37.311 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f076e1e0b80>, signals=[15]) 2023-12-20T04:10:37.311 INFO:teuthology.run_tasks:Running task internal.connect... 2023-12-20T04:10:37.318 INFO:teuthology.task.internal:Opening connections... 2023-12-20T04:10:37.319 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:10:37.320 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:10:37.399 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T04:10:37.400 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:10:37.474 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2023-12-20T04:10:37.486 DEBUG:teuthology.orchestra.run.smithi032:> uname -m 2023-12-20T04:10:37.494 INFO:teuthology.orchestra.run.smithi032.stdout:x86_64 2023-12-20T04:10:37.494 DEBUG:teuthology.orchestra.run.smithi032:> cat /etc/os-release 2023-12-20T04:10:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-20T04:10:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:NAME="Ubuntu" 2023-12-20T04:10:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_ID="22.04" 2023-12-20T04:10:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-20T04:10:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:VERSION_CODENAME=jammy 2023-12-20T04:10:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:ID=ubuntu 2023-12-20T04:10:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:ID_LIKE=debian 2023-12-20T04:10:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-20T04:10:37.540 INFO:teuthology.orchestra.run.smithi032.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-20T04:10:37.541 INFO:teuthology.orchestra.run.smithi032.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-20T04:10:37.541 INFO:teuthology.orchestra.run.smithi032.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-20T04:10:37.541 INFO:teuthology.orchestra.run.smithi032.stdout:UBUNTU_CODENAME=jammy 2023-12-20T04:10:37.541 INFO:teuthology.lock.ops:Updating smithi032.front.sepia.ceph.com on lock server 2023-12-20T04:10:37.599 DEBUG:teuthology.orchestra.run.smithi039:> uname -m 2023-12-20T04:10:37.606 INFO:teuthology.orchestra.run.smithi039.stdout:x86_64 2023-12-20T04:10:37.606 DEBUG:teuthology.orchestra.run.smithi039:> cat /etc/os-release 2023-12-20T04:10:37.652 INFO:teuthology.orchestra.run.smithi039.stdout:PRETTY_NAME="Ubuntu 22.04.1 LTS" 2023-12-20T04:10:37.652 INFO:teuthology.orchestra.run.smithi039.stdout:NAME="Ubuntu" 2023-12-20T04:10:37.652 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_ID="22.04" 2023-12-20T04:10:37.652 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION="22.04.1 LTS (Jammy Jellyfish)" 2023-12-20T04:10:37.652 INFO:teuthology.orchestra.run.smithi039.stdout:VERSION_CODENAME=jammy 2023-12-20T04:10:37.652 INFO:teuthology.orchestra.run.smithi039.stdout:ID=ubuntu 2023-12-20T04:10:37.652 INFO:teuthology.orchestra.run.smithi039.stdout:ID_LIKE=debian 2023-12-20T04:10:37.653 INFO:teuthology.orchestra.run.smithi039.stdout:HOME_URL="https://www.ubuntu.com/" 2023-12-20T04:10:37.653 INFO:teuthology.orchestra.run.smithi039.stdout:SUPPORT_URL="https://help.ubuntu.com/" 2023-12-20T04:10:37.653 INFO:teuthology.orchestra.run.smithi039.stdout:BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/" 2023-12-20T04:10:37.653 INFO:teuthology.orchestra.run.smithi039.stdout:PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy" 2023-12-20T04:10:37.653 INFO:teuthology.orchestra.run.smithi039.stdout:UBUNTU_CODENAME=jammy 2023-12-20T04:10:37.653 INFO:teuthology.lock.ops:Updating smithi039.front.sepia.ceph.com on lock server 2023-12-20T04:10:37.833 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2023-12-20T04:10:37.845 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2023-12-20T04:10:37.862 INFO:teuthology.task.internal:Checking for old test directory... 2023-12-20T04:10:37.863 DEBUG:teuthology.orchestra.run.smithi032:> test '!' -e /home/ubuntu/cephtest 2023-12-20T04:10:37.865 DEBUG:teuthology.orchestra.run.smithi039:> test '!' -e /home/ubuntu/cephtest 2023-12-20T04:10:37.870 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2023-12-20T04:10:37.881 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2023-12-20T04:10:37.882 DEBUG:teuthology.orchestra.run.smithi032:> test -z $(ls -A /var/lib/ceph) 2023-12-20T04:10:37.914 DEBUG:teuthology.orchestra.run.smithi039:> test -z $(ls -A /var/lib/ceph) 2023-12-20T04:10:37.936 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2023-12-20T04:10:37.993 INFO:teuthology.run_tasks:Running task kernel... 2023-12-20T04:10:38.014 INFO:teuthology.task.kernel:normalize config orig: {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'} 2023-12-20T04:10:38.014 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'branch': 'testing'}, 'client.2': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}, 'client.3': {'branch': 'testing'}} 2023-12-20T04:10:38.014 INFO:teuthology.task.kernel:config {'client.0': {'branch': 'testing'}, 'client.1': {'branch': 'testing'}}, timeout 300 2023-12-20T04:10:38.015 DEBUG:teuthology.orchestra.run.smithi032:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-20T04:10:38.015 DEBUG:teuthology.orchestra.run.smithi039:> test -f /run/.containerenv -o -f /.dockerenv 2023-12-20T04:10:38.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:10:38.020 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2023-12-20T04:10:38.024 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:10:38.025 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&ref=testing 2023-12-20T04:10:38.203 DEBUG:teuthology.task.kernel:sha1 for client.1 is 93572f31db899adbd1540d7954ab247e32731c84 2023-12-20T04:10:38.203 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "93572f31db899adbd1540d7954ab247e32731c84"... 2023-12-20T04:10:38.204 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2023-12-20T04:10:38.209 INFO:teuthology.orchestra.run.smithi039.stdout:5.15.0-56-generic 2023-12-20T04:10:38.209 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 93572f31db899adbd1540d7954ab247e32731c84 2023-12-20T04:10:38.210 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-20T04:10:38.210 INFO:teuthology.task.kernel:Installing linux-firmware on client.1... 2023-12-20T04:10:38.210 DEBUG:teuthology.orchestra.run.smithi039:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-12-20T04:10:38.264 INFO:teuthology.orchestra.run.smithi039.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-12-20T04:10:38.278 DEBUG:teuthology.task.kernel:sha1 for client.0 is 93572f31db899adbd1540d7954ab247e32731c84 2023-12-20T04:10:38.278 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "93572f31db899adbd1540d7954ab247e32731c84"... 2023-12-20T04:10:38.278 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-12-20T04:10:38.284 INFO:teuthology.orchestra.run.smithi032.stdout:5.15.0-56-generic 2023-12-20T04:10:38.284 DEBUG:teuthology.task.kernel:current kernel version is 5.15.0-56-generic vs 93572f31db899adbd1540d7954ab247e32731c84 2023-12-20T04:10:38.284 DEBUG:teuthology.task.kernel:failed to parse current kernel version 2023-12-20T04:10:38.285 INFO:teuthology.task.kernel:Installing linux-firmware on client.0... 2023-12-20T04:10:38.285 DEBUG:teuthology.orchestra.run.smithi032:> cd /lib/firmware/updates && test -d .git && test '!' -s .git/HEAD && sudo rm -rf .git ; sudo install -d -m0755 /lib/firmware/updates && cd /lib/firmware/updates && sudo git init 2023-12-20T04:10:38.358 INFO:teuthology.orchestra.run.smithi032.stderr:bash: line 1: cd: /lib/firmware/updates: No such file or directory 2023-12-20T04:10:38.399 INFO:teuthology.orchestra.run.smithi039.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-12-20T04:10:38.399 INFO:teuthology.orchestra.run.smithi039.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-12-20T04:10:38.400 INFO:teuthology.orchestra.run.smithi039.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-12-20T04:10:38.400 INFO:teuthology.orchestra.run.smithi039.stderr:hint: 2023-12-20T04:10:38.400 INFO:teuthology.orchestra.run.smithi039.stderr:hint: git config --global init.defaultBranch 2023-12-20T04:10:38.400 INFO:teuthology.orchestra.run.smithi039.stderr:hint: 2023-12-20T04:10:38.400 INFO:teuthology.orchestra.run.smithi039.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-12-20T04:10:38.400 INFO:teuthology.orchestra.run.smithi039.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-12-20T04:10:38.400 INFO:teuthology.orchestra.run.smithi039.stderr:hint: 2023-12-20T04:10:38.400 INFO:teuthology.orchestra.run.smithi039.stderr:hint: git branch -m 2023-12-20T04:10:38.400 INFO:teuthology.orchestra.run.smithi039.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-12-20T04:10:38.403 DEBUG:teuthology.orchestra.run.smithi039:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-12-20T04:10:38.469 DEBUG:teuthology.orchestra.run.smithi039:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-12-20T04:10:38.472 INFO:teuthology.orchestra.run.smithi032.stderr:hint: Using 'master' as the name for the initial branch. This default branch name 2023-12-20T04:10:38.472 INFO:teuthology.orchestra.run.smithi032.stderr:hint: is subject to change. To configure the initial branch name to use in all 2023-12-20T04:10:38.472 INFO:teuthology.orchestra.run.smithi032.stderr:hint: of your new repositories, which will suppress this warning, call: 2023-12-20T04:10:38.472 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 2023-12-20T04:10:38.472 INFO:teuthology.orchestra.run.smithi032.stderr:hint: git config --global init.defaultBranch 2023-12-20T04:10:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 2023-12-20T04:10:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:hint: Names commonly chosen instead of 'master' are 'main', 'trunk' and 2023-12-20T04:10:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 'development'. The just-created branch can be renamed via this command: 2023-12-20T04:10:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:hint: 2023-12-20T04:10:38.473 INFO:teuthology.orchestra.run.smithi032.stderr:hint: git branch -m 2023-12-20T04:10:38.473 INFO:teuthology.orchestra.run.smithi032.stdout:Initialized empty Git repository in /usr/lib/firmware/updates/.git/ 2023-12-20T04:10:38.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo git --git-dir=/lib/firmware/updates/.git config --get remote.origin.url >/dev/null || sudo git --git-dir=/lib/firmware/updates/.git remote add origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git 2023-12-20T04:10:38.526 DEBUG:teuthology.orchestra.run.smithi039:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-12-20T04:10:38.542 DEBUG:teuthology.orchestra.run.smithi032:> sudo git --git-dir=/lib/firmware/updates/.git remote set-url origin git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git >/dev/null 2023-12-20T04:10:38.599 DEBUG:teuthology.orchestra.run.smithi032:> cd /lib/firmware/updates && sudo git fetch origin && sudo git reset --hard origin/main 2023-12-20T04:12:05.524 INFO:teuthology.orchestra.run.smithi032.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-12-20T04:12:05.524 INFO:teuthology.orchestra.run.smithi032.stderr: * [new branch] main -> origin/main 2023-12-20T04:12:05.548 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190312 -> 20190312 2023-12-20T04:12:05.549 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190416 -> 20190416 2023-12-20T04:12:05.549 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190514 -> 20190514 2023-12-20T04:12:05.549 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190618 -> 20190618 2023-12-20T04:12:05.549 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190717 -> 20190717 2023-12-20T04:12:05.550 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190815 -> 20190815 2023-12-20T04:12:05.550 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20190923 -> 20190923 2023-12-20T04:12:05.550 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20191022 -> 20191022 2023-12-20T04:12:05.550 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20191215 -> 20191215 2023-12-20T04:12:05.550 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200122 -> 20200122 2023-12-20T04:12:05.550 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200316 -> 20200316 2023-12-20T04:12:05.550 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200421 -> 20200421 2023-12-20T04:12:05.551 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200519 -> 20200519 2023-12-20T04:12:05.551 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200619 -> 20200619 2023-12-20T04:12:05.551 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200721 -> 20200721 2023-12-20T04:12:05.551 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200817 -> 20200817 2023-12-20T04:12:05.551 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20200918 -> 20200918 2023-12-20T04:12:05.552 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201022 -> 20201022 2023-12-20T04:12:05.552 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201118 -> 20201118 2023-12-20T04:12:05.552 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20201218 -> 20201218 2023-12-20T04:12:05.552 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210208 -> 20210208 2023-12-20T04:12:05.552 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210315 -> 20210315 2023-12-20T04:12:05.556 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210511 -> 20210511 2023-12-20T04:12:05.556 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210716 -> 20210716 2023-12-20T04:12:05.556 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210818 -> 20210818 2023-12-20T04:12:05.556 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20210919 -> 20210919 2023-12-20T04:12:05.556 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20211027 -> 20211027 2023-12-20T04:12:05.556 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20211216 -> 20211216 2023-12-20T04:12:05.556 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220209 -> 20220209 2023-12-20T04:12:05.556 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220310 -> 20220310 2023-12-20T04:12:05.556 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220411 -> 20220411 2023-12-20T04:12:05.556 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220509 -> 20220509 2023-12-20T04:12:05.557 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220610 -> 20220610 2023-12-20T04:12:05.557 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220708 -> 20220708 2023-12-20T04:12:05.557 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220815 -> 20220815 2023-12-20T04:12:05.557 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20220913 -> 20220913 2023-12-20T04:12:05.557 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221012 -> 20221012 2023-12-20T04:12:05.557 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221109 -> 20221109 2023-12-20T04:12:05.558 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20221214 -> 20221214 2023-12-20T04:12:05.559 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230117 -> 20230117 2023-12-20T04:12:05.559 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230210 -> 20230210 2023-12-20T04:12:05.560 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230310 -> 20230310 2023-12-20T04:12:05.560 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230404 -> 20230404 2023-12-20T04:12:05.560 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230515 -> 20230515 2023-12-20T04:12:05.560 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230625 -> 20230625 2023-12-20T04:12:05.560 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230804 -> 20230804 2023-12-20T04:12:05.560 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20230919 -> 20230919 2023-12-20T04:12:05.561 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20231030 -> 20231030 2023-12-20T04:12:05.561 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20231111 -> 20231111 2023-12-20T04:12:05.561 INFO:teuthology.orchestra.run.smithi032.stderr: * [new tag] 20231211 -> 20231211 2023-12-20T04:12:08.006 INFO:teuthology.orchestra.run.smithi039.stderr:From git://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware 2023-12-20T04:12:08.006 INFO:teuthology.orchestra.run.smithi039.stderr: * [new branch] main -> origin/main 2023-12-20T04:12:08.039 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20190312 -> 20190312 2023-12-20T04:12:08.039 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20190416 -> 20190416 2023-12-20T04:12:08.039 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20190514 -> 20190514 2023-12-20T04:12:08.039 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20190618 -> 20190618 2023-12-20T04:12:08.040 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20190717 -> 20190717 2023-12-20T04:12:08.040 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20190815 -> 20190815 2023-12-20T04:12:08.040 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20190923 -> 20190923 2023-12-20T04:12:08.040 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20191022 -> 20191022 2023-12-20T04:12:08.040 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20191215 -> 20191215 2023-12-20T04:12:08.041 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20200122 -> 20200122 2023-12-20T04:12:08.041 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20200316 -> 20200316 2023-12-20T04:12:08.041 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20200421 -> 20200421 2023-12-20T04:12:08.041 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20200519 -> 20200519 2023-12-20T04:12:08.041 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20200619 -> 20200619 2023-12-20T04:12:08.042 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20200721 -> 20200721 2023-12-20T04:12:08.042 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20200817 -> 20200817 2023-12-20T04:12:08.042 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20200918 -> 20200918 2023-12-20T04:12:08.042 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20201022 -> 20201022 2023-12-20T04:12:08.042 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20201118 -> 20201118 2023-12-20T04:12:08.043 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20201218 -> 20201218 2023-12-20T04:12:08.044 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20210208 -> 20210208 2023-12-20T04:12:08.044 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20210315 -> 20210315 2023-12-20T04:12:08.044 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20210511 -> 20210511 2023-12-20T04:12:08.044 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20210716 -> 20210716 2023-12-20T04:12:08.045 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20210818 -> 20210818 2023-12-20T04:12:08.045 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20210919 -> 20210919 2023-12-20T04:12:08.045 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20211027 -> 20211027 2023-12-20T04:12:08.045 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20211216 -> 20211216 2023-12-20T04:12:08.045 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20220209 -> 20220209 2023-12-20T04:12:08.045 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20220310 -> 20220310 2023-12-20T04:12:08.046 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20220411 -> 20220411 2023-12-20T04:12:08.047 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20220509 -> 20220509 2023-12-20T04:12:08.047 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20220610 -> 20220610 2023-12-20T04:12:08.047 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20220708 -> 20220708 2023-12-20T04:12:08.047 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20220815 -> 20220815 2023-12-20T04:12:08.047 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20220913 -> 20220913 2023-12-20T04:12:08.047 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20221012 -> 20221012 2023-12-20T04:12:08.047 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20221109 -> 20221109 2023-12-20T04:12:08.047 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20221214 -> 20221214 2023-12-20T04:12:08.047 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20230117 -> 20230117 2023-12-20T04:12:08.048 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20230210 -> 20230210 2023-12-20T04:12:08.048 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20230310 -> 20230310 2023-12-20T04:12:08.048 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20230404 -> 20230404 2023-12-20T04:12:08.048 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20230515 -> 20230515 2023-12-20T04:12:08.048 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20230625 -> 20230625 2023-12-20T04:12:08.048 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20230804 -> 20230804 2023-12-20T04:12:08.049 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20230919 -> 20230919 2023-12-20T04:12:08.049 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20231030 -> 20231030 2023-12-20T04:12:08.049 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20231111 -> 20231111 2023-12-20T04:12:08.049 INFO:teuthology.orchestra.run.smithi039.stderr: * [new tag] 20231211 -> 20231211 2023-12-20T04:12:09.771 INFO:teuthology.orchestra.run.smithi032.stderr:Updating files: 49% (1598/3256) Updating files: 50% (1628/3256) Updating files: 51% (1661/3256) Updating files: 52% (1694/3256) Updating files: 53% (1726/3256) Updating files: 54% (1759/3256) Updating files: 55% (1791/3256) Updating files: 56% (1824/3256) Updating files: 57% (1856/3256) Updating files: 58% (1889/3256) Updating files: 59% (1922/3256) Updating files: 60% (1954/3256) Updating files: 61% (1987/3256) Updating files: 62% (2019/3256) Updating files: 63% (2052/3256) Updating files: 63% (2078/3256) Updating files: 64% (2084/3256) Updating files: 65% (2117/3256) Updating files: 66% (2149/3256) Updating files: 67% (2182/3256) Updating files: 68% (2215/3256) Updating files: 69% (2247/3256) Updating files: 70% (2280/3256) Updating files: 71% (2312/3256) Updating files: 72% (2345/3256) Updating files: 73% (2377/3256) Updating files: 74% (2410/3256) Updating files: 75% (2442/3256) Updating files: 76% (2475/3256) Updating files: 77% (2508/3256) Updating files: 78% (2540/3256) Updating files: 79% (2573/3256) Updating files: 80% (2605/3256) Updating files: 80% (2619/3256) Updating files: 81% (2638/3256) Updating files: 82% (2670/3256) Updating files: 83% (2703/3256) Updating files: 84% (2736/3256) Updating files: 85% (2768/3256) Updating files: 85% (2794/3256) Updating files: 86% (2801/3256) Updating files: 87% (2833/3256) Updating files: 88% (2866/3256) Updating files: 89% (2898/3256) Updating files: 90% (2931/3256) Updating files: 91% (2963/3256) Updating files: 92% (2996/3256) Updating files: 93% (3029/3256) Updating files: 94% (3061/3256) Updating files: 95% (3094/3256) Updating files: 96% (3126/3256) Updating files: 97% (3159/3256) Updating files: 98% (3191/3256) Updating files: 99% (3224/3256) Updating files: 100% (3256/3256) Updating files: 100% (3256/3256), done. 2023-12-20T04:12:09.776 INFO:teuthology.orchestra.run.smithi032.stdout:HEAD is now at c156e6be Merge branch 'robot/pr-0-1702625169' into 'main' 2023-12-20T04:12:09.798 INFO:teuthology.task.kernel:Downloading kernel 93572f31db899adbd1540d7954ab247e32731c84 on client.0... 2023-12-20T04:12:09.798 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=93572f31db899adbd1540d7954ab247e32731c84 2023-12-20T04:12:09.947 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/93572f31db899adbd1540d7954ab247e32731c84/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g93572f31db89/ 2023-12-20T04:12:09.947 DEBUG:teuthology.orchestra.run.smithi032:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-g93572f31db89_6.6.0-g93572f31db89-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/93572f31db899adbd1540d7954ab247e32731c84/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g93572f31db89/ --input-file=- 2023-12-20T04:12:09.950 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi032.front.sepia.ceph.com to complete... 2023-12-20T04:12:12.262 INFO:teuthology.orchestra.run.smithi039.stderr:Updating files: 48% (1589/3256) Updating files: 49% (1596/3256) Updating files: 50% (1628/3256) Updating files: 51% (1661/3256) Updating files: 52% (1694/3256) Updating files: 53% (1726/3256) Updating files: 54% (1759/3256) Updating files: 55% (1791/3256) Updating files: 56% (1824/3256) Updating files: 57% (1856/3256) Updating files: 58% (1889/3256) Updating files: 59% (1922/3256) Updating files: 60% (1954/3256) Updating files: 61% (1987/3256) Updating files: 62% (2019/3256) Updating files: 63% (2052/3256) Updating files: 63% (2074/3256) Updating files: 64% (2084/3256) Updating files: 65% (2117/3256) Updating files: 66% (2149/3256) Updating files: 67% (2182/3256) Updating files: 68% (2215/3256) Updating files: 69% (2247/3256) Updating files: 70% (2280/3256) Updating files: 71% (2312/3256) Updating files: 72% (2345/3256) Updating files: 73% (2377/3256) Updating files: 74% (2410/3256) Updating files: 75% (2442/3256) Updating files: 76% (2475/3256) Updating files: 77% (2508/3256) Updating files: 78% (2540/3256) Updating files: 78% (2566/3256) Updating files: 79% (2573/3256) Updating files: 80% (2605/3256) Updating files: 81% (2638/3256) Updating files: 82% (2670/3256) Updating files: 83% (2703/3256) Updating files: 84% (2736/3256) Updating files: 84% (2743/3256) Updating files: 85% (2768/3256) Updating files: 86% (2801/3256) Updating files: 87% (2833/3256) Updating files: 88% (2866/3256) Updating files: 89% (2898/3256) Updating files: 90% (2931/3256) Updating files: 91% (2963/3256) Updating files: 92% (2996/3256) Updating files: 93% (3029/3256) Updating files: 94% (3061/3256) Updating files: 95% (3094/3256) Updating files: 96% (3126/3256) Updating files: 97% (3159/3256) Updating files: 98% (3191/3256) Updating files: 99% (3224/3256) Updating files: 100% (3256/3256) Updating files: 100% (3256/3256), done. 2023-12-20T04:12:12.267 INFO:teuthology.orchestra.run.smithi039.stdout:HEAD is now at c156e6be Merge branch 'robot/pr-0-1702625169' into 'main' 2023-12-20T04:12:12.289 INFO:teuthology.task.kernel:Downloading kernel 93572f31db899adbd1540d7954ab247e32731c84 on client.1... 2023-12-20T04:12:12.289 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=kernel&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=93572f31db899adbd1540d7954ab247e32731c84 2023-12-20T04:12:12.446 INFO:teuthology.task.kernel:fetching, builder baseurl is https://2.chacra.ceph.com/r/kernel/testing/93572f31db899adbd1540d7954ab247e32731c84/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g93572f31db89/ 2023-12-20T04:12:12.446 DEBUG:teuthology.orchestra.run.smithi039:> rm -f /tmp/linux-image.deb && echo linux-image-6.6.0-g93572f31db89_6.6.0-g93572f31db89-1_amd64.deb | wget -nv -O /tmp/linux-image.deb --base=https://2.chacra.ceph.com/r/kernel/testing/93572f31db899adbd1540d7954ab247e32731c84/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g93572f31db89/ --input-file=- 2023-12-20T04:12:12.449 DEBUG:teuthology.task.kernel:Waiting for download/copy to ubuntu@smithi039.front.sepia.ceph.com to complete... 2023-12-20T04:12:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20 04:12:14 URL:https://2.chacra.ceph.com/r/kernel/testing/93572f31db899adbd1540d7954ab247e32731c84/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g93572f31db89/linux-image-6.6.0-g93572f31db89_6.6.0-g93572f31db89-1_amd64.deb [38935430/38935430] -> "/tmp/linux-image.deb" [1] 2023-12-20T04:12:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:FINISHED --2023-12-20 04:12:14-- 2023-12-20T04:12:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:Total wall clock time: 4.9s 2023-12-20T04:12:14.837 INFO:teuthology.orchestra.run.smithi032.stderr:Downloaded: 1 files, 37M in 4.6s (8.16 MB/s) 2023-12-20T04:12:14.842 INFO:teuthology.task.kernel:Installing kernel 93572f31db899adbd1540d7954ab247e32731c84 on client.0... 2023-12-20T04:12:14.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo dpkg -i /tmp/linux-image.deb 2023-12-20T04:12:15.105 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package linux-image-6.6.0-g93572f31db89. 2023-12-20T04:12:16.640 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20 04:12:16 URL:https://2.chacra.ceph.com/r/kernel/testing/93572f31db899adbd1540d7954ab247e32731c84/ubuntu/jammy/flavors/default/pool/main/l/linux-6.6.0-g93572f31db89/linux-image-6.6.0-g93572f31db89_6.6.0-g93572f31db89-1_amd64.deb [38935430/38935430] -> "/tmp/linux-image.deb" [1] 2023-12-20T04:12:16.641 INFO:teuthology.orchestra.run.smithi039.stderr:FINISHED --2023-12-20 04:12:16-- 2023-12-20T04:12:16.641 INFO:teuthology.orchestra.run.smithi039.stderr:Total wall clock time: 4.2s 2023-12-20T04:12:16.641 INFO:teuthology.orchestra.run.smithi039.stderr:Downloaded: 1 files, 37M in 3.8s (9.65 MB/s) 2023-12-20T04:12:16.646 INFO:teuthology.task.kernel:Installing kernel 93572f31db899adbd1540d7954ab247e32731c84 on client.1... 2023-12-20T04:12:16.646 DEBUG:teuthology.orchestra.run.smithi039:> sudo dpkg -i /tmp/linux-image.deb 2023-12-20T04:12:16.899 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package linux-image-6.6.0-g93572f31db89. 2023-12-20T04:12:17.175 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-12-20T04:12:17.179 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-12-20T04:12:17.379 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking linux-image-6.6.0-g93572f31db89 (6.6.0-g93572f31db89-1) ... 2023-12-20T04:12:18.831 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... 129542 files and directories currently installed.) 2023-12-20T04:12:18.835 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack /tmp/linux-image.deb ... 2023-12-20T04:12:18.954 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking linux-image-6.6.0-g93572f31db89 (6.6.0-g93572f31db89-1) ... 2023-12-20T04:12:20.868 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up linux-image-6.6.0-g93572f31db89 (6.6.0-g93572f31db89-1) ... 2023-12-20T04:12:20.973 INFO:teuthology.orchestra.run.smithi032.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-g93572f31db89 2023-12-20T04:12:22.098 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up linux-image-6.6.0-g93572f31db89 (6.6.0-g93572f31db89-1) ... 2023-12-20T04:12:22.205 INFO:teuthology.orchestra.run.smithi039.stderr:update-initramfs: Generating /boot/initrd.img-6.6.0-g93572f31db89 2023-12-20T04:12:42.283 INFO:teuthology.orchestra.run.smithi039.stderr:Sourcing file `/etc/default/grub' 2023-12-20T04:12:42.297 INFO:teuthology.orchestra.run.smithi039.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-20T04:12:42.325 INFO:teuthology.orchestra.run.smithi039.stderr:Generating grub configuration file ... 2023-12-20T04:12:42.630 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-6.6.0-g93572f31db89 2023-12-20T04:12:42.646 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-6.6.0-g93572f31db89 2023-12-20T04:12:42.984 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-20T04:12:42.998 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-20T04:12:43.058 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-20T04:12:43.064 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-20T04:12:43.170 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2023-12-20T04:12:43.176 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-20T04:12:43.199 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2023-12-20T04:12:43.207 INFO:teuthology.orchestra.run.smithi039.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-20T04:12:43.207 INFO:teuthology.orchestra.run.smithi039.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-12-20T04:12:43.207 INFO:teuthology.orchestra.run.smithi039.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-20T04:12:43.246 INFO:teuthology.orchestra.run.smithi039.stderr:done 2023-12-20T04:12:43.447 DEBUG:teuthology.orchestra.run.smithi039:> dpkg-deb -c /tmp/linux-image.deb 2023-12-20T04:12:43.506 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./ 2023-12-20T04:12:43.506 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./boot/ 2023-12-20T04:12:43.506 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 4415617 2023-12-20 01:01 ./boot/System.map-6.6.0-g93572f31db89 2023-12-20T04:12:43.519 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 169214 2023-12-20 01:01 ./boot/config-6.6.0-g93572f31db89 2023-12-20T04:12:43.519 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11352864 2023-12-20 01:01 ./boot/vmlinuz-6.6.0-g93572f31db89 2023-12-20T04:12:43.540 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/ 2023-12-20T04:12:43.540 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/kernel/ 2023-12-20T04:12:43.540 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/kernel/postinst.d/ 2023-12-20T04:12:43.540 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/kernel/postrm.d/ 2023-12-20T04:12:43.540 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/kernel/preinst.d/ 2023-12-20T04:12:43.540 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/kernel/prerm.d/ 2023-12-20T04:12:43.541 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/ 2023-12-20T04:12:43.541 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/ 2023-12-20T04:12:43.541 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/ 2023-12-20T04:12:43.541 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/ 2023-12-20T04:12:43.541 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/ 2023-12-20T04:12:43.541 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/ 2023-12-20T04:12:43.541 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/ 2023-12-20T04:12:43.541 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 490233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/aesni-intel.ko 2023-12-20T04:12:43.544 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-20T04:12:43.544 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-20T04:12:43.544 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-20T04:12:43.544 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 93113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-20T04:12:43.544 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 106729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-20T04:12:43.544 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 152241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-20T04:12:43.544 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-20T04:12:43.545 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-20T04:12:43.545 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 83889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-20T04:12:43.545 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-20T04:12:43.545 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-20T04:12:43.545 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-20T04:12:43.545 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-20T04:12:43.545 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-20T04:12:43.546 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 40529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-20T04:12:43.546 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 58585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-20T04:12:43.546 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-20T04:12:43.546 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-20T04:12:43.546 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-20T04:12:43.546 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/ 2023-12-20T04:12:43.546 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/cpu/ 2023-12-20T04:12:43.547 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/cpu/mce/ 2023-12-20T04:12:43.547 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 40113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-20T04:12:43.547 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/cpuid.ko 2023-12-20T04:12:43.547 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/msr.ko 2023-12-20T04:12:43.547 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kvm/ 2023-12-20T04:12:43.547 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 354433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-20T04:12:43.547 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 881625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-20T04:12:43.548 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 2275865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kvm/kvm.ko 2023-12-20T04:12:43.553 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-6.6.0-g93572f31db89 2023-12-20T04:12:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-6.6.0-g93572f31db89 2023-12-20T04:12:43.650 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/ 2023-12-20T04:12:43.650 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 53697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/af_alg.ko 2023-12-20T04:12:43.650 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/algif_hash.ko 2023-12-20T04:12:43.650 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/algif_skcipher.ko 2023-12-20T04:12:43.650 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/ansi_cprng.ko 2023-12-20T04:12:43.650 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/anubis.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/arc4.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/asymmetric_keys/ 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/ 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/async_memcpy.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/async_pq.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/async_tx.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/async_xor.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/raid6test.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/authenc.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/authencesn.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/blake2b_generic.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/blowfish_common.ko 2023-12-20T04:12:43.651 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/blowfish_generic.ko 2023-12-20T04:12:43.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/camellia_generic.ko 2023-12-20T04:12:43.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cast5_generic.ko 2023-12-20T04:12:43.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cast6_generic.ko 2023-12-20T04:12:43.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cast_common.ko 2023-12-20T04:12:43.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/ccm.ko 2023-12-20T04:12:43.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cmac.ko 2023-12-20T04:12:43.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/crc32_generic.ko 2023-12-20T04:12:43.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cryptd.ko 2023-12-20T04:12:43.652 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/crypto_engine.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/crypto_simd.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/crypto_user.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cts.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/des_generic.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/dh_generic.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/echainiv.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/essiv.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/fcrypt.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/khazad.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/lrw.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/lz4.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/lz4hc.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/md4.ko 2023-12-20T04:12:43.653 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/michael_mic.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/pcbc.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/pcrypt.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/rmd160.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/seed.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/serpent_generic.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 149193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/tcrypt.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/tea.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/twofish_common.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/twofish_generic.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/vmac.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/wp512.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/xcbc.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/xor.ko 2023-12-20T04:12:43.654 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/xts.ko 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/xxhash_generic.ko 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/ 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/acpi_extlog.ko 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/acpi_pad.ko 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/apei/ 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/apei/einj.ko 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/ec_sys.ko 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/platform_profile.ko 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/sbs.ko 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/sbshc.ko 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 103025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/video.ko 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/ 2023-12-20T04:12:43.655 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/acard-ahci.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 88865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/ahci.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/ahci_platform.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 88289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/libahci.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/libahci_platform.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_acpi.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_ali.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_amd.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_artop.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_atiixp.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_atp867x.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_cmd640.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_cmd64x.ko 2023-12-20T04:12:43.656 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_cypress.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_efar.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_hpt366.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_hpt37x.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_it8213.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_it821x.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_jmicron.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_legacy.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_marvell.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_mpiix.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_netcell.ko 2023-12-20T04:12:43.657 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_ninja32.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_ns87410.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_ns87415.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_oldpiix.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_opti.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_optidma.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_pcmcia.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_piccolo.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_radisys.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_rdc.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_rz1000.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_sch.ko 2023-12-20T04:12:43.658 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_serverworks.ko 2023-12-20T04:12:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_sil680.ko 2023-12-20T04:12:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_sl82c105.ko 2023-12-20T04:12:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_triflex.ko 2023-12-20T04:12:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_via.ko 2023-12-20T04:12:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pdc_adma.ko 2023-12-20T04:12:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_inic162x.ko 2023-12-20T04:12:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 83345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_mv.ko 2023-12-20T04:12:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 70393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_nv.ko 2023-12-20T04:12:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_promise.ko 2023-12-20T04:12:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_qstor.ko 2023-12-20T04:12:43.659 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_sil.ko 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_sil24.ko 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_sis.ko 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_svw.ko 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_sx4.ko 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_uli.ko 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_via.ko 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_vsc.ko 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/auxdisplay/ 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/auxdisplay/ks0108.ko 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/bcma/ 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 124473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/bcma/bcma.ko 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/ 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/aoe/ 2023-12-20T04:12:43.660 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 113665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/aoe/aoe.ko 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/drbd/ 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 965585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/drbd/drbd.ko 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 173225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/floppy.ko 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/mtip32xx/ 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 94297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 115793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/nbd.ko 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/null_blk/ 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 110385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/null_blk/null_blk.ko 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 98193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/pktcdvd.ko 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 263769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/rbd.ko 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/xen-blkback/ 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 98145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/zram/ 2023-12-20T04:12:43.661 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 60217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/zram/zram.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/applicom.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hangcheck-timer.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/ 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/via-rng.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 148601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 139313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-20T04:12:43.662 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 70921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 53513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/lp.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/mwave/ 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 84097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/mwave/mwave.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/nvram.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ppdev.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tlclk.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/ 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-20T04:12:43.663 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/xillybus/ 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/xillybus/xillybus_class.ko 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/xillybus/xillybus_core.ko 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/clk/ 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/clk/clk-palmas.ko 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/clk/clk-si5351.ko 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/clk/clk-twl6040.ko 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/clk/clk-wm831x.ko 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/cpufreq/ 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/ 2023-12-20T04:12:43.664 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/ccp/ 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 70801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 243025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/ccp/ccp.ko 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/ 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/qat/ 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/qat/qat_common/ 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 438761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/padlock-aes.ko 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/padlock-sha.ko 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/virtio/ 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 70273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dca/ 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dca/dca.ko 2023-12-20T04:12:43.665 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dma/ 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dma/dw/ 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dma/ioat/ 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 122569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/ 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 79953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/amd64_edac.ko 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/e752x_edac.ko 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 63617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/edac_mce_amd.ko 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i3000_edac.ko 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i3200_edac.ko 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i5100_edac.ko 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i5400_edac.ko 2023-12-20T04:12:43.666 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i7300_edac.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 52041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i7core_edac.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i82975x_edac.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/ie31200_edac.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 56825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/sb_edac.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/x38_edac.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/ 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-gpio.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-max14577.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-max77693.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-max8997.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-palmas.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-rt8973a.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-sm5502.ko 2023-12-20T04:12:43.667 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/firmware/ 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/firmware/dmi-sysfs.ko 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/firmware/efi/ 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/firmware/efi/efi-pstore.ko 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/ 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-adp5520.ko 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-amd8111.ko 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-arizona.ko 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-da9052.ko 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-da9055.ko 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-dln2.ko 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 77849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-f7188x.ko 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-generic.ko 2023-12-20T04:12:43.668 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-ich.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-kempld.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-lp3943.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-max7300.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-max730x.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-max732x.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-pca953x.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-pcf857x.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-rdc321x.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-sch.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-sch311x.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-tps65912.ko 2023-12-20T04:12:43.669 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-twl4030.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-twl6040.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-vx855.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-wm831x.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-wm8350.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-wm8994.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/ 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-a4tech.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-apple.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-appleir.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-aureal.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-axff.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-belkin.ko 2023-12-20T04:12:43.670 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-cherry.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-chicony.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 60649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-cp2112.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-cypress.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-dr.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-elecom.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-elo.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-emsff.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-ezkey.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-gaff.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-generic.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-gt683r.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-gyration.ko 2023-12-20T04:12:43.671 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-holtekff.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-hyperv.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-icade.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-kensington.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-keytouch.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-kye.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-lcpower.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-led.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-lenovo.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-lg-g15.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 58185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 101481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 80993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-logitech.ko 2023-12-20T04:12:43.672 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-magicmouse.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-microsoft.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-monterey.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-multitouch.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-ntrig.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-ortek.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-penmount.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-petalynx.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 72937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-picolcd.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-pl.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-plantronics.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-primax.ko 2023-12-20T04:12:43.673 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-rmi.ko 2023-12-20T04:12:43.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-20T04:12:43.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-common.ko 2023-12-20T04:12:43.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-20T04:12:43.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-20T04:12:43.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-20T04:12:43.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-20T04:12:43.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-20T04:12:43.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-20T04:12:43.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-20T04:12:43.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-20T04:12:43.674 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-saitek.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-samsung.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-sjoy.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-sony.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-speedlink.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-steelseries.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-sunplus.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-tivo.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-tmff.ko 2023-12-20T04:12:43.675 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-topseed.ko 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-twinhan.ko 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 71073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-uclogic.ko 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-waltop.ko 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 143473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-wiimote.ko 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-xinmo.ko 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-zpff.ko 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-zydacron.ko 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 267825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid.ko 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/uhid.ko 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/usbhid/ 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 125721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/usbhid/usbhid.ko 2023-12-20T04:12:43.676 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/usbhid/usbkbd.ko 2023-12-20T04:12:43.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/usbhid/usbmouse.ko 2023-12-20T04:12:43.677 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 217873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/wacom.ko 2023-12-20T04:12:43.677 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hv/ 2023-12-20T04:12:43.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 64201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hv/hv_balloon.ko 2023-12-20T04:12:43.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 89625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hv/hv_utils.ko 2023-12-20T04:12:43.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 305465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hv/hv_vmbus.ko 2023-12-20T04:12:43.678 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ 2023-12-20T04:12:43.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 66521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/abituguru.ko 2023-12-20T04:12:43.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 61393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/abituguru3.ko 2023-12-20T04:12:43.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-20T04:12:43.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ad7414.ko 2023-12-20T04:12:43.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ad7418.ko 2023-12-20T04:12:43.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adc128d818.ko 2023-12-20T04:12:43.678 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adm1025.ko 2023-12-20T04:12:43.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 97977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adm1026.ko 2023-12-20T04:12:43.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adm1029.ko 2023-12-20T04:12:43.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 45553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adm1031.ko 2023-12-20T04:12:43.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adm9240.ko 2023-12-20T04:12:43.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ads7828.ko 2023-12-20T04:12:43.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7410.ko 2023-12-20T04:12:43.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7411.ko 2023-12-20T04:12:43.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 74433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7462.ko 2023-12-20T04:12:43.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7470.ko 2023-12-20T04:12:43.679 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 67929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7475.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7x10.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/amc6821.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 55201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/applesmc.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/asb100.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 58617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/asc7621.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 40825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/atxp1.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/coretemp.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/da9052-hwmon.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/da9055-hwmon.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 99601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/dme1737.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ds1621.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ds620.ko 2023-12-20T04:12:43.680 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/emc1403.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/emc2103.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/emc6w201.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 66985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/f71805f.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 94457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/f71882fg.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/f75375s.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/fam15h_power.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 55449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/fschmd.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/g760a.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/g762.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 46801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/gl518sm.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 45529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/gl520sm.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/hih6130.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/i5500_temp.ko 2023-12-20T04:12:43.681 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/i5k_amb.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ibmaem.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ibmpex.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ina209.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ina2xx.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 130449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/it87.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/jc42.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/k10temp.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/k8temp.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lineage-pem.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 48601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm63.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm73.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm75.ko 2023-12-20T04:12:43.682 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm77.ko 2023-12-20T04:12:43.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm78.ko 2023-12-20T04:12:43.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm80.ko 2023-12-20T04:12:43.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm83.ko 2023-12-20T04:12:43.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 72833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm85.ko 2023-12-20T04:12:43.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 46153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm87.ko 2023-12-20T04:12:43.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm90.ko 2023-12-20T04:12:43.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm92.ko 2023-12-20T04:12:43.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 108841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm93.ko 2023-12-20T04:12:43.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm95234.ko 2023-12-20T04:12:43.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm95241.ko 2023-12-20T04:12:43.683 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm95245.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc2945.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4151.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4215.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4222.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4245.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4260.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4261.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max16065.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max1619.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max1668.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max197.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max6639.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max6650.ko 2023-12-20T04:12:43.684 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max6697.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/mc13783-adc.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/mcp3021.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/nct6683.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 119817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/nct6775-core.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 60705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/nct6775.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 59737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/nct7802.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 82337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pc87360.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 54433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pc87427.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pcf8591.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/ 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-20T04:12:43.685 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 82577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/powr1220.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sch5627.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sch5636.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sht15.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sht21.ko 2023-12-20T04:12:43.686 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/shtc1.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 40105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sis5595.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/smsc47b397.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/smsc47m1.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/smsc47m192.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/thmc50.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/tmp102.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/tmp103.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/tmp401.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/tmp421.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/via-cputemp.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/via686a.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 52457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/vt1211.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/vt8231.ko 2023-12-20T04:12:43.687 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 57449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83627ehf.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 78873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83627hf.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 71345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83781d.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 63225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83791d.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 71321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83792d.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 101473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83793.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 119505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83795.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83l785ts.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83l786ng.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/ 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/algos/ 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-20T04:12:43.688 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/ 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 6929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-20T04:12:43.689 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 62817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-12-20T04:12:43.690 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/i2c-mux.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/i2c-smbus.ko 2023-12-20T04:12:43.691 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/i2c-stub.ko 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/muxes/ 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/ 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 273609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 879969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/ib_core.ko 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 69273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 324217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 97985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 282273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 66201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/ 2023-12-20T04:12:43.692 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/cxgb4/ 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 493073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mlx4/ 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 419737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mlx5/ 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 739025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mthca/ 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 296625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/ocrdma/ 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 153697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/usnic/ 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 168969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-12-20T04:12:43.693 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/ 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/ipoib/ 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 263081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/iser/ 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 100241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/isert/ 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 111425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/srp/ 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 128753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/srpt/ 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 129529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/ 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/evbug.ko 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/ff-memless.ko 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/gameport/ 2023-12-20T04:12:43.694 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-20T04:12:43.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-20T04:12:43.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/gameport/gameport.ko 2023-12-20T04:12:43.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/joydev.ko 2023-12-20T04:12:43.695 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/ 2023-12-20T04:12:43.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-12-20T04:12:43.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-12-20T04:12:43.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-12-20T04:12:43.695 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/lkkbd.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/lm8323.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/lm8333.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/newtonkbd.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/qt1070.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/qt2160.ko 2023-12-20T04:12:43.696 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-12-20T04:12:43.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/stowaway.ko 2023-12-20T04:12:43.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/sunkbd.ko 2023-12-20T04:12:43.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-12-20T04:12:43.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-12-20T04:12:43.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-12-20T04:12:43.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/xtkbd.ko 2023-12-20T04:12:43.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/matrix-keymap.ko 2023-12-20T04:12:43.697 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/ 2023-12-20T04:12:43.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/appletouch.ko 2023-12-20T04:12:43.697 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/bcm5974.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 123681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/cyapatp.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 89449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/gpio_mouse.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 335929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/psmouse.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/sermouse.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/rmi4/ 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 109873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/ 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/altera_ps2.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/arc_ps2.ko 2023-12-20T04:12:43.698 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/ct82c710.ko 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/parkbd.ko 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/pcips2.ko 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/ps2mult.ko 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/serio_raw.ko 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/serport.ko 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/sparse-keymap.ko 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/iommu/ 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/iommu/amd/ 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/ 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-88pm860x.ko 2023-12-20T04:12:43.699 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-adp5520.ko 2023-12-20T04:12:43.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 59753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-bd2802.ko 2023-12-20T04:12:43.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-blinkm.ko 2023-12-20T04:12:43.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-da903x.ko 2023-12-20T04:12:43.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-da9052.ko 2023-12-20T04:12:43.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-gpio.ko 2023-12-20T04:12:43.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lm3530.ko 2023-12-20T04:12:43.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lm3533.ko 2023-12-20T04:12:43.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lm355x.ko 2023-12-20T04:12:43.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lm3642.ko 2023-12-20T04:12:43.700 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lp3944.ko 2023-12-20T04:12:43.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lp8788.ko 2023-12-20T04:12:43.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-max8997.ko 2023-12-20T04:12:43.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-mc13783.ko 2023-12-20T04:12:43.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-menf21bmc.ko 2023-12-20T04:12:43.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-pca9532.ko 2023-12-20T04:12:43.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-pca955x.ko 2023-12-20T04:12:43.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-pca963x.ko 2023-12-20T04:12:43.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-ss4200.ko 2023-12-20T04:12:43.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-tca6507.ko 2023-12-20T04:12:43.701 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-wm831x-status.ko 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-wm8350.ko 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/ 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/bcache/ 2023-12-20T04:12:43.702 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 654969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/bcache/bcache.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 40577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-bio-prison.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 89601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-bufio.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-cache-smq.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 156009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-cache.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 106897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-crypt.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-delay.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 53857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-era.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-flakey.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-log-userspace.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-log.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-mirror.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 78033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-multipath.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-queue-length.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 70825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-raid.ko 2023-12-20T04:12:43.703 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-region-hash.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-round-robin.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-service-time.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 109529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-snapshot.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-switch.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 181865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-thin-pool.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-verity.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-zero.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/faulty.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/linear.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/multipath.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/persistent-data/ 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 199937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/raid0.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 113073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/raid1.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 158681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/raid10.ko 2023-12-20T04:12:43.704 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 395721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/raid456.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/ 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/ 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 178137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptbase.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 63385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptctl.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptfc.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptlan.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 123321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptsas.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 79041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptscsih.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 53017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptspi.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/ 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/88pm800.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/88pm805.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/88pm80x.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/arizona-i2c.ko 2023-12-20T04:12:43.705 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 123721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/arizona.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/bcm590xx.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/cros_ec_dev.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/dln2.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/janz-cmodio.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 53585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/kempld-core.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/lm3533-core.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/lp3943.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 54569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/lpc_ich.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/lpc_sch.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/max8907.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/mc13xxx-core.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/menf21bmc.ko 2023-12-20T04:12:43.706 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/pcf50633-adc.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/pcf50633-gpio.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/pcf50633.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/retu-mfd.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 80425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/si476x-core.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 48457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/sm501.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/tps65010.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/tps6507x.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/viperboard.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/vx855.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/wl1273-core.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/wm8994.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-12-20T04:12:43.707 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 48489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ad525x_dpot.ko 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/altera-stapl/ 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 46585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/apds9802als.ko 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/apds990x.ko 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 45593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/bh1770glc.ko 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/c2port/ 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/c2port/core.ko 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/cb710/ 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/cb710/cb710.ko 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ds1682.ko 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/dummy-irq.ko 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/echo/ 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/echo/echo.ko 2023-12-20T04:12:43.708 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/eeprom/ 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/eeprom/at24.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/eeprom/max6875.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/enclosure.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/genwqe/ 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 152201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/hmc6352.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/hpilo.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ibmasm/ 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 72129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ics932s401.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/isl29003.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/isl29020.ko 2023-12-20T04:12:43.709 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/lis3lv02d/ 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/mei/ 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 125209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/mei/mei-me.ko 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 56521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/mei/mei-txe.ko 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 310113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/mei/mei.ko 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/phantom.ko 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ti-st/ 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 74265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ti-st/st_drv.ko 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/tifm_7xx1.ko 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/tifm_core.ko 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/tsl2550.ko 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 48697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/vmw_balloon.ko 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/vmw_vmci/ 2023-12-20T04:12:43.710 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 183609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/bonding/ 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 490129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/bonding/bonding.ko 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/ 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/b53/ 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 106033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 40417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/mv88e6xxx/ 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 453737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dummy.ko 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/eql.ko 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/ 2023-12-20T04:12:43.711 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/3com/ 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 45785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 130553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 68849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/8390/ 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 59689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 99049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/adaptec/ 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 54521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/agere/ 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 69137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/alteon/ 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 59585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-20T04:12:43.712 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/altera/ 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/amd/ 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 54609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 87297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/ 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/alx/ 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 97337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 104673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-12-20T04:12:43.713 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 90257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atlx/ 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 81569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 69729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/ 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 78401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 176761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1421497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 128569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/genet/ 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 132089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-20T04:12:43.714 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 312081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/brocade/ 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/brocade/bna/ 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 323513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/ 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 193497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 347417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1044497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 166193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-12-20T04:12:43.715 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/cisco/ 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/cisco/enic/ 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 204345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/ 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/ 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 69449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 61489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 141225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 62817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dlink/ 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 60873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 62833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-20T04:12:43.716 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 45721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dnet.ko 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/ec_bhf.ko 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/emulex/ 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/emulex/benet/ 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 281137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/ethoc.ko 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/fealnx.ko 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/fujitsu/ 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 45345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/ 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 90465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/e1000/ 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 295577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/e1000e/ 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 584497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-20T04:12:43.717 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/fm10k/ 2023-12-20T04:12:43.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 246209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-20T04:12:43.718 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/i40e/ 2023-12-20T04:12:43.718 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 925825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-20T04:12:43.778 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/iavf/ 2023-12-20T04:12:43.778 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 302937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-20T04:12:43.778 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/igb/ 2023-12-20T04:12:43.778 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 514641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-20T04:12:43.778 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/igbvf/ 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 93593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 734305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 151769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 86921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/jme.ko 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/marvell/ 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 102289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 122649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/ 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 727601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 278937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-20T04:12:43.779 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1113681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/micrel/ 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 53041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 40993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 95665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/myricom/ 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 108649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/natsemi/ 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 73441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-20T04:12:43.780 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/neterion/ 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 150905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/nvidia/ 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 136721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/packetengines/ 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 43145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/ 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 227897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 87081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 672049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-20T04:12:43.781 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/rdc/ 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 43017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/realtek/ 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 67385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 83985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 189801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/samsung/ 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 104241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sfc/ 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1052497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/silan/ 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sis/ 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 54817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-20T04:12:43.782 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 63337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/smsc/ 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 61537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 67193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 73409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 68177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/ 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 487025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sun/ 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 103337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 166097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-20T04:12:43.783 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 66329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 78801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/tehuti/ 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/ti/ 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 86529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/via/ 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 80273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 83633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/wiznet/ 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 46241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/xircom/ 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 59993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/fddi/ 2023-12-20T04:12:43.784 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/fddi/defxx.ko 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/fddi/skfp/ 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 216985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/fddi/skfp/skfp.ko 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 77345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/geneve.ko 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/hyperv/ 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 235521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ieee802154/ 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ifb.ko 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ipvlan/ 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 83769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 62145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/macvlan.ko 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/macvtap.ko 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio/ 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-20T04:12:43.785 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio/mdio-gpio.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mii.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 43953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/netconsole.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/nlmon.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/pcs/ 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/ 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/amd.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/bcm87xx.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/broadcom.ko 2023-12-20T04:12:43.786 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/cicada.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/davicom.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/et1011c.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/icplus.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/lxt.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 82049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/marvell.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 76113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/micrel.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/national.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 97857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/phylink.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/qsemi.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/realtek.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/smsc.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/ste10Xp.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/vitesse.ko 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/plip/ 2023-12-20T04:12:43.787 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 43505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/plip/plip.ko 2023-12-20T04:12:43.788 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/ 2023-12-20T04:12:43.788 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-20T04:12:43.788 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/ppp_async.ko 2023-12-20T04:12:43.788 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-20T04:12:43.788 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-20T04:12:43.788 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-20T04:12:43.788 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/pppoe.ko 2023-12-20T04:12:43.788 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/pppox.ko 2023-12-20T04:12:43.788 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/pptp.ko 2023-12-20T04:12:43.788 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/sb1000.ko 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/slip/ 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/slip/slip.ko 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/sungem_phy.ko 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 64449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/tap.ko 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/ 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 106169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team.ko 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team_mode_random.ko 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-20T04:12:43.789 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 87017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/veth.ko 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/vmxnet3/ 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 144297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/vxlan/ 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 297329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/vxlan/vxlan.ko 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/xen-netback/ 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 144585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ntb/ 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ntb/ntb.ko 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/nvme/ 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/nvme/host/ 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 265121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/nvme/host/nvme-core.ko 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 102209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/nvme/host/nvme.ko 2023-12-20T04:12:43.790 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/parport/ 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 133841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/parport/parport.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/parport/parport_cs.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 84465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/parport/parport_pc.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/parport/parport_serial.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/ 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/hotplug/ 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/pci-stub.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 46465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/xen-pcifront.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/ 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/i82092.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 145809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/pcmcia.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 62137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/pcmcia_core.ko 2023-12-20T04:12:43.791 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/pd6729.ko 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 112889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/phy/ 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/phy/broadcom/ 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/phy/samsung/ 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pinctrl/ 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pinctrl/intel/ 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 70153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/ 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/ 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_ec.ko 2023-12-20T04:12:43.792 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/ 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 67097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/acerhdf.ko 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 59665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 127441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-20T04:12:43.793 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 40265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/hdaps.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/ibm_rtl.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/intel/ 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 43497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/intel_ips.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-20T04:12:43.794 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 57569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/wmi.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/ 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/ 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/88pm860x_battery.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/88pm860x_charger.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 55057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/bq2415x_charger.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 54081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/bq24190_charger.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/bq24735-charger.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/da9030_battery.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/da9052-battery.ko 2023-12-20T04:12:43.795 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/ds2760_battery.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/ds2780_battery.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/ds2781_battery.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/ds2782_battery.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/gpio-charger.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/isp1704_charger.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/lp8727_charger.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/max14577_charger.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/max17040_battery.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/max17042_battery.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/max8903_charger.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/max8925_power.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/pcf50633-charger.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/sbs-battery.ko 2023-12-20T04:12:43.796 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/test_power.ko 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/tps65090-charger.ko 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/wm831x_backup.ko 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/wm831x_power.ko 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/wm8350_power.ko 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/powercap/ 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 57729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/ 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/clients/ 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/clients/pps_parport.ko 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/pps_core.ko 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ptp/ 2023-12-20T04:12:43.797 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 66385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ptp/ptp.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ptp/ptp_kvm.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/ 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-88pm80x.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-88pm860x.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-da9052.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-da9055.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-da9063.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 68577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-20T04:12:43.798 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-em3027.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-lp8788.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-m48t86.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-max6900.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-max8907.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-max8925.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-max8997.ko 2023-12-20T04:12:43.799 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-max8998.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-palmas.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 46633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf50633.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rc5t583.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-20T04:12:43.800 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-s35390a.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-tps6586x.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-tps65910.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-wm831x.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-wm8350.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-x1205.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 77617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/3w-9xxx.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 57665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/3w-sas.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 55105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/3w-xxxx.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 135185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/BusLogic.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/a100u2w.ko 2023-12-20T04:12:43.801 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aacraid/ 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 272305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 100617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/advansys.ko 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aic7xxx/ 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 356201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 295801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aic94xx/ 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 189185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/am53c974.ko 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/arcmsr/ 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 106881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/atp870u.ko 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/be2iscsi/ 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 246985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-20T04:12:43.802 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bfa/ 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1171945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bfa/bfa.ko 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bnx2fc/ 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 237273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bnx2i/ 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 115369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 40585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ch.ko 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/csiostor/ 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 308921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/ 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 76145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 101449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 146689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-20T04:12:43.803 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 75553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/dc395x.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/dmx3191d.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/esas2r/ 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 212425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 60505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/esp_scsi.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fcoe/ 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 78209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 158505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fdomain.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fnic/ 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 263489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fnic/fnic.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 201817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/hpsa.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/hptiop.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/hv_storvsc.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/imm.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/initio.ko 2023-12-20T04:12:43.804 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 266537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ipr.ko 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 101881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ips.ko 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/isci/ 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 317153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/isci/isci.ko 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 52961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libfc/ 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 314865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libfc/libfc.ko 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 145585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libiscsi.ko 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 53521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libsas/ 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 216665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libsas/libsas.ko 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/lpfc/ 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 2197249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-20T04:12:43.805 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/megaraid/ 2023-12-20T04:12:43.806 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 73569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-20T04:12:43.806 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-20T04:12:43.806 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 310617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-20T04:12:43.806 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 84721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/megaraid.ko 2023-12-20T04:12:43.806 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/mpt3sas/ 2023-12-20T04:12:43.806 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 659825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-20T04:12:43.806 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/mvsas/ 2023-12-20T04:12:43.808 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 119561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-20T04:12:43.808 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 78129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/mvumi.ko 2023-12-20T04:12:43.808 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pcmcia/ 2023-12-20T04:12:43.808 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 81625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-12-20T04:12:43.808 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-12-20T04:12:43.808 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pm8001/ 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 364105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 123617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pmcraid.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ppa.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 55361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla1280.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla2xxx/ 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1876689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 85289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla4xxx/ 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 559033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qlogicfas408.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/raid_class.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 238761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_debug.ko 2023-12-20T04:12:43.809 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 173937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 279313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 97729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 73881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ses.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 115889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/st.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 46825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/stex.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/sym53c8xx_2/ 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 162121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/virtio_scsi.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/wd719x.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/xen-scsifront.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ssb/ 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 129441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ssb/ssb.ko 2023-12-20T04:12:43.810 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/ 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/iscsi/ 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 736601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/loopback/ 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/target_core_file.ko 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 43025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/target_core_iblock.ko 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 930257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/target_core_mod.ko 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/target_core_pscsi.ko 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 122521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/target_core_user.ko 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/tcm_fc/ 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 75593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/ 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/ 2023-12-20T04:12:43.811 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/ 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-20T04:12:43.812 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/ 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/ipwireless/ 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 78425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 57865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/moxa.ko 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 50953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/mxser.ko 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 108601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/n_gsm.ko 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/n_hdlc.ko 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 67929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/nozomi.ko 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/ 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/8250/ 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 59497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-12-20T04:12:43.813 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/altera_uart.ko 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/arc_uart.ko 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 72425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/jsm/ 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 103073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/rp2.ko 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 52817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/sc16is7xx.ko 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/ 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/core/ 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 531265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/host/ 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/ 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio.ko 2023-12-20T04:12:43.814 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_aec.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_cif.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_dmem_genirq.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_mf624.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_netx.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_pci_generic.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_sercos3.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/ 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/c67x00/ 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 73297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/c67x00/c67x00.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/chipidea/ 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 144377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-12-20T04:12:43.815 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/class/ 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 87089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/class/cdc-acm.ko 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/class/usblp.ko 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 89857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/class/usbtmc.ko 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/common/ 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/common/ulpi.ko 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc2/ 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc3/ 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 448505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc3/dwc3.ko 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/ 2023-12-20T04:12:43.816 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/ 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 60177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/u_ether.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/u_serial.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 92489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 122081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 57681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 57729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 85609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-12-20T04:12:43.817 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 58113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 67337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/ 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-12-20T04:12:43.818 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 62185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 161225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/libcomposite.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/ 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/bdc/ 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 157681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 52721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 69705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 56033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 76057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/net2272.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 75073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/net2280.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 73169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-12-20T04:12:43.819 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 65809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 66097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 81505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 156073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/ 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/bcma-hcd.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 80113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/isp116x-hcd.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 87353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 78873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/sl811_cs.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/ssb-hcd.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/image/ 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/image/mdc800.ko 2023-12-20T04:12:43.820 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/image/microtek.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/ 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/adutux.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/appledisplay.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/cytherm.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/ehset.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/emi26.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/emi62.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/ezusb.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/idmouse.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/iowarrior.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-20T04:12:43.821 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/ldusb.ko 2023-12-20T04:12:43.822 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/legousbtower.ko 2023-12-20T04:12:43.822 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/lvstest.ko 2023-12-20T04:12:43.822 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/sisusbvga/ 2023-12-20T04:12:43.822 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 64841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-20T04:12:43.822 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-20T04:12:43.822 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/usb3503.ko 2023-12-20T04:12:43.822 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/usblcd.ko 2023-12-20T04:12:43.822 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-20T04:12:43.822 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 70809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/usbtest.ko 2023-12-20T04:12:43.822 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/uss720.ko 2023-12-20T04:12:43.822 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/yurex.ko 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/mon/ 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 72953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/mon/usbmon.ko 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/musb/ 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 301697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/musb/musb_hdrc.ko 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/phy/ 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/phy/phy-generic.ko 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/phy/phy-isp1301.ko 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/phy/phy-tahvo.ko 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/aircable.ko 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ark3116.ko 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-20T04:12:43.823 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ch341.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 70257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/cp210x.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/cyberjack.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 58217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/empeg.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/f81232.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 175137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 53113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 137081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 124833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/io_ti.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 67769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ipaq.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ipw.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ir-usb.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 58233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-20T04:12:43.824 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 107521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/keyspan.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/mct_u232.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/metro-usb.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 66449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/mos7720.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 78217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/mos7840.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 56337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/mxuport.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/navman.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/omninet.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/opticon.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 191137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/option.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/oti6858.ko 2023-12-20T04:12:43.825 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 54353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/pl2303.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/qcaux.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/qcserial.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/quatech2.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/safe_serial.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 51625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/sierra.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ssu100.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/symbolserial.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 61489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/usb_debug.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 125609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/usbserial.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/visor.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/whiteheat.ko 2023-12-20T04:12:43.826 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/wishbone-serial.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 52913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/uas.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-karma.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-20T04:12:43.827 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 179969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/usb-storage.ko 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/usbip/ 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 67481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 72177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 107001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/ 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/pci/ 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 154297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 108737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/vfio.ko 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 77609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vhost/ 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 113785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vhost/vhost.ko 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-20T04:12:43.828 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 58089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vhost/vhost_net.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 89249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vhost/vhost_scsi.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/ 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/ 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/aat2870_bl.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/adp5520_bl.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/adp8860_bl.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/adp8870_bl.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/apple_bl.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/as3711_bl.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/bd6107.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/da903x_bl.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/da9052_bl.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/gpio_backlight.ko 2023-12-20T04:12:43.829 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/kb3886_bl.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/lcd.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/lm3533_bl.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/lm3639_bl.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/lv5207lp.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/max8925_bl.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/pandora_bl.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/wm831x_bl.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/ 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/masters/ 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/masters/ds2482.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/masters/ds2490.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/masters/matrox_w1.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/masters/w1-gpio.ko 2023-12-20T04:12:43.830 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/ 2023-12-20T04:12:43.831 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-20T04:12:43.831 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-20T04:12:43.831 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-20T04:12:43.831 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-20T04:12:43.831 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-20T04:12:43.831 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-20T04:12:43.831 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-20T04:12:43.831 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-20T04:12:43.831 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 56425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 102673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/wire.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/ 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/acquirewdt.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/advantechwdt.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/cpu5wdt.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/da9052_wdt.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/da9055_wdt.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/da9063_wdt.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/dw_wdt.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/eurotechwdt.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-20T04:12:43.832 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/hpwdt.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/i6300esb.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/ib700wdt.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/ibmasr.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/it87_wdt.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/kempld_wdt.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/machzwd.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/mena21_wdt.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/nv_tco.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-12-20T04:12:43.833 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/pc87413_wdt.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/retu_wdt.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sc1200wdt.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/softdog.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/twl4030_wdt.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/via_wdt.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-20T04:12:43.834 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/wafer5823wdt.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/wdt_pci.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/wm831x_wdt.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/wm8350_wdt.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/xen_wdt.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/ 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-evtchn.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-gntalloc.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-gntdev.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-pciback/ 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 176377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 40121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-privcmd.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 71249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-scsiback.ko 2023-12-20T04:12:43.835 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xenfs/ 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/9p/ 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 166425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/9p/9p.ko 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/adfs/ 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 61073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/adfs/adfs.ko 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/affs/ 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 163593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/affs/affs.ko 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/afs/ 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 932817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/afs/kafs.ko 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/autofs/ 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 101057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/autofs/autofs4.ko 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/befs/ 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 64545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/befs/befs.ko 2023-12-20T04:12:43.836 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/bfs/ 2023-12-20T04:12:43.837 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/bfs/bfs.ko 2023-12-20T04:12:43.837 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/binfmt_misc.ko 2023-12-20T04:12:43.837 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/btrfs/ 2023-12-20T04:12:43.837 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 3402729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/btrfs/btrfs.ko 2023-12-20T04:12:43.837 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/cachefiles/ 2023-12-20T04:12:43.837 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 306009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/cachefiles/cachefiles.ko 2023-12-20T04:12:43.837 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ceph/ 2023-12-20T04:12:43.837 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1333017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ceph/ceph.ko 2023-12-20T04:12:43.837 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/coda/ 2023-12-20T04:12:43.837 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 100801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/coda/coda.ko 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/configfs/ 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 108393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/configfs/configfs.ko 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/cramfs/ 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 29625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/cramfs/cramfs.ko 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/dlm/ 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 642513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/dlm/dlm.ko 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/efs/ 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/efs/efs.ko 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/f2fs/ 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1505025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/f2fs/f2fs.ko 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fat/ 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fat/msdos.ko 2023-12-20T04:12:43.838 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/freevxfs/ 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/freevxfs/freevxfs.ko 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fscache/ 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 226481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fscache/fscache.ko 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fuse/ 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fuse/cuse.ko 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/gfs2/ 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1231529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/gfs2/gfs2.ko 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hfs/ 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 141209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hfs/hfs.ko 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hfsplus/ 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 222817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hfsplus/hfsplus.ko 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hpfs/ 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 192329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hpfs/hpfs.ko 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/isofs/ 2023-12-20T04:12:43.839 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 74065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/isofs/isofs.ko 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/jfs/ 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 419497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/jfs/jfs.ko 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/lockd/ 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 280313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/lockd/lockd.ko 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/minix/ 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 89193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/minix/minix.ko 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/netfs/ 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 124649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/netfs/netfs.ko 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/ 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/blocklayout/ 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 81449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/filelayout/ 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 59449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/flexfilelayout/ 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 117465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-20T04:12:43.840 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1055721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/nfs.ko 2023-12-20T04:12:43.841 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 66441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/nfsv2.ko 2023-12-20T04:12:43.841 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 120177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/nfsv3.ko 2023-12-20T04:12:43.841 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 2042137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/nfsv4.ko 2023-12-20T04:12:43.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs_common/ 2023-12-20T04:12:43.880 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs_common/grace.ko 2023-12-20T04:12:43.880 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs_common/nfs_acl.ko 2023-12-20T04:12:43.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfsd/ 2023-12-20T04:12:43.880 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1516873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfsd/nfsd.ko 2023-12-20T04:12:43.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nilfs2/ 2023-12-20T04:12:43.880 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 592833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nilfs2/nilfs2.ko 2023-12-20T04:12:43.880 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/ 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-celtic.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-centeuro.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-croatian.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-cyrillic.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-gaelic.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-greek.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-iceland.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-inuit.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-roman.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-romanian.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-turkish.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_ascii.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp1250.ko 2023-12-20T04:12:43.881 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp1251.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp1255.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp737.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp775.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp850.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp852.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp855.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp857.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp860.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp861.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp862.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp863.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp864.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp865.ko 2023-12-20T04:12:43.882 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp866.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp869.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp874.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 94137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp932.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 142673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp936.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 165745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp949.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 118121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp950.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_euc-jp.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-1.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-13.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-14.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-15.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-2.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-3.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-4.ko 2023-12-20T04:12:43.883 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-5.ko 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-6.ko 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-7.ko 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-9.ko 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_koi8-r.ko 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_koi8-ru.ko 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_koi8-u.ko 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_utf8.ko 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ntfs/ 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 219417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ntfs/ntfs.ko 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/ 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/cluster/ 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 244409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/dlm/ 2023-12-20T04:12:43.884 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 440737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/dlmfs/ 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 54049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 2592433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/ocfs2.ko 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/omfs/ 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 53433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/omfs/omfs.ko 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/overlayfs/ 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 352161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/overlayfs/overlay.ko 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/pstore/ 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 54809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/pstore/ramoops.ko 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/qnx4/ 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/qnx4/qnx4.ko 2023-12-20T04:12:43.885 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/qnx6/ 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 46689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/qnx6/qnx6.ko 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/quota/ 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/quota/quota_tree.ko 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/quota/quota_v1.ko 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/quota/quota_v2.ko 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/reiserfs/ 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 532673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/reiserfs/reiserfs.ko 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/romfs/ 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/romfs/romfs.ko 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/ 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/client/ 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 2668401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/client/cifs.ko 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/common/ 2023-12-20T04:12:43.886 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/common/cifs_arc4.ko 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/common/cifs_md4.ko 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/squashfs/ 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 134641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/squashfs/squashfs.ko 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/sysv/ 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 90721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/sysv/sysv.ko 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/udf/ 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 247737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/udf/udf.ko 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ufs/ 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 162497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ufs/ufs.ko 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/xfs/ 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 4011017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/xfs/xfs.ko 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/ 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/locking/ 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 75489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/locking/locktorture.ko 2023-12-20T04:12:43.887 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/time/ 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/time/test_udelay.ko 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 60553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/torture.ko 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/ 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 6833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crc-itu-t.ko 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 6529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crc7.ko 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crc8.ko 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crypto/ 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crypto/libarc4.ko 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crypto/libdes.ko 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/interval_tree_test.ko 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/libcrc32c.ko 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/lru_cache.ko 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/lz4/ 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/lz4/lz4_compress.ko 2023-12-20T04:12:43.888 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/lz4/lz4hc_compress.ko 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/math/ 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 6617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/math/cordic.ko 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/memory-notifier-error-inject.ko 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/notifier-error-inject.ko 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/percpu_test.ko 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/pm-notifier-error-inject.ko 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/raid6/ 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 143433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/raid6/raid6_pq.ko 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/rbtree_test.ko 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/reed_solomon/ 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test-kstrtox.ko 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test-string_helpers.ko 2023-12-20T04:12:43.889 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 4760425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test_bpf.ko 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test_firmware.ko 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 6369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test_module.ko 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test_user_copy.ko 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/ts_bm.ko 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/ts_fsm.ko 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/ts_kmp.ko 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/xz/ 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/xz/xz_dec_test.ko 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/mm/ 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/mm/hwpoison-inject.ko 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/ 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 55737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/6lowpan.ko 2023-12-20T04:12:43.890 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_dest.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_fragment.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_hop.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 6985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_mobility.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_routing.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_udp.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/ 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/garp.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/mrp.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/p8022.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/psnap.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/stp.ko 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/8021q/ 2023-12-20T04:12:43.891 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 81737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/8021q/8021q.ko 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/9p/ 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 198337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/9p/9pnet.ko 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/9p/9pnet_fd.ko 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/9p/9pnet_rdma.ko 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/9p/9pnet_virtio.ko 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/batman-adv/ 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 649297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/batman-adv/batman-adv.ko 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/ 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 63809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/br_netfilter.ko 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 777689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/bridge.ko 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-20T04:12:43.892 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-20T04:12:43.893 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 55929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebtables.ko 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ceph/ 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1046897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ceph/libceph.ko 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/core/ 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 125993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/core/pktgen.ko 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dccp/ 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 194209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dccp/dccp.ko 2023-12-20T04:12:43.894 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dccp/dccp_diag.ko 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 57881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dccp/dccp_ipv4.ko 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 60337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dccp/dccp_ipv6.ko 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/ 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 390473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/dsa_core.ko 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/tag_brcm.ko 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/tag_dsa.ko 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/tag_none.ko 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/tag_trailer.ko 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/hsr/ 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 99729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/hsr/hsr.ko 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ieee802154/ 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ieee802154/6lowpan/ 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 45233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 254625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ieee802154/ieee802154.ko 2023-12-20T04:12:43.895 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ah4.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/esp4.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 48337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/fou.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/gre.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 48657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/inet_diag.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 60633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ip_gre.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 61225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ip_tunnel.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ip_vti.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ipcomp.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ipip.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-20T04:12:43.896 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ip_tables.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-20T04:12:43.897 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-20T04:12:43.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-20T04:12:43.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-12-20T04:12:43.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-20T04:12:43.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-20T04:12:43.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-20T04:12:43.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-20T04:12:43.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-20T04:12:43.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_bic.ko 2023-12-20T04:12:43.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_dctcp.ko 2023-12-20T04:12:43.898 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_diag.ko 2023-12-20T04:12:43.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_highspeed.ko 2023-12-20T04:12:43.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_htcp.ko 2023-12-20T04:12:43.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_hybla.ko 2023-12-20T04:12:43.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_illinois.ko 2023-12-20T04:12:43.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_lp.ko 2023-12-20T04:12:43.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_scalable.ko 2023-12-20T04:12:43.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_vegas.ko 2023-12-20T04:12:43.899 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_veno.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_westwood.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_yeah.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tunnel4.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/udp_diag.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 41665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/udp_tunnel.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ah6.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/esp6.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/fou6.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 80897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ip6_gre.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 93577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ip6_tunnel.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-20T04:12:43.900 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 53273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ip6_vti.ko 2023-12-20T04:12:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ipcomp6.ko 2023-12-20T04:12:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/mip6.ko 2023-12-20T04:12:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ 2023-12-20T04:12:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-20T04:12:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-12-20T04:12:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-20T04:12:43.901 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-20T04:12:43.902 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-20T04:12:43.902 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-20T04:12:43.902 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-20T04:12:43.902 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-20T04:12:43.902 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-20T04:12:43.902 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-20T04:12:43.902 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-20T04:12:43.902 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-20T04:12:43.902 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-20T04:12:43.902 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 74073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/sit.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/tunnel6.ko 2023-12-20T04:12:43.903 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-20T04:12:43.904 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/key/ 2023-12-20T04:12:43.904 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 82833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/key/af_key.ko 2023-12-20T04:12:43.904 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/ 2023-12-20T04:12:43.904 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 125281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_core.ko 2023-12-20T04:12:43.904 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-20T04:12:43.904 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 25953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_eth.ko 2023-12-20T04:12:43.904 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_ip.ko 2023-12-20T04:12:43.904 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_ip6.ko 2023-12-20T04:12:43.904 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_netlink.ko 2023-12-20T04:12:43.904 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 53241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_ppp.ko 2023-12-20T04:12:43.904 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/lapb/ 2023-12-20T04:12:43.905 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 47977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/lapb/lapb.ko 2023-12-20T04:12:43.905 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/llc/ 2023-12-20T04:12:43.905 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/llc/llc.ko 2023-12-20T04:12:43.905 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 267913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/llc/llc2.ko 2023-12-20T04:12:43.905 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/mac802154/ 2023-12-20T04:12:43.905 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 290417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/mac802154/mac802154.ko 2023-12-20T04:12:43.905 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/mpls/ 2023-12-20T04:12:43.905 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/mpls/mpls_gso.ko 2023-12-20T04:12:43.905 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ 2023-12-20T04:12:43.906 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ 2023-12-20T04:12:43.906 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 119585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set.ko 2023-12-20T04:12:43.906 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-20T04:12:43.906 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-20T04:12:43.906 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-20T04:12:43.906 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 100297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-20T04:12:43.906 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 101441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 102817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 103417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 112849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 60329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 109593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 113129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 112585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 111585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 114465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 411729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-20T04:12:43.907 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conncount.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 408257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-20T04:12:43.908 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-20T04:12:43.909 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-20T04:12:43.909 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 118145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-20T04:12:43.909 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-20T04:12:43.909 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-20T04:12:43.911 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 107233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-20T04:12:43.911 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-20T04:12:43.911 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-20T04:12:43.911 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 66641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-20T04:12:43.911 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 89425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_flow_table.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 37129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_log_syslog.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 95353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat_irc.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat_sip.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 44065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 615649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_tables.ko 2023-12-20T04:12:43.912 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 42561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 39785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink_log.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_chain_nat.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_compat.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 40721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_ct.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_fib.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-20T04:12:43.913 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_hash.ko 2023-12-20T04:12:43.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_limit.ko 2023-12-20T04:12:43.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_log.ko 2023-12-20T04:12:43.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_masq.ko 2023-12-20T04:12:43.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_nat.ko 2023-12-20T04:12:43.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_numgen.ko 2023-12-20T04:12:43.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_osf.ko 2023-12-20T04:12:43.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_queue.ko 2023-12-20T04:12:43.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_redir.ko 2023-12-20T04:12:43.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_reject.ko 2023-12-20T04:12:43.914 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_reject_inet.ko 2023-12-20T04:12:43.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_socket.ko 2023-12-20T04:12:43.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_synproxy.ko 2023-12-20T04:12:43.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_tproxy.ko 2023-12-20T04:12:43.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 89313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/x_tables.ko 2023-12-20T04:12:43.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_AUDIT.ko 2023-12-20T04:12:43.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-20T04:12:43.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-20T04:12:43.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-20T04:12:43.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 18169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_CT.ko 2023-12-20T04:12:43.915 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_DSCP.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_HL.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_HMARK.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_LED.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_LOG.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_NETMAP.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_NFLOG.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_RATEEST.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 14337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_SECMARK.ko 2023-12-20T04:12:43.916 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-20T04:12:43.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-20T04:12:43.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_TEE.ko 2023-12-20T04:12:43.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_TPROXY.ko 2023-12-20T04:12:43.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_TRACE.ko 2023-12-20T04:12:43.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_addrtype.ko 2023-12-20T04:12:43.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_bpf.ko 2023-12-20T04:12:43.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_cgroup.ko 2023-12-20T04:12:43.917 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_cluster.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_comment.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_connbytes.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_connlabel.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_connlimit.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_connmark.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_conntrack.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_cpu.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_dccp.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_devgroup.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_dscp.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_ecn.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_esp.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 43289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_hashlimit.ko 2023-12-20T04:12:43.918 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_helper.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_hl.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_ipcomp.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_iprange.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_ipvs.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_l2tp.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_length.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_limit.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_mac.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_mark.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_multiport.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_nat.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_nfacct.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_osf.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_owner.ko 2023-12-20T04:12:43.919 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_physdev.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_pkttype.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_policy.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_quota.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_rateest.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_realm.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_recent.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_sctp.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_set.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 16089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_socket.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_state.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_statistic.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_string.ko 2023-12-20T04:12:43.920 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 8553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_tcpmss.ko 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_tcpudp.ko 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_time.ko 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_u32.ko 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netlink/ 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netlink/netlink_diag.ko 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/nsh/ 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 12425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/nsh/nsh.ko 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/openvswitch/ 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 380033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/openvswitch/openvswitch.ko 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/openvswitch/vport-geneve.ko 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/openvswitch/vport-gre.ko 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 11793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/openvswitch/vport-vxlan.ko 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/packet/ 2023-12-20T04:12:43.921 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/packet/af_packet_diag.ko 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/phonet/ 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 110201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/phonet/phonet.ko 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 54721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/phonet/pn_pep.ko 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rds/ 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 251481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rds/rds.ko 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 164937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rds/rds_rdma.ko 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 67377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rds/rds_tcp.ko 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rxrpc/ 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 819657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rxrpc/rxrpc.ko 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/ 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_csum.ko 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_gact.ko 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_ipt.ko 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_mirred.ko 2023-12-20T04:12:43.922 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_nat.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 43449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_pedit.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_police.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_simple.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_skbedit.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_vlan.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_basic.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_bpf.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_cgroup.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_flow.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_fw.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 38425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_route.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 61409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_u32.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_cmp.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 13409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_ipset.ko 2023-12-20T04:12:43.923 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_meta.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_nbyte.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_text.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_u32.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 21649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_choke.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_codel.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 34633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_drr.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 33609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_fq.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_fq_codel.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 35433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_gred.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 49817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_hfsc.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_hhf.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 66393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_htb.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_ingress.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 30273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_mqprio.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_mqprio_lib.ko 2023-12-20T04:12:43.924 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_multiq.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 36137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_netem.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 27497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_pie.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_plug.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 24729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_prio.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 48729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_qfq.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 32905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_red.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 26881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_sfb.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 31353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_sfq.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_tbf.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 28105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_teql.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sctp/ 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 929745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sctp/sctp.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 19033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sctp/sctp_diag.ko 2023-12-20T04:12:43.925 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/ 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/auth_gss/ 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 336529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 54737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 1520097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/sunrpc.ko 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/xprtrdma/ 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 707657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/tipc/ 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/tipc/diag.ko 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 862665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/tipc/tipc.ko 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/unix/ 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/unix/unix_diag.ko 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/ 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 85409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-20T04:12:43.926 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 68049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 96193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/vsock.ko 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 10705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 17521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/x25/ 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 125177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/x25/x25.ko 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/xfrm/ 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 22657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/xfrm/xfrm_algo.ko 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 20649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 102257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/xfrm/xfrm_user.ko 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/virt/ 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/virt/lib/ 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 15537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/virt/lib/irqbypass.ko 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 640081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.alias 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 624107 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.alias.bin 2023-12-20T04:12:43.927 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 7663 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.builtin 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 23428 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.builtin.alias.bin 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 9821 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.builtin.bin 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 70004 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.builtin.modinfo 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 146313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.dep 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 232548 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.dep.bin 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 210 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.devname 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 72606 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.order 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 879 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.softdep 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 254438 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.symbols 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 309363 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.symbols.bin 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./usr/ 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./usr/share/ 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./usr/share/doc/ 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./usr/share/doc/linux-image-6.6.0-g93572f31db89/ 2023-12-20T04:12:43.928 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 169 2023-12-20 01:01 ./usr/share/doc/linux-image-6.6.0-g93572f31db89/changelog.Debian.gz 2023-12-20T04:12:43.929 INFO:teuthology.orchestra.run.smithi039.stdout:-rw-r--r-- root/root 691 2023-12-20 01:01 ./usr/share/doc/linux-image-6.6.0-g93572f31db89/copyright 2023-12-20T04:12:43.929 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g93572f31db89 2023-12-20T04:12:43.930 INFO:teuthology.task.kernel:searching for kernel 6.6.0-g93572f31db89 2023-12-20T04:12:43.930 DEBUG:teuthology.orchestra.run.smithi039:> egrep '(submenu|menuentry.*6.6.0-g93572f31db89).*{' /boot/grub/grub.cfg 2023-12-20T04:12:43.931 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-20T04:12:43.931 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-20T04:12:43.937 INFO:teuthology.orchestra.run.smithi039.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-20T04:12:43.937 INFO:teuthology.orchestra.run.smithi039.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g93572f31db89' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g93572f31db89-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-20T04:12:43.937 INFO:teuthology.orchestra.run.smithi039.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g93572f31db89 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g93572f31db89-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-20T04:12:43.938 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-12-20T04:12:43.938 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-g93572f31db89 2023-12-20T04:12:43.938 DEBUG:teuthology.orchestra.run.smithi039:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-12-20T04:12:43.980 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi039.front.sepia.ceph.com to complete... 2023-12-20T04:12:43.996 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-20T04:12:44.002 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-20T04:12:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-20T04:12:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-12-20T04:12:44.179 INFO:teuthology.orchestra.run.smithi032.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-20T04:12:44.216 INFO:teuthology.orchestra.run.smithi032.stderr:done 2023-12-20T04:12:44.328 INFO:teuthology.orchestra.run.smithi039.stderr:Sourcing file `/etc/default/grub' 2023-12-20T04:12:44.328 INFO:teuthology.orchestra.run.smithi039.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-20T04:12:44.358 INFO:teuthology.orchestra.run.smithi039.stderr:Generating grub configuration file ... 2023-12-20T04:12:44.429 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-deb -c /tmp/linux-image.deb 2023-12-20T04:12:44.438 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./ 2023-12-20T04:12:44.438 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./boot/ 2023-12-20T04:12:44.438 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 4415617 2023-12-20 01:01 ./boot/System.map-6.6.0-g93572f31db89 2023-12-20T04:12:44.450 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 169214 2023-12-20 01:01 ./boot/config-6.6.0-g93572f31db89 2023-12-20T04:12:44.450 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11352864 2023-12-20 01:01 ./boot/vmlinuz-6.6.0-g93572f31db89 2023-12-20T04:12:44.472 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/ 2023-12-20T04:12:44.472 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/kernel/ 2023-12-20T04:12:44.472 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/kernel/postinst.d/ 2023-12-20T04:12:44.472 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/kernel/postrm.d/ 2023-12-20T04:12:44.472 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/kernel/preinst.d/ 2023-12-20T04:12:44.472 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./etc/kernel/prerm.d/ 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/ 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/ 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/ 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/ 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/ 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/ 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/ 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 490233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/aesni-intel.ko 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/blowfish-x86_64.ko 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/camellia-aesni-avx-x86_64.ko 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/camellia-aesni-avx2.ko 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/camellia-x86_64.ko 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/cast5-avx-x86_64.ko 2023-12-20T04:12:44.473 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 152241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/cast6-avx-x86_64.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/crc32-pclmul.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/crct10dif-pclmul.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/des3_ede-x86_64.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/ghash-clmulni-intel.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/serpent-avx-x86_64.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/serpent-avx2.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/serpent-sse2-x86_64.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/sha1-ssse3.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/sha256-ssse3.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/sha512-ssse3.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/twofish-avx-x86_64.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/twofish-x86_64-3way.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/crypto/twofish-x86_64.ko 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/ 2023-12-20T04:12:44.474 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/cpu/ 2023-12-20T04:12:44.475 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/cpu/mce/ 2023-12-20T04:12:44.475 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/cpu/mce/mce-inject.ko 2023-12-20T04:12:44.475 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/cpuid.ko 2023-12-20T04:12:44.475 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kernel/msr.ko 2023-12-20T04:12:44.475 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kvm/ 2023-12-20T04:12:44.475 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 354433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kvm/kvm-amd.ko 2023-12-20T04:12:44.475 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 881625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kvm/kvm-intel.ko 2023-12-20T04:12:44.476 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2275865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/arch/x86/kvm/kvm.ko 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/ 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/af_alg.ko 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/algif_hash.ko 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/algif_skcipher.ko 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/ansi_cprng.ko 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/anubis.ko 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/arc4.ko 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/asymmetric_keys/ 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/asymmetric_keys/pkcs7_test_key.ko 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/ 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/async_memcpy.ko 2023-12-20T04:12:44.530 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/async_pq.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/async_raid6_recov.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/async_tx.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/async_xor.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/async_tx/raid6test.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/authenc.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/authencesn.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/blake2b_generic.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/blowfish_common.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/blowfish_generic.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/camellia_generic.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cast5_generic.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cast6_generic.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cast_common.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/ccm.ko 2023-12-20T04:12:44.531 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cmac.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/crc32_generic.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cryptd.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/crypto_engine.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/crypto_simd.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/crypto_user.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/cts.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/des_generic.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/dh_generic.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/echainiv.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/essiv.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/fcrypt.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/khazad.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/lrw.ko 2023-12-20T04:12:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/lz4.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/lz4hc.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/md4.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/michael_mic.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/pcbc.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/pcrypt.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/rmd160.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/seed.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/serpent_generic.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 149193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/tcrypt.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/tea.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/twofish_common.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/twofish_generic.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/vmac.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/wp512.ko 2023-12-20T04:12:44.533 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/xcbc.ko 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/xor.ko 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/xts.ko 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/crypto/xxhash_generic.ko 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/ 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/acpi_extlog.ko 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/acpi_ipmi.ko 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/acpi_pad.ko 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/apei/ 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/apei/einj.ko 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/ec_sys.ko 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/platform_profile.ko 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/sbs.ko 2023-12-20T04:12:44.534 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/sbshc.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/acpi/video.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/ 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/acard-ahci.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 88865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/ahci.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/ahci_platform.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 88289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/libahci.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/libahci_platform.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_acpi.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_ali.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_amd.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_artop.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_atiixp.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_atp867x.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_cmd640.ko 2023-12-20T04:12:44.535 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_cmd64x.ko 2023-12-20T04:12:44.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_cypress.ko 2023-12-20T04:12:44.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_efar.ko 2023-12-20T04:12:44.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_hpt366.ko 2023-12-20T04:12:44.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_hpt37x.ko 2023-12-20T04:12:44.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_hpt3x2n.ko 2023-12-20T04:12:44.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_hpt3x3.ko 2023-12-20T04:12:44.536 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_it8213.ko 2023-12-20T04:12:44.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_it821x.ko 2023-12-20T04:12:44.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_jmicron.ko 2023-12-20T04:12:44.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_legacy.ko 2023-12-20T04:12:44.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_marvell.ko 2023-12-20T04:12:44.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_mpiix.ko 2023-12-20T04:12:44.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_netcell.ko 2023-12-20T04:12:44.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_ninja32.ko 2023-12-20T04:12:44.537 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_ns87410.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_ns87415.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_oldpiix.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_opti.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_optidma.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_pcmcia.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_pdc2027x.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_pdc202xx_old.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_piccolo.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_radisys.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_rdc.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_rz1000.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_sch.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_serverworks.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_sil680.ko 2023-12-20T04:12:44.538 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_sl82c105.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_triflex.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pata_via.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/pdc_adma.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_inic162x.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_mv.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_nv.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_promise.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_qstor.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_sil.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_sil24.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_sis.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_svw.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_sx4.ko 2023-12-20T04:12:44.539 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_uli.ko 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_via.ko 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ata/sata_vsc.ko 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/auxdisplay/ 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/auxdisplay/ks0108.ko 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/bcma/ 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/bcma/bcma.ko 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/ 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/aoe/ 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/aoe/aoe.ko 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/drbd/ 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 965585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/drbd/drbd.ko 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 173225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/floppy.ko 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/mtip32xx/ 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/mtip32xx/mtip32xx.ko 2023-12-20T04:12:44.540 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/nbd.ko 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/null_blk/ 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 110385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/null_blk/null_blk.ko 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 98193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/pktcdvd.ko 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 263769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/rbd.ko 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/xen-blkback/ 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 98145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/xen-blkback/xen-blkback.ko 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/zram/ 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/block/zram/zram.ko 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/applicom.ko 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hangcheck-timer.ko 2023-12-20T04:12:44.541 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/ 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/amd-rng.ko 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/intel-rng.ko 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/timeriomem-rng.ko 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/via-rng.ko 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/hw_random/virtio-rng.ko 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_devintf.ko 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 148601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_msghandler.ko 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_poweroff.ko 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 139313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_si.ko 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_ssif.ko 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ipmi/ipmi_watchdog.ko 2023-12-20T04:12:44.542 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/lp.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/mwave/ 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/mwave/mwave.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/nvram.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/ppdev.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tlclk.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/ 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_atmel.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_i2c_atmel.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_i2c_infineon.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_i2c_nuvoton.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_infineon.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/tpm_nsc.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/tpm/xen-tpmfront.ko 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/xillybus/ 2023-12-20T04:12:44.543 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/xillybus/xillybus_class.ko 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/xillybus/xillybus_core.ko 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/char/xillybus/xillybus_pcie.ko 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/clk/ 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/clk/clk-palmas.ko 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/clk/clk-si5351.ko 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/clk/clk-twl6040.ko 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/clk/clk-wm831x.ko 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/cpufreq/ 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/cpufreq/amd_freq_sensitivity.ko 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/cpufreq/p4-clockmod.ko 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/cpufreq/speedstep-lib.ko 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/ 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/ccp/ 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/ccp/ccp-crypto.ko 2023-12-20T04:12:44.544 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 243025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/ccp/ccp.ko 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/ 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/qat/ 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/qat/qat_common/ 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 438761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/qat/qat_common/intel_qat.ko 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/qat/qat_dh895xcc/ 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/intel/qat/qat_dh895xcc/qat_dh895xcc.ko 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/padlock-aes.ko 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/padlock-sha.ko 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/virtio/ 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/crypto/virtio/virtio_crypto.ko 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dca/ 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dca/dca.ko 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dma/ 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dma/dw/ 2023-12-20T04:12:44.545 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dma/dw/dw_dmac.ko 2023-12-20T04:12:44.546 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dma/ioat/ 2023-12-20T04:12:44.546 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 122569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/dma/ioat/ioatdma.ko 2023-12-20T04:12:44.546 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/ 2023-12-20T04:12:44.546 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/amd64_edac.ko 2023-12-20T04:12:44.546 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/e752x_edac.ko 2023-12-20T04:12:44.557 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/edac_mce_amd.ko 2023-12-20T04:12:44.557 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i3000_edac.ko 2023-12-20T04:12:44.557 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i3200_edac.ko 2023-12-20T04:12:44.557 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i5100_edac.ko 2023-12-20T04:12:44.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i5400_edac.ko 2023-12-20T04:12:44.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i7300_edac.ko 2023-12-20T04:12:44.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i7core_edac.ko 2023-12-20T04:12:44.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/i82975x_edac.ko 2023-12-20T04:12:44.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/ie31200_edac.ko 2023-12-20T04:12:44.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/sb_edac.ko 2023-12-20T04:12:44.558 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/edac/x38_edac.ko 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/ 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-gpio.ko 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-max14577.ko 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-max77693.ko 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-max8997.ko 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-palmas.ko 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-rt8973a.ko 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/extcon/extcon-sm5502.ko 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/firmware/ 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/firmware/dmi-sysfs.ko 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/firmware/efi/ 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/firmware/efi/efi-pstore.ko 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/firmware/iscsi_ibft.ko 2023-12-20T04:12:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/ 2023-12-20T04:12:44.560 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-adp5520.ko 2023-12-20T04:12:44.560 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-amd8111.ko 2023-12-20T04:12:44.560 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-arizona.ko 2023-12-20T04:12:44.560 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-da9052.ko 2023-12-20T04:12:44.560 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-da9055.ko 2023-12-20T04:12:44.560 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-dln2.ko 2023-12-20T04:12:44.560 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-f7188x.ko 2023-12-20T04:12:44.565 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-generic.ko 2023-12-20T04:12:44.565 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-ich.ko 2023-12-20T04:12:44.565 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-janz-ttl.ko 2023-12-20T04:12:44.565 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-kempld.ko 2023-12-20T04:12:44.565 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-lp3943.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-max7300.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-max730x.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-max732x.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-ml-ioh.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-pca953x.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-pcf857x.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-rdc321x.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-sch.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-sch311x.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-tps65912.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-twl4030.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-twl6040.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-viperboard.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-vx855.ko 2023-12-20T04:12:44.566 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-wm831x.ko 2023-12-20T04:12:44.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-wm8350.ko 2023-12-20T04:12:44.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/gpio/gpio-wm8994.ko 2023-12-20T04:12:44.567 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/ 2023-12-20T04:12:44.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-a4tech.ko 2023-12-20T04:12:44.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-apple.ko 2023-12-20T04:12:44.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-appleir.ko 2023-12-20T04:12:44.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-aureal.ko 2023-12-20T04:12:44.567 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-axff.ko 2023-12-20T04:12:44.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-belkin.ko 2023-12-20T04:12:44.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-cherry.ko 2023-12-20T04:12:44.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-chicony.ko 2023-12-20T04:12:44.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-cp2112.ko 2023-12-20T04:12:44.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-cypress.ko 2023-12-20T04:12:44.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-dr.ko 2023-12-20T04:12:44.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-elecom.ko 2023-12-20T04:12:44.568 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-elo.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-emsff.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-ezkey.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-gaff.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-generic.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-gt683r.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-gyration.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-holtek-kbd.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-holtek-mouse.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-holtekff.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-hyperv.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-icade.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-kensington.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-keytouch.ko 2023-12-20T04:12:44.569 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-kye.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-lcpower.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-led.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-lenovo.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-lg-g15.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-logitech-dj.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-logitech-hidpp.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-logitech.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-magicmouse.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-microsoft.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-monterey.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-multitouch.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-ntrig.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-ortek.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-penmount.ko 2023-12-20T04:12:44.570 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-petalynx.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-picolcd.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-pl.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-plantronics.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-primax.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-rmi.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-arvo.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-common.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-isku.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-kone.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-koneplus.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-konepure.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-kovaplus.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-lua.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-pyra.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-ryos.ko 2023-12-20T04:12:44.571 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat-savu.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-roccat.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-saitek.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-samsung.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-sensor-hub.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-sjoy.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-sony.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-speedlink.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-steelseries.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-sunplus.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-thrustmaster.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-tivo.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-tmff.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-topseed.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-twinhan.ko 2023-12-20T04:12:44.572 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-uclogic.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-waltop.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 143473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-wiimote.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-xinmo.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-zpff.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid-zydacron.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 267825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/hid.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/uhid.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/usbhid/ 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/usbhid/usbhid.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/usbhid/usbkbd.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/usbhid/usbmouse.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 217873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hid/wacom.ko 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hv/ 2023-12-20T04:12:44.573 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hv/hv_balloon.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hv/hv_utils.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 305465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hv/hv_vmbus.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/abituguru.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/abituguru3.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/acpi_power_meter.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ad7414.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ad7418.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adc128d818.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adm1025.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adm1026.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adm1029.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adm1031.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adm9240.ko 2023-12-20T04:12:44.574 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ads7828.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7410.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7411.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7462.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7470.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7475.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/adt7x10.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/amc6821.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/applesmc.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/asb100.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/asc7621.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/asus_atk0110.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/atxp1.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/coretemp.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/da9052-hwmon.ko 2023-12-20T04:12:44.575 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/da9055-hwmon.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/dell-smm-hwmon.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/dme1737.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ds1621.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ds620.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/emc1403.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/emc2103.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/emc6w201.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/f71805f.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/f71882fg.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/f75375s.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/fam15h_power.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/fschmd.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/g760a.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/g762.ko 2023-12-20T04:12:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/gl518sm.ko 2023-12-20T04:12:44.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/gl520sm.ko 2023-12-20T04:12:44.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/hih6130.ko 2023-12-20T04:12:44.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/hwmon-vid.ko 2023-12-20T04:12:44.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/i5500_temp.ko 2023-12-20T04:12:44.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/i5k_amb.ko 2023-12-20T04:12:44.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ibmaem.ko 2023-12-20T04:12:44.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ibmpex.ko 2023-12-20T04:12:44.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ina209.ko 2023-12-20T04:12:44.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ina2xx.ko 2023-12-20T04:12:44.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 130449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/it87.ko 2023-12-20T04:12:44.577 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/jc42.ko 2023-12-20T04:12:44.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/k10temp.ko 2023-12-20T04:12:44.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/k8temp.ko 2023-12-20T04:12:44.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lineage-pem.ko 2023-12-20T04:12:44.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm63.ko 2023-12-20T04:12:44.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm73.ko 2023-12-20T04:12:44.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm75.ko 2023-12-20T04:12:44.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm77.ko 2023-12-20T04:12:44.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm78.ko 2023-12-20T04:12:44.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm80.ko 2023-12-20T04:12:44.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm83.ko 2023-12-20T04:12:44.578 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm85.ko 2023-12-20T04:12:44.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm87.ko 2023-12-20T04:12:44.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm90.ko 2023-12-20T04:12:44.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm92.ko 2023-12-20T04:12:44.579 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm93.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm95234.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm95241.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/lm95245.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc2945.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4151.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4215.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4222.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4245.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4260.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/ltc4261.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max16065.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max1619.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max1668.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max197.ko 2023-12-20T04:12:44.580 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max6639.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max6650.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/max6697.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/mc13783-adc.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/mcp3021.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/menf21bmc_hwmon.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/nct6683.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/nct6775-core.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/nct6775.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/nct7802.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pc87360.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pc87427.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pcf8591.ko 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/ 2023-12-20T04:12:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/adm1275.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/lm25066.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/ltc2978.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/max16064.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/max34440.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/max8688.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/pmbus.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/pmbus_core.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/tps40422.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/ucd9000.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/ucd9200.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/pmbus/zl6100.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/powr1220.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sch5627.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sch5636.ko 2023-12-20T04:12:44.582 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sch56xx-common.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sht15.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sht21.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/shtc1.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/sis5595.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/smsc47b397.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/smsc47m1.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/smsc47m192.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/thmc50.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/tmp102.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/tmp103.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/tmp401.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/tmp421.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/via-cputemp.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/via686a.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/vt1211.ko 2023-12-20T04:12:44.583 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/vt8231.ko 2023-12-20T04:12:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83627ehf.ko 2023-12-20T04:12:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83627hf.ko 2023-12-20T04:12:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83781d.ko 2023-12-20T04:12:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83791d.ko 2023-12-20T04:12:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83792d.ko 2023-12-20T04:12:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83793.ko 2023-12-20T04:12:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83795.ko 2023-12-20T04:12:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83l785ts.ko 2023-12-20T04:12:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/w83l786ng.ko 2023-12-20T04:12:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/wm831x-hwmon.ko 2023-12-20T04:12:44.584 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/hwmon/wm8350-hwmon.ko 2023-12-20T04:12:44.585 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/ 2023-12-20T04:12:44.585 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/algos/ 2023-12-20T04:12:44.585 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/algos/i2c-algo-bit.ko 2023-12-20T04:12:44.585 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/algos/i2c-algo-pca.ko 2023-12-20T04:12:44.586 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/ 2023-12-20T04:12:44.586 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ali1535.ko 2023-12-20T04:12:44.586 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ali1563.ko 2023-12-20T04:12:44.586 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ali15x3.ko 2023-12-20T04:12:44.586 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-amd756-s4882.ko 2023-12-20T04:12:44.586 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-amd756.ko 2023-12-20T04:12:44.587 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-amd8111.ko 2023-12-20T04:12:44.587 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-cbus-gpio.ko 2023-12-20T04:12:44.587 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ccgx-ucsi.ko 2023-12-20T04:12:44.587 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-cros-ec-tunnel.ko 2023-12-20T04:12:44.587 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-designware-core.ko 2023-12-20T04:12:44.587 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-designware-pci.ko 2023-12-20T04:12:44.588 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-designware-platform.ko 2023-12-20T04:12:44.588 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-diolan-u2c.ko 2023-12-20T04:12:44.588 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-dln2.ko 2023-12-20T04:12:44.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-gpio.ko 2023-12-20T04:12:44.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-i801.ko 2023-12-20T04:12:44.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-isch.ko 2023-12-20T04:12:44.589 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ismt.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-kempld.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-nforce2-s4985.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-nforce2.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-ocores.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-parport.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-pca-platform.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-piix4.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-robotfuzz-osif.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-scmi.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-simtec.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-sis5595.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-sis630.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-sis96x.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-taos-evm.ko 2023-12-20T04:12:44.590 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-tiny-usb.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-via.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-viapro.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-viperboard.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/busses/i2c-xiic.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/i2c-mux.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/i2c-smbus.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/i2c-stub.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/muxes/ 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/muxes/i2c-mux-gpio.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/muxes/i2c-mux-pca9541.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/i2c/muxes/i2c-mux-pca954x.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/ 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 273609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/ib_cm.ko 2023-12-20T04:12:44.591 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 879969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/ib_core.ko 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/ib_umad.ko 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 324217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/ib_uverbs.ko 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/iw_cm.ko 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 282273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/rdma_cm.ko 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/core/rdma_ucm.ko 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/ 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/cxgb4/ 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 493073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/cxgb4/iw_cxgb4.ko 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mlx4/ 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 419737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mlx4/mlx4_ib.ko 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mlx5/ 2023-12-20T04:12:44.592 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 739025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mlx5/mlx5_ib.ko 2023-12-20T04:12:44.593 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-6.6.0-g93572f31db89 2023-12-20T04:12:44.601 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-6.6.0-g93572f31db89 2023-12-20T04:12:44.702 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mthca/ 2023-12-20T04:12:44.702 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 296625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/mthca/ib_mthca.ko 2023-12-20T04:12:44.702 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/ocrdma/ 2023-12-20T04:12:44.702 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 153697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/ocrdma/ocrdma.ko 2023-12-20T04:12:44.702 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/usnic/ 2023-12-20T04:12:44.702 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 168969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/hw/usnic/usnic_verbs.ko 2023-12-20T04:12:44.702 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/ 2023-12-20T04:12:44.702 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/ipoib/ 2023-12-20T04:12:44.702 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 263081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/ipoib/ib_ipoib.ko 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/iser/ 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/iser/ib_iser.ko 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/isert/ 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 111425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/isert/ib_isert.ko 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/srp/ 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 128753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/srp/ib_srp.ko 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/srpt/ 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 129529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/infiniband/ulp/srpt/ib_srpt.ko 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/ 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/evbug.ko 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/ff-memless.ko 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/gameport/ 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/gameport/emu10k1-gp.ko 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/gameport/fm801-gp.ko 2023-12-20T04:12:44.703 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/gameport/gameport.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/joydev.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/ 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/adp5520-keys.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/adp5588-keys.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/adp5589-keys.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/cros_ec_keyb.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/gpio_keys.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/gpio_keys_polled.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/lkkbd.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/lm8323.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/lm8333.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/matrix_keypad.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/max7359_keypad.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/mcs_touchkey.ko 2023-12-20T04:12:44.704 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/mpr121_touchkey.ko 2023-12-20T04:12:44.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/newtonkbd.ko 2023-12-20T04:12:44.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/opencores-kbd.ko 2023-12-20T04:12:44.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/qt1070.ko 2023-12-20T04:12:44.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/qt2160.ko 2023-12-20T04:12:44.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/samsung-keypad.ko 2023-12-20T04:12:44.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/stowaway.ko 2023-12-20T04:12:44.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/sunkbd.ko 2023-12-20T04:12:44.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/tca6416-keypad.ko 2023-12-20T04:12:44.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/tca8418_keypad.ko 2023-12-20T04:12:44.705 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/twl4030_keypad.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/keyboard/xtkbd.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/matrix-keymap.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/ 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/appletouch.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/bcm5974.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 123681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/cyapatp.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/elan_i2c.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/gpio_mouse.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 335929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/psmouse.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/sermouse.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/synaptics_i2c.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/synaptics_usb.ko 2023-12-20T04:12:44.706 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/mouse/vsxxxaa.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/rmi4/ 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 109873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/rmi4/rmi_core.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/ 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/altera_ps2.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/arc_ps2.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/ct82c710.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/hyperv-keyboard.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/parkbd.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/pcips2.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/ps2mult.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/serio_raw.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/serio/serport.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/input/sparse-keymap.ko 2023-12-20T04:12:44.707 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/iommu/ 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/iommu/amd/ 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/iommu/amd/iommu_v2.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/ 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-88pm860x.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-adp5520.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-bd2802.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-blinkm.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-da903x.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-da9052.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-gpio.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lm3530.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lm3533.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lm355x.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lm3642.ko 2023-12-20T04:12:44.708 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lp3944.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-lp8788.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-max8997.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-mc13783.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-menf21bmc.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-pca9532.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-pca955x.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-pca963x.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-ss4200.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-tca6507.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-wm831x-status.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/leds-wm8350.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-audio.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-backlight.ko 2023-12-20T04:12:44.709 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-camera.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-default-on.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-heartbeat.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-oneshot.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-timer.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/leds/trigger/ledtrig-transient.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/ 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/bcache/ 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 654969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/bcache/bcache.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-bio-prison.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-bufio.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-cache-smq.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 156009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-cache.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-crypt.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-delay.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-era.ko 2023-12-20T04:12:44.710 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-flakey.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-log-userspace.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-log.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-mirror.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-multipath.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-queue-length.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-raid.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-region-hash.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-round-robin.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-service-time.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 109529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-snapshot.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-switch.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 181865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-thin-pool.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-verity.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/dm-zero.ko 2023-12-20T04:12:44.711 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/faulty.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/linear.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/multipath.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/persistent-data/ 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 199937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/persistent-data/dm-persistent-data.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/raid0.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/raid1.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 158681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/raid10.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 395721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/md/raid456.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/ 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/ 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 178137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptbase.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptctl.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptfc.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptlan.ko 2023-12-20T04:12:44.712 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 123321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptsas.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 79041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptscsih.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/message/fusion/mptspi.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/ 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/88pm800.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/88pm805.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/88pm80x.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/arizona-i2c.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 123721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/arizona.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/bcm590xx.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/cros_ec_dev.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/dln2.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/janz-cmodio.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/kempld-core.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/lm3533-core.ko 2023-12-20T04:12:44.713 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/lm3533-ctrlbank.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/lp3943.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/lpc_ich.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/lpc_sch.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/max8907.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/mc13xxx-core.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/mc13xxx-i2c.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/menf21bmc.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/pcf50633-adc.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/pcf50633-gpio.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/pcf50633.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/rdc321x-southbridge.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/retu-mfd.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/si476x-core.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/sm501.ko 2023-12-20T04:12:44.714 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/ti_am335x_tscadc.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/tps65010.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/tps6507x.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/viperboard.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/vx855.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/wl1273-core.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/mfd/wm8994.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ad525x_dpot-i2c.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ad525x_dpot.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/altera-stapl/ 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/altera-stapl/altera-stapl.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/apds9802als.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/apds990x.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/bh1770glc.ko 2023-12-20T04:12:44.715 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/c2port/ 2023-12-20T04:12:44.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/c2port/c2port-duramar2150.ko 2023-12-20T04:12:44.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/c2port/core.ko 2023-12-20T04:12:44.716 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/cb710/ 2023-12-20T04:12:44.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/cb710/cb710.ko 2023-12-20T04:12:44.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ds1682.ko 2023-12-20T04:12:44.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/dummy-irq.ko 2023-12-20T04:12:44.716 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/echo/ 2023-12-20T04:12:44.716 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/echo/echo.ko 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/eeprom/ 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/eeprom/at24.ko 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/eeprom/eeprom.ko 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/eeprom/eeprom_93cx6.ko 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/eeprom/max6875.ko 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/enclosure.ko 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/genwqe/ 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 152201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/genwqe/genwqe_card.ko 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/hmc6352.ko 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/hpilo.ko 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ibmasm/ 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ibmasm/ibmasm.ko 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ics932s401.ko 2023-12-20T04:12:44.717 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/isl29003.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/isl29020.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/lis3lv02d/ 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/lis3lv02d/lis3lv02d.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/lis3lv02d/lis3lv02d_i2c.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/mei/ 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/mei/mei-me.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/mei/mei-txe.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 310113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/mei/mei.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/phantom.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ti-st/ 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/ti-st/st_drv.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/tifm_7xx1.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/tifm_core.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/tsl2550.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/vmw_balloon.ko 2023-12-20T04:12:44.718 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/vmw_vmci/ 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 183609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/misc/vmw_vmci/vmw_vmci.ko 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/bonding/ 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 490129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/bonding/bonding.ko 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/ 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/b53/ 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/b53/b53_common.ko 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/bcm-sf2.ko 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/mv88e6060.ko 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/mv88e6xxx/ 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 453737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dsa/mv88e6xxx/mv88e6xxx.ko 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/dummy.ko 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/eql.ko 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/ 2023-12-20T04:12:44.719 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/3com/ 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/3com/3c574_cs.ko 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/3com/3c589_cs.ko 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 130553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/3com/3c59x.ko 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/3com/typhoon.ko 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/8390/ 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/8390/8390.ko 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/8390/axnet_cs.ko 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/8390/ne2k-pci.ko 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/8390/pcnet_cs.ko 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/adaptec/ 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/adaptec/starfire.ko 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/agere/ 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/agere/et131x.ko 2023-12-20T04:12:44.720 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/alteon/ 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/alteon/acenic.ko 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/altera/ 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/altera/altera_tse.ko 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/amd/ 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/amd/amd8111e.ko 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/amd/nmclan_cs.ko 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/amd/pcnet32.ko 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/ 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/alx/ 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/alx/alx.ko 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atl1c/ 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 104673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atl1c/atl1c.ko 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atl1e/ 2023-12-20T04:12:44.721 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atl1e/atl1e.ko 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atlx/ 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atlx/atl1.ko 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/atheros/atlx/atl2.ko 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/ 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/b44.ko 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 176761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/bnx2.ko 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/bnx2x/ 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1421497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/bnx2x/bnx2x.ko 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 128569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/cnic.ko 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/genet/ 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 132089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/genet/genet.ko 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 312081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/broadcom/tg3.ko 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/brocade/ 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/brocade/bna/ 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 323513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/brocade/bna/bna.ko 2023-12-20T04:12:44.722 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/ 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb/ 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 193497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb/cxgb.ko 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb3/ 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 347417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb3/cxgb3.ko 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb4/ 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1044497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb4/cxgb4.ko 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb4vf/ 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 166193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/cxgb4vf/cxgb4vf.ko 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/libcxgb/ 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/chelsio/libcxgb/libcxgb.ko 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/cisco/ 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/cisco/enic/ 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 204345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/cisco/enic/enic.ko 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/ 2023-12-20T04:12:44.723 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/ 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/de2104x.ko 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/dmfe.ko 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 141225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/tulip.ko 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/uli526x.ko 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/winbond-840.ko 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dec/tulip/xircom_cb.ko 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dlink/ 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dlink/dl2k.ko 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dlink/sundance.ko 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/dnet.ko 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/ec_bhf.ko 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/emulex/ 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/emulex/benet/ 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 281137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/emulex/benet/be2net.ko 2023-12-20T04:12:44.724 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/ethoc.ko 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/fealnx.ko 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/fujitsu/ 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/fujitsu/fmvj18x_cs.ko 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/ 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/e100.ko 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/e1000/ 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 295577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/e1000/e1000.ko 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/e1000e/ 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 584497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/e1000e/e1000e.ko 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/fm10k/ 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 246209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/fm10k/fm10k.ko 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/i40e/ 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 925825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/i40e/i40e.ko 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/iavf/ 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 302937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/iavf/iavf.ko 2023-12-20T04:12:44.725 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/igb/ 2023-12-20T04:12:44.726 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 514641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/igb/igb.ko 2023-12-20T04:12:44.726 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/igbvf/ 2023-12-20T04:12:44.726 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/igbvf/igbvf.ko 2023-12-20T04:12:44.726 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/ixgbe/ 2023-12-20T04:12:44.726 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 734305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/ixgbe/ixgbe.ko 2023-12-20T04:12:44.726 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/ixgbevf/ 2023-12-20T04:12:44.726 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 151769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/intel/ixgbevf/ixgbevf.ko 2023-12-20T04:12:44.726 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 86921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/jme.ko 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/marvell/ 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/marvell/mvmdio.ko 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/marvell/skge.ko 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 122649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/marvell/sky2.ko 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/ 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx4/ 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 727601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_core.ko 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 278937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx4/mlx4_en.ko 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx5/ 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx5/core/ 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1113681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.ko 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/micrel/ 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/micrel/ks8842.ko 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/micrel/ks8851_common.ko 2023-12-20T04:12:44.727 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/micrel/ks8851_par.ko 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 95665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/micrel/ksz884x.ko 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/myricom/ 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/myricom/myri10ge/ 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/myricom/myri10ge/myri10ge.ko 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/natsemi/ 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/natsemi/natsemi.ko 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/natsemi/ns83820.ko 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/neterion/ 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 150905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/neterion/s2io.ko 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/nvidia/ 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 136721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/nvidia/forcedeth.ko 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/packetengines/ 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/packetengines/hamachi.ko 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/packetengines/yellowfin.ko 2023-12-20T04:12:44.728 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/ 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/netxen/ 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 227897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/netxen/netxen_nic.ko 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/qla3xxx.ko 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/qlcnic/ 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 672049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/qlogic/qlcnic/qlcnic.ko 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/rdc/ 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/rdc/r6040.ko 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/realtek/ 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/realtek/8139cp.ko 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/realtek/8139too.ko 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/realtek/atp.ko 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 189801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/realtek/r8169.ko 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/samsung/ 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/samsung/sxgbe/ 2023-12-20T04:12:44.729 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 104241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/samsung/sxgbe/samsung-sxgbe.ko 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sfc/ 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1052497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sfc/sfc.ko 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/silan/ 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/silan/sc92031.ko 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sis/ 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sis/sis190.ko 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sis/sis900.ko 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/smsc/ 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/smsc/epic100.ko 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/smsc/smc91c92_cs.ko 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/smsc/smsc911x.ko 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/smsc/smsc9420.ko 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/ 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/stmmac/ 2023-12-20T04:12:44.730 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-generic.ko 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/stmmac/dwmac-intel.ko 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac-platform.ko 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 487025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/stmicro/stmmac/stmmac.ko 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sun/ 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sun/cassini.ko 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 166097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sun/niu.ko 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sun/sungem.ko 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/sun/sunhme.ko 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/tehuti/ 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/tehuti/tehuti.ko 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/ti/ 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 86529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/ti/tlan.ko 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/via/ 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/via/via-rhine.ko 2023-12-20T04:12:44.731 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/via/via-velocity.ko 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/wiznet/ 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/wiznet/w5100.ko 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/wiznet/w5300.ko 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/xircom/ 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ethernet/xircom/xirc2ps_cs.ko 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/fddi/ 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/fddi/defxx.ko 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/fddi/skfp/ 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 216985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/fddi/skfp/skfp.ko 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/geneve.ko 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/hyperv/ 2023-12-20T04:12:44.732 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 235521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/hyperv/hv_netvsc.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ieee802154/ 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ieee802154/fakelb.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ifb.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ipvlan/ 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 83769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ipvlan/ipvlan.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/macvlan.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/macvtap.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio/ 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio/mdio-bcm-unimac.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio/mdio-bitbang.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio/mdio-gpio.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio/mdio-regmap.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mdio.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/mii.ko 2023-12-20T04:12:44.733 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/netconsole.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/nlmon.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/pcs/ 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/pcs/pcs-lynx.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/pcs/pcs_xpcs.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/ 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/amd.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/bcm-phy-lib.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/bcm7xxx.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/bcm87xx.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/broadcom.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/cicada.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/davicom.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/et1011c.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/icplus.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/lxt.ko 2023-12-20T04:12:44.734 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/marvell.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 76113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/micrel.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/national.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/phylink.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/qsemi.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/realtek.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/smsc.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/ste10Xp.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/phy/vitesse.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/plip/ 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/plip/plip.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/ 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/bsd_comp.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/ppp_async.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/ppp_deflate.ko 2023-12-20T04:12:44.735 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/ppp_mppe.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/ppp_synctty.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/pppoe.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/pppox.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/ppp/pptp.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/sb1000.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/slip/ 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/slip/slip.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/sungem_phy.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/tap.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/ 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team_mode_activebackup.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team_mode_broadcast.ko 2023-12-20T04:12:44.736 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team_mode_loadbalance.ko 2023-12-20T04:12:44.737 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team_mode_random.ko 2023-12-20T04:12:44.737 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/team/team_mode_roundrobin.ko 2023-12-20T04:12:44.737 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/veth.ko 2023-12-20T04:12:44.737 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/vmxnet3/ 2023-12-20T04:12:44.737 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 144297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/vmxnet3/vmxnet3.ko 2023-12-20T04:12:44.737 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/vxlan/ 2023-12-20T04:12:44.737 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 297329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/vxlan/vxlan.ko 2023-12-20T04:12:44.737 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/xen-netback/ 2023-12-20T04:12:44.737 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 144585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/net/xen-netback/xen-netback.ko 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ntb/ 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ntb/ntb.ko 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/nvme/ 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/nvme/host/ 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 265121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/nvme/host/nvme-core.ko 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/nvme/host/nvme.ko 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/parport/ 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 133841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/parport/parport.ko 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/parport/parport_cs.ko 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/parport/parport_pc.ko 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/parport/parport_serial.ko 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/ 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/hotplug/ 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/hotplug/acpiphp_ibm.ko 2023-12-20T04:12:44.738 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/hotplug/cpcihp_generic.ko 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/hotplug/cpcihp_zt5550.ko 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/pci-stub.ko 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pci/xen-pcifront.ko 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/ 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/i82092.ko 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 145809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/pcmcia.ko 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/pcmcia_core.ko 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/pcmcia_rsrc.ko 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/pd6729.ko 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 112889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pcmcia/yenta_socket.ko 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/phy/ 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/phy/broadcom/ 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/phy/broadcom/phy-bcm-kona-usb2.ko 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/phy/samsung/ 2023-12-20T04:12:44.739 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/phy/samsung/phy-exynos-usb2.ko 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pinctrl/ 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pinctrl/intel/ 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pinctrl/intel/pinctrl-cherryview.ko 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/ 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/ 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros-ec-sensorhub.ko 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_ec.ko 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_ec_chardev.ko 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_ec_debugfs.ko 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_ec_lightbar.ko 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_ec_sysfs.ko 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/chrome/cros_usbpd_notify.ko 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/ 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/acer-wmi.ko 2023-12-20T04:12:44.740 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/acerhdf.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/apple-gmux.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/asus-laptop.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/asus-nb-wmi.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 127441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/asus-wmi.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/classmate-laptop.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/eeepc-laptop.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/eeepc-wmi.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/fujitsu-laptop.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/fujitsu-tablet.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/hdaps.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/ibm_rtl.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/intel/ 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/intel/intel-rst.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/intel/intel-smartconnect.ko 2023-12-20T04:12:44.741 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/intel_ips.ko 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/msi-wmi.ko 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/mxm-wmi.ko 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/panasonic-laptop.ko 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/samsung-laptop.ko 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/samsung-q10.ko 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/topstar-laptop.ko 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/toshiba_bluetooth.ko 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/toshiba_haps.ko 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/wmi-bmof.ko 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/platform/x86/wmi.ko 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/ 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/ 2023-12-20T04:12:44.742 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/88pm860x_battery.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/88pm860x_charger.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/bq2415x_charger.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/bq24190_charger.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/bq24735-charger.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/cros_peripheral_charger.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/da9030_battery.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/da9052-battery.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/ds2760_battery.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/ds2780_battery.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/ds2781_battery.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/ds2782_battery.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/gpio-charger.ko 2023-12-20T04:12:44.743 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/isp1704_charger.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/lp8727_charger.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/max14577_charger.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/max17040_battery.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/max17042_battery.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/max8903_charger.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/max8925_power.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/pcf50633-charger.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/sbs-battery.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/test_power.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/tps65090-charger.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/wm831x_backup.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/wm831x_power.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/power/supply/wm8350_power.ko 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/powercap/ 2023-12-20T04:12:44.744 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/powercap/intel_rapl_common.ko 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/powercap/intel_rapl_msr.ko 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/ 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/clients/ 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/clients/pps-gpio.ko 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/clients/pps-ldisc.ko 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/clients/pps_parport.ko 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/pps/pps_core.ko 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ptp/ 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ptp/ptp.ko 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ptp/ptp_kvm.ko 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/ 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-88pm80x.ko 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-88pm860x.ko 2023-12-20T04:12:44.745 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-bq32k.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-da9052.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-da9055.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-da9063.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1286.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1307.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1374.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1511.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1553.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1672.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds1742.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds2404.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-ds3232.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-em3027.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-fm3130.ko 2023-12-20T04:12:44.746 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-isl12022.ko 2023-12-20T04:12:44.747 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-isl1208.ko 2023-12-20T04:12:44.747 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-lp8788.ko 2023-12-20T04:12:44.747 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-m41t80.ko 2023-12-20T04:12:44.747 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-m48t35.ko 2023-12-20T04:12:44.747 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-m48t59.ko 2023-12-20T04:12:44.747 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-m48t86.ko 2023-12-20T04:12:44.747 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-max6900.ko 2023-12-20T04:12:44.747 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-max8907.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-max8925.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-max8997.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-max8998.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-mc13xxx.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-msm6242.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-palmas.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf2127.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf50633.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf85063.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf8523.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf8563.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-pcf8583.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rc5t583.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rp5c01.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rs5c372.ko 2023-12-20T04:12:44.748 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rv3029c2.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rx8025.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-rx8581.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-s35390a.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-stk17ta8.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-tps6586x.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-tps65910.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-wm831x.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-wm8350.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/rtc/rtc-x1205.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/3w-9xxx.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/3w-sas.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/3w-xxxx.ko 2023-12-20T04:12:44.749 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 135185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/BusLogic.ko 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/a100u2w.ko 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aacraid/ 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 272305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aacraid/aacraid.ko 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/advansys.ko 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aic7xxx/ 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 356201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aic7xxx/aic79xx.ko 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 295801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aic7xxx/aic7xxx.ko 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aic94xx/ 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 189185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/aic94xx/aic94xx.ko 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/am53c974.ko 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/arcmsr/ 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 106881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/arcmsr/arcmsr.ko 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/atp870u.ko 2023-12-20T04:12:44.750 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/be2iscsi/ 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 246985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/be2iscsi/be2iscsi.ko 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bfa/ 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1171945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bfa/bfa.ko 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bnx2fc/ 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 237273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bnx2fc/bnx2fc.ko 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bnx2i/ 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/bnx2i/bnx2i.ko 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ch.ko 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/csiostor/ 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 308921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/csiostor/csiostor.ko 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/ 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/cxgb3i/ 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 76145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/cxgb3i/cxgb3i.ko 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/cxgb4i/ 2023-12-20T04:12:44.751 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/cxgb4i/cxgb4i.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 146689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/cxgbi/libcxgbi.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/dc395x.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/dmx3191d.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/esas2r/ 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 212425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/esas2r/esas2r.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/esp_scsi.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fcoe/ 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fcoe/fcoe.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 158505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fcoe/libfcoe.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fdomain.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fnic/ 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 263489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/fnic/fnic.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 201817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/hpsa.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/hptiop.ko 2023-12-20T04:12:44.752 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/hv_storvsc.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/imm.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/initio.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 266537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ipr.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ips.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/isci/ 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 317153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/isci/isci.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/iscsi_boot_sysfs.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/iscsi_tcp.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libfc/ 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 314865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libfc/libfc.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 145585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libiscsi.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libiscsi_tcp.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libsas/ 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 216665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/libsas/libsas.ko 2023-12-20T04:12:44.753 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/lpfc/ 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2197249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/lpfc/lpfc.ko 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/megaraid/ 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/megaraid/megaraid_mbox.ko 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/megaraid/megaraid_mm.ko 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 310617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/megaraid/megaraid_sas.ko 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 84721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/megaraid.ko 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/mpt3sas/ 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 659825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/mpt3sas/mpt3sas.ko 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/mvsas/ 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/mvsas/mvsas.ko 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/mvumi.ko 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pcmcia/ 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pcmcia/aha152x_cs.ko 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pcmcia/fdomain_cs.ko 2023-12-20T04:12:44.754 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pcmcia/qlogic_cs.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pcmcia/sym53c500_cs.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pm8001/ 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 364105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pm8001/pm80xx.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 123617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/pmcraid.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ppa.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla1280.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla2xxx/ 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1876689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla2xxx/qla2xxx.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla2xxx/tcm_qla2xxx.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla4xxx/ 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 559033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qla4xxx/qla4xxx.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/qlogicfas408.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/raid_class.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 238761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_debug.ko 2023-12-20T04:12:44.755 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 173937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_transport_fc.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 279313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_transport_iscsi.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 97729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_transport_sas.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_transport_spi.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/scsi_transport_srp.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/ses.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 115889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/st.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/stex.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/sym53c8xx_2/ 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 162121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/sym53c8xx_2/sym53c8xx.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/virtio_scsi.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/vmw_pvscsi.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/wd719x.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/scsi/xen-scsifront.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ssb/ 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 129441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ssb/ssb.ko 2023-12-20T04:12:44.756 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/ 2023-12-20T04:12:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/iscsi/ 2023-12-20T04:12:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 736601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/iscsi/iscsi_target_mod.ko 2023-12-20T04:12:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/loopback/ 2023-12-20T04:12:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/loopback/tcm_loop.ko 2023-12-20T04:12:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/target_core_file.ko 2023-12-20T04:12:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/target_core_iblock.ko 2023-12-20T04:12:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 930257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/target_core_mod.ko 2023-12-20T04:12:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/target_core_pscsi.ko 2023-12-20T04:12:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 122521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/target_core_user.ko 2023-12-20T04:12:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/tcm_fc/ 2023-12-20T04:12:44.757 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/target/tcm_fc/tcm_fc.ko 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/ 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/ 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/ 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/acpi_thermal_rel.ko 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/int3400_thermal.ko 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/int3401_thermal.ko 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/int3402_thermal.ko 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/int3403_thermal.ko 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/int340x_thermal_zone.ko 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device.ko 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci.ko 2023-12-20T04:12:44.758 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_device_pci_legacy.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_mbox.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rapl.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/int340x_thermal/processor_thermal_rfim.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/intel_powerclamp.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/intel_soc_dts_iosf.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/intel_soc_dts_thermal.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/thermal/intel/x86_pkg_temp_thermal.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/ 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/ipwireless/ 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/ipwireless/ipwireless.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/moxa.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 50953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/mxser.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/n_gsm.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/n_hdlc.ko 2023-12-20T04:12:44.759 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/nozomi.ko 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/ 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/8250/ 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/8250/8250_dw.ko 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/8250/serial_cs.ko 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/altera_jtaguart.ko 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/altera_uart.ko 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/arc_uart.ko 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/fsl_lpuart.ko 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/jsm/ 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/jsm/jsm.ko 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/rp2.ko 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/tty/serial/sc16is7xx.ko 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/ 2023-12-20T04:12:44.760 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/core/ 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 531265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/core/ufshcd-core.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/host/ 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/host/ufshcd-pci.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/ufs/host/ufshcd-pltfrm.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/ 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_aec.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_cif.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_dmem_genirq.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_mf624.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_netx.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_pci_generic.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_pdrv_genirq.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/uio/uio_sercos3.ko 2023-12-20T04:12:44.761 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/ 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/c67x00/ 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/c67x00/c67x00.ko 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/chipidea/ 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 144377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/chipidea/ci_hdrc.ko 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/chipidea/ci_hdrc_msm.ko 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/chipidea/ci_hdrc_pci.ko 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/chipidea/ci_hdrc_usb2.ko 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/class/ 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/class/cdc-acm.ko 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/class/cdc-wdm.ko 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/class/usblp.ko 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/class/usbtmc.ko 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/common/ 2023-12-20T04:12:44.762 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/common/ulpi.ko 2023-12-20T04:12:44.763 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc2/ 2023-12-20T04:12:44.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc2/dwc2_pci.ko 2023-12-20T04:12:44.763 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc3/ 2023-12-20T04:12:44.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc3/dwc3-haps.ko 2023-12-20T04:12:44.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc3/dwc3-pci.ko 2023-12-20T04:12:44.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 448505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/dwc3/dwc3.ko 2023-12-20T04:12:44.763 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/ 2023-12-20T04:12:44.763 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/ 2023-12-20T04:12:44.763 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/u_ether.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/u_serial.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_acm.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_ecm.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_ecm_subset.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_eem.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 92489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_fs.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_hid.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 122081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_mass_storage.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_ncm.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_obex.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_phonet.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_printer.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_rndis.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_serial.ko 2023-12-20T04:12:44.764 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_ss_lb.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/function/usb_f_tcm.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/ 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_acm_ms.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_cdc.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_dbgp.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_ether.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_ffs.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_hid.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_mass_storage.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_ncm.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_nokia.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_printer.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_serial.ko 2023-12-20T04:12:44.765 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/g_zero.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 62185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/gadgetfs.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/legacy/tcm_usb_gadget.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 161225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/libcomposite.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/ 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/amd5536udc_pci.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/bdc/ 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 157681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/bdc/bdc.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/goku_udc.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/gr_udc.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 69705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/mv_u3d_core.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/mv_udc.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 76057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/net2272.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/net2280.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 73169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/pch_udc.ko 2023-12-20T04:12:44.766 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 65809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/pxa27x_udc.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/r8a66597-udc.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/snps_udc_core.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 156073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/gadget/udc/udc-core.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/ 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/bcma-hcd.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/isp116x-hcd.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 87353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/oxu210hp-hcd.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/r8a66597-hcd.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/sl811-hcd.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/sl811_cs.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/ssb-hcd.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/host/xhci-plat-hcd.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/image/ 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/image/mdc800.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/image/microtek.ko 2023-12-20T04:12:44.767 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/ 2023-12-20T04:12:44.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/adutux.ko 2023-12-20T04:12:44.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/appledisplay.ko 2023-12-20T04:12:44.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/cypress_cy7c63.ko 2023-12-20T04:12:44.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/cytherm.ko 2023-12-20T04:12:44.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/ehset.ko 2023-12-20T04:12:44.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/emi26.ko 2023-12-20T04:12:44.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/emi62.ko 2023-12-20T04:12:44.768 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/ezusb.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/idmouse.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/iowarrior.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/isight_firmware.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/ldusb.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/legousbtower.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/lvstest.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/sisusbvga/ 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/sisusbvga/sisusbvga.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/trancevibrator.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/usb3503.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/usblcd.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/usbsevseg.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/usbtest.ko 2023-12-20T04:12:44.769 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/uss720.ko 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/misc/yurex.ko 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/mon/ 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/mon/usbmon.ko 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/musb/ 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 301697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/musb/musb_hdrc.ko 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/phy/ 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/phy/phy-generic.ko 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/phy/phy-gpio-vbus-usb.ko 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/phy/phy-isp1301.ko 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/phy/phy-tahvo.ko 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/aircable.ko 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ark3116.ko 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/belkin_sa.ko 2023-12-20T04:12:44.770 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ch341.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/cp210x.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/cyberjack.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/cypress_m8.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/digi_acceleport.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/empeg.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/f81232.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 175137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ftdi_sio.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/garmin_gps.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 137081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/io_edgeport.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/io_ti.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ipaq.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ipw.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ir-usb.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/iuu_phoenix.ko 2023-12-20T04:12:44.771 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 107521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/keyspan.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/keyspan_pda.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/kl5kusb105.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/kobil_sct.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/mct_u232.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/metro-usb.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/mos7720.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 78217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/mos7840.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/mxuport.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/navman.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/omninet.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/opticon.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 191137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/option.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/oti6858.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/pl2303.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/qcaux.ko 2023-12-20T04:12:44.772 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/qcserial.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/quatech2.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/safe_serial.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 51625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/sierra.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/spcp8x5.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ssu100.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/symbolserial.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/ti_usb_3410_5052.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/usb-serial-simple.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/usb_debug.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/usb_wwan.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/usbserial.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/visor.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/whiteheat.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/wishbone-serial.ko 2023-12-20T04:12:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/serial/xsens_mt.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 52913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/uas.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-alauda.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-cypress.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-datafab.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-eneub6250.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-freecom.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-isd200.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-jumpshot.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-karma.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-onetouch.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-realtek.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-sddr09.ko 2023-12-20T04:12:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-sddr55.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/ums-usbat.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 179969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/storage/usb-storage.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/usbip/ 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/usbip/usbip-core.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/usbip/usbip-host.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 107001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/usb/usbip/vhci-hcd.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/ 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/pci/ 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 154297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/pci/vfio-pci-core.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/pci/vfio-pci.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/vfio.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 77609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vfio/vfio_iommu_type1.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vhost/ 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vhost/vhost.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vhost/vhost_iotlb.ko 2023-12-20T04:12:44.775 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 58089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vhost/vhost_net.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/vhost/vhost_scsi.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/ 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/ 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/88pm860x_bl.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/aat2870_bl.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/adp5520_bl.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/adp8860_bl.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/adp8870_bl.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/apple_bl.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/as3711_bl.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/bd6107.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/da903x_bl.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/da9052_bl.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/gpio_backlight.ko 2023-12-20T04:12:44.776 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/kb3886_bl.ko 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/lcd.ko 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/lm3533_bl.ko 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/lm3639_bl.ko 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/lv5207lp.ko 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/max8925_bl.ko 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/pandora_bl.ko 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/pcf50633-backlight.ko 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/platform_lcd.ko 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/video/backlight/wm831x_bl.ko 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/ 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/masters/ 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/masters/ds2482.ko 2023-12-20T04:12:44.777 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/masters/ds2490.ko 2023-12-20T04:12:44.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/masters/matrox_w1.ko 2023-12-20T04:12:44.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/masters/w1-gpio.ko 2023-12-20T04:12:44.778 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/ 2023-12-20T04:12:44.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2406.ko 2023-12-20T04:12:44.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2408.ko 2023-12-20T04:12:44.778 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2413.ko 2023-12-20T04:12:44.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2423.ko 2023-12-20T04:12:44.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14153 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2431.ko 2023-12-20T04:12:44.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2433.ko 2023-12-20T04:12:44.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2780.ko 2023-12-20T04:12:44.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds2781.ko 2023-12-20T04:12:44.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_ds28e04.ko 2023-12-20T04:12:44.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_smem.ko 2023-12-20T04:12:44.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 56425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/slaves/w1_therm.ko 2023-12-20T04:12:44.779 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/w1/wire.ko 2023-12-20T04:12:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/ 2023-12-20T04:12:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/acquirewdt.ko 2023-12-20T04:12:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/advantechwdt.ko 2023-12-20T04:12:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/alim1535_wdt.ko 2023-12-20T04:12:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/alim7101_wdt.ko 2023-12-20T04:12:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/cpu5wdt.ko 2023-12-20T04:12:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/da9052_wdt.ko 2023-12-20T04:12:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/da9055_wdt.ko 2023-12-20T04:12:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/da9063_wdt.ko 2023-12-20T04:12:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/dw_wdt.ko 2023-12-20T04:12:44.780 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/eurotechwdt.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/f71808e_wdt.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/hpwdt.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/i6300esb.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/iTCO_vendor_support.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/iTCO_wdt.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/ib700wdt.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/ibmasr.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/ie6xx_wdt.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/it8712f_wdt.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/it87_wdt.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/kempld_wdt.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/machzwd.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/mena21_wdt.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/menf21bmc_wdt.ko 2023-12-20T04:12:44.781 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/nv_tco.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/of_xilinx_wdt.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22265 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/pc87413_wdt.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/pcwd_pci.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/pcwd_usb.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/retu_wdt.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sbc60xxwdt.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sbc_epx_c3.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sbc_fitpc2_wdt.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sc1200wdt.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sch311x_wdt.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/smsc37b787_wdt.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/softdog.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/sp5100_tco.ko 2023-12-20T04:12:44.782 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/twl4030_wdt.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/via_wdt.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/w83627hf_wdt.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/w83877f_wdt.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/w83977f_wdt.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/wafer5823wdt.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/wdt_pci.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/wm831x_wdt.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/wm8350_wdt.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/watchdog/xen_wdt.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/ 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-evtchn.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-gntalloc.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-gntdev.ko 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-pciback/ 2023-12-20T04:12:44.783 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 176377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-pciback/xen-pciback.ko 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-privcmd.ko 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 71249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xen-scsiback.ko 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xenfs/ 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/drivers/xen/xenfs/xenfs.ko 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/9p/ 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 166425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/9p/9p.ko 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/adfs/ 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/adfs/adfs.ko 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/affs/ 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 163593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/affs/affs.ko 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/afs/ 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 932817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/afs/kafs.ko 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/autofs/ 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/autofs/autofs4.ko 2023-12-20T04:12:44.784 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/befs/ 2023-12-20T04:12:44.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 64545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/befs/befs.ko 2023-12-20T04:12:44.785 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/bfs/ 2023-12-20T04:12:44.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/bfs/bfs.ko 2023-12-20T04:12:44.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/binfmt_misc.ko 2023-12-20T04:12:44.785 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/btrfs/ 2023-12-20T04:12:44.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 3402729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/btrfs/btrfs.ko 2023-12-20T04:12:44.785 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/cachefiles/ 2023-12-20T04:12:44.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 306009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/cachefiles/cachefiles.ko 2023-12-20T04:12:44.785 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ceph/ 2023-12-20T04:12:44.785 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1333017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ceph/ceph.ko 2023-12-20T04:12:44.829 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-20T04:12:44.831 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/coda/ 2023-12-20T04:12:44.831 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/coda/coda.ko 2023-12-20T04:12:44.831 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/configfs/ 2023-12-20T04:12:44.831 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 108393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/configfs/configfs.ko 2023-12-20T04:12:44.831 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/cramfs/ 2023-12-20T04:12:44.831 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 29625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/cramfs/cramfs.ko 2023-12-20T04:12:44.831 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/dlm/ 2023-12-20T04:12:44.831 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 642513 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/dlm/dlm.ko 2023-12-20T04:12:44.831 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/efs/ 2023-12-20T04:12:44.831 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/efs/efs.ko 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/f2fs/ 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1505025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/f2fs/f2fs.ko 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fat/ 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fat/msdos.ko 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/freevxfs/ 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/freevxfs/freevxfs.ko 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fscache/ 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 226481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fscache/fscache.ko 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fuse/ 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/fuse/cuse.ko 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/gfs2/ 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1231529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/gfs2/gfs2.ko 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hfs/ 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 141209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hfs/hfs.ko 2023-12-20T04:12:44.832 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hfsplus/ 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 222817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hfsplus/hfsplus.ko 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hpfs/ 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 192329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/hpfs/hpfs.ko 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/isofs/ 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/isofs/isofs.ko 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/jfs/ 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 419497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/jfs/jfs.ko 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/lockd/ 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 280313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/lockd/lockd.ko 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/minix/ 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/minix/minix.ko 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/netfs/ 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 124649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/netfs/netfs.ko 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/ 2023-12-20T04:12:44.833 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/blocklayout/ 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/blocklayout/blocklayoutdriver.ko 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/filelayout/ 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 59449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/filelayout/nfs_layout_nfsv41_files.ko 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/flexfilelayout/ 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 117465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/flexfilelayout/nfs_layout_flexfiles.ko 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1055721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/nfs.ko 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/nfsv2.ko 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 120177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/nfsv3.ko 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2042137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs/nfsv4.ko 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs_common/ 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs_common/grace.ko 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfs_common/nfs_acl.ko 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfsd/ 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1516873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nfsd/nfsd.ko 2023-12-20T04:12:44.834 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nilfs2/ 2023-12-20T04:12:44.835 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 592833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nilfs2/nilfs2.ko 2023-12-20T04:12:44.836 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/ 2023-12-20T04:12:44.836 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-celtic.ko 2023-12-20T04:12:44.836 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-centeuro.ko 2023-12-20T04:12:44.836 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-croatian.ko 2023-12-20T04:12:44.836 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-cyrillic.ko 2023-12-20T04:12:44.836 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-gaelic.ko 2023-12-20T04:12:44.836 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-greek.ko 2023-12-20T04:12:44.836 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-iceland.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-inuit.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-roman.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-romanian.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/mac-turkish.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_ascii.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp1250.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp1251.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp1255.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp737.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp775.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp850.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp852.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp855.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp857.ko 2023-12-20T04:12:44.837 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp860.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp861.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp862.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp863.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp864.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp865.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp866.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp869.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp874.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 94137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp932.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 142673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp936.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 165745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp949.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 118121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_cp950.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_euc-jp.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-1.ko 2023-12-20T04:12:44.838 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-13.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-14.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-15.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-2.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-3.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-4.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-5.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-6.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-7.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_iso8859-9.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_koi8-r.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_koi8-ru.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_koi8-u.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_ucs2_utils.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/nls/nls_utf8.ko 2023-12-20T04:12:44.839 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ntfs/ 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 219417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ntfs/ntfs.ko 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/ 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/cluster/ 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 244409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/cluster/ocfs2_nodemanager.ko 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/dlm/ 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 440737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/dlm/ocfs2_dlm.ko 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/dlmfs/ 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/dlmfs/ocfs2_dlmfs.ko 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2592433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/ocfs2.ko 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/ocfs2_stack_o2cb.ko 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/ocfs2_stack_user.ko 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ocfs2/ocfs2_stackglue.ko 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/omfs/ 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/omfs/omfs.ko 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/overlayfs/ 2023-12-20T04:12:44.840 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 352161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/overlayfs/overlay.ko 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/pstore/ 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/pstore/ramoops.ko 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/qnx4/ 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/qnx4/qnx4.ko 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/qnx6/ 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 46689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/qnx6/qnx6.ko 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/quota/ 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/quota/quota_tree.ko 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/quota/quota_v1.ko 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/quota/quota_v2.ko 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/reiserfs/ 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 532673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/reiserfs/reiserfs.ko 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/romfs/ 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/romfs/romfs.ko 2023-12-20T04:12:44.841 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/ 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/client/ 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 2668401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/client/cifs.ko 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/common/ 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/common/cifs_arc4.ko 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/smb/common/cifs_md4.ko 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/squashfs/ 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 134641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/squashfs/squashfs.ko 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/sysv/ 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 90721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/sysv/sysv.ko 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/udf/ 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 247737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/udf/udf.ko 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ufs/ 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 162497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/ufs/ufs.ko 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/xfs/ 2023-12-20T04:12:44.842 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 4011017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/fs/xfs/xfs.ko 2023-12-20T04:12:44.843 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/ 2023-12-20T04:12:44.843 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/locking/ 2023-12-20T04:12:44.843 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 75489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/locking/locktorture.ko 2023-12-20T04:12:44.843 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/time/ 2023-12-20T04:12:44.843 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/time/test_udelay.ko 2023-12-20T04:12:44.843 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/kernel/torture.ko 2023-12-20T04:12:44.843 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/ 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crc-itu-t.ko 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crc7.ko 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crc8.ko 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crypto/ 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crypto/libarc4.ko 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/crypto/libdes.ko 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/interval_tree_test.ko 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/libcrc32c.ko 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/lru_cache.ko 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/lz4/ 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/lz4/lz4_compress.ko 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/lz4/lz4hc_compress.ko 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/math/ 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/math/cordic.ko 2023-12-20T04:12:44.844 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/memory-notifier-error-inject.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/notifier-error-inject.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/percpu_test.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/pm-notifier-error-inject.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/raid6/ 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 143433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/raid6/raid6_pq.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/rbtree_test.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/reed_solomon/ 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/reed_solomon/reed_solomon.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test-kstrtox.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test-string_helpers.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 4760425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test_bpf.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test_firmware.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test_module.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/test_user_copy.ko 2023-12-20T04:12:44.845 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/ts_bm.ko 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/ts_fsm.ko 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/ts_kmp.ko 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/xz/ 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/lib/xz/xz_dec_test.ko 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/mm/ 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/mm/hwpoison-inject.ko 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/ 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/6lowpan.ko 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_dest.ko 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_fragment.ko 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_hop.ko 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 6985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_ipv6.ko 2023-12-20T04:12:44.846 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_mobility.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_routing.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/6lowpan/nhc_udp.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/ 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/garp.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/mrp.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/p8022.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/psnap.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/802/stp.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/8021q/ 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 81737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/8021q/8021q.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/9p/ 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 198337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/9p/9pnet.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/9p/9pnet_fd.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26345 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/9p/9pnet_rdma.ko 2023-12-20T04:12:44.847 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/9p/9pnet_virtio.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/batman-adv/ 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 649297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/batman-adv/batman-adv.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/ 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 63809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/br_netfilter.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 777689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/bridge.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_802_3.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_among.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_arp.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_arpreply.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_dnat.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_ip.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_ip6.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_limit.ko 2023-12-20T04:12:44.848 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13217 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_log.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_mark.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_mark_m.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8385 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_nflog.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_pkttype.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_redirect.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_snat.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_stp.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebt_vlan.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebtable_broute.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebtable_filter.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebtable_nat.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 55929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/ebtables.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/nft_meta_bridge.ko 2023-12-20T04:12:44.849 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/bridge/netfilter/nft_reject_bridge.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ceph/ 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1046897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ceph/libceph.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/core/ 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/core/pktgen.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dccp/ 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 194209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dccp/dccp.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dccp/dccp_diag.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 57881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dccp/dccp_ipv4.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dccp/dccp_ipv6.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/ 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 390473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/dsa_core.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/tag_brcm.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/tag_dsa.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/tag_none.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/dsa/tag_trailer.ko 2023-12-20T04:12:44.850 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/hsr/ 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 99729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/hsr/hsr.ko 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ieee802154/ 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ieee802154/6lowpan/ 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 45233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ieee802154/6lowpan/ieee802154_6lowpan.ko 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 254625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ieee802154/ieee802154.ko 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ieee802154/ieee802154_socket.ko 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ah4.ko 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/esp4.ko 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/fou.ko 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/gre.ko 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/inet_diag.ko 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ip_gre.ko 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61225 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ip_tunnel.ko 2023-12-20T04:12:44.851 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ip_vti.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ipcomp.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/ipip.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/arp_tables.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/arpt_mangle.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/arptable_filter.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ip_tables.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ipt_ECN.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ipt_REJECT.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ipt_SYNPROXY.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ipt_ah.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/ipt_rpfilter.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11305 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/iptable_filter.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/iptable_mangle.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/iptable_nat.ko 2023-12-20T04:12:44.852 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/iptable_raw.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/iptable_security.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_defrag_ipv4.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_dup_ipv4.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_nat_h323.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18161 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_nat_pptp.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_nat_snmp_basic.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_reject_ipv4.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_socket_ipv4.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nf_tproxy_ipv4.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nft_dup_ipv4.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nft_fib_ipv4.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/netfilter/nft_reject_ipv4.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_bic.ko 2023-12-20T04:12:44.853 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_dctcp.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_diag.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_highspeed.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_htcp.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_hybla.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_illinois.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_lp.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_scalable.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_vegas.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_veno.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_westwood.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tcp_yeah.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/tunnel4.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/udp_diag.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 41665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/udp_tunnel.ko 2023-12-20T04:12:44.854 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv4/xfrm4_tunnel.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ah6.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/esp6.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/fou6.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 80897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ip6_gre.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 93577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ip6_tunnel.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ip6_udp_tunnel.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ip6_vti.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/ipcomp6.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19017 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/mip6.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6_tables.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_NPT.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_REJECT.ko 2023-12-20T04:12:44.855 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_SYNPROXY.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_ah.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_eui64.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_frag.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_hbh.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_ipv6header.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_mh.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_rpfilter.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6t_rt.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6table_filter.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11121 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6table_mangle.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6table_nat.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6table_raw.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/ip6table_security.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nf_defrag_ipv6.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nf_dup_ipv6.ko 2023-12-20T04:12:44.856 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25945 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nf_reject_ipv6.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nf_socket_ipv6.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nf_tproxy_ipv6.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nft_dup_ipv6.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nft_fib_ipv6.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9009 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/netfilter/nft_reject_ipv6.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 74073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/sit.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/tunnel6.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/ipv6/xfrm6_tunnel.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/key/ 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 82833 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/key/af_key.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/ 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125281 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_core.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_debugfs.ko 2023-12-20T04:12:44.857 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 25953 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_eth.ko 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_ip.ko 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36713 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_ip6.ko 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_netlink.ko 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 53241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/l2tp/l2tp_ppp.ko 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/lapb/ 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 47977 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/lapb/lapb.ko 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/llc/ 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/llc/llc.ko 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 267913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/llc/llc2.ko 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/mac802154/ 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 290417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/mac802154/mac802154.ko 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/mpls/ 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/mpls/mpls_gso.ko 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ 2023-12-20T04:12:44.858 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 119585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_bitmap_ip.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26913 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_bitmap_ipmac.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_bitmap_port.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 100297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_ip.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 101441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_ipmark.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_ipport.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 103417 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_ipportip.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 112849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_ipportnet.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 60329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_mac.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 109593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_net.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 113129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_netiface.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 112585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_netnet.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 111585 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_netport.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 114465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_hash_netportnet.ko 2023-12-20T04:12:44.859 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipset/ip_set_list_set.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 411729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_dh.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_fo.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_ftp.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_lblc.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_lblcr.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_lc.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7689 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_nq.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10681 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_pe_sip.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_rr.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_sed.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_sh.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7697 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_wlc.ko 2023-12-20T04:12:44.860 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10993 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/ipvs/ip_vs_wrr.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conncount.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 408257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_amanda.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_broadcast.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33761 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_ftp.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 118145 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_h323.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_irc.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_netbios_ns.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 107233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_netlink.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_pptp.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_sane.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66641 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_sip.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_snmp.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_conntrack_tftp.ko 2023-12-20T04:12:44.861 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10113 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_dup_netdev.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_flow_table.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9593 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_flow_table_inet.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 37129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_log_syslog.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 95353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat_amanda.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat_ftp.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11857 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat_irc.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28929 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat_sip.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_nat_tftp.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 44065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_synproxy_core.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 615649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nf_tables.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 42561 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30233 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink_acct.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30569 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink_cttimeout.ko 2023-12-20T04:12:44.862 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 39785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink_log.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink_osf.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nfnetlink_queue.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10457 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_chain_nat.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_compat.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 40721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_ct.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9601 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_dup_netdev.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_fib.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_fwd_netdev.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14185 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_hash.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_limit.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_log.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_masq.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_nat.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13673 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_numgen.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11841 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_osf.ko 2023-12-20T04:12:44.863 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_queue.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_redir.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_reject.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_reject_inet.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_socket.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_synproxy.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/nft_tproxy.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 89313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/x_tables.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_AUDIT.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_CHECKSUM.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_CLASSIFY.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_CONNSECMARK.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 18169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_CT.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_DSCP.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_HL.ko 2023-12-20T04:12:44.864 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13369 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_HMARK.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_IDLETIMER.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13889 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_LED.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11489 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_LOG.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11753 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_MASQUERADE.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_NETMAP.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_NFLOG.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_NFQUEUE.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_RATEEST.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_REDIRECT.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 14337 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_SECMARK.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_TCPMSS.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_TCPOPTSTRIP.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17361 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_TEE.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_TPROXY.ko 2023-12-20T04:12:44.865 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8321 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_TRACE.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_addrtype.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_bpf.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_cgroup.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_cluster.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_comment.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10465 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_connbytes.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_connlabel.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_connlimit.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13129 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_connmark.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_conntrack.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_cpu.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_dccp.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_devgroup.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8801 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_dscp.ko 2023-12-20T04:12:44.866 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_ecn.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_esp.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_hashlimit.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_helper.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7865 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_hl.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11921 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_ipcomp.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_iprange.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11785 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_ipvs.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16041 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_l2tp.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8001 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_length.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10209 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_limit.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7545 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_mac.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_mark.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11825 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_multiport.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_nat.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_nfacct.ko 2023-12-20T04:12:44.867 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_osf.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_owner.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_physdev.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_pkttype.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11985 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_policy.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_quota.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_rateest.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_realm.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_recent.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13329 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_sctp.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_set.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 16089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_socket.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9505 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_state.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9473 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_statistic.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8809 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_string.ko 2023-12-20T04:12:44.868 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 8553 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_tcpmss.ko 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15777 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_tcpudp.ko 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_time.ko 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netfilter/xt_u32.ko 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netlink/ 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13969 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/netlink/netlink_diag.ko 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/nsh/ 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 12425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/nsh/nsh.ko 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/openvswitch/ 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 380033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/openvswitch/openvswitch.ko 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/openvswitch/vport-geneve.ko 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/openvswitch/vport-gre.ko 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 11793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/openvswitch/vport-vxlan.ko 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/packet/ 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13057 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/packet/af_packet_diag.ko 2023-12-20T04:12:44.869 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/phonet/ 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 110201 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/phonet/phonet.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54721 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/phonet/pn_pep.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rds/ 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 251481 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rds/rds.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 164937 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rds/rds_rdma.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 67377 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rds/rds_tcp.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rxrpc/ 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 819657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/rxrpc/rxrpc.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/ 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32241 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_csum.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23577 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_gact.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26873 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_ipt.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_mirred.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_nat.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 43449 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_pedit.ko 2023-12-20T04:12:44.870 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_police.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_simple.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35297 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_skbedit.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/act_vlan.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_basic.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33961 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_bpf.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19169 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_cgroup.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31401 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_flow.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31769 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_fw.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 38425 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_route.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 61409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/cls_u32.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7441 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_cmp.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 13409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_ipset.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33289 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_meta.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_nbyte.ko 2023-12-20T04:12:44.871 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_text.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7249 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/em_u32.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 21649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_choke.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20849 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_codel.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 34633 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_drr.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 33609 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_fq.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31617 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_fq_codel.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 35433 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_gred.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 49817 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_hfsc.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22089 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_hhf.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 66393 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_htb.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26073 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_ingress.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 30273 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_mqprio.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10793 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_mqprio_lib.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26065 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_multiq.ko 2023-12-20T04:12:44.872 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 36137 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_netem.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 27497 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_pie.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10625 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_plug.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 24729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_prio.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 48729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_qfq.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 32905 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_red.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 26881 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_sfb.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 31353 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_sfq.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28897 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_tbf.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 28105 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sched/sch_teql.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sctp/ 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 929745 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sctp/sctp.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 19033 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sctp/sctp_diag.ko 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/ 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/auth_gss/ 2023-12-20T04:12:44.873 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 336529 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/auth_gss/auth_rpcgss.ko 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 54737 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/auth_gss/rpcsec_gss_krb5.ko 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 1520097 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/sunrpc.ko 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/xprtrdma/ 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 707657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/sunrpc/xprtrdma/rpcrdma.ko 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/tipc/ 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10025 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/tipc/diag.ko 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 862665 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/tipc/tipc.ko 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/unix/ 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15729 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/unix/unix_diag.ko 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/ 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 85409 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/vmw_vsock_virtio_transport_common.ko 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 68049 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/vmw_vsock_vmci_transport.ko 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 96193 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/vsock.ko 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 10705 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/vsock_diag.ko 2023-12-20T04:12:44.874 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 17521 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/vmw_vsock/vsock_loopback.ko 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/x25/ 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 125177 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/x25/x25.ko 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/xfrm/ 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 22657 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/xfrm/xfrm_algo.ko 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 20649 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/xfrm/xfrm_ipcomp.ko 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 102257 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/net/xfrm/xfrm_user.ko 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/virt/ 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/virt/lib/ 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 15537 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/kernel/virt/lib/irqbypass.ko 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 640081 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.alias 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 624107 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.alias.bin 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 7663 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.builtin 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 23428 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.builtin.alias.bin 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 9821 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.builtin.bin 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 70004 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.builtin.modinfo 2023-12-20T04:12:44.875 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 146313 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.dep 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 232548 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.dep.bin 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 210 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.devname 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 72606 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.order 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 879 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.softdep 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 254438 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.symbols 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 309363 2023-12-20 01:01 ./lib/modules/6.6.0-g93572f31db89/modules.symbols.bin 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./usr/ 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./usr/share/ 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./usr/share/doc/ 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:drwxr-xr-x root/root 0 2023-12-20 01:01 ./usr/share/doc/linux-image-6.6.0-g93572f31db89/ 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 169 2023-12-20 01:01 ./usr/share/doc/linux-image-6.6.0-g93572f31db89/changelog.Debian.gz 2023-12-20T04:12:44.876 INFO:teuthology.orchestra.run.smithi032.stdout:-rw-r--r-- root/root 691 2023-12-20 01:01 ./usr/share/doc/linux-image-6.6.0-g93572f31db89/copyright 2023-12-20T04:12:44.877 DEBUG:teuthology.task.kernel:get_image_version: 6.6.0-g93572f31db89 2023-12-20T04:12:44.877 INFO:teuthology.task.kernel:searching for kernel 6.6.0-g93572f31db89 2023-12-20T04:12:44.877 DEBUG:teuthology.orchestra.run.smithi032:> egrep '(submenu|menuentry.*6.6.0-g93572f31db89).*{' /boot/grub/grub.cfg 2023-12-20T04:12:44.878 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-20T04:12:44.878 INFO:teuthology.orchestra.run.smithi039.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-20T04:12:44.879 INFO:teuthology.orchestra.run.smithi039.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-20T04:12:44.886 INFO:teuthology.orchestra.run.smithi032.stdout:submenu 'Advanced options for Ubuntu' $menuentry_id_option 'gnulinux-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-20T04:12:44.886 INFO:teuthology.orchestra.run.smithi032.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g93572f31db89' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g93572f31db89-advanced-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-20T04:12:44.886 INFO:teuthology.orchestra.run.smithi032.stdout: menuentry 'Ubuntu, with Linux 6.6.0-g93572f31db89 (recovery mode)' --class ubuntu --class gnu-linux --class gnu --class os $menuentry_id_option 'gnulinux-6.6.0-g93572f31db89-recovery-7e9e2103-8bb7-4452-b35a-3b1230743254' { 2023-12-20T04:12:44.887 INFO:teuthology.task.kernel:submenu_title:Advanced options for Ubuntu> 2023-12-20T04:12:44.887 INFO:teuthology.task.kernel:default_title:Ubuntu, with Linux 6.6.0-g93572f31db89 2023-12-20T04:12:44.887 DEBUG:teuthology.orchestra.run.smithi032:> /bin/echo -e 'cat </dev/null && sudo chmod a+x -- '/etc/grub.d/01_ceph_kernel.tmp~' && sudo mv -- '/etc/grub.d/01_ceph_kernel.tmp~' /etc/grub.d/01_ceph_kernel && sudo update-grub && rm /tmp/linux-image.deb && ( sleep 1 && sudo shutdown -r now & ) 2023-12-20T04:12:44.929 DEBUG:teuthology.task.kernel:Waiting for install on ubuntu@smithi032.front.sepia.ceph.com to complete... 2023-12-20T04:12:45.149 INFO:teuthology.orchestra.run.smithi039.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-20T04:12:45.149 INFO:teuthology.orchestra.run.smithi039.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-12-20T04:12:45.149 INFO:teuthology.orchestra.run.smithi039.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-20T04:12:45.170 INFO:teuthology.orchestra.run.smithi039.stderr:done 2023-12-20T04:12:45.401 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub' 2023-12-20T04:12:45.402 INFO:teuthology.orchestra.run.smithi032.stderr:Sourcing file `/etc/default/grub.d/init-select.cfg' 2023-12-20T04:12:45.418 INFO:teuthology.orchestra.run.smithi032.stderr:Generating grub configuration file ... 2023-12-20T04:12:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-6.6.0-g93572f31db89 2023-12-20T04:12:45.633 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-6.6.0-g93572f31db89 2023-12-20T04:12:45.904 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.15.0-56-generic 2023-12-20T04:12:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.15.0-56-generic 2023-12-20T04:12:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:Found linux image: /boot/vmlinuz-5.4.0-135-generic 2023-12-20T04:12:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:Found initrd image: /boot/initrd.img-5.4.0-135-generic 2023-12-20T04:12:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: os-prober will not be executed to detect other bootable partitions. 2023-12-20T04:12:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:Systems on them will not be added to the GRUB boot configuration. 2023-12-20T04:12:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:Check GRUB_DISABLE_OS_PROBER documentation entry. 2023-12-20T04:12:46.147 INFO:teuthology.orchestra.run.smithi032.stderr:done 2023-12-20T04:13:16.330 INFO:teuthology.task.kernel:Checking client client.1 for new kernel version... 2023-12-20T04:13:16.330 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-12-20T04:13:16.331 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:13:17.267 INFO:teuthology.task.kernel:Checking client client.0 for new kernel version... 2023-12-20T04:13:17.268 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-12-20T04:13:17.268 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:13:34.866 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2023-12-20T04:13:35.634 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-20T04:13:43.874 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-12-20T04:13:43.875 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:13:44.635 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-12-20T04:13:44.636 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:13:46.929 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2023-12-20T04:13:58.942 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-12-20T04:13:58.942 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:14:02.005 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.39 2023-12-20T04:14:03.025 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-20T04:14:15.037 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-12-20T04:14:15.038 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:14:17.009 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-12-20T04:14:17.009 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:14:18.097 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 172.21.15.32 2023-12-20T04:14:33.109 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-12-20T04:14:33.110 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:14:33.317 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-12-20T04:14:34.051 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-12-20T04:14:34.051 INFO:teuthology.task.kernel:Checking kernel version of client.0, want "6.6.0-g93572f31db89"... 2023-12-20T04:14:34.052 DEBUG:teuthology.orchestra.run.smithi032:> uname -r 2023-12-20T04:14:34.101 INFO:teuthology.orchestra.run.smithi032.stdout:6.6.0-g93572f31db89 2023-12-20T04:14:34.101 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g93572f31db89 vs 6.6.0-g93572f31db89 2023-12-20T04:14:34.101 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-20T04:14:34.101 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-12-20T04:14:35.102 DEBUG:teuthology.parallel:result is None 2023-12-20T04:15:17.012 DEBUG:teuthology.orchestra.remote:timed out 2023-12-20T04:15:35.013 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-12-20T04:15:35.014 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:15:35.250 DEBUG:teuthology.orchestra.run.smithi039:> true 2023-12-20T04:15:35.934 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-12-20T04:15:35.934 INFO:teuthology.task.kernel:Checking kernel version of client.1, want "6.6.0-g93572f31db89"... 2023-12-20T04:15:35.934 DEBUG:teuthology.orchestra.run.smithi039:> uname -r 2023-12-20T04:15:35.988 INFO:teuthology.orchestra.run.smithi039.stdout:6.6.0-g93572f31db89 2023-12-20T04:15:35.989 DEBUG:teuthology.task.kernel:current kernel version is 6.6.0-g93572f31db89 vs 6.6.0-g93572f31db89 2023-12-20T04:15:35.989 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2023-12-20T04:15:35.989 DEBUG:teuthology.task.kernel:Distro of this test job: ubuntu 2023-12-20T04:15:36.990 DEBUG:teuthology.parallel:result is None 2023-12-20T04:15:36.990 INFO:teuthology.run_tasks:Running task internal.base... 2023-12-20T04:15:37.000 INFO:teuthology.task.internal:Creating test directory... 2023-12-20T04:15:37.001 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-20T04:15:37.006 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2023-12-20T04:15:37.016 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2023-12-20T04:15:37.027 INFO:teuthology.run_tasks:Running task internal.archive... 2023-12-20T04:15:37.035 INFO:teuthology.task.internal:Creating archive directory... 2023-12-20T04:15:37.035 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-20T04:15:37.063 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2023-12-20T04:15:37.092 INFO:teuthology.run_tasks:Running task internal.coredump... 2023-12-20T04:15:37.099 INFO:teuthology.task.internal:Enabling coredump saving... 2023-12-20T04:15:37.099 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-20T04:15:37.128 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2023-12-20T04:15:37.217 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-20T04:15:37.236 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-20T04:15:37.246 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-20T04:15:37.286 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2023-12-20T04:15:37.287 INFO:teuthology.run_tasks:Running task internal.sudo... 2023-12-20T04:15:37.294 INFO:teuthology.task.internal:Configuring sudo... 2023-12-20T04:15:37.295 DEBUG:teuthology.orchestra.run.smithi032:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-20T04:15:37.305 DEBUG:teuthology.orchestra.run.smithi039:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2023-12-20T04:15:37.330 INFO:teuthology.run_tasks:Running task internal.syslog... 2023-12-20T04:15:37.340 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2023-12-20T04:15:37.340 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-20T04:15:37.371 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2023-12-20T04:15:37.381 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-20T04:15:37.433 DEBUG:teuthology.orchestra.run.smithi032:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-20T04:15:37.485 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:15:37.485 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-20T04:15:37.574 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2023-12-20T04:15:37.584 DEBUG:teuthology.orchestra.run.smithi039:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2023-12-20T04:15:37.636 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:15:37.636 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2023-12-20T04:15:37.710 DEBUG:teuthology.orchestra.run.smithi032:> sudo service rsyslog restart 2023-12-20T04:15:37.714 DEBUG:teuthology.orchestra.run.smithi039:> sudo service rsyslog restart 2023-12-20T04:15:37.838 INFO:teuthology.run_tasks:Running task internal.timer... 2023-12-20T04:15:37.846 INFO:teuthology.task.internal:Starting timer... 2023-12-20T04:15:37.846 INFO:teuthology.run_tasks:Running task pcp... 2023-12-20T04:15:37.867 INFO:teuthology.run_tasks:Running task selinux... 2023-12-20T04:15:37.878 DEBUG:teuthology.task.selinux:Excluding smithi032: OS 'ubuntu' does not support SELinux 2023-12-20T04:15:37.878 DEBUG:teuthology.task.selinux:Excluding smithi039: OS 'ubuntu' does not support SELinux 2023-12-20T04:15:37.879 DEBUG:teuthology.task.selinux:Getting current SELinux state 2023-12-20T04:15:37.879 DEBUG:teuthology.task.selinux:Existing SELinux modes: {} 2023-12-20T04:15:37.879 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2023-12-20T04:15:37.879 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2023-12-20T04:15:37.888 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main was just updated or references a specific commit; assuming it is current 2023-12-20T04:15:37.888 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2023-12-20T04:15:37.912 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2023-12-20T04:15:37.914 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi032.front.sepia.ceph.com,smithi039.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2023-12-20T04:24:41.826 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi032.front.sepia.ceph.com'), Remote(name='ubuntu@smithi039.front.sepia.ceph.com')] 2023-12-20T04:24:41.827 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-12-20T04:24:41.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi032.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:24:41.909 DEBUG:teuthology.orchestra.run.smithi032:> true 2023-12-20T04:24:42.032 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi032.front.sepia.ceph.com' 2023-12-20T04:24:42.033 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-12-20T04:24:42.033 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi039.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2023-12-20T04:24:42.111 DEBUG:teuthology.orchestra.run.smithi039:> true 2023-12-20T04:24:42.248 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi039.front.sepia.ceph.com' 2023-12-20T04:24:42.249 INFO:teuthology.run_tasks:Running task clock... 2023-12-20T04:24:42.260 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2023-12-20T04:24:42.260 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-20T04:24:42.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-20T04:24:42.265 INFO:teuthology.orchestra.run:Running command with timeout 360 2023-12-20T04:24:42.265 DEBUG:teuthology.orchestra.run.smithi039:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-20T04:24:42.303 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-12-20T04:24:42.303 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: Command line: ntpd -gq 2023-12-20T04:24:42.303 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: ---------------------------------------------------- 2023-12-20T04:24:42.303 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: ntp-4 is maintained by Network Time Foundation, 2023-12-20T04:24:42.303 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-12-20T04:24:42.303 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: corporation. Support and training for ntp-4 are 2023-12-20T04:24:42.303 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: available at https://www.nwtime.org/support 2023-12-20T04:24:42.303 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: ---------------------------------------------------- 2023-12-20T04:24:42.304 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: proto: precision = 0.044 usec (-24) 2023-12-20T04:24:42.304 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: basedate set to 2022-02-04 2023-12-20T04:24:42.304 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: gps base set to 2022-02-06 (week 2196) 2023-12-20T04:24:42.304 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-20T04:24:42.304 INFO:teuthology.orchestra.run.smithi032.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-20T04:24:42.305 INFO:teuthology.orchestra.run.smithi032.stderr:restrict ::: KOD does nothing without LIMITED. 2023-12-20T04:24:42.305 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: restrict ::: KOD does nothing without LIMITED. 2023-12-20T04:24:42.305 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: Listen and drop on 0 v6wildcard [::]:123 2023-12-20T04:24:42.305 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-12-20T04:24:42.306 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: Listen normally on 2 lo 127.0.0.1:123 2023-12-20T04:24:42.306 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: Listen normally on 3 ens1f0 172.21.15.32:123 2023-12-20T04:24:42.306 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: Listen normally on 4 lo [::1]:123 2023-12-20T04:24:42.306 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: Listen normally on 5 ens1f0 [fe80::ae1f:6bff:fef8:2602%3]:123 2023-12-20T04:24:42.306 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:42 ntpd[27885]: Listening on routing socket on fd #22 for interface updates 2023-12-20T04:24:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: ntpd 4.2.8p15@1.3728-o Wed Feb 16 17:13:02 UTC 2022 (1): Starting 2023-12-20T04:24:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: Command line: ntpd -gq 2023-12-20T04:24:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: ---------------------------------------------------- 2023-12-20T04:24:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: ntp-4 is maintained by Network Time Foundation, 2023-12-20T04:24:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: Inc. (NTF), a non-profit 501(c)(3) public-benefit 2023-12-20T04:24:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: corporation. Support and training for ntp-4 are 2023-12-20T04:24:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: available at https://www.nwtime.org/support 2023-12-20T04:24:42.330 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: ---------------------------------------------------- 2023-12-20T04:24:42.331 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: proto: precision = 0.043 usec (-24) 2023-12-20T04:24:42.331 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: basedate set to 2022-02-04 2023-12-20T04:24:42.331 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: gps base set to 2022-02-06 (week 2196) 2023-12-20T04:24:42.331 INFO:teuthology.orchestra.run.smithi039.stderr:restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-20T04:24:42.332 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: restrict 0.0.0.0: KOD does nothing without LIMITED. 2023-12-20T04:24:42.332 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: restrict ::: KOD does nothing without LIMITED. 2023-12-20T04:24:42.332 INFO:teuthology.orchestra.run.smithi039.stderr:restrict ::: KOD does nothing without LIMITED. 2023-12-20T04:24:42.333 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: Listen and drop on 0 v6wildcard [::]:123 2023-12-20T04:24:42.333 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: Listen and drop on 1 v4wildcard 0.0.0.0:123 2023-12-20T04:24:42.333 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: Listen normally on 2 lo 127.0.0.1:123 2023-12-20T04:24:42.334 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: Listen normally on 3 ens1f0 172.21.15.39:123 2023-12-20T04:24:42.334 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: Listen normally on 4 lo [::1]:123 2023-12-20T04:24:42.334 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: Listen normally on 5 ens1f0 [fe80::ec4:7aff:febd:1370%4]:123 2023-12-20T04:24:42.334 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:42 ntpd[27872]: Listening on routing socket on fd #22 for interface updates 2023-12-20T04:24:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:43 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:43 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:43 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:43 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:43 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:43.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:43 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:44.330 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:44 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:44.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:44 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:44.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:44 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:44.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:44 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:44.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:44 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:44.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:44 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:45 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:45.303 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:45 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:45.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:45.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:45.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:45.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:45.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:45.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:45.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:45.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:45.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:45.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:45.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:45.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:45 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:46.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:46 ntpd[27885]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:46 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:46 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:46 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:46 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:46 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:46 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:46 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:46.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:46 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:47 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:47 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:47.303 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:47 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:47 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:47 ntpd[27885]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:47.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:47 ntpd[27885]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:47.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:47.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:47.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:47.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:47 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:48.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:48.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:48.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:48.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:48 ntpd[27885]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:48.330 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:48 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:48 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:48 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:48 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:48 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:48.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:48 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:49 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:49 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:49 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:49 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:49 ntpd[27885]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:49 ntpd[27885]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:49.331 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:49.332 INFO:teuthology.orchestra.run.smithi039.stdout:20 Dec 04:24:49 ntpd[27872]: ntpd: time slew -0.001801 s 2023-12-20T04:24:49.332 INFO:teuthology.orchestra.run.smithi039.stdout:ntpd: time slew -0.001801s 2023-12-20T04:24:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-12-20T04:24:49.332 INFO:teuthology.orchestra.run.smithi039.stderr:20 Dec 04:24:49 ntpd[27872]: can't open /var/log/ntpstats/loopstats.20231220: Permission denied 2023-12-20T04:24:49.429 INFO:teuthology.orchestra.run.smithi039.stdout: remote refid st t when poll reach delay offset jitter 2023-12-20T04:24:49.429 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================== 2023-12-20T04:24:49.429 INFO:teuthology.orchestra.run.smithi039.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-20T04:24:49.429 INFO:teuthology.orchestra.run.smithi039.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-20T04:24:49.429 INFO:teuthology.orchestra.run.smithi039.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-20T04:24:49.429 INFO:teuthology.orchestra.run.smithi039.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-20T04:24:50.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:50.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:50.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:50.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:50.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:50 ntpd[27885]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:51 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:51 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:51 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:51 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:51 ntpd[27885]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:51.304 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:51 ntpd[27885]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:52.305 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:52 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:52 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:52 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:52 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:52 ntpd[27885]: couldn't unlink /var/log/ntpstats/rawstats: Permission denied 2023-12-20T04:24:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:52 ntpd[27885]: can't open /var/log/ntpstats/rawstats.20231220: Permission denied 2023-12-20T04:24:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:52 ntpd[27885]: couldn't unlink /var/log/ntpstats/peerstats: Permission denied 2023-12-20T04:24:52.306 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:52 ntpd[27885]: can't open /var/log/ntpstats/peerstats.20231220: Permission denied 2023-12-20T04:24:52.306 INFO:teuthology.orchestra.run.smithi032.stdout:20 Dec 04:24:52 ntpd[27885]: ntpd: time slew -0.001271 s 2023-12-20T04:24:52.306 INFO:teuthology.orchestra.run.smithi032.stdout:ntpd: time slew -0.001271s 2023-12-20T04:24:52.307 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:52 ntpd[27885]: couldn't unlink /var/log/ntpstats/loopstats: Permission denied 2023-12-20T04:24:52.307 INFO:teuthology.orchestra.run.smithi032.stderr:20 Dec 04:24:52 ntpd[27885]: can't open /var/log/ntpstats/loopstats.20231220: Permission denied 2023-12-20T04:24:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2023-12-20T04:24:52.402 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2023-12-20T04:24:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-20T04:24:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-20T04:24:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-20T04:24:52.402 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 +0.000 0.000 2023-12-20T04:24:52.403 INFO:teuthology.run_tasks:Running task install... 2023-12-20T04:24:52.412 DEBUG:teuthology.task.install:project ceph 2023-12-20T04:24:52.412 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}} 2023-12-20T04:24:52.413 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'} 2023-12-20T04:24:52.413 INFO:teuthology.task.install:Using flavor: default 2023-12-20T04:24:52.420 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2023-12-20T04:24:52.420 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2023-12-20T04:24:52.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-key list | grep Ceph 2023-12-20T04:24:52.421 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-key list | grep Ceph 2023-12-20T04:24:52.534 INFO:teuthology.orchestra.run.smithi039.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-12-20T04:24:52.559 INFO:teuthology.orchestra.run.smithi032.stderr:Warning: apt-key is deprecated. Manage keyring files in trusted.gpg.d instead (see apt-key(8)). 2023-12-20T04:24:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-12-20T04:24:52.589 INFO:teuthology.orchestra.run.smithi039.stdout:uid [ unknown] Ceph.com (release key) 2023-12-20T04:24:52.590 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-12-20T04:24:52.590 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-12-20T04:24:52.591 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-20T04:24:52.614 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph automated package build (Ceph automated package build) 2023-12-20T04:24:52.614 INFO:teuthology.orchestra.run.smithi032.stdout:uid [ unknown] Ceph.com (release key) 2023-12-20T04:24:52.615 INFO:teuthology.task.install.deb:Installing packages: ceph, cephadm, ceph-mds, ceph-mgr, ceph-common, ceph-fuse, ceph-test, ceph-volume, radosgw, python3-rados, python3-rgw, python3-cephfs, python3-rbd, libcephfs2, libcephfs-dev, librados2, librbd1, rbd-fuse, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror on remote deb x86_64 2023-12-20T04:24:52.615 INFO:teuthology.task.install.deb:Installing system (non-project) packages: bison, flex, libelf-dev, libssl-dev, network-manager, iproute2, util-linux, dump, indent, libaio-dev, libtool-bin, uuid-dev, xfslibs-dev, postgresql, postgresql-client, postgresql-common, postgresql-contrib on remote deb x86_64 2023-12-20T04:24:52.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-20T04:24:52.779 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default/ 2023-12-20T04:24:52.779 INFO:teuthology.task.install.deb:Package version is 18.2.1-132-g568df92b-1jammy 2023-12-20T04:24:52.820 INFO:teuthology.task.install.deb:Pulling from https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default/ 2023-12-20T04:24:52.820 INFO:teuthology.task.install.deb:Package version is 18.2.1-132-g568df92b-1jammy 2023-12-20T04:24:52.888 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:24:52.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-12-20T04:24:52.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo apt-get update 2023-12-20T04:24:52.990 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:24:52.990 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/apt/sources.list.d/ceph.list 2023-12-20T04:24:53.012 DEBUG:teuthology.orchestra.run.smithi039:> sudo apt-get update 2023-12-20T04:24:53.096 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-20T04:24:53.102 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-12-20T04:24:53.119 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-12-20T04:24:53.211 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy InRelease 2023-12-20T04:24:53.211 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:1 http://archive.ubuntu.com/ubuntu jammy InRelease 2023-12-20T04:24:53.213 INFO:teuthology.orchestra.run.smithi032.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-12-20T04:24:53.218 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:2 http://archive.ubuntu.com/ubuntu jammy-updates InRelease 2023-12-20T04:24:53.240 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:3 http://archive.ubuntu.com/ubuntu jammy-backports InRelease 2023-12-20T04:24:53.247 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-12-20T04:24:53.285 INFO:teuthology.orchestra.run.smithi032.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy Release.gpg 2023-12-20T04:24:53.321 INFO:teuthology.orchestra.run.smithi039.stdout:Ign:4 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy InRelease 2023-12-20T04:24:53.321 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2023-12-20T04:24:53.323 INFO:teuthology.orchestra.run.smithi039.stdout:Hit:5 http://security.ubuntu.com/ubuntu jammy-security InRelease 2023-12-20T04:24:53.357 INFO:teuthology.orchestra.run.smithi039.stdout:Get:6 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy Release [7,662 B] 2023-12-20T04:24:53.370 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-12-20T04:24:53.400 INFO:teuthology.orchestra.run.smithi039.stdout:Ign:7 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy Release.gpg 2023-12-20T04:24:53.440 INFO:teuthology.orchestra.run.smithi039.stdout:Get:8 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main i386 Packages [3,406 B] 2023-12-20T04:24:53.488 INFO:teuthology.orchestra.run.smithi039.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 Packages [17.4 kB] 2023-12-20T04:24:53.724 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 28.5 kB in 1s (48.4 kB/s) 2023-12-20T04:24:53.841 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 28.5 kB in 1s (47.8 kB/s) 2023-12-20T04:24:54.688 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-12-20T04:24:54.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.1-132-g568df92b-1jammy cephadm=18.2.1-132-g568df92b-1jammy ceph-mds=18.2.1-132-g568df92b-1jammy ceph-mgr=18.2.1-132-g568df92b-1jammy ceph-common=18.2.1-132-g568df92b-1jammy ceph-fuse=18.2.1-132-g568df92b-1jammy ceph-test=18.2.1-132-g568df92b-1jammy ceph-volume=18.2.1-132-g568df92b-1jammy radosgw=18.2.1-132-g568df92b-1jammy python3-rados=18.2.1-132-g568df92b-1jammy python3-rgw=18.2.1-132-g568df92b-1jammy python3-cephfs=18.2.1-132-g568df92b-1jammy python3-rbd=18.2.1-132-g568df92b-1jammy libcephfs2=18.2.1-132-g568df92b-1jammy libcephfs-dev=18.2.1-132-g568df92b-1jammy librados2=18.2.1-132-g568df92b-1jammy librbd1=18.2.1-132-g568df92b-1jammy rbd-fuse=18.2.1-132-g568df92b-1jammy python3-cephfs=18.2.1-132-g568df92b-1jammy cephfs-shell=18.2.1-132-g568df92b-1jammy cephfs-top=18.2.1-132-g568df92b-1jammy cephfs-mirror=18.2.1-132-g568df92b-1jammy 2023-12-20T04:24:54.780 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-12-20T04:24:54.803 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2023-12-20T04:24:54.817 DEBUG:teuthology.orchestra.run.smithi039:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install ceph=18.2.1-132-g568df92b-1jammy cephadm=18.2.1-132-g568df92b-1jammy ceph-mds=18.2.1-132-g568df92b-1jammy ceph-mgr=18.2.1-132-g568df92b-1jammy ceph-common=18.2.1-132-g568df92b-1jammy ceph-fuse=18.2.1-132-g568df92b-1jammy ceph-test=18.2.1-132-g568df92b-1jammy ceph-volume=18.2.1-132-g568df92b-1jammy radosgw=18.2.1-132-g568df92b-1jammy python3-rados=18.2.1-132-g568df92b-1jammy python3-rgw=18.2.1-132-g568df92b-1jammy python3-cephfs=18.2.1-132-g568df92b-1jammy python3-rbd=18.2.1-132-g568df92b-1jammy libcephfs2=18.2.1-132-g568df92b-1jammy libcephfs-dev=18.2.1-132-g568df92b-1jammy librados2=18.2.1-132-g568df92b-1jammy librbd1=18.2.1-132-g568df92b-1jammy rbd-fuse=18.2.1-132-g568df92b-1jammy python3-cephfs=18.2.1-132-g568df92b-1jammy cephfs-shell=18.2.1-132-g568df92b-1jammy cephfs-top=18.2.1-132-g568df92b-1jammy cephfs-mirror=18.2.1-132-g568df92b-1jammy 2023-12-20T04:24:54.895 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2023-12-20T04:24:54.952 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2023-12-20T04:24:54.953 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2023-12-20T04:24:55.070 INFO:teuthology.orchestra.run.smithi039.stdout:Building dependency tree... 2023-12-20T04:24:55.071 INFO:teuthology.orchestra.run.smithi039.stdout:Reading state information... 2023-12-20T04:24:55.113 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2023-12-20T04:24:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-20T04:24:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-20T04:24:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-20T04:24:55.113 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-20T04:24:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-20T04:24:55.114 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-20T04:24:55.114 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2023-12-20T04:24:55.114 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-12-20T04:24:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-12-20T04:24:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-12-20T04:24:55.115 INFO:teuthology.orchestra.run.smithi032.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-12-20T04:24:55.116 INFO:teuthology.orchestra.run.smithi032.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-12-20T04:24:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-12-20T04:24:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-12-20T04:24:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-12-20T04:24:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-12-20T04:24:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-12-20T04:24:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-12-20T04:24:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-12-20T04:24:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-12-20T04:24:55.117 INFO:teuthology.orchestra.run.smithi032.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-12-20T04:24:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-12-20T04:24:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-12-20T04:24:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-12-20T04:24:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-12-20T04:24:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-12-20T04:24:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-12-20T04:24:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-12-20T04:24:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-12-20T04:24:55.118 INFO:teuthology.orchestra.run.smithi032.stdout: qttranslations5-l10n socat xmlstarlet 2023-12-20T04:24:55.119 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2023-12-20T04:24:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-12-20T04:24:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-12-20T04:24:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-12-20T04:24:55.119 INFO:teuthology.orchestra.run.smithi032.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-12-20T04:24:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-12-20T04:24:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-12-20T04:24:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: python3-watchdog 2023-12-20T04:24:55.120 INFO:teuthology.orchestra.run.smithi032.stdout:Recommended packages: 2023-12-20T04:24:55.120 INFO:teuthology.orchestra.run.smithi032.stdout: btrfs-tools 2023-12-20T04:24:55.236 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages were automatically installed and are no longer required: 2023-12-20T04:24:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-20T04:24:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-20T04:24:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-20T04:24:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-20T04:24:55.236 INFO:teuthology.orchestra.run.smithi039.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-20T04:24:55.236 INFO:teuthology.orchestra.run.smithi039.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-20T04:24:55.238 INFO:teuthology.orchestra.run.smithi039.stdout:The following additional packages will be installed: 2023-12-20T04:24:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-base ceph-mgr-cephadm ceph-mgr-dashboard ceph-mgr-diskprediction-local 2023-12-20T04:24:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-k8sevents ceph-mgr-modules-core ceph-mon ceph-osd jq 2023-12-20T04:24:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-12-20T04:24:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 libradosstriper1 2023-12-20T04:24:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: librdkafka1 librgw2 libsqlite3-mod-ceph libthrift-0.16.0 nvme-cli 2023-12-20T04:24:55.238 INFO:teuthology.orchestra.run.smithi039.stdout: python-asyncssh-doc python-babel-localedata python-pastedeploy-tpl 2023-12-20T04:24:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: python3-asyncssh python3-babel python3-bcrypt python3-cachetools 2023-12-20T04:24:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: python3-ceph-argparse python3-ceph-common python3-certifi python3-cheroot 2023-12-20T04:24:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cherrypy3 python3-cmd2 python3-colorama python3-google-auth 2023-12-20T04:24:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: python3-idna python3-iniconfig python3-jaraco.classes 2023-12-20T04:24:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco.collections python3-jaraco.functools python3-jaraco.text 2023-12-20T04:24:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jinja2 python3-joblib python3-kubernetes python3-logutils 2023-12-20T04:24:55.239 INFO:teuthology.orchestra.run.smithi039.stdout: python3-mako python3-markupsafe python3-natsort python3-openssl 2023-12-20T04:24:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: python3-paste python3-pastedeploy python3-pastescript python3-pecan 2023-12-20T04:24:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pluggy python3-portend python3-prettytable python3-psutil python3-py 2023-12-20T04:24:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyasn1 python3-pyasn1-modules python3-pygments python3-pyinotify 2023-12-20T04:24:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pyperclip python3-pytest python3-repoze.lru python3-requests 2023-12-20T04:24:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib python3-routes python3-rsa python3-simplegeneric 2023-12-20T04:24:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplejson python3-singledispatch python3-sklearn 2023-12-20T04:24:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: python3-sklearn-lib python3-tempita python3-tempora python3-threadpoolctl 2023-12-20T04:24:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: python3-toml python3-urllib3 python3-waitress python3-wcwidth python3-webob 2023-12-20T04:24:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: python3-websocket python3-webtest python3-werkzeug python3-zc.lockfile 2023-12-20T04:24:55.240 INFO:teuthology.orchestra.run.smithi039.stdout: qttranslations5-l10n socat xmlstarlet 2023-12-20T04:24:55.241 INFO:teuthology.orchestra.run.smithi039.stdout:Suggested packages: 2023-12-20T04:24:55.241 INFO:teuthology.orchestra.run.smithi039.stdout: python3-influxdb python-jinja2-doc python3-beaker python-mako-doc 2023-12-20T04:24:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: python-natsort-doc python-openssl-doc python3-openssl-dbg httpd-wsgi 2023-12-20T04:24:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: libapache2-mod-python libapache2-mod-scgi libjs-mochikit python-pecan-doc 2023-12-20T04:24:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: python-psutil-doc subversion python-pygments-doc python-pyinotify-doc 2023-12-20T04:24:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: python3-socks python-requests-doc python3-dap python-sklearn-doc ipython3 2023-12-20T04:24:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: python-waitress-doc python-webob-doc python-webtest-doc python-werkzeug-doc 2023-12-20T04:24:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: python3-watchdog 2023-12-20T04:24:55.242 INFO:teuthology.orchestra.run.smithi039.stdout:Recommended packages: 2023-12-20T04:24:55.242 INFO:teuthology.orchestra.run.smithi039.stdout: btrfs-tools 2023-12-20T04:24:55.266 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2023-12-20T04:24:55.266 INFO:teuthology.orchestra.run.smithi032.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-12-20T04:24:55.266 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-12-20T04:24:55.266 INFO:teuthology.orchestra.run.smithi032.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-12-20T04:24:55.267 INFO:teuthology.orchestra.run.smithi032.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-12-20T04:24:55.267 INFO:teuthology.orchestra.run.smithi032.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-12-20T04:24:55.267 INFO:teuthology.orchestra.run.smithi032.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-12-20T04:24:55.267 INFO:teuthology.orchestra.run.smithi032.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-12-20T04:24:55.268 INFO:teuthology.orchestra.run.smithi032.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-12-20T04:24:55.268 INFO:teuthology.orchestra.run.smithi032.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-12-20T04:24:55.268 INFO:teuthology.orchestra.run.smithi032.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-12-20T04:24:55.268 INFO:teuthology.orchestra.run.smithi032.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-12-20T04:24:55.268 INFO:teuthology.orchestra.run.smithi032.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-12-20T04:24:55.268 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-12-20T04:24:55.269 INFO:teuthology.orchestra.run.smithi032.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-12-20T04:24:55.269 INFO:teuthology.orchestra.run.smithi032.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-12-20T04:24:55.269 INFO:teuthology.orchestra.run.smithi032.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-12-20T04:24:55.269 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-12-20T04:24:55.269 INFO:teuthology.orchestra.run.smithi032.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-12-20T04:24:55.269 INFO:teuthology.orchestra.run.smithi032.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-12-20T04:24:55.269 INFO:teuthology.orchestra.run.smithi032.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-12-20T04:24:55.269 INFO:teuthology.orchestra.run.smithi032.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-12-20T04:24:55.269 INFO:teuthology.orchestra.run.smithi032.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-12-20T04:24:55.269 INFO:teuthology.orchestra.run.smithi032.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-12-20T04:24:55.269 INFO:teuthology.orchestra.run.smithi032.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-12-20T04:24:55.270 INFO:teuthology.orchestra.run.smithi032.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-12-20T04:24:55.270 INFO:teuthology.orchestra.run.smithi032.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-12-20T04:24:55.270 INFO:teuthology.orchestra.run.smithi032.stdout: socat xmlstarlet 2023-12-20T04:24:55.391 INFO:teuthology.orchestra.run.smithi039.stdout:The following NEW packages will be installed: 2023-12-20T04:24:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: ceph ceph-base ceph-common ceph-fuse ceph-mds ceph-mgr ceph-mgr-cephadm 2023-12-20T04:24:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-k8sevents 2023-12-20T04:24:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: ceph-mgr-modules-core ceph-mon ceph-osd ceph-test ceph-volume cephadm 2023-12-20T04:24:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: cephfs-mirror cephfs-shell cephfs-top jq libcephfs-dev libcephfs2 2023-12-20T04:24:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: libdouble-conversion3 libfmt8 libjq1 liblttng-ust1 liboath0 libonig5 2023-12-20T04:24:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: libpcre2-16-0 libqt5core5a libqt5dbus5 libqt5network5 librados2 2023-12-20T04:24:55.391 INFO:teuthology.orchestra.run.smithi039.stdout: libradosstriper1 librbd1 librdkafka1 librgw2 libsqlite3-mod-ceph 2023-12-20T04:24:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: libthrift-0.16.0 nvme-cli python-asyncssh-doc python-babel-localedata 2023-12-20T04:24:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: python-pastedeploy-tpl python3-asyncssh python3-babel python3-bcrypt 2023-12-20T04:24:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: python3-cachetools python3-ceph-argparse python3-ceph-common python3-cephfs 2023-12-20T04:24:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: python3-certifi python3-cheroot python3-cherrypy3 python3-cmd2 2023-12-20T04:24:55.392 INFO:teuthology.orchestra.run.smithi039.stdout: python3-colorama python3-google-auth python3-idna python3-iniconfig 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco.classes python3-jaraco.collections python3-jaraco.functools 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-jaraco.text python3-jinja2 python3-joblib python3-kubernetes 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-logutils python3-mako python3-markupsafe python3-natsort 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-openssl python3-paste python3-pastedeploy python3-pastescript 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pecan python3-pluggy python3-portend python3-prettytable 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-psutil python3-py python3-pyasn1 python3-pyasn1-modules 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-pygments python3-pyinotify python3-pyperclip python3-pytest 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-rados python3-rbd python3-repoze.lru python3-requests 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-requests-oauthlib python3-rgw python3-routes python3-rsa 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-simplegeneric python3-simplejson python3-singledispatch 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-sklearn python3-sklearn-lib python3-tempita python3-tempora 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-threadpoolctl python3-toml python3-urllib3 python3-waitress 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-wcwidth python3-webob python3-websocket python3-webtest 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: python3-werkzeug python3-zc.lockfile qttranslations5-l10n radosgw rbd-fuse 2023-12-20T04:24:55.393 INFO:teuthology.orchestra.run.smithi039.stdout: socat xmlstarlet 2023-12-20T04:24:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:0 upgraded, 115 newly installed, 0 to remove and 268 not upgraded. 2023-12-20T04:24:55.463 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 181 MB of archives. 2023-12-20T04:24:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 812 MB of additional disk space will be used. 2023-12-20T04:24:55.464 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-12-20T04:24:55.466 INFO:teuthology.orchestra.run.smithi039.stdout:0 upgraded, 115 newly installed, 0 to remove and 268 not upgraded. 2023-12-20T04:24:55.466 INFO:teuthology.orchestra.run.smithi039.stdout:Need to get 181 MB of archives. 2023-12-20T04:24:55.466 INFO:teuthology.orchestra.run.smithi039.stdout:After this operation, 812 MB of additional disk space will be used. 2023-12-20T04:24:55.466 INFO:teuthology.orchestra.run.smithi039.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyperclip all 1.8.2-2 [11.5 kB] 2023-12-20T04:24:55.487 INFO:teuthology.orchestra.run.smithi039.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-12-20T04:24:55.493 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.1-132-g568df92b-1jammy [3,418 kB] 2023-12-20T04:24:55.498 INFO:teuthology.orchestra.run.smithi039.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-12-20T04:24:55.542 INFO:teuthology.orchestra.run.smithi039.stdout:Get:4 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librados2 amd64 18.2.1-132-g568df92b-1jammy [3,418 kB] 2023-12-20T04:24:55.545 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-wcwidth all 0.2.5+dfsg1-1 [21.9 kB] 2023-12-20T04:24:55.546 INFO:teuthology.orchestra.run.smithi039.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-12-20T04:24:55.555 INFO:teuthology.orchestra.run.smithi039.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-12-20T04:24:55.574 INFO:teuthology.orchestra.run.smithi039.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-12-20T04:24:55.577 INFO:teuthology.orchestra.run.smithi039.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-12-20T04:24:55.588 INFO:teuthology.orchestra.run.smithi039.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-12-20T04:24:55.591 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cmd2 all 2.3.3-1 [124 kB] 2023-12-20T04:24:55.631 INFO:teuthology.orchestra.run.smithi039.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-12-20T04:24:55.633 INFO:teuthology.orchestra.run.smithi039.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-12-20T04:24:55.644 INFO:teuthology.orchestra.run.smithi039.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-12-20T04:24:55.647 INFO:teuthology.orchestra.run.smithi039.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-12-20T04:24:55.648 INFO:teuthology.orchestra.run.smithi039.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-12-20T04:24:55.655 INFO:teuthology.orchestra.run.smithi039.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-12-20T04:24:55.655 INFO:teuthology.orchestra.run.smithi039.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-12-20T04:24:55.656 INFO:teuthology.orchestra.run.smithi039.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-12-20T04:24:55.657 INFO:teuthology.orchestra.run.smithi039.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-12-20T04:24:55.658 INFO:teuthology.orchestra.run.smithi039.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-12-20T04:24:55.678 INFO:teuthology.orchestra.run.smithi039.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-12-20T04:24:55.679 INFO:teuthology.orchestra.run.smithi039.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-12-20T04:24:55.679 INFO:teuthology.orchestra.run.smithi039.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-12-20T04:24:55.680 INFO:teuthology.orchestra.run.smithi039.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-12-20T04:24:55.680 INFO:teuthology.orchestra.run.smithi039.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-12-20T04:24:55.683 INFO:teuthology.orchestra.run.smithi039.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-12-20T04:24:55.683 INFO:teuthology.orchestra.run.smithi039.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-12-20T04:24:55.684 INFO:teuthology.orchestra.run.smithi039.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-12-20T04:24:55.684 INFO:teuthology.orchestra.run.smithi039.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-12-20T04:24:55.698 INFO:teuthology.orchestra.run.smithi039.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-12-20T04:24:55.705 INFO:teuthology.orchestra.run.smithi039.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-12-20T04:24:55.706 INFO:teuthology.orchestra.run.smithi039.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-12-20T04:24:55.707 INFO:teuthology.orchestra.run.smithi039.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-12-20T04:24:55.708 INFO:teuthology.orchestra.run.smithi039.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-12-20T04:24:55.708 INFO:teuthology.orchestra.run.smithi039.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-12-20T04:24:55.713 INFO:teuthology.orchestra.run.smithi039.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-12-20T04:24:55.713 INFO:teuthology.orchestra.run.smithi039.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-12-20T04:24:55.713 INFO:teuthology.orchestra.run.smithi039.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-12-20T04:24:55.718 INFO:teuthology.orchestra.run.smithi039.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-12-20T04:24:55.734 INFO:teuthology.orchestra.run.smithi039.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-12-20T04:24:55.736 INFO:teuthology.orchestra.run.smithi039.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-12-20T04:24:55.737 INFO:teuthology.orchestra.run.smithi039.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-12-20T04:24:55.738 INFO:teuthology.orchestra.run.smithi039.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-12-20T04:24:55.739 INFO:teuthology.orchestra.run.smithi039.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-12-20T04:24:55.741 INFO:teuthology.orchestra.run.smithi039.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-12-20T04:24:55.741 INFO:teuthology.orchestra.run.smithi039.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-12-20T04:24:55.743 INFO:teuthology.orchestra.run.smithi039.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-12-20T04:24:55.796 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libfmt8 amd64 8.1.1+ds1-2 [60.2 kB] 2023-12-20T04:24:55.811 INFO:teuthology.orchestra.run.smithi039.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-12-20T04:24:55.812 INFO:teuthology.orchestra.run.smithi039.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-12-20T04:24:55.813 INFO:teuthology.orchestra.run.smithi039.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-12-20T04:24:55.814 INFO:teuthology.orchestra.run.smithi039.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-12-20T04:24:55.814 INFO:teuthology.orchestra.run.smithi039.stdout:Get:51 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.1-132-g568df92b-1jammy [3,104 kB] 2023-12-20T04:24:55.815 INFO:teuthology.orchestra.run.smithi039.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-12-20T04:24:55.815 INFO:teuthology.orchestra.run.smithi039.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-12-20T04:24:55.823 INFO:teuthology.orchestra.run.smithi039.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-12-20T04:24:55.823 INFO:teuthology.orchestra.run.smithi039.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-12-20T04:24:55.827 INFO:teuthology.orchestra.run.smithi039.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-12-20T04:24:55.840 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblttng-ust1 amd64 2.13.1-1ubuntu1 [190 kB] 2023-12-20T04:24:55.842 INFO:teuthology.orchestra.run.smithi039.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-12-20T04:24:55.843 INFO:teuthology.orchestra.run.smithi039.stdout:Get:58 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-12-20T04:24:55.865 INFO:teuthology.orchestra.run.smithi039.stdout:Get:59 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-12-20T04:24:55.868 INFO:teuthology.orchestra.run.smithi039.stdout:Get:60 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-12-20T04:24:55.868 INFO:teuthology.orchestra.run.smithi039.stdout:Get:61 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-12-20T04:24:55.897 INFO:teuthology.orchestra.run.smithi039.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-12-20T04:24:55.897 INFO:teuthology.orchestra.run.smithi039.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-12-20T04:24:55.897 INFO:teuthology.orchestra.run.smithi039.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-12-20T04:24:55.898 INFO:teuthology.orchestra.run.smithi039.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-12-20T04:24:55.898 INFO:teuthology.orchestra.run.smithi039.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-12-20T04:24:55.912 INFO:teuthology.orchestra.run.smithi039.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-12-20T04:24:55.912 INFO:teuthology.orchestra.run.smithi039.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-12-20T04:24:55.912 INFO:teuthology.orchestra.run.smithi039.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-12-20T04:24:55.915 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libdouble-conversion3 amd64 3.1.7-4 [39.0 kB] 2023-12-20T04:24:55.915 INFO:teuthology.orchestra.run.smithi039.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-12-20T04:24:55.926 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpcre2-16-0 amd64 10.39-3ubuntu0.1 [203 kB] 2023-12-20T04:24:55.930 INFO:teuthology.orchestra.run.smithi039.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-12-20T04:24:55.932 INFO:teuthology.orchestra.run.smithi039.stdout:Get:72 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-12-20T04:24:55.933 INFO:teuthology.orchestra.run.smithi039.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-12-20T04:24:55.933 INFO:teuthology.orchestra.run.smithi039.stdout:Get:74 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.1-132-g568df92b-1jammy [670 kB] 2023-12-20T04:24:55.936 INFO:teuthology.orchestra.run.smithi039.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-12-20T04:24:55.939 INFO:teuthology.orchestra.run.smithi039.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-12-20T04:24:55.940 INFO:teuthology.orchestra.run.smithi039.stdout:Get:77 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-12-20T04:24:55.949 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librbd1 amd64 18.2.1-132-g568df92b-1jammy [3,104 kB] 2023-12-20T04:24:55.951 INFO:teuthology.orchestra.run.smithi039.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-12-20T04:24:55.954 INFO:teuthology.orchestra.run.smithi039.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-12-20T04:24:55.955 INFO:teuthology.orchestra.run.smithi039.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-12-20T04:24:55.957 INFO:teuthology.orchestra.run.smithi039.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-12-20T04:24:55.973 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5core5a amd64 5.15.3+dfsg-2ubuntu0.2 [2,006 kB] 2023-12-20T04:24:55.975 INFO:teuthology.orchestra.run.smithi039.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-12-20T04:24:55.975 INFO:teuthology.orchestra.run.smithi039.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-12-20T04:24:55.975 INFO:teuthology.orchestra.run.smithi039.stdout:Get:84 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.1-132-g568df92b-1jammy [356 kB] 2023-12-20T04:24:55.976 INFO:teuthology.orchestra.run.smithi039.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-12-20T04:24:55.993 INFO:teuthology.orchestra.run.smithi039.stdout:Get:86 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.1-132-g568df92b-1jammy [32.8 kB] 2023-12-20T04:24:55.994 INFO:teuthology.orchestra.run.smithi039.stdout:Get:87 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.1-132-g568df92b-1jammy [180 kB] 2023-12-20T04:24:56.005 INFO:teuthology.orchestra.run.smithi039.stdout:Get:88 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.1-132-g568df92b-1jammy [61.5 kB] 2023-12-20T04:24:56.010 INFO:teuthology.orchestra.run.smithi039.stdout:Get:89 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.1-132-g568df92b-1jammy [330 kB] 2023-12-20T04:24:56.021 INFO:teuthology.orchestra.run.smithi039.stdout:Get:90 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.1-132-g568df92b-1jammy [5,884 kB] 2023-12-20T04:24:56.138 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5dbus5 amd64 5.15.3+dfsg-2ubuntu0.2 [222 kB] 2023-12-20T04:24:56.146 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy-updates/universe amd64 libqt5network5 amd64 5.15.3+dfsg-2ubuntu0.2 [731 kB] 2023-12-20T04:24:56.173 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy/universe amd64 libthrift-0.16.0 amd64 0.16.0-2 [267 kB] 2023-12-20T04:24:56.201 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs2 amd64 18.2.1-132-g568df92b-1jammy [670 kB] 2023-12-20T04:24:56.253 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rados amd64 18.2.1-132-g568df92b-1jammy [356 kB] 2023-12-20T04:24:56.283 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-argparse all 18.2.1-132-g568df92b-1jammy [32.8 kB] 2023-12-20T04:24:56.285 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-cephfs amd64 18.2.1-132-g568df92b-1jammy [180 kB] 2023-12-20T04:24:56.300 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-ceph-common all 18.2.1-132-g568df92b-1jammy [61.5 kB] 2023-12-20T04:24:56.307 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rbd amd64 18.2.1-132-g568df92b-1jammy [330 kB] 2023-12-20T04:24:56.333 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 librgw2 amd64 18.2.1-132-g568df92b-1jammy [5,884 kB] 2023-12-20T04:24:56.344 INFO:teuthology.orchestra.run.smithi039.stdout:Get:91 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.1-132-g568df92b-1jammy [112 kB] 2023-12-20T04:24:56.345 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-prettytable all 2.5.0-2 [31.3 kB] 2023-12-20T04:24:56.349 INFO:teuthology.orchestra.run.smithi039.stdout:Get:92 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.1-132-g568df92b-1jammy [442 kB] 2023-12-20T04:24:56.371 INFO:teuthology.orchestra.run.smithi039.stdout:Get:93 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.1-132-g568df92b-1jammy [22.0 MB] 2023-12-20T04:24:56.548 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy/universe amd64 librdkafka1 amd64 1.8.0-1build1 [633 kB] 2023-12-20T04:24:56.829 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 python3-rgw amd64 18.2.1-132-g568df92b-1jammy [112 kB] 2023-12-20T04:24:56.837 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libradosstriper1 amd64 18.2.1-132-g568df92b-1jammy [442 kB] 2023-12-20T04:24:56.874 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-common amd64 18.2.1-132-g568df92b-1jammy [22.0 MB] 2023-12-20T04:24:56.946 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 liboath0 amd64 2.6.7-3build1 [41.1 kB] 2023-12-20T04:24:56.954 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.functools all 3.4.0-2 [9,030 B] 2023-12-20T04:24:56.956 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-cheroot all 8.5.2+ds1-1ubuntu3.1 [71.1 kB] 2023-12-20T04:24:56.967 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.classes all 3.2.1-3 [6,452 B] 2023-12-20T04:24:56.967 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.text all 3.6.0-2 [8,716 B] 2023-12-20T04:24:56.968 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jaraco.collections all 3.4.0-2 [11.4 kB] 2023-12-20T04:24:56.970 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempora all 4.1.2-1 [14.8 kB] 2023-12-20T04:24:56.972 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-portend all 3.0.0-1 [7,240 B] 2023-12-20T04:24:56.974 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-zc.lockfile all 2.0-1 [8,980 B] 2023-12-20T04:24:57.138 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cherrypy3 all 18.6.1-4 [208 kB] 2023-12-20T04:24:57.555 INFO:teuthology.orchestra.run.smithi039.stdout:Get:94 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.1-132-g568df92b-1jammy [6,835 kB] 2023-12-20T04:24:57.586 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-natsort all 8.0.2-1 [35.3 kB] 2023-12-20T04:24:57.596 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-logutils all 0.3.3-8 [17.6 kB] 2023-12-20T04:24:57.602 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-markupsafe amd64 2.0.1-2build1 [12.7 kB] 2023-12-20T04:24:57.606 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-mako all 1.1.3+ds1-2ubuntu0.1 [60.5 kB] 2023-12-20T04:24:57.624 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplegeneric all 0.8.1-3 [11.3 kB] 2023-12-20T04:24:57.628 INFO:teuthology.orchestra.run.smithi032.stdout:Get:41 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-singledispatch all 3.4.0.3-3 [7,320 B] 2023-12-20T04:24:57.632 INFO:teuthology.orchestra.run.smithi032.stdout:Get:42 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webob all 1:1.8.6-1.1 [85.2 kB] 2023-12-20T04:24:57.670 INFO:teuthology.orchestra.run.smithi032.stdout:Get:43 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-waitress all 1.4.4-1.1ubuntu1 [47.3 kB] 2023-12-20T04:24:57.684 INFO:teuthology.orchestra.run.smithi032.stdout:Get:44 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-tempita all 0.5.2-6ubuntu1 [15.1 kB] 2023-12-20T04:24:57.689 INFO:teuthology.orchestra.run.smithi032.stdout:Get:45 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-paste all 3.5.0+dfsg1-1 [456 kB] 2023-12-20T04:24:57.879 INFO:teuthology.orchestra.run.smithi039.stdout:Get:95 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.1-132-g568df92b-1jammy [237 kB] 2023-12-20T04:24:57.889 INFO:teuthology.orchestra.run.smithi039.stdout:Get:96 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.1-132-g568df92b-1jammy [130 kB] 2023-12-20T04:24:57.895 INFO:teuthology.orchestra.run.smithi039.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.1-132-g568df92b-1jammy [1,484 kB] 2023-12-20T04:24:57.942 INFO:teuthology.orchestra.run.smithi032.stdout:Get:46 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-pastedeploy-tpl all 2.1.1-1 [4,892 B] 2023-12-20T04:24:57.968 INFO:teuthology.orchestra.run.smithi039.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.1-132-g568df92b-1jammy [9,877 kB] 2023-12-20T04:24:58.025 INFO:teuthology.orchestra.run.smithi032.stdout:Get:47 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastedeploy all 2.1.1-1 [26.6 kB] 2023-12-20T04:24:58.120 INFO:teuthology.orchestra.run.smithi032.stdout:Get:48 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-webtest all 2.0.35-1 [28.5 kB] 2023-12-20T04:24:58.191 INFO:teuthology.orchestra.run.smithi032.stdout:Get:49 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pecan all 1.3.3-4ubuntu2 [87.3 kB] 2023-12-20T04:24:58.298 INFO:teuthology.orchestra.run.smithi032.stdout:Get:50 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-certifi all 2020.6.20-1 [150 kB] 2023-12-20T04:24:58.382 INFO:teuthology.orchestra.run.smithi032.stdout:Get:51 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-idna all 3.3-1 [49.3 kB] 2023-12-20T04:24:58.400 INFO:teuthology.orchestra.run.smithi032.stdout:Get:52 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-urllib3 all 1.26.5-1~exp1ubuntu0.1 [98.2 kB] 2023-12-20T04:24:58.430 INFO:teuthology.orchestra.run.smithi032.stdout:Get:53 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-requests all 2.25.1+dfsg-2ubuntu0.1 [48.8 kB] 2023-12-20T04:24:58.442 INFO:teuthology.orchestra.run.smithi032.stdout:Get:54 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 python3-werkzeug all 2.0.2+dfsg1-1ubuntu0.22.04.1 [181 kB] 2023-12-20T04:24:58.479 INFO:teuthology.orchestra.run.smithi032.stdout:Get:55 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-openssl all 21.0.0-1 [45.2 kB] 2023-12-20T04:24:58.487 INFO:teuthology.orchestra.run.smithi032.stdout:Get:56 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-asyncssh all 2.5.0-1 [185 kB] 2023-12-20T04:24:58.494 INFO:teuthology.orchestra.run.smithi039.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.1-132-g568df92b-1jammy [31.5 MB] 2023-12-20T04:24:58.677 INFO:teuthology.orchestra.run.smithi032.stdout:Get:57 http://archive.ubuntu.com/ubuntu jammy/main amd64 python-babel-localedata all 2.8.0+dfsg.1-7 [4,982 kB] 2023-12-20T04:24:58.826 INFO:teuthology.orchestra.run.smithi032.stdout:Get:58 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-base amd64 18.2.1-132-g568df92b-1jammy [6,835 kB] 2023-12-20T04:24:59.444 INFO:teuthology.orchestra.run.smithi032.stdout:Get:59 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-modules-core all 18.2.1-132-g568df92b-1jammy [237 kB] 2023-12-20T04:24:59.463 INFO:teuthology.orchestra.run.smithi032.stdout:Get:60 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libsqlite3-mod-ceph amd64 18.2.1-132-g568df92b-1jammy [130 kB] 2023-12-20T04:24:59.478 INFO:teuthology.orchestra.run.smithi032.stdout:Get:61 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr amd64 18.2.1-132-g568df92b-1jammy [1,484 kB] 2023-12-20T04:24:59.504 INFO:teuthology.orchestra.run.smithi032.stdout:Get:62 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-babel all 2.8.0+dfsg.1-7 [85.1 kB] 2023-12-20T04:24:59.505 INFO:teuthology.orchestra.run.smithi032.stdout:Get:63 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-jinja2 all 3.0.3-1 [108 kB] 2023-12-20T04:24:59.507 INFO:teuthology.orchestra.run.smithi032.stdout:Get:64 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-bcrypt amd64 3.2.0-1build1 [32.7 kB] 2023-12-20T04:24:59.507 INFO:teuthology.orchestra.run.smithi032.stdout:Get:65 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pluggy all 0.13.0-7.1 [19.0 kB] 2023-12-20T04:24:59.508 INFO:teuthology.orchestra.run.smithi032.stdout:Get:66 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-py all 1.10.0-1 [71.9 kB] 2023-12-20T04:24:59.509 INFO:teuthology.orchestra.run.smithi032.stdout:Get:67 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-iniconfig all 1.1.1-2 [6,024 B] 2023-12-20T04:24:59.509 INFO:teuthology.orchestra.run.smithi032.stdout:Get:68 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-toml all 0.10.2-1 [16.5 kB] 2023-12-20T04:24:59.510 INFO:teuthology.orchestra.run.smithi032.stdout:Get:69 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-pytest all 6.2.5-1ubuntu2 [214 kB] 2023-12-20T04:24:59.513 INFO:teuthology.orchestra.run.smithi032.stdout:Get:70 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-repoze.lru all 0.7-2 [12.1 kB] 2023-12-20T04:24:59.587 INFO:teuthology.orchestra.run.smithi032.stdout:Get:71 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-routes all 2.5.1-1ubuntu1 [89.0 kB] 2023-12-20T04:24:59.633 INFO:teuthology.orchestra.run.smithi032.stdout:Get:72 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mon amd64 18.2.1-132-g568df92b-1jammy [9,877 kB] 2023-12-20T04:24:59.751 INFO:teuthology.orchestra.run.smithi032.stdout:Get:73 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn-lib amd64 0.23.2-5ubuntu6 [2,058 kB] 2023-12-20T04:25:00.129 INFO:teuthology.orchestra.run.smithi039.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.1-132-g568df92b-1jammy [14.1 kB] 2023-12-20T04:25:00.129 INFO:teuthology.orchestra.run.smithi039.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.1-132-g568df92b-1jammy [824 kB] 2023-12-20T04:25:00.164 INFO:teuthology.orchestra.run.smithi039.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.1-132-g568df92b-1jammy [2,173 kB] 2023-12-20T04:25:00.278 INFO:teuthology.orchestra.run.smithi039.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.1-132-g568df92b-1jammy [235 kB] 2023-12-20T04:25:00.289 INFO:teuthology.orchestra.run.smithi039.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.1-132-g568df92b-1jammy [138 kB] 2023-12-20T04:25:00.296 INFO:teuthology.orchestra.run.smithi039.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.1-132-g568df92b-1jammy [1,898 kB] 2023-12-20T04:25:00.396 INFO:teuthology.orchestra.run.smithi039.stdout:Get:106 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.1-132-g568df92b-1jammy [8,625 kB] 2023-12-20T04:25:00.472 INFO:teuthology.orchestra.run.smithi032.stdout:Get:74 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-joblib all 0.17.0-4ubuntu1 [204 kB] 2023-12-20T04:25:00.477 INFO:teuthology.orchestra.run.smithi032.stdout:Get:75 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-threadpoolctl all 3.1.0-1 [21.3 kB] 2023-12-20T04:25:00.478 INFO:teuthology.orchestra.run.smithi032.stdout:Get:76 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-sklearn all 0.23.2-5ubuntu6 [1,829 kB] 2023-12-20T04:25:00.498 INFO:teuthology.orchestra.run.smithi032.stdout:Get:77 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-osd amd64 18.2.1-132-g568df92b-1jammy [31.5 MB] 2023-12-20T04:25:00.554 INFO:teuthology.orchestra.run.smithi032.stdout:Get:78 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-cachetools all 5.0.0-1 [9,722 B] 2023-12-20T04:25:00.554 INFO:teuthology.orchestra.run.smithi032.stdout:Get:79 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1 all 0.4.8-1 [50.9 kB] 2023-12-20T04:25:00.555 INFO:teuthology.orchestra.run.smithi032.stdout:Get:80 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyasn1-modules all 0.2.1-1 [33.0 kB] 2023-12-20T04:25:00.556 INFO:teuthology.orchestra.run.smithi032.stdout:Get:81 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-rsa all 4.8-1 [28.4 kB] 2023-12-20T04:25:00.556 INFO:teuthology.orchestra.run.smithi032.stdout:Get:82 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-google-auth all 1.5.1-3 [35.7 kB] 2023-12-20T04:25:00.557 INFO:teuthology.orchestra.run.smithi032.stdout:Get:83 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-requests-oauthlib all 1.3.0+ds-0.1 [18.7 kB] 2023-12-20T04:25:00.558 INFO:teuthology.orchestra.run.smithi032.stdout:Get:84 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-websocket all 1.2.3-1 [34.7 kB] 2023-12-20T04:25:00.722 INFO:teuthology.orchestra.run.smithi032.stdout:Get:85 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python3-kubernetes all 12.0.1-1ubuntu1 [353 kB] 2023-12-20T04:25:00.862 INFO:teuthology.orchestra.run.smithi039.stdout:Get:107 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.1-132-g568df92b-1jammy [14.2 kB] 2023-12-20T04:25:00.862 INFO:teuthology.orchestra.run.smithi039.stdout:Get:108 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.1-132-g568df92b-1jammy [46.4 MB] 2023-12-20T04:25:01.231 INFO:teuthology.orchestra.run.smithi032.stdout:Get:86 http://archive.ubuntu.com/ubuntu jammy/main amd64 libonig5 amd64 6.9.7.1-2build1 [172 kB] 2023-12-20T04:25:01.260 INFO:teuthology.orchestra.run.smithi032.stdout:Get:87 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjq1 amd64 1.6-2.1ubuntu3 [133 kB] 2023-12-20T04:25:01.282 INFO:teuthology.orchestra.run.smithi032.stdout:Get:88 http://archive.ubuntu.com/ubuntu jammy/main amd64 jq amd64 1.6-2.1ubuntu3 [52.5 kB] 2023-12-20T04:25:01.291 INFO:teuthology.orchestra.run.smithi032.stdout:Get:89 http://archive.ubuntu.com/ubuntu jammy/main amd64 socat amd64 1.7.4.1-3ubuntu4 [349 kB] 2023-12-20T04:25:01.368 INFO:teuthology.orchestra.run.smithi032.stdout:Get:90 http://archive.ubuntu.com/ubuntu jammy/universe amd64 xmlstarlet amd64 1.6.1-2.1 [265 kB] 2023-12-20T04:25:01.401 INFO:teuthology.orchestra.run.smithi032.stdout:Get:91 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-colorama all 0.4.4-1 [24.5 kB] 2023-12-20T04:25:01.403 INFO:teuthology.orchestra.run.smithi032.stdout:Get:92 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 nvme-cli amd64 1.16-3ubuntu0.1 [474 kB] 2023-12-20T04:25:01.439 INFO:teuthology.orchestra.run.smithi032.stdout:Get:93 http://archive.ubuntu.com/ubuntu jammy/universe amd64 python-asyncssh-doc all 2.5.0-1 [318 kB] 2023-12-20T04:25:01.462 INFO:teuthology.orchestra.run.smithi032.stdout:Get:94 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pastescript all 2.0.2-4 [54.6 kB] 2023-12-20T04:25:01.466 INFO:teuthology.orchestra.run.smithi032.stdout:Get:95 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-psutil amd64 5.9.0-1build1 [158 kB] 2023-12-20T04:25:01.641 INFO:teuthology.orchestra.run.smithi032.stdout:Get:96 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pygments all 2.11.2+dfsg-2 [750 kB] 2023-12-20T04:25:03.268 INFO:teuthology.orchestra.run.smithi032.stdout:Get:97 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph amd64 18.2.1-132-g568df92b-1jammy [14.1 kB] 2023-12-20T04:25:03.269 INFO:teuthology.orchestra.run.smithi032.stdout:Get:98 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-fuse amd64 18.2.1-132-g568df92b-1jammy [824 kB] 2023-12-20T04:25:03.303 INFO:teuthology.orchestra.run.smithi039.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.1-132-g568df92b-1jammy [121 kB] 2023-12-20T04:25:03.304 INFO:teuthology.orchestra.run.smithi039.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.1-132-g568df92b-1jammy [228 kB] 2023-12-20T04:25:03.306 INFO:teuthology.orchestra.run.smithi039.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.1-132-g568df92b-1jammy [34.0 kB] 2023-12-20T04:25:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.1-132-g568df92b-1jammy [23.3 kB] 2023-12-20T04:25:03.307 INFO:teuthology.orchestra.run.smithi039.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.1-132-g568df92b-1jammy [38.3 kB] 2023-12-20T04:25:03.309 INFO:teuthology.orchestra.run.smithi039.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.1-132-g568df92b-1jammy [11.6 MB] 2023-12-20T04:25:03.327 INFO:teuthology.orchestra.run.smithi032.stdout:Get:99 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mds amd64 18.2.1-132-g568df92b-1jammy [2,173 kB] 2023-12-20T04:25:03.491 INFO:teuthology.orchestra.run.smithi032.stdout:Get:100 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephadm amd64 18.2.1-132-g568df92b-1jammy [235 kB] 2023-12-20T04:25:03.508 INFO:teuthology.orchestra.run.smithi032.stdout:Get:101 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-cephadm all 18.2.1-132-g568df92b-1jammy [138 kB] 2023-12-20T04:25:03.519 INFO:teuthology.orchestra.run.smithi032.stdout:Get:102 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-dashboard all 18.2.1-132-g568df92b-1jammy [1,898 kB] 2023-12-20T04:25:03.667 INFO:teuthology.orchestra.run.smithi032.stdout:Get:103 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-diskprediction-local all 18.2.1-132-g568df92b-1jammy [8,625 kB] 2023-12-20T04:25:03.978 INFO:teuthology.orchestra.run.smithi039.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.1-132-g568df92b-1jammy [92.9 kB] 2023-12-20T04:25:04.172 INFO:teuthology.orchestra.run.smithi032.stdout:Get:104 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-mgr-k8sevents all 18.2.1-132-g568df92b-1jammy [14.2 kB] 2023-12-20T04:25:04.172 INFO:teuthology.orchestra.run.smithi032.stdout:Get:105 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-test amd64 18.2.1-132-g568df92b-1jammy [46.4 MB] 2023-12-20T04:25:04.566 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 181 MB in 9s (21.1 MB/s) 2023-12-20T04:25:04.585 INFO:teuthology.orchestra.run.smithi032.stdout:Get:106 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-pyinotify all 0.9.6-1.3 [24.8 kB] 2023-12-20T04:25:04.789 INFO:teuthology.orchestra.run.smithi032.stdout:Get:107 http://archive.ubuntu.com/ubuntu jammy/main amd64 python3-simplejson amd64 3.17.6-1build1 [54.7 kB] 2023-12-20T04:25:04.894 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pyperclip. 2023-12-20T04:25:04.966 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2023-12-20T04:25:04.970 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-12-20T04:25:05.002 INFO:teuthology.orchestra.run.smithi032.stdout:Get:108 http://archive.ubuntu.com/ubuntu jammy/universe amd64 qttranslations5-l10n all 5.15.3-1 [1,983 kB] 2023-12-20T04:25:05.076 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-12-20T04:25:05.370 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-wcwidth. 2023-12-20T04:25:05.384 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-12-20T04:25:05.420 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-20T04:25:05.731 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cmd2. 2023-12-20T04:25:05.745 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-12-20T04:25:05.789 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-12-20T04:25:05.847 INFO:teuthology.orchestra.run.smithi032.stdout:Get:109 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 ceph-volume all 18.2.1-132-g568df92b-1jammy [121 kB] 2023-12-20T04:25:05.848 INFO:teuthology.orchestra.run.smithi032.stdout:Get:110 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-mirror amd64 18.2.1-132-g568df92b-1jammy [228 kB] 2023-12-20T04:25:05.850 INFO:teuthology.orchestra.run.smithi032.stdout:Get:111 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-shell all 18.2.1-132-g568df92b-1jammy [34.0 kB] 2023-12-20T04:25:05.881 INFO:teuthology.orchestra.run.smithi032.stdout:Get:112 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 cephfs-top all 18.2.1-132-g568df92b-1jammy [23.3 kB] 2023-12-20T04:25:05.882 INFO:teuthology.orchestra.run.smithi032.stdout:Get:113 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 libcephfs-dev amd64 18.2.1-132-g568df92b-1jammy [38.3 kB] 2023-12-20T04:25:05.882 INFO:teuthology.orchestra.run.smithi032.stdout:Get:114 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 radosgw amd64 18.2.1-132-g568df92b-1jammy [11.6 MB] 2023-12-20T04:25:06.076 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libfmt8:amd64. 2023-12-20T04:25:06.090 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-12-20T04:25:06.159 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-12-20T04:25:06.254 INFO:teuthology.orchestra.run.smithi032.stdout:Get:115 https://3.chacra.ceph.com/r/ceph/wip-vshankar-testing1-2023-12-18-1207-reef-2/568df92b7e50b2108bce4c77c393ff294d6a5e5e/ubuntu/jammy/flavors/default jammy/main amd64 rbd-fuse amd64 18.2.1-132-g568df92b-1jammy [92.9 kB] 2023-12-20T04:25:06.554 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-12-20T04:25:06.569 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-12-20T04:25:06.612 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-20T04:25:06.856 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 181 MB in 11s (16.5 MB/s) 2023-12-20T04:25:06.949 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-12-20T04:25:06.964 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-12-20T04:25:07.007 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-20T04:25:07.033 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyperclip. 2023-12-20T04:25:07.103 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 132930 files and directories currently installed.) 2023-12-20T04:25:07.106 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../000-python3-pyperclip_1.8.2-2_all.deb ... 2023-12-20T04:25:07.157 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyperclip (1.8.2-2) ... 2023-12-20T04:25:07.343 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-12-20T04:25:07.359 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-12-20T04:25:07.427 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-20T04:25:07.526 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-wcwidth. 2023-12-20T04:25:07.540 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../001-python3-wcwidth_0.2.5+dfsg1-1_all.deb ... 2023-12-20T04:25:07.593 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-20T04:25:07.838 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-12-20T04:25:07.854 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-20T04:25:07.905 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:25:08.004 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cmd2. 2023-12-20T04:25:08.019 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../002-python3-cmd2_2.3.3-1_all.deb ... 2023-12-20T04:25:08.054 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cmd2 (2.3.3-1) ... 2023-12-20T04:25:08.306 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfmt8:amd64. 2023-12-20T04:25:08.308 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-12-20T04:25:08.321 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../003-libfmt8_8.1.1+ds1-2_amd64.deb ... 2023-12-20T04:25:08.323 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-20T04:25:08.366 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:25:08.407 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfmt8:amd64 (8.1.1+ds1-2) ... 2023-12-20T04:25:08.727 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-12-20T04:25:08.734 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblttng-ust1:amd64. 2023-12-20T04:25:08.743 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-20T04:25:08.750 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../004-liblttng-ust1_2.13.1-1ubuntu1_amd64.deb ... 2023-12-20T04:25:08.778 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:25:08.793 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-20T04:25:09.105 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-12-20T04:25:09.121 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-12-20T04:25:09.163 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-20T04:25:09.238 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libdouble-conversion3:amd64. 2023-12-20T04:25:09.253 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../005-libdouble-conversion3_3.1.7-4_amd64.deb ... 2023-12-20T04:25:09.304 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-20T04:25:09.475 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librados2. 2023-12-20T04:25:09.491 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../011-librados2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:09.533 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librados2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:09.690 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libpcre2-16-0:amd64. 2023-12-20T04:25:09.706 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../006-libpcre2-16-0_10.39-3ubuntu0.1_amd64.deb ... 2023-12-20T04:25:09.757 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-20T04:25:09.969 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librbd1. 2023-12-20T04:25:09.985 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../012-librbd1_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:10.027 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librbd1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:10.414 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libcephfs2. 2023-12-20T04:25:10.429 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../013-libcephfs2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:10.464 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libcephfs2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:10.644 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5core5a:amd64. 2023-12-20T04:25:10.660 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../007-libqt5core5a_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-20T04:25:10.719 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:25:10.741 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rados. 2023-12-20T04:25:10.757 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../014-python3-rados_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:10.808 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rados (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:11.127 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-12-20T04:25:11.143 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:11.185 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-ceph-argparse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:11.205 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5dbus5:amd64. 2023-12-20T04:25:11.252 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../008-libqt5dbus5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-20T04:25:11.263 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:25:11.437 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cephfs. 2023-12-20T04:25:11.453 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../016-python3-cephfs_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:11.496 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cephfs (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:11.692 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqt5network5:amd64. 2023-12-20T04:25:11.707 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../009-libqt5network5_5.15.3+dfsg-2ubuntu0.2_amd64.deb ... 2023-12-20T04:25:11.758 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:25:11.807 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-ceph-common. 2023-12-20T04:25:11.823 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:11.857 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:12.153 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libthrift-0.16.0:amd64. 2023-12-20T04:25:12.168 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../010-libthrift-0.16.0_0.16.0-2_amd64.deb ... 2023-12-20T04:25:12.185 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-prettytable. 2023-12-20T04:25:12.201 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-12-20T04:25:12.219 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-20T04:25:12.243 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-12-20T04:25:12.504 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rbd. 2023-12-20T04:25:12.519 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../019-python3-rbd_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:12.562 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rbd (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:12.664 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librados2. 2023-12-20T04:25:12.679 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../011-librados2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:12.730 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librados2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:12.923 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-12-20T04:25:12.939 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-12-20T04:25:12.981 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-20T04:25:13.183 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librbd1. 2023-12-20T04:25:13.198 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../012-librbd1_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:13.241 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librbd1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:13.284 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package librgw2. 2023-12-20T04:25:13.300 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../021-librgw2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:13.334 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking librgw2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:13.727 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs2. 2023-12-20T04:25:13.743 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../013-libcephfs2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:13.753 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rgw. 2023-12-20T04:25:13.769 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../022-python3-rgw_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:13.810 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:13.811 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:14.130 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rados. 2023-12-20T04:25:14.145 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../014-python3-rados_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:14.164 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package liboath0:amd64. 2023-12-20T04:25:14.180 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-12-20T04:25:14.196 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rados (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:14.222 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-12-20T04:25:14.534 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libradosstriper1. 2023-12-20T04:25:14.549 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../024-libradosstriper1_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:14.592 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libradosstriper1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:14.607 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-argparse. 2023-12-20T04:25:14.623 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../015-python3-ceph-argparse_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:14.682 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-argparse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:15.001 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cephfs. 2023-12-20T04:25:15.017 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../016-python3-cephfs_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:15.059 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cephfs (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:15.179 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-common. 2023-12-20T04:25:15.194 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../025-ceph-common_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:15.237 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:15.446 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-ceph-common. 2023-12-20T04:25:15.461 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../017-python3-ceph-common_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:15.512 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:15.881 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-prettytable. 2023-12-20T04:25:15.897 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../018-python3-prettytable_2.5.0-2_all.deb ... 2023-12-20T04:25:15.948 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-prettytable (2.5.0-2) ... 2023-12-20T04:25:16.251 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rbd. 2023-12-20T04:25:16.266 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../019-python3-rbd_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:16.318 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rbd (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:16.334 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-base. 2023-12-20T04:25:16.349 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../026-ceph-base_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:16.402 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-base (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:16.737 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librdkafka1:amd64. 2023-12-20T04:25:16.752 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../020-librdkafka1_1.8.0-1build1_amd64.deb ... 2023-12-20T04:25:16.803 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-20T04:25:17.080 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-12-20T04:25:17.094 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-12-20T04:25:17.115 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package librgw2. 2023-12-20T04:25:17.131 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../021-librgw2_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:17.138 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-12-20T04:25:17.172 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking librgw2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:17.449 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cheroot. 2023-12-20T04:25:17.465 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-12-20T04:25:17.508 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-20T04:25:17.667 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rgw. 2023-12-20T04:25:17.682 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../022-python3-rgw_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:17.725 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:17.794 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-12-20T04:25:17.809 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-12-20T04:25:17.844 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-12-20T04:25:18.061 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liboath0:amd64. 2023-12-20T04:25:18.077 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../023-liboath0_2.6.7-3build1_amd64.deb ... 2023-12-20T04:25:18.119 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liboath0:amd64 (2.6.7-3build1) ... 2023-12-20T04:25:18.130 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.text. 2023-12-20T04:25:18.145 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-12-20T04:25:18.188 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-12-20T04:25:18.380 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libradosstriper1. 2023-12-20T04:25:18.396 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../024-libradosstriper1_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:18.430 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libradosstriper1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:18.516 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-12-20T04:25:18.531 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-12-20T04:25:18.584 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-12-20T04:25:18.833 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-common. 2023-12-20T04:25:18.849 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../025-ceph-common_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:18.861 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-tempora. 2023-12-20T04:25:18.876 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-12-20T04:25:18.891 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:18.911 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-12-20T04:25:19.138 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-portend. 2023-12-20T04:25:19.154 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-12-20T04:25:19.196 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-12-20T04:25:19.499 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-12-20T04:25:19.515 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-12-20T04:25:19.574 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-12-20T04:25:19.877 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cherrypy3. 2023-12-20T04:25:19.893 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-12-20T04:25:19.936 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-12-20T04:25:19.997 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-base. 2023-12-20T04:25:20.013 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../026-ceph-base_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:20.065 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-base (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:20.422 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-natsort. 2023-12-20T04:25:20.438 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-12-20T04:25:20.480 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-12-20T04:25:20.809 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-logutils. 2023-12-20T04:25:20.824 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-12-20T04:25:20.867 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-12-20T04:25:20.926 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.functools. 2023-12-20T04:25:20.941 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../027-python3-jaraco.functools_3.4.0-2_all.deb ... 2023-12-20T04:25:20.984 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.functools (3.4.0-2) ... 2023-12-20T04:25:21.170 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-markupsafe. 2023-12-20T04:25:21.185 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-12-20T04:25:21.228 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-12-20T04:25:21.262 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cheroot. 2023-12-20T04:25:21.277 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../028-python3-cheroot_8.5.2+ds1-1ubuntu3.1_all.deb ... 2023-12-20T04:25:21.312 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-20T04:25:21.514 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-mako. 2023-12-20T04:25:21.530 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-12-20T04:25:21.564 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-20T04:25:21.581 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.classes. 2023-12-20T04:25:21.597 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../029-python3-jaraco.classes_3.2.1-3_all.deb ... 2023-12-20T04:25:21.631 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.classes (3.2.1-3) ... 2023-12-20T04:25:21.876 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-simplegeneric. 2023-12-20T04:25:21.892 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-12-20T04:25:21.917 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.text. 2023-12-20T04:25:21.926 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-12-20T04:25:21.932 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../030-python3-jaraco.text_3.6.0-2_all.deb ... 2023-12-20T04:25:21.975 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.text (3.6.0-2) ... 2023-12-20T04:25:22.228 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-singledispatch. 2023-12-20T04:25:22.244 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-12-20T04:25:22.287 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-12-20T04:25:22.311 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jaraco.collections. 2023-12-20T04:25:22.327 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../031-python3-jaraco.collections_3.4.0-2_all.deb ... 2023-12-20T04:25:22.369 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jaraco.collections (3.4.0-2) ... 2023-12-20T04:25:22.564 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-webob. 2023-12-20T04:25:22.580 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-12-20T04:25:22.614 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-12-20T04:25:22.647 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempora. 2023-12-20T04:25:22.663 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../032-python3-tempora_4.1.2-1_all.deb ... 2023-12-20T04:25:22.697 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempora (4.1.2-1) ... 2023-12-20T04:25:22.951 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-waitress. 2023-12-20T04:25:22.966 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-12-20T04:25:22.983 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-portend. 2023-12-20T04:25:22.999 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../033-python3-portend_3.0.0-1_all.deb ... 2023-12-20T04:25:23.004 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-20T04:25:23.041 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-portend (3.0.0-1) ... 2023-12-20T04:25:23.278 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-tempita. 2023-12-20T04:25:23.294 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-12-20T04:25:23.327 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-zc.lockfile. 2023-12-20T04:25:23.328 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-12-20T04:25:23.343 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../034-python3-zc.lockfile_2.0-1_all.deb ... 2023-12-20T04:25:23.386 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-zc.lockfile (2.0-1) ... 2023-12-20T04:25:23.615 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-paste. 2023-12-20T04:25:23.630 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-12-20T04:25:23.673 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-12-20T04:25:23.713 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cherrypy3. 2023-12-20T04:25:23.729 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../035-python3-cherrypy3_18.6.1-4_all.deb ... 2023-12-20T04:25:23.763 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cherrypy3 (18.6.1-4) ... 2023-12-20T04:25:23.984 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-12-20T04:25:24.000 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-12-20T04:25:24.034 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-12-20T04:25:24.110 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-natsort. 2023-12-20T04:25:24.126 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../036-python3-natsort_8.0.2-1_all.deb ... 2023-12-20T04:25:24.168 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-natsort (8.0.2-1) ... 2023-12-20T04:25:24.287 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pastedeploy. 2023-12-20T04:25:24.303 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-12-20T04:25:24.345 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-12-20T04:25:24.480 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-logutils. 2023-12-20T04:25:24.495 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../037-python3-logutils_0.3.3-8_all.deb ... 2023-12-20T04:25:24.538 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-logutils (0.3.3-8) ... 2023-12-20T04:25:24.657 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-webtest. 2023-12-20T04:25:24.672 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-12-20T04:25:24.715 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-12-20T04:25:24.832 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-markupsafe. 2023-12-20T04:25:24.847 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../038-python3-markupsafe_2.0.1-2build1_amd64.deb ... 2023-12-20T04:25:24.882 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-markupsafe (2.0.1-2build1) ... 2023-12-20T04:25:25.062 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pecan. 2023-12-20T04:25:25.077 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-12-20T04:25:25.126 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-12-20T04:25:25.176 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-mako. 2023-12-20T04:25:25.192 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../039-python3-mako_1.1.3+ds1-2ubuntu0.1_all.deb ... 2023-12-20T04:25:25.235 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-20T04:25:25.571 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplegeneric. 2023-12-20T04:25:25.587 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../040-python3-simplegeneric_0.8.1-3_all.deb ... 2023-12-20T04:25:25.629 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplegeneric (0.8.1-3) ... 2023-12-20T04:25:25.974 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-singledispatch. 2023-12-20T04:25:25.989 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../041-python3-singledispatch_3.4.0.3-3_all.deb ... 2023-12-20T04:25:26.023 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-singledispatch (3.4.0.3-3) ... 2023-12-20T04:25:26.214 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-certifi. 2023-12-20T04:25:26.230 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-12-20T04:25:26.264 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-12-20T04:25:26.326 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webob. 2023-12-20T04:25:26.342 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../042-python3-webob_1%3a1.8.6-1.1_all.deb ... 2023-12-20T04:25:26.384 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webob (1:1.8.6-1.1) ... 2023-12-20T04:25:26.525 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-idna. 2023-12-20T04:25:26.540 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-12-20T04:25:26.592 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-idna (3.3-1) ... 2023-12-20T04:25:26.712 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-waitress. 2023-12-20T04:25:26.728 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../043-python3-waitress_1.4.4-1.1ubuntu1_all.deb ... 2023-12-20T04:25:26.765 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-20T04:25:26.895 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-urllib3. 2023-12-20T04:25:26.910 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-12-20T04:25:26.953 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-20T04:25:27.048 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-tempita. 2023-12-20T04:25:27.064 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../044-python3-tempita_0.5.2-6ubuntu1_all.deb ... 2023-12-20T04:25:27.098 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-tempita (0.5.2-6ubuntu1) ... 2023-12-20T04:25:27.298 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-requests. 2023-12-20T04:25:27.313 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-12-20T04:25:27.356 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-20T04:25:27.434 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-paste. 2023-12-20T04:25:27.450 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../045-python3-paste_3.5.0+dfsg1-1_all.deb ... 2023-12-20T04:25:27.484 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-paste (3.5.0+dfsg1-1) ... 2023-12-20T04:25:27.667 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-werkzeug. 2023-12-20T04:25:27.682 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-12-20T04:25:27.734 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-20T04:25:27.805 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-pastedeploy-tpl. 2023-12-20T04:25:27.819 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../046-python-pastedeploy-tpl_2.1.1-1_all.deb ... 2023-12-20T04:25:27.862 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-pastedeploy-tpl (2.1.1-1) ... 2023-12-20T04:25:28.037 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-12-20T04:25:28.053 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:28.095 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-modules-core (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:28.156 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastedeploy. 2023-12-20T04:25:28.172 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../047-python3-pastedeploy_2.1.1-1_all.deb ... 2023-12-20T04:25:28.206 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastedeploy (2.1.1-1) ... 2023-12-20T04:25:28.500 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-12-20T04:25:28.515 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:28.558 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libsqlite3-mod-ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:28.877 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-openssl. 2023-12-20T04:25:28.893 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-12-20T04:25:28.935 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-12-20T04:25:29.084 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-webtest. 2023-12-20T04:25:29.100 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../048-python3-webtest_2.0.35-1_all.deb ... 2023-12-20T04:25:29.142 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-webtest (2.0.35-1) ... 2023-12-20T04:25:29.238 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr. 2023-12-20T04:25:29.255 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../058-ceph-mgr_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:29.288 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:29.462 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pecan. 2023-12-20T04:25:29.478 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../049-python3-pecan_1.3.3-4ubuntu2_all.deb ... 2023-12-20T04:25:29.529 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pecan (1.3.3-4ubuntu2) ... 2023-12-20T04:25:29.675 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mon. 2023-12-20T04:25:29.691 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../059-ceph-mon_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:29.733 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mon (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:29.900 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-certifi. 2023-12-20T04:25:29.916 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../050-python3-certifi_2020.6.20-1_all.deb ... 2023-12-20T04:25:29.959 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-certifi (2020.6.20-1) ... 2023-12-20T04:25:30.294 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-idna. 2023-12-20T04:25:30.310 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../051-python3-idna_3.3-1_all.deb ... 2023-12-20T04:25:30.352 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-idna (3.3-1) ... 2023-12-20T04:25:30.512 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-osd. 2023-12-20T04:25:30.527 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../060-ceph-osd_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:30.595 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-osd (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:30.663 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-urllib3. 2023-12-20T04:25:30.679 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../052-python3-urllib3_1.26.5-1~exp1ubuntu0.1_all.deb ... 2023-12-20T04:25:30.713 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-20T04:25:31.233 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests. 2023-12-20T04:25:31.249 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../053-python3-requests_2.25.1+dfsg-2ubuntu0.1_all.deb ... 2023-12-20T04:25:31.283 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-20T04:25:31.553 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-werkzeug. 2023-12-20T04:25:31.568 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../054-python3-werkzeug_2.0.2+dfsg1-1ubuntu0.22.04.1_all.deb ... 2023-12-20T04:25:31.602 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-20T04:25:31.790 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph. 2023-12-20T04:25:31.805 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../061-ceph_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:31.840 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:31.897 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-modules-core. 2023-12-20T04:25:31.912 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../055-ceph-mgr-modules-core_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:31.955 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-modules-core (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:32.126 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-fuse. 2023-12-20T04:25:32.142 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../062-ceph-fuse_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:32.184 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:32.351 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libsqlite3-mod-ceph. 2023-12-20T04:25:32.367 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../056-libsqlite3-mod-ceph_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:32.410 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libsqlite3-mod-ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:32.546 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mds. 2023-12-20T04:25:32.561 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../063-ceph-mds_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:32.604 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mds (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:32.720 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-openssl. 2023-12-20T04:25:32.736 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../057-python3-openssl_21.0.0-1_all.deb ... 2023-12-20T04:25:32.779 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-openssl (21.0.0-1) ... 2023-12-20T04:25:33.007 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package cephadm. 2023-12-20T04:25:33.022 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../064-cephadm_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:33.065 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:33.115 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr. 2023-12-20T04:25:33.131 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../058-ceph-mgr_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:33.173 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:33.327 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-asyncssh. 2023-12-20T04:25:33.342 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-12-20T04:25:33.384 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-12-20T04:25:33.526 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mon. 2023-12-20T04:25:33.542 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../059-ceph-mon_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:33.584 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mon (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:33.679 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python-babel-localedata. 2023-12-20T04:25:33.695 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-12-20T04:25:33.729 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-20T04:25:34.505 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-osd. 2023-12-20T04:25:34.520 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../060-ceph-osd_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:34.563 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-osd (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:34.734 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-babel. 2023-12-20T04:25:34.748 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-12-20T04:25:34.801 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-12-20T04:25:35.129 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-jinja2. 2023-12-20T04:25:35.143 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-12-20T04:25:35.178 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-12-20T04:25:35.490 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-12-20T04:25:35.505 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:35.540 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:35.983 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph. 2023-12-20T04:25:35.998 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../061-ceph_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:36.021 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-bcrypt. 2023-12-20T04:25:36.034 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-12-20T04:25:36.041 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:36.068 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-12-20T04:25:36.318 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-fuse. 2023-12-20T04:25:36.334 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../062-ceph-fuse_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:36.371 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pluggy. 2023-12-20T04:25:36.377 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:36.387 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2023-12-20T04:25:36.429 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-12-20T04:25:36.730 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mds. 2023-12-20T04:25:36.746 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../063-ceph-mds_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:36.749 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-py. 2023-12-20T04:25:36.765 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2023-12-20T04:25:36.788 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mds (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:36.807 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-py (1.10.0-1) ... 2023-12-20T04:25:37.128 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-iniconfig. 2023-12-20T04:25:37.143 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2023-12-20T04:25:37.186 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-12-20T04:25:37.190 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephadm. 2023-12-20T04:25:37.206 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../064-cephadm_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:37.249 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:37.488 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-toml. 2023-12-20T04:25:37.504 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2023-12-20T04:25:37.518 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-asyncssh. 2023-12-20T04:25:37.534 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../065-python3-asyncssh_2.5.0-1_all.deb ... 2023-12-20T04:25:37.538 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-12-20T04:25:37.568 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-asyncssh (2.5.0-1) ... 2023-12-20T04:25:37.791 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pytest. 2023-12-20T04:25:37.807 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-12-20T04:25:37.841 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-12-20T04:25:37.871 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-babel-localedata. 2023-12-20T04:25:37.887 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../066-python-babel-localedata_2.8.0+dfsg.1-7_all.deb ... 2023-12-20T04:25:37.929 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-20T04:25:38.187 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-repoze.lru. 2023-12-20T04:25:38.202 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2023-12-20T04:25:38.236 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-12-20T04:25:38.497 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-routes. 2023-12-20T04:25:38.513 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-12-20T04:25:38.547 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-12-20T04:25:38.817 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-12-20T04:25:38.833 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:38.867 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-dashboard (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:38.901 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-babel. 2023-12-20T04:25:38.915 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../067-python3-babel_2.8.0+dfsg.1-7_all.deb ... 2023-12-20T04:25:38.958 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-babel (2.8.0+dfsg.1-7) ... 2023-12-20T04:25:39.270 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-jinja2. 2023-12-20T04:25:39.285 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../068-python3-jinja2_3.0.3-1_all.deb ... 2023-12-20T04:25:39.304 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-12-20T04:25:39.320 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-12-20T04:25:39.345 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-jinja2 (3.0.3-1) ... 2023-12-20T04:25:39.362 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-20T04:25:39.673 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-cephadm. 2023-12-20T04:25:39.687 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../069-ceph-mgr-cephadm_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:39.731 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:39.874 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-joblib. 2023-12-20T04:25:39.890 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-12-20T04:25:39.932 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-12-20T04:25:40.068 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-bcrypt. 2023-12-20T04:25:40.082 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../070-python3-bcrypt_3.2.0-1build1_amd64.deb ... 2023-12-20T04:25:40.126 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-bcrypt (3.2.0-1build1) ... 2023-12-20T04:25:40.252 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-12-20T04:25:40.267 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-12-20T04:25:40.302 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-12-20T04:25:40.437 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pluggy. 2023-12-20T04:25:40.452 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../071-python3-pluggy_0.13.0-7.1_all.deb ... 2023-12-20T04:25:40.542 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pluggy (0.13.0-7.1) ... 2023-12-20T04:25:40.588 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-sklearn. 2023-12-20T04:25:40.604 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-12-20T04:25:40.646 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-20T04:25:40.857 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-py. 2023-12-20T04:25:40.873 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../072-python3-py_1.10.0-1_all.deb ... 2023-12-20T04:25:40.907 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-py (1.10.0-1) ... 2023-12-20T04:25:41.352 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-iniconfig. 2023-12-20T04:25:41.368 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../073-python3-iniconfig_1.1.1-2_all.deb ... 2023-12-20T04:25:41.410 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-iniconfig (1.1.1-2) ... 2023-12-20T04:25:41.434 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-12-20T04:25:41.449 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:41.492 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:41.671 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-toml. 2023-12-20T04:25:41.687 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../074-python3-toml_0.10.2-1_all.deb ... 2023-12-20T04:25:41.765 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-toml (0.10.2-1) ... 2023-12-20T04:25:42.065 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pytest. 2023-12-20T04:25:42.081 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../075-python3-pytest_6.2.5-1ubuntu2_all.deb ... 2023-12-20T04:25:42.124 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pytest (6.2.5-1ubuntu2) ... 2023-12-20T04:25:42.452 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-repoze.lru. 2023-12-20T04:25:42.468 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../076-python3-repoze.lru_0.7-2_all.deb ... 2023-12-20T04:25:42.471 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-cachetools. 2023-12-20T04:25:42.487 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2023-12-20T04:25:42.527 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-repoze.lru (0.7-2) ... 2023-12-20T04:25:42.529 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-12-20T04:25:42.815 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pyasn1. 2023-12-20T04:25:42.830 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-routes. 2023-12-20T04:25:42.832 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2023-12-20T04:25:42.846 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../077-python3-routes_2.5.1-1ubuntu1_all.deb ... 2023-12-20T04:25:42.873 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-12-20T04:25:42.888 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-routes (2.5.1-1ubuntu1) ... 2023-12-20T04:25:43.202 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-12-20T04:25:43.219 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-12-20T04:25:43.260 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-12-20T04:25:43.266 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-dashboard. 2023-12-20T04:25:43.282 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../078-ceph-mgr-dashboard_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:43.324 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-dashboard (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:43.589 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-rsa. 2023-12-20T04:25:43.605 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2023-12-20T04:25:43.647 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-rsa (4.8-1) ... 2023-12-20T04:25:43.678 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn-lib:amd64. 2023-12-20T04:25:43.692 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../079-python3-sklearn-lib_0.23.2-5ubuntu6_amd64.deb ... 2023-12-20T04:25:43.728 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-20T04:25:43.951 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-google-auth. 2023-12-20T04:25:43.967 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2023-12-20T04:25:44.009 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-12-20T04:25:44.206 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-joblib. 2023-12-20T04:25:44.221 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../080-python3-joblib_0.17.0-4ubuntu1_all.deb ... 2023-12-20T04:25:44.264 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-joblib (0.17.0-4ubuntu1) ... 2023-12-20T04:25:44.362 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-12-20T04:25:44.378 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-12-20T04:25:44.421 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-20T04:25:44.600 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-threadpoolctl. 2023-12-20T04:25:44.615 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../081-python3-threadpoolctl_3.1.0-1_all.deb ... 2023-12-20T04:25:44.659 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-threadpoolctl (3.1.0-1) ... 2023-12-20T04:25:44.699 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-websocket. 2023-12-20T04:25:44.714 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2023-12-20T04:25:44.748 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-12-20T04:25:44.970 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-sklearn. 2023-12-20T04:25:44.986 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../082-python3-sklearn_0.23.2-5ubuntu6_all.deb ... 2023-12-20T04:25:45.028 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-20T04:25:45.035 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-kubernetes. 2023-12-20T04:25:45.052 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-12-20T04:25:45.093 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-20T04:25:45.758 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-12-20T04:25:45.771 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-diskprediction-local. 2023-12-20T04:25:45.772 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:45.786 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../083-ceph-mgr-diskprediction-local_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:45.815 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-mgr-k8sevents (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:45.829 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-diskprediction-local (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:46.052 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libonig5:amd64. 2023-12-20T04:25:46.067 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-12-20T04:25:46.102 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-20T04:25:46.463 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libjq1:amd64. 2023-12-20T04:25:46.478 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-12-20T04:25:46.522 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-20T04:25:46.858 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package jq. 2023-12-20T04:25:46.873 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-12-20T04:25:46.899 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-cachetools. 2023-12-20T04:25:46.915 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../084-python3-cachetools_5.0.0-1_all.deb ... 2023-12-20T04:25:46.916 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-12-20T04:25:46.957 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-cachetools (5.0.0-1) ... 2023-12-20T04:25:47.144 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package socat. 2023-12-20T04:25:47.159 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-12-20T04:25:47.194 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-12-20T04:25:47.218 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1. 2023-12-20T04:25:47.235 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../085-python3-pyasn1_0.4.8-1_all.deb ... 2023-12-20T04:25:47.276 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1 (0.4.8-1) ... 2023-12-20T04:25:47.514 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package xmlstarlet. 2023-12-20T04:25:47.531 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-12-20T04:25:47.572 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-12-20T04:25:47.580 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyasn1-modules. 2023-12-20T04:25:47.597 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../086-python3-pyasn1-modules_0.2.1-1_all.deb ... 2023-12-20T04:25:47.629 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyasn1-modules (0.2.1-1) ... 2023-12-20T04:25:47.885 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-test. 2023-12-20T04:25:47.900 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../098-ceph-test_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:47.933 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-rsa. 2023-12-20T04:25:47.934 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-test (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:47.949 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../087-python3-rsa_4.8-1_all.deb ... 2023-12-20T04:25:47.982 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-rsa (4.8-1) ... 2023-12-20T04:25:48.303 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-google-auth. 2023-12-20T04:25:48.319 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../088-python3-google-auth_1.5.1-3_all.deb ... 2023-12-20T04:25:48.361 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-google-auth (1.5.1-3) ... 2023-12-20T04:25:48.681 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-requests-oauthlib. 2023-12-20T04:25:48.697 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../089-python3-requests-oauthlib_1.3.0+ds-0.1_all.deb ... 2023-12-20T04:25:48.739 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-20T04:25:49.092 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-websocket. 2023-12-20T04:25:49.108 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../090-python3-websocket_1.2.3-1_all.deb ... 2023-12-20T04:25:49.150 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-websocket (1.2.3-1) ... 2023-12-20T04:25:49.470 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-kubernetes. 2023-12-20T04:25:49.486 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../091-python3-kubernetes_12.0.1-1ubuntu1_all.deb ... 2023-12-20T04:25:49.527 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-20T04:25:49.689 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ceph-volume. 2023-12-20T04:25:49.706 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../099-ceph-volume_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:49.747 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ceph-volume (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:50.127 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package cephfs-mirror. 2023-12-20T04:25:50.143 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:50.185 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking cephfs-mirror (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:50.241 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-mgr-k8sevents. 2023-12-20T04:25:50.256 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../092-ceph-mgr-k8sevents_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:50.324 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-mgr-k8sevents (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:50.505 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-colorama. 2023-12-20T04:25:50.522 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2023-12-20T04:25:50.563 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-12-20T04:25:50.644 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libonig5:amd64. 2023-12-20T04:25:50.658 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../093-libonig5_6.9.7.1-2build1_amd64.deb ... 2023-12-20T04:25:50.702 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-20T04:25:50.816 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package cephfs-shell. 2023-12-20T04:25:50.832 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../102-cephfs-shell_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:50.866 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking cephfs-shell (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:51.013 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjq1:amd64. 2023-12-20T04:25:51.029 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../094-libjq1_1.6-2.1ubuntu3_amd64.deb ... 2023-12-20T04:25:51.063 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-20T04:25:51.307 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package jq. 2023-12-20T04:25:51.323 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../095-jq_1.6-2.1ubuntu3_amd64.deb ... 2023-12-20T04:25:51.335 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package cephfs-top. 2023-12-20T04:25:51.352 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../103-cephfs-top_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:51.365 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking jq (1.6-2.1ubuntu3) ... 2023-12-20T04:25:51.394 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking cephfs-top (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:51.647 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libcephfs-dev. 2023-12-20T04:25:51.651 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package socat. 2023-12-20T04:25:51.663 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:51.667 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../096-socat_1.7.4.1-3ubuntu4_amd64.deb ... 2023-12-20T04:25:51.705 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libcephfs-dev (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:51.709 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking socat (1.7.4.1-3ubuntu4) ... 2023-12-20T04:25:52.013 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package xmlstarlet. 2023-12-20T04:25:52.029 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../097-xmlstarlet_1.6.1-2.1_amd64.deb ... 2023-12-20T04:25:52.071 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking xmlstarlet (1.6.1-2.1) ... 2023-12-20T04:25:52.100 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package nvme-cli. 2023-12-20T04:25:52.116 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-12-20T04:25:52.158 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-12-20T04:25:52.425 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-test. 2023-12-20T04:25:52.439 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../098-ceph-test_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:52.483 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-test (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:52.497 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-12-20T04:25:52.513 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-12-20T04:25:52.555 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-12-20T04:25:53.000 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pastescript. 2023-12-20T04:25:53.016 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2023-12-20T04:25:53.058 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-12-20T04:25:53.378 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-psutil. 2023-12-20T04:25:53.394 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-12-20T04:25:53.428 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-12-20T04:25:53.731 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pygments. 2023-12-20T04:25:53.748 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-12-20T04:25:53.789 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-12-20T04:25:54.252 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-pyinotify. 2023-12-20T04:25:54.267 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-12-20T04:25:54.310 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-12-20T04:25:54.396 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ceph-volume. 2023-12-20T04:25:54.411 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../099-ceph-volume_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:54.462 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ceph-volume (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:54.596 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package python3-simplejson. 2023-12-20T04:25:54.611 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-12-20T04:25:54.655 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-12-20T04:25:54.874 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-mirror. 2023-12-20T04:25:54.890 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../100-cephfs-mirror_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:54.916 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-12-20T04:25:54.932 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-12-20T04:25:54.940 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-mirror (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:54.966 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-12-20T04:25:55.327 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-colorama. 2023-12-20T04:25:55.343 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../101-python3-colorama_0.4.4-1_all.deb ... 2023-12-20T04:25:55.402 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-colorama (0.4.4-1) ... 2023-12-20T04:25:55.653 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package radosgw. 2023-12-20T04:25:55.668 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../113-radosgw_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:55.710 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking radosgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:55.730 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-shell. 2023-12-20T04:25:55.746 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../102-cephfs-shell_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:55.796 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-shell (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:56.065 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package cephfs-top. 2023-12-20T04:25:56.081 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../103-cephfs-top_18.2.1-132-g568df92b-1jammy_all.deb ... 2023-12-20T04:25:56.125 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking cephfs-top (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:56.359 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libcephfs-dev. 2023-12-20T04:25:56.376 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../104-libcephfs-dev_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:56.410 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libcephfs-dev (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:56.431 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package rbd-fuse. 2023-12-20T04:25:56.447 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../114-rbd-fuse_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:25:56.506 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking rbd-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:56.793 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-12-20T04:25:57.021 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package nvme-cli. 2023-12-20T04:25:57.037 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../105-nvme-cli_1.16-3ubuntu0.1_amd64.deb ... 2023-12-20T04:25:57.077 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-20T04:25:57.079 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking nvme-cli (1.16-3ubuntu0.1) ... 2023-12-20T04:25:57.203 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-12-20T04:25:57.400 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python-asyncssh-doc. 2023-12-20T04:25:57.416 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../106-python-asyncssh-doc_2.5.0-1_all.deb ... 2023-12-20T04:25:57.446 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-12-20T04:25:57.458 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python-asyncssh-doc (2.5.0-1) ... 2023-12-20T04:25:57.866 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-12-20T04:25:57.903 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pastescript. 2023-12-20T04:25:57.919 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../107-python3-pastescript_2.0.2-4_all.deb ... 2023-12-20T04:25:57.961 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pastescript (2.0.2-4) ... 2023-12-20T04:25:58.290 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-psutil. 2023-12-20T04:25:58.306 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../108-python3-psutil_5.9.0-1build1_amd64.deb ... 2023-12-20T04:25:58.339 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-psutil (5.9.0-1build1) ... 2023-12-20T04:25:58.489 INFO:teuthology.orchestra.run.smithi039.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-12-20T04:25:58.548 INFO:teuthology.orchestra.run.smithi039.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-12-20T04:25:58.598 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:25:58.659 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pygments. 2023-12-20T04:25:58.675 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../109-python3-pygments_2.11.2+dfsg-2_all.deb ... 2023-12-20T04:25:58.717 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pygments (2.11.2+dfsg-2) ... 2023-12-20T04:25:59.146 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-pyinotify. 2023-12-20T04:25:59.151 INFO:teuthology.orchestra.run.smithi039.stdout:Adding system user cephadm....done 2023-12-20T04:25:59.161 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../110-python3-pyinotify_0.9.6-1.3_all.deb ... 2023-12-20T04:25:59.196 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-pyinotify (0.9.6-1.3) ... 2023-12-20T04:25:59.321 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-20T04:25:59.523 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package python3-simplejson. 2023-12-20T04:25:59.540 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../111-python3-simplejson_3.17.6-1build1_amd64.deb ... 2023-12-20T04:25:59.581 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking python3-simplejson (3.17.6-1build1) ... 2023-12-20T04:25:59.606 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-12-20T04:25:59.851 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package qttranslations5-l10n. 2023-12-20T04:25:59.867 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../112-qttranslations5-l10n_5.15.3-1_all.deb ... 2023-12-20T04:25:59.873 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-12-20T04:25:59.908 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking qttranslations5-l10n (5.15.3-1) ... 2023-12-20T04:25:59.999 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-12-20T04:26:00.242 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-12-20T04:26:00.511 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-12-20T04:26:00.588 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package radosgw. 2023-12-20T04:26:00.603 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../113-radosgw_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:26:00.636 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-py (1.10.0-1) ... 2023-12-20T04:26:00.646 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking radosgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:00.946 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-12-20T04:26:01.190 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package rbd-fuse. 2023-12-20T04:26:01.205 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../114-rbd-fuse_18.2.1-132-g568df92b-1jammy_amd64.deb ... 2023-12-20T04:26:01.248 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking rbd-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:01.324 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-12-20T04:26:01.541 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-iniconfig (1.1.1-2) ... 2023-12-20T04:26:01.591 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-12-20T04:26:01.860 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-12-20T04:26:01.895 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libdouble-conversion3:amd64 (3.1.7-4) ... 2023-12-20T04:26:01.996 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up nvme-cli (1.16-3ubuntu0.1) ... 2023-12-20T04:26:02.152 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-12-20T04:26:02.236 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service -> /lib/systemd/system/nvmefc-boot-connections.service. 2023-12-20T04:26:02.421 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-12-20T04:26:02.653 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmf-autoconnect.service -> /lib/systemd/system/nvmf-autoconnect.service. 2023-12-20T04:26:02.688 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-12-20T04:26:02.956 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-ceph-argparse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:03.232 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-20T04:26:03.328 INFO:teuthology.orchestra.run.smithi032.stdout:nvmf-connect.target is a disabled or a static unit, not starting it. 2023-12-20T04:26:03.350 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-12-20T04:26:03.364 INFO:teuthology.orchestra.run.smithi032.stdout:Could not execute systemctl: at /usr/bin/deb-systemd-invoke line 142. 2023-12-20T04:26:03.414 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:03.593 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-12-20T04:26:03.844 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-20T04:26:03.969 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-12-20T04:26:04.044 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user cephadm....done 2023-12-20T04:26:04.321 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-20T04:26:04.430 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-12-20T04:26:04.548 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-waitress (1.4.4-1.1ubuntu1) ... 2023-12-20T04:26:04.707 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-12-20T04:26:04.891 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.classes (3.2.1-3) ... 2023-12-20T04:26:04.991 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-12-20T04:26:05.152 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-asyncssh-doc (2.5.0-1) ... 2023-12-20T04:26:05.277 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.functools (3.4.0-2) ... 2023-12-20T04:26:05.318 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-12-20T04:26:05.578 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-repoze.lru (0.7-2) ... 2023-12-20T04:26:05.846 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liboath0:amd64 (2.6.7-3build1) ... 2023-12-20T04:26:05.971 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-py (1.10.0-1) ... 2023-12-20T04:26:06.019 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-12-20T04:26:06.271 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-12-20T04:26:06.282 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-joblib (0.17.0-4ubuntu1) ... 2023-12-20T04:26:06.438 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-12-20T04:26:06.565 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-20T04:26:06.650 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cachetools (5.0.0-1) ... 2023-12-20T04:26:06.899 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-12-20T04:26:06.926 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-colorama (0.4.4-1) ... 2023-12-20T04:26:07.194 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-openssl (21.0.0-1) ... 2023-12-20T04:26:07.301 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-12-20T04:26:07.463 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-bcrypt (3.2.0-1build1) ... 2023-12-20T04:26:07.560 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-12-20T04:26:07.713 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyinotify (0.9.6-1.3) ... 2023-12-20T04:26:07.961 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-20T04:26:08.023 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-threadpoolctl (3.1.0-1) ... 2023-12-20T04:26:08.263 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-12-20T04:26:08.291 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-argparse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:08.567 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn-lib:amd64 (0.23.2-5ubuntu6) ... 2023-12-20T04:26:08.606 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-20T04:26:08.692 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyperclip (1.8.2-2) ... 2023-12-20T04:26:08.974 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-12-20T04:26:08.985 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-markupsafe (2.0.1-2build1) ... 2023-12-20T04:26:09.236 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libpcre2-16-0:amd64 (10.39-3ubuntu0.1) ... 2023-12-20T04:26:09.250 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-12-20T04:26:09.354 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-psutil (5.9.0-1build1) ... 2023-12-20T04:26:09.376 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:09.711 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-idna (3.3-1) ... 2023-12-20T04:26:09.730 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-babel-localedata (2.8.0+dfsg.1-7) ... 2023-12-20T04:26:09.857 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-natsort (8.0.2-1) ... 2023-12-20T04:26:10.062 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-20T04:26:10.141 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-routes (2.5.1-1ubuntu1) ... 2023-12-20T04:26:10.450 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplejson (3.17.6-1build1) ... 2023-12-20T04:26:10.760 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pygments (2.11.2+dfsg-2) ... 2023-12-20T04:26:11.326 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:26:11.460 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-toml (0.10.2-1) ... 2023-12-20T04:26:11.478 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempita (0.5.2-6ubuntu1) ... 2023-12-20T04:26:11.746 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python-pastedeploy-tpl (2.1.1-1) ... 2023-12-20T04:26:11.872 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up qttranslations5-l10n (5.15.3-1) ... 2023-12-20T04:26:11.954 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-20T04:26:11.998 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-wcwidth (0.2.5+dfsg1-1) ... 2023-12-20T04:26:12.088 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-12-20T04:26:12.214 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-20T04:26:12.316 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-asyncssh (2.5.0-1) ... 2023-12-20T04:26:12.540 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-12-20T04:26:12.734 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-certifi (2020.6.20-1) ... 2023-12-20T04:26:12.817 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-12-20T04:26:13.010 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-paste (3.5.0+dfsg1-1) ... 2023-12-20T04:26:13.084 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:26:13.210 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-12-20T04:26:13.328 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-12-20T04:26:13.411 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cheroot (8.5.2+ds1-1ubuntu3.1) ... 2023-12-20T04:26:13.630 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-12-20T04:26:13.721 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cmd2 (2.3.3-1) ... 2023-12-20T04:26:13.898 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-12-20T04:26:14.047 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-werkzeug (2.0.2+dfsg1-1ubuntu0.22.04.1) ... 2023-12-20T04:26:14.174 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-12-20T04:26:14.407 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.text (3.6.0-2) ... 2023-12-20T04:26:14.450 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-12-20T04:26:14.675 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up socat (1.7.4.1-3ubuntu4) ... 2023-12-20T04:26:14.718 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-12-20T04:26:14.800 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:14.994 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-20T04:26:15.119 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-idna (3.3-1) ... 2023-12-20T04:26:15.119 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-12-20T04:26:15.429 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-20T04:26:15.469 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-sklearn (0.23.2-5ubuntu6) ... 2023-12-20T04:26:15.539 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-20T04:26:15.848 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-12-20T04:26:16.166 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-12-20T04:26:16.434 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-12-20T04:26:16.742 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5core5a:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:26:16.844 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-12-20T04:26:16.876 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-toml (0.10.2-1) ... 2023-12-20T04:26:17.095 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-20T04:26:17.205 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-12-20T04:26:17.277 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librdkafka1:amd64 (1.8.0-1build1) ... 2023-12-20T04:26:17.403 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up xmlstarlet (1.6.1-2.1) ... 2023-12-20T04:26:17.554 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-urllib3 (1.26.5-1~exp1ubuntu0.1) ... 2023-12-20T04:26:17.607 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-12-20T04:26:17.881 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pluggy (0.13.0-7.1) ... 2023-12-20T04:26:17.882 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-portend (3.0.0-1) ... 2023-12-20T04:26:18.132 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-zc.lockfile (2.0-1) ... 2023-12-20T04:26:18.142 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:26:18.268 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-12-20T04:26:18.391 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5dbus5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:26:18.517 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfmt8:amd64 (8.1.1+ds1-2) ... 2023-12-20T04:26:18.602 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-12-20T04:26:18.643 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1 (0.4.8-1) ... 2023-12-20T04:26:18.920 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-20T04:26:19.002 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-singledispatch (3.4.0.3-3) ... 2023-12-20T04:26:19.206 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-12-20T04:26:19.270 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-logutils (0.3.3-8) ... 2023-12-20T04:26:19.322 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rsa (4.8-1) ... 2023-12-20T04:26:19.546 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-tempora (4.1.2-1) ... 2023-12-20T04:26:19.599 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-12-20T04:26:19.806 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-simplegeneric (0.8.1-3) ... 2023-12-20T04:26:19.908 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-20T04:26:20.124 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-prettytable (2.5.0-2) ... 2023-12-20T04:26:20.185 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-12-20T04:26:20.400 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblttng-ust1:amd64 (2.13.1-1ubuntu1) ... 2023-12-20T04:26:20.526 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-websocket (1.2.3-1) ... 2023-12-20T04:26:20.578 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-12-20T04:26:20.818 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libonig5:amd64 (6.9.7.1-2build1) ... 2023-12-20T04:26:20.879 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-12-20T04:26:20.961 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-mako (1.1.3+ds1-2ubuntu0.1) ... 2023-12-20T04:26:21.232 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-20T04:26:21.270 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webob (1:1.8.6-1.1) ... 2023-12-20T04:26:21.374 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librados2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:21.500 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librgw2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:21.588 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jaraco.collections (3.4.0-2) ... 2023-12-20T04:26:21.703 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libsqlite3-mod-ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:21.805 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libcephfs2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:21.848 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-babel (2.8.0+dfsg.1-7) ... 2023-12-20T04:26:21.947 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libradosstriper1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:22.074 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-12-20T04:26:22.222 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/pybabel-python3 to provide /usr/bin/pybabel (pybabel) in auto mode 2023-12-20T04:26:22.369 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up librbd1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:22.484 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjq1:amd64 (1.6-2.1ubuntu3) ... 2023-12-20T04:26:22.484 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-modules-core (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:22.610 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pytest (6.2.5-1ubuntu2) ... 2023-12-20T04:26:22.611 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:22.829 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-20T04:26:22.830 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-20T04:26:23.011 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastedeploy (2.1.1-1) ... 2023-12-20T04:26:23.287 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-portend (3.0.0-1) ... 2023-12-20T04:26:23.512 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libcephfs-dev (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:23.589 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqt5network5:amd64 (5.15.3+dfsg-2ubuntu0.2) ... 2023-12-20T04:26:23.622 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rados (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:23.690 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-jinja2 (3.0.3-1) ... 2023-12-20T04:26:23.739 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up cephfs-top (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:23.865 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-20T04:26:24.007 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pyasn1-modules (0.2.1-1) ... 2023-12-20T04:26:24.325 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests (2.25.1+dfsg-2ubuntu0.1) ... 2023-12-20T04:26:24.626 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up jq (1.6-2.1ubuntu3) ... 2023-12-20T04:26:24.744 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rsa (4.8-1) ... 2023-12-20T04:26:25.036 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-webtest (2.0.35-1) ... 2023-12-20T04:26:25.220 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rbd (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:25.321 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-requests-oauthlib (1.3.0+ds-0.1) ... 2023-12-20T04:26:25.346 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up rbd-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:25.473 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-rgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:25.582 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up python3-cephfs (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:25.597 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cherrypy3 (18.6.1-4) ... 2023-12-20T04:26:25.683 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up cephfs-shell (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:25.793 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:26.007 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pastescript (2.0.2-4) ... 2023-12-20T04:26:26.308 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-pecan (1.3.3-4ubuntu2) ... 2023-12-20T04:26:26.347 INFO:teuthology.orchestra.run.smithi039.stdout:Adding group ceph....done 2023-12-20T04:26:26.669 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libthrift-0.16.0:amd64 (0.16.0-2) ... 2023-12-20T04:26:26.722 INFO:teuthology.orchestra.run.smithi039.stdout:Adding system user ceph....done 2023-12-20T04:26:26.778 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librados2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:26.904 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librgw2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:26.909 INFO:teuthology.orchestra.run.smithi039.stdout:Setting system user ceph properties....done 2023-12-20T04:26:26.928 INFO:teuthology.orchestra.run.smithi039.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-12-20T04:26:27.076 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-12-20T04:26:27.107 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libsqlite3-mod-ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:27.258 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs2 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:27.384 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libradosstriper1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:27.493 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-12-20T04:26:27.510 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-google-auth (1.5.1-3) ... 2023-12-20T04:26:27.795 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up librbd1 (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:27.921 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-modules-core (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:28.030 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:28.170 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up cephfs-mirror (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:28.245 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/remote-fs.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-20T04:26:28.245 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-fuse.target -> /lib/systemd/system/ceph-fuse.target. 2023-12-20T04:26:28.387 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-20T04:26:28.387 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-20T04:26:28.882 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libcephfs-dev (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:29.008 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rados (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:29.023 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-test (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:29.134 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-top (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:29.149 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up radosgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:29.260 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-kubernetes (12.0.1-1ubuntu1) ... 2023-12-20T04:26:29.697 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-20T04:26:29.697 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-20T04:26:30.351 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-base (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:30.590 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rbd (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:30.699 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up rbd-fuse (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:30.718 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-12-20T04:26:30.825 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-rgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:30.984 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up python3-cephfs (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:31.110 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-shell (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:31.236 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-common (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:31.377 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mds (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:31.607 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-20T04:26:31.607 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-20T04:26:31.807 INFO:teuthology.orchestra.run.smithi032.stdout:Adding group ceph....done 2023-12-20T04:26:32.200 INFO:teuthology.orchestra.run.smithi032.stdout:Adding system user ceph....done 2023-12-20T04:26:32.263 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:32.405 INFO:teuthology.orchestra.run.smithi032.stdout:Setting system user ceph properties....done 2023-12-20T04:26:32.421 INFO:teuthology.orchestra.run.smithi032.stdout:chown: cannot access '/var/log/ceph/*.log*': No such file or directory 2023-12-20T04:26:32.483 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-20T04:26:32.483 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-20T04:26:32.575 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target -> /lib/systemd/system/ceph.target. 2023-12-20T04:26:32.990 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/rbdmap.service -> /lib/systemd/system/rbdmap.service. 2023-12-20T04:26:33.115 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-osd (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:33.435 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-20T04:26:33.435 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-20T04:26:33.696 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up cephfs-mirror (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:33.913 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-20T04:26:33.913 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target -> /lib/systemd/system/cephfs-mirror.target. 2023-12-20T04:26:34.049 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-k8sevents (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:34.167 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-diskprediction-local (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:34.318 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mon (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:34.535 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-20T04:26:34.535 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-20T04:26:34.556 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-test (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:34.682 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up radosgw (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:35.186 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:35.224 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-20T04:26:35.224 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target -> /lib/systemd/system/ceph-radosgw.target. 2023-12-20T04:26:35.303 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:35.404 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-mgr-dashboard (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:35.539 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ceph-volume (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:35.875 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-base (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:35.900 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-20T04:26:36.247 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service -> /lib/systemd/system/ceph-crash.service. 2023-12-20T04:26:36.960 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mds (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:37.042 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-20T04:26:37.182 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-20T04:26:37.182 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target -> /lib/systemd/system/ceph-mds.target. 2023-12-20T04:26:37.513 INFO:teuthology.orchestra.run.smithi039.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-20T04:26:37.518 DEBUG:teuthology.orchestra.run.smithi039:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-12-20T04:26:37.595 INFO:teuthology.orchestra.run.smithi039.stdout:Reading package lists... 2023-12-20T04:26:37.767 INFO:teuthology.orchestra.run.smithi039.stdout:Building dependency tree... 2023-12-20T04:26:37.768 INFO:teuthology.orchestra.run.smithi039.stdout:Reading state information... 2023-12-20T04:26:37.846 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:37.928 INFO:teuthology.orchestra.run.smithi039.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-12-20T04:26:37.929 INFO:teuthology.orchestra.run.smithi039.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-12-20T04:26:37.929 INFO:teuthology.orchestra.run.smithi039.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-12-20T04:26:37.929 INFO:teuthology.orchestra.run.smithi039.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-12-20T04:26:37.929 INFO:teuthology.orchestra.run.smithi039.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-12-20T04:26:37.929 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages were automatically installed and are no longer required: 2023-12-20T04:26:37.929 INFO:teuthology.orchestra.run.smithi039.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-20T04:26:37.929 INFO:teuthology.orchestra.run.smithi039.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-20T04:26:37.929 INFO:teuthology.orchestra.run.smithi039.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-20T04:26:37.929 INFO:teuthology.orchestra.run.smithi039.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-20T04:26:37.930 INFO:teuthology.orchestra.run.smithi039.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-20T04:26:37.931 INFO:teuthology.orchestra.run.smithi039.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-20T04:26:37.931 INFO:teuthology.orchestra.run.smithi039.stdout:The following additional packages will be installed: 2023-12-20T04:26:37.931 INFO:teuthology.orchestra.run.smithi039.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-12-20T04:26:37.931 INFO:teuthology.orchestra.run.smithi039.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-12-20T04:26:37.931 INFO:teuthology.orchestra.run.smithi039.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-12-20T04:26:37.931 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-12-20T04:26:37.931 INFO:teuthology.orchestra.run.smithi039.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-20T04:26:37.932 INFO:teuthology.orchestra.run.smithi039.stdout:Suggested packages: 2023-12-20T04:26:37.932 INFO:teuthology.orchestra.run.smithi039.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-12-20T04:26:37.932 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-12-20T04:26:37.932 INFO:teuthology.orchestra.run.smithi039.stdout: libengine-pkcs11-openssl 2023-12-20T04:26:37.984 INFO:teuthology.orchestra.run.smithi039.stdout:The following NEW packages will be installed: 2023-12-20T04:26:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-12-20T04:26:37.984 INFO:teuthology.orchestra.run.smithi039.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-12-20T04:26:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-12-20T04:26:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-12-20T04:26:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-12-20T04:26:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-12-20T04:26:37.985 INFO:teuthology.orchestra.run.smithi039.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-20T04:26:37.985 INFO:teuthology.orchestra.run.smithi039.stdout:The following packages will be upgraded: 2023-12-20T04:26:37.986 INFO:teuthology.orchestra.run.smithi039.stdout: libssl3 2023-12-20T04:26:38.041 INFO:teuthology.orchestra.run.smithi039.stdout:1 upgraded, 39 newly installed, 0 to remove and 267 not upgraded. 2023-12-20T04:26:38.041 INFO:teuthology.orchestra.run.smithi039.stdout:Need to get 54.6 MB of archives. 2023-12-20T04:26:38.042 INFO:teuthology.orchestra.run.smithi039.stdout:After this operation, 202 MB of additional disk space will be used. 2023-12-20T04:26:38.042 INFO:teuthology.orchestra.run.smithi039.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-12-20T04:26:38.067 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-20T04:26:38.067 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target -> /lib/systemd/system/ceph-mgr.target. 2023-12-20T04:26:38.143 INFO:teuthology.orchestra.run.smithi039.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-12-20T04:26:38.198 INFO:teuthology.orchestra.run.smithi039.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-12-20T04:26:38.205 INFO:teuthology.orchestra.run.smithi039.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-12-20T04:26:38.206 INFO:teuthology.orchestra.run.smithi039.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-12-20T04:26:38.208 INFO:teuthology.orchestra.run.smithi039.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-12-20T04:26:38.209 INFO:teuthology.orchestra.run.smithi039.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-12-20T04:26:38.209 INFO:teuthology.orchestra.run.smithi039.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2023-12-20T04:26:38.210 INFO:teuthology.orchestra.run.smithi039.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-12-20T04:26:38.211 INFO:teuthology.orchestra.run.smithi039.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-12-20T04:26:38.212 INFO:teuthology.orchestra.run.smithi039.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-12-20T04:26:38.217 INFO:teuthology.orchestra.run.smithi039.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-12-20T04:26:38.229 INFO:teuthology.orchestra.run.smithi039.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-12-20T04:26:38.472 INFO:teuthology.orchestra.run.smithi039.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-12-20T04:26:38.474 INFO:teuthology.orchestra.run.smithi039.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-12-20T04:26:38.474 INFO:teuthology.orchestra.run.smithi039.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-12-20T04:26:38.477 INFO:teuthology.orchestra.run.smithi039.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-12-20T04:26:38.477 INFO:teuthology.orchestra.run.smithi039.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-12-20T04:26:38.482 INFO:teuthology.orchestra.run.smithi039.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.10-0ubuntu0.22.04.1 [143 kB] 2023-12-20T04:26:38.484 INFO:teuthology.orchestra.run.smithi039.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-12-20T04:26:38.491 INFO:teuthology.orchestra.run.smithi039.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-12-20T04:26:38.492 INFO:teuthology.orchestra.run.smithi039.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-12-20T04:26:38.515 INFO:teuthology.orchestra.run.smithi039.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-12-20T04:26:38.515 INFO:teuthology.orchestra.run.smithi039.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-12-20T04:26:38.526 INFO:teuthology.orchestra.run.smithi039.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-12-20T04:26:38.549 INFO:teuthology.orchestra.run.smithi039.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-12-20T04:26:38.553 INFO:teuthology.orchestra.run.smithi039.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-12-20T04:26:38.554 INFO:teuthology.orchestra.run.smithi039.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-12-20T04:26:38.555 INFO:teuthology.orchestra.run.smithi039.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-12-20T04:26:38.556 INFO:teuthology.orchestra.run.smithi039.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.10-0ubuntu0.22.04.1 [1,223 kB] 2023-12-20T04:26:38.569 INFO:teuthology.orchestra.run.smithi039.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-12-20T04:26:38.570 INFO:teuthology.orchestra.run.smithi039.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.10-0ubuntu0.22.04.1 [16.1 MB] 2023-12-20T04:26:38.722 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-osd (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:38.732 INFO:teuthology.orchestra.run.smithi039.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-12-20T04:26:38.732 INFO:teuthology.orchestra.run.smithi039.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-12-20T04:26:38.732 INFO:teuthology.orchestra.run.smithi039.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-12-20T04:26:38.732 INFO:teuthology.orchestra.run.smithi039.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-12-20T04:26:38.733 INFO:teuthology.orchestra.run.smithi039.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-12-20T04:26:38.733 INFO:teuthology.orchestra.run.smithi039.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-12-20T04:26:38.746 INFO:teuthology.orchestra.run.smithi039.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-12-20T04:26:38.747 INFO:teuthology.orchestra.run.smithi039.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-12-20T04:26:39.103 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-20T04:26:39.103 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target -> /lib/systemd/system/ceph-osd.target. 2023-12-20T04:26:39.142 INFO:teuthology.orchestra.run.smithi039.stdout:Preconfiguring packages ... 2023-12-20T04:26:39.291 INFO:teuthology.orchestra.run.smithi039.stdout:Fetched 54.6 MB in 1s (72.5 MB/s) 2023-12-20T04:26:39.424 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package flex. 2023-12-20T04:26:39.499 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139847 files and directories currently installed.) 2023-12-20T04:26:39.502 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-12-20T04:26:39.548 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking flex (2.6.4-8build2) ... 2023-12-20T04:26:39.741 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-k8sevents (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:39.866 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-diskprediction-local (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:40.001 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mon (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:40.051 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-20T04:26:40.218 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-20T04:26:40.218 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target -> /lib/systemd/system/ceph-mon.target. 2023-12-20T04:26:40.236 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-12-20T04:26:40.643 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-20T04:26:40.860 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-cephadm (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:41.003 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:41.129 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-mgr-dashboard (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:41.195 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package bison. 2023-12-20T04:26:41.269 INFO:teuthology.orchestra.run.smithi039.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139920 files and directories currently installed.) 2023-12-20T04:26:41.272 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ceph-volume (18.2.1-132-g568df92b-1jammy) ... 2023-12-20T04:26:41.273 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-12-20T04:26:41.323 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-12-20T04:26:41.633 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-20T04:26:41.765 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-12-20T04:26:41.780 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-12-20T04:26:41.823 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-12-20T04:26:42.101 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package dump. 2023-12-20T04:26:42.116 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-12-20T04:26:42.151 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking dump (0.4b47-3) ... 2023-12-20T04:26:42.429 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package indent. 2023-12-20T04:26:42.444 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:26:42.614 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-20T04:26:42.835 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-20T04:26:42.870 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-12-20T04:26:42.881 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-12-20T04:26:42.917 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-20T04:26:43.245 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-12-20T04:26:43.260 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2023-12-20T04:26:43.304 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-20T04:26:43.351 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-20T04:26:43.422 DEBUG:teuthology.orchestra.run.smithi032:> sudo DEBIAN_FRONTEND=noninteractive apt-get -y --force-yes -o Dpkg::Options::="--force-confdef" -o Dpkg::Options::="--force-confold" install bison flex libelf-dev libssl-dev network-manager iproute2 util-linux dump indent libaio-dev libtool-bin uuid-dev xfslibs-dev postgresql postgresql-client postgresql-common postgresql-contrib 2023-12-20T04:26:43.500 INFO:teuthology.orchestra.run.smithi032.stdout:Reading package lists... 2023-12-20T04:26:43.564 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-12-20T04:26:43.580 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-12-20T04:26:43.631 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-12-20T04:26:43.672 INFO:teuthology.orchestra.run.smithi032.stdout:Building dependency tree... 2023-12-20T04:26:43.672 INFO:teuthology.orchestra.run.smithi032.stdout:Reading state information... 2023-12-20T04:26:43.833 INFO:teuthology.orchestra.run.smithi032.stdout:iproute2 is already the newest version (5.15.0-1ubuntu2). 2023-12-20T04:26:43.833 INFO:teuthology.orchestra.run.smithi032.stdout:libtool-bin is already the newest version (2.4.6-15build2). 2023-12-20T04:26:43.833 INFO:teuthology.orchestra.run.smithi032.stdout:util-linux is already the newest version (2.37.2-4ubuntu3). 2023-12-20T04:26:43.833 INFO:teuthology.orchestra.run.smithi032.stdout:uuid-dev is already the newest version (2.37.2-4ubuntu3). 2023-12-20T04:26:43.833 INFO:teuthology.orchestra.run.smithi032.stdout:xfslibs-dev is already the newest version (5.13.0-1ubuntu2). 2023-12-20T04:26:43.833 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages were automatically installed and are no longer required: 2023-12-20T04:26:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: libboost-iostreams1.74.0 libboost-thread1.74.0 libgfapi0 libgfrpc0 libgfxdr0 2023-12-20T04:26:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: libglusterfs0 libiscsi7 libpmemobj1 libpython2-dev libpython2-stdlib 2023-12-20T04:26:43.833 INFO:teuthology.orchestra.run.smithi032.stdout: libpython2.7 libpython2.7-dev libpython2.7-minimal libpython2.7-stdlib 2023-12-20T04:26:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: linux-image-5.4.0-135-generic linux-modules-5.4.0-135-generic python2 2023-12-20T04:26:43.834 INFO:teuthology.orchestra.run.smithi032.stdout: python2-dev python2-minimal python2.7 python2.7-dev python2.7-minimal 2023-12-20T04:26:43.835 INFO:teuthology.orchestra.run.smithi032.stdout:Use 'sudo apt autoremove' to remove them. 2023-12-20T04:26:43.835 INFO:teuthology.orchestra.run.smithi032.stdout:The following additional packages will be installed: 2023-12-20T04:26:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: libbluetooth3 libfl-dev libfl2 libjson-perl libllvm14 liblzo2-2 2023-12-20T04:26:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: libmbim-glib4 libmbim-proxy libmm-glib0 libndp0 libnm0 libpq5 libqmi-glib5 2023-12-20T04:26:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: libqmi-proxy libssl3 libteamdctl0 modemmanager network-manager-pptp 2023-12-20T04:26:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-14 postgresql-client-14 postgresql-client-common ppp pptp-linux 2023-12-20T04:26:43.835 INFO:teuthology.orchestra.run.smithi032.stdout: tcl tcl8.6 usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-20T04:26:43.836 INFO:teuthology.orchestra.run.smithi032.stdout:Suggested packages: 2023-12-20T04:26:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: bison-doc flex-doc indent-doc libssl-doc avahi-autoipd libteam-utils 2023-12-20T04:26:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-doc postgresql-doc-14 tcl-tclreadline comgt wvdial wpagui 2023-12-20T04:26:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: libengine-pkcs11-openssl 2023-12-20T04:26:43.888 INFO:teuthology.orchestra.run.smithi032.stdout:The following NEW packages will be installed: 2023-12-20T04:26:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: bison dump flex indent libaio-dev libbluetooth3 libelf-dev libfl-dev libfl2 2023-12-20T04:26:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: libjson-perl libllvm14 liblzo2-2 libmbim-glib4 libmbim-proxy libmm-glib0 2023-12-20T04:26:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: libndp0 libnm0 libpq5 libqmi-glib5 libqmi-proxy libssl-dev libteamdctl0 2023-12-20T04:26:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: modemmanager network-manager network-manager-pptp postgresql postgresql-14 2023-12-20T04:26:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-client postgresql-client-14 postgresql-client-common 2023-12-20T04:26:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: postgresql-common postgresql-contrib ppp pptp-linux tcl tcl8.6 2023-12-20T04:26:43.889 INFO:teuthology.orchestra.run.smithi032.stdout: usb-modeswitch usb-modeswitch-data wpasupplicant 2023-12-20T04:26:43.890 INFO:teuthology.orchestra.run.smithi032.stdout:The following packages will be upgraded: 2023-12-20T04:26:43.891 INFO:teuthology.orchestra.run.smithi032.stdout: libssl3 2023-12-20T04:26:43.952 INFO:teuthology.orchestra.run.smithi032.stdout:1 upgraded, 39 newly installed, 0 to remove and 267 not upgraded. 2023-12-20T04:26:43.952 INFO:teuthology.orchestra.run.smithi032.stdout:Need to get 54.6 MB of archives. 2023-12-20T04:26:43.952 INFO:teuthology.orchestra.run.smithi032.stdout:After this operation, 202 MB of additional disk space will be used. 2023-12-20T04:26:43.953 INFO:teuthology.orchestra.run.smithi032.stdout:Get:1 http://archive.ubuntu.com/ubuntu jammy/main amd64 flex amd64 2.6.4-8build2 [307 kB] 2023-12-20T04:26:43.968 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libfl2:amd64. 2023-12-20T04:26:43.984 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-12-20T04:26:44.026 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-12-20T04:26:44.074 INFO:teuthology.orchestra.run.smithi032.stdout:Get:2 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl3 amd64 3.0.2-0ubuntu1.12 [1,901 kB] 2023-12-20T04:26:44.139 INFO:teuthology.orchestra.run.smithi032.stdout:Get:3 http://archive.ubuntu.com/ubuntu jammy/main amd64 bison amd64 2:3.8.2+dfsg-1build1 [748 kB] 2023-12-20T04:26:44.150 INFO:teuthology.orchestra.run.smithi032.stdout:Get:4 http://archive.ubuntu.com/ubuntu jammy/main amd64 liblzo2-2 amd64 2.10-2build3 [53.7 kB] 2023-12-20T04:26:44.151 INFO:teuthology.orchestra.run.smithi032.stdout:Get:5 http://archive.ubuntu.com/ubuntu jammy/universe amd64 dump amd64 0.4b47-3 [133 kB] 2023-12-20T04:26:44.153 INFO:teuthology.orchestra.run.smithi032.stdout:Get:6 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 indent amd64 2.2.12-1ubuntu0.22.04.1 [90.8 kB] 2023-12-20T04:26:44.155 INFO:teuthology.orchestra.run.smithi032.stdout:Get:7 http://archive.ubuntu.com/ubuntu jammy/main amd64 libaio-dev amd64 0.3.112-13build1 [21.2 kB] 2023-12-20T04:26:44.156 INFO:teuthology.orchestra.run.smithi032.stdout:Get:8 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libbluetooth3 amd64 5.64-0ubuntu1.1 [87.0 kB] 2023-12-20T04:26:44.157 INFO:teuthology.orchestra.run.smithi032.stdout:Get:9 http://archive.ubuntu.com/ubuntu jammy/main amd64 libelf-dev amd64 0.186-1build1 [64.4 kB] 2023-12-20T04:26:44.158 INFO:teuthology.orchestra.run.smithi032.stdout:Get:10 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl2 amd64 2.6.4-8build2 [10.7 kB] 2023-12-20T04:26:44.159 INFO:teuthology.orchestra.run.smithi032.stdout:Get:11 http://archive.ubuntu.com/ubuntu jammy/main amd64 libfl-dev amd64 2.6.4-8build2 [6,236 B] 2023-12-20T04:26:44.162 INFO:teuthology.orchestra.run.smithi032.stdout:Get:12 http://archive.ubuntu.com/ubuntu jammy/main amd64 libjson-perl all 4.04000-1 [81.8 kB] 2023-12-20T04:26:44.179 INFO:teuthology.orchestra.run.smithi032.stdout:Get:13 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libllvm14 amd64 1:14.0.0-1ubuntu1.1 [24.0 MB] 2023-12-20T04:26:44.211 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-12-20T04:26:44.227 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-12-20T04:26:44.270 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-20T04:26:44.476 INFO:teuthology.orchestra.run.smithi032.stdout:Get:14 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-glib4 amd64 1.28.0-1~ubuntu20.04.1 [191 kB] 2023-12-20T04:26:44.478 INFO:teuthology.orchestra.run.smithi032.stdout:Get:15 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmbim-proxy amd64 1.28.0-1~ubuntu20.04.1 [6,130 B] 2023-12-20T04:26:44.479 INFO:teuthology.orchestra.run.smithi032.stdout:Get:16 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libmm-glib0 amd64 1.20.0-1~ubuntu22.04.2 [263 kB] 2023-12-20T04:26:44.482 INFO:teuthology.orchestra.run.smithi032.stdout:Get:17 http://archive.ubuntu.com/ubuntu jammy/main amd64 libndp0 amd64 1.8-0ubuntu3 [12.3 kB] 2023-12-20T04:26:44.482 INFO:teuthology.orchestra.run.smithi032.stdout:Get:18 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libnm0 amd64 1.36.6-0ubuntu2 [456 kB] 2023-12-20T04:26:44.487 INFO:teuthology.orchestra.run.smithi032.stdout:Get:19 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libpq5 amd64 14.10-0ubuntu0.22.04.1 [143 kB] 2023-12-20T04:26:44.489 INFO:teuthology.orchestra.run.smithi032.stdout:Get:20 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-glib5 amd64 1.32.0-1ubuntu0.22.04.1 [772 kB] 2023-12-20T04:26:44.496 INFO:teuthology.orchestra.run.smithi032.stdout:Get:21 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libqmi-proxy amd64 1.32.0-1ubuntu0.22.04.1 [6,072 B] 2023-12-20T04:26:44.497 INFO:teuthology.orchestra.run.smithi032.stdout:Get:22 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 libssl-dev amd64 3.0.2-0ubuntu1.12 [2,373 kB] 2023-12-20T04:26:44.519 INFO:teuthology.orchestra.run.smithi032.stdout:Get:23 http://archive.ubuntu.com/ubuntu jammy/main amd64 libteamdctl0 amd64 1.31-1build2 [12.6 kB] 2023-12-20T04:26:44.521 INFO:teuthology.orchestra.run.smithi032.stdout:Get:24 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 modemmanager amd64 1.20.0-1~ubuntu22.04.2 [1,094 kB] 2023-12-20T04:26:44.530 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libjson-perl. 2023-12-20T04:26:44.532 INFO:teuthology.orchestra.run.smithi032.stdout:Get:25 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 network-manager amd64 1.36.6-0ubuntu2 [2,203 kB] 2023-12-20T04:26:44.547 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-12-20T04:26:44.556 INFO:teuthology.orchestra.run.smithi032.stdout:Get:26 http://archive.ubuntu.com/ubuntu jammy/main amd64 ppp amd64 2.4.9-1+1ubuntu3 [363 kB] 2023-12-20T04:26:44.559 INFO:teuthology.orchestra.run.smithi032.stdout:Get:27 http://archive.ubuntu.com/ubuntu jammy/main amd64 pptp-linux amd64 1.10.0-1build3 [42.5 kB] 2023-12-20T04:26:44.560 INFO:teuthology.orchestra.run.smithi032.stdout:Get:28 http://archive.ubuntu.com/ubuntu jammy/main amd64 network-manager-pptp amd64 1.2.10-1 [30.9 kB] 2023-12-20T04:26:44.561 INFO:teuthology.orchestra.run.smithi032.stdout:Get:29 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client-common all 238 [29.6 kB] 2023-12-20T04:26:44.562 INFO:teuthology.orchestra.run.smithi032.stdout:Get:30 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-client-14 amd64 14.10-0ubuntu0.22.04.1 [1,223 kB] 2023-12-20T04:26:44.576 INFO:teuthology.orchestra.run.smithi032.stdout:Get:31 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-common all 238 [169 kB] 2023-12-20T04:26:44.581 INFO:teuthology.orchestra.run.smithi032.stdout:Get:32 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 postgresql-14 amd64 14.10-0ubuntu0.22.04.1 [16.1 MB] 2023-12-20T04:26:44.589 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-12-20T04:26:44.773 INFO:teuthology.orchestra.run.smithi032.stdout:Get:33 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql all 14+238 [3,288 B] 2023-12-20T04:26:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:Get:34 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-client all 14+238 [3,292 B] 2023-12-20T04:26:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:Get:35 http://archive.ubuntu.com/ubuntu jammy/main amd64 postgresql-contrib all 14+238 [3,292 B] 2023-12-20T04:26:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:Get:36 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl8.6 amd64 8.6.12+dfsg-1build1 [15.0 kB] 2023-12-20T04:26:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:Get:37 http://archive.ubuntu.com/ubuntu jammy/main amd64 tcl amd64 8.6.11+1build2 [4,678 B] 2023-12-20T04:26:44.774 INFO:teuthology.orchestra.run.smithi032.stdout:Get:38 http://archive.ubuntu.com/ubuntu jammy-updates/main amd64 wpasupplicant amd64 2:2.10-6ubuntu2 [1,481 kB] 2023-12-20T04:26:44.788 INFO:teuthology.orchestra.run.smithi032.stdout:Get:39 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch-data all 20191128-4 [33.2 kB] 2023-12-20T04:26:44.789 INFO:teuthology.orchestra.run.smithi032.stdout:Get:40 http://archive.ubuntu.com/ubuntu jammy/main amd64 usb-modeswitch amd64 2.6.1-3ubuntu2 [46.0 kB] 2023-12-20T04:26:44.900 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libllvm14:amd64. 2023-12-20T04:26:44.917 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-12-20T04:26:44.959 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-20T04:26:45.186 INFO:teuthology.orchestra.run.smithi032.stdout:Preconfiguring packages ... 2023-12-20T04:26:45.350 INFO:teuthology.orchestra.run.smithi032.stdout:Fetched 54.6 MB in 1s (61.3 MB/s) 2023-12-20T04:26:45.524 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package flex. 2023-12-20T04:26:45.600 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139847 files and directories currently installed.) 2023-12-20T04:26:45.604 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../flex_2.6.4-8build2_amd64.deb ... 2023-12-20T04:26:45.648 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking flex (2.6.4-8build2) ... 2023-12-20T04:26:46.143 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../libssl3_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-20T04:26:46.279 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl3:amd64 (3.0.2-0ubuntu1.12) over (3.0.2-0ubuntu1.7) ... 2023-12-20T04:26:46.311 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-12-20T04:26:46.328 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-20T04:26:46.369 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-20T04:26:46.638 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libmbim-proxy. 2023-12-20T04:26:46.655 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-20T04:26:46.662 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl3:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-20T04:26:46.697 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-20T04:26:46.966 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-12-20T04:26:46.982 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-20T04:26:47.016 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-20T04:26:47.246 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package bison. 2023-12-20T04:26:47.322 INFO:teuthology.orchestra.run.smithi032.stdout:(Reading database ... (Reading database ... 5% (Reading database ... 10% (Reading database ... 15% (Reading database ... 20% (Reading database ... 25% (Reading database ... 30% (Reading database ... 35% (Reading database ... 40% (Reading database ... 45% (Reading database ... 50% (Reading database ... 55% (Reading database ... 60% (Reading database ... 65% (Reading database ... 70% (Reading database ... 75% (Reading database ... 80% (Reading database ... 85% (Reading database ... 90% (Reading database ... 95% (Reading database ... 100% (Reading database ... 139920 files and directories currently installed.) 2023-12-20T04:26:47.326 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../00-bison_2%3a3.8.2+dfsg-1build1_amd64.deb ... 2023-12-20T04:26:47.373 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking bison (2:3.8.2+dfsg-1build1) ... 2023-12-20T04:26:47.393 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libndp0:amd64. 2023-12-20T04:26:47.410 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-12-20T04:26:47.452 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-20T04:26:47.780 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libnm0:amd64. 2023-12-20T04:26:47.796 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-12-20T04:26:47.838 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-20T04:26:48.166 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libpq5:amd64. 2023-12-20T04:26:48.182 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../16-libpq5_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:26:48.223 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:26:48.310 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package liblzo2-2:amd64. 2023-12-20T04:26:48.327 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../01-liblzo2-2_2.10-2build3_amd64.deb ... 2023-12-20T04:26:48.367 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking liblzo2-2:amd64 (2.10-2build3) ... 2023-12-20T04:26:48.576 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-12-20T04:26:48.593 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:26:48.704 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package dump. 2023-12-20T04:26:48.721 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../02-dump_0.4b47-3_amd64.deb ... 2023-12-20T04:26:48.751 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-20T04:26:48.754 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking dump (0.4b47-3) ... 2023-12-20T04:26:49.024 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package indent. 2023-12-20T04:26:49.037 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libqmi-proxy. 2023-12-20T04:26:49.040 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../03-indent_2.2.12-1ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:26:49.053 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:26:49.083 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-20T04:26:49.095 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-20T04:26:49.347 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-12-20T04:26:49.363 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-20T04:26:49.393 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libaio-dev:amd64. 2023-12-20T04:26:49.409 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../04-libaio-dev_0.3.112-13build1_amd64.deb ... 2023-12-20T04:26:49.431 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-20T04:26:49.451 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-20T04:26:49.763 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libbluetooth3:amd64. 2023-12-20T04:26:49.780 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../05-libbluetooth3_5.64-0ubuntu1.1_amd64.deb ... 2023-12-20T04:26:49.813 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-20T04:26:49.985 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-12-20T04:26:50.001 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-12-20T04:26:50.041 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libelf-dev:amd64. 2023-12-20T04:26:50.043 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-12-20T04:26:50.056 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../06-libelf-dev_0.186-1build1_amd64.deb ... 2023-12-20T04:26:50.099 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libelf-dev:amd64 (0.186-1build1) ... 2023-12-20T04:26:50.346 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package modemmanager. 2023-12-20T04:26:50.362 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-20T04:26:50.396 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-20T04:26:50.436 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl2:amd64. 2023-12-20T04:26:50.452 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../07-libfl2_2.6.4-8build2_amd64.deb ... 2023-12-20T04:26:50.527 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl2:amd64 (2.6.4-8build2) ... 2023-12-20T04:26:50.780 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libfl-dev:amd64. 2023-12-20T04:26:50.796 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../08-libfl-dev_2.6.4-8build2_amd64.deb ... 2023-12-20T04:26:50.838 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-20T04:26:51.066 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libjson-perl. 2023-12-20T04:26:51.082 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../09-libjson-perl_4.04000-1_all.deb ... 2023-12-20T04:26:51.085 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package network-manager. 2023-12-20T04:26:51.101 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-12-20T04:26:51.124 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libjson-perl (4.04000-1) ... 2023-12-20T04:26:51.152 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-12-20T04:26:51.418 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libllvm14:amd64. 2023-12-20T04:26:51.435 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../10-libllvm14_1%3a14.0.0-1ubuntu1.1_amd64.deb ... 2023-12-20T04:26:51.477 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-20T04:26:51.783 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package ppp. 2023-12-20T04:26:51.799 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-12-20T04:26:52.162 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-12-20T04:26:52.454 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package pptp-linux. 2023-12-20T04:26:52.469 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-12-20T04:26:52.504 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-12-20T04:26:52.756 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package network-manager-pptp. 2023-12-20T04:26:52.771 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-12-20T04:26:52.838 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmbim-glib4:amd64. 2023-12-20T04:26:52.848 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-12-20T04:26:52.854 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../11-libmbim-glib4_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-20T04:26:52.896 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-20T04:26:53.134 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-client-common. 2023-12-20T04:26:53.150 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-12-20T04:26:53.184 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-client-common (238) ... 2023-12-20T04:26:53.216 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmbim-proxy. 2023-12-20T04:26:53.232 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../12-libmbim-proxy_1.28.0-1~ubuntu20.04.1_amd64.deb ... 2023-12-20T04:26:53.266 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-20T04:26:53.463 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-client-14. 2023-12-20T04:26:53.479 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../27-postgresql-client-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:26:53.521 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:26:53.585 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libmm-glib0:amd64. 2023-12-20T04:26:53.601 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../13-libmm-glib0_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-20T04:26:53.643 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-20T04:26:54.004 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libndp0:amd64. 2023-12-20T04:26:54.021 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../14-libndp0_1.8-0ubuntu3_amd64.deb ... 2023-12-20T04:26:54.062 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-20T04:26:54.168 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-common. 2023-12-20T04:26:54.184 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-12-20T04:26:54.245 INFO:teuthology.orchestra.run.smithi039.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-12-20T04:26:54.275 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-common (238) ... 2023-12-20T04:26:54.424 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libnm0:amd64. 2023-12-20T04:26:54.440 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../15-libnm0_1.36.6-0ubuntu2_amd64.deb ... 2023-12-20T04:26:54.482 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-20T04:26:54.708 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-14. 2023-12-20T04:26:54.725 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../29-postgresql-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:26:54.785 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libpq5:amd64. 2023-12-20T04:26:54.801 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../16-libpq5_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:26:54.820 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:26:54.843 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:26:55.179 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqmi-glib5:amd64. 2023-12-20T04:26:55.195 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../17-libqmi-glib5_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:26:55.245 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-20T04:26:55.548 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libqmi-proxy. 2023-12-20T04:26:55.564 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../18-libqmi-proxy_1.32.0-1ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:26:55.615 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-20T04:26:55.867 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libssl-dev:amd64. 2023-12-20T04:26:55.883 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../19-libssl-dev_3.0.2-0ubuntu1.12_amd64.deb ... 2023-12-20T04:26:55.925 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-20T04:26:56.120 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql. 2023-12-20T04:26:56.135 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-12-20T04:26:56.170 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql (14+238) ... 2023-12-20T04:26:56.387 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package libteamdctl0:amd64. 2023-12-20T04:26:56.402 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../20-libteamdctl0_1.31-1build2_amd64.deb ... 2023-12-20T04:26:56.445 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking libteamdctl0:amd64 (1.31-1build2) ... 2023-12-20T04:26:56.447 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-client. 2023-12-20T04:26:56.462 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-12-20T04:26:56.505 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-client (14+238) ... 2023-12-20T04:26:56.748 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package postgresql-contrib. 2023-12-20T04:26:56.764 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-12-20T04:26:56.816 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking postgresql-contrib (14+238) ... 2023-12-20T04:26:56.831 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package modemmanager. 2023-12-20T04:26:56.847 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../21-modemmanager_1.20.0-1~ubuntu22.04.2_amd64.deb ... 2023-12-20T04:26:56.889 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-20T04:26:57.135 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package tcl8.6. 2023-12-20T04:26:57.151 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-12-20T04:26:57.193 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-20T04:26:57.462 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package network-manager. 2023-12-20T04:26:57.478 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../22-network-manager_1.36.6-0ubuntu2_amd64.deb ... 2023-12-20T04:26:57.504 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package tcl. 2023-12-20T04:26:57.520 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-12-20T04:26:57.529 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking network-manager (1.36.6-0ubuntu2) ... 2023-12-20T04:26:57.559 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-12-20T04:26:57.898 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package wpasupplicant. 2023-12-20T04:26:57.914 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-12-20T04:26:57.956 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-20T04:26:58.144 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package ppp. 2023-12-20T04:26:58.160 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../23-ppp_2.4.9-1+1ubuntu3_amd64.deb ... 2023-12-20T04:26:58.277 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-12-20T04:26:58.294 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-12-20T04:26:58.335 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-12-20T04:26:58.503 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking ppp (2.4.9-1+1ubuntu3) ... 2023-12-20T04:26:58.730 INFO:teuthology.orchestra.run.smithi039.stdout:Selecting previously unselected package usb-modeswitch. 2023-12-20T04:26:58.747 INFO:teuthology.orchestra.run.smithi039.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-12-20T04:26:58.798 INFO:teuthology.orchestra.run.smithi039.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-20T04:26:58.833 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package pptp-linux. 2023-12-20T04:26:58.848 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../24-pptp-linux_1.10.0-1build3_amd64.deb ... 2023-12-20T04:26:58.882 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking pptp-linux (1.10.0-1build3) ... 2023-12-20T04:26:59.083 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-client-common (238) ... 2023-12-20T04:26:59.185 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package network-manager-pptp. 2023-12-20T04:26:59.201 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../25-network-manager-pptp_1.2.10-1_amd64.deb ... 2023-12-20T04:26:59.243 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking network-manager-pptp (1.2.10-1) ... 2023-12-20T04:26:59.252 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up flex (2.6.4-8build2) ... 2023-12-20T04:26:59.378 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-20T04:26:59.504 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:26:59.546 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client-common. 2023-12-20T04:26:59.562 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../26-postgresql-client-common_238_all.deb ... 2023-12-20T04:26:59.605 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client-common (238) ... 2023-12-20T04:26:59.655 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-12-20T04:26:59.781 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-20T04:26:59.877 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client-14. 2023-12-20T04:26:59.893 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../27-postgresql-client-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:26:59.944 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:27:00.358 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:27:00.499 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-common. 2023-12-20T04:27:00.515 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../28-postgresql-common_238_all.deb ... 2023-12-20T04:27:00.576 INFO:teuthology.orchestra.run.smithi032.stdout:Adding 'diversion of /usr/bin/pg_config to /usr/bin/pg_config.libpq-dev by postgresql-common' 2023-12-20T04:27:00.606 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-common (238) ... 2023-12-20T04:27:01.048 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-14. 2023-12-20T04:27:01.064 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../29-postgresql-14_14.10-0ubuntu0.22.04.1_amd64.deb ... 2023-12-20T04:27:01.109 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:27:01.390 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-12-20T04:27:01.488 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-20T04:27:01.875 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-20T04:27:01.875 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-20T04:27:02.302 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql. 2023-12-20T04:27:02.317 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../30-postgresql_14+238_all.deb ... 2023-12-20T04:27:02.360 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql (14+238) ... 2023-12-20T04:27:02.600 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-12-20T04:27:02.604 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-client. 2023-12-20T04:27:02.619 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../31-postgresql-client_14+238_all.deb ... 2023-12-20T04:27:02.662 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-client (14+238) ... 2023-12-20T04:27:02.725 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-20T04:27:02.834 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-12-20T04:27:02.931 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package postgresql-contrib. 2023-12-20T04:27:02.946 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../32-postgresql-contrib_14+238_all.deb ... 2023-12-20T04:27:02.969 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-12-20T04:27:02.989 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking postgresql-contrib (14+238) ... 2023-12-20T04:27:03.087 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-12-20T04:27:03.288 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-20T04:27:03.309 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package tcl8.6. 2023-12-20T04:27:03.325 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../33-tcl8.6_8.6.12+dfsg-1build1_amd64.deb ... 2023-12-20T04:27:03.367 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-20T04:27:03.413 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-12-20T04:27:03.653 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package tcl. 2023-12-20T04:27:03.669 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../34-tcl_8.6.11+1build2_amd64.deb ... 2023-12-20T04:27:03.716 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking tcl (8.6.11+1build2) ... 2023-12-20T04:27:04.102 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-20T04:27:04.203 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up dump (0.4b47-3) ... 2023-12-20T04:27:04.275 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package wpasupplicant. 2023-12-20T04:27:04.291 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../35-wpasupplicant_2%3a2.10-6ubuntu2_amd64.deb ... 2023-12-20T04:27:04.298 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-12-20T04:27:04.333 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-20T04:27:04.354 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-20T04:27:04.480 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-20T04:27:04.607 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up tcl (8.6.11+1build2) ... 2023-12-20T04:27:04.679 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package usb-modeswitch-data. 2023-12-20T04:27:04.696 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../36-usb-modeswitch-data_20191128-4_all.deb ... 2023-12-20T04:27:04.738 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking usb-modeswitch-data (20191128-4) ... 2023-12-20T04:27:04.741 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libjson-perl (4.04000-1) ... 2023-12-20T04:27:04.867 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-12-20T04:27:04.957 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-12-20T04:27:05.027 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-20T04:27:05.049 INFO:teuthology.orchestra.run.smithi032.stdout:Selecting previously unselected package usb-modeswitch. 2023-12-20T04:27:05.066 INFO:teuthology.orchestra.run.smithi032.stdout:Preparing to unpack .../37-usb-modeswitch_2.6.1-3ubuntu2_amd64.deb ... 2023-12-20T04:27:05.116 INFO:teuthology.orchestra.run.smithi032.stdout:Unpacking usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-20T04:27:05.177 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-20T04:27:05.337 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-20T04:27:05.426 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client-common (238) ... 2023-12-20T04:27:05.463 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-20T04:27:05.589 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-12-20T04:27:05.614 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up flex (2.6.4-8build2) ... 2023-12-20T04:27:05.740 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up tcl8.6 (8.6.12+dfsg-1build1) ... 2023-12-20T04:27:05.757 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-client (14+238) ... 2023-12-20T04:27:05.866 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libpq5:amd64 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:27:05.883 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-12-20T04:27:05.992 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up liblzo2-2:amd64 (2.10-2build3) ... 2023-12-20T04:27:06.118 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libaio-dev:amd64 (0.3.112-13build1) ... 2023-12-20T04:27:06.244 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:27:06.465 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:06.465 INFO:teuthology.orchestra.run.smithi039.stdout:The following network interfaces were found in /etc/network/interfaces 2023-12-20T04:27:06.465 INFO:teuthology.orchestra.run.smithi039.stdout:which means they are currently configured by ifupdown: 2023-12-20T04:27:06.465 INFO:teuthology.orchestra.run.smithi039.stdout:- ens1f0 2023-12-20T04:27:06.465 INFO:teuthology.orchestra.run.smithi039.stdout:If you want to manage those interfaces with NetworkManager instead 2023-12-20T04:27:06.465 INFO:teuthology.orchestra.run.smithi039.stdout:remove their configuration from /etc/network/interfaces. 2023-12-20T04:27:06.465 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:06.605 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-12-20T04:27:07.017 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-12-20T04:27:07.269 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/psql.1.gz to provide /usr/share/man/man1/psql.1.gz (psql.1.gz) in auto mode 2023-12-20T04:27:07.367 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up wpasupplicant (2:2.10-6ubuntu2) ... 2023-12-20T04:27:07.433 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-12-20T04:27:07.762 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-fi.w1.wpa_supplicant1.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-20T04:27:07.763 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/wpa_supplicant.service -> /lib/systemd/system/wpa_supplicant.service. 2023-12-20T04:27:08.160 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-20T04:27:08.278 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-20T04:27:08.413 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-12-20T04:27:08.462 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libteamdctl0:amd64 (1.31-1build2) ... 2023-12-20T04:27:08.564 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-common (238) ... 2023-12-20T04:27:08.579 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libssl-dev:amd64 (3.0.2-0ubuntu1.12) ... 2023-12-20T04:27:08.689 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up usb-modeswitch-data (20191128-4) ... 2023-12-20T04:27:08.815 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libelf-dev:amd64 (0.186-1build1) ... 2023-12-20T04:27:08.974 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl2:amd64 (2.6.4-8build2) ... 2023-12-20T04:27:09.100 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libnm0:amd64 (1.36.6-0ubuntu2) ... 2023-12-20T04:27:09.235 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up ppp (2.4.9-1+1ubuntu3) ... 2023-12-20T04:27:09.471 INFO:teuthology.orchestra.run.smithi039.stdout:Adding user postgres to group ssl-cert 2023-12-20T04:27:09.775 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:09.775 INFO:teuthology.orchestra.run.smithi039.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-12-20T04:27:09.925 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libllvm14:amd64 (1:14.0.0-1ubuntu1.1) ... 2023-12-20T04:27:10.038 INFO:teuthology.orchestra.run.smithi039.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-12-20T04:27:10.038 INFO:teuthology.orchestra.run.smithi039.stdout:Removing obsolete dictionary files: 2023-12-20T04:27:10.050 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up dump (0.4b47-3) ... 2023-12-20T04:27:10.179 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/sbin/rmt-dump to provide /usr/sbin/rmt (rmt) in auto mode 2023-12-20T04:27:10.243 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmm-glib0:amd64 (1.20.0-1~ubuntu22.04.2) ... 2023-12-20T04:27:10.370 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libbluetooth3:amd64 (5.64-0ubuntu1.1) ... 2023-12-20T04:27:10.496 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up tcl (8.6.11+1build2) ... 2023-12-20T04:27:10.605 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libjson-perl (4.04000-1) ... 2023-12-20T04:27:10.706 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up bison (2:3.8.2+dfsg-1build1) ... 2023-12-20T04:27:10.796 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/bin/bison.yacc to provide /usr/bin/yacc (yacc) in auto mode 2023-12-20T04:27:10.797 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-12-20T04:27:10.858 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up indent (2.2.12-1ubuntu0.22.04.1) ... 2023-12-20T04:27:10.984 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up usb-modeswitch (2.6.1-3ubuntu2) ... 2023-12-20T04:27:11.161 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libndp0:amd64 (1.8-0ubuntu3) ... 2023-12-20T04:27:11.325 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmbim-glib4:amd64 (1.28.0-1~ubuntu20.04.1) ... 2023-12-20T04:27:11.348 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:27:11.454 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up pptp-linux (1.10.0-1build3) ... 2023-12-20T04:27:11.622 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-client (14+238) ... 2023-12-20T04:27:11.697 INFO:teuthology.orchestra.run.smithi039.stdout:Creating new PostgreSQL cluster 14/main ... 2023-12-20T04:27:11.720 INFO:teuthology.orchestra.run.smithi039.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-12-20T04:27:11.723 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up network-manager (1.36.6-0ubuntu2) ... 2023-12-20T04:27:11.775 INFO:teuthology.orchestra.run.smithi039.stdout:The files belonging to this database system will be owned by user "postgres". 2023-12-20T04:27:11.775 INFO:teuthology.orchestra.run.smithi039.stdout:This user must also own the server process. 2023-12-20T04:27:11.775 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:11.775 INFO:teuthology.orchestra.run.smithi039.stdout:The database cluster will be initialized with locale "en_US". 2023-12-20T04:27:11.775 INFO:teuthology.orchestra.run.smithi039.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-12-20T04:27:11.775 INFO:teuthology.orchestra.run.smithi039.stdout:The default text search configuration will be set to "english". 2023-12-20T04:27:11.775 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:11.776 INFO:teuthology.orchestra.run.smithi039.stdout:Data page checksums are disabled. 2023-12-20T04:27:11.776 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:11.776 INFO:teuthology.orchestra.run.smithi039.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-12-20T04:27:11.777 INFO:teuthology.orchestra.run.smithi039.stdout:creating subdirectories ... ok 2023-12-20T04:27:11.777 INFO:teuthology.orchestra.run.smithi039.stdout:selecting dynamic shared memory implementation ... posix 2023-12-20T04:27:11.802 INFO:teuthology.orchestra.run.smithi039.stdout:selecting default max_connections ... 100 2023-12-20T04:27:11.839 INFO:teuthology.orchestra.run.smithi039.stdout:selecting default shared_buffers ... 128MB 2023-12-20T04:27:11.853 INFO:teuthology.orchestra.run.smithi039.stdout:selecting default time zone ... Etc/UTC 2023-12-20T04:27:11.854 INFO:teuthology.orchestra.run.smithi039.stdout:creating configuration files ... ok 2023-12-20T04:27:12.044 INFO:teuthology.orchestra.run.smithi039.stdout:running bootstrap script ... ok 2023-12-20T04:27:12.304 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:12.304 INFO:teuthology.orchestra.run.smithi032.stdout:The following network interfaces were found in /etc/network/interfaces 2023-12-20T04:27:12.304 INFO:teuthology.orchestra.run.smithi032.stdout:which means they are currently configured by ifupdown: 2023-12-20T04:27:12.304 INFO:teuthology.orchestra.run.smithi032.stdout:- ens1f0 2023-12-20T04:27:12.304 INFO:teuthology.orchestra.run.smithi032.stdout:If you want to manage those interfaces with NetworkManager instead 2023-12-20T04:27:12.304 INFO:teuthology.orchestra.run.smithi032.stdout:remove their configuration from /etc/network/interfaces. 2023-12-20T04:27:12.304 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:12.445 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.nm-dispatcher.service -> /lib/systemd/system/NetworkManager-dispatcher.service. 2023-12-20T04:27:12.701 INFO:teuthology.orchestra.run.smithi039.stdout:performing post-bootstrap initialization ... ok 2023-12-20T04:27:12.856 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/network-online.target.wants/NetworkManager-wait-online.service -> /lib/systemd/system/NetworkManager-wait-online.service. 2023-12-20T04:27:13.289 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/NetworkManager.service -> /lib/systemd/system/NetworkManager.service. 2023-12-20T04:27:13.292 INFO:teuthology.orchestra.run.smithi039.stdout:syncing data to disk ... ok 2023-12-20T04:27:13.810 INFO:teuthology.orchestra.run.smithi039.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-12-20T04:27:14.034 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libmbim-proxy (1.28.0-1~ubuntu20.04.1) ... 2023-12-20T04:27:14.170 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libfl-dev:amd64 (2.6.4-8build2) ... 2023-12-20T04:27:14.295 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up network-manager-pptp (1.2.10-1) ... 2023-12-20T04:27:14.421 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-common (238) ... 2023-12-20T04:27:15.534 INFO:teuthology.orchestra.run.smithi032.stdout:Adding user postgres to group ssl-cert 2023-12-20T04:27:15.796 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:15.797 INFO:teuthology.orchestra.run.smithi032.stdout:Creating config file /etc/postgresql-common/createcluster.conf with new version 2023-12-20T04:27:16.068 INFO:teuthology.orchestra.run.smithi032.stdout:Building PostgreSQL dictionaries from installed myspell/hunspell packages... 2023-12-20T04:27:16.068 INFO:teuthology.orchestra.run.smithi032.stdout:Removing obsolete dictionary files: 2023-12-20T04:27:16.819 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/postgresql.service -> /lib/systemd/system/postgresql.service. 2023-12-20T04:27:16.972 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-20T04:27:17.097 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql-contrib (14+238) ... 2023-12-20T04:27:17.232 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up postgresql (14+238) ... 2023-12-20T04:27:17.358 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-20T04:27:17.415 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-14 (14.10-0ubuntu0.22.04.1) ... 2023-12-20T04:27:17.507 INFO:teuthology.orchestra.run.smithi039.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-20T04:27:17.753 INFO:teuthology.orchestra.run.smithi032.stdout:Creating new PostgreSQL cluster 14/main ... 2023-12-20T04:27:17.760 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-12-20T04:27:17.760 INFO:teuthology.orchestra.run.smithi039.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-12-20T04:27:17.777 INFO:teuthology.orchestra.run.smithi032.stdout:/usr/lib/postgresql/14/bin/initdb -D /var/lib/postgresql/14/main --auth-local peer --auth-host scram-sha-256 --no-instructions 2023-12-20T04:27:17.831 INFO:teuthology.orchestra.run.smithi032.stdout:The files belonging to this database system will be owned by user "postgres". 2023-12-20T04:27:17.831 INFO:teuthology.orchestra.run.smithi032.stdout:This user must also own the server process. 2023-12-20T04:27:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:17.832 INFO:teuthology.orchestra.run.smithi032.stdout:The database cluster will be initialized with locale "en_US". 2023-12-20T04:27:17.832 INFO:teuthology.orchestra.run.smithi032.stdout:The default database encoding has accordingly been set to "LATIN1". 2023-12-20T04:27:17.832 INFO:teuthology.orchestra.run.smithi032.stdout:The default text search configuration will be set to "english". 2023-12-20T04:27:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:17.832 INFO:teuthology.orchestra.run.smithi032.stdout:Data page checksums are disabled. 2023-12-20T04:27:17.832 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:17.832 INFO:teuthology.orchestra.run.smithi032.stdout:fixing permissions on existing directory /var/lib/postgresql/14/main ... ok 2023-12-20T04:27:17.833 INFO:teuthology.orchestra.run.smithi032.stdout:creating subdirectories ... ok 2023-12-20T04:27:17.833 INFO:teuthology.orchestra.run.smithi032.stdout:selecting dynamic shared memory implementation ... posix 2023-12-20T04:27:17.858 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default max_connections ... 100 2023-12-20T04:27:17.894 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default shared_buffers ... 128MB 2023-12-20T04:27:17.908 INFO:teuthology.orchestra.run.smithi032.stdout:selecting default time zone ... Etc/UTC 2023-12-20T04:27:17.909 INFO:teuthology.orchestra.run.smithi032.stdout:creating configuration files ... ok 2023-12-20T04:27:18.088 INFO:teuthology.orchestra.run.smithi032.stdout:running bootstrap script ... ok 2023-12-20T04:27:18.511 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-12-20T04:27:18.717 INFO:teuthology.orchestra.run.smithi032.stdout:performing post-bootstrap initialization ... ok 2023-12-20T04:27:18.855 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-20T04:27:19.006 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-20T04:27:19.302 INFO:teuthology.orchestra.run.smithi032.stdout:syncing data to disk ... ok 2023-12-20T04:27:19.810 INFO:teuthology.orchestra.run.smithi032.stdout:update-alternatives: using /usr/share/postgresql/14/man/man1/postmaster.1.gz to provide /usr/share/man/man1/postmaster.1.gz (postmaster.1.gz) in auto mode 2023-12-20T04:27:20.469 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-12-20T04:27:20.569 INFO:teuthology.orchestra.run.smithi039.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-12-20T04:27:21.258 INFO:teuthology.orchestra.run.smithi039.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-20T04:27:21.264 DEBUG:teuthology.parallel:result is None 2023-12-20T04:27:23.015 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqmi-glib5:amd64 (1.32.0-1ubuntu0.22.04.1) ... 2023-12-20T04:27:23.141 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql-contrib (14+238) ... 2023-12-20T04:27:23.258 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up postgresql (14+238) ... 2023-12-20T04:27:23.368 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up libqmi-proxy (1.32.0-1ubuntu0.22.04.1) ... 2023-12-20T04:27:23.527 INFO:teuthology.orchestra.run.smithi032.stdout:Setting up modemmanager (1.20.0-1~ubuntu22.04.2) ... 2023-12-20T04:27:23.798 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/dbus-org.freedesktop.ModemManager1.service -> /lib/systemd/system/ModemManager.service. 2023-12-20T04:27:23.798 INFO:teuthology.orchestra.run.smithi032.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ModemManager.service -> /lib/systemd/system/ModemManager.service. 2023-12-20T04:27:24.571 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for hicolor-icon-theme (0.17-2) ... 2023-12-20T04:27:24.916 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for libc-bin (2.35-0ubuntu3.1) ... 2023-12-20T04:27:25.058 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for man-db (2.10.2-1) ... 2023-12-20T04:27:26.495 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for dbus (1.12.20-2ubuntu4.1) ... 2023-12-20T04:27:26.595 INFO:teuthology.orchestra.run.smithi032.stdout:Processing triggers for install-info (6.8-4build1) ... 2023-12-20T04:27:27.286 INFO:teuthology.orchestra.run.smithi032.stderr:W: --force-yes is deprecated, use one of the options starting with --allow instead. 2023-12-20T04:27:27.293 DEBUG:teuthology.parallel:result is None 2023-12-20T04:27:27.294 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-20T04:27:27.439 DEBUG:teuthology.orchestra.run.smithi032:> dpkg-query -W -f '${Version}' ceph 2023-12-20T04:27:27.463 INFO:teuthology.orchestra.run.smithi032.stdout:18.2.1-132-g568df92b-1jammy 2023-12-20T04:27:27.463 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-132-g568df92b-1jammy 2023-12-20T04:27:27.463 INFO:teuthology.task.install:The correct ceph version 18.2.1-132-g568df92b-1jammy is installed. 2023-12-20T04:27:27.465 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=ubuntu%2F22.04%2Fx86_64&sha1=568df92b7e50b2108bce4c77c393ff294d6a5e5e 2023-12-20T04:27:27.813 DEBUG:teuthology.orchestra.run.smithi039:> dpkg-query -W -f '${Version}' ceph 2023-12-20T04:27:27.838 INFO:teuthology.orchestra.run.smithi039.stdout:18.2.1-132-g568df92b-1jammy 2023-12-20T04:27:27.838 INFO:teuthology.packaging:The installed version of ceph is 18.2.1-132-g568df92b-1jammy 2023-12-20T04:27:27.838 INFO:teuthology.task.install:The correct ceph version 18.2.1-132-g568df92b-1jammy is installed. 2023-12-20T04:27:27.840 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2023-12-20T04:27:27.840 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:27:27.840 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-20T04:27:27.862 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:27:27.862 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2023-12-20T04:27:27.903 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2023-12-20T04:27:27.904 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:27:27.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/daemon-helper 2023-12-20T04:27:27.926 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-20T04:27:27.990 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:27:27.990 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/daemon-helper 2023-12-20T04:27:28.013 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/daemon-helper 2023-12-20T04:27:28.082 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2023-12-20T04:27:28.082 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:27:28.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-20T04:27:28.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-20T04:27:28.166 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:27:28.168 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/adjust-ulimits 2023-12-20T04:27:28.190 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2023-12-20T04:27:28.254 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2023-12-20T04:27:28.255 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:27:28.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/usr/bin/stdin-killer 2023-12-20T04:27:28.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-20T04:27:28.337 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:27:28.338 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/usr/bin/stdin-killer 2023-12-20T04:27:28.358 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod a=rx -- /usr/bin/stdin-killer 2023-12-20T04:27:28.421 INFO:teuthology.run_tasks:Running task ceph... 2023-12-20T04:27:28.493 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2023-12-20T04:27:28.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 777 /var/log/ceph 2023-12-20T04:27:28.497 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 777 /var/log/ceph 2023-12-20T04:27:28.519 INFO:tasks.ceph:Disabling ceph logrotate... 2023-12-20T04:27:28.519 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-20T04:27:28.560 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/logrotate.d/ceph 2023-12-20T04:27:28.584 INFO:tasks.ceph:Creating extra log directories... 2023-12-20T04:27:28.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-20T04:27:28.625 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2023-12-20T04:27:28.655 INFO:tasks.ceph:Setting up log rotation with {'ceph-mds': '10G', 'ceph-osd': '10G'} 2023-12-20T04:27:28.655 INFO:tasks.ceph:writing logrotate stanza for ceph-mds 2023-12-20T04:27:28.656 INFO:tasks.ceph:writing logrotate stanza for ceph-osd 2023-12-20T04:27:28.656 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:27:28.656 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-20T04:27:28.694 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-20T04:27:28.788 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:27:28.789 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/logrotate.ceph-test.conf 2023-12-20T04:27:28.798 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv /home/ubuntu/cephtest/logrotate.ceph-test.conf /etc/logrotate.d/ceph-test.conf && sudo chmod 0644 /etc/logrotate.d/ceph-test.conf && sudo chown root.root /etc/logrotate.d/ceph-test.conf 2023-12-20T04:27:28.904 INFO:tasks.ceph:Creating ceph cluster ceph... 2023-12-20T04:27:28.904 INFO:tasks.ceph:config {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2023-12-20T04:27:28.904 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395', 'branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2', 'description': 'fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule}', 'email': 'vshankar@redhat.com', 'first_in_suite': False, 'job_id': '7497395', 'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}, 'ktype': 'upstream', 'last_in_suite': False, 'log-rotate': {'ceph-mds': '10G', 'ceph-osd': '10G'}, 'machine_type': 'smithi', 'name': 'vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi', 'no_nested_subset': False, 'nuke-on-error': True, 'openstack': [{'volumes': {'count': 4, 'size': 20}}, {'machine': {'disk': 200}}], 'os_type': 'ubuntu', 'os_version': '22.04', 'overrides': {'admin_socket': {'branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2'}, 'ceph': {'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}, 'ceph-deploy': {'bluestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore'}}, 'fs': 'xfs'}, 'install': {'ceph': {'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}}, 'kclient': {'syntax': 'v2'}, 'thrashosds': {'bdev_inject_crash': 2, 'bdev_inject_crash_probability': 0.5}, 'workunit': {'branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}}, 'owner': 'scheduled_vshankar@teuthology', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.y', 'mds.a', 'mds.c', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0', 'client.2'], ['mon.b', 'mon.c', 'mgr.x', 'mds.b', 'mds.d', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1', 'client.3']], 'seed': 430, 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e', 'sleep_before_teardown': 0, 'subset': '301/1024', 'suite': 'fs', 'suite_branch': 'wip-vshankar-testing1-2023-12-18-1207-reef-2', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_568df92b7e50b2108bce4c77c393ff294d6a5e5e/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e', 'syslog': {'ignorelist': ['WARNING*.*check_session_state', 'WARNING*.*__ceph_remove_cap']}, 'targets': {'smithi032.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMVliqZJMAR+8i1UIgvLMOW2HwRq2iziIYQuIUOiaOB3RW74bqK8M0Px+7m7U1vi9gSVfien5K+r+Pq5a8Qk7kI=', 'smithi039.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLNmc+OQbgX+2td9QV+1fnzDuUIn09Fw7zDPWqNzdyN1TBzSxdTCN0yAAQXLDFQO9E0yC2E05BGvJuuIhqwybGY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'client': {'branch': 'testing'}, 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e'}}, {'ceph': {'create_rbd_pool': False, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'debug client': 20, 'debug finisher': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon op complaint time': 120}, 'osd': {'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'overall HEALTH_', '\\(FS_DEGRADED\\)', '\\(MDS_FAILED\\)', '\\(MDS_DEGRADED\\)', '\\(FS_WITH_FAILED_MDS\\)', '\\(MDS_DAMAGE\\)', '\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(FS_INLINE_DATA_DEPRECATED\\)', '\\(POOL_APP_NOT_ENABLED\\)', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding'], 'log-whitelist': ['OSD full dropping all updates', 'OSD near full', 'pausewr flag', 'failsafe engaged, dropping updates', 'failsafe disengaged, no longer dropping', 'is full \\(reached quota', 'POOL_FULL', 'POOL_BACKFILLFULL'], 'sha1': '568df92b7e50b2108bce4c77c393ff294d6a5e5e', 'cluster': 'ceph'}}, {'kclient': None}, {'cephfs_test_runner': {'modules': ['tasks.cephfs.test_snap_schedules']}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': ["local function is_kupstream() return yaml.ktype == 'upstream' end", "local function is_kdistro() return yaml.ktype == 'distro' end", 'local function is_fuse() return false end', 'local function syntax_version() return yaml.overrides.kclient.syntax end']}, 'teuthology_branch': 'main', 'teuthology_sha1': '5f3731ee8689ef3b0ef7d571e4cc2cb968c78d73', 'timestamp': '2023-12-20_03:59:06', 'tube': 'smithi', 'use_shaman': True, 'user': 'vshankar', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/dispatcher.smithi.1485078'} 2023-12-20T04:27:28.905 DEBUG:teuthology.orchestra.run.smithi032:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-20T04:27:28.909 DEBUG:teuthology.orchestra.run.smithi039:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2023-12-20T04:27:28.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo install -d -m0777 -- /var/run/ceph 2023-12-20T04:27:28.965 DEBUG:teuthology.orchestra.run.smithi039:> sudo install -d -m0777 -- /var/run/ceph 2023-12-20T04:27:29.014 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:27:29.015 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/scratch_devs of=/dev/stdout 2023-12-20T04:27:29.034 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-20T04:27:29.034 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_1 2023-12-20T04:27:29.086 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-20T04:27:29.086 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-20T04:27:29.086 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 904 Links: 1 2023-12-20T04:27:29.086 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-20T04:27:29.086 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-12-20 04:21:02.039012680 +0000 2023-12-20T04:27:29.086 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-12-20 04:21:01.607019271 +0000 2023-12-20T04:27:29.086 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-12-20 04:21:01.607019271 +0000 2023-12-20T04:27:29.087 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-12-20 04:21:01.607019271 +0000 2023-12-20T04:27:29.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-20T04:27:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-12-20T04:27:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-12-20T04:27:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000254698 s, 2.0 MB/s 2023-12-20T04:27:29.154 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-20T04:27:29.208 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_2 2023-12-20T04:27:29.258 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-20T04:27:29.258 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-20T04:27:29.259 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 921 Links: 1 2023-12-20T04:27:29.259 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-20T04:27:29.259 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-12-20 04:21:02.282991224 +0000 2023-12-20T04:27:29.259 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-12-20 04:21:02.278991576 +0000 2023-12-20T04:27:29.259 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-12-20 04:21:02.278991576 +0000 2023-12-20T04:27:29.259 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-12-20 04:21:02.278991576 +0000 2023-12-20T04:27:29.259 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-20T04:27:29.321 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-12-20T04:27:29.322 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-12-20T04:27:29.322 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000264664 s, 1.9 MB/s 2023-12-20T04:27:29.323 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-20T04:27:29.376 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_3 2023-12-20T04:27:29.427 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-20T04:27:29.427 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-20T04:27:29.427 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 938 Links: 1 2023-12-20T04:27:29.427 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-20T04:27:29.427 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-12-20 04:21:03.366896992 +0000 2023-12-20T04:27:29.427 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-12-20 04:21:02.962931430 +0000 2023-12-20T04:27:29.427 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-12-20 04:21:02.962931430 +0000 2023-12-20T04:27:29.427 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-12-20 04:21:02.962931430 +0000 2023-12-20T04:27:29.427 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-20T04:27:29.489 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-12-20T04:27:29.490 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-12-20T04:27:29.490 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000270691 s, 1.9 MB/s 2023-12-20T04:27:29.491 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-20T04:27:29.544 DEBUG:teuthology.orchestra.run.smithi032:> stat /dev/vg_nvme/lv_4 2023-12-20T04:27:29.602 INFO:teuthology.orchestra.run.smithi032.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-20T04:27:29.602 INFO:teuthology.orchestra.run.smithi032.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-20T04:27:29.602 INFO:teuthology.orchestra.run.smithi032.stdout:Device: 5h/5d Inode: 952 Links: 1 2023-12-20T04:27:29.602 INFO:teuthology.orchestra.run.smithi032.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-20T04:27:29.603 INFO:teuthology.orchestra.run.smithi032.stdout:Access: 2023-12-20 04:21:04.042839656 +0000 2023-12-20T04:27:29.603 INFO:teuthology.orchestra.run.smithi032.stdout:Modify: 2023-12-20 04:21:03.598877288 +0000 2023-12-20T04:27:29.603 INFO:teuthology.orchestra.run.smithi032.stdout:Change: 2023-12-20 04:21:03.598877288 +0000 2023-12-20T04:27:29.603 INFO:teuthology.orchestra.run.smithi032.stdout: Birth: 2023-12-20 04:21:03.598877288 +0000 2023-12-20T04:27:29.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-20T04:27:29.670 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records in 2023-12-20T04:27:29.670 INFO:teuthology.orchestra.run.smithi032.stderr:1+0 records out 2023-12-20T04:27:29.671 INFO:teuthology.orchestra.run.smithi032.stderr:512 bytes copied, 0.000243092 s, 2.1 MB/s 2023-12-20T04:27:29.672 DEBUG:teuthology.orchestra.run.smithi032:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-20T04:27:29.728 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-20T04:27:29.729 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:27:29.729 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/scratch_devs of=/dev/stdout 2023-12-20T04:27:29.739 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2023-12-20T04:27:29.739 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_1 2023-12-20T04:27:29.790 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2023-12-20T04:27:29.791 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-20T04:27:29.791 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 878 Links: 1 2023-12-20T04:27:29.791 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-20T04:27:29.791 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-12-20 04:21:04.152656213 +0000 2023-12-20T04:27:29.791 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-12-20 04:21:04.148656249 +0000 2023-12-20T04:27:29.791 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-12-20 04:21:04.148656249 +0000 2023-12-20T04:27:29.791 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2023-12-20 04:21:04.148656249 +0000 2023-12-20T04:27:29.791 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2023-12-20T04:27:29.859 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-12-20T04:27:29.859 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-12-20T04:27:29.859 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000515964 s, 992 kB/s 2023-12-20T04:27:29.861 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2023-12-20T04:27:29.912 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_2 2023-12-20T04:27:29.962 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2023-12-20T04:27:29.963 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-20T04:27:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 893 Links: 1 2023-12-20T04:27:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-20T04:27:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-12-20 04:21:05.284646102 +0000 2023-12-20T04:27:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-12-20 04:21:04.880649711 +0000 2023-12-20T04:27:29.963 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-12-20 04:21:04.880649711 +0000 2023-12-20T04:27:29.963 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2023-12-20 04:21:04.880649711 +0000 2023-12-20T04:27:29.963 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2023-12-20T04:27:30.026 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-12-20T04:27:30.026 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-12-20T04:27:30.026 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000468916 s, 1.1 MB/s 2023-12-20T04:27:30.028 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2023-12-20T04:27:30.080 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_3 2023-12-20T04:27:30.131 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2023-12-20T04:27:30.131 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-20T04:27:30.131 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 910 Links: 1 2023-12-20T04:27:30.131 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-20T04:27:30.131 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-12-20 04:21:05.968639991 +0000 2023-12-20T04:27:30.131 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-12-20 04:21:05.552643707 +0000 2023-12-20T04:27:30.131 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-12-20 04:21:05.552643707 +0000 2023-12-20T04:27:30.131 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2023-12-20 04:21:05.552643707 +0000 2023-12-20T04:27:30.131 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2023-12-20T04:27:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-12-20T04:27:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-12-20T04:27:30.198 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000804864 s, 636 kB/s 2023-12-20T04:27:30.200 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2023-12-20T04:27:30.252 DEBUG:teuthology.orchestra.run.smithi039:> stat /dev/vg_nvme/lv_4 2023-12-20T04:27:30.302 INFO:teuthology.orchestra.run.smithi039.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2023-12-20T04:27:30.302 INFO:teuthology.orchestra.run.smithi039.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2023-12-20T04:27:30.302 INFO:teuthology.orchestra.run.smithi039.stdout:Device: 5h/5d Inode: 930 Links: 1 2023-12-20T04:27:30.302 INFO:teuthology.orchestra.run.smithi039.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2023-12-20T04:27:30.302 INFO:teuthology.orchestra.run.smithi039.stdout:Access: 2023-12-20 04:21:06.652633882 +0000 2023-12-20T04:27:30.303 INFO:teuthology.orchestra.run.smithi039.stdout:Modify: 2023-12-20 04:21:06.244637526 +0000 2023-12-20T04:27:30.303 INFO:teuthology.orchestra.run.smithi039.stdout:Change: 2023-12-20 04:21:06.244637526 +0000 2023-12-20T04:27:30.303 INFO:teuthology.orchestra.run.smithi039.stdout: Birth: 2023-12-20 04:21:06.244637526 +0000 2023-12-20T04:27:30.303 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2023-12-20T04:27:30.365 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records in 2023-12-20T04:27:30.365 INFO:teuthology.orchestra.run.smithi039.stderr:1+0 records out 2023-12-20T04:27:30.366 INFO:teuthology.orchestra.run.smithi039.stderr:512 bytes copied, 0.000521619 s, 982 kB/s 2023-12-20T04:27:30.367 DEBUG:teuthology.orchestra.run.smithi039:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2023-12-20T04:27:30.421 INFO:tasks.ceph:osd dev map: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-20T04:27:30.421 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-20T04:27:30.421 INFO:tasks.ceph:Generating config... 2023-12-20T04:27:30.422 INFO:tasks.ceph:[client] client mount timeout = 600 2023-12-20T04:27:30.422 INFO:tasks.ceph:[client] debug client = 20 2023-12-20T04:27:30.422 INFO:tasks.ceph:[client] debug ms = 1 2023-12-20T04:27:30.422 INFO:tasks.ceph:[client] pid file = 2023-12-20T04:27:30.423 INFO:tasks.ceph:[client] rados mon op timeout = 900 2023-12-20T04:27:30.423 INFO:tasks.ceph:[client] rados osd op timeout = 900 2023-12-20T04:27:30.423 INFO:tasks.ceph:[global] ms die on skipped message = False 2023-12-20T04:27:30.423 INFO:tasks.ceph:[mds] debug mds = 20 2023-12-20T04:27:30.423 INFO:tasks.ceph:[mds] debug mds balancer = 20 2023-12-20T04:27:30.423 INFO:tasks.ceph:[mds] debug ms = 1 2023-12-20T04:27:30.423 INFO:tasks.ceph:[mds] mds debug frag = True 2023-12-20T04:27:30.423 INFO:tasks.ceph:[mds] mds debug scatterstat = True 2023-12-20T04:27:30.423 INFO:tasks.ceph:[mds] mds op complaint time = 180 2023-12-20T04:27:30.423 INFO:tasks.ceph:[mds] mds verify scatter = True 2023-12-20T04:27:30.423 INFO:tasks.ceph:[mds] osd op complaint time = 180 2023-12-20T04:27:30.423 INFO:tasks.ceph:[mds] rados mon op timeout = 900 2023-12-20T04:27:30.423 INFO:tasks.ceph:[mds] rados osd op timeout = 900 2023-12-20T04:27:30.424 INFO:tasks.ceph:[mgr] debug client = 20 2023-12-20T04:27:30.424 INFO:tasks.ceph:[mgr] debug finisher = 20 2023-12-20T04:27:30.424 INFO:tasks.ceph:[mgr] debug mgr = 20 2023-12-20T04:27:30.424 INFO:tasks.ceph:[mgr] debug ms = 1 2023-12-20T04:27:30.424 INFO:tasks.ceph:[mon] debug mon = 20 2023-12-20T04:27:30.424 INFO:tasks.ceph:[mon] debug ms = 1 2023-12-20T04:27:30.424 INFO:tasks.ceph:[mon] debug paxos = 20 2023-12-20T04:27:30.424 INFO:tasks.ceph:[mon] mon op complaint time = 120 2023-12-20T04:27:30.424 INFO:tasks.ceph:[osd] bdev async discard = True 2023-12-20T04:27:30.424 INFO:tasks.ceph:[osd] bdev enable discard = True 2023-12-20T04:27:30.424 INFO:tasks.ceph:[osd] bluestore allocator = bitmap 2023-12-20T04:27:30.424 INFO:tasks.ceph:[osd] bluestore block size = 96636764160 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] bluestore fsck on mount = True 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] debug bluefs = 1/20 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] debug bluestore = 1/20 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] debug ms = 1 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] debug osd = 20 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] debug rocksdb = 4/10 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] mon osd backfillfull_ratio = 0.85 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] mon osd full ratio = 0.9 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] mon osd nearfull ratio = 0.8 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] osd failsafe full ratio = 0.95 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] osd objectstore = bluestore 2023-12-20T04:27:30.425 INFO:tasks.ceph:[osd] osd op complaint time = 180 2023-12-20T04:27:30.425 INFO:tasks.ceph:Setting up mon.a... 2023-12-20T04:27:30.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2023-12-20T04:27:30.474 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.keyring 2023-12-20T04:27:30.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2023-12-20T04:27:30.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-20T04:27:30.566 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.32'), ('mon.b', '172.21.15.39'), ('mon.c', '[v2:172.21.15.39:3301,v1:172.21.15.39:6790]')] 2023-12-20T04:27:30.566 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '172.21.15.32,172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790]', 'ms die on skipped message': False}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'bdev async discard': True, 'bdev enable discard': True, 'bluestore allocator': 'bitmap', 'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': '1/20', 'debug bluestore': '1/20', 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': '4/10', 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false', 'debug client': 20, 'debug finisher': 20}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon op complaint time': 120}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok', 'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'pid file': '', 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mon.a': {}, 'mon.b': {}, 'mon.c': {}, 'mds.a': {}, 'mds.c': {}, 'mds.b': {}, 'mds.d': {}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}} 2023-12-20T04:27:30.567 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:27:30.567 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2023-12-20T04:27:30.621 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 172.21.15.32 --add b 172.21.15.39 --addv c '[v2:172.21.15.39:3301,v1:172.21.15.39:6790]' --print /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:27:30.714 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:27:30.714 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: generated fsid 9b496156-2b6f-4897-9af4-fbc686ad55e7 2023-12-20T04:27:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:setting min_mon_release = pacific 2023-12-20T04:27:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:epoch 0 2023-12-20T04:27:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:fsid 9b496156-2b6f-4897-9af4-fbc686ad55e7 2023-12-20T04:27:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:last_changed 2023-12-20T04:27:30.717071+0000 2023-12-20T04:27:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:created 2023-12-20T04:27:30.717071+0000 2023-12-20T04:27:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:min_mon_release 16 (pacific) 2023-12-20T04:27:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:election_strategy: 1 2023-12-20T04:27:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:0: [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] mon.a 2023-12-20T04:27:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:1: [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] mon.b 2023-12-20T04:27:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:2: [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] mon.c 2023-12-20T04:27:30.715 INFO:teuthology.orchestra.run.smithi032.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2023-12-20T04:27:30.717 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2023-12-20T04:27:30.726 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9b496156-2b6f-4897-9af4-fbc686ad55e7... 2023-12-20T04:27:30.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-20T04:27:30.772 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2023-12-20T04:27:30.815 INFO:teuthology.orchestra.run.smithi032.stdout:[global] 2023-12-20T04:27:30.815 INFO:teuthology.orchestra.run.smithi032.stdout: chdir = "" 2023-12-20T04:27:30.815 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-20T04:27:30.815 INFO:teuthology.orchestra.run.smithi032.stdout: auth supported = cephx 2023-12-20T04:27:30.815 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.815 INFO:teuthology.orchestra.run.smithi032.stdout: filestore xattr use omap = true 2023-12-20T04:27:30.815 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: mon clock drift allowed = 1.000 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: osd crush chooseleaf type = 0 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: auth debug = true 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on old message = true 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on bug = true 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: mon pg warn max object skew = 0 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: # disable pg_autoscaler by default for new pools 2023-12-20T04:27:30.816 INFO:teuthology.orchestra.run.smithi032.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default size = 2 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow primary affinity = true 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd allow pg remap = true 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on legacy crush tunables = false 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on crush straw calc version zero = false 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on no sortbitwise = false 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on osd down out interval zero = false 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: mon warn on too few osds = false 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: mon_allow_pool_size_one = true 2023-12-20T04:27:30.817 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: osd default data pool replay window = 5 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: mon allow pool delete = true 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: mon cluster log file level = debug 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: debug asserts on shutdown = true 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: mon health detail to clog = false 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: mon host = "172.21.15.32,172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790]" 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: ms die on skipped message = False 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: fsid = 9b496156-2b6f-4897-9af4-fbc686ad55e7 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.818 INFO:teuthology.orchestra.run.smithi032.stdout:[osd] 2023-12-20T04:27:30.819 INFO:teuthology.orchestra.run.smithi032.stdout: osd journal size = 100 2023-12-20T04:27:30.819 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.819 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub load threshold = 5.0 2023-12-20T04:27:30.819 INFO:teuthology.orchestra.run.smithi032.stdout: osd scrub max interval = 600 2023-12-20T04:27:30.819 INFO:teuthology.orchestra.run.smithi032.stdout: osd mclock profile = high_recovery_ops 2023-12-20T04:27:30.819 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.819 INFO:teuthology.orchestra.run.smithi032.stdout: osd recover clone overlap = true 2023-12-20T04:27:30.819 INFO:teuthology.orchestra.run.smithi032.stdout: osd recovery max chunk = 1048576 2023-12-20T04:27:30.819 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.820 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug shutdown = true 2023-12-20T04:27:30.820 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug op order = true 2023-12-20T04:27:30.820 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug verify stray on activate = true 2023-12-20T04:27:30.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.820 INFO:teuthology.orchestra.run.smithi032.stdout: osd open classes on start = true 2023-12-20T04:27:30.820 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug pg log writeout = true 2023-12-20T04:27:30.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.820 INFO:teuthology.orchestra.run.smithi032.stdout: osd deep scrub update digest min age = 30 2023-12-20T04:27:30.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.820 INFO:teuthology.orchestra.run.smithi032.stdout: osd map max advance = 10 2023-12-20T04:27:30.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: journal zero on create = true 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: filestore ondisk finisher threads = 3 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: filestore apply finisher threads = 3 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: bdev debug aio = true 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: osd debug misdirected ops = true 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: bdev async discard = True 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: bdev enable discard = True 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore allocator = bitmap 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore block size = 96636764160 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: bluestore fsck on mount = True 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluefs = 1/20 2023-12-20T04:27:30.821 INFO:teuthology.orchestra.run.smithi032.stdout: debug bluestore = 1/20 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: debug osd = 20 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: debug rocksdb = 4/10 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd backfillfull_ratio = 0.85 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd full ratio = 0.9 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd nearfull ratio = 0.8 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: osd failsafe full ratio = 0.95 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: osd objectstore = bluestore 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr] 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: debug mgr = 20 2023-12-20T04:27:30.822 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2023-12-20T04:27:30.825 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2023-12-20T04:27:30.825 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2023-12-20T04:27:30.825 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2023-12-20T04:27:30.825 INFO:teuthology.orchestra.run.smithi032.stdout: mgr/telemetry/nag = false 2023-12-20T04:27:30.825 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2023-12-20T04:27:30.825 INFO:teuthology.orchestra.run.smithi032.stdout: debug finisher = 20 2023-12-20T04:27:30.825 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.825 INFO:teuthology.orchestra.run.smithi032.stdout:[mon] 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: debug mon = 20 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: debug paxos = 20 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: debug auth = 20 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: mon data avail warn = 5 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: mon mgr mkfs grace = 240 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min pgs per osd = 4 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd reporter subtree level = osd 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: mon osd prime pg temp = true 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: mon reweight min bytes per osd = 10 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: auth mon ticket ttl = 660 # 11m 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: auth service ticket ttl = 240 # 4m 2023-12-20T04:27:30.826 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: # don't complain about insecure global_id in the test suite 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: # 1m isn't quite enough 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: mon_down_mkfs_grace = 2m 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: mon_warn_on_filestore_osds = false 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: mon op complaint time = 120 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout:[client] 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: rgw cache enabled = true 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable ops log = true 2023-12-20T04:27:30.827 INFO:teuthology.orchestra.run.smithi032.stdout: rgw enable usage log = true 2023-12-20T04:27:30.828 INFO:teuthology.orchestra.run.smithi032.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-20T04:27:30.828 INFO:teuthology.orchestra.run.smithi032.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-20T04:27:30.828 INFO:teuthology.orchestra.run.smithi032.stdout: client mount timeout = 600 2023-12-20T04:27:30.828 INFO:teuthology.orchestra.run.smithi032.stdout: debug client = 20 2023-12-20T04:27:30.828 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-12-20T04:27:30.828 INFO:teuthology.orchestra.run.smithi032.stdout: pid file = "" 2023-12-20T04:27:30.828 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2023-12-20T04:27:30.828 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2023-12-20T04:27:30.828 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.a] 2023-12-20T04:27:30.828 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.b] 2023-12-20T04:27:30.828 INFO:teuthology.orchestra.run.smithi032.stdout:[mon.c] 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout:[mds] 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds = 20 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout: debug mds balancer = 20 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout: debug ms = 1 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug frag = True 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout: mds debug scatterstat = True 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout: mds op complaint time = 180 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout: mds verify scatter = True 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout: osd op complaint time = 180 2023-12-20T04:27:30.829 INFO:teuthology.orchestra.run.smithi032.stdout: rados mon op timeout = 900 2023-12-20T04:27:30.830 INFO:teuthology.orchestra.run.smithi032.stdout: rados osd op timeout = 900 2023-12-20T04:27:30.831 INFO:teuthology.orchestra.run.smithi039.stdout:[global] 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: chdir = "" 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: auth supported = cephx 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: filestore xattr use omap = true 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: mon clock drift allowed = 1.000 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: osd crush chooseleaf type = 0 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: auth debug = true 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on old message = true 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on bug = true 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.832 INFO:teuthology.orchestra.run.smithi039.stdout: mon max pg per osd = 10000 # >= luminous 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: mon pg warn max object skew = 0 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: # disable pg_autoscaler by default for new pools 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: osd_pool_default_pg_autoscale_mode = off 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default size = 2 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow primary affinity = true 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd allow pg remap = true 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on legacy crush tunables = false 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on crush straw calc version zero = false 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on no sortbitwise = false 2023-12-20T04:27:30.833 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on osd down out interval zero = false 2023-12-20T04:27:30.834 INFO:teuthology.orchestra.run.smithi039.stdout: mon warn on too few osds = false 2023-12-20T04:27:30.834 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2023-12-20T04:27:30.834 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_pool_no_redundancy = false 2023-12-20T04:27:30.834 INFO:teuthology.orchestra.run.smithi039.stdout: mon_allow_pool_size_one = true 2023-12-20T04:27:30.834 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.834 INFO:teuthology.orchestra.run.smithi039.stdout: osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2023-12-20T04:27:30.834 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.834 INFO:teuthology.orchestra.run.smithi039.stdout: osd default data pool replay window = 5 2023-12-20T04:27:30.834 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.834 INFO:teuthology.orchestra.run.smithi039.stdout: mon allow pool delete = true 2023-12-20T04:27:30.834 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.835 INFO:teuthology.orchestra.run.smithi039.stdout: mon cluster log file level = debug 2023-12-20T04:27:30.835 INFO:teuthology.orchestra.run.smithi039.stdout: debug asserts on shutdown = true 2023-12-20T04:27:30.835 INFO:teuthology.orchestra.run.smithi039.stdout: mon health detail to clog = false 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: mon host = "172.21.15.32,172.21.15.39,[v2:172.21.15.39:3301,v1:172.21.15.39:6790]" 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: ms die on skipped message = False 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: fsid = 9b496156-2b6f-4897-9af4-fbc686ad55e7 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout:[osd] 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: osd journal size = 100 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub load threshold = 5.0 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: osd scrub max interval = 600 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: osd mclock profile = high_recovery_ops 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: osd recover clone overlap = true 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: osd recovery max chunk = 1048576 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug shutdown = true 2023-12-20T04:27:30.836 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug op order = true 2023-12-20T04:27:30.837 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug verify stray on activate = true 2023-12-20T04:27:30.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.837 INFO:teuthology.orchestra.run.smithi039.stdout: osd open classes on start = true 2023-12-20T04:27:30.837 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug pg log writeout = true 2023-12-20T04:27:30.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.837 INFO:teuthology.orchestra.run.smithi039.stdout: osd deep scrub update digest min age = 30 2023-12-20T04:27:30.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.837 INFO:teuthology.orchestra.run.smithi039.stdout: osd map max advance = 10 2023-12-20T04:27:30.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.837 INFO:teuthology.orchestra.run.smithi039.stdout: journal zero on create = true 2023-12-20T04:27:30.837 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: filestore ondisk finisher threads = 3 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: filestore apply finisher threads = 3 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: bdev debug aio = true 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: osd debug misdirected ops = true 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: bdev async discard = True 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: bdev enable discard = True 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore allocator = bitmap 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore block size = 96636764160 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: bluestore fsck on mount = True 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluefs = 1/20 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: debug bluestore = 1/20 2023-12-20T04:27:30.838 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: debug osd = 20 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: debug rocksdb = 4/10 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd backfillfull_ratio = 0.85 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd full ratio = 0.9 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd nearfull ratio = 0.8 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: osd failsafe full ratio = 0.95 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: osd objectstore = bluestore 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: osd op complaint time = 180 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr] 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: debug mgr = 20 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2023-12-20T04:27:30.839 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: mgr/telemetry/nag = false 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: debug client = 20 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: debug finisher = 20 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout:[mon] 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: debug mon = 20 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: debug paxos = 20 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: debug auth = 20 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: mon data avail warn = 5 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: mon mgr mkfs grace = 240 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min pgs per osd = 4 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd reporter subtree level = osd 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: mon osd prime pg temp = true 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: mon reweight min bytes per osd = 10 2023-12-20T04:27:30.840 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.841 INFO:teuthology.orchestra.run.smithi039.stdout: # rotate auth tickets quickly to exercise renewal paths 2023-12-20T04:27:30.841 INFO:teuthology.orchestra.run.smithi039.stdout: auth mon ticket ttl = 660 # 11m 2023-12-20T04:27:30.841 INFO:teuthology.orchestra.run.smithi039.stdout: auth service ticket ttl = 240 # 4m 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: # don't complain about insecure global_id in the test suite 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim = false 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: # 1m isn't quite enough 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: mon_down_mkfs_grace = 2m 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: mon_warn_on_filestore_osds = false 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: mon op complaint time = 120 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout:[client] 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: rgw cache enabled = true 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable ops log = true 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: rgw enable usage log = true 2023-12-20T04:27:30.842 INFO:teuthology.orchestra.run.smithi039.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout: client mount timeout = 600 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout: debug client = 20 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout: pid file = "" 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout: rados mon op timeout = 900 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout: rados osd op timeout = 900 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.a] 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.b] 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout:[mon.c] 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d] 2023-12-20T04:27:30.843 INFO:teuthology.orchestra.run.smithi039.stdout:[mds] 2023-12-20T04:27:30.844 INFO:teuthology.orchestra.run.smithi039.stdout: debug mds = 20 2023-12-20T04:27:30.844 INFO:teuthology.orchestra.run.smithi039.stdout: debug mds balancer = 20 2023-12-20T04:27:30.844 INFO:teuthology.orchestra.run.smithi039.stdout: debug ms = 1 2023-12-20T04:27:30.844 INFO:teuthology.orchestra.run.smithi039.stdout: mds debug frag = True 2023-12-20T04:27:30.844 INFO:teuthology.orchestra.run.smithi039.stdout: mds debug scatterstat = True 2023-12-20T04:27:30.844 INFO:teuthology.orchestra.run.smithi039.stdout: mds op complaint time = 180 2023-12-20T04:27:30.844 INFO:teuthology.orchestra.run.smithi039.stdout: mds verify scatter = True 2023-12-20T04:27:30.844 INFO:teuthology.orchestra.run.smithi039.stdout: osd op complaint time = 180 2023-12-20T04:27:30.844 INFO:teuthology.orchestra.run.smithi039.stdout: rados mon op timeout = 900 2023-12-20T04:27:30.844 INFO:teuthology.orchestra.run.smithi039.stdout: rados osd op timeout = 900 2023-12-20T04:27:30.846 INFO:tasks.ceph:Creating admin key on mon.a... 2023-12-20T04:27:30.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2023-12-20T04:27:30.948 INFO:tasks.ceph:Copying monmap to all nodes... 2023-12-20T04:27:30.948 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:27:30.948 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2023-12-20T04:27:30.957 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:27:30.957 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2023-12-20T04:27:31.005 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:27:31.006 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:27:31.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-20T04:27:31.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-20T04:27:31.088 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:27:31.088 DEBUG:teuthology.orchestra.run.smithi032:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:27:31.138 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T04:27:31.139 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:27:31.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd of=/etc/ceph/ceph.keyring 2023-12-20T04:27:31.139 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0644 /etc/ceph/ceph.keyring 2023-12-20T04:27:31.176 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:27:31.176 DEBUG:teuthology.orchestra.run.smithi039:> dd of=/home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:27:31.226 INFO:tasks.ceph:Setting up mon nodes... 2023-12-20T04:27:31.226 INFO:tasks.ceph:Setting up mgr nodes... 2023-12-20T04:27:31.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2023-12-20T04:27:31.285 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2023-12-20T04:27:31.291 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2023-12-20T04:27:31.351 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2023-12-20T04:27:31.365 INFO:tasks.ceph:Setting up mds nodes... 2023-12-20T04:27:31.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-a && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.a /var/lib/ceph/mds/ceph-a/keyring 2023-12-20T04:27:31.423 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-a/keyring 2023-12-20T04:27:31.428 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-a 2023-12-20T04:27:31.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mds/ceph-c && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.c /var/lib/ceph/mds/ceph-c/keyring 2023-12-20T04:27:31.550 INFO:teuthology.orchestra.run.smithi032.stdout:creating /var/lib/ceph/mds/ceph-c/keyring 2023-12-20T04:27:31.554 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-c 2023-12-20T04:27:31.575 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mds/ceph-b && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.b /var/lib/ceph/mds/ceph-b/keyring 2023-12-20T04:27:31.636 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mds/ceph-b/keyring 2023-12-20T04:27:31.640 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-b 2023-12-20T04:27:31.662 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mds/ceph-d && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mds.d /var/lib/ceph/mds/ceph-d/keyring 2023-12-20T04:27:31.763 INFO:teuthology.orchestra.run.smithi039.stdout:creating /var/lib/ceph/mds/ceph-d/keyring 2023-12-20T04:27:31.767 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mds/ceph-d 2023-12-20T04:27:31.788 INFO:tasks.ceph_client:Setting up client nodes... 2023-12-20T04:27:31.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2023-12-20T04:27:31.835 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.0.keyring 2023-12-20T04:27:31.854 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.2 /etc/ceph/ceph.client.2.keyring && sudo chmod 0644 /etc/ceph/ceph.client.2.keyring 2023-12-20T04:27:31.945 INFO:teuthology.orchestra.run.smithi032.stdout:creating /etc/ceph/ceph.client.2.keyring 2023-12-20T04:27:31.963 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2023-12-20T04:27:32.008 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.client.1.keyring 2023-12-20T04:27:32.025 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.3 /etc/ceph/ceph.client.3.keyring && sudo chmod 0644 /etc/ceph/ceph.client.3.keyring 2023-12-20T04:27:32.113 INFO:teuthology.orchestra.run.smithi039.stdout:creating /etc/ceph/ceph.client.3.keyring 2023-12-20T04:27:32.130 INFO:tasks.ceph:Running mkfs on osd nodes... 2023-12-20T04:27:32.131 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi032.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'}, Remote(name='ubuntu@smithi039.front.sepia.ceph.com'): {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'}} 2023-12-20T04:27:32.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2023-12-20T04:27:32.152 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-20T04:27:32.152 INFO:tasks.ceph:role: osd.0 2023-12-20T04:27:32.152 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:27:32.152 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-20T04:27:32.213 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-20T04:27:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-20T04:27:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-20T04:27:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-20T04:27:32.214 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-20T04:27:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-12-20T04:27:32.214 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-20T04:27:32.214 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-20T04:27:32.214 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-20T04:27:32.214 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-20T04:27:32.238 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-12-20T04:27:32.240 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-12-20T04:27:32.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2023-12-20T04:27:32.312 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2023-12-20T04:27:32.369 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-12-20T04:27:32.370 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:27:32.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2023-12-20T04:27:32.433 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-20T04:27:32.433 INFO:tasks.ceph:role: osd.1 2023-12-20T04:27:32.433 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:27:32.433 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-20T04:27:32.497 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-20T04:27:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-20T04:27:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-20T04:27:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-20T04:27:32.498 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-20T04:27:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-12-20T04:27:32.498 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-20T04:27:32.498 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-20T04:27:32.498 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-20T04:27:32.498 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-20T04:27:32.522 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-12-20T04:27:32.525 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-12-20T04:27:32.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2023-12-20T04:27:32.599 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2023-12-20T04:27:32.658 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-12-20T04:27:32.659 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:27:32.659 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2023-12-20T04:27:32.722 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-20T04:27:32.722 INFO:tasks.ceph:role: osd.2 2023-12-20T04:27:32.722 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:27:32.722 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-20T04:27:32.785 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-20T04:27:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-20T04:27:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-20T04:27:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-20T04:27:32.786 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-20T04:27:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-12-20T04:27:32.786 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-20T04:27:32.786 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-20T04:27:32.786 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-20T04:27:32.786 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-20T04:27:32.812 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-12-20T04:27:32.815 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-12-20T04:27:32.815 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2023-12-20T04:27:32.888 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2023-12-20T04:27:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-12-20T04:27:32.946 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:27:32.947 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2023-12-20T04:27:33.013 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3', 'osd.3': '/dev/vg_nvme/lv_4'} 2023-12-20T04:27:33.013 INFO:tasks.ceph:role: osd.3 2023-12-20T04:27:33.013 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:27:33.014 DEBUG:teuthology.orchestra.run.smithi032:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-20T04:27:33.077 INFO:teuthology.orchestra.run.smithi032.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-20T04:27:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-20T04:27:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-20T04:27:33.077 INFO:teuthology.orchestra.run.smithi032.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-20T04:27:33.077 INFO:teuthology.orchestra.run.smithi032.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-20T04:27:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: = sunit=0 swidth=0 blks 2023-12-20T04:27:33.078 INFO:teuthology.orchestra.run.smithi032.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-20T04:27:33.078 INFO:teuthology.orchestra.run.smithi032.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-20T04:27:33.078 INFO:teuthology.orchestra.run.smithi032.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-20T04:27:33.078 INFO:teuthology.orchestra.run.smithi032.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-20T04:27:33.101 INFO:teuthology.orchestra.run.smithi032.stdout:Discarding blocks...Done. 2023-12-20T04:27:33.105 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi032.front.sepia.ceph.com -o noatime 2023-12-20T04:27:33.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-3 2023-12-20T04:27:33.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2023-12-20T04:27:33.233 INFO:teuthology.orchestra.run.smithi032.stderr:sudo: /sbin/restorecon: command not found 2023-12-20T04:27:33.234 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:27:33.235 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:27:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:33.482+0000 7f5e0b32d640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2023-12-20T04:27:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:33.482+0000 7f5e0b32d640 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2023-12-20T04:27:33.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:33.482+0000 7f5e0b32d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-20T04:27:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:33.482+0000 7f5e0b32d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-20T04:27:33.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:33.482+0000 7f5e0b32d640 -1 bluestore(/var/lib/ceph/osd/ceph-0/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-0/block: (2) No such file or directory 2023-12-20T04:27:33.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:33.486+0000 7f5e0b32d640 -1 bluestore(/var/lib/ceph/osd/ceph-0) _read_fsid unparsable uuid 2023-12-20T04:27:36.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2023-12-20T04:27:36.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:27:36.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:36.590+0000 7fdb75afb640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2023-12-20T04:27:36.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:36.590+0000 7fdb75afb640 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2023-12-20T04:27:36.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:36.590+0000 7fdb75afb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-20T04:27:36.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:36.590+0000 7fdb75afb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-20T04:27:36.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:36.590+0000 7fdb75afb640 -1 bluestore(/var/lib/ceph/osd/ceph-1/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-1/block: (2) No such file or directory 2023-12-20T04:27:36.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:36.590+0000 7fdb75afb640 -1 bluestore(/var/lib/ceph/osd/ceph-1) _read_fsid unparsable uuid 2023-12-20T04:27:39.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2023-12-20T04:27:39.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:27:39.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:39.698+0000 7f22c1ecd640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2023-12-20T04:27:39.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:39.698+0000 7f22c1ecd640 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2023-12-20T04:27:39.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:39.698+0000 7f22c1ecd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-20T04:27:39.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:39.698+0000 7f22c1ecd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-20T04:27:39.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:39.698+0000 7f22c1ecd640 -1 bluestore(/var/lib/ceph/osd/ceph-2/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-2/block: (2) No such file or directory 2023-12-20T04:27:39.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:39.698+0000 7f22c1ecd640 -1 bluestore(/var/lib/ceph/osd/ceph-2) _read_fsid unparsable uuid 2023-12-20T04:27:42.724 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2023-12-20T04:27:42.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:27:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:42.846+0000 7f162d62c640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2023-12-20T04:27:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:42.850+0000 7f162d62c640 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2023-12-20T04:27:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:42.850+0000 7f162d62c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-20T04:27:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:42.850+0000 7f162d62c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-20T04:27:42.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:42.850+0000 7f162d62c640 -1 bluestore(/var/lib/ceph/osd/ceph-3/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-3/block: (2) No such file or directory 2023-12-20T04:27:42.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:27:42.850+0000 7f162d62c640 -1 bluestore(/var/lib/ceph/osd/ceph-3) _read_fsid unparsable uuid 2023-12-20T04:27:45.857 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2023-12-20T04:27:45.879 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2023-12-20T04:27:45.903 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-20T04:27:45.903 INFO:tasks.ceph:role: osd.4 2023-12-20T04:27:45.903 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T04:27:45.904 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2023-12-20T04:27:45.995 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2023-12-20T04:27:45.995 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-20T04:27:45.995 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-20T04:27:45.996 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-20T04:27:45.996 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-20T04:27:45.996 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2023-12-20T04:27:45.996 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-20T04:27:45.996 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-20T04:27:45.996 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-20T04:27:45.996 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-20T04:27:47.302 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2023-12-20T04:27:47.305 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2023-12-20T04:27:47.306 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-4 2023-12-20T04:27:47.439 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2023-12-20T04:27:47.458 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2023-12-20T04:27:47.458 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:27:47.459 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2023-12-20T04:27:47.522 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-20T04:27:47.523 INFO:tasks.ceph:role: osd.5 2023-12-20T04:27:47.523 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T04:27:47.523 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2023-12-20T04:27:47.614 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2023-12-20T04:27:47.615 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-20T04:27:47.615 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-20T04:27:47.615 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-20T04:27:47.615 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-20T04:27:47.615 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2023-12-20T04:27:47.615 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-20T04:27:47.615 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-20T04:27:47.615 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-20T04:27:47.615 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-20T04:27:48.920 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2023-12-20T04:27:48.923 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2023-12-20T04:27:48.923 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-5 2023-12-20T04:27:48.956 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2023-12-20T04:27:49.018 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2023-12-20T04:27:49.019 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:27:49.019 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2023-12-20T04:27:49.082 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-20T04:27:49.082 INFO:tasks.ceph:role: osd.6 2023-12-20T04:27:49.082 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T04:27:49.082 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2023-12-20T04:27:49.174 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2023-12-20T04:27:49.174 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-20T04:27:49.174 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-20T04:27:49.175 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-20T04:27:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-20T04:27:49.175 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2023-12-20T04:27:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-20T04:27:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-20T04:27:49.175 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-20T04:27:49.175 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-20T04:27:50.480 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2023-12-20T04:27:50.484 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2023-12-20T04:27:50.484 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-6 2023-12-20T04:27:50.515 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2023-12-20T04:27:50.579 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2023-12-20T04:27:50.579 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:27:50.580 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2023-12-20T04:27:50.643 INFO:tasks.ceph:roles_to_devs: {'osd.4': '/dev/vg_nvme/lv_1', 'osd.5': '/dev/vg_nvme/lv_2', 'osd.6': '/dev/vg_nvme/lv_3', 'osd.7': '/dev/vg_nvme/lv_4'} 2023-12-20T04:27:50.643 INFO:tasks.ceph:role: osd.7 2023-12-20T04:27:50.643 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T04:27:50.643 DEBUG:teuthology.orchestra.run.smithi039:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2023-12-20T04:27:50.735 INFO:teuthology.orchestra.run.smithi039.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2023-12-20T04:27:50.735 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 attr=2, projid32bit=1 2023-12-20T04:27:50.735 INFO:teuthology.orchestra.run.smithi039.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2023-12-20T04:27:50.735 INFO:teuthology.orchestra.run.smithi039.stdout: = reflink=1 bigtime=0 inobtcount=0 2023-12-20T04:27:50.735 INFO:teuthology.orchestra.run.smithi039.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2023-12-20T04:27:50.736 INFO:teuthology.orchestra.run.smithi039.stdout: = sunit=0 swidth=0 blks 2023-12-20T04:27:50.736 INFO:teuthology.orchestra.run.smithi039.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2023-12-20T04:27:50.736 INFO:teuthology.orchestra.run.smithi039.stdout:log =internal log bsize=4096 blocks=11443, version=2 2023-12-20T04:27:50.736 INFO:teuthology.orchestra.run.smithi039.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2023-12-20T04:27:50.736 INFO:teuthology.orchestra.run.smithi039.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2023-12-20T04:27:52.035 INFO:teuthology.orchestra.run.smithi039.stdout:Discarding blocks...Done. 2023-12-20T04:27:52.037 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi039.front.sepia.ceph.com -o noatime 2023-12-20T04:27:52.037 DEBUG:teuthology.orchestra.run.smithi039:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-7 2023-12-20T04:27:52.068 DEBUG:teuthology.orchestra.run.smithi039:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2023-12-20T04:27:52.130 INFO:teuthology.orchestra.run.smithi039.stderr:sudo: /sbin/restorecon: command not found 2023-12-20T04:27:52.131 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:27:52.132 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:27:52.272 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:52.267+0000 7fe325343640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2023-12-20T04:27:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:52.267+0000 7fe325343640 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2023-12-20T04:27:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:52.271+0000 7fe325343640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-20T04:27:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:52.271+0000 7fe325343640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-20T04:27:52.273 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:52.271+0000 7fe325343640 -1 bluestore(/var/lib/ceph/osd/ceph-4/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-4/block: (2) No such file or directory 2023-12-20T04:27:52.276 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:52.271+0000 7fe325343640 -1 bluestore(/var/lib/ceph/osd/ceph-4) _read_fsid unparsable uuid 2023-12-20T04:27:55.275 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2023-12-20T04:27:55.297 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:27:55.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:55.399+0000 7f46cdf84640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2023-12-20T04:27:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:55.399+0000 7f46cdf84640 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2023-12-20T04:27:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:55.399+0000 7f46cdf84640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-20T04:27:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:55.399+0000 7f46cdf84640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-20T04:27:55.405 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:55.399+0000 7f46cdf84640 -1 bluestore(/var/lib/ceph/osd/ceph-5/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-5/block: (2) No such file or directory 2023-12-20T04:27:55.406 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:55.403+0000 7f46cdf84640 -1 bluestore(/var/lib/ceph/osd/ceph-5) _read_fsid unparsable uuid 2023-12-20T04:27:58.419 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2023-12-20T04:27:58.441 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:27:58.542 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:58.539+0000 7f46e7472640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2023-12-20T04:27:58.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:58.539+0000 7f46e7472640 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2023-12-20T04:27:58.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:58.539+0000 7f46e7472640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-20T04:27:58.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:58.539+0000 7f46e7472640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-20T04:27:58.543 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:58.539+0000 7f46e7472640 -1 bluestore(/var/lib/ceph/osd/ceph-6/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-6/block: (2) No such file or directory 2023-12-20T04:27:58.545 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:27:58.543+0000 7f46e7472640 -1 bluestore(/var/lib/ceph/osd/ceph-6) _read_fsid unparsable uuid 2023-12-20T04:27:58.906 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:27:58.910 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:28:01.545 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2023-12-20T04:28:01.569 DEBUG:teuthology.orchestra.run.smithi039:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:28:01.673 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:01.667+0000 7f778e82b640 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2023-12-20T04:28:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:01.671+0000 7f778e82b640 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2023-12-20T04:28:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:01.671+0000 7f778e82b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-20T04:28:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:01.671+0000 7f778e82b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-20T04:28:01.674 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:01.671+0000 7f778e82b640 -1 bluestore(/var/lib/ceph/osd/ceph-7/block) _read_bdev_label failed to open /var/lib/ceph/osd/ceph-7/block: (2) No such file or directory 2023-12-20T04:28:01.675 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:01.671+0000 7f778e82b640 -1 bluestore(/var/lib/ceph/osd/ceph-7) _read_fsid unparsable uuid 2023-12-20T04:28:04.716 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2023-12-20T04:28:04.737 INFO:tasks.ceph:Reading keys from all nodes... 2023-12-20T04:28:04.738 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:04.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mgr/ceph-y/keyring of=/dev/stdout 2023-12-20T04:28:04.762 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:04.762 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-a/keyring of=/dev/stdout 2023-12-20T04:28:04.827 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:04.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/mds/ceph-c/keyring of=/dev/stdout 2023-12-20T04:28:04.891 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:04.891 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2023-12-20T04:28:04.959 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:04.959 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2023-12-20T04:28:05.022 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:05.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2023-12-20T04:28:05.091 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:05.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2023-12-20T04:28:05.155 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:05.155 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2023-12-20T04:28:05.177 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:05.177 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mds/ceph-b/keyring of=/dev/stdout 2023-12-20T04:28:05.244 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:05.245 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/mds/ceph-d/keyring of=/dev/stdout 2023-12-20T04:28:05.310 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:05.310 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-4/keyring of=/dev/stdout 2023-12-20T04:28:05.374 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:05.374 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-5/keyring of=/dev/stdout 2023-12-20T04:28:05.438 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:05.438 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-6/keyring of=/dev/stdout 2023-12-20T04:28:05.501 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:05.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-7/keyring of=/dev/stdout 2023-12-20T04:28:05.566 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:05.566 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2023-12-20T04:28:05.576 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:05.576 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/etc/ceph/ceph.client.2.keyring of=/dev/stdout 2023-12-20T04:28:05.630 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:05.630 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2023-12-20T04:28:05.639 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:05.639 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/etc/ceph/ceph.client.3.keyring of=/dev/stdout 2023-12-20T04:28:05.689 INFO:tasks.ceph:Adding keys to all mons... 2023-12-20T04:28:05.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-20T04:28:05.693 DEBUG:teuthology.orchestra.run.smithi039:> sudo tee -a /etc/ceph/ceph.keyring 2023-12-20T04:28:05.737 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.y] 2023-12-20T04:28:05.737 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA== 2023-12-20T04:28:05.737 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.a] 2023-12-20T04:28:05.737 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ== 2023-12-20T04:28:05.737 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.c] 2023-12-20T04:28:05.737 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA== 2023-12-20T04:28:05.737 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.0] 2023-12-20T04:28:05.737 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ== 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.1] 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w== 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.2] 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw== 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.3] 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA== 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout:[mgr.x] 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ== 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.b] 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg== 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout:[mds.d] 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA== 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.4] 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg== 2023-12-20T04:28:05.738 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.5] 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw== 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.6] 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g== 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout:[osd.7] 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA== 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout:[client.0] 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA== 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout:[client.2] 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A== 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout:[client.1] 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg== 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout:[client.3] 2023-12-20T04:28:05.739 INFO:teuthology.orchestra.run.smithi032.stdout: key = AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA== 2023-12-20T04:28:05.752 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.y] 2023-12-20T04:28:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA== 2023-12-20T04:28:05.752 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.a] 2023-12-20T04:28:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ== 2023-12-20T04:28:05.752 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.c] 2023-12-20T04:28:05.752 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA== 2023-12-20T04:28:05.752 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.0] 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ== 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.1] 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w== 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.2] 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw== 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.3] 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA== 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout:[mgr.x] 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ== 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.b] 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg== 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout:[mds.d] 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA== 2023-12-20T04:28:05.753 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.4] 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg== 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.5] 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw== 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.6] 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g== 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout:[osd.7] 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA== 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout:[client.0] 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA== 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout:[client.2] 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A== 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout:[client.1] 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg== 2023-12-20T04:28:05.754 INFO:teuthology.orchestra.run.smithi039.stdout:[client.3] 2023-12-20T04:28:05.755 INFO:teuthology.orchestra.run.smithi039.stdout: key = AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA== 2023-12-20T04:28:05.755 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-20T04:28:05.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-20T04:28:05.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-20T04:28:05.864 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.a --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-20T04:28:05.925 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-20T04:28:05.929 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.c --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-20T04:28:05.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:05.994 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.050 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.054 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.114 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.170 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.174 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.231 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-20T04:28:06.235 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2023-12-20T04:28:06.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-20T04:28:06.294 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.b --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-20T04:28:06.352 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-20T04:28:06.356 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mds.d --cap mon 'allow *' --cap mgr 'allow *' --cap osd 'allow *' --cap mds allow 2023-12-20T04:28:06.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.479 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.542 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.598 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.602 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2023-12-20T04:28:06.657 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-20T04:28:06.662 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-20T04:28:06.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-20T04:28:06.724 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.2 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-20T04:28:06.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-20T04:28:06.786 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-20T04:28:06.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-20T04:28:06.850 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.3 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2023-12-20T04:28:06.909 INFO:tasks.ceph:Running mkfs on mon nodes... 2023-12-20T04:28:06.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2023-12-20T04:28:06.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-20T04:28:07.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2023-12-20T04:28:07.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2023-12-20T04:28:07.089 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-20T04:28:07.249 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2023-12-20T04:28:07.270 DEBUG:teuthology.orchestra.run.smithi039:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2023-12-20T04:28:07.333 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2023-12-20T04:28:07.452 DEBUG:teuthology.orchestra.run.smithi039:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2023-12-20T04:28:07.473 DEBUG:teuthology.orchestra.run.smithi032:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:28:07.477 DEBUG:teuthology.orchestra.run.smithi039:> rm -- /home/ubuntu/cephtest/ceph.monmap 2023-12-20T04:28:07.528 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2023-12-20T04:28:07.529 INFO:tasks.ceph.mon.a:Restarting daemon 2023-12-20T04:28:07.529 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2023-12-20T04:28:07.533 INFO:tasks.ceph.mon.a:Started 2023-12-20T04:28:07.533 INFO:tasks.ceph.mon.b:Restarting daemon 2023-12-20T04:28:07.533 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i b 2023-12-20T04:28:07.580 INFO:tasks.ceph.mon.b:Started 2023-12-20T04:28:07.580 INFO:tasks.ceph.mon.c:Restarting daemon 2023-12-20T04:28:07.581 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i c 2023-12-20T04:28:07.584 INFO:tasks.ceph.mon.c:Started 2023-12-20T04:28:07.584 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2023-12-20T04:28:07.585 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-20T04:28:07.585 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-20T04:28:07.589 INFO:tasks.ceph.mgr.y:Started 2023-12-20T04:28:07.589 INFO:tasks.ceph.mgr.x:Restarting daemon 2023-12-20T04:28:07.589 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2023-12-20T04:28:07.593 INFO:tasks.ceph.mgr.x:Started 2023-12-20T04:28:07.593 INFO:tasks.ceph:Setting crush tunables to default 2023-12-20T04:28:07.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph osd crush tunables default 2023-12-20T04:28:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.734+0000 7f51bece5640 1 Processor -- start 2023-12-20T04:28:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.734+0000 7f51bece5640 1 -- start start 2023-12-20T04:28:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bece5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bece5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bece5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bece5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b806d410 con 0x7f51b80658e0 2023-12-20T04:28:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bece5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b815ba80 con 0x7f51b8158670 2023-12-20T04:28:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bece5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b815bd60 con 0x7f51b806ca60 2023-12-20T04:28:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51affff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51affff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35150/0 (socket says 172.21.15.32:35150) 2023-12-20T04:28:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 learned_addr learned my addr 172.21.15.32:0/2309350059 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 msgr2=0x7f51b8071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-20T04:28:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 msgr2=0x7f51b8071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:28:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bca81640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 msgr2=0x7f51b815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-20T04:28:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bca81640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 msgr2=0x7f51b815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:28:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bd282640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 msgr2=0x7f51b806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-20T04:28:07.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bd282640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 msgr2=0x7f51b806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:07.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:07.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.738+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:28:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 msgr2=0x7f51b8071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-20T04:28:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 msgr2=0x7f51b8071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51bca81640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 msgr2=0x7f51b815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-20T04:28:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51bca81640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 msgr2=0x7f51b815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:28:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51bd282640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 msgr2=0x7f51b806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-20T04:28:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51bd282640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 msgr2=0x7f51b806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:28:07.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:07.942+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:28:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 msgr2=0x7f51b8071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-20T04:28:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 msgr2=0x7f51b8071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:08.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51bca81640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 msgr2=0x7f51b815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-20T04:28:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T04:28:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51bca81640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 msgr2=0x7f51b815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51bd282640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 msgr2=0x7f51b806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-20T04:28:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51bd282640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 msgr2=0x7f51b806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T04:28:08.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:08.346+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T04:28:09.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:09.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 msgr2=0x7f51b8071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 13 2023-12-20T04:28:09.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 msgr2=0x7f51b8071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:09.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T04:28:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51bd282640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 msgr2=0x7f51b806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 14 2023-12-20T04:28:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51bd282640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 msgr2=0x7f51b806ced0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51bca81640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 msgr2=0x7f51b815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_bulk peer close file descriptor 15 2023-12-20T04:28:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51bca81640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 msgr2=0x7f51b815b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until read failed 2023-12-20T04:28:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2023-12-20T04:28:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T04:28:09.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:09.150+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T04:28:10.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:10.738+0000 7f51ae7fc640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 msgr2=0x7f51b806ced0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-20T04:28:10.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:10.738+0000 7f51ae7fc640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b806ca60 0x7f51b806ced0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:10.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:10.738+0000 7f51ae7fc640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 msgr2=0x7f51b815b1c0 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-20T04:28:10.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:10.738+0000 7f51ae7fc640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.39:3300/0 conn(0x7f51b8158670 0x7f51b815b1c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:10.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:10.738+0000 7f51ae7fc640 1 -- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 msgr2=0x7f51b8071200 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2023-12-20T04:28:10.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:10.738+0000 7f51ae7fc640 1 --2- 172.21.15.32:0/2309350059 >> v2:172.21.15.32:3300/0 conn(0x7f51b80658e0 0x7f51b8071200 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:10.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:10.738+0000 7f51ae7fc640 1 -- 172.21.15.32:0/2309350059 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b815c040 con 0x7f51a0003680 2023-12-20T04:28:10.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:10.738+0000 7f51ae7fc640 1 -- 172.21.15.32:0/2309350059 --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51a000c080 con 0x7f51a0007310 2023-12-20T04:28:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2659899088 0 0) 0x7f51b815c040 con 0x7f51a0003680 2023-12-20T04:28:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51a4003530 con 0x7f51a0003680 2023-12-20T04:28:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2540130320 0 0) 0x7f51a000c080 con 0x7f51a0007310 2023-12-20T04:28:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f51b815c040 con 0x7f51a0007310 2023-12-20T04:28:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1666271712 0 0) 0x7f51a4003530 con 0x7f51a0003680 2023-12-20T04:28:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 >> v1:172.21.15.39:6789/0 conn(0x7f51a0007310 legacy=0x7f51a00097d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a000d280 con 0x7f51a0003680 2023-12-20T04:28:12.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f51b400e510 con 0x7f51a0003680 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 >> v1:172.21.15.32:6789/0 conn(0x7f51a0003680 legacy=0x7f51a0005b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51bd282640 1 -- 172.21.15.32:0/2309350059 reap_dead start 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4004bf0 0x7f51a4004fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4003770 0x7f51a4003bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a4009980 0x7f51a400bd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a400c2e0 con 0x7f51a4009980 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a400c450 con 0x7f51a4003770 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a400c610 con 0x7f51a4004bf0 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a4009980 0x7f51a400bd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4003770 0x7f51a4003bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.910+0000 7f51bca81640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4004bf0 0x7f51a4004fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.914+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4004bf0 msgr2=0x7f51a4004fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.914+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4004bf0 0x7f51a4004fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.914+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a4009980 msgr2=0x7f51a400bd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.914+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a4009980 0x7f51a400bd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.914+0000 7f51affff640 1 -- 172.21.15.32:0/2309350059 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a4003530 con 0x7f51a4003770 2023-12-20T04:28:12.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.914+0000 7f51bd282640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a4009980 0x7f51a400bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:12.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51affff640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4003770 0x7f51a4003bf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009100 tx=0x7f51a801fae0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c4e8a59ff81c35b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a80a9710 con 0x7f51a4003770 2023-12-20T04:28:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a80ad070 con 0x7f51a4003770 2023-12-20T04:28:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51af7fe640 1 -- 172.21.15.32:0/2309350059 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a80a9ad0 con 0x7f51a4003770 2023-12-20T04:28:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51bece5640 1 -- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4003770 msgr2=0x7f51a4003bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51bece5640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4003770 0x7f51a4003bf0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f51a8009100 tx=0x7f51a801fae0 comp rx=0 tx=0).stop 2023-12-20T04:28:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51bece5640 1 -- 172.21.15.32:0/2309350059 shutdown_connections 2023-12-20T04:28:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51bece5640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a4009980 0x7f51a400bd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51bece5640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4003770 0x7f51a4003bf0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51bece5640 1 --2- 172.21.15.32:0/2309350059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4004bf0 0x7f51a4004fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:12.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51bece5640 1 -- 172.21.15.32:0/2309350059 >> 172.21.15.32:0/2309350059 conn(0x7f51b80cf8d0 msgr2=0x7f51b80cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:12.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51bece5640 1 -- 172.21.15.32:0/2309350059 shutdown_connections 2023-12-20T04:28:12.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.918+0000 7f51bece5640 1 -- 172.21.15.32:0/2309350059 wait complete. 2023-12-20T04:28:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bece5640 1 Processor -- start 2023-12-20T04:28:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bece5640 1 -- start start 2023-12-20T04:28:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bece5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4003770 0x7f51b814b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bece5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a4004bf0 0x7f51b80cda00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bece5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4009980 0x7f51b80cdf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bece5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b81f1d40 con 0x7f51a4004bf0 2023-12-20T04:28:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bece5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b8157900 con 0x7f51a4009980 2023-12-20T04:28:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bece5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b814ba50 con 0x7f51a4003770 2023-12-20T04:28:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bca81640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4003770 0x7f51b814b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51affff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a4004bf0 0x7f51b80cda00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bca81640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4003770 0x7f51b814b510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:39734/0 (socket says 172.21.15.32:39734) 2023-12-20T04:28:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bca81640 1 -- 172.21.15.32:0/2367765519 learned_addr learned my addr 172.21.15.32:0/2367765519 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bd282640 1 --2- 172.21.15.32:0/2367765519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4009980 0x7f51b80cdf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bd282640 1 -- 172.21.15.32:0/2367765519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4003770 msgr2=0x7f51b814b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bd282640 1 --2- 172.21.15.32:0/2367765519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4003770 0x7f51b814b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bd282640 1 -- 172.21.15.32:0/2367765519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a4004bf0 msgr2=0x7f51b80cda00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bd282640 1 --2- 172.21.15.32:0/2367765519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a4004bf0 0x7f51b80cda00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.922+0000 7f51bd282640 1 -- 172.21.15.32:0/2367765519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b8157620 con 0x7f51a4009980 2023-12-20T04:28:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.926+0000 7f51bd282640 1 --2- 172.21.15.32:0/2367765519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4009980 0x7f51b80cdf40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f51b80b1c40 tx=0x7f51b0015700 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.926+0000 7f51ad7fa640 1 -- 172.21.15.32:0/2367765519 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b00048b0 con 0x7f51a4009980 2023-12-20T04:28:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.926+0000 7f51ad7fa640 1 -- 172.21.15.32:0/2367765519 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51b000f040 con 0x7f51a4009980 2023-12-20T04:28:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.926+0000 7f51ad7fa640 1 -- 172.21.15.32:0/2367765519 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51b0021630 con 0x7f51a4009980 2023-12-20T04:28:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.926+0000 7f51bece5640 1 -- 172.21.15.32:0/2367765519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b80aba60 con 0x7f51a4009980 2023-12-20T04:28:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.926+0000 7f51bece5640 1 -- 172.21.15.32:0/2367765519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b80ac020 con 0x7f51a4009980 2023-12-20T04:28:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.926+0000 7f51ad7fa640 1 -- 172.21.15.32:0/2367765519 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f51b0021a90 con 0x7f51a4009980 2023-12-20T04:28:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.926+0000 7f51ad7fa640 1 -- 172.21.15.32:0/2367765519 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(1..1 src has 1..1) v4 ==== 725+0+0 (secure 0 0 0) 0x7f51b0017da0 con 0x7f51a4009980 2023-12-20T04:28:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.926+0000 7f51bece5640 1 -- 172.21.15.32:0/2367765519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b804e000 con 0x7f51a4009980 2023-12-20T04:28:12.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.930+0000 7f51ad7fa640 1 -- 172.21.15.32:0/2367765519 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f51b001c030 con 0x7f51a4009980 2023-12-20T04:28:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.994+0000 7f51bece5640 1 -- 172.21.15.32:0/2367765519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) v1 -- 0x7f51b80cc950 con 0x7f51a4009980 2023-12-20T04:28:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.998+0000 7f51ad7fa640 1 -- 172.21.15.32:0/2367765519 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v2) v1 ==== 124+0+0 (secure 0 0 0) 0x7f51b0021d40 con 0x7f51a4009980 2023-12-20T04:28:12.999 INFO:teuthology.orchestra.run.smithi032.stderr:adjusted tunables profile to default 2023-12-20T04:28:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.998+0000 7f51bece5640 1 -- 172.21.15.32:0/2367765519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4009980 msgr2=0x7f51b80cdf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.998+0000 7f51bece5640 1 --2- 172.21.15.32:0/2367765519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4009980 0x7f51b80cdf40 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f51b80b1c40 tx=0x7f51b0015700 comp rx=0 tx=0).stop 2023-12-20T04:28:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.998+0000 7f51bece5640 1 -- 172.21.15.32:0/2367765519 shutdown_connections 2023-12-20T04:28:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.998+0000 7f51bece5640 1 --2- 172.21.15.32:0/2367765519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4009980 0x7f51b80cdf40 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.998+0000 7f51bece5640 1 --2- 172.21.15.32:0/2367765519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a4004bf0 0x7f51b80cda00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.998+0000 7f51bece5640 1 --2- 172.21.15.32:0/2367765519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4003770 0x7f51b814b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.998+0000 7f51bece5640 1 -- 172.21.15.32:0/2367765519 >> 172.21.15.32:0/2367765519 conn(0x7f51b80cf8d0 msgr2=0x7f51b8158d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.998+0000 7f51bece5640 1 -- 172.21.15.32:0/2367765519 shutdown_connections 2023-12-20T04:28:13.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:12.998+0000 7f51bece5640 1 -- 172.21.15.32:0/2367765519 wait complete. 2023-12-20T04:28:13.020 INFO:tasks.ceph:check_enable_crimson: False 2023-12-20T04:28:13.021 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2023-12-20T04:28:13.021 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:13.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2023-12-20T04:28:13.042 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:13.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2023-12-20T04:28:13.106 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:13.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2023-12-20T04:28:13.118 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:13.118+0000 7f1cbfcdc180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:28:13.139 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:13.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2023-12-20T04:28:13.202 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:13.202 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-4/fsid of=/dev/stdout 2023-12-20T04:28:13.226 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:13.227 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-5/fsid of=/dev/stdout 2023-12-20T04:28:13.295 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:13.295 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-6/fsid of=/dev/stdout 2023-12-20T04:28:13.362 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:13.362 DEBUG:teuthology.orchestra.run.smithi039:> sudo dd if=/var/lib/ceph/osd/ceph-7/fsid of=/dev/stdout 2023-12-20T04:28:13.425 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new 6c8ebcf5-82d4-4280-9a4d-91671cb1b199 0 2023-12-20T04:28:13.612 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.607+0000 7f81f6b09640 1 Processor -- start 2023-12-20T04:28:13.613 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.607+0000 7f81f6b09640 1 -- start start 2023-12-20T04:28:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f6b09640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81f0158670 0x7f81f015b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f6b09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f006ced0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f6b09640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f015ba80 con 0x7f81f00658e0 2023-12-20T04:28:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f6b09640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f0071910 con 0x7f81f0158670 2023-12-20T04:28:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f6b09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f006d410 con 0x7f81f006ca60 2023-12-20T04:28:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f50a6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f006ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f50a6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f006ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:55112/0 (socket says 172.21.15.39:55112) 2023-12-20T04:28:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f50a6640 1 -- 172.21.15.39:0/1658608819 learned_addr learned my addr 172.21.15.39:0/1658608819 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:13.615 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f48a5640 1 --2- 172.21.15.39:0/1658608819 >> v2:172.21.15.39:3300/0 conn(0x7f81f0158670 0x7f81f015b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f50a6640 1 -- 172.21.15.39:0/1658608819 >> v2:172.21.15.39:3300/0 conn(0x7f81f0158670 msgr2=0x7f81f015b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f50a6640 1 --2- 172.21.15.39:0/1658608819 >> v2:172.21.15.39:3300/0 conn(0x7f81f0158670 0x7f81f015b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f50a6640 1 -- 172.21.15.39:0/1658608819 >> v1:172.21.15.39:6789/0 conn(0x7f81f00658e0 legacy=0x7f81f0071200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f50a6640 1 -- 172.21.15.39:0/1658608819 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f015bd60 con 0x7f81f006ca60 2023-12-20T04:28:13.616 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.611+0000 7f81f50a6640 1 --2- 172.21.15.39:0/1658608819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f006ced0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81f0071bb0 tx=0x7f81e801c990 comp rx=0 tx=0).ready entity=mon.2 client_cookie=591061c1a0c3ea09 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:13.617 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81ef7fe640 1 -- 172.21.15.39:0/1658608819 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e80a5700 con 0x7f81f006ca60 2023-12-20T04:28:13.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81ef7fe640 1 -- 172.21.15.39:0/1658608819 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e80a5860 con 0x7f81f006ca60 2023-12-20T04:28:13.619 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81ef7fe640 1 -- 172.21.15.39:0/1658608819 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e80a5bf0 con 0x7f81f006ca60 2023-12-20T04:28:13.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 -- 172.21.15.39:0/1658608819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 msgr2=0x7f81f006ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:13.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 --2- 172.21.15.39:0/1658608819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f006ced0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f81f0071bb0 tx=0x7f81e801c990 comp rx=0 tx=0).stop 2023-12-20T04:28:13.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 -- 172.21.15.39:0/1658608819 shutdown_connections 2023-12-20T04:28:13.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 --2- 172.21.15.39:0/1658608819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f006ced0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 --2- 172.21.15.39:0/1658608819 >> v2:172.21.15.39:3300/0 conn(0x7f81f0158670 0x7f81f015b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 -- 172.21.15.39:0/1658608819 >> 172.21.15.39:0/1658608819 conn(0x7f81f00cf8d0 msgr2=0x7f81f00cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:13.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 -- 172.21.15.39:0/1658608819 shutdown_connections 2023-12-20T04:28:13.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 -- 172.21.15.39:0/1658608819 wait complete. 2023-12-20T04:28:13.625 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 Processor -- start 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 -- start start 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f00658e0 0x7f81f00b08c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f0152580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f0158670 0x7f81f01574f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00b0e00 con 0x7f81f0158670 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00b15f0 con 0x7f81f00658e0 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f6b09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f00c2af0 con 0x7f81f006ca60 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81effff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f0152580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f48a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f00658e0 0x7f81f00b08c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81effff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f0152580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:55120/0 (socket says 172.21.15.39:55120) 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81effff640 1 -- 172.21.15.39:0/1820612803 learned_addr learned my addr 172.21.15.39:0/1820612803 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:13.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.615+0000 7f81f48a5640 1 --2- 172.21.15.39:0/1820612803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f00658e0 0x7f81f00b08c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36614/0 (socket says 172.21.15.39:36614) 2023-12-20T04:28:13.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81f50a6640 1 --2- 172.21.15.39:0/1820612803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f0158670 0x7f81f01574f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:13.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81effff640 1 -- 172.21.15.39:0/1820612803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f00658e0 msgr2=0x7f81f00b08c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:13.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81effff640 1 --2- 172.21.15.39:0/1820612803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f00658e0 0x7f81f00b08c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81effff640 1 -- 172.21.15.39:0/1820612803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f0158670 msgr2=0x7f81f01574f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:13.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81effff640 1 --2- 172.21.15.39:0/1820612803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f0158670 0x7f81f01574f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81effff640 1 -- 172.21.15.39:0/1820612803 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8009b30 con 0x7f81f006ca60 2023-12-20T04:28:13.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81f50a6640 1 --2- 172.21.15.39:0/1820612803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f0158670 0x7f81f01574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:13.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81f48a5640 1 --2- 172.21.15.39:0/1820612803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f00658e0 0x7f81f00b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:13.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81effff640 1 --2- 172.21.15.39:0/1820612803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f0152580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c480 tx=0x7f81dc00c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:13.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81ed7fa640 1 -- 172.21.15.39:0/1820612803 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc012830 con 0x7f81f006ca60 2023-12-20T04:28:13.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81ed7fa640 1 -- 172.21.15.39:0/1820612803 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81dc0129b0 con 0x7f81f006ca60 2023-12-20T04:28:13.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81ed7fa640 1 -- 172.21.15.39:0/1820612803 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81dc012d60 con 0x7f81f006ca60 2023-12-20T04:28:13.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81f6b09640 1 -- 172.21.15.39:0/1820612803 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f0152ac0 con 0x7f81f006ca60 2023-12-20T04:28:13.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81f6b09640 1 -- 172.21.15.39:0/1820612803 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f00c3300 con 0x7f81f006ca60 2023-12-20T04:28:13.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81ed7fa640 1 -- 172.21.15.39:0/1820612803 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f81dc01c760 con 0x7f81f006ca60 2023-12-20T04:28:13.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81ed7fa640 1 -- 172.21.15.39:0/1820612803 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(2..2 src has 1..2) v4 ==== 725+0+0 (secure 0 0 0) 0x7f81dc01b860 con 0x7f81f006ca60 2023-12-20T04:28:13.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.619+0000 7f81f6b09640 1 -- 172.21.15.39:0/1820612803 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f004e000 con 0x7f81f006ca60 2023-12-20T04:28:13.636 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.631+0000 7f81ed7fa640 1 -- 172.21.15.39:0/1820612803 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f81dc01b3c0 con 0x7f81f006ca60 2023-12-20T04:28:13.703 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:13.702+0000 7f1cbfcdc180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:28:13.704 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.699+0000 7f81f6b09640 1 -- 172.21.15.39:0/1820612803 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd new", "uuid": "6c8ebcf5-82d4-4280-9a4d-91671cb1b199", "id": 0} v 0) v1 -- 0x7f81f00ca810 con 0x7f81f006ca60 2023-12-20T04:28:13.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.707+0000 7f81ed7fa640 1 -- 172.21.15.39:0/1820612803 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "6c8ebcf5-82d4-4280-9a4d-91671cb1b199", "id": 0}]=0 v3) v1 ==== 112+0+2 (secure 0 0 0) 0x7f81dc02b0b0 con 0x7f81f006ca60 2023-12-20T04:28:13.710 INFO:teuthology.orchestra.run.smithi039.stdout:0 2023-12-20T04:28:13.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.707+0000 7f81f6b09640 1 -- 172.21.15.39:0/1820612803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 msgr2=0x7f81f0152580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:13.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.707+0000 7f81f6b09640 1 --2- 172.21.15.39:0/1820612803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f0152580 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f81dc00c480 tx=0x7f81dc00c940 comp rx=0 tx=0).stop 2023-12-20T04:28:13.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.707+0000 7f81f6b09640 1 -- 172.21.15.39:0/1820612803 shutdown_connections 2023-12-20T04:28:13.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.707+0000 7f81f6b09640 1 --2- 172.21.15.39:0/1820612803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f0158670 0x7f81f01574f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.707+0000 7f81f6b09640 1 --2- 172.21.15.39:0/1820612803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f006ca60 0x7f81f0152580 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.707+0000 7f81f6b09640 1 --2- 172.21.15.39:0/1820612803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f00658e0 0x7f81f00b08c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.707+0000 7f81f6b09640 1 -- 172.21.15.39:0/1820612803 >> 172.21.15.39:0/1820612803 conn(0x7f81f00cf8d0 msgr2=0x7f81f0165130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:13.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.707+0000 7f81f6b09640 1 -- 172.21.15.39:0/1820612803 shutdown_connections 2023-12-20T04:28:13.711 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.707+0000 7f81f6b09640 1 -- 172.21.15.39:0/1820612803 wait complete. 2023-12-20T04:28:13.730 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new 14ba777b-bae8-4e35-b63d-eafde55014d1 1 2023-12-20T04:28:13.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.867+0000 7f0bfacb6640 1 Processor -- start 2023-12-20T04:28:13.872 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.867+0000 7f0bfacb6640 1 -- start start 2023-12-20T04:28:13.873 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bfacb6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0bf40d14d0 0x7f0bf40b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bfacb6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf40d2650 con 0x7f0bf40d3300 2023-12-20T04:28:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bfacb6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0bf40d1a40 con 0x7f0bf415ce60 2023-12-20T04:28:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bfacb6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0bf40d1d20 con 0x7f0bf40d14d0 2023-12-20T04:28:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bf9253640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0bf40d14d0 0x7f0bf40b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bf9253640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0bf40d14d0 0x7f0bf40b6ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36638/0 (socket says 172.21.15.39:36638) 2023-12-20T04:28:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bf9253640 1 -- 172.21.15.39:0/800533758 learned_addr learned my addr 172.21.15.39:0/800533758 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:13.874 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bf9253640 1 -- 172.21.15.39:0/800533758 >> v1:172.21.15.39:6789/0 conn(0x7f0bf415ce60 legacy=0x7f0bf40d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bf9253640 1 -- 172.21.15.39:0/800533758 >> v1:172.21.15.32:6789/0 conn(0x7f0bf40d3300 legacy=0x7f0bf40d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bf9253640 1 -- 172.21.15.39:0/800533758 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bf40d2000 con 0x7f0bf40d14d0 2023-12-20T04:28:13.875 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bf37fe640 1 -- 172.21.15.39:0/800533758 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3880326662 0 0) 0x7f0bf40d2650 con 0x7f0bf40d3300 2023-12-20T04:28:13.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.871+0000 7f0bf9253640 1 --2- 172.21.15.39:0/800533758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bf40d14d0 0x7f0bf40b6ea0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009e50 tx=0x7f0bec01f1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=467912545361972a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:13.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bf37fe640 1 -- 172.21.15.39:0/800533758 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0af400 con 0x7f0bf40d14d0 2023-12-20T04:28:13.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bf37fe640 1 -- 172.21.15.39:0/800533758 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bec0ab050 con 0x7f0bf40d14d0 2023-12-20T04:28:13.877 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bf37fe640 1 -- 172.21.15.39:0/800533758 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bec0af780 con 0x7f0bf40d14d0 2023-12-20T04:28:13.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 -- 172.21.15.39:0/800533758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bf40d14d0 msgr2=0x7f0bf40b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:13.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 --2- 172.21.15.39:0/800533758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bf40d14d0 0x7f0bf40b6ea0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f0bec009e50 tx=0x7f0bec01f1f0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 -- 172.21.15.39:0/800533758 shutdown_connections 2023-12-20T04:28:13.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 --2- 172.21.15.39:0/800533758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bf40d14d0 0x7f0bf40b6ea0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 -- 172.21.15.39:0/800533758 >> 172.21.15.39:0/800533758 conn(0x7f0bf406b5e0 msgr2=0x7f0bf406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:13.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 -- 172.21.15.39:0/800533758 shutdown_connections 2023-12-20T04:28:13.878 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 -- 172.21.15.39:0/800533758 wait complete. 2023-12-20T04:28:13.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 Processor -- start 2023-12-20T04:28:13.879 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 -- start start 2023-12-20T04:28:13.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bf40d14d0 0x7f0bf41489d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:13.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bf40d3300 0x7f0bf40b41a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:13.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bf415ce60 0x7f0bf40b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:13.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf414f250 con 0x7f0bf40d3300 2023-12-20T04:28:13.881 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40c9670 con 0x7f0bf415ce60 2023-12-20T04:28:13.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.875+0000 7f0bfacb6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0bf40c91e0 con 0x7f0bf40d14d0 2023-12-20T04:28:13.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf9253640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bf415ce60 0x7f0bf40b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:13.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf8a52640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bf40d14d0 0x7f0bf41489d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:13.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf9253640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bf415ce60 0x7f0bf40b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36640/0 (socket says 172.21.15.39:36640) 2023-12-20T04:28:13.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf8a52640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bf40d14d0 0x7f0bf41489d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:55146/0 (socket says 172.21.15.39:55146) 2023-12-20T04:28:13.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf9253640 1 -- 172.21.15.39:0/2957449299 learned_addr learned my addr 172.21.15.39:0/2957449299 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:13.882 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf3fff640 1 --2- 172.21.15.39:0/2957449299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bf40d3300 0x7f0bf40b41a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:13.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf8a52640 1 -- 172.21.15.39:0/2957449299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bf415ce60 msgr2=0x7f0bf40b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:13.883 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf8a52640 1 --2- 172.21.15.39:0/2957449299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bf415ce60 0x7f0bf40b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf8a52640 1 -- 172.21.15.39:0/2957449299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bf40d3300 msgr2=0x7f0bf40b41a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf8a52640 1 --2- 172.21.15.39:0/2957449299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bf40d3300 0x7f0bf40b41a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf8a52640 1 -- 172.21.15.39:0/2957449299 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bec009b30 con 0x7f0bf40d14d0 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf9253640 1 --2- 172.21.15.39:0/2957449299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bf415ce60 0x7f0bf40b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf8a52640 1 --2- 172.21.15.39:0/2957449299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bf40d14d0 0x7f0bf41489d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0be800c3c0 tx=0x7f0be800c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf17fa640 1 -- 172.21.15.39:0/2957449299 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be80052d0 con 0x7f0bf40d14d0 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf17fa640 1 -- 172.21.15.39:0/2957449299 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0be8005430 con 0x7f0bf40d14d0 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf17fa640 1 -- 172.21.15.39:0/2957449299 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0be8007260 con 0x7f0bf40d14d0 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bfacb6640 1 -- 172.21.15.39:0/2957449299 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0bf40c85d0 con 0x7f0bf40d14d0 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bfacb6640 1 -- 172.21.15.39:0/2957449299 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0bf40c8b90 con 0x7f0bf40d14d0 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf17fa640 1 -- 172.21.15.39:0/2957449299 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f0be80067e0 con 0x7f0bf40d14d0 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bf17fa640 1 -- 172.21.15.39:0/2957449299 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(3..3 src has 1..3) v4 ==== 840+0+0 (secure 0 0 0) 0x7f0be80039f0 con 0x7f0bf40d14d0 2023-12-20T04:28:13.884 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.879+0000 7f0bfacb6640 1 -- 172.21.15.39:0/2957449299 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0bf404e000 con 0x7f0bf40d14d0 2023-12-20T04:28:13.886 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.883+0000 7f0bf17fa640 1 -- 172.21.15.39:0/2957449299 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f0be8020060 con 0x7f0bf40d14d0 2023-12-20T04:28:13.913 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:13.911+0000 7fb3fdf4f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:28:13.917 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:13.918+0000 7f1cbfcdc180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:28:13.956 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.951+0000 7f0bfacb6640 1 -- 172.21.15.39:0/2957449299 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd new", "uuid": "14ba777b-bae8-4e35-b63d-eafde55014d1", "id": 1} v 0) v1 -- 0x7f0bf40cdd70 con 0x7f0bf40d14d0 2023-12-20T04:28:13.960 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.955+0000 7f0bf17fa640 1 -- 172.21.15.39:0/2957449299 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "14ba777b-bae8-4e35-b63d-eafde55014d1", "id": 1}]=0 v4) v1 ==== 112+0+2 (secure 0 0 0) 0x7f0be80290b0 con 0x7f0bf40d14d0 2023-12-20T04:28:13.961 INFO:teuthology.orchestra.run.smithi039.stdout:1 2023-12-20T04:28:13.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.959+0000 7f0bfacb6640 1 -- 172.21.15.39:0/2957449299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bf40d14d0 msgr2=0x7f0bf41489d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:13.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.959+0000 7f0bfacb6640 1 --2- 172.21.15.39:0/2957449299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bf40d14d0 0x7f0bf41489d0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f0be800c3c0 tx=0x7f0be800c880 comp rx=0 tx=0).stop 2023-12-20T04:28:13.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.959+0000 7f0bfacb6640 1 -- 172.21.15.39:0/2957449299 shutdown_connections 2023-12-20T04:28:13.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.959+0000 7f0bfacb6640 1 --2- 172.21.15.39:0/2957449299 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0bf415ce60 0x7f0bf40b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.959+0000 7f0bfacb6640 1 --2- 172.21.15.39:0/2957449299 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0bf40d3300 0x7f0bf40b41a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.959+0000 7f0bfacb6640 1 --2- 172.21.15.39:0/2957449299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0bf40d14d0 0x7f0bf41489d0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:13.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.959+0000 7f0bfacb6640 1 -- 172.21.15.39:0/2957449299 >> 172.21.15.39:0/2957449299 conn(0x7f0bf406b5e0 msgr2=0x7f0bf406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:13.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.959+0000 7f0bfacb6640 1 -- 172.21.15.39:0/2957449299 shutdown_connections 2023-12-20T04:28:13.962 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:13.959+0000 7f0bfacb6640 1 -- 172.21.15.39:0/2957449299 wait complete. 2023-12-20T04:28:13.981 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new b5f84b62-e85d-43c1-ba26-54638415dfc3 2 2023-12-20T04:28:14.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.119+0000 7f1b37255640 1 Processor -- start 2023-12-20T04:28:14.122 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.119+0000 7f1b37255640 1 -- start start 2023-12-20T04:28:14.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.119+0000 7f1b37255640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1b3015cd60 0x7f1b30070c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.119+0000 7f1b37255640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1b300cb490 0x7f1b300cb900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.124 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.119+0000 7f1b37255640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1b300cdce0 con 0x7f1b3006cfa0 2023-12-20T04:28:14.125 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.119+0000 7f1b37255640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b30160ad0 con 0x7f1b300cb490 2023-12-20T04:28:14.125 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.119+0000 7f1b37255640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1b300cbe40 con 0x7f1b3015cd60 2023-12-20T04:28:14.125 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.119+0000 7f1b2ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1b3006cfa0 0x7f1b300cd5d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.39:53576/0 (socket says 172.21.15.39:53576) 2023-12-20T04:28:14.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.119+0000 7f1b2ffff640 1 -- 172.21.15.39:0/1402189843 learned_addr learned my addr 172.21.15.39:0/1402189843 (peer_addr_for_me v1:172.21.15.39:0/0) 2023-12-20T04:28:14.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.119+0000 7f1b357f2640 1 --2- 172.21.15.39:0/1402189843 >> v2:172.21.15.32:3300/0 conn(0x7f1b300cb490 0x7f1b300cb900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.119+0000 7f1b34ff1640 1 --2- 172.21.15.39:0/1402189843 >> v2:172.21.15.39:3300/0 conn(0x7f1b3015cd60 0x7f1b30070c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.123+0000 7f1b2f7fe640 1 -- 172.21.15.39:0/1402189843 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 948200400 0 0) 0x7f1b300cdce0 con 0x7f1b3006cfa0 2023-12-20T04:28:14.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.123+0000 7f1b2f7fe640 1 -- 172.21.15.39:0/1402189843 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1b20003530 con 0x7f1b3006cfa0 2023-12-20T04:28:14.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.123+0000 7f1b34ff1640 1 -- 172.21.15.39:0/1402189843 >> v2:172.21.15.32:3300/0 conn(0x7f1b300cb490 msgr2=0x7f1b300cb900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:14.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.123+0000 7f1b34ff1640 1 --2- 172.21.15.39:0/1402189843 >> v2:172.21.15.32:3300/0 conn(0x7f1b300cb490 0x7f1b300cb900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.123+0000 7f1b34ff1640 1 -- 172.21.15.39:0/1402189843 >> v1:172.21.15.32:6789/0 conn(0x7f1b3006cfa0 legacy=0x7f1b300cd5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.126 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.123+0000 7f1b34ff1640 1 -- 172.21.15.39:0/1402189843 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b300ca650 con 0x7f1b3015cd60 2023-12-20T04:28:14.128 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.123+0000 7f1b34ff1640 1 --2- 172.21.15.39:0/1402189843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3015cd60 0x7f1b30070c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1b3006ac80 tx=0x7f1b2401c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5640c72414d49ea server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:14.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.123+0000 7f1b2f7fe640 1 -- 172.21.15.39:0/1402189843 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b240a7070 con 0x7f1b3015cd60 2023-12-20T04:28:14.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.123+0000 7f1b2f7fe640 1 -- 172.21.15.39:0/1402189843 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b240ab430 con 0x7f1b3015cd60 2023-12-20T04:28:14.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.123+0000 7f1b2f7fe640 1 -- 172.21.15.39:0/1402189843 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b240ab7c0 con 0x7f1b3015cd60 2023-12-20T04:28:14.129 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 -- 172.21.15.39:0/1402189843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3015cd60 msgr2=0x7f1b30070c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 --2- 172.21.15.39:0/1402189843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3015cd60 0x7f1b30070c40 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f1b3006ac80 tx=0x7f1b2401c430 comp rx=0 tx=0).stop 2023-12-20T04:28:14.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 -- 172.21.15.39:0/1402189843 shutdown_connections 2023-12-20T04:28:14.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 --2- 172.21.15.39:0/1402189843 >> v2:172.21.15.32:3300/0 conn(0x7f1b300cb490 0x7f1b300cb900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 --2- 172.21.15.39:0/1402189843 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b3015cd60 0x7f1b30070c40 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 -- 172.21.15.39:0/1402189843 >> 172.21.15.39:0/1402189843 conn(0x7f1b30069d20 msgr2=0x7f1b3006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:14.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 -- 172.21.15.39:0/1402189843 shutdown_connections 2023-12-20T04:28:14.130 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 -- 172.21.15.39:0/1402189843 wait complete. 2023-12-20T04:28:14.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 Processor -- start 2023-12-20T04:28:14.131 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 -- start start 2023-12-20T04:28:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b3006cfa0 0x7f1b3015f250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b300cb490 0x7f1b3013f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.132 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1b3015cd60 0x7f1b3015ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b301509a0 con 0x7f1b3006cfa0 2023-12-20T04:28:14.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1b30151520 con 0x7f1b300cb490 2023-12-20T04:28:14.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b37255640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1b3015f790 con 0x7f1b3015cd60 2023-12-20T04:28:14.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b357f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1b3015cd60 0x7f1b3015ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b2ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b300cb490 0x7f1b3013f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b34ff1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b3006cfa0 0x7f1b3015f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b357f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1b3015cd60 0x7f1b3015ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:55156/0 (socket says 172.21.15.39:55156) 2023-12-20T04:28:14.133 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b357f2640 1 -- 172.21.15.39:0/1261447821 learned_addr learned my addr 172.21.15.39:0/1261447821 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:14.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b357f2640 1 -- 172.21.15.39:0/1261447821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b300cb490 msgr2=0x7f1b3013f300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b357f2640 1 --2- 172.21.15.39:0/1261447821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b300cb490 0x7f1b3013f300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.134 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b357f2640 1 -- 172.21.15.39:0/1261447821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b3006cfa0 msgr2=0x7f1b3015f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b357f2640 1 --2- 172.21.15.39:0/1261447821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b3006cfa0 0x7f1b3015f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b357f2640 1 -- 172.21.15.39:0/1261447821 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1b24009420 con 0x7f1b3015cd60 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.127+0000 7f1b2ffff640 1 --2- 172.21.15.39:0/1261447821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b300cb490 0x7f1b3013f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.131+0000 7f1b357f2640 1 --2- 172.21.15.39:0/1261447821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1b3015cd60 0x7f1b3015ae20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800efc0 tx=0x7f1b2800c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.131+0000 7f1b2d7fa640 1 -- 172.21.15.39:0/1261447821 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28014480 con 0x7f1b3015cd60 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.131+0000 7f1b2d7fa640 1 -- 172.21.15.39:0/1261447821 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1b2800f040 con 0x7f1b3015cd60 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.131+0000 7f1b37255640 1 -- 172.21.15.39:0/1261447821 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1b3013f840 con 0x7f1b3015cd60 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.131+0000 7f1b2d7fa640 1 -- 172.21.15.39:0/1261447821 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1b28014820 con 0x7f1b3015cd60 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.131+0000 7f1b37255640 1 -- 172.21.15.39:0/1261447821 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1b30150ed0 con 0x7f1b3015cd60 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.131+0000 7f1b2d7fa640 1 -- 172.21.15.39:0/1261447821 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f1b28014c80 con 0x7f1b3015cd60 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.131+0000 7f1b2d7fa640 1 -- 172.21.15.39:0/1261447821 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(4..4 src has 1..4) v4 ==== 955+0+0 (secure 0 0 0) 0x7f1b2801d820 con 0x7f1b3015cd60 2023-12-20T04:28:14.135 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.131+0000 7f1b37255640 1 -- 172.21.15.39:0/1261447821 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1b3006acb0 con 0x7f1b3015cd60 2023-12-20T04:28:14.137 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.135+0000 7f1b2d7fa640 1 -- 172.21.15.39:0/1261447821 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f1b2803bb40 con 0x7f1b3015cd60 2023-12-20T04:28:14.205 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.203+0000 7f1b37255640 1 -- 172.21.15.39:0/1261447821 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd new", "uuid": "b5f84b62-e85d-43c1-ba26-54638415dfc3", "id": 2} v 0) v1 -- 0x7f1b30151690 con 0x7f1b3015cd60 2023-12-20T04:28:14.210 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.207+0000 7f1b2d7fa640 1 -- 172.21.15.39:0/1261447821 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b5f84b62-e85d-43c1-ba26-54638415dfc3", "id": 2}]=0 v5) v1 ==== 112+0+2 (secure 0 0 0) 0x7f1b2802a070 con 0x7f1b3015cd60 2023-12-20T04:28:14.210 INFO:teuthology.orchestra.run.smithi039.stdout:2 2023-12-20T04:28:14.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.207+0000 7f1b37255640 1 -- 172.21.15.39:0/1261447821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1b3015cd60 msgr2=0x7f1b3015ae20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.207+0000 7f1b37255640 1 --2- 172.21.15.39:0/1261447821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1b3015cd60 0x7f1b3015ae20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1b2800efc0 tx=0x7f1b2800c490 comp rx=0 tx=0).stop 2023-12-20T04:28:14.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.207+0000 7f1b37255640 1 -- 172.21.15.39:0/1261447821 shutdown_connections 2023-12-20T04:28:14.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.207+0000 7f1b37255640 1 --2- 172.21.15.39:0/1261447821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1b3015cd60 0x7f1b3015ae20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.211 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.207+0000 7f1b37255640 1 --2- 172.21.15.39:0/1261447821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1b300cb490 0x7f1b3013f300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.207+0000 7f1b37255640 1 --2- 172.21.15.39:0/1261447821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1b3006cfa0 0x7f1b3015f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.207+0000 7f1b37255640 1 -- 172.21.15.39:0/1261447821 >> 172.21.15.39:0/1261447821 conn(0x7f1b30069d20 msgr2=0x7f1b300cee80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:14.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.207+0000 7f1b37255640 1 -- 172.21.15.39:0/1261447821 shutdown_connections 2023-12-20T04:28:14.212 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.207+0000 7f1b37255640 1 -- 172.21.15.39:0/1261447821 wait complete. 2023-12-20T04:28:14.231 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new b654e597-0ae4-42f3-a726-adcbd8c6d9a1 3 2023-12-20T04:28:14.371 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.367+0000 7f2ca9179640 1 Processor -- start 2023-12-20T04:28:14.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.367+0000 7f2ca9179640 1 -- start start 2023-12-20T04:28:14.372 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.367+0000 7f2ca9179640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca415cc30 0x7f2ca40b8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.367+0000 7f2ca9179640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca40d2090 con 0x7f2ca40d3060 2023-12-20T04:28:14.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.367+0000 7f2ca9179640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ca40d1480 con 0x7f2ca40d0ca0 2023-12-20T04:28:14.373 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.367+0000 7f2ca9179640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca415d010 con 0x7f2ca415cc30 2023-12-20T04:28:14.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.367+0000 7f2ca3fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca415cc30 0x7f2ca40b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.367+0000 7f2ca3fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca415cc30 0x7f2ca40b8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:55166/0 (socket says 172.21.15.39:55166) 2023-12-20T04:28:14.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.367+0000 7f2ca3fff640 1 -- 172.21.15.39:0/3398038981 learned_addr learned my addr 172.21.15.39:0/3398038981 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:14.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.371+0000 7f2ca3fff640 1 -- 172.21.15.39:0/3398038981 >> v1:172.21.15.39:6789/0 conn(0x7f2ca40d0ca0 legacy=0x7f2ca40d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.371+0000 7f2ca3fff640 1 -- 172.21.15.39:0/3398038981 >> v1:172.21.15.32:6789/0 conn(0x7f2ca40d3060 legacy=0x7f2ca40b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.374 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.371+0000 7f2ca3fff640 1 -- 172.21.15.39:0/3398038981 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ca40d1760 con 0x7f2ca415cc30 2023-12-20T04:28:14.376 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.371+0000 7f2ca3fff640 1 --2- 172.21.15.39:0/3398038981 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca415cc30 0x7f2ca40b8520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2c940098a0 tx=0x7f2c9401c530 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d63f375647a42bc server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:14.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.371+0000 7f2ca2ffd640 1 -- 172.21.15.39:0/3398038981 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c940a95f0 con 0x7f2ca415cc30 2023-12-20T04:28:14.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.371+0000 7f2ca2ffd640 1 -- 172.21.15.39:0/3398038981 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c940a9750 con 0x7f2ca415cc30 2023-12-20T04:28:14.377 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.371+0000 7f2ca2ffd640 1 -- 172.21.15.39:0/3398038981 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c940a9ae0 con 0x7f2ca415cc30 2023-12-20T04:28:14.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 -- 172.21.15.39:0/3398038981 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca415cc30 msgr2=0x7f2ca40b8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 --2- 172.21.15.39:0/3398038981 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca415cc30 0x7f2ca40b8520 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f2c940098a0 tx=0x7f2c9401c530 comp rx=0 tx=0).stop 2023-12-20T04:28:14.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 -- 172.21.15.39:0/3398038981 shutdown_connections 2023-12-20T04:28:14.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 --2- 172.21.15.39:0/3398038981 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca415cc30 0x7f2ca40b8520 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 -- 172.21.15.39:0/3398038981 >> 172.21.15.39:0/3398038981 conn(0x7f2ca4065ce0 msgr2=0x7f2ca40660f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:14.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 -- 172.21.15.39:0/3398038981 shutdown_connections 2023-12-20T04:28:14.378 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 -- 172.21.15.39:0/3398038981 wait complete. 2023-12-20T04:28:14.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 Processor -- start 2023-12-20T04:28:14.379 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 -- start start 2023-12-20T04:28:14.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca40d0ca0 0x7f2ca40b4400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca40d3060 0x7f2ca40b1820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca415cc30 0x7f2ca40b3340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca414e9d0 con 0x7f2ca415cc30 2023-12-20T04:28:14.382 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca414ecb0 con 0x7f2ca40d3060 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca9179640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ca4157070 con 0x7f2ca40d0ca0 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca40d3060 0x7f2ca40b1820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca3fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca40d0ca0 0x7f2ca40b4400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca37fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca40d3060 0x7f2ca40b1820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36680/0 (socket says 172.21.15.39:36680) 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca3fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca40d0ca0 0x7f2ca40b4400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:55174/0 (socket says 172.21.15.39:55174) 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca37fe640 1 -- 172.21.15.39:0/1724368009 learned_addr learned my addr 172.21.15.39:0/1724368009 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.375+0000 7f2ca8978640 1 --2- 172.21.15.39:0/1724368009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca415cc30 0x7f2ca40b3340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca37fe640 1 -- 172.21.15.39:0/1724368009 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca40d0ca0 msgr2=0x7f2ca40b4400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca37fe640 1 --2- 172.21.15.39:0/1724368009 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca40d0ca0 0x7f2ca40b4400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca37fe640 1 -- 172.21.15.39:0/1724368009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca415cc30 msgr2=0x7f2ca40b3340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca37fe640 1 --2- 172.21.15.39:0/1724368009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca415cc30 0x7f2ca40b3340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca37fe640 1 -- 172.21.15.39:0/1724368009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2c940094d0 con 0x7f2ca40d3060 2023-12-20T04:28:14.383 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca3fff640 1 --2- 172.21.15.39:0/1724368009 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca40d0ca0 0x7f2ca40b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca37fe640 1 --2- 172.21.15.39:0/1724368009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca40d3060 0x7f2ca40b1820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800c3c0 tx=0x7f2c9800c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca0ff9640 1 -- 172.21.15.39:0/1724368009 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c980052d0 con 0x7f2ca40d3060 2023-12-20T04:28:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca0ff9640 1 -- 172.21.15.39:0/1724368009 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2c98005430 con 0x7f2ca40d3060 2023-12-20T04:28:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca9179640 1 -- 172.21.15.39:0/1724368009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ca4157350 con 0x7f2ca40d3060 2023-12-20T04:28:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca0ff9640 1 -- 172.21.15.39:0/1724368009 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2c98003840 con 0x7f2ca40d3060 2023-12-20T04:28:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca9179640 1 -- 172.21.15.39:0/1724368009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ca40b4940 con 0x7f2ca40d3060 2023-12-20T04:28:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca0ff9640 1 -- 172.21.15.39:0/1724368009 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f2c98003ca0 con 0x7f2ca40d3060 2023-12-20T04:28:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca0ff9640 1 -- 172.21.15.39:0/1724368009 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(5..5 src has 1..5) v4 ==== 1070+0+0 (secure 0 0 0) 0x7f2c98011a20 con 0x7f2ca40d3060 2023-12-20T04:28:14.384 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.379+0000 7f2ca9179640 1 -- 172.21.15.39:0/1724368009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ca40d2500 con 0x7f2ca40d3060 2023-12-20T04:28:14.385 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.383+0000 7f2ca0ff9640 1 -- 172.21.15.39:0/1724368009 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f2ca40d2500 con 0x7f2ca40d3060 2023-12-20T04:28:14.454 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.451+0000 7f2ca9179640 1 -- 172.21.15.39:0/1724368009 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd new", "uuid": "b654e597-0ae4-42f3-a726-adcbd8c6d9a1", "id": 3} v 0) v1 -- 0x7f2ca4074100 con 0x7f2ca40d3060 2023-12-20T04:28:14.458 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.455+0000 7f2ca0ff9640 1 -- 172.21.15.39:0/1724368009 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "b654e597-0ae4-42f3-a726-adcbd8c6d9a1", "id": 3}]=0 v6) v1 ==== 112+0+2 (secure 0 0 0) 0x7f2c98017070 con 0x7f2ca40d3060 2023-12-20T04:28:14.459 INFO:teuthology.orchestra.run.smithi039.stdout:3 2023-12-20T04:28:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.455+0000 7f2ca9179640 1 -- 172.21.15.39:0/1724368009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca40d3060 msgr2=0x7f2ca40b1820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.455+0000 7f2ca9179640 1 --2- 172.21.15.39:0/1724368009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca40d3060 0x7f2ca40b1820 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f2c9800c3c0 tx=0x7f2c9800c880 comp rx=0 tx=0).stop 2023-12-20T04:28:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.455+0000 7f2ca9179640 1 -- 172.21.15.39:0/1724368009 shutdown_connections 2023-12-20T04:28:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.455+0000 7f2ca9179640 1 --2- 172.21.15.39:0/1724368009 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ca415cc30 0x7f2ca40b3340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.455+0000 7f2ca9179640 1 --2- 172.21.15.39:0/1724368009 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ca40d3060 0x7f2ca40b1820 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.455+0000 7f2ca9179640 1 --2- 172.21.15.39:0/1724368009 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ca40d0ca0 0x7f2ca40b4400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.455+0000 7f2ca9179640 1 -- 172.21.15.39:0/1724368009 >> 172.21.15.39:0/1724368009 conn(0x7f2ca4065ce0 msgr2=0x7f2ca415f790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:14.460 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.455+0000 7f2ca9179640 1 -- 172.21.15.39:0/1724368009 shutdown_connections 2023-12-20T04:28:14.461 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.455+0000 7f2ca9179640 1 -- 172.21.15.39:0/1724368009 wait complete. 2023-12-20T04:28:14.479 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new 90de0eda-35f4-45e0-b4ac-7d603612c980 4 2023-12-20T04:28:14.516 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:14.511+0000 7fb3fdf4f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:28:14.580 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:14.578+0000 7f1cbfcdc180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:28:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.615+0000 7fd8049e9640 1 Processor -- start 2023-12-20T04:28:14.620 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.615+0000 7fd8049e9640 1 -- start start 2023-12-20T04:28:14.621 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.619+0000 7fd8049e9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd80015ce60 0x7fd8000d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.619+0000 7fd8049e9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8000d2650 con 0x7fd8000d3130 2023-12-20T04:28:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.619+0000 7fd8049e9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8000d1a40 con 0x7fd8000d14d0 2023-12-20T04:28:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.619+0000 7fd8049e9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8000d35a0 con 0x7fd80015ce60 2023-12-20T04:28:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.619+0000 7fd7fe59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd80015ce60 0x7fd8000d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.619+0000 7fd7fe59c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd80015ce60 0x7fd8000d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36692/0 (socket says 172.21.15.39:36692) 2023-12-20T04:28:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.619+0000 7fd7fe59c640 1 -- 172.21.15.39:0/714525030 learned_addr learned my addr 172.21.15.39:0/714525030 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.619+0000 7fd7fe59c640 1 -- 172.21.15.39:0/714525030 >> v1:172.21.15.39:6789/0 conn(0x7fd8000d14d0 legacy=0x7fd8000b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.622 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.619+0000 7fd7fe59c640 1 -- 172.21.15.39:0/714525030 >> v1:172.21.15.32:6789/0 conn(0x7fd8000d3130 legacy=0x7fd8000d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.624 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.619+0000 7fd7fe59c640 1 -- 172.21.15.39:0/714525030 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8000d1d20 con 0x7fd80015ce60 2023-12-20T04:28:14.626 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.623+0000 7fd7fe59c640 1 --2- 172.21.15.39:0/714525030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd80015ce60 0x7fd8000d2bf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00098a0 tx=0x7fd7f001ed10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=47d73939e680890e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:14.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.623+0000 7fd7fd59a640 1 -- 172.21.15.39:0/714525030 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00a8a10 con 0x7fd80015ce60 2023-12-20T04:28:14.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.623+0000 7fd7fd59a640 1 -- 172.21.15.39:0/714525030 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f00a8b70 con 0x7fd80015ce60 2023-12-20T04:28:14.627 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.623+0000 7fd7fd59a640 1 -- 172.21.15.39:0/714525030 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00b0440 con 0x7fd80015ce60 2023-12-20T04:28:14.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.623+0000 7fd8049e9640 1 -- 172.21.15.39:0/714525030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd80015ce60 msgr2=0x7fd8000d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.623+0000 7fd8049e9640 1 --2- 172.21.15.39:0/714525030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd80015ce60 0x7fd8000d2bf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00098a0 tx=0x7fd7f001ed10 comp rx=0 tx=0).stop 2023-12-20T04:28:14.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.623+0000 7fd8049e9640 1 -- 172.21.15.39:0/714525030 shutdown_connections 2023-12-20T04:28:14.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.623+0000 7fd8049e9640 1 --2- 172.21.15.39:0/714525030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd80015ce60 0x7fd8000d2bf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.623+0000 7fd8049e9640 1 -- 172.21.15.39:0/714525030 >> 172.21.15.39:0/714525030 conn(0x7fd80006b5e0 msgr2=0x7fd80006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:14.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.623+0000 7fd8049e9640 1 -- 172.21.15.39:0/714525030 shutdown_connections 2023-12-20T04:28:14.628 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.623+0000 7fd8049e9640 1 -- 172.21.15.39:0/714525030 wait complete. 2023-12-20T04:28:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd8049e9640 1 Processor -- start 2023-12-20T04:28:14.629 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd8049e9640 1 -- start start 2023-12-20T04:28:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd8049e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8000d14d0 0x7fd8001522e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd8049e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8000d3130 0x7fd8000c9f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd8049e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd80015ce60 0x7fd80015e540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd8049e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80015f000 con 0x7fd8000d3130 2023-12-20T04:28:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd8049e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8001546c0 con 0x7fd8000d14d0 2023-12-20T04:28:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd8049e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd800154960 con 0x7fd80015ce60 2023-12-20T04:28:14.631 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7fe59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8000d14d0 0x7fd8001522e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7fe59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8000d14d0 0x7fd8001522e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36706/0 (socket says 172.21.15.39:36706) 2023-12-20T04:28:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7fe59c640 1 -- 172.21.15.39:0/2047652645 learned_addr learned my addr 172.21.15.39:0/2047652645 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7fdd9b640 1 --2- 172.21.15.39:0/2047652645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8000d3130 0x7fd8000c9f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7fed9d640 1 --2- 172.21.15.39:0/2047652645 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd80015ce60 0x7fd80015e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7fe59c640 1 -- 172.21.15.39:0/2047652645 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd80015ce60 msgr2=0x7fd80015e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7fe59c640 1 --2- 172.21.15.39:0/2047652645 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd80015ce60 0x7fd80015e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7fe59c640 1 -- 172.21.15.39:0/2047652645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8000d3130 msgr2=0x7fd8000c9f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7fe59c640 1 --2- 172.21.15.39:0/2047652645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8000d3130 0x7fd8000c9f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.632 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7fe59c640 1 -- 172.21.15.39:0/2047652645 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f00094d0 con 0x7fd8000d14d0 2023-12-20T04:28:14.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7fe59c640 1 --2- 172.21.15.39:0/2047652645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8000d14d0 0x7fd8001522e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00b3040 tx=0x7fd7f001bab0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:14.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7eeffd640 1 -- 172.21.15.39:0/2047652645 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00ba600 con 0x7fd8000d14d0 2023-12-20T04:28:14.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7eeffd640 1 -- 172.21.15.39:0/2047652645 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f00ba760 con 0x7fd8000d14d0 2023-12-20T04:28:14.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd8049e9640 1 -- 172.21.15.39:0/2047652645 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd800154c40 con 0x7fd8000d14d0 2023-12-20T04:28:14.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7eeffd640 1 -- 172.21.15.39:0/2047652645 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00bab10 con 0x7fd8000d14d0 2023-12-20T04:28:14.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd8049e9640 1 -- 172.21.15.39:0/2047652645 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd80015d7b0 con 0x7fd8000d14d0 2023-12-20T04:28:14.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7eeffd640 1 -- 172.21.15.39:0/2047652645 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd7f00c4460 con 0x7fd8000d14d0 2023-12-20T04:28:14.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd7eeffd640 1 -- 172.21.15.39:0/2047652645 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(6..6 src has 1..6) v4 ==== 1185+0+0 (secure 0 0 0) 0x7fd7f00c38d0 con 0x7fd8000d14d0 2023-12-20T04:28:14.634 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.627+0000 7fd8049e9640 1 -- 172.21.15.39:0/2047652645 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80004e000 con 0x7fd8000d14d0 2023-12-20T04:28:14.635 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.631+0000 7fd7eeffd640 1 -- 172.21.15.39:0/2047652645 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7fd7f00b5070 con 0x7fd8000d14d0 2023-12-20T04:28:14.672 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:14.670+0000 7f1cbfcdc180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:28:14.703 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.699+0000 7fd8049e9640 1 -- 172.21.15.39:0/2047652645 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd new", "uuid": "90de0eda-35f4-45e0-b4ac-7d603612c980", "id": 4} v 0) v1 -- 0x7fd8000d1130 con 0x7fd8000d14d0 2023-12-20T04:28:14.708 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.703+0000 7fd7eeffd640 1 -- 172.21.15.39:0/2047652645 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "90de0eda-35f4-45e0-b4ac-7d603612c980", "id": 4}]=0 v7) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd7f00c3c00 con 0x7fd8000d14d0 2023-12-20T04:28:14.709 INFO:teuthology.orchestra.run.smithi039.stdout:4 2023-12-20T04:28:14.709 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.707+0000 7fd8049e9640 1 -- 172.21.15.39:0/2047652645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8000d14d0 msgr2=0x7fd8001522e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.707+0000 7fd8049e9640 1 --2- 172.21.15.39:0/2047652645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8000d14d0 0x7fd8001522e0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7fd7f00b3040 tx=0x7fd7f001bab0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.707+0000 7fd8049e9640 1 -- 172.21.15.39:0/2047652645 shutdown_connections 2023-12-20T04:28:14.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.707+0000 7fd8049e9640 1 --2- 172.21.15.39:0/2047652645 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd80015ce60 0x7fd80015e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.707+0000 7fd8049e9640 1 --2- 172.21.15.39:0/2047652645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8000d3130 0x7fd8000c9f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.707+0000 7fd8049e9640 1 --2- 172.21.15.39:0/2047652645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8000d14d0 0x7fd8001522e0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.707+0000 7fd8049e9640 1 -- 172.21.15.39:0/2047652645 >> 172.21.15.39:0/2047652645 conn(0x7fd80006b5e0 msgr2=0x7fd8000b18b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:14.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.707+0000 7fd8049e9640 1 -- 172.21.15.39:0/2047652645 shutdown_connections 2023-12-20T04:28:14.710 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.707+0000 7fd8049e9640 1 -- 172.21.15.39:0/2047652645 wait complete. 2023-12-20T04:28:14.729 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new 81996767-dac0-469c-b381-07c0d34da475 5 2023-12-20T04:28:14.736 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:14.734+0000 7f1cbfcdc180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:28:14.747 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:14.743+0000 7fb3fdf4f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:28:14.829 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:14.826+0000 7f1cbfcdc180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:28:14.891 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.887+0000 7f01dd96d640 1 Processor -- start 2023-12-20T04:28:14.892 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.887+0000 7f01dd96d640 1 -- start start 2023-12-20T04:28:14.893 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.887+0000 7f01dd96d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01d80d3300 0x7f01d80d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.887+0000 7f01dd96d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d80d2650 con 0x7f01d815ce60 2023-12-20T04:28:14.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.887+0000 7f01dd96d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01d80d1a40 con 0x7f01d80d1300 2023-12-20T04:28:14.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.887+0000 7f01dd96d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01d80d1d20 con 0x7f01d80d3300 2023-12-20T04:28:14.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.887+0000 7f01cffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01d80d3300 0x7f01d80d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.887+0000 7f01cffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01d80d3300 0x7f01d80d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.39:50668/0 (socket says 172.21.15.39:50668) 2023-12-20T04:28:14.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.887+0000 7f01cffff640 1 -- 172.21.15.39:0/448266001 learned_addr learned my addr 172.21.15.39:0/448266001 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:14.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.887+0000 7f01d77fe640 1 --1- 172.21.15.39:0/448266001 >> v1:172.21.15.39:6789/0 conn(0x7f01d80d1300 0x7f01d80b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.39:44060/0 (socket says 172.21.15.39:44060) 2023-12-20T04:28:14.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.891+0000 7f01cffff640 1 -- 172.21.15.39:0/448266001 >> v1:172.21.15.39:6789/0 conn(0x7f01d80d1300 legacy=0x7f01d80b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.891+0000 7f01cffff640 1 -- 172.21.15.39:0/448266001 >> v1:172.21.15.32:6789/0 conn(0x7f01d815ce60 legacy=0x7f01d80d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.891+0000 7f01d67fc640 1 -- 172.21.15.39:0/448266001 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1547430825 0 0) 0x7f01d80d2650 con 0x7f01d815ce60 2023-12-20T04:28:14.894 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.891+0000 7f01cffff640 1 -- 172.21.15.39:0/448266001 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d80d2000 con 0x7f01d80d3300 2023-12-20T04:28:14.895 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:14.890+0000 7f1cbfcdc180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:28:14.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.891+0000 7f01cffff640 1 --2- 172.21.15.39:0/448266001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d80d3300 0x7f01d80d0dc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009870 tx=0x7f01c001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f724859ea0f28af5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:14.896 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.891+0000 7f01d67fc640 1 -- 172.21.15.39:0/448266001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c00ac630 con 0x7f01d80d3300 2023-12-20T04:28:14.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.891+0000 7f01d67fc640 1 -- 172.21.15.39:0/448266001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c00ac790 con 0x7f01d80d3300 2023-12-20T04:28:14.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.891+0000 7f01d67fc640 1 -- 172.21.15.39:0/448266001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c00acb20 con 0x7f01d80d3300 2023-12-20T04:28:14.897 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 -- 172.21.15.39:0/448266001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d80d3300 msgr2=0x7f01d80d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 --2- 172.21.15.39:0/448266001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d80d3300 0x7f01d80d0dc0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f01c0009870 tx=0x7f01c001c540 comp rx=0 tx=0).stop 2023-12-20T04:28:14.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 -- 172.21.15.39:0/448266001 shutdown_connections 2023-12-20T04:28:14.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 --2- 172.21.15.39:0/448266001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d80d3300 0x7f01d80d0dc0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 -- 172.21.15.39:0/448266001 >> 172.21.15.39:0/448266001 conn(0x7f01d806b5e0 msgr2=0x7f01d806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:14.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 -- 172.21.15.39:0/448266001 shutdown_connections 2023-12-20T04:28:14.898 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 -- 172.21.15.39:0/448266001 wait complete. 2023-12-20T04:28:14.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 Processor -- start 2023-12-20T04:28:14.899 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 -- start start 2023-12-20T04:28:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01d80d1300 0x7f01d814db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01d80d3300 0x7f01d8155cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d815ce60 0x7f01d80ca8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d81577b0 con 0x7f01d815ce60 2023-12-20T04:28:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01d80ca030 con 0x7f01d80d3300 2023-12-20T04:28:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01dd96d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01d8156200 con 0x7f01d80d1300 2023-12-20T04:28:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01d80d3300 0x7f01d8155cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01d6ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01d80d1300 0x7f01d814db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01d80d3300 0x7f01d8155cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36720/0 (socket says 172.21.15.39:36720) 2023-12-20T04:28:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01cffff640 1 -- 172.21.15.39:0/626667727 learned_addr learned my addr 172.21.15.39:0/626667727 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:14.902 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01d77fe640 1 --2- 172.21.15.39:0/626667727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d815ce60 0x7f01d80ca8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01d6ffd640 1 -- 172.21.15.39:0/626667727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01d80d3300 msgr2=0x7f01d8155cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01d6ffd640 1 --2- 172.21.15.39:0/626667727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01d80d3300 0x7f01d8155cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01d6ffd640 1 -- 172.21.15.39:0/626667727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d815ce60 msgr2=0x7f01d80ca8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01d6ffd640 1 --2- 172.21.15.39:0/626667727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d815ce60 0x7f01d80ca8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01d6ffd640 1 -- 172.21.15.39:0/626667727 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01c00094d0 con 0x7f01d80d1300 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01cffff640 1 --2- 172.21.15.39:0/626667727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01d80d3300 0x7f01d8155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.895+0000 7f01d77fe640 1 --2- 172.21.15.39:0/626667727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d815ce60 0x7f01d80ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.899+0000 7f01d6ffd640 1 --2- 172.21.15.39:0/626667727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01d80d1300 0x7f01d814db30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01c800c410 tx=0x7f01c800c8d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.899+0000 7f01d4ff9640 1 -- 172.21.15.39:0/626667727 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c80054e0 con 0x7f01d80d1300 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.899+0000 7f01d4ff9640 1 -- 172.21.15.39:0/626667727 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01c8003600 con 0x7f01d80d1300 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.899+0000 7f01d4ff9640 1 -- 172.21.15.39:0/626667727 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01c8003990 con 0x7f01d80d1300 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.899+0000 7f01dd96d640 1 -- 172.21.15.39:0/626667727 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01d81574d0 con 0x7f01d80d1300 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.899+0000 7f01dd96d640 1 -- 172.21.15.39:0/626667727 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01d81521b0 con 0x7f01d80d1300 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.899+0000 7f01d4ff9640 1 -- 172.21.15.39:0/626667727 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f01c801c440 con 0x7f01d80d1300 2023-12-20T04:28:14.903 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.899+0000 7f01d4ff9640 1 -- 172.21.15.39:0/626667727 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(7..7 src has 1..7) v4 ==== 1300+0+0 (secure 0 0 0) 0x7f01c8012980 con 0x7f01d80d1300 2023-12-20T04:28:14.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.899+0000 7f01dd96d640 1 -- 172.21.15.39:0/626667727 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01d804e000 con 0x7f01d80d1300 2023-12-20T04:28:14.904 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.899+0000 7f01d4ff9640 1 -- 172.21.15.39:0/626667727 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f01c8017070 con 0x7f01d80d1300 2023-12-20T04:28:14.975 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.971+0000 7f01dd96d640 1 -- 172.21.15.39:0/626667727 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd new", "uuid": "81996767-dac0-469c-b381-07c0d34da475", "id": 5} v 0) v1 -- 0x7f01d80cf230 con 0x7f01d80d1300 2023-12-20T04:28:14.979 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.975+0000 7f01d4ff9640 1 -- 172.21.15.39:0/626667727 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "81996767-dac0-469c-b381-07c0d34da475", "id": 5}]=0 v8) v1 ==== 112+0+2 (secure 0 0 0) 0x7f01c80124d0 con 0x7f01d80d1300 2023-12-20T04:28:14.980 INFO:teuthology.orchestra.run.smithi039.stdout:5 2023-12-20T04:28:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.975+0000 7f01dd96d640 1 -- 172.21.15.39:0/626667727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01d80d1300 msgr2=0x7f01d814db30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.975+0000 7f01dd96d640 1 --2- 172.21.15.39:0/626667727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01d80d1300 0x7f01d814db30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f01c800c410 tx=0x7f01c800c8d0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.975+0000 7f01dd96d640 1 -- 172.21.15.39:0/626667727 shutdown_connections 2023-12-20T04:28:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.975+0000 7f01dd96d640 1 --2- 172.21.15.39:0/626667727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01d815ce60 0x7f01d80ca8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.975+0000 7f01dd96d640 1 --2- 172.21.15.39:0/626667727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01d80d3300 0x7f01d8155cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.975+0000 7f01dd96d640 1 --2- 172.21.15.39:0/626667727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01d80d1300 0x7f01d814db30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.975+0000 7f01dd96d640 1 -- 172.21.15.39:0/626667727 >> 172.21.15.39:0/626667727 conn(0x7f01d806b5e0 msgr2=0x7f01d80b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.975+0000 7f01dd96d640 1 -- 172.21.15.39:0/626667727 shutdown_connections 2023-12-20T04:28:14.981 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:14.975+0000 7f01dd96d640 1 -- 172.21.15.39:0/626667727 wait complete. 2023-12-20T04:28:14.987 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:14.986+0000 7f1cbfcdc180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:28:14.999 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new e93092a0-e6d4-4ba7-97bd-50eab2ed529e 6 2023-12-20T04:28:15.050 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:15.050+0000 7f1cbfcdc180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:28:15.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.135+0000 7f7d0f666640 1 Processor -- start 2023-12-20T04:28:15.140 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.135+0000 7f7d0f666640 1 -- start start 2023-12-20T04:28:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.135+0000 7f7d0f666640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d0815ce60 0x7f7d080d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.141 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.135+0000 7f7d0f666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d3130 0x7f7d080d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.135+0000 7f7d0f666640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7d080d1300 0x7f7d080b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.135+0000 7f7d0f666640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d080d35a0 con 0x7f7d0815ce60 2023-12-20T04:28:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0f666640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d080d2650 con 0x7f7d080d1300 2023-12-20T04:28:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0f666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d080d1a40 con 0x7f7d080d3130 2023-12-20T04:28:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0d402640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d0815ce60 0x7f7d080d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0cc01640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d3130 0x7f7d080d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0d402640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d0815ce60 0x7f7d080d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.39:50686/0 (socket says 172.21.15.39:50686) 2023-12-20T04:28:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0d402640 1 -- 172.21.15.39:0/735922036 learned_addr learned my addr 172.21.15.39:0/735922036 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:15.142 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0dc03640 1 --2- 172.21.15.39:0/735922036 >> v2:172.21.15.39:3300/0 conn(0x7f7d080d1300 0x7f7d080b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0cc01640 1 -- 172.21.15.39:0/735922036 >> v2:172.21.15.39:3300/0 conn(0x7f7d080d1300 msgr2=0x7f7d080b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0cc01640 1 --2- 172.21.15.39:0/735922036 >> v2:172.21.15.39:3300/0 conn(0x7f7d080d1300 0x7f7d080b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0cc01640 1 -- 172.21.15.39:0/735922036 >> v2:172.21.15.32:3300/0 conn(0x7f7d0815ce60 msgr2=0x7f7d080d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0cc01640 1 --2- 172.21.15.39:0/735922036 >> v2:172.21.15.32:3300/0 conn(0x7f7d0815ce60 0x7f7d080d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0cc01640 1 -- 172.21.15.39:0/735922036 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d080d1d20 con 0x7f7d080d3130 2023-12-20T04:28:15.143 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0d402640 1 --2- 172.21.15.39:0/735922036 >> v2:172.21.15.32:3300/0 conn(0x7f7d0815ce60 0x7f7d080d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:15.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.139+0000 7f7d0cc01640 1 --2- 172.21.15.39:0/735922036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d3130 0x7f7d080d0dc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d080cfed0 tx=0x7f7cf801ec50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5f92b0b2b0d41560 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:15.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7cfffff640 1 -- 172.21.15.39:0/735922036 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf80a8910 con 0x7f7d080d3130 2023-12-20T04:28:15.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7cfffff640 1 -- 172.21.15.39:0/735922036 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7cf80a8a70 con 0x7f7d080d3130 2023-12-20T04:28:15.145 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7cfffff640 1 -- 172.21.15.39:0/735922036 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf80a8e00 con 0x7f7d080d3130 2023-12-20T04:28:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 -- 172.21.15.39:0/735922036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d3130 msgr2=0x7f7d080d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.146 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 --2- 172.21.15.39:0/735922036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d3130 0x7f7d080d0dc0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f7d080cfed0 tx=0x7f7cf801ec50 comp rx=0 tx=0).stop 2023-12-20T04:28:15.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 -- 172.21.15.39:0/735922036 shutdown_connections 2023-12-20T04:28:15.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 --2- 172.21.15.39:0/735922036 >> v2:172.21.15.39:3300/0 conn(0x7f7d080d1300 0x7f7d080b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 --2- 172.21.15.39:0/735922036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d3130 0x7f7d080d0dc0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 --2- 172.21.15.39:0/735922036 >> v2:172.21.15.32:3300/0 conn(0x7f7d0815ce60 0x7f7d080d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 -- 172.21.15.39:0/735922036 >> 172.21.15.39:0/735922036 conn(0x7f7d0806b5e0 msgr2=0x7f7d0806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:15.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 -- 172.21.15.39:0/735922036 shutdown_connections 2023-12-20T04:28:15.147 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 -- 172.21.15.39:0/735922036 wait complete. 2023-12-20T04:28:15.148 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 Processor -- start 2023-12-20T04:28:15.148 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 -- start start 2023-12-20T04:28:15.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.143+0000 7f7d0f666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d1300 0x7f7d081f55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0f666640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d080d3130 0x7f7d081e7b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0f666640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0815ce60 0x7f7d081e8050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0f666640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0814e450 con 0x7f7d080d3130 2023-12-20T04:28:15.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0f666640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0814d840 con 0x7f7d0815ce60 2023-12-20T04:28:15.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0f666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d0814db20 con 0x7f7d080d1300 2023-12-20T04:28:15.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0d402640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d1300 0x7f7d081f55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0d402640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d1300 0x7f7d081f55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:55200/0 (socket says 172.21.15.39:55200) 2023-12-20T04:28:15.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0d402640 1 -- 172.21.15.39:0/739117132 learned_addr learned my addr 172.21.15.39:0/739117132 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:15.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0dc03640 1 --2- 172.21.15.39:0/739117132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0815ce60 0x7f7d081e8050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.151 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0cc01640 1 --2- 172.21.15.39:0/739117132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d080d3130 0x7f7d081e7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0d402640 1 -- 172.21.15.39:0/739117132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0815ce60 msgr2=0x7f7d081e8050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0d402640 1 --2- 172.21.15.39:0/739117132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0815ce60 0x7f7d081e8050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0d402640 1 -- 172.21.15.39:0/739117132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d080d3130 msgr2=0x7f7d081e7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0d402640 1 --2- 172.21.15.39:0/739117132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d080d3130 0x7f7d081e7b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0d402640 1 -- 172.21.15.39:0/739117132 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d040094d0 con 0x7f7d080d1300 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0dc03640 1 --2- 172.21.15.39:0/739117132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0815ce60 0x7f7d081e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0cc01640 1 --2- 172.21.15.39:0/739117132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d080d3130 0x7f7d081e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0d402640 1 --2- 172.21.15.39:0/739117132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d1300 0x7f7d081f55f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e930 tx=0x7f7d0400edf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7cfdffb640 1 -- 172.21.15.39:0/739117132 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d0400ca00 con 0x7f7d080d1300 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7cfdffb640 1 -- 172.21.15.39:0/739117132 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d0400cb60 con 0x7f7d080d1300 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0f666640 1 -- 172.21.15.39:0/739117132 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf8009420 con 0x7f7d080d1300 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7cfdffb640 1 -- 172.21.15.39:0/739117132 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d04013470 con 0x7f7d080d1300 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7cfdffb640 1 -- 172.21.15.39:0/739117132 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f7d040138d0 con 0x7f7d080d1300 2023-12-20T04:28:15.152 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0f666640 1 -- 172.21.15.39:0/739117132 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d0814e0e0 con 0x7f7d080d1300 2023-12-20T04:28:15.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7cfdffb640 1 -- 172.21.15.39:0/739117132 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(8..8 src has 1..8) v4 ==== 1415+0+0 (secure 0 0 0) 0x7f7d04013d60 con 0x7f7d080d1300 2023-12-20T04:28:15.153 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.147+0000 7f7d0f666640 1 -- 172.21.15.39:0/739117132 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d0804e000 con 0x7f7d080d1300 2023-12-20T04:28:15.154 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.151+0000 7f7cfdffb640 1 -- 172.21.15.39:0/739117132 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f7d04021060 con 0x7f7d080d1300 2023-12-20T04:28:15.215 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:15.213+0000 7f1cbfcdc180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:28:15.222 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.219+0000 7f7d0f666640 1 -- 172.21.15.39:0/739117132 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd new", "uuid": "e93092a0-e6d4-4ba7-97bd-50eab2ed529e", "id": 6} v 0) v1 -- 0x7f7d080b2c70 con 0x7f7d080d1300 2023-12-20T04:28:15.227 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.223+0000 7f7cfdffb640 1 -- 172.21.15.39:0/739117132 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "e93092a0-e6d4-4ba7-97bd-50eab2ed529e", "id": 6}]=0 v9) v1 ==== 112+0+2 (secure 0 0 0) 0x7f7d0402a020 con 0x7f7d080d1300 2023-12-20T04:28:15.228 INFO:teuthology.orchestra.run.smithi039.stdout:6 2023-12-20T04:28:15.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.223+0000 7f7d0f666640 1 -- 172.21.15.39:0/739117132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d1300 msgr2=0x7f7d081f55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.223+0000 7f7d0f666640 1 --2- 172.21.15.39:0/739117132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d1300 0x7f7d081f55f0 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f7d0400e930 tx=0x7f7d0400edf0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.223+0000 7f7d0f666640 1 -- 172.21.15.39:0/739117132 shutdown_connections 2023-12-20T04:28:15.229 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.223+0000 7f7d0f666640 1 --2- 172.21.15.39:0/739117132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d0815ce60 0x7f7d081e8050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.223+0000 7f7d0f666640 1 --2- 172.21.15.39:0/739117132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d080d3130 0x7f7d081e7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.223+0000 7f7d0f666640 1 --2- 172.21.15.39:0/739117132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d080d1300 0x7f7d081f55f0 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.223+0000 7f7d0f666640 1 -- 172.21.15.39:0/739117132 >> 172.21.15.39:0/739117132 conn(0x7f7d0806b5e0 msgr2=0x7f7d080b5d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:15.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.223+0000 7f7d0f666640 1 -- 172.21.15.39:0/739117132 shutdown_connections 2023-12-20T04:28:15.230 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.223+0000 7f7d0f666640 1 -- 172.21.15.39:0/739117132 wait complete. 2023-12-20T04:28:15.249 DEBUG:teuthology.orchestra.run.smithi039:> sudo ceph --cluster ceph osd new 215a2838-2ba5-49d3-a86c-5ed5dd346fd5 7 2023-12-20T04:28:15.289 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:15.285+0000 7f1cbfcdc180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:28:15.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.387+0000 7fd921b19640 1 Processor -- start 2023-12-20T04:28:15.391 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.387+0000 7fd921b19640 1 -- start start 2023-12-20T04:28:15.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.387+0000 7fd921b19640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c15cc30 0x7fd91c0b8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.387+0000 7fd921b19640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd91c0d3060 0x7fd91c0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.387+0000 7fd921b19640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd91c0d2090 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.387+0000 7fd921b19640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd91c15d010 con 0x7fd91c0d3060 2023-12-20T04:28:15.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.387+0000 7fd921b19640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0d1480 con 0x7fd91c15cc30 2023-12-20T04:28:15.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.387+0000 7fd920b17640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c15cc30 0x7fd91c0b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.387+0000 7fd921318640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd91c0d3060 0x7fd91c0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.387+0000 7fd920b17640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c15cc30 0x7fd91c0b8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:55208/0 (socket says 172.21.15.39:55208) 2023-12-20T04:28:15.393 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.387+0000 7fd920b17640 1 -- 172.21.15.39:0/4106831527 learned_addr learned my addr 172.21.15.39:0/4106831527 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:15.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.391+0000 7fd920b17640 1 -- 172.21.15.39:0/4106831527 >> v2:172.21.15.39:3300/0 conn(0x7fd91c0d3060 msgr2=0x7fd91c0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:15.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.391+0000 7fd920b17640 1 --2- 172.21.15.39:0/4106831527 >> v2:172.21.15.39:3300/0 conn(0x7fd91c0d3060 0x7fd91c0b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.391+0000 7fd920b17640 1 -- 172.21.15.39:0/4106831527 >> v1:172.21.15.32:6789/0 conn(0x7fd91c0d0ca0 legacy=0x7fd91c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.391+0000 7fd920b17640 1 -- 172.21.15.39:0/4106831527 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd91c0d1760 con 0x7fd91c15cc30 2023-12-20T04:28:15.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.391+0000 7fd921318640 1 --2- 172.21.15.39:0/4106831527 >> v2:172.21.15.39:3300/0 conn(0x7fd91c0d3060 0x7fd91c0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:15.394 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.391+0000 7fd91b7fe640 1 -- 172.21.15.39:0/4106831527 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3862960092 0 0) 0x7fd91c0d2090 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.396 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.391+0000 7fd920b17640 1 --2- 172.21.15.39:0/4106831527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c15cc30 0x7fd91c0b8520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd910009870 tx=0x7fd91001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1387d883290ff48d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:15.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.391+0000 7fd91b7fe640 1 -- 172.21.15.39:0/4106831527 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100aa670 con 0x7fd91c15cc30 2023-12-20T04:28:15.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.391+0000 7fd91b7fe640 1 -- 172.21.15.39:0/4106831527 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9100aa7d0 con 0x7fd91c15cc30 2023-12-20T04:28:15.397 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.391+0000 7fd91b7fe640 1 -- 172.21.15.39:0/4106831527 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100aab60 con 0x7fd91c15cc30 2023-12-20T04:28:15.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 -- 172.21.15.39:0/4106831527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c15cc30 msgr2=0x7fd91c0b8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 --2- 172.21.15.39:0/4106831527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c15cc30 0x7fd91c0b8520 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fd910009870 tx=0x7fd91001c510 comp rx=0 tx=0).stop 2023-12-20T04:28:15.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 -- 172.21.15.39:0/4106831527 shutdown_connections 2023-12-20T04:28:15.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 --2- 172.21.15.39:0/4106831527 >> v2:172.21.15.39:3300/0 conn(0x7fd91c0d3060 0x7fd91c0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 --2- 172.21.15.39:0/4106831527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c15cc30 0x7fd91c0b8520 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 -- 172.21.15.39:0/4106831527 >> 172.21.15.39:0/4106831527 conn(0x7fd91c065ce0 msgr2=0x7fd91c0660f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:15.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 -- 172.21.15.39:0/4106831527 shutdown_connections 2023-12-20T04:28:15.398 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 -- 172.21.15.39:0/4106831527 wait complete. 2023-12-20T04:28:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 Processor -- start 2023-12-20T04:28:15.399 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 -- start start 2023-12-20T04:28:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c0d0ca0 0x7fd91c156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c0d3060 0x7fd91c156320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd91c15cc30 0x7fd91c15a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c0b31c0 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c156860 con 0x7fd91c15cc30 2023-12-20T04:28:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921b19640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd91c157140 con 0x7fd91c0d3060 2023-12-20T04:28:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd91bfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c0d3060 0x7fd91c156320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921318640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd91c15cc30 0x7fd91c15a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd920b17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c0d0ca0 0x7fd91c156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.401 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921318640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd91c15cc30 0x7fd91c15a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.39:36752/0 (socket says 172.21.15.39:36752) 2023-12-20T04:28:15.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd921318640 1 -- 172.21.15.39:0/3920178717 learned_addr learned my addr 172.21.15.39:0/3920178717 (peer_addr_for_me v2:172.21.15.39:0/0) 2023-12-20T04:28:15.402 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.395+0000 7fd91bfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c0d3060 0x7fd91c156320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.39:55212/0 (socket says 172.21.15.39:55212) 2023-12-20T04:28:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd920b17640 1 -- 172.21.15.39:0/3920178717 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c0d3060 msgr2=0x7fd91c156320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd920b17640 1 --2- 172.21.15.39:0/3920178717 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c0d3060 0x7fd91c156320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd920b17640 1 -- 172.21.15.39:0/3920178717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd91c15cc30 msgr2=0x7fd91c15a9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd920b17640 1 --2- 172.21.15.39:0/3920178717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd91c15cc30 0x7fd91c15a9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd920b17640 1 -- 172.21.15.39:0/3920178717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd914009a70 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd921318640 1 --2- 172.21.15.39:0/3920178717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd91c15cc30 0x7fd91c15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd920b17640 1 --2- 172.21.15.39:0/3920178717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c0d0ca0 0x7fd91c156a40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd9100ab000 tx=0x7fd91001ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd9197fa640 1 -- 172.21.15.39:0/3920178717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9100aad50 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd9197fa640 1 -- 172.21.15.39:0/3920178717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9100b7070 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.403 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd921b19640 1 -- 172.21.15.39:0/3920178717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9100094d0 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd9197fa640 1 -- 172.21.15.39:0/3920178717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd910004260 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd921b19640 1 -- 172.21.15.39:0/3920178717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd91c157700 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd9197fa640 1 -- 172.21.15.39:0/3920178717 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fd910003920 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd9197fa640 1 -- 172.21.15.39:0/3920178717 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(9..9 src has 1..9) v4 ==== 1530+0+0 (secure 0 0 0) 0x7fd9100bdb50 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.404 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.399+0000 7fd921b19640 1 -- 172.21.15.39:0/3920178717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd91c0b79c0 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.413 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.407+0000 7fd9197fa640 1 -- 172.21.15.39:0/3920178717 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7fd9100c2020 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.430 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:15.427+0000 7fb3fdf4f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:28:15.481 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.479+0000 7fd921b19640 1 -- 172.21.15.39:0/3920178717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd new", "uuid": "215a2838-2ba5-49d3-a86c-5ed5dd346fd5", "id": 7} v 0) v1 -- 0x7fd91c074100 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.485 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.483+0000 7fd9197fa640 1 -- 172.21.15.39:0/3920178717 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd new", "uuid": "215a2838-2ba5-49d3-a86c-5ed5dd346fd5", "id": 7}]=0 v10) v1 ==== 112+0+2 (secure 0 0 0) 0x7fd9100b9070 con 0x7fd91c0d0ca0 2023-12-20T04:28:15.485 INFO:teuthology.orchestra.run.smithi039.stdout:7 2023-12-20T04:28:15.486 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.483+0000 7fd921b19640 1 -- 172.21.15.39:0/3920178717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c0d0ca0 msgr2=0x7fd91c156a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.483+0000 7fd921b19640 1 --2- 172.21.15.39:0/3920178717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c0d0ca0 0x7fd91c156a40 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fd9100ab000 tx=0x7fd91001ad50 comp rx=0 tx=0).stop 2023-12-20T04:28:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.483+0000 7fd921b19640 1 -- 172.21.15.39:0/3920178717 shutdown_connections 2023-12-20T04:28:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.483+0000 7fd921b19640 1 --2- 172.21.15.39:0/3920178717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd91c15cc30 0x7fd91c15a9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.483+0000 7fd921b19640 1 --2- 172.21.15.39:0/3920178717 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd91c0d3060 0x7fd91c156320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.483+0000 7fd921b19640 1 --2- 172.21.15.39:0/3920178717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd91c0d0ca0 0x7fd91c156a40 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.483+0000 7fd921b19640 1 -- 172.21.15.39:0/3920178717 >> 172.21.15.39:0/3920178717 conn(0x7fd91c065ce0 msgr2=0x7fd91c15dde0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.483+0000 7fd921b19640 1 -- 172.21.15.39:0/3920178717 shutdown_connections 2023-12-20T04:28:15.487 INFO:teuthology.orchestra.run.smithi039.stderr:2023-12-20T04:28:15.483+0000 7fd921b19640 1 -- 172.21.15.39:0/3920178717 wait complete. 2023-12-20T04:28:15.506 INFO:tasks.ceph.osd.0:Restarting daemon 2023-12-20T04:28:15.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2023-12-20T04:28:15.510 INFO:tasks.ceph.osd.0:Started 2023-12-20T04:28:15.510 INFO:tasks.ceph.osd.1:Restarting daemon 2023-12-20T04:28:15.510 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2023-12-20T04:28:15.511 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:15.507+0000 7fb3fdf4f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:28:15.513 INFO:tasks.ceph.osd.1:Started 2023-12-20T04:28:15.514 INFO:tasks.ceph.osd.2:Restarting daemon 2023-12-20T04:28:15.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2023-12-20T04:28:15.517 INFO:tasks.ceph.osd.2:Started 2023-12-20T04:28:15.517 INFO:tasks.ceph.osd.3:Restarting daemon 2023-12-20T04:28:15.517 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2023-12-20T04:28:15.520 INFO:tasks.ceph.osd.3:Started 2023-12-20T04:28:15.521 INFO:tasks.ceph.osd.4:Restarting daemon 2023-12-20T04:28:15.521 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 4 2023-12-20T04:28:15.524 INFO:tasks.ceph.osd.4:Started 2023-12-20T04:28:15.524 INFO:tasks.ceph.osd.5:Restarting daemon 2023-12-20T04:28:15.524 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 5 2023-12-20T04:28:15.527 INFO:tasks.ceph.osd.5:Started 2023-12-20T04:28:15.527 INFO:tasks.ceph.osd.6:Restarting daemon 2023-12-20T04:28:15.528 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 6 2023-12-20T04:28:15.531 INFO:tasks.ceph.osd.6:Started 2023-12-20T04:28:15.531 INFO:tasks.ceph.osd.7:Restarting daemon 2023-12-20T04:28:15.531 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 7 2023-12-20T04:28:15.537 INFO:tasks.ceph.osd.7:Started 2023-12-20T04:28:15.538 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:15.581 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:15.575+0000 7fb3fdf4f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:28:15.701 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:15.695+0000 7fb3fdf4f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:28:15.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.741+0000 7fa64cce5640 1 Processor -- start 2023-12-20T04:28:15.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.741+0000 7fa64cce5640 1 -- start start 2023-12-20T04:28:15.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa64cce5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64815cdb0 0x7fa6480d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa64cce5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa64815d190 con 0x7fa6480cfc80 2023-12-20T04:28:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa64cce5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6480d16a0 con 0x7fa6480d2740 2023-12-20T04:28:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa64cce5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480d1980 con 0x7fa64815cdb0 2023-12-20T04:28:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa646d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa6480cfc80 0x7fa6480b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47826/0 (socket says 172.21.15.32:47826) 2023-12-20T04:28:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa646d9d640 1 -- 172.21.15.32:0/1294801818 learned_addr learned my addr 172.21.15.32:0/1294801818 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:15.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa64659c640 1 --2- 172.21.15.32:0/1294801818 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64815cdb0 0x7fa6480d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa645d9b640 1 -- 172.21.15.32:0/1294801818 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1622469825 0 0) 0x7fa6480d16a0 con 0x7fa6480d2740 2023-12-20T04:28:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa645d9b640 1 -- 172.21.15.32:0/1294801818 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa634003530 con 0x7fa6480d2740 2023-12-20T04:28:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa645d9b640 1 -- 172.21.15.32:0/1294801818 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817141716 0 0) 0x7fa64815d190 con 0x7fa6480cfc80 2023-12-20T04:28:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa64659c640 1 -- 172.21.15.32:0/1294801818 >> v1:172.21.15.39:6789/0 conn(0x7fa6480d2740 legacy=0x7fa6480cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa64659c640 1 -- 172.21.15.32:0/1294801818 >> v1:172.21.15.32:6789/0 conn(0x7fa6480cfc80 legacy=0x7fa6480b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.745+0000 7fa64659c640 1 -- 172.21.15.32:0/1294801818 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6480d1c60 con 0x7fa64815cdb0 2023-12-20T04:28:15.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa64659c640 1 --2- 172.21.15.32:0/1294801818 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64815cdb0 0x7fa6480d0be0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa6380097f0 tx=0x7fa63801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d111a461f63b1145 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:15.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa645d9b640 1 -- 172.21.15.32:0/1294801818 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380a89b0 con 0x7fa64815cdb0 2023-12-20T04:28:15.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa645d9b640 1 -- 172.21.15.32:0/1294801818 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6380a8b10 con 0x7fa64815cdb0 2023-12-20T04:28:15.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa645d9b640 1 -- 172.21.15.32:0/1294801818 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380a8ea0 con 0x7fa64815cdb0 2023-12-20T04:28:15.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa64cce5640 1 -- 172.21.15.32:0/1294801818 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64815cdb0 msgr2=0x7fa6480d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa64cce5640 1 --2- 172.21.15.32:0/1294801818 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64815cdb0 0x7fa6480d0be0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa6380097f0 tx=0x7fa63801ecb0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa64cce5640 1 -- 172.21.15.32:0/1294801818 shutdown_connections 2023-12-20T04:28:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa64cce5640 1 --2- 172.21.15.32:0/1294801818 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64815cdb0 0x7fa6480d0be0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa64cce5640 1 -- 172.21.15.32:0/1294801818 >> 172.21.15.32:0/1294801818 conn(0x7fa648065ee0 msgr2=0x7fa64806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa64cce5640 1 -- 172.21.15.32:0/1294801818 shutdown_connections 2023-12-20T04:28:15.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa64cce5640 1 -- 172.21.15.32:0/1294801818 wait complete. 2023-12-20T04:28:15.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa64cce5640 1 Processor -- start 2023-12-20T04:28:15.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.749+0000 7fa64cce5640 1 -- start start 2023-12-20T04:28:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64cce5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6480cfc80 0x7fa64814a970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64cce5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6480d2740 0x7fa64814f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64cce5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64815cdb0 0x7fa64814c9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64cce5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480cac80 con 0x7fa64815cdb0 2023-12-20T04:28:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64cce5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480caed0 con 0x7fa6480d2740 2023-12-20T04:28:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64cce5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6480cb1b0 con 0x7fa6480cfc80 2023-12-20T04:28:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64659c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6480cfc80 0x7fa64814a970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64659c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6480cfc80 0x7fa64814a970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:39790/0 (socket says 172.21.15.32:39790) 2023-12-20T04:28:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64659c640 1 -- 172.21.15.32:0/1292331624 learned_addr learned my addr 172.21.15.32:0/1292331624 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa646d9d640 1 --2- 172.21.15.32:0/1292331624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64815cdb0 0x7fa64814c9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa63ffff640 1 --2- 172.21.15.32:0/1292331624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6480d2740 0x7fa64814f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64659c640 1 -- 172.21.15.32:0/1292331624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6480d2740 msgr2=0x7fa64814f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64659c640 1 --2- 172.21.15.32:0/1292331624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6480d2740 0x7fa64814f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64659c640 1 -- 172.21.15.32:0/1292331624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64815cdb0 msgr2=0x7fa64814c9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64659c640 1 --2- 172.21.15.32:0/1292331624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64815cdb0 0x7fa64814c9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64659c640 1 -- 172.21.15.32:0/1292331624 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa638009420 con 0x7fa6480cfc80 2023-12-20T04:28:15.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64659c640 1 --2- 172.21.15.32:0/1292331624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6480cfc80 0x7fa64814a970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6380097f0 tx=0x7fa63801ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa63effd640 1 -- 172.21.15.32:0/1292331624 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380b0df0 con 0x7fa6480cfc80 2023-12-20T04:28:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa63effd640 1 -- 172.21.15.32:0/1292331624 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6380b9400 con 0x7fa6480cfc80 2023-12-20T04:28:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64cce5640 1 -- 172.21.15.32:0/1292331624 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6480cb490 con 0x7fa6480cfc80 2023-12-20T04:28:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa63effd640 1 -- 172.21.15.32:0/1292331624 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6380b9770 con 0x7fa6480cfc80 2023-12-20T04:28:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64cce5640 1 -- 172.21.15.32:0/1292331624 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa64814aeb0 con 0x7fa6480cfc80 2023-12-20T04:28:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa63effd640 1 -- 172.21.15.32:0/1292331624 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7fa6380b7040 con 0x7fa6480cfc80 2023-12-20T04:28:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa63effd640 1 -- 172.21.15.32:0/1292331624 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7fa6380c7d50 con 0x7fa6480cfc80 2023-12-20T04:28:15.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.753+0000 7fa64cce5640 1 -- 172.21.15.32:0/1292331624 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6480cfaf0 con 0x7fa6480cfc80 2023-12-20T04:28:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.757+0000 7fa63effd640 1 -- 172.21.15.32:0/1292331624 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7fa6380b5070 con 0x7fa6480cfc80 2023-12-20T04:28:15.764 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:15.759+0000 7fb3fdf4f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:28:15.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.833+0000 7fa64cce5640 1 -- 172.21.15.32:0/1292331624 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa64804e000 con 0x7fa6480cfc80 2023-12-20T04:28:15.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.833+0000 7fa63effd640 1 -- 172.21.15.32:0/1292331624 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7fa6380c7680 con 0x7fa6480cfc80 2023-12-20T04:28:15.835 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:15.835 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":10,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:15.484604+0000","last_up_change":"0.000000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:15.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.833+0000 7fa64cce5640 1 -- 172.21.15.32:0/1292331624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6480cfc80 msgr2=0x7fa64814a970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:15.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.833+0000 7fa64cce5640 1 --2- 172.21.15.32:0/1292331624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6480cfc80 0x7fa64814a970 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fa6380097f0 tx=0x7fa63801ba50 comp rx=0 tx=0).stop 2023-12-20T04:28:15.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.833+0000 7fa64cce5640 1 -- 172.21.15.32:0/1292331624 shutdown_connections 2023-12-20T04:28:15.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.833+0000 7fa64cce5640 1 --2- 172.21.15.32:0/1292331624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64815cdb0 0x7fa64814c9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.833+0000 7fa64cce5640 1 --2- 172.21.15.32:0/1292331624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6480d2740 0x7fa64814f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.833+0000 7fa64cce5640 1 --2- 172.21.15.32:0/1292331624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6480cfc80 0x7fa64814a970 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.833+0000 7fa64cce5640 1 -- 172.21.15.32:0/1292331624 >> 172.21.15.32:0/1292331624 conn(0x7fa648065ee0 msgr2=0x7fa64815bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.833+0000 7fa64cce5640 1 -- 172.21.15.32:0/1292331624 shutdown_connections 2023-12-20T04:28:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:15.833+0000 7fa64cce5640 1 -- 172.21.15.32:0/1292331624 wait complete. 2023-12-20T04:28:15.855 INFO:tasks.ceph.ceph_manager.ceph:[] 2023-12-20T04:28:15.855 INFO:tasks.ceph:Waiting for OSDs to come up 2023-12-20T04:28:15.858 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:15.855+0000 7fb3fdf4f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:28:15.921 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:15.919+0000 7fb3fdf4f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:28:15.969 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:28:15.969 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:28:15.969 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:28:16.019 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:16.017+0000 7f1cbfcdc180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:28:16.087 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:16.083+0000 7fb3fdf4f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:28:16.159 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:16.155+0000 7fb3fdf4f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:28:16.161 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:16.157+0000 7f1cbfcdc180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:28:16.175 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-12-20T04:28:16.173+0000 7f41ad50b640 -1 Falling back to public interface 2023-12-20T04:28:16.178 INFO:tasks.ceph.osd.4.smithi039.stderr:2023-12-20T04:28:16.175+0000 7fe4bf5d3640 -1 Falling back to public interface 2023-12-20T04:28:16.191 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-12-20T04:28:16.189+0000 7fa7a9cd2640 -1 Falling back to public interface 2023-12-20T04:28:16.202 INFO:tasks.ceph.osd.7.smithi039.stderr:2023-12-20T04:28:16.199+0000 7f4fc6b63640 -1 Falling back to public interface 2023-12-20T04:28:16.210 INFO:tasks.ceph.osd.6.smithi039.stderr:2023-12-20T04:28:16.207+0000 7f33150a2640 -1 Falling back to public interface 2023-12-20T04:28:16.219 INFO:tasks.ceph.osd.5.smithi039.stderr:2023-12-20T04:28:16.215+0000 7f34e9380640 -1 Falling back to public interface 2023-12-20T04:28:16.231 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-12-20T04:28:16.229+0000 7f5929c4a640 -1 Falling back to public interface 2023-12-20T04:28:16.231 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-12-20T04:28:16.229+0000 7f01d5edb640 -1 Falling back to public interface 2023-12-20T04:28:16.363 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:16.361+0000 7f1cbfcdc180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:28:16.439 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:16.437+0000 7f1cbfcdc180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:28:16.585 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:16.581+0000 7f1cbfcdc180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:28:16.660 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:16.793 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.789+0000 7f847dd39640 1 Processor -- start 2023-12-20T04:28:16.793 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.789+0000 7f847dd39640 1 -- start start 2023-12-20T04:28:16.794 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f847dd39640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f84780b7ab0 0x7f84780d17f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:16.794 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f847dd39640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84780b60b0 con 0x7f84780cfd30 2023-12-20T04:28:16.794 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f847dd39640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84780d23b0 con 0x7f847815d0d0 2023-12-20T04:28:16.794 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f847dd39640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84780b62e0 con 0x7f84780b7ab0 2023-12-20T04:28:16.794 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f8476ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f84780cfd30 0x7f84780d10e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47836/0 (socket says 172.21.15.32:47836) 2023-12-20T04:28:16.794 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f8476ffd640 1 -- 172.21.15.32:0/1952297513 learned_addr learned my addr 172.21.15.32:0/1952297513 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:16.794 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f8477fff640 1 --2- 172.21.15.32:0/1952297513 >> v2:172.21.15.39:3300/0 conn(0x7f84780b7ab0 0x7f84780d17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:16.795 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f84767fc640 1 -- 172.21.15.32:0/1952297513 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398985270 0 0) 0x7f84780b60b0 con 0x7f84780cfd30 2023-12-20T04:28:16.795 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f8477fff640 1 -- 172.21.15.32:0/1952297513 >> v1:172.21.15.39:6789/0 conn(0x7f847815d0d0 legacy=0x7f84780cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:16.795 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f8477fff640 1 -- 172.21.15.32:0/1952297513 >> v1:172.21.15.32:6789/0 conn(0x7f84780cfd30 legacy=0x7f84780d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:16.795 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f8477fff640 1 -- 172.21.15.32:0/1952297513 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84780d1df0 con 0x7f84780b7ab0 2023-12-20T04:28:16.797 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.793+0000 7f8477fff640 1 --2- 172.21.15.32:0/1952297513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b7ab0 0x7f84780d17f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8470009d90 tx=0x7f847001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b3c437f15f5cce3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:16.798 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f84767fc640 1 -- 172.21.15.32:0/1952297513 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700a59e0 con 0x7f84780b7ab0 2023-12-20T04:28:16.798 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f84767fc640 1 -- 172.21.15.32:0/1952297513 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84700a5b40 con 0x7f84780b7ab0 2023-12-20T04:28:16.798 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f84767fc640 1 -- 172.21.15.32:0/1952297513 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700ad440 con 0x7f84780b7ab0 2023-12-20T04:28:16.799 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 -- 172.21.15.32:0/1952297513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b7ab0 msgr2=0x7f84780d17f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:16.799 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 --2- 172.21.15.32:0/1952297513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b7ab0 0x7f84780d17f0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f8470009d90 tx=0x7f847001c920 comp rx=0 tx=0).stop 2023-12-20T04:28:16.799 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 -- 172.21.15.32:0/1952297513 shutdown_connections 2023-12-20T04:28:16.799 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 --2- 172.21.15.32:0/1952297513 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b7ab0 0x7f84780d17f0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:16.799 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 -- 172.21.15.32:0/1952297513 >> 172.21.15.32:0/1952297513 conn(0x7f847806b130 msgr2=0x7f847806b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:16.799 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 -- 172.21.15.32:0/1952297513 shutdown_connections 2023-12-20T04:28:16.800 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 -- 172.21.15.32:0/1952297513 wait complete. 2023-12-20T04:28:16.800 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 Processor -- start 2023-12-20T04:28:16.801 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 -- start start 2023-12-20T04:28:16.803 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b7ab0 0x7f847814a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:16.803 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cfd30 0x7f84780c79b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:16.803 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815d0d0 0x7f84780c7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:16.803 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84780c9ea0 con 0x7f84780cfd30 2023-12-20T04:28:16.803 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84780b2310 con 0x7f84780b7ab0 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f847dd39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84781577f0 con 0x7f847815d0d0 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.797+0000 7f8476ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cfd30 0x7f84780c79b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f8476ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cfd30 0x7f84780c79b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59562/0 (socket says 172.21.15.32:59562) 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f8476ffd640 1 -- 172.21.15.32:0/2188862502 learned_addr learned my addr 172.21.15.32:0/2188862502 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f84777fe640 1 --2- 172.21.15.32:0/2188862502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b7ab0 0x7f847814a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f8477fff640 1 --2- 172.21.15.32:0/2188862502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815d0d0 0x7f84780c7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f8476ffd640 1 -- 172.21.15.32:0/2188862502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815d0d0 msgr2=0x7f84780c7ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f8476ffd640 1 --2- 172.21.15.32:0/2188862502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815d0d0 0x7f84780c7ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f8476ffd640 1 -- 172.21.15.32:0/2188862502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b7ab0 msgr2=0x7f847814a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f8476ffd640 1 --2- 172.21.15.32:0/2188862502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b7ab0 0x7f847814a9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f8476ffd640 1 -- 172.21.15.32:0/2188862502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470009a70 con 0x7f84780cfd30 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f84777fe640 1 --2- 172.21.15.32:0/2188862502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b7ab0 0x7f847814a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f8476ffd640 1 --2- 172.21.15.32:0/2188862502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cfd30 0x7f84780c79b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f846c00c4d0 tx=0x7f846c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:16.804 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f845ffff640 1 -- 172.21.15.32:0/2188862502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c0074e0 con 0x7f84780cfd30 2023-12-20T04:28:16.805 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f845ffff640 1 -- 172.21.15.32:0/2188862502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f846c007640 con 0x7f84780cfd30 2023-12-20T04:28:16.805 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f847dd39640 1 -- 172.21.15.32:0/2188862502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847815c950 con 0x7f84780cfd30 2023-12-20T04:28:16.805 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f845ffff640 1 -- 172.21.15.32:0/2188862502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f846c004a30 con 0x7f84780cfd30 2023-12-20T04:28:16.805 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f847dd39640 1 -- 172.21.15.32:0/2188862502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847815fc00 con 0x7f84780cfd30 2023-12-20T04:28:16.805 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f845ffff640 1 -- 172.21.15.32:0/2188862502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 1) v1 ==== 660+0+0 (secure 0 0 0) 0x7f846c01d440 con 0x7f84780cfd30 2023-12-20T04:28:16.805 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f845ffff640 1 -- 172.21.15.32:0/2188862502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(10..10 src has 1..10) v4 ==== 1645+0+0 (secure 0 0 0) 0x7f846c012ad0 con 0x7f84780cfd30 2023-12-20T04:28:16.805 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.801+0000 7f847dd39640 1 -- 172.21.15.32:0/2188862502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84780671d0 con 0x7f84780cfd30 2023-12-20T04:28:16.806 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.805+0000 7f845ffff640 1 -- 172.21.15.32:0/2188862502 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+72491 (secure 0 0 0) 0x7f846c018070 con 0x7f84780cfd30 2023-12-20T04:28:16.829 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:28:16.830 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:28:16.830 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:28:16.864 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.861+0000 7f847dd39640 1 -- 172.21.15.32:0/2188862502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8478074170 con 0x7f84780cfd30 2023-12-20T04:28:16.864 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.861+0000 7f845ffff640 1 -- 172.21.15.32:0/2188862502 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v10) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f846c012400 con 0x7f84780cfd30 2023-12-20T04:28:16.865 INFO:teuthology.misc.health.smithi032.stdout: 2023-12-20T04:28:16.865 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":10,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:15.484604+0000","last_up_change":"0.000000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:16.865 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.861+0000 7f847dd39640 1 -- 172.21.15.32:0/2188862502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cfd30 msgr2=0x7f84780c79b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:16.866 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.861+0000 7f847dd39640 1 --2- 172.21.15.32:0/2188862502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cfd30 0x7f84780c79b0 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f846c00c4d0 tx=0x7f846c00c990 comp rx=0 tx=0).stop 2023-12-20T04:28:16.866 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.861+0000 7f847dd39640 1 -- 172.21.15.32:0/2188862502 shutdown_connections 2023-12-20T04:28:16.866 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.861+0000 7f847dd39640 1 --2- 172.21.15.32:0/2188862502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815d0d0 0x7f84780c7ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:16.866 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.861+0000 7f847dd39640 1 --2- 172.21.15.32:0/2188862502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cfd30 0x7f84780c79b0 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:16.866 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.861+0000 7f847dd39640 1 --2- 172.21.15.32:0/2188862502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b7ab0 0x7f847814a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:16.866 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.861+0000 7f847dd39640 1 -- 172.21.15.32:0/2188862502 >> 172.21.15.32:0/2188862502 conn(0x7f847806b130 msgr2=0x7f84780cdcf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:16.866 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.865+0000 7f847dd39640 1 -- 172.21.15.32:0/2188862502 shutdown_connections 2023-12-20T04:28:16.866 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:16.865+0000 7f847dd39640 1 -- 172.21.15.32:0/2188862502 wait complete. 2023-12-20T04:28:16.880 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:16.875+0000 7fb3fdf4f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:28:16.884 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-20T04:28:17.025 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:17.023+0000 7fb3fdf4f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:28:17.205 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:17.203+0000 7fb3fdf4f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:28:17.282 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:17.279+0000 7fb3fdf4f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:28:17.304 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:17.301+0000 7f1cbfcdc180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:28:17.395 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:17.393+0000 7f1cbfcdc180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:28:17.427 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:17.423+0000 7fb3fdf4f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:28:17.597 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:17.593+0000 7f1cbfcdc180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:28:17.686 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:17.685+0000 7f1cbfcdc180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:28:18.035 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:18.033+0000 7f1cbfcdc180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:28:18.108 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:18.105+0000 7f1cbfcdc180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:28:18.149 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:18.147+0000 7fb3fdf4f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:28:18.242 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:18.243+0000 7fb3fdf4f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:28:18.444 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:18.443+0000 7fb3fdf4f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:28:18.534 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:18.535+0000 7fb3fdf4f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:28:18.885 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:18.883+0000 7fb3fdf4f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:28:18.956 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:28:18.955+0000 7fb3fdf4f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:28:19.131 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:19.125+0000 7f1c6738d640 -1 mgr.server handle_report got status from non-daemon mon.b 2023-12-20T04:28:19.131 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:28:19.125+0000 7f1c6738d640 -1 mgr.server handle_report got status from non-daemon mon.c 2023-12-20T04:28:19.959 INFO:tasks.ceph.osd.7.smithi039.stderr:2023-12-20T04:28:19.959+0000 7f4fc6b63640 -1 osd.7 0 log_to_monitors true 2023-12-20T04:28:20.066 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-12-20T04:28:20.065+0000 7fa7a9cd2640 -1 osd.0 0 log_to_monitors true 2023-12-20T04:28:20.143 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-12-20T04:28:20.141+0000 7f41ad50b640 -1 osd.1 0 log_to_monitors true 2023-12-20T04:28:20.166 INFO:tasks.ceph.osd.4.smithi039.stderr:2023-12-20T04:28:20.167+0000 7fe4bf5d3640 -1 osd.4 0 log_to_monitors true 2023-12-20T04:28:20.169 INFO:tasks.ceph.osd.6.smithi039.stderr:2023-12-20T04:28:20.167+0000 7f33150a2640 -1 osd.6 0 log_to_monitors true 2023-12-20T04:28:20.210 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-12-20T04:28:20.205+0000 7f01d5edb640 -1 osd.2 0 log_to_monitors true 2023-12-20T04:28:20.231 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-12-20T04:28:20.229+0000 7f5929c4a640 -1 osd.3 0 log_to_monitors true 2023-12-20T04:28:20.257 INFO:tasks.ceph.osd.5.smithi039.stderr:2023-12-20T04:28:20.255+0000 7f34e9380640 -1 osd.5 0 log_to_monitors true 2023-12-20T04:28:23.479 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-12-20T04:28:23.477+0000 7f01d165d640 -1 osd.2 0 waiting for initial osdmap 2023-12-20T04:28:23.496 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-12-20T04:28:23.493+0000 7fa7a6c8f640 -1 osd.0 0 waiting for initial osdmap 2023-12-20T04:28:23.515 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-12-20T04:28:23.513+0000 7f41a8c8f640 -1 osd.1 0 waiting for initial osdmap 2023-12-20T04:28:23.531 INFO:tasks.ceph.osd.2.smithi032.stderr:2023-12-20T04:28:23.529+0000 7f01c9ca2640 -1 osd.2 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-20T04:28:23.531 INFO:tasks.ceph.osd.0.smithi032.stderr:2023-12-20T04:28:23.529+0000 7fa79da99640 -1 osd.0 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-20T04:28:23.546 INFO:tasks.ceph.osd.1.smithi032.stderr:2023-12-20T04:28:23.545+0000 7f41a12d4640 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-20T04:28:23.554 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-12-20T04:28:23.553+0000 7f59253ce640 -1 osd.3 0 waiting for initial osdmap 2023-12-20T04:28:23.611 INFO:tasks.ceph.osd.3.smithi032.stderr:2023-12-20T04:28:23.609+0000 7f591da13640 -1 osd.3 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-20T04:28:23.691 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:23.822 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.817+0000 7f9dac0b6640 1 Processor -- start 2023-12-20T04:28:23.822 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9dac0b6640 1 -- start start 2023-12-20T04:28:23.823 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9dac0b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da415d0d0 0x7f9da40cf620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:23.823 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9dac0b6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da40b60b0 con 0x7f9da40d17f0 2023-12-20T04:28:23.823 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9dac0b6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9da40cffa0 con 0x7f9da40cfb60 2023-12-20T04:28:23.824 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9dac0b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da40d23b0 con 0x7f9da415d0d0 2023-12-20T04:28:23.824 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9daa653640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9da40d17f0 0x7f9da40b15c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36748/0 (socket says 172.21.15.32:36748) 2023-12-20T04:28:23.824 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9daa653640 1 -- 172.21.15.32:0/4036573891 learned_addr learned my addr 172.21.15.32:0/4036573891 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:23.826 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3414005685 0 0) 0x7f9da40b60b0 con 0x7f9da40d17f0 2023-12-20T04:28:23.826 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da9e52640 1 --2- 172.21.15.32:0/4036573891 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da415d0d0 0x7f9da40cf620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:23.826 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d90003530 con 0x7f9da40d17f0 2023-12-20T04:28:23.826 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1255881339 0 0) 0x7f9d90003530 con 0x7f9da40d17f0 2023-12-20T04:28:23.826 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da415d0d0 msgr2=0x7f9da40cf620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:23.826 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 --2- 172.21.15.32:0/4036573891 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da415d0d0 0x7f9da40cf620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:23.826 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 >> v1:172.21.15.39:6789/0 conn(0x7f9da40cfb60 legacy=0x7f9da40d10e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:23.826 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da40b62e0 con 0x7f9da40d17f0 2023-12-20T04:28:23.826 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f9d9c001a00 con 0x7f9da40d17f0 2023-12-20T04:28:23.826 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 >> v1:172.21.15.32:6789/0 conn(0x7f9da40d17f0 legacy=0x7f9da40b15c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:23.826 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 --2- 172.21.15.32:0/4036573891 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da415d0d0 0x7f9d900039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:23.827 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 --2- 172.21.15.32:0/4036573891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d90003fe0 0x7f9d900047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:23.827 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 --2- 172.21.15.32:0/4036573891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 0x7f9d90005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:23.827 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da40d23b0 con 0x7f9d90004d10 2023-12-20T04:28:23.827 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9000bb50 con 0x7f9d90003fe0 2023-12-20T04:28:23.827 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d9000bd10 con 0x7f9da415d0d0 2023-12-20T04:28:23.827 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9daa653640 1 --2- 172.21.15.32:0/4036573891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 0x7f9d90005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:23.827 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9daa653640 1 -- 172.21.15.32:0/4036573891 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da415d0d0 msgr2=0x7f9d900039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:23.827 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9daa653640 1 --2- 172.21.15.32:0/4036573891 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da415d0d0 0x7f9d900039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:23.827 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9daa653640 1 -- 172.21.15.32:0/4036573891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d90003fe0 msgr2=0x7f9d900047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:23.827 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9daa653640 1 --2- 172.21.15.32:0/4036573891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d90003fe0 0x7f9d900047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:23.827 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.821+0000 7f9daa653640 1 -- 172.21.15.32:0/4036573891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d90003530 con 0x7f9d90004d10 2023-12-20T04:28:23.828 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.825+0000 7f9daa653640 1 --2- 172.21.15.32:0/4036573891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 0x7f9d90005190 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00a820 tx=0x7f9d9c01d320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cffaa50e60ed8784 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:23.828 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.825+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0072d0 con 0x7f9d90004d10 2023-12-20T04:28:23.828 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.825+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d9c0a8070 con 0x7f9d90004d10 2023-12-20T04:28:23.828 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.825+0000 7f9da8e50640 1 -- 172.21.15.32:0/4036573891 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d9c0076f0 con 0x7f9d90004d10 2023-12-20T04:28:23.829 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.825+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4036573891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 msgr2=0x7f9d90005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:23.829 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.825+0000 7f9dac0b6640 1 --2- 172.21.15.32:0/4036573891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 0x7f9d90005190 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9d9c00a820 tx=0x7f9d9c01d320 comp rx=0 tx=0).stop 2023-12-20T04:28:23.829 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.825+0000 7f9daa653640 1 -- 172.21.15.32:0/4036573891 reap_dead start 2023-12-20T04:28:23.829 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.825+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4036573891 shutdown_connections 2023-12-20T04:28:23.829 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.825+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4036573891 >> 172.21.15.32:0/4036573891 conn(0x7f9da406b130 msgr2=0x7f9da406b540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:23.829 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.825+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4036573891 shutdown_connections 2023-12-20T04:28:23.829 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.825+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4036573891 wait complete. 2023-12-20T04:28:23.830 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9dac0b6640 1 Processor -- start 2023-12-20T04:28:23.830 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9dac0b6640 1 -- start start 2023-12-20T04:28:23.833 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9dac0b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 0x7f9da4159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:23.833 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9dac0b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da41566e0 0x7f9da4148720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:23.833 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9dac0b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da4148c60 0x7f9da4157320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:23.833 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9dac0b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da41f3e80 con 0x7f9d90004d10 2023-12-20T04:28:23.833 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9dac0b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9da41f3240 con 0x7f9da41566e0 2023-12-20T04:28:23.833 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9dac0b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9da4156b60 con 0x7f9da4148c60 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9da9e52640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 0x7f9da4159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9da9e52640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 0x7f9da4159680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60036/0 (socket says 172.21.15.32:60036) 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9da9e52640 1 -- 172.21.15.32:0/4165005894 learned_addr learned my addr 172.21.15.32:0/4165005894 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9daa653640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da4148c60 0x7f9da4157320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9da9651640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da41566e0 0x7f9da4148720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9da9e52640 1 -- 172.21.15.32:0/4165005894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da4148c60 msgr2=0x7f9da4157320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9da9e52640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da4148c60 0x7f9da4157320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9da9e52640 1 -- 172.21.15.32:0/4165005894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da41566e0 msgr2=0x7f9da4148720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9da9e52640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da41566e0 0x7f9da4148720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9da9e52640 1 -- 172.21.15.32:0/4165005894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9da41f34e0 con 0x7f9d90004d10 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9da9e52640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 0x7f9da4159680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9da000d800 tx=0x7f9da000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9d9a7fc640 1 -- 172.21.15.32:0/4165005894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da000b6c0 con 0x7f9d90004d10 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9d9a7fc640 1 -- 172.21.15.32:0/4165005894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9da000b820 con 0x7f9d90004d10 2023-12-20T04:28:23.834 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9d9a7fc640 1 -- 172.21.15.32:0/4165005894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9da000b9b0 con 0x7f9d90004d10 2023-12-20T04:28:23.835 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4165005894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9da41f37c0 con 0x7f9d90004d10 2023-12-20T04:28:23.835 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.829+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4165005894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9da4157860 con 0x7f9d90004d10 2023-12-20T04:28:23.835 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.833+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4165005894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9da40c6090 con 0x7f9d90004d10 2023-12-20T04:28:23.835 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.833+0000 7f9d9a7fc640 1 -- 172.21.15.32:0/4165005894 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 4) v1 ==== 85172+0+0 (secure 0 0 0) 0x7f9da000bb10 con 0x7f9d90004d10 2023-12-20T04:28:23.837 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.833+0000 7f9d9a7fc640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d880659d0 0x7f9d88067e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:23.837 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.833+0000 7f9d9a7fc640 1 -- 172.21.15.32:0/4165005894 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(12..12 src has 1..12) v4 ==== 2231+0+0 (secure 0 0 0) 0x7f9da00825b0 con 0x7f9d90004d10 2023-12-20T04:28:23.837 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.833+0000 7f9da9651640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d880659d0 0x7f9d88067e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:23.837 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.833+0000 7f9da9651640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d880659d0 0x7f9d88067e90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9da415bfa0 tx=0x7f9d94009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:23.840 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.837+0000 7f9d9a7fc640 1 -- 172.21.15.32:0/4165005894 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9da0050dd0 con 0x7f9d90004d10 2023-12-20T04:28:24.000 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.997+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4165005894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9da40671d0 con 0x7f9d90004d10 2023-12-20T04:28:24.001 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:23.997+0000 7f9d9a7fc640 1 -- 172.21.15.32:0/4165005894 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v12) v1 ==== 74+0+7057 (secure 0 0 0) 0x7f9da0018090 con 0x7f9d90004d10 2023-12-20T04:28:24.001 INFO:teuthology.misc.health.smithi032.stdout: 2023-12-20T04:28:24.001 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":12,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:21.650418+0000","last_up_change":"0.000000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:24.003 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4165005894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d880659d0 msgr2=0x7f9d88067e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:24.003 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d880659d0 0x7f9d88067e90 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9da415bfa0 tx=0x7f9d94009040 comp rx=0 tx=0).stop 2023-12-20T04:28:24.003 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4165005894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 msgr2=0x7f9da4159680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:24.004 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 0x7f9da4159680 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f9da000d800 tx=0x7f9da000dcc0 comp rx=0 tx=0).stop 2023-12-20T04:28:24.004 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4165005894 shutdown_connections 2023-12-20T04:28:24.004 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d880659d0 0x7f9d88067e90 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:24.004 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9da4148c60 0x7f9da4157320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:24.004 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9da41566e0 0x7f9da4148720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:24.004 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 --2- 172.21.15.32:0/4165005894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d90004d10 0x7f9da4159680 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:24.004 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4165005894 >> 172.21.15.32:0/4165005894 conn(0x7f9da406b130 msgr2=0x7f9da40c6bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:24.004 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4165005894 shutdown_connections 2023-12-20T04:28:24.004 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:24.001+0000 7f9dac0b6640 1 -- 172.21.15.32:0/4165005894 wait complete. 2023-12-20T04:28:24.021 DEBUG:teuthology.misc:0 of 8 OSDs are up 2023-12-20T04:28:24.297 INFO:tasks.ceph.osd.4.smithi039.stderr:2023-12-20T04:28:24.294+0000 7fe4bad55640 -1 osd.4 0 waiting for initial osdmap 2023-12-20T04:28:24.318 INFO:tasks.ceph.osd.5.smithi039.stderr:2023-12-20T04:28:24.318+0000 7f34e633d640 -1 osd.5 0 waiting for initial osdmap 2023-12-20T04:28:24.338 INFO:tasks.ceph.osd.4.smithi039.stderr:2023-12-20T04:28:24.338+0000 7fe4b339a640 -1 osd.4 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-20T04:28:24.354 INFO:tasks.ceph.osd.5.smithi039.stderr:2023-12-20T04:28:24.354+0000 7f34dd147640 -1 osd.5 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-20T04:28:24.418 INFO:tasks.ceph.osd.6.smithi039.stderr:2023-12-20T04:28:24.414+0000 7f3310824640 -1 osd.6 0 waiting for initial osdmap 2023-12-20T04:28:24.428 INFO:tasks.ceph.osd.7.smithi039.stderr:2023-12-20T04:28:24.426+0000 7f4fc22e5640 -1 osd.7 0 waiting for initial osdmap 2023-12-20T04:28:24.450 INFO:tasks.ceph.osd.6.smithi039.stderr:2023-12-20T04:28:24.450+0000 7f3308e69640 -1 osd.6 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-20T04:28:24.470 INFO:tasks.ceph.osd.7.smithi039.stderr:2023-12-20T04:28:24.470+0000 7f4fba92a640 -1 osd.7 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2023-12-20T04:28:28.979 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:28:28.984 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:28:30.828 DEBUG:teuthology.orchestra.run.smithi032:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:30.974 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.968+0000 7efe90109640 1 Processor -- start 2023-12-20T04:28:30.974 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.968+0000 7efe90109640 1 -- start start 2023-12-20T04:28:30.975 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe90109640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efe8806a430 0x7efe881659d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:30.976 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe90109640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe88070180 con 0x7efe880b16f0 2023-12-20T04:28:30.976 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe90109640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe880703d0 con 0x7efe8815e140 2023-12-20T04:28:30.976 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe90109640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe880c9d60 con 0x7efe8806a430 2023-12-20T04:28:30.976 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8d6a4640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efe880b16f0 0x7efe88069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56378/0 (socket says 172.21.15.32:56378) 2023-12-20T04:28:30.976 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8d6a4640 1 -- 172.21.15.32:0/2082691192 learned_addr learned my addr 172.21.15.32:0/2082691192 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:30.978 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8e6a6640 1 --2- 172.21.15.32:0/2082691192 >> v2:172.21.15.39:3300/0 conn(0x7efe8806a430 0x7efe881659d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:30.978 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1736870632 0 0) 0x7efe88070180 con 0x7efe880b16f0 2023-12-20T04:28:30.978 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe74003530 con 0x7efe880b16f0 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2769242218 0 0) 0x7efe880703d0 con 0x7efe8815e140 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe88070180 con 0x7efe8815e140 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3520759481 0 0) 0x7efe74003530 con 0x7efe880b16f0 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 >> v2:172.21.15.39:3300/0 conn(0x7efe8806a430 msgr2=0x7efe881659d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 --2- 172.21.15.32:0/2082691192 >> v2:172.21.15.39:3300/0 conn(0x7efe8806a430 0x7efe881659d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 >> v1:172.21.15.39:6789/0 conn(0x7efe8815e140 legacy=0x7efe880ad740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe880706b0 con 0x7efe880b16f0 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7efe78001680 con 0x7efe880b16f0 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 >> v1:172.21.15.32:6789/0 conn(0x7efe880b16f0 legacy=0x7efe88069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 --2- 172.21.15.32:0/2082691192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe74003b10 0x7efe74003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 --2- 172.21.15.32:0/2082691192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe740047d0 0x7efe74004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 --2- 172.21.15.32:0/2082691192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74005190 0x7efe7400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe880c9d60 con 0x7efe74003b10 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe7400c0c0 con 0x7efe74005190 2023-12-20T04:28:30.979 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe7400c280 con 0x7efe740047d0 2023-12-20T04:28:30.980 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8e6a6640 1 --2- 172.21.15.32:0/2082691192 >> v2:172.21.15.39:3300/0 conn(0x7efe8806a430 0x7efe881659d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:30.980 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8dea5640 1 --2- 172.21.15.32:0/2082691192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe74003b10 0x7efe74003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:30.980 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8d6a4640 1 --2- 172.21.15.32:0/2082691192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe740047d0 0x7efe74004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:30.980 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8e6a6640 1 --2- 172.21.15.32:0/2082691192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74005190 0x7efe7400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:30.980 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8dea5640 1 -- 172.21.15.32:0/2082691192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe740047d0 msgr2=0x7efe74004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:30.980 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8dea5640 1 --2- 172.21.15.32:0/2082691192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe740047d0 0x7efe74004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:30.980 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8dea5640 1 -- 172.21.15.32:0/2082691192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74005190 msgr2=0x7efe7400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:30.980 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8dea5640 1 --2- 172.21.15.32:0/2082691192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74005190 0x7efe7400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:30.980 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8dea5640 1 -- 172.21.15.32:0/2082691192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe74003530 con 0x7efe74003b10 2023-12-20T04:28:30.980 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.972+0000 7efe8e6a6640 1 -- 172.21.15.32:0/2082691192 reap_dead start 2023-12-20T04:28:30.981 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.976+0000 7efe8dea5640 1 --2- 172.21.15.32:0/2082691192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe74003b10 0x7efe74003f10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe840018d0 tx=0x7efe8401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7b37b6d720e865e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:30.981 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.976+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840052d0 con 0x7efe74003b10 2023-12-20T04:28:30.981 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.976+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe840a7050 con 0x7efe74003b10 2023-12-20T04:28:30.982 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.976+0000 7efe8cea3640 1 -- 172.21.15.32:0/2082691192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe84003600 con 0x7efe74003b10 2023-12-20T04:28:30.982 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.976+0000 7efe90109640 1 -- 172.21.15.32:0/2082691192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe74003b10 msgr2=0x7efe74003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:30.982 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.976+0000 7efe90109640 1 --2- 172.21.15.32:0/2082691192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe74003b10 0x7efe74003f10 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7efe840018d0 tx=0x7efe8401c500 comp rx=0 tx=0).stop 2023-12-20T04:28:30.982 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.976+0000 7efe90109640 1 -- 172.21.15.32:0/2082691192 shutdown_connections 2023-12-20T04:28:30.982 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.976+0000 7efe90109640 1 --2- 172.21.15.32:0/2082691192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe74003b10 0x7efe74003f10 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:30.982 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.976+0000 7efe90109640 1 -- 172.21.15.32:0/2082691192 >> 172.21.15.32:0/2082691192 conn(0x7efe880ce7a0 msgr2=0x7efe880cebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:30.982 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.976+0000 7efe90109640 1 -- 172.21.15.32:0/2082691192 shutdown_connections 2023-12-20T04:28:30.982 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.976+0000 7efe90109640 1 -- 172.21.15.32:0/2082691192 wait complete. 2023-12-20T04:28:30.983 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe90109640 1 Processor -- start 2023-12-20T04:28:30.983 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe90109640 1 -- start start 2023-12-20T04:28:30.994 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe90109640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74003b10 0x7efe881f6b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:30.995 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe90109640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe880b3310 0x7efe880c78a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:30.995 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe90109640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe880b3790 0x7efe88156de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:30.995 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe90109640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe880c82f0 con 0x7efe880b3310 2023-12-20T04:28:30.995 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe90109640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe881f6030 con 0x7efe74003b10 2023-12-20T04:28:30.995 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe90109640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe881f6310 con 0x7efe880b3790 2023-12-20T04:28:30.995 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8d6a4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe880b3310 0x7efe880c78a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:30.995 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8d6a4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe880b3310 0x7efe880c78a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55760/0 (socket says 172.21.15.32:55760) 2023-12-20T04:28:30.995 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8d6a4640 1 -- 172.21.15.32:0/3549161622 learned_addr learned my addr 172.21.15.32:0/3549161622 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:30.995 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8dea5640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74003b10 0x7efe881f6b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8e6a6640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe880b3790 0x7efe88156de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8d6a4640 1 -- 172.21.15.32:0/3549161622 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe880b3790 msgr2=0x7efe88156de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8d6a4640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe880b3790 0x7efe88156de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8d6a4640 1 -- 172.21.15.32:0/3549161622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74003b10 msgr2=0x7efe881f6b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8d6a4640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74003b10 0x7efe881f6b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8d6a4640 1 -- 172.21.15.32:0/3549161622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe881f65f0 con 0x7efe880b3310 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8e6a6640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe880b3790 0x7efe88156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe8d6a4640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe880b3310 0x7efe880c78a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efe78004ed0 tx=0x7efe78007430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe7e7fc640 1 -- 172.21.15.32:0/3549161622 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7800f7b0 con 0x7efe880b3310 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe7e7fc640 1 -- 172.21.15.32:0/3549161622 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe7800f930 con 0x7efe880b3310 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe7e7fc640 1 -- 172.21.15.32:0/3549161622 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe7800fce0 con 0x7efe880b3310 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe90109640 1 -- 172.21.15.32:0/3549161622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe880c85d0 con 0x7efe880b3310 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.980+0000 7efe90109640 1 -- 172.21.15.32:0/3549161622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe88151140 con 0x7efe880b3310 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.984+0000 7efe90109640 1 -- 172.21.15.32:0/3549161622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe880c9f80 con 0x7efe880b3310 2023-12-20T04:28:30.996 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.984+0000 7efe7e7fc640 1 -- 172.21.15.32:0/3549161622 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7efe7801d460 con 0x7efe880b3310 2023-12-20T04:28:30.997 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.984+0000 7efe7e7fc640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe6c065a60 0x7efe6c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:30.997 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.984+0000 7efe7e7fc640 1 -- 172.21.15.32:0/3549161622 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7efe78082df0 con 0x7efe880b3310 2023-12-20T04:28:30.997 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.984+0000 7efe8dea5640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe6c065a60 0x7efe6c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:30.997 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.984+0000 7efe8dea5640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe6c065a60 0x7efe6c067f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efe840ac000 tx=0x7efe840a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:30.997 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:30.988+0000 7efe7e7fc640 1 -- 172.21.15.32:0/3549161622 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efe78050b00 con 0x7efe880b3310 2023-12-20T04:28:31.151 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 -- 172.21.15.32:0/3549161622 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe881f6830 con 0x7efe880b3310 2023-12-20T04:28:31.152 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe7e7fc640 1 -- 172.21.15.32:0/3549161622 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12334 (secure 0 0 0) 0x7efe881f6830 con 0x7efe880b3310 2023-12-20T04:28:31.152 INFO:teuthology.misc.health.smithi032.stdout: 2023-12-20T04:28:31.152 INFO:teuthology.misc.health.smithi032.stdout:{"epoch":17,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:28.847601+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:31.154 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 -- 172.21.15.32:0/3549161622 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe6c065a60 msgr2=0x7efe6c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.154 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe6c065a60 0x7efe6c067f20 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7efe840ac000 tx=0x7efe840a6000 comp rx=0 tx=0).stop 2023-12-20T04:28:31.154 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 -- 172.21.15.32:0/3549161622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe880b3310 msgr2=0x7efe880c78a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.154 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe880b3310 0x7efe880c78a0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7efe78004ed0 tx=0x7efe78007430 comp rx=0 tx=0).stop 2023-12-20T04:28:31.154 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 -- 172.21.15.32:0/3549161622 shutdown_connections 2023-12-20T04:28:31.154 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe6c065a60 0x7efe6c067f20 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.154 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe880b3790 0x7efe88156de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.155 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe880b3310 0x7efe880c78a0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.155 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 --2- 172.21.15.32:0/3549161622 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe74003b10 0x7efe881f6b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.155 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 -- 172.21.15.32:0/3549161622 >> 172.21.15.32:0/3549161622 conn(0x7efe880ce7a0 msgr2=0x7efe880ae200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:31.155 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 -- 172.21.15.32:0/3549161622 shutdown_connections 2023-12-20T04:28:31.155 INFO:teuthology.misc.health.smithi032.stderr:2023-12-20T04:28:31.148+0000 7efe90109640 1 -- 172.21.15.32:0/3549161622 wait complete. 2023-12-20T04:28:31.172 DEBUG:teuthology.misc:8 of 8 OSDs are up 2023-12-20T04:28:31.173 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2023-12-20T04:28:31.173 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T04:28:31.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T04:28:31.177 INFO:tasks.ceph.mds.a:Started 2023-12-20T04:28:31.177 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T04:28:31.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T04:28:31.180 INFO:tasks.ceph.mds.c:Started 2023-12-20T04:28:31.181 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T04:28:31.181 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T04:28:31.185 INFO:tasks.ceph.mds.b:Started 2023-12-20T04:28:31.185 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T04:28:31.185 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T04:28:31.188 INFO:tasks.ceph.mds.d:Started 2023-12-20T04:28:31.189 INFO:tasks.ceph:Setting up CephFS filesystem(s)... 2023-12-20T04:28:31.189 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:31.277 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T04:28:31.277 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T04:28:31.289 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T04:28:31.290 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T04:28:31.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.332+0000 7f6d868fc640 1 Processor -- start 2023-12-20T04:28:31.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.332+0000 7f6d868fc640 1 -- start start 2023-12-20T04:28:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d868fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d8015d080 0x7f6d800cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d868fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6d8015de00 0x7f6d8006a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d868fc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d8015d970 con 0x7f6d80069d20 2023-12-20T04:28:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d868fc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d800ce740 con 0x7f6d8015de00 2023-12-20T04:28:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d868fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d8006a640 con 0x7f6d8015d080 2023-12-20T04:28:31.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d7ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6d80069d20 0x7f6d800cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56386/0 (socket says 172.21.15.32:56386) 2023-12-20T04:28:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d7ffff640 1 -- 172.21.15.32:0/1801125388 learned_addr learned my addr 172.21.15.32:0/1801125388 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d84e99640 1 --2- 172.21.15.32:0/1801125388 >> v2:172.21.15.32:3300/0 conn(0x7f6d8015de00 0x7f6d8006a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d7f7fe640 1 --2- 172.21.15.32:0/1801125388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d8015d080 0x7f6d800cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d7effd640 1 -- 172.21.15.32:0/1801125388 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2866029713 0 0) 0x7f6d8015d970 con 0x7f6d80069d20 2023-12-20T04:28:31.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d84e99640 1 -- 172.21.15.32:0/1801125388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d8015d080 msgr2=0x7f6d800cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d84e99640 1 --2- 172.21.15.32:0/1801125388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d8015d080 0x7f6d800cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d84e99640 1 -- 172.21.15.32:0/1801125388 >> v1:172.21.15.32:6789/0 conn(0x7f6d80069d20 legacy=0x7f6d800cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.336+0000 7f6d84e99640 1 -- 172.21.15.32:0/1801125388 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d800ce180 con 0x7f6d8015de00 2023-12-20T04:28:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d84e99640 1 --2- 172.21.15.32:0/1801125388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8015de00 0x7f6d8006a100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009d90 tx=0x7f6d7801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a44bd44a4ba1614 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d7effd640 1 -- 172.21.15.32:0/1801125388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d780a59e0 con 0x7f6d8015de00 2023-12-20T04:28:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d7effd640 1 -- 172.21.15.32:0/1801125388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d780a5b40 con 0x7f6d8015de00 2023-12-20T04:28:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d7effd640 1 -- 172.21.15.32:0/1801125388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d780ad440 con 0x7f6d8015de00 2023-12-20T04:28:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d868fc640 1 -- 172.21.15.32:0/1801125388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8015de00 msgr2=0x7f6d8006a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d868fc640 1 --2- 172.21.15.32:0/1801125388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8015de00 0x7f6d8006a100 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6d78009d90 tx=0x7f6d7801c920 comp rx=0 tx=0).stop 2023-12-20T04:28:31.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d868fc640 1 -- 172.21.15.32:0/1801125388 shutdown_connections 2023-12-20T04:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d868fc640 1 --2- 172.21.15.32:0/1801125388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8015de00 0x7f6d8006a100 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d868fc640 1 --2- 172.21.15.32:0/1801125388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d8015d080 0x7f6d800cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d868fc640 1 -- 172.21.15.32:0/1801125388 >> 172.21.15.32:0/1801125388 conn(0x7f6d800cc310 msgr2=0x7f6d800cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d868fc640 1 -- 172.21.15.32:0/1801125388 shutdown_connections 2023-12-20T04:28:31.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d868fc640 1 -- 172.21.15.32:0/1801125388 wait complete. 2023-12-20T04:28:31.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d868fc640 1 Processor -- start 2023-12-20T04:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.340+0000 7f6d868fc640 1 -- start start 2023-12-20T04:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d868fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d80069d20 0x7f6d8014bc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d868fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8015d080 0x7f6d800b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d868fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d8015de00 0x7f6d8006f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d868fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800b1610 con 0x7f6d8015d080 2023-12-20T04:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d868fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800b18f0 con 0x7f6d80069d20 2023-12-20T04:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d868fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d800b1bd0 con 0x7f6d8015de00 2023-12-20T04:28:31.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8015d080 0x7f6d800b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d80069d20 0x7f6d8014bc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8015d080 0x7f6d800b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55818/0 (socket says 172.21.15.32:55818) 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7f7fe640 1 -- 172.21.15.32:0/1721714288 learned_addr learned my addr 172.21.15.32:0/1721714288 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d84e99640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d8015de00 0x7f6d8006f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7f7fe640 1 -- 172.21.15.32:0/1721714288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d8015de00 msgr2=0x7f6d8006f2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7f7fe640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d8015de00 0x7f6d8006f2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7f7fe640 1 -- 172.21.15.32:0/1721714288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d80069d20 msgr2=0x7f6d8014bc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7f7fe640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d80069d20 0x7f6d8014bc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7f7fe640 1 -- 172.21.15.32:0/1721714288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d78009a70 con 0x7f6d8015d080 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7ffff640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d80069d20 0x7f6d8014bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7f7fe640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8015d080 0x7f6d800b0bc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400e930 tx=0x7f6d7400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7cff9640 1 -- 172.21.15.32:0/1721714288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d74009690 con 0x7f6d8015d080 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7cff9640 1 -- 172.21.15.32:0/1721714288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d7400ca00 con 0x7f6d8015d080 2023-12-20T04:28:31.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d868fc640 1 -- 172.21.15.32:0/1721714288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d800b1eb0 con 0x7f6d8015d080 2023-12-20T04:28:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7cff9640 1 -- 172.21.15.32:0/1721714288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d7400cd70 con 0x7f6d8015d080 2023-12-20T04:28:31.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d868fc640 1 -- 172.21.15.32:0/1721714288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d800af450 con 0x7f6d8015d080 2023-12-20T04:28:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d868fc640 1 -- 172.21.15.32:0/1721714288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d8004e000 con 0x7f6d8015d080 2023-12-20T04:28:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.344+0000 7f6d7cff9640 1 -- 172.21.15.32:0/1721714288 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6d74013470 con 0x7f6d8015d080 2023-12-20T04:28:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.348+0000 7f6d7cff9640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d6c065a60 0x7f6d6c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.348+0000 7f6d7cff9640 1 -- 172.21.15.32:0/1721714288 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6d740833b0 con 0x7f6d8015d080 2023-12-20T04:28:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.348+0000 7f6d7ffff640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d6c065a60 0x7f6d6c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.348+0000 7f6d7ffff640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d6c065a60 0x7f6d6c067f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d70007fd0 tx=0x7f6d70007d60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:31.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.352+0000 7f6d7cff9640 1 -- 172.21.15.32:0/1721714288 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6d74051220 con 0x7f6d8015d080 2023-12-20T04:28:31.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.508+0000 7f6d868fc640 1 -- 172.21.15.32:0/1721714288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6d800b2040 con 0x7f6d8015d080 2023-12-20T04:28:31.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.508+0000 7f6d7cff9640 1 -- 172.21.15.32:0/1721714288 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12334 (secure 0 0 0) 0x7f6d74054840 con 0x7f6d8015d080 2023-12-20T04:28:31.514 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:31.514 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:28.847601+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:31.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 -- 172.21.15.32:0/1721714288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d6c065a60 msgr2=0x7f6d6c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d6c065a60 0x7f6d6c067f20 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f6d70007fd0 tx=0x7f6d70007d60 comp rx=0 tx=0).stop 2023-12-20T04:28:31.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 -- 172.21.15.32:0/1721714288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8015d080 msgr2=0x7f6d800b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8015d080 0x7f6d800b0bc0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f6d7400e930 tx=0x7f6d7400edf0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 -- 172.21.15.32:0/1721714288 shutdown_connections 2023-12-20T04:28:31.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d6c065a60 0x7f6d6c067f20 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d8015de00 0x7f6d8006f2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d8015d080 0x7f6d800b0bc0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 --2- 172.21.15.32:0/1721714288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d80069d20 0x7f6d8014bc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 -- 172.21.15.32:0/1721714288 >> 172.21.15.32:0/1721714288 conn(0x7f6d800cc310 msgr2=0x7f6d8015f150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:31.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 -- 172.21.15.32:0/1721714288 shutdown_connections 2023-12-20T04:28:31.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.512+0000 7f6d868fc640 1 -- 172.21.15.32:0/1721714288 wait complete. 2023-12-20T04:28:31.536 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:28:31.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:28:31.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.676+0000 7fb2dd62d640 1 Processor -- start 2023-12-20T04:28:31.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.676+0000 7fb2dd62d640 1 -- start start 2023-12-20T04:28:31.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2dd62d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb2d815abb0 0x7fb2d80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2dd62d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb2d8070230 0x7fb2d8070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2dd62d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2d8070bc0 con 0x7fb2d80b0cb0 2023-12-20T04:28:31.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2dd62d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d806e160 con 0x7fb2d815abb0 2023-12-20T04:28:31.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2dd62d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2d815b6a0 con 0x7fb2d8070230 2023-12-20T04:28:31.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2cffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb2d80b0cb0 0x7fb2d815bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56400/0 (socket says 172.21.15.32:56400) 2023-12-20T04:28:31.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2cffff640 1 -- 172.21.15.32:0/560888821 learned_addr learned my addr 172.21.15.32:0/560888821 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:31.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2d6ffd640 1 --2- 172.21.15.32:0/560888821 >> v2:172.21.15.32:3300/0 conn(0x7fb2d815abb0 0x7fb2d80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2d77fe640 1 --2- 172.21.15.32:0/560888821 >> v2:172.21.15.39:3300/0 conn(0x7fb2d8070230 0x7fb2d8070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/560888821 >> v2:172.21.15.39:3300/0 conn(0x7fb2d8070230 msgr2=0x7fb2d8070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2d6ffd640 1 --2- 172.21.15.32:0/560888821 >> v2:172.21.15.39:3300/0 conn(0x7fb2d8070230 0x7fb2d8070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/560888821 >> v1:172.21.15.32:6789/0 conn(0x7fb2d80b0cb0 legacy=0x7fb2d815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2d67fc640 1 -- 172.21.15.32:0/560888821 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2723057669 0 0) 0x7fb2c0001120 con 0x7fb2d80b0cb0 2023-12-20T04:28:31.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.680+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/560888821 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2d806d8c0 con 0x7fb2d815abb0 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2d6ffd640 1 --2- 172.21.15.32:0/560888821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d815abb0 0x7fb2d80caaf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80b0c40 tx=0x7fb2c801c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6bec6c1f2dd82b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2d67fc640 1 -- 172.21.15.32:0/560888821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80ab660 con 0x7fb2d815abb0 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2d67fc640 1 -- 172.21.15.32:0/560888821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c80ab7c0 con 0x7fb2d815abb0 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2d67fc640 1 -- 172.21.15.32:0/560888821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80abb50 con 0x7fb2d815abb0 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 -- 172.21.15.32:0/560888821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d815abb0 msgr2=0x7fb2d80caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 --2- 172.21.15.32:0/560888821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d815abb0 0x7fb2d80caaf0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fb2d80b0c40 tx=0x7fb2c801c460 comp rx=0 tx=0).stop 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 -- 172.21.15.32:0/560888821 shutdown_connections 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 --2- 172.21.15.32:0/560888821 >> v2:172.21.15.39:3300/0 conn(0x7fb2d8070230 0x7fb2d8070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 --2- 172.21.15.32:0/560888821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d815abb0 0x7fb2d80caaf0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 -- 172.21.15.32:0/560888821 >> 172.21.15.32:0/560888821 conn(0x7fb2d806a510 msgr2=0x7fb2d80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 -- 172.21.15.32:0/560888821 shutdown_connections 2023-12-20T04:28:31.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 -- 172.21.15.32:0/560888821 wait complete. 2023-12-20T04:28:31.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 Processor -- start 2023-12-20T04:28:31.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 -- start start 2023-12-20T04:28:31.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d8070230 0x7fb2d8159450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2d80b0cb0 0x7fb2d814f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d815abb0 0x7fb2d80a75c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d8151550 con 0x7fb2d8070230 2023-12-20T04:28:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d814f920 con 0x7fb2d815abb0 2023-12-20T04:28:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2dd62d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2d81525e0 con 0x7fb2d80b0cb0 2023-12-20T04:28:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.684+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d8070230 0x7fb2d8159450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2d6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d8070230 0x7fb2d8159450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55844/0 (socket says 172.21.15.32:55844) 2023-12-20T04:28:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/3808917421 learned_addr learned my addr 172.21.15.32:0/3808917421 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2cffff640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2d80b0cb0 0x7fb2d814f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2d77fe640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d815abb0 0x7fb2d80a75c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/3808917421 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2d80b0cb0 msgr2=0x7fb2d814f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2d6ffd640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2d80b0cb0 0x7fb2d814f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/3808917421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d815abb0 msgr2=0x7fb2d80a75c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2d6ffd640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d815abb0 0x7fb2d80a75c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2d6ffd640 1 -- 172.21.15.32:0/3808917421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2c8009420 con 0x7fb2d8070230 2023-12-20T04:28:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2d6ffd640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d8070230 0x7fb2d8159450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80ad040 tx=0x7fb2c801aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2cf7fe640 1 -- 172.21.15.32:0/3808917421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80bf610 con 0x7fb2d8070230 2023-12-20T04:28:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2cf7fe640 1 -- 172.21.15.32:0/3808917421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2c80bf790 con 0x7fb2d8070230 2023-12-20T04:28:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2dd62d640 1 -- 172.21.15.32:0/3808917421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2d81528c0 con 0x7fb2d8070230 2023-12-20T04:28:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2cf7fe640 1 -- 172.21.15.32:0/3808917421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2c80bfb40 con 0x7fb2d8070230 2023-12-20T04:28:31.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2dd62d640 1 -- 172.21.15.32:0/3808917421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2d81517a0 con 0x7fb2d8070230 2023-12-20T04:28:31.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2dd62d640 1 -- 172.21.15.32:0/3808917421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2d8152d70 con 0x7fb2d8070230 2023-12-20T04:28:31.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.688+0000 7fb2cf7fe640 1 -- 172.21.15.32:0/3808917421 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb2c80a9070 con 0x7fb2d8070230 2023-12-20T04:28:31.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.692+0000 7fb2cf7fe640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb2b8065a60 0x7fb2b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:31.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.692+0000 7fb2cf7fe640 1 -- 172.21.15.32:0/3808917421 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb2c811d760 con 0x7fb2d8070230 2023-12-20T04:28:31.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.692+0000 7fb2cffff640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb2b8065a60 0x7fb2b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:31.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.692+0000 7fb2cffff640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb2b8065a60 0x7fb2b8067f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8158b30 tx=0x7fb2c0007ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:31.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.696+0000 7fb2cf7fe640 1 -- 172.21.15.32:0/3808917421 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb2c80eb640 con 0x7fb2d8070230 2023-12-20T04:28:31.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.848+0000 7fb2dd62d640 1 -- 172.21.15.32:0/3808917421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb2d804e000 con 0x7fb2d8070230 2023-12-20T04:28:31.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2cf7fe640 1 -- 172.21.15.32:0/3808917421 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb2c80eec60 con 0x7fb2d8070230 2023-12-20T04:28:31.855 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:28:31.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 -- 172.21.15.32:0/3808917421 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb2b8065a60 msgr2=0x7fb2b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb2b8065a60 0x7fb2b8067f20 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fb2d8158b30 tx=0x7fb2c0007ba0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 -- 172.21.15.32:0/3808917421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d8070230 msgr2=0x7fb2d8159450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:31.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d8070230 0x7fb2d8159450 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb2c80ad040 tx=0x7fb2c801aca0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 -- 172.21.15.32:0/3808917421 shutdown_connections 2023-12-20T04:28:31.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb2b8065a60 0x7fb2b8067f20 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2d815abb0 0x7fb2d80a75c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2d80b0cb0 0x7fb2d814f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 --2- 172.21.15.32:0/3808917421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2d8070230 0x7fb2d8159450 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 -- 172.21.15.32:0/3808917421 >> 172.21.15.32:0/3808917421 conn(0x7fb2d806a510 msgr2=0x7fb2d80c47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 -- 172.21.15.32:0/3808917421 shutdown_connections 2023-12-20T04:28:31.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:31.852+0000 7fb2dd62d640 1 -- 172.21.15.32:0/3808917421 wait complete. 2023-12-20T04:28:31.876 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:28:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f593582d640 1 Processor -- start 2023-12-20T04:28:32.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f593582d640 1 -- start start 2023-12-20T04:28:32.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f593582d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cfc80 0x7f5930167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:32.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f593582d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59300b6ca0 con 0x7f593015cdb0 2023-12-20T04:28:32.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f593582d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f593015d190 con 0x7f59300d2740 2023-12-20T04:28:32.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f593582d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59300d16a0 con 0x7f59300cfc80 2023-12-20T04:28:32.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592effd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f593015cdb0 0x7f59300d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56410/0 (socket says 172.21.15.32:56410) 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592effd640 1 -- 172.21.15.32:0/1412864133 learned_addr learned my addr 172.21.15.32:0/1412864133 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592f7fe640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cfc80 0x7f5930167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 952421624 0 0) 0x7f5920001120 con 0x7f593015cdb0 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f591c003530 con 0x7f593015cdb0 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1365353002 0 0) 0x7f593015d190 con 0x7f59300d2740 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5920001120 con 0x7f59300d2740 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 203794745 0 0) 0x7f591c003530 con 0x7f593015cdb0 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cfc80 msgr2=0x7f5930167910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cfc80 0x7f5930167910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 >> v1:172.21.15.39:6789/0 conn(0x7f59300d2740 legacy=0x7f59300cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59300d1980 con 0x7f593015cdb0 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f5920001810 con 0x7f593015cdb0 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 >> v1:172.21.15.32:6789/0 conn(0x7f593015cdb0 legacy=0x7f59300d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:32.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591c003b40 0x7f591c003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cfc80 0x7f591c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f591c004d10 0x7f591c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59300d16a0 con 0x7f591c004d10 2023-12-20T04:28:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f591c00ba40 con 0x7f591c003b40 2023-12-20T04:28:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.016+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f591c00bc00 con 0x7f59300cfc80 2023-12-20T04:28:32.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f592f7fe640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f591c004d10 0x7f591c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f592effd640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591c003b40 0x7f591c003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f5927fff640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cfc80 0x7f591c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f592f7fe640 1 -- 172.21.15.32:0/1412864133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cfc80 msgr2=0x7f591c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f592f7fe640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cfc80 0x7f591c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f592f7fe640 1 -- 172.21.15.32:0/1412864133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591c003b40 msgr2=0x7f591c003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f592f7fe640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591c003b40 0x7f591c003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:32.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f592f7fe640 1 -- 172.21.15.32:0/1412864133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f591c003530 con 0x7f591c004d10 2023-12-20T04:28:32.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f592f7fe640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f591c004d10 0x7f591c005190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5928009f90 tx=0x7f592801cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90a640bce35da77 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59280a5c30 con 0x7f591c004d10 2023-12-20T04:28:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f592800f040 con 0x7f591c004d10 2023-12-20T04:28:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.020+0000 7f592e7fc640 1 -- 172.21.15.32:0/1412864133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59280ad440 con 0x7f591c004d10 2023-12-20T04:28:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 -- 172.21.15.32:0/1412864133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f591c004d10 msgr2=0x7f591c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 --2- 172.21.15.32:0/1412864133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f591c004d10 0x7f591c005190 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5928009f90 tx=0x7f592801cc20 comp rx=0 tx=0).stop 2023-12-20T04:28:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f592f7fe640 1 -- 172.21.15.32:0/1412864133 reap_dead start 2023-12-20T04:28:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 -- 172.21.15.32:0/1412864133 shutdown_connections 2023-12-20T04:28:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 -- 172.21.15.32:0/1412864133 >> 172.21.15.32:0/1412864133 conn(0x7f5930065ee0 msgr2=0x7f593006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 -- 172.21.15.32:0/1412864133 shutdown_connections 2023-12-20T04:28:32.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 -- 172.21.15.32:0/1412864133 wait complete. 2023-12-20T04:28:32.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 Processor -- start 2023-12-20T04:28:32.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 -- start start 2023-12-20T04:28:32.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591c004d10 0x7f593014a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:32.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cb960 0x7f593015dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:32.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59300cbde0 0x7f593015e270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:32.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59300cc970 con 0x7f59300cbde0 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59300cc3b0 con 0x7f591c004d10 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f593582d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59300cc690 con 0x7f59300cb960 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f592f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59300cbde0 0x7f593015e270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f592f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59300cbde0 0x7f593015e270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55860/0 (socket says 172.21.15.32:55860) 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f592effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591c004d10 0x7f593014a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f592f7fe640 1 -- 172.21.15.32:0/309310716 learned_addr learned my addr 172.21.15.32:0/309310716 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f5927fff640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cb960 0x7f593015dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f592f7fe640 1 -- 172.21.15.32:0/309310716 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cb960 msgr2=0x7f593015dd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f592f7fe640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cb960 0x7f593015dd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f592f7fe640 1 -- 172.21.15.32:0/309310716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591c004d10 msgr2=0x7f593014a210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f592f7fe640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591c004d10 0x7f593014a210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.024+0000 7f592f7fe640 1 -- 172.21.15.32:0/309310716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f593014be20 con 0x7f59300cbde0 2023-12-20T04:28:32.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.028+0000 7f592f7fe640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59300cbde0 0x7f593015e270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59280b0040 tx=0x7f5928009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.028+0000 7f592cff9640 1 -- 172.21.15.32:0/309310716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59280039a0 con 0x7f59300cbde0 2023-12-20T04:28:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.028+0000 7f592cff9640 1 -- 172.21.15.32:0/309310716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5928003b00 con 0x7f59300cbde0 2023-12-20T04:28:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.028+0000 7f593582d640 1 -- 172.21.15.32:0/309310716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f593014c100 con 0x7f59300cbde0 2023-12-20T04:28:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.028+0000 7f592cff9640 1 -- 172.21.15.32:0/309310716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59280ae890 con 0x7f59300cbde0 2023-12-20T04:28:32.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.028+0000 7f593582d640 1 -- 172.21.15.32:0/309310716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59301f2b80 con 0x7f59300cbde0 2023-12-20T04:28:32.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.028+0000 7f593582d640 1 -- 172.21.15.32:0/309310716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f593015c630 con 0x7f59300cbde0 2023-12-20T04:28:32.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.028+0000 7f592cff9640 1 -- 172.21.15.32:0/309310716 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f59280b4040 con 0x7f59300cbde0 2023-12-20T04:28:32.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.028+0000 7f592cff9640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5910065a90 0x7f5910067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:32.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.028+0000 7f592cff9640 1 -- 172.21.15.32:0/309310716 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f592811cb90 con 0x7f59300cbde0 2023-12-20T04:28:32.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.028+0000 7f592effd640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5910065a90 0x7f5910067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:32.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.032+0000 7f592effd640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5910065a90 0x7f5910067f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5920001de0 tx=0x7f5920007a00 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:32.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.032+0000 7f592cff9640 1 -- 172.21.15.32:0/309310716 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f59280ea8b0 con 0x7f59300cbde0 2023-12-20T04:28:32.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 -- 172.21.15.32:0/309310716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f593004e000 con 0x7f59300cbde0 2023-12-20T04:28:32.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f592cff9640 1 -- 172.21.15.32:0/309310716 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) v1 ==== 93+0+1253 (secure 0 0 0) 0x7f59280be500 con 0x7f59300cbde0 2023-12-20T04:28:32.229 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:32.229 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/797559820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":797559820},{"type":"v1","addr":"172.21.15.32:6836","nonce":797559820}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2}],"filesystems":[]} 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 2 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 -- 172.21.15.32:0/309310716 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5910065a90 msgr2=0x7f5910067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5910065a90 0x7f5910067f50 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5920001de0 tx=0x7f5920007a00 comp rx=0 tx=0).stop 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 -- 172.21.15.32:0/309310716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59300cbde0 msgr2=0x7f593015e270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59300cbde0 0x7f593015e270 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f59280b0040 tx=0x7f5928009750 comp rx=0 tx=0).stop 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 -- 172.21.15.32:0/309310716 shutdown_connections 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5910065a90 0x7f5910067f50 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59300cbde0 0x7f593015e270 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59300cb960 0x7f593015dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 --2- 172.21.15.32:0/309310716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f591c004d10 0x7f593014a210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 -- 172.21.15.32:0/309310716 >> 172.21.15.32:0/309310716 conn(0x7f5930065ee0 msgr2=0x7f59300d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:32.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 -- 172.21.15.32:0/309310716 shutdown_connections 2023-12-20T04:28:32.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:32.224+0000 7f593582d640 1 -- 172.21.15.32:0/309310716 wait complete. 2023-12-20T04:28:34.250 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:28:34.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.400+0000 7f891122d640 1 Processor -- start 2023-12-20T04:28:34.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.400+0000 7f891122d640 1 -- start start 2023-12-20T04:28:34.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.400+0000 7f891122d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f890c0aece0 0x7f890c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.400+0000 7f891122d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f890c0c98c0 con 0x7f890c0b0b30 2023-12-20T04:28:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.400+0000 7f891122d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f890c0c9ba0 con 0x7f890c1599e0 2023-12-20T04:28:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.400+0000 7f891122d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f890c0c9e80 con 0x7f890c0aece0 2023-12-20T04:28:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f890b59e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f890c0b0b30 0x7f890c0701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56414/0 (socket says 172.21.15.32:56414) 2023-12-20T04:28:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f8903fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f890c0aece0 0x7f890c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f890b59e640 1 -- 172.21.15.32:0/2939591363 learned_addr learned my addr 172.21.15.32:0/2939591363 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:34.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f890a59c640 1 -- 172.21.15.32:0/2939591363 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2079895746 0 0) 0x7f890c0c98c0 con 0x7f890c0b0b30 2023-12-20T04:28:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f890a59c640 1 -- 172.21.15.32:0/2939591363 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f88f8005740 con 0x7f890c0b0b30 2023-12-20T04:28:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f8903fff640 1 -- 172.21.15.32:0/2939591363 >> v1:172.21.15.39:6789/0 conn(0x7f890c1599e0 legacy=0x7f890c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f8903fff640 1 -- 172.21.15.32:0/2939591363 >> v1:172.21.15.32:6789/0 conn(0x7f890c0b0b30 legacy=0x7f890c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f8903fff640 1 -- 172.21.15.32:0/2939591363 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f890c0ca160 con 0x7f890c0aece0 2023-12-20T04:28:34.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f8903fff640 1 --2- 172.21.15.32:0/2939591363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c0aece0 0x7f890c0b05f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88f40097c0 tx=0x7f88f401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1f16690776f62c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:34.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f890a59c640 1 -- 172.21.15.32:0/2939591363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40a8050 con 0x7f890c0aece0 2023-12-20T04:28:34.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f890a59c640 1 -- 172.21.15.32:0/2939591363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f40ac5f0 con 0x7f890c0aece0 2023-12-20T04:28:34.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.404+0000 7f890a59c640 1 -- 172.21.15.32:0/2939591363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40ac980 con 0x7f890c0aece0 2023-12-20T04:28:34.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 -- 172.21.15.32:0/2939591363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c0aece0 msgr2=0x7f890c0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 --2- 172.21.15.32:0/2939591363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c0aece0 0x7f890c0b05f0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f88f40097c0 tx=0x7f88f401c480 comp rx=0 tx=0).stop 2023-12-20T04:28:34.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 -- 172.21.15.32:0/2939591363 shutdown_connections 2023-12-20T04:28:34.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 --2- 172.21.15.32:0/2939591363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c0aece0 0x7f890c0b05f0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 -- 172.21.15.32:0/2939591363 >> 172.21.15.32:0/2939591363 conn(0x7f890c06a320 msgr2=0x7f890c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:34.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 -- 172.21.15.32:0/2939591363 shutdown_connections 2023-12-20T04:28:34.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 -- 172.21.15.32:0/2939591363 wait complete. 2023-12-20T04:28:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 Processor -- start 2023-12-20T04:28:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 -- start start 2023-12-20T04:28:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f890c0aece0 0x7f890c15a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c0b0b30 0x7f890c14b890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f890c1599e0 0x7f890c06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c1554d0 con 0x7f890c0b0b30 2023-12-20T04:28:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f890c15a390 con 0x7f890c1599e0 2023-12-20T04:28:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f891122d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f890c0c8620 con 0x7f890c0aece0 2023-12-20T04:28:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f8903fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c0b0b30 0x7f890c14b890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f8903fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c0b0b30 0x7f890c14b890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55874/0 (socket says 172.21.15.32:55874) 2023-12-20T04:28:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f8903fff640 1 -- 172.21.15.32:0/3017855882 learned_addr learned my addr 172.21.15.32:0/3017855882 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f890b59e640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f890c1599e0 0x7f890c06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:34.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f890ad9d640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f890c0aece0 0x7f890c15a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f8903fff640 1 -- 172.21.15.32:0/3017855882 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f890c0aece0 msgr2=0x7f890c15a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.408+0000 7f8903fff640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f890c0aece0 0x7f890c15a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f8903fff640 1 -- 172.21.15.32:0/3017855882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f890c1599e0 msgr2=0x7f890c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f8903fff640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f890c1599e0 0x7f890c06c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f8903fff640 1 -- 172.21.15.32:0/3017855882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88f4009420 con 0x7f890c0b0b30 2023-12-20T04:28:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f8903fff640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c0b0b30 0x7f890c14b890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88f401a0b0 tx=0x7f88f4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f89037fe640 1 -- 172.21.15.32:0/3017855882 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f4006e20 con 0x7f890c0b0b30 2023-12-20T04:28:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f89037fe640 1 -- 172.21.15.32:0/3017855882 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88f40b5620 con 0x7f890c0b0b30 2023-12-20T04:28:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f891122d640 1 -- 172.21.15.32:0/3017855882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f890c0c8900 con 0x7f890c0b0b30 2023-12-20T04:28:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f89037fe640 1 -- 172.21.15.32:0/3017855882 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88f40b59b0 con 0x7f890c0b0b30 2023-12-20T04:28:34.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f891122d640 1 -- 172.21.15.32:0/3017855882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f890c14bdd0 con 0x7f890c0b0b30 2023-12-20T04:28:34.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f891122d640 1 -- 172.21.15.32:0/3017855882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f890c15a0f0 con 0x7f890c0b0b30 2023-12-20T04:28:34.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f89037fe640 1 -- 172.21.15.32:0/3017855882 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f88f40b3040 con 0x7f890c0b0b30 2023-12-20T04:28:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f89037fe640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f88ec065a60 0x7f88ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f89037fe640 1 -- 172.21.15.32:0/3017855882 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f88f411ca80 con 0x7f890c0b0b30 2023-12-20T04:28:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.412+0000 7f890ad9d640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f88ec065a60 0x7f88ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.416+0000 7f890ad9d640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f88ec065a60 0x7f88ec067f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f88fc000fc0 tx=0x7f88fc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.416+0000 7f89037fe640 1 -- 172.21.15.32:0/3017855882 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f890c15a0f0 con 0x7f890c0b0b30 2023-12-20T04:28:34.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 -- 172.21.15.32:0/3017855882 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f890c15a0f0 con 0x7f890c0b0b30 2023-12-20T04:28:34.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f89037fe640 1 -- 172.21.15.32:0/3017855882 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) v1 ==== 93+0+3521 (secure 0 0 0) 0x7f890c15a0f0 con 0x7f890c0b0b30 2023-12-20T04:28:34.616 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:34.616 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2247020919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2247020919},{"type":"v1","addr":"172.21.15.39:6834","nonce":2247020919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/797559820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":797559820},{"type":"v1","addr":"172.21.15.32:6836","nonce":797559820}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3209531230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3209531230},{"type":"v1","addr":"172.21.15.32:6837","nonce":3209531230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4300,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3547681098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3547681098},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547681098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[]} 2023-12-20T04:28:34.616 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 3 2023-12-20T04:28:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 -- 172.21.15.32:0/3017855882 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f88ec065a60 msgr2=0x7f88ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f88ec065a60 0x7f88ec067f20 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f88fc000fc0 tx=0x7f88fc009040 comp rx=0 tx=0).stop 2023-12-20T04:28:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 -- 172.21.15.32:0/3017855882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c0b0b30 msgr2=0x7f890c14b890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c0b0b30 0x7f890c14b890 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88f401a0b0 tx=0x7f88f4019220 comp rx=0 tx=0).stop 2023-12-20T04:28:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 -- 172.21.15.32:0/3017855882 shutdown_connections 2023-12-20T04:28:34.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f88ec065a60 0x7f88ec067f20 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f890c1599e0 0x7f890c06c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f890c0b0b30 0x7f890c14b890 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 --2- 172.21.15.32:0/3017855882 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f890c0aece0 0x7f890c15a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 -- 172.21.15.32:0/3017855882 >> 172.21.15.32:0/3017855882 conn(0x7f890c06a320 msgr2=0x7f890c156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 -- 172.21.15.32:0/3017855882 shutdown_connections 2023-12-20T04:28:34.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.612+0000 7f891122d640 1 -- 172.21.15.32:0/3017855882 wait complete. 2023-12-20T04:28:34.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:34.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.776+0000 7f8b682f9640 1 Processor -- start 2023-12-20T04:28:34.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.776+0000 7f8b682f9640 1 -- start start 2023-12-20T04:28:34.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.776+0000 7f8b682f9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b601599e0 0x7f8b600aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:34.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.776+0000 7f8b682f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b600c90e0 0x7f8b600b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:34.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.776+0000 7f8b682f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b600711d0 con 0x7f8b600b0b30 2023-12-20T04:28:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.776+0000 7f8b682f9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b600c98c0 con 0x7f8b601599e0 2023-12-20T04:28:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.776+0000 7f8b682f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600c9ba0 con 0x7f8b600c90e0 2023-12-20T04:28:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.776+0000 7f8b66095640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8b601599e0 0x7f8b600aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.776+0000 7f8b66896640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8b600b0b30 0x7f8b6015d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56418/0 (socket says 172.21.15.32:56418) 2023-12-20T04:28:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.776+0000 7f8b66896640 1 -- 172.21.15.32:0/2535698930 learned_addr learned my addr 172.21.15.32:0/2535698930 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:34.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.776+0000 7f8b65894640 1 --2- 172.21.15.32:0/2535698930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b600c90e0 0x7f8b600b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b66095640 1 -- 172.21.15.32:0/2535698930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b600c90e0 msgr2=0x7f8b600b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b66095640 1 --2- 172.21.15.32:0/2535698930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b600c90e0 0x7f8b600b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b66095640 1 -- 172.21.15.32:0/2535698930 >> v1:172.21.15.32:6789/0 conn(0x7f8b600b0b30 legacy=0x7f8b6015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b65093640 1 -- 172.21.15.32:0/2535698930 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2890820052 0 0) 0x7f8b600711d0 con 0x7f8b600b0b30 2023-12-20T04:28:34.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b66095640 1 -- 172.21.15.32:0/2535698930 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b600c9e80 con 0x7f8b601599e0 2023-12-20T04:28:34.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b66095640 1 --2- 172.21.15.32:0/2535698930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b601599e0 0x7f8b600aece0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0097a0 tx=0x7f8b5c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ecf80e396f04f2a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b65093640 1 -- 172.21.15.32:0/2535698930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c004230 con 0x7f8b601599e0 2023-12-20T04:28:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b65093640 1 -- 172.21.15.32:0/2535698930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5c0a7050 con 0x7f8b601599e0 2023-12-20T04:28:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b65093640 1 -- 172.21.15.32:0/2535698930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5c003620 con 0x7f8b601599e0 2023-12-20T04:28:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b682f9640 1 -- 172.21.15.32:0/2535698930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b601599e0 msgr2=0x7f8b600aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b682f9640 1 --2- 172.21.15.32:0/2535698930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b601599e0 0x7f8b600aece0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0097a0 tx=0x7f8b5c01c430 comp rx=0 tx=0).stop 2023-12-20T04:28:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b682f9640 1 -- 172.21.15.32:0/2535698930 shutdown_connections 2023-12-20T04:28:34.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b682f9640 1 --2- 172.21.15.32:0/2535698930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b600c90e0 0x7f8b600b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b682f9640 1 --2- 172.21.15.32:0/2535698930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b601599e0 0x7f8b600aece0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.780+0000 7f8b682f9640 1 -- 172.21.15.32:0/2535698930 >> 172.21.15.32:0/2535698930 conn(0x7f8b6006a320 msgr2=0x7f8b600ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b682f9640 1 -- 172.21.15.32:0/2535698930 shutdown_connections 2023-12-20T04:28:34.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b682f9640 1 -- 172.21.15.32:0/2535698930 wait complete. 2023-12-20T04:28:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b682f9640 1 Processor -- start 2023-12-20T04:28:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b682f9640 1 -- start start 2023-12-20T04:28:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b682f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b600b0b30 0x7f8b600c6cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b682f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b600c90e0 0x7f8b6015b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b682f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b601599e0 0x7f8b600c4140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b682f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600c7fd0 con 0x7f8b601599e0 2023-12-20T04:28:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b682f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600c7350 con 0x7f8b600b0b30 2023-12-20T04:28:34.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b682f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b600c7630 con 0x7f8b600c90e0 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b66896640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b601599e0 0x7f8b600c4140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b66896640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b601599e0 0x7f8b600c4140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55892/0 (socket says 172.21.15.32:55892) 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b66896640 1 -- 172.21.15.32:0/984876968 learned_addr learned my addr 172.21.15.32:0/984876968 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b65894640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b600c90e0 0x7f8b6015b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b66095640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b600b0b30 0x7f8b600c6cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b66896640 1 -- 172.21.15.32:0/984876968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b600c90e0 msgr2=0x7f8b6015b190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b66896640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b600c90e0 0x7f8b6015b190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b66896640 1 -- 172.21.15.32:0/984876968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b600b0b30 msgr2=0x7f8b600c6cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b66896640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b600b0b30 0x7f8b600c6cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b66896640 1 -- 172.21.15.32:0/984876968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b5c0093b0 con 0x7f8b601599e0 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b65894640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b600c90e0 0x7f8b6015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b66095640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b600b0b30 0x7f8b600c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b66896640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b601599e0 0x7f8b600c4140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c700 tx=0x7f8b5800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b56ffd640 1 -- 172.21.15.32:0/984876968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800ce10 con 0x7f8b601599e0 2023-12-20T04:28:34.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b682f9640 1 -- 172.21.15.32:0/984876968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b600c7910 con 0x7f8b601599e0 2023-12-20T04:28:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b56ffd640 1 -- 172.21.15.32:0/984876968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b5800a640 con 0x7f8b601599e0 2023-12-20T04:28:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.784+0000 7f8b56ffd640 1 -- 172.21.15.32:0/984876968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b5800aa30 con 0x7f8b601599e0 2023-12-20T04:28:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.788+0000 7f8b682f9640 1 -- 172.21.15.32:0/984876968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b6015b6d0 con 0x7f8b601599e0 2023-12-20T04:28:34.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.788+0000 7f8b682f9640 1 -- 172.21.15.32:0/984876968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b600b0f60 con 0x7f8b601599e0 2023-12-20T04:28:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.788+0000 7f8b56ffd640 1 -- 172.21.15.32:0/984876968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8b5800ab90 con 0x7f8b601599e0 2023-12-20T04:28:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.788+0000 7f8b56ffd640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b44065a60 0x7f8b44067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.788+0000 7f8b56ffd640 1 -- 172.21.15.32:0/984876968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f8b580825b0 con 0x7f8b601599e0 2023-12-20T04:28:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.788+0000 7f8b66095640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b44065a60 0x7f8b44067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.788+0000 7f8b66095640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b44065a60 0x7f8b44067f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0ad000 tx=0x7f8b5c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:34.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.792+0000 7f8b56ffd640 1 -- 172.21.15.32:0/984876968 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8b600b0f60 con 0x7f8b601599e0 2023-12-20T04:28:34.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.952+0000 7f8b682f9640 1 -- 172.21.15.32:0/984876968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8b600b0f60 con 0x7f8b601599e0 2023-12-20T04:28:34.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.952+0000 7f8b56ffd640 1 -- 172.21.15.32:0/984876968 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v17) v1 ==== 74+0+12334 (secure 0 0 0) 0x7f8b600b0f60 con 0x7f8b601599e0 2023-12-20T04:28:34.958 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:34.958 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":17,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:28.847601+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:34.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 -- 172.21.15.32:0/984876968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b44065a60 msgr2=0x7f8b44067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b44065a60 0x7f8b44067f20 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f8b5c0ad000 tx=0x7f8b5c008040 comp rx=0 tx=0).stop 2023-12-20T04:28:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 -- 172.21.15.32:0/984876968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b601599e0 msgr2=0x7f8b600c4140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b601599e0 0x7f8b600c4140 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8b5800c700 tx=0x7f8b5800cbc0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 -- 172.21.15.32:0/984876968 shutdown_connections 2023-12-20T04:28:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b44065a60 0x7f8b44067f20 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b601599e0 0x7f8b600c4140 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b600c90e0 0x7f8b6015b190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 --2- 172.21.15.32:0/984876968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b600b0b30 0x7f8b600c6cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 -- 172.21.15.32:0/984876968 >> 172.21.15.32:0/984876968 conn(0x7f8b6006a320 msgr2=0x7f8b600ce670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 -- 172.21.15.32:0/984876968 shutdown_connections 2023-12-20T04:28:34.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:34.956+0000 7f8b682f9640 1 -- 172.21.15.32:0/984876968 wait complete. 2023-12-20T04:28:34.979 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:28:34.979 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:28:35.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.120+0000 7f6279d39640 1 Processor -- start 2023-12-20T04:28:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.120+0000 7f6279d39640 1 -- start start 2023-12-20T04:28:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f6279d39640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62740b0cb0 0x7f62740caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f6279d39640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627415b6a0 con 0x7f6274070230 2023-12-20T04:28:35.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f6279d39640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f627406d8c0 con 0x7f627415abb0 2023-12-20T04:28:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f6279d39640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f627406e160 con 0x7f62740b0cb0 2023-12-20T04:28:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f6272ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62740b0cb0 0x7f62740caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f6273fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6274070230 0x7f627406c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56432/0 (socket says 172.21.15.32:56432) 2023-12-20T04:28:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f6273fff640 1 -- 172.21.15.32:0/1908651792 learned_addr learned my addr 172.21.15.32:0/1908651792 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f6272ffd640 1 -- 172.21.15.32:0/1908651792 >> v1:172.21.15.39:6789/0 conn(0x7f627415abb0 legacy=0x7f627415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f6272ffd640 1 -- 172.21.15.32:0/1908651792 >> v1:172.21.15.32:6789/0 conn(0x7f6274070230 legacy=0x7f627406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f6272ffd640 1 -- 172.21.15.32:0/1908651792 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f627406dba0 con 0x7f62740b0cb0 2023-12-20T04:28:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f6272ffd640 1 --2- 172.21.15.32:0/1908651792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62740b0cb0 0x7f62740caaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62680097c0 tx=0x7f626801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=98422fe5e1a6f235 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f62727fc640 1 -- 172.21.15.32:0/1908651792 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62680ab680 con 0x7f62740b0cb0 2023-12-20T04:28:35.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f62727fc640 1 -- 172.21.15.32:0/1908651792 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62680ab7e0 con 0x7f62740b0cb0 2023-12-20T04:28:35.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.124+0000 7f62727fc640 1 -- 172.21.15.32:0/1908651792 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62680abb70 con 0x7f62740b0cb0 2023-12-20T04:28:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 -- 172.21.15.32:0/1908651792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62740b0cb0 msgr2=0x7f62740caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 --2- 172.21.15.32:0/1908651792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62740b0cb0 0x7f62740caaf0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f62680097c0 tx=0x7f626801c480 comp rx=0 tx=0).stop 2023-12-20T04:28:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 -- 172.21.15.32:0/1908651792 shutdown_connections 2023-12-20T04:28:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 --2- 172.21.15.32:0/1908651792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62740b0cb0 0x7f62740caaf0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 -- 172.21.15.32:0/1908651792 >> 172.21.15.32:0/1908651792 conn(0x7f627406a510 msgr2=0x7f62740ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 -- 172.21.15.32:0/1908651792 shutdown_connections 2023-12-20T04:28:35.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 -- 172.21.15.32:0/1908651792 wait complete. 2023-12-20T04:28:35.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 Processor -- start 2023-12-20T04:28:35.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 -- start start 2023-12-20T04:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6274070230 0x7f62740c76c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62740b0cb0 0x7f62740c6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f627415abb0 0x7f627414e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62740c58b0 con 0x7f627415abb0 2023-12-20T04:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62740c5a90 con 0x7f6274070230 2023-12-20T04:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6279d39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62740c5d70 con 0x7f62740b0cb0 2023-12-20T04:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6273fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f627415abb0 0x7f627414e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6273fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f627415abb0 0x7f627414e820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55914/0 (socket says 172.21.15.32:55914) 2023-12-20T04:28:35.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6273fff640 1 -- 172.21.15.32:0/4215618393 learned_addr learned my addr 172.21.15.32:0/4215618393 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f62737fe640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6274070230 0x7f62740c76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.128+0000 7f6272ffd640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62740b0cb0 0x7f62740c6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f6273fff640 1 -- 172.21.15.32:0/4215618393 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62740b0cb0 msgr2=0x7f62740c6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f6273fff640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62740b0cb0 0x7f62740c6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f6273fff640 1 -- 172.21.15.32:0/4215618393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6274070230 msgr2=0x7f62740c76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f6273fff640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6274070230 0x7f62740c76c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f6273fff640 1 -- 172.21.15.32:0/4215618393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6268009420 con 0x7f627415abb0 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f6273fff640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f627415abb0 0x7f627414e820 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f626c00cbc0 tx=0x7f626c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f625bfff640 1 -- 172.21.15.32:0/4215618393 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c00ad40 con 0x7f627415abb0 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f625bfff640 1 -- 172.21.15.32:0/4215618393 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f626c012400 con 0x7f627415abb0 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f6279d39640 1 -- 172.21.15.32:0/4215618393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62740c6050 con 0x7f627415abb0 2023-12-20T04:28:35.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f625bfff640 1 -- 172.21.15.32:0/4215618393 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f626c00aea0 con 0x7f627415abb0 2023-12-20T04:28:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f6279d39640 1 -- 172.21.15.32:0/4215618393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f627414f2c0 con 0x7f627415abb0 2023-12-20T04:28:35.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f6279d39640 1 -- 172.21.15.32:0/4215618393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f627404e000 con 0x7f627415abb0 2023-12-20T04:28:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f625bfff640 1 -- 172.21.15.32:0/4215618393 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f626c007070 con 0x7f627415abb0 2023-12-20T04:28:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.132+0000 7f625bfff640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6260065a90 0x7f6260067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.136+0000 7f625bfff640 1 -- 172.21.15.32:0/4215618393 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f626c082d10 con 0x7f627415abb0 2023-12-20T04:28:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.136+0000 7f62737fe640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6260065a90 0x7f6260067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.136+0000 7f62737fe640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6260065a90 0x7f6260067f50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62740b05b0 tx=0x7f6264009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:35.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.136+0000 7f625bfff640 1 -- 172.21.15.32:0/4215618393 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f626c088020 con 0x7f627415abb0 2023-12-20T04:28:35.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.300+0000 7f6279d39640 1 -- 172.21.15.32:0/4215618393 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f62740c6500 con 0x7f627415abb0 2023-12-20T04:28:35.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.300+0000 7f625bfff640 1 -- 172.21.15.32:0/4215618393 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v17) v1 ==== 93+0+10 (secure 0 0 0) 0x7f626c017020 con 0x7f627415abb0 2023-12-20T04:28:35.306 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:28:35.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 -- 172.21.15.32:0/4215618393 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6260065a90 msgr2=0x7f6260067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6260065a90 0x7f6260067f50 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f62740b05b0 tx=0x7f6264009040 comp rx=0 tx=0).stop 2023-12-20T04:28:35.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 -- 172.21.15.32:0/4215618393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f627415abb0 msgr2=0x7f627414e820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f627415abb0 0x7f627414e820 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f626c00cbc0 tx=0x7f626c00a430 comp rx=0 tx=0).stop 2023-12-20T04:28:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 -- 172.21.15.32:0/4215618393 shutdown_connections 2023-12-20T04:28:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6260065a90 0x7f6260067f50 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f627415abb0 0x7f627414e820 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62740b0cb0 0x7f62740c6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 --2- 172.21.15.32:0/4215618393 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6274070230 0x7f62740c76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 -- 172.21.15.32:0/4215618393 >> 172.21.15.32:0/4215618393 conn(0x7f627406a510 msgr2=0x7f627415bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 -- 172.21.15.32:0/4215618393 shutdown_connections 2023-12-20T04:28:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.304+0000 7f6279d39640 1 -- 172.21.15.32:0/4215618393 wait complete. 2023-12-20T04:28:35.327 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T04:28:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.464+0000 7fe757c95640 1 Processor -- start 2023-12-20T04:28:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe757c95640 1 -- start start 2023-12-20T04:28:35.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe757c95640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7500af5e0 0x7fe7500af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe757c95640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe75006ce60 0x7fe75015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe757c95640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe75015bbd0 con 0x7fe75015f110 2023-12-20T04:28:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe757c95640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe75015beb0 con 0x7fe75006ce60 2023-12-20T04:28:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe757c95640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75015e910 con 0x7fe7500af5e0 2023-12-20T04:28:35.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe756232640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe75015f110 0x7fe75015c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56448/0 (socket says 172.21.15.32:56448) 2023-12-20T04:28:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe756232640 1 -- 172.21.15.32:0/206708937 learned_addr learned my addr 172.21.15.32:0/206708937 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe755a31640 1 --2- 172.21.15.32:0/206708937 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7500af5e0 0x7fe7500af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe755230640 1 --2- 172.21.15.32:0/206708937 >> v2:172.21.15.39:3300/0 conn(0x7fe75006ce60 0x7fe75015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe754a2f640 1 -- 172.21.15.32:0/206708937 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 647576689 0 0) 0x7fe75015bbd0 con 0x7fe75015f110 2023-12-20T04:28:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe754a2f640 1 -- 172.21.15.32:0/206708937 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe73c003530 con 0x7fe75015f110 2023-12-20T04:28:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe755230640 1 -- 172.21.15.32:0/206708937 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7500af5e0 msgr2=0x7fe7500af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:35.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe755230640 1 --2- 172.21.15.32:0/206708937 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7500af5e0 0x7fe7500af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe755230640 1 -- 172.21.15.32:0/206708937 >> v1:172.21.15.32:6789/0 conn(0x7fe75015f110 legacy=0x7fe75015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe754a2f640 1 -- 172.21.15.32:0/206708937 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2234930452 0 0) 0x7fe73c003530 con 0x7fe75015f110 2023-12-20T04:28:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe755230640 1 -- 172.21.15.32:0/206708937 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe75015ebf0 con 0x7fe75006ce60 2023-12-20T04:28:35.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.468+0000 7fe755a31640 1 --2- 172.21.15.32:0/206708937 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7500af5e0 0x7fe7500af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:35.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe755230640 1 --2- 172.21.15.32:0/206708937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75006ce60 0x7fe75015b280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7400097c0 tx=0x7fe74001c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2d2df9d89624b3b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe754a2f640 1 -- 172.21.15.32:0/206708937 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe748001a00 con 0x7fe75006ce60 2023-12-20T04:28:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe754a2f640 1 -- 172.21.15.32:0/206708937 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7400a7050 con 0x7fe75006ce60 2023-12-20T04:28:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe754a2f640 1 -- 172.21.15.32:0/206708937 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7400ab620 con 0x7fe75006ce60 2023-12-20T04:28:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe757c95640 1 -- 172.21.15.32:0/206708937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75006ce60 msgr2=0x7fe75015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe757c95640 1 --2- 172.21.15.32:0/206708937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75006ce60 0x7fe75015b280 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe7400097c0 tx=0x7fe74001c3a0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe757c95640 1 -- 172.21.15.32:0/206708937 shutdown_connections 2023-12-20T04:28:35.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe757c95640 1 --2- 172.21.15.32:0/206708937 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75006ce60 0x7fe75015b280 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe757c95640 1 --2- 172.21.15.32:0/206708937 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7500af5e0 0x7fe7500af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe757c95640 1 -- 172.21.15.32:0/206708937 >> 172.21.15.32:0/206708937 conn(0x7fe75006a320 msgr2=0x7fe7500ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe757c95640 1 -- 172.21.15.32:0/206708937 shutdown_connections 2023-12-20T04:28:35.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.472+0000 7fe757c95640 1 -- 172.21.15.32:0/206708937 wait complete. 2023-12-20T04:28:35.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe757c95640 1 Processor -- start 2023-12-20T04:28:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe757c95640 1 -- start start 2023-12-20T04:28:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe757c95640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe75006ce60 0x7fe7500c3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe757c95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7500af5e0 0x7fe750158f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe757c95640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75015f110 0x7fe750070b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe757c95640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7500cc660 con 0x7fe7500af5e0 2023-12-20T04:28:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe757c95640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7500c5fd0 con 0x7fe75015f110 2023-12-20T04:28:35.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe757c95640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe75015a680 con 0x7fe75006ce60 2023-12-20T04:28:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe755230640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7500af5e0 0x7fe750158f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe755230640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7500af5e0 0x7fe750158f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55922/0 (socket says 172.21.15.32:55922) 2023-12-20T04:28:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe755230640 1 -- 172.21.15.32:0/401305650 learned_addr learned my addr 172.21.15.32:0/401305650 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe755a31640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe75006ce60 0x7fe7500c3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe756232640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75015f110 0x7fe750070b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe755230640 1 -- 172.21.15.32:0/401305650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe75006ce60 msgr2=0x7fe7500c3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe755230640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe75006ce60 0x7fe7500c3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe755230640 1 -- 172.21.15.32:0/401305650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75015f110 msgr2=0x7fe750070b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe755230640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75015f110 0x7fe750070b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe755230640 1 -- 172.21.15.32:0/401305650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe74c0093b0 con 0x7fe7500af5e0 2023-12-20T04:28:35.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe755230640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7500af5e0 0x7fe750158f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe74001cc40 tx=0x7fe740019140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:35.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe7467fc640 1 -- 172.21.15.32:0/401305650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7400b4430 con 0x7fe7500af5e0 2023-12-20T04:28:35.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe7467fc640 1 -- 172.21.15.32:0/401305650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7400b45b0 con 0x7fe7500af5e0 2023-12-20T04:28:35.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe7467fc640 1 -- 172.21.15.32:0/401305650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7400b4960 con 0x7fe7500af5e0 2023-12-20T04:28:35.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe757c95640 1 -- 172.21.15.32:0/401305650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe740009420 con 0x7fe7500af5e0 2023-12-20T04:28:35.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe757c95640 1 -- 172.21.15.32:0/401305650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe75015ac40 con 0x7fe7500af5e0 2023-12-20T04:28:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.476+0000 7fe757c95640 1 -- 172.21.15.32:0/401305650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe75015ae30 con 0x7fe7500af5e0 2023-12-20T04:28:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.480+0000 7fe7467fc640 1 -- 172.21.15.32:0/401305650 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe7400b2040 con 0x7fe7500af5e0 2023-12-20T04:28:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.480+0000 7fe7467fc640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe734065a60 0x7fe734067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.480+0000 7fe7467fc640 1 -- 172.21.15.32:0/401305650 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe7400a4e40 con 0x7fe7500af5e0 2023-12-20T04:28:35.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.480+0000 7fe755a31640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe734065a60 0x7fe734067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.480+0000 7fe755a31640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe734065a60 0x7fe734067f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009cd0 tx=0x7fe74c006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:35.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.484+0000 7fe7467fc640 1 -- 172.21.15.32:0/401305650 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe7400ea940 con 0x7fe7500af5e0 2023-12-20T04:28:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 -- 172.21.15.32:0/401305650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fe75004e000 con 0x7fe7500af5e0 2023-12-20T04:28:35.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe7467fc640 1 -- 172.21.15.32:0/401305650 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v17) v1 ==== 84+0+65 (secure 0 0 0) 0x7fe7400f4070 con 0x7fe7500af5e0 2023-12-20T04:28:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:35.649 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:28:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:28:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T04:28:35.649 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T04:28:35.650 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:28:35.650 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:28:35.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 -- 172.21.15.32:0/401305650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe734065a60 msgr2=0x7fe734067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe734065a60 0x7fe734067f20 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fe74c009cd0 tx=0x7fe74c006f90 comp rx=0 tx=0).stop 2023-12-20T04:28:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 -- 172.21.15.32:0/401305650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7500af5e0 msgr2=0x7fe750158f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7500af5e0 0x7fe750158f10 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe74001cc40 tx=0x7fe740019140 comp rx=0 tx=0).stop 2023-12-20T04:28:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 -- 172.21.15.32:0/401305650 shutdown_connections 2023-12-20T04:28:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe734065a60 0x7fe734067f20 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe75015f110 0x7fe750070b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7500af5e0 0x7fe750158f10 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 --2- 172.21.15.32:0/401305650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe75006ce60 0x7fe7500c3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 -- 172.21.15.32:0/401305650 >> 172.21.15.32:0/401305650 conn(0x7fe75006a320 msgr2=0x7fe7500b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:35.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 -- 172.21.15.32:0/401305650 shutdown_connections 2023-12-20T04:28:35.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.644+0000 7fe757c95640 1 -- 172.21.15.32:0/401305650 wait complete. 2023-12-20T04:28:35.670 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T04:28:35.670 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T04:28:35.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.812+0000 7ffa070e5640 1 Processor -- start 2023-12-20T04:28:35.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.812+0000 7ffa070e5640 1 -- start start 2023-12-20T04:28:35.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa070e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ffa000b1930 0x7ffa000b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa070e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa000b0d30 0x7ffa000cd700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa070e5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa0006dd50 con 0x7ffa00159af0 2023-12-20T04:28:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa070e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa000b1da0 con 0x7ffa000b0d30 2023-12-20T04:28:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa070e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa000b1140 con 0x7ffa000b1930 2023-12-20T04:28:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa05682640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa000b0d30 0x7ffa000cd700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa05682640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffa000b0d30 0x7ffa000cd700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55932/0 (socket says 172.21.15.32:55932) 2023-12-20T04:28:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa05682640 1 -- 172.21.15.32:0/3833154000 learned_addr learned my addr 172.21.15.32:0/3833154000 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ff9fffff640 1 --2- 172.21.15.32:0/3833154000 >> v2:172.21.15.39:3300/0 conn(0x7ffa000b1930 0x7ffa000b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa05682640 1 -- 172.21.15.32:0/3833154000 >> v2:172.21.15.39:3300/0 conn(0x7ffa000b1930 msgr2=0x7ffa000b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa05682640 1 --2- 172.21.15.32:0/3833154000 >> v2:172.21.15.39:3300/0 conn(0x7ffa000b1930 0x7ffa000b07f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa05682640 1 -- 172.21.15.32:0/3833154000 >> v1:172.21.15.39:6789/0 conn(0x7ffa00159af0 legacy=0x7ffa0006d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa05682640 1 -- 172.21.15.32:0/3833154000 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa000cc940 con 0x7ffa000b0d30 2023-12-20T04:28:35.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.816+0000 7ffa05682640 1 --2- 172.21.15.32:0/3833154000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa000b0d30 0x7ffa000cd700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa000cf510 tx=0x7ff9f801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4af8b687714d6d2b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ff9ff7fe640 1 -- 172.21.15.32:0/3833154000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f800b2d0 con 0x7ffa000b0d30 2023-12-20T04:28:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ff9ff7fe640 1 -- 172.21.15.32:0/3833154000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f800b430 con 0x7ffa000b0d30 2023-12-20T04:28:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ff9ff7fe640 1 -- 172.21.15.32:0/3833154000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80a7c90 con 0x7ffa000b0d30 2023-12-20T04:28:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 -- 172.21.15.32:0/3833154000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa000b0d30 msgr2=0x7ffa000cd700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 --2- 172.21.15.32:0/3833154000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa000b0d30 0x7ffa000cd700 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ffa000cf510 tx=0x7ff9f801eba0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 -- 172.21.15.32:0/3833154000 shutdown_connections 2023-12-20T04:28:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 --2- 172.21.15.32:0/3833154000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa000b0d30 0x7ffa000cd700 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 --2- 172.21.15.32:0/3833154000 >> v2:172.21.15.39:3300/0 conn(0x7ffa000b1930 0x7ffa000b07f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 -- 172.21.15.32:0/3833154000 >> 172.21.15.32:0/3833154000 conn(0x7ffa0006a510 msgr2=0x7ffa000ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 -- 172.21.15.32:0/3833154000 shutdown_connections 2023-12-20T04:28:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 -- 172.21.15.32:0/3833154000 wait complete. 2023-12-20T04:28:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 Processor -- start 2023-12-20T04:28:35.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 -- start start 2023-12-20T04:28:35.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa000b0d30 0x7ffa0014a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa000b1930 0x7ffa00153fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa00159af0 0x7ffa000c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa0014a410 con 0x7ffa000b1930 2023-12-20T04:28:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa00149800 con 0x7ffa000b0d30 2023-12-20T04:28:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ffa070e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa00149ae0 con 0x7ffa00159af0 2023-12-20T04:28:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ff9fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa000b1930 0x7ffa00153fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.820+0000 7ff9fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa000b1930 0x7ffa00153fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55938/0 (socket says 172.21.15.32:55938) 2023-12-20T04:28:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fffff640 1 -- 172.21.15.32:0/1320070845 learned_addr learned my addr 172.21.15.32:0/1320070845 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ffa05682640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa00159af0 0x7ffa000c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ffa04e81640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa000b0d30 0x7ffa0014a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fffff640 1 -- 172.21.15.32:0/1320070845 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa00159af0 msgr2=0x7ffa000c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fffff640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa00159af0 0x7ffa000c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fffff640 1 -- 172.21.15.32:0/1320070845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa000b0d30 msgr2=0x7ffa0014a8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fffff640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa000b0d30 0x7ffa0014a8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:35.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fffff640 1 -- 172.21.15.32:0/1320070845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9f800bd40 con 0x7ffa000b1930 2023-12-20T04:28:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fffff640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa000b1930 0x7ffa00153fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000d800 tx=0x7ff9f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fd7fa640 1 -- 172.21.15.32:0/1320070845 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000b6c0 con 0x7ffa000b1930 2023-12-20T04:28:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fd7fa640 1 -- 172.21.15.32:0/1320070845 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f000b860 con 0x7ffa000b1930 2023-12-20T04:28:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ffa070e5640 1 -- 172.21.15.32:0/1320070845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa00149dc0 con 0x7ffa000b1930 2023-12-20T04:28:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fd7fa640 1 -- 172.21.15.32:0/1320070845 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f000bc10 con 0x7ffa000b1930 2023-12-20T04:28:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ffa070e5640 1 -- 172.21.15.32:0/1320070845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa00154520 con 0x7ffa000b1930 2023-12-20T04:28:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ffa070e5640 1 -- 172.21.15.32:0/1320070845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0014a580 con 0x7ffa000b1930 2023-12-20T04:28:35.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fd7fa640 1 -- 172.21.15.32:0/1320070845 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff9f001e460 con 0x7ffa000b1930 2023-12-20T04:28:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.824+0000 7ff9fd7fa640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9ec065a90 0x7ff9ec067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.828+0000 7ff9fd7fa640 1 -- 172.21.15.32:0/1320070845 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(17..17 src has 1..17) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff9f00832f0 con 0x7ffa000b1930 2023-12-20T04:28:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.828+0000 7ffa04e81640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9ec065a90 0x7ff9ec067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.828+0000 7ffa04e81640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9ec065a90 0x7ff9ec067f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4007fd0 tx=0x7ff9f4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:35.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.832+0000 7ff9fd7fa640 1 -- 172.21.15.32:0/1320070845 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff9f00510a0 con 0x7ffa000b1930 2023-12-20T04:28:35.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:35.988+0000 7ffa070e5640 1 -- 172.21.15.32:0/1320070845 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ffa0004e000 con 0x7ffa000b1930 2023-12-20T04:28:36.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.948+0000 7ff9fd7fa640 1 -- 172.21.15.32:0/1320070845 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v18) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff9f00546c0 con 0x7ffa000b1930 2023-12-20T04:28:36.955 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T04:28:36.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.952+0000 7ffa070e5640 1 -- 172.21.15.32:0/1320070845 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9ec065a90 msgr2=0x7ff9ec067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:36.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.952+0000 7ffa070e5640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9ec065a90 0x7ff9ec067f50 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7ff9f4007fd0 tx=0x7ff9f4009040 comp rx=0 tx=0).stop 2023-12-20T04:28:36.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.952+0000 7ffa070e5640 1 -- 172.21.15.32:0/1320070845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa000b1930 msgr2=0x7ffa00153fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:36.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.952+0000 7ffa070e5640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa000b1930 0x7ffa00153fe0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff9f000d800 tx=0x7ff9f000dcc0 comp rx=0 tx=0).stop 2023-12-20T04:28:36.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.952+0000 7ffa070e5640 1 -- 172.21.15.32:0/1320070845 shutdown_connections 2023-12-20T04:28:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.952+0000 7ffa070e5640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9ec065a90 0x7ff9ec067f50 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.952+0000 7ffa070e5640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa00159af0 0x7ffa000c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.952+0000 7ffa070e5640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa000b1930 0x7ffa00153fe0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.952+0000 7ffa070e5640 1 --2- 172.21.15.32:0/1320070845 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa000b0d30 0x7ffa0014a8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.952+0000 7ffa070e5640 1 -- 172.21.15.32:0/1320070845 >> 172.21.15.32:0/1320070845 conn(0x7ffa0006a510 msgr2=0x7ffa00070130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:36.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.956+0000 7ffa070e5640 1 -- 172.21.15.32:0/1320070845 shutdown_connections 2023-12-20T04:28:36.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:36.956+0000 7ffa070e5640 1 -- 172.21.15.32:0/1320070845 wait complete. 2023-12-20T04:28:36.987 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T04:28:37.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f14411bc640 1 Processor -- start 2023-12-20T04:28:37.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f14411bc640 1 -- start start 2023-12-20T04:28:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f14411bc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f143c159c60 0x7f143c0cbce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f14411bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f143c0b5330 0x7f143c0cc640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f14411bc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f143c0ccfb0 con 0x7f143c0ccb80 2023-12-20T04:28:37.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f14411bc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143c0aeff0 con 0x7f143c159c60 2023-12-20T04:28:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f14411bc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f143c0c33b0 con 0x7f143c0b5330 2023-12-20T04:28:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f143ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f143c159c60 0x7f143c0cbce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f143b59e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f143c0ccb80 0x7f143c0b26b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56460/0 (socket says 172.21.15.32:56460) 2023-12-20T04:28:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f143b59e640 1 -- 172.21.15.32:0/4055951997 learned_addr learned my addr 172.21.15.32:0/4055951997 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:37.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f143a59c640 1 --2- 172.21.15.32:0/4055951997 >> v2:172.21.15.39:3300/0 conn(0x7f143c0b5330 0x7f143c0cc640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:37.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f143ad9d640 1 -- 172.21.15.32:0/4055951997 >> v2:172.21.15.39:3300/0 conn(0x7f143c0b5330 msgr2=0x7f143c0cc640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f143ad9d640 1 --2- 172.21.15.32:0/4055951997 >> v2:172.21.15.39:3300/0 conn(0x7f143c0b5330 0x7f143c0cc640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f143ad9d640 1 -- 172.21.15.32:0/4055951997 >> v1:172.21.15.32:6789/0 conn(0x7f143c0ccb80 legacy=0x7f143c0b26b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f143ad9d640 1 -- 172.21.15.32:0/4055951997 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f143c0ae750 con 0x7f143c159c60 2023-12-20T04:28:37.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.140+0000 7f1439d9b640 1 -- 172.21.15.32:0/4055951997 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2174695411 0 0) 0x7f143c0ccfb0 con 0x7f143c0ccb80 2023-12-20T04:28:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f143ad9d640 1 --2- 172.21.15.32:0/4055951997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c159c60 0x7f143c0cbce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f143c0cac60 tx=0x7f142c01c340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7cba7a9f7add292 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f1439d9b640 1 -- 172.21.15.32:0/4055951997 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0ab600 con 0x7f143c159c60 2023-12-20T04:28:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f1439d9b640 1 -- 172.21.15.32:0/4055951997 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f142c0a7050 con 0x7f143c159c60 2023-12-20T04:28:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f1439d9b640 1 -- 172.21.15.32:0/4055951997 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f142c0ab980 con 0x7f143c159c60 2023-12-20T04:28:37.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f14411bc640 1 -- 172.21.15.32:0/4055951997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c159c60 msgr2=0x7f143c0cbce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f14411bc640 1 --2- 172.21.15.32:0/4055951997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c159c60 0x7f143c0cbce0 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f143c0cac60 tx=0x7f142c01c340 comp rx=0 tx=0).stop 2023-12-20T04:28:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f14411bc640 1 -- 172.21.15.32:0/4055951997 shutdown_connections 2023-12-20T04:28:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f14411bc640 1 --2- 172.21.15.32:0/4055951997 >> v2:172.21.15.39:3300/0 conn(0x7f143c0b5330 0x7f143c0cc640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:37.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f14411bc640 1 --2- 172.21.15.32:0/4055951997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c159c60 0x7f143c0cbce0 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:37.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f14411bc640 1 -- 172.21.15.32:0/4055951997 >> 172.21.15.32:0/4055951997 conn(0x7f143c06b030 msgr2=0x7f143c06b440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:37.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f14411bc640 1 -- 172.21.15.32:0/4055951997 shutdown_connections 2023-12-20T04:28:37.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.144+0000 7f14411bc640 1 -- 172.21.15.32:0/4055951997 wait complete. 2023-12-20T04:28:37.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14411bc640 1 Processor -- start 2023-12-20T04:28:37.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14411bc640 1 -- start start 2023-12-20T04:28:37.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14411bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f143c0b5330 0x7f143c0cfb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:37.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14411bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f143c0ccb80 0x7f143c152360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:37.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14411bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c159c60 0x7f143c162d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:37.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14411bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c158de0 con 0x7f143c159c60 2023-12-20T04:28:37.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14411bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f143c162350 con 0x7f143c0ccb80 2023-12-20T04:28:37.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14411bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f143c1628a0 con 0x7f143c0b5330 2023-12-20T04:28:37.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f143b59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c159c60 0x7f143c162d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:37.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f143b59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c159c60 0x7f143c162d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55958/0 (socket says 172.21.15.32:55958) 2023-12-20T04:28:37.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f143b59e640 1 -- 172.21.15.32:0/2601754153 learned_addr learned my addr 172.21.15.32:0/2601754153 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:37.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f143a59c640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f143c0ccb80 0x7f143c152360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:37.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f143ad9d640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f143c0b5330 0x7f143c0cfb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:37.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f143b59e640 1 -- 172.21.15.32:0/2601754153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f143c0b5330 msgr2=0x7f143c0cfb90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:37.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f143b59e640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f143c0b5330 0x7f143c0cfb90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:37.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f143b59e640 1 -- 172.21.15.32:0/2601754153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f143c0ccb80 msgr2=0x7f143c152360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:37.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f143b59e640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f143c0ccb80 0x7f143c152360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:37.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f143b59e640 1 -- 172.21.15.32:0/2601754153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f142c009420 con 0x7f143c159c60 2023-12-20T04:28:37.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f143b59e640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c159c60 0x7f143c162d30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f143400c7b0 tx=0x7f143400cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:37.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14237fe640 1 -- 172.21.15.32:0/2601754153 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143400a640 con 0x7f143c159c60 2023-12-20T04:28:37.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14237fe640 1 -- 172.21.15.32:0/2601754153 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f143400a7c0 con 0x7f143c159c60 2023-12-20T04:28:37.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14237fe640 1 -- 172.21.15.32:0/2601754153 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f143400ab70 con 0x7f143c159c60 2023-12-20T04:28:37.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14411bc640 1 -- 172.21.15.32:0/2601754153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f143c158fc0 con 0x7f143c159c60 2023-12-20T04:28:37.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.148+0000 7f14411bc640 1 -- 172.21.15.32:0/2601754153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f143c162530 con 0x7f143c159c60 2023-12-20T04:28:37.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.152+0000 7f14411bc640 1 -- 172.21.15.32:0/2601754153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f143c162a30 con 0x7f143c159c60 2023-12-20T04:28:37.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.152+0000 7f14237fe640 1 -- 172.21.15.32:0/2601754153 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f143400acd0 con 0x7f143c159c60 2023-12-20T04:28:37.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.152+0000 7f14237fe640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065a60 0x7f1428067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:37.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.152+0000 7f14237fe640 1 -- 172.21.15.32:0/2601754153 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(18..18 src has 1..18) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f14340826f0 con 0x7f143c159c60 2023-12-20T04:28:37.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.152+0000 7f143ad9d640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065a60 0x7f1428067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:37.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.152+0000 7f143ad9d640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065a60 0x7f1428067f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f142c0ad000 tx=0x7f142c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:37.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.156+0000 7f14237fe640 1 -- 172.21.15.32:0/2601754153 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1434050320 con 0x7f143c159c60 2023-12-20T04:28:37.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.312+0000 7f14411bc640 1 -- 172.21.15.32:0/2601754153 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f143c1594e0 con 0x7f143c159c60 2023-12-20T04:28:37.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.952+0000 7f14237fe640 1 -- 172.21.15.32:0/2601754153 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v19) v1 ==== 170+0+0 (secure 0 0 0) 0x7f14340180d0 con 0x7f143c159c60 2023-12-20T04:28:37.962 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T04:28:37.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 -- 172.21.15.32:0/2601754153 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065a60 msgr2=0x7f1428067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:37.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065a60 0x7f1428067f20 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f142c0ad000 tx=0x7f142c0a6000 comp rx=0 tx=0).stop 2023-12-20T04:28:37.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 -- 172.21.15.32:0/2601754153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c159c60 msgr2=0x7f143c162d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:37.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c159c60 0x7f143c162d30 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f143400c7b0 tx=0x7f143400cc70 comp rx=0 tx=0).stop 2023-12-20T04:28:37.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 -- 172.21.15.32:0/2601754153 shutdown_connections 2023-12-20T04:28:37.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065a60 0x7f1428067f20 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:37.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f143c159c60 0x7f143c162d30 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:37.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f143c0ccb80 0x7f143c152360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:37.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 --2- 172.21.15.32:0/2601754153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f143c0b5330 0x7f143c0cfb90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:37.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 -- 172.21.15.32:0/2601754153 >> 172.21.15.32:0/2601754153 conn(0x7f143c06b030 msgr2=0x7f143c06a630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:37.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 -- 172.21.15.32:0/2601754153 shutdown_connections 2023-12-20T04:28:37.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:37.960+0000 7f14411bc640 1 -- 172.21.15.32:0/2601754153 wait complete. 2023-12-20T04:28:38.003 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T04:28:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8ed128c640 1 Processor -- start 2023-12-20T04:28:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8ed128c640 1 -- start start 2023-12-20T04:28:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8ed128c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8ecc06bd50 0x7f8ecc0cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8ed128c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8ecc0cf5d0 0x7f8ecc0b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8ed128c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ecc0b0550 con 0x7f8ecc15ce50 2023-12-20T04:28:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8ed128c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ecc0aff20 con 0x7f8ecc06bd50 2023-12-20T04:28:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8ed128c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ecc0cfdc0 con 0x7f8ecc0cf5d0 2023-12-20T04:28:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8eca7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8ecc06bd50 0x7f8ecc0cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8ecaffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8ecc15ce50 0x7f8ecc06b640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56466/0 (socket says 172.21.15.32:56466) 2023-12-20T04:28:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8ecaffd640 1 -- 172.21.15.32:0/940609432 learned_addr learned my addr 172.21.15.32:0/940609432 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8eca7fc640 1 -- 172.21.15.32:0/940609432 >> v2:172.21.15.39:3300/0 conn(0x7f8ecc0cf5d0 msgr2=0x7f8ecc0b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8eca7fc640 1 --2- 172.21.15.32:0/940609432 >> v2:172.21.15.39:3300/0 conn(0x7f8ecc0cf5d0 0x7f8ecc0b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8eca7fc640 1 -- 172.21.15.32:0/940609432 >> v1:172.21.15.32:6789/0 conn(0x7f8ecc15ce50 legacy=0x7f8ecc06b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8eca7fc640 1 -- 172.21.15.32:0/940609432 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ecc0d00a0 con 0x7f8ecc06bd50 2023-12-20T04:28:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.148+0000 7f8ec9ffb640 1 -- 172.21.15.32:0/940609432 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2924364119 0 0) 0x7f8ecc0b0550 con 0x7f8ecc15ce50 2023-12-20T04:28:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8eca7fc640 1 --2- 172.21.15.32:0/940609432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ecc06bd50 0x7f8ecc0cf090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc06cff0 tx=0x7f8ec001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3be58963c5d1282 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8ec9ffb640 1 -- 172.21.15.32:0/940609432 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00ab650 con 0x7f8ecc06bd50 2023-12-20T04:28:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8ec9ffb640 1 -- 172.21.15.32:0/940609432 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ec00ab7b0 con 0x7f8ecc06bd50 2023-12-20T04:28:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8ec9ffb640 1 -- 172.21.15.32:0/940609432 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ec00abb40 con 0x7f8ecc06bd50 2023-12-20T04:28:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8ed128c640 1 -- 172.21.15.32:0/940609432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ecc06bd50 msgr2=0x7f8ecc0cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8ed128c640 1 --2- 172.21.15.32:0/940609432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ecc06bd50 0x7f8ecc0cf090 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc06cff0 tx=0x7f8ec001c430 comp rx=0 tx=0).stop 2023-12-20T04:28:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8ed128c640 1 -- 172.21.15.32:0/940609432 shutdown_connections 2023-12-20T04:28:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8ed128c640 1 --2- 172.21.15.32:0/940609432 >> v2:172.21.15.39:3300/0 conn(0x7f8ecc0cf5d0 0x7f8ecc0b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8ed128c640 1 --2- 172.21.15.32:0/940609432 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ecc06bd50 0x7f8ecc0cf090 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8ed128c640 1 -- 172.21.15.32:0/940609432 >> 172.21.15.32:0/940609432 conn(0x7f8ecc099d70 msgr2=0x7f8ecc0d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8ed128c640 1 -- 172.21.15.32:0/940609432 shutdown_connections 2023-12-20T04:28:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.152+0000 7f8ed128c640 1 -- 172.21.15.32:0/940609432 wait complete. 2023-12-20T04:28:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ed128c640 1 Processor -- start 2023-12-20T04:28:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ed128c640 1 -- start start 2023-12-20T04:28:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ed128c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ecc06bd50 0x7f8ecc0b6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ed128c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ecc0cf5d0 0x7f8ecc0c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ed128c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ecc15ce50 0x7f8ecc0c80b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ed128c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc150c30 con 0x7f8ecc06bd50 2023-12-20T04:28:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ed128c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc153500 con 0x7f8ecc0cf5d0 2023-12-20T04:28:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ed128c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ecc15a700 con 0x7f8ecc15ce50 2023-12-20T04:28:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ecaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ecc06bd50 0x7f8ecc0b6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ecaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ecc06bd50 0x7f8ecc0b6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55974/0 (socket says 172.21.15.32:55974) 2023-12-20T04:28:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ecaffd640 1 -- 172.21.15.32:0/794556012 learned_addr learned my addr 172.21.15.32:0/794556012 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8eca7fc640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ecc0cf5d0 0x7f8ecc0c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ecb7fe640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ecc15ce50 0x7f8ecc0c80b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ecaffd640 1 -- 172.21.15.32:0/794556012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ecc15ce50 msgr2=0x7f8ecc0c80b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ecaffd640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ecc15ce50 0x7f8ecc0c80b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ecaffd640 1 -- 172.21.15.32:0/794556012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ecc0cf5d0 msgr2=0x7f8ecc0c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ecaffd640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ecc0cf5d0 0x7f8ecc0c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ecaffd640 1 -- 172.21.15.32:0/794556012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec0009420 con 0x7f8ecc06bd50 2023-12-20T04:28:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ecb7fe640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ecc15ce50 0x7f8ecc0c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:28:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8eca7fc640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ecc0cf5d0 0x7f8ecc0c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ecaffd640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ecc06bd50 0x7f8ecc0b6360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c480 tx=0x7f8ebc00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8eb37fe640 1 -- 172.21.15.32:0/794556012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc0074e0 con 0x7f8ecc06bd50 2023-12-20T04:28:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8eb37fe640 1 -- 172.21.15.32:0/794556012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ebc007660 con 0x7f8ecc06bd50 2023-12-20T04:28:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8eb37fe640 1 -- 172.21.15.32:0/794556012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ebc004a30 con 0x7f8ecc06bd50 2023-12-20T04:28:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ed128c640 1 -- 172.21.15.32:0/794556012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ecc15a9e0 con 0x7f8ecc06bd50 2023-12-20T04:28:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ed128c640 1 -- 172.21.15.32:0/794556012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ecc15afa0 con 0x7f8ecc06bd50 2023-12-20T04:28:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.156+0000 7f8ed128c640 1 -- 172.21.15.32:0/794556012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ecc04e000 con 0x7f8ecc06bd50 2023-12-20T04:28:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.160+0000 7f8eb37fe640 1 -- 172.21.15.32:0/794556012 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8ebc004b90 con 0x7f8ecc06bd50 2023-12-20T04:28:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.160+0000 7f8eb37fe640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065a60 0x7f8eb8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.160+0000 7f8eb37fe640 1 -- 172.21.15.32:0/794556012 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(19..19 src has 1..19) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f8ebc082350 con 0x7f8ecc06bd50 2023-12-20T04:28:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.160+0000 7f8eca7fc640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065a60 0x7f8eb8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.160+0000 7f8eca7fc640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065a60 0x7f8eb8067f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00af000 tx=0x7f8ec00a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:38.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.164+0000 7f8eb37fe640 1 -- 172.21.15.32:0/794556012 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8ebc053830 con 0x7f8ecc06bd50 2023-12-20T04:28:38.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.356+0000 7f8ed128c640 1 -- 172.21.15.32:0/794556012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f8ecc0c9eb0 con 0x7f8ecc06bd50 2023-12-20T04:28:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.364+0000 7f8eb37fe640 1 -- 172.21.15.32:0/794556012 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v4) v1 ==== 172+0+0 (secure 0 0 0) 0x7f8ebc0533c0 con 0x7f8ecc06bd50 2023-12-20T04:28:38.367 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 2 and data pool 3 2023-12-20T04:28:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.368+0000 7f8ed128c640 1 -- 172.21.15.32:0/794556012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065a60 msgr2=0x7f8eb8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.368+0000 7f8ed128c640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065a60 0x7f8eb8067f20 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f8ec00af000 tx=0x7f8ec00a7000 comp rx=0 tx=0).stop 2023-12-20T04:28:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.368+0000 7f8ed128c640 1 -- 172.21.15.32:0/794556012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ecc06bd50 msgr2=0x7f8ecc0b6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.368+0000 7f8ed128c640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ecc06bd50 0x7f8ecc0b6360 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f8ebc00c480 tx=0x7f8ebc00c940 comp rx=0 tx=0).stop 2023-12-20T04:28:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.372+0000 7f8ed128c640 1 -- 172.21.15.32:0/794556012 shutdown_connections 2023-12-20T04:28:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.372+0000 7f8ed128c640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065a60 0x7f8eb8067f20 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.372+0000 7f8ed128c640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ecc15ce50 0x7f8ecc0c80b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.372+0000 7f8ed128c640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ecc0cf5d0 0x7f8ecc0c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.372+0000 7f8ed128c640 1 --2- 172.21.15.32:0/794556012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ecc06bd50 0x7f8ecc0b6360 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.372+0000 7f8ed128c640 1 -- 172.21.15.32:0/794556012 >> 172.21.15.32:0/794556012 conn(0x7f8ecc099d70 msgr2=0x7f8ecc0cd0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.372+0000 7f8ed128c640 1 -- 172.21.15.32:0/794556012 shutdown_connections 2023-12-20T04:28:38.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.372+0000 7f8ed128c640 1 -- 172.21.15.32:0/794556012 wait complete. 2023-12-20T04:28:38.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:38.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.552+0000 7f1f1cb7b640 1 Processor -- start 2023-12-20T04:28:38.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.552+0000 7f1f1cb7b640 1 -- start start 2023-12-20T04:28:38.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1cb7b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1f1815ce30 0x7f1f18161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1cb7b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f18071920 con 0x7f1f18070950 2023-12-20T04:28:38.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1cb7b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f18071080 con 0x7f1f1806deb0 2023-12-20T04:28:38.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1cb7b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f18071360 con 0x7f1f1815ce30 2023-12-20T04:28:38.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f15d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1f18070950 0x7f1f1806d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56470/0 (socket says 172.21.15.32:56470) 2023-12-20T04:28:38.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f15d9b640 1 -- 172.21.15.32:0/1962045750 learned_addr learned my addr 172.21.15.32:0/1962045750 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1659c640 1 --2- 172.21.15.32:0/1962045750 >> v2:172.21.15.39:3300/0 conn(0x7f1f1815ce30 0x7f1f18161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 404377342 0 0) 0x7f1f18071920 con 0x7f1f18070950 2023-12-20T04:28:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f00003530 con 0x7f1f18070950 2023-12-20T04:28:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2084909264 0 0) 0x7f1f00003530 con 0x7f1f18070950 2023-12-20T04:28:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 >> v2:172.21.15.39:3300/0 conn(0x7f1f1815ce30 msgr2=0x7f1f18161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 --2- 172.21.15.32:0/1962045750 >> v2:172.21.15.39:3300/0 conn(0x7f1f1815ce30 0x7f1f18161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 >> v1:172.21.15.39:6789/0 conn(0x7f1f1806deb0 legacy=0x7f1f180cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f18071640 con 0x7f1f18070950 2023-12-20T04:28:38.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f1f0c001680 con 0x7f1f18070950 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 >> v1:172.21.15.32:6789/0 conn(0x7f1f18070950 legacy=0x7f1f1806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 --2- 172.21.15.32:0/1962045750 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f00003980 0x7f1f00003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 --2- 172.21.15.32:0/1962045750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f00004380 0x7f1f00004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 --2- 172.21.15.32:0/1962045750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f000050f0 0x7f1f0000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18071360 con 0x7f1f000050f0 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0000c030 con 0x7f1f00004380 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f0000c1f0 con 0x7f1f00003980 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f1659c640 1 --2- 172.21.15.32:0/1962045750 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f00003980 0x7f1f00003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f16d9d640 1 --2- 172.21.15.32:0/1962045750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f000050f0 0x7f1f0000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f15d9b640 1 --2- 172.21.15.32:0/1962045750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f00004380 0x7f1f00004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f16d9d640 1 -- 172.21.15.32:0/1962045750 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f00003980 msgr2=0x7f1f00003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f16d9d640 1 --2- 172.21.15.32:0/1962045750 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f00003980 0x7f1f00003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f16d9d640 1 -- 172.21.15.32:0/1962045750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f00004380 msgr2=0x7f1f00004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:38.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f16d9d640 1 --2- 172.21.15.32:0/1962045750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f00004380 0x7f1f00004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.556+0000 7f1f16d9d640 1 -- 172.21.15.32:0/1962045750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f00003530 con 0x7f1f000050f0 2023-12-20T04:28:38.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f16d9d640 1 -- 172.21.15.32:0/1962045750 reap_dead start 2023-12-20T04:28:38.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f16d9d640 1 --2- 172.21.15.32:0/1962045750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f000050f0 0x7f1f0000b990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f10001f60 tx=0x7f1f1001c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c51da37fa53edc09 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:38.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f10003b80 con 0x7f1f000050f0 2023-12-20T04:28:38.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f10003ce0 con 0x7f1f000050f0 2023-12-20T04:28:38.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f1559a640 1 -- 172.21.15.32:0/1962045750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f100052f0 con 0x7f1f000050f0 2023-12-20T04:28:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/1962045750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f000050f0 msgr2=0x7f1f0000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f1cb7b640 1 --2- 172.21.15.32:0/1962045750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f000050f0 0x7f1f0000b990 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f1f10001f60 tx=0x7f1f1001c860 comp rx=0 tx=0).stop 2023-12-20T04:28:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/1962045750 shutdown_connections 2023-12-20T04:28:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f1cb7b640 1 --2- 172.21.15.32:0/1962045750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f000050f0 0x7f1f0000b990 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/1962045750 >> 172.21.15.32:0/1962045750 conn(0x7f1f180cba50 msgr2=0x7f1f180cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/1962045750 shutdown_connections 2023-12-20T04:28:38.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/1962045750 wait complete. 2023-12-20T04:28:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.560+0000 7f1f1cb7b640 1 Processor -- start 2023-12-20T04:28:38.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f1cb7b640 1 -- start start 2023-12-20T04:28:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f1cb7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f000050f0 0x7f1f180b0d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f1cb7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f181f5220 0x7f1f180c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f1cb7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f180c5430 0x7f1f180c3020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f1cb7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f180b12c0 con 0x7f1f181f5220 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f1cb7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18155b00 con 0x7f1f180c5430 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f1cb7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f18156080 con 0x7f1f000050f0 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f15d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f181f5220 0x7f1f180c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f15d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f181f5220 0x7f1f180c2ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55992/0 (socket says 172.21.15.32:55992) 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f15d9b640 1 -- 172.21.15.32:0/2093155066 learned_addr learned my addr 172.21.15.32:0/2093155066 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f16d9d640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f180c5430 0x7f1f180c3020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f1659c640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f000050f0 0x7f1f180b0d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f15d9b640 1 -- 172.21.15.32:0/2093155066 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f000050f0 msgr2=0x7f1f180b0d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f15d9b640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f000050f0 0x7f1f180b0d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f15d9b640 1 -- 172.21.15.32:0/2093155066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f180c5430 msgr2=0x7f1f180c3020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f15d9b640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f180c5430 0x7f1f180c3020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f15d9b640 1 -- 172.21.15.32:0/2093155066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f180c59c0 con 0x7f1f181f5220 2023-12-20T04:28:38.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f15d9b640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f181f5220 0x7f1f180c2ae0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0010f0 tx=0x7f1f0c004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:38.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f06ffd640 1 -- 172.21.15.32:0/2093155066 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00f6e0 con 0x7f1f181f5220 2023-12-20T04:28:38.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f06ffd640 1 -- 172.21.15.32:0/2093155066 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f0c00f880 con 0x7f1f181f5220 2023-12-20T04:28:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f06ffd640 1 -- 172.21.15.32:0/2093155066 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f0c00fc30 con 0x7f1f181f5220 2023-12-20T04:28:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/2093155066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f180c5ca0 con 0x7f1f181f5220 2023-12-20T04:28:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/2093155066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f18156510 con 0x7f1f181f5220 2023-12-20T04:28:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.564+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/2093155066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f180cb8c0 con 0x7f1f181f5220 2023-12-20T04:28:38.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.568+0000 7f1f06ffd640 1 -- 172.21.15.32:0/2093155066 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1f0c00fd90 con 0x7f1f181f5220 2023-12-20T04:28:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.568+0000 7f1f06ffd640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ef8065a60 0x7f1ef8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.568+0000 7f1f06ffd640 1 -- 172.21.15.32:0/2093155066 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1f0c083210 con 0x7f1f181f5220 2023-12-20T04:28:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.568+0000 7f1f1659c640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ef8065a60 0x7f1ef8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.568+0000 7f1f1659c640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ef8065a60 0x7f1ef8067f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009aa0 tx=0x7f1f08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:38.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.572+0000 7f1f06ffd640 1 -- 172.21.15.32:0/2093155066 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1f0c050be0 con 0x7f1f181f5220 2023-12-20T04:28:38.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.732+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/2093155066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1f180c5e90 con 0x7f1f181f5220 2023-12-20T04:28:38.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f06ffd640 1 -- 172.21.15.32:0/2093155066 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16002 (secure 0 0 0) 0x7f1f0c054200 con 0x7f1f181f5220 2023-12-20T04:28:38.739 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:38.739 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:38.361129+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/2093155066 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ef8065a60 msgr2=0x7f1ef8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ef8065a60 0x7f1ef8067f20 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f1f08009aa0 tx=0x7f1f08008040 comp rx=0 tx=0).stop 2023-12-20T04:28:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/2093155066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f181f5220 msgr2=0x7f1f180c2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f181f5220 0x7f1f180c2ae0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1f0c0010f0 tx=0x7f1f0c004f90 comp rx=0 tx=0).stop 2023-12-20T04:28:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/2093155066 shutdown_connections 2023-12-20T04:28:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ef8065a60 0x7f1ef8067f20 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f180c5430 0x7f1f180c3020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f181f5220 0x7f1f180c2ae0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 --2- 172.21.15.32:0/2093155066 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f000050f0 0x7f1f180b0d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/2093155066 >> 172.21.15.32:0/2093155066 conn(0x7f1f180cba50 msgr2=0x7f1f1809fd20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/2093155066 shutdown_connections 2023-12-20T04:28:38.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.736+0000 7f1f1cb7b640 1 -- 172.21.15.32:0/2093155066 wait complete. 2023-12-20T04:28:38.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.896+0000 7fa096efc640 1 Processor -- start 2023-12-20T04:28:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa096efc640 1 -- start start 2023-12-20T04:28:38.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa096efc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa09006deb0 0x7fa090161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa096efc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa090070950 0x7fa0900cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa096efc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa090071920 con 0x7fa09015ce30 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa096efc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa090070de0 con 0x7fa09006deb0 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa096efc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa090071080 con 0x7fa090070950 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa08ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa09006deb0 0x7fa090161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa08ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa09006deb0 0x7fa090161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48040/0 (socket says 172.21.15.32:48040) 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa08ffff640 1 -- 172.21.15.32:0/3919177010 learned_addr learned my addr 172.21.15.32:0/3919177010 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa095499640 1 --2- 172.21.15.32:0/3919177010 >> v2:172.21.15.39:3300/0 conn(0x7fa090070950 0x7fa0900cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa08ffff640 1 -- 172.21.15.32:0/3919177010 >> v2:172.21.15.39:3300/0 conn(0x7fa090070950 msgr2=0x7fa0900cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa08ffff640 1 --2- 172.21.15.32:0/3919177010 >> v2:172.21.15.39:3300/0 conn(0x7fa090070950 0x7fa0900cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa08ffff640 1 -- 172.21.15.32:0/3919177010 >> v1:172.21.15.39:6789/0 conn(0x7fa09015ce30 legacy=0x7fa09006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.900+0000 7fa08ffff640 1 -- 172.21.15.32:0/3919177010 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa090071360 con 0x7fa09006deb0 2023-12-20T04:28:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa08ffff640 1 --2- 172.21.15.32:0/3919177010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09006deb0 0x7fa090161560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa0900c95b0 tx=0x7fa07c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8847531ee4293b4b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa08f7fe640 1 -- 172.21.15.32:0/3919177010 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0ab5f0 con 0x7fa09006deb0 2023-12-20T04:28:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa08f7fe640 1 -- 172.21.15.32:0/3919177010 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07c0ab750 con 0x7fa09006deb0 2023-12-20T04:28:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa08f7fe640 1 -- 172.21.15.32:0/3919177010 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0abae0 con 0x7fa09006deb0 2023-12-20T04:28:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa096efc640 1 -- 172.21.15.32:0/3919177010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09006deb0 msgr2=0x7fa090161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa096efc640 1 --2- 172.21.15.32:0/3919177010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09006deb0 0x7fa090161560 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa0900c95b0 tx=0x7fa07c01c500 comp rx=0 tx=0).stop 2023-12-20T04:28:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa096efc640 1 -- 172.21.15.32:0/3919177010 shutdown_connections 2023-12-20T04:28:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa096efc640 1 --2- 172.21.15.32:0/3919177010 >> v2:172.21.15.39:3300/0 conn(0x7fa090070950 0x7fa0900cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa096efc640 1 --2- 172.21.15.32:0/3919177010 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa09006deb0 0x7fa090161560 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa096efc640 1 -- 172.21.15.32:0/3919177010 >> 172.21.15.32:0/3919177010 conn(0x7fa0900cba50 msgr2=0x7fa0900cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa096efc640 1 -- 172.21.15.32:0/3919177010 shutdown_connections 2023-12-20T04:28:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa096efc640 1 -- 172.21.15.32:0/3919177010 wait complete. 2023-12-20T04:28:38.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa096efc640 1 Processor -- start 2023-12-20T04:28:38.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.904+0000 7fa096efc640 1 -- start start 2023-12-20T04:28:38.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa096efc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa09006deb0 0x7fa0900c20d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa096efc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa090070950 0x7fa0900c32b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa096efc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa09015ce30 0x7fa0900c37f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa096efc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09015c2f0 con 0x7fa090070950 2023-12-20T04:28:38.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa096efc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa09015c710 con 0x7fa09006deb0 2023-12-20T04:28:38.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa096efc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0900c5940 con 0x7fa09015ce30 2023-12-20T04:28:38.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa090070950 0x7fa0900c32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa090070950 0x7fa0900c32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48050/0 (socket says 172.21.15.32:48050) 2023-12-20T04:28:38.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08ffff640 1 -- 172.21.15.32:0/2196252377 learned_addr learned my addr 172.21.15.32:0/2196252377 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:38.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa095499640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa09015ce30 0x7fa0900c37f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa094c98640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa09006deb0 0x7fa0900c20d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08ffff640 1 -- 172.21.15.32:0/2196252377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa09015ce30 msgr2=0x7fa0900c37f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08ffff640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa09015ce30 0x7fa0900c37f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08ffff640 1 -- 172.21.15.32:0/2196252377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa09006deb0 msgr2=0x7fa0900c20d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:38.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08ffff640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa09006deb0 0x7fa0900c20d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:38.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08ffff640 1 -- 172.21.15.32:0/2196252377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa07c0094d0 con 0x7fa090070950 2023-12-20T04:28:38.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08ffff640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa090070950 0x7fa0900c32b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa07c019110 tx=0x7fa07c01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:38.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08d7fa640 1 -- 172.21.15.32:0/2196252377 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0be5f0 con 0x7fa090070950 2023-12-20T04:28:38.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08d7fa640 1 -- 172.21.15.32:0/2196252377 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa07c0be750 con 0x7fa090070950 2023-12-20T04:28:38.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08d7fa640 1 -- 172.21.15.32:0/2196252377 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa07c0beb00 con 0x7fa090070950 2023-12-20T04:28:38.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa096efc640 1 -- 172.21.15.32:0/2196252377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0900c53e0 con 0x7fa090070950 2023-12-20T04:28:38.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa096efc640 1 -- 172.21.15.32:0/2196252377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa0900c5660 con 0x7fa090070950 2023-12-20T04:28:38.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa096efc640 1 -- 172.21.15.32:0/2196252377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa09015c430 con 0x7fa090070950 2023-12-20T04:28:38.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.908+0000 7fa08d7fa640 1 -- 172.21.15.32:0/2196252377 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa07c0b43f0 con 0x7fa090070950 2023-12-20T04:28:38.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.912+0000 7fa08d7fa640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa074065a60 0x7fa074067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:38.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.912+0000 7fa08d7fa640 1 -- 172.21.15.32:0/2196252377 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa07c0a8050 con 0x7fa090070950 2023-12-20T04:28:38.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.912+0000 7fa094c98640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa074065a60 0x7fa074067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:38.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.912+0000 7fa094c98640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa074065a60 0x7fa074067f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0900c51d0 tx=0x7fa084009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:38.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:38.916+0000 7fa08d7fa640 1 -- 172.21.15.32:0/2196252377 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa09015c430 con 0x7fa090070950 2023-12-20T04:28:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 -- 172.21.15.32:0/2196252377 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa09015c430 con 0x7fa090070950 2023-12-20T04:28:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa08d7fa640 1 -- 172.21.15.32:0/2196252377 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v20) v1 ==== 74+0+16002 (secure 0 0 0) 0x7fa09015c430 con 0x7fa090070950 2023-12-20T04:28:39.081 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:39.081 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":20,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:38.361129+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 -- 172.21.15.32:0/2196252377 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa074065a60 msgr2=0x7fa074067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa074065a60 0x7fa074067f20 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa0900c51d0 tx=0x7fa084009040 comp rx=0 tx=0).stop 2023-12-20T04:28:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 -- 172.21.15.32:0/2196252377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa090070950 msgr2=0x7fa0900c32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa090070950 0x7fa0900c32b0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa07c019110 tx=0x7fa07c01ad40 comp rx=0 tx=0).stop 2023-12-20T04:28:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 -- 172.21.15.32:0/2196252377 shutdown_connections 2023-12-20T04:28:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa074065a60 0x7fa074067f20 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:39.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa09015ce30 0x7fa0900c37f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa090070950 0x7fa0900c32b0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 --2- 172.21.15.32:0/2196252377 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa09006deb0 0x7fa0900c20d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 -- 172.21.15.32:0/2196252377 >> 172.21.15.32:0/2196252377 conn(0x7fa0900cba50 msgr2=0x7fa09015ea20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 -- 172.21.15.32:0/2196252377 shutdown_connections 2023-12-20T04:28:39.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.076+0000 7fa096efc640 1 -- 172.21.15.32:0/2196252377 wait complete. 2023-12-20T04:28:39.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238ff09640 1 Processor -- start 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238ff09640 1 -- start start 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238ff09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2388157480 0x7f23880c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238ff09640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f23880b0760 0x7f23880b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238ff09640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23880b1100 con 0x7f238815d660 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238ff09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23880cde20 con 0x7f2388157480 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238ff09640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23880c8e30 con 0x7f23880b0760 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238e4a6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f238815d660 0x7f238815da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35288/0 (socket says 172.21.15.32:35288) 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238e4a6640 1 -- 172.21.15.32:0/3385057287 learned_addr learned my addr 172.21.15.32:0/3385057287 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238dca5640 1 --2- 172.21.15.32:0/3385057287 >> v2:172.21.15.32:3300/0 conn(0x7f2388157480 0x7f23880c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238d4a4640 1 --2- 172.21.15.32:0/3385057287 >> v2:172.21.15.39:3300/0 conn(0x7f23880b0760 0x7f23880b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238dca5640 1 -- 172.21.15.32:0/3385057287 >> v2:172.21.15.39:3300/0 conn(0x7f23880b0760 msgr2=0x7f23880b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238dca5640 1 --2- 172.21.15.32:0/3385057287 >> v2:172.21.15.39:3300/0 conn(0x7f23880b0760 0x7f23880b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238dca5640 1 -- 172.21.15.32:0/3385057287 >> v1:172.21.15.32:6789/0 conn(0x7f238815d660 legacy=0x7f238815da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238dca5640 1 -- 172.21.15.32:0/3385057287 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23880ce070 con 0x7f2388157480 2023-12-20T04:28:39.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.244+0000 7f238cca3640 1 -- 172.21.15.32:0/3385057287 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1248726357 0 0) 0x7f23880b1100 con 0x7f238815d660 2023-12-20T04:28:39.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238dca5640 1 --2- 172.21.15.32:0/3385057287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2388157480 0x7f23880c9840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f238806e0d0 tx=0x7f238401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=278084aaaf0a449a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:39.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238cca3640 1 -- 172.21.15.32:0/3385057287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840ac600 con 0x7f2388157480 2023-12-20T04:28:39.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238cca3640 1 -- 172.21.15.32:0/3385057287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23840a8050 con 0x7f2388157480 2023-12-20T04:28:39.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238cca3640 1 -- 172.21.15.32:0/3385057287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840ac980 con 0x7f2388157480 2023-12-20T04:28:39.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238ff09640 1 -- 172.21.15.32:0/3385057287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2388157480 msgr2=0x7f23880c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:39.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238ff09640 1 --2- 172.21.15.32:0/3385057287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2388157480 0x7f23880c9840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f238806e0d0 tx=0x7f238401c3a0 comp rx=0 tx=0).stop 2023-12-20T04:28:39.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238ff09640 1 -- 172.21.15.32:0/3385057287 shutdown_connections 2023-12-20T04:28:39.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238ff09640 1 --2- 172.21.15.32:0/3385057287 >> v2:172.21.15.39:3300/0 conn(0x7f23880b0760 0x7f23880b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:39.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238ff09640 1 --2- 172.21.15.32:0/3385057287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2388157480 0x7f23880c9840 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:39.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238ff09640 1 -- 172.21.15.32:0/3385057287 >> 172.21.15.32:0/3385057287 conn(0x7f238806a510 msgr2=0x7f23880ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:39.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238ff09640 1 -- 172.21.15.32:0/3385057287 shutdown_connections 2023-12-20T04:28:39.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.248+0000 7f238ff09640 1 -- 172.21.15.32:0/3385057287 wait complete. 2023-12-20T04:28:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238ff09640 1 Processor -- start 2023-12-20T04:28:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238ff09640 1 -- start start 2023-12-20T04:28:39.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238ff09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23880b0760 0x7f238813dda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:39.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238ff09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2388157480 0x7f2388140540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:39.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238ff09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f238815d660 0x7f2388157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:39.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238ff09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f238813ea10 con 0x7f23880b0760 2023-12-20T04:28:39.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238ff09640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2388151940 con 0x7f238815d660 2023-12-20T04:28:39.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238ff09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2388140f90 con 0x7f2388157480 2023-12-20T04:28:39.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238dca5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23880b0760 0x7f238813dda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:39.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238dca5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23880b0760 0x7f238813dda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48072/0 (socket says 172.21.15.32:48072) 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238dca5640 1 -- 172.21.15.32:0/2760405305 learned_addr learned my addr 172.21.15.32:0/2760405305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238d4a4640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2388157480 0x7f2388140540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238e4a6640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f238815d660 0x7f2388157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238dca5640 1 -- 172.21.15.32:0/2760405305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2388157480 msgr2=0x7f2388140540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238dca5640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2388157480 0x7f2388140540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238dca5640 1 -- 172.21.15.32:0/2760405305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f238815d660 msgr2=0x7f2388157860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238dca5640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f238815d660 0x7f2388157860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238dca5640 1 -- 172.21.15.32:0/2760405305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2384009420 con 0x7f23880b0760 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238d4a4640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2388157480 0x7f2388140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238dca5640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23880b0760 0x7f238813dda0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2384018fb0 tx=0x7f238401abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f237e7fc640 1 -- 172.21.15.32:0/2760405305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840b7620 con 0x7f23880b0760 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f237e7fc640 1 -- 172.21.15.32:0/2760405305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23840b7780 con 0x7f23880b0760 2023-12-20T04:28:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f237e7fc640 1 -- 172.21.15.32:0/2760405305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23840b7af0 con 0x7f23880b0760 2023-12-20T04:28:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238ff09640 1 -- 172.21.15.32:0/2760405305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2388141270 con 0x7f23880b0760 2023-12-20T04:28:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.252+0000 7f238ff09640 1 -- 172.21.15.32:0/2760405305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2388162ed0 con 0x7f23880b0760 2023-12-20T04:28:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.256+0000 7f238ff09640 1 -- 172.21.15.32:0/2760405305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f238804e000 con 0x7f23880b0760 2023-12-20T04:28:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.256+0000 7f237e7fc640 1 -- 172.21.15.32:0/2760405305 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f23840b3040 con 0x7f23880b0760 2023-12-20T04:28:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.256+0000 7f237e7fc640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f236c065a60 0x7f236c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.256+0000 7f237e7fc640 1 -- 172.21.15.32:0/2760405305 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(20..20 src has 1..20) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f238411d4a0 con 0x7f23880b0760 2023-12-20T04:28:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.256+0000 7f238d4a4640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f236c065a60 0x7f236c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:39.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.256+0000 7f238d4a4640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f236c065a60 0x7f236c067f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2388144bc0 tx=0x7f2378008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.260+0000 7f237e7fc640 1 -- 172.21.15.32:0/2760405305 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f23840eadc0 con 0x7f23880b0760 2023-12-20T04:28:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:39.452+0000 7f238ff09640 1 -- 172.21.15.32:0/2760405305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f2388154e20 con 0x7f23880b0760 2023-12-20T04:28:40.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.376+0000 7f237e7fc640 1 -- 172.21.15.32:0/2760405305 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v7) v1 ==== 118+0+0 (secure 0 0 0) 0x7f23840ee3e0 con 0x7f23880b0760 2023-12-20T04:28:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 -- 172.21.15.32:0/2760405305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f236c065a60 msgr2=0x7f236c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f236c065a60 0x7f236c067f20 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f2388144bc0 tx=0x7f2378008040 comp rx=0 tx=0).stop 2023-12-20T04:28:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 -- 172.21.15.32:0/2760405305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23880b0760 msgr2=0x7f238813dda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23880b0760 0x7f238813dda0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2384018fb0 tx=0x7f238401abe0 comp rx=0 tx=0).stop 2023-12-20T04:28:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 -- 172.21.15.32:0/2760405305 shutdown_connections 2023-12-20T04:28:40.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f236c065a60 0x7f236c067f20 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f238815d660 0x7f2388157860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2388157480 0x7f2388140540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 --2- 172.21.15.32:0/2760405305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23880b0760 0x7f238813dda0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 -- 172.21.15.32:0/2760405305 >> 172.21.15.32:0/2760405305 conn(0x7f238806a510 msgr2=0x7f238806d730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 -- 172.21.15.32:0/2760405305 shutdown_connections 2023-12-20T04:28:40.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.380+0000 7f238ff09640 1 -- 172.21.15.32:0/2760405305 wait complete. 2023-12-20T04:28:40.406 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T04:28:40.406 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T04:28:40.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 Processor -- start 2023-12-20T04:28:40.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 -- start start 2023-12-20T04:28:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23e0161510 0x7f23e0071760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e00cdbd0 con 0x7f23e015cef0 2023-12-20T04:28:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e00cd460 con 0x7f23e00cde70 2023-12-20T04:28:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e00cd7b0 con 0x7f23e0161510 2023-12-20T04:28:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e565f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23e0161510 0x7f23e0071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e5e60640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f23e015cef0 0x7f23e00ca270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35292/0 (socket says 172.21.15.32:35292) 2023-12-20T04:28:40.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e5e60640 1 -- 172.21.15.32:0/2864591654 learned_addr learned my addr 172.21.15.32:0/2864591654 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e565f640 1 -- 172.21.15.32:0/2864591654 >> v1:172.21.15.39:6789/0 conn(0x7f23e00cde70 legacy=0x7f23e00cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e565f640 1 -- 172.21.15.32:0/2864591654 >> v1:172.21.15.32:6789/0 conn(0x7f23e015cef0 legacy=0x7f23e00ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e565f640 1 -- 172.21.15.32:0/2864591654 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e0071ef0 con 0x7f23e0161510 2023-12-20T04:28:40.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e4e5e640 1 -- 172.21.15.32:0/2864591654 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1887845027 0 0) 0x7f23e00cdbd0 con 0x7f23e015cef0 2023-12-20T04:28:40.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e565f640 1 --2- 172.21.15.32:0/2864591654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e0161510 0x7f23e0071760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23d00097c0 tx=0x7f23d001cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d24b30b343d108f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:40.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e4e5e640 1 -- 172.21.15.32:0/2864591654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0004230 con 0x7f23e0161510 2023-12-20T04:28:40.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e4e5e640 1 -- 172.21.15.32:0/2864591654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d00a7050 con 0x7f23e0161510 2023-12-20T04:28:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e4e5e640 1 -- 172.21.15.32:0/2864591654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0003620 con 0x7f23e0161510 2023-12-20T04:28:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 -- 172.21.15.32:0/2864591654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e0161510 msgr2=0x7f23e0071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 --2- 172.21.15.32:0/2864591654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e0161510 0x7f23e0071760 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f23d00097c0 tx=0x7f23d001cd90 comp rx=0 tx=0).stop 2023-12-20T04:28:40.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 -- 172.21.15.32:0/2864591654 shutdown_connections 2023-12-20T04:28:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 --2- 172.21.15.32:0/2864591654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e0161510 0x7f23e0071760 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 -- 172.21.15.32:0/2864591654 >> 172.21.15.32:0/2864591654 conn(0x7f23e006b230 msgr2=0x7f23e006b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 -- 172.21.15.32:0/2864591654 shutdown_connections 2023-12-20T04:28:40.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.556+0000 7f23e80c4640 1 -- 172.21.15.32:0/2864591654 wait complete. 2023-12-20T04:28:40.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e80c4640 1 Processor -- start 2023-12-20T04:28:40.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e80c4640 1 -- start start 2023-12-20T04:28:40.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e80c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e00cde70 0x7f23e00b4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:40.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e80c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e015cef0 0x7f23e00adff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:40.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e80c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e0161510 0x7f23e015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:40.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e80c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00b5300 con 0x7f23e015cef0 2023-12-20T04:28:40.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e80c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00b4000 con 0x7f23e0161510 2023-12-20T04:28:40.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e80c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e00b42e0 con 0x7f23e00cde70 2023-12-20T04:28:40.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e565f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e015cef0 0x7f23e00adff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:40.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e565f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e015cef0 0x7f23e00adff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48096/0 (socket says 172.21.15.32:48096) 2023-12-20T04:28:40.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e565f640 1 -- 172.21.15.32:0/2505388643 learned_addr learned my addr 172.21.15.32:0/2505388643 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:40.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e6661640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e0161510 0x7f23e015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e5e60640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e00cde70 0x7f23e00b4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e565f640 1 -- 172.21.15.32:0/2505388643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e00cde70 msgr2=0x7f23e00b4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e565f640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e00cde70 0x7f23e00b4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e565f640 1 -- 172.21.15.32:0/2505388643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e0161510 msgr2=0x7f23e015bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e565f640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e0161510 0x7f23e015bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e565f640 1 -- 172.21.15.32:0/2505388643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d0009420 con 0x7f23e015cef0 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e6661640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e0161510 0x7f23e015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e5e60640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e00cde70 0x7f23e00b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e565f640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e015cef0 0x7f23e00adff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f23d0018f40 tx=0x7f23d00190d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23d67fc640 1 -- 172.21.15.32:0/2505388643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0003cf0 con 0x7f23e015cef0 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23d67fc640 1 -- 172.21.15.32:0/2505388643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d0003e50 con 0x7f23e015cef0 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23d67fc640 1 -- 172.21.15.32:0/2505388643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d00ab680 con 0x7f23e015cef0 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e80c4640 1 -- 172.21.15.32:0/2505388643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e00b45c0 con 0x7f23e015cef0 2023-12-20T04:28:40.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.560+0000 7f23e80c4640 1 -- 172.21.15.32:0/2505388643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e00ae530 con 0x7f23e015cef0 2023-12-20T04:28:40.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.564+0000 7f23e80c4640 1 -- 172.21.15.32:0/2505388643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e00b4140 con 0x7f23e015cef0 2023-12-20T04:28:40.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.564+0000 7f23d67fc640 1 -- 172.21.15.32:0/2505388643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f23d00b2040 con 0x7f23e015cef0 2023-12-20T04:28:40.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.564+0000 7f23d67fc640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f23c4065a60 0x7f23c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:40.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.564+0000 7f23d67fc640 1 -- 172.21.15.32:0/2505388643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f23d011d2d0 con 0x7f23e015cef0 2023-12-20T04:28:40.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.564+0000 7f23e5e60640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f23c4065a60 0x7f23c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:40.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.564+0000 7f23e5e60640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f23c4065a60 0x7f23c4067f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f23dc000f70 tx=0x7f23dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:40.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.572+0000 7f23d67fc640 1 -- 172.21.15.32:0/2505388643 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f23e00b4140 con 0x7f23e015cef0 2023-12-20T04:28:40.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:40.760+0000 7f23e80c4640 1 -- 172.21.15.32:0/2505388643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f23e00b4140 con 0x7f23e015cef0 2023-12-20T04:28:41.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.380+0000 7f23d67fc640 1 -- 172.21.15.32:0/2505388643 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v8) v1 ==== 122+0+0 (secure 0 0 0) 0x7f23e00b4140 con 0x7f23e015cef0 2023-12-20T04:28:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 -- 172.21.15.32:0/2505388643 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f23c4065a60 msgr2=0x7f23c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f23c4065a60 0x7f23c4067f20 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f23dc000f70 tx=0x7f23dc009040 comp rx=0 tx=0).stop 2023-12-20T04:28:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 -- 172.21.15.32:0/2505388643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e015cef0 msgr2=0x7f23e00adff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e015cef0 0x7f23e00adff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f23d0018f40 tx=0x7f23d00190d0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 -- 172.21.15.32:0/2505388643 shutdown_connections 2023-12-20T04:28:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f23c4065a60 0x7f23c4067f20 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e0161510 0x7f23e015bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e015cef0 0x7f23e00adff0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 --2- 172.21.15.32:0/2505388643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e00cde70 0x7f23e00b4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 -- 172.21.15.32:0/2505388643 >> 172.21.15.32:0/2505388643 conn(0x7f23e006b230 msgr2=0x7f23e00af090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:41.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 -- 172.21.15.32:0/2505388643 shutdown_connections 2023-12-20T04:28:41.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.384+0000 7f23e80c4640 1 -- 172.21.15.32:0/2505388643 wait complete. 2023-12-20T04:28:41.407 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T04:28:41.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:28:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.552+0000 7f8f1dba2640 1 Processor -- start 2023-12-20T04:28:41.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.552+0000 7f8f1dba2640 1 -- start start 2023-12-20T04:28:41.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.552+0000 7f8f1dba2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f180b8480 0x7f8f1806af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.552+0000 7f8f1dba2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8f18161500 0x7f8f18153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.552+0000 7f8f1dba2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f180cd4d0 con 0x7f8f1806b460 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.552+0000 7f8f1dba2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f18161930 con 0x7f8f180b8480 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.552+0000 7f8f1dba2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f1806b8f0 con 0x7f8f18161500 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.552+0000 7f8f1cba0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f180b8480 0x7f8f1806af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.552+0000 7f8f1cba0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f180b8480 0x7f8f1806af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48100/0 (socket says 172.21.15.32:48100) 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.552+0000 7f8f1cba0640 1 -- 172.21.15.32:0/3251656291 learned_addr learned my addr 172.21.15.32:0/3251656291 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.552+0000 7f8f1d3a1640 1 --2- 172.21.15.32:0/3251656291 >> v2:172.21.15.39:3300/0 conn(0x7f8f18161500 0x7f8f18153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f1cba0640 1 -- 172.21.15.32:0/3251656291 >> v2:172.21.15.39:3300/0 conn(0x7f8f18161500 msgr2=0x7f8f18153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f1cba0640 1 --2- 172.21.15.32:0/3251656291 >> v2:172.21.15.39:3300/0 conn(0x7f8f18161500 0x7f8f18153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f1cba0640 1 -- 172.21.15.32:0/3251656291 >> v1:172.21.15.39:6789/0 conn(0x7f8f1806b460 legacy=0x7f8f18160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:41.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f1cba0640 1 -- 172.21.15.32:0/3251656291 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f180ce4e0 con 0x7f8f180b8480 2023-12-20T04:28:41.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f1cba0640 1 --2- 172.21.15.32:0/3251656291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f180b8480 0x7f8f1806af20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f180cef30 tx=0x7f8f0c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff1e025d99e6a0ff server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f177fe640 1 -- 172.21.15.32:0/3251656291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0ab5f0 con 0x7f8f180b8480 2023-12-20T04:28:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f177fe640 1 -- 172.21.15.32:0/3251656291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0ab750 con 0x7f8f180b8480 2023-12-20T04:28:41.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f177fe640 1 -- 172.21.15.32:0/3251656291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0abae0 con 0x7f8f180b8480 2023-12-20T04:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f1dba2640 1 -- 172.21.15.32:0/3251656291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f180b8480 msgr2=0x7f8f1806af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f1dba2640 1 --2- 172.21.15.32:0/3251656291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f180b8480 0x7f8f1806af20 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f8f180cef30 tx=0x7f8f0c01c500 comp rx=0 tx=0).stop 2023-12-20T04:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f1dba2640 1 -- 172.21.15.32:0/3251656291 shutdown_connections 2023-12-20T04:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f1dba2640 1 --2- 172.21.15.32:0/3251656291 >> v2:172.21.15.39:3300/0 conn(0x7f8f18161500 0x7f8f18153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f1dba2640 1 --2- 172.21.15.32:0/3251656291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f180b8480 0x7f8f1806af20 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.556+0000 7f8f1dba2640 1 -- 172.21.15.32:0/3251656291 >> 172.21.15.32:0/3251656291 conn(0x7f8f180d2a50 msgr2=0x7f8f180d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1dba2640 1 -- 172.21.15.32:0/3251656291 shutdown_connections 2023-12-20T04:28:41.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1dba2640 1 -- 172.21.15.32:0/3251656291 wait complete. 2023-12-20T04:28:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1dba2640 1 Processor -- start 2023-12-20T04:28:41.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1dba2640 1 -- start start 2023-12-20T04:28:41.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1dba2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1806b460 0x7f8f18154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1dba2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f180b8480 0x7f8f180c5720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1dba2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f18161500 0x7f8f1815e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1dba2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180c6b30 con 0x7f8f1806b460 2023-12-20T04:28:41.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1dba2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180c6e10 con 0x7f8f180b8480 2023-12-20T04:28:41.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1dba2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f180c70f0 con 0x7f8f18161500 2023-12-20T04:28:41.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1cba0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1806b460 0x7f8f18154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1cba0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1806b460 0x7f8f18154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48102/0 (socket says 172.21.15.32:48102) 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1cba0640 1 -- 172.21.15.32:0/2350381961 learned_addr learned my addr 172.21.15.32:0/2350381961 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f17fff640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f180b8480 0x7f8f180c5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1d3a1640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f18161500 0x7f8f1815e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1cba0640 1 -- 172.21.15.32:0/2350381961 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f18161500 msgr2=0x7f8f1815e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1cba0640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f18161500 0x7f8f1815e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1cba0640 1 -- 172.21.15.32:0/2350381961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f180b8480 msgr2=0x7f8f180c5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1cba0640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f180b8480 0x7f8f180c5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1cba0640 1 -- 172.21.15.32:0/2350381961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f0c0094d0 con 0x7f8f1806b460 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1d3a1640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f18161500 0x7f8f1815e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.560+0000 7f8f1cba0640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1806b460 0x7f8f18154580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c019110 tx=0x7f8f0c01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:41.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.564+0000 7f8f157fa640 1 -- 172.21.15.32:0/2350381961 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0be5f0 con 0x7f8f1806b460 2023-12-20T04:28:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.564+0000 7f8f157fa640 1 -- 172.21.15.32:0/2350381961 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f0c0be750 con 0x7f8f1806b460 2023-12-20T04:28:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.564+0000 7f8f157fa640 1 -- 172.21.15.32:0/2350381961 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f0c0beb00 con 0x7f8f1806b460 2023-12-20T04:28:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.564+0000 7f8f1dba2640 1 -- 172.21.15.32:0/2350381961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f180c73d0 con 0x7f8f1806b460 2023-12-20T04:28:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.564+0000 7f8f1dba2640 1 -- 172.21.15.32:0/2350381961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f180c5c60 con 0x7f8f1806b460 2023-12-20T04:28:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.564+0000 7f8f1dba2640 1 -- 172.21.15.32:0/2350381961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f18074100 con 0x7f8f1806b460 2023-12-20T04:28:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.564+0000 7f8f157fa640 1 -- 172.21.15.32:0/2350381961 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8f0c0b43f0 con 0x7f8f1806b460 2023-12-20T04:28:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.564+0000 7f8f157fa640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8efc065a60 0x7f8efc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.564+0000 7f8f157fa640 1 -- 172.21.15.32:0/2350381961 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8f0c0a8050 con 0x7f8f1806b460 2023-12-20T04:28:41.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.564+0000 7f8f17fff640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8efc065a60 0x7f8efc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:41.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.564+0000 7f8f17fff640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8efc065a60 0x7f8efc067f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f18157910 tx=0x7f8f04009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:41.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.568+0000 7f8f157fa640 1 -- 172.21.15.32:0/2350381961 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8f0c0eb730 con 0x7f8f1806b460 2023-12-20T04:28:41.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.760+0000 7f8f1dba2640 1 -- 172.21.15.32:0/2350381961 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8f180d0a70 con 0x7f8f1806b460 2023-12-20T04:28:41.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.760+0000 7f8f157fa640 1 -- 172.21.15.32:0/2350381961 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4700 (secure 0 0 0) 0x7f8f0c0eed50 con 0x7f8f1806b460 2023-12-20T04:28:41.761 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:41.761 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2247020919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2247020919},{"type":"v1","addr":"172.21.15.39:6834","nonce":2247020919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/797559820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":797559820},{"type":"v1","addr":"172.21.15.32:6836","nonce":797559820}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3209531230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3209531230},{"type":"v1","addr":"172.21.15.32:6837","nonce":3209531230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:28:38.361494+0000","modified":"2023-12-20T04:28:41.379636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547681098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3547681098},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547681098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-20T04:28:41.761 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.760+0000 7f8f1dba2640 1 -- 172.21.15.32:0/2350381961 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8efc065a60 msgr2=0x7f8efc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.760+0000 7f8f1dba2640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8efc065a60 0x7f8efc067f20 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f8f18157910 tx=0x7f8f04009040 comp rx=0 tx=0).stop 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.760+0000 7f8f1dba2640 1 -- 172.21.15.32:0/2350381961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1806b460 msgr2=0x7f8f18154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.760+0000 7f8f1dba2640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1806b460 0x7f8f18154580 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f0c019110 tx=0x7f8f0c01ad40 comp rx=0 tx=0).stop 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.764+0000 7f8f1dba2640 1 -- 172.21.15.32:0/2350381961 shutdown_connections 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.764+0000 7f8f1dba2640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8efc065a60 0x7f8efc067f20 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.764+0000 7f8f1dba2640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f18161500 0x7f8f1815e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.764+0000 7f8f1dba2640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f180b8480 0x7f8f180c5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.764+0000 7f8f1dba2640 1 --2- 172.21.15.32:0/2350381961 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f1806b460 0x7f8f18154580 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.764+0000 7f8f1dba2640 1 -- 172.21.15.32:0/2350381961 >> 172.21.15.32:0/2350381961 conn(0x7f8f180d2a50 msgr2=0x7f8f180b6190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.764+0000 7f8f1dba2640 1 -- 172.21.15.32:0/2350381961 shutdown_connections 2023-12-20T04:28:41.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.764+0000 7f8f1dba2640 1 -- 172.21.15.32:0/2350381961 wait complete. 2023-12-20T04:28:41.782 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:41.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.924+0000 7f8bf77e9640 1 Processor -- start 2023-12-20T04:28:41.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.924+0000 7f8bf77e9640 1 -- start start 2023-12-20T04:28:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.924+0000 7f8bf77e9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bf00af5e0 0x7f8bf00af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.924+0000 7f8bf77e9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8bf006ce60 0x7f8bf015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.924+0000 7f8bf77e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8bf015f110 0x7f8bf015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.924+0000 7f8bf77e9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf015bbd0 con 0x7f8bf00af5e0 2023-12-20T04:28:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.924+0000 7f8bf77e9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8bf015beb0 con 0x7f8bf006ce60 2023-12-20T04:28:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.924+0000 7f8bf77e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf015e910 con 0x7f8bf015f110 2023-12-20T04:28:41.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8bf5585640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bf00af5e0 0x7f8bf00af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8bf5585640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8bf00af5e0 0x7f8bf00af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48118/0 (socket says 172.21.15.32:48118) 2023-12-20T04:28:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8bf5585640 1 -- 172.21.15.32:0/1775477396 learned_addr learned my addr 172.21.15.32:0/1775477396 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8bf5585640 1 -- 172.21.15.32:0/1775477396 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8bf015f110 msgr2=0x7f8bf015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8bf5585640 1 --2- 172.21.15.32:0/1775477396 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8bf015f110 0x7f8bf015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8bf5585640 1 -- 172.21.15.32:0/1775477396 >> v2:172.21.15.39:3300/0 conn(0x7f8bf006ce60 msgr2=0x7f8bf015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8bf5585640 1 --2- 172.21.15.32:0/1775477396 >> v2:172.21.15.39:3300/0 conn(0x7f8bf006ce60 0x7f8bf015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8bf5585640 1 -- 172.21.15.32:0/1775477396 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf015ebf0 con 0x7f8bf00af5e0 2023-12-20T04:28:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8bf5585640 1 --2- 172.21.15.32:0/1775477396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bf00af5e0 0x7f8bf00af9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8bf006abe0 tx=0x7f8bec01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf0d915fab52f6ea server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8be7fff640 1 -- 172.21.15.32:0/1775477396 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0ab620 con 0x7f8bf00af5e0 2023-12-20T04:28:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8be7fff640 1 -- 172.21.15.32:0/1775477396 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bec0ab780 con 0x7f8bf00af5e0 2023-12-20T04:28:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.928+0000 7f8be7fff640 1 -- 172.21.15.32:0/1775477396 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bec0abb10 con 0x7f8bf00af5e0 2023-12-20T04:28:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 -- 172.21.15.32:0/1775477396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bf00af5e0 msgr2=0x7f8bf00af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 --2- 172.21.15.32:0/1775477396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bf00af5e0 0x7f8bf00af9c0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f8bf006abe0 tx=0x7f8bec01c500 comp rx=0 tx=0).stop 2023-12-20T04:28:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 -- 172.21.15.32:0/1775477396 shutdown_connections 2023-12-20T04:28:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 --2- 172.21.15.32:0/1775477396 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8bf015f110 0x7f8bf015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 --2- 172.21.15.32:0/1775477396 >> v2:172.21.15.39:3300/0 conn(0x7f8bf006ce60 0x7f8bf015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 --2- 172.21.15.32:0/1775477396 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bf00af5e0 0x7f8bf00af9c0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 -- 172.21.15.32:0/1775477396 >> 172.21.15.32:0/1775477396 conn(0x7f8bf006a320 msgr2=0x7f8bf00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 -- 172.21.15.32:0/1775477396 shutdown_connections 2023-12-20T04:28:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 -- 172.21.15.32:0/1775477396 wait complete. 2023-12-20T04:28:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 Processor -- start 2023-12-20T04:28:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 -- start start 2023-12-20T04:28:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf006ce60 0x7f8bf00b2990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8bf00af5e0 0x7f8bf00c6870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bf015f110 0x7f8bf00c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00c84b0 con 0x7f8bf015f110 2023-12-20T04:28:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00c7410 con 0x7f8bf006ce60 2023-12-20T04:28:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf77e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8bf00c76f0 con 0x7f8bf00af5e0 2023-12-20T04:28:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf5d86640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bf015f110 0x7f8bf00c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf5585640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf006ce60 0x7f8bf00b2990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf5d86640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bf015f110 0x7f8bf00c6db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48130/0 (socket says 172.21.15.32:48130) 2023-12-20T04:28:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf5d86640 1 -- 172.21.15.32:0/641078366 learned_addr learned my addr 172.21.15.32:0/641078366 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf4d84640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8bf00af5e0 0x7f8bf00c6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf5d86640 1 -- 172.21.15.32:0/641078366 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8bf00af5e0 msgr2=0x7f8bf00c6870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf5d86640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8bf00af5e0 0x7f8bf00c6870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf5d86640 1 -- 172.21.15.32:0/641078366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf006ce60 msgr2=0x7f8bf00b2990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf5d86640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf006ce60 0x7f8bf00b2990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.932+0000 7f8bf5d86640 1 -- 172.21.15.32:0/641078366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bec009480 con 0x7f8bf015f110 2023-12-20T04:28:41.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.936+0000 7f8bf5d86640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bf015f110 0x7f8bf00c6db0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8be800b970 tx=0x7f8be800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.936+0000 7f8be5ffb640 1 -- 172.21.15.32:0/641078366 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be800cb40 con 0x7f8bf015f110 2023-12-20T04:28:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.936+0000 7f8be5ffb640 1 -- 172.21.15.32:0/641078366 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8be800ccc0 con 0x7f8bf015f110 2023-12-20T04:28:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.936+0000 7f8bf77e9640 1 -- 172.21.15.32:0/641078366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8bf00c79d0 con 0x7f8bf015f110 2023-12-20T04:28:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.936+0000 7f8be5ffb640 1 -- 172.21.15.32:0/641078366 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8be8014680 con 0x7f8bf015f110 2023-12-20T04:28:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.936+0000 7f8bf77e9640 1 -- 172.21.15.32:0/641078366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8bf00c7f90 con 0x7f8bf015f110 2023-12-20T04:28:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.936+0000 7f8bf77e9640 1 -- 172.21.15.32:0/641078366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8bf00c81b0 con 0x7f8bf015f110 2023-12-20T04:28:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.936+0000 7f8be5ffb640 1 -- 172.21.15.32:0/641078366 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8be801d3b0 con 0x7f8bf015f110 2023-12-20T04:28:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.936+0000 7f8be5ffb640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8bd4065a60 0x7f8bd4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.936+0000 7f8be5ffb640 1 -- 172.21.15.32:0/641078366 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8be8083010 con 0x7f8bf015f110 2023-12-20T04:28:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.936+0000 7f8bf5585640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8bd4065a60 0x7f8bd4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.940+0000 7f8bf5585640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8bd4065a60 0x7f8bd4067f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0af000 tx=0x7f8bec0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:41.940+0000 7f8be5ffb640 1 -- 172.21.15.32:0/641078366 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8be8050a70 con 0x7f8bf015f110 2023-12-20T04:28:42.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.100+0000 7f8bf77e9640 1 -- 172.21.15.32:0/641078366 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8bf004e000 con 0x7f8bf015f110 2023-12-20T04:28:42.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.100+0000 7f8be5ffb640 1 -- 172.21.15.32:0/641078366 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f8be8019020 con 0x7f8bf015f110 2023-12-20T04:28:42.101 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:42.102 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 -- 172.21.15.32:0/641078366 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8bd4065a60 msgr2=0x7f8bd4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8bd4065a60 0x7f8bd4067f20 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f8bec0af000 tx=0x7f8bec0a7000 comp rx=0 tx=0).stop 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 -- 172.21.15.32:0/641078366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bf015f110 msgr2=0x7f8bf00c6db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bf015f110 0x7f8bf00c6db0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f8be800b970 tx=0x7f8be800be30 comp rx=0 tx=0).stop 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 -- 172.21.15.32:0/641078366 shutdown_connections 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8bd4065a60 0x7f8bd4067f20 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8bf015f110 0x7f8bf00c6db0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8bf00af5e0 0x7f8bf00c6870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 --2- 172.21.15.32:0/641078366 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8bf006ce60 0x7f8bf00b2990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 -- 172.21.15.32:0/641078366 >> 172.21.15.32:0/641078366 conn(0x7f8bf006a320 msgr2=0x7f8bf006c620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 -- 172.21.15.32:0/641078366 shutdown_connections 2023-12-20T04:28:42.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.104+0000 7f8bf77e9640 1 -- 172.21.15.32:0/641078366 wait complete. 2023-12-20T04:28:42.123 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T04:28:42.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:28:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.264+0000 7fde748e5640 1 Processor -- start 2023-12-20T04:28:42.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.264+0000 7fde748e5640 1 -- start start 2023-12-20T04:28:42.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde748e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde7015ce30 0x7fde70161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde748e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fde70070950 0x7fde7006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde748e5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fde70071920 con 0x7fde7006dce0 2023-12-20T04:28:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde748e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde70071080 con 0x7fde7015ce30 2023-12-20T04:28:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde748e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde70071360 con 0x7fde70070950 2023-12-20T04:28:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde6e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde7015ce30 0x7fde70161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde6e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde7015ce30 0x7fde70161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48136/0 (socket says 172.21.15.32:48136) 2023-12-20T04:28:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde6e59c640 1 -- 172.21.15.32:0/2323469952 learned_addr learned my addr 172.21.15.32:0/2323469952 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde67fff640 1 --2- 172.21.15.32:0/2323469952 >> v2:172.21.15.39:3300/0 conn(0x7fde70070950 0x7fde7006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:42.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde6e59c640 1 -- 172.21.15.32:0/2323469952 >> v2:172.21.15.39:3300/0 conn(0x7fde70070950 msgr2=0x7fde7006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:42.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde6e59c640 1 --2- 172.21.15.32:0/2323469952 >> v2:172.21.15.39:3300/0 conn(0x7fde70070950 0x7fde7006d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde6e59c640 1 -- 172.21.15.32:0/2323469952 >> v1:172.21.15.39:6789/0 conn(0x7fde7006dce0 legacy=0x7fde70069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.268+0000 7fde6e59c640 1 -- 172.21.15.32:0/2323469952 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde70071640 con 0x7fde7015ce30 2023-12-20T04:28:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde6e59c640 1 --2- 172.21.15.32:0/2323469952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7015ce30 0x7fde70161560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fde7006f400 tx=0x7fde6001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56f2bfc4da4e9dbb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde6dd9b640 1 -- 172.21.15.32:0/2323469952 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde600ab610 con 0x7fde7015ce30 2023-12-20T04:28:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde6dd9b640 1 -- 172.21.15.32:0/2323469952 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde600ab770 con 0x7fde7015ce30 2023-12-20T04:28:42.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde6dd9b640 1 -- 172.21.15.32:0/2323469952 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde600abb00 con 0x7fde7015ce30 2023-12-20T04:28:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 -- 172.21.15.32:0/2323469952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7015ce30 msgr2=0x7fde70161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 --2- 172.21.15.32:0/2323469952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7015ce30 0x7fde70161560 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7fde7006f400 tx=0x7fde6001c500 comp rx=0 tx=0).stop 2023-12-20T04:28:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 -- 172.21.15.32:0/2323469952 shutdown_connections 2023-12-20T04:28:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 --2- 172.21.15.32:0/2323469952 >> v2:172.21.15.39:3300/0 conn(0x7fde70070950 0x7fde7006d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 --2- 172.21.15.32:0/2323469952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7015ce30 0x7fde70161560 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 -- 172.21.15.32:0/2323469952 >> 172.21.15.32:0/2323469952 conn(0x7fde700cba50 msgr2=0x7fde700cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 -- 172.21.15.32:0/2323469952 shutdown_connections 2023-12-20T04:28:42.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 -- 172.21.15.32:0/2323469952 wait complete. 2023-12-20T04:28:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 Processor -- start 2023-12-20T04:28:42.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 -- start start 2023-12-20T04:28:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7006dce0 0x7fde701560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde70070950 0x7fde70152460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde7015ce30 0x7fde70154ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde700c75f0 con 0x7fde7006dce0 2023-12-20T04:28:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde70153730 con 0x7fde7015ce30 2023-12-20T04:28:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.272+0000 7fde748e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde70153a10 con 0x7fde70070950 2023-12-20T04:28:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde6e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7006dce0 0x7fde701560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde6e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7006dce0 0x7fde701560c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48140/0 (socket says 172.21.15.32:48140) 2023-12-20T04:28:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde6e59c640 1 -- 172.21.15.32:0/387036139 learned_addr learned my addr 172.21.15.32:0/387036139 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde67fff640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde70070950 0x7fde70152460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:42.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde6ed9d640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde7015ce30 0x7fde70154ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:42.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde6e59c640 1 -- 172.21.15.32:0/387036139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde70070950 msgr2=0x7fde70152460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde6e59c640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde70070950 0x7fde70152460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde6e59c640 1 -- 172.21.15.32:0/387036139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde7015ce30 msgr2=0x7fde70154ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde6e59c640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde7015ce30 0x7fde70154ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde6e59c640 1 -- 172.21.15.32:0/387036139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde600094d0 con 0x7fde7006dce0 2023-12-20T04:28:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde6ed9d640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde7015ce30 0x7fde70154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde6e59c640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7006dce0 0x7fde701560c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fde60019110 tx=0x7fde6001ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde66ffd640 1 -- 172.21.15.32:0/387036139 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde600b4800 con 0x7fde7006dce0 2023-12-20T04:28:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde66ffd640 1 -- 172.21.15.32:0/387036139 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde600b4960 con 0x7fde7006dce0 2023-12-20T04:28:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde66ffd640 1 -- 172.21.15.32:0/387036139 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde600b4cf0 con 0x7fde7006dce0 2023-12-20T04:28:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde748e5640 1 -- 172.21.15.32:0/387036139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde70153cf0 con 0x7fde7006dce0 2023-12-20T04:28:42.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde748e5640 1 -- 172.21.15.32:0/387036139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde70153fd0 con 0x7fde7006dce0 2023-12-20T04:28:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde748e5640 1 -- 172.21.15.32:0/387036139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde700c73c0 con 0x7fde7006dce0 2023-12-20T04:28:42.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.276+0000 7fde66ffd640 1 -- 172.21.15.32:0/387036139 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fde600c7390 con 0x7fde7006dce0 2023-12-20T04:28:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.280+0000 7fde66ffd640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde50065ab0 0x7fde50067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.280+0000 7fde66ffd640 1 -- 172.21.15.32:0/387036139 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde6011da80 con 0x7fde7006dce0 2023-12-20T04:28:42.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.280+0000 7fde67fff640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde50065ab0 0x7fde50067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:42.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.280+0000 7fde67fff640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde50065ab0 0x7fde50067f70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fde700b1b90 tx=0x7fde58009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:42.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.284+0000 7fde66ffd640 1 -- 172.21.15.32:0/387036139 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fde700c73c0 con 0x7fde7006dce0 2023-12-20T04:28:42.305 INFO:tasks.ceph.mds.d.smithi039.stderr:2023-12-20T04:28:42.302+0000 7f95e9e52640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:28:42.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.436+0000 7fde748e5640 1 -- 172.21.15.32:0/387036139 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde7006e0f0 con 0x7fde50065ab0 2023-12-20T04:28:42.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde66ffd640 1 -- 172.21.15.32:0/387036139 <== mgr.4104 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264612 (secure 0 0 0) 0x7fde58011a30 con 0x7fde50065ab0 2023-12-20T04:28:42.450 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:42.451 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:28:42.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 -- 172.21.15.32:0/387036139 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde50065ab0 msgr2=0x7fde50067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde50065ab0 0x7fde50067f70 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fde700b1b90 tx=0x7fde58009190 comp rx=0 tx=0).stop 2023-12-20T04:28:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 -- 172.21.15.32:0/387036139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7006dce0 msgr2=0x7fde701560c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7006dce0 0x7fde701560c0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fde60019110 tx=0x7fde6001ad40 comp rx=0 tx=0).stop 2023-12-20T04:28:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 -- 172.21.15.32:0/387036139 shutdown_connections 2023-12-20T04:28:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde50065ab0 0x7fde50067f70 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde7015ce30 0x7fde70154ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde70070950 0x7fde70152460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 --2- 172.21.15.32:0/387036139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde7006dce0 0x7fde701560c0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 -- 172.21.15.32:0/387036139 >> 172.21.15.32:0/387036139 conn(0x7fde700cba50 msgr2=0x7fde7015d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 -- 172.21.15.32:0/387036139 shutdown_connections 2023-12-20T04:28:42.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.448+0000 7fde748e5640 1 -- 172.21.15.32:0/387036139 wait complete. 2023-12-20T04:28:42.470 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-20T04:28:42.125170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234824,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739896,"statfs":{"total":773094113280,"available":772853653504,"internally_reserved":0,"allocated":2891776,"data_stored":1739612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.171780"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411165+0000","last_change":"2023-12-20T04:28:38.407268+0000","last_active":"2023-12-20T04:28:38.411165+0000","last_peered":"2023-12-20T04:28:38.411165+0000","last_clean":"2023-12-20T04:28:38.411165+0000","last_became_active":"2023-12-20T04:28:38.404491+0000","last_became_peered":"2023-12-20T04:28:38.404491+0000","last_unstale":"2023-12-20T04:28:38.411165+0000","last_undegraded":"2023-12-20T04:28:38.411165+0000","last_fullsized":"2023-12-20T04:28:38.411165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:36.363864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379971+0000","last_change":"2023-12-20T04:28:38.390082+0000","last_active":"2023-12-20T04:28:39.379971+0000","last_peered":"2023-12-20T04:28:39.379971+0000","last_clean":"2023-12-20T04:28:39.379971+0000","last_became_active":"2023-12-20T04:28:38.389688+0000","last_became_peered":"2023-12-20T04:28:38.389688+0000","last_unstale":"2023-12-20T04:28:39.379971+0000","last_undegraded":"2023-12-20T04:28:39.379971+0000","last_fullsized":"2023-12-20T04:28:39.379971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.395049+0000","last_change":"2023-12-20T04:28:38.388981+0000","last_active":"2023-12-20T04:28:38.395049+0000","last_peered":"2023-12-20T04:28:38.395049+0000","last_clean":"2023-12-20T04:28:38.395049+0000","last_became_active":"2023-12-20T04:28:38.388734+0000","last_became_peered":"2023-12-20T04:28:38.388734+0000","last_unstale":"2023-12-20T04:28:38.395049+0000","last_undegraded":"2023-12-20T04:28:38.395049+0000","last_fullsized":"2023-12-20T04:28:38.395049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385490+0000","last_change":"2023-12-20T04:28:38.384727+0000","last_active":"2023-12-20T04:28:39.385490+0000","last_peered":"2023-12-20T04:28:39.385490+0000","last_clean":"2023-12-20T04:28:39.385490+0000","last_became_active":"2023-12-20T04:28:38.384329+0000","last_became_peered":"2023-12-20T04:28:38.384329+0000","last_unstale":"2023-12-20T04:28:39.385490+0000","last_undegraded":"2023-12-20T04:28:39.385490+0000","last_fullsized":"2023-12-20T04:28:39.385490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:11:32.618631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.407744+0000","last_change":"2023-12-20T04:28:38.397089+0000","last_active":"2023-12-20T04:28:38.407744+0000","last_peered":"2023-12-20T04:28:38.407744+0000","last_clean":"2023-12-20T04:28:38.407744+0000","last_became_active":"2023-12-20T04:28:38.396774+0000","last_became_peered":"2023-12-20T04:28:38.396774+0000","last_unstale":"2023-12-20T04:28:38.407744+0000","last_undegraded":"2023-12-20T04:28:38.407744+0000","last_fullsized":"2023-12-20T04:28:38.407744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386104+0000","last_change":"2023-12-20T04:28:38.395384+0000","last_active":"2023-12-20T04:28:39.386104+0000","last_peered":"2023-12-20T04:28:39.386104+0000","last_clean":"2023-12-20T04:28:39.386104+0000","last_became_active":"2023-12-20T04:28:38.394254+0000","last_became_peered":"2023-12-20T04:28:38.394254+0000","last_unstale":"2023-12-20T04:28:39.386104+0000","last_undegraded":"2023-12-20T04:28:39.386104+0000","last_fullsized":"2023-12-20T04:28:39.386104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:55:26.717187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411155+0000","last_change":"2023-12-20T04:28:38.404597+0000","last_active":"2023-12-20T04:28:38.411155+0000","last_peered":"2023-12-20T04:28:38.411155+0000","last_clean":"2023-12-20T04:28:38.411155+0000","last_became_active":"2023-12-20T04:28:38.404108+0000","last_became_peered":"2023-12-20T04:28:38.404108+0000","last_unstale":"2023-12-20T04:28:38.411155+0000","last_undegraded":"2023-12-20T04:28:38.411155+0000","last_fullsized":"2023-12-20T04:28:38.411155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:41.399788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.387621+0000","last_change":"2023-12-20T04:28:38.385150+0000","last_active":"2023-12-20T04:28:38.387621+0000","last_peered":"2023-12-20T04:28:38.387621+0000","last_clean":"2023-12-20T04:28:38.387621+0000","last_became_active":"2023-12-20T04:28:38.384885+0000","last_became_peered":"2023-12-20T04:28:38.384885+0000","last_unstale":"2023-12-20T04:28:38.387621+0000","last_undegraded":"2023-12-20T04:28:38.387621+0000","last_fullsized":"2023-12-20T04:28:38.387621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.410227+0000","last_change":"2023-12-20T04:28:38.399891+0000","last_active":"2023-12-20T04:28:38.410227+0000","last_peered":"2023-12-20T04:28:38.410227+0000","last_clean":"2023-12-20T04:28:38.410227+0000","last_became_active":"2023-12-20T04:28:38.399599+0000","last_became_peered":"2023-12-20T04:28:38.399599+0000","last_unstale":"2023-12-20T04:28:38.410227+0000","last_undegraded":"2023-12-20T04:28:38.410227+0000","last_fullsized":"2023-12-20T04:28:38.410227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:46:52.340773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.406850+0000","last_change":"2023-12-20T04:28:38.398573+0000","last_active":"2023-12-20T04:28:38.406850+0000","last_peered":"2023-12-20T04:28:38.406850+0000","last_clean":"2023-12-20T04:28:38.406850+0000","last_became_active":"2023-12-20T04:28:38.398162+0000","last_became_peered":"2023-12-20T04:28:38.398162+0000","last_unstale":"2023-12-20T04:28:38.406850+0000","last_undegraded":"2023-12-20T04:28:38.406850+0000","last_fullsized":"2023-12-20T04:28:38.406850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411854+0000","last_change":"2023-12-20T04:28:38.405208+0000","last_active":"2023-12-20T04:28:38.411854+0000","last_peered":"2023-12-20T04:28:38.411854+0000","last_clean":"2023-12-20T04:28:38.411854+0000","last_became_active":"2023-12-20T04:28:38.404933+0000","last_became_peered":"2023-12-20T04:28:38.404933+0000","last_unstale":"2023-12-20T04:28:38.411854+0000","last_undegraded":"2023-12-20T04:28:38.411854+0000","last_fullsized":"2023-12-20T04:28:38.411854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:50.720992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386165+0000","last_change":"2023-12-20T04:28:38.412922+0000","last_active":"2023-12-20T04:28:39.386165+0000","last_peered":"2023-12-20T04:28:39.386165+0000","last_clean":"2023-12-20T04:28:39.386165+0000","last_became_active":"2023-12-20T04:28:38.412505+0000","last_became_peered":"2023-12-20T04:28:38.412505+0000","last_unstale":"2023-12-20T04:28:39.386165+0000","last_undegraded":"2023-12-20T04:28:39.386165+0000","last_fullsized":"2023-12-20T04:28:39.386165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:04:59.607569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.394176+0000","last_change":"2023-12-20T04:28:38.390684+0000","last_active":"2023-12-20T04:28:38.394176+0000","last_peered":"2023-12-20T04:28:38.394176+0000","last_clean":"2023-12-20T04:28:38.394176+0000","last_became_active":"2023-12-20T04:28:38.390483+0000","last_became_peered":"2023-12-20T04:28:38.390483+0000","last_unstale":"2023-12-20T04:28:38.394176+0000","last_undegraded":"2023-12-20T04:28:38.394176+0000","last_fullsized":"2023-12-20T04:28:38.394176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:55:26.717187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.392244+0000","last_change":"2023-12-20T04:28:38.379167+0000","last_active":"2023-12-20T04:28:38.392244+0000","last_peered":"2023-12-20T04:28:38.392244+0000","last_clean":"2023-12-20T04:28:38.392244+0000","last_became_active":"2023-12-20T04:28:38.378677+0000","last_became_peered":"2023-12-20T04:28:38.378677+0000","last_unstale":"2023-12-20T04:28:38.392244+0000","last_undegraded":"2023-12-20T04:28:38.392244+0000","last_fullsized":"2023-12-20T04:28:38.392244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379652+0000","last_change":"2023-12-20T04:28:38.391213+0000","last_active":"2023-12-20T04:28:39.379652+0000","last_peered":"2023-12-20T04:28:39.379652+0000","last_clean":"2023-12-20T04:28:39.379652+0000","last_became_active":"2023-12-20T04:28:38.390937+0000","last_became_peered":"2023-12-20T04:28:38.390937+0000","last_unstale":"2023-12-20T04:28:39.379652+0000","last_undegraded":"2023-12-20T04:28:39.379652+0000","last_fullsized":"2023-12-20T04:28:39.379652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385448+0000","last_change":"2023-12-20T04:28:38.403378+0000","last_active":"2023-12-20T04:28:39.385448+0000","last_peered":"2023-12-20T04:28:39.385448+0000","last_clean":"2023-12-20T04:28:39.385448+0000","last_became_active":"2023-12-20T04:28:38.403070+0000","last_became_peered":"2023-12-20T04:28:38.403070+0000","last_unstale":"2023-12-20T04:28:39.385448+0000","last_undegraded":"2023-12-20T04:28:39.385448+0000","last_fullsized":"2023-12-20T04:28:39.385448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:48:18.553482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385977+0000","last_change":"2023-12-20T04:28:38.399610+0000","last_active":"2023-12-20T04:28:39.385977+0000","last_peered":"2023-12-20T04:28:39.385977+0000","last_clean":"2023-12-20T04:28:39.385977+0000","last_became_active":"2023-12-20T04:28:38.399061+0000","last_became_peered":"2023-12-20T04:28:38.399061+0000","last_unstale":"2023-12-20T04:28:39.385977+0000","last_undegraded":"2023-12-20T04:28:39.385977+0000","last_fullsized":"2023-12-20T04:28:39.385977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.403366+0000","last_change":"2023-12-20T04:28:38.398099+0000","last_active":"2023-12-20T04:28:38.403366+0000","last_peered":"2023-12-20T04:28:38.403366+0000","last_clean":"2023-12-20T04:28:38.403366+0000","last_became_active":"2023-12-20T04:28:38.397333+0000","last_became_peered":"2023-12-20T04:28:38.397333+0000","last_unstale":"2023-12-20T04:28:38.403366+0000","last_undegraded":"2023-12-20T04:28:38.403366+0000","last_fullsized":"2023-12-20T04:28:38.403366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:12.041027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.407804+0000","last_change":"2023-12-20T04:28:38.397182+0000","last_active":"2023-12-20T04:28:38.407804+0000","last_peered":"2023-12-20T04:28:38.407804+0000","last_clean":"2023-12-20T04:28:38.407804+0000","last_became_active":"2023-12-20T04:28:38.396800+0000","last_became_peered":"2023-12-20T04:28:38.396800+0000","last_unstale":"2023-12-20T04:28:38.407804+0000","last_undegraded":"2023-12-20T04:28:38.407804+0000","last_fullsized":"2023-12-20T04:28:38.407804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411935+0000","last_change":"2023-12-20T04:28:38.410245+0000","last_active":"2023-12-20T04:28:38.411935+0000","last_peered":"2023-12-20T04:28:38.411935+0000","last_clean":"2023-12-20T04:28:38.411935+0000","last_became_active":"2023-12-20T04:28:38.403138+0000","last_became_peered":"2023-12-20T04:28:38.403138+0000","last_unstale":"2023-12-20T04:28:38.411935+0000","last_undegraded":"2023-12-20T04:28:38.411935+0000","last_fullsized":"2023-12-20T04:28:38.411935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.403794+0000","last_change":"2023-12-20T04:28:38.400262+0000","last_active":"2023-12-20T04:28:38.403794+0000","last_peered":"2023-12-20T04:28:38.403794+0000","last_clean":"2023-12-20T04:28:38.403794+0000","last_became_active":"2023-12-20T04:28:38.400021+0000","last_became_peered":"2023-12-20T04:28:38.400021+0000","last_unstale":"2023-12-20T04:28:38.403794+0000","last_undegraded":"2023-12-20T04:28:38.403794+0000","last_fullsized":"2023-12-20T04:28:38.403794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386133+0000","last_change":"2023-12-20T04:28:38.397515+0000","last_active":"2023-12-20T04:28:39.386133+0000","last_peered":"2023-12-20T04:28:39.386133+0000","last_clean":"2023-12-20T04:28:39.386133+0000","last_became_active":"2023-12-20T04:28:38.397234+0000","last_became_peered":"2023-12-20T04:28:38.397234+0000","last_unstale":"2023-12-20T04:28:39.386133+0000","last_undegraded":"2023-12-20T04:28:39.386133+0000","last_fullsized":"2023-12-20T04:28:39.386133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.392860+0000","last_change":"2023-12-20T04:28:38.390587+0000","last_active":"2023-12-20T04:28:38.392860+0000","last_peered":"2023-12-20T04:28:38.392860+0000","last_clean":"2023-12-20T04:28:38.392860+0000","last_became_active":"2023-12-20T04:28:38.390256+0000","last_became_peered":"2023-12-20T04:28:38.390256+0000","last_unstale":"2023-12-20T04:28:38.392860+0000","last_undegraded":"2023-12-20T04:28:38.392860+0000","last_fullsized":"2023-12-20T04:28:38.392860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385781+0000","last_change":"2023-12-20T04:28:38.390087+0000","last_active":"2023-12-20T04:28:39.385781+0000","last_peered":"2023-12-20T04:28:39.385781+0000","last_clean":"2023-12-20T04:28:39.385781+0000","last_became_active":"2023-12-20T04:28:38.389881+0000","last_became_peered":"2023-12-20T04:28:38.389881+0000","last_unstale":"2023-12-20T04:28:39.385781+0000","last_undegraded":"2023-12-20T04:28:39.385781+0000","last_fullsized":"2023-12-20T04:28:39.385781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:09.976618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.398819+0000","last_change":"2023-12-20T04:28:38.396543+0000","last_active":"2023-12-20T04:28:38.398819+0000","last_peered":"2023-12-20T04:28:38.398819+0000","last_clean":"2023-12-20T04:28:38.398819+0000","last_became_active":"2023-12-20T04:28:38.396247+0000","last_became_peered":"2023-12-20T04:28:38.396247+0000","last_unstale":"2023-12-20T04:28:38.398819+0000","last_undegraded":"2023-12-20T04:28:38.398819+0000","last_fullsized":"2023-12-20T04:28:38.398819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.406875+0000","last_change":"2023-12-20T04:28:38.401846+0000","last_active":"2023-12-20T04:28:38.406875+0000","last_peered":"2023-12-20T04:28:38.406875+0000","last_clean":"2023-12-20T04:28:38.406875+0000","last_became_active":"2023-12-20T04:28:38.401255+0000","last_became_peered":"2023-12-20T04:28:38.401255+0000","last_unstale":"2023-12-20T04:28:38.406875+0000","last_undegraded":"2023-12-20T04:28:38.406875+0000","last_fullsized":"2023-12-20T04:28:38.406875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379844+0000","last_change":"2023-12-20T04:28:38.385251+0000","last_active":"2023-12-20T04:28:39.379844+0000","last_peered":"2023-12-20T04:28:39.379844+0000","last_clean":"2023-12-20T04:28:39.379844+0000","last_became_active":"2023-12-20T04:28:38.384990+0000","last_became_peered":"2023-12-20T04:28:38.384990+0000","last_unstale":"2023-12-20T04:28:39.379844+0000","last_undegraded":"2023-12-20T04:28:39.379844+0000","last_fullsized":"2023-12-20T04:28:39.379844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.398594+0000","last_change":"2023-12-20T04:28:38.394851+0000","last_active":"2023-12-20T04:28:38.398594+0000","last_peered":"2023-12-20T04:28:38.398594+0000","last_clean":"2023-12-20T04:28:38.398594+0000","last_became_active":"2023-12-20T04:28:38.394584+0000","last_became_peered":"2023-12-20T04:28:38.394584+0000","last_unstale":"2023-12-20T04:28:38.398594+0000","last_undegraded":"2023-12-20T04:28:38.398594+0000","last_fullsized":"2023-12-20T04:28:38.398594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.400461+0000","last_change":"2023-12-20T04:28:38.383584+0000","last_active":"2023-12-20T04:28:38.400461+0000","last_peered":"2023-12-20T04:28:38.400461+0000","last_clean":"2023-12-20T04:28:38.400461+0000","last_became_active":"2023-12-20T04:28:38.383310+0000","last_became_peered":"2023-12-20T04:28:38.383310+0000","last_unstale":"2023-12-20T04:28:38.400461+0000","last_undegraded":"2023-12-20T04:28:38.400461+0000","last_fullsized":"2023-12-20T04:28:38.400461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386215+0000","last_change":"2023-12-20T04:28:38.388220+0000","last_active":"2023-12-20T04:28:39.386215+0000","last_peered":"2023-12-20T04:28:39.386215+0000","last_clean":"2023-12-20T04:28:39.386215+0000","last_became_active":"2023-12-20T04:28:38.386044+0000","last_became_peered":"2023-12-20T04:28:38.386044+0000","last_unstale":"2023-12-20T04:28:39.386215+0000","last_undegraded":"2023-12-20T04:28:39.386215+0000","last_fullsized":"2023-12-20T04:28:39.386215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:51.073749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.396856+0000","last_change":"2023-12-20T04:28:38.384460+0000","last_active":"2023-12-20T04:28:38.396856+0000","last_peered":"2023-12-20T04:28:38.396856+0000","last_clean":"2023-12-20T04:28:38.396856+0000","last_became_active":"2023-12-20T04:28:38.384190+0000","last_became_peered":"2023-12-20T04:28:38.384190+0000","last_unstale":"2023-12-20T04:28:38.396856+0000","last_undegraded":"2023-12-20T04:28:38.396856+0000","last_fullsized":"2023-12-20T04:28:38.396856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.409083+0000","last_change":"2023-12-20T04:28:38.403381+0000","last_active":"2023-12-20T04:28:38.409083+0000","last_peered":"2023-12-20T04:28:38.409083+0000","last_clean":"2023-12-20T04:28:38.409083+0000","last_became_active":"2023-12-20T04:28:38.402953+0000","last_became_peered":"2023-12-20T04:28:38.402953+0000","last_unstale":"2023-12-20T04:28:38.409083+0000","last_undegraded":"2023-12-20T04:28:38.409083+0000","last_fullsized":"2023-12-20T04:28:38.409083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385741+0000","last_change":"2023-12-20T04:28:38.392967+0000","last_active":"2023-12-20T04:28:39.385741+0000","last_peered":"2023-12-20T04:28:39.385741+0000","last_clean":"2023-12-20T04:28:39.385741+0000","last_became_active":"2023-12-20T04:28:38.392657+0000","last_became_peered":"2023-12-20T04:28:38.392657+0000","last_unstale":"2023-12-20T04:28:39.385741+0000","last_undegraded":"2023-12-20T04:28:39.385741+0000","last_fullsized":"2023-12-20T04:28:39.385741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:35.897773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.393945+0000","last_change":"2023-12-20T04:28:37.983099+0000","last_active":"2023-12-20T04:28:38.393945+0000","last_peered":"2023-12-20T04:28:38.393945+0000","last_clean":"2023-12-20T04:28:38.393945+0000","last_became_active":"2023-12-20T04:28:37.982857+0000","last_became_peered":"2023-12-20T04:28:37.982857+0000","last_unstale":"2023-12-20T04:28:38.393945+0000","last_undegraded":"2023-12-20T04:28:38.393945+0000","last_fullsized":"2023-12-20T04:28:38.393945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.408543+0000","last_change":"2023-12-20T04:28:38.407295+0000","last_active":"2023-12-20T04:28:38.408543+0000","last_peered":"2023-12-20T04:28:38.408543+0000","last_clean":"2023-12-20T04:28:38.408543+0000","last_became_active":"2023-12-20T04:28:38.406963+0000","last_became_peered":"2023-12-20T04:28:38.406963+0000","last_unstale":"2023-12-20T04:28:38.408543+0000","last_undegraded":"2023-12-20T04:28:38.408543+0000","last_fullsized":"2023-12-20T04:28:38.408543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:50.720992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.406335+0000","last_change":"2023-12-20T04:28:37.987627+0000","last_active":"2023-12-20T04:28:38.406335+0000","last_peered":"2023-12-20T04:28:38.406335+0000","last_clean":"2023-12-20T04:28:38.406335+0000","last_became_active":"2023-12-20T04:28:37.987309+0000","last_became_peered":"2023-12-20T04:28:37.987309+0000","last_unstale":"2023-12-20T04:28:38.406335+0000","last_undegraded":"2023-12-20T04:28:38.406335+0000","last_fullsized":"2023-12-20T04:28:38.406335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.372901+0000","last_change":"2023-12-20T04:28:37.996411+0000","last_active":"2023-12-20T04:28:38.372901+0000","last_peered":"2023-12-20T04:28:38.372901+0000","last_clean":"2023-12-20T04:28:38.372901+0000","last_became_active":"2023-12-20T04:28:37.996051+0000","last_became_peered":"2023-12-20T04:28:37.996051+0000","last_unstale":"2023-12-20T04:28:38.372901+0000","last_undegraded":"2023-12-20T04:28:38.372901+0000","last_fullsized":"2023-12-20T04:28:38.372901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.399071+0000","last_change":"2023-12-20T04:28:38.396714+0000","last_active":"2023-12-20T04:28:38.399071+0000","last_peered":"2023-12-20T04:28:38.399071+0000","last_clean":"2023-12-20T04:28:38.399071+0000","last_became_active":"2023-12-20T04:28:38.396337+0000","last_became_peered":"2023-12-20T04:28:38.396337+0000","last_unstale":"2023-12-20T04:28:38.399071+0000","last_undegraded":"2023-12-20T04:28:38.399071+0000","last_fullsized":"2023-12-20T04:28:38.399071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385551+0000","last_change":"2023-12-20T04:28:37.982942+0000","last_active":"2023-12-20T04:28:39.385551+0000","last_peered":"2023-12-20T04:28:39.385551+0000","last_clean":"2023-12-20T04:28:39.385551+0000","last_became_active":"2023-12-20T04:28:37.982287+0000","last_became_peered":"2023-12-20T04:28:37.982287+0000","last_unstale":"2023-12-20T04:28:39.385551+0000","last_undegraded":"2023-12-20T04:28:39.385551+0000","last_fullsized":"2023-12-20T04:28:39.385551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379613+0000","last_change":"2023-12-20T04:28:38.380510+0000","last_active":"2023-12-20T04:28:39.379613+0000","last_peered":"2023-12-20T04:28:39.379613+0000","last_clean":"2023-12-20T04:28:39.379613+0000","last_became_active":"2023-12-20T04:28:38.379896+0000","last_became_peered":"2023-12-20T04:28:38.379896+0000","last_unstale":"2023-12-20T04:28:39.379613+0000","last_undegraded":"2023-12-20T04:28:39.379613+0000","last_fullsized":"2023-12-20T04:28:39.379613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.391801+0000","last_change":"2023-12-20T04:28:37.992369+0000","last_active":"2023-12-20T04:28:38.391801+0000","last_peered":"2023-12-20T04:28:38.391801+0000","last_clean":"2023-12-20T04:28:38.391801+0000","last_became_active":"2023-12-20T04:28:37.992065+0000","last_became_peered":"2023-12-20T04:28:37.992065+0000","last_unstale":"2023-12-20T04:28:38.391801+0000","last_undegraded":"2023-12-20T04:28:38.391801+0000","last_fullsized":"2023-12-20T04:28:38.391801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385551+0000","last_change":"2023-12-20T04:28:38.390943+0000","last_active":"2023-12-20T04:28:39.385551+0000","last_peered":"2023-12-20T04:28:39.385551+0000","last_clean":"2023-12-20T04:28:39.385551+0000","last_became_active":"2023-12-20T04:28:38.390661+0000","last_became_peered":"2023-12-20T04:28:38.390661+0000","last_unstale":"2023-12-20T04:28:39.385551+0000","last_undegraded":"2023-12-20T04:28:39.385551+0000","last_fullsized":"2023-12-20T04:28:39.385551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:06.804397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385691+0000","last_change":"2023-12-20T04:28:37.986949+0000","last_active":"2023-12-20T04:28:39.385691+0000","last_peered":"2023-12-20T04:28:39.385691+0000","last_clean":"2023-12-20T04:28:39.385691+0000","last_became_active":"2023-12-20T04:28:37.986664+0000","last_became_peered":"2023-12-20T04:28:37.986664+0000","last_unstale":"2023-12-20T04:28:39.385691+0000","last_undegraded":"2023-12-20T04:28:39.385691+0000","last_fullsized":"2023-12-20T04:28:39.385691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.405825+0000","last_change":"2023-12-20T04:28:38.403067+0000","last_active":"2023-12-20T04:28:38.405825+0000","last_peered":"2023-12-20T04:28:38.405825+0000","last_clean":"2023-12-20T04:28:38.405825+0000","last_became_active":"2023-12-20T04:28:38.402312+0000","last_became_peered":"2023-12-20T04:28:38.402312+0000","last_unstale":"2023-12-20T04:28:38.405825+0000","last_undegraded":"2023-12-20T04:28:38.405825+0000","last_fullsized":"2023-12-20T04:28:38.405825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379842+0000","last_change":"2023-12-20T04:28:37.989273+0000","last_active":"2023-12-20T04:28:39.379842+0000","last_peered":"2023-12-20T04:28:39.379842+0000","last_clean":"2023-12-20T04:28:39.379842+0000","last_became_active":"2023-12-20T04:28:37.988921+0000","last_became_peered":"2023-12-20T04:28:37.988921+0000","last_unstale":"2023-12-20T04:28:39.379842+0000","last_undegraded":"2023-12-20T04:28:39.379842+0000","last_fullsized":"2023-12-20T04:28:39.379842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386234+0000","last_change":"2023-12-20T04:28:38.394680+0000","last_active":"2023-12-20T04:28:39.386234+0000","last_peered":"2023-12-20T04:28:39.386234+0000","last_clean":"2023-12-20T04:28:39.386234+0000","last_became_active":"2023-12-20T04:28:38.394369+0000","last_became_peered":"2023-12-20T04:28:38.394369+0000","last_unstale":"2023-12-20T04:28:39.386234+0000","last_undegraded":"2023-12-20T04:28:39.386234+0000","last_fullsized":"2023-12-20T04:28:39.386234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:12.041027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.415038+0000","last_change":"2023-12-20T04:28:37.992630+0000","last_active":"2023-12-20T04:28:38.415038+0000","last_peered":"2023-12-20T04:28:38.415038+0000","last_clean":"2023-12-20T04:28:38.415038+0000","last_became_active":"2023-12-20T04:28:37.989328+0000","last_became_peered":"2023-12-20T04:28:37.989328+0000","last_unstale":"2023-12-20T04:28:38.415038+0000","last_undegraded":"2023-12-20T04:28:38.415038+0000","last_fullsized":"2023-12-20T04:28:38.415038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.393559+0000","last_change":"2023-12-20T04:28:38.390252+0000","last_active":"2023-12-20T04:28:38.393559+0000","last_peered":"2023-12-20T04:28:38.393559+0000","last_clean":"2023-12-20T04:28:38.393559+0000","last_became_active":"2023-12-20T04:28:38.389945+0000","last_became_peered":"2023-12-20T04:28:38.389945+0000","last_unstale":"2023-12-20T04:28:38.393559+0000","last_undegraded":"2023-12-20T04:28:38.393559+0000","last_fullsized":"2023-12-20T04:28:38.393559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:11:32.618631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411064+0000","last_change":"2023-12-20T04:28:37.979978+0000","last_active":"2023-12-20T04:28:38.411064+0000","last_peered":"2023-12-20T04:28:38.411064+0000","last_clean":"2023-12-20T04:28:38.411064+0000","last_became_active":"2023-12-20T04:28:37.979639+0000","last_became_peered":"2023-12-20T04:28:37.979639+0000","last_unstale":"2023-12-20T04:28:38.411064+0000","last_undegraded":"2023-12-20T04:28:38.411064+0000","last_fullsized":"2023-12-20T04:28:38.411064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.403307+0000","last_change":"2023-12-20T04:28:38.400246+0000","last_active":"2023-12-20T04:28:38.403307+0000","last_peered":"2023-12-20T04:28:38.403307+0000","last_clean":"2023-12-20T04:28:38.403307+0000","last_became_active":"2023-12-20T04:28:38.399932+0000","last_became_peered":"2023-12-20T04:28:38.399932+0000","last_unstale":"2023-12-20T04:28:38.403307+0000","last_undegraded":"2023-12-20T04:28:38.403307+0000","last_fullsized":"2023-12-20T04:28:38.403307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:46:19.275821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386216+0000","last_change":"2023-12-20T04:28:37.985893+0000","last_active":"2023-12-20T04:28:39.386216+0000","last_peered":"2023-12-20T04:28:39.386216+0000","last_clean":"2023-12-20T04:28:39.386216+0000","last_became_active":"2023-12-20T04:28:37.985605+0000","last_became_peered":"2023-12-20T04:28:37.985605+0000","last_unstale":"2023-12-20T04:28:39.386216+0000","last_undegraded":"2023-12-20T04:28:39.386216+0000","last_fullsized":"2023-12-20T04:28:39.386216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411881+0000","last_change":"2023-12-20T04:28:38.409446+0000","last_active":"2023-12-20T04:28:38.411881+0000","last_peered":"2023-12-20T04:28:38.411881+0000","last_clean":"2023-12-20T04:28:38.411881+0000","last_became_active":"2023-12-20T04:28:38.409174+0000","last_became_peered":"2023-12-20T04:28:38.409174+0000","last_unstale":"2023-12-20T04:28:38.411881+0000","last_undegraded":"2023-12-20T04:28:38.411881+0000","last_fullsized":"2023-12-20T04:28:38.411881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:51.073749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.390234+0000","last_change":"2023-12-20T04:28:37.986147+0000","last_active":"2023-12-20T04:28:38.390234+0000","last_peered":"2023-12-20T04:28:38.390234+0000","last_clean":"2023-12-20T04:28:38.390234+0000","last_became_active":"2023-12-20T04:28:37.985323+0000","last_became_peered":"2023-12-20T04:28:37.985323+0000","last_unstale":"2023-12-20T04:28:38.390234+0000","last_undegraded":"2023-12-20T04:28:38.390234+0000","last_fullsized":"2023-12-20T04:28:38.390234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.409065+0000","last_change":"2023-12-20T04:28:38.403115+0000","last_active":"2023-12-20T04:28:38.409065+0000","last_peered":"2023-12-20T04:28:38.409065+0000","last_clean":"2023-12-20T04:28:38.409065+0000","last_became_active":"2023-12-20T04:28:38.402521+0000","last_became_peered":"2023-12-20T04:28:38.402521+0000","last_unstale":"2023-12-20T04:28:38.409065+0000","last_undegraded":"2023-12-20T04:28:38.409065+0000","last_fullsized":"2023-12-20T04:28:38.409065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.398155+0000","last_change":"2023-12-20T04:28:37.970816+0000","last_active":"2023-12-20T04:28:38.398155+0000","last_peered":"2023-12-20T04:28:38.398155+0000","last_clean":"2023-12-20T04:28:38.398155+0000","last_became_active":"2023-12-20T04:28:37.970330+0000","last_became_peered":"2023-12-20T04:28:37.970330+0000","last_unstale":"2023-12-20T04:28:38.398155+0000","last_undegraded":"2023-12-20T04:28:38.398155+0000","last_fullsized":"2023-12-20T04:28:38.398155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.408428+0000","last_change":"2023-12-20T04:28:38.398095+0000","last_active":"2023-12-20T04:28:38.408428+0000","last_peered":"2023-12-20T04:28:38.408428+0000","last_clean":"2023-12-20T04:28:38.408428+0000","last_became_active":"2023-12-20T04:28:38.397747+0000","last_became_peered":"2023-12-20T04:28:38.397747+0000","last_unstale":"2023-12-20T04:28:38.408428+0000","last_undegraded":"2023-12-20T04:28:38.408428+0000","last_fullsized":"2023-12-20T04:28:38.408428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:15:10.280675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386532+0000","last_change":"2023-12-20T04:28:37.980101+0000","last_active":"2023-12-20T04:28:39.386532+0000","last_peered":"2023-12-20T04:28:39.386532+0000","last_clean":"2023-12-20T04:28:39.386532+0000","last_became_active":"2023-12-20T04:28:37.979770+0000","last_became_peered":"2023-12-20T04:28:37.979770+0000","last_unstale":"2023-12-20T04:28:39.386532+0000","last_undegraded":"2023-12-20T04:28:39.386532+0000","last_fullsized":"2023-12-20T04:28:39.386532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411904+0000","last_change":"2023-12-20T04:28:38.409708+0000","last_active":"2023-12-20T04:28:38.411904+0000","last_peered":"2023-12-20T04:28:38.411904+0000","last_clean":"2023-12-20T04:28:38.411904+0000","last_became_active":"2023-12-20T04:28:38.408900+0000","last_became_peered":"2023-12-20T04:28:38.408900+0000","last_unstale":"2023-12-20T04:28:38.411904+0000","last_undegraded":"2023-12-20T04:28:38.411904+0000","last_fullsized":"2023-12-20T04:28:38.411904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:13.592803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.389306+0000","last_change":"2023-12-20T04:28:37.985187+0000","last_active":"2023-12-20T04:28:38.389306+0000","last_peered":"2023-12-20T04:28:38.389306+0000","last_clean":"2023-12-20T04:28:38.389306+0000","last_became_active":"2023-12-20T04:28:37.984909+0000","last_became_peered":"2023-12-20T04:28:37.984909+0000","last_unstale":"2023-12-20T04:28:38.389306+0000","last_undegraded":"2023-12-20T04:28:38.389306+0000","last_fullsized":"2023-12-20T04:28:38.389306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.408004+0000","last_change":"2023-12-20T04:28:38.397685+0000","last_active":"2023-12-20T04:28:38.408004+0000","last_peered":"2023-12-20T04:28:38.408004+0000","last_clean":"2023-12-20T04:28:38.408004+0000","last_became_active":"2023-12-20T04:28:38.396818+0000","last_became_peered":"2023-12-20T04:28:38.396818+0000","last_unstale":"2023-12-20T04:28:38.408004+0000","last_undegraded":"2023-12-20T04:28:38.408004+0000","last_fullsized":"2023-12-20T04:28:38.408004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:09.976618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379930+0000","last_change":"2023-12-20T04:28:37.981598+0000","last_active":"2023-12-20T04:28:39.379930+0000","last_peered":"2023-12-20T04:28:39.379930+0000","last_clean":"2023-12-20T04:28:39.379930+0000","last_became_active":"2023-12-20T04:28:37.981319+0000","last_became_peered":"2023-12-20T04:28:37.981319+0000","last_unstale":"2023-12-20T04:28:39.379930+0000","last_undegraded":"2023-12-20T04:28:39.379930+0000","last_fullsized":"2023-12-20T04:28:39.379930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385613+0000","last_change":"2023-12-20T04:28:38.385270+0000","last_active":"2023-12-20T04:28:39.385613+0000","last_peered":"2023-12-20T04:28:39.385613+0000","last_clean":"2023-12-20T04:28:39.385613+0000","last_became_active":"2023-12-20T04:28:38.384740+0000","last_became_peered":"2023-12-20T04:28:38.384740+0000","last_unstale":"2023-12-20T04:28:39.385613+0000","last_undegraded":"2023-12-20T04:28:39.385613+0000","last_fullsized":"2023-12-20T04:28:39.385613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:13.592803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.372577+0000","last_change":"2023-12-20T04:28:37.991119+0000","last_active":"2023-12-20T04:28:38.372577+0000","last_peered":"2023-12-20T04:28:38.372577+0000","last_clean":"2023-12-20T04:28:38.372577+0000","last_became_active":"2023-12-20T04:28:37.990872+0000","last_became_peered":"2023-12-20T04:28:37.990872+0000","last_unstale":"2023-12-20T04:28:38.372577+0000","last_undegraded":"2023-12-20T04:28:38.372577+0000","last_fullsized":"2023-12-20T04:28:38.372577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.407785+0000","last_change":"2023-12-20T04:28:38.394920+0000","last_active":"2023-12-20T04:28:38.407785+0000","last_peered":"2023-12-20T04:28:38.407785+0000","last_clean":"2023-12-20T04:28:38.407785+0000","last_became_active":"2023-12-20T04:28:38.394396+0000","last_became_peered":"2023-12-20T04:28:38.394396+0000","last_unstale":"2023-12-20T04:28:38.407785+0000","last_undegraded":"2023-12-20T04:28:38.407785+0000","last_fullsized":"2023-12-20T04:28:38.407785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-20T04:28:37.979939+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:28:37.979939+0000","last_peered":"2023-12-20T04:28:37.979939+0000","last_clean":"2023-12-20T04:28:37.979939+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:28:37.979939+0000","last_undegraded":"2023-12-20T04:28:37.979939+0000","last_fullsized":"2023-12-20T04:28:37.979939+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386014+0000","last_change":"2023-12-20T04:28:37.978792+0000","last_active":"2023-12-20T04:28:39.386014+0000","last_peered":"2023-12-20T04:28:39.386014+0000","last_clean":"2023-12-20T04:28:39.386014+0000","last_became_active":"2023-12-20T04:28:37.978509+0000","last_became_peered":"2023-12-20T04:28:37.978509+0000","last_unstale":"2023-12-20T04:28:39.386014+0000","last_undegraded":"2023-12-20T04:28:39.386014+0000","last_fullsized":"2023-12-20T04:28:39.386014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.400976+0000","last_change":"2023-12-20T04:28:38.400189+0000","last_active":"2023-12-20T04:28:38.400976+0000","last_peered":"2023-12-20T04:28:38.400976+0000","last_clean":"2023-12-20T04:28:38.400976+0000","last_became_active":"2023-12-20T04:28:38.399902+0000","last_became_peered":"2023-12-20T04:28:38.399902+0000","last_unstale":"2023-12-20T04:28:38.400976+0000","last_undegraded":"2023-12-20T04:28:38.400976+0000","last_fullsized":"2023-12-20T04:28:38.400976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:06.804397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.376745+0000","last_change":"2023-12-20T04:28:37.987850+0000","last_active":"2023-12-20T04:28:38.376745+0000","last_peered":"2023-12-20T04:28:38.376745+0000","last_clean":"2023-12-20T04:28:38.376745+0000","last_became_active":"2023-12-20T04:28:37.986972+0000","last_became_peered":"2023-12-20T04:28:37.986972+0000","last_unstale":"2023-12-20T04:28:38.376745+0000","last_undegraded":"2023-12-20T04:28:38.376745+0000","last_fullsized":"2023-12-20T04:28:38.376745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:19.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.403048+0000","last_change":"2023-12-20T04:28:38.397621+0000","last_active":"2023-12-20T04:28:38.403048+0000","last_peered":"2023-12-20T04:28:38.403048+0000","last_clean":"2023-12-20T04:28:38.403048+0000","last_became_active":"2023-12-20T04:28:38.397311+0000","last_became_peered":"2023-12-20T04:28:38.397311+0000","last_unstale":"2023-12-20T04:28:38.403048+0000","last_undegraded":"2023-12-20T04:28:38.403048+0000","last_fullsized":"2023-12-20T04:28:38.403048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:26:59.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.371710+0000","last_change":"2023-12-20T04:28:37.982113+0000","last_active":"2023-12-20T04:28:38.371710+0000","last_peered":"2023-12-20T04:28:38.371710+0000","last_clean":"2023-12-20T04:28:38.371710+0000","last_became_active":"2023-12-20T04:28:37.981812+0000","last_became_peered":"2023-12-20T04:28:37.981812+0000","last_unstale":"2023-12-20T04:28:38.371710+0000","last_undegraded":"2023-12-20T04:28:38.371710+0000","last_fullsized":"2023-12-20T04:28:38.371710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379680+0000","last_change":"2023-12-20T04:28:38.383867+0000","last_active":"2023-12-20T04:28:39.379680+0000","last_peered":"2023-12-20T04:28:39.379680+0000","last_clean":"2023-12-20T04:28:39.379680+0000","last_became_active":"2023-12-20T04:28:38.383222+0000","last_became_peered":"2023-12-20T04:28:38.383222+0000","last_unstale":"2023-12-20T04:28:39.379680+0000","last_undegraded":"2023-12-20T04:28:39.379680+0000","last_fullsized":"2023-12-20T04:28:39.379680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386301+0000","last_change":"2023-12-20T04:28:37.981934+0000","last_active":"2023-12-20T04:28:39.386301+0000","last_peered":"2023-12-20T04:28:39.386301+0000","last_clean":"2023-12-20T04:28:39.386301+0000","last_became_active":"2023-12-20T04:28:37.981000+0000","last_became_peered":"2023-12-20T04:28:37.981000+0000","last_unstale":"2023-12-20T04:28:39.386301+0000","last_undegraded":"2023-12-20T04:28:39.386301+0000","last_fullsized":"2023-12-20T04:28:39.386301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386046+0000","last_change":"2023-12-20T04:28:38.405946+0000","last_active":"2023-12-20T04:28:39.386046+0000","last_peered":"2023-12-20T04:28:39.386046+0000","last_clean":"2023-12-20T04:28:39.386046+0000","last_became_active":"2023-12-20T04:28:38.405609+0000","last_became_peered":"2023-12-20T04:28:38.405609+0000","last_unstale":"2023-12-20T04:28:39.386046+0000","last_undegraded":"2023-12-20T04:28:39.386046+0000","last_fullsized":"2023-12-20T04:28:39.386046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379821+0000","last_change":"2023-12-20T04:28:37.980797+0000","last_active":"2023-12-20T04:28:39.379821+0000","last_peered":"2023-12-20T04:28:39.379821+0000","last_clean":"2023-12-20T04:28:39.379821+0000","last_became_active":"2023-12-20T04:28:37.980430+0000","last_became_peered":"2023-12-20T04:28:37.980430+0000","last_unstale":"2023-12-20T04:28:39.379821+0000","last_undegraded":"2023-12-20T04:28:39.379821+0000","last_fullsized":"2023-12-20T04:28:39.379821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:19.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386153+0000","last_change":"2023-12-20T04:28:38.389261+0000","last_active":"2023-12-20T04:28:39.386153+0000","last_peered":"2023-12-20T04:28:39.386153+0000","last_clean":"2023-12-20T04:28:39.386153+0000","last_became_active":"2023-12-20T04:28:38.388004+0000","last_became_peered":"2023-12-20T04:28:38.388004+0000","last_unstale":"2023-12-20T04:28:39.386153+0000","last_undegraded":"2023-12-20T04:28:39.386153+0000","last_fullsized":"2023-12-20T04:28:39.386153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.378913+0000","last_change":"2023-12-20T04:28:37.991195+0000","last_active":"2023-12-20T04:28:38.378913+0000","last_peered":"2023-12-20T04:28:38.378913+0000","last_clean":"2023-12-20T04:28:38.378913+0000","last_became_active":"2023-12-20T04:28:37.990860+0000","last_became_peered":"2023-12-20T04:28:37.990860+0000","last_unstale":"2023-12-20T04:28:38.378913+0000","last_undegraded":"2023-12-20T04:28:38.378913+0000","last_fullsized":"2023-12-20T04:28:38.378913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:11.027603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379827+0000","last_change":"2023-12-20T04:28:38.377842+0000","last_active":"2023-12-20T04:28:39.379827+0000","last_peered":"2023-12-20T04:28:39.379827+0000","last_clean":"2023-12-20T04:28:39.379827+0000","last_became_active":"2023-12-20T04:28:38.377525+0000","last_became_peered":"2023-12-20T04:28:38.377525+0000","last_unstale":"2023-12-20T04:28:39.379827+0000","last_undegraded":"2023-12-20T04:28:39.379827+0000","last_fullsized":"2023-12-20T04:28:39.379827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.380013+0000","last_change":"2023-12-20T04:28:37.995878+0000","last_active":"2023-12-20T04:28:39.380013+0000","last_peered":"2023-12-20T04:28:39.380013+0000","last_clean":"2023-12-20T04:28:39.380013+0000","last_became_active":"2023-12-20T04:28:37.995604+0000","last_became_peered":"2023-12-20T04:28:37.995604+0000","last_unstale":"2023-12-20T04:28:39.380013+0000","last_undegraded":"2023-12-20T04:28:39.380013+0000","last_fullsized":"2023-12-20T04:28:39.380013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.399533+0000","last_change":"2023-12-20T04:28:38.398433+0000","last_active":"2023-12-20T04:28:38.399533+0000","last_peered":"2023-12-20T04:28:38.399533+0000","last_clean":"2023-12-20T04:28:38.399533+0000","last_became_active":"2023-12-20T04:28:38.398247+0000","last_became_peered":"2023-12-20T04:28:38.398247+0000","last_unstale":"2023-12-20T04:28:38.399533+0000","last_undegraded":"2023-12-20T04:28:38.399533+0000","last_fullsized":"2023-12-20T04:28:38.399533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:48:18.553482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411943+0000","last_change":"2023-12-20T04:28:37.993415+0000","last_active":"2023-12-20T04:28:38.411943+0000","last_peered":"2023-12-20T04:28:38.411943+0000","last_clean":"2023-12-20T04:28:38.411943+0000","last_became_active":"2023-12-20T04:28:37.993209+0000","last_became_peered":"2023-12-20T04:28:37.993209+0000","last_unstale":"2023-12-20T04:28:38.411943+0000","last_undegraded":"2023-12-20T04:28:38.411943+0000","last_fullsized":"2023-12-20T04:28:38.411943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.393420+0000","last_change":"2023-12-20T04:28:38.388800+0000","last_active":"2023-12-20T04:28:38.393420+0000","last_peered":"2023-12-20T04:28:38.393420+0000","last_clean":"2023-12-20T04:28:38.393420+0000","last_became_active":"2023-12-20T04:28:38.383908+0000","last_became_peered":"2023-12-20T04:28:38.383908+0000","last_unstale":"2023-12-20T04:28:38.393420+0000","last_undegraded":"2023-12-20T04:28:38.393420+0000","last_fullsized":"2023-12-20T04:28:38.393420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.376636+0000","last_change":"2023-12-20T04:28:37.996076+0000","last_active":"2023-12-20T04:28:38.376636+0000","last_peered":"2023-12-20T04:28:38.376636+0000","last_clean":"2023-12-20T04:28:38.376636+0000","last_became_active":"2023-12-20T04:28:37.995770+0000","last_became_peered":"2023-12-20T04:28:37.995770+0000","last_unstale":"2023-12-20T04:28:38.376636+0000","last_undegraded":"2023-12-20T04:28:38.376636+0000","last_fullsized":"2023-12-20T04:28:38.376636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:34.884350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385884+0000","last_change":"2023-12-20T04:28:38.393602+0000","last_active":"2023-12-20T04:28:39.385884+0000","last_peered":"2023-12-20T04:28:39.385884+0000","last_clean":"2023-12-20T04:28:39.385884+0000","last_became_active":"2023-12-20T04:28:38.392277+0000","last_became_peered":"2023-12-20T04:28:38.392277+0000","last_unstale":"2023-12-20T04:28:39.385884+0000","last_undegraded":"2023-12-20T04:28:39.385884+0000","last_fullsized":"2023-12-20T04:28:39.385884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:20.635968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379704+0000","last_change":"2023-12-20T04:28:37.987522+0000","last_active":"2023-12-20T04:28:39.379704+0000","last_peered":"2023-12-20T04:28:39.379704+0000","last_clean":"2023-12-20T04:28:39.379704+0000","last_became_active":"2023-12-20T04:28:37.986906+0000","last_became_peered":"2023-12-20T04:28:37.986906+0000","last_unstale":"2023-12-20T04:28:39.379704+0000","last_undegraded":"2023-12-20T04:28:39.379704+0000","last_fullsized":"2023-12-20T04:28:39.379704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:34.884350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.386583+0000","last_change":"2023-12-20T04:28:38.382504+0000","last_active":"2023-12-20T04:28:38.386583+0000","last_peered":"2023-12-20T04:28:38.386583+0000","last_clean":"2023-12-20T04:28:38.386583+0000","last_became_active":"2023-12-20T04:28:38.382224+0000","last_became_peered":"2023-12-20T04:28:38.382224+0000","last_unstale":"2023-12-20T04:28:38.386583+0000","last_undegraded":"2023-12-20T04:28:38.386583+0000","last_fullsized":"2023-12-20T04:28:38.386583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379771+0000","last_change":"2023-12-20T04:28:37.985344+0000","last_active":"2023-12-20T04:28:39.379771+0000","last_peered":"2023-12-20T04:28:39.379771+0000","last_clean":"2023-12-20T04:28:39.379771+0000","last_became_active":"2023-12-20T04:28:37.985074+0000","last_became_peered":"2023-12-20T04:28:37.985074+0000","last_unstale":"2023-12-20T04:28:39.379771+0000","last_undegraded":"2023-12-20T04:28:39.379771+0000","last_fullsized":"2023-12-20T04:28:39.379771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.404941+0000","last_change":"2023-12-20T04:28:38.402571+0000","last_active":"2023-12-20T04:28:38.404941+0000","last_peered":"2023-12-20T04:28:38.404941+0000","last_clean":"2023-12-20T04:28:38.404941+0000","last_became_active":"2023-12-20T04:28:38.402316+0000","last_became_peered":"2023-12-20T04:28:38.402316+0000","last_unstale":"2023-12-20T04:28:38.404941+0000","last_undegraded":"2023-12-20T04:28:38.404941+0000","last_fullsized":"2023-12-20T04:28:38.404941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.408942+0000","last_change":"2023-12-20T04:28:37.991378+0000","last_active":"2023-12-20T04:28:38.408942+0000","last_peered":"2023-12-20T04:28:38.408942+0000","last_clean":"2023-12-20T04:28:38.408942+0000","last_became_active":"2023-12-20T04:28:37.990976+0000","last_became_peered":"2023-12-20T04:28:37.990976+0000","last_unstale":"2023-12-20T04:28:38.408942+0000","last_undegraded":"2023-12-20T04:28:38.408942+0000","last_fullsized":"2023-12-20T04:28:38.408942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:18.611871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.413343+0000","last_change":"2023-12-20T04:28:38.411659+0000","last_active":"2023-12-20T04:28:38.413343+0000","last_peered":"2023-12-20T04:28:38.413343+0000","last_clean":"2023-12-20T04:28:38.413343+0000","last_became_active":"2023-12-20T04:28:38.411333+0000","last_became_peered":"2023-12-20T04:28:38.411333+0000","last_unstale":"2023-12-20T04:28:38.413343+0000","last_undegraded":"2023-12-20T04:28:38.413343+0000","last_fullsized":"2023-12-20T04:28:38.413343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:46:19.275821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386042+0000","last_change":"2023-12-20T04:28:38.392988+0000","last_active":"2023-12-20T04:28:39.386042+0000","last_peered":"2023-12-20T04:28:39.386042+0000","last_clean":"2023-12-20T04:28:39.386042+0000","last_became_active":"2023-12-20T04:28:38.392136+0000","last_became_peered":"2023-12-20T04:28:38.392136+0000","last_unstale":"2023-12-20T04:28:39.386042+0000","last_undegraded":"2023-12-20T04:28:39.386042+0000","last_fullsized":"2023-12-20T04:28:39.386042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.400965+0000","last_change":"2023-12-20T04:28:37.991881+0000","last_active":"2023-12-20T04:28:38.400965+0000","last_peered":"2023-12-20T04:28:38.400965+0000","last_clean":"2023-12-20T04:28:38.400965+0000","last_became_active":"2023-12-20T04:28:37.991520+0000","last_became_peered":"2023-12-20T04:28:37.991520+0000","last_unstale":"2023-12-20T04:28:38.400965+0000","last_undegraded":"2023-12-20T04:28:38.400965+0000","last_fullsized":"2023-12-20T04:28:38.400965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385497+0000","last_change":"2023-12-20T04:28:38.398174+0000","last_active":"2023-12-20T04:28:39.385497+0000","last_peered":"2023-12-20T04:28:39.385497+0000","last_clean":"2023-12-20T04:28:39.385497+0000","last_became_active":"2023-12-20T04:28:38.397909+0000","last_became_peered":"2023-12-20T04:28:38.397909+0000","last_unstale":"2023-12-20T04:28:39.385497+0000","last_undegraded":"2023-12-20T04:28:39.385497+0000","last_fullsized":"2023-12-20T04:28:39.385497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.401174+0000","last_change":"2023-12-20T04:28:37.997620+0000","last_active":"2023-12-20T04:28:38.401174+0000","last_peered":"2023-12-20T04:28:38.401174+0000","last_clean":"2023-12-20T04:28:38.401174+0000","last_became_active":"2023-12-20T04:28:37.997289+0000","last_became_peered":"2023-12-20T04:28:37.997289+0000","last_unstale":"2023-12-20T04:28:38.401174+0000","last_undegraded":"2023-12-20T04:28:38.401174+0000","last_fullsized":"2023-12-20T04:28:38.401174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:00.985871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.405734+0000","last_change":"2023-12-20T04:28:38.403356+0000","last_active":"2023-12-20T04:28:38.405734+0000","last_peered":"2023-12-20T04:28:38.405734+0000","last_clean":"2023-12-20T04:28:38.405734+0000","last_became_active":"2023-12-20T04:28:38.402695+0000","last_became_peered":"2023-12-20T04:28:38.402695+0000","last_unstale":"2023-12-20T04:28:38.405734+0000","last_undegraded":"2023-12-20T04:28:38.405734+0000","last_fullsized":"2023-12-20T04:28:38.405734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.391009+0000","last_change":"2023-12-20T04:28:37.986554+0000","last_active":"2023-12-20T04:28:38.391009+0000","last_peered":"2023-12-20T04:28:38.391009+0000","last_clean":"2023-12-20T04:28:38.391009+0000","last_became_active":"2023-12-20T04:28:37.985944+0000","last_became_peered":"2023-12-20T04:28:37.985944+0000","last_unstale":"2023-12-20T04:28:38.391009+0000","last_undegraded":"2023-12-20T04:28:38.391009+0000","last_fullsized":"2023-12-20T04:28:38.391009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.403613+0000","last_change":"2023-12-20T04:28:38.402052+0000","last_active":"2023-12-20T04:28:38.403613+0000","last_peered":"2023-12-20T04:28:38.403613+0000","last_clean":"2023-12-20T04:28:38.403613+0000","last_became_active":"2023-12-20T04:28:38.401578+0000","last_became_peered":"2023-12-20T04:28:38.401578+0000","last_unstale":"2023-12-20T04:28:38.403613+0000","last_undegraded":"2023-12-20T04:28:38.403613+0000","last_fullsized":"2023-12-20T04:28:38.403613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411161+0000","last_change":"2023-12-20T04:28:37.991899+0000","last_active":"2023-12-20T04:28:38.411161+0000","last_peered":"2023-12-20T04:28:38.411161+0000","last_clean":"2023-12-20T04:28:38.411161+0000","last_became_active":"2023-12-20T04:28:37.991545+0000","last_became_peered":"2023-12-20T04:28:37.991545+0000","last_unstale":"2023-12-20T04:28:38.411161+0000","last_undegraded":"2023-12-20T04:28:38.411161+0000","last_fullsized":"2023-12-20T04:28:38.411161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":802816,"data_stored":660219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":70833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":208896,"data_stored":69359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":69395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31260,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340580,"statfs":{"total":96636764160,"available":96604753920,"internally_reserved":0,"allocated":217088,"data_stored":70833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":212992,"data_stored":69429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":811008,"data_stored":660275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:28:42.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:28:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.652+0000 7f63455e9640 1 Processor -- start 2023-12-20T04:28:42.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.652+0000 7f63455e9640 1 -- start start 2023-12-20T04:28:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.652+0000 7f63455e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f634015cdb0 0x7f63400d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:42.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.652+0000 7f63455e9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f634015d190 con 0x7f63400cfc80 2023-12-20T04:28:42.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.652+0000 7f63455e9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63400d16a0 con 0x7f63400d2740 2023-12-20T04:28:42.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.652+0000 7f63455e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63400d1980 con 0x7f634015cdb0 2023-12-20T04:28:42.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.652+0000 7f633f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f63400cfc80 0x7f63400b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35302/0 (socket says 172.21.15.32:35302) 2023-12-20T04:28:42.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633f7fe640 1 -- 172.21.15.32:0/445177823 learned_addr learned my addr 172.21.15.32:0/445177823 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:42.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633effd640 1 --2- 172.21.15.32:0/445177823 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f634015cdb0 0x7f63400d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633e7fc640 1 -- 172.21.15.32:0/445177823 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2828927512 0 0) 0x7f634015d190 con 0x7f63400cfc80 2023-12-20T04:28:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633e7fc640 1 -- 172.21.15.32:0/445177823 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f632c003530 con 0x7f63400cfc80 2023-12-20T04:28:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633e7fc640 1 -- 172.21.15.32:0/445177823 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277331011 0 0) 0x7f63400d16a0 con 0x7f63400d2740 2023-12-20T04:28:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633e7fc640 1 -- 172.21.15.32:0/445177823 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f634015d190 con 0x7f63400d2740 2023-12-20T04:28:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633effd640 1 -- 172.21.15.32:0/445177823 >> v1:172.21.15.39:6789/0 conn(0x7f63400d2740 legacy=0x7f63400cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633effd640 1 -- 172.21.15.32:0/445177823 >> v1:172.21.15.32:6789/0 conn(0x7f63400cfc80 legacy=0x7f63400b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633effd640 1 -- 172.21.15.32:0/445177823 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63400d1c60 con 0x7f634015cdb0 2023-12-20T04:28:42.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633effd640 1 --2- 172.21.15.32:0/445177823 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f634015cdb0 0x7f63400d0be0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6330009dc0 tx=0x7f633001f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e47f981557d2db6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:42.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633e7fc640 1 -- 172.21.15.32:0/445177823 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300af600 con 0x7f634015cdb0 2023-12-20T04:28:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633e7fc640 1 -- 172.21.15.32:0/445177823 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63300ab050 con 0x7f634015cdb0 2023-12-20T04:28:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.656+0000 7f633e7fc640 1 -- 172.21.15.32:0/445177823 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63300af980 con 0x7f634015cdb0 2023-12-20T04:28:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 -- 172.21.15.32:0/445177823 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f634015cdb0 msgr2=0x7f63400d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 --2- 172.21.15.32:0/445177823 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f634015cdb0 0x7f63400d0be0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6330009dc0 tx=0x7f633001f150 comp rx=0 tx=0).stop 2023-12-20T04:28:42.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 -- 172.21.15.32:0/445177823 shutdown_connections 2023-12-20T04:28:42.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 --2- 172.21.15.32:0/445177823 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f634015cdb0 0x7f63400d0be0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 -- 172.21.15.32:0/445177823 >> 172.21.15.32:0/445177823 conn(0x7f6340065ee0 msgr2=0x7f634006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:42.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 -- 172.21.15.32:0/445177823 shutdown_connections 2023-12-20T04:28:42.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 -- 172.21.15.32:0/445177823 wait complete. 2023-12-20T04:28:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 Processor -- start 2023-12-20T04:28:42.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 -- start start 2023-12-20T04:28:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63400cfc80 0x7f6340149050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63400d2740 0x7f634014ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634015cdb0 0x7f63400caf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63400c7520 con 0x7f63400d2740 2023-12-20T04:28:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634014a2f0 con 0x7f634015cdb0 2023-12-20T04:28:42.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63455e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6340149d30 con 0x7f63400cfc80 2023-12-20T04:28:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63367fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63400d2740 0x7f634014ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63367fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63400d2740 0x7f634014ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48146/0 (socket says 172.21.15.32:48146) 2023-12-20T04:28:42.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.660+0000 7f63367fc640 1 -- 172.21.15.32:0/3492652704 learned_addr learned my addr 172.21.15.32:0/3492652704 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f633effd640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63400cfc80 0x7f6340149050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f633f7fe640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634015cdb0 0x7f63400caf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f63367fc640 1 -- 172.21.15.32:0/3492652704 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63400cfc80 msgr2=0x7f6340149050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f63367fc640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63400cfc80 0x7f6340149050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f63367fc640 1 -- 172.21.15.32:0/3492652704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634015cdb0 msgr2=0x7f63400caf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f63367fc640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634015cdb0 0x7f63400caf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f63367fc640 1 -- 172.21.15.32:0/3492652704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6330009a70 con 0x7f63400d2740 2023-12-20T04:28:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f63367fc640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63400d2740 0x7f634014ad70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f632800c500 tx=0x7f632800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f6337fff640 1 -- 172.21.15.32:0/3492652704 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280052d0 con 0x7f63400d2740 2023-12-20T04:28:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f6337fff640 1 -- 172.21.15.32:0/3492652704 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6328005450 con 0x7f63400d2740 2023-12-20T04:28:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f6337fff640 1 -- 172.21.15.32:0/3492652704 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6328012870 con 0x7f63400d2740 2023-12-20T04:28:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f63455e9640 1 -- 172.21.15.32:0/3492652704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634014a010 con 0x7f63400d2740 2023-12-20T04:28:42.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f63455e9640 1 -- 172.21.15.32:0/3492652704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f634014b2b0 con 0x7f63400d2740 2023-12-20T04:28:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f63455e9640 1 -- 172.21.15.32:0/3492652704 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63400c7660 con 0x7f63400d2740 2023-12-20T04:28:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f6337fff640 1 -- 172.21.15.32:0/3492652704 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f63280129d0 con 0x7f63400d2740 2023-12-20T04:28:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f6337fff640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065ae0 0x7f6320067fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f6337fff640 1 -- 172.21.15.32:0/3492652704 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6328082d70 con 0x7f63400d2740 2023-12-20T04:28:42.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.664+0000 7f633effd640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065ae0 0x7f6320067fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:42.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.668+0000 7f633effd640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065ae0 0x7f6320067fa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63300b1000 tx=0x7f63300aa000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:42.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.672+0000 7f6337fff640 1 -- 172.21.15.32:0/3492652704 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6328050720 con 0x7f63400d2740 2023-12-20T04:28:42.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.828+0000 7f63455e9640 1 -- 172.21.15.32:0/3492652704 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f63400d2220 con 0x7f6320065ae0 2023-12-20T04:28:42.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f6337fff640 1 -- 172.21.15.32:0/3492652704 <== mgr.4104 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264612 (secure 0 0 0) 0x7f63300af680 con 0x7f6320065ae0 2023-12-20T04:28:42.837 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:42.839 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 -- 172.21.15.32:0/3492652704 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065ae0 msgr2=0x7f6320067fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065ae0 0x7f6320067fa0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f63300b1000 tx=0x7f63300aa000 comp rx=0 tx=0).stop 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 -- 172.21.15.32:0/3492652704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63400d2740 msgr2=0x7f634014ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63400d2740 0x7f634014ad70 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f632800c500 tx=0x7f632800c9c0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 -- 172.21.15.32:0/3492652704 shutdown_connections 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065ae0 0x7f6320067fa0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634015cdb0 0x7f63400caf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63400d2740 0x7f634014ad70 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 --2- 172.21.15.32:0/3492652704 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63400cfc80 0x7f6340149050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 -- 172.21.15.32:0/3492652704 >> 172.21.15.32:0/3492652704 conn(0x7f6340065ee0 msgr2=0x7f63400d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 -- 172.21.15.32:0/3492652704 shutdown_connections 2023-12-20T04:28:42.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:42.836+0000 7f63455e9640 1 -- 172.21.15.32:0/3492652704 wait complete. 2023-12-20T04:28:42.858 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-20T04:28:42.125170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":203,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":155,"ondisk_log_size":155,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":80,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":234824,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":231936,"kb_avail":754739896,"statfs":{"total":773094113280,"available":772853653504,"internally_reserved":0,"allocated":2891776,"data_stored":1739612,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":237502464},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":39,"apply_latency_ms":39,"commit_latency_ns":39000000,"apply_latency_ns":39000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.171780"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411165+0000","last_change":"2023-12-20T04:28:38.407268+0000","last_active":"2023-12-20T04:28:38.411165+0000","last_peered":"2023-12-20T04:28:38.411165+0000","last_clean":"2023-12-20T04:28:38.411165+0000","last_became_active":"2023-12-20T04:28:38.404491+0000","last_became_peered":"2023-12-20T04:28:38.404491+0000","last_unstale":"2023-12-20T04:28:38.411165+0000","last_undegraded":"2023-12-20T04:28:38.411165+0000","last_fullsized":"2023-12-20T04:28:38.411165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:36.363864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379971+0000","last_change":"2023-12-20T04:28:38.390082+0000","last_active":"2023-12-20T04:28:39.379971+0000","last_peered":"2023-12-20T04:28:39.379971+0000","last_clean":"2023-12-20T04:28:39.379971+0000","last_became_active":"2023-12-20T04:28:38.389688+0000","last_became_peered":"2023-12-20T04:28:38.389688+0000","last_unstale":"2023-12-20T04:28:39.379971+0000","last_undegraded":"2023-12-20T04:28:39.379971+0000","last_fullsized":"2023-12-20T04:28:39.379971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.395049+0000","last_change":"2023-12-20T04:28:38.388981+0000","last_active":"2023-12-20T04:28:38.395049+0000","last_peered":"2023-12-20T04:28:38.395049+0000","last_clean":"2023-12-20T04:28:38.395049+0000","last_became_active":"2023-12-20T04:28:38.388734+0000","last_became_peered":"2023-12-20T04:28:38.388734+0000","last_unstale":"2023-12-20T04:28:38.395049+0000","last_undegraded":"2023-12-20T04:28:38.395049+0000","last_fullsized":"2023-12-20T04:28:38.395049+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385490+0000","last_change":"2023-12-20T04:28:38.384727+0000","last_active":"2023-12-20T04:28:39.385490+0000","last_peered":"2023-12-20T04:28:39.385490+0000","last_clean":"2023-12-20T04:28:39.385490+0000","last_became_active":"2023-12-20T04:28:38.384329+0000","last_became_peered":"2023-12-20T04:28:38.384329+0000","last_unstale":"2023-12-20T04:28:39.385490+0000","last_undegraded":"2023-12-20T04:28:39.385490+0000","last_fullsized":"2023-12-20T04:28:39.385490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:11:32.618631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.407744+0000","last_change":"2023-12-20T04:28:38.397089+0000","last_active":"2023-12-20T04:28:38.407744+0000","last_peered":"2023-12-20T04:28:38.407744+0000","last_clean":"2023-12-20T04:28:38.407744+0000","last_became_active":"2023-12-20T04:28:38.396774+0000","last_became_peered":"2023-12-20T04:28:38.396774+0000","last_unstale":"2023-12-20T04:28:38.407744+0000","last_undegraded":"2023-12-20T04:28:38.407744+0000","last_fullsized":"2023-12-20T04:28:38.407744+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386104+0000","last_change":"2023-12-20T04:28:38.395384+0000","last_active":"2023-12-20T04:28:39.386104+0000","last_peered":"2023-12-20T04:28:39.386104+0000","last_clean":"2023-12-20T04:28:39.386104+0000","last_became_active":"2023-12-20T04:28:38.394254+0000","last_became_peered":"2023-12-20T04:28:38.394254+0000","last_unstale":"2023-12-20T04:28:39.386104+0000","last_undegraded":"2023-12-20T04:28:39.386104+0000","last_fullsized":"2023-12-20T04:28:39.386104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:55:26.717187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411155+0000","last_change":"2023-12-20T04:28:38.404597+0000","last_active":"2023-12-20T04:28:38.411155+0000","last_peered":"2023-12-20T04:28:38.411155+0000","last_clean":"2023-12-20T04:28:38.411155+0000","last_became_active":"2023-12-20T04:28:38.404108+0000","last_became_peered":"2023-12-20T04:28:38.404108+0000","last_unstale":"2023-12-20T04:28:38.411155+0000","last_undegraded":"2023-12-20T04:28:38.411155+0000","last_fullsized":"2023-12-20T04:28:38.411155+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:41.399788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.387621+0000","last_change":"2023-12-20T04:28:38.385150+0000","last_active":"2023-12-20T04:28:38.387621+0000","last_peered":"2023-12-20T04:28:38.387621+0000","last_clean":"2023-12-20T04:28:38.387621+0000","last_became_active":"2023-12-20T04:28:38.384885+0000","last_became_peered":"2023-12-20T04:28:38.384885+0000","last_unstale":"2023-12-20T04:28:38.387621+0000","last_undegraded":"2023-12-20T04:28:38.387621+0000","last_fullsized":"2023-12-20T04:28:38.387621+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.410227+0000","last_change":"2023-12-20T04:28:38.399891+0000","last_active":"2023-12-20T04:28:38.410227+0000","last_peered":"2023-12-20T04:28:38.410227+0000","last_clean":"2023-12-20T04:28:38.410227+0000","last_became_active":"2023-12-20T04:28:38.399599+0000","last_became_peered":"2023-12-20T04:28:38.399599+0000","last_unstale":"2023-12-20T04:28:38.410227+0000","last_undegraded":"2023-12-20T04:28:38.410227+0000","last_fullsized":"2023-12-20T04:28:38.410227+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:46:52.340773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.406850+0000","last_change":"2023-12-20T04:28:38.398573+0000","last_active":"2023-12-20T04:28:38.406850+0000","last_peered":"2023-12-20T04:28:38.406850+0000","last_clean":"2023-12-20T04:28:38.406850+0000","last_became_active":"2023-12-20T04:28:38.398162+0000","last_became_peered":"2023-12-20T04:28:38.398162+0000","last_unstale":"2023-12-20T04:28:38.406850+0000","last_undegraded":"2023-12-20T04:28:38.406850+0000","last_fullsized":"2023-12-20T04:28:38.406850+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411854+0000","last_change":"2023-12-20T04:28:38.405208+0000","last_active":"2023-12-20T04:28:38.411854+0000","last_peered":"2023-12-20T04:28:38.411854+0000","last_clean":"2023-12-20T04:28:38.411854+0000","last_became_active":"2023-12-20T04:28:38.404933+0000","last_became_peered":"2023-12-20T04:28:38.404933+0000","last_unstale":"2023-12-20T04:28:38.411854+0000","last_undegraded":"2023-12-20T04:28:38.411854+0000","last_fullsized":"2023-12-20T04:28:38.411854+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:50.720992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386165+0000","last_change":"2023-12-20T04:28:38.412922+0000","last_active":"2023-12-20T04:28:39.386165+0000","last_peered":"2023-12-20T04:28:39.386165+0000","last_clean":"2023-12-20T04:28:39.386165+0000","last_became_active":"2023-12-20T04:28:38.412505+0000","last_became_peered":"2023-12-20T04:28:38.412505+0000","last_unstale":"2023-12-20T04:28:39.386165+0000","last_undegraded":"2023-12-20T04:28:39.386165+0000","last_fullsized":"2023-12-20T04:28:39.386165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:04:59.607569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.394176+0000","last_change":"2023-12-20T04:28:38.390684+0000","last_active":"2023-12-20T04:28:38.394176+0000","last_peered":"2023-12-20T04:28:38.394176+0000","last_clean":"2023-12-20T04:28:38.394176+0000","last_became_active":"2023-12-20T04:28:38.390483+0000","last_became_peered":"2023-12-20T04:28:38.390483+0000","last_unstale":"2023-12-20T04:28:38.394176+0000","last_undegraded":"2023-12-20T04:28:38.394176+0000","last_fullsized":"2023-12-20T04:28:38.394176+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:55:26.717187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.392244+0000","last_change":"2023-12-20T04:28:38.379167+0000","last_active":"2023-12-20T04:28:38.392244+0000","last_peered":"2023-12-20T04:28:38.392244+0000","last_clean":"2023-12-20T04:28:38.392244+0000","last_became_active":"2023-12-20T04:28:38.378677+0000","last_became_peered":"2023-12-20T04:28:38.378677+0000","last_unstale":"2023-12-20T04:28:38.392244+0000","last_undegraded":"2023-12-20T04:28:38.392244+0000","last_fullsized":"2023-12-20T04:28:38.392244+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379652+0000","last_change":"2023-12-20T04:28:38.391213+0000","last_active":"2023-12-20T04:28:39.379652+0000","last_peered":"2023-12-20T04:28:39.379652+0000","last_clean":"2023-12-20T04:28:39.379652+0000","last_became_active":"2023-12-20T04:28:38.390937+0000","last_became_peered":"2023-12-20T04:28:38.390937+0000","last_unstale":"2023-12-20T04:28:39.379652+0000","last_undegraded":"2023-12-20T04:28:39.379652+0000","last_fullsized":"2023-12-20T04:28:39.379652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385448+0000","last_change":"2023-12-20T04:28:38.403378+0000","last_active":"2023-12-20T04:28:39.385448+0000","last_peered":"2023-12-20T04:28:39.385448+0000","last_clean":"2023-12-20T04:28:39.385448+0000","last_became_active":"2023-12-20T04:28:38.403070+0000","last_became_peered":"2023-12-20T04:28:38.403070+0000","last_unstale":"2023-12-20T04:28:39.385448+0000","last_undegraded":"2023-12-20T04:28:39.385448+0000","last_fullsized":"2023-12-20T04:28:39.385448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:48:18.553482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385977+0000","last_change":"2023-12-20T04:28:38.399610+0000","last_active":"2023-12-20T04:28:39.385977+0000","last_peered":"2023-12-20T04:28:39.385977+0000","last_clean":"2023-12-20T04:28:39.385977+0000","last_became_active":"2023-12-20T04:28:38.399061+0000","last_became_peered":"2023-12-20T04:28:38.399061+0000","last_unstale":"2023-12-20T04:28:39.385977+0000","last_undegraded":"2023-12-20T04:28:39.385977+0000","last_fullsized":"2023-12-20T04:28:39.385977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.403366+0000","last_change":"2023-12-20T04:28:38.398099+0000","last_active":"2023-12-20T04:28:38.403366+0000","last_peered":"2023-12-20T04:28:38.403366+0000","last_clean":"2023-12-20T04:28:38.403366+0000","last_became_active":"2023-12-20T04:28:38.397333+0000","last_became_peered":"2023-12-20T04:28:38.397333+0000","last_unstale":"2023-12-20T04:28:38.403366+0000","last_undegraded":"2023-12-20T04:28:38.403366+0000","last_fullsized":"2023-12-20T04:28:38.403366+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:12.041027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.407804+0000","last_change":"2023-12-20T04:28:38.397182+0000","last_active":"2023-12-20T04:28:38.407804+0000","last_peered":"2023-12-20T04:28:38.407804+0000","last_clean":"2023-12-20T04:28:38.407804+0000","last_became_active":"2023-12-20T04:28:38.396800+0000","last_became_peered":"2023-12-20T04:28:38.396800+0000","last_unstale":"2023-12-20T04:28:38.407804+0000","last_undegraded":"2023-12-20T04:28:38.407804+0000","last_fullsized":"2023-12-20T04:28:38.407804+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411935+0000","last_change":"2023-12-20T04:28:38.410245+0000","last_active":"2023-12-20T04:28:38.411935+0000","last_peered":"2023-12-20T04:28:38.411935+0000","last_clean":"2023-12-20T04:28:38.411935+0000","last_became_active":"2023-12-20T04:28:38.403138+0000","last_became_peered":"2023-12-20T04:28:38.403138+0000","last_unstale":"2023-12-20T04:28:38.411935+0000","last_undegraded":"2023-12-20T04:28:38.411935+0000","last_fullsized":"2023-12-20T04:28:38.411935+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.403794+0000","last_change":"2023-12-20T04:28:38.400262+0000","last_active":"2023-12-20T04:28:38.403794+0000","last_peered":"2023-12-20T04:28:38.403794+0000","last_clean":"2023-12-20T04:28:38.403794+0000","last_became_active":"2023-12-20T04:28:38.400021+0000","last_became_peered":"2023-12-20T04:28:38.400021+0000","last_unstale":"2023-12-20T04:28:38.403794+0000","last_undegraded":"2023-12-20T04:28:38.403794+0000","last_fullsized":"2023-12-20T04:28:38.403794+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386133+0000","last_change":"2023-12-20T04:28:38.397515+0000","last_active":"2023-12-20T04:28:39.386133+0000","last_peered":"2023-12-20T04:28:39.386133+0000","last_clean":"2023-12-20T04:28:39.386133+0000","last_became_active":"2023-12-20T04:28:38.397234+0000","last_became_peered":"2023-12-20T04:28:38.397234+0000","last_unstale":"2023-12-20T04:28:39.386133+0000","last_undegraded":"2023-12-20T04:28:39.386133+0000","last_fullsized":"2023-12-20T04:28:39.386133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.392860+0000","last_change":"2023-12-20T04:28:38.390587+0000","last_active":"2023-12-20T04:28:38.392860+0000","last_peered":"2023-12-20T04:28:38.392860+0000","last_clean":"2023-12-20T04:28:38.392860+0000","last_became_active":"2023-12-20T04:28:38.390256+0000","last_became_peered":"2023-12-20T04:28:38.390256+0000","last_unstale":"2023-12-20T04:28:38.392860+0000","last_undegraded":"2023-12-20T04:28:38.392860+0000","last_fullsized":"2023-12-20T04:28:38.392860+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385781+0000","last_change":"2023-12-20T04:28:38.390087+0000","last_active":"2023-12-20T04:28:39.385781+0000","last_peered":"2023-12-20T04:28:39.385781+0000","last_clean":"2023-12-20T04:28:39.385781+0000","last_became_active":"2023-12-20T04:28:38.389881+0000","last_became_peered":"2023-12-20T04:28:38.389881+0000","last_unstale":"2023-12-20T04:28:39.385781+0000","last_undegraded":"2023-12-20T04:28:39.385781+0000","last_fullsized":"2023-12-20T04:28:39.385781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:09.976618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.398819+0000","last_change":"2023-12-20T04:28:38.396543+0000","last_active":"2023-12-20T04:28:38.398819+0000","last_peered":"2023-12-20T04:28:38.398819+0000","last_clean":"2023-12-20T04:28:38.398819+0000","last_became_active":"2023-12-20T04:28:38.396247+0000","last_became_peered":"2023-12-20T04:28:38.396247+0000","last_unstale":"2023-12-20T04:28:38.398819+0000","last_undegraded":"2023-12-20T04:28:38.398819+0000","last_fullsized":"2023-12-20T04:28:38.398819+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.406875+0000","last_change":"2023-12-20T04:28:38.401846+0000","last_active":"2023-12-20T04:28:38.406875+0000","last_peered":"2023-12-20T04:28:38.406875+0000","last_clean":"2023-12-20T04:28:38.406875+0000","last_became_active":"2023-12-20T04:28:38.401255+0000","last_became_peered":"2023-12-20T04:28:38.401255+0000","last_unstale":"2023-12-20T04:28:38.406875+0000","last_undegraded":"2023-12-20T04:28:38.406875+0000","last_fullsized":"2023-12-20T04:28:38.406875+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379844+0000","last_change":"2023-12-20T04:28:38.385251+0000","last_active":"2023-12-20T04:28:39.379844+0000","last_peered":"2023-12-20T04:28:39.379844+0000","last_clean":"2023-12-20T04:28:39.379844+0000","last_became_active":"2023-12-20T04:28:38.384990+0000","last_became_peered":"2023-12-20T04:28:38.384990+0000","last_unstale":"2023-12-20T04:28:39.379844+0000","last_undegraded":"2023-12-20T04:28:39.379844+0000","last_fullsized":"2023-12-20T04:28:39.379844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.398594+0000","last_change":"2023-12-20T04:28:38.394851+0000","last_active":"2023-12-20T04:28:38.398594+0000","last_peered":"2023-12-20T04:28:38.398594+0000","last_clean":"2023-12-20T04:28:38.398594+0000","last_became_active":"2023-12-20T04:28:38.394584+0000","last_became_peered":"2023-12-20T04:28:38.394584+0000","last_unstale":"2023-12-20T04:28:38.398594+0000","last_undegraded":"2023-12-20T04:28:38.398594+0000","last_fullsized":"2023-12-20T04:28:38.398594+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.400461+0000","last_change":"2023-12-20T04:28:38.383584+0000","last_active":"2023-12-20T04:28:38.400461+0000","last_peered":"2023-12-20T04:28:38.400461+0000","last_clean":"2023-12-20T04:28:38.400461+0000","last_became_active":"2023-12-20T04:28:38.383310+0000","last_became_peered":"2023-12-20T04:28:38.383310+0000","last_unstale":"2023-12-20T04:28:38.400461+0000","last_undegraded":"2023-12-20T04:28:38.400461+0000","last_fullsized":"2023-12-20T04:28:38.400461+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386215+0000","last_change":"2023-12-20T04:28:38.388220+0000","last_active":"2023-12-20T04:28:39.386215+0000","last_peered":"2023-12-20T04:28:39.386215+0000","last_clean":"2023-12-20T04:28:39.386215+0000","last_became_active":"2023-12-20T04:28:38.386044+0000","last_became_peered":"2023-12-20T04:28:38.386044+0000","last_unstale":"2023-12-20T04:28:39.386215+0000","last_undegraded":"2023-12-20T04:28:39.386215+0000","last_fullsized":"2023-12-20T04:28:39.386215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:51.073749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.396856+0000","last_change":"2023-12-20T04:28:38.384460+0000","last_active":"2023-12-20T04:28:38.396856+0000","last_peered":"2023-12-20T04:28:38.396856+0000","last_clean":"2023-12-20T04:28:38.396856+0000","last_became_active":"2023-12-20T04:28:38.384190+0000","last_became_peered":"2023-12-20T04:28:38.384190+0000","last_unstale":"2023-12-20T04:28:38.396856+0000","last_undegraded":"2023-12-20T04:28:38.396856+0000","last_fullsized":"2023-12-20T04:28:38.396856+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.409083+0000","last_change":"2023-12-20T04:28:38.403381+0000","last_active":"2023-12-20T04:28:38.409083+0000","last_peered":"2023-12-20T04:28:38.409083+0000","last_clean":"2023-12-20T04:28:38.409083+0000","last_became_active":"2023-12-20T04:28:38.402953+0000","last_became_peered":"2023-12-20T04:28:38.402953+0000","last_unstale":"2023-12-20T04:28:38.409083+0000","last_undegraded":"2023-12-20T04:28:38.409083+0000","last_fullsized":"2023-12-20T04:28:38.409083+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385741+0000","last_change":"2023-12-20T04:28:38.392967+0000","last_active":"2023-12-20T04:28:39.385741+0000","last_peered":"2023-12-20T04:28:39.385741+0000","last_clean":"2023-12-20T04:28:39.385741+0000","last_became_active":"2023-12-20T04:28:38.392657+0000","last_became_peered":"2023-12-20T04:28:38.392657+0000","last_unstale":"2023-12-20T04:28:39.385741+0000","last_undegraded":"2023-12-20T04:28:39.385741+0000","last_fullsized":"2023-12-20T04:28:39.385741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:35.897773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.393945+0000","last_change":"2023-12-20T04:28:37.983099+0000","last_active":"2023-12-20T04:28:38.393945+0000","last_peered":"2023-12-20T04:28:38.393945+0000","last_clean":"2023-12-20T04:28:38.393945+0000","last_became_active":"2023-12-20T04:28:37.982857+0000","last_became_peered":"2023-12-20T04:28:37.982857+0000","last_unstale":"2023-12-20T04:28:38.393945+0000","last_undegraded":"2023-12-20T04:28:38.393945+0000","last_fullsized":"2023-12-20T04:28:38.393945+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.408543+0000","last_change":"2023-12-20T04:28:38.407295+0000","last_active":"2023-12-20T04:28:38.408543+0000","last_peered":"2023-12-20T04:28:38.408543+0000","last_clean":"2023-12-20T04:28:38.408543+0000","last_became_active":"2023-12-20T04:28:38.406963+0000","last_became_peered":"2023-12-20T04:28:38.406963+0000","last_unstale":"2023-12-20T04:28:38.408543+0000","last_undegraded":"2023-12-20T04:28:38.408543+0000","last_fullsized":"2023-12-20T04:28:38.408543+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:50.720992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.406335+0000","last_change":"2023-12-20T04:28:37.987627+0000","last_active":"2023-12-20T04:28:38.406335+0000","last_peered":"2023-12-20T04:28:38.406335+0000","last_clean":"2023-12-20T04:28:38.406335+0000","last_became_active":"2023-12-20T04:28:37.987309+0000","last_became_peered":"2023-12-20T04:28:37.987309+0000","last_unstale":"2023-12-20T04:28:38.406335+0000","last_undegraded":"2023-12-20T04:28:38.406335+0000","last_fullsized":"2023-12-20T04:28:38.406335+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.372901+0000","last_change":"2023-12-20T04:28:37.996411+0000","last_active":"2023-12-20T04:28:38.372901+0000","last_peered":"2023-12-20T04:28:38.372901+0000","last_clean":"2023-12-20T04:28:38.372901+0000","last_became_active":"2023-12-20T04:28:37.996051+0000","last_became_peered":"2023-12-20T04:28:37.996051+0000","last_unstale":"2023-12-20T04:28:38.372901+0000","last_undegraded":"2023-12-20T04:28:38.372901+0000","last_fullsized":"2023-12-20T04:28:38.372901+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.399071+0000","last_change":"2023-12-20T04:28:38.396714+0000","last_active":"2023-12-20T04:28:38.399071+0000","last_peered":"2023-12-20T04:28:38.399071+0000","last_clean":"2023-12-20T04:28:38.399071+0000","last_became_active":"2023-12-20T04:28:38.396337+0000","last_became_peered":"2023-12-20T04:28:38.396337+0000","last_unstale":"2023-12-20T04:28:38.399071+0000","last_undegraded":"2023-12-20T04:28:38.399071+0000","last_fullsized":"2023-12-20T04:28:38.399071+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385551+0000","last_change":"2023-12-20T04:28:37.982942+0000","last_active":"2023-12-20T04:28:39.385551+0000","last_peered":"2023-12-20T04:28:39.385551+0000","last_clean":"2023-12-20T04:28:39.385551+0000","last_became_active":"2023-12-20T04:28:37.982287+0000","last_became_peered":"2023-12-20T04:28:37.982287+0000","last_unstale":"2023-12-20T04:28:39.385551+0000","last_undegraded":"2023-12-20T04:28:39.385551+0000","last_fullsized":"2023-12-20T04:28:39.385551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379613+0000","last_change":"2023-12-20T04:28:38.380510+0000","last_active":"2023-12-20T04:28:39.379613+0000","last_peered":"2023-12-20T04:28:39.379613+0000","last_clean":"2023-12-20T04:28:39.379613+0000","last_became_active":"2023-12-20T04:28:38.379896+0000","last_became_peered":"2023-12-20T04:28:38.379896+0000","last_unstale":"2023-12-20T04:28:39.379613+0000","last_undegraded":"2023-12-20T04:28:39.379613+0000","last_fullsized":"2023-12-20T04:28:39.379613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.391801+0000","last_change":"2023-12-20T04:28:37.992369+0000","last_active":"2023-12-20T04:28:38.391801+0000","last_peered":"2023-12-20T04:28:38.391801+0000","last_clean":"2023-12-20T04:28:38.391801+0000","last_became_active":"2023-12-20T04:28:37.992065+0000","last_became_peered":"2023-12-20T04:28:37.992065+0000","last_unstale":"2023-12-20T04:28:38.391801+0000","last_undegraded":"2023-12-20T04:28:38.391801+0000","last_fullsized":"2023-12-20T04:28:38.391801+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385551+0000","last_change":"2023-12-20T04:28:38.390943+0000","last_active":"2023-12-20T04:28:39.385551+0000","last_peered":"2023-12-20T04:28:39.385551+0000","last_clean":"2023-12-20T04:28:39.385551+0000","last_became_active":"2023-12-20T04:28:38.390661+0000","last_became_peered":"2023-12-20T04:28:38.390661+0000","last_unstale":"2023-12-20T04:28:39.385551+0000","last_undegraded":"2023-12-20T04:28:39.385551+0000","last_fullsized":"2023-12-20T04:28:39.385551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:06.804397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385691+0000","last_change":"2023-12-20T04:28:37.986949+0000","last_active":"2023-12-20T04:28:39.385691+0000","last_peered":"2023-12-20T04:28:39.385691+0000","last_clean":"2023-12-20T04:28:39.385691+0000","last_became_active":"2023-12-20T04:28:37.986664+0000","last_became_peered":"2023-12-20T04:28:37.986664+0000","last_unstale":"2023-12-20T04:28:39.385691+0000","last_undegraded":"2023-12-20T04:28:39.385691+0000","last_fullsized":"2023-12-20T04:28:39.385691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.405825+0000","last_change":"2023-12-20T04:28:38.403067+0000","last_active":"2023-12-20T04:28:38.405825+0000","last_peered":"2023-12-20T04:28:38.405825+0000","last_clean":"2023-12-20T04:28:38.405825+0000","last_became_active":"2023-12-20T04:28:38.402312+0000","last_became_peered":"2023-12-20T04:28:38.402312+0000","last_unstale":"2023-12-20T04:28:38.405825+0000","last_undegraded":"2023-12-20T04:28:38.405825+0000","last_fullsized":"2023-12-20T04:28:38.405825+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379842+0000","last_change":"2023-12-20T04:28:37.989273+0000","last_active":"2023-12-20T04:28:39.379842+0000","last_peered":"2023-12-20T04:28:39.379842+0000","last_clean":"2023-12-20T04:28:39.379842+0000","last_became_active":"2023-12-20T04:28:37.988921+0000","last_became_peered":"2023-12-20T04:28:37.988921+0000","last_unstale":"2023-12-20T04:28:39.379842+0000","last_undegraded":"2023-12-20T04:28:39.379842+0000","last_fullsized":"2023-12-20T04:28:39.379842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386234+0000","last_change":"2023-12-20T04:28:38.394680+0000","last_active":"2023-12-20T04:28:39.386234+0000","last_peered":"2023-12-20T04:28:39.386234+0000","last_clean":"2023-12-20T04:28:39.386234+0000","last_became_active":"2023-12-20T04:28:38.394369+0000","last_became_peered":"2023-12-20T04:28:38.394369+0000","last_unstale":"2023-12-20T04:28:39.386234+0000","last_undegraded":"2023-12-20T04:28:39.386234+0000","last_fullsized":"2023-12-20T04:28:39.386234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:12.041027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":16,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.415038+0000","last_change":"2023-12-20T04:28:37.992630+0000","last_active":"2023-12-20T04:28:38.415038+0000","last_peered":"2023-12-20T04:28:38.415038+0000","last_clean":"2023-12-20T04:28:38.415038+0000","last_became_active":"2023-12-20T04:28:37.989328+0000","last_became_peered":"2023-12-20T04:28:37.989328+0000","last_unstale":"2023-12-20T04:28:38.415038+0000","last_undegraded":"2023-12-20T04:28:38.415038+0000","last_fullsized":"2023-12-20T04:28:38.415038+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.393559+0000","last_change":"2023-12-20T04:28:38.390252+0000","last_active":"2023-12-20T04:28:38.393559+0000","last_peered":"2023-12-20T04:28:38.393559+0000","last_clean":"2023-12-20T04:28:38.393559+0000","last_became_active":"2023-12-20T04:28:38.389945+0000","last_became_peered":"2023-12-20T04:28:38.389945+0000","last_unstale":"2023-12-20T04:28:38.393559+0000","last_undegraded":"2023-12-20T04:28:38.393559+0000","last_fullsized":"2023-12-20T04:28:38.393559+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:11:32.618631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411064+0000","last_change":"2023-12-20T04:28:37.979978+0000","last_active":"2023-12-20T04:28:38.411064+0000","last_peered":"2023-12-20T04:28:38.411064+0000","last_clean":"2023-12-20T04:28:38.411064+0000","last_became_active":"2023-12-20T04:28:37.979639+0000","last_became_peered":"2023-12-20T04:28:37.979639+0000","last_unstale":"2023-12-20T04:28:38.411064+0000","last_undegraded":"2023-12-20T04:28:38.411064+0000","last_fullsized":"2023-12-20T04:28:38.411064+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.403307+0000","last_change":"2023-12-20T04:28:38.400246+0000","last_active":"2023-12-20T04:28:38.403307+0000","last_peered":"2023-12-20T04:28:38.403307+0000","last_clean":"2023-12-20T04:28:38.403307+0000","last_became_active":"2023-12-20T04:28:38.399932+0000","last_became_peered":"2023-12-20T04:28:38.399932+0000","last_unstale":"2023-12-20T04:28:38.403307+0000","last_undegraded":"2023-12-20T04:28:38.403307+0000","last_fullsized":"2023-12-20T04:28:38.403307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:46:19.275821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386216+0000","last_change":"2023-12-20T04:28:37.985893+0000","last_active":"2023-12-20T04:28:39.386216+0000","last_peered":"2023-12-20T04:28:39.386216+0000","last_clean":"2023-12-20T04:28:39.386216+0000","last_became_active":"2023-12-20T04:28:37.985605+0000","last_became_peered":"2023-12-20T04:28:37.985605+0000","last_unstale":"2023-12-20T04:28:39.386216+0000","last_undegraded":"2023-12-20T04:28:39.386216+0000","last_fullsized":"2023-12-20T04:28:39.386216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411881+0000","last_change":"2023-12-20T04:28:38.409446+0000","last_active":"2023-12-20T04:28:38.411881+0000","last_peered":"2023-12-20T04:28:38.411881+0000","last_clean":"2023-12-20T04:28:38.411881+0000","last_became_active":"2023-12-20T04:28:38.409174+0000","last_became_peered":"2023-12-20T04:28:38.409174+0000","last_unstale":"2023-12-20T04:28:38.411881+0000","last_undegraded":"2023-12-20T04:28:38.411881+0000","last_fullsized":"2023-12-20T04:28:38.411881+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:51.073749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.390234+0000","last_change":"2023-12-20T04:28:37.986147+0000","last_active":"2023-12-20T04:28:38.390234+0000","last_peered":"2023-12-20T04:28:38.390234+0000","last_clean":"2023-12-20T04:28:38.390234+0000","last_became_active":"2023-12-20T04:28:37.985323+0000","last_became_peered":"2023-12-20T04:28:37.985323+0000","last_unstale":"2023-12-20T04:28:38.390234+0000","last_undegraded":"2023-12-20T04:28:38.390234+0000","last_fullsized":"2023-12-20T04:28:38.390234+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.409065+0000","last_change":"2023-12-20T04:28:38.403115+0000","last_active":"2023-12-20T04:28:38.409065+0000","last_peered":"2023-12-20T04:28:38.409065+0000","last_clean":"2023-12-20T04:28:38.409065+0000","last_became_active":"2023-12-20T04:28:38.402521+0000","last_became_peered":"2023-12-20T04:28:38.402521+0000","last_unstale":"2023-12-20T04:28:38.409065+0000","last_undegraded":"2023-12-20T04:28:38.409065+0000","last_fullsized":"2023-12-20T04:28:38.409065+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":18,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.398155+0000","last_change":"2023-12-20T04:28:37.970816+0000","last_active":"2023-12-20T04:28:38.398155+0000","last_peered":"2023-12-20T04:28:38.398155+0000","last_clean":"2023-12-20T04:28:38.398155+0000","last_became_active":"2023-12-20T04:28:37.970330+0000","last_became_peered":"2023-12-20T04:28:37.970330+0000","last_unstale":"2023-12-20T04:28:38.398155+0000","last_undegraded":"2023-12-20T04:28:38.398155+0000","last_fullsized":"2023-12-20T04:28:38.398155+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.408428+0000","last_change":"2023-12-20T04:28:38.398095+0000","last_active":"2023-12-20T04:28:38.408428+0000","last_peered":"2023-12-20T04:28:38.408428+0000","last_clean":"2023-12-20T04:28:38.408428+0000","last_became_active":"2023-12-20T04:28:38.397747+0000","last_became_peered":"2023-12-20T04:28:38.397747+0000","last_unstale":"2023-12-20T04:28:38.408428+0000","last_undegraded":"2023-12-20T04:28:38.408428+0000","last_fullsized":"2023-12-20T04:28:38.408428+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:15:10.280675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386532+0000","last_change":"2023-12-20T04:28:37.980101+0000","last_active":"2023-12-20T04:28:39.386532+0000","last_peered":"2023-12-20T04:28:39.386532+0000","last_clean":"2023-12-20T04:28:39.386532+0000","last_became_active":"2023-12-20T04:28:37.979770+0000","last_became_peered":"2023-12-20T04:28:37.979770+0000","last_unstale":"2023-12-20T04:28:39.386532+0000","last_undegraded":"2023-12-20T04:28:39.386532+0000","last_fullsized":"2023-12-20T04:28:39.386532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411904+0000","last_change":"2023-12-20T04:28:38.409708+0000","last_active":"2023-12-20T04:28:38.411904+0000","last_peered":"2023-12-20T04:28:38.411904+0000","last_clean":"2023-12-20T04:28:38.411904+0000","last_became_active":"2023-12-20T04:28:38.408900+0000","last_became_peered":"2023-12-20T04:28:38.408900+0000","last_unstale":"2023-12-20T04:28:38.411904+0000","last_undegraded":"2023-12-20T04:28:38.411904+0000","last_fullsized":"2023-12-20T04:28:38.411904+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:13.592803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.389306+0000","last_change":"2023-12-20T04:28:37.985187+0000","last_active":"2023-12-20T04:28:38.389306+0000","last_peered":"2023-12-20T04:28:38.389306+0000","last_clean":"2023-12-20T04:28:38.389306+0000","last_became_active":"2023-12-20T04:28:37.984909+0000","last_became_peered":"2023-12-20T04:28:37.984909+0000","last_unstale":"2023-12-20T04:28:38.389306+0000","last_undegraded":"2023-12-20T04:28:38.389306+0000","last_fullsized":"2023-12-20T04:28:38.389306+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.408004+0000","last_change":"2023-12-20T04:28:38.397685+0000","last_active":"2023-12-20T04:28:38.408004+0000","last_peered":"2023-12-20T04:28:38.408004+0000","last_clean":"2023-12-20T04:28:38.408004+0000","last_became_active":"2023-12-20T04:28:38.396818+0000","last_became_peered":"2023-12-20T04:28:38.396818+0000","last_unstale":"2023-12-20T04:28:38.408004+0000","last_undegraded":"2023-12-20T04:28:38.408004+0000","last_fullsized":"2023-12-20T04:28:38.408004+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:09.976618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379930+0000","last_change":"2023-12-20T04:28:37.981598+0000","last_active":"2023-12-20T04:28:39.379930+0000","last_peered":"2023-12-20T04:28:39.379930+0000","last_clean":"2023-12-20T04:28:39.379930+0000","last_became_active":"2023-12-20T04:28:37.981319+0000","last_became_peered":"2023-12-20T04:28:37.981319+0000","last_unstale":"2023-12-20T04:28:39.379930+0000","last_undegraded":"2023-12-20T04:28:39.379930+0000","last_fullsized":"2023-12-20T04:28:39.379930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385613+0000","last_change":"2023-12-20T04:28:38.385270+0000","last_active":"2023-12-20T04:28:39.385613+0000","last_peered":"2023-12-20T04:28:39.385613+0000","last_clean":"2023-12-20T04:28:39.385613+0000","last_became_active":"2023-12-20T04:28:38.384740+0000","last_became_peered":"2023-12-20T04:28:38.384740+0000","last_unstale":"2023-12-20T04:28:39.385613+0000","last_undegraded":"2023-12-20T04:28:39.385613+0000","last_fullsized":"2023-12-20T04:28:39.385613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:13.592803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.372577+0000","last_change":"2023-12-20T04:28:37.991119+0000","last_active":"2023-12-20T04:28:38.372577+0000","last_peered":"2023-12-20T04:28:38.372577+0000","last_clean":"2023-12-20T04:28:38.372577+0000","last_became_active":"2023-12-20T04:28:37.990872+0000","last_became_peered":"2023-12-20T04:28:37.990872+0000","last_unstale":"2023-12-20T04:28:38.372577+0000","last_undegraded":"2023-12-20T04:28:38.372577+0000","last_fullsized":"2023-12-20T04:28:38.372577+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.407785+0000","last_change":"2023-12-20T04:28:38.394920+0000","last_active":"2023-12-20T04:28:38.407785+0000","last_peered":"2023-12-20T04:28:38.407785+0000","last_clean":"2023-12-20T04:28:38.407785+0000","last_became_active":"2023-12-20T04:28:38.394396+0000","last_became_peered":"2023-12-20T04:28:38.394396+0000","last_unstale":"2023-12-20T04:28:38.407785+0000","last_undegraded":"2023-12-20T04:28:38.407785+0000","last_fullsized":"2023-12-20T04:28:38.407785+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":163,"reported_epoch":19,"state":"active+clean","last_fresh":"2023-12-20T04:28:37.979939+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:28:37.979939+0000","last_peered":"2023-12-20T04:28:37.979939+0000","last_clean":"2023-12-20T04:28:37.979939+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:28:37.979939+0000","last_undegraded":"2023-12-20T04:28:37.979939+0000","last_fullsized":"2023-12-20T04:28:37.979939+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386014+0000","last_change":"2023-12-20T04:28:37.978792+0000","last_active":"2023-12-20T04:28:39.386014+0000","last_peered":"2023-12-20T04:28:39.386014+0000","last_clean":"2023-12-20T04:28:39.386014+0000","last_became_active":"2023-12-20T04:28:37.978509+0000","last_became_peered":"2023-12-20T04:28:37.978509+0000","last_unstale":"2023-12-20T04:28:39.386014+0000","last_undegraded":"2023-12-20T04:28:39.386014+0000","last_fullsized":"2023-12-20T04:28:39.386014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.400976+0000","last_change":"2023-12-20T04:28:38.400189+0000","last_active":"2023-12-20T04:28:38.400976+0000","last_peered":"2023-12-20T04:28:38.400976+0000","last_clean":"2023-12-20T04:28:38.400976+0000","last_became_active":"2023-12-20T04:28:38.399902+0000","last_became_peered":"2023-12-20T04:28:38.399902+0000","last_unstale":"2023-12-20T04:28:38.400976+0000","last_undegraded":"2023-12-20T04:28:38.400976+0000","last_fullsized":"2023-12-20T04:28:38.400976+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:06.804397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.376745+0000","last_change":"2023-12-20T04:28:37.987850+0000","last_active":"2023-12-20T04:28:38.376745+0000","last_peered":"2023-12-20T04:28:38.376745+0000","last_clean":"2023-12-20T04:28:38.376745+0000","last_became_active":"2023-12-20T04:28:37.986972+0000","last_became_peered":"2023-12-20T04:28:37.986972+0000","last_unstale":"2023-12-20T04:28:38.376745+0000","last_undegraded":"2023-12-20T04:28:38.376745+0000","last_fullsized":"2023-12-20T04:28:38.376745+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:19.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.403048+0000","last_change":"2023-12-20T04:28:38.397621+0000","last_active":"2023-12-20T04:28:38.403048+0000","last_peered":"2023-12-20T04:28:38.403048+0000","last_clean":"2023-12-20T04:28:38.403048+0000","last_became_active":"2023-12-20T04:28:38.397311+0000","last_became_peered":"2023-12-20T04:28:38.397311+0000","last_unstale":"2023-12-20T04:28:38.403048+0000","last_undegraded":"2023-12-20T04:28:38.403048+0000","last_fullsized":"2023-12-20T04:28:38.403048+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:26:59.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.371710+0000","last_change":"2023-12-20T04:28:37.982113+0000","last_active":"2023-12-20T04:28:38.371710+0000","last_peered":"2023-12-20T04:28:38.371710+0000","last_clean":"2023-12-20T04:28:38.371710+0000","last_became_active":"2023-12-20T04:28:37.981812+0000","last_became_peered":"2023-12-20T04:28:37.981812+0000","last_unstale":"2023-12-20T04:28:38.371710+0000","last_undegraded":"2023-12-20T04:28:38.371710+0000","last_fullsized":"2023-12-20T04:28:38.371710+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379680+0000","last_change":"2023-12-20T04:28:38.383867+0000","last_active":"2023-12-20T04:28:39.379680+0000","last_peered":"2023-12-20T04:28:39.379680+0000","last_clean":"2023-12-20T04:28:39.379680+0000","last_became_active":"2023-12-20T04:28:38.383222+0000","last_became_peered":"2023-12-20T04:28:38.383222+0000","last_unstale":"2023-12-20T04:28:39.379680+0000","last_undegraded":"2023-12-20T04:28:39.379680+0000","last_fullsized":"2023-12-20T04:28:39.379680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386301+0000","last_change":"2023-12-20T04:28:37.981934+0000","last_active":"2023-12-20T04:28:39.386301+0000","last_peered":"2023-12-20T04:28:39.386301+0000","last_clean":"2023-12-20T04:28:39.386301+0000","last_became_active":"2023-12-20T04:28:37.981000+0000","last_became_peered":"2023-12-20T04:28:37.981000+0000","last_unstale":"2023-12-20T04:28:39.386301+0000","last_undegraded":"2023-12-20T04:28:39.386301+0000","last_fullsized":"2023-12-20T04:28:39.386301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386046+0000","last_change":"2023-12-20T04:28:38.405946+0000","last_active":"2023-12-20T04:28:39.386046+0000","last_peered":"2023-12-20T04:28:39.386046+0000","last_clean":"2023-12-20T04:28:39.386046+0000","last_became_active":"2023-12-20T04:28:38.405609+0000","last_became_peered":"2023-12-20T04:28:38.405609+0000","last_unstale":"2023-12-20T04:28:39.386046+0000","last_undegraded":"2023-12-20T04:28:39.386046+0000","last_fullsized":"2023-12-20T04:28:39.386046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379821+0000","last_change":"2023-12-20T04:28:37.980797+0000","last_active":"2023-12-20T04:28:39.379821+0000","last_peered":"2023-12-20T04:28:39.379821+0000","last_clean":"2023-12-20T04:28:39.379821+0000","last_became_active":"2023-12-20T04:28:37.980430+0000","last_became_peered":"2023-12-20T04:28:37.980430+0000","last_unstale":"2023-12-20T04:28:39.379821+0000","last_undegraded":"2023-12-20T04:28:39.379821+0000","last_fullsized":"2023-12-20T04:28:39.379821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:19.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386153+0000","last_change":"2023-12-20T04:28:38.389261+0000","last_active":"2023-12-20T04:28:39.386153+0000","last_peered":"2023-12-20T04:28:39.386153+0000","last_clean":"2023-12-20T04:28:39.386153+0000","last_became_active":"2023-12-20T04:28:38.388004+0000","last_became_peered":"2023-12-20T04:28:38.388004+0000","last_unstale":"2023-12-20T04:28:39.386153+0000","last_undegraded":"2023-12-20T04:28:39.386153+0000","last_fullsized":"2023-12-20T04:28:39.386153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.378913+0000","last_change":"2023-12-20T04:28:37.991195+0000","last_active":"2023-12-20T04:28:38.378913+0000","last_peered":"2023-12-20T04:28:38.378913+0000","last_clean":"2023-12-20T04:28:38.378913+0000","last_became_active":"2023-12-20T04:28:37.990860+0000","last_became_peered":"2023-12-20T04:28:37.990860+0000","last_unstale":"2023-12-20T04:28:38.378913+0000","last_undegraded":"2023-12-20T04:28:38.378913+0000","last_fullsized":"2023-12-20T04:28:38.378913+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:11.027603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379827+0000","last_change":"2023-12-20T04:28:38.377842+0000","last_active":"2023-12-20T04:28:39.379827+0000","last_peered":"2023-12-20T04:28:39.379827+0000","last_clean":"2023-12-20T04:28:39.379827+0000","last_became_active":"2023-12-20T04:28:38.377525+0000","last_became_peered":"2023-12-20T04:28:38.377525+0000","last_unstale":"2023-12-20T04:28:39.379827+0000","last_undegraded":"2023-12-20T04:28:39.379827+0000","last_fullsized":"2023-12-20T04:28:39.379827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.380013+0000","last_change":"2023-12-20T04:28:37.995878+0000","last_active":"2023-12-20T04:28:39.380013+0000","last_peered":"2023-12-20T04:28:39.380013+0000","last_clean":"2023-12-20T04:28:39.380013+0000","last_became_active":"2023-12-20T04:28:37.995604+0000","last_became_peered":"2023-12-20T04:28:37.995604+0000","last_unstale":"2023-12-20T04:28:39.380013+0000","last_undegraded":"2023-12-20T04:28:39.380013+0000","last_fullsized":"2023-12-20T04:28:39.380013+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.399533+0000","last_change":"2023-12-20T04:28:38.398433+0000","last_active":"2023-12-20T04:28:38.399533+0000","last_peered":"2023-12-20T04:28:38.399533+0000","last_clean":"2023-12-20T04:28:38.399533+0000","last_became_active":"2023-12-20T04:28:38.398247+0000","last_became_peered":"2023-12-20T04:28:38.398247+0000","last_unstale":"2023-12-20T04:28:38.399533+0000","last_undegraded":"2023-12-20T04:28:38.399533+0000","last_fullsized":"2023-12-20T04:28:38.399533+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:48:18.553482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411943+0000","last_change":"2023-12-20T04:28:37.993415+0000","last_active":"2023-12-20T04:28:38.411943+0000","last_peered":"2023-12-20T04:28:38.411943+0000","last_clean":"2023-12-20T04:28:38.411943+0000","last_became_active":"2023-12-20T04:28:37.993209+0000","last_became_peered":"2023-12-20T04:28:37.993209+0000","last_unstale":"2023-12-20T04:28:38.411943+0000","last_undegraded":"2023-12-20T04:28:38.411943+0000","last_fullsized":"2023-12-20T04:28:38.411943+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.393420+0000","last_change":"2023-12-20T04:28:38.388800+0000","last_active":"2023-12-20T04:28:38.393420+0000","last_peered":"2023-12-20T04:28:38.393420+0000","last_clean":"2023-12-20T04:28:38.393420+0000","last_became_active":"2023-12-20T04:28:38.383908+0000","last_became_peered":"2023-12-20T04:28:38.383908+0000","last_unstale":"2023-12-20T04:28:38.393420+0000","last_undegraded":"2023-12-20T04:28:38.393420+0000","last_fullsized":"2023-12-20T04:28:38.393420+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.376636+0000","last_change":"2023-12-20T04:28:37.996076+0000","last_active":"2023-12-20T04:28:38.376636+0000","last_peered":"2023-12-20T04:28:38.376636+0000","last_clean":"2023-12-20T04:28:38.376636+0000","last_became_active":"2023-12-20T04:28:37.995770+0000","last_became_peered":"2023-12-20T04:28:37.995770+0000","last_unstale":"2023-12-20T04:28:38.376636+0000","last_undegraded":"2023-12-20T04:28:38.376636+0000","last_fullsized":"2023-12-20T04:28:38.376636+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:34.884350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385884+0000","last_change":"2023-12-20T04:28:38.393602+0000","last_active":"2023-12-20T04:28:39.385884+0000","last_peered":"2023-12-20T04:28:39.385884+0000","last_clean":"2023-12-20T04:28:39.385884+0000","last_became_active":"2023-12-20T04:28:38.392277+0000","last_became_peered":"2023-12-20T04:28:38.392277+0000","last_unstale":"2023-12-20T04:28:39.385884+0000","last_undegraded":"2023-12-20T04:28:39.385884+0000","last_fullsized":"2023-12-20T04:28:39.385884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:20.635968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379704+0000","last_change":"2023-12-20T04:28:37.987522+0000","last_active":"2023-12-20T04:28:39.379704+0000","last_peered":"2023-12-20T04:28:39.379704+0000","last_clean":"2023-12-20T04:28:39.379704+0000","last_became_active":"2023-12-20T04:28:37.986906+0000","last_became_peered":"2023-12-20T04:28:37.986906+0000","last_unstale":"2023-12-20T04:28:39.379704+0000","last_undegraded":"2023-12-20T04:28:39.379704+0000","last_fullsized":"2023-12-20T04:28:39.379704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:34.884350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.386583+0000","last_change":"2023-12-20T04:28:38.382504+0000","last_active":"2023-12-20T04:28:38.386583+0000","last_peered":"2023-12-20T04:28:38.386583+0000","last_clean":"2023-12-20T04:28:38.386583+0000","last_became_active":"2023-12-20T04:28:38.382224+0000","last_became_peered":"2023-12-20T04:28:38.382224+0000","last_unstale":"2023-12-20T04:28:38.386583+0000","last_undegraded":"2023-12-20T04:28:38.386583+0000","last_fullsized":"2023-12-20T04:28:38.386583+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379771+0000","last_change":"2023-12-20T04:28:37.985344+0000","last_active":"2023-12-20T04:28:39.379771+0000","last_peered":"2023-12-20T04:28:39.379771+0000","last_clean":"2023-12-20T04:28:39.379771+0000","last_became_active":"2023-12-20T04:28:37.985074+0000","last_became_peered":"2023-12-20T04:28:37.985074+0000","last_unstale":"2023-12-20T04:28:39.379771+0000","last_undegraded":"2023-12-20T04:28:39.379771+0000","last_fullsized":"2023-12-20T04:28:39.379771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.404941+0000","last_change":"2023-12-20T04:28:38.402571+0000","last_active":"2023-12-20T04:28:38.404941+0000","last_peered":"2023-12-20T04:28:38.404941+0000","last_clean":"2023-12-20T04:28:38.404941+0000","last_became_active":"2023-12-20T04:28:38.402316+0000","last_became_peered":"2023-12-20T04:28:38.402316+0000","last_unstale":"2023-12-20T04:28:38.404941+0000","last_undegraded":"2023-12-20T04:28:38.404941+0000","last_fullsized":"2023-12-20T04:28:38.404941+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.408942+0000","last_change":"2023-12-20T04:28:37.991378+0000","last_active":"2023-12-20T04:28:38.408942+0000","last_peered":"2023-12-20T04:28:38.408942+0000","last_clean":"2023-12-20T04:28:38.408942+0000","last_became_active":"2023-12-20T04:28:37.990976+0000","last_became_peered":"2023-12-20T04:28:37.990976+0000","last_unstale":"2023-12-20T04:28:38.408942+0000","last_undegraded":"2023-12-20T04:28:38.408942+0000","last_fullsized":"2023-12-20T04:28:38.408942+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:18.611871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.413343+0000","last_change":"2023-12-20T04:28:38.411659+0000","last_active":"2023-12-20T04:28:38.413343+0000","last_peered":"2023-12-20T04:28:38.413343+0000","last_clean":"2023-12-20T04:28:38.413343+0000","last_became_active":"2023-12-20T04:28:38.411333+0000","last_became_peered":"2023-12-20T04:28:38.411333+0000","last_unstale":"2023-12-20T04:28:38.413343+0000","last_undegraded":"2023-12-20T04:28:38.413343+0000","last_fullsized":"2023-12-20T04:28:38.413343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:46:19.275821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386042+0000","last_change":"2023-12-20T04:28:38.392988+0000","last_active":"2023-12-20T04:28:39.386042+0000","last_peered":"2023-12-20T04:28:39.386042+0000","last_clean":"2023-12-20T04:28:39.386042+0000","last_became_active":"2023-12-20T04:28:38.392136+0000","last_became_peered":"2023-12-20T04:28:38.392136+0000","last_unstale":"2023-12-20T04:28:39.386042+0000","last_undegraded":"2023-12-20T04:28:39.386042+0000","last_fullsized":"2023-12-20T04:28:39.386042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.400965+0000","last_change":"2023-12-20T04:28:37.991881+0000","last_active":"2023-12-20T04:28:38.400965+0000","last_peered":"2023-12-20T04:28:38.400965+0000","last_clean":"2023-12-20T04:28:38.400965+0000","last_became_active":"2023-12-20T04:28:37.991520+0000","last_became_peered":"2023-12-20T04:28:37.991520+0000","last_unstale":"2023-12-20T04:28:38.400965+0000","last_undegraded":"2023-12-20T04:28:38.400965+0000","last_fullsized":"2023-12-20T04:28:38.400965+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385497+0000","last_change":"2023-12-20T04:28:38.398174+0000","last_active":"2023-12-20T04:28:39.385497+0000","last_peered":"2023-12-20T04:28:39.385497+0000","last_clean":"2023-12-20T04:28:39.385497+0000","last_became_active":"2023-12-20T04:28:38.397909+0000","last_became_peered":"2023-12-20T04:28:38.397909+0000","last_unstale":"2023-12-20T04:28:39.385497+0000","last_undegraded":"2023-12-20T04:28:39.385497+0000","last_fullsized":"2023-12-20T04:28:39.385497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.401174+0000","last_change":"2023-12-20T04:28:37.997620+0000","last_active":"2023-12-20T04:28:38.401174+0000","last_peered":"2023-12-20T04:28:38.401174+0000","last_clean":"2023-12-20T04:28:38.401174+0000","last_became_active":"2023-12-20T04:28:37.997289+0000","last_became_peered":"2023-12-20T04:28:37.997289+0000","last_unstale":"2023-12-20T04:28:38.401174+0000","last_undegraded":"2023-12-20T04:28:38.401174+0000","last_fullsized":"2023-12-20T04:28:38.401174+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:00.985871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.405734+0000","last_change":"2023-12-20T04:28:38.403356+0000","last_active":"2023-12-20T04:28:38.405734+0000","last_peered":"2023-12-20T04:28:38.405734+0000","last_clean":"2023-12-20T04:28:38.405734+0000","last_became_active":"2023-12-20T04:28:38.402695+0000","last_became_peered":"2023-12-20T04:28:38.402695+0000","last_unstale":"2023-12-20T04:28:38.405734+0000","last_undegraded":"2023-12-20T04:28:38.405734+0000","last_fullsized":"2023-12-20T04:28:38.405734+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.391009+0000","last_change":"2023-12-20T04:28:37.986554+0000","last_active":"2023-12-20T04:28:38.391009+0000","last_peered":"2023-12-20T04:28:38.391009+0000","last_clean":"2023-12-20T04:28:38.391009+0000","last_became_active":"2023-12-20T04:28:37.985944+0000","last_became_peered":"2023-12-20T04:28:37.985944+0000","last_unstale":"2023-12-20T04:28:38.391009+0000","last_undegraded":"2023-12-20T04:28:38.391009+0000","last_fullsized":"2023-12-20T04:28:38.391009+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":9,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.403613+0000","last_change":"2023-12-20T04:28:38.402052+0000","last_active":"2023-12-20T04:28:38.403613+0000","last_peered":"2023-12-20T04:28:38.403613+0000","last_clean":"2023-12-20T04:28:38.403613+0000","last_became_active":"2023-12-20T04:28:38.401578+0000","last_became_peered":"2023-12-20T04:28:38.401578+0000","last_unstale":"2023-12-20T04:28:38.403613+0000","last_undegraded":"2023-12-20T04:28:38.403613+0000","last_fullsized":"2023-12-20T04:28:38.403613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":14,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.411161+0000","last_change":"2023-12-20T04:28:37.991899+0000","last_active":"2023-12-20T04:28:38.411161+0000","last_peered":"2023-12-20T04:28:38.411161+0000","last_clean":"2023-12-20T04:28:38.411161+0000","last_became_active":"2023-12-20T04:28:37.991545+0000","last_became_peered":"2023-12-20T04:28:37.991545+0000","last_unstale":"2023-12-20T04:28:38.411161+0000","last_undegraded":"2023-12-20T04:28:38.411161+0000","last_fullsized":"2023-12-20T04:28:38.411161+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574852,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31832,"kb_used_data":784,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340008,"statfs":{"total":96636764160,"available":96604168192,"internally_reserved":0,"allocated":802816,"data_stored":660219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.872}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77100000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.88200000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92000000000000004}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.64500000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574852,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":70833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56000000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75800000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60799999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.72399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.69699999999999995}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834574853,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27156,"kb_used_data":204,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344684,"statfs":{"total":96636764160,"available":96608956416,"internally_reserved":0,"allocated":208896,"data_stored":69359,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80100000000000005}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66500000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77300000000000002}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93300000000000005}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.89200000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574854,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27164,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344676,"statfs":{"total":96636764160,"available":96608948224,"internally_reserved":0,"allocated":217088,"data_stored":69395,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67700000000000005}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71399999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.97399999999999998}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.83499999999999996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91000000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.93700000000000006}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.95499999999999996}]}]},{"osd":3,"up_from":13,"seq":55834574853,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31260,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340580,"statfs":{"total":96636764160,"available":96604753920,"internally_reserved":0,"allocated":217088,"data_stored":70833,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.30199999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.28000000000000003}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48999999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54700000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67400000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]}]},{"osd":2,"up_from":13,"seq":55834574852,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27152,"kb_used_data":200,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344688,"statfs":{"total":96636764160,"available":96608960512,"internally_reserved":0,"allocated":204800,"data_stored":69269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.312}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.38}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.42599999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.873}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65600000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.79600000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.754}]}]},{"osd":1,"up_from":13,"seq":55834574854,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31256,"kb_used_data":208,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340584,"statfs":{"total":96636764160,"available":96604758016,"internally_reserved":0,"allocated":212992,"data_stored":69429,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.34499999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44500000000000001}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46899999999999997}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65000000000000002}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.75700000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.74099999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71999999999999997}]}]},{"osd":0,"up_from":13,"seq":55834574853,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31840,"kb_used_data":792,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340000,"statfs":{"total":96636764160,"available":96604160000,"internally_reserved":0,"allocated":811008,"data_stored":660275,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.45900000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60999999999999999}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.49199999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.014}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.92700000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.91800000000000004}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.98699999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:28:42.862 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T04:28:42.862 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2023-12-20T04:28:42.862 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2023-12-20T04:28:42.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-20T04:28:42.863 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2023-12-20T04:28:43.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df3fa6640 1 Processor -- start 2023-12-20T04:28:43.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df3fa6640 1 -- start start 2023-12-20T04:28:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df3fa6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3dec15d080 0x7f3dec0cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df3fa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dec15de00 0x7f3dec06a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df3fa6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dec15d970 con 0x7f3dec069d20 2023-12-20T04:28:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df3fa6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dec0ce740 con 0x7f3dec15d080 2023-12-20T04:28:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df3fa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec06a640 con 0x7f3dec15de00 2023-12-20T04:28:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df1541640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3dec15d080 0x7f3dec0cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df1541640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3dec15d080 0x7f3dec0cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48158/0 (socket says 172.21.15.32:48158) 2023-12-20T04:28:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df1541640 1 -- 172.21.15.32:0/507790027 learned_addr learned my addr 172.21.15.32:0/507790027 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df2543640 1 --2- 172.21.15.32:0/507790027 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dec15de00 0x7f3dec06a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df1541640 1 -- 172.21.15.32:0/507790027 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dec15de00 msgr2=0x7f3dec06a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df1541640 1 --2- 172.21.15.32:0/507790027 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dec15de00 0x7f3dec06a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df1541640 1 -- 172.21.15.32:0/507790027 >> v1:172.21.15.39:6789/0 conn(0x7f3dec069d20 legacy=0x7f3dec0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.040+0000 7f3df1541640 1 -- 172.21.15.32:0/507790027 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dec0ce180 con 0x7f3dec15d080 2023-12-20T04:28:43.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df1541640 1 --2- 172.21.15.32:0/507790027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec15d080 0x7f3dec0cbbb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009870 tx=0x7f3ddc01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f9d7cee5a74dc95 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df0d40640 1 -- 172.21.15.32:0/507790027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0aa670 con 0x7f3dec15d080 2023-12-20T04:28:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df0d40640 1 -- 172.21.15.32:0/507790027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc0aa7d0 con 0x7f3dec15d080 2023-12-20T04:28:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df0d40640 1 -- 172.21.15.32:0/507790027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0aab60 con 0x7f3dec15d080 2023-12-20T04:28:43.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df3fa6640 1 -- 172.21.15.32:0/507790027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec15d080 msgr2=0x7f3dec0cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df3fa6640 1 --2- 172.21.15.32:0/507790027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec15d080 0x7f3dec0cbbb0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc009870 tx=0x7f3ddc01c510 comp rx=0 tx=0).stop 2023-12-20T04:28:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df3fa6640 1 -- 172.21.15.32:0/507790027 shutdown_connections 2023-12-20T04:28:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df3fa6640 1 --2- 172.21.15.32:0/507790027 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dec15de00 0x7f3dec06a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df3fa6640 1 --2- 172.21.15.32:0/507790027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec15d080 0x7f3dec0cbbb0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df3fa6640 1 -- 172.21.15.32:0/507790027 >> 172.21.15.32:0/507790027 conn(0x7f3dec0cc310 msgr2=0x7f3dec0cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df3fa6640 1 -- 172.21.15.32:0/507790027 shutdown_connections 2023-12-20T04:28:43.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.044+0000 7f3df3fa6640 1 -- 172.21.15.32:0/507790027 wait complete. 2023-12-20T04:28:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df3fa6640 1 Processor -- start 2023-12-20T04:28:43.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df3fa6640 1 -- start start 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df3fa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dec069d20 0x7f3dec0c8210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df3fa6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec15d080 0x7f3dec1559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df3fa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dec15de00 0x7f3dec0b45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df3fa6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec154cb0 con 0x7f3dec15d080 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df3fa6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec155550 con 0x7f3dec069d20 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df3fa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dec0ac810 con 0x7f3dec15de00 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df1541640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec15d080 0x7f3dec1559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df1541640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec15d080 0x7f3dec1559e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48170/0 (socket says 172.21.15.32:48170) 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df1541640 1 -- 172.21.15.32:0/912954769 learned_addr learned my addr 172.21.15.32:0/912954769 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df1d42640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dec069d20 0x7f3dec0c8210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df2543640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dec15de00 0x7f3dec0b45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df1541640 1 -- 172.21.15.32:0/912954769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dec15de00 msgr2=0x7f3dec0b45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df1541640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dec15de00 0x7f3dec0b45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df1541640 1 -- 172.21.15.32:0/912954769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dec069d20 msgr2=0x7f3dec0c8210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df1541640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dec069d20 0x7f3dec0c8210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df1541640 1 -- 172.21.15.32:0/912954769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ddc0094d0 con 0x7f3dec15d080 2023-12-20T04:28:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3df1541640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec15d080 0x7f3dec1559e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0ac040 tx=0x7f3ddc01ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.048+0000 7f3de27fc640 1 -- 172.21.15.32:0/912954769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0b3800 con 0x7f3dec15d080 2023-12-20T04:28:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.052+0000 7f3de27fc640 1 -- 172.21.15.32:0/912954769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ddc0b3960 con 0x7f3dec15d080 2023-12-20T04:28:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.052+0000 7f3de27fc640 1 -- 172.21.15.32:0/912954769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ddc0b3cf0 con 0x7f3dec15d080 2023-12-20T04:28:43.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.052+0000 7f3df3fa6640 1 -- 172.21.15.32:0/912954769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3dec15e390 con 0x7f3dec15d080 2023-12-20T04:28:43.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.052+0000 7f3df3fa6640 1 -- 172.21.15.32:0/912954769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dec1551e0 con 0x7f3dec15d080 2023-12-20T04:28:43.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.052+0000 7f3df3fa6640 1 -- 172.21.15.32:0/912954769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dec04e000 con 0x7f3dec15d080 2023-12-20T04:28:43.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.052+0000 7f3de27fc640 1 -- 172.21.15.32:0/912954769 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3ddc0b2380 con 0x7f3dec15d080 2023-12-20T04:28:43.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.052+0000 7f3de27fc640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd0065a90 0x7f3dd0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.052+0000 7f3de27fc640 1 -- 172.21.15.32:0/912954769 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3ddc11db20 con 0x7f3dec15d080 2023-12-20T04:28:43.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.052+0000 7f3df1d42640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd0065a90 0x7f3dd0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.056+0000 7f3df1d42640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd0065a90 0x7f3dd0067f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3de8007fd0 tx=0x7f3de8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:43.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.056+0000 7f3de27fc640 1 -- 172.21.15.32:0/912954769 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3ddc0eb580 con 0x7f3dec15d080 2023-12-20T04:28:43.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.252+0000 7f3df3fa6640 1 -- 172.21.15.32:0/912954769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f3dec06f060 con 0x7f3dec15d080 2023-12-20T04:28:43.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.256+0000 7f3de27fc640 1 -- 172.21.15.32:0/912954769 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v5) v1 ==== 74+0+163361 (secure 0 0 0) 0x7f3ddc0c3020 con 0x7f3dec15d080 2023-12-20T04:28:43.258 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:43.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.256+0000 7f3df3fa6640 1 -- 172.21.15.32:0/912954769 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd0065a90 msgr2=0x7f3dd0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.256+0000 7f3df3fa6640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd0065a90 0x7f3dd0067f50 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f3de8007fd0 tx=0x7f3de8009040 comp rx=0 tx=0).stop 2023-12-20T04:28:43.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.256+0000 7f3df3fa6640 1 -- 172.21.15.32:0/912954769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec15d080 msgr2=0x7f3dec1559e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.256+0000 7f3df3fa6640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec15d080 0x7f3dec1559e0 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f3ddc0ac040 tx=0x7f3ddc01ad50 comp rx=0 tx=0).stop 2023-12-20T04:28:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.260+0000 7f3df3fa6640 1 -- 172.21.15.32:0/912954769 shutdown_connections 2023-12-20T04:28:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.260+0000 7f3df3fa6640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd0065a90 0x7f3dd0067f50 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.260+0000 7f3df3fa6640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dec15de00 0x7f3dec0b45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.260+0000 7f3df3fa6640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dec15d080 0x7f3dec1559e0 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.260+0000 7f3df3fa6640 1 --2- 172.21.15.32:0/912954769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dec069d20 0x7f3dec0c8210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.260+0000 7f3df3fa6640 1 -- 172.21.15.32:0/912954769 >> 172.21.15.32:0/912954769 conn(0x7f3dec0cc310 msgr2=0x7f3dec0cab00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.260+0000 7f3df3fa6640 1 -- 172.21.15.32:0/912954769 shutdown_connections 2023-12-20T04:28:43.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.260+0000 7f3df3fa6640 1 -- 172.21.15.32:0/912954769 wait complete. 2023-12-20T04:28:43.278 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":5,"active_gid":4104,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":38009},{"type":"v1","addr":"172.21.15.32:6833","nonce":38009}]},"active_addr":"172.21.15.32:6833/38009","active_change":"2023-12-20T04:28:18.111858+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":4112,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2782983206}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":728956527}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2331287763}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":856257861}]}]} 2023-12-20T04:28:43.282 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2023-12-20T04:28:43.282 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2023-12-20T04:28:43.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:43.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.424+0000 7f6344125640 1 Processor -- start 2023-12-20T04:28:43.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.424+0000 7f6344125640 1 -- start start 2023-12-20T04:28:43.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.424+0000 7f6344125640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f633c0ae010 0x7f633c071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.424+0000 7f6344125640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c0ae3f0 con 0x7f633c164ef0 2023-12-20T04:28:43.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.424+0000 7f6344125640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633c0ca4b0 con 0x7f633c06cf10 2023-12-20T04:28:43.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.424+0000 7f6344125640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633c161800 con 0x7f633c0ae010 2023-12-20T04:28:43.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.424+0000 7f6341ec1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f633c0ae010 0x7f633c071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.424+0000 7f63426c2640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f633c164ef0 0x7f633c0708d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35310/0 (socket says 172.21.15.32:35310) 2023-12-20T04:28:43.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.424+0000 7f63426c2640 1 -- 172.21.15.32:0/1935247563 learned_addr learned my addr 172.21.15.32:0/1935247563 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:43.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.424+0000 7f63416c0640 1 --1- 172.21.15.32:0/1935247563 >> v1:172.21.15.39:6789/0 conn(0x7f633c06cf10 0x7f633c0701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:52080/0 (socket says 172.21.15.32:52080) 2023-12-20T04:28:43.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6341ec1640 1 -- 172.21.15.32:0/1935247563 >> v1:172.21.15.39:6789/0 conn(0x7f633c06cf10 legacy=0x7f633c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6341ec1640 1 -- 172.21.15.32:0/1935247563 >> v1:172.21.15.32:6789/0 conn(0x7f633c164ef0 legacy=0x7f633c0708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6341ec1640 1 -- 172.21.15.32:0/1935247563 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633c070fe0 con 0x7f633c0ae010 2023-12-20T04:28:43.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6340ebf640 1 -- 172.21.15.32:0/1935247563 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156108650 0 0) 0x7f633c0ca4b0 con 0x7f633c06cf10 2023-12-20T04:28:43.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6341ec1640 1 --2- 172.21.15.32:0/1935247563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c0ae010 0x7f633c071470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f63380097a0 tx=0x7f633801c260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a252807523b2e34 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:43.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6340ebf640 1 -- 172.21.15.32:0/1935247563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63380ab600 con 0x7f633c0ae010 2023-12-20T04:28:43.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6340ebf640 1 -- 172.21.15.32:0/1935247563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63380a7050 con 0x7f633c0ae010 2023-12-20T04:28:43.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6340ebf640 1 -- 172.21.15.32:0/1935247563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63380ab980 con 0x7f633c0ae010 2023-12-20T04:28:43.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6344125640 1 -- 172.21.15.32:0/1935247563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c0ae010 msgr2=0x7f633c071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6344125640 1 --2- 172.21.15.32:0/1935247563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c0ae010 0x7f633c071470 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f63380097a0 tx=0x7f633801c260 comp rx=0 tx=0).stop 2023-12-20T04:28:43.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6344125640 1 -- 172.21.15.32:0/1935247563 shutdown_connections 2023-12-20T04:28:43.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6344125640 1 --2- 172.21.15.32:0/1935247563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c0ae010 0x7f633c071470 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.428+0000 7f6344125640 1 -- 172.21.15.32:0/1935247563 >> 172.21.15.32:0/1935247563 conn(0x7f633c06a320 msgr2=0x7f633c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:43.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f6344125640 1 -- 172.21.15.32:0/1935247563 shutdown_connections 2023-12-20T04:28:43.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f6344125640 1 -- 172.21.15.32:0/1935247563 wait complete. 2023-12-20T04:28:43.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f6344125640 1 Processor -- start 2023-12-20T04:28:43.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f6344125640 1 -- start start 2023-12-20T04:28:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f6344125640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c0ae010 0x7f633c0c34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f6344125640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f633c0c5830 0x7f633c0b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f6344125640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c0c2290 0x7f633c162a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f6344125640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0c69f0 con 0x7f633c0c2290 2023-12-20T04:28:43.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f6344125640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633c15e5a0 con 0x7f633c0ae010 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f6344125640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633c0c5d30 con 0x7f633c0c5830 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f63426c2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c0c2290 0x7f633c162a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f63426c2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c0c2290 0x7f633c162a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48184/0 (socket says 172.21.15.32:48184) 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f63426c2640 1 -- 172.21.15.32:0/1839442635 learned_addr learned my addr 172.21.15.32:0/1839442635 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f6341ec1640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c0ae010 0x7f633c0c34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f63416c0640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f633c0c5830 0x7f633c0b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f63426c2640 1 -- 172.21.15.32:0/1839442635 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f633c0c5830 msgr2=0x7f633c0b1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f63426c2640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f633c0c5830 0x7f633c0b1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f63426c2640 1 -- 172.21.15.32:0/1839442635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c0ae010 msgr2=0x7f633c0c34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f63426c2640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c0ae010 0x7f633c0c34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f63426c2640 1 -- 172.21.15.32:0/1839442635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63380093b0 con 0x7f633c0c2290 2023-12-20T04:28:43.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.432+0000 7f63426c2640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c0c2290 0x7f633c162a40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f633400e450 tx=0x7f633400e910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:43.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.436+0000 7f63327fc640 1 -- 172.21.15.32:0/1839442635 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334009480 con 0x7f633c0c2290 2023-12-20T04:28:43.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.436+0000 7f63327fc640 1 -- 172.21.15.32:0/1839442635 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63340095e0 con 0x7f633c0c2290 2023-12-20T04:28:43.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.436+0000 7f6344125640 1 -- 172.21.15.32:0/1839442635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633c0c6010 con 0x7f633c0c2290 2023-12-20T04:28:43.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.436+0000 7f63327fc640 1 -- 172.21.15.32:0/1839442635 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334009970 con 0x7f633c0c2290 2023-12-20T04:28:43.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.436+0000 7f6344125640 1 -- 172.21.15.32:0/1839442635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633c0c65d0 con 0x7f633c0c2290 2023-12-20T04:28:43.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.436+0000 7f6344125640 1 -- 172.21.15.32:0/1839442635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633c0c6790 con 0x7f633c0c2290 2023-12-20T04:28:43.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.436+0000 7f63327fc640 1 -- 172.21.15.32:0/1839442635 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f633401d460 con 0x7f633c0c2290 2023-12-20T04:28:43.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.436+0000 7f63327fc640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065a90 0x7f6320067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.436+0000 7f63327fc640 1 -- 172.21.15.32:0/1839442635 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6334085fe0 con 0x7f633c0c2290 2023-12-20T04:28:43.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.436+0000 7f6341ec1640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065a90 0x7f6320067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.440+0000 7f6341ec1640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065a90 0x7f6320067f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f63380ad000 tx=0x7f63380a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:43.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.440+0000 7f63327fc640 1 -- 172.21.15.32:0/1839442635 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f63340539f0 con 0x7f633c0c2290 2023-12-20T04:28:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 -- 172.21.15.32:0/1839442635 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f633c0c6b30 con 0x7f633c0c2290 2023-12-20T04:28:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f63327fc640 1 -- 172.21.15.32:0/1839442635 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f633c0c6b30 con 0x7f633c0c2290 2023-12-20T04:28:43.600 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:43.600 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:43.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 -- 172.21.15.32:0/1839442635 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065a90 msgr2=0x7f6320067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065a90 0x7f6320067f50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f63380ad000 tx=0x7f63380a6000 comp rx=0 tx=0).stop 2023-12-20T04:28:43.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 -- 172.21.15.32:0/1839442635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c0c2290 msgr2=0x7f633c162a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c0c2290 0x7f633c162a40 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f633400e450 tx=0x7f633400e910 comp rx=0 tx=0).stop 2023-12-20T04:28:43.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 -- 172.21.15.32:0/1839442635 shutdown_connections 2023-12-20T04:28:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6320065a90 0x7f6320067f50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633c0c2290 0x7f633c162a40 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f633c0c5830 0x7f633c0b1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 --2- 172.21.15.32:0/1839442635 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633c0ae010 0x7f633c0c34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 -- 172.21.15.32:0/1839442635 >> 172.21.15.32:0/1839442635 conn(0x7f633c06a320 msgr2=0x7f633c06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 -- 172.21.15.32:0/1839442635 shutdown_connections 2023-12-20T04:28:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.600+0000 7f6344125640 1 -- 172.21.15.32:0/1839442635 wait complete. 2023-12-20T04:28:43.621 INFO:tasks.ceph.ceph_manager.ceph:all up! 2023-12-20T04:28:43.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:43.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e2b041640 1 Processor -- start 2023-12-20T04:28:43.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e2b041640 1 -- start start 2023-12-20T04:28:43.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e2b041640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e2415cdb0 0x7f3e240d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e2b041640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 0x7f3e240cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e2b041640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e240b6ca0 con 0x7f3e240d1120 2023-12-20T04:28:43.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e2b041640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e2415d190 con 0x7f3e2415cdb0 2023-12-20T04:28:43.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e2b041640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240d16a0 con 0x7f3e240d2740 2023-12-20T04:28:43.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e28ddd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e2415cdb0 0x7f3e240d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e28ddd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e2415cdb0 0x7f3e240d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48194/0 (socket says 172.21.15.32:48194) 2023-12-20T04:28:43.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e28ddd640 1 -- 172.21.15.32:0/469966021 learned_addr learned my addr 172.21.15.32:0/469966021 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:43.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e23fff640 1 --2- 172.21.15.32:0/469966021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 0x7f3e240cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e28ddd640 1 -- 172.21.15.32:0/469966021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 msgr2=0x7f3e240cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:43.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e28ddd640 1 --2- 172.21.15.32:0/469966021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 0x7f3e240cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e28ddd640 1 -- 172.21.15.32:0/469966021 >> v1:172.21.15.39:6789/0 conn(0x7f3e240d1120 legacy=0x7f3e240b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.764+0000 7f3e28ddd640 1 -- 172.21.15.32:0/469966021 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e240d1980 con 0x7f3e2415cdb0 2023-12-20T04:28:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e28ddd640 1 --2- 172.21.15.32:0/469966021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2415cdb0 0x7f3e240d0be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009870 tx=0x7f3e1801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298a1bf3ae3a5ccb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e237fe640 1 -- 172.21.15.32:0/469966021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e180a8a30 con 0x7f3e2415cdb0 2023-12-20T04:28:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e237fe640 1 -- 172.21.15.32:0/469966021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e180a8b90 con 0x7f3e2415cdb0 2023-12-20T04:28:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e237fe640 1 -- 172.21.15.32:0/469966021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e180b0440 con 0x7f3e2415cdb0 2023-12-20T04:28:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e2b041640 1 -- 172.21.15.32:0/469966021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2415cdb0 msgr2=0x7f3e240d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e2b041640 1 --2- 172.21.15.32:0/469966021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2415cdb0 0x7f3e240d0be0 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f3e18009870 tx=0x7f3e1801ed30 comp rx=0 tx=0).stop 2023-12-20T04:28:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e2b041640 1 -- 172.21.15.32:0/469966021 shutdown_connections 2023-12-20T04:28:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e2b041640 1 --2- 172.21.15.32:0/469966021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 0x7f3e240cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e2b041640 1 --2- 172.21.15.32:0/469966021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2415cdb0 0x7f3e240d0be0 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e2b041640 1 -- 172.21.15.32:0/469966021 >> 172.21.15.32:0/469966021 conn(0x7f3e24065ee0 msgr2=0x7f3e2406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e2b041640 1 -- 172.21.15.32:0/469966021 shutdown_connections 2023-12-20T04:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.768+0000 7f3e2b041640 1 -- 172.21.15.32:0/469966021 wait complete. 2023-12-20T04:28:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e2b041640 1 Processor -- start 2023-12-20T04:28:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e2b041640 1 -- start start 2023-12-20T04:28:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e2b041640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e240d1120 0x7f3e2414cc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e2b041640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 0x7f3e2415fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e2b041640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2415cdb0 0x7f3e240c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e2b041640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e24160150 con 0x7f3e2415cdb0 2023-12-20T04:28:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e2b041640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240c8970 con 0x7f3e240d1120 2023-12-20T04:28:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e2b041640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2414f110 con 0x7f3e240d2740 2023-12-20T04:28:43.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e295de640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2415cdb0 0x7f3e240c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e23fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 0x7f3e2415fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e23fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 0x7f3e2415fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49314/0 (socket says 172.21.15.32:49314) 2023-12-20T04:28:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e23fff640 1 -- 172.21.15.32:0/3481818244 learned_addr learned my addr 172.21.15.32:0/3481818244 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:43.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e28ddd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e240d1120 0x7f3e2414cc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e23fff640 1 -- 172.21.15.32:0/3481818244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e240d1120 msgr2=0x7f3e2414cc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e23fff640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e240d1120 0x7f3e2414cc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e23fff640 1 -- 172.21.15.32:0/3481818244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2415cdb0 msgr2=0x7f3e240c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e23fff640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2415cdb0 0x7f3e240c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e23fff640 1 -- 172.21.15.32:0/3481818244 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e180094d0 con 0x7f3e240d2740 2023-12-20T04:28:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e295de640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2415cdb0 0x7f3e240c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e23fff640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 0x7f3e2415fc10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000d830 tx=0x7f3e1000dcf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e217fa640 1 -- 172.21.15.32:0/3481818244 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1000b6c0 con 0x7f3e240d2740 2023-12-20T04:28:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e217fa640 1 -- 172.21.15.32:0/3481818244 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e1000b820 con 0x7f3e240d2740 2023-12-20T04:28:43.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e217fa640 1 -- 172.21.15.32:0/3481818244 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e1000bb90 con 0x7f3e240d2740 2023-12-20T04:28:43.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.772+0000 7f3e2b041640 1 -- 172.21.15.32:0/3481818244 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e2414f3f0 con 0x7f3e240d2740 2023-12-20T04:28:43.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.776+0000 7f3e2b041640 1 -- 172.21.15.32:0/3481818244 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e2414f9b0 con 0x7f3e240d2740 2023-12-20T04:28:43.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.776+0000 7f3e2b041640 1 -- 172.21.15.32:0/3481818244 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2404e000 con 0x7f3e240d2740 2023-12-20T04:28:43.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.776+0000 7f3e217fa640 1 -- 172.21.15.32:0/3481818244 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3e1001e460 con 0x7f3e240d2740 2023-12-20T04:28:43.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.776+0000 7f3e217fa640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e08065ab0 0x7f3e08067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:43.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.776+0000 7f3e217fa640 1 -- 172.21.15.32:0/3481818244 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e10083750 con 0x7f3e240d2740 2023-12-20T04:28:43.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.776+0000 7f3e28ddd640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e08065ab0 0x7f3e08067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:43.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.776+0000 7f3e28ddd640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e08065ab0 0x7f3e08067f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e180b2000 tx=0x7f3e18009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:43.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.784+0000 7f3e217fa640 1 -- 172.21.15.32:0/3481818244 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3e10051230 con 0x7f3e240d2740 2023-12-20T04:28:43.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.940+0000 7f3e2b041640 1 -- 172.21.15.32:0/3481818244 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3e2406b8d0 con 0x7f3e240d2740 2023-12-20T04:28:43.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e217fa640 1 -- 172.21.15.32:0/3481818244 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f3e10054850 con 0x7f3e240d2740 2023-12-20T04:28:43.945 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:43.945 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 -- 172.21.15.32:0/3481818244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e08065ab0 msgr2=0x7f3e08067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e08065ab0 0x7f3e08067f70 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3e180b2000 tx=0x7f3e18009040 comp rx=0 tx=0).stop 2023-12-20T04:28:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 -- 172.21.15.32:0/3481818244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 msgr2=0x7f3e2415fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 0x7f3e2415fc10 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3e1000d830 tx=0x7f3e1000dcf0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 -- 172.21.15.32:0/3481818244 shutdown_connections 2023-12-20T04:28:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e08065ab0 0x7f3e08067f70 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2415cdb0 0x7f3e240c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e240d2740 0x7f3e2415fc10 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 --2- 172.21.15.32:0/3481818244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e240d1120 0x7f3e2414cc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:43.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 -- 172.21.15.32:0/3481818244 >> 172.21.15.32:0/3481818244 conn(0x7f3e24065ee0 msgr2=0x7f3e240cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:43.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 -- 172.21.15.32:0/3481818244 shutdown_connections 2023-12-20T04:28:43.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:43.944+0000 7f3e2b041640 1 -- 172.21.15.32:0/3481818244 wait complete. 2023-12-20T04:28:43.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2023-12-20T04:28:43.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2023-12-20T04:28:43.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2023-12-20T04:28:43.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2023-12-20T04:28:43.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2023-12-20T04:28:43.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2023-12-20T04:28:43.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2023-12-20T04:28:43.968 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2023-12-20T04:28:44.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f1e2956d640 1 Processor -- start 2023-12-20T04:28:44.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f1e2956d640 1 -- start start 2023-12-20T04:28:44.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f9d84bbc640 1 Processor -- start 2023-12-20T04:28:44.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f9d84bbc640 1 -- start start 2023-12-20T04:28:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f1e2956d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1e2406c3e0 0x7f1e2406c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f1e2956d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e2415bd50 con 0x7f1e240af4e0 2023-12-20T04:28:44.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f1e2956d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e24164eb0 con 0x7f1e2415b970 2023-12-20T04:28:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f1e2956d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e2406cdb0 con 0x7f1e2406c3e0 2023-12-20T04:28:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f9d84bbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d80158e40 0x7f9d800700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f9d84bbc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d80159220 con 0x7f9d800b3b40 2023-12-20T04:28:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f9d84bbc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d800b3f50 con 0x7f9d80070620 2023-12-20T04:28:44.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.216+0000 7f9d84bbc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d800ca120 con 0x7f9d80158e40 2023-12-20T04:28:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e227fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1e2406c3e0 0x7f1e2406c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e227fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1e2406c3e0 0x7f1e2406c870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48784/0 (socket says 172.21.15.32:48784) 2023-12-20T04:28:44.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e227fc640 1 -- 172.21.15.32:0/733027457 learned_addr learned my addr 172.21.15.32:0/733027457 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7dd9b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f9d80070620 0x7f9d800ca5a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:52120/0 (socket says 172.21.15.32:52120) 2023-12-20T04:28:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7dd9b640 1 -- 172.21.15.32:0/2042333841 learned_addr learned my addr 172.21.15.32:0/2042333841 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7e59c640 1 --2- 172.21.15.32:0/2042333841 >> v2:172.21.15.32:3300/0 conn(0x7f9d80158e40 0x7f9d800700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1723596883 0 0) 0x7f9d800b3f50 con 0x7f9d80070620 2023-12-20T04:28:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d68003530 con 0x7f9d80070620 2023-12-20T04:28:44.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 245769138 0 0) 0x7f9d780014a0 con 0x7f9d800b3b40 2023-12-20T04:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3615622763 0 0) 0x7f1e2415bd50 con 0x7f1e240af4e0 2023-12-20T04:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1e0c003530 con 0x7f1e240af4e0 2023-12-20T04:28:44.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9d800b3f50 con 0x7f9d800b3b40 2023-12-20T04:28:44.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 479357539 0 0) 0x7f1e0c003530 con 0x7f1e240af4e0 2023-12-20T04:28:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 >> v2:172.21.15.39:3300/0 conn(0x7f1e2406c3e0 msgr2=0x7f1e2406c870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 --2- 172.21.15.32:0/733027457 >> v2:172.21.15.39:3300/0 conn(0x7f1e2406c3e0 0x7f1e2406c870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 >> v1:172.21.15.39:6789/0 conn(0x7f1e2415b970 legacy=0x7f1e24071970 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-20T04:28:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e24165150 con 0x7f1e240af4e0 2023-12-20T04:28:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 <== mon.1 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1862750864 0 0) 0x7f9d68003530 con 0x7f9d80070620 2023-12-20T04:28:44.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 >> v2:172.21.15.32:3300/0 conn(0x7f9d80158e40 msgr2=0x7f9d800700e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 --2- 172.21.15.32:0/2042333841 >> v2:172.21.15.32:3300/0 conn(0x7f9d80158e40 0x7f9d800700e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 >> v1:172.21.15.32:6789/0 conn(0x7f9d800b3b40 legacy=0x7f9d800cacb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8006d4a0 con 0x7f9d80070620 2023-12-20T04:28:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 <== mon.1 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f9d70001680 con 0x7f9d80070620 2023-12-20T04:28:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 >> v1:172.21.15.39:6789/0 conn(0x7f9d80070620 legacy=0x7f9d800ca5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 --2- 172.21.15.32:0/2042333841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d68003b10 0x7f9d68003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 --2- 172.21.15.32:0/2042333841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d680047d0 0x7f9d68004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 --2- 172.21.15.32:0/2042333841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d68005190 0x7f9d68014590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800ca120 con 0x7f9d68003b10 2023-12-20T04:28:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d68009c00 con 0x7f9d68005190 2023-12-20T04:28:44.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d68014b20 con 0x7f9d680047d0 2023-12-20T04:28:44.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f9d7e59c640 1 --2- 172.21.15.32:0/2042333841 >> v2:172.21.15.32:3300/0 conn(0x7f9d80158e40 0x7f9d800700e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:28:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f1e1c001a00 con 0x7f1e240af4e0 2023-12-20T04:28:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 >> v1:172.21.15.32:6789/0 conn(0x7f1e240af4e0 legacy=0x7f1e240af930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 --2- 172.21.15.32:0/733027457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e0c003980 0x7f1e0c003d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 --2- 172.21.15.32:0/733027457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e0c0047d0 0x7f1e0c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 --2- 172.21.15.32:0/733027457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e0c005190 0x7f1e0c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2406cdb0 con 0x7f1e0c005190 2023-12-20T04:28:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c00c010 con 0x7f1e0c003980 2023-12-20T04:28:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e0c00c1d0 con 0x7f1e0c0047d0 2023-12-20T04:28:44.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.220+0000 7f1e22ffd640 1 --2- 172.21.15.32:0/733027457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e0c003980 0x7f1e0c003d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f9d7dd9b640 1 --2- 172.21.15.32:0/2042333841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d680047d0 0x7f9d68004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f9d7e59c640 1 --2- 172.21.15.32:0/2042333841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d68003b10 0x7f9d68003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f9d7dd9b640 1 -- 172.21.15.32:0/2042333841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d68005190 msgr2=0x7f9d68014590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f9d7dd9b640 1 --2- 172.21.15.32:0/2042333841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d68005190 0x7f9d68014590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f9d7dd9b640 1 -- 172.21.15.32:0/2042333841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d68003b10 msgr2=0x7f9d68003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f9d7dd9b640 1 --2- 172.21.15.32:0/2042333841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d68003b10 0x7f9d68003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f1e227fc640 1 --2- 172.21.15.32:0/733027457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e0c0047d0 0x7f1e0c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f1e237fe640 1 --2- 172.21.15.32:0/733027457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e0c005190 0x7f1e0c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f1e237fe640 1 -- 172.21.15.32:0/733027457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e0c0047d0 msgr2=0x7f1e0c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f1e237fe640 1 --2- 172.21.15.32:0/733027457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e0c0047d0 0x7f1e0c004c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f1e237fe640 1 -- 172.21.15.32:0/733027457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e0c003980 msgr2=0x7f1e0c003d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f1e237fe640 1 --2- 172.21.15.32:0/733027457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e0c003980 0x7f1e0c003d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f1e237fe640 1 -- 172.21.15.32:0/733027457 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e0c003530 con 0x7f1e0c005190 2023-12-20T04:28:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f771a1f0640 1 Processor -- start 2023-12-20T04:28:44.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f771a1f0640 1 -- start start 2023-12-20T04:28:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f9d7dd9b640 1 -- 172.21.15.32:0/2042333841 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d8006d4a0 con 0x7f9d680047d0 2023-12-20T04:28:44.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f9d7ed9d640 1 -- 172.21.15.32:0/2042333841 reap_dead start 2023-12-20T04:28:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f771a1f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f771406c3e0 0x7f771406c7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f771a1f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7714071990 0x7f7714071e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.224+0000 7f771a1f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f771415d7e0 con 0x7f77140c90e0 2023-12-20T04:28:44.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f771a1f0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7714072350 con 0x7f7714071990 2023-12-20T04:28:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f771a1f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f771415a500 con 0x7f771406c3e0 2023-12-20T04:28:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f77137fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f771406c3e0 0x7f771406c7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f77137fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f771406c3e0 0x7f771406c7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49330/0 (socket says 172.21.15.32:49330) 2023-12-20T04:28:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f77137fe640 1 -- 172.21.15.32:0/1922109743 learned_addr learned my addr 172.21.15.32:0/1922109743 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f77137fe640 1 -- 172.21.15.32:0/1922109743 >> v2:172.21.15.32:3300/0 conn(0x7f7714071990 msgr2=0x7f7714071e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f77137fe640 1 --2- 172.21.15.32:0/1922109743 >> v2:172.21.15.32:3300/0 conn(0x7f7714071990 0x7f7714071e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f77137fe640 1 -- 172.21.15.32:0/1922109743 >> v1:172.21.15.39:6789/0 conn(0x7f77140c90e0 legacy=0x7f77140c9570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f77137fe640 1 -- 172.21.15.32:0/1922109743 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f771406cd00 con 0x7f771406c3e0 2023-12-20T04:28:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f77127fc640 1 -- 172.21.15.32:0/1922109743 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1089237945 0 0) 0x7f771415d7e0 con 0x7f77140c90e0 2023-12-20T04:28:44.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7fc7afe73640 1 Processor -- start 2023-12-20T04:28:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f1e237fe640 1 -- 172.21.15.32:0/733027457 reap_dead start 2023-12-20T04:28:44.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f9d7dd9b640 1 --2- 172.21.15.32:0/2042333841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d680047d0 0x7f9d68004c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000c210 tx=0x7f9d7001cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=306b22c1d1c2070b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d700a5b60 con 0x7f9d680047d0 2023-12-20T04:28:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d700a5cc0 con 0x7f9d680047d0 2023-12-20T04:28:44.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7fc7afe73640 1 -- start start 2023-12-20T04:28:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7fc7afe73640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7a8156b00 0x7fc7a806c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7fc7afe73640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7a806c920 0x7fc7a81597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f9d7d59a640 1 -- 172.21.15.32:0/2042333841 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d70005b20 con 0x7f9d680047d0 2023-12-20T04:28:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f9d84bbc640 1 -- 172.21.15.32:0/2042333841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d680047d0 msgr2=0x7f9d68004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.228+0000 7f9d84bbc640 1 --2- 172.21.15.32:0/2042333841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d680047d0 0x7f9d68004c50 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9d7000c210 tx=0x7f9d7001cb90 comp rx=0 tx=0).stop 2023-12-20T04:28:44.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f1e237fe640 1 --2- 172.21.15.32:0/733027457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e0c005190 0x7f1e0c00b990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c007fd0 tx=0x7f1e1c01cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f622325ca5d0a04 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f9d84bbc640 1 -- 172.21.15.32:0/2042333841 shutdown_connections 2023-12-20T04:28:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f9d84bbc640 1 --2- 172.21.15.32:0/2042333841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d680047d0 0x7f9d68004c50 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f9d84bbc640 1 -- 172.21.15.32:0/2042333841 >> 172.21.15.32:0/2042333841 conn(0x7f9d80069d20 msgr2=0x7f9d8006a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7ad40e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7a806c920 0x7fc7a81597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7afe73640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7a8159ce0 0x7fc7a80701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7afe73640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a806cdd0 con 0x7fc7a806c920 2023-12-20T04:28:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7afe73640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7a815a510 con 0x7fc7a8156b00 2023-12-20T04:28:44.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7afe73640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a815da50 con 0x7fc7a8159ce0 2023-12-20T04:28:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f9d84bbc640 1 -- 172.21.15.32:0/2042333841 shutdown_connections 2023-12-20T04:28:44.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f9d84bbc640 1 -- 172.21.15.32:0/2042333841 wait complete. 2023-12-20T04:28:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f77137fe640 1 --2- 172.21.15.32:0/1922109743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f771406c3e0 0x7f771406c7c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7708006fd0 tx=0x7f770801c4f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a29bf76850b2700c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0a5de0 con 0x7f1e0c005190 2023-12-20T04:28:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f77127fc640 1 -- 172.21.15.32:0/1922109743 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080aa650 con 0x7f771406c3e0 2023-12-20T04:28:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f77127fc640 1 -- 172.21.15.32:0/1922109743 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77080aa7b0 con 0x7f771406c3e0 2023-12-20T04:28:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f77127fc640 1 -- 172.21.15.32:0/1922109743 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080aab40 con 0x7f771406c3e0 2023-12-20T04:28:44.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7adc0f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7a8156b00 0x7fc7a806c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7adc0f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7a8156b00 0x7fc7a806c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48798/0 (socket says 172.21.15.32:48798) 2023-12-20T04:28:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7adc0f640 1 -- 172.21.15.32:0/1459871977 learned_addr learned my addr 172.21.15.32:0/1459871977 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7ae410640 1 --2- 172.21.15.32:0/1459871977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7a8159ce0 0x7fc7a80701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7adc0f640 1 -- 172.21.15.32:0/1459871977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7a8159ce0 msgr2=0x7fc7a80701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7adc0f640 1 --2- 172.21.15.32:0/1459871977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7a8159ce0 0x7fc7a80701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7adc0f640 1 -- 172.21.15.32:0/1459871977 >> v2:172.21.15.32:3300/0 conn(0x7fc7a806c920 msgr2=0x7fc7a81597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7adc0f640 1 --2- 172.21.15.32:0/1459871977 >> v2:172.21.15.32:3300/0 conn(0x7fc7a806c920 0x7fc7a81597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7fc7adc0f640 1 -- 172.21.15.32:0/1459871977 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a815cc90 con 0x7fc7a8156b00 2023-12-20T04:28:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1c0ad440 con 0x7f1e0c005190 2023-12-20T04:28:44.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f1e21ffb640 1 -- 172.21.15.32:0/733027457 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0a5de0 con 0x7f1e0c005190 2023-12-20T04:28:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f771a1f0640 1 -- 172.21.15.32:0/1922109743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f771406c3e0 msgr2=0x7f771406c7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f771a1f0640 1 --2- 172.21.15.32:0/1922109743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f771406c3e0 0x7f771406c7c0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7708006fd0 tx=0x7f770801c4f0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f1e2956d640 1 -- 172.21.15.32:0/733027457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e0c005190 msgr2=0x7f1e0c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f1e2956d640 1 --2- 172.21.15.32:0/733027457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e0c005190 0x7f1e0c00b990 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c007fd0 tx=0x7f1e1c01cf20 comp rx=0 tx=0).stop 2023-12-20T04:28:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f771a1f0640 1 -- 172.21.15.32:0/1922109743 shutdown_connections 2023-12-20T04:28:44.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f771a1f0640 1 --2- 172.21.15.32:0/1922109743 >> v2:172.21.15.32:3300/0 conn(0x7f7714071990 0x7f7714071e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f771a1f0640 1 --2- 172.21.15.32:0/1922109743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f771406c3e0 0x7f771406c7c0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f771a1f0640 1 -- 172.21.15.32:0/1922109743 >> 172.21.15.32:0/1922109743 conn(0x7f771406a320 msgr2=0x7f77140ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f175d325640 1 Processor -- start 2023-12-20T04:28:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f1e2956d640 1 -- 172.21.15.32:0/733027457 shutdown_connections 2023-12-20T04:28:44.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f1e2956d640 1 --2- 172.21.15.32:0/733027457 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e0c005190 0x7f1e0c00b990 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f1e2956d640 1 -- 172.21.15.32:0/733027457 >> 172.21.15.32:0/733027457 conn(0x7f1e240cb1e0 msgr2=0x7f1e240cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f175d325640 1 -- start start 2023-12-20T04:28:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f1e2956d640 1 -- 172.21.15.32:0/733027457 shutdown_connections 2023-12-20T04:28:44.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f1e2956d640 1 -- 172.21.15.32:0/733027457 wait complete. 2023-12-20T04:28:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f9d84bbc640 1 Processor -- start 2023-12-20T04:28:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f9d84bbc640 1 -- start start 2023-12-20T04:28:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.232+0000 7f771a1f0640 1 -- 172.21.15.32:0/1922109743 shutdown_connections 2023-12-20T04:28:44.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f771a1f0640 1 -- 172.21.15.32:0/1922109743 wait complete. 2023-12-20T04:28:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f175d325640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f175815cdb0 0x7f17580d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d84bbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d680047d0 0x7f9d8015b3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f175d325640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f17580d2740 0x7f17580cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f175d325640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17580d1120 0x7f17580cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f175d325640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f175815d190 con 0x7f175815cdb0 2023-12-20T04:28:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f175d325640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17580d16a0 con 0x7f17580d2740 2023-12-20T04:28:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f175d325640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17580d1980 con 0x7f17580d1120 2023-12-20T04:28:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d84bbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d801593c0 0x7f9d801e4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d84bbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d800c0ff0 0x7f9d800c1470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d84bbc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d800c19b0 con 0x7f9d801593c0 2023-12-20T04:28:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d84bbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8015aea0 con 0x7f9d680047d0 2023-12-20T04:28:44.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d84bbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d8015b930 con 0x7f9d800c0ff0 2023-12-20T04:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f1e2956d640 1 Processor -- start 2023-12-20T04:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f1e2956d640 1 -- start start 2023-12-20T04:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f1756ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f175815cdb0 0x7f17580d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f1756ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f175815cdb0 0x7f17580d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48252/0 (socket says 172.21.15.32:48252) 2023-12-20T04:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f1756ffd640 1 -- 172.21.15.32:0/208018590 learned_addr learned my addr 172.21.15.32:0/208018590 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f17577fe640 1 --2- 172.21.15.32:0/208018590 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17580d1120 0x7f17580cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f17567fc640 1 --2- 172.21.15.32:0/208018590 >> v2:172.21.15.39:3300/0 conn(0x7f17580d2740 0x7f17580cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f4c56152640 1 Processor -- start 2023-12-20T04:28:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d7ed9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d800c0ff0 0x7f9d800c1470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d7e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d680047d0 0x7f9d8015b3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d7ed9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d800c0ff0 0x7f9d800c1470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49344/0 (socket says 172.21.15.32:49344) 2023-12-20T04:28:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d7ed9d640 1 -- 172.21.15.32:0/3488683684 learned_addr learned my addr 172.21.15.32:0/3488683684 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f17577fe640 1 -- 172.21.15.32:0/208018590 >> v2:172.21.15.39:3300/0 conn(0x7f17580d2740 msgr2=0x7f17580cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f17577fe640 1 --2- 172.21.15.32:0/208018590 >> v2:172.21.15.39:3300/0 conn(0x7f17580d2740 0x7f17580cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f17577fe640 1 -- 172.21.15.32:0/208018590 >> v2:172.21.15.32:3300/0 conn(0x7f175815cdb0 msgr2=0x7f17580d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f4c56152640 1 -- start start 2023-12-20T04:28:44.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d7ed9d640 1 -- 172.21.15.32:0/3488683684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d680047d0 msgr2=0x7f9d8015b3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d7ed9d640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d680047d0 0x7f9d8015b3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d7ed9d640 1 -- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d801593c0 msgr2=0x7f9d801e4cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d7ed9d640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d801593c0 0x7f9d801e4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d7ed9d640 1 -- 172.21.15.32:0/3488683684 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d80159960 con 0x7f9d800c0ff0 2023-12-20T04:28:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f1e2956d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e0c005190 0x7f1e240b2f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f1e2956d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e240c6f10 0x7f1e2420a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f1e2956d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e2415a860 0x7f1e2415ad10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e2956d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e2415b250 con 0x7f1e0c005190 2023-12-20T04:28:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e2956d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240c7390 con 0x7f1e240c6f10 2023-12-20T04:28:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e2956d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e240b3630 con 0x7f1e2415a860 2023-12-20T04:28:44.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7fc7adc0f640 1 --2- 172.21.15.32:0/1459871977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7a8156b00 0x7fc7a806c3e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7a806d6b0 tx=0x7fc7a401c3d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acce72d89874ba87 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f9d7ed9d640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d800c0ff0 0x7f9d800c1470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9d8006e160 tx=0x7f9d7800cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e237fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e2415a860 0x7f1e2415ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e237fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e2415a860 0x7f1e2415ad10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49358/0 (socket says 172.21.15.32:49358) 2023-12-20T04:28:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e237fe640 1 -- 172.21.15.32:0/2819818307 learned_addr learned my addr 172.21.15.32:0/2819818307 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e22ffd640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e0c005190 0x7f1e240b2f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7fc7acc0d640 1 -- 172.21.15.32:0/1459871977 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40ab600 con 0x7fc7a8156b00 2023-12-20T04:28:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7fc7acc0d640 1 -- 172.21.15.32:0/1459871977 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7a40a7050 con 0x7fc7a8156b00 2023-12-20T04:28:44.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7fc7acc0d640 1 -- 172.21.15.32:0/1459871977 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7a40ab980 con 0x7fc7a8156b00 2023-12-20T04:28:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e227fc640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e240c6f10 0x7f1e2420a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f9d6effd640 1 -- 172.21.15.32:0/3488683684 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78005d50 con 0x7f9d800c0ff0 2023-12-20T04:28:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f9d6effd640 1 -- 172.21.15.32:0/3488683684 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d78003980 con 0x7f9d800c0ff0 2023-12-20T04:28:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f9d6effd640 1 -- 172.21.15.32:0/3488683684 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d78003d30 con 0x7f9d800c0ff0 2023-12-20T04:28:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e237fe640 1 -- 172.21.15.32:0/2819818307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e240c6f10 msgr2=0x7f1e2420a0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e237fe640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e240c6f10 0x7f1e2420a0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e237fe640 1 -- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e0c005190 msgr2=0x7f1e240b2f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e237fe640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e0c005190 0x7f1e240b2f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e237fe640 1 -- 172.21.15.32:0/2819818307 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e240b3910 con 0x7f1e2415a860 2023-12-20T04:28:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f9d84bbc640 1 -- 172.21.15.32:0/3488683684 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d80159c40 con 0x7f9d800c0ff0 2023-12-20T04:28:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f9d84bbc640 1 -- 172.21.15.32:0/3488683684 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d8015a200 con 0x7f9d800c0ff0 2023-12-20T04:28:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c56152640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c50156b00 0x7f4c5006c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c56152640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4c5006c920 0x7f4c501597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e227fc640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e240c6f10 0x7f1e2420a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7fc7afe73640 1 -- 172.21.15.32:0/1459871977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7a8156b00 msgr2=0x7fc7a806c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7fc7afe73640 1 --2- 172.21.15.32:0/1459871977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7a8156b00 0x7fc7a806c3e0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fc7a806d6b0 tx=0x7fc7a401c3d0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f771a1f0640 1 Processor -- start 2023-12-20T04:28:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f771a1f0640 1 -- start start 2023-12-20T04:28:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f17577fe640 1 --2- 172.21.15.32:0/208018590 >> v2:172.21.15.32:3300/0 conn(0x7f175815cdb0 0x7f17580d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.236+0000 7f17577fe640 1 -- 172.21.15.32:0/208018590 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17580d1c60 con 0x7f17580d1120 2023-12-20T04:28:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f771a1f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f771406c3e0 0x7f771414c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f771a1f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7714071990 0x7f77140ce2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f771a1f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f77140c90e0 0x7f77141f94d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f771a1f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f771414d0e0 con 0x7f771406c3e0 2023-12-20T04:28:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e237fe640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e2415a860 0x7f1e2415ad10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c01cd50 tx=0x7f1e1c019cc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f771a1f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f77140c7610 con 0x7f7714071990 2023-12-20T04:28:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f771a1f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f77140c78b0 con 0x7f77140c90e0 2023-12-20T04:28:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c4effd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4c5006c920 0x7f4c501597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c4effd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4c5006c920 0x7f4c501597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48826/0 (socket says 172.21.15.32:48826) 2023-12-20T04:28:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c4effd640 1 -- 172.21.15.32:0/2616283435 learned_addr learned my addr 172.21.15.32:0/2616283435 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c56152640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c5015a510 con 0x7f4c50159ce0 2023-12-20T04:28:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c56152640 1 -- 172.21.15.32:0/2616283435 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c5006cdd0 con 0x7f4c50156b00 2023-12-20T04:28:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c56152640 1 -- 172.21.15.32:0/2616283435 --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c500ca0f0 con 0x7f4c5006c920 2023-12-20T04:28:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e0b7fe640 1 -- 172.21.15.32:0/2819818307 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c0a5de0 con 0x7f1e2415a860 2023-12-20T04:28:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e0b7fe640 1 -- 172.21.15.32:0/2819818307 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e1c0adbf0 con 0x7f1e2415a860 2023-12-20T04:28:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e0b7fe640 1 -- 172.21.15.32:0/2819818307 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e1c003980 con 0x7f1e2415a860 2023-12-20T04:28:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e2956d640 1 -- 172.21.15.32:0/2819818307 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e240b3bf0 con 0x7f1e2415a860 2023-12-20T04:28:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7fc7afe73640 1 -- 172.21.15.32:0/1459871977 shutdown_connections 2023-12-20T04:28:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7fc7afe73640 1 --2- 172.21.15.32:0/1459871977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7a8159ce0 0x7fc7a80701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7fc7afe73640 1 --2- 172.21.15.32:0/1459871977 >> v2:172.21.15.32:3300/0 conn(0x7fc7a806c920 0x7fc7a81597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7fc7afe73640 1 --2- 172.21.15.32:0/1459871977 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7a8156b00 0x7fc7a806c3e0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7fc7afe73640 1 -- 172.21.15.32:0/1459871977 >> 172.21.15.32:0/1459871977 conn(0x7fc7a80cb1e0 msgr2=0x7fc7a80cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f1e2956d640 1 -- 172.21.15.32:0/2819818307 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e241581a0 con 0x7f1e2415a860 2023-12-20T04:28:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f77137fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f771406c3e0 0x7f771414c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f7712ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7714071990 0x7f77140ce2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c4f7fe640 1 --2- 172.21.15.32:0/2616283435 >> v2:172.21.15.32:3300/0 conn(0x7f4c50156b00 0x7f4c5006c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f77137fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f771406c3e0 0x7f771414c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48278/0 (socket says 172.21.15.32:48278) 2023-12-20T04:28:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f77137fe640 1 -- 172.21.15.32:0/306052657 learned_addr learned my addr 172.21.15.32:0/306052657 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f9d84bbc640 1 -- 172.21.15.32:0/3488683684 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f9d800c9d40 con 0x7f9d800c0ff0 2023-12-20T04:28:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c4e7fc640 1 -- 172.21.15.32:0/2616283435 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727265427 0 0) 0x7f4c5015a510 con 0x7f4c50159ce0 2023-12-20T04:28:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f7713fff640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f77140c90e0 0x7f77141f94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c4e7fc640 1 -- 172.21.15.32:0/2616283435 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c38003530 con 0x7f4c50159ce0 2023-12-20T04:28:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f77137fe640 1 -- 172.21.15.32:0/306052657 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f77140c90e0 msgr2=0x7f77141f94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f77137fe640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f77140c90e0 0x7f77141f94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f77137fe640 1 -- 172.21.15.32:0/306052657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7714071990 msgr2=0x7f77140ce2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f77137fe640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7714071990 0x7f77140ce2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f77137fe640 1 -- 172.21.15.32:0/306052657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7708009480 con 0x7f771406c3e0 2023-12-20T04:28:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f77137fe640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f771406c3e0 0x7f771414c550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f770801aba0 tx=0x7f7708019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f17577fe640 1 --2- 172.21.15.32:0/208018590 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17580d1120 0x7f17580cfab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f17580cd570 tx=0x7f174801c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d059541aefdf8702 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f9d6effd640 1 -- 172.21.15.32:0/3488683684 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9d7800a640 con 0x7f9d800c0ff0 2023-12-20T04:28:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f1755ffb640 1 -- 172.21.15.32:0/208018590 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480a5960 con 0x7f17580d1120 2023-12-20T04:28:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f1755ffb640 1 -- 172.21.15.32:0/208018590 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17480a5ac0 con 0x7f17580d1120 2023-12-20T04:28:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f1755ffb640 1 -- 172.21.15.32:0/208018590 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17480a5e50 con 0x7f17580d1120 2023-12-20T04:28:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f1e097fa640 1 -- 172.21.15.32:0/2819818307 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f1e24165710 con 0x7f1e2415a860 2023-12-20T04:28:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f76fbfff640 1 -- 172.21.15.32:0/306052657 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f77080aad30 con 0x7f771406c3e0 2023-12-20T04:28:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c4effd640 1 -- 172.21.15.32:0/2616283435 >> v2:172.21.15.32:3300/0 conn(0x7f4c50156b00 msgr2=0x7f4c5006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c4effd640 1 --2- 172.21.15.32:0/2616283435 >> v2:172.21.15.32:3300/0 conn(0x7f4c50156b00 0x7f4c5006c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c4effd640 1 -- 172.21.15.32:0/2616283435 >> v1:172.21.15.39:6789/0 conn(0x7f4c50159ce0 legacy=0x7f4c500701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.240+0000 7f4c4effd640 1 -- 172.21.15.32:0/2616283435 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c5015da50 con 0x7f4c5006c920 2023-12-20T04:28:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f1e0b7fe640 1 -- 172.21.15.32:0/2819818307 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1e1c003ae0 con 0x7f1e2415a860 2023-12-20T04:28:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f4c4f7fe640 1 --2- 172.21.15.32:0/2616283435 >> v2:172.21.15.32:3300/0 conn(0x7f4c50156b00 0x7f4c5006c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f771a1f0640 1 -- 172.21.15.32:0/306052657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f77141666e0 con 0x7f771406c3e0 2023-12-20T04:28:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f76fbfff640 1 -- 172.21.15.32:0/306052657 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f77080aae90 con 0x7f771406c3e0 2023-12-20T04:28:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f771a1f0640 1 -- 172.21.15.32:0/306052657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f77140c7e70 con 0x7f771406c3e0 2023-12-20T04:28:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f9d6effd640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d60065b60 0x7f9d60068020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f76fbfff640 1 -- 172.21.15.32:0/306052657 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7708003860 con 0x7f771406c3e0 2023-12-20T04:28:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f175d325640 1 -- 172.21.15.32:0/208018590 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17580d1120 msgr2=0x7f17580cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f175d325640 1 --2- 172.21.15.32:0/208018590 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17580d1120 0x7f17580cfab0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f17580cd570 tx=0x7f174801c8e0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f9d6effd640 1 -- 172.21.15.32:0/3488683684 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9d78082490 con 0x7f9d800c0ff0 2023-12-20T04:28:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f1e0b7fe640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1e10065a90 0x7f1e10067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f1e0b7fe640 1 -- 172.21.15.32:0/2819818307 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1e1c11ce20 con 0x7f1e2415a860 2023-12-20T04:28:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f9d6effd640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] conn(0x7f9d6006dd80 0x7f9d600701a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f9d6effd640 1 -- 172.21.15.32:0/3488683684 --> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f9d60070850 con 0x7f9d6006dd80 2023-12-20T04:28:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f9d6effd640 1 -- 172.21.15.32:0/3488683684 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9d78084ab0 con 0x7f9d800c0ff0 2023-12-20T04:28:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7fc7afe73640 1 -- 172.21.15.32:0/1459871977 shutdown_connections 2023-12-20T04:28:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7fc7afe73640 1 -- 172.21.15.32:0/1459871977 wait complete. 2023-12-20T04:28:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f175d325640 1 -- 172.21.15.32:0/208018590 shutdown_connections 2023-12-20T04:28:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f175d325640 1 --2- 172.21.15.32:0/208018590 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17580d1120 0x7f17580cfab0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f175d325640 1 --2- 172.21.15.32:0/208018590 >> v2:172.21.15.39:3300/0 conn(0x7f17580d2740 0x7f17580cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f175d325640 1 --2- 172.21.15.32:0/208018590 >> v2:172.21.15.32:3300/0 conn(0x7f175815cdb0 0x7f17580d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.244+0000 7f175d325640 1 -- 172.21.15.32:0/208018590 >> 172.21.15.32:0/208018590 conn(0x7f1758065ee0 msgr2=0x7f175806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f175d325640 1 -- 172.21.15.32:0/208018590 shutdown_connections 2023-12-20T04:28:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f175d325640 1 -- 172.21.15.32:0/208018590 wait complete. 2023-12-20T04:28:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f1e0b7fe640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] conn(0x7f1e1006dc30 0x7f1e10070050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f1e0b7fe640 1 -- 172.21.15.32:0/2819818307 --> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f1e10070700 con 0x7f1e1006dc30 2023-12-20T04:28:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f1e0b7fe640 1 -- 172.21.15.32:0/2819818307 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1e1c11f400 con 0x7f1e2415a860 2023-12-20T04:28:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f9d7dd9b640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] conn(0x7f9d6006dd80 0x7f9d600701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f76f9ffb640 1 -- 172.21.15.32:0/306052657 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f77140cc660 con 0x7f771406c3e0 2023-12-20T04:28:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f1e22ffd640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1e10065a90 0x7f1e10067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f9d7e59c640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d60065b60 0x7f9d60068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f4c4effd640 1 --2- 172.21.15.32:0/2616283435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c5006c920 0x7f4c501597a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c5006fba0 tx=0x7f4c4401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31124c810c70d5ec server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f1e227fc640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] conn(0x7f1e1006dc30 0x7f1e10070050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f4c4e7fc640 1 -- 172.21.15.32:0/2616283435 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c440ac410 con 0x7f4c5006c920 2023-12-20T04:28:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f4c4e7fc640 1 -- 172.21.15.32:0/2616283435 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c440ac570 con 0x7f4c5006c920 2023-12-20T04:28:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f33fe96d640 1 Processor -- start 2023-12-20T04:28:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f33fe96d640 1 -- start start 2023-12-20T04:28:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f76fbfff640 1 -- 172.21.15.32:0/306052657 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f77080039c0 con 0x7f771406c3e0 2023-12-20T04:28:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f4c4e7fc640 1 -- 172.21.15.32:0/2616283435 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c440ac410 con 0x7f4c5006c920 2023-12-20T04:28:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f4c56152640 1 -- 172.21.15.32:0/2616283435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c5006c920 msgr2=0x7f4c501597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f4c56152640 1 --2- 172.21.15.32:0/2616283435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c5006c920 0x7f4c501597a0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4c5006fba0 tx=0x7f4c4401c3f0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7fc7afe73640 1 Processor -- start 2023-12-20T04:28:44.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7fc7afe73640 1 -- start start 2023-12-20T04:28:44.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f9d7dd9b640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] conn(0x7f9d6006dd80 0x7f9d600701a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f4c56152640 1 -- 172.21.15.32:0/2616283435 shutdown_connections 2023-12-20T04:28:44.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f4c56152640 1 --2- 172.21.15.32:0/2616283435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c5006c920 0x7f4c501597a0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f175d325640 1 Processor -- start 2023-12-20T04:28:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f4c56152640 1 --2- 172.21.15.32:0/2616283435 >> v2:172.21.15.32:3300/0 conn(0x7f4c50156b00 0x7f4c5006c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f4c56152640 1 -- 172.21.15.32:0/2616283435 >> 172.21.15.32:0/2616283435 conn(0x7f4c500cb1e0 msgr2=0x7f4c500cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f4c56152640 1 -- 172.21.15.32:0/2616283435 shutdown_connections 2023-12-20T04:28:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f1e227fc640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] conn(0x7f1e1006dc30 0x7f1e10070050 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f1e22ffd640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1e10065a90 0x7f1e10067f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1e240aef00 tx=0x7f1e18008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f9d6effd640 1 -- 172.21.15.32:0/3488683684 <== osd.1 v2:172.21.15.32:6800/38077 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f9d60070850 con 0x7f9d6006dd80 2023-12-20T04:28:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f9d7e59c640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d60065b60 0x7f9d60068020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d8002c850 tx=0x7f9d74004220 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.248+0000 7f175d325640 1 -- start start 2023-12-20T04:28:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f175d325640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580d1120 0x7f175814cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f175d325640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17580d2740 0x7f17580ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f175d325640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f175815cdb0 0x7f175814d730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f175d325640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580cbbc0 con 0x7f17580d2740 2023-12-20T04:28:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f175d325640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17580cafb0 con 0x7f17580d1120 2023-12-20T04:28:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f175d325640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17580cb290 con 0x7f175815cdb0 2023-12-20T04:28:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33fe96d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33f81599e0 0x7f33f80aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f4c56152640 1 -- 172.21.15.32:0/2616283435 wait complete. 2023-12-20T04:28:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f76fbfff640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7700065ac0 0x7f7700067f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f76fbfff640 1 -- 172.21.15.32:0/306052657 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f770811dd30 con 0x7f771406c3e0 2023-12-20T04:28:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f76fbfff640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] conn(0x7f770006dc60 0x7f7700070080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f76fbfff640 1 -- 172.21.15.32:0/306052657 --> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7700070730 con 0x7f770006dc60 2023-12-20T04:28:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f76fbfff640 1 -- 172.21.15.32:0/306052657 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7708003860 con 0x7f771406c3e0 2023-12-20T04:28:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f7712ffd640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7700065ac0 0x7f7700067f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7afe73640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7a806c920 0x7fc7a814c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7afe73640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7a8156b00 0x7fc7a80b1530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7afe73640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7a8159ce0 0x7fc7a80c7ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7afe73640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a8137280 con 0x7fc7a8159ce0 2023-12-20T04:28:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7afe73640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a814c960 con 0x7fc7a806c920 2023-12-20T04:28:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7afe73640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7a80b3080 con 0x7fc7a8156b00 2023-12-20T04:28:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f7713fff640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] conn(0x7f770006dc60 0x7f7700070080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f7712ffd640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7700065ac0 0x7f7700067f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f770c006fd0 tx=0x7f770c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f17577fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f175815cdb0 0x7f175814d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f17577fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f175815cdb0 0x7f175814d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49376/0 (socket says 172.21.15.32:49376) 2023-12-20T04:28:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f17577fe640 1 -- 172.21.15.32:0/2498496927 learned_addr learned my addr 172.21.15.32:0/2498496927 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33fe96d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f33f80c90e0 0x7f33f80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33fe96d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33f80711d0 con 0x7f33f80b0b30 2023-12-20T04:28:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33fe96d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33f80c98c0 con 0x7f33f80c90e0 2023-12-20T04:28:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33fe96d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80c9ba0 con 0x7f33f81599e0 2023-12-20T04:28:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33fcf0a640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f33f80b0b30 0x7f33f815d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:52138/0 (socket says 172.21.15.32:52138) 2023-12-20T04:28:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33fcf0a640 1 -- 172.21.15.32:0/3631427046 learned_addr learned my addr 172.21.15.32:0/3631427046 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7ad40e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7a8156b00 0x7fc7a80b1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7ad40e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7a8156b00 0x7fc7a80b1530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49400/0 (socket says 172.21.15.32:49400) 2023-12-20T04:28:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7ad40e640 1 -- 172.21.15.32:0/4270339776 learned_addr learned my addr 172.21.15.32:0/4270339776 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7adc0f640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7a806c920 0x7fc7a814c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7ad40e640 1 -- 172.21.15.32:0/4270339776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7a806c920 msgr2=0x7fc7a814c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7ad40e640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7a806c920 0x7fc7a814c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7ad40e640 1 -- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7a8159ce0 msgr2=0x7fc7a80c7ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7ad40e640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7a8159ce0 0x7fc7a80c7ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7fc7ad40e640 1 -- 172.21.15.32:0/4270339776 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7a40094d0 con 0x7fc7a8156b00 2023-12-20T04:28:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f1756ffd640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580d1120 0x7f175814cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f17567fc640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17580d2740 0x7f17580ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f7fff640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33f81599e0 0x7f33f80aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33ef7fe640 1 --2- 172.21.15.32:0/3631427046 >> v2:172.21.15.39:3300/0 conn(0x7f33f80c90e0 0x7f33f80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164889301 0 0) 0x7f33f80711d0 con 0x7f33f80b0b30 2023-12-20T04:28:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33e4003530 con 0x7f33f80b0b30 2023-12-20T04:28:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f043cf60640 1 Processor -- start 2023-12-20T04:28:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f043cf60640 1 -- start start 2023-12-20T04:28:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 <== mon.1 v1:172.21.15.39:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2972798535 0 0) 0x7f33e4003530 con 0x7f33f80b0b30 2023-12-20T04:28:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33f81599e0 msgr2=0x7f33f80aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33f81599e0 0x7f33f80aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 >> v2:172.21.15.39:3300/0 conn(0x7f33f80c90e0 msgr2=0x7f33f80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 --2- 172.21.15.32:0/3631427046 >> v2:172.21.15.39:3300/0 conn(0x7f33f80c90e0 0x7f33f80b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 --> v1:172.21.15.39:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f80c9e80 con 0x7f33f80b0b30 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 <== mon.1 v1:172.21.15.39:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f33f0001680 con 0x7f33f80b0b30 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 >> v1:172.21.15.39:6789/0 conn(0x7f33f80b0b30 legacy=0x7f33f815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e4003980 0x7f33e4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e4004320 0x7f33e40047f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e4004e60 0x7f33e400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80c98c0 con 0x7f33e4004e60 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f80c9ba0 con 0x7f33e4004320 2023-12-20T04:28:44.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.252+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e400bed0 con 0x7f33e4003980 2023-12-20T04:28:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7fc7ad40e640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7a8156b00 0x7fc7a80b1530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc79800c7f0 tx=0x7fc79800ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f1756ffd640 1 -- 172.21.15.32:0/2498496927 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f175815cdb0 msgr2=0x7f175814d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f1756ffd640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f175815cdb0 0x7f175814d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f1756ffd640 1 -- 172.21.15.32:0/2498496927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17580d2740 msgr2=0x7f17580ca610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f1756ffd640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17580d2740 0x7f17580ca610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f1756ffd640 1 -- 172.21.15.32:0/2498496927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1748009a70 con 0x7f17580d1120 2023-12-20T04:28:44.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f33ef7fe640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e4004320 0x7f33e40047f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f33f7fff640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e4003980 0x7f33e4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f33fcf0a640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e4004e60 0x7f33e400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f1756ffd640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580d1120 0x7f175814cf00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f175000d5d0 tx=0x7f175000da90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7fc79e7fc640 1 -- 172.21.15.32:0/4270339776 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798004220 con 0x7fc7a8156b00 2023-12-20T04:28:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7fc79e7fc640 1 -- 172.21.15.32:0/4270339776 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7980043a0 con 0x7fc7a8156b00 2023-12-20T04:28:44.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7fc79e7fc640 1 -- 172.21.15.32:0/4270339776 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc798003860 con 0x7fc7a8156b00 2023-12-20T04:28:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f33ef7fe640 1 -- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e4003980 msgr2=0x7f33e4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f33ef7fe640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e4003980 0x7f33e4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f33ef7fe640 1 -- 172.21.15.32:0/3631427046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e4004e60 msgr2=0x7f33e400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f33ef7fe640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e4004e60 0x7f33e400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f33ef7fe640 1 -- 172.21.15.32:0/3631427046 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f80c9e80 con 0x7f33e4004320 2023-12-20T04:28:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7fc7afe73640 1 -- 172.21.15.32:0/4270339776 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7a8136700 con 0x7fc7a8156b00 2023-12-20T04:28:44.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f4c56152640 1 Processor -- start 2023-12-20T04:28:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7fc7afe73640 1 -- 172.21.15.32:0/4270339776 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7a8136cc0 con 0x7fc7a8156b00 2023-12-20T04:28:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f043cf60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f043815cdb0 0x7f04380d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f043cf60640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04380d2740 0x7f04380cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f043cf60640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f04380b6ca0 con 0x7f04380d1120 2023-12-20T04:28:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f043cf60640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f043815d190 con 0x7f04380d2740 2023-12-20T04:28:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f043cf60640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04380d16a0 con 0x7f043815cdb0 2023-12-20T04:28:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f0435d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04380d2740 0x7f04380cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f043659c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f043815cdb0 0x7f04380d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f0435d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04380d2740 0x7f04380cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48320/0 (socket says 172.21.15.32:48320) 2023-12-20T04:28:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f0435d9b640 1 -- 172.21.15.32:0/1950522543 learned_addr learned my addr 172.21.15.32:0/1950522543 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f0435d9b640 1 -- 172.21.15.32:0/1950522543 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f043815cdb0 msgr2=0x7f04380d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f0435d9b640 1 --2- 172.21.15.32:0/1950522543 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f043815cdb0 0x7f04380d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f1e0b7fe640 1 -- 172.21.15.32:0/2819818307 <== osd.0 v2:172.21.15.32:6808/38080 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f1e10070700 con 0x7f1e1006dc30 2023-12-20T04:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f0435d9b640 1 -- 172.21.15.32:0/1950522543 >> v1:172.21.15.32:6789/0 conn(0x7f04380d1120 legacy=0x7f04380b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f0435d9b640 1 -- 172.21.15.32:0/1950522543 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04380d1980 con 0x7f04380d2740 2023-12-20T04:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7fc7afe73640 1 -- 172.21.15.32:0/4270339776 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7fc7a80af9f0 con 0x7fc7a8156b00 2023-12-20T04:28:44.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f043659c640 1 --2- 172.21.15.32:0/1950522543 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f043815cdb0 0x7f04380d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f173f7fe640 1 -- 172.21.15.32:0/2498496927 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750014620 con 0x7f17580d1120 2023-12-20T04:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f173f7fe640 1 -- 172.21.15.32:0/2498496927 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1750014780 con 0x7f17580d1120 2023-12-20T04:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f173f7fe640 1 -- 172.21.15.32:0/2498496927 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1750014af0 con 0x7f17580d1120 2023-12-20T04:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f4c56152640 1 -- start start 2023-12-20T04:28:44.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f175d325640 1 -- 172.21.15.32:0/2498496927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17580cb570 con 0x7f17580d1120 2023-12-20T04:28:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.256+0000 7f175d325640 1 -- 172.21.15.32:0/2498496927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17580cab50 con 0x7f17580d1120 2023-12-20T04:28:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c56152640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5006c920 0x7f4c50201b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c56152640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c50156b00 0x7f4c502059b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c56152640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c50159ce0 0x7f4c500ccc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c56152640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c5014d600 con 0x7f4c5006c920 2023-12-20T04:28:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c56152640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c50206430 con 0x7f4c50159ce0 2023-12-20T04:28:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c56152640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c50206710 con 0x7f4c50156b00 2023-12-20T04:28:44.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f175d325640 1 -- 172.21.15.32:0/2498496927 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f17580b8a60 con 0x7f17580d1120 2023-12-20T04:28:44.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f33ef7fe640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e4004320 0x7f33e40047f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009ae0 tx=0x7f33e001ca90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=57cb5f7672a45ad server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f7713fff640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] conn(0x7f770006dc60 0x7f7700070080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c4effd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c50156b00 0x7f4c502059b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c4effd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c50156b00 0x7f4c502059b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49428/0 (socket says 172.21.15.32:49428) 2023-12-20T04:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c4effd640 1 -- 172.21.15.32:0/1218624004 learned_addr learned my addr 172.21.15.32:0/1218624004 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c4ffff640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c50159ce0 0x7f4c500ccc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c4f7fe640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5006c920 0x7f4c50201b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c4f7fe640 1 -- 172.21.15.32:0/1218624004 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c50156b00 msgr2=0x7f4c502059b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c4f7fe640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c50156b00 0x7f4c502059b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c4f7fe640 1 -- 172.21.15.32:0/1218624004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c50159ce0 msgr2=0x7f4c500ccc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c4f7fe640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c50159ce0 0x7f4c500ccc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c4f7fe640 1 -- 172.21.15.32:0/1218624004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c4000ae50 con 0x7f4c5006c920 2023-12-20T04:28:44.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f4c4ffff640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c50159ce0 0x7f4c500ccc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7f173f7fe640 1 -- 172.21.15.32:0/2498496927 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1750014c50 con 0x7f17580d1120 2023-12-20T04:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.260+0000 7fc79e7fc640 1 -- 172.21.15.32:0/4270339776 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc798003bd0 con 0x7fc7a8156b00 2023-12-20T04:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7fc79e7fc640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc78c065b30 0x7fc78c067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7fc79e7fc640 1 -- 172.21.15.32:0/4270339776 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc798083550 con 0x7fc7a8156b00 2023-12-20T04:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7fc79e7fc640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] conn(0x7fc78c06dd50 0x7fc78c070170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7fc7adc0f640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc78c065b30 0x7fc78c067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f76fbfff640 1 -- 172.21.15.32:0/306052657 <== osd.3 v2:172.21.15.32:6816/38083 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7700070730 con 0x7f770006dc60 2023-12-20T04:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f173f7fe640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1744065ab0 0x7f1744067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f173f7fe640 1 -- 172.21.15.32:0/2498496927 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17500836a0 con 0x7f17580d1120 2023-12-20T04:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f173f7fe640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] conn(0x7f174406dcd0 0x7f17440700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f173f7fe640 1 -- 172.21.15.32:0/2498496927 --> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f17440707a0 con 0x7f174406dcd0 2023-12-20T04:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f173f7fe640 1 -- 172.21.15.32:0/2498496927 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f1750085c80 con 0x7f17580d1120 2023-12-20T04:28:44.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f4c4f7fe640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5006c920 0x7f4c50201b10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009860 tx=0x7f4c40009d20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e00a5680 con 0x7f33e4004320 2023-12-20T04:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e00a57e0 con 0x7f33e4004320 2023-12-20T04:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f33f77fe640 1 -- 172.21.15.32:0/3631427046 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e00a5bd0 con 0x7f33e4004320 2023-12-20T04:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f33fe96d640 1 -- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e4004320 msgr2=0x7f33e40047f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f33fe96d640 1 --2- 172.21.15.32:0/3631427046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e4004320 0x7f33e40047f0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f33e0009ae0 tx=0x7f33e001ca90 comp rx=0 tx=0).stop 2023-12-20T04:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f33fcf0a640 1 -- 172.21.15.32:0/3631427046 reap_dead start 2023-12-20T04:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f33fe96d640 1 -- 172.21.15.32:0/3631427046 shutdown_connections 2023-12-20T04:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f33fe96d640 1 -- 172.21.15.32:0/3631427046 >> 172.21.15.32:0/3631427046 conn(0x7f33f806a320 msgr2=0x7f33f80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f0435d9b640 1 --2- 172.21.15.32:0/1950522543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04380d2740 0x7f04380cf570 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0428009870 tx=0x7f042801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d5d200b44529c9a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f043559a640 1 -- 172.21.15.32:0/1950522543 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04280a89b0 con 0x7f04380d2740 2023-12-20T04:28:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f17577fe640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] conn(0x7f174406dcd0 0x7f17440700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f17567fc640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1744065ab0 0x7f1744067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f4c37fff640 1 -- 172.21.15.32:0/1218624004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4000eac0 con 0x7f4c5006c920 2023-12-20T04:28:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f4c37fff640 1 -- 172.21.15.32:0/1218624004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c40010040 con 0x7f4c5006c920 2023-12-20T04:28:44.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f4c37fff640 1 -- 172.21.15.32:0/1218624004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c4000ee40 con 0x7f4c5006c920 2023-12-20T04:28:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f043559a640 1 -- 172.21.15.32:0/1950522543 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04280a8b10 con 0x7f04380d2740 2023-12-20T04:28:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f043559a640 1 -- 172.21.15.32:0/1950522543 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04280a8ea0 con 0x7f04380d2740 2023-12-20T04:28:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7fc7adc0f640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc78c065b30 0x7fc78c067ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40ad000 tx=0x7fc7a40a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f17577fe640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] conn(0x7f174406dcd0 0x7f17440700f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f17567fc640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1744065ab0 0x7f1744067f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17581f49e0 tx=0x7f174c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f043cf60640 1 -- 172.21.15.32:0/1950522543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04380d2740 msgr2=0x7f04380cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f043cf60640 1 --2- 172.21.15.32:0/1950522543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04380d2740 0x7f04380cf570 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f0428009870 tx=0x7f042801ecf0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f4c56152640 1 -- 172.21.15.32:0/1218624004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c440094d0 con 0x7f4c5006c920 2023-12-20T04:28:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7fc79e7fc640 1 -- 172.21.15.32:0/4270339776 --> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7fc78c070820 con 0x7fc78c06dd50 2023-12-20T04:28:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7fc79e7fc640 1 -- 172.21.15.32:0/4270339776 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc798083900 con 0x7fc7a8156b00 2023-12-20T04:28:44.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.268+0000 7f173f7fe640 1 -- 172.21.15.32:0/2498496927 <== osd.4 v2:172.21.15.39:6800/38278 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f17440707a0 con 0x7f174406dcd0 2023-12-20T04:28:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.268+0000 7f4c56152640 1 -- 172.21.15.32:0/1218624004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c5014db30 con 0x7f4c5006c920 2023-12-20T04:28:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.268+0000 7f33fe96d640 1 -- 172.21.15.32:0/3631427046 shutdown_connections 2023-12-20T04:28:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f043cf60640 1 -- 172.21.15.32:0/1950522543 shutdown_connections 2023-12-20T04:28:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f043cf60640 1 --2- 172.21.15.32:0/1950522543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04380d2740 0x7f04380cf570 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f043cf60640 1 --2- 172.21.15.32:0/1950522543 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f043815cdb0 0x7f04380d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.264+0000 7f043cf60640 1 -- 172.21.15.32:0/1950522543 >> 172.21.15.32:0/1950522543 conn(0x7f0438065ee0 msgr2=0x7f043806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.268+0000 7f043cf60640 1 -- 172.21.15.32:0/1950522543 shutdown_connections 2023-12-20T04:28:44.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.268+0000 7f043cf60640 1 -- 172.21.15.32:0/1950522543 wait complete. 2023-12-20T04:28:44.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.268+0000 7f043cf60640 1 Processor -- start 2023-12-20T04:28:44.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.268+0000 7fc7ae410640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] conn(0x7fc78c06dd50 0x7fc78c070170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.268+0000 7f33fe96d640 1 -- 172.21.15.32:0/3631427046 wait complete. 2023-12-20T04:28:44.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.268+0000 7f33fe96d640 1 Processor -- start 2023-12-20T04:28:44.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.268+0000 7f4c56152640 1 -- 172.21.15.32:0/1218624004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c5015d530 con 0x7f4c5006c920 2023-12-20T04:28:44.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7fc7ae410640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] conn(0x7fc78c06dd50 0x7fc78c070170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f9d84bbc640 1 -- 172.21.15.32:0/3488683684 --> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f9d800c9a60 con 0x7f9d6006dd80 2023-12-20T04:28:44.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f9d6effd640 1 -- 172.21.15.32:0/3488683684 <== osd.1 v2:172.21.15.32:6800/38077 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f9d60070850 con 0x7f9d6006dd80 2023-12-20T04:28:44.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f043cf60640 1 -- start start 2023-12-20T04:28:44.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f4c37fff640 1 -- 172.21.15.32:0/1218624004 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4c40014410 con 0x7f4c5006c920 2023-12-20T04:28:44.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f4c37fff640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4c3c065b30 0x7f4c3c067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f4c37fff640 1 -- 172.21.15.32:0/1218624004 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4c40083390 con 0x7f4c5006c920 2023-12-20T04:28:44.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f4c37fff640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] conn(0x7f4c3c06dcd0 0x7f4c3c0700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f4c37fff640 1 -- 172.21.15.32:0/1218624004 --> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c3c0707a0 con 0x7f4c3c06dcd0 2023-12-20T04:28:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f043cf60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f04380d1120 0x7f04380c7900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f043cf60640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04380d2740 0x7f04380ca610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f043cf60640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043815cdb0 0x7f04381548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f043cf60640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04380cb020 con 0x7f043815cdb0 2023-12-20T04:28:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f043cf60640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f04380cb300 con 0x7f04380d2740 2023-12-20T04:28:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f043cf60640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04380cb5e0 con 0x7f04380d1120 2023-12-20T04:28:44.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f33fe96d640 1 -- start start 2023-12-20T04:28:44.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f9d84bbc640 1 -- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] conn(0x7f9d6006dd80 msgr2=0x7f9d600701a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f9d84bbc640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] conn(0x7f9d6006dd80 0x7f9d600701a0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f4c4effd640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4c3c065b30 0x7f4c3c067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f0436d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043815cdb0 0x7f04381548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f4c4ffff640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] conn(0x7f4c3c06dcd0 0x7f4c3c0700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f0435d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04380d2740 0x7f04380ca610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f043659c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f04380d1120 0x7f04380c7900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f0435d9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04380d2740 0x7f04380ca610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48872/0 (socket says 172.21.15.32:48872) 2023-12-20T04:28:44.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f0435d9b640 1 -- 172.21.15.32:0/564687914 learned_addr learned my addr 172.21.15.32:0/564687914 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f4c37fff640 1 -- 172.21.15.32:0/1218624004 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c40050d50 con 0x7f4c5006c920 2023-12-20T04:28:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.272+0000 7f4c4ffff640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] conn(0x7f4c3c06dcd0 0x7f4c3c0700f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f9d84bbc640 1 -- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d60065b60 msgr2=0x7f9d60068020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f9d84bbc640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d60065b60 0x7f9d60068020 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f9d8002c850 tx=0x7f9d74004220 comp rx=0 tx=0).stop 2023-12-20T04:28:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f9d84bbc640 1 -- 172.21.15.32:0/3488683684 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d800c0ff0 msgr2=0x7f9d800c1470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f9d84bbc640 1 --2- 172.21.15.32:0/3488683684 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d800c0ff0 0x7f9d800c1470 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f9d8006e160 tx=0x7f9d7800cb90 comp rx=0 tx=0).stop 2023-12-20T04:28:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f9d7ed9d640 1 -- 172.21.15.32:0/3488683684 reap_dead start 2023-12-20T04:28:44.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7fc79e7fc640 1 -- 172.21.15.32:0/4270339776 <== osd.2 v2:172.21.15.32:6818/38082 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7fc78c070820 con 0x7fc78c06dd50 2023-12-20T04:28:44.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f0435d9b640 1 -- 172.21.15.32:0/564687914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f04380d1120 msgr2=0x7f04380c7900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f0435d9b640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f04380d1120 0x7f04380c7900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f0435d9b640 1 -- 172.21.15.32:0/564687914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043815cdb0 msgr2=0x7f04381548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f043659c640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f04380d1120 0x7f04380c7900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:28:44.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f0435d9b640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043815cdb0 0x7f04381548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f0435d9b640 1 -- 172.21.15.32:0/564687914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04280094d0 con 0x7f04380d2740 2023-12-20T04:28:44.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f4c4effd640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4c3c065b30 0x7f4c3c067ff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4c4401a020 tx=0x7f4c44009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f0436d9d640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f043815cdb0 0x7f04381548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f9d84bbc640 1 -- 172.21.15.32:0/3488683684 shutdown_connections 2023-12-20T04:28:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f9d84bbc640 1 -- 172.21.15.32:0/3488683684 >> 172.21.15.32:0/3488683684 conn(0x7f9d80069d20 msgr2=0x7f9d800ca4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f4c37fff640 1 -- 172.21.15.32:0/1218624004 <== osd.5 v2:172.21.15.39:6824/38280 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4c3c0707a0 con 0x7f4c3c06dcd0 2023-12-20T04:28:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f0435d9b640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04380d2740 0x7f04380ca610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f042801c920 tx=0x7f042801ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f9d84bbc640 1 -- 172.21.15.32:0/3488683684 shutdown_connections 2023-12-20T04:28:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f9d84bbc640 1 -- 172.21.15.32:0/3488683684 wait complete. 2023-12-20T04:28:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f0426ffd640 1 -- 172.21.15.32:0/564687914 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04280be070 con 0x7f04380d2740 2023-12-20T04:28:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f0426ffd640 1 -- 172.21.15.32:0/564687914 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04280b1b90 con 0x7f04380d2740 2023-12-20T04:28:44.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f0426ffd640 1 -- 172.21.15.32:0/564687914 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04280b9440 con 0x7f04380d2740 2023-12-20T04:28:44.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33fe96d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e4004320 0x7f33f80c8460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33fe96d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f80b2fb0 0x7f33f814e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33fe96d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33f80b3430 0x7f33f814e810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33fe96d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8151830 con 0x7f33e4004320 2023-12-20T04:28:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33fe96d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33f814dad0 con 0x7f33f80b2fb0 2023-12-20T04:28:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33fe96d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33f8150340 con 0x7f33f80b3430 2023-12-20T04:28:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33fcf0a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33f80b3430 0x7f33f814e810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33fcf0a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33f80b3430 0x7f33f814e810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49450/0 (socket says 172.21.15.32:49450) 2023-12-20T04:28:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33ef7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f80b2fb0 0x7f33f814e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33fcf0a640 1 -- 172.21.15.32:0/2358294527 learned_addr learned my addr 172.21.15.32:0/2358294527 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33f7fff640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e4004320 0x7f33f80c8460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33ef7fe640 1 -- 172.21.15.32:0/2358294527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33f80b3430 msgr2=0x7f33f814e810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33ef7fe640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33f80b3430 0x7f33f814e810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33ef7fe640 1 -- 172.21.15.32:0/2358294527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e4004320 msgr2=0x7f33f80c8460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33ef7fe640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e4004320 0x7f33f80c8460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33ef7fe640 1 -- 172.21.15.32:0/2358294527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33f8150060 con 0x7f33f80b2fb0 2023-12-20T04:28:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.276+0000 7f33f7fff640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e4004320 0x7f33f80c8460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.280+0000 7f33ef7fe640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f80b2fb0 0x7f33f814e2d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f33e00ac000 tx=0x7f33e4005800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.280+0000 7f1e097fa640 1 -- 172.21.15.32:0/2819818307 --> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f1e24165430 con 0x7f1e1006dc30 2023-12-20T04:28:44.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.280+0000 7f33f57fa640 1 -- 172.21.15.32:0/2358294527 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0004220 con 0x7f33f80b2fb0 2023-12-20T04:28:44.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.280+0000 7f33f57fa640 1 -- 172.21.15.32:0/2358294527 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33e0004380 con 0x7f33f80b2fb0 2023-12-20T04:28:44.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.280+0000 7f33f57fa640 1 -- 172.21.15.32:0/2358294527 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33e0004220 con 0x7f33f80b2fb0 2023-12-20T04:28:44.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.280+0000 7f043cf60640 1 -- 172.21.15.32:0/564687914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f04380cb8c0 con 0x7f04380d2740 2023-12-20T04:28:44.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.280+0000 7f043cf60640 1 -- 172.21.15.32:0/564687914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f04380c7e40 con 0x7f04380d2740 2023-12-20T04:28:44.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.280+0000 7f043cf60640 1 -- 172.21.15.32:0/564687914 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f04380d1f40 con 0x7f04380d2740 2023-12-20T04:28:44.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.280+0000 7f33fe96d640 1 -- 172.21.15.32:0/2358294527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33f8151170 con 0x7f33f80b2fb0 2023-12-20T04:28:44.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.280+0000 7f33fe96d640 1 -- 172.21.15.32:0/2358294527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33f814fca0 con 0x7f33f80b2fb0 2023-12-20T04:28:44.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.280+0000 7f1e0b7fe640 1 -- 172.21.15.32:0/2819818307 <== osd.0 v2:172.21.15.32:6808/38080 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f1e14009c40 con 0x7f1e1006dc30 2023-12-20T04:28:44.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f33fe96d640 1 -- 172.21.15.32:0/2358294527 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f33f80ca440 con 0x7f33f80b2fb0 2023-12-20T04:28:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f1e097fa640 1 -- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] conn(0x7f1e1006dc30 msgr2=0x7f1e10070050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f1e097fa640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] conn(0x7f1e1006dc30 0x7f1e10070050 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f1e097fa640 1 -- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1e10065a90 msgr2=0x7f1e10067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f1e097fa640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1e10065a90 0x7f1e10067f50 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1e240aef00 tx=0x7f1e18008040 comp rx=0 tx=0).stop 2023-12-20T04:28:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f1e097fa640 1 -- 172.21.15.32:0/2819818307 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e2415a860 msgr2=0x7f1e2415ad10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f1e097fa640 1 --2- 172.21.15.32:0/2819818307 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e2415a860 0x7f1e2415ad10 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f1e1c01cd50 tx=0x7f1e1c019cc0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f1e237fe640 1 -- 172.21.15.32:0/2819818307 reap_dead start 2023-12-20T04:28:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f33f57fa640 1 -- 172.21.15.32:0/2358294527 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f33e00b4040 con 0x7f33f80b2fb0 2023-12-20T04:28:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f33f57fa640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f33d8065ac0 0x7f33d8067f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f33f57fa640 1 -- 172.21.15.32:0/2358294527 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f33e011d4b0 con 0x7f33f80b2fb0 2023-12-20T04:28:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f33f57fa640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] conn(0x7f33d806dce0 0x7f33d8070100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f33f7fff640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f33d8065ac0 0x7f33d8067f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f33f57fa640 1 -- 172.21.15.32:0/2358294527 --> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f33d80707b0 con 0x7f33d806dce0 2023-12-20T04:28:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f33f57fa640 1 -- 172.21.15.32:0/2358294527 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f33e011fa10 con 0x7f33f80b2fb0 2023-12-20T04:28:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f0426ffd640 1 -- 172.21.15.32:0/564687914 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f04280b7070 con 0x7f04380d2740 2023-12-20T04:28:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f0426ffd640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0418065ab0 0x7f0418067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.284+0000 7f0426ffd640 1 -- 172.21.15.32:0/564687914 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0428121310 con 0x7f04380d2740 2023-12-20T04:28:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.288+0000 7f1e097fa640 1 -- 172.21.15.32:0/2819818307 shutdown_connections 2023-12-20T04:28:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.288+0000 7f1e097fa640 1 -- 172.21.15.32:0/2819818307 >> 172.21.15.32:0/2819818307 conn(0x7f1e240cb1e0 msgr2=0x7f1e241642c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.288+0000 7f0426ffd640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] conn(0x7f041806dcd0 0x7f04180700f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.288+0000 7f0426ffd640 1 -- 172.21.15.32:0/564687914 --> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f04180707a0 con 0x7f041806dcd0 2023-12-20T04:28:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.288+0000 7f0426ffd640 1 -- 172.21.15.32:0/564687914 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_get_version_reply(handle=1 version=21) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0428121680 con 0x7f04380d2740 2023-12-20T04:28:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.288+0000 7f1e097fa640 1 -- 172.21.15.32:0/2819818307 shutdown_connections 2023-12-20T04:28:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.288+0000 7f1e097fa640 1 -- 172.21.15.32:0/2819818307 wait complete. 2023-12-20T04:28:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.288+0000 7f33f7fff640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f33d8065ac0 0x7f33d8067f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009fd0 tx=0x7f33e800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.292+0000 7f33fcf0a640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] conn(0x7f33d806dce0 0x7f33d8070100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.292+0000 7f043659c640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0418065ab0 0x7f0418067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.292+0000 7f0436d9d640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] conn(0x7f041806dcd0 0x7f04180700f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.292+0000 7f0436d9d640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] conn(0x7f041806dcd0 0x7f04180700f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.292+0000 7f043659c640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0418065ab0 0x7f0418067f70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f042c006fd0 tx=0x7f042c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.292+0000 7f33fcf0a640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] conn(0x7f33d806dce0 0x7f33d8070100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.292+0000 7f0426ffd640 1 -- 172.21.15.32:0/564687914 <== osd.7 v2:172.21.15.39:6808/38279 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f04180707a0 con 0x7f041806dcd0 2023-12-20T04:28:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.292+0000 7f33f57fa640 1 -- 172.21.15.32:0/2358294527 <== osd.6 v2:172.21.15.39:6816/38281 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f33d80707b0 con 0x7f33d806dce0 2023-12-20T04:28:44.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f76f9ffb640 1 -- 172.21.15.32:0/306052657 --> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f771415d190 con 0x7f770006dc60 2023-12-20T04:28:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.292+0000 7f175d325640 1 -- 172.21.15.32:0/2498496927 --> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f17580cbd00 con 0x7f174406dcd0 2023-12-20T04:28:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.292+0000 7f173f7fe640 1 -- 172.21.15.32:0/2498496927 <== osd.4 v2:172.21.15.39:6800/38278 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f17480a58a0 con 0x7f174406dcd0 2023-12-20T04:28:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f175d325640 1 -- 172.21.15.32:0/2498496927 >> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] conn(0x7f174406dcd0 msgr2=0x7f17440700f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f175d325640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] conn(0x7f174406dcd0 0x7f17440700f0 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f175d325640 1 -- 172.21.15.32:0/2498496927 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1744065ab0 msgr2=0x7f1744067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f175d325640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1744065ab0 0x7f1744067f70 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f17581f49e0 tx=0x7f174c008040 comp rx=0 tx=0).stop 2023-12-20T04:28:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f175d325640 1 -- 172.21.15.32:0/2498496927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580d1120 msgr2=0x7f175814cf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f175d325640 1 --2- 172.21.15.32:0/2498496927 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17580d1120 0x7f175814cf00 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f175000d5d0 tx=0x7f175000da90 comp rx=0 tx=0).stop 2023-12-20T04:28:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f17577fe640 1 -- 172.21.15.32:0/2498496927 reap_dead start 2023-12-20T04:28:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f175d325640 1 -- 172.21.15.32:0/2498496927 shutdown_connections 2023-12-20T04:28:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f175d325640 1 -- 172.21.15.32:0/2498496927 >> 172.21.15.32:0/2498496927 conn(0x7f1758065ee0 msgr2=0x7f17580b5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f76fbfff640 1 -- 172.21.15.32:0/306052657 <== osd.3 v2:172.21.15.32:6816/38083 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f77040019c0 con 0x7f770006dc60 2023-12-20T04:28:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f175d325640 1 -- 172.21.15.32:0/2498496927 shutdown_connections 2023-12-20T04:28:44.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f175d325640 1 -- 172.21.15.32:0/2498496927 wait complete. 2023-12-20T04:28:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f76f9ffb640 1 -- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] conn(0x7f770006dc60 msgr2=0x7f7700070080 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f76f9ffb640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] conn(0x7f770006dc60 0x7f7700070080 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f76f9ffb640 1 -- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7700065ac0 msgr2=0x7f7700067f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f76f9ffb640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7700065ac0 0x7f7700067f80 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f770c006fd0 tx=0x7f770c008040 comp rx=0 tx=0).stop 2023-12-20T04:28:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f76f9ffb640 1 -- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f771406c3e0 msgr2=0x7f771414c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f76f9ffb640 1 --2- 172.21.15.32:0/306052657 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f771406c3e0 0x7f771414c550 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f770801aba0 tx=0x7f7708019290 comp rx=0 tx=0).stop 2023-12-20T04:28:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.296+0000 7f7713fff640 1 -- 172.21.15.32:0/306052657 reap_dead start 2023-12-20T04:28:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.300+0000 7f76f9ffb640 1 -- 172.21.15.32:0/306052657 shutdown_connections 2023-12-20T04:28:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.300+0000 7f76f9ffb640 1 -- 172.21.15.32:0/306052657 >> 172.21.15.32:0/306052657 conn(0x7f771406a320 msgr2=0x7f7714151440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.300+0000 7f4c56152640 1 -- 172.21.15.32:0/1218624004 --> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f4c5015d250 con 0x7f4c3c06dcd0 2023-12-20T04:28:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.300+0000 7f76f9ffb640 1 -- 172.21.15.32:0/306052657 shutdown_connections 2023-12-20T04:28:44.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.300+0000 7f76f9ffb640 1 -- 172.21.15.32:0/306052657 wait complete. 2023-12-20T04:28:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.300+0000 7f4c37fff640 1 -- 172.21.15.32:0/1218624004 <== osd.5 v2:172.21.15.39:6824/38280 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f4c48001f80 con 0x7f4c3c06dcd0 2023-12-20T04:28:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7f4c56152640 1 -- 172.21.15.32:0/1218624004 >> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] conn(0x7f4c3c06dcd0 msgr2=0x7f4c3c0700f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7f4c56152640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] conn(0x7f4c3c06dcd0 0x7f4c3c0700f0 crc :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7f4c56152640 1 -- 172.21.15.32:0/1218624004 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4c3c065b30 msgr2=0x7f4c3c067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7f4c56152640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4c3c065b30 0x7f4c3c067ff0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4c4401a020 tx=0x7f4c44009040 comp rx=0 tx=0).stop 2023-12-20T04:28:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7f4c56152640 1 -- 172.21.15.32:0/1218624004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5006c920 msgr2=0x7f4c50201b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7f4c56152640 1 --2- 172.21.15.32:0/1218624004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c5006c920 0x7f4c50201b10 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f4c40009860 tx=0x7f4c40009d20 comp rx=0 tx=0).stop 2023-12-20T04:28:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7f4c4ffff640 1 -- 172.21.15.32:0/1218624004 reap_dead start 2023-12-20T04:28:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7f4c56152640 1 -- 172.21.15.32:0/1218624004 shutdown_connections 2023-12-20T04:28:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7f4c56152640 1 -- 172.21.15.32:0/1218624004 >> 172.21.15.32:0/1218624004 conn(0x7f4c500cb1e0 msgr2=0x7f4c500cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7f4c56152640 1 -- 172.21.15.32:0/1218624004 shutdown_connections 2023-12-20T04:28:44.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7fc7afe73640 1 -- 172.21.15.32:0/4270339776 --> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7fc7a815d530 con 0x7fc78c06dd50 2023-12-20T04:28:44.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.304+0000 7f4c56152640 1 -- 172.21.15.32:0/1218624004 wait complete. 2023-12-20T04:28:44.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc79e7fc640 1 -- 172.21.15.32:0/4270339776 <== osd.2 v2:172.21.15.32:6818/38082 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7fc7a0009bf0 con 0x7fc78c06dd50 2023-12-20T04:28:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc7afe73640 1 -- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] conn(0x7fc78c06dd50 msgr2=0x7fc78c070170 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc7afe73640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] conn(0x7fc78c06dd50 0x7fc78c070170 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc7afe73640 1 -- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc78c065b30 msgr2=0x7fc78c067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc7afe73640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc78c065b30 0x7fc78c067ff0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fc7a40ad000 tx=0x7fc7a40a6000 comp rx=0 tx=0).stop 2023-12-20T04:28:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc7afe73640 1 -- 172.21.15.32:0/4270339776 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7a8156b00 msgr2=0x7fc7a80b1530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc7afe73640 1 --2- 172.21.15.32:0/4270339776 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7a8156b00 0x7fc7a80b1530 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fc79800c7f0 tx=0x7fc79800ccb0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc7ae410640 1 -- 172.21.15.32:0/4270339776 reap_dead start 2023-12-20T04:28:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc7afe73640 1 -- 172.21.15.32:0/4270339776 shutdown_connections 2023-12-20T04:28:44.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc7afe73640 1 -- 172.21.15.32:0/4270339776 >> 172.21.15.32:0/4270339776 conn(0x7fc7a80cb1e0 msgr2=0x7fc7a806e2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc7afe73640 1 -- 172.21.15.32:0/4270339776 shutdown_connections 2023-12-20T04:28:44.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.308+0000 7fc7afe73640 1 -- 172.21.15.32:0/4270339776 wait complete. 2023-12-20T04:28:44.345 INFO:teuthology.orchestra.run.smithi032.stdout:55834574855 2023-12-20T04:28:44.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-20T04:28:44.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.316+0000 7f33fe96d640 1 -- 172.21.15.32:0/2358294527 --> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f33f80ca160 con 0x7f33d806dce0 2023-12-20T04:28:44.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.316+0000 7f33f57fa640 1 -- 172.21.15.32:0/2358294527 <== osd.6 v2:172.21.15.39:6816/38281 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f33f0007df0 con 0x7f33d806dce0 2023-12-20T04:28:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.320+0000 7f33fe96d640 1 -- 172.21.15.32:0/2358294527 >> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] conn(0x7f33d806dce0 msgr2=0x7f33d8070100 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.320+0000 7f33fe96d640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] conn(0x7f33d806dce0 0x7f33d8070100 crc :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.348 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-12-20T04:28:44.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-20T04:28:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.324+0000 7f33fe96d640 1 -- 172.21.15.32:0/2358294527 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f33d8065ac0 msgr2=0x7f33d8067f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.324+0000 7f33fe96d640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f33d8065ac0 0x7f33d8067f80 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f33e8009fd0 tx=0x7f33e800b040 comp rx=0 tx=0).stop 2023-12-20T04:28:44.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.324+0000 7f33fe96d640 1 -- 172.21.15.32:0/2358294527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f80b2fb0 msgr2=0x7f33f814e2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.324+0000 7f33fe96d640 1 --2- 172.21.15.32:0/2358294527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33f80b2fb0 0x7f33f814e2d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f33e00ac000 tx=0x7f33e4005800 comp rx=0 tx=0).stop 2023-12-20T04:28:44.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.324+0000 7f33fcf0a640 1 -- 172.21.15.32:0/2358294527 reap_dead start 2023-12-20T04:28:44.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.324+0000 7f33fe96d640 1 -- 172.21.15.32:0/2358294527 shutdown_connections 2023-12-20T04:28:44.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.324+0000 7f33fe96d640 1 -- 172.21.15.32:0/2358294527 >> 172.21.15.32:0/2358294527 conn(0x7f33f806a320 msgr2=0x7f33f80b1980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.324+0000 7f043cf60640 1 -- 172.21.15.32:0/564687914 --> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] -- command(tid 2: {"prefix": "flush_pg_stats"}) v1 -- 0x7f04380d1c60 con 0x7f041806dcd0 2023-12-20T04:28:44.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.324+0000 7f33fe96d640 1 -- 172.21.15.32:0/2358294527 shutdown_connections 2023-12-20T04:28:44.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.324+0000 7f33fe96d640 1 -- 172.21.15.32:0/2358294527 wait complete. 2023-12-20T04:28:44.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.324+0000 7f0426ffd640 1 -- 172.21.15.32:0/564687914 <== osd.7 v2:172.21.15.39:6808/38279 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+11 (crc 0 0 0) 0x7f04180707a0 con 0x7f041806dcd0 2023-12-20T04:28:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.332+0000 7f043cf60640 1 -- 172.21.15.32:0/564687914 >> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] conn(0x7f041806dcd0 msgr2=0x7f04180700f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.332+0000 7f043cf60640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] conn(0x7f041806dcd0 0x7f04180700f0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.332+0000 7f043cf60640 1 -- 172.21.15.32:0/564687914 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0418065ab0 msgr2=0x7f0418067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.332+0000 7f043cf60640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0418065ab0 0x7f0418067f70 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f042c006fd0 tx=0x7f042c008040 comp rx=0 tx=0).stop 2023-12-20T04:28:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.332+0000 7f043cf60640 1 -- 172.21.15.32:0/564687914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04380d2740 msgr2=0x7f04380ca610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.332+0000 7f043cf60640 1 --2- 172.21.15.32:0/564687914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f04380d2740 0x7f04380ca610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f042801c920 tx=0x7f042801ba90 comp rx=0 tx=0).stop 2023-12-20T04:28:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.332+0000 7f0436d9d640 1 -- 172.21.15.32:0/564687914 reap_dead start 2023-12-20T04:28:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.332+0000 7f043cf60640 1 -- 172.21.15.32:0/564687914 shutdown_connections 2023-12-20T04:28:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.332+0000 7f043cf60640 1 -- 172.21.15.32:0/564687914 >> 172.21.15.32:0/564687914 conn(0x7f0438065ee0 msgr2=0x7f043815b5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.332+0000 7f043cf60640 1 -- 172.21.15.32:0/564687914 shutdown_connections 2023-12-20T04:28:44.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.332+0000 7f043cf60640 1 -- 172.21.15.32:0/564687914 wait complete. 2023-12-20T04:28:44.359 INFO:teuthology.orchestra.run.smithi032.stdout:55834574855 2023-12-20T04:28:44.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-20T04:28:44.360 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-12-20T04:28:44.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-20T04:28:44.361 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-12-20T04:28:44.361 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-20T04:28:44.362 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-12-20T04:28:44.362 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-20T04:28:44.363 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-12-20T04:28:44.364 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-20T04:28:44.364 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-12-20T04:28:44.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-20T04:28:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.644+0000 7ff48f041640 1 Processor -- start 2023-12-20T04:28:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.644+0000 7ff48f041640 1 -- start start 2023-12-20T04:28:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.644+0000 7ff48f041640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff4880c1880 0x7ff4880af520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.644+0000 7ff48f041640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff48806caf0 0x7ff48816b3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.644+0000 7ff48f041640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4880b0ba0 con 0x7ff488165170 2023-12-20T04:28:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.644+0000 7ff48f041640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4880721d0 con 0x7ff48806caf0 2023-12-20T04:28:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.644+0000 7ff48f041640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4880aff90 con 0x7ff4880c1880 2023-12-20T04:28:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7fa7bb463640 1 Processor -- start 2023-12-20T04:28:44.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7fa7bb463640 1 -- start start 2023-12-20T04:28:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7fa7bb463640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b406cfd0 0x7fa7b406c840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7fa7bb463640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa7b4070cf0 0x7fa7b4160890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7ff48d5de640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff48806caf0 0x7ff48816b3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7ff48d5de640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff48806caf0 0x7ff48816b3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48374/0 (socket says 172.21.15.32:48374) 2023-12-20T04:28:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7ff48d5de640 1 -- 172.21.15.32:0/3545168003 learned_addr learned my addr 172.21.15.32:0/3545168003 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7fa7b89fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b406cfd0 0x7fa7b406c840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7fa7b89fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b406cfd0 0x7fa7b406c840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49458/0 (socket says 172.21.15.32:49458) 2023-12-20T04:28:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7fa7b89fe640 1 -- 172.21.15.32:0/4140135996 learned_addr learned my addr 172.21.15.32:0/4140135996 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7fa7bb463640 1 -- 172.21.15.32:0/4140135996 --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7b40b4680 con 0x7fa7b415cdf0 2023-12-20T04:28:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7fa7bb463640 1 -- 172.21.15.32:0/4140135996 --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7b415d1d0 con 0x7fa7b4070cf0 2023-12-20T04:28:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7fa7bb463640 1 -- 172.21.15.32:0/4140135996 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b40b43a0 con 0x7fa7b406cfd0 2023-12-20T04:28:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7ff48d5de640 1 -- 172.21.15.32:0/3545168003 >> v2:172.21.15.39:3300/0 conn(0x7ff4880c1880 msgr2=0x7ff4880af520 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-20T04:28:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7ff48d5de640 1 --2- 172.21.15.32:0/3545168003 >> v2:172.21.15.39:3300/0 conn(0x7ff4880c1880 0x7ff4880af520 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7ff48d5de640 1 -- 172.21.15.32:0/3545168003 >> v1:172.21.15.39:6789/0 conn(0x7ff488165170 legacy=0x7ff48806c3e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2023-12-20T04:28:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.648+0000 7ff48d5de640 1 -- 172.21.15.32:0/3545168003 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff4880b0270 con 0x7ff48806caf0 2023-12-20T04:28:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.652+0000 7fa7b89fe640 1 -- 172.21.15.32:0/4140135996 >> v2:172.21.15.32:3300/0 conn(0x7fa7b4070cf0 msgr2=0x7fa7b4160890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.652+0000 7fa7b89fe640 1 --2- 172.21.15.32:0/4140135996 >> v2:172.21.15.32:3300/0 conn(0x7fa7b4070cf0 0x7fa7b4160890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.652+0000 7fa7b89fe640 1 -- 172.21.15.32:0/4140135996 >> v1:172.21.15.32:6789/0 conn(0x7fa7b415cdf0 legacy=0x7fa7b40cae70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.652+0000 7fa7b89fe640 1 -- 172.21.15.32:0/4140135996 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7b4070210 con 0x7fa7b406cfd0 2023-12-20T04:28:44.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.652+0000 7ff48d5de640 1 --2- 172.21.15.32:0/3545168003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff48806caf0 0x7ff48816b3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff48806ad00 tx=0x7ff48001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a740ff049bfefe server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.656+0000 7ff4877fe640 1 -- 172.21.15.32:0/3545168003 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4800a5840 con 0x7ff48806caf0 2023-12-20T04:28:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.656+0000 7fb13c32c640 1 Processor -- start 2023-12-20T04:28:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.656+0000 7fa7b89fe640 1 --2- 172.21.15.32:0/4140135996 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b406cfd0 0x7fa7b406c840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009870 tx=0x7fa7a401c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e84c4c0830f9d2ba server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.656+0000 7fb13c32c640 1 -- start start 2023-12-20T04:28:44.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.656+0000 7fb13c32c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb1341575c0 0x7fb1340713e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.656+0000 7fb13c32c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb134071920 0x7fb13415d610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.656+0000 7fb13c32c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb13415db50 0x7fb1340701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.656+0000 7fb13c32c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb134071dd0 con 0x7fb134071920 2023-12-20T04:28:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.656+0000 7fb13c32c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1341579a0 con 0x7fb1341575c0 2023-12-20T04:28:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.656+0000 7fb13c32c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb13415e380 con 0x7fb13415db50 2023-12-20T04:28:44.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fb13a0c8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb1341575c0 0x7fb1340713e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fb13a0c8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb1341575c0 0x7fb1340713e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48904/0 (socket says 172.21.15.32:48904) 2023-12-20T04:28:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fb13a0c8640 1 -- 172.21.15.32:0/1233830398 learned_addr learned my addr 172.21.15.32:0/1233830398 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fb13a8c9640 1 --2- 172.21.15.32:0/1233830398 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb13415db50 0x7fb1340701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fa7abfff640 1 -- 172.21.15.32:0/4140135996 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40ab600 con 0x7fa7b406cfd0 2023-12-20T04:28:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fa7abfff640 1 -- 172.21.15.32:0/4140135996 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7a40a7050 con 0x7fa7b406cfd0 2023-12-20T04:28:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fa7abfff640 1 -- 172.21.15.32:0/4140135996 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7a40ab980 con 0x7fa7b406cfd0 2023-12-20T04:28:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fa7bb463640 1 -- 172.21.15.32:0/4140135996 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b406cfd0 msgr2=0x7fa7b406c840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fa7bb463640 1 --2- 172.21.15.32:0/4140135996 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b406cfd0 0x7fa7b406c840 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fa7a4009870 tx=0x7fa7a401c470 comp rx=0 tx=0).stop 2023-12-20T04:28:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fb13a0c8640 1 -- 172.21.15.32:0/1233830398 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb13415db50 msgr2=0x7fb1340701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fb13a0c8640 1 --2- 172.21.15.32:0/1233830398 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb13415db50 0x7fb1340701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fb13a0c8640 1 -- 172.21.15.32:0/1233830398 >> v2:172.21.15.32:3300/0 conn(0x7fb134071920 msgr2=0x7fb13415d610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fb13a0c8640 1 --2- 172.21.15.32:0/1233830398 >> v2:172.21.15.32:3300/0 conn(0x7fb134071920 0x7fb13415d610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fb13a0c8640 1 -- 172.21.15.32:0/1233830398 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb134160ae0 con 0x7fb1341575c0 2023-12-20T04:28:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fa7bb463640 1 -- 172.21.15.32:0/4140135996 shutdown_connections 2023-12-20T04:28:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fa7bb463640 1 --2- 172.21.15.32:0/4140135996 >> v2:172.21.15.32:3300/0 conn(0x7fa7b4070cf0 0x7fa7b4160890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fa7bb463640 1 --2- 172.21.15.32:0/4140135996 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b406cfd0 0x7fa7b406c840 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fa7bb463640 1 -- 172.21.15.32:0/4140135996 >> 172.21.15.32:0/4140135996 conn(0x7fa7b4069d20 msgr2=0x7fa7b406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7fa7bb463640 1 -- 172.21.15.32:0/4140135996 shutdown_connections 2023-12-20T04:28:44.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7fa7bb463640 1 -- 172.21.15.32:0/4140135996 wait complete. 2023-12-20T04:28:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7ff4877fe640 1 -- 172.21.15.32:0/3545168003 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4800a59a0 con 0x7ff48806caf0 2023-12-20T04:28:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.660+0000 7ff4877fe640 1 -- 172.21.15.32:0/3545168003 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4800a5840 con 0x7ff48806caf0 2023-12-20T04:28:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7ff48f041640 1 -- 172.21.15.32:0/3545168003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff48806caf0 msgr2=0x7ff48816b3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7ff48f041640 1 --2- 172.21.15.32:0/3545168003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff48806caf0 0x7ff48816b3c0 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7ff48806ad00 tx=0x7ff48001c980 comp rx=0 tx=0).stop 2023-12-20T04:28:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7ff48f041640 1 -- 172.21.15.32:0/3545168003 shutdown_connections 2023-12-20T04:28:44.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7ff48f041640 1 --2- 172.21.15.32:0/3545168003 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff48806caf0 0x7ff48816b3c0 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7ff48f041640 1 --2- 172.21.15.32:0/3545168003 >> v2:172.21.15.39:3300/0 conn(0x7ff4880c1880 0x7ff4880af520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7ff48f041640 1 -- 172.21.15.32:0/3545168003 >> 172.21.15.32:0/3545168003 conn(0x7ff48806a320 msgr2=0x7ff4880ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7fb13a0c8640 1 --2- 172.21.15.32:0/1233830398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1341575c0 0x7fb1340713e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb130009870 tx=0x7fb13001c210 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e94d66020b964406 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7fb1390c6640 1 -- 172.21.15.32:0/1233830398 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1300ab600 con 0x7fb1341575c0 2023-12-20T04:28:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7ff48f041640 1 -- 172.21.15.32:0/3545168003 shutdown_connections 2023-12-20T04:28:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7ff48f041640 1 -- 172.21.15.32:0/3545168003 wait complete. 2023-12-20T04:28:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7fb1390c6640 1 -- 172.21.15.32:0/1233830398 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1300a7050 con 0x7fb1341575c0 2023-12-20T04:28:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7fb1390c6640 1 -- 172.21.15.32:0/1233830398 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1300ab980 con 0x7fb1341575c0 2023-12-20T04:28:44.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7fa7bb463640 1 Processor -- start 2023-12-20T04:28:44.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7fbaa42c4640 1 Processor -- start 2023-12-20T04:28:44.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7fa7bb463640 1 -- start start 2023-12-20T04:28:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7fb13c32c640 1 -- 172.21.15.32:0/1233830398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1341575c0 msgr2=0x7fb1340713e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7fb13c32c640 1 --2- 172.21.15.32:0/1233830398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1341575c0 0x7fb1340713e0 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fb130009870 tx=0x7fb13001c210 comp rx=0 tx=0).stop 2023-12-20T04:28:44.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.664+0000 7fbaa42c4640 1 -- start start 2023-12-20T04:28:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff48f041640 1 Processor -- start 2023-12-20T04:28:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff48f041640 1 -- start start 2023-12-20T04:28:44.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7f435b717640 1 Processor -- start 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7bb463640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b406cfd0 0x7fa7b41596c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7bb463640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b4070cf0 0x7fa7b4158eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7bb463640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b415cdf0 0x7fa7b40c67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7bb463640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b4159f60 con 0x7fa7b406cfd0 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7bb463640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b41f8d50 con 0x7fa7b415cdf0 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7bb463640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7b41f9030 con 0x7fa7b4070cf0 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7b9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b415cdf0 0x7fa7b40c67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7b9a00640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b415cdf0 0x7fa7b40c67a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48910/0 (socket says 172.21.15.32:48910) 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7b9a00640 1 -- 172.21.15.32:0/1235498638 learned_addr learned my addr 172.21.15.32:0/1235498638 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7b91ff640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b406cfd0 0x7fa7b41596c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7b89fe640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b4070cf0 0x7fa7b4158eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7b9a00640 1 -- 172.21.15.32:0/1235498638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b4070cf0 msgr2=0x7fa7b4158eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7b9a00640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b4070cf0 0x7fa7b4158eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7b9a00640 1 -- 172.21.15.32:0/1235498638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b406cfd0 msgr2=0x7fa7b41596c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7b9a00640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b406cfd0 0x7fa7b41596c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7b9a00640 1 -- 172.21.15.32:0/1235498638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7a40094d0 con 0x7fa7b415cdf0 2023-12-20T04:28:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7f435b717640 1 -- start start 2023-12-20T04:28:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa42c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c1599e0 0x7fba9c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa42c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba9c0b0d00 0x7fba9c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa42c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9c0c98c0 con 0x7fba9c0c90e0 2023-12-20T04:28:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa42c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9c0c9ba0 con 0x7fba9c0b0d00 2023-12-20T04:28:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa42c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c0c9e80 con 0x7fba9c1599e0 2023-12-20T04:28:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa2060640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c1599e0 0x7fba9c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa2060640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c1599e0 0x7fba9c0aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49478/0 (socket says 172.21.15.32:49478) 2023-12-20T04:28:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa2060640 1 -- 172.21.15.32:0/1246139819 learned_addr learned my addr 172.21.15.32:0/1246139819 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7f8e5cf39640 1 Processor -- start 2023-12-20T04:28:44.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fa7b9a00640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b415cdf0 0x7fa7b40c67a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00b970 tx=0x7fa7ac00be30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff48f041640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff48806caf0 0x7ff4880c4a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff48f041640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4880c1880 0x7ff4880c29c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff48f041640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff488165170 0x7ff488148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff48f041640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff48815f330 con 0x7ff488165170 2023-12-20T04:28:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff48f041640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff488071090 con 0x7ff4880c1880 2023-12-20T04:28:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff48f041640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff488071370 con 0x7ff48806caf0 2023-12-20T04:28:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff487fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4880c1880 0x7ff4880c29c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff48cddd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff48806caf0 0x7ff4880c4a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff487fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4880c1880 0x7ff4880c29c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48926/0 (socket says 172.21.15.32:48926) 2023-12-20T04:28:44.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7ff487fff640 1 -- 172.21.15.32:0/1842946739 learned_addr learned my addr 172.21.15.32:0/1842946739 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7fb13c32c640 1 -- 172.21.15.32:0/1233830398 shutdown_connections 2023-12-20T04:28:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7fb13c32c640 1 --2- 172.21.15.32:0/1233830398 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb13415db50 0x7fb1340701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7fb13c32c640 1 --2- 172.21.15.32:0/1233830398 >> v2:172.21.15.32:3300/0 conn(0x7fb134071920 0x7fb13415d610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7fb13c32c640 1 --2- 172.21.15.32:0/1233830398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1341575c0 0x7fb1340713e0 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7fb13c32c640 1 -- 172.21.15.32:0/1233830398 >> 172.21.15.32:0/1233830398 conn(0x7fb1340cb1e0 msgr2=0x7fb1340cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7f435b717640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43540ae010 0x7f4354071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7f435b717640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4354164ef0 0x7f43540719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7f435b717640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43540ae3f0 con 0x7f435406cf10 2023-12-20T04:28:44.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7f435b717640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43540ca4b0 con 0x7f43540ae010 2023-12-20T04:28:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f435b717640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4354161800 con 0x7f4354164ef0 2023-12-20T04:28:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa2060640 1 -- 172.21.15.32:0/1246139819 >> v2:172.21.15.32:3300/0 conn(0x7fba9c0b0d00 msgr2=0x7fba9c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa2060640 1 --2- 172.21.15.32:0/1246139819 >> v2:172.21.15.32:3300/0 conn(0x7fba9c0b0d00 0x7fba9c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa2060640 1 -- 172.21.15.32:0/1246139819 >> v1:172.21.15.39:6789/0 conn(0x7fba9c0c90e0 legacy=0x7fba9c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7fbaa2060640 1 -- 172.21.15.32:0/1246139819 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c0ca160 con 0x7fba9c1599e0 2023-12-20T04:28:44.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f4359cb4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4354164ef0 0x7f43540719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f4359cb4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4354164ef0 0x7f43540719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49496/0 (socket says 172.21.15.32:49496) 2023-12-20T04:28:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f4359cb4640 1 -- 172.21.15.32:0/1129963395 learned_addr learned my addr 172.21.15.32:0/1129963395 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.668+0000 7f8e5cf39640 1 -- start start 2023-12-20T04:28:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f8e5cf39640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8e5806cf10 0x7f8e580c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f8e5cf39640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e58164ef0 0x7f8e580c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7ff487fff640 1 -- 172.21.15.32:0/1842946739 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff48806caf0 msgr2=0x7ff4880c4a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7ff487fff640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff48806caf0 0x7ff4880c4a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7ff487fff640 1 -- 172.21.15.32:0/1842946739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff488165170 msgr2=0x7ff488148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7ff487fff640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff488165170 0x7ff488148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7ff487fff640 1 -- 172.21.15.32:0/1842946739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff480009b30 con 0x7ff4880c1880 2023-12-20T04:28:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7fa7a9ffb640 1 -- 172.21.15.32:0/1235498638 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac00cb40 con 0x7fa7b415cdf0 2023-12-20T04:28:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7fa7a9ffb640 1 -- 172.21.15.32:0/1235498638 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7ac00ccc0 con 0x7fa7b415cdf0 2023-12-20T04:28:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7fa7a9ffb640 1 -- 172.21.15.32:0/1235498638 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7ac014680 con 0x7fa7b415cdf0 2023-12-20T04:28:44.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fa7bb463640 1 -- 172.21.15.32:0/1235498638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7b41f9310 con 0x7fa7b415cdf0 2023-12-20T04:28:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fa7bb463640 1 -- 172.21.15.32:0/1235498638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7b415d880 con 0x7fa7b415cdf0 2023-12-20T04:28:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7ff487fff640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4880c1880 0x7ff4880c29c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff47800b3e0 tx=0x7ff47800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7ff4857fa640 1 -- 172.21.15.32:0/1842946739 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478011830 con 0x7ff4880c1880 2023-12-20T04:28:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7ff48f041640 1 -- 172.21.15.32:0/1842946739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff488071650 con 0x7ff4880c1880 2023-12-20T04:28:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7ff48f041640 1 -- 172.21.15.32:0/1842946739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff488157d30 con 0x7ff4880c1880 2023-12-20T04:28:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7ff4857fa640 1 -- 172.21.15.32:0/1842946739 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff478011990 con 0x7ff4880c1880 2023-12-20T04:28:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7fb13c32c640 1 -- 172.21.15.32:0/1233830398 shutdown_connections 2023-12-20T04:28:44.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7fb13c32c640 1 -- 172.21.15.32:0/1233830398 wait complete. 2023-12-20T04:28:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13c32c640 1 Processor -- start 2023-12-20T04:28:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13c32c640 1 -- start start 2023-12-20T04:28:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f43594b3640 1 --2- 172.21.15.32:0/1129963395 >> v2:172.21.15.32:3300/0 conn(0x7f43540ae010 0x7f4354071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f43594b3640 1 -- 172.21.15.32:0/1129963395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4354164ef0 msgr2=0x7f43540719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f43594b3640 1 --2- 172.21.15.32:0/1129963395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4354164ef0 0x7f43540719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f43594b3640 1 -- 172.21.15.32:0/1129963395 >> v1:172.21.15.39:6789/0 conn(0x7f435406cf10 legacy=0x7f43540701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f8e5cf39640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e580ae3f0 con 0x7f8e580ae010 2023-12-20T04:28:44.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f8e5cf39640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e58161800 con 0x7f8e58164ef0 2023-12-20T04:28:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f8e5cf39640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e58165380 con 0x7f8e5806cf10 2023-12-20T04:28:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f8e56d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e58164ef0 0x7f8e580c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f8e56d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e58164ef0 0x7f8e580c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48424/0 (socket says 172.21.15.32:48424) 2023-12-20T04:28:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f8e56d9d640 1 -- 172.21.15.32:0/1364867039 learned_addr learned my addr 172.21.15.32:0/1364867039 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f8e55d9b640 1 --2- 172.21.15.32:0/1364867039 >> v2:172.21.15.39:3300/0 conn(0x7f8e5806cf10 0x7f8e580c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f8e56d9d640 1 -- 172.21.15.32:0/1364867039 >> v2:172.21.15.39:3300/0 conn(0x7f8e5806cf10 msgr2=0x7f8e580c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7f8e56d9d640 1 --2- 172.21.15.32:0/1364867039 >> v2:172.21.15.39:3300/0 conn(0x7f8e5806cf10 0x7f8e580c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fa7bb463640 1 -- 172.21.15.32:0/1235498638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7b415da70 con 0x7fa7b415cdf0 2023-12-20T04:28:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7ff4857fa640 1 -- 172.21.15.32:0/1842946739 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff478011d20 con 0x7ff4880c1880 2023-12-20T04:28:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13c32c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb134071920 0x7fb134201e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13c32c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1341575c0 0x7fb1340a9f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13c32c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13415db50 0x7fb1340aa460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13a0c8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb134071920 0x7fb134201e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13a0c8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb134071920 0x7fb134201e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48936/0 (socket says 172.21.15.32:48936) 2023-12-20T04:28:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb1398c7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1341575c0 0x7fb1340a9f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb1398c7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1341575c0 0x7fb1340a9f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49510/0 (socket says 172.21.15.32:49510) 2023-12-20T04:28:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13a0c8640 1 -- 172.21.15.32:0/269678483 learned_addr learned my addr 172.21.15.32:0/269678483 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340beec0 con 0x7fb13415db50 2023-12-20T04:28:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340ab520 con 0x7fb134071920 2023-12-20T04:28:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13a8c9640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13415db50 0x7fb1340aa460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1340aac80 con 0x7fb1341575c0 2023-12-20T04:28:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.672+0000 7f43594b3640 1 -- 172.21.15.32:0/1129963395 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4354165380 con 0x7f43540ae010 2023-12-20T04:28:44.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7f8e56d9d640 1 -- 172.21.15.32:0/1364867039 >> v1:172.21.15.32:6789/0 conn(0x7f8e580ae010 legacy=0x7f8e58071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.676+0000 7f8e56d9d640 1 -- 172.21.15.32:0/1364867039 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e58165660 con 0x7f8e58164ef0 2023-12-20T04:28:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7f8e55d9b640 1 --2- 172.21.15.32:0/1364867039 >> v2:172.21.15.39:3300/0 conn(0x7f8e5806cf10 0x7f8e580c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:28:44.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7ff48f041640 1 -- 172.21.15.32:0/1842946739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff468005260 con 0x7ff4880c1880 2023-12-20T04:28:44.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fb13a0c8640 1 -- 172.21.15.32:0/269678483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1341575c0 msgr2=0x7fb1340a9f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fb13a0c8640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1341575c0 0x7fb1340a9f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fb13a0c8640 1 -- 172.21.15.32:0/269678483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13415db50 msgr2=0x7fb1340aa460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fa7a9ffb640 1 -- 172.21.15.32:0/1235498638 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa7ac01d3b0 con 0x7fa7b415cdf0 2023-12-20T04:28:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa2060640 1 --2- 172.21.15.32:0/1246139819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c1599e0 0x7fba9c0aece0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba94009ea0 tx=0x7fba9401c9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=22dfdb78a204c167 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7f43594b3640 1 --2- 172.21.15.32:0/1129963395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43540ae010 0x7f4354071470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4344009de0 tx=0x7f434401c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e518cf9e6b6d9acd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa105e640 1 -- 172.21.15.32:0/1246139819 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba940a5760 con 0x7fba9c1599e0 2023-12-20T04:28:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa105e640 1 -- 172.21.15.32:0/1246139819 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba940a58c0 con 0x7fba9c1599e0 2023-12-20T04:28:44.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa105e640 1 -- 172.21.15.32:0/1246139819 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba940a5c50 con 0x7fba9c1599e0 2023-12-20T04:28:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7f4353fff640 1 -- 172.21.15.32:0/1129963395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43440a5990 con 0x7f43540ae010 2023-12-20T04:28:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7f4353fff640 1 -- 172.21.15.32:0/1129963395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43440a5af0 con 0x7f43540ae010 2023-12-20T04:28:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7ff4857fa640 1 -- 172.21.15.32:0/1842946739 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff47801a400 con 0x7ff4880c1880 2023-12-20T04:28:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7ff4857fa640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff474065a60 0x7ff474067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7ff4857fa640 1 -- 172.21.15.32:0/1842946739 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff478083200 con 0x7ff4880c1880 2023-12-20T04:28:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1246139819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c1599e0 msgr2=0x7fba9c0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa42c4640 1 --2- 172.21.15.32:0/1246139819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c1599e0 0x7fba9c0aece0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fba94009ea0 tx=0x7fba9401c9b0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fb13a0c8640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13415db50 0x7fb1340aa460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fb13a0c8640 1 -- 172.21.15.32:0/269678483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1300094d0 con 0x7fb134071920 2023-12-20T04:28:44.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7f4353fff640 1 -- 172.21.15.32:0/1129963395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43440a5990 con 0x7f43540ae010 2023-12-20T04:28:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7f8e56d9d640 1 --2- 172.21.15.32:0/1364867039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e58164ef0 0x7f8e580c99e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8e58164e50 tx=0x7f8e4c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecfab9424d4a59f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fb13a8c9640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13415db50 0x7fb1340aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7ff48cddd640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff474065a60 0x7ff474067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1246139819 shutdown_connections 2023-12-20T04:28:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fa7a9ffb640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa798065ae0 0x7fa798067fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fa7a9ffb640 1 -- 172.21.15.32:0/1235498638 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa7ac0830b0 con 0x7fa7b415cdf0 2023-12-20T04:28:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa42c4640 1 --2- 172.21.15.32:0/1246139819 >> v2:172.21.15.32:3300/0 conn(0x7fba9c0b0d00 0x7fba9c0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa42c4640 1 --2- 172.21.15.32:0/1246139819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c1599e0 0x7fba9c0aece0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1246139819 >> 172.21.15.32:0/1246139819 conn(0x7fba9c06a320 msgr2=0x7fba9c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7f471bbc0640 1 Processor -- start 2023-12-20T04:28:44.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fa7b91ff640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa798065ae0 0x7fa798067fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fb13a0c8640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb134071920 0x7fb134201e60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb13001cab0 tx=0x7fb130018fb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7f68b4196640 1 Processor -- start 2023-12-20T04:28:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1246139819 shutdown_connections 2023-12-20T04:28:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1246139819 wait complete. 2023-12-20T04:28:44.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7f471bbc0640 1 -- start start 2023-12-20T04:28:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.680+0000 7f68b4196640 1 -- start start 2023-12-20T04:28:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7fa7b91ff640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa798065ae0 0x7fa798067fa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7b415d3e0 tx=0x7fa7b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f435b717640 1 -- 172.21.15.32:0/1129963395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43540ae010 msgr2=0x7f4354071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f435b717640 1 --2- 172.21.15.32:0/1129963395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43540ae010 0x7f4354071470 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7f4344009de0 tx=0x7f434401c8d0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7ff48cddd640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff474065a60 0x7ff474067f20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff47c006fd0 tx=0x7ff47c0090e0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f435b717640 1 -- 172.21.15.32:0/1129963395 shutdown_connections 2023-12-20T04:28:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471bbc0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f47140cfc80 0x7f4714167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f435b717640 1 --2- 172.21.15.32:0/1129963395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4354164ef0 0x7f43540719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f435b717640 1 --2- 172.21.15.32:0/1129963395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43540ae010 0x7f4354071470 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f8e5559a640 1 -- 172.21.15.32:0/1364867039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c0ab640 con 0x7f8e58164ef0 2023-12-20T04:28:44.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f435b717640 1 -- 172.21.15.32:0/1129963395 >> 172.21.15.32:0/1129963395 conn(0x7f435406a320 msgr2=0x7f43540ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471bbc0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47140b6ca0 con 0x7f471415cdb0 2023-12-20T04:28:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f8e5559a640 1 -- 172.21.15.32:0/1364867039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c0a7070 con 0x7f8e58164ef0 2023-12-20T04:28:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471bbc0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f471415d190 con 0x7f47140d2740 2023-12-20T04:28:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471bbc0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47140d16a0 con 0x7f47140cfc80 2023-12-20T04:28:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b4196640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f68ac15cdb0 0x7f68ac0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b4196640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac15d190 con 0x7f68ac0d2740 2023-12-20T04:28:44.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471a15d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f47140cfc80 0x7f4714167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b4196640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68ac0d16a0 con 0x7f68ac0cfc80 2023-12-20T04:28:44.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b4196640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68ac0d1980 con 0x7f68ac15cdb0 2023-12-20T04:28:44.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f435b717640 1 -- 172.21.15.32:0/1129963395 shutdown_connections 2023-12-20T04:28:44.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471995c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f471415cdb0 0x7f47140d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35394/0 (socket says 172.21.15.32:35394) 2023-12-20T04:28:44.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471995c640 1 -- 172.21.15.32:0/521849900 learned_addr learned my addr 172.21.15.32:0/521849900 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:44.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f435b717640 1 -- 172.21.15.32:0/1129963395 wait complete. 2023-12-20T04:28:44.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7fbaa42c4640 1 Processor -- start 2023-12-20T04:28:44.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f8e5559a640 1 -- 172.21.15.32:0/1364867039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c0ab9c0 con 0x7f8e58164ef0 2023-12-20T04:28:44.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b1731640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f68ac0d2740 0x7f68ac0cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35404/0 (socket says 172.21.15.32:35404) 2023-12-20T04:28:44.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b1731640 1 -- 172.21.15.32:0/1965315107 learned_addr learned my addr 172.21.15.32:0/1965315107 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:44.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b1f32640 1 --2- 172.21.15.32:0/1965315107 >> v2:172.21.15.32:3300/0 conn(0x7f68ac15cdb0 0x7f68ac0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471895a640 1 -- 172.21.15.32:0/521849900 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 31929905 0 0) 0x7f4704001050 con 0x7f471415cdb0 2023-12-20T04:28:44.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471895a640 1 -- 172.21.15.32:0/521849900 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4700003530 con 0x7f471415cdb0 2023-12-20T04:28:44.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471a15d640 1 -- 172.21.15.32:0/521849900 >> v1:172.21.15.39:6789/0 conn(0x7f47140d2740 legacy=0x7f47140cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471a15d640 1 -- 172.21.15.32:0/521849900 >> v1:172.21.15.32:6789/0 conn(0x7f471415cdb0 legacy=0x7f47140d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f471a15d640 1 -- 172.21.15.32:0/521849900 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47140d1980 con 0x7f47140cfc80 2023-12-20T04:28:44.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1364867039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e58164ef0 msgr2=0x7f8e580c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f8e5cf39640 1 --2- 172.21.15.32:0/1364867039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e58164ef0 0x7f8e580c99e0 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7f8e58164e50 tx=0x7f8e4c01c480 comp rx=0 tx=0).stop 2023-12-20T04:28:44.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7fbaa42c4640 1 -- start start 2023-12-20T04:28:44.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b0f30640 1 -- 172.21.15.32:0/1965315107 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2028104048 0 0) 0x7f68ac15d190 con 0x7f68ac0d2740 2023-12-20T04:28:44.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b0f30640 1 -- 172.21.15.32:0/1965315107 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6898003530 con 0x7f68ac0d2740 2023-12-20T04:28:44.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b1f32640 1 -- 172.21.15.32:0/1965315107 >> v1:172.21.15.39:6789/0 conn(0x7f68ac0cfc80 legacy=0x7f68ac0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b1f32640 1 -- 172.21.15.32:0/1965315107 >> v1:172.21.15.32:6789/0 conn(0x7f68ac0d2740 legacy=0x7f68ac0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7f68b1f32640 1 -- 172.21.15.32:0/1965315107 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68ac0d1c60 con 0x7f68ac15cdb0 2023-12-20T04:28:44.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.684+0000 7ff4857fa640 1 -- 172.21.15.32:0/1842946739 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff478050bb0 con 0x7ff4880c1880 2023-12-20T04:28:44.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1364867039 shutdown_connections 2023-12-20T04:28:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f8e5cf39640 1 --2- 172.21.15.32:0/1364867039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e58164ef0 0x7f8e580c99e0 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f8e5cf39640 1 --2- 172.21.15.32:0/1364867039 >> v2:172.21.15.39:3300/0 conn(0x7f8e5806cf10 0x7f8e580c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1364867039 >> 172.21.15.32:0/1364867039 conn(0x7f8e5806a320 msgr2=0x7f8e580ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f435b717640 1 Processor -- start 2023-12-20T04:28:44.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1364867039 shutdown_connections 2023-12-20T04:28:44.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa42c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c0b0d00 0x7fba9c146f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa42c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba9c0c90e0 0x7fba9c14c690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1364867039 wait complete. 2023-12-20T04:28:44.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f435b717640 1 -- start start 2023-12-20T04:28:44.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f471a15d640 1 --2- 172.21.15.32:0/521849900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47140cfc80 0x7f4714167910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f470c009d90 tx=0x7f470c01f100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f1b956b636f5fcb1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa42c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c1599e0 0x7fba9c0ce130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa42c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c0c4d60 con 0x7fba9c1599e0 2023-12-20T04:28:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa42c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c1474d0 con 0x7fba9c0c90e0 2023-12-20T04:28:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa42c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c0c6c80 con 0x7fba9c0b0d00 2023-12-20T04:28:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa2060640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c0b0d00 0x7fba9c146f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa185f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba9c0c90e0 0x7fba9c14c690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa2060640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c0b0d00 0x7fba9c146f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49512/0 (socket says 172.21.15.32:49512) 2023-12-20T04:28:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa2060640 1 -- 172.21.15.32:0/1936865877 learned_addr learned my addr 172.21.15.32:0/1936865877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa2861640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c1599e0 0x7fba9c0ce130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa2060640 1 -- 172.21.15.32:0/1936865877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba9c0c90e0 msgr2=0x7fba9c14c690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa2060640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba9c0c90e0 0x7fba9c14c690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa2060640 1 -- 172.21.15.32:0/1936865877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c1599e0 msgr2=0x7fba9c0ce130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa2060640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c1599e0 0x7fba9c0ce130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa2060640 1 -- 172.21.15.32:0/1936865877 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba94009b30 con 0x7fba9c0b0d00 2023-12-20T04:28:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fbaa2861640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c1599e0 0x7fba9c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7fa7a9ffb640 1 -- 172.21.15.32:0/1235498638 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa7ac050ae0 con 0x7fa7b415cdf0 2023-12-20T04:28:44.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f435b717640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43540ae010 0x7f4354159a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f8e5cf39640 1 Processor -- start 2023-12-20T04:28:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f471895a640 1 -- 172.21.15.32:0/521849900 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c0af600 con 0x7f47140cfc80 2023-12-20T04:28:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f471895a640 1 -- 172.21.15.32:0/521849900 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c0ab050 con 0x7f47140cfc80 2023-12-20T04:28:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.688+0000 7f471895a640 1 -- 172.21.15.32:0/521849900 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c0af980 con 0x7f47140cfc80 2023-12-20T04:28:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f435b717640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4354160e90 0x7f435415fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f435b717640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43541f98e0 0x7f43541f9d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f435b717640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4354159fb0 con 0x7f43541f98e0 2023-12-20T04:28:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f435b717640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4354160370 con 0x7f4354160e90 2023-12-20T04:28:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f435b717640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4354160650 con 0x7f43540ae010 2023-12-20T04:28:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f43594b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43540ae010 0x7f4354159a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f43594b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43540ae010 0x7f4354159a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49528/0 (socket says 172.21.15.32:49528) 2023-12-20T04:28:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f43594b3640 1 -- 172.21.15.32:0/3369603903 learned_addr learned my addr 172.21.15.32:0/3369603903 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f471bbc0640 1 -- 172.21.15.32:0/521849900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47140cfc80 msgr2=0x7f4714167910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f471bbc0640 1 --2- 172.21.15.32:0/521849900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47140cfc80 0x7f4714167910 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f470c009d90 tx=0x7f470c01f100 comp rx=0 tx=0).stop 2023-12-20T04:28:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f4359cb4640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43541f98e0 0x7f43541f9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f4358cb2640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4354160e90 0x7f435415fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e5cf39640 1 -- start start 2023-12-20T04:28:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e5cf39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e580ae010 0x7f8e580b4150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e5cf39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e58164090 0x7f8e581565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e5cf39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e580c42e0 0x7f8e580c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e5cf39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e580c4c90 con 0x7f8e580c42e0 2023-12-20T04:28:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e5cf39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e580b4690 con 0x7f8e580ae010 2023-12-20T04:28:44.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e5cf39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e580b3d90 con 0x7f8e58164090 2023-12-20T04:28:44.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fbaa2060640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c0b0d00 0x7fba9c146f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba940af040 tx=0x7fba94019750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e56d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e580c42e0 0x7f8e580c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fb12affd640 1 -- 172.21.15.32:0/269678483 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1300b5400 con 0x7fb134071920 2023-12-20T04:28:44.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fb12affd640 1 -- 172.21.15.32:0/269678483 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1300b5560 con 0x7fb134071920 2023-12-20T04:28:44.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fb12affd640 1 -- 172.21.15.32:0/269678483 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1300b58f0 con 0x7fb134071920 2023-12-20T04:28:44.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1340aaf60 con 0x7fb134071920 2023-12-20T04:28:44.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1340aa9a0 con 0x7fb134071920 2023-12-20T04:28:44.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f4359cb4640 1 -- 172.21.15.32:0/3369603903 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43540ae010 msgr2=0x7f4354159a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f4359cb4640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43540ae010 0x7f4354159a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f4359cb4640 1 -- 172.21.15.32:0/3369603903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4354160e90 msgr2=0x7f435415fb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f4359cb4640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4354160e90 0x7f435415fb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f4359cb4640 1 -- 172.21.15.32:0/3369603903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4344009a70 con 0x7f43541f98e0 2023-12-20T04:28:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f43594b3640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43540ae010 0x7f4354159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fba92ffd640 1 -- 172.21.15.32:0/1936865877 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba940b8640 con 0x7fba9c0b0d00 2023-12-20T04:28:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fba92ffd640 1 -- 172.21.15.32:0/1936865877 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba940b87a0 con 0x7fba9c0b0d00 2023-12-20T04:28:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fba92ffd640 1 -- 172.21.15.32:0/1936865877 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba940b8b10 con 0x7fba9c0b0d00 2023-12-20T04:28:44.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1936865877 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9c0c6f20 con 0x7fba9c0b0d00 2023-12-20T04:28:44.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1936865877 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9c0c74e0 con 0x7fba9c0b0d00 2023-12-20T04:28:44.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f4359cb4640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43541f98e0 0x7f43541f9d60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e7c0 tx=0x7f434c00ec80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e56d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e580c42e0 0x7f8e580c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48474/0 (socket says 172.21.15.32:48474) 2023-12-20T04:28:44.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e56d9d640 1 -- 172.21.15.32:0/1302135305 learned_addr learned my addr 172.21.15.32:0/1302135305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f68b1f32640 1 --2- 172.21.15.32:0/1965315107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68ac15cdb0 0x7f68ac0d0be0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009d90 tx=0x7f68a401f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=537434c6969a3114 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e56d9d640 1 -- 172.21.15.32:0/1302135305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e58164090 msgr2=0x7f8e581565f0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-20T04:28:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e56d9d640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e58164090 0x7f8e581565f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f8e56d9d640 1 -- 172.21.15.32:0/1302135305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e580ae010 msgr2=0x7f8e580b4150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:28:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1936865877 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9c04e000 con 0x7fba9c0b0d00 2023-12-20T04:28:44.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb13404e000 con 0x7fb134071920 2023-12-20T04:28:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f471bbc0640 1 -- 172.21.15.32:0/521849900 shutdown_connections 2023-12-20T04:28:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f471bbc0640 1 --2- 172.21.15.32:0/521849900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47140cfc80 0x7f4714167910 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f471bbc0640 1 -- 172.21.15.32:0/521849900 >> 172.21.15.32:0/521849900 conn(0x7f4714065ee0 msgr2=0x7f471406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f4351ffb640 1 -- 172.21.15.32:0/3369603903 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c00c730 con 0x7f43541f98e0 2023-12-20T04:28:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f4351ffb640 1 -- 172.21.15.32:0/3369603903 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f434c00c890 con 0x7f43541f98e0 2023-12-20T04:28:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f4351ffb640 1 -- 172.21.15.32:0/3369603903 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f434c00cc20 con 0x7f43541f98e0 2023-12-20T04:28:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f435b717640 1 -- 172.21.15.32:0/3369603903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43540c6a90 con 0x7f43541f98e0 2023-12-20T04:28:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f435b717640 1 -- 172.21.15.32:0/3369603903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4354160080 con 0x7f43541f98e0 2023-12-20T04:28:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7fb12affd640 1 -- 172.21.15.32:0/269678483 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb1300b5a50 con 0x7fb134071920 2023-12-20T04:28:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f8e56d9d640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e580ae010 0x7f8e580b4150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.692+0000 7f68b0f30640 1 -- 172.21.15.32:0/1965315107 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a40af600 con 0x7f68ac15cdb0 2023-12-20T04:28:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f68b0f30640 1 -- 172.21.15.32:0/1965315107 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a40ab050 con 0x7f68ac15cdb0 2023-12-20T04:28:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f68b0f30640 1 -- 172.21.15.32:0/1965315107 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a40af960 con 0x7f68ac15cdb0 2023-12-20T04:28:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f8e56d9d640 1 -- 172.21.15.32:0/1302135305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e4c0094d0 con 0x7f8e580c42e0 2023-12-20T04:28:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7fba92ffd640 1 -- 172.21.15.32:0/1936865877 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fba94006d50 con 0x7fba9c0b0d00 2023-12-20T04:28:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7fba92ffd640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba80065ae0 0x7fba80067fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7fba92ffd640 1 -- 172.21.15.32:0/1936865877 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fba9411da80 con 0x7fba9c0b0d00 2023-12-20T04:28:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7fb12affd640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb118065b30 0x7fb118067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f8e56d9d640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e580c42e0 0x7f8e580c4750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0ad040 tx=0x7f8e4c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f435b717640 1 -- 172.21.15.32:0/3369603903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f435404e000 con 0x7f43541f98e0 2023-12-20T04:28:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f471bbc0640 1 -- 172.21.15.32:0/521849900 shutdown_connections 2023-12-20T04:28:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f68b4196640 1 -- 172.21.15.32:0/1965315107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68ac15cdb0 msgr2=0x7f68ac0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f68b4196640 1 --2- 172.21.15.32:0/1965315107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68ac15cdb0 0x7f68ac0d0be0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7f68a4009d90 tx=0x7f68a401f130 comp rx=0 tx=0).stop 2023-12-20T04:28:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7fbaa185f640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba80065ae0 0x7fba80067fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7fb12affd640 1 -- 172.21.15.32:0/269678483 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb13011d560 con 0x7fb134071920 2023-12-20T04:28:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7fbaa185f640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba80065ae0 0x7fba80067fa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0cebb0 tx=0x7fba98009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7fb1398c7640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb118065b30 0x7fb118067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7f68b4196640 1 -- 172.21.15.32:0/1965315107 shutdown_connections 2023-12-20T04:28:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7f68b4196640 1 --2- 172.21.15.32:0/1965315107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68ac15cdb0 0x7f68ac0d0be0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7f68b4196640 1 -- 172.21.15.32:0/1965315107 >> 172.21.15.32:0/1965315107 conn(0x7f68ac065ee0 msgr2=0x7f68ac06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:44.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7f4351ffb640 1 -- 172.21.15.32:0/3369603903 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f434c013470 con 0x7f43541f98e0 2023-12-20T04:28:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.696+0000 7f471bbc0640 1 -- 172.21.15.32:0/521849900 wait complete. 2023-12-20T04:28:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7f471bbc0640 1 Processor -- start 2023-12-20T04:28:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7fb1398c7640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb118065b30 0x7fb118067ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb134205a90 tx=0x7fb124006f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7fb12affd640 1 -- 172.21.15.32:0/269678483 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb1300eaf90 con 0x7fb134071920 2023-12-20T04:28:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7f68b4196640 1 -- 172.21.15.32:0/1965315107 shutdown_connections 2023-12-20T04:28:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7f68b4196640 1 -- 172.21.15.32:0/1965315107 wait complete. 2023-12-20T04:28:44.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7f471bbc0640 1 -- start start 2023-12-20T04:28:44.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7fba92ffd640 1 -- 172.21.15.32:0/1936865877 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fba940eb5d0 con 0x7fba9c0b0d00 2023-12-20T04:28:44.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.700+0000 7f4351ffb640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4340065b60 0x7f4340068020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f4351ffb640 1 -- 172.21.15.32:0/3369603903 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f434c083800 con 0x7f43541f98e0 2023-12-20T04:28:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471bbc0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47140cfc80 0x7f4714158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471bbc0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47140d2740 0x7f4714153950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471bbc0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f471415cdb0 0x7f4714153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471bbc0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714159710 con 0x7f47140cfc80 2023-12-20T04:28:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471bbc0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4714159150 con 0x7f471415cdb0 2023-12-20T04:28:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471bbc0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4714159430 con 0x7f47140d2740 2023-12-20T04:28:44.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f43594b3640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4340065b60 0x7f4340068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471a15d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f471415cdb0 0x7f4714153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471a15d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f471415cdb0 0x7f4714153e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:49000/0 (socket says 172.21.15.32:49000) 2023-12-20T04:28:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471a15d640 1 -- 172.21.15.32:0/232090260 learned_addr learned my addr 172.21.15.32:0/232090260 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f68b4196640 1 Processor -- start 2023-12-20T04:28:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471a15d640 1 -- 172.21.15.32:0/232090260 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47140d2740 msgr2=0x7f4714153950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-20T04:28:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471a15d640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47140d2740 0x7f4714153950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471a15d640 1 -- 172.21.15.32:0/232090260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47140cfc80 msgr2=0x7f4714158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471a15d640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47140cfc80 0x7f4714158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471a15d640 1 -- 172.21.15.32:0/232090260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f470c009a70 con 0x7f471415cdb0 2023-12-20T04:28:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f43594b3640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4340065b60 0x7f4340068020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f43540c7e80 tx=0x7f43440a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f471a15d640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f471415cdb0 0x7f4714153e90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f470c009d90 tx=0x7f470c01bea0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f470a7fc640 1 -- 172.21.15.32:0/232090260 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c0ba420 con 0x7f471415cdb0 2023-12-20T04:28:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f4351ffb640 1 -- 172.21.15.32:0/3369603903 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f434c0512e0 con 0x7f43541f98e0 2023-12-20T04:28:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f8e46ffd640 1 -- 172.21.15.32:0/1302135305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c0be5f0 con 0x7f8e580c42e0 2023-12-20T04:28:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f68b4196640 1 -- start start 2023-12-20T04:28:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f8e46ffd640 1 -- 172.21.15.32:0/1302135305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e4c0be770 con 0x7f8e580c42e0 2023-12-20T04:28:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f8e46ffd640 1 -- 172.21.15.32:0/1302135305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e4c0beb20 con 0x7f8e580c42e0 2023-12-20T04:28:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f470a7fc640 1 -- 172.21.15.32:0/232090260 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f470c0ba580 con 0x7f471415cdb0 2023-12-20T04:28:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.704+0000 7f470a7fc640 1 -- 172.21.15.32:0/232090260 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f470c0ba930 con 0x7f471415cdb0 2023-12-20T04:28:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1302135305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e580c6240 con 0x7f8e580c42e0 2023-12-20T04:28:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1302135305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e58164830 con 0x7f8e580c42e0 2023-12-20T04:28:44.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1302135305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e5804e000 con 0x7f8e580c42e0 2023-12-20T04:28:44.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f471bbc0640 1 -- 172.21.15.32:0/232090260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f471415d940 con 0x7f471415cdb0 2023-12-20T04:28:44.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f471bbc0640 1 -- 172.21.15.32:0/232090260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4714158e40 con 0x7f471415cdb0 2023-12-20T04:28:44.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f68b4196640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68ac0cfc80 0x7f68ac1f5230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f68b4196640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68ac0d2740 0x7f68ac15ef10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f68b4196640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68ac15cdb0 0x7f68ac15f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f68b4196640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac15ffe0 con 0x7f68ac0d2740 2023-12-20T04:28:44.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f68b4196640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac15d470 con 0x7f68ac15cdb0 2023-12-20T04:28:44.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f68b4196640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68ac15d750 con 0x7f68ac0cfc80 2023-12-20T04:28:44.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f68b1731640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68ac0d2740 0x7f68ac15ef10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f68b1731640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68ac0d2740 0x7f68ac15ef10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48492/0 (socket says 172.21.15.32:48492) 2023-12-20T04:28:44.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f68b1731640 1 -- 172.21.15.32:0/275501876 learned_addr learned my addr 172.21.15.32:0/275501876 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:44.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f68b1731640 1 -- 172.21.15.32:0/275501876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68ac0cfc80 msgr2=0x7f68ac1f5230 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-20T04:28:44.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.708+0000 7f68b1731640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68ac0cfc80 0x7f68ac1f5230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.712+0000 7f68b1731640 1 -- 172.21.15.32:0/275501876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68ac15cdb0 msgr2=0x7f68ac15f450 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-20T04:28:44.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.712+0000 7f68b1731640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68ac15cdb0 0x7f68ac15f450 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:44.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.712+0000 7f68b1731640 1 -- 172.21.15.32:0/275501876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68a4009a70 con 0x7f68ac0d2740 2023-12-20T04:28:44.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.712+0000 7f68b1731640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68ac0d2740 0x7f68ac15ef10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f68a8001c20 tx=0x7f68a8014420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.712+0000 7f470a7fc640 1 -- 172.21.15.32:0/232090260 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f470c0baa90 con 0x7f471415cdb0 2023-12-20T04:28:44.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.712+0000 7f471bbc0640 1 -- 172.21.15.32:0/232090260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f471415d7a0 con 0x7f471415cdb0 2023-12-20T04:28:44.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.712+0000 7f470a7fc640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46f8065a60 0x7f46f8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.712+0000 7f470a7fc640 1 -- 172.21.15.32:0/232090260 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f470c121e90 con 0x7f471415cdb0 2023-12-20T04:28:44.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.716+0000 7f471995c640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46f8065a60 0x7f46f8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.716+0000 7f8e46ffd640 1 -- 172.21.15.32:0/1302135305 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8e4c0c8410 con 0x7f8e580c42e0 2023-12-20T04:28:44.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.716+0000 7f471995c640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46f8065a60 0x7f46f8067f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4704001590 tx=0x7f4704007910 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.716+0000 7f8e46ffd640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e38065ab0 0x7f8e38067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.716+0000 7f8e46ffd640 1 -- 172.21.15.32:0/1302135305 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8e4c11de40 con 0x7f8e580c42e0 2023-12-20T04:28:44.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.716+0000 7f8e5659c640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e38065ab0 0x7f8e38067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.716+0000 7f470a7fc640 1 -- 172.21.15.32:0/232090260 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f471415d7a0 con 0x7f471415cdb0 2023-12-20T04:28:44.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.720+0000 7f689e7fc640 1 -- 172.21.15.32:0/275501876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a8017070 con 0x7f68ac0d2740 2023-12-20T04:28:44.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.720+0000 7f68b4196640 1 -- 172.21.15.32:0/275501876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68ac15fc70 con 0x7f68ac0d2740 2023-12-20T04:28:44.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.720+0000 7f68b4196640 1 -- 172.21.15.32:0/275501876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68ac15da30 con 0x7f68ac0d2740 2023-12-20T04:28:44.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.720+0000 7f689e7fc640 1 -- 172.21.15.32:0/275501876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68a800b020 con 0x7f68ac0d2740 2023-12-20T04:28:44.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.720+0000 7f689e7fc640 1 -- 172.21.15.32:0/275501876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68a80058c0 con 0x7f68ac0d2740 2023-12-20T04:28:44.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.720+0000 7f8e46ffd640 1 -- 172.21.15.32:0/1302135305 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8e4c0eb8e0 con 0x7f8e580c42e0 2023-12-20T04:28:44.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.720+0000 7f68b4196640 1 -- 172.21.15.32:0/275501876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68ac160190 con 0x7f68ac0d2740 2023-12-20T04:28:44.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.720+0000 7f8e5659c640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e38065ab0 0x7f8e38067f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8e581fcf60 tx=0x7f8e50007cc0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.724+0000 7f689e7fc640 1 -- 172.21.15.32:0/275501876 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f68a802a030 con 0x7f68ac0d2740 2023-12-20T04:28:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.724+0000 7f689e7fc640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6890065a90 0x7f6890067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.724+0000 7f689e7fc640 1 -- 172.21.15.32:0/275501876 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f68a800c030 con 0x7f68ac0d2740 2023-12-20T04:28:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.728+0000 7f68b1f32640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6890065a90 0x7f6890067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.732+0000 7f68b1f32640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6890065a90 0x7f6890067f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f68a40b1000 tx=0x7f68a40aa000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:44.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:44.732+0000 7f689e7fc640 1 -- 172.21.15.32:0/275501876 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f68a80c40a0 con 0x7f68ac0d2740 2023-12-20T04:28:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.012+0000 7fa7bb463640 1 -- 172.21.15.32:0/1235498638 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7fa7b4074100 con 0x7fa7b415cdf0 2023-12-20T04:28:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.016+0000 7fa7a9ffb640 1 -- 172.21.15.32:0/1235498638 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fa7ac019090 con 0x7fa7b415cdf0 2023-12-20T04:28:45.016 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-12-20T04:28:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 -- 172.21.15.32:0/1235498638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa798065ae0 msgr2=0x7fa798067fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa798065ae0 0x7fa798067fa0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fa7b415d3e0 tx=0x7fa7b0008040 comp rx=0 tx=0).stop 2023-12-20T04:28:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 -- 172.21.15.32:0/1235498638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b415cdf0 msgr2=0x7fa7b40c67a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b415cdf0 0x7fa7b40c67a0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fa7ac00b970 tx=0x7fa7ac00be30 comp rx=0 tx=0).stop 2023-12-20T04:28:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 -- 172.21.15.32:0/1235498638 shutdown_connections 2023-12-20T04:28:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa798065ae0 0x7fa798067fa0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7b415cdf0 0x7fa7b40c67a0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7b4070cf0 0x7fa7b4158eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 --2- 172.21.15.32:0/1235498638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7b406cfd0 0x7fa7b41596c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 -- 172.21.15.32:0/1235498638 >> 172.21.15.32:0/1235498638 conn(0x7fa7b4069d20 msgr2=0x7fa7b40cd4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 -- 172.21.15.32:0/1235498638 shutdown_connections 2023-12-20T04:28:45.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.020+0000 7fa7bb463640 1 -- 172.21.15.32:0/1235498638 wait complete. 2023-12-20T04:28:45.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.028+0000 7ff48f041640 1 -- 172.21.15.32:0/1842946739 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7ff468005510 con 0x7ff4880c1880 2023-12-20T04:28:45.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.028+0000 7ff4857fa640 1 -- 172.21.15.32:0/1842946739 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7ff478015020 con 0x7ff4880c1880 2023-12-20T04:28:45.029 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-12-20T04:28:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 -- 172.21.15.32:0/1842946739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff474065a60 msgr2=0x7ff474067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff474065a60 0x7ff474067f20 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7ff47c006fd0 tx=0x7ff47c0090e0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 -- 172.21.15.32:0/1842946739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4880c1880 msgr2=0x7ff4880c29c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4880c1880 0x7ff4880c29c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7ff47800b3e0 tx=0x7ff47800b8a0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 -- 172.21.15.32:0/1842946739 shutdown_connections 2023-12-20T04:28:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff474065a60 0x7ff474067f20 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff488165170 0x7ff488148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4880c1880 0x7ff4880c29c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 --2- 172.21.15.32:0/1842946739 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff48806caf0 0x7ff4880c4a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 -- 172.21.15.32:0/1842946739 >> 172.21.15.32:0/1842946739 conn(0x7ff48806a320 msgr2=0x7ff48815dc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1936865877 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7fba9c159210 con 0x7fba9c0b0d00 2023-12-20T04:28:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 -- 172.21.15.32:0/1842946739 shutdown_connections 2023-12-20T04:28:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.032+0000 7ff48f041640 1 -- 172.21.15.32:0/1842946739 wait complete. 2023-12-20T04:28:45.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.036+0000 7fba92ffd640 1 -- 172.21.15.32:0/1936865877 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fba940eebf0 con 0x7fba9c0b0d00 2023-12-20T04:28:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.036+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fb13406d490 con 0x7fb134071920 2023-12-20T04:28:45.037 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2023-12-20T04:28:45.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.036+0000 7fb12affd640 1 -- 172.21.15.32:0/269678483 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fb1300ee5b0 con 0x7fb134071920 2023-12-20T04:28:45.038 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-12-20T04:28:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb118065b30 msgr2=0x7fb118067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fb13c32c640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb118065b30 0x7fb118067ff0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb134205a90 tx=0x7fb124006f90 comp rx=0 tx=0).stop 2023-12-20T04:28:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1936865877 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba80065ae0 msgr2=0x7fba80067fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba80065ae0 0x7fba80067fa0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fba9c0cebb0 tx=0x7fba98009040 comp rx=0 tx=0).stop 2023-12-20T04:28:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1936865877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c0b0d00 msgr2=0x7fba9c146f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c0b0d00 0x7fba9c146f90 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fba940af040 tx=0x7fba94019750 comp rx=0 tx=0).stop 2023-12-20T04:28:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb134071920 msgr2=0x7fb134201e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fb13c32c640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb134071920 0x7fb134201e60 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fb13001cab0 tx=0x7fb130018fb0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1936865877 shutdown_connections 2023-12-20T04:28:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba80065ae0 0x7fba80067fa0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c1599e0 0x7fba9c0ce130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba9c0c90e0 0x7fba9c14c690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 --2- 172.21.15.32:0/1936865877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c0b0d00 0x7fba9c146f90 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1936865877 >> 172.21.15.32:0/1936865877 conn(0x7fba9c06a320 msgr2=0x7fba9c15afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7f435b717640 1 -- 172.21.15.32:0/3369603903 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f43540c6fb0 con 0x7f43541f98e0 2023-12-20T04:28:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1936865877 shutdown_connections 2023-12-20T04:28:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fbaa42c4640 1 -- 172.21.15.32:0/1936865877 wait complete. 2023-12-20T04:28:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.040+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 shutdown_connections 2023-12-20T04:28:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.044+0000 7fb13c32c640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb118065b30 0x7fb118067ff0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.044+0000 7fb13c32c640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb13415db50 0x7fb1340aa460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.044+0000 7fb13c32c640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1341575c0 0x7fb1340a9f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.044+0000 7fb13c32c640 1 --2- 172.21.15.32:0/269678483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb134071920 0x7fb134201e60 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.044+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 >> 172.21.15.32:0/269678483 conn(0x7fb1340cb1e0 msgr2=0x7fb13406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.044+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 shutdown_connections 2023-12-20T04:28:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.044+0000 7fb13c32c640 1 -- 172.21.15.32:0/269678483 wait complete. 2023-12-20T04:28:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.044+0000 7f4351ffb640 1 -- 172.21.15.32:0/3369603903 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f434c054900 con 0x7f43541f98e0 2023-12-20T04:28:45.046 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-12-20T04:28:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 -- 172.21.15.32:0/3369603903 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4340065b60 msgr2=0x7f4340068020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4340065b60 0x7f4340068020 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f43540c7e80 tx=0x7f43440a7070 comp rx=0 tx=0).stop 2023-12-20T04:28:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 -- 172.21.15.32:0/3369603903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43541f98e0 msgr2=0x7f43541f9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43541f98e0 0x7f43541f9d60 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f434c00e7c0 tx=0x7f434c00ec80 comp rx=0 tx=0).stop 2023-12-20T04:28:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 -- 172.21.15.32:0/3369603903 shutdown_connections 2023-12-20T04:28:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4340065b60 0x7f4340068020 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43541f98e0 0x7f43541f9d60 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4354160e90 0x7f435415fb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 --2- 172.21.15.32:0/3369603903 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43540ae010 0x7f4354159a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 -- 172.21.15.32:0/3369603903 >> 172.21.15.32:0/3369603903 conn(0x7f435406a320 msgr2=0x7f435415d9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 -- 172.21.15.32:0/3369603903 shutdown_connections 2023-12-20T04:28:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.048+0000 7f435b717640 1 -- 172.21.15.32:0/3369603903 wait complete. 2023-12-20T04:28:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.052+0000 7f471bbc0640 1 -- 172.21.15.32:0/232090260 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7f471415de60 con 0x7f471415cdb0 2023-12-20T04:28:45.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.052+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1302135305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f8e580cf450 con 0x7f8e580c42e0 2023-12-20T04:28:45.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.052+0000 7f470a7fc640 1 -- 172.21.15.32:0/232090260 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f471415de60 con 0x7f471415cdb0 2023-12-20T04:28:45.057 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2023-12-20T04:28:45.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.056+0000 7f8e46ffd640 1 -- 172.21.15.32:0/1302135305 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f8e4c0eef00 con 0x7f8e580c42e0 2023-12-20T04:28:45.058 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-12-20T04:28:45.059 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-12-20T04:28:45.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.056+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1302135305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e38065ab0 msgr2=0x7f8e38067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.056+0000 7f8e5cf39640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e38065ab0 0x7f8e38067f70 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8e581fcf60 tx=0x7f8e50007cc0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.056+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1302135305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e580c42e0 msgr2=0x7f8e580c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.056+0000 7f8e5cf39640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e580c42e0 0x7f8e580c4750 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f8e4c0ad040 tx=0x7f8e4c01acc0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.056+0000 7f471bbc0640 1 -- 172.21.15.32:0/232090260 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46f8065a60 msgr2=0x7f46f8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.056+0000 7f471bbc0640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46f8065a60 0x7f46f8067f20 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f4704001590 tx=0x7f4704007910 comp rx=0 tx=0).stop 2023-12-20T04:28:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.056+0000 7f471bbc0640 1 -- 172.21.15.32:0/232090260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f471415cdb0 msgr2=0x7f4714153e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.056+0000 7f471bbc0640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f471415cdb0 0x7f4714153e90 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f470c009d90 tx=0x7f470c01bea0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.056+0000 7f471bbc0640 1 -- 172.21.15.32:0/232090260 shutdown_connections 2023-12-20T04:28:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f471bbc0640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46f8065a60 0x7f46f8067f20 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f471bbc0640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f471415cdb0 0x7f4714153e90 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f471bbc0640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47140d2740 0x7f4714153950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f471bbc0640 1 --2- 172.21.15.32:0/232090260 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47140cfc80 0x7f4714158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f471bbc0640 1 -- 172.21.15.32:0/232090260 >> 172.21.15.32:0/232090260 conn(0x7f4714065ee0 msgr2=0x7f47140cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1302135305 shutdown_connections 2023-12-20T04:28:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f8e5cf39640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e38065ab0 0x7f8e38067f70 secure :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8e581fcf60 tx=0x7f8e50007cc0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f8e5cf39640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e580c42e0 0x7f8e580c4750 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f8e5cf39640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e58164090 0x7f8e581565f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f8e5cf39640 1 --2- 172.21.15.32:0/1302135305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e580ae010 0x7f8e580b4150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1302135305 >> 172.21.15.32:0/1302135305 conn(0x7f8e5806a320 msgr2=0x7f8e580cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f471bbc0640 1 -- 172.21.15.32:0/232090260 shutdown_connections 2023-12-20T04:28:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f471bbc0640 1 -- 172.21.15.32:0/232090260 wait complete. 2023-12-20T04:28:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1302135305 shutdown_connections 2023-12-20T04:28:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f8e5cf39640 1 -- 172.21.15.32:0/1302135305 wait complete. 2023-12-20T04:28:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.060+0000 7f68b4196640 1 -- 172.21.15.32:0/275501876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f68ac04e000 con 0x7f68ac0d2740 2023-12-20T04:28:45.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.064+0000 7f689e7fc640 1 -- 172.21.15.32:0/275501876 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f68ac160190 con 0x7f68ac0d2740 2023-12-20T04:28:45.068 INFO:teuthology.orchestra.run.smithi032.stdout:55834574852 2023-12-20T04:28:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.064+0000 7f68b4196640 1 -- 172.21.15.32:0/275501876 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6890065a90 msgr2=0x7f6890067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.064+0000 7f68b4196640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6890065a90 0x7f6890067f50 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f68a40b1000 tx=0x7f68a40aa000 comp rx=0 tx=0).stop 2023-12-20T04:28:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.064+0000 7f68b4196640 1 -- 172.21.15.32:0/275501876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68ac0d2740 msgr2=0x7f68ac15ef10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.064+0000 7f68b4196640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68ac0d2740 0x7f68ac15ef10 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f68a8001c20 tx=0x7f68a8014420 comp rx=0 tx=0).stop 2023-12-20T04:28:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.068+0000 7f68b4196640 1 -- 172.21.15.32:0/275501876 shutdown_connections 2023-12-20T04:28:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.068+0000 7f68b4196640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6890065a90 0x7f6890067f50 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.068+0000 7f68b4196640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68ac15cdb0 0x7f68ac15f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.068+0000 7f68b4196640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68ac0d2740 0x7f68ac15ef10 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.068+0000 7f68b4196640 1 --2- 172.21.15.32:0/275501876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68ac0cfc80 0x7f68ac1f5230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.068+0000 7f68b4196640 1 -- 172.21.15.32:0/275501876 >> 172.21.15.32:0/275501876 conn(0x7f68ac065ee0 msgr2=0x7f68ac15af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.068+0000 7f68b4196640 1 -- 172.21.15.32:0/275501876 shutdown_connections 2023-12-20T04:28:45.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:45.068+0000 7f68b4196640 1 -- 172.21.15.32:0/275501876 wait complete. 2023-12-20T04:28:45.074 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.1 2023-12-20T04:28:45.086 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574854 for osd.4 2023-12-20T04:28:45.090 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.6 2023-12-20T04:28:45.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.7 2023-12-20T04:28:45.093 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.5 2023-12-20T04:28:45.094 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.3 2023-12-20T04:28:45.101 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574852 for osd.2 2023-12-20T04:28:46.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-20T04:28:46.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2023-12-20T04:28:46.087 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2023-12-20T04:28:46.091 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2023-12-20T04:28:46.095 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2023-12-20T04:28:46.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2023-12-20T04:28:46.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2023-12-20T04:28:46.102 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2023-12-20T04:28:46.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.296+0000 7f6c9aa8c640 1 Processor -- start 2023-12-20T04:28:46.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.296+0000 7f6c9aa8c640 1 -- start start 2023-12-20T04:28:46.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c9aa8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c9415cbd0 0x7f6c940cb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c9aa8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c940ca600 0x7f6c940cb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c9aa8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c94070f40 con 0x7f6c9415cfb0 2023-12-20T04:28:46.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c9aa8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c94070180 con 0x7f6c9415cbd0 2023-12-20T04:28:46.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c9aa8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c94070460 con 0x7f6c940ca600 2023-12-20T04:28:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c98828640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c9415cbd0 0x7f6c940cb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c98828640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c9415cbd0 0x7f6c940cb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48504/0 (socket says 172.21.15.32:48504) 2023-12-20T04:28:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c98828640 1 -- 172.21.15.32:0/1915343352 learned_addr learned my addr 172.21.15.32:0/1915343352 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c98828640 1 -- 172.21.15.32:0/1915343352 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c940ca600 msgr2=0x7f6c940cb5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c98828640 1 --2- 172.21.15.32:0/1915343352 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c940ca600 0x7f6c940cb5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c98828640 1 -- 172.21.15.32:0/1915343352 >> v1:172.21.15.32:6789/0 conn(0x7f6c9415cfb0 legacy=0x7f6c940c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.300+0000 7f6c98828640 1 -- 172.21.15.32:0/1915343352 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c94070740 con 0x7f6c9415cbd0 2023-12-20T04:28:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.304+0000 7f6c98828640 1 --2- 172.21.15.32:0/1915343352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9415cbd0 0x7f6c940cb090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009870 tx=0x7f6c88024ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07ebe7d20206d7c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.304+0000 7f6c937fe640 1 -- 172.21.15.32:0/1915343352 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c880ada90 con 0x7f6c9415cbd0 2023-12-20T04:28:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.304+0000 7f6c937fe640 1 -- 172.21.15.32:0/1915343352 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c880adbf0 con 0x7f6c9415cbd0 2023-12-20T04:28:46.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.308+0000 7f6c937fe640 1 -- 172.21.15.32:0/1915343352 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c880ada90 con 0x7f6c9415cbd0 2023-12-20T04:28:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.308+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/1915343352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9415cbd0 msgr2=0x7f6c940cb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.308+0000 7f6c9aa8c640 1 --2- 172.21.15.32:0/1915343352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9415cbd0 0x7f6c940cb090 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009870 tx=0x7f6c88024ca0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.308+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/1915343352 shutdown_connections 2023-12-20T04:28:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.308+0000 7f6c9aa8c640 1 --2- 172.21.15.32:0/1915343352 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c940ca600 0x7f6c940cb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.308+0000 7f6c9aa8c640 1 --2- 172.21.15.32:0/1915343352 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c9415cbd0 0x7f6c940cb090 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.308+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/1915343352 >> 172.21.15.32:0/1915343352 conn(0x7f6c94065ee0 msgr2=0x7f6c94069d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.308+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/1915343352 shutdown_connections 2023-12-20T04:28:46.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.308+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/1915343352 wait complete. 2023-12-20T04:28:46.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.312+0000 7f6c9aa8c640 1 Processor -- start 2023-12-20T04:28:46.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.312+0000 7f6c9aa8c640 1 -- start start 2023-12-20T04:28:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.312+0000 7f6c9aa8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ca600 0x7f6c941585b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c9aa8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c9415cbd0 0x7f6c941598f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c98828640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ca600 0x7f6c941585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c98828640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ca600 0x7f6c941585b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48512/0 (socket says 172.21.15.32:48512) 2023-12-20T04:28:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c98828640 1 -- 172.21.15.32:0/2498994660 learned_addr learned my addr 172.21.15.32:0/2498994660 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c9aa8c640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9415cfb0 0x7f6c94159e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/2498994660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c941583f0 con 0x7f6c940ca600 2023-12-20T04:28:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/2498994660 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940c8540 con 0x7f6c9415cfb0 2023-12-20T04:28:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c93fff640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c9415cbd0 0x7f6c941598f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/2498994660 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c940c7930 con 0x7f6c9415cbd0 2023-12-20T04:28:46.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c99029640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9415cfb0 0x7f6c94159e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c98828640 1 -- 172.21.15.32:0/2498994660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c9415cbd0 msgr2=0x7f6c941598f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c98828640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c9415cbd0 0x7f6c941598f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c98828640 1 -- 172.21.15.32:0/2498994660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9415cfb0 msgr2=0x7f6c94159e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c98828640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9415cfb0 0x7f6c94159e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c98828640 1 -- 172.21.15.32:0/2498994660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c880094d0 con 0x7f6c940ca600 2023-12-20T04:28:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c98828640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ca600 0x7f6c941585b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c88000c10 tx=0x7f6c88021a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c917fa640 1 -- 172.21.15.32:0/2498994660 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c880ada90 con 0x7f6c940ca600 2023-12-20T04:28:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/2498994660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c940c7c10 con 0x7f6c940ca600 2023-12-20T04:28:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c917fa640 1 -- 172.21.15.32:0/2498994660 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c88014020 con 0x7f6c940ca600 2023-12-20T04:28:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.316+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/2498994660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c940c81d0 con 0x7f6c940ca600 2023-12-20T04:28:46.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.320+0000 7f6c7affd640 1 -- 172.21.15.32:0/2498994660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c6c005260 con 0x7f6c940ca600 2023-12-20T04:28:46.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.328+0000 7f6c917fa640 1 -- 172.21.15.32:0/2498994660 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88014390 con 0x7f6c940ca600 2023-12-20T04:28:46.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.328+0000 7f6c917fa640 1 -- 172.21.15.32:0/2498994660 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6c880144f0 con 0x7f6c940ca600 2023-12-20T04:28:46.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.328+0000 7f6c917fa640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c80065a60 0x7f6c80067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.328+0000 7f6c917fa640 1 -- 172.21.15.32:0/2498994660 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c8811e4a0 con 0x7f6c940ca600 2023-12-20T04:28:46.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.328+0000 7f6c93fff640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c80065a60 0x7f6c80067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.332+0000 7f6c93fff640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c80065a60 0x7f6c80067f20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c940c9620 tx=0x7f6c84009060 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.332+0000 7f6c917fa640 1 -- 172.21.15.32:0/2498994660 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c880ebfd0 con 0x7f6c940ca600 2023-12-20T04:28:46.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.364+0000 7fbad9b2c640 1 Processor -- start 2023-12-20T04:28:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.364+0000 7fbad9b2c640 1 -- start start 2023-12-20T04:28:46.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.364+0000 7f467157b640 1 Processor -- start 2023-12-20T04:28:46.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.364+0000 7f467157b640 1 -- start start 2023-12-20T04:28:46.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.364+0000 7fbad9b2c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbad415cea0 0x7fbad40cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.364+0000 7fbad9b2c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbad40b84d0 0x7fbad40cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.364+0000 7fbad9b2c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbad40d0c90 con 0x7fbad415cac0 2023-12-20T04:28:46.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.364+0000 7fbad9b2c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad4160f10 con 0x7fbad40b84d0 2023-12-20T04:28:46.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.364+0000 7fbad9b2c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbad4161110 con 0x7fbad415cea0 2023-12-20T04:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7fbad3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbad40b84d0 0x7fbad40cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7fbad2ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbad415cea0 0x7fbad40cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7fbad2ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbad415cea0 0x7fbad40cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:49020/0 (socket says 172.21.15.32:49020) 2023-12-20T04:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7fbad2ffd640 1 -- 172.21.15.32:0/2017598873 learned_addr learned my addr 172.21.15.32:0/2017598873 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7fbad3fff640 1 -- 172.21.15.32:0/2017598873 >> v2:172.21.15.39:3300/0 conn(0x7fbad415cea0 msgr2=0x7fbad40cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7fbad3fff640 1 --2- 172.21.15.32:0/2017598873 >> v2:172.21.15.39:3300/0 conn(0x7fbad415cea0 0x7fbad40cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7fbad3fff640 1 -- 172.21.15.32:0/2017598873 >> v1:172.21.15.32:6789/0 conn(0x7fbad415cac0 legacy=0x7fbad4160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7fbad3fff640 1 -- 172.21.15.32:0/2017598873 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbad41613f0 con 0x7fbad40b84d0 2023-12-20T04:28:46.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7f467157b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f466c0d2740 0x7f466c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7f467157b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 0x7f466c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7f467157b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466c0cff00 con 0x7f466c15cdb0 2023-12-20T04:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7f467157b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f466c15d190 con 0x7f466c0d2740 2023-12-20T04:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7f467157b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c0d16a0 con 0x7f466c0cfab0 2023-12-20T04:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7f466a7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f466c0d2740 0x7f466c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7f466a7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f466c0d2740 0x7f466c0cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:49024/0 (socket says 172.21.15.32:49024) 2023-12-20T04:28:46.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7f466a7fc640 1 -- 172.21.15.32:0/670576273 learned_addr learned my addr 172.21.15.32:0/670576273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.368+0000 7f466b7fe640 1 --2- 172.21.15.32:0/670576273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 0x7f466c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.372+0000 7fbad3fff640 1 --2- 172.21.15.32:0/2017598873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad40b84d0 0x7fbad40cdca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbad40cd070 tx=0x7fbacc01d270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ede9f1d37c7f1e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.372+0000 7fbad27fc640 1 -- 172.21.15.32:0/2017598873 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0a58f0 con 0x7fbad40b84d0 2023-12-20T04:28:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.372+0000 7fbad27fc640 1 -- 172.21.15.32:0/2017598873 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbacc0a5a50 con 0x7fbad40b84d0 2023-12-20T04:28:46.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.372+0000 7fbad27fc640 1 -- 172.21.15.32:0/2017598873 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbacc0a5de0 con 0x7fbad40b84d0 2023-12-20T04:28:46.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.372+0000 7fbad9b2c640 1 -- 172.21.15.32:0/2017598873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad40b84d0 msgr2=0x7fbad40cdca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.372+0000 7fbad9b2c640 1 --2- 172.21.15.32:0/2017598873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad40b84d0 0x7fbad40cdca0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7fbad40cd070 tx=0x7fbacc01d270 comp rx=0 tx=0).stop 2023-12-20T04:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.372+0000 7f466a7fc640 1 -- 172.21.15.32:0/670576273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 msgr2=0x7f466c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.372+0000 7f466a7fc640 1 --2- 172.21.15.32:0/670576273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 0x7f466c0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.372+0000 7f466a7fc640 1 -- 172.21.15.32:0/670576273 >> v1:172.21.15.32:6789/0 conn(0x7f466c15cdb0 legacy=0x7f466c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.372+0000 7f466a7fc640 1 -- 172.21.15.32:0/670576273 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f466c0d1980 con 0x7f466c0d2740 2023-12-20T04:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7fbad9b2c640 1 -- 172.21.15.32:0/2017598873 shutdown_connections 2023-12-20T04:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7fbad9b2c640 1 --2- 172.21.15.32:0/2017598873 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad40b84d0 0x7fbad40cdca0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7fbad9b2c640 1 --2- 172.21.15.32:0/2017598873 >> v2:172.21.15.39:3300/0 conn(0x7fbad415cea0 0x7fbad40cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7fbad9b2c640 1 -- 172.21.15.32:0/2017598873 >> 172.21.15.32:0/2017598873 conn(0x7fbad406afd0 msgr2=0x7fbad406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7fbad9b2c640 1 -- 172.21.15.32:0/2017598873 shutdown_connections 2023-12-20T04:28:46.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7fbad9b2c640 1 -- 172.21.15.32:0/2017598873 wait complete. 2023-12-20T04:28:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7f466a7fc640 1 --2- 172.21.15.32:0/670576273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c0d2740 0x7f466c0cf570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f465c009870 tx=0x7f465c01ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f7284fbf9279db83 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7f4669ffb640 1 -- 172.21.15.32:0/670576273 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0a8a30 con 0x7f466c0d2740 2023-12-20T04:28:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7f4669ffb640 1 -- 172.21.15.32:0/670576273 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465c0a8b90 con 0x7f466c0d2740 2023-12-20T04:28:46.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7f4669ffb640 1 -- 172.21.15.32:0/670576273 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465c0b0440 con 0x7f466c0d2740 2023-12-20T04:28:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7f467157b640 1 -- 172.21.15.32:0/670576273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c0d2740 msgr2=0x7f466c0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.376+0000 7f467157b640 1 --2- 172.21.15.32:0/670576273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c0d2740 0x7f466c0cf570 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f465c009870 tx=0x7f465c01ed30 comp rx=0 tx=0).stop 2023-12-20T04:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f467157b640 1 -- 172.21.15.32:0/670576273 shutdown_connections 2023-12-20T04:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f467157b640 1 --2- 172.21.15.32:0/670576273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c0d2740 0x7f466c0cf570 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f467157b640 1 --2- 172.21.15.32:0/670576273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 0x7f466c0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f467157b640 1 -- 172.21.15.32:0/670576273 >> 172.21.15.32:0/670576273 conn(0x7f466c065ee0 msgr2=0x7f466c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7fbad9b2c640 1 Processor -- start 2023-12-20T04:28:46.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f2d3862d640 1 Processor -- start 2023-12-20T04:28:46.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f2d3862d640 1 -- start start 2023-12-20T04:28:46.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7fbad9b2c640 1 -- start start 2023-12-20T04:28:46.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f467157b640 1 -- 172.21.15.32:0/670576273 shutdown_connections 2023-12-20T04:28:46.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f467157b640 1 -- 172.21.15.32:0/670576273 wait complete. 2023-12-20T04:28:46.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f467157b640 1 Processor -- start 2023-12-20T04:28:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f467157b640 1 -- start start 2023-12-20T04:28:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7fbad9b2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbad40b84d0 0x7fbad41590a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f98f7fce640 1 Processor -- start 2023-12-20T04:28:46.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f2d3862d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d3015cdb0 0x7f2d300d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f2d3862d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 0x7f2d300cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f2d3862d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d300b6ca0 con 0x7f2d300d1120 2023-12-20T04:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f2d3862d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d3015d190 con 0x7f2d3015cdb0 2023-12-20T04:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f2d3862d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d300d16a0 con 0x7f2d300d2740 2023-12-20T04:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7f98f7fce640 1 -- start start 2023-12-20T04:28:46.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f2d363c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d3015cdb0 0x7f2d300d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f2d363c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2d3015cdb0 0x7f2d300d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48530/0 (socket says 172.21.15.32:48530) 2023-12-20T04:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f2d363c9640 1 -- 172.21.15.32:0/3019789162 learned_addr learned my addr 172.21.15.32:0/3019789162 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f2d35bc8640 1 --2- 172.21.15.32:0/3019789162 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 0x7f2d300cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f467157b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 0x7f466c15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f467157b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0d2740 0x7f466c0c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f467157b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c15cdb0 0x7f466c0c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f467157b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c15e920 con 0x7f466c0d2740 2023-12-20T04:28:46.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f467157b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f466c14cd40 con 0x7f466c15cdb0 2023-12-20T04:28:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f467157b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466c15eb00 con 0x7f466c0cfab0 2023-12-20T04:28:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f466a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0d2740 0x7f466c0c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f466affd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 0x7f466c15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f2d35bc8640 1 -- 172.21.15.32:0/3019789162 >> v2:172.21.15.32:3300/0 conn(0x7f2d3015cdb0 msgr2=0x7f2d300d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f2d35bc8640 1 --2- 172.21.15.32:0/3019789162 >> v2:172.21.15.32:3300/0 conn(0x7f2d3015cdb0 0x7f2d300d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f2d35bc8640 1 -- 172.21.15.32:0/3019789162 >> v1:172.21.15.32:6789/0 conn(0x7f2d300d1120 legacy=0x7f2d300b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f2d35bc8640 1 -- 172.21.15.32:0/3019789162 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d300d1980 con 0x7f2d300d2740 2023-12-20T04:28:46.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f466affd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 0x7f466c15f5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49622/0 (socket says 172.21.15.32:49622) 2023-12-20T04:28:46.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f466affd640 1 -- 172.21.15.32:0/655624445 learned_addr learned my addr 172.21.15.32:0/655624445 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7fbad9b2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad415cac0 0x7fbad40c8c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad9b2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad415cea0 0x7fbad4146810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f466b7fe640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c15cdb0 0x7f466c0c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.380+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbad40b84d0 0x7fbad41590a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad37fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbad40b84d0 0x7fbad41590a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49598/0 (socket says 172.21.15.32:49598) 2023-12-20T04:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad9b2c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad41595e0 con 0x7fbad415cac0 2023-12-20T04:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad9b2c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbad4158c10 con 0x7fbad415cea0 2023-12-20T04:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad9b2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbad4159d10 con 0x7fbad40b84d0 2023-12-20T04:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f466affd640 1 -- 172.21.15.32:0/655624445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c15cdb0 msgr2=0x7f466c0c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f466affd640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c15cdb0 0x7f466c0c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f466affd640 1 -- 172.21.15.32:0/655624445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0d2740 msgr2=0x7f466c0c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f466affd640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0d2740 0x7f466c0c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f466affd640 1 -- 172.21.15.32:0/655624445 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0094d0 con 0x7f466c0cfab0 2023-12-20T04:28:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad37fe640 1 -- 172.21.15.32:0/699548005 learned_addr learned my addr 172.21.15.32:0/699548005 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad3fff640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad415cea0 0x7fbad4146810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f98f7fce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98f0156b00 0x7f98f006c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f98f7fce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98f006c920 0x7f98f01597a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f466affd640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 0x7f466c15f5e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f466000c500 tx=0x7f466000c9c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad37fe640 1 -- 172.21.15.32:0/699548005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad415cea0 msgr2=0x7fbad4146810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad37fe640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad415cea0 0x7fbad4146810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad37fe640 1 -- 172.21.15.32:0/699548005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad415cac0 msgr2=0x7fbad40c8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad37fe640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad415cac0 0x7fbad40c8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad37fe640 1 -- 172.21.15.32:0/699548005 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbacc009a70 con 0x7fbad40b84d0 2023-12-20T04:28:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f98f5d6a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98f0156b00 0x7f98f006c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f98f5d6a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98f0156b00 0x7f98f006c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48558/0 (socket says 172.21.15.32:48558) 2023-12-20T04:28:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fbad37fe640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbad40b84d0 0x7fbad41590a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbac400c4c0 tx=0x7fbac4001e80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f46537fe640 1 -- 172.21.15.32:0/655624445 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660005220 con 0x7f466c0cfab0 2023-12-20T04:28:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f46537fe640 1 -- 172.21.15.32:0/655624445 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4660005380 con 0x7f466c0cfab0 2023-12-20T04:28:46.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f46537fe640 1 -- 172.21.15.32:0/655624445 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4660007270 con 0x7f466c0cfab0 2023-12-20T04:28:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f467157b640 1 -- 172.21.15.32:0/655624445 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f466c15ede0 con 0x7f466c0cfab0 2023-12-20T04:28:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7f467157b640 1 -- 172.21.15.32:0/655624445 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f466c0c7080 con 0x7f466c0cfab0 2023-12-20T04:28:46.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7f98f5569640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98f006c920 0x7f98f01597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fbabbfff640 1 -- 172.21.15.32:0/699548005 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4005480 con 0x7fbad40b84d0 2023-12-20T04:28:46.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fbabbfff640 1 -- 172.21.15.32:0/699548005 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbac4003600 con 0x7fbad40b84d0 2023-12-20T04:28:46.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fbabbfff640 1 -- 172.21.15.32:0/699548005 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbac4003990 con 0x7fbad40b84d0 2023-12-20T04:28:46.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fbad9b2c640 1 -- 172.21.15.32:0/699548005 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbad4158000 con 0x7fbad40b84d0 2023-12-20T04:28:46.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fbad9b2c640 1 -- 172.21.15.32:0/699548005 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbad41585c0 con 0x7fbad40b84d0 2023-12-20T04:28:46.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f46537fe640 1 -- 172.21.15.32:0/655624445 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f46600073d0 con 0x7f466c0cfab0 2023-12-20T04:28:46.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f467157b640 1 -- 172.21.15.32:0/655624445 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f466c15f300 con 0x7f466c0cfab0 2023-12-20T04:28:46.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f46537fe640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4658065b30 0x7f4658067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7fbabbfff640 1 -- 172.21.15.32:0/699548005 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbac4003af0 con 0x7fbad40b84d0 2023-12-20T04:28:46.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7fbad9b2c640 1 -- 172.21.15.32:0/699548005 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbad4074100 con 0x7fbad40b84d0 2023-12-20T04:28:46.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7f98f5569640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98f006c920 0x7f98f01597a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:49058/0 (socket says 172.21.15.32:49058) 2023-12-20T04:28:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7f98f7fce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98f015a510 con 0x7f98f0159ce0 2023-12-20T04:28:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7f98f7fce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f006cdd0 con 0x7f98f0156b00 2023-12-20T04:28:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7f98f7fce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98f00ca0f0 con 0x7f98f006c920 2023-12-20T04:28:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7f98f5d6a640 1 -- 172.21.15.32:0/491130758 learned_addr learned my addr 172.21.15.32:0/491130758 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7f98f5569640 1 -- 172.21.15.32:0/491130758 >> v2:172.21.15.32:3300/0 conn(0x7f98f0156b00 msgr2=0x7f98f006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7f98f5569640 1 --2- 172.21.15.32:0/491130758 >> v2:172.21.15.32:3300/0 conn(0x7f98f0156b00 0x7f98f006c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7f98f5569640 1 -- 172.21.15.32:0/491130758 >> v1:172.21.15.39:6789/0 conn(0x7f98f0159ce0 legacy=0x7f98f00701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7f98f5569640 1 -- 172.21.15.32:0/491130758 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f015da50 con 0x7f98f006c920 2023-12-20T04:28:46.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.384+0000 7fd76d58e640 1 Processor -- start 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76d58e640 1 -- start start 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76d58e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd7680c9140 0x7fd7680afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76d58e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7680b0530 0x7fd7680ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76d58e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd768165cc0 con 0x7fd7681614b0 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76d58e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd768165510 con 0x7fd7680c9140 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76d58e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7680b09c0 con 0x7fd7680b0530 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76cd8d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7680b0530 0x7fd7680ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76cd8d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7680b0530 0x7fd7680ccf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:49064/0 (socket says 172.21.15.32:49064) 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76cd8d640 1 -- 172.21.15.32:0/286846721 learned_addr learned my addr 172.21.15.32:0/286846721 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76cd8d640 1 -- 172.21.15.32:0/286846721 >> v2:172.21.15.32:3300/0 conn(0x7fd7680c9140 msgr2=0x7fd7680afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76cd8d640 1 --2- 172.21.15.32:0/286846721 >> v2:172.21.15.32:3300/0 conn(0x7fd7680c9140 0x7fd7680afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76cd8d640 1 -- 172.21.15.32:0/286846721 >> v1:172.21.15.32:6789/0 conn(0x7fd7681614b0 legacy=0x7fd768164e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.388+0000 7fd76cd8d640 1 -- 172.21.15.32:0/286846721 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd768165700 con 0x7fd7680b0530 2023-12-20T04:28:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f189b8fc640 1 Processor -- start 2023-12-20T04:28:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f189b8fc640 1 -- start start 2023-12-20T04:28:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f2d35bc8640 1 --2- 172.21.15.32:0/3019789162 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 0x7f2d300cf570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d200097c0 tx=0x7f2d2001ec30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b741f4bdb975af8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f46537fe640 1 -- 172.21.15.32:0/655624445 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f46600827b0 con 0x7f466c0cfab0 2023-12-20T04:28:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f2d353c7640 1 -- 172.21.15.32:0/3019789162 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d200a88f0 con 0x7f2d300d2740 2023-12-20T04:28:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f2d353c7640 1 -- 172.21.15.32:0/3019789162 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d200a8a50 con 0x7f2d300d2740 2023-12-20T04:28:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f2d353c7640 1 -- 172.21.15.32:0/3019789162 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d200a8de0 con 0x7f2d300d2740 2023-12-20T04:28:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7fbabbfff640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbac0065ab0 0x7fbac0067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f466a7fc640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4658065b30 0x7f4658067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7fbabbfff640 1 -- 172.21.15.32:0/699548005 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbac4082620 con 0x7fbad40b84d0 2023-12-20T04:28:46.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7f764a2ca640 1 Processor -- start 2023-12-20T04:28:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.392+0000 7fbad2ffd640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbac0065ab0 0x7fbac0067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f764a2ca640 1 -- start start 2023-12-20T04:28:46.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f5569640 1 --2- 172.21.15.32:0/491130758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98f006c920 0x7f98f01597a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f98f006fba0 tx=0x7f98ec0249c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6b32eeee8cfed2a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f466a7fc640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4658065b30 0x7f4658067ff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f465c0b2000 tx=0x7f465c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f4d68640 1 -- 172.21.15.32:0/491130758 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec0112d0 con 0x7f98f006c920 2023-12-20T04:28:46.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f4d68640 1 -- 172.21.15.32:0/491130758 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ec011430 con 0x7f98f006c920 2023-12-20T04:28:46.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f4d68640 1 -- 172.21.15.32:0/491130758 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ec00f850 con 0x7f98f006c920 2023-12-20T04:28:46.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f189b8fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189415cdb0 0x7f18940d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f189b8fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18940d2740 0x7f18940cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f189b8fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18940d1120 0x7f18940cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f189b8fc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189415d190 con 0x7f18940d1120 2023-12-20T04:28:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f189b8fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18940d16a0 con 0x7f18940d2740 2023-12-20T04:28:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f189b8fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18940d1980 con 0x7f189415cdb0 2023-12-20T04:28:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f1898e97640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18940d2740 0x7f18940cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7fd76cd8d640 1 --2- 172.21.15.32:0/286846721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7680b0530 0x7fd7680ccf10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd76806fc30 tx=0x7fd76001c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f639ec6cd0bded36 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f1898e97640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18940d2740 0x7f18940cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:49068/0 (socket says 172.21.15.32:49068) 2023-12-20T04:28:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f1898e97640 1 -- 172.21.15.32:0/580883364 learned_addr learned my addr 172.21.15.32:0/580883364 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f2d3862d640 1 -- 172.21.15.32:0/3019789162 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 msgr2=0x7f2d300cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f2d3862d640 1 --2- 172.21.15.32:0/3019789162 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 0x7f2d300cf570 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d200097c0 tx=0x7f2d2001ec30 comp rx=0 tx=0).stop 2023-12-20T04:28:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f1899698640 1 --2- 172.21.15.32:0/580883364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189415cdb0 0x7f18940d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f1899e99640 1 --2- 172.21.15.32:0/580883364 >> v2:172.21.15.32:3300/0 conn(0x7f18940d1120 0x7f18940cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f7fce640 1 -- 172.21.15.32:0/491130758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98f006c920 msgr2=0x7f98f01597a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f7fce640 1 --2- 172.21.15.32:0/491130758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98f006c920 0x7f98f01597a0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f98f006fba0 tx=0x7f98ec0249c0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f7fce640 1 -- 172.21.15.32:0/491130758 shutdown_connections 2023-12-20T04:28:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f7fce640 1 --2- 172.21.15.32:0/491130758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98f006c920 0x7f98f01597a0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f7fce640 1 --2- 172.21.15.32:0/491130758 >> v2:172.21.15.32:3300/0 conn(0x7f98f0156b00 0x7f98f006c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f7fce640 1 -- 172.21.15.32:0/491130758 >> 172.21.15.32:0/491130758 conn(0x7f98f00cb1e0 msgr2=0x7f98f00cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7fd766ffd640 1 -- 172.21.15.32:0/286846721 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7600a5790 con 0x7fd7680b0530 2023-12-20T04:28:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7fd766ffd640 1 -- 172.21.15.32:0/286846721 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7600a58f0 con 0x7fd7680b0530 2023-12-20T04:28:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7fd766ffd640 1 -- 172.21.15.32:0/286846721 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7600a5c80 con 0x7fd7680b0530 2023-12-20T04:28:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f1898e97640 1 -- 172.21.15.32:0/580883364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189415cdb0 msgr2=0x7f18940d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f1898e97640 1 --2- 172.21.15.32:0/580883364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189415cdb0 0x7f18940d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f1898e97640 1 -- 172.21.15.32:0/580883364 >> v2:172.21.15.32:3300/0 conn(0x7f18940d1120 msgr2=0x7f18940cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f1898e97640 1 --2- 172.21.15.32:0/580883364 >> v2:172.21.15.32:3300/0 conn(0x7f18940d1120 0x7f18940cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f1898e97640 1 -- 172.21.15.32:0/580883364 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18940d1c60 con 0x7f18940d2740 2023-12-20T04:28:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f2d3862d640 1 -- 172.21.15.32:0/3019789162 shutdown_connections 2023-12-20T04:28:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f2d3862d640 1 --2- 172.21.15.32:0/3019789162 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 0x7f2d300cf570 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f2d3862d640 1 --2- 172.21.15.32:0/3019789162 >> v2:172.21.15.32:3300/0 conn(0x7f2d3015cdb0 0x7f2d300d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f2d3862d640 1 -- 172.21.15.32:0/3019789162 >> 172.21.15.32:0/3019789162 conn(0x7f2d30065ee0 msgr2=0x7f2d3006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f764a2ca640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 0x7f76440ad7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f764a2ca640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76440afff0 0x7f76440b0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f764a2ca640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f764415cd30 con 0x7f76440c8c30 2023-12-20T04:28:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f764a2ca640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f764415d6d0 con 0x7f76440afff0 2023-12-20T04:28:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f764a2ca640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76440b09a0 con 0x7f76440cbb10 2023-12-20T04:28:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f7fce640 1 -- 172.21.15.32:0/491130758 shutdown_connections 2023-12-20T04:28:46.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f98f7fce640 1 -- 172.21.15.32:0/491130758 wait complete. 2023-12-20T04:28:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7fbad2ffd640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbac0065ab0 0x7fbac0067f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbac8006f50 tx=0x7fbac8006ea0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f7648ac7640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f76440c8c30 0x7f7644158a60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35456/0 (socket says 172.21.15.32:35456) 2023-12-20T04:28:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f7648ac7640 1 -- 172.21.15.32:0/4046705576 learned_addr learned my addr 172.21.15.32:0/4046705576 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f76492c8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 0x7f76440ad7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f763bfff640 1 -- 172.21.15.32:0/4046705576 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2792659856 0 0) 0x7f764415cd30 con 0x7f76440c8c30 2023-12-20T04:28:46.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7f763bfff640 1 -- 172.21.15.32:0/4046705576 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7630003530 con 0x7f76440c8c30 2023-12-20T04:28:46.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7fd76d58e640 1 -- 172.21.15.32:0/286846721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7680b0530 msgr2=0x7fd7680ccf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.396+0000 7fd76d58e640 1 --2- 172.21.15.32:0/286846721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7680b0530 0x7fd7680ccf10 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fd76806fc30 tx=0x7fd76001c8d0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f2d3862d640 1 -- 172.21.15.32:0/3019789162 shutdown_connections 2023-12-20T04:28:46.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f76492c8640 1 -- 172.21.15.32:0/4046705576 >> v2:172.21.15.32:3300/0 conn(0x7f76440afff0 msgr2=0x7f76440b0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:46.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f76492c8640 1 --2- 172.21.15.32:0/4046705576 >> v2:172.21.15.32:3300/0 conn(0x7f76440afff0 0x7f76440b0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f76492c8640 1 -- 172.21.15.32:0/4046705576 >> v1:172.21.15.32:6789/0 conn(0x7f76440c8c30 legacy=0x7f7644158a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f76492c8640 1 -- 172.21.15.32:0/4046705576 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f764415d010 con 0x7f76440cbb10 2023-12-20T04:28:46.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7fd76d58e640 1 -- 172.21.15.32:0/286846721 shutdown_connections 2023-12-20T04:28:46.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7fd76d58e640 1 --2- 172.21.15.32:0/286846721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7680b0530 0x7fd7680ccf10 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7fd76d58e640 1 --2- 172.21.15.32:0/286846721 >> v2:172.21.15.32:3300/0 conn(0x7fd7680c9140 0x7fd7680afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7fd76d58e640 1 -- 172.21.15.32:0/286846721 >> 172.21.15.32:0/286846721 conn(0x7fd7680caff0 msgr2=0x7fd7680cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7fd76d58e640 1 -- 172.21.15.32:0/286846721 shutdown_connections 2023-12-20T04:28:46.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f46537fe640 1 -- 172.21.15.32:0/655624445 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f466c15f300 con 0x7f466c0cfab0 2023-12-20T04:28:46.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7fd76d58e640 1 -- 172.21.15.32:0/286846721 wait complete. 2023-12-20T04:28:46.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f2d3862d640 1 -- 172.21.15.32:0/3019789162 wait complete. 2023-12-20T04:28:46.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f98f7fce640 1 Processor -- start 2023-12-20T04:28:46.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f98f7fce640 1 -- start start 2023-12-20T04:28:46.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f1898e97640 1 --2- 172.21.15.32:0/580883364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940d2740 0x7f18940cf570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f18940d3c60 tx=0x7f188c01ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=986d530201166c79 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7fd76d58e640 1 Processor -- start 2023-12-20T04:28:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f1893fff640 1 -- 172.21.15.32:0/580883364 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0a89b0 con 0x7f18940d2740 2023-12-20T04:28:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f2d3862d640 1 Processor -- start 2023-12-20T04:28:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.400+0000 7f1893fff640 1 -- 172.21.15.32:0/580883364 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c0a8b10 con 0x7f18940d2740 2023-12-20T04:28:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd76d58e640 1 -- start start 2023-12-20T04:28:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f2d3862d640 1 -- start start 2023-12-20T04:28:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f189b8fc640 1 -- 172.21.15.32:0/580883364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940d2740 msgr2=0x7f18940cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f189b8fc640 1 --2- 172.21.15.32:0/580883364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940d2740 0x7f18940cf570 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f18940d3c60 tx=0x7f188c01ecf0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f7fce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f006c920 0x7f98f00b0560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f7fce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98f0156b00 0x7f98f0148170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f7fce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98f0159ce0 0x7f98f01486b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f5d6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f006c920 0x7f98f00b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f5d6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f006c920 0x7f98f00b0560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48604/0 (socket says 172.21.15.32:48604) 2023-12-20T04:28:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f656b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98f0159ce0 0x7f98f01486b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f656b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98f0159ce0 0x7f98f01486b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:49078/0 (socket says 172.21.15.32:49078) 2023-12-20T04:28:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f5569640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98f0156b00 0x7f98f0148170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f5569640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98f0156b00 0x7f98f0148170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49656/0 (socket says 172.21.15.32:49656) 2023-12-20T04:28:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fbabbfff640 1 -- 172.21.15.32:0/699548005 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbac4050100 con 0x7fbad40b84d0 2023-12-20T04:28:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f189b8fc640 1 -- 172.21.15.32:0/580883364 shutdown_connections 2023-12-20T04:28:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f189b8fc640 1 --2- 172.21.15.32:0/580883364 >> v2:172.21.15.32:3300/0 conn(0x7f18940d1120 0x7f18940cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f189b8fc640 1 --2- 172.21.15.32:0/580883364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18940d2740 0x7f18940cf570 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f189b8fc640 1 --2- 172.21.15.32:0/580883364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f189415cdb0 0x7f18940d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd76d58e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7680b0530 0x7fd768202e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f189b8fc640 1 -- 172.21.15.32:0/580883364 >> 172.21.15.32:0/580883364 conn(0x7f1894065ee0 msgr2=0x7f189406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd76d58e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7680c9140 0x7fd7681634f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f189b8fc640 1 -- 172.21.15.32:0/580883364 shutdown_connections 2023-12-20T04:28:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd76d58e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7681614b0 0x7fd7682049d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd76d58e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd768163a30 con 0x7fd7680c9140 2023-12-20T04:28:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd76d58e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680b1780 con 0x7fd7680b0530 2023-12-20T04:28:46.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd76d58e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7680b1a60 con 0x7fd7681614b0 2023-12-20T04:28:46.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f189b8fc640 1 -- 172.21.15.32:0/580883364 wait complete. 2023-12-20T04:28:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f7fce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f015bd00 con 0x7f98f006c920 2023-12-20T04:28:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f7fce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98f015b0f0 con 0x7f98f0159ce0 2023-12-20T04:28:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f7fce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98f015b3d0 con 0x7f98f0156b00 2023-12-20T04:28:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f98f5d6a640 1 -- 172.21.15.32:0/4043444511 learned_addr learned my addr 172.21.15.32:0/4043444511 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f98f656b640 1 -- 172.21.15.32:0/4043444511 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98f0156b00 msgr2=0x7f98f0148170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f98f656b640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98f0156b00 0x7f98f0148170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f98f656b640 1 -- 172.21.15.32:0/4043444511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f006c920 msgr2=0x7f98f00b0560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f98f656b640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f006c920 0x7f98f00b0560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f98f656b640 1 -- 172.21.15.32:0/4043444511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98ec011cc0 con 0x7f98f0159ce0 2023-12-20T04:28:46.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f98f5569640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98f0156b00 0x7f98f0148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:46.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f98f5d6a640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f006c920 0x7f98f00b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:28:46.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f189b8fc640 1 Processor -- start 2023-12-20T04:28:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f2d3862d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d300d1120 0x7f2d3015f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7f2d3862d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 0x7f2d3015b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d3862d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3015cdb0 0x7f2d3015bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d3862d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3014fce0 con 0x7f2d300d1120 2023-12-20T04:28:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d3862d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3015f8d0 con 0x7f2d3015cdb0 2023-12-20T04:28:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d3862d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d301ee210 con 0x7f2d300d2740 2023-12-20T04:28:46.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d35bc8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 0x7f2d3015b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d35bc8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 0x7f2d3015b6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49672/0 (socket says 172.21.15.32:49672) 2023-12-20T04:28:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d35bc8640 1 -- 172.21.15.32:0/1340876419 learned_addr learned my addr 172.21.15.32:0/1340876419 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d36bca640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3015cdb0 0x7f2d3015bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d363c9640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d300d1120 0x7f2d3015f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d35bc8640 1 -- 172.21.15.32:0/1340876419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3015cdb0 msgr2=0x7f2d3015bc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d35bc8640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3015cdb0 0x7f2d3015bc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d35bc8640 1 -- 172.21.15.32:0/1340876419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d300d1120 msgr2=0x7f2d3015f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f76492c8640 1 --2- 172.21.15.32:0/4046705576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 0x7f76440ad7d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f76400097c0 tx=0x7f764001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1ae2981d39576e42 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f763bfff640 1 -- 172.21.15.32:0/4046705576 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76400ab600 con 0x7f76440cbb10 2023-12-20T04:28:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f763bfff640 1 -- 172.21.15.32:0/4046705576 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76400a7050 con 0x7f76440cbb10 2023-12-20T04:28:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f763bfff640 1 -- 172.21.15.32:0/4046705576 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76400ab980 con 0x7f76440cbb10 2023-12-20T04:28:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f764a2ca640 1 -- 172.21.15.32:0/4046705576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 msgr2=0x7f76440ad7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f764a2ca640 1 --2- 172.21.15.32:0/4046705576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 0x7f76440ad7d0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f76400097c0 tx=0x7f764001c3e0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d35bc8640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d300d1120 0x7f2d3015f390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d35bc8640 1 -- 172.21.15.32:0/1340876419 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d20009420 con 0x7f2d300d2740 2023-12-20T04:28:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd767fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7680b0530 0x7fd768202e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd767fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7680b0530 0x7fd768202e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:49088/0 (socket says 172.21.15.32:49088) 2023-12-20T04:28:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd767fff640 1 -- 172.21.15.32:0/1664252233 learned_addr learned my addr 172.21.15.32:0/1664252233 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd767fff640 1 -- 172.21.15.32:0/1664252233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7681614b0 msgr2=0x7fd7682049d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-20T04:28:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd767fff640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7681614b0 0x7fd7682049d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd767fff640 1 -- 172.21.15.32:0/1664252233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7680c9140 msgr2=0x7fd7681634f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd767fff640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7680c9140 0x7fd7681634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.404+0000 7fd767fff640 1 -- 172.21.15.32:0/1664252233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd760009b30 con 0x7fd7680b0530 2023-12-20T04:28:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7fd767fff640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7680b0530 0x7fd768202e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b3e0 tx=0x7fd75800b8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7fd764ff9640 1 -- 172.21.15.32:0/1664252233 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd758011830 con 0x7fd7680b0530 2023-12-20T04:28:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7fd764ff9640 1 -- 172.21.15.32:0/1664252233 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7580119b0 con 0x7fd7680b0530 2023-12-20T04:28:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7fd764ff9640 1 -- 172.21.15.32:0/1664252233 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd758011d60 con 0x7fd7680b0530 2023-12-20T04:28:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d35bc8640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 0x7f2d3015b6e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d2001d2e0 tx=0x7f2d2001b9d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f98f656b640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98f0159ce0 0x7f98f01486b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f98e800c780 tx=0x7f98e800cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d26ffd640 1 -- 172.21.15.32:0/1340876419 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d200bd070 con 0x7f2d300d2740 2023-12-20T04:28:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f764a2ca640 1 -- 172.21.15.32:0/4046705576 shutdown_connections 2023-12-20T04:28:46.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f764a2ca640 1 --2- 172.21.15.32:0/4046705576 >> v2:172.21.15.32:3300/0 conn(0x7f76440afff0 0x7f76440b0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f764a2ca640 1 --2- 172.21.15.32:0/4046705576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 0x7f76440ad7d0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f764a2ca640 1 -- 172.21.15.32:0/4046705576 >> 172.21.15.32:0/4046705576 conn(0x7f76440caff0 msgr2=0x7f76440cb400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d26ffd640 1 -- 172.21.15.32:0/1340876419 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d200b07d0 con 0x7f2d300d2740 2023-12-20T04:28:46.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d26ffd640 1 -- 172.21.15.32:0/1340876419 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d200b0b60 con 0x7f2d300d2740 2023-12-20T04:28:46.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d3862d640 1 -- 172.21.15.32:0/1340876419 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d30156df0 con 0x7f2d300d2740 2023-12-20T04:28:46.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f189b8fc640 1 -- start start 2023-12-20T04:28:46.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7fd76d58e640 1 -- 172.21.15.32:0/1664252233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7680b1d40 con 0x7fd7680b0530 2023-12-20T04:28:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7fd76d58e640 1 -- 172.21.15.32:0/1664252233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7680b2300 con 0x7fd7680b0530 2023-12-20T04:28:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.408+0000 7f2d3862d640 1 -- 172.21.15.32:0/1340876419 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d301573b0 con 0x7f2d300d2740 2023-12-20T04:28:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f98e67fc640 1 -- 172.21.15.32:0/4043444511 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e800aa50 con 0x7f98f0159ce0 2023-12-20T04:28:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f98e67fc640 1 -- 172.21.15.32:0/4043444511 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e800abb0 con 0x7f98f0159ce0 2023-12-20T04:28:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f98e67fc640 1 -- 172.21.15.32:0/4043444511 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e800ce30 con 0x7f98f0159ce0 2023-12-20T04:28:46.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f189b8fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18940d1120 0x7f189414bde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7fd76d58e640 1 -- 172.21.15.32:0/1664252233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd768074100 con 0x7fd7680b0530 2023-12-20T04:28:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f98f7fce640 1 -- 172.21.15.32:0/4043444511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f015b6b0 con 0x7f98f0159ce0 2023-12-20T04:28:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f98f7fce640 1 -- 172.21.15.32:0/4043444511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98f015adf0 con 0x7f98f0159ce0 2023-12-20T04:28:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f189b8fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940d2740 0x7f18940cbdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f189b8fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f189415cdb0 0x7f18940cd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f189b8fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18941fa050 con 0x7f18940d2740 2023-12-20T04:28:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1898e97640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940d2740 0x7f18940cbdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f189b8fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18940cc850 con 0x7f189415cdb0 2023-12-20T04:28:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f189b8fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18941fa230 con 0x7f18940d1120 2023-12-20T04:28:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1898e97640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940d2740 0x7f18940cbdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48630/0 (socket says 172.21.15.32:48630) 2023-12-20T04:28:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1898e97640 1 -- 172.21.15.32:0/3939516350 learned_addr learned my addr 172.21.15.32:0/3939516350 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f2d3862d640 1 -- 172.21.15.32:0/1340876419 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d301ee430 con 0x7f2d300d2740 2023-12-20T04:28:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f764a2ca640 1 -- 172.21.15.32:0/4046705576 shutdown_connections 2023-12-20T04:28:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7fd764ff9640 1 -- 172.21.15.32:0/1664252233 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd75801a400 con 0x7fd7680b0530 2023-12-20T04:28:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1898e97640 1 -- 172.21.15.32:0/3939516350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18940d1120 msgr2=0x7f189414bde0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2023-12-20T04:28:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1898e97640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18940d1120 0x7f189414bde0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1898e97640 1 -- 172.21.15.32:0/3939516350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f189415cdb0 msgr2=0x7f18940cd5c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:28:46.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1898e97640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f189415cdb0 0x7f18940cd5c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1898e97640 1 -- 172.21.15.32:0/3939516350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c0094d0 con 0x7f18940d2740 2023-12-20T04:28:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f98f7fce640 1 -- 172.21.15.32:0/4043444511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98f015be70 con 0x7f98f0159ce0 2023-12-20T04:28:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1898e97640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940d2740 0x7f18940cbdb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f188c01f590 tx=0x7f188c01ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1891ffb640 1 -- 172.21.15.32:0/3939516350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0b0d00 con 0x7f18940d2740 2023-12-20T04:28:46.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1891ffb640 1 -- 172.21.15.32:0/3939516350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188c0b0e60 con 0x7f18940d2740 2023-12-20T04:28:46.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f1891ffb640 1 -- 172.21.15.32:0/3939516350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188c0ba600 con 0x7f18940d2740 2023-12-20T04:28:46.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.412+0000 7f764a2ca640 1 -- 172.21.15.32:0/4046705576 wait complete. 2023-12-20T04:28:46.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f2d26ffd640 1 -- 172.21.15.32:0/1340876419 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2d200b7040 con 0x7f2d300d2740 2023-12-20T04:28:46.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f189b8fc640 1 -- 172.21.15.32:0/3939516350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18941fa510 con 0x7f18940d2740 2023-12-20T04:28:46.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7fd764ff9640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd754065ab0 0x7fd754067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7fd764ff9640 1 -- 172.21.15.32:0/1664252233 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd758083270 con 0x7fd7680b0530 2023-12-20T04:28:46.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f189b8fc640 1 -- 172.21.15.32:0/3939516350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1894150ff0 con 0x7f18940d2740 2023-12-20T04:28:46.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7fd7677fe640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd754065ab0 0x7fd754067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f2d26ffd640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d14065a60 0x7f2d14067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f2d26ffd640 1 -- 172.21.15.32:0/1340876419 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d20120110 con 0x7f2d300d2740 2023-12-20T04:28:46.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f764a2ca640 1 Processor -- start 2023-12-20T04:28:46.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7fd7677fe640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd754065ab0 0x7fd754067f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd768152670 tx=0x7fd75c006ca0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f2d363c9640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d14065a60 0x7f2d14067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f187b7fe640 1 -- 172.21.15.32:0/3939516350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f189404e000 con 0x7f18940d2740 2023-12-20T04:28:46.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f2d363c9640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d14065a60 0x7f2d14067f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009ce0 tx=0x7f2d2800b040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f764a2ca640 1 -- start start 2023-12-20T04:28:46.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f764a2ca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76440afff0 0x7f76441458f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f764a2ca640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76440c8c30 0x7f76440c4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f764a2ca640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 0x7f76440c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f764a2ca640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76440c6eb0 con 0x7f76440afff0 2023-12-20T04:28:46.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f764a2ca640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76440c6050 con 0x7f76440c8c30 2023-12-20T04:28:46.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.416+0000 7f764a2ca640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7644145e30 con 0x7f76440cbb10 2023-12-20T04:28:46.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7649ac9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 0x7f76440c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7649ac9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 0x7f76440c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49690/0 (socket says 172.21.15.32:49690) 2023-12-20T04:28:46.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7649ac9640 1 -- 172.21.15.32:0/107957815 learned_addr learned my addr 172.21.15.32:0/107957815 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:46.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f76492c8640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76440afff0 0x7f76441458f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7fd764ff9640 1 -- 172.21.15.32:0/1664252233 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd758050c10 con 0x7fd7680b0530 2023-12-20T04:28:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7648ac7640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76440c8c30 0x7f76440c4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7649ac9640 1 -- 172.21.15.32:0/107957815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76440c8c30 msgr2=0x7f76440c4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7649ac9640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76440c8c30 0x7f76440c4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7649ac9640 1 -- 172.21.15.32:0/107957815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76440afff0 msgr2=0x7f76441458f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7649ac9640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76440afff0 0x7f76441458f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7649ac9640 1 -- 172.21.15.32:0/107957815 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7640009420 con 0x7f76440cbb10 2023-12-20T04:28:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f76492c8640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76440afff0 0x7f76441458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:28:46.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7648ac7640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76440c8c30 0x7f76440c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:28:46.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f1891ffb640 1 -- 172.21.15.32:0/3939516350 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f188c0ba760 con 0x7f18940d2740 2023-12-20T04:28:46.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7649ac9640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 0x7f76440c4b70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f763c00b970 tx=0x7f763c00be30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f2d26ffd640 1 -- 172.21.15.32:0/1340876419 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2d301ee430 con 0x7f2d300d2740 2023-12-20T04:28:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7639ffb640 1 -- 172.21.15.32:0/107957815 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c00cb40 con 0x7f76440cbb10 2023-12-20T04:28:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7639ffb640 1 -- 172.21.15.32:0/107957815 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f763c00ccc0 con 0x7f76440cbb10 2023-12-20T04:28:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f7639ffb640 1 -- 172.21.15.32:0/107957815 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f763c014680 con 0x7f76440cbb10 2023-12-20T04:28:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f764a2ca640 1 -- 172.21.15.32:0/107957815 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76440c62a0 con 0x7f76440cbb10 2023-12-20T04:28:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.424+0000 7f764a2ca640 1 -- 172.21.15.32:0/107957815 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76440c6860 con 0x7f76440cbb10 2023-12-20T04:28:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.420+0000 7f98e67fc640 1 -- 172.21.15.32:0/4043444511 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f98e8025480 con 0x7f98f0159ce0 2023-12-20T04:28:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.424+0000 7f98e67fc640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98d4065b60 0x7f98d4068020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.424+0000 7f98e67fc640 1 -- 172.21.15.32:0/4043444511 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98e8083a30 con 0x7f98f0159ce0 2023-12-20T04:28:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.424+0000 7f98e67fc640 1 -- 172.21.15.32:0/4043444511 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f98e80513d0 con 0x7f98f0159ce0 2023-12-20T04:28:46.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.428+0000 7f98f5d6a640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98d4065b60 0x7f98d4068020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.424+0000 7f1891ffb640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1880065b30 0x7f1880067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.424+0000 7f1891ffb640 1 -- 172.21.15.32:0/3939516350 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f188c120300 con 0x7f18940d2740 2023-12-20T04:28:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.428+0000 7f1899698640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1880065b30 0x7f1880067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.424+0000 7f764a2ca640 1 -- 172.21.15.32:0/107957815 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7644074100 con 0x7f76440cbb10 2023-12-20T04:28:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.424+0000 7f7639ffb640 1 -- 172.21.15.32:0/107957815 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f763c01d3b0 con 0x7f76440cbb10 2023-12-20T04:28:46.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.428+0000 7f7639ffb640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7628065b30 0x7f7628067ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.428+0000 7f7639ffb640 1 -- 172.21.15.32:0/107957815 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f763c0830c0 con 0x7f76440cbb10 2023-12-20T04:28:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.428+0000 7f76492c8640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7628065b30 0x7f7628067ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.428+0000 7f1899698640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1880065b30 0x7f1880067ff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1884009fa0 tx=0x7f188400d040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.432+0000 7f1891ffb640 1 -- 172.21.15.32:0/3939516350 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f188c0edd30 con 0x7f18940d2740 2023-12-20T04:28:46.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.432+0000 7f98f5d6a640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98d4065b60 0x7f98d4068020 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f98e0006aa0 tx=0x7f98e0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.432+0000 7f76492c8640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7628065b30 0x7f7628067ff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f76400097c0 tx=0x7f76400a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:46.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.432+0000 7f7639ffb640 1 -- 172.21.15.32:0/107957815 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f763c019080 con 0x7f76440cbb10 2023-12-20T04:28:46.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.676+0000 7f6c7affd640 1 -- 172.21.15.32:0/2498994660 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f6c6c0057e0 con 0x7f6c940ca600 2023-12-20T04:28:46.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.688+0000 7f6c917fa640 1 -- 172.21.15.32:0/2498994660 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f6c880ba090 con 0x7f6c940ca600 2023-12-20T04:28:46.688 INFO:teuthology.orchestra.run.smithi032.stdout:55834574853 2023-12-20T04:28:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.688+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/2498994660 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c80065a60 msgr2=0x7f6c80067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.688+0000 7f6c9aa8c640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c80065a60 0x7f6c80067f20 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f6c940c9620 tx=0x7f6c84009060 comp rx=0 tx=0).stop 2023-12-20T04:28:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.692+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/2498994660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ca600 msgr2=0x7f6c941585b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.692+0000 7f6c9aa8c640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ca600 0x7f6c941585b0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f6c88000c10 tx=0x7f6c88021a40 comp rx=0 tx=0).stop 2023-12-20T04:28:46.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.692+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/2498994660 shutdown_connections 2023-12-20T04:28:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.692+0000 7f6c9aa8c640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c80065a60 0x7f6c80067f20 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.692+0000 7f6c9aa8c640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c9415cfb0 0x7f6c94159e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.692+0000 7f6c9aa8c640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c9415cbd0 0x7f6c941598f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.692+0000 7f6c9aa8c640 1 --2- 172.21.15.32:0/2498994660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c940ca600 0x7f6c941585b0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.692+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/2498994660 >> 172.21.15.32:0/2498994660 conn(0x7f6c94065ee0 msgr2=0x7f6c940cb090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.692+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/2498994660 shutdown_connections 2023-12-20T04:28:46.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.692+0000 7f6c9aa8c640 1 -- 172.21.15.32:0/2498994660 wait complete. 2023-12-20T04:28:46.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbad9b2c640 1 -- 172.21.15.32:0/699548005 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) v1 -- 0x7fbad40d07a0 con 0x7fbad40b84d0 2023-12-20T04:28:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbabbfff640 1 -- 172.21.15.32:0/699548005 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fbac4017090 con 0x7fbad40b84d0 2023-12-20T04:28:46.728 INFO:teuthology.orchestra.run.smithi032.stdout:55834574856 2023-12-20T04:28:46.730 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574853 for osd.0 2023-12-20T04:28:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbad9b2c640 1 -- 172.21.15.32:0/699548005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbac0065ab0 msgr2=0x7fbac0067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbad9b2c640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbac0065ab0 0x7fbac0067f70 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbac8006f50 tx=0x7fbac8006ea0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbad9b2c640 1 -- 172.21.15.32:0/699548005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbad40b84d0 msgr2=0x7fbad41590a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbad9b2c640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbad40b84d0 0x7fbad41590a0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbac400c4c0 tx=0x7fbac4001e80 comp rx=0 tx=0).stop 2023-12-20T04:28:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbad9b2c640 1 -- 172.21.15.32:0/699548005 shutdown_connections 2023-12-20T04:28:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbad9b2c640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbac0065ab0 0x7fbac0067f70 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbad9b2c640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbad415cea0 0x7fbad4146810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbad9b2c640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbad415cac0 0x7fbad40c8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbad9b2c640 1 --2- 172.21.15.32:0/699548005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbad40b84d0 0x7fbad41590a0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.728+0000 7fbad9b2c640 1 -- 172.21.15.32:0/699548005 >> 172.21.15.32:0/699548005 conn(0x7fbad406afd0 msgr2=0x7fbad4156b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.732+0000 7fbad9b2c640 1 -- 172.21.15.32:0/699548005 shutdown_connections 2023-12-20T04:28:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.732+0000 7fbad9b2c640 1 -- 172.21.15.32:0/699548005 wait complete. 2023-12-20T04:28:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.740+0000 7f467157b640 1 -- 172.21.15.32:0/655624445 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) v1 -- 0x7f466c0cff00 con 0x7f466c0cfab0 2023-12-20T04:28:46.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.740+0000 7f46537fe640 1 -- 172.21.15.32:0/655624445 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f466c0cff00 con 0x7f466c0cfab0 2023-12-20T04:28:46.743 INFO:teuthology.orchestra.run.smithi032.stdout:55834574856 2023-12-20T04:28:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.744+0000 7f467157b640 1 -- 172.21.15.32:0/655624445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4658065b30 msgr2=0x7f4658067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.744+0000 7f467157b640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4658065b30 0x7f4658067ff0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f465c0b2000 tx=0x7f465c009040 comp rx=0 tx=0).stop 2023-12-20T04:28:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.744+0000 7f467157b640 1 -- 172.21.15.32:0/655624445 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 msgr2=0x7f466c15f5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.744+0000 7f467157b640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 0x7f466c15f5e0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f466000c500 tx=0x7f466000c9c0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.744+0000 7f467157b640 1 -- 172.21.15.32:0/655624445 shutdown_connections 2023-12-20T04:28:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.744+0000 7f467157b640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4658065b30 0x7f4658067ff0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.744+0000 7f467157b640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f466c15cdb0 0x7f466c0c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.744+0000 7f467157b640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f466c0d2740 0x7f466c0c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.744+0000 7f467157b640 1 --2- 172.21.15.32:0/655624445 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f466c0cfab0 0x7f466c15f5e0 secure :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f466000c500 tx=0x7f466000c9c0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.744+0000 7f467157b640 1 -- 172.21.15.32:0/655624445 >> 172.21.15.32:0/655624445 conn(0x7f466c065ee0 msgr2=0x7f466c1532d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.748+0000 7f467157b640 1 -- 172.21.15.32:0/655624445 shutdown_connections 2023-12-20T04:28:46.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.748+0000 7f467157b640 1 -- 172.21.15.32:0/655624445 wait complete. 2023-12-20T04:28:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.756+0000 7f98f7fce640 1 -- 172.21.15.32:0/4043444511 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) v1 -- 0x7f98f004e000 con 0x7f98f0159ce0 2023-12-20T04:28:46.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.756+0000 7f98e67fc640 1 -- 172.21.15.32:0/4043444511 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f98e8016090 con 0x7f98f0159ce0 2023-12-20T04:28:46.757 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-12-20T04:28:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.756+0000 7fd76d58e640 1 -- 172.21.15.32:0/1664252233 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) v1 -- 0x7fd7680cc450 con 0x7fd7680b0530 2023-12-20T04:28:46.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.756+0000 7fd764ff9640 1 -- 172.21.15.32:0/1664252233 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7fd758054230 con 0x7fd7680b0530 2023-12-20T04:28:46.761 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-12-20T04:28:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.756+0000 7f2d3862d640 1 -- 172.21.15.32:0/1340876419 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) v1 -- 0x7f2d300b6ca0 con 0x7f2d300d2740 2023-12-20T04:28:46.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d26ffd640 1 -- 172.21.15.32:0/1340876419 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f2d200b5020 con 0x7f2d300d2740 2023-12-20T04:28:46.763 INFO:teuthology.orchestra.run.smithi032.stdout:55834574855 2023-12-20T04:28:46.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 -- 172.21.15.32:0/4043444511 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98d4065b60 msgr2=0x7f98d4068020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98d4065b60 0x7f98d4068020 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f98e0006aa0 tx=0x7f98e0008040 comp rx=0 tx=0).stop 2023-12-20T04:28:46.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 -- 172.21.15.32:0/4043444511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98f0159ce0 msgr2=0x7f98f01486b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98f0159ce0 0x7f98f01486b0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f98e800c780 tx=0x7f98e800cc40 comp rx=0 tx=0).stop 2023-12-20T04:28:46.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 -- 172.21.15.32:0/4043444511 shutdown_connections 2023-12-20T04:28:46.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98d4065b60 0x7f98d4068020 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98f0159ce0 0x7f98f01486b0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98f0156b00 0x7f98f0148170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 --2- 172.21.15.32:0/4043444511 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98f006c920 0x7f98f00b0560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 -- 172.21.15.32:0/4043444511 >> 172.21.15.32:0/4043444511 conn(0x7f98f00cb1e0 msgr2=0x7f98f00cde90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.770 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.4 2023-12-20T04:28:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 -- 172.21.15.32:0/4043444511 shutdown_connections 2023-12-20T04:28:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f98f7fce640 1 -- 172.21.15.32:0/4043444511 wait complete. 2023-12-20T04:28:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 -- 172.21.15.32:0/1664252233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd754065ab0 msgr2=0x7fd754067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd754065ab0 0x7fd754067f70 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fd768152670 tx=0x7fd75c006ca0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 -- 172.21.15.32:0/1664252233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7680b0530 msgr2=0x7fd768202e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7680b0530 0x7fd768202e60 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fd75800b3e0 tx=0x7fd75800b8a0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.771 DEBUG:teuthology.parallel:result is None 2023-12-20T04:28:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 -- 172.21.15.32:0/1340876419 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d14065a60 msgr2=0x7f2d14067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d14065a60 0x7f2d14067f20 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f2d28009ce0 tx=0x7f2d2800b040 comp rx=0 tx=0).stop 2023-12-20T04:28:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 -- 172.21.15.32:0/1664252233 shutdown_connections 2023-12-20T04:28:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd754065ab0 0x7fd754067f70 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7681614b0 0x7fd7682049d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7680c9140 0x7fd7681634f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 --2- 172.21.15.32:0/1664252233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7680b0530 0x7fd768202e60 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 -- 172.21.15.32:0/1664252233 >> 172.21.15.32:0/1664252233 conn(0x7fd7680caff0 msgr2=0x7fd768152bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 -- 172.21.15.32:0/1340876419 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 msgr2=0x7f2d3015b6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 0x7f2d3015b6e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2d2001d2e0 tx=0x7f2d2001b9d0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 -- 172.21.15.32:0/1664252233 shutdown_connections 2023-12-20T04:28:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 -- 172.21.15.32:0/1340876419 shutdown_connections 2023-12-20T04:28:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d14065a60 0x7f2d14067f20 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d3015cdb0 0x7f2d3015bc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d300d2740 0x7f2d3015b6e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 --2- 172.21.15.32:0/1340876419 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d300d1120 0x7f2d3015f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 -- 172.21.15.32:0/1340876419 >> 172.21.15.32:0/1340876419 conn(0x7f2d30065ee0 msgr2=0x7f2d300cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 -- 172.21.15.32:0/1340876419 shutdown_connections 2023-12-20T04:28:46.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7f2d3862d640 1 -- 172.21.15.32:0/1340876419 wait complete. 2023-12-20T04:28:46.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.760+0000 7fd76d58e640 1 -- 172.21.15.32:0/1664252233 wait complete. 2023-12-20T04:28:46.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.764+0000 7f187b7fe640 1 -- 172.21.15.32:0/3939516350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) v1 -- 0x7f18940b36d0 con 0x7f18940d2740 2023-12-20T04:28:46.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.764+0000 7f1891ffb640 1 -- 172.21.15.32:0/3939516350 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f188c0f1350 con 0x7f18940d2740 2023-12-20T04:28:46.776 INFO:teuthology.orchestra.run.smithi032.stdout:55834574855 2023-12-20T04:28:46.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 -- 172.21.15.32:0/3939516350 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1880065b30 msgr2=0x7f1880067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1880065b30 0x7f1880067ff0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f1884009fa0 tx=0x7f188400d040 comp rx=0 tx=0).stop 2023-12-20T04:28:46.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 -- 172.21.15.32:0/3939516350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940d2740 msgr2=0x7f18940cbdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940d2740 0x7f18940cbdb0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f188c01f590 tx=0x7f188c01ba90 comp rx=0 tx=0).stop 2023-12-20T04:28:46.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 -- 172.21.15.32:0/3939516350 shutdown_connections 2023-12-20T04:28:46.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1880065b30 0x7f1880067ff0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f189415cdb0 0x7f18940cd5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18940d2740 0x7f18940cbdb0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 --2- 172.21.15.32:0/3939516350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18940d1120 0x7f189414bde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 -- 172.21.15.32:0/3939516350 >> 172.21.15.32:0/3939516350 conn(0x7f1894065ee0 msgr2=0x7f189415c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 -- 172.21.15.32:0/3939516350 shutdown_connections 2023-12-20T04:28:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.768+0000 7f187b7fe640 1 -- 172.21.15.32:0/3939516350 wait complete. 2023-12-20T04:28:46.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.776+0000 7f764a2ca640 1 -- 172.21.15.32:0/107957815 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) v1 -- 0x7f76440717a0 con 0x7f76440cbb10 2023-12-20T04:28:46.778 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.1 2023-12-20T04:28:46.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.776+0000 7f7639ffb640 1 -- 172.21.15.32:0/107957815 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f763c054310 con 0x7f76440cbb10 2023-12-20T04:28:46.778 INFO:teuthology.orchestra.run.smithi032.stdout:55834574854 2023-12-20T04:28:46.779 DEBUG:teuthology.parallel:result is None 2023-12-20T04:28:46.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 -- 172.21.15.32:0/107957815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7628065b30 msgr2=0x7f7628067ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7628065b30 0x7f7628067ff0 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f76400097c0 tx=0x7f76400a6000 comp rx=0 tx=0).stop 2023-12-20T04:28:46.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 -- 172.21.15.32:0/107957815 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 msgr2=0x7f76440c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:46.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 0x7f76440c4b70 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f763c00b970 tx=0x7f763c00be30 comp rx=0 tx=0).stop 2023-12-20T04:28:46.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 -- 172.21.15.32:0/107957815 shutdown_connections 2023-12-20T04:28:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7628065b30 0x7f7628067ff0 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76440cbb10 0x7f76440c4b70 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76440c8c30 0x7f76440c4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 --2- 172.21.15.32:0/107957815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76440afff0 0x7f76441458f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 -- 172.21.15.32:0/107957815 >> 172.21.15.32:0/107957815 conn(0x7f76440caff0 msgr2=0x7f76440af840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 -- 172.21.15.32:0/107957815 shutdown_connections 2023-12-20T04:28:46.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:46.780+0000 7f764a2ca640 1 -- 172.21.15.32:0/107957815 wait complete. 2023-12-20T04:28:46.793 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.7 2023-12-20T04:28:46.793 DEBUG:teuthology.parallel:result is None 2023-12-20T04:28:46.795 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.6 2023-12-20T04:28:46.795 DEBUG:teuthology.parallel:result is None 2023-12-20T04:28:46.801 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.3 2023-12-20T04:28:46.801 DEBUG:teuthology.parallel:result is None 2023-12-20T04:28:46.805 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.5 2023-12-20T04:28:46.805 DEBUG:teuthology.parallel:result is None 2023-12-20T04:28:46.806 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574853 got 55834574854 for osd.2 2023-12-20T04:28:46.806 DEBUG:teuthology.parallel:result is None 2023-12-20T04:28:47.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2023-12-20T04:28:47.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.884+0000 7f51a99a2640 1 Processor -- start 2023-12-20T04:28:47.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.884+0000 7f51a99a2640 1 -- start start 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a99a2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f51a40b8480 0x7f51a406af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a99a2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51a406b460 0x7f51a4160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a99a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4161330 0x7f51a4153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a99a2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a406b8f0 con 0x7f51a406b460 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a99a2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51a40cd4d0 con 0x7f51a40b8480 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a99a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40ce4e0 con 0x7f51a4161330 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51a406b460 0x7f51a4160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51a406b460 0x7f51a4160df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48644/0 (socket says 172.21.15.32:48644) 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a3fff640 1 -- 172.21.15.32:0/3933611997 learned_addr learned my addr 172.21.15.32:0/3933611997 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a89a0640 1 --2- 172.21.15.32:0/3933611997 >> v2:172.21.15.39:3300/0 conn(0x7f51a40b8480 0x7f51a406af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a91a1640 1 --2- 172.21.15.32:0/3933611997 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4161330 0x7f51a4153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a3fff640 1 -- 172.21.15.32:0/3933611997 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4161330 msgr2=0x7f51a4153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:47.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a3fff640 1 --2- 172.21.15.32:0/3933611997 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4161330 0x7f51a4153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:47.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a3fff640 1 -- 172.21.15.32:0/3933611997 >> v2:172.21.15.39:3300/0 conn(0x7f51a40b8480 msgr2=0x7f51a406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:47.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a3fff640 1 --2- 172.21.15.32:0/3933611997 >> v2:172.21.15.39:3300/0 conn(0x7f51a40b8480 0x7f51a406af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:47.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a3fff640 1 -- 172.21.15.32:0/3933611997 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a40cd720 con 0x7f51a406b460 2023-12-20T04:28:47.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.888+0000 7f51a91a1640 1 --2- 172.21.15.32:0/3933611997 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4161330 0x7f51a4153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:47.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a3fff640 1 --2- 172.21.15.32:0/3933611997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a406b460 0x7f51a4160df0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f51a406bb70 tx=0x7f519001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8f617bee54d6bb8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:47.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a37fe640 1 -- 172.21.15.32:0/3933611997 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51900ac410 con 0x7f51a406b460 2023-12-20T04:28:47.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a37fe640 1 -- 172.21.15.32:0/3933611997 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51900ac570 con 0x7f51a406b460 2023-12-20T04:28:47.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a37fe640 1 -- 172.21.15.32:0/3933611997 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51900ac900 con 0x7f51a406b460 2023-12-20T04:28:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a99a2640 1 -- 172.21.15.32:0/3933611997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a406b460 msgr2=0x7f51a4160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a99a2640 1 --2- 172.21.15.32:0/3933611997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a406b460 0x7f51a4160df0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f51a406bb70 tx=0x7f519001c500 comp rx=0 tx=0).stop 2023-12-20T04:28:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a99a2640 1 -- 172.21.15.32:0/3933611997 shutdown_connections 2023-12-20T04:28:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a99a2640 1 --2- 172.21.15.32:0/3933611997 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a4161330 0x7f51a4153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a99a2640 1 --2- 172.21.15.32:0/3933611997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a406b460 0x7f51a4160df0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a99a2640 1 --2- 172.21.15.32:0/3933611997 >> v2:172.21.15.39:3300/0 conn(0x7f51a40b8480 0x7f51a406af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a99a2640 1 -- 172.21.15.32:0/3933611997 >> 172.21.15.32:0/3933611997 conn(0x7f51a40d2a50 msgr2=0x7f51a40d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a99a2640 1 -- 172.21.15.32:0/3933611997 shutdown_connections 2023-12-20T04:28:47.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a99a2640 1 -- 172.21.15.32:0/3933611997 wait complete. 2023-12-20T04:28:47.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a99a2640 1 Processor -- start 2023-12-20T04:28:47.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.892+0000 7f51a99a2640 1 -- start start 2023-12-20T04:28:47.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a99a2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a406b460 0x7f51a40cc8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:47.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a99a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a40b8480 0x7f51a414c770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:47.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a99a2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4161330 0x7f51a40cad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:47.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a99a2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40cb820 con 0x7f51a406b460 2023-12-20T04:28:47.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a99a2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40cbb00 con 0x7f51a4161330 2023-12-20T04:28:47.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a99a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51a40cbde0 con 0x7f51a40b8480 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a89a0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a406b460 0x7f51a40cc8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a89a0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a406b460 0x7f51a40cc8c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48646/0 (socket says 172.21.15.32:48646) 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a89a0640 1 -- 172.21.15.32:0/154134640 learned_addr learned my addr 172.21.15.32:0/154134640 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a3fff640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a40b8480 0x7f51a414c770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a91a1640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4161330 0x7f51a40cad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a89a0640 1 -- 172.21.15.32:0/154134640 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a40b8480 msgr2=0x7f51a414c770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a89a0640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a40b8480 0x7f51a414c770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a89a0640 1 -- 172.21.15.32:0/154134640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4161330 msgr2=0x7f51a40cad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a89a0640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4161330 0x7f51a40cad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a89a0640 1 -- 172.21.15.32:0/154134640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51900094d0 con 0x7f51a406b460 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a91a1640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4161330 0x7f51a40cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a89a0640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a406b460 0x7f51a40cc8c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f519800e8b0 tx=0x7f519800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a17fa640 1 -- 172.21.15.32:0/154134640 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519800c940 con 0x7f51a406b460 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a17fa640 1 -- 172.21.15.32:0/154134640 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f519800caa0 con 0x7f51a406b460 2023-12-20T04:28:47.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a17fa640 1 -- 172.21.15.32:0/154134640 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f519800ce30 con 0x7f51a406b460 2023-12-20T04:28:47.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a99a2640 1 -- 172.21.15.32:0/154134640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51a40cc0c0 con 0x7f51a406b460 2023-12-20T04:28:47.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a99a2640 1 -- 172.21.15.32:0/154134640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51a40c5f90 con 0x7f51a406b460 2023-12-20T04:28:47.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.896+0000 7f51a99a2640 1 -- 172.21.15.32:0/154134640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51a4074100 con 0x7f51a406b460 2023-12-20T04:28:47.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.900+0000 7f51a17fa640 1 -- 172.21.15.32:0/154134640 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5198013470 con 0x7f51a406b460 2023-12-20T04:28:47.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.900+0000 7f51a17fa640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065a60 0x7f5188067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:47.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.900+0000 7f51a17fa640 1 -- 172.21.15.32:0/154134640 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5198083370 con 0x7f51a406b460 2023-12-20T04:28:47.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.900+0000 7f51a3fff640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065a60 0x7f5188067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:47.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.900+0000 7f51a3fff640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065a60 0x7f5188067f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f51900ae000 tx=0x7f5190009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:47.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:47.904+0000 7f51a17fa640 1 -- 172.21.15.32:0/154134640 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5198050d40 con 0x7f51a406b460 2023-12-20T04:28:48.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 -- 172.21.15.32:0/154134640 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) v1 -- 0x7f51a40d0a70 con 0x7f51a406b460 2023-12-20T04:28:48.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a17fa640 1 -- 172.21.15.32:0/154134640 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) v1 ==== 74+0+12 (secure 0 0 0) 0x7f5198054360 con 0x7f51a406b460 2023-12-20T04:28:48.065 INFO:teuthology.orchestra.run.smithi032.stdout:55834574855 2023-12-20T04:28:48.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 -- 172.21.15.32:0/154134640 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065a60 msgr2=0x7f5188067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065a60 0x7f5188067f20 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7f51900ae000 tx=0x7f5190009040 comp rx=0 tx=0).stop 2023-12-20T04:28:48.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 -- 172.21.15.32:0/154134640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a406b460 msgr2=0x7f51a40cc8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a406b460 0x7f51a40cc8c0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f519800e8b0 tx=0x7f519800ed70 comp rx=0 tx=0).stop 2023-12-20T04:28:48.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 -- 172.21.15.32:0/154134640 shutdown_connections 2023-12-20T04:28:48.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065a60 0x7f5188067f20 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51a4161330 0x7f51a40cad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51a40b8480 0x7f51a414c770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 --2- 172.21.15.32:0/154134640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51a406b460 0x7f51a40cc8c0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 -- 172.21.15.32:0/154134640 >> 172.21.15.32:0/154134640 conn(0x7f51a40d2a50 msgr2=0x7f51a40d1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:48.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 -- 172.21.15.32:0/154134640 shutdown_connections 2023-12-20T04:28:48.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.064+0000 7f51a99a2640 1 -- 172.21.15.32:0/154134640 wait complete. 2023-12-20T04:28:48.087 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574854 got 55834574855 for osd.0 2023-12-20T04:28:48.087 DEBUG:teuthology.parallel:result is None 2023-12-20T04:28:48.087 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-20T04:28:48.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:28:48.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.228+0000 7fd8db325640 1 Processor -- start 2023-12-20T04:28:48.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8db325640 1 -- start start 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8db325640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8d40ae010 0x7fd8d4071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8db325640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8d4164ef0 0x7fd8d40719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8db325640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8d40ae3f0 con 0x7fd8d406cf10 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8db325640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8d40ca4b0 con 0x7fd8d40ae010 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8db325640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d4161800 con 0x7fd8d4164ef0 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8d90c1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8d40ae010 0x7fd8d4071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8d90c1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8d40ae010 0x7fd8d4071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48660/0 (socket says 172.21.15.32:48660) 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8d90c1640 1 -- 172.21.15.32:0/1986027830 learned_addr learned my addr 172.21.15.32:0/1986027830 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8d98c2640 1 --2- 172.21.15.32:0/1986027830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8d4164ef0 0x7fd8d40719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8d90c1640 1 -- 172.21.15.32:0/1986027830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8d4164ef0 msgr2=0x7fd8d40719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8d90c1640 1 --2- 172.21.15.32:0/1986027830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8d4164ef0 0x7fd8d40719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8d90c1640 1 -- 172.21.15.32:0/1986027830 >> v1:172.21.15.39:6789/0 conn(0x7fd8d406cf10 legacy=0x7fd8d40701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:48.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8d90c1640 1 -- 172.21.15.32:0/1986027830 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d4165380 con 0x7fd8d40ae010 2023-12-20T04:28:48.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.232+0000 7fd8d98c2640 1 --2- 172.21.15.32:0/1986027830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8d4164ef0 0x7fd8d40719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:28:48.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8d90c1640 1 --2- 172.21.15.32:0/1986027830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d40ae010 0x7fd8d4071470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0006fd0 tx=0x7fd8d001c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f3a719def0b208b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:48.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8cbfff640 1 -- 172.21.15.32:0/1986027830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00ac640 con 0x7fd8d40ae010 2023-12-20T04:28:48.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8cbfff640 1 -- 172.21.15.32:0/1986027830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d00ac7a0 con 0x7fd8d40ae010 2023-12-20T04:28:48.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8cbfff640 1 -- 172.21.15.32:0/1986027830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00acb30 con 0x7fd8d40ae010 2023-12-20T04:28:48.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8db325640 1 -- 172.21.15.32:0/1986027830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d40ae010 msgr2=0x7fd8d4071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8db325640 1 --2- 172.21.15.32:0/1986027830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d40ae010 0x7fd8d4071470 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0006fd0 tx=0x7fd8d001c420 comp rx=0 tx=0).stop 2023-12-20T04:28:48.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8db325640 1 -- 172.21.15.32:0/1986027830 shutdown_connections 2023-12-20T04:28:48.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8db325640 1 --2- 172.21.15.32:0/1986027830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8d4164ef0 0x7fd8d40719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8db325640 1 --2- 172.21.15.32:0/1986027830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d40ae010 0x7fd8d4071470 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8db325640 1 -- 172.21.15.32:0/1986027830 >> 172.21.15.32:0/1986027830 conn(0x7fd8d406a320 msgr2=0x7fd8d40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:48.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8db325640 1 -- 172.21.15.32:0/1986027830 shutdown_connections 2023-12-20T04:28:48.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8db325640 1 -- 172.21.15.32:0/1986027830 wait complete. 2023-12-20T04:28:48.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8db325640 1 Processor -- start 2023-12-20T04:28:48.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.236+0000 7fd8db325640 1 -- start start 2023-12-20T04:28:48.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8db325640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d40ae010 0x7fd8d40c8040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8db325640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d41597f0 0x7fd8d414eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8db325640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8d415b800 0x7fd8d415bc80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8db325640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d4158b00 con 0x7fd8d40ae010 2023-12-20T04:28:48.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8db325640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d4159360 con 0x7fd8d41597f0 2023-12-20T04:28:48.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8db325640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8d40c8640 con 0x7fd8d415b800 2023-12-20T04:28:48.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8d90c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d40ae010 0x7fd8d40c8040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8d90c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d40ae010 0x7fd8d40c8040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48674/0 (socket says 172.21.15.32:48674) 2023-12-20T04:28:48.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8d90c1640 1 -- 172.21.15.32:0/2380779217 learned_addr learned my addr 172.21.15.32:0/2380779217 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:48.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8d98c2640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8d415b800 0x7fd8d415bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8d88c0640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d41597f0 0x7fd8d414eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8d90c1640 1 -- 172.21.15.32:0/2380779217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8d415b800 msgr2=0x7fd8d415bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8d90c1640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8d415b800 0x7fd8d415bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8d90c1640 1 -- 172.21.15.32:0/2380779217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d41597f0 msgr2=0x7fd8d414eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8d90c1640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d41597f0 0x7fd8d414eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8d90c1640 1 -- 172.21.15.32:0/2380779217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8d0009480 con 0x7fd8d40ae010 2023-12-20T04:28:48.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8d90c1640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d40ae010 0x7fd8d40c8040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0006fd0 tx=0x7fd8d00191c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:48.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8c9ffb640 1 -- 172.21.15.32:0/2380779217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00b9070 con 0x7fd8d40ae010 2023-12-20T04:28:48.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8c9ffb640 1 -- 172.21.15.32:0/2380779217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8d00ac5e0 con 0x7fd8d40ae010 2023-12-20T04:28:48.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8c9ffb640 1 -- 172.21.15.32:0/2380779217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8d00acd20 con 0x7fd8d40ae010 2023-12-20T04:28:48.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8db325640 1 -- 172.21.15.32:0/2380779217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8d4158d10 con 0x7fd8d40ae010 2023-12-20T04:28:48.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8db325640 1 -- 172.21.15.32:0/2380779217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8d4150630 con 0x7fd8d40ae010 2023-12-20T04:28:48.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.240+0000 7fd8db325640 1 -- 172.21.15.32:0/2380779217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8d404e000 con 0x7fd8d40ae010 2023-12-20T04:28:48.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.244+0000 7fd8c9ffb640 1 -- 172.21.15.32:0/2380779217 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd8d00b3070 con 0x7fd8d40ae010 2023-12-20T04:28:48.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.244+0000 7fd8c9ffb640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd8b8065a60 0x7fd8b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.244+0000 7fd8c9ffb640 1 -- 172.21.15.32:0/2380779217 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd8d011cad0 con 0x7fd8d40ae010 2023-12-20T04:28:48.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.244+0000 7fd8d88c0640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd8b8065a60 0x7fd8b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.244+0000 7fd8d88c0640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd8b8065a60 0x7fd8b8067f20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd8d416c250 tx=0x7fd8c4007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:48.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.248+0000 7fd8c9ffb640 1 -- 172.21.15.32:0/2380779217 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd8d00ea5d0 con 0x7fd8d40ae010 2023-12-20T04:28:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.400+0000 7fd8db325640 1 -- 172.21.15.32:0/2380779217 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd8d4165c20 con 0x7fd8b8065a60 2023-12-20T04:28:48.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.408+0000 7fd8c9ffb640 1 -- 172.21.15.32:0/2380779217 <== mgr.4104 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264528 (secure 0 0 0) 0x7fd8c4003bf0 con 0x7fd8b8065a60 2023-12-20T04:28:48.411 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:48.413 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:28:48.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 -- 172.21.15.32:0/2380779217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd8b8065a60 msgr2=0x7fd8b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd8b8065a60 0x7fd8b8067f20 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fd8d416c250 tx=0x7fd8c4007e40 comp rx=0 tx=0).stop 2023-12-20T04:28:48.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 -- 172.21.15.32:0/2380779217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d40ae010 msgr2=0x7fd8d40c8040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d40ae010 0x7fd8d40c8040 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fd8d0006fd0 tx=0x7fd8d00191c0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 -- 172.21.15.32:0/2380779217 shutdown_connections 2023-12-20T04:28:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd8b8065a60 0x7fd8b8067f20 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8d415b800 0x7fd8d415bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8d41597f0 0x7fd8d414eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 --2- 172.21.15.32:0/2380779217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8d40ae010 0x7fd8d40c8040 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 -- 172.21.15.32:0/2380779217 >> 172.21.15.32:0/2380779217 conn(0x7fd8d406a320 msgr2=0x7fd8d415a540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 -- 172.21.15.32:0/2380779217 shutdown_connections 2023-12-20T04:28:48.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.412+0000 7fd8db325640 1 -- 172.21.15.32:0/2380779217 wait complete. 2023-12-20T04:28:48.431 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-20T04:28:48.129580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239016,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":236032,"kb_avail":754735704,"statfs":{"total":773094113280,"available":772849360896,"internally_reserved":0,"allocated":2990080,"data_stored":1790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":241696768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.764412"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667779+0000","last_change":"2023-12-20T04:28:38.407268+0000","last_active":"2023-12-20T04:28:39.667779+0000","last_peered":"2023-12-20T04:28:39.667779+0000","last_clean":"2023-12-20T04:28:39.667779+0000","last_became_active":"2023-12-20T04:28:38.404491+0000","last_became_peered":"2023-12-20T04:28:38.404491+0000","last_unstale":"2023-12-20T04:28:39.667779+0000","last_undegraded":"2023-12-20T04:28:39.667779+0000","last_fullsized":"2023-12-20T04:28:39.667779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:36.363864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379971+0000","last_change":"2023-12-20T04:28:38.390082+0000","last_active":"2023-12-20T04:28:39.379971+0000","last_peered":"2023-12-20T04:28:39.379971+0000","last_clean":"2023-12-20T04:28:39.379971+0000","last_became_active":"2023-12-20T04:28:38.389688+0000","last_became_peered":"2023-12-20T04:28:38.389688+0000","last_unstale":"2023-12-20T04:28:39.379971+0000","last_undegraded":"2023-12-20T04:28:39.379971+0000","last_fullsized":"2023-12-20T04:28:39.379971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666298+0000","last_change":"2023-12-20T04:28:38.388981+0000","last_active":"2023-12-20T04:28:39.666298+0000","last_peered":"2023-12-20T04:28:39.666298+0000","last_clean":"2023-12-20T04:28:39.666298+0000","last_became_active":"2023-12-20T04:28:38.388734+0000","last_became_peered":"2023-12-20T04:28:38.388734+0000","last_unstale":"2023-12-20T04:28:39.666298+0000","last_undegraded":"2023-12-20T04:28:39.666298+0000","last_fullsized":"2023-12-20T04:28:39.666298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385490+0000","last_change":"2023-12-20T04:28:38.384727+0000","last_active":"2023-12-20T04:28:39.385490+0000","last_peered":"2023-12-20T04:28:39.385490+0000","last_clean":"2023-12-20T04:28:39.385490+0000","last_became_active":"2023-12-20T04:28:38.384329+0000","last_became_peered":"2023-12-20T04:28:38.384329+0000","last_unstale":"2023-12-20T04:28:39.385490+0000","last_undegraded":"2023-12-20T04:28:39.385490+0000","last_fullsized":"2023-12-20T04:28:39.385490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:11:32.618631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667485+0000","last_change":"2023-12-20T04:28:38.397089+0000","last_active":"2023-12-20T04:28:39.667485+0000","last_peered":"2023-12-20T04:28:39.667485+0000","last_clean":"2023-12-20T04:28:39.667485+0000","last_became_active":"2023-12-20T04:28:38.396774+0000","last_became_peered":"2023-12-20T04:28:38.396774+0000","last_unstale":"2023-12-20T04:28:39.667485+0000","last_undegraded":"2023-12-20T04:28:39.667485+0000","last_fullsized":"2023-12-20T04:28:39.667485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386104+0000","last_change":"2023-12-20T04:28:38.395384+0000","last_active":"2023-12-20T04:28:39.386104+0000","last_peered":"2023-12-20T04:28:39.386104+0000","last_clean":"2023-12-20T04:28:39.386104+0000","last_became_active":"2023-12-20T04:28:38.394254+0000","last_became_peered":"2023-12-20T04:28:38.394254+0000","last_unstale":"2023-12-20T04:28:39.386104+0000","last_undegraded":"2023-12-20T04:28:39.386104+0000","last_fullsized":"2023-12-20T04:28:39.386104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:55:26.717187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667445+0000","last_change":"2023-12-20T04:28:38.404597+0000","last_active":"2023-12-20T04:28:39.667445+0000","last_peered":"2023-12-20T04:28:39.667445+0000","last_clean":"2023-12-20T04:28:39.667445+0000","last_became_active":"2023-12-20T04:28:38.404108+0000","last_became_peered":"2023-12-20T04:28:38.404108+0000","last_unstale":"2023-12-20T04:28:39.667445+0000","last_undegraded":"2023-12-20T04:28:39.667445+0000","last_fullsized":"2023-12-20T04:28:39.667445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:41.399788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670310+0000","last_change":"2023-12-20T04:28:38.385150+0000","last_active":"2023-12-20T04:28:39.670310+0000","last_peered":"2023-12-20T04:28:39.670310+0000","last_clean":"2023-12-20T04:28:39.670310+0000","last_became_active":"2023-12-20T04:28:38.384885+0000","last_became_peered":"2023-12-20T04:28:38.384885+0000","last_unstale":"2023-12-20T04:28:39.670310+0000","last_undegraded":"2023-12-20T04:28:39.670310+0000","last_fullsized":"2023-12-20T04:28:39.670310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667828+0000","last_change":"2023-12-20T04:28:38.399891+0000","last_active":"2023-12-20T04:28:39.667828+0000","last_peered":"2023-12-20T04:28:39.667828+0000","last_clean":"2023-12-20T04:28:39.667828+0000","last_became_active":"2023-12-20T04:28:38.399599+0000","last_became_peered":"2023-12-20T04:28:38.399599+0000","last_unstale":"2023-12-20T04:28:39.667828+0000","last_undegraded":"2023-12-20T04:28:39.667828+0000","last_fullsized":"2023-12-20T04:28:39.667828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:46:52.340773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667813+0000","last_change":"2023-12-20T04:28:38.398573+0000","last_active":"2023-12-20T04:28:39.667813+0000","last_peered":"2023-12-20T04:28:39.667813+0000","last_clean":"2023-12-20T04:28:39.667813+0000","last_became_active":"2023-12-20T04:28:38.398162+0000","last_became_peered":"2023-12-20T04:28:38.398162+0000","last_unstale":"2023-12-20T04:28:39.667813+0000","last_undegraded":"2023-12-20T04:28:39.667813+0000","last_fullsized":"2023-12-20T04:28:39.667813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667799+0000","last_change":"2023-12-20T04:28:38.405208+0000","last_active":"2023-12-20T04:28:39.667799+0000","last_peered":"2023-12-20T04:28:39.667799+0000","last_clean":"2023-12-20T04:28:39.667799+0000","last_became_active":"2023-12-20T04:28:38.404933+0000","last_became_peered":"2023-12-20T04:28:38.404933+0000","last_unstale":"2023-12-20T04:28:39.667799+0000","last_undegraded":"2023-12-20T04:28:39.667799+0000","last_fullsized":"2023-12-20T04:28:39.667799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:50.720992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386165+0000","last_change":"2023-12-20T04:28:38.412922+0000","last_active":"2023-12-20T04:28:39.386165+0000","last_peered":"2023-12-20T04:28:39.386165+0000","last_clean":"2023-12-20T04:28:39.386165+0000","last_became_active":"2023-12-20T04:28:38.412505+0000","last_became_peered":"2023-12-20T04:28:38.412505+0000","last_unstale":"2023-12-20T04:28:39.386165+0000","last_undegraded":"2023-12-20T04:28:39.386165+0000","last_fullsized":"2023-12-20T04:28:39.386165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:04:59.607569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670386+0000","last_change":"2023-12-20T04:28:38.390684+0000","last_active":"2023-12-20T04:28:39.670386+0000","last_peered":"2023-12-20T04:28:39.670386+0000","last_clean":"2023-12-20T04:28:39.670386+0000","last_became_active":"2023-12-20T04:28:38.390483+0000","last_became_peered":"2023-12-20T04:28:38.390483+0000","last_unstale":"2023-12-20T04:28:39.670386+0000","last_undegraded":"2023-12-20T04:28:39.670386+0000","last_fullsized":"2023-12-20T04:28:39.670386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:55:26.717187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666230+0000","last_change":"2023-12-20T04:28:38.379167+0000","last_active":"2023-12-20T04:28:39.666230+0000","last_peered":"2023-12-20T04:28:39.666230+0000","last_clean":"2023-12-20T04:28:39.666230+0000","last_became_active":"2023-12-20T04:28:38.378677+0000","last_became_peered":"2023-12-20T04:28:38.378677+0000","last_unstale":"2023-12-20T04:28:39.666230+0000","last_undegraded":"2023-12-20T04:28:39.666230+0000","last_fullsized":"2023-12-20T04:28:39.666230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379652+0000","last_change":"2023-12-20T04:28:38.391213+0000","last_active":"2023-12-20T04:28:39.379652+0000","last_peered":"2023-12-20T04:28:39.379652+0000","last_clean":"2023-12-20T04:28:39.379652+0000","last_became_active":"2023-12-20T04:28:38.390937+0000","last_became_peered":"2023-12-20T04:28:38.390937+0000","last_unstale":"2023-12-20T04:28:39.379652+0000","last_undegraded":"2023-12-20T04:28:39.379652+0000","last_fullsized":"2023-12-20T04:28:39.379652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385448+0000","last_change":"2023-12-20T04:28:38.403378+0000","last_active":"2023-12-20T04:28:39.385448+0000","last_peered":"2023-12-20T04:28:39.385448+0000","last_clean":"2023-12-20T04:28:39.385448+0000","last_became_active":"2023-12-20T04:28:38.403070+0000","last_became_peered":"2023-12-20T04:28:38.403070+0000","last_unstale":"2023-12-20T04:28:39.385448+0000","last_undegraded":"2023-12-20T04:28:39.385448+0000","last_fullsized":"2023-12-20T04:28:39.385448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:48:18.553482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385977+0000","last_change":"2023-12-20T04:28:38.399610+0000","last_active":"2023-12-20T04:28:39.385977+0000","last_peered":"2023-12-20T04:28:39.385977+0000","last_clean":"2023-12-20T04:28:39.385977+0000","last_became_active":"2023-12-20T04:28:38.399061+0000","last_became_peered":"2023-12-20T04:28:38.399061+0000","last_unstale":"2023-12-20T04:28:39.385977+0000","last_undegraded":"2023-12-20T04:28:39.385977+0000","last_fullsized":"2023-12-20T04:28:39.385977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666152+0000","last_change":"2023-12-20T04:28:38.398099+0000","last_active":"2023-12-20T04:28:39.666152+0000","last_peered":"2023-12-20T04:28:39.666152+0000","last_clean":"2023-12-20T04:28:39.666152+0000","last_became_active":"2023-12-20T04:28:38.397333+0000","last_became_peered":"2023-12-20T04:28:38.397333+0000","last_unstale":"2023-12-20T04:28:39.666152+0000","last_undegraded":"2023-12-20T04:28:39.666152+0000","last_fullsized":"2023-12-20T04:28:39.666152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:12.041027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667726+0000","last_change":"2023-12-20T04:28:38.397182+0000","last_active":"2023-12-20T04:28:39.667726+0000","last_peered":"2023-12-20T04:28:39.667726+0000","last_clean":"2023-12-20T04:28:39.667726+0000","last_became_active":"2023-12-20T04:28:38.396800+0000","last_became_peered":"2023-12-20T04:28:38.396800+0000","last_unstale":"2023-12-20T04:28:39.667726+0000","last_undegraded":"2023-12-20T04:28:39.667726+0000","last_fullsized":"2023-12-20T04:28:39.667726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667761+0000","last_change":"2023-12-20T04:28:38.410245+0000","last_active":"2023-12-20T04:28:39.667761+0000","last_peered":"2023-12-20T04:28:39.667761+0000","last_clean":"2023-12-20T04:28:39.667761+0000","last_became_active":"2023-12-20T04:28:38.403138+0000","last_became_peered":"2023-12-20T04:28:38.403138+0000","last_unstale":"2023-12-20T04:28:39.667761+0000","last_undegraded":"2023-12-20T04:28:39.667761+0000","last_fullsized":"2023-12-20T04:28:39.667761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666258+0000","last_change":"2023-12-20T04:28:38.400262+0000","last_active":"2023-12-20T04:28:39.666258+0000","last_peered":"2023-12-20T04:28:39.666258+0000","last_clean":"2023-12-20T04:28:39.666258+0000","last_became_active":"2023-12-20T04:28:38.400021+0000","last_became_peered":"2023-12-20T04:28:38.400021+0000","last_unstale":"2023-12-20T04:28:39.666258+0000","last_undegraded":"2023-12-20T04:28:39.666258+0000","last_fullsized":"2023-12-20T04:28:39.666258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386133+0000","last_change":"2023-12-20T04:28:38.397515+0000","last_active":"2023-12-20T04:28:39.386133+0000","last_peered":"2023-12-20T04:28:39.386133+0000","last_clean":"2023-12-20T04:28:39.386133+0000","last_became_active":"2023-12-20T04:28:38.397234+0000","last_became_peered":"2023-12-20T04:28:38.397234+0000","last_unstale":"2023-12-20T04:28:39.386133+0000","last_undegraded":"2023-12-20T04:28:39.386133+0000","last_fullsized":"2023-12-20T04:28:39.386133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670343+0000","last_change":"2023-12-20T04:28:38.390587+0000","last_active":"2023-12-20T04:28:39.670343+0000","last_peered":"2023-12-20T04:28:39.670343+0000","last_clean":"2023-12-20T04:28:39.670343+0000","last_became_active":"2023-12-20T04:28:38.390256+0000","last_became_peered":"2023-12-20T04:28:38.390256+0000","last_unstale":"2023-12-20T04:28:39.670343+0000","last_undegraded":"2023-12-20T04:28:39.670343+0000","last_fullsized":"2023-12-20T04:28:39.670343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385781+0000","last_change":"2023-12-20T04:28:38.390087+0000","last_active":"2023-12-20T04:28:39.385781+0000","last_peered":"2023-12-20T04:28:39.385781+0000","last_clean":"2023-12-20T04:28:39.385781+0000","last_became_active":"2023-12-20T04:28:38.389881+0000","last_became_peered":"2023-12-20T04:28:38.389881+0000","last_unstale":"2023-12-20T04:28:39.385781+0000","last_undegraded":"2023-12-20T04:28:39.385781+0000","last_fullsized":"2023-12-20T04:28:39.385781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:09.976618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670450+0000","last_change":"2023-12-20T04:28:38.396543+0000","last_active":"2023-12-20T04:28:39.670450+0000","last_peered":"2023-12-20T04:28:39.670450+0000","last_clean":"2023-12-20T04:28:39.670450+0000","last_became_active":"2023-12-20T04:28:38.396247+0000","last_became_peered":"2023-12-20T04:28:38.396247+0000","last_unstale":"2023-12-20T04:28:39.670450+0000","last_undegraded":"2023-12-20T04:28:39.670450+0000","last_fullsized":"2023-12-20T04:28:39.670450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667823+0000","last_change":"2023-12-20T04:28:38.401846+0000","last_active":"2023-12-20T04:28:39.667823+0000","last_peered":"2023-12-20T04:28:39.667823+0000","last_clean":"2023-12-20T04:28:39.667823+0000","last_became_active":"2023-12-20T04:28:38.401255+0000","last_became_peered":"2023-12-20T04:28:38.401255+0000","last_unstale":"2023-12-20T04:28:39.667823+0000","last_undegraded":"2023-12-20T04:28:39.667823+0000","last_fullsized":"2023-12-20T04:28:39.667823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379844+0000","last_change":"2023-12-20T04:28:38.385251+0000","last_active":"2023-12-20T04:28:39.379844+0000","last_peered":"2023-12-20T04:28:39.379844+0000","last_clean":"2023-12-20T04:28:39.379844+0000","last_became_active":"2023-12-20T04:28:38.384990+0000","last_became_peered":"2023-12-20T04:28:38.384990+0000","last_unstale":"2023-12-20T04:28:39.379844+0000","last_undegraded":"2023-12-20T04:28:39.379844+0000","last_fullsized":"2023-12-20T04:28:39.379844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670414+0000","last_change":"2023-12-20T04:28:38.394851+0000","last_active":"2023-12-20T04:28:39.670414+0000","last_peered":"2023-12-20T04:28:39.670414+0000","last_clean":"2023-12-20T04:28:39.670414+0000","last_became_active":"2023-12-20T04:28:38.394584+0000","last_became_peered":"2023-12-20T04:28:38.394584+0000","last_unstale":"2023-12-20T04:28:39.670414+0000","last_undegraded":"2023-12-20T04:28:39.670414+0000","last_fullsized":"2023-12-20T04:28:39.670414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666271+0000","last_change":"2023-12-20T04:28:38.383584+0000","last_active":"2023-12-20T04:28:39.666271+0000","last_peered":"2023-12-20T04:28:39.666271+0000","last_clean":"2023-12-20T04:28:39.666271+0000","last_became_active":"2023-12-20T04:28:38.383310+0000","last_became_peered":"2023-12-20T04:28:38.383310+0000","last_unstale":"2023-12-20T04:28:39.666271+0000","last_undegraded":"2023-12-20T04:28:39.666271+0000","last_fullsized":"2023-12-20T04:28:39.666271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386215+0000","last_change":"2023-12-20T04:28:38.388220+0000","last_active":"2023-12-20T04:28:39.386215+0000","last_peered":"2023-12-20T04:28:39.386215+0000","last_clean":"2023-12-20T04:28:39.386215+0000","last_became_active":"2023-12-20T04:28:38.386044+0000","last_became_peered":"2023-12-20T04:28:38.386044+0000","last_unstale":"2023-12-20T04:28:39.386215+0000","last_undegraded":"2023-12-20T04:28:39.386215+0000","last_fullsized":"2023-12-20T04:28:39.386215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:51.073749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666072+0000","last_change":"2023-12-20T04:28:38.384460+0000","last_active":"2023-12-20T04:28:39.666072+0000","last_peered":"2023-12-20T04:28:39.666072+0000","last_clean":"2023-12-20T04:28:39.666072+0000","last_became_active":"2023-12-20T04:28:38.384190+0000","last_became_peered":"2023-12-20T04:28:38.384190+0000","last_unstale":"2023-12-20T04:28:39.666072+0000","last_undegraded":"2023-12-20T04:28:39.666072+0000","last_fullsized":"2023-12-20T04:28:39.666072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667624+0000","last_change":"2023-12-20T04:28:38.403381+0000","last_active":"2023-12-20T04:28:39.667624+0000","last_peered":"2023-12-20T04:28:39.667624+0000","last_clean":"2023-12-20T04:28:39.667624+0000","last_became_active":"2023-12-20T04:28:38.402953+0000","last_became_peered":"2023-12-20T04:28:38.402953+0000","last_unstale":"2023-12-20T04:28:39.667624+0000","last_undegraded":"2023-12-20T04:28:39.667624+0000","last_fullsized":"2023-12-20T04:28:39.667624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385741+0000","last_change":"2023-12-20T04:28:38.392967+0000","last_active":"2023-12-20T04:28:39.385741+0000","last_peered":"2023-12-20T04:28:39.385741+0000","last_clean":"2023-12-20T04:28:39.385741+0000","last_became_active":"2023-12-20T04:28:38.392657+0000","last_became_peered":"2023-12-20T04:28:38.392657+0000","last_unstale":"2023-12-20T04:28:39.385741+0000","last_undegraded":"2023-12-20T04:28:39.385741+0000","last_fullsized":"2023-12-20T04:28:39.385741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:35.897773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670404+0000","last_change":"2023-12-20T04:28:37.983099+0000","last_active":"2023-12-20T04:28:39.670404+0000","last_peered":"2023-12-20T04:28:39.670404+0000","last_clean":"2023-12-20T04:28:39.670404+0000","last_became_active":"2023-12-20T04:28:37.982857+0000","last_became_peered":"2023-12-20T04:28:37.982857+0000","last_unstale":"2023-12-20T04:28:39.670404+0000","last_undegraded":"2023-12-20T04:28:39.670404+0000","last_fullsized":"2023-12-20T04:28:39.670404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666279+0000","last_change":"2023-12-20T04:28:38.407295+0000","last_active":"2023-12-20T04:28:39.666279+0000","last_peered":"2023-12-20T04:28:39.666279+0000","last_clean":"2023-12-20T04:28:39.666279+0000","last_became_active":"2023-12-20T04:28:38.406963+0000","last_became_peered":"2023-12-20T04:28:38.406963+0000","last_unstale":"2023-12-20T04:28:39.666279+0000","last_undegraded":"2023-12-20T04:28:39.666279+0000","last_fullsized":"2023-12-20T04:28:39.666279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:50.720992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667740+0000","last_change":"2023-12-20T04:28:37.987627+0000","last_active":"2023-12-20T04:28:39.667740+0000","last_peered":"2023-12-20T04:28:39.667740+0000","last_clean":"2023-12-20T04:28:39.667740+0000","last_became_active":"2023-12-20T04:28:37.987309+0000","last_became_peered":"2023-12-20T04:28:37.987309+0000","last_unstale":"2023-12-20T04:28:39.667740+0000","last_undegraded":"2023-12-20T04:28:39.667740+0000","last_fullsized":"2023-12-20T04:28:39.667740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670368+0000","last_change":"2023-12-20T04:28:37.996411+0000","last_active":"2023-12-20T04:28:39.670368+0000","last_peered":"2023-12-20T04:28:39.670368+0000","last_clean":"2023-12-20T04:28:39.670368+0000","last_became_active":"2023-12-20T04:28:37.996051+0000","last_became_peered":"2023-12-20T04:28:37.996051+0000","last_unstale":"2023-12-20T04:28:39.670368+0000","last_undegraded":"2023-12-20T04:28:39.670368+0000","last_fullsized":"2023-12-20T04:28:39.670368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670324+0000","last_change":"2023-12-20T04:28:38.396714+0000","last_active":"2023-12-20T04:28:39.670324+0000","last_peered":"2023-12-20T04:28:39.670324+0000","last_clean":"2023-12-20T04:28:39.670324+0000","last_became_active":"2023-12-20T04:28:38.396337+0000","last_became_peered":"2023-12-20T04:28:38.396337+0000","last_unstale":"2023-12-20T04:28:39.670324+0000","last_undegraded":"2023-12-20T04:28:39.670324+0000","last_fullsized":"2023-12-20T04:28:39.670324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385551+0000","last_change":"2023-12-20T04:28:37.982942+0000","last_active":"2023-12-20T04:28:39.385551+0000","last_peered":"2023-12-20T04:28:39.385551+0000","last_clean":"2023-12-20T04:28:39.385551+0000","last_became_active":"2023-12-20T04:28:37.982287+0000","last_became_peered":"2023-12-20T04:28:37.982287+0000","last_unstale":"2023-12-20T04:28:39.385551+0000","last_undegraded":"2023-12-20T04:28:39.385551+0000","last_fullsized":"2023-12-20T04:28:39.385551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379613+0000","last_change":"2023-12-20T04:28:38.380510+0000","last_active":"2023-12-20T04:28:39.379613+0000","last_peered":"2023-12-20T04:28:39.379613+0000","last_clean":"2023-12-20T04:28:39.379613+0000","last_became_active":"2023-12-20T04:28:38.379896+0000","last_became_peered":"2023-12-20T04:28:38.379896+0000","last_unstale":"2023-12-20T04:28:39.379613+0000","last_undegraded":"2023-12-20T04:28:39.379613+0000","last_fullsized":"2023-12-20T04:28:39.379613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667837+0000","last_change":"2023-12-20T04:28:37.992369+0000","last_active":"2023-12-20T04:28:39.667837+0000","last_peered":"2023-12-20T04:28:39.667837+0000","last_clean":"2023-12-20T04:28:39.667837+0000","last_became_active":"2023-12-20T04:28:37.992065+0000","last_became_peered":"2023-12-20T04:28:37.992065+0000","last_unstale":"2023-12-20T04:28:39.667837+0000","last_undegraded":"2023-12-20T04:28:39.667837+0000","last_fullsized":"2023-12-20T04:28:39.667837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385551+0000","last_change":"2023-12-20T04:28:38.390943+0000","last_active":"2023-12-20T04:28:39.385551+0000","last_peered":"2023-12-20T04:28:39.385551+0000","last_clean":"2023-12-20T04:28:39.385551+0000","last_became_active":"2023-12-20T04:28:38.390661+0000","last_became_peered":"2023-12-20T04:28:38.390661+0000","last_unstale":"2023-12-20T04:28:39.385551+0000","last_undegraded":"2023-12-20T04:28:39.385551+0000","last_fullsized":"2023-12-20T04:28:39.385551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:06.804397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385691+0000","last_change":"2023-12-20T04:28:37.986949+0000","last_active":"2023-12-20T04:28:39.385691+0000","last_peered":"2023-12-20T04:28:39.385691+0000","last_clean":"2023-12-20T04:28:39.385691+0000","last_became_active":"2023-12-20T04:28:37.986664+0000","last_became_peered":"2023-12-20T04:28:37.986664+0000","last_unstale":"2023-12-20T04:28:39.385691+0000","last_undegraded":"2023-12-20T04:28:39.385691+0000","last_fullsized":"2023-12-20T04:28:39.385691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666354+0000","last_change":"2023-12-20T04:28:38.403067+0000","last_active":"2023-12-20T04:28:39.666354+0000","last_peered":"2023-12-20T04:28:39.666354+0000","last_clean":"2023-12-20T04:28:39.666354+0000","last_became_active":"2023-12-20T04:28:38.402312+0000","last_became_peered":"2023-12-20T04:28:38.402312+0000","last_unstale":"2023-12-20T04:28:39.666354+0000","last_undegraded":"2023-12-20T04:28:39.666354+0000","last_fullsized":"2023-12-20T04:28:39.666354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379842+0000","last_change":"2023-12-20T04:28:37.989273+0000","last_active":"2023-12-20T04:28:39.379842+0000","last_peered":"2023-12-20T04:28:39.379842+0000","last_clean":"2023-12-20T04:28:39.379842+0000","last_became_active":"2023-12-20T04:28:37.988921+0000","last_became_peered":"2023-12-20T04:28:37.988921+0000","last_unstale":"2023-12-20T04:28:39.379842+0000","last_undegraded":"2023-12-20T04:28:39.379842+0000","last_fullsized":"2023-12-20T04:28:39.379842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386234+0000","last_change":"2023-12-20T04:28:38.394680+0000","last_active":"2023-12-20T04:28:39.386234+0000","last_peered":"2023-12-20T04:28:39.386234+0000","last_clean":"2023-12-20T04:28:39.386234+0000","last_became_active":"2023-12-20T04:28:38.394369+0000","last_became_peered":"2023-12-20T04:28:38.394369+0000","last_unstale":"2023-12-20T04:28:39.386234+0000","last_undegraded":"2023-12-20T04:28:39.386234+0000","last_fullsized":"2023-12-20T04:28:39.386234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:12.041027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667779+0000","last_change":"2023-12-20T04:28:37.992630+0000","last_active":"2023-12-20T04:28:39.667779+0000","last_peered":"2023-12-20T04:28:39.667779+0000","last_clean":"2023-12-20T04:28:39.667779+0000","last_became_active":"2023-12-20T04:28:37.989328+0000","last_became_peered":"2023-12-20T04:28:37.989328+0000","last_unstale":"2023-12-20T04:28:39.667779+0000","last_undegraded":"2023-12-20T04:28:39.667779+0000","last_fullsized":"2023-12-20T04:28:39.667779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667598+0000","last_change":"2023-12-20T04:28:38.390252+0000","last_active":"2023-12-20T04:28:39.667598+0000","last_peered":"2023-12-20T04:28:39.667598+0000","last_clean":"2023-12-20T04:28:39.667598+0000","last_became_active":"2023-12-20T04:28:38.389945+0000","last_became_peered":"2023-12-20T04:28:38.389945+0000","last_unstale":"2023-12-20T04:28:39.667598+0000","last_undegraded":"2023-12-20T04:28:39.667598+0000","last_fullsized":"2023-12-20T04:28:39.667598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:11:32.618631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666289+0000","last_change":"2023-12-20T04:28:37.979978+0000","last_active":"2023-12-20T04:28:39.666289+0000","last_peered":"2023-12-20T04:28:39.666289+0000","last_clean":"2023-12-20T04:28:39.666289+0000","last_became_active":"2023-12-20T04:28:37.979639+0000","last_became_peered":"2023-12-20T04:28:37.979639+0000","last_unstale":"2023-12-20T04:28:39.666289+0000","last_undegraded":"2023-12-20T04:28:39.666289+0000","last_fullsized":"2023-12-20T04:28:39.666289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666242+0000","last_change":"2023-12-20T04:28:38.400246+0000","last_active":"2023-12-20T04:28:39.666242+0000","last_peered":"2023-12-20T04:28:39.666242+0000","last_clean":"2023-12-20T04:28:39.666242+0000","last_became_active":"2023-12-20T04:28:38.399932+0000","last_became_peered":"2023-12-20T04:28:38.399932+0000","last_unstale":"2023-12-20T04:28:39.666242+0000","last_undegraded":"2023-12-20T04:28:39.666242+0000","last_fullsized":"2023-12-20T04:28:39.666242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:46:19.275821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386216+0000","last_change":"2023-12-20T04:28:37.985893+0000","last_active":"2023-12-20T04:28:39.386216+0000","last_peered":"2023-12-20T04:28:39.386216+0000","last_clean":"2023-12-20T04:28:39.386216+0000","last_became_active":"2023-12-20T04:28:37.985605+0000","last_became_peered":"2023-12-20T04:28:37.985605+0000","last_unstale":"2023-12-20T04:28:39.386216+0000","last_undegraded":"2023-12-20T04:28:39.386216+0000","last_fullsized":"2023-12-20T04:28:39.386216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667505+0000","last_change":"2023-12-20T04:28:38.409446+0000","last_active":"2023-12-20T04:28:39.667505+0000","last_peered":"2023-12-20T04:28:39.667505+0000","last_clean":"2023-12-20T04:28:39.667505+0000","last_became_active":"2023-12-20T04:28:38.409174+0000","last_became_peered":"2023-12-20T04:28:38.409174+0000","last_unstale":"2023-12-20T04:28:39.667505+0000","last_undegraded":"2023-12-20T04:28:39.667505+0000","last_fullsized":"2023-12-20T04:28:39.667505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:51.073749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667680+0000","last_change":"2023-12-20T04:28:37.986147+0000","last_active":"2023-12-20T04:28:39.667680+0000","last_peered":"2023-12-20T04:28:39.667680+0000","last_clean":"2023-12-20T04:28:39.667680+0000","last_became_active":"2023-12-20T04:28:37.985323+0000","last_became_peered":"2023-12-20T04:28:37.985323+0000","last_unstale":"2023-12-20T04:28:39.667680+0000","last_undegraded":"2023-12-20T04:28:39.667680+0000","last_fullsized":"2023-12-20T04:28:39.667680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667652+0000","last_change":"2023-12-20T04:28:38.403115+0000","last_active":"2023-12-20T04:28:39.667652+0000","last_peered":"2023-12-20T04:28:39.667652+0000","last_clean":"2023-12-20T04:28:39.667652+0000","last_became_active":"2023-12-20T04:28:38.402521+0000","last_became_peered":"2023-12-20T04:28:38.402521+0000","last_unstale":"2023-12-20T04:28:39.667652+0000","last_undegraded":"2023-12-20T04:28:39.667652+0000","last_fullsized":"2023-12-20T04:28:39.667652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666046+0000","last_change":"2023-12-20T04:28:37.970816+0000","last_active":"2023-12-20T04:28:39.666046+0000","last_peered":"2023-12-20T04:28:39.666046+0000","last_clean":"2023-12-20T04:28:39.666046+0000","last_became_active":"2023-12-20T04:28:37.970330+0000","last_became_peered":"2023-12-20T04:28:37.970330+0000","last_unstale":"2023-12-20T04:28:39.666046+0000","last_undegraded":"2023-12-20T04:28:39.666046+0000","last_fullsized":"2023-12-20T04:28:39.666046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667464+0000","last_change":"2023-12-20T04:28:38.398095+0000","last_active":"2023-12-20T04:28:39.667464+0000","last_peered":"2023-12-20T04:28:39.667464+0000","last_clean":"2023-12-20T04:28:39.667464+0000","last_became_active":"2023-12-20T04:28:38.397747+0000","last_became_peered":"2023-12-20T04:28:38.397747+0000","last_unstale":"2023-12-20T04:28:39.667464+0000","last_undegraded":"2023-12-20T04:28:39.667464+0000","last_fullsized":"2023-12-20T04:28:39.667464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:15:10.280675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386532+0000","last_change":"2023-12-20T04:28:37.980101+0000","last_active":"2023-12-20T04:28:39.386532+0000","last_peered":"2023-12-20T04:28:39.386532+0000","last_clean":"2023-12-20T04:28:39.386532+0000","last_became_active":"2023-12-20T04:28:37.979770+0000","last_became_peered":"2023-12-20T04:28:37.979770+0000","last_unstale":"2023-12-20T04:28:39.386532+0000","last_undegraded":"2023-12-20T04:28:39.386532+0000","last_fullsized":"2023-12-20T04:28:39.386532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667762+0000","last_change":"2023-12-20T04:28:38.409708+0000","last_active":"2023-12-20T04:28:39.667762+0000","last_peered":"2023-12-20T04:28:39.667762+0000","last_clean":"2023-12-20T04:28:39.667762+0000","last_became_active":"2023-12-20T04:28:38.408900+0000","last_became_peered":"2023-12-20T04:28:38.408900+0000","last_unstale":"2023-12-20T04:28:39.667762+0000","last_undegraded":"2023-12-20T04:28:39.667762+0000","last_fullsized":"2023-12-20T04:28:39.667762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:13.592803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666173+0000","last_change":"2023-12-20T04:28:37.985187+0000","last_active":"2023-12-20T04:28:39.666173+0000","last_peered":"2023-12-20T04:28:39.666173+0000","last_clean":"2023-12-20T04:28:39.666173+0000","last_became_active":"2023-12-20T04:28:37.984909+0000","last_became_peered":"2023-12-20T04:28:37.984909+0000","last_unstale":"2023-12-20T04:28:39.666173+0000","last_undegraded":"2023-12-20T04:28:39.666173+0000","last_fullsized":"2023-12-20T04:28:39.666173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667848+0000","last_change":"2023-12-20T04:28:38.397685+0000","last_active":"2023-12-20T04:28:39.667848+0000","last_peered":"2023-12-20T04:28:39.667848+0000","last_clean":"2023-12-20T04:28:39.667848+0000","last_became_active":"2023-12-20T04:28:38.396818+0000","last_became_peered":"2023-12-20T04:28:38.396818+0000","last_unstale":"2023-12-20T04:28:39.667848+0000","last_undegraded":"2023-12-20T04:28:39.667848+0000","last_fullsized":"2023-12-20T04:28:39.667848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:09.976618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379930+0000","last_change":"2023-12-20T04:28:37.981598+0000","last_active":"2023-12-20T04:28:39.379930+0000","last_peered":"2023-12-20T04:28:39.379930+0000","last_clean":"2023-12-20T04:28:39.379930+0000","last_became_active":"2023-12-20T04:28:37.981319+0000","last_became_peered":"2023-12-20T04:28:37.981319+0000","last_unstale":"2023-12-20T04:28:39.379930+0000","last_undegraded":"2023-12-20T04:28:39.379930+0000","last_fullsized":"2023-12-20T04:28:39.379930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385613+0000","last_change":"2023-12-20T04:28:38.385270+0000","last_active":"2023-12-20T04:28:39.385613+0000","last_peered":"2023-12-20T04:28:39.385613+0000","last_clean":"2023-12-20T04:28:39.385613+0000","last_became_active":"2023-12-20T04:28:38.384740+0000","last_became_peered":"2023-12-20T04:28:38.384740+0000","last_unstale":"2023-12-20T04:28:39.385613+0000","last_undegraded":"2023-12-20T04:28:39.385613+0000","last_fullsized":"2023-12-20T04:28:39.385613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:13.592803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666105+0000","last_change":"2023-12-20T04:28:37.991119+0000","last_active":"2023-12-20T04:28:39.666105+0000","last_peered":"2023-12-20T04:28:39.666105+0000","last_clean":"2023-12-20T04:28:39.666105+0000","last_became_active":"2023-12-20T04:28:37.990872+0000","last_became_peered":"2023-12-20T04:28:37.990872+0000","last_unstale":"2023-12-20T04:28:39.666105+0000","last_undegraded":"2023-12-20T04:28:39.666105+0000","last_fullsized":"2023-12-20T04:28:39.666105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667615+0000","last_change":"2023-12-20T04:28:38.394920+0000","last_active":"2023-12-20T04:28:39.667615+0000","last_peered":"2023-12-20T04:28:39.667615+0000","last_clean":"2023-12-20T04:28:39.667615+0000","last_became_active":"2023-12-20T04:28:38.394396+0000","last_became_peered":"2023-12-20T04:28:38.394396+0000","last_unstale":"2023-12-20T04:28:39.667615+0000","last_undegraded":"2023-12-20T04:28:39.667615+0000","last_fullsized":"2023-12-20T04:28:39.667615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.376833+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:28:38.376833+0000","last_peered":"2023-12-20T04:28:38.376833+0000","last_clean":"2023-12-20T04:28:38.376833+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:28:38.376833+0000","last_undegraded":"2023-12-20T04:28:38.376833+0000","last_fullsized":"2023-12-20T04:28:38.376833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386014+0000","last_change":"2023-12-20T04:28:37.978792+0000","last_active":"2023-12-20T04:28:39.386014+0000","last_peered":"2023-12-20T04:28:39.386014+0000","last_clean":"2023-12-20T04:28:39.386014+0000","last_became_active":"2023-12-20T04:28:37.978509+0000","last_became_peered":"2023-12-20T04:28:37.978509+0000","last_unstale":"2023-12-20T04:28:39.386014+0000","last_undegraded":"2023-12-20T04:28:39.386014+0000","last_fullsized":"2023-12-20T04:28:39.386014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670377+0000","last_change":"2023-12-20T04:28:38.400189+0000","last_active":"2023-12-20T04:28:39.670377+0000","last_peered":"2023-12-20T04:28:39.670377+0000","last_clean":"2023-12-20T04:28:39.670377+0000","last_became_active":"2023-12-20T04:28:38.399902+0000","last_became_peered":"2023-12-20T04:28:38.399902+0000","last_unstale":"2023-12-20T04:28:39.670377+0000","last_undegraded":"2023-12-20T04:28:39.670377+0000","last_fullsized":"2023-12-20T04:28:39.670377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:06.804397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667400+0000","last_change":"2023-12-20T04:28:37.987850+0000","last_active":"2023-12-20T04:28:39.667400+0000","last_peered":"2023-12-20T04:28:39.667400+0000","last_clean":"2023-12-20T04:28:39.667400+0000","last_became_active":"2023-12-20T04:28:37.986972+0000","last_became_peered":"2023-12-20T04:28:37.986972+0000","last_unstale":"2023-12-20T04:28:39.667400+0000","last_undegraded":"2023-12-20T04:28:39.667400+0000","last_fullsized":"2023-12-20T04:28:39.667400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:19.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666208+0000","last_change":"2023-12-20T04:28:38.397621+0000","last_active":"2023-12-20T04:28:39.666208+0000","last_peered":"2023-12-20T04:28:39.666208+0000","last_clean":"2023-12-20T04:28:39.666208+0000","last_became_active":"2023-12-20T04:28:38.397311+0000","last_became_peered":"2023-12-20T04:28:38.397311+0000","last_unstale":"2023-12-20T04:28:39.666208+0000","last_undegraded":"2023-12-20T04:28:39.666208+0000","last_fullsized":"2023-12-20T04:28:39.666208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:26:59.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666090+0000","last_change":"2023-12-20T04:28:37.982113+0000","last_active":"2023-12-20T04:28:39.666090+0000","last_peered":"2023-12-20T04:28:39.666090+0000","last_clean":"2023-12-20T04:28:39.666090+0000","last_became_active":"2023-12-20T04:28:37.981812+0000","last_became_peered":"2023-12-20T04:28:37.981812+0000","last_unstale":"2023-12-20T04:28:39.666090+0000","last_undegraded":"2023-12-20T04:28:39.666090+0000","last_fullsized":"2023-12-20T04:28:39.666090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379680+0000","last_change":"2023-12-20T04:28:38.383867+0000","last_active":"2023-12-20T04:28:39.379680+0000","last_peered":"2023-12-20T04:28:39.379680+0000","last_clean":"2023-12-20T04:28:39.379680+0000","last_became_active":"2023-12-20T04:28:38.383222+0000","last_became_peered":"2023-12-20T04:28:38.383222+0000","last_unstale":"2023-12-20T04:28:39.379680+0000","last_undegraded":"2023-12-20T04:28:39.379680+0000","last_fullsized":"2023-12-20T04:28:39.379680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386301+0000","last_change":"2023-12-20T04:28:37.981934+0000","last_active":"2023-12-20T04:28:39.386301+0000","last_peered":"2023-12-20T04:28:39.386301+0000","last_clean":"2023-12-20T04:28:39.386301+0000","last_became_active":"2023-12-20T04:28:37.981000+0000","last_became_peered":"2023-12-20T04:28:37.981000+0000","last_unstale":"2023-12-20T04:28:39.386301+0000","last_undegraded":"2023-12-20T04:28:39.386301+0000","last_fullsized":"2023-12-20T04:28:39.386301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386046+0000","last_change":"2023-12-20T04:28:38.405946+0000","last_active":"2023-12-20T04:28:39.386046+0000","last_peered":"2023-12-20T04:28:39.386046+0000","last_clean":"2023-12-20T04:28:39.386046+0000","last_became_active":"2023-12-20T04:28:38.405609+0000","last_became_peered":"2023-12-20T04:28:38.405609+0000","last_unstale":"2023-12-20T04:28:39.386046+0000","last_undegraded":"2023-12-20T04:28:39.386046+0000","last_fullsized":"2023-12-20T04:28:39.386046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379821+0000","last_change":"2023-12-20T04:28:37.980797+0000","last_active":"2023-12-20T04:28:39.379821+0000","last_peered":"2023-12-20T04:28:39.379821+0000","last_clean":"2023-12-20T04:28:39.379821+0000","last_became_active":"2023-12-20T04:28:37.980430+0000","last_became_peered":"2023-12-20T04:28:37.980430+0000","last_unstale":"2023-12-20T04:28:39.379821+0000","last_undegraded":"2023-12-20T04:28:39.379821+0000","last_fullsized":"2023-12-20T04:28:39.379821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:19.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386153+0000","last_change":"2023-12-20T04:28:38.389261+0000","last_active":"2023-12-20T04:28:39.386153+0000","last_peered":"2023-12-20T04:28:39.386153+0000","last_clean":"2023-12-20T04:28:39.386153+0000","last_became_active":"2023-12-20T04:28:38.388004+0000","last_became_peered":"2023-12-20T04:28:38.388004+0000","last_unstale":"2023-12-20T04:28:39.386153+0000","last_undegraded":"2023-12-20T04:28:39.386153+0000","last_fullsized":"2023-12-20T04:28:39.386153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667746+0000","last_change":"2023-12-20T04:28:37.991195+0000","last_active":"2023-12-20T04:28:39.667746+0000","last_peered":"2023-12-20T04:28:39.667746+0000","last_clean":"2023-12-20T04:28:39.667746+0000","last_became_active":"2023-12-20T04:28:37.990860+0000","last_became_peered":"2023-12-20T04:28:37.990860+0000","last_unstale":"2023-12-20T04:28:39.667746+0000","last_undegraded":"2023-12-20T04:28:39.667746+0000","last_fullsized":"2023-12-20T04:28:39.667746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:11.027603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379827+0000","last_change":"2023-12-20T04:28:38.377842+0000","last_active":"2023-12-20T04:28:39.379827+0000","last_peered":"2023-12-20T04:28:39.379827+0000","last_clean":"2023-12-20T04:28:39.379827+0000","last_became_active":"2023-12-20T04:28:38.377525+0000","last_became_peered":"2023-12-20T04:28:38.377525+0000","last_unstale":"2023-12-20T04:28:39.379827+0000","last_undegraded":"2023-12-20T04:28:39.379827+0000","last_fullsized":"2023-12-20T04:28:39.379827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.674031+0000","last_change":"2023-12-20T04:28:37.995878+0000","last_active":"2023-12-20T04:28:39.674031+0000","last_peered":"2023-12-20T04:28:39.674031+0000","last_clean":"2023-12-20T04:28:39.674031+0000","last_became_active":"2023-12-20T04:28:37.995604+0000","last_became_peered":"2023-12-20T04:28:37.995604+0000","last_unstale":"2023-12-20T04:28:39.674031+0000","last_undegraded":"2023-12-20T04:28:39.674031+0000","last_fullsized":"2023-12-20T04:28:39.674031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670458+0000","last_change":"2023-12-20T04:28:38.398433+0000","last_active":"2023-12-20T04:28:39.670458+0000","last_peered":"2023-12-20T04:28:39.670458+0000","last_clean":"2023-12-20T04:28:39.670458+0000","last_became_active":"2023-12-20T04:28:38.398247+0000","last_became_peered":"2023-12-20T04:28:38.398247+0000","last_unstale":"2023-12-20T04:28:39.670458+0000","last_undegraded":"2023-12-20T04:28:39.670458+0000","last_fullsized":"2023-12-20T04:28:39.670458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:48:18.553482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667884+0000","last_change":"2023-12-20T04:28:37.993415+0000","last_active":"2023-12-20T04:28:39.667884+0000","last_peered":"2023-12-20T04:28:39.667884+0000","last_clean":"2023-12-20T04:28:39.667884+0000","last_became_active":"2023-12-20T04:28:37.993209+0000","last_became_peered":"2023-12-20T04:28:37.993209+0000","last_unstale":"2023-12-20T04:28:39.667884+0000","last_undegraded":"2023-12-20T04:28:39.667884+0000","last_fullsized":"2023-12-20T04:28:39.667884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666307+0000","last_change":"2023-12-20T04:28:38.388800+0000","last_active":"2023-12-20T04:28:39.666307+0000","last_peered":"2023-12-20T04:28:39.666307+0000","last_clean":"2023-12-20T04:28:39.666307+0000","last_became_active":"2023-12-20T04:28:38.383908+0000","last_became_peered":"2023-12-20T04:28:38.383908+0000","last_unstale":"2023-12-20T04:28:39.666307+0000","last_undegraded":"2023-12-20T04:28:39.666307+0000","last_fullsized":"2023-12-20T04:28:39.666307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667371+0000","last_change":"2023-12-20T04:28:37.996076+0000","last_active":"2023-12-20T04:28:39.667371+0000","last_peered":"2023-12-20T04:28:39.667371+0000","last_clean":"2023-12-20T04:28:39.667371+0000","last_became_active":"2023-12-20T04:28:37.995770+0000","last_became_peered":"2023-12-20T04:28:37.995770+0000","last_unstale":"2023-12-20T04:28:39.667371+0000","last_undegraded":"2023-12-20T04:28:39.667371+0000","last_fullsized":"2023-12-20T04:28:39.667371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:34.884350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385884+0000","last_change":"2023-12-20T04:28:38.393602+0000","last_active":"2023-12-20T04:28:39.385884+0000","last_peered":"2023-12-20T04:28:39.385884+0000","last_clean":"2023-12-20T04:28:39.385884+0000","last_became_active":"2023-12-20T04:28:38.392277+0000","last_became_peered":"2023-12-20T04:28:38.392277+0000","last_unstale":"2023-12-20T04:28:39.385884+0000","last_undegraded":"2023-12-20T04:28:39.385884+0000","last_fullsized":"2023-12-20T04:28:39.385884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:20.635968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379704+0000","last_change":"2023-12-20T04:28:37.987522+0000","last_active":"2023-12-20T04:28:39.379704+0000","last_peered":"2023-12-20T04:28:39.379704+0000","last_clean":"2023-12-20T04:28:39.379704+0000","last_became_active":"2023-12-20T04:28:37.986906+0000","last_became_peered":"2023-12-20T04:28:37.986906+0000","last_unstale":"2023-12-20T04:28:39.379704+0000","last_undegraded":"2023-12-20T04:28:39.379704+0000","last_fullsized":"2023-12-20T04:28:39.379704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:34.884350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670333+0000","last_change":"2023-12-20T04:28:38.382504+0000","last_active":"2023-12-20T04:28:39.670333+0000","last_peered":"2023-12-20T04:28:39.670333+0000","last_clean":"2023-12-20T04:28:39.670333+0000","last_became_active":"2023-12-20T04:28:38.382224+0000","last_became_peered":"2023-12-20T04:28:38.382224+0000","last_unstale":"2023-12-20T04:28:39.670333+0000","last_undegraded":"2023-12-20T04:28:39.670333+0000","last_fullsized":"2023-12-20T04:28:39.670333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379771+0000","last_change":"2023-12-20T04:28:37.985344+0000","last_active":"2023-12-20T04:28:39.379771+0000","last_peered":"2023-12-20T04:28:39.379771+0000","last_clean":"2023-12-20T04:28:39.379771+0000","last_became_active":"2023-12-20T04:28:37.985074+0000","last_became_peered":"2023-12-20T04:28:37.985074+0000","last_unstale":"2023-12-20T04:28:39.379771+0000","last_undegraded":"2023-12-20T04:28:39.379771+0000","last_fullsized":"2023-12-20T04:28:39.379771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670395+0000","last_change":"2023-12-20T04:28:38.402571+0000","last_active":"2023-12-20T04:28:39.670395+0000","last_peered":"2023-12-20T04:28:39.670395+0000","last_clean":"2023-12-20T04:28:39.670395+0000","last_became_active":"2023-12-20T04:28:38.402316+0000","last_became_peered":"2023-12-20T04:28:38.402316+0000","last_unstale":"2023-12-20T04:28:39.670395+0000","last_undegraded":"2023-12-20T04:28:39.670395+0000","last_fullsized":"2023-12-20T04:28:39.670395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667523+0000","last_change":"2023-12-20T04:28:37.991378+0000","last_active":"2023-12-20T04:28:39.667523+0000","last_peered":"2023-12-20T04:28:39.667523+0000","last_clean":"2023-12-20T04:28:39.667523+0000","last_became_active":"2023-12-20T04:28:37.990976+0000","last_became_peered":"2023-12-20T04:28:37.990976+0000","last_unstale":"2023-12-20T04:28:39.667523+0000","last_undegraded":"2023-12-20T04:28:39.667523+0000","last_fullsized":"2023-12-20T04:28:39.667523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:18.611871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667716+0000","last_change":"2023-12-20T04:28:38.411659+0000","last_active":"2023-12-20T04:28:39.667716+0000","last_peered":"2023-12-20T04:28:39.667716+0000","last_clean":"2023-12-20T04:28:39.667716+0000","last_became_active":"2023-12-20T04:28:38.411333+0000","last_became_peered":"2023-12-20T04:28:38.411333+0000","last_unstale":"2023-12-20T04:28:39.667716+0000","last_undegraded":"2023-12-20T04:28:39.667716+0000","last_fullsized":"2023-12-20T04:28:39.667716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:46:19.275821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386042+0000","last_change":"2023-12-20T04:28:38.392988+0000","last_active":"2023-12-20T04:28:39.386042+0000","last_peered":"2023-12-20T04:28:39.386042+0000","last_clean":"2023-12-20T04:28:39.386042+0000","last_became_active":"2023-12-20T04:28:38.392136+0000","last_became_peered":"2023-12-20T04:28:38.392136+0000","last_unstale":"2023-12-20T04:28:39.386042+0000","last_undegraded":"2023-12-20T04:28:39.386042+0000","last_fullsized":"2023-12-20T04:28:39.386042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667574+0000","last_change":"2023-12-20T04:28:37.991881+0000","last_active":"2023-12-20T04:28:39.667574+0000","last_peered":"2023-12-20T04:28:39.667574+0000","last_clean":"2023-12-20T04:28:39.667574+0000","last_became_active":"2023-12-20T04:28:37.991520+0000","last_became_peered":"2023-12-20T04:28:37.991520+0000","last_unstale":"2023-12-20T04:28:39.667574+0000","last_undegraded":"2023-12-20T04:28:39.667574+0000","last_fullsized":"2023-12-20T04:28:39.667574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385497+0000","last_change":"2023-12-20T04:28:38.398174+0000","last_active":"2023-12-20T04:28:39.385497+0000","last_peered":"2023-12-20T04:28:39.385497+0000","last_clean":"2023-12-20T04:28:39.385497+0000","last_became_active":"2023-12-20T04:28:38.397909+0000","last_became_peered":"2023-12-20T04:28:38.397909+0000","last_unstale":"2023-12-20T04:28:39.385497+0000","last_undegraded":"2023-12-20T04:28:39.385497+0000","last_fullsized":"2023-12-20T04:28:39.385497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667643+0000","last_change":"2023-12-20T04:28:37.997620+0000","last_active":"2023-12-20T04:28:39.667643+0000","last_peered":"2023-12-20T04:28:39.667643+0000","last_clean":"2023-12-20T04:28:39.667643+0000","last_became_active":"2023-12-20T04:28:37.997289+0000","last_became_peered":"2023-12-20T04:28:37.997289+0000","last_unstale":"2023-12-20T04:28:39.667643+0000","last_undegraded":"2023-12-20T04:28:39.667643+0000","last_fullsized":"2023-12-20T04:28:39.667643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:00.985871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666315+0000","last_change":"2023-12-20T04:28:38.403356+0000","last_active":"2023-12-20T04:28:39.666315+0000","last_peered":"2023-12-20T04:28:39.666315+0000","last_clean":"2023-12-20T04:28:39.666315+0000","last_became_active":"2023-12-20T04:28:38.402695+0000","last_became_peered":"2023-12-20T04:28:38.402695+0000","last_unstale":"2023-12-20T04:28:39.666315+0000","last_undegraded":"2023-12-20T04:28:39.666315+0000","last_fullsized":"2023-12-20T04:28:39.666315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670431+0000","last_change":"2023-12-20T04:28:37.986554+0000","last_active":"2023-12-20T04:28:39.670431+0000","last_peered":"2023-12-20T04:28:39.670431+0000","last_clean":"2023-12-20T04:28:39.670431+0000","last_became_active":"2023-12-20T04:28:37.985944+0000","last_became_peered":"2023-12-20T04:28:37.985944+0000","last_unstale":"2023-12-20T04:28:39.670431+0000","last_undegraded":"2023-12-20T04:28:39.670431+0000","last_fullsized":"2023-12-20T04:28:39.670431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670422+0000","last_change":"2023-12-20T04:28:38.402052+0000","last_active":"2023-12-20T04:28:39.670422+0000","last_peered":"2023-12-20T04:28:39.670422+0000","last_clean":"2023-12-20T04:28:39.670422+0000","last_became_active":"2023-12-20T04:28:38.401578+0000","last_became_peered":"2023-12-20T04:28:38.401578+0000","last_unstale":"2023-12-20T04:28:39.670422+0000","last_undegraded":"2023-12-20T04:28:39.670422+0000","last_fullsized":"2023-12-20T04:28:39.670422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666365+0000","last_change":"2023-12-20T04:28:37.991899+0000","last_active":"2023-12-20T04:28:39.666365+0000","last_peered":"2023-12-20T04:28:39.666365+0000","last_clean":"2023-12-20T04:28:39.666365+0000","last_became_active":"2023-12-20T04:28:37.991545+0000","last_became_peered":"2023-12-20T04:28:37.991545+0000","last_unstale":"2023-12-20T04:28:39.666365+0000","last_undegraded":"2023-12-20T04:28:39.666365+0000","last_fullsized":"2023-12-20T04:28:39.666365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.758}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31272,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340568,"statfs":{"total":96636764160,"available":96604741632,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0110000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9830000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6970000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31260,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340580,"statfs":{"total":96636764160,"available":96604753920,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31268,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340572,"statfs":{"total":96636764160,"available":96604745728,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":823296,"data_stored":666575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:28:48.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:28:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc59b196640 1 Processor -- start 2023-12-20T04:28:48.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc59b196640 1 -- start start 2023-12-20T04:28:48.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc59b196640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc5940af5e0 0x7fc5940af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc59b196640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc59415f110 0x7fc59415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc59b196640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc59415b990 con 0x7fc59406ce60 2023-12-20T04:28:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc59b196640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59415bbd0 con 0x7fc59415f110 2023-12-20T04:28:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc59b196640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc59415beb0 con 0x7fc5940af5e0 2023-12-20T04:28:48.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc599733640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc59415f110 0x7fc59415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc599733640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc59415f110 0x7fc59415c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48682/0 (socket says 172.21.15.32:48682) 2023-12-20T04:28:48.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc599733640 1 -- 172.21.15.32:0/3253440994 learned_addr learned my addr 172.21.15.32:0/3253440994 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:48.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc598f32640 1 --2- 172.21.15.32:0/3253440994 >> v2:172.21.15.39:3300/0 conn(0x7fc5940af5e0 0x7fc5940af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc599733640 1 -- 172.21.15.32:0/3253440994 >> v2:172.21.15.39:3300/0 conn(0x7fc5940af5e0 msgr2=0x7fc5940af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc599733640 1 --2- 172.21.15.32:0/3253440994 >> v2:172.21.15.39:3300/0 conn(0x7fc5940af5e0 0x7fc5940af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc599733640 1 -- 172.21.15.32:0/3253440994 >> v1:172.21.15.39:6789/0 conn(0x7fc59406ce60 legacy=0x7fc59415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.620+0000 7fc599733640 1 -- 172.21.15.32:0/3253440994 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc59415e910 con 0x7fc59415f110 2023-12-20T04:28:48.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc599733640 1 --2- 172.21.15.32:0/3253440994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc59415f110 0x7fc59415c440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc59406ac10 tx=0x7fc58c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff959f05f1a3e209 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:48.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc5937fe640 1 -- 172.21.15.32:0/3253440994 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0a5880 con 0x7fc59415f110 2023-12-20T04:28:48.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc5937fe640 1 -- 172.21.15.32:0/3253440994 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58c0a59e0 con 0x7fc59415f110 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc5937fe640 1 -- 172.21.15.32:0/3253440994 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58c0a5d70 con 0x7fc59415f110 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc59b196640 1 -- 172.21.15.32:0/3253440994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc59415f110 msgr2=0x7fc59415c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc59b196640 1 --2- 172.21.15.32:0/3253440994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc59415f110 0x7fc59415c440 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fc59406ac10 tx=0x7fc58c01c9a0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc59b196640 1 -- 172.21.15.32:0/3253440994 shutdown_connections 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc59b196640 1 --2- 172.21.15.32:0/3253440994 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc59415f110 0x7fc59415c440 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc59b196640 1 --2- 172.21.15.32:0/3253440994 >> v2:172.21.15.39:3300/0 conn(0x7fc5940af5e0 0x7fc5940af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc59b196640 1 -- 172.21.15.32:0/3253440994 >> 172.21.15.32:0/3253440994 conn(0x7fc59406a320 msgr2=0x7fc5940ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc59b196640 1 -- 172.21.15.32:0/3253440994 shutdown_connections 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.624+0000 7fc59b196640 1 -- 172.21.15.32:0/3253440994 wait complete. 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc59b196640 1 Processor -- start 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc59b196640 1 -- start start 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc59b196640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc59406ce60 0x7fc59414c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc59b196640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5940af5e0 0x7fc5940cddb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc59b196640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc59415f110 0x7fc594158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc59b196640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc59415a210 con 0x7fc59406ce60 2023-12-20T04:28:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc59b196640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5941f59d0 con 0x7fc5940af5e0 2023-12-20T04:28:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc59b196640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc594151ce0 con 0x7fc59415f110 2023-12-20T04:28:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc598f32640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc59406ce60 0x7fc59414c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc598f32640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc59406ce60 0x7fc59414c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48686/0 (socket says 172.21.15.32:48686) 2023-12-20T04:28:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc598f32640 1 -- 172.21.15.32:0/2008809162 learned_addr learned my addr 172.21.15.32:0/2008809162 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:48.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc593fff640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5940af5e0 0x7fc5940cddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc599733640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc59415f110 0x7fc594158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc598f32640 1 -- 172.21.15.32:0/2008809162 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc59415f110 msgr2=0x7fc594158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc598f32640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc59415f110 0x7fc594158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc598f32640 1 -- 172.21.15.32:0/2008809162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5940af5e0 msgr2=0x7fc5940cddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc598f32640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5940af5e0 0x7fc5940cddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc598f32640 1 -- 172.21.15.32:0/2008809162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc58c009b30 con 0x7fc59406ce60 2023-12-20T04:28:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc598f32640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc59406ce60 0x7fc59414c780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc58800e9b0 tx=0x7fc58800ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc5917fa640 1 -- 172.21.15.32:0/2008809162 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc588009670 con 0x7fc59406ce60 2023-12-20T04:28:48.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc5917fa640 1 -- 172.21.15.32:0/2008809162 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc58800c9a0 con 0x7fc59406ce60 2023-12-20T04:28:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc59b196640 1 -- 172.21.15.32:0/2008809162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc594151fc0 con 0x7fc59406ce60 2023-12-20T04:28:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc5917fa640 1 -- 172.21.15.32:0/2008809162 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc58800cd10 con 0x7fc59406ce60 2023-12-20T04:28:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.628+0000 7fc59b196640 1 -- 172.21.15.32:0/2008809162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc594152580 con 0x7fc59406ce60 2023-12-20T04:28:48.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.632+0000 7fc59b196640 1 -- 172.21.15.32:0/2008809162 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc59404e000 con 0x7fc59406ce60 2023-12-20T04:28:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.632+0000 7fc5917fa640 1 -- 172.21.15.32:0/2008809162 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc588013470 con 0x7fc59406ce60 2023-12-20T04:28:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.632+0000 7fc5917fa640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc578065a60 0x7fc578067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.632+0000 7fc5917fa640 1 -- 172.21.15.32:0/2008809162 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc588083700 con 0x7fc59406ce60 2023-12-20T04:28:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.632+0000 7fc593fff640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc578065a60 0x7fc578067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.632+0000 7fc593fff640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc578065a60 0x7fc578067f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc59415b020 tx=0x7fc580009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:48.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.636+0000 7fc5917fa640 1 -- 172.21.15.32:0/2008809162 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc588051160 con 0x7fc59406ce60 2023-12-20T04:28:48.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.796+0000 7fc59b196640 1 -- 172.21.15.32:0/2008809162 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc59415c190 con 0x7fc578065a60 2023-12-20T04:28:48.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc5917fa640 1 -- 172.21.15.32:0/2008809162 <== mgr.4104 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+264528 (secure 0 0 0) 0x7fc5800014f0 con 0x7fc578065a60 2023-12-20T04:28:48.806 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:48.808 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:28:48.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 -- 172.21.15.32:0/2008809162 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc578065a60 msgr2=0x7fc578067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc578065a60 0x7fc578067f20 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fc59415b020 tx=0x7fc580009040 comp rx=0 tx=0).stop 2023-12-20T04:28:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 -- 172.21.15.32:0/2008809162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc59406ce60 msgr2=0x7fc59414c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc59406ce60 0x7fc59414c780 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fc58800e9b0 tx=0x7fc58800ee70 comp rx=0 tx=0).stop 2023-12-20T04:28:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 -- 172.21.15.32:0/2008809162 shutdown_connections 2023-12-20T04:28:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc578065a60 0x7fc578067f20 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc59415f110 0x7fc594158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5940af5e0 0x7fc5940cddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 --2- 172.21.15.32:0/2008809162 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc59406ce60 0x7fc59414c780 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 -- 172.21.15.32:0/2008809162 >> 172.21.15.32:0/2008809162 conn(0x7fc59406a320 msgr2=0x7fc594164420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:48.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 -- 172.21.15.32:0/2008809162 shutdown_connections 2023-12-20T04:28:48.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:48.804+0000 7fc59b196640 1 -- 172.21.15.32:0/2008809162 wait complete. 2023-12-20T04:28:48.826 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":26,"stamp":"2023-12-20T04:28:48.129580+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":239016,"kb_used_data":2920,"kb_used_omap":0,"kb_used_meta":236032,"kb_avail":754735704,"statfs":{"total":773094113280,"available":772849360896,"internally_reserved":0,"allocated":2990080,"data_stored":1790012,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":241696768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.764412"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667779+0000","last_change":"2023-12-20T04:28:38.407268+0000","last_active":"2023-12-20T04:28:39.667779+0000","last_peered":"2023-12-20T04:28:39.667779+0000","last_clean":"2023-12-20T04:28:39.667779+0000","last_became_active":"2023-12-20T04:28:38.404491+0000","last_became_peered":"2023-12-20T04:28:38.404491+0000","last_unstale":"2023-12-20T04:28:39.667779+0000","last_undegraded":"2023-12-20T04:28:39.667779+0000","last_fullsized":"2023-12-20T04:28:39.667779+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:36.363864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379971+0000","last_change":"2023-12-20T04:28:38.390082+0000","last_active":"2023-12-20T04:28:39.379971+0000","last_peered":"2023-12-20T04:28:39.379971+0000","last_clean":"2023-12-20T04:28:39.379971+0000","last_became_active":"2023-12-20T04:28:38.389688+0000","last_became_peered":"2023-12-20T04:28:38.389688+0000","last_unstale":"2023-12-20T04:28:39.379971+0000","last_undegraded":"2023-12-20T04:28:39.379971+0000","last_fullsized":"2023-12-20T04:28:39.379971+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666298+0000","last_change":"2023-12-20T04:28:38.388981+0000","last_active":"2023-12-20T04:28:39.666298+0000","last_peered":"2023-12-20T04:28:39.666298+0000","last_clean":"2023-12-20T04:28:39.666298+0000","last_became_active":"2023-12-20T04:28:38.388734+0000","last_became_peered":"2023-12-20T04:28:38.388734+0000","last_unstale":"2023-12-20T04:28:39.666298+0000","last_undegraded":"2023-12-20T04:28:39.666298+0000","last_fullsized":"2023-12-20T04:28:39.666298+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385490+0000","last_change":"2023-12-20T04:28:38.384727+0000","last_active":"2023-12-20T04:28:39.385490+0000","last_peered":"2023-12-20T04:28:39.385490+0000","last_clean":"2023-12-20T04:28:39.385490+0000","last_became_active":"2023-12-20T04:28:38.384329+0000","last_became_peered":"2023-12-20T04:28:38.384329+0000","last_unstale":"2023-12-20T04:28:39.385490+0000","last_undegraded":"2023-12-20T04:28:39.385490+0000","last_fullsized":"2023-12-20T04:28:39.385490+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:11:32.618631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667485+0000","last_change":"2023-12-20T04:28:38.397089+0000","last_active":"2023-12-20T04:28:39.667485+0000","last_peered":"2023-12-20T04:28:39.667485+0000","last_clean":"2023-12-20T04:28:39.667485+0000","last_became_active":"2023-12-20T04:28:38.396774+0000","last_became_peered":"2023-12-20T04:28:38.396774+0000","last_unstale":"2023-12-20T04:28:39.667485+0000","last_undegraded":"2023-12-20T04:28:39.667485+0000","last_fullsized":"2023-12-20T04:28:39.667485+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386104+0000","last_change":"2023-12-20T04:28:38.395384+0000","last_active":"2023-12-20T04:28:39.386104+0000","last_peered":"2023-12-20T04:28:39.386104+0000","last_clean":"2023-12-20T04:28:39.386104+0000","last_became_active":"2023-12-20T04:28:38.394254+0000","last_became_peered":"2023-12-20T04:28:38.394254+0000","last_unstale":"2023-12-20T04:28:39.386104+0000","last_undegraded":"2023-12-20T04:28:39.386104+0000","last_fullsized":"2023-12-20T04:28:39.386104+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:55:26.717187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667445+0000","last_change":"2023-12-20T04:28:38.404597+0000","last_active":"2023-12-20T04:28:39.667445+0000","last_peered":"2023-12-20T04:28:39.667445+0000","last_clean":"2023-12-20T04:28:39.667445+0000","last_became_active":"2023-12-20T04:28:38.404108+0000","last_became_peered":"2023-12-20T04:28:38.404108+0000","last_unstale":"2023-12-20T04:28:39.667445+0000","last_undegraded":"2023-12-20T04:28:39.667445+0000","last_fullsized":"2023-12-20T04:28:39.667445+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:41.399788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670310+0000","last_change":"2023-12-20T04:28:38.385150+0000","last_active":"2023-12-20T04:28:39.670310+0000","last_peered":"2023-12-20T04:28:39.670310+0000","last_clean":"2023-12-20T04:28:39.670310+0000","last_became_active":"2023-12-20T04:28:38.384885+0000","last_became_peered":"2023-12-20T04:28:38.384885+0000","last_unstale":"2023-12-20T04:28:39.670310+0000","last_undegraded":"2023-12-20T04:28:39.670310+0000","last_fullsized":"2023-12-20T04:28:39.670310+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667828+0000","last_change":"2023-12-20T04:28:38.399891+0000","last_active":"2023-12-20T04:28:39.667828+0000","last_peered":"2023-12-20T04:28:39.667828+0000","last_clean":"2023-12-20T04:28:39.667828+0000","last_became_active":"2023-12-20T04:28:38.399599+0000","last_became_peered":"2023-12-20T04:28:38.399599+0000","last_unstale":"2023-12-20T04:28:39.667828+0000","last_undegraded":"2023-12-20T04:28:39.667828+0000","last_fullsized":"2023-12-20T04:28:39.667828+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:46:52.340773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667813+0000","last_change":"2023-12-20T04:28:38.398573+0000","last_active":"2023-12-20T04:28:39.667813+0000","last_peered":"2023-12-20T04:28:39.667813+0000","last_clean":"2023-12-20T04:28:39.667813+0000","last_became_active":"2023-12-20T04:28:38.398162+0000","last_became_peered":"2023-12-20T04:28:38.398162+0000","last_unstale":"2023-12-20T04:28:39.667813+0000","last_undegraded":"2023-12-20T04:28:39.667813+0000","last_fullsized":"2023-12-20T04:28:39.667813+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667799+0000","last_change":"2023-12-20T04:28:38.405208+0000","last_active":"2023-12-20T04:28:39.667799+0000","last_peered":"2023-12-20T04:28:39.667799+0000","last_clean":"2023-12-20T04:28:39.667799+0000","last_became_active":"2023-12-20T04:28:38.404933+0000","last_became_peered":"2023-12-20T04:28:38.404933+0000","last_unstale":"2023-12-20T04:28:39.667799+0000","last_undegraded":"2023-12-20T04:28:39.667799+0000","last_fullsized":"2023-12-20T04:28:39.667799+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:50.720992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386165+0000","last_change":"2023-12-20T04:28:38.412922+0000","last_active":"2023-12-20T04:28:39.386165+0000","last_peered":"2023-12-20T04:28:39.386165+0000","last_clean":"2023-12-20T04:28:39.386165+0000","last_became_active":"2023-12-20T04:28:38.412505+0000","last_became_peered":"2023-12-20T04:28:38.412505+0000","last_unstale":"2023-12-20T04:28:39.386165+0000","last_undegraded":"2023-12-20T04:28:39.386165+0000","last_fullsized":"2023-12-20T04:28:39.386165+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:04:59.607569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670386+0000","last_change":"2023-12-20T04:28:38.390684+0000","last_active":"2023-12-20T04:28:39.670386+0000","last_peered":"2023-12-20T04:28:39.670386+0000","last_clean":"2023-12-20T04:28:39.670386+0000","last_became_active":"2023-12-20T04:28:38.390483+0000","last_became_peered":"2023-12-20T04:28:38.390483+0000","last_unstale":"2023-12-20T04:28:39.670386+0000","last_undegraded":"2023-12-20T04:28:39.670386+0000","last_fullsized":"2023-12-20T04:28:39.670386+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:55:26.717187+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666230+0000","last_change":"2023-12-20T04:28:38.379167+0000","last_active":"2023-12-20T04:28:39.666230+0000","last_peered":"2023-12-20T04:28:39.666230+0000","last_clean":"2023-12-20T04:28:39.666230+0000","last_became_active":"2023-12-20T04:28:38.378677+0000","last_became_peered":"2023-12-20T04:28:38.378677+0000","last_unstale":"2023-12-20T04:28:39.666230+0000","last_undegraded":"2023-12-20T04:28:39.666230+0000","last_fullsized":"2023-12-20T04:28:39.666230+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379652+0000","last_change":"2023-12-20T04:28:38.391213+0000","last_active":"2023-12-20T04:28:39.379652+0000","last_peered":"2023-12-20T04:28:39.379652+0000","last_clean":"2023-12-20T04:28:39.379652+0000","last_became_active":"2023-12-20T04:28:38.390937+0000","last_became_peered":"2023-12-20T04:28:38.390937+0000","last_unstale":"2023-12-20T04:28:39.379652+0000","last_undegraded":"2023-12-20T04:28:39.379652+0000","last_fullsized":"2023-12-20T04:28:39.379652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385448+0000","last_change":"2023-12-20T04:28:38.403378+0000","last_active":"2023-12-20T04:28:39.385448+0000","last_peered":"2023-12-20T04:28:39.385448+0000","last_clean":"2023-12-20T04:28:39.385448+0000","last_became_active":"2023-12-20T04:28:38.403070+0000","last_became_peered":"2023-12-20T04:28:38.403070+0000","last_unstale":"2023-12-20T04:28:39.385448+0000","last_undegraded":"2023-12-20T04:28:39.385448+0000","last_fullsized":"2023-12-20T04:28:39.385448+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:48:18.553482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385977+0000","last_change":"2023-12-20T04:28:38.399610+0000","last_active":"2023-12-20T04:28:39.385977+0000","last_peered":"2023-12-20T04:28:39.385977+0000","last_clean":"2023-12-20T04:28:39.385977+0000","last_became_active":"2023-12-20T04:28:38.399061+0000","last_became_peered":"2023-12-20T04:28:38.399061+0000","last_unstale":"2023-12-20T04:28:39.385977+0000","last_undegraded":"2023-12-20T04:28:39.385977+0000","last_fullsized":"2023-12-20T04:28:39.385977+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666152+0000","last_change":"2023-12-20T04:28:38.398099+0000","last_active":"2023-12-20T04:28:39.666152+0000","last_peered":"2023-12-20T04:28:39.666152+0000","last_clean":"2023-12-20T04:28:39.666152+0000","last_became_active":"2023-12-20T04:28:38.397333+0000","last_became_peered":"2023-12-20T04:28:38.397333+0000","last_unstale":"2023-12-20T04:28:39.666152+0000","last_undegraded":"2023-12-20T04:28:39.666152+0000","last_fullsized":"2023-12-20T04:28:39.666152+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:12.041027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667726+0000","last_change":"2023-12-20T04:28:38.397182+0000","last_active":"2023-12-20T04:28:39.667726+0000","last_peered":"2023-12-20T04:28:39.667726+0000","last_clean":"2023-12-20T04:28:39.667726+0000","last_became_active":"2023-12-20T04:28:38.396800+0000","last_became_peered":"2023-12-20T04:28:38.396800+0000","last_unstale":"2023-12-20T04:28:39.667726+0000","last_undegraded":"2023-12-20T04:28:39.667726+0000","last_fullsized":"2023-12-20T04:28:39.667726+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667761+0000","last_change":"2023-12-20T04:28:38.410245+0000","last_active":"2023-12-20T04:28:39.667761+0000","last_peered":"2023-12-20T04:28:39.667761+0000","last_clean":"2023-12-20T04:28:39.667761+0000","last_became_active":"2023-12-20T04:28:38.403138+0000","last_became_peered":"2023-12-20T04:28:38.403138+0000","last_unstale":"2023-12-20T04:28:39.667761+0000","last_undegraded":"2023-12-20T04:28:39.667761+0000","last_fullsized":"2023-12-20T04:28:39.667761+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666258+0000","last_change":"2023-12-20T04:28:38.400262+0000","last_active":"2023-12-20T04:28:39.666258+0000","last_peered":"2023-12-20T04:28:39.666258+0000","last_clean":"2023-12-20T04:28:39.666258+0000","last_became_active":"2023-12-20T04:28:38.400021+0000","last_became_peered":"2023-12-20T04:28:38.400021+0000","last_unstale":"2023-12-20T04:28:39.666258+0000","last_undegraded":"2023-12-20T04:28:39.666258+0000","last_fullsized":"2023-12-20T04:28:39.666258+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386133+0000","last_change":"2023-12-20T04:28:38.397515+0000","last_active":"2023-12-20T04:28:39.386133+0000","last_peered":"2023-12-20T04:28:39.386133+0000","last_clean":"2023-12-20T04:28:39.386133+0000","last_became_active":"2023-12-20T04:28:38.397234+0000","last_became_peered":"2023-12-20T04:28:38.397234+0000","last_unstale":"2023-12-20T04:28:39.386133+0000","last_undegraded":"2023-12-20T04:28:39.386133+0000","last_fullsized":"2023-12-20T04:28:39.386133+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670343+0000","last_change":"2023-12-20T04:28:38.390587+0000","last_active":"2023-12-20T04:28:39.670343+0000","last_peered":"2023-12-20T04:28:39.670343+0000","last_clean":"2023-12-20T04:28:39.670343+0000","last_became_active":"2023-12-20T04:28:38.390256+0000","last_became_peered":"2023-12-20T04:28:38.390256+0000","last_unstale":"2023-12-20T04:28:39.670343+0000","last_undegraded":"2023-12-20T04:28:39.670343+0000","last_fullsized":"2023-12-20T04:28:39.670343+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385781+0000","last_change":"2023-12-20T04:28:38.390087+0000","last_active":"2023-12-20T04:28:39.385781+0000","last_peered":"2023-12-20T04:28:39.385781+0000","last_clean":"2023-12-20T04:28:39.385781+0000","last_became_active":"2023-12-20T04:28:38.389881+0000","last_became_peered":"2023-12-20T04:28:38.389881+0000","last_unstale":"2023-12-20T04:28:39.385781+0000","last_undegraded":"2023-12-20T04:28:39.385781+0000","last_fullsized":"2023-12-20T04:28:39.385781+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:09.976618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670450+0000","last_change":"2023-12-20T04:28:38.396543+0000","last_active":"2023-12-20T04:28:39.670450+0000","last_peered":"2023-12-20T04:28:39.670450+0000","last_clean":"2023-12-20T04:28:39.670450+0000","last_became_active":"2023-12-20T04:28:38.396247+0000","last_became_peered":"2023-12-20T04:28:38.396247+0000","last_unstale":"2023-12-20T04:28:39.670450+0000","last_undegraded":"2023-12-20T04:28:39.670450+0000","last_fullsized":"2023-12-20T04:28:39.670450+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667823+0000","last_change":"2023-12-20T04:28:38.401846+0000","last_active":"2023-12-20T04:28:39.667823+0000","last_peered":"2023-12-20T04:28:39.667823+0000","last_clean":"2023-12-20T04:28:39.667823+0000","last_became_active":"2023-12-20T04:28:38.401255+0000","last_became_peered":"2023-12-20T04:28:38.401255+0000","last_unstale":"2023-12-20T04:28:39.667823+0000","last_undegraded":"2023-12-20T04:28:39.667823+0000","last_fullsized":"2023-12-20T04:28:39.667823+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379844+0000","last_change":"2023-12-20T04:28:38.385251+0000","last_active":"2023-12-20T04:28:39.379844+0000","last_peered":"2023-12-20T04:28:39.379844+0000","last_clean":"2023-12-20T04:28:39.379844+0000","last_became_active":"2023-12-20T04:28:38.384990+0000","last_became_peered":"2023-12-20T04:28:38.384990+0000","last_unstale":"2023-12-20T04:28:39.379844+0000","last_undegraded":"2023-12-20T04:28:39.379844+0000","last_fullsized":"2023-12-20T04:28:39.379844+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670414+0000","last_change":"2023-12-20T04:28:38.394851+0000","last_active":"2023-12-20T04:28:39.670414+0000","last_peered":"2023-12-20T04:28:39.670414+0000","last_clean":"2023-12-20T04:28:39.670414+0000","last_became_active":"2023-12-20T04:28:38.394584+0000","last_became_peered":"2023-12-20T04:28:38.394584+0000","last_unstale":"2023-12-20T04:28:39.670414+0000","last_undegraded":"2023-12-20T04:28:39.670414+0000","last_fullsized":"2023-12-20T04:28:39.670414+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666271+0000","last_change":"2023-12-20T04:28:38.383584+0000","last_active":"2023-12-20T04:28:39.666271+0000","last_peered":"2023-12-20T04:28:39.666271+0000","last_clean":"2023-12-20T04:28:39.666271+0000","last_became_active":"2023-12-20T04:28:38.383310+0000","last_became_peered":"2023-12-20T04:28:38.383310+0000","last_unstale":"2023-12-20T04:28:39.666271+0000","last_undegraded":"2023-12-20T04:28:39.666271+0000","last_fullsized":"2023-12-20T04:28:39.666271+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386215+0000","last_change":"2023-12-20T04:28:38.388220+0000","last_active":"2023-12-20T04:28:39.386215+0000","last_peered":"2023-12-20T04:28:39.386215+0000","last_clean":"2023-12-20T04:28:39.386215+0000","last_became_active":"2023-12-20T04:28:38.386044+0000","last_became_peered":"2023-12-20T04:28:38.386044+0000","last_unstale":"2023-12-20T04:28:39.386215+0000","last_undegraded":"2023-12-20T04:28:39.386215+0000","last_fullsized":"2023-12-20T04:28:39.386215+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:51.073749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666072+0000","last_change":"2023-12-20T04:28:38.384460+0000","last_active":"2023-12-20T04:28:39.666072+0000","last_peered":"2023-12-20T04:28:39.666072+0000","last_clean":"2023-12-20T04:28:39.666072+0000","last_became_active":"2023-12-20T04:28:38.384190+0000","last_became_peered":"2023-12-20T04:28:38.384190+0000","last_unstale":"2023-12-20T04:28:39.666072+0000","last_undegraded":"2023-12-20T04:28:39.666072+0000","last_fullsized":"2023-12-20T04:28:39.666072+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667624+0000","last_change":"2023-12-20T04:28:38.403381+0000","last_active":"2023-12-20T04:28:39.667624+0000","last_peered":"2023-12-20T04:28:39.667624+0000","last_clean":"2023-12-20T04:28:39.667624+0000","last_became_active":"2023-12-20T04:28:38.402953+0000","last_became_peered":"2023-12-20T04:28:38.402953+0000","last_unstale":"2023-12-20T04:28:39.667624+0000","last_undegraded":"2023-12-20T04:28:39.667624+0000","last_fullsized":"2023-12-20T04:28:39.667624+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:23.707936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385741+0000","last_change":"2023-12-20T04:28:38.392967+0000","last_active":"2023-12-20T04:28:39.385741+0000","last_peered":"2023-12-20T04:28:39.385741+0000","last_clean":"2023-12-20T04:28:39.385741+0000","last_became_active":"2023-12-20T04:28:38.392657+0000","last_became_peered":"2023-12-20T04:28:38.392657+0000","last_unstale":"2023-12-20T04:28:39.385741+0000","last_undegraded":"2023-12-20T04:28:39.385741+0000","last_fullsized":"2023-12-20T04:28:39.385741+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:35.897773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1b","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670404+0000","last_change":"2023-12-20T04:28:37.983099+0000","last_active":"2023-12-20T04:28:39.670404+0000","last_peered":"2023-12-20T04:28:39.670404+0000","last_clean":"2023-12-20T04:28:39.670404+0000","last_became_active":"2023-12-20T04:28:37.982857+0000","last_became_peered":"2023-12-20T04:28:37.982857+0000","last_unstale":"2023-12-20T04:28:39.670404+0000","last_undegraded":"2023-12-20T04:28:39.670404+0000","last_fullsized":"2023-12-20T04:28:39.670404+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666279+0000","last_change":"2023-12-20T04:28:38.407295+0000","last_active":"2023-12-20T04:28:39.666279+0000","last_peered":"2023-12-20T04:28:39.666279+0000","last_clean":"2023-12-20T04:28:39.666279+0000","last_became_active":"2023-12-20T04:28:38.406963+0000","last_became_peered":"2023-12-20T04:28:38.406963+0000","last_unstale":"2023-12-20T04:28:39.666279+0000","last_undegraded":"2023-12-20T04:28:39.666279+0000","last_fullsized":"2023-12-20T04:28:39.666279+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:50.720992+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1a","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667740+0000","last_change":"2023-12-20T04:28:37.987627+0000","last_active":"2023-12-20T04:28:39.667740+0000","last_peered":"2023-12-20T04:28:39.667740+0000","last_clean":"2023-12-20T04:28:39.667740+0000","last_became_active":"2023-12-20T04:28:37.987309+0000","last_became_peered":"2023-12-20T04:28:37.987309+0000","last_unstale":"2023-12-20T04:28:39.667740+0000","last_undegraded":"2023-12-20T04:28:39.667740+0000","last_fullsized":"2023-12-20T04:28:39.667740+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670368+0000","last_change":"2023-12-20T04:28:37.996411+0000","last_active":"2023-12-20T04:28:39.670368+0000","last_peered":"2023-12-20T04:28:39.670368+0000","last_clean":"2023-12-20T04:28:39.670368+0000","last_became_active":"2023-12-20T04:28:37.996051+0000","last_became_peered":"2023-12-20T04:28:37.996051+0000","last_unstale":"2023-12-20T04:28:39.670368+0000","last_undegraded":"2023-12-20T04:28:39.670368+0000","last_fullsized":"2023-12-20T04:28:39.670368+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670324+0000","last_change":"2023-12-20T04:28:38.396714+0000","last_active":"2023-12-20T04:28:39.670324+0000","last_peered":"2023-12-20T04:28:39.670324+0000","last_clean":"2023-12-20T04:28:39.670324+0000","last_became_active":"2023-12-20T04:28:38.396337+0000","last_became_peered":"2023-12-20T04:28:38.396337+0000","last_unstale":"2023-12-20T04:28:39.670324+0000","last_undegraded":"2023-12-20T04:28:39.670324+0000","last_fullsized":"2023-12-20T04:28:39.670324+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385551+0000","last_change":"2023-12-20T04:28:37.982942+0000","last_active":"2023-12-20T04:28:39.385551+0000","last_peered":"2023-12-20T04:28:39.385551+0000","last_clean":"2023-12-20T04:28:39.385551+0000","last_became_active":"2023-12-20T04:28:37.982287+0000","last_became_peered":"2023-12-20T04:28:37.982287+0000","last_unstale":"2023-12-20T04:28:39.385551+0000","last_undegraded":"2023-12-20T04:28:39.385551+0000","last_fullsized":"2023-12-20T04:28:39.385551+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379613+0000","last_change":"2023-12-20T04:28:38.380510+0000","last_active":"2023-12-20T04:28:39.379613+0000","last_peered":"2023-12-20T04:28:39.379613+0000","last_clean":"2023-12-20T04:28:39.379613+0000","last_became_active":"2023-12-20T04:28:38.379896+0000","last_became_peered":"2023-12-20T04:28:38.379896+0000","last_unstale":"2023-12-20T04:28:39.379613+0000","last_undegraded":"2023-12-20T04:28:39.379613+0000","last_fullsized":"2023-12-20T04:28:39.379613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:49.972145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.17","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667837+0000","last_change":"2023-12-20T04:28:37.992369+0000","last_active":"2023-12-20T04:28:39.667837+0000","last_peered":"2023-12-20T04:28:39.667837+0000","last_clean":"2023-12-20T04:28:39.667837+0000","last_became_active":"2023-12-20T04:28:37.992065+0000","last_became_peered":"2023-12-20T04:28:37.992065+0000","last_unstale":"2023-12-20T04:28:39.667837+0000","last_undegraded":"2023-12-20T04:28:39.667837+0000","last_fullsized":"2023-12-20T04:28:39.667837+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385551+0000","last_change":"2023-12-20T04:28:38.390943+0000","last_active":"2023-12-20T04:28:39.385551+0000","last_peered":"2023-12-20T04:28:39.385551+0000","last_clean":"2023-12-20T04:28:39.385551+0000","last_became_active":"2023-12-20T04:28:38.390661+0000","last_became_peered":"2023-12-20T04:28:38.390661+0000","last_unstale":"2023-12-20T04:28:39.385551+0000","last_undegraded":"2023-12-20T04:28:39.385551+0000","last_fullsized":"2023-12-20T04:28:39.385551+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:06.804397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385691+0000","last_change":"2023-12-20T04:28:37.986949+0000","last_active":"2023-12-20T04:28:39.385691+0000","last_peered":"2023-12-20T04:28:39.385691+0000","last_clean":"2023-12-20T04:28:39.385691+0000","last_became_active":"2023-12-20T04:28:37.986664+0000","last_became_peered":"2023-12-20T04:28:37.986664+0000","last_unstale":"2023-12-20T04:28:39.385691+0000","last_undegraded":"2023-12-20T04:28:39.385691+0000","last_fullsized":"2023-12-20T04:28:39.385691+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666354+0000","last_change":"2023-12-20T04:28:38.403067+0000","last_active":"2023-12-20T04:28:39.666354+0000","last_peered":"2023-12-20T04:28:39.666354+0000","last_clean":"2023-12-20T04:28:39.666354+0000","last_became_active":"2023-12-20T04:28:38.402312+0000","last_became_peered":"2023-12-20T04:28:38.402312+0000","last_unstale":"2023-12-20T04:28:39.666354+0000","last_undegraded":"2023-12-20T04:28:39.666354+0000","last_fullsized":"2023-12-20T04:28:39.666354+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:23.372710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.15","version":"20'3","reported_seq":14,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379842+0000","last_change":"2023-12-20T04:28:37.989273+0000","last_active":"2023-12-20T04:28:39.379842+0000","last_peered":"2023-12-20T04:28:39.379842+0000","last_clean":"2023-12-20T04:28:39.379842+0000","last_became_active":"2023-12-20T04:28:37.988921+0000","last_became_peered":"2023-12-20T04:28:37.988921+0000","last_unstale":"2023-12-20T04:28:39.379842+0000","last_undegraded":"2023-12-20T04:28:39.379842+0000","last_fullsized":"2023-12-20T04:28:39.379842+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386234+0000","last_change":"2023-12-20T04:28:38.394680+0000","last_active":"2023-12-20T04:28:39.386234+0000","last_peered":"2023-12-20T04:28:39.386234+0000","last_clean":"2023-12-20T04:28:39.386234+0000","last_became_active":"2023-12-20T04:28:38.394369+0000","last_became_peered":"2023-12-20T04:28:38.394369+0000","last_unstale":"2023-12-20T04:28:39.386234+0000","last_undegraded":"2023-12-20T04:28:39.386234+0000","last_fullsized":"2023-12-20T04:28:39.386234+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:12.041027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.14","version":"20'6","reported_seq":17,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667779+0000","last_change":"2023-12-20T04:28:37.992630+0000","last_active":"2023-12-20T04:28:39.667779+0000","last_peered":"2023-12-20T04:28:39.667779+0000","last_clean":"2023-12-20T04:28:39.667779+0000","last_became_active":"2023-12-20T04:28:37.989328+0000","last_became_peered":"2023-12-20T04:28:37.989328+0000","last_unstale":"2023-12-20T04:28:39.667779+0000","last_undegraded":"2023-12-20T04:28:39.667779+0000","last_fullsized":"2023-12-20T04:28:39.667779+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667598+0000","last_change":"2023-12-20T04:28:38.390252+0000","last_active":"2023-12-20T04:28:39.667598+0000","last_peered":"2023-12-20T04:28:39.667598+0000","last_clean":"2023-12-20T04:28:39.667598+0000","last_became_active":"2023-12-20T04:28:38.389945+0000","last_became_peered":"2023-12-20T04:28:38.389945+0000","last_unstale":"2023-12-20T04:28:39.667598+0000","last_undegraded":"2023-12-20T04:28:39.667598+0000","last_fullsized":"2023-12-20T04:28:39.667598+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:11:32.618631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.13","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666289+0000","last_change":"2023-12-20T04:28:37.979978+0000","last_active":"2023-12-20T04:28:39.666289+0000","last_peered":"2023-12-20T04:28:39.666289+0000","last_clean":"2023-12-20T04:28:39.666289+0000","last_became_active":"2023-12-20T04:28:37.979639+0000","last_became_peered":"2023-12-20T04:28:37.979639+0000","last_unstale":"2023-12-20T04:28:39.666289+0000","last_undegraded":"2023-12-20T04:28:39.666289+0000","last_fullsized":"2023-12-20T04:28:39.666289+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666242+0000","last_change":"2023-12-20T04:28:38.400246+0000","last_active":"2023-12-20T04:28:39.666242+0000","last_peered":"2023-12-20T04:28:39.666242+0000","last_clean":"2023-12-20T04:28:39.666242+0000","last_became_active":"2023-12-20T04:28:38.399932+0000","last_became_peered":"2023-12-20T04:28:38.399932+0000","last_unstale":"2023-12-20T04:28:39.666242+0000","last_undegraded":"2023-12-20T04:28:39.666242+0000","last_fullsized":"2023-12-20T04:28:39.666242+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:46:19.275821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386216+0000","last_change":"2023-12-20T04:28:37.985893+0000","last_active":"2023-12-20T04:28:39.386216+0000","last_peered":"2023-12-20T04:28:39.386216+0000","last_clean":"2023-12-20T04:28:39.386216+0000","last_became_active":"2023-12-20T04:28:37.985605+0000","last_became_peered":"2023-12-20T04:28:37.985605+0000","last_unstale":"2023-12-20T04:28:39.386216+0000","last_undegraded":"2023-12-20T04:28:39.386216+0000","last_fullsized":"2023-12-20T04:28:39.386216+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667505+0000","last_change":"2023-12-20T04:28:38.409446+0000","last_active":"2023-12-20T04:28:39.667505+0000","last_peered":"2023-12-20T04:28:39.667505+0000","last_clean":"2023-12-20T04:28:39.667505+0000","last_became_active":"2023-12-20T04:28:38.409174+0000","last_became_peered":"2023-12-20T04:28:38.409174+0000","last_unstale":"2023-12-20T04:28:39.667505+0000","last_undegraded":"2023-12-20T04:28:39.667505+0000","last_fullsized":"2023-12-20T04:28:39.667505+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:51.073749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667680+0000","last_change":"2023-12-20T04:28:37.986147+0000","last_active":"2023-12-20T04:28:39.667680+0000","last_peered":"2023-12-20T04:28:39.667680+0000","last_clean":"2023-12-20T04:28:39.667680+0000","last_became_active":"2023-12-20T04:28:37.985323+0000","last_became_peered":"2023-12-20T04:28:37.985323+0000","last_unstale":"2023-12-20T04:28:39.667680+0000","last_undegraded":"2023-12-20T04:28:39.667680+0000","last_fullsized":"2023-12-20T04:28:39.667680+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667652+0000","last_change":"2023-12-20T04:28:38.403115+0000","last_active":"2023-12-20T04:28:39.667652+0000","last_peered":"2023-12-20T04:28:39.667652+0000","last_clean":"2023-12-20T04:28:39.667652+0000","last_became_active":"2023-12-20T04:28:38.402521+0000","last_became_peered":"2023-12-20T04:28:38.402521+0000","last_unstale":"2023-12-20T04:28:39.667652+0000","last_undegraded":"2023-12-20T04:28:39.667652+0000","last_fullsized":"2023-12-20T04:28:39.667652+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"20'8","reported_seq":19,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666046+0000","last_change":"2023-12-20T04:28:37.970816+0000","last_active":"2023-12-20T04:28:39.666046+0000","last_peered":"2023-12-20T04:28:39.666046+0000","last_clean":"2023-12-20T04:28:39.666046+0000","last_became_active":"2023-12-20T04:28:37.970330+0000","last_became_peered":"2023-12-20T04:28:37.970330+0000","last_unstale":"2023-12-20T04:28:39.666046+0000","last_undegraded":"2023-12-20T04:28:39.666046+0000","last_fullsized":"2023-12-20T04:28:39.666046+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667464+0000","last_change":"2023-12-20T04:28:38.398095+0000","last_active":"2023-12-20T04:28:39.667464+0000","last_peered":"2023-12-20T04:28:39.667464+0000","last_clean":"2023-12-20T04:28:39.667464+0000","last_became_active":"2023-12-20T04:28:38.397747+0000","last_became_peered":"2023-12-20T04:28:38.397747+0000","last_unstale":"2023-12-20T04:28:39.667464+0000","last_undegraded":"2023-12-20T04:28:39.667464+0000","last_fullsized":"2023-12-20T04:28:39.667464+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:15:10.280675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.f","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386532+0000","last_change":"2023-12-20T04:28:37.980101+0000","last_active":"2023-12-20T04:28:39.386532+0000","last_peered":"2023-12-20T04:28:39.386532+0000","last_clean":"2023-12-20T04:28:39.386532+0000","last_became_active":"2023-12-20T04:28:37.979770+0000","last_became_peered":"2023-12-20T04:28:37.979770+0000","last_unstale":"2023-12-20T04:28:39.386532+0000","last_undegraded":"2023-12-20T04:28:39.386532+0000","last_fullsized":"2023-12-20T04:28:39.386532+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667762+0000","last_change":"2023-12-20T04:28:38.409708+0000","last_active":"2023-12-20T04:28:39.667762+0000","last_peered":"2023-12-20T04:28:39.667762+0000","last_clean":"2023-12-20T04:28:39.667762+0000","last_became_active":"2023-12-20T04:28:38.408900+0000","last_became_peered":"2023-12-20T04:28:38.408900+0000","last_unstale":"2023-12-20T04:28:39.667762+0000","last_undegraded":"2023-12-20T04:28:39.667762+0000","last_fullsized":"2023-12-20T04:28:39.667762+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:13.592803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.e","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666173+0000","last_change":"2023-12-20T04:28:37.985187+0000","last_active":"2023-12-20T04:28:39.666173+0000","last_peered":"2023-12-20T04:28:39.666173+0000","last_clean":"2023-12-20T04:28:39.666173+0000","last_became_active":"2023-12-20T04:28:37.984909+0000","last_became_peered":"2023-12-20T04:28:37.984909+0000","last_unstale":"2023-12-20T04:28:39.666173+0000","last_undegraded":"2023-12-20T04:28:39.666173+0000","last_fullsized":"2023-12-20T04:28:39.666173+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667848+0000","last_change":"2023-12-20T04:28:38.397685+0000","last_active":"2023-12-20T04:28:39.667848+0000","last_peered":"2023-12-20T04:28:39.667848+0000","last_clean":"2023-12-20T04:28:39.667848+0000","last_became_active":"2023-12-20T04:28:38.396818+0000","last_became_peered":"2023-12-20T04:28:38.396818+0000","last_unstale":"2023-12-20T04:28:39.667848+0000","last_undegraded":"2023-12-20T04:28:39.667848+0000","last_fullsized":"2023-12-20T04:28:39.667848+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:09.976618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.d","version":"20'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379930+0000","last_change":"2023-12-20T04:28:37.981598+0000","last_active":"2023-12-20T04:28:39.379930+0000","last_peered":"2023-12-20T04:28:39.379930+0000","last_clean":"2023-12-20T04:28:39.379930+0000","last_became_active":"2023-12-20T04:28:37.981319+0000","last_became_peered":"2023-12-20T04:28:37.981319+0000","last_unstale":"2023-12-20T04:28:39.379930+0000","last_undegraded":"2023-12-20T04:28:39.379930+0000","last_fullsized":"2023-12-20T04:28:39.379930+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385613+0000","last_change":"2023-12-20T04:28:38.385270+0000","last_active":"2023-12-20T04:28:39.385613+0000","last_peered":"2023-12-20T04:28:39.385613+0000","last_clean":"2023-12-20T04:28:39.385613+0000","last_became_active":"2023-12-20T04:28:38.384740+0000","last_became_peered":"2023-12-20T04:28:38.384740+0000","last_unstale":"2023-12-20T04:28:39.385613+0000","last_undegraded":"2023-12-20T04:28:39.385613+0000","last_fullsized":"2023-12-20T04:28:39.385613+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:13.592803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666105+0000","last_change":"2023-12-20T04:28:37.991119+0000","last_active":"2023-12-20T04:28:39.666105+0000","last_peered":"2023-12-20T04:28:39.666105+0000","last_clean":"2023-12-20T04:28:39.666105+0000","last_became_active":"2023-12-20T04:28:37.990872+0000","last_became_peered":"2023-12-20T04:28:37.990872+0000","last_unstale":"2023-12-20T04:28:39.666105+0000","last_undegraded":"2023-12-20T04:28:39.666105+0000","last_fullsized":"2023-12-20T04:28:39.666105+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667615+0000","last_change":"2023-12-20T04:28:38.394920+0000","last_active":"2023-12-20T04:28:39.667615+0000","last_peered":"2023-12-20T04:28:39.667615+0000","last_clean":"2023-12-20T04:28:39.667615+0000","last_became_active":"2023-12-20T04:28:38.394396+0000","last_became_peered":"2023-12-20T04:28:38.394396+0000","last_unstale":"2023-12-20T04:28:39.667615+0000","last_undegraded":"2023-12-20T04:28:39.667615+0000","last_fullsized":"2023-12-20T04:28:39.667615+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:15.879363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":164,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:38.376833+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:28:38.376833+0000","last_peered":"2023-12-20T04:28:38.376833+0000","last_clean":"2023-12-20T04:28:38.376833+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:28:38.376833+0000","last_undegraded":"2023-12-20T04:28:38.376833+0000","last_fullsized":"2023-12-20T04:28:38.376833+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386014+0000","last_change":"2023-12-20T04:28:37.978792+0000","last_active":"2023-12-20T04:28:39.386014+0000","last_peered":"2023-12-20T04:28:39.386014+0000","last_clean":"2023-12-20T04:28:39.386014+0000","last_became_active":"2023-12-20T04:28:37.978509+0000","last_became_peered":"2023-12-20T04:28:37.978509+0000","last_unstale":"2023-12-20T04:28:39.386014+0000","last_undegraded":"2023-12-20T04:28:39.386014+0000","last_fullsized":"2023-12-20T04:28:39.386014+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:37.243233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670377+0000","last_change":"2023-12-20T04:28:38.400189+0000","last_active":"2023-12-20T04:28:39.670377+0000","last_peered":"2023-12-20T04:28:39.670377+0000","last_clean":"2023-12-20T04:28:39.670377+0000","last_became_active":"2023-12-20T04:28:38.399902+0000","last_became_peered":"2023-12-20T04:28:38.399902+0000","last_unstale":"2023-12-20T04:28:39.670377+0000","last_undegraded":"2023-12-20T04:28:39.670377+0000","last_fullsized":"2023-12-20T04:28:39.670377+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:06.804397+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667400+0000","last_change":"2023-12-20T04:28:37.987850+0000","last_active":"2023-12-20T04:28:39.667400+0000","last_peered":"2023-12-20T04:28:39.667400+0000","last_clean":"2023-12-20T04:28:39.667400+0000","last_became_active":"2023-12-20T04:28:37.986972+0000","last_became_peered":"2023-12-20T04:28:37.986972+0000","last_unstale":"2023-12-20T04:28:39.667400+0000","last_undegraded":"2023-12-20T04:28:39.667400+0000","last_fullsized":"2023-12-20T04:28:39.667400+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:19.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666208+0000","last_change":"2023-12-20T04:28:38.397621+0000","last_active":"2023-12-20T04:28:39.666208+0000","last_peered":"2023-12-20T04:28:39.666208+0000","last_clean":"2023-12-20T04:28:39.666208+0000","last_became_active":"2023-12-20T04:28:38.397311+0000","last_became_peered":"2023-12-20T04:28:38.397311+0000","last_unstale":"2023-12-20T04:28:39.666208+0000","last_undegraded":"2023-12-20T04:28:39.666208+0000","last_fullsized":"2023-12-20T04:28:39.666208+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:26:59.477008+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666090+0000","last_change":"2023-12-20T04:28:37.982113+0000","last_active":"2023-12-20T04:28:39.666090+0000","last_peered":"2023-12-20T04:28:39.666090+0000","last_clean":"2023-12-20T04:28:39.666090+0000","last_became_active":"2023-12-20T04:28:37.981812+0000","last_became_peered":"2023-12-20T04:28:37.981812+0000","last_unstale":"2023-12-20T04:28:39.666090+0000","last_undegraded":"2023-12-20T04:28:39.666090+0000","last_fullsized":"2023-12-20T04:28:39.666090+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:45.720333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379680+0000","last_change":"2023-12-20T04:28:38.383867+0000","last_active":"2023-12-20T04:28:39.379680+0000","last_peered":"2023-12-20T04:28:39.379680+0000","last_clean":"2023-12-20T04:28:39.379680+0000","last_became_active":"2023-12-20T04:28:38.383222+0000","last_became_peered":"2023-12-20T04:28:38.383222+0000","last_unstale":"2023-12-20T04:28:39.379680+0000","last_undegraded":"2023-12-20T04:28:39.379680+0000","last_fullsized":"2023-12-20T04:28:39.379680+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:44.416542+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386301+0000","last_change":"2023-12-20T04:28:37.981934+0000","last_active":"2023-12-20T04:28:39.386301+0000","last_peered":"2023-12-20T04:28:39.386301+0000","last_clean":"2023-12-20T04:28:39.386301+0000","last_became_active":"2023-12-20T04:28:37.981000+0000","last_became_peered":"2023-12-20T04:28:37.981000+0000","last_unstale":"2023-12-20T04:28:39.386301+0000","last_undegraded":"2023-12-20T04:28:39.386301+0000","last_fullsized":"2023-12-20T04:28:39.386301+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386046+0000","last_change":"2023-12-20T04:28:38.405946+0000","last_active":"2023-12-20T04:28:39.386046+0000","last_peered":"2023-12-20T04:28:39.386046+0000","last_clean":"2023-12-20T04:28:39.386046+0000","last_became_active":"2023-12-20T04:28:38.405609+0000","last_became_peered":"2023-12-20T04:28:38.405609+0000","last_unstale":"2023-12-20T04:28:39.386046+0000","last_undegraded":"2023-12-20T04:28:39.386046+0000","last_fullsized":"2023-12-20T04:28:39.386046+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379821+0000","last_change":"2023-12-20T04:28:37.980797+0000","last_active":"2023-12-20T04:28:39.379821+0000","last_peered":"2023-12-20T04:28:39.379821+0000","last_clean":"2023-12-20T04:28:39.379821+0000","last_became_active":"2023-12-20T04:28:37.980430+0000","last_became_peered":"2023-12-20T04:28:37.980430+0000","last_unstale":"2023-12-20T04:28:39.379821+0000","last_undegraded":"2023-12-20T04:28:39.379821+0000","last_fullsized":"2023-12-20T04:28:39.379821+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:19.622544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386153+0000","last_change":"2023-12-20T04:28:38.389261+0000","last_active":"2023-12-20T04:28:39.386153+0000","last_peered":"2023-12-20T04:28:39.386153+0000","last_clean":"2023-12-20T04:28:39.386153+0000","last_became_active":"2023-12-20T04:28:38.388004+0000","last_became_peered":"2023-12-20T04:28:38.388004+0000","last_unstale":"2023-12-20T04:28:39.386153+0000","last_undegraded":"2023-12-20T04:28:39.386153+0000","last_fullsized":"2023-12-20T04:28:39.386153+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667746+0000","last_change":"2023-12-20T04:28:37.991195+0000","last_active":"2023-12-20T04:28:39.667746+0000","last_peered":"2023-12-20T04:28:39.667746+0000","last_clean":"2023-12-20T04:28:39.667746+0000","last_became_active":"2023-12-20T04:28:37.990860+0000","last_became_peered":"2023-12-20T04:28:37.990860+0000","last_unstale":"2023-12-20T04:28:39.667746+0000","last_undegraded":"2023-12-20T04:28:39.667746+0000","last_fullsized":"2023-12-20T04:28:39.667746+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:28:11.027603+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379827+0000","last_change":"2023-12-20T04:28:38.377842+0000","last_active":"2023-12-20T04:28:39.379827+0000","last_peered":"2023-12-20T04:28:39.379827+0000","last_clean":"2023-12-20T04:28:39.379827+0000","last_became_active":"2023-12-20T04:28:38.377525+0000","last_became_peered":"2023-12-20T04:28:38.377525+0000","last_unstale":"2023-12-20T04:28:39.379827+0000","last_undegraded":"2023-12-20T04:28:39.379827+0000","last_fullsized":"2023-12-20T04:28:39.379827+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:41:44.497086+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"21'1","reported_seq":12,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.674031+0000","last_change":"2023-12-20T04:28:37.995878+0000","last_active":"2023-12-20T04:28:39.674031+0000","last_peered":"2023-12-20T04:28:39.674031+0000","last_clean":"2023-12-20T04:28:39.674031+0000","last_became_active":"2023-12-20T04:28:37.995604+0000","last_became_peered":"2023-12-20T04:28:37.995604+0000","last_unstale":"2023-12-20T04:28:39.674031+0000","last_undegraded":"2023-12-20T04:28:39.674031+0000","last_fullsized":"2023-12-20T04:28:39.674031+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670458+0000","last_change":"2023-12-20T04:28:38.398433+0000","last_active":"2023-12-20T04:28:39.670458+0000","last_peered":"2023-12-20T04:28:39.670458+0000","last_clean":"2023-12-20T04:28:39.670458+0000","last_became_active":"2023-12-20T04:28:38.398247+0000","last_became_peered":"2023-12-20T04:28:38.398247+0000","last_unstale":"2023-12-20T04:28:39.670458+0000","last_undegraded":"2023-12-20T04:28:39.670458+0000","last_fullsized":"2023-12-20T04:28:39.670458+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:48:18.553482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.7","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667884+0000","last_change":"2023-12-20T04:28:37.993415+0000","last_active":"2023-12-20T04:28:39.667884+0000","last_peered":"2023-12-20T04:28:39.667884+0000","last_clean":"2023-12-20T04:28:39.667884+0000","last_became_active":"2023-12-20T04:28:37.993209+0000","last_became_peered":"2023-12-20T04:28:37.993209+0000","last_unstale":"2023-12-20T04:28:39.667884+0000","last_undegraded":"2023-12-20T04:28:39.667884+0000","last_fullsized":"2023-12-20T04:28:39.667884+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:22.916734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666307+0000","last_change":"2023-12-20T04:28:38.388800+0000","last_active":"2023-12-20T04:28:39.666307+0000","last_peered":"2023-12-20T04:28:39.666307+0000","last_clean":"2023-12-20T04:28:39.666307+0000","last_became_active":"2023-12-20T04:28:38.383908+0000","last_became_peered":"2023-12-20T04:28:38.383908+0000","last_unstale":"2023-12-20T04:28:39.666307+0000","last_undegraded":"2023-12-20T04:28:39.666307+0000","last_fullsized":"2023-12-20T04:28:39.666307+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.8","version":"0'0","reported_seq":11,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667371+0000","last_change":"2023-12-20T04:28:37.996076+0000","last_active":"2023-12-20T04:28:39.667371+0000","last_peered":"2023-12-20T04:28:39.667371+0000","last_clean":"2023-12-20T04:28:39.667371+0000","last_became_active":"2023-12-20T04:28:37.995770+0000","last_became_peered":"2023-12-20T04:28:37.995770+0000","last_unstale":"2023-12-20T04:28:39.667371+0000","last_undegraded":"2023-12-20T04:28:39.667371+0000","last_fullsized":"2023-12-20T04:28:39.667371+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:34.884350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385884+0000","last_change":"2023-12-20T04:28:38.393602+0000","last_active":"2023-12-20T04:28:39.385884+0000","last_peered":"2023-12-20T04:28:39.385884+0000","last_clean":"2023-12-20T04:28:39.385884+0000","last_became_active":"2023-12-20T04:28:38.392277+0000","last_became_peered":"2023-12-20T04:28:38.392277+0000","last_unstale":"2023-12-20T04:28:39.385884+0000","last_undegraded":"2023-12-20T04:28:39.385884+0000","last_fullsized":"2023-12-20T04:28:39.385884+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:02:20.635968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.9","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379704+0000","last_change":"2023-12-20T04:28:37.987522+0000","last_active":"2023-12-20T04:28:39.379704+0000","last_peered":"2023-12-20T04:28:39.379704+0000","last_clean":"2023-12-20T04:28:39.379704+0000","last_became_active":"2023-12-20T04:28:37.986906+0000","last_became_peered":"2023-12-20T04:28:37.986906+0000","last_unstale":"2023-12-20T04:28:39.379704+0000","last_undegraded":"2023-12-20T04:28:39.379704+0000","last_fullsized":"2023-12-20T04:28:39.379704+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:34.884350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670333+0000","last_change":"2023-12-20T04:28:38.382504+0000","last_active":"2023-12-20T04:28:39.670333+0000","last_peered":"2023-12-20T04:28:39.670333+0000","last_clean":"2023-12-20T04:28:39.670333+0000","last_became_active":"2023-12-20T04:28:38.382224+0000","last_became_peered":"2023-12-20T04:28:38.382224+0000","last_unstale":"2023-12-20T04:28:39.670333+0000","last_undegraded":"2023-12-20T04:28:39.670333+0000","last_fullsized":"2023-12-20T04:28:39.670333+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:19.625295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":11,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.379771+0000","last_change":"2023-12-20T04:28:37.985344+0000","last_active":"2023-12-20T04:28:39.379771+0000","last_peered":"2023-12-20T04:28:39.379771+0000","last_clean":"2023-12-20T04:28:39.379771+0000","last_became_active":"2023-12-20T04:28:37.985074+0000","last_became_peered":"2023-12-20T04:28:37.985074+0000","last_unstale":"2023-12-20T04:28:39.379771+0000","last_undegraded":"2023-12-20T04:28:39.379771+0000","last_fullsized":"2023-12-20T04:28:39.379771+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:20.962267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670395+0000","last_change":"2023-12-20T04:28:38.402571+0000","last_active":"2023-12-20T04:28:39.670395+0000","last_peered":"2023-12-20T04:28:39.670395+0000","last_clean":"2023-12-20T04:28:39.670395+0000","last_became_active":"2023-12-20T04:28:38.402316+0000","last_became_peered":"2023-12-20T04:28:38.402316+0000","last_unstale":"2023-12-20T04:28:39.670395+0000","last_undegraded":"2023-12-20T04:28:39.670395+0000","last_fullsized":"2023-12-20T04:28:39.670395+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.b","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667523+0000","last_change":"2023-12-20T04:28:37.991378+0000","last_active":"2023-12-20T04:28:39.667523+0000","last_peered":"2023-12-20T04:28:39.667523+0000","last_clean":"2023-12-20T04:28:39.667523+0000","last_became_active":"2023-12-20T04:28:37.990976+0000","last_became_peered":"2023-12-20T04:28:37.990976+0000","last_unstale":"2023-12-20T04:28:39.667523+0000","last_undegraded":"2023-12-20T04:28:39.667523+0000","last_fullsized":"2023-12-20T04:28:39.667523+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:44:18.611871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667716+0000","last_change":"2023-12-20T04:28:38.411659+0000","last_active":"2023-12-20T04:28:39.667716+0000","last_peered":"2023-12-20T04:28:39.667716+0000","last_clean":"2023-12-20T04:28:39.667716+0000","last_became_active":"2023-12-20T04:28:38.411333+0000","last_became_peered":"2023-12-20T04:28:38.411333+0000","last_unstale":"2023-12-20T04:28:39.667716+0000","last_undegraded":"2023-12-20T04:28:39.667716+0000","last_fullsized":"2023-12-20T04:28:39.667716+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:46:19.275821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.386042+0000","last_change":"2023-12-20T04:28:38.392988+0000","last_active":"2023-12-20T04:28:39.386042+0000","last_peered":"2023-12-20T04:28:39.386042+0000","last_clean":"2023-12-20T04:28:39.386042+0000","last_became_active":"2023-12-20T04:28:38.392136+0000","last_became_peered":"2023-12-20T04:28:38.392136+0000","last_unstale":"2023-12-20T04:28:39.386042+0000","last_undegraded":"2023-12-20T04:28:39.386042+0000","last_fullsized":"2023-12-20T04:28:39.386042+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:57.264884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1c","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667574+0000","last_change":"2023-12-20T04:28:37.991881+0000","last_active":"2023-12-20T04:28:39.667574+0000","last_peered":"2023-12-20T04:28:39.667574+0000","last_clean":"2023-12-20T04:28:39.667574+0000","last_became_active":"2023-12-20T04:28:37.991520+0000","last_became_peered":"2023-12-20T04:28:37.991520+0000","last_unstale":"2023-12-20T04:28:39.667574+0000","last_undegraded":"2023-12-20T04:28:39.667574+0000","last_fullsized":"2023-12-20T04:28:39.667574+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:18.530174+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":10,"reported_epoch":21,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.385497+0000","last_change":"2023-12-20T04:28:38.398174+0000","last_active":"2023-12-20T04:28:39.385497+0000","last_peered":"2023-12-20T04:28:39.385497+0000","last_clean":"2023-12-20T04:28:39.385497+0000","last_became_active":"2023-12-20T04:28:38.397909+0000","last_became_peered":"2023-12-20T04:28:38.397909+0000","last_unstale":"2023-12-20T04:28:39.385497+0000","last_undegraded":"2023-12-20T04:28:39.385497+0000","last_fullsized":"2023-12-20T04:28:39.385497+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1d","version":"20'2","reported_seq":13,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.667643+0000","last_change":"2023-12-20T04:28:37.997620+0000","last_active":"2023-12-20T04:28:39.667643+0000","last_peered":"2023-12-20T04:28:39.667643+0000","last_clean":"2023-12-20T04:28:39.667643+0000","last_became_active":"2023-12-20T04:28:37.997289+0000","last_became_peered":"2023-12-20T04:28:37.997289+0000","last_unstale":"2023-12-20T04:28:39.667643+0000","last_undegraded":"2023-12-20T04:28:39.667643+0000","last_fullsized":"2023-12-20T04:28:39.667643+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:00.985871+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666315+0000","last_change":"2023-12-20T04:28:38.403356+0000","last_active":"2023-12-20T04:28:39.666315+0000","last_peered":"2023-12-20T04:28:39.666315+0000","last_clean":"2023-12-20T04:28:39.666315+0000","last_became_active":"2023-12-20T04:28:38.402695+0000","last_became_peered":"2023-12-20T04:28:38.402695+0000","last_unstale":"2023-12-20T04:28:39.666315+0000","last_undegraded":"2023-12-20T04:28:39.666315+0000","last_fullsized":"2023-12-20T04:28:39.666315+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:30:00.525973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1e","version":"20'1","reported_seq":12,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670431+0000","last_change":"2023-12-20T04:28:37.986554+0000","last_active":"2023-12-20T04:28:39.670431+0000","last_peered":"2023-12-20T04:28:39.670431+0000","last_clean":"2023-12-20T04:28:39.670431+0000","last_became_active":"2023-12-20T04:28:37.985944+0000","last_became_peered":"2023-12-20T04:28:37.985944+0000","last_unstale":"2023-12-20T04:28:39.670431+0000","last_undegraded":"2023-12-20T04:28:39.670431+0000","last_fullsized":"2023-12-20T04:28:39.670431+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:09:18.484037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":10,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.670422+0000","last_change":"2023-12-20T04:28:38.402052+0000","last_active":"2023-12-20T04:28:39.670422+0000","last_peered":"2023-12-20T04:28:39.670422+0000","last_clean":"2023-12-20T04:28:39.670422+0000","last_became_active":"2023-12-20T04:28:38.401578+0000","last_became_peered":"2023-12-20T04:28:38.401578+0000","last_unstale":"2023-12-20T04:28:39.670422+0000","last_undegraded":"2023-12-20T04:28:39.670422+0000","last_fullsized":"2023-12-20T04:28:39.670422+0000","mapping_epoch":19,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":20,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:37.952579+0000","last_clean_scrub_stamp":"2023-12-20T04:28:37.952579+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.126873+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.1f","version":"20'4","reported_seq":15,"reported_epoch":20,"state":"active+clean","last_fresh":"2023-12-20T04:28:39.666365+0000","last_change":"2023-12-20T04:28:37.991899+0000","last_active":"2023-12-20T04:28:39.666365+0000","last_peered":"2023-12-20T04:28:39.666365+0000","last_clean":"2023-12-20T04:28:39.666365+0000","last_became_active":"2023-12-20T04:28:37.991545+0000","last_became_peered":"2023-12-20T04:28:37.991545+0000","last_unstale":"2023-12-20T04:28:39.666365+0000","last_undegraded":"2023-12-20T04:28:39.666365+0000","last_fullsized":"2023-12-20T04:28:39.666365+0000","mapping_epoch":18,"log_start":"0'0","ondisk_log_start":"0'0","created":18,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:36.939156+0000","last_clean_scrub_stamp":"2023-12-20T04:28:36.939156+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:15.687125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]}],"pool_stats":[{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574854,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31844,"kb_used_data":796,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339996,"statfs":{"total":96636764160,"available":96604155904,"internally_reserved":0,"allocated":815104,"data_stored":666519,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52000000000000002}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.58799999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66000000000000003}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80900000000000005}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.879}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.85399999999999998}]}]},{"osd":6,"up_from":13,"seq":55834574854,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.60299999999999998}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.628}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.54300000000000004}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.56799999999999995}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.752}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77000000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.66300000000000003}]}]},{"osd":5,"up_from":13,"seq":55834574855,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27168,"kb_used_data":216,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344672,"statfs":{"total":96636764160,"available":96608944128,"internally_reserved":0,"allocated":221184,"data_stored":75659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.50600000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.71299999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.153}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.758}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73799999999999999}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.78500000000000003}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.81599999999999995}]}]},{"osd":4,"up_from":13,"seq":55834574856,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27176,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344664,"statfs":{"total":96636764160,"available":96608935936,"internally_reserved":0,"allocated":229376,"data_stored":75695,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.36299999999999999}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46999999999999997}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51700000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55000000000000004}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.76000000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.65100000000000002}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.77200000000000002}]}]},{"osd":3,"up_from":13,"seq":55834574855,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31272,"kb_used_data":224,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340568,"statfs":{"total":96636764160,"available":96604741632,"internally_reserved":0,"allocated":229376,"data_stored":77133,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.3700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":2.7130000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.0110000000000001}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.96799999999999997}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":3.9830000000000001}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.012}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":4.6970000000000001}]}]},{"osd":2,"up_from":13,"seq":55834574854,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31260,"kb_used_data":212,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340580,"statfs":{"total":96636764160,"available":96604753920,"internally_reserved":0,"allocated":217088,"data_stored":75569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.44700000000000001}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.47099999999999997}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.61899999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.996}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.044}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.0289999999999999}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":1.1080000000000001}]}]},{"osd":1,"up_from":13,"seq":55834574856,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31268,"kb_used_data":220,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340572,"statfs":{"total":96636764160,"available":96604745728,"internally_reserved":0,"allocated":225280,"data_stored":75729,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.32200000000000001}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.46100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.48699999999999999}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.63700000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.67900000000000005}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.73299999999999998}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.80300000000000005}]}]},{"osd":0,"up_from":13,"seq":55834574855,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31852,"kb_used_data":804,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339988,"statfs":{"total":96636764160,"available":96604147712,"internally_reserved":0,"allocated":823296,"data_stored":666575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.39700000000000002}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.52100000000000002}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.41399999999999998}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.51300000000000001}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.53500000000000003}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.57399999999999995}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0,"5min":0,"15min":0},"min":{"1min":0,"5min":0,"15min":0},"max":{"1min":0,"5min":0,"15min":0},"last":0.55600000000000005}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":126,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:28:48.834 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-20T04:28:48.834 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2023-12-20T04:28:48.835 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2023-12-20T04:28:48.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2023-12-20T04:28:49.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.012+0000 7f0b7e58f640 1 Processor -- start 2023-12-20T04:28:49.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.012+0000 7f0b7e58f640 1 -- start start 2023-12-20T04:28:49.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b7e58f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0b780aed40 0x7f0b780ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b7e58f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b780b0d30 0x7f0b781593e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b7e58f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b780706b0 con 0x7f0b78071ae0 2023-12-20T04:28:49.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b7e58f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b780b1110 con 0x7f0b780b0d30 2023-12-20T04:28:49.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b7e58f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b78070900 con 0x7f0b780aed40 2023-12-20T04:28:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b7cb2c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b780b0d30 0x7f0b781593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b777fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0b78071ae0 0x7f0b780b0620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52690/0 (socket says 172.21.15.32:52690) 2023-12-20T04:28:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b777fe640 1 -- 172.21.15.32:0/3917388096 learned_addr learned my addr 172.21.15.32:0/3917388096 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b77fff640 1 --2- 172.21.15.32:0/3917388096 >> v2:172.21.15.39:3300/0 conn(0x7f0b780aed40 0x7f0b780ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b7cb2c640 1 -- 172.21.15.32:0/3917388096 >> v2:172.21.15.39:3300/0 conn(0x7f0b780aed40 msgr2=0x7f0b780ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b7cb2c640 1 --2- 172.21.15.32:0/3917388096 >> v2:172.21.15.39:3300/0 conn(0x7f0b780aed40 0x7f0b780ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b7cb2c640 1 -- 172.21.15.32:0/3917388096 >> v1:172.21.15.32:6789/0 conn(0x7f0b78071ae0 legacy=0x7f0b780b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b7cb2c640 1 -- 172.21.15.32:0/3917388096 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b78070be0 con 0x7f0b780b0d30 2023-12-20T04:28:49.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.016+0000 7f0b76ffd640 1 -- 172.21.15.32:0/3917388096 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602477131 0 0) 0x7f0b780706b0 con 0x7f0b78071ae0 2023-12-20T04:28:49.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7cb2c640 1 --2- 172.21.15.32:0/3917388096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b780b0d30 0x7f0b781593e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0b7806acb0 tx=0x7f0b7001c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5a094f5aca8bf33 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:49.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b76ffd640 1 -- 172.21.15.32:0/3917388096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700a5880 con 0x7f0b780b0d30 2023-12-20T04:28:49.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b76ffd640 1 -- 172.21.15.32:0/3917388096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b700a59e0 con 0x7f0b780b0d30 2023-12-20T04:28:49.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b76ffd640 1 -- 172.21.15.32:0/3917388096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b700a5d70 con 0x7f0b780b0d30 2023-12-20T04:28:49.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 -- 172.21.15.32:0/3917388096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b780b0d30 msgr2=0x7f0b781593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 --2- 172.21.15.32:0/3917388096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b780b0d30 0x7f0b781593e0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f0b7806acb0 tx=0x7f0b7001c7c0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 -- 172.21.15.32:0/3917388096 shutdown_connections 2023-12-20T04:28:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 --2- 172.21.15.32:0/3917388096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b780b0d30 0x7f0b781593e0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 --2- 172.21.15.32:0/3917388096 >> v2:172.21.15.39:3300/0 conn(0x7f0b780aed40 0x7f0b780ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 -- 172.21.15.32:0/3917388096 >> 172.21.15.32:0/3917388096 conn(0x7f0b7806a320 msgr2=0x7f0b780ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 -- 172.21.15.32:0/3917388096 shutdown_connections 2023-12-20T04:28:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 -- 172.21.15.32:0/3917388096 wait complete. 2023-12-20T04:28:49.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 Processor -- start 2023-12-20T04:28:49.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 -- start start 2023-12-20T04:28:49.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0b78071ae0 0x7f0b781666c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b780aed40 0x7f0b781449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b780b0d30 0x7f0b780c9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b781458e0 con 0x7f0b780aed40 2023-12-20T04:28:49.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b78144ef0 con 0x7f0b780b0d30 2023-12-20T04:28:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.020+0000 7f0b7e58f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b78166ca0 con 0x7f0b78071ae0 2023-12-20T04:28:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b780aed40 0x7f0b781449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b780aed40 0x7f0b781449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45266/0 (socket says 172.21.15.32:45266) 2023-12-20T04:28:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b777fe640 1 -- 172.21.15.32:0/2521686527 learned_addr learned my addr 172.21.15.32:0/2521686527 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b777fe640 1 -- 172.21.15.32:0/2521686527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0b78071ae0 msgr2=0x7f0b781666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b777fe640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0b78071ae0 0x7f0b781666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b777fe640 1 -- 172.21.15.32:0/2521686527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b780b0d30 msgr2=0x7f0b780c9100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b777fe640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b780b0d30 0x7f0b780c9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b777fe640 1 -- 172.21.15.32:0/2521686527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b70009a70 con 0x7f0b780aed40 2023-12-20T04:28:49.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b777fe640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b780aed40 0x7f0b781449b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c480 tx=0x7f0b6c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:49.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b74ff9640 1 -- 172.21.15.32:0/2521686527 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c0074e0 con 0x7f0b780aed40 2023-12-20T04:28:49.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b74ff9640 1 -- 172.21.15.32:0/2521686527 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b6c007660 con 0x7f0b780aed40 2023-12-20T04:28:49.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b74ff9640 1 -- 172.21.15.32:0/2521686527 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b6c004a30 con 0x7f0b780aed40 2023-12-20T04:28:49.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b7e58f640 1 -- 172.21.15.32:0/2521686527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b78145220 con 0x7f0b780aed40 2023-12-20T04:28:49.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b7e58f640 1 -- 172.21.15.32:0/2521686527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b78166f80 con 0x7f0b780aed40 2023-12-20T04:28:49.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b7e58f640 1 -- 172.21.15.32:0/2521686527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b7804e000 con 0x7f0b780aed40 2023-12-20T04:28:49.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b74ff9640 1 -- 172.21.15.32:0/2521686527 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0b6c004b90 con 0x7f0b780aed40 2023-12-20T04:28:49.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.024+0000 7f0b74ff9640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0b64065a60 0x7f0b64067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.028+0000 7f0b74ff9640 1 -- 172.21.15.32:0/2521686527 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0b6c0823f0 con 0x7f0b780aed40 2023-12-20T04:28:49.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.028+0000 7f0b77fff640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0b64065a60 0x7f0b64067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.028+0000 7f0b77fff640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0b64065a60 0x7f0b64067f20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0b68006aa0 tx=0x7f0b68006a30 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:49.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.028+0000 7f0b74ff9640 1 -- 172.21.15.32:0/2521686527 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0b6c053830 con 0x7f0b780aed40 2023-12-20T04:28:49.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b7e58f640 1 -- 172.21.15.32:0/2521686527 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) v1 -- 0x7f0b78145a20 con 0x7f0b780aed40 2023-12-20T04:28:49.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b74ff9640 1 -- 172.21.15.32:0/2521686527 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) v1 ==== 72+0+46 (secure 0 0 0) 0x7f0b78145a20 con 0x7f0b780aed40 2023-12-20T04:28:49.221 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:49.221 INFO:teuthology.orchestra.run.smithi032.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2023-12-20T04:28:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b7e58f640 1 -- 172.21.15.32:0/2521686527 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0b64065a60 msgr2=0x7f0b64067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b7e58f640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0b64065a60 0x7f0b64067f20 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f0b68006aa0 tx=0x7f0b68006a30 comp rx=0 tx=0).stop 2023-12-20T04:28:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b7e58f640 1 -- 172.21.15.32:0/2521686527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b780aed40 msgr2=0x7f0b781449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b7e58f640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b780aed40 0x7f0b781449b0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f0b6c00c480 tx=0x7f0b6c00c940 comp rx=0 tx=0).stop 2023-12-20T04:28:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b7e58f640 1 -- 172.21.15.32:0/2521686527 shutdown_connections 2023-12-20T04:28:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b7e58f640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0b64065a60 0x7f0b64067f20 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b7e58f640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b780b0d30 0x7f0b780c9100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b7e58f640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b780aed40 0x7f0b781449b0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b7e58f640 1 --2- 172.21.15.32:0/2521686527 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0b78071ae0 0x7f0b781666c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.220+0000 7f0b7e58f640 1 -- 172.21.15.32:0/2521686527 >> 172.21.15.32:0/2521686527 conn(0x7f0b7806a320 msgr2=0x7f0b780ca350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:49.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.224+0000 7f0b7e58f640 1 -- 172.21.15.32:0/2521686527 shutdown_connections 2023-12-20T04:28:49.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.224+0000 7f0b7e58f640 1 -- 172.21.15.32:0/2521686527 wait complete. 2023-12-20T04:28:49.242 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2023-12-20T04:28:49.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:49.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.384+0000 7f283e82d640 1 Processor -- start 2023-12-20T04:28:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.384+0000 7f283e82d640 1 -- start start 2023-12-20T04:28:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.384+0000 7f283e82d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28380aece0 0x7f28380b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.384+0000 7f283e82d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f28380b0b30 0x7f28380701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.384+0000 7f283e82d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28380c98c0 con 0x7f28381599e0 2023-12-20T04:28:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.384+0000 7f283e82d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28380c9ba0 con 0x7f28380b0b30 2023-12-20T04:28:49.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.384+0000 7f283e82d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28380c9e80 con 0x7f28380aece0 2023-12-20T04:28:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f28377fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28380aece0 0x7f28380b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f28377fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28380aece0 0x7f28380b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49698/0 (socket says 172.21.15.32:49698) 2023-12-20T04:28:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f28377fe640 1 -- 172.21.15.32:0/2956980486 learned_addr learned my addr 172.21.15.32:0/2956980486 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f283cdca640 1 --2- 172.21.15.32:0/2956980486 >> v2:172.21.15.39:3300/0 conn(0x7f28380b0b30 0x7f28380701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f28377fe640 1 -- 172.21.15.32:0/2956980486 >> v2:172.21.15.39:3300/0 conn(0x7f28380b0b30 msgr2=0x7f28380701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f28377fe640 1 --2- 172.21.15.32:0/2956980486 >> v2:172.21.15.39:3300/0 conn(0x7f28380b0b30 0x7f28380701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f28377fe640 1 -- 172.21.15.32:0/2956980486 >> v1:172.21.15.39:6789/0 conn(0x7f28381599e0 legacy=0x7f28380c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f28377fe640 1 -- 172.21.15.32:0/2956980486 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28380ca160 con 0x7f28380aece0 2023-12-20T04:28:49.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f28377fe640 1 --2- 172.21.15.32:0/2956980486 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28380aece0 0x7f28380b05f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f282c009870 tx=0x7f282c01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2125400c248b4cc4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f2836ffd640 1 -- 172.21.15.32:0/2956980486 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c0ab650 con 0x7f28380aece0 2023-12-20T04:28:49.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f2836ffd640 1 -- 172.21.15.32:0/2956980486 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c0ab7b0 con 0x7f28380aece0 2023-12-20T04:28:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.388+0000 7f2836ffd640 1 -- 172.21.15.32:0/2956980486 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c0abb40 con 0x7f28380aece0 2023-12-20T04:28:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 -- 172.21.15.32:0/2956980486 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28380aece0 msgr2=0x7f28380b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 --2- 172.21.15.32:0/2956980486 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28380aece0 0x7f28380b05f0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f282c009870 tx=0x7f282c01c540 comp rx=0 tx=0).stop 2023-12-20T04:28:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 -- 172.21.15.32:0/2956980486 shutdown_connections 2023-12-20T04:28:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 --2- 172.21.15.32:0/2956980486 >> v2:172.21.15.39:3300/0 conn(0x7f28380b0b30 0x7f28380701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 --2- 172.21.15.32:0/2956980486 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28380aece0 0x7f28380b05f0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 -- 172.21.15.32:0/2956980486 >> 172.21.15.32:0/2956980486 conn(0x7f283806a320 msgr2=0x7f28380ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 -- 172.21.15.32:0/2956980486 shutdown_connections 2023-12-20T04:28:49.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 -- 172.21.15.32:0/2956980486 wait complete. 2023-12-20T04:28:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 Processor -- start 2023-12-20T04:28:49.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 -- start start 2023-12-20T04:28:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28380aece0 0x7f28380c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28380b0b30 0x7f283814bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28381599e0 0x7f28380cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2838151970 con 0x7f28380b0b30 2023-12-20T04:28:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28380c5d70 con 0x7f28380aece0 2023-12-20T04:28:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f283e82d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28380c8190 con 0x7f28381599e0 2023-12-20T04:28:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f28377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28380b0b30 0x7f283814bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f28377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28380b0b30 0x7f283814bd50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45282/0 (socket says 172.21.15.32:45282) 2023-12-20T04:28:49.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.392+0000 7f28377fe640 1 -- 172.21.15.32:0/1323719585 learned_addr learned my addr 172.21.15.32:0/1323719585 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f283cdca640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28381599e0 0x7f28380cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f2837fff640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28380aece0 0x7f28380c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f28377fe640 1 -- 172.21.15.32:0/1323719585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28381599e0 msgr2=0x7f28380cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f28377fe640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28381599e0 0x7f28380cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f28377fe640 1 -- 172.21.15.32:0/1323719585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28380aece0 msgr2=0x7f28380c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f28377fe640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28380aece0 0x7f28380c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f28377fe640 1 -- 172.21.15.32:0/1323719585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f282c0094d0 con 0x7f28380b0b30 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f28377fe640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28380b0b30 0x7f283814bd50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f282c01abf0 tx=0x7f282c01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f2834ff9640 1 -- 172.21.15.32:0/1323719585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c0be5f0 con 0x7f28380b0b30 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f2834ff9640 1 -- 172.21.15.32:0/1323719585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f282c0be770 con 0x7f28380b0b30 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f2834ff9640 1 -- 172.21.15.32:0/1323719585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f282c0beb20 con 0x7f28380b0b30 2023-12-20T04:28:49.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f283e82d640 1 -- 172.21.15.32:0/1323719585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28380c8470 con 0x7f28380b0b30 2023-12-20T04:28:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f283e82d640 1 -- 172.21.15.32:0/1323719585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28380c7e40 con 0x7f28380b0b30 2023-12-20T04:28:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f283e82d640 1 -- 172.21.15.32:0/1323719585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2838151ab0 con 0x7f28380b0b30 2023-12-20T04:28:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f2834ff9640 1 -- 172.21.15.32:0/1323719585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f282c0b43f0 con 0x7f28380b0b30 2023-12-20T04:28:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f2834ff9640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2824065ab0 0x7f2824067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.396+0000 7f2834ff9640 1 -- 172.21.15.32:0/1323719585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f282c11e200 con 0x7f28380b0b30 2023-12-20T04:28:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.400+0000 7f2837fff640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2824065ab0 0x7f2824067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.400+0000 7f2837fff640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2824065ab0 0x7f2824067f70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2828001600 tx=0x7f2828009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:49.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.400+0000 7f2834ff9640 1 -- 172.21.15.32:0/1323719585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2838151ab0 con 0x7f28380b0b30 2023-12-20T04:28:49.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f283e82d640 1 -- 172.21.15.32:0/1323719585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2838151ab0 con 0x7f28380b0b30 2023-12-20T04:28:49.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f2834ff9640 1 -- 172.21.15.32:0/1323719585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f2838151ab0 con 0x7f28380b0b30 2023-12-20T04:28:49.565 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:49.565 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f283e82d640 1 -- 172.21.15.32:0/1323719585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2824065ab0 msgr2=0x7f2824067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f283e82d640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2824065ab0 0x7f2824067f70 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f2828001600 tx=0x7f2828009040 comp rx=0 tx=0).stop 2023-12-20T04:28:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f283e82d640 1 -- 172.21.15.32:0/1323719585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28380b0b30 msgr2=0x7f283814bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f283e82d640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28380b0b30 0x7f283814bd50 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f282c01abf0 tx=0x7f282c01ad80 comp rx=0 tx=0).stop 2023-12-20T04:28:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f283e82d640 1 -- 172.21.15.32:0/1323719585 shutdown_connections 2023-12-20T04:28:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f283e82d640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2824065ab0 0x7f2824067f70 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f283e82d640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28381599e0 0x7f28380cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f283e82d640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28380b0b30 0x7f283814bd50 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f283e82d640 1 --2- 172.21.15.32:0/1323719585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28380aece0 0x7f28380c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.564+0000 7f283e82d640 1 -- 172.21.15.32:0/1323719585 >> 172.21.15.32:0/1323719585 conn(0x7f283806a320 msgr2=0x7f2838153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:49.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.568+0000 7f283e82d640 1 -- 172.21.15.32:0/1323719585 shutdown_connections 2023-12-20T04:28:49.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.568+0000 7f283e82d640 1 -- 172.21.15.32:0/1323719585 wait complete. 2023-12-20T04:28:49.587 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:28:36.040195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:28:37.364671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:28:49.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:28:49.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.732+0000 7fb9fa0c4640 1 Processor -- start 2023-12-20T04:28:49.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.732+0000 7fb9fa0c4640 1 -- start start 2023-12-20T04:28:49.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9fa0c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9f41591e0 0x7fb9f40caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9fa0c4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb9f4070940 0x7fb9f40cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9fa0c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9f40cc3b0 con 0x7fb9f406cdf0 2023-12-20T04:28:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9fa0c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f40c8e30 con 0x7fb9f41591e0 2023-12-20T04:28:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9fa0c4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9f406d280 con 0x7fb9f4070940 2023-12-20T04:28:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9f37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9f41591e0 0x7fb9f40caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9f2ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb9f406cdf0 0x7fb9f4070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52706/0 (socket says 172.21.15.32:52706) 2023-12-20T04:28:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9f2ffd640 1 -- 172.21.15.32:0/1890195678 learned_addr learned my addr 172.21.15.32:0/1890195678 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9f3fff640 1 --2- 172.21.15.32:0/1890195678 >> v2:172.21.15.39:3300/0 conn(0x7fb9f4070940 0x7fb9f40cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9f37fe640 1 -- 172.21.15.32:0/1890195678 >> v2:172.21.15.39:3300/0 conn(0x7fb9f4070940 msgr2=0x7fb9f40cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:49.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9f37fe640 1 --2- 172.21.15.32:0/1890195678 >> v2:172.21.15.39:3300/0 conn(0x7fb9f4070940 0x7fb9f40cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9f37fe640 1 -- 172.21.15.32:0/1890195678 >> v1:172.21.15.32:6789/0 conn(0x7fb9f406cdf0 legacy=0x7fb9f4070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9f37fe640 1 -- 172.21.15.32:0/1890195678 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f40c90d0 con 0x7fb9f41591e0 2023-12-20T04:28:49.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.736+0000 7fb9f27fc640 1 -- 172.21.15.32:0/1890195678 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 469974097 0 0) 0x7fb9f40cc3b0 con 0x7fb9f406cdf0 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9f37fe640 1 --2- 172.21.15.32:0/1890195678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f41591e0 0x7fb9f40caa30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40c5860 tx=0x7fb9e401c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc7ec4f75b9bae0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9f27fc640 1 -- 172.21.15.32:0/1890195678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40ab600 con 0x7fb9f41591e0 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9f27fc640 1 -- 172.21.15.32:0/1890195678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e40a7050 con 0x7fb9f41591e0 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9f27fc640 1 -- 172.21.15.32:0/1890195678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40ab980 con 0x7fb9f41591e0 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/1890195678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f41591e0 msgr2=0x7fb9f40caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9fa0c4640 1 --2- 172.21.15.32:0/1890195678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f41591e0 0x7fb9f40caa30 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fb9f40c5860 tx=0x7fb9e401c3d0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/1890195678 shutdown_connections 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9fa0c4640 1 --2- 172.21.15.32:0/1890195678 >> v2:172.21.15.39:3300/0 conn(0x7fb9f4070940 0x7fb9f40cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9fa0c4640 1 --2- 172.21.15.32:0/1890195678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f41591e0 0x7fb9f40caa30 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/1890195678 >> 172.21.15.32:0/1890195678 conn(0x7fb9f406a510 msgr2=0x7fb9f40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/1890195678 shutdown_connections 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/1890195678 wait complete. 2023-12-20T04:28:49.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9fa0c4640 1 Processor -- start 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.740+0000 7fb9fa0c4640 1 -- start start 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9fa0c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4070940 0x7fb9f4154680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9fa0c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9f41591e0 0x7fb9f413f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9fa0c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9f413fd50 0x7fb9f41f51e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9fa0c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f413cab0 con 0x7fb9f4070940 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9fa0c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f413e770 con 0x7fb9f413fd50 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9fa0c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9f413ea50 con 0x7fb9f41591e0 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4070940 0x7fb9f4154680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4070940 0x7fb9f4154680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45306/0 (socket says 172.21.15.32:45306) 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f37fe640 1 -- 172.21.15.32:0/2094206171 learned_addr learned my addr 172.21.15.32:0/2094206171 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f2ffd640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9f41591e0 0x7fb9f413f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f37fe640 1 -- 172.21.15.32:0/2094206171 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9f41591e0 msgr2=0x7fb9f413f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f3fff640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9f413fd50 0x7fb9f41f51e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f37fe640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9f41591e0 0x7fb9f413f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f37fe640 1 -- 172.21.15.32:0/2094206171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9f413fd50 msgr2=0x7fb9f41f51e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f37fe640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9f413fd50 0x7fb9f41f51e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f37fe640 1 -- 172.21.15.32:0/2094206171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e4009420 con 0x7fb9f4070940 2023-12-20T04:28:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f2ffd640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9f41591e0 0x7fb9f413f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:28:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9f37fe640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4070940 0x7fb9f4154680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40ae040 tx=0x7fb9e4019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:49.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9dbfff640 1 -- 172.21.15.32:0/2094206171 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40b4610 con 0x7fb9f4070940 2023-12-20T04:28:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9dbfff640 1 -- 172.21.15.32:0/2094206171 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e40b4770 con 0x7fb9f4070940 2023-12-20T04:28:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9dbfff640 1 -- 172.21.15.32:0/2094206171 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e40b4b20 con 0x7fb9f4070940 2023-12-20T04:28:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/2094206171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9f413ed30 con 0x7fb9f4070940 2023-12-20T04:28:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/2094206171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9f413f2f0 con 0x7fb9f4070940 2023-12-20T04:28:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.744+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/2094206171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9f40c54a0 con 0x7fb9f4070940 2023-12-20T04:28:49.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.748+0000 7fb9dbfff640 1 -- 172.21.15.32:0/2094206171 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb9e40b2040 con 0x7fb9f4070940 2023-12-20T04:28:49.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.748+0000 7fb9dbfff640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9e0065a60 0x7fb9e0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:49.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.748+0000 7fb9dbfff640 1 -- 172.21.15.32:0/2094206171 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb9e411d150 con 0x7fb9f4070940 2023-12-20T04:28:49.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.748+0000 7fb9f2ffd640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9e0065a60 0x7fb9e0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:49.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.748+0000 7fb9f2ffd640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9e0065a60 0x7fb9e0067f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4166930 tx=0x7fb9e8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:49.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.752+0000 7fb9dbfff640 1 -- 172.21.15.32:0/2094206171 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb9e40eab10 con 0x7fb9f4070940 2023-12-20T04:28:49.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.908+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/2094206171 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb9f404e000 con 0x7fb9f4070940 2023-12-20T04:28:49.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.908+0000 7fb9dbfff640 1 -- 172.21.15.32:0/2094206171 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb9e40ee130 con 0x7fb9f4070940 2023-12-20T04:28:49.911 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:28:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/2094206171 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9e0065a60 msgr2=0x7fb9e0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9e0065a60 0x7fb9e0067f20 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fb9f4166930 tx=0x7fb9e8009040 comp rx=0 tx=0).stop 2023-12-20T04:28:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/2094206171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4070940 msgr2=0x7fb9f4154680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4070940 0x7fb9f4154680 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7fb9e40ae040 tx=0x7fb9e4019170 comp rx=0 tx=0).stop 2023-12-20T04:28:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/2094206171 shutdown_connections 2023-12-20T04:28:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9e0065a60 0x7fb9e0067f20 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9f413fd50 0x7fb9f41f51e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9f41591e0 0x7fb9f413f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 --2- 172.21.15.32:0/2094206171 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9f4070940 0x7fb9f4154680 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/2094206171 >> 172.21.15.32:0/2094206171 conn(0x7fb9f406a510 msgr2=0x7fb9f40c78c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/2094206171 shutdown_connections 2023-12-20T04:28:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:49.912+0000 7fb9fa0c4640 1 -- 172.21.15.32:0/2094206171 wait complete. 2023-12-20T04:28:49.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:28:50.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.076+0000 7f268efcc640 1 Processor -- start 2023-12-20T04:28:50.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.076+0000 7f268efcc640 1 -- start start 2023-12-20T04:28:50.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.076+0000 7f268efcc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 0x7f26880caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.076+0000 7f268efcc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268815b6a0 con 0x7f268815abb0 2023-12-20T04:28:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.076+0000 7f268efcc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f268806d8c0 con 0x7f26880b0cb0 2023-12-20T04:28:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.076+0000 7f268efcc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f268806e160 con 0x7f2688070940 2023-12-20T04:28:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.076+0000 7f268cd68640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f268815abb0 0x7f268815bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52712/0 (socket says 172.21.15.32:52712) 2023-12-20T04:28:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.076+0000 7f268cd68640 1 -- 172.21.15.32:0/501330735 learned_addr learned my addr 172.21.15.32:0/501330735 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.076+0000 7f268d569640 1 --2- 172.21.15.32:0/501330735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 0x7f26880caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.080+0000 7f26877fe640 1 -- 172.21.15.32:0/501330735 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3702375946 0 0) 0x7f268815b6a0 con 0x7f268815abb0 2023-12-20T04:28:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.080+0000 7f26877fe640 1 -- 172.21.15.32:0/501330735 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2678003530 con 0x7f268815abb0 2023-12-20T04:28:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.080+0000 7f26877fe640 1 -- 172.21.15.32:0/501330735 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1053613434 0 0) 0x7f268806d8c0 con 0x7f26880b0cb0 2023-12-20T04:28:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.080+0000 7f26877fe640 1 -- 172.21.15.32:0/501330735 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f268815b6a0 con 0x7f26880b0cb0 2023-12-20T04:28:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.080+0000 7f268d569640 1 -- 172.21.15.32:0/501330735 >> v1:172.21.15.39:6789/0 conn(0x7f26880b0cb0 legacy=0x7f2688070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.080+0000 7f268d569640 1 -- 172.21.15.32:0/501330735 >> v1:172.21.15.32:6789/0 conn(0x7f268815abb0 legacy=0x7f268815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.080+0000 7f268d569640 1 -- 172.21.15.32:0/501330735 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f268806dba0 con 0x7f2688070940 2023-12-20T04:28:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.080+0000 7f268d569640 1 --2- 172.21.15.32:0/501330735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 0x7f26880caaf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2680009dc0 tx=0x7f268001c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3d5c52547ad1cc2a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.080+0000 7f26877fe640 1 -- 172.21.15.32:0/501330735 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26800a5a30 con 0x7f2688070940 2023-12-20T04:28:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.080+0000 7f26877fe640 1 -- 172.21.15.32:0/501330735 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26800a5b90 con 0x7f2688070940 2023-12-20T04:28:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.080+0000 7f26877fe640 1 -- 172.21.15.32:0/501330735 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26800ad440 con 0x7f2688070940 2023-12-20T04:28:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 -- 172.21.15.32:0/501330735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 msgr2=0x7f26880caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 --2- 172.21.15.32:0/501330735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 0x7f26880caaf0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f2680009dc0 tx=0x7f268001c970 comp rx=0 tx=0).stop 2023-12-20T04:28:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 -- 172.21.15.32:0/501330735 shutdown_connections 2023-12-20T04:28:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 --2- 172.21.15.32:0/501330735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 0x7f26880caaf0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 -- 172.21.15.32:0/501330735 >> 172.21.15.32:0/501330735 conn(0x7f268806a510 msgr2=0x7f26880ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:50.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 -- 172.21.15.32:0/501330735 shutdown_connections 2023-12-20T04:28:50.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 -- 172.21.15.32:0/501330735 wait complete. 2023-12-20T04:28:50.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 Processor -- start 2023-12-20T04:28:50.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 -- start start 2023-12-20T04:28:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 0x7f26880c3c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26880b0cb0 0x7f26880cc7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f268815abb0 0x7f26880ccd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26880cd440 con 0x7f26880b0cb0 2023-12-20T04:28:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26880cd720 con 0x7f268815abb0 2023-12-20T04:28:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268efcc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26880cda00 con 0x7f2688070940 2023-12-20T04:28:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f2687fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26880b0cb0 0x7f26880cc7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f2687fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26880b0cb0 0x7f26880cc7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45308/0 (socket says 172.21.15.32:45308) 2023-12-20T04:28:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f2687fff640 1 -- 172.21.15.32:0/1697247594 learned_addr learned my addr 172.21.15.32:0/1697247594 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268d569640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f268815abb0 0x7f26880ccd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268cd68640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 0x7f26880c3c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f2687fff640 1 -- 172.21.15.32:0/1697247594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 msgr2=0x7f26880c3c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f2687fff640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 0x7f26880c3c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f2687fff640 1 -- 172.21.15.32:0/1697247594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f268815abb0 msgr2=0x7f26880ccd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f2687fff640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f268815abb0 0x7f26880ccd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f2687fff640 1 -- 172.21.15.32:0/1697247594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2680009a70 con 0x7f26880b0cb0 2023-12-20T04:28:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.084+0000 7f268cd68640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 0x7f26880c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.088+0000 7f2687fff640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26880b0cb0 0x7f26880cc7f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f267400c500 tx=0x7f267400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.088+0000 7f26857fa640 1 -- 172.21.15.32:0/1697247594 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26740052d0 con 0x7f26880b0cb0 2023-12-20T04:28:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.088+0000 7f26857fa640 1 -- 172.21.15.32:0/1697247594 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2674005430 con 0x7f26880b0cb0 2023-12-20T04:28:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.088+0000 7f268efcc640 1 -- 172.21.15.32:0/1697247594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26880cdce0 con 0x7f26880b0cb0 2023-12-20T04:28:50.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.088+0000 7f26857fa640 1 -- 172.21.15.32:0/1697247594 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2674012870 con 0x7f26880b0cb0 2023-12-20T04:28:50.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.088+0000 7f268efcc640 1 -- 172.21.15.32:0/1697247594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26880c41d0 con 0x7f26880b0cb0 2023-12-20T04:28:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.088+0000 7f268efcc640 1 -- 172.21.15.32:0/1697247594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26880c43c0 con 0x7f26880b0cb0 2023-12-20T04:28:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.088+0000 7f26857fa640 1 -- 172.21.15.32:0/1697247594 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f26740129d0 con 0x7f26880b0cb0 2023-12-20T04:28:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.088+0000 7f26857fa640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f266c065a60 0x7f266c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.092+0000 7f26857fa640 1 -- 172.21.15.32:0/1697247594 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f26740836c0 con 0x7f26880b0cb0 2023-12-20T04:28:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.092+0000 7f268cd68640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f266c065a60 0x7f266c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.092+0000 7f268cd68640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f266c065a60 0x7f266c067f20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f267c0011f0 tx=0x7f267c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:50.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.096+0000 7f26857fa640 1 -- 172.21.15.32:0/1697247594 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2674051070 con 0x7f26880b0cb0 2023-12-20T04:28:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.252+0000 7f268efcc640 1 -- 172.21.15.32:0/1697247594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f26880c43c0 con 0x7f26880b0cb0 2023-12-20T04:28:50.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f26857fa640 1 -- 172.21.15.32:0/1697247594 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f26880c43c0 con 0x7f26880b0cb0 2023-12-20T04:28:50.255 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:28:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 -- 172.21.15.32:0/1697247594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f266c065a60 msgr2=0x7f266c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f266c065a60 0x7f266c067f20 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f267c0011f0 tx=0x7f267c009040 comp rx=0 tx=0).stop 2023-12-20T04:28:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 -- 172.21.15.32:0/1697247594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26880b0cb0 msgr2=0x7f26880cc7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26880b0cb0 0x7f26880cc7f0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f267400c500 tx=0x7f267400c9c0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 -- 172.21.15.32:0/1697247594 shutdown_connections 2023-12-20T04:28:50.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f266c065a60 0x7f266c067f20 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f268815abb0 0x7f26880ccd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26880b0cb0 0x7f26880cc7f0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 --2- 172.21.15.32:0/1697247594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2688070940 0x7f26880c3c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 -- 172.21.15.32:0/1697247594 >> 172.21.15.32:0/1697247594 conn(0x7f268806a510 msgr2=0x7f26880c4810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:50.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 -- 172.21.15.32:0/1697247594 shutdown_connections 2023-12-20T04:28:50.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.256+0000 7f268efcc640 1 -- 172.21.15.32:0/1697247594 wait complete. 2023-12-20T04:28:50.277 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:28:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.420+0000 7ff623c11640 1 Processor -- start 2023-12-20T04:28:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff623c11640 1 -- start start 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff623c11640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff61c0b0cb0 0x7ff61c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff623c11640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff61c070230 0x7ff61c070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff623c11640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff61c070bc0 con 0x7ff61c15abb0 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff623c11640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c06e160 con 0x7ff61c070230 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff623c11640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff61c15b6a0 con 0x7ff61c0b0cb0 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff6221ae640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff61c070230 0x7ff61c070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff6221ae640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff61c070230 0x7ff61c070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45314/0 (socket says 172.21.15.32:45314) 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff6221ae640 1 -- 172.21.15.32:0/1168194246 learned_addr learned my addr 172.21.15.32:0/1168194246 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff6211ac640 1 --2- 172.21.15.32:0/1168194246 >> v2:172.21.15.39:3300/0 conn(0x7ff61c0b0cb0 0x7ff61c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff6221ae640 1 -- 172.21.15.32:0/1168194246 >> v2:172.21.15.39:3300/0 conn(0x7ff61c0b0cb0 msgr2=0x7ff61c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff6221ae640 1 --2- 172.21.15.32:0/1168194246 >> v2:172.21.15.39:3300/0 conn(0x7ff61c0b0cb0 0x7ff61c0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff6221ae640 1 -- 172.21.15.32:0/1168194246 >> v1:172.21.15.39:6789/0 conn(0x7ff61c15abb0 legacy=0x7ff61c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:50.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.424+0000 7ff6221ae640 1 -- 172.21.15.32:0/1168194246 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff61c06d8c0 con 0x7ff61c070230 2023-12-20T04:28:50.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff6221ae640 1 --2- 172.21.15.32:0/1168194246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c070230 0x7ff61c070680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0b4470 tx=0x7ff61401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1281e2865076a304 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:50.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff6209ab640 1 -- 172.21.15.32:0/1168194246 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6140a5850 con 0x7ff61c070230 2023-12-20T04:28:50.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff6209ab640 1 -- 172.21.15.32:0/1168194246 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6140a59b0 con 0x7ff61c070230 2023-12-20T04:28:50.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff6209ab640 1 -- 172.21.15.32:0/1168194246 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6140a5d40 con 0x7ff61c070230 2023-12-20T04:28:50.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff623c11640 1 -- 172.21.15.32:0/1168194246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c070230 msgr2=0x7ff61c070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff623c11640 1 --2- 172.21.15.32:0/1168194246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c070230 0x7ff61c070680 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7ff61c0b4470 tx=0x7ff61401c990 comp rx=0 tx=0).stop 2023-12-20T04:28:50.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff623c11640 1 -- 172.21.15.32:0/1168194246 shutdown_connections 2023-12-20T04:28:50.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff623c11640 1 --2- 172.21.15.32:0/1168194246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c070230 0x7ff61c070680 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff623c11640 1 --2- 172.21.15.32:0/1168194246 >> v2:172.21.15.39:3300/0 conn(0x7ff61c0b0cb0 0x7ff61c0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff623c11640 1 -- 172.21.15.32:0/1168194246 >> 172.21.15.32:0/1168194246 conn(0x7ff61c06a510 msgr2=0x7ff61c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:50.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff623c11640 1 -- 172.21.15.32:0/1168194246 shutdown_connections 2023-12-20T04:28:50.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff623c11640 1 -- 172.21.15.32:0/1168194246 wait complete. 2023-12-20T04:28:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff623c11640 1 Processor -- start 2023-12-20T04:28:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.428+0000 7ff623c11640 1 -- start start 2023-12-20T04:28:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff623c11640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff61c070230 0x7ff61c135af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff623c11640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff61c0b0cb0 0x7ff61c14cd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff623c11640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c15abb0 0x7ff61c155080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff623c11640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c149cd0 con 0x7ff61c15abb0 2023-12-20T04:28:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff623c11640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c14c050 con 0x7ff61c0b0cb0 2023-12-20T04:28:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff623c11640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff61c136030 con 0x7ff61c070230 2023-12-20T04:28:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6221ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c15abb0 0x7ff61c155080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6221ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c15abb0 0x7ff61c155080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45316/0 (socket says 172.21.15.32:45316) 2023-12-20T04:28:50.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6221ae640 1 -- 172.21.15.32:0/1888945051 learned_addr learned my addr 172.21.15.32:0/1888945051 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:50.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6211ac640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff61c0b0cb0 0x7ff61c14cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6219ad640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff61c070230 0x7ff61c135af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6221ae640 1 -- 172.21.15.32:0/1888945051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff61c070230 msgr2=0x7ff61c135af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6221ae640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff61c070230 0x7ff61c135af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6221ae640 1 -- 172.21.15.32:0/1888945051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff61c0b0cb0 msgr2=0x7ff61c14cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6221ae640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff61c0b0cb0 0x7ff61c14cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6221ae640 1 -- 172.21.15.32:0/1888945051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff614009b30 con 0x7ff61c15abb0 2023-12-20T04:28:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6221ae640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c15abb0 0x7ff61c155080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff61401a5c0 tx=0x7ff614019730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6127fc640 1 -- 172.21.15.32:0/1888945051 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6140b8670 con 0x7ff61c15abb0 2023-12-20T04:28:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6127fc640 1 -- 172.21.15.32:0/1888945051 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6140b8810 con 0x7ff61c15abb0 2023-12-20T04:28:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff623c11640 1 -- 172.21.15.32:0/1888945051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff61c14c8b0 con 0x7ff61c15abb0 2023-12-20T04:28:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6127fc640 1 -- 172.21.15.32:0/1888945051 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6140b8bc0 con 0x7ff61c15abb0 2023-12-20T04:28:50.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff623c11640 1 -- 172.21.15.32:0/1888945051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff61c14c260 con 0x7ff61c15abb0 2023-12-20T04:28:50.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff623c11640 1 -- 172.21.15.32:0/1888945051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff61c04e000 con 0x7ff61c15abb0 2023-12-20T04:28:50.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.432+0000 7ff6127fc640 1 -- 172.21.15.32:0/1888945051 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff6140b8d20 con 0x7ff61c15abb0 2023-12-20T04:28:50.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.436+0000 7ff6127fc640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff600065a60 0x7ff600067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.436+0000 7ff6127fc640 1 -- 172.21.15.32:0/1888945051 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff61411e420 con 0x7ff61c15abb0 2023-12-20T04:28:50.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.436+0000 7ff6219ad640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff600065a60 0x7ff600067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.436+0000 7ff6219ad640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff600065a60 0x7ff600067f20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff618007fd0 tx=0x7ff618007e40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:50.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.440+0000 7ff6127fc640 1 -- 172.21.15.32:0/1888945051 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff6140ebe80 con 0x7ff61c15abb0 2023-12-20T04:28:50.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 -- 172.21.15.32:0/1888945051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff61c070bc0 con 0x7ff61c15abb0 2023-12-20T04:28:50.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff6127fc640 1 -- 172.21.15.32:0/1888945051 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff61c070bc0 con 0x7ff61c15abb0 2023-12-20T04:28:50.597 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:28:50.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 -- 172.21.15.32:0/1888945051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff600065a60 msgr2=0x7ff600067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff600065a60 0x7ff600067f20 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7ff618007fd0 tx=0x7ff618007e40 comp rx=0 tx=0).stop 2023-12-20T04:28:50.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 -- 172.21.15.32:0/1888945051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c15abb0 msgr2=0x7ff61c155080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c15abb0 0x7ff61c155080 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7ff61401a5c0 tx=0x7ff614019730 comp rx=0 tx=0).stop 2023-12-20T04:28:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 -- 172.21.15.32:0/1888945051 shutdown_connections 2023-12-20T04:28:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff600065a60 0x7ff600067f20 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff61c15abb0 0x7ff61c155080 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff61c0b0cb0 0x7ff61c14cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 --2- 172.21.15.32:0/1888945051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff61c070230 0x7ff61c135af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 -- 172.21.15.32:0/1888945051 >> 172.21.15.32:0/1888945051 conn(0x7ff61c06a510 msgr2=0x7ff61c156480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:50.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 -- 172.21.15.32:0/1888945051 shutdown_connections 2023-12-20T04:28:50.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.596+0000 7ff623c11640 1 -- 172.21.15.32:0/1888945051 wait complete. 2023-12-20T04:28:50.619 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:28:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65eba66640 1 Processor -- start 2023-12-20T04:28:50.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65eba66640 1 -- start start 2023-12-20T04:28:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65eba66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65e40cfc80 0x7f65e4167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65eba66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e40b6ca0 con 0x7f65e415cdb0 2023-12-20T04:28:50.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65eba66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65e415d190 con 0x7f65e40d2740 2023-12-20T04:28:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65eba66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e40d16a0 con 0x7f65e40cfc80 2023-12-20T04:28:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65e9001640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f65e40d2740 0x7f65e40cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:37604/0 (socket says 172.21.15.32:37604) 2023-12-20T04:28:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65e9001640 1 -- 172.21.15.32:0/946528043 learned_addr learned my addr 172.21.15.32:0/946528043 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65ea003640 1 --2- 172.21.15.32:0/946528043 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65e40cfc80 0x7f65e4167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65e8800640 1 -- 172.21.15.32:0/946528043 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 439665244 0 0) 0x7f65e0001120 con 0x7f65e415cdb0 2023-12-20T04:28:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65e8800640 1 -- 172.21.15.32:0/946528043 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65d0003530 con 0x7f65e415cdb0 2023-12-20T04:28:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65ea003640 1 -- 172.21.15.32:0/946528043 >> v1:172.21.15.39:6789/0 conn(0x7f65e40d2740 legacy=0x7f65e40cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65ea003640 1 -- 172.21.15.32:0/946528043 >> v1:172.21.15.32:6789/0 conn(0x7f65e415cdb0 legacy=0x7f65e40d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.764+0000 7f65ea003640 1 -- 172.21.15.32:0/946528043 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65e40d1980 con 0x7f65e40cfc80 2023-12-20T04:28:50.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.768+0000 7f65ea003640 1 --2- 172.21.15.32:0/946528043 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65e40cfc80 0x7f65e4167910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009dc0 tx=0x7f65dc01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d7862c847594195d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:50.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.772+0000 7f65e8800640 1 -- 172.21.15.32:0/946528043 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0af600 con 0x7f65e40cfc80 2023-12-20T04:28:50.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.772+0000 7f65e8800640 1 -- 172.21.15.32:0/946528043 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc0ab050 con 0x7f65e40cfc80 2023-12-20T04:28:50.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.772+0000 7f65e8800640 1 -- 172.21.15.32:0/946528043 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0af980 con 0x7f65e40cfc80 2023-12-20T04:28:50.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.772+0000 7f65eba66640 1 -- 172.21.15.32:0/946528043 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65e40cfc80 msgr2=0x7f65e4167910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.772+0000 7f65eba66640 1 --2- 172.21.15.32:0/946528043 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65e40cfc80 0x7f65e4167910 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f65dc009dc0 tx=0x7f65dc01f150 comp rx=0 tx=0).stop 2023-12-20T04:28:50.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.772+0000 7f65eba66640 1 -- 172.21.15.32:0/946528043 shutdown_connections 2023-12-20T04:28:50.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.772+0000 7f65eba66640 1 --2- 172.21.15.32:0/946528043 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65e40cfc80 0x7f65e4167910 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.772+0000 7f65eba66640 1 -- 172.21.15.32:0/946528043 >> 172.21.15.32:0/946528043 conn(0x7f65e4065ee0 msgr2=0x7f65e406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:50.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.772+0000 7f65eba66640 1 -- 172.21.15.32:0/946528043 shutdown_connections 2023-12-20T04:28:50.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.772+0000 7f65eba66640 1 -- 172.21.15.32:0/946528043 wait complete. 2023-12-20T04:28:50.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65eba66640 1 Processor -- start 2023-12-20T04:28:50.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65eba66640 1 -- start start 2023-12-20T04:28:50.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65eba66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e40cfc80 0x7f65e414c910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65eba66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65e40d2740 0x7f65e41f6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65eba66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65e415cdb0 0x7f65e4153500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65eba66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e4160130 con 0x7f65e415cdb0 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65eba66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65e414ce50 con 0x7f65e40cfc80 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65eba66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65e415f520 con 0x7f65e40d2740 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65ea003640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65e415cdb0 0x7f65e4153500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65ea003640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65e415cdb0 0x7f65e4153500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45322/0 (socket says 172.21.15.32:45322) 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65ea003640 1 -- 172.21.15.32:0/1258405493 learned_addr learned my addr 172.21.15.32:0/1258405493 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65e9802640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e40cfc80 0x7f65e414c910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65e9001640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65e40d2740 0x7f65e41f6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65ea003640 1 -- 172.21.15.32:0/1258405493 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65e40d2740 msgr2=0x7f65e41f6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65ea003640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65e40d2740 0x7f65e41f6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65ea003640 1 -- 172.21.15.32:0/1258405493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e40cfc80 msgr2=0x7f65e414c910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65ea003640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e40cfc80 0x7f65e414c910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65ea003640 1 -- 172.21.15.32:0/1258405493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65dc009a70 con 0x7f65e415cdb0 2023-12-20T04:28:50.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65ea003640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65e415cdb0 0x7f65e4153500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f65dc01bd60 tx=0x7f65dc01bef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65da7fc640 1 -- 172.21.15.32:0/1258405493 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc009450 con 0x7f65e415cdb0 2023-12-20T04:28:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65da7fc640 1 -- 172.21.15.32:0/1258405493 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65dc0b9410 con 0x7f65e415cdb0 2023-12-20T04:28:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65eba66640 1 -- 172.21.15.32:0/1258405493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65e415f800 con 0x7f65e415cdb0 2023-12-20T04:28:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65da7fc640 1 -- 172.21.15.32:0/1258405493 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65dc0b97c0 con 0x7f65e415cdb0 2023-12-20T04:28:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65eba66640 1 -- 172.21.15.32:0/1258405493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65e415fdc0 con 0x7f65e415cdb0 2023-12-20T04:28:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.776+0000 7f65eba66640 1 -- 172.21.15.32:0/1258405493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65e41602a0 con 0x7f65e415cdb0 2023-12-20T04:28:50.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.780+0000 7f65da7fc640 1 -- 172.21.15.32:0/1258405493 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f65dc0b6050 con 0x7f65e415cdb0 2023-12-20T04:28:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.780+0000 7f65da7fc640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65c8065a60 0x7f65c8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.780+0000 7f65da7fc640 1 -- 172.21.15.32:0/1258405493 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f65dc1214d0 con 0x7f65e415cdb0 2023-12-20T04:28:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.780+0000 7f65e9802640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65c8065a60 0x7f65c8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:50.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.780+0000 7f65e9802640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65c8065a60 0x7f65c8067f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65e0001380 tx=0x7f65e0009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:50.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.784+0000 7f65da7fc640 1 -- 172.21.15.32:0/1258405493 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f65dc0eeff0 con 0x7f65e415cdb0 2023-12-20T04:28:50.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.976+0000 7f65eba66640 1 -- 172.21.15.32:0/1258405493 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f65e404e000 con 0x7f65e415cdb0 2023-12-20T04:28:50.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.976+0000 7f65da7fc640 1 -- 172.21.15.32:0/1258405493 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4700 (secure 0 0 0) 0x7f65dc0f2610 con 0x7f65e415cdb0 2023-12-20T04:28:50.978 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:50.978 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2247020919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2247020919},{"type":"v1","addr":"172.21.15.39:6834","nonce":2247020919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/797559820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":797559820},{"type":"v1","addr":"172.21.15.32:6836","nonce":797559820}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3209531230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3209531230},{"type":"v1","addr":"172.21.15.32:6837","nonce":3209531230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:28:38.361494+0000","modified":"2023-12-20T04:28:41.379636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547681098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3547681098},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547681098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-20T04:28:50.978 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-12-20T04:28:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 -- 172.21.15.32:0/1258405493 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65c8065a60 msgr2=0x7f65c8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65c8065a60 0x7f65c8067f20 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f65e0001380 tx=0x7f65e0009040 comp rx=0 tx=0).stop 2023-12-20T04:28:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 -- 172.21.15.32:0/1258405493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65e415cdb0 msgr2=0x7f65e4153500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65e415cdb0 0x7f65e4153500 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f65dc01bd60 tx=0x7f65dc01bef0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 -- 172.21.15.32:0/1258405493 shutdown_connections 2023-12-20T04:28:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65c8065a60 0x7f65c8067f20 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65e415cdb0 0x7f65e4153500 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65e40d2740 0x7f65e41f6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 --2- 172.21.15.32:0/1258405493 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65e40cfc80 0x7f65e414c910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 -- 172.21.15.32:0/1258405493 >> 172.21.15.32:0/1258405493 conn(0x7f65e4065ee0 msgr2=0x7f65e414dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 -- 172.21.15.32:0/1258405493 shutdown_connections 2023-12-20T04:28:50.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:50.980+0000 7f65eba66640 1 -- 172.21.15.32:0/1258405493 wait complete. 2023-12-20T04:28:50.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:51.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.144+0000 7f03bf1e9640 1 Processor -- start 2023-12-20T04:28:51.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.144+0000 7f03bf1e9640 1 -- start start 2023-12-20T04:28:51.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.144+0000 7f03bf1e9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f03b80af5e0 0x7f03b80af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.144+0000 7f03bf1e9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b815b990 con 0x7f03b806ce60 2023-12-20T04:28:51.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.144+0000 7f03bf1e9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03b815bbd0 con 0x7f03b815f110 2023-12-20T04:28:51.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.144+0000 7f03bf1e9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03b815beb0 con 0x7f03b80af5e0 2023-12-20T04:28:51.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.144+0000 7f03b7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f03b806ce60 0x7f03b815b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52732/0 (socket says 172.21.15.32:52732) 2023-12-20T04:28:51.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.144+0000 7f03b7fff640 1 -- 172.21.15.32:0/4081619324 learned_addr learned my addr 172.21.15.32:0/4081619324 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.144+0000 7f03bcf85640 1 --2- 172.21.15.32:0/4081619324 >> v2:172.21.15.39:3300/0 conn(0x7f03b80af5e0 0x7f03b80af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.144+0000 7f03b77fe640 1 -- 172.21.15.32:0/4081619324 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 382533687 0 0) 0x7f03a4001120 con 0x7f03b806ce60 2023-12-20T04:28:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.144+0000 7f03b77fe640 1 -- 172.21.15.32:0/4081619324 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f03a8003530 con 0x7f03b806ce60 2023-12-20T04:28:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.148+0000 7f03bcf85640 1 -- 172.21.15.32:0/4081619324 >> v1:172.21.15.39:6789/0 conn(0x7f03b815f110 legacy=0x7f03b815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.148+0000 7f03b77fe640 1 -- 172.21.15.32:0/4081619324 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1053835590 0 0) 0x7f03a8003530 con 0x7f03b806ce60 2023-12-20T04:28:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.148+0000 7f03bcf85640 1 -- 172.21.15.32:0/4081619324 >> v1:172.21.15.32:6789/0 conn(0x7f03b806ce60 legacy=0x7f03b815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.148+0000 7f03bcf85640 1 -- 172.21.15.32:0/4081619324 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03b815e910 con 0x7f03b80af5e0 2023-12-20T04:28:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.148+0000 7f03bcf85640 1 --2- 172.21.15.32:0/4081619324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b80af5e0 0x7f03b80af9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0097a0 tx=0x7f03ac01c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2e2513f9cff166e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.148+0000 7f03b77fe640 1 -- 172.21.15.32:0/4081619324 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03a4001810 con 0x7f03b80af5e0 2023-12-20T04:28:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.148+0000 7f03b77fe640 1 -- 172.21.15.32:0/4081619324 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03ac0a7050 con 0x7f03b80af5e0 2023-12-20T04:28:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.148+0000 7f03b77fe640 1 -- 172.21.15.32:0/4081619324 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03ac0ab620 con 0x7f03b80af5e0 2023-12-20T04:28:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.148+0000 7f03bf1e9640 1 -- 172.21.15.32:0/4081619324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b80af5e0 msgr2=0x7f03b80af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.148+0000 7f03bf1e9640 1 --2- 172.21.15.32:0/4081619324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b80af5e0 0x7f03b80af9c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0097a0 tx=0x7f03ac01c350 comp rx=0 tx=0).stop 2023-12-20T04:28:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 -- 172.21.15.32:0/4081619324 shutdown_connections 2023-12-20T04:28:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 --2- 172.21.15.32:0/4081619324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b80af5e0 0x7f03b80af9c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 -- 172.21.15.32:0/4081619324 >> 172.21.15.32:0/4081619324 conn(0x7f03b806a320 msgr2=0x7f03b80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:51.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 -- 172.21.15.32:0/4081619324 shutdown_connections 2023-12-20T04:28:51.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 -- 172.21.15.32:0/4081619324 wait complete. 2023-12-20T04:28:51.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 Processor -- start 2023-12-20T04:28:51.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 -- start start 2023-12-20T04:28:51.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b806ce60 0x7f03b814e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f03b80af5e0 0x7f03b80c7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b815f110 0x7f03b815dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b814e140 con 0x7f03b815f110 2023-12-20T04:28:51.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03b80c86b0 con 0x7f03b806ce60 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bf1e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03b814f280 con 0x7f03b80af5e0 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bd786640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b815f110 0x7f03b815dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bd786640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b815f110 0x7f03b815dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45328/0 (socket says 172.21.15.32:45328) 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bd786640 1 -- 172.21.15.32:0/3339693630 learned_addr learned my addr 172.21.15.32:0/3339693630 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bcf85640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b806ce60 0x7f03b814e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03b7fff640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f03b80af5e0 0x7f03b80c7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bd786640 1 -- 172.21.15.32:0/3339693630 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f03b80af5e0 msgr2=0x7f03b80c7eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bd786640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f03b80af5e0 0x7f03b80c7eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bd786640 1 -- 172.21.15.32:0/3339693630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b806ce60 msgr2=0x7f03b814e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bd786640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b806ce60 0x7f03b814e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bd786640 1 -- 172.21.15.32:0/3339693630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03ac0093b0 con 0x7f03b815f110 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.152+0000 7f03bd786640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b815f110 0x7f03b815dc40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c730 tx=0x7f03b000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:51.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.156+0000 7f03b57fa640 1 -- 172.21.15.32:0/3339693630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b0003b80 con 0x7f03b815f110 2023-12-20T04:28:51.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.156+0000 7f03b57fa640 1 -- 172.21.15.32:0/3339693630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03b0003ce0 con 0x7f03b815f110 2023-12-20T04:28:51.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.156+0000 7f03b57fa640 1 -- 172.21.15.32:0/3339693630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03b000aa30 con 0x7f03b815f110 2023-12-20T04:28:51.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.156+0000 7f03bf1e9640 1 -- 172.21.15.32:0/3339693630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03b80c83f0 con 0x7f03b815f110 2023-12-20T04:28:51.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.156+0000 7f03bf1e9640 1 -- 172.21.15.32:0/3339693630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03b80c6650 con 0x7f03b815f110 2023-12-20T04:28:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.156+0000 7f03bf1e9640 1 -- 172.21.15.32:0/3339693630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03b814e2b0 con 0x7f03b815f110 2023-12-20T04:28:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.156+0000 7f03b57fa640 1 -- 172.21.15.32:0/3339693630 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f03b000ab90 con 0x7f03b815f110 2023-12-20T04:28:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.156+0000 7f03b57fa640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f039c065a90 0x7f039c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.156+0000 7f03b57fa640 1 -- 172.21.15.32:0/3339693630 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f03b0082f30 con 0x7f03b815f110 2023-12-20T04:28:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.156+0000 7f03bcf85640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f039c065a90 0x7f039c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.160+0000 7f03bcf85640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f039c065a90 0x7f039c067f50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0ad000 tx=0x7f03ac0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:51.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.160+0000 7f03b57fa640 1 -- 172.21.15.32:0/3339693630 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f03b814e2b0 con 0x7f03b815f110 2023-12-20T04:28:51.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.320+0000 7f03bf1e9640 1 -- 172.21.15.32:0/3339693630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f03b814e280 con 0x7f03b815f110 2023-12-20T04:28:51.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.320+0000 7f03b57fa640 1 -- 172.21.15.32:0/3339693630 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f03b00508e0 con 0x7f03b815f110 2023-12-20T04:28:51.327 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:51.327 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 -- 172.21.15.32:0/3339693630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f039c065a90 msgr2=0x7f039c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f039c065a90 0x7f039c067f50 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7f03ac0ad000 tx=0x7f03ac0a6000 comp rx=0 tx=0).stop 2023-12-20T04:28:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 -- 172.21.15.32:0/3339693630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b815f110 msgr2=0x7f03b815dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b815f110 0x7f03b815dc40 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f03b000c730 tx=0x7f03b000cbf0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 -- 172.21.15.32:0/3339693630 shutdown_connections 2023-12-20T04:28:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f039c065a90 0x7f039c067f50 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03b815f110 0x7f03b815dc40 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f03b80af5e0 0x7f03b80c7eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 --2- 172.21.15.32:0/3339693630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03b806ce60 0x7f03b814e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 -- 172.21.15.32:0/3339693630 >> 172.21.15.32:0/3339693630 conn(0x7f03b806a320 msgr2=0x7f03b80ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:51.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 -- 172.21.15.32:0/3339693630 shutdown_connections 2023-12-20T04:28:51.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.324+0000 7f03bf1e9640 1 -- 172.21.15.32:0/3339693630 wait complete. 2023-12-20T04:28:51.343 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:28:36.040195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:28:37.364671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:28:51.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:28:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.488+0000 7f3c04f09640 1 Processor -- start 2023-12-20T04:28:51.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.488+0000 7f3c04f09640 1 -- start start 2023-12-20T04:28:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3c04f09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3c000b0cb0 0x7f3c000caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3c04f09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3c00070230 0x7f3c000706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3c04f09640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3c0015b6a0 con 0x7f3c0015abb0 2023-12-20T04:28:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3c04f09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3c0006e160 con 0x7f3c000b0cb0 2023-12-20T04:28:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3c04f09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c00070c00 con 0x7f3c00070230 2023-12-20T04:28:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3bfe59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3c0015abb0 0x7f3c0015bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52742/0 (socket says 172.21.15.32:52742) 2023-12-20T04:28:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3bfe59c640 1 -- 172.21.15.32:0/997660640 learned_addr learned my addr 172.21.15.32:0/997660640 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:51.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3bf7fff640 1 --2- 172.21.15.32:0/997660640 >> v2:172.21.15.32:3300/0 conn(0x7f3c000b0cb0 0x7f3c000caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3bfed9d640 1 --2- 172.21.15.32:0/997660640 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3c00070230 0x7f3c000706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3bf7fff640 1 -- 172.21.15.32:0/997660640 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3c00070230 msgr2=0x7f3c000706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3bf7fff640 1 --2- 172.21.15.32:0/997660640 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3c00070230 0x7f3c000706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3bfdd9b640 1 -- 172.21.15.32:0/997660640 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2921875652 0 0) 0x7f3c0015b6a0 con 0x7f3c0015abb0 2023-12-20T04:28:51.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3bf7fff640 1 -- 172.21.15.32:0/997660640 >> v1:172.21.15.32:6789/0 conn(0x7f3c0015abb0 legacy=0x7f3c0015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.492+0000 7f3bf7fff640 1 -- 172.21.15.32:0/997660640 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3c0006d8c0 con 0x7f3c000b0cb0 2023-12-20T04:28:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3bf7fff640 1 --2- 172.21.15.32:0/997660640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c000b0cb0 0x7f3c000caaf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3be80097c0 tx=0x7f3be801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3fb0e3826d8e4fda server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3bfdd9b640 1 -- 172.21.15.32:0/997660640 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be80ab680 con 0x7f3c000b0cb0 2023-12-20T04:28:51.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3bfdd9b640 1 -- 172.21.15.32:0/997660640 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be80ab7e0 con 0x7f3c000b0cb0 2023-12-20T04:28:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3bfdd9b640 1 -- 172.21.15.32:0/997660640 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be80abb70 con 0x7f3c000b0cb0 2023-12-20T04:28:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3c04f09640 1 -- 172.21.15.32:0/997660640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c000b0cb0 msgr2=0x7f3c000caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3c04f09640 1 --2- 172.21.15.32:0/997660640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c000b0cb0 0x7f3c000caaf0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f3be80097c0 tx=0x7f3be801c480 comp rx=0 tx=0).stop 2023-12-20T04:28:51.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3c04f09640 1 -- 172.21.15.32:0/997660640 shutdown_connections 2023-12-20T04:28:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3c04f09640 1 --2- 172.21.15.32:0/997660640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c000b0cb0 0x7f3c000caaf0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3c04f09640 1 --2- 172.21.15.32:0/997660640 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3c00070230 0x7f3c000706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3c04f09640 1 -- 172.21.15.32:0/997660640 >> 172.21.15.32:0/997660640 conn(0x7f3c0006a510 msgr2=0x7f3c000ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3c04f09640 1 -- 172.21.15.32:0/997660640 shutdown_connections 2023-12-20T04:28:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3c04f09640 1 -- 172.21.15.32:0/997660640 wait complete. 2023-12-20T04:28:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3c04f09640 1 Processor -- start 2023-12-20T04:28:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.496+0000 7f3c04f09640 1 -- start start 2023-12-20T04:28:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3c04f09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3c00070230 0x7f3c0014f350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3c04f09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c000b0cb0 0x7f3c001f5c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3c04f09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c0015abb0 0x7f3c001f8770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3c04f09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c00150600 con 0x7f3c000b0cb0 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3c04f09640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3c001f6140 con 0x7f3c0015abb0 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3c04f09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3c0014f9f0 con 0x7f3c00070230 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c000b0cb0 0x7f3c001f5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c000b0cb0 0x7f3c001f5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45340/0 (socket says 172.21.15.32:45340) 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf7fff640 1 -- 172.21.15.32:0/36173320 learned_addr learned my addr 172.21.15.32:0/36173320 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bfe59c640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3c00070230 0x7f3c0014f350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bfed9d640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c0015abb0 0x7f3c001f8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf7fff640 1 -- 172.21.15.32:0/36173320 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3c00070230 msgr2=0x7f3c0014f350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf7fff640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3c00070230 0x7f3c0014f350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf7fff640 1 -- 172.21.15.32:0/36173320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c0015abb0 msgr2=0x7f3c001f8770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf7fff640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c0015abb0 0x7f3c001f8770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf7fff640 1 -- 172.21.15.32:0/36173320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3be8009420 con 0x7f3c000b0cb0 2023-12-20T04:28:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bfed9d640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c0015abb0 0x7f3c001f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf7fff640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c000b0cb0 0x7f3c001f5c00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3be801ab30 tx=0x7f3be801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf6ffd640 1 -- 172.21.15.32:0/36173320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be80b5810 con 0x7f3c000b0cb0 2023-12-20T04:28:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf6ffd640 1 -- 172.21.15.32:0/36173320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3be80b5970 con 0x7f3c000b0cb0 2023-12-20T04:28:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3c04f09640 1 -- 172.21.15.32:0/36173320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3c0014fcd0 con 0x7f3c000b0cb0 2023-12-20T04:28:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf6ffd640 1 -- 172.21.15.32:0/36173320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3be80b5d20 con 0x7f3c000b0cb0 2023-12-20T04:28:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3c04f09640 1 -- 172.21.15.32:0/36173320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3c00150290 con 0x7f3c000b0cb0 2023-12-20T04:28:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3c04f09640 1 -- 172.21.15.32:0/36173320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3c001507b0 con 0x7f3c000b0cb0 2023-12-20T04:28:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.500+0000 7f3bf6ffd640 1 -- 172.21.15.32:0/36173320 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3be80bf410 con 0x7f3c000b0cb0 2023-12-20T04:28:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.504+0000 7f3bf6ffd640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3be0065a60 0x7f3be0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.504+0000 7f3bf6ffd640 1 -- 172.21.15.32:0/36173320 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3be811dfe0 con 0x7f3c000b0cb0 2023-12-20T04:28:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.504+0000 7f3bfe59c640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3be0065a60 0x7f3be0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.504+0000 7f3bfe59c640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3be0065a60 0x7f3be0067f20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0000f70 tx=0x7f3bf0007c10 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:51.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.508+0000 7f3bf6ffd640 1 -- 172.21.15.32:0/36173320 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3c001507b0 con 0x7f3c000b0cb0 2023-12-20T04:28:51.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 -- 172.21.15.32:0/36173320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3c001507b0 con 0x7f3c000b0cb0 2023-12-20T04:28:51.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3bf6ffd640 1 -- 172.21.15.32:0/36173320 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3c001507b0 con 0x7f3c000b0cb0 2023-12-20T04:28:51.664 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:28:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 -- 172.21.15.32:0/36173320 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3be0065a60 msgr2=0x7f3be0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3be0065a60 0x7f3be0067f20 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f3bf0000f70 tx=0x7f3bf0007c10 comp rx=0 tx=0).stop 2023-12-20T04:28:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 -- 172.21.15.32:0/36173320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c000b0cb0 msgr2=0x7f3c001f5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c000b0cb0 0x7f3c001f5c00 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3be801ab30 tx=0x7f3be801acc0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 -- 172.21.15.32:0/36173320 shutdown_connections 2023-12-20T04:28:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3be0065a60 0x7f3be0067f20 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3c0015abb0 0x7f3c001f8770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3c000b0cb0 0x7f3c001f5c00 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 --2- 172.21.15.32:0/36173320 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3c00070230 0x7f3c0014f350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 -- 172.21.15.32:0/36173320 >> 172.21.15.32:0/36173320 conn(0x7f3c0006a510 msgr2=0x7f3c000cb710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:51.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 -- 172.21.15.32:0/36173320 shutdown_connections 2023-12-20T04:28:51.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.664+0000 7f3c04f09640 1 -- 172.21.15.32:0/36173320 wait complete. 2023-12-20T04:28:51.685 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:28:51.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f164a925640 1 Processor -- start 2023-12-20T04:28:51.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f164a925640 1 -- start start 2023-12-20T04:28:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f164a925640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f164415ce60 0x7f16440d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f164a925640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16440d2650 con 0x7f16440d3130 2023-12-20T04:28:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f164a925640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16440d1a40 con 0x7f16440d14d0 2023-12-20T04:28:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f164a925640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16440d35a0 con 0x7f164415ce60 2023-12-20T04:28:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f16437fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f16440d3130 0x7f16440d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52746/0 (socket says 172.21.15.32:52746) 2023-12-20T04:28:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f1643fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f164415ce60 0x7f16440d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f16437fe640 1 -- 172.21.15.32:0/2285377418 learned_addr learned my addr 172.21.15.32:0/2285377418 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f1643fff640 1 -- 172.21.15.32:0/2285377418 >> v1:172.21.15.39:6789/0 conn(0x7f16440d14d0 legacy=0x7f16440b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f1643fff640 1 -- 172.21.15.32:0/2285377418 >> v1:172.21.15.32:6789/0 conn(0x7f16440d3130 legacy=0x7f16440d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f1643fff640 1 -- 172.21.15.32:0/2285377418 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16440d1d20 con 0x7f164415ce60 2023-12-20T04:28:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.832+0000 7f1642ffd640 1 -- 172.21.15.32:0/2285377418 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3341764385 0 0) 0x7f16440d2650 con 0x7f16440d3130 2023-12-20T04:28:51.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.836+0000 7f1643fff640 1 --2- 172.21.15.32:0/2285377418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f164415ce60 0x7f16440d2bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16340097f0 tx=0x7f163401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8358e2aff3b3e926 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.836+0000 7f1642ffd640 1 -- 172.21.15.32:0/2285377418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16340a8970 con 0x7f164415ce60 2023-12-20T04:28:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.836+0000 7f1642ffd640 1 -- 172.21.15.32:0/2285377418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16340a8ad0 con 0x7f164415ce60 2023-12-20T04:28:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.836+0000 7f1642ffd640 1 -- 172.21.15.32:0/2285377418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16340a8e60 con 0x7f164415ce60 2023-12-20T04:28:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.836+0000 7f164a925640 1 -- 172.21.15.32:0/2285377418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f164415ce60 msgr2=0x7f16440d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.836+0000 7f164a925640 1 --2- 172.21.15.32:0/2285377418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f164415ce60 0x7f16440d2bf0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f16340097f0 tx=0x7f163401ec70 comp rx=0 tx=0).stop 2023-12-20T04:28:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.836+0000 7f164a925640 1 -- 172.21.15.32:0/2285377418 shutdown_connections 2023-12-20T04:28:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.836+0000 7f164a925640 1 --2- 172.21.15.32:0/2285377418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f164415ce60 0x7f16440d2bf0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.836+0000 7f164a925640 1 -- 172.21.15.32:0/2285377418 >> 172.21.15.32:0/2285377418 conn(0x7f164406b5e0 msgr2=0x7f164406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.836+0000 7f164a925640 1 -- 172.21.15.32:0/2285377418 shutdown_connections 2023-12-20T04:28:51.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.836+0000 7f164a925640 1 -- 172.21.15.32:0/2285377418 wait complete. 2023-12-20T04:28:51.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f164a925640 1 Processor -- start 2023-12-20T04:28:51.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f164a925640 1 -- start start 2023-12-20T04:28:51.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f164a925640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16440d14d0 0x7f1644152930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f164a925640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16440d3130 0x7f16440ca950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f164a925640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f164415ce60 0x7f1644151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f164a925640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164415f5f0 con 0x7f164415ce60 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f164a925640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f164415ed30 con 0x7f16440d14d0 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f164a925640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f164415efa0 con 0x7f16440d3130 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1648ec2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f164415ce60 0x7f1644151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1648ec2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f164415ce60 0x7f1644151a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45358/0 (socket says 172.21.15.32:45358) 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1648ec2640 1 -- 172.21.15.32:0/2143891263 learned_addr learned my addr 172.21.15.32:0/2143891263 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1643fff640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16440d14d0 0x7f1644152930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f16437fe640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16440d3130 0x7f16440ca950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1648ec2640 1 -- 172.21.15.32:0/2143891263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16440d3130 msgr2=0x7f16440ca950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1648ec2640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16440d3130 0x7f16440ca950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1648ec2640 1 -- 172.21.15.32:0/2143891263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16440d14d0 msgr2=0x7f1644152930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1648ec2640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16440d14d0 0x7f1644152930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1648ec2640 1 -- 172.21.15.32:0/2143891263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1634009420 con 0x7f164415ce60 2023-12-20T04:28:51.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1648ec2640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f164415ce60 0x7f1644151a30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f163c00c730 tx=0x7f163c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:51.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1640ff9640 1 -- 172.21.15.32:0/2143891263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c003b80 con 0x7f164415ce60 2023-12-20T04:28:51.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1640ff9640 1 -- 172.21.15.32:0/2143891263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f163c003ce0 con 0x7f164415ce60 2023-12-20T04:28:51.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f1640ff9640 1 -- 172.21.15.32:0/2143891263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f163c005310 con 0x7f164415ce60 2023-12-20T04:28:51.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f164a925640 1 -- 172.21.15.32:0/2143891263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f164415f280 con 0x7f164415ce60 2023-12-20T04:28:51.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.840+0000 7f164a925640 1 -- 172.21.15.32:0/2143891263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16440cb360 con 0x7f164415ce60 2023-12-20T04:28:51.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.844+0000 7f164a925640 1 -- 172.21.15.32:0/2143891263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f164415f7a0 con 0x7f164415ce60 2023-12-20T04:28:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.844+0000 7f1640ff9640 1 -- 172.21.15.32:0/2143891263 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f163c005680 con 0x7f164415ce60 2023-12-20T04:28:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.844+0000 7f1640ff9640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1630065a60 0x7f1630067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.844+0000 7f1640ff9640 1 -- 172.21.15.32:0/2143891263 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f163c082ff0 con 0x7f164415ce60 2023-12-20T04:28:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.844+0000 7f1643fff640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1630065a60 0x7f1630067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.844+0000 7f1643fff640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1630065a60 0x7f1630067f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f16340b2000 tx=0x7f1634008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:51.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:51.848+0000 7f1640ff9640 1 -- 172.21.15.32:0/2143891263 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f163c0b40a0 con 0x7f164415ce60 2023-12-20T04:28:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 -- 172.21.15.32:0/2143891263 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f16440d1fd0 con 0x7f164415ce60 2023-12-20T04:28:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f1640ff9640 1 -- 172.21.15.32:0/2143891263 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f164415f7a0 con 0x7f164415ce60 2023-12-20T04:28:52.008 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:28:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 -- 172.21.15.32:0/2143891263 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1630065a60 msgr2=0x7f1630067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1630065a60 0x7f1630067f20 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f16340b2000 tx=0x7f1634008040 comp rx=0 tx=0).stop 2023-12-20T04:28:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 -- 172.21.15.32:0/2143891263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f164415ce60 msgr2=0x7f1644151a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f164415ce60 0x7f1644151a30 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f163c00c730 tx=0x7f163c00cbf0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 -- 172.21.15.32:0/2143891263 shutdown_connections 2023-12-20T04:28:52.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1630065a60 0x7f1630067f20 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f164415ce60 0x7f1644151a30 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16440d3130 0x7f16440ca950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 --2- 172.21.15.32:0/2143891263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16440d14d0 0x7f1644152930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 -- 172.21.15.32:0/2143891263 >> 172.21.15.32:0/2143891263 conn(0x7f164406b5e0 msgr2=0x7f164406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 -- 172.21.15.32:0/2143891263 shutdown_connections 2023-12-20T04:28:52.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.008+0000 7f164a925640 1 -- 172.21.15.32:0/2143891263 wait complete. 2023-12-20T04:28:52.029 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:28:52.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.171+0000 7ff9e722d640 1 Processor -- start 2023-12-20T04:28:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.171+0000 7ff9e722d640 1 -- start start 2023-12-20T04:28:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e722d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9e00cb050 0x7ff9e00cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e722d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9e006d170 0x7ff9e0161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e722d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9e006a1f0 con 0x7ff9e0065d40 2023-12-20T04:28:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e722d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9e0161860 con 0x7ff9e00cb050 2023-12-20T04:28:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e722d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e006a440 con 0x7ff9e006d170 2023-12-20T04:28:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e4fc9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9e00cb050 0x7ff9e00cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e4fc9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9e00cb050 0x7ff9e00cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45360/0 (socket says 172.21.15.32:45360) 2023-12-20T04:28:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e4fc9640 1 -- 172.21.15.32:0/556884510 learned_addr learned my addr 172.21.15.32:0/556884510 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e57ca640 1 --2- 172.21.15.32:0/556884510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9e006d170 0x7ff9e0161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e4fc9640 1 -- 172.21.15.32:0/556884510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9e006d170 msgr2=0x7ff9e0161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e4fc9640 1 --2- 172.21.15.32:0/556884510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9e006d170 0x7ff9e0161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e4fc9640 1 -- 172.21.15.32:0/556884510 >> v1:172.21.15.39:6789/0 conn(0x7ff9e0065d40 legacy=0x7ff9e006ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.175+0000 7ff9e4fc9640 1 -- 172.21.15.32:0/556884510 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9e006a720 con 0x7ff9e00cb050 2023-12-20T04:28:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9e4fc9640 1 --2- 172.21.15.32:0/556884510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9e00cb050 0x7ff9e00cb430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009870 tx=0x7ff9d401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2af49d22171e8130 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9df7fe640 1 -- 172.21.15.32:0/556884510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40aa670 con 0x7ff9e00cb050 2023-12-20T04:28:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9df7fe640 1 -- 172.21.15.32:0/556884510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9d40aa7d0 con 0x7ff9e00cb050 2023-12-20T04:28:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9df7fe640 1 -- 172.21.15.32:0/556884510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9d40aab60 con 0x7ff9e00cb050 2023-12-20T04:28:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9e722d640 1 -- 172.21.15.32:0/556884510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9e00cb050 msgr2=0x7ff9e00cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9e722d640 1 --2- 172.21.15.32:0/556884510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9e00cb050 0x7ff9e00cb430 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7ff9d4009870 tx=0x7ff9d401c510 comp rx=0 tx=0).stop 2023-12-20T04:28:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9e722d640 1 -- 172.21.15.32:0/556884510 shutdown_connections 2023-12-20T04:28:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9e722d640 1 --2- 172.21.15.32:0/556884510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9e006d170 0x7ff9e0161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9e722d640 1 --2- 172.21.15.32:0/556884510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9e00cb050 0x7ff9e00cb430 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9e722d640 1 -- 172.21.15.32:0/556884510 >> 172.21.15.32:0/556884510 conn(0x7ff9e00cf8d0 msgr2=0x7ff9e00cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9e722d640 1 -- 172.21.15.32:0/556884510 shutdown_connections 2023-12-20T04:28:52.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9e722d640 1 -- 172.21.15.32:0/556884510 wait complete. 2023-12-20T04:28:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9e722d640 1 Processor -- start 2023-12-20T04:28:52.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.179+0000 7ff9e722d640 1 -- start start 2023-12-20T04:28:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9e722d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e0065d40 0x7ff9e015b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9e722d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9e006d170 0x7ff9e00c7070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9e722d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9e00cb050 0x7ff9e015a7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9e722d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c8580 con 0x7ff9e006d170 2023-12-20T04:28:52.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9e722d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c89a0 con 0x7ff9e0065d40 2023-12-20T04:28:52.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9e722d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9e00c7900 con 0x7ff9e00cb050 2023-12-20T04:28:52.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9e006d170 0x7ff9e00c7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9e006d170 0x7ff9e00c7070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45366/0 (socket says 172.21.15.32:45366) 2023-12-20T04:28:52.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dffff640 1 -- 172.21.15.32:0/1821856154 learned_addr learned my addr 172.21.15.32:0/1821856154 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:52.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9e4fc9640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e0065d40 0x7ff9e015b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9e57ca640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9e00cb050 0x7ff9e015a7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dffff640 1 -- 172.21.15.32:0/1821856154 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9e00cb050 msgr2=0x7ff9e015a7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dffff640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9e00cb050 0x7ff9e015a7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dffff640 1 -- 172.21.15.32:0/1821856154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e0065d40 msgr2=0x7ff9e015b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dffff640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e0065d40 0x7ff9e015b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dffff640 1 -- 172.21.15.32:0/1821856154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9d40094d0 con 0x7ff9e006d170 2023-12-20T04:28:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dffff640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9e006d170 0x7ff9e00c7070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00c3c0 tx=0x7ff9cc00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dd7fa640 1 -- 172.21.15.32:0/1821856154 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc0052d0 con 0x7ff9e006d170 2023-12-20T04:28:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dd7fa640 1 -- 172.21.15.32:0/1821856154 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9cc005430 con 0x7ff9e006d170 2023-12-20T04:28:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9e722d640 1 -- 172.21.15.32:0/1821856154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9e00c7be0 con 0x7ff9e006d170 2023-12-20T04:28:52.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dd7fa640 1 -- 172.21.15.32:0/1821856154 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9cc003840 con 0x7ff9e006d170 2023-12-20T04:28:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9e722d640 1 -- 172.21.15.32:0/1821856154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9e00c81a0 con 0x7ff9e006d170 2023-12-20T04:28:52.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9e722d640 1 -- 172.21.15.32:0/1821856154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9e00c86c0 con 0x7ff9e006d170 2023-12-20T04:28:52.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.183+0000 7ff9dd7fa640 1 -- 172.21.15.32:0/1821856154 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff9cc0039a0 con 0x7ff9e006d170 2023-12-20T04:28:52.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.187+0000 7ff9dd7fa640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9c4065a60 0x7ff9c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.187+0000 7ff9dd7fa640 1 -- 172.21.15.32:0/1821856154 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff9cc082530 con 0x7ff9e006d170 2023-12-20T04:28:52.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.187+0000 7ff9e4fc9640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9c4065a60 0x7ff9c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.187+0000 7ff9e4fc9640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9c4065a60 0x7ff9c4067f20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40ae000 tx=0x7ff9d40b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:52.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.191+0000 7ff9dd7fa640 1 -- 172.21.15.32:0/1821856154 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff9e00c86c0 con 0x7ff9e006d170 2023-12-20T04:28:52.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 -- 172.21.15.32:0/1821856154 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ff9e00c86c0 con 0x7ff9e006d170 2023-12-20T04:28:52.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9dd7fa640 1 -- 172.21.15.32:0/1821856154 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7ff9e00c86c0 con 0x7ff9e006d170 2023-12-20T04:28:52.348 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:28:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 -- 172.21.15.32:0/1821856154 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9c4065a60 msgr2=0x7ff9c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9c4065a60 0x7ff9c4067f20 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7ff9d40ae000 tx=0x7ff9d40b5040 comp rx=0 tx=0).stop 2023-12-20T04:28:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 -- 172.21.15.32:0/1821856154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9e006d170 msgr2=0x7ff9e00c7070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9e006d170 0x7ff9e00c7070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7ff9cc00c3c0 tx=0x7ff9cc00c880 comp rx=0 tx=0).stop 2023-12-20T04:28:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 -- 172.21.15.32:0/1821856154 shutdown_connections 2023-12-20T04:28:52.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff9c4065a60 0x7ff9c4067f20 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9e00cb050 0x7ff9e015a7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9e006d170 0x7ff9e00c7070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 --2- 172.21.15.32:0/1821856154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9e0065d40 0x7ff9e015b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 -- 172.21.15.32:0/1821856154 >> 172.21.15.32:0/1821856154 conn(0x7ff9e00cf8d0 msgr2=0x7ff9e015fc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 -- 172.21.15.32:0/1821856154 shutdown_connections 2023-12-20T04:28:52.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.347+0000 7ff9e722d640 1 -- 172.21.15.32:0/1821856154 wait complete. 2023-12-20T04:28:52.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:28:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e94803640 1 Processor -- start 2023-12-20T04:28:52.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e94803640 1 -- start start 2023-12-20T04:28:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e94803640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e900b8480 0x7f3e9006af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e94803640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e9006b460 0x7f3e90160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e94803640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3e90161330 0x7f3e90153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e94803640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e9006b8c0 con 0x7f3e9006b460 2023-12-20T04:28:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e94803640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e900cd4d0 con 0x7f3e90161330 2023-12-20T04:28:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e94803640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900ce4e0 con 0x7f3e900b8480 2023-12-20T04:28:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e8e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e9006b460 0x7f3e90160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e8e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e9006b460 0x7f3e90160df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45380/0 (socket says 172.21.15.32:45380) 2023-12-20T04:28:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e8e59c640 1 -- 172.21.15.32:0/3785958924 learned_addr learned my addr 172.21.15.32:0/3785958924 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e8e59c640 1 -- 172.21.15.32:0/3785958924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e900b8480 msgr2=0x7f3e9006af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e8e59c640 1 --2- 172.21.15.32:0/3785958924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e900b8480 0x7f3e9006af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e8e59c640 1 -- 172.21.15.32:0/3785958924 >> v2:172.21.15.39:3300/0 conn(0x7f3e90161330 msgr2=0x7f3e90153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e8e59c640 1 --2- 172.21.15.32:0/3785958924 >> v2:172.21.15.39:3300/0 conn(0x7f3e90161330 0x7f3e90153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.511+0000 7f3e8e59c640 1 -- 172.21.15.32:0/3785958924 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e900cd720 con 0x7f3e9006b460 2023-12-20T04:28:52.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e8e59c640 1 --2- 172.21.15.32:0/3785958924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e9006b460 0x7f3e90160df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3e9006bae0 tx=0x7f3e8401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b73858decbd990a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e8dd9b640 1 -- 172.21.15.32:0/3785958924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e840ac410 con 0x7f3e9006b460 2023-12-20T04:28:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e8dd9b640 1 -- 172.21.15.32:0/3785958924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e840ac570 con 0x7f3e9006b460 2023-12-20T04:28:52.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e8dd9b640 1 -- 172.21.15.32:0/3785958924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e840ac900 con 0x7f3e9006b460 2023-12-20T04:28:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e94803640 1 -- 172.21.15.32:0/3785958924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e9006b460 msgr2=0x7f3e90160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e94803640 1 --2- 172.21.15.32:0/3785958924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e9006b460 0x7f3e90160df0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3e9006bae0 tx=0x7f3e8401c500 comp rx=0 tx=0).stop 2023-12-20T04:28:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e94803640 1 -- 172.21.15.32:0/3785958924 shutdown_connections 2023-12-20T04:28:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e94803640 1 --2- 172.21.15.32:0/3785958924 >> v2:172.21.15.39:3300/0 conn(0x7f3e90161330 0x7f3e90153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e94803640 1 --2- 172.21.15.32:0/3785958924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e9006b460 0x7f3e90160df0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e94803640 1 --2- 172.21.15.32:0/3785958924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e900b8480 0x7f3e9006af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e94803640 1 -- 172.21.15.32:0/3785958924 >> 172.21.15.32:0/3785958924 conn(0x7f3e900d2a50 msgr2=0x7f3e900d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e94803640 1 -- 172.21.15.32:0/3785958924 shutdown_connections 2023-12-20T04:28:52.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.515+0000 7f3e94803640 1 -- 172.21.15.32:0/3785958924 wait complete. 2023-12-20T04:28:52.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e94803640 1 Processor -- start 2023-12-20T04:28:52.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e94803640 1 -- start start 2023-12-20T04:28:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e94803640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e9006b460 0x7f3e90155b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e94803640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e900b8480 0x7f3e90147c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e94803640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e90161330 0x7f3e900c6fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e94803640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900c6850 con 0x7f3e900b8480 2023-12-20T04:28:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e94803640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e900c6b30 con 0x7f3e90161330 2023-12-20T04:28:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e94803640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e90156620 con 0x7f3e9006b460 2023-12-20T04:28:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e900b8480 0x7f3e90147c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e900b8480 0x7f3e90147c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45392/0 (socket says 172.21.15.32:45392) 2023-12-20T04:28:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8e59c640 1 -- 172.21.15.32:0/2737016819 learned_addr learned my addr 172.21.15.32:0/2737016819 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8f59e640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e90161330 0x7f3e900c6fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8ed9d640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e9006b460 0x7f3e90155b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8e59c640 1 -- 172.21.15.32:0/2737016819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e9006b460 msgr2=0x7f3e90155b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8e59c640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e9006b460 0x7f3e90155b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8e59c640 1 -- 172.21.15.32:0/2737016819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e90161330 msgr2=0x7f3e900c6fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8e59c640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e90161330 0x7f3e900c6fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8e59c640 1 -- 172.21.15.32:0/2737016819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e840094d0 con 0x7f3e900b8480 2023-12-20T04:28:52.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8f59e640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e90161330 0x7f3e900c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:52.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e8e59c640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e900b8480 0x7f3e90147c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3e840af040 tx=0x7f3e840192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e777fe640 1 -- 172.21.15.32:0/2737016819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e840acea0 con 0x7f3e900b8480 2023-12-20T04:28:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e777fe640 1 -- 172.21.15.32:0/2737016819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e840b5630 con 0x7f3e900b8480 2023-12-20T04:28:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e94803640 1 -- 172.21.15.32:0/2737016819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e90156900 con 0x7f3e900b8480 2023-12-20T04:28:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e777fe640 1 -- 172.21.15.32:0/2737016819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e840ac410 con 0x7f3e900b8480 2023-12-20T04:28:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.519+0000 7f3e94803640 1 -- 172.21.15.32:0/2737016819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e901562b0 con 0x7f3e900b8480 2023-12-20T04:28:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.523+0000 7f3e94803640 1 -- 172.21.15.32:0/2737016819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e90074100 con 0x7f3e900b8480 2023-12-20T04:28:52.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.523+0000 7f3e777fe640 1 -- 172.21.15.32:0/2737016819 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3e840b3070 con 0x7f3e900b8480 2023-12-20T04:28:52.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.523+0000 7f3e777fe640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e7c065a60 0x7f3e7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.523+0000 7f3e777fe640 1 -- 172.21.15.32:0/2737016819 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e8411d8c0 con 0x7f3e900b8480 2023-12-20T04:28:52.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.523+0000 7f3e8ed9d640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e7c065a60 0x7f3e7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.523+0000 7f3e8ed9d640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e7c065a60 0x7f3e7c067f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3e80006fd0 tx=0x7f3e80008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:52.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.527+0000 7f3e777fe640 1 -- 172.21.15.32:0/2737016819 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3e840eb270 con 0x7f3e900b8480 2023-12-20T04:28:52.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.715+0000 7f3e94803640 1 -- 172.21.15.32:0/2737016819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e90156090 con 0x7f3e900b8480 2023-12-20T04:28:52.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.715+0000 7f3e777fe640 1 -- 172.21.15.32:0/2737016819 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4700 (secure 0 0 0) 0x7f3e840b1090 con 0x7f3e900b8480 2023-12-20T04:28:52.718 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:52.719 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2247020919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2247020919},{"type":"v1","addr":"172.21.15.39:6834","nonce":2247020919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/797559820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":797559820},{"type":"v1","addr":"172.21.15.32:6836","nonce":797559820}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3209531230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3209531230},{"type":"v1","addr":"172.21.15.32:6837","nonce":3209531230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:28:38.361494+0000","modified":"2023-12-20T04:28:41.379636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547681098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3547681098},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547681098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-20T04:28:52.719 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-12-20T04:28:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 -- 172.21.15.32:0/2737016819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e7c065a60 msgr2=0x7f3e7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e7c065a60 0x7f3e7c067f20 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3e80006fd0 tx=0x7f3e80008040 comp rx=0 tx=0).stop 2023-12-20T04:28:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 -- 172.21.15.32:0/2737016819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e900b8480 msgr2=0x7f3e90147c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e900b8480 0x7f3e90147c90 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f3e840af040 tx=0x7f3e840192a0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 -- 172.21.15.32:0/2737016819 shutdown_connections 2023-12-20T04:28:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e7c065a60 0x7f3e7c067f20 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e90161330 0x7f3e900c6fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e900b8480 0x7f3e90147c90 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 --2- 172.21.15.32:0/2737016819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e9006b460 0x7f3e90155b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 -- 172.21.15.32:0/2737016819 >> 172.21.15.32:0/2737016819 conn(0x7f3e900d2a50 msgr2=0x7f3e900d2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:52.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 -- 172.21.15.32:0/2737016819 shutdown_connections 2023-12-20T04:28:52.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.719+0000 7f3e94803640 1 -- 172.21.15.32:0/2737016819 wait complete. 2023-12-20T04:28:52.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:28:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e6991640 1 Processor -- start 2023-12-20T04:28:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e6991640 1 -- start start 2023-12-20T04:28:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e6991640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01e006b630 0x7f01e0160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e6991640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f01e0161330 0x7f01e0153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e6991640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01e00cd4d0 con 0x7f01e00b8480 2023-12-20T04:28:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e6991640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e00ce4e0 con 0x7f01e006b630 2023-12-20T04:28:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e6991640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01e00cd720 con 0x7f01e0161330 2023-12-20T04:28:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e598f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01e00b8480 0x7f01e006af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52762/0 (socket says 172.21.15.32:52762) 2023-12-20T04:28:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e598f640 1 -- 172.21.15.32:0/3852287822 learned_addr learned my addr 172.21.15.32:0/3852287822 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e518e640 1 --2- 172.21.15.32:0/3852287822 >> v2:172.21.15.32:3300/0 conn(0x7f01e006b630 0x7f01e0160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e6190640 1 --2- 172.21.15.32:0/3852287822 >> v2:172.21.15.39:3300/0 conn(0x7f01e0161330 0x7f01e0153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e498d640 1 -- 172.21.15.32:0/3852287822 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2749933001 0 0) 0x7f01e00cd4d0 con 0x7f01e00b8480 2023-12-20T04:28:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e498d640 1 -- 172.21.15.32:0/3852287822 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f01cc003530 con 0x7f01e00b8480 2023-12-20T04:28:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e518e640 1 -- 172.21.15.32:0/3852287822 >> v2:172.21.15.39:3300/0 conn(0x7f01e0161330 msgr2=0x7f01e0153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e518e640 1 --2- 172.21.15.32:0/3852287822 >> v2:172.21.15.39:3300/0 conn(0x7f01e0161330 0x7f01e0153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e518e640 1 -- 172.21.15.32:0/3852287822 >> v1:172.21.15.32:6789/0 conn(0x7f01e00b8480 legacy=0x7f01e006af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.883+0000 7f01e518e640 1 -- 172.21.15.32:0/3852287822 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01e00cda00 con 0x7f01e006b630 2023-12-20T04:28:52.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e518e640 1 --2- 172.21.15.32:0/3852287822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e006b630 0x7f01e0160df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01e0161b20 tx=0x7f01d001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46d8c4d167616854 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:52.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e498d640 1 -- 172.21.15.32:0/3852287822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0004230 con 0x7f01e006b630 2023-12-20T04:28:52.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e498d640 1 -- 172.21.15.32:0/3852287822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d00a7050 con 0x7f01e006b630 2023-12-20T04:28:52.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e498d640 1 -- 172.21.15.32:0/3852287822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d0003620 con 0x7f01e006b630 2023-12-20T04:28:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e6991640 1 -- 172.21.15.32:0/3852287822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e006b630 msgr2=0x7f01e0160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e6991640 1 --2- 172.21.15.32:0/3852287822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e006b630 0x7f01e0160df0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f01e0161b20 tx=0x7f01d001c460 comp rx=0 tx=0).stop 2023-12-20T04:28:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e6991640 1 -- 172.21.15.32:0/3852287822 shutdown_connections 2023-12-20T04:28:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e6991640 1 --2- 172.21.15.32:0/3852287822 >> v2:172.21.15.39:3300/0 conn(0x7f01e0161330 0x7f01e0153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e6991640 1 --2- 172.21.15.32:0/3852287822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e006b630 0x7f01e0160df0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e6991640 1 -- 172.21.15.32:0/3852287822 >> 172.21.15.32:0/3852287822 conn(0x7f01e00d2a50 msgr2=0x7f01e00d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e6991640 1 -- 172.21.15.32:0/3852287822 shutdown_connections 2023-12-20T04:28:52.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.887+0000 7f01e6991640 1 -- 172.21.15.32:0/3852287822 wait complete. 2023-12-20T04:28:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6991640 1 Processor -- start 2023-12-20T04:28:52.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6991640 1 -- start start 2023-12-20T04:28:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6991640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e006b630 0x7f01e00ccf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6991640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01e00b8480 0x7f01e0157b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6991640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e0161330 0x7f01e0158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6991640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00b6320 con 0x7f01e0161330 2023-12-20T04:28:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6991640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01e013fc90 con 0x7f01e006b630 2023-12-20T04:28:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6991640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01e00cd450 con 0x7f01e00b8480 2023-12-20T04:28:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6190640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e0161330 0x7f01e0158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6190640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e0161330 0x7f01e0158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45414/0 (socket says 172.21.15.32:45414) 2023-12-20T04:28:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6190640 1 -- 172.21.15.32:0/3572332270 learned_addr learned my addr 172.21.15.32:0/3572332270 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:52.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e598f640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e006b630 0x7f01e00ccf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e518e640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01e00b8480 0x7f01e0157b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6190640 1 -- 172.21.15.32:0/3572332270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01e00b8480 msgr2=0x7f01e0157b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6190640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01e00b8480 0x7f01e0157b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6190640 1 -- 172.21.15.32:0/3572332270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e006b630 msgr2=0x7f01e00ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6190640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e006b630 0x7f01e00ccf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6190640 1 -- 172.21.15.32:0/3572332270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01d0009420 con 0x7f01e0161330 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e598f640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e006b630 0x7f01e00ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6190640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e0161330 0x7f01e0158060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f01d800efc0 tx=0x7f01d800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01d67fc640 1 -- 172.21.15.32:0/3572332270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d8009ba0 con 0x7f01e0161330 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01d67fc640 1 -- 172.21.15.32:0/3572332270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01d800f040 con 0x7f01e0161330 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01d67fc640 1 -- 172.21.15.32:0/3572332270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01d80146c0 con 0x7f01e0161330 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6991640 1 -- 172.21.15.32:0/3572332270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01e015c100 con 0x7f01e0161330 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.891+0000 7f01e6991640 1 -- 172.21.15.32:0/3572332270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01e015c6c0 con 0x7f01e0161330 2023-12-20T04:28:52.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.895+0000 7f01e6991640 1 -- 172.21.15.32:0/3572332270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01e0074100 con 0x7f01e0161330 2023-12-20T04:28:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.895+0000 7f01d67fc640 1 -- 172.21.15.32:0/3572332270 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f01d8014820 con 0x7f01e0161330 2023-12-20T04:28:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.895+0000 7f01d67fc640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01c4065a90 0x7f01c4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:52.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.895+0000 7f01d67fc640 1 -- 172.21.15.32:0/3572332270 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f01d8083d50 con 0x7f01e0161330 2023-12-20T04:28:52.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.895+0000 7f01e598f640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01c4065a90 0x7f01c4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:52.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.895+0000 7f01e598f640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01c4065a90 0x7f01c4067f50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0011f0 tx=0x7f01dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:52.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:52.899+0000 7f01d67fc640 1 -- 172.21.15.32:0/3572332270 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f01d8051670 con 0x7f01e0161330 2023-12-20T04:28:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.059+0000 7f01e6991640 1 -- 172.21.15.32:0/3572332270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f01e013fdd0 con 0x7f01e0161330 2023-12-20T04:28:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01d67fc640 1 -- 172.21.15.32:0/3572332270 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f01d8054c90 con 0x7f01e0161330 2023-12-20T04:28:53.063 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:53.063 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:28:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 -- 172.21.15.32:0/3572332270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01c4065a90 msgr2=0x7f01c4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01c4065a90 0x7f01c4067f50 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f01dc0011f0 tx=0x7f01dc009040 comp rx=0 tx=0).stop 2023-12-20T04:28:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 -- 172.21.15.32:0/3572332270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e0161330 msgr2=0x7f01e0158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e0161330 0x7f01e0158060 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f01d800efc0 tx=0x7f01d800c490 comp rx=0 tx=0).stop 2023-12-20T04:28:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 -- 172.21.15.32:0/3572332270 shutdown_connections 2023-12-20T04:28:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01c4065a90 0x7f01c4067f50 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01e0161330 0x7f01e0158060 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01e00b8480 0x7f01e0157b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 --2- 172.21.15.32:0/3572332270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01e006b630 0x7f01e00ccf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 -- 172.21.15.32:0/3572332270 >> 172.21.15.32:0/3572332270 conn(0x7f01e00d2a50 msgr2=0x7f01e015d770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 -- 172.21.15.32:0/3572332270 shutdown_connections 2023-12-20T04:28:53.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.063+0000 7f01e6991640 1 -- 172.21.15.32:0/3572332270 wait complete. 2023-12-20T04:28:53.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:28:53.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.227+0000 7fae4bfa5640 1 Processor -- start 2023-12-20T04:28:53.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.227+0000 7fae4bfa5640 1 -- start start 2023-12-20T04:28:53.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae4bfa5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fae4415cdb0 0x7fae440d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:53.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae4bfa5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae440cfc80 0x7fae440b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:53.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae4bfa5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fae4415d190 con 0x7fae440d2740 2023-12-20T04:28:53.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae4bfa5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae440d16a0 con 0x7fae440cfc80 2023-12-20T04:28:53.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae4bfa5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fae440d1980 con 0x7fae4415cdb0 2023-12-20T04:28:53.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae49540640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fae440d2740 0x7fae440cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52770/0 (socket says 172.21.15.32:52770) 2023-12-20T04:28:53.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae4a542640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fae440cfc80 0x7fae440b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:53.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae49540640 1 -- 172.21.15.32:0/2371098172 learned_addr learned my addr 172.21.15.32:0/2371098172 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:53.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae49d41640 1 --2- 172.21.15.32:0/2371098172 >> v2:172.21.15.39:3300/0 conn(0x7fae4415cdb0 0x7fae440d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:53.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae48d3f640 1 -- 172.21.15.32:0/2371098172 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 536325960 0 0) 0x7fae4415d190 con 0x7fae440d2740 2023-12-20T04:28:53.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae48d3f640 1 -- 172.21.15.32:0/2371098172 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fae30003530 con 0x7fae440d2740 2023-12-20T04:28:53.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae4a542640 1 -- 172.21.15.32:0/2371098172 >> v2:172.21.15.39:3300/0 conn(0x7fae4415cdb0 msgr2=0x7fae440d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:53.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae4a542640 1 --2- 172.21.15.32:0/2371098172 >> v2:172.21.15.39:3300/0 conn(0x7fae4415cdb0 0x7fae440d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae4a542640 1 -- 172.21.15.32:0/2371098172 >> v1:172.21.15.32:6789/0 conn(0x7fae440d2740 legacy=0x7fae440cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:53.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae4a542640 1 -- 172.21.15.32:0/2371098172 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae440d1c60 con 0x7fae440cfc80 2023-12-20T04:28:53.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.231+0000 7fae4a542640 1 --2- 172.21.15.32:0/2371098172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae440cfc80 0x7fae440b5c90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fae440667b0 tx=0x7fae3c01f0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c721ac9bcb18ba2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:53.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae48d3f640 1 -- 172.21.15.32:0/2371098172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0b0440 con 0x7fae440cfc80 2023-12-20T04:28:53.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae48d3f640 1 -- 172.21.15.32:0/2371098172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae3c0b05a0 con 0x7fae440cfc80 2023-12-20T04:28:53.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae48d3f640 1 -- 172.21.15.32:0/2371098172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae3c0b0930 con 0x7fae440cfc80 2023-12-20T04:28:53.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 -- 172.21.15.32:0/2371098172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae440cfc80 msgr2=0x7fae440b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:53.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 --2- 172.21.15.32:0/2371098172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae440cfc80 0x7fae440b5c90 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fae440667b0 tx=0x7fae3c01f0f0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 -- 172.21.15.32:0/2371098172 shutdown_connections 2023-12-20T04:28:53.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 --2- 172.21.15.32:0/2371098172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae440cfc80 0x7fae440b5c90 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 --2- 172.21.15.32:0/2371098172 >> v2:172.21.15.39:3300/0 conn(0x7fae4415cdb0 0x7fae440d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 -- 172.21.15.32:0/2371098172 >> 172.21.15.32:0/2371098172 conn(0x7fae44065ee0 msgr2=0x7fae4406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:53.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 -- 172.21.15.32:0/2371098172 shutdown_connections 2023-12-20T04:28:53.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 -- 172.21.15.32:0/2371098172 wait complete. 2023-12-20T04:28:53.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 Processor -- start 2023-12-20T04:28:53.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 -- start start 2023-12-20T04:28:53.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fae440cfc80 0x7fae440c86e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:53.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae440d2740 0x7fae440cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:53.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae4415cdb0 0x7fae441548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:53.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae4414e950 con 0x7fae440d2740 2023-12-20T04:28:53.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fae440c8c20 con 0x7fae4415cdb0 2023-12-20T04:28:53.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae4bfa5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fae440cbfa0 con 0x7fae440cfc80 2023-12-20T04:28:53.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae49540640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae440d2740 0x7fae440cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:53.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae49540640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae440d2740 0x7fae440cb770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45430/0 (socket says 172.21.15.32:45430) 2023-12-20T04:28:53.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.235+0000 7fae49540640 1 -- 172.21.15.32:0/1881269721 learned_addr learned my addr 172.21.15.32:0/1881269721 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:53.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae4a542640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae4415cdb0 0x7fae441548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:53.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae49d41640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fae440cfc80 0x7fae440c86e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:53.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae49540640 1 -- 172.21.15.32:0/1881269721 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fae440cfc80 msgr2=0x7fae440c86e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:53.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae49540640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fae440cfc80 0x7fae440c86e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae49540640 1 -- 172.21.15.32:0/1881269721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae4415cdb0 msgr2=0x7fae441548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae49540640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae4415cdb0 0x7fae441548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae49540640 1 -- 172.21.15.32:0/1881269721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fae3c009a70 con 0x7fae440d2740 2023-12-20T04:28:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae49d41640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fae440cfc80 0x7fae440c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae49540640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae440d2740 0x7fae440cb770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c500 tx=0x7fae3400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae3a7fc640 1 -- 172.21.15.32:0/1881269721 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae34005cf0 con 0x7fae440d2740 2023-12-20T04:28:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae3a7fc640 1 -- 172.21.15.32:0/1881269721 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fae34005e50 con 0x7fae440d2740 2023-12-20T04:28:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae4bfa5640 1 -- 172.21.15.32:0/1881269721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fae440cc280 con 0x7fae440d2740 2023-12-20T04:28:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae3a7fc640 1 -- 172.21.15.32:0/1881269721 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fae340072f0 con 0x7fae440d2740 2023-12-20T04:28:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae4bfa5640 1 -- 172.21.15.32:0/1881269721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fae4414eb60 con 0x7fae440d2740 2023-12-20T04:28:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae4bfa5640 1 -- 172.21.15.32:0/1881269721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fae440ce7e0 con 0x7fae440d2740 2023-12-20T04:28:53.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae3a7fc640 1 -- 172.21.15.32:0/1881269721 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fae34007450 con 0x7fae440d2740 2023-12-20T04:28:53.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae3a7fc640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fae28065a90 0x7fae28067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:53.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae3a7fc640 1 -- 172.21.15.32:0/1881269721 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fae34082680 con 0x7fae440d2740 2023-12-20T04:28:53.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.239+0000 7fae49d41640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fae28065a90 0x7fae28067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:53.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.243+0000 7fae49d41640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fae28065a90 0x7fae28067f50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae40009b10 tx=0x7fae40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:53.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.243+0000 7fae3a7fc640 1 -- 172.21.15.32:0/1881269721 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fae340500e0 con 0x7fae440d2740 2023-12-20T04:28:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.435+0000 7fae4bfa5640 1 -- 172.21.15.32:0/1881269721 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fae4404e000 con 0x7fae440d2740 2023-12-20T04:28:53.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.435+0000 7fae3a7fc640 1 -- 172.21.15.32:0/1881269721 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4700 (secure 0 0 0) 0x7fae34053700 con 0x7fae440d2740 2023-12-20T04:28:53.437 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:28:53.437 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2247020919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2247020919},{"type":"v1","addr":"172.21.15.39:6834","nonce":2247020919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/797559820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":797559820},{"type":"v1","addr":"172.21.15.32:6836","nonce":797559820}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3209531230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3209531230},{"type":"v1","addr":"172.21.15.32:6837","nonce":3209531230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:28:38.361494+0000","modified":"2023-12-20T04:28:41.379636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547681098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3547681098},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547681098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-20T04:28:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-12-20T04:28:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 -- 172.21.15.32:0/1881269721 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fae28065a90 msgr2=0x7fae28067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fae28065a90 0x7fae28067f50 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fae40009b10 tx=0x7fae40008040 comp rx=0 tx=0).stop 2023-12-20T04:28:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 -- 172.21.15.32:0/1881269721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae440d2740 msgr2=0x7fae440cb770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae440d2740 0x7fae440cb770 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fae3400c500 tx=0x7fae3400c9c0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 -- 172.21.15.32:0/1881269721 shutdown_connections 2023-12-20T04:28:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fae28065a90 0x7fae28067f50 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fae4415cdb0 0x7fae441548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fae440d2740 0x7fae440cb770 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 --2- 172.21.15.32:0/1881269721 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fae440cfc80 0x7fae440c86e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 -- 172.21.15.32:0/1881269721 >> 172.21.15.32:0/1881269721 conn(0x7fae44065ee0 msgr2=0x7fae440b6d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:53.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 -- 172.21.15.32:0/1881269721 shutdown_connections 2023-12-20T04:28:53.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:53.439+0000 7fae4bfa5640 1 -- 172.21.15.32:0/1881269721 wait complete. 2023-12-20T04:28:53.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 8, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T04:28:38.361494+0000', 'modified': '2023-12-20T04:28:41.379636+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 4300}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_4300': {'gid': 4300, 'name': 'd', 'rank': 0, 'incarnation': 4, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/3547681098', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6833', 'nonce': 3547681098}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 3547681098}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [3], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T04:28:53.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T04:28:53.861 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-20T04:28:54.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb97bc640 1 Processor -- start 2023-12-20T04:28:54.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb97bc640 1 -- start start 2023-12-20T04:28:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb97bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb415cd70 0x7fbeb40cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb97bc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbeb4160ff0 0x7fbeb40b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb97bc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb40b2d70 con 0x7fbeb40b23d0 2023-12-20T04:28:54.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb97bc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb41618b0 con 0x7fbeb4160ff0 2023-12-20T04:28:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb97bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40ce6c0 con 0x7fbeb415cd70 2023-12-20T04:28:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbeb4160ff0 0x7fbeb40b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeabfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbeb40b23d0 0x7fbeb41608e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52786/0 (socket says 172.21.15.32:52786) 2023-12-20T04:28:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeabfff640 1 -- 172.21.15.32:0/2630915279 learned_addr learned my addr 172.21.15.32:0/2630915279 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/2630915279 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb415cd70 0x7fbeb40cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb37fe640 1 -- 172.21.15.32:0/2630915279 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb415cd70 msgr2=0x7fbeb40cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:54.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb37fe640 1 --2- 172.21.15.32:0/2630915279 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb415cd70 0x7fbeb40cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb37fe640 1 -- 172.21.15.32:0/2630915279 >> v1:172.21.15.32:6789/0 conn(0x7fbeb40b23d0 legacy=0x7fbeb41608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb37fe640 1 -- 172.21.15.32:0/2630915279 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb41615d0 con 0x7fbeb4160ff0 2023-12-20T04:28:54.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.011+0000 7fbeb27fc640 1 -- 172.21.15.32:0/2630915279 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4085398275 0 0) 0x7fbeb40b2d70 con 0x7fbeb40b23d0 2023-12-20T04:28:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb37fe640 1 --2- 172.21.15.32:0/2630915279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb4160ff0 0x7fbeb40b2830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009dc0 tx=0x7fbeac01c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05585d95c3a1180 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb27fc640 1 -- 172.21.15.32:0/2630915279 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac0a59c0 con 0x7fbeb4160ff0 2023-12-20T04:28:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb27fc640 1 -- 172.21.15.32:0/2630915279 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeac0a5b20 con 0x7fbeb4160ff0 2023-12-20T04:28:54.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb27fc640 1 -- 172.21.15.32:0/2630915279 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeac0ad440 con 0x7fbeb4160ff0 2023-12-20T04:28:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb97bc640 1 -- 172.21.15.32:0/2630915279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb4160ff0 msgr2=0x7fbeb40b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb97bc640 1 --2- 172.21.15.32:0/2630915279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb4160ff0 0x7fbeb40b2830 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7fbeac009dc0 tx=0x7fbeac01c900 comp rx=0 tx=0).stop 2023-12-20T04:28:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb97bc640 1 -- 172.21.15.32:0/2630915279 shutdown_connections 2023-12-20T04:28:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb97bc640 1 --2- 172.21.15.32:0/2630915279 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb4160ff0 0x7fbeb40b2830 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb97bc640 1 --2- 172.21.15.32:0/2630915279 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb415cd70 0x7fbeb40cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb97bc640 1 -- 172.21.15.32:0/2630915279 >> 172.21.15.32:0/2630915279 conn(0x7fbeb40cd3d0 msgr2=0x7fbeb40cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb97bc640 1 -- 172.21.15.32:0/2630915279 shutdown_connections 2023-12-20T04:28:54.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb97bc640 1 -- 172.21.15.32:0/2630915279 wait complete. 2023-12-20T04:28:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb97bc640 1 Processor -- start 2023-12-20T04:28:54.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.015+0000 7fbeb97bc640 1 -- start start 2023-12-20T04:28:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeb97bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb40b23d0 0x7fbeb40c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeb97bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb415cd70 0x7fbeb40ca160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeb97bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4160ff0 0x7fbeb4157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeb97bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40ab8e0 con 0x7fbeb415cd70 2023-12-20T04:28:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeb97bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40b4d30 con 0x7fbeb4160ff0 2023-12-20T04:28:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeb97bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40c89a0 con 0x7fbeb40b23d0 2023-12-20T04:28:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb415cd70 0x7fbeb40ca160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb415cd70 0x7fbeb40ca160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45442/0 (socket says 172.21.15.32:45442) 2023-12-20T04:28:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeabfff640 1 -- 172.21.15.32:0/987831121 learned_addr learned my addr 172.21.15.32:0/987831121 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb40b23d0 0x7fbeb40c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeb37fe640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4160ff0 0x7fbeb4157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeabfff640 1 -- 172.21.15.32:0/987831121 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb40b23d0 msgr2=0x7fbeb40c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeabfff640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb40b23d0 0x7fbeb40c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeabfff640 1 -- 172.21.15.32:0/987831121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4160ff0 msgr2=0x7fbeb4157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeabfff640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4160ff0 0x7fbeb4157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeabfff640 1 -- 172.21.15.32:0/987831121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac009a70 con 0x7fbeb415cd70 2023-12-20T04:28:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeabfff640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb415cd70 0x7fbeb40ca160 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00c520 tx=0x7fbe9c00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeab7fe640 1 -- 172.21.15.32:0/987831121 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0074e0 con 0x7fbeb415cd70 2023-12-20T04:28:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeab7fe640 1 -- 172.21.15.32:0/987831121 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c007640 con 0x7fbeb415cd70 2023-12-20T04:28:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeb97bc640 1 -- 172.21.15.32:0/987831121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbeb40b4400 con 0x7fbeb415cd70 2023-12-20T04:28:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeab7fe640 1 -- 172.21.15.32:0/987831121 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c0049f0 con 0x7fbeb415cd70 2023-12-20T04:28:54.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeb97bc640 1 -- 172.21.15.32:0/987831121 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb40b49c0 con 0x7fbeb415cd70 2023-12-20T04:28:54.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.019+0000 7fbeab7fe640 1 -- 172.21.15.32:0/987831121 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbe9c004b50 con 0x7fbeb415cd70 2023-12-20T04:28:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeab7fe640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbe94065a90 0x7fbe94067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeab7fe640 1 -- 172.21.15.32:0/987831121 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe9c082750 con 0x7fbeb415cd70 2023-12-20T04:28:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbe94065a90 0x7fbe94067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbebcb1b000 1 Processor -- start 2023-12-20T04:28:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbebcb1b000 1 -- start start 2023-12-20T04:28:54.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbe94065a90 0x7fbe94067f50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009b60 tx=0x7fbea4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144e68450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68990 0x557144e1dc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e1e1c0 0x557144e1e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557144c41c90 con 0x557144e68070 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557144c73220 con 0x557144e1e1c0 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x557144c707f0 con 0x557144e68990 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144e68450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144e68450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45446/0 (socket says 172.21.15.32:45446) 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2056371608 learned_addr learned my addr 172.21.15.32:0/2056371608 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb37fe640 1 --2- 172.21.15.32:0/2056371608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e1e1c0 0x557144e1e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeabfff640 1 --2- 172.21.15.32:0/2056371608 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68990 0x557144e1dc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2056371608 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68990 msgr2=0x557144e1dc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/2056371608 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68990 0x557144e1dc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2056371608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e1e1c0 msgr2=0x557144e1e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:54.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/2056371608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e1e1c0 0x557144e1e5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2056371608 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557144968890 con 0x557144e68070 2023-12-20T04:28:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeabfff640 1 --2- 172.21.15.32:0/2056371608 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68990 0x557144e1dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.023+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/2056371608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144e68450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbea4011ec0 tx=0x7fbea4014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=695f1f5a228bd258 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbea8ff9640 1 -- 172.21.15.32:0/2056371608 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea401c040 con 0x557144e68070 2023-12-20T04:28:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbea8ff9640 1 -- 172.21.15.32:0/2056371608 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea4003830 con 0x557144e68070 2023-12-20T04:28:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbea8ff9640 1 -- 172.21.15.32:0/2056371608 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4003ba0 con 0x557144e68070 2023-12-20T04:28:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2056371608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 msgr2=0x557144e68450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2056371608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144e68450 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fbea4011ec0 tx=0x7fbea4014000 comp rx=0 tx=0).stop 2023-12-20T04:28:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2056371608 shutdown_connections 2023-12-20T04:28:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2056371608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e1e1c0 0x557144e1e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2056371608 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68990 0x557144e1dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2056371608 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144e68450 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2056371608 >> 172.21.15.32:0/2056371608 conn(0x557144e24950 msgr2=0x557144e1c560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:54.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2056371608 shutdown_connections 2023-12-20T04:28:54.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2056371608 wait complete. 2023-12-20T04:28:54.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 -- start start 2023-12-20T04:28:54.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 Processor -- start 2023-12-20T04:28:54.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144e23750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68070 0x557144e2a080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e68990 0x557144e2a5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557144c6c0e0 con 0x557144e1e1c0 2023-12-20T04:28:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557144c0b310 con 0x557144e68990 2023-12-20T04:28:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x557144c29770 con 0x557144e68070 2023-12-20T04:28:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbebcb1b000 10 client.0 _ll_register_callbacks cb 0x557144e09b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:28:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeb2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144e23750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeb2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144e23750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45460/0 (socket says 172.21.15.32:45460) 2023-12-20T04:28:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2322457902 learned_addr learned my addr 172.21.15.32:0/2322457902 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeabfff640 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68070 0x557144e2a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeb37fe640 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e68990 0x557144e2a5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68070 msgr2=0x557144e2a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68070 0x557144e2a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e68990 msgr2=0x557144e2a5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e68990 0x557144e2a5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2322457902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbea4011ba0 con 0x557144e1e1c0 2023-12-20T04:28:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeabfff640 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68070 0x557144e2a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144e23750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbea4024040 tx=0x7fbea4004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442b8b05fcbb8e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.027+0000 7fbe91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:28:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2322457902 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4022880 con 0x557144e1e1c0 2023-12-20T04:28:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2322457902 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbea40229e0 con 0x557144e1e1c0 2023-12-20T04:28:54.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2322457902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557144c3bc10 con 0x557144e1e1c0 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2322457902 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbea4022be0 con 0x557144e1e1c0 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 10 client.4650 fetch_fsmap learned FSMap version 8 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2322457902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x557144c31260 con 0x557144e1e1c0 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2322457902 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557144c3bc10 con 0x557144e1e1c0 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2322457902 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbea4036470 con 0x557144e1e1c0 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 10 client.4650 fetch_fsmap finished waiting for FSMap version 8 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 10 client.4650 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 20 client.4650 populate_metadata read hostname 'smithi032' 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:6833/3547681098,v1:172.21.15.39:6835/3547681098] conn(0x557144e6f8d0 0x557144e6fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 4 client.4650 mds_command: new command op to 4300 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2322457902 --> [v2:172.21.15.39:6833/3547681098,v1:172.21.15.39:6835/3547681098] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x557144c6aa30 con 0x557144e6f8d0 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbeabfff640 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:6833/3547681098,v1:172.21.15.39:6835/3547681098] conn(0x557144e6f8d0 0x557144e6fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbeabfff640 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:6833/3547681098,v1:172.21.15.39:6835/3547681098] conn(0x557144e6f8d0 0x557144e6fcb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7573fd2fce188099 server_cookie=3556d024579b9210 in_seq=0 out_seq=0 2023-12-20T04:28:54.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbe91ffb640 10 client.4650 ms_handle_connect on v2:172.21.15.39:6833/3547681098 2023-12-20T04:28:54.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2322457902 <== mds.0 v2:172.21.15.39:6833/3547681098 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x557144c6aa30 con 0x557144e6f8d0 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbe91ffb640 10 client.4650 handle_command_reply: tid=0 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 client.4650 shutdown 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2322457902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 msgr2=0x557144e23750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144e23750 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fbea4024040 tx=0x7fbea4004740 comp rx=0 tx=0).stop 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2322457902 shutdown_connections 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:6833/3547681098,v1:172.21.15.39:6835/3547681098] conn(0x557144e6f8d0 0x557144e6fcb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e68990 0x557144e2a5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68070 0x557144e2a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2322457902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144e23750 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbe91ffb640 0 client.4650 ms_handle_reset on v2:172.21.15.39:6833/3547681098 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2322457902 >> 172.21.15.32:0/2322457902 conn(0x557144e24950 msgr2=0x557144e1cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2322457902 shutdown_connections 2023-12-20T04:28:54.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2322457902 wait complete. 2023-12-20T04:28:54.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.031+0000 7fbebcb1b000 20 client.4650 trim_cache size 0 max 16384 2023-12-20T04:28:54.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbebcb1b000 1 Processor -- start 2023-12-20T04:28:54.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbebcb1b000 1 -- start start 2023-12-20T04:28:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e1e1c0 0x557144ebfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144ec0410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68990 0x557144ec0950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557144929940 con 0x557144e68070 2023-12-20T04:28:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557144c6d670 con 0x557144e1e1c0 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x557144e20d70 con 0x557144e68990 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144ec0410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbeabfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144ec0410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45468/0 (socket says 172.21.15.32:45468) 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbeabfff640 1 -- 172.21.15.32:0/3847171800 learned_addr learned my addr 172.21.15.32:0/3847171800 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbeabfff640 1 -- 172.21.15.32:0/3847171800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68990 msgr2=0x557144ec0950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.047+0000 7fbeabfff640 1 --2- 172.21.15.32:0/3847171800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68990 0x557144ec0950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeabfff640 1 -- 172.21.15.32:0/3847171800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e1e1c0 msgr2=0x557144ebfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeabfff640 1 --2- 172.21.15.32:0/3847171800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e1e1c0 0x557144ebfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeabfff640 1 -- 172.21.15.32:0/3847171800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x557144c173b0 con 0x557144e68070 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeabfff640 1 --2- 172.21.15.32:0/3847171800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144ec0410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c000fc0 tx=0x7fbe9c082040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61e49764c58ddd35 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbea97fa640 1 -- 172.21.15.32:0/3847171800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c080cf0 con 0x557144e68070 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbea97fa640 1 -- 172.21.15.32:0/3847171800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe9c052210 con 0x557144e68070 2023-12-20T04:28:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbea97fa640 1 -- 172.21.15.32:0/3847171800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe9c06b1d0 con 0x557144e68070 2023-12-20T04:28:54.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 -- 172.21.15.32:0/3847171800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 msgr2=0x557144ec0410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/3847171800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144ec0410 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c000fc0 tx=0x7fbe9c082040 comp rx=0 tx=0).stop 2023-12-20T04:28:54.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 -- 172.21.15.32:0/3847171800 shutdown_connections 2023-12-20T04:28:54.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/3847171800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68990 0x557144ec0950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/3847171800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e68070 0x557144ec0410 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/3847171800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e1e1c0 0x557144ebfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 -- 172.21.15.32:0/3847171800 >> 172.21.15.32:0/3847171800 conn(0x557144e24950 msgr2=0x557144e29f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:54.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 -- 172.21.15.32:0/3847171800 shutdown_connections 2023-12-20T04:28:54.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 -- 172.21.15.32:0/3847171800 wait complete. 2023-12-20T04:28:54.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 -- start start 2023-12-20T04:28:54.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 Processor -- start 2023-12-20T04:28:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144eaafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68070 0x557144ec00e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e68990 0x557144ec0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x557144c2e9e0 con 0x557144e1e1c0 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x557144e04040 con 0x557144e68990 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x557144c3bc10 con 0x557144e68070 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbebcb1b000 10 client.0 _ll_register_callbacks cb 0x557144c1a640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeb2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144eaafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeb2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144eaafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45484/0 (socket says 172.21.15.32:45484) 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2554235692 learned_addr learned my addr 172.21.15.32:0/2554235692 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeabfff640 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68070 0x557144ec00e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeb37fe640 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e68990 0x557144ec0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68070 msgr2=0x557144ec00e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68070 0x557144ec00e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e68990 msgr2=0x557144ec0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e68990 0x557144ec0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeb2ffd640 1 -- 172.21.15.32:0/2554235692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb40b49c0 con 0x557144e1e1c0 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.051+0000 7fbeb37fe640 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e68990 0x557144ec0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb2ffd640 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144eaafa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbea4022de0 tx=0x7fbea40046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=887869af10e5711d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbe91ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2554235692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea4003ee0 con 0x557144e1e1c0 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2554235692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbea4036bc0 con 0x557144e1e1c0 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2554235692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x557144c31e00 con 0x557144e1e1c0 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2554235692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=8) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbea4009030 con 0x557144e1e1c0 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 10 client.4656 fetch_fsmap learned FSMap version 8 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2554235692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=8,monmap=2+}) v3 -- 0x557144968cc0 con 0x557144e1e1c0 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2554235692 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x557144c31e00 con 0x557144e1e1c0 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2554235692 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 8) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbea4014440 con 0x557144e1e1c0 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 10 client.4656 fetch_fsmap finished waiting for FSMap version 8 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 10 client.4656 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 20 client.4656 populate_metadata read hostname 'smithi032' 2023-12-20T04:28:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:6833/3547681098,v1:172.21.15.39:6835/3547681098] conn(0x557144ec0b60 0x557144ec0f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 4 client.4656 mds_command: new command op to 4300 tid=0[{"prefix": "status"}] 2023-12-20T04:28:54.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2554235692 --> [v2:172.21.15.39:6833/3547681098,v1:172.21.15.39:6835/3547681098] -- command(tid 0: {"prefix": "status"}) v1 -- 0x557144c65160 con 0x557144ec0b60 2023-12-20T04:28:54.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeabfff640 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:6833/3547681098,v1:172.21.15.39:6835/3547681098] conn(0x557144ec0b60 0x557144ec0f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeabfff640 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:6833/3547681098,v1:172.21.15.39:6835/3547681098] conn(0x557144ec0b60 0x557144ec0f40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35b897857ce5d77e server_cookie=d8bc809f1c54070 in_seq=0 out_seq=0 2023-12-20T04:28:54.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbe91ffb640 10 client.4656 ms_handle_connect on v2:172.21.15.39:6833/3547681098 2023-12-20T04:28:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbe91ffb640 1 -- 172.21.15.32:0/2554235692 <== mds.0 v2:172.21.15.39:6833/3547681098 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+330 (crc 0 0 0) 0x557144c65160 con 0x557144ec0b60 2023-12-20T04:28:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbe91ffb640 10 client.4656 handle_command_reply: tid=0 2023-12-20T04:28:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 client.4656 shutdown 2023-12-20T04:28:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2554235692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 msgr2=0x557144eaafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144eaafa0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fbea4022de0 tx=0x7fbea40046d0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2554235692 shutdown_connections 2023-12-20T04:28:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:6833/3547681098,v1:172.21.15.39:6835/3547681098] conn(0x557144ec0b60 0x557144ec0f40 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x557144e68990 0x557144ec0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x557144e68070 0x557144ec00e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 --2- 172.21.15.32:0/2554235692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x557144e1e1c0 0x557144eaafa0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbe91ffb640 0 client.4656 ms_handle_reset on v2:172.21.15.39:6833/3547681098 2023-12-20T04:28:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2554235692 >> 172.21.15.32:0/2554235692 conn(0x557144e24950 msgr2=0x557144e65030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2554235692 shutdown_connections 2023-12-20T04:28:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 1 -- 172.21.15.32:0/2554235692 wait complete. 2023-12-20T04:28:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbebcb1b000 20 client.4656 trim_cache size 0 max 16384 2023-12-20T04:28:54.061 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:28:54.061 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T04:28:54.061 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T04:28:54.061 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 4300, 2023-12-20T04:28:54.061 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T04:28:54.062 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T04:28:54.062 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T04:28:54.062 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 15.691524577999999, 2023-12-20T04:28:54.062 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 8, 2023-12-20T04:28:54.062 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 21, 2023-12-20T04:28:54.062 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 21, 2023-12-20T04:28:54.062 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 22.761625811999998 2023-12-20T04:28:54.062 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 -- 172.21.15.32:0/987831121 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbe94065a90 msgr2=0x7fbe94067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbe94065a90 0x7fbe94067f50 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7fbea4009b60 tx=0x7fbea4009190 comp rx=0 tx=0).stop 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 -- 172.21.15.32:0/987831121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb415cd70 msgr2=0x7fbeb40ca160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb415cd70 0x7fbeb40ca160 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c00c520 tx=0x7fbe9c00c9e0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 -- 172.21.15.32:0/987831121 shutdown_connections 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbe94065a90 0x7fbe94067f50 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4160ff0 0x7fbeb4157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb415cd70 0x7fbeb40ca160 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 --2- 172.21.15.32:0/987831121 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb40b23d0 0x7fbeb40c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 -- 172.21.15.32:0/987831121 >> 172.21.15.32:0/987831121 conn(0x7fbeb40cd3d0 msgr2=0x7fbeb4158890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 -- 172.21.15.32:0/987831121 shutdown_connections 2023-12-20T04:28:54.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.055+0000 7fbeb97bc640 1 -- 172.21.15.32:0/987831121 wait complete. 2023-12-20T04:28:54.077 INFO:teuthology.run_tasks:Running task kclient... 2023-12-20T04:28:54.101 INFO:tasks.kclient:Mounting kernel clients... 2023-12-20T04:28:54.101 INFO:tasks.kclient:config is {'client.0': {}, 'client.2': {}, 'client.1': {}, 'client.3': {}} 2023-12-20T04:28:54.102 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:54.102 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T04:28:54.113 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:54.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-12-20T04:28:54.178 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-12-20T04:28:54.180 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:28:54.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:54.181 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T04:28:54.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:54.232 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-12-20T04:28:54.298 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-12-20T04:28:54.300 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:28:54.300 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:54.300 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2023-12-20T04:28:54.311 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:54.312 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2023-12-20T04:28:54.380 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2023-12-20T04:28:54.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:28:54.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:54.382 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2023-12-20T04:28:54.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:54.440 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2023-12-20T04:28:54.511 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2023-12-20T04:28:54.513 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T04:28:54.514 INFO:tasks.kclient:client.0 config is {'syntax': 'v2'} 2023-12-20T04:28:54.514 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-20T04:28:54.514 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T04:28:54.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T04:28:54.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.655+0000 7fa1e8efc640 1 Processor -- start 2023-12-20T04:28:54.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.655+0000 7fa1e8efc640 1 -- start start 2023-12-20T04:28:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.655+0000 7fa1e8efc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa1e406ce60 0x7fa1e415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.655+0000 7fa1e8efc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e415bbd0 con 0x7fa1e40af5e0 2023-12-20T04:28:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.655+0000 7fa1e8efc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1e415beb0 con 0x7fa1e415f110 2023-12-20T04:28:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.655+0000 7fa1e8efc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1e415e910 con 0x7fa1e406ce60 2023-12-20T04:28:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.655+0000 7fa1e259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa1e40af5e0 0x7fa1e40af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52796/0 (socket says 172.21.15.32:52796) 2023-12-20T04:28:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1e259c640 1 -- 172.21.15.32:0/1209773364 learned_addr learned my addr 172.21.15.32:0/1209773364 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1dbfff640 1 --2- 172.21.15.32:0/1209773364 >> v2:172.21.15.39:3300/0 conn(0x7fa1e406ce60 0x7fa1e415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1e1d9b640 1 -- 172.21.15.32:0/1209773364 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 828862808 0 0) 0x7fa1e415bbd0 con 0x7fa1e40af5e0 2023-12-20T04:28:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1e1d9b640 1 -- 172.21.15.32:0/1209773364 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d0003530 con 0x7fa1e40af5e0 2023-12-20T04:28:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1dbfff640 1 -- 172.21.15.32:0/1209773364 >> v1:172.21.15.39:6789/0 conn(0x7fa1e415f110 legacy=0x7fa1e415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1dbfff640 1 -- 172.21.15.32:0/1209773364 >> v1:172.21.15.32:6789/0 conn(0x7fa1e40af5e0 legacy=0x7fa1e40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1e1d9b640 1 -- 172.21.15.32:0/1209773364 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3463773176 0 0) 0x7fa1d0003530 con 0x7fa1e40af5e0 2023-12-20T04:28:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1dbfff640 1 -- 172.21.15.32:0/1209773364 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e415ebf0 con 0x7fa1e406ce60 2023-12-20T04:28:54.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1dbfff640 1 --2- 172.21.15.32:0/1209773364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e406ce60 0x7fa1e415b280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0097c0 tx=0x7fa1cc01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a82c8d70e9446f5c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1e1d9b640 1 -- 172.21.15.32:0/1209773364 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0a8050 con 0x7fa1e406ce60 2023-12-20T04:28:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1e1d9b640 1 -- 172.21.15.32:0/1209773364 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc0ac5f0 con 0x7fa1e406ce60 2023-12-20T04:28:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.659+0000 7fa1e1d9b640 1 -- 172.21.15.32:0/1209773364 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e415b9d0 con 0x7fa1e406ce60 2023-12-20T04:28:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1209773364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e406ce60 msgr2=0x7fa1e415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 --2- 172.21.15.32:0/1209773364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e406ce60 0x7fa1e415b280 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0097c0 tx=0x7fa1cc01c3f0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1209773364 shutdown_connections 2023-12-20T04:28:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 --2- 172.21.15.32:0/1209773364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e406ce60 0x7fa1e415b280 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1209773364 >> 172.21.15.32:0/1209773364 conn(0x7fa1e406a320 msgr2=0x7fa1e40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1209773364 shutdown_connections 2023-12-20T04:28:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1209773364 wait complete. 2023-12-20T04:28:54.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 Processor -- start 2023-12-20T04:28:54.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 -- start start 2023-12-20T04:28:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1e406ce60 0x7fa1e4157fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e40af5e0 0x7fa1e4163dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e415f110 0x7fa1e414f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e4071610 con 0x7fa1e40af5e0 2023-12-20T04:28:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e4070c70 con 0x7fa1e415f110 2023-12-20T04:28:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e8efc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1e4070f50 con 0x7fa1e406ce60 2023-12-20T04:28:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1dbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e40af5e0 0x7fa1e4163dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1dbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e40af5e0 0x7fa1e4163dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45488/0 (socket says 172.21.15.32:45488) 2023-12-20T04:28:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1dbfff640 1 -- 172.21.15.32:0/1525062585 learned_addr learned my addr 172.21.15.32:0/1525062585 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e259c640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1e406ce60 0x7fa1e4157fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.663+0000 7fa1e2d9d640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e415f110 0x7fa1e414f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1dbfff640 1 -- 172.21.15.32:0/1525062585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1e406ce60 msgr2=0x7fa1e4157fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1dbfff640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1e406ce60 0x7fa1e4157fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1dbfff640 1 -- 172.21.15.32:0/1525062585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e415f110 msgr2=0x7fa1e414f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1dbfff640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e415f110 0x7fa1e414f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1dbfff640 1 -- 172.21.15.32:0/1525062585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1cc009420 con 0x7fa1e40af5e0 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1e259c640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1e406ce60 0x7fa1e4157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1dbfff640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e40af5e0 0x7fa1e4163dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0097c0 tx=0x7fa1cc01ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1daffd640 1 -- 172.21.15.32:0/1525062585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc006e20 con 0x7fa1e40af5e0 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1daffd640 1 -- 172.21.15.32:0/1525062585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1cc0b8660 con 0x7fa1e40af5e0 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1525062585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1e4071230 con 0x7fa1e40af5e0 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1daffd640 1 -- 172.21.15.32:0/1525062585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1cc0b8a10 con 0x7fa1e40af5e0 2023-12-20T04:28:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1525062585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1e41644d0 con 0x7fa1e40af5e0 2023-12-20T04:28:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1525062585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1e4071750 con 0x7fa1e40af5e0 2023-12-20T04:28:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1daffd640 1 -- 172.21.15.32:0/1525062585 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa1cc0b8b70 con 0x7fa1e40af5e0 2023-12-20T04:28:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1daffd640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1c4065a60 0x7fa1c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1daffd640 1 -- 172.21.15.32:0/1525062585 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1cc11da10 con 0x7fa1e40af5e0 2023-12-20T04:28:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.667+0000 7fa1e259c640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1c4065a60 0x7fa1c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.671+0000 7fa1e259c640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1c4065a60 0x7fa1c4067f20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4001010 tx=0x7fa1d4009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:54.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.671+0000 7fa1daffd640 1 -- 172.21.15.32:0/1525062585 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa1e4071750 con 0x7fa1e40af5e0 2023-12-20T04:28:54.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1525062585 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa1e4071750 con 0x7fa1e40af5e0 2023-12-20T04:28:54.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1daffd640 1 -- 172.21.15.32:0/1525062585 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa1cc0bc060 con 0x7fa1e40af5e0 2023-12-20T04:28:54.869 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T04:28:54.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1525062585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1c4065a60 msgr2=0x7fa1c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1c4065a60 0x7fa1c4067f20 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7fa1d4001010 tx=0x7fa1d4009040 comp rx=0 tx=0).stop 2023-12-20T04:28:54.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1525062585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e40af5e0 msgr2=0x7fa1e4163dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:54.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e40af5e0 0x7fa1e4163dd0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fa1cc0097c0 tx=0x7fa1cc01ac30 comp rx=0 tx=0).stop 2023-12-20T04:28:54.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1525062585 shutdown_connections 2023-12-20T04:28:54.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1c4065a60 0x7fa1c4067f20 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1e415f110 0x7fa1e414f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1e40af5e0 0x7fa1e4163dd0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 --2- 172.21.15.32:0/1525062585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1e406ce60 0x7fa1e4157fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:54.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1525062585 >> 172.21.15.32:0/1525062585 conn(0x7fa1e406a320 msgr2=0x7fa1e415c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:54.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1525062585 shutdown_connections 2023-12-20T04:28:54.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:54.867+0000 7fa1e8efc640 1 -- 172.21.15.32:0/1525062585 wait complete. 2023-12-20T04:28:54.890 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-20T04:28:54.890 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T04:28:54.890 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T04:28:54.891 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T04:28:54.891 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T04:28:54.891 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-20T04:28:54.891 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-20T04:28:54.891 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T04:28:54.891 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:28:54.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:54.891 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T04:28:54.901 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:28:54.901 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:28:54.901 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:28:54.901 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:54.901 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T04:28:54.901 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42325sec preferred_lft 42325sec 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:28:54.902 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:54.903 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T04:28:54.903 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T04:28:54.903 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:54.903 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T04:28:54.903 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:54.903 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-20T04:28:54.903 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:54.903 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:28:54.903 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:28:54.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add name ceph-brx type bridge 2023-12-20T04:28:54.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr flush dev ceph-brx 2023-12-20T04:28:54.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set ceph-brx up 2023-12-20T04:28:54.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-20T04:28:54.904 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:28:55.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:55.018 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T04:28:55.037 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T04:28:55.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:55.040 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T04:28:55.106 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T04:28:55.106 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T04:28:55.106 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T04:28:55.106 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T04:28:55.106 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T04:28:55.106 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T04:28:55.107 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:55.107 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:28:55.107 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:28:55.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T04:28:55.107 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T04:28:55.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T04:28:55.108 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:28:55.199 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:55.199 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T04:28:55.210 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:55.210 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-12-20T04:28:55.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:55.264 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:28:55.264 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:28:55.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T04:28:55.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2023-12-20T04:28:55.264 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:28:55.348 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2023-12-20T04:28:55.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:55.348 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:28:55.348 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:28:55.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2023-12-20T04:28:55.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-20T04:28:55.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2023-12-20T04:28:55.349 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2023-12-20T04:28:55.349 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2023-12-20T04:28:55.349 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:28:55.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:55.528 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:28:55.528 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:28:55.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T04:28:55.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.0 master ceph-brx 2023-12-20T04:28:55.529 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:28:55.570 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:28:55.570 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T04:28:55.620 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:28:55.620 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:28:55.621 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:28:55.671 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T04:28:55.671 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:55.671 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:28:55.775 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:28:55.775 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:28:55.876 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:28:55.876 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:28:55.877 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T04:28:55.877 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T04:28:55.877 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T04:28:55.880 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T04:28:55.881 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:55.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:28:55.904 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:55.904 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:28:55.980 INFO:teuthology.orchestra.run.smithi032.stdout:client4645 2023-12-20T04:28:55.980 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:55.980 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:28:56.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:56.032 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:28:56.095 INFO:teuthology.orchestra.run.smithi032.stdout:client4645 2023-12-20T04:28:56.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:56.096 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:28:56.157 INFO:teuthology.orchestra.run.smithi032.stdout:client4645 2023-12-20T04:28:56.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:56.157 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9b496156-2b6f-4897-9af4-fbc686ad55e7.client4645/status') 2023-12-20T04:28:56.224 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.4645 (3)172.21.15.32:0/1426299651 2023-12-20T04:28:56.225 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2023-12-20T04:28:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2023-12-20T04:28:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2023-12-20T04:28:56.225 INFO:teuthology.orchestra.run.smithi032.stderr:70 bytes copied, 5.3226e-05 s, 1.3 MB/s 2023-12-20T04:28:56.227 INFO:tasks.kclient:client.2 config is {'syntax': 'v2'} 2023-12-20T04:28:56.227 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-20T04:28:56.227 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-20T04:28:56.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T04:28:56.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.411+0000 7fd7d7339640 1 Processor -- start 2023-12-20T04:28:56.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.411+0000 7fd7d7339640 1 -- start start 2023-12-20T04:28:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.411+0000 7fd7d7339640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7d015ce30 0x7fd7d0161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.411+0000 7fd7d7339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d0070950 0x7fd7d006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.411+0000 7fd7d7339640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d0071920 con 0x7fd7d006dce0 2023-12-20T04:28:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.411+0000 7fd7d7339640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d0071080 con 0x7fd7d015ce30 2023-12-20T04:28:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.411+0000 7fd7d7339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d0071360 con 0x7fd7d0070950 2023-12-20T04:28:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.411+0000 7fd7d50d5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7d015ce30 0x7fd7d0161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.411+0000 7fd7d48d4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d0070950 0x7fd7d006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.411+0000 7fd7d50d5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7d015ce30 0x7fd7d0161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:57504/0 (socket says 172.21.15.32:57504) 2023-12-20T04:28:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.411+0000 7fd7d50d5640 1 -- 172.21.15.32:0/3373056255 learned_addr learned my addr 172.21.15.32:0/3373056255 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:56.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.415+0000 7fd7d50d5640 1 -- 172.21.15.32:0/3373056255 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d0070950 msgr2=0x7fd7d006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.415+0000 7fd7d50d5640 1 --2- 172.21.15.32:0/3373056255 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d0070950 0x7fd7d006d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.415+0000 7fd7d50d5640 1 -- 172.21.15.32:0/3373056255 >> v1:172.21.15.39:6789/0 conn(0x7fd7d006dce0 legacy=0x7fd7d0069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.415+0000 7fd7d50d5640 1 -- 172.21.15.32:0/3373056255 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d0071640 con 0x7fd7d015ce30 2023-12-20T04:28:56.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.415+0000 7fd7c7fff640 1 -- 172.21.15.32:0/3373056255 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 764248603 0 0) 0x7fd7d0071920 con 0x7fd7d006dce0 2023-12-20T04:28:56.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.415+0000 7fd7d50d5640 1 --2- 172.21.15.32:0/3373056255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d015ce30 0x7fd7d0161560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009870 tx=0x7fd7cc01c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=719ac36088b8982b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:56.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.415+0000 7fd7c7fff640 1 -- 172.21.15.32:0/3373056255 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0ac620 con 0x7fd7d015ce30 2023-12-20T04:28:56.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.415+0000 7fd7c7fff640 1 -- 172.21.15.32:0/3373056255 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc0ac780 con 0x7fd7d015ce30 2023-12-20T04:28:56.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.415+0000 7fd7c7fff640 1 -- 172.21.15.32:0/3373056255 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0acb10 con 0x7fd7d015ce30 2023-12-20T04:28:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 -- 172.21.15.32:0/3373056255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d015ce30 msgr2=0x7fd7d0161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 --2- 172.21.15.32:0/3373056255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d015ce30 0x7fd7d0161560 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc009870 tx=0x7fd7cc01c540 comp rx=0 tx=0).stop 2023-12-20T04:28:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 -- 172.21.15.32:0/3373056255 shutdown_connections 2023-12-20T04:28:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 --2- 172.21.15.32:0/3373056255 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d0070950 0x7fd7d006d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 --2- 172.21.15.32:0/3373056255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d015ce30 0x7fd7d0161560 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:56.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 -- 172.21.15.32:0/3373056255 >> 172.21.15.32:0/3373056255 conn(0x7fd7d00cba50 msgr2=0x7fd7d00cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:56.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 -- 172.21.15.32:0/3373056255 shutdown_connections 2023-12-20T04:28:56.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 -- 172.21.15.32:0/3373056255 wait complete. 2023-12-20T04:28:56.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 Processor -- start 2023-12-20T04:28:56.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 -- start start 2023-12-20T04:28:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d006dce0 0x7fd7d015aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d0070950 0x7fd7d0159460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d015ce30 0x7fd7d01599a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00c2050 con 0x7fd7d006dce0 2023-12-20T04:28:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00c1e00 con 0x7fd7d015ce30 2023-12-20T04:28:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d7339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00c1560 con 0x7fd7d0070950 2023-12-20T04:28:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d50d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d006dce0 0x7fd7d015aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d50d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d006dce0 0x7fd7d015aea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45504/0 (socket says 172.21.15.32:45504) 2023-12-20T04:28:56.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.419+0000 7fd7d50d5640 1 -- 172.21.15.32:0/2832156988 learned_addr learned my addr 172.21.15.32:0/2832156988 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7d58d6640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d015ce30 0x7fd7d01599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7d48d4640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d0070950 0x7fd7d0159460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7d50d5640 1 -- 172.21.15.32:0/2832156988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d0070950 msgr2=0x7fd7d0159460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7d50d5640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d0070950 0x7fd7d0159460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7d50d5640 1 -- 172.21.15.32:0/2832156988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d015ce30 msgr2=0x7fd7d01599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7d50d5640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d015ce30 0x7fd7d01599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7d50d5640 1 -- 172.21.15.32:0/2832156988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7cc0094d0 con 0x7fd7d006dce0 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7d50d5640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d006dce0 0x7fd7d015aea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0af040 tx=0x7fd7cc01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7c5ffb640 1 -- 172.21.15.32:0/2832156988 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0b7420 con 0x7fd7d006dce0 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7c5ffb640 1 -- 172.21.15.32:0/2832156988 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7cc0b7580 con 0x7fd7d006dce0 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7c5ffb640 1 -- 172.21.15.32:0/2832156988 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7cc0b7930 con 0x7fd7d006dce0 2023-12-20T04:28:56.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7d7339640 1 -- 172.21.15.32:0/2832156988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7d00c2250 con 0x7fd7d006dce0 2023-12-20T04:28:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7d7339640 1 -- 172.21.15.32:0/2832156988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d00c1a90 con 0x7fd7d006dce0 2023-12-20T04:28:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7d7339640 1 -- 172.21.15.32:0/2832156988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d00c5eb0 con 0x7fd7d006dce0 2023-12-20T04:28:56.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7c5ffb640 1 -- 172.21.15.32:0/2832156988 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd7cc0b3040 con 0x7fd7d006dce0 2023-12-20T04:28:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7c5ffb640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7b4065a60 0x7fd7b4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.423+0000 7fd7c5ffb640 1 -- 172.21.15.32:0/2832156988 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd7cc11d590 con 0x7fd7d006dce0 2023-12-20T04:28:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.427+0000 7fd7d48d4640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7b4065a60 0x7fd7b4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:56.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.427+0000 7fd7d48d4640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7b4065a60 0x7fd7b4067f20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0157180 tx=0x7fd7c0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:56.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.427+0000 7fd7c5ffb640 1 -- 172.21.15.32:0/2832156988 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd7cc0eaf40 con 0x7fd7d006dce0 2023-12-20T04:28:56.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.619+0000 7fd7d7339640 1 -- 172.21.15.32:0/2832156988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fd7d004e000 con 0x7fd7d006dce0 2023-12-20T04:28:56.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.619+0000 7fd7c5ffb640 1 -- 172.21.15.32:0/2832156988 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7fd7cc0ee560 con 0x7fd7d006dce0 2023-12-20T04:28:56.623 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 -- 172.21.15.32:0/2832156988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7b4065a60 msgr2=0x7fd7b4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7b4065a60 0x7fd7b4067f20 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fd7d0157180 tx=0x7fd7c0008040 comp rx=0 tx=0).stop 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 -- 172.21.15.32:0/2832156988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d006dce0 msgr2=0x7fd7d015aea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d006dce0 0x7fd7d015aea0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7fd7cc0af040 tx=0x7fd7cc01ad80 comp rx=0 tx=0).stop 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 -- 172.21.15.32:0/2832156988 shutdown_connections 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7b4065a60 0x7fd7b4067f20 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d015ce30 0x7fd7d01599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d0070950 0x7fd7d0159460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 --2- 172.21.15.32:0/2832156988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d006dce0 0x7fd7d015aea0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 -- 172.21.15.32:0/2832156988 >> 172.21.15.32:0/2832156988 conn(0x7fd7d00cba50 msgr2=0x7fd7d015d570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 -- 172.21.15.32:0/2832156988 shutdown_connections 2023-12-20T04:28:56.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:56.623+0000 7fd7d7339640 1 -- 172.21.15.32:0/2832156988 wait complete. 2023-12-20T04:28:56.644 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-20T04:28:56.644 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T04:28:56.644 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T04:28:56.644 INFO:tasks.cephfs.mount:self.client.name = client.2 2023-12-20T04:28:56.644 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.2 2023-12-20T04:28:56.644 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-20T04:28:56.645 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-20T04:28:56.645 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T04:28:56.645 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.2' 2023-12-20T04:28:56.645 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:56.645 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T04:28:56.659 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:28:56.659 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:28:56.659 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:28:56.659 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:56.659 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T04:28:56.659 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:56.659 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:28:56.659 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42323sec preferred_lft 42323sec 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:56.660 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T04:28:56.661 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T04:28:56.661 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:56.661 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T04:28:56.661 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:56.661 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:28:56.661 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:56.661 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T04:28:56.661 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:56.661 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-12-20T04:28:56.661 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:56.662 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:28:56.662 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T04:28:56.662 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link tentative 2023-12-20T04:28:56.662 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:56.662 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:56.662 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T04:28:56.726 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T04:28:56.728 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:56.729 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T04:28:56.785 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T04:28:56.785 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T04:28:56.785 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T04:28:56.785 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T04:28:56.786 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T04:28:56.786 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T04:28:56.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:56.786 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:28:56.786 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:28:56.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T04:28:56.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T04:28:56.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T04:28:56.786 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:28:56.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:56.883 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T04:28:56.893 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T04:28:56.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:56.894 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list-id 2023-12-20T04:28:56.943 INFO:teuthology.orchestra.run.smithi032.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.0) 2023-12-20T04:28:56.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:56.944 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:28:56.944 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:28:56.945 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T04:28:56.945 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.2 1 2023-12-20T04:28:56.945 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:28:57.031 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:57.031 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-20T04:28:57.101 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:28:57.101 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:28:57.101 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:28:57.101 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:57.101 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T04:28:57.102 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:57.102 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:28:57.102 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 92:36:1e:dc:ab:dc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-20T04:28:57.102 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-20T04:28:57.102 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:57.102 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::9036:1eff:fedc:abdc/64 scope link 2023-12-20T04:28:57.102 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:57.103 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:57.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr 2023-12-20T04:28:57.174 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:28:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:28:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:28:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T04:28:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:57.175 INFO:teuthology.orchestra.run.smithi032.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:28:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 92:36:1e:dc:ab:dc brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-20T04:28:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-20T04:28:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::9036:1eff:fedc:abdc/64 scope link 2023-12-20T04:28:57.175 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:57.176 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.2' with 192.168.0.2/16 2023-12-20T04:28:57.177 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:57.177 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:28:57.177 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:28:57.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.2 type veth peer name brx.1 2023-12-20T04:28:57.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-20T04:28:57.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set veth0 up 2023-12-20T04:28:57.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip link set lo up 2023-12-20T04:28:57.177 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.2 ip route add default via 192.168.255.254 2023-12-20T04:28:57.177 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:28:57.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:57.323 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:28:57.323 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:28:57.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.1 up 2023-12-20T04:28:57.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set dev brx.1 master ceph-brx 2023-12-20T04:28:57.323 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:28:57.363 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:28:57.364 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.2 2023-12-20T04:28:57.419 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.2' 2023-12-20T04:28:57.419 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:28:57.419 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.2 2023-12-20T04:28:57.475 INFO:tasks.cephfs.kernel_mount:mounting using device: 2@.cephfs=/ 2023-12-20T04:28:57.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:57.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 2@.cephfs=/ /home/ubuntu/cephtest/mnt.2 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:28:57.571 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:28:57.571 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:28:57.584 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:28:57.584 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:28:57.584 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T04:28:57.584 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 2@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T04:28:57.584 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=2,ms_mode=prefer-crc,key=2,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T04:28:57.587 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T04:28:57.588 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:57.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.2 2023-12-20T04:28:57.609 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:57.609 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:28:57.664 INFO:teuthology.orchestra.run.smithi032.stdout:client4657 2023-12-20T04:28:57.664 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:28:57.664 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:28:57.719 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:57.720 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:28:57.775 INFO:teuthology.orchestra.run.smithi032.stdout:client4657 2023-12-20T04:28:57.775 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:57.775 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:28:57.831 INFO:teuthology.orchestra.run.smithi032.stdout:client4657 2023-12-20T04:28:57.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:57.831 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.2 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9b496156-2b6f-4897-9af4-fbc686ad55e7.client4657/status') 2023-12-20T04:28:57.900 INFO:teuthology.orchestra.run.smithi032.stdout:instance: client.4657 (3)172.21.15.32:0/1172713097 2023-12-20T04:28:57.900 INFO:teuthology.orchestra.run.smithi032.stdout:blocklisted: false 2023-12-20T04:28:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records in 2023-12-20T04:28:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:0+1 records out 2023-12-20T04:28:57.901 INFO:teuthology.orchestra.run.smithi032.stderr:70 bytes copied, 5.4578e-05 s, 1.3 MB/s 2023-12-20T04:28:57.902 INFO:tasks.kclient:client.1 config is {'syntax': 'v2'} 2023-12-20T04:28:57.902 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-20T04:28:57.903 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-20T04:28:57.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T04:28:58.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.087+0000 7ff309695640 1 Processor -- start 2023-12-20T04:28:58.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.087+0000 7ff309695640 1 -- start start 2023-12-20T04:28:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.087+0000 7ff309695640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3041600a0 0x7ff304161170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.087+0000 7ff309695640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff3040b2770 0x7ff3040703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.087+0000 7ff309695640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3040caf50 con 0x7ff3040b2310 2023-12-20T04:28:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.087+0000 7ff309695640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3041616b0 con 0x7ff3040b2770 2023-12-20T04:28:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.087+0000 7ff309695640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040ce840 con 0x7ff3041600a0 2023-12-20T04:28:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff3027fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff3040b2310 0x7ff30406fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52810/0 (socket says 172.21.15.32:52810) 2023-12-20T04:28:58.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff3027fc640 1 -- 172.21.15.32:0/1714033103 learned_addr learned my addr 172.21.15.32:0/1714033103 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff3037fe640 1 --2- 172.21.15.32:0/1714033103 >> v2:172.21.15.32:3300/0 conn(0x7ff3040b2770 0x7ff3040703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:58.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff302ffd640 1 --2- 172.21.15.32:0/1714033103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3041600a0 0x7ff304161170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:58.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff3037fe640 1 -- 172.21.15.32:0/1714033103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3041600a0 msgr2=0x7ff304161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:58.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff3037fe640 1 --2- 172.21.15.32:0/1714033103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3041600a0 0x7ff304161170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:58.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff3037fe640 1 -- 172.21.15.32:0/1714033103 >> v1:172.21.15.32:6789/0 conn(0x7ff3040b2310 legacy=0x7ff30406fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:58.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff301ffb640 1 -- 172.21.15.32:0/1714033103 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 650752354 0 0) 0x7ff3040caf50 con 0x7ff3040b2310 2023-12-20T04:28:58.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff3037fe640 1 -- 172.21.15.32:0/1714033103 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3040ceb20 con 0x7ff3040b2770 2023-12-20T04:28:58.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff3037fe640 1 --2- 172.21.15.32:0/1714033103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3040b2770 0x7ff3040703e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009d90 tx=0x7ff2fc01f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb83f4a73a593c74 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:58.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff301ffb640 1 -- 172.21.15.32:0/1714033103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0af600 con 0x7ff3040b2770 2023-12-20T04:28:58.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff301ffb640 1 -- 172.21.15.32:0/1714033103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc0ab050 con 0x7ff3040b2770 2023-12-20T04:28:58.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.091+0000 7ff301ffb640 1 -- 172.21.15.32:0/1714033103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0af980 con 0x7ff3040b2770 2023-12-20T04:28:58.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 -- 172.21.15.32:0/1714033103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3040b2770 msgr2=0x7ff3040703e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:58.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 --2- 172.21.15.32:0/1714033103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3040b2770 0x7ff3040703e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc009d90 tx=0x7ff2fc01f130 comp rx=0 tx=0).stop 2023-12-20T04:28:58.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 -- 172.21.15.32:0/1714033103 shutdown_connections 2023-12-20T04:28:58.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 --2- 172.21.15.32:0/1714033103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3040b2770 0x7ff3040703e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:58.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 --2- 172.21.15.32:0/1714033103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3041600a0 0x7ff304161170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:58.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 -- 172.21.15.32:0/1714033103 >> 172.21.15.32:0/1714033103 conn(0x7ff3040cd310 msgr2=0x7ff3040cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:58.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 -- 172.21.15.32:0/1714033103 shutdown_connections 2023-12-20T04:28:58.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 -- 172.21.15.32:0/1714033103 wait complete. 2023-12-20T04:28:58.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 Processor -- start 2023-12-20T04:28:58.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 -- start start 2023-12-20T04:28:58.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3040b2310 0x7ff3040c5340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:58.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040b2770 0x7ff3040c7090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:58.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3041600a0 0x7ff3041f1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:58.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff30415d620 con 0x7ff3041600a0 2023-12-20T04:28:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040c5a00 con 0x7ff3040b2770 2023-12-20T04:28:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff309695640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3040c5ce0 con 0x7ff3040b2310 2023-12-20T04:28:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff3037fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3041600a0 0x7ff3041f1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff3037fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3041600a0 0x7ff3041f1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45520/0 (socket says 172.21.15.32:45520) 2023-12-20T04:28:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff3037fe640 1 -- 172.21.15.32:0/1671712110 learned_addr learned my addr 172.21.15.32:0/1671712110 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:58.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff302ffd640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3040b2310 0x7ff3040c5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.095+0000 7ff3027fc640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040b2770 0x7ff3040c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff3037fe640 1 -- 172.21.15.32:0/1671712110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3040b2310 msgr2=0x7ff3040c5340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff3037fe640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3040b2310 0x7ff3040c5340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff3037fe640 1 -- 172.21.15.32:0/1671712110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040b2770 msgr2=0x7ff3040c7090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff3037fe640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040b2770 0x7ff3040c7090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:58.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff3037fe640 1 -- 172.21.15.32:0/1671712110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2fc009a70 con 0x7ff3041600a0 2023-12-20T04:28:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff3037fe640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3041600a0 0x7ff3041f1bd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0b2040 tx=0x7ff2fc01bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:58.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff2eb7fe640 1 -- 172.21.15.32:0/1671712110 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc009450 con 0x7ff3041600a0 2023-12-20T04:28:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff2eb7fe640 1 -- 172.21.15.32:0/1671712110 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2fc0b9410 con 0x7ff3041600a0 2023-12-20T04:28:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff2eb7fe640 1 -- 172.21.15.32:0/1671712110 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2fc0b97c0 con 0x7ff3041600a0 2023-12-20T04:28:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff309695640 1 -- 172.21.15.32:0/1671712110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3040c5fc0 con 0x7ff3041600a0 2023-12-20T04:28:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff309695640 1 -- 172.21.15.32:0/1671712110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3041f2110 con 0x7ff3041600a0 2023-12-20T04:28:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff309695640 1 -- 172.21.15.32:0/1671712110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff30415d3f0 con 0x7ff3041600a0 2023-12-20T04:28:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff2eb7fe640 1 -- 172.21.15.32:0/1671712110 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff2fc0b6050 con 0x7ff3041600a0 2023-12-20T04:28:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff2eb7fe640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2f0065a90 0x7ff2f0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff2eb7fe640 1 -- 172.21.15.32:0/1671712110 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff2fc121570 con 0x7ff3041600a0 2023-12-20T04:28:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.099+0000 7ff302ffd640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2f0065a90 0x7ff2f0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.103+0000 7ff302ffd640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2f0065a90 0x7ff2f0067f50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff3040ca6f0 tx=0x7ff2f4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:58.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.103+0000 7ff2eb7fe640 1 -- 172.21.15.32:0/1671712110 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff30415d3f0 con 0x7ff3041600a0 2023-12-20T04:28:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 -- 172.21.15.32:0/1671712110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7ff30415d3f0 con 0x7ff3041600a0 2023-12-20T04:28:58.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff2eb7fe640 1 -- 172.21.15.32:0/1671712110 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7ff2fc0b4060 con 0x7ff3041600a0 2023-12-20T04:28:58.301 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T04:28:58.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 -- 172.21.15.32:0/1671712110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2f0065a90 msgr2=0x7ff2f0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2f0065a90 0x7ff2f0067f50 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7ff3040ca6f0 tx=0x7ff2f4009190 comp rx=0 tx=0).stop 2023-12-20T04:28:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 -- 172.21.15.32:0/1671712110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3041600a0 msgr2=0x7ff3041f1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3041600a0 0x7ff3041f1bd0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7ff2fc0b2040 tx=0x7ff2fc01bed0 comp rx=0 tx=0).stop 2023-12-20T04:28:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 -- 172.21.15.32:0/1671712110 shutdown_connections 2023-12-20T04:28:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2f0065a90 0x7ff2f0067f50 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3041600a0 0x7ff3041f1bd0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3040b2770 0x7ff3040c7090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 --2- 172.21.15.32:0/1671712110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3040b2310 0x7ff3040c5340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 -- 172.21.15.32:0/1671712110 >> 172.21.15.32:0/1671712110 conn(0x7ff3040cd310 msgr2=0x7ff3040b6070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 -- 172.21.15.32:0/1671712110 shutdown_connections 2023-12-20T04:28:58.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:58.299+0000 7ff309695640 1 -- 172.21.15.32:0/1671712110 wait complete. 2023-12-20T04:28:58.322 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-20T04:28:58.322 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T04:28:58.322 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2023-12-20T04:28:58.322 INFO:tasks.cephfs.mount:self.client.name = client.1 2023-12-20T04:28:58.322 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.1 2023-12-20T04:28:58.322 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-20T04:28:58.322 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-20T04:28:58.322 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T04:28:58.322 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.1' 2023-12-20T04:28:58.322 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:58.323 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2023-12-20T04:28:58.333 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:28:58.333 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:28:58.333 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:28:58.333 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:58.333 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42367sec preferred_lft 42367sec 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2023-12-20T04:28:58.334 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:58.335 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:28:58.335 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:58.335 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2023-12-20T04:28:58.335 INFO:teuthology.orchestra.run.smithi039.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T04:28:58.335 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 02:42:7e:18:c0:fb brd ff:ff:ff:ff:ff:ff 2023-12-20T04:28:58.335 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T04:28:58.335 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:28:58.336 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2023-12-20T04:28:58.336 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:58.336 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-12-20T04:28:58.336 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-12-20T04:28:58.336 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add name ceph-brx type bridge 2023-12-20T04:28:58.336 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr flush dev ceph-brx 2023-12-20T04:28:58.336 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set ceph-brx up 2023-12-20T04:28:58.336 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2023-12-20T04:28:58.336 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-12-20T04:28:58.455 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:58.455 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T04:28:58.476 INFO:teuthology.orchestra.run.smithi039.stdout:1 2023-12-20T04:28:58.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:58.478 DEBUG:teuthology.orchestra.run.smithi039:> route 2023-12-20T04:28:58.548 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2023-12-20T04:28:58.548 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T04:28:58.548 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T04:28:58.548 INFO:teuthology.orchestra.run.smithi039.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T04:28:58.549 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T04:28:58.549 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T04:28:58.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:58.549 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-12-20T04:28:58.549 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-12-20T04:28:58.549 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T04:28:58.549 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T04:28:58.549 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T04:28:58.549 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-12-20T04:28:58.644 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:58.644 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2023-12-20T04:28:58.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:58.654 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2023-12-20T04:28:58.710 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:58.711 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-12-20T04:28:58.711 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-12-20T04:28:58.711 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-20T04:28:58.711 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.1 0 2023-12-20T04:28:58.711 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-12-20T04:28:58.796 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.1' with 192.168.0.1/16 2023-12-20T04:28:58.796 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:58.796 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-12-20T04:28:58.796 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-12-20T04:28:58.796 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.1 type veth peer name brx.0 2023-12-20T04:28:58.796 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2023-12-20T04:28:58.796 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set veth0 up 2023-12-20T04:28:58.797 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip link set lo up 2023-12-20T04:28:58.797 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip route add default via 192.168.255.254 2023-12-20T04:28:58.797 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-12-20T04:28:58.959 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:58.960 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-12-20T04:28:58.960 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-12-20T04:28:58.960 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.0 up 2023-12-20T04:28:58.960 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.0 master ceph-brx 2023-12-20T04:28:58.961 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-12-20T04:28:59.003 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:28:59.004 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.1 2023-12-20T04:28:59.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:28:59.038 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:28:59.058 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.1' 2023-12-20T04:28:59.058 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:28:59.059 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.1 2023-12-20T04:28:59.149 INFO:tasks.cephfs.kernel_mount:mounting using device: 1@.cephfs=/ 2023-12-20T04:28:59.149 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:59.149 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 1@.cephfs=/ /home/ubuntu/cephtest/mnt.1 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:28:59.249 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:28:59.249 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:28:59.354 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:28:59.354 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:28:59.354 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T04:28:59.354 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 1@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T04:28:59.354 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=1,ms_mode=prefer-crc,key=1,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T04:28:59.358 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T04:28:59.358 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:59.358 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.1 2023-12-20T04:28:59.381 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:59.381 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:28:59.483 INFO:teuthology.orchestra.run.smithi039.stdout:client4663 2023-12-20T04:28:59.483 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:28:59.483 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:28:59.493 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:59.494 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:28:59.548 INFO:teuthology.orchestra.run.smithi039.stdout:client4663 2023-12-20T04:28:59.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:59.549 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:28:59.600 INFO:teuthology.orchestra.run.smithi039.stdout:client4663 2023-12-20T04:28:59.601 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:28:59.601 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.1 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9b496156-2b6f-4897-9af4-fbc686ad55e7.client4663/status') 2023-12-20T04:28:59.662 INFO:teuthology.orchestra.run.smithi039.stdout:instance: client.4663 (3)192.168.0.1:0/2618674163 2023-12-20T04:28:59.662 INFO:teuthology.orchestra.run.smithi039.stdout:blocklisted: false 2023-12-20T04:28:59.663 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records in 2023-12-20T04:28:59.663 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records out 2023-12-20T04:28:59.663 INFO:teuthology.orchestra.run.smithi039.stderr:69 bytes copied, 5.3476e-05 s, 1.3 MB/s 2023-12-20T04:28:59.664 INFO:tasks.kclient:client.3 config is {'syntax': 'v2'} 2023-12-20T04:28:59.664 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2023-12-20T04:28:59.664 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-20T04:28:59.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T04:28:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.815+0000 7f2da3bbc640 1 Processor -- start 2023-12-20T04:28:59.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da3bbc640 1 -- start start 2023-12-20T04:28:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da3bbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d9c15cdb0 0x7f2d9c0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da3bbc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2d9c0d2740 0x7f2d9c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da3bbc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d9c0b6ca0 con 0x7f2d9c0d1120 2023-12-20T04:28:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da3bbc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d9c15d190 con 0x7f2d9c0d2740 2023-12-20T04:28:59.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da3bbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c0d16a0 con 0x7f2d9c15cdb0 2023-12-20T04:28:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da1157640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2d9c0d2740 0x7f2d9c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da1958640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d9c15cdb0 0x7f2d9c0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da2159640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2d9c0d1120 0x7f2d9c0b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:48196/0 (socket says 172.21.15.32:48196) 2023-12-20T04:28:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da2159640 1 -- 172.21.15.32:0/3092154151 learned_addr learned my addr 172.21.15.32:0/3092154151 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:28:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da1157640 1 -- 172.21.15.32:0/3092154151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d9c15cdb0 msgr2=0x7f2d9c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:28:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da1157640 1 --2- 172.21.15.32:0/3092154151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d9c15cdb0 0x7f2d9c0d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da1157640 1 -- 172.21.15.32:0/3092154151 >> v1:172.21.15.39:6789/0 conn(0x7f2d9c0d1120 legacy=0x7f2d9c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:59.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.819+0000 7f2da1157640 1 -- 172.21.15.32:0/3092154151 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d9c0d1980 con 0x7f2d9c0d2740 2023-12-20T04:28:59.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da1157640 1 --2- 172.21.15.32:0/3092154151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c0d2740 0x7f2d9c0cf570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0097c0 tx=0x7f2d8c01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ca9e39538915acf6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:59.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da0956640 1 -- 172.21.15.32:0/3092154151 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c0a8970 con 0x7f2d9c0d2740 2023-12-20T04:28:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da0956640 1 -- 172.21.15.32:0/3092154151 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d8c0a8ad0 con 0x7f2d9c0d2740 2023-12-20T04:28:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da0956640 1 -- 172.21.15.32:0/3092154151 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d8c0a8e60 con 0x7f2d9c0d2740 2023-12-20T04:28:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da3bbc640 1 -- 172.21.15.32:0/3092154151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c0d2740 msgr2=0x7f2d9c0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da3bbc640 1 --2- 172.21.15.32:0/3092154151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c0d2740 0x7f2d9c0cf570 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0097c0 tx=0x7f2d8c01ec70 comp rx=0 tx=0).stop 2023-12-20T04:28:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da3bbc640 1 -- 172.21.15.32:0/3092154151 shutdown_connections 2023-12-20T04:28:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da3bbc640 1 --2- 172.21.15.32:0/3092154151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c0d2740 0x7f2d9c0cf570 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da3bbc640 1 --2- 172.21.15.32:0/3092154151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d9c15cdb0 0x7f2d9c0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da3bbc640 1 -- 172.21.15.32:0/3092154151 >> 172.21.15.32:0/3092154151 conn(0x7f2d9c065ee0 msgr2=0x7f2d9c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:28:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da3bbc640 1 -- 172.21.15.32:0/3092154151 shutdown_connections 2023-12-20T04:28:59.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da3bbc640 1 -- 172.21.15.32:0/3092154151 wait complete. 2023-12-20T04:28:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da3bbc640 1 Processor -- start 2023-12-20T04:28:59.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.823+0000 7f2da3bbc640 1 -- start start 2023-12-20T04:28:59.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da3bbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c0d1120 0x7f2d9c1503d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da3bbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d9c0d2740 0x7f2d9c1f3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da3bbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c15cdb0 0x7f2d9c14dd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da3bbc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c1f3d60 con 0x7f2d9c0d1120 2023-12-20T04:28:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da3bbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c14e2b0 con 0x7f2d9c15cdb0 2023-12-20T04:28:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da3bbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d9c150a70 con 0x7f2d9c0d2740 2023-12-20T04:28:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da1958640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c0d1120 0x7f2d9c1503d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da1958640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c0d1120 0x7f2d9c1503d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47584/0 (socket says 172.21.15.32:47584) 2023-12-20T04:28:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da1958640 1 -- 172.21.15.32:0/1069923273 learned_addr learned my addr 172.21.15.32:0/1069923273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:28:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da1157640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d9c0d2740 0x7f2d9c1f3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da2159640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c15cdb0 0x7f2d9c14dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:59.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da1958640 1 -- 172.21.15.32:0/1069923273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d9c0d2740 msgr2=0x7f2d9c1f3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da1958640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d9c0d2740 0x7f2d9c1f3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da1958640 1 -- 172.21.15.32:0/1069923273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c15cdb0 msgr2=0x7f2d9c14dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da1958640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c15cdb0 0x7f2d9c14dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da1958640 1 -- 172.21.15.32:0/1069923273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8c009420 con 0x7f2d9c0d1120 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da1157640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d9c0d2740 0x7f2d9c1f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da1958640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c0d1120 0x7f2d9c1503d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800e930 tx=0x7f2d9800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2d927fc640 1 -- 172.21.15.32:0/1069923273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d98009690 con 0x7f2d9c0d1120 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2d927fc640 1 -- 172.21.15.32:0/1069923273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d9800ca00 con 0x7f2d9c0d1120 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2d927fc640 1 -- 172.21.15.32:0/1069923273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d9800cd70 con 0x7f2d9c0d1120 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da3bbc640 1 -- 172.21.15.32:0/1069923273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d9c150d50 con 0x7f2d9c0d1120 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da3bbc640 1 -- 172.21.15.32:0/1069923273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d9c151310 con 0x7f2d9c0d1120 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2da3bbc640 1 -- 172.21.15.32:0/1069923273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d9c04e000 con 0x7f2d9c0d1120 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.827+0000 7f2d927fc640 1 -- 172.21.15.32:0/1069923273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2d98013470 con 0x7f2d9c0d1120 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.831+0000 7f2d927fc640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d80065a60 0x7f2d80067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:28:59.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.831+0000 7f2d927fc640 1 -- 172.21.15.32:0/1069923273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2d98083520 con 0x7f2d9c0d1120 2023-12-20T04:28:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.831+0000 7f2da1157640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d80065a60 0x7f2d80067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:28:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.831+0000 7f2da1157640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d80065a60 0x7f2d80067f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0b2000 tx=0x7f2d8c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:28:59.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:28:59.835+0000 7f2d927fc640 1 -- 172.21.15.32:0/1069923273 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2d98050f00 con 0x7f2d9c0d1120 2023-12-20T04:29:00.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.023+0000 7f2da3bbc640 1 -- 172.21.15.32:0/1069923273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f2d9c157c00 con 0x7f2d9c0d1120 2023-12-20T04:29:00.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.023+0000 7f2d927fc640 1 -- 172.21.15.32:0/1069923273 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v8) v1 ==== 53+0+73 (secure 0 0 0) 0x7f2d98054520 con 0x7f2d9c0d1120 2023-12-20T04:29:00.027 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T04:29:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 -- 172.21.15.32:0/1069923273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d80065a60 msgr2=0x7f2d80067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d80065a60 0x7f2d80067f20 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f2d8c0b2000 tx=0x7f2d8c008040 comp rx=0 tx=0).stop 2023-12-20T04:29:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 -- 172.21.15.32:0/1069923273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c0d1120 msgr2=0x7f2d9c1503d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c0d1120 0x7f2d9c1503d0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f2d9800e930 tx=0x7f2d9800edf0 comp rx=0 tx=0).stop 2023-12-20T04:29:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 -- 172.21.15.32:0/1069923273 shutdown_connections 2023-12-20T04:29:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d80065a60 0x7f2d80067f20 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:00.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d9c15cdb0 0x7f2d9c14dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d9c0d2740 0x7f2d9c1f3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 --2- 172.21.15.32:0/1069923273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d9c0d1120 0x7f2d9c1503d0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 -- 172.21.15.32:0/1069923273 >> 172.21.15.32:0/1069923273 conn(0x7f2d9c065ee0 msgr2=0x7f2d9c0cd860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 -- 172.21.15.32:0/1069923273 shutdown_connections 2023-12-20T04:29:00.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:00.027+0000 7f2da3bbc640 1 -- 172.21.15.32:0/1069923273 wait complete. 2023-12-20T04:29:00.048 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2023-12-20T04:29:00.048 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T04:29:00.048 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi039.front.sepia.ceph.com 2023-12-20T04:29:00.048 INFO:tasks.cephfs.mount:self.client.name = client.3 2023-12-20T04:29:00.048 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.3 2023-12-20T04:29:00.048 INFO:tasks.cephfs.mount:self.cephfs_name = None 2023-12-20T04:29:00.048 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2023-12-20T04:29:00.048 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T04:29:00.048 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.3' 2023-12-20T04:29:00.049 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.049 DEBUG:teuthology.orchestra.run.smithi039:> ip addr 2023-12-20T04:29:00.060 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:29:00.060 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:29:00.060 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c6 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f0 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:3: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:6b:fe:c7 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp7s0f1 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout:4: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:70 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f0 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.21.15.39/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T04:29:00.061 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft 42365sec preferred_lft 42365sec 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::ec4:7aff:febd:1370/64 scope link 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0c:c4:7a:bd:13:71 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: altname enp1s0f1 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 02:42:7e:18:c0:fb brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link tentative 2023-12-20T04:29:00.062 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.063 INFO:teuthology.orchestra.run.smithi039.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:29:00.063 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-20T04:29:00.063 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T04:29:00.063 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.063 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.063 DEBUG:teuthology.orchestra.run.smithi039:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T04:29:00.121 INFO:teuthology.orchestra.run.smithi039.stdout:1 2023-12-20T04:29:00.123 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.123 DEBUG:teuthology.orchestra.run.smithi039:> route 2023-12-20T04:29:00.175 INFO:teuthology.orchestra.run.smithi039.stdout:Kernel IP routing table 2023-12-20T04:29:00.175 INFO:teuthology.orchestra.run.smithi039.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T04:29:00.175 INFO:teuthology.orchestra.run.smithi039.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T04:29:00.175 INFO:teuthology.orchestra.run.smithi039.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T04:29:00.176 INFO:teuthology.orchestra.run.smithi039.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T04:29:00.176 INFO:teuthology.orchestra.run.smithi039.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T04:29:00.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.176 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-12-20T04:29:00.176 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-12-20T04:29:00.176 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T04:29:00.176 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T04:29:00.176 DEBUG:teuthology.orchestra.run.smithi039:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T04:29:00.177 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-12-20T04:29:00.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.271 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2023-12-20T04:29:00.280 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-20T04:29:00.281 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.281 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list-id 2023-12-20T04:29:00.331 INFO:teuthology.orchestra.run.smithi039.stdout:nsid 0 (iproute2 netns name: ceph-ns--home-ubuntu-cephtest-mnt.1) 2023-12-20T04:29:00.331 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.331 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-12-20T04:29:00.331 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-12-20T04:29:00.331 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-20T04:29:00.331 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.3 1 2023-12-20T04:29:00.331 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-12-20T04:29:00.415 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.415 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-20T04:29:00.479 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:29:00.480 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:29:00.480 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:29:00.480 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.480 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2023-12-20T04:29:00.480 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.480 INFO:teuthology.orchestra.run.smithi039.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:29:00.480 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether e6:6c:89:be:29:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-20T04:29:00.480 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-20T04:29:00.480 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.480 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::e46c:89ff:febe:2965/64 scope link tentative 2023-12-20T04:29:00.480 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.481 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.481 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.1 ip addr 2023-12-20T04:29:00.547 INFO:teuthology.orchestra.run.smithi039.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:29:00.547 INFO:teuthology.orchestra.run.smithi039.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:29:00.547 INFO:teuthology.orchestra.run.smithi039.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:29:00.548 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.548 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 ::1/128 scope host 2023-12-20T04:29:00.548 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.548 INFO:teuthology.orchestra.run.smithi039.stdout:2: veth0@if8: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:29:00.548 INFO:teuthology.orchestra.run.smithi039.stdout: link/ether e6:6c:89:be:29:65 brd ff:ff:ff:ff:ff:ff link-netnsid 0 2023-12-20T04:29:00.548 INFO:teuthology.orchestra.run.smithi039.stdout: inet 192.168.0.1/16 brd 192.168.255.255 scope global veth0 2023-12-20T04:29:00.548 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.548 INFO:teuthology.orchestra.run.smithi039.stdout: inet6 fe80::e46c:89ff:febe:2965/64 scope link tentative 2023-12-20T04:29:00.548 INFO:teuthology.orchestra.run.smithi039.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:00.549 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.3' with 192.168.0.2/16 2023-12-20T04:29:00.549 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.549 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-12-20T04:29:00.550 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-12-20T04:29:00.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.3 type veth peer name brx.1 2023-12-20T04:29:00.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip addr add 192.168.0.2/16 brd 192.168.255.255 dev veth0 2023-12-20T04:29:00.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set veth0 up 2023-12-20T04:29:00.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip link set lo up 2023-12-20T04:29:00.550 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.3 ip route add default via 192.168.255.254 2023-12-20T04:29:00.550 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-12-20T04:29:00.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.691 DEBUG:teuthology.orchestra.run.smithi039:> (cd / && exec bash -c ' 2023-12-20T04:29:00.691 DEBUG:teuthology.orchestra.run.smithi039:> set -e 2023-12-20T04:29:00.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set brx.1 up 2023-12-20T04:29:00.691 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link set dev brx.1 master ceph-brx 2023-12-20T04:29:00.691 DEBUG:teuthology.orchestra.run.smithi039:> ') 2023-12-20T04:29:00.731 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:29:00.732 DEBUG:teuthology.orchestra.run.smithi039:> mkdir -p -v /home/ubuntu/cephtest/mnt.3 2023-12-20T04:29:00.782 INFO:teuthology.orchestra.run.smithi039.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.3' 2023-12-20T04:29:00.783 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:29:00.783 DEBUG:teuthology.orchestra.run.smithi039:> chmod 0000 /home/ubuntu/cephtest/mnt.3 2023-12-20T04:29:00.838 INFO:tasks.cephfs.kernel_mount:mounting using device: 3@.cephfs=/ 2023-12-20T04:29:00.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.838 DEBUG:teuthology.orchestra.run.smithi039:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 3@.cephfs=/ /home/ubuntu/cephtest/mnt.3 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:29:00.933 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:29:00.933 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:29:00.947 INFO:teuthology.orchestra.run.smithi039.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:29:00.947 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:29:00.947 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T04:29:00.947 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: trying mount with new device syntax: 3@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T04:29:00.947 INFO:teuthology.orchestra.run.smithi039.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=3,ms_mode=prefer-crc,key=3,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T04:29:00.951 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T04:29:00.951 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.951 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.3 2023-12-20T04:29:00.973 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:00.973 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:29:01.029 INFO:teuthology.orchestra.run.smithi039.stdout:client4680 2023-12-20T04:29:01.029 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:29:01.029 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:01.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:01.078 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:29:01.132 INFO:teuthology.orchestra.run.smithi039.stdout:client4680 2023-12-20T04:29:01.133 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:01.133 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:29:01.184 INFO:teuthology.orchestra.run.smithi039.stdout:client4680 2023-12-20T04:29:01.185 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:01.185 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest/mnt.3 && exec bash -c 'sudo dd if=/sys/kernel/debug/ceph/9b496156-2b6f-4897-9af4-fbc686ad55e7.client4680/status') 2023-12-20T04:29:01.246 INFO:teuthology.orchestra.run.smithi039.stdout:instance: client.4680 (3)172.21.15.39:0/3562047173 2023-12-20T04:29:01.246 INFO:teuthology.orchestra.run.smithi039.stdout:blocklisted: false 2023-12-20T04:29:01.246 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records in 2023-12-20T04:29:01.247 INFO:teuthology.orchestra.run.smithi039.stderr:0+1 records out 2023-12-20T04:29:01.247 INFO:teuthology.orchestra.run.smithi039.stderr:70 bytes copied, 7.6103e-05 s, 920 kB/s 2023-12-20T04:29:01.248 INFO:teuthology.run_tasks:Running task cephfs_test_runner... 2023-12-20T04:29:01.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:01.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.419+0000 7fa10cb7b640 1 Processor -- start 2023-12-20T04:29:01.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.419+0000 7fa10cb7b640 1 -- start start 2023-12-20T04:29:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.419+0000 7fa10cb7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa10806ce60 0x7fa10815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.419+0000 7fa10cb7b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa10815f110 0x7fa10815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.419+0000 7fa10cb7b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa10815bbd0 con 0x7fa1080af5e0 2023-12-20T04:29:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.419+0000 7fa10cb7b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa10815beb0 con 0x7fa10815f110 2023-12-20T04:29:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.419+0000 7fa10cb7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa10815e910 con 0x7fa10806ce60 2023-12-20T04:29:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.419+0000 7fa106d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa10815f110 0x7fa10815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.419+0000 7fa106d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa10815f110 0x7fa10815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47600/0 (socket says 172.21.15.32:47600) 2023-12-20T04:29:01.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.419+0000 7fa106d9d640 1 -- 172.21.15.32:0/117285620 learned_addr learned my addr 172.21.15.32:0/117285620 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.419+0000 7fa0fffff640 1 --2- 172.21.15.32:0/117285620 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa10806ce60 0x7fa10815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa106d9d640 1 -- 172.21.15.32:0/117285620 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa10806ce60 msgr2=0x7fa10815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa106d9d640 1 --2- 172.21.15.32:0/117285620 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa10806ce60 0x7fa10815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa106d9d640 1 -- 172.21.15.32:0/117285620 >> v1:172.21.15.39:6789/0 conn(0x7fa1080af5e0 legacy=0x7fa1080af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa106d9d640 1 -- 172.21.15.32:0/117285620 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa10815ebf0 con 0x7fa10815f110 2023-12-20T04:29:01.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa106d9d640 1 --2- 172.21.15.32:0/117285620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10815f110 0x7fa10815c440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa100009e50 tx=0x7fa10001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27cda729be31495 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa105d9b640 1 -- 172.21.15.32:0/117285620 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1000a57e0 con 0x7fa10815f110 2023-12-20T04:29:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa105d9b640 1 -- 172.21.15.32:0/117285620 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1000a5940 con 0x7fa10815f110 2023-12-20T04:29:01.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa105d9b640 1 -- 172.21.15.32:0/117285620 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1000a5cd0 con 0x7fa10815f110 2023-12-20T04:29:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa10cb7b640 1 -- 172.21.15.32:0/117285620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10815f110 msgr2=0x7fa10815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa10cb7b640 1 --2- 172.21.15.32:0/117285620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10815f110 0x7fa10815c440 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa100009e50 tx=0x7fa10001c980 comp rx=0 tx=0).stop 2023-12-20T04:29:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa10cb7b640 1 -- 172.21.15.32:0/117285620 shutdown_connections 2023-12-20T04:29:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa10cb7b640 1 --2- 172.21.15.32:0/117285620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10815f110 0x7fa10815c440 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa10cb7b640 1 --2- 172.21.15.32:0/117285620 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa10806ce60 0x7fa10815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa10cb7b640 1 -- 172.21.15.32:0/117285620 >> 172.21.15.32:0/117285620 conn(0x7fa10806a320 msgr2=0x7fa1080ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa10cb7b640 1 -- 172.21.15.32:0/117285620 shutdown_connections 2023-12-20T04:29:01.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.423+0000 7fa10cb7b640 1 -- 172.21.15.32:0/117285620 wait complete. 2023-12-20T04:29:01.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa10cb7b640 1 Processor -- start 2023-12-20T04:29:01.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa10cb7b640 1 -- start start 2023-12-20T04:29:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa10cb7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa10806ce60 0x7fa108071d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa10cb7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1080af5e0 0x7fa10815d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa10cb7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10815f110 0x7fa108149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa10cb7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa108072260 con 0x7fa10815f110 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa10cb7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa108070710 con 0x7fa10806ce60 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa10cb7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa108153760 con 0x7fa1080af5e0 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa106d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10815f110 0x7fa108149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa0fffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1080af5e0 0x7fa10815d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa10659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa10806ce60 0x7fa108071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa106d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10815f110 0x7fa108149890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47604/0 (socket says 172.21.15.32:47604) 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa0fffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1080af5e0 0x7fa10815d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:42534/0 (socket says 172.21.15.32:42534) 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa106d9d640 1 -- 172.21.15.32:0/2432864922 learned_addr learned my addr 172.21.15.32:0/2432864922 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa106d9d640 1 -- 172.21.15.32:0/2432864922 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1080af5e0 msgr2=0x7fa10815d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa106d9d640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1080af5e0 0x7fa10815d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa106d9d640 1 -- 172.21.15.32:0/2432864922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa10806ce60 msgr2=0x7fa108071d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa106d9d640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa10806ce60 0x7fa108071d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa106d9d640 1 -- 172.21.15.32:0/2432864922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa100009b30 con 0x7fa10815f110 2023-12-20T04:29:01.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.427+0000 7fa106d9d640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10815f110 0x7fa108149890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa1000af040 tx=0x7fa100019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:01.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.431+0000 7fa0feffd640 1 -- 172.21.15.32:0/2432864922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1000b8640 con 0x7fa10815f110 2023-12-20T04:29:01.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.431+0000 7fa0feffd640 1 -- 172.21.15.32:0/2432864922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1000b87a0 con 0x7fa10815f110 2023-12-20T04:29:01.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.431+0000 7fa0feffd640 1 -- 172.21.15.32:0/2432864922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1000b8b30 con 0x7fa10815f110 2023-12-20T04:29:01.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.431+0000 7fa10cb7b640 1 -- 172.21.15.32:0/2432864922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa108153a40 con 0x7fa10815f110 2023-12-20T04:29:01.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.431+0000 7fa10cb7b640 1 -- 172.21.15.32:0/2432864922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa108154000 con 0x7fa10815f110 2023-12-20T04:29:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.431+0000 7fa10cb7b640 1 -- 172.21.15.32:0/2432864922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1080c8b70 con 0x7fa10815f110 2023-12-20T04:29:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.431+0000 7fa0feffd640 1 -- 172.21.15.32:0/2432864922 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa1000b8c90 con 0x7fa10815f110 2023-12-20T04:29:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.431+0000 7fa0feffd640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa0e8065a60 0x7fa0e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.431+0000 7fa0feffd640 1 -- 172.21.15.32:0/2432864922 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa10011dcd0 con 0x7fa10815f110 2023-12-20T04:29:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.431+0000 7fa10659c640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa0e8065a60 0x7fa0e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.431+0000 7fa10659c640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa0e8065a60 0x7fa0e8067f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000fc0 tx=0x7fa0f8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.435+0000 7fa0feffd640 1 -- 172.21.15.32:0/2432864922 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa1000eb6a0 con 0x7fa10815f110 2023-12-20T04:29:01.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.599+0000 7fa10cb7b640 1 -- 172.21.15.32:0/2432864922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa1081538a0 con 0x7fa10815f110 2023-12-20T04:29:01.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa0feffd640 1 -- 172.21.15.32:0/2432864922 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7fa1081538a0 con 0x7fa10815f110 2023-12-20T04:29:01.604 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:01.605 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 -- 172.21.15.32:0/2432864922 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa0e8065a60 msgr2=0x7fa0e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa0e8065a60 0x7fa0e8067f20 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fa0f8000fc0 tx=0x7fa0f8009040 comp rx=0 tx=0).stop 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 -- 172.21.15.32:0/2432864922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10815f110 msgr2=0x7fa108149890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10815f110 0x7fa108149890 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fa1000af040 tx=0x7fa100019720 comp rx=0 tx=0).stop 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 -- 172.21.15.32:0/2432864922 shutdown_connections 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa0e8065a60 0x7fa0e8067f20 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa10815f110 0x7fa108149890 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1080af5e0 0x7fa10815d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 --2- 172.21.15.32:0/2432864922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa10806ce60 0x7fa108071d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 -- 172.21.15.32:0/2432864922 >> 172.21.15.32:0/2432864922 conn(0x7fa10806a320 msgr2=0x7fa108157a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 -- 172.21.15.32:0/2432864922 shutdown_connections 2023-12-20T04:29:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.603+0000 7fa10cb7b640 1 -- 172.21.15.32:0/2432864922 wait complete. 2023-12-20T04:29:01.627 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:28:36.040195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:28:37.364671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:29:01.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:29:01.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.771+0000 7ff713539640 1 Processor -- start 2023-12-20T04:29:01.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.771+0000 7ff713539640 1 -- start start 2023-12-20T04:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff713539640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c0ca1a0 0x7ff70c0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff713539640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70c0cb0b0 con 0x7ff70c164670 2023-12-20T04:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff713539640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff70c0ca810 con 0x7ff70c070230 2023-12-20T04:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff713539640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0ca600 con 0x7ff70c0ca1a0 2023-12-20T04:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff7112d5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff70c164670 0x7ff70c06da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57244/0 (socket says 172.21.15.32:57244) 2023-12-20T04:29:01.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff7112d5640 1 -- 172.21.15.32:0/3404044901 learned_addr learned my addr 172.21.15.32:0/3404044901 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff710ad4640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c0ca1a0 0x7ff70c0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3943346775 0 0) 0x7ff70c0cb0b0 con 0x7ff70c164670 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff6f8003530 con 0x7ff70c164670 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4236203412 0 0) 0x7ff70c0ca810 con 0x7ff70c070230 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff70c0cb0b0 con 0x7ff70c070230 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3226914913 0 0) 0x7ff6f8003530 con 0x7ff70c164670 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c0ca1a0 msgr2=0x7ff70c0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c0ca1a0 0x7ff70c0cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 >> v1:172.21.15.39:6789/0 conn(0x7ff70c070230 legacy=0x7ff70c06ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c0caaf0 con 0x7ff70c164670 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7ff708001680 con 0x7ff70c164670 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff710ad4640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c0ca1a0 0x7ff70c0cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:01.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 >> v1:172.21.15.32:6789/0 conn(0x7ff70c164670 legacy=0x7ff70c06da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f8003b40 0x7ff6f8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f80047d0 0x7ff6f8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c0ca1a0 0x7ff6f8005200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c0ca600 con 0x7ff6f8003b40 2023-12-20T04:29:01.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f800bb20 con 0x7ff6f80047d0 2023-12-20T04:29:01.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f800bcb0 con 0x7ff70c0ca1a0 2023-12-20T04:29:01.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff7112d5640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f8003b40 0x7ff6f8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff711ad6640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c0ca1a0 0x7ff6f8005200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff710ad4640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f80047d0 0x7ff6f8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff7112d5640 1 -- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c0ca1a0 msgr2=0x7ff6f8005200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:01.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff7112d5640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c0ca1a0 0x7ff6f8005200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff7112d5640 1 -- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f80047d0 msgr2=0x7ff6f8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:01.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff7112d5640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f80047d0 0x7ff6f8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.775+0000 7ff7112d5640 1 -- 172.21.15.32:0/3404044901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6fc009420 con 0x7ff6f8003b40 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff7112d5640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f8003b40 0x7ff6f8003f40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff70800c190 tx=0x7ff70801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abbdd028466560ea server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7080058e0 con 0x7ff6f8003b40 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff708005a40 con 0x7ff6f8003b40 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff703fff640 1 -- 172.21.15.32:0/3404044901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff708005df0 con 0x7ff6f8003b40 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff713539640 1 -- 172.21.15.32:0/3404044901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f8003b40 msgr2=0x7ff6f8003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff713539640 1 --2- 172.21.15.32:0/3404044901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f8003b40 0x7ff6f8003f40 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7ff70800c190 tx=0x7ff70801cbc0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff711ad6640 1 -- 172.21.15.32:0/3404044901 reap_dead start 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff713539640 1 -- 172.21.15.32:0/3404044901 shutdown_connections 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff713539640 1 -- 172.21.15.32:0/3404044901 >> 172.21.15.32:0/3404044901 conn(0x7ff70c06a510 msgr2=0x7ff70c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff713539640 1 -- 172.21.15.32:0/3404044901 shutdown_connections 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff713539640 1 -- 172.21.15.32:0/3404044901 wait complete. 2023-12-20T04:29:01.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff713539640 1 Processor -- start 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.779+0000 7ff713539640 1 -- start start 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff713539640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f8003b40 0x7ff70c15fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff713539640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c158a70 0x7ff70c0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff713539640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff70c0b2d90 0x7ff70c0c3c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff713539640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c160410 con 0x7ff70c0b2d90 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff713539640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c1fd690 con 0x7ff6f8003b40 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff713539640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff70c1fd970 con 0x7ff70c158a70 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff711ad6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff70c0b2d90 0x7ff70c0c3c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff711ad6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff70c0b2d90 0x7ff70c0c3c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47612/0 (socket says 172.21.15.32:47612) 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff711ad6640 1 -- 172.21.15.32:0/3064314522 learned_addr learned my addr 172.21.15.32:0/3064314522 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff7112d5640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f8003b40 0x7ff70c15fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff710ad4640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c158a70 0x7ff70c0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff711ad6640 1 -- 172.21.15.32:0/3064314522 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c158a70 msgr2=0x7ff70c0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff711ad6640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c158a70 0x7ff70c0b2850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff711ad6640 1 -- 172.21.15.32:0/3064314522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f8003b40 msgr2=0x7ff70c15fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff711ad6640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f8003b40 0x7ff70c15fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff711ad6640 1 -- 172.21.15.32:0/3064314522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff70c070bd0 con 0x7ff70c0b2d90 2023-12-20T04:29:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff7112d5640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f8003b40 0x7ff70c15fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff710ad4640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c158a70 0x7ff70c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff711ad6640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff70c0b2d90 0x7ff70c0c3c20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff70c06e610 tx=0x7ff70400ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff701ffb640 1 -- 172.21.15.32:0/3064314522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704006d70 con 0x7ff70c0b2d90 2023-12-20T04:29:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff701ffb640 1 -- 172.21.15.32:0/3064314522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff704003980 con 0x7ff70c0b2d90 2023-12-20T04:29:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff701ffb640 1 -- 172.21.15.32:0/3064314522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff704003d30 con 0x7ff70c0b2d90 2023-12-20T04:29:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff713539640 1 -- 172.21.15.32:0/3064314522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff70c1fdc50 con 0x7ff70c0b2d90 2023-12-20T04:29:01.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff713539640 1 -- 172.21.15.32:0/3064314522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff70c1fd2b0 con 0x7ff70c0b2d90 2023-12-20T04:29:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff713539640 1 -- 172.21.15.32:0/3064314522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff70c04e000 con 0x7ff70c0b2d90 2023-12-20T04:29:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.783+0000 7ff701ffb640 1 -- 172.21.15.32:0/3064314522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff7040050d0 con 0x7ff70c0b2d90 2023-12-20T04:29:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.787+0000 7ff701ffb640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6f0065a90 0x7ff6f0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.787+0000 7ff701ffb640 1 -- 172.21.15.32:0/3064314522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff7040822b0 con 0x7ff70c0b2d90 2023-12-20T04:29:01.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.787+0000 7ff7112d5640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6f0065a90 0x7ff6f0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:01.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.787+0000 7ff7112d5640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6f0065a90 0x7ff6f0067f50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff7080aa000 tx=0x7ff708001990 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:01.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.791+0000 7ff701ffb640 1 -- 172.21.15.32:0/3064314522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff70404fd00 con 0x7ff70c0b2d90 2023-12-20T04:29:01.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 -- 172.21.15.32:0/3064314522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff70c165de0 con 0x7ff70c0b2d90 2023-12-20T04:29:01.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff701ffb640 1 -- 172.21.15.32:0/3064314522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff704053320 con 0x7ff70c0b2d90 2023-12-20T04:29:01.953 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 -- 172.21.15.32:0/3064314522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6f0065a90 msgr2=0x7ff6f0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6f0065a90 0x7ff6f0067f50 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7ff7080aa000 tx=0x7ff708001990 comp rx=0 tx=0).stop 2023-12-20T04:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 -- 172.21.15.32:0/3064314522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff70c0b2d90 msgr2=0x7ff70c0c3c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff70c0b2d90 0x7ff70c0c3c20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7ff70c06e610 tx=0x7ff70400ff70 comp rx=0 tx=0).stop 2023-12-20T04:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 -- 172.21.15.32:0/3064314522 shutdown_connections 2023-12-20T04:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6f0065a90 0x7ff6f0067f50 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff70c0b2d90 0x7ff70c0c3c20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff70c158a70 0x7ff70c0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 --2- 172.21.15.32:0/3064314522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f8003b40 0x7ff70c15fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 -- 172.21.15.32:0/3064314522 >> 172.21.15.32:0/3064314522 conn(0x7ff70c06a510 msgr2=0x7ff70c15d830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:01.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 -- 172.21.15.32:0/3064314522 shutdown_connections 2023-12-20T04:29:01.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:01.951+0000 7ff713539640 1 -- 172.21.15.32:0/3064314522 wait complete. 2023-12-20T04:29:01.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:29:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.119+0000 7fee69d19640 1 Processor -- start 2023-12-20T04:29:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.119+0000 7fee69d19640 1 -- start start 2023-12-20T04:29:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69d19640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee64158e40 0x7fee640ccc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69d19640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee640cb310 0x7fee64071270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69d19640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee6415cd40 con 0x7fee640cd150 2023-12-20T04:29:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69d19640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee640cd590 con 0x7fee640cb310 2023-12-20T04:29:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69d19640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640cd780 con 0x7fee64158e40 2023-12-20T04:29:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69518640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee640cb310 0x7fee64071270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69518640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee640cb310 0x7fee64071270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47620/0 (socket says 172.21.15.32:47620) 2023-12-20T04:29:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69518640 1 -- 172.21.15.32:0/3567552751 learned_addr learned my addr 172.21.15.32:0/3567552751 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69518640 1 -- 172.21.15.32:0/3567552751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee64158e40 msgr2=0x7fee640ccc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69518640 1 --2- 172.21.15.32:0/3567552751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee64158e40 0x7fee640ccc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69518640 1 -- 172.21.15.32:0/3567552751 >> v1:172.21.15.39:6789/0 conn(0x7fee640cd150 legacy=0x7fee640cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69518640 1 -- 172.21.15.32:0/3567552751 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee640cda60 con 0x7fee640cb310 2023-12-20T04:29:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.123+0000 7fee69518640 1 --2- 172.21.15.32:0/3567552751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640cb310 0x7fee64071270 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009e50 tx=0x7fee5c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7370c28712250eae server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:02.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee637fe640 1 -- 172.21.15.32:0/3567552751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0a5710 con 0x7fee640cb310 2023-12-20T04:29:02.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee637fe640 1 -- 172.21.15.32:0/3567552751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee5c0a5870 con 0x7fee640cb310 2023-12-20T04:29:02.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee637fe640 1 -- 172.21.15.32:0/3567552751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee5c0a5c00 con 0x7fee640cb310 2023-12-20T04:29:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 -- 172.21.15.32:0/3567552751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640cb310 msgr2=0x7fee64071270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 --2- 172.21.15.32:0/3567552751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640cb310 0x7fee64071270 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fee5c009e50 tx=0x7fee5c01c910 comp rx=0 tx=0).stop 2023-12-20T04:29:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 -- 172.21.15.32:0/3567552751 shutdown_connections 2023-12-20T04:29:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 --2- 172.21.15.32:0/3567552751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640cb310 0x7fee64071270 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 --2- 172.21.15.32:0/3567552751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee64158e40 0x7fee640ccc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 -- 172.21.15.32:0/3567552751 >> 172.21.15.32:0/3567552751 conn(0x7fee6406a4e0 msgr2=0x7fee6406bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 -- 172.21.15.32:0/3567552751 shutdown_connections 2023-12-20T04:29:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 -- 172.21.15.32:0/3567552751 wait complete. 2023-12-20T04:29:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 Processor -- start 2023-12-20T04:29:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 -- start start 2023-12-20T04:29:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640cb310 0x7fee640ace10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640cd150 0x7fee641f8380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee64158e40 0x7fee640c6010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee641f88c0 con 0x7fee640cd150 2023-12-20T04:29:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee640c6550 con 0x7fee64158e40 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee69d19640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee640c6820 con 0x7fee640cb310 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.127+0000 7fee63fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640cd150 0x7fee641f8380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee63fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640cd150 0x7fee641f8380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47634/0 (socket says 172.21.15.32:47634) 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee63fff640 1 -- 172.21.15.32:0/2911819267 learned_addr learned my addr 172.21.15.32:0/2911819267 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee69518640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee64158e40 0x7fee640c6010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee68d17640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640cb310 0x7fee640ace10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee63fff640 1 -- 172.21.15.32:0/2911819267 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640cb310 msgr2=0x7fee640ace10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee63fff640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640cb310 0x7fee640ace10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee63fff640 1 -- 172.21.15.32:0/2911819267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee64158e40 msgr2=0x7fee640c6010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee63fff640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee64158e40 0x7fee640c6010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee63fff640 1 -- 172.21.15.32:0/2911819267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee5c009b30 con 0x7fee640cd150 2023-12-20T04:29:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee68d17640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640cb310 0x7fee640ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:02.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee63fff640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640cd150 0x7fee641f8380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee5000c450 tx=0x7fee5000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:02.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee617fa640 1 -- 172.21.15.32:0/2911819267 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50012830 con 0x7fee640cd150 2023-12-20T04:29:02.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee617fa640 1 -- 172.21.15.32:0/2911819267 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee500129b0 con 0x7fee640cd150 2023-12-20T04:29:02.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee617fa640 1 -- 172.21.15.32:0/2911819267 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee50012d60 con 0x7fee640cd150 2023-12-20T04:29:02.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee69d19640 1 -- 172.21.15.32:0/2911819267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee640c6b00 con 0x7fee640cd150 2023-12-20T04:29:02.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee69d19640 1 -- 172.21.15.32:0/2911819267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee640c70c0 con 0x7fee640cd150 2023-12-20T04:29:02.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee69d19640 1 -- 172.21.15.32:0/2911819267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee640cb980 con 0x7fee640cd150 2023-12-20T04:29:02.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.131+0000 7fee617fa640 1 -- 172.21.15.32:0/2911819267 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fee5001c460 con 0x7fee640cd150 2023-12-20T04:29:02.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.135+0000 7fee617fa640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee48065a60 0x7fee48067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.135+0000 7fee617fa640 1 -- 172.21.15.32:0/2911819267 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fee50083050 con 0x7fee640cd150 2023-12-20T04:29:02.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.135+0000 7fee68d17640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee48065a60 0x7fee48067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.135+0000 7fee68d17640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee48065a60 0x7fee48067f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fee58006fd0 tx=0x7fee58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:02.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.139+0000 7fee617fa640 1 -- 172.21.15.32:0/2911819267 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee500b40a0 con 0x7fee640cd150 2023-12-20T04:29:02.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.291+0000 7fee69d19640 1 -- 172.21.15.32:0/2911819267 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee64074100 con 0x7fee640cd150 2023-12-20T04:29:02.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.291+0000 7fee617fa640 1 -- 172.21.15.32:0/2911819267 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee500508e0 con 0x7fee640cd150 2023-12-20T04:29:02.295 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:29:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 -- 172.21.15.32:0/2911819267 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee48065a60 msgr2=0x7fee48067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee48065a60 0x7fee48067f20 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7fee58006fd0 tx=0x7fee58008040 comp rx=0 tx=0).stop 2023-12-20T04:29:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 -- 172.21.15.32:0/2911819267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640cd150 msgr2=0x7fee641f8380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640cd150 0x7fee641f8380 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fee5000c450 tx=0x7fee5000c910 comp rx=0 tx=0).stop 2023-12-20T04:29:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 -- 172.21.15.32:0/2911819267 shutdown_connections 2023-12-20T04:29:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee48065a60 0x7fee48067f20 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee64158e40 0x7fee640c6010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640cd150 0x7fee641f8380 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 --2- 172.21.15.32:0/2911819267 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640cb310 0x7fee640ace10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 -- 172.21.15.32:0/2911819267 >> 172.21.15.32:0/2911819267 conn(0x7fee6406a4e0 msgr2=0x7fee640c9c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:02.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 -- 172.21.15.32:0/2911819267 shutdown_connections 2023-12-20T04:29:02.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.295+0000 7fee69d19640 1 -- 172.21.15.32:0/2911819267 wait complete. 2023-12-20T04:29:02.316 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:29:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45aee63640 1 Processor -- start 2023-12-20T04:29:02.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45aee63640 1 -- start start 2023-12-20T04:29:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45aee63640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f45a8164670 0x7f45a80ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45aee63640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45a806da20 0x7f45a80cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45aee63640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a8070230 0x7f45a8070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45aee63640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a80cb0b0 con 0x7f45a806da20 2023-12-20T04:29:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45aee63640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a8070ba0 con 0x7f45a8164670 2023-12-20T04:29:02.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45aee63640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a80ca810 con 0x7f45a8070230 2023-12-20T04:29:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45a7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45a806da20 0x7f45a80cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45a7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45a806da20 0x7f45a80cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47638/0 (socket says 172.21.15.32:47638) 2023-12-20T04:29:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45a7fff640 1 -- 172.21.15.32:0/3893493562 learned_addr learned my addr 172.21.15.32:0/3893493562 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45acbff640 1 --2- 172.21.15.32:0/3893493562 >> v2:172.21.15.39:3300/0 conn(0x7f45a8164670 0x7f45a80ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.463+0000 7f45ad400640 1 --2- 172.21.15.32:0/3893493562 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a8070230 0x7f45a8070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45a7fff640 1 -- 172.21.15.32:0/3893493562 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a8070230 msgr2=0x7f45a8070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45a7fff640 1 --2- 172.21.15.32:0/3893493562 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a8070230 0x7f45a8070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45a7fff640 1 -- 172.21.15.32:0/3893493562 >> v2:172.21.15.39:3300/0 conn(0x7f45a8164670 msgr2=0x7f45a80ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45a7fff640 1 --2- 172.21.15.32:0/3893493562 >> v2:172.21.15.39:3300/0 conn(0x7f45a8164670 0x7f45a80ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45a7fff640 1 -- 172.21.15.32:0/3893493562 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a80caaf0 con 0x7f45a806da20 2023-12-20T04:29:02.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45ad400640 1 --2- 172.21.15.32:0/3893493562 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a8070230 0x7f45a8070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45a7fff640 1 --2- 172.21.15.32:0/3893493562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a806da20 0x7f45a80cc3b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f45a806e430 tx=0x7f459401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cbefa7c4962df19 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45a77fe640 1 -- 172.21.15.32:0/3893493562 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940ab600 con 0x7f45a806da20 2023-12-20T04:29:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45a77fe640 1 -- 172.21.15.32:0/3893493562 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45940a7050 con 0x7f45a806da20 2023-12-20T04:29:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45a77fe640 1 -- 172.21.15.32:0/3893493562 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940ab980 con 0x7f45a806da20 2023-12-20T04:29:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45aee63640 1 -- 172.21.15.32:0/3893493562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a806da20 msgr2=0x7f45a80cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.467+0000 7f45aee63640 1 --2- 172.21.15.32:0/3893493562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a806da20 0x7f45a80cc3b0 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f45a806e430 tx=0x7f459401c3b0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 -- 172.21.15.32:0/3893493562 shutdown_connections 2023-12-20T04:29:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 --2- 172.21.15.32:0/3893493562 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a8070230 0x7f45a8070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 --2- 172.21.15.32:0/3893493562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a806da20 0x7f45a80cc3b0 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 --2- 172.21.15.32:0/3893493562 >> v2:172.21.15.39:3300/0 conn(0x7f45a8164670 0x7f45a80ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 -- 172.21.15.32:0/3893493562 >> 172.21.15.32:0/3893493562 conn(0x7f45a806a510 msgr2=0x7f45a80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 -- 172.21.15.32:0/3893493562 shutdown_connections 2023-12-20T04:29:02.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 -- 172.21.15.32:0/3893493562 wait complete. 2023-12-20T04:29:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 Processor -- start 2023-12-20T04:29:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 -- start start 2023-12-20T04:29:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45a806da20 0x7f45a80c7350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a8070230 0x7f45a815f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a8164670 0x7f45a8163120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a8160660 con 0x7f45a8070230 2023-12-20T04:29:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a80c6900 con 0x7f45a806da20 2023-12-20T04:29:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45aee63640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a81600a0 con 0x7f45a8164670 2023-12-20T04:29:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45a7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a8070230 0x7f45a815f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45a7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a8070230 0x7f45a815f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47642/0 (socket says 172.21.15.32:47642) 2023-12-20T04:29:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45a7fff640 1 -- 172.21.15.32:0/97104378 learned_addr learned my addr 172.21.15.32:0/97104378 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:02.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45acbff640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45a806da20 0x7f45a80c7350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45ad400640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a8164670 0x7f45a8163120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45a7fff640 1 -- 172.21.15.32:0/97104378 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a8164670 msgr2=0x7f45a8163120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45a7fff640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a8164670 0x7f45a8163120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45a7fff640 1 -- 172.21.15.32:0/97104378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45a806da20 msgr2=0x7f45a80c7350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45a7fff640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45a806da20 0x7f45a80c7350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45a7fff640 1 -- 172.21.15.32:0/97104378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45940094d0 con 0x7f45a8070230 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45acbff640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45a806da20 0x7f45a80c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.471+0000 7f45a7fff640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a8070230 0x7f45a815f830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f459401aa60 tx=0x7f4594019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.475+0000 7f45a57fa640 1 -- 172.21.15.32:0/97104378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940b4420 con 0x7f45a8070230 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.475+0000 7f45a57fa640 1 -- 172.21.15.32:0/97104378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45940b4580 con 0x7f45a8070230 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.475+0000 7f45a57fa640 1 -- 172.21.15.32:0/97104378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45940b4930 con 0x7f45a8070230 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.475+0000 7f45aee63640 1 -- 172.21.15.32:0/97104378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a8160380 con 0x7f45a8070230 2023-12-20T04:29:02.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.475+0000 7f45aee63640 1 -- 172.21.15.32:0/97104378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a80c6e30 con 0x7f45a8070230 2023-12-20T04:29:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.475+0000 7f45aee63640 1 -- 172.21.15.32:0/97104378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a804e000 con 0x7f45a8070230 2023-12-20T04:29:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.475+0000 7f45a57fa640 1 -- 172.21.15.32:0/97104378 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f45940b2070 con 0x7f45a8070230 2023-12-20T04:29:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.475+0000 7f45a57fa640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065a60 0x7f458c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.475+0000 7f45a57fa640 1 -- 172.21.15.32:0/97104378 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f459411cd90 con 0x7f45a8070230 2023-12-20T04:29:02.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.475+0000 7f45acbff640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065a60 0x7f458c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.479+0000 7f45acbff640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065a60 0x7f458c067f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f459c009bb0 tx=0x7f459c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:02.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.479+0000 7f45a57fa640 1 -- 172.21.15.32:0/97104378 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f45940ea740 con 0x7f45a8070230 2023-12-20T04:29:02.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.635+0000 7f45aee63640 1 -- 172.21.15.32:0/97104378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f45a80b0bc0 con 0x7f45a8070230 2023-12-20T04:29:02.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.635+0000 7f45a57fa640 1 -- 172.21.15.32:0/97104378 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f45940edd60 con 0x7f45a8070230 2023-12-20T04:29:02.639 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:29:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 -- 172.21.15.32:0/97104378 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065a60 msgr2=0x7f458c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065a60 0x7f458c067f20 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f459c009bb0 tx=0x7f459c008040 comp rx=0 tx=0).stop 2023-12-20T04:29:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 -- 172.21.15.32:0/97104378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a8070230 msgr2=0x7f45a815f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a8070230 0x7f45a815f830 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f459401aa60 tx=0x7f4594019150 comp rx=0 tx=0).stop 2023-12-20T04:29:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 -- 172.21.15.32:0/97104378 shutdown_connections 2023-12-20T04:29:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065a60 0x7f458c067f20 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a8164670 0x7f45a8163120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a8070230 0x7f45a815f830 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 --2- 172.21.15.32:0/97104378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45a806da20 0x7f45a80c7350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 -- 172.21.15.32:0/97104378 >> 172.21.15.32:0/97104378 conn(0x7f45a806a510 msgr2=0x7f45a80c4c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 -- 172.21.15.32:0/97104378 shutdown_connections 2023-12-20T04:29:02.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.639+0000 7f45aee63640 1 -- 172.21.15.32:0/97104378 wait complete. 2023-12-20T04:29:02.661 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:02.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe84496d640 1 Processor -- start 2023-12-20T04:29:02.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe84496d640 1 -- start start 2023-12-20T04:29:02.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe84496d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe84015cdb0 0x7fe8400d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe84496d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe84015d190 con 0x7fe8400cfc80 2023-12-20T04:29:02.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe84496d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8400d16a0 con 0x7fe8400d2740 2023-12-20T04:29:02.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe84496d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400d1980 con 0x7fe84015cdb0 2023-12-20T04:29:02.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe83ed9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe8400cfc80 0x7fe8400b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57246/0 (socket says 172.21.15.32:57246) 2023-12-20T04:29:02.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe83ed9d640 1 -- 172.21.15.32:0/2645022584 learned_addr learned my addr 172.21.15.32:0/2645022584 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:02.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe83e59c640 1 --2- 172.21.15.32:0/2645022584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe84015cdb0 0x7fe8400d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe83d59a640 1 -- 172.21.15.32:0/2645022584 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2067797278 0 0) 0x7fe84015d190 con 0x7fe8400cfc80 2023-12-20T04:29:02.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe83d59a640 1 -- 172.21.15.32:0/2645022584 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe828003530 con 0x7fe8400cfc80 2023-12-20T04:29:02.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe83d59a640 1 -- 172.21.15.32:0/2645022584 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322482253 0 0) 0x7fe8400d16a0 con 0x7fe8400d2740 2023-12-20T04:29:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe83e59c640 1 -- 172.21.15.32:0/2645022584 >> v1:172.21.15.39:6789/0 conn(0x7fe8400d2740 legacy=0x7fe8400cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe83e59c640 1 -- 172.21.15.32:0/2645022584 >> v1:172.21.15.32:6789/0 conn(0x7fe8400cfc80 legacy=0x7fe8400b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.803+0000 7fe83e59c640 1 -- 172.21.15.32:0/2645022584 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400d1c60 con 0x7fe84015cdb0 2023-12-20T04:29:02.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.807+0000 7fe83e59c640 1 --2- 172.21.15.32:0/2645022584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe84015cdb0 0x7fe8400d0be0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe8300097f0 tx=0x7fe83001ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a1bfed87f82292b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:02.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.807+0000 7fe83d59a640 1 -- 172.21.15.32:0/2645022584 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300a89b0 con 0x7fe84015cdb0 2023-12-20T04:29:02.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.807+0000 7fe83d59a640 1 -- 172.21.15.32:0/2645022584 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8300a8b10 con 0x7fe84015cdb0 2023-12-20T04:29:02.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.807+0000 7fe83d59a640 1 -- 172.21.15.32:0/2645022584 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300a8ea0 con 0x7fe84015cdb0 2023-12-20T04:29:02.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.807+0000 7fe84496d640 1 -- 172.21.15.32:0/2645022584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe84015cdb0 msgr2=0x7fe8400d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.807+0000 7fe84496d640 1 --2- 172.21.15.32:0/2645022584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe84015cdb0 0x7fe8400d0be0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fe8300097f0 tx=0x7fe83001ecb0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.807+0000 7fe84496d640 1 -- 172.21.15.32:0/2645022584 shutdown_connections 2023-12-20T04:29:02.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.807+0000 7fe84496d640 1 --2- 172.21.15.32:0/2645022584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe84015cdb0 0x7fe8400d0be0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.807+0000 7fe84496d640 1 -- 172.21.15.32:0/2645022584 >> 172.21.15.32:0/2645022584 conn(0x7fe840065ee0 msgr2=0x7fe84006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:02.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.807+0000 7fe84496d640 1 -- 172.21.15.32:0/2645022584 shutdown_connections 2023-12-20T04:29:02.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.807+0000 7fe84496d640 1 -- 172.21.15.32:0/2645022584 wait complete. 2023-12-20T04:29:02.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe84496d640 1 Processor -- start 2023-12-20T04:29:02.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe84496d640 1 -- start start 2023-12-20T04:29:02.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe84496d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400cfc80 0x7fe8401f6970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe84496d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8400d2740 0x7fe84014d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe84496d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe84015cdb0 0x7fe8401f30d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe84496d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8401f2cb0 con 0x7fe8400cfc80 2023-12-20T04:29:02.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe84496d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe84014c7e0 con 0x7fe8400d2740 2023-12-20T04:29:02.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe84496d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe84014cac0 con 0x7fe84015cdb0 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400cfc80 0x7fe8401f6970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400cfc80 0x7fe8401f6970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47650/0 (socket says 172.21.15.32:47650) 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83e59c640 1 -- 172.21.15.32:0/1041101784 learned_addr learned my addr 172.21.15.32:0/1041101784 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83ed9d640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe84015cdb0 0x7fe8401f30d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83dd9b640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8400d2740 0x7fe84014d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83e59c640 1 -- 172.21.15.32:0/1041101784 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe84015cdb0 msgr2=0x7fe8401f30d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83e59c640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe84015cdb0 0x7fe8401f30d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83e59c640 1 -- 172.21.15.32:0/1041101784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8400d2740 msgr2=0x7fe84014d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83e59c640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8400d2740 0x7fe84014d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83e59c640 1 -- 172.21.15.32:0/1041101784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe830009420 con 0x7fe8400cfc80 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83dd9b640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8400d2740 0x7fe84014d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:02.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe83e59c640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400cfc80 0x7fe8401f6970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8300097f0 tx=0x7fe83001ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe82effd640 1 -- 172.21.15.32:0/1041101784 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300b0df0 con 0x7fe8400cfc80 2023-12-20T04:29:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe82effd640 1 -- 172.21.15.32:0/1041101784 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8300b9400 con 0x7fe8400cfc80 2023-12-20T04:29:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe82effd640 1 -- 172.21.15.32:0/1041101784 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8300b9770 con 0x7fe8400cfc80 2023-12-20T04:29:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe84496d640 1 -- 172.21.15.32:0/1041101784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8401f28d0 con 0x7fe8400cfc80 2023-12-20T04:29:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.811+0000 7fe84496d640 1 -- 172.21.15.32:0/1041101784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe84014d620 con 0x7fe8400cfc80 2023-12-20T04:29:02.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.815+0000 7fe84496d640 1 -- 172.21.15.32:0/1041101784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84004e000 con 0x7fe8400cfc80 2023-12-20T04:29:02.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.815+0000 7fe82effd640 1 -- 172.21.15.32:0/1041101784 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe8300b7040 con 0x7fe8400cfc80 2023-12-20T04:29:02.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.815+0000 7fe82effd640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe820065a60 0x7fe820067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:02.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.815+0000 7fe82effd640 1 -- 172.21.15.32:0/1041101784 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe8301201c0 con 0x7fe8400cfc80 2023-12-20T04:29:02.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.815+0000 7fe83dd9b640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe820065a60 0x7fe820067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:02.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.819+0000 7fe83dd9b640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe820065a60 0x7fe820067f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe8401f6eb0 tx=0x7fe834009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:02.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.823+0000 7fe82effd640 1 -- 172.21.15.32:0/1041101784 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe8300edc10 con 0x7fe8400cfc80 2023-12-20T04:29:02.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.979+0000 7fe84496d640 1 -- 172.21.15.32:0/1041101784 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe84006b8d0 con 0x7fe8400cfc80 2023-12-20T04:29:02.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.979+0000 7fe82effd640 1 -- 172.21.15.32:0/1041101784 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7fe8300bd090 con 0x7fe8400cfc80 2023-12-20T04:29:02.982 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:02.982 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 -- 172.21.15.32:0/1041101784 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe820065a60 msgr2=0x7fe820067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe820065a60 0x7fe820067f20 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fe8401f6eb0 tx=0x7fe834009040 comp rx=0 tx=0).stop 2023-12-20T04:29:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 -- 172.21.15.32:0/1041101784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400cfc80 msgr2=0x7fe8401f6970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400cfc80 0x7fe8401f6970 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fe8300097f0 tx=0x7fe83001ba50 comp rx=0 tx=0).stop 2023-12-20T04:29:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 -- 172.21.15.32:0/1041101784 shutdown_connections 2023-12-20T04:29:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe820065a60 0x7fe820067f20 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe84015cdb0 0x7fe8401f30d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8400d2740 0x7fe84014d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 --2- 172.21.15.32:0/1041101784 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400cfc80 0x7fe8401f6970 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:02.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 -- 172.21.15.32:0/1041101784 >> 172.21.15.32:0/1041101784 conn(0x7fe840065ee0 msgr2=0x7fe84015c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:02.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 -- 172.21.15.32:0/1041101784 shutdown_connections 2023-12-20T04:29:02.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:02.983+0000 7fe84496d640 1 -- 172.21.15.32:0/1041101784 wait complete. 2023-12-20T04:29:03.003 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:28:36.040195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:28:37.364671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:29:03.004 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:29:03.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fadeb20b640 1 Processor -- start 2023-12-20T04:29:03.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fadeb20b640 1 -- start start 2023-12-20T04:29:03.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fadeb20b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fade4157480 0x7fade40c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fadeb20b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fade40b0760 0x7fade40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fadeb20b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fade40b1100 con 0x7fade415d660 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fadeb20b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fade40cde20 con 0x7fade4157480 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fadeb20b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c8e30 con 0x7fade40b0760 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fade8fa7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fade4157480 0x7fade40c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fade8fa7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fade4157480 0x7fade40c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47658/0 (socket says 172.21.15.32:47658) 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fade8fa7640 1 -- 172.21.15.32:0/785791989 learned_addr learned my addr 172.21.15.32:0/785791989 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fade3fff640 1 --2- 172.21.15.32:0/785791989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fade40b0760 0x7fade40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fade8fa7640 1 -- 172.21.15.32:0/785791989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fade40b0760 msgr2=0x7fade40b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fade8fa7640 1 --2- 172.21.15.32:0/785791989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fade40b0760 0x7fade40b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fade8fa7640 1 -- 172.21.15.32:0/785791989 >> v1:172.21.15.32:6789/0 conn(0x7fade415d660 legacy=0x7fade415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fade8fa7640 1 -- 172.21.15.32:0/785791989 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fade40ce070 con 0x7fade4157480 2023-12-20T04:29:03.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.147+0000 7fade3fff640 1 --2- 172.21.15.32:0/785791989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fade40b0760 0x7fade40b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fade8fa7640 1 --2- 172.21.15.32:0/785791989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4157480 0x7fade40c9840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009870 tx=0x7fadd801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7f0525e65a198ce server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fade37fe640 1 -- 172.21.15.32:0/785791989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd80ab680 con 0x7fade4157480 2023-12-20T04:29:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fade37fe640 1 -- 172.21.15.32:0/785791989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fadd80ab7e0 con 0x7fade4157480 2023-12-20T04:29:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fade37fe640 1 -- 172.21.15.32:0/785791989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fadd80abb70 con 0x7fade4157480 2023-12-20T04:29:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fadeb20b640 1 -- 172.21.15.32:0/785791989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4157480 msgr2=0x7fade40c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fadeb20b640 1 --2- 172.21.15.32:0/785791989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4157480 0x7fade40c9840 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fadd8009870 tx=0x7fadd801c480 comp rx=0 tx=0).stop 2023-12-20T04:29:03.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fadeb20b640 1 -- 172.21.15.32:0/785791989 shutdown_connections 2023-12-20T04:29:03.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fadeb20b640 1 --2- 172.21.15.32:0/785791989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fade40b0760 0x7fade40b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fadeb20b640 1 --2- 172.21.15.32:0/785791989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade4157480 0x7fade40c9840 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fadeb20b640 1 -- 172.21.15.32:0/785791989 >> 172.21.15.32:0/785791989 conn(0x7fade406a510 msgr2=0x7fade40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:03.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fadeb20b640 1 -- 172.21.15.32:0/785791989 shutdown_connections 2023-12-20T04:29:03.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.151+0000 7fadeb20b640 1 -- 172.21.15.32:0/785791989 wait complete. 2023-12-20T04:29:03.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fadeb20b640 1 Processor -- start 2023-12-20T04:29:03.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fadeb20b640 1 -- start start 2023-12-20T04:29:03.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fadeb20b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fade40b0760 0x7fade4151060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fadeb20b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade4157480 0x7fade40c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fadeb20b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade415d660 0x7fade40c46e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fadeb20b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade415afd0 con 0x7fade415d660 2023-12-20T04:29:03.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fadeb20b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c7880 con 0x7fade4157480 2023-12-20T04:29:03.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fadeb20b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fade40c7b20 con 0x7fade40b0760 2023-12-20T04:29:03.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade97a8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade415d660 0x7fade40c46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade97a8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade415d660 0x7fade40c46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47672/0 (socket says 172.21.15.32:47672) 2023-12-20T04:29:03.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade97a8640 1 -- 172.21.15.32:0/3158020282 learned_addr learned my addr 172.21.15.32:0/3158020282 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:03.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade8fa7640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fade40b0760 0x7fade4151060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade3fff640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade4157480 0x7fade40c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade97a8640 1 -- 172.21.15.32:0/3158020282 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fade40b0760 msgr2=0x7fade4151060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade97a8640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fade40b0760 0x7fade4151060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade97a8640 1 -- 172.21.15.32:0/3158020282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade4157480 msgr2=0x7fade40c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade97a8640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade4157480 0x7fade40c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade97a8640 1 -- 172.21.15.32:0/3158020282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fadd80094d0 con 0x7fade415d660 2023-12-20T04:29:03.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade97a8640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade415d660 0x7fade40c46e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faddc00c770 tx=0x7faddc00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:03.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade17fa640 1 -- 172.21.15.32:0/3158020282 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc003b80 con 0x7fade415d660 2023-12-20T04:29:03.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade17fa640 1 -- 172.21.15.32:0/3158020282 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faddc003ce0 con 0x7fade415d660 2023-12-20T04:29:03.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fade17fa640 1 -- 172.21.15.32:0/3158020282 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faddc00aa90 con 0x7fade415d660 2023-12-20T04:29:03.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.155+0000 7fadeb20b640 1 -- 172.21.15.32:0/3158020282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fade40c7e00 con 0x7fade415d660 2023-12-20T04:29:03.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.159+0000 7fadeb20b640 1 -- 172.21.15.32:0/3158020282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fade40c80e0 con 0x7fade415d660 2023-12-20T04:29:03.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.159+0000 7fadeb20b640 1 -- 172.21.15.32:0/3158020282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fade415b180 con 0x7fade415d660 2023-12-20T04:29:03.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.159+0000 7fade17fa640 1 -- 172.21.15.32:0/3158020282 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7faddc00abf0 con 0x7fade415d660 2023-12-20T04:29:03.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.159+0000 7fade17fa640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fadc8065a60 0x7fadc8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.159+0000 7fade17fa640 1 -- 172.21.15.32:0/3158020282 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faddc083070 con 0x7fade415d660 2023-12-20T04:29:03.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.159+0000 7fade8fa7640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fadc8065a60 0x7fadc8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.159+0000 7fade8fa7640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fadc8065a60 0x7fadc8067f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fadd80af000 tx=0x7fadd80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:03.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.163+0000 7fade17fa640 1 -- 172.21.15.32:0/3158020282 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faddc0b40a0 con 0x7fade415d660 2023-12-20T04:29:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 -- 172.21.15.32:0/3158020282 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fade404e000 con 0x7fade415d660 2023-12-20T04:29:03.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fade17fa640 1 -- 172.21.15.32:0/3158020282 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fade415b180 con 0x7fade415d660 2023-12-20T04:29:03.324 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:29:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 -- 172.21.15.32:0/3158020282 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fadc8065a60 msgr2=0x7fadc8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fadc8065a60 0x7fadc8067f20 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7fadd80af000 tx=0x7fadd80a7000 comp rx=0 tx=0).stop 2023-12-20T04:29:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 -- 172.21.15.32:0/3158020282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade415d660 msgr2=0x7fade40c46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade415d660 0x7fade40c46e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faddc00c770 tx=0x7faddc00cc30 comp rx=0 tx=0).stop 2023-12-20T04:29:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 -- 172.21.15.32:0/3158020282 shutdown_connections 2023-12-20T04:29:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fadc8065a60 0x7fadc8067f20 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fade415d660 0x7fade40c46e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fade4157480 0x7fade40c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 --2- 172.21.15.32:0/3158020282 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fade40b0760 0x7fade4151060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 -- 172.21.15.32:0/3158020282 >> 172.21.15.32:0/3158020282 conn(0x7fade406a510 msgr2=0x7fade4151ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:03.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 -- 172.21.15.32:0/3158020282 shutdown_connections 2023-12-20T04:29:03.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.323+0000 7fadeb20b640 1 -- 172.21.15.32:0/3158020282 wait complete. 2023-12-20T04:29:03.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:29:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f5335118640 1 Processor -- start 2023-12-20T04:29:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f5335118640 1 -- start start 2023-12-20T04:29:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f5335118640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f53301591e0 0x7f53300caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f5335118640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f533006cdf0 0x7f5330070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f5335118640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53300cc3b0 con 0x7f5330070770 2023-12-20T04:29:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f5335118640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5330070ba0 con 0x7f533006cdf0 2023-12-20T04:29:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f5335118640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53300c8e30 con 0x7f53301591e0 2023-12-20T04:29:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f532e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f533006cdf0 0x7f5330070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f532e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f533006cdf0 0x7f5330070230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47684/0 (socket says 172.21.15.32:47684) 2023-12-20T04:29:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f532e59c640 1 -- 172.21.15.32:0/2802596183 learned_addr learned my addr 172.21.15.32:0/2802596183 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f532ed9d640 1 --2- 172.21.15.32:0/2802596183 >> v2:172.21.15.39:3300/0 conn(0x7f53301591e0 0x7f53300caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f532e59c640 1 -- 172.21.15.32:0/2802596183 >> v2:172.21.15.39:3300/0 conn(0x7f53301591e0 msgr2=0x7f53300caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f532e59c640 1 --2- 172.21.15.32:0/2802596183 >> v2:172.21.15.39:3300/0 conn(0x7f53301591e0 0x7f53300caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f532e59c640 1 -- 172.21.15.32:0/2802596183 >> v1:172.21.15.39:6789/0 conn(0x7f5330070770 legacy=0x7f533015cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.491+0000 7f532e59c640 1 -- 172.21.15.32:0/2802596183 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f533006d280 con 0x7f533006cdf0 2023-12-20T04:29:03.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f532e59c640 1 --2- 172.21.15.32:0/2802596183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f533006cdf0 0x7f5330070230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f53300cf890 tx=0x7f532401d540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f598b9fa2560088 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:03.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f532dd9b640 1 -- 172.21.15.32:0/2802596183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53240ab610 con 0x7f533006cdf0 2023-12-20T04:29:03.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f532dd9b640 1 -- 172.21.15.32:0/2802596183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53240a7040 con 0x7f533006cdf0 2023-12-20T04:29:03.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f532dd9b640 1 -- 172.21.15.32:0/2802596183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53240ab990 con 0x7f533006cdf0 2023-12-20T04:29:03.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f5335118640 1 -- 172.21.15.32:0/2802596183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f533006cdf0 msgr2=0x7f5330070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f5335118640 1 --2- 172.21.15.32:0/2802596183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f533006cdf0 0x7f5330070230 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f53300cf890 tx=0x7f532401d540 comp rx=0 tx=0).stop 2023-12-20T04:29:03.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f5335118640 1 -- 172.21.15.32:0/2802596183 shutdown_connections 2023-12-20T04:29:03.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f5335118640 1 --2- 172.21.15.32:0/2802596183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f533006cdf0 0x7f5330070230 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f5335118640 1 --2- 172.21.15.32:0/2802596183 >> v2:172.21.15.39:3300/0 conn(0x7f53301591e0 0x7f53300caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f5335118640 1 -- 172.21.15.32:0/2802596183 >> 172.21.15.32:0/2802596183 conn(0x7f533006a510 msgr2=0x7f53300ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:03.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f5335118640 1 -- 172.21.15.32:0/2802596183 shutdown_connections 2023-12-20T04:29:03.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.495+0000 7f5335118640 1 -- 172.21.15.32:0/2802596183 wait complete. 2023-12-20T04:29:03.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f5335118640 1 Processor -- start 2023-12-20T04:29:03.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f5335118640 1 -- start start 2023-12-20T04:29:03.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f5335118640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5330070770 0x7f5330141420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f5335118640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53301591e0 0x7f533013ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f5335118640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5330167f30 0x7f53301668d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f5335118640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f533013f480 con 0x7f5330070770 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f5335118640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5330140380 con 0x7f5330167f30 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f5335118640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5330140660 con 0x7f53301591e0 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f532ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5330070770 0x7f5330141420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f532ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5330070770 0x7f5330141420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47688/0 (socket says 172.21.15.32:47688) 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f532ed9d640 1 -- 172.21.15.32:0/3307251385 learned_addr learned my addr 172.21.15.32:0/3307251385 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f532e59c640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53301591e0 0x7f533013ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f532f59e640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5330167f30 0x7f53301668d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f532ed9d640 1 -- 172.21.15.32:0/3307251385 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53301591e0 msgr2=0x7f533013ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f532ed9d640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53301591e0 0x7f533013ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f532ed9d640 1 -- 172.21.15.32:0/3307251385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5330167f30 msgr2=0x7f53301668d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f532ed9d640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5330167f30 0x7f53301668d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f532ed9d640 1 -- 172.21.15.32:0/3307251385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53240094d0 con 0x7f5330070770 2023-12-20T04:29:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f532ed9d640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5330070770 0x7f5330141420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f532000d5f0 tx=0x7f532000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f53177fe640 1 -- 172.21.15.32:0/3307251385 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53200125f0 con 0x7f5330070770 2023-12-20T04:29:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f53177fe640 1 -- 172.21.15.32:0/3307251385 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5320012750 con 0x7f5330070770 2023-12-20T04:29:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f53177fe640 1 -- 172.21.15.32:0/3307251385 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5320012ae0 con 0x7f5330070770 2023-12-20T04:29:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f5335118640 1 -- 172.21.15.32:0/3307251385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5330140940 con 0x7f5330070770 2023-12-20T04:29:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f5335118640 1 -- 172.21.15.32:0/3307251385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5330140f00 con 0x7f5330070770 2023-12-20T04:29:03.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.499+0000 7f5335118640 1 -- 172.21.15.32:0/3307251385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f533004e000 con 0x7f5330070770 2023-12-20T04:29:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.503+0000 7f53177fe640 1 -- 172.21.15.32:0/3307251385 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5320012c40 con 0x7f5330070770 2023-12-20T04:29:03.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.503+0000 7f53177fe640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f531c065a60 0x7f531c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.503+0000 7f53177fe640 1 -- 172.21.15.32:0/3307251385 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5320083790 con 0x7f5330070770 2023-12-20T04:29:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.503+0000 7f532e59c640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f531c065a60 0x7f531c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.503+0000 7f532e59c640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f531c065a60 0x7f531c067f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f53240af000 tx=0x7f5324008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:03.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.507+0000 7f53177fe640 1 -- 172.21.15.32:0/3307251385 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5320051020 con 0x7f5330070770 2023-12-20T04:29:03.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 -- 172.21.15.32:0/3307251385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f533014faa0 con 0x7f5330070770 2023-12-20T04:29:03.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f53177fe640 1 -- 172.21.15.32:0/3307251385 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5320007380 con 0x7f5330070770 2023-12-20T04:29:03.665 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 -- 172.21.15.32:0/3307251385 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f531c065a60 msgr2=0x7f531c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f531c065a60 0x7f531c067f20 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f53240af000 tx=0x7f5324008040 comp rx=0 tx=0).stop 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 -- 172.21.15.32:0/3307251385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5330070770 msgr2=0x7f5330141420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5330070770 0x7f5330141420 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f532000d5f0 tx=0x7f532000dab0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 -- 172.21.15.32:0/3307251385 shutdown_connections 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f531c065a60 0x7f531c067f20 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5330167f30 0x7f53301668d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53301591e0 0x7f533013ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 --2- 172.21.15.32:0/3307251385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5330070770 0x7f5330141420 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 -- 172.21.15.32:0/3307251385 >> 172.21.15.32:0/3307251385 conn(0x7f533006a510 msgr2=0x7f53300c4bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 -- 172.21.15.32:0/3307251385 shutdown_connections 2023-12-20T04:29:03.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.663+0000 7f5335118640 1 -- 172.21.15.32:0/3307251385 wait complete. 2023-12-20T04:29:03.686 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:29:03.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f064962a640 1 Processor -- start 2023-12-20T04:29:03.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f064962a640 1 -- start start 2023-12-20T04:29:03.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f064962a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f064415ccf0 0x7f06440cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f064962a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06440cd7b0 0x7f06440cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f064962a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06440cfd80 con 0x7f06440cf920 2023-12-20T04:29:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f064962a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06440ce2f0 con 0x7f064415ccf0 2023-12-20T04:29:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f064962a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06440d0b70 con 0x7f06440cd7b0 2023-12-20T04:29:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f0648e29640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f06440cf920 0x7f06440b4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57264/0 (socket says 172.21.15.32:57264) 2023-12-20T04:29:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f0648e29640 1 -- 172.21.15.32:0/1642538972 learned_addr learned my addr 172.21.15.32:0/1642538972 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f0643fff640 1 --2- 172.21.15.32:0/1642538972 >> v2:172.21.15.39:3300/0 conn(0x7f064415ccf0 0x7f06440cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f06437fe640 1 --2- 172.21.15.32:0/1642538972 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06440cd7b0 0x7f06440cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f0642ffd640 1 -- 172.21.15.32:0/1642538972 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184314401 0 0) 0x7f06440cfd80 con 0x7f06440cf920 2023-12-20T04:29:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.831+0000 7f0642ffd640 1 -- 172.21.15.32:0/1642538972 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f062c003530 con 0x7f06440cf920 2023-12-20T04:29:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.835+0000 7f0643fff640 1 -- 172.21.15.32:0/1642538972 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06440cd7b0 msgr2=0x7f06440cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.835+0000 7f0643fff640 1 --2- 172.21.15.32:0/1642538972 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06440cd7b0 0x7f06440cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.835+0000 7f0643fff640 1 -- 172.21.15.32:0/1642538972 >> v1:172.21.15.32:6789/0 conn(0x7f06440cf920 legacy=0x7f06440b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.835+0000 7f0643fff640 1 -- 172.21.15.32:0/1642538972 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06440d10f0 con 0x7f064415ccf0 2023-12-20T04:29:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.835+0000 7f06437fe640 1 --2- 172.21.15.32:0/1642538972 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06440cd7b0 0x7f06440cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.835+0000 7f0643fff640 1 --2- 172.21.15.32:0/1642538972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f064415ccf0 0x7f06440cd270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06340097c0 tx=0x7f063401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f01c3220d5d128f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.835+0000 7f0642ffd640 1 -- 172.21.15.32:0/1642538972 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06340ab600 con 0x7f064415ccf0 2023-12-20T04:29:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.835+0000 7f0642ffd640 1 -- 172.21.15.32:0/1642538972 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06340a7050 con 0x7f064415ccf0 2023-12-20T04:29:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.835+0000 7f0642ffd640 1 -- 172.21.15.32:0/1642538972 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06340ab980 con 0x7f064415ccf0 2023-12-20T04:29:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 -- 172.21.15.32:0/1642538972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f064415ccf0 msgr2=0x7f06440cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 --2- 172.21.15.32:0/1642538972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f064415ccf0 0x7f06440cd270 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f06340097c0 tx=0x7f063401c3c0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 -- 172.21.15.32:0/1642538972 shutdown_connections 2023-12-20T04:29:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 --2- 172.21.15.32:0/1642538972 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06440cd7b0 0x7f06440cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 --2- 172.21.15.32:0/1642538972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f064415ccf0 0x7f06440cd270 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 -- 172.21.15.32:0/1642538972 >> 172.21.15.32:0/1642538972 conn(0x7f06440658e0 msgr2=0x7f0644065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 -- 172.21.15.32:0/1642538972 shutdown_connections 2023-12-20T04:29:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 -- 172.21.15.32:0/1642538972 wait complete. 2023-12-20T04:29:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 Processor -- start 2023-12-20T04:29:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 -- start start 2023-12-20T04:29:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06440cd7b0 0x7f06441543b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06440cf920 0x7f06440cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f064415ccf0 0x7f06440cbfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06440cc500 con 0x7f06440cf920 2023-12-20T04:29:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06441548f0 con 0x7f064415ccf0 2023-12-20T04:29:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f064962a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06440c93a0 con 0x7f06440cd7b0 2023-12-20T04:29:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f06437fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06440cf920 0x7f06440cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f06437fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06440cf920 0x7f06440cba80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47702/0 (socket says 172.21.15.32:47702) 2023-12-20T04:29:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f06437fe640 1 -- 172.21.15.32:0/2700806324 learned_addr learned my addr 172.21.15.32:0/2700806324 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f0648e29640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f064415ccf0 0x7f06440cbfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.839+0000 7f0643fff640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06440cd7b0 0x7f06441543b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f06437fe640 1 -- 172.21.15.32:0/2700806324 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06440cd7b0 msgr2=0x7f06441543b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f06437fe640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06440cd7b0 0x7f06441543b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f06437fe640 1 -- 172.21.15.32:0/2700806324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f064415ccf0 msgr2=0x7f06440cbfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f06437fe640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f064415ccf0 0x7f06440cbfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f06437fe640 1 -- 172.21.15.32:0/2700806324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0638009420 con 0x7f06440cf920 2023-12-20T04:29:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f06437fe640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06440cf920 0x7f06440cba80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f063800e980 tx=0x7f063800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f0640ff9640 1 -- 172.21.15.32:0/2700806324 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f063800ca00 con 0x7f06440cf920 2023-12-20T04:29:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f0640ff9640 1 -- 172.21.15.32:0/2700806324 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f063800cb60 con 0x7f06440cf920 2023-12-20T04:29:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f064962a640 1 -- 172.21.15.32:0/2700806324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0634009420 con 0x7f06440cf920 2023-12-20T04:29:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f0640ff9640 1 -- 172.21.15.32:0/2700806324 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0638013470 con 0x7f06440cf920 2023-12-20T04:29:03.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f064962a640 1 -- 172.21.15.32:0/2700806324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06440c8b00 con 0x7f06440cf920 2023-12-20T04:29:03.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f064962a640 1 -- 172.21.15.32:0/2700806324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0644074100 con 0x7f06440cf920 2023-12-20T04:29:03.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f0640ff9640 1 -- 172.21.15.32:0/2700806324 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f06380135d0 con 0x7f06440cf920 2023-12-20T04:29:03.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f0640ff9640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0630065a60 0x7f0630067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:03.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f0640ff9640 1 -- 172.21.15.32:0/2700806324 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0638083370 con 0x7f06440cf920 2023-12-20T04:29:03.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.843+0000 7f0643fff640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0630065a60 0x7f0630067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:03.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.847+0000 7f0643fff640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0630065a60 0x7f0630067f20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f06340ad000 tx=0x7f0634008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:03.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:03.847+0000 7f0640ff9640 1 -- 172.21.15.32:0/2700806324 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0638050dd0 con 0x7f06440cf920 2023-12-20T04:29:04.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 -- 172.21.15.32:0/2700806324 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f06440d0720 con 0x7f06440cf920 2023-12-20T04:29:04.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f0640ff9640 1 -- 172.21.15.32:0/2700806324 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f06380543f0 con 0x7f06440cf920 2023-12-20T04:29:04.008 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:29:04.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 -- 172.21.15.32:0/2700806324 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0630065a60 msgr2=0x7f0630067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0630065a60 0x7f0630067f20 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f06340ad000 tx=0x7f0634008040 comp rx=0 tx=0).stop 2023-12-20T04:29:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 -- 172.21.15.32:0/2700806324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06440cf920 msgr2=0x7f06440cba80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06440cf920 0x7f06440cba80 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f063800e980 tx=0x7f063800ee40 comp rx=0 tx=0).stop 2023-12-20T04:29:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 -- 172.21.15.32:0/2700806324 shutdown_connections 2023-12-20T04:29:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0630065a60 0x7f0630067f20 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f064415ccf0 0x7f06440cbfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06440cf920 0x7f06440cba80 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 --2- 172.21.15.32:0/2700806324 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06440cd7b0 0x7f06441543b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 -- 172.21.15.32:0/2700806324 >> 172.21.15.32:0/2700806324 conn(0x7f06440658e0 msgr2=0x7f06440b6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 -- 172.21.15.32:0/2700806324 shutdown_connections 2023-12-20T04:29:04.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.007+0000 7f064962a640 1 -- 172.21.15.32:0/2700806324 wait complete. 2023-12-20T04:29:04.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:04.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.171+0000 7f6e98c2d640 1 Processor -- start 2023-12-20T04:29:04.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.171+0000 7f6e98c2d640 1 -- start start 2023-12-20T04:29:04.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e98c2d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6e9406ce60 0x7f6e9415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e98c2d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9415bbd0 con 0x7f6e940af5e0 2023-12-20T04:29:04.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e98c2d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e9415beb0 con 0x7f6e9415f110 2023-12-20T04:29:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e98c2d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e9415e910 con 0x7f6e9406ce60 2023-12-20T04:29:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6e940af5e0 0x7f6e940af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57270/0 (socket says 172.21.15.32:57270) 2023-12-20T04:29:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9259c640 1 -- 172.21.15.32:0/1369069579 learned_addr learned my addr 172.21.15.32:0/1369069579 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e91d9b640 1 --2- 172.21.15.32:0/1369069579 >> v2:172.21.15.39:3300/0 conn(0x7f6e9406ce60 0x7f6e9415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3477005119 0 0) 0x7f6e9415bbd0 con 0x7f6e940af5e0 2023-12-20T04:29:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e7c003530 con 0x7f6e940af5e0 2023-12-20T04:29:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 780208802 0 0) 0x7f6e9415beb0 con 0x7f6e9415f110 2023-12-20T04:29:04.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e9415bbd0 con 0x7f6e9415f110 2023-12-20T04:29:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1258577193 0 0) 0x7f6e7c003530 con 0x7f6e940af5e0 2023-12-20T04:29:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 >> v2:172.21.15.39:3300/0 conn(0x7f6e9406ce60 msgr2=0x7f6e9415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 --2- 172.21.15.32:0/1369069579 >> v2:172.21.15.39:3300/0 conn(0x7f6e9406ce60 0x7f6e9415b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 >> v1:172.21.15.39:6789/0 conn(0x7f6e9415f110 legacy=0x7f6e9415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e9415ebf0 con 0x7f6e940af5e0 2023-12-20T04:29:04.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f6e84001800 con 0x7f6e940af5e0 2023-12-20T04:29:04.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 >> v1:172.21.15.32:6789/0 conn(0x7f6e940af5e0 legacy=0x7f6e940af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7c003b10 0x7f6e7c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e7c0047d0 0x7f6e7c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e7c005190 0x7f6e7c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9415e910 con 0x7f6e7c0047d0 2023-12-20T04:29:04.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c00c0c0 con 0x7f6e7c003b10 2023-12-20T04:29:04.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e7c00c280 con 0x7f6e7c005190 2023-12-20T04:29:04.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e91d9b640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e7c0047d0 0x7f6e7c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9259c640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7c003b10 0x7f6e7c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e92d9d640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e7c005190 0x7f6e7c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e91d9b640 1 -- 172.21.15.32:0/1369069579 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e7c005190 msgr2=0x7f6e7c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:04.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e91d9b640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e7c005190 0x7f6e7c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e91d9b640 1 -- 172.21.15.32:0/1369069579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7c003b10 msgr2=0x7f6e7c003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:04.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e91d9b640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7c003b10 0x7f6e7c003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e91d9b640 1 -- 172.21.15.32:0/1369069579 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7c003530 con 0x7f6e7c0047d0 2023-12-20T04:29:04.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e92d9d640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e7c005190 0x7f6e7c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:04.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e9259c640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7c003b10 0x7f6e7c003f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:04.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.175+0000 7f6e92d9d640 1 -- 172.21.15.32:0/1369069579 reap_dead start 2023-12-20T04:29:04.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.179+0000 7f6e91d9b640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e7c0047d0 0x7f6e7c004c50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009ae0 tx=0x7f6e8801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0301e438d684da server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.179+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880a64c0 con 0x7f6e7c0047d0 2023-12-20T04:29:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.179+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e880a6620 con 0x7f6e7c0047d0 2023-12-20T04:29:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.179+0000 7f6e9159a640 1 -- 172.21.15.32:0/1369069579 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880a69d0 con 0x7f6e7c0047d0 2023-12-20T04:29:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.179+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1369069579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e7c0047d0 msgr2=0x7f6e7c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.179+0000 7f6e98c2d640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e7c0047d0 0x7f6e7c004c50 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f6e88009ae0 tx=0x7f6e8801ca50 comp rx=0 tx=0).stop 2023-12-20T04:29:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.179+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1369069579 shutdown_connections 2023-12-20T04:29:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.179+0000 7f6e98c2d640 1 --2- 172.21.15.32:0/1369069579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e7c0047d0 0x7f6e7c004c50 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.179+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1369069579 >> 172.21.15.32:0/1369069579 conn(0x7f6e9406a320 msgr2=0x7f6e940ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:04.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.179+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1369069579 shutdown_connections 2023-12-20T04:29:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.179+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1369069579 wait complete. 2023-12-20T04:29:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e98c2d640 1 Processor -- start 2023-12-20T04:29:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e98c2d640 1 -- start start 2023-12-20T04:29:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e98c2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7c0047d0 0x7f6e940c1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e98c2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9415d490 0x7f6e9415d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e98c2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e940b3250 0x7f6e940b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e98c2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e940af0a0 con 0x7f6e9415d490 2023-12-20T04:29:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e98c2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9415de50 con 0x7f6e7c0047d0 2023-12-20T04:29:04.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e98c2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e9414c480 con 0x7f6e940b3250 2023-12-20T04:29:04.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e91d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9415d490 0x7f6e9415d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e91d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9415d490 0x7f6e9415d910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47724/0 (socket says 172.21.15.32:47724) 2023-12-20T04:29:04.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e91d9b640 1 -- 172.21.15.32:0/1796376202 learned_addr learned my addr 172.21.15.32:0/1796376202 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:04.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e92d9d640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e940b3250 0x7f6e940b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e9259c640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7c0047d0 0x7f6e940c1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e91d9b640 1 -- 172.21.15.32:0/1796376202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e940b3250 msgr2=0x7f6e940b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e91d9b640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e940b3250 0x7f6e940b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e91d9b640 1 -- 172.21.15.32:0/1796376202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7c0047d0 msgr2=0x7f6e940c1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e91d9b640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7c0047d0 0x7f6e940c1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e91d9b640 1 -- 172.21.15.32:0/1796376202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e940701c0 con 0x7f6e9415d490 2023-12-20T04:29:04.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e9259c640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7c0047d0 0x7f6e940c1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:04.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e91d9b640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9415d490 0x7f6e9415d910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6e880b0040 tx=0x7f6e880197f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:04.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e82ffd640 1 -- 172.21.15.32:0/1796376202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880a64c0 con 0x7f6e9415d490 2023-12-20T04:29:04.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e82ffd640 1 -- 172.21.15.32:0/1796376202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e880a6790 con 0x7f6e9415d490 2023-12-20T04:29:04.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e82ffd640 1 -- 172.21.15.32:0/1796376202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e880a6b00 con 0x7f6e9415d490 2023-12-20T04:29:04.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1796376202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e940b3c10 con 0x7f6e9415d490 2023-12-20T04:29:04.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1796376202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e94070410 con 0x7f6e9415d490 2023-12-20T04:29:04.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.183+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1796376202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e9404e000 con 0x7f6e9415d490 2023-12-20T04:29:04.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.187+0000 7f6e82ffd640 1 -- 172.21.15.32:0/1796376202 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6e880a6c60 con 0x7f6e9415d490 2023-12-20T04:29:04.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.187+0000 7f6e82ffd640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e74065a60 0x7f6e74067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.187+0000 7f6e82ffd640 1 -- 172.21.15.32:0/1796376202 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e880aa070 con 0x7f6e9415d490 2023-12-20T04:29:04.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.187+0000 7f6e9259c640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e74065a60 0x7f6e74067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.187+0000 7f6e9259c640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e74065a60 0x7f6e74067f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6e84001390 tx=0x7f6e84009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:04.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.191+0000 7f6e82ffd640 1 -- 172.21.15.32:0/1796376202 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6e880eb400 con 0x7f6e9415d490 2023-12-20T04:29:04.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.351+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1796376202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6e940cf450 con 0x7f6e9415d490 2023-12-20T04:29:04.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e82ffd640 1 -- 172.21.15.32:0/1796376202 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f6e880eea20 con 0x7f6e9415d490 2023-12-20T04:29:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:04.355 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1796376202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e74065a60 msgr2=0x7f6e74067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e74065a60 0x7f6e74067f20 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f6e84001390 tx=0x7f6e84009040 comp rx=0 tx=0).stop 2023-12-20T04:29:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1796376202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9415d490 msgr2=0x7f6e9415d910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9415d490 0x7f6e9415d910 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f6e880b0040 tx=0x7f6e880197f0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1796376202 shutdown_connections 2023-12-20T04:29:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e74065a60 0x7f6e74067f20 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e940b3250 0x7f6e940b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e9415d490 0x7f6e9415d910 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 --2- 172.21.15.32:0/1796376202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7c0047d0 0x7f6e940c1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1796376202 >> 172.21.15.32:0/1796376202 conn(0x7f6e9406a320 msgr2=0x7f6e94157e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1796376202 shutdown_connections 2023-12-20T04:29:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.355+0000 7f6e98c2d640 1 -- 172.21.15.32:0/1796376202 wait complete. 2023-12-20T04:29:04.377 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:28:36.040195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:28:37.364671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:29:04.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:29:04.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.519+0000 7fc60f58f640 1 Processor -- start 2023-12-20T04:29:04.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60f58f640 1 -- start start 2023-12-20T04:29:04.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60f58f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc608157480 0x7fc6080c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60f58f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6080cde20 con 0x7fc6080cd4c0 2023-12-20T04:29:04.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60f58f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6080c8e30 con 0x7fc6080b0760 2023-12-20T04:29:04.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60f58f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc60815e480 con 0x7fc608157480 2023-12-20T04:29:04.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60d32b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc608157480 0x7fc6080c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60db2c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc6080cd4c0 0x7fc60815dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57282/0 (socket says 172.21.15.32:57282) 2023-12-20T04:29:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60d32b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc608157480 0x7fc6080c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47734/0 (socket says 172.21.15.32:47734) 2023-12-20T04:29:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60db2c640 1 -- 172.21.15.32:0/514705415 learned_addr learned my addr 172.21.15.32:0/514705415 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60d32b640 1 -- 172.21.15.32:0/514705415 >> v1:172.21.15.39:6789/0 conn(0x7fc6080b0760 legacy=0x7fc60815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60d32b640 1 -- 172.21.15.32:0/514705415 >> v1:172.21.15.32:6789/0 conn(0x7fc6080cd4c0 legacy=0x7fc60815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc60d32b640 1 -- 172.21.15.32:0/514705415 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6080ce070 con 0x7fc608157480 2023-12-20T04:29:04.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.523+0000 7fc5fffff640 1 -- 172.21.15.32:0/514705415 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 142802014 0 0) 0x7fc6080cde20 con 0x7fc6080cd4c0 2023-12-20T04:29:04.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc60d32b640 1 --2- 172.21.15.32:0/514705415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc608157480 0x7fc6080c9840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc60400ba60 tx=0x7fc60401ea90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1990a6876907d58b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:04.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc5fffff640 1 -- 172.21.15.32:0/514705415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6040a7a60 con 0x7fc608157480 2023-12-20T04:29:04.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc5fffff640 1 -- 172.21.15.32:0/514705415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6040a7bc0 con 0x7fc608157480 2023-12-20T04:29:04.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc5fffff640 1 -- 172.21.15.32:0/514705415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6040af440 con 0x7fc608157480 2023-12-20T04:29:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc60f58f640 1 -- 172.21.15.32:0/514705415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc608157480 msgr2=0x7fc6080c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc60f58f640 1 --2- 172.21.15.32:0/514705415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc608157480 0x7fc6080c9840 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc60400ba60 tx=0x7fc60401ea90 comp rx=0 tx=0).stop 2023-12-20T04:29:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc60f58f640 1 -- 172.21.15.32:0/514705415 shutdown_connections 2023-12-20T04:29:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc60f58f640 1 --2- 172.21.15.32:0/514705415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc608157480 0x7fc6080c9840 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc60f58f640 1 -- 172.21.15.32:0/514705415 >> 172.21.15.32:0/514705415 conn(0x7fc60806a510 msgr2=0x7fc6080ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc60f58f640 1 -- 172.21.15.32:0/514705415 shutdown_connections 2023-12-20T04:29:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc60f58f640 1 -- 172.21.15.32:0/514705415 wait complete. 2023-12-20T04:29:04.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc60f58f640 1 Processor -- start 2023-12-20T04:29:04.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.527+0000 7fc60f58f640 1 -- start start 2023-12-20T04:29:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60f58f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6080b0760 0x7fc60815a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60f58f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc6080cd4c0 0x7fc608145540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60f58f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc608157480 0x7fc6080c8270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60f58f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6081f9290 con 0x7fc608157480 2023-12-20T04:29:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60f58f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc608145a80 con 0x7fc6080b0760 2023-12-20T04:29:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60f58f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc60815aa70 con 0x7fc6080cd4c0 2023-12-20T04:29:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60db2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc608157480 0x7fc6080c8270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60db2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc608157480 0x7fc6080c8270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47736/0 (socket says 172.21.15.32:47736) 2023-12-20T04:29:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60db2c640 1 -- 172.21.15.32:0/388690185 learned_addr learned my addr 172.21.15.32:0/388690185 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60d32b640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6080b0760 0x7fc60815a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60cb2a640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc6080cd4c0 0x7fc608145540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60db2c640 1 -- 172.21.15.32:0/388690185 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc6080cd4c0 msgr2=0x7fc608145540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60db2c640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc6080cd4c0 0x7fc608145540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60db2c640 1 -- 172.21.15.32:0/388690185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6080b0760 msgr2=0x7fc60815a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60db2c640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6080b0760 0x7fc60815a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60db2c640 1 -- 172.21.15.32:0/388690185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc60400b6c0 con 0x7fc608157480 2023-12-20T04:29:04.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60db2c640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc608157480 0x7fc6080c8270 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc600001570 tx=0x7fc60000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:04.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc5fdffb640 1 -- 172.21.15.32:0/388690185 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60000a640 con 0x7fc608157480 2023-12-20T04:29:04.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc5fdffb640 1 -- 172.21.15.32:0/388690185 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc60000a7c0 con 0x7fc608157480 2023-12-20T04:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc5fdffb640 1 -- 172.21.15.32:0/388690185 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc60000ab70 con 0x7fc608157480 2023-12-20T04:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60f58f640 1 -- 172.21.15.32:0/388690185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6081f88f0 con 0x7fc608157480 2023-12-20T04:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60f58f640 1 -- 172.21.15.32:0/388690185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6081f8eb0 con 0x7fc608157480 2023-12-20T04:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc60f58f640 1 -- 172.21.15.32:0/388690185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6081f93d0 con 0x7fc608157480 2023-12-20T04:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.531+0000 7fc5fdffb640 1 -- 172.21.15.32:0/388690185 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc60000acd0 con 0x7fc608157480 2023-12-20T04:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.535+0000 7fc5fdffb640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5ec065a60 0x7fc5ec067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.535+0000 7fc5fdffb640 1 -- 172.21.15.32:0/388690185 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc600082760 con 0x7fc608157480 2023-12-20T04:29:04.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.535+0000 7fc60d32b640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5ec065a60 0x7fc5ec067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.535+0000 7fc60d32b640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5ec065a60 0x7fc5ec067f20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc6040b1000 tx=0x7fc604008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:04.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.539+0000 7fc5fdffb640 1 -- 172.21.15.32:0/388690185 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc6081f93d0 con 0x7fc608157480 2023-12-20T04:29:04.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.695+0000 7fc60f58f640 1 -- 172.21.15.32:0/388690185 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6081f93d0 con 0x7fc608157480 2023-12-20T04:29:04.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.695+0000 7fc5fdffb640 1 -- 172.21.15.32:0/388690185 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6081f93d0 con 0x7fc608157480 2023-12-20T04:29:04.698 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:29:04.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 -- 172.21.15.32:0/388690185 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5ec065a60 msgr2=0x7fc5ec067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5ec065a60 0x7fc5ec067f20 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fc6040b1000 tx=0x7fc604008040 comp rx=0 tx=0).stop 2023-12-20T04:29:04.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 -- 172.21.15.32:0/388690185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc608157480 msgr2=0x7fc6080c8270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc608157480 0x7fc6080c8270 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7fc600001570 tx=0x7fc60000cca0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 -- 172.21.15.32:0/388690185 shutdown_connections 2023-12-20T04:29:04.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5ec065a60 0x7fc5ec067f20 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc608157480 0x7fc6080c8270 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc6080cd4c0 0x7fc608145540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 --2- 172.21.15.32:0/388690185 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6080b0760 0x7fc60815a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 -- 172.21.15.32:0/388690185 >> 172.21.15.32:0/388690185 conn(0x7fc60806a510 msgr2=0x7fc6080ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:04.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 -- 172.21.15.32:0/388690185 shutdown_connections 2023-12-20T04:29:04.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.699+0000 7fc60f58f640 1 -- 172.21.15.32:0/388690185 wait complete. 2023-12-20T04:29:04.720 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:29:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.863+0000 7f855ab0a640 1 Processor -- start 2023-12-20T04:29:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.863+0000 7f855ab0a640 1 -- start start 2023-12-20T04:29:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f855ab0a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f85541609d0 0x7f8554160e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f855ab0a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85540cb870 0x7f85540cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f855ab0a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85541607c0 con 0x7f85541603e0 2023-12-20T04:29:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f855ab0a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85540d0540 con 0x7f85540cb870 2023-12-20T04:29:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f855ab0a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85540cecd0 con 0x7f85541609d0 2023-12-20T04:29:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f85590a7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85540cb870 0x7f85540cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f85590a7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85540cb870 0x7f85540cbc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47742/0 (socket says 172.21.15.32:47742) 2023-12-20T04:29:04.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f85590a7640 1 -- 172.21.15.32:0/3587110638 learned_addr learned my addr 172.21.15.32:0/3587110638 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f8553fff640 1 --2- 172.21.15.32:0/3587110638 >> v2:172.21.15.39:3300/0 conn(0x7f85541609d0 0x7f8554160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f85590a7640 1 -- 172.21.15.32:0/3587110638 >> v2:172.21.15.39:3300/0 conn(0x7f85541609d0 msgr2=0x7f8554160e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f85590a7640 1 --2- 172.21.15.32:0/3587110638 >> v2:172.21.15.39:3300/0 conn(0x7f85541609d0 0x7f8554160e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f85590a7640 1 -- 172.21.15.32:0/3587110638 >> v1:172.21.15.39:6789/0 conn(0x7f85541603e0 legacy=0x7f85540c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.867+0000 7f85590a7640 1 -- 172.21.15.32:0/3587110638 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85540cff80 con 0x7f85540cb870 2023-12-20T04:29:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f85590a7640 1 --2- 172.21.15.32:0/3587110638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85540cb870 0x7f85540cbc80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f85540ce690 tx=0x7f854c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392adb0b747af1c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f85537fe640 1 -- 172.21.15.32:0/3587110638 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c0a57d0 con 0x7f85540cb870 2023-12-20T04:29:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f85537fe640 1 -- 172.21.15.32:0/3587110638 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f854c0a5930 con 0x7f85540cb870 2023-12-20T04:29:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f85537fe640 1 -- 172.21.15.32:0/3587110638 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f854c0a5cc0 con 0x7f85540cb870 2023-12-20T04:29:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f855ab0a640 1 -- 172.21.15.32:0/3587110638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85540cb870 msgr2=0x7f85540cbc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f855ab0a640 1 --2- 172.21.15.32:0/3587110638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85540cb870 0x7f85540cbc80 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f85540ce690 tx=0x7f854c01c910 comp rx=0 tx=0).stop 2023-12-20T04:29:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f855ab0a640 1 -- 172.21.15.32:0/3587110638 shutdown_connections 2023-12-20T04:29:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f855ab0a640 1 --2- 172.21.15.32:0/3587110638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85540cb870 0x7f85540cbc80 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f855ab0a640 1 --2- 172.21.15.32:0/3587110638 >> v2:172.21.15.39:3300/0 conn(0x7f85541609d0 0x7f8554160e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f855ab0a640 1 -- 172.21.15.32:0/3587110638 >> 172.21.15.32:0/3587110638 conn(0x7f8554098c20 msgr2=0x7f85540cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f855ab0a640 1 -- 172.21.15.32:0/3587110638 shutdown_connections 2023-12-20T04:29:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f855ab0a640 1 -- 172.21.15.32:0/3587110638 wait complete. 2023-12-20T04:29:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f855ab0a640 1 Processor -- start 2023-12-20T04:29:04.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.871+0000 7f855ab0a640 1 -- start start 2023-12-20T04:29:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f855ab0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85540cb870 0x7f8554154bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f855ab0a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85541603e0 0x7f855415ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f855ab0a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85541609d0 0x7f85540cd420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f855ab0a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855415c550 con 0x7f85540cb870 2023-12-20T04:29:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f855ab0a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f855415c830 con 0x7f85541609d0 2023-12-20T04:29:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f855ab0a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f855415cb10 con 0x7f85541603e0 2023-12-20T04:29:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85588a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85540cb870 0x7f8554154bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85588a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85540cb870 0x7f8554154bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47748/0 (socket says 172.21.15.32:47748) 2023-12-20T04:29:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85588a6640 1 -- 172.21.15.32:0/613183988 learned_addr learned my addr 172.21.15.32:0/613183988 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f8553fff640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85541603e0 0x7f855415ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85590a7640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85541609d0 0x7f85540cd420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85588a6640 1 -- 172.21.15.32:0/613183988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85541603e0 msgr2=0x7f855415ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85588a6640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85541603e0 0x7f855415ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85588a6640 1 -- 172.21.15.32:0/613183988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85541609d0 msgr2=0x7f85540cd420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85588a6640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85541609d0 0x7f85540cd420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85588a6640 1 -- 172.21.15.32:0/613183988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f854c009b30 con 0x7f85540cb870 2023-12-20T04:29:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f8553fff640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85541603e0 0x7f855415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85590a7640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85541609d0 0x7f85540cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85588a6640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85540cb870 0x7f8554154bc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f854800c450 tx=0x7f854800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85517fa640 1 -- 172.21.15.32:0/613183988 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85480052d0 con 0x7f85540cb870 2023-12-20T04:29:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85517fa640 1 -- 172.21.15.32:0/613183988 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8548005430 con 0x7f85540cb870 2023-12-20T04:29:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f855ab0a640 1 -- 172.21.15.32:0/613183988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f855413ece0 con 0x7f85540cb870 2023-12-20T04:29:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85517fa640 1 -- 172.21.15.32:0/613183988 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8548003890 con 0x7f85540cb870 2023-12-20T04:29:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f855ab0a640 1 -- 172.21.15.32:0/613183988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f855415bfb0 con 0x7f85540cb870 2023-12-20T04:29:04.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f855ab0a640 1 -- 172.21.15.32:0/613183988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f855413eb40 con 0x7f85540cb870 2023-12-20T04:29:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.875+0000 7f85517fa640 1 -- 172.21.15.32:0/613183988 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f85480039f0 con 0x7f85540cb870 2023-12-20T04:29:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.879+0000 7f85517fa640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8538065a60 0x7f8538067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.879+0000 7f85517fa640 1 -- 172.21.15.32:0/613183988 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8548082ba0 con 0x7f85540cb870 2023-12-20T04:29:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.879+0000 7f8553fff640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8538065a60 0x7f8538067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:04.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.879+0000 7f8553fff640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8538065a60 0x7f8538067f20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f855409d7d0 tx=0x7f8540008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:04.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:04.883+0000 7f85517fa640 1 -- 172.21.15.32:0/613183988 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f855413eb40 con 0x7f85540cb870 2023-12-20T04:29:05.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.039+0000 7f855ab0a640 1 -- 172.21.15.32:0/613183988 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f85541607c0 con 0x7f85540cb870 2023-12-20T04:29:05.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.039+0000 7f85517fa640 1 -- 172.21.15.32:0/613183988 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f85541607c0 con 0x7f85540cb870 2023-12-20T04:29:05.042 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:29:05.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 -- 172.21.15.32:0/613183988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8538065a60 msgr2=0x7f8538067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8538065a60 0x7f8538067f20 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f855409d7d0 tx=0x7f8540008040 comp rx=0 tx=0).stop 2023-12-20T04:29:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 -- 172.21.15.32:0/613183988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85540cb870 msgr2=0x7f8554154bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85540cb870 0x7f8554154bc0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f854800c450 tx=0x7f854800c910 comp rx=0 tx=0).stop 2023-12-20T04:29:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 -- 172.21.15.32:0/613183988 shutdown_connections 2023-12-20T04:29:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8538065a60 0x7f8538067f20 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85541609d0 0x7f85540cd420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85541603e0 0x7f855415ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 --2- 172.21.15.32:0/613183988 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85540cb870 0x7f8554154bc0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 -- 172.21.15.32:0/613183988 >> 172.21.15.32:0/613183988 conn(0x7f8554098c20 msgr2=0x7f85540ac7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 -- 172.21.15.32:0/613183988 shutdown_connections 2023-12-20T04:29:05.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.043+0000 7f855ab0a640 1 -- 172.21.15.32:0/613183988 wait complete. 2023-12-20T04:29:05.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:29:05.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.207+0000 7f0c28241640 1 Processor -- start 2023-12-20T04:29:05.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.207+0000 7f0c28241640 1 -- start start 2023-12-20T04:29:05.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c28241640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c20164670 0x7f0c200ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c28241640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c200cb0b0 con 0x7f0c2006da20 2023-12-20T04:29:05.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c28241640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c200ca810 con 0x7f0c20070230 2023-12-20T04:29:05.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c28241640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c200ccac0 con 0x7f0c20164670 2023-12-20T04:29:05.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c25fdd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c20164670 0x7f0c200ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c257dc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c2006da20 0x7f0c200cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57294/0 (socket says 172.21.15.32:57294) 2023-12-20T04:29:05.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c25fdd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c20164670 0x7f0c200ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47752/0 (socket says 172.21.15.32:47752) 2023-12-20T04:29:05.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c25fdd640 1 -- 172.21.15.32:0/716808652 learned_addr learned my addr 172.21.15.32:0/716808652 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:05.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c25fdd640 1 -- 172.21.15.32:0/716808652 >> v1:172.21.15.39:6789/0 conn(0x7f0c20070230 legacy=0x7f0c20070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:05.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c25fdd640 1 -- 172.21.15.32:0/716808652 >> v1:172.21.15.32:6789/0 conn(0x7f0c2006da20 legacy=0x7f0c200cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c25fdd640 1 -- 172.21.15.32:0/716808652 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c200caaf0 con 0x7f0c20164670 2023-12-20T04:29:05.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.211+0000 7f0c24fdb640 1 -- 172.21.15.32:0/716808652 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1418180994 0 0) 0x7f0c200cb0b0 con 0x7f0c2006da20 2023-12-20T04:29:05.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c25fdd640 1 --2- 172.21.15.32:0/716808652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c20164670 0x7f0c200ca1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009870 tx=0x7f0c1c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=addd12f9629fe827 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:05.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c24fdb640 1 -- 172.21.15.32:0/716808652 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0ac410 con 0x7f0c20164670 2023-12-20T04:29:05.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c24fdb640 1 -- 172.21.15.32:0/716808652 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c1c0ac570 con 0x7f0c20164670 2023-12-20T04:29:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c24fdb640 1 -- 172.21.15.32:0/716808652 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c1c0ac900 con 0x7f0c20164670 2023-12-20T04:29:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c28241640 1 -- 172.21.15.32:0/716808652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c20164670 msgr2=0x7f0c200ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c28241640 1 --2- 172.21.15.32:0/716808652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c20164670 0x7f0c200ca1a0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c009870 tx=0x7f0c1c01c3b0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c28241640 1 -- 172.21.15.32:0/716808652 shutdown_connections 2023-12-20T04:29:05.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c28241640 1 --2- 172.21.15.32:0/716808652 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c20164670 0x7f0c200ca1a0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c28241640 1 -- 172.21.15.32:0/716808652 >> 172.21.15.32:0/716808652 conn(0x7f0c2006a510 msgr2=0x7f0c200ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c28241640 1 -- 172.21.15.32:0/716808652 shutdown_connections 2023-12-20T04:29:05.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c28241640 1 -- 172.21.15.32:0/716808652 wait complete. 2023-12-20T04:29:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c28241640 1 Processor -- start 2023-12-20T04:29:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.215+0000 7f0c28241640 1 -- start start 2023-12-20T04:29:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c28241640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c2006da20 0x7f0c201596e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c28241640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c20070230 0x7f0c20165570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c28241640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c20164670 0x7f0c2015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c28241640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c20159c20 con 0x7f0c20070230 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c28241640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201fc5e0 con 0x7f0c20164670 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c28241640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c201fc8c0 con 0x7f0c2006da20 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c257dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c20070230 0x7f0c20165570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c257dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c20070230 0x7f0c20165570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47764/0 (socket says 172.21.15.32:47764) 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c257dc640 1 -- 172.21.15.32:0/4108169548 learned_addr learned my addr 172.21.15.32:0/4108169548 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c25fdd640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c2006da20 0x7f0c201596e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c267de640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c20164670 0x7f0c2015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c257dc640 1 -- 172.21.15.32:0/4108169548 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c2006da20 msgr2=0x7f0c201596e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c257dc640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c2006da20 0x7f0c201596e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c257dc640 1 -- 172.21.15.32:0/4108169548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c20164670 msgr2=0x7f0c2015fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c257dc640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c20164670 0x7f0c2015fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c257dc640 1 -- 172.21.15.32:0/4108169548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c1c0094d0 con 0x7f0c20070230 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c267de640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c20164670 0x7f0c2015fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c257dc640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c20070230 0x7f0c20165570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c480 tx=0x7f0c1000c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c167fc640 1 -- 172.21.15.32:0/4108169548 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c100074e0 con 0x7f0c20070230 2023-12-20T04:29:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c167fc640 1 -- 172.21.15.32:0/4108169548 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c10007660 con 0x7f0c20070230 2023-12-20T04:29:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c167fc640 1 -- 172.21.15.32:0/4108169548 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c10004a30 con 0x7f0c20070230 2023-12-20T04:29:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c28241640 1 -- 172.21.15.32:0/4108169548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c201fcba0 con 0x7f0c20070230 2023-12-20T04:29:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c28241640 1 -- 172.21.15.32:0/4108169548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c20165ab0 con 0x7f0c20070230 2023-12-20T04:29:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c28241640 1 -- 172.21.15.32:0/4108169548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c201fd3a0 con 0x7f0c20070230 2023-12-20T04:29:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.219+0000 7f0c167fc640 1 -- 172.21.15.32:0/4108169548 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0c10004b90 con 0x7f0c20070230 2023-12-20T04:29:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.223+0000 7f0c167fc640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c04065a60 0x7f0c04067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.223+0000 7f0c167fc640 1 -- 172.21.15.32:0/4108169548 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c100822b0 con 0x7f0c20070230 2023-12-20T04:29:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.223+0000 7f0c25fdd640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c04065a60 0x7f0c04067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.223+0000 7f0c25fdd640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c04065a60 0x7f0c04067f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0ae000 tx=0x7f0c1c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:05.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.227+0000 7f0c167fc640 1 -- 172.21.15.32:0/4108169548 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0c201fd3a0 con 0x7f0c20070230 2023-12-20T04:29:05.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 -- 172.21.15.32:0/4108169548 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f0c201fd3a0 con 0x7f0c20070230 2023-12-20T04:29:05.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c167fc640 1 -- 172.21.15.32:0/4108169548 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f0c201fd3a0 con 0x7f0c20070230 2023-12-20T04:29:05.388 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:29:05.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 -- 172.21.15.32:0/4108169548 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c04065a60 msgr2=0x7f0c04067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c04065a60 0x7f0c04067f20 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f0c1c0ae000 tx=0x7f0c1c008040 comp rx=0 tx=0).stop 2023-12-20T04:29:05.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 -- 172.21.15.32:0/4108169548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c20070230 msgr2=0x7f0c20165570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c20070230 0x7f0c20165570 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f0c1000c480 tx=0x7f0c1000c940 comp rx=0 tx=0).stop 2023-12-20T04:29:05.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 -- 172.21.15.32:0/4108169548 shutdown_connections 2023-12-20T04:29:05.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c04065a60 0x7f0c04067f20 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c20164670 0x7f0c2015fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c20070230 0x7f0c20165570 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 --2- 172.21.15.32:0/4108169548 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c2006da20 0x7f0c201596e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 -- 172.21.15.32:0/4108169548 >> 172.21.15.32:0/4108169548 conn(0x7f0c2006a510 msgr2=0x7f0c20162830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:05.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 -- 172.21.15.32:0/4108169548 shutdown_connections 2023-12-20T04:29:05.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.387+0000 7f0c28241640 1 -- 172.21.15.32:0/4108169548 wait complete. 2023-12-20T04:29:05.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:05.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cc20fc640 1 Processor -- start 2023-12-20T04:29:05.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cc20fc640 1 -- start start 2023-12-20T04:29:05.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cc20fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc15cdb0 0x7f1cbc0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cc20fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1cbc0d2740 0x7f1cbc0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cc20fc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1cbc0b6ca0 con 0x7f1cbc0d1120 2023-12-20T04:29:05.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cc20fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1cbc15d190 con 0x7f1cbc0d2740 2023-12-20T04:29:05.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cc20fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc0d16a0 con 0x7f1cbc15cdb0 2023-12-20T04:29:05.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cbb7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc15cdb0 0x7f1cbc0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cbb7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc15cdb0 0x7f1cbc0d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:42732/0 (socket says 172.21.15.32:42732) 2023-12-20T04:29:05.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cbb7fe640 1 -- 172.21.15.32:0/3538426322 learned_addr learned my addr 172.21.15.32:0/3538426322 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:05.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cbaffd640 1 --2- 172.21.15.32:0/3538426322 >> v2:172.21.15.39:3300/0 conn(0x7f1cbc0d2740 0x7f1cbc0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cbb7fe640 1 -- 172.21.15.32:0/3538426322 >> v2:172.21.15.39:3300/0 conn(0x7f1cbc0d2740 msgr2=0x7f1cbc0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:05.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cbb7fe640 1 --2- 172.21.15.32:0/3538426322 >> v2:172.21.15.39:3300/0 conn(0x7f1cbc0d2740 0x7f1cbc0cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cbb7fe640 1 -- 172.21.15.32:0/3538426322 >> v1:172.21.15.39:6789/0 conn(0x7f1cbc0d1120 legacy=0x7f1cbc0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.551+0000 7f1cbb7fe640 1 -- 172.21.15.32:0/3538426322 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cbc0d1980 con 0x7f1cbc15cdb0 2023-12-20T04:29:05.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cbb7fe640 1 --2- 172.21.15.32:0/3538426322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc15cdb0 0x7f1cbc0d0be0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009870 tx=0x7f1cac01ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3096157a58609a0c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:05.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cba7fc640 1 -- 172.21.15.32:0/3538426322 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0a8a30 con 0x7f1cbc15cdb0 2023-12-20T04:29:05.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cba7fc640 1 -- 172.21.15.32:0/3538426322 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cac0a8b90 con 0x7f1cbc15cdb0 2023-12-20T04:29:05.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cba7fc640 1 -- 172.21.15.32:0/3538426322 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cac0b0440 con 0x7f1cbc15cdb0 2023-12-20T04:29:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cc20fc640 1 -- 172.21.15.32:0/3538426322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc15cdb0 msgr2=0x7f1cbc0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cc20fc640 1 --2- 172.21.15.32:0/3538426322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc15cdb0 0x7f1cbc0d0be0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f1cac009870 tx=0x7f1cac01ed30 comp rx=0 tx=0).stop 2023-12-20T04:29:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cc20fc640 1 -- 172.21.15.32:0/3538426322 shutdown_connections 2023-12-20T04:29:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cc20fc640 1 --2- 172.21.15.32:0/3538426322 >> v2:172.21.15.39:3300/0 conn(0x7f1cbc0d2740 0x7f1cbc0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cc20fc640 1 --2- 172.21.15.32:0/3538426322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc15cdb0 0x7f1cbc0d0be0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cc20fc640 1 -- 172.21.15.32:0/3538426322 >> 172.21.15.32:0/3538426322 conn(0x7f1cbc065ee0 msgr2=0x7f1cbc06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cc20fc640 1 -- 172.21.15.32:0/3538426322 shutdown_connections 2023-12-20T04:29:05.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.555+0000 7f1cc20fc640 1 -- 172.21.15.32:0/3538426322 wait complete. 2023-12-20T04:29:05.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cc20fc640 1 Processor -- start 2023-12-20T04:29:05.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cc20fc640 1 -- start start 2023-12-20T04:29:05.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cc20fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc0d1120 0x7f1cbc0cc1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cc20fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc0d2740 0x7f1cbc15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cc20fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15cdb0 0x7f1cbc15f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cc20fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc14ea60 con 0x7f1cbc0d2740 2023-12-20T04:29:05.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cc20fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc15d810 con 0x7f1cbc15cdb0 2023-12-20T04:29:05.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cc20fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1cbc15dab0 con 0x7f1cbc0d1120 2023-12-20T04:29:05.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cbaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc0d2740 0x7f1cbc15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cbaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc0d2740 0x7f1cbc15d2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47780/0 (socket says 172.21.15.32:47780) 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cbaffd640 1 -- 172.21.15.32:0/162725802 learned_addr learned my addr 172.21.15.32:0/162725802 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cbb7fe640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc0d1120 0x7f1cbc0cc1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cbbfff640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15cdb0 0x7f1cbc15f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cbaffd640 1 -- 172.21.15.32:0/162725802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc0d1120 msgr2=0x7f1cbc0cc1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cbaffd640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc0d1120 0x7f1cbc0cc1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cbaffd640 1 -- 172.21.15.32:0/162725802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15cdb0 msgr2=0x7f1cbc15f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cbaffd640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15cdb0 0x7f1cbc15f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cbaffd640 1 -- 172.21.15.32:0/162725802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1cac0094d0 con 0x7f1cbc0d2740 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cbaffd640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc0d2740 0x7f1cbc15d2d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000e930 tx=0x7f1cb000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1ca3fff640 1 -- 172.21.15.32:0/162725802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb0009690 con 0x7f1cbc0d2740 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1ca3fff640 1 -- 172.21.15.32:0/162725802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1cb000ca00 con 0x7f1cbc0d2740 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1ca3fff640 1 -- 172.21.15.32:0/162725802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1cb000cd70 con 0x7f1cbc0d2740 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cc20fc640 1 -- 172.21.15.32:0/162725802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1cbc15dd90 con 0x7f1cbc0d2740 2023-12-20T04:29:05.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.559+0000 7f1cc20fc640 1 -- 172.21.15.32:0/162725802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1cbc15e350 con 0x7f1cbc0d2740 2023-12-20T04:29:05.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.563+0000 7f1cc20fc640 1 -- 172.21.15.32:0/162725802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1cbc14ebd0 con 0x7f1cbc0d2740 2023-12-20T04:29:05.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.563+0000 7f1ca3fff640 1 -- 172.21.15.32:0/162725802 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1cb0013470 con 0x7f1cbc0d2740 2023-12-20T04:29:05.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.563+0000 7f1ca3fff640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ca8065a90 0x7f1ca8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.563+0000 7f1ca3fff640 1 -- 172.21.15.32:0/162725802 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1cb00832e0 con 0x7f1cbc0d2740 2023-12-20T04:29:05.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.563+0000 7f1cbb7fe640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ca8065a90 0x7f1ca8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.563+0000 7f1cbb7fe640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ca8065a90 0x7f1ca8067f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0b2000 tx=0x7f1cac0aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:05.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.567+0000 7f1ca3fff640 1 -- 172.21.15.32:0/162725802 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1cb0050e00 con 0x7f1cbc0d2740 2023-12-20T04:29:05.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.727+0000 7f1cc20fc640 1 -- 172.21.15.32:0/162725802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1cbc04e000 con 0x7f1cbc0d2740 2023-12-20T04:29:05.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.727+0000 7f1ca3fff640 1 -- 172.21.15.32:0/162725802 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f1cb0054420 con 0x7f1cbc0d2740 2023-12-20T04:29:05.729 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:05.729 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 -- 172.21.15.32:0/162725802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ca8065a90 msgr2=0x7f1ca8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ca8065a90 0x7f1ca8067f50 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f1cac0b2000 tx=0x7f1cac0aa070 comp rx=0 tx=0).stop 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 -- 172.21.15.32:0/162725802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc0d2740 msgr2=0x7f1cbc15d2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc0d2740 0x7f1cbc15d2d0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f1cb000e930 tx=0x7f1cb000edf0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 -- 172.21.15.32:0/162725802 shutdown_connections 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1ca8065a90 0x7f1ca8067f50 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1cbc15cdb0 0x7f1cbc15f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1cbc0d2740 0x7f1cbc15d2d0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 --2- 172.21.15.32:0/162725802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1cbc0d1120 0x7f1cbc0cc1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 -- 172.21.15.32:0/162725802 >> 172.21.15.32:0/162725802 conn(0x7f1cbc065ee0 msgr2=0x7f1cbc0b6350 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 -- 172.21.15.32:0/162725802 shutdown_connections 2023-12-20T04:29:05.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.731+0000 7f1cc20fc640 1 -- 172.21.15.32:0/162725802 wait complete. 2023-12-20T04:29:05.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:28:36.040195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:28:37.364671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:29:05.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:29:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.895+0000 7fd9bb98f640 1 Processor -- start 2023-12-20T04:29:05.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.895+0000 7fd9bb98f640 1 -- start start 2023-12-20T04:29:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.895+0000 7fd9bb98f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd9b40cb050 0x7fd9b40cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.895+0000 7fd9bb98f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b4065d40 0x7fd9b4161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.895+0000 7fd9bb98f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b406a1f0 con 0x7fd9b406ca60 2023-12-20T04:29:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.895+0000 7fd9bb98f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b4161860 con 0x7fd9b40cb050 2023-12-20T04:29:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.895+0000 7fd9bb98f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b406a440 con 0x7fd9b4065d40 2023-12-20T04:29:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.895+0000 7fd9b9f2c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd9b406ca60 0x7fd9b4070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57310/0 (socket says 172.21.15.32:57310) 2023-12-20T04:29:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.895+0000 7fd9b9f2c640 1 -- 172.21.15.32:0/3393175978 learned_addr learned my addr 172.21.15.32:0/3393175978 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.895+0000 7fd9b972b640 1 --2- 172.21.15.32:0/3393175978 >> v2:172.21.15.39:3300/0 conn(0x7fd9b40cb050 0x7fd9b40cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.895+0000 7fd9b8f2a640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b4065d40 0x7fd9b4161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3474834622 0 0) 0x7fd9b406a1f0 con 0x7fd9b406ca60 2023-12-20T04:29:05.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd9a0003530 con 0x7fd9b406ca60 2023-12-20T04:29:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3155554389 0 0) 0x7fd9a0003530 con 0x7fd9b406ca60 2023-12-20T04:29:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b4065d40 msgr2=0x7fd9b4161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b4065d40 0x7fd9b4161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 >> v2:172.21.15.39:3300/0 conn(0x7fd9b40cb050 msgr2=0x7fd9b40cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 --2- 172.21.15.32:0/3393175978 >> v2:172.21.15.39:3300/0 conn(0x7fd9b40cb050 0x7fd9b40cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b406a720 con 0x7fd9b406ca60 2023-12-20T04:29:05.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fd9ac001a00 con 0x7fd9b406ca60 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9b972b640 1 --2- 172.21.15.32:0/3393175978 >> v2:172.21.15.39:3300/0 conn(0x7fd9b40cb050 0x7fd9b40cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 >> v1:172.21.15.32:6789/0 conn(0x7fd9b406ca60 legacy=0x7fd9b4070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b4065d40 0x7fd9a0003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a0003f00 0x7fd9a00047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 0x7fd9a0005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b4161860 con 0x7fd9a0004d10 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b406a440 con 0x7fd9a0003f00 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9a000b990 con 0x7fd9b4065d40 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9b9f2c640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 0x7fd9a0005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9b972b640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b4065d40 0x7fd9a0003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9b8f2a640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a0003f00 0x7fd9a00047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9b9f2c640 1 -- 172.21.15.32:0/3393175978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b4065d40 msgr2=0x7fd9a0003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9b9f2c640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b4065d40 0x7fd9a0003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9b9f2c640 1 -- 172.21.15.32:0/3393175978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a0003f00 msgr2=0x7fd9a00047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9b9f2c640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9a0003f00 0x7fd9a00047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9b9f2c640 1 -- 172.21.15.32:0/3393175978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9a0003530 con 0x7fd9a0004d10 2023-12-20T04:29:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.899+0000 7fd9b9f2c640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 0x7fd9a0005190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00a860 tx=0x7fd9ac01d300 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c8ce100b81818a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0ad630 con 0x7fd9a0004d10 2023-12-20T04:29:05.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac0a8070 con 0x7fd9a0004d10 2023-12-20T04:29:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9abfff640 1 -- 172.21.15.32:0/3393175978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0ada30 con 0x7fd9a0004d10 2023-12-20T04:29:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 -- 172.21.15.32:0/3393175978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 msgr2=0x7fd9a0005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 --2- 172.21.15.32:0/3393175978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 0x7fd9a0005190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00a860 tx=0x7fd9ac01d300 comp rx=0 tx=0).stop 2023-12-20T04:29:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9b9f2c640 1 -- 172.21.15.32:0/3393175978 reap_dead start 2023-12-20T04:29:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 -- 172.21.15.32:0/3393175978 shutdown_connections 2023-12-20T04:29:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 -- 172.21.15.32:0/3393175978 >> 172.21.15.32:0/3393175978 conn(0x7fd9b40cf8d0 msgr2=0x7fd9b40cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 -- 172.21.15.32:0/3393175978 shutdown_connections 2023-12-20T04:29:05.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 -- 172.21.15.32:0/3393175978 wait complete. 2023-12-20T04:29:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 Processor -- start 2023-12-20T04:29:05.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 -- start start 2023-12-20T04:29:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 0x7fd9b41500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b40ad650 0x7fd9b40adab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b40ab8e0 0x7fd9b40abd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40c5e20 con 0x7fd9a0004d10 2023-12-20T04:29:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b41509c0 con 0x7fd9b40ad650 2023-12-20T04:29:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9bb98f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b40adff0 con 0x7fd9b40ab8e0 2023-12-20T04:29:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9b972b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 0x7fd9b41500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9b972b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 0x7fd9b41500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47810/0 (socket says 172.21.15.32:47810) 2023-12-20T04:29:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.903+0000 7fd9b972b640 1 -- 172.21.15.32:0/2693145871 learned_addr learned my addr 172.21.15.32:0/2693145871 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9b9f2c640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b40ab8e0 0x7fd9b40abd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9b8f2a640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b40ad650 0x7fd9b40adab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9b972b640 1 -- 172.21.15.32:0/2693145871 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b40ab8e0 msgr2=0x7fd9b40abd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9b972b640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b40ab8e0 0x7fd9b40abd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9b972b640 1 -- 172.21.15.32:0/2693145871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b40ad650 msgr2=0x7fd9b40adab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9b972b640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b40ad650 0x7fd9b40adab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9b972b640 1 -- 172.21.15.32:0/2693145871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b40ac2a0 con 0x7fd9a0004d10 2023-12-20T04:29:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9b972b640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 0x7fd9b41500a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0011cd0 tx=0x7fd9b000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9a9ffb640 1 -- 172.21.15.32:0/2693145871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b000fcc0 con 0x7fd9a0004d10 2023-12-20T04:29:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9a9ffb640 1 -- 172.21.15.32:0/2693145871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9b000fe20 con 0x7fd9a0004d10 2023-12-20T04:29:05.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9a9ffb640 1 -- 172.21.15.32:0/2693145871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9b0004240 con 0x7fd9a0004d10 2023-12-20T04:29:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9bb98f640 1 -- 172.21.15.32:0/2693145871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b40c70e0 con 0x7fd9a0004d10 2023-12-20T04:29:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9bb98f640 1 -- 172.21.15.32:0/2693145871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b40ac7d0 con 0x7fd9a0004d10 2023-12-20T04:29:05.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9bb98f640 1 -- 172.21.15.32:0/2693145871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b404e000 con 0x7fd9a0004d10 2023-12-20T04:29:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9a9ffb640 1 -- 172.21.15.32:0/2693145871 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd9b00043a0 con 0x7fd9a0004d10 2023-12-20T04:29:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9a9ffb640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd998065a90 0x7fd998067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.907+0000 7fd9a9ffb640 1 -- 172.21.15.32:0/2693145871 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd9b0083880 con 0x7fd9a0004d10 2023-12-20T04:29:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.911+0000 7fd9b8f2a640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd998065a90 0x7fd998067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:05.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.911+0000 7fd9b8f2a640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd998065a90 0x7fd998067f50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4150bf0 tx=0x7fd9a4009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:05.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:05.911+0000 7fd9a9ffb640 1 -- 172.21.15.32:0/2693145871 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd9b00512f0 con 0x7fd9a0004d10 2023-12-20T04:29:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.067+0000 7fd9bb98f640 1 -- 172.21.15.32:0/2693145871 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd9b40b4c90 con 0x7fd9a0004d10 2023-12-20T04:29:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9a9ffb640 1 -- 172.21.15.32:0/2693145871 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd9b0022070 con 0x7fd9a0004d10 2023-12-20T04:29:06.071 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:29:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 -- 172.21.15.32:0/2693145871 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd998065a90 msgr2=0x7fd998067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd998065a90 0x7fd998067f50 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7fd9b4150bf0 tx=0x7fd9a4009190 comp rx=0 tx=0).stop 2023-12-20T04:29:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 -- 172.21.15.32:0/2693145871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 msgr2=0x7fd9b41500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 0x7fd9b41500a0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0011cd0 tx=0x7fd9b000f610 comp rx=0 tx=0).stop 2023-12-20T04:29:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 -- 172.21.15.32:0/2693145871 shutdown_connections 2023-12-20T04:29:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd998065a90 0x7fd998067f50 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b40ab8e0 0x7fd9b40abd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b40ad650 0x7fd9b40adab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 --2- 172.21.15.32:0/2693145871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9a0004d10 0x7fd9b41500a0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 -- 172.21.15.32:0/2693145871 >> 172.21.15.32:0/2693145871 conn(0x7fd9b40cf8d0 msgr2=0x7fd9b415ec80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 -- 172.21.15.32:0/2693145871 shutdown_connections 2023-12-20T04:29:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.071+0000 7fd9bb98f640 1 -- 172.21.15.32:0/2693145871 wait complete. 2023-12-20T04:29:06.093 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:29:06.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.235+0000 7f1012241640 1 Processor -- start 2023-12-20T04:29:06.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f1012241640 1 -- start start 2023-12-20T04:29:06.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f1012241640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f100c0b0760 0x7f100c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f1012241640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100c0b1100 con 0x7f100c157480 2023-12-20T04:29:06.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f1012241640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f100c0cde20 con 0x7f100c0cc4b0 2023-12-20T04:29:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f1012241640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f100c0c9f50 con 0x7f100c0b0760 2023-12-20T04:29:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f100c157480 0x7f100c0c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57316/0 (socket says 172.21.15.32:57316) 2023-12-20T04:29:06.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100b7fe640 1 -- 172.21.15.32:0/314880702 learned_addr learned my addr 172.21.15.32:0/314880702 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:06.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100affd640 1 --2- 172.21.15.32:0/314880702 >> v2:172.21.15.39:3300/0 conn(0x7f100c0b0760 0x7f100c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4275528791 0 0) 0x7f100c0b1100 con 0x7f100c157480 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0ff4003530 con 0x7f100c157480 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1691496069 0 0) 0x7f0ff4003530 con 0x7f100c157480 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 >> v2:172.21.15.39:3300/0 conn(0x7f100c0b0760 msgr2=0x7f100c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 --2- 172.21.15.32:0/314880702 >> v2:172.21.15.39:3300/0 conn(0x7f100c0b0760 0x7f100c0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 >> v1:172.21.15.39:6789/0 conn(0x7f100c0cc4b0 legacy=0x7f100c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c0c8e30 con 0x7f100c157480 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f0ffc001680 con 0x7f100c157480 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 >> v1:172.21.15.32:6789/0 conn(0x7f100c157480 legacy=0x7f100c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 --2- 172.21.15.32:0/314880702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 0x7f0ff4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 --2- 172.21.15.32:0/314880702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ff4004380 0x7f0ff4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 --2- 172.21.15.32:0/314880702 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ff40050f0 0x7f0ff400ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c0c9f50 con 0x7f0ff4003980 2023-12-20T04:29:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff40104e0 con 0x7f0ff4004380 2023-12-20T04:29:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ff400bb90 con 0x7f0ff40050f0 2023-12-20T04:29:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100b7fe640 1 --2- 172.21.15.32:0/314880702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 0x7f0ff4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100affd640 1 --2- 172.21.15.32:0/314880702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ff4004380 0x7f0ff4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100bfff640 1 --2- 172.21.15.32:0/314880702 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ff40050f0 0x7f0ff400ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100b7fe640 1 -- 172.21.15.32:0/314880702 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ff40050f0 msgr2=0x7f0ff400ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100b7fe640 1 --2- 172.21.15.32:0/314880702 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ff40050f0 0x7f0ff400ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100b7fe640 1 -- 172.21.15.32:0/314880702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ff4004380 msgr2=0x7f0ff4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100b7fe640 1 --2- 172.21.15.32:0/314880702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ff4004380 0x7f0ff4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100b7fe640 1 -- 172.21.15.32:0/314880702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ff4003530 con 0x7f0ff4003980 2023-12-20T04:29:06.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.239+0000 7f100bfff640 1 -- 172.21.15.32:0/314880702 reap_dead start 2023-12-20T04:29:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.243+0000 7f100b7fe640 1 --2- 172.21.15.32:0/314880702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 0x7f0ff4003d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c190 tx=0x7f0ffc01cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a34138ce90eecbd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.243+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc005ae0 con 0x7f0ff4003980 2023-12-20T04:29:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.243+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc005c60 con 0x7f0ff4003980 2023-12-20T04:29:06.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.243+0000 7f100a7fc640 1 -- 172.21.15.32:0/314880702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0a5a30 con 0x7f0ff4003980 2023-12-20T04:29:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.243+0000 7f1012241640 1 -- 172.21.15.32:0/314880702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 msgr2=0x7f0ff4003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.243+0000 7f1012241640 1 --2- 172.21.15.32:0/314880702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 0x7f0ff4003d60 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc00c190 tx=0x7f0ffc01cb70 comp rx=0 tx=0).stop 2023-12-20T04:29:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.243+0000 7f1012241640 1 -- 172.21.15.32:0/314880702 shutdown_connections 2023-12-20T04:29:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.243+0000 7f1012241640 1 --2- 172.21.15.32:0/314880702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 0x7f0ff4003d60 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.243+0000 7f1012241640 1 -- 172.21.15.32:0/314880702 >> 172.21.15.32:0/314880702 conn(0x7f100c06a510 msgr2=0x7f100c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.243+0000 7f1012241640 1 -- 172.21.15.32:0/314880702 shutdown_connections 2023-12-20T04:29:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.243+0000 7f1012241640 1 -- 172.21.15.32:0/314880702 wait complete. 2023-12-20T04:29:06.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f1012241640 1 Processor -- start 2023-12-20T04:29:06.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f1012241640 1 -- start start 2023-12-20T04:29:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f1012241640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 0x7f100c1524d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f1012241640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c154340 0x7f100c158220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f1012241640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f100c158760 0x7f100c153780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f1012241640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c1501d0 con 0x7f0ff4003980 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f1012241640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f100c158be0 con 0x7f100c154340 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f1012241640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f100c158ec0 con 0x7f100c158760 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f100b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 0x7f100c1524d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f100b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 0x7f100c1524d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47826/0 (socket says 172.21.15.32:47826) 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f100b7fe640 1 -- 172.21.15.32:0/1142682328 learned_addr learned my addr 172.21.15.32:0/1142682328 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f100affd640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c154340 0x7f100c158220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f100bfff640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f100c158760 0x7f100c153780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f100b7fe640 1 -- 172.21.15.32:0/1142682328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f100c158760 msgr2=0x7f100c153780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f100b7fe640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f100c158760 0x7f100c153780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f100b7fe640 1 -- 172.21.15.32:0/1142682328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c154340 msgr2=0x7f100c158220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f100b7fe640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c154340 0x7f100c158220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f100b7fe640 1 -- 172.21.15.32:0/1142682328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f100c1591a0 con 0x7f0ff4003980 2023-12-20T04:29:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f100b7fe640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 0x7f100c1524d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc01c9a0 tx=0x7f0ff4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f0ff3fff640 1 -- 172.21.15.32:0/1142682328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0b4070 con 0x7f0ff4003980 2023-12-20T04:29:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f0ff3fff640 1 -- 172.21.15.32:0/1142682328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0ffc001480 con 0x7f0ff4003980 2023-12-20T04:29:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f0ff3fff640 1 -- 172.21.15.32:0/1142682328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0ffc0a5a10 con 0x7f0ff4003980 2023-12-20T04:29:06.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f1012241640 1 -- 172.21.15.32:0/1142682328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f100c159480 con 0x7f0ff4003980 2023-12-20T04:29:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.247+0000 7f1012241640 1 -- 172.21.15.32:0/1142682328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f100c153e90 con 0x7f0ff4003980 2023-12-20T04:29:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.251+0000 7f1012241640 1 -- 172.21.15.32:0/1142682328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f100c04e000 con 0x7f0ff4003980 2023-12-20T04:29:06.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.251+0000 7f0ff3fff640 1 -- 172.21.15.32:0/1142682328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0ffc0a5b70 con 0x7f0ff4003980 2023-12-20T04:29:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.251+0000 7f0ff3fff640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0ff8065ae0 0x7f0ff8067fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.251+0000 7f0ff3fff640 1 -- 172.21.15.32:0/1142682328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0ffc11cbe0 con 0x7f0ff4003980 2023-12-20T04:29:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.251+0000 7f100affd640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0ff8065ae0 0x7f0ff8067fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.251+0000 7f100affd640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0ff8065ae0 0x7f0ff8067fa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f100c1f49f0 tx=0x7f1000009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:06.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.255+0000 7f0ff3fff640 1 -- 172.21.15.32:0/1142682328 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0ffc0ea580 con 0x7f0ff4003980 2023-12-20T04:29:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.411+0000 7f1012241640 1 -- 172.21.15.32:0/1142682328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f100c150310 con 0x7f0ff4003980 2023-12-20T04:29:06.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.411+0000 7f0ff3fff640 1 -- 172.21.15.32:0/1142682328 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f0ffc0edba0 con 0x7f0ff4003980 2023-12-20T04:29:06.415 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:29:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 -- 172.21.15.32:0/1142682328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0ff8065ae0 msgr2=0x7f0ff8067fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0ff8065ae0 0x7f0ff8067fa0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f100c1f49f0 tx=0x7f1000009190 comp rx=0 tx=0).stop 2023-12-20T04:29:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 -- 172.21.15.32:0/1142682328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 msgr2=0x7f100c1524d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 0x7f100c1524d0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f0ffc01c9a0 tx=0x7f0ff4005800 comp rx=0 tx=0).stop 2023-12-20T04:29:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 -- 172.21.15.32:0/1142682328 shutdown_connections 2023-12-20T04:29:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0ff8065ae0 0x7f0ff8067fa0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f100c158760 0x7f100c153780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f100c154340 0x7f100c158220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 --2- 172.21.15.32:0/1142682328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ff4003980 0x7f100c1524d0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 -- 172.21.15.32:0/1142682328 >> 172.21.15.32:0/1142682328 conn(0x7f100c06a510 msgr2=0x7f100c0c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 -- 172.21.15.32:0/1142682328 shutdown_connections 2023-12-20T04:29:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.415+0000 7f1012241640 1 -- 172.21.15.32:0/1142682328 wait complete. 2023-12-20T04:29:06.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:29:06.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.579+0000 7fd101efc640 1 Processor -- start 2023-12-20T04:29:06.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.579+0000 7fd101efc640 1 -- start start 2023-12-20T04:29:06.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd101efc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd0fc0b0cb0 0x7fd0fc0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd101efc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd0fc070230 0x7fd0fc0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd101efc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd0fc15b6a0 con 0x7fd0fc15abb0 2023-12-20T04:29:06.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd101efc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd0fc06e160 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd101efc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc070c00 con 0x7fd0fc070230 2023-12-20T04:29:06.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd0faffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd0fc0b0cb0 0x7fd0fc0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd0fbfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd0fc070230 0x7fd0fc0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd0fb7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd0fc15abb0 0x7fd0fc15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:48308/0 (socket says 172.21.15.32:48308) 2023-12-20T04:29:06.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd0fb7fe640 1 -- 172.21.15.32:0/1351430498 learned_addr learned my addr 172.21.15.32:0/1351430498 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:06.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd0fa7fc640 1 -- 172.21.15.32:0/1351430498 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3833534021 0 0) 0x7fd0fc15b6a0 con 0x7fd0fc15abb0 2023-12-20T04:29:06.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd0fa7fc640 1 -- 172.21.15.32:0/1351430498 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd0e4003530 con 0x7fd0fc15abb0 2023-12-20T04:29:06.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd0faffd640 1 -- 172.21.15.32:0/1351430498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd0fc070230 msgr2=0x7fd0fc0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:06.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd0faffd640 1 --2- 172.21.15.32:0/1351430498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd0fc070230 0x7fd0fc0706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd0faffd640 1 -- 172.21.15.32:0/1351430498 >> v1:172.21.15.39:6789/0 conn(0x7fd0fc15abb0 legacy=0x7fd0fc15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd0faffd640 1 -- 172.21.15.32:0/1351430498 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0fc06d8c0 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.583+0000 7fd0fbfff640 1 --2- 172.21.15.32:0/1351430498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd0fc070230 0x7fd0fc0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:06.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd0faffd640 1 --2- 172.21.15.32:0/1351430498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0fc0b0cb0 0x7fd0fc0caaf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00097c0 tx=0x7fd0f001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=69598923c5108311 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:06.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd0fa7fc640 1 -- 172.21.15.32:0/1351430498 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00ab680 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd0fa7fc640 1 -- 172.21.15.32:0/1351430498 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f00ab7e0 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd0fa7fc640 1 -- 172.21.15.32:0/1351430498 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00abb70 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd101efc640 1 -- 172.21.15.32:0/1351430498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0fc0b0cb0 msgr2=0x7fd0fc0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd101efc640 1 --2- 172.21.15.32:0/1351430498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0fc0b0cb0 0x7fd0fc0caaf0 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00097c0 tx=0x7fd0f001c480 comp rx=0 tx=0).stop 2023-12-20T04:29:06.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd101efc640 1 -- 172.21.15.32:0/1351430498 shutdown_connections 2023-12-20T04:29:06.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd101efc640 1 --2- 172.21.15.32:0/1351430498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd0fc070230 0x7fd0fc0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd101efc640 1 --2- 172.21.15.32:0/1351430498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0fc0b0cb0 0x7fd0fc0caaf0 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd101efc640 1 -- 172.21.15.32:0/1351430498 >> 172.21.15.32:0/1351430498 conn(0x7fd0fc06a510 msgr2=0x7fd0fc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:06.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd101efc640 1 -- 172.21.15.32:0/1351430498 shutdown_connections 2023-12-20T04:29:06.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.587+0000 7fd101efc640 1 -- 172.21.15.32:0/1351430498 wait complete. 2023-12-20T04:29:06.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd101efc640 1 Processor -- start 2023-12-20T04:29:06.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd101efc640 1 -- start start 2023-12-20T04:29:06.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd101efc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0fc070230 0x7fd0fc137f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd101efc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0fc0b0cb0 0x7fd0fc152350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd101efc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd0fc15abb0 0x7fd0fc158900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd101efc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc150ac0 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd101efc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc1384e0 con 0x7fd0fc070230 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd101efc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd0fc1387c0 con 0x7fd0fc15abb0 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0faffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0fc0b0cb0 0x7fd0fc152350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0faffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0fc0b0cb0 0x7fd0fc152350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47836/0 (socket says 172.21.15.32:47836) 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0faffd640 1 -- 172.21.15.32:0/1141182214 learned_addr learned my addr 172.21.15.32:0/1141182214 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0fb7fe640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0fc070230 0x7fd0fc137f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0fbfff640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd0fc15abb0 0x7fd0fc158900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0faffd640 1 -- 172.21.15.32:0/1141182214 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd0fc15abb0 msgr2=0x7fd0fc158900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0faffd640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd0fc15abb0 0x7fd0fc158900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0faffd640 1 -- 172.21.15.32:0/1141182214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0fc070230 msgr2=0x7fd0fc137f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0faffd640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0fc070230 0x7fd0fc137f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0faffd640 1 -- 172.21.15.32:0/1141182214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd0f4009a70 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0faffd640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0fc0b0cb0 0x7fd0fc152350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00ad040 tx=0x7fd0f001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0e3fff640 1 -- 172.21.15.32:0/1141182214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00bf610 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0e3fff640 1 -- 172.21.15.32:0/1141182214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd0f00bf790 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd101efc640 1 -- 172.21.15.32:0/1141182214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd0f0009420 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd0e3fff640 1 -- 172.21.15.32:0/1141182214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd0f00bfb40 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.591+0000 7fd101efc640 1 -- 172.21.15.32:0/1141182214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd0fc138d80 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.595+0000 7fd101efc640 1 -- 172.21.15.32:0/1141182214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd0fc04e000 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.595+0000 7fd0e3fff640 1 -- 172.21.15.32:0/1141182214 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd0f00b5400 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.595+0000 7fd0e3fff640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd0e8065a60 0x7fd0e8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:06.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.595+0000 7fd0e3fff640 1 -- 172.21.15.32:0/1141182214 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd0f011dba0 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.595+0000 7fd0fb7fe640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd0e8065a60 0x7fd0e8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:06.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.595+0000 7fd0fb7fe640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd0e8065a60 0x7fd0e8067f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0011f0 tx=0x7fd0ec007d40 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:06.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.599+0000 7fd0e3fff640 1 -- 172.21.15.32:0/1141182214 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd0f00eb620 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.755+0000 7fd101efc640 1 -- 172.21.15.32:0/1141182214 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fd0fc0cea00 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.755+0000 7fd0e3fff640 1 -- 172.21.15.32:0/1141182214 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fd0f00eec40 con 0x7fd0fc0b0cb0 2023-12-20T04:29:06.812 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:29:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 -- 172.21.15.32:0/1141182214 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd0e8065a60 msgr2=0x7fd0e8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd0e8065a60 0x7fd0e8067f20 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7fd0ec0011f0 tx=0x7fd0ec007d40 comp rx=0 tx=0).stop 2023-12-20T04:29:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 -- 172.21.15.32:0/1141182214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0fc0b0cb0 msgr2=0x7fd0fc152350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0fc0b0cb0 0x7fd0fc152350 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fd0f00ad040 tx=0x7fd0f001acc0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 -- 172.21.15.32:0/1141182214 shutdown_connections 2023-12-20T04:29:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd0e8065a60 0x7fd0e8067f20 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd0fc15abb0 0x7fd0fc158900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd0fc0b0cb0 0x7fd0fc152350 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 --2- 172.21.15.32:0/1141182214 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd0fc070230 0x7fd0fc137f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 -- 172.21.15.32:0/1141182214 >> 172.21.15.32:0/1141182214 conn(0x7fd0fc06a510 msgr2=0x7fd0fc0cb6a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 -- 172.21.15.32:0/1141182214 shutdown_connections 2023-12-20T04:29:06.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.759+0000 7fd101efc640 1 -- 172.21.15.32:0/1141182214 wait complete. 2023-12-20T04:29:06.851 INFO:tasks.cephfs_test_runner:Starting test: test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test concurrent snap creates in same file-system without db issues 2023-12-20T04:29:06.851 INFO:tasks.cephfs_test_runner:test_concurrent_snap_creates (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-20T04:29:06.851 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-12-20T04:29:06.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.995+0000 7feb8b073640 1 Processor -- start 2023-12-20T04:29:06.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.995+0000 7feb8b073640 1 -- start start 2023-12-20T04:29:07.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb8b073640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb840d2740 0x7feb840cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:07.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb8b073640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb840cfec0 con 0x7feb840cfab0 2023-12-20T04:29:07.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb8b073640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb8415d190 con 0x7feb8415cdb0 2023-12-20T04:29:07.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb8b073640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb840d16a0 con 0x7feb840d2740 2023-12-20T04:29:07.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb83fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb840d2740 0x7feb840cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:07.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb89610640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feb840cfab0 0x7feb840b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57330/0 (socket says 172.21.15.32:57330) 2023-12-20T04:29:07.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb89610640 1 -- 172.21.15.32:0/2809823666 learned_addr learned my addr 172.21.15.32:0/2809823666 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:07.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb837fe640 1 -- 172.21.15.32:0/2809823666 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 321386946 0 0) 0x7feb840cfec0 con 0x7feb840cfab0 2023-12-20T04:29:07.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb837fe640 1 -- 172.21.15.32:0/2809823666 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb74003530 con 0x7feb840cfab0 2023-12-20T04:29:07.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb83fff640 1 -- 172.21.15.32:0/2809823666 >> v1:172.21.15.39:6789/0 conn(0x7feb8415cdb0 legacy=0x7feb840d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:07.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb83fff640 1 -- 172.21.15.32:0/2809823666 >> v1:172.21.15.32:6789/0 conn(0x7feb840cfab0 legacy=0x7feb840b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:07.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:06.999+0000 7feb83fff640 1 -- 172.21.15.32:0/2809823666 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb840d1980 con 0x7feb840d2740 2023-12-20T04:29:07.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb83fff640 1 --2- 172.21.15.32:0/2809823666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb840d2740 0x7feb840cf570 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feb700097c0 tx=0x7feb7001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34a2698fd2f7ecea server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:07.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb837fe640 1 -- 172.21.15.32:0/2809823666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb700a8970 con 0x7feb840d2740 2023-12-20T04:29:07.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb837fe640 1 -- 172.21.15.32:0/2809823666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb700a8ad0 con 0x7feb840d2740 2023-12-20T04:29:07.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb837fe640 1 -- 172.21.15.32:0/2809823666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb700a8e60 con 0x7feb840d2740 2023-12-20T04:29:07.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb8b073640 1 -- 172.21.15.32:0/2809823666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb840d2740 msgr2=0x7feb840cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:07.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb8b073640 1 --2- 172.21.15.32:0/2809823666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb840d2740 0x7feb840cf570 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7feb700097c0 tx=0x7feb7001ec70 comp rx=0 tx=0).stop 2023-12-20T04:29:07.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb8b073640 1 -- 172.21.15.32:0/2809823666 shutdown_connections 2023-12-20T04:29:07.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb8b073640 1 --2- 172.21.15.32:0/2809823666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb840d2740 0x7feb840cf570 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:07.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb8b073640 1 -- 172.21.15.32:0/2809823666 >> 172.21.15.32:0/2809823666 conn(0x7feb84065ee0 msgr2=0x7feb8406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:07.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb8b073640 1 -- 172.21.15.32:0/2809823666 shutdown_connections 2023-12-20T04:29:07.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb8b073640 1 -- 172.21.15.32:0/2809823666 wait complete. 2023-12-20T04:29:07.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb8b073640 1 Processor -- start 2023-12-20T04:29:07.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.003+0000 7feb8b073640 1 -- start start 2023-12-20T04:29:07.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb8b073640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb840cfab0 0x7feb84156150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:07.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb8b073640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb840d2740 0x7feb840c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb8b073640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb8415cdb0 0x7feb841f63f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb8b073640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb841f5c30 con 0x7feb840d2740 2023-12-20T04:29:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb8b073640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb841567e0 con 0x7feb840cfab0 2023-12-20T04:29:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb8b073640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb84156ac0 con 0x7feb8415cdb0 2023-12-20T04:29:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb83fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb840d2740 0x7feb840c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb83fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb840d2740 0x7feb840c84a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47864/0 (socket says 172.21.15.32:47864) 2023-12-20T04:29:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb83fff640 1 -- 172.21.15.32:0/1625530928 learned_addr learned my addr 172.21.15.32:0/1625530928 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb88e0f640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb840cfab0 0x7feb84156150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb89610640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb8415cdb0 0x7feb841f63f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb83fff640 1 -- 172.21.15.32:0/1625530928 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb8415cdb0 msgr2=0x7feb841f63f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:07.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb83fff640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb8415cdb0 0x7feb841f63f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:07.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb83fff640 1 -- 172.21.15.32:0/1625530928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb840cfab0 msgr2=0x7feb84156150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:07.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb83fff640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb840cfab0 0x7feb84156150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:07.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb83fff640 1 -- 172.21.15.32:0/1625530928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb70009420 con 0x7feb840d2740 2023-12-20T04:29:07.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb89610640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb8415cdb0 0x7feb841f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:07.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb83fff640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb840d2740 0x7feb840c84a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb700b3040 tx=0x7feb7001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb817fa640 1 -- 172.21.15.32:0/1625530928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb700b9630 con 0x7feb840d2740 2023-12-20T04:29:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb817fa640 1 -- 172.21.15.32:0/1625530928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb700b97b0 con 0x7feb840d2740 2023-12-20T04:29:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb817fa640 1 -- 172.21.15.32:0/1625530928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb700b9b60 con 0x7feb840d2740 2023-12-20T04:29:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb8b073640 1 -- 172.21.15.32:0/1625530928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb84156da0 con 0x7feb840d2740 2023-12-20T04:29:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb8b073640 1 -- 172.21.15.32:0/1625530928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb84153720 con 0x7feb840d2740 2023-12-20T04:29:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.007+0000 7feb8b073640 1 -- 172.21.15.32:0/1625530928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb8404e000 con 0x7feb840d2740 2023-12-20T04:29:07.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.011+0000 7feb817fa640 1 -- 172.21.15.32:0/1625530928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7feb700b7040 con 0x7feb840d2740 2023-12-20T04:29:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.011+0000 7feb817fa640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb68065a60 0x7feb68067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.011+0000 7feb817fa640 1 -- 172.21.15.32:0/1625530928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7feb70121030 con 0x7feb840d2740 2023-12-20T04:29:07.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.011+0000 7feb88e0f640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb68065a60 0x7feb68067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.011+0000 7feb88e0f640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb68065a60 0x7feb68067f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feb78007fd0 tx=0x7feb78009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.015+0000 7feb817fa640 1 -- 172.21.15.32:0/1625530928 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feb700ee9f0 con 0x7feb840d2740 2023-12-20T04:29:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.231+0000 7feb8b073640 1 -- 172.21.15.32:0/1625530928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7feb84156c00 con 0x7feb840d2740 2023-12-20T04:29:07.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.851+0000 7feb817fa640 1 -- 172.21.15.32:0/1625530928 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v54) v1 ==== 161+0+0 (secure 0 0 0) 0x7feb84156c00 con 0x7feb840d2740 2023-12-20T04:29:07.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.851+0000 7feb8b073640 1 -- 172.21.15.32:0/1625530928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb68065a60 msgr2=0x7feb68067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:07.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.851+0000 7feb8b073640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb68065a60 0x7feb68067f20 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7feb78007fd0 tx=0x7feb78009040 comp rx=0 tx=0).stop 2023-12-20T04:29:07.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.851+0000 7feb8b073640 1 -- 172.21.15.32:0/1625530928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb840d2740 msgr2=0x7feb840c84a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:07.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.851+0000 7feb8b073640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb840d2740 0x7feb840c84a0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7feb700b3040 tx=0x7feb7001ba10 comp rx=0 tx=0).stop 2023-12-20T04:29:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.855+0000 7feb8b073640 1 -- 172.21.15.32:0/1625530928 shutdown_connections 2023-12-20T04:29:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.855+0000 7feb8b073640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb68065a60 0x7feb68067f20 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.855+0000 7feb8b073640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb8415cdb0 0x7feb841f63f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.855+0000 7feb8b073640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb840d2740 0x7feb840c84a0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.855+0000 7feb8b073640 1 --2- 172.21.15.32:0/1625530928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb840cfab0 0x7feb84156150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.855+0000 7feb8b073640 1 -- 172.21.15.32:0/1625530928 >> 172.21.15.32:0/1625530928 conn(0x7feb84065ee0 msgr2=0x7feb841586d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.855+0000 7feb8b073640 1 -- 172.21.15.32:0/1625530928 shutdown_connections 2023-12-20T04:29:07.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:07.855+0000 7feb8b073640 1 -- 172.21.15.32:0/1625530928 wait complete. 2023-12-20T04:29:07.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T04:29:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.027+0000 7f6dbc066640 1 Processor -- start 2023-12-20T04:29:08.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.027+0000 7f6dbc066640 1 -- start start 2023-12-20T04:29:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6dbc066640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6db4070490 0x7f6db415fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6dbc066640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6db4070870 0x7f6db40ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6dbc066640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6db406c480 con 0x7f6db406d240 2023-12-20T04:29:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6dbc066640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6db406c760 con 0x7f6db4070490 2023-12-20T04:29:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6dbc066640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db406ca40 con 0x7f6db4070870 2023-12-20T04:29:08.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6db9e02640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6db4070490 0x7f6db415fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6db9601640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6db406d240 0x7f6db40ca1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57334/0 (socket says 172.21.15.32:57334) 2023-12-20T04:29:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6db9601640 1 -- 172.21.15.32:0/187328546 learned_addr learned my addr 172.21.15.32:0/187328546 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6dba603640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6db4070870 0x7f6db40ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6db9e02640 1 -- 172.21.15.32:0/187328546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6db4070870 msgr2=0x7f6db40ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6db9e02640 1 --2- 172.21.15.32:0/187328546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6db4070870 0x7f6db40ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6db9e02640 1 -- 172.21.15.32:0/187328546 >> v1:172.21.15.32:6789/0 conn(0x7f6db406d240 legacy=0x7f6db40ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6db9e02640 1 -- 172.21.15.32:0/187328546 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db406cd20 con 0x7f6db4070490 2023-12-20T04:29:08.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.031+0000 7f6db8e00640 1 -- 172.21.15.32:0/187328546 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132662717 0 0) 0x7f6db406c480 con 0x7f6db406d240 2023-12-20T04:29:08.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6db9e02640 1 --2- 172.21.15.32:0/187328546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6db4070490 0x7f6db415fcd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6db00097c0 tx=0x7f6db001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9336d8ff9f1ba7f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6db8e00640 1 -- 172.21.15.32:0/187328546 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db00ad660 con 0x7f6db4070490 2023-12-20T04:29:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6db8e00640 1 -- 172.21.15.32:0/187328546 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db00ad7c0 con 0x7f6db4070490 2023-12-20T04:29:08.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6db8e00640 1 -- 172.21.15.32:0/187328546 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db00adb50 con 0x7f6db4070490 2023-12-20T04:29:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6dbc066640 1 -- 172.21.15.32:0/187328546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6db4070490 msgr2=0x7f6db415fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6dbc066640 1 --2- 172.21.15.32:0/187328546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6db4070490 0x7f6db415fcd0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f6db00097c0 tx=0x7f6db001c3c0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6dbc066640 1 -- 172.21.15.32:0/187328546 shutdown_connections 2023-12-20T04:29:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6dbc066640 1 --2- 172.21.15.32:0/187328546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6db4070870 0x7f6db40ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6dbc066640 1 --2- 172.21.15.32:0/187328546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6db4070490 0x7f6db415fcd0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6dbc066640 1 -- 172.21.15.32:0/187328546 >> 172.21.15.32:0/187328546 conn(0x7f6db406a510 msgr2=0x7f6db40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6dbc066640 1 -- 172.21.15.32:0/187328546 shutdown_connections 2023-12-20T04:29:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6dbc066640 1 -- 172.21.15.32:0/187328546 wait complete. 2023-12-20T04:29:08.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6dbc066640 1 Processor -- start 2023-12-20T04:29:08.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.035+0000 7f6dbc066640 1 -- start start 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6dbc066640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db406d240 0x7f6db40c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6dbc066640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6db4070490 0x7f6db4155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6dbc066640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6db4070870 0x7f6db4146fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6dbc066640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4147bb0 con 0x7f6db406d240 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6dbc066640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6db40c8b20 con 0x7f6db4070870 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6dbc066640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6db4147e00 con 0x7f6db4070490 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6db9e02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db406d240 0x7f6db40c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6db9e02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db406d240 0x7f6db40c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47880/0 (socket says 172.21.15.32:47880) 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6db9e02640 1 -- 172.21.15.32:0/176397637 learned_addr learned my addr 172.21.15.32:0/176397637 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6db9601640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6db4070490 0x7f6db4155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6dba603640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6db4070870 0x7f6db4146fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6db9e02640 1 -- 172.21.15.32:0/176397637 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6db4070490 msgr2=0x7f6db4155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6db9e02640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6db4070490 0x7f6db4155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6db9e02640 1 -- 172.21.15.32:0/176397637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6db4070870 msgr2=0x7f6db4146fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6db9e02640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6db4070870 0x7f6db4146fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6db9e02640 1 -- 172.21.15.32:0/176397637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6db0009420 con 0x7f6db406d240 2023-12-20T04:29:08.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6db9e02640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db406d240 0x7f6db40c84b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6db00b0040 tx=0x7f6db001ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:08.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6daa7fc640 1 -- 172.21.15.32:0/176397637 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db00b9650 con 0x7f6db406d240 2023-12-20T04:29:08.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6daa7fc640 1 -- 172.21.15.32:0/176397637 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6db00b97f0 con 0x7f6db406d240 2023-12-20T04:29:08.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6dbc066640 1 -- 172.21.15.32:0/176397637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6db41480e0 con 0x7f6db406d240 2023-12-20T04:29:08.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6daa7fc640 1 -- 172.21.15.32:0/176397637 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6db00b9ba0 con 0x7f6db406d240 2023-12-20T04:29:08.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6dbc066640 1 -- 172.21.15.32:0/176397637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6db40c6140 con 0x7f6db406d240 2023-12-20T04:29:08.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.039+0000 7f6dbc066640 1 -- 172.21.15.32:0/176397637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6db404e000 con 0x7f6db406d240 2023-12-20T04:29:08.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.043+0000 7f6daa7fc640 1 -- 172.21.15.32:0/176397637 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6db00b9d00 con 0x7f6db406d240 2023-12-20T04:29:08.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.043+0000 7f6daa7fc640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d98065a60 0x7f6d98067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.043+0000 7f6daa7fc640 1 -- 172.21.15.32:0/176397637 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6db011e080 con 0x7f6db406d240 2023-12-20T04:29:08.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.043+0000 7f6db9601640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d98065a60 0x7f6d98067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.043+0000 7f6db9601640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d98065a60 0x7f6d98067f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6db4158740 tx=0x7f6da4007b60 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:08.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.047+0000 7f6daa7fc640 1 -- 172.21.15.32:0/176397637 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6db00ebbf0 con 0x7f6db406d240 2023-12-20T04:29:08.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.203+0000 7f6dbc066640 1 -- 172.21.15.32:0/176397637 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f6db40cf4d0 con 0x7f6db406d240 2023-12-20T04:29:08.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6daa7fc640 1 -- 172.21.15.32:0/176397637 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v10)=0 v10) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6db00ef1d0 con 0x7f6db406d240 2023-12-20T04:29:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 -- 172.21.15.32:0/176397637 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d98065a60 msgr2=0x7f6d98067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d98065a60 0x7f6d98067f20 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6db4158740 tx=0x7f6da4007b60 comp rx=0 tx=0).stop 2023-12-20T04:29:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 -- 172.21.15.32:0/176397637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db406d240 msgr2=0x7f6db40c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db406d240 0x7f6db40c84b0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f6db00b0040 tx=0x7f6db001ac00 comp rx=0 tx=0).stop 2023-12-20T04:29:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 -- 172.21.15.32:0/176397637 shutdown_connections 2023-12-20T04:29:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d98065a60 0x7f6d98067f20 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6db4070870 0x7f6db4146fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6db4070490 0x7f6db4155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 --2- 172.21.15.32:0/176397637 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6db406d240 0x7f6db40c84b0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 -- 172.21.15.32:0/176397637 >> 172.21.15.32:0/176397637 conn(0x7f6db406a510 msgr2=0x7f6db4158c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 -- 172.21.15.32:0/176397637 shutdown_connections 2023-12-20T04:29:08.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.211+0000 7f6dbc066640 1 -- 172.21.15.32:0/176397637 wait complete. 2023-12-20T04:29:08.234 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:29:08.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:29:08.286 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:29:08.376 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:29:08.377 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:29:08.377 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:08.386 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:29:08.386 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:08.440 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:29:08.440 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:08.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.679+0000 7f38412c4640 1 Processor -- start 2023-12-20T04:29:08.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.679+0000 7f38412c4640 1 -- start start 2023-12-20T04:29:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.679+0000 7f38412c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c161240 0x7f383c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.679+0000 7f38412c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383c0c9a40 con 0x7f383c15ce60 2023-12-20T04:29:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.679+0000 7f38412c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383c0c9180 con 0x7f383c071af0 2023-12-20T04:29:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.679+0000 7f38412c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383c1616b0 con 0x7f383c161240 2023-12-20T04:29:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f383c15ce60 0x7f383c0713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57336/0 (socket says 172.21.15.32:57336) 2023-12-20T04:29:08.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383affd640 1 -- 172.21.15.32:0/1102386290 learned_addr learned my addr 172.21.15.32:0/1102386290 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:08.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383b7fe640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c161240 0x7f383c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4026556096 0 0) 0x7f383c0c9a40 con 0x7f383c15ce60 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3828003530 con 0x7f383c15ce60 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716503963 0 0) 0x7f383c0c9180 con 0x7f383c071af0 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f383c0c9a40 con 0x7f383c071af0 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3253309250 0 0) 0x7f3828003530 con 0x7f383c15ce60 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c161240 msgr2=0x7f383c0c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c161240 0x7f383c0c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 >> v1:172.21.15.39:6789/0 conn(0x7f383c071af0 legacy=0x7f383c160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c0c93f0 con 0x7f383c15ce60 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f382c001800 con 0x7f383c15ce60 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 >> v1:172.21.15.32:6789/0 conn(0x7f383c15ce60 legacy=0x7f383c0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 0x7f3828003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c161240 0x7f38280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3828004d10 0x7f3828005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c1616b0 con 0x7f3828003b40 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f382800ba40 con 0x7f3828004d10 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f382800bc00 con 0x7f383c161240 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383affd640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 0x7f3828003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f3833fff640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c161240 0x7f38280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383b7fe640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3828004d10 0x7f3828005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383affd640 1 -- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c161240 msgr2=0x7f38280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383affd640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c161240 0x7f38280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383affd640 1 -- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3828004d10 msgr2=0x7f3828005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383affd640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3828004d10 0x7f3828005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383affd640 1 -- 172.21.15.32:0/1102386290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3828003530 con 0x7f3828003b40 2023-12-20T04:29:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.683+0000 7f383b7fe640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3828004d10 0x7f3828005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f383affd640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 0x7f3828003f40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f382c007fa0 tx=0x7f382c01cab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=840acb52b12c5055 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c007270 con 0x7f3828003b40 2023-12-20T04:29:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c0073d0 con 0x7f3828003b40 2023-12-20T04:29:08.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f383a7fc640 1 -- 172.21.15.32:0/1102386290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c0057b0 con 0x7f3828003b40 2023-12-20T04:29:08.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f38412c4640 1 -- 172.21.15.32:0/1102386290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 msgr2=0x7f3828003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f38412c4640 1 --2- 172.21.15.32:0/1102386290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 0x7f3828003f40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f382c007fa0 tx=0x7f382c01cab0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f383b7fe640 1 -- 172.21.15.32:0/1102386290 reap_dead start 2023-12-20T04:29:08.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f38412c4640 1 -- 172.21.15.32:0/1102386290 shutdown_connections 2023-12-20T04:29:08.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f38412c4640 1 -- 172.21.15.32:0/1102386290 >> 172.21.15.32:0/1102386290 conn(0x7f383c06a320 msgr2=0x7f383c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:08.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f38412c4640 1 -- 172.21.15.32:0/1102386290 shutdown_connections 2023-12-20T04:29:08.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f38412c4640 1 -- 172.21.15.32:0/1102386290 wait complete. 2023-12-20T04:29:08.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f38412c4640 1 Processor -- start 2023-12-20T04:29:08.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.687+0000 7f38412c4640 1 -- start start 2023-12-20T04:29:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38412c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 0x7f383c155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38412c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c0cb930 0x7f383c15ff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38412c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c15dbf0 0x7f383c15e070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38412c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c15e5b0 con 0x7f3828003b40 2023-12-20T04:29:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38412c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c0c4720 con 0x7f383c0cb930 2023-12-20T04:29:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38412c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383c1563a0 con 0x7f383c15dbf0 2023-12-20T04:29:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f383affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 0x7f383c155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f383affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 0x7f383c155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47896/0 (socket says 172.21.15.32:47896) 2023-12-20T04:29:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f383affd640 1 -- 172.21.15.32:0/2851056611 learned_addr learned my addr 172.21.15.32:0/2851056611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f383b7fe640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c15dbf0 0x7f383c15e070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f3833fff640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c0cb930 0x7f383c15ff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f383affd640 1 -- 172.21.15.32:0/2851056611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c15dbf0 msgr2=0x7f383c15e070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f383affd640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c15dbf0 0x7f383c15e070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f383affd640 1 -- 172.21.15.32:0/2851056611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c0cb930 msgr2=0x7f383c15ff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f383affd640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c0cb930 0x7f383c15ff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f383affd640 1 -- 172.21.15.32:0/2851056611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c0c3b10 con 0x7f3828003b40 2023-12-20T04:29:08.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f383affd640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 0x7f383c155e60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f382c01c8e0 tx=0x7f3828005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:08.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38337fe640 1 -- 172.21.15.32:0/2851056611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c0b4070 con 0x7f3828003b40 2023-12-20T04:29:08.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38337fe640 1 -- 172.21.15.32:0/2851056611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c005e50 con 0x7f3828003b40 2023-12-20T04:29:08.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38337fe640 1 -- 172.21.15.32:0/2851056611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c000f80 con 0x7f3828003b40 2023-12-20T04:29:08.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38412c4640 1 -- 172.21.15.32:0/2851056611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f383c0c3df0 con 0x7f3828003b40 2023-12-20T04:29:08.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38412c4640 1 -- 172.21.15.32:0/2851056611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f383c0c43b0 con 0x7f3828003b40 2023-12-20T04:29:08.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38412c4640 1 -- 172.21.15.32:0/2851056611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383c04e000 con 0x7f3828003b40 2023-12-20T04:29:08.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.691+0000 7f38337fe640 1 -- 172.21.15.32:0/2851056611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f382c001600 con 0x7f3828003b40 2023-12-20T04:29:08.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.695+0000 7f38337fe640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065a90 0x7f381c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:08.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.695+0000 7f38337fe640 1 -- 172.21.15.32:0/2851056611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f382c11cb30 con 0x7f3828003b40 2023-12-20T04:29:08.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.695+0000 7f3833fff640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065a90 0x7f381c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:08.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.695+0000 7f3833fff640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065a90 0x7f381c067f50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f383c0c55f0 tx=0x7f3824009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.699+0000 7f38337fe640 1 -- 172.21.15.32:0/2851056611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f382c0ea4e0 con 0x7f3828003b40 2023-12-20T04:29:08.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.855+0000 7f38412c4640 1 -- 172.21.15.32:0/2851056611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f383c0cf450 con 0x7f3828003b40 2023-12-20T04:29:08.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.855+0000 7f38337fe640 1 -- 172.21.15.32:0/2851056611 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f382c0edb00 con 0x7f3828003b40 2023-12-20T04:29:08.859 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:08.859 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 -- 172.21.15.32:0/2851056611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065a90 msgr2=0x7f381c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065a90 0x7f381c067f50 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f383c0c55f0 tx=0x7f3824009040 comp rx=0 tx=0).stop 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 -- 172.21.15.32:0/2851056611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 msgr2=0x7f383c155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 0x7f383c155e60 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f382c01c8e0 tx=0x7f3828005800 comp rx=0 tx=0).stop 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 -- 172.21.15.32:0/2851056611 shutdown_connections 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065a90 0x7f381c067f50 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c15dbf0 0x7f383c15e070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c0cb930 0x7f383c15ff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 --2- 172.21.15.32:0/2851056611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3828003b40 0x7f383c155e60 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 -- 172.21.15.32:0/2851056611 >> 172.21.15.32:0/2851056611 conn(0x7f383c06a320 msgr2=0x7f383c06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 -- 172.21.15.32:0/2851056611 shutdown_connections 2023-12-20T04:29:08.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:08.859+0000 7f38412c4640 1 -- 172.21.15.32:0/2851056611 wait complete. 2023-12-20T04:29:08.881 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:28:36.040195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:28:37.364671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:29:08.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:29:09.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.023+0000 7f2f73295640 1 Processor -- start 2023-12-20T04:29:09.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.023+0000 7f2f73295640 1 -- start start 2023-12-20T04:29:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f73295640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2f6c0658e0 0x7f2f6c06ca60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f73295640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f6c0711c0 0x7f2f6c06cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f73295640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f6c06a6f0 con 0x7f2f6c065cc0 2023-12-20T04:29:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f73295640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f6c15a6c0 con 0x7f2f6c0658e0 2023-12-20T04:29:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f73295640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c06a9d0 con 0x7f2f6c0711c0 2023-12-20T04:29:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f71832640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f6c0711c0 0x7f2f6c06cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f70830640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2f6c065cc0 0x7f2f6c070ab0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:49460/0 (socket says 172.21.15.32:49460) 2023-12-20T04:29:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f71031640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2f6c0658e0 0x7f2f6c06ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f70830640 1 -- 172.21.15.32:0/2456531230 learned_addr learned my addr 172.21.15.32:0/2456531230 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f71031640 1 -- 172.21.15.32:0/2456531230 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f6c0711c0 msgr2=0x7f2f6c06cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:09.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f71031640 1 --2- 172.21.15.32:0/2456531230 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f6c0711c0 0x7f2f6c06cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f71031640 1 -- 172.21.15.32:0/2456531230 >> v1:172.21.15.39:6789/0 conn(0x7f2f6c065cc0 legacy=0x7f2f6c070ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.027+0000 7f2f71031640 1 -- 172.21.15.32:0/2456531230 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c06acb0 con 0x7f2f6c0658e0 2023-12-20T04:29:09.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f71031640 1 --2- 172.21.15.32:0/2456531230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f6c0658e0 0x7f2f6c06ca60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2f680097f0 tx=0x7f2f6801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b6ff416f84513ea server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:09.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f63fff640 1 -- 172.21.15.32:0/2456531230 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680ad660 con 0x7f2f6c0658e0 2023-12-20T04:29:09.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f63fff640 1 -- 172.21.15.32:0/2456531230 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f680ad7c0 con 0x7f2f6c0658e0 2023-12-20T04:29:09.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f63fff640 1 -- 172.21.15.32:0/2456531230 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680adb50 con 0x7f2f6c0658e0 2023-12-20T04:29:09.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f73295640 1 -- 172.21.15.32:0/2456531230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f6c0658e0 msgr2=0x7f2f6c06ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f73295640 1 --2- 172.21.15.32:0/2456531230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f6c0658e0 0x7f2f6c06ca60 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f2f680097f0 tx=0x7f2f6801c450 comp rx=0 tx=0).stop 2023-12-20T04:29:09.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f73295640 1 -- 172.21.15.32:0/2456531230 shutdown_connections 2023-12-20T04:29:09.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f73295640 1 --2- 172.21.15.32:0/2456531230 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f6c0711c0 0x7f2f6c06cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f73295640 1 --2- 172.21.15.32:0/2456531230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f6c0658e0 0x7f2f6c06ca60 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f73295640 1 -- 172.21.15.32:0/2456531230 >> 172.21.15.32:0/2456531230 conn(0x7f2f6c0cf8d0 msgr2=0x7f2f6c0cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:09.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f73295640 1 -- 172.21.15.32:0/2456531230 shutdown_connections 2023-12-20T04:29:09.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f73295640 1 -- 172.21.15.32:0/2456531230 wait complete. 2023-12-20T04:29:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f73295640 1 Processor -- start 2023-12-20T04:29:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.031+0000 7f2f73295640 1 -- start start 2023-12-20T04:29:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f73295640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f6c0658e0 0x7f2f6c0c1b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f73295640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f6c065cc0 0x7f2f6c0c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f73295640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f6c0711c0 0x7f2f6c15b850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f73295640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0c8bf0 con 0x7f2f6c0658e0 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f73295640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0c7f70 con 0x7f2f6c065cc0 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f73295640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f6c0c8250 con 0x7f2f6c0711c0 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f71031640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f6c0658e0 0x7f2f6c0c1b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f71031640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f6c0658e0 0x7f2f6c0c1b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59070/0 (socket says 172.21.15.32:59070) 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f71031640 1 -- 172.21.15.32:0/2560297528 learned_addr learned my addr 172.21.15.32:0/2560297528 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f71832640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f6c0711c0 0x7f2f6c15b850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f70830640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f6c065cc0 0x7f2f6c0c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f71031640 1 -- 172.21.15.32:0/2560297528 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f6c0711c0 msgr2=0x7f2f6c15b850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f71031640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f6c0711c0 0x7f2f6c15b850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f71031640 1 -- 172.21.15.32:0/2560297528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f6c065cc0 msgr2=0x7f2f6c0c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f71031640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f6c065cc0 0x7f2f6c0c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f71031640 1 -- 172.21.15.32:0/2560297528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f68009420 con 0x7f2f6c0658e0 2023-12-20T04:29:09.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f71031640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f6c0658e0 0x7f2f6c0c1b20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2f680b0040 tx=0x7f2f680191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f61ffb640 1 -- 172.21.15.32:0/2560297528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680b6610 con 0x7f2f6c0658e0 2023-12-20T04:29:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f61ffb640 1 -- 172.21.15.32:0/2560297528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f680b6790 con 0x7f2f6c0658e0 2023-12-20T04:29:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f61ffb640 1 -- 172.21.15.32:0/2560297528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f680b6b40 con 0x7f2f6c0658e0 2023-12-20T04:29:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f73295640 1 -- 172.21.15.32:0/2560297528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f6c0c8530 con 0x7f2f6c0658e0 2023-12-20T04:29:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f73295640 1 -- 172.21.15.32:0/2560297528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f6c0c2060 con 0x7f2f6c0658e0 2023-12-20T04:29:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.035+0000 7f2f73295640 1 -- 172.21.15.32:0/2560297528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f6c04e000 con 0x7f2f6c0658e0 2023-12-20T04:29:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.039+0000 7f2f61ffb640 1 -- 172.21.15.32:0/2560297528 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2f680b4040 con 0x7f2f6c0658e0 2023-12-20T04:29:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.039+0000 7f2f61ffb640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f50065a60 0x7f2f50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.039+0000 7f2f61ffb640 1 -- 172.21.15.32:0/2560297528 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f6811d410 con 0x7f2f6c0658e0 2023-12-20T04:29:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.039+0000 7f2f70830640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f50065a60 0x7f2f50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.039+0000 7f2f70830640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f50065a60 0x7f2f50067f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c156340 tx=0x7f2f5c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:09.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.043+0000 7f2f61ffb640 1 -- 172.21.15.32:0/2560297528 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2f680ead40 con 0x7f2f6c0658e0 2023-12-20T04:29:09.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.199+0000 7f2f73295640 1 -- 172.21.15.32:0/2560297528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f2f6c0b4fd0 con 0x7f2f6c0658e0 2023-12-20T04:29:09.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.199+0000 7f2f61ffb640 1 -- 172.21.15.32:0/2560297528 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f2f680ee360 con 0x7f2f6c0658e0 2023-12-20T04:29:09.201 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:29:09.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 -- 172.21.15.32:0/2560297528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f50065a60 msgr2=0x7f2f50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f50065a60 0x7f2f50067f20 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c156340 tx=0x7f2f5c009040 comp rx=0 tx=0).stop 2023-12-20T04:29:09.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 -- 172.21.15.32:0/2560297528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f6c0658e0 msgr2=0x7f2f6c0c1b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f6c0658e0 0x7f2f6c0c1b20 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f2f680b0040 tx=0x7f2f680191f0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 -- 172.21.15.32:0/2560297528 shutdown_connections 2023-12-20T04:29:09.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f50065a60 0x7f2f50067f20 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f6c0711c0 0x7f2f6c15b850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f6c065cc0 0x7f2f6c0c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 --2- 172.21.15.32:0/2560297528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f6c0658e0 0x7f2f6c0c1b20 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 -- 172.21.15.32:0/2560297528 >> 172.21.15.32:0/2560297528 conn(0x7f2f6c0cf8d0 msgr2=0x7f2f6c153800 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:09.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 -- 172.21.15.32:0/2560297528 shutdown_connections 2023-12-20T04:29:09.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.203+0000 7f2f73295640 1 -- 172.21.15.32:0/2560297528 wait complete. 2023-12-20T04:29:09.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:29:09.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62e09e9640 1 Processor -- start 2023-12-20T04:29:09.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62e09e9640 1 -- start start 2023-12-20T04:29:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62e09e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62dc15abb0 0x7f62dc0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62e09e9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f62dc0b0cb0 0x7f62dc15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62e09e9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62dc070230 0x7f62dc0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62e09e9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62dc06e160 con 0x7f62dc070230 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62e09e9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62dc070be0 con 0x7f62dc0b0cb0 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62e09e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15b6a0 con 0x7f62dc15abb0 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62dad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62dc070230 0x7f62dc0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62dad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62dc070230 0x7f62dc0706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59078/0 (socket says 172.21.15.32:59078) 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62dad9d640 1 -- 172.21.15.32:0/3385513152 learned_addr learned my addr 172.21.15.32:0/3385513152 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62dad9d640 1 -- 172.21.15.32:0/3385513152 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62dc15abb0 msgr2=0x7f62dc0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62da59c640 1 --2- 172.21.15.32:0/3385513152 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62dc15abb0 0x7f62dc0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62d1d9b640 1 --2- 172.21.15.32:0/3385513152 >> v2:172.21.15.39:3300/0 conn(0x7f62dc0b0cb0 0x7f62dc15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62dad9d640 1 --2- 172.21.15.32:0/3385513152 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62dc15abb0 0x7f62dc0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62dad9d640 1 -- 172.21.15.32:0/3385513152 >> v2:172.21.15.39:3300/0 conn(0x7f62dc0b0cb0 msgr2=0x7f62dc15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62dad9d640 1 --2- 172.21.15.32:0/3385513152 >> v2:172.21.15.39:3300/0 conn(0x7f62dc0b0cb0 0x7f62dc15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62dad9d640 1 -- 172.21.15.32:0/3385513152 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62dc15c070 con 0x7f62dc070230 2023-12-20T04:29:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62da59c640 1 --2- 172.21.15.32:0/3385513152 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62dc15abb0 0x7f62dc0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.367+0000 7f62d1d9b640 1 --2- 172.21.15.32:0/3385513152 >> v2:172.21.15.39:3300/0 conn(0x7f62dc0b0cb0 0x7f62dc15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62dad9d640 1 --2- 172.21.15.32:0/3385513152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62dc070230 0x7f62dc0706a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0716d0 tx=0x7f62d401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228dba3603421c5a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:09.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62d9d9b640 1 -- 172.21.15.32:0/3385513152 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d40aa620 con 0x7f62dc070230 2023-12-20T04:29:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62d9d9b640 1 -- 172.21.15.32:0/3385513152 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d40aa780 con 0x7f62dc070230 2023-12-20T04:29:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62d9d9b640 1 -- 172.21.15.32:0/3385513152 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d40aab10 con 0x7f62dc070230 2023-12-20T04:29:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62e09e9640 1 -- 172.21.15.32:0/3385513152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62dc070230 msgr2=0x7f62dc0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62e09e9640 1 --2- 172.21.15.32:0/3385513152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62dc070230 0x7f62dc0706a0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f62dc0716d0 tx=0x7f62d401c500 comp rx=0 tx=0).stop 2023-12-20T04:29:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62e09e9640 1 -- 172.21.15.32:0/3385513152 shutdown_connections 2023-12-20T04:29:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62e09e9640 1 --2- 172.21.15.32:0/3385513152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62dc070230 0x7f62dc0706a0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62e09e9640 1 --2- 172.21.15.32:0/3385513152 >> v2:172.21.15.39:3300/0 conn(0x7f62dc0b0cb0 0x7f62dc15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62e09e9640 1 --2- 172.21.15.32:0/3385513152 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62dc15abb0 0x7f62dc0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62e09e9640 1 -- 172.21.15.32:0/3385513152 >> 172.21.15.32:0/3385513152 conn(0x7f62dc06a510 msgr2=0x7f62dc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:09.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62e09e9640 1 -- 172.21.15.32:0/3385513152 shutdown_connections 2023-12-20T04:29:09.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.371+0000 7f62e09e9640 1 -- 172.21.15.32:0/3385513152 wait complete. 2023-12-20T04:29:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62e09e9640 1 Processor -- start 2023-12-20T04:29:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62e09e9640 1 -- start start 2023-12-20T04:29:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62e09e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62dc070230 0x7f62dc151770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62e09e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62dc0b0cb0 0x7f62dc0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62e09e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62dc15abb0 0x7f62dc0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62e09e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15d030 con 0x7f62dc15abb0 2023-12-20T04:29:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62e09e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15c420 con 0x7f62dc070230 2023-12-20T04:29:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62e09e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62dc15c700 con 0x7f62dc0b0cb0 2023-12-20T04:29:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62dad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62dc15abb0 0x7f62dc0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62dad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62dc15abb0 0x7f62dc0c3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59086/0 (socket says 172.21.15.32:59086) 2023-12-20T04:29:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62dad9d640 1 -- 172.21.15.32:0/2624432864 learned_addr learned my addr 172.21.15.32:0/2624432864 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62d1d9b640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62dc0b0cb0 0x7f62dc0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62da59c640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62dc070230 0x7f62dc151770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62dad9d640 1 -- 172.21.15.32:0/2624432864 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62dc0b0cb0 msgr2=0x7f62dc0c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62dad9d640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62dc0b0cb0 0x7f62dc0c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62dad9d640 1 -- 172.21.15.32:0/2624432864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62dc070230 msgr2=0x7f62dc151770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62dad9d640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62dc070230 0x7f62dc151770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62dad9d640 1 -- 172.21.15.32:0/2624432864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d40094d0 con 0x7f62dc15abb0 2023-12-20T04:29:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62dad9d640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62dc15abb0 0x7f62dc0c3ba0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62d40ac040 tx=0x7f62d401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62d37fe640 1 -- 172.21.15.32:0/2624432864 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d40bd3f0 con 0x7f62dc15abb0 2023-12-20T04:29:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62d37fe640 1 -- 172.21.15.32:0/2624432864 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d40bd550 con 0x7f62dc15abb0 2023-12-20T04:29:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62d37fe640 1 -- 172.21.15.32:0/2624432864 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d40bd900 con 0x7f62dc15abb0 2023-12-20T04:29:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62e09e9640 1 -- 172.21.15.32:0/2624432864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62dc15c9e0 con 0x7f62dc15abb0 2023-12-20T04:29:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.375+0000 7f62e09e9640 1 -- 172.21.15.32:0/2624432864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62dc151cb0 con 0x7f62dc15abb0 2023-12-20T04:29:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.379+0000 7f62e09e9640 1 -- 172.21.15.32:0/2624432864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62dc15d1e0 con 0x7f62dc15abb0 2023-12-20T04:29:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.379+0000 7f62d37fe640 1 -- 172.21.15.32:0/2624432864 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f62d40b33f0 con 0x7f62dc15abb0 2023-12-20T04:29:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.379+0000 7f62d37fe640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f62bc065a60 0x7f62bc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.379+0000 7f62d37fe640 1 -- 172.21.15.32:0/2624432864 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f62d411d440 con 0x7f62dc15abb0 2023-12-20T04:29:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.379+0000 7f62da59c640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f62bc065a60 0x7f62bc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.379+0000 7f62da59c640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f62bc065a60 0x7f62bc067f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00d620 tx=0x7f62cc007810 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.383+0000 7f62d37fe640 1 -- 172.21.15.32:0/2624432864 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f62dc15d1e0 con 0x7f62dc15abb0 2023-12-20T04:29:09.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 -- 172.21.15.32:0/2624432864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62dc04e000 con 0x7f62dc15abb0 2023-12-20T04:29:09.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62d37fe640 1 -- 172.21.15.32:0/2624432864 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62d40eafc0 con 0x7f62dc15abb0 2023-12-20T04:29:09.541 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:29:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 -- 172.21.15.32:0/2624432864 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f62bc065a60 msgr2=0x7f62bc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f62bc065a60 0x7f62bc067f20 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f62cc00d620 tx=0x7f62cc007810 comp rx=0 tx=0).stop 2023-12-20T04:29:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 -- 172.21.15.32:0/2624432864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62dc15abb0 msgr2=0x7f62dc0c3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62dc15abb0 0x7f62dc0c3ba0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f62d40ac040 tx=0x7f62d401ad40 comp rx=0 tx=0).stop 2023-12-20T04:29:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 -- 172.21.15.32:0/2624432864 shutdown_connections 2023-12-20T04:29:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f62bc065a60 0x7f62bc067f20 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62dc15abb0 0x7f62dc0c3ba0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62dc0b0cb0 0x7f62dc0c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 --2- 172.21.15.32:0/2624432864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62dc070230 0x7f62dc151770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 -- 172.21.15.32:0/2624432864 >> 172.21.15.32:0/2624432864 conn(0x7f62dc06a510 msgr2=0x7f62dc0c4820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 -- 172.21.15.32:0/2624432864 shutdown_connections 2023-12-20T04:29:09.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.539+0000 7f62e09e9640 1 -- 172.21.15.32:0/2624432864 wait complete. 2023-12-20T04:29:09.562 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:29:09.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.707+0000 7fe22190a640 1 Processor -- start 2023-12-20T04:29:09.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.707+0000 7fe22190a640 1 -- start start 2023-12-20T04:29:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe22190a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe21c0b0cb0 0x7fe21c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe22190a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c070230 0x7fe21c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe22190a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe21c15b6a0 con 0x7fe21c15abb0 2023-12-20T04:29:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe22190a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe21c06e160 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe22190a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c070c00 con 0x7fe21c070230 2023-12-20T04:29:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe21a7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe21c0b0cb0 0x7fe21c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe21affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe21c15abb0 0x7fe21c15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57426/0 (socket says 172.21.15.32:57426) 2023-12-20T04:29:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe21a7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe21c0b0cb0 0x7fe21c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59090/0 (socket says 172.21.15.32:59090) 2023-12-20T04:29:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe21affd640 1 -- 172.21.15.32:0/3686192048 learned_addr learned my addr 172.21.15.32:0/3686192048 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:09.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe21b7fe640 1 --2- 172.21.15.32:0/3686192048 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c070230 0x7fe21c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe21a7fc640 1 -- 172.21.15.32:0/3686192048 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c070230 msgr2=0x7fe21c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe21a7fc640 1 --2- 172.21.15.32:0/3686192048 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c070230 0x7fe21c0706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe21a7fc640 1 -- 172.21.15.32:0/3686192048 >> v1:172.21.15.32:6789/0 conn(0x7fe21c15abb0 legacy=0x7fe21c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe21a7fc640 1 -- 172.21.15.32:0/3686192048 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe21c06d8c0 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe21b7fe640 1 --2- 172.21.15.32:0/3686192048 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c070230 0x7fe21c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:09.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.711+0000 7fe21a7fc640 1 --2- 172.21.15.32:0/3686192048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe21c0b0cb0 0x7fe21c0caaf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2100097c0 tx=0x7fe21001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cdf7a3b54277f9f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:09.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe219ffb640 1 -- 172.21.15.32:0/3686192048 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100ab680 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe219ffb640 1 -- 172.21.15.32:0/3686192048 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2100ab7e0 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe219ffb640 1 -- 172.21.15.32:0/3686192048 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100abb70 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 -- 172.21.15.32:0/3686192048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe21c0b0cb0 msgr2=0x7fe21c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 --2- 172.21.15.32:0/3686192048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe21c0b0cb0 0x7fe21c0caaf0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fe2100097c0 tx=0x7fe21001c480 comp rx=0 tx=0).stop 2023-12-20T04:29:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 -- 172.21.15.32:0/3686192048 shutdown_connections 2023-12-20T04:29:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 --2- 172.21.15.32:0/3686192048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe21c0b0cb0 0x7fe21c0caaf0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 --2- 172.21.15.32:0/3686192048 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c070230 0x7fe21c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 -- 172.21.15.32:0/3686192048 >> 172.21.15.32:0/3686192048 conn(0x7fe21c06a510 msgr2=0x7fe21c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 -- 172.21.15.32:0/3686192048 shutdown_connections 2023-12-20T04:29:09.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 -- 172.21.15.32:0/3686192048 wait complete. 2023-12-20T04:29:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 Processor -- start 2023-12-20T04:29:09.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 -- start start 2023-12-20T04:29:09.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe21c070230 0x7fe21c14a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe21c0b0cb0 0x7fe21c152090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c15abb0 0x7fe21c0cce00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c0cc9e0 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c14a820 con 0x7fe21c070230 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe22190a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe21c159940 con 0x7fe21c15abb0 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe21c0b0cb0 0x7fe21c152090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe21a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe21c0b0cb0 0x7fe21c152090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59106/0 (socket says 172.21.15.32:59106) 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe21a7fc640 1 -- 172.21.15.32:0/1696836029 learned_addr learned my addr 172.21.15.32:0/1696836029 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.715+0000 7fe21affd640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe21c070230 0x7fe21c14a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe21b7fe640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c15abb0 0x7fe21c0cce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe21a7fc640 1 -- 172.21.15.32:0/1696836029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c15abb0 msgr2=0x7fe21c0cce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe21a7fc640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c15abb0 0x7fe21c0cce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe21a7fc640 1 -- 172.21.15.32:0/1696836029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe21c070230 msgr2=0x7fe21c14a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe21a7fc640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe21c070230 0x7fe21c14a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe21a7fc640 1 -- 172.21.15.32:0/1696836029 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe210009420 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe21b7fe640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c15abb0 0x7fe21c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:09.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe21a7fc640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe21c0b0cb0 0x7fe21c152090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe21001cd20 tx=0x7fe21001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:09.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe203fff640 1 -- 172.21.15.32:0/1696836029 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100b5810 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe203fff640 1 -- 172.21.15.32:0/1696836029 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2100b5970 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe22190a640 1 -- 172.21.15.32:0/1696836029 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe21c159380 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe203fff640 1 -- 172.21.15.32:0/1696836029 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2100b5d00 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe22190a640 1 -- 172.21.15.32:0/1696836029 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe21c0cc600 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe22190a640 1 -- 172.21.15.32:0/1696836029 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe21c0ccb20 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe203fff640 1 -- 172.21.15.32:0/1696836029 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe2100c83f0 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe203fff640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe208065a60 0x7fe208067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe203fff640 1 -- 172.21.15.32:0/1696836029 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe21011e230 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.719+0000 7fe21affd640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe208065a60 0x7fe208067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.723+0000 7fe21affd640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe208065a60 0x7fe208067f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe20c000f70 tx=0x7fe20c007e90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:09.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.723+0000 7fe203fff640 1 -- 172.21.15.32:0/1696836029 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe21c0ccb20 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe22190a640 1 -- 172.21.15.32:0/1696836029 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fe21c0ccb20 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe203fff640 1 -- 172.21.15.32:0/1696836029 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fe21c0ccb20 con 0x7fe21c0b0cb0 2023-12-20T04:29:09.885 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:29:09.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe22190a640 1 -- 172.21.15.32:0/1696836029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe208065a60 msgr2=0x7fe208067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe22190a640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe208065a60 0x7fe208067f20 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7fe20c000f70 tx=0x7fe20c007e90 comp rx=0 tx=0).stop 2023-12-20T04:29:09.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe22190a640 1 -- 172.21.15.32:0/1696836029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe21c0b0cb0 msgr2=0x7fe21c152090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:09.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe22190a640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe21c0b0cb0 0x7fe21c152090 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fe21001cd20 tx=0x7fe21001acc0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe22190a640 1 -- 172.21.15.32:0/1696836029 shutdown_connections 2023-12-20T04:29:09.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe22190a640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe208065a60 0x7fe208067f20 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe22190a640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe21c15abb0 0x7fe21c0cce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe22190a640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe21c0b0cb0 0x7fe21c152090 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe22190a640 1 --2- 172.21.15.32:0/1696836029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe21c070230 0x7fe21c14a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.883+0000 7fe22190a640 1 -- 172.21.15.32:0/1696836029 >> 172.21.15.32:0/1696836029 conn(0x7fe21c06a510 msgr2=0x7fe21c0c4b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.887+0000 7fe22190a640 1 -- 172.21.15.32:0/1696836029 shutdown_connections 2023-12-20T04:29:09.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:09.887+0000 7fe22190a640 1 -- 172.21.15.32:0/1696836029 wait complete. 2023-12-20T04:29:09.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T04:29:10.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.055+0000 7f96d8e2d640 1 Processor -- start 2023-12-20T04:29:10.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.055+0000 7f96d8e2d640 1 -- start start 2023-12-20T04:29:10.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.055+0000 7f96d8e2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d415ce30 0x7f96d4161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.055+0000 7f96d8e2d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f96d4070950 0x7f96d406d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.055+0000 7f96d8e2d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96d4071920 con 0x7f96d406dce0 2023-12-20T04:29:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.055+0000 7f96d8e2d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96d4071080 con 0x7f96d4070950 2023-12-20T04:29:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.055+0000 7f96d8e2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d4071360 con 0x7f96d415ce30 2023-12-20T04:29:10.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.055+0000 7f96d259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d415ce30 0x7f96d4161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d1d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f96d4070950 0x7f96d406d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d415ce30 0x7f96d4161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45330/0 (socket says 172.21.15.32:45330) 2023-12-20T04:29:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d259c640 1 -- 172.21.15.32:0/179424377 learned_addr learned my addr 172.21.15.32:0/179424377 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d2d9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f96d406dce0 0x7f96d4069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:49474/0 (socket says 172.21.15.32:49474) 2023-12-20T04:29:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d259c640 1 -- 172.21.15.32:0/179424377 >> v2:172.21.15.39:3300/0 conn(0x7f96d4070950 msgr2=0x7f96d406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d259c640 1 --2- 172.21.15.32:0/179424377 >> v2:172.21.15.39:3300/0 conn(0x7f96d4070950 0x7f96d406d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d259c640 1 -- 172.21.15.32:0/179424377 >> v1:172.21.15.39:6789/0 conn(0x7f96d406dce0 legacy=0x7f96d4069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d259c640 1 -- 172.21.15.32:0/179424377 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d4071640 con 0x7f96d415ce30 2023-12-20T04:29:10.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d1d9b640 1 --2- 172.21.15.32:0/179424377 >> v2:172.21.15.39:3300/0 conn(0x7f96d4070950 0x7f96d406d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:10.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d259c640 1 --2- 172.21.15.32:0/179424377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d415ce30 0x7f96d4161560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009870 tx=0x7f96c401c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3aab358891e3126f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d159a640 1 -- 172.21.15.32:0/179424377 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40ab650 con 0x7f96d415ce30 2023-12-20T04:29:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d159a640 1 -- 172.21.15.32:0/179424377 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96c40ab7b0 con 0x7f96d415ce30 2023-12-20T04:29:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.059+0000 7f96d159a640 1 -- 172.21.15.32:0/179424377 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96c40abb40 con 0x7f96d415ce30 2023-12-20T04:29:10.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 -- 172.21.15.32:0/179424377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d415ce30 msgr2=0x7f96d4161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 --2- 172.21.15.32:0/179424377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d415ce30 0x7f96d4161560 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f96c4009870 tx=0x7f96c401c540 comp rx=0 tx=0).stop 2023-12-20T04:29:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 -- 172.21.15.32:0/179424377 shutdown_connections 2023-12-20T04:29:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 --2- 172.21.15.32:0/179424377 >> v2:172.21.15.39:3300/0 conn(0x7f96d4070950 0x7f96d406d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 --2- 172.21.15.32:0/179424377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d415ce30 0x7f96d4161560 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 -- 172.21.15.32:0/179424377 >> 172.21.15.32:0/179424377 conn(0x7f96d40cba50 msgr2=0x7f96d40cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 -- 172.21.15.32:0/179424377 shutdown_connections 2023-12-20T04:29:10.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 -- 172.21.15.32:0/179424377 wait complete. 2023-12-20T04:29:10.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 Processor -- start 2023-12-20T04:29:10.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 -- start start 2023-12-20T04:29:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d406dce0 0x7f96d41593c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d4070950 0x7f96d4157530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96d415ce30 0x7f96d4157a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40b3480 con 0x7f96d415ce30 2023-12-20T04:29:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40b3760 con 0x7f96d4070950 2023-12-20T04:29:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d8e2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96d40b3a40 con 0x7f96d406dce0 2023-12-20T04:29:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96d415ce30 0x7f96d4157a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96d415ce30 0x7f96d4157a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59122/0 (socket says 172.21.15.32:59122) 2023-12-20T04:29:10.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d2d9d640 1 -- 172.21.15.32:0/1273790841 learned_addr learned my addr 172.21.15.32:0/1273790841 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:10.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.063+0000 7f96d259c640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d406dce0 0x7f96d41593c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96d1d9b640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d4070950 0x7f96d4157530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96d2d9d640 1 -- 172.21.15.32:0/1273790841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d406dce0 msgr2=0x7f96d41593c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96d2d9d640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d406dce0 0x7f96d41593c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96d2d9d640 1 -- 172.21.15.32:0/1273790841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d4070950 msgr2=0x7f96d4157530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96d2d9d640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d4070950 0x7f96d4157530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96d2d9d640 1 -- 172.21.15.32:0/1273790841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96c8009420 con 0x7f96d415ce30 2023-12-20T04:29:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96d259c640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d406dce0 0x7f96d41593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96d2d9d640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96d415ce30 0x7f96d4157a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00c7a0 tx=0x7f96cc00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96c2ffd640 1 -- 172.21.15.32:0/1273790841 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc00a640 con 0x7f96d415ce30 2023-12-20T04:29:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96c2ffd640 1 -- 172.21.15.32:0/1273790841 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96cc00a7a0 con 0x7f96d415ce30 2023-12-20T04:29:10.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96c2ffd640 1 -- 172.21.15.32:0/1273790841 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96cc00ab10 con 0x7f96d415ce30 2023-12-20T04:29:10.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96d8e2d640 1 -- 172.21.15.32:0/1273790841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96c40094d0 con 0x7f96d415ce30 2023-12-20T04:29:10.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96d8e2d640 1 -- 172.21.15.32:0/1273790841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96d40b3130 con 0x7f96d415ce30 2023-12-20T04:29:10.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96d8e2d640 1 -- 172.21.15.32:0/1273790841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96d404e000 con 0x7f96d415ce30 2023-12-20T04:29:10.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.067+0000 7f96c2ffd640 1 -- 172.21.15.32:0/1273790841 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f96cc00ac70 con 0x7f96d415ce30 2023-12-20T04:29:10.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.071+0000 7f96c2ffd640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f96b4065a90 0x7f96b4067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.071+0000 7f96c2ffd640 1 -- 172.21.15.32:0/1273790841 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f96cc082990 con 0x7f96d415ce30 2023-12-20T04:29:10.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.071+0000 7f96d259c640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f96b4065a90 0x7f96b4067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.071+0000 7f96d259c640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f96b4065a90 0x7f96b4067f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f96c40af000 tx=0x7f96c40b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:10.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.075+0000 7f96c2ffd640 1 -- 172.21.15.32:0/1273790841 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f96cc050360 con 0x7f96d415ce30 2023-12-20T04:29:10.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.243+0000 7f96d8e2d640 1 -- 172.21.15.32:0/1273790841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f96d40b35c0 con 0x7f96d415ce30 2023-12-20T04:29:10.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.243+0000 7f96c2ffd640 1 -- 172.21.15.32:0/1273790841 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f96d40b35c0 con 0x7f96d415ce30 2023-12-20T04:29:10.247 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-12-20T04:29:10.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 -- 172.21.15.32:0/1273790841 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f96b4065a90 msgr2=0x7f96b4067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f96b4065a90 0x7f96b4067f50 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f96c40af000 tx=0x7f96c40b6040 comp rx=0 tx=0).stop 2023-12-20T04:29:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 -- 172.21.15.32:0/1273790841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96d415ce30 msgr2=0x7f96d4157a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96d415ce30 0x7f96d4157a70 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f96cc00c7a0 tx=0x7f96cc00cc60 comp rx=0 tx=0).stop 2023-12-20T04:29:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 -- 172.21.15.32:0/1273790841 shutdown_connections 2023-12-20T04:29:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f96b4065a90 0x7f96b4067f50 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96d415ce30 0x7f96d4157a70 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96d4070950 0x7f96d4157530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 --2- 172.21.15.32:0/1273790841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96d406dce0 0x7f96d41593c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 -- 172.21.15.32:0/1273790841 >> 172.21.15.32:0/1273790841 conn(0x7f96d40cba50 msgr2=0x7f96d415f4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 -- 172.21.15.32:0/1273790841 shutdown_connections 2023-12-20T04:29:10.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.247+0000 7f96d8e2d640 1 -- 172.21.15.32:0/1273790841 wait complete. 2023-12-20T04:29:10.267 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T04:29:10.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:10.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 -f 2023-12-20T04:29:10.300 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:29:10.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:10.301 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T04:29:10.367 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:29:10.367 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:10.377 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:29:10.378 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:10.435 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:29:10.435 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:10.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:10.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.627+0000 7f74cc909640 1 Processor -- start 2023-12-20T04:29:10.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.627+0000 7f74cc909640 1 -- start start 2023-12-20T04:29:10.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74cc909640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80aece0 0x7f74c80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74cc909640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74c80b0b30 0x7f74c80701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74cc909640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c80c98c0 con 0x7f74c81599e0 2023-12-20T04:29:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74cc909640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c80c9ba0 con 0x7f74c80b0b30 2023-12-20T04:29:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74cc909640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80c9e80 con 0x7f74c80aece0 2023-12-20T04:29:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74c6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74c80b0b30 0x7f74c80701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74c659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f74c81599e0 0x7f74c80c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57430/0 (socket says 172.21.15.32:57430) 2023-12-20T04:29:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74c659c640 1 -- 172.21.15.32:0/1468185280 learned_addr learned my addr 172.21.15.32:0/1468185280 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74c6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74c80b0b30 0x7f74c80701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59130/0 (socket says 172.21.15.32:59130) 2023-12-20T04:29:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74bffff640 1 --2- 172.21.15.32:0/1468185280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80aece0 0x7f74c80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74c6d9d640 1 -- 172.21.15.32:0/1468185280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80aece0 msgr2=0x7f74c80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:10.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74c6d9d640 1 --2- 172.21.15.32:0/1468185280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80aece0 0x7f74c80b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74c6d9d640 1 -- 172.21.15.32:0/1468185280 >> v1:172.21.15.32:6789/0 conn(0x7f74c81599e0 legacy=0x7f74c80c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74c6d9d640 1 -- 172.21.15.32:0/1468185280 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80ca160 con 0x7f74c80b0b30 2023-12-20T04:29:10.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.631+0000 7f74c5d9b640 1 -- 172.21.15.32:0/1468185280 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2484259703 0 0) 0x7f74c80c98c0 con 0x7f74c81599e0 2023-12-20T04:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74c6d9d640 1 --2- 172.21.15.32:0/1468185280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80b0b30 0x7f74c80701c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009d90 tx=0x7f74c001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b845bcf1f3db3e29 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:10.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74c5d9b640 1 -- 172.21.15.32:0/1468185280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00a5920 con 0x7f74c80b0b30 2023-12-20T04:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74c5d9b640 1 -- 172.21.15.32:0/1468185280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c00a5a80 con 0x7f74c80b0b30 2023-12-20T04:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74c5d9b640 1 -- 172.21.15.32:0/1468185280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00a5e10 con 0x7f74c80b0b30 2023-12-20T04:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74cc909640 1 -- 172.21.15.32:0/1468185280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80b0b30 msgr2=0x7f74c80701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74cc909640 1 --2- 172.21.15.32:0/1468185280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80b0b30 0x7f74c80701c0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009d90 tx=0x7f74c001c950 comp rx=0 tx=0).stop 2023-12-20T04:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74cc909640 1 -- 172.21.15.32:0/1468185280 shutdown_connections 2023-12-20T04:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74cc909640 1 --2- 172.21.15.32:0/1468185280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80b0b30 0x7f74c80701c0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74cc909640 1 --2- 172.21.15.32:0/1468185280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80aece0 0x7f74c80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74cc909640 1 -- 172.21.15.32:0/1468185280 >> 172.21.15.32:0/1468185280 conn(0x7f74c806a320 msgr2=0x7f74c80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74cc909640 1 -- 172.21.15.32:0/1468185280 shutdown_connections 2023-12-20T04:29:10.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74cc909640 1 -- 172.21.15.32:0/1468185280 wait complete. 2023-12-20T04:29:10.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74cc909640 1 Processor -- start 2023-12-20T04:29:10.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.635+0000 7f74cc909640 1 -- start start 2023-12-20T04:29:10.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74cc909640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80aece0 0x7f74c815a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74cc909640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c80b0b30 0x7f74c815aaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74cc909640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c81599e0 0x7f74c81588d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74cc909640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c8150fe0 con 0x7f74c80aece0 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74cc909640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c815afe0 con 0x7f74c80b0b30 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74cc909640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81528c0 con 0x7f74c81599e0 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74c659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80aece0 0x7f74c815a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74c659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80aece0 0x7f74c815a160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59142/0 (socket says 172.21.15.32:59142) 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74c659c640 1 -- 172.21.15.32:0/2201588273 learned_addr learned my addr 172.21.15.32:0/2201588273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74bffff640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c80b0b30 0x7f74c815aaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74c6d9d640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c81599e0 0x7f74c81588d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74c659c640 1 -- 172.21.15.32:0/2201588273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c81599e0 msgr2=0x7f74c81588d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74c659c640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c81599e0 0x7f74c81588d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74c659c640 1 -- 172.21.15.32:0/2201588273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c80b0b30 msgr2=0x7f74c815aaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74c659c640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c80b0b30 0x7f74c815aaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74c659c640 1 -- 172.21.15.32:0/2201588273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c0009a70 con 0x7f74c80aece0 2023-12-20T04:29:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74bffff640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c80b0b30 0x7f74c815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74c6d9d640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c81599e0 0x7f74c81588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74c659c640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80aece0 0x7f74c815a160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f74b80010f0 tx=0x7f74b800c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74beffd640 1 -- 172.21.15.32:0/2201588273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80074e0 con 0x7f74c80aece0 2023-12-20T04:29:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74beffd640 1 -- 172.21.15.32:0/2201588273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b8007640 con 0x7f74c80aece0 2023-12-20T04:29:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74cc909640 1 -- 172.21.15.32:0/2201588273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c8152ba0 con 0x7f74c80aece0 2023-12-20T04:29:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74beffd640 1 -- 172.21.15.32:0/2201588273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b80059d0 con 0x7f74c80aece0 2023-12-20T04:29:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74cc909640 1 -- 172.21.15.32:0/2201588273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c8153160 con 0x7f74c80aece0 2023-12-20T04:29:10.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74cc909640 1 -- 172.21.15.32:0/2201588273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c8153320 con 0x7f74c80aece0 2023-12-20T04:29:10.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.639+0000 7f74beffd640 1 -- 172.21.15.32:0/2201588273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f74b8005b30 con 0x7f74c80aece0 2023-12-20T04:29:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.643+0000 7f74beffd640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74a8065ab0 0x7f74a8067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.643+0000 7f74beffd640 1 -- 172.21.15.32:0/2201588273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f74b8082750 con 0x7f74c80aece0 2023-12-20T04:29:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.643+0000 7f74bffff640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74a8065ab0 0x7f74a8067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.643+0000 7f74bffff640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74a8065ab0 0x7f74a8067f70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f74c8150f70 tx=0x7f74b0008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:10.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.647+0000 7f74beffd640 1 -- 172.21.15.32:0/2201588273 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f74b8050180 con 0x7f74c80aece0 2023-12-20T04:29:10.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.803+0000 7f74cc909640 1 -- 172.21.15.32:0/2201588273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74c804e000 con 0x7f74c80aece0 2023-12-20T04:29:10.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.803+0000 7f74beffd640 1 -- 172.21.15.32:0/2201588273 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f74b80537a0 con 0x7f74c80aece0 2023-12-20T04:29:10.807 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:10.807 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 -- 172.21.15.32:0/2201588273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74a8065ab0 msgr2=0x7f74a8067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74a8065ab0 0x7f74a8067f70 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f74c8150f70 tx=0x7f74b0008040 comp rx=0 tx=0).stop 2023-12-20T04:29:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 -- 172.21.15.32:0/2201588273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80aece0 msgr2=0x7f74c815a160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80aece0 0x7f74c815a160 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7f74b80010f0 tx=0x7f74b800c8b0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 -- 172.21.15.32:0/2201588273 shutdown_connections 2023-12-20T04:29:10.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74a8065ab0 0x7f74a8067f70 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c81599e0 0x7f74c81588d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74c80b0b30 0x7f74c815aaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 --2- 172.21.15.32:0/2201588273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80aece0 0x7f74c815a160 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 -- 172.21.15.32:0/2201588273 >> 172.21.15.32:0/2201588273 conn(0x7f74c806a320 msgr2=0x7f74c80b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 -- 172.21.15.32:0/2201588273 shutdown_connections 2023-12-20T04:29:10.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.807+0000 7f74cc909640 1 -- 172.21.15.32:0/2201588273 wait complete. 2023-12-20T04:29:10.828 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:28:36.040195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:28:37.364671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:29:10.829 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:29:10.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f4979fbc640 1 Processor -- start 2023-12-20T04:29:10.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f4979fbc640 1 -- start start 2023-12-20T04:29:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f4979fbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4974157480 0x7f49740c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f4979fbc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f49740b0760 0x7f49740b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f4979fbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f497415d660 0x7f497415dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f4979fbc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49740b1100 con 0x7f4974157480 2023-12-20T04:29:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f4979fbc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49740cde20 con 0x7f49740b0760 2023-12-20T04:29:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f4979fbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49740c8e30 con 0x7f497415d660 2023-12-20T04:29:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f49737fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4974157480 0x7f49740c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f49737fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4974157480 0x7f49740c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59156/0 (socket says 172.21.15.32:59156) 2023-12-20T04:29:10.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f49737fe640 1 -- 172.21.15.32:0/3715860105 learned_addr learned my addr 172.21.15.32:0/3715860105 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f49737fe640 1 -- 172.21.15.32:0/3715860105 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f497415d660 msgr2=0x7f497415dad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f49737fe640 1 --2- 172.21.15.32:0/3715860105 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f497415d660 0x7f497415dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f49737fe640 1 -- 172.21.15.32:0/3715860105 >> v2:172.21.15.39:3300/0 conn(0x7f49740b0760 msgr2=0x7f49740b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f49737fe640 1 --2- 172.21.15.32:0/3715860105 >> v2:172.21.15.39:3300/0 conn(0x7f49740b0760 0x7f49740b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.975+0000 7f49737fe640 1 -- 172.21.15.32:0/3715860105 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49740ce070 con 0x7f4974157480 2023-12-20T04:29:10.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f49737fe640 1 --2- 172.21.15.32:0/3715860105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4974157480 0x7f49740c9840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f497406e0d0 tx=0x7f496401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f7e4a199a6dd002 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:10.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f49727fc640 1 -- 172.21.15.32:0/3715860105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49640ab600 con 0x7f4974157480 2023-12-20T04:29:10.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f49727fc640 1 -- 172.21.15.32:0/3715860105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49640ab760 con 0x7f4974157480 2023-12-20T04:29:10.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f49727fc640 1 -- 172.21.15.32:0/3715860105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49640abaf0 con 0x7f4974157480 2023-12-20T04:29:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f4979fbc640 1 -- 172.21.15.32:0/3715860105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4974157480 msgr2=0x7f49740c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f4979fbc640 1 --2- 172.21.15.32:0/3715860105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4974157480 0x7f49740c9840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f497406e0d0 tx=0x7f496401c440 comp rx=0 tx=0).stop 2023-12-20T04:29:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f4979fbc640 1 -- 172.21.15.32:0/3715860105 shutdown_connections 2023-12-20T04:29:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f4979fbc640 1 --2- 172.21.15.32:0/3715860105 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f497415d660 0x7f497415dad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f4979fbc640 1 --2- 172.21.15.32:0/3715860105 >> v2:172.21.15.39:3300/0 conn(0x7f49740b0760 0x7f49740b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f4979fbc640 1 --2- 172.21.15.32:0/3715860105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4974157480 0x7f49740c9840 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f4979fbc640 1 -- 172.21.15.32:0/3715860105 >> 172.21.15.32:0/3715860105 conn(0x7f497406a510 msgr2=0x7f49740ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f4979fbc640 1 -- 172.21.15.32:0/3715860105 shutdown_connections 2023-12-20T04:29:10.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.979+0000 7f4979fbc640 1 -- 172.21.15.32:0/3715860105 wait complete. 2023-12-20T04:29:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4979fbc640 1 Processor -- start 2023-12-20T04:29:10.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4979fbc640 1 -- start start 2023-12-20T04:29:10.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4979fbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49740b0760 0x7f49740b0180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4979fbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4974157480 0x7f497414e2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4979fbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f497415d660 0x7f49740c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4979fbc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49740b32f0 con 0x7f497415d660 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4979fbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f497414e810 con 0x7f4974157480 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4979fbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49740c8740 con 0x7f49740b0760 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4973fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f497415d660 0x7f49740c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f49737fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49740b0760 0x7f49740b0180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4973fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f497415d660 0x7f49740c6250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59162/0 (socket says 172.21.15.32:59162) 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f49737fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49740b0760 0x7f49740b0180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45360/0 (socket says 172.21.15.32:45360) 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4973fff640 1 -- 172.21.15.32:0/3346931560 learned_addr learned my addr 172.21.15.32:0/3346931560 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4972ffd640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4974157480 0x7f497414e2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4973fff640 1 -- 172.21.15.32:0/3346931560 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49740b0760 msgr2=0x7f49740b0180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4973fff640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49740b0760 0x7f49740b0180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4973fff640 1 -- 172.21.15.32:0/3346931560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4974157480 msgr2=0x7f497414e2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4973fff640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4974157480 0x7f497414e2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:10.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4973fff640 1 -- 172.21.15.32:0/3346931560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49640094d0 con 0x7f497415d660 2023-12-20T04:29:10.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f4973fff640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f497415d660 0x7f49740c6250 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f496c00b970 tx=0x7f496c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:10.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f495bfff640 1 -- 172.21.15.32:0/3346931560 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c00cb40 con 0x7f497415d660 2023-12-20T04:29:10.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.983+0000 7f495bfff640 1 -- 172.21.15.32:0/3346931560 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f496c00ccc0 con 0x7f497415d660 2023-12-20T04:29:10.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.987+0000 7f495bfff640 1 -- 172.21.15.32:0/3346931560 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f496c014680 con 0x7f497415d660 2023-12-20T04:29:10.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.987+0000 7f4979fbc640 1 -- 172.21.15.32:0/3346931560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49740c8a20 con 0x7f497415d660 2023-12-20T04:29:10.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.987+0000 7f4979fbc640 1 -- 172.21.15.32:0/3346931560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4974156b90 con 0x7f497415d660 2023-12-20T04:29:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.987+0000 7f4979fbc640 1 -- 172.21.15.32:0/3346931560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f497404e000 con 0x7f497415d660 2023-12-20T04:29:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.987+0000 7f495bfff640 1 -- 172.21.15.32:0/3346931560 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f496c007370 con 0x7f497415d660 2023-12-20T04:29:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.987+0000 7f495bfff640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4960065a60 0x7f4960067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.987+0000 7f495bfff640 1 -- 172.21.15.32:0/3346931560 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f496c0830e0 con 0x7f497415d660 2023-12-20T04:29:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.987+0000 7f49737fe640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4960065a60 0x7f4960067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:10.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.987+0000 7f49737fe640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4960065a60 0x7f4960067f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f49640af000 tx=0x7f49640b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:10.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:10.991+0000 7f495bfff640 1 -- 172.21.15.32:0/3346931560 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f496c050ab0 con 0x7f497415d660 2023-12-20T04:29:11.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.151+0000 7f4979fbc640 1 -- 172.21.15.32:0/3346931560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f49740b3770 con 0x7f497415d660 2023-12-20T04:29:11.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.151+0000 7f495bfff640 1 -- 172.21.15.32:0/3346931560 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f496c0190a0 con 0x7f497415d660 2023-12-20T04:29:11.155 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:29:11.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 -- 172.21.15.32:0/3346931560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4960065a60 msgr2=0x7f4960067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4960065a60 0x7f4960067f20 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f49640af000 tx=0x7f49640b6040 comp rx=0 tx=0).stop 2023-12-20T04:29:11.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 -- 172.21.15.32:0/3346931560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f497415d660 msgr2=0x7f49740c6250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f497415d660 0x7f49740c6250 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f496c00b970 tx=0x7f496c00be30 comp rx=0 tx=0).stop 2023-12-20T04:29:11.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 -- 172.21.15.32:0/3346931560 shutdown_connections 2023-12-20T04:29:11.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4960065a60 0x7f4960067f20 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f497415d660 0x7f49740c6250 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4974157480 0x7f497414e2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 --2- 172.21.15.32:0/3346931560 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49740b0760 0x7f49740b0180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 -- 172.21.15.32:0/3346931560 >> 172.21.15.32:0/3346931560 conn(0x7f497406a510 msgr2=0x7f49740cccc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:11.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 -- 172.21.15.32:0/3346931560 shutdown_connections 2023-12-20T04:29:11.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.155+0000 7f4979fbc640 1 -- 172.21.15.32:0/3346931560 wait complete. 2023-12-20T04:29:11.176 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:29:11.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.319+0000 7fc5d4c2a640 1 Processor -- start 2023-12-20T04:29:11.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.319+0000 7fc5d4c2a640 1 -- start start 2023-12-20T04:29:11.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5d4c2a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5d015ccf0 0x7fc5d00cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5d4c2a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc5d00cd7b0 0x7fc5d00cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5d4c2a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5d00d10f0 con 0x7fc5d00cf920 2023-12-20T04:29:11.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5d4c2a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d00cfdb0 con 0x7fc5d015ccf0 2023-12-20T04:29:11.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5d4c2a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5d00ce2f0 con 0x7fc5d00cd7b0 2023-12-20T04:29:11.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5cf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5d015ccf0 0x7fc5d00cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5cf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc5d015ccf0 0x7fc5d00cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59178/0 (socket says 172.21.15.32:59178) 2023-12-20T04:29:11.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5cf7fe640 1 -- 172.21.15.32:0/10559687 learned_addr learned my addr 172.21.15.32:0/10559687 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:11.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5cf7fe640 1 -- 172.21.15.32:0/10559687 >> v2:172.21.15.39:3300/0 conn(0x7fc5d00cd7b0 msgr2=0x7fc5d00cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:11.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5ceffd640 1 --2- 172.21.15.32:0/10559687 >> v2:172.21.15.39:3300/0 conn(0x7fc5d00cd7b0 0x7fc5d00cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5cf7fe640 1 --2- 172.21.15.32:0/10559687 >> v2:172.21.15.39:3300/0 conn(0x7fc5d00cd7b0 0x7fc5d00cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5cf7fe640 1 -- 172.21.15.32:0/10559687 >> v1:172.21.15.39:6789/0 conn(0x7fc5d00cf920 legacy=0x7fc5d00b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:11.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5cf7fe640 1 -- 172.21.15.32:0/10559687 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5d00cdd30 con 0x7fc5d015ccf0 2023-12-20T04:29:11.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.323+0000 7fc5ceffd640 1 --2- 172.21.15.32:0/10559687 >> v2:172.21.15.39:3300/0 conn(0x7fc5d00cd7b0 0x7fc5d00cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:11.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5cf7fe640 1 --2- 172.21.15.32:0/10559687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5d015ccf0 0x7fc5d00cd270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00ccfc0 tx=0x7fc5c001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab09d9377b2610 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:11.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5ce7fc640 1 -- 172.21.15.32:0/10559687 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00ab600 con 0x7fc5d015ccf0 2023-12-20T04:29:11.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5ce7fc640 1 -- 172.21.15.32:0/10559687 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c00a7050 con 0x7fc5d015ccf0 2023-12-20T04:29:11.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5ce7fc640 1 -- 172.21.15.32:0/10559687 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c00ab980 con 0x7fc5d015ccf0 2023-12-20T04:29:11.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/10559687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5d015ccf0 msgr2=0x7fc5d00cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 --2- 172.21.15.32:0/10559687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5d015ccf0 0x7fc5d00cd270 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7fc5d00ccfc0 tx=0x7fc5c001c430 comp rx=0 tx=0).stop 2023-12-20T04:29:11.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/10559687 shutdown_connections 2023-12-20T04:29:11.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 --2- 172.21.15.32:0/10559687 >> v2:172.21.15.39:3300/0 conn(0x7fc5d00cd7b0 0x7fc5d00cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 --2- 172.21.15.32:0/10559687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5d015ccf0 0x7fc5d00cd270 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/10559687 >> 172.21.15.32:0/10559687 conn(0x7fc5d00658e0 msgr2=0x7fc5d0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:11.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/10559687 shutdown_connections 2023-12-20T04:29:11.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/10559687 wait complete. 2023-12-20T04:29:11.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 Processor -- start 2023-12-20T04:29:11.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 -- start start 2023-12-20T04:29:11.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5d00cd7b0 0x7fc5d0145e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc5d00cf920 0x7fc5d00b6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5d015ccf0 0x7fc5d00b7260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00c9f80 con 0x7fc5d015ccf0 2023-12-20T04:29:11.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00c5460 con 0x7fc5d00cd7b0 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.327+0000 7fc5d4c2a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5d00ca190 con 0x7fc5d00cf920 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5d015ccf0 0x7fc5d00b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5d015ccf0 0x7fc5d00b7260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59194/0 (socket says 172.21.15.32:59194) 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5cffff640 1 -- 172.21.15.32:0/1927293149 learned_addr learned my addr 172.21.15.32:0/1927293149 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5cf7fe640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5d00cd7b0 0x7fc5d0145e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5ceffd640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc5d00cf920 0x7fc5d00b6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5cffff640 1 -- 172.21.15.32:0/1927293149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc5d00cf920 msgr2=0x7fc5d00b6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5cffff640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc5d00cf920 0x7fc5d00b6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5cffff640 1 -- 172.21.15.32:0/1927293149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5d00cd7b0 msgr2=0x7fc5d0145e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5cffff640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5d00cd7b0 0x7fc5d0145e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5cffff640 1 -- 172.21.15.32:0/1927293149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5c00094d0 con 0x7fc5d015ccf0 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5ceffd640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc5d00cf920 0x7fc5d00b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:11.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5cffff640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5d015ccf0 0x7fc5d00b7260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c780 tx=0x7fc5c800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5b7fff640 1 -- 172.21.15.32:0/1927293149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c8004b70 con 0x7fc5d015ccf0 2023-12-20T04:29:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5b7fff640 1 -- 172.21.15.32:0/1927293149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5c800ce90 con 0x7fc5d015ccf0 2023-12-20T04:29:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5b7fff640 1 -- 172.21.15.32:0/1927293149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5c800a850 con 0x7fc5d015ccf0 2023-12-20T04:29:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/1927293149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5d00c59d0 con 0x7fc5d015ccf0 2023-12-20T04:29:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/1927293149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5d00cc860 con 0x7fc5d015ccf0 2023-12-20T04:29:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/1927293149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5d0074100 con 0x7fc5d015ccf0 2023-12-20T04:29:11.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.331+0000 7fc5b7fff640 1 -- 172.21.15.32:0/1927293149 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc5c800a9b0 con 0x7fc5d015ccf0 2023-12-20T04:29:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.335+0000 7fc5b7fff640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5bc065ab0 0x7fc5bc067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.335+0000 7fc5b7fff640 1 -- 172.21.15.32:0/1927293149 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc5c80831a0 con 0x7fc5d015ccf0 2023-12-20T04:29:11.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.335+0000 7fc5cf7fe640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5bc065ab0 0x7fc5bc067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.335+0000 7fc5cf7fe640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5bc065ab0 0x7fc5bc067f70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00ad000 tx=0x7fc5c00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:11.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.339+0000 7fc5b7fff640 1 -- 172.21.15.32:0/1927293149 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc5c8050c10 con 0x7fc5d015ccf0 2023-12-20T04:29:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.495+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/1927293149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc5d0160a70 con 0x7fc5d015ccf0 2023-12-20T04:29:11.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.495+0000 7fc5b7fff640 1 -- 172.21.15.32:0/1927293149 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc5c8054230 con 0x7fc5d015ccf0 2023-12-20T04:29:11.499 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:29:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/1927293149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5bc065ab0 msgr2=0x7fc5bc067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5bc065ab0 0x7fc5bc067f70 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fc5c00ad000 tx=0x7fc5c00a6000 comp rx=0 tx=0).stop 2023-12-20T04:29:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/1927293149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5d015ccf0 msgr2=0x7fc5d00b7260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5d015ccf0 0x7fc5d00b7260 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7fc5c800c780 tx=0x7fc5c800cc40 comp rx=0 tx=0).stop 2023-12-20T04:29:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/1927293149 shutdown_connections 2023-12-20T04:29:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc5bc065ab0 0x7fc5bc067f70 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5d015ccf0 0x7fc5d00b7260 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc5d00cf920 0x7fc5d00b6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 --2- 172.21.15.32:0/1927293149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5d00cd7b0 0x7fc5d0145e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/1927293149 >> 172.21.15.32:0/1927293149 conn(0x7fc5d00658e0 msgr2=0x7fc5d00b5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/1927293149 shutdown_connections 2023-12-20T04:29:11.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.499+0000 7fc5d4c2a640 1 -- 172.21.15.32:0/1927293149 wait complete. 2023-12-20T04:29:11.520 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:29:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.663+0000 7fab36b60640 1 Processor -- start 2023-12-20T04:29:11.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.663+0000 7fab36b60640 1 -- start start 2023-12-20T04:29:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab36b60640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab30065d40 0x7fab300cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab36b60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab3006d170 0x7fab30161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab36b60640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab3006a1f0 con 0x7fab300cb050 2023-12-20T04:29:11.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab36b60640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab30161860 con 0x7fab30065d40 2023-12-20T04:29:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab36b60640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab3006a440 con 0x7fab3006d170 2023-12-20T04:29:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab2ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab30065d40 0x7fab300cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab2ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab30065d40 0x7fab300cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59196/0 (socket says 172.21.15.32:59196) 2023-12-20T04:29:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab2ffff640 1 -- 172.21.15.32:0/3677842104 learned_addr learned my addr 172.21.15.32:0/3677842104 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab350fd640 1 --2- 172.21.15.32:0/3677842104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab3006d170 0x7fab30161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab2ffff640 1 -- 172.21.15.32:0/3677842104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab3006d170 msgr2=0x7fab30161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab2ffff640 1 --2- 172.21.15.32:0/3677842104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab3006d170 0x7fab30161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab2ffff640 1 -- 172.21.15.32:0/3677842104 >> v1:172.21.15.39:6789/0 conn(0x7fab300cb050 legacy=0x7fab3006ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab2ffff640 1 -- 172.21.15.32:0/3677842104 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab3006a720 con 0x7fab30065d40 2023-12-20T04:29:11.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.667+0000 7fab2f7fe640 1 -- 172.21.15.32:0/3677842104 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3739701054 0 0) 0x7fab3006a1f0 con 0x7fab300cb050 2023-12-20T04:29:11.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab2ffff640 1 --2- 172.21.15.32:0/3677842104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab30065d40 0x7fab300cb430 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009870 tx=0x7fab1c01c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52c0c04b0e1f5229 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab2f7fe640 1 -- 172.21.15.32:0/3677842104 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0aa670 con 0x7fab30065d40 2023-12-20T04:29:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab2f7fe640 1 -- 172.21.15.32:0/3677842104 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab1c0aa7d0 con 0x7fab30065d40 2023-12-20T04:29:11.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab2f7fe640 1 -- 172.21.15.32:0/3677842104 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0aab60 con 0x7fab30065d40 2023-12-20T04:29:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab36b60640 1 -- 172.21.15.32:0/3677842104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab30065d40 msgr2=0x7fab300cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab36b60640 1 --2- 172.21.15.32:0/3677842104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab30065d40 0x7fab300cb430 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab1c009870 tx=0x7fab1c01c510 comp rx=0 tx=0).stop 2023-12-20T04:29:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab36b60640 1 -- 172.21.15.32:0/3677842104 shutdown_connections 2023-12-20T04:29:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab36b60640 1 --2- 172.21.15.32:0/3677842104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab3006d170 0x7fab30161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab36b60640 1 --2- 172.21.15.32:0/3677842104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab30065d40 0x7fab300cb430 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab36b60640 1 -- 172.21.15.32:0/3677842104 >> 172.21.15.32:0/3677842104 conn(0x7fab300cf8d0 msgr2=0x7fab300cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab36b60640 1 -- 172.21.15.32:0/3677842104 shutdown_connections 2023-12-20T04:29:11.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab36b60640 1 -- 172.21.15.32:0/3677842104 wait complete. 2023-12-20T04:29:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab36b60640 1 Processor -- start 2023-12-20T04:29:11.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.671+0000 7fab36b60640 1 -- start start 2023-12-20T04:29:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab36b60640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab30065d40 0x7fab300c7970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab36b60640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab3006d170 0x7fab30150390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab36b60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab300cb050 0x7fab300c32a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab36b60640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab3014ff70 con 0x7fab30065d40 2023-12-20T04:29:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab36b60640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab300c7eb0 con 0x7fab3006d170 2023-12-20T04:29:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab36b60640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab300c6f30 con 0x7fab300cb050 2023-12-20T04:29:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab348fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab30065d40 0x7fab300c7970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab348fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab30065d40 0x7fab300c7970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59202/0 (socket says 172.21.15.32:59202) 2023-12-20T04:29:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab348fc640 1 -- 172.21.15.32:0/3524662923 learned_addr learned my addr 172.21.15.32:0/3524662923 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab350fd640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab300cb050 0x7fab300c32a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab2ffff640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab3006d170 0x7fab30150390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab348fc640 1 -- 172.21.15.32:0/3524662923 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab300cb050 msgr2=0x7fab300c32a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab348fc640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab300cb050 0x7fab300c32a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab348fc640 1 -- 172.21.15.32:0/3524662923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab3006d170 msgr2=0x7fab30150390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab348fc640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab3006d170 0x7fab30150390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab348fc640 1 -- 172.21.15.32:0/3524662923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab1c0094d0 con 0x7fab30065d40 2023-12-20T04:29:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab348fc640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab30065d40 0x7fab300c7970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab2400c4d0 tx=0x7fab2400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab2d7fa640 1 -- 172.21.15.32:0/3524662923 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab24005480 con 0x7fab30065d40 2023-12-20T04:29:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab2d7fa640 1 -- 172.21.15.32:0/3524662923 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab24003600 con 0x7fab30065d40 2023-12-20T04:29:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab36b60640 1 -- 172.21.15.32:0/3524662923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab300c7210 con 0x7fab30065d40 2023-12-20T04:29:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab2d7fa640 1 -- 172.21.15.32:0/3524662923 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab240039b0 con 0x7fab30065d40 2023-12-20T04:29:11.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab36b60640 1 -- 172.21.15.32:0/3524662923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab3014fb90 con 0x7fab30065d40 2023-12-20T04:29:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab36b60640 1 -- 172.21.15.32:0/3524662923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab301501e0 con 0x7fab30065d40 2023-12-20T04:29:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.675+0000 7fab2d7fa640 1 -- 172.21.15.32:0/3524662923 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fab24003b10 con 0x7fab30065d40 2023-12-20T04:29:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.679+0000 7fab2d7fa640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab14065a60 0x7fab14067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:11.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.679+0000 7fab2d7fa640 1 -- 172.21.15.32:0/3524662923 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fab24082840 con 0x7fab30065d40 2023-12-20T04:29:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.679+0000 7fab2ffff640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab14065a60 0x7fab14067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:11.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.679+0000 7fab2ffff640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab14065a60 0x7fab14067f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0ae000 tx=0x7fab1c0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:11.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.683+0000 7fab2d7fa640 1 -- 172.21.15.32:0/3524662923 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fab24050220 con 0x7fab30065d40 2023-12-20T04:29:11.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.839+0000 7fab36b60640 1 -- 172.21.15.32:0/3524662923 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fab3004e000 con 0x7fab30065d40 2023-12-20T04:29:11.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab2d7fa640 1 -- 172.21.15.32:0/3524662923 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fab24017020 con 0x7fab30065d40 2023-12-20T04:29:11.844 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:29:11.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 -- 172.21.15.32:0/3524662923 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab14065a60 msgr2=0x7fab14067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab14065a60 0x7fab14067f20 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7fab1c0ae000 tx=0x7fab1c0a7000 comp rx=0 tx=0).stop 2023-12-20T04:29:11.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 -- 172.21.15.32:0/3524662923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab30065d40 msgr2=0x7fab300c7970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:11.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab30065d40 0x7fab300c7970 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7fab2400c4d0 tx=0x7fab2400c990 comp rx=0 tx=0).stop 2023-12-20T04:29:11.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 -- 172.21.15.32:0/3524662923 shutdown_connections 2023-12-20T04:29:11.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab14065a60 0x7fab14067f20 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab300cb050 0x7fab300c32a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab3006d170 0x7fab30150390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 --2- 172.21.15.32:0/3524662923 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab30065d40 0x7fab300c7970 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:11.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 -- 172.21.15.32:0/3524662923 >> 172.21.15.32:0/3524662923 conn(0x7fab300cf8d0 msgr2=0x7fab300c82e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:11.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 -- 172.21.15.32:0/3524662923 shutdown_connections 2023-12-20T04:29:11.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:11.843+0000 7fab36b60640 1 -- 172.21.15.32:0/3524662923 wait complete. 2023-12-20T04:29:11.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T04:29:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.007+0000 7fc8012f2640 1 Processor -- start 2023-12-20T04:29:12.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.007+0000 7fc8012f2640 1 -- start start 2023-12-20T04:29:12.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc8012f2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7fc15ce30 0x7fc7fc161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc8012f2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7fc071920 con 0x7fc7fc06deb0 2023-12-20T04:29:12.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc8012f2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7fc071080 con 0x7fc7fc070950 2023-12-20T04:29:12.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc8012f2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7fc071360 con 0x7fc7fc15ce30 2023-12-20T04:29:12.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc7fb7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc7fc06deb0 0x7fc7fc0cd520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57438/0 (socket says 172.21.15.32:57438) 2023-12-20T04:29:12.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc7fb7fe640 1 -- 172.21.15.32:0/2707517030 learned_addr learned my addr 172.21.15.32:0/2707517030 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:12.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc7faffd640 1 --2- 172.21.15.32:0/2707517030 >> v2:172.21.15.39:3300/0 conn(0x7fc7fc15ce30 0x7fc7fc161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc7fa7fc640 1 -- 172.21.15.32:0/2707517030 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510215018 0 0) 0x7fc7fc071920 con 0x7fc7fc06deb0 2023-12-20T04:29:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc7fa7fc640 1 -- 172.21.15.32:0/2707517030 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc7e8003530 con 0x7fc7fc06deb0 2023-12-20T04:29:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc7faffd640 1 -- 172.21.15.32:0/2707517030 >> v1:172.21.15.39:6789/0 conn(0x7fc7fc070950 legacy=0x7fc7fc06d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc7faffd640 1 -- 172.21.15.32:0/2707517030 >> v1:172.21.15.32:6789/0 conn(0x7fc7fc06deb0 legacy=0x7fc7fc0cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.011+0000 7fc7faffd640 1 -- 172.21.15.32:0/2707517030 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7fc071640 con 0x7fc7fc15ce30 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc7faffd640 1 --2- 172.21.15.32:0/2707517030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7fc15ce30 0x7fc7fc161560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0097c0 tx=0x7fc7ec01c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b3fa9f75463226c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc7fa7fc640 1 -- 172.21.15.32:0/2707517030 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0ab600 con 0x7fc7fc15ce30 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc7fa7fc640 1 -- 172.21.15.32:0/2707517030 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7ec0a7050 con 0x7fc7fc15ce30 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc7fa7fc640 1 -- 172.21.15.32:0/2707517030 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7ec0ab980 con 0x7fc7fc15ce30 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc8012f2640 1 -- 172.21.15.32:0/2707517030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7fc15ce30 msgr2=0x7fc7fc161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc8012f2640 1 --2- 172.21.15.32:0/2707517030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7fc15ce30 0x7fc7fc161560 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0097c0 tx=0x7fc7ec01c3f0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc8012f2640 1 -- 172.21.15.32:0/2707517030 shutdown_connections 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc8012f2640 1 --2- 172.21.15.32:0/2707517030 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7fc15ce30 0x7fc7fc161560 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc8012f2640 1 -- 172.21.15.32:0/2707517030 >> 172.21.15.32:0/2707517030 conn(0x7fc7fc0cba50 msgr2=0x7fc7fc0cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc8012f2640 1 -- 172.21.15.32:0/2707517030 shutdown_connections 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc8012f2640 1 -- 172.21.15.32:0/2707517030 wait complete. 2023-12-20T04:29:12.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc8012f2640 1 Processor -- start 2023-12-20T04:29:12.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.015+0000 7fc8012f2640 1 -- start start 2023-12-20T04:29:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc8012f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7fc06deb0 0x7fc7fc0c1950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc8012f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc070950 0x7fc7fc1559a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc8012f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7fc15ce30 0x7fc7fc155ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc8012f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc1565f0 con 0x7fc7fc070950 2023-12-20T04:29:12.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc8012f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc0c1130 con 0x7fc7fc15ce30 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc8012f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7fc0c1410 con 0x7fc7fc06deb0 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc070950 0x7fc7fc1559a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f27fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc070950 0x7fc7fc1559a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59210/0 (socket says 172.21.15.32:59210) 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f27fc640 1 -- 172.21.15.32:0/2626834069 learned_addr learned my addr 172.21.15.32:0/2626834069 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7faffd640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7fc06deb0 0x7fc7fc0c1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7fb7fe640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7fc15ce30 0x7fc7fc155ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f27fc640 1 -- 172.21.15.32:0/2626834069 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7fc06deb0 msgr2=0x7fc7fc0c1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f27fc640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7fc06deb0 0x7fc7fc0c1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f27fc640 1 -- 172.21.15.32:0/2626834069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7fc15ce30 msgr2=0x7fc7fc155ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f27fc640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7fc15ce30 0x7fc7fc155ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f27fc640 1 -- 172.21.15.32:0/2626834069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7ec009420 con 0x7fc7fc070950 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f27fc640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc070950 0x7fc7fc1559a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400c500 tx=0x7fc7e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f3fff640 1 -- 172.21.15.32:0/2626834069 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4005220 con 0x7fc7fc070950 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f3fff640 1 -- 172.21.15.32:0/2626834069 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7e4005380 con 0x7fc7fc070950 2023-12-20T04:29:12.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc7f3fff640 1 -- 172.21.15.32:0/2626834069 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7e4007270 con 0x7fc7fc070950 2023-12-20T04:29:12.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc8012f2640 1 -- 172.21.15.32:0/2626834069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7fc0c16f0 con 0x7fc7fc070950 2023-12-20T04:29:12.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc8012f2640 1 -- 172.21.15.32:0/2626834069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7fc0c1f60 con 0x7fc7fc070950 2023-12-20T04:29:12.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.019+0000 7fc8012f2640 1 -- 172.21.15.32:0/2626834069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7fc0c2e80 con 0x7fc7fc070950 2023-12-20T04:29:12.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.023+0000 7fc7f3fff640 1 -- 172.21.15.32:0/2626834069 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc7e40073d0 con 0x7fc7fc070950 2023-12-20T04:29:12.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.023+0000 7fc7f3fff640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7dc065a90 0x7fc7dc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.023+0000 7fc7f3fff640 1 -- 172.21.15.32:0/2626834069 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc7e4082900 con 0x7fc7fc070950 2023-12-20T04:29:12.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.023+0000 7fc7faffd640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7dc065a90 0x7fc7dc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.023+0000 7fc7faffd640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7dc065a90 0x7fc7dc067f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0ad000 tx=0x7fc7ec008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:12.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.027+0000 7fc7f3fff640 1 -- 172.21.15.32:0/2626834069 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc7fc0c2e80 con 0x7fc7fc070950 2023-12-20T04:29:12.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 -- 172.21.15.32:0/2626834069 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc7fc0c2e80 con 0x7fc7fc070950 2023-12-20T04:29:12.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc7f3fff640 1 -- 172.21.15.32:0/2626834069 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7fc7fc0c2e80 con 0x7fc7fc070950 2023-12-20T04:29:12.197 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-12-20T04:29:12.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 -- 172.21.15.32:0/2626834069 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7dc065a90 msgr2=0x7fc7dc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7dc065a90 0x7fc7dc067f50 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc7ec0ad000 tx=0x7fc7ec008040 comp rx=0 tx=0).stop 2023-12-20T04:29:12.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 -- 172.21.15.32:0/2626834069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc070950 msgr2=0x7fc7fc1559a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc070950 0x7fc7fc1559a0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fc7e400c500 tx=0x7fc7e400c9c0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 -- 172.21.15.32:0/2626834069 shutdown_connections 2023-12-20T04:29:12.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7dc065a90 0x7fc7dc067f50 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7fc15ce30 0x7fc7fc155ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7fc070950 0x7fc7fc1559a0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 --2- 172.21.15.32:0/2626834069 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7fc06deb0 0x7fc7fc0c1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 -- 172.21.15.32:0/2626834069 >> 172.21.15.32:0/2626834069 conn(0x7fc7fc0cba50 msgr2=0x7fc7fc1579c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:12.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 -- 172.21.15.32:0/2626834069 shutdown_connections 2023-12-20T04:29:12.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.195+0000 7fc8012f2640 1 -- 172.21.15.32:0/2626834069 wait complete. 2023-12-20T04:29:12.218 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.1... 2023-12-20T04:29:12.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:12.219 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.1 -f 2023-12-20T04:29:12.262 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T04:29:12.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:12.262 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.1) 2023-12-20T04:29:12.330 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:29:12.330 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:12.339 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:29:12.339 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:12.395 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:29:12.395 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:12.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.631+0000 7f2f7ec8c640 1 Processor -- start 2023-12-20T04:29:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.631+0000 7f2f7ec8c640 1 -- start start 2023-12-20T04:29:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ec8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f781599e0 0x7f2f780aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ec8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2f780b0d00 0x7f2f780701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ec8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2f780c98c0 con 0x7f2f780c90e0 2023-12-20T04:29:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ec8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f780c9ba0 con 0x7f2f781599e0 2023-12-20T04:29:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ec8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2f780c9e80 con 0x7f2f780b0d00 2023-12-20T04:29:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ca28640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2f781599e0 0x7f2f780aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f77fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2f780c90e0 0x7f2f780b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57448/0 (socket says 172.21.15.32:57448) 2023-12-20T04:29:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f77fff640 1 -- 172.21.15.32:0/3025885671 learned_addr learned my addr 172.21.15.32:0/3025885671 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:12.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ca28640 1 --2- 172.21.15.32:0/3025885671 >> v2:172.21.15.32:3300/0 conn(0x7f2f781599e0 0x7f2f780aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59224/0 (socket says 172.21.15.32:59224) 2023-12-20T04:29:12.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7d229640 1 --2- 172.21.15.32:0/3025885671 >> v2:172.21.15.39:3300/0 conn(0x7f2f780b0d00 0x7f2f780701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ca28640 1 -- 172.21.15.32:0/3025885671 >> v2:172.21.15.39:3300/0 conn(0x7f2f780b0d00 msgr2=0x7f2f780701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:12.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ca28640 1 --2- 172.21.15.32:0/3025885671 >> v2:172.21.15.39:3300/0 conn(0x7f2f780b0d00 0x7f2f780701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ca28640 1 -- 172.21.15.32:0/3025885671 >> v1:172.21.15.32:6789/0 conn(0x7f2f780c90e0 legacy=0x7f2f780b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ca28640 1 -- 172.21.15.32:0/3025885671 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f780ca160 con 0x7f2f781599e0 2023-12-20T04:29:12.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f777fe640 1 -- 172.21.15.32:0/3025885671 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1907453801 0 0) 0x7f2f780c98c0 con 0x7f2f780c90e0 2023-12-20T04:29:12.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.635+0000 7f2f7ca28640 1 --2- 172.21.15.32:0/3025885671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f781599e0 0x7f2f780aece0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2f780b0060 tx=0x7f2f6c01c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab67fb15418d1014 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:12.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f777fe640 1 -- 172.21.15.32:0/3025885671 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0ab600 con 0x7f2f781599e0 2023-12-20T04:29:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f777fe640 1 -- 172.21.15.32:0/3025885671 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f6c0a7050 con 0x7f2f781599e0 2023-12-20T04:29:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f777fe640 1 -- 172.21.15.32:0/3025885671 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f6c0ab980 con 0x7f2f781599e0 2023-12-20T04:29:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/3025885671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f781599e0 msgr2=0x7f2f780aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 --2- 172.21.15.32:0/3025885671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f781599e0 0x7f2f780aece0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f2f780b0060 tx=0x7f2f6c01c410 comp rx=0 tx=0).stop 2023-12-20T04:29:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/3025885671 shutdown_connections 2023-12-20T04:29:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 --2- 172.21.15.32:0/3025885671 >> v2:172.21.15.39:3300/0 conn(0x7f2f780b0d00 0x7f2f780701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 --2- 172.21.15.32:0/3025885671 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f781599e0 0x7f2f780aece0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/3025885671 >> 172.21.15.32:0/3025885671 conn(0x7f2f7806a320 msgr2=0x7f2f780ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/3025885671 shutdown_connections 2023-12-20T04:29:12.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/3025885671 wait complete. 2023-12-20T04:29:12.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 Processor -- start 2023-12-20T04:29:12.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 -- start start 2023-12-20T04:29:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f780b0d00 0x7f2f7814ba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f780c90e0 0x7f2f7814c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f781599e0 0x7f2f780b33b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f78155d20 con 0x7f2f781599e0 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2f78154f10 con 0x7f2f780b0d00 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7ec8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2f781551f0 con 0x7f2f780c90e0 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.639+0000 7f2f7d229640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f781599e0 0x7f2f780b33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7d229640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f781599e0 0x7f2f780b33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59226/0 (socket says 172.21.15.32:59226) 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7d229640 1 -- 172.21.15.32:0/440226942 learned_addr learned my addr 172.21.15.32:0/440226942 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7ca28640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f780b0d00 0x7f2f7814ba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f77fff640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f780c90e0 0x7f2f7814c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7d229640 1 -- 172.21.15.32:0/440226942 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f780c90e0 msgr2=0x7f2f7814c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7d229640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f780c90e0 0x7f2f7814c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7d229640 1 -- 172.21.15.32:0/440226942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f780b0d00 msgr2=0x7f2f7814ba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7d229640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f780b0d00 0x7f2f7814ba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7d229640 1 -- 172.21.15.32:0/440226942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2f6c0093b0 con 0x7f2f781599e0 2023-12-20T04:29:12.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7ca28640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f780b0d00 0x7f2f7814ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:12.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7d229640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f781599e0 0x7f2f780b33b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f70007e60 tx=0x7f2f7000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:12.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f757fa640 1 -- 172.21.15.32:0/440226942 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70014420 con 0x7f2f781599e0 2023-12-20T04:29:12.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f757fa640 1 -- 172.21.15.32:0/440226942 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2f700145a0 con 0x7f2f781599e0 2023-12-20T04:29:12.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f757fa640 1 -- 172.21.15.32:0/440226942 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2f70014950 con 0x7f2f781599e0 2023-12-20T04:29:12.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/440226942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2f78155f70 con 0x7f2f781599e0 2023-12-20T04:29:12.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/440226942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2f7814bfb0 con 0x7f2f781599e0 2023-12-20T04:29:12.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/440226942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2f78155410 con 0x7f2f781599e0 2023-12-20T04:29:12.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f757fa640 1 -- 172.21.15.32:0/440226942 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2f70014ab0 con 0x7f2f781599e0 2023-12-20T04:29:12.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.643+0000 7f2f757fa640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f5c065a60 0x7f2f5c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.647+0000 7f2f757fa640 1 -- 172.21.15.32:0/440226942 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2f70083b60 con 0x7f2f781599e0 2023-12-20T04:29:12.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.647+0000 7f2f7ca28640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f5c065a60 0x7f2f5c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.647+0000 7f2f7ca28640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f5c065a60 0x7f2f5c067f20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0ad000 tx=0x7f2f6c0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:12.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.651+0000 7f2f757fa640 1 -- 172.21.15.32:0/440226942 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2f70051480 con 0x7f2f781599e0 2023-12-20T04:29:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/440226942 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2f7804e000 con 0x7f2f781599e0 2023-12-20T04:29:12.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f757fa640 1 -- 172.21.15.32:0/440226942 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f2f70054aa0 con 0x7f2f781599e0 2023-12-20T04:29:12.813 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:12.813 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/440226942 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f5c065a60 msgr2=0x7f2f5c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f5c065a60 0x7f2f5c067f20 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f2f6c0ad000 tx=0x7f2f6c0a6000 comp rx=0 tx=0).stop 2023-12-20T04:29:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/440226942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f781599e0 msgr2=0x7f2f780b33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f781599e0 0x7f2f780b33b0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f2f70007e60 tx=0x7f2f7000e4b0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/440226942 shutdown_connections 2023-12-20T04:29:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2f5c065a60 0x7f2f5c067f20 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2f781599e0 0x7f2f780b33b0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2f780c90e0 0x7f2f7814c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 --2- 172.21.15.32:0/440226942 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2f780b0d00 0x7f2f7814ba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/440226942 >> 172.21.15.32:0/440226942 conn(0x7f2f7806a320 msgr2=0x7f2f7814fcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/440226942 shutdown_connections 2023-12-20T04:29:12.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.811+0000 7f2f7ec8c640 1 -- 172.21.15.32:0/440226942 wait complete. 2023-12-20T04:29:12.835 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:28:36.040195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:28:37.364671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:29:12.835 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:29:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b29b09640 1 Processor -- start 2023-12-20T04:29:12.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b29b09640 1 -- start start 2023-12-20T04:29:12.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b29b09640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3b240b0760 0x7f3b240b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b29b09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b240cc4b0 0x7f3b240cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b29b09640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3b240b1100 con 0x7f3b24157480 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b29b09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b240c9f50 con 0x7f3b240cc4b0 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b29b09640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b240cde20 con 0x7f3b240b0760 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b23fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b240cc4b0 0x7f3b240cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b23fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b240cc4b0 0x7f3b240cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59232/0 (socket says 172.21.15.32:59232) 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b23fff640 1 -- 172.21.15.32:0/2029322702 learned_addr learned my addr 172.21.15.32:0/2029322702 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b22ffd640 1 --2- 172.21.15.32:0/2029322702 >> v2:172.21.15.39:3300/0 conn(0x7f3b240b0760 0x7f3b240b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b23fff640 1 -- 172.21.15.32:0/2029322702 >> v2:172.21.15.39:3300/0 conn(0x7f3b240b0760 msgr2=0x7f3b240b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b23fff640 1 --2- 172.21.15.32:0/2029322702 >> v2:172.21.15.39:3300/0 conn(0x7f3b240b0760 0x7f3b240b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b23fff640 1 -- 172.21.15.32:0/2029322702 >> v1:172.21.15.39:6789/0 conn(0x7f3b24157480 legacy=0x7f3b240c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.983+0000 7f3b23fff640 1 -- 172.21.15.32:0/2029322702 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b240c8e30 con 0x7f3b240cc4b0 2023-12-20T04:29:12.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b23fff640 1 --2- 172.21.15.32:0/2029322702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b240cc4b0 0x7f3b240cc8d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3b2406e0d0 tx=0x7f3b1c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6395560534311ad6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:12.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b227fc640 1 -- 172.21.15.32:0/2029322702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0a57d0 con 0x7f3b240cc4b0 2023-12-20T04:29:12.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b227fc640 1 -- 172.21.15.32:0/2029322702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b1c0a5930 con 0x7f3b240cc4b0 2023-12-20T04:29:12.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b227fc640 1 -- 172.21.15.32:0/2029322702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b1c0a5cc0 con 0x7f3b240cc4b0 2023-12-20T04:29:12.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b29b09640 1 -- 172.21.15.32:0/2029322702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b240cc4b0 msgr2=0x7f3b240cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b29b09640 1 --2- 172.21.15.32:0/2029322702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b240cc4b0 0x7f3b240cc8d0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f3b2406e0d0 tx=0x7f3b1c01c910 comp rx=0 tx=0).stop 2023-12-20T04:29:12.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b29b09640 1 -- 172.21.15.32:0/2029322702 shutdown_connections 2023-12-20T04:29:12.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b29b09640 1 --2- 172.21.15.32:0/2029322702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b240cc4b0 0x7f3b240cc8d0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b29b09640 1 --2- 172.21.15.32:0/2029322702 >> v2:172.21.15.39:3300/0 conn(0x7f3b240b0760 0x7f3b240b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b29b09640 1 -- 172.21.15.32:0/2029322702 >> 172.21.15.32:0/2029322702 conn(0x7f3b2406a510 msgr2=0x7f3b240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:12.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b29b09640 1 -- 172.21.15.32:0/2029322702 shutdown_connections 2023-12-20T04:29:12.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b29b09640 1 -- 172.21.15.32:0/2029322702 wait complete. 2023-12-20T04:29:12.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.987+0000 7f3b29b09640 1 Processor -- start 2023-12-20T04:29:12.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b29b09640 1 -- start start 2023-12-20T04:29:12.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b29b09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b240b0760 0x7f3b24141d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b29b09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b240cc4b0 0x7f3b2415b020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b29b09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24157480 0x7f3b24070390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b29b09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2415b560 con 0x7f3b240b0760 2023-12-20T04:29:12.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b29b09640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2415b800 con 0x7f3b24157480 2023-12-20T04:29:12.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b29b09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b2415bae0 con 0x7f3b240cc4b0 2023-12-20T04:29:12.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b240b0760 0x7f3b24141d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b240b0760 0x7f3b24141d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59242/0 (socket says 172.21.15.32:59242) 2023-12-20T04:29:12.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b237fe640 1 -- 172.21.15.32:0/3080922550 learned_addr learned my addr 172.21.15.32:0/3080922550 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:12.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b23fff640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24157480 0x7f3b24070390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b22ffd640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b240cc4b0 0x7f3b2415b020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b237fe640 1 -- 172.21.15.32:0/3080922550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b240cc4b0 msgr2=0x7f3b2415b020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b237fe640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b240cc4b0 0x7f3b2415b020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b237fe640 1 -- 172.21.15.32:0/3080922550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24157480 msgr2=0x7f3b24070390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b237fe640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24157480 0x7f3b24070390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b237fe640 1 -- 172.21.15.32:0/3080922550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b1c009b30 con 0x7f3b240b0760 2023-12-20T04:29:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b237fe640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b240b0760 0x7f3b24141d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400c500 tx=0x7f3b1400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b0bfff640 1 -- 172.21.15.32:0/3080922550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14005220 con 0x7f3b240b0760 2023-12-20T04:29:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b0bfff640 1 -- 172.21.15.32:0/3080922550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b14005380 con 0x7f3b240b0760 2023-12-20T04:29:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b29b09640 1 -- 172.21.15.32:0/3080922550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b2415bdc0 con 0x7f3b240b0760 2023-12-20T04:29:12.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b0bfff640 1 -- 172.21.15.32:0/3080922550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b14007270 con 0x7f3b240b0760 2023-12-20T04:29:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b29b09640 1 -- 172.21.15.32:0/3080922550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b2415c380 con 0x7f3b240b0760 2023-12-20T04:29:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.991+0000 7f3b29b09640 1 -- 172.21.15.32:0/3080922550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b2404e000 con 0x7f3b240b0760 2023-12-20T04:29:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.995+0000 7f3b0bfff640 1 -- 172.21.15.32:0/3080922550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f3b140073d0 con 0x7f3b240b0760 2023-12-20T04:29:12.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.995+0000 7f3b0bfff640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b10065a60 0x7f3b10067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.995+0000 7f3b0bfff640 1 -- 172.21.15.32:0/3080922550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b140829c0 con 0x7f3b240b0760 2023-12-20T04:29:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.995+0000 7f3b22ffd640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b10065a60 0x7f3b10067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.995+0000 7f3b22ffd640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b10065a60 0x7f3b10067f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3b240b39c0 tx=0x7f3b18009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:13.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:12.999+0000 7f3b0bfff640 1 -- 172.21.15.32:0/3080922550 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3b140503a0 con 0x7f3b240b0760 2023-12-20T04:29:13.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.155+0000 7f3b29b09640 1 -- 172.21.15.32:0/3080922550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3b2415c570 con 0x7f3b240b0760 2023-12-20T04:29:13.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.155+0000 7f3b0bfff640 1 -- 172.21.15.32:0/3080922550 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3b140539c0 con 0x7f3b240b0760 2023-12-20T04:29:13.159 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 -- 172.21.15.32:0/3080922550 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b10065a60 msgr2=0x7f3b10067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b10065a60 0x7f3b10067f20 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f3b240b39c0 tx=0x7f3b18009190 comp rx=0 tx=0).stop 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 -- 172.21.15.32:0/3080922550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b240b0760 msgr2=0x7f3b24141d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b240b0760 0x7f3b24141d10 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f3b1400c500 tx=0x7f3b1400c9c0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 -- 172.21.15.32:0/3080922550 shutdown_connections 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b10065a60 0x7f3b10067f20 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b24157480 0x7f3b24070390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b240cc4b0 0x7f3b2415b020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 --2- 172.21.15.32:0/3080922550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b240b0760 0x7f3b24141d10 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 -- 172.21.15.32:0/3080922550 >> 172.21.15.32:0/3080922550 conn(0x7f3b2406a510 msgr2=0x7f3b240c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 -- 172.21.15.32:0/3080922550 shutdown_connections 2023-12-20T04:29:13.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.159+0000 7f3b29b09640 1 -- 172.21.15.32:0/3080922550 wait complete. 2023-12-20T04:29:13.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:29:13.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.323+0000 7f93bce20640 1 Processor -- start 2023-12-20T04:29:13.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.323+0000 7f93bce20640 1 -- start start 2023-12-20T04:29:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.323+0000 7f93bce20640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f93b815abb0 0x7f93b80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.323+0000 7f93bce20640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93b80b0cb0 0x7f93b815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.323+0000 7f93bce20640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93b815b6a0 con 0x7f93b8070230 2023-12-20T04:29:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.323+0000 7f93bce20640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b806e160 con 0x7f93b80b0cb0 2023-12-20T04:29:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.323+0000 7f93bce20640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93b815c070 con 0x7f93b815abb0 2023-12-20T04:29:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.323+0000 7f93affff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93b80b0cb0 0x7f93b815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93affff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93b80b0cb0 0x7f93b815bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59246/0 (socket says 172.21.15.32:59246) 2023-12-20T04:29:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93affff640 1 -- 172.21.15.32:0/2752974074 learned_addr learned my addr 172.21.15.32:0/2752974074 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93affff640 1 -- 172.21.15.32:0/2752974074 >> v2:172.21.15.39:3300/0 conn(0x7f93b815abb0 msgr2=0x7f93b80caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93b659c640 1 --2- 172.21.15.32:0/2752974074 >> v2:172.21.15.39:3300/0 conn(0x7f93b815abb0 0x7f93b80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93affff640 1 --2- 172.21.15.32:0/2752974074 >> v2:172.21.15.39:3300/0 conn(0x7f93b815abb0 0x7f93b80caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93affff640 1 -- 172.21.15.32:0/2752974074 >> v1:172.21.15.39:6789/0 conn(0x7f93b8070230 legacy=0x7f93b806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93affff640 1 -- 172.21.15.32:0/2752974074 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93b806d8c0 con 0x7f93b80b0cb0 2023-12-20T04:29:13.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93b659c640 1 --2- 172.21.15.32:0/2752974074 >> v2:172.21.15.39:3300/0 conn(0x7f93b815abb0 0x7f93b80caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:13.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93affff640 1 --2- 172.21.15.32:0/2752974074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b80b0cb0 0x7f93b815bb30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f93b80713f0 tx=0x7f93a001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc131968638943f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93b5d9b640 1 -- 172.21.15.32:0/2752974074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a00aa620 con 0x7f93b80b0cb0 2023-12-20T04:29:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93b5d9b640 1 -- 172.21.15.32:0/2752974074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93a00aa780 con 0x7f93b80b0cb0 2023-12-20T04:29:13.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.327+0000 7f93b5d9b640 1 -- 172.21.15.32:0/2752974074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93a00aab10 con 0x7f93b80b0cb0 2023-12-20T04:29:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 -- 172.21.15.32:0/2752974074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b80b0cb0 msgr2=0x7f93b815bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 --2- 172.21.15.32:0/2752974074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b80b0cb0 0x7f93b815bb30 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f93b80713f0 tx=0x7f93a001c4c0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 -- 172.21.15.32:0/2752974074 shutdown_connections 2023-12-20T04:29:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 --2- 172.21.15.32:0/2752974074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b80b0cb0 0x7f93b815bb30 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 --2- 172.21.15.32:0/2752974074 >> v2:172.21.15.39:3300/0 conn(0x7f93b815abb0 0x7f93b80caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 -- 172.21.15.32:0/2752974074 >> 172.21.15.32:0/2752974074 conn(0x7f93b806a510 msgr2=0x7f93b80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 -- 172.21.15.32:0/2752974074 shutdown_connections 2023-12-20T04:29:13.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 -- 172.21.15.32:0/2752974074 wait complete. 2023-12-20T04:29:13.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 Processor -- start 2023-12-20T04:29:13.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 -- start start 2023-12-20T04:29:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93b8070230 0x7f93b814b630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b80b0cb0 0x7f93b81599b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b815abb0 0x7f93b80cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b814cc30 con 0x7f93b815abb0 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93b814bfb0 con 0x7f93b80b0cb0 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93bce20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93b814c290 con 0x7f93b8070230 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93b6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b815abb0 0x7f93b80cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93b659c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93b8070230 0x7f93b814b630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93b6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b815abb0 0x7f93b80cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59250/0 (socket says 172.21.15.32:59250) 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93b659c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93b8070230 0x7f93b814b630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45428/0 (socket says 172.21.15.32:45428) 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93b6d9d640 1 -- 172.21.15.32:0/3582287494 learned_addr learned my addr 172.21.15.32:0/3582287494 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93affff640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b80b0cb0 0x7f93b81599b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93b6d9d640 1 -- 172.21.15.32:0/3582287494 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93b8070230 msgr2=0x7f93b814b630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93b6d9d640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93b8070230 0x7f93b814b630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93b6d9d640 1 -- 172.21.15.32:0/3582287494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b80b0cb0 msgr2=0x7f93b81599b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93b6d9d640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b80b0cb0 0x7f93b81599b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.331+0000 7f93b6d9d640 1 -- 172.21.15.32:0/3582287494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93a00094d0 con 0x7f93b815abb0 2023-12-20T04:29:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.335+0000 7f93b6d9d640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b815abb0 0x7f93b80cb150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c780 tx=0x7f93b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.335+0000 7f93aeffd640 1 -- 172.21.15.32:0/3582287494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b0004b70 con 0x7f93b815abb0 2023-12-20T04:29:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.335+0000 7f93aeffd640 1 -- 172.21.15.32:0/3582287494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93b000ce90 con 0x7f93b815abb0 2023-12-20T04:29:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.335+0000 7f93aeffd640 1 -- 172.21.15.32:0/3582287494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93b000a850 con 0x7f93b815abb0 2023-12-20T04:29:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.335+0000 7f93bce20640 1 -- 172.21.15.32:0/3582287494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93b814c570 con 0x7f93b815abb0 2023-12-20T04:29:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.335+0000 7f93bce20640 1 -- 172.21.15.32:0/3582287494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93b814bb70 con 0x7f93b815abb0 2023-12-20T04:29:13.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.335+0000 7f93bce20640 1 -- 172.21.15.32:0/3582287494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93b814cd70 con 0x7f93b815abb0 2023-12-20T04:29:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.335+0000 7f93aeffd640 1 -- 172.21.15.32:0/3582287494 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f93b000a9b0 con 0x7f93b815abb0 2023-12-20T04:29:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.335+0000 7f93aeffd640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9398065a60 0x7f9398067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.335+0000 7f93aeffd640 1 -- 172.21.15.32:0/3582287494 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f93b00831f0 con 0x7f93b815abb0 2023-12-20T04:29:13.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.335+0000 7f93b659c640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9398065a60 0x7f9398067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.339+0000 7f93b659c640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9398065a60 0x7f9398067f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f93a8006fd0 tx=0x7f93a8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:13.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.339+0000 7f93aeffd640 1 -- 172.21.15.32:0/3582287494 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f93b814cd70 con 0x7f93b815abb0 2023-12-20T04:29:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 -- 172.21.15.32:0/3582287494 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f93b814cd70 con 0x7f93b815abb0 2023-12-20T04:29:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93aeffd640 1 -- 172.21.15.32:0/3582287494 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f93b814cd70 con 0x7f93b815abb0 2023-12-20T04:29:13.501 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:29:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 -- 172.21.15.32:0/3582287494 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9398065a60 msgr2=0x7f9398067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9398065a60 0x7f9398067f20 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f93a8006fd0 tx=0x7f93a8008040 comp rx=0 tx=0).stop 2023-12-20T04:29:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 -- 172.21.15.32:0/3582287494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b815abb0 msgr2=0x7f93b80cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b815abb0 0x7f93b80cb150 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7f93b000c780 tx=0x7f93b000cc40 comp rx=0 tx=0).stop 2023-12-20T04:29:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 -- 172.21.15.32:0/3582287494 shutdown_connections 2023-12-20T04:29:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9398065a60 0x7f9398067f20 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93b815abb0 0x7f93b80cb150 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93b80b0cb0 0x7f93b81599b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 --2- 172.21.15.32:0/3582287494 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93b8070230 0x7f93b814b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 -- 172.21.15.32:0/3582287494 >> 172.21.15.32:0/3582287494 conn(0x7f93b806a510 msgr2=0x7f93b80c4a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 -- 172.21.15.32:0/3582287494 shutdown_connections 2023-12-20T04:29:13.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.499+0000 7f93bce20640 1 -- 172.21.15.32:0/3582287494 wait complete. 2023-12-20T04:29:13.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:29:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.663+0000 7fc24f96d640 1 Processor -- start 2023-12-20T04:29:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.663+0000 7fc24f96d640 1 -- start start 2023-12-20T04:29:13.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24f96d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc248157480 0x7fc2480c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24f96d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2480cd4c0 0x7fc2480b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24f96d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2480b1100 con 0x7fc2480b0760 2023-12-20T04:29:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24f96d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2480cde20 con 0x7fc2480cd4c0 2023-12-20T04:29:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24f96d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480c8e30 con 0x7fc248157480 2023-12-20T04:29:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24df0a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2480cd4c0 0x7fc2480b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24cf08640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2480b0760 0x7fc24815d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57458/0 (socket says 172.21.15.32:57458) 2023-12-20T04:29:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24cf08640 1 -- 172.21.15.32:0/994030406 learned_addr learned my addr 172.21.15.32:0/994030406 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:13.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24d709640 1 --2- 172.21.15.32:0/994030406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc248157480 0x7fc2480c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24df0a640 1 -- 172.21.15.32:0/994030406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc248157480 msgr2=0x7fc2480c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24df0a640 1 --2- 172.21.15.32:0/994030406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc248157480 0x7fc2480c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24df0a640 1 -- 172.21.15.32:0/994030406 >> v1:172.21.15.32:6789/0 conn(0x7fc2480b0760 legacy=0x7fc24815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc23ffff640 1 -- 172.21.15.32:0/994030406 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033754629 0 0) 0x7fc2480b1100 con 0x7fc2480b0760 2023-12-20T04:29:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24df0a640 1 -- 172.21.15.32:0/994030406 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2480ce070 con 0x7fc2480cd4c0 2023-12-20T04:29:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.667+0000 7fc24d709640 1 --2- 172.21.15.32:0/994030406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc248157480 0x7fc2480c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc24df0a640 1 --2- 172.21.15.32:0/994030406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2480cd4c0 0x7fc2480b0bc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc240009d90 tx=0x7fc24001c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58ad224cba9ab2d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc23ffff640 1 -- 172.21.15.32:0/994030406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2400a5870 con 0x7fc2480cd4c0 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc23ffff640 1 -- 172.21.15.32:0/994030406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2400a59d0 con 0x7fc2480cd4c0 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc23ffff640 1 -- 172.21.15.32:0/994030406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2400a5d60 con 0x7fc2480cd4c0 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc24f96d640 1 -- 172.21.15.32:0/994030406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2480cd4c0 msgr2=0x7fc2480b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc24f96d640 1 --2- 172.21.15.32:0/994030406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2480cd4c0 0x7fc2480b0bc0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7fc240009d90 tx=0x7fc24001c890 comp rx=0 tx=0).stop 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc24f96d640 1 -- 172.21.15.32:0/994030406 shutdown_connections 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc24f96d640 1 --2- 172.21.15.32:0/994030406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2480cd4c0 0x7fc2480b0bc0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc24f96d640 1 --2- 172.21.15.32:0/994030406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc248157480 0x7fc2480c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc24f96d640 1 -- 172.21.15.32:0/994030406 >> 172.21.15.32:0/994030406 conn(0x7fc24806a510 msgr2=0x7fc2480ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc24f96d640 1 -- 172.21.15.32:0/994030406 shutdown_connections 2023-12-20T04:29:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc24f96d640 1 -- 172.21.15.32:0/994030406 wait complete. 2023-12-20T04:29:13.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc24f96d640 1 Processor -- start 2023-12-20T04:29:13.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.671+0000 7fc24f96d640 1 -- start start 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24f96d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2480b0760 0x7fc24815d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24f96d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2480cd4c0 0x7fc2480b2f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24f96d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc248157480 0x7fc24815add0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24f96d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc248136560 con 0x7fc2480cd4c0 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24f96d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480b2ab0 con 0x7fc2480b0760 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24f96d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2480b3480 con 0x7fc248157480 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24cf08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2480cd4c0 0x7fc2480b2f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24cf08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2480cd4c0 0x7fc2480b2f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59266/0 (socket says 172.21.15.32:59266) 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24cf08640 1 -- 172.21.15.32:0/3620827182 learned_addr learned my addr 172.21.15.32:0/3620827182 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24df0a640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc248157480 0x7fc24815add0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24d709640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2480b0760 0x7fc24815d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24cf08640 1 -- 172.21.15.32:0/3620827182 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc248157480 msgr2=0x7fc24815add0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24cf08640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc248157480 0x7fc24815add0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24cf08640 1 -- 172.21.15.32:0/3620827182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2480b0760 msgr2=0x7fc24815d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24cf08640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2480b0760 0x7fc24815d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24cf08640 1 -- 172.21.15.32:0/3620827182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc240009a70 con 0x7fc2480cd4c0 2023-12-20T04:29:13.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24cf08640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2480cd4c0 0x7fc2480b2f40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c4b0 tx=0x7fc23800c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:13.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc23dffb640 1 -- 172.21.15.32:0/3620827182 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2380074e0 con 0x7fc2480cd4c0 2023-12-20T04:29:13.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc23dffb640 1 -- 172.21.15.32:0/3620827182 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc238007660 con 0x7fc2480cd4c0 2023-12-20T04:29:13.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc23dffb640 1 -- 172.21.15.32:0/3620827182 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc238004a30 con 0x7fc2480cd4c0 2023-12-20T04:29:13.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24f96d640 1 -- 172.21.15.32:0/3620827182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2480b1ea0 con 0x7fc2480cd4c0 2023-12-20T04:29:13.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24f96d640 1 -- 172.21.15.32:0/3620827182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2480b2460 con 0x7fc2480cd4c0 2023-12-20T04:29:13.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.675+0000 7fc24f96d640 1 -- 172.21.15.32:0/3620827182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24804e000 con 0x7fc2480cd4c0 2023-12-20T04:29:13.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.679+0000 7fc23dffb640 1 -- 172.21.15.32:0/3620827182 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc238004b90 con 0x7fc2480cd4c0 2023-12-20T04:29:13.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.679+0000 7fc23dffb640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc22c065a60 0x7fc22c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:13.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.679+0000 7fc23dffb640 1 -- 172.21.15.32:0/3620827182 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2380822c0 con 0x7fc2480cd4c0 2023-12-20T04:29:13.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.679+0000 7fc24d709640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc22c065a60 0x7fc22c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:13.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.679+0000 7fc24d709640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc22c065a60 0x7fc22c067f20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc2480c3c20 tx=0x7fc244008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:13.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.683+0000 7fc23dffb640 1 -- 172.21.15.32:0/3620827182 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc23804fc90 con 0x7fc2480cd4c0 2023-12-20T04:29:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 -- 172.21.15.32:0/3620827182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fc2480b2bf0 con 0x7fc2480cd4c0 2023-12-20T04:29:13.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc23dffb640 1 -- 172.21.15.32:0/3620827182 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fc2380532b0 con 0x7fc2480cd4c0 2023-12-20T04:29:13.841 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 -- 172.21.15.32:0/3620827182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc22c065a60 msgr2=0x7fc22c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc22c065a60 0x7fc22c067f20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc2480c3c20 tx=0x7fc244008040 comp rx=0 tx=0).stop 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 -- 172.21.15.32:0/3620827182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2480cd4c0 msgr2=0x7fc2480b2f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2480cd4c0 0x7fc2480b2f40 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7fc23800c4b0 tx=0x7fc23800c970 comp rx=0 tx=0).stop 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 -- 172.21.15.32:0/3620827182 shutdown_connections 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc22c065a60 0x7fc22c067f20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc248157480 0x7fc24815add0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2480cd4c0 0x7fc2480b2f40 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 --2- 172.21.15.32:0/3620827182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2480b0760 0x7fc24815d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 -- 172.21.15.32:0/3620827182 >> 172.21.15.32:0/3620827182 conn(0x7fc24806a510 msgr2=0x7fc2481524f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 -- 172.21.15.32:0/3620827182 shutdown_connections 2023-12-20T04:29:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:13.839+0000 7fc24f96d640 1 -- 172.21.15.32:0/3620827182 wait complete. 2023-12-20T04:29:13.862 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T04:29:14.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.003+0000 7f5cc1241640 1 Processor -- start 2023-12-20T04:29:14.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.003+0000 7f5cc1241640 1 -- start start 2023-12-20T04:29:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.007+0000 7f5cc1241640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cbc1599e0 0x7f5cbc0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.007+0000 7f5cc1241640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cbc0c98c0 con 0x7f5cbc0b0d00 2023-12-20T04:29:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.007+0000 7f5cc1241640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5cbc0c9ba0 con 0x7f5cbc0c90e0 2023-12-20T04:29:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.007+0000 7f5cc1241640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5cbc0c9e80 con 0x7f5cbc1599e0 2023-12-20T04:29:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.007+0000 7f5cbad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cbc1599e0 0x7f5cbc0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.007+0000 7f5cbad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5cbc1599e0 0x7f5cbc0aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59272/0 (socket says 172.21.15.32:59272) 2023-12-20T04:29:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.007+0000 7f5cbad9d640 1 -- 172.21.15.32:0/895263435 learned_addr learned my addr 172.21.15.32:0/895263435 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.007+0000 7f5cbb59e640 1 --1- 172.21.15.32:0/895263435 >> v1:172.21.15.32:6789/0 conn(0x7f5cbc0b0d00 0x7f5cbc0701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57468/0 (socket says 172.21.15.32:57468) 2023-12-20T04:29:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.007+0000 7f5cbad9d640 1 -- 172.21.15.32:0/895263435 >> v1:172.21.15.39:6789/0 conn(0x7f5cbc0c90e0 legacy=0x7f5cbc0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.007+0000 7f5cbad9d640 1 -- 172.21.15.32:0/895263435 >> v1:172.21.15.32:6789/0 conn(0x7f5cbc0b0d00 legacy=0x7f5cbc0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.007+0000 7f5cbad9d640 1 -- 172.21.15.32:0/895263435 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cbc0ca160 con 0x7f5cbc1599e0 2023-12-20T04:29:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cbad9d640 1 --2- 172.21.15.32:0/895263435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cbc1599e0 0x7f5cbc0aece0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5cac006fd0 tx=0x7f5cac01c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e14bce82eabb058 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cba59c640 1 -- 172.21.15.32:0/895263435 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0ab600 con 0x7f5cbc1599e0 2023-12-20T04:29:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cba59c640 1 -- 172.21.15.32:0/895263435 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cac0ab760 con 0x7f5cbc1599e0 2023-12-20T04:29:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cba59c640 1 -- 172.21.15.32:0/895263435 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cac0abaf0 con 0x7f5cbc1599e0 2023-12-20T04:29:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cc1241640 1 -- 172.21.15.32:0/895263435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cbc1599e0 msgr2=0x7f5cbc0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cc1241640 1 --2- 172.21.15.32:0/895263435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cbc1599e0 0x7f5cbc0aece0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5cac006fd0 tx=0x7f5cac01c4f0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cc1241640 1 -- 172.21.15.32:0/895263435 shutdown_connections 2023-12-20T04:29:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cc1241640 1 --2- 172.21.15.32:0/895263435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cbc1599e0 0x7f5cbc0aece0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cc1241640 1 -- 172.21.15.32:0/895263435 >> 172.21.15.32:0/895263435 conn(0x7f5cbc06a320 msgr2=0x7f5cbc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cc1241640 1 -- 172.21.15.32:0/895263435 shutdown_connections 2023-12-20T04:29:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cc1241640 1 -- 172.21.15.32:0/895263435 wait complete. 2023-12-20T04:29:14.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.011+0000 7f5cc1241640 1 Processor -- start 2023-12-20T04:29:14.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cc1241640 1 -- start start 2023-12-20T04:29:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cc1241640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cbc0b0d00 0x7f5cbc14b680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cc1241640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5cbc0c90e0 0x7f5cbc0c4b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cc1241640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cbc1599e0 0x7f5cbc146be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cc1241640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc14c870 con 0x7f5cbc1599e0 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cc1241640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc148690 con 0x7f5cbc0b0d00 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cc1241640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5cbc0c5050 con 0x7f5cbc0c90e0 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cbb59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cbc1599e0 0x7f5cbc146be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cbb59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cbc1599e0 0x7f5cbc146be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59280/0 (socket says 172.21.15.32:59280) 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cbb59e640 1 -- 172.21.15.32:0/3070338979 learned_addr learned my addr 172.21.15.32:0/3070338979 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cbad9d640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cbc0b0d00 0x7f5cbc14b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cb3fff640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5cbc0c90e0 0x7f5cbc0c4b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cbb59e640 1 -- 172.21.15.32:0/3070338979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5cbc0c90e0 msgr2=0x7f5cbc0c4b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cbb59e640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5cbc0c90e0 0x7f5cbc0c4b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cbb59e640 1 -- 172.21.15.32:0/3070338979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cbc0b0d00 msgr2=0x7f5cbc14b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cbb59e640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cbc0b0d00 0x7f5cbc14b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cbb59e640 1 -- 172.21.15.32:0/3070338979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5cac009480 con 0x7f5cbc1599e0 2023-12-20T04:29:14.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cbb59e640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cbc1599e0 0x7f5cbc146be0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400caa0 tx=0x7f5cb400cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cb37fe640 1 -- 172.21.15.32:0/3070338979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb400a730 con 0x7f5cbc1599e0 2023-12-20T04:29:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cb37fe640 1 -- 172.21.15.32:0/3070338979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5cb400a890 con 0x7f5cbc1599e0 2023-12-20T04:29:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cb37fe640 1 -- 172.21.15.32:0/3070338979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5cb400ac40 con 0x7f5cbc1599e0 2023-12-20T04:29:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cc1241640 1 -- 172.21.15.32:0/3070338979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5cbc14bc60 con 0x7f5cbc1599e0 2023-12-20T04:29:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cc1241640 1 -- 172.21.15.32:0/3070338979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5cbc14c220 con 0x7f5cbc1599e0 2023-12-20T04:29:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.015+0000 7f5cc1241640 1 -- 172.21.15.32:0/3070338979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5cbc14c9e0 con 0x7f5cbc1599e0 2023-12-20T04:29:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.019+0000 7f5cb37fe640 1 -- 172.21.15.32:0/3070338979 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5cb4006070 con 0x7f5cbc1599e0 2023-12-20T04:29:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.019+0000 7f5cb37fe640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c9c065a90 0x7f5c9c067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.019+0000 7f5cb37fe640 1 -- 172.21.15.32:0/3070338979 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5cb4082bd0 con 0x7f5cbc1599e0 2023-12-20T04:29:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.019+0000 7f5cbad9d640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c9c065a90 0x7f5c9c067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.019+0000 7f5cbad9d640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c9c065a90 0x7f5c9c067f50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0af000 tx=0x7f5cac0a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:14.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.023+0000 7f5cb37fe640 1 -- 172.21.15.32:0/3070338979 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5cb40505a0 con 0x7f5cbc1599e0 2023-12-20T04:29:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.191+0000 7f5cc1241640 1 -- 172.21.15.32:0/3070338979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f5cbc04e000 con 0x7f5cbc1599e0 2023-12-20T04:29:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.191+0000 7f5cb37fe640 1 -- 172.21.15.32:0/3070338979 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7f5cb4017020 con 0x7f5cbc1599e0 2023-12-20T04:29:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-12-20T04:29:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 -- 172.21.15.32:0/3070338979 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c9c065a90 msgr2=0x7f5c9c067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c9c065a90 0x7f5c9c067f50 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7f5cac0af000 tx=0x7f5cac0a7000 comp rx=0 tx=0).stop 2023-12-20T04:29:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 -- 172.21.15.32:0/3070338979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cbc1599e0 msgr2=0x7f5cbc146be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cbc1599e0 0x7f5cbc146be0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f5cb400caa0 tx=0x7f5cb400cea0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 -- 172.21.15.32:0/3070338979 shutdown_connections 2023-12-20T04:29:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c9c065a90 0x7f5c9c067f50 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5cbc1599e0 0x7f5cbc146be0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5cbc0c90e0 0x7f5cbc0c4b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 --2- 172.21.15.32:0/3070338979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5cbc0b0d00 0x7f5cbc14b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 -- 172.21.15.32:0/3070338979 >> 172.21.15.32:0/3070338979 conn(0x7f5cbc06a320 msgr2=0x7f5cbc0ce1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 -- 172.21.15.32:0/3070338979 shutdown_connections 2023-12-20T04:29:14.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.195+0000 7f5cc1241640 1 -- 172.21.15.32:0/3070338979 wait complete. 2023-12-20T04:29:14.216 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.2... 2023-12-20T04:29:14.216 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:14.217 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.2 -f 2023-12-20T04:29:14.241 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:29:14.241 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:14.242 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.2) 2023-12-20T04:29:14.293 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:29:14.293 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:14.304 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:29:14.304 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:14.354 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:29:14.355 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:14.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:14.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.547+0000 7f578ca12640 1 Processor -- start 2023-12-20T04:29:14.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.547+0000 7f578ca12640 1 -- start start 2023-12-20T04:29:14.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f578ca12640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f57880b0d00 0x7f5788164a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f578ca12640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57880711d0 con 0x7f57880aece0 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f578ca12640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57880c98c0 con 0x7f57881599e0 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f578ca12640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57880c9ba0 con 0x7f57880b0d00 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f577ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57880aece0 0x7f57880b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57474/0 (socket says 172.21.15.32:57474) 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f577ffff640 1 -- 172.21.15.32:0/3217531424 learned_addr learned my addr 172.21.15.32:0/3217531424 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5786d9d640 1 --2- 172.21.15.32:0/3217531424 >> v2:172.21.15.39:3300/0 conn(0x7f57880b0d00 0x7f5788164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3174094393 0 0) 0x7f57880711d0 con 0x7f57880aece0 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5774003530 con 0x7f57880aece0 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2709966047 0 0) 0x7f57880c98c0 con 0x7f57881599e0 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f57880711d0 con 0x7f57881599e0 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3133899139 0 0) 0x7f5774003530 con 0x7f57880aece0 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 >> v2:172.21.15.39:3300/0 conn(0x7f57880b0d00 msgr2=0x7f5788164a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 --2- 172.21.15.32:0/3217531424 >> v2:172.21.15.39:3300/0 conn(0x7f57880b0d00 0x7f5788164a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 >> v1:172.21.15.39:6789/0 conn(0x7f57881599e0 legacy=0x7f57880c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57880c9e80 con 0x7f57880aece0 2023-12-20T04:29:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f5770001680 con 0x7f57880aece0 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 >> v1:172.21.15.32:6789/0 conn(0x7f57880aece0 legacy=0x7f57880b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 --2- 172.21.15.32:0/3217531424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5774003b60 0x7f5774003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 --2- 172.21.15.32:0/3217531424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f57740047d0 0x7f5774004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 --2- 172.21.15.32:0/3217531424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5774005190 0x7f577400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57880c9ba0 con 0x7f5774005190 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577400e270 con 0x7f5774003b60 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577400e430 con 0x7f57740047d0 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5786d9d640 1 --2- 172.21.15.32:0/3217531424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5774005190 0x7f577400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f578659c640 1 --2- 172.21.15.32:0/3217531424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5774003b60 0x7f5774003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f577ffff640 1 --2- 172.21.15.32:0/3217531424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f57740047d0 0x7f5774004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5786d9d640 1 -- 172.21.15.32:0/3217531424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f57740047d0 msgr2=0x7f5774004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5786d9d640 1 --2- 172.21.15.32:0/3217531424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f57740047d0 0x7f5774004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5786d9d640 1 -- 172.21.15.32:0/3217531424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5774003b60 msgr2=0x7f5774003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5786d9d640 1 --2- 172.21.15.32:0/3217531424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5774003b60 0x7f5774003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.551+0000 7f5786d9d640 1 -- 172.21.15.32:0/3217531424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5774003530 con 0x7f5774005190 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f5786d9d640 1 -- 172.21.15.32:0/3217531424 reap_dead start 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f5786d9d640 1 --2- 172.21.15.32:0/3217531424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5774005190 0x7f577400dba0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5780009fc0 tx=0x7f578001cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26401ceb5d554776 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57800a5cb0 con 0x7f5774005190 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f578000f040 con 0x7f5774005190 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f5785d9b640 1 -- 172.21.15.32:0/3217531424 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5780003be0 con 0x7f5774005190 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f578ca12640 1 -- 172.21.15.32:0/3217531424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5774005190 msgr2=0x7f577400dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f578ca12640 1 --2- 172.21.15.32:0/3217531424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5774005190 0x7f577400dba0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f5780009fc0 tx=0x7f578001cca0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f578ca12640 1 -- 172.21.15.32:0/3217531424 shutdown_connections 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f578ca12640 1 --2- 172.21.15.32:0/3217531424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5774005190 0x7f577400dba0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f578ca12640 1 -- 172.21.15.32:0/3217531424 >> 172.21.15.32:0/3217531424 conn(0x7f578806a320 msgr2=0x7f57880ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f578ca12640 1 -- 172.21.15.32:0/3217531424 shutdown_connections 2023-12-20T04:29:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f578ca12640 1 -- 172.21.15.32:0/3217531424 wait complete. 2023-12-20T04:29:14.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f578ca12640 1 Processor -- start 2023-12-20T04:29:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.555+0000 7f578ca12640 1 -- start start 2023-12-20T04:29:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f578ca12640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5774005190 0x7f578815b820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f578ca12640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815bd60 0x7f57881edfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f578ca12640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57881ee4e0 0x7f57881ee960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f578ca12640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f57881eeea0 con 0x7f578815bd60 2023-12-20T04:29:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f578ca12640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5788155a50 con 0x7f57881ee4e0 2023-12-20T04:29:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f578ca12640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f578815c390 con 0x7f5774005190 2023-12-20T04:29:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815bd60 0x7f57881edfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815bd60 0x7f57881edfa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59304/0 (socket says 172.21.15.32:59304) 2023-12-20T04:29:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577ffff640 1 -- 172.21.15.32:0/4278873034 learned_addr learned my addr 172.21.15.32:0/4278873034 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f578659c640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5774005190 0x7f578815b820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f5786d9d640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57881ee4e0 0x7f57881ee960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577ffff640 1 -- 172.21.15.32:0/4278873034 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5774005190 msgr2=0x7f578815b820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577ffff640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5774005190 0x7f578815b820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577ffff640 1 -- 172.21.15.32:0/4278873034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57881ee4e0 msgr2=0x7f57881ee960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577ffff640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57881ee4e0 0x7f57881ee960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577ffff640 1 -- 172.21.15.32:0/4278873034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f578815c670 con 0x7f578815bd60 2023-12-20T04:29:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577ffff640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815bd60 0x7f57881edfa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5770001910 tx=0x7f5770011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577effd640 1 -- 172.21.15.32:0/4278873034 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f577000fb60 con 0x7f578815bd60 2023-12-20T04:29:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577effd640 1 -- 172.21.15.32:0/4278873034 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f577000fce0 con 0x7f578815bd60 2023-12-20T04:29:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f577effd640 1 -- 172.21.15.32:0/4278873034 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5770015650 con 0x7f578815bd60 2023-12-20T04:29:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f578ca12640 1 -- 172.21.15.32:0/4278873034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f57881576e0 con 0x7f578815bd60 2023-12-20T04:29:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f578ca12640 1 -- 172.21.15.32:0/4278873034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5788157300 con 0x7f578815bd60 2023-12-20T04:29:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.559+0000 7f578ca12640 1 -- 172.21.15.32:0/4278873034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5788157520 con 0x7f578815bd60 2023-12-20T04:29:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.563+0000 7f577effd640 1 -- 172.21.15.32:0/4278873034 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f57700157b0 con 0x7f578815bd60 2023-12-20T04:29:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.563+0000 7f577effd640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5768065a90 0x7f5768067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.563+0000 7f577effd640 1 -- 172.21.15.32:0/4278873034 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5770086380 con 0x7f578815bd60 2023-12-20T04:29:14.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.563+0000 7f578659c640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5768065a90 0x7f5768067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.563+0000 7f578659c640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5768065a90 0x7f5768067f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5778007fd0 tx=0x7f5778009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:14.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.567+0000 7f577effd640 1 -- 172.21.15.32:0/4278873034 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5770053d50 con 0x7f578815bd60 2023-12-20T04:29:14.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.727+0000 7f578ca12640 1 -- 172.21.15.32:0/4278873034 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f578804e000 con 0x7f578815bd60 2023-12-20T04:29:14.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.727+0000 7f577effd640 1 -- 172.21.15.32:0/4278873034 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f5770057370 con 0x7f578815bd60 2023-12-20T04:29:14.730 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:14.730 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 -- 172.21.15.32:0/4278873034 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5768065a90 msgr2=0x7f5768067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5768065a90 0x7f5768067f50 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7f5778007fd0 tx=0x7f5778009040 comp rx=0 tx=0).stop 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 -- 172.21.15.32:0/4278873034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815bd60 msgr2=0x7f57881edfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815bd60 0x7f57881edfa0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f5770001910 tx=0x7f5770011c20 comp rx=0 tx=0).stop 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 -- 172.21.15.32:0/4278873034 shutdown_connections 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5768065a90 0x7f5768067f50 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57881ee4e0 0x7f57881ee960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f578815bd60 0x7f57881edfa0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 --2- 172.21.15.32:0/4278873034 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5774005190 0x7f578815b820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 -- 172.21.15.32:0/4278873034 >> 172.21.15.32:0/4278873034 conn(0x7f578806a320 msgr2=0x7f578814ecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 -- 172.21.15.32:0/4278873034 shutdown_connections 2023-12-20T04:29:14.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.731+0000 7f578ca12640 1 -- 172.21.15.32:0/4278873034 wait complete. 2023-12-20T04:29:14.751 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:28:36.040195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:28:37.364671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:29:14.751 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:29:14.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.895+0000 7fbb69c41640 1 Processor -- start 2023-12-20T04:29:14.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.895+0000 7fbb69c41640 1 -- start start 2023-12-20T04:29:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb69c41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb640d3300 0x7fbb640d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb69c41640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb640d2650 con 0x7fbb6415ce60 2023-12-20T04:29:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb69c41640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb640d1a40 con 0x7fbb640d1300 2023-12-20T04:29:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb69c41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640d1d20 con 0x7fbb640d3300 2023-12-20T04:29:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb637fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbb6415ce60 0x7fbb640d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57490/0 (socket says 172.21.15.32:57490) 2023-12-20T04:29:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb637fe640 1 -- 172.21.15.32:0/4129225647 learned_addr learned my addr 172.21.15.32:0/4129225647 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:14.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb62ffd640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb640d3300 0x7fbb640d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 280741100 0 0) 0x7fbb640d2650 con 0x7fbb6415ce60 2023-12-20T04:29:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb4c003530 con 0x7fbb6415ce60 2023-12-20T04:29:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1371496969 0 0) 0x7fbb4c003530 con 0x7fbb6415ce60 2023-12-20T04:29:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb640d3300 msgr2=0x7fbb640d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb640d3300 0x7fbb640d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 >> v1:172.21.15.39:6789/0 conn(0x7fbb640d1300 legacy=0x7fbb640b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640d2000 con 0x7fbb6415ce60 2023-12-20T04:29:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fbb54001680 con 0x7fbb6415ce60 2023-12-20T04:29:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 >> v1:172.21.15.32:6789/0 conn(0x7fbb6415ce60 legacy=0x7fbb640d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb4c0039b0 0x7fbb4c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 0x7fbb4c004870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c005070 0x7fbb4c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640d1d20 con 0x7fbb4c0043a0 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c00bf90 con 0x7fbb4c005070 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb4c00c150 con 0x7fbb4c0039b0 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb62ffd640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 0x7fbb4c004870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb63fff640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c005070 0x7fbb4c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb637fe640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb4c0039b0 0x7fbb4c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb62ffd640 1 -- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb4c0039b0 msgr2=0x7fbb4c003dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb62ffd640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb4c0039b0 0x7fbb4c003dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb62ffd640 1 -- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c005070 msgr2=0x7fbb4c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb62ffd640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c005070 0x7fbb4c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb62ffd640 1 -- 172.21.15.32:0/4129225647 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb4c003530 con 0x7fbb4c0043a0 2023-12-20T04:29:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.899+0000 7fbb63fff640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb4c005070 0x7fbb4c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.903+0000 7fbb62ffd640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 0x7fbb4c004870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbb580099d0 tx=0x7fbb5801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29c7ebdf617e3b43 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:14.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.903+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb580a5540 con 0x7fbb4c0043a0 2023-12-20T04:29:14.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.903+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb580a56a0 con 0x7fbb4c0043a0 2023-12-20T04:29:14.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.903+0000 7fbb627fc640 1 -- 172.21.15.32:0/4129225647 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb580a5a50 con 0x7fbb4c0043a0 2023-12-20T04:29:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.903+0000 7fbb69c41640 1 -- 172.21.15.32:0/4129225647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 msgr2=0x7fbb4c004870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.903+0000 7fbb69c41640 1 --2- 172.21.15.32:0/4129225647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 0x7fbb4c004870 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fbb580099d0 tx=0x7fbb5801c950 comp rx=0 tx=0).stop 2023-12-20T04:29:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.903+0000 7fbb63fff640 1 -- 172.21.15.32:0/4129225647 reap_dead start 2023-12-20T04:29:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.903+0000 7fbb69c41640 1 -- 172.21.15.32:0/4129225647 shutdown_connections 2023-12-20T04:29:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.903+0000 7fbb69c41640 1 -- 172.21.15.32:0/4129225647 >> 172.21.15.32:0/4129225647 conn(0x7fbb6406b5e0 msgr2=0x7fbb6406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.903+0000 7fbb69c41640 1 -- 172.21.15.32:0/4129225647 shutdown_connections 2023-12-20T04:29:14.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.903+0000 7fbb69c41640 1 -- 172.21.15.32:0/4129225647 wait complete. 2023-12-20T04:29:14.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb69c41640 1 Processor -- start 2023-12-20T04:29:14.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb69c41640 1 -- start start 2023-12-20T04:29:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb69c41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 0x7fbb6414e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb69c41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb640b7000 0x7fbb640b7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb69c41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb640cad80 0x7fbb640cb200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb69c41640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6414e800 con 0x7fbb4c0043a0 2023-12-20T04:29:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb69c41640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb640b79c0 con 0x7fbb640cad80 2023-12-20T04:29:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb69c41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb6415dcf0 con 0x7fbb640b7000 2023-12-20T04:29:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 0x7fbb6414e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 0x7fbb6414e2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59316/0 (socket says 172.21.15.32:59316) 2023-12-20T04:29:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb637fe640 1 -- 172.21.15.32:0/2995757413 learned_addr learned my addr 172.21.15.32:0/2995757413 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb62ffd640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb640b7000 0x7fbb640b7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb63fff640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb640cad80 0x7fbb640cb200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb637fe640 1 -- 172.21.15.32:0/2995757413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb640b7000 msgr2=0x7fbb640b7480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb637fe640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb640b7000 0x7fbb640b7480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb637fe640 1 -- 172.21.15.32:0/2995757413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb640cad80 msgr2=0x7fbb640cb200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb637fe640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb640cad80 0x7fbb640cb200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb637fe640 1 -- 172.21.15.32:0/2995757413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb640cbd90 con 0x7fbb4c0043a0 2023-12-20T04:29:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb62ffd640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb640b7000 0x7fbb640b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb637fe640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 0x7fbb6414e2c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400e820 tx=0x7fbb5400ece0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb4bfff640 1 -- 172.21.15.32:0/2995757413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5400c8d0 con 0x7fbb4c0043a0 2023-12-20T04:29:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb4bfff640 1 -- 172.21.15.32:0/2995757413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb5400ca50 con 0x7fbb4c0043a0 2023-12-20T04:29:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb4bfff640 1 -- 172.21.15.32:0/2995757413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5400ce00 con 0x7fbb4c0043a0 2023-12-20T04:29:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb69c41640 1 -- 172.21.15.32:0/2995757413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb640cbab0 con 0x7fbb4c0043a0 2023-12-20T04:29:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb69c41640 1 -- 172.21.15.32:0/2995757413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb6415d980 con 0x7fbb4c0043a0 2023-12-20T04:29:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.907+0000 7fbb69c41640 1 -- 172.21.15.32:0/2995757413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb640d1710 con 0x7fbb4c0043a0 2023-12-20T04:29:14.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.911+0000 7fbb4bfff640 1 -- 172.21.15.32:0/2995757413 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbb5401d460 con 0x7fbb4c0043a0 2023-12-20T04:29:14.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.911+0000 7fbb4bfff640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb50065a60 0x7fbb50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:14.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.911+0000 7fbb4bfff640 1 -- 172.21.15.32:0/2995757413 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb54082fa0 con 0x7fbb4c0043a0 2023-12-20T04:29:14.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.911+0000 7fbb62ffd640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb50065a60 0x7fbb50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:14.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.911+0000 7fbb62ffd640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb50065a60 0x7fbb50067f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb580af000 tx=0x7fbb580046d0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:14.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:14.915+0000 7fbb4bfff640 1 -- 172.21.15.32:0/2995757413 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbb540b40a0 con 0x7fbb4c0043a0 2023-12-20T04:29:15.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.071+0000 7fbb69c41640 1 -- 172.21.15.32:0/2995757413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fbb6404e000 con 0x7fbb4c0043a0 2023-12-20T04:29:15.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.071+0000 7fbb4bfff640 1 -- 172.21.15.32:0/2995757413 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7fbb54050aa0 con 0x7fbb4c0043a0 2023-12-20T04:29:15.074 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 -- 172.21.15.32:0/2995757413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb50065a60 msgr2=0x7fbb50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb50065a60 0x7fbb50067f20 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fbb580af000 tx=0x7fbb580046d0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 -- 172.21.15.32:0/2995757413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 msgr2=0x7fbb6414e2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 0x7fbb6414e2c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fbb5400e820 tx=0x7fbb5400ece0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 -- 172.21.15.32:0/2995757413 shutdown_connections 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb50065a60 0x7fbb50067f20 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb640cad80 0x7fbb640cb200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb640b7000 0x7fbb640b7480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 --2- 172.21.15.32:0/2995757413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb4c0043a0 0x7fbb6414e2c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 -- 172.21.15.32:0/2995757413 >> 172.21.15.32:0/2995757413 conn(0x7fbb6406b5e0 msgr2=0x7fbb6415c080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 -- 172.21.15.32:0/2995757413 shutdown_connections 2023-12-20T04:29:15.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.075+0000 7fbb69c41640 1 -- 172.21.15.32:0/2995757413 wait complete. 2023-12-20T04:29:15.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:29:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.239+0000 7f1d47bce640 1 Processor -- start 2023-12-20T04:29:15.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.239+0000 7f1d47bce640 1 -- start start 2023-12-20T04:29:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.239+0000 7f1d47bce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d4015ce60 0x7f1d400d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d47bce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d400d2650 con 0x7f1d400d14d0 2023-12-20T04:29:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d47bce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d400d1a40 con 0x7f1d400d3130 2023-12-20T04:29:15.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d47bce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d400d35a0 con 0x7f1d4015ce60 2023-12-20T04:29:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d4596a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d4015ce60 0x7f1d400d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d4616b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1d400d14d0 0x7f1d400b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57496/0 (socket says 172.21.15.32:57496) 2023-12-20T04:29:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d4616b640 1 -- 172.21.15.32:0/3354344638 learned_addr learned my addr 172.21.15.32:0/3354344638 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d4596a640 1 -- 172.21.15.32:0/3354344638 >> v1:172.21.15.39:6789/0 conn(0x7f1d400d3130 legacy=0x7f1d400d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d4596a640 1 -- 172.21.15.32:0/3354344638 >> v1:172.21.15.32:6789/0 conn(0x7f1d400d14d0 legacy=0x7f1d400b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d4596a640 1 -- 172.21.15.32:0/3354344638 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d400d1d20 con 0x7f1d4015ce60 2023-12-20T04:29:15.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d44968640 1 -- 172.21.15.32:0/3354344638 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3220701650 0 0) 0x7f1d400d2650 con 0x7f1d400d14d0 2023-12-20T04:29:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d4596a640 1 --2- 172.21.15.32:0/3354344638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d4015ce60 0x7f1d400d2bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0097f0 tx=0x7f1d3c01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65a9ae88375dbbd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d44968640 1 -- 172.21.15.32:0/3354344638 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0a8930 con 0x7f1d4015ce60 2023-12-20T04:29:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d44968640 1 -- 172.21.15.32:0/3354344638 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d3c0a8a90 con 0x7f1d4015ce60 2023-12-20T04:29:15.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.243+0000 7f1d44968640 1 -- 172.21.15.32:0/3354344638 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d3c0a8e20 con 0x7f1d4015ce60 2023-12-20T04:29:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 -- 172.21.15.32:0/3354344638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d4015ce60 msgr2=0x7f1d400d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 --2- 172.21.15.32:0/3354344638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d4015ce60 0x7f1d400d2bf0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0097f0 tx=0x7f1d3c01ec30 comp rx=0 tx=0).stop 2023-12-20T04:29:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 -- 172.21.15.32:0/3354344638 shutdown_connections 2023-12-20T04:29:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 --2- 172.21.15.32:0/3354344638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d4015ce60 0x7f1d400d2bf0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 -- 172.21.15.32:0/3354344638 >> 172.21.15.32:0/3354344638 conn(0x7f1d4006b5e0 msgr2=0x7f1d4006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 -- 172.21.15.32:0/3354344638 shutdown_connections 2023-12-20T04:29:15.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 -- 172.21.15.32:0/3354344638 wait complete. 2023-12-20T04:29:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 Processor -- start 2023-12-20T04:29:15.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 -- start start 2023-12-20T04:29:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d400d14d0 0x7f1d40151800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400d3130 0x7f1d401e5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d4015ce60 0x7f1d400b2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40151d40 con 0x7f1d400d3130 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d40151fa0 con 0x7f1d4015ce60 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d47bce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d401f5db0 con 0x7f1d400d14d0 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d45169640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400d3130 0x7f1d401e5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d45169640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400d3130 0x7f1d401e5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59336/0 (socket says 172.21.15.32:59336) 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d45169640 1 -- 172.21.15.32:0/1349346905 learned_addr learned my addr 172.21.15.32:0/1349346905 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d4596a640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d400d14d0 0x7f1d40151800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.247+0000 7f1d4616b640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d4015ce60 0x7f1d400b2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d45169640 1 -- 172.21.15.32:0/1349346905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d400d14d0 msgr2=0x7f1d40151800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d45169640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d400d14d0 0x7f1d40151800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d45169640 1 -- 172.21.15.32:0/1349346905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d4015ce60 msgr2=0x7f1d400b2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d45169640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d4015ce60 0x7f1d400b2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d45169640 1 -- 172.21.15.32:0/1349346905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c009420 con 0x7f1d400d3130 2023-12-20T04:29:15.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d45169640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400d3130 0x7f1d401e5f80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000c450 tx=0x7f1d3000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d367fc640 1 -- 172.21.15.32:0/1349346905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30012830 con 0x7f1d400d3130 2023-12-20T04:29:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d367fc640 1 -- 172.21.15.32:0/1349346905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d300129b0 con 0x7f1d400d3130 2023-12-20T04:29:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d367fc640 1 -- 172.21.15.32:0/1349346905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d30012d60 con 0x7f1d400d3130 2023-12-20T04:29:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d47bce640 1 -- 172.21.15.32:0/1349346905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d401f6090 con 0x7f1d400d3130 2023-12-20T04:29:15.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d47bce640 1 -- 172.21.15.32:0/1349346905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d401e64c0 con 0x7f1d400d3130 2023-12-20T04:29:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d47bce640 1 -- 172.21.15.32:0/1349346905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d40152150 con 0x7f1d400d3130 2023-12-20T04:29:15.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d367fc640 1 -- 172.21.15.32:0/1349346905 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f1d3001c460 con 0x7f1d400d3130 2023-12-20T04:29:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d367fc640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d24065a90 0x7f1d24067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d367fc640 1 -- 172.21.15.32:0/1349346905 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1d30082ed0 con 0x7f1d400d3130 2023-12-20T04:29:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.251+0000 7f1d4596a640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d24065a90 0x7f1d24067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.255+0000 7f1d4596a640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d24065a90 0x7f1d24067f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0b2000 tx=0x7f1d3c0aa070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:15.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.255+0000 7f1d367fc640 1 -- 172.21.15.32:0/1349346905 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1d40152150 con 0x7f1d400d3130 2023-12-20T04:29:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 -- 172.21.15.32:0/1349346905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f1d40152110 con 0x7f1d400d3130 2023-12-20T04:29:15.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d367fc640 1 -- 172.21.15.32:0/1349346905 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f1d300508a0 con 0x7f1d400d3130 2023-12-20T04:29:15.416 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:29:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 -- 172.21.15.32:0/1349346905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d24065a90 msgr2=0x7f1d24067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d24065a90 0x7f1d24067f50 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f1d3c0b2000 tx=0x7f1d3c0aa070 comp rx=0 tx=0).stop 2023-12-20T04:29:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 -- 172.21.15.32:0/1349346905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400d3130 msgr2=0x7f1d401e5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400d3130 0x7f1d401e5f80 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f1d3000c450 tx=0x7f1d3000c910 comp rx=0 tx=0).stop 2023-12-20T04:29:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 -- 172.21.15.32:0/1349346905 shutdown_connections 2023-12-20T04:29:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d24065a90 0x7f1d24067f50 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d4015ce60 0x7f1d400b2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d400d3130 0x7f1d401e5f80 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 --2- 172.21.15.32:0/1349346905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d400d14d0 0x7f1d40151800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 -- 172.21.15.32:0/1349346905 >> 172.21.15.32:0/1349346905 conn(0x7f1d4006b5e0 msgr2=0x7f1d4006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:15.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 -- 172.21.15.32:0/1349346905 shutdown_connections 2023-12-20T04:29:15.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.415+0000 7f1d47bce640 1 -- 172.21.15.32:0/1349346905 wait complete. 2023-12-20T04:29:15.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:29:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.579+0000 7fa028598640 1 Processor -- start 2023-12-20T04:29:15.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.579+0000 7fa028598640 1 -- start start 2023-12-20T04:29:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa028598640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa02015abb0 0x7fa0200caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa028598640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa0200b0cb0 0x7fa02015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa028598640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa02015b6a0 con 0x7fa020070230 2023-12-20T04:29:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa028598640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa02006e160 con 0x7fa02015abb0 2023-12-20T04:29:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa028598640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa02015c070 con 0x7fa0200b0cb0 2023-12-20T04:29:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa026b35640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa020070230 0x7fa02006c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57512/0 (socket says 172.21.15.32:57512) 2023-12-20T04:29:15.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa026b35640 1 -- 172.21.15.32:0/1036130299 learned_addr learned my addr 172.21.15.32:0/1036130299 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa025b33640 1 --2- 172.21.15.32:0/1036130299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa0200b0cb0 0x7fa02015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa026334640 1 --2- 172.21.15.32:0/1036130299 >> v2:172.21.15.39:3300/0 conn(0x7fa02015abb0 0x7fa0200caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa025332640 1 -- 172.21.15.32:0/1036130299 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2123926692 0 0) 0x7fa02015b6a0 con 0x7fa020070230 2023-12-20T04:29:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa025332640 1 -- 172.21.15.32:0/1036130299 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa00c003530 con 0x7fa020070230 2023-12-20T04:29:15.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa025332640 1 -- 172.21.15.32:0/1036130299 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3622298584 0 0) 0x7fa00c003530 con 0x7fa020070230 2023-12-20T04:29:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa025b33640 1 -- 172.21.15.32:0/1036130299 >> v2:172.21.15.39:3300/0 conn(0x7fa02015abb0 msgr2=0x7fa0200caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa025b33640 1 --2- 172.21.15.32:0/1036130299 >> v2:172.21.15.39:3300/0 conn(0x7fa02015abb0 0x7fa0200caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa025b33640 1 -- 172.21.15.32:0/1036130299 >> v1:172.21.15.32:6789/0 conn(0x7fa020070230 legacy=0x7fa02006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.583+0000 7fa025b33640 1 -- 172.21.15.32:0/1036130299 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa02006d8c0 con 0x7fa0200b0cb0 2023-12-20T04:29:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa025b33640 1 --2- 172.21.15.32:0/1036130299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa0200b0cb0 0x7fa02015bb30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0100097c0 tx=0x7fa01001c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=979f10da399e1c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa025332640 1 -- 172.21.15.32:0/1036130299 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa018001a00 con 0x7fa0200b0cb0 2023-12-20T04:29:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa025332640 1 -- 172.21.15.32:0/1036130299 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa0100a7050 con 0x7fa0200b0cb0 2023-12-20T04:29:15.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa025332640 1 -- 172.21.15.32:0/1036130299 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0100ab620 con 0x7fa0200b0cb0 2023-12-20T04:29:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa028598640 1 -- 172.21.15.32:0/1036130299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa0200b0cb0 msgr2=0x7fa02015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa028598640 1 --2- 172.21.15.32:0/1036130299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa0200b0cb0 0x7fa02015bb30 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa0100097c0 tx=0x7fa01001c3a0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa028598640 1 -- 172.21.15.32:0/1036130299 shutdown_connections 2023-12-20T04:29:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa028598640 1 --2- 172.21.15.32:0/1036130299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa0200b0cb0 0x7fa02015bb30 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa028598640 1 --2- 172.21.15.32:0/1036130299 >> v2:172.21.15.39:3300/0 conn(0x7fa02015abb0 0x7fa0200caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa028598640 1 -- 172.21.15.32:0/1036130299 >> 172.21.15.32:0/1036130299 conn(0x7fa02006a510 msgr2=0x7fa0200ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa028598640 1 -- 172.21.15.32:0/1036130299 shutdown_connections 2023-12-20T04:29:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa028598640 1 -- 172.21.15.32:0/1036130299 wait complete. 2023-12-20T04:29:15.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa028598640 1 Processor -- start 2023-12-20T04:29:15.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.587+0000 7fa028598640 1 -- start start 2023-12-20T04:29:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa028598640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa020070230 0x7fa020146040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa028598640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa0200b0cb0 0x7fa0201f3d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa028598640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02015abb0 0x7fa02014b660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa028598640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa020148f20 con 0x7fa02015abb0 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa028598640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa020149200 con 0x7fa020070230 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa028598640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa0201494e0 con 0x7fa0200b0cb0 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa026b35640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02015abb0 0x7fa02014b660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa026b35640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02015abb0 0x7fa02014b660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59346/0 (socket says 172.21.15.32:59346) 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa026b35640 1 -- 172.21.15.32:0/2000897291 learned_addr learned my addr 172.21.15.32:0/2000897291 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa026334640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa020070230 0x7fa020146040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa025b33640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa0200b0cb0 0x7fa0201f3d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa026b35640 1 -- 172.21.15.32:0/2000897291 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa0200b0cb0 msgr2=0x7fa0201f3d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa026b35640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa0200b0cb0 0x7fa0201f3d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa026b35640 1 -- 172.21.15.32:0/2000897291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa020070230 msgr2=0x7fa020146040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa026b35640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa020070230 0x7fa020146040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa026b35640 1 -- 172.21.15.32:0/2000897291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa010009420 con 0x7fa02015abb0 2023-12-20T04:29:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa026b35640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02015abb0 0x7fa02014b660 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa01800cc70 tx=0x7fa01800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa016ffd640 1 -- 172.21.15.32:0/2000897291 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa01800ce80 con 0x7fa02015abb0 2023-12-20T04:29:15.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa016ffd640 1 -- 172.21.15.32:0/2000897291 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa018012400 con 0x7fa02015abb0 2023-12-20T04:29:15.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa016ffd640 1 -- 172.21.15.32:0/2000897291 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa0180127d0 con 0x7fa02015abb0 2023-12-20T04:29:15.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa028598640 1 -- 172.21.15.32:0/2000897291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa0201497c0 con 0x7fa02015abb0 2023-12-20T04:29:15.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa028598640 1 -- 172.21.15.32:0/2000897291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa020149fc0 con 0x7fa02015abb0 2023-12-20T04:29:15.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa028598640 1 -- 172.21.15.32:0/2000897291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa02004e000 con 0x7fa02015abb0 2023-12-20T04:29:15.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.591+0000 7fa016ffd640 1 -- 172.21.15.32:0/2000897291 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa018007070 con 0x7fa02015abb0 2023-12-20T04:29:15.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.595+0000 7fa016ffd640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa004065a60 0x7fa004067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.595+0000 7fa016ffd640 1 -- 172.21.15.32:0/2000897291 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa018083300 con 0x7fa02015abb0 2023-12-20T04:29:15.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.595+0000 7fa026334640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa004065a60 0x7fa004067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.595+0000 7fa026334640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa004065a60 0x7fa004067f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009b60 tx=0x7fa01c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:15.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.599+0000 7fa016ffd640 1 -- 172.21.15.32:0/2000897291 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa018050cd0 con 0x7fa02015abb0 2023-12-20T04:29:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.755+0000 7fa028598640 1 -- 172.21.15.32:0/2000897291 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fa020149c70 con 0x7fa02015abb0 2023-12-20T04:29:15.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.755+0000 7fa016ffd640 1 -- 172.21.15.32:0/2000897291 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7fa0180542f0 con 0x7fa02015abb0 2023-12-20T04:29:15.758 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:29:15.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 -- 172.21.15.32:0/2000897291 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa004065a60 msgr2=0x7fa004067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa004065a60 0x7fa004067f20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7fa01c009b60 tx=0x7fa01c009190 comp rx=0 tx=0).stop 2023-12-20T04:29:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 -- 172.21.15.32:0/2000897291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02015abb0 msgr2=0x7fa02014b660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02015abb0 0x7fa02014b660 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7fa01800cc70 tx=0x7fa01800a430 comp rx=0 tx=0).stop 2023-12-20T04:29:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 -- 172.21.15.32:0/2000897291 shutdown_connections 2023-12-20T04:29:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa004065a60 0x7fa004067f20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa02015abb0 0x7fa02014b660 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa0200b0cb0 0x7fa0201f3d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 --2- 172.21.15.32:0/2000897291 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa020070230 0x7fa020146040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 -- 172.21.15.32:0/2000897291 >> 172.21.15.32:0/2000897291 conn(0x7fa02006a510 msgr2=0x7fa0200cb8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 -- 172.21.15.32:0/2000897291 shutdown_connections 2023-12-20T04:29:15.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.759+0000 7fa028598640 1 -- 172.21.15.32:0/2000897291 wait complete. 2023-12-20T04:29:15.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T04:29:15.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa2ee5640 1 Processor -- start 2023-12-20T04:29:15.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa2ee5640 1 -- start start 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa2ee5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7eff9c15ce30 0x7eff9c161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa2ee5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff9c070950 0x7eff9c06d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa2ee5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff9c071920 con 0x7eff9c06dce0 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa2ee5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff9c071080 con 0x7eff9c15ce30 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa2ee5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c071360 con 0x7eff9c070950 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa0c81640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7eff9c15ce30 0x7eff9c161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa0c81640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7eff9c15ce30 0x7eff9c161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59352/0 (socket says 172.21.15.32:59352) 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa0c81640 1 -- 172.21.15.32:0/695733617 learned_addr learned my addr 172.21.15.32:0/695733617 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7eff93fff640 1 --2- 172.21.15.32:0/695733617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff9c070950 0x7eff9c06d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa0c81640 1 -- 172.21.15.32:0/695733617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff9c070950 msgr2=0x7eff9c06d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa0c81640 1 --2- 172.21.15.32:0/695733617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff9c070950 0x7eff9c06d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa0c81640 1 -- 172.21.15.32:0/695733617 >> v1:172.21.15.39:6789/0 conn(0x7eff9c06dce0 legacy=0x7eff9c069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.923+0000 7effa0c81640 1 -- 172.21.15.32:0/695733617 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff9c071640 con 0x7eff9c15ce30 2023-12-20T04:29:15.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7effa0c81640 1 --2- 172.21.15.32:0/695733617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff9c15ce30 0x7eff9c161560 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7eff98009870 tx=0x7eff9801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3077b7f0c5bc33ae server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:15.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7eff937fe640 1 -- 172.21.15.32:0/695733617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff980ab650 con 0x7eff9c15ce30 2023-12-20T04:29:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7eff937fe640 1 -- 172.21.15.32:0/695733617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff980ab7b0 con 0x7eff9c15ce30 2023-12-20T04:29:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7eff937fe640 1 -- 172.21.15.32:0/695733617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff980abb40 con 0x7eff9c15ce30 2023-12-20T04:29:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7effa2ee5640 1 -- 172.21.15.32:0/695733617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff9c15ce30 msgr2=0x7eff9c161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7effa2ee5640 1 --2- 172.21.15.32:0/695733617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff9c15ce30 0x7eff9c161560 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7eff98009870 tx=0x7eff9801c540 comp rx=0 tx=0).stop 2023-12-20T04:29:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7effa2ee5640 1 -- 172.21.15.32:0/695733617 shutdown_connections 2023-12-20T04:29:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7effa2ee5640 1 --2- 172.21.15.32:0/695733617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff9c070950 0x7eff9c06d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7effa2ee5640 1 --2- 172.21.15.32:0/695733617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff9c15ce30 0x7eff9c161560 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7effa2ee5640 1 -- 172.21.15.32:0/695733617 >> 172.21.15.32:0/695733617 conn(0x7eff9c0cba50 msgr2=0x7eff9c0cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7effa2ee5640 1 -- 172.21.15.32:0/695733617 shutdown_connections 2023-12-20T04:29:15.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.927+0000 7effa2ee5640 1 -- 172.21.15.32:0/695733617 wait complete. 2023-12-20T04:29:15.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa2ee5640 1 Processor -- start 2023-12-20T04:29:15.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa2ee5640 1 -- start start 2023-12-20T04:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa2ee5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff9c06dce0 0x7eff9c0b1080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa2ee5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff9c070950 0x7eff9c154250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa2ee5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff9c15ce30 0x7eff9c156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa2ee5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0b0bf0 con 0x7eff9c070950 2023-12-20T04:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa2ee5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0affe0 con 0x7eff9c06dce0 2023-12-20T04:29:15.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa2ee5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff9c0b02c0 con 0x7eff9c15ce30 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff93fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff9c070950 0x7eff9c154250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff93fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff9c070950 0x7eff9c154250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59368/0 (socket says 172.21.15.32:59368) 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff93fff640 1 -- 172.21.15.32:0/3319931546 learned_addr learned my addr 172.21.15.32:0/3319931546 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa0c81640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff9c06dce0 0x7eff9c0b1080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa1482640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff9c15ce30 0x7eff9c156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff93fff640 1 -- 172.21.15.32:0/3319931546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff9c15ce30 msgr2=0x7eff9c156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff93fff640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff9c15ce30 0x7eff9c156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff93fff640 1 -- 172.21.15.32:0/3319931546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff9c06dce0 msgr2=0x7eff9c0b1080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff93fff640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff9c06dce0 0x7eff9c0b1080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff93fff640 1 -- 172.21.15.32:0/3319931546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff980094d0 con 0x7eff9c070950 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa1482640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff9c15ce30 0x7eff9c156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:15.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff93fff640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff9c070950 0x7eff9c154250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00d800 tx=0x7eff8c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:15.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff917fa640 1 -- 172.21.15.32:0/3319931546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00b6c0 con 0x7eff9c070950 2023-12-20T04:29:15.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff917fa640 1 -- 172.21.15.32:0/3319931546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff8c00b820 con 0x7eff9c070950 2023-12-20T04:29:15.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7eff917fa640 1 -- 172.21.15.32:0/3319931546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff8c00bb90 con 0x7eff9c070950 2023-12-20T04:29:15.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa2ee5640 1 -- 172.21.15.32:0/3319931546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff9c0b05a0 con 0x7eff9c070950 2023-12-20T04:29:15.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa2ee5640 1 -- 172.21.15.32:0/3319931546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff9c154790 con 0x7eff9c070950 2023-12-20T04:29:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.931+0000 7effa2ee5640 1 -- 172.21.15.32:0/3319931546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff9c0b0da0 con 0x7eff9c070950 2023-12-20T04:29:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.935+0000 7eff917fa640 1 -- 172.21.15.32:0/3319931546 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7eff8c00bcf0 con 0x7eff9c070950 2023-12-20T04:29:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.935+0000 7eff917fa640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7eff88065a90 0x7eff88067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.935+0000 7eff917fa640 1 -- 172.21.15.32:0/3319931546 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7eff8c0836e0 con 0x7eff9c070950 2023-12-20T04:29:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.935+0000 7effa0c81640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7eff88065a90 0x7eff88067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:15.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.935+0000 7effa0c81640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7eff88065a90 0x7eff88067f50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff980af000 tx=0x7eff980b6040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:15.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:15.939+0000 7eff917fa640 1 -- 172.21.15.32:0/3319931546 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7eff9c0b0da0 con 0x7eff9c070950 2023-12-20T04:29:16.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 -- 172.21.15.32:0/3319931546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7eff9c04e000 con 0x7eff9c070950 2023-12-20T04:29:16.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7eff917fa640 1 -- 172.21.15.32:0/3319931546 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 0 entries v21) v1 ==== 80+0+0 (secure 0 0 0) 0x7eff8c051000 con 0x7eff9c070950 2023-12-20T04:29:16.113 INFO:teuthology.orchestra.run.smithi032.stderr:listed 0 entries 2023-12-20T04:29:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 -- 172.21.15.32:0/3319931546 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7eff88065a90 msgr2=0x7eff88067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7eff88065a90 0x7eff88067f50 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7eff980af000 tx=0x7eff980b6040 comp rx=0 tx=0).stop 2023-12-20T04:29:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 -- 172.21.15.32:0/3319931546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff9c070950 msgr2=0x7eff9c154250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff9c070950 0x7eff9c154250 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7eff8c00d800 tx=0x7eff8c00dcc0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 -- 172.21.15.32:0/3319931546 shutdown_connections 2023-12-20T04:29:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7eff88065a90 0x7eff88067f50 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff9c15ce30 0x7eff9c156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff9c070950 0x7eff9c154250 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 --2- 172.21.15.32:0/3319931546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff9c06dce0 0x7eff9c0b1080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 -- 172.21.15.32:0/3319931546 >> 172.21.15.32:0/3319931546 conn(0x7eff9c0cba50 msgr2=0x7eff9c0cbfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:16.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 -- 172.21.15.32:0/3319931546 shutdown_connections 2023-12-20T04:29:16.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.111+0000 7effa2ee5640 1 -- 172.21.15.32:0/3319931546 wait complete. 2023-12-20T04:29:16.135 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.3... 2023-12-20T04:29:16.135 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:16.135 DEBUG:teuthology.orchestra.run.smithi039:> sudo umount /home/ubuntu/cephtest/mnt.3 -f 2023-12-20T04:29:16.164 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T04:29:16.164 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:16.164 DEBUG:teuthology.orchestra.run.smithi039:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.3) 2023-12-20T04:29:16.220 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T04:29:16.221 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:29:16.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd12a62640 1 Processor -- start 2023-12-20T04:29:16.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd12a62640 1 -- start start 2023-12-20T04:29:16.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd12a62640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd0c0b8480 0x7fbd0c06af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd12a62640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbd0c06b460 0x7fbd0c160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd12a62640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd0c06b8c0 con 0x7fbd0c161330 2023-12-20T04:29:16.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd12a62640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd0c0cd4d0 con 0x7fbd0c06b460 2023-12-20T04:29:16.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd12a62640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0ce4e0 con 0x7fbd0c0b8480 2023-12-20T04:29:16.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd12261640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fbd0c161330 0x7fbd0c153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:49576/0 (socket says 172.21.15.32:49576) 2023-12-20T04:29:16.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd1125f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbd0c06b460 0x7fbd0c160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd12261640 1 -- 172.21.15.32:0/59846925 learned_addr learned my addr 172.21.15.32:0/59846925 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:16.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd11a60640 1 --2- 172.21.15.32:0/59846925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd0c0b8480 0x7fbd0c06af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd1125f640 1 -- 172.21.15.32:0/59846925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd0c0b8480 msgr2=0x7fbd0c06af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:16.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd1125f640 1 --2- 172.21.15.32:0/59846925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd0c0b8480 0x7fbd0c06af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd1125f640 1 -- 172.21.15.32:0/59846925 >> v1:172.21.15.39:6789/0 conn(0x7fbd0c161330 legacy=0x7fbd0c153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.363+0000 7fbd1125f640 1 -- 172.21.15.32:0/59846925 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd0c0cd720 con 0x7fbd0c06b460 2023-12-20T04:29:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd1125f640 1 --2- 172.21.15.32:0/59846925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd0c06b460 0x7fbd0c160df0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0097c0 tx=0x7fbcfc01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9bf5b613d135494d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd10a5e640 1 -- 172.21.15.32:0/59846925 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc0ab600 con 0x7fbd0c06b460 2023-12-20T04:29:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd10a5e640 1 -- 172.21.15.32:0/59846925 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcfc0a7050 con 0x7fbd0c06b460 2023-12-20T04:29:16.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd10a5e640 1 -- 172.21.15.32:0/59846925 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcfc0ab980 con 0x7fbd0c06b460 2023-12-20T04:29:16.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd12a62640 1 -- 172.21.15.32:0/59846925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd0c06b460 msgr2=0x7fbd0c160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd12a62640 1 --2- 172.21.15.32:0/59846925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd0c06b460 0x7fbd0c160df0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0097c0 tx=0x7fbcfc01c480 comp rx=0 tx=0).stop 2023-12-20T04:29:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd12a62640 1 -- 172.21.15.32:0/59846925 shutdown_connections 2023-12-20T04:29:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd12a62640 1 --2- 172.21.15.32:0/59846925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd0c06b460 0x7fbd0c160df0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd12a62640 1 --2- 172.21.15.32:0/59846925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd0c0b8480 0x7fbd0c06af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd12a62640 1 -- 172.21.15.32:0/59846925 >> 172.21.15.32:0/59846925 conn(0x7fbd0c0d2a50 msgr2=0x7fbd0c0d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd12a62640 1 -- 172.21.15.32:0/59846925 shutdown_connections 2023-12-20T04:29:16.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.367+0000 7fbd12a62640 1 -- 172.21.15.32:0/59846925 wait complete. 2023-12-20T04:29:16.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd12a62640 1 Processor -- start 2023-12-20T04:29:16.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd12a62640 1 -- start start 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd12a62640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0c06b460 0x7fbd0c154580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd12a62640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd0c0b8480 0x7fbd0c154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd12a62640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd0c161330 0x7fbd0c14c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd12a62640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c14c190 con 0x7fbd0c06b460 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd12a62640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c0c5d80 con 0x7fbd0c0b8480 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd12a62640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd0c14b830 con 0x7fbd0c161330 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd11a60640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0c06b460 0x7fbd0c154580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd11a60640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0c06b460 0x7fbd0c154580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59380/0 (socket says 172.21.15.32:59380) 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd11a60640 1 -- 172.21.15.32:0/1054315196 learned_addr learned my addr 172.21.15.32:0/1054315196 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd12261640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd0c161330 0x7fbd0c14c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd1125f640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd0c0b8480 0x7fbd0c154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd11a60640 1 -- 172.21.15.32:0/1054315196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd0c161330 msgr2=0x7fbd0c14c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd11a60640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd0c161330 0x7fbd0c14c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd11a60640 1 -- 172.21.15.32:0/1054315196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd0c0b8480 msgr2=0x7fbd0c154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd11a60640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd0c0b8480 0x7fbd0c154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd11a60640 1 -- 172.21.15.32:0/1054315196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcfc009420 con 0x7fbd0c06b460 2023-12-20T04:29:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd1125f640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd0c0b8480 0x7fbd0c154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:16.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd11a60640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0c06b460 0x7fbd0c154580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800e930 tx=0x7fbd0800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:16.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd027fc640 1 -- 172.21.15.32:0/1054315196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd08009690 con 0x7fbd0c06b460 2023-12-20T04:29:16.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd027fc640 1 -- 172.21.15.32:0/1054315196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd0800ca00 con 0x7fbd0c06b460 2023-12-20T04:29:16.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd12a62640 1 -- 172.21.15.32:0/1054315196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd0c14bad0 con 0x7fbd0c06b460 2023-12-20T04:29:16.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd027fc640 1 -- 172.21.15.32:0/1054315196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd0800cd70 con 0x7fbd0c06b460 2023-12-20T04:29:16.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.371+0000 7fbd12a62640 1 -- 172.21.15.32:0/1054315196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd0c1603b0 con 0x7fbd0c06b460 2023-12-20T04:29:16.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.375+0000 7fbd12a62640 1 -- 172.21.15.32:0/1054315196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd0c14c400 con 0x7fbd0c06b460 2023-12-20T04:29:16.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.375+0000 7fbd027fc640 1 -- 172.21.15.32:0/1054315196 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbd08013470 con 0x7fbd0c06b460 2023-12-20T04:29:16.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.375+0000 7fbd027fc640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcf0065a60 0x7fbcf0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.375+0000 7fbd027fc640 1 -- 172.21.15.32:0/1054315196 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd08083520 con 0x7fbd0c06b460 2023-12-20T04:29:16.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.375+0000 7fbd1125f640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcf0065a60 0x7fbcf0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.375+0000 7fbd1125f640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcf0065a60 0x7fbcf0067f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0ad000 tx=0x7fbcfc008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:16.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.379+0000 7fbd027fc640 1 -- 172.21.15.32:0/1054315196 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbd08050f00 con 0x7fbd0c06b460 2023-12-20T04:29:16.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.567+0000 7fbd12a62640 1 -- 172.21.15.32:0/1054315196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbd0c074100 con 0x7fbd0c06b460 2023-12-20T04:29:16.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd027fc640 1 -- 172.21.15.32:0/1054315196 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4700 (secure 0 0 0) 0x7fbd08054520 con 0x7fbd0c06b460 2023-12-20T04:29:16.571 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:16.572 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2247020919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2247020919},{"type":"v1","addr":"172.21.15.39:6834","nonce":2247020919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/797559820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":797559820},{"type":"v1","addr":"172.21.15.32:6836","nonce":797559820}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3209531230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3209531230},{"type":"v1","addr":"172.21.15.32:6837","nonce":3209531230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:28:38.361494+0000","modified":"2023-12-20T04:28:41.379636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547681098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3547681098},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547681098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-20T04:29:16.572 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-12-20T04:29:16.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 -- 172.21.15.32:0/1054315196 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcf0065a60 msgr2=0x7fbcf0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcf0065a60 0x7fbcf0067f20 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fbcfc0ad000 tx=0x7fbcfc008040 comp rx=0 tx=0).stop 2023-12-20T04:29:16.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 -- 172.21.15.32:0/1054315196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0c06b460 msgr2=0x7fbd0c154580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0c06b460 0x7fbd0c154580 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fbd0800e930 tx=0x7fbd0800edf0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 -- 172.21.15.32:0/1054315196 shutdown_connections 2023-12-20T04:29:16.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcf0065a60 0x7fbcf0067f20 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd0c161330 0x7fbd0c14c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd0c0b8480 0x7fbd0c154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 --2- 172.21.15.32:0/1054315196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd0c06b460 0x7fbd0c154580 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 -- 172.21.15.32:0/1054315196 >> 172.21.15.32:0/1054315196 conn(0x7fbd0c0d2a50 msgr2=0x7fbd0c0b7210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:16.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 -- 172.21.15.32:0/1054315196 shutdown_connections 2023-12-20T04:29:16.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.571+0000 7fbd12a62640 1 -- 172.21.15.32:0/1054315196 wait complete. 2023-12-20T04:29:16.593 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:16.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f0254bbc640 1 Processor -- start 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f0254bbc640 1 -- start start 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f0254bbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025015cdb0 0x7f02500d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f0254bbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02500d2740 0x7f02500cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f0254bbc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f02500d1120 0x7f02500cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f0254bbc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025015d190 con 0x7f02500d2740 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f0254bbc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02500d16a0 con 0x7f02500d1120 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f0254bbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02500d1980 con 0x7f025015cdb0 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f024dd9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02500d2740 0x7f02500cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f024dd9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02500d2740 0x7f02500cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59396/0 (socket says 172.21.15.32:59396) 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f024dd9b640 1 -- 172.21.15.32:0/3962753090 learned_addr learned my addr 172.21.15.32:0/3962753090 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f024e59c640 1 --2- 172.21.15.32:0/3962753090 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025015cdb0 0x7f02500d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f024ed9d640 1 --2- 172.21.15.32:0/3962753090 >> v2:172.21.15.39:3300/0 conn(0x7f02500d1120 0x7f02500cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f024dd9b640 1 -- 172.21.15.32:0/3962753090 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025015cdb0 msgr2=0x7f02500d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:16.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f024dd9b640 1 --2- 172.21.15.32:0/3962753090 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025015cdb0 0x7f02500d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f024dd9b640 1 -- 172.21.15.32:0/3962753090 >> v2:172.21.15.39:3300/0 conn(0x7f02500d1120 msgr2=0x7f02500cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:16.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f024dd9b640 1 --2- 172.21.15.32:0/3962753090 >> v2:172.21.15.39:3300/0 conn(0x7f02500d1120 0x7f02500cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.735+0000 7f024dd9b640 1 -- 172.21.15.32:0/3962753090 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02500d1c60 con 0x7f02500d2740 2023-12-20T04:29:16.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f024dd9b640 1 --2- 172.21.15.32:0/3962753090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02500d2740 0x7f02500cf570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f02500d3bc0 tx=0x7f024401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35220012c1de784b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:16.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f024d59a640 1 -- 172.21.15.32:0/3962753090 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02440a89b0 con 0x7f02500d2740 2023-12-20T04:29:16.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f024d59a640 1 -- 172.21.15.32:0/3962753090 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02440a8b10 con 0x7f02500d2740 2023-12-20T04:29:16.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f024d59a640 1 -- 172.21.15.32:0/3962753090 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02440a8ea0 con 0x7f02500d2740 2023-12-20T04:29:16.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f0254bbc640 1 -- 172.21.15.32:0/3962753090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02500d2740 msgr2=0x7f02500cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f0254bbc640 1 --2- 172.21.15.32:0/3962753090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02500d2740 0x7f02500cf570 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f02500d3bc0 tx=0x7f024401ecf0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f0254bbc640 1 -- 172.21.15.32:0/3962753090 shutdown_connections 2023-12-20T04:29:16.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f0254bbc640 1 --2- 172.21.15.32:0/3962753090 >> v2:172.21.15.39:3300/0 conn(0x7f02500d1120 0x7f02500cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f0254bbc640 1 --2- 172.21.15.32:0/3962753090 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02500d2740 0x7f02500cf570 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f0254bbc640 1 --2- 172.21.15.32:0/3962753090 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025015cdb0 0x7f02500d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f0254bbc640 1 -- 172.21.15.32:0/3962753090 >> 172.21.15.32:0/3962753090 conn(0x7f0250065ee0 msgr2=0x7f025006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:16.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f0254bbc640 1 -- 172.21.15.32:0/3962753090 shutdown_connections 2023-12-20T04:29:16.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.739+0000 7f0254bbc640 1 -- 172.21.15.32:0/3962753090 wait complete. 2023-12-20T04:29:16.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f0254bbc640 1 Processor -- start 2023-12-20T04:29:16.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f0254bbc640 1 -- start start 2023-12-20T04:29:16.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f0254bbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02500d1120 0x7f025014e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f0254bbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02500d2740 0x7f02500cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f0254bbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025015cdb0 0x7f02500c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f0254bbc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02500cc680 con 0x7f02500d2740 2023-12-20T04:29:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f0254bbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02500cbbd0 con 0x7f025015cdb0 2023-12-20T04:29:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f0254bbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02500cbde0 con 0x7f02500d1120 2023-12-20T04:29:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02500d2740 0x7f02500cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f024dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02500d2740 0x7f02500cb690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59410/0 (socket says 172.21.15.32:59410) 2023-12-20T04:29:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f024dd9b640 1 -- 172.21.15.32:0/2752106896 learned_addr learned my addr 172.21.15.32:0/2752106896 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f024e59c640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02500d1120 0x7f025014e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f024ed9d640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025015cdb0 0x7f02500c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f024dd9b640 1 -- 172.21.15.32:0/2752106896 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02500d1120 msgr2=0x7f025014e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f024dd9b640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02500d1120 0x7f025014e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f024dd9b640 1 -- 172.21.15.32:0/2752106896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025015cdb0 msgr2=0x7f02500c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f024dd9b640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025015cdb0 0x7f02500c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f024dd9b640 1 -- 172.21.15.32:0/2752106896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02440094d0 con 0x7f02500d2740 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f024dd9b640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02500d2740 0x7f02500cb690 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f024401d3a0 tx=0x7f024401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f023effd640 1 -- 172.21.15.32:0/2752106896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02440b0d40 con 0x7f02500d2740 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f023effd640 1 -- 172.21.15.32:0/2752106896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02440bb460 con 0x7f02500d2740 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f023effd640 1 -- 172.21.15.32:0/2752106896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02440b0ea0 con 0x7f02500d2740 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f0254bbc640 1 -- 172.21.15.32:0/2752106896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02500cc0c0 con 0x7f02500d2740 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.743+0000 7f0254bbc640 1 -- 172.21.15.32:0/2752106896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025015da90 con 0x7f02500d2740 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.747+0000 7f0254bbc640 1 -- 172.21.15.32:0/2752106896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025015d8f0 con 0x7f02500d2740 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.747+0000 7f023effd640 1 -- 172.21.15.32:0/2752106896 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f02440b83f0 con 0x7f02500d2740 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.747+0000 7f023effd640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0230065a60 0x7f0230067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.747+0000 7f023effd640 1 -- 172.21.15.32:0/2752106896 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0244120a80 con 0x7f02500d2740 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.747+0000 7f024e59c640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0230065a60 0x7f0230067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:16.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.747+0000 7f024e59c640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0230065a60 0x7f0230067f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0240009a40 tx=0x7f0240009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:16.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.751+0000 7f023effd640 1 -- 172.21.15.32:0/2752106896 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f025015d8f0 con 0x7f02500d2740 2023-12-20T04:29:16.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 -- 172.21.15.32:0/2752106896 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f025015d8f0 con 0x7f02500d2740 2023-12-20T04:29:16.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f023effd640 1 -- 172.21.15.32:0/2752106896 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f025015d8f0 con 0x7f02500d2740 2023-12-20T04:29:16.913 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 -- 172.21.15.32:0/2752106896 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0230065a60 msgr2=0x7f0230067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0230065a60 0x7f0230067f20 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f0240009a40 tx=0x7f0240009190 comp rx=0 tx=0).stop 2023-12-20T04:29:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 -- 172.21.15.32:0/2752106896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02500d2740 msgr2=0x7f02500cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02500d2740 0x7f02500cb690 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f024401d3a0 tx=0x7f024401ba90 comp rx=0 tx=0).stop 2023-12-20T04:29:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 -- 172.21.15.32:0/2752106896 shutdown_connections 2023-12-20T04:29:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0230065a60 0x7f0230067f20 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025015cdb0 0x7f02500c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02500d2740 0x7f02500cb690 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 --2- 172.21.15.32:0/2752106896 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02500d1120 0x7f025014e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 -- 172.21.15.32:0/2752106896 >> 172.21.15.32:0/2752106896 conn(0x7f0250065ee0 msgr2=0x7f02500cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 -- 172.21.15.32:0/2752106896 shutdown_connections 2023-12-20T04:29:16.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:16.911+0000 7f0254bbc640 1 -- 172.21.15.32:0/2752106896 wait complete. 2023-12-20T04:29:16.934 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:28:36.040195+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '20', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:28:37.364671+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:29:16.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:29:17.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41dc68c640 1 Processor -- start 2023-12-20T04:29:17.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41dc68c640 1 -- start start 2023-12-20T04:29:17.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41dc68c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f41d4157480 0x7f41d40c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41dc68c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d40cde20 con 0x7f41d40b0760 2023-12-20T04:29:17.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41dc68c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41d40c8e30 con 0x7f41d40cd4c0 2023-12-20T04:29:17.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41dc68c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41d415e480 con 0x7f41d4157480 2023-12-20T04:29:17.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41d9c27640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f41d40b0760 0x7f41d415d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57528/0 (socket says 172.21.15.32:57528) 2023-12-20T04:29:17.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41d9c27640 1 -- 172.21.15.32:0/609168047 learned_addr learned my addr 172.21.15.32:0/609168047 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:17.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41da428640 1 --2- 172.21.15.32:0/609168047 >> v2:172.21.15.39:3300/0 conn(0x7f41d4157480 0x7f41d40c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41d9426640 1 -- 172.21.15.32:0/609168047 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3680262275 0 0) 0x7f41d40cde20 con 0x7f41d40b0760 2023-12-20T04:29:17.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41d9426640 1 -- 172.21.15.32:0/609168047 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41c0003530 con 0x7f41d40b0760 2023-12-20T04:29:17.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41d9426640 1 -- 172.21.15.32:0/609168047 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1619070493 0 0) 0x7f41c0003530 con 0x7f41d40b0760 2023-12-20T04:29:17.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41da428640 1 -- 172.21.15.32:0/609168047 >> v1:172.21.15.39:6789/0 conn(0x7f41d40cd4c0 legacy=0x7f41d415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41da428640 1 -- 172.21.15.32:0/609168047 >> v1:172.21.15.32:6789/0 conn(0x7f41d40b0760 legacy=0x7f41d415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.079+0000 7f41da428640 1 -- 172.21.15.32:0/609168047 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d40ce070 con 0x7f41d4157480 2023-12-20T04:29:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.083+0000 7f41da428640 1 --2- 172.21.15.32:0/609168047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41d4157480 0x7f41d40c9840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f41d00097c0 tx=0x7f41d001c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8eb377fa1647fa2f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.083+0000 7f41d9426640 1 -- 172.21.15.32:0/609168047 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41c4001680 con 0x7f41d4157480 2023-12-20T04:29:17.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.083+0000 7f41d9426640 1 -- 172.21.15.32:0/609168047 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41d00a7050 con 0x7f41d4157480 2023-12-20T04:29:17.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.083+0000 7f41d9426640 1 -- 172.21.15.32:0/609168047 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41d00ab600 con 0x7f41d4157480 2023-12-20T04:29:17.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.083+0000 7f41dc68c640 1 -- 172.21.15.32:0/609168047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41d4157480 msgr2=0x7f41d40c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 --2- 172.21.15.32:0/609168047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41d4157480 0x7f41d40c9840 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f41d00097c0 tx=0x7f41d001c2e0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 -- 172.21.15.32:0/609168047 shutdown_connections 2023-12-20T04:29:17.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 --2- 172.21.15.32:0/609168047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41d4157480 0x7f41d40c9840 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 -- 172.21.15.32:0/609168047 >> 172.21.15.32:0/609168047 conn(0x7f41d406a510 msgr2=0x7f41d40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:17.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 -- 172.21.15.32:0/609168047 shutdown_connections 2023-12-20T04:29:17.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 -- 172.21.15.32:0/609168047 wait complete. 2023-12-20T04:29:17.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 Processor -- start 2023-12-20T04:29:17.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 -- start start 2023-12-20T04:29:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41d40b0760 0x7f41d414b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41d40cd4c0 0x7f41d40c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41d4157480 0x7f41d40c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d414bf70 con 0x7f41d4157480 2023-12-20T04:29:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41d4150a70 con 0x7f41d40cd4c0 2023-12-20T04:29:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dc68c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41d414fe60 con 0x7f41d40b0760 2023-12-20T04:29:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dac29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41d4157480 0x7f41d40c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dac29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41d4157480 0x7f41d40c3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59416/0 (socket says 172.21.15.32:59416) 2023-12-20T04:29:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dac29640 1 -- 172.21.15.32:0/338310674 learned_addr learned my addr 172.21.15.32:0/338310674 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41d9c27640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41d40cd4c0 0x7f41d40c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41da428640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41d40b0760 0x7f41d414b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dac29640 1 -- 172.21.15.32:0/338310674 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41d40b0760 msgr2=0x7f41d414b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dac29640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41d40b0760 0x7f41d414b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dac29640 1 -- 172.21.15.32:0/338310674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41d40cd4c0 msgr2=0x7f41d40c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dac29640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41d40cd4c0 0x7f41d40c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.087+0000 7f41dac29640 1 -- 172.21.15.32:0/338310674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41d0009420 con 0x7f41d4157480 2023-12-20T04:29:17.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.091+0000 7f41dac29640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41d4157480 0x7f41d40c3ba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00c730 tx=0x7f41cc00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:17.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.091+0000 7f41caffd640 1 -- 172.21.15.32:0/338310674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc003b80 con 0x7f41d4157480 2023-12-20T04:29:17.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.091+0000 7f41caffd640 1 -- 172.21.15.32:0/338310674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41cc003ce0 con 0x7f41d4157480 2023-12-20T04:29:17.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.091+0000 7f41dc68c640 1 -- 172.21.15.32:0/338310674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41d4150140 con 0x7f41d4157480 2023-12-20T04:29:17.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.091+0000 7f41caffd640 1 -- 172.21.15.32:0/338310674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41cc00aa30 con 0x7f41d4157480 2023-12-20T04:29:17.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.091+0000 7f41dc68c640 1 -- 172.21.15.32:0/338310674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41d4150700 con 0x7f41d4157480 2023-12-20T04:29:17.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.091+0000 7f41dc68c640 1 -- 172.21.15.32:0/338310674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41d4150c00 con 0x7f41d4157480 2023-12-20T04:29:17.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.091+0000 7f41caffd640 1 -- 172.21.15.32:0/338310674 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f41cc00ab90 con 0x7f41d4157480 2023-12-20T04:29:17.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.091+0000 7f41caffd640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f41b8065a60 0x7f41b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.091+0000 7f41caffd640 1 -- 172.21.15.32:0/338310674 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f41cc082dd0 con 0x7f41d4157480 2023-12-20T04:29:17.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.091+0000 7f41da428640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f41b8065a60 0x7f41b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.095+0000 7f41da428640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f41b8065a60 0x7f41b8067f20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f41d00ad000 tx=0x7f41d00a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:17.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.095+0000 7f41caffd640 1 -- 172.21.15.32:0/338310674 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f41cc050840 con 0x7f41d4157480 2023-12-20T04:29:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.251+0000 7f41dc68c640 1 -- 172.21.15.32:0/338310674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f41d404e000 con 0x7f41d4157480 2023-12-20T04:29:17.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.251+0000 7f41caffd640 1 -- 172.21.15.32:0/338310674 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v21) v1 ==== 93+0+10 (secure 0 0 0) 0x7f41cc084020 con 0x7f41d4157480 2023-12-20T04:29:17.255 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:29:17.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 -- 172.21.15.32:0/338310674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f41b8065a60 msgr2=0x7f41b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f41b8065a60 0x7f41b8067f20 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f41d00ad000 tx=0x7f41d00a6000 comp rx=0 tx=0).stop 2023-12-20T04:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 -- 172.21.15.32:0/338310674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41d4157480 msgr2=0x7f41d40c3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41d4157480 0x7f41d40c3ba0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f41cc00c730 tx=0x7f41cc00cbf0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 -- 172.21.15.32:0/338310674 shutdown_connections 2023-12-20T04:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f41b8065a60 0x7f41b8067f20 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41d4157480 0x7f41d40c3ba0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41d40cd4c0 0x7f41d40c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 --2- 172.21.15.32:0/338310674 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41d40b0760 0x7f41d414b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 -- 172.21.15.32:0/338310674 >> 172.21.15.32:0/338310674 conn(0x7f41d406a510 msgr2=0x7f41d40706b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 -- 172.21.15.32:0/338310674 shutdown_connections 2023-12-20T04:29:17.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.255+0000 7f41dc68c640 1 -- 172.21.15.32:0/338310674 wait complete. 2023-12-20T04:29:17.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:29:17.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5de07bc640 1 Processor -- start 2023-12-20T04:29:17.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5de07bc640 1 -- start start 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5de07bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd8157480 0x7f5dd80c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5de07bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5dd80cd4c0 0x7f5dd80b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5de07bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5dd80b1100 con 0x7f5dd80b0760 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5de07bc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5dd80cde20 con 0x7f5dd80cd4c0 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5de07bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd80c8e30 con 0x7f5dd8157480 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5dde558640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd8157480 0x7f5dd80c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5dde558640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd8157480 0x7f5dd80c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45580/0 (socket says 172.21.15.32:45580) 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5dde558640 1 -- 172.21.15.32:0/1359763771 learned_addr learned my addr 172.21.15.32:0/1359763771 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5dded59640 1 --2- 172.21.15.32:0/1359763771 >> v2:172.21.15.39:3300/0 conn(0x7f5dd80cd4c0 0x7f5dd80b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5dde558640 1 -- 172.21.15.32:0/1359763771 >> v2:172.21.15.39:3300/0 conn(0x7f5dd80cd4c0 msgr2=0x7f5dd80b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5dde558640 1 --2- 172.21.15.32:0/1359763771 >> v2:172.21.15.39:3300/0 conn(0x7f5dd80cd4c0 0x7f5dd80b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5dde558640 1 -- 172.21.15.32:0/1359763771 >> v1:172.21.15.39:6789/0 conn(0x7f5dd80b0760 legacy=0x7f5dd815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.423+0000 7f5dde558640 1 -- 172.21.15.32:0/1359763771 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd80ce070 con 0x7f5dd8157480 2023-12-20T04:29:17.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5dde558640 1 --2- 172.21.15.32:0/1359763771 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd8157480 0x7f5dd80c9840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009870 tx=0x7f5dd401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=deb826c79d1702c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:17.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5ddd556640 1 -- 172.21.15.32:0/1359763771 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40ac600 con 0x7f5dd8157480 2023-12-20T04:29:17.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5ddd556640 1 -- 172.21.15.32:0/1359763771 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd40a8050 con 0x7f5dd8157480 2023-12-20T04:29:17.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5ddd556640 1 -- 172.21.15.32:0/1359763771 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40ac980 con 0x7f5dd8157480 2023-12-20T04:29:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5de07bc640 1 -- 172.21.15.32:0/1359763771 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd8157480 msgr2=0x7f5dd80c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5de07bc640 1 --2- 172.21.15.32:0/1359763771 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd8157480 0x7f5dd80c9840 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4009870 tx=0x7f5dd401c480 comp rx=0 tx=0).stop 2023-12-20T04:29:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5de07bc640 1 -- 172.21.15.32:0/1359763771 shutdown_connections 2023-12-20T04:29:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5de07bc640 1 --2- 172.21.15.32:0/1359763771 >> v2:172.21.15.39:3300/0 conn(0x7f5dd80cd4c0 0x7f5dd80b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5de07bc640 1 --2- 172.21.15.32:0/1359763771 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd8157480 0x7f5dd80c9840 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5de07bc640 1 -- 172.21.15.32:0/1359763771 >> 172.21.15.32:0/1359763771 conn(0x7f5dd806a510 msgr2=0x7f5dd80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5de07bc640 1 -- 172.21.15.32:0/1359763771 shutdown_connections 2023-12-20T04:29:17.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.427+0000 7f5de07bc640 1 -- 172.21.15.32:0/1359763771 wait complete. 2023-12-20T04:29:17.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5de07bc640 1 Processor -- start 2023-12-20T04:29:17.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5de07bc640 1 -- start start 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5de07bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd80b0760 0x7f5dd80b4290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5de07bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dd80cd4c0 0x7f5dd8162b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5de07bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dd8157480 0x7f5dd81630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5de07bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd815c750 con 0x7f5dd80cd4c0 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5de07bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd8152b10 con 0x7f5dd8157480 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5de07bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5dd815bb40 con 0x7f5dd80b0760 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dde558640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd80b0760 0x7f5dd80b4290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dded59640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dd8157480 0x7f5dd81630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dde558640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd80b0760 0x7f5dd80b4290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45592/0 (socket says 172.21.15.32:45592) 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dde558640 1 -- 172.21.15.32:0/1539990313 learned_addr learned my addr 172.21.15.32:0/1539990313 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dddd57640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dd80cd4c0 0x7f5dd8162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dde558640 1 -- 172.21.15.32:0/1539990313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dd8157480 msgr2=0x7f5dd81630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dde558640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dd8157480 0x7f5dd81630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dde558640 1 -- 172.21.15.32:0/1539990313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dd80cd4c0 msgr2=0x7f5dd8162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dde558640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dd80cd4c0 0x7f5dd8162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dde558640 1 -- 172.21.15.32:0/1539990313 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5dd40094d0 con 0x7f5dd80b0760 2023-12-20T04:29:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dded59640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dd8157480 0x7f5dd81630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:17.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dde558640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd80b0760 0x7f5dd80b4290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5dd401ab30 tx=0x7f5dd401acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dceffd640 1 -- 172.21.15.32:0/1539990313 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40b7420 con 0x7f5dd80b0760 2023-12-20T04:29:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dceffd640 1 -- 172.21.15.32:0/1539990313 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5dd40b75a0 con 0x7f5dd80b0760 2023-12-20T04:29:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5dceffd640 1 -- 172.21.15.32:0/1539990313 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5dd40b7950 con 0x7f5dd80b0760 2023-12-20T04:29:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5de07bc640 1 -- 172.21.15.32:0/1539990313 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5dd815be20 con 0x7f5dd80b0760 2023-12-20T04:29:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.431+0000 7f5de07bc640 1 -- 172.21.15.32:0/1539990313 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5dd815c3e0 con 0x7f5dd80b0760 2023-12-20T04:29:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.435+0000 7f5de07bc640 1 -- 172.21.15.32:0/1539990313 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5dd804e000 con 0x7f5dd80b0760 2023-12-20T04:29:17.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.435+0000 7f5dceffd640 1 -- 172.21.15.32:0/1539990313 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f5dd40b3040 con 0x7f5dd80b0760 2023-12-20T04:29:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.435+0000 7f5dceffd640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5dbc065a60 0x7f5dbc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.435+0000 7f5dceffd640 1 -- 172.21.15.32:0/1539990313 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5dd411d1a0 con 0x7f5dd80b0760 2023-12-20T04:29:17.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.435+0000 7f5dddd57640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5dbc065a60 0x7f5dbc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.435+0000 7f5dddd57640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5dbc065a60 0x7f5dbc067f20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5dd81f8530 tx=0x7f5dc8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:17.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.439+0000 7f5dceffd640 1 -- 172.21.15.32:0/1539990313 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5dd40eac00 con 0x7f5dd80b0760 2023-12-20T04:29:17.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 -- 172.21.15.32:0/1539990313 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f5dd81533a0 con 0x7f5dd80b0760 2023-12-20T04:29:17.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5dceffd640 1 -- 172.21.15.32:0/1539990313 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v21) v1 ==== 104+0+11 (secure 0 0 0) 0x7f5dd40ee220 con 0x7f5dd80b0760 2023-12-20T04:29:17.601 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:29:17.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 -- 172.21.15.32:0/1539990313 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5dbc065a60 msgr2=0x7f5dbc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5dbc065a60 0x7f5dbc067f20 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f5dd81f8530 tx=0x7f5dc8009040 comp rx=0 tx=0).stop 2023-12-20T04:29:17.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 -- 172.21.15.32:0/1539990313 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd80b0760 msgr2=0x7f5dd80b4290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd80b0760 0x7f5dd80b4290 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f5dd401ab30 tx=0x7f5dd401acc0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 -- 172.21.15.32:0/1539990313 shutdown_connections 2023-12-20T04:29:17.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5dbc065a60 0x7f5dbc067f20 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5dd8157480 0x7f5dd81630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5dd80cd4c0 0x7f5dd8162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 --2- 172.21.15.32:0/1539990313 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5dd80b0760 0x7f5dd80b4290 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 -- 172.21.15.32:0/1539990313 >> 172.21.15.32:0/1539990313 conn(0x7f5dd806a510 msgr2=0x7f5dd806d790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:17.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 -- 172.21.15.32:0/1539990313 shutdown_connections 2023-12-20T04:29:17.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.599+0000 7f5de07bc640 1 -- 172.21.15.32:0/1539990313 wait complete. 2023-12-20T04:29:17.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:29:17.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f161de12640 1 Processor -- start 2023-12-20T04:29:17.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f161de12640 1 -- start start 2023-12-20T04:29:17.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f161de12640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1618070940 0x7f16180caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f161de12640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161815b6a0 con 0x7f16180b0cb0 2023-12-20T04:29:17.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f161de12640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f161806d8c0 con 0x7f161815abb0 2023-12-20T04:29:17.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f161de12640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f161806e160 con 0x7f1618070940 2023-12-20T04:29:17.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f1616ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f16180b0cb0 0x7f1618070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57536/0 (socket says 172.21.15.32:57536) 2023-12-20T04:29:17.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f1616ffd640 1 -- 172.21.15.32:0/2507025109 learned_addr learned my addr 172.21.15.32:0/2507025109 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:17.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f1617fff640 1 --2- 172.21.15.32:0/2507025109 >> v2:172.21.15.32:3300/0 conn(0x7f1618070940 0x7f16180caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f1617fff640 1 -- 172.21.15.32:0/2507025109 >> v1:172.21.15.39:6789/0 conn(0x7f161815abb0 legacy=0x7f161815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f1617fff640 1 -- 172.21.15.32:0/2507025109 >> v1:172.21.15.32:6789/0 conn(0x7f16180b0cb0 legacy=0x7f1618070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f1617fff640 1 -- 172.21.15.32:0/2507025109 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161806dba0 con 0x7f1618070940 2023-12-20T04:29:17.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.767+0000 7f16167fc640 1 -- 172.21.15.32:0/2507025109 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1040098094 0 0) 0x7f161815b6a0 con 0x7f16180b0cb0 2023-12-20T04:29:17.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.771+0000 7f1617fff640 1 --2- 172.21.15.32:0/2507025109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1618070940 0x7f16180caaf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1610009d90 tx=0x7f161001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=954074f4207d0f2b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:17.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.771+0000 7f16167fc640 1 -- 172.21.15.32:0/2507025109 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16100a5a10 con 0x7f1618070940 2023-12-20T04:29:17.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.771+0000 7f16167fc640 1 -- 172.21.15.32:0/2507025109 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16100a5b70 con 0x7f1618070940 2023-12-20T04:29:17.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.771+0000 7f16167fc640 1 -- 172.21.15.32:0/2507025109 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16100ad440 con 0x7f1618070940 2023-12-20T04:29:17.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.771+0000 7f161de12640 1 -- 172.21.15.32:0/2507025109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1618070940 msgr2=0x7f16180caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.771+0000 7f161de12640 1 --2- 172.21.15.32:0/2507025109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1618070940 0x7f16180caaf0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f1610009d90 tx=0x7f161001c950 comp rx=0 tx=0).stop 2023-12-20T04:29:17.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.771+0000 7f161de12640 1 -- 172.21.15.32:0/2507025109 shutdown_connections 2023-12-20T04:29:17.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.771+0000 7f161de12640 1 --2- 172.21.15.32:0/2507025109 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1618070940 0x7f16180caaf0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.771+0000 7f161de12640 1 -- 172.21.15.32:0/2507025109 >> 172.21.15.32:0/2507025109 conn(0x7f161806a510 msgr2=0x7f16180ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:17.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.771+0000 7f161de12640 1 -- 172.21.15.32:0/2507025109 shutdown_connections 2023-12-20T04:29:17.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.771+0000 7f161de12640 1 -- 172.21.15.32:0/2507025109 wait complete. 2023-12-20T04:29:17.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f161de12640 1 Processor -- start 2023-12-20T04:29:17.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f161de12640 1 -- start start 2023-12-20T04:29:17.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f161de12640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1618070940 0x7f1618135b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f161de12640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16180b0cb0 0x7f161814f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f161de12640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161815abb0 0x7f161814c390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f161de12640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1618145ae0 con 0x7f161815abb0 2023-12-20T04:29:17.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f161de12640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1618145f00 con 0x7f1618070940 2023-12-20T04:29:17.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f161de12640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1618144e60 con 0x7f16180b0cb0 2023-12-20T04:29:17.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f1617fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161815abb0 0x7f161814c390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f1617fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161815abb0 0x7f161814c390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59430/0 (socket says 172.21.15.32:59430) 2023-12-20T04:29:17.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f1617fff640 1 -- 172.21.15.32:0/2117700413 learned_addr learned my addr 172.21.15.32:0/2117700413 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:17.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f1616ffd640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16180b0cb0 0x7f161814f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f16177fe640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1618070940 0x7f1618135b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f1617fff640 1 -- 172.21.15.32:0/2117700413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16180b0cb0 msgr2=0x7f161814f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f1617fff640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16180b0cb0 0x7f161814f560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f1617fff640 1 -- 172.21.15.32:0/2117700413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1618070940 msgr2=0x7f1618135b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f1617fff640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1618070940 0x7f1618135b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f1617fff640 1 -- 172.21.15.32:0/2117700413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1610009a70 con 0x7f161815abb0 2023-12-20T04:29:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f1617fff640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161815abb0 0x7f161814c390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f161001a580 tx=0x7f16100196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f15fffff640 1 -- 172.21.15.32:0/2117700413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16100ade40 con 0x7f161815abb0 2023-12-20T04:29:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f15fffff640 1 -- 172.21.15.32:0/2117700413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16100b9450 con 0x7f161815abb0 2023-12-20T04:29:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f15fffff640 1 -- 172.21.15.32:0/2117700413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16100b97e0 con 0x7f161815abb0 2023-12-20T04:29:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f161de12640 1 -- 172.21.15.32:0/2117700413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1618145140 con 0x7f161815abb0 2023-12-20T04:29:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f161de12640 1 -- 172.21.15.32:0/2117700413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1618145700 con 0x7f161815abb0 2023-12-20T04:29:17.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.775+0000 7f161de12640 1 -- 172.21.15.32:0/2117700413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1618145c20 con 0x7f161815abb0 2023-12-20T04:29:17.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.779+0000 7f15fffff640 1 -- 172.21.15.32:0/2117700413 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f16100b9940 con 0x7f161815abb0 2023-12-20T04:29:17.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.779+0000 7f15fffff640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065a60 0x7f1604067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:17.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.779+0000 7f15fffff640 1 -- 172.21.15.32:0/2117700413 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f161011e9f0 con 0x7f161815abb0 2023-12-20T04:29:17.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.779+0000 7f16177fe640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065a60 0x7f1604067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:17.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.779+0000 7f16177fe640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065a60 0x7f1604067f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1608000f70 tx=0x7f1608009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:17.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.783+0000 7f15fffff640 1 -- 172.21.15.32:0/2117700413 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f16100ec460 con 0x7f161815abb0 2023-12-20T04:29:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 -- 172.21.15.32:0/2117700413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f161804e000 con 0x7f161815abb0 2023-12-20T04:29:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f15fffff640 1 -- 172.21.15.32:0/2117700413 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v21) v1 ==== 100+0+11 (secure 0 0 0) 0x7f16100efa80 con 0x7f161815abb0 2023-12-20T04:29:17.944 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:29:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 -- 172.21.15.32:0/2117700413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065a60 msgr2=0x7f1604067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065a60 0x7f1604067f20 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f1608000f70 tx=0x7f1608009040 comp rx=0 tx=0).stop 2023-12-20T04:29:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 -- 172.21.15.32:0/2117700413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161815abb0 msgr2=0x7f161814c390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161815abb0 0x7f161814c390 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f161001a580 tx=0x7f16100196f0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 -- 172.21.15.32:0/2117700413 shutdown_connections 2023-12-20T04:29:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065a60 0x7f1604067f20 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f161815abb0 0x7f161814c390 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16180b0cb0 0x7f161814f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 --2- 172.21.15.32:0/2117700413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1618070940 0x7f1618135b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 -- 172.21.15.32:0/2117700413 >> 172.21.15.32:0/2117700413 conn(0x7f161806a510 msgr2=0x7f16180c4270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 -- 172.21.15.32:0/2117700413 shutdown_connections 2023-12-20T04:29:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:17.943+0000 7f161de12640 1 -- 172.21.15.32:0/2117700413 wait complete. 2023-12-20T04:29:17.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:29:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.107+0000 7fed03841640 1 Processor -- start 2023-12-20T04:29:18.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.107+0000 7fed03841640 1 -- start start 2023-12-20T04:29:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.107+0000 7fed03841640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fecfc15f110 0x7fecfc15c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.107+0000 7fed03841640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecfc15b990 con 0x7fecfc0af5e0 2023-12-20T04:29:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.107+0000 7fed03841640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fecfc15bbd0 con 0x7fecfc06ce60 2023-12-20T04:29:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.107+0000 7fed03841640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fecfc15beb0 con 0x7fecfc15f110 2023-12-20T04:29:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.107+0000 7fed015dd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fecfc0af5e0 0x7fecfc0af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57550/0 (socket says 172.21.15.32:57550) 2023-12-20T04:29:18.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.107+0000 7fed015dd640 1 -- 172.21.15.32:0/3012831967 learned_addr learned my addr 172.21.15.32:0/3012831967 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fed01dde640 1 --2- 172.21.15.32:0/3012831967 >> v2:172.21.15.39:3300/0 conn(0x7fecfc15f110 0x7fecfc15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1128004804 0 0) 0x7fecf80012a0 con 0x7fecfc0af5e0 2023-12-20T04:29:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fece8003530 con 0x7fecfc0af5e0 2023-12-20T04:29:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 446649833 0 0) 0x7fece8003530 con 0x7fecfc0af5e0 2023-12-20T04:29:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 >> v2:172.21.15.39:3300/0 conn(0x7fecfc15f110 msgr2=0x7fecfc15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 --2- 172.21.15.32:0/3012831967 >> v2:172.21.15.39:3300/0 conn(0x7fecfc15f110 0x7fecfc15c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 >> v1:172.21.15.39:6789/0 conn(0x7fecfc06ce60 legacy=0x7fecfc15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc15e910 con 0x7fecfc0af5e0 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fecf8001990 con 0x7fecfc0af5e0 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 >> v1:172.21.15.32:6789/0 conn(0x7fecfc0af5e0 legacy=0x7fecfc0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 --2- 172.21.15.32:0/3012831967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fece8003980 0x7fece8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 --2- 172.21.15.32:0/3012831967 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fece80047d0 0x7fece8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 --2- 172.21.15.32:0/3012831967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fece8005190 0x7fece800fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc15beb0 con 0x7fece8005190 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fece8007970 con 0x7fece8003980 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fece8007b30 con 0x7fece80047d0 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fed01dde640 1 --2- 172.21.15.32:0/3012831967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fece8005190 0x7fece800fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fed015dd640 1 --2- 172.21.15.32:0/3012831967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fece8003980 0x7fece8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fed00ddc640 1 --2- 172.21.15.32:0/3012831967 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fece80047d0 0x7fece8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fed01dde640 1 -- 172.21.15.32:0/3012831967 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fece80047d0 msgr2=0x7fece8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fed01dde640 1 --2- 172.21.15.32:0/3012831967 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fece80047d0 0x7fece8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fed01dde640 1 -- 172.21.15.32:0/3012831967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fece8003980 msgr2=0x7fece8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:18.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fed01dde640 1 --2- 172.21.15.32:0/3012831967 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fece8003980 0x7fece8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.111+0000 7fed01dde640 1 -- 172.21.15.32:0/3012831967 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fece8003530 con 0x7fece8005190 2023-12-20T04:29:18.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fed01dde640 1 -- 172.21.15.32:0/3012831967 reap_dead start 2023-12-20T04:29:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fed01dde640 1 --2- 172.21.15.32:0/3012831967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fece8005190 0x7fece800fda0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009fc0 tx=0x7fecf401cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd6f2b57c553b101 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4003ba0 con 0x7fece8005190 2023-12-20T04:29:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf400f040 con 0x7fece8005190 2023-12-20T04:29:18.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fecf3fff640 1 -- 172.21.15.32:0/3012831967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40a5890 con 0x7fece8005190 2023-12-20T04:29:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fed03841640 1 -- 172.21.15.32:0/3012831967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fece8005190 msgr2=0x7fece800fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fed03841640 1 --2- 172.21.15.32:0/3012831967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fece8005190 0x7fece800fda0 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7fecf4009fc0 tx=0x7fecf401cca0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fed03841640 1 -- 172.21.15.32:0/3012831967 shutdown_connections 2023-12-20T04:29:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fed03841640 1 --2- 172.21.15.32:0/3012831967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fece8005190 0x7fece800fda0 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fed03841640 1 -- 172.21.15.32:0/3012831967 >> 172.21.15.32:0/3012831967 conn(0x7fecfc06a320 msgr2=0x7fecfc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fed03841640 1 -- 172.21.15.32:0/3012831967 shutdown_connections 2023-12-20T04:29:18.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fed03841640 1 -- 172.21.15.32:0/3012831967 wait complete. 2023-12-20T04:29:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fed03841640 1 Processor -- start 2023-12-20T04:29:18.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.115+0000 7fed03841640 1 -- start start 2023-12-20T04:29:18.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed03841640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fece8005190 0x7fecfc0c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed03841640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fecfc0c83a0 0x7fecfc0c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed03841640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecfc071110 0x7fecfc071590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed03841640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc071ad0 con 0x7fecfc071110 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed03841640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc0b1aa0 con 0x7fece8005190 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed03841640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fecfc0b1d80 con 0x7fecfc0c83a0 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed01dde640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecfc071110 0x7fecfc071590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed01dde640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecfc071110 0x7fecfc071590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59448/0 (socket says 172.21.15.32:59448) 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed01dde640 1 -- 172.21.15.32:0/3363080105 learned_addr learned my addr 172.21.15.32:0/3363080105 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed015dd640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fece8005190 0x7fecfc0c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed00ddc640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fecfc0c83a0 0x7fecfc0c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed01dde640 1 -- 172.21.15.32:0/3363080105 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fecfc0c83a0 msgr2=0x7fecfc0c4010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed01dde640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fecfc0c83a0 0x7fecfc0c4010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed01dde640 1 -- 172.21.15.32:0/3363080105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fece8005190 msgr2=0x7fecfc0c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed01dde640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fece8005190 0x7fecfc0c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed01dde640 1 -- 172.21.15.32:0/3363080105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fecfc0b2060 con 0x7fecfc071110 2023-12-20T04:29:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed00ddc640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fecfc0c83a0 0x7fecfc0c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed015dd640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fece8005190 0x7fecfc0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed01dde640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecfc071110 0x7fecfc071590 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fecf401c710 tx=0x7fecf4009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fecf1ffb640 1 -- 172.21.15.32:0/3363080105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf4003e70 con 0x7fecfc071110 2023-12-20T04:29:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fecf1ffb640 1 -- 172.21.15.32:0/3363080105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fecf4004030 con 0x7fecfc071110 2023-12-20T04:29:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed03841640 1 -- 172.21.15.32:0/3363080105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fecfc0b2340 con 0x7fecfc071110 2023-12-20T04:29:18.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fecf1ffb640 1 -- 172.21.15.32:0/3363080105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fecf40043a0 con 0x7fecfc071110 2023-12-20T04:29:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed03841640 1 -- 172.21.15.32:0/3363080105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fecfc0c5d70 con 0x7fecfc071110 2023-12-20T04:29:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fed03841640 1 -- 172.21.15.32:0/3363080105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fecfc04e000 con 0x7fecfc071110 2023-12-20T04:29:18.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.119+0000 7fecf1ffb640 1 -- 172.21.15.32:0/3363080105 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fecf4004500 con 0x7fecfc071110 2023-12-20T04:29:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.123+0000 7fecf1ffb640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fece0065a90 0x7fece0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.123+0000 7fecf1ffb640 1 -- 172.21.15.32:0/3363080105 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fecf411dd80 con 0x7fecfc071110 2023-12-20T04:29:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.123+0000 7fed015dd640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fece0065a90 0x7fece0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.123+0000 7fed015dd640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fece0065a90 0x7fece0067f50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fecf8007fd0 tx=0x7fecf8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:18.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.127+0000 7fecf1ffb640 1 -- 172.21.15.32:0/3363080105 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fecf40eb6a0 con 0x7fecfc071110 2023-12-20T04:29:18.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.315+0000 7fed03841640 1 -- 172.21.15.32:0/3363080105 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fecfc0cf450 con 0x7fecfc071110 2023-12-20T04:29:18.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.315+0000 7fecf1ffb640 1 -- 172.21.15.32:0/3363080105 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4700 (secure 0 0 0) 0x7fecf40bd020 con 0x7fecfc071110 2023-12-20T04:29:18.319 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:18.319 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2247020919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2247020919},{"type":"v1","addr":"172.21.15.39:6834","nonce":2247020919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/797559820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":797559820},{"type":"v1","addr":"172.21.15.32:6836","nonce":797559820}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3209531230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3209531230},{"type":"v1","addr":"172.21.15.32:6837","nonce":3209531230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:28:38.361494+0000","modified":"2023-12-20T04:28:41.379636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547681098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3547681098},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547681098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-20T04:29:18.319 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-12-20T04:29:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 -- 172.21.15.32:0/3363080105 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fece0065a90 msgr2=0x7fece0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fece0065a90 0x7fece0067f50 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fecf8007fd0 tx=0x7fecf8009040 comp rx=0 tx=0).stop 2023-12-20T04:29:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 -- 172.21.15.32:0/3363080105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecfc071110 msgr2=0x7fecfc071590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecfc071110 0x7fecfc071590 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7fecf401c710 tx=0x7fecf4009750 comp rx=0 tx=0).stop 2023-12-20T04:29:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 -- 172.21.15.32:0/3363080105 shutdown_connections 2023-12-20T04:29:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fece0065a90 0x7fece0067f50 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fecfc071110 0x7fecfc071590 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fecfc0c83a0 0x7fecfc0c4010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 --2- 172.21.15.32:0/3363080105 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fece8005190 0x7fecfc0c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 -- 172.21.15.32:0/3363080105 >> 172.21.15.32:0/3363080105 conn(0x7fecfc06a320 msgr2=0x7fecfc158df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:18.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 -- 172.21.15.32:0/3363080105 shutdown_connections 2023-12-20T04:29:18.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.319+0000 7fed03841640 1 -- 172.21.15.32:0/3363080105 wait complete. 2023-12-20T04:29:18.340 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.483+0000 7f6eee77b640 1 Processor -- start 2023-12-20T04:29:18.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.483+0000 7f6eee77b640 1 -- start start 2023-12-20T04:29:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6eee77b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ee815cd70 0x7f6ee80cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6eee77b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6ee80b23d0 0x7f6ee80b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6eee77b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6ee80b2d70 con 0x7f6ee81608e0 2023-12-20T04:29:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6eee77b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6ee81618b0 con 0x7f6ee815cd70 2023-12-20T04:29:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6eee77b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee80ce6c0 con 0x7f6ee80b23d0 2023-12-20T04:29:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6ee7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6ee815cd70 0x7f6ee80cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6eecd18640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6ee81608e0 0x7f6ee8160d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57562/0 (socket says 172.21.15.32:57562) 2023-12-20T04:29:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6eecd18640 1 -- 172.21.15.32:0/2033142280 learned_addr learned my addr 172.21.15.32:0/2033142280 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6ee77fe640 1 --2- 172.21.15.32:0/2033142280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6ee80b23d0 0x7f6ee80b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6ee7fff640 1 -- 172.21.15.32:0/2033142280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6ee80b23d0 msgr2=0x7f6ee80b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6ee7fff640 1 --2- 172.21.15.32:0/2033142280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6ee80b23d0 0x7f6ee80b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6ee7fff640 1 -- 172.21.15.32:0/2033142280 >> v1:172.21.15.32:6789/0 conn(0x7f6ee81608e0 legacy=0x7f6ee8160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6ee7fff640 1 -- 172.21.15.32:0/2033142280 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ee81615d0 con 0x7f6ee815cd70 2023-12-20T04:29:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6ee77fe640 1 --2- 172.21.15.32:0/2033142280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6ee80b23d0 0x7f6ee80b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:18.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.487+0000 7f6ee6ffd640 1 -- 172.21.15.32:0/2033142280 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2089434753 0 0) 0x7f6ee80b2d70 con 0x7f6ee81608e0 2023-12-20T04:29:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6ee7fff640 1 --2- 172.21.15.32:0/2033142280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee815cd70 0x7f6ee80cf160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80097f0 tx=0x7f6ed801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d10ce10604e284a8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6ee6ffd640 1 -- 172.21.15.32:0/2033142280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80ab650 con 0x7f6ee815cd70 2023-12-20T04:29:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6ee6ffd640 1 -- 172.21.15.32:0/2033142280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ed80ab7b0 con 0x7f6ee815cd70 2023-12-20T04:29:18.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6ee6ffd640 1 -- 172.21.15.32:0/2033142280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ed80abb40 con 0x7f6ee815cd70 2023-12-20T04:29:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6eee77b640 1 -- 172.21.15.32:0/2033142280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee815cd70 msgr2=0x7f6ee80cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6eee77b640 1 --2- 172.21.15.32:0/2033142280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee815cd70 0x7f6ee80cf160 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80097f0 tx=0x7f6ed801c450 comp rx=0 tx=0).stop 2023-12-20T04:29:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6eee77b640 1 -- 172.21.15.32:0/2033142280 shutdown_connections 2023-12-20T04:29:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6eee77b640 1 --2- 172.21.15.32:0/2033142280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6ee80b23d0 0x7f6ee80b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6eee77b640 1 --2- 172.21.15.32:0/2033142280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee815cd70 0x7f6ee80cf160 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6eee77b640 1 -- 172.21.15.32:0/2033142280 >> 172.21.15.32:0/2033142280 conn(0x7f6ee80cd3d0 msgr2=0x7f6ee80cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6eee77b640 1 -- 172.21.15.32:0/2033142280 shutdown_connections 2023-12-20T04:29:18.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6eee77b640 1 -- 172.21.15.32:0/2033142280 wait complete. 2023-12-20T04:29:18.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6eee77b640 1 Processor -- start 2023-12-20T04:29:18.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.491+0000 7f6eee77b640 1 -- start start 2023-12-20T04:29:18.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eee77b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6ee80b23d0 0x7f6ee80c6410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eee77b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ee815cd70 0x7f6ee815ad20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eee77b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee81608e0 0x7f6ee80cd7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eee77b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee815b700 con 0x7f6ee81608e0 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eee77b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee815b9e0 con 0x7f6ee815cd70 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eee77b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6ee815bcc0 con 0x7f6ee80b23d0 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eecd18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee81608e0 0x7f6ee80cd7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eecd18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee81608e0 0x7f6ee80cd7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59470/0 (socket says 172.21.15.32:59470) 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eecd18640 1 -- 172.21.15.32:0/30306434 learned_addr learned my addr 172.21.15.32:0/30306434 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6ee7fff640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6ee80b23d0 0x7f6ee80c6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6ee77fe640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ee815cd70 0x7f6ee815ad20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eecd18640 1 -- 172.21.15.32:0/30306434 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6ee80b23d0 msgr2=0x7f6ee80c6410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eecd18640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6ee80b23d0 0x7f6ee80c6410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eecd18640 1 -- 172.21.15.32:0/30306434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ee815cd70 msgr2=0x7f6ee815ad20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eecd18640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ee815cd70 0x7f6ee815ad20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eecd18640 1 -- 172.21.15.32:0/30306434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ed8009420 con 0x7f6ee81608e0 2023-12-20T04:29:18.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eecd18640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee81608e0 0x7f6ee80cd7b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000c780 tx=0x7f6ee000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:18.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6ee4ff9640 1 -- 172.21.15.32:0/30306434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee000ce90 con 0x7f6ee81608e0 2023-12-20T04:29:18.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6ee4ff9640 1 -- 172.21.15.32:0/30306434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ee000a640 con 0x7f6ee81608e0 2023-12-20T04:29:18.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6ee4ff9640 1 -- 172.21.15.32:0/30306434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ee000a9d0 con 0x7f6ee81608e0 2023-12-20T04:29:18.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eee77b640 1 -- 172.21.15.32:0/30306434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ee815bfa0 con 0x7f6ee81608e0 2023-12-20T04:29:18.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eee77b640 1 -- 172.21.15.32:0/30306434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6ee80c6950 con 0x7f6ee81608e0 2023-12-20T04:29:18.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6eee77b640 1 -- 172.21.15.32:0/30306434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6ee80d0270 con 0x7f6ee81608e0 2023-12-20T04:29:18.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.495+0000 7f6ee4ff9640 1 -- 172.21.15.32:0/30306434 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6ee000ab30 con 0x7f6ee81608e0 2023-12-20T04:29:18.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.499+0000 7f6ee4ff9640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ed4065a60 0x7f6ed4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.499+0000 7f6ee4ff9640 1 -- 172.21.15.32:0/30306434 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6ee00825b0 con 0x7f6ee81608e0 2023-12-20T04:29:18.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.499+0000 7f6ee7fff640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ed4065a60 0x7f6ed4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.499+0000 7f6ee7fff640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ed4065a60 0x7f6ed4067f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80af000 tx=0x7f6ed80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:18.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.503+0000 7f6ee4ff9640 1 -- 172.21.15.32:0/30306434 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6ee00500b0 con 0x7f6ee81608e0 2023-12-20T04:29:18.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.659+0000 7f6eee77b640 1 -- 172.21.15.32:0/30306434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6ee804e000 con 0x7f6ee81608e0 2023-12-20T04:29:18.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.659+0000 7f6ee4ff9640 1 -- 172.21.15.32:0/30306434 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v21) v1 ==== 74+0+15989 (secure 0 0 0) 0x7f6ee0018090 con 0x7f6ee81608e0 2023-12-20T04:29:18.662 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:18.663 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":21,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:28:39.364898+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:18.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 -- 172.21.15.32:0/30306434 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ed4065a60 msgr2=0x7f6ed4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ed4065a60 0x7f6ed4067f20 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80af000 tx=0x7f6ed80a7000 comp rx=0 tx=0).stop 2023-12-20T04:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 -- 172.21.15.32:0/30306434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee81608e0 msgr2=0x7f6ee80cd7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee81608e0 0x7f6ee80cd7b0 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f6ee000c780 tx=0x7f6ee000cc40 comp rx=0 tx=0).stop 2023-12-20T04:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 -- 172.21.15.32:0/30306434 shutdown_connections 2023-12-20T04:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ed4065a60 0x7f6ed4067f20 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6ee81608e0 0x7f6ee80cd7b0 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6ee815cd70 0x7f6ee815ad20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 --2- 172.21.15.32:0/30306434 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6ee80b23d0 0x7f6ee80c6410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 -- 172.21.15.32:0/30306434 >> 172.21.15.32:0/30306434 conn(0x7f6ee80cd3d0 msgr2=0x7f6ee80cb190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 -- 172.21.15.32:0/30306434 shutdown_connections 2023-12-20T04:29:18.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.663+0000 7f6eee77b640 1 -- 172.21.15.32:0/30306434 wait complete. 2023-12-20T04:29:18.684 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T04:29:18.684 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:29:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.823+0000 7f0a6d3f3640 1 Processor -- start 2023-12-20T04:29:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a6d3f3640 1 -- start start 2023-12-20T04:29:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a6d3f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a681599e0 0x7f0a680aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a6d3f3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0a680c90e0 0x7f0a680b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a6d3f3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a680711d0 con 0x7f0a680b0b30 2023-12-20T04:29:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a6d3f3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a680c98c0 con 0x7f0a680c90e0 2023-12-20T04:29:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a6d3f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a680c9ba0 con 0x7f0a681599e0 2023-12-20T04:29:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a677fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a680b0b30 0x7f0a6815d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52590/0 (socket says 172.21.15.32:52590) 2023-12-20T04:29:18.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a677fe640 1 -- 172.21.15.32:0/364116615 learned_addr learned my addr 172.21.15.32:0/364116615 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a66ffd640 1 --2- 172.21.15.32:0/364116615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a681599e0 0x7f0a680aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a5e7fc640 1 --2- 172.21.15.32:0/364116615 >> v2:172.21.15.39:3300/0 conn(0x7f0a680c90e0 0x7f0a680b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a667fc640 1 -- 172.21.15.32:0/364116615 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1134188531 0 0) 0x7f0a680711d0 con 0x7f0a680b0b30 2023-12-20T04:29:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a667fc640 1 -- 172.21.15.32:0/364116615 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a54003530 con 0x7f0a680b0b30 2023-12-20T04:29:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a667fc640 1 -- 172.21.15.32:0/364116615 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 677802318 0 0) 0x7f0a54003530 con 0x7f0a680b0b30 2023-12-20T04:29:18.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a66ffd640 1 -- 172.21.15.32:0/364116615 >> v2:172.21.15.39:3300/0 conn(0x7f0a680c90e0 msgr2=0x7f0a680b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a66ffd640 1 --2- 172.21.15.32:0/364116615 >> v2:172.21.15.39:3300/0 conn(0x7f0a680c90e0 0x7f0a680b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a66ffd640 1 -- 172.21.15.32:0/364116615 >> v1:172.21.15.32:6789/0 conn(0x7f0a680b0b30 legacy=0x7f0a6815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.827+0000 7f0a66ffd640 1 -- 172.21.15.32:0/364116615 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a680c9e80 con 0x7f0a681599e0 2023-12-20T04:29:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a66ffd640 1 --2- 172.21.15.32:0/364116615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a681599e0 0x7f0a680aece0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009de0 tx=0x7f0a5801c880 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7efa9f4ef3f94454 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a667fc640 1 -- 172.21.15.32:0/364116615 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a60001680 con 0x7f0a681599e0 2023-12-20T04:29:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a667fc640 1 -- 172.21.15.32:0/364116615 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a580a55a0 con 0x7f0a681599e0 2023-12-20T04:29:18.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a667fc640 1 -- 172.21.15.32:0/364116615 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a580a5910 con 0x7f0a681599e0 2023-12-20T04:29:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/364116615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a681599e0 msgr2=0x7f0a680aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a6d3f3640 1 --2- 172.21.15.32:0/364116615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a681599e0 0x7f0a680aece0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0a58009de0 tx=0x7f0a5801c880 comp rx=0 tx=0).stop 2023-12-20T04:29:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/364116615 shutdown_connections 2023-12-20T04:29:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a6d3f3640 1 --2- 172.21.15.32:0/364116615 >> v2:172.21.15.39:3300/0 conn(0x7f0a680c90e0 0x7f0a680b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a6d3f3640 1 --2- 172.21.15.32:0/364116615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a681599e0 0x7f0a680aece0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/364116615 >> 172.21.15.32:0/364116615 conn(0x7f0a6806a320 msgr2=0x7f0a680ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/364116615 shutdown_connections 2023-12-20T04:29:18.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.831+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/364116615 wait complete. 2023-12-20T04:29:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a6d3f3640 1 Processor -- start 2023-12-20T04:29:18.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a6d3f3640 1 -- start start 2023-12-20T04:29:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a6d3f3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a680b0b30 0x7f0a681f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a6d3f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a680c90e0 0x7f0a681585a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a6d3f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a681599e0 0x7f0a68158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a6d3f3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a680c86f0 con 0x7f0a680b0b30 2023-12-20T04:29:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a6d3f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a681f64c0 con 0x7f0a681599e0 2023-12-20T04:29:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a6d3f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a680c7ae0 con 0x7f0a680c90e0 2023-12-20T04:29:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a66ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a680b0b30 0x7f0a681f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a66ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a680b0b30 0x7f0a681f5f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37906/0 (socket says 172.21.15.32:37906) 2023-12-20T04:29:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a66ffd640 1 -- 172.21.15.32:0/796223981 learned_addr learned my addr 172.21.15.32:0/796223981 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a5e7fc640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a680c90e0 0x7f0a681585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a677fe640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a681599e0 0x7f0a68158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a66ffd640 1 -- 172.21.15.32:0/796223981 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a680c90e0 msgr2=0x7f0a681585a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a66ffd640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a680c90e0 0x7f0a681585a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a66ffd640 1 -- 172.21.15.32:0/796223981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a681599e0 msgr2=0x7f0a68158ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a66ffd640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a681599e0 0x7f0a68158ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a66ffd640 1 -- 172.21.15.32:0/796223981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a58009a70 con 0x7f0a680b0b30 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a677fe640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a681599e0 0x7f0a68158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a66ffd640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a680b0b30 0x7f0a681f5f80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0a5801d120 tx=0x7f0a58019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a5ffff640 1 -- 172.21.15.32:0/796223981 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a580bd070 con 0x7f0a680b0b30 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a5ffff640 1 -- 172.21.15.32:0/796223981 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a580a5920 con 0x7f0a680b0b30 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a5ffff640 1 -- 172.21.15.32:0/796223981 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a580a5cd0 con 0x7f0a680b0b30 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/796223981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a680c7dc0 con 0x7f0a680b0b30 2023-12-20T04:29:18.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/796223981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a680c8380 con 0x7f0a680b0b30 2023-12-20T04:29:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.835+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/796223981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a680b0f80 con 0x7f0a680b0b30 2023-12-20T04:29:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.839+0000 7f0a5ffff640 1 -- 172.21.15.32:0/796223981 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f0a58006d50 con 0x7f0a680b0b30 2023-12-20T04:29:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.839+0000 7f0a5ffff640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a48065a60 0x7f0a48067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.839+0000 7f0a5ffff640 1 -- 172.21.15.32:0/796223981 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(21..21 src has 1..21) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0a5811da70 con 0x7f0a680b0b30 2023-12-20T04:29:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.839+0000 7f0a5e7fc640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a48065a60 0x7f0a48067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:18.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.839+0000 7f0a5e7fc640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a48065a60 0x7f0a48067f20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0a681610b0 tx=0x7f0a50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:18.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:18.843+0000 7f0a5ffff640 1 -- 172.21.15.32:0/796223981 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0a580eb420 con 0x7f0a680b0b30 2023-12-20T04:29:19.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/796223981 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a6804e000 con 0x7f0a680b0b30 2023-12-20T04:29:19.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a5ffff640 1 -- 172.21.15.32:0/796223981 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4700 (secure 0 0 0) 0x7f0a580b2070 con 0x7f0a680b0b30 2023-12-20T04:29:19.041 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:19.041 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2247020919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2247020919},{"type":"v1","addr":"172.21.15.39:6834","nonce":2247020919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/797559820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":797559820},{"type":"v1","addr":"172.21.15.32:6836","nonce":797559820}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3209531230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3209531230},{"type":"v1","addr":"172.21.15.32:6837","nonce":3209531230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:28:38.361494+0000","modified":"2023-12-20T04:28:41.379636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547681098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3547681098},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547681098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-20T04:29:19.041 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-12-20T04:29:19.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/796223981 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a48065a60 msgr2=0x7f0a48067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a48065a60 0x7f0a48067f20 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f0a681610b0 tx=0x7f0a50008040 comp rx=0 tx=0).stop 2023-12-20T04:29:19.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/796223981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a680b0b30 msgr2=0x7f0a681f5f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a680b0b30 0x7f0a681f5f80 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f0a5801d120 tx=0x7f0a58019620 comp rx=0 tx=0).stop 2023-12-20T04:29:19.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/796223981 shutdown_connections 2023-12-20T04:29:19.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a48065a60 0x7f0a48067f20 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a681599e0 0x7f0a68158ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a680c90e0 0x7f0a681585a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 --2- 172.21.15.32:0/796223981 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a680b0b30 0x7f0a681f5f80 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/796223981 >> 172.21.15.32:0/796223981 conn(0x7f0a6806a320 msgr2=0x7f0a68070b10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:19.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/796223981 shutdown_connections 2023-12-20T04:29:19.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.039+0000 7f0a6d3f3640 1 -- 172.21.15.32:0/796223981 wait complete. 2023-12-20T04:29:19.064 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:29:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.203+0000 7f2a0bbf0640 1 Processor -- start 2023-12-20T04:29:19.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.203+0000 7f2a0bbf0640 1 -- start start 2023-12-20T04:29:19.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0bbf0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2a040af5e0 0x7f2a040af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0bbf0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a0415f110 0x7f2a0415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0bbf0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a0415b990 con 0x7f2a0406ce60 2023-12-20T04:29:19.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0bbf0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0415bbd0 con 0x7f2a0415f110 2023-12-20T04:29:19.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0bbf0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2a0415beb0 con 0x7f2a040af5e0 2023-12-20T04:29:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0a18d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2a0415f110 0x7f2a0415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0918b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2a0406ce60 0x7f2a0415b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52594/0 (socket says 172.21.15.32:52594) 2023-12-20T04:29:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0918b640 1 -- 172.21.15.32:0/3045726776 learned_addr learned my addr 172.21.15.32:0/3045726776 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0998c640 1 --2- 172.21.15.32:0/3045726776 >> v2:172.21.15.39:3300/0 conn(0x7f2a040af5e0 0x7f2a040af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0a18d640 1 -- 172.21.15.32:0/3045726776 >> v2:172.21.15.39:3300/0 conn(0x7f2a040af5e0 msgr2=0x7f2a040af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0a18d640 1 --2- 172.21.15.32:0/3045726776 >> v2:172.21.15.39:3300/0 conn(0x7f2a040af5e0 0x7f2a040af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0a18d640 1 -- 172.21.15.32:0/3045726776 >> v1:172.21.15.32:6789/0 conn(0x7f2a0406ce60 legacy=0x7f2a0415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0a18d640 1 -- 172.21.15.32:0/3045726776 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a0415e910 con 0x7f2a0415f110 2023-12-20T04:29:19.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.207+0000 7f2a0898a640 1 -- 172.21.15.32:0/3045726776 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3018944595 0 0) 0x7f29f4001120 con 0x7f2a0406ce60 2023-12-20T04:29:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0a18d640 1 --2- 172.21.15.32:0/3045726776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a0415f110 0x7f2a0415c440 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2a0406ac10 tx=0x7f29fc01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d61ac8a99aff5e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0898a640 1 -- 172.21.15.32:0/3045726776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0a59c0 con 0x7f2a0415f110 2023-12-20T04:29:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0898a640 1 -- 172.21.15.32:0/3045726776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29fc0a5b20 con 0x7f2a0415f110 2023-12-20T04:29:19.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0898a640 1 -- 172.21.15.32:0/3045726776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29fc0ad440 con 0x7f2a0415f110 2023-12-20T04:29:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/3045726776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a0415f110 msgr2=0x7f2a0415c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0bbf0640 1 --2- 172.21.15.32:0/3045726776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a0415f110 0x7f2a0415c440 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f2a0406ac10 tx=0x7f29fc01c8e0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/3045726776 shutdown_connections 2023-12-20T04:29:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0bbf0640 1 --2- 172.21.15.32:0/3045726776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a0415f110 0x7f2a0415c440 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0bbf0640 1 --2- 172.21.15.32:0/3045726776 >> v2:172.21.15.39:3300/0 conn(0x7f2a040af5e0 0x7f2a040af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/3045726776 >> 172.21.15.32:0/3045726776 conn(0x7f2a0406a320 msgr2=0x7f2a040ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/3045726776 shutdown_connections 2023-12-20T04:29:19.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/3045726776 wait complete. 2023-12-20T04:29:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0bbf0640 1 Processor -- start 2023-12-20T04:29:19.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.211+0000 7f2a0bbf0640 1 -- start start 2023-12-20T04:29:19.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0bbf0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a0406ce60 0x7f2a040c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0bbf0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040af5e0 0x7f2a04158e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0bbf0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a0415f110 0x7f2a04159390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0bbf0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04071a20 con 0x7f2a0406ce60 2023-12-20T04:29:19.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0bbf0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04071d00 con 0x7f2a040af5e0 2023-12-20T04:29:19.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0bbf0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a04071fe0 con 0x7f2a0415f110 2023-12-20T04:29:19.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0998c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a0406ce60 0x7f2a040c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0998c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a0406ce60 0x7f2a040c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37912/0 (socket says 172.21.15.32:37912) 2023-12-20T04:29:19.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0998c640 1 -- 172.21.15.32:0/2928926889 learned_addr learned my addr 172.21.15.32:0/2928926889 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0918b640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040af5e0 0x7f2a04158e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0a18d640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a0415f110 0x7f2a04159390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0998c640 1 -- 172.21.15.32:0/2928926889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a0415f110 msgr2=0x7f2a04159390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0998c640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a0415f110 0x7f2a04159390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0998c640 1 -- 172.21.15.32:0/2928926889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040af5e0 msgr2=0x7f2a04158e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0998c640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040af5e0 0x7f2a04158e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0998c640 1 -- 172.21.15.32:0/2928926889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29fc009a70 con 0x7f2a0406ce60 2023-12-20T04:29:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0998c640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a0406ce60 0x7f2a040c80e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000e9b0 tx=0x7f2a0000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f29fa7fc640 1 -- 172.21.15.32:0/2928926889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a00009670 con 0x7f2a0406ce60 2023-12-20T04:29:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f29fa7fc640 1 -- 172.21.15.32:0/2928926889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a0000c9a0 con 0x7f2a0406ce60 2023-12-20T04:29:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f29fa7fc640 1 -- 172.21.15.32:0/2928926889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a0000cd10 con 0x7f2a0406ce60 2023-12-20T04:29:19.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/2928926889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a040c6000 con 0x7f2a0406ce60 2023-12-20T04:29:19.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/2928926889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a04164920 con 0x7f2a0406ce60 2023-12-20T04:29:19.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/2928926889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a040c5e60 con 0x7f2a0406ce60 2023-12-20T04:29:19.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.215+0000 7f29fa7fc640 1 -- 172.21.15.32:0/2928926889 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f2a00013470 con 0x7f2a0406ce60 2023-12-20T04:29:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.219+0000 7f29fa7fc640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f29e8065a90 0x7f29e8067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.219+0000 7f29fa7fc640 1 -- 172.21.15.32:0/2928926889 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f2a00083810 con 0x7f2a0406ce60 2023-12-20T04:29:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.219+0000 7f2a0918b640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f29e8065a90 0x7f29e8067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.219+0000 7f2a0918b640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f29e8065a90 0x7f29e8067f50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2a04159b80 tx=0x7f29f4007ba0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:19.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.223+0000 7f29fa7fc640 1 -- 172.21.15.32:0/2928926889 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2a040c5e60 con 0x7f2a0406ce60 2023-12-20T04:29:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/2928926889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2a040c5e60 con 0x7f2a0406ce60 2023-12-20T04:29:19.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f29fa7fc640 1 -- 172.21.15.32:0/2928926889 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) v1 ==== 93+0+4700 (secure 0 0 0) 0x7f2a040c5e60 con 0x7f2a0406ce60 2023-12-20T04:29:19.417 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:19.417 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":8,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4241,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2247020919","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2247020919},{"type":"v1","addr":"172.21.15.39:6834","nonce":2247020919}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3},{"gid":4254,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/797559820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":797559820},{"type":"v1","addr":"172.21.15.32:6836","nonce":797559820}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":2},{"gid":4257,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3209531230","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3209531230},{"type":"v1","addr":"172.21.15.32:6837","nonce":3209531230}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":3}],"filesystems":[{"mdsmap":{"epoch":8,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:28:38.361494+0000","modified":"2023-12-20T04:28:41.379636+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":4300},"failed":[],"damaged":[],"stopped":[],"info":{"gid_4300":{"gid":4300,"name":"d","rank":0,"incarnation":4,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/3547681098","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3547681098},{"type":"v1","addr":"172.21.15.39:6835","nonce":3547681098}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[3],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":1}]} 2023-12-20T04:29:19.418 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 8 2023-12-20T04:29:19.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/2928926889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f29e8065a90 msgr2=0x7f29e8067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f29e8065a90 0x7f29e8067f50 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f2a04159b80 tx=0x7f29f4007ba0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/2928926889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a0406ce60 msgr2=0x7f2a040c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a0406ce60 0x7f2a040c80e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f2a0000e9b0 tx=0x7f2a0000ee70 comp rx=0 tx=0).stop 2023-12-20T04:29:19.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/2928926889 shutdown_connections 2023-12-20T04:29:19.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f29e8065a90 0x7f29e8067f50 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a0415f110 0x7f2a04159390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a040af5e0 0x7f2a04158e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 --2- 172.21.15.32:0/2928926889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a0406ce60 0x7f2a040c80e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/2928926889 >> 172.21.15.32:0/2928926889 conn(0x7f2a0406a320 msgr2=0x7f2a040cb850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/2928926889 shutdown_connections 2023-12-20T04:29:19.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.415+0000 7f2a0bbf0640 1 -- 172.21.15.32:0/2928926889 wait complete. 2023-12-20T04:29:19.438 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:19.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.579+0000 7fd4001b3640 1 Processor -- start 2023-12-20T04:29:19.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.579+0000 7fd4001b3640 1 -- start start 2023-12-20T04:29:19.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd4001b3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd3f806cf10 0x7fd3f80c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd4001b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f8164ef0 0x7fd3f80c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd4001b3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3f80ae3f0 con 0x7fd3f80ae010 2023-12-20T04:29:19.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd4001b3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3f8161800 con 0x7fd3f806cf10 2023-12-20T04:29:19.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd4001b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f8165380 con 0x7fd3f8164ef0 2023-12-20T04:29:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fe750640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f8164ef0 0x7fd3f80c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fe750640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f8164ef0 0x7fd3f80c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:38466/0 (socket says 172.21.15.32:38466) 2023-12-20T04:29:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fd74e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd3f806cf10 0x7fd3f80c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fe750640 1 -- 172.21.15.32:0/2934296424 learned_addr learned my addr 172.21.15.32:0/2934296424 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fdf4f640 1 --1- 172.21.15.32:0/2934296424 >> v1:172.21.15.39:6789/0 conn(0x7fd3f80ae010 0x7fd3f8071470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:54764/0 (socket says 172.21.15.32:54764) 2023-12-20T04:29:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fcf4d640 1 -- 172.21.15.32:0/2934296424 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3181020706 0 0) 0x7fd3f80ae3f0 con 0x7fd3f80ae010 2023-12-20T04:29:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fcf4d640 1 -- 172.21.15.32:0/2934296424 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd3e4003530 con 0x7fd3f80ae010 2023-12-20T04:29:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fd74e640 1 -- 172.21.15.32:0/2934296424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f8164ef0 msgr2=0x7fd3f80c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fd74e640 1 --2- 172.21.15.32:0/2934296424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f8164ef0 0x7fd3f80c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fd74e640 1 -- 172.21.15.32:0/2934296424 >> v1:172.21.15.39:6789/0 conn(0x7fd3f80ae010 legacy=0x7fd3f8071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fd74e640 1 -- 172.21.15.32:0/2934296424 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f8165660 con 0x7fd3f806cf10 2023-12-20T04:29:19.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.583+0000 7fd3fe750640 1 --2- 172.21.15.32:0/2934296424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f8164ef0 0x7fd3f80c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:19.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd3fd74e640 1 --2- 172.21.15.32:0/2934296424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3f806cf10 0x7fd3f80c94a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80097c0 tx=0x7fd3e801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=725ecae8113d1c9f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:19.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd3fcf4d640 1 -- 172.21.15.32:0/2934296424 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80ac620 con 0x7fd3f806cf10 2023-12-20T04:29:19.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd3fcf4d640 1 -- 172.21.15.32:0/2934296424 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3e80ac780 con 0x7fd3f806cf10 2023-12-20T04:29:19.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd3fcf4d640 1 -- 172.21.15.32:0/2934296424 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3e80acb10 con 0x7fd3f806cf10 2023-12-20T04:29:19.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd4001b3640 1 -- 172.21.15.32:0/2934296424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3f806cf10 msgr2=0x7fd3f80c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd4001b3640 1 --2- 172.21.15.32:0/2934296424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3f806cf10 0x7fd3f80c94a0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80097c0 tx=0x7fd3e801c3c0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd4001b3640 1 -- 172.21.15.32:0/2934296424 shutdown_connections 2023-12-20T04:29:19.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd4001b3640 1 --2- 172.21.15.32:0/2934296424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f8164ef0 0x7fd3f80c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd4001b3640 1 --2- 172.21.15.32:0/2934296424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3f806cf10 0x7fd3f80c94a0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd4001b3640 1 -- 172.21.15.32:0/2934296424 >> 172.21.15.32:0/2934296424 conn(0x7fd3f806a320 msgr2=0x7fd3f80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:19.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd4001b3640 1 -- 172.21.15.32:0/2934296424 shutdown_connections 2023-12-20T04:29:19.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd4001b3640 1 -- 172.21.15.32:0/2934296424 wait complete. 2023-12-20T04:29:19.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd4001b3640 1 Processor -- start 2023-12-20T04:29:19.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.587+0000 7fd4001b3640 1 -- start start 2023-12-20T04:29:19.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd4001b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3f80ae010 0x7fd3f8155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd4001b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f80c47d0 0x7fd3f81629c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd4001b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f80c4c50 0x7fd3f8162f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd4001b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80c5cb0 con 0x7fd3f80c4c50 2023-12-20T04:29:19.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd4001b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80c3aa0 con 0x7fd3f80ae010 2023-12-20T04:29:19.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd4001b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3f80c34e0 con 0x7fd3f80c47d0 2023-12-20T04:29:19.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3fe750640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f80c4c50 0x7fd3f8162f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3fdf4f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3f80ae010 0x7fd3f8155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3fe750640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f80c4c50 0x7fd3f8162f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37918/0 (socket says 172.21.15.32:37918) 2023-12-20T04:29:19.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3fe750640 1 -- 172.21.15.32:0/1203869221 learned_addr learned my addr 172.21.15.32:0/1203869221 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:19.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3fd74e640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f80c47d0 0x7fd3f81629c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3fdf4f640 1 -- 172.21.15.32:0/1203869221 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f80c47d0 msgr2=0x7fd3f81629c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3fdf4f640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f80c47d0 0x7fd3f81629c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3fdf4f640 1 -- 172.21.15.32:0/1203869221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f80c4c50 msgr2=0x7fd3f8162f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3fdf4f640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f80c4c50 0x7fd3f8162f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3fdf4f640 1 -- 172.21.15.32:0/1203869221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3f0009b30 con 0x7fd3f80ae010 2023-12-20T04:29:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3fdf4f640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3f80ae010 0x7fd3f8155d60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400c430 tx=0x7fd3f400c8f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3ee7fc640 1 -- 172.21.15.32:0/1203869221 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f4005c80 con 0x7fd3f80ae010 2023-12-20T04:29:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3ee7fc640 1 -- 172.21.15.32:0/1203869221 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3f4005de0 con 0x7fd3f80ae010 2023-12-20T04:29:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd3ee7fc640 1 -- 172.21.15.32:0/1203869221 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3f40072f0 con 0x7fd3f80ae010 2023-12-20T04:29:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd4001b3640 1 -- 172.21.15.32:0/1203869221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3e8009420 con 0x7fd3f80ae010 2023-12-20T04:29:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd4001b3640 1 -- 172.21.15.32:0/1203869221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3f80c58d0 con 0x7fd3f80ae010 2023-12-20T04:29:19.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.591+0000 7fd4001b3640 1 -- 172.21.15.32:0/1203869221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3f8159620 con 0x7fd3f80ae010 2023-12-20T04:29:19.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.595+0000 7fd3ee7fc640 1 -- 172.21.15.32:0/1203869221 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd3f4007450 con 0x7fd3f80ae010 2023-12-20T04:29:19.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.595+0000 7fd3ee7fc640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd3dc065a60 0x7fd3dc067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.595+0000 7fd3ee7fc640 1 -- 172.21.15.32:0/1203869221 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(22..22 src has 1..22) v4 ==== 5825+0+0 (secure 0 0 0) 0x7fd3f40827b0 con 0x7fd3f80ae010 2023-12-20T04:29:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.595+0000 7fd3fd74e640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd3dc065a60 0x7fd3dc067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.595+0000 7fd3fd74e640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd3dc065a60 0x7fd3dc067f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80ae000 tx=0x7fd3e8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:19.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.599+0000 7fd3ee7fc640 1 -- 172.21.15.32:0/1203869221 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd3f4050190 con 0x7fd3f80ae010 2023-12-20T04:29:19.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.759+0000 7fd4001b3640 1 -- 172.21.15.32:0/1203869221 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd3f804e000 con 0x7fd3f80ae010 2023-12-20T04:29:19.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.759+0000 7fd3ee7fc640 1 -- 172.21.15.32:0/1203869221 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v22) v1 ==== 74+0+16238 (secure 0 0 0) 0x7fd3f40537b0 con 0x7fd3f80ae010 2023-12-20T04:29:19.762 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:19.763 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":22,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:29:18.906694+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:28:36.040195+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2023-12-20T04:28:37.364671+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"3.d","mappings":[{"from":7,"to":2}]},{"pgid":"3.16","mappings":[{"from":5,"to":2}]},{"pgid":"3.1f","mappings":[{"from":5,"to":1}]},{"pgid":"3.38","mappings":[{"from":3,"to":2}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:19.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 -- 172.21.15.32:0/1203869221 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd3dc065a60 msgr2=0x7fd3dc067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd3dc065a60 0x7fd3dc067f20 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7fd3e80ae000 tx=0x7fd3e8008040 comp rx=0 tx=0).stop 2023-12-20T04:29:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 -- 172.21.15.32:0/1203869221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3f80ae010 msgr2=0x7fd3f8155d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3f80ae010 0x7fd3f8155d60 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7fd3f400c430 tx=0x7fd3f400c8f0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 -- 172.21.15.32:0/1203869221 shutdown_connections 2023-12-20T04:29:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd3dc065a60 0x7fd3dc067f20 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3f80c4c50 0x7fd3f8162f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3f80c47d0 0x7fd3f81629c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 --2- 172.21.15.32:0/1203869221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3f80ae010 0x7fd3f8155d60 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 -- 172.21.15.32:0/1203869221 >> 172.21.15.32:0/1203869221 conn(0x7fd3f806a320 msgr2=0x7fd3f80b0a90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 -- 172.21.15.32:0/1203869221 shutdown_connections 2023-12-20T04:29:19.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.763+0000 7fd4001b3640 1 -- 172.21.15.32:0/1203869221 wait complete. 2023-12-20T04:29:19.784 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T04:29:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.927+0000 7f576b48c640 1 Processor -- start 2023-12-20T04:29:19.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.927+0000 7f576b48c640 1 -- start start 2023-12-20T04:29:19.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f576b48c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57640b0d00 0x7f5764164a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f576b48c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57640711d0 con 0x7f57640aece0 2023-12-20T04:29:19.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f576b48c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f57640c98c0 con 0x7f57641599e0 2023-12-20T04:29:19.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f576b48c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f57640c9ba0 con 0x7f57640b0d00 2023-12-20T04:29:19.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f5769a29640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f57640b0d00 0x7f5764164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f5768a27640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f57640aece0 0x7f57640b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52602/0 (socket says 172.21.15.32:52602) 2023-12-20T04:29:19.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f5768a27640 1 -- 172.21.15.32:0/4039951330 learned_addr learned my addr 172.21.15.32:0/4039951330 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f5769a29640 1 -- 172.21.15.32:0/4039951330 >> v1:172.21.15.39:6789/0 conn(0x7f57641599e0 legacy=0x7f57640c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f5769a29640 1 -- 172.21.15.32:0/4039951330 >> v1:172.21.15.32:6789/0 conn(0x7f57640aece0 legacy=0x7f57640b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f575bfff640 1 -- 172.21.15.32:0/4039951330 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 889986526 0 0) 0x7f57640711d0 con 0x7f57640aece0 2023-12-20T04:29:19.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f5769a29640 1 -- 172.21.15.32:0/4039951330 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f57640c9e80 con 0x7f57640b0d00 2023-12-20T04:29:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.931+0000 7f5769a29640 1 --2- 172.21.15.32:0/4039951330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57640b0d00 0x7f5764164a70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f575c009d90 tx=0x7f575c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c281e0daafb3f2d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f575bfff640 1 -- 172.21.15.32:0/4039951330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0a5900 con 0x7f57640b0d00 2023-12-20T04:29:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f575bfff640 1 -- 172.21.15.32:0/4039951330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c0a5a60 con 0x7f57640b0d00 2023-12-20T04:29:19.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f575bfff640 1 -- 172.21.15.32:0/4039951330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0a5df0 con 0x7f57640b0d00 2023-12-20T04:29:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f576b48c640 1 -- 172.21.15.32:0/4039951330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57640b0d00 msgr2=0x7f5764164a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f576b48c640 1 --2- 172.21.15.32:0/4039951330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57640b0d00 0x7f5764164a70 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f575c009d90 tx=0x7f575c01c950 comp rx=0 tx=0).stop 2023-12-20T04:29:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f576b48c640 1 -- 172.21.15.32:0/4039951330 shutdown_connections 2023-12-20T04:29:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f576b48c640 1 --2- 172.21.15.32:0/4039951330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57640b0d00 0x7f5764164a70 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f576b48c640 1 -- 172.21.15.32:0/4039951330 >> 172.21.15.32:0/4039951330 conn(0x7f576406a320 msgr2=0x7f57640ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:19.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f576b48c640 1 -- 172.21.15.32:0/4039951330 shutdown_connections 2023-12-20T04:29:19.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f576b48c640 1 -- 172.21.15.32:0/4039951330 wait complete. 2023-12-20T04:29:19.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f576b48c640 1 Processor -- start 2023-12-20T04:29:19.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f576b48c640 1 -- start start 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.935+0000 7f576b48c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57640aece0 0x7f576414f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f576b48c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f57640b0d00 0x7f5764161a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f576b48c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57641599e0 0x7f57641633f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f576b48c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f576414f610 con 0x7f57641599e0 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f576b48c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5764161f80 con 0x7f57640aece0 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f576b48c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5764153300 con 0x7f57640b0d00 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5769a29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57641599e0 0x7f57641633f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5769a29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57641599e0 0x7f57641633f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37934/0 (socket says 172.21.15.32:37934) 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5769228640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57640aece0 0x7f576414f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5769a29640 1 -- 172.21.15.32:0/2865777670 learned_addr learned my addr 172.21.15.32:0/2865777670 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5769a29640 1 -- 172.21.15.32:0/2865777670 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f57640b0d00 msgr2=0x7f5764161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5769a29640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f57640b0d00 0x7f5764161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5769a29640 1 -- 172.21.15.32:0/2865777670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57640aece0 msgr2=0x7f576414f0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5769a29640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57640aece0 0x7f576414f0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5769a29640 1 -- 172.21.15.32:0/2865777670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f575c009a70 con 0x7f57641599e0 2023-12-20T04:29:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5769a29640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57641599e0 0x7f57641633f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f575c01a580 tx=0x7f575c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5759ffb640 1 -- 172.21.15.32:0/2865777670 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0ad950 con 0x7f57641599e0 2023-12-20T04:29:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5759ffb640 1 -- 172.21.15.32:0/2865777670 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f575c0adab0 con 0x7f57641599e0 2023-12-20T04:29:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f5759ffb640 1 -- 172.21.15.32:0/2865777670 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f575c0ade20 con 0x7f57641599e0 2023-12-20T04:29:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f576b48c640 1 -- 172.21.15.32:0/2865777670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5764153020 con 0x7f57641599e0 2023-12-20T04:29:19.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f576b48c640 1 -- 172.21.15.32:0/2865777670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5764154e80 con 0x7f57641599e0 2023-12-20T04:29:19.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.939+0000 7f576b48c640 1 -- 172.21.15.32:0/2865777670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f576404e000 con 0x7f57641599e0 2023-12-20T04:29:19.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.943+0000 7f5759ffb640 1 -- 172.21.15.32:0/2865777670 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f575c0c4460 con 0x7f57641599e0 2023-12-20T04:29:19.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.943+0000 7f5759ffb640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5748065a60 0x7f5748067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:19.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.943+0000 7f5759ffb640 1 -- 172.21.15.32:0/2865777670 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(23..23 src has 1..23) v4 ==== 5825+0+0 (secure 0 0 0) 0x7f575c11e790 con 0x7f57641599e0 2023-12-20T04:29:19.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.943+0000 7f5769228640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5748065a60 0x7f5748067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:19.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.943+0000 7f5769228640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5748065a60 0x7f5748067f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5760000fc0 tx=0x7f5760009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:19.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:19.947+0000 7f5759ffb640 1 -- 172.21.15.32:0/2865777670 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f575c0ec150 con 0x7f57641599e0 2023-12-20T04:29:20.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.143+0000 7f576b48c640 1 -- 172.21.15.32:0/2865777670 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f57640cf550 con 0x7f57641599e0 2023-12-20T04:29:20.148 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T04:29:20.147+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T04:29:20.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.159+0000 7f5759ffb640 1 -- 172.21.15.32:0/2865777670 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v9) v1 ==== 161+0+0 (secure 0 0 0) 0x7f575c0ef770 con 0x7f57641599e0 2023-12-20T04:29:20.163 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T04:29:20.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 -- 172.21.15.32:0/2865777670 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5748065a60 msgr2=0x7f5748067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:20.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5748065a60 0x7f5748067f20 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f5760000fc0 tx=0x7f5760009040 comp rx=0 tx=0).stop 2023-12-20T04:29:20.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 -- 172.21.15.32:0/2865777670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57641599e0 msgr2=0x7f57641633f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57641599e0 0x7f57641633f0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f575c01a580 tx=0x7f575c0196f0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 -- 172.21.15.32:0/2865777670 shutdown_connections 2023-12-20T04:29:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5748065a60 0x7f5748067f20 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f57641599e0 0x7f57641633f0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f57640b0d00 0x7f5764161a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 --2- 172.21.15.32:0/2865777670 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f57640aece0 0x7f576414f0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 -- 172.21.15.32:0/2865777670 >> 172.21.15.32:0/2865777670 conn(0x7f576406a320 msgr2=0x7f5764070850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 -- 172.21.15.32:0/2865777670 shutdown_connections 2023-12-20T04:29:20.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.163+0000 7f576b48c640 1 -- 172.21.15.32:0/2865777670 wait complete. 2023-12-20T04:29:20.177 INFO:tasks.ceph.mds.d.smithi039.stderr: -6644> 2023-12-20T04:28:42.302+0000 7f95e9e52640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:29:20.185 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T04:29:20.230 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T04:29:20.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.327+0000 7f32223bc640 1 Processor -- start 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.327+0000 7f32223bc640 1 -- start start 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f32223bc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f321c15abb0 0x7f321c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f32223bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c0b0cb0 0x7f321c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f32223bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f321c070230 0x7f321c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f32223bc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321c06e160 con 0x7f321c15abb0 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f32223bc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f321c070be0 con 0x7f321c070230 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f32223bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c15b6a0 con 0x7f321c0b0cb0 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f321bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f321c15abb0 0x7f321c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f3220959640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f321c070230 0x7f321c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f321b7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c0b0cb0 0x7f321c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f321bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f321c15abb0 0x7f321c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37942/0 (socket says 172.21.15.32:37942) 2023-12-20T04:29:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f321bfff640 1 -- 172.21.15.32:0/3549862879 learned_addr learned my addr 172.21.15.32:0/3549862879 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:20.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f321b7fe640 1 -- 172.21.15.32:0/3549862879 >> v2:172.21.15.39:3300/0 conn(0x7f321c070230 msgr2=0x7f321c0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:20.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f321b7fe640 1 --2- 172.21.15.32:0/3549862879 >> v2:172.21.15.39:3300/0 conn(0x7f321c070230 0x7f321c0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f321b7fe640 1 -- 172.21.15.32:0/3549862879 >> v2:172.21.15.32:3300/0 conn(0x7f321c15abb0 msgr2=0x7f321c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:20.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f321b7fe640 1 --2- 172.21.15.32:0/3549862879 >> v2:172.21.15.32:3300/0 conn(0x7f321c15abb0 0x7f321c0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f321b7fe640 1 -- 172.21.15.32:0/3549862879 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f321c15c070 con 0x7f321c0b0cb0 2023-12-20T04:29:20.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.331+0000 7f321bfff640 1 --2- 172.21.15.32:0/3549862879 >> v2:172.21.15.32:3300/0 conn(0x7f321c15abb0 0x7f321c0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:20.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f321b7fe640 1 --2- 172.21.15.32:0/3549862879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c0b0cb0 0x7f321c15bb30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f321c0b4470 tx=0x7f321001c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec6e88ed6808e1cf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:20.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f321affd640 1 -- 172.21.15.32:0/3549862879 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32100ab620 con 0x7f321c0b0cb0 2023-12-20T04:29:20.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f321affd640 1 -- 172.21.15.32:0/3549862879 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32100ab780 con 0x7f321c0b0cb0 2023-12-20T04:29:20.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f321affd640 1 -- 172.21.15.32:0/3549862879 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32100abb10 con 0x7f321c0b0cb0 2023-12-20T04:29:20.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f32223bc640 1 -- 172.21.15.32:0/3549862879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c0b0cb0 msgr2=0x7f321c15bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:20.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f32223bc640 1 --2- 172.21.15.32:0/3549862879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c0b0cb0 0x7f321c15bb30 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f321c0b4470 tx=0x7f321001c460 comp rx=0 tx=0).stop 2023-12-20T04:29:20.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f32223bc640 1 -- 172.21.15.32:0/3549862879 shutdown_connections 2023-12-20T04:29:20.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f32223bc640 1 --2- 172.21.15.32:0/3549862879 >> v2:172.21.15.39:3300/0 conn(0x7f321c070230 0x7f321c0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f32223bc640 1 --2- 172.21.15.32:0/3549862879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c0b0cb0 0x7f321c15bb30 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f32223bc640 1 --2- 172.21.15.32:0/3549862879 >> v2:172.21.15.32:3300/0 conn(0x7f321c15abb0 0x7f321c0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f32223bc640 1 -- 172.21.15.32:0/3549862879 >> 172.21.15.32:0/3549862879 conn(0x7f321c06a510 msgr2=0x7f321c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f32223bc640 1 -- 172.21.15.32:0/3549862879 shutdown_connections 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f32223bc640 1 -- 172.21.15.32:0/3549862879 wait complete. 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f32223bc640 1 Processor -- start 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.335+0000 7f32223bc640 1 -- start start 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f32223bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f321c070230 0x7f321c1fd3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f32223bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f321c0b0cb0 0x7f321c0cd660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f32223bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c15abb0 0x7f321c145c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f32223bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c1531a0 con 0x7f321c070230 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f32223bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f321c1fd930 con 0x7f321c0b0cb0 2023-12-20T04:29:20.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f32223bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f321c152590 con 0x7f321c15abb0 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f321bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f321c070230 0x7f321c1fd3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f321bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f321c070230 0x7f321c1fd3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37952/0 (socket says 172.21.15.32:37952) 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f321bfff640 1 -- 172.21.15.32:0/741522021 learned_addr learned my addr 172.21.15.32:0/741522021 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f321b7fe640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f321c0b0cb0 0x7f321c0cd660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f3220959640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c15abb0 0x7f321c145c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f321bfff640 1 -- 172.21.15.32:0/741522021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c15abb0 msgr2=0x7f321c145c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f321bfff640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c15abb0 0x7f321c145c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f321bfff640 1 -- 172.21.15.32:0/741522021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f321c0b0cb0 msgr2=0x7f321c0cd660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f321bfff640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f321c0b0cb0 0x7f321c0cd660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f321bfff640 1 -- 172.21.15.32:0/741522021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f320c0094d0 con 0x7f321c070230 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f321b7fe640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f321c0b0cb0 0x7f321c0cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f3220959640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c15abb0 0x7f321c145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f321bfff640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f321c070230 0x7f321c1fd3f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f320c00e9d0 tx=0x7f320c00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f3218ff9640 1 -- 172.21.15.32:0/741522021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c00ca60 con 0x7f321c070230 2023-12-20T04:29:20.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f3218ff9640 1 -- 172.21.15.32:0/741522021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f320c00cbc0 con 0x7f321c070230 2023-12-20T04:29:20.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f32223bc640 1 -- 172.21.15.32:0/741522021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3210009420 con 0x7f321c070230 2023-12-20T04:29:20.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f3218ff9640 1 -- 172.21.15.32:0/741522021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f320c013470 con 0x7f321c070230 2023-12-20T04:29:20.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f32223bc640 1 -- 172.21.15.32:0/741522021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f321c152b50 con 0x7f321c070230 2023-12-20T04:29:20.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f32223bc640 1 -- 172.21.15.32:0/741522021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f321c04e000 con 0x7f321c070230 2023-12-20T04:29:20.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.339+0000 7f3218ff9640 1 -- 172.21.15.32:0/741522021 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f320c0135d0 con 0x7f321c070230 2023-12-20T04:29:20.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.343+0000 7f3218ff9640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3208065a90 0x7f3208067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:20.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.343+0000 7f3218ff9640 1 -- 172.21.15.32:0/741522021 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f320c083560 con 0x7f321c070230 2023-12-20T04:29:20.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.343+0000 7f321b7fe640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3208065a90 0x7f3208067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:20.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.343+0000 7f321b7fe640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3208065a90 0x7f3208067f50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f32100af000 tx=0x7f32100a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:20.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.347+0000 7f3218ff9640 1 -- 172.21.15.32:0/741522021 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f320c050e40 con 0x7f321c070230 2023-12-20T04:29:20.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:20.539+0000 7f32223bc640 1 -- 172.21.15.32:0/741522021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f321c155230 con 0x7f321c070230 2023-12-20T04:29:21.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.163+0000 7f3218ff9640 1 -- 172.21.15.32:0/741522021 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v10) v1 ==== 104+0+0 (secure 0 0 0) 0x7f320c054460 con 0x7f321c070230 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 -- 172.21.15.32:0/741522021 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3208065a90 msgr2=0x7f3208067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3208065a90 0x7f3208067f50 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f32100af000 tx=0x7f32100a7000 comp rx=0 tx=0).stop 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 -- 172.21.15.32:0/741522021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f321c070230 msgr2=0x7f321c1fd3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f321c070230 0x7f321c1fd3f0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f320c00e9d0 tx=0x7f320c00ee90 comp rx=0 tx=0).stop 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 -- 172.21.15.32:0/741522021 shutdown_connections 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3208065a90 0x7f3208067f50 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f321c15abb0 0x7f321c145c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f321c0b0cb0 0x7f321c0cd660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 --2- 172.21.15.32:0/741522021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f321c070230 0x7f321c1fd3f0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 -- 172.21.15.32:0/741522021 >> 172.21.15.32:0/741522021 conn(0x7f321c06a510 msgr2=0x7f321c0c4b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 -- 172.21.15.32:0/741522021 shutdown_connections 2023-12-20T04:29:21.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.167+0000 7f32223bc640 1 -- 172.21.15.32:0/741522021 wait complete. 2023-12-20T04:29:21.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T04:29:21.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.339+0000 7f6c2be8c640 1 Processor -- start 2023-12-20T04:29:21.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.339+0000 7f6c2be8c640 1 -- start start 2023-12-20T04:29:21.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.339+0000 7f6c2be8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c24159250 0x7f6c240caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:21.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.339+0000 7f6c2be8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c24070940 0x7f6c240cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:21.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.339+0000 7f6c2be8c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c24159630 con 0x7f6c240ce290 2023-12-20T04:29:21.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.339+0000 7f6c2be8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c240c9480 con 0x7f6c24070940 2023-12-20T04:29:21.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.339+0000 7f6c2be8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2406cf90 con 0x7f6c24159250 2023-12-20T04:29:21.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c2a429640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c24070940 0x7f6c240cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:21.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c2a429640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c24070940 0x7f6c240cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37954/0 (socket says 172.21.15.32:37954) 2023-12-20T04:29:21.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c2a429640 1 -- 172.21.15.32:0/2393157758 learned_addr learned my addr 172.21.15.32:0/2393157758 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:21.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c29c28640 1 --2- 172.21.15.32:0/2393157758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c24159250 0x7f6c240caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:21.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c2a429640 1 -- 172.21.15.32:0/2393157758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c24159250 msgr2=0x7f6c240caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:21.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c2a429640 1 --2- 172.21.15.32:0/2393157758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c24159250 0x7f6c240caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:21.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c2a429640 1 -- 172.21.15.32:0/2393157758 >> v1:172.21.15.39:6789/0 conn(0x7f6c240ce290 legacy=0x7f6c24070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:21.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c2a429640 1 -- 172.21.15.32:0/2393157758 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2406d270 con 0x7f6c24070940 2023-12-20T04:29:21.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c28c26640 1 -- 172.21.15.32:0/2393157758 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3165111766 0 0) 0x7f6c24159630 con 0x7f6c240ce290 2023-12-20T04:29:21.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c2a429640 1 --2- 172.21.15.32:0/2393157758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c24070940 0x7f6c240cc5b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009e50 tx=0x7f6c1c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f029385483c568d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:21.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c28c26640 1 -- 172.21.15.32:0/2393157758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0a5820 con 0x7f6c24070940 2023-12-20T04:29:21.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c28c26640 1 -- 172.21.15.32:0/2393157758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c0a5980 con 0x7f6c24070940 2023-12-20T04:29:21.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.343+0000 7f6c28c26640 1 -- 172.21.15.32:0/2393157758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0a5d10 con 0x7f6c24070940 2023-12-20T04:29:21.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2393157758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c24070940 msgr2=0x7f6c240cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:21.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 --2- 172.21.15.32:0/2393157758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c24070940 0x7f6c240cc5b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c009e50 tx=0x7f6c1c01c9e0 comp rx=0 tx=0).stop 2023-12-20T04:29:21.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2393157758 shutdown_connections 2023-12-20T04:29:21.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 --2- 172.21.15.32:0/2393157758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c24070940 0x7f6c240cc5b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:21.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 --2- 172.21.15.32:0/2393157758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c24159250 0x7f6c240caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:21.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2393157758 >> 172.21.15.32:0/2393157758 conn(0x7f6c2406a510 msgr2=0x7f6c240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:21.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2393157758 shutdown_connections 2023-12-20T04:29:21.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2393157758 wait complete. 2023-12-20T04:29:21.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 Processor -- start 2023-12-20T04:29:21.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 -- start start 2023-12-20T04:29:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c24070940 0x7f6c2414c5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c240ce290 0x7f6c240c3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c24159250 0x7f6c240c4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2414caf0 con 0x7f6c24159250 2023-12-20T04:29:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2415a760 con 0x7f6c24070940 2023-12-20T04:29:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2be8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c2415aa40 con 0x7f6c240ce290 2023-12-20T04:29:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2a429640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c24159250 0x7f6c240c4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2a429640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c24159250 0x7f6c240c4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37960/0 (socket says 172.21.15.32:37960) 2023-12-20T04:29:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2a429640 1 -- 172.21.15.32:0/2713348651 learned_addr learned my addr 172.21.15.32:0/2713348651 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c29c28640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c24070940 0x7f6c2414c5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:21.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c29427640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c240ce290 0x7f6c240c3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2a429640 1 -- 172.21.15.32:0/2713348651 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c240ce290 msgr2=0x7f6c240c3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2a429640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c240ce290 0x7f6c240c3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2a429640 1 -- 172.21.15.32:0/2713348651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c24070940 msgr2=0x7f6c2414c5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.347+0000 7f6c2a429640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c24070940 0x7f6c2414c5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c2a429640 1 -- 172.21.15.32:0/2713348651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c1c009b30 con 0x7f6c24159250 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c29c28640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c24070940 0x7f6c2414c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c29427640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c240ce290 0x7f6c240c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c2a429640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c24159250 0x7f6c240c4440 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0af040 tx=0x7f6c1c019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c1a7fc640 1 -- 172.21.15.32:0/2713348651 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0b8670 con 0x7f6c24159250 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c1a7fc640 1 -- 172.21.15.32:0/2713348651 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c1c0b87d0 con 0x7f6c24159250 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c1a7fc640 1 -- 172.21.15.32:0/2713348651 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c1c0b8b80 con 0x7f6c24159250 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2713348651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c24156970 con 0x7f6c24159250 2023-12-20T04:29:21.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2713348651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c2415ad20 con 0x7f6c24159250 2023-12-20T04:29:21.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2713348651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c2404e000 con 0x7f6c24159250 2023-12-20T04:29:21.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c1a7fc640 1 -- 172.21.15.32:0/2713348651 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6c1c0b8ce0 con 0x7f6c24159250 2023-12-20T04:29:21.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c1a7fc640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c08065a60 0x7f6c08067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:21.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c1a7fc640 1 -- 172.21.15.32:0/2713348651 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(24..24 src has 1..24) v4 ==== 5911+0+0 (secure 0 0 0) 0x7f6c1c11dd50 con 0x7f6c24159250 2023-12-20T04:29:21.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.351+0000 7f6c29c28640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c08065a60 0x7f6c08067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:21.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.355+0000 7f6c29c28640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c08065a60 0x7f6c08067f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009a40 tx=0x7f6c20008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:21.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.355+0000 7f6c1a7fc640 1 -- 172.21.15.32:0/2713348651 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c1c0eb5a0 con 0x7f6c24159250 2023-12-20T04:29:21.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:21.514+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2713348651 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6c2415ab80 con 0x7f6c24159250 2023-12-20T04:29:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.166+0000 7f6c1a7fc640 1 -- 172.21.15.32:0/2713348651 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v25) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6c2415ab80 con 0x7f6c24159250 2023-12-20T04:29:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T04:29:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.170+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2713348651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c08065a60 msgr2=0x7f6c08067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.170+0000 7f6c2be8c640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c08065a60 0x7f6c08067f20 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f6c20009a40 tx=0x7f6c20008040 comp rx=0 tx=0).stop 2023-12-20T04:29:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.170+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2713348651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c24159250 msgr2=0x7f6c240c4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:22.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.170+0000 7f6c2be8c640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c24159250 0x7f6c240c4440 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f6c1c0af040 tx=0x7f6c1c019780 comp rx=0 tx=0).stop 2023-12-20T04:29:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.174+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2713348651 shutdown_connections 2023-12-20T04:29:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.174+0000 7f6c2be8c640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c08065a60 0x7f6c08067f20 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.174+0000 7f6c2be8c640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c24159250 0x7f6c240c4440 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.174+0000 7f6c2be8c640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c240ce290 0x7f6c240c3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.174+0000 7f6c2be8c640 1 --2- 172.21.15.32:0/2713348651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c24070940 0x7f6c2414c5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.174+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2713348651 >> 172.21.15.32:0/2713348651 conn(0x7f6c2406a510 msgr2=0x7f6c240c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.174+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2713348651 shutdown_connections 2023-12-20T04:29:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.174+0000 7f6c2be8c640 1 -- 172.21.15.32:0/2713348651 wait complete. 2023-12-20T04:29:22.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T04:29:22.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.354+0000 7f63c5441640 1 Processor -- start 2023-12-20T04:29:22.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.354+0000 7f63c5441640 1 -- start start 2023-12-20T04:29:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63c5441640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63c0070940 0x7f63c00ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63c5441640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c006e100 con 0x7f63c006dcb0 2023-12-20T04:29:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63c5441640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63c00ca590 con 0x7f63c0158520 2023-12-20T04:29:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63c5441640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00ca7f0 con 0x7f63c0070940 2023-12-20T04:29:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63b7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f63c006dcb0 0x7f63c0070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52608/0 (socket says 172.21.15.32:52608) 2023-12-20T04:29:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63b7fff640 1 -- 172.21.15.32:0/230635354 learned_addr learned my addr 172.21.15.32:0/230635354 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63bf7fe640 1 --2- 172.21.15.32:0/230635354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63c0070940 0x7f63c00ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63be7fc640 1 -- 172.21.15.32:0/230635354 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 546766366 0 0) 0x7f63a8001120 con 0x7f63c006dcb0 2023-12-20T04:29:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63be7fc640 1 -- 172.21.15.32:0/230635354 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63ac003530 con 0x7f63c006dcb0 2023-12-20T04:29:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63be7fc640 1 -- 172.21.15.32:0/230635354 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1991418710 0 0) 0x7f63c00ca590 con 0x7f63c0158520 2023-12-20T04:29:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63be7fc640 1 -- 172.21.15.32:0/230635354 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f63a8001120 con 0x7f63c0158520 2023-12-20T04:29:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63bf7fe640 1 -- 172.21.15.32:0/230635354 >> v1:172.21.15.39:6789/0 conn(0x7f63c0158520 legacy=0x7f63c015ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63bf7fe640 1 -- 172.21.15.32:0/230635354 >> v1:172.21.15.32:6789/0 conn(0x7f63c006dcb0 legacy=0x7f63c0070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:22.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.358+0000 7f63bf7fe640 1 -- 172.21.15.32:0/230635354 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63c00caad0 con 0x7f63c0070940 2023-12-20T04:29:22.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63bf7fe640 1 --2- 172.21.15.32:0/230635354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63c0070940 0x7f63c00ca050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009dc0 tx=0x7f63b801c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d155ed237b32dc6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:22.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63be7fc640 1 -- 172.21.15.32:0/230635354 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b80a5970 con 0x7f63c0070940 2023-12-20T04:29:22.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63be7fc640 1 -- 172.21.15.32:0/230635354 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b80a5ad0 con 0x7f63c0070940 2023-12-20T04:29:22.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63be7fc640 1 -- 172.21.15.32:0/230635354 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b80a5e60 con 0x7f63c0070940 2023-12-20T04:29:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63c5441640 1 -- 172.21.15.32:0/230635354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63c0070940 msgr2=0x7f63c00ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63c5441640 1 --2- 172.21.15.32:0/230635354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63c0070940 0x7f63c00ca050 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f63b8009dc0 tx=0x7f63b801c8b0 comp rx=0 tx=0).stop 2023-12-20T04:29:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63c5441640 1 -- 172.21.15.32:0/230635354 shutdown_connections 2023-12-20T04:29:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63c5441640 1 --2- 172.21.15.32:0/230635354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63c0070940 0x7f63c00ca050 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63c5441640 1 -- 172.21.15.32:0/230635354 >> 172.21.15.32:0/230635354 conn(0x7f63c006a510 msgr2=0x7f63c00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63c5441640 1 -- 172.21.15.32:0/230635354 shutdown_connections 2023-12-20T04:29:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63c5441640 1 -- 172.21.15.32:0/230635354 wait complete. 2023-12-20T04:29:22.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63c5441640 1 Processor -- start 2023-12-20T04:29:22.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63c5441640 1 -- start start 2023-12-20T04:29:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63c5441640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c006dcb0 0x7f63c0165c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.362+0000 7f63c5441640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63c0070940 0x7f63c014ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63c5441640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63c0158520 0x7f63c0150130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63c5441640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c0154720 con 0x7f63c006dcb0 2023-12-20T04:29:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63c5441640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00c4690 con 0x7f63c0158520 2023-12-20T04:29:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63c5441640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63c00c4970 con 0x7f63c0070940 2023-12-20T04:29:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63beffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c006dcb0 0x7f63c0165c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:22.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63beffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c006dcb0 0x7f63c0165c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37976/0 (socket says 172.21.15.32:37976) 2023-12-20T04:29:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63beffd640 1 -- 172.21.15.32:0/3033712202 learned_addr learned my addr 172.21.15.32:0/3033712202 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63b7fff640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63c0070940 0x7f63c014ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63bf7fe640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63c0158520 0x7f63c0150130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63beffd640 1 -- 172.21.15.32:0/3033712202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63c0070940 msgr2=0x7f63c014ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63beffd640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63c0070940 0x7f63c014ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63beffd640 1 -- 172.21.15.32:0/3033712202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63c0158520 msgr2=0x7f63c0150130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63beffd640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63c0158520 0x7f63c0150130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:22.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63beffd640 1 -- 172.21.15.32:0/3033712202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63b8009a70 con 0x7f63c006dcb0 2023-12-20T04:29:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63beffd640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c006dcb0 0x7f63c0165c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f63b000c430 tx=0x7f63b000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:22.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63b77fe640 1 -- 172.21.15.32:0/3033712202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b00052d0 con 0x7f63c006dcb0 2023-12-20T04:29:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63b77fe640 1 -- 172.21.15.32:0/3033712202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63b0005450 con 0x7f63c006dcb0 2023-12-20T04:29:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63b77fe640 1 -- 172.21.15.32:0/3033712202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63b0003840 con 0x7f63c006dcb0 2023-12-20T04:29:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63c5441640 1 -- 172.21.15.32:0/3033712202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63c014f930 con 0x7f63c006dcb0 2023-12-20T04:29:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63c5441640 1 -- 172.21.15.32:0/3033712202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63c01661c0 con 0x7f63c006dcb0 2023-12-20T04:29:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63c5441640 1 -- 172.21.15.32:0/3033712202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63c004e000 con 0x7f63c006dcb0 2023-12-20T04:29:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.366+0000 7f63b77fe640 1 -- 172.21.15.32:0/3033712202 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f63b00039a0 con 0x7f63c006dcb0 2023-12-20T04:29:22.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.370+0000 7f63b77fe640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63a0065a60 0x7f63a0067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:22.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.370+0000 7f63b77fe640 1 -- 172.21.15.32:0/3033712202 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(25..25 src has 1..25) v4 ==== 5473+0+0 (secure 0 0 0) 0x7f63b00827d0 con 0x7f63c006dcb0 2023-12-20T04:29:22.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.370+0000 7f63b7fff640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63a0065a60 0x7f63a0067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:22.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.370+0000 7f63b7fff640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63a0065a60 0x7f63a0067f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f63c015e4b0 tx=0x7f63a8009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:22.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.374+0000 7f63b77fe640 1 -- 172.21.15.32:0/3033712202 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f63b00501d0 con 0x7f63c006dcb0 2023-12-20T04:29:22.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:22.530+0000 7f63c5441640 1 -- 172.21.15.32:0/3033712202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f63c014fde0 con 0x7f63c006dcb0 2023-12-20T04:29:23.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.174+0000 7f63b77fe640 1 -- 172.21.15.32:0/3033712202 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v26) v1 ==== 169+0+0 (secure 0 0 0) 0x7f63b00537f0 con 0x7f63c006dcb0 2023-12-20T04:29:23.180 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T04:29:23.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.178+0000 7f63c5441640 1 -- 172.21.15.32:0/3033712202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63a0065a60 msgr2=0x7f63a0067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:23.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.178+0000 7f63c5441640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63a0065a60 0x7f63a0067f20 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f63c015e4b0 tx=0x7f63a8009040 comp rx=0 tx=0).stop 2023-12-20T04:29:23.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.178+0000 7f63c5441640 1 -- 172.21.15.32:0/3033712202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c006dcb0 msgr2=0x7f63c0165c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:23.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.178+0000 7f63c5441640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c006dcb0 0x7f63c0165c80 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f63b000c430 tx=0x7f63b000c8f0 comp rx=0 tx=0).stop 2023-12-20T04:29:23.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.178+0000 7f63c5441640 1 -- 172.21.15.32:0/3033712202 shutdown_connections 2023-12-20T04:29:23.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.178+0000 7f63c5441640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63a0065a60 0x7f63a0067f20 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:23.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.178+0000 7f63c5441640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63c0158520 0x7f63c0150130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:23.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.178+0000 7f63c5441640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63c0070940 0x7f63c014ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:23.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.178+0000 7f63c5441640 1 --2- 172.21.15.32:0/3033712202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63c006dcb0 0x7f63c0165c80 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:23.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.178+0000 7f63c5441640 1 -- 172.21.15.32:0/3033712202 >> 172.21.15.32:0/3033712202 conn(0x7f63c006a510 msgr2=0x7f63c00c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:23.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.182+0000 7f63c5441640 1 -- 172.21.15.32:0/3033712202 shutdown_connections 2023-12-20T04:29:23.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.182+0000 7f63c5441640 1 -- 172.21.15.32:0/3033712202 wait complete. 2023-12-20T04:29:23.214 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T04:29:23.214 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T04:29:23.214 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T04:29:23.214 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:29:23.215 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T04:29:23.215 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T04:29:23.215 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T04:29:23.215 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:29:23.215 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T04:29:23.215 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T04:29:23.215 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T04:29:23.215 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:29:23.216 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T04:29:23.216 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T04:29:23.216 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T04:29:23.216 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:29:23.233 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T04:29:23.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T04:29:23.234 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T04:29:23.235 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T04:29:23.237 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T04:29:23.237 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T04:29:23.239 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T04:29:23.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T04:29:23.283 INFO:tasks.ceph.mds.a:Started 2023-12-20T04:29:23.283 DEBUG:teuthology.parallel:result is None 2023-12-20T04:29:23.285 INFO:tasks.ceph.mds.c:Started 2023-12-20T04:29:23.285 DEBUG:teuthology.parallel:result is None 2023-12-20T04:29:23.288 INFO:tasks.ceph.mds.b:Started 2023-12-20T04:29:23.289 DEBUG:teuthology.parallel:result is None 2023-12-20T04:29:23.291 INFO:tasks.ceph.mds.d:Started 2023-12-20T04:29:23.291 DEBUG:teuthology.parallel:result is None 2023-12-20T04:29:23.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T04:29:23.374 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T04:29:23.374 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T04:29:23.377 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T04:29:23.378 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T04:29:23.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.462+0000 7fca295df640 1 Processor -- start 2023-12-20T04:29:23.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.462+0000 7fca295df640 1 -- start start 2023-12-20T04:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.462+0000 7fca295df640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca24161500 0x7fca24153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.462+0000 7fca295df640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca240cd4d0 con 0x7fca2406b630 2023-12-20T04:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.462+0000 7fca295df640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca240ce4e0 con 0x7fca240b8480 2023-12-20T04:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.462+0000 7fca295df640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca240cd720 con 0x7fca24161500 2023-12-20T04:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.462+0000 7fca237fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fca2406b630 0x7fca24160df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52622/0 (socket says 172.21.15.32:52622) 2023-12-20T04:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.462+0000 7fca237fe640 1 -- 172.21.15.32:0/3538729126 learned_addr learned my addr 172.21.15.32:0/3538729126 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:23.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.462+0000 7fca28dde640 1 --2- 172.21.15.32:0/3538729126 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca24161500 0x7fca24153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.466+0000 7fca22ffd640 1 -- 172.21.15.32:0/3538729126 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350100605 0 0) 0x7fca240cd4d0 con 0x7fca2406b630 2023-12-20T04:29:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.466+0000 7fca22ffd640 1 -- 172.21.15.32:0/3538729126 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca0c003530 con 0x7fca2406b630 2023-12-20T04:29:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.466+0000 7fca22ffd640 1 -- 172.21.15.32:0/3538729126 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1001197930 0 0) 0x7fca240ce4e0 con 0x7fca240b8480 2023-12-20T04:29:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.466+0000 7fca28dde640 1 -- 172.21.15.32:0/3538729126 >> v1:172.21.15.39:6789/0 conn(0x7fca240b8480 legacy=0x7fca2406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.466+0000 7fca28dde640 1 -- 172.21.15.32:0/3538729126 >> v1:172.21.15.32:6789/0 conn(0x7fca2406b630 legacy=0x7fca24160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:23.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.466+0000 7fca28dde640 1 -- 172.21.15.32:0/3538729126 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca240cda00 con 0x7fca24161500 2023-12-20T04:29:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.466+0000 7fca28dde640 1 --2- 172.21.15.32:0/3538729126 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca24161500 0x7fca24153610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009dc0 tx=0x7fca1c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3c072cae05f0b52b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.466+0000 7fca22ffd640 1 -- 172.21.15.32:0/3538729126 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0a5810 con 0x7fca24161500 2023-12-20T04:29:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.466+0000 7fca22ffd640 1 -- 172.21.15.32:0/3538729126 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca1c0a5970 con 0x7fca24161500 2023-12-20T04:29:23.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.466+0000 7fca22ffd640 1 -- 172.21.15.32:0/3538729126 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca1c0a5d00 con 0x7fca24161500 2023-12-20T04:29:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 -- 172.21.15.32:0/3538729126 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca24161500 msgr2=0x7fca24153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 --2- 172.21.15.32:0/3538729126 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca24161500 0x7fca24153610 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fca1c009dc0 tx=0x7fca1c01c970 comp rx=0 tx=0).stop 2023-12-20T04:29:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 -- 172.21.15.32:0/3538729126 shutdown_connections 2023-12-20T04:29:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 --2- 172.21.15.32:0/3538729126 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca24161500 0x7fca24153610 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 -- 172.21.15.32:0/3538729126 >> 172.21.15.32:0/3538729126 conn(0x7fca240d2a50 msgr2=0x7fca240d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 -- 172.21.15.32:0/3538729126 shutdown_connections 2023-12-20T04:29:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 -- 172.21.15.32:0/3538729126 wait complete. 2023-12-20T04:29:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 Processor -- start 2023-12-20T04:29:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 -- start start 2023-12-20T04:29:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca2406b630 0x7fca240c8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca240b8480 0x7fca2415e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca24161500 0x7fca2415eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca240c6dc0 con 0x7fca240b8480 2023-12-20T04:29:23.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca240c64f0 con 0x7fca24161500 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca295df640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca240bbcf0 con 0x7fca2406b630 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca240b8480 0x7fca2415e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca240b8480 0x7fca2415e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38002/0 (socket says 172.21.15.32:38002) 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca237fe640 1 -- 172.21.15.32:0/788847543 learned_addr learned my addr 172.21.15.32:0/788847543 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca28dde640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca24161500 0x7fca2415eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca23fff640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca2406b630 0x7fca240c8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca237fe640 1 -- 172.21.15.32:0/788847543 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca2406b630 msgr2=0x7fca240c8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca237fe640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca2406b630 0x7fca240c8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca237fe640 1 -- 172.21.15.32:0/788847543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca24161500 msgr2=0x7fca2415eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca237fe640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca24161500 0x7fca2415eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.470+0000 7fca237fe640 1 -- 172.21.15.32:0/788847543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca1c009a70 con 0x7fca240b8480 2023-12-20T04:29:23.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.474+0000 7fca237fe640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca240b8480 0x7fca2415e620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fca1800c480 tx=0x7fca1800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.474+0000 7fca20ff9640 1 -- 172.21.15.32:0/788847543 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18005cf0 con 0x7fca240b8480 2023-12-20T04:29:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.474+0000 7fca20ff9640 1 -- 172.21.15.32:0/788847543 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca18005e50 con 0x7fca240b8480 2023-12-20T04:29:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.474+0000 7fca295df640 1 -- 172.21.15.32:0/788847543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca240beb20 con 0x7fca240b8480 2023-12-20T04:29:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.474+0000 7fca20ff9640 1 -- 172.21.15.32:0/788847543 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca18007290 con 0x7fca240b8480 2023-12-20T04:29:23.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.474+0000 7fca295df640 1 -- 172.21.15.32:0/788847543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca240c69e0 con 0x7fca240b8480 2023-12-20T04:29:23.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.474+0000 7fca295df640 1 -- 172.21.15.32:0/788847543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca240c8360 con 0x7fca240b8480 2023-12-20T04:29:23.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.474+0000 7fca20ff9640 1 -- 172.21.15.32:0/788847543 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fca180073f0 con 0x7fca240b8480 2023-12-20T04:29:23.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.474+0000 7fca20ff9640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fca10065a60 0x7fca10067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.474+0000 7fca20ff9640 1 -- 172.21.15.32:0/788847543 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(30..30 src has 1..30) v4 ==== 4943+0+0 (secure 0 0 0) 0x7fca18082430 con 0x7fca240b8480 2023-12-20T04:29:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.474+0000 7fca23fff640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fca10065a60 0x7fca10067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:23.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.478+0000 7fca23fff640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fca10065a60 0x7fca10067f20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca14000f70 tx=0x7fca14009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:23.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.482+0000 7fca20ff9640 1 -- 172.21.15.32:0/788847543 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fca18050190 con 0x7fca240b8480 2023-12-20T04:29:23.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:23.662+0000 7fca295df640 1 -- 172.21.15.32:0/788847543 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fca24074100 con 0x7fca240b8480 2023-12-20T04:29:24.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.442+0000 7fca20ff9640 1 -- 172.21.15.32:0/788847543 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v31) v1 ==== 97+0+0 (secure 0 0 0) 0x7fca18017080 con 0x7fca240b8480 2023-12-20T04:29:24.447 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T04:29:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 -- 172.21.15.32:0/788847543 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fca10065a60 msgr2=0x7fca10067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fca10065a60 0x7fca10067f20 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fca14000f70 tx=0x7fca14009040 comp rx=0 tx=0).stop 2023-12-20T04:29:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 -- 172.21.15.32:0/788847543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca240b8480 msgr2=0x7fca2415e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca240b8480 0x7fca2415e620 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fca1800c480 tx=0x7fca1800c940 comp rx=0 tx=0).stop 2023-12-20T04:29:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 -- 172.21.15.32:0/788847543 shutdown_connections 2023-12-20T04:29:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fca10065a60 0x7fca10067f20 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca24161500 0x7fca2415eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca240b8480 0x7fca2415e620 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 --2- 172.21.15.32:0/788847543 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca2406b630 0x7fca240c8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 -- 172.21.15.32:0/788847543 >> 172.21.15.32:0/788847543 conn(0x7fca240d2a50 msgr2=0x7fca240d2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:24.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 -- 172.21.15.32:0/788847543 shutdown_connections 2023-12-20T04:29:24.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.446+0000 7fca295df640 1 -- 172.21.15.32:0/788847543 wait complete. 2023-12-20T04:29:24.473 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T04:29:24.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.618+0000 7fe2c7ab6640 1 Processor -- start 2023-12-20T04:29:24.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.618+0000 7fe2c7ab6640 1 -- start start 2023-12-20T04:29:24.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.618+0000 7fe2c7ab6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2c00aece0 0x7fe2c00b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.618+0000 7fe2c7ab6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2c00b0b30 0x7fe2c00701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.618+0000 7fe2c7ab6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2c00c98c0 con 0x7fe2c01599e0 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.618+0000 7fe2c7ab6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2c00c9ba0 con 0x7fe2c00aece0 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.618+0000 7fe2c7ab6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00c9e80 con 0x7fe2c00b0b30 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.618+0000 7fe2c5051640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2c00aece0 0x7fe2c00b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.618+0000 7fe2c5051640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2c00aece0 0x7fe2c00b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38004/0 (socket says 172.21.15.32:38004) 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.618+0000 7fe2c5051640 1 -- 172.21.15.32:0/755530086 learned_addr learned my addr 172.21.15.32:0/755530086 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.618+0000 7fe2c6053640 1 --2- 172.21.15.32:0/755530086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2c00b0b30 0x7fe2c00701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.622+0000 7fe2c5051640 1 -- 172.21.15.32:0/755530086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2c00b0b30 msgr2=0x7fe2c00701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.622+0000 7fe2c5051640 1 --2- 172.21.15.32:0/755530086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2c00b0b30 0x7fe2c00701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.622+0000 7fe2c5051640 1 -- 172.21.15.32:0/755530086 >> v1:172.21.15.39:6789/0 conn(0x7fe2c01599e0 legacy=0x7fe2c00c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.622+0000 7fe2c5051640 1 -- 172.21.15.32:0/755530086 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c00ca160 con 0x7fe2c00aece0 2023-12-20T04:29:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.622+0000 7fe2c5051640 1 --2- 172.21.15.32:0/755530086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2c00aece0 0x7fe2c00b05f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009870 tx=0x7fe2b001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=586ba401a40bcfe0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:24.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.622+0000 7fe2c4850640 1 -- 172.21.15.32:0/755530086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00ab650 con 0x7fe2c00aece0 2023-12-20T04:29:24.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.622+0000 7fe2c4850640 1 -- 172.21.15.32:0/755530086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b00ab7b0 con 0x7fe2c00aece0 2023-12-20T04:29:24.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.622+0000 7fe2c4850640 1 -- 172.21.15.32:0/755530086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b00abb40 con 0x7fe2c00aece0 2023-12-20T04:29:24.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.622+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/755530086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2c00aece0 msgr2=0x7fe2c00b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:24.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.622+0000 7fe2c7ab6640 1 --2- 172.21.15.32:0/755530086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2c00aece0 0x7fe2c00b05f0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fe2b0009870 tx=0x7fe2b001c540 comp rx=0 tx=0).stop 2023-12-20T04:29:24.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/755530086 shutdown_connections 2023-12-20T04:29:24.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 --2- 172.21.15.32:0/755530086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2c00b0b30 0x7fe2c00701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 --2- 172.21.15.32:0/755530086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2c00aece0 0x7fe2c00b05f0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/755530086 >> 172.21.15.32:0/755530086 conn(0x7fe2c006a320 msgr2=0x7fe2c00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:24.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/755530086 shutdown_connections 2023-12-20T04:29:24.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/755530086 wait complete. 2023-12-20T04:29:24.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 Processor -- start 2023-12-20T04:29:24.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 -- start start 2023-12-20T04:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2c00aece0 0x7fe2c00ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c00b0b30 0x7fe2c00adbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2c01599e0 0x7fe2c01f4e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00ae340 con 0x7fe2c01599e0 2023-12-20T04:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c00ae0f0 con 0x7fe2c00b0b30 2023-12-20T04:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c7ab6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2c01436f0 con 0x7fe2c00aece0 2023-12-20T04:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c6053640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2c01599e0 0x7fe2c01f4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c6053640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2c01599e0 0x7fe2c01f4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38016/0 (socket says 172.21.15.32:38016) 2023-12-20T04:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c6053640 1 -- 172.21.15.32:0/3181704593 learned_addr learned my addr 172.21.15.32:0/3181704593 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c5852640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2c00aece0 0x7fe2c00ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c5051640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c00b0b30 0x7fe2c00adbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c6053640 1 -- 172.21.15.32:0/3181704593 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2c00aece0 msgr2=0x7fe2c00ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c6053640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2c00aece0 0x7fe2c00ac2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c6053640 1 -- 172.21.15.32:0/3181704593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c00b0b30 msgr2=0x7fe2c00adbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c6053640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c00b0b30 0x7fe2c00adbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.626+0000 7fe2c6053640 1 -- 172.21.15.32:0/3181704593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2b00094d0 con 0x7fe2c01599e0 2023-12-20T04:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.630+0000 7fe2c6053640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2c01599e0 0x7fe2c01f4e40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8007ea0 tx=0x7fe2b8007f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.630+0000 7fe2b67fc640 1 -- 172.21.15.32:0/3181704593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8014620 con 0x7fe2c01599e0 2023-12-20T04:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.630+0000 7fe2b67fc640 1 -- 172.21.15.32:0/3181704593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2b80147a0 con 0x7fe2c01599e0 2023-12-20T04:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.630+0000 7fe2b67fc640 1 -- 172.21.15.32:0/3181704593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2b8014b50 con 0x7fe2c01599e0 2023-12-20T04:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.630+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/3181704593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2c01f9a50 con 0x7fe2c01599e0 2023-12-20T04:29:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.630+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/3181704593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2c01f8f70 con 0x7fe2c01599e0 2023-12-20T04:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.630+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/3181704593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2c004e000 con 0x7fe2c01599e0 2023-12-20T04:29:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.630+0000 7fe2b67fc640 1 -- 172.21.15.32:0/3181704593 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe2b8014cb0 con 0x7fe2c01599e0 2023-12-20T04:29:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.630+0000 7fe2b67fc640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2a4065a60 0x7fe2a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.630+0000 7fe2b67fc640 1 -- 172.21.15.32:0/3181704593 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fe2b8083680 con 0x7fe2c01599e0 2023-12-20T04:29:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.630+0000 7fe2c5852640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2a4065a60 0x7fe2a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.634+0000 7fe2c5852640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2a4065a60 0x7fe2a4067f20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc000fc0 tx=0x7fe2bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:24.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.638+0000 7fe2b67fc640 1 -- 172.21.15.32:0/3181704593 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe2b8051430 con 0x7fe2c01599e0 2023-12-20T04:29:24.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.846+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/3181704593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fe2c00ae8f0 con 0x7fe2c01599e0 2023-12-20T04:29:24.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.846+0000 7fe2b67fc640 1 -- 172.21.15.32:0/3181704593 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v6) v1 ==== 80+0+6645 (secure 0 0 0) 0x7fe2b8054a50 con 0x7fe2c01599e0 2023-12-20T04:29:24.850 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:24.850 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:29:24.850 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T04:29:24.850 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.850 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T04:29:24.850 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T04:29:24.850 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.850 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T04:29:24.852 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T04:29:24.853 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.853 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:29:24.853 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:29:24.853 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:29:24.853 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.853 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.853 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.854 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.855 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T04:29:24.855 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T04:29:24.855 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.855 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:29:24.855 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:29:24.855 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.855 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.855 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.855 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.855 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:29:24.856 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.857 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.857 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.857 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.857 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T04:29:24.857 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T04:29:24.857 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.857 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:29:24.857 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.858 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.859 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.860 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.861 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:29:24.862 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.863 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mds", 2023-12-20T04:29:24.864 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBcbYJlu9WKNRAACPmel2685tIMLwVutd5mpg==", 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mds" 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-mgr", 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBcbYJlX+aKNRAAglTdpGtkH1TuTwmm93c78Q==", 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-mgr" 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.865 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-osd", 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBcbYJlUvSKNRAAXJ2ajoV4hUHXRIvzjy2waw==", 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-osd" 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd", 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBcbYJlXwKLNRAAnjuzFsDqGp0oH9A8zayk0g==", 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd" 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.866 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.867 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rbd-mirror", 2023-12-20T04:29:24.867 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBcbYJlwBGLNRAA6GHT6I3DD2NvkTakZRjsaw==", 2023-12-20T04:29:24.867 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.867 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rbd-mirror" 2023-12-20T04:29:24.867 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.867 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.867 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.867 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.bootstrap-rgw", 2023-12-20T04:29:24.867 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBcbYJlmCCLNRAAUA2zZ6HfKB7zt6MsUI/VOA==", 2023-12-20T04:29:24.867 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.867 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile bootstrap-rgw" 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:29:24.868 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:24.869 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T04:29:24.869 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T04:29:24.869 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:29:24.869 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:29:24.869 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:29:24.869 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:29:24.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.869 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:24.869 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T04:29:24.869 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:29:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.846+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/3181704593 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2a4065a60 msgr2=0x7fe2a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.846+0000 7fe2c7ab6640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2a4065a60 0x7fe2a4067f20 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7fe2bc000fc0 tx=0x7fe2bc009040 comp rx=0 tx=0).stop 2023-12-20T04:29:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.846+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/3181704593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2c01599e0 msgr2=0x7fe2c01f4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.846+0000 7fe2c7ab6640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2c01599e0 0x7fe2c01f4e40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7fe2b8007ea0 tx=0x7fe2b8007f90 comp rx=0 tx=0).stop 2023-12-20T04:29:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.850+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/3181704593 shutdown_connections 2023-12-20T04:29:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.850+0000 7fe2c7ab6640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2a4065a60 0x7fe2a4067f20 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.850+0000 7fe2c7ab6640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2c01599e0 0x7fe2c01f4e40 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.850+0000 7fe2c7ab6640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2c00b0b30 0x7fe2c00adbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.850+0000 7fe2c7ab6640 1 --2- 172.21.15.32:0/3181704593 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2c00aece0 0x7fe2c00ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.850+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/3181704593 >> 172.21.15.32:0/3181704593 conn(0x7fe2c006a320 msgr2=0x7fe2c00cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:24.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.850+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/3181704593 shutdown_connections 2023-12-20T04:29:24.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:24.850+0000 7fe2c7ab6640 1 -- 172.21.15.32:0/3181704593 wait complete. 2023-12-20T04:29:24.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mds 2023-12-20T04:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd791a12640 1 Processor -- start 2023-12-20T04:29:25.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd791a12640 1 -- start start 2023-12-20T04:29:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd791a12640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c0d2740 0x7fd78c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd791a12640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd78c0cfab0 0x7fd78c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd791a12640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd78c15d190 con 0x7fd78c15cdb0 2023-12-20T04:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd791a12640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd78c0cff40 con 0x7fd78c0cfab0 2023-12-20T04:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd791a12640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c0d16a0 con 0x7fd78c0d2740 2023-12-20T04:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd78affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd78c15cdb0 0x7fd78c0d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52638/0 (socket says 172.21.15.32:52638) 2023-12-20T04:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd78affd640 1 -- 172.21.15.32:0/2507115909 learned_addr learned my addr 172.21.15.32:0/2507115909 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd78a7fc640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c0d2740 0x7fd78c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd78b7fe640 1 --2- 172.21.15.32:0/2507115909 >> v2:172.21.15.39:3300/0 conn(0x7fd78c0cfab0 0x7fd78c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3134124570 0 0) 0x7fd78c15d190 con 0x7fd78c15cdb0 2023-12-20T04:29:25.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd774003530 con 0x7fd78c15cdb0 2023-12-20T04:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1717623448 0 0) 0x7fd774003530 con 0x7fd78c15cdb0 2023-12-20T04:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c0d2740 msgr2=0x7fd78c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c0d2740 0x7fd78c0cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 >> v2:172.21.15.39:3300/0 conn(0x7fd78c0cfab0 msgr2=0x7fd78c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 --2- 172.21.15.32:0/2507115909 >> v2:172.21.15.39:3300/0 conn(0x7fd78c0cfab0 0x7fd78c0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c0d1980 con 0x7fd78c15cdb0 2023-12-20T04:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fd77c001680 con 0x7fd78c15cdb0 2023-12-20T04:29:25.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd78a7fc640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c0d2740 0x7fd78c0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 >> v1:172.21.15.32:6789/0 conn(0x7fd78c15cdb0 legacy=0x7fd78c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd774003980 0x7fd774003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c0d2740 0x7fd7740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd774004d10 0x7fd774005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c0cff40 con 0x7fd774004d10 2023-12-20T04:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c0d16a0 con 0x7fd774003980 2023-12-20T04:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd77400b990 con 0x7fd78c0d2740 2023-12-20T04:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd78affd640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd774003980 0x7fd774003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.014+0000 7fd78b7fe640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd774004d10 0x7fd774005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.018+0000 7fd78a7fc640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c0d2740 0x7fd7740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.018+0000 7fd78b7fe640 1 -- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c0d2740 msgr2=0x7fd7740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.018+0000 7fd78b7fe640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c0d2740 0x7fd7740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.018+0000 7fd78b7fe640 1 -- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd774003980 msgr2=0x7fd774003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.018+0000 7fd78b7fe640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd774003980 0x7fd774003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.018+0000 7fd78b7fe640 1 -- 172.21.15.32:0/2507115909 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd780009420 con 0x7fd774004d10 2023-12-20T04:29:25.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.018+0000 7fd78affd640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd774003980 0x7fd774003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:25.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.018+0000 7fd78b7fe640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd774004d10 0x7fd774005190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd78400e170 tx=0x7fd78401cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e5ec22fc4d89d3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:25.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.018+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7840a5c90 con 0x7fd774004d10 2023-12-20T04:29:25.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.018+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78400f040 con 0x7fd774004d10 2023-12-20T04:29:25.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.018+0000 7fd789ffb640 1 -- 172.21.15.32:0/2507115909 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7840ad440 con 0x7fd774004d10 2023-12-20T04:29:25.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 -- 172.21.15.32:0/2507115909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd774004d10 msgr2=0x7fd774005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 --2- 172.21.15.32:0/2507115909 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd774004d10 0x7fd774005190 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fd78400e170 tx=0x7fd78401cc80 comp rx=0 tx=0).stop 2023-12-20T04:29:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd78b7fe640 1 -- 172.21.15.32:0/2507115909 reap_dead start 2023-12-20T04:29:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 -- 172.21.15.32:0/2507115909 shutdown_connections 2023-12-20T04:29:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 -- 172.21.15.32:0/2507115909 >> 172.21.15.32:0/2507115909 conn(0x7fd78c065ee0 msgr2=0x7fd78c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 -- 172.21.15.32:0/2507115909 shutdown_connections 2023-12-20T04:29:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 -- 172.21.15.32:0/2507115909 wait complete. 2023-12-20T04:29:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 Processor -- start 2023-12-20T04:29:25.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 -- start start 2023-12-20T04:29:25.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd78c0b6d00 0x7fd78c158000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c158540 0x7fd78c1f5c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c1f6170 0x7fd78c155220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c154a40 con 0x7fd78c158540 2023-12-20T04:29:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c1f6680 con 0x7fd78c0b6d00 2023-12-20T04:29:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd791a12640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd78c1f6960 con 0x7fd78c1f6170 2023-12-20T04:29:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c158540 0x7fd78c1f5c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd78a7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c158540 0x7fd78c1f5c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38022/0 (socket says 172.21.15.32:38022) 2023-12-20T04:29:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd78a7fc640 1 -- 172.21.15.32:0/1383377589 learned_addr learned my addr 172.21.15.32:0/1383377589 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:25.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd78a7fc640 1 -- 172.21.15.32:0/1383377589 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c1f6170 msgr2=0x7fd78c155220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd78a7fc640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c1f6170 0x7fd78c155220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd78a7fc640 1 -- 172.21.15.32:0/1383377589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd78c0b6d00 msgr2=0x7fd78c158000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd78a7fc640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd78c0b6d00 0x7fd78c158000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.022+0000 7fd78a7fc640 1 -- 172.21.15.32:0/1383377589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd78c154d00 con 0x7fd78c158540 2023-12-20T04:29:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd78a7fc640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c158540 0x7fd78c1f5c30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd780011cd0 tx=0x7fd78000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd773fff640 1 -- 172.21.15.32:0/1383377589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd78000fca0 con 0x7fd78c158540 2023-12-20T04:29:25.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd773fff640 1 -- 172.21.15.32:0/1383377589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd78000fe00 con 0x7fd78c158540 2023-12-20T04:29:25.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd773fff640 1 -- 172.21.15.32:0/1383377589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd780004240 con 0x7fd78c158540 2023-12-20T04:29:25.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd791a12640 1 -- 172.21.15.32:0/1383377589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd78c15f380 con 0x7fd78c158540 2023-12-20T04:29:25.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd791a12640 1 -- 172.21.15.32:0/1383377589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd78c0b72b0 con 0x7fd78c158540 2023-12-20T04:29:25.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd791a12640 1 -- 172.21.15.32:0/1383377589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd78c04e000 con 0x7fd78c158540 2023-12-20T04:29:25.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd773fff640 1 -- 172.21.15.32:0/1383377589 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fd7800043a0 con 0x7fd78c158540 2023-12-20T04:29:25.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd773fff640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd778065a60 0x7fd778067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd773fff640 1 -- 172.21.15.32:0/1383377589 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd780083590 con 0x7fd78c158540 2023-12-20T04:29:25.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd78affd640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd778065a60 0x7fd778067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.026+0000 7fd78affd640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd778065a60 0x7fd778067f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd77c007cf0 tx=0x7fd77c007c80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:25.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.030+0000 7fd773fff640 1 -- 172.21.15.32:0/1383377589 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd780051290 con 0x7fd78c158540 2023-12-20T04:29:25.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.242+0000 7fd791a12640 1 -- 172.21.15.32:0/1383377589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mds"} v 0) v1 -- 0x7fd78c1548a0 con 0x7fd78c158540 2023-12-20T04:29:25.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.246+0000 7fd773fff640 1 -- 172.21.15.32:0/1383377589 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mds"}]=0 v7) v1 ==== 90+0+0 (secure 0 0 0) 0x7fd78c1548a0 con 0x7fd78c158540 2023-12-20T04:29:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 -- 172.21.15.32:0/1383377589 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd778065a60 msgr2=0x7fd778067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd778065a60 0x7fd778067f20 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fd77c007cf0 tx=0x7fd77c007c80 comp rx=0 tx=0).stop 2023-12-20T04:29:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 -- 172.21.15.32:0/1383377589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c158540 msgr2=0x7fd78c1f5c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c158540 0x7fd78c1f5c30 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7fd780011cd0 tx=0x7fd78000f610 comp rx=0 tx=0).stop 2023-12-20T04:29:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 -- 172.21.15.32:0/1383377589 shutdown_connections 2023-12-20T04:29:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd778065a60 0x7fd778067f20 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd78c1f6170 0x7fd78c155220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd78c158540 0x7fd78c1f5c30 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 --2- 172.21.15.32:0/1383377589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd78c0b6d00 0x7fd78c158000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 -- 172.21.15.32:0/1383377589 >> 172.21.15.32:0/1383377589 conn(0x7fd78c065ee0 msgr2=0x7fd78c0ce010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 -- 172.21.15.32:0/1383377589 shutdown_connections 2023-12-20T04:29:25.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.250+0000 7fd791a12640 1 -- 172.21.15.32:0/1383377589 wait complete. 2023-12-20T04:29:25.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-mgr 2023-12-20T04:29:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f51721db640 1 Processor -- start 2023-12-20T04:29:25.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f51721db640 1 -- start start 2023-12-20T04:29:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f51721db640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f516c0d2740 0x7f516c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f51721db640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f516c0cfab0 0x7f516c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f51721db640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f516c0cfec0 con 0x7f516c15cdb0 2023-12-20T04:29:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f51721db640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c15d190 con 0x7f516c0d2740 2023-12-20T04:29:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f51721db640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f516c0d16a0 con 0x7f516c0cfab0 2023-12-20T04:29:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f516affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f516c0d2740 0x7f516c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f516affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f516c0d2740 0x7f516c0cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38028/0 (socket says 172.21.15.32:38028) 2023-12-20T04:29:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f516affd640 1 -- 172.21.15.32:0/1610214017 learned_addr learned my addr 172.21.15.32:0/1610214017 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:25.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f516bfff640 1 --2- 172.21.15.32:0/1610214017 >> v2:172.21.15.39:3300/0 conn(0x7f516c0cfab0 0x7f516c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f516affd640 1 -- 172.21.15.32:0/1610214017 >> v2:172.21.15.39:3300/0 conn(0x7f516c0cfab0 msgr2=0x7f516c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f516affd640 1 --2- 172.21.15.32:0/1610214017 >> v2:172.21.15.39:3300/0 conn(0x7f516c0cfab0 0x7f516c0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f516affd640 1 -- 172.21.15.32:0/1610214017 >> v1:172.21.15.39:6789/0 conn(0x7f516c15cdb0 legacy=0x7f516c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f516affd640 1 -- 172.21.15.32:0/1610214017 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f516c0d1980 con 0x7f516c0d2740 2023-12-20T04:29:25.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.414+0000 7f516a7fc640 1 -- 172.21.15.32:0/1610214017 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 289705634 0 0) 0x7f516c0cfec0 con 0x7f516c15cdb0 2023-12-20T04:29:25.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f516affd640 1 --2- 172.21.15.32:0/1610214017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c0d2740 0x7f516c0cf570 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f516c06c100 tx=0x7f516001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fe85c53206bda35 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:25.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f516a7fc640 1 -- 172.21.15.32:0/1610214017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600a89f0 con 0x7f516c0d2740 2023-12-20T04:29:25.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f516a7fc640 1 -- 172.21.15.32:0/1610214017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51600a8b50 con 0x7f516c0d2740 2023-12-20T04:29:25.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f516a7fc640 1 -- 172.21.15.32:0/1610214017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51600b0440 con 0x7f516c0d2740 2023-12-20T04:29:25.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f51721db640 1 -- 172.21.15.32:0/1610214017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c0d2740 msgr2=0x7f516c0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f51721db640 1 --2- 172.21.15.32:0/1610214017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c0d2740 0x7f516c0cf570 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f516c06c100 tx=0x7f516001ecf0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f51721db640 1 -- 172.21.15.32:0/1610214017 shutdown_connections 2023-12-20T04:29:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f51721db640 1 --2- 172.21.15.32:0/1610214017 >> v2:172.21.15.39:3300/0 conn(0x7f516c0cfab0 0x7f516c0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f51721db640 1 --2- 172.21.15.32:0/1610214017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c0d2740 0x7f516c0cf570 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f51721db640 1 -- 172.21.15.32:0/1610214017 >> 172.21.15.32:0/1610214017 conn(0x7f516c065ee0 msgr2=0x7f516c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f51721db640 1 -- 172.21.15.32:0/1610214017 shutdown_connections 2023-12-20T04:29:25.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.418+0000 7f51721db640 1 -- 172.21.15.32:0/1610214017 wait complete. 2023-12-20T04:29:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f51721db640 1 Processor -- start 2023-12-20T04:29:25.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f51721db640 1 -- start start 2023-12-20T04:29:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f51721db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c0cfab0 0x7f516c151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f51721db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f516c0d2740 0x7f516c1571e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f51721db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516c15cdb0 0x7f516c157720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f51721db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c158b90 con 0x7f516c0cfab0 2023-12-20T04:29:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f51721db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f516c1f6890 con 0x7f516c15cdb0 2023-12-20T04:29:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f51721db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f516c157c60 con 0x7f516c0d2740 2023-12-20T04:29:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c0cfab0 0x7f516c151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c0cfab0 0x7f516c151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38044/0 (socket says 172.21.15.32:38044) 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516b7fe640 1 -- 172.21.15.32:0/1780277815 learned_addr learned my addr 172.21.15.32:0/1780277815 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516affd640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f516c0d2740 0x7f516c1571e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516bfff640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516c15cdb0 0x7f516c157720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516b7fe640 1 -- 172.21.15.32:0/1780277815 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f516c0d2740 msgr2=0x7f516c1571e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516b7fe640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f516c0d2740 0x7f516c1571e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516b7fe640 1 -- 172.21.15.32:0/1780277815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516c15cdb0 msgr2=0x7f516c157720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516b7fe640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516c15cdb0 0x7f516c157720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516b7fe640 1 -- 172.21.15.32:0/1780277815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51600094d0 con 0x7f516c0cfab0 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516bfff640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516c15cdb0 0x7f516c157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f516b7fe640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c0cfab0 0x7f516c151620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f515c001ec0 tx=0x7f515c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f5153fff640 1 -- 172.21.15.32:0/1780277815 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0054e0 con 0x7f516c0cfab0 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f5153fff640 1 -- 172.21.15.32:0/1780277815 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f515c003600 con 0x7f516c0cfab0 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f5153fff640 1 -- 172.21.15.32:0/1780277815 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f515c0039b0 con 0x7f516c0cfab0 2023-12-20T04:29:25.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f51721db640 1 -- 172.21.15.32:0/1780277815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f516c1f6d20 con 0x7f516c0cfab0 2023-12-20T04:29:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.422+0000 7f51721db640 1 -- 172.21.15.32:0/1780277815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f516c1f6240 con 0x7f516c0cfab0 2023-12-20T04:29:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.426+0000 7f51721db640 1 -- 172.21.15.32:0/1780277815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f516c04e000 con 0x7f516c0cfab0 2023-12-20T04:29:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.426+0000 7f5153fff640 1 -- 172.21.15.32:0/1780277815 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f515c003b10 con 0x7f516c0cfab0 2023-12-20T04:29:25.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.426+0000 7f5153fff640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5158065ab0 0x7f5158067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.426+0000 7f5153fff640 1 -- 172.21.15.32:0/1780277815 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f515c0823e0 con 0x7f516c0cfab0 2023-12-20T04:29:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.426+0000 7f516affd640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5158065ab0 0x7f5158067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.426+0000 7f516affd640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5158065ab0 0x7f5158067f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f51600b2000 tx=0x7f5160009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:25.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.430+0000 7f5153fff640 1 -- 172.21.15.32:0/1780277815 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f515c050180 con 0x7f516c0cfab0 2023-12-20T04:29:25.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.646+0000 7f51721db640 1 -- 172.21.15.32:0/1780277815 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-mgr"} v 0) v1 -- 0x7f516c0d1f10 con 0x7f516c0cfab0 2023-12-20T04:29:25.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.650+0000 7f5153fff640 1 -- 172.21.15.32:0/1780277815 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-mgr"}]=0 v8) v1 ==== 90+0+0 (secure 0 0 0) 0x7f515c0170a0 con 0x7f516c0cfab0 2023-12-20T04:29:25.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 -- 172.21.15.32:0/1780277815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5158065ab0 msgr2=0x7f5158067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5158065ab0 0x7f5158067f70 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f51600b2000 tx=0x7f5160009040 comp rx=0 tx=0).stop 2023-12-20T04:29:25.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 -- 172.21.15.32:0/1780277815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c0cfab0 msgr2=0x7f516c151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c0cfab0 0x7f516c151620 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f515c001ec0 tx=0x7f515c00c8e0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 -- 172.21.15.32:0/1780277815 shutdown_connections 2023-12-20T04:29:25.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5158065ab0 0x7f5158067f70 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f516c15cdb0 0x7f516c157720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f516c0d2740 0x7f516c1571e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 --2- 172.21.15.32:0/1780277815 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f516c0cfab0 0x7f516c151620 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 -- 172.21.15.32:0/1780277815 >> 172.21.15.32:0/1780277815 conn(0x7f516c065ee0 msgr2=0x7f516c0cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:25.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 -- 172.21.15.32:0/1780277815 shutdown_connections 2023-12-20T04:29:25.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.654+0000 7f51721db640 1 -- 172.21.15.32:0/1780277815 wait complete. 2023-12-20T04:29:25.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-osd 2023-12-20T04:29:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822c76d640 1 Processor -- start 2023-12-20T04:29:25.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822c76d640 1 -- start start 2023-12-20T04:29:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822c76d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82240af5e0 0x7f82240af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822c76d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822415b990 con 0x7f822415f110 2023-12-20T04:29:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822c76d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f822415bbd0 con 0x7f822406ce60 2023-12-20T04:29:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822c76d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f822415beb0 con 0x7f82240af5e0 2023-12-20T04:29:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822ad0a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f822415f110 0x7f822415c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52646/0 (socket says 172.21.15.32:52646) 2023-12-20T04:29:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822ad0a640 1 -- 172.21.15.32:0/1810845442 learned_addr learned my addr 172.21.15.32:0/1810845442 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822a509640 1 --2- 172.21.15.32:0/1810845442 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82240af5e0 0x7f82240af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f8229507640 1 -- 172.21.15.32:0/1810845442 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 605167622 0 0) 0x7f821c0014a0 con 0x7f822415f110 2023-12-20T04:29:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f8229507640 1 -- 172.21.15.32:0/1810845442 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8210003530 con 0x7f822415f110 2023-12-20T04:29:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f8229507640 1 -- 172.21.15.32:0/1810845442 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2017256818 0 0) 0x7f822415bbd0 con 0x7f822406ce60 2023-12-20T04:29:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f8229507640 1 -- 172.21.15.32:0/1810845442 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f821c0014a0 con 0x7f822406ce60 2023-12-20T04:29:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f8229507640 1 -- 172.21.15.32:0/1810845442 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3054920776 0 0) 0x7f8210003530 con 0x7f822415f110 2023-12-20T04:29:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822a509640 1 -- 172.21.15.32:0/1810845442 >> v1:172.21.15.39:6789/0 conn(0x7f822406ce60 legacy=0x7f822415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822a509640 1 -- 172.21.15.32:0/1810845442 >> v1:172.21.15.32:6789/0 conn(0x7f822415f110 legacy=0x7f822415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.818+0000 7f822a509640 1 -- 172.21.15.32:0/1810845442 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f822415e910 con 0x7f82240af5e0 2023-12-20T04:29:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.822+0000 7f822a509640 1 --2- 172.21.15.32:0/1810845442 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82240af5e0 0x7f82240af9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f82200097d0 tx=0x7f822001c390 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6a344776f3b04295 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.822+0000 7f8229507640 1 -- 172.21.15.32:0/1810845442 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c001b90 con 0x7f82240af5e0 2023-12-20T04:29:25.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.822+0000 7f8229507640 1 -- 172.21.15.32:0/1810845442 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82200a7050 con 0x7f82240af5e0 2023-12-20T04:29:25.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.822+0000 7f8229507640 1 -- 172.21.15.32:0/1810845442 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82200ab620 con 0x7f82240af5e0 2023-12-20T04:29:25.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.822+0000 7f822c76d640 1 -- 172.21.15.32:0/1810845442 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82240af5e0 msgr2=0x7f82240af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.822+0000 7f822c76d640 1 --2- 172.21.15.32:0/1810845442 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82240af5e0 0x7f82240af9c0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f82200097d0 tx=0x7f822001c390 comp rx=0 tx=0).stop 2023-12-20T04:29:25.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.822+0000 7f822c76d640 1 -- 172.21.15.32:0/1810845442 shutdown_connections 2023-12-20T04:29:25.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.822+0000 7f822c76d640 1 --2- 172.21.15.32:0/1810845442 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82240af5e0 0x7f82240af9c0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.822+0000 7f822c76d640 1 -- 172.21.15.32:0/1810845442 >> 172.21.15.32:0/1810845442 conn(0x7f822406a320 msgr2=0x7f82240ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:25.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.822+0000 7f822c76d640 1 -- 172.21.15.32:0/1810845442 shutdown_connections 2023-12-20T04:29:25.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.822+0000 7f822c76d640 1 -- 172.21.15.32:0/1810845442 wait complete. 2023-12-20T04:29:25.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822c76d640 1 Processor -- start 2023-12-20T04:29:25.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822c76d640 1 -- start start 2023-12-20T04:29:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822c76d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f822406ce60 0x7f8224070d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822c76d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82240af5e0 0x7f8224136190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822c76d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822415f110 0x7f82241366d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822c76d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8224072190 con 0x7f822415f110 2023-12-20T04:29:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822c76d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8224071580 con 0x7f82240af5e0 2023-12-20T04:29:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822c76d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8224071860 con 0x7f822406ce60 2023-12-20T04:29:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822ad0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822415f110 0x7f82241366d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822ad0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822415f110 0x7f82241366d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38060/0 (socket says 172.21.15.32:38060) 2023-12-20T04:29:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822ad0a640 1 -- 172.21.15.32:0/148912625 learned_addr learned my addr 172.21.15.32:0/148912625 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822a509640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f822406ce60 0x7f8224070d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f8229d08640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82240af5e0 0x7f8224136190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822ad0a640 1 -- 172.21.15.32:0/148912625 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f822406ce60 msgr2=0x7f8224070d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822ad0a640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f822406ce60 0x7f8224070d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822ad0a640 1 -- 172.21.15.32:0/148912625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82240af5e0 msgr2=0x7f8224136190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822ad0a640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82240af5e0 0x7f8224136190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822ad0a640 1 -- 172.21.15.32:0/148912625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82200093b0 con 0x7f822415f110 2023-12-20T04:29:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822ad0a640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822415f110 0x7f82241366d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f821c0020a0 tx=0x7f821c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f821affd640 1 -- 172.21.15.32:0/148912625 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c012600 con 0x7f822415f110 2023-12-20T04:29:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f821affd640 1 -- 172.21.15.32:0/148912625 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f821c012760 con 0x7f822415f110 2023-12-20T04:29:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822c76d640 1 -- 172.21.15.32:0/148912625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8224071b40 con 0x7f822415f110 2023-12-20T04:29:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f821affd640 1 -- 172.21.15.32:0/148912625 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f821c012b10 con 0x7f822415f110 2023-12-20T04:29:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822c76d640 1 -- 172.21.15.32:0/148912625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8224071290 con 0x7f822415f110 2023-12-20T04:29:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.826+0000 7f822c76d640 1 -- 172.21.15.32:0/148912625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8224158ea0 con 0x7f822415f110 2023-12-20T04:29:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.830+0000 7f821affd640 1 -- 172.21.15.32:0/148912625 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f821c012c70 con 0x7f822415f110 2023-12-20T04:29:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.830+0000 7f821affd640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8208065a60 0x7f8208067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.830+0000 7f821affd640 1 -- 172.21.15.32:0/148912625 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f821c083560 con 0x7f822415f110 2023-12-20T04:29:25.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.830+0000 7f822a509640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8208065a60 0x7f8208067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:25.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.830+0000 7f822a509640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8208065a60 0x7f8208067f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f82200ad000 tx=0x7f82200a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:25.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:25.834+0000 7f821affd640 1 -- 172.21.15.32:0/148912625 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8224158ea0 con 0x7f822415f110 2023-12-20T04:29:26.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.046+0000 7f822c76d640 1 -- 172.21.15.32:0/148912625 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-osd"} v 0) v1 -- 0x7f822415b990 con 0x7f822415f110 2023-12-20T04:29:26.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f821affd640 1 -- 172.21.15.32:0/148912625 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-osd"}]=0 v9) v1 ==== 90+0+0 (secure 0 0 0) 0x7f822415b990 con 0x7f822415f110 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 -- 172.21.15.32:0/148912625 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8208065a60 msgr2=0x7f8208067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8208065a60 0x7f8208067f20 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f82200ad000 tx=0x7f82200a6000 comp rx=0 tx=0).stop 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 -- 172.21.15.32:0/148912625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822415f110 msgr2=0x7f82241366d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822415f110 0x7f82241366d0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f821c0020a0 tx=0x7f821c00a430 comp rx=0 tx=0).stop 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 -- 172.21.15.32:0/148912625 shutdown_connections 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8208065a60 0x7f8208067f20 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f822415f110 0x7f82241366d0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82240af5e0 0x7f8224136190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 --2- 172.21.15.32:0/148912625 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f822406ce60 0x7f8224070d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 -- 172.21.15.32:0/148912625 >> 172.21.15.32:0/148912625 conn(0x7f822406a320 msgr2=0x7f822406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 -- 172.21.15.32:0/148912625 shutdown_connections 2023-12-20T04:29:26.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.054+0000 7f822c76d640 1 -- 172.21.15.32:0/148912625 wait complete. 2023-12-20T04:29:26.076 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd 2023-12-20T04:29:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.218+0000 7f9d77eb6640 1 Processor -- start 2023-12-20T04:29:26.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.218+0000 7f9d77eb6640 1 -- start start 2023-12-20T04:29:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.218+0000 7f9d77eb6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d700ae010 0x7f9d70071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.218+0000 7f9d77eb6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9d7006cf10 0x7f9d700719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.218+0000 7f9d77eb6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d70164ef0 0x7f9d700701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d77eb6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d700ae3f0 con 0x7f9d70164ef0 2023-12-20T04:29:26.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d77eb6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d70161800 con 0x7f9d7006cf10 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d77eb6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70165380 con 0x7f9d700ae010 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d76453640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d70164ef0 0x7f9d700701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d76453640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d70164ef0 0x7f9d700701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38070/0 (socket says 172.21.15.32:38070) 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d76453640 1 -- 172.21.15.32:0/3563698714 learned_addr learned my addr 172.21.15.32:0/3563698714 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d75c52640 1 --2- 172.21.15.32:0/3563698714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d700ae010 0x7f9d70071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d75451640 1 --2- 172.21.15.32:0/3563698714 >> v2:172.21.15.39:3300/0 conn(0x7f9d7006cf10 0x7f9d700719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d76453640 1 -- 172.21.15.32:0/3563698714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d700ae010 msgr2=0x7f9d70071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d76453640 1 --2- 172.21.15.32:0/3563698714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d700ae010 0x7f9d70071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d76453640 1 -- 172.21.15.32:0/3563698714 >> v2:172.21.15.39:3300/0 conn(0x7f9d7006cf10 msgr2=0x7f9d700719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d76453640 1 --2- 172.21.15.32:0/3563698714 >> v2:172.21.15.39:3300/0 conn(0x7f9d7006cf10 0x7f9d700719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d76453640 1 -- 172.21.15.32:0/3563698714 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d70165660 con 0x7f9d70164ef0 2023-12-20T04:29:26.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d75c52640 1 --2- 172.21.15.32:0/3563698714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d700ae010 0x7f9d70071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:26.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.222+0000 7f9d76453640 1 --2- 172.21.15.32:0/3563698714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d70164ef0 0x7f9d700701c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9d7006ad00 tx=0x7f9d6801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72df4ef57a4a4f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:26.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d74c50640 1 -- 172.21.15.32:0/3563698714 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d680a5790 con 0x7f9d70164ef0 2023-12-20T04:29:26.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d74c50640 1 -- 172.21.15.32:0/3563698714 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d680a58f0 con 0x7f9d70164ef0 2023-12-20T04:29:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d74c50640 1 -- 172.21.15.32:0/3563698714 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d680a5c80 con 0x7f9d70164ef0 2023-12-20T04:29:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 -- 172.21.15.32:0/3563698714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d70164ef0 msgr2=0x7f9d700701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 --2- 172.21.15.32:0/3563698714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d70164ef0 0x7f9d700701c0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f9d7006ad00 tx=0x7f9d6801c910 comp rx=0 tx=0).stop 2023-12-20T04:29:26.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 -- 172.21.15.32:0/3563698714 shutdown_connections 2023-12-20T04:29:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 --2- 172.21.15.32:0/3563698714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d70164ef0 0x7f9d700701c0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 --2- 172.21.15.32:0/3563698714 >> v2:172.21.15.39:3300/0 conn(0x7f9d7006cf10 0x7f9d700719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 --2- 172.21.15.32:0/3563698714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d700ae010 0x7f9d70071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 -- 172.21.15.32:0/3563698714 >> 172.21.15.32:0/3563698714 conn(0x7f9d7006a320 msgr2=0x7f9d700ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 -- 172.21.15.32:0/3563698714 shutdown_connections 2023-12-20T04:29:26.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 -- 172.21.15.32:0/3563698714 wait complete. 2023-12-20T04:29:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 Processor -- start 2023-12-20T04:29:26.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 -- start start 2023-12-20T04:29:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d700ae010 0x7f9d701510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700b33f0 0x7f9d701591e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d70159720 0x7f9d7015ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d70159ba0 con 0x7f9d700b33f0 2023-12-20T04:29:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d701fa930 con 0x7f9d700ae010 2023-12-20T04:29:26.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d77eb6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d701fac10 con 0x7f9d70159720 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d75451640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700b33f0 0x7f9d701591e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d75451640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700b33f0 0x7f9d701591e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38084/0 (socket says 172.21.15.32:38084) 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d75451640 1 -- 172.21.15.32:0/1213204225 learned_addr learned my addr 172.21.15.32:0/1213204225 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d75c52640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d700ae010 0x7f9d701510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.226+0000 7f9d76453640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d70159720 0x7f9d7015ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d75451640 1 -- 172.21.15.32:0/1213204225 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d70159720 msgr2=0x7f9d7015ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d75451640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d70159720 0x7f9d7015ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d75451640 1 -- 172.21.15.32:0/1213204225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d700ae010 msgr2=0x7f9d701510f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d75451640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d700ae010 0x7f9d701510f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d75451640 1 -- 172.21.15.32:0/1213204225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d68009b30 con 0x7f9d700b33f0 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d75c52640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d700ae010 0x7f9d701510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d75451640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700b33f0 0x7f9d701591e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000e8b0 tx=0x7f9d6000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d667fc640 1 -- 172.21.15.32:0/1213204225 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000c940 con 0x7f9d700b33f0 2023-12-20T04:29:26.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d667fc640 1 -- 172.21.15.32:0/1213204225 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d6000caa0 con 0x7f9d700b33f0 2023-12-20T04:29:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d667fc640 1 -- 172.21.15.32:0/1213204225 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d6000ce30 con 0x7f9d700b33f0 2023-12-20T04:29:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d77eb6640 1 -- 172.21.15.32:0/1213204225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d701faef0 con 0x7f9d700b33f0 2023-12-20T04:29:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d77eb6640 1 -- 172.21.15.32:0/1213204225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d701fb4b0 con 0x7f9d700b33f0 2023-12-20T04:29:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d77eb6640 1 -- 172.21.15.32:0/1213204225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d7004e000 con 0x7f9d700b33f0 2023-12-20T04:29:26.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d667fc640 1 -- 172.21.15.32:0/1213204225 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f9d60013470 con 0x7f9d700b33f0 2023-12-20T04:29:26.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d667fc640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d54065a60 0x7f9d54067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d667fc640 1 -- 172.21.15.32:0/1213204225 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9d60083010 con 0x7f9d700b33f0 2023-12-20T04:29:26.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.230+0000 7f9d75c52640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d54065a60 0x7f9d54067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.234+0000 7f9d75c52640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d54065a60 0x7f9d54067f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c006fd0 tx=0x7f9d6c008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:26.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.234+0000 7f9d667fc640 1 -- 172.21.15.32:0/1213204225 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9d60050d50 con 0x7f9d700b33f0 2023-12-20T04:29:26.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.450+0000 7f9d77eb6640 1 -- 172.21.15.32:0/1213204225 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd"} v 0) v1 -- 0x7f9d700cf450 con 0x7f9d700b33f0 2023-12-20T04:29:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d667fc640 1 -- 172.21.15.32:0/1213204225 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd"}]=0 v10) v1 ==== 90+0+0 (secure 0 0 0) 0x7f9d60054370 con 0x7f9d700b33f0 2023-12-20T04:29:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 -- 172.21.15.32:0/1213204225 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d54065a60 msgr2=0x7f9d54067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d54065a60 0x7f9d54067f20 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f9d6c006fd0 tx=0x7f9d6c008040 comp rx=0 tx=0).stop 2023-12-20T04:29:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 -- 172.21.15.32:0/1213204225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700b33f0 msgr2=0x7f9d701591e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700b33f0 0x7f9d701591e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7f9d6000e8b0 tx=0x7f9d6000ed70 comp rx=0 tx=0).stop 2023-12-20T04:29:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 -- 172.21.15.32:0/1213204225 shutdown_connections 2023-12-20T04:29:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d54065a60 0x7f9d54067f20 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d70159720 0x7f9d7015ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d700b33f0 0x7f9d701591e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 --2- 172.21.15.32:0/1213204225 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d700ae010 0x7f9d701510f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 -- 172.21.15.32:0/1213204225 >> 172.21.15.32:0/1213204225 conn(0x7f9d7006a320 msgr2=0x7f9d7015e3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 -- 172.21.15.32:0/1213204225 shutdown_connections 2023-12-20T04:29:26.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.454+0000 7f9d77eb6640 1 -- 172.21.15.32:0/1213204225 wait complete. 2023-12-20T04:29:26.477 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rbd-mirror 2023-12-20T04:29:26.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75bad87640 1 Processor -- start 2023-12-20T04:29:26.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75bad87640 1 -- start start 2023-12-20T04:29:26.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75bad87640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75b40b8480 0x7f75b406af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75bad87640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b40cd4d0 con 0x7f75b406b460 2023-12-20T04:29:26.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75bad87640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75b40ce4e0 con 0x7f75b4161500 2023-12-20T04:29:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75bad87640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75b4161930 con 0x7f75b40b8480 2023-12-20T04:29:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75b9d85640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75b40b8480 0x7f75b406af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75b9d85640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75b40b8480 0x7f75b406af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38090/0 (socket says 172.21.15.32:38090) 2023-12-20T04:29:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75b9d85640 1 -- 172.21.15.32:0/2113090798 learned_addr learned my addr 172.21.15.32:0/2113090798 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75b9584640 1 --1- 172.21.15.32:0/2113090798 >> v1:172.21.15.32:6789/0 conn(0x7f75b406b460 0x7f75b4160df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52660/0 (socket says 172.21.15.32:52660) 2023-12-20T04:29:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75b9d85640 1 -- 172.21.15.32:0/2113090798 >> v1:172.21.15.39:6789/0 conn(0x7f75b4161500 legacy=0x7f75b4153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75b9d85640 1 -- 172.21.15.32:0/2113090798 >> v1:172.21.15.32:6789/0 conn(0x7f75b406b460 legacy=0x7f75b4160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75b9d85640 1 -- 172.21.15.32:0/2113090798 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b40cd720 con 0x7f75b40b8480 2023-12-20T04:29:26.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.618+0000 7f75b8d83640 1 -- 172.21.15.32:0/2113090798 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 144848302 0 0) 0x7f75b40cd4d0 con 0x7f75b406b460 2023-12-20T04:29:26.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.622+0000 7f75b9d85640 1 --2- 172.21.15.32:0/2113090798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b40b8480 0x7f75b406af20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009870 tx=0x7f75b001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c8f428a10767a7c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.622+0000 7f75b8d83640 1 -- 172.21.15.32:0/2113090798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00ab600 con 0x7f75b40b8480 2023-12-20T04:29:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.622+0000 7f75b8d83640 1 -- 172.21.15.32:0/2113090798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75b00a7050 con 0x7f75b40b8480 2023-12-20T04:29:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.622+0000 7f75b8d83640 1 -- 172.21.15.32:0/2113090798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75b00ab980 con 0x7f75b40b8480 2023-12-20T04:29:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.622+0000 7f75bad87640 1 -- 172.21.15.32:0/2113090798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b40b8480 msgr2=0x7f75b406af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.622+0000 7f75bad87640 1 --2- 172.21.15.32:0/2113090798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b40b8480 0x7f75b406af20 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7f75b0009870 tx=0x7f75b001c500 comp rx=0 tx=0).stop 2023-12-20T04:29:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.622+0000 7f75bad87640 1 -- 172.21.15.32:0/2113090798 shutdown_connections 2023-12-20T04:29:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.622+0000 7f75bad87640 1 --2- 172.21.15.32:0/2113090798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b40b8480 0x7f75b406af20 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.622+0000 7f75bad87640 1 -- 172.21.15.32:0/2113090798 >> 172.21.15.32:0/2113090798 conn(0x7f75b40d2a50 msgr2=0x7f75b40d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.622+0000 7f75bad87640 1 -- 172.21.15.32:0/2113090798 shutdown_connections 2023-12-20T04:29:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.622+0000 7f75bad87640 1 -- 172.21.15.32:0/2113090798 wait complete. 2023-12-20T04:29:26.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75bad87640 1 Processor -- start 2023-12-20T04:29:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75bad87640 1 -- start start 2023-12-20T04:29:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75bad87640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f75b406b460 0x7f75b4149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75bad87640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b40b8480 0x7f75b40c7170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75bad87640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b4161500 0x7f75b414d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75bad87640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40c66d0 con 0x7f75b4161500 2023-12-20T04:29:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75bad87640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75b4151f80 con 0x7f75b40b8480 2023-12-20T04:29:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75bad87640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75b40c6970 con 0x7f75b406b460 2023-12-20T04:29:26.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75ba586640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b4161500 0x7f75b414d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75ba586640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b4161500 0x7f75b414d2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38104/0 (socket says 172.21.15.32:38104) 2023-12-20T04:29:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75ba586640 1 -- 172.21.15.32:0/4162878446 learned_addr learned my addr 172.21.15.32:0/4162878446 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75b9d85640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f75b406b460 0x7f75b4149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75b9584640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b40b8480 0x7f75b40c7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75ba586640 1 -- 172.21.15.32:0/4162878446 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f75b406b460 msgr2=0x7f75b4149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75ba586640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f75b406b460 0x7f75b4149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75ba586640 1 -- 172.21.15.32:0/4162878446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b40b8480 msgr2=0x7f75b40c7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75ba586640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b40b8480 0x7f75b40c7170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75ba586640 1 -- 172.21.15.32:0/4162878446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75b00094d0 con 0x7f75b4161500 2023-12-20T04:29:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75ba586640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b4161500 0x7f75b414d2c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00b970 tx=0x7f75ac00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75aa7fc640 1 -- 172.21.15.32:0/4162878446 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac00cb40 con 0x7f75b4161500 2023-12-20T04:29:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75aa7fc640 1 -- 172.21.15.32:0/4162878446 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75ac00cca0 con 0x7f75b4161500 2023-12-20T04:29:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75aa7fc640 1 -- 172.21.15.32:0/4162878446 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75ac014680 con 0x7f75b4161500 2023-12-20T04:29:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75bad87640 1 -- 172.21.15.32:0/4162878446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75b40c6c50 con 0x7f75b4161500 2023-12-20T04:29:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.626+0000 7f75bad87640 1 -- 172.21.15.32:0/4162878446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75b4152260 con 0x7f75b4161500 2023-12-20T04:29:26.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.630+0000 7f75bad87640 1 -- 172.21.15.32:0/4162878446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75b40c6e70 con 0x7f75b4161500 2023-12-20T04:29:26.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.630+0000 7f75aa7fc640 1 -- 172.21.15.32:0/4162878446 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f75ac01d3b0 con 0x7f75b4161500 2023-12-20T04:29:26.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.630+0000 7f75aa7fc640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7598065a60 0x7f7598067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:26.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.630+0000 7f75aa7fc640 1 -- 172.21.15.32:0/4162878446 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f75ac082bd0 con 0x7f75b4161500 2023-12-20T04:29:26.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.630+0000 7f75b9d85640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7598065a60 0x7f7598067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:26.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.630+0000 7f75b9d85640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7598065a60 0x7f7598067f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f75b00ad000 tx=0x7f75b00a4f90 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:26.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.634+0000 7f75aa7fc640 1 -- 172.21.15.32:0/4162878446 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f75ac050980 con 0x7f75b4161500 2023-12-20T04:29:26.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.850+0000 7f75bad87640 1 -- 172.21.15.32:0/4162878446 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"} v 0) v1 -- 0x7f75b4074100 con 0x7f75b4161500 2023-12-20T04:29:26.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75aa7fc640 1 -- 172.21.15.32:0/4162878446 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rbd-mirror"}]=0 v11) v1 ==== 97+0+0 (secure 0 0 0) 0x7f75ac053fa0 con 0x7f75b4161500 2023-12-20T04:29:26.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 -- 172.21.15.32:0/4162878446 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7598065a60 msgr2=0x7f7598067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7598065a60 0x7f7598067f20 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f75b00ad000 tx=0x7f75b00a4f90 comp rx=0 tx=0).stop 2023-12-20T04:29:26.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 -- 172.21.15.32:0/4162878446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b4161500 msgr2=0x7f75b414d2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:26.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b4161500 0x7f75b414d2c0 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f75ac00b970 tx=0x7f75ac00be30 comp rx=0 tx=0).stop 2023-12-20T04:29:26.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 -- 172.21.15.32:0/4162878446 shutdown_connections 2023-12-20T04:29:26.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7598065a60 0x7f7598067f20 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75b4161500 0x7f75b414d2c0 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75b40b8480 0x7f75b40c7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 --2- 172.21.15.32:0/4162878446 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f75b406b460 0x7f75b4149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 -- 172.21.15.32:0/4162878446 >> 172.21.15.32:0/4162878446 conn(0x7f75b40d2a50 msgr2=0x7f75b40d1720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 -- 172.21.15.32:0/4162878446 shutdown_connections 2023-12-20T04:29:26.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:26.854+0000 7f75bad87640 1 -- 172.21.15.32:0/4162878446 wait complete. 2023-12-20T04:29:26.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth del client.bootstrap-rgw 2023-12-20T04:29:27.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.018+0000 7f517fb98640 1 Processor -- start 2023-12-20T04:29:27.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.018+0000 7f517fb98640 1 -- start start 2023-12-20T04:29:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517fb98640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51780ae010 0x7f5178071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517fb98640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f517806cf10 0x7f51780719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517fb98640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51780ae3f0 con 0x7f5178164ef0 2023-12-20T04:29:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517fb98640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51780ca4b0 con 0x7f51780ae010 2023-12-20T04:29:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517fb98640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5178161800 con 0x7f517806cf10 2023-12-20T04:29:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517d934640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51780ae010 0x7f5178071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517d934640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51780ae010 0x7f5178071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38112/0 (socket says 172.21.15.32:38112) 2023-12-20T04:29:27.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517e135640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5178164ef0 0x7f51780701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52674/0 (socket says 172.21.15.32:52674) 2023-12-20T04:29:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517d934640 1 -- 172.21.15.32:0/2376337111 learned_addr learned my addr 172.21.15.32:0/2376337111 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517d133640 1 --2- 172.21.15.32:0/2376337111 >> v2:172.21.15.39:3300/0 conn(0x7f517806cf10 0x7f51780719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517d934640 1 -- 172.21.15.32:0/2376337111 >> v2:172.21.15.39:3300/0 conn(0x7f517806cf10 msgr2=0x7f51780719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517d934640 1 --2- 172.21.15.32:0/2376337111 >> v2:172.21.15.39:3300/0 conn(0x7f517806cf10 0x7f51780719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517d934640 1 -- 172.21.15.32:0/2376337111 >> v1:172.21.15.32:6789/0 conn(0x7f5178164ef0 legacy=0x7f51780701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517d934640 1 -- 172.21.15.32:0/2376337111 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5178165380 con 0x7f51780ae010 2023-12-20T04:29:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517c932640 1 -- 172.21.15.32:0/2376337111 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 42401777 0 0) 0x7f51780ae3f0 con 0x7f5178164ef0 2023-12-20T04:29:27.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.022+0000 7f517d133640 1 --2- 172.21.15.32:0/2376337111 >> v2:172.21.15.39:3300/0 conn(0x7f517806cf10 0x7f51780719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:27.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517d934640 1 --2- 172.21.15.32:0/2376337111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51780ae010 0x7f5178071470 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5178164e50 tx=0x7f517401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffca80919057b8b8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517c932640 1 -- 172.21.15.32:0/2376337111 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740ab640 con 0x7f51780ae010 2023-12-20T04:29:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517c932640 1 -- 172.21.15.32:0/2376337111 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51740a7070 con 0x7f51780ae010 2023-12-20T04:29:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517c932640 1 -- 172.21.15.32:0/2376337111 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740ab9c0 con 0x7f51780ae010 2023-12-20T04:29:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 -- 172.21.15.32:0/2376337111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51780ae010 msgr2=0x7f5178071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 --2- 172.21.15.32:0/2376337111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51780ae010 0x7f5178071470 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f5178164e50 tx=0x7f517401c440 comp rx=0 tx=0).stop 2023-12-20T04:29:27.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 -- 172.21.15.32:0/2376337111 shutdown_connections 2023-12-20T04:29:27.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 --2- 172.21.15.32:0/2376337111 >> v2:172.21.15.39:3300/0 conn(0x7f517806cf10 0x7f51780719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 --2- 172.21.15.32:0/2376337111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51780ae010 0x7f5178071470 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 -- 172.21.15.32:0/2376337111 >> 172.21.15.32:0/2376337111 conn(0x7f517806a320 msgr2=0x7f51780ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:27.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 -- 172.21.15.32:0/2376337111 shutdown_connections 2023-12-20T04:29:27.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 -- 172.21.15.32:0/2376337111 wait complete. 2023-12-20T04:29:27.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 Processor -- start 2023-12-20T04:29:27.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 -- start start 2023-12-20T04:29:27.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51780ae010 0x7f51781633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51780c45d0 0x7f51780c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51780c2340 0x7f51780c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51780c4f90 con 0x7f51780ae010 2023-12-20T04:29:27.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51780c30b0 con 0x7f51780c2340 2023-12-20T04:29:27.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.026+0000 7f517fb98640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51780c3390 con 0x7f51780c45d0 2023-12-20T04:29:27.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517d934640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51780ae010 0x7f51781633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517d934640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51780ae010 0x7f51781633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38126/0 (socket says 172.21.15.32:38126) 2023-12-20T04:29:27.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517d934640 1 -- 172.21.15.32:0/1709347088 learned_addr learned my addr 172.21.15.32:0/1709347088 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:27.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517d133640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51780c45d0 0x7f51780c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517e135640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51780c2340 0x7f51780c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517d934640 1 -- 172.21.15.32:0/1709347088 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51780c45d0 msgr2=0x7f51780c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517d934640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51780c45d0 0x7f51780c4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517d934640 1 -- 172.21.15.32:0/1709347088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51780c2340 msgr2=0x7f51780c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517d934640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51780c2340 0x7f51780c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517d934640 1 -- 172.21.15.32:0/1709347088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5174009480 con 0x7f51780ae010 2023-12-20T04:29:27.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517d934640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51780ae010 0x7f51781633e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f517401a070 tx=0x7f51740191e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:27.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f516e7fc640 1 -- 172.21.15.32:0/1709347088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5174003a20 con 0x7f51780ae010 2023-12-20T04:29:27.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f516e7fc640 1 -- 172.21.15.32:0/1709347088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5174003b80 con 0x7f51780ae010 2023-12-20T04:29:27.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f516e7fc640 1 -- 172.21.15.32:0/1709347088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51740ab440 con 0x7f51780ae010 2023-12-20T04:29:27.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517fb98640 1 -- 172.21.15.32:0/1709347088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51780c3670 con 0x7f51780ae010 2023-12-20T04:29:27.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517fb98640 1 -- 172.21.15.32:0/1709347088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51780c3c30 con 0x7f51780ae010 2023-12-20T04:29:27.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f517fb98640 1 -- 172.21.15.32:0/1709347088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f517804e000 con 0x7f51780ae010 2023-12-20T04:29:27.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.030+0000 7f516e7fc640 1 -- 172.21.15.32:0/1709347088 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f51740ab5a0 con 0x7f51780ae010 2023-12-20T04:29:27.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.034+0000 7f516e7fc640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f515c065a60 0x7f515c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.034+0000 7f516e7fc640 1 -- 172.21.15.32:0/1709347088 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f517411d650 con 0x7f51780ae010 2023-12-20T04:29:27.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.034+0000 7f517d133640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f515c065a60 0x7f515c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.034+0000 7f517d133640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f515c065a60 0x7f515c067f20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f51780ce320 tx=0x7f5168008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:27.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.038+0000 7f516e7fc640 1 -- 172.21.15.32:0/1709347088 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f51740eb3e0 con 0x7f51780ae010 2023-12-20T04:29:27.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.254+0000 7f517fb98640 1 -- 172.21.15.32:0/1709347088 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth del", "entity": "client.bootstrap-rgw"} v 0) v1 -- 0x7f51780cf450 con 0x7f51780ae010 2023-12-20T04:29:27.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.258+0000 7f516e7fc640 1 -- 172.21.15.32:0/1709347088 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth del", "entity": "client.bootstrap-rgw"}]=0 v12) v1 ==== 90+0+0 (secure 0 0 0) 0x7f51740eea00 con 0x7f51780ae010 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.258+0000 7f517fb98640 1 -- 172.21.15.32:0/1709347088 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f515c065a60 msgr2=0x7f515c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.258+0000 7f517fb98640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f515c065a60 0x7f515c067f20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f51780ce320 tx=0x7f5168008040 comp rx=0 tx=0).stop 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.262+0000 7f517fb98640 1 -- 172.21.15.32:0/1709347088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51780ae010 msgr2=0x7f51781633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.262+0000 7f517fb98640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51780ae010 0x7f51781633e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f517401a070 tx=0x7f51740191e0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.262+0000 7f517fb98640 1 -- 172.21.15.32:0/1709347088 shutdown_connections 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.262+0000 7f517fb98640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f515c065a60 0x7f515c067f20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.262+0000 7f517fb98640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51780c2340 0x7f51780c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.262+0000 7f517fb98640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51780c45d0 0x7f51780c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.262+0000 7f517fb98640 1 --2- 172.21.15.32:0/1709347088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51780ae010 0x7f51781633e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.262+0000 7f517fb98640 1 -- 172.21.15.32:0/1709347088 >> 172.21.15.32:0/1709347088 conn(0x7f517806a320 msgr2=0x7f517815d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.262+0000 7f517fb98640 1 -- 172.21.15.32:0/1709347088 shutdown_connections 2023-12-20T04:29:27.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.262+0000 7f517fb98640 1 -- 172.21.15.32:0/1709347088 wait complete. 2023-12-20T04:29:27.283 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:27.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f743168c640 1 Processor -- start 2023-12-20T04:29:27.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f743168c640 1 -- start start 2023-12-20T04:29:27.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f743168c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f742c15cd70 0x7f742c0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f743168c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f742c160ff0 0x7f742c0b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f743168c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f742c0b2d70 con 0x7f742c0b23d0 2023-12-20T04:29:27.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f743168c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f742c1618b0 con 0x7f742c160ff0 2023-12-20T04:29:27.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f743168c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c0ce6c0 con 0x7f742c15cd70 2023-12-20T04:29:27.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f742b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f742c160ff0 0x7f742c0b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f742affd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f742c15cd70 0x7f742c0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f742b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f742c160ff0 0x7f742c0b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:57698/0 (socket says 172.21.15.32:57698) 2023-12-20T04:29:27.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f742b7fe640 1 -- 172.21.15.32:0/1239160580 learned_addr learned my addr 172.21.15.32:0/1239160580 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:27.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f7429ffb640 1 -- 172.21.15.32:0/1239160580 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1018947003 0 0) 0x7f742c0b2d70 con 0x7f742c0b23d0 2023-12-20T04:29:27.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f7429ffb640 1 -- 172.21.15.32:0/1239160580 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7414003530 con 0x7f742c0b23d0 2023-12-20T04:29:27.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f742b7fe640 1 -- 172.21.15.32:0/1239160580 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f742c15cd70 msgr2=0x7f742c0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:27.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f742b7fe640 1 --2- 172.21.15.32:0/1239160580 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f742c15cd70 0x7f742c0cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f742b7fe640 1 -- 172.21.15.32:0/1239160580 >> v1:172.21.15.39:6789/0 conn(0x7f742c0b23d0 legacy=0x7f742c1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.426+0000 7f742b7fe640 1 -- 172.21.15.32:0/1239160580 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f742c1615d0 con 0x7f742c160ff0 2023-12-20T04:29:27.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f742b7fe640 1 --2- 172.21.15.32:0/1239160580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742c160ff0 0x7f742c0b2830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7424009e80 tx=0x7f742401c9c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b5264beb75ee3b94 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:27.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f7429ffb640 1 -- 172.21.15.32:0/1239160580 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240a5880 con 0x7f742c160ff0 2023-12-20T04:29:27.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f7429ffb640 1 -- 172.21.15.32:0/1239160580 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74240a59e0 con 0x7f742c160ff0 2023-12-20T04:29:27.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f7429ffb640 1 -- 172.21.15.32:0/1239160580 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74240a5d70 con 0x7f742c160ff0 2023-12-20T04:29:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f743168c640 1 -- 172.21.15.32:0/1239160580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742c160ff0 msgr2=0x7f742c0b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f743168c640 1 --2- 172.21.15.32:0/1239160580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742c160ff0 0x7f742c0b2830 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7424009e80 tx=0x7f742401c9c0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f743168c640 1 -- 172.21.15.32:0/1239160580 shutdown_connections 2023-12-20T04:29:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f743168c640 1 --2- 172.21.15.32:0/1239160580 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742c160ff0 0x7f742c0b2830 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f743168c640 1 --2- 172.21.15.32:0/1239160580 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f742c15cd70 0x7f742c0cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f743168c640 1 -- 172.21.15.32:0/1239160580 >> 172.21.15.32:0/1239160580 conn(0x7f742c0cd3d0 msgr2=0x7f742c0cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f743168c640 1 -- 172.21.15.32:0/1239160580 shutdown_connections 2023-12-20T04:29:27.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.430+0000 7f743168c640 1 -- 172.21.15.32:0/1239160580 wait complete. 2023-12-20T04:29:27.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f743168c640 1 Processor -- start 2023-12-20T04:29:27.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f743168c640 1 -- start start 2023-12-20T04:29:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f743168c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f742c0b23d0 0x7f742c159ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f743168c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742c15cd70 0x7f742c15c190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f743168c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f742c160ff0 0x7f742c15e4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f743168c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c15ebb0 con 0x7f742c0b23d0 2023-12-20T04:29:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f743168c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f742c15ee90 con 0x7f742c15cd70 2023-12-20T04:29:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f743168c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f742c15f170 con 0x7f742c160ff0 2023-12-20T04:29:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f742c0b23d0 0x7f742c159ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f742c0b23d0 0x7f742c159ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38138/0 (socket says 172.21.15.32:38138) 2023-12-20T04:29:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742affd640 1 -- 172.21.15.32:0/2127352500 learned_addr learned my addr 172.21.15.32:0/2127352500 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742a7fc640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742c15cd70 0x7f742c15c190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742b7fe640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f742c160ff0 0x7f742c15e4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742affd640 1 -- 172.21.15.32:0/2127352500 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f742c160ff0 msgr2=0x7f742c15e4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742affd640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f742c160ff0 0x7f742c15e4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742affd640 1 -- 172.21.15.32:0/2127352500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742c15cd70 msgr2=0x7f742c15c190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742affd640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742c15cd70 0x7f742c15c190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742affd640 1 -- 172.21.15.32:0/2127352500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7424009b30 con 0x7f742c0b23d0 2023-12-20T04:29:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742b7fe640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f742c160ff0 0x7f742c15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f742affd640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f742c0b23d0 0x7f742c159ff0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f741c00e930 tx=0x7f741c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f74137fe640 1 -- 172.21.15.32:0/2127352500 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c009690 con 0x7f742c0b23d0 2023-12-20T04:29:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f74137fe640 1 -- 172.21.15.32:0/2127352500 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f741c00ca00 con 0x7f742c0b23d0 2023-12-20T04:29:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f743168c640 1 -- 172.21.15.32:0/2127352500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f742c15f450 con 0x7f742c0b23d0 2023-12-20T04:29:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f74137fe640 1 -- 172.21.15.32:0/2127352500 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f741c00cd70 con 0x7f742c0b23d0 2023-12-20T04:29:27.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f743168c640 1 -- 172.21.15.32:0/2127352500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f742c0b3530 con 0x7f742c0b23d0 2023-12-20T04:29:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.434+0000 7f743168c640 1 -- 172.21.15.32:0/2127352500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f742c04e000 con 0x7f742c0b23d0 2023-12-20T04:29:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.438+0000 7f74137fe640 1 -- 172.21.15.32:0/2127352500 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f741c013470 con 0x7f742c0b23d0 2023-12-20T04:29:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.438+0000 7f74137fe640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7418065a60 0x7f7418067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.438+0000 7f74137fe640 1 -- 172.21.15.32:0/2127352500 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f741c0830b0 con 0x7f742c0b23d0 2023-12-20T04:29:27.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.438+0000 7f742a7fc640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7418065a60 0x7f7418067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.438+0000 7f742a7fc640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7418065a60 0x7f7418067f20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f742c1f8610 tx=0x7f7420009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:27.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.442+0000 7f74137fe640 1 -- 172.21.15.32:0/2127352500 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f741c050e90 con 0x7f742c0b23d0 2023-12-20T04:29:27.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.598+0000 7f743168c640 1 -- 172.21.15.32:0/2127352500 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f742c0b68d0 con 0x7f742c0b23d0 2023-12-20T04:29:27.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f74137fe640 1 -- 172.21.15.32:0/2127352500 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v31) v1 ==== 74+0+12341 (secure 0 0 0) 0x7f741c0544b0 con 0x7f742c0b23d0 2023-12-20T04:29:27.604 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:27.604 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:29:24.436745+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":22,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":19,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 -- 172.21.15.32:0/2127352500 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7418065a60 msgr2=0x7f7418067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7418065a60 0x7f7418067f20 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f742c1f8610 tx=0x7f7420009040 comp rx=0 tx=0).stop 2023-12-20T04:29:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 -- 172.21.15.32:0/2127352500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f742c0b23d0 msgr2=0x7f742c159ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f742c0b23d0 0x7f742c159ff0 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f741c00e930 tx=0x7f741c00edf0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 -- 172.21.15.32:0/2127352500 shutdown_connections 2023-12-20T04:29:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7418065a60 0x7f7418067f20 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f742c160ff0 0x7f742c15e4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f742c15cd70 0x7f742c15c190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 --2- 172.21.15.32:0/2127352500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f742c0b23d0 0x7f742c159ff0 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 -- 172.21.15.32:0/2127352500 >> 172.21.15.32:0/2127352500 conn(0x7f742c0cd3d0 msgr2=0x7f742c0cc480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:27.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 -- 172.21.15.32:0/2127352500 shutdown_connections 2023-12-20T04:29:27.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.602+0000 7f743168c640 1 -- 172.21.15.32:0/2127352500 wait complete. 2023-12-20T04:29:27.625 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:29:27.626 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:29:27.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa77ab7b640 1 Processor -- start 2023-12-20T04:29:27.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa77ab7b640 1 -- start start 2023-12-20T04:29:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa77ab7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa77415ce50 0x7fa77406b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa77ab7b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7740b0550 con 0x7fa77406bb80 2023-12-20T04:29:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa77ab7b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa77406bfc0 con 0x7fa7740cf7a0 2023-12-20T04:29:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa77ab7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740aff20 con 0x7fa77415ce50 2023-12-20T04:29:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa773fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa77406bb80 0x7fa7740cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52682/0 (socket says 172.21.15.32:52682) 2023-12-20T04:29:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa773fff640 1 -- 172.21.15.32:0/4163359190 learned_addr learned my addr 172.21.15.32:0/4163359190 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa778917640 1 --2- 172.21.15.32:0/4163359190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa77415ce50 0x7fa77406b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa7737fe640 1 -- 172.21.15.32:0/4163359190 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1039791525 0 0) 0x7fa7740b0550 con 0x7fa77406bb80 2023-12-20T04:29:27.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa7737fe640 1 -- 172.21.15.32:0/4163359190 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa764003530 con 0x7fa77406bb80 2023-12-20T04:29:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa7737fe640 1 -- 172.21.15.32:0/4163359190 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2842146880 0 0) 0x7fa77406bfc0 con 0x7fa7740cf7a0 2023-12-20T04:29:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa7737fe640 1 -- 172.21.15.32:0/4163359190 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7740b0550 con 0x7fa7740cf7a0 2023-12-20T04:29:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa778917640 1 -- 172.21.15.32:0/4163359190 >> v1:172.21.15.39:6789/0 conn(0x7fa7740cf7a0 legacy=0x7fa7740b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa778917640 1 -- 172.21.15.32:0/4163359190 >> v1:172.21.15.32:6789/0 conn(0x7fa77406bb80 legacy=0x7fa7740cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.770+0000 7fa778917640 1 -- 172.21.15.32:0/4163359190 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7740cfdc0 con 0x7fa77415ce50 2023-12-20T04:29:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.774+0000 7fa778917640 1 --2- 172.21.15.32:0/4163359190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa77415ce50 0x7fa77406b640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa7680097f0 tx=0x7fa76801c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e9b3216a37f553b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.774+0000 7fa7737fe640 1 -- 172.21.15.32:0/4163359190 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680ac600 con 0x7fa77415ce50 2023-12-20T04:29:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.774+0000 7fa7737fe640 1 -- 172.21.15.32:0/4163359190 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7680a8050 con 0x7fa77415ce50 2023-12-20T04:29:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.774+0000 7fa7737fe640 1 -- 172.21.15.32:0/4163359190 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680ac960 con 0x7fa77415ce50 2023-12-20T04:29:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.774+0000 7fa77ab7b640 1 -- 172.21.15.32:0/4163359190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa77415ce50 msgr2=0x7fa77406b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.774+0000 7fa77ab7b640 1 --2- 172.21.15.32:0/4163359190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa77415ce50 0x7fa77406b640 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7fa7680097f0 tx=0x7fa76801c490 comp rx=0 tx=0).stop 2023-12-20T04:29:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.774+0000 7fa77ab7b640 1 -- 172.21.15.32:0/4163359190 shutdown_connections 2023-12-20T04:29:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.774+0000 7fa77ab7b640 1 --2- 172.21.15.32:0/4163359190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa77415ce50 0x7fa77406b640 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.774+0000 7fa77ab7b640 1 -- 172.21.15.32:0/4163359190 >> 172.21.15.32:0/4163359190 conn(0x7fa774099d70 msgr2=0x7fa7740d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.774+0000 7fa77ab7b640 1 -- 172.21.15.32:0/4163359190 shutdown_connections 2023-12-20T04:29:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.774+0000 7fa77ab7b640 1 -- 172.21.15.32:0/4163359190 wait complete. 2023-12-20T04:29:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa77ab7b640 1 Processor -- start 2023-12-20T04:29:27.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa77ab7b640 1 -- start start 2023-12-20T04:29:27.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa77ab7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77406bb80 0x7fa77414e210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa77ab7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7740cf7a0 0x7fa77406f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa77ab7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77415ce50 0x7fa77406fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa77ab7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740c82f0 con 0x7fa77406bb80 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa77ab7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740c85d0 con 0x7fa77415ce50 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa77ab7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7740c88b0 con 0x7fa7740cf7a0 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa778917640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77406bb80 0x7fa77414e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa778917640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77406bb80 0x7fa77414e210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38154/0 (socket says 172.21.15.32:38154) 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa778917640 1 -- 172.21.15.32:0/180211266 learned_addr learned my addr 172.21.15.32:0/180211266 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa773fff640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7740cf7a0 0x7fa77406f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa779118640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77415ce50 0x7fa77406fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa778917640 1 -- 172.21.15.32:0/180211266 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7740cf7a0 msgr2=0x7fa77406f680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa778917640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7740cf7a0 0x7fa77406f680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa778917640 1 -- 172.21.15.32:0/180211266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77415ce50 msgr2=0x7fa77406fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa778917640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77415ce50 0x7fa77406fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa778917640 1 -- 172.21.15.32:0/180211266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa768009420 con 0x7fa77406bb80 2023-12-20T04:29:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa779118640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77415ce50 0x7fa77406fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa778917640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77406bb80 0x7fa77414e210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa7680097f0 tx=0x7fa768019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa7717fa640 1 -- 172.21.15.32:0/180211266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa768006e20 con 0x7fa77406bb80 2023-12-20T04:29:27.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa7717fa640 1 -- 172.21.15.32:0/180211266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7680b5400 con 0x7fa77406bb80 2023-12-20T04:29:27.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa7717fa640 1 -- 172.21.15.32:0/180211266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7680b5770 con 0x7fa77406bb80 2023-12-20T04:29:27.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.778+0000 7fa77ab7b640 1 -- 172.21.15.32:0/180211266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7740c8b90 con 0x7fa77406bb80 2023-12-20T04:29:27.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.782+0000 7fa77ab7b640 1 -- 172.21.15.32:0/180211266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa77415d3b0 con 0x7fa77406bb80 2023-12-20T04:29:27.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.782+0000 7fa77ab7b640 1 -- 172.21.15.32:0/180211266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa77404e000 con 0x7fa77406bb80 2023-12-20T04:29:27.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.782+0000 7fa7717fa640 1 -- 172.21.15.32:0/180211266 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa7680b3040 con 0x7fa77406bb80 2023-12-20T04:29:27.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.782+0000 7fa7717fa640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa758065a90 0x7fa758067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:27.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.782+0000 7fa7717fa640 1 -- 172.21.15.32:0/180211266 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fa76811c640 con 0x7fa77406bb80 2023-12-20T04:29:27.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.782+0000 7fa773fff640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa758065a90 0x7fa758067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:27.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.782+0000 7fa773fff640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa758065a90 0x7fa758067f50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa7740b6a50 tx=0x7fa760007d80 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:27.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.786+0000 7fa7717fa640 1 -- 172.21.15.32:0/180211266 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa7680ea340 con 0x7fa77406bb80 2023-12-20T04:29:27.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 -- 172.21.15.32:0/180211266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa7740c89f0 con 0x7fa77406bb80 2023-12-20T04:29:27.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa7717fa640 1 -- 172.21.15.32:0/180211266 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v31) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7740c89f0 con 0x7fa77406bb80 2023-12-20T04:29:27.948 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:29:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 -- 172.21.15.32:0/180211266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa758065a90 msgr2=0x7fa758067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa758065a90 0x7fa758067f50 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa7740b6a50 tx=0x7fa760007d80 comp rx=0 tx=0).stop 2023-12-20T04:29:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 -- 172.21.15.32:0/180211266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77406bb80 msgr2=0x7fa77414e210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77406bb80 0x7fa77414e210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7fa7680097f0 tx=0x7fa768019230 comp rx=0 tx=0).stop 2023-12-20T04:29:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 -- 172.21.15.32:0/180211266 shutdown_connections 2023-12-20T04:29:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa758065a90 0x7fa758067f50 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa77415ce50 0x7fa77406fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7740cf7a0 0x7fa77406f680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 --2- 172.21.15.32:0/180211266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa77406bb80 0x7fa77414e210 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 -- 172.21.15.32:0/180211266 >> 172.21.15.32:0/180211266 conn(0x7fa774099d70 msgr2=0x7fa7740ae740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 -- 172.21.15.32:0/180211266 shutdown_connections 2023-12-20T04:29:27.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:27.946+0000 7fa77ab7b640 1 -- 172.21.15.32:0/180211266 wait complete. 2023-12-20T04:29:27.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T04:29:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb5882f9640 1 Processor -- start 2023-12-20T04:29:28.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb5882f9640 1 -- start start 2023-12-20T04:29:28.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb5882f9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb58015cdb0 0x7fb5800d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb5882f9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5800cfc80 0x7fb5800b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb5882f9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb58015d190 con 0x7fb5800d2740 2023-12-20T04:29:28.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb5882f9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5800d16a0 con 0x7fb5800cfc80 2023-12-20T04:29:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb5882f9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5800d1980 con 0x7fb58015cdb0 2023-12-20T04:29:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb586896640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5800cfc80 0x7fb5800b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb586095640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb58015cdb0 0x7fb5800d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb586896640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5800cfc80 0x7fb5800b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38162/0 (socket says 172.21.15.32:38162) 2023-12-20T04:29:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb586896640 1 -- 172.21.15.32:0/2054785602 learned_addr learned my addr 172.21.15.32:0/2054785602 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb586896640 1 -- 172.21.15.32:0/2054785602 >> v2:172.21.15.39:3300/0 conn(0x7fb58015cdb0 msgr2=0x7fb5800d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb586896640 1 --2- 172.21.15.32:0/2054785602 >> v2:172.21.15.39:3300/0 conn(0x7fb58015cdb0 0x7fb5800d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb586896640 1 -- 172.21.15.32:0/2054785602 >> v1:172.21.15.39:6789/0 conn(0x7fb5800d2740 legacy=0x7fb5800cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:28.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.110+0000 7fb586896640 1 -- 172.21.15.32:0/2054785602 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5800d1c60 con 0x7fb5800cfc80 2023-12-20T04:29:28.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb586896640 1 --2- 172.21.15.32:0/2054785602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5800cfc80 0x7fb5800b5c90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb5800667b0 tx=0x7fb57801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45d3d7af969fcaf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:28.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb585093640 1 -- 172.21.15.32:0/2054785602 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5780af400 con 0x7fb5800cfc80 2023-12-20T04:29:28.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb585093640 1 -- 172.21.15.32:0/2054785602 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5780ab050 con 0x7fb5800cfc80 2023-12-20T04:29:28.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb585093640 1 -- 172.21.15.32:0/2054785602 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5780af780 con 0x7fb5800cfc80 2023-12-20T04:29:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb5882f9640 1 -- 172.21.15.32:0/2054785602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5800cfc80 msgr2=0x7fb5800b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb5882f9640 1 --2- 172.21.15.32:0/2054785602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5800cfc80 0x7fb5800b5c90 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7fb5800667b0 tx=0x7fb57801f1c0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb5882f9640 1 -- 172.21.15.32:0/2054785602 shutdown_connections 2023-12-20T04:29:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb5882f9640 1 --2- 172.21.15.32:0/2054785602 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5800cfc80 0x7fb5800b5c90 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb5882f9640 1 --2- 172.21.15.32:0/2054785602 >> v2:172.21.15.39:3300/0 conn(0x7fb58015cdb0 0x7fb5800d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb5882f9640 1 -- 172.21.15.32:0/2054785602 >> 172.21.15.32:0/2054785602 conn(0x7fb580065ee0 msgr2=0x7fb58006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb5882f9640 1 -- 172.21.15.32:0/2054785602 shutdown_connections 2023-12-20T04:29:28.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.114+0000 7fb5882f9640 1 -- 172.21.15.32:0/2054785602 wait complete. 2023-12-20T04:29:28.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb5882f9640 1 Processor -- start 2023-12-20T04:29:28.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb5882f9640 1 -- start start 2023-12-20T04:29:28.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb5882f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5800cfc80 0x7fb58014e330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb5882f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5800d2740 0x7fb5801528f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb5882f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb58015cdb0 0x7fb580155210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb5882f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58015e750 con 0x7fb58015cdb0 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb5882f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb58015e400 con 0x7fb5800cfc80 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb5882f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5801535e0 con 0x7fb5800d2740 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb586896640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb58015cdb0 0x7fb580155210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb586896640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb58015cdb0 0x7fb580155210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38170/0 (socket says 172.21.15.32:38170) 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb586896640 1 -- 172.21.15.32:0/3235926038 learned_addr learned my addr 172.21.15.32:0/3235926038 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb586095640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5800cfc80 0x7fb58014e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb585894640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5800d2740 0x7fb5801528f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb586896640 1 -- 172.21.15.32:0/3235926038 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5800d2740 msgr2=0x7fb5801528f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb586896640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5800d2740 0x7fb5801528f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb586896640 1 -- 172.21.15.32:0/3235926038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5800cfc80 msgr2=0x7fb58014e330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb586896640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5800cfc80 0x7fb58014e330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb586896640 1 -- 172.21.15.32:0/3235926038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb578009b30 con 0x7fb58015cdb0 2023-12-20T04:29:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb586896640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb58015cdb0 0x7fb580155210 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb57801cdf0 tx=0x7fb57801bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb576ffd640 1 -- 172.21.15.32:0/3235926038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5780afc90 con 0x7fb58015cdb0 2023-12-20T04:29:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb576ffd640 1 -- 172.21.15.32:0/3235926038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5780afdf0 con 0x7fb58015cdb0 2023-12-20T04:29:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb5882f9640 1 -- 172.21.15.32:0/3235926038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5801538c0 con 0x7fb58015cdb0 2023-12-20T04:29:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb576ffd640 1 -- 172.21.15.32:0/3235926038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5780b9650 con 0x7fb58015cdb0 2023-12-20T04:29:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.118+0000 7fb5882f9640 1 -- 172.21.15.32:0/3235926038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb58014e870 con 0x7fb58015cdb0 2023-12-20T04:29:28.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.122+0000 7fb5882f9640 1 -- 172.21.15.32:0/3235926038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb58015e890 con 0x7fb58015cdb0 2023-12-20T04:29:28.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.122+0000 7fb576ffd640 1 -- 172.21.15.32:0/3235926038 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb5780b6070 con 0x7fb58015cdb0 2023-12-20T04:29:28.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.122+0000 7fb576ffd640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb564065a60 0x7fb564067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.122+0000 7fb576ffd640 1 -- 172.21.15.32:0/3235926038 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb578120c80 con 0x7fb58015cdb0 2023-12-20T04:29:28.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.122+0000 7fb586095640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb564065a60 0x7fb564067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.122+0000 7fb586095640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb564065a60 0x7fb564067f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009ac0 tx=0x7fb57c009190 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.126+0000 7fb576ffd640 1 -- 172.21.15.32:0/3235926038 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb58015e890 con 0x7fb58015cdb0 2023-12-20T04:29:28.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 -- 172.21.15.32:0/3235926038 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fb58015e890 con 0x7fb58015cdb0 2023-12-20T04:29:28.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb576ffd640 1 -- 172.21.15.32:0/3235926038 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v31) v1 ==== 84+0+65 (secure 0 0 0) 0x7fb58015e890 con 0x7fb58015cdb0 2023-12-20T04:29:28.293 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:28.293 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:29:28.293 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:28.294 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T04:29:28.294 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T04:29:28.294 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:28.294 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:29:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 -- 172.21.15.32:0/3235926038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb564065a60 msgr2=0x7fb564067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb564065a60 0x7fb564067f20 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fb57c009ac0 tx=0x7fb57c009190 comp rx=0 tx=0).stop 2023-12-20T04:29:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 -- 172.21.15.32:0/3235926038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb58015cdb0 msgr2=0x7fb580155210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb58015cdb0 0x7fb580155210 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fb57801cdf0 tx=0x7fb57801bf60 comp rx=0 tx=0).stop 2023-12-20T04:29:28.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 -- 172.21.15.32:0/3235926038 shutdown_connections 2023-12-20T04:29:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb564065a60 0x7fb564067f20 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb58015cdb0 0x7fb580155210 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5800d2740 0x7fb5801528f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 --2- 172.21.15.32:0/3235926038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5800cfc80 0x7fb58014e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 -- 172.21.15.32:0/3235926038 >> 172.21.15.32:0/3235926038 conn(0x7fb580065ee0 msgr2=0x7fb580158cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 -- 172.21.15.32:0/3235926038 shutdown_connections 2023-12-20T04:29:28.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.290+0000 7fb5882f9640 1 -- 172.21.15.32:0/3235926038 wait complete. 2023-12-20T04:29:28.314 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T04:29:28.314 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T04:29:28.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.458+0000 7fed1e76d640 1 Processor -- start 2023-12-20T04:29:28.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.458+0000 7fed1e76d640 1 -- start start 2023-12-20T04:29:28.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.458+0000 7fed1e76d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18159250 0x7fed180caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.458+0000 7fed1e76d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fed18070940 0x7fed180cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.458+0000 7fed1e76d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed18159630 con 0x7fed180ce290 2023-12-20T04:29:28.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.458+0000 7fed1e76d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed180c9480 con 0x7fed18070940 2023-12-20T04:29:28.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.458+0000 7fed1e76d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1806cf90 con 0x7fed18159250 2023-12-20T04:29:28.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.458+0000 7fed177fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fed180ce290 0x7fed18070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52694/0 (socket says 172.21.15.32:52694) 2023-12-20T04:29:28.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.458+0000 7fed177fe640 1 -- 172.21.15.32:0/2214699732 learned_addr learned my addr 172.21.15.32:0/2214699732 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.458+0000 7fed17fff640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18159250 0x7fed180caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.458+0000 7fed1cd0a640 1 --2- 172.21.15.32:0/2214699732 >> v2:172.21.15.39:3300/0 conn(0x7fed18070940 0x7fed180cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1468671858 0 0) 0x7fed18159630 con 0x7fed180ce290 2023-12-20T04:29:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed00003530 con 0x7fed180ce290 2023-12-20T04:29:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2845174240 0 0) 0x7fed00003530 con 0x7fed180ce290 2023-12-20T04:29:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18159250 msgr2=0x7fed180caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:28.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18159250 0x7fed180caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 >> v2:172.21.15.39:3300/0 conn(0x7fed18070940 msgr2=0x7fed180cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 --2- 172.21.15.32:0/2214699732 >> v2:172.21.15.39:3300/0 conn(0x7fed18070940 0x7fed180cc5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1806d270 con 0x7fed180ce290 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fed0c001680 con 0x7fed180ce290 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 >> v1:172.21.15.32:6789/0 conn(0x7fed180ce290 legacy=0x7fed18070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed00003980 0x7fed00003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18159250 0x7fed000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed00004d10 0x7fed00005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed180c9480 con 0x7fed00004d10 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1806cf90 con 0x7fed00003980 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed0000dba0 con 0x7fed18159250 2023-12-20T04:29:28.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed17fff640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed00003980 0x7fed00003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed1cd0a640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed00004d10 0x7fed00005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed177fe640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18159250 0x7fed000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed1cd0a640 1 -- 172.21.15.32:0/2214699732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18159250 msgr2=0x7fed000047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:28.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed1cd0a640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18159250 0x7fed000047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed1cd0a640 1 -- 172.21.15.32:0/2214699732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed00003980 msgr2=0x7fed00003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:28.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed1cd0a640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed00003980 0x7fed00003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.462+0000 7fed1cd0a640 1 -- 172.21.15.32:0/2214699732 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed00003530 con 0x7fed00004d10 2023-12-20T04:29:28.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1cd0a640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed00004d10 0x7fed00005190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fed18070010 tx=0x7fed1001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a0242bdace5b40 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed100a5c50 con 0x7fed00004d10 2023-12-20T04:29:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed1000f040 con 0x7fed00004d10 2023-12-20T04:29:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed16ffd640 1 -- 172.21.15.32:0/2214699732 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed100ad440 con 0x7fed00004d10 2023-12-20T04:29:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1e76d640 1 -- 172.21.15.32:0/2214699732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed00004d10 msgr2=0x7fed00005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1e76d640 1 --2- 172.21.15.32:0/2214699732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed00004d10 0x7fed00005190 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fed18070010 tx=0x7fed1001cc40 comp rx=0 tx=0).stop 2023-12-20T04:29:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1cd0a640 1 -- 172.21.15.32:0/2214699732 reap_dead start 2023-12-20T04:29:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1e76d640 1 -- 172.21.15.32:0/2214699732 shutdown_connections 2023-12-20T04:29:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1e76d640 1 -- 172.21.15.32:0/2214699732 >> 172.21.15.32:0/2214699732 conn(0x7fed1806a510 msgr2=0x7fed180ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1e76d640 1 -- 172.21.15.32:0/2214699732 shutdown_connections 2023-12-20T04:29:28.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1e76d640 1 -- 172.21.15.32:0/2214699732 wait complete. 2023-12-20T04:29:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1e76d640 1 Processor -- start 2023-12-20T04:29:28.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1e76d640 1 -- start start 2023-12-20T04:29:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1e76d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed00004d10 0x7fed180c6a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.466+0000 7fed1e76d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18152050 0x7fed18153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1e76d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed181524d0 0x7fed18154330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1e76d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed18154970 con 0x7fed181524d0 2023-12-20T04:29:28.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1e76d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed1814f770 con 0x7fed00004d10 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1e76d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed1814fa50 con 0x7fed18152050 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1cd0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed181524d0 0x7fed18154330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1cd0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed181524d0 0x7fed18154330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38188/0 (socket says 172.21.15.32:38188) 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1cd0a640 1 -- 172.21.15.32:0/539995934 learned_addr learned my addr 172.21.15.32:0/539995934 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed177fe640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18152050 0x7fed18153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed17fff640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed00004d10 0x7fed180c6a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1cd0a640 1 -- 172.21.15.32:0/539995934 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18152050 msgr2=0x7fed18153df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1cd0a640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18152050 0x7fed18153df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1cd0a640 1 -- 172.21.15.32:0/539995934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed00004d10 msgr2=0x7fed180c6a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1cd0a640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed00004d10 0x7fed180c6a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1cd0a640 1 -- 172.21.15.32:0/539995934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed1814fd30 con 0x7fed181524d0 2023-12-20T04:29:28.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1cd0a640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed181524d0 0x7fed18154330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fed100b0040 tx=0x7fed10009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed14ff9640 1 -- 172.21.15.32:0/539995934 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed100039a0 con 0x7fed181524d0 2023-12-20T04:29:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed14ff9640 1 -- 172.21.15.32:0/539995934 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed10003b20 con 0x7fed181524d0 2023-12-20T04:29:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1e76d640 1 -- 172.21.15.32:0/539995934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed18150010 con 0x7fed181524d0 2023-12-20T04:29:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed14ff9640 1 -- 172.21.15.32:0/539995934 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed100ae890 con 0x7fed181524d0 2023-12-20T04:29:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1e76d640 1 -- 172.21.15.32:0/539995934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed18150530 con 0x7fed181524d0 2023-12-20T04:29:28.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed1e76d640 1 -- 172.21.15.32:0/539995934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed1804e000 con 0x7fed181524d0 2023-12-20T04:29:28.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.470+0000 7fed14ff9640 1 -- 172.21.15.32:0/539995934 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fed100b4040 con 0x7fed181524d0 2023-12-20T04:29:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.474+0000 7fed14ff9640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed04065a60 0x7fed04067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.474+0000 7fed14ff9640 1 -- 172.21.15.32:0/539995934 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(31..31 src has 1..31) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fed1011ce30 con 0x7fed181524d0 2023-12-20T04:29:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.474+0000 7fed17fff640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed04065a60 0x7fed04067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:28.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.474+0000 7fed17fff640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed04065a60 0x7fed04067f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fed180c3aa0 tx=0x7fed08008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:28.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.478+0000 7fed14ff9640 1 -- 172.21.15.32:0/539995934 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fed100eaa10 con 0x7fed181524d0 2023-12-20T04:29:28.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:28.634+0000 7fed1e76d640 1 -- 172.21.15.32:0/539995934 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fed18154ab0 con 0x7fed181524d0 2023-12-20T04:29:29.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:29:29.106 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:29:29.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.194+0000 7fed14ff9640 1 -- 172.21.15.32:0/539995934 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v32) v1 ==== 138+0+0 (secure 0 0 0) 0x7fed100b2020 con 0x7fed181524d0 2023-12-20T04:29:29.197 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T04:29:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.198+0000 7fed1e76d640 1 -- 172.21.15.32:0/539995934 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed04065a60 msgr2=0x7fed04067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.198+0000 7fed1e76d640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed04065a60 0x7fed04067f20 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fed180c3aa0 tx=0x7fed08008040 comp rx=0 tx=0).stop 2023-12-20T04:29:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.198+0000 7fed1e76d640 1 -- 172.21.15.32:0/539995934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed181524d0 msgr2=0x7fed18154330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.198+0000 7fed1e76d640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed181524d0 0x7fed18154330 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7fed100b0040 tx=0x7fed10009750 comp rx=0 tx=0).stop 2023-12-20T04:29:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.198+0000 7fed1e76d640 1 -- 172.21.15.32:0/539995934 shutdown_connections 2023-12-20T04:29:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.198+0000 7fed1e76d640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed04065a60 0x7fed04067f20 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.198+0000 7fed1e76d640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed181524d0 0x7fed18154330 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.198+0000 7fed1e76d640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed18152050 0x7fed18153df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.198+0000 7fed1e76d640 1 --2- 172.21.15.32:0/539995934 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed00004d10 0x7fed180c6a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.198+0000 7fed1e76d640 1 -- 172.21.15.32:0/539995934 >> 172.21.15.32:0/539995934 conn(0x7fed1806a510 msgr2=0x7fed180c4130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.198+0000 7fed1e76d640 1 -- 172.21.15.32:0/539995934 shutdown_connections 2023-12-20T04:29:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.202+0000 7fed1e76d640 1 -- 172.21.15.32:0/539995934 wait complete. 2023-12-20T04:29:29.227 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T04:29:29.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.418+0000 7fb4b5055640 1 Processor -- start 2023-12-20T04:29:29.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.418+0000 7fb4b5055640 1 -- start start 2023-12-20T04:29:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4b5055640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4b015a940 0x7fb4b00cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4b5055640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4b015ad20 0x7fb4b00ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4b5055640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4b00cc9d0 con 0x7fb4b00cb1c0 2023-12-20T04:29:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4b5055640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b00c9b90 con 0x7fb4b015a940 2023-12-20T04:29:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4b5055640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4b00c9e70 con 0x7fb4b015ad20 2023-12-20T04:29:29.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4aed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4b015a940 0x7fb4b00cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:29.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4aed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb4b015a940 0x7fb4b00cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45500/0 (socket says 172.21.15.32:45500) 2023-12-20T04:29:29.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4aed9d640 1 -- 172.21.15.32:0/222925775 learned_addr learned my addr 172.21.15.32:0/222925775 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:29.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4a7fff640 1 --2- 172.21.15.32:0/222925775 >> v2:172.21.15.39:3300/0 conn(0x7fb4b015ad20 0x7fb4b00ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:29.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4aed9d640 1 -- 172.21.15.32:0/222925775 >> v2:172.21.15.39:3300/0 conn(0x7fb4b015ad20 msgr2=0x7fb4b00ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:29.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4aed9d640 1 --2- 172.21.15.32:0/222925775 >> v2:172.21.15.39:3300/0 conn(0x7fb4b015ad20 0x7fb4b00ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:29.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4aed9d640 1 -- 172.21.15.32:0/222925775 >> v1:172.21.15.39:6789/0 conn(0x7fb4b00cb1c0 legacy=0x7fb4b0070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:29.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4aed9d640 1 -- 172.21.15.32:0/222925775 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4b00ca150 con 0x7fb4b015a940 2023-12-20T04:29:29.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.422+0000 7fb4a7fff640 1 --2- 172.21.15.32:0/222925775 >> v2:172.21.15.39:3300/0 conn(0x7fb4b015ad20 0x7fb4b00ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4aed9d640 1 --2- 172.21.15.32:0/222925775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015a940 0x7fb4b00cc380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00ce460 tx=0x7fb4a001cef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36710db616b2d3c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4ae59c640 1 -- 172.21.15.32:0/222925775 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00aa610 con 0x7fb4b015a940 2023-12-20T04:29:29.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4ae59c640 1 -- 172.21.15.32:0/222925775 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a00a6040 con 0x7fb4b015a940 2023-12-20T04:29:29.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4ae59c640 1 -- 172.21.15.32:0/222925775 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a00aa990 con 0x7fb4b015a940 2023-12-20T04:29:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4b5055640 1 -- 172.21.15.32:0/222925775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015a940 msgr2=0x7fb4b00cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4b5055640 1 --2- 172.21.15.32:0/222925775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015a940 0x7fb4b00cc380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fb4b00ce460 tx=0x7fb4a001cef0 comp rx=0 tx=0).stop 2023-12-20T04:29:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4b5055640 1 -- 172.21.15.32:0/222925775 shutdown_connections 2023-12-20T04:29:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4b5055640 1 --2- 172.21.15.32:0/222925775 >> v2:172.21.15.39:3300/0 conn(0x7fb4b015ad20 0x7fb4b00ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:29.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4b5055640 1 --2- 172.21.15.32:0/222925775 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015a940 0x7fb4b00cc380 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:29.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4b5055640 1 -- 172.21.15.32:0/222925775 >> 172.21.15.32:0/222925775 conn(0x7fb4b006a4b0 msgr2=0x7fb4b00b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:29.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4b5055640 1 -- 172.21.15.32:0/222925775 shutdown_connections 2023-12-20T04:29:29.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4b5055640 1 -- 172.21.15.32:0/222925775 wait complete. 2023-12-20T04:29:29.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4b5055640 1 Processor -- start 2023-12-20T04:29:29.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.426+0000 7fb4b5055640 1 -- start start 2023-12-20T04:29:29.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4b5055640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4b00cb1c0 0x7fb4b00c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:29.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4b5055640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4b015a940 0x7fb4b0159160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:29.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4b5055640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015ad20 0x7fb4b014d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4b5055640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c5dc0 con 0x7fb4b015ad20 2023-12-20T04:29:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4b5055640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c6060 con 0x7fb4b015a940 2023-12-20T04:29:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4b5055640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4b00c6340 con 0x7fb4b00cb1c0 2023-12-20T04:29:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4af59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015ad20 0x7fb4b014d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4af59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015ad20 0x7fb4b014d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45502/0 (socket says 172.21.15.32:45502) 2023-12-20T04:29:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4af59e640 1 -- 172.21.15.32:0/523995862 learned_addr learned my addr 172.21.15.32:0/523995862 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4a7fff640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4b015a940 0x7fb4b0159160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4aed9d640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4b00cb1c0 0x7fb4b00c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:29.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4af59e640 1 -- 172.21.15.32:0/523995862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4b00cb1c0 msgr2=0x7fb4b00c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:29.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4af59e640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4b00cb1c0 0x7fb4b00c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:29.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4af59e640 1 -- 172.21.15.32:0/523995862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4b015a940 msgr2=0x7fb4b0159160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:29.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4af59e640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4b015a940 0x7fb4b0159160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:29.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4af59e640 1 -- 172.21.15.32:0/523995862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4a00094d0 con 0x7fb4b015ad20 2023-12-20T04:29:29.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4af59e640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015ad20 0x7fb4b014d1e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800c780 tx=0x7fb4a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:29.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4a77fe640 1 -- 172.21.15.32:0/523995862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a8004b70 con 0x7fb4b015ad20 2023-12-20T04:29:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4a77fe640 1 -- 172.21.15.32:0/523995862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4a800ce90 con 0x7fb4b015ad20 2023-12-20T04:29:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4b5055640 1 -- 172.21.15.32:0/523995862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4b00c6620 con 0x7fb4b015ad20 2023-12-20T04:29:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4a77fe640 1 -- 172.21.15.32:0/523995862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4a800a850 con 0x7fb4b015ad20 2023-12-20T04:29:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4b5055640 1 -- 172.21.15.32:0/523995862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4b015bdf0 con 0x7fb4b015ad20 2023-12-20T04:29:29.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.430+0000 7fb4b5055640 1 -- 172.21.15.32:0/523995862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4b015c010 con 0x7fb4b015ad20 2023-12-20T04:29:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.434+0000 7fb4a77fe640 1 -- 172.21.15.32:0/523995862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb4a800a9b0 con 0x7fb4b015ad20 2023-12-20T04:29:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.434+0000 7fb4a77fe640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb490065a60 0x7fb490067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.434+0000 7fb4a77fe640 1 -- 172.21.15.32:0/523995862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(32..32 src has 1..32) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fb4a80830e0 con 0x7fb4b015ad20 2023-12-20T04:29:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.434+0000 7fb4aed9d640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb490065a60 0x7fb490067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.434+0000 7fb4aed9d640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb490065a60 0x7fb490067f20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00ae000 tx=0x7fb4a00b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:29.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.438+0000 7fb4a77fe640 1 -- 172.21.15.32:0/523995862 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb4a8050c90 con 0x7fb4b015ad20 2023-12-20T04:29:29.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:29.598+0000 7fb4b5055640 1 -- 172.21.15.32:0/523995862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb4b004e000 con 0x7fb4b015ad20 2023-12-20T04:29:30.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.198+0000 7fb4a77fe640 1 -- 172.21.15.32:0/523995862 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v33) v1 ==== 170+0+0 (secure 0 0 0) 0x7fb4a80542b0 con 0x7fb4b015ad20 2023-12-20T04:29:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T04:29:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.210+0000 7fb4b5055640 1 -- 172.21.15.32:0/523995862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb490065a60 msgr2=0x7fb490067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.210+0000 7fb4b5055640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb490065a60 0x7fb490067f20 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fb4a00ae000 tx=0x7fb4a00b5040 comp rx=0 tx=0).stop 2023-12-20T04:29:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.210+0000 7fb4b5055640 1 -- 172.21.15.32:0/523995862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015ad20 msgr2=0x7fb4b014d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.210+0000 7fb4b5055640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015ad20 0x7fb4b014d1e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fb4a800c780 tx=0x7fb4a800cc40 comp rx=0 tx=0).stop 2023-12-20T04:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.210+0000 7fb4b5055640 1 -- 172.21.15.32:0/523995862 shutdown_connections 2023-12-20T04:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.210+0000 7fb4b5055640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb490065a60 0x7fb490067f20 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.210+0000 7fb4b5055640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4b015ad20 0x7fb4b014d1e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.210+0000 7fb4b5055640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4b015a940 0x7fb4b0159160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.210+0000 7fb4b5055640 1 --2- 172.21.15.32:0/523995862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4b00cb1c0 0x7fb4b00c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.210+0000 7fb4b5055640 1 -- 172.21.15.32:0/523995862 >> 172.21.15.32:0/523995862 conn(0x7fb4b006a4b0 msgr2=0x7fb4b00ad140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.214+0000 7fb4b5055640 1 -- 172.21.15.32:0/523995862 shutdown_connections 2023-12-20T04:29:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.214+0000 7fb4b5055640 1 -- 172.21.15.32:0/523995862 wait complete. 2023-12-20T04:29:30.251 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T04:29:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13f5295640 1 Processor -- start 2023-12-20T04:29:30.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13f5295640 1 -- start start 2023-12-20T04:29:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13f5295640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f13f00cb050 0x7f13f00cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13f5295640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f13f0065d40 0x7f13f0161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13f5295640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f13f006a1f0 con 0x7f13f006ca60 2023-12-20T04:29:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13f5295640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f13f0161860 con 0x7f13f00cb050 2023-12-20T04:29:30.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13f5295640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f006a440 con 0x7f13f0065d40 2023-12-20T04:29:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13eeffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f13f00cb050 0x7f13f00cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13e7fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f13f0065d40 0x7f13f0161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13eeffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f13f00cb050 0x7f13f00cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48470/0 (socket says 172.21.15.32:48470) 2023-12-20T04:29:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13eeffd640 1 -- 172.21.15.32:0/3954510130 learned_addr learned my addr 172.21.15.32:0/3954510130 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13ef7fe640 1 --1- 172.21.15.32:0/3954510130 >> v1:172.21.15.39:6789/0 conn(0x7f13f006ca60 0x7f13f0070960 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:34236/0 (socket says 172.21.15.32:34236) 2023-12-20T04:29:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13e7fff640 1 -- 172.21.15.32:0/3954510130 >> v2:172.21.15.39:3300/0 conn(0x7f13f00cb050 msgr2=0x7f13f00cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:30.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13e7fff640 1 --2- 172.21.15.32:0/3954510130 >> v2:172.21.15.39:3300/0 conn(0x7f13f00cb050 0x7f13f00cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13e7fff640 1 -- 172.21.15.32:0/3954510130 >> v1:172.21.15.39:6789/0 conn(0x7f13f006ca60 legacy=0x7f13f0070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.398+0000 7f13e7fff640 1 -- 172.21.15.32:0/3954510130 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13f006a720 con 0x7f13f0065d40 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13e7fff640 1 --2- 172.21.15.32:0/3954510130 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f13f0065d40 0x7f13f0161320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f13d80097c0 tx=0x7f13d801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f20e31f2bbad93ab server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13ee7fc640 1 -- 172.21.15.32:0/3954510130 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d80ab650 con 0x7f13f0065d40 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13ee7fc640 1 -- 172.21.15.32:0/3954510130 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13d80ab7b0 con 0x7f13f0065d40 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13ee7fc640 1 -- 172.21.15.32:0/3954510130 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13d80abb40 con 0x7f13f0065d40 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13f5295640 1 -- 172.21.15.32:0/3954510130 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f13f0065d40 msgr2=0x7f13f0161320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13f5295640 1 --2- 172.21.15.32:0/3954510130 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f13f0065d40 0x7f13f0161320 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f13d80097c0 tx=0x7f13d801c450 comp rx=0 tx=0).stop 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13f5295640 1 -- 172.21.15.32:0/3954510130 shutdown_connections 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13f5295640 1 --2- 172.21.15.32:0/3954510130 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f13f0065d40 0x7f13f0161320 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13f5295640 1 --2- 172.21.15.32:0/3954510130 >> v2:172.21.15.39:3300/0 conn(0x7f13f00cb050 0x7f13f00cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13f5295640 1 -- 172.21.15.32:0/3954510130 >> 172.21.15.32:0/3954510130 conn(0x7f13f00cf8d0 msgr2=0x7f13f00cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13f5295640 1 -- 172.21.15.32:0/3954510130 shutdown_connections 2023-12-20T04:29:30.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.402+0000 7f13f5295640 1 -- 172.21.15.32:0/3954510130 wait complete. 2023-12-20T04:29:30.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13f5295640 1 Processor -- start 2023-12-20T04:29:30.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13f5295640 1 -- start start 2023-12-20T04:29:30.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13f5295640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13f0065d40 0x7f13f00c7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13f5295640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f13f006ca60 0x7f13f00c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13f5295640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f00cb050 0x7f13f0154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13f5295640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0153cc0 con 0x7f13f00cb050 2023-12-20T04:29:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13f5295640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f13f00c8480 con 0x7f13f0065d40 2023-12-20T04:29:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13f5295640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f13f0153990 con 0x7f13f006ca60 2023-12-20T04:29:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13ef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f00cb050 0x7f13f0154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13eeffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13f0065d40 0x7f13f00c7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13ef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f00cb050 0x7f13f0154150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45516/0 (socket says 172.21.15.32:45516) 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13ef7fe640 1 -- 172.21.15.32:0/3892287449 learned_addr learned my addr 172.21.15.32:0/3892287449 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13e7fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f13f006ca60 0x7f13f00c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13eeffd640 1 -- 172.21.15.32:0/3892287449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f13f006ca60 msgr2=0x7f13f00c5810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13eeffd640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f13f006ca60 0x7f13f00c5810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13eeffd640 1 -- 172.21.15.32:0/3892287449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f00cb050 msgr2=0x7f13f0154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13eeffd640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f00cb050 0x7f13f0154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13eeffd640 1 -- 172.21.15.32:0/3892287449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f13d8009420 con 0x7f13f0065d40 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13ef7fe640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f00cb050 0x7f13f0154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13eeffd640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13f0065d40 0x7f13f00c7bd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13e000e870 tx=0x7f13e000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13e77fe640 1 -- 172.21.15.32:0/3892287449 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e000c940 con 0x7f13f0065d40 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13e77fe640 1 -- 172.21.15.32:0/3892287449 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f13e000caa0 con 0x7f13f0065d40 2023-12-20T04:29:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13e77fe640 1 -- 172.21.15.32:0/3892287449 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f13e000ce30 con 0x7f13f0065d40 2023-12-20T04:29:30.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13f5295640 1 -- 172.21.15.32:0/3892287449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f13f00c8720 con 0x7f13f0065d40 2023-12-20T04:29:30.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.406+0000 7f13f5295640 1 -- 172.21.15.32:0/3892287449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f13f00c8110 con 0x7f13f0065d40 2023-12-20T04:29:30.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.410+0000 7f13f5295640 1 -- 172.21.15.32:0/3892287449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f13f004e000 con 0x7f13f0065d40 2023-12-20T04:29:30.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.410+0000 7f13e77fe640 1 -- 172.21.15.32:0/3892287449 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f13e0013470 con 0x7f13f0065d40 2023-12-20T04:29:30.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.410+0000 7f13e77fe640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f13d0065a90 0x7f13d0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.410+0000 7f13e77fe640 1 -- 172.21.15.32:0/3892287449 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(33..33 src has 1..33) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f13e0083270 con 0x7f13f0065d40 2023-12-20T04:29:30.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.410+0000 7f13e7fff640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f13d0065a90 0x7f13d0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.410+0000 7f13e7fff640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f13d0065a90 0x7f13d0067f50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f13d80af000 tx=0x7f13d80a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:30.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.414+0000 7f13e77fe640 1 -- 172.21.15.32:0/3892287449 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f13e0050ca0 con 0x7f13f0065d40 2023-12-20T04:29:30.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.606+0000 7f13f5295640 1 -- 172.21.15.32:0/3892287449 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f13f0065a20 con 0x7f13f0065d40 2023-12-20T04:29:30.627 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 4 and data pool 5 2023-12-20T04:29:30.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.622+0000 7f13e77fe640 1 -- 172.21.15.32:0/3892287449 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 4 and data pool 5 v12) v1 ==== 172+0+0 (secure 0 0 0) 0x7f13e00542c0 con 0x7f13f0065d40 2023-12-20T04:29:30.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.626+0000 7f13f5295640 1 -- 172.21.15.32:0/3892287449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f13d0065a90 msgr2=0x7f13d0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.626+0000 7f13f5295640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f13d0065a90 0x7f13d0067f50 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f13d80af000 tx=0x7f13d80a7000 comp rx=0 tx=0).stop 2023-12-20T04:29:30.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.626+0000 7f13f5295640 1 -- 172.21.15.32:0/3892287449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13f0065d40 msgr2=0x7f13f00c7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.626+0000 7f13f5295640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13f0065d40 0x7f13f00c7bd0 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f13e000e870 tx=0x7f13e000ed30 comp rx=0 tx=0).stop 2023-12-20T04:29:30.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.630+0000 7f13f5295640 1 -- 172.21.15.32:0/3892287449 shutdown_connections 2023-12-20T04:29:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.630+0000 7f13f5295640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f13d0065a90 0x7f13d0067f50 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.630+0000 7f13f5295640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f13f00cb050 0x7f13f0154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.630+0000 7f13f5295640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f13f006ca60 0x7f13f00c5810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.630+0000 7f13f5295640 1 --2- 172.21.15.32:0/3892287449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f13f0065d40 0x7f13f00c7bd0 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.630+0000 7f13f5295640 1 -- 172.21.15.32:0/3892287449 >> 172.21.15.32:0/3892287449 conn(0x7f13f00cf8d0 msgr2=0x7f13f0070960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.630+0000 7f13f5295640 1 -- 172.21.15.32:0/3892287449 shutdown_connections 2023-12-20T04:29:30.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.630+0000 7f13f5295640 1 -- 172.21.15.32:0/3892287449 wait complete. 2023-12-20T04:29:30.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:30.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc72dbc640 1 Processor -- start 2023-12-20T04:29:30.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc72dbc640 1 -- start start 2023-12-20T04:29:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc72dbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbc6c1599e0 0x7fbc6c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc72dbc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbc6c0c90e0 0x7fbc6c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc72dbc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbc6c0711d0 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc72dbc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6c0c98c0 con 0x7fbc6c1599e0 2023-12-20T04:29:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc72dbc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbc6c0c9ba0 con 0x7fbc6c0c90e0 2023-12-20T04:29:30.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc70b58640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbc6c1599e0 0x7fbc6c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc70b58640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbc6c1599e0 0x7fbc6c0aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45532/0 (socket says 172.21.15.32:45532) 2023-12-20T04:29:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc70b58640 1 -- 172.21.15.32:0/205356216 learned_addr learned my addr 172.21.15.32:0/205356216 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc6bfff640 1 --2- 172.21.15.32:0/205356216 >> v2:172.21.15.39:3300/0 conn(0x7fbc6c0c90e0 0x7fbc6c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc70b58640 1 -- 172.21.15.32:0/205356216 >> v2:172.21.15.39:3300/0 conn(0x7fbc6c0c90e0 msgr2=0x7fbc6c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc70b58640 1 --2- 172.21.15.32:0/205356216 >> v2:172.21.15.39:3300/0 conn(0x7fbc6c0c90e0 0x7fbc6c0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc70b58640 1 -- 172.21.15.32:0/205356216 >> v1:172.21.15.39:6789/0 conn(0x7fbc6c0b0b30 legacy=0x7fbc6c15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:30.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.810+0000 7fbc70b58640 1 -- 172.21.15.32:0/205356216 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc6c0c9e80 con 0x7fbc6c1599e0 2023-12-20T04:29:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc70b58640 1 --2- 172.21.15.32:0/205356216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c1599e0 0x7fbc6c0aece0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c06ab20 tx=0x7fbc6001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2683c86391a38211 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc6b7fe640 1 -- 172.21.15.32:0/205356216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc600ac410 con 0x7fbc6c1599e0 2023-12-20T04:29:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc6b7fe640 1 -- 172.21.15.32:0/205356216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc600ac570 con 0x7fbc6c1599e0 2023-12-20T04:29:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc6b7fe640 1 -- 172.21.15.32:0/205356216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc600ac900 con 0x7fbc6c1599e0 2023-12-20T04:29:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc72dbc640 1 -- 172.21.15.32:0/205356216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c1599e0 msgr2=0x7fbc6c0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc72dbc640 1 --2- 172.21.15.32:0/205356216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c1599e0 0x7fbc6c0aece0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c06ab20 tx=0x7fbc6001c490 comp rx=0 tx=0).stop 2023-12-20T04:29:30.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc72dbc640 1 -- 172.21.15.32:0/205356216 shutdown_connections 2023-12-20T04:29:30.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc72dbc640 1 --2- 172.21.15.32:0/205356216 >> v2:172.21.15.39:3300/0 conn(0x7fbc6c0c90e0 0x7fbc6c0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc72dbc640 1 --2- 172.21.15.32:0/205356216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c1599e0 0x7fbc6c0aece0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc72dbc640 1 -- 172.21.15.32:0/205356216 >> 172.21.15.32:0/205356216 conn(0x7fbc6c06a320 msgr2=0x7fbc6c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:30.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc72dbc640 1 -- 172.21.15.32:0/205356216 shutdown_connections 2023-12-20T04:29:30.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.814+0000 7fbc72dbc640 1 -- 172.21.15.32:0/205356216 wait complete. 2023-12-20T04:29:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc72dbc640 1 Processor -- start 2023-12-20T04:29:30.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc72dbc640 1 -- start start 2023-12-20T04:29:30.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc72dbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c0b0b30 0x7fbc6c15c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc72dbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc6c0c90e0 0x7fbc6c149190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc72dbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbc6c1599e0 0x7fbc6c146540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc72dbc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c15cbe0 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc72dbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c15c3c0 con 0x7fbc6c0c90e0 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc72dbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbc6c0b3ce0 con 0x7fbc6c1599e0 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc70b58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c0b0b30 0x7fbc6c15c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc70b58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c0b0b30 0x7fbc6c15c6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45548/0 (socket says 172.21.15.32:45548) 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc70b58640 1 -- 172.21.15.32:0/4198298199 learned_addr learned my addr 172.21.15.32:0/4198298199 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc6bfff640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc6c0c90e0 0x7fbc6c149190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc71359640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbc6c1599e0 0x7fbc6c146540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc70b58640 1 -- 172.21.15.32:0/4198298199 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbc6c1599e0 msgr2=0x7fbc6c146540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc70b58640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbc6c1599e0 0x7fbc6c146540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc70b58640 1 -- 172.21.15.32:0/4198298199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc6c0c90e0 msgr2=0x7fbc6c149190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc70b58640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc6c0c90e0 0x7fbc6c149190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc70b58640 1 -- 172.21.15.32:0/4198298199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc60009480 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc70b58640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c0b0b30 0x7fbc6c15c6a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc6001a0c0 tx=0x7fbc60019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc697fa640 1 -- 172.21.15.32:0/4198298199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc600059a0 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc697fa640 1 -- 172.21.15.32:0/4198298199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc60005b00 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc697fa640 1 -- 172.21.15.32:0/4198298199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc60005e70 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc72dbc640 1 -- 172.21.15.32:0/4198298199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbc6c0b3fc0 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc72dbc640 1 -- 172.21.15.32:0/4198298199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbc6c1496d0 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.818+0000 7fbc72dbc640 1 -- 172.21.15.32:0/4198298199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbc6c1498c0 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.822+0000 7fbc697fa640 1 -- 172.21.15.32:0/4198298199 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fbc600b3040 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.822+0000 7fbc697fa640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbc50065a60 0x7fbc50067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.822+0000 7fbc697fa640 1 -- 172.21.15.32:0/4198298199 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbc6011c5b0 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.822+0000 7fbc6bfff640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbc50065a60 0x7fbc50067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:30.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.822+0000 7fbc6bfff640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbc50065a60 0x7fbc50067f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0b36d0 tx=0x7fbc58008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:30.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.826+0000 7fbc697fa640 1 -- 172.21.15.32:0/4198298199 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbc600e9f50 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.982+0000 7fbc72dbc640 1 -- 172.21.15.32:0/4198298199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbc6c1498c0 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc697fa640 1 -- 172.21.15.32:0/4198298199 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16066 (secure 0 0 0) 0x7fbc6c1498c0 con 0x7fbc6c0b0b30 2023-12-20T04:29:30.989 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:30.989 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:29:30.609855+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:29:28.685343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-20T04:29:29.647655+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 -- 172.21.15.32:0/4198298199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbc50065a60 msgr2=0x7fbc50067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbc50065a60 0x7fbc50067f20 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7fbc6c0b36d0 tx=0x7fbc58008040 comp rx=0 tx=0).stop 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 -- 172.21.15.32:0/4198298199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c0b0b30 msgr2=0x7fbc6c15c6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c0b0b30 0x7fbc6c15c6a0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbc6001a0c0 tx=0x7fbc60019230 comp rx=0 tx=0).stop 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 -- 172.21.15.32:0/4198298199 shutdown_connections 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbc50065a60 0x7fbc50067f20 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbc6c1599e0 0x7fbc6c146540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbc6c0c90e0 0x7fbc6c149190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 --2- 172.21.15.32:0/4198298199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbc6c0b0b30 0x7fbc6c15c6a0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 -- 172.21.15.32:0/4198298199 >> 172.21.15.32:0/4198298199 conn(0x7fbc6c06a320 msgr2=0x7fbc6c071360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 -- 172.21.15.32:0/4198298199 shutdown_connections 2023-12-20T04:29:30.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:30.986+0000 7fbc72dbc640 1 -- 172.21.15.32:0/4198298199 wait complete. 2023-12-20T04:29:31.010 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:31.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.150+0000 7f28d2717640 1 Processor -- start 2023-12-20T04:29:31.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.150+0000 7f28d2717640 1 -- start start 2023-12-20T04:29:31.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.150+0000 7f28d2717640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28cc06ce60 0x7f28cc15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:31.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.150+0000 7f28d2717640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28cc15bbd0 con 0x7f28cc15f110 2023-12-20T04:29:31.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.150+0000 7f28d2717640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28cc15beb0 con 0x7f28cc0af5e0 2023-12-20T04:29:31.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.150+0000 7f28d2717640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28cc15e910 con 0x7f28cc06ce60 2023-12-20T04:29:31.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28cb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28cc06ce60 0x7f28cc15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:31.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28cb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28cc06ce60 0x7f28cc15b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45564/0 (socket says 172.21.15.32:45564) 2023-12-20T04:29:31.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28cb7fe640 1 -- 172.21.15.32:0/442246940 learned_addr learned my addr 172.21.15.32:0/442246940 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:31.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28d0cb4640 1 --1- 172.21.15.32:0/442246940 >> v1:172.21.15.32:6789/0 conn(0x7f28cc15f110 0x7f28cc15c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47534/0 (socket says 172.21.15.32:47534) 2023-12-20T04:29:31.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28cb7fe640 1 -- 172.21.15.32:0/442246940 >> v1:172.21.15.39:6789/0 conn(0x7f28cc0af5e0 legacy=0x7f28cc0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:31.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28cb7fe640 1 -- 172.21.15.32:0/442246940 >> v1:172.21.15.32:6789/0 conn(0x7f28cc15f110 legacy=0x7f28cc15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:31.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28cb7fe640 1 -- 172.21.15.32:0/442246940 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28cc15ebf0 con 0x7f28cc06ce60 2023-12-20T04:29:31.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28caffd640 1 -- 172.21.15.32:0/442246940 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1942816315 0 0) 0x7f28cc15bbd0 con 0x7f28cc15f110 2023-12-20T04:29:31.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28cb7fe640 1 --2- 172.21.15.32:0/442246940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc06ce60 0x7f28cc15b280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009870 tx=0x7f28c001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4be1074e190104 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:31.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28caffd640 1 -- 172.21.15.32:0/442246940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28cc15b9d0 con 0x7f28cc06ce60 2023-12-20T04:29:31.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28caffd640 1 -- 172.21.15.32:0/442246940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c00ab650 con 0x7f28cc06ce60 2023-12-20T04:29:31.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.154+0000 7f28caffd640 1 -- 172.21.15.32:0/442246940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c00ab9e0 con 0x7f28cc06ce60 2023-12-20T04:29:31.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 -- 172.21.15.32:0/442246940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc06ce60 msgr2=0x7f28cc15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:31.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 --2- 172.21.15.32:0/442246940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc06ce60 0x7f28cc15b280 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009870 tx=0x7f28c001c4b0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 -- 172.21.15.32:0/442246940 shutdown_connections 2023-12-20T04:29:31.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 --2- 172.21.15.32:0/442246940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc06ce60 0x7f28cc15b280 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 -- 172.21.15.32:0/442246940 >> 172.21.15.32:0/442246940 conn(0x7f28cc06a320 msgr2=0x7f28cc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:31.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 -- 172.21.15.32:0/442246940 shutdown_connections 2023-12-20T04:29:31.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 -- 172.21.15.32:0/442246940 wait complete. 2023-12-20T04:29:31.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 Processor -- start 2023-12-20T04:29:31.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 -- start start 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28cc06ce60 0x7f28cc159350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc0af5e0 0x7f28cc15db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc15f110 0x7f28cc146900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc149310 con 0x7f28cc15f110 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc1477f0 con 0x7f28cc0af5e0 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d2717640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28cc146470 con 0x7f28cc06ce60 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d0cb4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc15f110 0x7f28cc146900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d0cb4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc15f110 0x7f28cc146900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45580/0 (socket says 172.21.15.32:45580) 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d0cb4640 1 -- 172.21.15.32:0/885302032 learned_addr learned my addr 172.21.15.32:0/885302032 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28cbfff640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28cc06ce60 0x7f28cc159350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28cb7fe640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc0af5e0 0x7f28cc15db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d0cb4640 1 -- 172.21.15.32:0/885302032 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28cc06ce60 msgr2=0x7f28cc159350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d0cb4640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28cc06ce60 0x7f28cc159350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d0cb4640 1 -- 172.21.15.32:0/885302032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc0af5e0 msgr2=0x7f28cc15db60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d0cb4640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc0af5e0 0x7f28cc15db60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.158+0000 7f28d0cb4640 1 -- 172.21.15.32:0/885302032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28c00094d0 con 0x7f28cc15f110 2023-12-20T04:29:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.162+0000 7f28d0cb4640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc15f110 0x7f28cc146900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c780 tx=0x7f28c400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.162+0000 7f28c8ff9640 1 -- 172.21.15.32:0/885302032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c400ce90 con 0x7f28cc15f110 2023-12-20T04:29:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.162+0000 7f28c8ff9640 1 -- 172.21.15.32:0/885302032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28c400a640 con 0x7f28cc15f110 2023-12-20T04:29:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.162+0000 7f28c8ff9640 1 -- 172.21.15.32:0/885302032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28c400a9d0 con 0x7f28cc15f110 2023-12-20T04:29:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.162+0000 7f28d2717640 1 -- 172.21.15.32:0/885302032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28cc148d50 con 0x7f28cc15f110 2023-12-20T04:29:31.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.162+0000 7f28d2717640 1 -- 172.21.15.32:0/885302032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28cc146100 con 0x7f28cc15f110 2023-12-20T04:29:31.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.162+0000 7f28d2717640 1 -- 172.21.15.32:0/885302032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28cc146620 con 0x7f28cc15f110 2023-12-20T04:29:31.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.162+0000 7f28c8ff9640 1 -- 172.21.15.32:0/885302032 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f28c400ab30 con 0x7f28cc15f110 2023-12-20T04:29:31.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.162+0000 7f28c8ff9640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28b8065a60 0x7f28b8067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:31.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.162+0000 7f28c8ff9640 1 -- 172.21.15.32:0/885302032 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28c40826f0 con 0x7f28cc15f110 2023-12-20T04:29:31.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.162+0000 7f28cbfff640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28b8065a60 0x7f28b8067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:31.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.166+0000 7f28cbfff640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28b8065a60 0x7f28b8067f20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f28cc148c40 tx=0x7f28bc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:31.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.166+0000 7f28c8ff9640 1 -- 172.21.15.32:0/885302032 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f28cc146620 con 0x7f28cc15f110 2023-12-20T04:29:31.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.326+0000 7f28d2717640 1 -- 172.21.15.32:0/885302032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f28cc146620 con 0x7f28cc15f110 2023-12-20T04:29:31.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28c8ff9640 1 -- 172.21.15.32:0/885302032 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v34) v1 ==== 74+0+16066 (secure 0 0 0) 0x7f28cc146620 con 0x7f28cc15f110 2023-12-20T04:29:31.332 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:31.332 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":34,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:29:30.609855+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:29:28.685343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-20T04:29:29.647655+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:31.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 -- 172.21.15.32:0/885302032 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28b8065a60 msgr2=0x7f28b8067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:31.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28b8065a60 0x7f28b8067f20 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f28cc148c40 tx=0x7f28bc009040 comp rx=0 tx=0).stop 2023-12-20T04:29:31.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 -- 172.21.15.32:0/885302032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc15f110 msgr2=0x7f28cc146900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:31.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc15f110 0x7f28cc146900 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f28c400c780 tx=0x7f28c400cc40 comp rx=0 tx=0).stop 2023-12-20T04:29:31.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 -- 172.21.15.32:0/885302032 shutdown_connections 2023-12-20T04:29:31.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28b8065a60 0x7f28b8067f20 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28cc15f110 0x7f28cc146900 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28cc0af5e0 0x7f28cc15db60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 --2- 172.21.15.32:0/885302032 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28cc06ce60 0x7f28cc159350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 -- 172.21.15.32:0/885302032 >> 172.21.15.32:0/885302032 conn(0x7f28cc06a320 msgr2=0x7f28cc15c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:31.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 -- 172.21.15.32:0/885302032 shutdown_connections 2023-12-20T04:29:31.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.330+0000 7f28d2717640 1 -- 172.21.15.32:0/885302032 wait complete. 2023-12-20T04:29:31.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T04:29:31.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcbe160640 1 Processor -- start 2023-12-20T04:29:31.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcbe160640 1 -- start start 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcbe160640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcb8157480 0x7fdcb80c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcbe160640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcb80cd4c0 0x7fdcb80b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcbe160640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdcb80b1100 con 0x7fdcb80b0760 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcbe160640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdcb80cde20 con 0x7fdcb80cd4c0 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcbe160640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb80c8e30 con 0x7fdcb8157480 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcb7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcb80cd4c0 0x7fdcb80b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcb6ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdcb80b0760 0x7fdcb815d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47540/0 (socket says 172.21.15.32:47540) 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcb7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdcb80cd4c0 0x7fdcb80b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45590/0 (socket says 172.21.15.32:45590) 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcb6ffd640 1 -- 172.21.15.32:0/2482060494 learned_addr learned my addr 172.21.15.32:0/2482060494 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcb77fe640 1 --2- 172.21.15.32:0/2482060494 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcb8157480 0x7fdcb80c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcb7fff640 1 -- 172.21.15.32:0/2482060494 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcb8157480 msgr2=0x7fdcb80c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:31.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcb7fff640 1 --2- 172.21.15.32:0/2482060494 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcb8157480 0x7fdcb80c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcb7fff640 1 -- 172.21.15.32:0/2482060494 >> v1:172.21.15.32:6789/0 conn(0x7fdcb80b0760 legacy=0x7fdcb815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:31.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.498+0000 7fdcb7fff640 1 -- 172.21.15.32:0/2482060494 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb80ce070 con 0x7fdcb80cd4c0 2023-12-20T04:29:31.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcb7fff640 1 --2- 172.21.15.32:0/2482060494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcb80cd4c0 0x7fdcb80b0bc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009dc0 tx=0x7fdcb001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78838252a1b5e8db server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:31.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcb67fc640 1 -- 172.21.15.32:0/2482060494 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00a5850 con 0x7fdcb80cd4c0 2023-12-20T04:29:31.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcb67fc640 1 -- 172.21.15.32:0/2482060494 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb00a59b0 con 0x7fdcb80cd4c0 2023-12-20T04:29:31.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcb67fc640 1 -- 172.21.15.32:0/2482060494 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00a5d40 con 0x7fdcb80cd4c0 2023-12-20T04:29:31.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcbe160640 1 -- 172.21.15.32:0/2482060494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcb80cd4c0 msgr2=0x7fdcb80b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:31.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcbe160640 1 --2- 172.21.15.32:0/2482060494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcb80cd4c0 0x7fdcb80b0bc0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7fdcb0009dc0 tx=0x7fdcb001c870 comp rx=0 tx=0).stop 2023-12-20T04:29:31.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcbe160640 1 -- 172.21.15.32:0/2482060494 shutdown_connections 2023-12-20T04:29:31.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcbe160640 1 --2- 172.21.15.32:0/2482060494 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcb80cd4c0 0x7fdcb80b0bc0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcbe160640 1 --2- 172.21.15.32:0/2482060494 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcb8157480 0x7fdcb80c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcbe160640 1 -- 172.21.15.32:0/2482060494 >> 172.21.15.32:0/2482060494 conn(0x7fdcb806a510 msgr2=0x7fdcb80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:31.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcbe160640 1 -- 172.21.15.32:0/2482060494 shutdown_connections 2023-12-20T04:29:31.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcbe160640 1 -- 172.21.15.32:0/2482060494 wait complete. 2023-12-20T04:29:31.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.502+0000 7fdcbe160640 1 Processor -- start 2023-12-20T04:29:31.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcbe160640 1 -- start start 2023-12-20T04:29:31.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcbe160640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb80b0760 0x7fdcb81647d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:31.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcbe160640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcb80cd4c0 0x7fdcb81f8bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:31.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcbe160640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcb8157480 0x7fdcb81f9100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:31.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcbe160640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb8165290 con 0x7fdcb8157480 2023-12-20T04:29:31.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcbe160640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb8165570 con 0x7fdcb80b0760 2023-12-20T04:29:31.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcbe160640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdcb8165850 con 0x7fdcb80cd4c0 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcb8157480 0x7fdcb81f9100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcb8157480 0x7fdcb81f9100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45596/0 (socket says 172.21.15.32:45596) 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb7fff640 1 -- 172.21.15.32:0/4219321739 learned_addr learned my addr 172.21.15.32:0/4219321739 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb6ffd640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcb80cd4c0 0x7fdcb81f8bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb77fe640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb80b0760 0x7fdcb81647d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb7fff640 1 -- 172.21.15.32:0/4219321739 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcb80cd4c0 msgr2=0x7fdcb81f8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb7fff640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcb80cd4c0 0x7fdcb81f8bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb7fff640 1 -- 172.21.15.32:0/4219321739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb80b0760 msgr2=0x7fdcb81647d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb7fff640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb80b0760 0x7fdcb81647d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb7fff640 1 -- 172.21.15.32:0/4219321739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcb0009a70 con 0x7fdcb8157480 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb77fe640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb80b0760 0x7fdcb81647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:31.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcb7fff640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcb8157480 0x7fdcb81f9100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdcb001a4a0 tx=0x7fdcb0019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:31.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdc9ffff640 1 -- 172.21.15.32:0/4219321739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00b9860 con 0x7fdcb8157480 2023-12-20T04:29:31.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdc9ffff640 1 -- 172.21.15.32:0/4219321739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcb00b99c0 con 0x7fdcb8157480 2023-12-20T04:29:31.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdc9ffff640 1 -- 172.21.15.32:0/4219321739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcb00b9d70 con 0x7fdcb8157480 2023-12-20T04:29:31.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcbe160640 1 -- 172.21.15.32:0/4219321739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdcb8165b30 con 0x7fdcb8157480 2023-12-20T04:29:31.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcbe160640 1 -- 172.21.15.32:0/4219321739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdcb80c5030 con 0x7fdcb8157480 2023-12-20T04:29:31.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.506+0000 7fdcbe160640 1 -- 172.21.15.32:0/4219321739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdcb804e000 con 0x7fdcb8157480 2023-12-20T04:29:31.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.510+0000 7fdc9ffff640 1 -- 172.21.15.32:0/4219321739 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fdcb00ad440 con 0x7fdcb8157480 2023-12-20T04:29:31.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.510+0000 7fdc9ffff640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdca4065a60 0x7fdca4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:31.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.510+0000 7fdc9ffff640 1 -- 172.21.15.32:0/4219321739 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(34..34 src has 1..34) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdcb011e560 con 0x7fdcb8157480 2023-12-20T04:29:31.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.510+0000 7fdcb77fe640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdca4065a60 0x7fdca4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:31.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.510+0000 7fdcb77fe640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdca4065a60 0x7fdca4067f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdca8007a70 tx=0x7fdca8008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:31.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.514+0000 7fdc9ffff640 1 -- 172.21.15.32:0/4219321739 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdcb00ebf10 con 0x7fdcb8157480 2023-12-20T04:29:31.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:31.706+0000 7fdcbe160640 1 -- 172.21.15.32:0/4219321739 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fdcb806d650 con 0x7fdcb8157480 2023-12-20T04:29:32.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.626+0000 7fdc9ffff640 1 -- 172.21.15.32:0/4219321739 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v15) v1 ==== 118+0+0 (secure 0 0 0) 0x7fdcb00ef530 con 0x7fdcb8157480 2023-12-20T04:29:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.626+0000 7fdcbe160640 1 -- 172.21.15.32:0/4219321739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdca4065a60 msgr2=0x7fdca4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:32.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.626+0000 7fdcbe160640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdca4065a60 0x7fdca4067f20 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdca8007a70 tx=0x7fdca8008040 comp rx=0 tx=0).stop 2023-12-20T04:29:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.626+0000 7fdcbe160640 1 -- 172.21.15.32:0/4219321739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcb8157480 msgr2=0x7fdcb81f9100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.626+0000 7fdcbe160640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcb8157480 0x7fdcb81f9100 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fdcb001a4a0 tx=0x7fdcb0019610 comp rx=0 tx=0).stop 2023-12-20T04:29:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.626+0000 7fdcbe160640 1 -- 172.21.15.32:0/4219321739 shutdown_connections 2023-12-20T04:29:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.626+0000 7fdcbe160640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdca4065a60 0x7fdca4067f20 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.626+0000 7fdcbe160640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdcb8157480 0x7fdcb81f9100 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.626+0000 7fdcbe160640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdcb80cd4c0 0x7fdcb81f8bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.626+0000 7fdcbe160640 1 --2- 172.21.15.32:0/4219321739 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdcb80b0760 0x7fdcb81647d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.626+0000 7fdcbe160640 1 -- 172.21.15.32:0/4219321739 >> 172.21.15.32:0/4219321739 conn(0x7fdcb806a510 msgr2=0x7fdcb80c3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.630+0000 7fdcbe160640 1 -- 172.21.15.32:0/4219321739 shutdown_connections 2023-12-20T04:29:32.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.630+0000 7fdcbe160640 1 -- 172.21.15.32:0/4219321739 wait complete. 2023-12-20T04:29:32.658 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T04:29:32.659 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T04:29:32.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f493388c640 1 Processor -- start 2023-12-20T04:29:32.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f493388c640 1 -- start start 2023-12-20T04:29:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f493388c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f492c15abb0 0x7f492c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f493388c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492c070230 0x7f492c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f493388c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f492c15b6a0 con 0x7f492c0b0cb0 2023-12-20T04:29:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f493388c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f492c06e160 con 0x7f492c15abb0 2023-12-20T04:29:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f493388c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c070c00 con 0x7f492c070230 2023-12-20T04:29:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f4931628640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f492c15abb0 0x7f492c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f4931628640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f492c15abb0 0x7f492c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45606/0 (socket says 172.21.15.32:45606) 2023-12-20T04:29:32.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f4931628640 1 -- 172.21.15.32:0/2403517905 learned_addr learned my addr 172.21.15.32:0/2403517905 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f4931e29640 1 --2- 172.21.15.32:0/2403517905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492c070230 0x7f492c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f4931628640 1 -- 172.21.15.32:0/2403517905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492c070230 msgr2=0x7f492c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f4931628640 1 --2- 172.21.15.32:0/2403517905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492c070230 0x7f492c0706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f4931628640 1 -- 172.21.15.32:0/2403517905 >> v1:172.21.15.39:6789/0 conn(0x7f492c0b0cb0 legacy=0x7f492c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:32.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.806+0000 7f4931628640 1 -- 172.21.15.32:0/2403517905 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f492c06d8c0 con 0x7f492c15abb0 2023-12-20T04:29:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f4931628640 1 --2- 172.21.15.32:0/2403517905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492c15abb0 0x7f492c0caaf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4928009870 tx=0x7f492801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4455723f7a55219 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f4923fff640 1 -- 172.21.15.32:0/2403517905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49280aa620 con 0x7f492c15abb0 2023-12-20T04:29:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f4923fff640 1 -- 172.21.15.32:0/2403517905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49280aa780 con 0x7f492c15abb0 2023-12-20T04:29:32.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f4923fff640 1 -- 172.21.15.32:0/2403517905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49280aab10 con 0x7f492c15abb0 2023-12-20T04:29:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f493388c640 1 -- 172.21.15.32:0/2403517905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492c15abb0 msgr2=0x7f492c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f493388c640 1 --2- 172.21.15.32:0/2403517905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492c15abb0 0x7f492c0caaf0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f4928009870 tx=0x7f492801c500 comp rx=0 tx=0).stop 2023-12-20T04:29:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f493388c640 1 -- 172.21.15.32:0/2403517905 shutdown_connections 2023-12-20T04:29:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f493388c640 1 --2- 172.21.15.32:0/2403517905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492c070230 0x7f492c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f493388c640 1 --2- 172.21.15.32:0/2403517905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492c15abb0 0x7f492c0caaf0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:32.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f493388c640 1 -- 172.21.15.32:0/2403517905 >> 172.21.15.32:0/2403517905 conn(0x7f492c06a510 msgr2=0x7f492c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f493388c640 1 -- 172.21.15.32:0/2403517905 shutdown_connections 2023-12-20T04:29:32.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.810+0000 7f493388c640 1 -- 172.21.15.32:0/2403517905 wait complete. 2023-12-20T04:29:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f493388c640 1 Processor -- start 2023-12-20T04:29:32.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f493388c640 1 -- start start 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f493388c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492c070230 0x7f492c135410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f493388c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492c0b0cb0 0x7f492c0b3230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f493388c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492c15abb0 0x7f492c0b3770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f493388c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c138d20 con 0x7f492c15abb0 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f493388c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492c14f2c0 con 0x7f492c0b0cb0 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f493388c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492c14f560 con 0x7f492c070230 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4931e29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492c15abb0 0x7f492c0b3770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4931e29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492c15abb0 0x7f492c0b3770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45614/0 (socket says 172.21.15.32:45614) 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4931e29640 1 -- 172.21.15.32:0/1069115332 learned_addr learned my addr 172.21.15.32:0/1069115332 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4931628640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492c070230 0x7f492c135410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4930e27640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492c0b0cb0 0x7f492c0b3230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4931e29640 1 -- 172.21.15.32:0/1069115332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492c070230 msgr2=0x7f492c135410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4931e29640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492c070230 0x7f492c135410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4931e29640 1 -- 172.21.15.32:0/1069115332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492c0b0cb0 msgr2=0x7f492c0b3230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4931e29640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492c0b0cb0 0x7f492c0b3230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4931e29640 1 -- 172.21.15.32:0/1069115332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49280094d0 con 0x7f492c15abb0 2023-12-20T04:29:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4931628640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492c070230 0x7f492c135410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4931e29640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492c15abb0 0x7f492c0b3770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4924007e40 tx=0x7f492400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4921ffb640 1 -- 172.21.15.32:0/1069115332 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4924014620 con 0x7f492c15abb0 2023-12-20T04:29:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4921ffb640 1 -- 172.21.15.32:0/1069115332 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f492400ee90 con 0x7f492c15abb0 2023-12-20T04:29:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f4921ffb640 1 -- 172.21.15.32:0/1069115332 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49240149e0 con 0x7f492c15abb0 2023-12-20T04:29:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f493388c640 1 -- 172.21.15.32:0/1069115332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492c14de20 con 0x7f492c15abb0 2023-12-20T04:29:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.814+0000 7f493388c640 1 -- 172.21.15.32:0/1069115332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492c14db40 con 0x7f492c15abb0 2023-12-20T04:29:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.818+0000 7f493388c640 1 -- 172.21.15.32:0/1069115332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492c14f720 con 0x7f492c15abb0 2023-12-20T04:29:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.818+0000 7f4921ffb640 1 -- 172.21.15.32:0/1069115332 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f4924014b40 con 0x7f492c15abb0 2023-12-20T04:29:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.818+0000 7f4921ffb640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4910065a60 0x7f4910067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.818+0000 7f4921ffb640 1 -- 172.21.15.32:0/1069115332 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4924083bb0 con 0x7f492c15abb0 2023-12-20T04:29:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.818+0000 7f4931628640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4910065a60 0x7f4910067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.818+0000 7f4931628640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4910065a60 0x7f4910067f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f49280ae000 tx=0x7f49280a7000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:32.822+0000 7f4921ffb640 1 -- 172.21.15.32:0/1069115332 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f49240514d0 con 0x7f492c15abb0 2023-12-20T04:29:33.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.014+0000 7f493388c640 1 -- 172.21.15.32:0/1069115332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f492c04e000 con 0x7f492c15abb0 2023-12-20T04:29:33.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.662+0000 7f4921ffb640 1 -- 172.21.15.32:0/1069115332 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4924054af0 con 0x7f492c15abb0 2023-12-20T04:29:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 -- 172.21.15.32:0/1069115332 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4910065a60 msgr2=0x7f4910067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4910065a60 0x7f4910067f20 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f49280ae000 tx=0x7f49280a7000 comp rx=0 tx=0).stop 2023-12-20T04:29:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 -- 172.21.15.32:0/1069115332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492c15abb0 msgr2=0x7f492c0b3770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:33.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492c15abb0 0x7f492c0b3770 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f4924007e40 tx=0x7f492400e4b0 comp rx=0 tx=0).stop 2023-12-20T04:29:33.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 -- 172.21.15.32:0/1069115332 shutdown_connections 2023-12-20T04:29:33.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4910065a60 0x7f4910067f20 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:33.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f492c15abb0 0x7f492c0b3770 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:33.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f492c0b0cb0 0x7f492c0b3230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:33.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 --2- 172.21.15.32:0/1069115332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492c070230 0x7f492c135410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:33.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 -- 172.21.15.32:0/1069115332 >> 172.21.15.32:0/1069115332 conn(0x7f492c06a510 msgr2=0x7f492c153090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:33.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 -- 172.21.15.32:0/1069115332 shutdown_connections 2023-12-20T04:29:33.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.666+0000 7f493388c640 1 -- 172.21.15.32:0/1069115332 wait complete. 2023-12-20T04:29:33.692 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T04:29:33.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:29:33.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.834+0000 7f489a8f9640 1 Processor -- start 2023-12-20T04:29:33.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f489a8f9640 1 -- start start 2023-12-20T04:29:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f489a8f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f489406ce60 0x7f489415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f489a8f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489415bbd0 con 0x7f48940af5e0 2023-12-20T04:29:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f489a8f9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f489415beb0 con 0x7f489415f110 2023-12-20T04:29:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f489a8f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489415e910 con 0x7f489406ce60 2023-12-20T04:29:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4893fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f48940af5e0 0x7f48940af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47544/0 (socket says 172.21.15.32:47544) 2023-12-20T04:29:33.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4893fff640 1 -- 172.21.15.32:0/665337901 learned_addr learned my addr 172.21.15.32:0/665337901 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f48937fe640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f489406ce60 0x7f489415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 524564897 0 0) 0x7f489415bbd0 con 0x7f48940af5e0 2023-12-20T04:29:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f487c003530 con 0x7f48940af5e0 2023-12-20T04:29:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2892782307 0 0) 0x7f487c003530 con 0x7f48940af5e0 2023-12-20T04:29:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f489406ce60 msgr2=0x7f489415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f489406ce60 0x7f489415b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 >> v1:172.21.15.39:6789/0 conn(0x7f489415f110 legacy=0x7f489415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f489415ebf0 con 0x7f48940af5e0 2023-12-20T04:29:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f4884001800 con 0x7f48940af5e0 2023-12-20T04:29:33.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 >> v1:172.21.15.32:6789/0 conn(0x7f48940af5e0 legacy=0x7f48940af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0039b0 0x7f487c003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f489406ce60 0x7f487c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f487c004d10 0x7f487c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f489415e910 con 0x7f487c0039b0 2023-12-20T04:29:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f487c00bac0 con 0x7f487c004d10 2023-12-20T04:29:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f487c00bc80 con 0x7f489406ce60 2023-12-20T04:29:33.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4893fff640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0039b0 0x7f487c003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4898e96640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f487c004d10 0x7f487c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f48937fe640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f489406ce60 0x7f487c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4893fff640 1 -- 172.21.15.32:0/665337901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f489406ce60 msgr2=0x7f487c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4893fff640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f489406ce60 0x7f487c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4893fff640 1 -- 172.21.15.32:0/665337901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f487c004d10 msgr2=0x7f487c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4893fff640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f487c004d10 0x7f487c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:33.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.838+0000 7f4893fff640 1 -- 172.21.15.32:0/665337901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f487c003530 con 0x7f487c0039b0 2023-12-20T04:29:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.842+0000 7f4893fff640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0039b0 0x7f487c003dd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f488400c240 tx=0x7f488401cb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0fe90e54dbdc386 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.842+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4884007270 con 0x7f487c0039b0 2023-12-20T04:29:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.842+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f48840073f0 con 0x7f487c0039b0 2023-12-20T04:29:33.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.842+0000 7f4892ffd640 1 -- 172.21.15.32:0/665337901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48840a57e0 con 0x7f487c0039b0 2023-12-20T04:29:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.842+0000 7f489a8f9640 1 -- 172.21.15.32:0/665337901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0039b0 msgr2=0x7f487c003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.842+0000 7f489a8f9640 1 --2- 172.21.15.32:0/665337901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f487c0039b0 0x7f487c003dd0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f488400c240 tx=0x7f488401cb60 comp rx=0 tx=0).stop 2023-12-20T04:29:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.842+0000 7f4898e96640 1 -- 172.21.15.32:0/665337901 reap_dead start 2023-12-20T04:29:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.842+0000 7f489a8f9640 1 -- 172.21.15.32:0/665337901 shutdown_connections 2023-12-20T04:29:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.842+0000 7f489a8f9640 1 -- 172.21.15.32:0/665337901 >> 172.21.15.32:0/665337901 conn(0x7f489406a320 msgr2=0x7f48940ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.842+0000 7f489a8f9640 1 -- 172.21.15.32:0/665337901 shutdown_connections 2023-12-20T04:29:33.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.842+0000 7f489a8f9640 1 -- 172.21.15.32:0/665337901 wait complete. 2023-12-20T04:29:33.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f489a8f9640 1 Processor -- start 2023-12-20T04:29:33.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f489a8f9640 1 -- start start 2023-12-20T04:29:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f489a8f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f487c0039b0 0x7f4894163bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f489a8f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4894161790 0x7f4894159b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f489a8f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48940cb170 0x7f48940cb5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f489a8f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940711b0 con 0x7f48940cb170 2023-12-20T04:29:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f489a8f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48940cbb30 con 0x7f4894161790 2023-12-20T04:29:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f489a8f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f489415ad20 con 0x7f487c0039b0 2023-12-20T04:29:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4898e96640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48940cb170 0x7f48940cb5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4898e96640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48940cb170 0x7f48940cb5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45636/0 (socket says 172.21.15.32:45636) 2023-12-20T04:29:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4898e96640 1 -- 172.21.15.32:0/3312410802 learned_addr learned my addr 172.21.15.32:0/3312410802 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4893fff640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f487c0039b0 0x7f4894163bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:33.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f48937fe640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4894161790 0x7f4894159b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4898e96640 1 -- 172.21.15.32:0/3312410802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f487c0039b0 msgr2=0x7f4894163bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4898e96640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f487c0039b0 0x7f4894163bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4898e96640 1 -- 172.21.15.32:0/3312410802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4894161790 msgr2=0x7f4894159b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4898e96640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4894161790 0x7f4894159b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:33.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4898e96640 1 -- 172.21.15.32:0/3312410802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4894151560 con 0x7f48940cb170 2023-12-20T04:29:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4898e96640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48940cb170 0x7f48940cb5f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f489415e7d0 tx=0x7f488c00ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4890ff9640 1 -- 172.21.15.32:0/3312410802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c003980 con 0x7f48940cb170 2023-12-20T04:29:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4890ff9640 1 -- 172.21.15.32:0/3312410802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f488c003ae0 con 0x7f48940cb170 2023-12-20T04:29:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f4890ff9640 1 -- 172.21.15.32:0/3312410802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f488c003e90 con 0x7f48940cb170 2023-12-20T04:29:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f489a8f9640 1 -- 172.21.15.32:0/3312410802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4894070530 con 0x7f48940cb170 2023-12-20T04:29:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.846+0000 7f489a8f9640 1 -- 172.21.15.32:0/3312410802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4894070af0 con 0x7f48940cb170 2023-12-20T04:29:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.850+0000 7f489a8f9640 1 -- 172.21.15.32:0/3312410802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f48940712f0 con 0x7f48940cb170 2023-12-20T04:29:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.850+0000 7f4890ff9640 1 -- 172.21.15.32:0/3312410802 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f488c0050d0 con 0x7f48940cb170 2023-12-20T04:29:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.850+0000 7f4890ff9640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4880065a60 0x7f4880067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.850+0000 7f4890ff9640 1 -- 172.21.15.32:0/3312410802 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f488c082de0 con 0x7f48940cb170 2023-12-20T04:29:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.850+0000 7f4893fff640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4880065a60 0x7f4880067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:33.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.850+0000 7f4893fff640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4880065a60 0x7f4880067f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f48840ab000 tx=0x7f48840a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:33.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:33.854+0000 7f4890ff9640 1 -- 172.21.15.32:0/3312410802 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f48940712f0 con 0x7f48940cb170 2023-12-20T04:29:34.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f489a8f9640 1 -- 172.21.15.32:0/3312410802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f48940712f0 con 0x7f48940cb170 2023-12-20T04:29:34.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f4890ff9640 1 -- 172.21.15.32:0/3312410802 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f48940712f0 con 0x7f48940cb170 2023-12-20T04:29:34.050 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:34.050 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2541430692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2541430692},{"type":"v1","addr":"172.21.15.39:6835","nonce":2541430692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1379533068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1379533068},{"type":"v1","addr":"172.21.15.32:6837","nonce":1379533068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4021567802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4021567802},{"type":"v1","addr":"172.21.15.39:6834","nonce":4021567802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:29:30.610189+0000","modified":"2023-12-20T04:29:33.661905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3391762708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3391762708},{"type":"v1","addr":"172.21.15.32:6836","nonce":3391762708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-20T04:29:34.050 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2023-12-20T04:29:34.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f489a8f9640 1 -- 172.21.15.32:0/3312410802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4880065a60 msgr2=0x7f4880067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f489a8f9640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4880065a60 0x7f4880067f20 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f48840ab000 tx=0x7f48840a7070 comp rx=0 tx=0).stop 2023-12-20T04:29:34.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f489a8f9640 1 -- 172.21.15.32:0/3312410802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48940cb170 msgr2=0x7f48940cb5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f489a8f9640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48940cb170 0x7f48940cb5f0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f489415e7d0 tx=0x7f488c00ec10 comp rx=0 tx=0).stop 2023-12-20T04:29:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f489a8f9640 1 -- 172.21.15.32:0/3312410802 shutdown_connections 2023-12-20T04:29:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f489a8f9640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4880065a60 0x7f4880067f20 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f489a8f9640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f48940cb170 0x7f48940cb5f0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f489a8f9640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4894161790 0x7f4894159b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f489a8f9640 1 --2- 172.21.15.32:0/3312410802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f487c0039b0 0x7f4894163bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.046+0000 7f489a8f9640 1 -- 172.21.15.32:0/3312410802 >> 172.21.15.32:0/3312410802 conn(0x7f489406a320 msgr2=0x7f48940ccae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.050+0000 7f489a8f9640 1 -- 172.21.15.32:0/3312410802 shutdown_connections 2023-12-20T04:29:34.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.050+0000 7f489a8f9640 1 -- 172.21.15.32:0/3312410802 wait complete. 2023-12-20T04:29:34.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:29:34.189 INFO:tasks.ceph.mds.a.smithi032.stderr:2023-12-20T04:29:34.186+0000 7f3e571c3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:29:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4ab5e640 1 Processor -- start 2023-12-20T04:29:34.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4ab5e640 1 -- start start 2023-12-20T04:29:34.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4ab5e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec440b8480 0x7fec4406af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4ab5e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fec44161500 0x7fec44153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4ab5e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec4406b8c0 con 0x7fec4406b460 2023-12-20T04:29:34.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4ab5e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec440cd4d0 con 0x7fec44161500 2023-12-20T04:29:34.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4ab5e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec440ce4e0 con 0x7fec440b8480 2023-12-20T04:29:34.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4a35d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fec44161500 0x7fec44153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec49b5c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec440b8480 0x7fec4406af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4a35d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fec44161500 0x7fec44153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48536/0 (socket says 172.21.15.32:48536) 2023-12-20T04:29:34.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4a35d640 1 -- 172.21.15.32:0/162476524 learned_addr learned my addr 172.21.15.32:0/162476524 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:34.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4935b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fec4406b460 0x7fec44160df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:34276/0 (socket says 172.21.15.32:34276) 2023-12-20T04:29:34.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4a35d640 1 -- 172.21.15.32:0/162476524 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec440b8480 msgr2=0x7fec4406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:34.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4a35d640 1 --2- 172.21.15.32:0/162476524 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec440b8480 0x7fec4406af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4a35d640 1 -- 172.21.15.32:0/162476524 >> v1:172.21.15.39:6789/0 conn(0x7fec4406b460 legacy=0x7fec44160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec4a35d640 1 -- 172.21.15.32:0/162476524 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec440cd720 con 0x7fec44161500 2023-12-20T04:29:34.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.214+0000 7fec48b5a640 1 -- 172.21.15.32:0/162476524 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2336099616 0 0) 0x7fec4406b8c0 con 0x7fec4406b460 2023-12-20T04:29:34.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec4a35d640 1 --2- 172.21.15.32:0/162476524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec44161500 0x7fec44153610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009e50 tx=0x7fec3c01ca10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9a837e71f3b94cfd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:34.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec48b5a640 1 -- 172.21.15.32:0/162476524 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0a56e0 con 0x7fec44161500 2023-12-20T04:29:34.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec48b5a640 1 -- 172.21.15.32:0/162476524 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c0a5840 con 0x7fec44161500 2023-12-20T04:29:34.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec48b5a640 1 -- 172.21.15.32:0/162476524 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0a5bd0 con 0x7fec44161500 2023-12-20T04:29:34.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec4ab5e640 1 -- 172.21.15.32:0/162476524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec44161500 msgr2=0x7fec44153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec4ab5e640 1 --2- 172.21.15.32:0/162476524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec44161500 0x7fec44153610 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fec3c009e50 tx=0x7fec3c01ca10 comp rx=0 tx=0).stop 2023-12-20T04:29:34.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec4ab5e640 1 -- 172.21.15.32:0/162476524 shutdown_connections 2023-12-20T04:29:34.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec4ab5e640 1 --2- 172.21.15.32:0/162476524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec44161500 0x7fec44153610 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec4ab5e640 1 --2- 172.21.15.32:0/162476524 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec440b8480 0x7fec4406af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec4ab5e640 1 -- 172.21.15.32:0/162476524 >> 172.21.15.32:0/162476524 conn(0x7fec440d2a50 msgr2=0x7fec440d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:34.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec4ab5e640 1 -- 172.21.15.32:0/162476524 shutdown_connections 2023-12-20T04:29:34.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.218+0000 7fec4ab5e640 1 -- 172.21.15.32:0/162476524 wait complete. 2023-12-20T04:29:34.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4ab5e640 1 Processor -- start 2023-12-20T04:29:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4ab5e640 1 -- start start 2023-12-20T04:29:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4ab5e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4406b460 0x7fec44151d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4ab5e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec440b8480 0x7fec44147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4ab5e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec44161500 0x7fec440c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4ab5e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec44151320 con 0x7fec44161500 2023-12-20T04:29:34.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4ab5e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec440c7f70 con 0x7fec440b8480 2023-12-20T04:29:34.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4ab5e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec440c8250 con 0x7fec4406b460 2023-12-20T04:29:34.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4a35d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec44161500 0x7fec440c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4a35d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec44161500 0x7fec440c7520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45638/0 (socket says 172.21.15.32:45638) 2023-12-20T04:29:34.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4a35d640 1 -- 172.21.15.32:0/2957236787 learned_addr learned my addr 172.21.15.32:0/2957236787 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:34.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec49b5c640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4406b460 0x7fec44151d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4935b640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec440b8480 0x7fec44147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4a35d640 1 -- 172.21.15.32:0/2957236787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4406b460 msgr2=0x7fec44151d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4a35d640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4406b460 0x7fec44151d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4a35d640 1 -- 172.21.15.32:0/2957236787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec440b8480 msgr2=0x7fec44147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4a35d640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec440b8480 0x7fec44147d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4a35d640 1 -- 172.21.15.32:0/2957236787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c009b30 con 0x7fec44161500 2023-12-20T04:29:34.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4935b640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec440b8480 0x7fec44147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4a35d640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec44161500 0x7fec440c7520 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fec3c01a640 tx=0x7fec3c0197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec3a7fc640 1 -- 172.21.15.32:0/2957236787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0b8640 con 0x7fec44161500 2023-12-20T04:29:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec3a7fc640 1 -- 172.21.15.32:0/2957236787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c0b87c0 con 0x7fec44161500 2023-12-20T04:29:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4ab5e640 1 -- 172.21.15.32:0/2957236787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec44151570 con 0x7fec44161500 2023-12-20T04:29:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec3a7fc640 1 -- 172.21.15.32:0/2957236787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c0b8b70 con 0x7fec44161500 2023-12-20T04:29:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4ab5e640 1 -- 172.21.15.32:0/2957236787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec441522b0 con 0x7fec44161500 2023-12-20T04:29:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.222+0000 7fec4ab5e640 1 -- 172.21.15.32:0/2957236787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec440c8410 con 0x7fec44161500 2023-12-20T04:29:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.226+0000 7fec3a7fc640 1 -- 172.21.15.32:0/2957236787 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fec3c0b8cd0 con 0x7fec44161500 2023-12-20T04:29:34.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.226+0000 7fec3a7fc640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec28065a90 0x7fec28067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.226+0000 7fec3a7fc640 1 -- 172.21.15.32:0/2957236787 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fec3c11e350 con 0x7fec44161500 2023-12-20T04:29:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.226+0000 7fec49b5c640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec28065a90 0x7fec28067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.226+0000 7fec49b5c640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec28065a90 0x7fec28067f50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fec40009af0 tx=0x7fec40008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:34.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.230+0000 7fec3a7fc640 1 -- 172.21.15.32:0/2957236787 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fec3c0ebe20 con 0x7fec44161500 2023-12-20T04:29:34.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 -- 172.21.15.32:0/2957236787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fec44074100 con 0x7fec44161500 2023-12-20T04:29:34.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec3a7fc640 1 -- 172.21.15.32:0/2957236787 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v35) v1 ==== 74+0+16053 (secure 0 0 0) 0x7fec3c009510 con 0x7fec44161500 2023-12-20T04:29:34.401 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:34.401 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":35,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:29:31.613058+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:29:28.685343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-20T04:29:29.647655+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"35","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:29:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 -- 172.21.15.32:0/2957236787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec28065a90 msgr2=0x7fec28067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec28065a90 0x7fec28067f50 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7fec40009af0 tx=0x7fec40008040 comp rx=0 tx=0).stop 2023-12-20T04:29:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 -- 172.21.15.32:0/2957236787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec44161500 msgr2=0x7fec440c7520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec44161500 0x7fec440c7520 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fec3c01a640 tx=0x7fec3c0197b0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 -- 172.21.15.32:0/2957236787 shutdown_connections 2023-12-20T04:29:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec28065a90 0x7fec28067f50 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec44161500 0x7fec440c7520 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec440b8480 0x7fec44147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 --2- 172.21.15.32:0/2957236787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4406b460 0x7fec44151d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 -- 172.21.15.32:0/2957236787 >> 172.21.15.32:0/2957236787 conn(0x7fec440d2a50 msgr2=0x7fec440b6e70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 -- 172.21.15.32:0/2957236787 shutdown_connections 2023-12-20T04:29:34.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.398+0000 7fec4ab5e640 1 -- 172.21.15.32:0/2957236787 wait complete. 2023-12-20T04:29:34.423 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T04:29:34.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:29:34.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.562+0000 7ff05abbc640 1 Processor -- start 2023-12-20T04:29:34.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.562+0000 7ff05abbc640 1 -- start start 2023-12-20T04:29:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.562+0000 7ff05abbc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff0540af5e0 0x7ff0540af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.562+0000 7ff05abbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff05415f110 0x7ff05415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.562+0000 7ff05abbc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff05415b990 con 0x7ff05406ce60 2023-12-20T04:29:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.562+0000 7ff05abbc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff05415bbd0 con 0x7ff0540af5e0 2023-12-20T04:29:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.562+0000 7ff05abbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff05415beb0 con 0x7ff05415f110 2023-12-20T04:29:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff058958640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff0540af5e0 0x7ff0540af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff059159640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff05415f110 0x7ff05415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff058958640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff0540af5e0 0x7ff0540af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48540/0 (socket says 172.21.15.32:48540) 2023-12-20T04:29:34.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff058958640 1 -- 172.21.15.32:0/1600615989 learned_addr learned my addr 172.21.15.32:0/1600615989 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff058958640 1 -- 172.21.15.32:0/1600615989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff05415f110 msgr2=0x7ff05415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff058958640 1 --2- 172.21.15.32:0/1600615989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff05415f110 0x7ff05415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff058958640 1 -- 172.21.15.32:0/1600615989 >> v1:172.21.15.39:6789/0 conn(0x7ff05406ce60 legacy=0x7ff05415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff058958640 1 -- 172.21.15.32:0/1600615989 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff05415e910 con 0x7ff0540af5e0 2023-12-20T04:29:34.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff058958640 1 --2- 172.21.15.32:0/1600615989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0540af5e0 0x7ff0540af9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff048006fd0 tx=0x7ff04801c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9d3b1ec9b3dab137 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:34.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff0537fe640 1 -- 172.21.15.32:0/1600615989 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0480aa670 con 0x7ff0540af5e0 2023-12-20T04:29:34.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff0537fe640 1 -- 172.21.15.32:0/1600615989 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0480aa7d0 con 0x7ff0540af5e0 2023-12-20T04:29:34.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.566+0000 7ff0537fe640 1 -- 172.21.15.32:0/1600615989 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0480aab60 con 0x7ff0540af5e0 2023-12-20T04:29:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 -- 172.21.15.32:0/1600615989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0540af5e0 msgr2=0x7ff0540af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 --2- 172.21.15.32:0/1600615989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0540af5e0 0x7ff0540af9c0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7ff048006fd0 tx=0x7ff04801c4f0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 -- 172.21.15.32:0/1600615989 shutdown_connections 2023-12-20T04:29:34.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 --2- 172.21.15.32:0/1600615989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff05415f110 0x7ff05415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 --2- 172.21.15.32:0/1600615989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0540af5e0 0x7ff0540af9c0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 -- 172.21.15.32:0/1600615989 >> 172.21.15.32:0/1600615989 conn(0x7ff05406a320 msgr2=0x7ff0540ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 -- 172.21.15.32:0/1600615989 shutdown_connections 2023-12-20T04:29:34.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 -- 172.21.15.32:0/1600615989 wait complete. 2023-12-20T04:29:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 Processor -- start 2023-12-20T04:29:34.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 -- start start 2023-12-20T04:29:34.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff05406ce60 0x7ff054157de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0540af5e0 0x7ff05414a500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff05415f110 0x7ff0541513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff054156490 con 0x7ff0540af5e0 2023-12-20T04:29:34.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff054158320 con 0x7ff05406ce60 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff05abbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0540b2e50 con 0x7ff05415f110 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff053fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0540af5e0 0x7ff05414a500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff053fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0540af5e0 0x7ff05414a500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45642/0 (socket says 172.21.15.32:45642) 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff053fff640 1 -- 172.21.15.32:0/3843200110 learned_addr learned my addr 172.21.15.32:0/3843200110 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff058958640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff05406ce60 0x7ff054157de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.570+0000 7ff059159640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff05415f110 0x7ff0541513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff053fff640 1 -- 172.21.15.32:0/3843200110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff05415f110 msgr2=0x7ff0541513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff053fff640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff05415f110 0x7ff0541513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff053fff640 1 -- 172.21.15.32:0/3843200110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff05406ce60 msgr2=0x7ff054157de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff053fff640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff05406ce60 0x7ff054157de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff053fff640 1 -- 172.21.15.32:0/3843200110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff048009480 con 0x7ff0540af5e0 2023-12-20T04:29:34.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff053fff640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0540af5e0 0x7ff05414a500 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff04000c420 tx=0x7ff04000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:34.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff0517fa640 1 -- 172.21.15.32:0/3843200110 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0400052d0 con 0x7ff0540af5e0 2023-12-20T04:29:34.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff0517fa640 1 -- 172.21.15.32:0/3843200110 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff040005430 con 0x7ff0540af5e0 2023-12-20T04:29:34.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff05abbc640 1 -- 172.21.15.32:0/3843200110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff054156730 con 0x7ff0540af5e0 2023-12-20T04:29:34.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff0517fa640 1 -- 172.21.15.32:0/3843200110 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff040007260 con 0x7ff0540af5e0 2023-12-20T04:29:34.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff05abbc640 1 -- 172.21.15.32:0/3843200110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff054156cf0 con 0x7ff0540af5e0 2023-12-20T04:29:34.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff05abbc640 1 -- 172.21.15.32:0/3843200110 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff05404e000 con 0x7ff0540af5e0 2023-12-20T04:29:34.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff0517fa640 1 -- 172.21.15.32:0/3843200110 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7ff0400073c0 con 0x7ff0540af5e0 2023-12-20T04:29:34.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff0517fa640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff038065a60 0x7ff038067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff0517fa640 1 -- 172.21.15.32:0/3843200110 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff0400823e0 con 0x7ff0540af5e0 2023-12-20T04:29:34.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.574+0000 7ff058958640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff038065a60 0x7ff038067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.578+0000 7ff058958640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff038065a60 0x7ff038067f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff0480ae000 tx=0x7ff0480b5040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:34.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.578+0000 7ff0517fa640 1 -- 172.21.15.32:0/3843200110 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff04004fdb0 con 0x7ff0540af5e0 2023-12-20T04:29:34.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.738+0000 7ff05abbc640 1 -- 172.21.15.32:0/3843200110 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff05415c190 con 0x7ff038065a60 2023-12-20T04:29:34.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.750+0000 7ff0517fa640 1 -- 172.21.15.32:0/3843200110 <== mgr.4104 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278026 (secure 0 0 0) 0x7ff048004040 con 0x7ff038065a60 2023-12-20T04:29:34.756 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:34.758 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:29:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.750+0000 7ff05abbc640 1 -- 172.21.15.32:0/3843200110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff038065a60 msgr2=0x7ff038067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.750+0000 7ff05abbc640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff038065a60 0x7ff038067f20 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7ff0480ae000 tx=0x7ff0480b5040 comp rx=0 tx=0).stop 2023-12-20T04:29:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.750+0000 7ff05abbc640 1 -- 172.21.15.32:0/3843200110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0540af5e0 msgr2=0x7ff05414a500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.750+0000 7ff05abbc640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0540af5e0 0x7ff05414a500 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7ff04000c420 tx=0x7ff04000c8e0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.750+0000 7ff05abbc640 1 -- 172.21.15.32:0/3843200110 shutdown_connections 2023-12-20T04:29:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.750+0000 7ff05abbc640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff038065a60 0x7ff038067f20 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.750+0000 7ff05abbc640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff05415f110 0x7ff0541513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.750+0000 7ff05abbc640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0540af5e0 0x7ff05414a500 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.750+0000 7ff05abbc640 1 --2- 172.21.15.32:0/3843200110 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff05406ce60 0x7ff054157de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.750+0000 7ff05abbc640 1 -- 172.21.15.32:0/3843200110 >> 172.21.15.32:0/3843200110 conn(0x7ff05406a320 msgr2=0x7ff0540cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:34.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.754+0000 7ff05abbc640 1 -- 172.21.15.32:0/3843200110 shutdown_connections 2023-12-20T04:29:34.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.754+0000 7ff05abbc640 1 -- 172.21.15.32:0/3843200110 wait complete. 2023-12-20T04:29:34.775 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-12-20T04:29:34.137815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240000,"kb_used_data":3904,"kb_used_omap":0,"kb_used_meta":236032,"kb_avail":754734720,"statfs":{"total":773094113280,"available":772848353280,"internally_reserved":0,"allocated":3997696,"data_stored":2243562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":241696768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.942055"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.206667+0000","last_change":"2023-12-20T04:29:29.206667+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.206667+0000","last_undegraded":"2023-12-20T04:29:29.206667+0000","last_fullsized":"2023-12-20T04:29:29.206667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.481203+0000","last_change":"2023-12-20T04:29:29.481203+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.481203+0000","last_undegraded":"2023-12-20T04:29:29.481203+0000","last_fullsized":"2023-12-20T04:29:29.481203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199892+0000","last_change":"2023-12-20T04:29:29.199892+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199892+0000","last_undegraded":"2023-12-20T04:29:29.199892+0000","last_fullsized":"2023-12-20T04:29:29.199892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203092+0000","last_change":"2023-12-20T04:29:29.203092+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203092+0000","last_undegraded":"2023-12-20T04:29:29.203092+0000","last_fullsized":"2023-12-20T04:29:29.203092+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.204371+0000","last_change":"2023-12-20T04:29:29.204371+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.204371+0000","last_undegraded":"2023-12-20T04:29:29.204371+0000","last_fullsized":"2023-12-20T04:29:29.204371+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:17:46.344276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203682+0000","last_change":"2023-12-20T04:29:29.203682+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203682+0000","last_undegraded":"2023-12-20T04:29:29.203682+0000","last_fullsized":"2023-12-20T04:29:29.203682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:35:23.040608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199827+0000","last_change":"2023-12-20T04:29:29.199827+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199827+0000","last_undegraded":"2023-12-20T04:29:29.199827+0000","last_fullsized":"2023-12-20T04:29:29.199827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:42:48.712967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202317+0000","last_change":"2023-12-20T04:29:29.202317+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202317+0000","last_undegraded":"2023-12-20T04:29:29.202317+0000","last_fullsized":"2023-12-20T04:29:29.202317+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:50:04.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199119+0000","last_change":"2023-12-20T04:29:29.199119+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199119+0000","last_undegraded":"2023-12-20T04:29:29.199119+0000","last_fullsized":"2023-12-20T04:29:29.199119+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:00.360055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.479944+0000","last_change":"2023-12-20T04:29:29.479944+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.479944+0000","last_undegraded":"2023-12-20T04:29:29.479944+0000","last_fullsized":"2023-12-20T04:29:29.479944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202337+0000","last_change":"2023-12-20T04:29:29.202337+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202337+0000","last_undegraded":"2023-12-20T04:29:29.202337+0000","last_fullsized":"2023-12-20T04:29:29.202337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199098+0000","last_change":"2023-12-20T04:29:29.199098+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199098+0000","last_undegraded":"2023-12-20T04:29:29.199098+0000","last_fullsized":"2023-12-20T04:29:29.199098+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:29:32.922907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203882+0000","last_change":"2023-12-20T04:29:29.203882+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203882+0000","last_undegraded":"2023-12-20T04:29:29.203882+0000","last_fullsized":"2023-12-20T04:29:29.203882+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202693+0000","last_change":"2023-12-20T04:29:29.202693+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202693+0000","last_undegraded":"2023-12-20T04:29:29.202693+0000","last_fullsized":"2023-12-20T04:29:29.202693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199100+0000","last_change":"2023-12-20T04:29:29.199100+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199100+0000","last_undegraded":"2023-12-20T04:29:29.199100+0000","last_fullsized":"2023-12-20T04:29:29.199100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:42.345201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199792+0000","last_change":"2023-12-20T04:29:29.199792+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199792+0000","last_undegraded":"2023-12-20T04:29:29.199792+0000","last_fullsized":"2023-12-20T04:29:29.199792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202839+0000","last_change":"2023-12-20T04:29:29.202839+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202839+0000","last_undegraded":"2023-12-20T04:29:29.202839+0000","last_fullsized":"2023-12-20T04:29:29.202839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:31:17.467772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.204190+0000","last_change":"2023-12-20T04:29:29.204190+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.204190+0000","last_undegraded":"2023-12-20T04:29:29.204190+0000","last_fullsized":"2023-12-20T04:29:29.204190+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202994+0000","last_change":"2023-12-20T04:29:29.202994+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202994+0000","last_undegraded":"2023-12-20T04:29:29.202994+0000","last_fullsized":"2023-12-20T04:29:29.202994+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:44:48.959029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.204884+0000","last_change":"2023-12-20T04:29:29.204884+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.204884+0000","last_undegraded":"2023-12-20T04:29:29.204884+0000","last_fullsized":"2023-12-20T04:29:29.204884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.480661+0000","last_change":"2023-12-20T04:29:29.480661+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.480661+0000","last_undegraded":"2023-12-20T04:29:29.480661+0000","last_fullsized":"2023-12-20T04:29:29.480661+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.206556+0000","last_change":"2023-12-20T04:29:29.206556+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.206556+0000","last_undegraded":"2023-12-20T04:29:29.206556+0000","last_fullsized":"2023-12-20T04:29:29.206556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203598+0000","last_change":"2023-12-20T04:29:29.203598+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203598+0000","last_undegraded":"2023-12-20T04:29:29.203598+0000","last_fullsized":"2023-12-20T04:29:29.203598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:00.538815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202904+0000","last_change":"2023-12-20T04:29:29.202904+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202904+0000","last_undegraded":"2023-12-20T04:29:29.202904+0000","last_fullsized":"2023-12-20T04:29:29.202904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202359+0000","last_change":"2023-12-20T04:29:29.202359+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202359+0000","last_undegraded":"2023-12-20T04:29:29.202359+0000","last_fullsized":"2023-12-20T04:29:29.202359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:00.162402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.480657+0000","last_change":"2023-12-20T04:29:29.480657+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.480657+0000","last_undegraded":"2023-12-20T04:29:29.480657+0000","last_fullsized":"2023-12-20T04:29:29.480657+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.201950+0000","last_change":"2023-12-20T04:29:29.201950+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.201950+0000","last_undegraded":"2023-12-20T04:29:29.201950+0000","last_fullsized":"2023-12-20T04:29:29.201950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:19:07.467835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.479944+0000","last_change":"2023-12-20T04:29:29.479944+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.479944+0000","last_undegraded":"2023-12-20T04:29:29.479944+0000","last_fullsized":"2023-12-20T04:29:29.479944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202137+0000","last_change":"2023-12-20T04:29:29.202137+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202137+0000","last_undegraded":"2023-12-20T04:29:29.202137+0000","last_fullsized":"2023-12-20T04:29:29.202137+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202917+0000","last_change":"2023-12-20T04:29:29.202917+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202917+0000","last_undegraded":"2023-12-20T04:29:29.202917+0000","last_fullsized":"2023-12-20T04:29:29.202917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:00:05.822869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199150+0000","last_change":"2023-12-20T04:29:29.199150+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199150+0000","last_undegraded":"2023-12-20T04:29:29.199150+0000","last_fullsized":"2023-12-20T04:29:29.199150+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:29.961522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.206658+0000","last_change":"2023-12-20T04:29:29.206658+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.206658+0000","last_undegraded":"2023-12-20T04:29:29.206658+0000","last_fullsized":"2023-12-20T04:29:29.206658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-12-20T04:29:25.089751+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:29:25.089751+0000","last_peered":"2023-12-20T04:29:25.089751+0000","last_clean":"2023-12-20T04:29:25.089751+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:29:25.089751+0000","last_undegraded":"2023-12-20T04:29:25.089751+0000","last_fullsized":"2023-12-20T04:29:25.089751+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":942080,"data_stored":721767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.57699999999999996}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.629},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.85699999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.68400000000000005}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.746}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.746},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.443}]},{"osd":3,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.73299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":360448,"data_stored":137216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65300000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.95399999999999996}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.89400000000000002}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.77000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.68899999999999995}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.752},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.627}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":360448,"data_stored":137216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.3700000000000001,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.378},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.7160000000000002,"5min":3.7160000000000002,"15min":3.7160000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0110000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":6.1399999999999997,"5min":6.1399999999999997,"15min":6.1399999999999997},"last":0.439}]},{"osd":4,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.59599999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.9830000000000001,"5min":3.9830000000000001,"15min":3.9830000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.57499999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.622},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.8900000000000001,"5min":3.8900000000000001,"15min":3.8900000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.6970000000000001,"5min":4.6970000000000001,"15min":4.6970000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.58799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.318}]},{"osd":1,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.49199999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.752}]},{"osd":7,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.39700000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.82499999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.748},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.879},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.93999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":942080,"data_stored":721767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.26900000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.436}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.55300000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:29:34.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:29:34.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.958+0000 7fe462cfc640 1 Processor -- start 2023-12-20T04:29:34.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.958+0000 7fe462cfc640 1 -- start start 2023-12-20T04:29:34.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.958+0000 7fe462cfc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe45c1603e0 0x7fe45c0cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.958+0000 7fe462cfc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe45c0cd840 0x7fe45c160af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.958+0000 7fe462cfc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe45c161030 0x7fe45c06c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.958+0000 7fe462cfc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45c0719b0 con 0x7fe45c1603e0 2023-12-20T04:29:34.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.958+0000 7fe462cfc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe45c1614c0 con 0x7fe45c0cd840 2023-12-20T04:29:34.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.958+0000 7fe462cfc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0713f0 con 0x7fe45c161030 2023-12-20T04:29:34.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe460a98640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe45c1603e0 0x7fe45c0cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe460a98640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe45c1603e0 0x7fe45c0cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45652/0 (socket says 172.21.15.32:45652) 2023-12-20T04:29:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe460a98640 1 -- 172.21.15.32:0/1095400157 learned_addr learned my addr 172.21.15.32:0/1095400157 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe45bfff640 1 --2- 172.21.15.32:0/1095400157 >> v2:172.21.15.39:3300/0 conn(0x7fe45c0cd840 0x7fe45c160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe461299640 1 --2- 172.21.15.32:0/1095400157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe45c161030 0x7fe45c06c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe460a98640 1 -- 172.21.15.32:0/1095400157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe45c161030 msgr2=0x7fe45c06c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe460a98640 1 --2- 172.21.15.32:0/1095400157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe45c161030 0x7fe45c06c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe460a98640 1 -- 172.21.15.32:0/1095400157 >> v2:172.21.15.39:3300/0 conn(0x7fe45c0cd840 msgr2=0x7fe45c160af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe460a98640 1 --2- 172.21.15.32:0/1095400157 >> v2:172.21.15.39:3300/0 conn(0x7fe45c0cd840 0x7fe45c160af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe460a98640 1 -- 172.21.15.32:0/1095400157 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe45c0716d0 con 0x7fe45c1603e0 2023-12-20T04:29:34.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe45bfff640 1 --2- 172.21.15.32:0/1095400157 >> v2:172.21.15.39:3300/0 conn(0x7fe45c0cd840 0x7fe45c160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:34.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe460a98640 1 --2- 172.21.15.32:0/1095400157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45c1603e0 0x7fe45c0cd300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe450009870 tx=0x7fe45001c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afa40891b5cfafe9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:34.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe45b7fe640 1 -- 172.21.15.32:0/1095400157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4500ab600 con 0x7fe45c1603e0 2023-12-20T04:29:34.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe45b7fe640 1 -- 172.21.15.32:0/1095400157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4500a7050 con 0x7fe45c1603e0 2023-12-20T04:29:34.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.962+0000 7fe45b7fe640 1 -- 172.21.15.32:0/1095400157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4500ab980 con 0x7fe45c1603e0 2023-12-20T04:29:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 -- 172.21.15.32:0/1095400157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45c1603e0 msgr2=0x7fe45c0cd300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 --2- 172.21.15.32:0/1095400157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45c1603e0 0x7fe45c0cd300 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7fe450009870 tx=0x7fe45001c0f0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 -- 172.21.15.32:0/1095400157 shutdown_connections 2023-12-20T04:29:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 --2- 172.21.15.32:0/1095400157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe45c161030 0x7fe45c06c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 --2- 172.21.15.32:0/1095400157 >> v2:172.21.15.39:3300/0 conn(0x7fe45c0cd840 0x7fe45c160af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 --2- 172.21.15.32:0/1095400157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45c1603e0 0x7fe45c0cd300 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 -- 172.21.15.32:0/1095400157 >> 172.21.15.32:0/1095400157 conn(0x7fe45c06e110 msgr2=0x7fe45c06e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 -- 172.21.15.32:0/1095400157 shutdown_connections 2023-12-20T04:29:34.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 -- 172.21.15.32:0/1095400157 wait complete. 2023-12-20T04:29:34.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 Processor -- start 2023-12-20T04:29:34.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 -- start start 2023-12-20T04:29:34.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe45c0cd840 0x7fe45c15d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe45c1603e0 0x7fe45c15f0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45c161030 0x7fe45c0b38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0c69b0 con 0x7fe45c161030 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c15da00 con 0x7fe45c0cd840 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe462cfc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe45c0c5da0 con 0x7fe45c1603e0 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe461299640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45c161030 0x7fe45c0b38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe461299640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45c161030 0x7fe45c0b38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45654/0 (socket says 172.21.15.32:45654) 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe461299640 1 -- 172.21.15.32:0/4260608570 learned_addr learned my addr 172.21.15.32:0/4260608570 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe460a98640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe45c0cd840 0x7fe45c15d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.966+0000 7fe45bfff640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe45c1603e0 0x7fe45c15f0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe461299640 1 -- 172.21.15.32:0/4260608570 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe45c1603e0 msgr2=0x7fe45c15f0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe461299640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe45c1603e0 0x7fe45c15f0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe461299640 1 -- 172.21.15.32:0/4260608570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe45c0cd840 msgr2=0x7fe45c15d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe461299640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe45c0cd840 0x7fe45c15d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:34.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe461299640 1 -- 172.21.15.32:0/4260608570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4500094d0 con 0x7fe45c161030 2023-12-20T04:29:34.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe461299640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45c161030 0x7fe45c0b38e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe454007e40 tx=0x7fe45400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:34.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe4597fa640 1 -- 172.21.15.32:0/4260608570 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe454014620 con 0x7fe45c161030 2023-12-20T04:29:34.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe4597fa640 1 -- 172.21.15.32:0/4260608570 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe45400ee90 con 0x7fe45c161030 2023-12-20T04:29:34.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe4597fa640 1 -- 172.21.15.32:0/4260608570 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4540149e0 con 0x7fe45c161030 2023-12-20T04:29:34.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe462cfc640 1 -- 172.21.15.32:0/4260608570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe45c0c6080 con 0x7fe45c161030 2023-12-20T04:29:34.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe462cfc640 1 -- 172.21.15.32:0/4260608570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe45c0c6640 con 0x7fe45c161030 2023-12-20T04:29:34.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe462cfc640 1 -- 172.21.15.32:0/4260608570 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe45c04e000 con 0x7fe45c161030 2023-12-20T04:29:34.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe4597fa640 1 -- 172.21.15.32:0/4260608570 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fe454014b40 con 0x7fe45c161030 2023-12-20T04:29:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe4597fa640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe440065a60 0x7fe440067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe4597fa640 1 -- 172.21.15.32:0/4260608570 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe454083a30 con 0x7fe45c161030 2023-12-20T04:29:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.970+0000 7fe460a98640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe440065a60 0x7fe440067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:34.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.974+0000 7fe460a98640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe440065a60 0x7fe440067f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4500ad000 tx=0x7fe4500a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:34.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:34.974+0000 7fe4597fa640 1 -- 172.21.15.32:0/4260608570 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe454051400 con 0x7fe45c161030 2023-12-20T04:29:35.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.130+0000 7fe462cfc640 1 -- 172.21.15.32:0/4260608570 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe45c1607c0 con 0x7fe440065a60 2023-12-20T04:29:35.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.138+0000 7fe4597fa640 1 -- 172.21.15.32:0/4260608570 <== mgr.4104 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278026 (secure 0 0 0) 0x7fe4500ab620 con 0x7fe440065a60 2023-12-20T04:29:35.142 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:35.145 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:29:35.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.138+0000 7fe462cfc640 1 -- 172.21.15.32:0/4260608570 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe440065a60 msgr2=0x7fe440067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.138+0000 7fe462cfc640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe440065a60 0x7fe440067f20 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7fe4500ad000 tx=0x7fe4500a6000 comp rx=0 tx=0).stop 2023-12-20T04:29:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.138+0000 7fe462cfc640 1 -- 172.21.15.32:0/4260608570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45c161030 msgr2=0x7fe45c0b38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.138+0000 7fe462cfc640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45c161030 0x7fe45c0b38e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7fe454007e40 tx=0x7fe45400e4b0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.142+0000 7fe462cfc640 1 -- 172.21.15.32:0/4260608570 shutdown_connections 2023-12-20T04:29:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.142+0000 7fe462cfc640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe440065a60 0x7fe440067f20 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.142+0000 7fe462cfc640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe45c161030 0x7fe45c0b38e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.142+0000 7fe462cfc640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe45c1603e0 0x7fe45c15f0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.142+0000 7fe462cfc640 1 --2- 172.21.15.32:0/4260608570 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe45c0cd840 0x7fe45c15d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.142+0000 7fe462cfc640 1 -- 172.21.15.32:0/4260608570 >> 172.21.15.32:0/4260608570 conn(0x7fe45c06e110 msgr2=0x7fe45c0ad540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.142+0000 7fe462cfc640 1 -- 172.21.15.32:0/4260608570 shutdown_connections 2023-12-20T04:29:35.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.142+0000 7fe462cfc640 1 -- 172.21.15.32:0/4260608570 wait complete. 2023-12-20T04:29:35.163 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-12-20T04:29:34.137815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240000,"kb_used_data":3904,"kb_used_omap":0,"kb_used_meta":236032,"kb_avail":754734720,"statfs":{"total":773094113280,"available":772848353280,"internally_reserved":0,"allocated":3997696,"data_stored":2243562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":241696768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.942055"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.206667+0000","last_change":"2023-12-20T04:29:29.206667+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.206667+0000","last_undegraded":"2023-12-20T04:29:29.206667+0000","last_fullsized":"2023-12-20T04:29:29.206667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.481203+0000","last_change":"2023-12-20T04:29:29.481203+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.481203+0000","last_undegraded":"2023-12-20T04:29:29.481203+0000","last_fullsized":"2023-12-20T04:29:29.481203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199892+0000","last_change":"2023-12-20T04:29:29.199892+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199892+0000","last_undegraded":"2023-12-20T04:29:29.199892+0000","last_fullsized":"2023-12-20T04:29:29.199892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203092+0000","last_change":"2023-12-20T04:29:29.203092+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203092+0000","last_undegraded":"2023-12-20T04:29:29.203092+0000","last_fullsized":"2023-12-20T04:29:29.203092+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.204371+0000","last_change":"2023-12-20T04:29:29.204371+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.204371+0000","last_undegraded":"2023-12-20T04:29:29.204371+0000","last_fullsized":"2023-12-20T04:29:29.204371+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:17:46.344276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203682+0000","last_change":"2023-12-20T04:29:29.203682+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203682+0000","last_undegraded":"2023-12-20T04:29:29.203682+0000","last_fullsized":"2023-12-20T04:29:29.203682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:35:23.040608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199827+0000","last_change":"2023-12-20T04:29:29.199827+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199827+0000","last_undegraded":"2023-12-20T04:29:29.199827+0000","last_fullsized":"2023-12-20T04:29:29.199827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:42:48.712967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202317+0000","last_change":"2023-12-20T04:29:29.202317+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202317+0000","last_undegraded":"2023-12-20T04:29:29.202317+0000","last_fullsized":"2023-12-20T04:29:29.202317+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:50:04.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199119+0000","last_change":"2023-12-20T04:29:29.199119+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199119+0000","last_undegraded":"2023-12-20T04:29:29.199119+0000","last_fullsized":"2023-12-20T04:29:29.199119+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:00.360055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.479944+0000","last_change":"2023-12-20T04:29:29.479944+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.479944+0000","last_undegraded":"2023-12-20T04:29:29.479944+0000","last_fullsized":"2023-12-20T04:29:29.479944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202337+0000","last_change":"2023-12-20T04:29:29.202337+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202337+0000","last_undegraded":"2023-12-20T04:29:29.202337+0000","last_fullsized":"2023-12-20T04:29:29.202337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199098+0000","last_change":"2023-12-20T04:29:29.199098+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199098+0000","last_undegraded":"2023-12-20T04:29:29.199098+0000","last_fullsized":"2023-12-20T04:29:29.199098+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:29:32.922907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203882+0000","last_change":"2023-12-20T04:29:29.203882+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203882+0000","last_undegraded":"2023-12-20T04:29:29.203882+0000","last_fullsized":"2023-12-20T04:29:29.203882+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202693+0000","last_change":"2023-12-20T04:29:29.202693+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202693+0000","last_undegraded":"2023-12-20T04:29:29.202693+0000","last_fullsized":"2023-12-20T04:29:29.202693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199100+0000","last_change":"2023-12-20T04:29:29.199100+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199100+0000","last_undegraded":"2023-12-20T04:29:29.199100+0000","last_fullsized":"2023-12-20T04:29:29.199100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:42.345201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199792+0000","last_change":"2023-12-20T04:29:29.199792+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199792+0000","last_undegraded":"2023-12-20T04:29:29.199792+0000","last_fullsized":"2023-12-20T04:29:29.199792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202839+0000","last_change":"2023-12-20T04:29:29.202839+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202839+0000","last_undegraded":"2023-12-20T04:29:29.202839+0000","last_fullsized":"2023-12-20T04:29:29.202839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:31:17.467772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.204190+0000","last_change":"2023-12-20T04:29:29.204190+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.204190+0000","last_undegraded":"2023-12-20T04:29:29.204190+0000","last_fullsized":"2023-12-20T04:29:29.204190+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202994+0000","last_change":"2023-12-20T04:29:29.202994+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202994+0000","last_undegraded":"2023-12-20T04:29:29.202994+0000","last_fullsized":"2023-12-20T04:29:29.202994+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:44:48.959029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.204884+0000","last_change":"2023-12-20T04:29:29.204884+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.204884+0000","last_undegraded":"2023-12-20T04:29:29.204884+0000","last_fullsized":"2023-12-20T04:29:29.204884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.480661+0000","last_change":"2023-12-20T04:29:29.480661+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.480661+0000","last_undegraded":"2023-12-20T04:29:29.480661+0000","last_fullsized":"2023-12-20T04:29:29.480661+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.206556+0000","last_change":"2023-12-20T04:29:29.206556+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.206556+0000","last_undegraded":"2023-12-20T04:29:29.206556+0000","last_fullsized":"2023-12-20T04:29:29.206556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203598+0000","last_change":"2023-12-20T04:29:29.203598+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203598+0000","last_undegraded":"2023-12-20T04:29:29.203598+0000","last_fullsized":"2023-12-20T04:29:29.203598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:00.538815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202904+0000","last_change":"2023-12-20T04:29:29.202904+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202904+0000","last_undegraded":"2023-12-20T04:29:29.202904+0000","last_fullsized":"2023-12-20T04:29:29.202904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202359+0000","last_change":"2023-12-20T04:29:29.202359+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202359+0000","last_undegraded":"2023-12-20T04:29:29.202359+0000","last_fullsized":"2023-12-20T04:29:29.202359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:00.162402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.480657+0000","last_change":"2023-12-20T04:29:29.480657+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.480657+0000","last_undegraded":"2023-12-20T04:29:29.480657+0000","last_fullsized":"2023-12-20T04:29:29.480657+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.201950+0000","last_change":"2023-12-20T04:29:29.201950+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.201950+0000","last_undegraded":"2023-12-20T04:29:29.201950+0000","last_fullsized":"2023-12-20T04:29:29.201950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:19:07.467835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.479944+0000","last_change":"2023-12-20T04:29:29.479944+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.479944+0000","last_undegraded":"2023-12-20T04:29:29.479944+0000","last_fullsized":"2023-12-20T04:29:29.479944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202137+0000","last_change":"2023-12-20T04:29:29.202137+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202137+0000","last_undegraded":"2023-12-20T04:29:29.202137+0000","last_fullsized":"2023-12-20T04:29:29.202137+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202917+0000","last_change":"2023-12-20T04:29:29.202917+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202917+0000","last_undegraded":"2023-12-20T04:29:29.202917+0000","last_fullsized":"2023-12-20T04:29:29.202917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:00:05.822869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199150+0000","last_change":"2023-12-20T04:29:29.199150+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199150+0000","last_undegraded":"2023-12-20T04:29:29.199150+0000","last_fullsized":"2023-12-20T04:29:29.199150+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:29.961522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.206658+0000","last_change":"2023-12-20T04:29:29.206658+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.206658+0000","last_undegraded":"2023-12-20T04:29:29.206658+0000","last_fullsized":"2023-12-20T04:29:29.206658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-12-20T04:29:25.089751+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:29:25.089751+0000","last_peered":"2023-12-20T04:29:25.089751+0000","last_clean":"2023-12-20T04:29:25.089751+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:29:25.089751+0000","last_undegraded":"2023-12-20T04:29:25.089751+0000","last_fullsized":"2023-12-20T04:29:25.089751+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":942080,"data_stored":721767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.57699999999999996}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.629},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.85699999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.68400000000000005}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.746}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.746},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.443}]},{"osd":3,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.73299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":360448,"data_stored":137216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65300000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.95399999999999996}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.89400000000000002}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.77000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.68899999999999995}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.752},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.627}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":360448,"data_stored":137216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.3700000000000001,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.378},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.7160000000000002,"5min":3.7160000000000002,"15min":3.7160000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0110000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":6.1399999999999997,"5min":6.1399999999999997,"15min":6.1399999999999997},"last":0.439}]},{"osd":4,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.59599999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.9830000000000001,"5min":3.9830000000000001,"15min":3.9830000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.57499999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.622},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.8900000000000001,"5min":3.8900000000000001,"15min":3.8900000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.6970000000000001,"5min":4.6970000000000001,"15min":4.6970000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.58799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.318}]},{"osd":1,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.49199999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.752}]},{"osd":7,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.39700000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.82499999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.748},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.879},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.93999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":942080,"data_stored":721767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.26900000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.436}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.55300000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:29:35.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:29:35.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa621d39640 1 Processor -- start 2023-12-20T04:29:35.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa621d39640 1 -- start start 2023-12-20T04:29:35.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa621d39640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa61c0d2740 0x7fa61c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa621d39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa61c0cfab0 0x7fa61c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa621d39640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa61c0cff00 con 0x7fa61c15cdb0 2023-12-20T04:29:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa621d39640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa61c15d190 con 0x7fa61c0d2740 2023-12-20T04:29:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa621d39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c0d16a0 con 0x7fa61c0cfab0 2023-12-20T04:29:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa61affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa61c0d2740 0x7fa61c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa61affd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa61c0d2740 0x7fa61c0cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48572/0 (socket says 172.21.15.32:48572) 2023-12-20T04:29:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa61affd640 1 -- 172.21.15.32:0/2633679051 learned_addr learned my addr 172.21.15.32:0/2633679051 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa61b7fe640 1 --1- 172.21.15.32:0/2633679051 >> v1:172.21.15.39:6789/0 conn(0x7fa61c15cdb0 0x7fa61c0d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:34308/0 (socket says 172.21.15.32:34308) 2023-12-20T04:29:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.346+0000 7fa61bfff640 1 --2- 172.21.15.32:0/2633679051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa61c0cfab0 0x7fa61c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa61affd640 1 -- 172.21.15.32:0/2633679051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa61c0cfab0 msgr2=0x7fa61c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa61affd640 1 --2- 172.21.15.32:0/2633679051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa61c0cfab0 0x7fa61c0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa61affd640 1 -- 172.21.15.32:0/2633679051 >> v1:172.21.15.39:6789/0 conn(0x7fa61c15cdb0 legacy=0x7fa61c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa61affd640 1 -- 172.21.15.32:0/2633679051 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa61c0d1980 con 0x7fa61c0d2740 2023-12-20T04:29:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa61affd640 1 --2- 172.21.15.32:0/2633679051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c0d2740 0x7fa61c0cf570 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa610009870 tx=0x7fa61001ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4bb3f6a9793f5dbf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa61a7fc640 1 -- 172.21.15.32:0/2633679051 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6100a8a30 con 0x7fa61c0d2740 2023-12-20T04:29:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa61a7fc640 1 -- 172.21.15.32:0/2633679051 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6100a8b90 con 0x7fa61c0d2740 2023-12-20T04:29:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa61a7fc640 1 -- 172.21.15.32:0/2633679051 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6100b0440 con 0x7fa61c0d2740 2023-12-20T04:29:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa621d39640 1 -- 172.21.15.32:0/2633679051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c0d2740 msgr2=0x7fa61c0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa621d39640 1 --2- 172.21.15.32:0/2633679051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c0d2740 0x7fa61c0cf570 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fa610009870 tx=0x7fa61001ed30 comp rx=0 tx=0).stop 2023-12-20T04:29:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa621d39640 1 -- 172.21.15.32:0/2633679051 shutdown_connections 2023-12-20T04:29:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa621d39640 1 --2- 172.21.15.32:0/2633679051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa61c0cfab0 0x7fa61c0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa621d39640 1 --2- 172.21.15.32:0/2633679051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c0d2740 0x7fa61c0cf570 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.350+0000 7fa621d39640 1 -- 172.21.15.32:0/2633679051 >> 172.21.15.32:0/2633679051 conn(0x7fa61c065ee0 msgr2=0x7fa61c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa621d39640 1 -- 172.21.15.32:0/2633679051 shutdown_connections 2023-12-20T04:29:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa621d39640 1 -- 172.21.15.32:0/2633679051 wait complete. 2023-12-20T04:29:35.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa621d39640 1 Processor -- start 2023-12-20T04:29:35.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa621d39640 1 -- start start 2023-12-20T04:29:35.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa621d39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa61c0cfab0 0x7fa61c15f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa621d39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa61c0d2740 0x7fa61c0c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa621d39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c15cdb0 0x7fa61c0c73e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa621d39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c15e920 con 0x7fa61c0d2740 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa621d39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c14cd40 con 0x7fa61c15cdb0 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa621d39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa61c15eb00 con 0x7fa61c0cfab0 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa61affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa61c0d2740 0x7fa61c0c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa61affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa61c0d2740 0x7fa61c0c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45670/0 (socket says 172.21.15.32:45670) 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa61affd640 1 -- 172.21.15.32:0/1192034917 learned_addr learned my addr 172.21.15.32:0/1192034917 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa61bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c15cdb0 0x7fa61c0c73e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa61b7fe640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa61c0cfab0 0x7fa61c15f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa61affd640 1 -- 172.21.15.32:0/1192034917 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa61c0cfab0 msgr2=0x7fa61c15f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa61affd640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa61c0cfab0 0x7fa61c15f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa61affd640 1 -- 172.21.15.32:0/1192034917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c15cdb0 msgr2=0x7fa61c0c73e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa61affd640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c15cdb0 0x7fa61c0c73e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.354+0000 7fa61affd640 1 -- 172.21.15.32:0/1192034917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6100094d0 con 0x7fa61c0d2740 2023-12-20T04:29:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.358+0000 7fa61affd640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa61c0d2740 0x7fa61c0c6b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa6100b3040 tx=0x7fa61001bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:35.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.358+0000 7fa603fff640 1 -- 172.21.15.32:0/1192034917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6100b0dd0 con 0x7fa61c0d2740 2023-12-20T04:29:35.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.358+0000 7fa603fff640 1 -- 172.21.15.32:0/1192034917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6100bb460 con 0x7fa61c0d2740 2023-12-20T04:29:35.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.358+0000 7fa621d39640 1 -- 172.21.15.32:0/1192034917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa61c15ede0 con 0x7fa61c0d2740 2023-12-20T04:29:35.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.358+0000 7fa603fff640 1 -- 172.21.15.32:0/1192034917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6100bb810 con 0x7fa61c0d2740 2023-12-20T04:29:35.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.358+0000 7fa621d39640 1 -- 172.21.15.32:0/1192034917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa61c0c7080 con 0x7fa61c0d2740 2023-12-20T04:29:35.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.358+0000 7fa621d39640 1 -- 172.21.15.32:0/1192034917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa61c04e000 con 0x7fa61c0d2740 2023-12-20T04:29:35.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.358+0000 7fa603fff640 1 -- 172.21.15.32:0/1192034917 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fa6100b83f0 con 0x7fa61c0d2740 2023-12-20T04:29:35.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.358+0000 7fa603fff640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa608065a60 0x7fa608067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.358+0000 7fa603fff640 1 -- 172.21.15.32:0/1192034917 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa610120a20 con 0x7fa61c0d2740 2023-12-20T04:29:35.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.358+0000 7fa61b7fe640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa608065a60 0x7fa608067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.362+0000 7fa61b7fe640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa608065a60 0x7fa608067f20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa60c000f70 tx=0x7fa60c007e50 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:35.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.362+0000 7fa603fff640 1 -- 172.21.15.32:0/1192034917 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa6100ee480 con 0x7fa61c0d2740 2023-12-20T04:29:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.518+0000 7fa621d39640 1 -- 172.21.15.32:0/1192034917 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa61c0d1f40 con 0x7fa608065a60 2023-12-20T04:29:35.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.526+0000 7fa603fff640 1 -- 172.21.15.32:0/1192034917 <== mgr.4104 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+278026 (secure 0 0 0) 0x7fa60c004a70 con 0x7fa608065a60 2023-12-20T04:29:35.531 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:35.533 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:29:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 -- 172.21.15.32:0/1192034917 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa608065a60 msgr2=0x7fa608067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa608065a60 0x7fa608067f20 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7fa60c000f70 tx=0x7fa60c007e50 comp rx=0 tx=0).stop 2023-12-20T04:29:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 -- 172.21.15.32:0/1192034917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa61c0d2740 msgr2=0x7fa61c0c6b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa61c0d2740 0x7fa61c0c6b40 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7fa6100b3040 tx=0x7fa61001bad0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 -- 172.21.15.32:0/1192034917 shutdown_connections 2023-12-20T04:29:35.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa608065a60 0x7fa608067f20 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa61c15cdb0 0x7fa61c0c73e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa61c0d2740 0x7fa61c0c6b40 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 --2- 172.21.15.32:0/1192034917 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa61c0cfab0 0x7fa61c15f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 -- 172.21.15.32:0/1192034917 >> 172.21.15.32:0/1192034917 conn(0x7fa61c065ee0 msgr2=0x7fa61c1532d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:35.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 -- 172.21.15.32:0/1192034917 shutdown_connections 2023-12-20T04:29:35.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.530+0000 7fa621d39640 1 -- 172.21.15.32:0/1192034917 wait complete. 2023-12-20T04:29:35.552 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":63,"stamp":"2023-12-20T04:29:34.137815+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":66,"acting":66,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":8,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":240000,"kb_used_data":3904,"kb_used_omap":0,"kb_used_meta":236032,"kb_avail":754734720,"statfs":{"total":773094113280,"available":772848353280,"internally_reserved":0,"allocated":3997696,"data_stored":2243562,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":241696768},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.942055"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.206667+0000","last_change":"2023-12-20T04:29:29.206667+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.206667+0000","last_undegraded":"2023-12-20T04:29:29.206667+0000","last_fullsized":"2023-12-20T04:29:29.206667+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.481203+0000","last_change":"2023-12-20T04:29:29.481203+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.481203+0000","last_undegraded":"2023-12-20T04:29:29.481203+0000","last_fullsized":"2023-12-20T04:29:29.481203+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199892+0000","last_change":"2023-12-20T04:29:29.199892+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199892+0000","last_undegraded":"2023-12-20T04:29:29.199892+0000","last_fullsized":"2023-12-20T04:29:29.199892+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203092+0000","last_change":"2023-12-20T04:29:29.203092+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203092+0000","last_undegraded":"2023-12-20T04:29:29.203092+0000","last_fullsized":"2023-12-20T04:29:29.203092+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.204371+0000","last_change":"2023-12-20T04:29:29.204371+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.204371+0000","last_undegraded":"2023-12-20T04:29:29.204371+0000","last_fullsized":"2023-12-20T04:29:29.204371+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:17:46.344276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203682+0000","last_change":"2023-12-20T04:29:29.203682+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203682+0000","last_undegraded":"2023-12-20T04:29:29.203682+0000","last_fullsized":"2023-12-20T04:29:29.203682+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:35:23.040608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199827+0000","last_change":"2023-12-20T04:29:29.199827+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199827+0000","last_undegraded":"2023-12-20T04:29:29.199827+0000","last_fullsized":"2023-12-20T04:29:29.199827+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:42:48.712967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202317+0000","last_change":"2023-12-20T04:29:29.202317+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202317+0000","last_undegraded":"2023-12-20T04:29:29.202317+0000","last_fullsized":"2023-12-20T04:29:29.202317+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:50:04.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199119+0000","last_change":"2023-12-20T04:29:29.199119+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199119+0000","last_undegraded":"2023-12-20T04:29:29.199119+0000","last_fullsized":"2023-12-20T04:29:29.199119+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:00.360055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.479944+0000","last_change":"2023-12-20T04:29:29.479944+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.479944+0000","last_undegraded":"2023-12-20T04:29:29.479944+0000","last_fullsized":"2023-12-20T04:29:29.479944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202337+0000","last_change":"2023-12-20T04:29:29.202337+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202337+0000","last_undegraded":"2023-12-20T04:29:29.202337+0000","last_fullsized":"2023-12-20T04:29:29.202337+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199098+0000","last_change":"2023-12-20T04:29:29.199098+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199098+0000","last_undegraded":"2023-12-20T04:29:29.199098+0000","last_fullsized":"2023-12-20T04:29:29.199098+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:29:32.922907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203882+0000","last_change":"2023-12-20T04:29:29.203882+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203882+0000","last_undegraded":"2023-12-20T04:29:29.203882+0000","last_fullsized":"2023-12-20T04:29:29.203882+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202693+0000","last_change":"2023-12-20T04:29:29.202693+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202693+0000","last_undegraded":"2023-12-20T04:29:29.202693+0000","last_fullsized":"2023-12-20T04:29:29.202693+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199100+0000","last_change":"2023-12-20T04:29:29.199100+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199100+0000","last_undegraded":"2023-12-20T04:29:29.199100+0000","last_fullsized":"2023-12-20T04:29:29.199100+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:42.345201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199792+0000","last_change":"2023-12-20T04:29:29.199792+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199792+0000","last_undegraded":"2023-12-20T04:29:29.199792+0000","last_fullsized":"2023-12-20T04:29:29.199792+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202839+0000","last_change":"2023-12-20T04:29:29.202839+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202839+0000","last_undegraded":"2023-12-20T04:29:29.202839+0000","last_fullsized":"2023-12-20T04:29:29.202839+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:31:17.467772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.204190+0000","last_change":"2023-12-20T04:29:29.204190+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.204190+0000","last_undegraded":"2023-12-20T04:29:29.204190+0000","last_fullsized":"2023-12-20T04:29:29.204190+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202994+0000","last_change":"2023-12-20T04:29:29.202994+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202994+0000","last_undegraded":"2023-12-20T04:29:29.202994+0000","last_fullsized":"2023-12-20T04:29:29.202994+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:44:48.959029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.204884+0000","last_change":"2023-12-20T04:29:29.204884+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.204884+0000","last_undegraded":"2023-12-20T04:29:29.204884+0000","last_fullsized":"2023-12-20T04:29:29.204884+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.480661+0000","last_change":"2023-12-20T04:29:29.480661+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.480661+0000","last_undegraded":"2023-12-20T04:29:29.480661+0000","last_fullsized":"2023-12-20T04:29:29.480661+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.206556+0000","last_change":"2023-12-20T04:29:29.206556+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.206556+0000","last_undegraded":"2023-12-20T04:29:29.206556+0000","last_fullsized":"2023-12-20T04:29:29.206556+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.203598+0000","last_change":"2023-12-20T04:29:29.203598+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.203598+0000","last_undegraded":"2023-12-20T04:29:29.203598+0000","last_fullsized":"2023-12-20T04:29:29.203598+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:00.538815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202904+0000","last_change":"2023-12-20T04:29:29.202904+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202904+0000","last_undegraded":"2023-12-20T04:29:29.202904+0000","last_fullsized":"2023-12-20T04:29:29.202904+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202359+0000","last_change":"2023-12-20T04:29:29.202359+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202359+0000","last_undegraded":"2023-12-20T04:29:29.202359+0000","last_fullsized":"2023-12-20T04:29:29.202359+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:00.162402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.480657+0000","last_change":"2023-12-20T04:29:29.480657+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.480657+0000","last_undegraded":"2023-12-20T04:29:29.480657+0000","last_fullsized":"2023-12-20T04:29:29.480657+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.201950+0000","last_change":"2023-12-20T04:29:29.201950+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.201950+0000","last_undegraded":"2023-12-20T04:29:29.201950+0000","last_fullsized":"2023-12-20T04:29:29.201950+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:19:07.467835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.479944+0000","last_change":"2023-12-20T04:29:29.479944+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.479944+0000","last_undegraded":"2023-12-20T04:29:29.479944+0000","last_fullsized":"2023-12-20T04:29:29.479944+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202137+0000","last_change":"2023-12-20T04:29:29.202137+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202137+0000","last_undegraded":"2023-12-20T04:29:29.202137+0000","last_fullsized":"2023-12-20T04:29:29.202137+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.202917+0000","last_change":"2023-12-20T04:29:29.202917+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.202917+0000","last_undegraded":"2023-12-20T04:29:29.202917+0000","last_fullsized":"2023-12-20T04:29:29.202917+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:00:05.822869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.199150+0000","last_change":"2023-12-20T04:29:29.199150+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.199150+0000","last_undegraded":"2023-12-20T04:29:29.199150+0000","last_fullsized":"2023-12-20T04:29:29.199150+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:29.961522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:29:30.195147+0000","last_change":"2023-12-20T04:29:30.195147+0000","last_active":"2023-12-20T04:29:30.195147+0000","last_peered":"2023-12-20T04:29:30.195147+0000","last_clean":"2023-12-20T04:29:30.195147+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:30.195147+0000","last_undegraded":"2023-12-20T04:29:30.195147+0000","last_fullsized":"2023-12-20T04:29:30.195147+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":1,"reported_epoch":32,"state":"creating+peering","last_fresh":"2023-12-20T04:29:29.206658+0000","last_change":"2023-12-20T04:29:29.206658+0000","last_active":"2023-12-20T04:29:29.192336+0000","last_peered":"2023-12-20T04:29:29.192336+0000","last_clean":"2023-12-20T04:29:29.192336+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:29:29.206658+0000","last_undegraded":"2023-12-20T04:29:29.206658+0000","last_fullsized":"2023-12-20T04:29:29.206658+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":171,"reported_epoch":31,"state":"active+clean","last_fresh":"2023-12-20T04:29:25.089751+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:29:25.089751+0000","last_peered":"2023-12-20T04:29:25.089751+0000","last_clean":"2023-12-20T04:29:25.089751+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:29:25.089751+0000","last_undegraded":"2023-12-20T04:29:25.089751+0000","last_fullsized":"2023-12-20T04:29:25.089751+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":64,"acting":64,"num_store_stats":7},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":942080,"data_stored":721767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.57699999999999996}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.629},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.85699999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.68400000000000005}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.746}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574863,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.746},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.60599999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.54900000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.443}]},{"osd":3,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.58099999999999996}]},{"osd":4,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.71699999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.86299999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.73299999999999998}]}]},{"osd":5,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27304,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344536,"statfs":{"total":96636764160,"available":96608804864,"internally_reserved":0,"allocated":360448,"data_stored":137216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.65300000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.92800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.54800000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.95399999999999996}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.69099999999999995}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.89400000000000002}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.77000000000000002}]}]},{"osd":4,"up_from":13,"seq":55834574865,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27292,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344548,"statfs":{"total":96636764160,"available":96608817152,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.52300000000000002},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.68899999999999995}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.752},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.67200000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.627}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.71199999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.74199999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.72399999999999998}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.84899999999999998}]}]},{"osd":3,"up_from":13,"seq":55834574864,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31400,"kb_used_data":352,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340440,"statfs":{"total":96636764160,"available":96604610560,"internally_reserved":0,"allocated":360448,"data_stored":137216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.3700000000000001,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.378},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.41499999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.7160000000000002,"5min":3.7160000000000002,"15min":3.7160000000000002},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0110000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":0.45500000000000002},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":6.1399999999999997,"5min":6.1399999999999997,"15min":6.1399999999999997},"last":0.439}]},{"osd":4,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.59599999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.9830000000000001,"5min":3.9830000000000001,"15min":3.9830000000000001},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.57499999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.622},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.8900000000000001,"5min":3.8900000000000001,"15min":3.8900000000000001},"last":0.67800000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.6970000000000001,"5min":4.6970000000000001,"15min":4.6970000000000001},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.58799999999999997}]}]},{"osd":2,"up_from":13,"seq":55834574863,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.318}]},{"osd":1,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.49199999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.54900000000000004}]},{"osd":4,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.56399999999999995}]},{"osd":5,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.60999999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.752}]},{"osd":7,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.68100000000000005}]}]},{"osd":1,"up_from":13,"seq":55834574865,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31388,"kb_used_data":340,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340452,"statfs":{"total":96636764160,"available":96604622848,"internally_reserved":0,"allocated":348160,"data_stored":131399,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.26500000000000001},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.30299999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.38500000000000001},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.39700000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.82499999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.748},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.80600000000000005}]},{"osd":6,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.879},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.79800000000000004}]},{"osd":7,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.93999999999999995}]}]},{"osd":0,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31968,"kb_used_data":920,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339872,"statfs":{"total":96636764160,"available":96604028928,"internally_reserved":0,"allocated":942080,"data_stored":721767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.26900000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.28399999999999997},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.436}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.57599999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.70699999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.55300000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.53900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:29:35.557 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3f is not active+clean 2023-12-20T04:29:35.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3e is not active+clean 2023-12-20T04:29:35.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3d is not active+clean 2023-12-20T04:29:35.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3c is not active+clean 2023-12-20T04:29:35.558 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.558 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3b is not active+clean 2023-12-20T04:29:35.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3a is not active+clean 2023-12-20T04:29:35.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.39 is not active+clean 2023-12-20T04:29:35.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.39', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.38 is not active+clean 2023-12-20T04:29:35.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.38', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.37 is not active+clean 2023-12-20T04:29:35.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.37', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.559 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.36 is not active+clean 2023-12-20T04:29:35.559 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.36', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.35 is not active+clean 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.35', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.34 is not active+clean 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.34', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.33 is not active+clean 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.33', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.32 is not active+clean 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.32', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.31 is not active+clean 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.31', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.30 is not active+clean 2023-12-20T04:29:35.560 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.30', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2f is not active+clean 2023-12-20T04:29:35.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2e is not active+clean 2023-12-20T04:29:35.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2d is not active+clean 2023-12-20T04:29:35.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.561 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2c is not active+clean 2023-12-20T04:29:35.561 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2b is not active+clean 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2a is not active+clean 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.29 is not active+clean 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.29', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.28 is not active+clean 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.28', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.27 is not active+clean 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.27', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.26 is not active+clean 2023-12-20T04:29:35.562 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.26', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.25 is not active+clean 2023-12-20T04:29:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.25', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.24 is not active+clean 2023-12-20T04:29:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.24', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.23 is not active+clean 2023-12-20T04:29:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.23', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.22 is not active+clean 2023-12-20T04:29:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.22', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.21 is not active+clean 2023-12-20T04:29:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.21', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.563 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.20 is not active+clean 2023-12-20T04:29:35.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.20', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1e is not active+clean 2023-12-20T04:29:35.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1f is not active+clean 2023-12-20T04:29:35.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.206667+0000', 'last_change': '2023-12-20T04:29:29.206667+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.206667+0000', 'last_undegraded': '2023-12-20T04:29:29.206667+0000', 'last_fullsized': '2023-12-20T04:29:29.206667+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 5], 'acting': [6, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-20T04:29:35.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1f is not active+clean 2023-12-20T04:29:35.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1e is not active+clean 2023-12-20T04:29:35.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.481203+0000', 'last_change': '2023-12-20T04:29:29.481203+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.481203+0000', 'last_undegraded': '2023-12-20T04:29:29.481203+0000', 'last_fullsized': '2023-12-20T04:29:29.481203+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 7], 'acting': [0, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-20T04:29:35.564 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1c is not active+clean 2023-12-20T04:29:35.564 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1d is not active+clean 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.199892+0000', 'last_change': '2023-12-20T04:29:29.199892+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.199892+0000', 'last_undegraded': '2023-12-20T04:29:29.199892+0000', 'last_fullsized': '2023-12-20T04:29:29.199892+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1d is not active+clean 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1c is not active+clean 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.203092+0000', 'last_change': '2023-12-20T04:29:29.203092+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.203092+0000', 'last_undegraded': '2023-12-20T04:29:29.203092+0000', 'last_fullsized': '2023-12-20T04:29:29.203092+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [2, 1], 'acting': [2, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 2, 'acting_primary': 2, 'purged_snaps': []} 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1a is not active+clean 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1b is not active+clean 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.204371+0000', 'last_change': '2023-12-20T04:29:29.204371+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.204371+0000', 'last_undegraded': '2023-12-20T04:29:29.204371+0000', 'last_fullsized': '2023-12-20T04:29:29.204371+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T05:17:46.344276+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1b is not active+clean 2023-12-20T04:29:35.565 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1a is not active+clean 2023-12-20T04:29:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.203682+0000', 'last_change': '2023-12-20T04:29:29.203682+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.203682+0000', 'last_undegraded': '2023-12-20T04:29:29.203682+0000', 'last_fullsized': '2023-12-20T04:29:29.203682+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T04:35:23.040608+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-20T04:29:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.18 is not active+clean 2023-12-20T04:29:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.18', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.19 is not active+clean 2023-12-20T04:29:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.19', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.199827+0000', 'last_change': '2023-12-20T04:29:29.199827+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.199827+0000', 'last_undegraded': '2023-12-20T04:29:29.199827+0000', 'last_fullsized': '2023-12-20T04:29:29.199827+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T09:42:48.712967+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 2], 'acting': [3, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-20T04:29:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.19 is not active+clean 2023-12-20T04:29:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.19', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.18 is not active+clean 2023-12-20T04:29:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.18', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.202317+0000', 'last_change': '2023-12-20T04:29:29.202317+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.202317+0000', 'last_undegraded': '2023-12-20T04:29:29.202317+0000', 'last_fullsized': '2023-12-20T04:29:29.202317+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T09:50:04.001754+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-20T04:29:35.566 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.16 is not active+clean 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.16', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.17 is not active+clean 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.17', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.199119+0000', 'last_change': '2023-12-20T04:29:29.199119+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.199119+0000', 'last_undegraded': '2023-12-20T04:29:29.199119+0000', 'last_fullsized': '2023-12-20T04:29:29.199119+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T05:00:00.360055+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.17 is not active+clean 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.17', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.16 is not active+clean 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.16', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.479944+0000', 'last_change': '2023-12-20T04:29:29.479944+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.479944+0000', 'last_undegraded': '2023-12-20T04:29:29.479944+0000', 'last_fullsized': '2023-12-20T04:29:29.479944+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 3], 'acting': [0, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.14 is not active+clean 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.14', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.15 is not active+clean 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.15', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.202337+0000', 'last_change': '2023-12-20T04:29:29.202337+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.202337+0000', 'last_undegraded': '2023-12-20T04:29:29.202337+0000', 'last_fullsized': '2023-12-20T04:29:29.202337+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-20T04:29:35.567 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.15 is not active+clean 2023-12-20T04:29:35.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.15', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.14 is not active+clean 2023-12-20T04:29:35.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.14', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.199098+0000', 'last_change': '2023-12-20T04:29:29.199098+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.199098+0000', 'last_undegraded': '2023-12-20T04:29:29.199098+0000', 'last_fullsized': '2023-12-20T04:29:29.199098+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T08:29:32.922907+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-20T04:29:35.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.12 is not active+clean 2023-12-20T04:29:35.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.12', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.13 is not active+clean 2023-12-20T04:29:35.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.13', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.203882+0000', 'last_change': '2023-12-20T04:29:29.203882+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.203882+0000', 'last_undegraded': '2023-12-20T04:29:29.203882+0000', 'last_fullsized': '2023-12-20T04:29:29.203882+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-20T04:29:35.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.13 is not active+clean 2023-12-20T04:29:35.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.13', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.568 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.12 is not active+clean 2023-12-20T04:29:35.568 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.12', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.202693+0000', 'last_change': '2023-12-20T04:29:29.202693+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.202693+0000', 'last_undegraded': '2023-12-20T04:29:29.202693+0000', 'last_fullsized': '2023-12-20T04:29:29.202693+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 6], 'acting': [1, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-20T04:29:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.10 is not active+clean 2023-12-20T04:29:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.10', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.11 is not active+clean 2023-12-20T04:29:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.11', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.199100+0000', 'last_change': '2023-12-20T04:29:29.199100+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.199100+0000', 'last_undegraded': '2023-12-20T04:29:29.199100+0000', 'last_fullsized': '2023-12-20T04:29:29.199100+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T12:17:42.345201+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-20T04:29:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.11 is not active+clean 2023-12-20T04:29:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.11', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.10 is not active+clean 2023-12-20T04:29:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.10', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.199792+0000', 'last_change': '2023-12-20T04:29:29.199792+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.199792+0000', 'last_undegraded': '2023-12-20T04:29:29.199792+0000', 'last_fullsized': '2023-12-20T04:29:29.199792+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 1], 'acting': [3, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-20T04:29:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.e is not active+clean 2023-12-20T04:29:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.e', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.569 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.f is not active+clean 2023-12-20T04:29:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.f', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.202839+0000', 'last_change': '2023-12-20T04:29:29.202839+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.202839+0000', 'last_undegraded': '2023-12-20T04:29:29.202839+0000', 'last_fullsized': '2023-12-20T04:29:29.202839+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T09:31:17.467772+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 3], 'acting': [1, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-20T04:29:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.f is not active+clean 2023-12-20T04:29:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.f', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.e is not active+clean 2023-12-20T04:29:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.e', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.204190+0000', 'last_change': '2023-12-20T04:29:29.204190+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.204190+0000', 'last_undegraded': '2023-12-20T04:29:29.204190+0000', 'last_fullsized': '2023-12-20T04:29:29.204190+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 6], 'acting': [4, 6], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-20T04:29:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.c is not active+clean 2023-12-20T04:29:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.c', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.d is not active+clean 2023-12-20T04:29:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.d', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.202994+0000', 'last_change': '2023-12-20T04:29:29.202994+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.202994+0000', 'last_undegraded': '2023-12-20T04:29:29.202994+0000', 'last_fullsized': '2023-12-20T04:29:29.202994+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T07:44:48.959029+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 2], 'acting': [4, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-20T04:29:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.d is not active+clean 2023-12-20T04:29:35.570 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.d', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.c is not active+clean 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.c', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.204884+0000', 'last_change': '2023-12-20T04:29:29.204884+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.204884+0000', 'last_undegraded': '2023-12-20T04:29:29.204884+0000', 'last_fullsized': '2023-12-20T04:29:29.204884+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 3], 'acting': [4, 3], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.a is not active+clean 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.a', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.b is not active+clean 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.b', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.480661+0000', 'last_change': '2023-12-20T04:29:29.480661+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.480661+0000', 'last_undegraded': '2023-12-20T04:29:29.480661+0000', 'last_fullsized': '2023-12-20T04:29:29.480661+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.b is not active+clean 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.b', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.a is not active+clean 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.a', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.206556+0000', 'last_change': '2023-12-20T04:29:29.206556+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.206556+0000', 'last_undegraded': '2023-12-20T04:29:29.206556+0000', 'last_fullsized': '2023-12-20T04:29:29.206556+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 1], 'acting': [6, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.8 is not active+clean 2023-12-20T04:29:35.571 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.8', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.9 is not active+clean 2023-12-20T04:29:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.9', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.203598+0000', 'last_change': '2023-12-20T04:29:29.203598+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.203598+0000', 'last_undegraded': '2023-12-20T04:29:29.203598+0000', 'last_fullsized': '2023-12-20T04:29:29.203598+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T10:39:00.538815+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 1], 'acting': [4, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-20T04:29:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.9 is not active+clean 2023-12-20T04:29:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.9', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.8 is not active+clean 2023-12-20T04:29:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.8', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.202904+0000', 'last_change': '2023-12-20T04:29:29.202904+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.202904+0000', 'last_undegraded': '2023-12-20T04:29:29.202904+0000', 'last_fullsized': '2023-12-20T04:29:29.202904+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [5, 7], 'acting': [5, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 5, 'acting_primary': 5, 'purged_snaps': []} 2023-12-20T04:29:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.6 is not active+clean 2023-12-20T04:29:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.6', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.7 is not active+clean 2023-12-20T04:29:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.7', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.202359+0000', 'last_change': '2023-12-20T04:29:29.202359+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.202359+0000', 'last_undegraded': '2023-12-20T04:29:29.202359+0000', 'last_fullsized': '2023-12-20T04:29:29.202359+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T11:05:00.162402+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-20T04:29:35.572 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.7 is not active+clean 2023-12-20T04:29:35.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.7', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.6 is not active+clean 2023-12-20T04:29:35.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.6', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.480657+0000', 'last_change': '2023-12-20T04:29:29.480657+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.480657+0000', 'last_undegraded': '2023-12-20T04:29:29.480657+0000', 'last_fullsized': '2023-12-20T04:29:29.480657+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 1], 'acting': [0, 1], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-20T04:29:35.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.5 is not active+clean 2023-12-20T04:29:35.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.5', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.4 is not active+clean 2023-12-20T04:29:35.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.4', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.201950+0000', 'last_change': '2023-12-20T04:29:29.201950+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.201950+0000', 'last_undegraded': '2023-12-20T04:29:29.201950+0000', 'last_fullsized': '2023-12-20T04:29:29.201950+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T05:19:07.467835+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 2], 'acting': [1, 2], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-20T04:29:35.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.2 is not active+clean 2023-12-20T04:29:35.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.2', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.573 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.3 is not active+clean 2023-12-20T04:29:35.573 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.3', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.479944+0000', 'last_change': '2023-12-20T04:29:29.479944+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.479944+0000', 'last_undegraded': '2023-12-20T04:29:29.479944+0000', 'last_fullsized': '2023-12-20T04:29:29.479944+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [0, 5], 'acting': [0, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 0, 'acting_primary': 0, 'purged_snaps': []} 2023-12-20T04:29:35.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.3 is not active+clean 2023-12-20T04:29:35.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.3', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.2 is not active+clean 2023-12-20T04:29:35.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.2', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.202137+0000', 'last_change': '2023-12-20T04:29:29.202137+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.202137+0000', 'last_undegraded': '2023-12-20T04:29:29.202137+0000', 'last_fullsized': '2023-12-20T04:29:29.202137+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [1, 5], 'acting': [1, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 1, 'acting_primary': 1, 'purged_snaps': []} 2023-12-20T04:29:35.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.0 is not active+clean 2023-12-20T04:29:35.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.0', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.1 is not active+clean 2023-12-20T04:29:35.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.1', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.202917+0000', 'last_change': '2023-12-20T04:29:29.202917+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.202917+0000', 'last_undegraded': '2023-12-20T04:29:29.202917+0000', 'last_fullsized': '2023-12-20T04:29:29.202917+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T12:00:05.822869+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [4, 5], 'acting': [4, 5], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 4, 'acting_primary': 4, 'purged_snaps': []} 2023-12-20T04:29:35.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.1 is not active+clean 2023-12-20T04:29:35.574 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.1', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.574 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.0 is not active+clean 2023-12-20T04:29:35.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.0', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.199150+0000', 'last_change': '2023-12-20T04:29:29.199150+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.199150+0000', 'last_undegraded': '2023-12-20T04:29:29.199150+0000', 'last_fullsized': '2023-12-20T04:29:29.199150+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T08:53:29.961522+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [3, 7], 'acting': [3, 7], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 3, 'acting_primary': 3, 'purged_snaps': []} 2023-12-20T04:29:35.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 5.4 is not active+clean 2023-12-20T04:29:35.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '5.4', 'version': "0'0", 'reported_seq': 0, 'reported_epoch': 0, 'state': 'unknown', 'last_fresh': '2023-12-20T04:29:30.195147+0000', 'last_change': '2023-12-20T04:29:30.195147+0000', 'last_active': '2023-12-20T04:29:30.195147+0000', 'last_peered': '2023-12-20T04:29:30.195147+0000', 'last_clean': '2023-12-20T04:29:30.195147+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:30.195147+0000', 'last_undegraded': '2023-12-20T04:29:30.195147+0000', 'last_fullsized': '2023-12-20T04:29:30.195147+0000', 'mapping_epoch': 0, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 0, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:30.195147+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': '--', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [], 'acting': [], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': -1, 'acting_primary': -1, 'purged_snaps': []} 2023-12-20T04:29:35.575 INFO:tasks.cephfs.filesystem.ceph_manager:PG 4.5 is not active+clean 2023-12-20T04:29:35.575 INFO:tasks.cephfs.filesystem.ceph_manager:{'pgid': '4.5', 'version': "0'0", 'reported_seq': 1, 'reported_epoch': 32, 'state': 'creating+peering', 'last_fresh': '2023-12-20T04:29:29.206658+0000', 'last_change': '2023-12-20T04:29:29.206658+0000', 'last_active': '2023-12-20T04:29:29.192336+0000', 'last_peered': '2023-12-20T04:29:29.192336+0000', 'last_clean': '2023-12-20T04:29:29.192336+0000', 'last_became_active': '0.000000', 'last_became_peered': '0.000000', 'last_unstale': '2023-12-20T04:29:29.206658+0000', 'last_undegraded': '2023-12-20T04:29:29.206658+0000', 'last_fullsized': '2023-12-20T04:29:29.206658+0000', 'mapping_epoch': 32, 'log_start': "0'0", 'ondisk_log_start': "0'0", 'created': 32, 'last_epoch_clean': 0, 'parent': '0.0', 'parent_split_bits': 0, 'last_scrub': "0'0", 'last_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_deep_scrub': "0'0", 'last_deep_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'last_clean_scrub_stamp': '2023-12-20T04:29:29.192336+0000', 'objects_scrubbed': 0, 'log_size': 0, 'log_dups_size': 0, 'ondisk_log_size': 0, 'stats_invalid': False, 'dirty_stats_invalid': False, 'omap_stats_invalid': False, 'hitset_stats_invalid': False, 'hitset_bytes_stats_invalid': False, 'pin_stats_invalid': False, 'manifest_stats_invalid': False, 'snaptrimq_len': 0, 'last_scrub_duration': 0, 'scrub_schedule': 'periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000', 'scrub_duration': 0, 'objects_trimmed': 0, 'snaptrim_duration': 0, 'stat_sum': {'num_bytes': 0, 'num_objects': 0, 'num_object_clones': 0, 'num_object_copies': 0, 'num_objects_missing_on_primary': 0, 'num_objects_missing': 0, 'num_objects_degraded': 0, 'num_objects_misplaced': 0, 'num_objects_unfound': 0, 'num_objects_dirty': 0, 'num_whiteouts': 0, 'num_read': 0, 'num_read_kb': 0, 'num_write': 0, 'num_write_kb': 0, 'num_scrub_errors': 0, 'num_shallow_scrub_errors': 0, 'num_deep_scrub_errors': 0, 'num_objects_recovered': 0, 'num_bytes_recovered': 0, 'num_keys_recovered': 0, 'num_objects_omap': 0, 'num_objects_hit_set_archive': 0, 'num_bytes_hit_set_archive': 0, 'num_flush': 0, 'num_flush_kb': 0, 'num_evict': 0, 'num_evict_kb': 0, 'num_promote': 0, 'num_flush_mode_high': 0, 'num_flush_mode_low': 0, 'num_evict_mode_some': 0, 'num_evict_mode_full': 0, 'num_objects_pinned': 0, 'num_legacy_snapsets': 0, 'num_large_omap_objects': 0, 'num_objects_manifest': 0, 'num_omap_bytes': 0, 'num_omap_keys': 0, 'num_objects_repaired': 0}, 'up': [6, 4], 'acting': [6, 4], 'avail_no_missing': [], 'object_location_counts': [], 'blocked_by': [], 'up_primary': 6, 'acting_primary': 6, 'purged_snaps': []} 2023-12-20T04:29:35.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph status --format=json 2023-12-20T04:29:35.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.734+0000 7f8c7df60640 1 Processor -- start 2023-12-20T04:29:35.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.734+0000 7f8c7df60640 1 -- start start 2023-12-20T04:29:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.734+0000 7f8c7df60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c780aed40 0x7f8c780ad9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.734+0000 7f8c7df60640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8c780b0d30 0x7f8c781593e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.734+0000 7f8c7df60640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c780706b0 con 0x7f8c78071ae0 2023-12-20T04:29:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.734+0000 7f8c7df60640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c78070900 con 0x7f8c780b0d30 2023-12-20T04:29:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.734+0000 7f8c7df60640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78070be0 con 0x7f8c780aed40 2023-12-20T04:29:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.734+0000 7f8c77fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8c780b0d30 0x7f8c781593e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.734+0000 7f8c77fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8c780b0d30 0x7f8c781593e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45684/0 (socket says 172.21.15.32:45684) 2023-12-20T04:29:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.734+0000 7f8c77fff640 1 -- 172.21.15.32:0/3899162621 learned_addr learned my addr 172.21.15.32:0/3899162621 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:35.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.734+0000 7f8c777fe640 1 --2- 172.21.15.32:0/3899162621 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c780aed40 0x7f8c780ad9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c77fff640 1 -- 172.21.15.32:0/3899162621 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c780aed40 msgr2=0x7f8c780ad9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c77fff640 1 --2- 172.21.15.32:0/3899162621 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c780aed40 0x7f8c780ad9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c77fff640 1 -- 172.21.15.32:0/3899162621 >> v1:172.21.15.39:6789/0 conn(0x7f8c78071ae0 legacy=0x7f8c780b0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c77fff640 1 -- 172.21.15.32:0/3899162621 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c78070ec0 con 0x7f8c780b0d30 2023-12-20T04:29:35.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c767fc640 1 -- 172.21.15.32:0/3899162621 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 511645973 0 0) 0x7f8c780706b0 con 0x7f8c78071ae0 2023-12-20T04:29:35.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c77fff640 1 --2- 172.21.15.32:0/3899162621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c780b0d30 0x7f8c781593e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009e50 tx=0x7f8c7001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=51066e25b63a610c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c767fc640 1 -- 172.21.15.32:0/3899162621 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c700a5690 con 0x7f8c780b0d30 2023-12-20T04:29:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c767fc640 1 -- 172.21.15.32:0/3899162621 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c700a57f0 con 0x7f8c780b0d30 2023-12-20T04:29:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c767fc640 1 -- 172.21.15.32:0/3899162621 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c700a5b80 con 0x7f8c780b0d30 2023-12-20T04:29:35.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c7df60640 1 -- 172.21.15.32:0/3899162621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c780b0d30 msgr2=0x7f8c781593e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c7df60640 1 --2- 172.21.15.32:0/3899162621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c780b0d30 0x7f8c781593e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f8c70009e50 tx=0x7f8c7001c8c0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c7df60640 1 -- 172.21.15.32:0/3899162621 shutdown_connections 2023-12-20T04:29:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c7df60640 1 --2- 172.21.15.32:0/3899162621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c780b0d30 0x7f8c781593e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c7df60640 1 --2- 172.21.15.32:0/3899162621 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c780aed40 0x7f8c780ad9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c7df60640 1 -- 172.21.15.32:0/3899162621 >> 172.21.15.32:0/3899162621 conn(0x7f8c7806a320 msgr2=0x7f8c780ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c7df60640 1 -- 172.21.15.32:0/3899162621 shutdown_connections 2023-12-20T04:29:35.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.738+0000 7f8c7df60640 1 -- 172.21.15.32:0/3899162621 wait complete. 2023-12-20T04:29:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c7df60640 1 Processor -- start 2023-12-20T04:29:35.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c7df60640 1 -- start start 2023-12-20T04:29:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c7df60640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c78071ae0 0x7f8c781449b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c7df60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c780aed40 0x7f8c780ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c7df60640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c780b0d30 0x7f8c780b1630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c7df60640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c780ad290 con 0x7f8c78071ae0 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c7df60640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c78165fb0 con 0x7f8c780b0d30 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c7df60640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c780accd0 con 0x7f8c780aed40 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c78071ae0 0x7f8c781449b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c78071ae0 0x7f8c781449b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45686/0 (socket says 172.21.15.32:45686) 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c777fe640 1 -- 172.21.15.32:0/1759765000 learned_addr learned my addr 172.21.15.32:0/1759765000 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c76ffd640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c780aed40 0x7f8c780ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c77fff640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c780b0d30 0x7f8c780b1630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c777fe640 1 -- 172.21.15.32:0/1759765000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c780aed40 msgr2=0x7f8c780ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c777fe640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c780aed40 0x7f8c780ac2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c777fe640 1 -- 172.21.15.32:0/1759765000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c780b0d30 msgr2=0x7f8c780b1630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c777fe640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c780b0d30 0x7f8c780b1630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c777fe640 1 -- 172.21.15.32:0/1759765000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c70009b30 con 0x7f8c78071ae0 2023-12-20T04:29:35.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.742+0000 7f8c777fe640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c78071ae0 0x7f8c781449b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800e870 tx=0x7f8c6800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:35.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.746+0000 7f8c5ffff640 1 -- 172.21.15.32:0/1759765000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c68009510 con 0x7f8c78071ae0 2023-12-20T04:29:35.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.746+0000 7f8c5ffff640 1 -- 172.21.15.32:0/1759765000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c6800eec0 con 0x7f8c78071ae0 2023-12-20T04:29:35.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.746+0000 7f8c5ffff640 1 -- 172.21.15.32:0/1759765000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c6800cb40 con 0x7f8c78071ae0 2023-12-20T04:29:35.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.746+0000 7f8c7df60640 1 -- 172.21.15.32:0/1759765000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c780acfb0 con 0x7f8c78071ae0 2023-12-20T04:29:35.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.746+0000 7f8c7df60640 1 -- 172.21.15.32:0/1759765000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c78144ef0 con 0x7f8c78071ae0 2023-12-20T04:29:35.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.746+0000 7f8c7df60640 1 -- 172.21.15.32:0/1759765000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c78165aa0 con 0x7f8c78071ae0 2023-12-20T04:29:35.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.746+0000 7f8c5ffff640 1 -- 172.21.15.32:0/1759765000 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f8c6800cca0 con 0x7f8c78071ae0 2023-12-20T04:29:35.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.746+0000 7f8c5ffff640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c64065a90 0x7f8c64067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:35.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.746+0000 7f8c5ffff640 1 -- 172.21.15.32:0/1759765000 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8c680832b0 con 0x7f8c78071ae0 2023-12-20T04:29:35.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.746+0000 7f8c76ffd640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c64065a90 0x7f8c64067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:35.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.750+0000 7f8c76ffd640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c64065a90 0x7f8c64067f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8c7815dbd0 tx=0x7f8c6c009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:35.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.750+0000 7f8c5ffff640 1 -- 172.21.15.32:0/1759765000 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8c78165aa0 con 0x7f8c78071ae0 2023-12-20T04:29:35.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.962+0000 7f8c7df60640 1 -- 172.21.15.32:0/1759765000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "status", "format": "json"} v 0) v1 -- 0x7f8c78165aa0 con 0x7f8c78071ae0 2023-12-20T04:29:35.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.962+0000 7f8c5ffff640 1 -- 172.21.15.32:0/1759765000 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json"}]=0 v0) v1 ==== 72+0+1655 (secure 0 0 0) 0x7f8c78165aa0 con 0x7f8c78071ae0 2023-12-20T04:29:35.968 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:35.968 INFO:teuthology.orchestra.run.smithi032.stdout:{"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","health":{"status":"HEALTH_OK","checks":{},"mutes":[]},"election_epoch":6,"quorum":[0,1,2],"quorum_names":["a","b","c"],"quorum_age":83,"monmap":{"epoch":1,"min_mon_release_name":"reef","num_mons":3},"osdmap":{"epoch":35,"num_osds":8,"num_up_osds":8,"osd_up_since":1703046504,"num_in_osds":8,"osd_in_since":1703046495,"num_remapped_pgs":0},"pgmap":{"pgs_by_state":[{"state_name":"unknown","count":64},{"state_name":"creating+peering","count":32},{"state_name":"active+clean","count":1}],"num_pgs":97,"num_pools":3,"num_objects":2,"data_bytes":590368,"bytes_used":245760000,"bytes_avail":772848353280,"bytes_total":773094113280,"unknown_pgs_ratio":0.65979379415512085,"inactive_pgs_ratio":0.32989689707756042},"fsmap":{"epoch":16,"id":2,"up":1,"in":1,"max":1,"by_rank":[{"filesystem_id":2,"rank":0,"name":"a","status":"up:active","gid":5031}],"up:standby":3},"mgrmap":{"available":true,"num_standbys":1,"modules":["iostat","nfs","restful"],"services":{}},"servicemap":{"epoch":3,"modified":"2023-12-20T04:29:26.136619+0000","services":{"mds":{"daemons":{"summary":"","a":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"b":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"c":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}},"d":{"start_epoch":0,"start_stamp":"0.000000","gid":0,"addr":"(unrecognized address family 0)/0","metadata":{},"task_status":{}}}}}},"progress_events":{}} 2023-12-20T04:29:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 -- 172.21.15.32:0/1759765000 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c64065a90 msgr2=0x7f8c64067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c64065a90 0x7f8c64067f50 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f8c7815dbd0 tx=0x7f8c6c009040 comp rx=0 tx=0).stop 2023-12-20T04:29:35.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 -- 172.21.15.32:0/1759765000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c78071ae0 msgr2=0x7f8c781449b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c78071ae0 0x7f8c781449b0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8c6800e870 tx=0x7f8c6800ec70 comp rx=0 tx=0).stop 2023-12-20T04:29:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 -- 172.21.15.32:0/1759765000 shutdown_connections 2023-12-20T04:29:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c64065a90 0x7f8c64067f50 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c780b0d30 0x7f8c780b1630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c780aed40 0x7f8c780ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 --2- 172.21.15.32:0/1759765000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c78071ae0 0x7f8c781449b0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:35.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 -- 172.21.15.32:0/1759765000 >> 172.21.15.32:0/1759765000 conn(0x7f8c7806a320 msgr2=0x7f8c780cc710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:35.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 -- 172.21.15.32:0/1759765000 shutdown_connections 2023-12-20T04:29:35.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:35.966+0000 7f8c7df60640 1 -- 172.21.15.32:0/1759765000 wait complete. 2023-12-20T04:29:35.990 INFO:tasks.cephfs.filesystem.ceph_manager:no progress seen, keeping timeout for now 2023-12-20T04:29:35.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:29:36.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc72a4fc640 1 Processor -- start 2023-12-20T04:29:36.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc72a4fc640 1 -- start start 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc72a4fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7241603e0 0x7fc7240cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc72a4fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7240cd840 0x7fc724160af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc72a4fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc724161030 0x7fc72406c640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc72a4fc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7240719b0 con 0x7fc7240cd840 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc72a4fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7241614c0 con 0x7fc724161030 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc72a4fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240713f0 con 0x7fc7241603e0 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc7237fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7240cd840 0x7fc724160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc7237fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc7240cd840 0x7fc724160af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45690/0 (socket says 172.21.15.32:45690) 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc7237fe640 1 -- 172.21.15.32:0/1709673077 learned_addr learned my addr 172.21.15.32:0/1709673077 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc723fff640 1 --2- 172.21.15.32:0/1709673077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7241603e0 0x7fc7240cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc728a99640 1 --2- 172.21.15.32:0/1709673077 >> v2:172.21.15.39:3300/0 conn(0x7fc724161030 0x7fc72406c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.130+0000 7fc7237fe640 1 -- 172.21.15.32:0/1709673077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7241603e0 msgr2=0x7fc7240cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc7237fe640 1 --2- 172.21.15.32:0/1709673077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7241603e0 0x7fc7240cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc7237fe640 1 -- 172.21.15.32:0/1709673077 >> v2:172.21.15.39:3300/0 conn(0x7fc724161030 msgr2=0x7fc72406c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:36.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc7237fe640 1 --2- 172.21.15.32:0/1709673077 >> v2:172.21.15.39:3300/0 conn(0x7fc724161030 0x7fc72406c640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc7237fe640 1 -- 172.21.15.32:0/1709673077 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7240716d0 con 0x7fc7240cd840 2023-12-20T04:29:36.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc7237fe640 1 --2- 172.21.15.32:0/1709673077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7240cd840 0x7fc724160af0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc7240b4c00 tx=0x7fc71801c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45792d6117439f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:36.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc722ffd640 1 -- 172.21.15.32:0/1709673077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180ab600 con 0x7fc7240cd840 2023-12-20T04:29:36.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc722ffd640 1 -- 172.21.15.32:0/1709673077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7180a7050 con 0x7fc7240cd840 2023-12-20T04:29:36.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc722ffd640 1 -- 172.21.15.32:0/1709673077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180ab980 con 0x7fc7240cd840 2023-12-20T04:29:36.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1709673077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7240cd840 msgr2=0x7fc724160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:36.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc72a4fc640 1 --2- 172.21.15.32:0/1709673077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7240cd840 0x7fc724160af0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fc7240b4c00 tx=0x7fc71801c0f0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1709673077 shutdown_connections 2023-12-20T04:29:36.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc72a4fc640 1 --2- 172.21.15.32:0/1709673077 >> v2:172.21.15.39:3300/0 conn(0x7fc724161030 0x7fc72406c640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc72a4fc640 1 --2- 172.21.15.32:0/1709673077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7240cd840 0x7fc724160af0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc72a4fc640 1 --2- 172.21.15.32:0/1709673077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7241603e0 0x7fc7240cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.134+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1709673077 >> 172.21.15.32:0/1709673077 conn(0x7fc72406e110 msgr2=0x7fc72406e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:36.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1709673077 shutdown_connections 2023-12-20T04:29:36.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1709673077 wait complete. 2023-12-20T04:29:36.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc72a4fc640 1 Processor -- start 2023-12-20T04:29:36.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc72a4fc640 1 -- start start 2023-12-20T04:29:36.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc72a4fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7240cd840 0x7fc72414cac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:36.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc72a4fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7241603e0 0x7fc7240b3450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:36.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc72a4fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc724161030 0x7fc7240b3990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:36.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc72a4fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240c2ba0 con 0x7fc7241603e0 2023-12-20T04:29:36.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc72a4fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240b2270 con 0x7fc724161030 2023-12-20T04:29:36.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc72a4fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7240b24c0 con 0x7fc7240cd840 2023-12-20T04:29:36.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc7237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7241603e0 0x7fc7240b3450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc7237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7241603e0 0x7fc7240b3450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45696/0 (socket says 172.21.15.32:45696) 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc7237fe640 1 -- 172.21.15.32:0/1419759253 learned_addr learned my addr 172.21.15.32:0/1419759253 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc723fff640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7240cd840 0x7fc72414cac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc728a99640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc724161030 0x7fc7240b3990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc7237fe640 1 -- 172.21.15.32:0/1419759253 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7240cd840 msgr2=0x7fc72414cac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc7237fe640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7240cd840 0x7fc72414cac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc7237fe640 1 -- 172.21.15.32:0/1419759253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc724161030 msgr2=0x7fc7240b3990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc7237fe640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc724161030 0x7fc7240b3990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.138+0000 7fc7237fe640 1 -- 172.21.15.32:0/1419759253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7180094d0 con 0x7fc7241603e0 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.142+0000 7fc7237fe640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7241603e0 0x7fc7240b3450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc7180ae040 tx=0x7fc718018e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.142+0000 7fc720ff9640 1 -- 172.21.15.32:0/1419759253 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180b4440 con 0x7fc7241603e0 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.142+0000 7fc720ff9640 1 -- 172.21.15.32:0/1419759253 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7180b45a0 con 0x7fc7241603e0 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.142+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1419759253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7240b27a0 con 0x7fc7241603e0 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.142+0000 7fc720ff9640 1 -- 172.21.15.32:0/1419759253 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7180b4930 con 0x7fc7241603e0 2023-12-20T04:29:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.142+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1419759253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7240b2d60 con 0x7fc7241603e0 2023-12-20T04:29:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.142+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1419759253 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc72404e000 con 0x7fc7241603e0 2023-12-20T04:29:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.142+0000 7fc720ff9640 1 -- 172.21.15.32:0/1419759253 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc7180b2070 con 0x7fc7241603e0 2023-12-20T04:29:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.142+0000 7fc720ff9640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc710065a60 0x7fc710067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.142+0000 7fc720ff9640 1 -- 172.21.15.32:0/1419759253 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc71811d110 con 0x7fc7241603e0 2023-12-20T04:29:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.142+0000 7fc723fff640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc710065a60 0x7fc710067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.146+0000 7fc723fff640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc710065a60 0x7fc710067f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7140045f0 tx=0x7fc714008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:36.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.146+0000 7fc720ff9640 1 -- 172.21.15.32:0/1419759253 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc7180eaae0 con 0x7fc7241603e0 2023-12-20T04:29:36.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.302+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1419759253 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc7241607c0 con 0x7fc710065a60 2023-12-20T04:29:36.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.310+0000 7fc720ff9640 1 -- 172.21.15.32:0/1419759253 <== mgr.4104 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288938 (secure 0 0 0) 0x7fc71400b740 con 0x7fc710065a60 2023-12-20T04:29:36.317 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:36.319 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:29:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1419759253 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc710065a60 msgr2=0x7fc710067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc710065a60 0x7fc710067f20 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fc7140045f0 tx=0x7fc714008040 comp rx=0 tx=0).stop 2023-12-20T04:29:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1419759253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7241603e0 msgr2=0x7fc7240b3450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7241603e0 0x7fc7240b3450 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fc7180ae040 tx=0x7fc718018e90 comp rx=0 tx=0).stop 2023-12-20T04:29:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1419759253 shutdown_connections 2023-12-20T04:29:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc710065a60 0x7fc710067f20 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc724161030 0x7fc7240b3990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7241603e0 0x7fc7240b3450 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 --2- 172.21.15.32:0/1419759253 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7240cd840 0x7fc72414cac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:36.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1419759253 >> 172.21.15.32:0/1419759253 conn(0x7fc72406e110 msgr2=0x7fc7241582b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:36.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1419759253 shutdown_connections 2023-12-20T04:29:36.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:36.314+0000 7fc72a4fc640 1 -- 172.21.15.32:0/1419759253 wait complete. 2023-12-20T04:29:36.338 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":64,"stamp":"2023-12-20T04:29:36.138147+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244520,"kb_used_data":4328,"kb_used_omap":0,"kb_used_meta":240128,"kb_avail":754730200,"statfs":{"total":773094113280,"available":772843724800,"internally_reserved":0,"allocated":4431872,"data_stored":2437924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":245891072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001038"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.189645+0000","last_change":"2023-12-20T04:29:30.655298+0000","last_active":"2023-12-20T04:29:32.189645+0000","last_peered":"2023-12-20T04:29:32.189645+0000","last_clean":"2023-12-20T04:29:32.189645+0000","last_became_active":"2023-12-20T04:29:30.654910+0000","last_became_peered":"2023-12-20T04:29:30.654910+0000","last_unstale":"2023-12-20T04:29:32.189645+0000","last_undegraded":"2023-12-20T04:29:32.189645+0000","last_fullsized":"2023-12-20T04:29:32.189645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:42.225653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854278+0000","last_change":"2023-12-20T04:29:30.634459+0000","last_active":"2023-12-20T04:29:31.854278+0000","last_peered":"2023-12-20T04:29:31.854278+0000","last_clean":"2023-12-20T04:29:31.854278+0000","last_became_active":"2023-12-20T04:29:30.633546+0000","last_became_peered":"2023-12-20T04:29:30.633546+0000","last_unstale":"2023-12-20T04:29:31.854278+0000","last_undegraded":"2023-12-20T04:29:31.854278+0000","last_fullsized":"2023-12-20T04:29:31.854278+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:06.756922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955380+0000","last_change":"2023-12-20T04:29:30.655533+0000","last_active":"2023-12-20T04:29:32.955380+0000","last_peered":"2023-12-20T04:29:32.955380+0000","last_clean":"2023-12-20T04:29:32.955380+0000","last_became_active":"2023-12-20T04:29:30.650108+0000","last_became_peered":"2023-12-20T04:29:30.650108+0000","last_unstale":"2023-12-20T04:29:32.955380+0000","last_undegraded":"2023-12-20T04:29:32.955380+0000","last_fullsized":"2023-12-20T04:29:32.955380+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:34:12.655851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622395+0000","last_change":"2023-12-20T04:29:30.646047+0000","last_active":"2023-12-20T04:29:31.622395+0000","last_peered":"2023-12-20T04:29:31.622395+0000","last_clean":"2023-12-20T04:29:31.622395+0000","last_became_active":"2023-12-20T04:29:30.645668+0000","last_became_peered":"2023-12-20T04:29:30.645668+0000","last_unstale":"2023-12-20T04:29:31.622395+0000","last_undegraded":"2023-12-20T04:29:31.622395+0000","last_fullsized":"2023-12-20T04:29:31.622395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:00:35.598112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928493+0000","last_change":"2023-12-20T04:29:30.625945+0000","last_active":"2023-12-20T04:29:31.928493+0000","last_peered":"2023-12-20T04:29:31.928493+0000","last_clean":"2023-12-20T04:29:31.928493+0000","last_became_active":"2023-12-20T04:29:30.625635+0000","last_became_peered":"2023-12-20T04:29:30.625635+0000","last_unstale":"2023-12-20T04:29:31.928493+0000","last_undegraded":"2023-12-20T04:29:31.928493+0000","last_fullsized":"2023-12-20T04:29:31.928493+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:29:33.925717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854088+0000","last_change":"2023-12-20T04:29:30.653519+0000","last_active":"2023-12-20T04:29:31.854088+0000","last_peered":"2023-12-20T04:29:31.854088+0000","last_clean":"2023-12-20T04:29:31.854088+0000","last_became_active":"2023-12-20T04:29:30.653242+0000","last_became_peered":"2023-12-20T04:29:30.653242+0000","last_unstale":"2023-12-20T04:29:31.854088+0000","last_undegraded":"2023-12-20T04:29:31.854088+0000","last_fullsized":"2023-12-20T04:29:31.854088+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:24:31.947504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625700+0000","last_change":"2023-12-20T04:29:30.661216+0000","last_active":"2023-12-20T04:29:31.625700+0000","last_peered":"2023-12-20T04:29:31.625700+0000","last_clean":"2023-12-20T04:29:31.625700+0000","last_became_active":"2023-12-20T04:29:30.660768+0000","last_became_peered":"2023-12-20T04:29:30.660768+0000","last_unstale":"2023-12-20T04:29:31.625700+0000","last_undegraded":"2023-12-20T04:29:31.625700+0000","last_fullsized":"2023-12-20T04:29:31.625700+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:11:19.882387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625467+0000","last_change":"2023-12-20T04:29:30.648261+0000","last_active":"2023-12-20T04:29:31.625467+0000","last_peered":"2023-12-20T04:29:31.625467+0000","last_clean":"2023-12-20T04:29:31.625467+0000","last_became_active":"2023-12-20T04:29:30.647929+0000","last_became_peered":"2023-12-20T04:29:30.647929+0000","last_unstale":"2023-12-20T04:29:31.625467+0000","last_undegraded":"2023-12-20T04:29:31.625467+0000","last_fullsized":"2023-12-20T04:29:31.625467+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:19:15.119662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626646+0000","last_change":"2023-12-20T04:29:30.652472+0000","last_active":"2023-12-20T04:29:31.626646+0000","last_peered":"2023-12-20T04:29:31.626646+0000","last_clean":"2023-12-20T04:29:31.626646+0000","last_became_active":"2023-12-20T04:29:30.652114+0000","last_became_peered":"2023-12-20T04:29:30.652114+0000","last_unstale":"2023-12-20T04:29:31.626646+0000","last_undegraded":"2023-12-20T04:29:31.626646+0000","last_fullsized":"2023-12-20T04:29:31.626646+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:13.683201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622430+0000","last_change":"2023-12-20T04:29:30.642253+0000","last_active":"2023-12-20T04:29:31.622430+0000","last_peered":"2023-12-20T04:29:31.622430+0000","last_clean":"2023-12-20T04:29:31.622430+0000","last_became_active":"2023-12-20T04:29:30.641994+0000","last_became_peered":"2023-12-20T04:29:30.641994+0000","last_unstale":"2023-12-20T04:29:31.622430+0000","last_undegraded":"2023-12-20T04:29:31.622430+0000","last_fullsized":"2023-12-20T04:29:31.622430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:36.244013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626151+0000","last_change":"2023-12-20T04:29:30.656023+0000","last_active":"2023-12-20T04:29:31.626151+0000","last_peered":"2023-12-20T04:29:31.626151+0000","last_clean":"2023-12-20T04:29:31.626151+0000","last_became_active":"2023-12-20T04:29:30.655720+0000","last_became_peered":"2023-12-20T04:29:30.655720+0000","last_unstale":"2023-12-20T04:29:31.626151+0000","last_undegraded":"2023-12-20T04:29:31.626151+0000","last_fullsized":"2023-12-20T04:29:31.626151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:34:12.655851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955241+0000","last_change":"2023-12-20T04:29:30.640182+0000","last_active":"2023-12-20T04:29:32.955241+0000","last_peered":"2023-12-20T04:29:32.955241+0000","last_clean":"2023-12-20T04:29:32.955241+0000","last_became_active":"2023-12-20T04:29:30.639672+0000","last_became_peered":"2023-12-20T04:29:30.639672+0000","last_unstale":"2023-12-20T04:29:32.955241+0000","last_undegraded":"2023-12-20T04:29:32.955241+0000","last_fullsized":"2023-12-20T04:29:32.955241+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:24:31.947504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928428+0000","last_change":"2023-12-20T04:29:30.635754+0000","last_active":"2023-12-20T04:29:31.928428+0000","last_peered":"2023-12-20T04:29:31.928428+0000","last_clean":"2023-12-20T04:29:31.928428+0000","last_became_active":"2023-12-20T04:29:30.635447+0000","last_became_peered":"2023-12-20T04:29:30.635447+0000","last_unstale":"2023-12-20T04:29:31.928428+0000","last_undegraded":"2023-12-20T04:29:31.928428+0000","last_fullsized":"2023-12-20T04:29:31.928428+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:45.142508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854122+0000","last_change":"2023-12-20T04:29:30.627375+0000","last_active":"2023-12-20T04:29:31.854122+0000","last_peered":"2023-12-20T04:29:31.854122+0000","last_clean":"2023-12-20T04:29:31.854122+0000","last_became_active":"2023-12-20T04:29:30.627110+0000","last_became_peered":"2023-12-20T04:29:30.627110+0000","last_unstale":"2023-12-20T04:29:31.854122+0000","last_undegraded":"2023-12-20T04:29:31.854122+0000","last_fullsized":"2023-12-20T04:29:31.854122+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:49.286363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.954997+0000","last_change":"2023-12-20T04:29:30.653951+0000","last_active":"2023-12-20T04:29:32.954997+0000","last_peered":"2023-12-20T04:29:32.954997+0000","last_clean":"2023-12-20T04:29:32.954997+0000","last_became_active":"2023-12-20T04:29:30.653649+0000","last_became_peered":"2023-12-20T04:29:30.653649+0000","last_unstale":"2023-12-20T04:29:32.954997+0000","last_undegraded":"2023-12-20T04:29:32.954997+0000","last_fullsized":"2023-12-20T04:29:32.954997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:49.286363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625807+0000","last_change":"2023-12-20T04:29:30.653428+0000","last_active":"2023-12-20T04:29:31.625807+0000","last_peered":"2023-12-20T04:29:31.625807+0000","last_clean":"2023-12-20T04:29:31.625807+0000","last_became_active":"2023-12-20T04:29:30.653143+0000","last_became_peered":"2023-12-20T04:29:30.653143+0000","last_unstale":"2023-12-20T04:29:31.625807+0000","last_undegraded":"2023-12-20T04:29:31.625807+0000","last_fullsized":"2023-12-20T04:29:31.625807+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:49.286363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626369+0000","last_change":"2023-12-20T04:29:30.661644+0000","last_active":"2023-12-20T04:29:31.626369+0000","last_peered":"2023-12-20T04:29:31.626369+0000","last_clean":"2023-12-20T04:29:31.626369+0000","last_became_active":"2023-12-20T04:29:30.661393+0000","last_became_peered":"2023-12-20T04:29:30.661393+0000","last_unstale":"2023-12-20T04:29:31.626369+0000","last_undegraded":"2023-12-20T04:29:31.626369+0000","last_fullsized":"2023-12-20T04:29:31.626369+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:06.756922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854314+0000","last_change":"2023-12-20T04:29:30.638164+0000","last_active":"2023-12-20T04:29:31.854314+0000","last_peered":"2023-12-20T04:29:31.854314+0000","last_clean":"2023-12-20T04:29:31.854314+0000","last_became_active":"2023-12-20T04:29:30.637901+0000","last_became_peered":"2023-12-20T04:29:30.637901+0000","last_unstale":"2023-12-20T04:29:31.854314+0000","last_undegraded":"2023-12-20T04:29:31.854314+0000","last_fullsized":"2023-12-20T04:29:31.854314+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:00:58.157886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854084+0000","last_change":"2023-12-20T04:29:30.634579+0000","last_active":"2023-12-20T04:29:31.854084+0000","last_peered":"2023-12-20T04:29:31.854084+0000","last_clean":"2023-12-20T04:29:31.854084+0000","last_became_active":"2023-12-20T04:29:30.634314+0000","last_became_peered":"2023-12-20T04:29:30.634314+0000","last_unstale":"2023-12-20T04:29:31.854084+0000","last_undegraded":"2023-12-20T04:29:31.854084+0000","last_fullsized":"2023-12-20T04:29:31.854084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:29.127931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928453+0000","last_change":"2023-12-20T04:29:30.645563+0000","last_active":"2023-12-20T04:29:31.928453+0000","last_peered":"2023-12-20T04:29:31.928453+0000","last_clean":"2023-12-20T04:29:31.928453+0000","last_became_active":"2023-12-20T04:29:30.645324+0000","last_became_peered":"2023-12-20T04:29:30.645324+0000","last_unstale":"2023-12-20T04:29:31.928453+0000","last_undegraded":"2023-12-20T04:29:31.928453+0000","last_fullsized":"2023-12-20T04:29:31.928453+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:50.074492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625811+0000","last_change":"2023-12-20T04:29:30.651654+0000","last_active":"2023-12-20T04:29:31.625811+0000","last_peered":"2023-12-20T04:29:31.625811+0000","last_clean":"2023-12-20T04:29:31.625811+0000","last_became_active":"2023-12-20T04:29:30.651333+0000","last_became_peered":"2023-12-20T04:29:30.651333+0000","last_unstale":"2023-12-20T04:29:31.625811+0000","last_undegraded":"2023-12-20T04:29:31.625811+0000","last_fullsized":"2023-12-20T04:29:31.625811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:43.348011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625561+0000","last_change":"2023-12-20T04:29:30.649495+0000","last_active":"2023-12-20T04:29:31.625561+0000","last_peered":"2023-12-20T04:29:31.625561+0000","last_clean":"2023-12-20T04:29:31.625561+0000","last_became_active":"2023-12-20T04:29:30.649215+0000","last_became_peered":"2023-12-20T04:29:30.649215+0000","last_unstale":"2023-12-20T04:29:31.625561+0000","last_undegraded":"2023-12-20T04:29:31.625561+0000","last_fullsized":"2023-12-20T04:29:31.625561+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:45.142508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.930181+0000","last_change":"2023-12-20T04:29:30.644827+0000","last_active":"2023-12-20T04:29:32.930181+0000","last_peered":"2023-12-20T04:29:32.930181+0000","last_clean":"2023-12-20T04:29:32.930181+0000","last_became_active":"2023-12-20T04:29:30.641962+0000","last_became_peered":"2023-12-20T04:29:30.641962+0000","last_unstale":"2023-12-20T04:29:32.930181+0000","last_undegraded":"2023-12-20T04:29:32.930181+0000","last_fullsized":"2023-12-20T04:29:32.930181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:44:31.379624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955000+0000","last_change":"2023-12-20T04:29:30.645099+0000","last_active":"2023-12-20T04:29:32.955000+0000","last_peered":"2023-12-20T04:29:32.955000+0000","last_clean":"2023-12-20T04:29:32.955000+0000","last_became_active":"2023-12-20T04:29:30.644820+0000","last_became_peered":"2023-12-20T04:29:30.644820+0000","last_unstale":"2023-12-20T04:29:32.955000+0000","last_undegraded":"2023-12-20T04:29:32.955000+0000","last_fullsized":"2023-12-20T04:29:32.955000+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:13:08.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622700+0000","last_change":"2023-12-20T04:29:30.651618+0000","last_active":"2023-12-20T04:29:31.622700+0000","last_peered":"2023-12-20T04:29:31.622700+0000","last_clean":"2023-12-20T04:29:31.622700+0000","last_became_active":"2023-12-20T04:29:30.651281+0000","last_became_peered":"2023-12-20T04:29:30.651281+0000","last_unstale":"2023-12-20T04:29:31.622700+0000","last_undegraded":"2023-12-20T04:29:31.622700+0000","last_fullsized":"2023-12-20T04:29:31.622700+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:13.683201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854368+0000","last_change":"2023-12-20T04:29:30.654081+0000","last_active":"2023-12-20T04:29:31.854368+0000","last_peered":"2023-12-20T04:29:31.854368+0000","last_clean":"2023-12-20T04:29:31.854368+0000","last_became_active":"2023-12-20T04:29:30.653794+0000","last_became_peered":"2023-12-20T04:29:30.653794+0000","last_unstale":"2023-12-20T04:29:31.854368+0000","last_undegraded":"2023-12-20T04:29:31.854368+0000","last_fullsized":"2023-12-20T04:29:31.854368+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:41.192000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622635+0000","last_change":"2023-12-20T04:29:30.647664+0000","last_active":"2023-12-20T04:29:31.622635+0000","last_peered":"2023-12-20T04:29:31.622635+0000","last_clean":"2023-12-20T04:29:31.622635+0000","last_became_active":"2023-12-20T04:29:30.646694+0000","last_became_peered":"2023-12-20T04:29:30.646694+0000","last_unstale":"2023-12-20T04:29:31.622635+0000","last_undegraded":"2023-12-20T04:29:31.622635+0000","last_fullsized":"2023-12-20T04:29:31.622635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:42.395721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955333+0000","last_change":"2023-12-20T04:29:30.636178+0000","last_active":"2023-12-20T04:29:32.955333+0000","last_peered":"2023-12-20T04:29:32.955333+0000","last_clean":"2023-12-20T04:29:32.955333+0000","last_became_active":"2023-12-20T04:29:30.635857+0000","last_became_peered":"2023-12-20T04:29:30.635857+0000","last_unstale":"2023-12-20T04:29:32.955333+0000","last_undegraded":"2023-12-20T04:29:32.955333+0000","last_fullsized":"2023-12-20T04:29:32.955333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:57.967703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626019+0000","last_change":"2023-12-20T04:29:30.644814+0000","last_active":"2023-12-20T04:29:31.626019+0000","last_peered":"2023-12-20T04:29:31.626019+0000","last_clean":"2023-12-20T04:29:31.626019+0000","last_became_active":"2023-12-20T04:29:30.644526+0000","last_became_peered":"2023-12-20T04:29:30.644526+0000","last_unstale":"2023-12-20T04:29:31.626019+0000","last_undegraded":"2023-12-20T04:29:31.626019+0000","last_fullsized":"2023-12-20T04:29:31.626019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:50.074492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625710+0000","last_change":"2023-12-20T04:29:30.661180+0000","last_active":"2023-12-20T04:29:31.625710+0000","last_peered":"2023-12-20T04:29:31.625710+0000","last_clean":"2023-12-20T04:29:31.625710+0000","last_became_active":"2023-12-20T04:29:30.660878+0000","last_became_peered":"2023-12-20T04:29:30.660878+0000","last_unstale":"2023-12-20T04:29:31.625710+0000","last_undegraded":"2023-12-20T04:29:31.625710+0000","last_fullsized":"2023-12-20T04:29:31.625710+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:24:31.947504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.189678+0000","last_change":"2023-12-20T04:29:30.636109+0000","last_active":"2023-12-20T04:29:32.189678+0000","last_peered":"2023-12-20T04:29:32.189678+0000","last_clean":"2023-12-20T04:29:32.189678+0000","last_became_active":"2023-12-20T04:29:30.635710+0000","last_became_peered":"2023-12-20T04:29:30.635710+0000","last_unstale":"2023-12-20T04:29:32.189678+0000","last_undegraded":"2023-12-20T04:29:32.189678+0000","last_fullsized":"2023-12-20T04:29:32.189678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:30.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625574+0000","last_change":"2023-12-20T04:29:30.648795+0000","last_active":"2023-12-20T04:29:31.625574+0000","last_peered":"2023-12-20T04:29:31.625574+0000","last_clean":"2023-12-20T04:29:31.625574+0000","last_became_active":"2023-12-20T04:29:30.648490+0000","last_became_peered":"2023-12-20T04:29:30.648490+0000","last_unstale":"2023-12-20T04:29:31.625574+0000","last_undegraded":"2023-12-20T04:29:31.625574+0000","last_fullsized":"2023-12-20T04:29:31.625574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:36.244013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.930570+0000","last_change":"2023-12-20T04:29:30.642782+0000","last_active":"2023-12-20T04:29:32.930570+0000","last_peered":"2023-12-20T04:29:32.930570+0000","last_clean":"2023-12-20T04:29:32.930570+0000","last_became_active":"2023-12-20T04:29:30.642444+0000","last_became_peered":"2023-12-20T04:29:30.642444+0000","last_unstale":"2023-12-20T04:29:32.930570+0000","last_undegraded":"2023-12-20T04:29:32.930570+0000","last_fullsized":"2023-12-20T04:29:32.930570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:13.683201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.655050+0000","last_change":"2023-12-20T04:29:30.233682+0000","last_active":"2023-12-20T04:29:30.655050+0000","last_peered":"2023-12-20T04:29:30.655050+0000","last_clean":"2023-12-20T04:29:30.655050+0000","last_became_active":"2023-12-20T04:29:30.233450+0000","last_became_peered":"2023-12-20T04:29:30.233450+0000","last_unstale":"2023-12-20T04:29:30.655050+0000","last_undegraded":"2023-12-20T04:29:30.655050+0000","last_fullsized":"2023-12-20T04:29:30.655050+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.189749+0000","last_change":"2023-12-20T04:29:30.655215+0000","last_active":"2023-12-20T04:29:32.189749+0000","last_peered":"2023-12-20T04:29:32.189749+0000","last_clean":"2023-12-20T04:29:32.189749+0000","last_became_active":"2023-12-20T04:29:30.654666+0000","last_became_peered":"2023-12-20T04:29:30.654666+0000","last_unstale":"2023-12-20T04:29:32.189749+0000","last_undegraded":"2023-12-20T04:29:32.189749+0000","last_fullsized":"2023-12-20T04:29:32.189749+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:18.283893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.633102+0000","last_change":"2023-12-20T04:29:30.225518+0000","last_active":"2023-12-20T04:29:30.633102+0000","last_peered":"2023-12-20T04:29:30.633102+0000","last_clean":"2023-12-20T04:29:30.633102+0000","last_became_active":"2023-12-20T04:29:30.225211+0000","last_became_peered":"2023-12-20T04:29:30.225211+0000","last_unstale":"2023-12-20T04:29:30.633102+0000","last_undegraded":"2023-12-20T04:29:30.633102+0000","last_fullsized":"2023-12-20T04:29:30.633102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626443+0000","last_change":"2023-12-20T04:29:30.652588+0000","last_active":"2023-12-20T04:29:31.626443+0000","last_peered":"2023-12-20T04:29:31.626443+0000","last_clean":"2023-12-20T04:29:31.626443+0000","last_became_active":"2023-12-20T04:29:30.652296+0000","last_became_peered":"2023-12-20T04:29:30.652296+0000","last_unstale":"2023-12-20T04:29:31.626443+0000","last_undegraded":"2023-12-20T04:29:31.626443+0000","last_fullsized":"2023-12-20T04:29:31.626443+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:30.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.639776+0000","last_change":"2023-12-20T04:29:30.233148+0000","last_active":"2023-12-20T04:29:30.639776+0000","last_peered":"2023-12-20T04:29:30.639776+0000","last_clean":"2023-12-20T04:29:30.639776+0000","last_became_active":"2023-12-20T04:29:30.232686+0000","last_became_peered":"2023-12-20T04:29:30.232686+0000","last_unstale":"2023-12-20T04:29:30.639776+0000","last_undegraded":"2023-12-20T04:29:30.639776+0000","last_fullsized":"2023-12-20T04:29:30.639776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622736+0000","last_change":"2023-12-20T04:29:30.648689+0000","last_active":"2023-12-20T04:29:31.622736+0000","last_peered":"2023-12-20T04:29:31.622736+0000","last_clean":"2023-12-20T04:29:31.622736+0000","last_became_active":"2023-12-20T04:29:30.648308+0000","last_became_peered":"2023-12-20T04:29:30.648308+0000","last_unstale":"2023-12-20T04:29:31.622736+0000","last_undegraded":"2023-12-20T04:29:31.622736+0000","last_fullsized":"2023-12-20T04:29:31.622736+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:13:08.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.638569+0000","last_change":"2023-12-20T04:29:30.224172+0000","last_active":"2023-12-20T04:29:30.638569+0000","last_peered":"2023-12-20T04:29:30.638569+0000","last_clean":"2023-12-20T04:29:30.638569+0000","last_became_active":"2023-12-20T04:29:30.223865+0000","last_became_peered":"2023-12-20T04:29:30.223865+0000","last_unstale":"2023-12-20T04:29:30.638569+0000","last_undegraded":"2023-12-20T04:29:30.638569+0000","last_fullsized":"2023-12-20T04:29:30.638569+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625879+0000","last_change":"2023-12-20T04:29:30.652768+0000","last_active":"2023-12-20T04:29:31.625879+0000","last_peered":"2023-12-20T04:29:31.625879+0000","last_clean":"2023-12-20T04:29:31.625879+0000","last_became_active":"2023-12-20T04:29:30.652427+0000","last_became_peered":"2023-12-20T04:29:30.652427+0000","last_unstale":"2023-12-20T04:29:31.625879+0000","last_undegraded":"2023-12-20T04:29:31.625879+0000","last_fullsized":"2023-12-20T04:29:31.625879+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:57.967703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.649107+0000","last_change":"2023-12-20T04:29:30.231289+0000","last_active":"2023-12-20T04:29:30.649107+0000","last_peered":"2023-12-20T04:29:30.649107+0000","last_clean":"2023-12-20T04:29:30.649107+0000","last_became_active":"2023-12-20T04:29:30.230075+0000","last_became_peered":"2023-12-20T04:29:30.230075+0000","last_unstale":"2023-12-20T04:29:30.649107+0000","last_undegraded":"2023-12-20T04:29:30.649107+0000","last_fullsized":"2023-12-20T04:29:30.649107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:17:46.344276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955112+0000","last_change":"2023-12-20T04:29:30.646861+0000","last_active":"2023-12-20T04:29:32.955112+0000","last_peered":"2023-12-20T04:29:32.955112+0000","last_clean":"2023-12-20T04:29:32.955112+0000","last_became_active":"2023-12-20T04:29:30.646560+0000","last_became_peered":"2023-12-20T04:29:30.646560+0000","last_unstale":"2023-12-20T04:29:32.955112+0000","last_undegraded":"2023-12-20T04:29:32.955112+0000","last_fullsized":"2023-12-20T04:29:32.955112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:42.395721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.649412+0000","last_change":"2023-12-20T04:29:30.232127+0000","last_active":"2023-12-20T04:29:30.649412+0000","last_peered":"2023-12-20T04:29:30.649412+0000","last_clean":"2023-12-20T04:29:30.649412+0000","last_became_active":"2023-12-20T04:29:30.231754+0000","last_became_peered":"2023-12-20T04:29:30.231754+0000","last_unstale":"2023-12-20T04:29:30.649412+0000","last_undegraded":"2023-12-20T04:29:30.649412+0000","last_fullsized":"2023-12-20T04:29:30.649412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:35:23.040608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626165+0000","last_change":"2023-12-20T04:29:30.660699+0000","last_active":"2023-12-20T04:29:31.626165+0000","last_peered":"2023-12-20T04:29:31.626165+0000","last_clean":"2023-12-20T04:29:31.626165+0000","last_became_active":"2023-12-20T04:29:30.660412+0000","last_became_peered":"2023-12-20T04:29:30.660412+0000","last_unstale":"2023-12-20T04:29:31.626165+0000","last_undegraded":"2023-12-20T04:29:31.626165+0000","last_fullsized":"2023-12-20T04:29:31.626165+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:44:31.379624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.615460+0000","last_change":"2023-12-20T04:29:30.221627+0000","last_active":"2023-12-20T04:29:30.615460+0000","last_peered":"2023-12-20T04:29:30.615460+0000","last_clean":"2023-12-20T04:29:30.615460+0000","last_became_active":"2023-12-20T04:29:30.221347+0000","last_became_peered":"2023-12-20T04:29:30.221347+0000","last_unstale":"2023-12-20T04:29:30.615460+0000","last_undegraded":"2023-12-20T04:29:30.615460+0000","last_fullsized":"2023-12-20T04:29:30.615460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:42:48.712967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622155+0000","last_change":"2023-12-20T04:29:30.638486+0000","last_active":"2023-12-20T04:29:31.622155+0000","last_peered":"2023-12-20T04:29:31.622155+0000","last_clean":"2023-12-20T04:29:31.622155+0000","last_became_active":"2023-12-20T04:29:30.638192+0000","last_became_peered":"2023-12-20T04:29:30.638192+0000","last_unstale":"2023-12-20T04:29:31.622155+0000","last_undegraded":"2023-12-20T04:29:31.622155+0000","last_fullsized":"2023-12-20T04:29:31.622155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:57.967703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.640646+0000","last_change":"2023-12-20T04:29:30.231286+0000","last_active":"2023-12-20T04:29:30.640646+0000","last_peered":"2023-12-20T04:29:30.640646+0000","last_clean":"2023-12-20T04:29:30.640646+0000","last_became_active":"2023-12-20T04:29:30.230988+0000","last_became_peered":"2023-12-20T04:29:30.230988+0000","last_unstale":"2023-12-20T04:29:30.640646+0000","last_undegraded":"2023-12-20T04:29:30.640646+0000","last_fullsized":"2023-12-20T04:29:30.640646+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:50:04.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955455+0000","last_change":"2023-12-20T04:29:30.652262+0000","last_active":"2023-12-20T04:29:32.955455+0000","last_peered":"2023-12-20T04:29:32.955455+0000","last_clean":"2023-12-20T04:29:32.955455+0000","last_became_active":"2023-12-20T04:29:30.647614+0000","last_became_peered":"2023-12-20T04:29:30.647614+0000","last_unstale":"2023-12-20T04:29:32.955455+0000","last_undegraded":"2023-12-20T04:29:32.955455+0000","last_fullsized":"2023-12-20T04:29:32.955455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:36.244013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.616961+0000","last_change":"2023-12-20T04:29:30.223687+0000","last_active":"2023-12-20T04:29:30.616961+0000","last_peered":"2023-12-20T04:29:30.616961+0000","last_clean":"2023-12-20T04:29:30.616961+0000","last_became_active":"2023-12-20T04:29:30.223353+0000","last_became_peered":"2023-12-20T04:29:30.223353+0000","last_unstale":"2023-12-20T04:29:30.616961+0000","last_undegraded":"2023-12-20T04:29:30.616961+0000","last_fullsized":"2023-12-20T04:29:30.616961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:00.360055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854127+0000","last_change":"2023-12-20T04:29:30.647207+0000","last_active":"2023-12-20T04:29:31.854127+0000","last_peered":"2023-12-20T04:29:31.854127+0000","last_clean":"2023-12-20T04:29:31.854127+0000","last_became_active":"2023-12-20T04:29:30.646695+0000","last_became_peered":"2023-12-20T04:29:30.646695+0000","last_unstale":"2023-12-20T04:29:31.854127+0000","last_undegraded":"2023-12-20T04:29:31.854127+0000","last_fullsized":"2023-12-20T04:29:31.854127+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:06:15.798745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.623178+0000","last_change":"2023-12-20T04:29:30.230906+0000","last_active":"2023-12-20T04:29:30.623178+0000","last_peered":"2023-12-20T04:29:30.623178+0000","last_clean":"2023-12-20T04:29:30.623178+0000","last_became_active":"2023-12-20T04:29:30.230515+0000","last_became_peered":"2023-12-20T04:29:30.230515+0000","last_unstale":"2023-12-20T04:29:30.623178+0000","last_undegraded":"2023-12-20T04:29:30.623178+0000","last_fullsized":"2023-12-20T04:29:30.623178+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854241+0000","last_change":"2023-12-20T04:29:30.654536+0000","last_active":"2023-12-20T04:29:31.854241+0000","last_peered":"2023-12-20T04:29:31.854241+0000","last_clean":"2023-12-20T04:29:31.854241+0000","last_became_active":"2023-12-20T04:29:30.653861+0000","last_became_peered":"2023-12-20T04:29:30.653861+0000","last_unstale":"2023-12-20T04:29:31.854241+0000","last_undegraded":"2023-12-20T04:29:31.854241+0000","last_fullsized":"2023-12-20T04:29:31.854241+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:11:19.882387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.647354+0000","last_change":"2023-12-20T04:29:30.226215+0000","last_active":"2023-12-20T04:29:30.647354+0000","last_peered":"2023-12-20T04:29:30.647354+0000","last_clean":"2023-12-20T04:29:30.647354+0000","last_became_active":"2023-12-20T04:29:30.225949+0000","last_became_peered":"2023-12-20T04:29:30.225949+0000","last_unstale":"2023-12-20T04:29:30.647354+0000","last_undegraded":"2023-12-20T04:29:30.647354+0000","last_fullsized":"2023-12-20T04:29:30.647354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955333+0000","last_change":"2023-12-20T04:29:30.652886+0000","last_active":"2023-12-20T04:29:32.955333+0000","last_peered":"2023-12-20T04:29:32.955333+0000","last_clean":"2023-12-20T04:29:32.955333+0000","last_became_active":"2023-12-20T04:29:30.652626+0000","last_became_peered":"2023-12-20T04:29:30.652626+0000","last_unstale":"2023-12-20T04:29:32.955333+0000","last_undegraded":"2023-12-20T04:29:32.955333+0000","last_fullsized":"2023-12-20T04:29:32.955333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:07.590492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.657824+0000","last_change":"2023-12-20T04:29:30.228644+0000","last_active":"2023-12-20T04:29:30.657824+0000","last_peered":"2023-12-20T04:29:30.657824+0000","last_clean":"2023-12-20T04:29:30.657824+0000","last_became_active":"2023-12-20T04:29:30.228268+0000","last_became_peered":"2023-12-20T04:29:30.228268+0000","last_unstale":"2023-12-20T04:29:30.657824+0000","last_undegraded":"2023-12-20T04:29:30.657824+0000","last_fullsized":"2023-12-20T04:29:30.657824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:29:32.922907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622166+0000","last_change":"2023-12-20T04:29:30.651226+0000","last_active":"2023-12-20T04:29:31.622166+0000","last_peered":"2023-12-20T04:29:31.622166+0000","last_clean":"2023-12-20T04:29:31.622166+0000","last_became_active":"2023-12-20T04:29:30.650941+0000","last_became_peered":"2023-12-20T04:29:30.650941+0000","last_unstale":"2023-12-20T04:29:31.622166+0000","last_undegraded":"2023-12-20T04:29:31.622166+0000","last_fullsized":"2023-12-20T04:29:31.622166+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:52.889158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.660131+0000","last_change":"2023-12-20T04:29:30.233204+0000","last_active":"2023-12-20T04:29:30.660131+0000","last_peered":"2023-12-20T04:29:30.660131+0000","last_clean":"2023-12-20T04:29:30.660131+0000","last_became_active":"2023-12-20T04:29:30.232924+0000","last_became_peered":"2023-12-20T04:29:30.232924+0000","last_unstale":"2023-12-20T04:29:30.660131+0000","last_undegraded":"2023-12-20T04:29:30.660131+0000","last_fullsized":"2023-12-20T04:29:30.660131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854180+0000","last_change":"2023-12-20T04:29:30.634013+0000","last_active":"2023-12-20T04:29:31.854180+0000","last_peered":"2023-12-20T04:29:31.854180+0000","last_clean":"2023-12-20T04:29:31.854180+0000","last_became_active":"2023-12-20T04:29:30.633703+0000","last_became_peered":"2023-12-20T04:29:30.633703+0000","last_unstale":"2023-12-20T04:29:31.854180+0000","last_undegraded":"2023-12-20T04:29:31.854180+0000","last_fullsized":"2023-12-20T04:29:31.854180+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:36.244013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.629155+0000","last_change":"2023-12-20T04:29:30.231628+0000","last_active":"2023-12-20T04:29:30.629155+0000","last_peered":"2023-12-20T04:29:30.629155+0000","last_clean":"2023-12-20T04:29:30.629155+0000","last_became_active":"2023-12-20T04:29:30.231363+0000","last_became_peered":"2023-12-20T04:29:30.231363+0000","last_unstale":"2023-12-20T04:29:30.629155+0000","last_undegraded":"2023-12-20T04:29:30.629155+0000","last_fullsized":"2023-12-20T04:29:30.629155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625688+0000","last_change":"2023-12-20T04:29:30.650833+0000","last_active":"2023-12-20T04:29:31.625688+0000","last_peered":"2023-12-20T04:29:31.625688+0000","last_clean":"2023-12-20T04:29:31.625688+0000","last_became_active":"2023-12-20T04:29:30.650576+0000","last_became_peered":"2023-12-20T04:29:30.650576+0000","last_unstale":"2023-12-20T04:29:31.625688+0000","last_undegraded":"2023-12-20T04:29:31.625688+0000","last_fullsized":"2023-12-20T04:29:31.625688+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:52.889158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.615507+0000","last_change":"2023-12-20T04:29:30.227573+0000","last_active":"2023-12-20T04:29:30.615507+0000","last_peered":"2023-12-20T04:29:30.615507+0000","last_clean":"2023-12-20T04:29:30.615507+0000","last_became_active":"2023-12-20T04:29:30.227266+0000","last_became_peered":"2023-12-20T04:29:30.227266+0000","last_unstale":"2023-12-20T04:29:30.615507+0000","last_undegraded":"2023-12-20T04:29:30.615507+0000","last_fullsized":"2023-12-20T04:29:30.615507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:42.345201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.189818+0000","last_change":"2023-12-20T04:29:30.659850+0000","last_active":"2023-12-20T04:29:32.189818+0000","last_peered":"2023-12-20T04:29:32.189818+0000","last_clean":"2023-12-20T04:29:32.189818+0000","last_became_active":"2023-12-20T04:29:30.659274+0000","last_became_peered":"2023-12-20T04:29:30.659274+0000","last_unstale":"2023-12-20T04:29:32.189818+0000","last_undegraded":"2023-12-20T04:29:32.189818+0000","last_fullsized":"2023-12-20T04:29:32.189818+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:01.362866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.647976+0000","last_change":"2023-12-20T04:29:30.229228+0000","last_active":"2023-12-20T04:29:30.647976+0000","last_peered":"2023-12-20T04:29:30.647976+0000","last_clean":"2023-12-20T04:29:30.647976+0000","last_became_active":"2023-12-20T04:29:30.228933+0000","last_became_peered":"2023-12-20T04:29:30.228933+0000","last_unstale":"2023-12-20T04:29:30.647976+0000","last_undegraded":"2023-12-20T04:29:30.647976+0000","last_fullsized":"2023-12-20T04:29:30.647976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626218+0000","last_change":"2023-12-20T04:29:30.658500+0000","last_active":"2023-12-20T04:29:31.626218+0000","last_peered":"2023-12-20T04:29:31.626218+0000","last_clean":"2023-12-20T04:29:31.626218+0000","last_became_active":"2023-12-20T04:29:30.658220+0000","last_became_peered":"2023-12-20T04:29:30.658220+0000","last_unstale":"2023-12-20T04:29:31.626218+0000","last_undegraded":"2023-12-20T04:29:31.626218+0000","last_fullsized":"2023-12-20T04:29:31.626218+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:29:33.925717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.644729+0000","last_change":"2023-12-20T04:29:30.230687+0000","last_active":"2023-12-20T04:29:30.644729+0000","last_peered":"2023-12-20T04:29:30.644729+0000","last_clean":"2023-12-20T04:29:30.644729+0000","last_became_active":"2023-12-20T04:29:30.230395+0000","last_became_peered":"2023-12-20T04:29:30.230395+0000","last_unstale":"2023-12-20T04:29:30.644729+0000","last_undegraded":"2023-12-20T04:29:30.644729+0000","last_fullsized":"2023-12-20T04:29:30.644729+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:31:17.467772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955511+0000","last_change":"2023-12-20T04:29:30.652005+0000","last_active":"2023-12-20T04:29:32.955511+0000","last_peered":"2023-12-20T04:29:32.955511+0000","last_clean":"2023-12-20T04:29:32.955511+0000","last_became_active":"2023-12-20T04:29:30.651689+0000","last_became_peered":"2023-12-20T04:29:30.651689+0000","last_unstale":"2023-12-20T04:29:32.955511+0000","last_undegraded":"2023-12-20T04:29:32.955511+0000","last_fullsized":"2023-12-20T04:29:32.955511+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:52.889158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.660255+0000","last_change":"2023-12-20T04:29:30.232810+0000","last_active":"2023-12-20T04:29:30.660255+0000","last_peered":"2023-12-20T04:29:30.660255+0000","last_clean":"2023-12-20T04:29:30.660255+0000","last_became_active":"2023-12-20T04:29:30.232515+0000","last_became_peered":"2023-12-20T04:29:30.232515+0000","last_unstale":"2023-12-20T04:29:30.660255+0000","last_undegraded":"2023-12-20T04:29:30.660255+0000","last_fullsized":"2023-12-20T04:29:30.660255+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622401+0000","last_change":"2023-12-20T04:29:30.645904+0000","last_active":"2023-12-20T04:29:31.622401+0000","last_peered":"2023-12-20T04:29:31.622401+0000","last_clean":"2023-12-20T04:29:31.622401+0000","last_became_active":"2023-12-20T04:29:30.645393+0000","last_became_peered":"2023-12-20T04:29:30.645393+0000","last_unstale":"2023-12-20T04:29:31.622401+0000","last_undegraded":"2023-12-20T04:29:31.622401+0000","last_fullsized":"2023-12-20T04:29:31.622401+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:44:31.379624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.655227+0000","last_change":"2023-12-20T04:29:30.221914+0000","last_active":"2023-12-20T04:29:30.655227+0000","last_peered":"2023-12-20T04:29:30.655227+0000","last_clean":"2023-12-20T04:29:30.655227+0000","last_became_active":"2023-12-20T04:29:30.221627+0000","last_became_peered":"2023-12-20T04:29:30.221627+0000","last_unstale":"2023-12-20T04:29:30.655227+0000","last_undegraded":"2023-12-20T04:29:30.655227+0000","last_fullsized":"2023-12-20T04:29:30.655227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:44:48.959029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928549+0000","last_change":"2023-12-20T04:29:30.648693+0000","last_active":"2023-12-20T04:29:31.928549+0000","last_peered":"2023-12-20T04:29:31.928549+0000","last_clean":"2023-12-20T04:29:31.928549+0000","last_became_active":"2023-12-20T04:29:30.648393+0000","last_became_peered":"2023-12-20T04:29:30.648393+0000","last_unstale":"2023-12-20T04:29:31.928549+0000","last_undegraded":"2023-12-20T04:29:31.928549+0000","last_fullsized":"2023-12-20T04:29:31.928549+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:18.283893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.639338+0000","last_change":"2023-12-20T04:29:30.230694+0000","last_active":"2023-12-20T04:29:30.639338+0000","last_peered":"2023-12-20T04:29:30.639338+0000","last_clean":"2023-12-20T04:29:30.639338+0000","last_became_active":"2023-12-20T04:29:30.230428+0000","last_became_peered":"2023-12-20T04:29:30.230428+0000","last_unstale":"2023-12-20T04:29:30.639338+0000","last_undegraded":"2023-12-20T04:29:30.639338+0000","last_fullsized":"2023-12-20T04:29:30.639338+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928323+0000","last_change":"2023-12-20T04:29:30.646456+0000","last_active":"2023-12-20T04:29:31.928323+0000","last_peered":"2023-12-20T04:29:31.928323+0000","last_clean":"2023-12-20T04:29:31.928323+0000","last_became_active":"2023-12-20T04:29:30.646027+0000","last_became_peered":"2023-12-20T04:29:30.646027+0000","last_unstale":"2023-12-20T04:29:31.928323+0000","last_undegraded":"2023-12-20T04:29:31.928323+0000","last_fullsized":"2023-12-20T04:29:31.928323+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:01.362866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.649074+0000","last_change":"2023-12-20T04:29:30.218619+0000","last_active":"2023-12-20T04:29:30.649074+0000","last_peered":"2023-12-20T04:29:30.649074+0000","last_clean":"2023-12-20T04:29:30.649074+0000","last_became_active":"2023-12-20T04:29:30.218180+0000","last_became_peered":"2023-12-20T04:29:30.218180+0000","last_unstale":"2023-12-20T04:29:30.649074+0000","last_undegraded":"2023-12-20T04:29:30.649074+0000","last_fullsized":"2023-12-20T04:29:30.649074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928509+0000","last_change":"2023-12-20T04:29:30.633870+0000","last_active":"2023-12-20T04:29:31.928509+0000","last_peered":"2023-12-20T04:29:31.928509+0000","last_clean":"2023-12-20T04:29:31.928509+0000","last_became_active":"2023-12-20T04:29:30.633586+0000","last_became_peered":"2023-12-20T04:29:30.633586+0000","last_unstale":"2023-12-20T04:29:31.928509+0000","last_undegraded":"2023-12-20T04:29:31.928509+0000","last_fullsized":"2023-12-20T04:29:31.928509+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:30.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.628414+0000","last_change":"2023-12-20T04:29:30.232351+0000","last_active":"2023-12-20T04:29:30.628414+0000","last_peered":"2023-12-20T04:29:30.628414+0000","last_clean":"2023-12-20T04:29:30.628414+0000","last_became_active":"2023-12-20T04:29:30.231883+0000","last_became_peered":"2023-12-20T04:29:30.231883+0000","last_unstale":"2023-12-20T04:29:30.628414+0000","last_undegraded":"2023-12-20T04:29:30.628414+0000","last_fullsized":"2023-12-20T04:29:30.628414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928377+0000","last_change":"2023-12-20T04:29:30.633390+0000","last_active":"2023-12-20T04:29:31.928377+0000","last_peered":"2023-12-20T04:29:31.928377+0000","last_clean":"2023-12-20T04:29:31.928377+0000","last_became_active":"2023-12-20T04:29:30.632629+0000","last_became_peered":"2023-12-20T04:29:30.632629+0000","last_unstale":"2023-12-20T04:29:31.928377+0000","last_undegraded":"2023-12-20T04:29:31.928377+0000","last_fullsized":"2023-12-20T04:29:31.928377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:43.348011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.642615+0000","last_change":"2023-12-20T04:29:30.221611+0000","last_active":"2023-12-20T04:29:30.642615+0000","last_peered":"2023-12-20T04:29:30.642615+0000","last_clean":"2023-12-20T04:29:30.642615+0000","last_became_active":"2023-12-20T04:29:30.221315+0000","last_became_peered":"2023-12-20T04:29:30.221315+0000","last_unstale":"2023-12-20T04:29:30.642615+0000","last_undegraded":"2023-12-20T04:29:30.642615+0000","last_fullsized":"2023-12-20T04:29:30.642615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:00.538815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625495+0000","last_change":"2023-12-20T04:29:30.661222+0000","last_active":"2023-12-20T04:29:31.625495+0000","last_peered":"2023-12-20T04:29:31.625495+0000","last_clean":"2023-12-20T04:29:31.625495+0000","last_became_active":"2023-12-20T04:29:30.660457+0000","last_became_peered":"2023-12-20T04:29:30.660457+0000","last_unstale":"2023-12-20T04:29:31.625495+0000","last_undegraded":"2023-12-20T04:29:31.625495+0000","last_fullsized":"2023-12-20T04:29:31.625495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:41.192000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.629159+0000","last_change":"2023-12-20T04:29:30.227738+0000","last_active":"2023-12-20T04:29:30.629159+0000","last_peered":"2023-12-20T04:29:30.629159+0000","last_clean":"2023-12-20T04:29:30.629159+0000","last_became_active":"2023-12-20T04:29:30.227464+0000","last_became_peered":"2023-12-20T04:29:30.227464+0000","last_unstale":"2023-12-20T04:29:30.629159+0000","last_undegraded":"2023-12-20T04:29:30.629159+0000","last_fullsized":"2023-12-20T04:29:30.629159+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928500+0000","last_change":"2023-12-20T04:29:30.640153+0000","last_active":"2023-12-20T04:29:31.928500+0000","last_peered":"2023-12-20T04:29:31.928500+0000","last_clean":"2023-12-20T04:29:31.928500+0000","last_became_active":"2023-12-20T04:29:30.639884+0000","last_became_peered":"2023-12-20T04:29:30.639884+0000","last_unstale":"2023-12-20T04:29:31.928500+0000","last_undegraded":"2023-12-20T04:29:31.928500+0000","last_fullsized":"2023-12-20T04:29:31.928500+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:42:49.715777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.651264+0000","last_change":"2023-12-20T04:29:30.226593+0000","last_active":"2023-12-20T04:29:30.651264+0000","last_peered":"2023-12-20T04:29:30.651264+0000","last_clean":"2023-12-20T04:29:30.651264+0000","last_became_active":"2023-12-20T04:29:30.226331+0000","last_became_peered":"2023-12-20T04:29:30.226331+0000","last_unstale":"2023-12-20T04:29:30.651264+0000","last_undegraded":"2023-12-20T04:29:30.651264+0000","last_fullsized":"2023-12-20T04:29:30.651264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:00.162402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955532+0000","last_change":"2023-12-20T04:29:30.657429+0000","last_active":"2023-12-20T04:29:32.955532+0000","last_peered":"2023-12-20T04:29:32.955532+0000","last_clean":"2023-12-20T04:29:32.955532+0000","last_became_active":"2023-12-20T04:29:30.656974+0000","last_became_peered":"2023-12-20T04:29:30.656974+0000","last_unstale":"2023-12-20T04:29:32.955532+0000","last_undegraded":"2023-12-20T04:29:32.955532+0000","last_fullsized":"2023-12-20T04:29:32.955532+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:19:15.119662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.930657+0000","last_change":"2023-12-20T04:29:30.222922+0000","last_active":"2023-12-20T04:29:32.930657+0000","last_peered":"2023-12-20T04:29:32.930657+0000","last_clean":"2023-12-20T04:29:32.930657+0000","last_became_active":"2023-12-20T04:29:30.222618+0000","last_became_peered":"2023-12-20T04:29:30.222618+0000","last_unstale":"2023-12-20T04:29:32.930657+0000","last_undegraded":"2023-12-20T04:29:32.930657+0000","last_fullsized":"2023-12-20T04:29:32.930657+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.930429+0000","last_change":"2023-12-20T04:29:30.636221+0000","last_active":"2023-12-20T04:29:32.930429+0000","last_peered":"2023-12-20T04:29:32.930429+0000","last_clean":"2023-12-20T04:29:32.930429+0000","last_became_active":"2023-12-20T04:29:30.634994+0000","last_became_peered":"2023-12-20T04:29:30.634994+0000","last_unstale":"2023-12-20T04:29:32.930429+0000","last_undegraded":"2023-12-20T04:29:32.930429+0000","last_fullsized":"2023-12-20T04:29:32.930429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:45.142508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.627288+0000","last_change":"2023-12-20T04:29:30.221192+0000","last_active":"2023-12-20T04:29:30.627288+0000","last_peered":"2023-12-20T04:29:30.627288+0000","last_clean":"2023-12-20T04:29:30.627288+0000","last_became_active":"2023-12-20T04:29:30.220915+0000","last_became_peered":"2023-12-20T04:29:30.220915+0000","last_unstale":"2023-12-20T04:29:30.627288+0000","last_undegraded":"2023-12-20T04:29:30.627288+0000","last_fullsized":"2023-12-20T04:29:30.627288+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:19:07.467835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.189527+0000","last_change":"2023-12-20T04:29:30.647250+0000","last_active":"2023-12-20T04:29:32.189527+0000","last_peered":"2023-12-20T04:29:32.189527+0000","last_clean":"2023-12-20T04:29:32.189527+0000","last_became_active":"2023-12-20T04:29:30.646942+0000","last_became_peered":"2023-12-20T04:29:30.646942+0000","last_unstale":"2023-12-20T04:29:32.189527+0000","last_undegraded":"2023-12-20T04:29:32.189527+0000","last_fullsized":"2023-12-20T04:29:32.189527+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:30.023462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.622694+0000","last_change":"2023-12-20T04:29:30.226803+0000","last_active":"2023-12-20T04:29:30.622694+0000","last_peered":"2023-12-20T04:29:30.622694+0000","last_clean":"2023-12-20T04:29:30.622694+0000","last_became_active":"2023-12-20T04:29:30.226262+0000","last_became_peered":"2023-12-20T04:29:30.226262+0000","last_unstale":"2023-12-20T04:29:30.622694+0000","last_undegraded":"2023-12-20T04:29:30.622694+0000","last_fullsized":"2023-12-20T04:29:30.622694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.930362+0000","last_change":"2023-12-20T04:29:30.635889+0000","last_active":"2023-12-20T04:29:32.930362+0000","last_peered":"2023-12-20T04:29:32.930362+0000","last_clean":"2023-12-20T04:29:32.930362+0000","last_became_active":"2023-12-20T04:29:30.635589+0000","last_became_peered":"2023-12-20T04:29:30.635589+0000","last_unstale":"2023-12-20T04:29:32.930362+0000","last_undegraded":"2023-12-20T04:29:32.930362+0000","last_fullsized":"2023-12-20T04:29:32.930362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:00:35.598112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.629242+0000","last_change":"2023-12-20T04:29:30.220925+0000","last_active":"2023-12-20T04:29:30.629242+0000","last_peered":"2023-12-20T04:29:30.629242+0000","last_clean":"2023-12-20T04:29:30.629242+0000","last_became_active":"2023-12-20T04:29:30.220649+0000","last_became_peered":"2023-12-20T04:29:30.220649+0000","last_unstale":"2023-12-20T04:29:30.629242+0000","last_undegraded":"2023-12-20T04:29:30.629242+0000","last_fullsized":"2023-12-20T04:29:30.629242+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.853919+0000","last_change":"2023-12-20T04:29:30.637566+0000","last_active":"2023-12-20T04:29:31.853919+0000","last_peered":"2023-12-20T04:29:31.853919+0000","last_clean":"2023-12-20T04:29:31.853919+0000","last_became_active":"2023-12-20T04:29:30.637316+0000","last_became_peered":"2023-12-20T04:29:30.637316+0000","last_unstale":"2023-12-20T04:29:31.853919+0000","last_undegraded":"2023-12-20T04:29:31.853919+0000","last_fullsized":"2023-12-20T04:29:31.853919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:21:29.467415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.642716+0000","last_change":"2023-12-20T04:29:30.226051+0000","last_active":"2023-12-20T04:29:30.642716+0000","last_peered":"2023-12-20T04:29:30.642716+0000","last_clean":"2023-12-20T04:29:30.642716+0000","last_became_active":"2023-12-20T04:29:30.225759+0000","last_became_peered":"2023-12-20T04:29:30.225759+0000","last_unstale":"2023-12-20T04:29:30.642716+0000","last_undegraded":"2023-12-20T04:29:30.642716+0000","last_fullsized":"2023-12-20T04:29:30.642716+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:00:05.822869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625469+0000","last_change":"2023-12-20T04:29:30.649586+0000","last_active":"2023-12-20T04:29:31.625469+0000","last_peered":"2023-12-20T04:29:31.625469+0000","last_clean":"2023-12-20T04:29:31.625469+0000","last_became_active":"2023-12-20T04:29:30.649187+0000","last_became_peered":"2023-12-20T04:29:30.649187+0000","last_unstale":"2023-12-20T04:29:31.625469+0000","last_undegraded":"2023-12-20T04:29:31.625469+0000","last_fullsized":"2023-12-20T04:29:31.625469+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:42:49.715777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.615563+0000","last_change":"2023-12-20T04:29:30.227077+0000","last_active":"2023-12-20T04:29:30.615563+0000","last_peered":"2023-12-20T04:29:30.615563+0000","last_clean":"2023-12-20T04:29:30.615563+0000","last_became_active":"2023-12-20T04:29:30.226520+0000","last_became_peered":"2023-12-20T04:29:30.226520+0000","last_unstale":"2023-12-20T04:29:30.615563+0000","last_undegraded":"2023-12-20T04:29:30.615563+0000","last_fullsized":"2023-12-20T04:29:30.615563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:29.961522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625847+0000","last_change":"2023-12-20T04:29:30.648340+0000","last_active":"2023-12-20T04:29:31.625847+0000","last_peered":"2023-12-20T04:29:31.625847+0000","last_clean":"2023-12-20T04:29:31.625847+0000","last_became_active":"2023-12-20T04:29:30.647992+0000","last_became_peered":"2023-12-20T04:29:30.647992+0000","last_unstale":"2023-12-20T04:29:31.625847+0000","last_undegraded":"2023-12-20T04:29:31.625847+0000","last_fullsized":"2023-12-20T04:29:31.625847+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:07.590492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.626176+0000","last_change":"2023-12-20T04:29:30.233314+0000","last_active":"2023-12-20T04:29:30.626176+0000","last_peered":"2023-12-20T04:29:30.626176+0000","last_clean":"2023-12-20T04:29:30.626176+0000","last_became_active":"2023-12-20T04:29:30.233037+0000","last_became_peered":"2023-12-20T04:29:30.233037+0000","last_unstale":"2023-12-20T04:29:30.626176+0000","last_undegraded":"2023-12-20T04:29:30.626176+0000","last_fullsized":"2023-12-20T04:29:30.626176+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":174,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.637861+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:29:30.637861+0000","last_peered":"2023-12-20T04:29:30.637861+0000","last_clean":"2023-12-20T04:29:30.637861+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:29:30.637861+0000","last_undegraded":"2023-12-20T04:29:30.637861+0000","last_fullsized":"2023-12-20T04:29:30.637861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":995328,"data_stored":747016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.57699999999999996}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.629},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.85699999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.68400000000000005}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.746}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":157174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.71599999999999997}]},{"osd":2,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.90800000000000003}]},{"osd":3,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.80200000000000005}]},{"osd":4,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.84199999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.872}]},{"osd":7,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.89100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":405504,"data_stored":157230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.72799999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.752}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68300000000000005}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.64500000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.622}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54000000000000004}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.71199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":156662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54500000000000004}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.84799999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.82899999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65300000000000002}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.88700000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.90700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":413696,"data_stored":158144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.3700000000000001,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.309},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.51400000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.7160000000000002,"5min":3.7160000000000002,"15min":3.7160000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0110000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":0.497},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":6.1399999999999997,"5min":6.1399999999999997,"15min":6.1399999999999997},"last":0.42799999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.749}]},{"osd":5,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.9830000000000001,"5min":3.9830000000000001,"15min":3.9830000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.85699999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.8900000000000001,"5min":3.8900000000000001,"15min":3.8900000000000001},"last":0.88300000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.6970000000000001,"5min":4.6970000000000001,"15min":4.6970000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.77200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":401408,"data_stored":156628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35799999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.315},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.59699999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.623}]},{"osd":4,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.72499999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.73999999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.94499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":413696,"data_stored":158144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.434},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.30099999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.41299999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.58399999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.59799999999999998}]},{"osd":7,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.628},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.65800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":991232,"data_stored":746926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.31900000000000001}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.371}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42899999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.70699999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.623},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.79000000000000004}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.69299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:29:39.345 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:29:39.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.498+0000 7fde67b96640 1 Processor -- start 2023-12-20T04:29:39.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.498+0000 7fde67b96640 1 -- start start 2023-12-20T04:29:39.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde67b96640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fde601599e0 0x7fde600aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde67b96640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde600c90e0 0x7fde600b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde67b96640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde600b0b30 0x7fde6015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde67b96640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde600c98c0 con 0x7fde600b0b30 2023-12-20T04:29:39.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde67b96640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fde600c9ba0 con 0x7fde601599e0 2023-12-20T04:29:39.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde67b96640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde600c9e80 con 0x7fde600c90e0 2023-12-20T04:29:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde66133640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde600b0b30 0x7fde6015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde66133640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fde600b0b30 0x7fde6015d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41094/0 (socket says 172.21.15.32:41094) 2023-12-20T04:29:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde66133640 1 -- 172.21.15.32:0/2331874374 learned_addr learned my addr 172.21.15.32:0/2331874374 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde65932640 1 --2- 172.21.15.32:0/2331874374 >> v2:172.21.15.39:3300/0 conn(0x7fde601599e0 0x7fde600aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde65131640 1 --2- 172.21.15.32:0/2331874374 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde600c90e0 0x7fde600b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde66133640 1 -- 172.21.15.32:0/2331874374 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde600c90e0 msgr2=0x7fde600b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde66133640 1 --2- 172.21.15.32:0/2331874374 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde600c90e0 0x7fde600b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde66133640 1 -- 172.21.15.32:0/2331874374 >> v2:172.21.15.39:3300/0 conn(0x7fde601599e0 msgr2=0x7fde600aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde66133640 1 --2- 172.21.15.32:0/2331874374 >> v2:172.21.15.39:3300/0 conn(0x7fde601599e0 0x7fde600aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.502+0000 7fde66133640 1 -- 172.21.15.32:0/2331874374 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde600ca160 con 0x7fde600b0b30 2023-12-20T04:29:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde66133640 1 --2- 172.21.15.32:0/2331874374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde600b0b30 0x7fde6015d6e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde6006e3c0 tx=0x7fde5801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7cfc0e251d1b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde64930640 1 -- 172.21.15.32:0/2331874374 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde580a5580 con 0x7fde600b0b30 2023-12-20T04:29:39.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde64930640 1 -- 172.21.15.32:0/2331874374 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde580a56e0 con 0x7fde600b0b30 2023-12-20T04:29:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde64930640 1 -- 172.21.15.32:0/2331874374 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde580a5a70 con 0x7fde600b0b30 2023-12-20T04:29:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde67b96640 1 -- 172.21.15.32:0/2331874374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde600b0b30 msgr2=0x7fde6015d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:39.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde67b96640 1 --2- 172.21.15.32:0/2331874374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde600b0b30 0x7fde6015d6e0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7fde6006e3c0 tx=0x7fde5801c6e0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde67b96640 1 -- 172.21.15.32:0/2331874374 shutdown_connections 2023-12-20T04:29:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde67b96640 1 --2- 172.21.15.32:0/2331874374 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde600b0b30 0x7fde6015d6e0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde67b96640 1 --2- 172.21.15.32:0/2331874374 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde600c90e0 0x7fde600b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde67b96640 1 --2- 172.21.15.32:0/2331874374 >> v2:172.21.15.39:3300/0 conn(0x7fde601599e0 0x7fde600aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde67b96640 1 -- 172.21.15.32:0/2331874374 >> 172.21.15.32:0/2331874374 conn(0x7fde6006a320 msgr2=0x7fde600ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde67b96640 1 -- 172.21.15.32:0/2331874374 shutdown_connections 2023-12-20T04:29:39.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde67b96640 1 -- 172.21.15.32:0/2331874374 wait complete. 2023-12-20T04:29:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde67b96640 1 Processor -- start 2023-12-20T04:29:39.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.506+0000 7fde67b96640 1 -- start start 2023-12-20T04:29:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde67b96640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde600b0b30 0x7fde60152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde67b96640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde600c90e0 0x7fde6014c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde67b96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde601599e0 0x7fde6014c830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde67b96640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde601f5330 con 0x7fde600b0b30 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde67b96640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fde600c7340 con 0x7fde601599e0 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde67b96640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fde600c7620 con 0x7fde600c90e0 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde65932640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde600b0b30 0x7fde60152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde65932640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde600b0b30 0x7fde60152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41100/0 (socket says 172.21.15.32:41100) 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde65932640 1 -- 172.21.15.32:0/324225965 learned_addr learned my addr 172.21.15.32:0/324225965 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde65131640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde600c90e0 0x7fde6014c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde66133640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde601599e0 0x7fde6014c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde65932640 1 -- 172.21.15.32:0/324225965 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde600c90e0 msgr2=0x7fde6014c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde65932640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde600c90e0 0x7fde6014c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde65932640 1 -- 172.21.15.32:0/324225965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde601599e0 msgr2=0x7fde6014c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde65932640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde601599e0 0x7fde6014c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde65932640 1 -- 172.21.15.32:0/324225965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fde58009b30 con 0x7fde600b0b30 2023-12-20T04:29:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde65932640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde600b0b30 0x7fde60152ab0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00e870 tx=0x7fde5c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde567fc640 1 -- 172.21.15.32:0/324225965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c0095d0 con 0x7fde600b0b30 2023-12-20T04:29:39.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde567fc640 1 -- 172.21.15.32:0/324225965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fde5c00c940 con 0x7fde600b0b30 2023-12-20T04:29:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde567fc640 1 -- 172.21.15.32:0/324225965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fde5c00ccb0 con 0x7fde600b0b30 2023-12-20T04:29:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde67b96640 1 -- 172.21.15.32:0/324225965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fde600c7900 con 0x7fde600b0b30 2023-12-20T04:29:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde67b96640 1 -- 172.21.15.32:0/324225965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fde600c7ec0 con 0x7fde600b0b30 2023-12-20T04:29:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde67b96640 1 -- 172.21.15.32:0/324225965 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fde6004e000 con 0x7fde600b0b30 2023-12-20T04:29:39.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.510+0000 7fde567fc640 1 -- 172.21.15.32:0/324225965 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fde5c013470 con 0x7fde600b0b30 2023-12-20T04:29:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.514+0000 7fde567fc640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde44065a60 0x7fde44067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.514+0000 7fde567fc640 1 -- 172.21.15.32:0/324225965 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fde5c083670 con 0x7fde600b0b30 2023-12-20T04:29:39.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.514+0000 7fde65131640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde44065a60 0x7fde44067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.514+0000 7fde65131640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde44065a60 0x7fde44067f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde601557f0 tx=0x7fde50008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:39.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.518+0000 7fde567fc640 1 -- 172.21.15.32:0/324225965 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fde5c0510f0 con 0x7fde600b0b30 2023-12-20T04:29:39.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.674+0000 7fde67b96640 1 -- 172.21.15.32:0/324225965 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fde600c7790 con 0x7fde44065a60 2023-12-20T04:29:39.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.682+0000 7fde567fc640 1 -- 172.21.15.32:0/324225965 <== mgr.4104 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288938 (secure 0 0 0) 0x7fde600c7790 con 0x7fde44065a60 2023-12-20T04:29:39.689 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:39.692 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 -- 172.21.15.32:0/324225965 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde44065a60 msgr2=0x7fde44067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde44065a60 0x7fde44067f20 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fde601557f0 tx=0x7fde50008040 comp rx=0 tx=0).stop 2023-12-20T04:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 -- 172.21.15.32:0/324225965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde600b0b30 msgr2=0x7fde60152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde600b0b30 0x7fde60152ab0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7fde5c00e870 tx=0x7fde5c00ed30 comp rx=0 tx=0).stop 2023-12-20T04:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 -- 172.21.15.32:0/324225965 shutdown_connections 2023-12-20T04:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fde44065a60 0x7fde44067f20 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fde601599e0 0x7fde6014c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fde600c90e0 0x7fde6014c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 --2- 172.21.15.32:0/324225965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fde600b0b30 0x7fde60152ab0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 -- 172.21.15.32:0/324225965 >> 172.21.15.32:0/324225965 conn(0x7fde6006a320 msgr2=0x7fde60153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:39.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 -- 172.21.15.32:0/324225965 shutdown_connections 2023-12-20T04:29:39.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.686+0000 7fde67b96640 1 -- 172.21.15.32:0/324225965 wait complete. 2023-12-20T04:29:39.710 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":65,"stamp":"2023-12-20T04:29:38.138676+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":24,"num_object_clones":0,"num_object_copies":48,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":24,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":204,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":156,"ondisk_log_size":156,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":171,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":244520,"kb_used_data":4328,"kb_used_omap":0,"kb_used_meta":240128,"kb_avail":754730200,"statfs":{"total":773094113280,"available":772843724800,"internally_reserved":0,"allocated":4431872,"data_stored":2437924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":245891072},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"7.939139"},"pg_stats":[{"pgid":"5.3f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.189645+0000","last_change":"2023-12-20T04:29:30.655298+0000","last_active":"2023-12-20T04:29:32.189645+0000","last_peered":"2023-12-20T04:29:32.189645+0000","last_clean":"2023-12-20T04:29:32.189645+0000","last_became_active":"2023-12-20T04:29:30.654910+0000","last_became_peered":"2023-12-20T04:29:30.654910+0000","last_unstale":"2023-12-20T04:29:32.189645+0000","last_undegraded":"2023-12-20T04:29:32.189645+0000","last_fullsized":"2023-12-20T04:29:32.189645+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:42.225653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.3e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854278+0000","last_change":"2023-12-20T04:29:30.634459+0000","last_active":"2023-12-20T04:29:31.854278+0000","last_peered":"2023-12-20T04:29:31.854278+0000","last_clean":"2023-12-20T04:29:31.854278+0000","last_became_active":"2023-12-20T04:29:30.633546+0000","last_became_peered":"2023-12-20T04:29:30.633546+0000","last_unstale":"2023-12-20T04:29:31.854278+0000","last_undegraded":"2023-12-20T04:29:31.854278+0000","last_fullsized":"2023-12-20T04:29:31.854278+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:06.756922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.3d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955380+0000","last_change":"2023-12-20T04:29:30.655533+0000","last_active":"2023-12-20T04:29:32.955380+0000","last_peered":"2023-12-20T04:29:32.955380+0000","last_clean":"2023-12-20T04:29:32.955380+0000","last_became_active":"2023-12-20T04:29:30.650108+0000","last_became_peered":"2023-12-20T04:29:30.650108+0000","last_unstale":"2023-12-20T04:29:32.955380+0000","last_undegraded":"2023-12-20T04:29:32.955380+0000","last_fullsized":"2023-12-20T04:29:32.955380+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:34:12.655851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.3c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622395+0000","last_change":"2023-12-20T04:29:30.646047+0000","last_active":"2023-12-20T04:29:31.622395+0000","last_peered":"2023-12-20T04:29:31.622395+0000","last_clean":"2023-12-20T04:29:31.622395+0000","last_became_active":"2023-12-20T04:29:30.645668+0000","last_became_peered":"2023-12-20T04:29:30.645668+0000","last_unstale":"2023-12-20T04:29:31.622395+0000","last_undegraded":"2023-12-20T04:29:31.622395+0000","last_fullsized":"2023-12-20T04:29:31.622395+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:00:35.598112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.3b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928493+0000","last_change":"2023-12-20T04:29:30.625945+0000","last_active":"2023-12-20T04:29:31.928493+0000","last_peered":"2023-12-20T04:29:31.928493+0000","last_clean":"2023-12-20T04:29:31.928493+0000","last_became_active":"2023-12-20T04:29:30.625635+0000","last_became_peered":"2023-12-20T04:29:30.625635+0000","last_unstale":"2023-12-20T04:29:31.928493+0000","last_undegraded":"2023-12-20T04:29:31.928493+0000","last_fullsized":"2023-12-20T04:29:31.928493+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:29:33.925717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.3a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854088+0000","last_change":"2023-12-20T04:29:30.653519+0000","last_active":"2023-12-20T04:29:31.854088+0000","last_peered":"2023-12-20T04:29:31.854088+0000","last_clean":"2023-12-20T04:29:31.854088+0000","last_became_active":"2023-12-20T04:29:30.653242+0000","last_became_peered":"2023-12-20T04:29:30.653242+0000","last_unstale":"2023-12-20T04:29:31.854088+0000","last_undegraded":"2023-12-20T04:29:31.854088+0000","last_fullsized":"2023-12-20T04:29:31.854088+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:24:31.947504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.39","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625700+0000","last_change":"2023-12-20T04:29:30.661216+0000","last_active":"2023-12-20T04:29:31.625700+0000","last_peered":"2023-12-20T04:29:31.625700+0000","last_clean":"2023-12-20T04:29:31.625700+0000","last_became_active":"2023-12-20T04:29:30.660768+0000","last_became_peered":"2023-12-20T04:29:30.660768+0000","last_unstale":"2023-12-20T04:29:31.625700+0000","last_undegraded":"2023-12-20T04:29:31.625700+0000","last_fullsized":"2023-12-20T04:29:31.625700+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:11:19.882387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.38","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625467+0000","last_change":"2023-12-20T04:29:30.648261+0000","last_active":"2023-12-20T04:29:31.625467+0000","last_peered":"2023-12-20T04:29:31.625467+0000","last_clean":"2023-12-20T04:29:31.625467+0000","last_became_active":"2023-12-20T04:29:30.647929+0000","last_became_peered":"2023-12-20T04:29:30.647929+0000","last_unstale":"2023-12-20T04:29:31.625467+0000","last_undegraded":"2023-12-20T04:29:31.625467+0000","last_fullsized":"2023-12-20T04:29:31.625467+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:19:15.119662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.37","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626646+0000","last_change":"2023-12-20T04:29:30.652472+0000","last_active":"2023-12-20T04:29:31.626646+0000","last_peered":"2023-12-20T04:29:31.626646+0000","last_clean":"2023-12-20T04:29:31.626646+0000","last_became_active":"2023-12-20T04:29:30.652114+0000","last_became_peered":"2023-12-20T04:29:30.652114+0000","last_unstale":"2023-12-20T04:29:31.626646+0000","last_undegraded":"2023-12-20T04:29:31.626646+0000","last_fullsized":"2023-12-20T04:29:31.626646+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:13.683201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.36","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622430+0000","last_change":"2023-12-20T04:29:30.642253+0000","last_active":"2023-12-20T04:29:31.622430+0000","last_peered":"2023-12-20T04:29:31.622430+0000","last_clean":"2023-12-20T04:29:31.622430+0000","last_became_active":"2023-12-20T04:29:30.641994+0000","last_became_peered":"2023-12-20T04:29:30.641994+0000","last_unstale":"2023-12-20T04:29:31.622430+0000","last_undegraded":"2023-12-20T04:29:31.622430+0000","last_fullsized":"2023-12-20T04:29:31.622430+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:36.244013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.35","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626151+0000","last_change":"2023-12-20T04:29:30.656023+0000","last_active":"2023-12-20T04:29:31.626151+0000","last_peered":"2023-12-20T04:29:31.626151+0000","last_clean":"2023-12-20T04:29:31.626151+0000","last_became_active":"2023-12-20T04:29:30.655720+0000","last_became_peered":"2023-12-20T04:29:30.655720+0000","last_unstale":"2023-12-20T04:29:31.626151+0000","last_undegraded":"2023-12-20T04:29:31.626151+0000","last_fullsized":"2023-12-20T04:29:31.626151+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:34:12.655851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.34","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955241+0000","last_change":"2023-12-20T04:29:30.640182+0000","last_active":"2023-12-20T04:29:32.955241+0000","last_peered":"2023-12-20T04:29:32.955241+0000","last_clean":"2023-12-20T04:29:32.955241+0000","last_became_active":"2023-12-20T04:29:30.639672+0000","last_became_peered":"2023-12-20T04:29:30.639672+0000","last_unstale":"2023-12-20T04:29:32.955241+0000","last_undegraded":"2023-12-20T04:29:32.955241+0000","last_fullsized":"2023-12-20T04:29:32.955241+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:24:31.947504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.33","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928428+0000","last_change":"2023-12-20T04:29:30.635754+0000","last_active":"2023-12-20T04:29:31.928428+0000","last_peered":"2023-12-20T04:29:31.928428+0000","last_clean":"2023-12-20T04:29:31.928428+0000","last_became_active":"2023-12-20T04:29:30.635447+0000","last_became_peered":"2023-12-20T04:29:30.635447+0000","last_unstale":"2023-12-20T04:29:31.928428+0000","last_undegraded":"2023-12-20T04:29:31.928428+0000","last_fullsized":"2023-12-20T04:29:31.928428+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:45.142508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.32","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854122+0000","last_change":"2023-12-20T04:29:30.627375+0000","last_active":"2023-12-20T04:29:31.854122+0000","last_peered":"2023-12-20T04:29:31.854122+0000","last_clean":"2023-12-20T04:29:31.854122+0000","last_became_active":"2023-12-20T04:29:30.627110+0000","last_became_peered":"2023-12-20T04:29:30.627110+0000","last_unstale":"2023-12-20T04:29:31.854122+0000","last_undegraded":"2023-12-20T04:29:31.854122+0000","last_fullsized":"2023-12-20T04:29:31.854122+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:49.286363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.31","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.954997+0000","last_change":"2023-12-20T04:29:30.653951+0000","last_active":"2023-12-20T04:29:32.954997+0000","last_peered":"2023-12-20T04:29:32.954997+0000","last_clean":"2023-12-20T04:29:32.954997+0000","last_became_active":"2023-12-20T04:29:30.653649+0000","last_became_peered":"2023-12-20T04:29:30.653649+0000","last_unstale":"2023-12-20T04:29:32.954997+0000","last_undegraded":"2023-12-20T04:29:32.954997+0000","last_fullsized":"2023-12-20T04:29:32.954997+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:49.286363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.30","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625807+0000","last_change":"2023-12-20T04:29:30.653428+0000","last_active":"2023-12-20T04:29:31.625807+0000","last_peered":"2023-12-20T04:29:31.625807+0000","last_clean":"2023-12-20T04:29:31.625807+0000","last_became_active":"2023-12-20T04:29:30.653143+0000","last_became_peered":"2023-12-20T04:29:30.653143+0000","last_unstale":"2023-12-20T04:29:31.625807+0000","last_undegraded":"2023-12-20T04:29:31.625807+0000","last_fullsized":"2023-12-20T04:29:31.625807+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:49.286363+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626369+0000","last_change":"2023-12-20T04:29:30.661644+0000","last_active":"2023-12-20T04:29:31.626369+0000","last_peered":"2023-12-20T04:29:31.626369+0000","last_clean":"2023-12-20T04:29:31.626369+0000","last_became_active":"2023-12-20T04:29:30.661393+0000","last_became_peered":"2023-12-20T04:29:30.661393+0000","last_unstale":"2023-12-20T04:29:31.626369+0000","last_undegraded":"2023-12-20T04:29:31.626369+0000","last_fullsized":"2023-12-20T04:29:31.626369+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:06.756922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.2e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854314+0000","last_change":"2023-12-20T04:29:30.638164+0000","last_active":"2023-12-20T04:29:31.854314+0000","last_peered":"2023-12-20T04:29:31.854314+0000","last_clean":"2023-12-20T04:29:31.854314+0000","last_became_active":"2023-12-20T04:29:30.637901+0000","last_became_peered":"2023-12-20T04:29:30.637901+0000","last_unstale":"2023-12-20T04:29:31.854314+0000","last_undegraded":"2023-12-20T04:29:31.854314+0000","last_fullsized":"2023-12-20T04:29:31.854314+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:00:58.157886+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854084+0000","last_change":"2023-12-20T04:29:30.634579+0000","last_active":"2023-12-20T04:29:31.854084+0000","last_peered":"2023-12-20T04:29:31.854084+0000","last_clean":"2023-12-20T04:29:31.854084+0000","last_became_active":"2023-12-20T04:29:30.634314+0000","last_became_peered":"2023-12-20T04:29:30.634314+0000","last_unstale":"2023-12-20T04:29:31.854084+0000","last_undegraded":"2023-12-20T04:29:31.854084+0000","last_fullsized":"2023-12-20T04:29:31.854084+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:29.127931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.2c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928453+0000","last_change":"2023-12-20T04:29:30.645563+0000","last_active":"2023-12-20T04:29:31.928453+0000","last_peered":"2023-12-20T04:29:31.928453+0000","last_clean":"2023-12-20T04:29:31.928453+0000","last_became_active":"2023-12-20T04:29:30.645324+0000","last_became_peered":"2023-12-20T04:29:30.645324+0000","last_unstale":"2023-12-20T04:29:31.928453+0000","last_undegraded":"2023-12-20T04:29:31.928453+0000","last_fullsized":"2023-12-20T04:29:31.928453+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:50.074492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.2b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625811+0000","last_change":"2023-12-20T04:29:30.651654+0000","last_active":"2023-12-20T04:29:31.625811+0000","last_peered":"2023-12-20T04:29:31.625811+0000","last_clean":"2023-12-20T04:29:31.625811+0000","last_became_active":"2023-12-20T04:29:30.651333+0000","last_became_peered":"2023-12-20T04:29:30.651333+0000","last_unstale":"2023-12-20T04:29:31.625811+0000","last_undegraded":"2023-12-20T04:29:31.625811+0000","last_fullsized":"2023-12-20T04:29:31.625811+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:43.348011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.2a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625561+0000","last_change":"2023-12-20T04:29:30.649495+0000","last_active":"2023-12-20T04:29:31.625561+0000","last_peered":"2023-12-20T04:29:31.625561+0000","last_clean":"2023-12-20T04:29:31.625561+0000","last_became_active":"2023-12-20T04:29:30.649215+0000","last_became_peered":"2023-12-20T04:29:30.649215+0000","last_unstale":"2023-12-20T04:29:31.625561+0000","last_undegraded":"2023-12-20T04:29:31.625561+0000","last_fullsized":"2023-12-20T04:29:31.625561+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:45.142508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.29","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.930181+0000","last_change":"2023-12-20T04:29:30.644827+0000","last_active":"2023-12-20T04:29:32.930181+0000","last_peered":"2023-12-20T04:29:32.930181+0000","last_clean":"2023-12-20T04:29:32.930181+0000","last_became_active":"2023-12-20T04:29:30.641962+0000","last_became_peered":"2023-12-20T04:29:30.641962+0000","last_unstale":"2023-12-20T04:29:32.930181+0000","last_undegraded":"2023-12-20T04:29:32.930181+0000","last_fullsized":"2023-12-20T04:29:32.930181+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:44:31.379624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.28","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955000+0000","last_change":"2023-12-20T04:29:30.645099+0000","last_active":"2023-12-20T04:29:32.955000+0000","last_peered":"2023-12-20T04:29:32.955000+0000","last_clean":"2023-12-20T04:29:32.955000+0000","last_became_active":"2023-12-20T04:29:30.644820+0000","last_became_peered":"2023-12-20T04:29:30.644820+0000","last_unstale":"2023-12-20T04:29:32.955000+0000","last_undegraded":"2023-12-20T04:29:32.955000+0000","last_fullsized":"2023-12-20T04:29:32.955000+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:13:08.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.27","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622700+0000","last_change":"2023-12-20T04:29:30.651618+0000","last_active":"2023-12-20T04:29:31.622700+0000","last_peered":"2023-12-20T04:29:31.622700+0000","last_clean":"2023-12-20T04:29:31.622700+0000","last_became_active":"2023-12-20T04:29:30.651281+0000","last_became_peered":"2023-12-20T04:29:30.651281+0000","last_unstale":"2023-12-20T04:29:31.622700+0000","last_undegraded":"2023-12-20T04:29:31.622700+0000","last_fullsized":"2023-12-20T04:29:31.622700+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:13.683201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.26","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854368+0000","last_change":"2023-12-20T04:29:30.654081+0000","last_active":"2023-12-20T04:29:31.854368+0000","last_peered":"2023-12-20T04:29:31.854368+0000","last_clean":"2023-12-20T04:29:31.854368+0000","last_became_active":"2023-12-20T04:29:30.653794+0000","last_became_peered":"2023-12-20T04:29:30.653794+0000","last_unstale":"2023-12-20T04:29:31.854368+0000","last_undegraded":"2023-12-20T04:29:31.854368+0000","last_fullsized":"2023-12-20T04:29:31.854368+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:41.192000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.25","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622635+0000","last_change":"2023-12-20T04:29:30.647664+0000","last_active":"2023-12-20T04:29:31.622635+0000","last_peered":"2023-12-20T04:29:31.622635+0000","last_clean":"2023-12-20T04:29:31.622635+0000","last_became_active":"2023-12-20T04:29:30.646694+0000","last_became_peered":"2023-12-20T04:29:30.646694+0000","last_unstale":"2023-12-20T04:29:31.622635+0000","last_undegraded":"2023-12-20T04:29:31.622635+0000","last_fullsized":"2023-12-20T04:29:31.622635+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:42.395721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.24","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955333+0000","last_change":"2023-12-20T04:29:30.636178+0000","last_active":"2023-12-20T04:29:32.955333+0000","last_peered":"2023-12-20T04:29:32.955333+0000","last_clean":"2023-12-20T04:29:32.955333+0000","last_became_active":"2023-12-20T04:29:30.635857+0000","last_became_peered":"2023-12-20T04:29:30.635857+0000","last_unstale":"2023-12-20T04:29:32.955333+0000","last_undegraded":"2023-12-20T04:29:32.955333+0000","last_fullsized":"2023-12-20T04:29:32.955333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:57.967703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.23","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626019+0000","last_change":"2023-12-20T04:29:30.644814+0000","last_active":"2023-12-20T04:29:31.626019+0000","last_peered":"2023-12-20T04:29:31.626019+0000","last_clean":"2023-12-20T04:29:31.626019+0000","last_became_active":"2023-12-20T04:29:30.644526+0000","last_became_peered":"2023-12-20T04:29:30.644526+0000","last_unstale":"2023-12-20T04:29:31.626019+0000","last_undegraded":"2023-12-20T04:29:31.626019+0000","last_fullsized":"2023-12-20T04:29:31.626019+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:50.074492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.22","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625710+0000","last_change":"2023-12-20T04:29:30.661180+0000","last_active":"2023-12-20T04:29:31.625710+0000","last_peered":"2023-12-20T04:29:31.625710+0000","last_clean":"2023-12-20T04:29:31.625710+0000","last_became_active":"2023-12-20T04:29:30.660878+0000","last_became_peered":"2023-12-20T04:29:30.660878+0000","last_unstale":"2023-12-20T04:29:31.625710+0000","last_undegraded":"2023-12-20T04:29:31.625710+0000","last_fullsized":"2023-12-20T04:29:31.625710+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:24:31.947504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.21","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.189678+0000","last_change":"2023-12-20T04:29:30.636109+0000","last_active":"2023-12-20T04:29:32.189678+0000","last_peered":"2023-12-20T04:29:32.189678+0000","last_clean":"2023-12-20T04:29:32.189678+0000","last_became_active":"2023-12-20T04:29:30.635710+0000","last_became_peered":"2023-12-20T04:29:30.635710+0000","last_unstale":"2023-12-20T04:29:32.189678+0000","last_undegraded":"2023-12-20T04:29:32.189678+0000","last_fullsized":"2023-12-20T04:29:32.189678+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:30.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.20","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625574+0000","last_change":"2023-12-20T04:29:30.648795+0000","last_active":"2023-12-20T04:29:31.625574+0000","last_peered":"2023-12-20T04:29:31.625574+0000","last_clean":"2023-12-20T04:29:31.625574+0000","last_became_active":"2023-12-20T04:29:30.648490+0000","last_became_peered":"2023-12-20T04:29:30.648490+0000","last_unstale":"2023-12-20T04:29:31.625574+0000","last_undegraded":"2023-12-20T04:29:31.625574+0000","last_fullsized":"2023-12-20T04:29:31.625574+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:36.244013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"5.1e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.930570+0000","last_change":"2023-12-20T04:29:30.642782+0000","last_active":"2023-12-20T04:29:32.930570+0000","last_peered":"2023-12-20T04:29:32.930570+0000","last_clean":"2023-12-20T04:29:32.930570+0000","last_became_active":"2023-12-20T04:29:30.642444+0000","last_became_peered":"2023-12-20T04:29:30.642444+0000","last_unstale":"2023-12-20T04:29:32.930570+0000","last_undegraded":"2023-12-20T04:29:32.930570+0000","last_fullsized":"2023-12-20T04:29:32.930570+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:13.683201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1f","version":"34'4","reported_seq":15,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.655050+0000","last_change":"2023-12-20T04:29:30.233682+0000","last_active":"2023-12-20T04:29:30.655050+0000","last_peered":"2023-12-20T04:29:30.655050+0000","last_clean":"2023-12-20T04:29:30.655050+0000","last_became_active":"2023-12-20T04:29:30.233450+0000","last_became_peered":"2023-12-20T04:29:30.233450+0000","last_unstale":"2023-12-20T04:29:30.655050+0000","last_undegraded":"2023-12-20T04:29:30.655050+0000","last_fullsized":"2023-12-20T04:29:30.655050+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.1f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.189749+0000","last_change":"2023-12-20T04:29:30.655215+0000","last_active":"2023-12-20T04:29:32.189749+0000","last_peered":"2023-12-20T04:29:32.189749+0000","last_clean":"2023-12-20T04:29:32.189749+0000","last_became_active":"2023-12-20T04:29:30.654666+0000","last_became_peered":"2023-12-20T04:29:30.654666+0000","last_unstale":"2023-12-20T04:29:32.189749+0000","last_undegraded":"2023-12-20T04:29:32.189749+0000","last_fullsized":"2023-12-20T04:29:32.189749+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:18.283893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.1e","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.633102+0000","last_change":"2023-12-20T04:29:30.225518+0000","last_active":"2023-12-20T04:29:30.633102+0000","last_peered":"2023-12-20T04:29:30.633102+0000","last_clean":"2023-12-20T04:29:30.633102+0000","last_became_active":"2023-12-20T04:29:30.225211+0000","last_became_peered":"2023-12-20T04:29:30.225211+0000","last_unstale":"2023-12-20T04:29:30.633102+0000","last_undegraded":"2023-12-20T04:29:30.633102+0000","last_fullsized":"2023-12-20T04:29:30.633102+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.1c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626443+0000","last_change":"2023-12-20T04:29:30.652588+0000","last_active":"2023-12-20T04:29:31.626443+0000","last_peered":"2023-12-20T04:29:31.626443+0000","last_clean":"2023-12-20T04:29:31.626443+0000","last_became_active":"2023-12-20T04:29:30.652296+0000","last_became_peered":"2023-12-20T04:29:30.652296+0000","last_unstale":"2023-12-20T04:29:31.626443+0000","last_undegraded":"2023-12-20T04:29:31.626443+0000","last_fullsized":"2023-12-20T04:29:31.626443+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:30.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.1d","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.639776+0000","last_change":"2023-12-20T04:29:30.233148+0000","last_active":"2023-12-20T04:29:30.639776+0000","last_peered":"2023-12-20T04:29:30.639776+0000","last_clean":"2023-12-20T04:29:30.639776+0000","last_became_active":"2023-12-20T04:29:30.232686+0000","last_became_peered":"2023-12-20T04:29:30.232686+0000","last_unstale":"2023-12-20T04:29:30.639776+0000","last_undegraded":"2023-12-20T04:29:30.639776+0000","last_fullsized":"2023-12-20T04:29:30.639776+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:17.281082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.1d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622736+0000","last_change":"2023-12-20T04:29:30.648689+0000","last_active":"2023-12-20T04:29:31.622736+0000","last_peered":"2023-12-20T04:29:31.622736+0000","last_clean":"2023-12-20T04:29:31.622736+0000","last_became_active":"2023-12-20T04:29:30.648308+0000","last_became_peered":"2023-12-20T04:29:30.648308+0000","last_unstale":"2023-12-20T04:29:31.622736+0000","last_undegraded":"2023-12-20T04:29:31.622736+0000","last_fullsized":"2023-12-20T04:29:31.622736+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:13:08.163569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1c","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.638569+0000","last_change":"2023-12-20T04:29:30.224172+0000","last_active":"2023-12-20T04:29:30.638569+0000","last_peered":"2023-12-20T04:29:30.638569+0000","last_clean":"2023-12-20T04:29:30.638569+0000","last_became_active":"2023-12-20T04:29:30.223865+0000","last_became_peered":"2023-12-20T04:29:30.223865+0000","last_unstale":"2023-12-20T04:29:30.638569+0000","last_undegraded":"2023-12-20T04:29:30.638569+0000","last_fullsized":"2023-12-20T04:29:30.638569+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"5.1a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625879+0000","last_change":"2023-12-20T04:29:30.652768+0000","last_active":"2023-12-20T04:29:31.625879+0000","last_peered":"2023-12-20T04:29:31.625879+0000","last_clean":"2023-12-20T04:29:31.625879+0000","last_became_active":"2023-12-20T04:29:30.652427+0000","last_became_peered":"2023-12-20T04:29:30.652427+0000","last_unstale":"2023-12-20T04:29:31.625879+0000","last_undegraded":"2023-12-20T04:29:31.625879+0000","last_fullsized":"2023-12-20T04:29:31.625879+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:57.967703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.1b","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.649107+0000","last_change":"2023-12-20T04:29:30.231289+0000","last_active":"2023-12-20T04:29:30.649107+0000","last_peered":"2023-12-20T04:29:30.649107+0000","last_clean":"2023-12-20T04:29:30.649107+0000","last_became_active":"2023-12-20T04:29:30.230075+0000","last_became_peered":"2023-12-20T04:29:30.230075+0000","last_unstale":"2023-12-20T04:29:30.649107+0000","last_undegraded":"2023-12-20T04:29:30.649107+0000","last_fullsized":"2023-12-20T04:29:30.649107+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:17:46.344276+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955112+0000","last_change":"2023-12-20T04:29:30.646861+0000","last_active":"2023-12-20T04:29:32.955112+0000","last_peered":"2023-12-20T04:29:32.955112+0000","last_clean":"2023-12-20T04:29:32.955112+0000","last_became_active":"2023-12-20T04:29:30.646560+0000","last_became_peered":"2023-12-20T04:29:30.646560+0000","last_unstale":"2023-12-20T04:29:32.955112+0000","last_undegraded":"2023-12-20T04:29:32.955112+0000","last_fullsized":"2023-12-20T04:29:32.955112+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:42.395721+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1a","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.649412+0000","last_change":"2023-12-20T04:29:30.232127+0000","last_active":"2023-12-20T04:29:30.649412+0000","last_peered":"2023-12-20T04:29:30.649412+0000","last_clean":"2023-12-20T04:29:30.649412+0000","last_became_active":"2023-12-20T04:29:30.231754+0000","last_became_peered":"2023-12-20T04:29:30.231754+0000","last_unstale":"2023-12-20T04:29:30.649412+0000","last_undegraded":"2023-12-20T04:29:30.649412+0000","last_fullsized":"2023-12-20T04:29:30.649412+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:35:23.040608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.18","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626165+0000","last_change":"2023-12-20T04:29:30.660699+0000","last_active":"2023-12-20T04:29:31.626165+0000","last_peered":"2023-12-20T04:29:31.626165+0000","last_clean":"2023-12-20T04:29:31.626165+0000","last_became_active":"2023-12-20T04:29:30.660412+0000","last_became_peered":"2023-12-20T04:29:30.660412+0000","last_unstale":"2023-12-20T04:29:31.626165+0000","last_undegraded":"2023-12-20T04:29:31.626165+0000","last_fullsized":"2023-12-20T04:29:31.626165+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:44:31.379624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.615460+0000","last_change":"2023-12-20T04:29:30.221627+0000","last_active":"2023-12-20T04:29:30.615460+0000","last_peered":"2023-12-20T04:29:30.615460+0000","last_clean":"2023-12-20T04:29:30.615460+0000","last_became_active":"2023-12-20T04:29:30.221347+0000","last_became_peered":"2023-12-20T04:29:30.221347+0000","last_unstale":"2023-12-20T04:29:30.615460+0000","last_undegraded":"2023-12-20T04:29:30.615460+0000","last_fullsized":"2023-12-20T04:29:30.615460+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:42:48.712967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.19","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622155+0000","last_change":"2023-12-20T04:29:30.638486+0000","last_active":"2023-12-20T04:29:31.622155+0000","last_peered":"2023-12-20T04:29:31.622155+0000","last_clean":"2023-12-20T04:29:31.622155+0000","last_became_active":"2023-12-20T04:29:30.638192+0000","last_became_peered":"2023-12-20T04:29:30.638192+0000","last_unstale":"2023-12-20T04:29:31.622155+0000","last_undegraded":"2023-12-20T04:29:31.622155+0000","last_fullsized":"2023-12-20T04:29:31.622155+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:57.967703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.640646+0000","last_change":"2023-12-20T04:29:30.231286+0000","last_active":"2023-12-20T04:29:30.640646+0000","last_peered":"2023-12-20T04:29:30.640646+0000","last_clean":"2023-12-20T04:29:30.640646+0000","last_became_active":"2023-12-20T04:29:30.230988+0000","last_became_peered":"2023-12-20T04:29:30.230988+0000","last_unstale":"2023-12-20T04:29:30.640646+0000","last_undegraded":"2023-12-20T04:29:30.640646+0000","last_fullsized":"2023-12-20T04:29:30.640646+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:50:04.001754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.16","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955455+0000","last_change":"2023-12-20T04:29:30.652262+0000","last_active":"2023-12-20T04:29:32.955455+0000","last_peered":"2023-12-20T04:29:32.955455+0000","last_clean":"2023-12-20T04:29:32.955455+0000","last_became_active":"2023-12-20T04:29:30.647614+0000","last_became_peered":"2023-12-20T04:29:30.647614+0000","last_unstale":"2023-12-20T04:29:32.955455+0000","last_undegraded":"2023-12-20T04:29:32.955455+0000","last_fullsized":"2023-12-20T04:29:32.955455+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:36.244013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.616961+0000","last_change":"2023-12-20T04:29:30.223687+0000","last_active":"2023-12-20T04:29:30.616961+0000","last_peered":"2023-12-20T04:29:30.616961+0000","last_clean":"2023-12-20T04:29:30.616961+0000","last_became_active":"2023-12-20T04:29:30.223353+0000","last_became_peered":"2023-12-20T04:29:30.223353+0000","last_unstale":"2023-12-20T04:29:30.616961+0000","last_undegraded":"2023-12-20T04:29:30.616961+0000","last_fullsized":"2023-12-20T04:29:30.616961+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:00.360055+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.17","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854127+0000","last_change":"2023-12-20T04:29:30.647207+0000","last_active":"2023-12-20T04:29:31.854127+0000","last_peered":"2023-12-20T04:29:31.854127+0000","last_clean":"2023-12-20T04:29:31.854127+0000","last_became_active":"2023-12-20T04:29:30.646695+0000","last_became_peered":"2023-12-20T04:29:30.646695+0000","last_unstale":"2023-12-20T04:29:31.854127+0000","last_undegraded":"2023-12-20T04:29:31.854127+0000","last_fullsized":"2023-12-20T04:29:31.854127+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:06:15.798745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.623178+0000","last_change":"2023-12-20T04:29:30.230906+0000","last_active":"2023-12-20T04:29:30.623178+0000","last_peered":"2023-12-20T04:29:30.623178+0000","last_clean":"2023-12-20T04:29:30.623178+0000","last_became_active":"2023-12-20T04:29:30.230515+0000","last_became_peered":"2023-12-20T04:29:30.230515+0000","last_unstale":"2023-12-20T04:29:30.623178+0000","last_undegraded":"2023-12-20T04:29:30.623178+0000","last_fullsized":"2023-12-20T04:29:30.623178+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.14","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854241+0000","last_change":"2023-12-20T04:29:30.654536+0000","last_active":"2023-12-20T04:29:31.854241+0000","last_peered":"2023-12-20T04:29:31.854241+0000","last_clean":"2023-12-20T04:29:31.854241+0000","last_became_active":"2023-12-20T04:29:30.653861+0000","last_became_peered":"2023-12-20T04:29:30.653861+0000","last_unstale":"2023-12-20T04:29:31.854241+0000","last_undegraded":"2023-12-20T04:29:31.854241+0000","last_fullsized":"2023-12-20T04:29:31.854241+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:11:19.882387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.15","version":"34'3","reported_seq":14,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.647354+0000","last_change":"2023-12-20T04:29:30.226215+0000","last_active":"2023-12-20T04:29:30.647354+0000","last_peered":"2023-12-20T04:29:30.647354+0000","last_clean":"2023-12-20T04:29:30.647354+0000","last_became_active":"2023-12-20T04:29:30.225949+0000","last_became_peered":"2023-12-20T04:29:30.225949+0000","last_unstale":"2023-12-20T04:29:30.647354+0000","last_undegraded":"2023-12-20T04:29:30.647354+0000","last_fullsized":"2023-12-20T04:29:30.647354+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.15","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955333+0000","last_change":"2023-12-20T04:29:30.652886+0000","last_active":"2023-12-20T04:29:32.955333+0000","last_peered":"2023-12-20T04:29:32.955333+0000","last_clean":"2023-12-20T04:29:32.955333+0000","last_became_active":"2023-12-20T04:29:30.652626+0000","last_became_peered":"2023-12-20T04:29:30.652626+0000","last_unstale":"2023-12-20T04:29:32.955333+0000","last_undegraded":"2023-12-20T04:29:32.955333+0000","last_fullsized":"2023-12-20T04:29:32.955333+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:07.590492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.14","version":"34'6","reported_seq":17,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.657824+0000","last_change":"2023-12-20T04:29:30.228644+0000","last_active":"2023-12-20T04:29:30.657824+0000","last_peered":"2023-12-20T04:29:30.657824+0000","last_clean":"2023-12-20T04:29:30.657824+0000","last_became_active":"2023-12-20T04:29:30.228268+0000","last_became_peered":"2023-12-20T04:29:30.228268+0000","last_unstale":"2023-12-20T04:29:30.657824+0000","last_undegraded":"2023-12-20T04:29:30.657824+0000","last_fullsized":"2023-12-20T04:29:30.657824+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:29:32.922907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.12","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622166+0000","last_change":"2023-12-20T04:29:30.651226+0000","last_active":"2023-12-20T04:29:31.622166+0000","last_peered":"2023-12-20T04:29:31.622166+0000","last_clean":"2023-12-20T04:29:31.622166+0000","last_became_active":"2023-12-20T04:29:30.650941+0000","last_became_peered":"2023-12-20T04:29:30.650941+0000","last_unstale":"2023-12-20T04:29:31.622166+0000","last_undegraded":"2023-12-20T04:29:31.622166+0000","last_fullsized":"2023-12-20T04:29:31.622166+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:52.889158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.13","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.660131+0000","last_change":"2023-12-20T04:29:30.233204+0000","last_active":"2023-12-20T04:29:30.660131+0000","last_peered":"2023-12-20T04:29:30.660131+0000","last_clean":"2023-12-20T04:29:30.660131+0000","last_became_active":"2023-12-20T04:29:30.232924+0000","last_became_peered":"2023-12-20T04:29:30.232924+0000","last_unstale":"2023-12-20T04:29:30.660131+0000","last_undegraded":"2023-12-20T04:29:30.660131+0000","last_fullsized":"2023-12-20T04:29:30.660131+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.13","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.854180+0000","last_change":"2023-12-20T04:29:30.634013+0000","last_active":"2023-12-20T04:29:31.854180+0000","last_peered":"2023-12-20T04:29:31.854180+0000","last_clean":"2023-12-20T04:29:31.854180+0000","last_became_active":"2023-12-20T04:29:30.633703+0000","last_became_peered":"2023-12-20T04:29:30.633703+0000","last_unstale":"2023-12-20T04:29:31.854180+0000","last_undegraded":"2023-12-20T04:29:31.854180+0000","last_fullsized":"2023-12-20T04:29:31.854180+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:36.244013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.629155+0000","last_change":"2023-12-20T04:29:30.231628+0000","last_active":"2023-12-20T04:29:30.629155+0000","last_peered":"2023-12-20T04:29:30.629155+0000","last_clean":"2023-12-20T04:29:30.629155+0000","last_became_active":"2023-12-20T04:29:30.231363+0000","last_became_peered":"2023-12-20T04:29:30.231363+0000","last_unstale":"2023-12-20T04:29:30.629155+0000","last_undegraded":"2023-12-20T04:29:30.629155+0000","last_fullsized":"2023-12-20T04:29:30.629155+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:29.020652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.10","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625688+0000","last_change":"2023-12-20T04:29:30.650833+0000","last_active":"2023-12-20T04:29:31.625688+0000","last_peered":"2023-12-20T04:29:31.625688+0000","last_clean":"2023-12-20T04:29:31.625688+0000","last_became_active":"2023-12-20T04:29:30.650576+0000","last_became_peered":"2023-12-20T04:29:30.650576+0000","last_unstale":"2023-12-20T04:29:31.625688+0000","last_undegraded":"2023-12-20T04:29:31.625688+0000","last_fullsized":"2023-12-20T04:29:31.625688+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:52.889158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.615507+0000","last_change":"2023-12-20T04:29:30.227573+0000","last_active":"2023-12-20T04:29:30.615507+0000","last_peered":"2023-12-20T04:29:30.615507+0000","last_clean":"2023-12-20T04:29:30.615507+0000","last_became_active":"2023-12-20T04:29:30.227266+0000","last_became_peered":"2023-12-20T04:29:30.227266+0000","last_unstale":"2023-12-20T04:29:30.615507+0000","last_undegraded":"2023-12-20T04:29:30.615507+0000","last_fullsized":"2023-12-20T04:29:30.615507+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:42.345201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.11","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.189818+0000","last_change":"2023-12-20T04:29:30.659850+0000","last_active":"2023-12-20T04:29:32.189818+0000","last_peered":"2023-12-20T04:29:32.189818+0000","last_clean":"2023-12-20T04:29:32.189818+0000","last_became_active":"2023-12-20T04:29:30.659274+0000","last_became_peered":"2023-12-20T04:29:30.659274+0000","last_unstale":"2023-12-20T04:29:32.189818+0000","last_undegraded":"2023-12-20T04:29:32.189818+0000","last_fullsized":"2023-12-20T04:29:32.189818+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:01.362866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.10","version":"34'8","reported_seq":19,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.647976+0000","last_change":"2023-12-20T04:29:30.229228+0000","last_active":"2023-12-20T04:29:30.647976+0000","last_peered":"2023-12-20T04:29:30.647976+0000","last_clean":"2023-12-20T04:29:30.647976+0000","last_became_active":"2023-12-20T04:29:30.228933+0000","last_became_peered":"2023-12-20T04:29:30.228933+0000","last_unstale":"2023-12-20T04:29:30.647976+0000","last_undegraded":"2023-12-20T04:29:30.647976+0000","last_fullsized":"2023-12-20T04:29:30.647976+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.e","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.626218+0000","last_change":"2023-12-20T04:29:30.658500+0000","last_active":"2023-12-20T04:29:31.626218+0000","last_peered":"2023-12-20T04:29:31.626218+0000","last_clean":"2023-12-20T04:29:31.626218+0000","last_became_active":"2023-12-20T04:29:30.658220+0000","last_became_peered":"2023-12-20T04:29:30.658220+0000","last_unstale":"2023-12-20T04:29:31.626218+0000","last_undegraded":"2023-12-20T04:29:31.626218+0000","last_fullsized":"2023-12-20T04:29:31.626218+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:29:33.925717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.f","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.644729+0000","last_change":"2023-12-20T04:29:30.230687+0000","last_active":"2023-12-20T04:29:30.644729+0000","last_peered":"2023-12-20T04:29:30.644729+0000","last_clean":"2023-12-20T04:29:30.644729+0000","last_became_active":"2023-12-20T04:29:30.230395+0000","last_became_peered":"2023-12-20T04:29:30.230395+0000","last_unstale":"2023-12-20T04:29:30.644729+0000","last_undegraded":"2023-12-20T04:29:30.644729+0000","last_fullsized":"2023-12-20T04:29:30.644729+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:31:17.467772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.f","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955511+0000","last_change":"2023-12-20T04:29:30.652005+0000","last_active":"2023-12-20T04:29:32.955511+0000","last_peered":"2023-12-20T04:29:32.955511+0000","last_clean":"2023-12-20T04:29:32.955511+0000","last_became_active":"2023-12-20T04:29:30.651689+0000","last_became_peered":"2023-12-20T04:29:30.651689+0000","last_unstale":"2023-12-20T04:29:32.955511+0000","last_undegraded":"2023-12-20T04:29:32.955511+0000","last_fullsized":"2023-12-20T04:29:32.955511+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:52.889158+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.e","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.660255+0000","last_change":"2023-12-20T04:29:30.232810+0000","last_active":"2023-12-20T04:29:30.660255+0000","last_peered":"2023-12-20T04:29:30.660255+0000","last_clean":"2023-12-20T04:29:30.660255+0000","last_became_active":"2023-12-20T04:29:30.232515+0000","last_became_peered":"2023-12-20T04:29:30.232515+0000","last_unstale":"2023-12-20T04:29:30.660255+0000","last_undegraded":"2023-12-20T04:29:30.660255+0000","last_fullsized":"2023-12-20T04:29:30.660255+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.097973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.c","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.622401+0000","last_change":"2023-12-20T04:29:30.645904+0000","last_active":"2023-12-20T04:29:31.622401+0000","last_peered":"2023-12-20T04:29:31.622401+0000","last_clean":"2023-12-20T04:29:31.622401+0000","last_became_active":"2023-12-20T04:29:30.645393+0000","last_became_peered":"2023-12-20T04:29:30.645393+0000","last_unstale":"2023-12-20T04:29:31.622401+0000","last_undegraded":"2023-12-20T04:29:31.622401+0000","last_fullsized":"2023-12-20T04:29:31.622401+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:44:31.379624+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.d","version":"34'1","reported_seq":12,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.655227+0000","last_change":"2023-12-20T04:29:30.221914+0000","last_active":"2023-12-20T04:29:30.655227+0000","last_peered":"2023-12-20T04:29:30.655227+0000","last_clean":"2023-12-20T04:29:30.655227+0000","last_became_active":"2023-12-20T04:29:30.221627+0000","last_became_peered":"2023-12-20T04:29:30.221627+0000","last_unstale":"2023-12-20T04:29:30.655227+0000","last_undegraded":"2023-12-20T04:29:30.655227+0000","last_fullsized":"2023-12-20T04:29:30.655227+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:44:48.959029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.d","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928549+0000","last_change":"2023-12-20T04:29:30.648693+0000","last_active":"2023-12-20T04:29:31.928549+0000","last_peered":"2023-12-20T04:29:31.928549+0000","last_clean":"2023-12-20T04:29:31.928549+0000","last_became_active":"2023-12-20T04:29:30.648393+0000","last_became_peered":"2023-12-20T04:29:30.648393+0000","last_unstale":"2023-12-20T04:29:31.928549+0000","last_undegraded":"2023-12-20T04:29:31.928549+0000","last_fullsized":"2023-12-20T04:29:31.928549+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:10:18.283893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.639338+0000","last_change":"2023-12-20T04:29:30.230694+0000","last_active":"2023-12-20T04:29:30.639338+0000","last_peered":"2023-12-20T04:29:30.639338+0000","last_clean":"2023-12-20T04:29:30.639338+0000","last_became_active":"2023-12-20T04:29:30.230428+0000","last_became_peered":"2023-12-20T04:29:30.230428+0000","last_unstale":"2023-12-20T04:29:30.639338+0000","last_undegraded":"2023-12-20T04:29:30.639338+0000","last_fullsized":"2023-12-20T04:29:30.639338+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.a","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928323+0000","last_change":"2023-12-20T04:29:30.646456+0000","last_active":"2023-12-20T04:29:31.928323+0000","last_peered":"2023-12-20T04:29:31.928323+0000","last_clean":"2023-12-20T04:29:31.928323+0000","last_became_active":"2023-12-20T04:29:30.646027+0000","last_became_peered":"2023-12-20T04:29:30.646027+0000","last_unstale":"2023-12-20T04:29:31.928323+0000","last_undegraded":"2023-12-20T04:29:31.928323+0000","last_fullsized":"2023-12-20T04:29:31.928323+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:00:01.362866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.b","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.649074+0000","last_change":"2023-12-20T04:29:30.218619+0000","last_active":"2023-12-20T04:29:30.649074+0000","last_peered":"2023-12-20T04:29:30.649074+0000","last_clean":"2023-12-20T04:29:30.649074+0000","last_became_active":"2023-12-20T04:29:30.218180+0000","last_became_peered":"2023-12-20T04:29:30.218180+0000","last_unstale":"2023-12-20T04:29:30.649074+0000","last_undegraded":"2023-12-20T04:29:30.649074+0000","last_fullsized":"2023-12-20T04:29:30.649074+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.b","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928509+0000","last_change":"2023-12-20T04:29:30.633870+0000","last_active":"2023-12-20T04:29:31.928509+0000","last_peered":"2023-12-20T04:29:31.928509+0000","last_clean":"2023-12-20T04:29:31.928509+0000","last_became_active":"2023-12-20T04:29:30.633586+0000","last_became_peered":"2023-12-20T04:29:30.633586+0000","last_unstale":"2023-12-20T04:29:31.928509+0000","last_undegraded":"2023-12-20T04:29:31.928509+0000","last_fullsized":"2023-12-20T04:29:31.928509+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:30.964333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.628414+0000","last_change":"2023-12-20T04:29:30.232351+0000","last_active":"2023-12-20T04:29:30.628414+0000","last_peered":"2023-12-20T04:29:30.628414+0000","last_clean":"2023-12-20T04:29:30.628414+0000","last_became_active":"2023-12-20T04:29:30.231883+0000","last_became_peered":"2023-12-20T04:29:30.231883+0000","last_unstale":"2023-12-20T04:29:30.628414+0000","last_undegraded":"2023-12-20T04:29:30.628414+0000","last_fullsized":"2023-12-20T04:29:30.628414+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:34:46.020178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"5.8","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928377+0000","last_change":"2023-12-20T04:29:30.633390+0000","last_active":"2023-12-20T04:29:31.928377+0000","last_peered":"2023-12-20T04:29:31.928377+0000","last_clean":"2023-12-20T04:29:31.928377+0000","last_became_active":"2023-12-20T04:29:30.632629+0000","last_became_peered":"2023-12-20T04:29:30.632629+0000","last_unstale":"2023-12-20T04:29:31.928377+0000","last_undegraded":"2023-12-20T04:29:31.928377+0000","last_fullsized":"2023-12-20T04:29:31.928377+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:43.348011+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.642615+0000","last_change":"2023-12-20T04:29:30.221611+0000","last_active":"2023-12-20T04:29:30.642615+0000","last_peered":"2023-12-20T04:29:30.642615+0000","last_clean":"2023-12-20T04:29:30.642615+0000","last_became_active":"2023-12-20T04:29:30.221315+0000","last_became_peered":"2023-12-20T04:29:30.221315+0000","last_unstale":"2023-12-20T04:29:30.642615+0000","last_undegraded":"2023-12-20T04:29:30.642615+0000","last_fullsized":"2023-12-20T04:29:30.642615+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:00.538815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.9","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625495+0000","last_change":"2023-12-20T04:29:30.661222+0000","last_active":"2023-12-20T04:29:31.625495+0000","last_peered":"2023-12-20T04:29:31.625495+0000","last_clean":"2023-12-20T04:29:31.625495+0000","last_became_active":"2023-12-20T04:29:30.660457+0000","last_became_peered":"2023-12-20T04:29:30.660457+0000","last_unstale":"2023-12-20T04:29:31.625495+0000","last_undegraded":"2023-12-20T04:29:31.625495+0000","last_fullsized":"2023-12-20T04:29:31.625495+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:41.192000+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.629159+0000","last_change":"2023-12-20T04:29:30.227738+0000","last_active":"2023-12-20T04:29:30.629159+0000","last_peered":"2023-12-20T04:29:30.629159+0000","last_clean":"2023-12-20T04:29:30.629159+0000","last_became_active":"2023-12-20T04:29:30.227464+0000","last_became_peered":"2023-12-20T04:29:30.227464+0000","last_unstale":"2023-12-20T04:29:30.629159+0000","last_undegraded":"2023-12-20T04:29:30.629159+0000","last_fullsized":"2023-12-20T04:29:30.629159+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:41.222843+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"5.6","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.928500+0000","last_change":"2023-12-20T04:29:30.640153+0000","last_active":"2023-12-20T04:29:31.928500+0000","last_peered":"2023-12-20T04:29:31.928500+0000","last_clean":"2023-12-20T04:29:31.928500+0000","last_became_active":"2023-12-20T04:29:30.639884+0000","last_became_peered":"2023-12-20T04:29:30.639884+0000","last_unstale":"2023-12-20T04:29:31.928500+0000","last_undegraded":"2023-12-20T04:29:31.928500+0000","last_fullsized":"2023-12-20T04:29:31.928500+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:42:49.715777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.7","version":"34'2","reported_seq":13,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.651264+0000","last_change":"2023-12-20T04:29:30.226593+0000","last_active":"2023-12-20T04:29:30.651264+0000","last_peered":"2023-12-20T04:29:30.651264+0000","last_clean":"2023-12-20T04:29:30.651264+0000","last_became_active":"2023-12-20T04:29:30.226331+0000","last_became_peered":"2023-12-20T04:29:30.226331+0000","last_unstale":"2023-12-20T04:29:30.651264+0000","last_undegraded":"2023-12-20T04:29:30.651264+0000","last_fullsized":"2023-12-20T04:29:30.651264+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:05:00.162402+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.7","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.955532+0000","last_change":"2023-12-20T04:29:30.657429+0000","last_active":"2023-12-20T04:29:32.955532+0000","last_peered":"2023-12-20T04:29:32.955532+0000","last_clean":"2023-12-20T04:29:32.955532+0000","last_became_active":"2023-12-20T04:29:30.656974+0000","last_became_peered":"2023-12-20T04:29:30.656974+0000","last_unstale":"2023-12-20T04:29:32.955532+0000","last_undegraded":"2023-12-20T04:29:32.955532+0000","last_fullsized":"2023-12-20T04:29:32.955532+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:19:15.119662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.6","version":"35'1","reported_seq":12,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.930657+0000","last_change":"2023-12-20T04:29:30.222922+0000","last_active":"2023-12-20T04:29:32.930657+0000","last_peered":"2023-12-20T04:29:32.930657+0000","last_clean":"2023-12-20T04:29:32.930657+0000","last_became_active":"2023-12-20T04:29:30.222618+0000","last_became_peered":"2023-12-20T04:29:30.222618+0000","last_unstale":"2023-12-20T04:29:32.930657+0000","last_undegraded":"2023-12-20T04:29:32.930657+0000","last_fullsized":"2023-12-20T04:29:32.930657+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:56:52.764129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.5","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.930429+0000","last_change":"2023-12-20T04:29:30.636221+0000","last_active":"2023-12-20T04:29:32.930429+0000","last_peered":"2023-12-20T04:29:32.930429+0000","last_clean":"2023-12-20T04:29:32.930429+0000","last_became_active":"2023-12-20T04:29:30.634994+0000","last_became_peered":"2023-12-20T04:29:30.634994+0000","last_unstale":"2023-12-20T04:29:32.930429+0000","last_undegraded":"2023-12-20T04:29:32.930429+0000","last_fullsized":"2023-12-20T04:29:32.930429+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:45.142508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.627288+0000","last_change":"2023-12-20T04:29:30.221192+0000","last_active":"2023-12-20T04:29:30.627288+0000","last_peered":"2023-12-20T04:29:30.627288+0000","last_clean":"2023-12-20T04:29:30.627288+0000","last_became_active":"2023-12-20T04:29:30.220915+0000","last_became_peered":"2023-12-20T04:29:30.220915+0000","last_unstale":"2023-12-20T04:29:30.627288+0000","last_undegraded":"2023-12-20T04:29:30.627288+0000","last_fullsized":"2023-12-20T04:29:30.627288+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:19:07.467835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.2","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.189527+0000","last_change":"2023-12-20T04:29:30.647250+0000","last_active":"2023-12-20T04:29:32.189527+0000","last_peered":"2023-12-20T04:29:32.189527+0000","last_clean":"2023-12-20T04:29:32.189527+0000","last_became_active":"2023-12-20T04:29:30.646942+0000","last_became_peered":"2023-12-20T04:29:30.646942+0000","last_unstale":"2023-12-20T04:29:32.189527+0000","last_undegraded":"2023-12-20T04:29:32.189527+0000","last_fullsized":"2023-12-20T04:29:32.189527+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:30.023462+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.622694+0000","last_change":"2023-12-20T04:29:30.226803+0000","last_active":"2023-12-20T04:29:30.622694+0000","last_peered":"2023-12-20T04:29:30.622694+0000","last_clean":"2023-12-20T04:29:30.622694+0000","last_became_active":"2023-12-20T04:29:30.226262+0000","last_became_peered":"2023-12-20T04:29:30.226262+0000","last_unstale":"2023-12-20T04:29:30.622694+0000","last_undegraded":"2023-12-20T04:29:30.622694+0000","last_fullsized":"2023-12-20T04:29:30.622694+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"5.3","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:32.930362+0000","last_change":"2023-12-20T04:29:30.635889+0000","last_active":"2023-12-20T04:29:32.930362+0000","last_peered":"2023-12-20T04:29:32.930362+0000","last_clean":"2023-12-20T04:29:32.930362+0000","last_became_active":"2023-12-20T04:29:30.635589+0000","last_became_peered":"2023-12-20T04:29:30.635589+0000","last_unstale":"2023-12-20T04:29:32.930362+0000","last_undegraded":"2023-12-20T04:29:32.930362+0000","last_fullsized":"2023-12-20T04:29:32.930362+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:00:35.598112+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.629242+0000","last_change":"2023-12-20T04:29:30.220925+0000","last_active":"2023-12-20T04:29:30.629242+0000","last_peered":"2023-12-20T04:29:30.629242+0000","last_clean":"2023-12-20T04:29:30.629242+0000","last_became_active":"2023-12-20T04:29:30.220649+0000","last_became_peered":"2023-12-20T04:29:30.220649+0000","last_unstale":"2023-12-20T04:29:30.629242+0000","last_undegraded":"2023-12-20T04:29:30.629242+0000","last_fullsized":"2023-12-20T04:29:30.629242+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:50:38.234160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"5.0","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.853919+0000","last_change":"2023-12-20T04:29:30.637566+0000","last_active":"2023-12-20T04:29:31.853919+0000","last_peered":"2023-12-20T04:29:31.853919+0000","last_clean":"2023-12-20T04:29:31.853919+0000","last_became_active":"2023-12-20T04:29:30.637316+0000","last_became_peered":"2023-12-20T04:29:30.637316+0000","last_unstale":"2023-12-20T04:29:31.853919+0000","last_undegraded":"2023-12-20T04:29:31.853919+0000","last_fullsized":"2023-12-20T04:29:31.853919+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:21:29.467415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.642716+0000","last_change":"2023-12-20T04:29:30.226051+0000","last_active":"2023-12-20T04:29:30.642716+0000","last_peered":"2023-12-20T04:29:30.642716+0000","last_clean":"2023-12-20T04:29:30.642716+0000","last_became_active":"2023-12-20T04:29:30.225759+0000","last_became_peered":"2023-12-20T04:29:30.225759+0000","last_unstale":"2023-12-20T04:29:30.642716+0000","last_undegraded":"2023-12-20T04:29:30.642716+0000","last_fullsized":"2023-12-20T04:29:30.642716+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:00:05.822869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"5.1","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625469+0000","last_change":"2023-12-20T04:29:30.649586+0000","last_active":"2023-12-20T04:29:31.625469+0000","last_peered":"2023-12-20T04:29:31.625469+0000","last_clean":"2023-12-20T04:29:31.625469+0000","last_became_active":"2023-12-20T04:29:30.649187+0000","last_became_peered":"2023-12-20T04:29:30.649187+0000","last_unstale":"2023-12-20T04:29:31.625469+0000","last_undegraded":"2023-12-20T04:29:31.625469+0000","last_fullsized":"2023-12-20T04:29:31.625469+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:42:49.715777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.615563+0000","last_change":"2023-12-20T04:29:30.227077+0000","last_active":"2023-12-20T04:29:30.615563+0000","last_peered":"2023-12-20T04:29:30.615563+0000","last_clean":"2023-12-20T04:29:30.615563+0000","last_became_active":"2023-12-20T04:29:30.226520+0000","last_became_peered":"2023-12-20T04:29:30.226520+0000","last_unstale":"2023-12-20T04:29:30.615563+0000","last_undegraded":"2023-12-20T04:29:30.615563+0000","last_fullsized":"2023-12-20T04:29:30.615563+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:29.961522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"5.4","version":"0'0","reported_seq":10,"reported_epoch":35,"state":"active+clean","last_fresh":"2023-12-20T04:29:31.625847+0000","last_change":"2023-12-20T04:29:30.648340+0000","last_active":"2023-12-20T04:29:31.625847+0000","last_peered":"2023-12-20T04:29:31.625847+0000","last_clean":"2023-12-20T04:29:31.625847+0000","last_became_active":"2023-12-20T04:29:30.647992+0000","last_became_peered":"2023-12-20T04:29:30.647992+0000","last_unstale":"2023-12-20T04:29:31.625847+0000","last_undegraded":"2023-12-20T04:29:31.625847+0000","last_fullsized":"2023-12-20T04:29:31.625847+0000","mapping_epoch":33,"log_start":"0'0","ondisk_log_start":"0'0","created":33,"last_epoch_clean":34,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:30.195147+0000","last_clean_scrub_stamp":"2023-12-20T04:29:30.195147+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:07.590492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":11,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.626176+0000","last_change":"2023-12-20T04:29:30.233314+0000","last_active":"2023-12-20T04:29:30.626176+0000","last_peered":"2023-12-20T04:29:30.626176+0000","last_clean":"2023-12-20T04:29:30.626176+0000","last_became_active":"2023-12-20T04:29:30.233037+0000","last_became_peered":"2023-12-20T04:29:30.233037+0000","last_unstale":"2023-12-20T04:29:30.626176+0000","last_undegraded":"2023-12-20T04:29:30.626176+0000","last_fullsized":"2023-12-20T04:29:30.626176+0000","mapping_epoch":32,"log_start":"0'0","ondisk_log_start":"0'0","created":32,"last_epoch_clean":33,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:29:29.192336+0000","last_clean_scrub_stamp":"2023-12-20T04:29:29.192336+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:55:41.484211+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"17'117","reported_seq":174,"reported_epoch":34,"state":"active+clean","last_fresh":"2023-12-20T04:29:30.637861+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:29:30.637861+0000","last_peered":"2023-12-20T04:29:30.637861+0000","last_clean":"2023-12-20T04:29:30.637861+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:29:30.637861+0000","last_undegraded":"2023-12-20T04:29:30.637861+0000","last_fullsized":"2023-12-20T04:29:30.637861+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":117,"log_dups_size":0,"ondisk_log_size":117,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":5,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":116,"num_read_kb":100,"num_write":159,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":117,"ondisk_log_size":117,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834574864,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32020,"kb_used_data":972,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339820,"statfs":{"total":96636764160,"available":96603975680,"internally_reserved":0,"allocated":995328,"data_stored":747016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.624,"15min":0.624},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":0.90200000000000002,"5min":0.90200000000000002,"15min":0.90200000000000002},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.66300000000000003,"15min":0.66300000000000003},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.079,"5min":1.079,"15min":1.079},"last":0.57699999999999996}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64000000000000001,"5min":0.64000000000000001,"15min":0.64000000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":1.2190000000000001,"5min":1.2190000000000001,"15min":1.2190000000000001},"last":0.629},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.78600000000000003,"15min":0.78600000000000003},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":2.649,"5min":2.649,"15min":2.649},"last":0.85699999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.753,"15min":0.753},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.5960000000000001,"5min":1.5960000000000001,"15min":1.5960000000000001},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.70599999999999996,"15min":0.70599999999999996},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.135,"5min":1.135,"15min":1.135},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.641,"5min":1.641,"15min":1.641},"last":0.54900000000000004},{"interface":"front","average":{"1min":0.86899999999999999,"5min":0.86899999999999999,"15min":0.86899999999999999},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":2.5990000000000002,"5min":2.5990000000000002,"15min":2.5990000000000002},"last":0.68400000000000005}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":0.97899999999999998},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.71499999999999997,"15min":0.71499999999999997},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.746}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.71799999999999997},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.75600000000000001,"15min":0.75600000000000001},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0549999999999999,"5min":1.0549999999999999,"15min":1.0549999999999999},"last":0.76000000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.55700000000000005,"5min":0.55700000000000005,"15min":0.55700000000000005},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.98199999999999998},"last":0.79200000000000004}]}]},{"osd":6,"up_from":13,"seq":55834574864,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":157174,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.59599999999999997,"5min":0.59599999999999997,"15min":0.59599999999999997},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.32000000000000001},"max":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.60899999999999999,"15min":0.60899999999999999},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":0.88700000000000001},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63200000000000001,"15min":0.63200000000000001},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.629,"5min":0.629,"15min":0.629},"min":{"1min":0.36699999999999999,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"last":0.71599999999999997}]},{"osd":2,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68899999999999995,"15min":0.68899999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":0.90300000000000002},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.64600000000000002,"15min":0.64600000000000002},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.90800000000000003}]},{"osd":3,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.45600000000000002,"5min":0.45600000000000002,"15min":0.45600000000000002},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.71199999999999997,"15min":0.71199999999999997},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":0.91600000000000004,"5min":0.91600000000000004,"15min":0.91600000000000004},"last":0.80200000000000005}]},{"osd":4,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68200000000000005,"15min":0.68200000000000005},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.97099999999999997,"5min":0.97099999999999997,"15min":0.97099999999999997},"last":0.82499999999999996},{"interface":"front","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.56999999999999995,"5min":0.56999999999999995,"15min":0.56999999999999995},"max":{"1min":1.071,"5min":1.071,"15min":1.071},"last":0.84199999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74099999999999999,"15min":0.74099999999999999},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.96299999999999997,"5min":0.96299999999999997,"15min":0.96299999999999997},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.872}]},{"osd":7,"last update":"Wed Dec 20 04:29:26 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.2270000000000001,"5min":1.2270000000000001,"15min":1.2270000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.84399999999999997,"5min":0.84399999999999997,"15min":0.84399999999999997},"min":{"1min":0.67700000000000005,"5min":0.67700000000000005,"15min":0.67700000000000005},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.89100000000000001}]}]},{"osd":5,"up_from":13,"seq":55834574865,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31444,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340396,"statfs":{"total":96636764160,"available":96604565504,"internally_reserved":0,"allocated":405504,"data_stored":157230,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.56599999999999995},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.42799999999999999},"max":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.66600000000000004,"15min":0.66600000000000004},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.72799999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63500000000000001,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.89400000000000002,"5min":0.89400000000000002,"15min":0.89400000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.71599999999999997,"15min":0.71599999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.48099999999999998},"max":{"1min":1.1140000000000001,"5min":1.1140000000000001,"15min":1.1140000000000001},"last":0.752}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40600000000000003},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69299999999999995,"15min":0.69299999999999995},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":0.91500000000000004,"5min":0.91500000000000004,"15min":0.91500000000000004},"last":0.68300000000000005}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.47499999999999998,"5min":0.47499999999999998,"15min":0.47499999999999998},"max":{"1min":1.758,"5min":1.758,"15min":1.758},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":1.0049999999999999,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.64500000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.67100000000000004,"15min":0.67100000000000004},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.76100000000000001},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.70299999999999996,"15min":0.70299999999999996},"min":{"1min":0.5,"5min":0.5,"15min":0.5},"max":{"1min":0.94099999999999995,"5min":0.94099999999999995,"15min":0.94099999999999995},"last":0.622}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":1.0700000000000001,"5min":1.0700000000000001,"15min":1.0700000000000001},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.71899999999999997,"15min":0.71899999999999997},"min":{"1min":0.55300000000000005,"5min":0.55300000000000005,"15min":0.55300000000000005},"max":{"1min":0.90400000000000003,"5min":0.90400000000000003,"15min":0.90400000000000003},"last":0.54000000000000004}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.77500000000000002},"min":{"1min":0.54900000000000004,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.034,"5min":1.034,"15min":1.034},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.77200000000000002,"15min":0.77200000000000002},"min":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"max":{"1min":1.119,"5min":1.119,"15min":1.119},"last":0.71199999999999997}]}]},{"osd":4,"up_from":13,"seq":55834574866,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":27348,"kb_used_data":396,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94344492,"statfs":{"total":96636764160,"available":96608759808,"internally_reserved":0,"allocated":405504,"data_stored":156662,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.64400000000000002},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67900000000000005,"15min":0.67900000000000005},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.92500000000000004},"last":0.54500000000000004}]},{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.64800000000000002,"15min":0.64800000000000002},"min":{"1min":0.42299999999999999,"5min":0.42299999999999999,"15min":0.42299999999999999},"max":{"1min":0.81999999999999995,"5min":0.81999999999999995,"15min":0.81999999999999995},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.48299999999999998,"5min":0.48299999999999998,"15min":0.48299999999999998},"max":{"1min":0.99399999999999999,"5min":0.99399999999999999,"15min":0.99399999999999999},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.68100000000000005,"15min":0.68100000000000005},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.46100000000000002},"max":{"1min":0.97399999999999998,"5min":0.97399999999999998,"15min":0.97399999999999998},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.75,"15min":0.75},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.84799999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.70199999999999996,"15min":0.70199999999999996},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96499999999999997,"5min":0.96499999999999997,"15min":0.96499999999999997},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75800000000000001,"15min":0.75800000000000001},"min":{"1min":0.56000000000000005,"5min":0.56000000000000005,"15min":0.56000000000000005},"max":{"1min":0.92800000000000005,"5min":0.92800000000000005,"15min":0.92800000000000005},"last":0.82899999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.72199999999999998,"15min":0.72199999999999998},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":0.96099999999999997},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.73199999999999998,"15min":0.73199999999999998},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":0.98299999999999998,"5min":0.98299999999999998,"15min":0.98299999999999998},"last":0.65300000000000002}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77000000000000002,"5min":0.77000000000000002,"15min":0.77000000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.435},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.78900000000000003,"15min":0.78900000000000003},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.88700000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77600000000000002,"15min":0.77600000000000002},"min":{"1min":0.46400000000000002,"5min":0.46400000000000002,"15min":0.46400000000000002},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.0189999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":0.80800000000000005},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"last":0.90700000000000003}]}]},{"osd":3,"up_from":13,"seq":55834574865,"num_pgs":31,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":413696,"data_stored":158144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.747,"15min":0.747},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":4.3700000000000001,"5min":4.3700000000000001,"15min":4.3700000000000001},"last":0.309},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":3.355,"5min":3.355,"15min":3.355},"last":0.51400000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.66000000000000003,"5min":0.66000000000000003,"15min":0.66000000000000003},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":3.7160000000000002,"5min":3.7160000000000002,"15min":3.7160000000000002},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.68000000000000005,"15min":0.68000000000000005},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":3.3109999999999999,"5min":3.3109999999999999,"15min":3.3109999999999999},"last":0.46800000000000003}]},{"osd":2,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.76500000000000001,"15min":0.76500000000000001},"min":{"1min":0.36499999999999999,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":4.0110000000000001,"5min":4.0110000000000001,"15min":4.0110000000000001},"last":0.497},{"interface":"front","average":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":0.95799999999999996},"min":{"1min":0.316,"5min":0.316,"15min":0.316},"max":{"1min":6.1399999999999997,"5min":6.1399999999999997,"15min":6.1399999999999997},"last":0.42799999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.503,"5min":0.503,"15min":0.503},"max":{"1min":1.3089999999999999,"5min":1.3089999999999999,"15min":1.3089999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":1.0309999999999999,"5min":1.0309999999999999,"15min":1.0309999999999999},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":3.9510000000000001,"5min":3.9510000000000001,"15min":3.9510000000000001},"last":0.749}]},{"osd":5,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":1.0720000000000001,"5min":1.0720000000000001,"15min":1.0720000000000001},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":3.9830000000000001,"5min":3.9830000000000001,"15min":3.9830000000000001},"last":0.82099999999999995},{"interface":"front","average":{"1min":1.103,"5min":1.103,"15min":1.103},"min":{"1min":0.57899999999999996,"5min":0.57899999999999996,"15min":0.57899999999999996},"max":{"1min":4.6550000000000002,"5min":4.6550000000000002,"15min":4.6550000000000002},"last":0.85699999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.82999999999999996},"min":{"1min":0.57999999999999996,"5min":0.57999999999999996,"15min":0.57999999999999996},"max":{"1min":1.1950000000000001,"5min":1.1950000000000001,"15min":1.1950000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":1.034,"5min":1.034,"15min":1.034},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":3.8900000000000001,"5min":3.8900000000000001,"15min":3.8900000000000001},"last":0.88300000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:29:24 2023","interfaces":[{"interface":"back","average":{"1min":1.111,"5min":1.111,"15min":1.111},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":4.6970000000000001,"5min":4.6970000000000001,"15min":4.6970000000000001},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":0.86399999999999999},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.60299999999999998},"max":{"1min":1.1180000000000001,"5min":1.1180000000000001,"15min":1.1180000000000001},"last":0.77200000000000002}]}]},{"osd":2,"up_from":13,"seq":55834574864,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31440,"kb_used_data":392,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340400,"statfs":{"total":96636764160,"available":96604569600,"internally_reserved":0,"allocated":401408,"data_stored":156628,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.435,"5min":0.435,"15min":0.435},"min":{"1min":0.25,"5min":0.25,"15min":0.25},"max":{"1min":0.89600000000000002,"5min":0.89600000000000002,"15min":0.89600000000000002},"last":0.28799999999999998},{"interface":"front","average":{"1min":0.439,"5min":0.439,"15min":0.439},"min":{"1min":0.28799999999999998,"5min":0.28799999999999998,"15min":0.28799999999999998},"max":{"1min":0.66400000000000003,"5min":0.66400000000000003,"15min":0.66400000000000003},"last":0.35799999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"last":0.315},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.59699999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.34999999999999998,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.80700000000000005,"5min":0.80700000000000005,"15min":0.80700000000000005},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.59499999999999997,"5min":0.59499999999999997,"15min":0.59499999999999997},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.623}]},{"osd":4,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1399999999999999,"5min":1.1399999999999999,"15min":1.1399999999999999},"last":0.70099999999999996},{"interface":"front","average":{"1min":0.754,"5min":0.754,"15min":0.754},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.41399999999999998},"max":{"1min":1.2010000000000001,"5min":1.2010000000000001,"15min":1.2010000000000001},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.76200000000000001,"15min":0.76200000000000001},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.327,"5min":1.327,"15min":1.327},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.80400000000000005,"15min":0.80400000000000005},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.381,"5min":1.381,"15min":1.381},"last":0.72499999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.78700000000000003,"15min":0.78700000000000003},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.2809999999999999,"5min":1.2809999999999999,"15min":1.2809999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.78400000000000003,"15min":0.78400000000000003},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.73999999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.82499999999999996,"15min":0.82499999999999996},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3520000000000001,"5min":1.3520000000000001,"15min":1.3520000000000001},"last":0.88700000000000001},{"interface":"front","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.59099999999999997,"5min":0.59099999999999997,"15min":0.59099999999999997},"max":{"1min":1.401,"5min":1.401,"15min":1.401},"last":0.94499999999999995}]}]},{"osd":1,"up_from":13,"seq":55834574866,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31452,"kb_used_data":404,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340388,"statfs":{"total":96636764160,"available":96604557312,"internally_reserved":0,"allocated":413696,"data_stored":158144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"min":{"1min":0.26800000000000002,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.69999999999999996,"5min":0.69999999999999996,"15min":0.69999999999999996},"last":0.434},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.46000000000000002},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.69699999999999995,"5min":0.69699999999999995,"15min":0.69699999999999995},"last":0.30099999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"min":{"1min":0.32800000000000001,"5min":0.32800000000000001,"15min":0.32800000000000001},"max":{"1min":0.85399999999999998,"5min":0.85399999999999998,"15min":0.85399999999999998},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.91300000000000003,"5min":0.91300000000000003,"15min":0.91300000000000003},"last":0.36199999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.55400000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.48399999999999999},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.751,"5min":0.751,"15min":0.751},"last":0.41299999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.752,"15min":0.752},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.4419999999999999,"5min":1.4419999999999999,"15min":1.4419999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76100000000000001,"15min":0.76100000000000001},"min":{"1min":0.48899999999999999,"5min":0.48899999999999999,"15min":0.48899999999999999},"max":{"1min":1.3380000000000001,"5min":1.3380000000000001,"15min":1.3380000000000001},"last":0.55400000000000005}]},{"osd":5,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.71699999999999997,"15min":0.71699999999999997},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.0629999999999999,"5min":1.0629999999999999,"15min":1.0629999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77900000000000003,"15min":0.77900000000000003},"min":{"1min":0.56100000000000005,"5min":0.56100000000000005,"15min":0.56100000000000005},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.58399999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.76800000000000002,"15min":0.76800000000000002},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":1.415,"5min":1.415,"15min":1.415},"last":0.64300000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.77300000000000002,"15min":0.77300000000000002},"min":{"1min":0.50600000000000001,"5min":0.50600000000000001,"15min":0.50600000000000001},"max":{"1min":1.268,"5min":1.268,"15min":1.268},"last":0.59799999999999998}]},{"osd":7,"last update":"Wed Dec 20 04:29:27 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82899999999999996,"15min":0.82899999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.399,"5min":1.399,"15min":1.399},"last":0.628},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.57599999999999996},"max":{"1min":1.373,"5min":1.373,"15min":1.373},"last":0.65800000000000003}]}]},{"osd":0,"up_from":13,"seq":55834574865,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32016,"kb_used_data":968,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339824,"statfs":{"total":96636764160,"available":96603979776,"internally_reserved":0,"allocated":991232,"data_stored":746926,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"min":{"1min":0.26200000000000001,"5min":0.26200000000000001,"15min":0.26200000000000001},"max":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.58599999999999997},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.81000000000000005,"5min":0.81000000000000005,"15min":0.81000000000000005},"last":0.31900000000000001}]},{"osd":2,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.505,"5min":0.505,"15min":0.505},"min":{"1min":0.32900000000000001,"5min":0.32900000000000001,"15min":0.32900000000000001},"max":{"1min":0.69399999999999995,"5min":0.69399999999999995,"15min":0.69399999999999995},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.53700000000000003,"15min":0.53700000000000003},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.371}]},{"osd":3,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.47799999999999998},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.66800000000000004,"5min":0.66800000000000004,"15min":0.66800000000000004},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"min":{"1min":0.39600000000000002,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.42899999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.89500000000000002,"5min":0.89500000000000002,"15min":0.89500000000000002},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":3.7469999999999999,"5min":3.7469999999999999,"15min":3.7469999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.76000000000000001,"15min":0.76000000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":1.1279999999999999,"5min":1.1279999999999999,"15min":1.1279999999999999},"last":0.70699999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.81100000000000005,"15min":0.81100000000000005},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.877,"5min":1.877,"15min":1.877},"last":0.623},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.79600000000000004},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.61499999999999999},"max":{"1min":1.7450000000000001,"5min":1.7450000000000001,"15min":1.7450000000000001},"last":0.77100000000000002}]},{"osd":6,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.80000000000000004,"15min":0.80000000000000004},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":1.613,"5min":1.613,"15min":1.613},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.83299999999999996},"min":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.57699999999999996},"max":{"1min":1.492,"5min":1.492,"15min":1.492},"last":0.79000000000000004}]},{"osd":7,"last update":"Wed Dec 20 04:29:25 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.85699999999999998,"15min":0.85699999999999998},"min":{"1min":0.55600000000000005,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.6539999999999999,"5min":1.6539999999999999,"15min":1.6539999999999999},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.84599999999999997,"5min":0.84599999999999997,"15min":0.84599999999999997},"min":{"1min":0.59199999999999997,"5min":0.59199999999999997,"15min":0.59199999999999997},"max":{"1min":2.113,"5min":2.113,"15min":2.113},"last":0.69299999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":5,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:29:39.720 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T04:29:39.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T04:29:39.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb920812640 1 Processor -- start 2023-12-20T04:29:39.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb920812640 1 -- start start 2023-12-20T04:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb920812640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb91c1615d0 0x7fb91c06ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb920812640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0cd3f0 0x7fb91c0c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb920812640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb91c0cb710 con 0x7fb91c06ca40 2023-12-20T04:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb920812640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb91c0cddd0 con 0x7fb91c1615d0 2023-12-20T04:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb920812640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c06a7f0 con 0x7fb91c0cd3f0 2023-12-20T04:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91a59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb91c06ca40 0x7fb91c06ce20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34072/0 (socket says 172.21.15.32:34072) 2023-12-20T04:29:39.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91a59c640 1 -- 172.21.15.32:0/1933353378 learned_addr learned my addr 172.21.15.32:0/1933353378 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:39.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb919d9b640 1 --2- 172.21.15.32:0/1933353378 >> v2:172.21.15.39:3300/0 conn(0x7fb91c1615d0 0x7fb91c06ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91ad9d640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0cd3f0 0x7fb91c0c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2405342856 0 0) 0x7fb91c0cb710 con 0x7fb91c06ca40 2023-12-20T04:29:39.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb904003530 con 0x7fb91c06ca40 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1866771637 0 0) 0x7fb904003530 con 0x7fb91c06ca40 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0cd3f0 msgr2=0x7fb91c0c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0cd3f0 0x7fb91c0c9800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 >> v2:172.21.15.39:3300/0 conn(0x7fb91c1615d0 msgr2=0x7fb91c06ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 --2- 172.21.15.32:0/1933353378 >> v2:172.21.15.39:3300/0 conn(0x7fb91c1615d0 0x7fb91c06ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c069ec0 con 0x7fb91c06ca40 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fb90c001680 con 0x7fb91c06ca40 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 >> v1:172.21.15.32:6789/0 conn(0x7fb91c06ca40 legacy=0x7fb91c06ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0cd3f0 0x7fb904003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb904003f00 0x7fb9040047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 0x7fb904005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0cddd0 con 0x7fb904004d10 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c06a7f0 con 0x7fb904003f00 2023-12-20T04:29:39.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90400b9c0 con 0x7fb91c0cd3f0 2023-12-20T04:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91ad9d640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 0x7fb904005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.898+0000 7fb91a59c640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0cd3f0 0x7fb904003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.902+0000 7fb919d9b640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb904003f00 0x7fb9040047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.902+0000 7fb91ad9d640 1 -- 172.21.15.32:0/1933353378 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0cd3f0 msgr2=0x7fb904003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.902+0000 7fb91ad9d640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0cd3f0 0x7fb904003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.902+0000 7fb91ad9d640 1 -- 172.21.15.32:0/1933353378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb904003f00 msgr2=0x7fb9040047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.902+0000 7fb91ad9d640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb904003f00 0x7fb9040047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.902+0000 7fb91ad9d640 1 -- 172.21.15.32:0/1933353378 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb904003530 con 0x7fb904004d10 2023-12-20T04:29:39.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.902+0000 7fb91ad9d640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 0x7fb904005190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb914009fc0 tx=0x7fb91401cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdafe497d6d5b24b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.902+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9140a5c70 con 0x7fb904004d10 2023-12-20T04:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.902+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb91400f040 con 0x7fb904004d10 2023-12-20T04:29:39.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.902+0000 7fb91959a640 1 -- 172.21.15.32:0/1933353378 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9140ad440 con 0x7fb904004d10 2023-12-20T04:29:39.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 -- 172.21.15.32:0/1933353378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 msgr2=0x7fb904005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:39.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 --2- 172.21.15.32:0/1933353378 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 0x7fb904005190 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb914009fc0 tx=0x7fb91401cc60 comp rx=0 tx=0).stop 2023-12-20T04:29:39.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb91ad9d640 1 -- 172.21.15.32:0/1933353378 reap_dead start 2023-12-20T04:29:39.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 -- 172.21.15.32:0/1933353378 shutdown_connections 2023-12-20T04:29:39.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 -- 172.21.15.32:0/1933353378 >> 172.21.15.32:0/1933353378 conn(0x7fb91c0cf8e0 msgr2=0x7fb91c0cbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:39.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 -- 172.21.15.32:0/1933353378 shutdown_connections 2023-12-20T04:29:39.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 -- 172.21.15.32:0/1933353378 wait complete. 2023-12-20T04:29:39.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 Processor -- start 2023-12-20T04:29:39.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 -- start start 2023-12-20T04:29:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 0x7fb91c15a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0b3a10 0x7fb91c0c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb91c0c4f90 0x7fb91c15c090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0c5550 con 0x7fb904004d10 2023-12-20T04:29:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0b2c50 con 0x7fb91c0c4f90 2023-12-20T04:29:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb920812640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb91c0b2f30 con 0x7fb91c0b3a10 2023-12-20T04:29:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb91a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 0x7fb91c15a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb91a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 0x7fb91c15a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41118/0 (socket says 172.21.15.32:41118) 2023-12-20T04:29:39.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb91a59c640 1 -- 172.21.15.32:0/1121139709 learned_addr learned my addr 172.21.15.32:0/1121139709 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb91a59c640 1 -- 172.21.15.32:0/1121139709 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0b3a10 msgr2=0x7fb91c0c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb91a59c640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0b3a10 0x7fb91c0c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb91a59c640 1 -- 172.21.15.32:0/1121139709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb91c0c4f90 msgr2=0x7fb91c15c090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb91a59c640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb91c0c4f90 0x7fb91c15c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.906+0000 7fb91a59c640 1 -- 172.21.15.32:0/1121139709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb91c0b3210 con 0x7fb904004d10 2023-12-20T04:29:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb91a59c640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 0x7fb91c15a3d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb90c007310 tx=0x7fb90c007710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb90affd640 1 -- 172.21.15.32:0/1121139709 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c00fa00 con 0x7fb904004d10 2023-12-20T04:29:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb90affd640 1 -- 172.21.15.32:0/1121139709 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c00fba0 con 0x7fb904004d10 2023-12-20T04:29:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb920812640 1 -- 172.21.15.32:0/1121139709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb91c0b34f0 con 0x7fb904004d10 2023-12-20T04:29:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb90affd640 1 -- 172.21.15.32:0/1121139709 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0133f0 con 0x7fb904004d10 2023-12-20T04:29:39.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb920812640 1 -- 172.21.15.32:0/1121139709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91c15a910 con 0x7fb904004d10 2023-12-20T04:29:39.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb920812640 1 -- 172.21.15.32:0/1121139709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91c0b3710 con 0x7fb904004d10 2023-12-20T04:29:39.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb90affd640 1 -- 172.21.15.32:0/1121139709 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb90c013550 con 0x7fb904004d10 2023-12-20T04:29:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb90affd640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8fc065a90 0x7fb8fc067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb90affd640 1 -- 172.21.15.32:0/1121139709 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb90c083440 con 0x7fb904004d10 2023-12-20T04:29:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb919d9b640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8fc065a90 0x7fb8fc067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:39.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.910+0000 7fb919d9b640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8fc065a90 0x7fb8fc067f50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb910006b20 tx=0x7fb910006ab0 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:39.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:39.914+0000 7fb90affd640 1 -- 172.21.15.32:0/1121139709 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb90c050df0 con 0x7fb904004d10 2023-12-20T04:29:40.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.126+0000 7fb920812640 1 -- 172.21.15.32:0/1121139709 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fb91c0c5690 con 0x7fb904004d10 2023-12-20T04:29:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb90affd640 1 -- 172.21.15.32:0/1121139709 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v13) v1 ==== 190+0+0 (secure 0 0 0) 0x7fb90c054410 con 0x7fb904004d10 2023-12-20T04:29:40.136 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T04:29:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 -- 172.21.15.32:0/1121139709 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8fc065a90 msgr2=0x7fb8fc067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8fc065a90 0x7fb8fc067f50 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7fb910006b20 tx=0x7fb910006ab0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 -- 172.21.15.32:0/1121139709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 msgr2=0x7fb91c15a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 0x7fb91c15a3d0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7fb90c007310 tx=0x7fb90c007710 comp rx=0 tx=0).stop 2023-12-20T04:29:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 -- 172.21.15.32:0/1121139709 shutdown_connections 2023-12-20T04:29:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8fc065a90 0x7fb8fc067f50 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb91c0c4f90 0x7fb91c15c090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb91c0b3a10 0x7fb91c0c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 --2- 172.21.15.32:0/1121139709 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb904004d10 0x7fb91c15a3d0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 -- 172.21.15.32:0/1121139709 >> 172.21.15.32:0/1121139709 conn(0x7fb91c0cf8e0 msgr2=0x7fb91c15b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 -- 172.21.15.32:0/1121139709 shutdown_connections 2023-12-20T04:29:40.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.134+0000 7fb920812640 1 -- 172.21.15.32:0/1121139709 wait complete. 2023-12-20T04:29:40.158 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:29:40.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb713d6d640 1 Processor -- start 2023-12-20T04:29:40.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb713d6d640 1 -- start start 2023-12-20T04:29:40.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb713d6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb70c0b0d00 0x7fb70c164a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:40.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb713d6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70c0711d0 con 0x7fb70c1599e0 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb713d6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb70c0c98c0 con 0x7fb70c0aece0 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb713d6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0c9ba0 con 0x7fb70c0b0d00 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb711b09640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb70c1599e0 0x7fb70c0c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34082/0 (socket says 172.21.15.32:34082) 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb711b09640 1 -- 172.21.15.32:0/3123317542 learned_addr learned my addr 172.21.15.32:0/3123317542 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb71230a640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb70c0b0d00 0x7fb70c164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 738815949 0 0) 0x7fb70c0711d0 con 0x7fb70c1599e0 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6f8003530 con 0x7fb70c1599e0 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 119621443 0 0) 0x7fb6f8003530 con 0x7fb70c1599e0 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb70c0b0d00 msgr2=0x7fb70c164a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb70c0b0d00 0x7fb70c164a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 >> v1:172.21.15.39:6789/0 conn(0x7fb70c0aece0 legacy=0x7fb70c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c0c9e80 con 0x7fb70c1599e0 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fb708001800 con 0x7fb70c1599e0 2023-12-20T04:29:40.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 >> v1:172.21.15.32:6789/0 conn(0x7fb70c1599e0 legacy=0x7fb70c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:40.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80039b0 0x7fb6f8003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:40.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f8004360 0x7fb6f8004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:40.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb70c0b0d00 0x7fb6f8005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:40.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0c9ba0 con 0x7fb6f80039b0 2023-12-20T04:29:40.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f800dc50 con 0x7fb6f8004360 2023-12-20T04:29:40.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f800de10 con 0x7fb70c0b0d00 2023-12-20T04:29:40.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.298+0000 7fb711b09640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80039b0 0x7fb6f8003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:40.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb711308640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f8004360 0x7fb6f8004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:40.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb71230a640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb70c0b0d00 0x7fb6f8005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:40.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb711b09640 1 -- 172.21.15.32:0/3123317542 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb70c0b0d00 msgr2=0x7fb6f8005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:40.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb711b09640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb70c0b0d00 0x7fb6f8005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb711b09640 1 -- 172.21.15.32:0/3123317542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f8004360 msgr2=0x7fb6f8004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:40.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb711b09640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6f8004360 0x7fb6f8004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb711b09640 1 -- 172.21.15.32:0/3123317542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f8003530 con 0x7fb6f80039b0 2023-12-20T04:29:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb711b09640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80039b0 0x7fb6f8003d90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c1c0 tx=0x7fb70801cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2a0708fdbfd501b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb708007270 con 0x7fb6f80039b0 2023-12-20T04:29:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7080073d0 con 0x7fb6f80039b0 2023-12-20T04:29:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb710b07640 1 -- 172.21.15.32:0/3123317542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7080a5810 con 0x7fb6f80039b0 2023-12-20T04:29:40.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb713d6d640 1 -- 172.21.15.32:0/3123317542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80039b0 msgr2=0x7fb6f8003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:40.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb713d6d640 1 --2- 172.21.15.32:0/3123317542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f80039b0 0x7fb6f8003d90 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7fb70800c1c0 tx=0x7fb70801cc40 comp rx=0 tx=0).stop 2023-12-20T04:29:40.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.302+0000 7fb71230a640 1 -- 172.21.15.32:0/3123317542 reap_dead start 2023-12-20T04:29:40.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 -- 172.21.15.32:0/3123317542 shutdown_connections 2023-12-20T04:29:40.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 -- 172.21.15.32:0/3123317542 >> 172.21.15.32:0/3123317542 conn(0x7fb70c06a320 msgr2=0x7fb70c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:40.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 -- 172.21.15.32:0/3123317542 shutdown_connections 2023-12-20T04:29:40.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 -- 172.21.15.32:0/3123317542 wait complete. 2023-12-20T04:29:40.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 Processor -- start 2023-12-20T04:29:40.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 -- start start 2023-12-20T04:29:40.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6f80039b0 0x7fb70c15a160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:40.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb70c071620 0x7fb70c0b1220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:40.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70c0b2f40 0x7fb70c13d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:40.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c14bc00 con 0x7fb70c0b2f40 2023-12-20T04:29:40.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c15a6a0 con 0x7fb70c071620 2023-12-20T04:29:40.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb713d6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb70c0b1a70 con 0x7fb6f80039b0 2023-12-20T04:29:40.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb71230a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70c0b2f40 0x7fb70c13d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:40.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb71230a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70c0b2f40 0x7fb70c13d5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41144/0 (socket says 172.21.15.32:41144) 2023-12-20T04:29:40.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb71230a640 1 -- 172.21.15.32:0/2853255611 learned_addr learned my addr 172.21.15.32:0/2853255611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:40.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb711b09640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6f80039b0 0x7fb70c15a160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:40.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb711308640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb70c071620 0x7fb70c0b1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:40.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb71230a640 1 -- 172.21.15.32:0/2853255611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6f80039b0 msgr2=0x7fb70c15a160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:40.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb71230a640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6f80039b0 0x7fb70c15a160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb71230a640 1 -- 172.21.15.32:0/2853255611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb70c071620 msgr2=0x7fb70c0b1220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:40.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb71230a640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb70c071620 0x7fb70c0b1220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb71230a640 1 -- 172.21.15.32:0/2853255611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb70c0b1d50 con 0x7fb70c0b2f40 2023-12-20T04:29:40.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb71230a640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70c0b2f40 0x7fb70c13d5c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0b4540 tx=0x7fb70400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:40.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.306+0000 7fb7027fc640 1 -- 172.21.15.32:0/2853255611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb70400e040 con 0x7fb70c0b2f40 2023-12-20T04:29:40.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.310+0000 7fb7027fc640 1 -- 172.21.15.32:0/2853255611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb70400f040 con 0x7fb70c0b2f40 2023-12-20T04:29:40.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.310+0000 7fb7027fc640 1 -- 172.21.15.32:0/2853255611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb704015670 con 0x7fb70c0b2f40 2023-12-20T04:29:40.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.310+0000 7fb713d6d640 1 -- 172.21.15.32:0/2853255611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb70c0b2030 con 0x7fb70c0b2f40 2023-12-20T04:29:40.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.310+0000 7fb713d6d640 1 -- 172.21.15.32:0/2853255611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb70c0b1760 con 0x7fb70c0b2f40 2023-12-20T04:29:40.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.310+0000 7fb713d6d640 1 -- 172.21.15.32:0/2853255611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb70c14c090 con 0x7fb70c0b2f40 2023-12-20T04:29:40.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.310+0000 7fb7027fc640 1 -- 172.21.15.32:0/2853255611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fb7040157d0 con 0x7fb70c0b2f40 2023-12-20T04:29:40.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.310+0000 7fb7027fc640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6f0065a90 0x7fb6f0067f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:40.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.310+0000 7fb7027fc640 1 -- 172.21.15.32:0/2853255611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb704083760 con 0x7fb70c0b2f40 2023-12-20T04:29:40.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.310+0000 7fb711b09640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6f0065a90 0x7fb6f0067f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:40.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.310+0000 7fb711b09640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6f0065a90 0x7fb6f0067f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb7080ab000 tx=0x7fb7080a7070 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:40.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.314+0000 7fb7027fc640 1 -- 172.21.15.32:0/2853255611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb70c14c090 con 0x7fb70c0b2f40 2023-12-20T04:29:40.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.506+0000 7fb713d6d640 1 -- 172.21.15.32:0/2853255611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb70c14c090 con 0x7fb70c0b2f40 2023-12-20T04:29:40.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.506+0000 7fb7027fc640 1 -- 172.21.15.32:0/2853255611 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4709 (secure 0 0 0) 0x7fb70c14c090 con 0x7fb70c0b2f40 2023-12-20T04:29:40.511 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:29:40.511 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2541430692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2541430692},{"type":"v1","addr":"172.21.15.39:6835","nonce":2541430692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1379533068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1379533068},{"type":"v1","addr":"172.21.15.32:6837","nonce":1379533068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4021567802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4021567802},{"type":"v1","addr":"172.21.15.39:6834","nonce":4021567802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:29:30.610189+0000","modified":"2023-12-20T04:29:33.661905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3391762708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3391762708},{"type":"v1","addr":"172.21.15.32:6836","nonce":3391762708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-20T04:29:40.511 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2023-12-20T04:29:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 -- 172.21.15.32:0/2853255611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6f0065a90 msgr2=0x7fb6f0067f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6f0065a90 0x7fb6f0067f50 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fb7080ab000 tx=0x7fb7080a7070 comp rx=0 tx=0).stop 2023-12-20T04:29:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 -- 172.21.15.32:0/2853255611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70c0b2f40 msgr2=0x7fb70c13d5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70c0b2f40 0x7fb70c13d5c0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fb70c0b4540 tx=0x7fb70400f580 comp rx=0 tx=0).stop 2023-12-20T04:29:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 -- 172.21.15.32:0/2853255611 shutdown_connections 2023-12-20T04:29:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6f0065a90 0x7fb6f0067f50 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70c0b2f40 0x7fb70c13d5c0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb70c071620 0x7fb70c0b1220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 --2- 172.21.15.32:0/2853255611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6f80039b0 0x7fb70c15a160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 -- 172.21.15.32:0/2853255611 >> 172.21.15.32:0/2853255611 conn(0x7fb70c06a320 msgr2=0x7fb70c14d630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 -- 172.21.15.32:0/2853255611 shutdown_connections 2023-12-20T04:29:40.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:40.510+0000 7fb713d6d640 1 -- 172.21.15.32:0/2853255611 wait complete. 2023-12-20T04:29:40.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 16, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T04:29:30.610189+0000', 'modified': '2023-12-20T04:29:33.661905+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5031}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5031': {'gid': 5031, 'name': 'a', 'rank': 0, 'incarnation': 12, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6836/3391762708', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 3391762708}, {'type': 'v1', 'addr': '172.21.15.32:6836', 'nonce': 3391762708}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [5], 'metadata_pool': 4, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T04:29:40.939 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T04:29:40.939 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-20T04:29:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.082+0000 7fc2e7d18640 1 Processor -- start 2023-12-20T04:29:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.082+0000 7fc2e7d18640 1 -- start start 2023-12-20T04:29:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.082+0000 7fc2e7d18640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2e006deb0 0x7fc2e0161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.082+0000 7fc2e7d18640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2e0070950 0x7fc2e00cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.082+0000 7fc2e7d18640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2e0071920 con 0x7fc2e015ce30 2023-12-20T04:29:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.082+0000 7fc2e7d18640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2e0070de0 con 0x7fc2e006deb0 2023-12-20T04:29:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.082+0000 7fc2e7d18640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e0071080 con 0x7fc2e0070950 2023-12-20T04:29:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.082+0000 7fc2e52b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2e006deb0 0x7fc2e0161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.082+0000 7fc2e52b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2e006deb0 0x7fc2e0161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41160/0 (socket says 172.21.15.32:41160) 2023-12-20T04:29:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.082+0000 7fc2e52b3640 1 -- 172.21.15.32:0/1058333867 learned_addr learned my addr 172.21.15.32:0/1058333867 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.086+0000 7fc2e62b5640 1 --2- 172.21.15.32:0/1058333867 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2e0070950 0x7fc2e00cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.086+0000 7fc2e52b3640 1 -- 172.21.15.32:0/1058333867 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2e0070950 msgr2=0x7fc2e00cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.086+0000 7fc2e52b3640 1 --2- 172.21.15.32:0/1058333867 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2e0070950 0x7fc2e00cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.086+0000 7fc2e52b3640 1 -- 172.21.15.32:0/1058333867 >> v1:172.21.15.39:6789/0 conn(0x7fc2e015ce30 legacy=0x7fc2e006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.086+0000 7fc2e52b3640 1 -- 172.21.15.32:0/1058333867 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2e0071360 con 0x7fc2e006deb0 2023-12-20T04:29:41.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.086+0000 7fc2e52b3640 1 --2- 172.21.15.32:0/1058333867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2e006deb0 0x7fc2e0161560 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009870 tx=0x7fc2d001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a8854dd8ed68f74 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:41.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.086+0000 7fc2e4ab2640 1 -- 172.21.15.32:0/1058333867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00a7070 con 0x7fc2e006deb0 2023-12-20T04:29:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.086+0000 7fc2e4ab2640 1 -- 172.21.15.32:0/1058333867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d00ab430 con 0x7fc2e006deb0 2023-12-20T04:29:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.086+0000 7fc2e4ab2640 1 -- 172.21.15.32:0/1058333867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d00ab7c0 con 0x7fc2e006deb0 2023-12-20T04:29:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.086+0000 7fc2e7d18640 1 -- 172.21.15.32:0/1058333867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2e006deb0 msgr2=0x7fc2e0161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.086+0000 7fc2e7d18640 1 --2- 172.21.15.32:0/1058333867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2e006deb0 0x7fc2e0161560 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fc2d0009870 tx=0x7fc2d001c540 comp rx=0 tx=0).stop 2023-12-20T04:29:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 -- 172.21.15.32:0/1058333867 shutdown_connections 2023-12-20T04:29:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 --2- 172.21.15.32:0/1058333867 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2e0070950 0x7fc2e00cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 --2- 172.21.15.32:0/1058333867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2e006deb0 0x7fc2e0161560 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 -- 172.21.15.32:0/1058333867 >> 172.21.15.32:0/1058333867 conn(0x7fc2e00cba50 msgr2=0x7fc2e00cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 -- 172.21.15.32:0/1058333867 shutdown_connections 2023-12-20T04:29:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 -- 172.21.15.32:0/1058333867 wait complete. 2023-12-20T04:29:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 Processor -- start 2023-12-20T04:29:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 -- start start 2023-12-20T04:29:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2e006deb0 0x7fc2e0154fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e0070950 0x7fc2e00c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2e015ce30 0x7fc2e00b0c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e015eb20 con 0x7fc2e015ce30 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e015da80 con 0x7fc2e0070950 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e7d18640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2e015dd60 con 0x7fc2e006deb0 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e62b5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2e015ce30 0x7fc2e00b0c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e62b5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2e015ce30 0x7fc2e00b0c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41170/0 (socket says 172.21.15.32:41170) 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e62b5640 1 -- 172.21.15.32:0/797498447 learned_addr learned my addr 172.21.15.32:0/797498447 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e52b3640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e0070950 0x7fc2e00c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2e006deb0 0x7fc2e0154fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e62b5640 1 -- 172.21.15.32:0/797498447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2e006deb0 msgr2=0x7fc2e0154fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e62b5640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2e006deb0 0x7fc2e0154fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e62b5640 1 -- 172.21.15.32:0/797498447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e0070950 msgr2=0x7fc2e00c89c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e62b5640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e0070950 0x7fc2e00c89c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e62b5640 1 -- 172.21.15.32:0/797498447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2d00094d0 con 0x7fc2e015ce30 2023-12-20T04:29:41.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2e006deb0 0x7fc2e0154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.090+0000 7fc2e62b5640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2e015ce30 0x7fc2e00b0c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8007e40 tx=0x7fc2d800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.094+0000 7fc2d67fc640 1 -- 172.21.15.32:0/797498447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8014620 con 0x7fc2e015ce30 2023-12-20T04:29:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.094+0000 7fc2d67fc640 1 -- 172.21.15.32:0/797498447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2d800ee90 con 0x7fc2e015ce30 2023-12-20T04:29:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.094+0000 7fc2d67fc640 1 -- 172.21.15.32:0/797498447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d80149e0 con 0x7fc2e015ce30 2023-12-20T04:29:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.094+0000 7fc2e7d18640 1 -- 172.21.15.32:0/797498447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2e015e040 con 0x7fc2e015ce30 2023-12-20T04:29:41.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.094+0000 7fc2e7d18640 1 -- 172.21.15.32:0/797498447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2e015e600 con 0x7fc2e015ce30 2023-12-20T04:29:41.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.094+0000 7fc2d67fc640 1 -- 172.21.15.32:0/797498447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fc2d8014b40 con 0x7fc2e015ce30 2023-12-20T04:29:41.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.094+0000 7fc2d67fc640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2c4065a60 0x7fc2c4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.094+0000 7fc2d67fc640 1 -- 172.21.15.32:0/797498447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2d8083b50 con 0x7fc2e015ce30 2023-12-20T04:29:41.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.094+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2c4065a60 0x7fc2c4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.094+0000 7fc2eb085000 1 Processor -- start 2023-12-20T04:29:41.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.094+0000 7fc2eb085000 1 -- start start 2023-12-20T04:29:41.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2c4065a60 0x7fc2c4067f20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc000f70 tx=0x7fc2dc009040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:41.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e070 0x558702f3e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e990 0x558702ef3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702ef41c0 0x558702ef45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558702d17c90 con 0x558702f3e070 2023-12-20T04:29:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558702d49220 con 0x558702f3e990 2023-12-20T04:29:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x558702d467f0 con 0x558702ef41c0 2023-12-20T04:29:41.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e5ab4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e070 0x558702f3e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e5ab4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e070 0x558702f3e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41176/0 (socket says 172.21.15.32:41176) 2023-12-20T04:29:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/1302622271 learned_addr learned my addr 172.21.15.32:0/1302622271 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:41.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e52b3640 1 --2- 172.21.15.32:0/1302622271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e990 0x558702ef3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e62b5640 1 --2- 172.21.15.32:0/1302622271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702ef41c0 0x558702ef45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/1302622271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702ef41c0 msgr2=0x558702ef45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/1302622271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702ef41c0 0x558702ef45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/1302622271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e990 msgr2=0x558702ef3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/1302622271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e990 0x558702ef3c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/1302622271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558702a3e890 con 0x558702f3e070 2023-12-20T04:29:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/1302622271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e070 0x558702f3e450 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc001eb0 tx=0x7fc2dc011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78cf0cde5f7157fb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2c37fe640 1 -- 172.21.15.32:0/1302622271 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc003600 con 0x558702f3e070 2023-12-20T04:29:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2c37fe640 1 -- 172.21.15.32:0/1302622271 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc003760 con 0x558702f3e070 2023-12-20T04:29:41.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2c37fe640 1 -- 172.21.15.32:0/1302622271 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc003ad0 con 0x558702f3e070 2023-12-20T04:29:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 -- 172.21.15.32:0/1302622271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e070 msgr2=0x558702f3e450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 --2- 172.21.15.32:0/1302622271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e070 0x558702f3e450 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc001eb0 tx=0x7fc2dc011000 comp rx=0 tx=0).stop 2023-12-20T04:29:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 -- 172.21.15.32:0/1302622271 shutdown_connections 2023-12-20T04:29:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 --2- 172.21.15.32:0/1302622271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702ef41c0 0x558702ef45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 --2- 172.21.15.32:0/1302622271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e990 0x558702ef3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 --2- 172.21.15.32:0/1302622271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e070 0x558702f3e450 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 -- 172.21.15.32:0/1302622271 >> 172.21.15.32:0/1302622271 conn(0x558702efa950 msgr2=0x558702ef2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:41.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 -- 172.21.15.32:0/1302622271 shutdown_connections 2023-12-20T04:29:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 -- 172.21.15.32:0/1302622271 wait complete. 2023-12-20T04:29:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 -- start start 2023-12-20T04:29:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.098+0000 7fc2eb085000 1 Processor -- start 2023-12-20T04:29:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702ef41c0 0x558702ef9710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e070 0x558702f00080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e990 0x558702f005c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558702d420e0 con 0x558702f3e990 2023-12-20T04:29:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558702ce1310 con 0x558702ef41c0 2023-12-20T04:29:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x558702cff770 con 0x558702f3e070 2023-12-20T04:29:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 10 client.0 _ll_register_callbacks cb 0x558702edfb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:29:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e62b5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e990 0x558702f005c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e62b5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e990 0x558702f005c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41188/0 (socket says 172.21.15.32:41188) 2023-12-20T04:29:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e62b5640 1 -- 172.21.15.32:0/3147614777 learned_addr learned my addr 172.21.15.32:0/3147614777 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:41.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702ef41c0 0x558702ef9710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e52b3640 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e070 0x558702f00080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e62b5640 1 -- 172.21.15.32:0/3147614777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e070 msgr2=0x558702f00080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e62b5640 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e070 0x558702f00080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e62b5640 1 -- 172.21.15.32:0/3147614777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702ef41c0 msgr2=0x558702ef9710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e62b5640 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702ef41c0 0x558702ef9710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e62b5640 1 -- 172.21.15.32:0/3147614777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2dc00f040 con 0x558702f3e990 2023-12-20T04:29:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e62b5640 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e990 0x558702f005c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8081fe0 tx=0x7fc2d80551f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73a0a0309e92697 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:29:41.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/3147614777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2d8051840 con 0x558702f3e990 2023-12-20T04:29:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/3147614777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2d80519a0 con 0x558702f3e990 2023-12-20T04:29:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 1 -- 172.21.15.32:0/3147614777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558702d11c10 con 0x558702f3e990 2023-12-20T04:29:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/3147614777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2d8054620 con 0x558702f3e990 2023-12-20T04:29:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 10 client.5217 fetch_fsmap learned FSMap version 16 2023-12-20T04:29:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 1 -- 172.21.15.32:0/3147614777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x558702d07260 con 0x558702f3e990 2023-12-20T04:29:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/3147614777 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558702d11c10 con 0x558702f3e990 2023-12-20T04:29:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/3147614777 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc2d8005d20 con 0x558702f3e990 2023-12-20T04:29:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 10 client.5217 fetch_fsmap finished waiting for FSMap version 16 2023-12-20T04:29:41.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 10 client.5217 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T04:29:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 20 client.5217 populate_metadata read hostname 'smithi032' 2023-12-20T04:29:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.32:6834/3391762708,v1:172.21.15.32:6836/3391762708] conn(0x558702f458d0 0x558702f45cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 4 client.5217 mds_command: new command op to 5031 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T04:29:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2eb085000 1 -- 172.21.15.32:0/3147614777 --> [v2:172.21.15.32:6834/3391762708,v1:172.21.15.32:6836/3391762708] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558702d40a30 con 0x558702f458d0 2023-12-20T04:29:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.32:6834/3391762708,v1:172.21.15.32:6836/3391762708] conn(0x558702f458d0 0x558702f45cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.32:6834/3391762708,v1:172.21.15.32:6836/3391762708] conn(0x558702f458d0 0x558702f45cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=76736cc7ce098b8f server_cookie=24dfcb9c2de25837 in_seq=0 out_seq=0 2023-12-20T04:29:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.102+0000 7fc2c0ff9640 10 client.5217 ms_handle_connect on v2:172.21.15.32:6834/3391762708 2023-12-20T04:29:41.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/3147614777 <== mds.0 v2:172.21.15.32:6834/3391762708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558702d40a30 con 0x558702f458d0 2023-12-20T04:29:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2c0ff9640 10 client.5217 handle_command_reply: tid=0 2023-12-20T04:29:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 1 client.5217 shutdown 2023-12-20T04:29:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 1 -- 172.21.15.32:0/3147614777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e990 msgr2=0x558702f005c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e990 0x558702f005c0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8081fe0 tx=0x7fc2d80551f0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 1 -- 172.21.15.32:0/3147614777 shutdown_connections 2023-12-20T04:29:41.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702f3e990 0x558702f005c0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e070 0x558702f00080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.32:6834/3391762708,v1:172.21.15.32:6836/3391762708] conn(0x558702f458d0 0x558702f45cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 1 --2- 172.21.15.32:0/3147614777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702ef41c0 0x558702ef9710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 1 -- 172.21.15.32:0/3147614777 >> 172.21.15.32:0/3147614777 conn(0x558702efa950 msgr2=0x558702ef2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2c0ff9640 0 client.5217 ms_handle_reset on v2:172.21.15.32:6834/3391762708 2023-12-20T04:29:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 1 -- 172.21.15.32:0/3147614777 shutdown_connections 2023-12-20T04:29:41.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 1 -- 172.21.15.32:0/3147614777 wait complete. 2023-12-20T04:29:41.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.106+0000 7fc2eb085000 20 client.5217 trim_cache size 0 max 16384 2023-12-20T04:29:41.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 Processor -- start 2023-12-20T04:29:41.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 -- start start 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f962f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e070 0x558702f96830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e990 0x558702f96d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5587029ff940 con 0x558702ef41c0 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558702cea860 con 0x558702f3e990 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x558702eeb9d0 con 0x558702f3e070 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2e5ab4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f962f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2e5ab4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f962f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41192/0 (socket says 172.21.15.32:41192) 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/1108929866 learned_addr learned my addr 172.21.15.32:0/1108929866 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2e52b3640 1 --2- 172.21.15.32:0/1108929866 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e070 0x558702f96830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2e62b5640 1 --2- 172.21.15.32:0/1108929866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e990 0x558702f96d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/1108929866 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e070 msgr2=0x558702f96830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/1108929866 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e070 0x558702f96830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/1108929866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e990 msgr2=0x558702f96d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:41.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/1108929866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e990 0x558702f96d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/1108929866 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558702ced3b0 con 0x558702ef41c0 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/1108929866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f962f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc014000 tx=0x7fc2dc017070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=218006de32863bd5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2c3fff640 1 -- 172.21.15.32:0/1108929866 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc028070 con 0x558702ef41c0 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2c3fff640 1 -- 172.21.15.32:0/1108929866 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2dc019040 con 0x558702ef41c0 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2c3fff640 1 -- 172.21.15.32:0/1108929866 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc02f800 con 0x558702ef41c0 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 -- 172.21.15.32:0/1108929866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 msgr2=0x558702f962f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 --2- 172.21.15.32:0/1108929866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f962f0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc014000 tx=0x7fc2dc017070 comp rx=0 tx=0).stop 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 -- 172.21.15.32:0/1108929866 shutdown_connections 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 --2- 172.21.15.32:0/1108929866 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e990 0x558702f96d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 --2- 172.21.15.32:0/1108929866 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e070 0x558702f96830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 --2- 172.21.15.32:0/1108929866 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f962f0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 -- 172.21.15.32:0/1108929866 >> 172.21.15.32:0/1108929866 conn(0x558702efa950 msgr2=0x558702efff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 -- 172.21.15.32:0/1108929866 shutdown_connections 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 -- 172.21.15.32:0/1108929866 wait complete. 2023-12-20T04:29:41.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 -- start start 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.122+0000 7fc2eb085000 1 Processor -- start 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f7f360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e070 0x558702f7f8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e990 0x558702f7fde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558702d3f4e0 con 0x558702ef41c0 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558702d43670 con 0x558702f3e070 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x558702d85ce0 con 0x558702f3e990 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 10 client.0 _ll_register_callbacks cb 0x558702cf0640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e5ab4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f7f360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e5ab4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f7f360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41198/0 (socket says 172.21.15.32:41198) 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/2436474403 learned_addr learned my addr 172.21.15.32:0/2436474403 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e62b5640 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e990 0x558702f7fde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e52b3640 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e070 0x558702f7f8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/2436474403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e990 msgr2=0x558702f7fde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:41.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e990 0x558702f7fde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/2436474403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e070 msgr2=0x558702f7f8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e070 0x558702f7f8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e5ab4640 1 -- 172.21.15.32:0/2436474403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x558702a3e890 con 0x558702ef41c0 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e5ab4640 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f7f360 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00e530 tx=0x7fc2dc000ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a37d9e848a7054b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2c0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/2436474403 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2dc028070 con 0x558702ef41c0 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/2436474403 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc2dc02f3b0 con 0x558702ef41c0 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 -- 172.21.15.32:0/2436474403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558702ef8ba0 con 0x558702ef41c0 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/2436474403 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=16) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc2dc02fa10 con 0x558702ef41c0 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 10 client.5223 fetch_fsmap learned FSMap version 16 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 -- 172.21.15.32:0/2436474403 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=16,monmap=2+}) v3 -- 0x558702a3ecc0 con 0x558702ef41c0 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/2436474403 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558702ef8ba0 con 0x558702ef41c0 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/2436474403 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 16) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc2dc036070 con 0x558702ef41c0 2023-12-20T04:29:41.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 10 client.5223 fetch_fsmap finished waiting for FSMap version 16 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 10 client.5223 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 20 client.5223 populate_metadata read hostname 'smithi032' 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.32:6834/3391762708,v1:172.21.15.32:6836/3391762708] conn(0x558702f80320 0x558702fa3590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 4 client.5223 mds_command: new command op to 5031 tid=0[{"prefix": "status"}] 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 -- 172.21.15.32:0/2436474403 --> [v2:172.21.15.32:6834/3391762708,v1:172.21.15.32:6836/3391762708] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558702d07e00 con 0x558702f80320 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e52b3640 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.32:6834/3391762708,v1:172.21.15.32:6836/3391762708] conn(0x558702f80320 0x558702fa3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2e52b3640 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.32:6834/3391762708,v1:172.21.15.32:6836/3391762708] conn(0x558702f80320 0x558702fa3590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f64bb2a46e4f9fd3 server_cookie=b4e4d769646407bd in_seq=0 out_seq=0 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2c0ff9640 10 client.5223 ms_handle_connect on v2:172.21.15.32:6834/3391762708 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2c0ff9640 1 -- 172.21.15.32:0/2436474403 <== mds.0 v2:172.21.15.32:6834/3391762708 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x558702d07e00 con 0x558702f80320 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2c0ff9640 10 client.5223 handle_command_reply: tid=0 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 client.5223 shutdown 2023-12-20T04:29:41.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 -- 172.21.15.32:0/2436474403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 msgr2=0x558702f7f360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f7f360 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc00e530 tx=0x7fc2dc000ee0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 -- 172.21.15.32:0/2436474403 shutdown_connections 2023-12-20T04:29:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558702f3e990 0x558702f7fde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.32:6834/3391762708,v1:172.21.15.32:6836/3391762708] conn(0x558702f80320 0x558702fa3590 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558702f3e070 0x558702f7f8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 --2- 172.21.15.32:0/2436474403 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558702ef41c0 0x558702f7f360 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2c0ff9640 0 client.5223 ms_handle_reset on v2:172.21.15.32:6834/3391762708 2023-12-20T04:29:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.126+0000 7fc2eb085000 1 -- 172.21.15.32:0/2436474403 >> 172.21.15.32:0/2436474403 conn(0x558702efa950 msgr2=0x558702f3b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2eb085000 1 -- 172.21.15.32:0/2436474403 shutdown_connections 2023-12-20T04:29:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2eb085000 1 -- 172.21.15.32:0/2436474403 wait complete. 2023-12-20T04:29:41.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2eb085000 20 client.5223 trim_cache size 0 max 16384 2023-12-20T04:29:41.140 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:29:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T04:29:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T04:29:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5031, 2023-12-20T04:29:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T04:29:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T04:29:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T04:29:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 10.518589216000001, 2023-12-20T04:29:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 16, 2023-12-20T04:29:41.140 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 35, 2023-12-20T04:29:41.141 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 35, 2023-12-20T04:29:41.141 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 17.75408814 2023-12-20T04:29:41.141 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:29:41.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 -- 172.21.15.32:0/797498447 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2c4065a60 msgr2=0x7fc2c4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2c4065a60 0x7fc2c4067f20 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fc2dc000f70 tx=0x7fc2dc009040 comp rx=0 tx=0).stop 2023-12-20T04:29:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 -- 172.21.15.32:0/797498447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2e015ce30 msgr2=0x7fc2e00b0c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2e015ce30 0x7fc2e00b0c10 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7fc2d8007e40 tx=0x7fc2d800e4b0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 -- 172.21.15.32:0/797498447 shutdown_connections 2023-12-20T04:29:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2c4065a60 0x7fc2c4067f20 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2e015ce30 0x7fc2e00b0c10 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2e0070950 0x7fc2e00c89c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 --2- 172.21.15.32:0/797498447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2e006deb0 0x7fc2e0154fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 -- 172.21.15.32:0/797498447 >> 172.21.15.32:0/797498447 conn(0x7fc2e00cba50 msgr2=0x7fc2e00ce6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 -- 172.21.15.32:0/797498447 shutdown_connections 2023-12-20T04:29:41.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.130+0000 7fc2e7d18640 1 -- 172.21.15.32:0/797498447 wait complete. 2023-12-20T04:29:41.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T04:29:41.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98bd579640 1 Processor -- start 2023-12-20T04:29:41.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98bd579640 1 -- start start 2023-12-20T04:29:41.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98bd579640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98b80b8480 0x7f98b806af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98bd579640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b806b460 0x7f98b8160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98bd579640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98b806b8c0 con 0x7f98b8161330 2023-12-20T04:29:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98bd579640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98b80cd4d0 con 0x7f98b80b8480 2023-12-20T04:29:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98bd579640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b80ce4e0 con 0x7f98b806b460 2023-12-20T04:29:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98b7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98b80b8480 0x7f98b806af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98b77fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b806b460 0x7f98b8160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98b7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98b80b8480 0x7f98b806af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:60916/0 (socket says 172.21.15.32:60916) 2023-12-20T04:29:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98b77fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b806b460 0x7f98b8160df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:47066/0 (socket says 172.21.15.32:47066) 2023-12-20T04:29:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98b7fff640 1 -- 172.21.15.32:0/404510181 learned_addr learned my addr 172.21.15.32:0/404510181 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98b77fe640 1 -- 172.21.15.32:0/404510181 >> v2:172.21.15.39:3300/0 conn(0x7f98b80b8480 msgr2=0x7f98b806af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98b77fe640 1 --2- 172.21.15.32:0/404510181 >> v2:172.21.15.39:3300/0 conn(0x7f98b80b8480 0x7f98b806af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98b77fe640 1 -- 172.21.15.32:0/404510181 >> v1:172.21.15.39:6789/0 conn(0x7f98b8161330 legacy=0x7f98b8153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98b77fe640 1 -- 172.21.15.32:0/404510181 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98b80cd720 con 0x7f98b806b460 2023-12-20T04:29:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.294+0000 7f98b7fff640 1 --2- 172.21.15.32:0/404510181 >> v2:172.21.15.39:3300/0 conn(0x7f98b80b8480 0x7f98b806af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:41.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98b77fe640 1 --2- 172.21.15.32:0/404510181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b806b460 0x7f98b8160df0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0097c0 tx=0x7f98ac01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f0af38bfbcb42edf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:41.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98b6ffd640 1 -- 172.21.15.32:0/404510181 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0ab600 con 0x7f98b806b460 2023-12-20T04:29:41.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98b6ffd640 1 -- 172.21.15.32:0/404510181 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98ac0a7050 con 0x7f98b806b460 2023-12-20T04:29:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98b6ffd640 1 -- 172.21.15.32:0/404510181 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98ac0ab980 con 0x7f98b806b460 2023-12-20T04:29:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98bd579640 1 -- 172.21.15.32:0/404510181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b806b460 msgr2=0x7f98b8160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98bd579640 1 --2- 172.21.15.32:0/404510181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b806b460 0x7f98b8160df0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0097c0 tx=0x7f98ac01c480 comp rx=0 tx=0).stop 2023-12-20T04:29:41.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98bd579640 1 -- 172.21.15.32:0/404510181 shutdown_connections 2023-12-20T04:29:41.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98bd579640 1 --2- 172.21.15.32:0/404510181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b806b460 0x7f98b8160df0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98bd579640 1 --2- 172.21.15.32:0/404510181 >> v2:172.21.15.39:3300/0 conn(0x7f98b80b8480 0x7f98b806af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98bd579640 1 -- 172.21.15.32:0/404510181 >> 172.21.15.32:0/404510181 conn(0x7f98b80d2a50 msgr2=0x7f98b80d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:41.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98bd579640 1 -- 172.21.15.32:0/404510181 shutdown_connections 2023-12-20T04:29:41.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.298+0000 7f98bd579640 1 -- 172.21.15.32:0/404510181 wait complete. 2023-12-20T04:29:41.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bd579640 1 Processor -- start 2023-12-20T04:29:41.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bd579640 1 -- start start 2023-12-20T04:29:41.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bd579640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98b806b460 0x7f98b814dc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bd579640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b80b8480 0x7f98b8147ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bd579640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98b8161330 0x7f98b81561d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bd579640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b8154f50 con 0x7f98b8161330 2023-12-20T04:29:41.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bd579640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98b814e160 con 0x7f98b806b460 2023-12-20T04:29:41.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bd579640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98b8155130 con 0x7f98b80b8480 2023-12-20T04:29:41.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bcd78640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98b8161330 0x7f98b81561d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98b806b460 0x7f98b814dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bcd78640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98b8161330 0x7f98b81561d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41212/0 (socket says 172.21.15.32:41212) 2023-12-20T04:29:41.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bcd78640 1 -- 172.21.15.32:0/3673254594 learned_addr learned my addr 172.21.15.32:0/3673254594 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:41.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98b77fe640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b80b8480 0x7f98b8147ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98b7fff640 1 -- 172.21.15.32:0/3673254594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b80b8480 msgr2=0x7f98b8147ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98b7fff640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b80b8480 0x7f98b8147ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98b7fff640 1 -- 172.21.15.32:0/3673254594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98b8161330 msgr2=0x7f98b81561d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98b7fff640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98b8161330 0x7f98b81561d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98b7fff640 1 -- 172.21.15.32:0/3673254594 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98a80094d0 con 0x7f98b806b460 2023-12-20T04:29:41.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bcd78640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98b8161330 0x7f98b81561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:41.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98b7fff640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98b806b460 0x7f98b814dc20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f98a800e910 tx=0x7f98a800edd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:41.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98b4ff9640 1 -- 172.21.15.32:0/3673254594 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a800ca00 con 0x7f98b806b460 2023-12-20T04:29:41.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98b4ff9640 1 -- 172.21.15.32:0/3673254594 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98a800cb60 con 0x7f98b806b460 2023-12-20T04:29:41.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98b4ff9640 1 -- 172.21.15.32:0/3673254594 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98a8004030 con 0x7f98b806b460 2023-12-20T04:29:41.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bd579640 1 -- 172.21.15.32:0/3673254594 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98ac009420 con 0x7f98b806b460 2023-12-20T04:29:41.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.302+0000 7f98bd579640 1 -- 172.21.15.32:0/3673254594 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98b81556f0 con 0x7f98b806b460 2023-12-20T04:29:41.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.306+0000 7f98bd579640 1 -- 172.21.15.32:0/3673254594 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98b8074100 con 0x7f98b806b460 2023-12-20T04:29:41.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.306+0000 7f98b4ff9640 1 -- 172.21.15.32:0/3673254594 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f98a8004190 con 0x7f98b806b460 2023-12-20T04:29:41.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.306+0000 7f98b4ff9640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98a4065a60 0x7f98a4067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:41.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.306+0000 7f98b4ff9640 1 -- 172.21.15.32:0/3673254594 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f98a80833c0 con 0x7f98b806b460 2023-12-20T04:29:41.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.306+0000 7f98b77fe640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98a4065a60 0x7f98a4067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:41.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.306+0000 7f98b77fe640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98a4065a60 0x7f98a4067f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0ad000 tx=0x7f98ac0a6000 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:41.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.310+0000 7f98b4ff9640 1 -- 172.21.15.32:0/3673254594 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f98a8050e10 con 0x7f98b806b460 2023-12-20T04:29:41.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.502+0000 7f98bd579640 1 -- 172.21.15.32:0/3673254594 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f98b80d0a70 con 0x7f98b806b460 2023-12-20T04:29:41.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.502+0000 7f98b4ff9640 1 -- 172.21.15.32:0/3673254594 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v16) v1 ==== 53+0+73 (secure 0 0 0) 0x7f98a8054430 con 0x7f98b806b460 2023-12-20T04:29:41.506 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T04:29:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 -- 172.21.15.32:0/3673254594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98a4065a60 msgr2=0x7f98a4067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98a4065a60 0x7f98a4067f20 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f98ac0ad000 tx=0x7f98ac0a6000 comp rx=0 tx=0).stop 2023-12-20T04:29:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 -- 172.21.15.32:0/3673254594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98b806b460 msgr2=0x7f98b814dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98b806b460 0x7f98b814dc20 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f98a800e910 tx=0x7f98a800edd0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 -- 172.21.15.32:0/3673254594 shutdown_connections 2023-12-20T04:29:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98a4065a60 0x7f98a4067f20 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98b8161330 0x7f98b81561d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98b80b8480 0x7f98b8147ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 --2- 172.21.15.32:0/3673254594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98b806b460 0x7f98b814dc20 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:41.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 -- 172.21.15.32:0/3673254594 >> 172.21.15.32:0/3673254594 conn(0x7f98b80d2a50 msgr2=0x7f98b814c2f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:41.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 -- 172.21.15.32:0/3673254594 shutdown_connections 2023-12-20T04:29:41.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:41.506+0000 7f98bd579640 1 -- 172.21.15.32:0/3673254594 wait complete. 2023-12-20T04:29:41.528 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T04:29:41.528 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T04:29:41.528 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T04:29:41.528 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T04:29:41.528 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T04:29:41.528 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T04:29:41.528 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T04:29:41.528 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T04:29:41.528 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:29:41.528 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:41.529 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T04:29:41.539 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:29:41.539 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:29:41.539 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:29:41.539 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:41.539 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T04:29:41.539 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:41.539 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 42278sec preferred_lft 42278sec 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T04:29:41.540 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T04:29:41.541 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:41.541 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T04:29:41.541 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:41.541 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:29:41.541 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T04:29:41.541 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T04:29:41.541 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:41.541 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T04:29:41.541 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:41.541 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:29:41.541 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T04:29:41.542 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T04:29:41.542 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:41.542 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:29:41.542 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T04:29:41.542 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T04:29:41.542 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:29:41.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:41.542 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T04:29:41.603 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T04:29:41.605 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:41.605 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T04:29:41.657 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T04:29:41.658 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T04:29:41.658 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T04:29:41.658 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T04:29:41.658 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T04:29:41.658 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T04:29:41.658 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:41.658 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:29:41.658 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:29:41.658 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T04:29:41.658 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T04:29:41.659 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T04:29:41.659 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:29:41.752 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:41.752 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T04:29:41.762 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T04:29:41.762 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T04:29:41.762 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:29:41.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:41.763 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T04:29:41.826 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:29:41.826 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T04:29:41.880 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:29:41.880 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:29:41.880 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:29:41.931 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T04:29:41.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:41.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:29:42.023 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:29:42.023 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:29:42.036 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:29:42.036 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:29:42.036 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T04:29:42.036 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T04:29:42.036 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T04:29:42.040 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T04:29:42.040 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:42.040 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:29:42.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:42.061 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:29:42.125 INFO:teuthology.orchestra.run.smithi032.stdout:client5194 2023-12-20T04:29:42.125 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:29:42.125 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:29:42.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T04:29:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.362+0000 7f6e7fd7b640 1 Processor -- start 2023-12-20T04:29:42.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.362+0000 7f6e7fd7b640 1 -- start start 2023-12-20T04:29:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.362+0000 7f6e7fd7b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6e780af5e0 0x7f6e780af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7fd7b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7815b990 con 0x7f6e7815f110 2023-12-20T04:29:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7fd7b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e7815bbd0 con 0x7f6e7806ce60 2023-12-20T04:29:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7fd7b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e7815beb0 con 0x7f6e780af5e0 2023-12-20T04:29:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7e318640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6e7815f110 0x7f6e7815c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34090/0 (socket says 172.21.15.32:34090) 2023-12-20T04:29:42.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7e318640 1 -- 172.21.15.32:0/3471021972 learned_addr learned my addr 172.21.15.32:0/3471021972 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7db17640 1 --2- 172.21.15.32:0/3471021972 >> v2:172.21.15.39:3300/0 conn(0x7f6e780af5e0 0x7f6e780af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7cb15640 1 -- 172.21.15.32:0/3471021972 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704011629 0 0) 0x7f6e700014a0 con 0x7f6e7815f110 2023-12-20T04:29:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7cb15640 1 -- 172.21.15.32:0/3471021972 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e64003530 con 0x7f6e7815f110 2023-12-20T04:29:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7cb15640 1 -- 172.21.15.32:0/3471021972 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 56174270 0 0) 0x7f6e7815bbd0 con 0x7f6e7806ce60 2023-12-20T04:29:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7cb15640 1 -- 172.21.15.32:0/3471021972 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6e700014a0 con 0x7f6e7806ce60 2023-12-20T04:29:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7db17640 1 -- 172.21.15.32:0/3471021972 >> v1:172.21.15.39:6789/0 conn(0x7f6e7806ce60 legacy=0x7f6e7815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:42.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7db17640 1 -- 172.21.15.32:0/3471021972 >> v1:172.21.15.32:6789/0 conn(0x7f6e7815f110 legacy=0x7f6e7815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7cb15640 1 -- 172.21.15.32:0/3471021972 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2303624279 0 0) 0x7f6e64003530 con 0x7f6e7815f110 2023-12-20T04:29:42.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.366+0000 7f6e7db17640 1 -- 172.21.15.32:0/3471021972 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e7815e910 con 0x7f6e780af5e0 2023-12-20T04:29:42.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7db17640 1 --2- 172.21.15.32:0/3471021972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e780af5e0 0x7f6e780af9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6e740097a0 tx=0x7f6e7401c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5122e48a4f0a495c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7cb15640 1 -- 172.21.15.32:0/3471021972 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e70001b90 con 0x7f6e780af5e0 2023-12-20T04:29:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7cb15640 1 -- 172.21.15.32:0/3471021972 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e740a7050 con 0x7f6e780af5e0 2023-12-20T04:29:42.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7cb15640 1 -- 172.21.15.32:0/3471021972 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e740ab620 con 0x7f6e780af5e0 2023-12-20T04:29:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/3471021972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e780af5e0 msgr2=0x7f6e780af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 --2- 172.21.15.32:0/3471021972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e780af5e0 0x7f6e780af9c0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6e740097a0 tx=0x7f6e7401c350 comp rx=0 tx=0).stop 2023-12-20T04:29:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/3471021972 shutdown_connections 2023-12-20T04:29:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 --2- 172.21.15.32:0/3471021972 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e780af5e0 0x7f6e780af9c0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/3471021972 >> 172.21.15.32:0/3471021972 conn(0x7f6e7806a320 msgr2=0x7f6e780ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/3471021972 shutdown_connections 2023-12-20T04:29:42.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/3471021972 wait complete. 2023-12-20T04:29:42.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 Processor -- start 2023-12-20T04:29:42.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 -- start start 2023-12-20T04:29:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e7806ce60 0x7f6e78159150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e780af5e0 0x7f6e78165830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:42.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7815f110 0x7f6e78151660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780701c0 con 0x7f6e780af5e0 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e78070d00 con 0x7f6e7815f110 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.370+0000 7f6e7fd7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e780703d0 con 0x7f6e7806ce60 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7d316640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e780af5e0 0x7f6e78165830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7d316640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e780af5e0 0x7f6e78165830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41226/0 (socket says 172.21.15.32:41226) 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7d316640 1 -- 172.21.15.32:0/2646308963 learned_addr learned my addr 172.21.15.32:0/2646308963 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7db17640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e7806ce60 0x7f6e78159150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7e318640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7815f110 0x7f6e78151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7d316640 1 -- 172.21.15.32:0/2646308963 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e7806ce60 msgr2=0x7f6e78159150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7d316640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e7806ce60 0x7f6e78159150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7d316640 1 -- 172.21.15.32:0/2646308963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7815f110 msgr2=0x7f6e78151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7d316640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7815f110 0x7f6e78151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7d316640 1 -- 172.21.15.32:0/2646308963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e740093b0 con 0x7f6e780af5e0 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7db17640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e7806ce60 0x7f6e78159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7e318640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7815f110 0x7f6e78151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:42.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7d316640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e780af5e0 0x7f6e78165830 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c4c0 tx=0x7f6e6800c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e6e7fc640 1 -- 172.21.15.32:0/2646308963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e680052d0 con 0x7f6e780af5e0 2023-12-20T04:29:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e6e7fc640 1 -- 172.21.15.32:0/2646308963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e68005430 con 0x7f6e780af5e0 2023-12-20T04:29:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/2646308963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e780706b0 con 0x7f6e780af5e0 2023-12-20T04:29:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e6e7fc640 1 -- 172.21.15.32:0/2646308963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e68012870 con 0x7f6e780af5e0 2023-12-20T04:29:42.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/2646308963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e78153360 con 0x7f6e780af5e0 2023-12-20T04:29:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/2646308963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e7804e000 con 0x7f6e780af5e0 2023-12-20T04:29:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e6e7fc640 1 -- 172.21.15.32:0/2646308963 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7f6e680129d0 con 0x7f6e780af5e0 2023-12-20T04:29:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e6e7fc640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e5c065a60 0x7f6e5c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.374+0000 7f6e6e7fc640 1 -- 172.21.15.32:0/2646308963 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6e68082d70 con 0x7f6e780af5e0 2023-12-20T04:29:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.378+0000 7f6e7db17640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e5c065a60 0x7f6e5c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:42.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.378+0000 7f6e7db17640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e5c065a60 0x7f6e5c067f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e740ad000 tx=0x7f6e74008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:42.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.378+0000 7f6e6e7fc640 1 -- 172.21.15.32:0/2646308963 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6e68050720 con 0x7f6e780af5e0 2023-12-20T04:29:42.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.558+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/2646308963 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f6e7815c190 con 0x7f6e5c065a60 2023-12-20T04:29:42.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.558+0000 7f6e6e7fc640 1 -- 172.21.15.32:0/2646308963 <== mgr.4104 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f6e740bb080 con 0x7f6e5c065a60 2023-12-20T04:29:42.563 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:29:42.563 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:29:42.563 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T04:29:42.563 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:29:42.563 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:29:42.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/2646308963 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e5c065a60 msgr2=0x7f6e5c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:42.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e5c065a60 0x7f6e5c067f20 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f6e740ad000 tx=0x7f6e74008040 comp rx=0 tx=0).stop 2023-12-20T04:29:42.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/2646308963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e780af5e0 msgr2=0x7f6e78165830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:42.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e780af5e0 0x7f6e78165830 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7f6e6800c4c0 tx=0x7f6e6800c980 comp rx=0 tx=0).stop 2023-12-20T04:29:42.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/2646308963 shutdown_connections 2023-12-20T04:29:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e5c065a60 0x7f6e5c067f20 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e7815f110 0x7f6e78151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e780af5e0 0x7f6e78165830 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 --2- 172.21.15.32:0/2646308963 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e7806ce60 0x7f6e78159150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/2646308963 >> 172.21.15.32:0/2646308963 conn(0x7f6e7806a320 msgr2=0x7f6e780ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/2646308963 shutdown_connections 2023-12-20T04:29:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.562+0000 7f6e7fd7b640 1 -- 172.21.15.32:0/2646308963 wait complete. 2023-12-20T04:29:42.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T04:29:42.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.726+0000 7fcf95d7b640 1 Processor -- start 2023-12-20T04:29:42.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.726+0000 7fcf95d7b640 1 -- start start 2023-12-20T04:29:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf95d7b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf90157480 0x7fcf900c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf95d7b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcf900b0760 0x7fcf900b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf95d7b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf900b1100 con 0x7fcf9015d660 2023-12-20T04:29:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf95d7b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf900cde20 con 0x7fcf90157480 2023-12-20T04:29:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf95d7b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf900c8e30 con 0x7fcf900b0760 2023-12-20T04:29:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf8f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf90157480 0x7fcf900c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf8f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf90157480 0x7fcf900c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41230/0 (socket says 172.21.15.32:41230) 2023-12-20T04:29:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf8f7fe640 1 -- 172.21.15.32:0/3317530849 learned_addr learned my addr 172.21.15.32:0/3317530849 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf8ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcf9015d660 0x7fcf9015da80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34102/0 (socket says 172.21.15.32:34102) 2023-12-20T04:29:42.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf8effd640 1 --2- 172.21.15.32:0/3317530849 >> v2:172.21.15.39:3300/0 conn(0x7fcf900b0760 0x7fcf900b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf8f7fe640 1 -- 172.21.15.32:0/3317530849 >> v2:172.21.15.39:3300/0 conn(0x7fcf900b0760 msgr2=0x7fcf900b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf8f7fe640 1 --2- 172.21.15.32:0/3317530849 >> v2:172.21.15.39:3300/0 conn(0x7fcf900b0760 0x7fcf900b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf8f7fe640 1 -- 172.21.15.32:0/3317530849 >> v1:172.21.15.32:6789/0 conn(0x7fcf9015d660 legacy=0x7fcf9015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf8f7fe640 1 -- 172.21.15.32:0/3317530849 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf900ce070 con 0x7fcf90157480 2023-12-20T04:29:42.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.730+0000 7fcf8e7fc640 1 -- 172.21.15.32:0/3317530849 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198717203 0 0) 0x7fcf900b1100 con 0x7fcf9015d660 2023-12-20T04:29:42.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf8f7fe640 1 --2- 172.21.15.32:0/3317530849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf90157480 0x7fcf900c9840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcf9006e0d0 tx=0x7fcf8001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9a2ece5e86ece3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:42.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf8e7fc640 1 -- 172.21.15.32:0/3317530849 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf800ab640 con 0x7fcf90157480 2023-12-20T04:29:42.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf8e7fc640 1 -- 172.21.15.32:0/3317530849 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf800a7070 con 0x7fcf90157480 2023-12-20T04:29:42.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf8e7fc640 1 -- 172.21.15.32:0/3317530849 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf800ab9c0 con 0x7fcf90157480 2023-12-20T04:29:42.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf95d7b640 1 -- 172.21.15.32:0/3317530849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf90157480 msgr2=0x7fcf900c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf95d7b640 1 --2- 172.21.15.32:0/3317530849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf90157480 0x7fcf900c9840 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fcf9006e0d0 tx=0x7fcf8001c440 comp rx=0 tx=0).stop 2023-12-20T04:29:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf95d7b640 1 -- 172.21.15.32:0/3317530849 shutdown_connections 2023-12-20T04:29:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf95d7b640 1 --2- 172.21.15.32:0/3317530849 >> v2:172.21.15.39:3300/0 conn(0x7fcf900b0760 0x7fcf900b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf95d7b640 1 --2- 172.21.15.32:0/3317530849 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf90157480 0x7fcf900c9840 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf95d7b640 1 -- 172.21.15.32:0/3317530849 >> 172.21.15.32:0/3317530849 conn(0x7fcf9006a510 msgr2=0x7fcf900ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf95d7b640 1 -- 172.21.15.32:0/3317530849 shutdown_connections 2023-12-20T04:29:42.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf95d7b640 1 -- 172.21.15.32:0/3317530849 wait complete. 2023-12-20T04:29:42.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf95d7b640 1 Processor -- start 2023-12-20T04:29:42.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.734+0000 7fcf95d7b640 1 -- start start 2023-12-20T04:29:42.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf95d7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf900b0760 0x7fcf9014dc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:42.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf95d7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf90157480 0x7fcf90159400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:42.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf95d7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf9015d660 0x7fcf90162b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:42.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf95d7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90157860 con 0x7fcf900b0760 2023-12-20T04:29:42.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf95d7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90158680 con 0x7fcf9015d660 2023-12-20T04:29:42.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf95d7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf90158b10 con 0x7fcf90157480 2023-12-20T04:29:42.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf900b0760 0x7fcf9014dc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:42.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf900b0760 0x7fcf9014dc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41244/0 (socket says 172.21.15.32:41244) 2023-12-20T04:29:42.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8f7fe640 1 -- 172.21.15.32:0/60528630 learned_addr learned my addr 172.21.15.32:0/60528630 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:42.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8ffff640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf9015d660 0x7fcf90162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:42.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8effd640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf90157480 0x7fcf90159400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:42.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8f7fe640 1 -- 172.21.15.32:0/60528630 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf90157480 msgr2=0x7fcf90159400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:42.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8f7fe640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf90157480 0x7fcf90159400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8f7fe640 1 -- 172.21.15.32:0/60528630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf9015d660 msgr2=0x7fcf90162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:42.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8f7fe640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf9015d660 0x7fcf90162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:42.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8f7fe640 1 -- 172.21.15.32:0/60528630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf800094d0 con 0x7fcf900b0760 2023-12-20T04:29:42.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8effd640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf90157480 0x7fcf90159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:42.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf8f7fe640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf900b0760 0x7fcf9014dc40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcf800ad040 tx=0x7fcf8001ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:42.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf77fff640 1 -- 172.21.15.32:0/60528630 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf800be3f0 con 0x7fcf900b0760 2023-12-20T04:29:42.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf77fff640 1 -- 172.21.15.32:0/60528630 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf800be570 con 0x7fcf900b0760 2023-12-20T04:29:42.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf95d7b640 1 -- 172.21.15.32:0/60528630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf9014e180 con 0x7fcf900b0760 2023-12-20T04:29:42.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf77fff640 1 -- 172.21.15.32:0/60528630 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf800be920 con 0x7fcf900b0760 2023-12-20T04:29:42.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf95d7b640 1 -- 172.21.15.32:0/60528630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf90157d50 con 0x7fcf900b0760 2023-12-20T04:29:42.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf95d7b640 1 -- 172.21.15.32:0/60528630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf9004e000 con 0x7fcf900b0760 2023-12-20T04:29:42.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.738+0000 7fcf77fff640 1 -- 172.21.15.32:0/60528630 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 5) v1 ==== 85228+0+0 (secure 0 0 0) 0x7fcf800b43f0 con 0x7fcf900b0760 2023-12-20T04:29:42.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.742+0000 7fcf77fff640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf7c065a60 0x7fcf7c067f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:42.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.742+0000 7fcf77fff640 1 -- 172.21.15.32:0/60528630 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcf8011d750 con 0x7fcf900b0760 2023-12-20T04:29:42.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.742+0000 7fcf8effd640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf7c065a60 0x7fcf7c067f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:42.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.742+0000 7fcf8effd640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf7c065a60 0x7fcf7c067f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcf900713f0 tx=0x7fcf84008040 comp rx=0 tx=0).ready entity=mgr.4104 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:42.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.746+0000 7fcf77fff640 1 -- 172.21.15.32:0/60528630 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcf800eb120 con 0x7fcf900b0760 2023-12-20T04:29:42.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:42.942+0000 7fcf95d7b640 1 -- 172.21.15.32:0/60528630 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fcf90150920 con 0x7fcf900b0760 2023-12-20T04:29:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.706+0000 7fcf77fff640 1 -- 172.21.15.32:0/60528630 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 6) v1 ==== 85245+0+0 (secure 0 0 0) 0x7fcf800eb4f0 con 0x7fcf900b0760 2023-12-20T04:29:43.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.706+0000 7fcf77fff640 1 -- 172.21.15.32:0/60528630 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v6) v1 ==== 92+0+0 (secure 0 0 0) 0x7fcf800eb120 con 0x7fcf900b0760 2023-12-20T04:29:43.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 -- 172.21.15.32:0/60528630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf7c065a60 msgr2=0x7fcf7c067f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:43.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf7c065a60 0x7fcf7c067f20 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcf900713f0 tx=0x7fcf84008040 comp rx=0 tx=0).stop 2023-12-20T04:29:43.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 -- 172.21.15.32:0/60528630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf900b0760 msgr2=0x7fcf9014dc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:43.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf900b0760 0x7fcf9014dc40 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fcf800ad040 tx=0x7fcf8001ac80 comp rx=0 tx=0).stop 2023-12-20T04:29:43.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 -- 172.21.15.32:0/60528630 shutdown_connections 2023-12-20T04:29:43.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf7c065a60 0x7fcf7c067f20 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:43.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf9015d660 0x7fcf90162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:43.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf90157480 0x7fcf90159400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:43.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 --2- 172.21.15.32:0/60528630 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf900b0760 0x7fcf9014dc40 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:43.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 -- 172.21.15.32:0/60528630 >> 172.21.15.32:0/60528630 conn(0x7fcf9006a510 msgr2=0x7fcf90070ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:43.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 -- 172.21.15.32:0/60528630 shutdown_connections 2023-12-20T04:29:43.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.710+0000 7fcf95d7b640 1 -- 172.21.15.32:0/60528630 wait complete. 2023-12-20T04:29:43.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T04:29:43.888 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:43.884+0000 7f7ae049b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:29:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b5577b640 1 Processor -- start 2023-12-20T04:29:43.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b5577b640 1 -- start start 2023-12-20T04:29:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b5577b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b5015cb30 0x7f3b500b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b5577b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b501606b0 0x7f3b500cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b5577b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3b500cdbf0 0x7f3b500b4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b5577b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b500ce060 con 0x7f3b501606b0 2023-12-20T04:29:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b5577b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3b500d0d40 con 0x7f3b500cdbf0 2023-12-20T04:29:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b5577b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b50160bb0 con 0x7f3b5015cb30 2023-12-20T04:29:43.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4e7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b501606b0 0x7f3b500cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4e7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3b501606b0 0x7f3b500cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41260/0 (socket says 172.21.15.32:41260) 2023-12-20T04:29:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4e7fc640 1 -- 172.21.15.32:0/656185939 learned_addr learned my addr 172.21.15.32:0/656185939 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4f7fe640 1 --2- 172.21.15.32:0/656185939 >> v2:172.21.15.39:3300/0 conn(0x7f3b500cdbf0 0x7f3b500b4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4effd640 1 --2- 172.21.15.32:0/656185939 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b5015cb30 0x7f3b500b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4e7fc640 1 -- 172.21.15.32:0/656185939 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b5015cb30 msgr2=0x7f3b500b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4e7fc640 1 --2- 172.21.15.32:0/656185939 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b5015cb30 0x7f3b500b8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4e7fc640 1 -- 172.21.15.32:0/656185939 >> v2:172.21.15.39:3300/0 conn(0x7f3b500cdbf0 msgr2=0x7f3b500b4460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4e7fc640 1 --2- 172.21.15.32:0/656185939 >> v2:172.21.15.39:3300/0 conn(0x7f3b500cdbf0 0x7f3b500b4460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4e7fc640 1 -- 172.21.15.32:0/656185939 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b50160e90 con 0x7f3b501606b0 2023-12-20T04:29:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4effd640 1 --2- 172.21.15.32:0/656185939 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b5015cb30 0x7f3b500b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:43.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.902+0000 7f3b4f7fe640 1 --2- 172.21.15.32:0/656185939 >> v2:172.21.15.39:3300/0 conn(0x7f3b500cdbf0 0x7f3b500b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:43.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b4e7fc640 1 --2- 172.21.15.32:0/656185939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b501606b0 0x7f3b500cd6b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3b500b58c0 tx=0x7f3b4401c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac3bc324de25158e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:43.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b4dffb640 1 -- 172.21.15.32:0/656185939 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b440ab600 con 0x7f3b501606b0 2023-12-20T04:29:43.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b4dffb640 1 -- 172.21.15.32:0/656185939 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b440a7050 con 0x7f3b501606b0 2023-12-20T04:29:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b4dffb640 1 -- 172.21.15.32:0/656185939 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b440ab980 con 0x7f3b501606b0 2023-12-20T04:29:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b5577b640 1 -- 172.21.15.32:0/656185939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b501606b0 msgr2=0x7f3b500cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b5577b640 1 --2- 172.21.15.32:0/656185939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b501606b0 0x7f3b500cd6b0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f3b500b58c0 tx=0x7f3b4401c210 comp rx=0 tx=0).stop 2023-12-20T04:29:43.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b5577b640 1 -- 172.21.15.32:0/656185939 shutdown_connections 2023-12-20T04:29:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b5577b640 1 --2- 172.21.15.32:0/656185939 >> v2:172.21.15.39:3300/0 conn(0x7f3b500cdbf0 0x7f3b500b4460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b5577b640 1 --2- 172.21.15.32:0/656185939 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b501606b0 0x7f3b500cd6b0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b5577b640 1 --2- 172.21.15.32:0/656185939 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b5015cb30 0x7f3b500b8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b5577b640 1 -- 172.21.15.32:0/656185939 >> 172.21.15.32:0/656185939 conn(0x7f3b5006af60 msgr2=0x7f3b5006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b5577b640 1 -- 172.21.15.32:0/656185939 shutdown_connections 2023-12-20T04:29:43.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.906+0000 7f3b5577b640 1 -- 172.21.15.32:0/656185939 wait complete. 2023-12-20T04:29:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b5577b640 1 Processor -- start 2023-12-20T04:29:43.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b5577b640 1 -- start start 2023-12-20T04:29:43.913 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b8809b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:29:43.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b5577b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b500cdbf0 0x7f3b500c7af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b5577b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b5015cb30 0x7f3b500cb770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b5577b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b501606b0 0x7f3b500cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b5577b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500c8460 con 0x7f3b500cdbf0 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b5577b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500c8740 con 0x7f3b5015cb30 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b5577b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3b500c8a20 con 0x7f3b501606b0 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b4effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b500cdbf0 0x7f3b500c7af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b4effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b500cdbf0 0x7f3b500c7af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41276/0 (socket says 172.21.15.32:41276) 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b4effd640 1 -- 172.21.15.32:0/2222809286 learned_addr learned my addr 172.21.15.32:0/2222809286 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b4f7fe640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b501606b0 0x7f3b500cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b4e7fc640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b5015cb30 0x7f3b500cb770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b4effd640 1 -- 172.21.15.32:0/2222809286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b501606b0 msgr2=0x7f3b500cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b4effd640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b501606b0 0x7f3b500cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b4effd640 1 -- 172.21.15.32:0/2222809286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b5015cb30 msgr2=0x7f3b500cb770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b4effd640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b5015cb30 0x7f3b500cb770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:43.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b4effd640 1 -- 172.21.15.32:0/2222809286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3b48009a70 con 0x7f3b500cdbf0 2023-12-20T04:29:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b4effd640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b500cdbf0 0x7f3b500c7af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000ebe0 tx=0x7f3b40009e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b377fe640 1 -- 172.21.15.32:0/2222809286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b4000ca00 con 0x7f3b500cdbf0 2023-12-20T04:29:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b377fe640 1 -- 172.21.15.32:0/2222809286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3b4000cb60 con 0x7f3b500cdbf0 2023-12-20T04:29:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b377fe640 1 -- 172.21.15.32:0/2222809286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3b400059a0 con 0x7f3b500cdbf0 2023-12-20T04:29:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b5577b640 1 -- 172.21.15.32:0/2222809286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3b400093b0 con 0x7f3b500cdbf0 2023-12-20T04:29:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.910+0000 7f3b5577b640 1 -- 172.21.15.32:0/2222809286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3b440094d0 con 0x7f3b500cdbf0 2023-12-20T04:29:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.914+0000 7f3b5577b640 1 -- 172.21.15.32:0/2222809286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3b50074100 con 0x7f3b500cdbf0 2023-12-20T04:29:43.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.914+0000 7f3b377fe640 1 -- 172.21.15.32:0/2222809286 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 6) v1 ==== 85245+0+0 (secure 0 0 0) 0x7f3b40005b00 con 0x7f3b500cdbf0 2023-12-20T04:29:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.914+0000 7f3b377fe640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b3c065ab0 0x7f3b3c067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.914+0000 7f3b4e7fc640 1 -- 172.21.15.32:0/2222809286 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b3c065ab0 msgr2=0x7f3b3c067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:29:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.914+0000 7f3b377fe640 1 -- 172.21.15.32:0/2222809286 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3b40083610 con 0x7f3b500cdbf0 2023-12-20T04:29:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.914+0000 7f3b4e7fc640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b3c065ab0 0x7f3b3c067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:29:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:43.918+0000 7f3b377fe640 1 -- 172.21.15.32:0/2222809286 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3b40050fe0 con 0x7f3b500cdbf0 2023-12-20T04:29:44.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.082+0000 7f3b5577b640 1 -- 172.21.15.32:0/2222809286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f3b500c9300 con 0x7f3b500cdbf0 2023-12-20T04:29:44.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.086+0000 7f3b377fe640 1 -- 172.21.15.32:0/2222809286 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v11)=0 v11) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3b40054600 con 0x7f3b500cdbf0 2023-12-20T04:29:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 -- 172.21.15.32:0/2222809286 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b3c065ab0 msgr2=0x7f3b3c067f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:29:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b3c065ab0 0x7f3b3c067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 -- 172.21.15.32:0/2222809286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b500cdbf0 msgr2=0x7f3b500c7af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b500cdbf0 0x7f3b500c7af0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f3b4000ebe0 tx=0x7f3b40009e20 comp rx=0 tx=0).stop 2023-12-20T04:29:44.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 -- 172.21.15.32:0/2222809286 shutdown_connections 2023-12-20T04:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3b3c065ab0 0x7f3b3c067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3b501606b0 0x7f3b500cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3b5015cb30 0x7f3b500cb770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 --2- 172.21.15.32:0/2222809286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3b500cdbf0 0x7f3b500c7af0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 -- 172.21.15.32:0/2222809286 >> 172.21.15.32:0/2222809286 conn(0x7f3b5006af60 msgr2=0x7f3b5006b690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 -- 172.21.15.32:0/2222809286 shutdown_connections 2023-12-20T04:29:44.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.090+0000 7f3b5577b640 1 -- 172.21.15.32:0/2222809286 wait complete. 2023-12-20T04:29:44.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T04:29:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.258+0000 7fe7c39db640 1 Processor -- start 2023-12-20T04:29:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.258+0000 7fe7c39db640 1 -- start start 2023-12-20T04:29:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.258+0000 7fe7c39db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc164670 0x7fe7bc0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.258+0000 7fe7c39db640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc0cb0b0 con 0x7fe7bc070230 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.258+0000 7fe7c39db640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7bc0ca810 con 0x7fe7bc06da20 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.258+0000 7fe7c39db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0ccac0 con 0x7fe7bc164670 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.258+0000 7fe7c1f78640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe7bc070230 0x7fe7bc06ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34122/0 (socket says 172.21.15.32:34122) 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.258+0000 7fe7c1f78640 1 -- 172.21.15.32:0/3063054800 learned_addr learned my addr 172.21.15.32:0/3063054800 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.258+0000 7fe7c1777640 1 --2- 172.21.15.32:0/3063054800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc164670 0x7fe7bc0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7b3fff640 1 -- 172.21.15.32:0/3063054800 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011726290 0 0) 0x7fe7bc0cb0b0 con 0x7fe7bc070230 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7b3fff640 1 -- 172.21.15.32:0/3063054800 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7a8003530 con 0x7fe7bc070230 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7b3fff640 1 -- 172.21.15.32:0/3063054800 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2916458332 0 0) 0x7fe7bc0ca810 con 0x7fe7bc06da20 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7b3fff640 1 -- 172.21.15.32:0/3063054800 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe7bc0cb0b0 con 0x7fe7bc06da20 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7b3fff640 1 -- 172.21.15.32:0/3063054800 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3889908315 0 0) 0x7fe7a8003530 con 0x7fe7bc070230 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7c1777640 1 -- 172.21.15.32:0/3063054800 >> v1:172.21.15.39:6789/0 conn(0x7fe7bc06da20 legacy=0x7fe7bc0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7c1777640 1 -- 172.21.15.32:0/3063054800 >> v1:172.21.15.32:6789/0 conn(0x7fe7bc070230 legacy=0x7fe7bc06ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7c1777640 1 -- 172.21.15.32:0/3063054800 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7bc0caaf0 con 0x7fe7bc164670 2023-12-20T04:29:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7c1777640 1 --2- 172.21.15.32:0/3063054800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc164670 0x7fe7bc0ca1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80097f0 tx=0x7fe7b801c290 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34105ba0eb9e983e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7b3fff640 1 -- 172.21.15.32:0/3063054800 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80a4ea0 con 0x7fe7bc164670 2023-12-20T04:29:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7b3fff640 1 -- 172.21.15.32:0/3063054800 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b80a7050 con 0x7fe7bc164670 2023-12-20T04:29:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.262+0000 7fe7b3fff640 1 -- 172.21.15.32:0/3063054800 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b4001a00 con 0x7fe7bc164670 2023-12-20T04:29:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 -- 172.21.15.32:0/3063054800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc164670 msgr2=0x7fe7bc0ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 --2- 172.21.15.32:0/3063054800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc164670 0x7fe7bc0ca1a0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80097f0 tx=0x7fe7b801c290 comp rx=0 tx=0).stop 2023-12-20T04:29:44.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 -- 172.21.15.32:0/3063054800 shutdown_connections 2023-12-20T04:29:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 --2- 172.21.15.32:0/3063054800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc164670 0x7fe7bc0ca1a0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 -- 172.21.15.32:0/3063054800 >> 172.21.15.32:0/3063054800 conn(0x7fe7bc06a510 msgr2=0x7fe7bc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 -- 172.21.15.32:0/3063054800 shutdown_connections 2023-12-20T04:29:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 -- 172.21.15.32:0/3063054800 wait complete. 2023-12-20T04:29:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 Processor -- start 2023-12-20T04:29:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 -- start start 2023-12-20T04:29:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc06da20 0x7fe7bc1fcd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7bc070230 0x7fe7bc0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7bc164670 0x7fe7bc154120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0b17d0 con 0x7fe7bc070230 2023-12-20T04:29:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc0c8770 con 0x7fe7bc164670 2023-12-20T04:29:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c39db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7bc1fd310 con 0x7fe7bc06da20 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c0f76640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7bc070230 0x7fe7bc0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c1777640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc06da20 0x7fe7bc1fcd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c0f76640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7bc070230 0x7fe7bc0c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41286/0 (socket says 172.21.15.32:41286) 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c0f76640 1 -- 172.21.15.32:0/70054731 learned_addr learned my addr 172.21.15.32:0/70054731 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.266+0000 7fe7c1f78640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7bc164670 0x7fe7bc154120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7c1777640 1 -- 172.21.15.32:0/70054731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7bc164670 msgr2=0x7fe7bc154120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7c1777640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7bc164670 0x7fe7bc154120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7c1777640 1 -- 172.21.15.32:0/70054731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7bc070230 msgr2=0x7fe7bc0c8230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7c1777640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7bc070230 0x7fe7bc0c8230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7c1777640 1 -- 172.21.15.32:0/70054731 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7b8009420 con 0x7fe7bc06da20 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7c0f76640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7bc070230 0x7fe7bc0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7c1777640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc06da20 0x7fe7bc1fcd30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80ab000 tx=0x7fe7b801aad0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7b1ffb640 1 -- 172.21.15.32:0/70054731 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b8006e20 con 0x7fe7bc06da20 2023-12-20T04:29:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7b1ffb640 1 -- 172.21.15.32:0/70054731 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7b80b7450 con 0x7fe7bc06da20 2023-12-20T04:29:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7b1ffb640 1 -- 172.21.15.32:0/70054731 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7b80b77e0 con 0x7fe7bc06da20 2023-12-20T04:29:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7c39db640 1 -- 172.21.15.32:0/70054731 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7bc1fd5f0 con 0x7fe7bc06da20 2023-12-20T04:29:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7c39db640 1 -- 172.21.15.32:0/70054731 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7bc154660 con 0x7fe7bc06da20 2023-12-20T04:29:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7c39db640 1 -- 172.21.15.32:0/70054731 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7bc04e000 con 0x7fe7bc06da20 2023-12-20T04:29:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.270+0000 7fe7b1ffb640 1 -- 172.21.15.32:0/70054731 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 6) v1 ==== 85245+0+0 (secure 0 0 0) 0x7fe7b80b7940 con 0x7fe7bc06da20 2023-12-20T04:29:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.274+0000 7fe7b1ffb640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe7a0065ab0 0x7fe7a0067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.274+0000 7fe7b1ffb640 1 -- 172.21.15.32:0/70054731 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe7b811d5f0 con 0x7fe7bc06da20 2023-12-20T04:29:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.274+0000 7fe7c0f76640 1 -- 172.21.15.32:0/70054731 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe7a0065ab0 msgr2=0x7fe7a0067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:29:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.274+0000 7fe7c0f76640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe7a0065ab0 0x7fe7a0067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:29:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.278+0000 7fe7b1ffb640 1 -- 172.21.15.32:0/70054731 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe7b80eb0e0 con 0x7fe7bc06da20 2023-12-20T04:29:44.330 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:44.324+0000 7f7ae049b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:29:44.362 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:44.358+0000 7f3b8809b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:29:44.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.442+0000 7fe7c39db640 1 -- 172.21.15.32:0/70054731 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fe7bc0cf4d0 con 0x7fe7bc06da20 2023-12-20T04:29:44.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7b1ffb640 1 -- 172.21.15.32:0/70054731 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v12)=0 v12) v1 ==== 133+0+0 (secure 0 0 0) 0x7fe7b80ee700 con 0x7fe7bc06da20 2023-12-20T04:29:44.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 -- 172.21.15.32:0/70054731 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe7a0065ab0 msgr2=0x7fe7a0067f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:29:44.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe7a0065ab0 0x7fe7a0067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 -- 172.21.15.32:0/70054731 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc06da20 msgr2=0x7fe7bc1fcd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc06da20 0x7fe7bc1fcd30 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fe7b80ab000 tx=0x7fe7b801aad0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 -- 172.21.15.32:0/70054731 shutdown_connections 2023-12-20T04:29:44.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe7a0065ab0 0x7fe7a0067f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7bc164670 0x7fe7bc154120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7bc070230 0x7fe7bc0c8230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 --2- 172.21.15.32:0/70054731 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7bc06da20 0x7fe7bc1fcd30 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 -- 172.21.15.32:0/70054731 >> 172.21.15.32:0/70054731 conn(0x7fe7bc06a510 msgr2=0x7fe7bc158670 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 -- 172.21.15.32:0/70054731 shutdown_connections 2023-12-20T04:29:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.454+0000 7fe7c39db640 1 -- 172.21.15.32:0/70054731 wait complete. 2023-12-20T04:29:44.479 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:44.480 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-20T04:29:44.491 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:44.488+0000 7f7ae049b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:29:44.494 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:44.495 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir0) 2023-12-20T04:29:44.523 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:44.518+0000 7f3b8809b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:29:44.535 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir0 1M 2023-12-20T04:29:44.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.718+0000 7f3842560640 1 Processor -- start 2023-12-20T04:29:44.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.718+0000 7f3842560640 1 -- start start 2023-12-20T04:29:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.718+0000 7f3842560640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f383c0d3300 0x7f383c0d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.718+0000 7f3842560640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383c0d2650 con 0x7f383c15ce60 2023-12-20T04:29:44.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.718+0000 7f3842560640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f383c0d1a40 con 0x7f383c0d1300 2023-12-20T04:29:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.718+0000 7f3842560640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f383c0d1d20 con 0x7f383c0d3300 2023-12-20T04:29:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.718+0000 7f3833fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f383c0d3300 0x7f383c0d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.718+0000 7f3833fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f383c0d3300 0x7f383c0d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41294/0 (socket says 172.21.15.32:41294) 2023-12-20T04:29:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.718+0000 7f3833fff640 1 -- 172.21.15.32:0/194194875 learned_addr learned my addr 172.21.15.32:0/194194875 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:44.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.718+0000 7f383bfff640 1 --1- 172.21.15.32:0/194194875 >> v1:172.21.15.32:6789/0 conn(0x7f383c15ce60 0x7f383c0d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34134/0 (socket says 172.21.15.32:34134) 2023-12-20T04:29:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.722+0000 7f3833fff640 1 -- 172.21.15.32:0/194194875 >> v1:172.21.15.39:6789/0 conn(0x7f383c0d1300 legacy=0x7f383c0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.722+0000 7f3833fff640 1 -- 172.21.15.32:0/194194875 >> v1:172.21.15.32:6789/0 conn(0x7f383c15ce60 legacy=0x7f383c0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.722+0000 7f3833fff640 1 -- 172.21.15.32:0/194194875 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f383c0d2000 con 0x7f383c0d3300 2023-12-20T04:29:44.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.722+0000 7f383b7fe640 1 -- 172.21.15.32:0/194194875 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1774825656 0 0) 0x7f383c0d1a40 con 0x7f383c0d1300 2023-12-20T04:29:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.722+0000 7f3833fff640 1 --2- 172.21.15.32:0/194194875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f383c0d3300 0x7f383c0d0dc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3824009870 tx=0x7f382401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c838a2a55b9cd08f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.722+0000 7f383b7fe640 1 -- 172.21.15.32:0/194194875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38240ac630 con 0x7f383c0d3300 2023-12-20T04:29:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.722+0000 7f383b7fe640 1 -- 172.21.15.32:0/194194875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38240ac790 con 0x7f383c0d3300 2023-12-20T04:29:44.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.722+0000 7f383b7fe640 1 -- 172.21.15.32:0/194194875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38240acb20 con 0x7f383c0d3300 2023-12-20T04:29:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.726+0000 7f3842560640 1 -- 172.21.15.32:0/194194875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f383c0d3300 msgr2=0x7f383c0d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.726+0000 7f3842560640 1 --2- 172.21.15.32:0/194194875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f383c0d3300 0x7f383c0d0dc0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f3824009870 tx=0x7f382401c540 comp rx=0 tx=0).stop 2023-12-20T04:29:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.726+0000 7f3842560640 1 -- 172.21.15.32:0/194194875 shutdown_connections 2023-12-20T04:29:44.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.726+0000 7f3842560640 1 --2- 172.21.15.32:0/194194875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f383c0d3300 0x7f383c0d0dc0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.726+0000 7f3842560640 1 -- 172.21.15.32:0/194194875 >> 172.21.15.32:0/194194875 conn(0x7f383c06b5e0 msgr2=0x7f383c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.726+0000 7f3842560640 1 -- 172.21.15.32:0/194194875 shutdown_connections 2023-12-20T04:29:44.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.726+0000 7f3842560640 1 -- 172.21.15.32:0/194194875 wait complete. 2023-12-20T04:29:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.726+0000 7f3842560640 1 Processor -- start 2023-12-20T04:29:44.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.726+0000 7f3842560640 1 -- start start 2023-12-20T04:29:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3842560640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f383c0d1300 0x7f383c156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3842560640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c0d3300 0x7f383c0b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3842560640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c15ce60 0x7f383c0cae50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3842560640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c1599e0 con 0x7f383c0d1300 2023-12-20T04:29:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3842560640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f383c158d60 con 0x7f383c0d3300 2023-12-20T04:29:44.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3842560640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f383c159040 con 0x7f383c15ce60 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f383bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f383c0d1300 0x7f383c156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3833fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c0d3300 0x7f383c0b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f383bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f383c0d1300 0x7f383c156430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41304/0 (socket says 172.21.15.32:41304) 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f383bfff640 1 -- 172.21.15.32:0/1249381417 learned_addr learned my addr 172.21.15.32:0/1249381417 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3840afd640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c15ce60 0x7f383c0cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f383bfff640 1 -- 172.21.15.32:0/1249381417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c15ce60 msgr2=0x7f383c0cae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f383bfff640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c15ce60 0x7f383c0cae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f383bfff640 1 -- 172.21.15.32:0/1249381417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c0d3300 msgr2=0x7f383c0b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f383bfff640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c0d3300 0x7f383c0b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f383bfff640 1 -- 172.21.15.32:0/1249381417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38240094d0 con 0x7f383c0d1300 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3840afd640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c15ce60 0x7f383c0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:44.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f383bfff640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f383c0d1300 0x7f383c156430 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c430 tx=0x7f382c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f38397fa640 1 -- 172.21.15.32:0/1249381417 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c0074e0 con 0x7f383c0d1300 2023-12-20T04:29:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f38397fa640 1 -- 172.21.15.32:0/1249381417 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f382c007640 con 0x7f383c0d1300 2023-12-20T04:29:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f38397fa640 1 -- 172.21.15.32:0/1249381417 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f382c004a10 con 0x7f383c0d1300 2023-12-20T04:29:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3842560640 1 -- 172.21.15.32:0/1249381417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f383c159320 con 0x7f383c0d1300 2023-12-20T04:29:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3842560640 1 -- 172.21.15.32:0/1249381417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f383c156970 con 0x7f383c0d1300 2023-12-20T04:29:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.730+0000 7f3842560640 1 -- 172.21.15.32:0/1249381417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f383c04e000 con 0x7f383c0d1300 2023-12-20T04:29:44.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.734+0000 7f38397fa640 1 -- 172.21.15.32:0/1249381417 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 6) v1 ==== 85245+0+0 (secure 0 0 0) 0x7f382c004b70 con 0x7f383c0d1300 2023-12-20T04:29:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.734+0000 7f38397fa640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 0x7f381c067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.734+0000 7f38397fa640 1 -- 172.21.15.32:0/1249381417 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(35..35 src has 1..35) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f382c082c30 con 0x7f383c0d1300 2023-12-20T04:29:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.734+0000 7f3833fff640 1 -- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 msgr2=0x7f381c067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:29:44.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.734+0000 7f3833fff640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 0x7f381c067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:29:44.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.738+0000 7f38397fa640 1 -- 172.21.15.32:0/1249381417 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f382c050550 con 0x7f383c0d1300 2023-12-20T04:29:44.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.918+0000 7f3842560640 1 -- 172.21.15.32:0/1249381417 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f383c0b6db0 con 0x7f381c065ab0 2023-12-20T04:29:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.938+0000 7f3833fff640 1 -- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 msgr2=0x7f381c067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:29:44.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:44.938+0000 7f3833fff640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 0x7f381c067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:29:45.004 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:45.000+0000 7f7ae049b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:29:45.039 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:45.034+0000 7f3b8809b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:29:45.065 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:45.060+0000 7f7ae049b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:29:45.100 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:45.098+0000 7f3b8809b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:29:45.123 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:45.120+0000 7f7ae049b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:29:45.159 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:45.154+0000 7f3b8809b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:29:45.216 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:45.212+0000 7f7ae049b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:29:45.250 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:45.246+0000 7f3b8809b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:29:45.275 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:45.272+0000 7f7ae049b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:29:45.310 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:45.306+0000 7f3b8809b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:29:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:45.342+0000 7f3833fff640 1 -- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 msgr2=0x7f381c067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:29:45.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:45.342+0000 7f3833fff640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 0x7f381c067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T04:29:45.363 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:45.360+0000 7f7ae049b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:29:45.397 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:45.394+0000 7f3b8809b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:29:45.422 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:45.416+0000 7f7ae049b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:29:45.457 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:45.454+0000 7f3b8809b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:29:45.573 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:45.568+0000 7f7ae049b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:29:45.607 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:45.602+0000 7f3b8809b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:29:45.634 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:45.632+0000 7f7ae049b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:29:45.669 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:45.666+0000 7f3b8809b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:29:45.885 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:29:45.885 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:29:45.885 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:29:45.898 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:45.888+0000 7f7ae049b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:29:45.919 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:29:45.919 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:29:45.919 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:29:45.928 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:45.926+0000 7f3b8809b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:29:46.038 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:46.032+0000 7f7ae049b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:29:46.070 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:46.066+0000 7f3b8809b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:29:46.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:46.146+0000 7f3833fff640 1 -- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 msgr2=0x7f381c067f70 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:29:46.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:46.146+0000 7f3833fff640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 0x7f381c067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T04:29:46.168 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:46.164+0000 7f7ae049b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:29:46.201 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:46.198+0000 7f3b8809b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:29:46.243 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:46.240+0000 7f7ae049b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:29:46.277 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:46.274+0000 7f3b8809b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:29:46.387 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:46.380+0000 7f7ae049b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:29:46.420 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:46.418+0000 7f3b8809b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:29:46.963 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:46.956+0000 7f7ae049b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:29:46.992 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:46.990+0000 7f3b8809b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:29:47.050 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:47.044+0000 7f7ae049b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:29:47.081 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.078+0000 7f3b8809b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:29:47.207 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:47.204+0000 7f7ae049b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:29:47.237 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.234+0000 7f3b8809b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:29:47.296 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:47.292+0000 7f7ae049b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:29:47.325 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.322+0000 7f3b8809b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:29:47.632 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:47.628+0000 7f7ae049b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:29:47.660 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.658+0000 7f3b8809b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:29:47.698 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:29:47.692+0000 7f7ae049b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:29:47.724 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.722+0000 7f3b8809b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:29:47.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:47.738+0000 7f38397fa640 1 -- 172.21.15.32:0/1249381417 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 7) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f382c050920 con 0x7f383c0d1300 2023-12-20T04:29:47.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:47.738+0000 7f38397fa640 1 -- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 msgr2=0x7f381c067f70 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:29:47.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:47.738+0000 7f38397fa640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 0x7f381c067f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:47.862 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.858+0000 7f3b0b00d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:29:47.862 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.858+0000 7f3b0b00d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:29:47.862 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.858+0000 7f3b0b00d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:29:47.863 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.858+0000 7f3b0b00d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:29:47.863 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.858+0000 7f3b0b00d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:29:47.863 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.858+0000 7f3b0d011640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:29:47.863 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.858+0000 7f3b0d011640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:29:47.863 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.858+0000 7f3b0d011640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:29:47.863 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.858+0000 7f3b0d011640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:29:47.864 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:29:47.858+0000 7f3b0d011640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:29:48.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.742+0000 7f38397fa640 1 -- 172.21.15.32:0/1249381417 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 8) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f382c0582a0 con 0x7f383c0d1300 2023-12-20T04:29:48.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.742+0000 7f38397fa640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 0x7f381c067f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:48.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.742+0000 7f38397fa640 1 -- 172.21.15.32:0/1249381417 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir0", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f381c01bfa0 con 0x7f381c065ab0 2023-12-20T04:29:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.746+0000 7f3833fff640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 0x7f381c067f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.750+0000 7f3833fff640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 0x7f381c067f70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f38240ae000 tx=0x7f3824008040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:48.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.834+0000 7f38397fa640 1 -- 172.21.15.32:0/1249381417 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f381c01bfa0 con 0x7f381c065ab0 2023-12-20T04:29:48.839 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1/dir0 2023-12-20T04:29:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 -- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 msgr2=0x7f381c067f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 0x7f381c067f70 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f38240ae000 tx=0x7f3824008040 comp rx=0 tx=0).stop 2023-12-20T04:29:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 -- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f383c0d1300 msgr2=0x7f383c156430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f383c0d1300 0x7f383c156430 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f382c00c430 tx=0x7f382c00c8f0 comp rx=0 tx=0).stop 2023-12-20T04:29:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 -- 172.21.15.32:0/1249381417 shutdown_connections 2023-12-20T04:29:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f381c065ab0 0x7f381c067f70 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f383c15ce60 0x7f383c0cae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f383c0d3300 0x7f383c0b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 --2- 172.21.15.32:0/1249381417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f383c0d1300 0x7f383c156430 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 -- 172.21.15.32:0/1249381417 >> 172.21.15.32:0/1249381417 conn(0x7f383c06b5e0 msgr2=0x7f383c158390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 -- 172.21.15.32:0/1249381417 shutdown_connections 2023-12-20T04:29:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:48.838+0000 7f3842560640 1 -- 172.21.15.32:0/1249381417 wait complete. 2023-12-20T04:29:48.862 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir0 2023-12-20T04:29:48.862 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:48.862 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir1) 2023-12-20T04:29:48.873 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir1 1M 2023-12-20T04:29:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.058+0000 7faf879ce640 1 Processor -- start 2023-12-20T04:29:49.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.058+0000 7faf879ce640 1 -- start start 2023-12-20T04:29:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf879ce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faf80164670 0x7faf800ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf879ce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf800cb0b0 con 0x7faf80070230 2023-12-20T04:29:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf879ce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf800ca810 con 0x7faf8006da20 2023-12-20T04:29:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf879ce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf800ccac0 con 0x7faf80164670 2023-12-20T04:29:49.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf8576a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faf80164670 0x7faf800ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf84f69640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7faf8006da20 0x7faf800cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:56676/0 (socket says 172.21.15.32:56676) 2023-12-20T04:29:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf84f69640 1 -- 172.21.15.32:0/3852717540 learned_addr learned my addr 172.21.15.32:0/3852717540 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf77fff640 1 -- 172.21.15.32:0/3852717540 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2721943123 0 0) 0x7faf800ca810 con 0x7faf8006da20 2023-12-20T04:29:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf77fff640 1 -- 172.21.15.32:0/3852717540 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faf6c003530 con 0x7faf8006da20 2023-12-20T04:29:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf77fff640 1 -- 172.21.15.32:0/3852717540 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1399082098 0 0) 0x7faf800cb0b0 con 0x7faf80070230 2023-12-20T04:29:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf8576a640 1 -- 172.21.15.32:0/3852717540 >> v1:172.21.15.39:6789/0 conn(0x7faf8006da20 legacy=0x7faf800cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf8576a640 1 -- 172.21.15.32:0/3852717540 >> v1:172.21.15.32:6789/0 conn(0x7faf80070230 legacy=0x7faf80070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.062+0000 7faf8576a640 1 -- 172.21.15.32:0/3852717540 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf800caaf0 con 0x7faf80164670 2023-12-20T04:29:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf8576a640 1 --2- 172.21.15.32:0/3852717540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf80164670 0x7faf800ca1a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0097c0 tx=0x7faf7c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c8ff64f2ca9a97f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf77fff640 1 -- 172.21.15.32:0/3852717540 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0ab600 con 0x7faf80164670 2023-12-20T04:29:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf77fff640 1 -- 172.21.15.32:0/3852717540 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf7c0a7050 con 0x7faf80164670 2023-12-20T04:29:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf77fff640 1 -- 172.21.15.32:0/3852717540 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf7c0ab980 con 0x7faf80164670 2023-12-20T04:29:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf879ce640 1 -- 172.21.15.32:0/3852717540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf80164670 msgr2=0x7faf800ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf879ce640 1 --2- 172.21.15.32:0/3852717540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf80164670 0x7faf800ca1a0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0097c0 tx=0x7faf7c01c330 comp rx=0 tx=0).stop 2023-12-20T04:29:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf879ce640 1 -- 172.21.15.32:0/3852717540 shutdown_connections 2023-12-20T04:29:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf879ce640 1 --2- 172.21.15.32:0/3852717540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf80164670 0x7faf800ca1a0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf879ce640 1 -- 172.21.15.32:0/3852717540 >> 172.21.15.32:0/3852717540 conn(0x7faf8006a510 msgr2=0x7faf800ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf879ce640 1 -- 172.21.15.32:0/3852717540 shutdown_connections 2023-12-20T04:29:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf879ce640 1 -- 172.21.15.32:0/3852717540 wait complete. 2023-12-20T04:29:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf879ce640 1 Processor -- start 2023-12-20T04:29:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.066+0000 7faf879ce640 1 -- start start 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf879ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf8006da20 0x7faf80143a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf879ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf80070230 0x7faf80158700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf879ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf80164670 0x7faf80159500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf879ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf80143f80 con 0x7faf80070230 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf879ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf800c6f70 con 0x7faf80164670 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf879ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf800c6360 con 0x7faf8006da20 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf84f69640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf80070230 0x7faf80158700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf84f69640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf80070230 0x7faf80158700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34462/0 (socket says 172.21.15.32:34462) 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf84f69640 1 -- 172.21.15.32:0/152239310 learned_addr learned my addr 172.21.15.32:0/152239310 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf8576a640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf8006da20 0x7faf80143a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf85f6b640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf80164670 0x7faf80159500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf84f69640 1 -- 172.21.15.32:0/152239310 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf8006da20 msgr2=0x7faf80143a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf84f69640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf8006da20 0x7faf80143a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf84f69640 1 -- 172.21.15.32:0/152239310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf80164670 msgr2=0x7faf80159500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf84f69640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf80164670 0x7faf80159500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf84f69640 1 -- 172.21.15.32:0/152239310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf7c009420 con 0x7faf80070230 2023-12-20T04:29:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf84f69640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf80070230 0x7faf80158700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c3f0 tx=0x7faf7000c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf75ffb640 1 -- 172.21.15.32:0/152239310 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf70005cf0 con 0x7faf80070230 2023-12-20T04:29:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf75ffb640 1 -- 172.21.15.32:0/152239310 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf70005e50 con 0x7faf80070230 2023-12-20T04:29:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf75ffb640 1 -- 172.21.15.32:0/152239310 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf700072f0 con 0x7faf80070230 2023-12-20T04:29:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf879ce640 1 -- 172.21.15.32:0/152239310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf800c6640 con 0x7faf80070230 2023-12-20T04:29:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf879ce640 1 -- 172.21.15.32:0/152239310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf800c6c00 con 0x7faf80070230 2023-12-20T04:29:49.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf879ce640 1 -- 172.21.15.32:0/152239310 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf8016cee0 con 0x7faf80070230 2023-12-20T04:29:49.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.070+0000 7faf75ffb640 1 -- 172.21.15.32:0/152239310 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 85083+0+0 (secure 0 0 0) 0x7faf70007450 con 0x7faf80070230 2023-12-20T04:29:49.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.074+0000 7faf75ffb640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf64065900 0x7faf64067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.074+0000 7faf75ffb640 1 -- 172.21.15.32:0/152239310 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7faf70082630 con 0x7faf80070230 2023-12-20T04:29:49.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.074+0000 7faf8576a640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf64065900 0x7faf64067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.074+0000 7faf8576a640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf64065900 0x7faf64067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0ad000 tx=0x7faf7c0a6000 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:49.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.078+0000 7faf75ffb640 1 -- 172.21.15.32:0/152239310 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf70050160 con 0x7faf80070230 2023-12-20T04:29:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.254+0000 7faf879ce640 1 -- 172.21.15.32:0/152239310 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7faf80164a50 con 0x7faf64065900 2023-12-20T04:29:49.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.318+0000 7faf75ffb640 1 -- 172.21.15.32:0/152239310 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7faf7c0ab740 con 0x7faf64065900 2023-12-20T04:29:49.322 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1/dir1 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.318+0000 7faf879ce640 1 -- 172.21.15.32:0/152239310 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf64065900 msgr2=0x7faf64067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.318+0000 7faf879ce640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf64065900 0x7faf64067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7faf7c0ad000 tx=0x7faf7c0a6000 comp rx=0 tx=0).stop 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.318+0000 7faf879ce640 1 -- 172.21.15.32:0/152239310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf80070230 msgr2=0x7faf80158700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.318+0000 7faf879ce640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf80070230 0x7faf80158700 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7faf7000c3f0 tx=0x7faf7000c8b0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.318+0000 7faf879ce640 1 -- 172.21.15.32:0/152239310 shutdown_connections 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.318+0000 7faf879ce640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf64065900 0x7faf64067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.318+0000 7faf879ce640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf80164670 0x7faf80159500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.318+0000 7faf879ce640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf80070230 0x7faf80158700 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.318+0000 7faf879ce640 1 --2- 172.21.15.32:0/152239310 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf8006da20 0x7faf80143a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.318+0000 7faf879ce640 1 -- 172.21.15.32:0/152239310 >> 172.21.15.32:0/152239310 conn(0x7faf8006a510 msgr2=0x7faf800c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.322+0000 7faf879ce640 1 -- 172.21.15.32:0/152239310 shutdown_connections 2023-12-20T04:29:49.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.322+0000 7faf879ce640 1 -- 172.21.15.32:0/152239310 wait complete. 2023-12-20T04:29:49.343 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir1 2023-12-20T04:29:49.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:49.343 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir2) 2023-12-20T04:29:49.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir2 1M 2023-12-20T04:29:49.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.542+0000 7fdbff12c640 1 Processor -- start 2023-12-20T04:29:49.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.542+0000 7fdbff12c640 1 -- start start 2023-12-20T04:29:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.542+0000 7fdbff12c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdbf815abb0 0x7fdbf80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.542+0000 7fdbff12c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbf80b0cb0 0x7fdbf815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.542+0000 7fdbff12c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdbf8070230 0x7fdbf80706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.542+0000 7fdbff12c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf806e160 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.542+0000 7fdbff12c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdbf8070be0 con 0x7fdbf8070230 2023-12-20T04:29:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.542+0000 7fdbff12c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf815b6a0 con 0x7fdbf815abb0 2023-12-20T04:29:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbf80b0cb0 0x7fdbf815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdbf80b0cb0 0x7fdbf815bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34470/0 (socket says 172.21.15.32:34470) 2023-12-20T04:29:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf7fff640 1 -- 172.21.15.32:0/502961214 learned_addr learned my addr 172.21.15.32:0/502961214 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbfcec8640 1 --2- 172.21.15.32:0/502961214 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdbf815abb0 0x7fdbf80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbfd6c9640 1 --2- 172.21.15.32:0/502961214 >> v2:172.21.15.39:3300/0 conn(0x7fdbf8070230 0x7fdbf80706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf7fff640 1 -- 172.21.15.32:0/502961214 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdbf815abb0 msgr2=0x7fdbf80caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:49.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf7fff640 1 --2- 172.21.15.32:0/502961214 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdbf815abb0 0x7fdbf80caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf7fff640 1 -- 172.21.15.32:0/502961214 >> v2:172.21.15.39:3300/0 conn(0x7fdbf8070230 msgr2=0x7fdbf80706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:49.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf7fff640 1 --2- 172.21.15.32:0/502961214 >> v2:172.21.15.39:3300/0 conn(0x7fdbf8070230 0x7fdbf80706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf7fff640 1 -- 172.21.15.32:0/502961214 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbf815c070 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf7fff640 1 --2- 172.21.15.32:0/502961214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf80b0cb0 0x7fdbf815bb30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80b4470 tx=0x7fdbe401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b96bc24ebd976339 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:49.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf77fe640 1 -- 172.21.15.32:0/502961214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40aa620 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf77fe640 1 -- 172.21.15.32:0/502961214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe40aa780 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.546+0000 7fdbf77fe640 1 -- 172.21.15.32:0/502961214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40aab10 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 -- 172.21.15.32:0/502961214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf80b0cb0 msgr2=0x7fdbf815bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 --2- 172.21.15.32:0/502961214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf80b0cb0 0x7fdbf815bb30 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fdbf80b4470 tx=0x7fdbe401c500 comp rx=0 tx=0).stop 2023-12-20T04:29:49.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 -- 172.21.15.32:0/502961214 shutdown_connections 2023-12-20T04:29:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 --2- 172.21.15.32:0/502961214 >> v2:172.21.15.39:3300/0 conn(0x7fdbf8070230 0x7fdbf80706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 --2- 172.21.15.32:0/502961214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf80b0cb0 0x7fdbf815bb30 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 --2- 172.21.15.32:0/502961214 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdbf815abb0 0x7fdbf80caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 -- 172.21.15.32:0/502961214 >> 172.21.15.32:0/502961214 conn(0x7fdbf806a510 msgr2=0x7fdbf80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 -- 172.21.15.32:0/502961214 shutdown_connections 2023-12-20T04:29:49.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 -- 172.21.15.32:0/502961214 wait complete. 2023-12-20T04:29:49.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 Processor -- start 2023-12-20T04:29:49.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 -- start start 2023-12-20T04:29:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdbf8070230 0x7fdbf814f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf80b0cb0 0x7fdbf80cd8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf815abb0 0x7fdbf81597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80cd210 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80cc970 con 0x7fdbf815abb0 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbff12c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdbf80ccc50 con 0x7fdbf8070230 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf80b0cb0 0x7fdbf80cd8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf80b0cb0 0x7fdbf80cd8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34484/0 (socket says 172.21.15.32:34484) 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbf7fff640 1 -- 172.21.15.32:0/4009003662 learned_addr learned my addr 172.21.15.32:0/4009003662 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbfcec8640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdbf8070230 0x7fdbf814f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.550+0000 7fdbfd6c9640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf815abb0 0x7fdbf81597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf7fff640 1 -- 172.21.15.32:0/4009003662 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdbf8070230 msgr2=0x7fdbf814f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf7fff640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdbf8070230 0x7fdbf814f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf7fff640 1 -- 172.21.15.32:0/4009003662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf815abb0 msgr2=0x7fdbf81597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf7fff640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf815abb0 0x7fdbf81597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf7fff640 1 -- 172.21.15.32:0/4009003662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbe40094d0 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf7fff640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf80b0cb0 0x7fdbf80cd8f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdbe401abb0 tx=0x7fdbe401ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:49.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf57fa640 1 -- 172.21.15.32:0/4009003662 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40aad80 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf57fa640 1 -- 172.21.15.32:0/4009003662 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbe40bd3f0 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbff12c640 1 -- 172.21.15.32:0/4009003662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdbf80ccf30 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf57fa640 1 -- 172.21.15.32:0/4009003662 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbe40bd7a0 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbff12c640 1 -- 172.21.15.32:0/4009003662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdbf814f8d0 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbff12c640 1 -- 172.21.15.32:0/4009003662 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdbf8142f00 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf57fa640 1 -- 172.21.15.32:0/4009003662 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 8) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fdbe40b33f0 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf57fa640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbdc065900 0x7fdbdc067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:49.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbf57fa640 1 -- 172.21.15.32:0/4009003662 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fdbe411d350 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.554+0000 7fdbfcec8640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbdc065900 0x7fdbdc067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:49.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.558+0000 7fdbfcec8640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbdc065900 0x7fdbdc067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009b60 tx=0x7fdbec009190 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:49.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.558+0000 7fdbf57fa640 1 -- 172.21.15.32:0/4009003662 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdbf8142f00 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.738+0000 7fdbff12c640 1 -- 172.21.15.32:0/4009003662 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir2", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdbf815b330 con 0x7fdbdc065900 2023-12-20T04:29:49.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.750+0000 7fdbf57fa640 1 -- 172.21.15.32:0/4009003662 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 9) v1 ==== 85348+0+0 (secure 0 0 0) 0x7fdbe40eb420 con 0x7fdbf80b0cb0 2023-12-20T04:29:49.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbf57fa640 1 -- 172.21.15.32:0/4009003662 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fdbec011ac0 con 0x7fdbdc065900 2023-12-20T04:29:49.812 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1/dir2 2023-12-20T04:29:49.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 -- 172.21.15.32:0/4009003662 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbdc065900 msgr2=0x7fdbdc067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbdc065900 0x7fdbdc067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdbec009b60 tx=0x7fdbec009190 comp rx=0 tx=0).stop 2023-12-20T04:29:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 -- 172.21.15.32:0/4009003662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf80b0cb0 msgr2=0x7fdbf80cd8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf80b0cb0 0x7fdbf80cd8f0 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fdbe401abb0 tx=0x7fdbe401ad40 comp rx=0 tx=0).stop 2023-12-20T04:29:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 -- 172.21.15.32:0/4009003662 shutdown_connections 2023-12-20T04:29:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbdc065900 0x7fdbdc067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdbf815abb0 0x7fdbf81597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdbf80b0cb0 0x7fdbf80cd8f0 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 --2- 172.21.15.32:0/4009003662 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdbf8070230 0x7fdbf814f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 -- 172.21.15.32:0/4009003662 >> 172.21.15.32:0/4009003662 conn(0x7fdbf806a510 msgr2=0x7fdbf80c3970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 -- 172.21.15.32:0/4009003662 shutdown_connections 2023-12-20T04:29:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:49.810+0000 7fdbff12c640 1 -- 172.21.15.32:0/4009003662 wait complete. 2023-12-20T04:29:49.838 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir2 2023-12-20T04:29:49.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:49.838 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir3) 2023-12-20T04:29:49.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir3 1M 2023-12-20T04:29:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.058+0000 7f429e339640 1 Processor -- start 2023-12-20T04:29:50.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.058+0000 7f429e339640 1 -- start start 2023-12-20T04:29:50.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.058+0000 7f429e339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f429815abb0 0x7f42980caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.058+0000 7f429e339640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429815b6a0 con 0x7f4298070230 2023-12-20T04:29:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.058+0000 7f429e339640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f429806d8c0 con 0x7f42980b0cb0 2023-12-20T04:29:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.058+0000 7f429e339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f429806e160 con 0x7f429815abb0 2023-12-20T04:29:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f429c8d6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4298070230 0x7f429806c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40280/0 (socket says 172.21.15.32:40280) 2023-12-20T04:29:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f429c8d6640 1 -- 172.21.15.32:0/2387935130 learned_addr learned my addr 172.21.15.32:0/2387935130 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:50.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f4297fff640 1 --2- 172.21.15.32:0/2387935130 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f429815abb0 0x7f42980caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f4296ffd640 1 -- 172.21.15.32:0/2387935130 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3481938248 0 0) 0x7f429815b6a0 con 0x7f4298070230 2023-12-20T04:29:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f4296ffd640 1 -- 172.21.15.32:0/2387935130 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4280003530 con 0x7f4298070230 2023-12-20T04:29:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f4297fff640 1 -- 172.21.15.32:0/2387935130 >> v1:172.21.15.39:6789/0 conn(0x7f42980b0cb0 legacy=0x7f429815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f4297fff640 1 -- 172.21.15.32:0/2387935130 >> v1:172.21.15.32:6789/0 conn(0x7f4298070230 legacy=0x7f429806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f4297fff640 1 -- 172.21.15.32:0/2387935130 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f429806dba0 con 0x7f429815abb0 2023-12-20T04:29:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f4297fff640 1 --2- 172.21.15.32:0/2387935130 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f429815abb0 0x7f42980caaf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f42880097f0 tx=0x7f428801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4d99d877283ecd98 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f4296ffd640 1 -- 172.21.15.32:0/2387935130 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880aa620 con 0x7f429815abb0 2023-12-20T04:29:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f4296ffd640 1 -- 172.21.15.32:0/2387935130 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42880aa780 con 0x7f429815abb0 2023-12-20T04:29:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.062+0000 7f4296ffd640 1 -- 172.21.15.32:0/2387935130 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42880aab10 con 0x7f429815abb0 2023-12-20T04:29:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 -- 172.21.15.32:0/2387935130 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f429815abb0 msgr2=0x7f42980caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 --2- 172.21.15.32:0/2387935130 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f429815abb0 0x7f42980caaf0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f42880097f0 tx=0x7f428801c4c0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 -- 172.21.15.32:0/2387935130 shutdown_connections 2023-12-20T04:29:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 --2- 172.21.15.32:0/2387935130 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f429815abb0 0x7f42980caaf0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 -- 172.21.15.32:0/2387935130 >> 172.21.15.32:0/2387935130 conn(0x7f429806a510 msgr2=0x7f42980ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 -- 172.21.15.32:0/2387935130 shutdown_connections 2023-12-20T04:29:50.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 -- 172.21.15.32:0/2387935130 wait complete. 2023-12-20T04:29:50.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 Processor -- start 2023-12-20T04:29:50.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 -- start start 2023-12-20T04:29:50.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4298070230 0x7f4298150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42980b0cb0 0x7f4298156760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f429815abb0 0x7f4298156ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4298159ef0 con 0x7f42980b0cb0 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4298137bd0 con 0x7f4298070230 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f429e339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42981571e0 con 0x7f429815abb0 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f42977fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42980b0cb0 0x7f4298156760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f42977fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42980b0cb0 0x7f4298156760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34506/0 (socket says 172.21.15.32:34506) 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.066+0000 7f42977fe640 1 -- 172.21.15.32:0/4284769540 learned_addr learned my addr 172.21.15.32:0/4284769540 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f429c8d6640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f429815abb0 0x7f4298156ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f4297fff640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4298070230 0x7f4298150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f42977fe640 1 -- 172.21.15.32:0/4284769540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f429815abb0 msgr2=0x7f4298156ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f42977fe640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f429815abb0 0x7f4298156ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f42977fe640 1 -- 172.21.15.32:0/4284769540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4298070230 msgr2=0x7f4298150fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f42977fe640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4298070230 0x7f4298150fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f42977fe640 1 -- 172.21.15.32:0/4284769540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4288009420 con 0x7f42980b0cb0 2023-12-20T04:29:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f42977fe640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42980b0cb0 0x7f4298156760 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c530 tx=0x7f428c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f4294ff9640 1 -- 172.21.15.32:0/4284769540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c005220 con 0x7f42980b0cb0 2023-12-20T04:29:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f4294ff9640 1 -- 172.21.15.32:0/4284769540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f428c005380 con 0x7f42980b0cb0 2023-12-20T04:29:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f4294ff9640 1 -- 172.21.15.32:0/4284769540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f428c007270 con 0x7f42980b0cb0 2023-12-20T04:29:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f429e339640 1 -- 172.21.15.32:0/4284769540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f429815a310 con 0x7f42980b0cb0 2023-12-20T04:29:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f429e339640 1 -- 172.21.15.32:0/4284769540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4298159550 con 0x7f42980b0cb0 2023-12-20T04:29:50.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f429e339640 1 -- 172.21.15.32:0/4284769540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f429813e210 con 0x7f42980b0cb0 2023-12-20T04:29:50.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f4294ff9640 1 -- 172.21.15.32:0/4284769540 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 85348+0+0 (secure 0 0 0) 0x7f428c0073d0 con 0x7f42980b0cb0 2023-12-20T04:29:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f4294ff9640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4284065bd0 0x7f4284068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.070+0000 7f4294ff9640 1 -- 172.21.15.32:0/4284769540 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f428c082cf0 con 0x7f42980b0cb0 2023-12-20T04:29:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.074+0000 7f4297fff640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4284065bd0 0x7f4284068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.074+0000 7f4297fff640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4284065bd0 0x7f4284068090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f42880ae000 tx=0x7f42880b5040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:50.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.082+0000 7f4294ff9640 1 -- 172.21.15.32:0/4284769540 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f428c087070 con 0x7f42980b0cb0 2023-12-20T04:29:50.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.258+0000 7f429e339640 1 -- 172.21.15.32:0/4284769540 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir3", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f429804e070 con 0x7f4284065bd0 2023-12-20T04:29:50.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f4294ff9640 1 -- 172.21.15.32:0/4284769540 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f429804e070 con 0x7f4284065bd0 2023-12-20T04:29:50.320 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1/dir3 2023-12-20T04:29:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 -- 172.21.15.32:0/4284769540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4284065bd0 msgr2=0x7f4284068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4284065bd0 0x7f4284068090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f42880ae000 tx=0x7f42880b5040 comp rx=0 tx=0).stop 2023-12-20T04:29:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 -- 172.21.15.32:0/4284769540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42980b0cb0 msgr2=0x7f4298156760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42980b0cb0 0x7f4298156760 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f428c00c530 tx=0x7f428c00c9f0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 -- 172.21.15.32:0/4284769540 shutdown_connections 2023-12-20T04:29:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4284065bd0 0x7f4284068090 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f429815abb0 0x7f4298156ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42980b0cb0 0x7f4298156760 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 --2- 172.21.15.32:0/4284769540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4298070230 0x7f4298150fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 -- 172.21.15.32:0/4284769540 >> 172.21.15.32:0/4284769540 conn(0x7f429806a510 msgr2=0x7f42980b2a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:50.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 -- 172.21.15.32:0/4284769540 shutdown_connections 2023-12-20T04:29:50.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.318+0000 7f429e339640 1 -- 172.21.15.32:0/4284769540 wait complete. 2023-12-20T04:29:50.342 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir3 2023-12-20T04:29:50.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:50.342 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir4) 2023-12-20T04:29:50.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir4 1M 2023-12-20T04:29:50.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ef6d1640 1 Processor -- start 2023-12-20T04:29:50.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ef6d1640 1 -- start start 2023-12-20T04:29:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ef6d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 0x7f78e80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ef6d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f78e8070230 0x7f78e80706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ef6d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f78e815b6a0 con 0x7f78e815abb0 2023-12-20T04:29:50.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ef6d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f78e806e160 con 0x7f78e8070230 2023-12-20T04:29:50.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ef6d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8070be0 con 0x7f78e80b0cb0 2023-12-20T04:29:50.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ecc6c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 0x7f78e80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:50.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78edc6e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f78e8070230 0x7f78e80706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ed46d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f78e815abb0 0x7f78e815bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:56690/0 (socket says 172.21.15.32:56690) 2023-12-20T04:29:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ecc6c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 0x7f78e80caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49360/0 (socket says 172.21.15.32:49360) 2023-12-20T04:29:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ed46d640 1 -- 172.21.15.32:0/2395868470 learned_addr learned my addr 172.21.15.32:0/2395868470 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ecc6c640 1 -- 172.21.15.32:0/2395868470 >> v2:172.21.15.39:3300/0 conn(0x7f78e8070230 msgr2=0x7f78e80706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ecc6c640 1 --2- 172.21.15.32:0/2395868470 >> v2:172.21.15.39:3300/0 conn(0x7f78e8070230 0x7f78e80706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ecc6c640 1 -- 172.21.15.32:0/2395868470 >> v1:172.21.15.39:6789/0 conn(0x7f78e815abb0 legacy=0x7f78e815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78dffff640 1 -- 172.21.15.32:0/2395868470 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1408715475 0 0) 0x7f78e815b6a0 con 0x7f78e815abb0 2023-12-20T04:29:50.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.538+0000 7f78ecc6c640 1 -- 172.21.15.32:0/2395868470 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78e806d8c0 con 0x7f78e80b0cb0 2023-12-20T04:29:50.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78ecc6c640 1 --2- 172.21.15.32:0/2395868470 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 0x7f78e80caaf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f78d80097c0 tx=0x7f78d801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dc56ca679aad2c91 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:50.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78dffff640 1 -- 172.21.15.32:0/2395868470 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d80ab680 con 0x7f78e80b0cb0 2023-12-20T04:29:50.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78dffff640 1 -- 172.21.15.32:0/2395868470 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78d80ab7e0 con 0x7f78e80b0cb0 2023-12-20T04:29:50.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78dffff640 1 -- 172.21.15.32:0/2395868470 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78d80abb70 con 0x7f78e80b0cb0 2023-12-20T04:29:50.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78ef6d1640 1 -- 172.21.15.32:0/2395868470 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 msgr2=0x7f78e80caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78ef6d1640 1 --2- 172.21.15.32:0/2395868470 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 0x7f78e80caaf0 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f78d80097c0 tx=0x7f78d801c480 comp rx=0 tx=0).stop 2023-12-20T04:29:50.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78ef6d1640 1 -- 172.21.15.32:0/2395868470 shutdown_connections 2023-12-20T04:29:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78ef6d1640 1 --2- 172.21.15.32:0/2395868470 >> v2:172.21.15.39:3300/0 conn(0x7f78e8070230 0x7f78e80706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78ef6d1640 1 --2- 172.21.15.32:0/2395868470 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 0x7f78e80caaf0 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78ef6d1640 1 -- 172.21.15.32:0/2395868470 >> 172.21.15.32:0/2395868470 conn(0x7f78e806a510 msgr2=0x7f78e80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:50.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78ef6d1640 1 -- 172.21.15.32:0/2395868470 shutdown_connections 2023-12-20T04:29:50.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.542+0000 7f78ef6d1640 1 -- 172.21.15.32:0/2395868470 wait complete. 2023-12-20T04:29:50.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ef6d1640 1 Processor -- start 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ef6d1640 1 -- start start 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ef6d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e8070230 0x7f78e813e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ef6d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 0x7f78e8140840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ef6d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78e815abb0 0x7f78e8143b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ef6d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e813f350 con 0x7f78e8070230 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ef6d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f78e813fd60 con 0x7f78e815abb0 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ef6d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f78e8140040 con 0x7f78e80b0cb0 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ed46d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e8070230 0x7f78e813e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ed46d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e8070230 0x7f78e813e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34510/0 (socket says 172.21.15.32:34510) 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ed46d640 1 -- 172.21.15.32:0/1649733893 learned_addr learned my addr 172.21.15.32:0/1649733893 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ecc6c640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 0x7f78e8140840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:50.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78edc6e640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78e815abb0 0x7f78e8143b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ed46d640 1 -- 172.21.15.32:0/1649733893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 msgr2=0x7f78e8140840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ed46d640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 0x7f78e8140840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ed46d640 1 -- 172.21.15.32:0/1649733893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78e815abb0 msgr2=0x7f78e8143b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ed46d640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78e815abb0 0x7f78e8143b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ed46d640 1 -- 172.21.15.32:0/1649733893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78d8009420 con 0x7f78e8070230 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ed46d640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e8070230 0x7f78e813e620 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f78e400c480 tx=0x7f78e400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ddffb640 1 -- 172.21.15.32:0/1649733893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e40074e0 con 0x7f78e8070230 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ddffb640 1 -- 172.21.15.32:0/1649733893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78e4007660 con 0x7f78e8070230 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ddffb640 1 -- 172.21.15.32:0/1649733893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78e4004a30 con 0x7f78e8070230 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.546+0000 7f78ef6d1640 1 -- 172.21.15.32:0/1649733893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78e8140320 con 0x7f78e8070230 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.550+0000 7f78ef6d1640 1 -- 172.21.15.32:0/1649733893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f78e8140d80 con 0x7f78e8070230 2023-12-20T04:29:50.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.550+0000 7f78ef6d1640 1 -- 172.21.15.32:0/1649733893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f78e813fbc0 con 0x7f78e8070230 2023-12-20T04:29:50.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.550+0000 7f78ddffb640 1 -- 172.21.15.32:0/1649733893 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 85348+0+0 (secure 0 0 0) 0x7f78e4004b90 con 0x7f78e8070230 2023-12-20T04:29:50.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.550+0000 7f78ddffb640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f78cc065bd0 0x7f78cc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:50.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.550+0000 7f78ddffb640 1 -- 172.21.15.32:0/1649733893 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f78e4082590 con 0x7f78e8070230 2023-12-20T04:29:50.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.550+0000 7f78ecc6c640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f78cc065bd0 0x7f78cc068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:50.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.550+0000 7f78ecc6c640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f78cc065bd0 0x7f78cc068090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f78d80af000 tx=0x7f78d80a7000 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:50.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.554+0000 7f78ddffb640 1 -- 172.21.15.32:0/1649733893 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f78e813fbc0 con 0x7f78e8070230 2023-12-20T04:29:50.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.730+0000 7f78ef6d1640 1 -- 172.21.15.32:0/1649733893 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir4", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f78e813fed0 con 0x7f78cc065bd0 2023-12-20T04:29:50.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.794+0000 7f78ddffb640 1 -- 172.21.15.32:0/1649733893 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f78e813fed0 con 0x7f78cc065bd0 2023-12-20T04:29:50.801 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1/dir4 2023-12-20T04:29:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 -- 172.21.15.32:0/1649733893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f78cc065bd0 msgr2=0x7f78cc068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f78cc065bd0 0x7f78cc068090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f78d80af000 tx=0x7f78d80a7000 comp rx=0 tx=0).stop 2023-12-20T04:29:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 -- 172.21.15.32:0/1649733893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e8070230 msgr2=0x7f78e813e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e8070230 0x7f78e813e620 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f78e400c480 tx=0x7f78e400c940 comp rx=0 tx=0).stop 2023-12-20T04:29:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 -- 172.21.15.32:0/1649733893 shutdown_connections 2023-12-20T04:29:50.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f78cc065bd0 0x7f78cc068090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f78e815abb0 0x7f78e8143b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f78e80b0cb0 0x7f78e8140840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 --2- 172.21.15.32:0/1649733893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f78e8070230 0x7f78e813e620 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 -- 172.21.15.32:0/1649733893 >> 172.21.15.32:0/1649733893 conn(0x7f78e806a510 msgr2=0x7f78e80c4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 -- 172.21.15.32:0/1649733893 shutdown_connections 2023-12-20T04:29:50.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:50.798+0000 7f78ef6d1640 1 -- 172.21.15.32:0/1649733893 wait complete. 2023-12-20T04:29:50.820 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir4 2023-12-20T04:29:50.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:50.821 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir5) 2023-12-20T04:29:50.832 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir5 1M 2023-12-20T04:29:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01ba22d640 1 Processor -- start 2023-12-20T04:29:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01ba22d640 1 -- start start 2023-12-20T04:29:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01ba22d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01b4157480 0x7f01b40c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01ba22d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b40cde20 con 0x7f01b40b0760 2023-12-20T04:29:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01ba22d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f01b40c8e30 con 0x7f01b40cd4c0 2023-12-20T04:29:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01ba22d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f01b415e480 con 0x7f01b4157480 2023-12-20T04:29:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01b37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01b4157480 0x7f01b40c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01b2ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f01b40b0760 0x7f01b415d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40292/0 (socket says 172.21.15.32:40292) 2023-12-20T04:29:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01b37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f01b4157480 0x7f01b40c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34518/0 (socket says 172.21.15.32:34518) 2023-12-20T04:29:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01b2ffd640 1 -- 172.21.15.32:0/3715161058 learned_addr learned my addr 172.21.15.32:0/3715161058 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01b37fe640 1 -- 172.21.15.32:0/3715161058 >> v1:172.21.15.39:6789/0 conn(0x7f01b40cd4c0 legacy=0x7f01b415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01b37fe640 1 -- 172.21.15.32:0/3715161058 >> v1:172.21.15.32:6789/0 conn(0x7f01b40b0760 legacy=0x7f01b415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01b37fe640 1 -- 172.21.15.32:0/3715161058 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01b40ce070 con 0x7f01b4157480 2023-12-20T04:29:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.018+0000 7f01b27fc640 1 -- 172.21.15.32:0/3715161058 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 620495974 0 0) 0x7f01b40cde20 con 0x7f01b40b0760 2023-12-20T04:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.022+0000 7f01b37fe640 1 --2- 172.21.15.32:0/3715161058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b4157480 0x7f01b40c9840 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f01a40097c0 tx=0x7f01a401c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcef6f1d23150605 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.022+0000 7f01b27fc640 1 -- 172.21.15.32:0/3715161058 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a40ac620 con 0x7f01b4157480 2023-12-20T04:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.022+0000 7f01b27fc640 1 -- 172.21.15.32:0/3715161058 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01a40ac780 con 0x7f01b4157480 2023-12-20T04:29:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.022+0000 7f01b27fc640 1 -- 172.21.15.32:0/3715161058 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01a40acb10 con 0x7f01b4157480 2023-12-20T04:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.022+0000 7f01ba22d640 1 -- 172.21.15.32:0/3715161058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b4157480 msgr2=0x7f01b40c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.022+0000 7f01ba22d640 1 --2- 172.21.15.32:0/3715161058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b4157480 0x7f01b40c9840 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f01a40097c0 tx=0x7f01a401c3c0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.022+0000 7f01ba22d640 1 -- 172.21.15.32:0/3715161058 shutdown_connections 2023-12-20T04:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.022+0000 7f01ba22d640 1 --2- 172.21.15.32:0/3715161058 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b4157480 0x7f01b40c9840 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.022+0000 7f01ba22d640 1 -- 172.21.15.32:0/3715161058 >> 172.21.15.32:0/3715161058 conn(0x7f01b406a510 msgr2=0x7f01b40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.022+0000 7f01ba22d640 1 -- 172.21.15.32:0/3715161058 shutdown_connections 2023-12-20T04:29:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.022+0000 7f01ba22d640 1 -- 172.21.15.32:0/3715161058 wait complete. 2023-12-20T04:29:51.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01ba22d640 1 Processor -- start 2023-12-20T04:29:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01ba22d640 1 -- start start 2023-12-20T04:29:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01ba22d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01b40b0760 0x7f01b40c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01ba22d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01b40cd4c0 0x7f01b4070c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01ba22d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b4157480 0x7f01b41440b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01ba22d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c6fc0 con 0x7f01b4157480 2023-12-20T04:29:51.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01ba22d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f01b4071190 con 0x7f01b40b0760 2023-12-20T04:29:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01ba22d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f01b40c5f20 con 0x7f01b40cd4c0 2023-12-20T04:29:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b4157480 0x7f01b41440b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b4157480 0x7f01b41440b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34522/0 (socket says 172.21.15.32:34522) 2023-12-20T04:29:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01b3fff640 1 -- 172.21.15.32:0/4181780464 learned_addr learned my addr 172.21.15.32:0/4181780464 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:51.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01b37fe640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01b40b0760 0x7f01b40c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01b2ffd640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01b40cd4c0 0x7f01b4070c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01b3fff640 1 -- 172.21.15.32:0/4181780464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01b40cd4c0 msgr2=0x7f01b4070c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01b3fff640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01b40cd4c0 0x7f01b4070c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01b3fff640 1 -- 172.21.15.32:0/4181780464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01b40b0760 msgr2=0x7f01b40c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01b3fff640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01b40b0760 0x7f01b40c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01b3fff640 1 -- 172.21.15.32:0/4181780464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f01a4009420 con 0x7f01b4157480 2023-12-20T04:29:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01b3fff640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b4157480 0x7f01b41440b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00c770 tx=0x7f01ac00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f019bfff640 1 -- 172.21.15.32:0/4181780464 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac003b80 con 0x7f01b4157480 2023-12-20T04:29:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f019bfff640 1 -- 172.21.15.32:0/4181780464 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f01ac003ce0 con 0x7f01b4157480 2023-12-20T04:29:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f019bfff640 1 -- 172.21.15.32:0/4181780464 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f01ac00aa90 con 0x7f01b4157480 2023-12-20T04:29:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01ba22d640 1 -- 172.21.15.32:0/4181780464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f01b40c6200 con 0x7f01b4157480 2023-12-20T04:29:51.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.026+0000 7f01ba22d640 1 -- 172.21.15.32:0/4181780464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f01b40c67c0 con 0x7f01b4157480 2023-12-20T04:29:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.030+0000 7f01ba22d640 1 -- 172.21.15.32:0/4181780464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f01b404e000 con 0x7f01b4157480 2023-12-20T04:29:51.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.030+0000 7f019bfff640 1 -- 172.21.15.32:0/4181780464 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 85348+0+0 (secure 0 0 0) 0x7f01ac00abf0 con 0x7f01b4157480 2023-12-20T04:29:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.030+0000 7f019bfff640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01a0065bd0 0x7f01a0068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.030+0000 7f019bfff640 1 -- 172.21.15.32:0/4181780464 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f01ac083180 con 0x7f01b4157480 2023-12-20T04:29:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.030+0000 7f01b37fe640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01a0065bd0 0x7f01a0068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.030+0000 7f01b37fe640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01a0065bd0 0x7f01a0068090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f01a40ae000 tx=0x7f01a40a7000 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:51.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.034+0000 7f019bfff640 1 -- 172.21.15.32:0/4181780464 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f01ac0509d0 con 0x7f01b4157480 2023-12-20T04:29:51.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.210+0000 7f01ba22d640 1 -- 172.21.15.32:0/4181780464 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir5", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f01b40c9360 con 0x7f01a0065bd0 2023-12-20T04:29:51.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.266+0000 7f019bfff640 1 -- 172.21.15.32:0/4181780464 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f01a40040c0 con 0x7f01a0065bd0 2023-12-20T04:29:51.271 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1/dir5 2023-12-20T04:29:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 -- 172.21.15.32:0/4181780464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01a0065bd0 msgr2=0x7f01a0068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01a0065bd0 0x7f01a0068090 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f01a40ae000 tx=0x7f01a40a7000 comp rx=0 tx=0).stop 2023-12-20T04:29:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 -- 172.21.15.32:0/4181780464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b4157480 msgr2=0x7f01b41440b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b4157480 0x7f01b41440b0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f01ac00c770 tx=0x7f01ac00cc30 comp rx=0 tx=0).stop 2023-12-20T04:29:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 -- 172.21.15.32:0/4181780464 shutdown_connections 2023-12-20T04:29:51.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f01a0065bd0 0x7f01a0068090 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f01b4157480 0x7f01b41440b0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f01b40cd4c0 0x7f01b4070c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 --2- 172.21.15.32:0/4181780464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f01b40b0760 0x7f01b40c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 -- 172.21.15.32:0/4181780464 >> 172.21.15.32:0/4181780464 conn(0x7f01b406a510 msgr2=0x7f01b40c49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 -- 172.21.15.32:0/4181780464 shutdown_connections 2023-12-20T04:29:51.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.270+0000 7f01ba22d640 1 -- 172.21.15.32:0/4181780464 wait complete. 2023-12-20T04:29:51.293 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir5 2023-12-20T04:29:51.293 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:51.294 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir6) 2023-12-20T04:29:51.307 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir6 1M 2023-12-20T04:29:51.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe249e66640 1 Processor -- start 2023-12-20T04:29:51.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe249e66640 1 -- start start 2023-12-20T04:29:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe249e66640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe24406d170 0x7fe2440cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe249e66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24406a1f0 con 0x7fe244065d40 2023-12-20T04:29:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe249e66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe24406a440 con 0x7fe2440cb050 2023-12-20T04:29:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe249e66640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe24406a720 con 0x7fe24406d170 2023-12-20T04:29:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe243fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe24406d170 0x7fe2440cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe243fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe24406d170 0x7fe2440cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34536/0 (socket says 172.21.15.32:34536) 2023-12-20T04:29:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe23bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe244065d40 0x7fe2440c9720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40296/0 (socket says 172.21.15.32:40296) 2023-12-20T04:29:51.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe243fff640 1 -- 172.21.15.32:0/3879538726 learned_addr learned my addr 172.21.15.32:0/3879538726 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe243fff640 1 -- 172.21.15.32:0/3879538726 >> v1:172.21.15.39:6789/0 conn(0x7fe2440cb050 legacy=0x7fe24406ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe243fff640 1 -- 172.21.15.32:0/3879538726 >> v1:172.21.15.32:6789/0 conn(0x7fe244065d40 legacy=0x7fe2440c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe243fff640 1 -- 172.21.15.32:0/3879538726 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe24406aa00 con 0x7fe24406d170 2023-12-20T04:29:51.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.494+0000 7fe242ffd640 1 -- 172.21.15.32:0/3879538726 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1727898701 0 0) 0x7fe24406a1f0 con 0x7fe244065d40 2023-12-20T04:29:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.498+0000 7fe243fff640 1 --2- 172.21.15.32:0/3879538726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24406d170 0x7fe2440cb430 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009e50 tx=0x7fe23c01c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=505b990a19cbb3ea server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.498+0000 7fe242ffd640 1 -- 172.21.15.32:0/3879538726 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0a58a0 con 0x7fe24406d170 2023-12-20T04:29:51.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.498+0000 7fe242ffd640 1 -- 172.21.15.32:0/3879538726 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c0a5a00 con 0x7fe24406d170 2023-12-20T04:29:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.498+0000 7fe242ffd640 1 -- 172.21.15.32:0/3879538726 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0a5d90 con 0x7fe24406d170 2023-12-20T04:29:51.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.498+0000 7fe249e66640 1 -- 172.21.15.32:0/3879538726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24406d170 msgr2=0x7fe2440cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.498+0000 7fe249e66640 1 --2- 172.21.15.32:0/3879538726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24406d170 0x7fe2440cb430 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7fe23c009e50 tx=0x7fe23c01c9e0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.498+0000 7fe249e66640 1 -- 172.21.15.32:0/3879538726 shutdown_connections 2023-12-20T04:29:51.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.498+0000 7fe249e66640 1 --2- 172.21.15.32:0/3879538726 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe24406d170 0x7fe2440cb430 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.498+0000 7fe249e66640 1 -- 172.21.15.32:0/3879538726 >> 172.21.15.32:0/3879538726 conn(0x7fe2440cf8d0 msgr2=0x7fe2440cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:51.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.498+0000 7fe249e66640 1 -- 172.21.15.32:0/3879538726 shutdown_connections 2023-12-20T04:29:51.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.498+0000 7fe249e66640 1 -- 172.21.15.32:0/3879538726 wait complete. 2023-12-20T04:29:51.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe249e66640 1 Processor -- start 2023-12-20T04:29:51.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe249e66640 1 -- start start 2023-12-20T04:29:51.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe249e66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe244065d40 0x7fe24414b510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe249e66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe24406d170 0x7fe2440c7b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe249e66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2440cb050 0x7fe244156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe249e66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24414ba50 con 0x7fe2440cb050 2023-12-20T04:29:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe249e66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe24415a620 con 0x7fe244065d40 2023-12-20T04:29:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe249e66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe24415a900 con 0x7fe24406d170 2023-12-20T04:29:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe243fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2440cb050 0x7fe244156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe243fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2440cb050 0x7fe244156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34540/0 (socket says 172.21.15.32:34540) 2023-12-20T04:29:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe243fff640 1 -- 172.21.15.32:0/1585060860 learned_addr learned my addr 172.21.15.32:0/1585060860 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe2437fe640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe244065d40 0x7fe24414b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe23bfff640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe24406d170 0x7fe2440c7b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe243fff640 1 -- 172.21.15.32:0/1585060860 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe24406d170 msgr2=0x7fe2440c7b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe243fff640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe24406d170 0x7fe2440c7b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe243fff640 1 -- 172.21.15.32:0/1585060860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe244065d40 msgr2=0x7fe24414b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe243fff640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe244065d40 0x7fe24414b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe243fff640 1 -- 172.21.15.32:0/1585060860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe23c009b30 con 0x7fe2440cb050 2023-12-20T04:29:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe243fff640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2440cb050 0x7fe244156dc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0af040 tx=0x7fe23c019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe240ff9640 1 -- 172.21.15.32:0/1585060860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0b8880 con 0x7fe2440cb050 2023-12-20T04:29:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe240ff9640 1 -- 172.21.15.32:0/1585060860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe23c0b8a00 con 0x7fe2440cb050 2023-12-20T04:29:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe240ff9640 1 -- 172.21.15.32:0/1585060860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe23c0b8db0 con 0x7fe2440cb050 2023-12-20T04:29:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe249e66640 1 -- 172.21.15.32:0/1585060860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe24415abe0 con 0x7fe2440cb050 2023-12-20T04:29:51.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.502+0000 7fe249e66640 1 -- 172.21.15.32:0/1585060860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe24415b1a0 con 0x7fe2440cb050 2023-12-20T04:29:51.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.506+0000 7fe249e66640 1 -- 172.21.15.32:0/1585060860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe24415b390 con 0x7fe2440cb050 2023-12-20T04:29:51.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.506+0000 7fe240ff9640 1 -- 172.21.15.32:0/1585060860 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 9) v1 ==== 85348+0+0 (secure 0 0 0) 0x7fe23c0b6440 con 0x7fe2440cb050 2023-12-20T04:29:51.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.506+0000 7fe240ff9640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe224065bd0 0x7fe224068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.506+0000 7fe240ff9640 1 -- 172.21.15.32:0/1585060860 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe23c11e010 con 0x7fe2440cb050 2023-12-20T04:29:51.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.506+0000 7fe2437fe640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe224065bd0 0x7fe224068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.506+0000 7fe2437fe640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe224065bd0 0x7fe224068090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe234007fd0 tx=0x7fe234009040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:51.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.510+0000 7fe240ff9640 1 -- 172.21.15.32:0/1585060860 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe23c0eb730 con 0x7fe2440cb050 2023-12-20T04:29:51.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.686+0000 7fe249e66640 1 -- 172.21.15.32:0/1585060860 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir6", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe244161730 con 0x7fe224065bd0 2023-12-20T04:29:51.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.746+0000 7fe240ff9640 1 -- 172.21.15.32:0/1585060860 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7fe234001570 con 0x7fe224065bd0 2023-12-20T04:29:51.752 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1/dir6 2023-12-20T04:29:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 -- 172.21.15.32:0/1585060860 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe224065bd0 msgr2=0x7fe224068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe224065bd0 0x7fe224068090 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fe234007fd0 tx=0x7fe234009040 comp rx=0 tx=0).stop 2023-12-20T04:29:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 -- 172.21.15.32:0/1585060860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2440cb050 msgr2=0x7fe244156dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2440cb050 0x7fe244156dc0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7fe23c0af040 tx=0x7fe23c019780 comp rx=0 tx=0).stop 2023-12-20T04:29:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 -- 172.21.15.32:0/1585060860 shutdown_connections 2023-12-20T04:29:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe224065bd0 0x7fe224068090 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2440cb050 0x7fe244156dc0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe24406d170 0x7fe2440c7b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 --2- 172.21.15.32:0/1585060860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe244065d40 0x7fe24414b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 -- 172.21.15.32:0/1585060860 >> 172.21.15.32:0/1585060860 conn(0x7fe2440cf8d0 msgr2=0x7fe244158b40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:51.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 -- 172.21.15.32:0/1585060860 shutdown_connections 2023-12-20T04:29:51.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.750+0000 7fe249e66640 1 -- 172.21.15.32:0/1585060860 wait complete. 2023-12-20T04:29:51.777 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir6 2023-12-20T04:29:51.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:51.777 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir7) 2023-12-20T04:29:51.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir7 1M 2023-12-20T04:29:51.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.978+0000 7f830b56d640 1 Processor -- start 2023-12-20T04:29:51.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.978+0000 7f830b56d640 1 -- start start 2023-12-20T04:29:51.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.978+0000 7f830b56d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f83040cb050 0x7f83040cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.978+0000 7f830b56d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f830406d170 0x7f8304161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.978+0000 7f830b56d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f830406a1f0 con 0x7f8304065d40 2023-12-20T04:29:51.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f830b56d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8304161860 con 0x7f83040cb050 2023-12-20T04:29:51.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f830b56d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830406a440 con 0x7f830406d170 2023-12-20T04:29:51.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8308b08640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8304065d40 0x7f830406ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40310/0 (socket says 172.21.15.32:40310) 2023-12-20T04:29:51.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8308b08640 1 -- 172.21.15.32:0/3988538655 learned_addr learned my addr 172.21.15.32:0/3988538655 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:51.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8309309640 1 --2- 172.21.15.32:0/3988538655 >> v2:172.21.15.39:3300/0 conn(0x7f83040cb050 0x7f83040cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8309b0a640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f830406d170 0x7f8304161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2294892527 0 0) 0x7f830406a1f0 con 0x7f8304065d40 2023-12-20T04:29:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f82f0003530 con 0x7f8304065d40 2023-12-20T04:29:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2059024322 0 0) 0x7f82f0003530 con 0x7f8304065d40 2023-12-20T04:29:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f830406d170 msgr2=0x7f8304161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f830406d170 0x7f8304161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 >> v2:172.21.15.39:3300/0 conn(0x7f83040cb050 msgr2=0x7f83040cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 --2- 172.21.15.32:0/3988538655 >> v2:172.21.15.39:3300/0 conn(0x7f83040cb050 0x7f83040cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f830406a720 con 0x7f8304065d40 2023-12-20T04:29:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f82f4001680 con 0x7f8304065d40 2023-12-20T04:29:51.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 >> v1:172.21.15.32:6789/0 conn(0x7f8304065d40 legacy=0x7f830406ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82f0003980 0x7f82f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 0x7f82f0004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82f0004ec0 0x7f82f000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8304161860 con 0x7f82f0004370 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f830406a440 con 0x7f82f0004ec0 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82f000bed0 con 0x7f82f0003980 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8309309640 1 --2- 172.21.15.32:0/3988538655 >> v2:172.21.15.39:3300/0 conn(0x7f83040cb050 0x7f83040cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8309b0a640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f830406d170 0x7f8304161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8308b08640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 0x7f82f0004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8309309640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82f0003980 0x7f82f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8309b0a640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82f0004ec0 0x7f82f000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8308b08640 1 -- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82f0003980 msgr2=0x7f82f0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8308b08640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82f0003980 0x7f82f0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8308b08640 1 -- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82f0004ec0 msgr2=0x7f82f000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:51.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8308b08640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82f0004ec0 0x7f82f000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8308b08640 1 -- 172.21.15.32:0/3988538655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82f0003530 con 0x7f82f0004370 2023-12-20T04:29:51.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.982+0000 7f8309b0a640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82f0004ec0 0x7f82f000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:51.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f8308b08640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 0x7f82f0004850 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f82f400c240 tx=0x7f82f401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=adfa071a3beb1b29 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:51.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4005ae0 con 0x7f82f0004370 2023-12-20T04:29:51.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82f40ab070 con 0x7f82f0004370 2023-12-20T04:29:51.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f82fbfff640 1 -- 172.21.15.32:0/3988538655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82f4007070 con 0x7f82f0004370 2023-12-20T04:29:51.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f830b56d640 1 -- 172.21.15.32:0/3988538655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 msgr2=0x7f82f0004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:51.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f830b56d640 1 --2- 172.21.15.32:0/3988538655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 0x7f82f0004850 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f82f400c240 tx=0x7f82f401f3d0 comp rx=0 tx=0).stop 2023-12-20T04:29:51.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f8309b0a640 1 -- 172.21.15.32:0/3988538655 reap_dead start 2023-12-20T04:29:51.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f830b56d640 1 -- 172.21.15.32:0/3988538655 shutdown_connections 2023-12-20T04:29:51.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f830b56d640 1 -- 172.21.15.32:0/3988538655 >> 172.21.15.32:0/3988538655 conn(0x7f83040cf8d0 msgr2=0x7f83040cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:51.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f830b56d640 1 -- 172.21.15.32:0/3988538655 shutdown_connections 2023-12-20T04:29:51.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f830b56d640 1 -- 172.21.15.32:0/3988538655 wait complete. 2023-12-20T04:29:51.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f830b56d640 1 Processor -- start 2023-12-20T04:29:51.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.986+0000 7f830b56d640 1 -- start start 2023-12-20T04:29:52.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f830b56d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 0x7f8304156e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f830b56d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f830414ce40 0x7f8304151cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f830b56d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83040c6300 0x7f830415d810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f830b56d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8304153090 con 0x7f82f0004370 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f830b56d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8304152200 con 0x7f83040c6300 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f830b56d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f830414d2a0 con 0x7f830414ce40 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8309309640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 0x7f8304156e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8309309640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 0x7f8304156e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34560/0 (socket says 172.21.15.32:34560) 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8309309640 1 -- 172.21.15.32:0/3715873435 learned_addr learned my addr 172.21.15.32:0/3715873435 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8308b08640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f830414ce40 0x7f8304151cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8309b0a640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83040c6300 0x7f830415d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8309309640 1 -- 172.21.15.32:0/3715873435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f830414ce40 msgr2=0x7f8304151cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8309309640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f830414ce40 0x7f8304151cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8309309640 1 -- 172.21.15.32:0/3715873435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83040c6300 msgr2=0x7f830415d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8309309640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83040c6300 0x7f830415d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8309309640 1 -- 172.21.15.32:0/3715873435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8304152480 con 0x7f82f0004370 2023-12-20T04:29:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8309b0a640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83040c6300 0x7f830415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f8309309640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 0x7f8304156e30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8300011cd0 tx=0x7f830000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f82f9ffb640 1 -- 172.21.15.32:0/3715873435 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f830000fd20 con 0x7f82f0004370 2023-12-20T04:29:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f82f9ffb640 1 -- 172.21.15.32:0/3715873435 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f830000fe80 con 0x7f82f0004370 2023-12-20T04:29:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f82f9ffb640 1 -- 172.21.15.32:0/3715873435 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8300004270 con 0x7f82f0004370 2023-12-20T04:29:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f830b56d640 1 -- 172.21.15.32:0/3715873435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8304152760 con 0x7f82f0004370 2023-12-20T04:29:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f830b56d640 1 -- 172.21.15.32:0/3715873435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8304152d20 con 0x7f82f0004370 2023-12-20T04:29:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f830b56d640 1 -- 172.21.15.32:0/3715873435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f830404e000 con 0x7f82f0004370 2023-12-20T04:29:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.990+0000 7f82f9ffb640 1 -- 172.21.15.32:0/3715873435 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f83000043d0 con 0x7f82f0004370 2023-12-20T04:29:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.994+0000 7f82f9ffb640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f82e8065c90 0x7f82e8068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.994+0000 7f82f9ffb640 1 -- 172.21.15.32:0/3715873435 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8300083e20 con 0x7f82f0004370 2023-12-20T04:29:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.994+0000 7f8308b08640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f82e8065c90 0x7f82e8068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.994+0000 7f8308b08640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f82e8065c90 0x7f82e8068150 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f82f40ae000 tx=0x7f82f4001a80 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:51.998+0000 7f82f9ffb640 1 -- 172.21.15.32:0/3715873435 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8300051460 con 0x7f82f0004370 2023-12-20T04:29:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.174+0000 7f830b56d640 1 -- 172.21.15.32:0/3715873435 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir7", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f830406ace0 con 0x7f82e8065c90 2023-12-20T04:29:52.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.234+0000 7f82f9ffb640 1 -- 172.21.15.32:0/3715873435 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f82f40b8040 con 0x7f82e8065c90 2023-12-20T04:29:52.239 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1/dir7 2023-12-20T04:29:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.234+0000 7f830b56d640 1 -- 172.21.15.32:0/3715873435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f82e8065c90 msgr2=0x7f82e8068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.234+0000 7f830b56d640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f82e8065c90 0x7f82e8068150 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f82f40ae000 tx=0x7f82f4001a80 comp rx=0 tx=0).stop 2023-12-20T04:29:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.238+0000 7f830b56d640 1 -- 172.21.15.32:0/3715873435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 msgr2=0x7f8304156e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.238+0000 7f830b56d640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 0x7f8304156e30 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f8300011cd0 tx=0x7f830000f610 comp rx=0 tx=0).stop 2023-12-20T04:29:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.238+0000 7f830b56d640 1 -- 172.21.15.32:0/3715873435 shutdown_connections 2023-12-20T04:29:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.238+0000 7f830b56d640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f82e8065c90 0x7f82e8068150 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.238+0000 7f830b56d640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83040c6300 0x7f830415d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.238+0000 7f830b56d640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f830414ce40 0x7f8304151cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.238+0000 7f830b56d640 1 --2- 172.21.15.32:0/3715873435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82f0004370 0x7f8304156e30 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.238+0000 7f830b56d640 1 -- 172.21.15.32:0/3715873435 >> 172.21.15.32:0/3715873435 conn(0x7f83040cf8d0 msgr2=0x7f8304160780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:52.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.238+0000 7f830b56d640 1 -- 172.21.15.32:0/3715873435 shutdown_connections 2023-12-20T04:29:52.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.238+0000 7f830b56d640 1 -- 172.21.15.32:0/3715873435 wait complete. 2023-12-20T04:29:52.261 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir7 2023-12-20T04:29:52.261 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:52.261 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir8) 2023-12-20T04:29:52.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir8 1M 2023-12-20T04:29:52.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.458+0000 7f326f518640 1 Processor -- start 2023-12-20T04:29:52.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.458+0000 7f326f518640 1 -- start start 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.458+0000 7f326f518640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3268070ab0 0x7f3268070f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.458+0000 7f326f518640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32680cd260 0x7f32680cd6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.458+0000 7f326f518640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f326806a6f0 con 0x7f326806ca60 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.458+0000 7f326f518640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326815a6c0 con 0x7f32680cd260 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.458+0000 7f326f518640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f326806a9d0 con 0x7f3268070ab0 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.458+0000 7f326dab5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32680cd260 0x7f32680cd6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.458+0000 7f326dab5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f32680cd260 0x7f32680cd6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34568/0 (socket says 172.21.15.32:34568) 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.458+0000 7f326dab5640 1 -- 172.21.15.32:0/3580981888 learned_addr learned my addr 172.21.15.32:0/3580981888 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f326cab3640 1 --2- 172.21.15.32:0/3580981888 >> v2:172.21.15.39:3300/0 conn(0x7f3268070ab0 0x7f3268070f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f326dab5640 1 -- 172.21.15.32:0/3580981888 >> v2:172.21.15.39:3300/0 conn(0x7f3268070ab0 msgr2=0x7f3268070f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f326dab5640 1 --2- 172.21.15.32:0/3580981888 >> v2:172.21.15.39:3300/0 conn(0x7f3268070ab0 0x7f3268070f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f326dab5640 1 -- 172.21.15.32:0/3580981888 >> v1:172.21.15.39:6789/0 conn(0x7f326806ca60 legacy=0x7f326806ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f326dab5640 1 -- 172.21.15.32:0/3580981888 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326806acb0 con 0x7f32680cd260 2023-12-20T04:29:52.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f326cab3640 1 --2- 172.21.15.32:0/3580981888 >> v2:172.21.15.39:3300/0 conn(0x7f3268070ab0 0x7f3268070f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:52.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f326dab5640 1 --2- 172.21.15.32:0/3580981888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32680cd260 0x7f32680cd6d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f326000af60 tx=0x7f326001daa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db33917dc05ef9c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:52.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f325ffff640 1 -- 172.21.15.32:0/3580981888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f326000a2d0 con 0x7f32680cd260 2023-12-20T04:29:52.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f325ffff640 1 -- 172.21.15.32:0/3580981888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f32600a6910 con 0x7f32680cd260 2023-12-20T04:29:52.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f325ffff640 1 -- 172.21.15.32:0/3580981888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600a6ca0 con 0x7f32680cd260 2023-12-20T04:29:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f326f518640 1 -- 172.21.15.32:0/3580981888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32680cd260 msgr2=0x7f32680cd6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.462+0000 7f326f518640 1 --2- 172.21.15.32:0/3580981888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32680cd260 0x7f32680cd6d0 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f326000af60 tx=0x7f326001daa0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 -- 172.21.15.32:0/3580981888 shutdown_connections 2023-12-20T04:29:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 --2- 172.21.15.32:0/3580981888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32680cd260 0x7f32680cd6d0 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 --2- 172.21.15.32:0/3580981888 >> v2:172.21.15.39:3300/0 conn(0x7f3268070ab0 0x7f3268070f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 -- 172.21.15.32:0/3580981888 >> 172.21.15.32:0/3580981888 conn(0x7f32680cf8d0 msgr2=0x7f32680cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 -- 172.21.15.32:0/3580981888 shutdown_connections 2023-12-20T04:29:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 -- 172.21.15.32:0/3580981888 wait complete. 2023-12-20T04:29:52.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 Processor -- start 2023-12-20T04:29:52.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 -- start start 2023-12-20T04:29:52.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326806ca60 0x7f326814e080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3268070ab0 0x7f32681523a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32680cd260 0x7f3268153d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f32680c8910 con 0x7f32680cd260 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f326814e5c0 con 0x7f326806ca60 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326f518640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3268152c80 con 0x7f3268070ab0 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326dab5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32680cd260 0x7f3268153d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326dab5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32680cd260 0x7f3268153d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34574/0 (socket says 172.21.15.32:34574) 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326dab5640 1 -- 172.21.15.32:0/407065254 learned_addr learned my addr 172.21.15.32:0/407065254 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326cab3640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3268070ab0 0x7f32681523a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326d2b4640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326806ca60 0x7f326814e080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326dab5640 1 -- 172.21.15.32:0/407065254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3268070ab0 msgr2=0x7f32681523a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326dab5640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3268070ab0 0x7f32681523a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326dab5640 1 -- 172.21.15.32:0/407065254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326806ca60 msgr2=0x7f326814e080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326dab5640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326806ca60 0x7f326814e080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326dab5640 1 -- 172.21.15.32:0/407065254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f326000ac40 con 0x7f32680cd260 2023-12-20T04:29:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.466+0000 7f326dab5640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32680cd260 0x7f3268153d20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f326001a6b0 tx=0x7f326001a840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.470+0000 7f325dffb640 1 -- 172.21.15.32:0/407065254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600bc070 con 0x7f32680cd260 2023-12-20T04:29:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.470+0000 7f325dffb640 1 -- 172.21.15.32:0/407065254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3260008500 con 0x7f32680cd260 2023-12-20T04:29:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.470+0000 7f326f518640 1 -- 172.21.15.32:0/407065254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3268152f60 con 0x7f32680cd260 2023-12-20T04:29:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.470+0000 7f325dffb640 1 -- 172.21.15.32:0/407065254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f32600088d0 con 0x7f32680cd260 2023-12-20T04:29:52.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.470+0000 7f326f518640 1 -- 172.21.15.32:0/407065254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3268153520 con 0x7f32680cd260 2023-12-20T04:29:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.470+0000 7f326f518640 1 -- 172.21.15.32:0/407065254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3268071430 con 0x7f32680cd260 2023-12-20T04:29:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.470+0000 7f325dffb640 1 -- 172.21.15.32:0/407065254 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f3260008a30 con 0x7f32680cd260 2023-12-20T04:29:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.470+0000 7f325dffb640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f324c065c60 0x7f324c068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.470+0000 7f325dffb640 1 -- 172.21.15.32:0/407065254 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f326011e3f0 con 0x7f32680cd260 2023-12-20T04:29:52.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.470+0000 7f326d2b4640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f324c065c60 0x7f324c068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.474+0000 7f326d2b4640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f324c065c60 0x7f324c068120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3264007fd0 tx=0x7f3264009040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:52.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.474+0000 7f325dffb640 1 -- 172.21.15.32:0/407065254 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f32600eba40 con 0x7f32680cd260 2023-12-20T04:29:52.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.650+0000 7f326f518640 1 -- 172.21.15.32:0/407065254 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir8", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f32680717e0 con 0x7f324c065c60 2023-12-20T04:29:52.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f325dffb640 1 -- 172.21.15.32:0/407065254 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7f32640014f0 con 0x7f324c065c60 2023-12-20T04:29:52.713 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1/dir8 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 -- 172.21.15.32:0/407065254 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f324c065c60 msgr2=0x7f324c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f324c065c60 0x7f324c068120 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3264007fd0 tx=0x7f3264009040 comp rx=0 tx=0).stop 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 -- 172.21.15.32:0/407065254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32680cd260 msgr2=0x7f3268153d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32680cd260 0x7f3268153d20 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f326001a6b0 tx=0x7f326001a840 comp rx=0 tx=0).stop 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 -- 172.21.15.32:0/407065254 shutdown_connections 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f324c065c60 0x7f324c068120 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f32680cd260 0x7f3268153d20 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3268070ab0 0x7f32681523a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 --2- 172.21.15.32:0/407065254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f326806ca60 0x7f326814e080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 -- 172.21.15.32:0/407065254 >> 172.21.15.32:0/407065254 conn(0x7f32680cf8d0 msgr2=0x7f32680cba30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 -- 172.21.15.32:0/407065254 shutdown_connections 2023-12-20T04:29:52.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.710+0000 7f326f518640 1 -- 172.21.15.32:0/407065254 wait complete. 2023-12-20T04:29:52.734 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir8 2023-12-20T04:29:52.734 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:29:52.734 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/dir9) 2023-12-20T04:29:52.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/dir9 1M 2023-12-20T04:29:52.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.934+0000 7efef99cc640 1 Processor -- start 2023-12-20T04:29:52.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.934+0000 7efef99cc640 1 -- start start 2023-12-20T04:29:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.934+0000 7efef99cc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef40cb050 0x7efef40cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.934+0000 7efef99cc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efef406d170 0x7efef4161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.934+0000 7efef99cc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef406a1f0 con 0x7efef4065d40 2023-12-20T04:29:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.934+0000 7efef99cc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef4161860 con 0x7efef406d170 2023-12-20T04:29:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.934+0000 7efef99cc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef406a440 con 0x7efef40cb050 2023-12-20T04:29:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efeebfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efef4065d40 0x7efef406ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40322/0 (socket says 172.21.15.32:40322) 2023-12-20T04:29:52.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efeebfff640 1 -- 172.21.15.32:0/1071794231 learned_addr learned my addr 172.21.15.32:0/1071794231 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:52.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efef37fe640 1 --2- 172.21.15.32:0/1071794231 >> v2:172.21.15.39:3300/0 conn(0x7efef406d170 0x7efef4161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efef2ffd640 1 --2- 172.21.15.32:0/1071794231 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef40cb050 0x7efef40cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efef27fc640 1 -- 172.21.15.32:0/1071794231 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 315740933 0 0) 0x7efef406a1f0 con 0x7efef4065d40 2023-12-20T04:29:52.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efef27fc640 1 -- 172.21.15.32:0/1071794231 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee0003530 con 0x7efef4065d40 2023-12-20T04:29:52.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efef37fe640 1 -- 172.21.15.32:0/1071794231 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef40cb050 msgr2=0x7efef40cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:52.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efef37fe640 1 --2- 172.21.15.32:0/1071794231 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef40cb050 0x7efef40cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efef37fe640 1 -- 172.21.15.32:0/1071794231 >> v1:172.21.15.32:6789/0 conn(0x7efef4065d40 legacy=0x7efef406ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efef27fc640 1 -- 172.21.15.32:0/1071794231 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3569436370 0 0) 0x7efee0003530 con 0x7efef4065d40 2023-12-20T04:29:52.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efef37fe640 1 -- 172.21.15.32:0/1071794231 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef406a720 con 0x7efef406d170 2023-12-20T04:29:52.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.938+0000 7efef37fe640 1 --2- 172.21.15.32:0/1071794231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef406d170 0x7efef4161320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efeec009d90 tx=0x7efeec01c840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=45009efe438b3ccc server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:52.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef27fc640 1 -- 172.21.15.32:0/1071794231 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efedc001680 con 0x7efef406d170 2023-12-20T04:29:52.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef27fc640 1 -- 172.21.15.32:0/1071794231 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efeec0a5640 con 0x7efef406d170 2023-12-20T04:29:52.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef27fc640 1 -- 172.21.15.32:0/1071794231 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efeec0a59d0 con 0x7efef406d170 2023-12-20T04:29:52.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 -- 172.21.15.32:0/1071794231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef406d170 msgr2=0x7efef4161320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 --2- 172.21.15.32:0/1071794231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef406d170 0x7efef4161320 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7efeec009d90 tx=0x7efeec01c840 comp rx=0 tx=0).stop 2023-12-20T04:29:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 -- 172.21.15.32:0/1071794231 shutdown_connections 2023-12-20T04:29:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 --2- 172.21.15.32:0/1071794231 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef406d170 0x7efef4161320 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 --2- 172.21.15.32:0/1071794231 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef40cb050 0x7efef40cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 -- 172.21.15.32:0/1071794231 >> 172.21.15.32:0/1071794231 conn(0x7efef40cf8d0 msgr2=0x7efef40cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:52.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 -- 172.21.15.32:0/1071794231 shutdown_connections 2023-12-20T04:29:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 -- 172.21.15.32:0/1071794231 wait complete. 2023-12-20T04:29:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 Processor -- start 2023-12-20T04:29:52.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 -- start start 2023-12-20T04:29:52.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef4065d40 0x7efef4147660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef406d170 0x7efef41537a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef40cb050 0x7efef4153ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef4152f20 con 0x7efef4065d40 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef415dc90 con 0x7efef406d170 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef99cc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef4152310 con 0x7efef40cb050 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef4065d40 0x7efef4147660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef4065d40 0x7efef4147660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34576/0 (socket says 172.21.15.32:34576) 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef2ffd640 1 -- 172.21.15.32:0/1568355050 learned_addr learned my addr 172.21.15.32:0/1568355050 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efeebfff640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef406d170 0x7efef41537a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.942+0000 7efef37fe640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef40cb050 0x7efef4153ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef2ffd640 1 -- 172.21.15.32:0/1568355050 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef40cb050 msgr2=0x7efef4153ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef2ffd640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef40cb050 0x7efef4153ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef2ffd640 1 -- 172.21.15.32:0/1568355050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef406d170 msgr2=0x7efef41537a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef2ffd640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef406d170 0x7efef41537a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef2ffd640 1 -- 172.21.15.32:0/1568355050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efeec009a70 con 0x7efef4065d40 2023-12-20T04:29:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efeebfff640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef406d170 0x7efef41537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef2ffd640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef4065d40 0x7efef4147660 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efee400e960 tx=0x7efee400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef0ff9640 1 -- 172.21.15.32:0/1568355050 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee4009690 con 0x7efef4065d40 2023-12-20T04:29:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef0ff9640 1 -- 172.21.15.32:0/1568355050 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee400ca00 con 0x7efef4065d40 2023-12-20T04:29:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef99cc640 1 -- 172.21.15.32:0/1568355050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef41525f0 con 0x7efef4065d40 2023-12-20T04:29:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef0ff9640 1 -- 172.21.15.32:0/1568355050 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee400cd70 con 0x7efef4065d40 2023-12-20T04:29:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef99cc640 1 -- 172.21.15.32:0/1568355050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef4152bb0 con 0x7efef4065d40 2023-12-20T04:29:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef99cc640 1 -- 172.21.15.32:0/1568355050 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef404e000 con 0x7efef4065d40 2023-12-20T04:29:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef0ff9640 1 -- 172.21.15.32:0/1568355050 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7efee4013470 con 0x7efef4065d40 2023-12-20T04:29:52.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef0ff9640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efed4065c90 0x7efed4068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:52.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efef0ff9640 1 -- 172.21.15.32:0/1568355050 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7efee40838b0 con 0x7efef4065d40 2023-12-20T04:29:52.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.946+0000 7efeebfff640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efed4065c90 0x7efed4068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:52.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.950+0000 7efeebfff640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efed4065c90 0x7efed4068150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efef4167c50 tx=0x7efedc009040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:52.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:52.950+0000 7efef0ff9640 1 -- 172.21.15.32:0/1568355050 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efee4050ee0 con 0x7efef4065d40 2023-12-20T04:29:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.126+0000 7efef99cc640 1 -- 172.21.15.32:0/1568355050 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/dir9", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7efef40c6e80 con 0x7efed4065c90 2023-12-20T04:29:53.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.186+0000 7efef0ff9640 1 -- 172.21.15.32:0/1568355050 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+43 (secure 0 0 0) 0x7efef40c6e80 con 0x7efed4065c90 2023-12-20T04:29:53.192 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1/dir9 2023-12-20T04:29:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 -- 172.21.15.32:0/1568355050 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efed4065c90 msgr2=0x7efed4068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efed4065c90 0x7efed4068150 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7efef4167c50 tx=0x7efedc009040 comp rx=0 tx=0).stop 2023-12-20T04:29:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 -- 172.21.15.32:0/1568355050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef4065d40 msgr2=0x7efef4147660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef4065d40 0x7efef4147660 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7efee400e960 tx=0x7efee400ee20 comp rx=0 tx=0).stop 2023-12-20T04:29:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 -- 172.21.15.32:0/1568355050 shutdown_connections 2023-12-20T04:29:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efed4065c90 0x7efed4068150 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef40cb050 0x7efef4153ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efef406d170 0x7efef41537a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 --2- 172.21.15.32:0/1568355050 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef4065d40 0x7efef4147660 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 -- 172.21.15.32:0/1568355050 >> 172.21.15.32:0/1568355050 conn(0x7efef40cf8d0 msgr2=0x7efef415fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 -- 172.21.15.32:0/1568355050 shutdown_connections 2023-12-20T04:29:53.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.190+0000 7efef99cc640 1 -- 172.21.15.32:0/1568355050 wait complete. 2023-12-20T04:29:53.213 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/dir9 2023-12-20T04:29:53.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir0 1M 2023-12-20T04:29:53.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.354+0000 7f7674ff0640 1 Processor -- start 2023-12-20T04:29:53.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.354+0000 7f7674ff0640 1 -- start start 2023-12-20T04:29:53.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f7674ff0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f76700b0cb0 0x7f76700caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f7674ff0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7670070230 0x7f76700706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f7674ff0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f767015b6a0 con 0x7f767015abb0 2023-12-20T04:29:53.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f7674ff0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f767006e160 con 0x7f76700b0cb0 2023-12-20T04:29:53.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f7674ff0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7670070c00 con 0x7f7670070230 2023-12-20T04:29:53.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766e59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f767015abb0 0x7f767015bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40338/0 (socket says 172.21.15.32:40338) 2023-12-20T04:29:53.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766e59c640 1 -- 172.21.15.32:0/4273023160 learned_addr learned my addr 172.21.15.32:0/4273023160 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:53.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766dd9b640 1 --2- 172.21.15.32:0/4273023160 >> v2:172.21.15.39:3300/0 conn(0x7f76700b0cb0 0x7f76700caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766ed9d640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7670070230 0x7f76700706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427481341 0 0) 0x7f767015b6a0 con 0x7f767015abb0 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7658003530 con 0x7f767015abb0 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2551360918 0 0) 0x7f7658003530 con 0x7f767015abb0 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7670070230 msgr2=0x7f76700706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7670070230 0x7f76700706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 >> v2:172.21.15.39:3300/0 conn(0x7f76700b0cb0 msgr2=0x7f76700caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 --2- 172.21.15.32:0/4273023160 >> v2:172.21.15.39:3300/0 conn(0x7f76700b0cb0 0x7f76700caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f767006d8c0 con 0x7f767015abb0 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f7660001680 con 0x7f767015abb0 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 >> v1:172.21.15.32:6789/0 conn(0x7f767015abb0 legacy=0x7f767015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7658003980 0x7f7658003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7670070230 0x7f76580047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7658004d10 0x7f7658005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f767006e160 con 0x7f7658004d10 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7670070c00 con 0x7f7658003980 2023-12-20T04:29:53.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f765800b990 con 0x7f7670070230 2023-12-20T04:29:53.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766ed9d640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7658004d10 0x7f7658005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766e59c640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7658003980 0x7f7658003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766dd9b640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7670070230 0x7f76580047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766ed9d640 1 -- 172.21.15.32:0/4273023160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7670070230 msgr2=0x7f76580047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:53.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766ed9d640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7670070230 0x7f76580047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766ed9d640 1 -- 172.21.15.32:0/4273023160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7658003980 msgr2=0x7f7658003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:53.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766ed9d640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7658003980 0x7f7658003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.358+0000 7f766ed9d640 1 -- 172.21.15.32:0/4273023160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7658003530 con 0x7f7658004d10 2023-12-20T04:29:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.362+0000 7f766ed9d640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7658004d10 0x7f7658005190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7668009fc0 tx=0x7f766801cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ef2c01c0a7b33ac server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.362+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76680a5c70 con 0x7f7658004d10 2023-12-20T04:29:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.362+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766800f040 con 0x7f7658004d10 2023-12-20T04:29:53.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.362+0000 7f766d59a640 1 -- 172.21.15.32:0/4273023160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76680ad680 con 0x7f7658004d10 2023-12-20T04:29:53.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.362+0000 7f7674ff0640 1 -- 172.21.15.32:0/4273023160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7658004d10 msgr2=0x7f7658005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.362+0000 7f7674ff0640 1 --2- 172.21.15.32:0/4273023160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7658004d10 0x7f7658005190 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f7668009fc0 tx=0x7f766801cc60 comp rx=0 tx=0).stop 2023-12-20T04:29:53.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.362+0000 7f766ed9d640 1 -- 172.21.15.32:0/4273023160 reap_dead start 2023-12-20T04:29:53.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.362+0000 7f7674ff0640 1 -- 172.21.15.32:0/4273023160 shutdown_connections 2023-12-20T04:29:53.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.362+0000 7f7674ff0640 1 -- 172.21.15.32:0/4273023160 >> 172.21.15.32:0/4273023160 conn(0x7f767006a510 msgr2=0x7f76700ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:53.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.362+0000 7f7674ff0640 1 -- 172.21.15.32:0/4273023160 shutdown_connections 2023-12-20T04:29:53.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.362+0000 7f7674ff0640 1 -- 172.21.15.32:0/4273023160 wait complete. 2023-12-20T04:29:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f7674ff0640 1 Processor -- start 2023-12-20T04:29:53.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f7674ff0640 1 -- start start 2023-12-20T04:29:53.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f7674ff0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7658004d10 0x7f76701f2cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f7674ff0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76701e7050 0x7f767014f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f7674ff0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76700c4670 0x7f76700c4b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f7674ff0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76700c5040 con 0x7f76701e7050 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f7674ff0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76701f3210 con 0x7f7658004d10 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f7674ff0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76701e5360 con 0x7f76700c4670 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f766dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76701e7050 0x7f767014f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f766dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76701e7050 0x7f767014f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34594/0 (socket says 172.21.15.32:34594) 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f766dd9b640 1 -- 172.21.15.32:0/1151518817 learned_addr learned my addr 172.21.15.32:0/1151518817 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f766ed9d640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76700c4670 0x7f76700c4b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f766e59c640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7658004d10 0x7f76701f2cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f766dd9b640 1 -- 172.21.15.32:0/1151518817 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76700c4670 msgr2=0x7f76700c4b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f766dd9b640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76700c4670 0x7f76700c4b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f766dd9b640 1 -- 172.21.15.32:0/1151518817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7658004d10 msgr2=0x7f76701f2cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f766dd9b640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7658004d10 0x7f76701f2cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f766dd9b640 1 -- 172.21.15.32:0/1151518817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76701e5640 con 0x7f76701e7050 2023-12-20T04:29:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f766dd9b640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76701e7050 0x7f767014f6b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f76640119a0 tx=0x7f7664011da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f765effd640 1 -- 172.21.15.32:0/1151518817 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f766400fa90 con 0x7f76701e7050 2023-12-20T04:29:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f765effd640 1 -- 172.21.15.32:0/1151518817 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f766400fbf0 con 0x7f76701e7050 2023-12-20T04:29:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f765effd640 1 -- 172.21.15.32:0/1151518817 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7664014440 con 0x7f76701e7050 2023-12-20T04:29:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f7674ff0640 1 -- 172.21.15.32:0/1151518817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76701e5920 con 0x7f76701e7050 2023-12-20T04:29:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.366+0000 7f7674ff0640 1 -- 172.21.15.32:0/1151518817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76701e5ee0 con 0x7f76701e7050 2023-12-20T04:29:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.370+0000 7f7674ff0640 1 -- 172.21.15.32:0/1151518817 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f767004e000 con 0x7f76701e7050 2023-12-20T04:29:53.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.370+0000 7f765effd640 1 -- 172.21.15.32:0/1151518817 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f76640145a0 con 0x7f76701e7050 2023-12-20T04:29:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.370+0000 7f765effd640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7650065c90 0x7f7650068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.370+0000 7f765effd640 1 -- 172.21.15.32:0/1151518817 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7664083e50 con 0x7f76701e7050 2023-12-20T04:29:53.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.370+0000 7f766e59c640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7650065c90 0x7f7650068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.370+0000 7f766e59c640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7650065c90 0x7f7650068150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7660000c10 tx=0x7f7660007900 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:53.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.374+0000 7f765effd640 1 -- 172.21.15.32:0/1151518817 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7664051480 con 0x7f76701e7050 2023-12-20T04:29:53.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.554+0000 7f7674ff0640 1 -- 172.21.15.32:0/1151518817 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f767006de80 con 0x7f7650065c90 2023-12-20T04:29:53.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.594+0000 7f765effd640 1 -- 172.21.15.32:0/1151518817 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f7660007e60 con 0x7f7650065c90 2023-12-20T04:29:53.600 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path /snap_test_dir1/dir0 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 -- 172.21.15.32:0/1151518817 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7650065c90 msgr2=0x7f7650068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7650065c90 0x7f7650068150 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f7660000c10 tx=0x7f7660007900 comp rx=0 tx=0).stop 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 -- 172.21.15.32:0/1151518817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76701e7050 msgr2=0x7f767014f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76701e7050 0x7f767014f6b0 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f76640119a0 tx=0x7f7664011da0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 -- 172.21.15.32:0/1151518817 shutdown_connections 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7650065c90 0x7f7650068150 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76700c4670 0x7f76700c4b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76701e7050 0x7f767014f6b0 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 --2- 172.21.15.32:0/1151518817 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7658004d10 0x7f76701f2cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 -- 172.21.15.32:0/1151518817 >> 172.21.15.32:0/1151518817 conn(0x7f767006a510 msgr2=0x7f76700cdb70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 -- 172.21.15.32:0/1151518817 shutdown_connections 2023-12-20T04:29:53.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.598+0000 7f7674ff0640 1 -- 172.21.15.32:0/1151518817 wait complete. 2023-12-20T04:29:53.621 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir0 2023-12-20T04:29:53.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir1 1M 2023-12-20T04:29:53.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.766+0000 7fe6ff59e640 1 Processor -- start 2023-12-20T04:29:53.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.766+0000 7fe6ff59e640 1 -- start start 2023-12-20T04:29:53.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.766+0000 7fe6ff59e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe7000cf840 0x7fe7000d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.766+0000 7fe6ff59e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7000d10f0 con 0x7fe70015ccf0 2023-12-20T04:29:53.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.766+0000 7fe6ff59e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7000cdd30 con 0x7fe7000cf3e0 2023-12-20T04:29:53.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.766+0000 7fe6ff59e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7000ce2f0 con 0x7fe7000cf840 2023-12-20T04:29:53.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.766+0000 7fe6fe59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe70015ccf0 0x7fe7000cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40352/0 (socket says 172.21.15.32:40352) 2023-12-20T04:29:53.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.766+0000 7fe6fe59c640 1 -- 172.21.15.32:0/1907448174 learned_addr learned my addr 172.21.15.32:0/1907448174 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:53.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.766+0000 7fe6fed9d640 1 --2- 172.21.15.32:0/1907448174 >> v2:172.21.15.39:3300/0 conn(0x7fe7000cf840 0x7fe7000d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.766+0000 7fe6fdd9b640 1 -- 172.21.15.32:0/1907448174 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3582754926 0 0) 0x7fe7000d10f0 con 0x7fe70015ccf0 2023-12-20T04:29:53.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.770+0000 7fe6fdd9b640 1 -- 172.21.15.32:0/1907448174 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe6ec003530 con 0x7fe70015ccf0 2023-12-20T04:29:53.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.770+0000 7fe6fed9d640 1 -- 172.21.15.32:0/1907448174 >> v1:172.21.15.39:6789/0 conn(0x7fe7000cf3e0 legacy=0x7fe7000d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.770+0000 7fe6fed9d640 1 -- 172.21.15.32:0/1907448174 >> v1:172.21.15.32:6789/0 conn(0x7fe70015ccf0 legacy=0x7fe7000cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.770+0000 7fe6fed9d640 1 -- 172.21.15.32:0/1907448174 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7000ce010 con 0x7fe7000cf840 2023-12-20T04:29:53.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.774+0000 7fe6fed9d640 1 --2- 172.21.15.32:0/1907448174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7000cf840 0x7fe7000d1c90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009d90 tx=0x7fe6f801c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a3a7c60e28ff05f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.774+0000 7fe6fdd9b640 1 -- 172.21.15.32:0/1907448174 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80a5870 con 0x7fe7000cf840 2023-12-20T04:29:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.774+0000 7fe6fdd9b640 1 -- 172.21.15.32:0/1907448174 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f80a59d0 con 0x7fe7000cf840 2023-12-20T04:29:53.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.774+0000 7fe6fdd9b640 1 -- 172.21.15.32:0/1907448174 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80a5d60 con 0x7fe7000cf840 2023-12-20T04:29:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.774+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1907448174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7000cf840 msgr2=0x7fe7000d1c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.774+0000 7fe6ff59e640 1 --2- 172.21.15.32:0/1907448174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7000cf840 0x7fe7000d1c90 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe6f8009d90 tx=0x7fe6f801c8c0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.774+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1907448174 shutdown_connections 2023-12-20T04:29:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.774+0000 7fe6ff59e640 1 --2- 172.21.15.32:0/1907448174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7000cf840 0x7fe7000d1c90 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.774+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1907448174 >> 172.21.15.32:0/1907448174 conn(0x7fe7000658e0 msgr2=0x7fe700065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.774+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1907448174 shutdown_connections 2023-12-20T04:29:53.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.774+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1907448174 wait complete. 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6ff59e640 1 Processor -- start 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6ff59e640 1 -- start start 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6ff59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7000cf3e0 0x7fe7000c8f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6ff59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7000cf840 0x7fe7000b73b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6ff59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe70015ccf0 0x7fe7000c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6ff59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000c7210 con 0x7fe70015ccf0 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6ff59e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000c74f0 con 0x7fe7000cf3e0 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6ff59e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7000c77d0 con 0x7fe7000cf840 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6fed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe70015ccf0 0x7fe7000c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6fed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe70015ccf0 0x7fe7000c6540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34602/0 (socket says 172.21.15.32:34602) 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6fed9d640 1 -- 172.21.15.32:0/1233487036 learned_addr learned my addr 172.21.15.32:0/1233487036 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6fe59c640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7000cf3e0 0x7fe7000c8f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6f7fff640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7000cf840 0x7fe7000b73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6fed9d640 1 -- 172.21.15.32:0/1233487036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7000cf840 msgr2=0x7fe7000b73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6fed9d640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7000cf840 0x7fe7000b73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6fed9d640 1 -- 172.21.15.32:0/1233487036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7000cf3e0 msgr2=0x7fe7000c8f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6fed9d640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7000cf3e0 0x7fe7000c8f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6fed9d640 1 -- 172.21.15.32:0/1233487036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f8009a70 con 0x7fe70015ccf0 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6fed9d640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe70015ccf0 0x7fe7000c6540 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80b0040 tx=0x7fe6f8019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6f6ffd640 1 -- 172.21.15.32:0/1233487036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80ad850 con 0x7fe70015ccf0 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6f6ffd640 1 -- 172.21.15.32:0/1233487036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6f80ad9b0 con 0x7fe70015ccf0 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6f6ffd640 1 -- 172.21.15.32:0/1233487036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6f80adda0 con 0x7fe70015ccf0 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1233487036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7000c8720 con 0x7fe70015ccf0 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1233487036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7000b78f0 con 0x7fe70015ccf0 2023-12-20T04:29:53.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.778+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1233487036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe700074100 con 0x7fe70015ccf0 2023-12-20T04:29:53.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.782+0000 7fe6f6ffd640 1 -- 172.21.15.32:0/1233487036 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7fe6f80b4050 con 0x7fe70015ccf0 2023-12-20T04:29:53.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.782+0000 7fe6f6ffd640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6e0065c60 0x7fe6e0068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:53.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.782+0000 7fe6f6ffd640 1 -- 172.21.15.32:0/1233487036 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe6f811e350 con 0x7fe70015ccf0 2023-12-20T04:29:53.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.782+0000 7fe6fe59c640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6e0065c60 0x7fe6e0068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:53.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.782+0000 7fe6fe59c640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6e0065c60 0x7fe6e0068120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00011f0 tx=0x7fe6f0007d40 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:53.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.786+0000 7fe6f6ffd640 1 -- 172.21.15.32:0/1233487036 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe6f80ebaf0 con 0x7fe70015ccf0 2023-12-20T04:29:53.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:53.966+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1233487036 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe70015d0d0 con 0x7fe6e0065c60 2023-12-20T04:29:54.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6f6ffd640 1 -- 172.21.15.32:0/1233487036 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe6f0006b70 con 0x7fe6e0065c60 2023-12-20T04:29:54.014 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path /snap_test_dir1/dir1 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1233487036 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6e0065c60 msgr2=0x7fe6e0068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6e0065c60 0x7fe6e0068120 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00011f0 tx=0x7fe6f0007d40 comp rx=0 tx=0).stop 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1233487036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe70015ccf0 msgr2=0x7fe7000c6540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe70015ccf0 0x7fe7000c6540 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7fe6f80b0040 tx=0x7fe6f8019660 comp rx=0 tx=0).stop 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1233487036 shutdown_connections 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6e0065c60 0x7fe6e0068120 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe70015ccf0 0x7fe7000c6540 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7000cf840 0x7fe7000b73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 --2- 172.21.15.32:0/1233487036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7000cf3e0 0x7fe7000c8f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1233487036 >> 172.21.15.32:0/1233487036 conn(0x7fe7000658e0 msgr2=0x7fe700152e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1233487036 shutdown_connections 2023-12-20T04:29:54.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.010+0000 7fe6ff59e640 1 -- 172.21.15.32:0/1233487036 wait complete. 2023-12-20T04:29:54.034 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir1 2023-12-20T04:29:54.035 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir2 1M 2023-12-20T04:29:54.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc6032c640 1 Processor -- start 2023-12-20T04:29:54.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc6032c640 1 -- start start 2023-12-20T04:29:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc6032c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc580b0760 0x7fdc580b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:54.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc6032c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdc580cc4b0 0x7fdc580cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc6032c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc580b1100 con 0x7fdc58157480 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc6032c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc580c9f50 con 0x7fdc580b0760 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc6032c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc580cde20 con 0x7fdc580cc4b0 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc5d8c7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc580b0760 0x7fdc580b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc5d8c7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc580b0760 0x7fdc580b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34618/0 (socket says 172.21.15.32:34618) 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc5d8c7640 1 -- 172.21.15.32:0/3904722707 learned_addr learned my addr 172.21.15.32:0/3904722707 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc5e0c8640 1 --1- 172.21.15.32:0/3904722707 >> v1:172.21.15.32:6789/0 conn(0x7fdc58157480 0x7fdc580c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40360/0 (socket says 172.21.15.32:40360) 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc5e8c9640 1 --2- 172.21.15.32:0/3904722707 >> v2:172.21.15.39:3300/0 conn(0x7fdc580cc4b0 0x7fdc580cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc5d8c7640 1 -- 172.21.15.32:0/3904722707 >> v2:172.21.15.39:3300/0 conn(0x7fdc580cc4b0 msgr2=0x7fdc580cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc5d8c7640 1 --2- 172.21.15.32:0/3904722707 >> v2:172.21.15.39:3300/0 conn(0x7fdc580cc4b0 0x7fdc580cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc5d8c7640 1 -- 172.21.15.32:0/3904722707 >> v1:172.21.15.32:6789/0 conn(0x7fdc58157480 legacy=0x7fdc580c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc5d8c7640 1 -- 172.21.15.32:0/3904722707 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc580c8e30 con 0x7fdc580b0760 2023-12-20T04:29:54.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.178+0000 7fdc5d0c6640 1 -- 172.21.15.32:0/3904722707 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780823269 0 0) 0x7fdc580b1100 con 0x7fdc58157480 2023-12-20T04:29:54.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc5d8c7640 1 --2- 172.21.15.32:0/3904722707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc580b0760 0x7fdc580b0bc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdc58071420 tx=0x7fdc4801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0f5be05674d1e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc5d0c6640 1 -- 172.21.15.32:0/3904722707 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480ab640 con 0x7fdc580b0760 2023-12-20T04:29:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc5d0c6640 1 -- 172.21.15.32:0/3904722707 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc480a7070 con 0x7fdc580b0760 2023-12-20T04:29:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc5d0c6640 1 -- 172.21.15.32:0/3904722707 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc480ab9c0 con 0x7fdc580b0760 2023-12-20T04:29:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc6032c640 1 -- 172.21.15.32:0/3904722707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc580b0760 msgr2=0x7fdc580b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc6032c640 1 --2- 172.21.15.32:0/3904722707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc580b0760 0x7fdc580b0bc0 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fdc58071420 tx=0x7fdc4801c440 comp rx=0 tx=0).stop 2023-12-20T04:29:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc6032c640 1 -- 172.21.15.32:0/3904722707 shutdown_connections 2023-12-20T04:29:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc6032c640 1 --2- 172.21.15.32:0/3904722707 >> v2:172.21.15.39:3300/0 conn(0x7fdc580cc4b0 0x7fdc580cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc6032c640 1 --2- 172.21.15.32:0/3904722707 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc580b0760 0x7fdc580b0bc0 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc6032c640 1 -- 172.21.15.32:0/3904722707 >> 172.21.15.32:0/3904722707 conn(0x7fdc5806a510 msgr2=0x7fdc580ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc6032c640 1 -- 172.21.15.32:0/3904722707 shutdown_connections 2023-12-20T04:29:54.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.182+0000 7fdc6032c640 1 -- 172.21.15.32:0/3904722707 wait complete. 2023-12-20T04:29:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc6032c640 1 Processor -- start 2023-12-20T04:29:54.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc6032c640 1 -- start start 2023-12-20T04:29:54.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc6032c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc580b0760 0x7fdc581f62e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc6032c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc580cc4b0 0x7fdc581f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc6032c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc58157480 0x7fdc581f80a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc6032c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc5806cb50 con 0x7fdc58157480 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc6032c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc580b1a20 con 0x7fdc580b0760 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc6032c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc580b1d00 con 0x7fdc580cc4b0 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5e8c9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc58157480 0x7fdc581f80a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5e8c9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc58157480 0x7fdc581f80a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34632/0 (socket says 172.21.15.32:34632) 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5e8c9640 1 -- 172.21.15.32:0/2180366288 learned_addr learned my addr 172.21.15.32:0/2180366288 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5d8c7640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc580cc4b0 0x7fdc581f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5e0c8640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc580b0760 0x7fdc581f62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5e8c9640 1 -- 172.21.15.32:0/2180366288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc580cc4b0 msgr2=0x7fdc581f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5e8c9640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc580cc4b0 0x7fdc581f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5e8c9640 1 -- 172.21.15.32:0/2180366288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc580b0760 msgr2=0x7fdc581f62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5e8c9640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc580b0760 0x7fdc581f62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5e8c9640 1 -- 172.21.15.32:0/2180366288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc480094d0 con 0x7fdc58157480 2023-12-20T04:29:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5e0c8640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc580b0760 0x7fdc581f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc5e8c9640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc58157480 0x7fdc581f80a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdc50007e40 tx=0x7fdc5000e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc4effd640 1 -- 172.21.15.32:0/2180366288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc50014620 con 0x7fdc58157480 2023-12-20T04:29:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc4effd640 1 -- 172.21.15.32:0/2180366288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5000ee90 con 0x7fdc58157480 2023-12-20T04:29:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc4effd640 1 -- 172.21.15.32:0/2180366288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc500149e0 con 0x7fdc58157480 2023-12-20T04:29:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.186+0000 7fdc6032c640 1 -- 172.21.15.32:0/2180366288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc580b1fe0 con 0x7fdc58157480 2023-12-20T04:29:54.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.190+0000 7fdc6032c640 1 -- 172.21.15.32:0/2180366288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc5806c3e0 con 0x7fdc58157480 2023-12-20T04:29:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.190+0000 7fdc6032c640 1 -- 172.21.15.32:0/2180366288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc5815adf0 con 0x7fdc58157480 2023-12-20T04:29:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.190+0000 7fdc4effd640 1 -- 172.21.15.32:0/2180366288 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7fdc50014b40 con 0x7fdc58157480 2023-12-20T04:29:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.190+0000 7fdc4effd640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdc3c065c60 0x7fdc3c068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.190+0000 7fdc4effd640 1 -- 172.21.15.32:0/2180366288 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fdc50083fa0 con 0x7fdc58157480 2023-12-20T04:29:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.190+0000 7fdc5e0c8640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdc3c065c60 0x7fdc3c068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.190+0000 7fdc5e0c8640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdc3c065c60 0x7fdc3c068120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdc54000f70 tx=0x7fdc54009040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.194+0000 7fdc4effd640 1 -- 172.21.15.32:0/2180366288 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdc5815adf0 con 0x7fdc58157480 2023-12-20T04:29:54.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.378+0000 7fdc6032c640 1 -- 172.21.15.32:0/2180366288 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdc580b1b90 con 0x7fdc3c065c60 2023-12-20T04:29:54.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.422+0000 7fdc4effd640 1 -- 172.21.15.32:0/2180366288 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fdc580b1b90 con 0x7fdc3c065c60 2023-12-20T04:29:54.426 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path /snap_test_dir1/dir2 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 -- 172.21.15.32:0/2180366288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdc3c065c60 msgr2=0x7fdc3c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdc3c065c60 0x7fdc3c068120 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdc54000f70 tx=0x7fdc54009040 comp rx=0 tx=0).stop 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 -- 172.21.15.32:0/2180366288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc58157480 msgr2=0x7fdc581f80a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc58157480 0x7fdc581f80a0 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fdc50007e40 tx=0x7fdc5000e4b0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 -- 172.21.15.32:0/2180366288 shutdown_connections 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdc3c065c60 0x7fdc3c068120 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc58157480 0x7fdc581f80a0 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc580cc4b0 0x7fdc581f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 --2- 172.21.15.32:0/2180366288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc580b0760 0x7fdc581f62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 -- 172.21.15.32:0/2180366288 >> 172.21.15.32:0/2180366288 conn(0x7fdc5806a510 msgr2=0x7fdc580c4f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 -- 172.21.15.32:0/2180366288 shutdown_connections 2023-12-20T04:29:54.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.426+0000 7fdc6032c640 1 -- 172.21.15.32:0/2180366288 wait complete. 2023-12-20T04:29:54.447 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir2 2023-12-20T04:29:54.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir3 1M 2023-12-20T04:29:54.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.590+0000 7f8dd337b640 1 Processor -- start 2023-12-20T04:29:54.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.590+0000 7f8dd337b640 1 -- start start 2023-12-20T04:29:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.590+0000 7f8dd337b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8dcc0b0cb0 0x7f8dcc0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.590+0000 7f8dd337b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dcc15b6a0 con 0x7f8dcc15abb0 2023-12-20T04:29:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.590+0000 7f8dd337b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dcc06d8c0 con 0x7f8dcc070230 2023-12-20T04:29:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.590+0000 7f8dd337b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dcc06e160 con 0x7f8dcc0b0cb0 2023-12-20T04:29:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.594+0000 7f8dd1117640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8dcc15abb0 0x7f8dcc15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40372/0 (socket says 172.21.15.32:40372) 2023-12-20T04:29:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.594+0000 7f8dd1117640 1 -- 172.21.15.32:0/3052052191 learned_addr learned my addr 172.21.15.32:0/3052052191 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:54.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.594+0000 7f8dd0916640 1 --2- 172.21.15.32:0/3052052191 >> v2:172.21.15.39:3300/0 conn(0x7f8dcc0b0cb0 0x7f8dcc0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.594+0000 7f8dc3fff640 1 -- 172.21.15.32:0/3052052191 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229635375 0 0) 0x7f8dcc15b6a0 con 0x7f8dcc15abb0 2023-12-20T04:29:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.594+0000 7f8dc3fff640 1 -- 172.21.15.32:0/3052052191 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8db8003530 con 0x7f8dcc15abb0 2023-12-20T04:29:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.594+0000 7f8dd0916640 1 -- 172.21.15.32:0/3052052191 >> v1:172.21.15.39:6789/0 conn(0x7f8dcc070230 legacy=0x7f8dcc06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.594+0000 7f8dd0916640 1 -- 172.21.15.32:0/3052052191 >> v1:172.21.15.32:6789/0 conn(0x7f8dcc15abb0 legacy=0x7f8dcc15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.594+0000 7f8dd0916640 1 -- 172.21.15.32:0/3052052191 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dcc06dba0 con 0x7f8dcc0b0cb0 2023-12-20T04:29:54.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.594+0000 7f8dd0916640 1 --2- 172.21.15.32:0/3052052191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dcc0b0cb0 0x7f8dcc0caaf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0097c0 tx=0x7f8dbc01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b9de5c4a83a530e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:54.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dc3fff640 1 -- 172.21.15.32:0/3052052191 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc0ab680 con 0x7f8dcc0b0cb0 2023-12-20T04:29:54.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dc3fff640 1 -- 172.21.15.32:0/3052052191 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dbc0ab7e0 con 0x7f8dcc0b0cb0 2023-12-20T04:29:54.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dc3fff640 1 -- 172.21.15.32:0/3052052191 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc0abb70 con 0x7f8dcc0b0cb0 2023-12-20T04:29:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 -- 172.21.15.32:0/3052052191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dcc0b0cb0 msgr2=0x7f8dcc0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 --2- 172.21.15.32:0/3052052191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dcc0b0cb0 0x7f8dcc0caaf0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc0097c0 tx=0x7f8dbc01c480 comp rx=0 tx=0).stop 2023-12-20T04:29:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 -- 172.21.15.32:0/3052052191 shutdown_connections 2023-12-20T04:29:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 --2- 172.21.15.32:0/3052052191 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dcc0b0cb0 0x7f8dcc0caaf0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 -- 172.21.15.32:0/3052052191 >> 172.21.15.32:0/3052052191 conn(0x7f8dcc06a510 msgr2=0x7f8dcc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 -- 172.21.15.32:0/3052052191 shutdown_connections 2023-12-20T04:29:54.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 -- 172.21.15.32:0/3052052191 wait complete. 2023-12-20T04:29:54.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 Processor -- start 2023-12-20T04:29:54.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 -- start start 2023-12-20T04:29:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dcc070230 0x7f8dcc14c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dcc0b0cb0 0x7f8dcc1503e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dcc15abb0 0x7f8dcc148c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc0cc550 con 0x7f8dcc15abb0 2023-12-20T04:29:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc14a2d0 con 0x7f8dcc070230 2023-12-20T04:29:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd337b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dcc14a5b0 con 0x7f8dcc0b0cb0 2023-12-20T04:29:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd1918640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dcc15abb0 0x7f8dcc148c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd1918640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dcc15abb0 0x7f8dcc148c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34636/0 (socket says 172.21.15.32:34636) 2023-12-20T04:29:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.598+0000 7f8dd1918640 1 -- 172.21.15.32:0/1246740685 learned_addr learned my addr 172.21.15.32:0/1246740685 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dd0916640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dcc0b0cb0 0x7f8dcc1503e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:54.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dd1117640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dcc070230 0x7f8dcc14c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:54.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dd1918640 1 -- 172.21.15.32:0/1246740685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dcc0b0cb0 msgr2=0x7f8dcc1503e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dd1918640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dcc0b0cb0 0x7f8dcc1503e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dd1918640 1 -- 172.21.15.32:0/1246740685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dcc070230 msgr2=0x7f8dcc14c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dd1918640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dcc070230 0x7f8dcc14c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dd1918640 1 -- 172.21.15.32:0/1246740685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dbc009420 con 0x7f8dcc15abb0 2023-12-20T04:29:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dd1918640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dcc15abb0 0x7f8dcc148c80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c770 tx=0x7f8dc400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dc1ffb640 1 -- 172.21.15.32:0/1246740685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc4003b80 con 0x7f8dcc15abb0 2023-12-20T04:29:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dc1ffb640 1 -- 172.21.15.32:0/1246740685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc4003ce0 con 0x7f8dcc15abb0 2023-12-20T04:29:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dc1ffb640 1 -- 172.21.15.32:0/1246740685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc400aa90 con 0x7f8dcc15abb0 2023-12-20T04:29:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dd337b640 1 -- 172.21.15.32:0/1246740685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dcc14a890 con 0x7f8dcc15abb0 2023-12-20T04:29:54.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dd337b640 1 -- 172.21.15.32:0/1246740685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dcc151590 con 0x7f8dcc15abb0 2023-12-20T04:29:54.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dd337b640 1 -- 172.21.15.32:0/1246740685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dcc04e000 con 0x7f8dcc15abb0 2023-12-20T04:29:54.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.602+0000 7f8dc1ffb640 1 -- 172.21.15.32:0/1246740685 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f8dc400abf0 con 0x7f8dcc15abb0 2023-12-20T04:29:54.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.606+0000 7f8dc1ffb640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8db0065c60 0x7f8db0068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:54.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.606+0000 7f8dc1ffb640 1 -- 172.21.15.32:0/1246740685 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8dc4083300 con 0x7f8dcc15abb0 2023-12-20T04:29:54.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.606+0000 7f8dd1117640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8db0065c60 0x7f8db0068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:54.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.606+0000 7f8dd1117640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8db0065c60 0x7f8db0068120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80011f0 tx=0x7f8dc8007d40 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:54.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.610+0000 7f8dc1ffb640 1 -- 172.21.15.32:0/1246740685 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8dc40509e0 con 0x7f8dcc15abb0 2023-12-20T04:29:54.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.790+0000 7f8dd337b640 1 -- 172.21.15.32:0/1246740685 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dcc06cf00 con 0x7f8db0065c60 2023-12-20T04:29:54.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dc1ffb640 1 -- 172.21.15.32:0/1246740685 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8dc8006b50 con 0x7f8db0065c60 2023-12-20T04:29:54.837 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path /snap_test_dir1/dir3 2023-12-20T04:29:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 -- 172.21.15.32:0/1246740685 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8db0065c60 msgr2=0x7f8db0068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8db0065c60 0x7f8db0068120 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f8dc80011f0 tx=0x7f8dc8007d40 comp rx=0 tx=0).stop 2023-12-20T04:29:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 -- 172.21.15.32:0/1246740685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dcc15abb0 msgr2=0x7f8dcc148c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dcc15abb0 0x7f8dcc148c80 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8dc400c770 tx=0x7f8dc400cc30 comp rx=0 tx=0).stop 2023-12-20T04:29:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 -- 172.21.15.32:0/1246740685 shutdown_connections 2023-12-20T04:29:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8db0065c60 0x7f8db0068120 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dcc15abb0 0x7f8dcc148c80 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dcc0b0cb0 0x7f8dcc1503e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 --2- 172.21.15.32:0/1246740685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dcc070230 0x7f8dcc14c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:54.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 -- 172.21.15.32:0/1246740685 >> 172.21.15.32:0/1246740685 conn(0x7f8dcc06a510 msgr2=0x7f8dcc0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:54.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 -- 172.21.15.32:0/1246740685 shutdown_connections 2023-12-20T04:29:54.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.834+0000 7f8dd337b640 1 -- 172.21.15.32:0/1246740685 wait complete. 2023-12-20T04:29:54.857 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir3 2023-12-20T04:29:54.858 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir4 1M 2023-12-20T04:29:55.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:54.998+0000 7f45a69db640 1 Processor -- start 2023-12-20T04:29:55.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f45a69db640 1 -- start start 2023-12-20T04:29:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f45a69db640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45a0157480 0x7f45a00c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f45a69db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a00cd4c0 0x7f45a00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f45a69db640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45a00b1100 con 0x7f45a00b0760 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f45a69db640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45a00cde20 con 0x7f45a0157480 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f45a69db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00c8e30 con 0x7f45a00cd4c0 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f459ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45a0157480 0x7f45a00c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f459ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45a0157480 0x7f45a00c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34638/0 (socket says 172.21.15.32:34638) 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f459ffff640 1 -- 172.21.15.32:0/3609381736 learned_addr learned my addr 172.21.15.32:0/3609381736 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f459f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f45a00b0760 0x7f45a015d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40374/0 (socket says 172.21.15.32:40374) 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f45a4f78640 1 --2- 172.21.15.32:0/3609381736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a00cd4c0 0x7f45a00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f459ffff640 1 -- 172.21.15.32:0/3609381736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a00cd4c0 msgr2=0x7f45a00b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f459ffff640 1 --2- 172.21.15.32:0/3609381736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a00cd4c0 0x7f45a00b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f459ffff640 1 -- 172.21.15.32:0/3609381736 >> v1:172.21.15.32:6789/0 conn(0x7f45a00b0760 legacy=0x7f45a015d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f459ffff640 1 -- 172.21.15.32:0/3609381736 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45a00ce070 con 0x7f45a0157480 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f459effd640 1 -- 172.21.15.32:0/3609381736 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 270055335 0 0) 0x7f45a00b1100 con 0x7f45a00b0760 2023-12-20T04:29:55.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.002+0000 7f45a4f78640 1 --2- 172.21.15.32:0/3609381736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a00cd4c0 0x7f45a00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:55.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f459ffff640 1 --2- 172.21.15.32:0/3609381736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a0157480 0x7f45a00c9840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4590009870 tx=0x7f459001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae48ccada1cab29d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:55.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f459effd640 1 -- 172.21.15.32:0/3609381736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45900ac600 con 0x7f45a0157480 2023-12-20T04:29:55.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f459effd640 1 -- 172.21.15.32:0/3609381736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45900a8050 con 0x7f45a0157480 2023-12-20T04:29:55.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f459effd640 1 -- 172.21.15.32:0/3609381736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45900ac980 con 0x7f45a0157480 2023-12-20T04:29:55.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f45a69db640 1 -- 172.21.15.32:0/3609381736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a0157480 msgr2=0x7f45a00c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f45a69db640 1 --2- 172.21.15.32:0/3609381736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a0157480 0x7f45a00c9840 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f4590009870 tx=0x7f459001c480 comp rx=0 tx=0).stop 2023-12-20T04:29:55.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f45a69db640 1 -- 172.21.15.32:0/3609381736 shutdown_connections 2023-12-20T04:29:55.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f45a69db640 1 --2- 172.21.15.32:0/3609381736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a00cd4c0 0x7f45a00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f45a69db640 1 --2- 172.21.15.32:0/3609381736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a0157480 0x7f45a00c9840 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f45a69db640 1 -- 172.21.15.32:0/3609381736 >> 172.21.15.32:0/3609381736 conn(0x7f45a006a510 msgr2=0x7f45a00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:55.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f45a69db640 1 -- 172.21.15.32:0/3609381736 shutdown_connections 2023-12-20T04:29:55.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f45a69db640 1 -- 172.21.15.32:0/3609381736 wait complete. 2023-12-20T04:29:55.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.006+0000 7f45a69db640 1 Processor -- start 2023-12-20T04:29:55.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f45a69db640 1 -- start start 2023-12-20T04:29:55.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f45a69db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a00b0760 0x7f45a0145020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f45a69db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00cd4c0 0x7f45a015af30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f45a69db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45a0157480 0x7f45a0154a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f45a69db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a00c4e10 con 0x7f45a00cd4c0 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f45a69db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45a006d6f0 con 0x7f45a0157480 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f45a69db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45a0145560 con 0x7f45a00b0760 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00cd4c0 0x7f45a015af30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00cd4c0 0x7f45a015af30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34646/0 (socket says 172.21.15.32:34646) 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459f7fe640 1 -- 172.21.15.32:0/2097391135 learned_addr learned my addr 172.21.15.32:0/2097391135 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459ffff640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a00b0760 0x7f45a0145020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f45a4f78640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45a0157480 0x7f45a0154a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459f7fe640 1 -- 172.21.15.32:0/2097391135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a00b0760 msgr2=0x7f45a0145020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459f7fe640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a00b0760 0x7f45a0145020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459f7fe640 1 -- 172.21.15.32:0/2097391135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45a0157480 msgr2=0x7f45a0154a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459f7fe640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45a0157480 0x7f45a0154a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459f7fe640 1 -- 172.21.15.32:0/2097391135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45900094d0 con 0x7f45a00cd4c0 2023-12-20T04:29:55.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459f7fe640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00cd4c0 0x7f45a015af30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f459400c450 tx=0x7f459400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:55.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459cff9640 1 -- 172.21.15.32:0/2097391135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594007480 con 0x7f45a00cd4c0 2023-12-20T04:29:55.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459cff9640 1 -- 172.21.15.32:0/2097391135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4594007600 con 0x7f45a00cd4c0 2023-12-20T04:29:55.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f459cff9640 1 -- 172.21.15.32:0/2097391135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4594004a30 con 0x7f45a00cd4c0 2023-12-20T04:29:55.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f45a69db640 1 -- 172.21.15.32:0/2097391135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45a01545c0 con 0x7f45a00cd4c0 2023-12-20T04:29:55.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f45a69db640 1 -- 172.21.15.32:0/2097391135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45a006d410 con 0x7f45a00cd4c0 2023-12-20T04:29:55.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.010+0000 7f45a69db640 1 -- 172.21.15.32:0/2097391135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45a00c46b0 con 0x7f45a00cd4c0 2023-12-20T04:29:55.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.014+0000 7f459cff9640 1 -- 172.21.15.32:0/2097391135 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f4594004b90 con 0x7f45a00cd4c0 2023-12-20T04:29:55.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.014+0000 7f459cff9640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065c60 0x7f458c068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.014+0000 7f459cff9640 1 -- 172.21.15.32:0/2097391135 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f45940827c0 con 0x7f45a00cd4c0 2023-12-20T04:29:55.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.014+0000 7f459ffff640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065c60 0x7f458c068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.014+0000 7f459ffff640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065c60 0x7f458c068120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f45900ae000 tx=0x7f45900a7000 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:55.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.018+0000 7f459cff9640 1 -- 172.21.15.32:0/2097391135 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f45a00c46b0 con 0x7f45a00cd4c0 2023-12-20T04:29:55.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.194+0000 7f45a69db640 1 -- 172.21.15.32:0/2097391135 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f45a00c46b0 con 0x7f458c065c60 2023-12-20T04:29:55.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f459cff9640 1 -- 172.21.15.32:0/2097391135 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f45a00c46b0 con 0x7f458c065c60 2023-12-20T04:29:55.241 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path /snap_test_dir1/dir4 2023-12-20T04:29:55.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 -- 172.21.15.32:0/2097391135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065c60 msgr2=0x7f458c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065c60 0x7f458c068120 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f45900ae000 tx=0x7f45900a7000 comp rx=0 tx=0).stop 2023-12-20T04:29:55.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 -- 172.21.15.32:0/2097391135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00cd4c0 msgr2=0x7f45a015af30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00cd4c0 0x7f45a015af30 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f459400c450 tx=0x7f459400c910 comp rx=0 tx=0).stop 2023-12-20T04:29:55.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 -- 172.21.15.32:0/2097391135 shutdown_connections 2023-12-20T04:29:55.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f458c065c60 0x7f458c068120 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45a0157480 0x7f45a0154a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45a00cd4c0 0x7f45a015af30 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 --2- 172.21.15.32:0/2097391135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45a00b0760 0x7f45a0145020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 -- 172.21.15.32:0/2097391135 >> 172.21.15.32:0/2097391135 conn(0x7f45a006a510 msgr2=0x7f45a0163090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:55.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 -- 172.21.15.32:0/2097391135 shutdown_connections 2023-12-20T04:29:55.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.238+0000 7f45a69db640 1 -- 172.21.15.32:0/2097391135 wait complete. 2023-12-20T04:29:55.262 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir4 2023-12-20T04:29:55.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir5 1M 2023-12-20T04:29:55.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.405+0000 7f9128066640 1 Processor -- start 2023-12-20T04:29:55.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.405+0000 7f9128066640 1 -- start start 2023-12-20T04:29:55.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.405+0000 7f9128066640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91200b0cb0 0x7f91200caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.405+0000 7f9128066640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9120070230 0x7f91200706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.405+0000 7f9128066640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f912015b6a0 con 0x7f912015abb0 2023-12-20T04:29:55.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.405+0000 7f9128066640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f912006e160 con 0x7f91200b0cb0 2023-12-20T04:29:55.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.405+0000 7f9128066640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9120070c00 con 0x7f9120070230 2023-12-20T04:29:55.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.405+0000 7f9125e02640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f912015abb0 0x7f912015bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40376/0 (socket says 172.21.15.32:40376) 2023-12-20T04:29:55.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.405+0000 7f9125e02640 1 -- 172.21.15.32:0/399123956 learned_addr learned my addr 172.21.15.32:0/399123956 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:55.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.405+0000 7f9125601640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91200b0cb0 0x7f91200caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.405+0000 7f9126603640 1 --2- 172.21.15.32:0/399123956 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9120070230 0x7f91200706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9124e00640 1 -- 172.21.15.32:0/399123956 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1741286956 0 0) 0x7f912015b6a0 con 0x7f912015abb0 2023-12-20T04:29:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9124e00640 1 -- 172.21.15.32:0/399123956 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f910c003530 con 0x7f912015abb0 2023-12-20T04:29:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9125601640 1 -- 172.21.15.32:0/399123956 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9120070230 msgr2=0x7f91200706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9125601640 1 --2- 172.21.15.32:0/399123956 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9120070230 0x7f91200706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9125601640 1 -- 172.21.15.32:0/399123956 >> v1:172.21.15.32:6789/0 conn(0x7f912015abb0 legacy=0x7f912015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9125601640 1 -- 172.21.15.32:0/399123956 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f912006d8c0 con 0x7f91200b0cb0 2023-12-20T04:29:55.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9125601640 1 --2- 172.21.15.32:0/399123956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91200b0cb0 0x7f91200caaf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f91100097c0 tx=0x7f911001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f541dd55995088ec server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:55.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9124e00640 1 -- 172.21.15.32:0/399123956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91100ab680 con 0x7f91200b0cb0 2023-12-20T04:29:55.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9124e00640 1 -- 172.21.15.32:0/399123956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91100ab7e0 con 0x7f91200b0cb0 2023-12-20T04:29:55.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9124e00640 1 -- 172.21.15.32:0/399123956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91100abb70 con 0x7f91200b0cb0 2023-12-20T04:29:55.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9128066640 1 -- 172.21.15.32:0/399123956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91200b0cb0 msgr2=0x7f91200caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.409+0000 7f9128066640 1 --2- 172.21.15.32:0/399123956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91200b0cb0 0x7f91200caaf0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f91100097c0 tx=0x7f911001c480 comp rx=0 tx=0).stop 2023-12-20T04:29:55.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 -- 172.21.15.32:0/399123956 shutdown_connections 2023-12-20T04:29:55.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 --2- 172.21.15.32:0/399123956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91200b0cb0 0x7f91200caaf0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 --2- 172.21.15.32:0/399123956 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9120070230 0x7f91200706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 -- 172.21.15.32:0/399123956 >> 172.21.15.32:0/399123956 conn(0x7f912006a510 msgr2=0x7f91200ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:55.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 -- 172.21.15.32:0/399123956 shutdown_connections 2023-12-20T04:29:55.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 -- 172.21.15.32:0/399123956 wait complete. 2023-12-20T04:29:55.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 Processor -- start 2023-12-20T04:29:55.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 -- start start 2023-12-20T04:29:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9120070230 0x7f9120148ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91200b0cb0 0x7f91200c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912015abb0 0x7f9120157420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9120149410 con 0x7f9120070230 2023-12-20T04:29:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912014ed40 con 0x7f91200b0cb0 2023-12-20T04:29:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9128066640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912014f020 con 0x7f912015abb0 2023-12-20T04:29:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9125e02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9120070230 0x7f9120148ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9125e02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9120070230 0x7f9120148ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34670/0 (socket says 172.21.15.32:34670) 2023-12-20T04:29:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9125e02640 1 -- 172.21.15.32:0/1173099624 learned_addr learned my addr 172.21.15.32:0/1173099624 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9125601640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91200b0cb0 0x7f91200c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9126603640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912015abb0 0x7f9120157420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9125e02640 1 -- 172.21.15.32:0/1173099624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912015abb0 msgr2=0x7f9120157420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9125e02640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912015abb0 0x7f9120157420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9125e02640 1 -- 172.21.15.32:0/1173099624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91200b0cb0 msgr2=0x7f91200c38c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9125e02640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91200b0cb0 0x7f91200c38c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9125e02640 1 -- 172.21.15.32:0/1173099624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9110009420 con 0x7f9120070230 2023-12-20T04:29:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9125601640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91200b0cb0 0x7f91200c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9126603640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912015abb0 0x7f9120157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:55.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.413+0000 7f9125e02640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9120070230 0x7f9120148ba0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f911c00c420 tx=0x7f911c00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:55.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.417+0000 7f91167fc640 1 -- 172.21.15.32:0/1173099624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c005cf0 con 0x7f9120070230 2023-12-20T04:29:55.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.417+0000 7f91167fc640 1 -- 172.21.15.32:0/1173099624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c005e50 con 0x7f9120070230 2023-12-20T04:29:55.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.417+0000 7f9128066640 1 -- 172.21.15.32:0/1173099624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f912014f300 con 0x7f9120070230 2023-12-20T04:29:55.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.417+0000 7f91167fc640 1 -- 172.21.15.32:0/1173099624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0072d0 con 0x7f9120070230 2023-12-20T04:29:55.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.417+0000 7f9128066640 1 -- 172.21.15.32:0/1173099624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f912014e980 con 0x7f9120070230 2023-12-20T04:29:55.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.417+0000 7f9128066640 1 -- 172.21.15.32:0/1173099624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912004e000 con 0x7f9120070230 2023-12-20T04:29:55.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.417+0000 7f91167fc640 1 -- 172.21.15.32:0/1173099624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f911c007430 con 0x7f9120070230 2023-12-20T04:29:55.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.417+0000 7f91167fc640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9104065c60 0x7f9104068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.417+0000 7f91167fc640 1 -- 172.21.15.32:0/1173099624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f911c082af0 con 0x7f9120070230 2023-12-20T04:29:55.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.417+0000 7f9125601640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9104065c60 0x7f9104068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.417+0000 7f9125601640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9104065c60 0x7f9104068120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f91100af000 tx=0x7f91100a7000 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:55.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.421+0000 7f91167fc640 1 -- 172.21.15.32:0/1173099624 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f911c0501e0 con 0x7f9120070230 2023-12-20T04:29:55.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.601+0000 7f9128066640 1 -- 172.21.15.32:0/1173099624 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f912015b330 con 0x7f9104065c60 2023-12-20T04:29:55.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f91167fc640 1 -- 172.21.15.32:0/1173099624 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f91100ab660 con 0x7f9104065c60 2023-12-20T04:29:55.650 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path /snap_test_dir1/dir5 2023-12-20T04:29:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f9128066640 1 -- 172.21.15.32:0/1173099624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9104065c60 msgr2=0x7f9104068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f9128066640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9104065c60 0x7f9104068120 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f91100af000 tx=0x7f91100a7000 comp rx=0 tx=0).stop 2023-12-20T04:29:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f9128066640 1 -- 172.21.15.32:0/1173099624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9120070230 msgr2=0x7f9120148ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f9128066640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9120070230 0x7f9120148ba0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f911c00c420 tx=0x7f911c00c8e0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f9128066640 1 -- 172.21.15.32:0/1173099624 shutdown_connections 2023-12-20T04:29:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f9128066640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9104065c60 0x7f9104068120 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f9128066640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912015abb0 0x7f9120157420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f9128066640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91200b0cb0 0x7f91200c38c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f9128066640 1 --2- 172.21.15.32:0/1173099624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9120070230 0x7f9120148ba0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f9128066640 1 -- 172.21.15.32:0/1173099624 >> 172.21.15.32:0/1173099624 conn(0x7f912006a510 msgr2=0x7f91200c41c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.645+0000 7f9128066640 1 -- 172.21.15.32:0/1173099624 shutdown_connections 2023-12-20T04:29:55.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.649+0000 7f9128066640 1 -- 172.21.15.32:0/1173099624 wait complete. 2023-12-20T04:29:55.671 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir5 2023-12-20T04:29:55.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir6 1M 2023-12-20T04:29:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.817+0000 7fcdf616d640 1 Processor -- start 2023-12-20T04:29:55.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.817+0000 7fcdf616d640 1 -- start start 2023-12-20T04:29:55.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdf616d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf00b0bc0 0x7fcdf00ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdf616d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf00cde20 con 0x7fcdf00b0760 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdf616d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf00c8e30 con 0x7fcdf0157480 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdf616d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00c9f50 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdeeffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcdf00b0760 0x7fcdf00cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40388/0 (socket says 172.21.15.32:40388) 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdeeffd640 1 -- 172.21.15.32:0/2527878071 learned_addr learned my addr 172.21.15.32:0/2527878071 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdeffff640 1 --2- 172.21.15.32:0/2527878071 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf00b0bc0 0x7fcdf00ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdee7fc640 1 -- 172.21.15.32:0/2527878071 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2304224853 0 0) 0x7fcdf00cde20 con 0x7fcdf00b0760 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdee7fc640 1 -- 172.21.15.32:0/2527878071 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdd8003530 con 0x7fcdf00b0760 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdee7fc640 1 -- 172.21.15.32:0/2527878071 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1958754677 0 0) 0x7fcdf00c8e30 con 0x7fcdf0157480 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdee7fc640 1 -- 172.21.15.32:0/2527878071 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcdf00cde20 con 0x7fcdf0157480 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdeffff640 1 -- 172.21.15.32:0/2527878071 >> v1:172.21.15.39:6789/0 conn(0x7fcdf0157480 legacy=0x7fcdf00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdeffff640 1 -- 172.21.15.32:0/2527878071 >> v1:172.21.15.32:6789/0 conn(0x7fcdf00b0760 legacy=0x7fcdf00cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdee7fc640 1 -- 172.21.15.32:0/2527878071 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 259799957 0 0) 0x7fcdd8003530 con 0x7fcdf00b0760 2023-12-20T04:29:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.821+0000 7fcdeffff640 1 -- 172.21.15.32:0/2527878071 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf00ce070 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdeffff640 1 --2- 172.21.15.32:0/2527878071 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf00b0bc0 0x7fcdf00ccbc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009dc0 tx=0x7fcde801c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4935af1b2220c07f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdee7fc640 1 -- 172.21.15.32:0/2527878071 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4001680 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdee7fc640 1 -- 172.21.15.32:0/2527878071 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde80a5550 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdee7fc640 1 -- 172.21.15.32:0/2527878071 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde80a58e0 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdf616d640 1 -- 172.21.15.32:0/2527878071 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf00b0bc0 msgr2=0x7fcdf00ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdf616d640 1 --2- 172.21.15.32:0/2527878071 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf00b0bc0 0x7fcdf00ccbc0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fcde8009dc0 tx=0x7fcde801c7a0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdf616d640 1 -- 172.21.15.32:0/2527878071 shutdown_connections 2023-12-20T04:29:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdf616d640 1 --2- 172.21.15.32:0/2527878071 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf00b0bc0 0x7fcdf00ccbc0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdf616d640 1 -- 172.21.15.32:0/2527878071 >> 172.21.15.32:0/2527878071 conn(0x7fcdf006a510 msgr2=0x7fcdf00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdf616d640 1 -- 172.21.15.32:0/2527878071 shutdown_connections 2023-12-20T04:29:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdf616d640 1 -- 172.21.15.32:0/2527878071 wait complete. 2023-12-20T04:29:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdf616d640 1 Processor -- start 2023-12-20T04:29:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.825+0000 7fcdf616d640 1 -- start start 2023-12-20T04:29:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdf616d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf00b0760 0x7fcdf01432d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdf616d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00b0bc0 0x7fcdf00b1b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdf616d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf0157480 0x7fcdf0158f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdf616d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0158b60 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdf616d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf00b4390 con 0x7fcdf00b0760 2023-12-20T04:29:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdf616d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf0143810 con 0x7fcdf0157480 2023-12-20T04:29:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdeeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00b0bc0 0x7fcdf00b1b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdeeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00b0bc0 0x7fcdf00b1b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34672/0 (socket says 172.21.15.32:34672) 2023-12-20T04:29:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdeeffd640 1 -- 172.21.15.32:0/87828040 learned_addr learned my addr 172.21.15.32:0/87828040 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:55.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdef7fe640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf00b0760 0x7fcdf01432d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdeffff640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf0157480 0x7fcdf0158f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdeeffd640 1 -- 172.21.15.32:0/87828040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf0157480 msgr2=0x7fcdf0158f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdeeffd640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf0157480 0x7fcdf0158f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdeeffd640 1 -- 172.21.15.32:0/87828040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf00b0760 msgr2=0x7fcdf01432d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdeeffd640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf00b0760 0x7fcdf01432d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdeeffd640 1 -- 172.21.15.32:0/87828040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde8009a70 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdef7fe640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf00b0760 0x7fcdf01432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:29:55.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdeeffd640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00b0bc0 0x7fcdf00b1b50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c780 tx=0x7fcde400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdd7fff640 1 -- 172.21.15.32:0/87828040 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4005ae0 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdd7fff640 1 -- 172.21.15.32:0/87828040 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcde4005c60 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdf616d640 1 -- 172.21.15.32:0/87828040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf015d820 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdd7fff640 1 -- 172.21.15.32:0/87828040 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcde4007070 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdf616d640 1 -- 172.21.15.32:0/87828040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf00b3dd0 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdf616d640 1 -- 172.21.15.32:0/87828040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf004e000 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.829+0000 7fcdd7fff640 1 -- 172.21.15.32:0/87828040 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7fcde40071d0 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.833+0000 7fcdd7fff640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcddc065c60 0x7fcddc068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.833+0000 7fcdd7fff640 1 -- 172.21.15.32:0/87828040 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fcde4082bd0 con 0x7fcdf00b0bc0 2023-12-20T04:29:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.833+0000 7fcdef7fe640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcddc065c60 0x7fcddc068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:55.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.833+0000 7fcdef7fe640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcddc065c60 0x7fcddc068120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcde00011f0 tx=0x7fcde0009040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:55.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:55.837+0000 7fcdd7fff640 1 -- 172.21.15.32:0/87828040 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcde4050370 con 0x7fcdf00b0bc0 2023-12-20T04:29:56.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.013+0000 7fcdf616d640 1 -- 172.21.15.32:0/87828040 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdf00b1030 con 0x7fcddc065c60 2023-12-20T04:29:56.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdd7fff640 1 -- 172.21.15.32:0/87828040 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fcde0001d50 con 0x7fcddc065c60 2023-12-20T04:29:56.060 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path /snap_test_dir1/dir6 2023-12-20T04:29:56.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 -- 172.21.15.32:0/87828040 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcddc065c60 msgr2=0x7fcddc068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcddc065c60 0x7fcddc068120 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcde00011f0 tx=0x7fcde0009040 comp rx=0 tx=0).stop 2023-12-20T04:29:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 -- 172.21.15.32:0/87828040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00b0bc0 msgr2=0x7fcdf00b1b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00b0bc0 0x7fcdf00b1b50 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7fcde400c780 tx=0x7fcde400cc40 comp rx=0 tx=0).stop 2023-12-20T04:29:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 -- 172.21.15.32:0/87828040 shutdown_connections 2023-12-20T04:29:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcddc065c60 0x7fcddc068120 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf0157480 0x7fcdf0158f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf00b0bc0 0x7fcdf00b1b50 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 --2- 172.21.15.32:0/87828040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf00b0760 0x7fcdf01432d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 -- 172.21.15.32:0/87828040 >> 172.21.15.32:0/87828040 conn(0x7fcdf006a510 msgr2=0x7fcdf00c4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 -- 172.21.15.32:0/87828040 shutdown_connections 2023-12-20T04:29:56.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.057+0000 7fcdf616d640 1 -- 172.21.15.32:0/87828040 wait complete. 2023-12-20T04:29:56.082 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir6 2023-12-20T04:29:56.082 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir7 1M 2023-12-20T04:29:56.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847e6e5640 1 Processor -- start 2023-12-20T04:29:56.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847e6e5640 1 -- start start 2023-12-20T04:29:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847e6e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8478157480 0x7f84780c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847e6e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84780cd4c0 0x7f84780b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847e6e5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84780b1100 con 0x7f84780b0760 2023-12-20T04:29:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847e6e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84780cde20 con 0x7f84780cd4c0 2023-12-20T04:29:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847e6e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84780c8e30 con 0x7f8478157480 2023-12-20T04:29:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847cc82640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84780cd4c0 0x7f84780b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f84777fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f84780b0760 0x7f847815d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40398/0 (socket says 172.21.15.32:40398) 2023-12-20T04:29:56.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f84777fe640 1 -- 172.21.15.32:0/377708887 learned_addr learned my addr 172.21.15.32:0/377708887 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f8477fff640 1 --2- 172.21.15.32:0/377708887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8478157480 0x7f84780c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847cc82640 1 -- 172.21.15.32:0/377708887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8478157480 msgr2=0x7f84780c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847cc82640 1 --2- 172.21.15.32:0/377708887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8478157480 0x7f84780c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847cc82640 1 -- 172.21.15.32:0/377708887 >> v1:172.21.15.32:6789/0 conn(0x7f84780b0760 legacy=0x7f847815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f8476ffd640 1 -- 172.21.15.32:0/377708887 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2187824581 0 0) 0x7f84780b1100 con 0x7f84780b0760 2023-12-20T04:29:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f847cc82640 1 -- 172.21.15.32:0/377708887 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84780ce070 con 0x7f84780cd4c0 2023-12-20T04:29:56.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.225+0000 7f8477fff640 1 --2- 172.21.15.32:0/377708887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8478157480 0x7f84780c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f847cc82640 1 --2- 172.21.15.32:0/377708887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cd4c0 0x7f84780b0bc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8470009dc0 tx=0x7f847001c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bab97925629e0210 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f8476ffd640 1 -- 172.21.15.32:0/377708887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700a5850 con 0x7f84780cd4c0 2023-12-20T04:29:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f8476ffd640 1 -- 172.21.15.32:0/377708887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84700a59b0 con 0x7f84780cd4c0 2023-12-20T04:29:56.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f8476ffd640 1 -- 172.21.15.32:0/377708887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700a5d40 con 0x7f84780cd4c0 2023-12-20T04:29:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f847e6e5640 1 -- 172.21.15.32:0/377708887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cd4c0 msgr2=0x7f84780b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f847e6e5640 1 --2- 172.21.15.32:0/377708887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cd4c0 0x7f84780b0bc0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7f8470009dc0 tx=0x7f847001c870 comp rx=0 tx=0).stop 2023-12-20T04:29:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f847e6e5640 1 -- 172.21.15.32:0/377708887 shutdown_connections 2023-12-20T04:29:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f847e6e5640 1 --2- 172.21.15.32:0/377708887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cd4c0 0x7f84780b0bc0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f847e6e5640 1 --2- 172.21.15.32:0/377708887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8478157480 0x7f84780c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f847e6e5640 1 -- 172.21.15.32:0/377708887 >> 172.21.15.32:0/377708887 conn(0x7f847806a510 msgr2=0x7f84780ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f847e6e5640 1 -- 172.21.15.32:0/377708887 shutdown_connections 2023-12-20T04:29:56.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f847e6e5640 1 -- 172.21.15.32:0/377708887 wait complete. 2023-12-20T04:29:56.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.229+0000 7f847e6e5640 1 Processor -- start 2023-12-20T04:29:56.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847e6e5640 1 -- start start 2023-12-20T04:29:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847e6e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b0760 0x7f847815d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847e6e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f84780cd4c0 0x7f847814d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847e6e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8478157480 0x7f8478070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847e6e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8478155870 con 0x7f8478157480 2023-12-20T04:29:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847e6e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847814e7f0 con 0x7f84780b0760 2023-12-20T04:29:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847e6e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f847814db70 con 0x7f84780cd4c0 2023-12-20T04:29:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847cc82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8478157480 0x7f8478070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847cc82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8478157480 0x7f8478070130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34690/0 (socket says 172.21.15.32:34690) 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847cc82640 1 -- 172.21.15.32:0/2127611765 learned_addr learned my addr 172.21.15.32:0/2127611765 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f84777fe640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f84780cd4c0 0x7f847814d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f8477fff640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b0760 0x7f847815d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847cc82640 1 -- 172.21.15.32:0/2127611765 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f84780cd4c0 msgr2=0x7f847814d120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847cc82640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f84780cd4c0 0x7f847814d120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847cc82640 1 -- 172.21.15.32:0/2127611765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b0760 msgr2=0x7f847815d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847cc82640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b0760 0x7f847815d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847cc82640 1 -- 172.21.15.32:0/2127611765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470009a70 con 0x7f8478157480 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f8477fff640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b0760 0x7f847815d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847cc82640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8478157480 0x7f8478070130 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f847001cde0 tx=0x7f8470019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f8474ff9640 1 -- 172.21.15.32:0/2127611765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700ad850 con 0x7f8478157480 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f8474ff9640 1 -- 172.21.15.32:0/2127611765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84700ad9b0 con 0x7f8478157480 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f8474ff9640 1 -- 172.21.15.32:0/2127611765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700add20 con 0x7f8478157480 2023-12-20T04:29:56.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847e6e5640 1 -- 172.21.15.32:0/2127611765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f847814de50 con 0x7f8478157480 2023-12-20T04:29:56.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847e6e5640 1 -- 172.21.15.32:0/2127611765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f847814e410 con 0x7f8478157480 2023-12-20T04:29:56.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.233+0000 7f847e6e5640 1 -- 172.21.15.32:0/2127611765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f847814e930 con 0x7f8478157480 2023-12-20T04:29:56.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.237+0000 7f8474ff9640 1 -- 172.21.15.32:0/2127611765 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f84700b4070 con 0x7f8478157480 2023-12-20T04:29:56.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.237+0000 7f8474ff9640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8464065c90 0x7f8464068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.237+0000 7f8474ff9640 1 -- 172.21.15.32:0/2127611765 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f847011e1e0 con 0x7f8478157480 2023-12-20T04:29:56.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.237+0000 7f8477fff640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8464065c90 0x7f8464068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.237+0000 7f8477fff640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8464065c90 0x7f8464068150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8468006fd0 tx=0x7f8468008040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:56.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.241+0000 7f8474ff9640 1 -- 172.21.15.32:0/2127611765 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f847814e930 con 0x7f8478157480 2023-12-20T04:29:56.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.417+0000 7f847e6e5640 1 -- 172.21.15.32:0/2127611765 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f84780c9360 con 0x7f8464065c90 2023-12-20T04:29:56.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.461+0000 7f8474ff9640 1 -- 172.21.15.32:0/2127611765 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f846800f860 con 0x7f8464065c90 2023-12-20T04:29:56.467 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path /snap_test_dir1/dir7 2023-12-20T04:29:56.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 -- 172.21.15.32:0/2127611765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8464065c90 msgr2=0x7f8464068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8464065c90 0x7f8464068150 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f8468006fd0 tx=0x7f8468008040 comp rx=0 tx=0).stop 2023-12-20T04:29:56.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 -- 172.21.15.32:0/2127611765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8478157480 msgr2=0x7f8478070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8478157480 0x7f8478070130 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7f847001cde0 tx=0x7f8470019610 comp rx=0 tx=0).stop 2023-12-20T04:29:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 -- 172.21.15.32:0/2127611765 shutdown_connections 2023-12-20T04:29:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8464065c90 0x7f8464068150 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8478157480 0x7f8478070130 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f84780cd4c0 0x7f847814d120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 --2- 172.21.15.32:0/2127611765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780b0760 0x7f847815d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 -- 172.21.15.32:0/2127611765 >> 172.21.15.32:0/2127611765 conn(0x7f847806a510 msgr2=0x7f84780c4cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 -- 172.21.15.32:0/2127611765 shutdown_connections 2023-12-20T04:29:56.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.465+0000 7f847e6e5640 1 -- 172.21.15.32:0/2127611765 wait complete. 2023-12-20T04:29:56.489 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir7 2023-12-20T04:29:56.489 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir8 1M 2023-12-20T04:29:56.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.629+0000 7f8eaf59e640 1 Processor -- start 2023-12-20T04:29:56.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.629+0000 7f8eaf59e640 1 -- start start 2023-12-20T04:29:56.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eaf59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb00affe0 0x7f8eb0071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eaf59e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb0071820 con 0x7f8eb00b13b0 2023-12-20T04:29:56.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eaf59e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8eb0071b00 con 0x7f8eb00cdce0 2023-12-20T04:29:56.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eaf59e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00723a0 con 0x7f8eb00affe0 2023-12-20T04:29:56.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8ea7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8eb00b13b0 0x7f8eb00b03c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40406/0 (socket says 172.21.15.32:40406) 2023-12-20T04:29:56.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8ea7fff640 1 -- 172.21.15.32:0/1667526095 learned_addr learned my addr 172.21.15.32:0/1667526095 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:29:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eae59c640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb00affe0 0x7f8eb0071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2554275985 0 0) 0x7f8eb0071820 con 0x7f8eb00b13b0 2023-12-20T04:29:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e9c003530 con 0x7f8eb00b13b0 2023-12-20T04:29:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2732584340 0 0) 0x7f8e9c003530 con 0x7f8eb00b13b0 2023-12-20T04:29:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb00affe0 msgr2=0x7f8eb0071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb00affe0 0x7f8eb0071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 >> v1:172.21.15.39:6789/0 conn(0x7f8eb00cdce0 legacy=0x7f8eb00cc510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb0071de0 con 0x7f8eb00b13b0 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f8e98001680 con 0x7f8eb00b13b0 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 >> v1:172.21.15.32:6789/0 conn(0x7f8eb00b13b0 legacy=0x7f8eb00b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e9c0039b0 0x7f8e9c003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb00affe0 0x7f8e9c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e9c004d10 0x7f8e9c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00723a0 con 0x7f8e9c004d10 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c00b990 con 0x7f8e9c0039b0 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e9c00bb50 con 0x7f8eb00affe0 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eaed9d640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e9c004d10 0x7f8e9c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8ea7fff640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb00affe0 0x7f8e9c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eae59c640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e9c0039b0 0x7f8e9c003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eaed9d640 1 -- 172.21.15.32:0/1667526095 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb00affe0 msgr2=0x7f8e9c0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:56.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eaed9d640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb00affe0 0x7f8e9c0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eaed9d640 1 -- 172.21.15.32:0/1667526095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e9c0039b0 msgr2=0x7f8e9c003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:56.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eaed9d640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e9c0039b0 0x7f8e9c003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.633+0000 7f8eaed9d640 1 -- 172.21.15.32:0/1667526095 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e9c003530 con 0x7f8e9c004d10 2023-12-20T04:29:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.637+0000 7f8eaed9d640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e9c004d10 0x7f8e9c005190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8001fc0 tx=0x7f8ea801c840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d8650c2245b3df server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.637+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80a6630 con 0x7f8e9c004d10 2023-12-20T04:29:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.637+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea80a67b0 con 0x7f8e9c004d10 2023-12-20T04:29:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.637+0000 7f8eadd9b640 1 -- 172.21.15.32:0/1667526095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80a6b60 con 0x7f8e9c004d10 2023-12-20T04:29:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.637+0000 7f8eaf59e640 1 -- 172.21.15.32:0/1667526095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e9c004d10 msgr2=0x7f8e9c005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.637+0000 7f8eaf59e640 1 --2- 172.21.15.32:0/1667526095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e9c004d10 0x7f8e9c005190 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f8ea8001fc0 tx=0x7f8ea801c840 comp rx=0 tx=0).stop 2023-12-20T04:29:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.637+0000 7f8eaed9d640 1 -- 172.21.15.32:0/1667526095 reap_dead start 2023-12-20T04:29:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.637+0000 7f8eaf59e640 1 -- 172.21.15.32:0/1667526095 shutdown_connections 2023-12-20T04:29:56.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.637+0000 7f8eaf59e640 1 -- 172.21.15.32:0/1667526095 >> 172.21.15.32:0/1667526095 conn(0x7f8eb006a4e0 msgr2=0x7f8eb006bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:56.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.637+0000 7f8eaf59e640 1 -- 172.21.15.32:0/1667526095 shutdown_connections 2023-12-20T04:29:56.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.637+0000 7f8eaf59e640 1 -- 172.21.15.32:0/1667526095 wait complete. 2023-12-20T04:29:56.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaf59e640 1 Processor -- start 2023-12-20T04:29:56.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaf59e640 1 -- start start 2023-12-20T04:29:56.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaf59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e9c004d10 0x7f8eb01f6440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaf59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb0152e60 0x7f8eb01544c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaf59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0156170 0x7f8eb01565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaf59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb00b17f0 con 0x7f8eb0156170 2023-12-20T04:29:56.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaf59e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0156b30 con 0x7f8e9c004d10 2023-12-20T04:29:56.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaf59e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8eb0154cf0 con 0x7f8eb0152e60 2023-12-20T04:29:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0156170 0x7f8eb01565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0156170 0x7f8eb01565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34706/0 (socket says 172.21.15.32:34706) 2023-12-20T04:29:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaed9d640 1 -- 172.21.15.32:0/2033711582 learned_addr learned my addr 172.21.15.32:0/2033711582 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eae59c640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e9c004d10 0x7f8eb01f6440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8ea7fff640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb0152e60 0x7f8eb01544c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaed9d640 1 -- 172.21.15.32:0/2033711582 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb0152e60 msgr2=0x7f8eb01544c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaed9d640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb0152e60 0x7f8eb01544c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaed9d640 1 -- 172.21.15.32:0/2033711582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e9c004d10 msgr2=0x7f8eb01f6440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaed9d640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e9c004d10 0x7f8eb01f6440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaed9d640 1 -- 172.21.15.32:0/2033711582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8eb0154fd0 con 0x7f8eb0156170 2023-12-20T04:29:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaed9d640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0156170 0x7f8eb01565f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8ea801cd20 tx=0x7f8e9c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8ea6ffd640 1 -- 172.21.15.32:0/2033711582 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80b6070 con 0x7f8eb0156170 2023-12-20T04:29:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8ea6ffd640 1 -- 172.21.15.32:0/2033711582 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ea80a6640 con 0x7f8eb0156170 2023-12-20T04:29:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaf59e640 1 -- 172.21.15.32:0/2033711582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8eb01552b0 con 0x7f8eb0156170 2023-12-20T04:29:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8ea6ffd640 1 -- 172.21.15.32:0/2033711582 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ea80a6b20 con 0x7f8eb0156170 2023-12-20T04:29:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaf59e640 1 -- 172.21.15.32:0/2033711582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8eb0155870 con 0x7f8eb0156170 2023-12-20T04:29:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.641+0000 7f8eaf59e640 1 -- 172.21.15.32:0/2033711582 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8eb0074100 con 0x7f8eb0156170 2023-12-20T04:29:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.645+0000 7f8ea6ffd640 1 -- 172.21.15.32:0/2033711582 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7f8ea80a6c80 con 0x7f8eb0156170 2023-12-20T04:29:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.645+0000 7f8ea6ffd640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e90065c90 0x7f8e90068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.645+0000 7f8ea6ffd640 1 -- 172.21.15.32:0/2033711582 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8ea811d240 con 0x7f8eb0156170 2023-12-20T04:29:56.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.645+0000 7f8eae59c640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e90065c90 0x7f8e90068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.645+0000 7f8eae59c640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e90065c90 0x7f8e90068150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009be0 tx=0x7f8ea0009190 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.649+0000 7f8ea6ffd640 1 -- 172.21.15.32:0/2033711582 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8ea80ea790 con 0x7f8eb0156170 2023-12-20T04:29:56.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.829+0000 7f8eaf59e640 1 -- 172.21.15.32:0/2033711582 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8eb00720c0 con 0x7f8e90065c90 2023-12-20T04:29:56.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.869+0000 7f8ea6ffd640 1 -- 172.21.15.32:0/2033711582 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7f8ea0019070 con 0x7f8e90065c90 2023-12-20T04:29:56.876 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path /snap_test_dir1/dir8 2023-12-20T04:29:56.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 -- 172.21.15.32:0/2033711582 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e90065c90 msgr2=0x7f8e90068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e90065c90 0x7f8e90068150 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f8ea0009be0 tx=0x7f8ea0009190 comp rx=0 tx=0).stop 2023-12-20T04:29:56.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 -- 172.21.15.32:0/2033711582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0156170 msgr2=0x7f8eb01565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:56.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0156170 0x7f8eb01565f0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f8ea801cd20 tx=0x7f8e9c005800 comp rx=0 tx=0).stop 2023-12-20T04:29:56.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 -- 172.21.15.32:0/2033711582 shutdown_connections 2023-12-20T04:29:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8e90065c90 0x7f8e90068150 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8eb0156170 0x7f8eb01565f0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8eb0152e60 0x7f8eb01544c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 --2- 172.21.15.32:0/2033711582 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e9c004d10 0x7f8eb01f6440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 -- 172.21.15.32:0/2033711582 >> 172.21.15.32:0/2033711582 conn(0x7f8eb006a4e0 msgr2=0x7f8eb00adec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 -- 172.21.15.32:0/2033711582 shutdown_connections 2023-12-20T04:29:56.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:56.873+0000 7f8eaf59e640 1 -- 172.21.15.32:0/2033711582 wait complete. 2023-12-20T04:29:56.897 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir8 2023-12-20T04:29:56.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/dir9 1M 2023-12-20T04:29:57.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5da7ce640 1 Processor -- start 2023-12-20T04:29:57.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5da7ce640 1 -- start start 2023-12-20T04:29:57.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5da7ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5d4157480 0x7fe5d40c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:57.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5da7ce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe5d40b0760 0x7fe5d40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:57.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5da7ce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d40b1100 con 0x7fe5d415d660 2023-12-20T04:29:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5da7ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40cde20 con 0x7fe5d4157480 2023-12-20T04:29:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5da7ce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40c8e30 con 0x7fe5d40b0760 2023-12-20T04:29:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5d3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5d4157480 0x7fe5d40c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5d3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5d4157480 0x7fe5d40c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34718/0 (socket says 172.21.15.32:34718) 2023-12-20T04:29:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5d3fff640 1 -- 172.21.15.32:0/1570560456 learned_addr learned my addr 172.21.15.32:0/1570560456 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5d3fff640 1 -- 172.21.15.32:0/1570560456 >> v2:172.21.15.39:3300/0 conn(0x7fe5d40b0760 msgr2=0x7fe5d40b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5d37fe640 1 --2- 172.21.15.32:0/1570560456 >> v2:172.21.15.39:3300/0 conn(0x7fe5d40b0760 0x7fe5d40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5d3fff640 1 --2- 172.21.15.32:0/1570560456 >> v2:172.21.15.39:3300/0 conn(0x7fe5d40b0760 0x7fe5d40b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5d3fff640 1 -- 172.21.15.32:0/1570560456 >> v1:172.21.15.39:6789/0 conn(0x7fe5d415d660 legacy=0x7fe5d415da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:29:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5d3fff640 1 -- 172.21.15.32:0/1570560456 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d40ce070 con 0x7fe5d4157480 2023-12-20T04:29:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.041+0000 7fe5d37fe640 1 --2- 172.21.15.32:0/1570560456 >> v2:172.21.15.39:3300/0 conn(0x7fe5d40b0760 0x7fe5d40b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:29:57.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5d3fff640 1 --2- 172.21.15.32:0/1570560456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d4157480 0x7fe5d40c9840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe5d406e0d0 tx=0x7fe5c401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b66dbbd0e2fa694 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:57.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5d2ffd640 1 -- 172.21.15.32:0/1570560456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40ab600 con 0x7fe5d4157480 2023-12-20T04:29:57.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5d2ffd640 1 -- 172.21.15.32:0/1570560456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c40ab760 con 0x7fe5d4157480 2023-12-20T04:29:57.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5d2ffd640 1 -- 172.21.15.32:0/1570560456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40abaf0 con 0x7fe5d4157480 2023-12-20T04:29:57.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5da7ce640 1 -- 172.21.15.32:0/1570560456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d4157480 msgr2=0x7fe5d40c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5da7ce640 1 --2- 172.21.15.32:0/1570560456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d4157480 0x7fe5d40c9840 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7fe5d406e0d0 tx=0x7fe5c401c400 comp rx=0 tx=0).stop 2023-12-20T04:29:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5da7ce640 1 -- 172.21.15.32:0/1570560456 shutdown_connections 2023-12-20T04:29:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5da7ce640 1 --2- 172.21.15.32:0/1570560456 >> v2:172.21.15.39:3300/0 conn(0x7fe5d40b0760 0x7fe5d40b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5da7ce640 1 --2- 172.21.15.32:0/1570560456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d4157480 0x7fe5d40c9840 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5da7ce640 1 -- 172.21.15.32:0/1570560456 >> 172.21.15.32:0/1570560456 conn(0x7fe5d406a510 msgr2=0x7fe5d40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5da7ce640 1 -- 172.21.15.32:0/1570560456 shutdown_connections 2023-12-20T04:29:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.045+0000 7fe5da7ce640 1 -- 172.21.15.32:0/1570560456 wait complete. 2023-12-20T04:29:57.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5da7ce640 1 Processor -- start 2023-12-20T04:29:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5da7ce640 1 -- start start 2023-12-20T04:29:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5da7ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40b0760 0x7fe5d4151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5da7ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d4157480 0x7fe5d40c5960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5da7ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d415d660 0x7fe5d40b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5da7ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4166740 con 0x7fe5d40b0760 2023-12-20T04:29:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5da7ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4165ac0 con 0x7fe5d4157480 2023-12-20T04:29:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5da7ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4165da0 con 0x7fe5d415d660 2023-12-20T04:29:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40b0760 0x7fe5d4151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40b0760 0x7fe5d4151d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34722/0 (socket says 172.21.15.32:34722) 2023-12-20T04:29:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d3fff640 1 -- 172.21.15.32:0/3057420076 learned_addr learned my addr 172.21.15.32:0/3057420076 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:29:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d37fe640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d4157480 0x7fe5d40c5960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d8d6b640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d415d660 0x7fe5d40b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d3fff640 1 -- 172.21.15.32:0/3057420076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d415d660 msgr2=0x7fe5d40b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d3fff640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d415d660 0x7fe5d40b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d3fff640 1 -- 172.21.15.32:0/3057420076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d4157480 msgr2=0x7fe5d40c5960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d3fff640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d4157480 0x7fe5d40c5960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d3fff640 1 -- 172.21.15.32:0/3057420076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c40094d0 con 0x7fe5d40b0760 2023-12-20T04:29:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d8d6b640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d415d660 0x7fe5d40b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:29:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d3fff640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40b0760 0x7fe5d4151d30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40ad040 tx=0x7fe5c401ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/3057420076 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40b4800 con 0x7fe5d40b0760 2023-12-20T04:29:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/3057420076 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c40b4980 con 0x7fe5d40b0760 2023-12-20T04:29:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/3057420076 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40b4d30 con 0x7fe5d40b0760 2023-12-20T04:29:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5da7ce640 1 -- 172.21.15.32:0/3057420076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d4166080 con 0x7fe5d40b0760 2023-12-20T04:29:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.049+0000 7fe5da7ce640 1 -- 172.21.15.32:0/3057420076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d4152820 con 0x7fe5d40b0760 2023-12-20T04:29:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.053+0000 7fe5da7ce640 1 -- 172.21.15.32:0/3057420076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d404e000 con 0x7fe5d40b0760 2023-12-20T04:29:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.053+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/3057420076 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 10) v1 ==== 85404+0+0 (secure 0 0 0) 0x7fe5c40b3380 con 0x7fe5d40b0760 2023-12-20T04:29:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.053+0000 7fe5d0ff9640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c0065c60 0x7fe5c0068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:29:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.053+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/3057420076 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe5c411dd70 con 0x7fe5d40b0760 2023-12-20T04:29:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.053+0000 7fe5d37fe640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c0065c60 0x7fe5c0068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:29:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.053+0000 7fe5d37fe640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c0065c60 0x7fe5c0068120 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe5d415a270 tx=0x7fe5c8008040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:29:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.057+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/3057420076 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe5c40eb350 con 0x7fe5d40b0760 2023-12-20T04:29:57.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.233+0000 7fe5da7ce640 1 -- 172.21.15.32:0/3057420076 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe5d40c9360 con 0x7fe5c0065c60 2023-12-20T04:29:57.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.277+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/3057420076 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+49 (secure 0 0 0) 0x7fe5c800c040 con 0x7fe5c0065c60 2023-12-20T04:29:57.282 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path /snap_test_dir1/dir9 2023-12-20T04:29:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 -- 172.21.15.32:0/3057420076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c0065c60 msgr2=0x7fe5c0068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c0065c60 0x7fe5c0068120 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fe5d415a270 tx=0x7fe5c8008040 comp rx=0 tx=0).stop 2023-12-20T04:29:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 -- 172.21.15.32:0/3057420076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40b0760 msgr2=0x7fe5d4151d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:29:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40b0760 0x7fe5d4151d30 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40ad040 tx=0x7fe5c401ac40 comp rx=0 tx=0).stop 2023-12-20T04:29:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 -- 172.21.15.32:0/3057420076 shutdown_connections 2023-12-20T04:29:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c0065c60 0x7fe5c0068120 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d415d660 0x7fe5d40b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:57.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d4157480 0x7fe5d40c5960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:57.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 --2- 172.21.15.32:0/3057420076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40b0760 0x7fe5d4151d30 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:29:57.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 -- 172.21.15.32:0/3057420076 >> 172.21.15.32:0/3057420076 conn(0x7fe5d406a510 msgr2=0x7fe5d40730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:29:57.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 -- 172.21.15.32:0/3057420076 shutdown_connections 2023-12-20T04:29:57.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:29:57.281+0000 7fe5da7ce640 1 -- 172.21.15.32:0/3057420076 wait complete. 2023-12-20T04:29:57.303 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/dir9 2023-12-20T04:29:59.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:29:59.173 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:30:00.026 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:30:00.021+0000 7f3afd7f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:30:00.027 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:30:00.021+0000 7f3afd7f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:30:00.027 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:30:00.021+0000 7f3afd7f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:30:00.027 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:30:00.021+0000 7f3afd7f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:30:00.027 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:30:00.021+0000 7f3afd7f2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:30:29.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:30:29.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:30:59.364 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:30:59.368 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:31:29.428 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:31:29.432 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:31:59.488 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:31:59.493 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:32:29.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:32:29.548 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:32:59.601 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:32:59.606 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:33:29.659 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:33:29.665 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:33:59.724 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:33:59.728 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:34:29.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:34:29.784 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:34:59.852 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:34:59.857 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:35:00.259 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:35:00.257+0000 7f3af46e0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:35:00.259 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:35:00.257+0000 7f3af46e0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:35:00.259 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:35:00.257+0000 7f3af46e0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:35:00.259 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:35:00.257+0000 7f3af46e0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:35:00.259 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:35:00.257+0000 7f3af46e0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:35:29.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:35:29.930 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:35:59.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:35:59.993 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:36:30.054 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:36:30.059 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:37:00.110 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:37:00.114 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:37:30.184 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:37:30.189 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:38:00.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:38:00.275 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:38:30.326 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:38:30.330 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:39:00.389 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:39:00.395 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:39:30.451 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:39:30.456 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:40:00.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:40:00.529 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:40:19.305 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir0 1M 2023-12-20T04:40:19.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.462+0000 7fa1f35ce640 1 Processor -- start 2023-12-20T04:40:19.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.462+0000 7fa1f35ce640 1 -- start start 2023-12-20T04:40:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1f35ce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa1ec070940 0x7fa1ec0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1f35ce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec15b6a0 con 0x7fa1ec0b0cb0 2023-12-20T04:40:19.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1f35ce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1ec06d8c0 con 0x7fa1ec15abb0 2023-12-20T04:40:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1f35ce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1ec06e160 con 0x7fa1ec070940 2023-12-20T04:40:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1f0b69640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa1ec0b0cb0 0x7fa1ec070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58920/0 (socket says 172.21.15.32:58920) 2023-12-20T04:40:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1f0b69640 1 -- 172.21.15.32:0/4127025442 learned_addr learned my addr 172.21.15.32:0/4127025442 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1f1b6b640 1 --2- 172.21.15.32:0/4127025442 >> v2:172.21.15.39:3300/0 conn(0x7fa1ec070940 0x7fa1ec0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1e3fff640 1 -- 172.21.15.32:0/4127025442 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1996552285 0 0) 0x7fa1ec15b6a0 con 0x7fa1ec0b0cb0 2023-12-20T04:40:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1e3fff640 1 -- 172.21.15.32:0/4127025442 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa1d8003530 con 0x7fa1ec0b0cb0 2023-12-20T04:40:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1f1b6b640 1 -- 172.21.15.32:0/4127025442 >> v1:172.21.15.39:6789/0 conn(0x7fa1ec15abb0 legacy=0x7fa1ec15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1f1b6b640 1 -- 172.21.15.32:0/4127025442 >> v1:172.21.15.32:6789/0 conn(0x7fa1ec0b0cb0 legacy=0x7fa1ec070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.466+0000 7fa1f1b6b640 1 -- 172.21.15.32:0/4127025442 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1ec06dba0 con 0x7fa1ec070940 2023-12-20T04:40:19.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1f1b6b640 1 --2- 172.21.15.32:0/4127025442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1ec070940 0x7fa1ec0caaf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009d90 tx=0x7fa1e401c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=31303717e421eab1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:19.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1e3fff640 1 -- 172.21.15.32:0/4127025442 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40a5a10 con 0x7fa1ec070940 2023-12-20T04:40:19.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1e3fff640 1 -- 172.21.15.32:0/4127025442 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e40a5b70 con 0x7fa1ec070940 2023-12-20T04:40:19.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1e3fff640 1 -- 172.21.15.32:0/4127025442 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40ad440 con 0x7fa1ec070940 2023-12-20T04:40:19.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1f35ce640 1 -- 172.21.15.32:0/4127025442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1ec070940 msgr2=0x7fa1ec0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1f35ce640 1 --2- 172.21.15.32:0/4127025442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1ec070940 0x7fa1ec0caaf0 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7fa1e4009d90 tx=0x7fa1e401c950 comp rx=0 tx=0).stop 2023-12-20T04:40:19.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1f35ce640 1 -- 172.21.15.32:0/4127025442 shutdown_connections 2023-12-20T04:40:19.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1f35ce640 1 --2- 172.21.15.32:0/4127025442 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1ec070940 0x7fa1ec0caaf0 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1f35ce640 1 -- 172.21.15.32:0/4127025442 >> 172.21.15.32:0/4127025442 conn(0x7fa1ec06a510 msgr2=0x7fa1ec0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:19.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1f35ce640 1 -- 172.21.15.32:0/4127025442 shutdown_connections 2023-12-20T04:40:19.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1f35ce640 1 -- 172.21.15.32:0/4127025442 wait complete. 2023-12-20T04:40:19.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1f35ce640 1 Processor -- start 2023-12-20T04:40:19.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.470+0000 7fa1f35ce640 1 -- start start 2023-12-20T04:40:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f35ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1ec070940 0x7fa1ec0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f35ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1ec0b0cb0 0x7fa1ec154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f35ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1ec15abb0 0x7fa1ec0c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f35ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec0c9840 con 0x7fa1ec15abb0 2023-12-20T04:40:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f35ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec0c8c70 con 0x7fa1ec0b0cb0 2023-12-20T04:40:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f35ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1ec157da0 con 0x7fa1ec070940 2023-12-20T04:40:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f1b6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1ec15abb0 0x7fa1ec0c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f1b6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1ec15abb0 0x7fa1ec0c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44052/0 (socket says 172.21.15.32:44052) 2023-12-20T04:40:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f1b6b640 1 -- 172.21.15.32:0/3312908959 learned_addr learned my addr 172.21.15.32:0/3312908959 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f0b69640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1ec0b0cb0 0x7fa1ec154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f136a640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1ec070940 0x7fa1ec0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f1b6b640 1 -- 172.21.15.32:0/3312908959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1ec070940 msgr2=0x7fa1ec0c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f1b6b640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1ec070940 0x7fa1ec0c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f1b6b640 1 -- 172.21.15.32:0/3312908959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1ec0b0cb0 msgr2=0x7fa1ec154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f1b6b640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1ec0b0cb0 0x7fa1ec154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f1b6b640 1 -- 172.21.15.32:0/3312908959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1e4009a70 con 0x7fa1ec15abb0 2023-12-20T04:40:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f0b69640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1ec0b0cb0 0x7fa1ec154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f136a640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1ec070940 0x7fa1ec0c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f1b6b640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1ec15abb0 0x7fa1ec0c9300 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa1e401a580 tx=0x7fa1e40196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1e1ffb640 1 -- 172.21.15.32:0/3312908959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40add90 con 0x7fa1ec15abb0 2023-12-20T04:40:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1e1ffb640 1 -- 172.21.15.32:0/3312908959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1e4009450 con 0x7fa1ec15abb0 2023-12-20T04:40:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f35ce640 1 -- 172.21.15.32:0/3312908959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1ec158080 con 0x7fa1ec15abb0 2023-12-20T04:40:19.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1e1ffb640 1 -- 172.21.15.32:0/3312908959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1e40b9670 con 0x7fa1ec15abb0 2023-12-20T04:40:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f35ce640 1 -- 172.21.15.32:0/3312908959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1ec1579c0 con 0x7fa1ec15abb0 2023-12-20T04:40:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.474+0000 7fa1f35ce640 1 -- 172.21.15.32:0/3312908959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1ec06acf0 con 0x7fa1ec15abb0 2023-12-20T04:40:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.478+0000 7fa1e1ffb640 1 -- 172.21.15.32:0/3312908959 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fa1e40b97d0 con 0x7fa1ec15abb0 2023-12-20T04:40:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.478+0000 7fa1e1ffb640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1d0065c60 0x7fa1d0068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.478+0000 7fa1e1ffb640 1 -- 172.21.15.32:0/3312908959 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa1e411ece0 con 0x7fa1ec15abb0 2023-12-20T04:40:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.478+0000 7fa1f136a640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1d0065c60 0x7fa1d0068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.478+0000 7fa1f136a640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1d0065c60 0x7fa1d0068120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8000f70 tx=0x7fa1e8009040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.482+0000 7fa1e1ffb640 1 -- 172.21.15.32:0/3312908959 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa1e40ec310 con 0x7fa1ec15abb0 2023-12-20T04:40:19.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.658+0000 7fa1f35ce640 1 -- 172.21.15.32:0/3312908959 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa1ec157f10 con 0x7fa1d0065c60 2023-12-20T04:40:19.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.722+0000 7fa1e1ffb640 1 -- 172.21.15.32:0/3312908959 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fa1ec157f10 con 0x7fa1d0065c60 2023-12-20T04:40:19.722 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path /snap_test_dir1/dir0 2023-12-20T04:40:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 -- 172.21.15.32:0/3312908959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1d0065c60 msgr2=0x7fa1d0068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1d0065c60 0x7fa1d0068120 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fa1e8000f70 tx=0x7fa1e8009040 comp rx=0 tx=0).stop 2023-12-20T04:40:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 -- 172.21.15.32:0/3312908959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1ec15abb0 msgr2=0x7fa1ec0c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1ec15abb0 0x7fa1ec0c9300 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7fa1e401a580 tx=0x7fa1e40196f0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 -- 172.21.15.32:0/3312908959 shutdown_connections 2023-12-20T04:40:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1d0065c60 0x7fa1d0068120 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1ec15abb0 0x7fa1ec0c9300 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1ec0b0cb0 0x7fa1ec154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 --2- 172.21.15.32:0/3312908959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1ec070940 0x7fa1ec0c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 -- 172.21.15.32:0/3312908959 >> 172.21.15.32:0/3312908959 conn(0x7fa1ec06a510 msgr2=0x7fa1ec0c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 -- 172.21.15.32:0/3312908959 shutdown_connections 2023-12-20T04:40:19.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.726+0000 7fa1f35ce640 1 -- 172.21.15.32:0/3312908959 wait complete. 2023-12-20T04:40:19.743 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir0 2023-12-20T04:40:19.743 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir1 1M 2023-12-20T04:40:19.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.890+0000 7fd295d25640 1 Processor -- start 2023-12-20T04:40:19.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.890+0000 7fd295d25640 1 -- start start 2023-12-20T04:40:19.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.890+0000 7fd295d25640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd29006cdf0 0x7fd290070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.890+0000 7fd295d25640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2900cc3b0 con 0x7fd290070770 2023-12-20T04:40:19.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.890+0000 7fd295d25640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd290070ba0 con 0x7fd2901591e0 2023-12-20T04:40:19.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.890+0000 7fd295d25640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2900c8e30 con 0x7fd29006cdf0 2023-12-20T04:40:19.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.890+0000 7fd28ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd290070770 0x7fd2900cc840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58930/0 (socket says 172.21.15.32:58930) 2023-12-20T04:40:19.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.890+0000 7fd28ffff640 1 -- 172.21.15.32:0/3058291779 learned_addr learned my addr 172.21.15.32:0/3058291779 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28effd640 1 --2- 172.21.15.32:0/3058291779 >> v2:172.21.15.39:3300/0 conn(0x7fd29006cdf0 0x7fd290070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3985690225 0 0) 0x7fd2900cc3b0 con 0x7fd290070770 2023-12-20T04:40:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd278003530 con 0x7fd290070770 2023-12-20T04:40:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1125330434 0 0) 0x7fd278003530 con 0x7fd290070770 2023-12-20T04:40:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 >> v2:172.21.15.39:3300/0 conn(0x7fd29006cdf0 msgr2=0x7fd290070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 --2- 172.21.15.32:0/3058291779 >> v2:172.21.15.39:3300/0 conn(0x7fd29006cdf0 0x7fd290070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 >> v1:172.21.15.39:6789/0 conn(0x7fd2901591e0 legacy=0x7fd2900caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29006d280 con 0x7fd290070770 2023-12-20T04:40:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fd288001a00 con 0x7fd290070770 2023-12-20T04:40:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 >> v1:172.21.15.32:6789/0 conn(0x7fd290070770 legacy=0x7fd2900cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 --2- 172.21.15.32:0/3058291779 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd278003980 0x7fd278003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 --2- 172.21.15.32:0/3058291779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 0x7fd278004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 --2- 172.21.15.32:0/3058291779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd278005190 0x7fd27800ffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2900c8e30 con 0x7fd2780047d0 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780104e0 con 0x7fd278005190 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd2780106a0 con 0x7fd278003980 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28effd640 1 --2- 172.21.15.32:0/3058291779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 0x7fd278004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28ffff640 1 --2- 172.21.15.32:0/3058291779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd278005190 0x7fd27800ffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28f7fe640 1 --2- 172.21.15.32:0/3058291779 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd278003980 0x7fd278003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28effd640 1 -- 172.21.15.32:0/3058291779 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd278003980 msgr2=0x7fd278003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28effd640 1 --2- 172.21.15.32:0/3058291779 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd278003980 0x7fd278003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28effd640 1 -- 172.21.15.32:0/3058291779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd278005190 msgr2=0x7fd27800ffa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28effd640 1 --2- 172.21.15.32:0/3058291779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd278005190 0x7fd27800ffa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28effd640 1 -- 172.21.15.32:0/3058291779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd278003530 con 0x7fd2780047d0 2023-12-20T04:40:19.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.894+0000 7fd28ffff640 1 -- 172.21.15.32:0/3058291779 reap_dead start 2023-12-20T04:40:19.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd28effd640 1 --2- 172.21.15.32:0/3058291779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 0x7fd278004c50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd284009ae0 tx=0x7fd28401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed071f74a92b7a71 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:19.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2840a8460 con 0x7fd2780047d0 2023-12-20T04:40:19.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2840a85c0 con 0x7fd2780047d0 2023-12-20T04:40:19.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd28e7fc640 1 -- 172.21.15.32:0/3058291779 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2840a89b0 con 0x7fd2780047d0 2023-12-20T04:40:19.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd295d25640 1 -- 172.21.15.32:0/3058291779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 msgr2=0x7fd278004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd295d25640 1 --2- 172.21.15.32:0/3058291779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 0x7fd278004c50 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7fd284009ae0 tx=0x7fd28401c950 comp rx=0 tx=0).stop 2023-12-20T04:40:19.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd295d25640 1 -- 172.21.15.32:0/3058291779 shutdown_connections 2023-12-20T04:40:19.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd295d25640 1 --2- 172.21.15.32:0/3058291779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 0x7fd278004c50 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd295d25640 1 -- 172.21.15.32:0/3058291779 >> 172.21.15.32:0/3058291779 conn(0x7fd29006a510 msgr2=0x7fd2900ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:19.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd295d25640 1 -- 172.21.15.32:0/3058291779 shutdown_connections 2023-12-20T04:40:19.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd295d25640 1 -- 172.21.15.32:0/3058291779 wait complete. 2023-12-20T04:40:19.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd295d25640 1 Processor -- start 2023-12-20T04:40:19.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.898+0000 7fd295d25640 1 -- start start 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd295d25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 0x7fd29015c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd295d25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd29015b9f0 0x7fd29013f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd295d25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd29015a190 0x7fd2901e62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd295d25640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29013fad0 con 0x7fd2780047d0 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd295d25640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd29015a950 con 0x7fd29015b9f0 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd295d25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd29015ac30 con 0x7fd29015a190 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 0x7fd29015c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 0x7fd29015c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44074/0 (socket says 172.21.15.32:44074) 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28f7fe640 1 -- 172.21.15.32:0/986214354 learned_addr learned my addr 172.21.15.32:0/986214354 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28effd640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd29015b9f0 0x7fd29013f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28ffff640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd29015a190 0x7fd2901e62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28f7fe640 1 -- 172.21.15.32:0/986214354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd29015a190 msgr2=0x7fd2901e62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28f7fe640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd29015a190 0x7fd2901e62c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28f7fe640 1 -- 172.21.15.32:0/986214354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd29015b9f0 msgr2=0x7fd29013f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28f7fe640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd29015b9f0 0x7fd29013f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28f7fe640 1 -- 172.21.15.32:0/986214354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd29015af10 con 0x7fd2780047d0 2023-12-20T04:40:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28effd640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd29015b9f0 0x7fd29013f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:19.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd28f7fe640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 0x7fd29015c1a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd280001ec0 tx=0x7fd28000e8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:19.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd277fff640 1 -- 172.21.15.32:0/986214354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2800052e0 con 0x7fd2780047d0 2023-12-20T04:40:19.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd277fff640 1 -- 172.21.15.32:0/986214354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd280005440 con 0x7fd2780047d0 2023-12-20T04:40:19.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd277fff640 1 -- 172.21.15.32:0/986214354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd280003820 con 0x7fd2780047d0 2023-12-20T04:40:19.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd295d25640 1 -- 172.21.15.32:0/986214354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd29015b1f0 con 0x7fd2780047d0 2023-12-20T04:40:19.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd295d25640 1 -- 172.21.15.32:0/986214354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd29015a610 con 0x7fd2780047d0 2023-12-20T04:40:19.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd295d25640 1 -- 172.21.15.32:0/986214354 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd29004e000 con 0x7fd2780047d0 2023-12-20T04:40:19.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.902+0000 7fd277fff640 1 -- 172.21.15.32:0/986214354 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fd280003980 con 0x7fd2780047d0 2023-12-20T04:40:19.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.906+0000 7fd277fff640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd27c065c90 0x7fd27c068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:19.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.906+0000 7fd277fff640 1 -- 172.21.15.32:0/986214354 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fd280082e40 con 0x7fd2780047d0 2023-12-20T04:40:19.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.906+0000 7fd28effd640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd27c065c90 0x7fd27c068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:19.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.906+0000 7fd28effd640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd27c065c90 0x7fd27c068150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd2840af000 tx=0x7fd2840aa040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:19.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:19.910+0000 7fd277fff640 1 -- 172.21.15.32:0/986214354 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd2800504b0 con 0x7fd2780047d0 2023-12-20T04:40:20.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.090+0000 7fd295d25640 1 -- 172.21.15.32:0/986214354 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd2900ce4d0 con 0x7fd27c065c90 2023-12-20T04:40:20.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd277fff640 1 -- 172.21.15.32:0/986214354 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fd2840b5040 con 0x7fd27c065c90 2023-12-20T04:40:20.153 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path /snap_test_dir1/dir1 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 -- 172.21.15.32:0/986214354 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd27c065c90 msgr2=0x7fd27c068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd27c065c90 0x7fd27c068150 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd2840af000 tx=0x7fd2840aa040 comp rx=0 tx=0).stop 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 -- 172.21.15.32:0/986214354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 msgr2=0x7fd29015c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 0x7fd29015c1a0 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7fd280001ec0 tx=0x7fd28000e8a0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 -- 172.21.15.32:0/986214354 shutdown_connections 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd27c065c90 0x7fd27c068150 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd29015a190 0x7fd2901e62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd29015b9f0 0x7fd29013f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 --2- 172.21.15.32:0/986214354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2780047d0 0x7fd29015c1a0 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 -- 172.21.15.32:0/986214354 >> 172.21.15.32:0/986214354 conn(0x7fd29006a510 msgr2=0x7fd2900b14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 -- 172.21.15.32:0/986214354 shutdown_connections 2023-12-20T04:40:20.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.154+0000 7fd295d25640 1 -- 172.21.15.32:0/986214354 wait complete. 2023-12-20T04:40:20.173 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir1 2023-12-20T04:40:20.174 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir2 1M 2023-12-20T04:40:20.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0d23f0640 1 Processor -- start 2023-12-20T04:40:20.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0d23f0640 1 -- start start 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0d23f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0cc164670 0x7ff0cc0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0d23f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0cc06da20 0x7ff0cc0cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0d23f0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff0cc070230 0x7ff0cc070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0d23f0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0cc0cb0b0 con 0x7ff0cc164670 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0d23f0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0cc070ba0 con 0x7ff0cc070230 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0d23f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc0ca810 con 0x7ff0cc06da20 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0cbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0cc164670 0x7ff0cc0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0cbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff0cc164670 0x7ff0cc0ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44078/0 (socket says 172.21.15.32:44078) 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0cbfff640 1 -- 172.21.15.32:0/2261928067 learned_addr learned my addr 172.21.15.32:0/2261928067 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0cbfff640 1 -- 172.21.15.32:0/2261928067 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0cc06da20 msgr2=0x7ff0cc0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0cb7fe640 1 --2- 172.21.15.32:0/2261928067 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0cc06da20 0x7ff0cc0cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0d098d640 1 --2- 172.21.15.32:0/2261928067 >> v2:172.21.15.39:3300/0 conn(0x7ff0cc070230 0x7ff0cc070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0cbfff640 1 --2- 172.21.15.32:0/2261928067 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0cc06da20 0x7ff0cc0cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0cbfff640 1 -- 172.21.15.32:0/2261928067 >> v2:172.21.15.39:3300/0 conn(0x7ff0cc070230 msgr2=0x7ff0cc070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0cbfff640 1 --2- 172.21.15.32:0/2261928067 >> v2:172.21.15.39:3300/0 conn(0x7ff0cc070230 0x7ff0cc070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0cbfff640 1 -- 172.21.15.32:0/2261928067 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0cc0caaf0 con 0x7ff0cc164670 2023-12-20T04:40:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0cb7fe640 1 --2- 172.21.15.32:0/2261928067 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0cc06da20 0x7ff0cc0cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:20.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.322+0000 7ff0d098d640 1 --2- 172.21.15.32:0/2261928067 >> v2:172.21.15.39:3300/0 conn(0x7ff0cc070230 0x7ff0cc070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:20.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0cbfff640 1 --2- 172.21.15.32:0/2261928067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc164670 0x7ff0cc0ca1a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0c8a00 tx=0x7ff0bc01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=815f216d939d0266 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:20.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0caffd640 1 -- 172.21.15.32:0/2261928067 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0ab600 con 0x7ff0cc164670 2023-12-20T04:40:20.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0caffd640 1 -- 172.21.15.32:0/2261928067 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0a7050 con 0x7ff0cc164670 2023-12-20T04:40:20.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0caffd640 1 -- 172.21.15.32:0/2261928067 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0ab980 con 0x7ff0cc164670 2023-12-20T04:40:20.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2261928067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc164670 msgr2=0x7ff0cc0ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:20.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0d23f0640 1 --2- 172.21.15.32:0/2261928067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc164670 0x7ff0cc0ca1a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0c8a00 tx=0x7ff0bc01c3b0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2261928067 shutdown_connections 2023-12-20T04:40:20.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0d23f0640 1 --2- 172.21.15.32:0/2261928067 >> v2:172.21.15.39:3300/0 conn(0x7ff0cc070230 0x7ff0cc070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0d23f0640 1 --2- 172.21.15.32:0/2261928067 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0cc06da20 0x7ff0cc0cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0d23f0640 1 --2- 172.21.15.32:0/2261928067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc164670 0x7ff0cc0ca1a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2261928067 >> 172.21.15.32:0/2261928067 conn(0x7ff0cc06a510 msgr2=0x7ff0cc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:20.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2261928067 shutdown_connections 2023-12-20T04:40:20.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2261928067 wait complete. 2023-12-20T04:40:20.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.326+0000 7ff0d23f0640 1 Processor -- start 2023-12-20T04:40:20.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0d23f0640 1 -- start start 2023-12-20T04:40:20.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0d23f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc06da20 0x7ff0cc15c590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0d23f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0cc070230 0x7ff0cc15aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0d23f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0cc164670 0x7ff0cc15b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0d23f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc1601a0 con 0x7ff0cc06da20 2023-12-20T04:40:20.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0d23f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc15cad0 con 0x7ff0cc164670 2023-12-20T04:40:20.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0d23f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0cc15f3e0 con 0x7ff0cc070230 2023-12-20T04:40:20.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc06da20 0x7ff0cc15c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc06da20 0x7ff0cc15c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44084/0 (socket says 172.21.15.32:44084) 2023-12-20T04:40:20.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0cbfff640 1 -- 172.21.15.32:0/2869710450 learned_addr learned my addr 172.21.15.32:0/2869710450 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:20.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0cb7fe640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0cc070230 0x7ff0cc15aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0d098d640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0cc164670 0x7ff0cc15b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0cbfff640 1 -- 172.21.15.32:0/2869710450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0cc070230 msgr2=0x7ff0cc15aa10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:20.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0cbfff640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0cc070230 0x7ff0cc15aa10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0cbfff640 1 -- 172.21.15.32:0/2869710450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0cc164670 msgr2=0x7ff0cc15b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:20.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0cbfff640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0cc164670 0x7ff0cc15b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0cbfff640 1 -- 172.21.15.32:0/2869710450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0bc0094d0 con 0x7ff0cc06da20 2023-12-20T04:40:20.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0cbfff640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc06da20 0x7ff0cc15c590 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0ae040 tx=0x7ff0bc019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:20.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0c8ff9640 1 -- 172.21.15.32:0/2869710450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0b4640 con 0x7ff0cc06da20 2023-12-20T04:40:20.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0c8ff9640 1 -- 172.21.15.32:0/2869710450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0b47c0 con 0x7ff0cc06da20 2023-12-20T04:40:20.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2869710450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0cc15f6c0 con 0x7ff0cc06da20 2023-12-20T04:40:20.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0c8ff9640 1 -- 172.21.15.32:0/2869710450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0b4b70 con 0x7ff0cc06da20 2023-12-20T04:40:20.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2869710450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0cc15fc80 con 0x7ff0cc06da20 2023-12-20T04:40:20.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.330+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2869710450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0cc15fe70 con 0x7ff0cc06da20 2023-12-20T04:40:20.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.334+0000 7ff0c8ff9640 1 -- 172.21.15.32:0/2869710450 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7ff0bc0b2070 con 0x7ff0cc06da20 2023-12-20T04:40:20.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.334+0000 7ff0c8ff9640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff0b8065c60 0x7ff0b8068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.334+0000 7ff0c8ff9640 1 -- 172.21.15.32:0/2869710450 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7ff0bc11d7b0 con 0x7ff0cc06da20 2023-12-20T04:40:20.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.334+0000 7ff0cb7fe640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff0b8065c60 0x7ff0b8068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.334+0000 7ff0cb7fe640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff0b8065c60 0x7ff0b8068120 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0cdfb0 tx=0x7ff0c0008040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:20.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.338+0000 7ff0c8ff9640 1 -- 172.21.15.32:0/2869710450 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff0bc0eacd0 con 0x7ff0cc06da20 2023-12-20T04:40:20.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.518+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2869710450 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff0cc164a50 con 0x7ff0b8065c60 2023-12-20T04:40:20.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.578+0000 7ff0c8ff9640 1 -- 172.21.15.32:0/2869710450 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7ff0c000c040 con 0x7ff0b8065c60 2023-12-20T04:40:20.579 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path /snap_test_dir1/dir2 2023-12-20T04:40:20.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2869710450 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff0b8065c60 msgr2=0x7ff0b8068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:20.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff0b8065c60 0x7ff0b8068120 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc0cdfb0 tx=0x7ff0c0008040 comp rx=0 tx=0).stop 2023-12-20T04:40:20.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2869710450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc06da20 msgr2=0x7ff0cc15c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:20.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc06da20 0x7ff0cc15c590 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc0ae040 tx=0x7ff0bc019150 comp rx=0 tx=0).stop 2023-12-20T04:40:20.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2869710450 shutdown_connections 2023-12-20T04:40:20.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff0b8065c60 0x7ff0b8068120 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0cc164670 0x7ff0cc15b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0cc070230 0x7ff0cc15aa10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 --2- 172.21.15.32:0/2869710450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0cc06da20 0x7ff0cc15c590 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2869710450 >> 172.21.15.32:0/2869710450 conn(0x7ff0cc06a510 msgr2=0x7ff0cc0c37f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:20.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2869710450 shutdown_connections 2023-12-20T04:40:20.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.582+0000 7ff0d23f0640 1 -- 172.21.15.32:0/2869710450 wait complete. 2023-12-20T04:40:20.600 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir2 2023-12-20T04:40:20.601 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir3 1M 2023-12-20T04:40:20.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.746+0000 7f7ee220b640 1 Processor -- start 2023-12-20T04:40:20.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.746+0000 7f7ee220b640 1 -- start start 2023-12-20T04:40:20.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7ee220b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7edc157480 0x7f7edc0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7ee220b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7edc0cd4c0 0x7f7edc0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7ee220b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7edc0b1100 con 0x7f7edc0b0760 2023-12-20T04:40:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7ee220b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7edc0cde20 con 0x7f7edc157480 2023-12-20T04:40:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7ee220b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7edc0c8e30 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7edb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7edc157480 0x7f7edc0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7edb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7edc157480 0x7f7edc0c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44098/0 (socket says 172.21.15.32:44098) 2023-12-20T04:40:20.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7edb7fe640 1 -- 172.21.15.32:0/3205391714 learned_addr learned my addr 172.21.15.32:0/3205391714 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:20.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7edaffd640 1 --1- 172.21.15.32:0/3205391714 >> v1:172.21.15.32:6789/0 conn(0x7f7edc0b0760 0x7f7edc15d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58946/0 (socket says 172.21.15.32:58946) 2023-12-20T04:40:20.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7edbfff640 1 --2- 172.21.15.32:0/3205391714 >> v2:172.21.15.39:3300/0 conn(0x7f7edc0cd4c0 0x7f7edc0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7edb7fe640 1 -- 172.21.15.32:0/3205391714 >> v2:172.21.15.39:3300/0 conn(0x7f7edc0cd4c0 msgr2=0x7f7edc0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:20.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7edb7fe640 1 --2- 172.21.15.32:0/3205391714 >> v2:172.21.15.39:3300/0 conn(0x7f7edc0cd4c0 0x7f7edc0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7edb7fe640 1 -- 172.21.15.32:0/3205391714 >> v1:172.21.15.32:6789/0 conn(0x7f7edc0b0760 legacy=0x7f7edc15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:20.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.750+0000 7f7edb7fe640 1 -- 172.21.15.32:0/3205391714 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7edc0ce070 con 0x7f7edc157480 2023-12-20T04:40:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7edb7fe640 1 --2- 172.21.15.32:0/3205391714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc157480 0x7f7edc0c9840 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7edc071420 tx=0x7f7ecc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3c92c19e9d78bc server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7eda7fc640 1 -- 172.21.15.32:0/3205391714 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc0ad660 con 0x7f7edc157480 2023-12-20T04:40:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7eda7fc640 1 -- 172.21.15.32:0/3205391714 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ecc0ad7c0 con 0x7f7edc157480 2023-12-20T04:40:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7eda7fc640 1 -- 172.21.15.32:0/3205391714 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc0adb50 con 0x7f7edc157480 2023-12-20T04:40:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7ee220b640 1 -- 172.21.15.32:0/3205391714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc157480 msgr2=0x7f7edc0c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7ee220b640 1 --2- 172.21.15.32:0/3205391714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc157480 0x7f7edc0c9840 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f7edc071420 tx=0x7f7ecc01c440 comp rx=0 tx=0).stop 2023-12-20T04:40:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7ee220b640 1 -- 172.21.15.32:0/3205391714 shutdown_connections 2023-12-20T04:40:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7ee220b640 1 --2- 172.21.15.32:0/3205391714 >> v2:172.21.15.39:3300/0 conn(0x7f7edc0cd4c0 0x7f7edc0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7ee220b640 1 --2- 172.21.15.32:0/3205391714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc157480 0x7f7edc0c9840 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7ee220b640 1 -- 172.21.15.32:0/3205391714 >> 172.21.15.32:0/3205391714 conn(0x7f7edc06a510 msgr2=0x7f7edc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7ee220b640 1 -- 172.21.15.32:0/3205391714 shutdown_connections 2023-12-20T04:40:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7ee220b640 1 -- 172.21.15.32:0/3205391714 wait complete. 2023-12-20T04:40:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7ee220b640 1 Processor -- start 2023-12-20T04:40:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.754+0000 7f7ee220b640 1 -- start start 2023-12-20T04:40:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ee220b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7edc0b0760 0x7f7edc15aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ee220b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0cd4c0 0x7f7edc14ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ee220b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7edc157480 0x7f7edc1505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ee220b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc14f110 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ee220b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc150b20 con 0x7f7edc0b0760 2023-12-20T04:40:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ee220b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7edc156580 con 0x7f7edc157480 2023-12-20T04:40:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7edaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0cd4c0 0x7f7edc14ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7edaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0cd4c0 0x7f7edc14ebd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44112/0 (socket says 172.21.15.32:44112) 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7edaffd640 1 -- 172.21.15.32:0/1087917919 learned_addr learned my addr 172.21.15.32:0/1087917919 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7edb7fe640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7edc0b0760 0x7f7edc15aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7edbfff640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7edc157480 0x7f7edc1505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7edaffd640 1 -- 172.21.15.32:0/1087917919 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7edc157480 msgr2=0x7f7edc1505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7edaffd640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7edc157480 0x7f7edc1505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7edaffd640 1 -- 172.21.15.32:0/1087917919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7edc0b0760 msgr2=0x7f7edc15aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7edaffd640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7edc0b0760 0x7f7edc15aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7edaffd640 1 -- 172.21.15.32:0/1087917919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ecc0094d0 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7edaffd640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0cd4c0 0x7f7edc14ebd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000c430 tx=0x7f7ed000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ec3fff640 1 -- 172.21.15.32:0/1087917919 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed00074e0 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ec3fff640 1 -- 172.21.15.32:0/1087917919 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ed0007640 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ee220b640 1 -- 172.21.15.32:0/1087917919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7edc156860 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ec3fff640 1 -- 172.21.15.32:0/1087917919 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ed0004a10 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ee220b640 1 -- 172.21.15.32:0/1087917919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7edc150d50 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ee220b640 1 -- 172.21.15.32:0/1087917919 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7edc04e000 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.758+0000 7f7ec3fff640 1 -- 172.21.15.32:0/1087917919 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f7ed0004b70 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.762+0000 7f7ec3fff640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ec8067e70 0x7f7ec806a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.762+0000 7f7ec3fff640 1 -- 172.21.15.32:0/1087917919 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7ed0083150 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.762+0000 7f7edb7fe640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ec8067e70 0x7f7ec806a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:20.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.762+0000 7f7edb7fe640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ec8067e70 0x7f7ec806a330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0af000 tx=0x7f7ecc008040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:20.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.766+0000 7f7ec3fff640 1 -- 172.21.15.32:0/1087917919 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7ed00506f0 con 0x7f7edc0cd4c0 2023-12-20T04:40:20.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:20.942+0000 7f7ee220b640 1 -- 172.21.15.32:0/1087917919 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7edc0c9360 con 0x7f7ec8067e70 2023-12-20T04:40:21.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ec3fff640 1 -- 172.21.15.32:0/1087917919 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f7ecc0bc0c0 con 0x7f7ec8067e70 2023-12-20T04:40:21.004 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path /snap_test_dir1/dir3 2023-12-20T04:40:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 -- 172.21.15.32:0/1087917919 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ec8067e70 msgr2=0x7f7ec806a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ec8067e70 0x7f7ec806a330 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc0af000 tx=0x7f7ecc008040 comp rx=0 tx=0).stop 2023-12-20T04:40:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 -- 172.21.15.32:0/1087917919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0cd4c0 msgr2=0x7f7edc14ebd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0cd4c0 0x7f7edc14ebd0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7f7ed000c430 tx=0x7f7ed000c8f0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 -- 172.21.15.32:0/1087917919 shutdown_connections 2023-12-20T04:40:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ec8067e70 0x7f7ec806a330 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7edc157480 0x7f7edc1505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7edc0cd4c0 0x7f7edc14ebd0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 --2- 172.21.15.32:0/1087917919 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7edc0b0760 0x7f7edc15aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 -- 172.21.15.32:0/1087917919 >> 172.21.15.32:0/1087917919 conn(0x7f7edc06a510 msgr2=0x7f7edc0ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:21.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 -- 172.21.15.32:0/1087917919 shutdown_connections 2023-12-20T04:40:21.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.006+0000 7f7ee220b640 1 -- 172.21.15.32:0/1087917919 wait complete. 2023-12-20T04:40:21.025 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir3 2023-12-20T04:40:21.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir4 1M 2023-12-20T04:40:21.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b2042d640 1 Processor -- start 2023-12-20T04:40:21.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b2042d640 1 -- start start 2023-12-20T04:40:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b2042d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0b1815abb0 0x7f0b180caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b2042d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b180b0cb0 0x7f0b1815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b2042d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0b1815b6a0 con 0x7f0b18070230 2023-12-20T04:40:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b2042d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b1806e160 con 0x7f0b180b0cb0 2023-12-20T04:40:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b2042d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0b1815c070 con 0x7f0b1815abb0 2023-12-20T04:40:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b1d9c8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b180b0cb0 0x7f0b1815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:21.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b1d9c8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0b180b0cb0 0x7f0b1815bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44116/0 (socket says 172.21.15.32:44116) 2023-12-20T04:40:21.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b1d9c8640 1 -- 172.21.15.32:0/339703088 learned_addr learned my addr 172.21.15.32:0/339703088 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:21.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b1e1c9640 1 --2- 172.21.15.32:0/339703088 >> v2:172.21.15.39:3300/0 conn(0x7f0b1815abb0 0x7f0b180caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:21.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b1d9c8640 1 -- 172.21.15.32:0/339703088 >> v2:172.21.15.39:3300/0 conn(0x7f0b1815abb0 msgr2=0x7f0b180caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:21.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b1d9c8640 1 --2- 172.21.15.32:0/339703088 >> v2:172.21.15.39:3300/0 conn(0x7f0b1815abb0 0x7f0b180caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b1d9c8640 1 -- 172.21.15.32:0/339703088 >> v1:172.21.15.32:6789/0 conn(0x7f0b18070230 legacy=0x7f0b1806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.174+0000 7f0b1d9c8640 1 -- 172.21.15.32:0/339703088 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b1806d8c0 con 0x7f0b180b0cb0 2023-12-20T04:40:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b1d9c8640 1 --2- 172.21.15.32:0/339703088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b180b0cb0 0x7f0b1815bb30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0b180713f0 tx=0x7f0b0801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6458d11074a98f72 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b1d1c7640 1 -- 172.21.15.32:0/339703088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b080aa660 con 0x7f0b180b0cb0 2023-12-20T04:40:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b1d1c7640 1 -- 172.21.15.32:0/339703088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b080aa7c0 con 0x7f0b180b0cb0 2023-12-20T04:40:21.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b1d1c7640 1 -- 172.21.15.32:0/339703088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b080aab50 con 0x7f0b180b0cb0 2023-12-20T04:40:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b2042d640 1 -- 172.21.15.32:0/339703088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b180b0cb0 msgr2=0x7f0b1815bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b2042d640 1 --2- 172.21.15.32:0/339703088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b180b0cb0 0x7f0b1815bb30 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f0b180713f0 tx=0x7f0b0801c500 comp rx=0 tx=0).stop 2023-12-20T04:40:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b2042d640 1 -- 172.21.15.32:0/339703088 shutdown_connections 2023-12-20T04:40:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b2042d640 1 --2- 172.21.15.32:0/339703088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b180b0cb0 0x7f0b1815bb30 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b2042d640 1 --2- 172.21.15.32:0/339703088 >> v2:172.21.15.39:3300/0 conn(0x7f0b1815abb0 0x7f0b180caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b2042d640 1 -- 172.21.15.32:0/339703088 >> 172.21.15.32:0/339703088 conn(0x7f0b1806a510 msgr2=0x7f0b180ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b2042d640 1 -- 172.21.15.32:0/339703088 shutdown_connections 2023-12-20T04:40:21.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.178+0000 7f0b2042d640 1 -- 172.21.15.32:0/339703088 wait complete. 2023-12-20T04:40:21.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b2042d640 1 Processor -- start 2023-12-20T04:40:21.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b2042d640 1 -- start start 2023-12-20T04:40:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b2042d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18070230 0x7f0b1813d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b2042d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0b180b0cb0 0x7f0b180c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b2042d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b1815abb0 0x7f0b18158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b2042d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b1813d790 con 0x7f0b18070230 2023-12-20T04:40:21.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b2042d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180c7fd0 con 0x7f0b1815abb0 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b2042d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0b180c82b0 con 0x7f0b180b0cb0 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b1e1c9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18070230 0x7f0b1813d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b1e1c9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18070230 0x7f0b1813d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44132/0 (socket says 172.21.15.32:44132) 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b1e1c9640 1 -- 172.21.15.32:0/3608659902 learned_addr learned my addr 172.21.15.32:0/3608659902 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b1e1c9640 1 -- 172.21.15.32:0/3608659902 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0b180b0cb0 msgr2=0x7f0b180c9070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b1e1c9640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0b180b0cb0 0x7f0b180c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b1e1c9640 1 -- 172.21.15.32:0/3608659902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b1815abb0 msgr2=0x7f0b18158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b1e1c9640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b1815abb0 0x7f0b18158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b1e1c9640 1 -- 172.21.15.32:0/3608659902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0b080094d0 con 0x7f0b18070230 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b1e1c9640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18070230 0x7f0b1813d250 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400d5f0 tx=0x7f0b1400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b0effd640 1 -- 172.21.15.32:0/3608659902 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b140125f0 con 0x7f0b18070230 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b0effd640 1 -- 172.21.15.32:0/3608659902 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0b14012750 con 0x7f0b18070230 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b0effd640 1 -- 172.21.15.32:0/3608659902 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0b14012ae0 con 0x7f0b18070230 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b2042d640 1 -- 172.21.15.32:0/3608659902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0b180c8590 con 0x7f0b18070230 2023-12-20T04:40:21.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.182+0000 7f0b2042d640 1 -- 172.21.15.32:0/3608659902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0b180c8b50 con 0x7f0b18070230 2023-12-20T04:40:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.186+0000 7f0b2042d640 1 -- 172.21.15.32:0/3608659902 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0b1806acf0 con 0x7f0b18070230 2023-12-20T04:40:21.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.186+0000 7f0b0effd640 1 -- 172.21.15.32:0/3608659902 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f0b14012c40 con 0x7f0b18070230 2023-12-20T04:40:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.186+0000 7f0b0effd640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0afc065c60 0x7f0afc068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.186+0000 7f0b0effd640 1 -- 172.21.15.32:0/3608659902 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0b14083bb0 con 0x7f0b18070230 2023-12-20T04:40:21.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.186+0000 7f0b1d9c8640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0afc065c60 0x7f0afc068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:21.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.186+0000 7f0b1d9c8640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0afc065c60 0x7f0afc068120 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b080ae000 tx=0x7f0b08008040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:21.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.190+0000 7f0b0effd640 1 -- 172.21.15.32:0/3608659902 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0b14051150 con 0x7f0b18070230 2023-12-20T04:40:21.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.366+0000 7f0b2042d640 1 -- 172.21.15.32:0/3608659902 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0b1806cf00 con 0x7f0afc065c60 2023-12-20T04:40:21.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b0effd640 1 -- 172.21.15.32:0/3608659902 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f0b080aa930 con 0x7f0afc065c60 2023-12-20T04:40:21.429 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path /snap_test_dir1/dir4 2023-12-20T04:40:21.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 -- 172.21.15.32:0/3608659902 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0afc065c60 msgr2=0x7f0afc068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0afc065c60 0x7f0afc068120 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0b080ae000 tx=0x7f0b08008040 comp rx=0 tx=0).stop 2023-12-20T04:40:21.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 -- 172.21.15.32:0/3608659902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18070230 msgr2=0x7f0b1813d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18070230 0x7f0b1813d250 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f0b1400d5f0 tx=0x7f0b1400dab0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 -- 172.21.15.32:0/3608659902 shutdown_connections 2023-12-20T04:40:21.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0afc065c60 0x7f0afc068120 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0b1815abb0 0x7f0b18158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0b180b0cb0 0x7f0b180c9070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 --2- 172.21.15.32:0/3608659902 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0b18070230 0x7f0b1813d250 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 -- 172.21.15.32:0/3608659902 >> 172.21.15.32:0/3608659902 conn(0x7f0b1806a510 msgr2=0x7f0b180b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:21.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 -- 172.21.15.32:0/3608659902 shutdown_connections 2023-12-20T04:40:21.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.430+0000 7f0b2042d640 1 -- 172.21.15.32:0/3608659902 wait complete. 2023-12-20T04:40:21.449 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir4 2023-12-20T04:40:21.449 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir5 1M 2023-12-20T04:40:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.598+0000 7f78947f0640 1 Processor -- start 2023-12-20T04:40:21.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.598+0000 7f78947f0640 1 -- start start 2023-12-20T04:40:21.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f78947f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f788c15abb0 0x7f788c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f78947f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f788c0b0cb0 0x7f788c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f78947f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f788c15b6a0 con 0x7f788c070230 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f78947f0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f788c06e160 con 0x7f788c15abb0 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f78947f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c15c070 con 0x7f788c0b0cb0 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f789258c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f788c15abb0 0x7f788c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f789258c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f788c15abb0 0x7f788c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44146/0 (socket says 172.21.15.32:44146) 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f789258c640 1 -- 172.21.15.32:0/2903656742 learned_addr learned my addr 172.21.15.32:0/2903656742 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f7891d8b640 1 --2- 172.21.15.32:0/2903656742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f788c0b0cb0 0x7f788c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f789258c640 1 -- 172.21.15.32:0/2903656742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f788c0b0cb0 msgr2=0x7f788c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f789258c640 1 --2- 172.21.15.32:0/2903656742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f788c0b0cb0 0x7f788c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f789258c640 1 -- 172.21.15.32:0/2903656742 >> v1:172.21.15.39:6789/0 conn(0x7f788c070230 legacy=0x7f788c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.602+0000 7f789258c640 1 -- 172.21.15.32:0/2903656742 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f788c06d8c0 con 0x7f788c15abb0 2023-12-20T04:40:21.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f789258c640 1 --2- 172.21.15.32:0/2903656742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788c15abb0 0x7f788c0caaf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7888009870 tx=0x7f788801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2915e432941a7b8a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:21.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f789158a640 1 -- 172.21.15.32:0/2903656742 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880a9400 con 0x7f788c15abb0 2023-12-20T04:40:21.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f789158a640 1 -- 172.21.15.32:0/2903656742 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78880a9560 con 0x7f788c15abb0 2023-12-20T04:40:21.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f789158a640 1 -- 172.21.15.32:0/2903656742 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880a98f0 con 0x7f788c15abb0 2023-12-20T04:40:21.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f78947f0640 1 -- 172.21.15.32:0/2903656742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788c15abb0 msgr2=0x7f788c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f78947f0640 1 --2- 172.21.15.32:0/2903656742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788c15abb0 0x7f788c0caaf0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f7888009870 tx=0x7f788801c540 comp rx=0 tx=0).stop 2023-12-20T04:40:21.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f78947f0640 1 -- 172.21.15.32:0/2903656742 shutdown_connections 2023-12-20T04:40:21.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f78947f0640 1 --2- 172.21.15.32:0/2903656742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f788c0b0cb0 0x7f788c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f78947f0640 1 --2- 172.21.15.32:0/2903656742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788c15abb0 0x7f788c0caaf0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f78947f0640 1 -- 172.21.15.32:0/2903656742 >> 172.21.15.32:0/2903656742 conn(0x7f788c06a510 msgr2=0x7f788c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:21.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f78947f0640 1 -- 172.21.15.32:0/2903656742 shutdown_connections 2023-12-20T04:40:21.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f78947f0640 1 -- 172.21.15.32:0/2903656742 wait complete. 2023-12-20T04:40:21.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f78947f0640 1 Processor -- start 2023-12-20T04:40:21.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.606+0000 7f78947f0640 1 -- start start 2023-12-20T04:40:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f78947f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788c070230 0x7f788c14f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f78947f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f788c0b0cb0 0x7f788c14eab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f78947f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f788c15abb0 0x7f788c1493a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f78947f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c14fa70 con 0x7f788c070230 2023-12-20T04:40:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f78947f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f788c14eff0 con 0x7f788c15abb0 2023-12-20T04:40:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f78947f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f788c1498e0 con 0x7f788c0b0cb0 2023-12-20T04:40:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f789258c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788c070230 0x7f788c14f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:21.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f789258c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788c070230 0x7f788c14f530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44150/0 (socket says 172.21.15.32:44150) 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f789258c640 1 -- 172.21.15.32:0/2276910737 learned_addr learned my addr 172.21.15.32:0/2276910737 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f7891d8b640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f788c0b0cb0 0x7f788c14eab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f7892d8d640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f788c15abb0 0x7f788c1493a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f789258c640 1 -- 172.21.15.32:0/2276910737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f788c0b0cb0 msgr2=0x7f788c14eab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f789258c640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f788c0b0cb0 0x7f788c14eab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f789258c640 1 -- 172.21.15.32:0/2276910737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f788c15abb0 msgr2=0x7f788c1493a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f789258c640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f788c15abb0 0x7f788c1493a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f789258c640 1 -- 172.21.15.32:0/2276910737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f78880094d0 con 0x7f788c070230 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f7891d8b640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f788c0b0cb0 0x7f788c14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f789258c640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788c070230 0x7f788c14f530 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f788801a170 tx=0x7f788801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f7882ffd640 1 -- 172.21.15.32:0/2276910737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880a9400 con 0x7f788c070230 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f7882ffd640 1 -- 172.21.15.32:0/2276910737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78880b1820 con 0x7f788c070230 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f7882ffd640 1 -- 172.21.15.32:0/2276910737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78880b1bd0 con 0x7f788c070230 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f78947f0640 1 -- 172.21.15.32:0/2276910737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f788c14aa40 con 0x7f788c070230 2023-12-20T04:40:21.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f78947f0640 1 -- 172.21.15.32:0/2276910737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f788c14b000 con 0x7f788c070230 2023-12-20T04:40:21.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f78947f0640 1 -- 172.21.15.32:0/2276910737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f788c04e000 con 0x7f788c070230 2023-12-20T04:40:21.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.610+0000 7f7882ffd640 1 -- 172.21.15.32:0/2276910737 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f78880b1d30 con 0x7f788c070230 2023-12-20T04:40:21.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.614+0000 7f7882ffd640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7870065cb0 0x7f7870068170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:21.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.614+0000 7f7882ffd640 1 -- 172.21.15.32:0/2276910737 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f788811e200 con 0x7f788c070230 2023-12-20T04:40:21.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.614+0000 7f7891d8b640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7870065cb0 0x7f7870068170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:21.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.614+0000 7f7891d8b640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7870065cb0 0x7f7870068170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f788c1f8c50 tx=0x7f787c008040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:21.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.618+0000 7f7882ffd640 1 -- 172.21.15.32:0/2276910737 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f78880eb830 con 0x7f788c070230 2023-12-20T04:40:21.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.798+0000 7f78947f0640 1 -- 172.21.15.32:0/2276910737 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f788c06cf00 con 0x7f7870065cb0 2023-12-20T04:40:21.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.858+0000 7f7882ffd640 1 -- 172.21.15.32:0/2276910737 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f787c0098e0 con 0x7f7870065cb0 2023-12-20T04:40:21.859 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path /snap_test_dir1/dir5 2023-12-20T04:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 -- 172.21.15.32:0/2276910737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7870065cb0 msgr2=0x7f7870068170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7870065cb0 0x7f7870068170 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f788c1f8c50 tx=0x7f787c008040 comp rx=0 tx=0).stop 2023-12-20T04:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 -- 172.21.15.32:0/2276910737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788c070230 msgr2=0x7f788c14f530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788c070230 0x7f788c14f530 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f788801a170 tx=0x7f788801ad80 comp rx=0 tx=0).stop 2023-12-20T04:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 -- 172.21.15.32:0/2276910737 shutdown_connections 2023-12-20T04:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7870065cb0 0x7f7870068170 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f788c15abb0 0x7f788c1493a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f788c0b0cb0 0x7f788c14eab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 --2- 172.21.15.32:0/2276910737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f788c070230 0x7f788c14f530 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 -- 172.21.15.32:0/2276910737 >> 172.21.15.32:0/2276910737 conn(0x7f788c06a510 msgr2=0x7f788c0c47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:21.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 -- 172.21.15.32:0/2276910737 shutdown_connections 2023-12-20T04:40:21.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:21.862+0000 7f78947f0640 1 -- 172.21.15.32:0/2276910737 wait complete. 2023-12-20T04:40:21.880 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir5 2023-12-20T04:40:21.881 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir6 1M 2023-12-20T04:40:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.030+0000 7fc5b8c8c640 1 Processor -- start 2023-12-20T04:40:22.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.030+0000 7fc5b8c8c640 1 -- start start 2023-12-20T04:40:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.030+0000 7fc5b8c8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc5b4157480 0x7fc5b40c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.030+0000 7fc5b8c8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b40cde20 con 0x7fc5b40b0760 2023-12-20T04:40:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.030+0000 7fc5b8c8c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5b40c8e30 con 0x7fc5b40cd4c0 2023-12-20T04:40:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.030+0000 7fc5b8c8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5b415e480 con 0x7fc5b4157480 2023-12-20T04:40:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.030+0000 7fc5b1d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc5b40b0760 0x7fc5b415d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58972/0 (socket says 172.21.15.32:58972) 2023-12-20T04:40:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.030+0000 7fc5b1d9b640 1 -- 172.21.15.32:0/2869090860 learned_addr learned my addr 172.21.15.32:0/2869090860 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:22.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.034+0000 7fc5b259c640 1 --2- 172.21.15.32:0/2869090860 >> v2:172.21.15.39:3300/0 conn(0x7fc5b4157480 0x7fc5b40c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.034+0000 7fc5b159a640 1 -- 172.21.15.32:0/2869090860 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1768851569 0 0) 0x7fc5b40cde20 con 0x7fc5b40b0760 2023-12-20T04:40:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.034+0000 7fc5b159a640 1 -- 172.21.15.32:0/2869090860 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc59c003530 con 0x7fc5b40b0760 2023-12-20T04:40:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.034+0000 7fc5b259c640 1 -- 172.21.15.32:0/2869090860 >> v1:172.21.15.39:6789/0 conn(0x7fc5b40cd4c0 legacy=0x7fc5b415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.034+0000 7fc5b259c640 1 -- 172.21.15.32:0/2869090860 >> v1:172.21.15.32:6789/0 conn(0x7fc5b40b0760 legacy=0x7fc5b415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.034+0000 7fc5b259c640 1 -- 172.21.15.32:0/2869090860 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5b40ce070 con 0x7fc5b4157480 2023-12-20T04:40:22.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.034+0000 7fc5b259c640 1 --2- 172.21.15.32:0/2869090860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5b4157480 0x7fc5b40c9840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40097c0 tx=0x7fc5a401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cc83d78f956e2040 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:22.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.034+0000 7fc5b159a640 1 -- 172.21.15.32:0/2869090860 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a40ac620 con 0x7fc5b4157480 2023-12-20T04:40:22.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.034+0000 7fc5b159a640 1 -- 172.21.15.32:0/2869090860 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5a40ac780 con 0x7fc5b4157480 2023-12-20T04:40:22.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.034+0000 7fc5b159a640 1 -- 172.21.15.32:0/2869090860 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5a40acb10 con 0x7fc5b4157480 2023-12-20T04:40:22.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2869090860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5b4157480 msgr2=0x7fc5b40c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 --2- 172.21.15.32:0/2869090860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5b4157480 0x7fc5b40c9840 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40097c0 tx=0x7fc5a401c3c0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2869090860 shutdown_connections 2023-12-20T04:40:22.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 --2- 172.21.15.32:0/2869090860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5b4157480 0x7fc5b40c9840 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2869090860 >> 172.21.15.32:0/2869090860 conn(0x7fc5b406a510 msgr2=0x7fc5b40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:22.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2869090860 shutdown_connections 2023-12-20T04:40:22.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2869090860 wait complete. 2023-12-20T04:40:22.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 Processor -- start 2023-12-20T04:40:22.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 -- start start 2023-12-20T04:40:22.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5b40b0760 0x7fc5b41f6030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc5b40cd4c0 0x7fc5b41f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5b4157480 0x7fc5b414d1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b414e7e0 con 0x7fc5b4157480 2023-12-20T04:40:22.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b4163510 con 0x7fc5b40b0760 2023-12-20T04:40:22.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b8c8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5b41f9050 con 0x7fc5b40cd4c0 2023-12-20T04:40:22.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5b4157480 0x7fc5b414d1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5b4157480 0x7fc5b414d1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44166/0 (socket says 172.21.15.32:44166) 2023-12-20T04:40:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b2d9d640 1 -- 172.21.15.32:0/2884167482 learned_addr learned my addr 172.21.15.32:0/2884167482 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b1d9b640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc5b40cd4c0 0x7fc5b41f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.038+0000 7fc5b259c640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5b40b0760 0x7fc5b41f6030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5b2d9d640 1 -- 172.21.15.32:0/2884167482 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc5b40cd4c0 msgr2=0x7fc5b41f8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5b2d9d640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc5b40cd4c0 0x7fc5b41f8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5b2d9d640 1 -- 172.21.15.32:0/2884167482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5b40b0760 msgr2=0x7fc5b41f6030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5b2d9d640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5b40b0760 0x7fc5b41f6030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5b2d9d640 1 -- 172.21.15.32:0/2884167482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5a4009420 con 0x7fc5b4157480 2023-12-20T04:40:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5b259c640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5b40b0760 0x7fc5b41f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:22.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5b2d9d640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5b4157480 0x7fc5b414d1e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c7a0 tx=0x7fc5ac00cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5a2ffd640 1 -- 172.21.15.32:0/2884167482 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac003b80 con 0x7fc5b4157480 2023-12-20T04:40:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5a2ffd640 1 -- 172.21.15.32:0/2884167482 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5ac003ce0 con 0x7fc5b4157480 2023-12-20T04:40:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5a2ffd640 1 -- 172.21.15.32:0/2884167482 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5ac00aa90 con 0x7fc5b4157480 2023-12-20T04:40:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2884167482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5b414d720 con 0x7fc5b4157480 2023-12-20T04:40:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2884167482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5b41f9560 con 0x7fc5b4157480 2023-12-20T04:40:22.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2884167482 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5b4163650 con 0x7fc5b4157480 2023-12-20T04:40:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5a2ffd640 1 -- 172.21.15.32:0/2884167482 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fc5ac00abf0 con 0x7fc5b4157480 2023-12-20T04:40:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5a2ffd640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc594065c60 0x7fc594068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5a2ffd640 1 -- 172.21.15.32:0/2884167482 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fc5ac083470 con 0x7fc5b4157480 2023-12-20T04:40:22.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.042+0000 7fc5b259c640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc594065c60 0x7fc594068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.046+0000 7fc5b259c640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc594065c60 0x7fc594068120 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40ae000 tx=0x7fc5a4008040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:22.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.046+0000 7fc5a2ffd640 1 -- 172.21.15.32:0/2884167482 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc5b4163650 con 0x7fc5b4157480 2023-12-20T04:40:22.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.226+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2884167482 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fc5b4163650 con 0x7fc594065c60 2023-12-20T04:40:22.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5a2ffd640 1 -- 172.21.15.32:0/2884167482 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fc5b4163650 con 0x7fc594065c60 2023-12-20T04:40:22.291 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path /snap_test_dir1/dir6 2023-12-20T04:40:22.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2884167482 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc594065c60 msgr2=0x7fc594068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc594065c60 0x7fc594068120 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fc5a40ae000 tx=0x7fc5a4008040 comp rx=0 tx=0).stop 2023-12-20T04:40:22.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2884167482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5b4157480 msgr2=0x7fc5b414d1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5b4157480 0x7fc5b414d1e0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac00c7a0 tx=0x7fc5ac00cc60 comp rx=0 tx=0).stop 2023-12-20T04:40:22.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2884167482 shutdown_connections 2023-12-20T04:40:22.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc594065c60 0x7fc594068120 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc5b4157480 0x7fc5b414d1e0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc5b40cd4c0 0x7fc5b41f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 --2- 172.21.15.32:0/2884167482 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc5b40b0760 0x7fc5b41f6030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2884167482 >> 172.21.15.32:0/2884167482 conn(0x7fc5b406a510 msgr2=0x7fc5b40c3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:22.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2884167482 shutdown_connections 2023-12-20T04:40:22.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.294+0000 7fc5b8c8c640 1 -- 172.21.15.32:0/2884167482 wait complete. 2023-12-20T04:40:22.313 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir6 2023-12-20T04:40:22.313 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir7 1M 2023-12-20T04:40:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8abf0640 1 Processor -- start 2023-12-20T04:40:22.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8abf0640 1 -- start start 2023-12-20T04:40:22.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8abf0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2d840cb050 0x7f2d840cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8abf0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8406a1f0 con 0x7f2d8406d170 2023-12-20T04:40:22.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8abf0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d8406a440 con 0x7f2d84065d40 2023-12-20T04:40:22.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8abf0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d8406a720 con 0x7f2d840cb050 2023-12-20T04:40:22.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8918d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2d8406d170 0x7f2d840c9720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58974/0 (socket says 172.21.15.32:58974) 2023-12-20T04:40:22.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8918d640 1 -- 172.21.15.32:0/1308847814 learned_addr learned my addr 172.21.15.32:0/1308847814 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:22.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8898c640 1 --2- 172.21.15.32:0/1308847814 >> v2:172.21.15.39:3300/0 conn(0x7f2d840cb050 0x7f2d840cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d837fe640 1 -- 172.21.15.32:0/1308847814 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4147892184 0 0) 0x7f2d8406a1f0 con 0x7f2d8406d170 2023-12-20T04:40:22.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d837fe640 1 -- 172.21.15.32:0/1308847814 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d74003530 con 0x7f2d8406d170 2023-12-20T04:40:22.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d837fe640 1 -- 172.21.15.32:0/1308847814 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 994169063 0 0) 0x7f2d74003530 con 0x7f2d8406d170 2023-12-20T04:40:22.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8898c640 1 -- 172.21.15.32:0/1308847814 >> v1:172.21.15.39:6789/0 conn(0x7f2d84065d40 legacy=0x7f2d8406ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8898c640 1 -- 172.21.15.32:0/1308847814 >> v1:172.21.15.32:6789/0 conn(0x7f2d8406d170 legacy=0x7f2d840c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.462+0000 7f2d8898c640 1 -- 172.21.15.32:0/1308847814 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d8406aa00 con 0x7f2d840cb050 2023-12-20T04:40:22.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.466+0000 7f2d8898c640 1 --2- 172.21.15.32:0/1308847814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d840cb050 0x7f2d840cb430 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2d780097c0 tx=0x7f2d7801c370 comp rx=0 tx=0).ready entity=mon.1 client_cookie=606a6d87d4d1aacf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:22.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.466+0000 7f2d837fe640 1 -- 172.21.15.32:0/1308847814 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d7c001a00 con 0x7f2d840cb050 2023-12-20T04:40:22.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.466+0000 7f2d837fe640 1 -- 172.21.15.32:0/1308847814 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d780a7050 con 0x7f2d840cb050 2023-12-20T04:40:22.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.466+0000 7f2d837fe640 1 -- 172.21.15.32:0/1308847814 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d780ab620 con 0x7f2d840cb050 2023-12-20T04:40:22.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.466+0000 7f2d8abf0640 1 -- 172.21.15.32:0/1308847814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d840cb050 msgr2=0x7f2d840cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.466+0000 7f2d8abf0640 1 --2- 172.21.15.32:0/1308847814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d840cb050 0x7f2d840cb430 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2d780097c0 tx=0x7f2d7801c370 comp rx=0 tx=0).stop 2023-12-20T04:40:22.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.466+0000 7f2d8abf0640 1 -- 172.21.15.32:0/1308847814 shutdown_connections 2023-12-20T04:40:22.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.466+0000 7f2d8abf0640 1 --2- 172.21.15.32:0/1308847814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d840cb050 0x7f2d840cb430 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.466+0000 7f2d8abf0640 1 -- 172.21.15.32:0/1308847814 >> 172.21.15.32:0/1308847814 conn(0x7f2d840cf8d0 msgr2=0x7f2d840cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:22.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.466+0000 7f2d8abf0640 1 -- 172.21.15.32:0/1308847814 shutdown_connections 2023-12-20T04:40:22.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.466+0000 7f2d8abf0640 1 -- 172.21.15.32:0/1308847814 wait complete. 2023-12-20T04:40:22.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8abf0640 1 Processor -- start 2023-12-20T04:40:22.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8abf0640 1 -- start start 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8abf0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d84065d40 0x7f2d840c7ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8abf0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d8406d170 0x7f2d8414b550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8abf0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d840cb050 0x7f2d8414abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8abf0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840c6d40 con 0x7f2d84065d40 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8abf0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840c8280 con 0x7f2d840cb050 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8abf0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d840c8560 con 0x7f2d8406d170 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8898c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d84065d40 0x7f2d840c7ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8898c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d84065d40 0x7f2d840c7ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44176/0 (socket says 172.21.15.32:44176) 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8898c640 1 -- 172.21.15.32:0/3582223900 learned_addr learned my addr 172.21.15.32:0/3582223900 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8918d640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d840cb050 0x7f2d8414abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d83fff640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d8406d170 0x7f2d8414b550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8898c640 1 -- 172.21.15.32:0/3582223900 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d8406d170 msgr2=0x7f2d8414b550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8898c640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d8406d170 0x7f2d8414b550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8898c640 1 -- 172.21.15.32:0/3582223900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d840cb050 msgr2=0x7f2d8414abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8898c640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d840cb050 0x7f2d8414abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8898c640 1 -- 172.21.15.32:0/3582223900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d78009420 con 0x7f2d84065d40 2023-12-20T04:40:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d83fff640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d8406d170 0x7f2d8414b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8898c640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d84065d40 0x7f2d840c7ca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2d780097c0 tx=0x7f2d7801abb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d817fa640 1 -- 172.21.15.32:0/3582223900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78004020 con 0x7f2d84065d40 2023-12-20T04:40:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d817fa640 1 -- 172.21.15.32:0/3582223900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d78004180 con 0x7f2d84065d40 2023-12-20T04:40:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d817fa640 1 -- 172.21.15.32:0/3582223900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d78003620 con 0x7f2d84065d40 2023-12-20T04:40:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8abf0640 1 -- 172.21.15.32:0/3582223900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d840c69d0 con 0x7f2d84065d40 2023-12-20T04:40:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.470+0000 7f2d8abf0640 1 -- 172.21.15.32:0/3582223900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d840c8840 con 0x7f2d84065d40 2023-12-20T04:40:22.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.474+0000 7f2d8abf0640 1 -- 172.21.15.32:0/3582223900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d8404e000 con 0x7f2d84065d40 2023-12-20T04:40:22.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.474+0000 7f2d817fa640 1 -- 172.21.15.32:0/3582223900 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f2d78003780 con 0x7f2d84065d40 2023-12-20T04:40:22.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.474+0000 7f2d817fa640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d68065c60 0x7f2d68068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.474+0000 7f2d817fa640 1 -- 172.21.15.32:0/3582223900 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f2d7811d110 con 0x7f2d84065d40 2023-12-20T04:40:22.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.474+0000 7f2d83fff640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d68065c60 0x7f2d68068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.474+0000 7f2d83fff640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d68065c60 0x7f2d68068120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d840c90f0 tx=0x7f2d70009040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:22.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.478+0000 7f2d817fa640 1 -- 172.21.15.32:0/3582223900 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2d780ea5a0 con 0x7f2d84065d40 2023-12-20T04:40:22.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.662+0000 7f2d8abf0640 1 -- 172.21.15.32:0/3582223900 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f2d840c6e80 con 0x7f2d68065c60 2023-12-20T04:40:22.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.726+0000 7f2d817fa640 1 -- 172.21.15.32:0/3582223900 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f2d70001350 con 0x7f2d68065c60 2023-12-20T04:40:22.725 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path /snap_test_dir1/dir7 2023-12-20T04:40:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.726+0000 7f2d8abf0640 1 -- 172.21.15.32:0/3582223900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d68065c60 msgr2=0x7f2d68068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.726+0000 7f2d8abf0640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d68065c60 0x7f2d68068120 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2d840c90f0 tx=0x7f2d70009040 comp rx=0 tx=0).stop 2023-12-20T04:40:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.726+0000 7f2d8abf0640 1 -- 172.21.15.32:0/3582223900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d84065d40 msgr2=0x7f2d840c7ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.726+0000 7f2d8abf0640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d84065d40 0x7f2d840c7ca0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7f2d780097c0 tx=0x7f2d7801abb0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.726+0000 7f2d8abf0640 1 -- 172.21.15.32:0/3582223900 shutdown_connections 2023-12-20T04:40:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.726+0000 7f2d8abf0640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d68065c60 0x7f2d68068120 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.726+0000 7f2d8abf0640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d840cb050 0x7f2d8414abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.726+0000 7f2d8abf0640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d8406d170 0x7f2d8414b550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.726+0000 7f2d8abf0640 1 --2- 172.21.15.32:0/3582223900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d84065d40 0x7f2d840c7ca0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.726+0000 7f2d8abf0640 1 -- 172.21.15.32:0/3582223900 >> 172.21.15.32:0/3582223900 conn(0x7f2d840cf8d0 msgr2=0x7f2d841582c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:22.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.730+0000 7f2d8abf0640 1 -- 172.21.15.32:0/3582223900 shutdown_connections 2023-12-20T04:40:22.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.730+0000 7f2d8abf0640 1 -- 172.21.15.32:0/3582223900 wait complete. 2023-12-20T04:40:22.746 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir7 2023-12-20T04:40:22.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir8 1M 2023-12-20T04:40:22.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7ace2a640 1 Processor -- start 2023-12-20T04:40:22.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7ace2a640 1 -- start start 2023-12-20T04:40:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7ace2a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe7a80cf3e0 0x7fe7a80cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7ace2a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80d1580 0x7fe7a80d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7ace2a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7a80cfd80 con 0x7fe7a815ccf0 2023-12-20T04:40:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7ace2a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7a80ce2f0 con 0x7fe7a80cf3e0 2023-12-20T04:40:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7ace2a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80d0b70 con 0x7fe7a80d1580 2023-12-20T04:40:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80d1580 0x7fe7a80d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7a6ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe7a80cf3e0 0x7fe7a80cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7a7fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80d1580 0x7fe7a80d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:41836/0 (socket says 172.21.15.32:41836) 2023-12-20T04:40:22.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7a7fff640 1 -- 172.21.15.32:0/1833603742 learned_addr learned my addr 172.21.15.32:0/1833603742 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:22.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7a7fff640 1 -- 172.21.15.32:0/1833603742 >> v2:172.21.15.39:3300/0 conn(0x7fe7a80cf3e0 msgr2=0x7fe7a80cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:22.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7a7fff640 1 --2- 172.21.15.32:0/1833603742 >> v2:172.21.15.39:3300/0 conn(0x7fe7a80cf3e0 0x7fe7a80cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7a67fc640 1 -- 172.21.15.32:0/1833603742 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 25025976 0 0) 0x7fe7a80cfd80 con 0x7fe7a815ccf0 2023-12-20T04:40:22.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7a7fff640 1 -- 172.21.15.32:0/1833603742 >> v1:172.21.15.39:6789/0 conn(0x7fe7a815ccf0 legacy=0x7fe7a80cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7a7fff640 1 -- 172.21.15.32:0/1833603742 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a80d10f0 con 0x7fe7a80d1580 2023-12-20T04:40:22.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.898+0000 7fe7a6ffd640 1 --2- 172.21.15.32:0/1833603742 >> v2:172.21.15.39:3300/0 conn(0x7fe7a80cf3e0 0x7fe7a80cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:22.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7a7fff640 1 --2- 172.21.15.32:0/1833603742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80d1580 0x7fe7a80d1a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009e50 tx=0x7fe7a001c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=302cb7b3ceeeeabf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7a67fc640 1 -- 172.21.15.32:0/1833603742 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00a5810 con 0x7fe7a80d1580 2023-12-20T04:40:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7a67fc640 1 -- 172.21.15.32:0/1833603742 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a00a5970 con 0x7fe7a80d1580 2023-12-20T04:40:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7a67fc640 1 -- 172.21.15.32:0/1833603742 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00a5d00 con 0x7fe7a80d1580 2023-12-20T04:40:22.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1833603742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80d1580 msgr2=0x7fe7a80d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7ace2a640 1 --2- 172.21.15.32:0/1833603742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80d1580 0x7fe7a80d1a10 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fe7a0009e50 tx=0x7fe7a001c950 comp rx=0 tx=0).stop 2023-12-20T04:40:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1833603742 shutdown_connections 2023-12-20T04:40:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7ace2a640 1 --2- 172.21.15.32:0/1833603742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80d1580 0x7fe7a80d1a10 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7ace2a640 1 --2- 172.21.15.32:0/1833603742 >> v2:172.21.15.39:3300/0 conn(0x7fe7a80cf3e0 0x7fe7a80cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1833603742 >> 172.21.15.32:0/1833603742 conn(0x7fe7a80658e0 msgr2=0x7fe7a8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1833603742 shutdown_connections 2023-12-20T04:40:22.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.902+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1833603742 wait complete. 2023-12-20T04:40:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7ace2a640 1 Processor -- start 2023-12-20T04:40:22.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7ace2a640 1 -- start start 2023-12-20T04:40:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7ace2a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80cf3e0 0x7fe7a814b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7ace2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7a80d1580 0x7fe7a81489b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7ace2a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7a815ccf0 0x7fe7a80d29f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7ace2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a8150960 con 0x7fe7a80d1580 2023-12-20T04:40:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7ace2a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80d3520 con 0x7fe7a815ccf0 2023-12-20T04:40:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7ace2a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7a80d3240 con 0x7fe7a80cf3e0 2023-12-20T04:40:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7a6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7a80d1580 0x7fe7a81489b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7a6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7a80d1580 0x7fe7a81489b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44184/0 (socket says 172.21.15.32:44184) 2023-12-20T04:40:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7a6ffd640 1 -- 172.21.15.32:0/1150308575 learned_addr learned my addr 172.21.15.32:0/1150308575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7a77fe640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80cf3e0 0x7fe7a814b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7a7fff640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7a815ccf0 0x7fe7a80d29f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7a6ffd640 1 -- 172.21.15.32:0/1150308575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80cf3e0 msgr2=0x7fe7a814b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7a6ffd640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80cf3e0 0x7fe7a814b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7a6ffd640 1 -- 172.21.15.32:0/1150308575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7a815ccf0 msgr2=0x7fe7a80d29f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:22.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7a6ffd640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7a815ccf0 0x7fe7a80d29f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7a6ffd640 1 -- 172.21.15.32:0/1150308575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe79c009420 con 0x7fe7a80d1580 2023-12-20T04:40:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7a6ffd640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7a80d1580 0x7fe7a81489b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00e980 tx=0x7fe79c00ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe78ffff640 1 -- 172.21.15.32:0/1150308575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c00ca00 con 0x7fe7a80d1580 2023-12-20T04:40:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1150308575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7a0009b30 con 0x7fe7a80d1580 2023-12-20T04:40:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe78ffff640 1 -- 172.21.15.32:0/1150308575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe79c00cb60 con 0x7fe7a80d1580 2023-12-20T04:40:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe78ffff640 1 -- 172.21.15.32:0/1150308575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe79c013470 con 0x7fe7a80d1580 2023-12-20T04:40:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1150308575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7a81505f0 con 0x7fe7a80d1580 2023-12-20T04:40:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.906+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1150308575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7a8150b10 con 0x7fe7a80d1580 2023-12-20T04:40:22.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.910+0000 7fe78ffff640 1 -- 172.21.15.32:0/1150308575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fe79c0135d0 con 0x7fe7a80d1580 2023-12-20T04:40:22.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.910+0000 7fe78ffff640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe794065c60 0x7fe794068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.910+0000 7fe78ffff640 1 -- 172.21.15.32:0/1150308575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe79c0837f0 con 0x7fe7a80d1580 2023-12-20T04:40:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.910+0000 7fe7a77fe640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe794065c60 0x7fe794068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:22.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.910+0000 7fe7a77fe640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe794065c60 0x7fe794068120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe798000f70 tx=0x7fe798009040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:22.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:22.914+0000 7fe78ffff640 1 -- 172.21.15.32:0/1150308575 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe7a8150b10 con 0x7fe7a80d1580 2023-12-20T04:40:23.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.090+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1150308575 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe7a80b5c40 con 0x7fe794065c60 2023-12-20T04:40:23.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe78ffff640 1 -- 172.21.15.32:0/1150308575 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe798001560 con 0x7fe794065c60 2023-12-20T04:40:23.157 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path /snap_test_dir1/dir8 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1150308575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe794065c60 msgr2=0x7fe794068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe794065c60 0x7fe794068120 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fe798000f70 tx=0x7fe798009040 comp rx=0 tx=0).stop 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1150308575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7a80d1580 msgr2=0x7fe7a81489b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7a80d1580 0x7fe7a81489b0 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe79c00e980 tx=0x7fe79c00ee40 comp rx=0 tx=0).stop 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1150308575 shutdown_connections 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe794065c60 0x7fe794068120 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7a815ccf0 0x7fe7a80d29f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7a80d1580 0x7fe7a81489b0 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 --2- 172.21.15.32:0/1150308575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7a80cf3e0 0x7fe7a814b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1150308575 >> 172.21.15.32:0/1150308575 conn(0x7fe7a80658e0 msgr2=0x7fe7a815d410 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1150308575 shutdown_connections 2023-12-20T04:40:23.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.158+0000 7fe7ace2a640 1 -- 172.21.15.32:0/1150308575 wait complete. 2023-12-20T04:40:23.178 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir8 2023-12-20T04:40:23.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/dir9 1M 2023-12-20T04:40:23.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2eeefc640 1 Processor -- start 2023-12-20T04:40:23.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2eeefc640 1 -- start start 2023-12-20T04:40:23.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2eeefc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe2e8157480 0x7fe2e80c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:23.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2eeefc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e80cde20 con 0x7fe2e80cd4c0 2023-12-20T04:40:23.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2eeefc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2e80c8e30 con 0x7fe2e80b0760 2023-12-20T04:40:23.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2eeefc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2e815e480 con 0x7fe2e8157480 2023-12-20T04:40:23.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2ed499640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe2e80cd4c0 0x7fe2e815dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58986/0 (socket says 172.21.15.32:58986) 2023-12-20T04:40:23.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2ed499640 1 -- 172.21.15.32:0/789767417 learned_addr learned my addr 172.21.15.32:0/789767417 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:23.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2ecc98640 1 --2- 172.21.15.32:0/789767417 >> v2:172.21.15.39:3300/0 conn(0x7fe2e8157480 0x7fe2e80c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:23.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2e77fe640 1 -- 172.21.15.32:0/789767417 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2234040528 0 0) 0x7fe2e80cde20 con 0x7fe2e80cd4c0 2023-12-20T04:40:23.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2e77fe640 1 -- 172.21.15.32:0/789767417 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2d8003530 con 0x7fe2e80cd4c0 2023-12-20T04:40:23.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2e77fe640 1 -- 172.21.15.32:0/789767417 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2744849940 0 0) 0x7fe2e80c8e30 con 0x7fe2e80b0760 2023-12-20T04:40:23.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2e77fe640 1 -- 172.21.15.32:0/789767417 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe2e80cde20 con 0x7fe2e80b0760 2023-12-20T04:40:23.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2e77fe640 1 -- 172.21.15.32:0/789767417 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2390384683 0 0) 0x7fe2d8003530 con 0x7fe2e80cd4c0 2023-12-20T04:40:23.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2ecc98640 1 -- 172.21.15.32:0/789767417 >> v1:172.21.15.39:6789/0 conn(0x7fe2e80b0760 legacy=0x7fe2e815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2ecc98640 1 -- 172.21.15.32:0/789767417 >> v1:172.21.15.32:6789/0 conn(0x7fe2e80cd4c0 legacy=0x7fe2e815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.326+0000 7fe2ecc98640 1 -- 172.21.15.32:0/789767417 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2e80ce070 con 0x7fe2e8157480 2023-12-20T04:40:23.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.330+0000 7fe2ecc98640 1 --2- 172.21.15.32:0/789767417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e8157480 0x7fe2e80c9840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0097c0 tx=0x7fe2dc01c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3f79dd59b5b9c30 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:23.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.330+0000 7fe2e77fe640 1 -- 172.21.15.32:0/789767417 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0001a00 con 0x7fe2e8157480 2023-12-20T04:40:23.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.330+0000 7fe2e77fe640 1 -- 172.21.15.32:0/789767417 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2dc0a7050 con 0x7fe2e8157480 2023-12-20T04:40:23.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.330+0000 7fe2e77fe640 1 -- 172.21.15.32:0/789767417 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2dc0ab620 con 0x7fe2e8157480 2023-12-20T04:40:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.330+0000 7fe2eeefc640 1 -- 172.21.15.32:0/789767417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e8157480 msgr2=0x7fe2e80c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.330+0000 7fe2eeefc640 1 --2- 172.21.15.32:0/789767417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e8157480 0x7fe2e80c9840 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0097c0 tx=0x7fe2dc01c2e0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.330+0000 7fe2eeefc640 1 -- 172.21.15.32:0/789767417 shutdown_connections 2023-12-20T04:40:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.330+0000 7fe2eeefc640 1 --2- 172.21.15.32:0/789767417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e8157480 0x7fe2e80c9840 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.330+0000 7fe2eeefc640 1 -- 172.21.15.32:0/789767417 >> 172.21.15.32:0/789767417 conn(0x7fe2e806a510 msgr2=0x7fe2e80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.330+0000 7fe2eeefc640 1 -- 172.21.15.32:0/789767417 shutdown_connections 2023-12-20T04:40:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.330+0000 7fe2eeefc640 1 -- 172.21.15.32:0/789767417 wait complete. 2023-12-20T04:40:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2eeefc640 1 Processor -- start 2023-12-20T04:40:23.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2eeefc640 1 -- start start 2023-12-20T04:40:23.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2eeefc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e80b0760 0x7fe2e81562f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:23.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2eeefc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2e80cd4c0 0x7fe2e8162b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:23.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2eeefc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8157480 0x7fe2e80b19c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:23.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2eeefc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e8164090 con 0x7fe2e8157480 2023-12-20T04:40:23.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2eeefc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e8163480 con 0x7fe2e80b0760 2023-12-20T04:40:23.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2eeefc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2e8163760 con 0x7fe2e80cd4c0 2023-12-20T04:40:23.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2ed499640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8157480 0x7fe2e80b19c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:23.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2ed499640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8157480 0x7fe2e80b19c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44192/0 (socket says 172.21.15.32:44192) 2023-12-20T04:40:23.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2ed499640 1 -- 172.21.15.32:0/3776468051 learned_addr learned my addr 172.21.15.32:0/3776468051 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:23.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2ecc98640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e80b0760 0x7fe2e81562f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:23.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2e7fff640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2e80cd4c0 0x7fe2e8162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2ed499640 1 -- 172.21.15.32:0/3776468051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2e80cd4c0 msgr2=0x7fe2e8162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2ed499640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2e80cd4c0 0x7fe2e8162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2ed499640 1 -- 172.21.15.32:0/3776468051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e80b0760 msgr2=0x7fe2e81562f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2ed499640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e80b0760 0x7fe2e81562f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2ed499640 1 -- 172.21.15.32:0/3776468051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2dc009420 con 0x7fe2e8157480 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2ed499640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8157480 0x7fe2e80b19c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0001f70 tx=0x7fe2e000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2e57fa640 1 -- 172.21.15.32:0/3776468051 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0012400 con 0x7fe2e8157480 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2e57fa640 1 -- 172.21.15.32:0/3776468051 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2e0012560 con 0x7fe2e8157480 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2eeefc640 1 -- 172.21.15.32:0/3776468051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2e8163a40 con 0x7fe2e8157480 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2e57fa640 1 -- 172.21.15.32:0/3776468051 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2e0012910 con 0x7fe2e8157480 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.334+0000 7fe2eeefc640 1 -- 172.21.15.32:0/3776468051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2e8156830 con 0x7fe2e8157480 2023-12-20T04:40:23.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.338+0000 7fe2eeefc640 1 -- 172.21.15.32:0/3776468051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2e8164240 con 0x7fe2e8157480 2023-12-20T04:40:23.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.338+0000 7fe2e57fa640 1 -- 172.21.15.32:0/3776468051 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fe2e0012a70 con 0x7fe2e8157480 2023-12-20T04:40:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.338+0000 7fe2e57fa640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2cc065c90 0x7fe2cc068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.338+0000 7fe2e57fa640 1 -- 172.21.15.32:0/3776468051 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fe2e0083da0 con 0x7fe2e8157480 2023-12-20T04:40:23.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.338+0000 7fe2ecc98640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2cc065c90 0x7fe2cc068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:23.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.338+0000 7fe2ecc98640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2cc065c90 0x7fe2cc068150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0ad000 tx=0x7fe2dc0a6000 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:23.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.342+0000 7fe2e57fa640 1 -- 172.21.15.32:0/3776468051 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe2e8164240 con 0x7fe2e8157480 2023-12-20T04:40:23.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.522+0000 7fe2eeefc640 1 -- 172.21.15.32:0/3776468051 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe2e80c9360 con 0x7fe2cc065c90 2023-12-20T04:40:23.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.582+0000 7fe2e57fa640 1 -- 172.21.15.32:0/3776468051 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7fe2dc0ab580 con 0x7fe2cc065c90 2023-12-20T04:40:23.582 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path /snap_test_dir1/dir9 2023-12-20T04:40:23.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 -- 172.21.15.32:0/3776468051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2cc065c90 msgr2=0x7fe2cc068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2cc065c90 0x7fe2cc068150 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe2dc0ad000 tx=0x7fe2dc0a6000 comp rx=0 tx=0).stop 2023-12-20T04:40:23.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 -- 172.21.15.32:0/3776468051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8157480 msgr2=0x7fe2e80b19c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8157480 0x7fe2e80b19c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fe2e0001f70 tx=0x7fe2e000a430 comp rx=0 tx=0).stop 2023-12-20T04:40:23.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 -- 172.21.15.32:0/3776468051 shutdown_connections 2023-12-20T04:40:23.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe2cc065c90 0x7fe2cc068150 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2e8157480 0x7fe2e80b19c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2e80cd4c0 0x7fe2e8162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 --2- 172.21.15.32:0/3776468051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2e80b0760 0x7fe2e81562f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 -- 172.21.15.32:0/3776468051 >> 172.21.15.32:0/3776468051 conn(0x7fe2e806a510 msgr2=0x7fe2e80c4b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:23.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 -- 172.21.15.32:0/3776468051 shutdown_connections 2023-12-20T04:40:23.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.586+0000 7fe2eeefc640 1 -- 172.21.15.32:0/3776468051 wait complete. 2023-12-20T04:40:23.602 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/dir9 2023-12-20T04:40:23.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:23.603 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-12-20T04:40:23.616 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:23.616 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:23.616 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:23.616 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:23.616 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:23.617 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:23.617 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:23.617 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:23.617 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:23.617 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:23.617 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:23.618 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-04_30_00_UTC', 'scheduled-2023-12-20-04_31_00_UTC', 'scheduled-2023-12-20-04_32_00_UTC', 'scheduled-2023-12-20-04_33_00_UTC', 'scheduled-2023-12-20-04_34_00_UTC', 'scheduled-2023-12-20-04_35_00_UTC', 'scheduled-2023-12-20-04_36_00_UTC', 'scheduled-2023-12-20-04_37_00_UTC', 'scheduled-2023-12-20-04_38_00_UTC', 'scheduled-2023-12-20-04_39_00_UTC', 'scheduled-2023-12-20-04_40_00_UTC'] 2023-12-20T04:40:23.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir0 2023-12-20T04:40:23.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.802+0000 7fbd68811640 1 Processor -- start 2023-12-20T04:40:23.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd68811640 1 -- start start 2023-12-20T04:40:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd68811640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd6415ce60 0x7fbd640d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd68811640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd640d3130 0x7fbd640d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd68811640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd640d2650 con 0x7fbd640d1300 2023-12-20T04:40:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd68811640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd640d35a0 con 0x7fbd6415ce60 2023-12-20T04:40:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd68811640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640d1a40 con 0x7fbd640d3130 2023-12-20T04:40:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd6259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd6415ce60 0x7fbd640d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd6259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd6415ce60 0x7fbd640d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44202/0 (socket says 172.21.15.32:44202) 2023-12-20T04:40:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd6259c640 1 -- 172.21.15.32:0/2796470777 learned_addr learned my addr 172.21.15.32:0/2796470777 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd5bfff640 1 --2- 172.21.15.32:0/2796470777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd640d3130 0x7fbd640d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:23.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd6259c640 1 -- 172.21.15.32:0/2796470777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd640d3130 msgr2=0x7fbd640d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd6259c640 1 --2- 172.21.15.32:0/2796470777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd640d3130 0x7fbd640d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd6259c640 1 -- 172.21.15.32:0/2796470777 >> v1:172.21.15.39:6789/0 conn(0x7fbd640d1300 legacy=0x7fbd640b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd6259c640 1 -- 172.21.15.32:0/2796470777 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd640d1d20 con 0x7fbd6415ce60 2023-12-20T04:40:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.806+0000 7fbd5bfff640 1 --2- 172.21.15.32:0/2796470777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd640d3130 0x7fbd640d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd6259c640 1 --2- 172.21.15.32:0/2796470777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6415ce60 0x7fbd640d2bf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009870 tx=0x7fbd5401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4fafc295fc13a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd61d9b640 1 -- 172.21.15.32:0/2796470777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd540a89b0 con 0x7fbd6415ce60 2023-12-20T04:40:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd61d9b640 1 -- 172.21.15.32:0/2796470777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd540a8b10 con 0x7fbd6415ce60 2023-12-20T04:40:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd61d9b640 1 -- 172.21.15.32:0/2796470777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd540a8ea0 con 0x7fbd6415ce60 2023-12-20T04:40:23.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd68811640 1 -- 172.21.15.32:0/2796470777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6415ce60 msgr2=0x7fbd640d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd68811640 1 --2- 172.21.15.32:0/2796470777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6415ce60 0x7fbd640d2bf0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7fbd54009870 tx=0x7fbd5401ecf0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd68811640 1 -- 172.21.15.32:0/2796470777 shutdown_connections 2023-12-20T04:40:23.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd68811640 1 --2- 172.21.15.32:0/2796470777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd640d3130 0x7fbd640d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd68811640 1 --2- 172.21.15.32:0/2796470777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd6415ce60 0x7fbd640d2bf0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd68811640 1 -- 172.21.15.32:0/2796470777 >> 172.21.15.32:0/2796470777 conn(0x7fbd6406b5e0 msgr2=0x7fbd6406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:23.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd68811640 1 -- 172.21.15.32:0/2796470777 shutdown_connections 2023-12-20T04:40:23.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd68811640 1 -- 172.21.15.32:0/2796470777 wait complete. 2023-12-20T04:40:23.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.810+0000 7fbd68811640 1 Processor -- start 2023-12-20T04:40:23.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd68811640 1 -- start start 2023-12-20T04:40:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd68811640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd640d1300 0x7fbd6414cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd68811640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd640d3130 0x7fbd640cb600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd68811640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd6415ce60 0x7fbd640a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd68811640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd6414d2f0 con 0x7fbd640d3130 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd68811640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd64147780 con 0x7fbd6415ce60 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd68811640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd640cbfc0 con 0x7fbd640d1300 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd640d3130 0x7fbd640cb600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd640d3130 0x7fbd640cb600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44216/0 (socket says 172.21.15.32:44216) 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5bfff640 1 -- 172.21.15.32:0/852155980 learned_addr learned my addr 172.21.15.32:0/852155980 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd6259c640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd640d1300 0x7fbd6414cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd62d9d640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd6415ce60 0x7fbd640a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5bfff640 1 -- 172.21.15.32:0/852155980 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd640d1300 msgr2=0x7fbd6414cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5bfff640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd640d1300 0x7fbd6414cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5bfff640 1 -- 172.21.15.32:0/852155980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd6415ce60 msgr2=0x7fbd640a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5bfff640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd6415ce60 0x7fbd640a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:23.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5bfff640 1 -- 172.21.15.32:0/852155980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd540094d0 con 0x7fbd640d3130 2023-12-20T04:40:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5bfff640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd640d3130 0x7fbd640cb600 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c7c0 tx=0x7fbd4c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5affd640 1 -- 172.21.15.32:0/852155980 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c007a30 con 0x7fbd640d3130 2023-12-20T04:40:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5affd640 1 -- 172.21.15.32:0/852155980 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd4c007b90 con 0x7fbd640d3130 2023-12-20T04:40:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd68811640 1 -- 172.21.15.32:0/852155980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd640cc2a0 con 0x7fbd640d3130 2023-12-20T04:40:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd5affd640 1 -- 172.21.15.32:0/852155980 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd4c00ce70 con 0x7fbd640d3130 2023-12-20T04:40:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd68811640 1 -- 172.21.15.32:0/852155980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd64147410 con 0x7fbd640d3130 2023-12-20T04:40:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.814+0000 7fbd68811640 1 -- 172.21.15.32:0/852155980 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd6404e000 con 0x7fbd640d3130 2023-12-20T04:40:23.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.818+0000 7fbd5affd640 1 -- 172.21.15.32:0/852155980 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fbd4c0113f0 con 0x7fbd640d3130 2023-12-20T04:40:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.818+0000 7fbd5affd640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd44065c90 0x7fbd44068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.818+0000 7fbd5affd640 1 -- 172.21.15.32:0/852155980 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fbd4c083ad0 con 0x7fbd640d3130 2023-12-20T04:40:23.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.818+0000 7fbd6259c640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd44065c90 0x7fbd44068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:23.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.818+0000 7fbd6259c640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd44065c90 0x7fbd44068150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd540b2000 tx=0x7fbd54009040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:23.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:23.822+0000 7fbd5affd640 1 -- 172.21.15.32:0/852155980 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbd4c051150 con 0x7fbd640d3130 2023-12-20T04:40:23.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.002+0000 7fbd68811640 1 -- 172.21.15.32:0/852155980 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir0", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd640d22e0 con 0x7fbd44065c90 2023-12-20T04:40:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd5affd640 1 -- 172.21.15.32:0/852155980 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7fbd540b05c0 con 0x7fbd44065c90 2023-12-20T04:40:24.008 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:24.008 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:48", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 -- 172.21.15.32:0/852155980 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd44065c90 msgr2=0x7fbd44068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd44065c90 0x7fbd44068150 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fbd540b2000 tx=0x7fbd54009040 comp rx=0 tx=0).stop 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 -- 172.21.15.32:0/852155980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd640d3130 msgr2=0x7fbd640cb600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd640d3130 0x7fbd640cb600 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7fbd4c00c7c0 tx=0x7fbd4c00cc80 comp rx=0 tx=0).stop 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 -- 172.21.15.32:0/852155980 shutdown_connections 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd44065c90 0x7fbd44068150 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd6415ce60 0x7fbd640a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd640d3130 0x7fbd640cb600 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 --2- 172.21.15.32:0/852155980 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd640d1300 0x7fbd6414cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 -- 172.21.15.32:0/852155980 >> 172.21.15.32:0/852155980 conn(0x7fbd6406b5e0 msgr2=0x7fbd6415afe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 -- 172.21.15.32:0/852155980 shutdown_connections 2023-12-20T04:40:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.010+0000 7fbd68811640 1 -- 172.21.15.32:0/852155980 wait complete. 2023-12-20T04:40:24.029 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir0", "rel_path": "/snap_test_dir1/dir0", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:48", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:24.029 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir0', 'rel_path': '/snap_test_dir1/dir0', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T04:29:48', 'first': '2023-12-20T04:30:00', 'last': '2023-12-20T04:40:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-20T04:40:24.030 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-20T04:40:24.030 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-20T04:40:24.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:24.030 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-12-20T04:40:24.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:24.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:24.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:24.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:24.041 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:24.041 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:24.041 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:24.041 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:24.041 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:24.041 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:24.041 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:24.042 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-04_30_00_UTC', 'scheduled-2023-12-20-04_31_00_UTC', 'scheduled-2023-12-20-04_32_00_UTC', 'scheduled-2023-12-20-04_33_00_UTC', 'scheduled-2023-12-20-04_34_00_UTC', 'scheduled-2023-12-20-04_35_00_UTC', 'scheduled-2023-12-20-04_36_00_UTC', 'scheduled-2023-12-20-04_37_00_UTC', 'scheduled-2023-12-20-04_38_00_UTC', 'scheduled-2023-12-20-04_39_00_UTC', 'scheduled-2023-12-20-04_40_00_UTC'] 2023-12-20T04:40:24.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir1 2023-12-20T04:40:24.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.226+0000 7f4163bb3640 1 Processor -- start 2023-12-20T04:40:24.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f4163bb3640 1 -- start start 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f4163bb3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f415c15abb0 0x7f415c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f4163bb3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f415c070230 0x7f415c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f4163bb3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f415c15b6a0 con 0x7f415c0b0cb0 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f4163bb3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f415c06e160 con 0x7f415c15abb0 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f4163bb3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c070c00 con 0x7f415c070230 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f416194f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f415c15abb0 0x7f415c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f416194f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f415c15abb0 0x7f415c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44224/0 (socket says 172.21.15.32:44224) 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f416194f640 1 -- 172.21.15.32:0/131964643 learned_addr learned my addr 172.21.15.32:0/131964643 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f416194f640 1 -- 172.21.15.32:0/131964643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f415c070230 msgr2=0x7f415c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f416194f640 1 --2- 172.21.15.32:0/131964643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f415c070230 0x7f415c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f416194f640 1 -- 172.21.15.32:0/131964643 >> v1:172.21.15.39:6789/0 conn(0x7f415c0b0cb0 legacy=0x7f415c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:24.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.230+0000 7f416194f640 1 -- 172.21.15.32:0/131964643 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f415c06d8c0 con 0x7f415c15abb0 2023-12-20T04:40:24.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f416194f640 1 --2- 172.21.15.32:0/131964643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c15abb0 0x7f415c0caaf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4158009870 tx=0x7f415801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f98cc4382b1d73f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:24.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f416094d640 1 -- 172.21.15.32:0/131964643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41580aa620 con 0x7f415c15abb0 2023-12-20T04:40:24.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f416094d640 1 -- 172.21.15.32:0/131964643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41580aa780 con 0x7f415c15abb0 2023-12-20T04:40:24.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f416094d640 1 -- 172.21.15.32:0/131964643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41580aab10 con 0x7f415c15abb0 2023-12-20T04:40:24.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f4163bb3640 1 -- 172.21.15.32:0/131964643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c15abb0 msgr2=0x7f415c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f4163bb3640 1 --2- 172.21.15.32:0/131964643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c15abb0 0x7f415c0caaf0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f4158009870 tx=0x7f415801c500 comp rx=0 tx=0).stop 2023-12-20T04:40:24.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f4163bb3640 1 -- 172.21.15.32:0/131964643 shutdown_connections 2023-12-20T04:40:24.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f4163bb3640 1 --2- 172.21.15.32:0/131964643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f415c070230 0x7f415c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f4163bb3640 1 --2- 172.21.15.32:0/131964643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c15abb0 0x7f415c0caaf0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f4163bb3640 1 -- 172.21.15.32:0/131964643 >> 172.21.15.32:0/131964643 conn(0x7f415c06a510 msgr2=0x7f415c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:24.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f4163bb3640 1 -- 172.21.15.32:0/131964643 shutdown_connections 2023-12-20T04:40:24.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f4163bb3640 1 -- 172.21.15.32:0/131964643 wait complete. 2023-12-20T04:40:24.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f4163bb3640 1 Processor -- start 2023-12-20T04:40:24.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.234+0000 7f4163bb3640 1 -- start start 2023-12-20T04:40:24.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f4163bb3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415c070230 0x7f415c0c68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f4163bb3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0b0cb0 0x7f415c158e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f4163bb3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f415c15abb0 0x7f415c0c9330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f4163bb3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c0c6df0 con 0x7f415c0b0cb0 2023-12-20T04:40:24.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f4163bb3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f415c14cbd0 con 0x7f415c070230 2023-12-20T04:40:24.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f4163bb3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f415c14ceb0 con 0x7f415c15abb0 2023-12-20T04:40:24.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f416114e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0b0cb0 0x7f415c158e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:24.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f416114e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0b0cb0 0x7f415c158e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44226/0 (socket says 172.21.15.32:44226) 2023-12-20T04:40:24.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f416114e640 1 -- 172.21.15.32:0/3944416821 learned_addr learned my addr 172.21.15.32:0/3944416821 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f4162150640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f415c15abb0 0x7f415c0c9330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f416194f640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415c070230 0x7f415c0c68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f416114e640 1 -- 172.21.15.32:0/3944416821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f415c15abb0 msgr2=0x7f415c0c9330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f416114e640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f415c15abb0 0x7f415c0c9330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f416114e640 1 -- 172.21.15.32:0/3944416821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415c070230 msgr2=0x7f415c0c68b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f416114e640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415c070230 0x7f415c0c68b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f416114e640 1 -- 172.21.15.32:0/3944416821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41580094d0 con 0x7f415c0b0cb0 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f416114e640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0b0cb0 0x7f415c158e60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f414c00c410 tx=0x7f414c00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f41527fc640 1 -- 172.21.15.32:0/3944416821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c003600 con 0x7f415c0b0cb0 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f41527fc640 1 -- 172.21.15.32:0/3944416821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f414c003780 con 0x7f415c0b0cb0 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f4163bb3640 1 -- 172.21.15.32:0/3944416821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f415c14d190 con 0x7f415c0b0cb0 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f41527fc640 1 -- 172.21.15.32:0/3944416821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f414c003b30 con 0x7f415c0b0cb0 2023-12-20T04:40:24.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f4163bb3640 1 -- 172.21.15.32:0/3944416821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f415c14d750 con 0x7f415c0b0cb0 2023-12-20T04:40:24.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f4163bb3640 1 -- 172.21.15.32:0/3944416821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f415c14d970 con 0x7f415c0b0cb0 2023-12-20T04:40:24.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.238+0000 7f41527fc640 1 -- 172.21.15.32:0/3944416821 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f414c003c90 con 0x7f415c0b0cb0 2023-12-20T04:40:24.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.242+0000 7f41527fc640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4140065c60 0x7f4140068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.242+0000 7f41527fc640 1 -- 172.21.15.32:0/3944416821 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f414c082970 con 0x7f415c0b0cb0 2023-12-20T04:40:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.242+0000 7f416194f640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4140065c60 0x7f4140068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:24.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.242+0000 7f416194f640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4140065c60 0x7f4140068120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41580ae000 tx=0x7f41580a7000 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:24.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.246+0000 7f41527fc640 1 -- 172.21.15.32:0/3944416821 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f414c04ffa0 con 0x7f415c0b0cb0 2023-12-20T04:40:24.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.418+0000 7f4163bb3640 1 -- 172.21.15.32:0/3944416821 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f415c15b330 con 0x7f4140065c60 2023-12-20T04:40:24.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.426+0000 7f41527fc640 1 -- 172.21.15.32:0/3944416821 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f41580aa8f0 con 0x7f4140065c60 2023-12-20T04:40:24.427 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:24.427 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:49", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 -- 172.21.15.32:0/3944416821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4140065c60 msgr2=0x7f4140068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4140065c60 0x7f4140068120 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f41580ae000 tx=0x7f41580a7000 comp rx=0 tx=0).stop 2023-12-20T04:40:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 -- 172.21.15.32:0/3944416821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0b0cb0 msgr2=0x7f415c158e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0b0cb0 0x7f415c158e60 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f414c00c410 tx=0x7f414c00c8d0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 -- 172.21.15.32:0/3944416821 shutdown_connections 2023-12-20T04:40:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4140065c60 0x7f4140068120 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f415c15abb0 0x7f415c0c9330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f415c0b0cb0 0x7f415c158e60 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 --2- 172.21.15.32:0/3944416821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f415c070230 0x7f415c0c68b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 -- 172.21.15.32:0/3944416821 >> 172.21.15.32:0/3944416821 conn(0x7f415c06a510 msgr2=0x7f415c06d1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 -- 172.21.15.32:0/3944416821 shutdown_connections 2023-12-20T04:40:24.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.430+0000 7f4163bb3640 1 -- 172.21.15.32:0/3944416821 wait complete. 2023-12-20T04:40:24.448 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir1", "rel_path": "/snap_test_dir1/dir1", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:49", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:24.448 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir1', 'rel_path': '/snap_test_dir1/dir1', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T04:29:49', 'first': '2023-12-20T04:30:00', 'last': '2023-12-20T04:40:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-20T04:40:24.448 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-20T04:40:24.448 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-20T04:40:24.448 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:24.449 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-12-20T04:40:24.460 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:24.460 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:24.460 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:24.460 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:24.460 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:24.460 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:24.460 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:24.460 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:24.461 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:24.461 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:24.461 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:24.462 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-04_30_00_UTC', 'scheduled-2023-12-20-04_31_00_UTC', 'scheduled-2023-12-20-04_32_00_UTC', 'scheduled-2023-12-20-04_33_00_UTC', 'scheduled-2023-12-20-04_34_00_UTC', 'scheduled-2023-12-20-04_35_00_UTC', 'scheduled-2023-12-20-04_36_00_UTC', 'scheduled-2023-12-20-04_37_00_UTC', 'scheduled-2023-12-20-04_38_00_UTC', 'scheduled-2023-12-20-04_39_00_UTC', 'scheduled-2023-12-20-04_40_00_UTC'] 2023-12-20T04:40:24.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir2 2023-12-20T04:40:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.646+0000 7f7c7c695640 1 Processor -- start 2023-12-20T04:40:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.646+0000 7f7c7c695640 1 -- start start 2023-12-20T04:40:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7c695640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7c7415abb0 0x7f7c740caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7c695640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7c74070230 0x7f7c740706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7c695640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7c7415b6a0 con 0x7f7c740b0cb0 2023-12-20T04:40:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7c695640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7c7406e160 con 0x7f7c7415abb0 2023-12-20T04:40:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7c695640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74070c00 con 0x7f7c74070230 2023-12-20T04:40:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7a431640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7c7415abb0 0x7f7c740caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7ac32640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7c74070230 0x7f7c740706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7a431640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7c7415abb0 0x7f7c740caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:34214/0 (socket says 172.21.15.32:34214) 2023-12-20T04:40:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7a431640 1 -- 172.21.15.32:0/3920814153 learned_addr learned my addr 172.21.15.32:0/3920814153 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:24.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c79c30640 1 --1- 172.21.15.32:0/3920814153 >> v1:172.21.15.39:6789/0 conn(0x7f7c740b0cb0 0x7f7c7415bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:60786/0 (socket says 172.21.15.32:60786) 2023-12-20T04:40:24.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7a431640 1 -- 172.21.15.32:0/3920814153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7c74070230 msgr2=0x7f7c740706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:24.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7a431640 1 --2- 172.21.15.32:0/3920814153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7c74070230 0x7f7c740706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7a431640 1 -- 172.21.15.32:0/3920814153 >> v1:172.21.15.39:6789/0 conn(0x7f7c740b0cb0 legacy=0x7f7c7415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7a431640 1 -- 172.21.15.32:0/3920814153 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c7406d8c0 con 0x7f7c7415abb0 2023-12-20T04:40:24.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.650+0000 7f7c7ac32640 1 --2- 172.21.15.32:0/3920814153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7c74070230 0x7f7c740706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:24.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7a431640 1 --2- 172.21.15.32:0/3920814153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c7415abb0 0x7f7c740caaf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009870 tx=0x7f7c7001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1e4103380370f6df server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7942f640 1 -- 172.21.15.32:0/3920814153 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700a9400 con 0x7f7c7415abb0 2023-12-20T04:40:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7942f640 1 -- 172.21.15.32:0/3920814153 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c700a9560 con 0x7f7c7415abb0 2023-12-20T04:40:24.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7942f640 1 -- 172.21.15.32:0/3920814153 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700a98f0 con 0x7f7c7415abb0 2023-12-20T04:40:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7c695640 1 -- 172.21.15.32:0/3920814153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c7415abb0 msgr2=0x7f7c740caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7c695640 1 --2- 172.21.15.32:0/3920814153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c7415abb0 0x7f7c740caaf0 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f7c70009870 tx=0x7f7c7001c540 comp rx=0 tx=0).stop 2023-12-20T04:40:24.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7c695640 1 -- 172.21.15.32:0/3920814153 shutdown_connections 2023-12-20T04:40:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7c695640 1 --2- 172.21.15.32:0/3920814153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7c74070230 0x7f7c740706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7c695640 1 --2- 172.21.15.32:0/3920814153 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c7415abb0 0x7f7c740caaf0 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7c695640 1 -- 172.21.15.32:0/3920814153 >> 172.21.15.32:0/3920814153 conn(0x7f7c7406a510 msgr2=0x7f7c740ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7c695640 1 -- 172.21.15.32:0/3920814153 shutdown_connections 2023-12-20T04:40:24.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7c695640 1 -- 172.21.15.32:0/3920814153 wait complete. 2023-12-20T04:40:24.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.654+0000 7f7c7c695640 1 Processor -- start 2023-12-20T04:40:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7c695640 1 -- start start 2023-12-20T04:40:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7c695640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c74070230 0x7f7c74152ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7c695640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7c740b0cb0 0x7f7c7414a1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7c695640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c7415abb0 0x7f7c740cd8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7c695640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c741493e0 con 0x7f7c74070230 2023-12-20T04:40:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7c695640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7c74151780 con 0x7f7c7415abb0 2023-12-20T04:40:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7c695640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7c7414a8b0 con 0x7f7c740b0cb0 2023-12-20T04:40:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7a431640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c74070230 0x7f7c74152ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:24.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7a431640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c74070230 0x7f7c74152ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44240/0 (socket says 172.21.15.32:44240) 2023-12-20T04:40:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7a431640 1 -- 172.21.15.32:0/2525587615 learned_addr learned my addr 172.21.15.32:0/2525587615 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c79c30640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7c740b0cb0 0x7f7c7414a1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7ac32640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c7415abb0 0x7f7c740cd8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7a431640 1 -- 172.21.15.32:0/2525587615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7c740b0cb0 msgr2=0x7f7c7414a1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7a431640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7c740b0cb0 0x7f7c7414a1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7a431640 1 -- 172.21.15.32:0/2525587615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c7415abb0 msgr2=0x7f7c740cd8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7a431640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c7415abb0 0x7f7c740cd8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7a431640 1 -- 172.21.15.32:0/2525587615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7c6c009a70 con 0x7f7c74070230 2023-12-20T04:40:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7a431640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c74070230 0x7f7c74152ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7c7001a170 tx=0x7f7c7001ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c6affd640 1 -- 172.21.15.32:0/2525587615 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700a9400 con 0x7f7c74070230 2023-12-20T04:40:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c6affd640 1 -- 172.21.15.32:0/2525587615 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7c700b1820 con 0x7f7c74070230 2023-12-20T04:40:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c6affd640 1 -- 172.21.15.32:0/2525587615 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7c700b1bd0 con 0x7f7c74070230 2023-12-20T04:40:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7c695640 1 -- 172.21.15.32:0/2525587615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7c700094d0 con 0x7f7c74070230 2023-12-20T04:40:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7c695640 1 -- 172.21.15.32:0/2525587615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7c74151130 con 0x7f7c74070230 2023-12-20T04:40:24.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.658+0000 7f7c7c695640 1 -- 172.21.15.32:0/2525587615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7c7404e000 con 0x7f7c74070230 2023-12-20T04:40:24.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.662+0000 7f7c6affd640 1 -- 172.21.15.32:0/2525587615 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f7c700b1d30 con 0x7f7c74070230 2023-12-20T04:40:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.662+0000 7f7c6affd640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7c58065c60 0x7f7c58068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.662+0000 7f7c6affd640 1 -- 172.21.15.32:0/2525587615 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7c7011da70 con 0x7f7c74070230 2023-12-20T04:40:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.662+0000 7f7c79c30640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7c58065c60 0x7f7c58068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:24.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.662+0000 7f7c79c30640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7c58065c60 0x7f7c58068120 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c74149aa0 tx=0x7f7c64007e50 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:24.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.666+0000 7f7c6affd640 1 -- 172.21.15.32:0/2525587615 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7c700eaf90 con 0x7f7c74070230 2023-12-20T04:40:24.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.842+0000 7f7c7c695640 1 -- 172.21.15.32:0/2525587615 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir2", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7c7415b330 con 0x7f7c58065c60 2023-12-20T04:40:24.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c6affd640 1 -- 172.21.15.32:0/2525587615 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f7c64004a70 con 0x7f7c58065c60 2023-12-20T04:40:24.851 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:24.852 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:49", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:24.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 -- 172.21.15.32:0/2525587615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7c58065c60 msgr2=0x7f7c58068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7c58065c60 0x7f7c58068120 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f7c74149aa0 tx=0x7f7c64007e50 comp rx=0 tx=0).stop 2023-12-20T04:40:24.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 -- 172.21.15.32:0/2525587615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c74070230 msgr2=0x7f7c74152ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:24.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c74070230 0x7f7c74152ab0 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f7c7001a170 tx=0x7f7c7001ad80 comp rx=0 tx=0).stop 2023-12-20T04:40:24.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 -- 172.21.15.32:0/2525587615 shutdown_connections 2023-12-20T04:40:24.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7c58065c60 0x7f7c58068120 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7c7415abb0 0x7f7c740cd8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7c740b0cb0 0x7f7c7414a1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 --2- 172.21.15.32:0/2525587615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7c74070230 0x7f7c74152ab0 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:24.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 -- 172.21.15.32:0/2525587615 >> 172.21.15.32:0/2525587615 conn(0x7f7c7406a510 msgr2=0x7f7c740cb880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:24.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 -- 172.21.15.32:0/2525587615 shutdown_connections 2023-12-20T04:40:24.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:24.854+0000 7f7c7c695640 1 -- 172.21.15.32:0/2525587615 wait complete. 2023-12-20T04:40:24.875 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir2", "rel_path": "/snap_test_dir1/dir2", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:49", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:24.875 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir2', 'rel_path': '/snap_test_dir1/dir2', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T04:29:49', 'first': '2023-12-20T04:30:00', 'last': '2023-12-20T04:40:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-20T04:40:24.875 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-20T04:40:24.876 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-20T04:40:24.876 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:24.876 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-12-20T04:40:24.888 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:24.888 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:24.888 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:24.889 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:24.889 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:24.889 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:24.889 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:24.889 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:24.889 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:24.889 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:24.889 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:24.890 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-04_30_00_UTC', 'scheduled-2023-12-20-04_31_00_UTC', 'scheduled-2023-12-20-04_32_00_UTC', 'scheduled-2023-12-20-04_33_00_UTC', 'scheduled-2023-12-20-04_34_00_UTC', 'scheduled-2023-12-20-04_35_00_UTC', 'scheduled-2023-12-20-04_36_00_UTC', 'scheduled-2023-12-20-04_37_00_UTC', 'scheduled-2023-12-20-04_38_00_UTC', 'scheduled-2023-12-20-04_39_00_UTC', 'scheduled-2023-12-20-04_40_00_UTC'] 2023-12-20T04:40:24.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir3 2023-12-20T04:40:25.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d25495640 1 Processor -- start 2023-12-20T04:40:25.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d25495640 1 -- start start 2023-12-20T04:40:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d25495640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6d20157480 0x7f6d200c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d25495640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d200b0760 0x7f6d200b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d25495640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d200b1100 con 0x7f6d2015d660 2023-12-20T04:40:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d25495640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d200cde20 con 0x7f6d20157480 2023-12-20T04:40:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d25495640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d200c8e30 con 0x7f6d200b0760 2023-12-20T04:40:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d17fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d200b0760 0x7f6d200b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d1effd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6d20157480 0x7f6d200c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d17fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d200b0760 0x7f6d200b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:41920/0 (socket says 172.21.15.32:41920) 2023-12-20T04:40:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d17fff640 1 -- 172.21.15.32:0/1875373515 learned_addr learned my addr 172.21.15.32:0/1875373515 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d17fff640 1 -- 172.21.15.32:0/1875373515 >> v2:172.21.15.39:3300/0 conn(0x7f6d20157480 msgr2=0x7f6d200c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d17fff640 1 --2- 172.21.15.32:0/1875373515 >> v2:172.21.15.39:3300/0 conn(0x7f6d20157480 0x7f6d200c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d17fff640 1 -- 172.21.15.32:0/1875373515 >> v1:172.21.15.39:6789/0 conn(0x7f6d2015d660 legacy=0x7f6d2015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d17fff640 1 -- 172.21.15.32:0/1875373515 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d200ce070 con 0x7f6d200b0760 2023-12-20T04:40:25.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.078+0000 7f6d1effd640 1 --2- 172.21.15.32:0/1875373515 >> v2:172.21.15.39:3300/0 conn(0x7f6d20157480 0x7f6d200c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:25.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d17fff640 1 --2- 172.21.15.32:0/1875373515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d200b0760 0x7f6d200b0bc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009870 tx=0x7f6d0801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fff2461d75ed35b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d1e7fc640 1 -- 172.21.15.32:0/1875373515 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080ab680 con 0x7f6d200b0760 2023-12-20T04:40:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d1e7fc640 1 -- 172.21.15.32:0/1875373515 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d080ab7e0 con 0x7f6d200b0760 2023-12-20T04:40:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d1e7fc640 1 -- 172.21.15.32:0/1875373515 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d080abb70 con 0x7f6d200b0760 2023-12-20T04:40:25.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d25495640 1 -- 172.21.15.32:0/1875373515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d200b0760 msgr2=0x7f6d200b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d25495640 1 --2- 172.21.15.32:0/1875373515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d200b0760 0x7f6d200b0bc0 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6d08009870 tx=0x7f6d0801c480 comp rx=0 tx=0).stop 2023-12-20T04:40:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d25495640 1 -- 172.21.15.32:0/1875373515 shutdown_connections 2023-12-20T04:40:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d25495640 1 --2- 172.21.15.32:0/1875373515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d200b0760 0x7f6d200b0bc0 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d25495640 1 --2- 172.21.15.32:0/1875373515 >> v2:172.21.15.39:3300/0 conn(0x7f6d20157480 0x7f6d200c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d25495640 1 -- 172.21.15.32:0/1875373515 >> 172.21.15.32:0/1875373515 conn(0x7f6d2006a510 msgr2=0x7f6d200ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d25495640 1 -- 172.21.15.32:0/1875373515 shutdown_connections 2023-12-20T04:40:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.082+0000 7f6d25495640 1 -- 172.21.15.32:0/1875373515 wait complete. 2023-12-20T04:40:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d25495640 1 Processor -- start 2023-12-20T04:40:25.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d25495640 1 -- start start 2023-12-20T04:40:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d25495640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d200b0760 0x7f6d201510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d25495640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d20157480 0x7f6d2006d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d25495640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d2015d660 0x7f6d2006d8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d25495640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d2015b730 con 0x7f6d200b0760 2023-12-20T04:40:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d25495640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d20151630 con 0x7f6d20157480 2023-12-20T04:40:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d25495640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d201f9830 con 0x7f6d2015d660 2023-12-20T04:40:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d1effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d200b0760 0x7f6d201510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d1effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d200b0760 0x7f6d201510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44252/0 (socket says 172.21.15.32:44252) 2023-12-20T04:40:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d1effd640 1 -- 172.21.15.32:0/985979481 learned_addr learned my addr 172.21.15.32:0/985979481 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d17fff640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d20157480 0x7f6d2006d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d1f7fe640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d2015d660 0x7f6d2006d8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d1effd640 1 -- 172.21.15.32:0/985979481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d2015d660 msgr2=0x7f6d2006d8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d1effd640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d2015d660 0x7f6d2006d8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d1effd640 1 -- 172.21.15.32:0/985979481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d20157480 msgr2=0x7f6d2006d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d1effd640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d20157480 0x7f6d2006d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d1effd640 1 -- 172.21.15.32:0/985979481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d10009420 con 0x7f6d200b0760 2023-12-20T04:40:25.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d1effd640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d200b0760 0x7f6d201510f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000e9b0 tx=0x7f6d1000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d177fe640 1 -- 172.21.15.32:0/985979481 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d1000ca00 con 0x7f6d200b0760 2023-12-20T04:40:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d177fe640 1 -- 172.21.15.32:0/985979481 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d1000cb60 con 0x7f6d200b0760 2023-12-20T04:40:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d177fe640 1 -- 172.21.15.32:0/985979481 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d10013470 con 0x7f6d200b0760 2023-12-20T04:40:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d25495640 1 -- 172.21.15.32:0/985979481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d080094d0 con 0x7f6d200b0760 2023-12-20T04:40:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.086+0000 7f6d25495640 1 -- 172.21.15.32:0/985979481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d2015b070 con 0x7f6d200b0760 2023-12-20T04:40:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.090+0000 7f6d25495640 1 -- 172.21.15.32:0/985979481 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d2004e000 con 0x7f6d200b0760 2023-12-20T04:40:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.090+0000 7f6d177fe640 1 -- 172.21.15.32:0/985979481 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f6d100135d0 con 0x7f6d200b0760 2023-12-20T04:40:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.090+0000 7f6d177fe640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d00065c60 0x7f6d00068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.090+0000 7f6d177fe640 1 -- 172.21.15.32:0/985979481 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f6d10083800 con 0x7f6d200b0760 2023-12-20T04:40:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.090+0000 7f6d17fff640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d00065c60 0x7f6d00068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.090+0000 7f6d17fff640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d00065c60 0x7f6d00068120 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d080af000 tx=0x7f6d080b6040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:25.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.094+0000 7f6d177fe640 1 -- 172.21.15.32:0/985979481 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6d10050f00 con 0x7f6d200b0760 2023-12-20T04:40:25.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.270+0000 7f6d25495640 1 -- 172.21.15.32:0/985979481 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir3", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d200c9360 con 0x7f6d00065c60 2023-12-20T04:40:25.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.278+0000 7f6d177fe640 1 -- 172.21.15.32:0/985979481 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f6d080ab950 con 0x7f6d00065c60 2023-12-20T04:40:25.277 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:25.277 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:50", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.278+0000 7f6d25495640 1 -- 172.21.15.32:0/985979481 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d00065c60 msgr2=0x7f6d00068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.278+0000 7f6d25495640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d00065c60 0x7f6d00068120 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f6d080af000 tx=0x7f6d080b6040 comp rx=0 tx=0).stop 2023-12-20T04:40:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.278+0000 7f6d25495640 1 -- 172.21.15.32:0/985979481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d200b0760 msgr2=0x7f6d201510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.278+0000 7f6d25495640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d200b0760 0x7f6d201510f0 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7f6d1000e9b0 tx=0x7f6d1000ee70 comp rx=0 tx=0).stop 2023-12-20T04:40:25.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.282+0000 7f6d25495640 1 -- 172.21.15.32:0/985979481 shutdown_connections 2023-12-20T04:40:25.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.282+0000 7f6d25495640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d00065c60 0x7f6d00068120 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.282+0000 7f6d25495640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d2015d660 0x7f6d2006d8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.282+0000 7f6d25495640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d20157480 0x7f6d2006d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.282+0000 7f6d25495640 1 --2- 172.21.15.32:0/985979481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d200b0760 0x7f6d201510f0 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.282+0000 7f6d25495640 1 -- 172.21.15.32:0/985979481 >> 172.21.15.32:0/985979481 conn(0x7f6d2006a510 msgr2=0x7f6d20164030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:25.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.282+0000 7f6d25495640 1 -- 172.21.15.32:0/985979481 shutdown_connections 2023-12-20T04:40:25.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.282+0000 7f6d25495640 1 -- 172.21.15.32:0/985979481 wait complete. 2023-12-20T04:40:25.299 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir3", "rel_path": "/snap_test_dir1/dir3", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:50", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:25.299 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir3', 'rel_path': '/snap_test_dir1/dir3', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T04:29:50', 'first': '2023-12-20T04:30:00', 'last': '2023-12-20T04:40:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-20T04:40:25.299 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-20T04:40:25.299 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-20T04:40:25.299 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:25.299 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-12-20T04:40:25.309 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:25.310 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:25.310 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:25.310 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:25.310 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:25.310 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:25.310 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:25.310 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:25.310 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:25.310 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:25.310 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:25.311 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-04_30_00_UTC', 'scheduled-2023-12-20-04_31_00_UTC', 'scheduled-2023-12-20-04_32_00_UTC', 'scheduled-2023-12-20-04_33_00_UTC', 'scheduled-2023-12-20-04_34_00_UTC', 'scheduled-2023-12-20-04_35_00_UTC', 'scheduled-2023-12-20-04_36_00_UTC', 'scheduled-2023-12-20-04_37_00_UTC', 'scheduled-2023-12-20-04_38_00_UTC', 'scheduled-2023-12-20-04_39_00_UTC', 'scheduled-2023-12-20-04_40_00_UTC'] 2023-12-20T04:40:25.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir4 2023-12-20T04:40:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.506+0000 7f15fdd96640 1 Processor -- start 2023-12-20T04:40:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.506+0000 7f15fdd96640 1 -- start start 2023-12-20T04:40:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15fdd96640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f15f8070940 0x7f15f80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15fdd96640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f815b6a0 con 0x7f15f815abb0 2023-12-20T04:40:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15fdd96640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f806d8c0 con 0x7f15f80b0cb0 2023-12-20T04:40:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15fdd96640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f806e160 con 0x7f15f8070940 2023-12-20T04:40:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15f77fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f15f815abb0 0x7f15f815bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58996/0 (socket says 172.21.15.32:58996) 2023-12-20T04:40:25.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15f77fe640 1 -- 172.21.15.32:0/3128138275 learned_addr learned my addr 172.21.15.32:0/3128138275 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:25.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15f7fff640 1 --2- 172.21.15.32:0/3128138275 >> v2:172.21.15.39:3300/0 conn(0x7f15f8070940 0x7f15f80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15f67fc640 1 -- 172.21.15.32:0/3128138275 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 26597444 0 0) 0x7f15f815b6a0 con 0x7f15f815abb0 2023-12-20T04:40:25.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15f67fc640 1 -- 172.21.15.32:0/3128138275 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f15e0003530 con 0x7f15f815abb0 2023-12-20T04:40:25.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15f7fff640 1 -- 172.21.15.32:0/3128138275 >> v1:172.21.15.39:6789/0 conn(0x7f15f80b0cb0 legacy=0x7f15f8070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15f67fc640 1 -- 172.21.15.32:0/3128138275 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 460067932 0 0) 0x7f15f806d8c0 con 0x7f15f80b0cb0 2023-12-20T04:40:25.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15f7fff640 1 -- 172.21.15.32:0/3128138275 >> v1:172.21.15.32:6789/0 conn(0x7f15f815abb0 legacy=0x7f15f815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15f7fff640 1 -- 172.21.15.32:0/3128138275 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f806dba0 con 0x7f15f8070940 2023-12-20T04:40:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.510+0000 7f15f7fff640 1 --2- 172.21.15.32:0/3128138275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f8070940 0x7f15f80caaf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009d90 tx=0x7f15f001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3686e34dc94f59c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15f67fc640 1 -- 172.21.15.32:0/3128138275 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00a5a10 con 0x7f15f8070940 2023-12-20T04:40:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15f67fc640 1 -- 172.21.15.32:0/3128138275 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f00a5b70 con 0x7f15f8070940 2023-12-20T04:40:25.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15f67fc640 1 -- 172.21.15.32:0/3128138275 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00ad440 con 0x7f15f8070940 2023-12-20T04:40:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 -- 172.21.15.32:0/3128138275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f8070940 msgr2=0x7f15f80caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 --2- 172.21.15.32:0/3128138275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f8070940 0x7f15f80caaf0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009d90 tx=0x7f15f001c950 comp rx=0 tx=0).stop 2023-12-20T04:40:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 -- 172.21.15.32:0/3128138275 shutdown_connections 2023-12-20T04:40:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 --2- 172.21.15.32:0/3128138275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f8070940 0x7f15f80caaf0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 -- 172.21.15.32:0/3128138275 >> 172.21.15.32:0/3128138275 conn(0x7f15f806a510 msgr2=0x7f15f80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 -- 172.21.15.32:0/3128138275 shutdown_connections 2023-12-20T04:40:25.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 -- 172.21.15.32:0/3128138275 wait complete. 2023-12-20T04:40:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 Processor -- start 2023-12-20T04:40:25.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 -- start start 2023-12-20T04:40:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f8070940 0x7f15f8145260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15f80b0cb0 0x7f15f8149cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f815abb0 0x7f15f814efc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f814a230 con 0x7f15f815abb0 2023-12-20T04:40:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f814a4d0 con 0x7f15f8070940 2023-12-20T04:40:25.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15fdd96640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f814a7b0 con 0x7f15f80b0cb0 2023-12-20T04:40:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.514+0000 7f15f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f815abb0 0x7f15f814efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f815abb0 0x7f15f814efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44260/0 (socket says 172.21.15.32:44260) 2023-12-20T04:40:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15f7fff640 1 -- 172.21.15.32:0/3475398471 learned_addr learned my addr 172.21.15.32:0/3475398471 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15f77fe640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f8070940 0x7f15f8145260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15f6ffd640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15f80b0cb0 0x7f15f8149cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15f7fff640 1 -- 172.21.15.32:0/3475398471 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15f80b0cb0 msgr2=0x7f15f8149cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15f7fff640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15f80b0cb0 0x7f15f8149cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15f7fff640 1 -- 172.21.15.32:0/3475398471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f8070940 msgr2=0x7f15f8145260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15f7fff640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f8070940 0x7f15f8145260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15f7fff640 1 -- 172.21.15.32:0/3475398471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f0009a70 con 0x7f15f815abb0 2023-12-20T04:40:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15f7fff640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f815abb0 0x7f15f814efc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f15f00b0040 tx=0x7f15f00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15dffff640 1 -- 172.21.15.32:0/3475398471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f00add30 con 0x7f15f815abb0 2023-12-20T04:40:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15dffff640 1 -- 172.21.15.32:0/3475398471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15f00ade90 con 0x7f15f815abb0 2023-12-20T04:40:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15fdd96640 1 -- 172.21.15.32:0/3475398471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f814aa90 con 0x7f15f815abb0 2023-12-20T04:40:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15dffff640 1 -- 172.21.15.32:0/3475398471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15f0003be0 con 0x7f15f815abb0 2023-12-20T04:40:25.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15fdd96640 1 -- 172.21.15.32:0/3475398471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f814eaa0 con 0x7f15f815abb0 2023-12-20T04:40:25.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15fdd96640 1 -- 172.21.15.32:0/3475398471 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f804e000 con 0x7f15f815abb0 2023-12-20T04:40:25.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.518+0000 7f15dffff640 1 -- 172.21.15.32:0/3475398471 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f15f00c4450 con 0x7f15f815abb0 2023-12-20T04:40:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.522+0000 7f15dffff640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f15e4065c60 0x7f15e4068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.522+0000 7f15dffff640 1 -- 172.21.15.32:0/3475398471 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f15f011e0a0 con 0x7f15f815abb0 2023-12-20T04:40:25.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.522+0000 7f15f77fe640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f15e4065c60 0x7f15e4068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.522+0000 7f15f77fe640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f15e4065c60 0x7f15e4068120 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f15e8007fd0 tx=0x7f15e8009040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:25.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.526+0000 7f15dffff640 1 -- 172.21.15.32:0/3475398471 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f15f00eb6f0 con 0x7f15f815abb0 2023-12-20T04:40:25.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.702+0000 7f15fdd96640 1 -- 172.21.15.32:0/3475398471 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir4", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f15f806cbf0 con 0x7f15e4065c60 2023-12-20T04:40:25.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.710+0000 7f15dffff640 1 -- 172.21.15.32:0/3475398471 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f15e80011c0 con 0x7f15e4065c60 2023-12-20T04:40:25.710 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:25.710 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:50", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 -- 172.21.15.32:0/3475398471 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f15e4065c60 msgr2=0x7f15e4068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f15e4065c60 0x7f15e4068120 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f15e8007fd0 tx=0x7f15e8009040 comp rx=0 tx=0).stop 2023-12-20T04:40:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 -- 172.21.15.32:0/3475398471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f815abb0 msgr2=0x7f15f814efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f815abb0 0x7f15f814efc0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7f15f00b0040 tx=0x7f15f00196f0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 -- 172.21.15.32:0/3475398471 shutdown_connections 2023-12-20T04:40:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f15e4065c60 0x7f15e4068120 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f815abb0 0x7f15f814efc0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15f80b0cb0 0x7f15f8149cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 --2- 172.21.15.32:0/3475398471 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f8070940 0x7f15f8145260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 -- 172.21.15.32:0/3475398471 >> 172.21.15.32:0/3475398471 conn(0x7f15f806a510 msgr2=0x7f15f80c4870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 -- 172.21.15.32:0/3475398471 shutdown_connections 2023-12-20T04:40:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.714+0000 7f15fdd96640 1 -- 172.21.15.32:0/3475398471 wait complete. 2023-12-20T04:40:25.731 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir4", "rel_path": "/snap_test_dir1/dir4", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:50", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:25.731 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir4', 'rel_path': '/snap_test_dir1/dir4', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T04:29:50', 'first': '2023-12-20T04:30:00', 'last': '2023-12-20T04:40:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-20T04:40:25.731 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-20T04:40:25.731 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-20T04:40:25.732 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:25.732 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-12-20T04:40:25.742 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:25.742 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:25.742 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:25.742 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:25.743 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:25.743 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:25.743 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:25.743 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:25.743 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:25.743 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:25.743 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:25.744 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-04_30_00_UTC', 'scheduled-2023-12-20-04_31_00_UTC', 'scheduled-2023-12-20-04_32_00_UTC', 'scheduled-2023-12-20-04_33_00_UTC', 'scheduled-2023-12-20-04_34_00_UTC', 'scheduled-2023-12-20-04_35_00_UTC', 'scheduled-2023-12-20-04_36_00_UTC', 'scheduled-2023-12-20-04_37_00_UTC', 'scheduled-2023-12-20-04_38_00_UTC', 'scheduled-2023-12-20-04_39_00_UTC', 'scheduled-2023-12-20-04_40_00_UTC'] 2023-12-20T04:40:25.744 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir5 2023-12-20T04:40:25.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4567739640 1 Processor -- start 2023-12-20T04:40:25.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4567739640 1 -- start start 2023-12-20T04:40:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4567739640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4560157480 0x7f45600c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4567739640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45600b0760 0x7f45600b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4567739640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45600b1100 con 0x7f456015d660 2023-12-20T04:40:25.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4567739640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45600cde20 con 0x7f45600b0760 2023-12-20T04:40:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4567739640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45600c8e30 con 0x7f4560157480 2023-12-20T04:40:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4564cd4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45600b0760 0x7f45600b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4564cd4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45600b0760 0x7f45600b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44268/0 (socket says 172.21.15.32:44268) 2023-12-20T04:40:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4564cd4640 1 -- 172.21.15.32:0/2960250689 learned_addr learned my addr 172.21.15.32:0/2960250689 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f45654d5640 1 --2- 172.21.15.32:0/2960250689 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4560157480 0x7f45600c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4564cd4640 1 -- 172.21.15.32:0/2960250689 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4560157480 msgr2=0x7f45600c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4564cd4640 1 --2- 172.21.15.32:0/2960250689 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4560157480 0x7f45600c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4564cd4640 1 -- 172.21.15.32:0/2960250689 >> v1:172.21.15.39:6789/0 conn(0x7f456015d660 legacy=0x7f456015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.934+0000 7f4564cd4640 1 -- 172.21.15.32:0/2960250689 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45600ce070 con 0x7f45600b0760 2023-12-20T04:40:25.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4564cd4640 1 --2- 172.21.15.32:0/2960250689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45600b0760 0x7f45600b0bc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4550009870 tx=0x7f455001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfb1d7aa8ac7fc51 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:25.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4557fff640 1 -- 172.21.15.32:0/2960250689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500ab680 con 0x7f45600b0760 2023-12-20T04:40:25.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4557fff640 1 -- 172.21.15.32:0/2960250689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45500ab7e0 con 0x7f45600b0760 2023-12-20T04:40:25.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4557fff640 1 -- 172.21.15.32:0/2960250689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500abb70 con 0x7f45600b0760 2023-12-20T04:40:25.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4567739640 1 -- 172.21.15.32:0/2960250689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45600b0760 msgr2=0x7f45600b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4567739640 1 --2- 172.21.15.32:0/2960250689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45600b0760 0x7f45600b0bc0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7f4550009870 tx=0x7f455001c480 comp rx=0 tx=0).stop 2023-12-20T04:40:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4567739640 1 -- 172.21.15.32:0/2960250689 shutdown_connections 2023-12-20T04:40:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4567739640 1 --2- 172.21.15.32:0/2960250689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45600b0760 0x7f45600b0bc0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4567739640 1 --2- 172.21.15.32:0/2960250689 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4560157480 0x7f45600c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4567739640 1 -- 172.21.15.32:0/2960250689 >> 172.21.15.32:0/2960250689 conn(0x7f456006a510 msgr2=0x7f45600ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4567739640 1 -- 172.21.15.32:0/2960250689 shutdown_connections 2023-12-20T04:40:25.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.938+0000 7f4567739640 1 -- 172.21.15.32:0/2960250689 wait complete. 2023-12-20T04:40:25.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4567739640 1 Processor -- start 2023-12-20T04:40:25.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4567739640 1 -- start start 2023-12-20T04:40:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4567739640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45600b0760 0x7f4560151ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4567739640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4560157480 0x7f45600b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4567739640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456015d660 0x7f45600c7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4567739640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4560152420 con 0x7f4560157480 2023-12-20T04:40:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4567739640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45600b3460 con 0x7f456015d660 2023-12-20T04:40:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4567739640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45600b3740 con 0x7f45600b0760 2023-12-20T04:40:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4564cd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4560157480 0x7f45600b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4564cd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4560157480 0x7f45600b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44276/0 (socket says 172.21.15.32:44276) 2023-12-20T04:40:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4564cd4640 1 -- 172.21.15.32:0/2804257743 learned_addr learned my addr 172.21.15.32:0/2804257743 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:25.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f45654d5640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45600b0760 0x7f4560151ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4565cd6640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456015d660 0x7f45600c7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4564cd4640 1 -- 172.21.15.32:0/2804257743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45600b0760 msgr2=0x7f4560151ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4564cd4640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45600b0760 0x7f4560151ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4564cd4640 1 -- 172.21.15.32:0/2804257743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456015d660 msgr2=0x7f45600c7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:25.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4564cd4640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456015d660 0x7f45600c7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:25.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4564cd4640 1 -- 172.21.15.32:0/2804257743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45500094d0 con 0x7f4560157480 2023-12-20T04:40:25.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4564cd4640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4560157480 0x7f45600b2af0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f45500ad040 tx=0x7f455001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:25.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4555ffb640 1 -- 172.21.15.32:0/2804257743 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500b4800 con 0x7f4560157480 2023-12-20T04:40:25.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4555ffb640 1 -- 172.21.15.32:0/2804257743 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45500b4980 con 0x7f4560157480 2023-12-20T04:40:25.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4555ffb640 1 -- 172.21.15.32:0/2804257743 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45500b4d30 con 0x7f4560157480 2023-12-20T04:40:25.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.942+0000 7f4567739640 1 -- 172.21.15.32:0/2804257743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f45600c6c90 con 0x7f4560157480 2023-12-20T04:40:25.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.946+0000 7f4567739640 1 -- 172.21.15.32:0/2804257743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45600c68b0 con 0x7f4560157480 2023-12-20T04:40:25.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.946+0000 7f4567739640 1 -- 172.21.15.32:0/2804257743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45600c6aa0 con 0x7f4560157480 2023-12-20T04:40:25.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.946+0000 7f4555ffb640 1 -- 172.21.15.32:0/2804257743 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f45500b3380 con 0x7f4560157480 2023-12-20T04:40:25.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.946+0000 7f4555ffb640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4544065c60 0x7f4544068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:25.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.946+0000 7f4555ffb640 1 -- 172.21.15.32:0/2804257743 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f455011e0e0 con 0x7f4560157480 2023-12-20T04:40:25.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.946+0000 7f45654d5640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4544065c60 0x7f4544068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:25.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.946+0000 7f45654d5640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4544065c60 0x7f4544068120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f455c0079c0 tx=0x7f455c008040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:25.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:25.950+0000 7f4555ffb640 1 -- 172.21.15.32:0/2804257743 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f45500eb6d0 con 0x7f4560157480 2023-12-20T04:40:26.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.126+0000 7f4567739640 1 -- 172.21.15.32:0/2804257743 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir5", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f45600b35d0 con 0x7f4544065c60 2023-12-20T04:40:26.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4555ffb640 1 -- 172.21.15.32:0/2804257743 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f45600b35d0 con 0x7f4544065c60 2023-12-20T04:40:26.135 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:26.135 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:51", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 -- 172.21.15.32:0/2804257743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4544065c60 msgr2=0x7f4544068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4544065c60 0x7f4544068120 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f455c0079c0 tx=0x7f455c008040 comp rx=0 tx=0).stop 2023-12-20T04:40:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 -- 172.21.15.32:0/2804257743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4560157480 msgr2=0x7f45600b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4560157480 0x7f45600b2af0 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f45500ad040 tx=0x7f455001acc0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 -- 172.21.15.32:0/2804257743 shutdown_connections 2023-12-20T04:40:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4544065c60 0x7f4544068120 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f456015d660 0x7f45600c7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4560157480 0x7f45600b2af0 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 --2- 172.21.15.32:0/2804257743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45600b0760 0x7f4560151ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 -- 172.21.15.32:0/2804257743 >> 172.21.15.32:0/2804257743 conn(0x7f456006a510 msgr2=0x7f45600cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 -- 172.21.15.32:0/2804257743 shutdown_connections 2023-12-20T04:40:26.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.138+0000 7f4567739640 1 -- 172.21.15.32:0/2804257743 wait complete. 2023-12-20T04:40:26.156 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir5", "rel_path": "/snap_test_dir1/dir5", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:51", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:26.157 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir5', 'rel_path': '/snap_test_dir1/dir5', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T04:29:51', 'first': '2023-12-20T04:30:00', 'last': '2023-12-20T04:40:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-20T04:40:26.157 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-20T04:40:26.157 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-20T04:40:26.157 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:26.157 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-12-20T04:40:26.168 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:26.168 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:26.168 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:26.168 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:26.168 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:26.168 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:26.168 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:26.168 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:26.168 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:26.168 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:26.168 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:26.169 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-04_30_00_UTC', 'scheduled-2023-12-20-04_31_00_UTC', 'scheduled-2023-12-20-04_32_00_UTC', 'scheduled-2023-12-20-04_33_00_UTC', 'scheduled-2023-12-20-04_34_00_UTC', 'scheduled-2023-12-20-04_35_00_UTC', 'scheduled-2023-12-20-04_36_00_UTC', 'scheduled-2023-12-20-04_37_00_UTC', 'scheduled-2023-12-20-04_38_00_UTC', 'scheduled-2023-12-20-04_39_00_UTC', 'scheduled-2023-12-20-04_40_00_UTC'] 2023-12-20T04:40:26.169 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir6 2023-12-20T04:40:26.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0262560640 1 Processor -- start 2023-12-20T04:40:26.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0262560640 1 -- start start 2023-12-20T04:40:26.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0262560640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f025c0b0cb0 0x7f025c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0262560640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c070230 0x7f025c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0262560640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f025c15b6a0 con 0x7f025c15abb0 2023-12-20T04:40:26.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0262560640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f025c06e160 con 0x7f025c0b0cb0 2023-12-20T04:40:26.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0262560640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c070c00 con 0x7f025c070230 2023-12-20T04:40:26.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0260afd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c070230 0x7f025c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f025b7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f025c0b0cb0 0x7f025c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0260afd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c070230 0x7f025c0706c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:41950/0 (socket says 172.21.15.32:41950) 2023-12-20T04:40:26.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0260afd640 1 -- 172.21.15.32:0/520518584 learned_addr learned my addr 172.21.15.32:0/520518584 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:26.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0260afd640 1 -- 172.21.15.32:0/520518584 >> v2:172.21.15.39:3300/0 conn(0x7f025c0b0cb0 msgr2=0x7f025c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:26.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0260afd640 1 --2- 172.21.15.32:0/520518584 >> v2:172.21.15.39:3300/0 conn(0x7f025c0b0cb0 0x7f025c0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0260afd640 1 -- 172.21.15.32:0/520518584 >> v1:172.21.15.39:6789/0 conn(0x7f025c15abb0 legacy=0x7f025c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.358+0000 7f0260afd640 1 -- 172.21.15.32:0/520518584 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f025c06d8c0 con 0x7f025c070230 2023-12-20T04:40:26.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f0260afd640 1 --2- 172.21.15.32:0/520518584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c070230 0x7f025c0706c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0254009e50 tx=0x7f025401ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5a550f192351cd21 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:26.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f025affd640 1 -- 172.21.15.32:0/520518584 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02540a58e0 con 0x7f025c070230 2023-12-20T04:40:26.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f025affd640 1 -- 172.21.15.32:0/520518584 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02540a5a40 con 0x7f025c070230 2023-12-20T04:40:26.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f025affd640 1 -- 172.21.15.32:0/520518584 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02540a5dd0 con 0x7f025c070230 2023-12-20T04:40:26.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f0262560640 1 -- 172.21.15.32:0/520518584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c070230 msgr2=0x7f025c0706c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f0262560640 1 --2- 172.21.15.32:0/520518584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c070230 0x7f025c0706c0 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0254009e50 tx=0x7f025401ca10 comp rx=0 tx=0).stop 2023-12-20T04:40:26.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f0262560640 1 -- 172.21.15.32:0/520518584 shutdown_connections 2023-12-20T04:40:26.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f0262560640 1 --2- 172.21.15.32:0/520518584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c070230 0x7f025c0706c0 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f0262560640 1 --2- 172.21.15.32:0/520518584 >> v2:172.21.15.39:3300/0 conn(0x7f025c0b0cb0 0x7f025c0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f0262560640 1 -- 172.21.15.32:0/520518584 >> 172.21.15.32:0/520518584 conn(0x7f025c06a510 msgr2=0x7f025c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:26.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f0262560640 1 -- 172.21.15.32:0/520518584 shutdown_connections 2023-12-20T04:40:26.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.362+0000 7f0262560640 1 -- 172.21.15.32:0/520518584 wait complete. 2023-12-20T04:40:26.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0262560640 1 Processor -- start 2023-12-20T04:40:26.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0262560640 1 -- start start 2023-12-20T04:40:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0262560640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f025c070230 0x7f025c0c3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0262560640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c0b0cb0 0x7f025c133f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0262560640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c15abb0 0x7f025c134490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0262560640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c150540 con 0x7f025c070230 2023-12-20T04:40:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0262560640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f025c131f60 con 0x7f025c15abb0 2023-12-20T04:40:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0262560640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f025c132240 con 0x7f025c0b0cb0 2023-12-20T04:40:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f025bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f025c070230 0x7f025c0c3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f025bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f025c070230 0x7f025c0c3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44280/0 (socket says 172.21.15.32:44280) 2023-12-20T04:40:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f025bfff640 1 -- 172.21.15.32:0/3326351847 learned_addr learned my addr 172.21.15.32:0/3326351847 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:26.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f025b7fe640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c0b0cb0 0x7f025c133f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0260afd640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c15abb0 0x7f025c134490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f025bfff640 1 -- 172.21.15.32:0/3326351847 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c0b0cb0 msgr2=0x7f025c133f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f025bfff640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c0b0cb0 0x7f025c133f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f025bfff640 1 -- 172.21.15.32:0/3326351847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c15abb0 msgr2=0x7f025c134490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f025bfff640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c15abb0 0x7f025c134490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f025bfff640 1 -- 172.21.15.32:0/3326351847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0254009b30 con 0x7f025c070230 2023-12-20T04:40:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0260afd640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c15abb0 0x7f025c134490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f025bfff640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f025c070230 0x7f025c0c3cd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c500 tx=0x7f024c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0258ff9640 1 -- 172.21.15.32:0/3326351847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c005220 con 0x7f025c070230 2023-12-20T04:40:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0258ff9640 1 -- 172.21.15.32:0/3326351847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f024c005380 con 0x7f025c070230 2023-12-20T04:40:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0262560640 1 -- 172.21.15.32:0/3326351847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f025c132520 con 0x7f025c070230 2023-12-20T04:40:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0258ff9640 1 -- 172.21.15.32:0/3326351847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f024c007270 con 0x7f025c070230 2023-12-20T04:40:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.366+0000 7f0262560640 1 -- 172.21.15.32:0/3326351847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f025c0cc3b0 con 0x7f025c070230 2023-12-20T04:40:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.370+0000 7f0262560640 1 -- 172.21.15.32:0/3326351847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f025c04e000 con 0x7f025c070230 2023-12-20T04:40:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.370+0000 7f0258ff9640 1 -- 172.21.15.32:0/3326351847 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f024c0073d0 con 0x7f025c070230 2023-12-20T04:40:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.370+0000 7f0258ff9640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0248065c60 0x7f0248068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.370+0000 7f0258ff9640 1 -- 172.21.15.32:0/3326351847 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f024c082f00 con 0x7f025c070230 2023-12-20T04:40:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.370+0000 7f025b7fe640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0248065c60 0x7f0248068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.370+0000 7f025b7fe640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0248065c60 0x7f0248068120 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f025c0afed0 tx=0x7f0250009190 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:26.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.374+0000 7f0258ff9640 1 -- 172.21.15.32:0/3326351847 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f024c0504b0 con 0x7f025c070230 2023-12-20T04:40:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.550+0000 7f0262560640 1 -- 172.21.15.32:0/3326351847 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir6", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f025c15b300 con 0x7f0248065c60 2023-12-20T04:40:26.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.558+0000 7f0258ff9640 1 -- 172.21.15.32:0/3326351847 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f025c15b300 con 0x7f0248065c60 2023-12-20T04:40:26.557 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:26.557 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:51", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:26.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 -- 172.21.15.32:0/3326351847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0248065c60 msgr2=0x7f0248068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0248065c60 0x7f0248068120 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f025c0afed0 tx=0x7f0250009190 comp rx=0 tx=0).stop 2023-12-20T04:40:26.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 -- 172.21.15.32:0/3326351847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f025c070230 msgr2=0x7f025c0c3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f025c070230 0x7f025c0c3cd0 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7f024c00c500 tx=0x7f024c00c9c0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 -- 172.21.15.32:0/3326351847 shutdown_connections 2023-12-20T04:40:26.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0248065c60 0x7f0248068120 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f025c15abb0 0x7f025c134490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f025c0b0cb0 0x7f025c133f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 --2- 172.21.15.32:0/3326351847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f025c070230 0x7f025c0c3cd0 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 -- 172.21.15.32:0/3326351847 >> 172.21.15.32:0/3326351847 conn(0x7f025c06a510 msgr2=0x7f025c0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:26.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 -- 172.21.15.32:0/3326351847 shutdown_connections 2023-12-20T04:40:26.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.562+0000 7f0262560640 1 -- 172.21.15.32:0/3326351847 wait complete. 2023-12-20T04:40:26.579 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir6", "rel_path": "/snap_test_dir1/dir6", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:51", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:26.579 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir6', 'rel_path': '/snap_test_dir1/dir6', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T04:29:51', 'first': '2023-12-20T04:30:00', 'last': '2023-12-20T04:40:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-20T04:40:26.579 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-20T04:40:26.579 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-20T04:40:26.579 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:26.579 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-12-20T04:40:26.590 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:26.590 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:26.590 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:26.590 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:26.590 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:26.590 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:26.591 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:26.591 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:26.591 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:26.591 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:26.591 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:26.592 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-04_30_00_UTC', 'scheduled-2023-12-20-04_31_00_UTC', 'scheduled-2023-12-20-04_32_00_UTC', 'scheduled-2023-12-20-04_33_00_UTC', 'scheduled-2023-12-20-04_34_00_UTC', 'scheduled-2023-12-20-04_35_00_UTC', 'scheduled-2023-12-20-04_36_00_UTC', 'scheduled-2023-12-20-04_37_00_UTC', 'scheduled-2023-12-20-04_38_00_UTC', 'scheduled-2023-12-20-04_39_00_UTC', 'scheduled-2023-12-20-04_40_00_UTC'] 2023-12-20T04:40:26.592 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir7 2023-12-20T04:40:26.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.782+0000 7f7ebe01f640 1 Processor -- start 2023-12-20T04:40:26.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.782+0000 7f7ebe01f640 1 -- start start 2023-12-20T04:40:26.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7ebe01f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7eb806cdf0 0x7f7eb8070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7ebe01f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7eb8070770 0x7f7eb80cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7ebe01f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7eb80cc3b0 con 0x7f7eb81591e0 2023-12-20T04:40:26.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7ebe01f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb8070ba0 con 0x7f7eb806cdf0 2023-12-20T04:40:26.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7ebe01f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7eb80c8e30 con 0x7f7eb8070770 2023-12-20T04:40:26.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7eb6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7eb806cdf0 0x7f7eb8070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7eb6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7eb806cdf0 0x7f7eb8070230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44290/0 (socket says 172.21.15.32:44290) 2023-12-20T04:40:26.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7eb77fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7eb81591e0 0x7f7eb80caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59012/0 (socket says 172.21.15.32:59012) 2023-12-20T04:40:26.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7eb6ffd640 1 -- 172.21.15.32:0/4253319885 learned_addr learned my addr 172.21.15.32:0/4253319885 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:26.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7eb7fff640 1 --2- 172.21.15.32:0/4253319885 >> v2:172.21.15.39:3300/0 conn(0x7f7eb8070770 0x7f7eb80cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7eb6ffd640 1 -- 172.21.15.32:0/4253319885 >> v2:172.21.15.39:3300/0 conn(0x7f7eb8070770 msgr2=0x7f7eb80cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:26.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7eb6ffd640 1 --2- 172.21.15.32:0/4253319885 >> v2:172.21.15.39:3300/0 conn(0x7f7eb8070770 0x7f7eb80cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7eb6ffd640 1 -- 172.21.15.32:0/4253319885 >> v1:172.21.15.32:6789/0 conn(0x7f7eb81591e0 legacy=0x7f7eb80caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7eb6ffd640 1 -- 172.21.15.32:0/4253319885 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eb806d280 con 0x7f7eb806cdf0 2023-12-20T04:40:26.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.786+0000 7f7eb67fc640 1 -- 172.21.15.32:0/4253319885 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193080809 0 0) 0x7f7eb80cc3b0 con 0x7f7eb81591e0 2023-12-20T04:40:26.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7eb6ffd640 1 --2- 172.21.15.32:0/4253319885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eb806cdf0 0x7f7eb8070230 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80c5860 tx=0x7f7eac01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed58b1797e783f37 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:26.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7eb67fc640 1 -- 172.21.15.32:0/4253319885 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0ab600 con 0x7f7eb806cdf0 2023-12-20T04:40:26.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7eb67fc640 1 -- 172.21.15.32:0/4253319885 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac0a7050 con 0x7f7eb806cdf0 2023-12-20T04:40:26.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7eb67fc640 1 -- 172.21.15.32:0/4253319885 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0ab980 con 0x7f7eb806cdf0 2023-12-20T04:40:26.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 -- 172.21.15.32:0/4253319885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eb806cdf0 msgr2=0x7f7eb8070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 --2- 172.21.15.32:0/4253319885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eb806cdf0 0x7f7eb8070230 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7f7eb80c5860 tx=0x7f7eac01c470 comp rx=0 tx=0).stop 2023-12-20T04:40:26.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 -- 172.21.15.32:0/4253319885 shutdown_connections 2023-12-20T04:40:26.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 --2- 172.21.15.32:0/4253319885 >> v2:172.21.15.39:3300/0 conn(0x7f7eb8070770 0x7f7eb80cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 --2- 172.21.15.32:0/4253319885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eb806cdf0 0x7f7eb8070230 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 -- 172.21.15.32:0/4253319885 >> 172.21.15.32:0/4253319885 conn(0x7f7eb806a510 msgr2=0x7f7eb80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:26.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 -- 172.21.15.32:0/4253319885 shutdown_connections 2023-12-20T04:40:26.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 -- 172.21.15.32:0/4253319885 wait complete. 2023-12-20T04:40:26.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 Processor -- start 2023-12-20T04:40:26.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 -- start start 2023-12-20T04:40:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb8070770 0x7f7eb8163740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eb81591e0 0x7f7eb8156a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7eb8163c80 0x7f7eb8156f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb81633a0 con 0x7f7eb81591e0 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb8162f10 con 0x7f7eb8070770 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.790+0000 7f7ebe01f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7eb8162370 con 0x7f7eb8163c80 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7eb6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eb81591e0 0x7f7eb8156a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7eb6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eb81591e0 0x7f7eb8156a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44296/0 (socket says 172.21.15.32:44296) 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7eb6ffd640 1 -- 172.21.15.32:0/2927865416 learned_addr learned my addr 172.21.15.32:0/2927865416 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7eb7fff640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7eb8163c80 0x7f7eb8156f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7eb77fe640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb8070770 0x7f7eb8163740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7eb6ffd640 1 -- 172.21.15.32:0/2927865416 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7eb8163c80 msgr2=0x7f7eb8156f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7eb6ffd640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7eb8163c80 0x7f7eb8156f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7eb6ffd640 1 -- 172.21.15.32:0/2927865416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb8070770 msgr2=0x7f7eb8163740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7eb6ffd640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb8070770 0x7f7eb8163740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7eb6ffd640 1 -- 172.21.15.32:0/2927865416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7eac0094d0 con 0x7f7eb81591e0 2023-12-20T04:40:26.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7eb6ffd640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eb81591e0 0x7f7eb8156a20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0ae040 tx=0x7f7eac019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7e9ffff640 1 -- 172.21.15.32:0/2927865416 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0b4420 con 0x7f7eb81591e0 2023-12-20T04:40:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7e9ffff640 1 -- 172.21.15.32:0/2927865416 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7eac0b4580 con 0x7f7eb81591e0 2023-12-20T04:40:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7ebe01f640 1 -- 172.21.15.32:0/2927865416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7eb81625e0 con 0x7f7eb81591e0 2023-12-20T04:40:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7e9ffff640 1 -- 172.21.15.32:0/2927865416 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7eac0b4930 con 0x7f7eb81591e0 2023-12-20T04:40:26.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7ebe01f640 1 -- 172.21.15.32:0/2927865416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7eb8162ba0 con 0x7f7eb81591e0 2023-12-20T04:40:26.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7ebe01f640 1 -- 172.21.15.32:0/2927865416 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7eb8163080 con 0x7f7eb81591e0 2023-12-20T04:40:26.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.794+0000 7f7e9ffff640 1 -- 172.21.15.32:0/2927865416 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f7eac0b2070 con 0x7f7eb81591e0 2023-12-20T04:40:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.798+0000 7f7e9ffff640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ea4065c90 0x7f7ea4068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.798+0000 7f7e9ffff640 1 -- 172.21.15.32:0/2927865416 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f7eac11d1a0 con 0x7f7eb81591e0 2023-12-20T04:40:26.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.798+0000 7f7eb77fe640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ea4065c90 0x7f7ea4068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:26.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.798+0000 7f7eb77fe640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ea4065c90 0x7f7ea4068150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8000f70 tx=0x7f7ea8007c10 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:26.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.802+0000 7f7e9ffff640 1 -- 172.21.15.32:0/2927865416 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7eac0ea7e0 con 0x7f7eb81591e0 2023-12-20T04:40:26.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.982+0000 7f7ebe01f640 1 -- 172.21.15.32:0/2927865416 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir7", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7eb80ce4d0 con 0x7f7ea4065c90 2023-12-20T04:40:26.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.990+0000 7f7e9ffff640 1 -- 172.21.15.32:0/2927865416 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f7ea8007270 con 0x7f7ea4065c90 2023-12-20T04:40:26.989 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:26.990 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:52", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.990+0000 7f7ebe01f640 1 -- 172.21.15.32:0/2927865416 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ea4065c90 msgr2=0x7f7ea4068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.990+0000 7f7ebe01f640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ea4065c90 0x7f7ea4068150 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7ea8000f70 tx=0x7f7ea8007c10 comp rx=0 tx=0).stop 2023-12-20T04:40:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.990+0000 7f7ebe01f640 1 -- 172.21.15.32:0/2927865416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eb81591e0 msgr2=0x7f7eb8156a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.990+0000 7f7ebe01f640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eb81591e0 0x7f7eb8156a20 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7f7eac0ae040 tx=0x7f7eac019210 comp rx=0 tx=0).stop 2023-12-20T04:40:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.990+0000 7f7ebe01f640 1 -- 172.21.15.32:0/2927865416 shutdown_connections 2023-12-20T04:40:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.990+0000 7f7ebe01f640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ea4065c90 0x7f7ea4068150 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.990+0000 7f7ebe01f640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7eb8163c80 0x7f7eb8156f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.990+0000 7f7ebe01f640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7eb81591e0 0x7f7eb8156a20 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.990+0000 7f7ebe01f640 1 --2- 172.21.15.32:0/2927865416 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7eb8070770 0x7f7eb8163740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.990+0000 7f7ebe01f640 1 -- 172.21.15.32:0/2927865416 >> 172.21.15.32:0/2927865416 conn(0x7f7eb806a510 msgr2=0x7f7eb80ce650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.994+0000 7f7ebe01f640 1 -- 172.21.15.32:0/2927865416 shutdown_connections 2023-12-20T04:40:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:26.994+0000 7f7ebe01f640 1 -- 172.21.15.32:0/2927865416 wait complete. 2023-12-20T04:40:27.011 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir7", "rel_path": "/snap_test_dir1/dir7", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:52", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:27.011 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir7', 'rel_path': '/snap_test_dir1/dir7', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T04:29:52', 'first': '2023-12-20T04:30:00', 'last': '2023-12-20T04:40:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-20T04:40:27.011 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-20T04:40:27.011 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-20T04:40:27.011 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:27.011 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-12-20T04:40:27.022 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:27.022 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:27.022 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:27.022 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:27.022 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:27.022 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:27.022 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:27.022 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:27.022 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:27.022 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:27.022 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:27.023 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-04_30_00_UTC', 'scheduled-2023-12-20-04_31_00_UTC', 'scheduled-2023-12-20-04_32_00_UTC', 'scheduled-2023-12-20-04_33_00_UTC', 'scheduled-2023-12-20-04_34_00_UTC', 'scheduled-2023-12-20-04_35_00_UTC', 'scheduled-2023-12-20-04_36_00_UTC', 'scheduled-2023-12-20-04_37_00_UTC', 'scheduled-2023-12-20-04_38_00_UTC', 'scheduled-2023-12-20-04_39_00_UTC', 'scheduled-2023-12-20-04_40_00_UTC'] 2023-12-20T04:40:27.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir8 2023-12-20T04:40:27.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8ab06e5640 1 Processor -- start 2023-12-20T04:40:27.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8ab06e5640 1 -- start start 2023-12-20T04:40:27.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8ab06e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8aa8065d40 0x7f8aa80cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8ab06e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8aa806d170 0x7f8aa8161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8ab06e5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8aa806a1f0 con 0x7f8aa80cb050 2023-12-20T04:40:27.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8ab06e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8aa8161860 con 0x7f8aa8065d40 2023-12-20T04:40:27.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8ab06e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa806a440 con 0x7f8aa806d170 2023-12-20T04:40:27.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8aadc80640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8aa8065d40 0x7f8aa80cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8aadc80640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8aa8065d40 0x7f8aa80cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44312/0 (socket says 172.21.15.32:44312) 2023-12-20T04:40:27.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8aadc80640 1 -- 172.21.15.32:0/662843545 learned_addr learned my addr 172.21.15.32:0/662843545 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:27.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8aae481640 1 --1- 172.21.15.32:0/662843545 >> v1:172.21.15.32:6789/0 conn(0x7f8aa80cb050 0x7f8aa806ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59026/0 (socket says 172.21.15.32:59026) 2023-12-20T04:40:27.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8aaec82640 1 --2- 172.21.15.32:0/662843545 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8aa806d170 0x7f8aa8161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8aadc80640 1 -- 172.21.15.32:0/662843545 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8aa806d170 msgr2=0x7f8aa8161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:27.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8aadc80640 1 --2- 172.21.15.32:0/662843545 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8aa806d170 0x7f8aa8161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8aadc80640 1 -- 172.21.15.32:0/662843545 >> v1:172.21.15.32:6789/0 conn(0x7f8aa80cb050 legacy=0x7f8aa806ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8aadc80640 1 -- 172.21.15.32:0/662843545 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8aa806a720 con 0x7f8aa8065d40 2023-12-20T04:40:27.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.214+0000 7f8aad47f640 1 -- 172.21.15.32:0/662843545 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 993359835 0 0) 0x7f8aa806a1f0 con 0x7f8aa80cb050 2023-12-20T04:40:27.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8aadc80640 1 --2- 172.21.15.32:0/662843545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa8065d40 0x7f8aa80cb430 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009870 tx=0x7f8a9801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f732ba5dea0bc8df server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:27.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8aad47f640 1 -- 172.21.15.32:0/662843545 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a980aa670 con 0x7f8aa8065d40 2023-12-20T04:40:27.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8aad47f640 1 -- 172.21.15.32:0/662843545 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a980aa7d0 con 0x7f8aa8065d40 2023-12-20T04:40:27.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8aad47f640 1 -- 172.21.15.32:0/662843545 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a980aab60 con 0x7f8aa8065d40 2023-12-20T04:40:27.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8ab06e5640 1 -- 172.21.15.32:0/662843545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa8065d40 msgr2=0x7f8aa80cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8ab06e5640 1 --2- 172.21.15.32:0/662843545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa8065d40 0x7f8aa80cb430 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x7f8a98009870 tx=0x7f8a9801c510 comp rx=0 tx=0).stop 2023-12-20T04:40:27.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8ab06e5640 1 -- 172.21.15.32:0/662843545 shutdown_connections 2023-12-20T04:40:27.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8ab06e5640 1 --2- 172.21.15.32:0/662843545 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa8065d40 0x7f8aa80cb430 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8ab06e5640 1 --2- 172.21.15.32:0/662843545 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8aa806d170 0x7f8aa8161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8ab06e5640 1 -- 172.21.15.32:0/662843545 >> 172.21.15.32:0/662843545 conn(0x7f8aa80cf8d0 msgr2=0x7f8aa80cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:27.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8ab06e5640 1 -- 172.21.15.32:0/662843545 shutdown_connections 2023-12-20T04:40:27.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.218+0000 7f8ab06e5640 1 -- 172.21.15.32:0/662843545 wait complete. 2023-12-20T04:40:27.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8ab06e5640 1 Processor -- start 2023-12-20T04:40:27.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8ab06e5640 1 -- start start 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8ab06e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8aa8065d40 0x7f8aa814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8ab06e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa806d170 0x7f8aa8154940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8ab06e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aa80cb050 0x7f8aa8151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8ab06e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa8152f80 con 0x7f8aa806d170 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8ab06e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa8150fb0 con 0x7f8aa80cb050 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8ab06e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8aa814f8a0 con 0x7f8aa8065d40 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8aadc80640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa806d170 0x7f8aa8154940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8aadc80640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa806d170 0x7f8aa8154940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44328/0 (socket says 172.21.15.32:44328) 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8aadc80640 1 -- 172.21.15.32:0/1350805195 learned_addr learned my addr 172.21.15.32:0/1350805195 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8aaec82640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aa80cb050 0x7f8aa8151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8aae481640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8aa8065d40 0x7f8aa814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8aadc80640 1 -- 172.21.15.32:0/1350805195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8aa8065d40 msgr2=0x7f8aa814e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8aadc80640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8aa8065d40 0x7f8aa814e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8aadc80640 1 -- 172.21.15.32:0/1350805195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aa80cb050 msgr2=0x7f8aa8151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8aadc80640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aa80cb050 0x7f8aa8151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8aadc80640 1 -- 172.21.15.32:0/1350805195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a980094d0 con 0x7f8aa806d170 2023-12-20T04:40:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8aadc80640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa806d170 0x7f8aa8154940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8a980ac040 tx=0x7f8a9801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8a9effd640 1 -- 172.21.15.32:0/1350805195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a980b3800 con 0x7f8aa806d170 2023-12-20T04:40:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8a9effd640 1 -- 172.21.15.32:0/1350805195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a980b3980 con 0x7f8aa806d170 2023-12-20T04:40:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8a9effd640 1 -- 172.21.15.32:0/1350805195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a980b3d30 con 0x7f8aa806d170 2023-12-20T04:40:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.222+0000 7f8ab06e5640 1 -- 172.21.15.32:0/1350805195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8aa814f5c0 con 0x7f8aa806d170 2023-12-20T04:40:27.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.226+0000 7f8ab06e5640 1 -- 172.21.15.32:0/1350805195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8aa8151530 con 0x7f8aa806d170 2023-12-20T04:40:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.226+0000 7f8ab06e5640 1 -- 172.21.15.32:0/1350805195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8aa804e000 con 0x7f8aa806d170 2023-12-20T04:40:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.226+0000 7f8a9effd640 1 -- 172.21.15.32:0/1350805195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f8a980bd3f0 con 0x7f8aa806d170 2023-12-20T04:40:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.226+0000 7f8a9effd640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a8c065c60 0x7f8a8c068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.226+0000 7f8a9effd640 1 -- 172.21.15.32:0/1350805195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f8a9811e3b0 con 0x7f8aa806d170 2023-12-20T04:40:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.226+0000 7f8aae481640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a8c065c60 0x7f8a8c068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.226+0000 7f8aae481640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a8c065c60 0x7f8a8c068120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4000f70 tx=0x7f8aa4007b60 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:27.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.230+0000 7f8a9effd640 1 -- 172.21.15.32:0/1350805195 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8a980eb980 con 0x7f8aa806d170 2023-12-20T04:40:27.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.410+0000 7f8ab06e5640 1 -- 172.21.15.32:0/1350805195 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir8", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8aa806afc0 con 0x7f8a8c065c60 2023-12-20T04:40:27.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8a9effd640 1 -- 172.21.15.32:0/1350805195 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f8aa40071f0 con 0x7f8a8c065c60 2023-12-20T04:40:27.416 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:27.416 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:52", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 -- 172.21.15.32:0/1350805195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a8c065c60 msgr2=0x7f8a8c068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a8c065c60 0x7f8a8c068120 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8aa4000f70 tx=0x7f8aa4007b60 comp rx=0 tx=0).stop 2023-12-20T04:40:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 -- 172.21.15.32:0/1350805195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa806d170 msgr2=0x7f8aa8154940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa806d170 0x7f8aa8154940 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7f8a980ac040 tx=0x7f8a9801ad50 comp rx=0 tx=0).stop 2023-12-20T04:40:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 -- 172.21.15.32:0/1350805195 shutdown_connections 2023-12-20T04:40:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a8c065c60 0x7f8a8c068120 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8aa80cb050 0x7f8aa8151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8aa806d170 0x7f8aa8154940 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 --2- 172.21.15.32:0/1350805195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8aa8065d40 0x7f8aa814e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 -- 172.21.15.32:0/1350805195 >> 172.21.15.32:0/1350805195 conn(0x7f8aa80cf8d0 msgr2=0x7f8aa80c8e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 -- 172.21.15.32:0/1350805195 shutdown_connections 2023-12-20T04:40:27.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.418+0000 7f8ab06e5640 1 -- 172.21.15.32:0/1350805195 wait complete. 2023-12-20T04:40:27.437 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir8", "rel_path": "/snap_test_dir1/dir8", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:52", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:27.437 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir8', 'rel_path': '/snap_test_dir1/dir8', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T04:29:52', 'first': '2023-12-20T04:30:00', 'last': '2023-12-20T04:40:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-20T04:40:27.437 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-20T04:40:27.437 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-20T04:40:27.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:27.437 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-12-20T04:40:27.448 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:27.448 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:27.448 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:27.448 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:27.448 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:27.449 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:27.449 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:27.449 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:27.449 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:27.449 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:27.449 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:27.450 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-04_30_00_UTC', 'scheduled-2023-12-20-04_31_00_UTC', 'scheduled-2023-12-20-04_32_00_UTC', 'scheduled-2023-12-20-04_33_00_UTC', 'scheduled-2023-12-20-04_34_00_UTC', 'scheduled-2023-12-20-04_35_00_UTC', 'scheduled-2023-12-20-04_36_00_UTC', 'scheduled-2023-12-20-04_37_00_UTC', 'scheduled-2023-12-20-04_38_00_UTC', 'scheduled-2023-12-20-04_39_00_UTC', 'scheduled-2023-12-20-04_40_00_UTC'] 2023-12-20T04:40:27.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/dir9 2023-12-20T04:40:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66bcbcb640 1 Processor -- start 2023-12-20T04:40:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66bcbcb640 1 -- start start 2023-12-20T04:40:27.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66bcbcb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f66b80b6120 0x7f66b815b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66bcbcb640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b815ed90 con 0x7f66b80b2240 2023-12-20T04:40:27.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66bcbcb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b815f070 con 0x7f66b8158650 2023-12-20T04:40:27.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66bcbcb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b815eba0 con 0x7f66b80b6120 2023-12-20T04:40:27.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f66b80b2240 0x7f66b815c060 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59040/0 (socket says 172.21.15.32:59040) 2023-12-20T04:40:27.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b7fff640 1 -- 172.21.15.32:0/397641675 learned_addr learned my addr 172.21.15.32:0/397641675 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:27.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b6ffd640 1 --2- 172.21.15.32:0/397641675 >> v2:172.21.15.39:3300/0 conn(0x7f66b80b6120 0x7f66b815b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1677916474 0 0) 0x7f66b815ed90 con 0x7f66b80b2240 2023-12-20T04:40:27.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66a0003530 con 0x7f66b80b2240 2023-12-20T04:40:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1681849283 0 0) 0x7f66a0003530 con 0x7f66b80b2240 2023-12-20T04:40:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 >> v2:172.21.15.39:3300/0 conn(0x7f66b80b6120 msgr2=0x7f66b815b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:27.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 --2- 172.21.15.32:0/397641675 >> v2:172.21.15.39:3300/0 conn(0x7f66b80b6120 0x7f66b815b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 >> v1:172.21.15.39:6789/0 conn(0x7f66b8158650 legacy=0x7f66b815e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b815f350 con 0x7f66b80b2240 2023-12-20T04:40:27.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f66b0001a00 con 0x7f66b80b2240 2023-12-20T04:40:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 >> v1:172.21.15.32:6789/0 conn(0x7f66b80b2240 legacy=0x7f66b815c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 --2- 172.21.15.32:0/397641675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 0x7f66a0003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 --2- 172.21.15.32:0/397641675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66a00047d0 0x7f66a0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 --2- 172.21.15.32:0/397641675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66a0005190 0x7f66a000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a000c0b0 con 0x7f66a0003980 2023-12-20T04:40:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66a000c220 con 0x7f66a00047d0 2023-12-20T04:40:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66a000c3e0 con 0x7f66a0005190 2023-12-20T04:40:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b77fe640 1 --2- 172.21.15.32:0/397641675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 0x7f66a0003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.638+0000 7f66b6ffd640 1 --2- 172.21.15.32:0/397641675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66a00047d0 0x7f66a0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66b7fff640 1 --2- 172.21.15.32:0/397641675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66a0005190 0x7f66a000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66b77fe640 1 -- 172.21.15.32:0/397641675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66a0005190 msgr2=0x7f66a000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:27.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66b77fe640 1 --2- 172.21.15.32:0/397641675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66a0005190 0x7f66a000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66b77fe640 1 -- 172.21.15.32:0/397641675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66a00047d0 msgr2=0x7f66a0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:27.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66b77fe640 1 --2- 172.21.15.32:0/397641675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66a00047d0 0x7f66a0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66b77fe640 1 -- 172.21.15.32:0/397641675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66a0003530 con 0x7f66a0003980 2023-12-20T04:40:27.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66b7fff640 1 -- 172.21.15.32:0/397641675 reap_dead start 2023-12-20T04:40:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66b77fe640 1 --2- 172.21.15.32:0/397641675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 0x7f66a0003d60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f66a8001860 tx=0x7f66a801c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32074b93b1c909c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a80ab600 con 0x7f66a0003980 2023-12-20T04:40:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a80a7050 con 0x7f66a0003980 2023-12-20T04:40:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66b67fc640 1 -- 172.21.15.32:0/397641675 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a80aba20 con 0x7f66a0003980 2023-12-20T04:40:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66bcbcb640 1 -- 172.21.15.32:0/397641675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 msgr2=0x7f66a0003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.642+0000 7f66bcbcb640 1 --2- 172.21.15.32:0/397641675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 0x7f66a0003d60 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f66a8001860 tx=0x7f66a801c4e0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 -- 172.21.15.32:0/397641675 shutdown_connections 2023-12-20T04:40:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 --2- 172.21.15.32:0/397641675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 0x7f66a0003d60 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 -- 172.21.15.32:0/397641675 >> 172.21.15.32:0/397641675 conn(0x7f66b80cd240 msgr2=0x7f66b80cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 -- 172.21.15.32:0/397641675 shutdown_connections 2023-12-20T04:40:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 -- 172.21.15.32:0/397641675 wait complete. 2023-12-20T04:40:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 Processor -- start 2023-12-20T04:40:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 -- start start 2023-12-20T04:40:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 0x7f66b80c61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b8155fc0 0x7f66b80c6710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66b806b840 0x7f66b80c6c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b806bd50 con 0x7f66a0003980 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b806c030 con 0x7f66b806b840 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66bcbcb640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b806c310 con 0x7f66b8155fc0 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66b77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 0x7f66b80c61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66b77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 0x7f66b80c61d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44354/0 (socket says 172.21.15.32:44354) 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66b77fe640 1 -- 172.21.15.32:0/3055688328 learned_addr learned my addr 172.21.15.32:0/3055688328 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66b6ffd640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b8155fc0 0x7f66b80c6710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66b7fff640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66b806b840 0x7f66b80c6c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66b77fe640 1 -- 172.21.15.32:0/3055688328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b8155fc0 msgr2=0x7f66b80c6710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66b77fe640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b8155fc0 0x7f66b80c6710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66b77fe640 1 -- 172.21.15.32:0/3055688328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66b806b840 msgr2=0x7f66b80c6c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66b77fe640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66b806b840 0x7f66b80c6c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66b77fe640 1 -- 172.21.15.32:0/3055688328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b806c5f0 con 0x7f66a0003980 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.646+0000 7f66b77fe640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 0x7f66b80c61d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f66a801c310 tx=0x7f66a0005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.650+0000 7f669ffff640 1 -- 172.21.15.32:0/3055688328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a8007240 con 0x7f66a0003980 2023-12-20T04:40:27.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.650+0000 7f669ffff640 1 -- 172.21.15.32:0/3055688328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a80a4e90 con 0x7f66a0003980 2023-12-20T04:40:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.650+0000 7f669ffff640 1 -- 172.21.15.32:0/3055688328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a80075c0 con 0x7f66a0003980 2023-12-20T04:40:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.650+0000 7f66bcbcb640 1 -- 172.21.15.32:0/3055688328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b80c8d60 con 0x7f66a0003980 2023-12-20T04:40:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.650+0000 7f66bcbcb640 1 -- 172.21.15.32:0/3055688328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b80c4800 con 0x7f66a0003980 2023-12-20T04:40:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.650+0000 7f66bcbcb640 1 -- 172.21.15.32:0/3055688328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b806c780 con 0x7f66a0003980 2023-12-20T04:40:27.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.650+0000 7f669ffff640 1 -- 172.21.15.32:0/3055688328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f66a80b2050 con 0x7f66a0003980 2023-12-20T04:40:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.650+0000 7f669ffff640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f66a4065c90 0x7f66a4068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.650+0000 7f669ffff640 1 -- 172.21.15.32:0/3055688328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f66a811d150 con 0x7f66a0003980 2023-12-20T04:40:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.650+0000 7f66b6ffd640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f66a4065c90 0x7f66a4068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:27.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.650+0000 7f66b6ffd640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f66a4065c90 0x7f66a4068150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66b80d00a0 tx=0x7f66ac009190 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:27.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.654+0000 7f669ffff640 1 -- 172.21.15.32:0/3055688328 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f66a80ea740 con 0x7f66a0003980 2023-12-20T04:40:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.834+0000 7f66bcbcb640 1 -- 172.21.15.32:0/3055688328 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/dir9", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f66b806c1a0 con 0x7f66a4065c90 2023-12-20T04:40:27.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.842+0000 7f669ffff640 1 -- 172.21.15.32:0/3055688328 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+345 (secure 0 0 0) 0x7f66b806c1a0 con 0x7f66a4065c90 2023-12-20T04:40:27.841 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:27.842 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:53", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.842+0000 7f66bcbcb640 1 -- 172.21.15.32:0/3055688328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f66a4065c90 msgr2=0x7f66a4068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.842+0000 7f66bcbcb640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f66a4065c90 0x7f66a4068150 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f66b80d00a0 tx=0x7f66ac009190 comp rx=0 tx=0).stop 2023-12-20T04:40:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.842+0000 7f66bcbcb640 1 -- 172.21.15.32:0/3055688328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 msgr2=0x7f66b80c61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.842+0000 7f66bcbcb640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 0x7f66b80c61d0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7f66a801c310 tx=0x7f66a0005800 comp rx=0 tx=0).stop 2023-12-20T04:40:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.846+0000 7f66bcbcb640 1 -- 172.21.15.32:0/3055688328 shutdown_connections 2023-12-20T04:40:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.846+0000 7f66bcbcb640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f66a4065c90 0x7f66a4068150 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.846+0000 7f66bcbcb640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66b806b840 0x7f66b80c6c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.846+0000 7f66bcbcb640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b8155fc0 0x7f66b80c6710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.846+0000 7f66bcbcb640 1 --2- 172.21.15.32:0/3055688328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66a0003980 0x7f66b80c61d0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.846+0000 7f66bcbcb640 1 -- 172.21.15.32:0/3055688328 >> 172.21.15.32:0/3055688328 conn(0x7f66b80cd240 msgr2=0x7f66b80cbdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.846+0000 7f66bcbcb640 1 -- 172.21.15.32:0/3055688328 shutdown_connections 2023-12-20T04:40:27.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:27.846+0000 7f66bcbcb640 1 -- 172.21.15.32:0/3055688328 wait complete. 2023-12-20T04:40:27.862 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/dir9", "rel_path": "/snap_test_dir1/dir9", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:29:53", "first": "2023-12-20T04:30:00", "last": "2023-12-20T04:40:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": false}] 2023-12-20T04:40:27.863 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/dir9', 'rel_path': '/snap_test_dir1/dir9', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T04:29:53', 'first': '2023-12-20T04:30:00', 'last': '2023-12-20T04:40:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': False} 2023-12-20T04:40:27.863 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-20T04:40:27.863 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-20T04:40:27.863 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir0 1M 2023-12-20T04:40:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e7ce8c640 1 Processor -- start 2023-12-20T04:40:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e7ce8c640 1 -- start start 2023-12-20T04:40:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e7ce8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e7815cef0 0x7f0e78161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e7ce8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e78071760 0x7f0e780ca270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e7ce8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e780cdbd0 con 0x7f0e780ca7b0 2023-12-20T04:40:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e7ce8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e780cd7b0 con 0x7f0e78071760 2023-12-20T04:40:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e7ce8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e78071be0 con 0x7f0e7815cef0 2023-12-20T04:40:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e75d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e78071760 0x7f0e780ca270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e75d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e78071760 0x7f0e780ca270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44356/0 (socket says 172.21.15.32:44356) 2023-12-20T04:40:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e75d9b640 1 -- 172.21.15.32:0/369493595 learned_addr learned my addr 172.21.15.32:0/369493595 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:28.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e76d9d640 1 --1- 172.21.15.32:0/369493595 >> v1:172.21.15.32:6789/0 conn(0x7f0e780ca7b0 0x7f0e780cde70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59044/0 (socket says 172.21.15.32:59044) 2023-12-20T04:40:28.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e7659c640 1 --2- 172.21.15.32:0/369493595 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e7815cef0 0x7f0e78161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:28.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e75d9b640 1 -- 172.21.15.32:0/369493595 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e7815cef0 msgr2=0x7f0e78161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:28.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e75d9b640 1 --2- 172.21.15.32:0/369493595 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e7815cef0 0x7f0e78161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:28.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e75d9b640 1 -- 172.21.15.32:0/369493595 >> v1:172.21.15.32:6789/0 conn(0x7f0e780ca7b0 legacy=0x7f0e780cde70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:28.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e75d9b640 1 -- 172.21.15.32:0/369493595 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e780cd460 con 0x7f0e78071760 2023-12-20T04:40:28.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.014+0000 7f0e7559a640 1 -- 172.21.15.32:0/369493595 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1495673263 0 0) 0x7f0e780cdbd0 con 0x7f0e780ca7b0 2023-12-20T04:40:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e75d9b640 1 --2- 172.21.15.32:0/369493595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e78071760 0x7f0e780ca270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009870 tx=0x7f0e6c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ac967af2e7df4c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e7559a640 1 -- 172.21.15.32:0/369493595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0ab680 con 0x7f0e78071760 2023-12-20T04:40:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e7559a640 1 -- 172.21.15.32:0/369493595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c0ab7e0 con 0x7f0e78071760 2023-12-20T04:40:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e7559a640 1 -- 172.21.15.32:0/369493595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0abb70 con 0x7f0e78071760 2023-12-20T04:40:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/369493595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e78071760 msgr2=0x7f0e780ca270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e7ce8c640 1 --2- 172.21.15.32:0/369493595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e78071760 0x7f0e780ca270 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c009870 tx=0x7f0e6c01c480 comp rx=0 tx=0).stop 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/369493595 shutdown_connections 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e7ce8c640 1 --2- 172.21.15.32:0/369493595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e78071760 0x7f0e780ca270 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e7ce8c640 1 --2- 172.21.15.32:0/369493595 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e7815cef0 0x7f0e78161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/369493595 >> 172.21.15.32:0/369493595 conn(0x7f0e7806b230 msgr2=0x7f0e7806b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/369493595 shutdown_connections 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.018+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/369493595 wait complete. 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e7ce8c640 1 Processor -- start 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e7ce8c640 1 -- start start 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e7ce8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e78071760 0x7f0e7814a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e7ce8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e780ca7b0 0x7f0e7815b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e7ce8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7815cef0 0x7f0e7815bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e7ce8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7814b880 con 0x7f0e780ca7b0 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e7ce8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7814a620 con 0x7f0e7815cef0 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e7ce8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e7815af90 con 0x7f0e78071760 2023-12-20T04:40:28.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e75d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e780ca7b0 0x7f0e7815b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e75d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e780ca7b0 0x7f0e7815b5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44370/0 (socket says 172.21.15.32:44370) 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e75d9b640 1 -- 172.21.15.32:0/2835755611 learned_addr learned my addr 172.21.15.32:0/2835755611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e75d9b640 1 -- 172.21.15.32:0/2835755611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e78071760 msgr2=0x7f0e7814a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e75d9b640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e78071760 0x7f0e7814a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e75d9b640 1 -- 172.21.15.32:0/2835755611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7815cef0 msgr2=0x7f0e7815bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e75d9b640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7815cef0 0x7f0e7815bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e75d9b640 1 -- 172.21.15.32:0/2835755611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e6c0094d0 con 0x7f0e780ca7b0 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e75d9b640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e780ca7b0 0x7f0e7815b5d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0ad040 tx=0x7f0e6c01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e66ffd640 1 -- 172.21.15.32:0/2835755611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0b4800 con 0x7f0e780ca7b0 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e66ffd640 1 -- 172.21.15.32:0/2835755611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e6c0b4980 con 0x7f0e780ca7b0 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/2835755611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e7815b270 con 0x7f0e780ca7b0 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e66ffd640 1 -- 172.21.15.32:0/2835755611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c0b4d30 con 0x7f0e780ca7b0 2023-12-20T04:40:28.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/2835755611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e7814b4a0 con 0x7f0e780ca7b0 2023-12-20T04:40:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.022+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/2835755611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e7804e000 con 0x7f0e780ca7b0 2023-12-20T04:40:28.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.026+0000 7f0e66ffd640 1 -- 172.21.15.32:0/2835755611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f0e6c0c7440 con 0x7f0e780ca7b0 2023-12-20T04:40:28.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.026+0000 7f0e66ffd640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e58065c60 0x7f0e58068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:28.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.026+0000 7f0e66ffd640 1 -- 172.21.15.32:0/2835755611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0e6c11e610 con 0x7f0e780ca7b0 2023-12-20T04:40:28.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.026+0000 7f0e7659c640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e58065c60 0x7f0e58068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:28.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.026+0000 7f0e7659c640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e58065c60 0x7f0e58068120 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e68006aa0 tx=0x7f0e68006a30 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:28.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.030+0000 7f0e66ffd640 1 -- 172.21.15.32:0/2835755611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0e6c0ebc50 con 0x7f0e780ca7b0 2023-12-20T04:40:28.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.206+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/2835755611 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir0", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0e780b15a0 con 0x7f0e58065c60 2023-12-20T04:40:28.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.282+0000 7f0e66ffd640 1 -- 172.21.15.32:0/2835755611 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f0e68009640 con 0x7f0e58065c60 2023-12-20T04:40:28.281 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1/dir0 2023-12-20T04:40:28.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.282+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/2835755611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e58065c60 msgr2=0x7f0e58068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.282+0000 7f0e7ce8c640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e58065c60 0x7f0e58068120 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f0e68006aa0 tx=0x7f0e68006a30 comp rx=0 tx=0).stop 2023-12-20T04:40:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.282+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/2835755611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e780ca7b0 msgr2=0x7f0e7815b5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.282+0000 7f0e7ce8c640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e780ca7b0 0x7f0e7815b5d0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c0ad040 tx=0x7f0e6c01acc0 comp rx=0 tx=0).stop 2023-12-20T04:40:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.282+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/2835755611 shutdown_connections 2023-12-20T04:40:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.282+0000 7f0e7ce8c640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e58065c60 0x7f0e58068120 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:28.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.282+0000 7f0e7ce8c640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e7815cef0 0x7f0e7815bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:28.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.282+0000 7f0e7ce8c640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e780ca7b0 0x7f0e7815b5d0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:28.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.282+0000 7f0e7ce8c640 1 --2- 172.21.15.32:0/2835755611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e78071760 0x7f0e7814a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:28.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.282+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/2835755611 >> 172.21.15.32:0/2835755611 conn(0x7f0e7806b230 msgr2=0x7f0e780afa90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:28.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.286+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/2835755611 shutdown_connections 2023-12-20T04:40:28.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:28.286+0000 7f0e7ce8c640 1 -- 172.21.15.32:0/2835755611 wait complete. 2023-12-20T04:40:28.302 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir0 2023-12-20T04:40:28.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.303 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir0/.snap) 2023-12-20T04:40:28.312 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:28.313 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:28.314 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:28.314 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:28.314 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:28.314 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:28.315 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:28.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.316 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_30_00_UTC) 2023-12-20T04:40:28.375 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:28.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.375 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_31_00_UTC) 2023-12-20T04:40:28.437 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:28.437 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.438 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_32_00_UTC) 2023-12-20T04:40:28.498 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:28.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.498 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_33_00_UTC) 2023-12-20T04:40:28.554 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:28.554 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.554 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_34_00_UTC) 2023-12-20T04:40:28.610 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:28.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.610 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_35_00_UTC) 2023-12-20T04:40:28.665 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:28.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.665 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_36_00_UTC) 2023-12-20T04:40:28.722 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:28.722 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.722 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_37_00_UTC) 2023-12-20T04:40:28.777 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:28.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.778 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_38_00_UTC) 2023-12-20T04:40:28.838 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:28.838 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.838 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_39_00_UTC) 2023-12-20T04:40:28.894 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:28.894 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.894 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0/.snap/scheduled-2023-12-20-04_40_00_UTC) 2023-12-20T04:40:28.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:28.954 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir0) 2023-12-20T04:40:29.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir1 1M 2023-12-20T04:40:29.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.202+0000 7f0c53a95640 1 Processor -- start 2023-12-20T04:40:29.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.202+0000 7f0c53a95640 1 -- start start 2023-12-20T04:40:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.202+0000 7f0c53a95640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0c4c0b0760 0x7f0c4c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.202+0000 7f0c53a95640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4c0b1100 con 0x7f0c4c157480 2023-12-20T04:40:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.202+0000 7f0c53a95640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c4c0cde20 con 0x7f0c4c0cc4b0 2023-12-20T04:40:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.202+0000 7f0c53a95640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c4c0c9f50 con 0x7f0c4c0b0760 2023-12-20T04:40:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.202+0000 7f0c51831640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c4c157480 0x7f0c4c0c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57018/0 (socket says 172.21.15.32:57018) 2023-12-20T04:40:29.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.202+0000 7f0c51831640 1 -- 172.21.15.32:0/2147323305 learned_addr learned my addr 172.21.15.32:0/2147323305 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.202+0000 7f0c51030640 1 --2- 172.21.15.32:0/2147323305 >> v2:172.21.15.39:3300/0 conn(0x7f0c4c0b0760 0x7f0c4c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:29.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.202+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1842865985 0 0) 0x7f0c4c0b1100 con 0x7f0c4c157480 2023-12-20T04:40:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.202+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0c38003530 con 0x7f0c4c157480 2023-12-20T04:40:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 500154701 0 0) 0x7f0c38003530 con 0x7f0c4c157480 2023-12-20T04:40:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 >> v2:172.21.15.39:3300/0 conn(0x7f0c4c0b0760 msgr2=0x7f0c4c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 --2- 172.21.15.32:0/2147323305 >> v2:172.21.15.39:3300/0 conn(0x7f0c4c0b0760 0x7f0c4c0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 >> v1:172.21.15.39:6789/0 conn(0x7f0c4c0cc4b0 legacy=0x7f0c4c0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c0c8e30 con 0x7f0c4c157480 2023-12-20T04:40:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f0c48001680 con 0x7f0c4c157480 2023-12-20T04:40:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 >> v1:172.21.15.32:6789/0 conn(0x7f0c4c157480 legacy=0x7f0c4c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:29.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c38003980 0x7f0c38003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c380047d0 0x7f0c38004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c38005190 0x7f0c3800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c0c9f50 con 0x7f0c380047d0 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3800e340 con 0x7f0c38003980 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c3800e500 con 0x7f0c38005190 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c51030640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c380047d0 0x7f0c38004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c51831640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c38003980 0x7f0c38003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c52032640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c38005190 0x7f0c3800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c51030640 1 -- 172.21.15.32:0/2147323305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c38005190 msgr2=0x7f0c3800dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c51030640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c38005190 0x7f0c3800dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c51030640 1 -- 172.21.15.32:0/2147323305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c38003980 msgr2=0x7f0c38003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c51030640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c38003980 0x7f0c38003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c51030640 1 -- 172.21.15.32:0/2147323305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c38003530 con 0x7f0c380047d0 2023-12-20T04:40:29.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c52032640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c38005190 0x7f0c3800dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:29.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c52032640 1 -- 172.21.15.32:0/2147323305 reap_dead start 2023-12-20T04:40:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c51030640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c380047d0 0x7f0c38004c50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009ae0 tx=0x7f0c3c01ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff653daf21d87b57 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0a5610 con 0x7f0c380047d0 2023-12-20T04:40:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c3c0a5770 con 0x7f0c380047d0 2023-12-20T04:40:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.206+0000 7f0c5082f640 1 -- 172.21.15.32:0/2147323305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c3c0a5b20 con 0x7f0c380047d0 2023-12-20T04:40:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 -- 172.21.15.32:0/2147323305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c380047d0 msgr2=0x7f0c38004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c380047d0 0x7f0c38004c50 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f0c3c009ae0 tx=0x7f0c3c01ca20 comp rx=0 tx=0).stop 2023-12-20T04:40:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 -- 172.21.15.32:0/2147323305 shutdown_connections 2023-12-20T04:40:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 --2- 172.21.15.32:0/2147323305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c380047d0 0x7f0c38004c50 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 -- 172.21.15.32:0/2147323305 >> 172.21.15.32:0/2147323305 conn(0x7f0c4c06a510 msgr2=0x7f0c4c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 -- 172.21.15.32:0/2147323305 shutdown_connections 2023-12-20T04:40:29.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 -- 172.21.15.32:0/2147323305 wait complete. 2023-12-20T04:40:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 Processor -- start 2023-12-20T04:40:29.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 -- start start 2023-12-20T04:40:29.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c380047d0 0x7f0c4c15b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:29.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c152530 0x7f0c4c1548b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:29.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c4c0c5a90 0x7f0c4c152c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:29.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c15c990 con 0x7f0c4c0c5a90 2023-12-20T04:40:29.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c06cad0 con 0x7f0c4c152530 2023-12-20T04:40:29.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c53a95640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c4c15b650 con 0x7f0c380047d0 2023-12-20T04:40:29.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c52032640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c4c0c5a90 0x7f0c4c152c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:29.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c51831640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c380047d0 0x7f0c4c15b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:29.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c51030640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c152530 0x7f0c4c1548b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:29.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c52032640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c4c0c5a90 0x7f0c4c152c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36856/0 (socket says 172.21.15.32:36856) 2023-12-20T04:40:29.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.210+0000 7f0c52032640 1 -- 172.21.15.32:0/1520303502 learned_addr learned my addr 172.21.15.32:0/1520303502 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:29.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c52032640 1 -- 172.21.15.32:0/1520303502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c380047d0 msgr2=0x7f0c4c15b070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:29.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c52032640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c380047d0 0x7f0c4c15b070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:29.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c52032640 1 -- 172.21.15.32:0/1520303502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c152530 msgr2=0x7f0c4c1548b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:29.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c52032640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c152530 0x7f0c4c1548b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:29.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c52032640 1 -- 172.21.15.32:0/1520303502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c4c06c480 con 0x7f0c4c0c5a90 2023-12-20T04:40:29.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c51831640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c380047d0 0x7f0c4c15b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:29.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c52032640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c4c0c5a90 0x7f0c4c152c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c06e270 tx=0x7f0c4400ead0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:29.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c427fc640 1 -- 172.21.15.32:0/1520303502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44003b90 con 0x7f0c4c0c5a90 2023-12-20T04:40:29.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c427fc640 1 -- 172.21.15.32:0/1520303502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c44003cf0 con 0x7f0c4c0c5a90 2023-12-20T04:40:29.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c427fc640 1 -- 172.21.15.32:0/1520303502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c44005310 con 0x7f0c4c0c5a90 2023-12-20T04:40:29.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c53a95640 1 -- 172.21.15.32:0/1520303502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c4c06c760 con 0x7f0c4c0c5a90 2023-12-20T04:40:29.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c53a95640 1 -- 172.21.15.32:0/1520303502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c4c154df0 con 0x7f0c4c0c5a90 2023-12-20T04:40:29.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c53a95640 1 -- 172.21.15.32:0/1520303502 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c4c04e000 con 0x7f0c4c0c5a90 2023-12-20T04:40:29.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c427fc640 1 -- 172.21.15.32:0/1520303502 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f0c44005470 con 0x7f0c4c0c5a90 2023-12-20T04:40:29.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.214+0000 7f0c427fc640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c30065c60 0x7f0c30068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:29.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.218+0000 7f0c427fc640 1 -- 172.21.15.32:0/1520303502 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f0c44082830 con 0x7f0c4c0c5a90 2023-12-20T04:40:29.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.218+0000 7f0c51831640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c30065c60 0x7f0c30068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:29.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.218+0000 7f0c51831640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c30065c60 0x7f0c30068120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c15d330 tx=0x7f0c48001180 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:29.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.222+0000 7f0c427fc640 1 -- 172.21.15.32:0/1520303502 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0c4404fe80 con 0x7f0c4c0c5a90 2023-12-20T04:40:29.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.394+0000 7f0c53a95640 1 -- 172.21.15.32:0/1520303502 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir1", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f0c4c0b0f50 con 0x7f0c30065c60 2023-12-20T04:40:29.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.474+0000 7f0c427fc640 1 -- 172.21.15.32:0/1520303502 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f0c48007610 con 0x7f0c30065c60 2023-12-20T04:40:29.475 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1/dir1 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 -- 172.21.15.32:0/1520303502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c30065c60 msgr2=0x7f0c30068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c30065c60 0x7f0c30068120 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c15d330 tx=0x7f0c48001180 comp rx=0 tx=0).stop 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 -- 172.21.15.32:0/1520303502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c4c0c5a90 msgr2=0x7f0c4c152c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c4c0c5a90 0x7f0c4c152c80 secure :-1 s=READY pgs=484 cs=0 l=1 rev1=1 crypto rx=0x7f0c4c06e270 tx=0x7f0c4400ead0 comp rx=0 tx=0).stop 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 -- 172.21.15.32:0/1520303502 shutdown_connections 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c30065c60 0x7f0c30068120 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c4c0c5a90 0x7f0c4c152c80 unknown :-1 s=CLOSED pgs=484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c4c152530 0x7f0c4c1548b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 --2- 172.21.15.32:0/1520303502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c380047d0 0x7f0c4c15b070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 -- 172.21.15.32:0/1520303502 >> 172.21.15.32:0/1520303502 conn(0x7f0c4c06a510 msgr2=0x7f0c4c156750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 -- 172.21.15.32:0/1520303502 shutdown_connections 2023-12-20T04:40:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:29.478+0000 7f0c53a95640 1 -- 172.21.15.32:0/1520303502 wait complete. 2023-12-20T04:40:29.496 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir1 2023-12-20T04:40:29.496 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:29.496 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir1/.snap) 2023-12-20T04:40:29.506 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:29.506 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:29.506 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:29.506 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:29.506 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:29.506 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:29.506 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:29.506 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:29.506 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:29.507 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:29.507 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:29.507 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:29.508 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:29.508 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_30_00_UTC) 2023-12-20T04:40:29.574 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:29.574 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:29.574 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_31_00_UTC) 2023-12-20T04:40:29.631 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:29.631 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:29.631 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_32_00_UTC) 2023-12-20T04:40:29.694 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:29.694 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:29.694 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_33_00_UTC) 2023-12-20T04:40:29.750 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:29.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:29.751 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_34_00_UTC) 2023-12-20T04:40:29.814 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:29.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:29.815 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_35_00_UTC) 2023-12-20T04:40:29.879 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:29.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:29.879 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_36_00_UTC) 2023-12-20T04:40:29.938 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:29.938 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:29.938 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_37_00_UTC) 2023-12-20T04:40:29.994 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:29.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:29.995 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_38_00_UTC) 2023-12-20T04:40:30.054 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:30.054 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:30.054 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_39_00_UTC) 2023-12-20T04:40:30.110 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:30.110 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:30.111 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1/.snap/scheduled-2023-12-20-04_40_00_UTC) 2023-12-20T04:40:30.170 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:30.170 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir1) 2023-12-20T04:40:30.222 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir2 1M 2023-12-20T04:40:30.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32c3bc640 1 Processor -- start 2023-12-20T04:40:30.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32c3bc640 1 -- start start 2023-12-20T04:40:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32c3bc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa324157480 0x7fa3240c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32c3bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3240cd4c0 0x7fa3240b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32c3bc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3240b1100 con 0x7fa3240b0760 2023-12-20T04:40:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32c3bc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3240cde20 con 0x7fa324157480 2023-12-20T04:40:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32c3bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240c8e30 con 0x7fa3240cd4c0 2023-12-20T04:40:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32a158640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa324157480 0x7fa3240c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32a158640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa324157480 0x7fa3240c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36860/0 (socket says 172.21.15.32:36860) 2023-12-20T04:40:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32a158640 1 -- 172.21.15.32:0/2681304669 learned_addr learned my addr 172.21.15.32:0/2681304669 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa329957640 1 --1- 172.21.15.32:0/2681304669 >> v1:172.21.15.32:6789/0 conn(0x7fa3240b0760 0x7fa32415d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57026/0 (socket says 172.21.15.32:57026) 2023-12-20T04:40:30.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32a158640 1 -- 172.21.15.32:0/2681304669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3240cd4c0 msgr2=0x7fa3240b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32a959640 1 --2- 172.21.15.32:0/2681304669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3240cd4c0 0x7fa3240b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32a158640 1 --2- 172.21.15.32:0/2681304669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3240cd4c0 0x7fa3240b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32a158640 1 -- 172.21.15.32:0/2681304669 >> v1:172.21.15.32:6789/0 conn(0x7fa3240b0760 legacy=0x7fa32415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32a158640 1 -- 172.21.15.32:0/2681304669 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3240ce070 con 0x7fa324157480 2023-12-20T04:40:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa329156640 1 -- 172.21.15.32:0/2681304669 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3716163511 0 0) 0x7fa3240b1100 con 0x7fa3240b0760 2023-12-20T04:40:30.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.410+0000 7fa32a959640 1 --2- 172.21.15.32:0/2681304669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3240cd4c0 0x7fa3240b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:30.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa32a158640 1 --2- 172.21.15.32:0/2681304669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa324157480 0x7fa3240c9840 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa3200098a0 tx=0x7fa32001c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1401670e83e31e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:30.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa329156640 1 -- 172.21.15.32:0/2681304669 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200ad680 con 0x7fa324157480 2023-12-20T04:40:30.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa329156640 1 -- 172.21.15.32:0/2681304669 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3200ad7e0 con 0x7fa324157480 2023-12-20T04:40:30.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa329156640 1 -- 172.21.15.32:0/2681304669 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200adb70 con 0x7fa324157480 2023-12-20T04:40:30.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa32c3bc640 1 -- 172.21.15.32:0/2681304669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa324157480 msgr2=0x7fa3240c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:30.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa32c3bc640 1 --2- 172.21.15.32:0/2681304669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa324157480 0x7fa3240c9840 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa3200098a0 tx=0x7fa32001c460 comp rx=0 tx=0).stop 2023-12-20T04:40:30.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa32c3bc640 1 -- 172.21.15.32:0/2681304669 shutdown_connections 2023-12-20T04:40:30.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa32c3bc640 1 --2- 172.21.15.32:0/2681304669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3240cd4c0 0x7fa3240b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:30.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa32c3bc640 1 --2- 172.21.15.32:0/2681304669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa324157480 0x7fa3240c9840 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:30.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa32c3bc640 1 -- 172.21.15.32:0/2681304669 >> 172.21.15.32:0/2681304669 conn(0x7fa32406a510 msgr2=0x7fa3240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:30.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa32c3bc640 1 -- 172.21.15.32:0/2681304669 shutdown_connections 2023-12-20T04:40:30.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.414+0000 7fa32c3bc640 1 -- 172.21.15.32:0/2681304669 wait complete. 2023-12-20T04:40:30.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32c3bc640 1 Processor -- start 2023-12-20T04:40:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32c3bc640 1 -- start start 2023-12-20T04:40:30.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32c3bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240b0760 0x7fa3240b2a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32c3bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3240cd4c0 0x7fa3240c3c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32c3bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa324157480 0x7fa3240c7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32c3bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240c58b0 con 0x7fa3240b0760 2023-12-20T04:40:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32c3bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3240c8230 con 0x7fa3240cd4c0 2023-12-20T04:40:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32c3bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa324146d10 con 0x7fa324157480 2023-12-20T04:40:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32a158640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240b0760 0x7fa3240b2a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32a158640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240b0760 0x7fa3240b2a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36874/0 (socket says 172.21.15.32:36874) 2023-12-20T04:40:30.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32a158640 1 -- 172.21.15.32:0/3890341177 learned_addr learned my addr 172.21.15.32:0/3890341177 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32a158640 1 -- 172.21.15.32:0/3890341177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa324157480 msgr2=0x7fa3240c7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32a158640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa324157480 0x7fa3240c7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32a158640 1 -- 172.21.15.32:0/3890341177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3240cd4c0 msgr2=0x7fa3240c3c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32a158640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3240cd4c0 0x7fa3240c3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32a158640 1 -- 172.21.15.32:0/3890341177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3200094d0 con 0x7fa3240b0760 2023-12-20T04:40:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32a158640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240b0760 0x7fa3240b2a20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa32001a090 tx=0x7fa32001aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa31affd640 1 -- 172.21.15.32:0/3890341177 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200b8420 con 0x7fa3240b0760 2023-12-20T04:40:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa31affd640 1 -- 172.21.15.32:0/3890341177 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3200b85a0 con 0x7fa3240b0760 2023-12-20T04:40:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32c3bc640 1 -- 172.21.15.32:0/3890341177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3240b3000 con 0x7fa3240b0760 2023-12-20T04:40:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa31affd640 1 -- 172.21.15.32:0/3890341177 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3200b8950 con 0x7fa3240b0760 2023-12-20T04:40:30.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32c3bc640 1 -- 172.21.15.32:0/3890341177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3240b35c0 con 0x7fa3240b0760 2023-12-20T04:40:30.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.418+0000 7fa32c3bc640 1 -- 172.21.15.32:0/3890341177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa32404e000 con 0x7fa3240b0760 2023-12-20T04:40:30.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.422+0000 7fa31affd640 1 -- 172.21.15.32:0/3890341177 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fa3200b4040 con 0x7fa3240b0760 2023-12-20T04:40:30.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.422+0000 7fa31affd640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa308065c60 0x7fa308068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:30.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.422+0000 7fa31affd640 1 -- 172.21.15.32:0/3890341177 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7fa32011dce0 con 0x7fa3240b0760 2023-12-20T04:40:30.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.422+0000 7fa329957640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa308065c60 0x7fa308068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:30.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.422+0000 7fa329957640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa308065c60 0x7fa308068120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3240c4260 tx=0x7fa314007990 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:30.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.426+0000 7fa31affd640 1 -- 172.21.15.32:0/3890341177 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa3200eb210 con 0x7fa3240b0760 2023-12-20T04:40:30.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.602+0000 7fa32c3bc640 1 -- 172.21.15.32:0/3890341177 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir2", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa3240c9360 con 0x7fa308065c60 2023-12-20T04:40:30.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:40:30.624 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:40:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.682+0000 7fa31affd640 1 -- 172.21.15.32:0/3890341177 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fa314001230 con 0x7fa308065c60 2023-12-20T04:40:30.683 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1/dir2 2023-12-20T04:40:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 -- 172.21.15.32:0/3890341177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa308065c60 msgr2=0x7fa308068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa308065c60 0x7fa308068120 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa3240c4260 tx=0x7fa314007990 comp rx=0 tx=0).stop 2023-12-20T04:40:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 -- 172.21.15.32:0/3890341177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240b0760 msgr2=0x7fa3240b2a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240b0760 0x7fa3240b2a20 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa32001a090 tx=0x7fa32001aca0 comp rx=0 tx=0).stop 2023-12-20T04:40:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 -- 172.21.15.32:0/3890341177 shutdown_connections 2023-12-20T04:40:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa308065c60 0x7fa308068120 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa324157480 0x7fa3240c7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3240cd4c0 0x7fa3240c3c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 --2- 172.21.15.32:0/3890341177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3240b0760 0x7fa3240b2a20 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 -- 172.21.15.32:0/3890341177 >> 172.21.15.32:0/3890341177 conn(0x7fa32406a510 msgr2=0x7fa324070130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 -- 172.21.15.32:0/3890341177 shutdown_connections 2023-12-20T04:40:30.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:30.686+0000 7fa32c3bc640 1 -- 172.21.15.32:0/3890341177 wait complete. 2023-12-20T04:40:30.702 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir2 2023-12-20T04:40:30.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:30.703 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir2/.snap) 2023-12-20T04:40:30.761 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:30.761 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:30.761 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:30.761 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:30.761 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:30.761 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:30.761 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:30.762 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:30.762 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:30.762 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:30.762 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:30.763 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:30.763 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:30.763 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_30_00_UTC) 2023-12-20T04:40:30.824 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:30.824 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:30.824 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_31_00_UTC) 2023-12-20T04:40:30.882 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:30.883 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:30.883 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_32_00_UTC) 2023-12-20T04:40:30.942 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:30.942 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:30.943 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_33_00_UTC) 2023-12-20T04:40:30.998 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:30.999 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:30.999 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_34_00_UTC) 2023-12-20T04:40:31.062 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:31.062 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:31.062 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_35_00_UTC) 2023-12-20T04:40:31.118 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:31.118 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:31.118 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_36_00_UTC) 2023-12-20T04:40:31.174 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:31.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:31.174 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_37_00_UTC) 2023-12-20T04:40:31.230 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:31.230 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:31.231 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_38_00_UTC) 2023-12-20T04:40:31.288 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:31.289 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:31.289 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_39_00_UTC) 2023-12-20T04:40:31.346 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:31.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:31.346 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2/.snap/scheduled-2023-12-20-04_40_00_UTC) 2023-12-20T04:40:31.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:31.402 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir2) 2023-12-20T04:40:31.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir3 1M 2023-12-20T04:40:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d65918640 1 Processor -- start 2023-12-20T04:40:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d65918640 1 -- start start 2023-12-20T04:40:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d65918640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d60157480 0x7f9d600c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d65918640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d600b0760 0x7f9d600b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d65918640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d600b1100 con 0x7f9d6015d660 2023-12-20T04:40:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d65918640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d600cde20 con 0x7f9d60157480 2023-12-20T04:40:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d65918640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d600c8e30 con 0x7f9d600b0760 2023-12-20T04:40:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d5effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d60157480 0x7f9d600c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d5f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9d6015d660 0x7f9d6015daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57040/0 (socket says 172.21.15.32:57040) 2023-12-20T04:40:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d5f7fe640 1 -- 172.21.15.32:0/21626354 learned_addr learned my addr 172.21.15.32:0/21626354 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d5effd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d60157480 0x7f9d600c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36878/0 (socket says 172.21.15.32:36878) 2023-12-20T04:40:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d57fff640 1 --2- 172.21.15.32:0/21626354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d600b0760 0x7f9d600b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d5effd640 1 -- 172.21.15.32:0/21626354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d600b0760 msgr2=0x7f9d600b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d5effd640 1 --2- 172.21.15.32:0/21626354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d600b0760 0x7f9d600b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d5effd640 1 -- 172.21.15.32:0/21626354 >> v1:172.21.15.32:6789/0 conn(0x7f9d6015d660 legacy=0x7f9d6015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d5effd640 1 -- 172.21.15.32:0/21626354 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d600ce070 con 0x7f9d60157480 2023-12-20T04:40:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.646+0000 7f9d5e7fc640 1 -- 172.21.15.32:0/21626354 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4028968355 0 0) 0x7f9d600b1100 con 0x7f9d6015d660 2023-12-20T04:40:31.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d5effd640 1 --2- 172.21.15.32:0/21626354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d60157480 0x7f9d600c9840 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9d500097c0 tx=0x7f9d5001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e084333a0c713533 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:31.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d5e7fc640 1 -- 172.21.15.32:0/21626354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d500ac620 con 0x7f9d60157480 2023-12-20T04:40:31.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d5e7fc640 1 -- 172.21.15.32:0/21626354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d500ac780 con 0x7f9d60157480 2023-12-20T04:40:31.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d5e7fc640 1 -- 172.21.15.32:0/21626354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d500acb10 con 0x7f9d60157480 2023-12-20T04:40:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d65918640 1 -- 172.21.15.32:0/21626354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d60157480 msgr2=0x7f9d600c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d65918640 1 --2- 172.21.15.32:0/21626354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d60157480 0x7f9d600c9840 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7f9d500097c0 tx=0x7f9d5001c3c0 comp rx=0 tx=0).stop 2023-12-20T04:40:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d65918640 1 -- 172.21.15.32:0/21626354 shutdown_connections 2023-12-20T04:40:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d65918640 1 --2- 172.21.15.32:0/21626354 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d600b0760 0x7f9d600b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d65918640 1 --2- 172.21.15.32:0/21626354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d60157480 0x7f9d600c9840 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d65918640 1 -- 172.21.15.32:0/21626354 >> 172.21.15.32:0/21626354 conn(0x7f9d6006a510 msgr2=0x7f9d600ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d65918640 1 -- 172.21.15.32:0/21626354 shutdown_connections 2023-12-20T04:40:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.650+0000 7f9d65918640 1 -- 172.21.15.32:0/21626354 wait complete. 2023-12-20T04:40:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d65918640 1 Processor -- start 2023-12-20T04:40:31.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d65918640 1 -- start start 2023-12-20T04:40:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d65918640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d600b0760 0x7f9d60152ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d65918640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d60157480 0x7f9d600b2b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:31.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d65918640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d6015d660 0x7f9d600b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d65918640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d60153530 con 0x7f9d60157480 2023-12-20T04:40:31.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d65918640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d601560f0 con 0x7f9d600b0760 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d65918640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d601563d0 con 0x7f9d6015d660 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d57fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d60157480 0x7f9d600b2b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d57fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d60157480 0x7f9d600b2b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36888/0 (socket says 172.21.15.32:36888) 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d57fff640 1 -- 172.21.15.32:0/1458730220 learned_addr learned my addr 172.21.15.32:0/1458730220 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d5effd640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d600b0760 0x7f9d60152ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d5f7fe640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d6015d660 0x7f9d600b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d57fff640 1 -- 172.21.15.32:0/1458730220 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d6015d660 msgr2=0x7f9d600b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d57fff640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d6015d660 0x7f9d600b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d57fff640 1 -- 172.21.15.32:0/1458730220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d600b0760 msgr2=0x7f9d60152ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d57fff640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d600b0760 0x7f9d60152ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d57fff640 1 -- 172.21.15.32:0/1458730220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d50009420 con 0x7f9d60157480 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d57fff640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d60157480 0x7f9d600b2b50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800e870 tx=0x7f9d4800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d5cff9640 1 -- 172.21.15.32:0/1458730220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d48009510 con 0x7f9d60157480 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d5cff9640 1 -- 172.21.15.32:0/1458730220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d4800eec0 con 0x7f9d60157480 2023-12-20T04:40:31.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d5cff9640 1 -- 172.21.15.32:0/1458730220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d4800cb40 con 0x7f9d60157480 2023-12-20T04:40:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d65918640 1 -- 172.21.15.32:0/1458730220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d601566b0 con 0x7f9d60157480 2023-12-20T04:40:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.654+0000 7f9d65918640 1 -- 172.21.15.32:0/1458730220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d60156c70 con 0x7f9d60157480 2023-12-20T04:40:31.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.658+0000 7f9d65918640 1 -- 172.21.15.32:0/1458730220 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d6004e000 con 0x7f9d60157480 2023-12-20T04:40:31.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.658+0000 7f9d5cff9640 1 -- 172.21.15.32:0/1458730220 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f9d4800ccc0 con 0x7f9d60157480 2023-12-20T04:40:31.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.658+0000 7f9d5cff9640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d40065c60 0x7f9d40068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:31.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.658+0000 7f9d5cff9640 1 -- 172.21.15.32:0/1458730220 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f9d48083620 con 0x7f9d60157480 2023-12-20T04:40:31.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.658+0000 7f9d5effd640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d40065c60 0x7f9d40068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:31.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.658+0000 7f9d5effd640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d40065c60 0x7f9d40068120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d500ae000 tx=0x7f9d500a7000 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:31.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.662+0000 7f9d5cff9640 1 -- 172.21.15.32:0/1458730220 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9d48050d20 con 0x7f9d60157480 2023-12-20T04:40:31.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.842+0000 7f9d65918640 1 -- 172.21.15.32:0/1458730220 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir3", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f9d600c9360 con 0x7f9d40065c60 2023-12-20T04:40:31.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.914+0000 7f9d5cff9640 1 -- 172.21.15.32:0/1458730220 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f9d500ac8f0 con 0x7f9d40065c60 2023-12-20T04:40:31.914 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1/dir3 2023-12-20T04:40:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 -- 172.21.15.32:0/1458730220 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d40065c60 msgr2=0x7f9d40068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d40065c60 0x7f9d40068120 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f9d500ae000 tx=0x7f9d500a7000 comp rx=0 tx=0).stop 2023-12-20T04:40:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 -- 172.21.15.32:0/1458730220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d60157480 msgr2=0x7f9d600b2b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d60157480 0x7f9d600b2b50 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7f9d4800e870 tx=0x7f9d4800ec70 comp rx=0 tx=0).stop 2023-12-20T04:40:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 -- 172.21.15.32:0/1458730220 shutdown_connections 2023-12-20T04:40:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d40065c60 0x7f9d40068120 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d6015d660 0x7f9d600b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d60157480 0x7f9d600b2b50 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 --2- 172.21.15.32:0/1458730220 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d600b0760 0x7f9d60152ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 -- 172.21.15.32:0/1458730220 >> 172.21.15.32:0/1458730220 conn(0x7f9d6006a510 msgr2=0x7f9d60163da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 -- 172.21.15.32:0/1458730220 shutdown_connections 2023-12-20T04:40:31.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:31.918+0000 7f9d65918640 1 -- 172.21.15.32:0/1458730220 wait complete. 2023-12-20T04:40:31.935 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir3 2023-12-20T04:40:31.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:31.935 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir3/.snap) 2023-12-20T04:40:31.946 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:31.946 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:31.946 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:31.946 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:31.946 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:31.946 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:31.946 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:31.947 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:31.947 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:31.947 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:31.947 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:31.948 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:31.948 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:31.948 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_30_00_UTC) 2023-12-20T04:40:32.008 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:32.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:32.008 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_31_00_UTC) 2023-12-20T04:40:32.067 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:32.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:32.067 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_32_00_UTC) 2023-12-20T04:40:32.132 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:32.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:32.133 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_33_00_UTC) 2023-12-20T04:40:32.195 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:32.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:32.195 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_34_00_UTC) 2023-12-20T04:40:32.258 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:32.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:32.258 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_35_00_UTC) 2023-12-20T04:40:32.314 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:32.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:32.315 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_36_00_UTC) 2023-12-20T04:40:32.380 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:32.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:32.380 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_37_00_UTC) 2023-12-20T04:40:32.439 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:32.439 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:32.439 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_38_00_UTC) 2023-12-20T04:40:32.499 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:32.499 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:32.499 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_39_00_UTC) 2023-12-20T04:40:32.558 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:32.559 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:32.559 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3/.snap/scheduled-2023-12-20-04_40_00_UTC) 2023-12-20T04:40:32.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:32.623 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir3) 2023-12-20T04:40:32.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir4 1M 2023-12-20T04:40:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f40558d1640 1 Processor -- start 2023-12-20T04:40:32.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f40558d1640 1 -- start start 2023-12-20T04:40:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f40558d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4050157480 0x7f40500c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f40558d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f40500b0760 0x7f40500b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f40558d1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f405015d660 0x7f405015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f40558d1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40500b1100 con 0x7f405015d660 2023-12-20T04:40:32.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f40558d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40500cde20 con 0x7f40500b0760 2023-12-20T04:40:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f40558d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40500c8e30 con 0x7f4050157480 2023-12-20T04:40:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f404f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f405015d660 0x7f405015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f404f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f405015d660 0x7f405015daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36898/0 (socket says 172.21.15.32:36898) 2023-12-20T04:40:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f404f7fe640 1 -- 172.21.15.32:0/56982071 learned_addr learned my addr 172.21.15.32:0/56982071 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f40467fc640 1 --2- 172.21.15.32:0/56982071 >> v2:172.21.15.39:3300/0 conn(0x7f40500b0760 0x7f40500b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f404effd640 1 --2- 172.21.15.32:0/56982071 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4050157480 0x7f40500c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f404f7fe640 1 -- 172.21.15.32:0/56982071 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4050157480 msgr2=0x7f40500c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f404f7fe640 1 --2- 172.21.15.32:0/56982071 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4050157480 0x7f40500c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f404f7fe640 1 -- 172.21.15.32:0/56982071 >> v2:172.21.15.39:3300/0 conn(0x7f40500b0760 msgr2=0x7f40500b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f404f7fe640 1 --2- 172.21.15.32:0/56982071 >> v2:172.21.15.39:3300/0 conn(0x7f40500b0760 0x7f40500b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:32.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f404f7fe640 1 -- 172.21.15.32:0/56982071 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40500ce070 con 0x7f405015d660 2023-12-20T04:40:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f40467fc640 1 --2- 172.21.15.32:0/56982071 >> v2:172.21.15.39:3300/0 conn(0x7f40500b0760 0x7f40500b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.878+0000 7f404effd640 1 --2- 172.21.15.32:0/56982071 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4050157480 0x7f40500c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:32.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.882+0000 7f404f7fe640 1 --2- 172.21.15.32:0/56982071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f405015d660 0x7f405015daf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f405006e030 tx=0x7f404801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4e3e384e74f16f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f404e7fc640 1 -- 172.21.15.32:0/56982071 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480ab600 con 0x7f405015d660 2023-12-20T04:40:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f404e7fc640 1 -- 172.21.15.32:0/56982071 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40480ab760 con 0x7f405015d660 2023-12-20T04:40:32.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f404e7fc640 1 -- 172.21.15.32:0/56982071 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40480abaf0 con 0x7f405015d660 2023-12-20T04:40:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 -- 172.21.15.32:0/56982071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f405015d660 msgr2=0x7f405015daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:32.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 --2- 172.21.15.32:0/56982071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f405015d660 0x7f405015daf0 secure :-1 s=READY pgs=491 cs=0 l=1 rev1=1 crypto rx=0x7f405006e030 tx=0x7f404801c440 comp rx=0 tx=0).stop 2023-12-20T04:40:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 -- 172.21.15.32:0/56982071 shutdown_connections 2023-12-20T04:40:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 --2- 172.21.15.32:0/56982071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f405015d660 0x7f405015daf0 unknown :-1 s=CLOSED pgs=491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 --2- 172.21.15.32:0/56982071 >> v2:172.21.15.39:3300/0 conn(0x7f40500b0760 0x7f40500b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 --2- 172.21.15.32:0/56982071 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4050157480 0x7f40500c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 -- 172.21.15.32:0/56982071 >> 172.21.15.32:0/56982071 conn(0x7f405006a510 msgr2=0x7f40500ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:32.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 -- 172.21.15.32:0/56982071 shutdown_connections 2023-12-20T04:40:32.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 -- 172.21.15.32:0/56982071 wait complete. 2023-12-20T04:40:32.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 Processor -- start 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 -- start start 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40500b0760 0x7f40501f4d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4050157480 0x7f40500c6fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405015d660 0x7f40500c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40500c5df0 con 0x7f4050157480 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40501549c0 con 0x7f405015d660 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.886+0000 7f40558d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40500c6020 con 0x7f40500b0760 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40467fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4050157480 0x7f40500c6fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40467fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4050157480 0x7f40500c6fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36902/0 (socket says 172.21.15.32:36902) 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40467fc640 1 -- 172.21.15.32:0/1157108751 learned_addr learned my addr 172.21.15.32:0/1157108751 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f404f7fe640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405015d660 0x7f40500c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:32.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f404effd640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40500b0760 0x7f40501f4d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40467fc640 1 -- 172.21.15.32:0/1157108751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40500b0760 msgr2=0x7f40501f4d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40467fc640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40500b0760 0x7f40501f4d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40467fc640 1 -- 172.21.15.32:0/1157108751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405015d660 msgr2=0x7f40500c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40467fc640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405015d660 0x7f40500c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40467fc640 1 -- 172.21.15.32:0/1157108751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40480094d0 con 0x7f4050157480 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f404f7fe640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405015d660 0x7f40500c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40467fc640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4050157480 0x7f40500c6fa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f403800c7f0 tx=0x7f403800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f404cff9640 1 -- 172.21.15.32:0/1157108751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038007820 con 0x7f4050157480 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f404cff9640 1 -- 172.21.15.32:0/1157108751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4038007980 con 0x7f4050157480 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f404cff9640 1 -- 172.21.15.32:0/1157108751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4038007d30 con 0x7f4050157480 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40558d1640 1 -- 172.21.15.32:0/1157108751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40500c6300 con 0x7f4050157480 2023-12-20T04:40:32.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40558d1640 1 -- 172.21.15.32:0/1157108751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4050154c10 con 0x7f4050157480 2023-12-20T04:40:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f40558d1640 1 -- 172.21.15.32:0/1157108751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f405004e000 con 0x7f4050157480 2023-12-20T04:40:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f404cff9640 1 -- 172.21.15.32:0/1157108751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f40380113f0 con 0x7f4050157480 2023-12-20T04:40:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.890+0000 7f404cff9640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4030065c60 0x7f4030068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.894+0000 7f404cff9640 1 -- 172.21.15.32:0/1157108751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(36..36 src has 1..36) v4 ==== 5967+0+0 (secure 0 0 0) 0x7f4038083b80 con 0x7f4050157480 2023-12-20T04:40:32.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.894+0000 7f404effd640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4030065c60 0x7f4030068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:32.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.894+0000 7f404effd640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4030065c60 0x7f4030068120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4040007510 tx=0x7f40400074a0 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:32.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:32.898+0000 7f404cff9640 1 -- 172.21.15.32:0/1157108751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4038050f80 con 0x7f4050157480 2023-12-20T04:40:33.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.070+0000 7f40558d1640 1 -- 172.21.15.32:0/1157108751 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir4", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f40500c9360 con 0x7f4030065c60 2023-12-20T04:40:33.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.146+0000 7f404cff9640 1 -- 172.21.15.32:0/1157108751 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f4040013070 con 0x7f4030065c60 2023-12-20T04:40:33.145 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1/dir4 2023-12-20T04:40:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 -- 172.21.15.32:0/1157108751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4030065c60 msgr2=0x7f4030068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4030065c60 0x7f4030068120 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f4040007510 tx=0x7f40400074a0 comp rx=0 tx=0).stop 2023-12-20T04:40:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 -- 172.21.15.32:0/1157108751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4050157480 msgr2=0x7f40500c6fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4050157480 0x7f40500c6fa0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7f403800c7f0 tx=0x7f403800ccb0 comp rx=0 tx=0).stop 2023-12-20T04:40:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 -- 172.21.15.32:0/1157108751 shutdown_connections 2023-12-20T04:40:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4030065c60 0x7f4030068120 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f405015d660 0x7f40500c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4050157480 0x7f40500c6fa0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 --2- 172.21.15.32:0/1157108751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40500b0760 0x7f40501f4d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 -- 172.21.15.32:0/1157108751 >> 172.21.15.32:0/1157108751 conn(0x7f405006a510 msgr2=0x7f40500c3780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 -- 172.21.15.32:0/1157108751 shutdown_connections 2023-12-20T04:40:33.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:33.150+0000 7f40558d1640 1 -- 172.21.15.32:0/1157108751 wait complete. 2023-12-20T04:40:33.166 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir4 2023-12-20T04:40:33.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.167 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir4/.snap) 2023-12-20T04:40:33.177 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:33.177 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:33.177 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:33.177 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:33.177 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:33.177 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:33.178 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:33.178 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:33.178 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:33.178 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:33.178 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:33.179 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:33.179 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.179 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_30_00_UTC) 2023-12-20T04:40:33.244 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:33.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.244 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_31_00_UTC) 2023-12-20T04:40:33.306 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:33.306 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.307 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_32_00_UTC) 2023-12-20T04:40:33.366 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:33.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.367 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_33_00_UTC) 2023-12-20T04:40:33.428 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:33.428 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.428 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_34_00_UTC) 2023-12-20T04:40:33.491 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:33.491 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.491 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_35_00_UTC) 2023-12-20T04:40:33.553 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:33.553 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.553 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_36_00_UTC) 2023-12-20T04:40:33.615 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:33.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.615 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_37_00_UTC) 2023-12-20T04:40:33.674 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:33.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.675 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_38_00_UTC) 2023-12-20T04:40:33.731 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:33.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.731 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_39_00_UTC) 2023-12-20T04:40:33.790 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:33.790 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.790 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4/.snap/scheduled-2023-12-20-04_40_00_UTC) 2023-12-20T04:40:33.846 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:33.847 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir4) 2023-12-20T04:40:33.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir5 1M 2023-12-20T04:40:34.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.170+0000 7fdaca56d640 1 Processor -- start 2023-12-20T04:40:34.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.170+0000 7fdaca56d640 1 -- start start 2023-12-20T04:40:34.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdaca56d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdac4070230 0x7fdac4070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdaca56d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac40cb0b0 con 0x7fdac40ca1a0 2023-12-20T04:40:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdaca56d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdac4070b50 con 0x7fdac4164670 2023-12-20T04:40:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdaca56d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac40ca630 con 0x7fdac4070230 2023-12-20T04:40:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdabbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdac40ca1a0 0x7fdac40cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57042/0 (socket says 172.21.15.32:57042) 2023-12-20T04:40:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdabbfff640 1 -- 172.21.15.32:0/1866683278 learned_addr learned my addr 172.21.15.32:0/1866683278 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:34.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdac8b0a640 1 --2- 172.21.15.32:0/1866683278 >> v2:172.21.15.39:3300/0 conn(0x7fdac4070230 0x7fdac4070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:34.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdac37fe640 1 -- 172.21.15.32:0/1866683278 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 113847997 0 0) 0x7fdac40cb0b0 con 0x7fdac40ca1a0 2023-12-20T04:40:34.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdac37fe640 1 -- 172.21.15.32:0/1866683278 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdab0003530 con 0x7fdac40ca1a0 2023-12-20T04:40:34.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdac8b0a640 1 -- 172.21.15.32:0/1866683278 >> v1:172.21.15.39:6789/0 conn(0x7fdac4164670 legacy=0x7fdac406da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:34.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdac8b0a640 1 -- 172.21.15.32:0/1866683278 >> v1:172.21.15.32:6789/0 conn(0x7fdac40ca1a0 legacy=0x7fdac40cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:34.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.174+0000 7fdac8b0a640 1 -- 172.21.15.32:0/1866683278 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac40ca810 con 0x7fdac4070230 2023-12-20T04:40:34.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.178+0000 7fdac8b0a640 1 --2- 172.21.15.32:0/1866683278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac4070230 0x7fdac4070610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009d90 tx=0x7fdabc01c830 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9e74976977312f13 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:34.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.178+0000 7fdac37fe640 1 -- 172.21.15.32:0/1866683278 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0a5800 con 0x7fdac4070230 2023-12-20T04:40:34.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.178+0000 7fdac37fe640 1 -- 172.21.15.32:0/1866683278 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdabc0a5960 con 0x7fdac4070230 2023-12-20T04:40:34.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.178+0000 7fdac37fe640 1 -- 172.21.15.32:0/1866683278 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdabc0a5cf0 con 0x7fdac4070230 2023-12-20T04:40:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.182+0000 7fdaca56d640 1 -- 172.21.15.32:0/1866683278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac4070230 msgr2=0x7fdac4070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.182+0000 7fdaca56d640 1 --2- 172.21.15.32:0/1866683278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac4070230 0x7fdac4070610 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7fdabc009d90 tx=0x7fdabc01c830 comp rx=0 tx=0).stop 2023-12-20T04:40:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.182+0000 7fdaca56d640 1 -- 172.21.15.32:0/1866683278 shutdown_connections 2023-12-20T04:40:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.182+0000 7fdaca56d640 1 --2- 172.21.15.32:0/1866683278 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac4070230 0x7fdac4070610 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.182+0000 7fdaca56d640 1 -- 172.21.15.32:0/1866683278 >> 172.21.15.32:0/1866683278 conn(0x7fdac406a510 msgr2=0x7fdac40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:34.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.182+0000 7fdaca56d640 1 -- 172.21.15.32:0/1866683278 shutdown_connections 2023-12-20T04:40:34.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.182+0000 7fdaca56d640 1 -- 172.21.15.32:0/1866683278 wait complete. 2023-12-20T04:40:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.182+0000 7fdaca56d640 1 Processor -- start 2023-12-20T04:40:34.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.182+0000 7fdaca56d640 1 -- start start 2023-12-20T04:40:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdaca56d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac4070230 0x7fdac4155940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdaca56d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac40ca1a0 0x7fdac40cd210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdaca56d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac4164670 0x7fdac41496a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdaca56d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac415e540 con 0x7fdac4164670 2023-12-20T04:40:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdaca56d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac4155e80 con 0x7fdac40ca1a0 2023-12-20T04:40:34.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdaca56d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac415df80 con 0x7fdac4070230 2023-12-20T04:40:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac3fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac4070230 0x7fdac4155940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac3fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac4070230 0x7fdac4155940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49318/0 (socket says 172.21.15.32:49318) 2023-12-20T04:40:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac3fff640 1 -- 172.21.15.32:0/2428366502 learned_addr learned my addr 172.21.15.32:0/2428366502 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdabbfff640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac40ca1a0 0x7fdac40cd210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:34.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac8b0a640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac4164670 0x7fdac41496a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac3fff640 1 -- 172.21.15.32:0/2428366502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac40ca1a0 msgr2=0x7fdac40cd210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac3fff640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac40ca1a0 0x7fdac40cd210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac3fff640 1 -- 172.21.15.32:0/2428366502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac4164670 msgr2=0x7fdac41496a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:34.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac3fff640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac4164670 0x7fdac41496a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac3fff640 1 -- 172.21.15.32:0/2428366502 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdabc009a70 con 0x7fdac4070230 2023-12-20T04:40:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac3fff640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac4070230 0x7fdac4155940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdab400c3f0 tx=0x7fdab400c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:34.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac17fa640 1 -- 172.21.15.32:0/2428366502 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab40052d0 con 0x7fdac4070230 2023-12-20T04:40:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac17fa640 1 -- 172.21.15.32:0/2428366502 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab4005430 con 0x7fdac4070230 2023-12-20T04:40:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdac17fa640 1 -- 172.21.15.32:0/2428366502 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab4003840 con 0x7fdac4070230 2023-12-20T04:40:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.186+0000 7fdaca56d640 1 -- 172.21.15.32:0/2428366502 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac415e260 con 0x7fdac4070230 2023-12-20T04:40:34.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.190+0000 7fdaca56d640 1 -- 172.21.15.32:0/2428366502 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac40b1810 con 0x7fdac4070230 2023-12-20T04:40:34.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.190+0000 7fdaca56d640 1 -- 172.21.15.32:0/2428366502 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac40b1390 con 0x7fdac4070230 2023-12-20T04:40:34.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.190+0000 7fdac17fa640 1 -- 172.21.15.32:0/2428366502 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fdab40039a0 con 0x7fdac4070230 2023-12-20T04:40:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.190+0000 7fdac17fa640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdaa4065c60 0x7fdaa4068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.194+0000 7fdac17fa640 1 -- 172.21.15.32:0/2428366502 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(37..37 src has 1..37) v4 ==== 7399+0+0 (secure 0 0 0) 0x7fdab4083090 con 0x7fdac4070230 2023-12-20T04:40:34.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.194+0000 7fdabbfff640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdaa4065c60 0x7fdaa4068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:34.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.194+0000 7fdabbfff640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdaa4065c60 0x7fdaa4068120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdac415e6a0 tx=0x7fdaac007d40 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:34.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.202+0000 7fdac17fa640 1 -- 172.21.15.32:0/2428366502 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdac40b1390 con 0x7fdac4070230 2023-12-20T04:40:34.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.454+0000 7fdaca56d640 1 -- 172.21.15.32:0/2428366502 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir5", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdac4164a50 con 0x7fdaa4065c60 2023-12-20T04:40:34.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.742+0000 7fdac17fa640 1 -- 172.21.15.32:0/2428366502 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fdaac006b50 con 0x7fdaa4065c60 2023-12-20T04:40:34.743 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1/dir5 2023-12-20T04:40:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 -- 172.21.15.32:0/2428366502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdaa4065c60 msgr2=0x7fdaa4068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdaa4065c60 0x7fdaa4068120 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fdac415e6a0 tx=0x7fdaac007d40 comp rx=0 tx=0).stop 2023-12-20T04:40:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 -- 172.21.15.32:0/2428366502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac4070230 msgr2=0x7fdac4155940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac4070230 0x7fdac4155940 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fdab400c3f0 tx=0x7fdab400c8b0 comp rx=0 tx=0).stop 2023-12-20T04:40:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 -- 172.21.15.32:0/2428366502 shutdown_connections 2023-12-20T04:40:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdaa4065c60 0x7fdaa4068120 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac4164670 0x7fdac41496a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac40ca1a0 0x7fdac40cd210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 --2- 172.21.15.32:0/2428366502 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac4070230 0x7fdac4155940 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:34.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 -- 172.21.15.32:0/2428366502 >> 172.21.15.32:0/2428366502 conn(0x7fdac406a510 msgr2=0x7fdac41567c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:34.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 -- 172.21.15.32:0/2428366502 shutdown_connections 2023-12-20T04:40:34.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:34.746+0000 7fdaca56d640 1 -- 172.21.15.32:0/2428366502 wait complete. 2023-12-20T04:40:34.764 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir5 2023-12-20T04:40:34.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:34.764 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir5/.snap) 2023-12-20T04:40:34.774 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:34.775 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:34.775 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:34.775 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:34.775 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:34.775 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:34.775 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:34.775 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:34.775 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:34.775 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:34.775 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:34.776 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:34.776 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:34.776 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_30_00_UTC) 2023-12-20T04:40:34.836 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:34.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:34.836 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_31_00_UTC) 2023-12-20T04:40:34.895 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:34.895 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:34.896 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_32_00_UTC) 2023-12-20T04:40:34.978 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:34.978 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:34.979 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_33_00_UTC) 2023-12-20T04:40:35.039 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:35.039 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:35.039 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_34_00_UTC) 2023-12-20T04:40:35.099 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:35.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:35.099 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_35_00_UTC) 2023-12-20T04:40:35.158 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:35.158 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:35.159 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_36_00_UTC) 2023-12-20T04:40:35.221 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:35.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:35.221 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_37_00_UTC) 2023-12-20T04:40:35.279 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:35.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:35.279 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_38_00_UTC) 2023-12-20T04:40:35.342 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:35.342 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:35.342 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_39_00_UTC) 2023-12-20T04:40:35.397 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:35.398 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:35.398 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5/.snap/scheduled-2023-12-20-04_40_00_UTC) 2023-12-20T04:40:35.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:35.453 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir5) 2023-12-20T04:40:35.508 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir6 1M 2023-12-20T04:40:35.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e7241640 1 Processor -- start 2023-12-20T04:40:35.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e7241640 1 -- start start 2023-12-20T04:40:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e7241640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71e0157480 0x7f71e00c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e7241640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71e00b0760 0x7f71e00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e7241640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71e00b1100 con 0x7f71e015d660 2023-12-20T04:40:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e7241640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71e00cde20 con 0x7f71e0157480 2023-12-20T04:40:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e7241640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00c8e30 con 0x7f71e00b0760 2023-12-20T04:40:35.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e4fdd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71e0157480 0x7f71e00c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:35.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e4fdd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71e0157480 0x7f71e00c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36932/0 (socket says 172.21.15.32:36932) 2023-12-20T04:40:35.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e57de640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f71e015d660 0x7f71e015daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57054/0 (socket says 172.21.15.32:57054) 2023-12-20T04:40:35.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e4fdd640 1 -- 172.21.15.32:0/3772857049 learned_addr learned my addr 172.21.15.32:0/3772857049 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:35.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71dffff640 1 --2- 172.21.15.32:0/3772857049 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71e00b0760 0x7f71e00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:35.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e4fdd640 1 -- 172.21.15.32:0/3772857049 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71e00b0760 msgr2=0x7f71e00b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e4fdd640 1 --2- 172.21.15.32:0/3772857049 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71e00b0760 0x7f71e00b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e4fdd640 1 -- 172.21.15.32:0/3772857049 >> v1:172.21.15.32:6789/0 conn(0x7f71e015d660 legacy=0x7f71e015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71e4fdd640 1 -- 172.21.15.32:0/3772857049 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e00ce070 con 0x7f71e0157480 2023-12-20T04:40:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71df7fe640 1 -- 172.21.15.32:0/3772857049 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 610666169 0 0) 0x7f71e00b1100 con 0x7f71e015d660 2023-12-20T04:40:35.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.698+0000 7f71dffff640 1 --2- 172.21.15.32:0/3772857049 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71e00b0760 0x7f71e00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:35.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71e4fdd640 1 --2- 172.21.15.32:0/3772857049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e0157480 0x7f71e00c9840 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009870 tx=0x7f71d401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f88714e4f6c3dc7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71df7fe640 1 -- 172.21.15.32:0/3772857049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d40ab680 con 0x7f71e0157480 2023-12-20T04:40:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71df7fe640 1 -- 172.21.15.32:0/3772857049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d40ab7e0 con 0x7f71e0157480 2023-12-20T04:40:35.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71df7fe640 1 -- 172.21.15.32:0/3772857049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d40abb70 con 0x7f71e0157480 2023-12-20T04:40:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71e7241640 1 -- 172.21.15.32:0/3772857049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e0157480 msgr2=0x7f71e00c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71e7241640 1 --2- 172.21.15.32:0/3772857049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e0157480 0x7f71e00c9840 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f71d4009870 tx=0x7f71d401c480 comp rx=0 tx=0).stop 2023-12-20T04:40:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71e7241640 1 -- 172.21.15.32:0/3772857049 shutdown_connections 2023-12-20T04:40:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71e7241640 1 --2- 172.21.15.32:0/3772857049 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71e00b0760 0x7f71e00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71e7241640 1 --2- 172.21.15.32:0/3772857049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e0157480 0x7f71e00c9840 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71e7241640 1 -- 172.21.15.32:0/3772857049 >> 172.21.15.32:0/3772857049 conn(0x7f71e006a510 msgr2=0x7f71e00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71e7241640 1 -- 172.21.15.32:0/3772857049 shutdown_connections 2023-12-20T04:40:35.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.702+0000 7f71e7241640 1 -- 172.21.15.32:0/3772857049 wait complete. 2023-12-20T04:40:35.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e7241640 1 Processor -- start 2023-12-20T04:40:35.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e7241640 1 -- start start 2023-12-20T04:40:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e7241640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b0760 0x7f71e0148750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e7241640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71e0157480 0x7f71e006cdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e7241640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e015d660 0x7f71e006d2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e7241640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e00b21b0 con 0x7f71e015d660 2023-12-20T04:40:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e7241640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71e014cd60 con 0x7f71e00b0760 2023-12-20T04:40:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e7241640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71e0152360 con 0x7f71e0157480 2023-12-20T04:40:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e57de640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e015d660 0x7f71e006d2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e57de640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e015d660 0x7f71e006d2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36938/0 (socket says 172.21.15.32:36938) 2023-12-20T04:40:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e57de640 1 -- 172.21.15.32:0/1899709699 learned_addr learned my addr 172.21.15.32:0/1899709699 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71dffff640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71e0157480 0x7f71e006cdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:35.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e4fdd640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b0760 0x7f71e0148750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:35.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e57de640 1 -- 172.21.15.32:0/1899709699 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71e0157480 msgr2=0x7f71e006cdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e57de640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71e0157480 0x7f71e006cdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e57de640 1 -- 172.21.15.32:0/1899709699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b0760 msgr2=0x7f71e0148750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e57de640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b0760 0x7f71e0148750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e57de640 1 -- 172.21.15.32:0/1899709699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71d40094d0 con 0x7f71e015d660 2023-12-20T04:40:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e57de640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e015d660 0x7f71e006d2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f71d800c7d0 tx=0x7f71d800cc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71dd7fa640 1 -- 172.21.15.32:0/1899709699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d800a640 con 0x7f71e015d660 2023-12-20T04:40:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71dd7fa640 1 -- 172.21.15.32:0/1899709699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71d800a7a0 con 0x7f71e015d660 2023-12-20T04:40:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e7241640 1 -- 172.21.15.32:0/1899709699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71e0152080 con 0x7f71e015d660 2023-12-20T04:40:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71dd7fa640 1 -- 172.21.15.32:0/1899709699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71d800ab90 con 0x7f71e015d660 2023-12-20T04:40:35.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.706+0000 7f71e7241640 1 -- 172.21.15.32:0/1899709699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71e00b1e40 con 0x7f71e015d660 2023-12-20T04:40:35.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.710+0000 7f71e7241640 1 -- 172.21.15.32:0/1899709699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71e00b19c0 con 0x7f71e015d660 2023-12-20T04:40:35.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.710+0000 7f71dd7fa640 1 -- 172.21.15.32:0/1899709699 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f71d800acf0 con 0x7f71e015d660 2023-12-20T04:40:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.710+0000 7f71dd7fa640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71c4065c90 0x7f71c4068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.710+0000 7f71dd7fa640 1 -- 172.21.15.32:0/1899709699 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(38..38 src has 1..38) v4 ==== 6967+0+0 (secure 0 0 0) 0x7f71d8082d40 con 0x7f71e015d660 2023-12-20T04:40:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.710+0000 7f71e4fdd640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71c4065c90 0x7f71c4068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:35.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.710+0000 7f71e4fdd640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71c4065c90 0x7f71c4068150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f71d40af000 tx=0x7f71d40a7000 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:35.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.714+0000 7f71dd7fa640 1 -- 172.21.15.32:0/1899709699 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f71e00b19c0 con 0x7f71e015d660 2023-12-20T04:40:35.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.890+0000 7f71e7241640 1 -- 172.21.15.32:0/1899709699 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir6", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f71e00c9360 con 0x7f71c4065c90 2023-12-20T04:40:35.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71dd7fa640 1 -- 172.21.15.32:0/1899709699 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f71d40ab640 con 0x7f71c4065c90 2023-12-20T04:40:35.969 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1/dir6 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 -- 172.21.15.32:0/1899709699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71c4065c90 msgr2=0x7f71c4068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71c4065c90 0x7f71c4068150 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f71d40af000 tx=0x7f71d40a7000 comp rx=0 tx=0).stop 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 -- 172.21.15.32:0/1899709699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e015d660 msgr2=0x7f71e006d2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e015d660 0x7f71e006d2f0 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7f71d800c7d0 tx=0x7f71d800cc90 comp rx=0 tx=0).stop 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 -- 172.21.15.32:0/1899709699 shutdown_connections 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71c4065c90 0x7f71c4068150 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71e015d660 0x7f71e006d2f0 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71e0157480 0x7f71e006cdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 --2- 172.21.15.32:0/1899709699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71e00b0760 0x7f71e0148750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 -- 172.21.15.32:0/1899709699 >> 172.21.15.32:0/1899709699 conn(0x7f71e006a510 msgr2=0x7f71e00b0c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 -- 172.21.15.32:0/1899709699 shutdown_connections 2023-12-20T04:40:35.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:35.970+0000 7f71e7241640 1 -- 172.21.15.32:0/1899709699 wait complete. 2023-12-20T04:40:35.989 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir6 2023-12-20T04:40:35.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:35.990 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir6/.snap) 2023-12-20T04:40:36.000 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:36.000 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:36.000 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:36.000 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:36.000 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:36.000 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:36.000 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:36.001 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:36.001 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:36.001 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:36.001 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:36.002 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:36.002 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.002 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_30_00_UTC) 2023-12-20T04:40:36.069 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:36.069 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.069 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_31_00_UTC) 2023-12-20T04:40:36.125 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:36.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.126 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_32_00_UTC) 2023-12-20T04:40:36.183 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:36.183 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.183 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_33_00_UTC) 2023-12-20T04:40:36.241 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:36.242 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.242 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_34_00_UTC) 2023-12-20T04:40:36.298 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:36.298 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.298 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_35_00_UTC) 2023-12-20T04:40:36.353 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:36.354 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.354 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_36_00_UTC) 2023-12-20T04:40:36.414 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:36.414 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.414 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_37_00_UTC) 2023-12-20T04:40:36.470 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:36.470 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.470 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_38_00_UTC) 2023-12-20T04:40:36.526 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:36.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.526 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_39_00_UTC) 2023-12-20T04:40:36.581 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:36.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.581 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6/.snap/scheduled-2023-12-20-04_40_00_UTC) 2023-12-20T04:40:36.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:36.638 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir6) 2023-12-20T04:40:36.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir7 1M 2023-12-20T04:40:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2ab8f640 1 Processor -- start 2023-12-20T04:40:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2ab8f640 1 -- start start 2023-12-20T04:40:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2ab8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 0x7fac240c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2ab8f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac240cd4c0 0x7fac240b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:36.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2ab8f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac240b1100 con 0x7fac240b0760 2023-12-20T04:40:36.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2ab8f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac240cde20 con 0x7fac240cd4c0 2023-12-20T04:40:36.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2ab8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac240c8e30 con 0x7fac24157480 2023-12-20T04:40:36.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2912c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac240cd4c0 0x7fac240b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:36.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2912c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fac240cd4c0 0x7fac240b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36942/0 (socket says 172.21.15.32:36942) 2023-12-20T04:40:36.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2912c640 1 -- 172.21.15.32:0/4148934250 learned_addr learned my addr 172.21.15.32:0/4148934250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:36.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2892b640 1 --2- 172.21.15.32:0/4148934250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 0x7fac240c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:36.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2912c640 1 -- 172.21.15.32:0/4148934250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 msgr2=0x7fac240c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:36.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2912c640 1 --2- 172.21.15.32:0/4148934250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 0x7fac240c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:36.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2912c640 1 -- 172.21.15.32:0/4148934250 >> v1:172.21.15.39:6789/0 conn(0x7fac240b0760 legacy=0x7fac2415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:36.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2912c640 1 -- 172.21.15.32:0/4148934250 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac240ce070 con 0x7fac240cd4c0 2023-12-20T04:40:36.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.886+0000 7fac2892b640 1 --2- 172.21.15.32:0/4148934250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 0x7fac240c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac2912c640 1 --2- 172.21.15.32:0/4148934250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac240cd4c0 0x7fac240b0bc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009e50 tx=0x7fac1c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=831a52b55df0d957 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac237fe640 1 -- 172.21.15.32:0/4148934250 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0a5730 con 0x7fac240cd4c0 2023-12-20T04:40:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac237fe640 1 -- 172.21.15.32:0/4148934250 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac1c0a5890 con 0x7fac240cd4c0 2023-12-20T04:40:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac237fe640 1 -- 172.21.15.32:0/4148934250 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac1c0a5c20 con 0x7fac240cd4c0 2023-12-20T04:40:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac2ab8f640 1 -- 172.21.15.32:0/4148934250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac240cd4c0 msgr2=0x7fac240b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac2ab8f640 1 --2- 172.21.15.32:0/4148934250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac240cd4c0 0x7fac240b0bc0 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fac1c009e50 tx=0x7fac1c01c950 comp rx=0 tx=0).stop 2023-12-20T04:40:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac2ab8f640 1 -- 172.21.15.32:0/4148934250 shutdown_connections 2023-12-20T04:40:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac2ab8f640 1 --2- 172.21.15.32:0/4148934250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac240cd4c0 0x7fac240b0bc0 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac2ab8f640 1 --2- 172.21.15.32:0/4148934250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 0x7fac240c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac2ab8f640 1 -- 172.21.15.32:0/4148934250 >> 172.21.15.32:0/4148934250 conn(0x7fac2406a510 msgr2=0x7fac240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:36.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac2ab8f640 1 -- 172.21.15.32:0/4148934250 shutdown_connections 2023-12-20T04:40:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.890+0000 7fac2ab8f640 1 -- 172.21.15.32:0/4148934250 wait complete. 2023-12-20T04:40:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2ab8f640 1 Processor -- start 2023-12-20T04:40:36.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2ab8f640 1 -- start start 2023-12-20T04:40:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2ab8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac240b0760 0x7fac24153e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2ab8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac240cd4c0 0x7fac240c80e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2ab8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 0x7fac2414dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2ab8f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac241f60d0 con 0x7fac240cd4c0 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2ab8f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac241f5450 con 0x7fac240b0760 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2ab8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac241f5730 con 0x7fac24157480 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac23fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac240cd4c0 0x7fac240c80e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac23fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac240cd4c0 0x7fac240c80e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36944/0 (socket says 172.21.15.32:36944) 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac23fff640 1 -- 172.21.15.32:0/1438802611 learned_addr learned my addr 172.21.15.32:0/1438802611 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2892b640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac240b0760 0x7fac24153e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2912c640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 0x7fac2414dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac23fff640 1 -- 172.21.15.32:0/1438802611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 msgr2=0x7fac2414dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac23fff640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 0x7fac2414dbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac23fff640 1 -- 172.21.15.32:0/1438802611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac240b0760 msgr2=0x7fac24153e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac23fff640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac240b0760 0x7fac24153e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac23fff640 1 -- 172.21.15.32:0/1438802611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac1c009b30 con 0x7fac240cd4c0 2023-12-20T04:40:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2912c640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 0x7fac2414dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac23fff640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac240cd4c0 0x7fac240c80e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fac1000c3c0 tx=0x7fac1000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac217fa640 1 -- 172.21.15.32:0/1438802611 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac100052d0 con 0x7fac240cd4c0 2023-12-20T04:40:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac217fa640 1 -- 172.21.15.32:0/1438802611 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac10005430 con 0x7fac240cd4c0 2023-12-20T04:40:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac217fa640 1 -- 172.21.15.32:0/1438802611 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac10003840 con 0x7fac240cd4c0 2023-12-20T04:40:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2ab8f640 1 -- 172.21.15.32:0/1438802611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac241f5a10 con 0x7fac240cd4c0 2023-12-20T04:40:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.894+0000 7fac2ab8f640 1 -- 172.21.15.32:0/1438802611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac24150bd0 con 0x7fac240cd4c0 2023-12-20T04:40:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.898+0000 7fac2ab8f640 1 -- 172.21.15.32:0/1438802611 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac240c3660 con 0x7fac240cd4c0 2023-12-20T04:40:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.898+0000 7fac217fa640 1 -- 172.21.15.32:0/1438802611 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fac100039a0 con 0x7fac240cd4c0 2023-12-20T04:40:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.898+0000 7fac217fa640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac08065c90 0x7fac08068150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.898+0000 7fac217fa640 1 -- 172.21.15.32:0/1438802611 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(39..39 src has 1..39) v4 ==== 6871+0+0 (secure 0 0 0) 0x7fac10082eb0 con 0x7fac240cd4c0 2023-12-20T04:40:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.898+0000 7fac2892b640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac08065c90 0x7fac08068150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:36.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.898+0000 7fac2892b640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac08065c90 0x7fac08068150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac2414ff70 tx=0x7fac18008040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:36.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:36.902+0000 7fac217fa640 1 -- 172.21.15.32:0/1438802611 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fac240c3660 con 0x7fac240cd4c0 2023-12-20T04:40:37.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.114+0000 7fac2ab8f640 1 -- 172.21.15.32:0/1438802611 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir7", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fac240c9360 con 0x7fac08065c90 2023-12-20T04:40:37.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.226+0000 7fac217fa640 1 -- 172.21.15.32:0/1438802611 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fac1800c590 con 0x7fac08065c90 2023-12-20T04:40:37.226 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1/dir7 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 -- 172.21.15.32:0/1438802611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac08065c90 msgr2=0x7fac08068150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac08065c90 0x7fac08068150 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7fac2414ff70 tx=0x7fac18008040 comp rx=0 tx=0).stop 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 -- 172.21.15.32:0/1438802611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac240cd4c0 msgr2=0x7fac240c80e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac240cd4c0 0x7fac240c80e0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7fac1000c3c0 tx=0x7fac1000c880 comp rx=0 tx=0).stop 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 -- 172.21.15.32:0/1438802611 shutdown_connections 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac08065c90 0x7fac08068150 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac24157480 0x7fac2414dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac240cd4c0 0x7fac240c80e0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 --2- 172.21.15.32:0/1438802611 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac240b0760 0x7fac24153e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 -- 172.21.15.32:0/1438802611 >> 172.21.15.32:0/1438802611 conn(0x7fac2406a510 msgr2=0x7fac241563a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 -- 172.21.15.32:0/1438802611 shutdown_connections 2023-12-20T04:40:37.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:37.230+0000 7fac2ab8f640 1 -- 172.21.15.32:0/1438802611 wait complete. 2023-12-20T04:40:37.247 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir7 2023-12-20T04:40:37.247 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.247 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir7/.snap) 2023-12-20T04:40:37.257 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:37.258 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:37.258 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:37.258 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:37.258 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:37.258 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:37.258 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:37.258 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:37.258 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:37.258 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:37.258 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:37.259 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:37.259 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.259 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_30_00_UTC) 2023-12-20T04:40:37.319 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:37.319 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.320 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_31_00_UTC) 2023-12-20T04:40:37.380 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:37.380 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.380 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_32_00_UTC) 2023-12-20T04:40:37.443 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:37.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.444 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_33_00_UTC) 2023-12-20T04:40:37.502 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:37.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.502 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_34_00_UTC) 2023-12-20T04:40:37.558 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:37.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.558 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_35_00_UTC) 2023-12-20T04:40:37.613 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:37.613 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.614 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_36_00_UTC) 2023-12-20T04:40:37.673 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:37.673 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.673 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_37_00_UTC) 2023-12-20T04:40:37.730 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:37.730 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.730 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_38_00_UTC) 2023-12-20T04:40:37.786 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:37.786 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.786 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_39_00_UTC) 2023-12-20T04:40:37.842 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:37.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.842 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7/.snap/scheduled-2023-12-20-04_40_00_UTC) 2023-12-20T04:40:37.898 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:37.898 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir7) 2023-12-20T04:40:37.958 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir8 1M 2023-12-20T04:40:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefdd020640 1 Processor -- start 2023-12-20T04:40:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefdd020640 1 -- start start 2023-12-20T04:40:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefdd020640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fefd815abb0 0x7fefd80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefdd020640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fefd80b0cb0 0x7fefd815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefdd020640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fefd815b6a0 con 0x7fefd8070230 2023-12-20T04:40:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefdd020640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd806e160 con 0x7fefd815abb0 2023-12-20T04:40:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefdd020640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fefd815c070 con 0x7fefd80b0cb0 2023-12-20T04:40:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefd6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fefd815abb0 0x7fefd80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:38.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefd6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fefd815abb0 0x7fefd80caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36948/0 (socket says 172.21.15.32:36948) 2023-12-20T04:40:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefd6d9d640 1 -- 172.21.15.32:0/1987952118 learned_addr learned my addr 172.21.15.32:0/1987952118 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefd659c640 1 --2- 172.21.15.32:0/1987952118 >> v2:172.21.15.39:3300/0 conn(0x7fefd80b0cb0 0x7fefd815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefd6d9d640 1 -- 172.21.15.32:0/1987952118 >> v2:172.21.15.39:3300/0 conn(0x7fefd80b0cb0 msgr2=0x7fefd815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefd6d9d640 1 --2- 172.21.15.32:0/1987952118 >> v2:172.21.15.39:3300/0 conn(0x7fefd80b0cb0 0x7fefd815bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefd6d9d640 1 -- 172.21.15.32:0/1987952118 >> v1:172.21.15.39:6789/0 conn(0x7fefd8070230 legacy=0x7fefd806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefd6d9d640 1 -- 172.21.15.32:0/1987952118 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefd806d8c0 con 0x7fefd815abb0 2023-12-20T04:40:38.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.154+0000 7fefd659c640 1 --2- 172.21.15.32:0/1987952118 >> v2:172.21.15.39:3300/0 conn(0x7fefd80b0cb0 0x7fefd815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefd6d9d640 1 --2- 172.21.15.32:0/1987952118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd815abb0 0x7fefd80caaf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fefd80b4470 tx=0x7fefc801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9cb17fb81fd5c8e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefd5d9b640 1 -- 172.21.15.32:0/1987952118 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80aa660 con 0x7fefd815abb0 2023-12-20T04:40:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefd5d9b640 1 -- 172.21.15.32:0/1987952118 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefc80aa7c0 con 0x7fefd815abb0 2023-12-20T04:40:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefd5d9b640 1 -- 172.21.15.32:0/1987952118 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefc80aab50 con 0x7fefd815abb0 2023-12-20T04:40:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefdd020640 1 -- 172.21.15.32:0/1987952118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd815abb0 msgr2=0x7fefd80caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefdd020640 1 --2- 172.21.15.32:0/1987952118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd815abb0 0x7fefd80caaf0 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7fefd80b4470 tx=0x7fefc801c500 comp rx=0 tx=0).stop 2023-12-20T04:40:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefdd020640 1 -- 172.21.15.32:0/1987952118 shutdown_connections 2023-12-20T04:40:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefdd020640 1 --2- 172.21.15.32:0/1987952118 >> v2:172.21.15.39:3300/0 conn(0x7fefd80b0cb0 0x7fefd815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefdd020640 1 --2- 172.21.15.32:0/1987952118 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd815abb0 0x7fefd80caaf0 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefdd020640 1 -- 172.21.15.32:0/1987952118 >> 172.21.15.32:0/1987952118 conn(0x7fefd806a510 msgr2=0x7fefd80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefdd020640 1 -- 172.21.15.32:0/1987952118 shutdown_connections 2023-12-20T04:40:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefdd020640 1 -- 172.21.15.32:0/1987952118 wait complete. 2023-12-20T04:40:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefdd020640 1 Processor -- start 2023-12-20T04:40:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.158+0000 7fefdd020640 1 -- start start 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefdd020640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fefd8070230 0x7fefd80c47c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefdd020640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefd80b0cb0 0x7fefd814dec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefdd020640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd815abb0 0x7fefd80c3dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefdd020640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80cdcb0 con 0x7fefd815abb0 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefdd020640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fefd814f860 con 0x7fefd80b0cb0 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefdd020640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fefd80c4300 con 0x7fefd8070230 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefd759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd815abb0 0x7fefd80c3dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefd659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefd80b0cb0 0x7fefd814dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefd6d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fefd8070230 0x7fefd80c47c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefd759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd815abb0 0x7fefd80c3dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36962/0 (socket says 172.21.15.32:36962) 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefd759e640 1 -- 172.21.15.32:0/779263303 learned_addr learned my addr 172.21.15.32:0/779263303 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefd759e640 1 -- 172.21.15.32:0/779263303 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fefd8070230 msgr2=0x7fefd80c47c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefd759e640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fefd8070230 0x7fefd80c47c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefd759e640 1 -- 172.21.15.32:0/779263303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefd80b0cb0 msgr2=0x7fefd814dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefd759e640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefd80b0cb0 0x7fefd814dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefd759e640 1 -- 172.21.15.32:0/779263303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fefc80094d0 con 0x7fefd815abb0 2023-12-20T04:40:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefd759e640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd815abb0 0x7fefd80c3dc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c730 tx=0x7fefd000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefbf7fe640 1 -- 172.21.15.32:0/779263303 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd0003b80 con 0x7fefd815abb0 2023-12-20T04:40:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefbf7fe640 1 -- 172.21.15.32:0/779263303 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fefd0003ce0 con 0x7fefd815abb0 2023-12-20T04:40:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefbf7fe640 1 -- 172.21.15.32:0/779263303 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fefd000aa30 con 0x7fefd815abb0 2023-12-20T04:40:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefdd020640 1 -- 172.21.15.32:0/779263303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fefd80cdf00 con 0x7fefd815abb0 2023-12-20T04:40:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefdd020640 1 -- 172.21.15.32:0/779263303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fefd814fcf0 con 0x7fefd815abb0 2023-12-20T04:40:38.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.162+0000 7fefdd020640 1 -- 172.21.15.32:0/779263303 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fefd814f9d0 con 0x7fefd815abb0 2023-12-20T04:40:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.166+0000 7fefbf7fe640 1 -- 172.21.15.32:0/779263303 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fefd000ab90 con 0x7fefd815abb0 2023-12-20T04:40:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.166+0000 7fefbf7fe640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fefc4065c60 0x7fefc4068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.166+0000 7fefbf7fe640 1 -- 172.21.15.32:0/779263303 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(41..41 src has 1..41) v4 ==== 6711+0+0 (secure 0 0 0) 0x7fefd0083690 con 0x7fefd815abb0 2023-12-20T04:40:38.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.166+0000 7fefd6d9d640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fefc4065c60 0x7fefc4068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:38.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.166+0000 7fefd6d9d640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fefc4065c60 0x7fefc4068120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefc80ae000 tx=0x7fefc80b5040 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:38.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.170+0000 7fefbf7fe640 1 -- 172.21.15.32:0/779263303 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fefd0050970 con 0x7fefd815abb0 2023-12-20T04:40:38.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.390+0000 7fefdd020640 1 -- 172.21.15.32:0/779263303 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir8", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fefd814f650 con 0x7fefc4065c60 2023-12-20T04:40:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefbf7fe640 1 -- 172.21.15.32:0/779263303 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7fefc80aa930 con 0x7fefc4065c60 2023-12-20T04:40:38.496 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1/dir8 2023-12-20T04:40:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 -- 172.21.15.32:0/779263303 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fefc4065c60 msgr2=0x7fefc4068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fefc4065c60 0x7fefc4068120 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fefc80ae000 tx=0x7fefc80b5040 comp rx=0 tx=0).stop 2023-12-20T04:40:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 -- 172.21.15.32:0/779263303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd815abb0 msgr2=0x7fefd80c3dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd815abb0 0x7fefd80c3dc0 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fefd000c730 tx=0x7fefd000cbf0 comp rx=0 tx=0).stop 2023-12-20T04:40:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 -- 172.21.15.32:0/779263303 shutdown_connections 2023-12-20T04:40:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fefc4065c60 0x7fefc4068120 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fefd815abb0 0x7fefd80c3dc0 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fefd80b0cb0 0x7fefd814dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 --2- 172.21.15.32:0/779263303 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fefd8070230 0x7fefd80c47c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 -- 172.21.15.32:0/779263303 >> 172.21.15.32:0/779263303 conn(0x7fefd806a510 msgr2=0x7fefd80cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 -- 172.21.15.32:0/779263303 shutdown_connections 2023-12-20T04:40:38.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:38.498+0000 7fefdd020640 1 -- 172.21.15.32:0/779263303 wait complete. 2023-12-20T04:40:38.517 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir8 2023-12-20T04:40:38.517 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:38.517 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir8/.snap) 2023-12-20T04:40:38.528 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:38.528 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:38.529 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:38.529 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:38.529 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:38.529 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:38.529 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:38.529 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:38.529 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:38.529 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:38.529 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:38.529 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:38.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:38.530 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_30_00_UTC) 2023-12-20T04:40:38.585 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:38.586 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:38.586 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_31_00_UTC) 2023-12-20T04:40:38.641 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:38.642 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:38.642 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_32_00_UTC) 2023-12-20T04:40:38.698 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:38.698 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:38.698 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_33_00_UTC) 2023-12-20T04:40:38.753 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:38.753 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:38.753 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_34_00_UTC) 2023-12-20T04:40:38.813 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:38.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:38.814 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_35_00_UTC) 2023-12-20T04:40:38.870 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:38.870 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:38.871 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_36_00_UTC) 2023-12-20T04:40:38.934 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:38.934 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:38.934 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_37_00_UTC) 2023-12-20T04:40:39.008 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:39.008 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:39.008 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_38_00_UTC) 2023-12-20T04:40:39.067 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:39.067 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:39.067 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_39_00_UTC) 2023-12-20T04:40:39.127 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:39.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:39.127 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8/.snap/scheduled-2023-12-20-04_40_00_UTC) 2023-12-20T04:40:39.191 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:39.191 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir8) 2023-12-20T04:40:39.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/dir9 1M 2023-12-20T04:40:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.466+0000 7f8ac0e2a640 1 Processor -- start 2023-12-20T04:40:39.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.466+0000 7f8ac0e2a640 1 -- start start 2023-12-20T04:40:39.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.466+0000 7f8ac0e2a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8abc0cf3e0 0x7f8abc0cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:39.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.466+0000 7f8ac0e2a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8abc0d1580 0x7f8abc0d19e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:39.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.466+0000 7f8ac0e2a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8abc0cfd80 con 0x7f8abc15ccf0 2023-12-20T04:40:39.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.466+0000 7f8ac0e2a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8abc0ce2f0 con 0x7f8abc0d1580 2023-12-20T04:40:39.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.466+0000 7f8ac0e2a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0d0b70 con 0x7f8abc0cf3e0 2023-12-20T04:40:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.466+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8abc0cf3e0 0x7f8abc0cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.466+0000 7f8abbfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8abc0d1580 0x7f8abc0d19e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8abb7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8abc15ccf0 0x7f8abc0cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:32988/0 (socket says 172.21.15.32:32988) 2023-12-20T04:40:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8abbfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8abc0d1580 0x7f8abc0d19e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:44300/0 (socket says 172.21.15.32:44300) 2023-12-20T04:40:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8abb7fe640 1 -- 172.21.15.32:0/1579005732 learned_addr learned my addr 172.21.15.32:0/1579005732 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8aba7fc640 1 -- 172.21.15.32:0/1579005732 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1539262442 0 0) 0x7f8abc0cfd80 con 0x7f8abc15ccf0 2023-12-20T04:40:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8abaffd640 1 -- 172.21.15.32:0/1579005732 >> v2:172.21.15.39:3300/0 conn(0x7f8abc0d1580 msgr2=0x7f8abc0d19e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8abaffd640 1 --2- 172.21.15.32:0/1579005732 >> v2:172.21.15.39:3300/0 conn(0x7f8abc0d1580 0x7f8abc0d19e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8abaffd640 1 -- 172.21.15.32:0/1579005732 >> v1:172.21.15.39:6789/0 conn(0x7f8abc15ccf0 legacy=0x7f8abc0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:39.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8abaffd640 1 -- 172.21.15.32:0/1579005732 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8abc0d10f0 con 0x7f8abc0cf3e0 2023-12-20T04:40:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8abaffd640 1 --2- 172.21.15.32:0/1579005732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8abc0cf3e0 0x7f8abc0cf840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00097c0 tx=0x7f8ab001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=197652e6e8a0878b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8aba7fc640 1 -- 172.21.15.32:0/1579005732 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00ac620 con 0x7f8abc0cf3e0 2023-12-20T04:40:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8aba7fc640 1 -- 172.21.15.32:0/1579005732 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab00ac780 con 0x7f8abc0cf3e0 2023-12-20T04:40:39.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.470+0000 7f8aba7fc640 1 -- 172.21.15.32:0/1579005732 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00acb10 con 0x7f8abc0cf3e0 2023-12-20T04:40:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/1579005732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8abc0cf3e0 msgr2=0x7f8abc0cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 --2- 172.21.15.32:0/1579005732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8abc0cf3e0 0x7f8abc0cf840 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8ab00097c0 tx=0x7f8ab001c3c0 comp rx=0 tx=0).stop 2023-12-20T04:40:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/1579005732 shutdown_connections 2023-12-20T04:40:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 --2- 172.21.15.32:0/1579005732 >> v2:172.21.15.39:3300/0 conn(0x7f8abc0d1580 0x7f8abc0d19e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 --2- 172.21.15.32:0/1579005732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8abc0cf3e0 0x7f8abc0cf840 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:39.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/1579005732 >> 172.21.15.32:0/1579005732 conn(0x7f8abc0658e0 msgr2=0x7f8abc065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/1579005732 shutdown_connections 2023-12-20T04:40:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/1579005732 wait complete. 2023-12-20T04:40:39.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 Processor -- start 2023-12-20T04:40:39.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 -- start start 2023-12-20T04:40:39.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8abc0cf3e0 0x7f8abc0c65b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:39.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8abc0d1580 0x7f8abc0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:39.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8abc15ccf0 0x7f8abc146500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:39.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc1470d0 con 0x7f8abc0d1580 2023-12-20T04:40:39.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc0cc8b0 con 0x7f8abc15ccf0 2023-12-20T04:40:39.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8ac0e2a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8abc147370 con 0x7f8abc0cf3e0 2023-12-20T04:40:39.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8abc0d1580 0x7f8abc0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:39.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8abaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8abc0d1580 0x7f8abc0cc370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48450/0 (socket says 172.21.15.32:48450) 2023-12-20T04:40:39.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8abaffd640 1 -- 172.21.15.32:0/4222488406 learned_addr learned my addr 172.21.15.32:0/4222488406 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:39.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8abbfff640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8abc15ccf0 0x7f8abc146500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:39.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8abb7fe640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8abc0cf3e0 0x7f8abc0c65b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:39.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8abaffd640 1 -- 172.21.15.32:0/4222488406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8abc0cf3e0 msgr2=0x7f8abc0c65b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:39.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8abaffd640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8abc0cf3e0 0x7f8abc0c65b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:39.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8abaffd640 1 -- 172.21.15.32:0/4222488406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8abc15ccf0 msgr2=0x7f8abc146500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:39.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8abaffd640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8abc15ccf0 0x7f8abc146500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:39.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8abaffd640 1 -- 172.21.15.32:0/4222488406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ab0009420 con 0x7f8abc0d1580 2023-12-20T04:40:39.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.474+0000 7f8abbfff640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8abc15ccf0 0x7f8abc146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:39.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.478+0000 7f8abaffd640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8abc0d1580 0x7f8abc0cc370 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0019ff0 tx=0x7f8ab0019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:39.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.478+0000 7f8aa3fff640 1 -- 172.21.15.32:0/4222488406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00b5410 con 0x7f8abc0d1580 2023-12-20T04:40:39.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.478+0000 7f8aa3fff640 1 -- 172.21.15.32:0/4222488406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ab00b5590 con 0x7f8abc0d1580 2023-12-20T04:40:39.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.478+0000 7f8aa3fff640 1 -- 172.21.15.32:0/4222488406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ab00b5940 con 0x7f8abc0d1580 2023-12-20T04:40:39.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.478+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/4222488406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8abc147650 con 0x7f8abc0d1580 2023-12-20T04:40:39.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.478+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/4222488406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8abc147c10 con 0x7f8abc0d1580 2023-12-20T04:40:39.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.478+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/4222488406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8abc074100 con 0x7f8abc0d1580 2023-12-20T04:40:39.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.478+0000 7f8aa3fff640 1 -- 172.21.15.32:0/4222488406 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7f8ab00b3040 con 0x7f8abc0d1580 2023-12-20T04:40:39.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.478+0000 7f8aa3fff640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8aa8065c60 0x7f8aa8068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:39.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.478+0000 7f8aa3fff640 1 -- 172.21.15.32:0/4222488406 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(42..42 src has 1..42) v4 ==== 6535+0+0 (secure 0 0 0) 0x7f8ab011db50 con 0x7f8abc0d1580 2023-12-20T04:40:39.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.478+0000 7f8abb7fe640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8aa8065c60 0x7f8aa8068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:39.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.482+0000 7f8abb7fe640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8aa8065c60 0x7f8aa8068120 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000f70 tx=0x7f8aac007b60 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:39.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.482+0000 7f8aa3fff640 1 -- 172.21.15.32:0/4222488406 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8ab00eaed0 con 0x7f8abc0d1580 2023-12-20T04:40:39.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.662+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/4222488406 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/dir9", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8abc0b5c40 con 0x7f8aa8065c60 2023-12-20T04:40:39.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.734+0000 7f8aa3fff640 1 -- 172.21.15.32:0/4222488406 <== mgr.5218 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+47 (secure 0 0 0) 0x7f8aac0071f0 con 0x7f8aa8065c60 2023-12-20T04:40:39.734 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1/dir9 2023-12-20T04:40:39.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/4222488406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8aa8065c60 msgr2=0x7f8aa8068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:39.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8aa8065c60 0x7f8aa8068120 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f8aac000f70 tx=0x7f8aac007b60 comp rx=0 tx=0).stop 2023-12-20T04:40:39.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/4222488406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8abc0d1580 msgr2=0x7f8abc0cc370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:39.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8abc0d1580 0x7f8abc0cc370 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7f8ab0019ff0 tx=0x7f8ab0019160 comp rx=0 tx=0).stop 2023-12-20T04:40:39.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/4222488406 shutdown_connections 2023-12-20T04:40:39.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8aa8065c60 0x7f8aa8068120 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:39.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8abc15ccf0 0x7f8abc146500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:39.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8abc0d1580 0x7f8abc0cc370 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:39.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 --2- 172.21.15.32:0/4222488406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8abc0cf3e0 0x7f8abc0c65b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:39.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/4222488406 >> 172.21.15.32:0/4222488406 conn(0x7f8abc0658e0 msgr2=0x7f8abc0cf7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:39.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/4222488406 shutdown_connections 2023-12-20T04:40:39.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:39.738+0000 7f8ac0e2a640 1 -- 172.21.15.32:0/4222488406 wait complete. 2023-12-20T04:40:39.754 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/dir9 2023-12-20T04:40:39.755 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:39.755 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/dir9/.snap) 2023-12-20T04:40:39.766 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:39.766 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:39.766 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:39.767 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:39.767 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:39.767 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:39.767 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:39.767 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:39.767 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:39.767 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:39.767 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:39.769 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_30_00_UTC 2023-12-20T04:40:39.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:39.769 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_30_00_UTC) 2023-12-20T04:40:39.827 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_31_00_UTC 2023-12-20T04:40:39.828 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:39.828 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_31_00_UTC) 2023-12-20T04:40:39.886 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_32_00_UTC 2023-12-20T04:40:39.886 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:39.886 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_32_00_UTC) 2023-12-20T04:40:39.944 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_33_00_UTC 2023-12-20T04:40:39.945 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:39.945 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_33_00_UTC) 2023-12-20T04:40:40.029 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_34_00_UTC 2023-12-20T04:40:40.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:40.029 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_34_00_UTC) 2023-12-20T04:40:40.087 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_35_00_UTC 2023-12-20T04:40:40.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:40.087 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_35_00_UTC) 2023-12-20T04:40:40.151 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_36_00_UTC 2023-12-20T04:40:40.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:40.151 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_36_00_UTC) 2023-12-20T04:40:40.211 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_37_00_UTC 2023-12-20T04:40:40.211 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:40.211 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_37_00_UTC) 2023-12-20T04:40:40.280 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_38_00_UTC 2023-12-20T04:40:40.280 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:40.280 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_38_00_UTC) 2023-12-20T04:40:40.338 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_39_00_UTC 2023-12-20T04:40:40.339 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:40.339 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_39_00_UTC) 2023-12-20T04:40:40.402 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_40_00_UTC 2023-12-20T04:40:40.402 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:40.402 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9/.snap/scheduled-2023-12-20-04_40_00_UTC) 2023-12-20T04:40:40.462 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:40.462 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/dir9) 2023-12-20T04:40:40.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T04:40:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd4bbbc640 1 Processor -- start 2023-12-20T04:40:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd4bbbc640 1 -- start start 2023-12-20T04:40:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd4bbbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd440b0760 0x7fbd440b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:40.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd4bbbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd440cc4b0 0x7fbd440cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:40.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd4bbbc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd440b1100 con 0x7fbd44157480 2023-12-20T04:40:40.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd4bbbc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd440c9f50 con 0x7fbd440b0760 2023-12-20T04:40:40.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd4bbbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd440cde20 con 0x7fbd440cc4b0 2023-12-20T04:40:40.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd49157640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd440b0760 0x7fbd440b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:40.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd49157640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd440b0760 0x7fbd440b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48458/0 (socket says 172.21.15.32:48458) 2023-12-20T04:40:40.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd49157640 1 -- 172.21.15.32:0/2694114340 learned_addr learned my addr 172.21.15.32:0/2694114340 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:40.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd4a159640 1 --2- 172.21.15.32:0/2694114340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd440cc4b0 0x7fbd440cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:40.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd49157640 1 -- 172.21.15.32:0/2694114340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd440cc4b0 msgr2=0x7fbd440cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:40.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd49157640 1 --2- 172.21.15.32:0/2694114340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd440cc4b0 0x7fbd440cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:40.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd49157640 1 -- 172.21.15.32:0/2694114340 >> v1:172.21.15.39:6789/0 conn(0x7fbd44157480 legacy=0x7fbd440c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:40.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd49157640 1 -- 172.21.15.32:0/2694114340 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd440c8e30 con 0x7fbd440b0760 2023-12-20T04:40:40.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.702+0000 7fbd4a159640 1 --2- 172.21.15.32:0/2694114340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd440cc4b0 0x7fbd440cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd49157640 1 --2- 172.21.15.32:0/2694114340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd440b0760 0x7fbd440b0bc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbd4406e0d0 tx=0x7fbd3401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b56ffe75419875c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd48956640 1 -- 172.21.15.32:0/2694114340 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340ac620 con 0x7fbd440b0760 2023-12-20T04:40:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd48956640 1 -- 172.21.15.32:0/2694114340 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd340ac780 con 0x7fbd440b0760 2023-12-20T04:40:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd48956640 1 -- 172.21.15.32:0/2694114340 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340acb10 con 0x7fbd440b0760 2023-12-20T04:40:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/2694114340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd440b0760 msgr2=0x7fbd440b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd4bbbc640 1 --2- 172.21.15.32:0/2694114340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd440b0760 0x7fbd440b0bc0 secure :-1 s=READY pgs=502 cs=0 l=1 rev1=1 crypto rx=0x7fbd4406e0d0 tx=0x7fbd3401c400 comp rx=0 tx=0).stop 2023-12-20T04:40:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/2694114340 shutdown_connections 2023-12-20T04:40:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd4bbbc640 1 --2- 172.21.15.32:0/2694114340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd440cc4b0 0x7fbd440cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:40.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd4bbbc640 1 --2- 172.21.15.32:0/2694114340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd440b0760 0x7fbd440b0bc0 unknown :-1 s=CLOSED pgs=502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:40.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/2694114340 >> 172.21.15.32:0/2694114340 conn(0x7fbd4406a510 msgr2=0x7fbd440ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:40.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/2694114340 shutdown_connections 2023-12-20T04:40:40.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/2694114340 wait complete. 2023-12-20T04:40:40.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.706+0000 7fbd4bbbc640 1 Processor -- start 2023-12-20T04:40:40.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd4bbbc640 1 -- start start 2023-12-20T04:40:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd4bbbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd440b0760 0x7fbd4413c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd4bbbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd440cc4b0 0x7fbd4413aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd4bbbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd44157480 0x7fbd44162b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd4bbbc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4413b610 con 0x7fbd440cc4b0 2023-12-20T04:40:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd4bbbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4413b8f0 con 0x7fbd440b0760 2023-12-20T04:40:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd4bbbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4413bbd0 con 0x7fbd44157480 2023-12-20T04:40:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd49157640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd440cc4b0 0x7fbd4413aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd49157640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd440cc4b0 0x7fbd4413aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48462/0 (socket says 172.21.15.32:48462) 2023-12-20T04:40:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd49157640 1 -- 172.21.15.32:0/1324400507 learned_addr learned my addr 172.21.15.32:0/1324400507 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd49958640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd440b0760 0x7fbd4413c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd4a159640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd44157480 0x7fbd44162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd49157640 1 -- 172.21.15.32:0/1324400507 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd44157480 msgr2=0x7fbd44162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd49157640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd44157480 0x7fbd44162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd49157640 1 -- 172.21.15.32:0/1324400507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd440b0760 msgr2=0x7fbd4413c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd49157640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd440b0760 0x7fbd4413c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd49157640 1 -- 172.21.15.32:0/1324400507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd340094d0 con 0x7fbd440cc4b0 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd49157640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd440cc4b0 0x7fbd4413aca0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd340af040 tx=0x7fbd340191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd3a7fc640 1 -- 172.21.15.32:0/1324400507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340b7670 con 0x7fbd440cc4b0 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd3a7fc640 1 -- 172.21.15.32:0/1324400507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd340b77f0 con 0x7fbd440cc4b0 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd3a7fc640 1 -- 172.21.15.32:0/1324400507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd340b7ba0 con 0x7fbd440cc4b0 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/1324400507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd4413beb0 con 0x7fbd440cc4b0 2023-12-20T04:40:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/1324400507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4413b1e0 con 0x7fbd440cc4b0 2023-12-20T04:40:40.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.710+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/1324400507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4413b3d0 con 0x7fbd440cc4b0 2023-12-20T04:40:40.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.714+0000 7fbd3a7fc640 1 -- 172.21.15.32:0/1324400507 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 13) v1 ==== 85416+0+0 (secure 0 0 0) 0x7fbd340b3070 con 0x7fbd440cc4b0 2023-12-20T04:40:40.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.714+0000 7fbd3a7fc640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd28065c60 0x7fbd28068120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:40.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.714+0000 7fbd3a7fc640 1 -- 172.21.15.32:0/1324400507 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(43..43 src has 1..43) v4 ==== 6375+0+0 (secure 0 0 0) 0x7fbd3411dff0 con 0x7fbd440cc4b0 2023-12-20T04:40:40.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.714+0000 7fbd49958640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd28065c60 0x7fbd28068120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:40.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.714+0000 7fbd49958640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd28065c60 0x7fbd28068120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd40007fd0 tx=0x7fbd40007e40 comp rx=0 tx=0).ready entity=mgr.5218 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:40.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.718+0000 7fbd3a7fc640 1 -- 172.21.15.32:0/1324400507 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbd3414f0a0 con 0x7fbd440cc4b0 2023-12-20T04:40:40.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.910+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/1324400507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fbd440c9050 con 0x7fbd440cc4b0 2023-12-20T04:40:40.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.986+0000 7fbd3a7fc640 1 -- 172.21.15.32:0/1324400507 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7fbd340aa070 con 0x7fbd440cc4b0 2023-12-20T04:40:40.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.986+0000 7fbd3a7fc640 1 -- 172.21.15.32:0/1324400507 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v14) v1 ==== 93+0+0 (secure 0 0 0) 0x7fbd340e7bb0 con 0x7fbd440cc4b0 2023-12-20T04:40:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.994+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/1324400507 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd28065c60 msgr2=0x7fbd28068120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.994+0000 7fbd4bbbc640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd28065c60 0x7fbd28068120 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7fbd40007fd0 tx=0x7fbd40007e40 comp rx=0 tx=0).stop 2023-12-20T04:40:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.994+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/1324400507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd440cc4b0 msgr2=0x7fbd4413aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:40.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.994+0000 7fbd4bbbc640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd440cc4b0 0x7fbd4413aca0 secure :-1 s=READY pgs=503 cs=0 l=1 rev1=1 crypto rx=0x7fbd340af040 tx=0x7fbd340191a0 comp rx=0 tx=0).stop 2023-12-20T04:40:40.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.998+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/1324400507 shutdown_connections 2023-12-20T04:40:40.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.998+0000 7fbd4bbbc640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd28065c60 0x7fbd28068120 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:40.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.998+0000 7fbd4bbbc640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd44157480 0x7fbd44162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:40.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.998+0000 7fbd4bbbc640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd440cc4b0 0x7fbd4413aca0 unknown :-1 s=CLOSED pgs=503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:40.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.998+0000 7fbd4bbbc640 1 --2- 172.21.15.32:0/1324400507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd440b0760 0x7fbd4413c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:40.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.998+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/1324400507 >> 172.21.15.32:0/1324400507 conn(0x7fbd4406a510 msgr2=0x7fbd440c4230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:40.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.998+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/1324400507 shutdown_connections 2023-12-20T04:40:40.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:40.998+0000 7fbd4bbbc640 1 -- 172.21.15.32:0/1324400507 wait complete. 2023-12-20T04:40:41.021 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:40:41.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:40:41.130 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:40:41.177 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:41.178+0000 7f6d88262180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:40:41.185 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:41.186+0000 7fcb0e255180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:40:41.213 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:40:41.214 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:40:41.214 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:40:41.229 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:40:41.229 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:40:41.280 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:40:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f2742d640 1 Processor -- start 2023-12-20T04:40:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f2742d640 1 -- start start 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f2742d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4f201599e0 0x7f4f200aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f2742d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 0x7f4f200b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f2742d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f200b0b30 0x7f4f2015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f2742d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f200c98c0 con 0x7f4f200b0b30 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f2742d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f200c9ba0 con 0x7f4f201599e0 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f2742d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f200c9e80 con 0x7f4f200c90e0 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f259ca640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f200b0b30 0x7f4f2015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f259ca640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f200b0b30 0x7f4f2015d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48480/0 (socket says 172.21.15.32:48480) 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f259ca640 1 -- 172.21.15.32:0/3889927539 learned_addr learned my addr 172.21.15.32:0/3889927539 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f249c8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 0x7f4f200b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f251c9640 1 --2- 172.21.15.32:0/3889927539 >> v2:172.21.15.39:3300/0 conn(0x7f4f201599e0 0x7f4f200aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f259ca640 1 -- 172.21.15.32:0/3889927539 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 msgr2=0x7f4f200b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f259ca640 1 --2- 172.21.15.32:0/3889927539 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 0x7f4f200b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f259ca640 1 -- 172.21.15.32:0/3889927539 >> v2:172.21.15.39:3300/0 conn(0x7f4f201599e0 msgr2=0x7f4f200aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f259ca640 1 --2- 172.21.15.32:0/3889927539 >> v2:172.21.15.39:3300/0 conn(0x7f4f201599e0 0x7f4f200aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.470+0000 7f4f259ca640 1 -- 172.21.15.32:0/3889927539 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f200ca160 con 0x7f4f200b0b30 2023-12-20T04:40:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f259ca640 1 --2- 172.21.15.32:0/3889927539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f200b0b30 0x7f4f2015d6e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4f2006e3c0 tx=0x7f4f1801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6458bf97ebb39777 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f17fff640 1 -- 172.21.15.32:0/3889927539 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f180a5580 con 0x7f4f200b0b30 2023-12-20T04:40:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f17fff640 1 -- 172.21.15.32:0/3889927539 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f180a56e0 con 0x7f4f200b0b30 2023-12-20T04:40:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f17fff640 1 -- 172.21.15.32:0/3889927539 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f180a5a70 con 0x7f4f200b0b30 2023-12-20T04:40:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f2742d640 1 -- 172.21.15.32:0/3889927539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f200b0b30 msgr2=0x7f4f2015d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f2742d640 1 --2- 172.21.15.32:0/3889927539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f200b0b30 0x7f4f2015d6e0 secure :-1 s=READY pgs=508 cs=0 l=1 rev1=1 crypto rx=0x7f4f2006e3c0 tx=0x7f4f1801c6e0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f2742d640 1 -- 172.21.15.32:0/3889927539 shutdown_connections 2023-12-20T04:40:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f2742d640 1 --2- 172.21.15.32:0/3889927539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f200b0b30 0x7f4f2015d6e0 unknown :-1 s=CLOSED pgs=508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f2742d640 1 --2- 172.21.15.32:0/3889927539 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 0x7f4f200b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f2742d640 1 --2- 172.21.15.32:0/3889927539 >> v2:172.21.15.39:3300/0 conn(0x7f4f201599e0 0x7f4f200aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f2742d640 1 -- 172.21.15.32:0/3889927539 >> 172.21.15.32:0/3889927539 conn(0x7f4f2006a320 msgr2=0x7f4f200ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f2742d640 1 -- 172.21.15.32:0/3889927539 shutdown_connections 2023-12-20T04:40:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.474+0000 7f4f2742d640 1 -- 172.21.15.32:0/3889927539 wait complete. 2023-12-20T04:40:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f2742d640 1 Processor -- start 2023-12-20T04:40:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f2742d640 1 -- start start 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f2742d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f200b0b30 0x7f4f2014fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f2742d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 0x7f4f2015bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f2742d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f201599e0 0x7f4f20150e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f2742d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2015ba80 con 0x7f4f201599e0 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f2742d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2015ae70 con 0x7f4f200b0b30 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f2742d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f2015b150 con 0x7f4f200c90e0 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f249c8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 0x7f4f2015bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f251c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f200b0b30 0x7f4f2014fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f251c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f200b0b30 0x7f4f2014fb70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:44344/0 (socket says 172.21.15.32:44344) 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f251c9640 1 -- 172.21.15.32:0/2431360427 learned_addr learned my addr 172.21.15.32:0/2431360427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f249c8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 0x7f4f2015bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:47510/0 (socket says 172.21.15.32:47510) 2023-12-20T04:40:41.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f259ca640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f201599e0 0x7f4f20150e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f249c8640 1 -- 172.21.15.32:0/2431360427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f200b0b30 msgr2=0x7f4f2014fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f249c8640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f200b0b30 0x7f4f2014fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f249c8640 1 -- 172.21.15.32:0/2431360427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f201599e0 msgr2=0x7f4f20150e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f249c8640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f201599e0 0x7f4f20150e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f249c8640 1 -- 172.21.15.32:0/2431360427 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f18009b30 con 0x7f4f200c90e0 2023-12-20T04:40:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f249c8640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 0x7f4f2015bf10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000e870 tx=0x7f4f1000ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f15ffb640 1 -- 172.21.15.32:0/2431360427 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f100095d0 con 0x7f4f200c90e0 2023-12-20T04:40:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f15ffb640 1 -- 172.21.15.32:0/2431360427 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1000c940 con 0x7f4f200c90e0 2023-12-20T04:40:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f15ffb640 1 -- 172.21.15.32:0/2431360427 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1000ccb0 con 0x7f4f200c90e0 2023-12-20T04:40:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f2742d640 1 -- 172.21.15.32:0/2431360427 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f2015b430 con 0x7f4f200c90e0 2023-12-20T04:40:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.478+0000 7f4f2742d640 1 -- 172.21.15.32:0/2431360427 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f201500b0 con 0x7f4f200c90e0 2023-12-20T04:40:41.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.482+0000 7f4f2742d640 1 -- 172.21.15.32:0/2431360427 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f2004e000 con 0x7f4f200c90e0 2023-12-20T04:40:41.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.482+0000 7f4f15ffb640 1 -- 172.21.15.32:0/2431360427 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f4f10013470 con 0x7f4f200c90e0 2023-12-20T04:40:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.482+0000 7f4f15ffb640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f04065c10 0x7f4f040680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.482+0000 7f4f15ffb640 1 -- 172.21.15.32:0/2431360427 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6279+0+0 (secure 0 0 0) 0x7f4f100837e0 con 0x7f4f200c90e0 2023-12-20T04:40:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.482+0000 7f4f251c9640 1 -- 172.21.15.32:0/2431360427 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f04065c10 msgr2=0x7f4f040680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:40:41.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.482+0000 7f4f251c9640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f04065c10 0x7f4f040680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:40:41.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.486+0000 7f4f15ffb640 1 -- 172.21.15.32:0/2431360427 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4f10050da0 con 0x7f4f200c90e0 2023-12-20T04:40:41.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.650+0000 7f4f2742d640 1 -- 172.21.15.32:0/2431360427 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4f2015b290 con 0x7f4f200c90e0 2023-12-20T04:40:41.650 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:41.650+0000 7f6d88262180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:40:41.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.650+0000 7f4f15ffb640 1 -- 172.21.15.32:0/2431360427 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v44) v1 ==== 74+0+16924 (secure 0 0 0) 0x7f4f2015b290 con 0x7f4f200c90e0 2023-12-20T04:40:41.651 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:41.651 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":44,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:40:40.970414+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:29:28.685343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-20T04:29:29.647655+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"44","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":212,"snap_epoch":44,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2331287763":"2023-12-21T04:29:47.726855+0000","172.21.15.32:6833/38009":"2023-12-21T04:29:47.726855+0000","172.21.15.32:6832/38009":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/728956527":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/856257861":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/2782983206":"2023-12-21T04:29:47.726855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":2,"length":24},{"begin":27,"length":10},{"begin":38,"length":9},{"begin":48,"length":9},{"begin":58,"length":10},{"begin":69,"length":10},{"begin":80,"length":9},{"begin":90,"length":10},{"begin":101,"length":10},{"begin":112,"length":101}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":15,"length":1},{"begin":64,"length":1},{"begin":75,"length":1},{"begin":85,"length":1},{"begin":96,"length":1},{"begin":107,"length":1},{"begin":206,"length":7}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:40:41.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 -- 172.21.15.32:0/2431360427 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f04065c10 msgr2=0x7f4f040680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:40:41.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f04065c10 0x7f4f040680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 -- 172.21.15.32:0/2431360427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 msgr2=0x7f4f2015bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:41.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 0x7f4f2015bf10 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f4f1000e870 tx=0x7f4f1000ed30 comp rx=0 tx=0).stop 2023-12-20T04:40:41.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 -- 172.21.15.32:0/2431360427 shutdown_connections 2023-12-20T04:40:41.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f04065c10 0x7f4f040680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f201599e0 0x7f4f20150e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f200c90e0 0x7f4f2015bf10 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 --2- 172.21.15.32:0/2431360427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f200b0b30 0x7f4f2014fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 -- 172.21.15.32:0/2431360427 >> 172.21.15.32:0/2431360427 conn(0x7f4f2006a320 msgr2=0x7f4f200caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:41.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 -- 172.21.15.32:0/2431360427 shutdown_connections 2023-12-20T04:40:41.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.654+0000 7f4f2742d640 1 -- 172.21.15.32:0/2431360427 wait complete. 2023-12-20T04:40:41.661 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:41.662+0000 7fcb0e255180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:40:41.674 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:29:28.685343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:29:29.647655+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '44', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 212, 'snap_epoch': 44, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:40:41.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:40:41.807 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:41.810+0000 7f6d88262180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:40:41.822 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:41.822+0000 7fcb0e255180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:40:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.822+0000 7f91219bd640 1 Processor -- start 2023-12-20T04:40:41.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.822+0000 7f91219bd640 1 -- start start 2023-12-20T04:40:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f91219bd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f911c15ccf0 0x7f911c0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f91219bd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f911c0cd7b0 0x7f911c0cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f91219bd640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f911c0cfd80 con 0x7f911c0cf920 2023-12-20T04:40:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f91219bd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f911c0ce2f0 con 0x7f911c0cd7b0 2023-12-20T04:40:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f91219bd640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0d0b70 con 0x7f911c15ccf0 2023-12-20T04:40:41.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f911bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f911c0cd7b0 0x7f911c0cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f911bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f911c0cd7b0 0x7f911c0cf3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48494/0 (socket says 172.21.15.32:48494) 2023-12-20T04:40:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f911bfff640 1 -- 172.21.15.32:0/2761910331 learned_addr learned my addr 172.21.15.32:0/2761910331 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f91209bb640 1 --2- 172.21.15.32:0/2761910331 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f911c15ccf0 0x7f911c0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f911bfff640 1 -- 172.21.15.32:0/2761910331 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f911c15ccf0 msgr2=0x7f911c0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f911bfff640 1 --2- 172.21.15.32:0/2761910331 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f911c15ccf0 0x7f911c0cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f911bfff640 1 -- 172.21.15.32:0/2761910331 >> v1:172.21.15.39:6789/0 conn(0x7f911c0cf920 legacy=0x7f911c0b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:41.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.826+0000 7f911bfff640 1 -- 172.21.15.32:0/2761910331 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0d10f0 con 0x7f911c0cd7b0 2023-12-20T04:40:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f911bfff640 1 --2- 172.21.15.32:0/2761910331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c0cd7b0 0x7f911c0cf3e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9108009870 tx=0x7f910801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12bcf7ae1d5df72b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:41.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f911b7fe640 1 -- 172.21.15.32:0/2761910331 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91080ac600 con 0x7f911c0cd7b0 2023-12-20T04:40:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f911b7fe640 1 -- 172.21.15.32:0/2761910331 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91080a8050 con 0x7f911c0cd7b0 2023-12-20T04:40:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f911b7fe640 1 -- 172.21.15.32:0/2761910331 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91080ac980 con 0x7f911c0cd7b0 2023-12-20T04:40:41.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f91219bd640 1 -- 172.21.15.32:0/2761910331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c0cd7b0 msgr2=0x7f911c0cf3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f91219bd640 1 --2- 172.21.15.32:0/2761910331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c0cd7b0 0x7f911c0cf3e0 secure :-1 s=READY pgs=509 cs=0 l=1 rev1=1 crypto rx=0x7f9108009870 tx=0x7f910801c480 comp rx=0 tx=0).stop 2023-12-20T04:40:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f91219bd640 1 -- 172.21.15.32:0/2761910331 shutdown_connections 2023-12-20T04:40:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f91219bd640 1 --2- 172.21.15.32:0/2761910331 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c0cd7b0 0x7f911c0cf3e0 unknown :-1 s=CLOSED pgs=509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f91219bd640 1 --2- 172.21.15.32:0/2761910331 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f911c15ccf0 0x7f911c0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f91219bd640 1 -- 172.21.15.32:0/2761910331 >> 172.21.15.32:0/2761910331 conn(0x7f911c0658e0 msgr2=0x7f911c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f91219bd640 1 -- 172.21.15.32:0/2761910331 shutdown_connections 2023-12-20T04:40:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f91219bd640 1 -- 172.21.15.32:0/2761910331 wait complete. 2023-12-20T04:40:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f91219bd640 1 Processor -- start 2023-12-20T04:40:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.830+0000 7f91219bd640 1 -- start start 2023-12-20T04:40:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91219bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c0cd7b0 0x7f911c0c2e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91219bd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c0cf920 0x7f911c0cbe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91219bd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f911c15ccf0 0x7f911c147730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91219bd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0cc950 con 0x7f911c0cd7b0 2023-12-20T04:40:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91219bd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0caa40 con 0x7f911c0cf920 2023-12-20T04:40:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91219bd640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f911c0c9c80 con 0x7f911c15ccf0 2023-12-20T04:40:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91209bb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c0cd7b0 0x7f911c0c2e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:41.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91209bb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c0cd7b0 0x7f911c0c2e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48504/0 (socket says 172.21.15.32:48504) 2023-12-20T04:40:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91209bb640 1 -- 172.21.15.32:0/466835723 learned_addr learned my addr 172.21.15.32:0/466835723 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f911bfff640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c0cf920 0x7f911c0cbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91211bc640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f911c15ccf0 0x7f911c147730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91211bc640 1 -- 172.21.15.32:0/466835723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c0cf920 msgr2=0x7f911c0cbe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91211bc640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c0cf920 0x7f911c0cbe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91211bc640 1 -- 172.21.15.32:0/466835723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c0cd7b0 msgr2=0x7f911c0c2e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91211bc640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c0cd7b0 0x7f911c0c2e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91211bc640 1 -- 172.21.15.32:0/466835723 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91080094d0 con 0x7f911c15ccf0 2023-12-20T04:40:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f911bfff640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c0cf920 0x7f911c0cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91209bb640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c0cd7b0 0x7f911c0c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:41.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91211bc640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f911c15ccf0 0x7f911c147730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f911400c730 tx=0x7f911400cbf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91197fa640 1 -- 172.21.15.32:0/466835723 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9114003b80 con 0x7f911c15ccf0 2023-12-20T04:40:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91197fa640 1 -- 172.21.15.32:0/466835723 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9114003ce0 con 0x7f911c15ccf0 2023-12-20T04:40:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91197fa640 1 -- 172.21.15.32:0/466835723 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911400aa30 con 0x7f911c15ccf0 2023-12-20T04:40:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91219bd640 1 -- 172.21.15.32:0/466835723 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f911c0c9f60 con 0x7f911c15ccf0 2023-12-20T04:40:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91219bd640 1 -- 172.21.15.32:0/466835723 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f911c0ca520 con 0x7f911c15ccf0 2023-12-20T04:40:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91219bd640 1 -- 172.21.15.32:0/466835723 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f911c0ca710 con 0x7f911c15ccf0 2023-12-20T04:40:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.834+0000 7f91197fa640 1 -- 172.21.15.32:0/466835723 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f911400ab90 con 0x7f911c15ccf0 2023-12-20T04:40:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.838+0000 7f91197fa640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9100065c10 0x7f91000680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.838+0000 7f91197fa640 1 -- 172.21.15.32:0/466835723 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(44..44 src has 1..44) v4 ==== 6279+0+0 (secure 0 0 0) 0x7f91140833c0 con 0x7f911c15ccf0 2023-12-20T04:40:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.838+0000 7f91209bb640 1 -- 172.21.15.32:0/466835723 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9100065c10 msgr2=0x7f91000680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:40:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.838+0000 7f91209bb640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9100065c10 0x7f91000680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:40:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:41.842+0000 7f91197fa640 1 -- 172.21.15.32:0/466835723 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f91140508d0 con 0x7f911c15ccf0 2023-12-20T04:40:42.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.014+0000 7f91219bd640 1 -- 172.21.15.32:0/466835723 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f911c0c9dc0 con 0x7f911c15ccf0 2023-12-20T04:40:42.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91197fa640 1 -- 172.21.15.32:0/466835723 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v45) v1 ==== 93+0+10 (secure 0 0 0) 0x7f911c0c9dc0 con 0x7f911c15ccf0 2023-12-20T04:40:42.015 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:40:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 -- 172.21.15.32:0/466835723 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9100065c10 msgr2=0x7f91000680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:40:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9100065c10 0x7f91000680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 -- 172.21.15.32:0/466835723 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f911c15ccf0 msgr2=0x7f911c147730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f911c15ccf0 0x7f911c147730 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f911400c730 tx=0x7f911400cbf0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 -- 172.21.15.32:0/466835723 shutdown_connections 2023-12-20T04:40:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9100065c10 0x7f91000680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f911c15ccf0 0x7f911c147730 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f911c0cf920 0x7f911c0cbe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 --2- 172.21.15.32:0/466835723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f911c0cd7b0 0x7f911c0c2e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 -- 172.21.15.32:0/466835723 >> 172.21.15.32:0/466835723 conn(0x7f911c0658e0 msgr2=0x7f911c0b73f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 -- 172.21.15.32:0/466835723 shutdown_connections 2023-12-20T04:40:42.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.018+0000 7f91219bd640 1 -- 172.21.15.32:0/466835723 wait complete. 2023-12-20T04:40:42.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:40:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1cc17b640 1 Processor -- start 2023-12-20T04:40:42.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1cc17b640 1 -- start start 2023-12-20T04:40:42.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1cc17b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40b0760 0x7fb1c40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1cc17b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40b1100 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1cc17b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c40cde20 con 0x7fb1c4157480 2023-12-20T04:40:42.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1cc17b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40c9f50 con 0x7fb1c40b0760 2023-12-20T04:40:42.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1ca718640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb1c40cc4b0 0x7fb1c40cc940 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38780/0 (socket says 172.21.15.32:38780) 2023-12-20T04:40:42.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1ca718640 1 -- 172.21.15.32:0/1390022140 learned_addr learned my addr 172.21.15.32:0/1390022140 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:42.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1c9716640 1 --2- 172.21.15.32:0/1390022140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40b0760 0x7fb1c40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1c8f15640 1 -- 172.21.15.32:0/1390022140 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459479196 0 0) 0x7fb1c40cde20 con 0x7fb1c4157480 2023-12-20T04:40:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1c8f15640 1 -- 172.21.15.32:0/1390022140 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1b0005740 con 0x7fb1c4157480 2023-12-20T04:40:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.230+0000 7fb1c8f15640 1 -- 172.21.15.32:0/1390022140 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2437418701 0 0) 0x7fb1c40b1100 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.234+0000 7fb1c9716640 1 -- 172.21.15.32:0/1390022140 >> v1:172.21.15.39:6789/0 conn(0x7fb1c4157480 legacy=0x7fb1c40c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.234+0000 7fb1c9716640 1 -- 172.21.15.32:0/1390022140 >> v1:172.21.15.32:6789/0 conn(0x7fb1c40cc4b0 legacy=0x7fb1c40cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.234+0000 7fb1c9716640 1 -- 172.21.15.32:0/1390022140 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c40c8e30 con 0x7fb1c40b0760 2023-12-20T04:40:42.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.234+0000 7fb1c9716640 1 --2- 172.21.15.32:0/1390022140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40b0760 0x7fb1c40b0bc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40097f0 tx=0x7fb1b401c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ab85557ccf4c289d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.234+0000 7fb1c8f15640 1 -- 172.21.15.32:0/1390022140 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40ab600 con 0x7fb1c40b0760 2023-12-20T04:40:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.234+0000 7fb1c8f15640 1 -- 172.21.15.32:0/1390022140 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b40ab760 con 0x7fb1c40b0760 2023-12-20T04:40:42.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.234+0000 7fb1c8f15640 1 -- 172.21.15.32:0/1390022140 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40abaf0 con 0x7fb1c40b0760 2023-12-20T04:40:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.234+0000 7fb1cc17b640 1 -- 172.21.15.32:0/1390022140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40b0760 msgr2=0x7fb1c40b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.234+0000 7fb1cc17b640 1 --2- 172.21.15.32:0/1390022140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40b0760 0x7fb1c40b0bc0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40097f0 tx=0x7fb1b401c400 comp rx=0 tx=0).stop 2023-12-20T04:40:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 -- 172.21.15.32:0/1390022140 shutdown_connections 2023-12-20T04:40:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 --2- 172.21.15.32:0/1390022140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40b0760 0x7fb1c40b0bc0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 -- 172.21.15.32:0/1390022140 >> 172.21.15.32:0/1390022140 conn(0x7fb1c406a510 msgr2=0x7fb1c40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 -- 172.21.15.32:0/1390022140 shutdown_connections 2023-12-20T04:40:42.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 -- 172.21.15.32:0/1390022140 wait complete. 2023-12-20T04:40:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 Processor -- start 2023-12-20T04:40:42.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 -- start start 2023-12-20T04:40:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c40b0760 0x7fb1c40c3d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40cc4b0 0x7fb1c4158fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4157480 0x7fb1c414eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4151f90 con 0x7fb1c4157480 2023-12-20T04:40:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4150890 con 0x7fb1c40b0760 2023-12-20T04:40:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1cc17b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c4151940 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1ca718640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4157480 0x7fb1c414eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1c9716640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40cc4b0 0x7fb1c4158fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1c9716640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40cc4b0 0x7fb1c4158fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:47538/0 (socket says 172.21.15.32:47538) 2023-12-20T04:40:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1c9716640 1 -- 172.21.15.32:0/2951619795 learned_addr learned my addr 172.21.15.32:0/2951619795 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1c9f17640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c40b0760 0x7fb1c40c3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1c9716640 1 -- 172.21.15.32:0/2951619795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c40b0760 msgr2=0x7fb1c40c3d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1c9716640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c40b0760 0x7fb1c40c3d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1c9716640 1 -- 172.21.15.32:0/2951619795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4157480 msgr2=0x7fb1c414eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1c9716640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4157480 0x7fb1c414eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1c9716640 1 -- 172.21.15.32:0/2951619795 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b4009420 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1ca718640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4157480 0x7fb1c414eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.238+0000 7fb1c9716640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40cc4b0 0x7fb1c4158fb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40ad040 tx=0x7fb1b401ac40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.242+0000 7fb1ba7fc640 1 -- 172.21.15.32:0/2951619795 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4004220 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.242+0000 7fb1ba7fc640 1 -- 172.21.15.32:0/2951619795 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b4004380 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.242+0000 7fb1cc17b640 1 -- 172.21.15.32:0/2951619795 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c4151c20 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.242+0000 7fb1ba7fc640 1 -- 172.21.15.32:0/2951619795 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4003840 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.242+0000 7fb1cc17b640 1 -- 172.21.15.32:0/2951619795 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c41594f0 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.242+0000 7fb1cc17b640 1 -- 172.21.15.32:0/2951619795 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c41596e0 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.242+0000 7fb1ba7fc640 1 -- 172.21.15.32:0/2951619795 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7fb1b40039a0 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.246+0000 7fb1ba7fc640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1a8065c40 0x7fb1a8068100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.246+0000 7fb1ba7fc640 1 -- 172.21.15.32:0/2951619795 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6167+0+0 (secure 0 0 0) 0x7fb1b411e1c0 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.246+0000 7fb1c9f17640 1 -- 172.21.15.32:0/2951619795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1a8065c40 msgr2=0x7fb1a8068100 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:40:42.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.246+0000 7fb1c9f17640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1a8065c40 0x7fb1a8068100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:40:42.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.250+0000 7fb1ba7fc640 1 -- 172.21.15.32:0/2951619795 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb1c41596e0 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.381 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:42.382+0000 7f6d88262180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:40:42.408 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:42.410+0000 7fcb0e255180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:40:42.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.410+0000 7fb1cc17b640 1 -- 172.21.15.32:0/2951619795 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fb1c41517a0 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.410+0000 7fb1ba7fc640 1 -- 172.21.15.32:0/2951619795 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v45) v1 ==== 104+0+11 (secure 0 0 0) 0x7fb1c41517a0 con 0x7fb1c40cc4b0 2023-12-20T04:40:42.409 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:40:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 -- 172.21.15.32:0/2951619795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1a8065c40 msgr2=0x7fb1a8068100 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:40:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1a8065c40 0x7fb1a8068100 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 -- 172.21.15.32:0/2951619795 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40cc4b0 msgr2=0x7fb1c4158fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40cc4b0 0x7fb1c4158fb0 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fb1b40ad040 tx=0x7fb1b401ac40 comp rx=0 tx=0).stop 2023-12-20T04:40:42.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 -- 172.21.15.32:0/2951619795 shutdown_connections 2023-12-20T04:40:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1a8065c40 0x7fb1a8068100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4157480 0x7fb1c414eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c40cc4b0 0x7fb1c4158fb0 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 --2- 172.21.15.32:0/2951619795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c40b0760 0x7fb1c40c3d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 -- 172.21.15.32:0/2951619795 >> 172.21.15.32:0/2951619795 conn(0x7fb1c406a510 msgr2=0x7fb1c406c590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 -- 172.21.15.32:0/2951619795 shutdown_connections 2023-12-20T04:40:42.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.414+0000 7fb1cc17b640 1 -- 172.21.15.32:0/2951619795 wait complete. 2023-12-20T04:40:42.431 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:40:42.443 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:42.446+0000 7f6d88262180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:40:42.474 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:42.474+0000 7fcb0e255180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:40:42.503 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:42.506+0000 7f6d88262180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:40:42.535 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:42.534+0000 7fcb0e255180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:40:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb41ab6640 1 Processor -- start 2023-12-20T04:40:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb41ab6640 1 -- start start 2023-12-20T04:40:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb41ab6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcb3c0b0760 0x7fcb3c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb41ab6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb3c0cc4b0 0x7fcb3c0cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb41ab6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb3c0b1100 con 0x7fcb3c157480 2023-12-20T04:40:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb41ab6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3c0c9f50 con 0x7fcb3c0cc4b0 2023-12-20T04:40:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb41ab6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb3c0cde20 con 0x7fcb3c0b0760 2023-12-20T04:40:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb3bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb3c0cc4b0 0x7fcb3c0cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb3b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcb3c157480 0x7fcb3c0c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38796/0 (socket says 172.21.15.32:38796) 2023-12-20T04:40:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb3bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcb3c0cc4b0 0x7fcb3c0cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48528/0 (socket says 172.21.15.32:48528) 2023-12-20T04:40:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb3b7fe640 1 -- 172.21.15.32:0/1457455116 learned_addr learned my addr 172.21.15.32:0/1457455116 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb3affd640 1 --2- 172.21.15.32:0/1457455116 >> v2:172.21.15.39:3300/0 conn(0x7fcb3c0b0760 0x7fcb3c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb3bfff640 1 -- 172.21.15.32:0/1457455116 >> v2:172.21.15.39:3300/0 conn(0x7fcb3c0b0760 msgr2=0x7fcb3c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:42.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb3bfff640 1 --2- 172.21.15.32:0/1457455116 >> v2:172.21.15.39:3300/0 conn(0x7fcb3c0b0760 0x7fcb3c0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb3bfff640 1 -- 172.21.15.32:0/1457455116 >> v1:172.21.15.32:6789/0 conn(0x7fcb3c157480 legacy=0x7fcb3c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb3bfff640 1 -- 172.21.15.32:0/1457455116 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb3c0c8e30 con 0x7fcb3c0cc4b0 2023-12-20T04:40:42.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.582+0000 7fcb3a7fc640 1 -- 172.21.15.32:0/1457455116 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4174004282 0 0) 0x7fcb3c0b1100 con 0x7fcb3c157480 2023-12-20T04:40:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb3bfff640 1 --2- 172.21.15.32:0/1457455116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0cc4b0 0x7fcb3c0cc8d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c06e0d0 tx=0x7fcb3401c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b608886efc4b04ff server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb3a7fc640 1 -- 172.21.15.32:0/1457455116 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340a5910 con 0x7fcb3c0cc4b0 2023-12-20T04:40:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb3a7fc640 1 -- 172.21.15.32:0/1457455116 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb340a5a70 con 0x7fcb3c0cc4b0 2023-12-20T04:40:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb3a7fc640 1 -- 172.21.15.32:0/1457455116 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340a5e00 con 0x7fcb3c0cc4b0 2023-12-20T04:40:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb41ab6640 1 -- 172.21.15.32:0/1457455116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0cc4b0 msgr2=0x7fcb3c0cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb41ab6640 1 --2- 172.21.15.32:0/1457455116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0cc4b0 0x7fcb3c0cc8d0 secure :-1 s=READY pgs=512 cs=0 l=1 rev1=1 crypto rx=0x7fcb3c06e0d0 tx=0x7fcb3401c850 comp rx=0 tx=0).stop 2023-12-20T04:40:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb41ab6640 1 -- 172.21.15.32:0/1457455116 shutdown_connections 2023-12-20T04:40:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb41ab6640 1 --2- 172.21.15.32:0/1457455116 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c0cc4b0 0x7fcb3c0cc8d0 unknown :-1 s=CLOSED pgs=512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb41ab6640 1 --2- 172.21.15.32:0/1457455116 >> v2:172.21.15.39:3300/0 conn(0x7fcb3c0b0760 0x7fcb3c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb41ab6640 1 -- 172.21.15.32:0/1457455116 >> 172.21.15.32:0/1457455116 conn(0x7fcb3c06a510 msgr2=0x7fcb3c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:42.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb41ab6640 1 -- 172.21.15.32:0/1457455116 shutdown_connections 2023-12-20T04:40:42.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.586+0000 7fcb41ab6640 1 -- 172.21.15.32:0/1457455116 wait complete. 2023-12-20T04:40:42.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb41ab6640 1 Processor -- start 2023-12-20T04:40:42.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb41ab6640 1 -- start start 2023-12-20T04:40:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb41ab6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcb3c0b0760 0x7fcb3c14e0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb41ab6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb3c0cc4b0 0x7fcb3c15da70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb41ab6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c157480 0x7fcb3c15b1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb41ab6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c14bc10 con 0x7fcb3c157480 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb41ab6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c166dd0 con 0x7fcb3c0cc4b0 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb41ab6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb3c1670b0 con 0x7fcb3c0b0760 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb3bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c157480 0x7fcb3c15b1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb3b7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcb3c0b0760 0x7fcb3c14e0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb3bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c157480 0x7fcb3c15b1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48540/0 (socket says 172.21.15.32:48540) 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb3bfff640 1 -- 172.21.15.32:0/2048541147 learned_addr learned my addr 172.21.15.32:0/2048541147 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb3affd640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb3c0cc4b0 0x7fcb3c15da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb3bfff640 1 -- 172.21.15.32:0/2048541147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcb3c0b0760 msgr2=0x7fcb3c14e0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb3bfff640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcb3c0b0760 0x7fcb3c14e0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb3bfff640 1 -- 172.21.15.32:0/2048541147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb3c0cc4b0 msgr2=0x7fcb3c15da70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb3bfff640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb3c0cc4b0 0x7fcb3c15da70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb3bfff640 1 -- 172.21.15.32:0/2048541147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb34009a70 con 0x7fcb3c157480 2023-12-20T04:40:42.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.590+0000 7fcb3bfff640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c157480 0x7fcb3c15b1a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcb3401cd30 tx=0x7fcb340195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:42.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.594+0000 7fcb23fff640 1 -- 172.21.15.32:0/2048541147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340ad850 con 0x7fcb3c157480 2023-12-20T04:40:42.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.594+0000 7fcb23fff640 1 -- 172.21.15.32:0/2048541147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb340ad9b0 con 0x7fcb3c157480 2023-12-20T04:40:42.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.594+0000 7fcb23fff640 1 -- 172.21.15.32:0/2048541147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb340add40 con 0x7fcb3c157480 2023-12-20T04:40:42.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.594+0000 7fcb41ab6640 1 -- 172.21.15.32:0/2048541147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb3c167390 con 0x7fcb3c157480 2023-12-20T04:40:42.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.594+0000 7fcb41ab6640 1 -- 172.21.15.32:0/2048541147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb3c167950 con 0x7fcb3c157480 2023-12-20T04:40:42.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.594+0000 7fcb41ab6640 1 -- 172.21.15.32:0/2048541147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb3c15adf0 con 0x7fcb3c157480 2023-12-20T04:40:42.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.594+0000 7fcb23fff640 1 -- 172.21.15.32:0/2048541147 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7fcb340b4070 con 0x7fcb3c157480 2023-12-20T04:40:42.595 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:42.594+0000 7f6d88262180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:40:42.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.594+0000 7fcb23fff640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcb28065c10 0x7fcb280680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.594+0000 7fcb23fff640 1 -- 172.21.15.32:0/2048541147 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(45..45 src has 1..45) v4 ==== 6167+0+0 (secure 0 0 0) 0x7fcb3411dec0 con 0x7fcb3c157480 2023-12-20T04:40:42.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.594+0000 7fcb3b7fe640 1 -- 172.21.15.32:0/2048541147 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcb28065c10 msgr2=0x7fcb280680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:40:42.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.594+0000 7fcb3b7fe640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcb28065c10 0x7fcb280680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:40:42.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.598+0000 7fcb23fff640 1 -- 172.21.15.32:0/2048541147 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcb3c15adf0 con 0x7fcb3c157480 2023-12-20T04:40:42.629 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:42.630+0000 7fcb0e255180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:40:42.654 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:42.654+0000 7f6d88262180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:40:42.693 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:42.694+0000 7fcb0e255180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:40:42.744 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:42.746+0000 7f6d88262180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:40:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.758+0000 7fcb41ab6640 1 -- 172.21.15.32:0/2048541147 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fcb3c15adf0 con 0x7fcb3c157480 2023-12-20T04:40:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.758+0000 7fcb23fff640 1 -- 172.21.15.32:0/2048541147 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v45) v1 ==== 100+0+11 (secure 0 0 0) 0x7fcb3c15adf0 con 0x7fcb3c157480 2023-12-20T04:40:42.759 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:40:42.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 -- 172.21.15.32:0/2048541147 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcb28065c10 msgr2=0x7fcb280680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:40:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcb28065c10 0x7fcb280680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 -- 172.21.15.32:0/2048541147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c157480 msgr2=0x7fcb3c15b1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c157480 0x7fcb3c15b1a0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7fcb3401cd30 tx=0x7fcb340195f0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 -- 172.21.15.32:0/2048541147 shutdown_connections 2023-12-20T04:40:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcb28065c10 0x7fcb280680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb3c157480 0x7fcb3c15b1a0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb3c0cc4b0 0x7fcb3c15da70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 --2- 172.21.15.32:0/2048541147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcb3c0b0760 0x7fcb3c14e0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 -- 172.21.15.32:0/2048541147 >> 172.21.15.32:0/2048541147 conn(0x7fcb3c06a510 msgr2=0x7fcb3c0c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 -- 172.21.15.32:0/2048541147 shutdown_connections 2023-12-20T04:40:42.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.762+0000 7fcb41ab6640 1 -- 172.21.15.32:0/2048541147 wait complete. 2023-12-20T04:40:42.781 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T04:40:42.784 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:42.786+0000 7fcb0e255180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:40:42.803 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:42.806+0000 7f6d88262180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:40:42.845 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:42.846+0000 7fcb0e255180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:40:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b59efc640 1 Processor -- start 2023-12-20T04:40:42.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b59efc640 1 -- start start 2023-12-20T04:40:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b59efc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8b541599e0 0x7f8b540aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b59efc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b540c90e0 0x7f8b540b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b59efc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8b540711d0 con 0x7f8b540b0b30 2023-12-20T04:40:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b59efc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8b540c98c0 con 0x7f8b541599e0 2023-12-20T04:40:42.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b59efc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b540c9ba0 con 0x7f8b540c90e0 2023-12-20T04:40:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b537fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8b541599e0 0x7f8b540aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b4bfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b540c90e0 0x7f8b540b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b4bfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b540c90e0 0x7f8b540b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:47546/0 (socket says 172.21.15.32:47546) 2023-12-20T04:40:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b4bfff640 1 -- 172.21.15.32:0/1314400599 learned_addr learned my addr 172.21.15.32:0/1314400599 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b537fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8b541599e0 0x7f8b540aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:44392/0 (socket says 172.21.15.32:44392) 2023-12-20T04:40:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b4bfff640 1 -- 172.21.15.32:0/1314400599 >> v2:172.21.15.39:3300/0 conn(0x7f8b541599e0 msgr2=0x7f8b540aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b4bfff640 1 --2- 172.21.15.32:0/1314400599 >> v2:172.21.15.39:3300/0 conn(0x7f8b541599e0 0x7f8b540aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b4bfff640 1 -- 172.21.15.32:0/1314400599 >> v1:172.21.15.39:6789/0 conn(0x7f8b540b0b30 legacy=0x7f8b5415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b4bfff640 1 -- 172.21.15.32:0/1314400599 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b540c9e80 con 0x7f8b540c90e0 2023-12-20T04:40:42.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.938+0000 7f8b537fe640 1 --2- 172.21.15.32:0/1314400599 >> v2:172.21.15.39:3300/0 conn(0x7f8b541599e0 0x7f8b540aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:42.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b4bfff640 1 --2- 172.21.15.32:0/1314400599 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b540c90e0 0x7f8b540b05f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009870 tx=0x7f8b3c01c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b7e2874e12bde43 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:42.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b52ffd640 1 -- 172.21.15.32:0/1314400599 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0a7070 con 0x7f8b540c90e0 2023-12-20T04:40:42.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b52ffd640 1 -- 172.21.15.32:0/1314400599 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b3c0ab430 con 0x7f8b540c90e0 2023-12-20T04:40:42.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b52ffd640 1 -- 172.21.15.32:0/1314400599 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b3c0ab7c0 con 0x7f8b540c90e0 2023-12-20T04:40:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b59efc640 1 -- 172.21.15.32:0/1314400599 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b540c90e0 msgr2=0x7f8b540b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b59efc640 1 --2- 172.21.15.32:0/1314400599 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b540c90e0 0x7f8b540b05f0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8b3c009870 tx=0x7f8b3c01c510 comp rx=0 tx=0).stop 2023-12-20T04:40:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b59efc640 1 -- 172.21.15.32:0/1314400599 shutdown_connections 2023-12-20T04:40:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b59efc640 1 --2- 172.21.15.32:0/1314400599 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b540c90e0 0x7f8b540b05f0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b59efc640 1 --2- 172.21.15.32:0/1314400599 >> v2:172.21.15.39:3300/0 conn(0x7f8b541599e0 0x7f8b540aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b59efc640 1 -- 172.21.15.32:0/1314400599 >> 172.21.15.32:0/1314400599 conn(0x7f8b5406a320 msgr2=0x7f8b540ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b59efc640 1 -- 172.21.15.32:0/1314400599 shutdown_connections 2023-12-20T04:40:42.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.942+0000 7f8b59efc640 1 -- 172.21.15.32:0/1314400599 wait complete. 2023-12-20T04:40:42.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b59efc640 1 Processor -- start 2023-12-20T04:40:42.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b59efc640 1 -- start start 2023-12-20T04:40:42.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b59efc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b540b0b30 0x7f8b5414ed70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b59efc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b540c90e0 0x7f8b5415b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b59efc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b541599e0 0x7f8b5415bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b59efc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5415b3f0 con 0x7f8b540b0b30 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b59efc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5414e620 con 0x7f8b540c90e0 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b59efc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8b5415a560 con 0x7f8b541599e0 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b537fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b540b0b30 0x7f8b5414ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b4bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b540c90e0 0x7f8b5415b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b537fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b540b0b30 0x7f8b5414ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48552/0 (socket says 172.21.15.32:48552) 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b537fe640 1 -- 172.21.15.32:0/644871669 learned_addr learned my addr 172.21.15.32:0/644871669 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b53fff640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b541599e0 0x7f8b5415bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b537fe640 1 -- 172.21.15.32:0/644871669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b541599e0 msgr2=0x7f8b5415bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b537fe640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b541599e0 0x7f8b5415bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b537fe640 1 -- 172.21.15.32:0/644871669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b540c90e0 msgr2=0x7f8b5415b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b537fe640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b540c90e0 0x7f8b5415b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b537fe640 1 -- 172.21.15.32:0/644871669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8b440093b0 con 0x7f8b540b0b30 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b4bfff640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b540c90e0 0x7f8b5415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:42.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b537fe640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b540b0b30 0x7f8b5414ed70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400e990 tx=0x7f8b4400ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:42.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b50ff9640 1 -- 172.21.15.32:0/644871669 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b440059a0 con 0x7f8b540b0b30 2023-12-20T04:40:42.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b50ff9640 1 -- 172.21.15.32:0/644871669 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8b44005b00 con 0x7f8b540b0b30 2023-12-20T04:40:42.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b50ff9640 1 -- 172.21.15.32:0/644871669 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8b44005e90 con 0x7f8b540b0b30 2023-12-20T04:40:42.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b59efc640 1 -- 172.21.15.32:0/644871669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8b3c0094d0 con 0x7f8b540b0b30 2023-12-20T04:40:42.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b59efc640 1 -- 172.21.15.32:0/644871669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8b5415aac0 con 0x7f8b540b0b30 2023-12-20T04:40:42.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.946+0000 7f8b59efc640 1 -- 172.21.15.32:0/644871669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8b5404e000 con 0x7f8b540b0b30 2023-12-20T04:40:42.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.950+0000 7f8b50ff9640 1 -- 172.21.15.32:0/644871669 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f8b4400c9e0 con 0x7f8b540b0b30 2023-12-20T04:40:42.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.950+0000 7f8b50ff9640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b34065c10 0x7f8b340680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:42.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.950+0000 7f8b50ff9640 1 -- 172.21.15.32:0/644871669 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f8b44083de0 con 0x7f8b540b0b30 2023-12-20T04:40:42.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.950+0000 7f8b4bfff640 1 -- 172.21.15.32:0/644871669 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b34065c10 msgr2=0x7f8b340680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:40:42.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.950+0000 7f8b4bfff640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b34065c10 0x7f8b340680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:40:42.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:42.954+0000 7f8b50ff9640 1 -- 172.21.15.32:0/644871669 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8b440510b0 con 0x7f8b540b0b30 2023-12-20T04:40:42.959 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:42.962+0000 7f6d88262180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:40:43.018 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:43.018+0000 7fcb0e255180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:40:43.022 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:43.022+0000 7f6d88262180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:40:43.087 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:43.090+0000 7fcb0e255180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:40:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.146+0000 7f8b59efc640 1 -- 172.21.15.32:0/644871669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f8b540cf550 con 0x7f8b540b0b30 2023-12-20T04:40:43.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.146+0000 7f8b50ff9640 1 -- 172.21.15.32:0/644871669 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 6 entries v46) v1 ==== 80+0+342 (secure 0 0 0) 0x7f8b44020020 con 0x7f8b540b0b30 2023-12-20T04:40:43.145 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2331287763 2023-12-21T04:29:47.726855+0000 2023-12-20T04:40:43.145 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/38009 2023-12-21T04:29:47.726855+0000 2023-12-20T04:40:43.145 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/38009 2023-12-21T04:29:47.726855+0000 2023-12-20T04:40:43.146 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/728956527 2023-12-21T04:29:47.726855+0000 2023-12-20T04:40:43.146 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/856257861 2023-12-21T04:29:47.726855+0000 2023-12-20T04:40:43.146 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2782983206 2023-12-21T04:29:47.726855+0000 2023-12-20T04:40:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:listed 6 entries 2023-12-20T04:40:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 -- 172.21.15.32:0/644871669 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b34065c10 msgr2=0x7f8b340680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:40:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b34065c10 0x7f8b340680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 -- 172.21.15.32:0/644871669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b540b0b30 msgr2=0x7f8b5414ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b540b0b30 0x7f8b5414ed70 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7f8b4400e990 tx=0x7f8b4400ee50 comp rx=0 tx=0).stop 2023-12-20T04:40:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 -- 172.21.15.32:0/644871669 shutdown_connections 2023-12-20T04:40:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8b34065c10 0x7f8b340680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8b541599e0 0x7f8b5415bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8b540c90e0 0x7f8b5415b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 --2- 172.21.15.32:0/644871669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8b540b0b30 0x7f8b5414ed70 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 -- 172.21.15.32:0/644871669 >> 172.21.15.32:0/644871669 conn(0x7f8b5406a320 msgr2=0x7f8b540700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 -- 172.21.15.32:0/644871669 shutdown_connections 2023-12-20T04:40:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.150+0000 7f8b59efc640 1 -- 172.21.15.32:0/644871669 wait complete. 2023-12-20T04:40:43.167 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T04:40:43.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:43.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T04:40:43.206 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:40:43.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:40:43.206 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T04:40:43.271 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:40:43.272 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:40:43.277 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:40:43.277 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:40:43.277 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:40:43.281 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:40:43.281 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:40:43.281 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:40:43.286 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:43.286+0000 7f6d88262180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:40:43.323 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:40:43.324 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:40:43.324 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:40:43.333 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:40:43.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:40:43.343 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:40:43.343 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:40:43.343 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:40:43.352 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:43.354+0000 7fcb0e255180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:40:43.429 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:43.430+0000 7f6d88262180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:40:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899ea9a640 1 Processor -- start 2023-12-20T04:40:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899ea9a640 1 -- start start 2023-12-20T04:40:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899ea9a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89980b8480 0x7f899806af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899ea9a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899806b460 0x7f8998160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899ea9a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8998161330 0x7f8998153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899ea9a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f899806b8c0 con 0x7f89980b8480 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899ea9a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89980cd4d0 con 0x7f8998161330 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899ea9a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89980ce4e0 con 0x7f899806b460 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899da98640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89980b8480 0x7f899806af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899da98640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89980b8480 0x7f899806af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48560/0 (socket says 172.21.15.32:48560) 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899da98640 1 -- 172.21.15.32:0/1486768566 learned_addr learned my addr 172.21.15.32:0/1486768566 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899e299640 1 --2- 172.21.15.32:0/1486768566 >> v2:172.21.15.39:3300/0 conn(0x7f8998161330 0x7f8998153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899d297640 1 --2- 172.21.15.32:0/1486768566 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899806b460 0x7f8998160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899da98640 1 -- 172.21.15.32:0/1486768566 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899806b460 msgr2=0x7f8998160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899da98640 1 --2- 172.21.15.32:0/1486768566 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899806b460 0x7f8998160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899da98640 1 -- 172.21.15.32:0/1486768566 >> v2:172.21.15.39:3300/0 conn(0x7f8998161330 msgr2=0x7f8998153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899da98640 1 --2- 172.21.15.32:0/1486768566 >> v2:172.21.15.39:3300/0 conn(0x7f8998161330 0x7f8998153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.490+0000 7f899da98640 1 -- 172.21.15.32:0/1486768566 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89980cd720 con 0x7f89980b8480 2023-12-20T04:40:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899da98640 1 --2- 172.21.15.32:0/1486768566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89980b8480 0x7f899806af20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f899806bae0 tx=0x7f899401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34d0eba62cfb4cd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ca96640 1 -- 172.21.15.32:0/1486768566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89940ac410 con 0x7f89980b8480 2023-12-20T04:40:43.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ca96640 1 -- 172.21.15.32:0/1486768566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89940ac570 con 0x7f89980b8480 2023-12-20T04:40:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ca96640 1 -- 172.21.15.32:0/1486768566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89940ac900 con 0x7f89980b8480 2023-12-20T04:40:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ea9a640 1 -- 172.21.15.32:0/1486768566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89980b8480 msgr2=0x7f899806af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ea9a640 1 --2- 172.21.15.32:0/1486768566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89980b8480 0x7f899806af20 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7f899806bae0 tx=0x7f899401c500 comp rx=0 tx=0).stop 2023-12-20T04:40:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ea9a640 1 -- 172.21.15.32:0/1486768566 shutdown_connections 2023-12-20T04:40:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ea9a640 1 --2- 172.21.15.32:0/1486768566 >> v2:172.21.15.39:3300/0 conn(0x7f8998161330 0x7f8998153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ea9a640 1 --2- 172.21.15.32:0/1486768566 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899806b460 0x7f8998160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ea9a640 1 --2- 172.21.15.32:0/1486768566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89980b8480 0x7f899806af20 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ea9a640 1 -- 172.21.15.32:0/1486768566 >> 172.21.15.32:0/1486768566 conn(0x7f89980d2a50 msgr2=0x7f89980d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ea9a640 1 -- 172.21.15.32:0/1486768566 shutdown_connections 2023-12-20T04:40:43.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.494+0000 7f899ea9a640 1 -- 172.21.15.32:0/1486768566 wait complete. 2023-12-20T04:40:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899ea9a640 1 Processor -- start 2023-12-20T04:40:43.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899ea9a640 1 -- start start 2023-12-20T04:40:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899ea9a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899806b460 0x7f899814b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899ea9a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89980b8480 0x7f899814cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899ea9a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8998161330 0x7f8998156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899ea9a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8998156e60 con 0x7f8998161330 2023-12-20T04:40:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899ea9a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899814bec0 con 0x7f89980b8480 2023-12-20T04:40:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899ea9a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899814c1a0 con 0x7f899806b460 2023-12-20T04:40:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899e299640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8998161330 0x7f8998156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:43.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899e299640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8998161330 0x7f8998156920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48568/0 (socket says 172.21.15.32:48568) 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899e299640 1 -- 172.21.15.32:0/2212378413 learned_addr learned my addr 172.21.15.32:0/2212378413 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899da98640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899806b460 0x7f899814b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899d297640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89980b8480 0x7f899814cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899e299640 1 -- 172.21.15.32:0/2212378413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899806b460 msgr2=0x7f899814b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899e299640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899806b460 0x7f899814b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899e299640 1 -- 172.21.15.32:0/2212378413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89980b8480 msgr2=0x7f899814cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899e299640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89980b8480 0x7f899814cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899e299640 1 -- 172.21.15.32:0/2212378413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89940094d0 con 0x7f8998161330 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899e299640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8998161330 0x7f8998156920 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f899000efc0 tx=0x7f899000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f898e7fc640 1 -- 172.21.15.32:0/2212378413 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8990009ba0 con 0x7f8998161330 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f898e7fc640 1 -- 172.21.15.32:0/2212378413 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f899000f040 con 0x7f8998161330 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f898e7fc640 1 -- 172.21.15.32:0/2212378413 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89900146c0 con 0x7f8998161330 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899ea9a640 1 -- 172.21.15.32:0/2212378413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899814c480 con 0x7f8998161330 2023-12-20T04:40:43.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.498+0000 7f899ea9a640 1 -- 172.21.15.32:0/2212378413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899814ca40 con 0x7f8998161330 2023-12-20T04:40:43.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.502+0000 7f899ea9a640 1 -- 172.21.15.32:0/2212378413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899814cc60 con 0x7f8998161330 2023-12-20T04:40:43.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.502+0000 7f898e7fc640 1 -- 172.21.15.32:0/2212378413 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f8990014820 con 0x7f8998161330 2023-12-20T04:40:43.500 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:43.502+0000 7fcb0e255180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:40:43.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.502+0000 7f898e7fc640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065c10 0x7f897c0680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.502+0000 7f899da98640 1 -- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065c10 msgr2=0x7f897c0680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:40:43.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.502+0000 7f898e7fc640 1 -- 172.21.15.32:0/2212378413 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f8990084510 con 0x7f8998161330 2023-12-20T04:40:43.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.502+0000 7f899da98640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065c10 0x7f897c0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:40:43.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.506+0000 7f898e7fc640 1 -- 172.21.15.32:0/2212378413 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8990051620 con 0x7f8998161330 2023-12-20T04:40:43.560 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:43.562+0000 7f6d88262180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:40:43.634 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:43.634+0000 7f6d88262180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:40:43.641 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:43.642+0000 7fcb0e255180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:40:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.706+0000 7f899da98640 1 -- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065c10 msgr2=0x7f897c0680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:40:43.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.706+0000 7f899da98640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065c10 0x7f897c0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:40:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.706+0000 7f899ea9a640 1 -- 172.21.15.32:0/2212378413 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8998074100 con 0x7f8998161330 2023-12-20T04:40:43.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.706+0000 7f898e7fc640 1 -- 172.21.15.32:0/2212378413 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f8990054c40 con 0x7f8998161330 2023-12-20T04:40:43.706 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:43.706 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2541430692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2541430692},{"type":"v1","addr":"172.21.15.39:6835","nonce":2541430692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1379533068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1379533068},{"type":"v1","addr":"172.21.15.32:6837","nonce":1379533068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4021567802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4021567802},{"type":"v1","addr":"172.21.15.39:6834","nonce":4021567802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:29:30.610189+0000","modified":"2023-12-20T04:29:33.661905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3391762708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3391762708},{"type":"v1","addr":"172.21.15.32:6836","nonce":3391762708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-20T04:40:43.706 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2023-12-20T04:40:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 -- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065c10 msgr2=0x7f897c0680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:40:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065c10 0x7f897c0680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 -- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8998161330 msgr2=0x7f8998156920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8998161330 0x7f8998156920 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f899000efc0 tx=0x7f899000c490 comp rx=0 tx=0).stop 2023-12-20T04:40:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 -- 172.21.15.32:0/2212378413 shutdown_connections 2023-12-20T04:40:43.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065c10 0x7f897c0680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8998161330 0x7f8998156920 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89980b8480 0x7f899814cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 --2- 172.21.15.32:0/2212378413 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899806b460 0x7f899814b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 -- 172.21.15.32:0/2212378413 >> 172.21.15.32:0/2212378413 conn(0x7f89980d2a50 msgr2=0x7f899815e620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 -- 172.21.15.32:0/2212378413 shutdown_connections 2023-12-20T04:40:43.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.710+0000 7f899ea9a640 1 -- 172.21.15.32:0/2212378413 wait complete. 2023-12-20T04:40:43.717 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:43.718+0000 7fcb0e255180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:40:43.728 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:40:43.777 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:43.778+0000 7f6d88262180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:40:43.863 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:43.866+0000 7fcb0e255180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:40:43.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.918+0000 7f20e4598640 1 Processor -- start 2023-12-20T04:40:43.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.918+0000 7f20e4598640 1 -- start start 2023-12-20T04:40:43.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e4598640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20dc0cfc80 0x7f20dc167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e4598640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20dc0b6ca0 con 0x7f20dc0d2740 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e4598640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20dc15d190 con 0x7f20dc15cdb0 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e4598640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc0d16a0 con 0x7f20dc0cfc80 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e1b33640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f20dc0d2740 0x7f20dc0cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38806/0 (socket says 172.21.15.32:38806) 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e1b33640 1 -- 172.21.15.32:0/904965893 learned_addr learned my addr 172.21.15.32:0/904965893 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e2b35640 1 --2- 172.21.15.32:0/904965893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20dc0cfc80 0x7f20dc167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e1332640 1 -- 172.21.15.32:0/904965893 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3376253672 0 0) 0x7f20cc001120 con 0x7f20dc0d2740 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e1332640 1 -- 172.21.15.32:0/904965893 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20c8003530 con 0x7f20dc0d2740 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e1332640 1 -- 172.21.15.32:0/904965893 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4139241063 0 0) 0x7f20dc15d190 con 0x7f20dc15cdb0 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e1332640 1 -- 172.21.15.32:0/904965893 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f20cc001120 con 0x7f20dc15cdb0 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e2b35640 1 -- 172.21.15.32:0/904965893 >> v1:172.21.15.39:6789/0 conn(0x7f20dc15cdb0 legacy=0x7f20dc0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:43.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e2b35640 1 -- 172.21.15.32:0/904965893 >> v1:172.21.15.32:6789/0 conn(0x7f20dc0d2740 legacy=0x7f20dc0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.922+0000 7f20e2b35640 1 -- 172.21.15.32:0/904965893 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20dc0d1980 con 0x7f20dc0cfc80 2023-12-20T04:40:43.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e2b35640 1 --2- 172.21.15.32:0/904965893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20dc0cfc80 0x7f20dc167910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009dc0 tx=0x7f20d401f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6e58a53cc22f726b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e1332640 1 -- 172.21.15.32:0/904965893 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40af600 con 0x7f20dc0cfc80 2023-12-20T04:40:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e1332640 1 -- 172.21.15.32:0/904965893 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20d40ab050 con 0x7f20dc0cfc80 2023-12-20T04:40:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e1332640 1 -- 172.21.15.32:0/904965893 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20d40af980 con 0x7f20dc0cfc80 2023-12-20T04:40:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e4598640 1 -- 172.21.15.32:0/904965893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20dc0cfc80 msgr2=0x7f20dc167910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e4598640 1 --2- 172.21.15.32:0/904965893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20dc0cfc80 0x7f20dc167910 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f20d4009dc0 tx=0x7f20d401f150 comp rx=0 tx=0).stop 2023-12-20T04:40:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e4598640 1 -- 172.21.15.32:0/904965893 shutdown_connections 2023-12-20T04:40:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e4598640 1 --2- 172.21.15.32:0/904965893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20dc0cfc80 0x7f20dc167910 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e4598640 1 -- 172.21.15.32:0/904965893 >> 172.21.15.32:0/904965893 conn(0x7f20dc065ee0 msgr2=0x7f20dc06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e4598640 1 -- 172.21.15.32:0/904965893 shutdown_connections 2023-12-20T04:40:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e4598640 1 -- 172.21.15.32:0/904965893 wait complete. 2023-12-20T04:40:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e4598640 1 Processor -- start 2023-12-20T04:40:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.926+0000 7f20e4598640 1 -- start start 2023-12-20T04:40:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e4598640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20dc0cfc80 0x7f20dc1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e4598640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc0d2740 0x7f20dc0c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e4598640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20dc15cdb0 0x7f20dc15ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e4598640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc145410 con 0x7f20dc0d2740 2023-12-20T04:40:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e4598640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc144790 con 0x7f20dc0cfc80 2023-12-20T04:40:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e4598640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20dc144a70 con 0x7f20dc15cdb0 2023-12-20T04:40:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e1b33640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc0d2740 0x7f20dc0c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e1b33640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc0d2740 0x7f20dc0c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48582/0 (socket says 172.21.15.32:48582) 2023-12-20T04:40:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e1b33640 1 -- 172.21.15.32:0/2910555607 learned_addr learned my addr 172.21.15.32:0/2910555607 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e2b35640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20dc15cdb0 0x7f20dc15ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e2334640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20dc0cfc80 0x7f20dc1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e1b33640 1 -- 172.21.15.32:0/2910555607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20dc15cdb0 msgr2=0x7f20dc15ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e1b33640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20dc15cdb0 0x7f20dc15ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e1b33640 1 -- 172.21.15.32:0/2910555607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20dc0cfc80 msgr2=0x7f20dc1548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e1b33640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20dc0cfc80 0x7f20dc1548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e1b33640 1 -- 172.21.15.32:0/2910555607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20d4009a70 con 0x7f20dc0d2740 2023-12-20T04:40:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e1b33640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc0d2740 0x7f20dc0c7f30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00c650 tx=0x7f20cc00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:43.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20d2ffd640 1 -- 172.21.15.32:0/2910555607 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc005ac0 con 0x7f20dc0d2740 2023-12-20T04:40:43.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20d2ffd640 1 -- 172.21.15.32:0/2910555607 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f20cc005c40 con 0x7f20dc0d2740 2023-12-20T04:40:43.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e4598640 1 -- 172.21.15.32:0/2910555607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20dc144d50 con 0x7f20dc0d2740 2023-12-20T04:40:43.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20d2ffd640 1 -- 172.21.15.32:0/2910555607 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f20cc007070 con 0x7f20dc0d2740 2023-12-20T04:40:43.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e4598640 1 -- 172.21.15.32:0/2910555607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20dc153ad0 con 0x7f20dc0d2740 2023-12-20T04:40:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.930+0000 7f20e4598640 1 -- 172.21.15.32:0/2910555607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f20dc145550 con 0x7f20dc0d2740 2023-12-20T04:40:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.934+0000 7f20d2ffd640 1 -- 172.21.15.32:0/2910555607 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f20cc0071f0 con 0x7f20dc0d2740 2023-12-20T04:40:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.934+0000 7f20d2ffd640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f20c0065c10 0x7f20c00680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.934+0000 7f20d2ffd640 1 -- 172.21.15.32:0/2910555607 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(46..46 src has 1..46) v4 ==== 7031+0+0 (secure 0 0 0) 0x7f20cc082e30 con 0x7f20dc0d2740 2023-12-20T04:40:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.934+0000 7f20e2334640 1 -- 172.21.15.32:0/2910555607 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f20c0065c10 msgr2=0x7f20c00680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:40:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.934+0000 7f20e2334640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f20c0065c10 0x7f20c00680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:40:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:43.938+0000 7f20d2ffd640 1 -- 172.21.15.32:0/2910555607 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f20dc145550 con 0x7f20dc0d2740 2023-12-20T04:40:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.106+0000 7f20e4598640 1 -- 172.21.15.32:0/2910555607 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20dc0b6ca0 con 0x7f20dc0d2740 2023-12-20T04:40:44.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.106+0000 7f20d2ffd640 1 -- 172.21.15.32:0/2910555607 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v47) v1 ==== 74+0+17329 (secure 0 0 0) 0x7f20dc0b6ca0 con 0x7f20dc0d2740 2023-12-20T04:40:44.106 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:44.107 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":47,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:40:43.981980+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:29:28.685343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-20T04:29:29.647655+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2331287763":"2023-12-21T04:29:47.726855+0000","172.21.15.32:6833/38009":"2023-12-21T04:29:47.726855+0000","172.21.15.32:6832/38009":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/728956527":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/856257861":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/2782983206":"2023-12-21T04:29:47.726855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":57,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":68,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":2},{"begin":96,"length":1},{"begin":100,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":188,"length":34}]}],"new_removed_snaps":[{"pool":5,"snaps":[{"begin":111,"length":1},{"begin":221,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:40:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 -- 172.21.15.32:0/2910555607 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f20c0065c10 msgr2=0x7f20c00680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:40:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f20c0065c10 0x7f20c00680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 -- 172.21.15.32:0/2910555607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc0d2740 msgr2=0x7f20dc0c7f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc0d2740 0x7f20dc0c7f30 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7f20cc00c650 tx=0x7f20cc00cb10 comp rx=0 tx=0).stop 2023-12-20T04:40:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 -- 172.21.15.32:0/2910555607 shutdown_connections 2023-12-20T04:40:44.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f20c0065c10 0x7f20c00680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20dc15cdb0 0x7f20dc15ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20dc0d2740 0x7f20dc0c7f30 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 --2- 172.21.15.32:0/2910555607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20dc0cfc80 0x7f20dc1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 -- 172.21.15.32:0/2910555607 >> 172.21.15.32:0/2910555607 conn(0x7f20dc065ee0 msgr2=0x7f20dc0b6140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 -- 172.21.15.32:0/2910555607 shutdown_connections 2023-12-20T04:40:44.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.110+0000 7f20e4598640 1 -- 172.21.15.32:0/2910555607 wait complete. 2023-12-20T04:40:44.129 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:29:28.685343+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '34', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 5, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:29:29.647655+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '47', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 221, 'snap_epoch': 47, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:40:44.129 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:40:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.282+0000 7ff6e8803640 1 Processor -- start 2023-12-20T04:40:44.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.282+0000 7ff6e8803640 1 -- start start 2023-12-20T04:40:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.282+0000 7ff6e8803640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6e40cf3e0 0x7ff6e40cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.282+0000 7ff6e8803640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e40cfd80 con 0x7ff6e40d1580 2023-12-20T04:40:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.282+0000 7ff6e8803640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6e40d10f0 con 0x7ff6e415ccf0 2023-12-20T04:40:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.282+0000 7ff6e8803640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6e40ce2f0 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.282+0000 7ff6e259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6e40cf3e0 0x7ff6e40cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.282+0000 7ff6e2d9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff6e415ccf0 0x7ff6e40cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:33036/0 (socket says 172.21.15.32:33036) 2023-12-20T04:40:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.282+0000 7ff6e2d9d640 1 -- 172.21.15.32:0/1462408137 learned_addr learned my addr 172.21.15.32:0/1462408137 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.282+0000 7ff6e259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6e40cf3e0 0x7ff6e40cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48592/0 (socket says 172.21.15.32:48592) 2023-12-20T04:40:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.286+0000 7ff6e259c640 1 -- 172.21.15.32:0/1462408137 >> v1:172.21.15.39:6789/0 conn(0x7ff6e415ccf0 legacy=0x7ff6e40cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.286+0000 7ff6e259c640 1 -- 172.21.15.32:0/1462408137 >> v1:172.21.15.32:6789/0 conn(0x7ff6e40d1580 legacy=0x7ff6e40d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.286+0000 7ff6e1d9b640 1 -- 172.21.15.32:0/1462408137 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3955337793 0 0) 0x7ff6e40d10f0 con 0x7ff6e415ccf0 2023-12-20T04:40:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.286+0000 7ff6e259c640 1 -- 172.21.15.32:0/1462408137 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e40cdd30 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.286+0000 7ff6e259c640 1 --2- 172.21.15.32:0/1462408137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e40cf3e0 0x7ff6e40cf840 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80097c0 tx=0x7ff6d801c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dfdd5146f6b729e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.286+0000 7ff6e1d9b640 1 -- 172.21.15.32:0/1462408137 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80ab600 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.286+0000 7ff6e1d9b640 1 -- 172.21.15.32:0/1462408137 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d80a7050 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.286+0000 7ff6e1d9b640 1 -- 172.21.15.32:0/1462408137 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d80ab980 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.286+0000 7ff6e8803640 1 -- 172.21.15.32:0/1462408137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e40cf3e0 msgr2=0x7ff6e40cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.286+0000 7ff6e8803640 1 --2- 172.21.15.32:0/1462408137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e40cf3e0 0x7ff6e40cf840 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7ff6d80097c0 tx=0x7ff6d801c280 comp rx=0 tx=0).stop 2023-12-20T04:40:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.290+0000 7ff6e8803640 1 -- 172.21.15.32:0/1462408137 shutdown_connections 2023-12-20T04:40:44.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.290+0000 7ff6e8803640 1 --2- 172.21.15.32:0/1462408137 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e40cf3e0 0x7ff6e40cf840 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.290+0000 7ff6e8803640 1 -- 172.21.15.32:0/1462408137 >> 172.21.15.32:0/1462408137 conn(0x7ff6e40658e0 msgr2=0x7ff6e4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.290+0000 7ff6e8803640 1 -- 172.21.15.32:0/1462408137 shutdown_connections 2023-12-20T04:40:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.290+0000 7ff6e8803640 1 -- 172.21.15.32:0/1462408137 wait complete. 2023-12-20T04:40:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.290+0000 7ff6e8803640 1 Processor -- start 2023-12-20T04:40:44.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.290+0000 7ff6e8803640 1 -- start start 2023-12-20T04:40:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e8803640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6e40cf3e0 0x7ff6e415bf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e8803640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6e40d1580 0x7ff6e40b3060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e8803640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e415ccf0 0x7ff6e40c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e8803640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e415c4b0 con 0x7ff6e415ccf0 2023-12-20T04:40:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e8803640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40b4ae0 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e8803640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6e40b3a40 con 0x7ff6e40d1580 2023-12-20T04:40:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6e40cf3e0 0x7ff6e415bf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6e40d1580 0x7ff6e40b3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e415ccf0 0x7ff6e40c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:44.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6e40d1580 0x7ff6e40b3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:47592/0 (socket says 172.21.15.32:47592) 2023-12-20T04:40:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e2d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6e40cf3e0 0x7ff6e415bf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:44442/0 (socket says 172.21.15.32:44442) 2023-12-20T04:40:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e259c640 1 -- 172.21.15.32:0/1416965060 learned_addr learned my addr 172.21.15.32:0/1416965060 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e2d9d640 1 -- 172.21.15.32:0/1416965060 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6e40d1580 msgr2=0x7ff6e40b3060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e2d9d640 1 --2- 172.21.15.32:0/1416965060 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6e40d1580 0x7ff6e40b3060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e2d9d640 1 -- 172.21.15.32:0/1416965060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e415ccf0 msgr2=0x7ff6e40c7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e2d9d640 1 --2- 172.21.15.32:0/1416965060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e415ccf0 0x7ff6e40c7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e2d9d640 1 -- 172.21.15.32:0/1416965060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d8009420 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e2d9d640 1 --2- 172.21.15.32:0/1416965060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6e40cf3e0 0x7ff6e415bf70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400c3e0 tx=0x7ff6d400c8a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6cb7fe640 1 -- 172.21.15.32:0/1416965060 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d40074e0 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6cb7fe640 1 -- 172.21.15.32:0/1416965060 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6d4007640 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6cb7fe640 1 -- 172.21.15.32:0/1416965060 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6d4004a30 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e8803640 1 -- 172.21.15.32:0/1416965060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6e40b3d20 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e8803640 1 -- 172.21.15.32:0/1416965060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6e40b42e0 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.294+0000 7ff6e8803640 1 -- 172.21.15.32:0/1416965060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6e4074100 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.298+0000 7ff6cb7fe640 1 -- 172.21.15.32:0/1416965060 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7ff6d4004b90 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.298+0000 7ff6cb7fe640 1 --2- 172.21.15.32:0/1416965060 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6d0065c10 0x7ff6d00680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.298+0000 7ff6cb7fe640 1 -- 172.21.15.32:0/1416965060 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6551+0+0 (secure 0 0 0) 0x7ff6d4083470 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.298+0000 7ff6e259c640 1 -- 172.21.15.32:0/1416965060 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6d0065c10 msgr2=0x7ff6d00680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:40:44.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.298+0000 7ff6e259c640 1 --2- 172.21.15.32:0/1416965060 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6d0065c10 0x7ff6d00680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:40:44.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.302+0000 7ff6cb7fe640 1 -- 172.21.15.32:0/1416965060 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff6d4050880 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.359 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:44.362+0000 7f6d88262180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:40:44.449 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:44.450+0000 7f6d88262180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:40:44.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.462+0000 7ff6e8803640 1 -- 172.21.15.32:0/1416965060 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff6e40cd5b0 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.462+0000 7ff6cb7fe640 1 -- 172.21.15.32:0/1416965060 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v47) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff6d4001070 con 0x7ff6e40cf3e0 2023-12-20T04:40:44.460 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:40:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.462+0000 7ff6e8803640 1 -- 172.21.15.32:0/1416965060 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6d0065c10 msgr2=0x7ff6d00680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:40:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.462+0000 7ff6e8803640 1 --2- 172.21.15.32:0/1416965060 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6d0065c10 0x7ff6d00680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.462+0000 7ff6e8803640 1 -- 172.21.15.32:0/1416965060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6e40cf3e0 msgr2=0x7ff6e415bf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.462+0000 7ff6e8803640 1 --2- 172.21.15.32:0/1416965060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6e40cf3e0 0x7ff6e415bf70 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7ff6d400c3e0 tx=0x7ff6d400c8a0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.466+0000 7ff6e8803640 1 -- 172.21.15.32:0/1416965060 shutdown_connections 2023-12-20T04:40:44.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.466+0000 7ff6e8803640 1 --2- 172.21.15.32:0/1416965060 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6d0065c10 0x7ff6d00680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.466+0000 7ff6e8803640 1 --2- 172.21.15.32:0/1416965060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6e415ccf0 0x7ff6e40c7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.466+0000 7ff6e8803640 1 --2- 172.21.15.32:0/1416965060 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6e40d1580 0x7ff6e40b3060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.466+0000 7ff6e8803640 1 --2- 172.21.15.32:0/1416965060 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6e40cf3e0 0x7ff6e415bf70 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.466+0000 7ff6e8803640 1 -- 172.21.15.32:0/1416965060 >> 172.21.15.32:0/1416965060 conn(0x7ff6e40658e0 msgr2=0x7ff6e40caa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:44.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.466+0000 7ff6e8803640 1 -- 172.21.15.32:0/1416965060 shutdown_connections 2023-12-20T04:40:44.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.466+0000 7ff6e8803640 1 -- 172.21.15.32:0/1416965060 wait complete. 2023-12-20T04:40:44.468 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:44.470+0000 7fcb0e255180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:40:44.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:40:44.563 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:44.566+0000 7fcb0e255180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:40:44.608 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:44.610+0000 7f6d88262180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:40:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc138895640 1 Processor -- start 2023-12-20T04:40:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc138895640 1 -- start start 2023-12-20T04:40:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc138895640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1340d3300 0x7fc1340d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc138895640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1340d2650 con 0x7fc13415ce60 2023-12-20T04:40:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc138895640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1340d1a40 con 0x7fc1340d1300 2023-12-20T04:40:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc138895640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1340d1d20 con 0x7fc1340d3300 2023-12-20T04:40:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc131d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1340d3300 0x7fc1340d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc131d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1340d3300 0x7fc1340d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48608/0 (socket says 172.21.15.32:48608) 2023-12-20T04:40:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc131d9b640 1 -- 172.21.15.32:0/2181873536 learned_addr learned my addr 172.21.15.32:0/2181873536 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc13259c640 1 --1- 172.21.15.32:0/2181873536 >> v1:172.21.15.32:6789/0 conn(0x7fc13415ce60 0x7fc1340d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38832/0 (socket says 172.21.15.32:38832) 2023-12-20T04:40:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc131d9b640 1 -- 172.21.15.32:0/2181873536 >> v1:172.21.15.39:6789/0 conn(0x7fc1340d1300 legacy=0x7fc1340b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc131d9b640 1 -- 172.21.15.32:0/2181873536 >> v1:172.21.15.32:6789/0 conn(0x7fc13415ce60 legacy=0x7fc1340d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc13159a640 1 -- 172.21.15.32:0/2181873536 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3427103041 0 0) 0x7fc1340d2650 con 0x7fc13415ce60 2023-12-20T04:40:44.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.638+0000 7fc131d9b640 1 -- 172.21.15.32:0/2181873536 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1340d2000 con 0x7fc1340d3300 2023-12-20T04:40:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc131d9b640 1 --2- 172.21.15.32:0/2181873536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1340d3300 0x7fc1340d0dc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc128009870 tx=0x7fc12801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc87eaac8c63f088 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc13159a640 1 -- 172.21.15.32:0/2181873536 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280ac630 con 0x7fc1340d3300 2023-12-20T04:40:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc13159a640 1 -- 172.21.15.32:0/2181873536 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1280ac790 con 0x7fc1340d3300 2023-12-20T04:40:44.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc13159a640 1 -- 172.21.15.32:0/2181873536 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280acb20 con 0x7fc1340d3300 2023-12-20T04:40:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc138895640 1 -- 172.21.15.32:0/2181873536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1340d3300 msgr2=0x7fc1340d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc138895640 1 --2- 172.21.15.32:0/2181873536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1340d3300 0x7fc1340d0dc0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7fc128009870 tx=0x7fc12801c540 comp rx=0 tx=0).stop 2023-12-20T04:40:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc138895640 1 -- 172.21.15.32:0/2181873536 shutdown_connections 2023-12-20T04:40:44.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc138895640 1 --2- 172.21.15.32:0/2181873536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1340d3300 0x7fc1340d0dc0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc138895640 1 -- 172.21.15.32:0/2181873536 >> 172.21.15.32:0/2181873536 conn(0x7fc13406b5e0 msgr2=0x7fc13406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:44.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc138895640 1 -- 172.21.15.32:0/2181873536 shutdown_connections 2023-12-20T04:40:44.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc138895640 1 -- 172.21.15.32:0/2181873536 wait complete. 2023-12-20T04:40:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc138895640 1 Processor -- start 2023-12-20T04:40:44.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.646+0000 7fc138895640 1 -- start start 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc138895640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1340d1300 0x7fc1340c8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc138895640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1340d3300 0x7fc13415d830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc138895640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc13415ce60 0x7fc1340b2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc138895640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340b70e0 con 0x7fc1340d3300 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc138895640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340b72f0 con 0x7fc13415ce60 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc138895640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1340b75d0 con 0x7fc1340d1300 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc131d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1340d3300 0x7fc13415d830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc13259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1340d1300 0x7fc1340c8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc132d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc13415ce60 0x7fc1340b2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc131d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1340d3300 0x7fc13415d830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48612/0 (socket says 172.21.15.32:48612) 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc13259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1340d1300 0x7fc1340c8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:47598/0 (socket says 172.21.15.32:47598) 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc132d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc13415ce60 0x7fc1340b2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:44458/0 (socket says 172.21.15.32:44458) 2023-12-20T04:40:44.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc131d9b640 1 -- 172.21.15.32:0/1292877948 learned_addr learned my addr 172.21.15.32:0/1292877948 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc131d9b640 1 -- 172.21.15.32:0/1292877948 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1340d1300 msgr2=0x7fc1340c8f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc131d9b640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1340d1300 0x7fc1340c8f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc131d9b640 1 -- 172.21.15.32:0/1292877948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc13415ce60 msgr2=0x7fc1340b2dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc131d9b640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc13415ce60 0x7fc1340b2dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc131d9b640 1 -- 172.21.15.32:0/1292877948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1280094d0 con 0x7fc1340d3300 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc132d9d640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc13415ce60 0x7fc1340b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc13259c640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1340d1300 0x7fc1340c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc131d9b640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1340d3300 0x7fc13415d830 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc1280af040 tx=0x7fc1280192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc122ffd640 1 -- 172.21.15.32:0/1292877948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280b5850 con 0x7fc1340d3300 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc122ffd640 1 -- 172.21.15.32:0/1292877948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1280b59d0 con 0x7fc1340d3300 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc122ffd640 1 -- 172.21.15.32:0/1292877948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1280b5d80 con 0x7fc1340d3300 2023-12-20T04:40:44.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc138895640 1 -- 172.21.15.32:0/1292877948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc13415e230 con 0x7fc1340d3300 2023-12-20T04:40:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc138895640 1 -- 172.21.15.32:0/1292877948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc13415b8b0 con 0x7fc1340d3300 2023-12-20T04:40:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.650+0000 7fc138895640 1 -- 172.21.15.32:0/1292877948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc13404e000 con 0x7fc1340d3300 2023-12-20T04:40:44.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.654+0000 7fc122ffd640 1 -- 172.21.15.32:0/1292877948 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7fc1280b3070 con 0x7fc1340d3300 2023-12-20T04:40:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.654+0000 7fc122ffd640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc114065c10 0x7fc1140680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.654+0000 7fc122ffd640 1 -- 172.21.15.32:0/1292877948 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6551+0+0 (secure 0 0 0) 0x7fc12811d8f0 con 0x7fc1340d3300 2023-12-20T04:40:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.654+0000 7fc13259c640 1 -- 172.21.15.32:0/1292877948 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc114065c10 msgr2=0x7fc1140680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:40:44.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.654+0000 7fc13259c640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc114065c10 0x7fc1140680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:40:44.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.658+0000 7fc122ffd640 1 -- 172.21.15.32:0/1292877948 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc1280ead10 con 0x7fc1340d3300 2023-12-20T04:40:44.696 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:44.698+0000 7f6d88262180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:40:44.728 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:44.730+0000 7fcb0e255180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:40:44.817 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:44.818+0000 7fcb0e255180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:40:44.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.822+0000 7fc138895640 1 -- 172.21.15.32:0/1292877948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc13415e6e0 con 0x7fc1340d3300 2023-12-20T04:40:44.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.822+0000 7fc122ffd640 1 -- 172.21.15.32:0/1292877948 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v47) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc1280ee330 con 0x7fc1340d3300 2023-12-20T04:40:44.822 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 -- 172.21.15.32:0/1292877948 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc114065c10 msgr2=0x7fc1140680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc114065c10 0x7fc1140680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 -- 172.21.15.32:0/1292877948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1340d3300 msgr2=0x7fc13415d830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1340d3300 0x7fc13415d830 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7fc1280af040 tx=0x7fc1280192e0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 -- 172.21.15.32:0/1292877948 shutdown_connections 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc114065c10 0x7fc1140680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc13415ce60 0x7fc1340b2dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1340d3300 0x7fc13415d830 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 --2- 172.21.15.32:0/1292877948 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1340d1300 0x7fc1340c8f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 -- 172.21.15.32:0/1292877948 >> 172.21.15.32:0/1292877948 conn(0x7fc13406b5e0 msgr2=0x7fc1340cbd40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 -- 172.21.15.32:0/1292877948 shutdown_connections 2023-12-20T04:40:44.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.826+0000 7fc138895640 1 -- 172.21.15.32:0/1292877948 wait complete. 2023-12-20T04:40:44.844 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:40:44.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d4796640 1 Processor -- start 2023-12-20T04:40:44.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d4796640 1 -- start start 2023-12-20T04:40:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d4796640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f39cc0b0cb0 0x7f39cc0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d4796640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f39cc070230 0x7f39cc0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d4796640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f39cc15b6a0 con 0x7f39cc15abb0 2023-12-20T04:40:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d4796640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f39cc06e160 con 0x7f39cc070230 2023-12-20T04:40:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d4796640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc070be0 con 0x7f39cc0b0cb0 2023-12-20T04:40:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d2532640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f39cc15abb0 0x7f39cc15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38834/0 (socket says 172.21.15.32:38834) 2023-12-20T04:40:44.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d2532640 1 -- 172.21.15.32:0/4073645446 learned_addr learned my addr 172.21.15.32:0/4073645446 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d1d31640 1 --2- 172.21.15.32:0/4073645446 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f39cc0b0cb0 0x7f39cc0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d2d33640 1 --2- 172.21.15.32:0/4073645446 >> v2:172.21.15.39:3300/0 conn(0x7f39cc070230 0x7f39cc0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d1530640 1 -- 172.21.15.32:0/4073645446 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1583422320 0 0) 0x7f39cc15b6a0 con 0x7f39cc15abb0 2023-12-20T04:40:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d1530640 1 -- 172.21.15.32:0/4073645446 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f39b8003530 con 0x7f39cc15abb0 2023-12-20T04:40:44.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d1d31640 1 -- 172.21.15.32:0/4073645446 >> v2:172.21.15.39:3300/0 conn(0x7f39cc070230 msgr2=0x7f39cc0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d1d31640 1 --2- 172.21.15.32:0/4073645446 >> v2:172.21.15.39:3300/0 conn(0x7f39cc070230 0x7f39cc0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.994+0000 7f39d1d31640 1 -- 172.21.15.32:0/4073645446 >> v1:172.21.15.32:6789/0 conn(0x7f39cc15abb0 legacy=0x7f39cc15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d1d31640 1 -- 172.21.15.32:0/4073645446 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39cc06d8c0 con 0x7f39cc0b0cb0 2023-12-20T04:40:44.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d1d31640 1 --2- 172.21.15.32:0/4073645446 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f39cc0b0cb0 0x7f39cc0caaf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0097c0 tx=0x7f39bc01c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fd0d26406063e5d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d1530640 1 -- 172.21.15.32:0/4073645446 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0ab680 con 0x7f39cc0b0cb0 2023-12-20T04:40:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d1530640 1 -- 172.21.15.32:0/4073645446 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39bc0ab7e0 con 0x7f39cc0b0cb0 2023-12-20T04:40:44.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d1530640 1 -- 172.21.15.32:0/4073645446 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0abb70 con 0x7f39cc0b0cb0 2023-12-20T04:40:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d4796640 1 -- 172.21.15.32:0/4073645446 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f39cc0b0cb0 msgr2=0x7f39cc0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d4796640 1 --2- 172.21.15.32:0/4073645446 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f39cc0b0cb0 0x7f39cc0caaf0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f39bc0097c0 tx=0x7f39bc01c480 comp rx=0 tx=0).stop 2023-12-20T04:40:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d4796640 1 -- 172.21.15.32:0/4073645446 shutdown_connections 2023-12-20T04:40:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d4796640 1 --2- 172.21.15.32:0/4073645446 >> v2:172.21.15.39:3300/0 conn(0x7f39cc070230 0x7f39cc0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d4796640 1 --2- 172.21.15.32:0/4073645446 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f39cc0b0cb0 0x7f39cc0caaf0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d4796640 1 -- 172.21.15.32:0/4073645446 >> 172.21.15.32:0/4073645446 conn(0x7f39cc06a510 msgr2=0x7f39cc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:44.998+0000 7f39d4796640 1 -- 172.21.15.32:0/4073645446 shutdown_connections 2023-12-20T04:40:44.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d4796640 1 -- 172.21.15.32:0/4073645446 wait complete. 2023-12-20T04:40:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d4796640 1 Processor -- start 2023-12-20T04:40:45.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d4796640 1 -- start start 2023-12-20T04:40:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d4796640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39cc070230 0x7f39cc1552e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d4796640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc0b0cb0 0x7f39cc159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d4796640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f39cc15abb0 0x7f39cc1495c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d4796640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc14d2c0 con 0x7f39cc070230 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d4796640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc154e50 con 0x7f39cc0b0cb0 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d4796640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f39cc14cd00 con 0x7f39cc15abb0 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d2532640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39cc070230 0x7f39cc1552e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d2532640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39cc070230 0x7f39cc1552e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48618/0 (socket says 172.21.15.32:48618) 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d2532640 1 -- 172.21.15.32:0/3625633694 learned_addr learned my addr 172.21.15.32:0/3625633694 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d1d31640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc0b0cb0 0x7f39cc159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d2d33640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f39cc15abb0 0x7f39cc1495c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d2532640 1 -- 172.21.15.32:0/3625633694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f39cc15abb0 msgr2=0x7f39cc1495c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d2532640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f39cc15abb0 0x7f39cc1495c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d2532640 1 -- 172.21.15.32:0/3625633694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc0b0cb0 msgr2=0x7f39cc159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d2532640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc0b0cb0 0x7f39cc159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d2532640 1 -- 172.21.15.32:0/3625633694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f39bc009420 con 0x7f39cc070230 2023-12-20T04:40:45.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39d2532640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39cc070230 0x7f39cc1552e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c420 tx=0x7f39c800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.002+0000 7f39c2ffd640 1 -- 172.21.15.32:0/3625633694 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c8005cf0 con 0x7f39cc070230 2023-12-20T04:40:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.006+0000 7f39c2ffd640 1 -- 172.21.15.32:0/3625633694 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39c8005e50 con 0x7f39cc070230 2023-12-20T04:40:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.006+0000 7f39c2ffd640 1 -- 172.21.15.32:0/3625633694 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39c80072d0 con 0x7f39cc070230 2023-12-20T04:40:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.006+0000 7f39d4796640 1 -- 172.21.15.32:0/3625633694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f39cc14cfe0 con 0x7f39cc070230 2023-12-20T04:40:45.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.006+0000 7f39d4796640 1 -- 172.21.15.32:0/3625633694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f39cc155820 con 0x7f39cc070230 2023-12-20T04:40:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.006+0000 7f39d4796640 1 -- 172.21.15.32:0/3625633694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f39cc154940 con 0x7f39cc070230 2023-12-20T04:40:45.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.006+0000 7f39c2ffd640 1 -- 172.21.15.32:0/3625633694 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 14) v1 ==== 85399+0+0 (secure 0 0 0) 0x7f39c8007430 con 0x7f39cc070230 2023-12-20T04:40:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.006+0000 7f39c2ffd640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f39b0065c10 0x7f39b00680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.006+0000 7f39c2ffd640 1 -- 172.21.15.32:0/3625633694 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(47..47 src has 1..47) v4 ==== 6551+0+0 (secure 0 0 0) 0x7f39c8082c40 con 0x7f39cc070230 2023-12-20T04:40:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.006+0000 7f39d1d31640 1 -- 172.21.15.32:0/3625633694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f39b0065c10 msgr2=0x7f39b00680d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:40:45.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.006+0000 7f39d1d31640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f39b0065c10 0x7f39b00680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:40:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.014+0000 7f39c2ffd640 1 -- 172.21.15.32:0/3625633694 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f39cc154940 con 0x7f39cc070230 2023-12-20T04:40:45.031 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:45.034+0000 7f6d88262180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:40:45.095 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:40:45.098+0000 7f6d88262180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:40:45.168 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.170+0000 7fcb0e255180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:40:45.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.174+0000 7f39d4796640 1 -- 172.21.15.32:0/3625633694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f39cc154940 con 0x7f39cc070230 2023-12-20T04:40:45.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.174+0000 7f39c2ffd640 1 -- 172.21.15.32:0/3625633694 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v47) v1 ==== 100+0+11 (secure 0 0 0) 0x7f39cc154940 con 0x7f39cc070230 2023-12-20T04:40:45.173 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:40:45.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 -- 172.21.15.32:0/3625633694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f39b0065c10 msgr2=0x7f39b00680d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:40:45.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f39b0065c10 0x7f39b00680d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 -- 172.21.15.32:0/3625633694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39cc070230 msgr2=0x7f39cc1552e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39cc070230 0x7f39cc1552e0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f39c800c420 tx=0x7f39c800c8e0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 -- 172.21.15.32:0/3625633694 shutdown_connections 2023-12-20T04:40:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f39b0065c10 0x7f39b00680d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f39cc15abb0 0x7f39cc1495c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f39cc0b0cb0 0x7f39cc159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 --2- 172.21.15.32:0/3625633694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f39cc070230 0x7f39cc1552e0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 -- 172.21.15.32:0/3625633694 >> 172.21.15.32:0/3625633694 conn(0x7f39cc06a510 msgr2=0x7f39cc156af0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 -- 172.21.15.32:0/3625633694 shutdown_connections 2023-12-20T04:40:45.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.178+0000 7f39d4796640 1 -- 172.21.15.32:0/3625633694 wait complete. 2023-12-20T04:40:45.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:40:45.234 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.234+0000 7fcb0e255180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:40:45.308 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.310+0000 7fca999d3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:40:45.309 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.310+0000 7fca999d3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:40:45.309 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.310+0000 7fca999d3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:40:45.309 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.310+0000 7fca999d3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:40:45.309 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.310+0000 7fca999d3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:40:45.309 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.310+0000 7fca969cd640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:40:45.309 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.310+0000 7fca969cd640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:40:45.309 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.310+0000 7fca969cd640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:40:45.309 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.310+0000 7fca969cd640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:40:45.310 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:40:45.310+0000 7fca969cd640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:40:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.354+0000 7f42bc82c640 1 Processor -- start 2023-12-20T04:40:45.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42bc82c640 1 -- start start 2023-12-20T04:40:45.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42bc82c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f42b8159ab0 0x7f42b80714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42bc82c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42b80b3e90 0x7f42b8071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42bc82c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42b8071f40 con 0x7f42b80cd2e0 2023-12-20T04:40:45.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42bc82c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b80ca0b0 con 0x7f42b80b3e90 2023-12-20T04:40:45.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42bc82c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42b80ad7e0 con 0x7f42b8159ab0 2023-12-20T04:40:45.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42b759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42b80b3e90 0x7f42b8071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42b759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42b80b3e90 0x7f42b8071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48734/0 (socket says 172.21.15.32:48734) 2023-12-20T04:40:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42b759e640 1 -- 172.21.15.32:0/2128062172 learned_addr learned my addr 172.21.15.32:0/2128062172 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42b6d9d640 1 --2- 172.21.15.32:0/2128062172 >> v2:172.21.15.39:3300/0 conn(0x7f42b8159ab0 0x7f42b80714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42b759e640 1 -- 172.21.15.32:0/2128062172 >> v2:172.21.15.39:3300/0 conn(0x7f42b8159ab0 msgr2=0x7f42b80714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42b759e640 1 --2- 172.21.15.32:0/2128062172 >> v2:172.21.15.39:3300/0 conn(0x7f42b8159ab0 0x7f42b80714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42b759e640 1 -- 172.21.15.32:0/2128062172 >> v1:172.21.15.39:6789/0 conn(0x7f42b80cd2e0 legacy=0x7f42b80cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42b759e640 1 -- 172.21.15.32:0/2128062172 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b80ada80 con 0x7f42b80b3e90 2023-12-20T04:40:45.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.358+0000 7f42b5d9b640 1 -- 172.21.15.32:0/2128062172 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1962718995 0 0) 0x7f42b8071f40 con 0x7f42b80cd2e0 2023-12-20T04:40:45.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42b759e640 1 --2- 172.21.15.32:0/2128062172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80b3e90 0x7f42b8071a00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f42b806e110 tx=0x7f42b001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6a71aad4ef74733 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:45.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42b5d9b640 1 -- 172.21.15.32:0/2128062172 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b00a5780 con 0x7f42b80b3e90 2023-12-20T04:40:45.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42b5d9b640 1 -- 172.21.15.32:0/2128062172 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b00a58e0 con 0x7f42b80b3e90 2023-12-20T04:40:45.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42b5d9b640 1 -- 172.21.15.32:0/2128062172 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b00a5c70 con 0x7f42b80b3e90 2023-12-20T04:40:45.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42bc82c640 1 -- 172.21.15.32:0/2128062172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80b3e90 msgr2=0x7f42b8071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42bc82c640 1 --2- 172.21.15.32:0/2128062172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80b3e90 0x7f42b8071a00 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f42b806e110 tx=0x7f42b001c9a0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42bc82c640 1 -- 172.21.15.32:0/2128062172 shutdown_connections 2023-12-20T04:40:45.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42bc82c640 1 --2- 172.21.15.32:0/2128062172 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80b3e90 0x7f42b8071a00 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42bc82c640 1 --2- 172.21.15.32:0/2128062172 >> v2:172.21.15.39:3300/0 conn(0x7f42b8159ab0 0x7f42b80714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42bc82c640 1 -- 172.21.15.32:0/2128062172 >> 172.21.15.32:0/2128062172 conn(0x7f42b806c060 msgr2=0x7f42b806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:45.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42bc82c640 1 -- 172.21.15.32:0/2128062172 shutdown_connections 2023-12-20T04:40:45.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42bc82c640 1 -- 172.21.15.32:0/2128062172 wait complete. 2023-12-20T04:40:45.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42bc82c640 1 Processor -- start 2023-12-20T04:40:45.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.362+0000 7f42bc82c640 1 -- start start 2023-12-20T04:40:45.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42bc82c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80b3e90 0x7f42b8151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42bc82c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b80cd2e0 0x7f42b815be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42bc82c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42b8159ab0 0x7f42b80ac2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42bc82c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b81f5b70 con 0x7f42b80b3e90 2023-12-20T04:40:45.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42bc82c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42b815c380 con 0x7f42b80cd2e0 2023-12-20T04:40:45.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42bc82c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42b815b080 con 0x7f42b8159ab0 2023-12-20T04:40:45.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42b6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80b3e90 0x7f42b8151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42b6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80b3e90 0x7f42b8151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48746/0 (socket says 172.21.15.32:48746) 2023-12-20T04:40:45.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42b6d9d640 1 -- 172.21.15.32:0/3942532936 learned_addr learned my addr 172.21.15.32:0/3942532936 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:45.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42b759e640 1 --2- 172.21.15.32:0/3942532936 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42b8159ab0 0x7f42b80ac2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42b659c640 1 --2- 172.21.15.32:0/3942532936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b80cd2e0 0x7f42b815be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42b6d9d640 1 -- 172.21.15.32:0/3942532936 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42b8159ab0 msgr2=0x7f42b80ac2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42b6d9d640 1 --2- 172.21.15.32:0/3942532936 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42b8159ab0 0x7f42b80ac2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42b6d9d640 1 -- 172.21.15.32:0/3942532936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b80cd2e0 msgr2=0x7f42b815be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42b6d9d640 1 --2- 172.21.15.32:0/3942532936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b80cd2e0 0x7f42b815be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42b6d9d640 1 -- 172.21.15.32:0/3942532936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b0009b30 con 0x7f42b80b3e90 2023-12-20T04:40:45.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42b6d9d640 1 --2- 172.21.15.32:0/3942532936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80b3e90 0x7f42b8151900 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f42a800e870 tx=0x7f42a800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:45.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f429f7fe640 1 -- 172.21.15.32:0/3942532936 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a8009510 con 0x7f42b80b3e90 2023-12-20T04:40:45.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f429f7fe640 1 -- 172.21.15.32:0/3942532936 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42a800eec0 con 0x7f42b80b3e90 2023-12-20T04:40:45.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f429f7fe640 1 -- 172.21.15.32:0/3942532936 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a800cb40 con 0x7f42b80b3e90 2023-12-20T04:40:45.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42bc82c640 1 -- 172.21.15.32:0/3942532936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42b815b360 con 0x7f42b80b3e90 2023-12-20T04:40:45.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42bc82c640 1 -- 172.21.15.32:0/3942532936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42b815b920 con 0x7f42b80b3e90 2023-12-20T04:40:45.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.366+0000 7f42bc82c640 1 -- 172.21.15.32:0/3942532936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42b8074100 con 0x7f42b80b3e90 2023-12-20T04:40:45.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.370+0000 7f429f7fe640 1 -- 172.21.15.32:0/3942532936 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 15) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f42a800cca0 con 0x7f42b80b3e90 2023-12-20T04:40:45.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.370+0000 7f429f7fe640 1 -- 172.21.15.32:0/3942532936 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f42a8083200 con 0x7f42b80b3e90 2023-12-20T04:40:45.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.374+0000 7f429f7fe640 1 -- 172.21.15.32:0/3942532936 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f42a8050b80 con 0x7f42b80b3e90 2023-12-20T04:40:45.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.562+0000 7f42bc82c640 1 -- 172.21.15.32:0/3942532936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42b8159e90 con 0x7f42b80b3e90 2023-12-20T04:40:45.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.566+0000 7f429f7fe640 1 -- 172.21.15.32:0/3942532936 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f42a80541a0 con 0x7f42b80b3e90 2023-12-20T04:40:45.563 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:45.564 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2541430692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2541430692},{"type":"v1","addr":"172.21.15.39:6835","nonce":2541430692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1379533068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1379533068},{"type":"v1","addr":"172.21.15.32:6837","nonce":1379533068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4021567802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4021567802},{"type":"v1","addr":"172.21.15.39:6834","nonce":4021567802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:29:30.610189+0000","modified":"2023-12-20T04:29:33.661905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3391762708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3391762708},{"type":"v1","addr":"172.21.15.32:6836","nonce":3391762708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-20T04:40:45.564 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2023-12-20T04:40:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.566+0000 7f42bc82c640 1 -- 172.21.15.32:0/3942532936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80b3e90 msgr2=0x7f42b8151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.566+0000 7f42bc82c640 1 --2- 172.21.15.32:0/3942532936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80b3e90 0x7f42b8151900 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f42a800e870 tx=0x7f42a800ec70 comp rx=0 tx=0).stop 2023-12-20T04:40:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.566+0000 7f42bc82c640 1 -- 172.21.15.32:0/3942532936 shutdown_connections 2023-12-20T04:40:45.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.566+0000 7f42bc82c640 1 --2- 172.21.15.32:0/3942532936 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42b8159ab0 0x7f42b80ac2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.566+0000 7f42bc82c640 1 --2- 172.21.15.32:0/3942532936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42b80cd2e0 0x7f42b815be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.566+0000 7f42bc82c640 1 --2- 172.21.15.32:0/3942532936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42b80b3e90 0x7f42b8151900 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.566+0000 7f42bc82c640 1 -- 172.21.15.32:0/3942532936 >> 172.21.15.32:0/3942532936 conn(0x7f42b806c060 msgr2=0x7f42b80ad040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:45.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.566+0000 7f42bc82c640 1 -- 172.21.15.32:0/3942532936 shutdown_connections 2023-12-20T04:40:45.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.566+0000 7f42bc82c640 1 -- 172.21.15.32:0/3942532936 wait complete. 2023-12-20T04:40:45.585 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:40:45.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.770+0000 7fb8d448c640 1 Processor -- start 2023-12-20T04:40:45.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.770+0000 7fb8d448c640 1 -- start start 2023-12-20T04:40:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d448c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb8cc0aece0 0x7fb8cc0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d448c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8cc0b0b30 0x7fb8cc0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d448c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8cc0c98c0 con 0x7fb8cc1599e0 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d448c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8cc0c9ba0 con 0x7fb8cc0b0b30 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d448c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8cc0c9e80 con 0x7fb8cc0aece0 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d2a29640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8cc0b0b30 0x7fb8cc0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d2a29640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8cc0b0b30 0x7fb8cc0701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48748/0 (socket says 172.21.15.32:48748) 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d2a29640 1 -- 172.21.15.32:0/1742073366 learned_addr learned my addr 172.21.15.32:0/1742073366 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d1a27640 1 --2- 172.21.15.32:0/1742073366 >> v2:172.21.15.39:3300/0 conn(0x7fb8cc0aece0 0x7fb8cc0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d2a29640 1 -- 172.21.15.32:0/1742073366 >> v2:172.21.15.39:3300/0 conn(0x7fb8cc0aece0 msgr2=0x7fb8cc0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d2a29640 1 --2- 172.21.15.32:0/1742073366 >> v2:172.21.15.39:3300/0 conn(0x7fb8cc0aece0 0x7fb8cc0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d2a29640 1 -- 172.21.15.32:0/1742073366 >> v1:172.21.15.39:6789/0 conn(0x7fb8cc1599e0 legacy=0x7fb8cc0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d2a29640 1 -- 172.21.15.32:0/1742073366 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8cc0ca160 con 0x7fb8cc0b0b30 2023-12-20T04:40:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d2a29640 1 --2- 172.21.15.32:0/1742073366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8cc0b0b30 0x7fb8cc0701c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0cf520 tx=0x7fb8c401f630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d03ec28a4fc00bd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.774+0000 7fb8d1226640 1 -- 172.21.15.32:0/1742073366 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c400b2d0 con 0x7fb8cc0b0b30 2023-12-20T04:40:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d1226640 1 -- 172.21.15.32:0/1742073366 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c400b430 con 0x7fb8cc0b0b30 2023-12-20T04:40:45.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d1226640 1 -- 172.21.15.32:0/1742073366 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40a7bd0 con 0x7fb8cc0b0b30 2023-12-20T04:40:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 -- 172.21.15.32:0/1742073366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8cc0b0b30 msgr2=0x7fb8cc0701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 --2- 172.21.15.32:0/1742073366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8cc0b0b30 0x7fb8cc0701c0 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc0cf520 tx=0x7fb8c401f630 comp rx=0 tx=0).stop 2023-12-20T04:40:45.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 -- 172.21.15.32:0/1742073366 shutdown_connections 2023-12-20T04:40:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 --2- 172.21.15.32:0/1742073366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8cc0b0b30 0x7fb8cc0701c0 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 --2- 172.21.15.32:0/1742073366 >> v2:172.21.15.39:3300/0 conn(0x7fb8cc0aece0 0x7fb8cc0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 -- 172.21.15.32:0/1742073366 >> 172.21.15.32:0/1742073366 conn(0x7fb8cc06a320 msgr2=0x7fb8cc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 -- 172.21.15.32:0/1742073366 shutdown_connections 2023-12-20T04:40:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 -- 172.21.15.32:0/1742073366 wait complete. 2023-12-20T04:40:45.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 Processor -- start 2023-12-20T04:40:45.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 -- start start 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8cc0aece0 0x7fb8cc146e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8cc0b0b30 0x7fb8cc14aea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8cc1599e0 0x7fb8cc070f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc1474c0 con 0x7fb8cc1599e0 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc14b3e0 con 0x7fb8cc0b0b30 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d448c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8cc14f640 con 0x7fb8cc0aece0 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d2a29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8cc1599e0 0x7fb8cc070f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d2228640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8cc0aece0 0x7fb8cc146e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d1a27640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8cc0b0b30 0x7fb8cc14aea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d2a29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8cc1599e0 0x7fb8cc070f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48764/0 (socket says 172.21.15.32:48764) 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.778+0000 7fb8d2a29640 1 -- 172.21.15.32:0/2234180850 learned_addr learned my addr 172.21.15.32:0/2234180850 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:45.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8d2a29640 1 -- 172.21.15.32:0/2234180850 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8cc0aece0 msgr2=0x7fb8cc146e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8d2a29640 1 --2- 172.21.15.32:0/2234180850 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8cc0aece0 0x7fb8cc146e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8d2a29640 1 -- 172.21.15.32:0/2234180850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8cc0b0b30 msgr2=0x7fb8cc14aea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8d2a29640 1 --2- 172.21.15.32:0/2234180850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8cc0b0b30 0x7fb8cc14aea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8d2a29640 1 -- 172.21.15.32:0/2234180850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c400bd40 con 0x7fb8cc1599e0 2023-12-20T04:40:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8d2a29640 1 --2- 172.21.15.32:0/2234180850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8cc1599e0 0x7fb8cc070f10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb8c401b7e0 tx=0x7fb8c401b970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8c2ffd640 1 -- 172.21.15.32:0/2234180850 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40bd070 con 0x7fb8cc1599e0 2023-12-20T04:40:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8c2ffd640 1 -- 172.21.15.32:0/2234180850 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c4009600 con 0x7fb8cc1599e0 2023-12-20T04:40:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8c2ffd640 1 -- 172.21.15.32:0/2234180850 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40099b0 con 0x7fb8cc1599e0 2023-12-20T04:40:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8d448c640 1 -- 172.21.15.32:0/2234180850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8cc14f920 con 0x7fb8cc1599e0 2023-12-20T04:40:45.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8d448c640 1 -- 172.21.15.32:0/2234180850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8cc1479f0 con 0x7fb8cc1599e0 2023-12-20T04:40:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8d448c640 1 -- 172.21.15.32:0/2234180850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8cc147be0 con 0x7fb8cc1599e0 2023-12-20T04:40:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8c2ffd640 1 -- 172.21.15.32:0/2234180850 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 15) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fb8c4009b10 con 0x7fb8cc1599e0 2023-12-20T04:40:45.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.782+0000 7fb8c2ffd640 1 -- 172.21.15.32:0/2234180850 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6808+0+0 (secure 0 0 0) 0x7fb8c411e1a0 con 0x7fb8cc1599e0 2023-12-20T04:40:45.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.786+0000 7fb8c2ffd640 1 -- 172.21.15.32:0/2234180850 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb8c40ebc40 con 0x7fb8cc1599e0 2023-12-20T04:40:45.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.946+0000 7fb8d448c640 1 -- 172.21.15.32:0/2234180850 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8cc04e000 con 0x7fb8cc1599e0 2023-12-20T04:40:45.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.946+0000 7fb8c2ffd640 1 -- 172.21.15.32:0/2234180850 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+17691 (secure 0 0 0) 0x7fb8c40ef260 con 0x7fb8cc1599e0 2023-12-20T04:40:45.945 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:45.945 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":48,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:40:45.241441+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:29:28.685343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-20T04:29:29.647655+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1043195128":"2023-12-21T04:40:45.241408+0000","172.21.15.32:0/158547636":"2023-12-21T04:40:45.241408+0000","172.21.15.32:0/2331287763":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/1794234007":"2023-12-21T04:40:45.241408+0000","172.21.15.32:0/2945853272":"2023-12-21T04:40:45.241408+0000","172.21.15.32:6833/38009":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/827611945":"2023-12-21T04:40:45.241408+0000","172.21.15.32:6832/38009":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/728956527":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/3777108374":"2023-12-21T04:40:45.241408+0000","172.21.15.32:0/4147638565":"2023-12-21T04:40:45.241408+0000","172.21.15.32:0/856257861":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/2782983206":"2023-12-21T04:29:47.726855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":57,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":68,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":2},{"begin":96,"length":1},{"begin":100,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:40:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.946+0000 7fb8d448c640 1 -- 172.21.15.32:0/2234180850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8cc1599e0 msgr2=0x7fb8cc070f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.946+0000 7fb8d448c640 1 --2- 172.21.15.32:0/2234180850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8cc1599e0 0x7fb8cc070f10 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fb8c401b7e0 tx=0x7fb8c401b970 comp rx=0 tx=0).stop 2023-12-20T04:40:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.946+0000 7fb8d448c640 1 -- 172.21.15.32:0/2234180850 shutdown_connections 2023-12-20T04:40:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.946+0000 7fb8d448c640 1 --2- 172.21.15.32:0/2234180850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8cc1599e0 0x7fb8cc070f10 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.946+0000 7fb8d448c640 1 --2- 172.21.15.32:0/2234180850 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8cc0b0b30 0x7fb8cc14aea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.946+0000 7fb8d448c640 1 --2- 172.21.15.32:0/2234180850 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8cc0aece0 0x7fb8cc146e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.946+0000 7fb8d448c640 1 -- 172.21.15.32:0/2234180850 >> 172.21.15.32:0/2234180850 conn(0x7fb8cc06a320 msgr2=0x7fb8cc0b39b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.950+0000 7fb8d448c640 1 -- 172.21.15.32:0/2234180850 shutdown_connections 2023-12-20T04:40:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:45.950+0000 7fb8d448c640 1 -- 172.21.15.32:0/2234180850 wait complete. 2023-12-20T04:40:45.966 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T04:40:45.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:40:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.110+0000 7fc869066640 1 Processor -- start 2023-12-20T04:40:46.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.110+0000 7fc869066640 1 -- start start 2023-12-20T04:40:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc869066640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc86415ce30 0x7fc864161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc869066640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc86406deb0 0x7fc8640cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc869066640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc864071920 con 0x7fc864070950 2023-12-20T04:40:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc869066640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc864071080 con 0x7fc86406deb0 2023-12-20T04:40:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc869066640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc864071360 con 0x7fc86415ce30 2023-12-20T04:40:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc86359e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc86406deb0 0x7fc8640cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc86359e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc86406deb0 0x7fc8640cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48768/0 (socket says 172.21.15.32:48768) 2023-12-20T04:40:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc86359e640 1 -- 172.21.15.32:0/1871386065 learned_addr learned my addr 172.21.15.32:0/1871386065 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc862d9d640 1 --2- 172.21.15.32:0/1871386065 >> v2:172.21.15.39:3300/0 conn(0x7fc86415ce30 0x7fc864161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc86359e640 1 -- 172.21.15.32:0/1871386065 >> v2:172.21.15.39:3300/0 conn(0x7fc86415ce30 msgr2=0x7fc864161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc86359e640 1 --2- 172.21.15.32:0/1871386065 >> v2:172.21.15.39:3300/0 conn(0x7fc86415ce30 0x7fc864161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc86359e640 1 -- 172.21.15.32:0/1871386065 >> v1:172.21.15.39:6789/0 conn(0x7fc864070950 legacy=0x7fc86406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc86359e640 1 -- 172.21.15.32:0/1871386065 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc864071640 con 0x7fc86406deb0 2023-12-20T04:40:46.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.114+0000 7fc862d9d640 1 --2- 172.21.15.32:0/1871386065 >> v2:172.21.15.39:3300/0 conn(0x7fc86415ce30 0x7fc864161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc86359e640 1 --2- 172.21.15.32:0/1871386065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86406deb0 0x7fc8640cd520 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc8640cef30 tx=0x7fc85c01c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4b68055806f8f5c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc86259c640 1 -- 172.21.15.32:0/1871386065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0a5710 con 0x7fc86406deb0 2023-12-20T04:40:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc86259c640 1 -- 172.21.15.32:0/1871386065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c0a5870 con 0x7fc86406deb0 2023-12-20T04:40:46.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc86259c640 1 -- 172.21.15.32:0/1871386065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0a5c00 con 0x7fc86406deb0 2023-12-20T04:40:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 -- 172.21.15.32:0/1871386065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86406deb0 msgr2=0x7fc8640cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 --2- 172.21.15.32:0/1871386065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86406deb0 0x7fc8640cd520 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fc8640cef30 tx=0x7fc85c01c940 comp rx=0 tx=0).stop 2023-12-20T04:40:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 -- 172.21.15.32:0/1871386065 shutdown_connections 2023-12-20T04:40:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 --2- 172.21.15.32:0/1871386065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86406deb0 0x7fc8640cd520 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 --2- 172.21.15.32:0/1871386065 >> v2:172.21.15.39:3300/0 conn(0x7fc86415ce30 0x7fc864161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 -- 172.21.15.32:0/1871386065 >> 172.21.15.32:0/1871386065 conn(0x7fc8640cba50 msgr2=0x7fc8640cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 -- 172.21.15.32:0/1871386065 shutdown_connections 2023-12-20T04:40:46.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 -- 172.21.15.32:0/1871386065 wait complete. 2023-12-20T04:40:46.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 Processor -- start 2023-12-20T04:40:46.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 -- start start 2023-12-20T04:40:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc86406deb0 0x7fc8640c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc864070950 0x7fc864159a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86415ce30 0x7fc864159f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc864158410 con 0x7fc86415ce30 2023-12-20T04:40:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8641586f0 con 0x7fc864070950 2023-12-20T04:40:46.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.118+0000 7fc869066640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8641589d0 con 0x7fc86406deb0 2023-12-20T04:40:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc86359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86415ce30 0x7fc864159f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc86359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86415ce30 0x7fc864159f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48770/0 (socket says 172.21.15.32:48770) 2023-12-20T04:40:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc86359e640 1 -- 172.21.15.32:0/1659470578 learned_addr learned my addr 172.21.15.32:0/1659470578 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc862d9d640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc86406deb0 0x7fc8640c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc85bfff640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc864070950 0x7fc864159a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc86359e640 1 -- 172.21.15.32:0/1659470578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc86406deb0 msgr2=0x7fc8640c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc86359e640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc86406deb0 0x7fc8640c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc86359e640 1 -- 172.21.15.32:0/1659470578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc864070950 msgr2=0x7fc864159a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc86359e640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc864070950 0x7fc864159a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc86359e640 1 -- 172.21.15.32:0/1659470578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc85c009b30 con 0x7fc86415ce30 2023-12-20T04:40:46.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc86359e640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86415ce30 0x7fc864159f40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0af040 tx=0x7fc85c0196e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc85b7fe640 1 -- 172.21.15.32:0/1659470578 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0b8670 con 0x7fc86415ce30 2023-12-20T04:40:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc85b7fe640 1 -- 172.21.15.32:0/1659470578 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c0b87d0 con 0x7fc86415ce30 2023-12-20T04:40:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc85b7fe640 1 -- 172.21.15.32:0/1659470578 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0b8b80 con 0x7fc86415ce30 2023-12-20T04:40:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc869066640 1 -- 172.21.15.32:0/1659470578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc864158cb0 con 0x7fc86415ce30 2023-12-20T04:40:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc869066640 1 -- 172.21.15.32:0/1659470578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8640c65e0 con 0x7fc86415ce30 2023-12-20T04:40:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc869066640 1 -- 172.21.15.32:0/1659470578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8640c67d0 con 0x7fc86415ce30 2023-12-20T04:40:46.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.122+0000 7fc85b7fe640 1 -- 172.21.15.32:0/1659470578 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 15) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fc85c0b8ce0 con 0x7fc86415ce30 2023-12-20T04:40:46.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.126+0000 7fc85b7fe640 1 -- 172.21.15.32:0/1659470578 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6808+0+0 (secure 0 0 0) 0x7fc85c11e660 con 0x7fc86415ce30 2023-12-20T04:40:46.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.130+0000 7fc85b7fe640 1 -- 172.21.15.32:0/1659470578 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc8640c67d0 con 0x7fc86415ce30 2023-12-20T04:40:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.250+0000 7fc85b7fe640 1 -- 172.21.15.32:0/1659470578 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 16) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fc85c0ebbe0 con 0x7fc86415ce30 2023-12-20T04:40:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.250+0000 7fc85b7fe640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc84406b5b0 0x7fc84406d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.254+0000 7fc862d9d640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc84406b5b0 0x7fc84406d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.258+0000 7fc862d9d640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc84406b5b0 0x7fc84406d9a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc854006fd0 tx=0x7fc854008040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 -- 172.21.15.32:0/1659470578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc8640c67d0 con 0x7fc86415ce30 2023-12-20T04:40:46.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc85b7fe640 1 -- 172.21.15.32:0/1659470578 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4709 (secure 0 0 0) 0x7fc8640c67d0 con 0x7fc86415ce30 2023-12-20T04:40:46.325 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:46.325 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2541430692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2541430692},{"type":"v1","addr":"172.21.15.39:6835","nonce":2541430692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1379533068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1379533068},{"type":"v1","addr":"172.21.15.32:6837","nonce":1379533068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4021567802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4021567802},{"type":"v1","addr":"172.21.15.39:6834","nonce":4021567802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:29:30.610189+0000","modified":"2023-12-20T04:29:33.661905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3391762708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3391762708},{"type":"v1","addr":"172.21.15.32:6836","nonce":3391762708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-20T04:40:46.325 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2023-12-20T04:40:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 -- 172.21.15.32:0/1659470578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc84406b5b0 msgr2=0x7fc84406d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc84406b5b0 0x7fc84406d9a0 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fc854006fd0 tx=0x7fc854008040 comp rx=0 tx=0).stop 2023-12-20T04:40:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 -- 172.21.15.32:0/1659470578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86415ce30 msgr2=0x7fc864159f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86415ce30 0x7fc864159f40 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fc85c0af040 tx=0x7fc85c0196e0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 -- 172.21.15.32:0/1659470578 shutdown_connections 2023-12-20T04:40:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc84406b5b0 0x7fc84406d9a0 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc86415ce30 0x7fc864159f40 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc864070950 0x7fc864159a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 --2- 172.21.15.32:0/1659470578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc86406deb0 0x7fc8640c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 -- 172.21.15.32:0/1659470578 >> 172.21.15.32:0/1659470578 conn(0x7fc8640cba50 msgr2=0x7fc86409fa40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:46.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.326+0000 7fc869066640 1 -- 172.21.15.32:0/1659470578 shutdown_connections 2023-12-20T04:40:46.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.330+0000 7fc869066640 1 -- 172.21.15.32:0/1659470578 wait complete. 2023-12-20T04:40:46.346 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:40:46.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.490+0000 7f02a71bc640 1 Processor -- start 2023-12-20T04:40:46.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.490+0000 7f02a71bc640 1 -- start start 2023-12-20T04:40:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a71bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f02a00af5e0 0x7f02a00af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a71bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02a015f110 0x7f02a015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a71bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f02a015b990 con 0x7f02a006ce60 2023-12-20T04:40:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a71bc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f02a015bbd0 con 0x7f02a00af5e0 2023-12-20T04:40:46.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a71bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a015beb0 con 0x7f02a015f110 2023-12-20T04:40:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a4f58640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f02a00af5e0 0x7f02a00af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a4f58640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f02a00af5e0 0x7f02a00af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:44600/0 (socket says 172.21.15.32:44600) 2023-12-20T04:40:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a4f58640 1 -- 172.21.15.32:0/2877695247 learned_addr learned my addr 172.21.15.32:0/2877695247 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a5759640 1 --2- 172.21.15.32:0/2877695247 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02a015f110 0x7f02a015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a4f58640 1 -- 172.21.15.32:0/2877695247 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02a015f110 msgr2=0x7f02a015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a4f58640 1 --2- 172.21.15.32:0/2877695247 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02a015f110 0x7f02a015c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a4f58640 1 -- 172.21.15.32:0/2877695247 >> v1:172.21.15.39:6789/0 conn(0x7f02a006ce60 legacy=0x7f02a015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.494+0000 7f02a4f58640 1 -- 172.21.15.32:0/2877695247 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02a015e910 con 0x7f02a00af5e0 2023-12-20T04:40:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f02a4f58640 1 --2- 172.21.15.32:0/2877695247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02a00af5e0 0x7f02a00af9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0294006fd0 tx=0x7f029401c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=61c7b47d7518116 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f029f7fe640 1 -- 172.21.15.32:0/2877695247 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02940aa670 con 0x7f02a00af5e0 2023-12-20T04:40:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f029f7fe640 1 -- 172.21.15.32:0/2877695247 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02940aa7d0 con 0x7f02a00af5e0 2023-12-20T04:40:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f029f7fe640 1 -- 172.21.15.32:0/2877695247 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02940aab60 con 0x7f02a00af5e0 2023-12-20T04:40:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f02a71bc640 1 -- 172.21.15.32:0/2877695247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02a00af5e0 msgr2=0x7f02a00af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f02a71bc640 1 --2- 172.21.15.32:0/2877695247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02a00af5e0 0x7f02a00af9c0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7f0294006fd0 tx=0x7f029401c4f0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f02a71bc640 1 -- 172.21.15.32:0/2877695247 shutdown_connections 2023-12-20T04:40:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f02a71bc640 1 --2- 172.21.15.32:0/2877695247 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02a015f110 0x7f02a015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f02a71bc640 1 --2- 172.21.15.32:0/2877695247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02a00af5e0 0x7f02a00af9c0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f02a71bc640 1 -- 172.21.15.32:0/2877695247 >> 172.21.15.32:0/2877695247 conn(0x7f02a006a320 msgr2=0x7f02a00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f02a71bc640 1 -- 172.21.15.32:0/2877695247 shutdown_connections 2023-12-20T04:40:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f02a71bc640 1 -- 172.21.15.32:0/2877695247 wait complete. 2023-12-20T04:40:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f02a71bc640 1 Processor -- start 2023-12-20T04:40:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.498+0000 7f02a71bc640 1 -- start start 2023-12-20T04:40:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a71bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a006ce60 0x7f02a015a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a71bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02a00af5e0 0x7f02a0154dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a71bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02a015f110 0x7f02a0151550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a71bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00c5830 con 0x7f02a006ce60 2023-12-20T04:40:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a71bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00c8210 con 0x7f02a00af5e0 2023-12-20T04:40:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a71bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02a00c86a0 con 0x7f02a015f110 2023-12-20T04:40:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a4f58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a006ce60 0x7f02a015a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a4f58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a006ce60 0x7f02a015a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48786/0 (socket says 172.21.15.32:48786) 2023-12-20T04:40:46.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a4f58640 1 -- 172.21.15.32:0/2718992261 learned_addr learned my addr 172.21.15.32:0/2718992261 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a5759640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02a015f110 0x7f02a0151550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f029ffff640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02a00af5e0 0x7f02a0154dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a4f58640 1 -- 172.21.15.32:0/2718992261 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02a015f110 msgr2=0x7f02a0151550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a4f58640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02a015f110 0x7f02a0151550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a4f58640 1 -- 172.21.15.32:0/2718992261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02a00af5e0 msgr2=0x7f02a0154dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a4f58640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02a00af5e0 0x7f02a0154dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a4f58640 1 -- 172.21.15.32:0/2718992261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0294009480 con 0x7f02a006ce60 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a5759640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02a015f110 0x7f02a0151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f029ffff640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02a00af5e0 0x7f02a0154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a4f58640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a006ce60 0x7f02a015a950 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f029401c9d0 tx=0x7f0294019290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f029d7fa640 1 -- 172.21.15.32:0/2718992261 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0294005ba0 con 0x7f02a006ce60 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f029d7fa640 1 -- 172.21.15.32:0/2718992261 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0294005d00 con 0x7f02a006ce60 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a71bc640 1 -- 172.21.15.32:0/2718992261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f02a00c4e80 con 0x7f02a006ce60 2023-12-20T04:40:46.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f029d7fa640 1 -- 172.21.15.32:0/2718992261 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02940aab60 con 0x7f02a006ce60 2023-12-20T04:40:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a71bc640 1 -- 172.21.15.32:0/2718992261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f02a00c5d60 con 0x7f02a006ce60 2023-12-20T04:40:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f02a71bc640 1 -- 172.21.15.32:0/2718992261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f02a015c170 con 0x7f02a006ce60 2023-12-20T04:40:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.502+0000 7f029d7fa640 1 -- 172.21.15.32:0/2718992261 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f02940aacc0 con 0x7f02a006ce60 2023-12-20T04:40:46.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.506+0000 7f029d7fa640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f02840658b0 0x7f0284067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.506+0000 7f029d7fa640 1 -- 172.21.15.32:0/2718992261 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f029411d5b0 con 0x7f02a006ce60 2023-12-20T04:40:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.506+0000 7f029ffff640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f02840658b0 0x7f0284067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.506+0000 7f029ffff640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f02840658b0 0x7f0284067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f02a00b3bf0 tx=0x7f028c009040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:46.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.510+0000 7f029d7fa640 1 -- 172.21.15.32:0/2718992261 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f02a015c170 con 0x7f02a006ce60 2023-12-20T04:40:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 -- 172.21.15.32:0/2718992261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f02a015c170 con 0x7f02a006ce60 2023-12-20T04:40:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f029d7fa640 1 -- 172.21.15.32:0/2718992261 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 16 v16) v1 ==== 94+0+4709 (secure 0 0 0) 0x7f02a015c170 con 0x7f02a006ce60 2023-12-20T04:40:46.700 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:46.700 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":16,"default_fscid":2,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2541430692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2541430692},{"type":"v1","addr":"172.21.15.39:6835","nonce":2541430692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1379533068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1379533068},{"type":"v1","addr":"172.21.15.32:6837","nonce":1379533068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4021567802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4021567802},{"type":"v1","addr":"172.21.15.39:6834","nonce":4021567802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11}],"filesystems":[{"mdsmap":{"epoch":16,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:29:30.610189+0000","modified":"2023-12-20T04:29:33.661905+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5031},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5031":{"gid":5031,"name":"a","rank":0,"incarnation":12,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3391762708","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3391762708},{"type":"v1","addr":"172.21.15.32:6836","nonce":3391762708}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[5],"metadata_pool":4,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":2}]} 2023-12-20T04:40:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 16 2023-12-20T04:40:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 -- 172.21.15.32:0/2718992261 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f02840658b0 msgr2=0x7f0284067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f02840658b0 0x7f0284067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f02a00b3bf0 tx=0x7f028c009040 comp rx=0 tx=0).stop 2023-12-20T04:40:46.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 -- 172.21.15.32:0/2718992261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a006ce60 msgr2=0x7f02a015a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a006ce60 0x7f02a015a950 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f029401c9d0 tx=0x7f0294019290 comp rx=0 tx=0).stop 2023-12-20T04:40:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 -- 172.21.15.32:0/2718992261 shutdown_connections 2023-12-20T04:40:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f02840658b0 0x7f0284067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02a015f110 0x7f02a0151550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f02a00af5e0 0x7f02a0154dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 --2- 172.21.15.32:0/2718992261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02a006ce60 0x7f02a015a950 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 -- 172.21.15.32:0/2718992261 >> 172.21.15.32:0/2718992261 conn(0x7f02a006a320 msgr2=0x7f02a0152930 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 -- 172.21.15.32:0/2718992261 shutdown_connections 2023-12-20T04:40:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.702+0000 7f02a71bc640 1 -- 172.21.15.32:0/2718992261 wait complete. 2023-12-20T04:40:46.721 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:40:46.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.866+0000 7f5ae758f640 1 Processor -- start 2023-12-20T04:40:46.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.866+0000 7f5ae758f640 1 -- start start 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.866+0000 7f5ae758f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ae00ae010 0x7f5ae0071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.866+0000 7f5ae758f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae00ae3f0 con 0x7f5ae006cf10 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ae758f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ae00ca4b0 con 0x7f5ae0164ef0 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ae758f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae0161800 con 0x7f5ae00ae010 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ae4b2a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ae006cf10 0x7f5ae00701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38840/0 (socket says 172.21.15.32:38840) 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ae4b2a640 1 -- 172.21.15.32:0/2705496931 learned_addr learned my addr 172.21.15.32:0/2705496931 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ae532b640 1 --2- 172.21.15.32:0/2705496931 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ae00ae010 0x7f5ae0071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ad7fff640 1 -- 172.21.15.32:0/2705496931 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1821648501 0 0) 0x7f5ae00ae3f0 con 0x7f5ae006cf10 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ad7fff640 1 -- 172.21.15.32:0/2705496931 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5acc003530 con 0x7f5ae006cf10 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ad7fff640 1 -- 172.21.15.32:0/2705496931 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2821448370 0 0) 0x7f5ae00ca4b0 con 0x7f5ae0164ef0 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ad7fff640 1 -- 172.21.15.32:0/2705496931 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ae00ae3f0 con 0x7f5ae0164ef0 2023-12-20T04:40:46.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ae532b640 1 -- 172.21.15.32:0/2705496931 >> v1:172.21.15.39:6789/0 conn(0x7f5ae0164ef0 legacy=0x7f5ae00708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ae532b640 1 -- 172.21.15.32:0/2705496931 >> v1:172.21.15.32:6789/0 conn(0x7f5ae006cf10 legacy=0x7f5ae00701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ae532b640 1 -- 172.21.15.32:0/2705496931 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ae0070fe0 con 0x7f5ae00ae010 2023-12-20T04:40:46.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.870+0000 7f5ae532b640 1 --2- 172.21.15.32:0/2705496931 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ae00ae010 0x7f5ae0071470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0097d0 tx=0x7f5adc01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=999c58536866abbf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:46.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ad7fff640 1 -- 172.21.15.32:0/2705496931 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0ac640 con 0x7f5ae00ae010 2023-12-20T04:40:46.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ad7fff640 1 -- 172.21.15.32:0/2705496931 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5adc0ac7a0 con 0x7f5ae00ae010 2023-12-20T04:40:46.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ad7fff640 1 -- 172.21.15.32:0/2705496931 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5adc0acb30 con 0x7f5ae00ae010 2023-12-20T04:40:46.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 -- 172.21.15.32:0/2705496931 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ae00ae010 msgr2=0x7f5ae0071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 --2- 172.21.15.32:0/2705496931 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ae00ae010 0x7f5ae0071470 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0097d0 tx=0x7f5adc01c3e0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 -- 172.21.15.32:0/2705496931 shutdown_connections 2023-12-20T04:40:46.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 --2- 172.21.15.32:0/2705496931 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ae00ae010 0x7f5ae0071470 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 -- 172.21.15.32:0/2705496931 >> 172.21.15.32:0/2705496931 conn(0x7f5ae006a320 msgr2=0x7f5ae00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:46.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 -- 172.21.15.32:0/2705496931 shutdown_connections 2023-12-20T04:40:46.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 -- 172.21.15.32:0/2705496931 wait complete. 2023-12-20T04:40:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 Processor -- start 2023-12-20T04:40:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 -- start start 2023-12-20T04:40:46.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ae00ae010 0x7f5ae0152950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ae0154ff0 0x7f5ae0155470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ae01559b0 0x7f5ae0156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae015dbc0 con 0x7f5ae01559b0 2023-12-20T04:40:46.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae0155e30 con 0x7f5ae00ae010 2023-12-20T04:40:46.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae758f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ae0156110 con 0x7f5ae0154ff0 2023-12-20T04:40:46.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae5b2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ae01559b0 0x7f5ae0156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae5b2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ae01559b0 0x7f5ae0156b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48796/0 (socket says 172.21.15.32:48796) 2023-12-20T04:40:46.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.874+0000 7f5ae5b2c640 1 -- 172.21.15.32:0/2627461910 learned_addr learned my addr 172.21.15.32:0/2627461910 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:46.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae4b2a640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ae0154ff0 0x7f5ae0155470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae532b640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ae00ae010 0x7f5ae0152950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae5b2c640 1 -- 172.21.15.32:0/2627461910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ae0154ff0 msgr2=0x7f5ae0155470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae5b2c640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ae0154ff0 0x7f5ae0155470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae5b2c640 1 -- 172.21.15.32:0/2627461910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ae00ae010 msgr2=0x7f5ae0152950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:46.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae5b2c640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ae00ae010 0x7f5ae0152950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:46.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae5b2c640 1 -- 172.21.15.32:0/2627461910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5adc0093b0 con 0x7f5ae01559b0 2023-12-20T04:40:46.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae5b2c640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ae01559b0 0x7f5ae0156b90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800c730 tx=0x7f5ad800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:46.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ad5ffb640 1 -- 172.21.15.32:0/2627461910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad8003b80 con 0x7f5ae01559b0 2023-12-20T04:40:46.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ad5ffb640 1 -- 172.21.15.32:0/2627461910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ad8003ce0 con 0x7f5ae01559b0 2023-12-20T04:40:46.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ad5ffb640 1 -- 172.21.15.32:0/2627461910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ad800aa90 con 0x7f5ae01559b0 2023-12-20T04:40:46.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae758f640 1 -- 172.21.15.32:0/2627461910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ae01563f0 con 0x7f5ae01559b0 2023-12-20T04:40:46.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae758f640 1 -- 172.21.15.32:0/2627461910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ae0152e90 con 0x7f5ae01559b0 2023-12-20T04:40:46.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae758f640 1 -- 172.21.15.32:0/2627461910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ae015dd70 con 0x7f5ae01559b0 2023-12-20T04:40:46.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ad5ffb640 1 -- 172.21.15.32:0/2627461910 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f5ad800abf0 con 0x7f5ae01559b0 2023-12-20T04:40:46.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ad5ffb640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5ac40658b0 0x7f5ac4067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:46.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ad5ffb640 1 -- 172.21.15.32:0/2627461910 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f5ad80830f0 con 0x7f5ae01559b0 2023-12-20T04:40:46.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.878+0000 7f5ae532b640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5ac40658b0 0x7f5ac4067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:46.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.882+0000 7f5ae532b640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5ac40658b0 0x7f5ac4067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0ae000 tx=0x7f5adc0a7000 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:46.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:46.882+0000 7f5ad5ffb640 1 -- 172.21.15.32:0/2627461910 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5ae015dd70 con 0x7f5ae01559b0 2023-12-20T04:40:47.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.042+0000 7f5ae758f640 1 -- 172.21.15.32:0/2627461910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ae004e000 con 0x7f5ae01559b0 2023-12-20T04:40:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ad5ffb640 1 -- 172.21.15.32:0/2627461910 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v48) v1 ==== 74+0+17691 (secure 0 0 0) 0x7f5ad80508d0 con 0x7f5ae01559b0 2023-12-20T04:40:47.043 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:47.043 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":48,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:40:45.241441+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:29:28.685343+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"34","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":5,"pool_name":"cephfs_data","create_time":"2023-12-20T04:29:29.647655+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"47","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":221,"snap_epoch":47,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1043195128":"2023-12-21T04:40:45.241408+0000","172.21.15.32:0/158547636":"2023-12-21T04:40:45.241408+0000","172.21.15.32:0/2331287763":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/1794234007":"2023-12-21T04:40:45.241408+0000","172.21.15.32:0/2945853272":"2023-12-21T04:40:45.241408+0000","172.21.15.32:6833/38009":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/827611945":"2023-12-21T04:40:45.241408+0000","172.21.15.32:6832/38009":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/728956527":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/3777108374":"2023-12-21T04:40:45.241408+0000","172.21.15.32:0/4147638565":"2023-12-21T04:40:45.241408+0000","172.21.15.32:0/856257861":"2023-12-21T04:29:47.726855+0000","172.21.15.32:0/2782983206":"2023-12-21T04:29:47.726855+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":5,"snaps":[{"begin":5,"length":1},{"begin":7,"length":1},{"begin":11,"length":1},{"begin":15,"length":1},{"begin":17,"length":1},{"begin":21,"length":2},{"begin":26,"length":1},{"begin":28,"length":1},{"begin":33,"length":1},{"begin":37,"length":1},{"begin":39,"length":1},{"begin":43,"length":1},{"begin":47,"length":1},{"begin":49,"length":1},{"begin":53,"length":1},{"begin":57,"length":1},{"begin":59,"length":1},{"begin":64,"length":1},{"begin":68,"length":1},{"begin":70,"length":1},{"begin":75,"length":1},{"begin":79,"length":1},{"begin":81,"length":1},{"begin":85,"length":1},{"begin":89,"length":1},{"begin":91,"length":2},{"begin":96,"length":1},{"begin":100,"length":1},{"begin":103,"length":1},{"begin":107,"length":1},{"begin":109,"length":1},{"begin":111,"length":1},{"begin":188,"length":34}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:40:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 -- 172.21.15.32:0/2627461910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5ac40658b0 msgr2=0x7f5ac4067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5ac40658b0 0x7f5ac4067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5adc0ae000 tx=0x7f5adc0a7000 comp rx=0 tx=0).stop 2023-12-20T04:40:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 -- 172.21.15.32:0/2627461910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ae01559b0 msgr2=0x7f5ae0156b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ae01559b0 0x7f5ae0156b90 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5ad800c730 tx=0x7f5ad800cbf0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 -- 172.21.15.32:0/2627461910 shutdown_connections 2023-12-20T04:40:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5ac40658b0 0x7f5ac4067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ae01559b0 0x7f5ae0156b90 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ae0154ff0 0x7f5ae0155470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 --2- 172.21.15.32:0/2627461910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ae00ae010 0x7f5ae0152950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 -- 172.21.15.32:0/2627461910 >> 172.21.15.32:0/2627461910 conn(0x7f5ae006a320 msgr2=0x7f5ae0160da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 -- 172.21.15.32:0/2627461910 shutdown_connections 2023-12-20T04:40:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.046+0000 7f5ae758f640 1 -- 172.21.15.32:0/2627461910 wait complete. 2023-12-20T04:40:47.065 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T04:40:47.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f8909640 1 Processor -- start 2023-12-20T04:40:47.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f8909640 1 -- start start 2023-12-20T04:40:47.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f8909640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40cfc80 0x7f53f4167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f8909640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f40b6ca0 con 0x7f53f415cdb0 2023-12-20T04:40:47.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f8909640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53f415d190 con 0x7f53f40d2740 2023-12-20T04:40:47.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f8909640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f40d16a0 con 0x7f53f40cfc80 2023-12-20T04:40:47.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f53f415cdb0 0x7f53f40d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38848/0 (socket says 172.21.15.32:38848) 2023-12-20T04:40:47.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f259c640 1 -- 172.21.15.32:0/1656980092 learned_addr learned my addr 172.21.15.32:0/1656980092 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:47.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f2d9d640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40cfc80 0x7f53f4167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3413590176 0 0) 0x7f53e4001120 con 0x7f53f415cdb0 2023-12-20T04:40:47.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53e0003530 con 0x7f53f415cdb0 2023-12-20T04:40:47.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 100850178 0 0) 0x7f53e0003530 con 0x7f53f415cdb0 2023-12-20T04:40:47.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40cfc80 msgr2=0x7f53f4167910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:47.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40cfc80 0x7f53f4167910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 >> v1:172.21.15.39:6789/0 conn(0x7f53f40d2740 legacy=0x7f53f40cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f40d1980 con 0x7f53f415cdb0 2023-12-20T04:40:47.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f53e4001810 con 0x7f53f415cdb0 2023-12-20T04:40:47.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 >> v1:172.21.15.32:6789/0 conn(0x7f53f415cdb0 legacy=0x7f53f40d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e00039b0 0x7f53e0003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53e0004360 0x7f53e0004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40cfc80 0x7f53e0005090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f40d16a0 con 0x7f53e00039b0 2023-12-20T04:40:47.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53e000b9c0 con 0x7f53e0004360 2023-12-20T04:40:47.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.210+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53e000bb80 con 0x7f53f40cfc80 2023-12-20T04:40:47.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f259c640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e00039b0 0x7f53e0003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53ebfff640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53e0004360 0x7f53e0004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f2d9d640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40cfc80 0x7f53e0005090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f259c640 1 -- 172.21.15.32:0/1656980092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40cfc80 msgr2=0x7f53e0005090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:47.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f259c640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40cfc80 0x7f53e0005090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f259c640 1 -- 172.21.15.32:0/1656980092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53e0004360 msgr2=0x7f53e0004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:47.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f259c640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53e0004360 0x7f53e0004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f259c640 1 -- 172.21.15.32:0/1656980092 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53e0003530 con 0x7f53e00039b0 2023-12-20T04:40:47.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f259c640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e00039b0 0x7f53e0003d90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f53e400c190 tx=0x7f53e401cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2abc4a493471833c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:47.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e4005ae0 con 0x7f53e00039b0 2023-12-20T04:40:47.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53e4005c60 con 0x7f53e00039b0 2023-12-20T04:40:47.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f1d9b640 1 -- 172.21.15.32:0/1656980092 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53e40a58e0 con 0x7f53e00039b0 2023-12-20T04:40:47.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f8909640 1 -- 172.21.15.32:0/1656980092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e00039b0 msgr2=0x7f53e0003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f8909640 1 --2- 172.21.15.32:0/1656980092 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53e00039b0 0x7f53e0003d90 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f53e400c190 tx=0x7f53e401cbe0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.214+0000 7f53f2d9d640 1 -- 172.21.15.32:0/1656980092 reap_dead start 2023-12-20T04:40:47.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 -- 172.21.15.32:0/1656980092 shutdown_connections 2023-12-20T04:40:47.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 -- 172.21.15.32:0/1656980092 >> 172.21.15.32:0/1656980092 conn(0x7f53f4065ee0 msgr2=0x7f53f406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:47.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 -- 172.21.15.32:0/1656980092 shutdown_connections 2023-12-20T04:40:47.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 -- 172.21.15.32:0/1656980092 wait complete. 2023-12-20T04:40:47.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 Processor -- start 2023-12-20T04:40:47.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 -- start start 2023-12-20T04:40:47.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53e00039b0 0x7f53f4153230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40c8650 0x7f53f414c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f414c9d0 0x7f53f413e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f414c070 con 0x7f53f414c9d0 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53f414b3f0 con 0x7f53e00039b0 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f8909640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53f414b6d0 con 0x7f53f40c8650 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f414c9d0 0x7f53f413e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53e00039b0 0x7f53f4153230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f414c9d0 0x7f53f413e7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48812/0 (socket says 172.21.15.32:48812) 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f2d9d640 1 -- 172.21.15.32:0/291688223 learned_addr learned my addr 172.21.15.32:0/291688223 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53ebfff640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40c8650 0x7f53f414c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f2d9d640 1 -- 172.21.15.32:0/291688223 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40c8650 msgr2=0x7f53f414c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f2d9d640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40c8650 0x7f53f414c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f2d9d640 1 -- 172.21.15.32:0/291688223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53e00039b0 msgr2=0x7f53f4153230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f2d9d640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53e00039b0 0x7f53f4153230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f2d9d640 1 -- 172.21.15.32:0/291688223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53f414b9b0 con 0x7f53f414c9d0 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.218+0000 7f53f259c640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53e00039b0 0x7f53f4153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.222+0000 7f53f2d9d640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f414c9d0 0x7f53f413e7e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f53f40cf420 tx=0x7f53ec0115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:47.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.222+0000 7f53eaffd640 1 -- 172.21.15.32:0/291688223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec00e070 con 0x7f53f414c9d0 2023-12-20T04:40:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.222+0000 7f53eaffd640 1 -- 172.21.15.32:0/291688223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53ec011e70 con 0x7f53f414c9d0 2023-12-20T04:40:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.222+0000 7f53f8909640 1 -- 172.21.15.32:0/291688223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53f414bc90 con 0x7f53f414c9d0 2023-12-20T04:40:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.222+0000 7f53eaffd640 1 -- 172.21.15.32:0/291688223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53ec0039a0 con 0x7f53f414c9d0 2023-12-20T04:40:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.222+0000 7f53f8909640 1 -- 172.21.15.32:0/291688223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53f414a520 con 0x7f53f414c9d0 2023-12-20T04:40:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.222+0000 7f53f8909640 1 -- 172.21.15.32:0/291688223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53f414c1b0 con 0x7f53f414c9d0 2023-12-20T04:40:47.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.222+0000 7f53eaffd640 1 -- 172.21.15.32:0/291688223 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 16) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f53ec003b00 con 0x7f53f414c9d0 2023-12-20T04:40:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.222+0000 7f53eaffd640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f53d40658e0 0x7f53d4067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.222+0000 7f53eaffd640 1 -- 172.21.15.32:0/291688223 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(48..48 src has 1..48) v4 ==== 6808+0+0 (secure 0 0 0) 0x7f53ec0836f0 con 0x7f53f414c9d0 2023-12-20T04:40:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.222+0000 7f53f259c640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f53d40658e0 0x7f53d4067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.226+0000 7f53f259c640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f53d40658e0 0x7f53d4067da0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f53e40ab000 tx=0x7f53e40a7070 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:47.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.226+0000 7f53eaffd640 1 -- 172.21.15.32:0/291688223 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f53f414c1b0 con 0x7f53f414c9d0 2023-12-20T04:40:47.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.262+0000 7f53eaffd640 1 -- 172.21.15.32:0/291688223 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 17) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f53ec051350 con 0x7f53f414c9d0 2023-12-20T04:40:47.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.422+0000 7f53f8909640 1 -- 172.21.15.32:0/291688223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f53f414c1b0 con 0x7f53f414c9d0 2023-12-20T04:40:47.422 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T04:40:47.422+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T04:40:47.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.438+0000 7f53eaffd640 1 -- 172.21.15.32:0/291688223 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v17) v1 ==== 161+0+0 (secure 0 0 0) 0x7f53f414c1b0 con 0x7f53f414c9d0 2023-12-20T04:40:47.437 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T04:40:47.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 -- 172.21.15.32:0/291688223 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f53d40658e0 msgr2=0x7f53d4067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f53d40658e0 0x7f53d4067da0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f53e40ab000 tx=0x7f53e40a7070 comp rx=0 tx=0).stop 2023-12-20T04:40:47.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 -- 172.21.15.32:0/291688223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f414c9d0 msgr2=0x7f53f413e7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f414c9d0 0x7f53f413e7e0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f53f40cf420 tx=0x7f53ec0115a0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 -- 172.21.15.32:0/291688223 shutdown_connections 2023-12-20T04:40:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f53d40658e0 0x7f53d4067da0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53f414c9d0 0x7f53f413e7e0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53f40c8650 0x7f53f414c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 --2- 172.21.15.32:0/291688223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53e00039b0 0x7f53f4153230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 -- 172.21.15.32:0/291688223 >> 172.21.15.32:0/291688223 conn(0x7f53f4065ee0 msgr2=0x7f53f415a960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 -- 172.21.15.32:0/291688223 shutdown_connections 2023-12-20T04:40:47.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.442+0000 7f53f8909640 1 -- 172.21.15.32:0/291688223 wait complete. 2023-12-20T04:40:47.460 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T04:40:47.501 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T04:40:47.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f9533b25640 1 Processor -- start 2023-12-20T04:40:47.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f9533b25640 1 -- start start 2023-12-20T04:40:47.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f9533b25640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f952c15abb0 0x7f952c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f9533b25640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c15b6a0 con 0x7f952c0b0cb0 2023-12-20T04:40:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f9533b25640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f952c06d8c0 con 0x7f952c070230 2023-12-20T04:40:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f9533b25640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f952c06e160 con 0x7f952c15abb0 2023-12-20T04:40:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f95310c0640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f952c0b0cb0 0x7f952c15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:38868/0 (socket says 172.21.15.32:38868) 2023-12-20T04:40:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f95310c0640 1 -- 172.21.15.32:0/3094632131 learned_addr learned my addr 172.21.15.32:0/3094632131 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f95318c1640 1 --2- 172.21.15.32:0/3094632131 >> v2:172.21.15.39:3300/0 conn(0x7f952c15abb0 0x7f952c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f95308bf640 1 -- 172.21.15.32:0/3094632131 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3460271637 0 0) 0x7f952c15b6a0 con 0x7f952c0b0cb0 2023-12-20T04:40:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f95308bf640 1 -- 172.21.15.32:0/3094632131 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9518003530 con 0x7f952c0b0cb0 2023-12-20T04:40:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f95308bf640 1 -- 172.21.15.32:0/3094632131 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2287444977 0 0) 0x7f952c06d8c0 con 0x7f952c070230 2023-12-20T04:40:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f95308bf640 1 -- 172.21.15.32:0/3094632131 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f952c15b6a0 con 0x7f952c070230 2023-12-20T04:40:47.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f95318c1640 1 -- 172.21.15.32:0/3094632131 >> v1:172.21.15.39:6789/0 conn(0x7f952c070230 legacy=0x7f952c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f95318c1640 1 -- 172.21.15.32:0/3094632131 >> v1:172.21.15.32:6789/0 conn(0x7f952c0b0cb0 legacy=0x7f952c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.610+0000 7f95318c1640 1 -- 172.21.15.32:0/3094632131 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f952c06dba0 con 0x7f952c15abb0 2023-12-20T04:40:47.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.614+0000 7f95318c1640 1 --2- 172.21.15.32:0/3094632131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c15abb0 0x7f952c0caaf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95280097c0 tx=0x7f952801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7fa113f151c62378 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.614+0000 7f95308bf640 1 -- 172.21.15.32:0/3094632131 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95280ab680 con 0x7f952c15abb0 2023-12-20T04:40:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.614+0000 7f95308bf640 1 -- 172.21.15.32:0/3094632131 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95280ab7e0 con 0x7f952c15abb0 2023-12-20T04:40:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.614+0000 7f95308bf640 1 -- 172.21.15.32:0/3094632131 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95280abb70 con 0x7f952c15abb0 2023-12-20T04:40:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.614+0000 7f9533b25640 1 -- 172.21.15.32:0/3094632131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c15abb0 msgr2=0x7f952c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.614+0000 7f9533b25640 1 --2- 172.21.15.32:0/3094632131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c15abb0 0x7f952c0caaf0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f95280097c0 tx=0x7f952801c480 comp rx=0 tx=0).stop 2023-12-20T04:40:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.614+0000 7f9533b25640 1 -- 172.21.15.32:0/3094632131 shutdown_connections 2023-12-20T04:40:47.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.614+0000 7f9533b25640 1 --2- 172.21.15.32:0/3094632131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c15abb0 0x7f952c0caaf0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.614+0000 7f9533b25640 1 -- 172.21.15.32:0/3094632131 >> 172.21.15.32:0/3094632131 conn(0x7f952c06a510 msgr2=0x7f952c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.614+0000 7f9533b25640 1 -- 172.21.15.32:0/3094632131 shutdown_connections 2023-12-20T04:40:47.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.614+0000 7f9533b25640 1 -- 172.21.15.32:0/3094632131 wait complete. 2023-12-20T04:40:47.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f9533b25640 1 Processor -- start 2023-12-20T04:40:47.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f9533b25640 1 -- start start 2023-12-20T04:40:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f9533b25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f952c070230 0x7f952c0c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f9533b25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b0cb0 0x7f952c143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f9533b25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c15abb0 0x7f952c152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f9533b25640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c0c98f0 con 0x7f952c0b0cb0 2023-12-20T04:40:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f9533b25640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952c14ce50 con 0x7f952c15abb0 2023-12-20T04:40:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f9533b25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952c1469b0 con 0x7f952c070230 2023-12-20T04:40:47.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95310c0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b0cb0 0x7f952c143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95310c0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b0cb0 0x7f952c143120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48864/0 (socket says 172.21.15.32:48864) 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95310c0640 1 -- 172.21.15.32:0/1588566443 learned_addr learned my addr 172.21.15.32:0/1588566443 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95318c1640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f952c070230 0x7f952c0c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95320c2640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c15abb0 0x7f952c152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95310c0640 1 -- 172.21.15.32:0/1588566443 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f952c070230 msgr2=0x7f952c0c93b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95310c0640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f952c070230 0x7f952c0c93b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95310c0640 1 -- 172.21.15.32:0/1588566443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c15abb0 msgr2=0x7f952c152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95310c0640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c15abb0 0x7f952c152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95310c0640 1 -- 172.21.15.32:0/1588566443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9528009420 con 0x7f952c0b0cb0 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95310c0640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b0cb0 0x7f952c143120 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f951c005890 tx=0x7f951c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95227fc640 1 -- 172.21.15.32:0/1588566443 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c005cf0 con 0x7f952c0b0cb0 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95227fc640 1 -- 172.21.15.32:0/1588566443 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f951c005e50 con 0x7f952c0b0cb0 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f9533b25640 1 -- 172.21.15.32:0/1588566443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952c146c90 con 0x7f952c0b0cb0 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f95227fc640 1 -- 172.21.15.32:0/1588566443 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f951c007290 con 0x7f952c0b0cb0 2023-12-20T04:40:47.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.618+0000 7f9533b25640 1 -- 172.21.15.32:0/1588566443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952c147250 con 0x7f952c0b0cb0 2023-12-20T04:40:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.622+0000 7f9533b25640 1 -- 172.21.15.32:0/1588566443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f952c04e000 con 0x7f952c0b0cb0 2023-12-20T04:40:47.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.622+0000 7f95227fc640 1 -- 172.21.15.32:0/1588566443 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f951c0073f0 con 0x7f952c0b0cb0 2023-12-20T04:40:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.622+0000 7f95227fc640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9510065b20 0x7f9510067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.622+0000 7f95227fc640 1 -- 172.21.15.32:0/1588566443 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(49..49 src has 1..49) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f951c082b30 con 0x7f952c0b0cb0 2023-12-20T04:40:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.622+0000 7f95318c1640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9510065b20 0x7f9510067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:47.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.626+0000 7f95318c1640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9510065b20 0x7f9510067fe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f95280af000 tx=0x7f95280a7000 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:47.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.626+0000 7f95227fc640 1 -- 172.21.15.32:0/1588566443 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f951c050110 con 0x7f952c0b0cb0 2023-12-20T04:40:47.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:47.818+0000 7f9533b25640 1 -- 172.21.15.32:0/1588566443 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f952c0b1d20 con 0x7f952c0b0cb0 2023-12-20T04:40:48.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f95227fc640 1 -- 172.21.15.32:0/1588566443 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v18) v1 ==== 104+0+0 (secure 0 0 0) 0x7f951c053730 con 0x7f952c0b0cb0 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 -- 172.21.15.32:0/1588566443 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9510065b20 msgr2=0x7f9510067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9510065b20 0x7f9510067fe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f95280af000 tx=0x7f95280a7000 comp rx=0 tx=0).stop 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 -- 172.21.15.32:0/1588566443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b0cb0 msgr2=0x7f952c143120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b0cb0 0x7f952c143120 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7f951c005890 tx=0x7f951c00c960 comp rx=0 tx=0).stop 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 -- 172.21.15.32:0/1588566443 shutdown_connections 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9510065b20 0x7f9510067fe0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952c15abb0 0x7f952c152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952c0b0cb0 0x7f952c143120 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 --2- 172.21.15.32:0/1588566443 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f952c070230 0x7f952c0c93b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 -- 172.21.15.32:0/1588566443 >> 172.21.15.32:0/1588566443 conn(0x7f952c06a510 msgr2=0x7f952c0c4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 -- 172.21.15.32:0/1588566443 shutdown_connections 2023-12-20T04:40:48.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.441+0000 7f9533b25640 1 -- 172.21.15.32:0/1588566443 wait complete. 2023-12-20T04:40:48.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T04:40:48.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.649+0000 7f824a4fc640 1 Processor -- start 2023-12-20T04:40:48.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.649+0000 7f824a4fc640 1 -- start start 2023-12-20T04:40:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.649+0000 7f824a4fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8244159af0 0x7f82440b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.649+0000 7f824a4fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f824406d2d0 0x7f824406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.649+0000 7f824a4fc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f824406dd50 con 0x7f82440b07f0 2023-12-20T04:40:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.649+0000 7f824a4fc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82440cc940 con 0x7f8244159af0 2023-12-20T04:40:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.649+0000 7f824a4fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82440ccc20 con 0x7f824406d2d0 2023-12-20T04:40:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f8243fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8244159af0 0x7f82440b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f8243fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8244159af0 0x7f82440b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48870/0 (socket says 172.21.15.32:48870) 2023-12-20T04:40:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f8243fff640 1 -- 172.21.15.32:0/3723512946 learned_addr learned my addr 172.21.15.32:0/3723512946 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f82437fe640 1 --2- 172.21.15.32:0/3723512946 >> v2:172.21.15.39:3300/0 conn(0x7f824406d2d0 0x7f824406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f8243fff640 1 -- 172.21.15.32:0/3723512946 >> v2:172.21.15.39:3300/0 conn(0x7f824406d2d0 msgr2=0x7f824406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:48.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f8243fff640 1 --2- 172.21.15.32:0/3723512946 >> v2:172.21.15.39:3300/0 conn(0x7f824406d2d0 0x7f824406d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:48.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f8243fff640 1 -- 172.21.15.32:0/3723512946 >> v1:172.21.15.39:6789/0 conn(0x7f82440b07f0 legacy=0x7f82440cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:48.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f8243fff640 1 -- 172.21.15.32:0/3723512946 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82440ccf00 con 0x7f8244159af0 2023-12-20T04:40:48.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f8243fff640 1 --2- 172.21.15.32:0/3723512946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8244159af0 0x7f82440b1930 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f824406e440 tx=0x7f823401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f434afd58d5c446 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:48.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f8242ffd640 1 -- 172.21.15.32:0/3723512946 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82340aa650 con 0x7f8244159af0 2023-12-20T04:40:48.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f8242ffd640 1 -- 172.21.15.32:0/3723512946 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82340aa7b0 con 0x7f8244159af0 2023-12-20T04:40:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.653+0000 7f8242ffd640 1 -- 172.21.15.32:0/3723512946 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82340aab40 con 0x7f8244159af0 2023-12-20T04:40:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 -- 172.21.15.32:0/3723512946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8244159af0 msgr2=0x7f82440b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 --2- 172.21.15.32:0/3723512946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8244159af0 0x7f82440b1930 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7f824406e440 tx=0x7f823401c4d0 comp rx=0 tx=0).stop 2023-12-20T04:40:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 -- 172.21.15.32:0/3723512946 shutdown_connections 2023-12-20T04:40:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 --2- 172.21.15.32:0/3723512946 >> v2:172.21.15.39:3300/0 conn(0x7f824406d2d0 0x7f824406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 --2- 172.21.15.32:0/3723512946 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8244159af0 0x7f82440b1930 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 -- 172.21.15.32:0/3723512946 >> 172.21.15.32:0/3723512946 conn(0x7f824406a510 msgr2=0x7f82440ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 -- 172.21.15.32:0/3723512946 shutdown_connections 2023-12-20T04:40:48.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 -- 172.21.15.32:0/3723512946 wait complete. 2023-12-20T04:40:48.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 Processor -- start 2023-12-20T04:40:48.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 -- start start 2023-12-20T04:40:48.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f824406d2d0 0x7f8244142280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82440b07f0 0x7f824414c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8244159af0 0x7f824414f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824414c310 con 0x7f824406d2d0 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f824414fae0 con 0x7f82440b07f0 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f824a4fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f824414d4d0 con 0x7f8244159af0 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f8243fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f824406d2d0 0x7f8244142280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f8243fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f824406d2d0 0x7f8244142280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48880/0 (socket says 172.21.15.32:48880) 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f8243fff640 1 -- 172.21.15.32:0/2512603555 learned_addr learned my addr 172.21.15.32:0/2512603555 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f82437fe640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82440b07f0 0x7f824414c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.657+0000 7f8248a99640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8244159af0 0x7f824414f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f8243fff640 1 -- 172.21.15.32:0/2512603555 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8244159af0 msgr2=0x7f824414f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:48.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f8243fff640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8244159af0 0x7f824414f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f8243fff640 1 -- 172.21.15.32:0/2512603555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82440b07f0 msgr2=0x7f824414c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f8243fff640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82440b07f0 0x7f824414c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f8243fff640 1 -- 172.21.15.32:0/2512603555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f82340094d0 con 0x7f824406d2d0 2023-12-20T04:40:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f8243fff640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f824406d2d0 0x7f8244142280 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f82340ac040 tx=0x7f823401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f8240ff9640 1 -- 172.21.15.32:0/2512603555 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82340b3800 con 0x7f824406d2d0 2023-12-20T04:40:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f8240ff9640 1 -- 172.21.15.32:0/2512603555 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82340b3960 con 0x7f824406d2d0 2023-12-20T04:40:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f824a4fc640 1 -- 172.21.15.32:0/2512603555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82440b1190 con 0x7f824406d2d0 2023-12-20T04:40:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f8240ff9640 1 -- 172.21.15.32:0/2512603555 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82340b3cd0 con 0x7f824406d2d0 2023-12-20T04:40:48.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f824a4fc640 1 -- 172.21.15.32:0/2512603555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f824414b9e0 con 0x7f824406d2d0 2023-12-20T04:40:48.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f824a4fc640 1 -- 172.21.15.32:0/2512603555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f82440ca8c0 con 0x7f824406d2d0 2023-12-20T04:40:48.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.661+0000 7f8240ff9640 1 -- 172.21.15.32:0/2512603555 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 17) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f82340b2380 con 0x7f824406d2d0 2023-12-20T04:40:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.665+0000 7f8240ff9640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8230065af0 0x7f8230067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.665+0000 7f8240ff9640 1 -- 172.21.15.32:0/2512603555 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(50..50 src has 1..50) v4 ==== 6894+0+0 (secure 0 0 0) 0x7f823411d8a0 con 0x7f824406d2d0 2023-12-20T04:40:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.665+0000 7f82437fe640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8230065af0 0x7f8230067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:48.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.665+0000 7f82437fe640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8230065af0 0x7f8230067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8244144590 tx=0x7f8238009190 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:48.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.669+0000 7f8240ff9640 1 -- 172.21.15.32:0/2512603555 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f82340eac70 con 0x7f824406d2d0 2023-12-20T04:40:48.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:48.829+0000 7f824a4fc640 1 -- 172.21.15.32:0/2512603555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f824404e000 con 0x7f824406d2d0 2023-12-20T04:40:49.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.453+0000 7f8240ff9640 1 -- 172.21.15.32:0/2512603555 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f82340eb040 con 0x7f824406d2d0 2023-12-20T04:40:49.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f8240ff9640 1 -- 172.21.15.32:0/2512603555 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v51) v1 ==== 181+0+0 (secure 0 0 0) 0x7f8234105de0 con 0x7f824406d2d0 2023-12-20T04:40:49.456 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T04:40:49.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 -- 172.21.15.32:0/2512603555 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8230065af0 msgr2=0x7f8230067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:49.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8230065af0 0x7f8230067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f8244144590 tx=0x7f8238009190 comp rx=0 tx=0).stop 2023-12-20T04:40:49.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 -- 172.21.15.32:0/2512603555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f824406d2d0 msgr2=0x7f8244142280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:49.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f824406d2d0 0x7f8244142280 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f82340ac040 tx=0x7f823401ad10 comp rx=0 tx=0).stop 2023-12-20T04:40:49.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 -- 172.21.15.32:0/2512603555 shutdown_connections 2023-12-20T04:40:49.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8230065af0 0x7f8230067fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:49.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8244159af0 0x7f824414f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:49.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82440b07f0 0x7f824414c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:49.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 --2- 172.21.15.32:0/2512603555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f824406d2d0 0x7f8244142280 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:49.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 -- 172.21.15.32:0/2512603555 >> 172.21.15.32:0/2512603555 conn(0x7f824406a510 msgr2=0x7f824414e260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:49.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 -- 172.21.15.32:0/2512603555 shutdown_connections 2023-12-20T04:40:49.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.457+0000 7f824a4fc640 1 -- 172.21.15.32:0/2512603555 wait complete. 2023-12-20T04:40:49.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T04:40:49.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72ca0c4640 1 Processor -- start 2023-12-20T04:40:49.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72ca0c4640 1 -- start start 2023-12-20T04:40:49.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72ca0c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72c4159af0 0x7f72c40b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:49.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72ca0c4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f72c406d2d0 0x7f72c406d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:49.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72ca0c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72c406dd50 con 0x7f72c40b07f0 2023-12-20T04:40:49.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72ca0c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c40cc940 con 0x7f72c4159af0 2023-12-20T04:40:49.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72ca0c4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72c40ccc20 con 0x7f72c406d2d0 2023-12-20T04:40:49.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72c37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72c4159af0 0x7f72c40b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:49.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72c37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72c4159af0 0x7f72c40b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46118/0 (socket says 172.21.15.32:46118) 2023-12-20T04:40:49.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72c37fe640 1 -- 172.21.15.32:0/3454538230 learned_addr learned my addr 172.21.15.32:0/3454538230 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72c2ffd640 1 --2- 172.21.15.32:0/3454538230 >> v2:172.21.15.39:3300/0 conn(0x7f72c406d2d0 0x7f72c406d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72c37fe640 1 -- 172.21.15.32:0/3454538230 >> v2:172.21.15.39:3300/0 conn(0x7f72c406d2d0 msgr2=0x7f72c406d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72c37fe640 1 --2- 172.21.15.32:0/3454538230 >> v2:172.21.15.39:3300/0 conn(0x7f72c406d2d0 0x7f72c406d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72c37fe640 1 -- 172.21.15.32:0/3454538230 >> v1:172.21.15.39:6789/0 conn(0x7f72c40b07f0 legacy=0x7f72c40cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72c37fe640 1 -- 172.21.15.32:0/3454538230 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72c40ccf00 con 0x7f72c4159af0 2023-12-20T04:40:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.633+0000 7f72c2ffd640 1 --2- 172.21.15.32:0/3454538230 >> v2:172.21.15.39:3300/0 conn(0x7f72c406d2d0 0x7f72c406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:49.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72c37fe640 1 --2- 172.21.15.32:0/3454538230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c4159af0 0x7f72c40b1930 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f72c406e440 tx=0x7f72b401c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0f48ffccc62d8d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72c27fc640 1 -- 172.21.15.32:0/3454538230 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b40aa610 con 0x7f72c4159af0 2023-12-20T04:40:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72c27fc640 1 -- 172.21.15.32:0/3454538230 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72b40a6040 con 0x7f72c4159af0 2023-12-20T04:40:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72c27fc640 1 -- 172.21.15.32:0/3454538230 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72b40aa990 con 0x7f72c4159af0 2023-12-20T04:40:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72ca0c4640 1 -- 172.21.15.32:0/3454538230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c4159af0 msgr2=0x7f72c40b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72ca0c4640 1 --2- 172.21.15.32:0/3454538230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c4159af0 0x7f72c40b1930 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f72c406e440 tx=0x7f72b401c490 comp rx=0 tx=0).stop 2023-12-20T04:40:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72ca0c4640 1 -- 172.21.15.32:0/3454538230 shutdown_connections 2023-12-20T04:40:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72ca0c4640 1 --2- 172.21.15.32:0/3454538230 >> v2:172.21.15.39:3300/0 conn(0x7f72c406d2d0 0x7f72c406d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72ca0c4640 1 --2- 172.21.15.32:0/3454538230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c4159af0 0x7f72c40b1930 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72ca0c4640 1 -- 172.21.15.32:0/3454538230 >> 172.21.15.32:0/3454538230 conn(0x7f72c406a510 msgr2=0x7f72c40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72ca0c4640 1 -- 172.21.15.32:0/3454538230 shutdown_connections 2023-12-20T04:40:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.637+0000 7f72ca0c4640 1 -- 172.21.15.32:0/3454538230 wait complete. 2023-12-20T04:40:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72ca0c4640 1 Processor -- start 2023-12-20T04:40:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72ca0c4640 1 -- start start 2023-12-20T04:40:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72ca0c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72c406d2d0 0x7f72c41587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72ca0c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72c40b07f0 0x7f72c4143ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72ca0c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c4159af0 0x7f72c4156b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72ca0c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4157050 con 0x7f72c4159af0 2023-12-20T04:40:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72ca0c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4157320 con 0x7f72c40b07f0 2023-12-20T04:40:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72ca0c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72c4157600 con 0x7f72c406d2d0 2023-12-20T04:40:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c4159af0 0x7f72c4156b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c4159af0 0x7f72c4156b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46126/0 (socket says 172.21.15.32:46126) 2023-12-20T04:40:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72c3fff640 1 -- 172.21.15.32:0/4163150445 learned_addr learned my addr 172.21.15.32:0/4163150445 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72c2ffd640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72c40b07f0 0x7f72c4143ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72c37fe640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72c406d2d0 0x7f72c41587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72c3fff640 1 -- 172.21.15.32:0/4163150445 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72c406d2d0 msgr2=0x7f72c41587b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72c3fff640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72c406d2d0 0x7f72c41587b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72c3fff640 1 -- 172.21.15.32:0/4163150445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72c40b07f0 msgr2=0x7f72c4143ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72c3fff640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72c40b07f0 0x7f72c4143ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72c3fff640 1 -- 172.21.15.32:0/4163150445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72b40094d0 con 0x7f72c4159af0 2023-12-20T04:40:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72c3fff640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c4159af0 0x7f72c4156b10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00c780 tx=0x7f72bc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72abfff640 1 -- 172.21.15.32:0/4163150445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc004b70 con 0x7f72c4159af0 2023-12-20T04:40:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72abfff640 1 -- 172.21.15.32:0/4163150445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72bc00ce90 con 0x7f72c4159af0 2023-12-20T04:40:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72abfff640 1 -- 172.21.15.32:0/4163150445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72bc00a850 con 0x7f72c4159af0 2023-12-20T04:40:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.641+0000 7f72ca0c4640 1 -- 172.21.15.32:0/4163150445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72c41578e0 con 0x7f72c4159af0 2023-12-20T04:40:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.645+0000 7f72ca0c4640 1 -- 172.21.15.32:0/4163150445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72c40c7640 con 0x7f72c4159af0 2023-12-20T04:40:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.645+0000 7f72ca0c4640 1 -- 172.21.15.32:0/4163150445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72c40c7b90 con 0x7f72c4159af0 2023-12-20T04:40:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.645+0000 7f72abfff640 1 -- 172.21.15.32:0/4163150445 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f72bc00a9b0 con 0x7f72c4159af0 2023-12-20T04:40:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.645+0000 7f72abfff640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f72b0065b80 0x7f72b0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.645+0000 7f72abfff640 1 -- 172.21.15.32:0/4163150445 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(51..51 src has 1..51) v4 ==== 5916+0+0 (secure 0 0 0) 0x7f72bc083510 con 0x7f72c4159af0 2023-12-20T04:40:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.645+0000 7f72c37fe640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f72b0065b80 0x7f72b0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.645+0000 7f72c37fe640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f72b0065b80 0x7f72b0068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f72b40ae000 tx=0x7f72b40b5040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.649+0000 7f72abfff640 1 -- 172.21.15.32:0/4163150445 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f72c40c7b90 con 0x7f72c4159af0 2023-12-20T04:40:49.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:49.809+0000 7f72ca0c4640 1 -- 172.21.15.32:0/4163150445 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f72c40c7b90 con 0x7f72c4159af0 2023-12-20T04:40:50.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.461+0000 7f72abfff640 1 -- 172.21.15.32:0/4163150445 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v52) v1 ==== 169+0+0 (secure 0 0 0) 0x7f72c40c7b90 con 0x7f72c4159af0 2023-12-20T04:40:50.462 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T04:40:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 -- 172.21.15.32:0/4163150445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f72b0065b80 msgr2=0x7f72b0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f72b0065b80 0x7f72b0068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f72b40ae000 tx=0x7f72b40b5040 comp rx=0 tx=0).stop 2023-12-20T04:40:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 -- 172.21.15.32:0/4163150445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c4159af0 msgr2=0x7f72c4156b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c4159af0 0x7f72c4156b10 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f72bc00c780 tx=0x7f72bc00cc40 comp rx=0 tx=0).stop 2023-12-20T04:40:50.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 -- 172.21.15.32:0/4163150445 shutdown_connections 2023-12-20T04:40:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f72b0065b80 0x7f72b0068040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72c4159af0 0x7f72c4156b10 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72c40b07f0 0x7f72c4143ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 --2- 172.21.15.32:0/4163150445 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72c406d2d0 0x7f72c41587b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 -- 172.21.15.32:0/4163150445 >> 172.21.15.32:0/4163150445 conn(0x7f72c406a510 msgr2=0x7f72c4070880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 -- 172.21.15.32:0/4163150445 shutdown_connections 2023-12-20T04:40:50.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.465+0000 7f72ca0c4640 1 -- 172.21.15.32:0/4163150445 wait complete. 2023-12-20T04:40:50.494 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T04:40:50.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c9357b640 1 Processor -- start 2023-12-20T04:40:50.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c9357b640 1 -- start start 2023-12-20T04:40:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c9357b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c8c15abb0 0x7f6c8c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c9357b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8c15b6a0 con 0x7f6c8c070230 2023-12-20T04:40:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c9357b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c8c06d8c0 con 0x7f6c8c0b0cb0 2023-12-20T04:40:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c9357b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c8c06e160 con 0x7f6c8c15abb0 2023-12-20T04:40:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c91317640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c8c15abb0 0x7f6c8c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c91317640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c8c15abb0 0x7f6c8c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46142/0 (socket says 172.21.15.32:46142) 2023-12-20T04:40:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c91317640 1 -- 172.21.15.32:0/2728141619 learned_addr learned my addr 172.21.15.32:0/2728141619 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c91b18640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6c8c070230 0x7f6c8c06c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46642/0 (socket says 172.21.15.32:46642) 2023-12-20T04:40:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c91317640 1 -- 172.21.15.32:0/2728141619 >> v1:172.21.15.39:6789/0 conn(0x7f6c8c0b0cb0 legacy=0x7f6c8c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c91317640 1 -- 172.21.15.32:0/2728141619 >> v1:172.21.15.32:6789/0 conn(0x7f6c8c070230 legacy=0x7f6c8c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:50.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.641+0000 7f6c91317640 1 -- 172.21.15.32:0/2728141619 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c8c06dba0 con 0x7f6c8c15abb0 2023-12-20T04:40:50.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.645+0000 7f6c91317640 1 --2- 172.21.15.32:0/2728141619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c8c15abb0 0x7f6c8c0caaf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009870 tx=0x7f6c8801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71f67d08133cf974 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:50.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.645+0000 7f6c83fff640 1 -- 172.21.15.32:0/2728141619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c880a9400 con 0x7f6c8c15abb0 2023-12-20T04:40:50.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.645+0000 7f6c83fff640 1 -- 172.21.15.32:0/2728141619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c880a9560 con 0x7f6c8c15abb0 2023-12-20T04:40:50.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.645+0000 7f6c83fff640 1 -- 172.21.15.32:0/2728141619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c880a98f0 con 0x7f6c8c15abb0 2023-12-20T04:40:50.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.645+0000 7f6c9357b640 1 -- 172.21.15.32:0/2728141619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c8c15abb0 msgr2=0x7f6c8c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:50.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.645+0000 7f6c9357b640 1 --2- 172.21.15.32:0/2728141619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c8c15abb0 0x7f6c8c0caaf0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f6c88009870 tx=0x7f6c8801c540 comp rx=0 tx=0).stop 2023-12-20T04:40:50.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.645+0000 7f6c9357b640 1 -- 172.21.15.32:0/2728141619 shutdown_connections 2023-12-20T04:40:50.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.645+0000 7f6c9357b640 1 --2- 172.21.15.32:0/2728141619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c8c15abb0 0x7f6c8c0caaf0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:50.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.645+0000 7f6c9357b640 1 -- 172.21.15.32:0/2728141619 >> 172.21.15.32:0/2728141619 conn(0x7f6c8c06a510 msgr2=0x7f6c8c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:50.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.645+0000 7f6c9357b640 1 -- 172.21.15.32:0/2728141619 shutdown_connections 2023-12-20T04:40:50.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.645+0000 7f6c9357b640 1 -- 172.21.15.32:0/2728141619 wait complete. 2023-12-20T04:40:50.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c9357b640 1 Processor -- start 2023-12-20T04:40:50.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c9357b640 1 -- start start 2023-12-20T04:40:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c9357b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c8c070230 0x7f6c8c156850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c9357b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c8c0b0cb0 0x7f6c8c0b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c9357b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c8c15abb0 0x7f6c8c0b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c9357b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c156e30 con 0x7f6c8c15abb0 2023-12-20T04:40:50.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c9357b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c157110 con 0x7f6c8c070230 2023-12-20T04:40:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c9357b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c8c1573f0 con 0x7f6c8c0b0cb0 2023-12-20T04:40:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c91b18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c8c15abb0 0x7f6c8c0b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c91b18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c8c15abb0 0x7f6c8c0b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46148/0 (socket says 172.21.15.32:46148) 2023-12-20T04:40:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c91b18640 1 -- 172.21.15.32:0/944248720 learned_addr learned my addr 172.21.15.32:0/944248720 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c90b16640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c8c0b0cb0 0x7f6c8c0b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:50.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c91317640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c8c070230 0x7f6c8c156850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:50.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c91b18640 1 -- 172.21.15.32:0/944248720 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c8c0b0cb0 msgr2=0x7f6c8c0b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:50.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c91b18640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c8c0b0cb0 0x7f6c8c0b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:50.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c91b18640 1 -- 172.21.15.32:0/944248720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c8c070230 msgr2=0x7f6c8c156850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:50.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c91b18640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c8c070230 0x7f6c8c156850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:50.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c91b18640 1 -- 172.21.15.32:0/944248720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c880094d0 con 0x7f6c8c15abb0 2023-12-20T04:40:50.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c91317640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c8c070230 0x7f6c8c156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c91b18640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c8c15abb0 0x7f6c8c0b2fb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c7a0 tx=0x7f6c8400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c81ffb640 1 -- 172.21.15.32:0/944248720 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400a640 con 0x7f6c8c15abb0 2023-12-20T04:40:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c81ffb640 1 -- 172.21.15.32:0/944248720 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c8400a7a0 con 0x7f6c8c15abb0 2023-12-20T04:40:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c81ffb640 1 -- 172.21.15.32:0/944248720 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c8400ab10 con 0x7f6c8c15abb0 2023-12-20T04:40:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.649+0000 7f6c9357b640 1 -- 172.21.15.32:0/944248720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c8c1576d0 con 0x7f6c8c15abb0 2023-12-20T04:40:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.653+0000 7f6c9357b640 1 -- 172.21.15.32:0/944248720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c8c14e730 con 0x7f6c8c15abb0 2023-12-20T04:40:50.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.653+0000 7f6c9357b640 1 -- 172.21.15.32:0/944248720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c8c04e000 con 0x7f6c8c15abb0 2023-12-20T04:40:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.653+0000 7f6c81ffb640 1 -- 172.21.15.32:0/944248720 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6c8400ac70 con 0x7f6c8c15abb0 2023-12-20T04:40:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.653+0000 7f6c81ffb640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c70065bb0 0x7f6c70068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.653+0000 7f6c81ffb640 1 -- 172.21.15.32:0/944248720 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f6c84082ab0 con 0x7f6c8c15abb0 2023-12-20T04:40:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.653+0000 7f6c91317640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c70065bb0 0x7f6c70068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:50.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.653+0000 7f6c91317640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c70065bb0 0x7f6c70068070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6c880ae000 tx=0x7f6c880b3040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:50.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.657+0000 7f6c81ffb640 1 -- 172.21.15.32:0/944248720 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c840503b0 con 0x7f6c8c15abb0 2023-12-20T04:40:50.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.833+0000 7f6c9357b640 1 -- 172.21.15.32:0/944248720 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f6c8c071180 con 0x7f6c8c15abb0 2023-12-20T04:40:50.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.837+0000 7f6c81ffb640 1 -- 172.21.15.32:0/944248720 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v13) v1 ==== 120+0+0 (secure 0 0 0) 0x7f6c84017020 con 0x7f6c8c15abb0 2023-12-20T04:40:50.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 -- 172.21.15.32:0/944248720 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c70065bb0 msgr2=0x7f6c70068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:50.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c70065bb0 0x7f6c70068070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f6c880ae000 tx=0x7f6c880b3040 comp rx=0 tx=0).stop 2023-12-20T04:40:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 -- 172.21.15.32:0/944248720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c8c15abb0 msgr2=0x7f6c8c0b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c8c15abb0 0x7f6c8c0b2fb0 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f6c8400c7a0 tx=0x7f6c8400cc60 comp rx=0 tx=0).stop 2023-12-20T04:40:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 -- 172.21.15.32:0/944248720 shutdown_connections 2023-12-20T04:40:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c70065bb0 0x7f6c70068070 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c8c15abb0 0x7f6c8c0b2fb0 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c8c0b0cb0 0x7f6c8c0b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 --2- 172.21.15.32:0/944248720 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c8c070230 0x7f6c8c156850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 -- 172.21.15.32:0/944248720 >> 172.21.15.32:0/944248720 conn(0x7f6c8c06a510 msgr2=0x7f6c8c0c3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 -- 172.21.15.32:0/944248720 shutdown_connections 2023-12-20T04:40:50.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:50.841+0000 7f6c9357b640 1 -- 172.21.15.32:0/944248720 wait complete. 2023-12-20T04:40:50.860 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T04:40:51.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.017+0000 7f3fe5b39640 1 Processor -- start 2023-12-20T04:40:51.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.017+0000 7f3fe5b39640 1 -- start start 2023-12-20T04:40:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fe5b39640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fe0157480 0x7f3fe00c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fe5b39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fe00b0760 0x7f3fe00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fe5b39640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3fe015d660 0x7f3fe015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fe5b39640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe00b1100 con 0x7f3fe0157480 2023-12-20T04:40:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fe5b39640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fe00cde20 con 0x7f3fe015d660 2023-12-20T04:40:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fe5b39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe00c8e30 con 0x7f3fe00b0760 2023-12-20T04:40:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fdf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fe0157480 0x7f3fe00c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fdf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fe0157480 0x7f3fe00c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46158/0 (socket says 172.21.15.32:46158) 2023-12-20T04:40:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fdf7fe640 1 -- 172.21.15.32:0/752305896 learned_addr learned my addr 172.21.15.32:0/752305896 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fdf7fe640 1 -- 172.21.15.32:0/752305896 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fe00b0760 msgr2=0x7f3fe00b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fdf7fe640 1 --2- 172.21.15.32:0/752305896 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fe00b0760 0x7f3fe00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fdf7fe640 1 -- 172.21.15.32:0/752305896 >> v2:172.21.15.39:3300/0 conn(0x7f3fe015d660 msgr2=0x7f3fe015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fdf7fe640 1 --2- 172.21.15.32:0/752305896 >> v2:172.21.15.39:3300/0 conn(0x7f3fe015d660 0x7f3fe015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.021+0000 7f3fdf7fe640 1 -- 172.21.15.32:0/752305896 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fe00ce070 con 0x7f3fe0157480 2023-12-20T04:40:51.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fdf7fe640 1 --2- 172.21.15.32:0/752305896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fe0157480 0x7f3fe00c9840 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00b49b0 tx=0x7f3fd001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=333656ac1c0fc98c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fde7fc640 1 -- 172.21.15.32:0/752305896 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd00ab600 con 0x7f3fe0157480 2023-12-20T04:40:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fde7fc640 1 -- 172.21.15.32:0/752305896 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd00ab760 con 0x7f3fe0157480 2023-12-20T04:40:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fde7fc640 1 -- 172.21.15.32:0/752305896 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd00abaf0 con 0x7f3fe0157480 2023-12-20T04:40:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fe5b39640 1 -- 172.21.15.32:0/752305896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fe0157480 msgr2=0x7f3fe00c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fe5b39640 1 --2- 172.21.15.32:0/752305896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fe0157480 0x7f3fe00c9840 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f3fe00b49b0 tx=0x7f3fd001c440 comp rx=0 tx=0).stop 2023-12-20T04:40:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fe5b39640 1 -- 172.21.15.32:0/752305896 shutdown_connections 2023-12-20T04:40:51.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fe5b39640 1 --2- 172.21.15.32:0/752305896 >> v2:172.21.15.39:3300/0 conn(0x7f3fe015d660 0x7f3fe015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fe5b39640 1 --2- 172.21.15.32:0/752305896 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fe00b0760 0x7f3fe00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fe5b39640 1 --2- 172.21.15.32:0/752305896 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fe0157480 0x7f3fe00c9840 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fe5b39640 1 -- 172.21.15.32:0/752305896 >> 172.21.15.32:0/752305896 conn(0x7f3fe006a510 msgr2=0x7f3fe00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fe5b39640 1 -- 172.21.15.32:0/752305896 shutdown_connections 2023-12-20T04:40:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fe5b39640 1 -- 172.21.15.32:0/752305896 wait complete. 2023-12-20T04:40:51.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fe5b39640 1 Processor -- start 2023-12-20T04:40:51.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.025+0000 7f3fe5b39640 1 -- start start 2023-12-20T04:40:51.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fe5b39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fe00b0760 0x7f3fe014ef40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fe5b39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fe0157480 0x7f3fe015bf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fe5b39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fe015d660 0x7f3fe015c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fe5b39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0166850 con 0x7f3fe0157480 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fe5b39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0166b30 con 0x7f3fe015d660 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fe5b39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fe0166e10 con 0x7f3fe00b0760 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fdeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fe0157480 0x7f3fe015bf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fdeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fe0157480 0x7f3fe015bf90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46172/0 (socket says 172.21.15.32:46172) 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fdeffd640 1 -- 172.21.15.32:0/3012213523 learned_addr learned my addr 172.21.15.32:0/3012213523 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fdffff640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fe015d660 0x7f3fe015c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fdf7fe640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fe00b0760 0x7f3fe014ef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fdeffd640 1 -- 172.21.15.32:0/3012213523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fe00b0760 msgr2=0x7f3fe014ef40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fdeffd640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fe00b0760 0x7f3fe014ef40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fdeffd640 1 -- 172.21.15.32:0/3012213523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fe015d660 msgr2=0x7f3fe015c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fdeffd640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fe015d660 0x7f3fe015c4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fdeffd640 1 -- 172.21.15.32:0/3012213523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd00094d0 con 0x7f3fe0157480 2023-12-20T04:40:51.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fdeffd640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fe0157480 0x7f3fe015bf90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400b410 tx=0x7f3fd400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fc7fff640 1 -- 172.21.15.32:0/3012213523 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4011830 con 0x7f3fe0157480 2023-12-20T04:40:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fc7fff640 1 -- 172.21.15.32:0/3012213523 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fd4011990 con 0x7f3fe0157480 2023-12-20T04:40:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fc7fff640 1 -- 172.21.15.32:0/3012213523 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fd4011d40 con 0x7f3fe0157480 2023-12-20T04:40:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fe5b39640 1 -- 172.21.15.32:0/3012213523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fe01670f0 con 0x7f3fe0157480 2023-12-20T04:40:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fe5b39640 1 -- 172.21.15.32:0/3012213523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fe014f480 con 0x7f3fe0157480 2023-12-20T04:40:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fe5b39640 1 -- 172.21.15.32:0/3012213523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fe004e000 con 0x7f3fe0157480 2023-12-20T04:40:51.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.029+0000 7f3fc7fff640 1 -- 172.21.15.32:0/3012213523 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3fd401a400 con 0x7f3fe0157480 2023-12-20T04:40:51.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.033+0000 7f3fc7fff640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3fcc065b80 0x7f3fcc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.033+0000 7f3fc7fff640 1 -- 172.21.15.32:0/3012213523 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f3fd40834e0 con 0x7f3fe0157480 2023-12-20T04:40:51.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.033+0000 7f3fdf7fe640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3fcc065b80 0x7f3fcc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.033+0000 7f3fdf7fe640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3fcc065b80 0x7f3fcc068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00af000 tx=0x7f3fd00a7000 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:51.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.037+0000 7f3fc7fff640 1 -- 172.21.15.32:0/3012213523 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3fd4050dc0 con 0x7f3fe0157480 2023-12-20T04:40:51.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.193+0000 7f3fe5b39640 1 -- 172.21.15.32:0/3012213523 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f3fe01555c0 con 0x7f3fe0157480 2023-12-20T04:40:51.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.197+0000 7f3fc7fff640 1 -- 172.21.15.32:0/3012213523 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v14) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3fd4081e60 con 0x7f3fe0157480 2023-12-20T04:40:51.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 -- 172.21.15.32:0/3012213523 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3fcc065b80 msgr2=0x7f3fcc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3fcc065b80 0x7f3fcc068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00af000 tx=0x7f3fd00a7000 comp rx=0 tx=0).stop 2023-12-20T04:40:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 -- 172.21.15.32:0/3012213523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fe0157480 msgr2=0x7f3fe015bf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fe0157480 0x7f3fe015bf90 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f3fd400b410 tx=0x7f3fd400b8d0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 -- 172.21.15.32:0/3012213523 shutdown_connections 2023-12-20T04:40:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3fcc065b80 0x7f3fcc068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fe015d660 0x7f3fe015c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fe0157480 0x7f3fe015bf90 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 --2- 172.21.15.32:0/3012213523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fe00b0760 0x7f3fe014ef40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 -- 172.21.15.32:0/3012213523 >> 172.21.15.32:0/3012213523 conn(0x7f3fe006a510 msgr2=0x7f3fe0162c10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 -- 172.21.15.32:0/3012213523 shutdown_connections 2023-12-20T04:40:51.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.201+0000 7f3fe5b39640 1 -- 172.21.15.32:0/3012213523 wait complete. 2023-12-20T04:40:51.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T04:40:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.365+0000 7fbf2b666640 1 Processor -- start 2023-12-20T04:40:51.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.365+0000 7fbf2b666640 1 -- start start 2023-12-20T04:40:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf2b666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240ca1a0 0x7fbf240cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf2b666640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf240cb0b0 con 0x7fbf24164670 2023-12-20T04:40:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf2b666640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf240ca810 con 0x7fbf24070230 2023-12-20T04:40:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf2b666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240ca600 con 0x7fbf240ca1a0 2023-12-20T04:40:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf29402640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbf24164670 0x7fbf2406da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46648/0 (socket says 172.21.15.32:46648) 2023-12-20T04:40:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf29402640 1 -- 172.21.15.32:0/4246248830 learned_addr learned my addr 172.21.15.32:0/4246248830 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf28c01640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240ca1a0 0x7fbf240cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2801458732 0 0) 0x7fbf240cb0b0 con 0x7fbf24164670 2023-12-20T04:40:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf10003530 con 0x7fbf24164670 2023-12-20T04:40:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4059926028 0 0) 0x7fbf240ca810 con 0x7fbf24070230 2023-12-20T04:40:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf240cb0b0 con 0x7fbf24070230 2023-12-20T04:40:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2644405564 0 0) 0x7fbf10003530 con 0x7fbf24164670 2023-12-20T04:40:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240ca1a0 msgr2=0x7fbf240cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240ca1a0 0x7fbf240cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 >> v1:172.21.15.39:6789/0 conn(0x7fbf24070230 legacy=0x7fbf2406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf240caaf0 con 0x7fbf24164670 2023-12-20T04:40:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fbf20001680 con 0x7fbf24164670 2023-12-20T04:40:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 >> v1:172.21.15.32:6789/0 conn(0x7fbf24164670 legacy=0x7fbf2406da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf10003b40 0x7fbf10003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf100047d0 0x7fbf10004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240ca1a0 0x7fbf10005240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240ca600 con 0x7fbf10003b40 2023-12-20T04:40:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1000bb50 con 0x7fbf100047d0 2023-12-20T04:40:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1000bd10 con 0x7fbf240ca1a0 2023-12-20T04:40:51.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf29402640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf10003b40 0x7fbf10003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf28c01640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf100047d0 0x7fbf10004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf29c03640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240ca1a0 0x7fbf10005240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf29402640 1 -- 172.21.15.32:0/4246248830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240ca1a0 msgr2=0x7fbf10005240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf29402640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240ca1a0 0x7fbf10005240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf29402640 1 -- 172.21.15.32:0/4246248830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf100047d0 msgr2=0x7fbf10004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf29402640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf100047d0 0x7fbf10004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.369+0000 7fbf29402640 1 -- 172.21.15.32:0/4246248830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10003530 con 0x7fbf10003b40 2023-12-20T04:40:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.373+0000 7fbf29402640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf10003b40 0x7fbf10003f40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000c190 tx=0x7fbf2001cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b455c12102a07cb5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.373+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf200058e0 con 0x7fbf10003b40 2023-12-20T04:40:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.373+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf20005a40 con 0x7fbf10003b40 2023-12-20T04:40:51.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.373+0000 7fbf1bfff640 1 -- 172.21.15.32:0/4246248830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf20005e10 con 0x7fbf10003b40 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.373+0000 7fbf2b666640 1 -- 172.21.15.32:0/4246248830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf10003b40 msgr2=0x7fbf10003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.373+0000 7fbf2b666640 1 --2- 172.21.15.32:0/4246248830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf10003b40 0x7fbf10003f40 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7fbf2000c190 tx=0x7fbf2001cbc0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.373+0000 7fbf29c03640 1 -- 172.21.15.32:0/4246248830 reap_dead start 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.373+0000 7fbf2b666640 1 -- 172.21.15.32:0/4246248830 shutdown_connections 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.373+0000 7fbf2b666640 1 -- 172.21.15.32:0/4246248830 >> 172.21.15.32:0/4246248830 conn(0x7fbf2406a510 msgr2=0x7fbf240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.373+0000 7fbf2b666640 1 -- 172.21.15.32:0/4246248830 shutdown_connections 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.373+0000 7fbf2b666640 1 -- 172.21.15.32:0/4246248830 wait complete. 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf2b666640 1 Processor -- start 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf2b666640 1 -- start start 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf2b666640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf10003b40 0x7fbf24157d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf2b666640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2415cdc0 0x7fbf2415d750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf2b666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240b2c10 0x7fbf240b3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf2b666640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf24158270 con 0x7fbf2415cdc0 2023-12-20T04:40:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf2b666640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240b35d0 con 0x7fbf10003b40 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf2b666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf240c7940 con 0x7fbf240b2c10 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf28c01640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2415cdc0 0x7fbf2415d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf28c01640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2415cdc0 0x7fbf2415d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46204/0 (socket says 172.21.15.32:46204) 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf28c01640 1 -- 172.21.15.32:0/1662867 learned_addr learned my addr 172.21.15.32:0/1662867 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf29402640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf10003b40 0x7fbf24157d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf29c03640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240b2c10 0x7fbf240b3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf28c01640 1 -- 172.21.15.32:0/1662867 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240b2c10 msgr2=0x7fbf240b3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf28c01640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240b2c10 0x7fbf240b3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf28c01640 1 -- 172.21.15.32:0/1662867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf10003b40 msgr2=0x7fbf24157d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf28c01640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf10003b40 0x7fbf24157d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf28c01640 1 -- 172.21.15.32:0/1662867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf24070bd0 con 0x7fbf2415cdc0 2023-12-20T04:40:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf28c01640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2415cdc0 0x7fbf2415d750 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbf14011960 tx=0x7fbf14011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf19ffb640 1 -- 172.21.15.32:0/1662867 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf1400fa90 con 0x7fbf2415cdc0 2023-12-20T04:40:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf19ffb640 1 -- 172.21.15.32:0/1662867 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf1400fc10 con 0x7fbf2415cdc0 2023-12-20T04:40:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf19ffb640 1 -- 172.21.15.32:0/1662867 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf14014440 con 0x7fbf2415cdc0 2023-12-20T04:40:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf2b666640 1 -- 172.21.15.32:0/1662867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf2415dc90 con 0x7fbf2415cdc0 2023-12-20T04:40:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.377+0000 7fbf2b666640 1 -- 172.21.15.32:0/1662867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf240c7010 con 0x7fbf2415cdc0 2023-12-20T04:40:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.381+0000 7fbf2b666640 1 -- 172.21.15.32:0/1662867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2404e000 con 0x7fbf2415cdc0 2023-12-20T04:40:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.381+0000 7fbf19ffb640 1 -- 172.21.15.32:0/1662867 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbf140145a0 con 0x7fbf2415cdc0 2023-12-20T04:40:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.381+0000 7fbf19ffb640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf08065b80 0x7fbf08068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.381+0000 7fbf19ffb640 1 -- 172.21.15.32:0/1662867 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fbf14083a50 con 0x7fbf2415cdc0 2023-12-20T04:40:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.381+0000 7fbf29402640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf08065b80 0x7fbf08068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.381+0000 7fbf29402640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf08065b80 0x7fbf08068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbf200aa000 tx=0x7fbf20001990 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:51.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.385+0000 7fbf19ffb640 1 -- 172.21.15.32:0/1662867 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbf140512d0 con 0x7fbf2415cdc0 2023-12-20T04:40:51.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.545+0000 7fbf2b666640 1 -- 172.21.15.32:0/1662867 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fbf241653c0 con 0x7fbf2415cdc0 2023-12-20T04:40:51.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.549+0000 7fbf19ffb640 1 -- 172.21.15.32:0/1662867 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v15) v1 ==== 104+0+0 (secure 0 0 0) 0x7fbf140220a0 con 0x7fbf2415cdc0 2023-12-20T04:40:51.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 -- 172.21.15.32:0/1662867 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf08065b80 msgr2=0x7fbf08068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf08065b80 0x7fbf08068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fbf200aa000 tx=0x7fbf20001990 comp rx=0 tx=0).stop 2023-12-20T04:40:51.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 -- 172.21.15.32:0/1662867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2415cdc0 msgr2=0x7fbf2415d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2415cdc0 0x7fbf2415d750 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7fbf14011960 tx=0x7fbf14011e20 comp rx=0 tx=0).stop 2023-12-20T04:40:51.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 -- 172.21.15.32:0/1662867 shutdown_connections 2023-12-20T04:40:51.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf08065b80 0x7fbf08068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf240b2c10 0x7fbf240b3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf2415cdc0 0x7fbf2415d750 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 --2- 172.21.15.32:0/1662867 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf10003b40 0x7fbf24157d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 -- 172.21.15.32:0/1662867 >> 172.21.15.32:0/1662867 conn(0x7fbf2406a510 msgr2=0x7fbf240ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:51.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 -- 172.21.15.32:0/1662867 shutdown_connections 2023-12-20T04:40:51.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.553+0000 7fbf2b666640 1 -- 172.21.15.32:0/1662867 wait complete. 2023-12-20T04:40:51.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates' 2023-12-20T04:40:51.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.717+0000 7fce5c466640 1 Processor -- start 2023-12-20T04:40:51.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.717+0000 7fce5c466640 1 -- start start 2023-12-20T04:40:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.717+0000 7fce5c466640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fce5406cf10 0x7fce540c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.717+0000 7fce5c466640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce540ae3f0 con 0x7fce54164ef0 2023-12-20T04:40:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.717+0000 7fce5c466640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce54161800 con 0x7fce540ae010 2023-12-20T04:40:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.717+0000 7fce5c466640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce54165380 con 0x7fce5406cf10 2023-12-20T04:40:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.717+0000 7fce5aa03640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fce54164ef0 0x7fce540c99e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46664/0 (socket says 172.21.15.32:46664) 2023-12-20T04:40:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.717+0000 7fce5aa03640 1 -- 172.21.15.32:0/4244453467 learned_addr learned my addr 172.21.15.32:0/4244453467 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.717+0000 7fce59a01640 1 --2- 172.21.15.32:0/4244453467 >> v2:172.21.15.39:3300/0 conn(0x7fce5406cf10 0x7fce540c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.721+0000 7fce59200640 1 -- 172.21.15.32:0/4244453467 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2950096214 0 0) 0x7fce540ae3f0 con 0x7fce54164ef0 2023-12-20T04:40:51.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.721+0000 7fce59200640 1 -- 172.21.15.32:0/4244453467 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce40003530 con 0x7fce54164ef0 2023-12-20T04:40:51.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.721+0000 7fce59200640 1 -- 172.21.15.32:0/4244453467 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 898551651 0 0) 0x7fce54161800 con 0x7fce540ae010 2023-12-20T04:40:51.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.721+0000 7fce59200640 1 -- 172.21.15.32:0/4244453467 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce540ae3f0 con 0x7fce540ae010 2023-12-20T04:40:51.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.721+0000 7fce59a01640 1 -- 172.21.15.32:0/4244453467 >> v1:172.21.15.39:6789/0 conn(0x7fce540ae010 legacy=0x7fce54071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.721+0000 7fce59a01640 1 -- 172.21.15.32:0/4244453467 >> v1:172.21.15.32:6789/0 conn(0x7fce54164ef0 legacy=0x7fce540c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.721+0000 7fce59a01640 1 -- 172.21.15.32:0/4244453467 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce54165660 con 0x7fce5406cf10 2023-12-20T04:40:51.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.721+0000 7fce59a01640 1 --2- 172.21.15.32:0/4244453467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce5406cf10 0x7fce540c94a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fce440097c0 tx=0x7fce4401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8df75d469e3e670b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.721+0000 7fce59200640 1 -- 172.21.15.32:0/4244453467 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440ac620 con 0x7fce5406cf10 2023-12-20T04:40:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.721+0000 7fce59200640 1 -- 172.21.15.32:0/4244453467 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce440ac780 con 0x7fce5406cf10 2023-12-20T04:40:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.721+0000 7fce59200640 1 -- 172.21.15.32:0/4244453467 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440acb10 con 0x7fce5406cf10 2023-12-20T04:40:51.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 -- 172.21.15.32:0/4244453467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce5406cf10 msgr2=0x7fce540c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 --2- 172.21.15.32:0/4244453467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce5406cf10 0x7fce540c94a0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fce440097c0 tx=0x7fce4401c3c0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 -- 172.21.15.32:0/4244453467 shutdown_connections 2023-12-20T04:40:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 --2- 172.21.15.32:0/4244453467 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce5406cf10 0x7fce540c94a0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 -- 172.21.15.32:0/4244453467 >> 172.21.15.32:0/4244453467 conn(0x7fce5406a320 msgr2=0x7fce540ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 -- 172.21.15.32:0/4244453467 shutdown_connections 2023-12-20T04:40:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 -- 172.21.15.32:0/4244453467 wait complete. 2023-12-20T04:40:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 Processor -- start 2023-12-20T04:40:51.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 -- start start 2023-12-20T04:40:51.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce540ae010 0x7fce54158200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce5414ec80 0x7fce540b1440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce540b1980 0x7fce540b1e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540b3610 con 0x7fce5414ec80 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce540b42d0 con 0x7fce540b1980 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5c466640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce5414f640 con 0x7fce540ae010 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce59a01640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce5414ec80 0x7fce540b1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce59a01640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce5414ec80 0x7fce540b1440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46208/0 (socket says 172.21.15.32:46208) 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce59a01640 1 -- 172.21.15.32:0/874234077 learned_addr learned my addr 172.21.15.32:0/874234077 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5aa03640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce540b1980 0x7fce540b1e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.725+0000 7fce5a202640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce540ae010 0x7fce54158200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce59a01640 1 -- 172.21.15.32:0/874234077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce540ae010 msgr2=0x7fce54158200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce59a01640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce540ae010 0x7fce54158200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce59a01640 1 -- 172.21.15.32:0/874234077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce540b1980 msgr2=0x7fce540b1e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce59a01640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce540b1980 0x7fce540b1e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce59a01640 1 -- 172.21.15.32:0/874234077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce44009420 con 0x7fce5414ec80 2023-12-20T04:40:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce5a202640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce540ae010 0x7fce54158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce59a01640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce5414ec80 0x7fce540b1440 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fce440097c0 tx=0x7fce44019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce4affd640 1 -- 172.21.15.32:0/874234077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440ac620 con 0x7fce5414ec80 2023-12-20T04:40:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce4affd640 1 -- 172.21.15.32:0/874234077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce440b5410 con 0x7fce5414ec80 2023-12-20T04:40:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce4affd640 1 -- 172.21.15.32:0/874234077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce440b57c0 con 0x7fce5414ec80 2023-12-20T04:40:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce5c466640 1 -- 172.21.15.32:0/874234077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce5414f100 con 0x7fce5414ec80 2023-12-20T04:40:51.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce5c466640 1 -- 172.21.15.32:0/874234077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce540b37f0 con 0x7fce5414ec80 2023-12-20T04:40:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce5c466640 1 -- 172.21.15.32:0/874234077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce5404e000 con 0x7fce5414ec80 2023-12-20T04:40:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce4affd640 1 -- 172.21.15.32:0/874234077 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fce440b3040 con 0x7fce5414ec80 2023-12-20T04:40:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce4affd640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fce38065b80 0x7fce38068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce4affd640 1 -- 172.21.15.32:0/874234077 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5502+0+0 (secure 0 0 0) 0x7fce4411d1f0 con 0x7fce5414ec80 2023-12-20T04:40:51.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.729+0000 7fce5a202640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fce38065b80 0x7fce38068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:51.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.733+0000 7fce5a202640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fce38065b80 0x7fce38068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fce50001240 tx=0x7fce50009040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:51.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.733+0000 7fce4affd640 1 -- 172.21.15.32:0/874234077 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fce440eaae0 con 0x7fce5414ec80 2023-12-20T04:40:51.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:51.945+0000 7fce5c466640 1 -- 172.21.15.32:0/874234077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]} v 0) v1 -- 0x7fce540cf450 con 0x7fce5414ec80 2023-12-20T04:40:52.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.465+0000 7fce4affd640 1 -- 172.21.15.32:0/874234077 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_concurrent_snap_creates"]}]=0 v441) v1 ==== 158+0+0 (secure 0 0 0) 0x7fce440ee100 con 0x7fce5414ec80 2023-12-20T04:40:52.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 -- 172.21.15.32:0/874234077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fce38065b80 msgr2=0x7fce38068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fce38065b80 0x7fce38068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fce50001240 tx=0x7fce50009040 comp rx=0 tx=0).stop 2023-12-20T04:40:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 -- 172.21.15.32:0/874234077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce5414ec80 msgr2=0x7fce540b1440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce5414ec80 0x7fce540b1440 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7fce440097c0 tx=0x7fce44019160 comp rx=0 tx=0).stop 2023-12-20T04:40:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 -- 172.21.15.32:0/874234077 shutdown_connections 2023-12-20T04:40:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fce38065b80 0x7fce38068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce540b1980 0x7fce540b1e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce5414ec80 0x7fce540b1440 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 --2- 172.21.15.32:0/874234077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce540ae010 0x7fce54158200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 -- 172.21.15.32:0/874234077 >> 172.21.15.32:0/874234077 conn(0x7fce5406a320 msgr2=0x7fce540c94a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 -- 172.21.15.32:0/874234077 shutdown_connections 2023-12-20T04:40:52.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.469+0000 7fce5c466640 1 -- 172.21.15.32:0/874234077 wait complete. 2023-12-20T04:40:52.492 INFO:tasks.cephfs_test_runner:Test concurrent snap creates in same file-system without db issues ... ok 2023-12-20T04:40:52.492 INFO:tasks.cephfs_test_runner:Starting test: test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test exisitence of multiple scheduled snapshots 2023-12-20T04:40:52.492 INFO:tasks.cephfs_test_runner:test_multi_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-20T04:40:52.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-12-20T04:40:52.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.641+0000 7efe1efa6640 1 Processor -- start 2023-12-20T04:40:52.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.641+0000 7efe1efa6640 1 -- start start 2023-12-20T04:40:52.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.641+0000 7efe1efa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe181599e0 0x7efe180aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:52.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.641+0000 7efe1efa6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efe180c90e0 0x7efe180b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:52.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.641+0000 7efe1efa6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe180711d0 con 0x7efe180b0b30 2023-12-20T04:40:52.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.641+0000 7efe1efa6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe180c98c0 con 0x7efe180c90e0 2023-12-20T04:40:52.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.641+0000 7efe1efa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe180c9ba0 con 0x7efe181599e0 2023-12-20T04:40:52.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.645+0000 7efe1d543640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efe180b0b30 0x7efe1815d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46680/0 (socket says 172.21.15.32:46680) 2023-12-20T04:40:52.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.645+0000 7efe1d543640 1 -- 172.21.15.32:0/2646411504 learned_addr learned my addr 172.21.15.32:0/2646411504 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:52.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.645+0000 7efe17fff640 1 --2- 172.21.15.32:0/2646411504 >> v2:172.21.15.39:3300/0 conn(0x7efe180c90e0 0x7efe180b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:52.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.645+0000 7efe1cd42640 1 --2- 172.21.15.32:0/2646411504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe181599e0 0x7efe180aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:52.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.645+0000 7efe177fe640 1 -- 172.21.15.32:0/2646411504 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1023784862 0 0) 0x7efe180711d0 con 0x7efe180b0b30 2023-12-20T04:40:52.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.645+0000 7efe177fe640 1 -- 172.21.15.32:0/2646411504 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe08003530 con 0x7efe180b0b30 2023-12-20T04:40:52.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.645+0000 7efe17fff640 1 -- 172.21.15.32:0/2646411504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe181599e0 msgr2=0x7efe180aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:52.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.645+0000 7efe17fff640 1 --2- 172.21.15.32:0/2646411504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe181599e0 0x7efe180aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:52.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.645+0000 7efe17fff640 1 -- 172.21.15.32:0/2646411504 >> v1:172.21.15.32:6789/0 conn(0x7efe180b0b30 legacy=0x7efe1815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:52.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.645+0000 7efe17fff640 1 -- 172.21.15.32:0/2646411504 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe180c9e80 con 0x7efe180c90e0 2023-12-20T04:40:52.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.645+0000 7efe17fff640 1 --2- 172.21.15.32:0/2646411504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180c90e0 0x7efe180b05f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe040097c0 tx=0x7efe0401c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b4e0fd4b6acab5c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:52.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe177fe640 1 -- 172.21.15.32:0/2646411504 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe040ab600 con 0x7efe180c90e0 2023-12-20T04:40:52.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe177fe640 1 -- 172.21.15.32:0/2646411504 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe040a7050 con 0x7efe180c90e0 2023-12-20T04:40:52.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe177fe640 1 -- 172.21.15.32:0/2646411504 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe040ab960 con 0x7efe180c90e0 2023-12-20T04:40:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 -- 172.21.15.32:0/2646411504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180c90e0 msgr2=0x7efe180b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 --2- 172.21.15.32:0/2646411504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180c90e0 0x7efe180b05f0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7efe040097c0 tx=0x7efe0401c450 comp rx=0 tx=0).stop 2023-12-20T04:40:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 -- 172.21.15.32:0/2646411504 shutdown_connections 2023-12-20T04:40:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 --2- 172.21.15.32:0/2646411504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180c90e0 0x7efe180b05f0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 --2- 172.21.15.32:0/2646411504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe181599e0 0x7efe180aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 -- 172.21.15.32:0/2646411504 >> 172.21.15.32:0/2646411504 conn(0x7efe1806a320 msgr2=0x7efe180ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 -- 172.21.15.32:0/2646411504 shutdown_connections 2023-12-20T04:40:52.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 -- 172.21.15.32:0/2646411504 wait complete. 2023-12-20T04:40:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 Processor -- start 2023-12-20T04:40:52.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 -- start start 2023-12-20T04:40:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180b0b30 0x7efe1815a150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe180c90e0 0x7efe181f6d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe181599e0 0x7efe181f72b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180b30d0 con 0x7efe181599e0 2023-12-20T04:40:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe180c6b30 con 0x7efe180b0b30 2023-12-20T04:40:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1efa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe1815a690 con 0x7efe180c90e0 2023-12-20T04:40:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1d543640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe181599e0 0x7efe181f72b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1d543640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe181599e0 0x7efe181f72b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46210/0 (socket says 172.21.15.32:46210) 2023-12-20T04:40:52.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1d543640 1 -- 172.21.15.32:0/3681198094 learned_addr learned my addr 172.21.15.32:0/3681198094 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:52.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe1cd42640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180b0b30 0x7efe1815a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:52.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.649+0000 7efe17fff640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe180c90e0 0x7efe181f6d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:52.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe1d543640 1 -- 172.21.15.32:0/3681198094 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe180c90e0 msgr2=0x7efe181f6d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:52.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe1d543640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe180c90e0 0x7efe181f6d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:52.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe1d543640 1 -- 172.21.15.32:0/3681198094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180b0b30 msgr2=0x7efe1815a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:52.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe1d543640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180b0b30 0x7efe1815a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:52.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe1d543640 1 -- 172.21.15.32:0/3681198094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe04009420 con 0x7efe181599e0 2023-12-20T04:40:52.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe17fff640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe180c90e0 0x7efe181f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:52.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe1d543640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe181599e0 0x7efe181f72b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efe1000cb40 tx=0x7efe1000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:52.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe157fa640 1 -- 172.21.15.32:0/3681198094 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe1000aca0 con 0x7efe181599e0 2023-12-20T04:40:52.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe157fa640 1 -- 172.21.15.32:0/3681198094 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe1000ae00 con 0x7efe181599e0 2023-12-20T04:40:52.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe157fa640 1 -- 172.21.15.32:0/3681198094 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe10012620 con 0x7efe181599e0 2023-12-20T04:40:52.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe1efa6640 1 -- 172.21.15.32:0/3681198094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe180c85d0 con 0x7efe181599e0 2023-12-20T04:40:52.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe1efa6640 1 -- 172.21.15.32:0/3681198094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe180c67c0 con 0x7efe181599e0 2023-12-20T04:40:52.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe1efa6640 1 -- 172.21.15.32:0/3681198094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe180c6ce0 con 0x7efe181599e0 2023-12-20T04:40:52.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.653+0000 7efe157fa640 1 -- 172.21.15.32:0/3681198094 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7efe10007070 con 0x7efe181599e0 2023-12-20T04:40:52.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.657+0000 7efe157fa640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efdfc065b80 0x7efdfc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:52.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.657+0000 7efe157fa640 1 -- 172.21.15.32:0/3681198094 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5502+0+0 (secure 0 0 0) 0x7efe10083280 con 0x7efe181599e0 2023-12-20T04:40:52.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.657+0000 7efe1cd42640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efdfc065b80 0x7efdfc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:52.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.657+0000 7efe1cd42640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efdfc065b80 0x7efdfc068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009aa0 tx=0x7efe0c006f90 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:52.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.661+0000 7efe157fa640 1 -- 172.21.15.32:0/3681198094 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efe180c6ce0 con 0x7efe181599e0 2023-12-20T04:40:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:52.873+0000 7efe1efa6640 1 -- 172.21.15.32:0/3681198094 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7efe180c6ce0 con 0x7efe181599e0 2023-12-20T04:40:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.469+0000 7efe157fa640 1 -- 172.21.15.32:0/3681198094 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v442) v1 ==== 157+0+0 (secure 0 0 0) 0x7efe180c6ce0 con 0x7efe181599e0 2023-12-20T04:40:53.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.469+0000 7efe1efa6640 1 -- 172.21.15.32:0/3681198094 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efdfc065b80 msgr2=0x7efdfc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:53.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.469+0000 7efe1efa6640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efdfc065b80 0x7efdfc068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7efe0c009aa0 tx=0x7efe0c006f90 comp rx=0 tx=0).stop 2023-12-20T04:40:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.469+0000 7efe1efa6640 1 -- 172.21.15.32:0/3681198094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe181599e0 msgr2=0x7efe181f72b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.469+0000 7efe1efa6640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe181599e0 0x7efe181f72b0 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efe1000cb40 tx=0x7efe1000a060 comp rx=0 tx=0).stop 2023-12-20T04:40:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.473+0000 7efe1efa6640 1 -- 172.21.15.32:0/3681198094 shutdown_connections 2023-12-20T04:40:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.473+0000 7efe1efa6640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efdfc065b80 0x7efdfc068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.473+0000 7efe1efa6640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe181599e0 0x7efe181f72b0 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.473+0000 7efe1efa6640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe180c90e0 0x7efe181f6d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.473+0000 7efe1efa6640 1 --2- 172.21.15.32:0/3681198094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe180b0b30 0x7efe1815a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.473+0000 7efe1efa6640 1 -- 172.21.15.32:0/3681198094 >> 172.21.15.32:0/3681198094 conn(0x7efe1806a320 msgr2=0x7efe1814fbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:53.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.473+0000 7efe1efa6640 1 -- 172.21.15.32:0/3681198094 shutdown_connections 2023-12-20T04:40:53.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.473+0000 7efe1efa6640 1 -- 172.21.15.32:0/3681198094 wait complete. 2023-12-20T04:40:53.495 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T04:40:53.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.645+0000 7f81e42fc640 1 Processor -- start 2023-12-20T04:40:53.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.645+0000 7f81e42fc640 1 -- start start 2023-12-20T04:40:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.645+0000 7f81e42fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81dc15abb0 0x7f81dc0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.645+0000 7f81e42fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81dc0b0cb0 0x7f81dc15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e42fc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81dc15b6a0 con 0x7f81dc070230 2023-12-20T04:40:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e42fc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81dc06e160 con 0x7f81dc15abb0 2023-12-20T04:40:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e42fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc15c070 con 0x7f81dc0b0cb0 2023-12-20T04:40:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e2098640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81dc15abb0 0x7f81dc0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e2098640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81dc15abb0 0x7f81dc0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46222/0 (socket says 172.21.15.32:46222) 2023-12-20T04:40:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e2098640 1 -- 172.21.15.32:0/3541519683 learned_addr learned my addr 172.21.15.32:0/3541519683 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e2098640 1 -- 172.21.15.32:0/3541519683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81dc0b0cb0 msgr2=0x7f81dc15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e1897640 1 --2- 172.21.15.32:0/3541519683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81dc0b0cb0 0x7f81dc15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e2098640 1 --2- 172.21.15.32:0/3541519683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81dc0b0cb0 0x7f81dc15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e2098640 1 -- 172.21.15.32:0/3541519683 >> v1:172.21.15.39:6789/0 conn(0x7f81dc070230 legacy=0x7f81dc06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:53.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e2098640 1 -- 172.21.15.32:0/3541519683 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81dc06d8c0 con 0x7f81dc15abb0 2023-12-20T04:40:53.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e1897640 1 --2- 172.21.15.32:0/3541519683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81dc0b0cb0 0x7f81dc15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:53.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e2098640 1 --2- 172.21.15.32:0/3541519683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81dc15abb0 0x7f81dc0caaf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009870 tx=0x7f81d801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33bf03fdb442d9cd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:53.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.649+0000 7f81e1096640 1 -- 172.21.15.32:0/3541519683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d80a9400 con 0x7f81dc15abb0 2023-12-20T04:40:53.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e1096640 1 -- 172.21.15.32:0/3541519683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d80a9560 con 0x7f81dc15abb0 2023-12-20T04:40:53.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e1096640 1 -- 172.21.15.32:0/3541519683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d80a98f0 con 0x7f81dc15abb0 2023-12-20T04:40:53.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 -- 172.21.15.32:0/3541519683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81dc15abb0 msgr2=0x7f81dc0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:53.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 --2- 172.21.15.32:0/3541519683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81dc15abb0 0x7f81dc0caaf0 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7f81d8009870 tx=0x7f81d801c540 comp rx=0 tx=0).stop 2023-12-20T04:40:53.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 -- 172.21.15.32:0/3541519683 shutdown_connections 2023-12-20T04:40:53.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 --2- 172.21.15.32:0/3541519683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81dc0b0cb0 0x7f81dc15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 --2- 172.21.15.32:0/3541519683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81dc15abb0 0x7f81dc0caaf0 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 -- 172.21.15.32:0/3541519683 >> 172.21.15.32:0/3541519683 conn(0x7f81dc06a510 msgr2=0x7f81dc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:53.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 -- 172.21.15.32:0/3541519683 shutdown_connections 2023-12-20T04:40:53.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 -- 172.21.15.32:0/3541519683 wait complete. 2023-12-20T04:40:53.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 Processor -- start 2023-12-20T04:40:53.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 -- start start 2023-12-20T04:40:53.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81dc070230 0x7f81dc13a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:53.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81dc0b0cb0 0x7f81dc0c90f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:53.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81dc15abb0 0x7f81dc13ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:53.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc1508e0 con 0x7f81dc15abb0 2023-12-20T04:40:53.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc14fcd0 con 0x7f81dc0b0cb0 2023-12-20T04:40:53.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.653+0000 7f81e42fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81dc14ffb0 con 0x7f81dc070230 2023-12-20T04:40:53.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e2899640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81dc15abb0 0x7f81dc13ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:53.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e2098640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81dc070230 0x7f81dc13a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e2899640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81dc15abb0 0x7f81dc13ff10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46230/0 (socket says 172.21.15.32:46230) 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e2899640 1 -- 172.21.15.32:0/238039230 learned_addr learned my addr 172.21.15.32:0/238039230 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e1897640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81dc0b0cb0 0x7f81dc0c90f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e2899640 1 -- 172.21.15.32:0/238039230 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81dc070230 msgr2=0x7f81dc13a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e2899640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81dc070230 0x7f81dc13a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e2899640 1 -- 172.21.15.32:0/238039230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81dc0b0cb0 msgr2=0x7f81dc0c90f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e2899640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81dc0b0cb0 0x7f81dc0c90f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e2899640 1 -- 172.21.15.32:0/238039230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81d80094d0 con 0x7f81dc15abb0 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e2899640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81dc15abb0 0x7f81dc13ff10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f81d400c730 tx=0x7f81d400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81d2ffd640 1 -- 172.21.15.32:0/238039230 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d4003b80 con 0x7f81dc15abb0 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81d2ffd640 1 -- 172.21.15.32:0/238039230 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81d4003ce0 con 0x7f81dc15abb0 2023-12-20T04:40:53.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81d2ffd640 1 -- 172.21.15.32:0/238039230 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81d400aa30 con 0x7f81dc15abb0 2023-12-20T04:40:53.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e42fc640 1 -- 172.21.15.32:0/238039230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81dc150290 con 0x7f81dc15abb0 2023-12-20T04:40:53.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e42fc640 1 -- 172.21.15.32:0/238039230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81dc13a660 con 0x7f81dc15abb0 2023-12-20T04:40:53.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81e42fc640 1 -- 172.21.15.32:0/238039230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81dc04e000 con 0x7f81dc15abb0 2023-12-20T04:40:53.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.657+0000 7f81d2ffd640 1 -- 172.21.15.32:0/238039230 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f81d400ab90 con 0x7f81dc15abb0 2023-12-20T04:40:53.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.661+0000 7f81d2ffd640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81c0065b80 0x7f81c0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:53.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.661+0000 7f81d2ffd640 1 -- 172.21.15.32:0/238039230 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f81d4082ec0 con 0x7f81dc15abb0 2023-12-20T04:40:53.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.661+0000 7f81e2098640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81c0065b80 0x7f81c0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:53.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.661+0000 7f81e2098640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81c0065b80 0x7f81c0068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f81d80ae000 tx=0x7f81d80b3040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:53.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.665+0000 7f81d2ffd640 1 -- 172.21.15.32:0/238039230 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f81d40508e0 con 0x7f81dc15abb0 2023-12-20T04:40:53.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.821+0000 7f81e42fc640 1 -- 172.21.15.32:0/238039230 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f81dc150a20 con 0x7f81dc15abb0 2023-12-20T04:40:53.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.825+0000 7f81d2ffd640 1 -- 172.21.15.32:0/238039230 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v16)=0 v16) v1 ==== 122+0+0 (secure 0 0 0) 0x7f81dc150a20 con 0x7f81dc15abb0 2023-12-20T04:40:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 -- 172.21.15.32:0/238039230 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81c0065b80 msgr2=0x7f81c0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81c0065b80 0x7f81c0068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f81d80ae000 tx=0x7f81d80b3040 comp rx=0 tx=0).stop 2023-12-20T04:40:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 -- 172.21.15.32:0/238039230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81dc15abb0 msgr2=0x7f81dc13ff10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81dc15abb0 0x7f81dc13ff10 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f81d400c730 tx=0x7f81d400cbf0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 -- 172.21.15.32:0/238039230 shutdown_connections 2023-12-20T04:40:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81c0065b80 0x7f81c0068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81dc15abb0 0x7f81dc13ff10 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81dc0b0cb0 0x7f81dc0c90f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 --2- 172.21.15.32:0/238039230 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81dc070230 0x7f81dc13a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:53.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 -- 172.21.15.32:0/238039230 >> 172.21.15.32:0/238039230 conn(0x7f81dc06a510 msgr2=0x7f81dc150f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:53.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 -- 172.21.15.32:0/238039230 shutdown_connections 2023-12-20T04:40:53.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:53.829+0000 7f81e42fc640 1 -- 172.21.15.32:0/238039230 wait complete. 2023-12-20T04:40:53.848 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:40:53.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:40:53.957 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:40:54.034 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:40:54.034 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:40:54.034 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:40:54.044 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:40:54.044 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:40:54.044 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:40:54.054 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:40:54.054 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:40:54.054 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:40:54.095 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:40:54.095 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:40:54.095 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:40:54.105 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:40:54.105 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T04:40:54.105 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:40:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.281+0000 7f7ead7a2640 1 Processor -- start 2023-12-20T04:40:54.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.281+0000 7f7ead7a2640 1 -- start start 2023-12-20T04:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7ead7a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ea815ace0 0x7f7ea80b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7ead7a2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ea80b0700 0x7f7ea80cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7ead7a2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ea8070570 con 0x7f7ea80709d0 2023-12-20T04:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7ead7a2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ea80cab80 con 0x7f7ea80b0700 2023-12-20T04:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7ead7a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea80c9c20 con 0x7f7ea815ace0 2023-12-20T04:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7eacfa1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ea80b0700 0x7f7ea80cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7eacfa1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ea80b0700 0x7f7ea80cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46240/0 (socket says 172.21.15.32:46240) 2023-12-20T04:40:54.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7eacfa1640 1 -- 172.21.15.32:0/1137711838 learned_addr learned my addr 172.21.15.32:0/1137711838 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7eacfa1640 1 -- 172.21.15.32:0/1137711838 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ea815ace0 msgr2=0x7f7ea80b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7ea7fff640 1 --2- 172.21.15.32:0/1137711838 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ea815ace0 0x7f7ea80b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7eacfa1640 1 --2- 172.21.15.32:0/1137711838 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ea815ace0 0x7f7ea80b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7eacfa1640 1 -- 172.21.15.32:0/1137711838 >> v1:172.21.15.39:6789/0 conn(0x7f7ea80709d0 legacy=0x7f7ea80afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7eacfa1640 1 -- 172.21.15.32:0/1137711838 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea80c9f00 con 0x7f7ea80b0700 2023-12-20T04:40:54.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.285+0000 7f7ea7fff640 1 --2- 172.21.15.32:0/1137711838 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ea815ace0 0x7f7ea80b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:54.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7eacfa1640 1 --2- 172.21.15.32:0/1137711838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea80b0700 0x7f7ea80cafa0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009e50 tx=0x7f7ea001c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95fa6286facdf568 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:54.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ea6ffd640 1 -- 172.21.15.32:0/1137711838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00a57c0 con 0x7f7ea80b0700 2023-12-20T04:40:54.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ea6ffd640 1 -- 172.21.15.32:0/1137711838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ea00a5920 con 0x7f7ea80b0700 2023-12-20T04:40:54.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ea6ffd640 1 -- 172.21.15.32:0/1137711838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ea00a5cb0 con 0x7f7ea80b0700 2023-12-20T04:40:54.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ead7a2640 1 -- 172.21.15.32:0/1137711838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea80b0700 msgr2=0x7f7ea80cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:54.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ead7a2640 1 --2- 172.21.15.32:0/1137711838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea80b0700 0x7f7ea80cafa0 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f7ea0009e50 tx=0x7f7ea001c9e0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ead7a2640 1 -- 172.21.15.32:0/1137711838 shutdown_connections 2023-12-20T04:40:54.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ead7a2640 1 --2- 172.21.15.32:0/1137711838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea80b0700 0x7f7ea80cafa0 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ead7a2640 1 --2- 172.21.15.32:0/1137711838 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ea815ace0 0x7f7ea80b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ead7a2640 1 -- 172.21.15.32:0/1137711838 >> 172.21.15.32:0/1137711838 conn(0x7f7ea806c060 msgr2=0x7f7ea806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:54.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ead7a2640 1 -- 172.21.15.32:0/1137711838 shutdown_connections 2023-12-20T04:40:54.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ead7a2640 1 -- 172.21.15.32:0/1137711838 wait complete. 2023-12-20T04:40:54.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ead7a2640 1 Processor -- start 2023-12-20T04:40:54.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.289+0000 7f7ead7a2640 1 -- start start 2023-12-20T04:40:54.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ead7a2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea80709d0 0x7f7ea8151900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ead7a2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ea80b0700 0x7f7ea80c7e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ead7a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ea815ace0 0x7f7ea81f8d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ead7a2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea815cc00 con 0x7f7ea80709d0 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ead7a2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea815cee0 con 0x7f7ea80b0700 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ead7a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ea815d1c0 con 0x7f7ea815ace0 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea80709d0 0x7f7ea8151900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea80709d0 0x7f7ea8151900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46256/0 (socket says 172.21.15.32:46256) 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea7fff640 1 -- 172.21.15.32:0/2922811633 learned_addr learned my addr 172.21.15.32:0/2922811633 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea77fe640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ea80b0700 0x7f7ea80c7e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7eacfa1640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ea815ace0 0x7f7ea81f8d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea7fff640 1 -- 172.21.15.32:0/2922811633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ea815ace0 msgr2=0x7f7ea81f8d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea7fff640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ea815ace0 0x7f7ea81f8d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea7fff640 1 -- 172.21.15.32:0/2922811633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ea80b0700 msgr2=0x7f7ea80c7e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:54.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea7fff640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ea80b0700 0x7f7ea80c7e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea7fff640 1 -- 172.21.15.32:0/2922811633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ea0009b30 con 0x7f7ea80709d0 2023-12-20T04:40:54.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea7fff640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea80709d0 0x7f7ea8151900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800c7f0 tx=0x7f7e9800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:54.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea4ff9640 1 -- 172.21.15.32:0/2922811633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98007820 con 0x7f7ea80709d0 2023-12-20T04:40:54.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea4ff9640 1 -- 172.21.15.32:0/2922811633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e98007980 con 0x7f7ea80709d0 2023-12-20T04:40:54.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ead7a2640 1 -- 172.21.15.32:0/2922811633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ea815d4a0 con 0x7f7ea80709d0 2023-12-20T04:40:54.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea4ff9640 1 -- 172.21.15.32:0/2922811633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e98007d30 con 0x7f7ea80709d0 2023-12-20T04:40:54.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ead7a2640 1 -- 172.21.15.32:0/2922811633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ea815dca0 con 0x7f7ea80709d0 2023-12-20T04:40:54.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ead7a2640 1 -- 172.21.15.32:0/2922811633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ea8074100 con 0x7f7ea80709d0 2023-12-20T04:40:54.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.293+0000 7f7ea4ff9640 1 -- 172.21.15.32:0/2922811633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7e980113f0 con 0x7f7ea80709d0 2023-12-20T04:40:54.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.297+0000 7f7ea4ff9640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e94065b80 0x7f7e94068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.297+0000 7f7ea4ff9640 1 -- 172.21.15.32:0/2922811633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(52..52 src has 1..52) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f7e980835a0 con 0x7f7ea80709d0 2023-12-20T04:40:54.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.297+0000 7f7ea77fe640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e94065b80 0x7f7e94068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.297+0000 7f7ea77fe640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e94065b80 0x7f7e94068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7ea81f8cd0 tx=0x7f7e9c009040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:54.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.301+0000 7f7ea4ff9640 1 -- 172.21.15.32:0/2922811633 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7e98050ea0 con 0x7f7ea80709d0 2023-12-20T04:40:54.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.489+0000 7f7ead7a2640 1 -- 172.21.15.32:0/2922811633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7ea80c8d30 con 0x7f7ea80709d0 2023-12-20T04:40:54.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.489+0000 7f7ea4ff9640 1 -- 172.21.15.32:0/2922811633 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 18 v18) v1 ==== 94+0+3529 (secure 0 0 0) 0x7f7e9801fde0 con 0x7f7ea80709d0 2023-12-20T04:40:54.490 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:54.490 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":18,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":4763,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2541430692","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2541430692},{"type":"v1","addr":"172.21.15.39:6835","nonce":2541430692}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4996,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1379533068","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1379533068},{"type":"v1","addr":"172.21.15.32:6837","nonce":1379533068}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":4999,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4021567802","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4021567802},{"type":"v1","addr":"172.21.15.39:6834","nonce":4021567802}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":11},{"gid":5805,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/2566907975","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2566907975},{"type":"v1","addr":"172.21.15.32:6836","nonce":2566907975}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":18}],"filesystems":[]} 2023-12-20T04:40:54.490 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 18 2023-12-20T04:40:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 -- 172.21.15.32:0/2922811633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e94065b80 msgr2=0x7f7e94068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e94065b80 0x7f7e94068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7ea81f8cd0 tx=0x7f7e9c009040 comp rx=0 tx=0).stop 2023-12-20T04:40:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 -- 172.21.15.32:0/2922811633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea80709d0 msgr2=0x7f7ea8151900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea80709d0 0x7f7ea8151900 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f7e9800c7f0 tx=0x7f7e9800ccb0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 -- 172.21.15.32:0/2922811633 shutdown_connections 2023-12-20T04:40:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e94065b80 0x7f7e94068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ea815ace0 0x7f7ea81f8d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ea80b0700 0x7f7ea80c7e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 --2- 172.21.15.32:0/2922811633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ea80709d0 0x7f7ea8151900 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 -- 172.21.15.32:0/2922811633 >> 172.21.15.32:0/2922811633 conn(0x7f7ea806c060 msgr2=0x7f7ea80af480 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 -- 172.21.15.32:0/2922811633 shutdown_connections 2023-12-20T04:40:54.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.493+0000 7f7ead7a2640 1 -- 172.21.15.32:0/2922811633 wait complete. 2023-12-20T04:40:54.512 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T04:40:54.512 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T04:40:54.512 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T04:40:54.512 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:40:54.513 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T04:40:54.513 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T04:40:54.513 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T04:40:54.513 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:40:54.513 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T04:40:54.513 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T04:40:54.514 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T04:40:54.514 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:40:54.514 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T04:40:54.514 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T04:40:54.514 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T04:40:54.514 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:40:54.524 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T04:40:54.524 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T04:40:54.526 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T04:40:54.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T04:40:54.528 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T04:40:54.528 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T04:40:54.530 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T04:40:54.530 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T04:40:54.574 INFO:tasks.ceph.mds.a:Started 2023-12-20T04:40:54.574 DEBUG:teuthology.parallel:result is None 2023-12-20T04:40:54.575 INFO:tasks.ceph.mds.c:Started 2023-12-20T04:40:54.576 DEBUG:teuthology.parallel:result is None 2023-12-20T04:40:54.577 INFO:tasks.ceph.mds.b:Started 2023-12-20T04:40:54.578 DEBUG:teuthology.parallel:result is None 2023-12-20T04:40:54.580 INFO:tasks.ceph.mds.d:Started 2023-12-20T04:40:54.580 DEBUG:teuthology.parallel:result is None 2023-12-20T04:40:54.580 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T04:40:54.663 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T04:40:54.664 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T04:40:54.674 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T04:40:54.676 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T04:40:54.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f12c20640 1 Processor -- start 2023-12-20T04:40:54.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f12c20640 1 -- start start 2023-12-20T04:40:54.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f12c20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f0c071af0 0x7f6f0c0c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f12c20640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6f0c160b30 0x7f6f0c160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f12c20640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f0c0c9a40 con 0x7f6f0c15ce60 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f12c20640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f0c0c9180 con 0x7f6f0c160b30 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f12c20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0c93f0 con 0x7f6f0c071af0 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f109bc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6f0c15ce60 0x7f6f0c0713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46694/0 (socket says 172.21.15.32:46694) 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f109bc640 1 -- 172.21.15.32:0/1877847941 learned_addr learned my addr 172.21.15.32:0/1877847941 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f0bfff640 1 --2- 172.21.15.32:0/1877847941 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f0c071af0 0x7f6f0c0c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f111bd640 1 --2- 172.21.15.32:0/1877847941 >> v2:172.21.15.39:3300/0 conn(0x7f6f0c160b30 0x7f6f0c160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f0b7fe640 1 -- 172.21.15.32:0/1877847941 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 984836400 0 0) 0x7f6f0c0c9a40 con 0x7f6f0c15ce60 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f0b7fe640 1 -- 172.21.15.32:0/1877847941 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6efc003530 con 0x7f6f0c15ce60 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f0bfff640 1 -- 172.21.15.32:0/1877847941 >> v2:172.21.15.39:3300/0 conn(0x7f6f0c160b30 msgr2=0x7f6f0c160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f0bfff640 1 --2- 172.21.15.32:0/1877847941 >> v2:172.21.15.39:3300/0 conn(0x7f6f0c160b30 0x7f6f0c160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f0bfff640 1 -- 172.21.15.32:0/1877847941 >> v1:172.21.15.32:6789/0 conn(0x7f6f0c15ce60 legacy=0x7f6f0c0713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f0bfff640 1 -- 172.21.15.32:0/1877847941 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f0c0c96d0 con 0x7f6f0c071af0 2023-12-20T04:40:54.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.737+0000 7f6f111bd640 1 --2- 172.21.15.32:0/1877847941 >> v2:172.21.15.39:3300/0 conn(0x7f6f0c160b30 0x7f6f0c160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:54.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f0bfff640 1 --2- 172.21.15.32:0/1877847941 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f0c071af0 0x7f6f0c0c9ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80097c0 tx=0x7f6ef801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4c1dd28fc35fa0a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f0b7fe640 1 -- 172.21.15.32:0/1877847941 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80ad640 con 0x7f6f0c071af0 2023-12-20T04:40:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f0b7fe640 1 -- 172.21.15.32:0/1877847941 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ef80ad7a0 con 0x7f6f0c071af0 2023-12-20T04:40:54.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f0b7fe640 1 -- 172.21.15.32:0/1877847941 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ef80adb30 con 0x7f6f0c071af0 2023-12-20T04:40:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f12c20640 1 -- 172.21.15.32:0/1877847941 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f0c071af0 msgr2=0x7f6f0c0c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f12c20640 1 --2- 172.21.15.32:0/1877847941 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f0c071af0 0x7f6f0c0c9ed0 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80097c0 tx=0x7f6ef801c480 comp rx=0 tx=0).stop 2023-12-20T04:40:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f12c20640 1 -- 172.21.15.32:0/1877847941 shutdown_connections 2023-12-20T04:40:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f12c20640 1 --2- 172.21.15.32:0/1877847941 >> v2:172.21.15.39:3300/0 conn(0x7f6f0c160b30 0x7f6f0c160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f12c20640 1 --2- 172.21.15.32:0/1877847941 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f0c071af0 0x7f6f0c0c9ed0 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f12c20640 1 -- 172.21.15.32:0/1877847941 >> 172.21.15.32:0/1877847941 conn(0x7f6f0c06a320 msgr2=0x7f6f0c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f12c20640 1 -- 172.21.15.32:0/1877847941 shutdown_connections 2023-12-20T04:40:54.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.741+0000 7f6f12c20640 1 -- 172.21.15.32:0/1877847941 wait complete. 2023-12-20T04:40:54.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f12c20640 1 Processor -- start 2023-12-20T04:40:54.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f12c20640 1 -- start start 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f12c20640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f0c071af0 0x7f6f0c0c84b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f12c20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f0c15ce60 0x7f6f0c1543e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f12c20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f0c160b30 0x7f6f0c0cdb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f12c20640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c0c4340 con 0x7f6f0c071af0 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f12c20640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c15d530 con 0x7f6f0c15ce60 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f12c20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f0c15ee60 con 0x7f6f0c160b30 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f109bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f0c071af0 0x7f6f0c0c84b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f109bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f0c071af0 0x7f6f0c0c84b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46312/0 (socket says 172.21.15.32:46312) 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f109bc640 1 -- 172.21.15.32:0/2487870363 learned_addr learned my addr 172.21.15.32:0/2487870363 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f0bfff640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f0c15ce60 0x7f6f0c1543e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f111bd640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f0c160b30 0x7f6f0c0cdb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f109bc640 1 -- 172.21.15.32:0/2487870363 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f0c160b30 msgr2=0x7f6f0c0cdb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f109bc640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f0c160b30 0x7f6f0c0cdb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f109bc640 1 -- 172.21.15.32:0/2487870363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f0c15ce60 msgr2=0x7f6f0c1543e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f109bc640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f0c15ce60 0x7f6f0c1543e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f109bc640 1 -- 172.21.15.32:0/2487870363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f04009a70 con 0x7f6f0c071af0 2023-12-20T04:40:54.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f109bc640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f0c071af0 0x7f6f0c0c84b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000c700 tx=0x7f6f0000cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:54.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f097fa640 1 -- 172.21.15.32:0/2487870363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f000074e0 con 0x7f6f0c071af0 2023-12-20T04:40:54.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f097fa640 1 -- 172.21.15.32:0/2487870363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f00007660 con 0x7f6f0c071af0 2023-12-20T04:40:54.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f097fa640 1 -- 172.21.15.32:0/2487870363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f000059f0 con 0x7f6f0c071af0 2023-12-20T04:40:54.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f12c20640 1 -- 172.21.15.32:0/2487870363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6ef8009420 con 0x7f6f0c071af0 2023-12-20T04:40:54.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f12c20640 1 -- 172.21.15.32:0/2487870363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f0c161680 con 0x7f6f0c071af0 2023-12-20T04:40:54.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.745+0000 7f6f12c20640 1 -- 172.21.15.32:0/2487870363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f0c15da10 con 0x7f6f0c071af0 2023-12-20T04:40:54.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.749+0000 7f6f097fa640 1 -- 172.21.15.32:0/2487870363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6f00005b50 con 0x7f6f0c071af0 2023-12-20T04:40:54.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.749+0000 7f6f097fa640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ef0065c00 0x7f6ef00680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:54.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.749+0000 7f6f097fa640 1 -- 172.21.15.32:0/2487870363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(56..56 src has 1..56) v4 ==== 5502+0+0 (secure 0 0 0) 0x7f6f00082760 con 0x7f6f0c071af0 2023-12-20T04:40:54.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.749+0000 7f6f0bfff640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ef0065c00 0x7f6ef00680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:54.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.749+0000 7f6f0bfff640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ef0065c00 0x7f6ef00680c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80af000 tx=0x7f6ef80a7070 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:54.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.753+0000 7f6f097fa640 1 -- 172.21.15.32:0/2487870363 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6f00053c00 con 0x7f6f0c071af0 2023-12-20T04:40:54.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:54.921+0000 7f6f12c20640 1 -- 172.21.15.32:0/2487870363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f6f0c15efd0 con 0x7f6f0c071af0 2023-12-20T04:40:55.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.713+0000 7f6f097fa640 1 -- 172.21.15.32:0/2487870363 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v57) v1 ==== 97+0+0 (secure 0 0 0) 0x7f6f0c15efd0 con 0x7f6f0c071af0 2023-12-20T04:40:55.713 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T04:40:55.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 -- 172.21.15.32:0/2487870363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ef0065c00 msgr2=0x7f6ef00680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:55.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ef0065c00 0x7f6ef00680c0 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f6ef80af000 tx=0x7f6ef80a7070 comp rx=0 tx=0).stop 2023-12-20T04:40:55.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 -- 172.21.15.32:0/2487870363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f0c071af0 msgr2=0x7f6f0c0c84b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:55.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f0c071af0 0x7f6f0c0c84b0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f6f0000c700 tx=0x7f6f0000cb00 comp rx=0 tx=0).stop 2023-12-20T04:40:55.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 -- 172.21.15.32:0/2487870363 shutdown_connections 2023-12-20T04:40:55.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6ef0065c00 0x7f6ef00680c0 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:55.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f0c160b30 0x7f6f0c0cdb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:55.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f0c15ce60 0x7f6f0c1543e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:55.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 --2- 172.21.15.32:0/2487870363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f0c071af0 0x7f6f0c0c84b0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:55.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 -- 172.21.15.32:0/2487870363 >> 172.21.15.32:0/2487870363 conn(0x7f6f0c06a320 msgr2=0x7f6f0c0713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:55.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 -- 172.21.15.32:0/2487870363 shutdown_connections 2023-12-20T04:40:55.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.717+0000 7f6f12c20640 1 -- 172.21.15.32:0/2487870363 wait complete. 2023-12-20T04:40:55.739 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T04:40:55.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facefd2c640 1 Processor -- start 2023-12-20T04:40:55.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facefd2c640 1 -- start start 2023-12-20T04:40:55.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facefd2c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7face806cf10 0x7face80c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:55.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facefd2c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7face8164ef0 0x7face80c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:55.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facefd2c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7face80ae3f0 con 0x7face80ae010 2023-12-20T04:40:55.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facefd2c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face8161800 con 0x7face8164ef0 2023-12-20T04:40:55.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facefd2c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7face8165380 con 0x7face806cf10 2023-12-20T04:40:55.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facedac8640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7face80ae010 0x7face8071470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46704/0 (socket says 172.21.15.32:46704) 2023-12-20T04:40:55.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facedac8640 1 -- 172.21.15.32:0/1631963053 learned_addr learned my addr 172.21.15.32:0/1631963053 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:55.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facee2c9640 1 --2- 172.21.15.32:0/1631963053 >> v2:172.21.15.32:3300/0 conn(0x7face8164ef0 0x7face80c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:55.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7faced2c7640 1 --2- 172.21.15.32:0/1631963053 >> v2:172.21.15.39:3300/0 conn(0x7face806cf10 0x7face80c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:55.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facee2c9640 1 -- 172.21.15.32:0/1631963053 >> v2:172.21.15.39:3300/0 conn(0x7face806cf10 msgr2=0x7face80c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:55.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facee2c9640 1 --2- 172.21.15.32:0/1631963053 >> v2:172.21.15.39:3300/0 conn(0x7face806cf10 0x7face80c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:55.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facee2c9640 1 -- 172.21.15.32:0/1631963053 >> v1:172.21.15.32:6789/0 conn(0x7face80ae010 legacy=0x7face8071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:55.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facee2c9640 1 -- 172.21.15.32:0/1631963053 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face8165660 con 0x7face8164ef0 2023-12-20T04:40:55.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.889+0000 7facecac6640 1 -- 172.21.15.32:0/1631963053 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 398340775 0 0) 0x7face80ae3f0 con 0x7face80ae010 2023-12-20T04:40:55.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facee2c9640 1 --2- 172.21.15.32:0/1631963053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7face8164ef0 0x7face80c99e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7face8164e50 tx=0x7face001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbaf00226dab12e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:55.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facecac6640 1 -- 172.21.15.32:0/1631963053 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face00a5910 con 0x7face8164ef0 2023-12-20T04:40:55.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facecac6640 1 -- 172.21.15.32:0/1631963053 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face00a5a70 con 0x7face8164ef0 2023-12-20T04:40:55.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facecac6640 1 -- 172.21.15.32:0/1631963053 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face00a5e00 con 0x7face8164ef0 2023-12-20T04:40:55.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facefd2c640 1 -- 172.21.15.32:0/1631963053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7face8164ef0 msgr2=0x7face80c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:55.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facefd2c640 1 --2- 172.21.15.32:0/1631963053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7face8164ef0 0x7face80c99e0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7face8164e50 tx=0x7face001c850 comp rx=0 tx=0).stop 2023-12-20T04:40:55.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facefd2c640 1 -- 172.21.15.32:0/1631963053 shutdown_connections 2023-12-20T04:40:55.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facefd2c640 1 --2- 172.21.15.32:0/1631963053 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7face8164ef0 0x7face80c99e0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:55.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facefd2c640 1 --2- 172.21.15.32:0/1631963053 >> v2:172.21.15.39:3300/0 conn(0x7face806cf10 0x7face80c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:55.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facefd2c640 1 -- 172.21.15.32:0/1631963053 >> 172.21.15.32:0/1631963053 conn(0x7face806a320 msgr2=0x7face80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:55.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facefd2c640 1 -- 172.21.15.32:0/1631963053 shutdown_connections 2023-12-20T04:40:55.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facefd2c640 1 -- 172.21.15.32:0/1631963053 wait complete. 2023-12-20T04:40:55.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.893+0000 7facefd2c640 1 Processor -- start 2023-12-20T04:40:55.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facefd2c640 1 -- start start 2023-12-20T04:40:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facefd2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7face80ae010 0x7face81fddd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facefd2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face81619d0 0x7face814d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facefd2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face80c7910 0x7face80c7dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facefd2c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80c8300 con 0x7face80ae010 2023-12-20T04:40:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facefd2c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7face80c8660 con 0x7face81619d0 2023-12-20T04:40:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facefd2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7face81638f0 con 0x7face80c7910 2023-12-20T04:40:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facedac8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7face80ae010 0x7face81fddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:55.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facedac8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7face80ae010 0x7face81fddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46322/0 (socket says 172.21.15.32:46322) 2023-12-20T04:40:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facedac8640 1 -- 172.21.15.32:0/2099953673 learned_addr learned my addr 172.21.15.32:0/2099953673 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7faced2c7640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face81619d0 0x7face814d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facee2c9640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face80c7910 0x7face80c7dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facedac8640 1 -- 172.21.15.32:0/2099953673 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face80c7910 msgr2=0x7face80c7dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facedac8640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face80c7910 0x7face80c7dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facedac8640 1 -- 172.21.15.32:0/2099953673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face81619d0 msgr2=0x7face814d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facedac8640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face81619d0 0x7face814d700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:55.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facedac8640 1 -- 172.21.15.32:0/2099953673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7face0009a70 con 0x7face80ae010 2023-12-20T04:40:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facedac8640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7face80ae010 0x7face81fddd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7face40010f0 tx=0x7face400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facde7fc640 1 -- 172.21.15.32:0/2099953673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face4007270 con 0x7face80ae010 2023-12-20T04:40:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facde7fc640 1 -- 172.21.15.32:0/2099953673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7face40073d0 con 0x7face80ae010 2023-12-20T04:40:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facefd2c640 1 -- 172.21.15.32:0/2099953673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7face8163bd0 con 0x7face80ae010 2023-12-20T04:40:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facde7fc640 1 -- 172.21.15.32:0/2099953673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7face40057b0 con 0x7face80ae010 2023-12-20T04:40:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facefd2c640 1 -- 172.21.15.32:0/2099953673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7face8164190 con 0x7face80ae010 2023-12-20T04:40:55.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.897+0000 7facefd2c640 1 -- 172.21.15.32:0/2099953673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7face81643b0 con 0x7face80ae010 2023-12-20T04:40:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.901+0000 7facde7fc640 1 -- 172.21.15.32:0/2099953673 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7face4005910 con 0x7face80ae010 2023-12-20T04:40:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.901+0000 7facde7fc640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faccc065b80 0x7faccc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.901+0000 7facde7fc640 1 -- 172.21.15.32:0/2099953673 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4857+0+0 (secure 0 0 0) 0x7face40825e0 con 0x7face80ae010 2023-12-20T04:40:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.901+0000 7faced2c7640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faccc065b80 0x7faccc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:55.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.901+0000 7faced2c7640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faccc065b80 0x7faccc068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7face80ca570 tx=0x7facd8008040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:55.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:55.905+0000 7facde7fc640 1 -- 172.21.15.32:0/2099953673 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7face40501e0 con 0x7face80ae010 2023-12-20T04:40:56.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 -- 172.21.15.32:0/2099953673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7face8066de0 con 0x7face80ae010 2023-12-20T04:40:56.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facde7fc640 1 -- 172.21.15.32:0/2099953673 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v15) v1 ==== 80+0+5304 (secure 0 0 0) 0x7face4017020 con 0x7face80ae010 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.116 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T04:40:56.117 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.118 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.119 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T04:40:56.120 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:40:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:40:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T04:40:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T04:40:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:40:56.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:40:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T04:40:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T04:40:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T04:40:56.124 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:40:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:40:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T04:40:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T04:40:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:40:56.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T04:40:56.126 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:40:56.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.128 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:40:56.129 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:40:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:56.130 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T04:40:56.130 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 -- 172.21.15.32:0/2099953673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faccc065b80 msgr2=0x7faccc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faccc065b80 0x7faccc068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7face80ca570 tx=0x7facd8008040 comp rx=0 tx=0).stop 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 -- 172.21.15.32:0/2099953673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7face80ae010 msgr2=0x7face81fddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7face80ae010 0x7face81fddd0 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7face40010f0 tx=0x7face400cbe0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 -- 172.21.15.32:0/2099953673 shutdown_connections 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faccc065b80 0x7faccc068040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7face80c7910 0x7face80c7dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7face81619d0 0x7face814d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 --2- 172.21.15.32:0/2099953673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7face80ae010 0x7face81fddd0 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 -- 172.21.15.32:0/2099953673 >> 172.21.15.32:0/2099953673 conn(0x7face806a320 msgr2=0x7face80cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 -- 172.21.15.32:0/2099953673 shutdown_connections 2023-12-20T04:40:56.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.117+0000 7facefd2c640 1 -- 172.21.15.32:0/2099953673 wait complete. 2023-12-20T04:40:56.137 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:40:56.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.281+0000 7f2d4de20640 1 Processor -- start 2023-12-20T04:40:56.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.281+0000 7f2d4de20640 1 -- start start 2023-12-20T04:40:56.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.281+0000 7f2d4de20640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2d4815d9a0 0x7f2d4806c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.281+0000 7f2d4de20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d4806c920 0x7f2d480705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.281+0000 7f2d4de20640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2d480ae140 con 0x7f2d481593c0 2023-12-20T04:40:56.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.281+0000 7f2d4de20640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2d480ae9e0 con 0x7f2d4815d9a0 2023-12-20T04:40:56.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.281+0000 7f2d4de20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4806cd90 con 0x7f2d4806c920 2023-12-20T04:40:56.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.281+0000 7f2d477fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2d481593c0 0x7f2d480afdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46716/0 (socket says 172.21.15.32:46716) 2023-12-20T04:40:56.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.281+0000 7f2d477fe640 1 -- 172.21.15.32:0/270320521 learned_addr learned my addr 172.21.15.32:0/270320521 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d47fff640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d4806c920 0x7f2d480705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d46ffd640 1 --2- 172.21.15.32:0/270320521 >> v2:172.21.15.39:3300/0 conn(0x7f2d4815d9a0 0x7f2d4806c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2512500363 0 0) 0x7f2d480ae140 con 0x7f2d481593c0 2023-12-20T04:40:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2d30005740 con 0x7f2d481593c0 2023-12-20T04:40:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3314795075 0 0) 0x7f2d30005740 con 0x7f2d481593c0 2023-12-20T04:40:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d4806c920 msgr2=0x7f2d480705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d4806c920 0x7f2d480705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 >> v2:172.21.15.39:3300/0 conn(0x7f2d4815d9a0 msgr2=0x7f2d4806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 --2- 172.21.15.32:0/270320521 >> v2:172.21.15.39:3300/0 conn(0x7f2d4815d9a0 0x7f2d4806c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d48072170 con 0x7f2d481593c0 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f2d38001800 con 0x7f2d481593c0 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 >> v1:172.21.15.32:6789/0 conn(0x7f2d481593c0 legacy=0x7f2d480afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 0x7f2d30005fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d4806c920 0x7f2d300069e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d30006f20 0x7f2d300073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d4806cd90 con 0x7f2d30005bc0 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3000dbd0 con 0x7f2d30006f20 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d3000dd90 con 0x7f2d4806c920 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d477fe640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 0x7f2d30005fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d47fff640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d30006f20 0x7f2d300073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d46ffd640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d4806c920 0x7f2d300069e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d477fe640 1 -- 172.21.15.32:0/270320521 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d4806c920 msgr2=0x7f2d300069e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d477fe640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d4806c920 0x7f2d300069e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d477fe640 1 -- 172.21.15.32:0/270320521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d30006f20 msgr2=0x7f2d300073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d477fe640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d30006f20 0x7f2d300073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.285+0000 7f2d477fe640 1 -- 172.21.15.32:0/270320521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d30005740 con 0x7f2d30005bc0 2023-12-20T04:40:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d477fe640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 0x7f2d30005fe0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800c240 tx=0x7f2d3801cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ab837e09dd9be1b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38007270 con 0x7f2d30005bc0 2023-12-20T04:40:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d380073f0 con 0x7f2d30005bc0 2023-12-20T04:40:56.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d467fc640 1 -- 172.21.15.32:0/270320521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d380a5850 con 0x7f2d30005bc0 2023-12-20T04:40:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 -- 172.21.15.32:0/270320521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 msgr2=0x7f2d30005fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 --2- 172.21.15.32:0/270320521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 0x7f2d30005fe0 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f2d3800c240 tx=0x7f2d3801cbf0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d47fff640 1 -- 172.21.15.32:0/270320521 reap_dead start 2023-12-20T04:40:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 -- 172.21.15.32:0/270320521 shutdown_connections 2023-12-20T04:40:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 -- 172.21.15.32:0/270320521 >> 172.21.15.32:0/270320521 conn(0x7f2d4806a320 msgr2=0x7f2d480ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 -- 172.21.15.32:0/270320521 shutdown_connections 2023-12-20T04:40:56.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 -- 172.21.15.32:0/270320521 wait complete. 2023-12-20T04:40:56.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 Processor -- start 2023-12-20T04:40:56.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 -- start start 2023-12-20T04:40:56.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 0x7f2d480b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d480b4000 0x7f2d481f50f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d481f5630 0x7f2d481f5ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d481f6000 con 0x7f2d30005bc0 2023-12-20T04:40:56.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480c6eb0 con 0x7f2d481f5630 2023-12-20T04:40:56.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.289+0000 7f2d4de20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2d480b21b0 con 0x7f2d480b4000 2023-12-20T04:40:56.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 0x7f2d480b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 0x7f2d480b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46344/0 (socket says 172.21.15.32:46344) 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d477fe640 1 -- 172.21.15.32:0/3838970051 learned_addr learned my addr 172.21.15.32:0/3838970051 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d47fff640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d481f5630 0x7f2d481f5ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d46ffd640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d480b4000 0x7f2d481f50f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d477fe640 1 -- 172.21.15.32:0/3838970051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d480b4000 msgr2=0x7f2d481f50f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d477fe640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d480b4000 0x7f2d481f50f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d477fe640 1 -- 172.21.15.32:0/3838970051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d481f5630 msgr2=0x7f2d481f5ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d477fe640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d481f5630 0x7f2d481f5ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d477fe640 1 -- 172.21.15.32:0/3838970051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2d480b2490 con 0x7f2d30005bc0 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d47fff640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d481f5630 0x7f2d481f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d477fe640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 0x7f2d480b3ac0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2d38000f00 tx=0x7f2d30007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d2ffff640 1 -- 172.21.15.32:0/3838970051 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d380b4070 con 0x7f2d30005bc0 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d4de20640 1 -- 172.21.15.32:0/3838970051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2d480b2770 con 0x7f2d30005bc0 2023-12-20T04:40:56.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d4de20640 1 -- 172.21.15.32:0/3838970051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2d480b1e30 con 0x7f2d30005bc0 2023-12-20T04:40:56.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d2ffff640 1 -- 172.21.15.32:0/3838970051 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2d38007560 con 0x7f2d30005bc0 2023-12-20T04:40:56.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d2ffff640 1 -- 172.21.15.32:0/3838970051 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2d38001180 con 0x7f2d30005bc0 2023-12-20T04:40:56.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d4de20640 1 -- 172.21.15.32:0/3838970051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2d28005260 con 0x7f2d30005bc0 2023-12-20T04:40:56.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.293+0000 7f2d2ffff640 1 -- 172.21.15.32:0/3838970051 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2d380015c0 con 0x7f2d30005bc0 2023-12-20T04:40:56.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.297+0000 7f2d2ffff640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d34065b80 0x7f2d34068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.297+0000 7f2d2ffff640 1 -- 172.21.15.32:0/3838970051 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2d3811c7d0 con 0x7f2d30005bc0 2023-12-20T04:40:56.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.297+0000 7f2d46ffd640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d34065b80 0x7f2d34068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.297+0000 7f2d46ffd640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d34065b80 0x7f2d34068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d480b4420 tx=0x7f2d3c009190 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:56.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.301+0000 7f2d2ffff640 1 -- 172.21.15.32:0/3838970051 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2d380ea330 con 0x7f2d30005bc0 2023-12-20T04:40:56.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.457+0000 7f2d4de20640 1 -- 172.21.15.32:0/3838970051 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2d280057e0 con 0x7f2d30005bc0 2023-12-20T04:40:56.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.457+0000 7f2d2ffff640 1 -- 172.21.15.32:0/3838970051 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v57) v1 ==== 74+0+12341 (secure 0 0 0) 0x7f2d380bd090 con 0x7f2d30005bc0 2023-12-20T04:40:56.458 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:56.458 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":57,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:40:55.708616+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":5,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":33,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:40:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 -- 172.21.15.32:0/3838970051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d34065b80 msgr2=0x7f2d34068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d34065b80 0x7f2d34068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f2d480b4420 tx=0x7f2d3c009190 comp rx=0 tx=0).stop 2023-12-20T04:40:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 -- 172.21.15.32:0/3838970051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 msgr2=0x7f2d480b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 0x7f2d480b3ac0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f2d38000f00 tx=0x7f2d30007a10 comp rx=0 tx=0).stop 2023-12-20T04:40:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 -- 172.21.15.32:0/3838970051 shutdown_connections 2023-12-20T04:40:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2d34065b80 0x7f2d34068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2d481f5630 0x7f2d481f5ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2d480b4000 0x7f2d481f50f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 --2- 172.21.15.32:0/3838970051 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2d30005bc0 0x7f2d480b3ac0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 -- 172.21.15.32:0/3838970051 >> 172.21.15.32:0/3838970051 conn(0x7f2d4806a320 msgr2=0x7f2d480cd180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 -- 172.21.15.32:0/3838970051 shutdown_connections 2023-12-20T04:40:56.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.461+0000 7f2d4de20640 1 -- 172.21.15.32:0/3838970051 wait complete. 2023-12-20T04:40:56.480 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:40:56.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:40:56.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9547d7b640 1 Processor -- start 2023-12-20T04:40:56.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9547d7b640 1 -- start start 2023-12-20T04:40:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9547d7b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f954006d170 0x7f95400cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9547d7b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954006a1f0 con 0x7f9540065d40 2023-12-20T04:40:56.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9547d7b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f954006a440 con 0x7f95400cb050 2023-12-20T04:40:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9547d7b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f954006a720 con 0x7f954006d170 2023-12-20T04:40:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9546318640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f954006d170 0x7f95400cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9545316640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9540065d40 0x7f95400c9720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46726/0 (socket says 172.21.15.32:46726) 2023-12-20T04:40:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9545316640 1 -- 172.21.15.32:0/4047026234 learned_addr learned my addr 172.21.15.32:0/4047026234 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9546318640 1 -- 172.21.15.32:0/4047026234 >> v1:172.21.15.39:6789/0 conn(0x7f95400cb050 legacy=0x7f954006ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9546318640 1 -- 172.21.15.32:0/4047026234 >> v1:172.21.15.32:6789/0 conn(0x7f9540065d40 legacy=0x7f95400c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.629+0000 7f9546318640 1 -- 172.21.15.32:0/4047026234 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f954006aa00 con 0x7f954006d170 2023-12-20T04:40:56.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9546318640 1 --2- 172.21.15.32:0/4047026234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f954006d170 0x7f95400cb430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9538009d90 tx=0x7f953801c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8aa7a8f45fee4df2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:56.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9544b15640 1 -- 172.21.15.32:0/4047026234 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95380a59e0 con 0x7f954006d170 2023-12-20T04:40:56.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9544b15640 1 -- 172.21.15.32:0/4047026234 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95380a5b40 con 0x7f954006d170 2023-12-20T04:40:56.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9544b15640 1 -- 172.21.15.32:0/4047026234 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95380ad440 con 0x7f954006d170 2023-12-20T04:40:56.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9547d7b640 1 -- 172.21.15.32:0/4047026234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f954006d170 msgr2=0x7f95400cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9547d7b640 1 --2- 172.21.15.32:0/4047026234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f954006d170 0x7f95400cb430 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7f9538009d90 tx=0x7f953801c920 comp rx=0 tx=0).stop 2023-12-20T04:40:56.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9547d7b640 1 -- 172.21.15.32:0/4047026234 shutdown_connections 2023-12-20T04:40:56.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9547d7b640 1 --2- 172.21.15.32:0/4047026234 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f954006d170 0x7f95400cb430 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9547d7b640 1 -- 172.21.15.32:0/4047026234 >> 172.21.15.32:0/4047026234 conn(0x7f95400cf8d0 msgr2=0x7f95400cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:56.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9547d7b640 1 -- 172.21.15.32:0/4047026234 shutdown_connections 2023-12-20T04:40:56.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9547d7b640 1 -- 172.21.15.32:0/4047026234 wait complete. 2023-12-20T04:40:56.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9547d7b640 1 Processor -- start 2023-12-20T04:40:56.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.633+0000 7f9547d7b640 1 -- start start 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9547d7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9540065d40 0x7f954014bfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9547d7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f954006d170 0x7f95400c6fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9547d7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95400cb050 0x7f954014d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9547d7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95400c83a0 con 0x7f954006d170 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9547d7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95400c7510 con 0x7f95400cb050 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9547d7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95400c7790 con 0x7f9540065d40 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9545316640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f954006d170 0x7f95400c6fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9545316640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f954006d170 0x7f95400c6fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46374/0 (socket says 172.21.15.32:46374) 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9545316640 1 -- 172.21.15.32:0/872003072 learned_addr learned my addr 172.21.15.32:0/872003072 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9546318640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95400cb050 0x7f954014d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9545b17640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9540065d40 0x7f954014bfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9545316640 1 -- 172.21.15.32:0/872003072 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9540065d40 msgr2=0x7f954014bfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9545316640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9540065d40 0x7f954014bfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9545316640 1 -- 172.21.15.32:0/872003072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95400cb050 msgr2=0x7f954014d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9545316640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95400cb050 0x7f954014d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9545316640 1 -- 172.21.15.32:0/872003072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9538009a70 con 0x7f954006d170 2023-12-20T04:40:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9545b17640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9540065d40 0x7f954014bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9546318640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95400cb050 0x7f954014d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9545316640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f954006d170 0x7f95400c6fd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f953000c430 tx=0x7f953000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f95367fc640 1 -- 172.21.15.32:0/872003072 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9530005cf0 con 0x7f954006d170 2023-12-20T04:40:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f95367fc640 1 -- 172.21.15.32:0/872003072 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9530005e50 con 0x7f954006d170 2023-12-20T04:40:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f95367fc640 1 -- 172.21.15.32:0/872003072 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95300072d0 con 0x7f954006d170 2023-12-20T04:40:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9547d7b640 1 -- 172.21.15.32:0/872003072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95400c7a70 con 0x7f954006d170 2023-12-20T04:40:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9547d7b640 1 -- 172.21.15.32:0/872003072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95400c8030 con 0x7f954006d170 2023-12-20T04:40:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.637+0000 7f9547d7b640 1 -- 172.21.15.32:0/872003072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954004e000 con 0x7f954006d170 2023-12-20T04:40:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.641+0000 7f95367fc640 1 -- 172.21.15.32:0/872003072 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9530007430 con 0x7f954006d170 2023-12-20T04:40:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.641+0000 7f95367fc640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9524065b80 0x7f9524068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.641+0000 7f95367fc640 1 -- 172.21.15.32:0/872003072 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9530082450 con 0x7f954006d170 2023-12-20T04:40:56.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.641+0000 7f9545b17640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9524065b80 0x7f9524068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.641+0000 7f9545b17640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9524065b80 0x7f9524068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f95401f9190 tx=0x7f953c009040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:56.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.645+0000 7f95367fc640 1 -- 172.21.15.32:0/872003072 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9530050100 con 0x7f954006d170 2023-12-20T04:40:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 -- 172.21.15.32:0/872003072 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f95401617c0 con 0x7f954006d170 2023-12-20T04:40:56.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f95367fc640 1 -- 172.21.15.32:0/872003072 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v57) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9530017020 con 0x7f954006d170 2023-12-20T04:40:56.801 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:40:56.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 -- 172.21.15.32:0/872003072 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9524065b80 msgr2=0x7f9524068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9524065b80 0x7f9524068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f95401f9190 tx=0x7f953c009040 comp rx=0 tx=0).stop 2023-12-20T04:40:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 -- 172.21.15.32:0/872003072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f954006d170 msgr2=0x7f95400c6fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f954006d170 0x7f95400c6fd0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f953000c430 tx=0x7f953000c8f0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 -- 172.21.15.32:0/872003072 shutdown_connections 2023-12-20T04:40:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9524065b80 0x7f9524068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95400cb050 0x7f954014d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f954006d170 0x7f95400c6fd0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 --2- 172.21.15.32:0/872003072 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9540065d40 0x7f954014bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 -- 172.21.15.32:0/872003072 >> 172.21.15.32:0/872003072 conn(0x7f95400cf8d0 msgr2=0x7f95400c6090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 -- 172.21.15.32:0/872003072 shutdown_connections 2023-12-20T04:40:56.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.801+0000 7f9547d7b640 1 -- 172.21.15.32:0/872003072 wait complete. 2023-12-20T04:40:56.823 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T04:40:56.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.965+0000 7f348056d640 1 Processor -- start 2023-12-20T04:40:56.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f348056d640 1 -- start start 2023-12-20T04:40:56.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f348056d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f347815cd70 0x7f34780cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f348056d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f34780b23d0 0x7f34780b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f348056d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34780b2d70 con 0x7f34781608e0 2023-12-20T04:40:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f348056d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34781618b0 con 0x7f347815cd70 2023-12-20T04:40:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f348056d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34780ce6c0 con 0x7f34780b23d0 2023-12-20T04:40:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f347e309640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f347815cd70 0x7f34780cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f347e309640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f347815cd70 0x7f34780cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46378/0 (socket says 172.21.15.32:46378) 2023-12-20T04:40:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f347e309640 1 -- 172.21.15.32:0/3305308500 learned_addr learned my addr 172.21.15.32:0/3305308500 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f347eb0a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f34781608e0 0x7f3478160d30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46738/0 (socket says 172.21.15.32:46738) 2023-12-20T04:40:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f347db08640 1 --2- 172.21.15.32:0/3305308500 >> v2:172.21.15.39:3300/0 conn(0x7f34780b23d0 0x7f34780b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f347e309640 1 -- 172.21.15.32:0/3305308500 >> v2:172.21.15.39:3300/0 conn(0x7f34780b23d0 msgr2=0x7f34780b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f347e309640 1 --2- 172.21.15.32:0/3305308500 >> v2:172.21.15.39:3300/0 conn(0x7f34780b23d0 0x7f34780b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f347e309640 1 -- 172.21.15.32:0/3305308500 >> v1:172.21.15.32:6789/0 conn(0x7f34781608e0 legacy=0x7f3478160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f347e309640 1 -- 172.21.15.32:0/3305308500 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34781615d0 con 0x7f347815cd70 2023-12-20T04:40:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f347d307640 1 -- 172.21.15.32:0/3305308500 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 241817951 0 0) 0x7f34780b2d70 con 0x7f34781608e0 2023-12-20T04:40:56.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.969+0000 7f347db08640 1 --2- 172.21.15.32:0/3305308500 >> v2:172.21.15.39:3300/0 conn(0x7f34780b23d0 0x7f34780b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f347e309640 1 --2- 172.21.15.32:0/3305308500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347815cd70 0x7f34780cf160 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f34780d08b0 tx=0x7f347401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e91306d21c16fc1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f347d307640 1 -- 172.21.15.32:0/3305308500 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34740aa430 con 0x7f347815cd70 2023-12-20T04:40:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f347d307640 1 -- 172.21.15.32:0/3305308500 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34740aa590 con 0x7f347815cd70 2023-12-20T04:40:56.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f347d307640 1 -- 172.21.15.32:0/3305308500 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34740aa920 con 0x7f347815cd70 2023-12-20T04:40:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f348056d640 1 -- 172.21.15.32:0/3305308500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347815cd70 msgr2=0x7f34780cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f348056d640 1 --2- 172.21.15.32:0/3305308500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347815cd70 0x7f34780cf160 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7f34780d08b0 tx=0x7f347401c4d0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f348056d640 1 -- 172.21.15.32:0/3305308500 shutdown_connections 2023-12-20T04:40:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f348056d640 1 --2- 172.21.15.32:0/3305308500 >> v2:172.21.15.39:3300/0 conn(0x7f34780b23d0 0x7f34780b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f348056d640 1 --2- 172.21.15.32:0/3305308500 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347815cd70 0x7f34780cf160 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f348056d640 1 -- 172.21.15.32:0/3305308500 >> 172.21.15.32:0/3305308500 conn(0x7f34780cd3d0 msgr2=0x7f34780cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f348056d640 1 -- 172.21.15.32:0/3305308500 shutdown_connections 2023-12-20T04:40:56.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f348056d640 1 -- 172.21.15.32:0/3305308500 wait complete. 2023-12-20T04:40:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f348056d640 1 Processor -- start 2023-12-20T04:40:56.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.973+0000 7f348056d640 1 -- start start 2023-12-20T04:40:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f348056d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34780b23d0 0x7f3478160370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f348056d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347815cd70 0x7f3478153cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f348056d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34781608e0 0x7f34780b1070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f348056d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3478154210 con 0x7f347815cd70 2023-12-20T04:40:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f348056d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f347815f2d0 con 0x7f34780b23d0 2023-12-20T04:40:56.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f348056d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f347815f5b0 con 0x7f34781608e0 2023-12-20T04:40:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347db08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347815cd70 0x7f3478153cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347db08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347815cd70 0x7f3478153cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46394/0 (socket says 172.21.15.32:46394) 2023-12-20T04:40:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347db08640 1 -- 172.21.15.32:0/3881028516 learned_addr learned my addr 172.21.15.32:0/3881028516 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347e309640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34780b23d0 0x7f3478160370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347eb0a640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34781608e0 0x7f34780b1070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347db08640 1 -- 172.21.15.32:0/3881028516 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34781608e0 msgr2=0x7f34780b1070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347db08640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34781608e0 0x7f34780b1070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347db08640 1 -- 172.21.15.32:0/3881028516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34780b23d0 msgr2=0x7f3478160370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347db08640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34780b23d0 0x7f3478160370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347db08640 1 -- 172.21.15.32:0/3881028516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34740094d0 con 0x7f347815cd70 2023-12-20T04:40:56.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347e309640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34780b23d0 0x7f3478160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f347db08640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347815cd70 0x7f3478153cd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f346800e810 tx=0x7f346800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f346effd640 1 -- 172.21.15.32:0/3881028516 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346800c8d0 con 0x7f347815cd70 2023-12-20T04:40:56.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f346effd640 1 -- 172.21.15.32:0/3881028516 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f346800ca30 con 0x7f347815cd70 2023-12-20T04:40:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f348056d640 1 -- 172.21.15.32:0/3881028516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f347815f890 con 0x7f347815cd70 2023-12-20T04:40:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f346effd640 1 -- 172.21.15.32:0/3881028516 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f346800cde0 con 0x7f347815cd70 2023-12-20T04:40:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f348056d640 1 -- 172.21.15.32:0/3881028516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f347815fe50 con 0x7f347815cd70 2023-12-20T04:40:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.977+0000 7f348056d640 1 -- 172.21.15.32:0/3881028516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f347804e000 con 0x7f347815cd70 2023-12-20T04:40:56.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.981+0000 7f346effd640 1 -- 172.21.15.32:0/3881028516 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3468013410 con 0x7f347815cd70 2023-12-20T04:40:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.981+0000 7f346effd640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f345c065b80 0x7f345c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.981+0000 7f346effd640 1 -- 172.21.15.32:0/3881028516 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f3468082e20 con 0x7f347815cd70 2023-12-20T04:40:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.981+0000 7f347e309640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f345c065b80 0x7f345c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:56.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.981+0000 7f347e309640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f345c065b80 0x7f345c068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34740ae000 tx=0x7f34740b5040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:56.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:56.985+0000 7f346effd640 1 -- 172.21.15.32:0/3881028516 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f34680509a0 con 0x7f347815cd70 2023-12-20T04:40:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.141+0000 7f348056d640 1 -- 172.21.15.32:0/3881028516 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f3478167a50 con 0x7f347815cd70 2023-12-20T04:40:57.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f346effd640 1 -- 172.21.15.32:0/3881028516 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v57) v1 ==== 84+0+65 (secure 0 0 0) 0x7f3468053fc0 con 0x7f347815cd70 2023-12-20T04:40:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:40:57.143 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:40:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:40:57.143 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T04:40:57.144 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T04:40:57.144 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:40:57.144 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:40:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 -- 172.21.15.32:0/3881028516 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f345c065b80 msgr2=0x7f345c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f345c065b80 0x7f345c068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f34740ae000 tx=0x7f34740b5040 comp rx=0 tx=0).stop 2023-12-20T04:40:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 -- 172.21.15.32:0/3881028516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347815cd70 msgr2=0x7f3478153cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347815cd70 0x7f3478153cd0 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7f346800e810 tx=0x7f346800ecd0 comp rx=0 tx=0).stop 2023-12-20T04:40:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 -- 172.21.15.32:0/3881028516 shutdown_connections 2023-12-20T04:40:57.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f345c065b80 0x7f345c068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34781608e0 0x7f34780b1070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f347815cd70 0x7f3478153cd0 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 --2- 172.21.15.32:0/3881028516 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34780b23d0 0x7f3478160370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 -- 172.21.15.32:0/3881028516 >> 172.21.15.32:0/3881028516 conn(0x7f34780cd3d0 msgr2=0x7f34780cb1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 -- 172.21.15.32:0/3881028516 shutdown_connections 2023-12-20T04:40:57.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.145+0000 7f348056d640 1 -- 172.21.15.32:0/3881028516 wait complete. 2023-12-20T04:40:57.164 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T04:40:57.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T04:40:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd80d36d640 1 Processor -- start 2023-12-20T04:40:57.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd80d36d640 1 -- start start 2023-12-20T04:40:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd80d36d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8081603e0 0x7fd8080b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd80d36d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080d1030 con 0x7fd8080d2040 2023-12-20T04:40:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd80d36d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8080b8ac0 con 0x7fd8080b1d90 2023-12-20T04:40:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd80d36d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8080d1280 con 0x7fd8081603e0 2023-12-20T04:40:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd806ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8081603e0 0x7fd8080b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd8077fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd8080d2040 0x7fd8080d2420 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46742/0 (socket says 172.21.15.32:46742) 2023-12-20T04:40:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd8077fe640 1 -- 172.21.15.32:0/2104826535 learned_addr learned my addr 172.21.15.32:0/2104826535 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd8067fc640 1 -- 172.21.15.32:0/2104826535 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1904748473 0 0) 0x7fd8080d1030 con 0x7fd8080d2040 2023-12-20T04:40:57.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd8067fc640 1 -- 172.21.15.32:0/2104826535 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7f4003530 con 0x7fd8080d2040 2023-12-20T04:40:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd806ffd640 1 -- 172.21.15.32:0/2104826535 >> v1:172.21.15.39:6789/0 conn(0x7fd8080b1d90 legacy=0x7fd8080b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd806ffd640 1 -- 172.21.15.32:0/2104826535 >> v1:172.21.15.32:6789/0 conn(0x7fd8080d2040 legacy=0x7fd8080d2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:57.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.313+0000 7fd806ffd640 1 -- 172.21.15.32:0/2104826535 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8080d1560 con 0x7fd8081603e0 2023-12-20T04:40:57.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.317+0000 7fd806ffd640 1 --2- 172.21.15.32:0/2104826535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8081603e0 0x7fd8080b8580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80097c0 tx=0x7fd7f801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50268904e3428342 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:57.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.317+0000 7fd8067fc640 1 -- 172.21.15.32:0/2104826535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80ac620 con 0x7fd8081603e0 2023-12-20T04:40:57.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.317+0000 7fd8067fc640 1 -- 172.21.15.32:0/2104826535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f80ac780 con 0x7fd8081603e0 2023-12-20T04:40:57.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.317+0000 7fd8067fc640 1 -- 172.21.15.32:0/2104826535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f80acb10 con 0x7fd8081603e0 2023-12-20T04:40:57.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.317+0000 7fd80d36d640 1 -- 172.21.15.32:0/2104826535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8081603e0 msgr2=0x7fd8080b8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:57.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.317+0000 7fd80d36d640 1 --2- 172.21.15.32:0/2104826535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8081603e0 0x7fd8080b8580 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80097c0 tx=0x7fd7f801c3c0 comp rx=0 tx=0).stop 2023-12-20T04:40:57.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.317+0000 7fd80d36d640 1 -- 172.21.15.32:0/2104826535 shutdown_connections 2023-12-20T04:40:57.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.317+0000 7fd80d36d640 1 --2- 172.21.15.32:0/2104826535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8081603e0 0x7fd8080b8580 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:57.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.317+0000 7fd80d36d640 1 -- 172.21.15.32:0/2104826535 >> 172.21.15.32:0/2104826535 conn(0x7fd80806b5e0 msgr2=0x7fd80806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:57.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.317+0000 7fd80d36d640 1 -- 172.21.15.32:0/2104826535 shutdown_connections 2023-12-20T04:40:57.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.317+0000 7fd80d36d640 1 -- 172.21.15.32:0/2104826535 wait complete. 2023-12-20T04:40:57.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd80d36d640 1 Processor -- start 2023-12-20T04:40:57.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd80d36d640 1 -- start start 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd80d36d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8080b1d90 0x7fd808154d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd80d36d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8080d2040 0x7fd808154260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd80d36d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8081603e0 0x7fd8080c8b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd80d36d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8081552c0 con 0x7fd8080d2040 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd80d36d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd80815a800 con 0x7fd8080b1d90 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd80d36d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd80815aae0 con 0x7fd8081603e0 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8080d2040 0x7fd808154260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8080d2040 0x7fd808154260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46414/0 (socket says 172.21.15.32:46414) 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7fffff640 1 -- 172.21.15.32:0/1213018748 learned_addr learned my addr 172.21.15.32:0/1213018748 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd806ffd640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8080b1d90 0x7fd808154d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd8077fe640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8081603e0 0x7fd8080c8b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7fffff640 1 -- 172.21.15.32:0/1213018748 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8081603e0 msgr2=0x7fd8080c8b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7fffff640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8081603e0 0x7fd8080c8b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:57.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7fffff640 1 -- 172.21.15.32:0/1213018748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8080b1d90 msgr2=0x7fd808154d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:57.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7fffff640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8080b1d90 0x7fd808154d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:57.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7fffff640 1 -- 172.21.15.32:0/1213018748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7f8009420 con 0x7fd8080d2040 2023-12-20T04:40:57.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd806ffd640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8080b1d90 0x7fd808154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:40:57.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7fffff640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8080d2040 0x7fd808154260 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000c3c0 tx=0x7fd7f000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:57.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7ff7fe640 1 -- 172.21.15.32:0/1213018748 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f00052d0 con 0x7fd8080d2040 2023-12-20T04:40:57.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7ff7fe640 1 -- 172.21.15.32:0/1213018748 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7f0005430 con 0x7fd8080d2040 2023-12-20T04:40:57.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd7ff7fe640 1 -- 172.21.15.32:0/1213018748 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7f0003840 con 0x7fd8080d2040 2023-12-20T04:40:57.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd80d36d640 1 -- 172.21.15.32:0/1213018748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8080ca190 con 0x7fd8080d2040 2023-12-20T04:40:57.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.321+0000 7fd80d36d640 1 -- 172.21.15.32:0/1213018748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8080ca750 con 0x7fd8080d2040 2023-12-20T04:40:57.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.325+0000 7fd80d36d640 1 -- 172.21.15.32:0/1213018748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd80804e000 con 0x7fd8080d2040 2023-12-20T04:40:57.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.325+0000 7fd7ff7fe640 1 -- 172.21.15.32:0/1213018748 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd7f00039a0 con 0x7fd8080d2040 2023-12-20T04:40:57.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.325+0000 7fd7ff7fe640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7e8065b80 0x7fd7e8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:57.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.325+0000 7fd7ff7fe640 1 -- 172.21.15.32:0/1213018748 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(57..57 src has 1..57) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd7f0082450 con 0x7fd8080d2040 2023-12-20T04:40:57.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.325+0000 7fd806ffd640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7e8065b80 0x7fd7e8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:57.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.325+0000 7fd806ffd640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7e8065b80 0x7fd7e8068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80ae000 tx=0x7fd7f8008040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:57.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.329+0000 7fd7ff7fe640 1 -- 172.21.15.32:0/1213018748 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd7f00500e0 con 0x7fd8080d2040 2023-12-20T04:40:57.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:57.489+0000 7fd80d36d640 1 -- 172.21.15.32:0/1213018748 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fd8080cd980 con 0x7fd8080d2040 2023-12-20T04:40:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.505+0000 7fd7ff7fe640 1 -- 172.21.15.32:0/1213018748 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v58) v1 ==== 138+0+0 (secure 0 0 0) 0x7fd7f0053700 con 0x7fd8080d2040 2023-12-20T04:40:58.507 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T04:40:58.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.509+0000 7fd80d36d640 1 -- 172.21.15.32:0/1213018748 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7e8065b80 msgr2=0x7fd7e8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.509+0000 7fd80d36d640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7e8065b80 0x7fd7e8068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fd7f80ae000 tx=0x7fd7f8008040 comp rx=0 tx=0).stop 2023-12-20T04:40:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.509+0000 7fd80d36d640 1 -- 172.21.15.32:0/1213018748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8080d2040 msgr2=0x7fd808154260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.509+0000 7fd80d36d640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8080d2040 0x7fd808154260 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7fd7f000c3c0 tx=0x7fd7f000c880 comp rx=0 tx=0).stop 2023-12-20T04:40:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.513+0000 7fd80d36d640 1 -- 172.21.15.32:0/1213018748 shutdown_connections 2023-12-20T04:40:58.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.513+0000 7fd80d36d640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd7e8065b80 0x7fd7e8068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:58.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.513+0000 7fd80d36d640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8081603e0 0x7fd8080c8b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:58.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.513+0000 7fd80d36d640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8080d2040 0x7fd808154260 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:58.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.513+0000 7fd80d36d640 1 --2- 172.21.15.32:0/1213018748 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8080b1d90 0x7fd808154d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:58.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.513+0000 7fd80d36d640 1 -- 172.21.15.32:0/1213018748 >> 172.21.15.32:0/1213018748 conn(0x7fd80806b5e0 msgr2=0x7fd8080d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:58.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.513+0000 7fd80d36d640 1 -- 172.21.15.32:0/1213018748 shutdown_connections 2023-12-20T04:40:58.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.513+0000 7fd80d36d640 1 -- 172.21.15.32:0/1213018748 wait complete. 2023-12-20T04:40:58.536 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T04:40:58.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c8dd8f640 1 Processor -- start 2023-12-20T04:40:58.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c8dd8f640 1 -- start start 2023-12-20T04:40:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c8dd8f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5c880adf00 0x7f5c880cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c8dd8f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c880c4320 0x7f5c880cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c8dd8f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c88159e60 con 0x7f5c8806da40 2023-12-20T04:40:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c8dd8f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c880c3630 con 0x7f5c880c4320 2023-12-20T04:40:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c8dd8f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c880ce380 con 0x7f5c880adf00 2023-12-20T04:40:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c86ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c880c4320 0x7f5c880cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:58.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c87fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5c8806da40 0x7f5c8815d550 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46754/0 (socket says 172.21.15.32:46754) 2023-12-20T04:40:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c86ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c880c4320 0x7f5c880cdd90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46416/0 (socket says 172.21.15.32:46416) 2023-12-20T04:40:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c87fff640 1 -- 172.21.15.32:0/516749448 learned_addr learned my addr 172.21.15.32:0/516749448 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c877fe640 1 --2- 172.21.15.32:0/516749448 >> v2:172.21.15.39:3300/0 conn(0x7f5c880adf00 0x7f5c880cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c86ffd640 1 -- 172.21.15.32:0/516749448 >> v2:172.21.15.39:3300/0 conn(0x7f5c880adf00 msgr2=0x7f5c880cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:40:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c86ffd640 1 --2- 172.21.15.32:0/516749448 >> v2:172.21.15.39:3300/0 conn(0x7f5c880adf00 0x7f5c880cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c86ffd640 1 -- 172.21.15.32:0/516749448 >> v1:172.21.15.32:6789/0 conn(0x7f5c8806da40 legacy=0x7f5c8815d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c86ffd640 1 -- 172.21.15.32:0/516749448 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c880c3840 con 0x7f5c880c4320 2023-12-20T04:40:58.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.689+0000 7f5c867fc640 1 -- 172.21.15.32:0/516749448 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3801211557 0 0) 0x7f5c88159e60 con 0x7f5c8806da40 2023-12-20T04:40:58.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c86ffd640 1 --2- 172.21.15.32:0/516749448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c880c4320 0x7f5c880cdd90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5c880b0b30 tx=0x7f5c7c01eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9fe35d1790922be server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:58.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c867fc640 1 -- 172.21.15.32:0/516749448 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0a7bd0 con 0x7f5c880c4320 2023-12-20T04:40:58.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c867fc640 1 -- 172.21.15.32:0/516749448 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c7c0a7d30 con 0x7f5c880c4320 2023-12-20T04:40:58.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c867fc640 1 -- 172.21.15.32:0/516749448 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0a7ea0 con 0x7f5c880c4320 2023-12-20T04:40:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/516749448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c880c4320 msgr2=0x7f5c880cdd90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c8dd8f640 1 --2- 172.21.15.32:0/516749448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c880c4320 0x7f5c880cdd90 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f5c880b0b30 tx=0x7f5c7c01eb00 comp rx=0 tx=0).stop 2023-12-20T04:40:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/516749448 shutdown_connections 2023-12-20T04:40:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c8dd8f640 1 --2- 172.21.15.32:0/516749448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c880c4320 0x7f5c880cdd90 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c8dd8f640 1 --2- 172.21.15.32:0/516749448 >> v2:172.21.15.39:3300/0 conn(0x7f5c880adf00 0x7f5c880cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/516749448 >> 172.21.15.32:0/516749448 conn(0x7f5c8806a4b0 msgr2=0x7f5c880b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/516749448 shutdown_connections 2023-12-20T04:40:58.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.693+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/516749448 wait complete. 2023-12-20T04:40:58.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c8dd8f640 1 Processor -- start 2023-12-20T04:40:58.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c8dd8f640 1 -- start start 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c8dd8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8806da40 0x7f5c881f61f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c8dd8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c880adf00 0x7f5c881f7f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c8dd8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c880c4320 0x7f5c880b3590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c8dd8f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c881f8450 con 0x7f5c880c4320 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c8dd8f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815a180 con 0x7f5c8806da40 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c8dd8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c8815a460 con 0x7f5c880adf00 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c880c4320 0x7f5c880b3590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c87fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c880c4320 0x7f5c880b3590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46430/0 (socket says 172.21.15.32:46430) 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c87fff640 1 -- 172.21.15.32:0/3375137901 learned_addr learned my addr 172.21.15.32:0/3375137901 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c86ffd640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c880adf00 0x7f5c881f7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c877fe640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8806da40 0x7f5c881f61f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:58.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c87fff640 1 -- 172.21.15.32:0/3375137901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c880adf00 msgr2=0x7f5c881f7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c87fff640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c880adf00 0x7f5c881f7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c87fff640 1 -- 172.21.15.32:0/3375137901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8806da40 msgr2=0x7f5c881f61f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c87fff640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8806da40 0x7f5c881f61f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c87fff640 1 -- 172.21.15.32:0/3375137901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c7c00b6c0 con 0x7f5c880c4320 2023-12-20T04:40:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c877fe640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8806da40 0x7f5c881f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:40:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c87fff640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c880c4320 0x7f5c880b3590 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000c7a0 tx=0x7f5c8000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c6ffff640 1 -- 172.21.15.32:0/3375137901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8000a640 con 0x7f5c880c4320 2023-12-20T04:40:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c6ffff640 1 -- 172.21.15.32:0/3375137901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c8000a7a0 con 0x7f5c880c4320 2023-12-20T04:40:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/3375137901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c8806e3e0 con 0x7f5c880c4320 2023-12-20T04:40:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c6ffff640 1 -- 172.21.15.32:0/3375137901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c8000abd0 con 0x7f5c880c4320 2023-12-20T04:40:58.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.697+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/3375137901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c8815aa20 con 0x7f5c880c4320 2023-12-20T04:40:58.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.701+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/3375137901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c8815ac10 con 0x7f5c880c4320 2023-12-20T04:40:58.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.701+0000 7f5c6ffff640 1 -- 172.21.15.32:0/3375137901 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5c8000ad30 con 0x7f5c880c4320 2023-12-20T04:40:58.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.701+0000 7f5c6ffff640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c74065b80 0x7f5c74068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:58.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.701+0000 7f5c6ffff640 1 -- 172.21.15.32:0/3375137901 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(58..58 src has 1..58) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f5c80082b80 con 0x7f5c880c4320 2023-12-20T04:40:58.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.701+0000 7f5c877fe640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c74065b80 0x7f5c74068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:58.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.701+0000 7f5c877fe640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c74065b80 0x7f5c74068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009840 tx=0x7f5c78008040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:58.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.705+0000 7f5c6ffff640 1 -- 172.21.15.32:0/3375137901 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5c80050780 con 0x7f5c880c4320 2023-12-20T04:40:58.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:58.861+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/3375137901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f5c8804e000 con 0x7f5c880c4320 2023-12-20T04:40:59.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.517+0000 7f5c6ffff640 1 -- 172.21.15.32:0/3375137901 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v59) v1 ==== 170+0+0 (secure 0 0 0) 0x7f5c800180d0 con 0x7f5c880c4320 2023-12-20T04:40:59.516 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T04:40:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.521+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/3375137901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c74065b80 msgr2=0x7f5c74068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.521+0000 7f5c8dd8f640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c74065b80 0x7f5c74068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f5c78009840 tx=0x7f5c78008040 comp rx=0 tx=0).stop 2023-12-20T04:40:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.521+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/3375137901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c880c4320 msgr2=0x7f5c880b3590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:59.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.521+0000 7f5c8dd8f640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c880c4320 0x7f5c880b3590 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f5c8000c7a0 tx=0x7f5c8000cc60 comp rx=0 tx=0).stop 2023-12-20T04:40:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.521+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/3375137901 shutdown_connections 2023-12-20T04:40:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.521+0000 7f5c8dd8f640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5c74065b80 0x7f5c74068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.521+0000 7f5c8dd8f640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c880c4320 0x7f5c880b3590 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.521+0000 7f5c8dd8f640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c880adf00 0x7f5c881f7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.521+0000 7f5c8dd8f640 1 --2- 172.21.15.32:0/3375137901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c8806da40 0x7f5c881f61f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.521+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/3375137901 >> 172.21.15.32:0/3375137901 conn(0x7f5c8806a4b0 msgr2=0x7f5c880b0320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:59.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.525+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/3375137901 shutdown_connections 2023-12-20T04:40:59.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.525+0000 7f5c8dd8f640 1 -- 172.21.15.32:0/3375137901 wait complete. 2023-12-20T04:40:59.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T04:40:59.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.705+0000 7f375a52c640 1 Processor -- start 2023-12-20T04:40:59.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.705+0000 7f375a52c640 1 -- start start 2023-12-20T04:40:59.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f375a52c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f375406d170 0x7f37540cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:59.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f375a52c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375406a1f0 con 0x7f37540cb050 2023-12-20T04:40:59.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f375a52c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f375406a440 con 0x7f3754065d40 2023-12-20T04:40:59.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f375a52c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f375406a720 con 0x7f375406d170 2023-12-20T04:40:59.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f3753fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f37540cb050 0x7f375406ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35806/0 (socket says 172.21.15.32:35806) 2023-12-20T04:40:59.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f3753fff640 1 -- 172.21.15.32:0/255065869 learned_addr learned my addr 172.21.15.32:0/255065869 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:40:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f3758ac9640 1 --2- 172.21.15.32:0/255065869 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f375406d170 0x7f37540cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f3752ffd640 1 -- 172.21.15.32:0/255065869 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1930613251 0 0) 0x7f375406a1f0 con 0x7f37540cb050 2023-12-20T04:40:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f3752ffd640 1 -- 172.21.15.32:0/255065869 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f373c003530 con 0x7f37540cb050 2023-12-20T04:40:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f3752ffd640 1 -- 172.21.15.32:0/255065869 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3093026499 0 0) 0x7f375406a440 con 0x7f3754065d40 2023-12-20T04:40:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f3752ffd640 1 -- 172.21.15.32:0/255065869 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f375406a1f0 con 0x7f3754065d40 2023-12-20T04:40:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f3758ac9640 1 -- 172.21.15.32:0/255065869 >> v1:172.21.15.39:6789/0 conn(0x7f3754065d40 legacy=0x7f37540c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:59.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f3758ac9640 1 -- 172.21.15.32:0/255065869 >> v1:172.21.15.32:6789/0 conn(0x7f37540cb050 legacy=0x7f375406ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:59.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f3758ac9640 1 -- 172.21.15.32:0/255065869 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f375406aa00 con 0x7f375406d170 2023-12-20T04:40:59.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.709+0000 7f3758ac9640 1 --2- 172.21.15.32:0/255065869 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f375406d170 0x7f37540cb430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f374c009dc0 tx=0x7f374c01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9b21544de5c8b337 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f3752ffd640 1 -- 172.21.15.32:0/255065869 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c0a5a00 con 0x7f375406d170 2023-12-20T04:40:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f3752ffd640 1 -- 172.21.15.32:0/255065869 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f374c0a5b60 con 0x7f375406d170 2023-12-20T04:40:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f3752ffd640 1 -- 172.21.15.32:0/255065869 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f374c0ad440 con 0x7f375406d170 2023-12-20T04:40:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 -- 172.21.15.32:0/255065869 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f375406d170 msgr2=0x7f37540cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:59.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 --2- 172.21.15.32:0/255065869 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f375406d170 0x7f37540cb430 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f374c009dc0 tx=0x7f374c01c940 comp rx=0 tx=0).stop 2023-12-20T04:40:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 -- 172.21.15.32:0/255065869 shutdown_connections 2023-12-20T04:40:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 --2- 172.21.15.32:0/255065869 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f375406d170 0x7f37540cb430 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 -- 172.21.15.32:0/255065869 >> 172.21.15.32:0/255065869 conn(0x7f37540cf8d0 msgr2=0x7f37540cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 -- 172.21.15.32:0/255065869 shutdown_connections 2023-12-20T04:40:59.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 -- 172.21.15.32:0/255065869 wait complete. 2023-12-20T04:40:59.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 Processor -- start 2023-12-20T04:40:59.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 -- start start 2023-12-20T04:40:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3754065d40 0x7f37540c5950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f375406d170 0x7f3754156dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37540cb050 0x7f3754147fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3754154490 con 0x7f375406d170 2023-12-20T04:40:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3754159770 con 0x7f37540cb050 2023-12-20T04:40:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.713+0000 7f375a52c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37540c5e90 con 0x7f3754065d40 2023-12-20T04:40:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f37537fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f375406d170 0x7f3754156dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3753fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3754065d40 0x7f37540c5950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f37537fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f375406d170 0x7f3754156dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51306/0 (socket says 172.21.15.32:51306) 2023-12-20T04:40:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f37537fe640 1 -- 172.21.15.32:0/3469239423 learned_addr learned my addr 172.21.15.32:0/3469239423 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:40:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3758ac9640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37540cb050 0x7f3754147fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3753fff640 1 -- 172.21.15.32:0/3469239423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37540cb050 msgr2=0x7f3754147fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3753fff640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37540cb050 0x7f3754147fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3753fff640 1 -- 172.21.15.32:0/3469239423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f375406d170 msgr2=0x7f3754156dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3753fff640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f375406d170 0x7f3754156dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3753fff640 1 -- 172.21.15.32:0/3469239423 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f374c009a70 con 0x7f3754065d40 2023-12-20T04:40:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f37537fe640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f375406d170 0x7f3754156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:40:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3753fff640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3754065d40 0x7f37540c5950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f374400c400 tx=0x7f374400c8c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3750ff9640 1 -- 172.21.15.32:0/3469239423 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3744005cf0 con 0x7f3754065d40 2023-12-20T04:40:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3750ff9640 1 -- 172.21.15.32:0/3469239423 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3744005e50 con 0x7f3754065d40 2023-12-20T04:40:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f375a52c640 1 -- 172.21.15.32:0/3469239423 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3754153ed0 con 0x7f3754065d40 2023-12-20T04:40:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3750ff9640 1 -- 172.21.15.32:0/3469239423 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37440072d0 con 0x7f3754065d40 2023-12-20T04:40:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f375a52c640 1 -- 172.21.15.32:0/3469239423 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3754159400 con 0x7f3754065d40 2023-12-20T04:40:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f375a52c640 1 -- 172.21.15.32:0/3469239423 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3754159260 con 0x7f3754065d40 2023-12-20T04:40:59.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.717+0000 7f3750ff9640 1 -- 172.21.15.32:0/3469239423 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3744007430 con 0x7f3754065d40 2023-12-20T04:40:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.721+0000 7f3750ff9640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3740065bb0 0x7f3740068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:40:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.721+0000 7f3750ff9640 1 -- 172.21.15.32:0/3469239423 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(59..59 src has 1..59) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f37440827c0 con 0x7f3754065d40 2023-12-20T04:40:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.721+0000 7f37537fe640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3740065bb0 0x7f3740068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:40:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.721+0000 7f37537fe640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3740065bb0 0x7f3740068070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f375414f300 tx=0x7f3748009040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:40:59.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.725+0000 7f3750ff9640 1 -- 172.21.15.32:0/3469239423 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3754159260 con 0x7f3754065d40 2023-12-20T04:40:59.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.913+0000 7f375a52c640 1 -- 172.21.15.32:0/3469239423 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f3754159260 con 0x7f3754065d40 2023-12-20T04:40:59.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.937+0000 7f3750ff9640 1 -- 172.21.15.32:0/3469239423 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 6 and data pool 7 v20) v1 ==== 172+0+0 (secure 0 0 0) 0x7f3754159260 con 0x7f3754065d40 2023-12-20T04:40:59.937 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 6 and data pool 7 2023-12-20T04:40:59.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 -- 172.21.15.32:0/3469239423 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3740065bb0 msgr2=0x7f3740068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3740065bb0 0x7f3740068070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f375414f300 tx=0x7f3748009040 comp rx=0 tx=0).stop 2023-12-20T04:40:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 -- 172.21.15.32:0/3469239423 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3754065d40 msgr2=0x7f37540c5950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:40:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3754065d40 0x7f37540c5950 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f374400c400 tx=0x7f374400c8c0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 -- 172.21.15.32:0/3469239423 shutdown_connections 2023-12-20T04:40:59.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3740065bb0 0x7f3740068070 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37540cb050 0x7f3754147fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f375406d170 0x7f3754156dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 --2- 172.21.15.32:0/3469239423 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3754065d40 0x7f37540c5950 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:40:59.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 -- 172.21.15.32:0/3469239423 >> 172.21.15.32:0/3469239423 conn(0x7f37540cf8d0 msgr2=0x7f37540ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:40:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 -- 172.21.15.32:0/3469239423 shutdown_connections 2023-12-20T04:40:59.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:40:59.945+0000 7f375a52c640 1 -- 172.21.15.32:0/3469239423 wait complete. 2023-12-20T04:40:59.973 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:41:00.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.117+0000 7f6c4da2d640 1 Processor -- start 2023-12-20T04:41:00.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.117+0000 7f6c4da2d640 1 -- start start 2023-12-20T04:41:00.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c4da2d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6c481599e0 0x7f6c480aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c4da2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c480c90e0 0x7f6c480b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c4da2d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c480b0b30 0x7f6c4815d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c4da2d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c480c98c0 con 0x7f6c480b0b30 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c4da2d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c480c9ba0 con 0x7f6c481599e0 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c4da2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c480c9e80 con 0x7f6c480c90e0 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c477fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c480b0b30 0x7f6c4815d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c477fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c480b0b30 0x7f6c4815d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51316/0 (socket says 172.21.15.32:51316) 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c477fe640 1 -- 172.21.15.32:0/2655848559 learned_addr learned my addr 172.21.15.32:0/2655848559 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c46ffd640 1 --2- 172.21.15.32:0/2655848559 >> v2:172.21.15.39:3300/0 conn(0x7f6c481599e0 0x7f6c480aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c3ffff640 1 --2- 172.21.15.32:0/2655848559 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c480c90e0 0x7f6c480b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c477fe640 1 -- 172.21.15.32:0/2655848559 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c480c90e0 msgr2=0x7f6c480b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c477fe640 1 --2- 172.21.15.32:0/2655848559 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c480c90e0 0x7f6c480b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c477fe640 1 -- 172.21.15.32:0/2655848559 >> v2:172.21.15.39:3300/0 conn(0x7f6c481599e0 msgr2=0x7f6c480aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c477fe640 1 --2- 172.21.15.32:0/2655848559 >> v2:172.21.15.39:3300/0 conn(0x7f6c481599e0 0x7f6c480aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c477fe640 1 -- 172.21.15.32:0/2655848559 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c480ca160 con 0x7f6c480b0b30 2023-12-20T04:41:00.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.121+0000 7f6c477fe640 1 --2- 172.21.15.32:0/2655848559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c480b0b30 0x7f6c4815d6e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6c4806e3c0 tx=0x7f6c4001e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8938849f96364ce7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:00.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c467fc640 1 -- 172.21.15.32:0/2655848559 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4000b2d0 con 0x7f6c480b0b30 2023-12-20T04:41:00.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c467fc640 1 -- 172.21.15.32:0/2655848559 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4000b430 con 0x7f6c480b0b30 2023-12-20T04:41:00.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c467fc640 1 -- 172.21.15.32:0/2655848559 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400a79b0 con 0x7f6c480b0b30 2023-12-20T04:41:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 -- 172.21.15.32:0/2655848559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c480b0b30 msgr2=0x7f6c4815d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 --2- 172.21.15.32:0/2655848559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c480b0b30 0x7f6c4815d6e0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f6c4806e3c0 tx=0x7f6c4001e8e0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 -- 172.21.15.32:0/2655848559 shutdown_connections 2023-12-20T04:41:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 --2- 172.21.15.32:0/2655848559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c480b0b30 0x7f6c4815d6e0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 --2- 172.21.15.32:0/2655848559 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c480c90e0 0x7f6c480b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 --2- 172.21.15.32:0/2655848559 >> v2:172.21.15.39:3300/0 conn(0x7f6c481599e0 0x7f6c480aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 -- 172.21.15.32:0/2655848559 >> 172.21.15.32:0/2655848559 conn(0x7f6c4806a320 msgr2=0x7f6c480ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 -- 172.21.15.32:0/2655848559 shutdown_connections 2023-12-20T04:41:00.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 -- 172.21.15.32:0/2655848559 wait complete. 2023-12-20T04:41:00.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 Processor -- start 2023-12-20T04:41:00.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 -- start start 2023-12-20T04:41:00.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c480b0b30 0x7f6c4814f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c480c90e0 0x7f6c48159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.125+0000 7f6c4da2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c481599e0 0x7f6c480b33f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c4da2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4815cf80 con 0x7f6c480c90e0 2023-12-20T04:41:00.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c4da2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4814f810 con 0x7f6c481599e0 2023-12-20T04:41:00.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c4da2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c48158330 con 0x7f6c480b0b30 2023-12-20T04:41:00.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c3ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c480c90e0 0x7f6c48159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c3ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c480c90e0 0x7f6c48159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51324/0 (socket says 172.21.15.32:51324) 2023-12-20T04:41:00.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c3ffff640 1 -- 172.21.15.32:0/3319870575 learned_addr learned my addr 172.21.15.32:0/3319870575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:00.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c46ffd640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c480b0b30 0x7f6c4814f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c477fe640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c481599e0 0x7f6c480b33f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c3ffff640 1 -- 172.21.15.32:0/3319870575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c480b0b30 msgr2=0x7f6c4814f2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c3ffff640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c480b0b30 0x7f6c4814f2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c3ffff640 1 -- 172.21.15.32:0/3319870575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c481599e0 msgr2=0x7f6c480b33f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c3ffff640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c481599e0 0x7f6c480b33f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c3ffff640 1 -- 172.21.15.32:0/3319870575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4000bd40 con 0x7f6c480c90e0 2023-12-20T04:41:00.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c3ffff640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c480c90e0 0x7f6c48159dc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000e930 tx=0x7f6c3000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:00.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c44ff9640 1 -- 172.21.15.32:0/3319870575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c30009690 con 0x7f6c480c90e0 2023-12-20T04:41:00.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c44ff9640 1 -- 172.21.15.32:0/3319870575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c3000ca00 con 0x7f6c480c90e0 2023-12-20T04:41:00.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c44ff9640 1 -- 172.21.15.32:0/3319870575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c3000cd70 con 0x7f6c480c90e0 2023-12-20T04:41:00.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c4da2d640 1 -- 172.21.15.32:0/3319870575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c48158610 con 0x7f6c480c90e0 2023-12-20T04:41:00.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c4da2d640 1 -- 172.21.15.32:0/3319870575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4815a300 con 0x7f6c480c90e0 2023-12-20T04:41:00.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c4da2d640 1 -- 172.21.15.32:0/3319870575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4804e000 con 0x7f6c480c90e0 2023-12-20T04:41:00.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.129+0000 7f6c44ff9640 1 -- 172.21.15.32:0/3319870575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6c30013470 con 0x7f6c480c90e0 2023-12-20T04:41:00.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.133+0000 7f6c44ff9640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c28065b80 0x7f6c28068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.133+0000 7f6c44ff9640 1 -- 172.21.15.32:0/3319870575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c30083580 con 0x7f6c480c90e0 2023-12-20T04:41:00.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.133+0000 7f6c46ffd640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c28065b80 0x7f6c28068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.133+0000 7f6c46ffd640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c28065b80 0x7f6c28068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6c481f4b70 tx=0x7f6c38006f90 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:00.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.137+0000 7f6c44ff9640 1 -- 172.21.15.32:0/3319870575 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c30050f00 con 0x7f6c480c90e0 2023-12-20T04:41:00.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.297+0000 7f6c4da2d640 1 -- 172.21.15.32:0/3319870575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6c4815d0c0 con 0x7f6c480c90e0 2023-12-20T04:41:00.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.297+0000 7f6c44ff9640 1 -- 172.21.15.32:0/3319870575 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16010 (secure 0 0 0) 0x7f6c4815d0c0 con 0x7f6c480c90e0 2023-12-20T04:41:00.298 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:41:00.298 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":60,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:40:59.920038+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:40:57.540206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-20T04:40:58.913164+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 -- 172.21.15.32:0/3319870575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c28065b80 msgr2=0x7f6c28068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c28065b80 0x7f6c28068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f6c481f4b70 tx=0x7f6c38006f90 comp rx=0 tx=0).stop 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 -- 172.21.15.32:0/3319870575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c480c90e0 msgr2=0x7f6c48159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c480c90e0 0x7f6c48159dc0 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f6c3000e930 tx=0x7f6c3000edf0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 -- 172.21.15.32:0/3319870575 shutdown_connections 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c28065b80 0x7f6c28068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c481599e0 0x7f6c480b33f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c480c90e0 0x7f6c48159dc0 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 --2- 172.21.15.32:0/3319870575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c480b0b30 0x7f6c4814f2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 -- 172.21.15.32:0/3319870575 >> 172.21.15.32:0/3319870575 conn(0x7f6c4806a320 msgr2=0x7f6c480caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 -- 172.21.15.32:0/3319870575 shutdown_connections 2023-12-20T04:41:00.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.301+0000 7f6c4da2d640 1 -- 172.21.15.32:0/3319870575 wait complete. 2023-12-20T04:41:00.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:41:00.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96e8b6640 1 Processor -- start 2023-12-20T04:41:00.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96e8b6640 1 -- start start 2023-12-20T04:41:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96e8b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff96815ace0 0x7ff9680b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96e8b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9680709d0 0x7ff9680afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96e8b6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff968070570 con 0x7ff968070e30 2023-12-20T04:41:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96e8b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9680cab80 con 0x7ff9680709d0 2023-12-20T04:41:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96e8b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9680c9c20 con 0x7ff96815ace0 2023-12-20T04:41:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96d0b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9680709d0 0x7ff9680afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96e0b5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff968070e30 0x7ff96815e870 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35816/0 (socket says 172.21.15.32:35816) 2023-12-20T04:41:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96e0b5640 1 -- 172.21.15.32:0/3833440727 learned_addr learned my addr 172.21.15.32:0/3833440727 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:41:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96d8b4640 1 --2- 172.21.15.32:0/3833440727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff96815ace0 0x7ff9680b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96c8b2640 1 -- 172.21.15.32:0/3833440727 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 968229101 0 0) 0x7ff968070570 con 0x7ff968070e30 2023-12-20T04:41:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96d0b3640 1 -- 172.21.15.32:0/3833440727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff96815ace0 msgr2=0x7ff9680b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96d0b3640 1 --2- 172.21.15.32:0/3833440727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff96815ace0 0x7ff9680b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96d0b3640 1 -- 172.21.15.32:0/3833440727 >> v1:172.21.15.32:6789/0 conn(0x7ff968070e30 legacy=0x7ff96815e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.465+0000 7ff96d0b3640 1 -- 172.21.15.32:0/3833440727 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9680c9f00 con 0x7ff9680709d0 2023-12-20T04:41:00.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96d0b3640 1 --2- 172.21.15.32:0/3833440727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680709d0 0x7ff9680afff0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff9580097c0 tx=0x7ff95801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf81c39a629dff80 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96c8b2640 1 -- 172.21.15.32:0/3833440727 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9580ad660 con 0x7ff9680709d0 2023-12-20T04:41:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96c8b2640 1 -- 172.21.15.32:0/3833440727 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9580ad7c0 con 0x7ff9680709d0 2023-12-20T04:41:00.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96c8b2640 1 -- 172.21.15.32:0/3833440727 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9580adb50 con 0x7ff9680709d0 2023-12-20T04:41:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96e8b6640 1 -- 172.21.15.32:0/3833440727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680709d0 msgr2=0x7ff9680afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96e8b6640 1 --2- 172.21.15.32:0/3833440727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680709d0 0x7ff9680afff0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7ff9580097c0 tx=0x7ff95801c450 comp rx=0 tx=0).stop 2023-12-20T04:41:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96e8b6640 1 -- 172.21.15.32:0/3833440727 shutdown_connections 2023-12-20T04:41:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96e8b6640 1 --2- 172.21.15.32:0/3833440727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680709d0 0x7ff9680afff0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96e8b6640 1 --2- 172.21.15.32:0/3833440727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff96815ace0 0x7ff9680b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96e8b6640 1 -- 172.21.15.32:0/3833440727 >> 172.21.15.32:0/3833440727 conn(0x7ff96806c060 msgr2=0x7ff96806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96e8b6640 1 -- 172.21.15.32:0/3833440727 shutdown_connections 2023-12-20T04:41:00.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96e8b6640 1 -- 172.21.15.32:0/3833440727 wait complete. 2023-12-20T04:41:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.469+0000 7ff96e8b6640 1 Processor -- start 2023-12-20T04:41:00.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96e8b6640 1 -- start start 2023-12-20T04:41:00.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96e8b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680709d0 0x7ff9680c85c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96e8b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff968070e30 0x7ff9680c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96e8b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96815ace0 0x7ff9681515e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96e8b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9680c6690 con 0x7ff9680709d0 2023-12-20T04:41:00.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96e8b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9681555c0 con 0x7ff96815ace0 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96e8b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9680c57e0 con 0x7ff968070e30 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96d8b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680709d0 0x7ff9680c85c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96d8b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680709d0 0x7ff9680c85c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51346/0 (socket says 172.21.15.32:51346) 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96d8b4640 1 -- 172.21.15.32:0/2376760778 learned_addr learned my addr 172.21.15.32:0/2376760778 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96d0b3640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff968070e30 0x7ff9680c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96e0b5640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96815ace0 0x7ff9681515e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96d8b4640 1 -- 172.21.15.32:0/2376760778 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff968070e30 msgr2=0x7ff9680c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96d8b4640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff968070e30 0x7ff9680c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96d8b4640 1 -- 172.21.15.32:0/2376760778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96815ace0 msgr2=0x7ff9681515e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96d8b4640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96815ace0 0x7ff9681515e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96d8b4640 1 -- 172.21.15.32:0/2376760778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff958009420 con 0x7ff9680709d0 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96d8b4640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680709d0 0x7ff9680c85c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff96400e870 tx=0x7ff96400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff95e7fc640 1 -- 172.21.15.32:0/2376760778 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff964009510 con 0x7ff9680709d0 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff95e7fc640 1 -- 172.21.15.32:0/2376760778 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff96400eec0 con 0x7ff9680709d0 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff95e7fc640 1 -- 172.21.15.32:0/2376760778 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff96400cb40 con 0x7ff9680709d0 2023-12-20T04:41:00.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96e8b6640 1 -- 172.21.15.32:0/2376760778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9680c5ac0 con 0x7ff9680709d0 2023-12-20T04:41:00.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96e8b6640 1 -- 172.21.15.32:0/2376760778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff968151b20 con 0x7ff9680709d0 2023-12-20T04:41:00.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.473+0000 7ff96e8b6640 1 -- 172.21.15.32:0/2376760778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff968074100 con 0x7ff9680709d0 2023-12-20T04:41:00.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.477+0000 7ff95e7fc640 1 -- 172.21.15.32:0/2376760778 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff96400cca0 con 0x7ff9680709d0 2023-12-20T04:41:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.477+0000 7ff95e7fc640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff94c065b80 0x7ff94c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.477+0000 7ff95e7fc640 1 -- 172.21.15.32:0/2376760778 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff964083b00 con 0x7ff9680709d0 2023-12-20T04:41:00.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.477+0000 7ff96d0b3640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff94c065b80 0x7ff94c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.477+0000 7ff96d0b3640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff94c065b80 0x7ff94c068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9580af000 tx=0x7ff9580a7000 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:00.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.481+0000 7ff95e7fc640 1 -- 172.21.15.32:0/2376760778 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff964051520 con 0x7ff9680709d0 2023-12-20T04:41:00.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.641+0000 7ff96e8b6640 1 -- 172.21.15.32:0/2376760778 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff9680c8d30 con 0x7ff9680709d0 2023-12-20T04:41:00.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.641+0000 7ff95e7fc640 1 -- 172.21.15.32:0/2376760778 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v60) v1 ==== 74+0+16010 (secure 0 0 0) 0x7ff964054b40 con 0x7ff9680709d0 2023-12-20T04:41:00.641 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:41:00.641 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":60,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:40:59.920038+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:40:57.540206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-20T04:40:58.913164+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:41:00.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 -- 172.21.15.32:0/2376760778 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff94c065b80 msgr2=0x7ff94c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff94c065b80 0x7ff94c068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7ff9580af000 tx=0x7ff9580a7000 comp rx=0 tx=0).stop 2023-12-20T04:41:00.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 -- 172.21.15.32:0/2376760778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680709d0 msgr2=0x7ff9680c85c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680709d0 0x7ff9680c85c0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7ff96400e870 tx=0x7ff96400ec70 comp rx=0 tx=0).stop 2023-12-20T04:41:00.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 -- 172.21.15.32:0/2376760778 shutdown_connections 2023-12-20T04:41:00.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff94c065b80 0x7ff94c068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff96815ace0 0x7ff9681515e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff968070e30 0x7ff9680c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 --2- 172.21.15.32:0/2376760778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9680709d0 0x7ff9680c85c0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 -- 172.21.15.32:0/2376760778 >> 172.21.15.32:0/2376760778 conn(0x7ff96806c060 msgr2=0x7ff9680c8f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:00.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 -- 172.21.15.32:0/2376760778 shutdown_connections 2023-12-20T04:41:00.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.645+0000 7ff96e8b6640 1 -- 172.21.15.32:0/2376760778 wait complete. 2023-12-20T04:41:00.663 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T04:41:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.809+0000 7f4e5b37b640 1 Processor -- start 2023-12-20T04:41:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.809+0000 7f4e5b37b640 1 -- start start 2023-12-20T04:41:00.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.809+0000 7f4e5b37b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e5415abb0 0x7f4e540caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.809+0000 7f4e5b37b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e54070230 0x7f4e540706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.809+0000 7f4e5b37b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e5415b6a0 con 0x7f4e540b0cb0 2023-12-20T04:41:00.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.809+0000 7f4e5b37b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e5406e160 con 0x7f4e5415abb0 2023-12-20T04:41:00.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.809+0000 7f4e5b37b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e54070c00 con 0x7f4e54070230 2023-12-20T04:41:00.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e59117640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e5415abb0 0x7f4e540caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e59117640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e5415abb0 0x7f4e540caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51352/0 (socket says 172.21.15.32:51352) 2023-12-20T04:41:00.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e59117640 1 -- 172.21.15.32:0/1261788471 learned_addr learned my addr 172.21.15.32:0/1261788471 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:00.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e59918640 1 --2- 172.21.15.32:0/1261788471 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e54070230 0x7f4e540706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e59117640 1 -- 172.21.15.32:0/1261788471 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e54070230 msgr2=0x7f4e540706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:00.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e59117640 1 --2- 172.21.15.32:0/1261788471 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e54070230 0x7f4e540706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e59117640 1 -- 172.21.15.32:0/1261788471 >> v1:172.21.15.39:6789/0 conn(0x7f4e540b0cb0 legacy=0x7f4e5415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e59117640 1 -- 172.21.15.32:0/1261788471 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e5406d8c0 con 0x7f4e5415abb0 2023-12-20T04:41:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e59918640 1 --2- 172.21.15.32:0/1261788471 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e54070230 0x7f4e540706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:41:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e59117640 1 --2- 172.21.15.32:0/1261788471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e5415abb0 0x7f4e540caaf0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009870 tx=0x7f4e5001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db71526debf38f61 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e4bfff640 1 -- 172.21.15.32:0/1261788471 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500a9400 con 0x7f4e5415abb0 2023-12-20T04:41:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e4bfff640 1 -- 172.21.15.32:0/1261788471 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e500a9560 con 0x7f4e5415abb0 2023-12-20T04:41:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.813+0000 7f4e4bfff640 1 -- 172.21.15.32:0/1261788471 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500a98f0 con 0x7f4e5415abb0 2023-12-20T04:41:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 -- 172.21.15.32:0/1261788471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e5415abb0 msgr2=0x7f4e540caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 --2- 172.21.15.32:0/1261788471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e5415abb0 0x7f4e540caaf0 secure :-1 s=READY pgs=603 cs=0 l=1 rev1=1 crypto rx=0x7f4e50009870 tx=0x7f4e5001c540 comp rx=0 tx=0).stop 2023-12-20T04:41:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 -- 172.21.15.32:0/1261788471 shutdown_connections 2023-12-20T04:41:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 --2- 172.21.15.32:0/1261788471 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e54070230 0x7f4e540706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 --2- 172.21.15.32:0/1261788471 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e5415abb0 0x7f4e540caaf0 unknown :-1 s=CLOSED pgs=603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 -- 172.21.15.32:0/1261788471 >> 172.21.15.32:0/1261788471 conn(0x7f4e5406a510 msgr2=0x7f4e540ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 -- 172.21.15.32:0/1261788471 shutdown_connections 2023-12-20T04:41:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 -- 172.21.15.32:0/1261788471 wait complete. 2023-12-20T04:41:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 Processor -- start 2023-12-20T04:41:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 -- start start 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54070230 0x7f4e54143f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e540b0cb0 0x7f4e5415dd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5415abb0 0x7f4e540c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5415ccb0 con 0x7f4e54070230 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5415cf90 con 0x7f4e5415abb0 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e5b37b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e5415d270 con 0x7f4e540b0cb0 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e59117640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54070230 0x7f4e54143f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e59117640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54070230 0x7f4e54143f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51354/0 (socket says 172.21.15.32:51354) 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e59117640 1 -- 172.21.15.32:0/2897093217 learned_addr learned my addr 172.21.15.32:0/2897093217 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e59918640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5415abb0 0x7f4e540c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e58916640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e540b0cb0 0x7f4e5415dd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e59117640 1 -- 172.21.15.32:0/2897093217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e540b0cb0 msgr2=0x7f4e5415dd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e59117640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e540b0cb0 0x7f4e5415dd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e59117640 1 -- 172.21.15.32:0/2897093217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5415abb0 msgr2=0x7f4e540c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:00.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.817+0000 7f4e59117640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5415abb0 0x7f4e540c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:00.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e59117640 1 -- 172.21.15.32:0/2897093217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e500094d0 con 0x7f4e54070230 2023-12-20T04:41:00.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e59117640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54070230 0x7f4e54143f40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4e5001abf0 tx=0x7f4e500192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:00.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e49ffb640 1 -- 172.21.15.32:0/2897093217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500a9400 con 0x7f4e54070230 2023-12-20T04:41:00.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e49ffb640 1 -- 172.21.15.32:0/2897093217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e500b5070 con 0x7f4e54070230 2023-12-20T04:41:00.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e49ffb640 1 -- 172.21.15.32:0/2897093217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e500b1a20 con 0x7f4e54070230 2023-12-20T04:41:00.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e5b37b640 1 -- 172.21.15.32:0/2897093217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e5415d550 con 0x7f4e54070230 2023-12-20T04:41:00.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e5b37b640 1 -- 172.21.15.32:0/2897093217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e5415e290 con 0x7f4e54070230 2023-12-20T04:41:00.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e5b37b640 1 -- 172.21.15.32:0/2897093217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e5404e000 con 0x7f4e54070230 2023-12-20T04:41:00.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e49ffb640 1 -- 172.21.15.32:0/2897093217 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4e500b1b80 con 0x7f4e54070230 2023-12-20T04:41:00.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:41:00.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e49ffb640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4e38065b80 0x7f4e38068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:00.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e49ffb640 1 -- 172.21.15.32:0/2897093217 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(60..60 src has 1..60) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4e500b7070 con 0x7f4e54070230 2023-12-20T04:41:00.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.821+0000 7f4e58916640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4e38065b80 0x7f4e38068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:00.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.825+0000 7f4e58916640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4e38065b80 0x7f4e38068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4e540c4c40 tx=0x7f4e44009040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:00.826 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:41:00.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:00.825+0000 7f4e49ffb640 1 -- 172.21.15.32:0/2897093217 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4e500eb110 con 0x7f4e54070230 2023-12-20T04:41:01.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.021+0000 7f4e5b37b640 1 -- 172.21.15.32:0/2897093217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4e5415d3b0 con 0x7f4e54070230 2023-12-20T04:41:01.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.941+0000 7f4e49ffb640 1 -- 172.21.15.32:0/2897093217 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v23) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4e5415d3b0 con 0x7f4e54070230 2023-12-20T04:41:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.941+0000 7f4e5b37b640 1 -- 172.21.15.32:0/2897093217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4e38065b80 msgr2=0x7f4e38068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.945+0000 7f4e5b37b640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4e38065b80 0x7f4e38068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4e540c4c40 tx=0x7f4e44009040 comp rx=0 tx=0).stop 2023-12-20T04:41:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.945+0000 7f4e5b37b640 1 -- 172.21.15.32:0/2897093217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54070230 msgr2=0x7f4e54143f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.945+0000 7f4e5b37b640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54070230 0x7f4e54143f40 secure :-1 s=READY pgs=604 cs=0 l=1 rev1=1 crypto rx=0x7f4e5001abf0 tx=0x7f4e500192e0 comp rx=0 tx=0).stop 2023-12-20T04:41:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.945+0000 7f4e5b37b640 1 -- 172.21.15.32:0/2897093217 shutdown_connections 2023-12-20T04:41:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.945+0000 7f4e5b37b640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4e38065b80 0x7f4e38068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.945+0000 7f4e5b37b640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e5415abb0 0x7f4e540c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.945+0000 7f4e5b37b640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e540b0cb0 0x7f4e5415dd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:01.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.945+0000 7f4e5b37b640 1 --2- 172.21.15.32:0/2897093217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e54070230 0x7f4e54143f40 unknown :-1 s=CLOSED pgs=604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.945+0000 7f4e5b37b640 1 -- 172.21.15.32:0/2897093217 >> 172.21.15.32:0/2897093217 conn(0x7f4e5406a510 msgr2=0x7f4e54159d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.945+0000 7f4e5b37b640 1 -- 172.21.15.32:0/2897093217 shutdown_connections 2023-12-20T04:41:01.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:01.945+0000 7f4e5b37b640 1 -- 172.21.15.32:0/2897093217 wait complete. 2023-12-20T04:41:01.966 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T04:41:01.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T04:41:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.117+0000 7fe093725640 1 Processor -- start 2023-12-20T04:41:02.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.117+0000 7fe093725640 1 -- start start 2023-12-20T04:41:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe093725640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe08c15abb0 0x7fe08c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe093725640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe08c0b0cb0 0x7fe08c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe093725640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe08c070230 0x7fe08c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe093725640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08c06e160 con 0x7fe08c0b0cb0 2023-12-20T04:41:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe093725640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe08c070be0 con 0x7fe08c070230 2023-12-20T04:41:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe093725640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c15b6a0 con 0x7fe08c15abb0 2023-12-20T04:41:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe090cc0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe08c0b0cb0 0x7fe08c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe090cc0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe08c0b0cb0 0x7fe08c15bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51362/0 (socket says 172.21.15.32:51362) 2023-12-20T04:41:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe090cc0640 1 -- 172.21.15.32:0/1001596508 learned_addr learned my addr 172.21.15.32:0/1001596508 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe0914c1640 1 --2- 172.21.15.32:0/1001596508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe08c15abb0 0x7fe08c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe091cc2640 1 --2- 172.21.15.32:0/1001596508 >> v2:172.21.15.39:3300/0 conn(0x7fe08c070230 0x7fe08c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe090cc0640 1 -- 172.21.15.32:0/1001596508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe08c15abb0 msgr2=0x7fe08c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe090cc0640 1 --2- 172.21.15.32:0/1001596508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe08c15abb0 0x7fe08c0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe090cc0640 1 -- 172.21.15.32:0/1001596508 >> v2:172.21.15.39:3300/0 conn(0x7fe08c070230 msgr2=0x7fe08c0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe090cc0640 1 --2- 172.21.15.32:0/1001596508 >> v2:172.21.15.39:3300/0 conn(0x7fe08c070230 0x7fe08c0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe090cc0640 1 -- 172.21.15.32:0/1001596508 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe08c15c070 con 0x7fe08c0b0cb0 2023-12-20T04:41:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.121+0000 7fe091cc2640 1 --2- 172.21.15.32:0/1001596508 >> v2:172.21.15.39:3300/0 conn(0x7fe08c070230 0x7fe08c0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:41:02.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe090cc0640 1 --2- 172.21.15.32:0/1001596508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0b0cb0 0x7fe08c15bb30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0b4470 tx=0x7fe07c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=342cc13c7bd289ce server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe083fff640 1 -- 172.21.15.32:0/1001596508 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0aa620 con 0x7fe08c0b0cb0 2023-12-20T04:41:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe083fff640 1 -- 172.21.15.32:0/1001596508 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe07c0aa780 con 0x7fe08c0b0cb0 2023-12-20T04:41:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe083fff640 1 -- 172.21.15.32:0/1001596508 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe07c0aab10 con 0x7fe08c0b0cb0 2023-12-20T04:41:02.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe093725640 1 -- 172.21.15.32:0/1001596508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0b0cb0 msgr2=0x7fe08c15bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe093725640 1 --2- 172.21.15.32:0/1001596508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0b0cb0 0x7fe08c15bb30 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7fe08c0b4470 tx=0x7fe07c01c500 comp rx=0 tx=0).stop 2023-12-20T04:41:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe093725640 1 -- 172.21.15.32:0/1001596508 shutdown_connections 2023-12-20T04:41:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe093725640 1 --2- 172.21.15.32:0/1001596508 >> v2:172.21.15.39:3300/0 conn(0x7fe08c070230 0x7fe08c0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe093725640 1 --2- 172.21.15.32:0/1001596508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c0b0cb0 0x7fe08c15bb30 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe093725640 1 --2- 172.21.15.32:0/1001596508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe08c15abb0 0x7fe08c0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe093725640 1 -- 172.21.15.32:0/1001596508 >> 172.21.15.32:0/1001596508 conn(0x7fe08c06a510 msgr2=0x7fe08c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe093725640 1 -- 172.21.15.32:0/1001596508 shutdown_connections 2023-12-20T04:41:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe093725640 1 -- 172.21.15.32:0/1001596508 wait complete. 2023-12-20T04:41:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe093725640 1 Processor -- start 2023-12-20T04:41:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.125+0000 7fe093725640 1 -- start start 2023-12-20T04:41:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe093725640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe08c070230 0x7fe08c14bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe093725640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe08c0b0cb0 0x7fe08c155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe093725640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c15abb0 0x7fe08c152d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:02.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe093725640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c138ab0 con 0x7fe08c15abb0 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe093725640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c137ea0 con 0x7fe08c070230 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe093725640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe08c138180 con 0x7fe08c0b0cb0 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe091cc2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c15abb0 0x7fe08c152d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe091cc2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c15abb0 0x7fe08c152d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51366/0 (socket says 172.21.15.32:51366) 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe091cc2640 1 -- 172.21.15.32:0/3394542348 learned_addr learned my addr 172.21.15.32:0/3394542348 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe0914c1640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe08c070230 0x7fe08c14bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe090cc0640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe08c0b0cb0 0x7fe08c155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe091cc2640 1 -- 172.21.15.32:0/3394542348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe08c0b0cb0 msgr2=0x7fe08c155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe091cc2640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe08c0b0cb0 0x7fe08c155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe091cc2640 1 -- 172.21.15.32:0/3394542348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe08c070230 msgr2=0x7fe08c14bf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe091cc2640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe08c070230 0x7fe08c14bf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe091cc2640 1 -- 172.21.15.32:0/3394542348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe07c0094d0 con 0x7fe08c15abb0 2023-12-20T04:41:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe091cc2640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c15abb0 0x7fe08c152d80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe08400cd10 tx=0x7fe08400a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe081ffb640 1 -- 172.21.15.32:0/3394542348 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08400f680 con 0x7fe08c15abb0 2023-12-20T04:41:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe081ffb640 1 -- 172.21.15.32:0/3394542348 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe08400f800 con 0x7fe08c15abb0 2023-12-20T04:41:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe093725640 1 -- 172.21.15.32:0/3394542348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe08c138460 con 0x7fe08c15abb0 2023-12-20T04:41:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe081ffb640 1 -- 172.21.15.32:0/3394542348 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe08400fbb0 con 0x7fe08c15abb0 2023-12-20T04:41:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe093725640 1 -- 172.21.15.32:0/3394542348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe08c155d60 con 0x7fe08c15abb0 2023-12-20T04:41:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.129+0000 7fe093725640 1 -- 172.21.15.32:0/3394542348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe08c138c20 con 0x7fe08c15abb0 2023-12-20T04:41:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.133+0000 7fe081ffb640 1 -- 172.21.15.32:0/3394542348 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe08400b040 con 0x7fe08c15abb0 2023-12-20T04:41:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.133+0000 7fe081ffb640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe070065b80 0x7fe070068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.133+0000 7fe081ffb640 1 -- 172.21.15.32:0/3394542348 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe084083c80 con 0x7fe08c15abb0 2023-12-20T04:41:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.133+0000 7fe0914c1640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe070065b80 0x7fe070068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.133+0000 7fe0914c1640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe070065b80 0x7fe070068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe088007a60 tx=0x7fe088008040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:02.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.137+0000 7fe081ffb640 1 -- 172.21.15.32:0/3394542348 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe0840514d0 con 0x7fe08c15abb0 2023-12-20T04:41:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.333+0000 7fe093725640 1 -- 172.21.15.32:0/3394542348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fe08c04e000 con 0x7fe08c15abb0 2023-12-20T04:41:02.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.941+0000 7fe081ffb640 1 -- 172.21.15.32:0/3394542348 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v24) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe08401d0c0 con 0x7fe08c15abb0 2023-12-20T04:41:02.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 -- 172.21.15.32:0/3394542348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe070065b80 msgr2=0x7fe070068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:02.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe070065b80 0x7fe070068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fe088007a60 tx=0x7fe088008040 comp rx=0 tx=0).stop 2023-12-20T04:41:02.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 -- 172.21.15.32:0/3394542348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c15abb0 msgr2=0x7fe08c152d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:02.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c15abb0 0x7fe08c152d80 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fe08400cd10 tx=0x7fe08400a650 comp rx=0 tx=0).stop 2023-12-20T04:41:02.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 -- 172.21.15.32:0/3394542348 shutdown_connections 2023-12-20T04:41:02.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe070065b80 0x7fe070068040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:02.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe08c15abb0 0x7fe08c152d80 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:02.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe08c0b0cb0 0x7fe08c155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:02.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 --2- 172.21.15.32:0/3394542348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe08c070230 0x7fe08c14bf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:02.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 -- 172.21.15.32:0/3394542348 >> 172.21.15.32:0/3394542348 conn(0x7fe08c06a510 msgr2=0x7fe08c0b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:02.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 -- 172.21.15.32:0/3394542348 shutdown_connections 2023-12-20T04:41:02.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:02.945+0000 7fe093725640 1 -- 172.21.15.32:0/3394542348 wait complete. 2023-12-20T04:41:02.965 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T04:41:02.965 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:41:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.109+0000 7f273590b640 1 Processor -- start 2023-12-20T04:41:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f273590b640 1 -- start start 2023-12-20T04:41:03.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f273590b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27300b8480 0x7f273006af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f273590b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f273006b460 0x7f2730160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f273590b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2730161330 0x7f2730153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f273590b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f273006b8c0 con 0x7f27300b8480 2023-12-20T04:41:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f273590b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27300cd4d0 con 0x7f2730161330 2023-12-20T04:41:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f273590b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27300ce4e0 con 0x7f273006b460 2023-12-20T04:41:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f2734909640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27300b8480 0x7f273006af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f2734909640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27300b8480 0x7f273006af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51382/0 (socket says 172.21.15.32:51382) 2023-12-20T04:41:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f2734909640 1 -- 172.21.15.32:0/4125973132 learned_addr learned my addr 172.21.15.32:0/4125973132 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f272ffff640 1 --2- 172.21.15.32:0/4125973132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f273006b460 0x7f2730160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f273510a640 1 --2- 172.21.15.32:0/4125973132 >> v2:172.21.15.39:3300/0 conn(0x7f2730161330 0x7f2730153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f2734909640 1 -- 172.21.15.32:0/4125973132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f273006b460 msgr2=0x7f2730160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f2734909640 1 --2- 172.21.15.32:0/4125973132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f273006b460 0x7f2730160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f2734909640 1 -- 172.21.15.32:0/4125973132 >> v2:172.21.15.39:3300/0 conn(0x7f2730161330 msgr2=0x7f2730153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f2734909640 1 --2- 172.21.15.32:0/4125973132 >> v2:172.21.15.39:3300/0 conn(0x7f2730161330 0x7f2730153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.113+0000 7f2734909640 1 -- 172.21.15.32:0/4125973132 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27300cd720 con 0x7f27300b8480 2023-12-20T04:41:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f2734909640 1 --2- 172.21.15.32:0/4125973132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27300b8480 0x7f273006af20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f273006bae0 tx=0x7f272401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a66406f34b3414 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f272f7fe640 1 -- 172.21.15.32:0/4125973132 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27240ac410 con 0x7f27300b8480 2023-12-20T04:41:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f272f7fe640 1 -- 172.21.15.32:0/4125973132 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27240ac570 con 0x7f27300b8480 2023-12-20T04:41:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f272f7fe640 1 -- 172.21.15.32:0/4125973132 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27240ac900 con 0x7f27300b8480 2023-12-20T04:41:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f273590b640 1 -- 172.21.15.32:0/4125973132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27300b8480 msgr2=0x7f273006af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f273590b640 1 --2- 172.21.15.32:0/4125973132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27300b8480 0x7f273006af20 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f273006bae0 tx=0x7f272401c500 comp rx=0 tx=0).stop 2023-12-20T04:41:03.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f273590b640 1 -- 172.21.15.32:0/4125973132 shutdown_connections 2023-12-20T04:41:03.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f273590b640 1 --2- 172.21.15.32:0/4125973132 >> v2:172.21.15.39:3300/0 conn(0x7f2730161330 0x7f2730153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f273590b640 1 --2- 172.21.15.32:0/4125973132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f273006b460 0x7f2730160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f273590b640 1 --2- 172.21.15.32:0/4125973132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27300b8480 0x7f273006af20 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f273590b640 1 -- 172.21.15.32:0/4125973132 >> 172.21.15.32:0/4125973132 conn(0x7f27300d2a50 msgr2=0x7f27300d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:03.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f273590b640 1 -- 172.21.15.32:0/4125973132 shutdown_connections 2023-12-20T04:41:03.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f273590b640 1 -- 172.21.15.32:0/4125973132 wait complete. 2023-12-20T04:41:03.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f273590b640 1 Processor -- start 2023-12-20T04:41:03.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.117+0000 7f273590b640 1 -- start start 2023-12-20T04:41:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f273590b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f273006b460 0x7f273014b830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f273590b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27300b8480 0x7f273014cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f273590b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2730161330 0x7f2730156920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f273590b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2730156e60 con 0x7f27300b8480 2023-12-20T04:41:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f273590b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f273014bec0 con 0x7f273006b460 2023-12-20T04:41:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f273590b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f273014c1a0 con 0x7f2730161330 2023-12-20T04:41:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27300b8480 0x7f273014cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27300b8480 0x7f273014cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51396/0 (socket says 172.21.15.32:51396) 2023-12-20T04:41:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272ffff640 1 -- 172.21.15.32:0/2527392821 learned_addr learned my addr 172.21.15.32:0/2527392821 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f2734909640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f273006b460 0x7f273014b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f273510a640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2730161330 0x7f2730156920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272ffff640 1 -- 172.21.15.32:0/2527392821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2730161330 msgr2=0x7f2730156920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272ffff640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2730161330 0x7f2730156920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272ffff640 1 -- 172.21.15.32:0/2527392821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f273006b460 msgr2=0x7f273014b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272ffff640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f273006b460 0x7f273014b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272ffff640 1 -- 172.21.15.32:0/2527392821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27240094d0 con 0x7f27300b8480 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f273510a640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2730161330 0x7f2730156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272ffff640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27300b8480 0x7f273014cf60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f271c00d800 tx=0x7f271c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272d7fa640 1 -- 172.21.15.32:0/2527392821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c0096a0 con 0x7f27300b8480 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272d7fa640 1 -- 172.21.15.32:0/2527392821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f271c009800 con 0x7f27300b8480 2023-12-20T04:41:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f273590b640 1 -- 172.21.15.32:0/2527392821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f273014c480 con 0x7f27300b8480 2023-12-20T04:41:03.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272d7fa640 1 -- 172.21.15.32:0/2527392821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f271c009b70 con 0x7f27300b8480 2023-12-20T04:41:03.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f273590b640 1 -- 172.21.15.32:0/2527392821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f273014ca40 con 0x7f27300b8480 2023-12-20T04:41:03.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f273590b640 1 -- 172.21.15.32:0/2527392821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2730074100 con 0x7f27300b8480 2023-12-20T04:41:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.121+0000 7f272d7fa640 1 -- 172.21.15.32:0/2527392821 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f271c009cd0 con 0x7f27300b8480 2023-12-20T04:41:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.125+0000 7f272d7fa640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2714065b80 0x7f2714068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.125+0000 7f272d7fa640 1 -- 172.21.15.32:0/2527392821 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f271c083a80 con 0x7f27300b8480 2023-12-20T04:41:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.125+0000 7f2734909640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2714065b80 0x7f2714068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.125+0000 7f2734909640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2714065b80 0x7f2714068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27240ae000 tx=0x7f2724009040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.129+0000 7f272d7fa640 1 -- 172.21.15.32:0/2527392821 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f271c051290 con 0x7f27300b8480 2023-12-20T04:41:03.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 -- 172.21.15.32:0/2527392821 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f27300d01a0 con 0x7f27300b8480 2023-12-20T04:41:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f272d7fa640 1 -- 172.21.15.32:0/2527392821 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4700 (secure 0 0 0) 0x7f271c0548b0 con 0x7f27300b8480 2023-12-20T04:41:03.320 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:41:03.320 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/536564441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":536564441},{"type":"v1","addr":"172.21.15.32:6835","nonce":536564441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2898125535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2898125535},{"type":"v1","addr":"172.21.15.39:6835","nonce":2898125535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/319656564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":319656564},{"type":"v1","addr":"172.21.15.32:6837","nonce":319656564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:40:59.920384+0000","modified":"2023-12-20T04:41:02.943287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5895},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5895":{"gid":5895,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/192084837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":192084837},{"type":"v1","addr":"172.21.15.39:6834","nonce":192084837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-20T04:41:03.320 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 -- 172.21.15.32:0/2527392821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2714065b80 msgr2=0x7f2714068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2714065b80 0x7f2714068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f27240ae000 tx=0x7f2724009040 comp rx=0 tx=0).stop 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 -- 172.21.15.32:0/2527392821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27300b8480 msgr2=0x7f273014cf60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27300b8480 0x7f273014cf60 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f271c00d800 tx=0x7f271c00dcc0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 -- 172.21.15.32:0/2527392821 shutdown_connections 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2714065b80 0x7f2714068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2730161330 0x7f2730156920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27300b8480 0x7f273014cf60 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 --2- 172.21.15.32:0/2527392821 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f273006b460 0x7f273014b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 -- 172.21.15.32:0/2527392821 >> 172.21.15.32:0/2527392821 conn(0x7f27300d2a50 msgr2=0x7f273015e620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 -- 172.21.15.32:0/2527392821 shutdown_connections 2023-12-20T04:41:03.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.321+0000 7f273590b640 1 -- 172.21.15.32:0/2527392821 wait complete. 2023-12-20T04:41:03.341 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:41:03.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f89663db640 1 Processor -- start 2023-12-20T04:41:03.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f89663db640 1 -- start start 2023-12-20T04:41:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f89663db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f896015cdb0 0x7f89600d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f89663db640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89600d2740 0x7f89600cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f89663db640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89600b6ca0 con 0x7f89600d1120 2023-12-20T04:41:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f89663db640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f896015d190 con 0x7f89600d2740 2023-12-20T04:41:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f89663db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89600d16a0 con 0x7f896015cdb0 2023-12-20T04:41:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f895f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89600d2740 0x7f89600cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f8964978640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f89600d1120 0x7f89600b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35824/0 (socket says 172.21.15.32:35824) 2023-12-20T04:41:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f8964978640 1 -- 172.21.15.32:0/889191487 learned_addr learned my addr 172.21.15.32:0/889191487 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:41:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f895ffff640 1 --2- 172.21.15.32:0/889191487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f896015cdb0 0x7f89600d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f895f7fe640 1 -- 172.21.15.32:0/889191487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f896015cdb0 msgr2=0x7f89600d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f895f7fe640 1 --2- 172.21.15.32:0/889191487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f896015cdb0 0x7f89600d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f895f7fe640 1 -- 172.21.15.32:0/889191487 >> v1:172.21.15.32:6789/0 conn(0x7f89600d1120 legacy=0x7f89600b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f895effd640 1 -- 172.21.15.32:0/889191487 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436134275 0 0) 0x7f89580014a0 con 0x7f89600d1120 2023-12-20T04:41:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f895f7fe640 1 -- 172.21.15.32:0/889191487 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89600d1980 con 0x7f89600d2740 2023-12-20T04:41:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.485+0000 7f895ffff640 1 --2- 172.21.15.32:0/889191487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f896015cdb0 0x7f89600d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:41:03.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f895f7fe640 1 --2- 172.21.15.32:0/889191487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89600d2740 0x7f89600cf570 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f89540097c0 tx=0x7f895401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d9b63ae12ed7bf7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:03.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f895effd640 1 -- 172.21.15.32:0/889191487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89540a8970 con 0x7f89600d2740 2023-12-20T04:41:03.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f895effd640 1 -- 172.21.15.32:0/889191487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89540a8ad0 con 0x7f89600d2740 2023-12-20T04:41:03.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f895effd640 1 -- 172.21.15.32:0/889191487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89540a8e60 con 0x7f89600d2740 2023-12-20T04:41:03.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f89663db640 1 -- 172.21.15.32:0/889191487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89600d2740 msgr2=0x7f89600cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f89663db640 1 --2- 172.21.15.32:0/889191487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89600d2740 0x7f89600cf570 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7f89540097c0 tx=0x7f895401ec70 comp rx=0 tx=0).stop 2023-12-20T04:41:03.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f89663db640 1 -- 172.21.15.32:0/889191487 shutdown_connections 2023-12-20T04:41:03.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f89663db640 1 --2- 172.21.15.32:0/889191487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89600d2740 0x7f89600cf570 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f89663db640 1 --2- 172.21.15.32:0/889191487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f896015cdb0 0x7f89600d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f89663db640 1 -- 172.21.15.32:0/889191487 >> 172.21.15.32:0/889191487 conn(0x7f8960065ee0 msgr2=0x7f896006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:03.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f89663db640 1 -- 172.21.15.32:0/889191487 shutdown_connections 2023-12-20T04:41:03.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.489+0000 7f89663db640 1 -- 172.21.15.32:0/889191487 wait complete. 2023-12-20T04:41:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f89663db640 1 Processor -- start 2023-12-20T04:41:03.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f89663db640 1 -- start start 2023-12-20T04:41:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f89663db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89600d1120 0x7f89601f6670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f89663db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89600d2740 0x7f89601f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f89663db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f896015cdb0 0x7f89600d2b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f89663db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89600cbca0 con 0x7f89600d1120 2023-12-20T04:41:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f89663db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89601f6bb0 con 0x7f896015cdb0 2023-12-20T04:41:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f89663db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89600cb020 con 0x7f89600d2740 2023-12-20T04:41:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89600d1120 0x7f89601f6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89600d1120 0x7f89601f6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51426/0 (socket says 172.21.15.32:51426) 2023-12-20T04:41:03.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895ffff640 1 -- 172.21.15.32:0/3112221764 learned_addr learned my addr 172.21.15.32:0/3112221764 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895f7fe640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89600d2740 0x7f89601f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f8964978640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f896015cdb0 0x7f89600d2b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895ffff640 1 -- 172.21.15.32:0/3112221764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89600d2740 msgr2=0x7f89601f5420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895ffff640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89600d2740 0x7f89601f5420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895ffff640 1 -- 172.21.15.32:0/3112221764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f896015cdb0 msgr2=0x7f89600d2b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895ffff640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f896015cdb0 0x7f89600d2b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895ffff640 1 -- 172.21.15.32:0/3112221764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8954009420 con 0x7f89600d1120 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895f7fe640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89600d2740 0x7f89601f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f8964978640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f896015cdb0 0x7f89600d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895ffff640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89600d1120 0x7f89601f6670 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f895000c7c0 tx=0x7f895000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895cff9640 1 -- 172.21.15.32:0/3112221764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8950007a30 con 0x7f89600d1120 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895cff9640 1 -- 172.21.15.32:0/3112221764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8950007b90 con 0x7f89600d1120 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f895cff9640 1 -- 172.21.15.32:0/3112221764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f895000ce70 con 0x7f89600d1120 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f89663db640 1 -- 172.21.15.32:0/3112221764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89600cb300 con 0x7f89600d1120 2023-12-20T04:41:03.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.493+0000 7f89663db640 1 -- 172.21.15.32:0/3112221764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89600cb8c0 con 0x7f89600d1120 2023-12-20T04:41:03.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.497+0000 7f89663db640 1 -- 172.21.15.32:0/3112221764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f896004e000 con 0x7f89600d1120 2023-12-20T04:41:03.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.497+0000 7f895cff9640 1 -- 172.21.15.32:0/3112221764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f89500113f0 con 0x7f89600d1120 2023-12-20T04:41:03.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.497+0000 7f895cff9640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f894c065bb0 0x7f894c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.497+0000 7f895cff9640 1 -- 172.21.15.32:0/3112221764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8950083830 con 0x7f89600d1120 2023-12-20T04:41:03.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.497+0000 7f895f7fe640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f894c065bb0 0x7f894c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.497+0000 7f895f7fe640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f894c065bb0 0x7f894c068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f89540b2000 tx=0x7f89540aa070 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:03.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.501+0000 7f895cff9640 1 -- 172.21.15.32:0/3112221764 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8950051060 con 0x7f89600d1120 2023-12-20T04:41:03.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.657+0000 7f89663db640 1 -- 172.21.15.32:0/3112221764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f896006bc20 con 0x7f89600d1120 2023-12-20T04:41:03.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f895cff9640 1 -- 172.21.15.32:0/3112221764 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v61) v1 ==== 74+0+15997 (secure 0 0 0) 0x7f895001e090 con 0x7f89600d1120 2023-12-20T04:41:03.660 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:41:03.660 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":61,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:41:00.922929+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:40:57.540206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-20T04:40:58.913164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:41:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 -- 172.21.15.32:0/3112221764 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f894c065bb0 msgr2=0x7f894c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f894c065bb0 0x7f894c068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f89540b2000 tx=0x7f89540aa070 comp rx=0 tx=0).stop 2023-12-20T04:41:03.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 -- 172.21.15.32:0/3112221764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89600d1120 msgr2=0x7f89601f6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89600d1120 0x7f89601f6670 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7f895000c7c0 tx=0x7f895000cc80 comp rx=0 tx=0).stop 2023-12-20T04:41:03.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 -- 172.21.15.32:0/3112221764 shutdown_connections 2023-12-20T04:41:03.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f894c065bb0 0x7f894c068070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f896015cdb0 0x7f89600d2b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89600d2740 0x7f89601f5420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 --2- 172.21.15.32:0/3112221764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89600d1120 0x7f89601f6670 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 -- 172.21.15.32:0/3112221764 >> 172.21.15.32:0/3112221764 conn(0x7f8960065ee0 msgr2=0x7f8960155060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:03.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 -- 172.21.15.32:0/3112221764 shutdown_connections 2023-12-20T04:41:03.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.661+0000 7f89663db640 1 -- 172.21.15.32:0/3112221764 wait complete. 2023-12-20T04:41:03.682 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T04:41:03.682 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:41:03.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.825+0000 7fd484a8c640 1 Processor -- start 2023-12-20T04:41:03.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.825+0000 7fd484a8c640 1 -- start start 2023-12-20T04:41:03.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.829+0000 7fd484a8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4801599e0 0x7fd4800aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.829+0000 7fd484a8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4800c98c0 con 0x7fd4800b0d00 2023-12-20T04:41:03.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.829+0000 7fd484a8c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd4800c9ba0 con 0x7fd4800c90e0 2023-12-20T04:41:03.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.829+0000 7fd484a8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4800c9e80 con 0x7fd4801599e0 2023-12-20T04:41:03.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.829+0000 7fd47e59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4801599e0 0x7fd4800aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.829+0000 7fd47ed9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4800b0d00 0x7fd4800701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35840/0 (socket says 172.21.15.32:35840) 2023-12-20T04:41:03.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.829+0000 7fd47ed9d640 1 -- 172.21.15.32:0/3851137664 learned_addr learned my addr 172.21.15.32:0/3851137664 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:41:03.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.829+0000 7fd47e59c640 1 -- 172.21.15.32:0/3851137664 >> v1:172.21.15.39:6789/0 conn(0x7fd4800c90e0 legacy=0x7fd4800b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.829+0000 7fd47e59c640 1 -- 172.21.15.32:0/3851137664 >> v1:172.21.15.32:6789/0 conn(0x7fd4800b0d00 legacy=0x7fd4800701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.829+0000 7fd47e59c640 1 -- 172.21.15.32:0/3851137664 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4800ca160 con 0x7fd4801599e0 2023-12-20T04:41:03.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.829+0000 7fd47dd9b640 1 -- 172.21.15.32:0/3851137664 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704999060 0 0) 0x7fd4800c98c0 con 0x7fd4800b0d00 2023-12-20T04:41:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd47e59c640 1 --2- 172.21.15.32:0/3851137664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801599e0 0x7fd4800aece0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd4700097a0 tx=0x7fd47001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283a84af12d09426 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd47dd9b640 1 -- 172.21.15.32:0/3851137664 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4700ab650 con 0x7fd4801599e0 2023-12-20T04:41:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd47dd9b640 1 -- 172.21.15.32:0/3851137664 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4700ab7b0 con 0x7fd4801599e0 2023-12-20T04:41:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd47dd9b640 1 -- 172.21.15.32:0/3851137664 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4700abb40 con 0x7fd4801599e0 2023-12-20T04:41:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd484a8c640 1 -- 172.21.15.32:0/3851137664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801599e0 msgr2=0x7fd4800aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd484a8c640 1 --2- 172.21.15.32:0/3851137664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801599e0 0x7fd4800aece0 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd4700097a0 tx=0x7fd47001c430 comp rx=0 tx=0).stop 2023-12-20T04:41:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd484a8c640 1 -- 172.21.15.32:0/3851137664 shutdown_connections 2023-12-20T04:41:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd484a8c640 1 --2- 172.21.15.32:0/3851137664 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801599e0 0x7fd4800aece0 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd484a8c640 1 -- 172.21.15.32:0/3851137664 >> 172.21.15.32:0/3851137664 conn(0x7fd48006a320 msgr2=0x7fd4800ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd484a8c640 1 -- 172.21.15.32:0/3851137664 shutdown_connections 2023-12-20T04:41:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd484a8c640 1 -- 172.21.15.32:0/3851137664 wait complete. 2023-12-20T04:41:03.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd484a8c640 1 Processor -- start 2023-12-20T04:41:03.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.833+0000 7fd484a8c640 1 -- start start 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd484a8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800b0d00 0x7fd480158210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd484a8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4800c90e0 0x7fd48014f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd484a8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801599e0 0x7fd480156790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd484a8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd480158750 con 0x7fd4801599e0 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd484a8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd480157170 con 0x7fd4800b0d00 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd484a8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd480157450 con 0x7fd4800c90e0 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801599e0 0x7fd480156790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd47ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801599e0 0x7fd480156790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51454/0 (socket says 172.21.15.32:51454) 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd47ed9d640 1 -- 172.21.15.32:0/2340902743 learned_addr learned my addr 172.21.15.32:0/2340902743 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd47e59c640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800b0d00 0x7fd480158210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd477fff640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4800c90e0 0x7fd48014f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd47ed9d640 1 -- 172.21.15.32:0/2340902743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4800c90e0 msgr2=0x7fd48014f140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd47ed9d640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4800c90e0 0x7fd48014f140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd47ed9d640 1 -- 172.21.15.32:0/2340902743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800b0d00 msgr2=0x7fd480158210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd47ed9d640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800b0d00 0x7fd480158210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd47ed9d640 1 -- 172.21.15.32:0/2340902743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4700093b0 con 0x7fd4801599e0 2023-12-20T04:41:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd47e59c640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800b0d00 0x7fd480158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:41:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd47ed9d640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801599e0 0x7fd480156790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c700 tx=0x7fd47800cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd476ffd640 1 -- 172.21.15.32:0/2340902743 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47800ce10 con 0x7fd4801599e0 2023-12-20T04:41:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd476ffd640 1 -- 172.21.15.32:0/2340902743 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd47800a640 con 0x7fd4801599e0 2023-12-20T04:41:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd484a8c640 1 -- 172.21.15.32:0/2340902743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd480157730 con 0x7fd4801599e0 2023-12-20T04:41:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd476ffd640 1 -- 172.21.15.32:0/2340902743 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd47800aa70 con 0x7fd4801599e0 2023-12-20T04:41:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd484a8c640 1 -- 172.21.15.32:0/2340902743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd480157cf0 con 0x7fd4801599e0 2023-12-20T04:41:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd484a8c640 1 -- 172.21.15.32:0/2340902743 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd48004e000 con 0x7fd4801599e0 2023-12-20T04:41:03.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.837+0000 7fd476ffd640 1 -- 172.21.15.32:0/2340902743 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd47800abd0 con 0x7fd4801599e0 2023-12-20T04:41:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.841+0000 7fd476ffd640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd460065b80 0x7fd460068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.841+0000 7fd476ffd640 1 -- 172.21.15.32:0/2340902743 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd478082720 con 0x7fd4801599e0 2023-12-20T04:41:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.841+0000 7fd47e59c640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd460065b80 0x7fd460068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.841+0000 7fd47e59c640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd460065b80 0x7fd460068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd4700af000 tx=0x7fd4700a7000 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:03.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:03.845+0000 7fd476ffd640 1 -- 172.21.15.32:0/2340902743 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd478050070 con 0x7fd4801599e0 2023-12-20T04:41:04.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.001+0000 7fd484a8c640 1 -- 172.21.15.32:0/2340902743 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fd480159dc0 con 0x7fd460065b80 2023-12-20T04:41:04.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.009+0000 7fd476ffd640 1 -- 172.21.15.32:0/2340902743 <== mgr.5664 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288499 (secure 0 0 0) 0x7fd4700ab920 con 0x7fd460065b80 2023-12-20T04:41:04.013 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:41:04.014 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:41:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 -- 172.21.15.32:0/2340902743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd460065b80 msgr2=0x7fd460068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd460065b80 0x7fd460068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fd4700af000 tx=0x7fd4700a7000 comp rx=0 tx=0).stop 2023-12-20T04:41:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 -- 172.21.15.32:0/2340902743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801599e0 msgr2=0x7fd480156790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801599e0 0x7fd480156790 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd47800c700 tx=0x7fd47800cbc0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 -- 172.21.15.32:0/2340902743 shutdown_connections 2023-12-20T04:41:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd460065b80 0x7fd460068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4801599e0 0x7fd480156790 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4800c90e0 0x7fd48014f140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 --2- 172.21.15.32:0/2340902743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4800b0d00 0x7fd480158210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 -- 172.21.15.32:0/2340902743 >> 172.21.15.32:0/2340902743 conn(0x7fd48006a320 msgr2=0x7fd48015be30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 -- 172.21.15.32:0/2340902743 shutdown_connections 2023-12-20T04:41:04.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.013+0000 7fd484a8c640 1 -- 172.21.15.32:0/2340902743 wait complete. 2023-12-20T04:41:04.033 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-20T04:41:03.265840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":297812,"kb_used_data":6804,"kb_used_omap":0,"kb_used_meta":290944,"kb_avail":754676908,"statfs":{"total":773094113280,"available":772789153792,"internally_reserved":0,"allocated":6967296,"data_stored":3811328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":297926656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.756120"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.961113+0000","last_change":"2023-12-20T04:40:59.950537+0000","last_active":"2023-12-20T04:40:59.961113+0000","last_peered":"2023-12-20T04:40:59.961113+0000","last_clean":"2023-12-20T04:40:59.961113+0000","last_became_active":"2023-12-20T04:40:59.950185+0000","last_became_peered":"2023-12-20T04:40:59.950185+0000","last_unstale":"2023-12-20T04:40:59.961113+0000","last_undegraded":"2023-12-20T04:40:59.961113+0000","last_fullsized":"2023-12-20T04:40:59.961113+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:52:07.362486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.930127+0000","last_change":"2023-12-20T04:40:59.953877+0000","last_active":"2023-12-20T04:41:00.930127+0000","last_peered":"2023-12-20T04:41:00.930127+0000","last_clean":"2023-12-20T04:41:00.930127+0000","last_became_active":"2023-12-20T04:40:59.953540+0000","last_became_peered":"2023-12-20T04:40:59.953540+0000","last_unstale":"2023-12-20T04:41:00.930127+0000","last_undegraded":"2023-12-20T04:41:00.930127+0000","last_fullsized":"2023-12-20T04:41:00.930127+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:07.685523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928093+0000","last_change":"2023-12-20T04:40:59.952341+0000","last_active":"2023-12-20T04:41:00.928093+0000","last_peered":"2023-12-20T04:41:00.928093+0000","last_clean":"2023-12-20T04:41:00.928093+0000","last_became_active":"2023-12-20T04:40:59.952037+0000","last_became_peered":"2023-12-20T04:40:59.952037+0000","last_unstale":"2023-12-20T04:41:00.928093+0000","last_undegraded":"2023-12-20T04:41:00.928093+0000","last_fullsized":"2023-12-20T04:41:00.928093+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:24.973589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.965928+0000","last_change":"2023-12-20T04:40:59.964203+0000","last_active":"2023-12-20T04:40:59.965928+0000","last_peered":"2023-12-20T04:40:59.965928+0000","last_clean":"2023-12-20T04:40:59.965928+0000","last_became_active":"2023-12-20T04:40:59.963916+0000","last_became_peered":"2023-12-20T04:40:59.963916+0000","last_unstale":"2023-12-20T04:40:59.965928+0000","last_undegraded":"2023-12-20T04:40:59.965928+0000","last_fullsized":"2023-12-20T04:40:59.965928+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:33:20.043278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929917+0000","last_change":"2023-12-20T04:40:59.954330+0000","last_active":"2023-12-20T04:41:00.929917+0000","last_peered":"2023-12-20T04:41:00.929917+0000","last_clean":"2023-12-20T04:41:00.929917+0000","last_became_active":"2023-12-20T04:40:59.954017+0000","last_became_peered":"2023-12-20T04:40:59.954017+0000","last_unstale":"2023-12-20T04:41:00.929917+0000","last_undegraded":"2023-12-20T04:41:00.929917+0000","last_fullsized":"2023-12-20T04:41:00.929917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:24.973589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259696+0000","last_change":"2023-12-20T04:40:59.948940+0000","last_active":"2023-12-20T04:41:01.259696+0000","last_peered":"2023-12-20T04:41:01.259696+0000","last_clean":"2023-12-20T04:41:01.259696+0000","last_became_active":"2023-12-20T04:40:59.948668+0000","last_became_peered":"2023-12-20T04:40:59.948668+0000","last_unstale":"2023-12-20T04:41:01.259696+0000","last_undegraded":"2023-12-20T04:41:01.259696+0000","last_fullsized":"2023-12-20T04:41:01.259696+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:14:57.533936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.956104+0000","last_change":"2023-12-20T04:40:59.947725+0000","last_active":"2023-12-20T04:40:59.956104+0000","last_peered":"2023-12-20T04:40:59.956104+0000","last_clean":"2023-12-20T04:40:59.956104+0000","last_became_active":"2023-12-20T04:40:59.947430+0000","last_became_peered":"2023-12-20T04:40:59.947430+0000","last_unstale":"2023-12-20T04:40:59.956104+0000","last_undegraded":"2023-12-20T04:40:59.956104+0000","last_fullsized":"2023-12-20T04:40:59.956104+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:27.433632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966348+0000","last_change":"2023-12-20T04:40:59.965242+0000","last_active":"2023-12-20T04:40:59.966348+0000","last_peered":"2023-12-20T04:40:59.966348+0000","last_clean":"2023-12-20T04:40:59.966348+0000","last_became_active":"2023-12-20T04:40:59.964302+0000","last_became_peered":"2023-12-20T04:40:59.964302+0000","last_unstale":"2023-12-20T04:40:59.966348+0000","last_undegraded":"2023-12-20T04:40:59.966348+0000","last_fullsized":"2023-12-20T04:40:59.966348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:36:49.912753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957043+0000","last_change":"2023-12-20T04:40:59.947035+0000","last_active":"2023-12-20T04:40:59.957043+0000","last_peered":"2023-12-20T04:40:59.957043+0000","last_clean":"2023-12-20T04:40:59.957043+0000","last_became_active":"2023-12-20T04:40:59.946725+0000","last_became_peered":"2023-12-20T04:40:59.946725+0000","last_unstale":"2023-12-20T04:40:59.957043+0000","last_undegraded":"2023-12-20T04:40:59.957043+0000","last_fullsized":"2023-12-20T04:40:59.957043+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:32:00.691348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966256+0000","last_change":"2023-12-20T04:40:59.964374+0000","last_active":"2023-12-20T04:40:59.966256+0000","last_peered":"2023-12-20T04:40:59.966256+0000","last_clean":"2023-12-20T04:40:59.966256+0000","last_became_active":"2023-12-20T04:40:59.964095+0000","last_became_peered":"2023-12-20T04:40:59.964095+0000","last_unstale":"2023-12-20T04:40:59.966256+0000","last_undegraded":"2023-12-20T04:40:59.966256+0000","last_fullsized":"2023-12-20T04:40:59.966256+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:00:37.983783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.953303+0000","last_change":"2023-12-20T04:40:59.942545+0000","last_active":"2023-12-20T04:40:59.953303+0000","last_peered":"2023-12-20T04:40:59.953303+0000","last_clean":"2023-12-20T04:40:59.953303+0000","last_became_active":"2023-12-20T04:40:59.942205+0000","last_became_peered":"2023-12-20T04:40:59.942205+0000","last_unstale":"2023-12-20T04:40:59.953303+0000","last_undegraded":"2023-12-20T04:40:59.953303+0000","last_fullsized":"2023-12-20T04:40:59.953303+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:12:25.173510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929933+0000","last_change":"2023-12-20T04:40:59.949515+0000","last_active":"2023-12-20T04:41:00.929933+0000","last_peered":"2023-12-20T04:41:00.929933+0000","last_clean":"2023-12-20T04:41:00.929933+0000","last_became_active":"2023-12-20T04:40:59.948482+0000","last_became_peered":"2023-12-20T04:40:59.948482+0000","last_unstale":"2023-12-20T04:41:00.929933+0000","last_undegraded":"2023-12-20T04:41:00.929933+0000","last_fullsized":"2023-12-20T04:41:00.929933+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:11:05.165428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927806+0000","last_change":"2023-12-20T04:40:59.955753+0000","last_active":"2023-12-20T04:41:00.927806+0000","last_peered":"2023-12-20T04:41:00.927806+0000","last_clean":"2023-12-20T04:41:00.927806+0000","last_became_active":"2023-12-20T04:40:59.955493+0000","last_became_peered":"2023-12-20T04:40:59.955493+0000","last_unstale":"2023-12-20T04:41:00.927806+0000","last_undegraded":"2023-12-20T04:41:00.927806+0000","last_fullsized":"2023-12-20T04:41:00.927806+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:00:46.881125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.953199+0000","last_change":"2023-12-20T04:40:59.942919+0000","last_active":"2023-12-20T04:40:59.953199+0000","last_peered":"2023-12-20T04:40:59.953199+0000","last_clean":"2023-12-20T04:40:59.953199+0000","last_became_active":"2023-12-20T04:40:59.942609+0000","last_became_peered":"2023-12-20T04:40:59.942609+0000","last_unstale":"2023-12-20T04:40:59.953199+0000","last_undegraded":"2023-12-20T04:40:59.953199+0000","last_fullsized":"2023-12-20T04:40:59.953199+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:05.391543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259640+0000","last_change":"2023-12-20T04:40:59.954208+0000","last_active":"2023-12-20T04:41:01.259640+0000","last_peered":"2023-12-20T04:41:01.259640+0000","last_clean":"2023-12-20T04:41:01.259640+0000","last_became_active":"2023-12-20T04:40:59.953953+0000","last_became_peered":"2023-12-20T04:40:59.953953+0000","last_unstale":"2023-12-20T04:41:01.259640+0000","last_undegraded":"2023-12-20T04:41:01.259640+0000","last_fullsized":"2023-12-20T04:41:01.259640+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:32:17.808232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.964769+0000","last_change":"2023-12-20T04:40:59.959481+0000","last_active":"2023-12-20T04:40:59.964769+0000","last_peered":"2023-12-20T04:40:59.964769+0000","last_clean":"2023-12-20T04:40:59.964769+0000","last_became_active":"2023-12-20T04:40:59.958981+0000","last_became_peered":"2023-12-20T04:40:59.958981+0000","last_unstale":"2023-12-20T04:40:59.964769+0000","last_undegraded":"2023-12-20T04:40:59.964769+0000","last_fullsized":"2023-12-20T04:40:59.964769+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:52:01.969606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928230+0000","last_change":"2023-12-20T04:40:59.953605+0000","last_active":"2023-12-20T04:41:00.928230+0000","last_peered":"2023-12-20T04:41:00.928230+0000","last_clean":"2023-12-20T04:41:00.928230+0000","last_became_active":"2023-12-20T04:40:59.953199+0000","last_became_peered":"2023-12-20T04:40:59.953199+0000","last_unstale":"2023-12-20T04:41:00.928230+0000","last_undegraded":"2023-12-20T04:41:00.928230+0000","last_fullsized":"2023-12-20T04:41:00.928230+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:27.433632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928193+0000","last_change":"2023-12-20T04:40:59.956790+0000","last_active":"2023-12-20T04:41:00.928193+0000","last_peered":"2023-12-20T04:41:00.928193+0000","last_clean":"2023-12-20T04:41:00.928193+0000","last_became_active":"2023-12-20T04:40:59.956516+0000","last_became_peered":"2023-12-20T04:40:59.956516+0000","last_unstale":"2023-12-20T04:41:00.928193+0000","last_undegraded":"2023-12-20T04:41:00.928193+0000","last_fullsized":"2023-12-20T04:41:00.928193+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:43:16.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966456+0000","last_change":"2023-12-20T04:40:59.962564+0000","last_active":"2023-12-20T04:40:59.966456+0000","last_peered":"2023-12-20T04:40:59.966456+0000","last_clean":"2023-12-20T04:40:59.966456+0000","last_became_active":"2023-12-20T04:40:59.962252+0000","last_became_peered":"2023-12-20T04:40:59.962252+0000","last_unstale":"2023-12-20T04:40:59.966456+0000","last_undegraded":"2023-12-20T04:40:59.966456+0000","last_fullsized":"2023-12-20T04:40:59.966456+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:14:36.769806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927980+0000","last_change":"2023-12-20T04:40:59.951480+0000","last_active":"2023-12-20T04:41:00.927980+0000","last_peered":"2023-12-20T04:41:00.927980+0000","last_clean":"2023-12-20T04:41:00.927980+0000","last_became_active":"2023-12-20T04:40:59.951214+0000","last_became_peered":"2023-12-20T04:40:59.951214+0000","last_unstale":"2023-12-20T04:41:00.927980+0000","last_undegraded":"2023-12-20T04:41:00.927980+0000","last_fullsized":"2023-12-20T04:41:00.927980+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:33:20.033497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259749+0000","last_change":"2023-12-20T04:40:59.968026+0000","last_active":"2023-12-20T04:41:01.259749+0000","last_peered":"2023-12-20T04:41:01.259749+0000","last_clean":"2023-12-20T04:41:01.259749+0000","last_became_active":"2023-12-20T04:40:59.967679+0000","last_became_peered":"2023-12-20T04:40:59.967679+0000","last_unstale":"2023-12-20T04:41:01.259749+0000","last_undegraded":"2023-12-20T04:41:01.259749+0000","last_fullsized":"2023-12-20T04:41:01.259749+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:01.174229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.968504+0000","last_change":"2023-12-20T04:40:59.967936+0000","last_active":"2023-12-20T04:40:59.968504+0000","last_peered":"2023-12-20T04:40:59.968504+0000","last_clean":"2023-12-20T04:40:59.968504+0000","last_became_active":"2023-12-20T04:40:59.967638+0000","last_became_peered":"2023-12-20T04:40:59.967638+0000","last_unstale":"2023-12-20T04:40:59.968504+0000","last_undegraded":"2023-12-20T04:40:59.968504+0000","last_fullsized":"2023-12-20T04:40:59.968504+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:01.467255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966404+0000","last_change":"2023-12-20T04:40:59.962694+0000","last_active":"2023-12-20T04:40:59.966404+0000","last_peered":"2023-12-20T04:40:59.966404+0000","last_clean":"2023-12-20T04:40:59.966404+0000","last_became_active":"2023-12-20T04:40:59.962343+0000","last_became_peered":"2023-12-20T04:40:59.962343+0000","last_unstale":"2023-12-20T04:40:59.966404+0000","last_undegraded":"2023-12-20T04:40:59.966404+0000","last_fullsized":"2023-12-20T04:40:59.966404+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:00:46.881125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.964792+0000","last_change":"2023-12-20T04:40:59.959782+0000","last_active":"2023-12-20T04:40:59.964792+0000","last_peered":"2023-12-20T04:40:59.964792+0000","last_clean":"2023-12-20T04:40:59.964792+0000","last_became_active":"2023-12-20T04:40:59.959426+0000","last_became_peered":"2023-12-20T04:40:59.959426+0000","last_unstale":"2023-12-20T04:40:59.964792+0000","last_undegraded":"2023-12-20T04:40:59.964792+0000","last_fullsized":"2023-12-20T04:40:59.964792+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:05:05.313155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.964849+0000","last_change":"2023-12-20T04:40:59.960365+0000","last_active":"2023-12-20T04:40:59.964849+0000","last_peered":"2023-12-20T04:40:59.964849+0000","last_clean":"2023-12-20T04:40:59.964849+0000","last_became_active":"2023-12-20T04:40:59.959117+0000","last_became_peered":"2023-12-20T04:40:59.959117+0000","last_unstale":"2023-12-20T04:40:59.964849+0000","last_undegraded":"2023-12-20T04:40:59.964849+0000","last_fullsized":"2023-12-20T04:40:59.964849+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:38:04.392962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927866+0000","last_change":"2023-12-20T04:40:59.955893+0000","last_active":"2023-12-20T04:41:00.927866+0000","last_peered":"2023-12-20T04:41:00.927866+0000","last_clean":"2023-12-20T04:41:00.927866+0000","last_became_active":"2023-12-20T04:40:59.955591+0000","last_became_peered":"2023-12-20T04:40:59.955591+0000","last_unstale":"2023-12-20T04:41:00.927866+0000","last_undegraded":"2023-12-20T04:41:00.927866+0000","last_fullsized":"2023-12-20T04:41:00.927866+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:09:00.207233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928098+0000","last_change":"2023-12-20T04:40:59.955297+0000","last_active":"2023-12-20T04:41:00.928098+0000","last_peered":"2023-12-20T04:41:00.928098+0000","last_clean":"2023-12-20T04:41:00.928098+0000","last_became_active":"2023-12-20T04:40:59.954028+0000","last_became_peered":"2023-12-20T04:40:59.954028+0000","last_unstale":"2023-12-20T04:41:00.928098+0000","last_undegraded":"2023-12-20T04:41:00.928098+0000","last_fullsized":"2023-12-20T04:41:00.928098+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:51.240859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929957+0000","last_change":"2023-12-20T04:40:59.943197+0000","last_active":"2023-12-20T04:41:00.929957+0000","last_peered":"2023-12-20T04:41:00.929957+0000","last_clean":"2023-12-20T04:41:00.929957+0000","last_became_active":"2023-12-20T04:40:59.942874+0000","last_became_peered":"2023-12-20T04:40:59.942874+0000","last_unstale":"2023-12-20T04:41:00.929957+0000","last_undegraded":"2023-12-20T04:41:00.929957+0000","last_fullsized":"2023-12-20T04:41:00.929957+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:16:42.925904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.967289+0000","last_change":"2023-12-20T04:40:59.964866+0000","last_active":"2023-12-20T04:40:59.967289+0000","last_peered":"2023-12-20T04:40:59.967289+0000","last_clean":"2023-12-20T04:40:59.967289+0000","last_became_active":"2023-12-20T04:40:59.964568+0000","last_became_peered":"2023-12-20T04:40:59.964568+0000","last_unstale":"2023-12-20T04:40:59.967289+0000","last_undegraded":"2023-12-20T04:40:59.967289+0000","last_fullsized":"2023-12-20T04:40:59.967289+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:45.584995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927946+0000","last_change":"2023-12-20T04:40:59.954287+0000","last_active":"2023-12-20T04:41:00.927946+0000","last_peered":"2023-12-20T04:41:00.927946+0000","last_clean":"2023-12-20T04:41:00.927946+0000","last_became_active":"2023-12-20T04:40:59.953065+0000","last_became_peered":"2023-12-20T04:40:59.953065+0000","last_unstale":"2023-12-20T04:41:00.927946+0000","last_undegraded":"2023-12-20T04:41:00.927946+0000","last_fullsized":"2023-12-20T04:41:00.927946+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:52:07.362486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259709+0000","last_change":"2023-12-20T04:40:59.952866+0000","last_active":"2023-12-20T04:41:01.259709+0000","last_peered":"2023-12-20T04:41:01.259709+0000","last_clean":"2023-12-20T04:41:01.259709+0000","last_became_active":"2023-12-20T04:40:59.952032+0000","last_became_peered":"2023-12-20T04:40:59.952032+0000","last_unstale":"2023-12-20T04:41:01.259709+0000","last_undegraded":"2023-12-20T04:41:01.259709+0000","last_fullsized":"2023-12-20T04:41:01.259709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:30:10.242145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927665+0000","last_change":"2023-12-20T04:40:59.940706+0000","last_active":"2023-12-20T04:41:00.927665+0000","last_peered":"2023-12-20T04:41:00.927665+0000","last_clean":"2023-12-20T04:41:00.927665+0000","last_became_active":"2023-12-20T04:40:59.940362+0000","last_became_peered":"2023-12-20T04:40:59.940362+0000","last_unstale":"2023-12-20T04:41:00.927665+0000","last_undegraded":"2023-12-20T04:41:00.927665+0000","last_fullsized":"2023-12-20T04:41:00.927665+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:05:05.313155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.955693+0000","last_change":"2023-12-20T04:40:59.947838+0000","last_active":"2023-12-20T04:40:59.955693+0000","last_peered":"2023-12-20T04:40:59.955693+0000","last_clean":"2023-12-20T04:40:59.955693+0000","last_became_active":"2023-12-20T04:40:59.947367+0000","last_became_peered":"2023-12-20T04:40:59.947367+0000","last_unstale":"2023-12-20T04:40:59.955693+0000","last_undegraded":"2023-12-20T04:40:59.955693+0000","last_fullsized":"2023-12-20T04:40:59.955693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:25:19.139121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"60'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.260064+0000","last_change":"2023-12-20T04:40:59.546885+0000","last_active":"2023-12-20T04:41:01.260064+0000","last_peered":"2023-12-20T04:41:01.260064+0000","last_clean":"2023-12-20T04:41:01.260064+0000","last_became_active":"2023-12-20T04:40:59.546555+0000","last_became_peered":"2023-12-20T04:40:59.546555+0000","last_unstale":"2023-12-20T04:41:01.260064+0000","last_undegraded":"2023-12-20T04:41:01.260064+0000","last_fullsized":"2023-12-20T04:41:01.260064+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:01.459895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966303+0000","last_change":"2023-12-20T04:40:59.962424+0000","last_active":"2023-12-20T04:40:59.966303+0000","last_peered":"2023-12-20T04:40:59.966303+0000","last_clean":"2023-12-20T04:40:59.966303+0000","last_became_active":"2023-12-20T04:40:59.962106+0000","last_became_peered":"2023-12-20T04:40:59.962106+0000","last_unstale":"2023-12-20T04:40:59.966303+0000","last_undegraded":"2023-12-20T04:40:59.966303+0000","last_fullsized":"2023-12-20T04:40:59.966303+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:33:20.033497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928280+0000","last_change":"2023-12-20T04:40:59.545033+0000","last_active":"2023-12-20T04:41:00.928280+0000","last_peered":"2023-12-20T04:41:00.928280+0000","last_clean":"2023-12-20T04:41:00.928280+0000","last_became_active":"2023-12-20T04:40:59.544671+0000","last_became_peered":"2023-12-20T04:40:59.544671+0000","last_unstale":"2023-12-20T04:41:00.928280+0000","last_undegraded":"2023-12-20T04:41:00.928280+0000","last_fullsized":"2023-12-20T04:41:00.928280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:25:18.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.930032+0000","last_change":"2023-12-20T04:40:59.950545+0000","last_active":"2023-12-20T04:41:00.930032+0000","last_peered":"2023-12-20T04:41:00.930032+0000","last_clean":"2023-12-20T04:41:00.930032+0000","last_became_active":"2023-12-20T04:40:59.949051+0000","last_became_peered":"2023-12-20T04:40:59.949051+0000","last_unstale":"2023-12-20T04:41:00.930032+0000","last_undegraded":"2023-12-20T04:41:00.930032+0000","last_fullsized":"2023-12-20T04:41:00.930032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:03:02.302456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.948084+0000","last_change":"2023-12-20T04:40:59.547531+0000","last_active":"2023-12-20T04:40:59.948084+0000","last_peered":"2023-12-20T04:40:59.948084+0000","last_clean":"2023-12-20T04:40:59.948084+0000","last_became_active":"2023-12-20T04:40:59.546128+0000","last_became_peered":"2023-12-20T04:40:59.546128+0000","last_unstale":"2023-12-20T04:40:59.948084+0000","last_undegraded":"2023-12-20T04:40:59.948084+0000","last_fullsized":"2023-12-20T04:40:59.948084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:28.929814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.967608+0000","last_change":"2023-12-20T04:40:59.966631+0000","last_active":"2023-12-20T04:40:59.967608+0000","last_peered":"2023-12-20T04:40:59.967608+0000","last_clean":"2023-12-20T04:40:59.967608+0000","last_became_active":"2023-12-20T04:40:59.966323+0000","last_became_peered":"2023-12-20T04:40:59.966323+0000","last_unstale":"2023-12-20T04:40:59.967608+0000","last_undegraded":"2023-12-20T04:40:59.967608+0000","last_fullsized":"2023-12-20T04:40:59.967608+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:37:49.708430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.951423+0000","last_change":"2023-12-20T04:40:59.549225+0000","last_active":"2023-12-20T04:40:59.951423+0000","last_peered":"2023-12-20T04:40:59.951423+0000","last_clean":"2023-12-20T04:40:59.951423+0000","last_became_active":"2023-12-20T04:40:59.548921+0000","last_became_peered":"2023-12-20T04:40:59.548921+0000","last_unstale":"2023-12-20T04:40:59.951423+0000","last_undegraded":"2023-12-20T04:40:59.951423+0000","last_fullsized":"2023-12-20T04:40:59.951423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:37:48.689217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928017+0000","last_change":"2023-12-20T04:40:59.965437+0000","last_active":"2023-12-20T04:41:00.928017+0000","last_peered":"2023-12-20T04:41:00.928017+0000","last_clean":"2023-12-20T04:41:00.928017+0000","last_became_active":"2023-12-20T04:40:59.962778+0000","last_became_peered":"2023-12-20T04:40:59.962778+0000","last_unstale":"2023-12-20T04:41:00.928017+0000","last_undegraded":"2023-12-20T04:41:00.928017+0000","last_fullsized":"2023-12-20T04:41:00.928017+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:04.693874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259851+0000","last_change":"2023-12-20T04:40:59.541677+0000","last_active":"2023-12-20T04:41:01.259851+0000","last_peered":"2023-12-20T04:41:01.259851+0000","last_clean":"2023-12-20T04:41:01.259851+0000","last_became_active":"2023-12-20T04:40:59.541386+0000","last_became_peered":"2023-12-20T04:40:59.541386+0000","last_unstale":"2023-12-20T04:41:01.259851+0000","last_undegraded":"2023-12-20T04:41:01.259851+0000","last_fullsized":"2023-12-20T04:41:01.259851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:43:15.224288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957243+0000","last_change":"2023-12-20T04:40:59.949214+0000","last_active":"2023-12-20T04:40:59.957243+0000","last_peered":"2023-12-20T04:40:59.957243+0000","last_clean":"2023-12-20T04:40:59.957243+0000","last_became_active":"2023-12-20T04:40:59.948204+0000","last_became_peered":"2023-12-20T04:40:59.948204+0000","last_unstale":"2023-12-20T04:40:59.957243+0000","last_undegraded":"2023-12-20T04:40:59.957243+0000","last_fullsized":"2023-12-20T04:40:59.957243+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:11:54.734047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928063+0000","last_change":"2023-12-20T04:40:59.542954+0000","last_active":"2023-12-20T04:41:00.928063+0000","last_peered":"2023-12-20T04:41:00.928063+0000","last_clean":"2023-12-20T04:41:00.928063+0000","last_became_active":"2023-12-20T04:40:59.541023+0000","last_became_peered":"2023-12-20T04:40:59.541023+0000","last_unstale":"2023-12-20T04:41:00.928063+0000","last_undegraded":"2023-12-20T04:41:00.928063+0000","last_fullsized":"2023-12-20T04:41:00.928063+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:21:17.066394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.965504+0000","last_change":"2023-12-20T04:40:59.964322+0000","last_active":"2023-12-20T04:40:59.965504+0000","last_peered":"2023-12-20T04:40:59.965504+0000","last_clean":"2023-12-20T04:40:59.965504+0000","last_became_active":"2023-12-20T04:40:59.964033+0000","last_became_peered":"2023-12-20T04:40:59.964033+0000","last_unstale":"2023-12-20T04:40:59.965504+0000","last_undegraded":"2023-12-20T04:40:59.965504+0000","last_fullsized":"2023-12-20T04:40:59.965504+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:11:42.857428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929798+0000","last_change":"2023-12-20T04:40:59.543696+0000","last_active":"2023-12-20T04:41:00.929798+0000","last_peered":"2023-12-20T04:41:00.929798+0000","last_clean":"2023-12-20T04:41:00.929798+0000","last_became_active":"2023-12-20T04:40:59.540121+0000","last_became_peered":"2023-12-20T04:40:59.540121+0000","last_unstale":"2023-12-20T04:41:00.929798+0000","last_undegraded":"2023-12-20T04:41:00.929798+0000","last_fullsized":"2023-12-20T04:41:00.929798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:43:20.804066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966425+0000","last_change":"2023-12-20T04:40:59.962756+0000","last_active":"2023-12-20T04:40:59.966425+0000","last_peered":"2023-12-20T04:40:59.966425+0000","last_clean":"2023-12-20T04:40:59.966425+0000","last_became_active":"2023-12-20T04:40:59.962286+0000","last_became_peered":"2023-12-20T04:40:59.962286+0000","last_unstale":"2023-12-20T04:40:59.966425+0000","last_undegraded":"2023-12-20T04:40:59.966425+0000","last_fullsized":"2023-12-20T04:40:59.966425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:09:00.207233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.936883+0000","last_change":"2023-12-20T04:40:59.534257+0000","last_active":"2023-12-20T04:40:59.936883+0000","last_peered":"2023-12-20T04:40:59.936883+0000","last_clean":"2023-12-20T04:40:59.936883+0000","last_became_active":"2023-12-20T04:40:59.533910+0000","last_became_peered":"2023-12-20T04:40:59.533910+0000","last_unstale":"2023-12-20T04:40:59.936883+0000","last_undegraded":"2023-12-20T04:40:59.936883+0000","last_fullsized":"2023-12-20T04:40:59.936883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:25:18.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928264+0000","last_change":"2023-12-20T04:40:59.958070+0000","last_active":"2023-12-20T04:41:00.928264+0000","last_peered":"2023-12-20T04:41:00.928264+0000","last_clean":"2023-12-20T04:41:00.928264+0000","last_became_active":"2023-12-20T04:40:59.957793+0000","last_became_peered":"2023-12-20T04:40:59.957793+0000","last_unstale":"2023-12-20T04:41:00.928264+0000","last_undegraded":"2023-12-20T04:41:00.928264+0000","last_fullsized":"2023-12-20T04:41:00.928264+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:10.439930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928281+0000","last_change":"2023-12-20T04:40:59.537199+0000","last_active":"2023-12-20T04:41:00.928281+0000","last_peered":"2023-12-20T04:41:00.928281+0000","last_clean":"2023-12-20T04:41:00.928281+0000","last_became_active":"2023-12-20T04:40:59.536871+0000","last_became_peered":"2023-12-20T04:40:59.536871+0000","last_unstale":"2023-12-20T04:41:00.928281+0000","last_undegraded":"2023-12-20T04:41:00.928281+0000","last_fullsized":"2023-12-20T04:41:00.928281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:39:48.794770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957153+0000","last_change":"2023-12-20T04:40:59.948780+0000","last_active":"2023-12-20T04:40:59.957153+0000","last_peered":"2023-12-20T04:40:59.957153+0000","last_clean":"2023-12-20T04:40:59.957153+0000","last_became_active":"2023-12-20T04:40:59.948494+0000","last_became_peered":"2023-12-20T04:40:59.948494+0000","last_unstale":"2023-12-20T04:40:59.957153+0000","last_undegraded":"2023-12-20T04:40:59.957153+0000","last_fullsized":"2023-12-20T04:40:59.957153+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:51.240859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.937510+0000","last_change":"2023-12-20T04:40:59.546225+0000","last_active":"2023-12-20T04:40:59.937510+0000","last_peered":"2023-12-20T04:40:59.937510+0000","last_clean":"2023-12-20T04:40:59.937510+0000","last_became_active":"2023-12-20T04:40:59.545871+0000","last_became_peered":"2023-12-20T04:40:59.545871+0000","last_unstale":"2023-12-20T04:40:59.937510+0000","last_undegraded":"2023-12-20T04:40:59.937510+0000","last_fullsized":"2023-12-20T04:40:59.937510+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:53.616685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.260042+0000","last_change":"2023-12-20T04:40:59.941167+0000","last_active":"2023-12-20T04:41:01.260042+0000","last_peered":"2023-12-20T04:41:01.260042+0000","last_clean":"2023-12-20T04:41:01.260042+0000","last_became_active":"2023-12-20T04:40:59.940889+0000","last_became_peered":"2023-12-20T04:40:59.940889+0000","last_unstale":"2023-12-20T04:41:01.260042+0000","last_undegraded":"2023-12-20T04:41:01.260042+0000","last_fullsized":"2023-12-20T04:41:01.260042+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:39:33.817277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"60'3","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.960337+0000","last_change":"2023-12-20T04:40:59.548265+0000","last_active":"2023-12-20T04:40:59.960337+0000","last_peered":"2023-12-20T04:40:59.960337+0000","last_clean":"2023-12-20T04:40:59.960337+0000","last_became_active":"2023-12-20T04:40:59.547975+0000","last_became_peered":"2023-12-20T04:40:59.547975+0000","last_unstale":"2023-12-20T04:40:59.960337+0000","last_undegraded":"2023-12-20T04:40:59.960337+0000","last_fullsized":"2023-12-20T04:40:59.960337+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:09:33.906588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.956915+0000","last_change":"2023-12-20T04:40:59.947855+0000","last_active":"2023-12-20T04:40:59.956915+0000","last_peered":"2023-12-20T04:40:59.956915+0000","last_clean":"2023-12-20T04:40:59.956915+0000","last_became_active":"2023-12-20T04:40:59.947545+0000","last_became_peered":"2023-12-20T04:40:59.947545+0000","last_unstale":"2023-12-20T04:40:59.956915+0000","last_undegraded":"2023-12-20T04:40:59.956915+0000","last_fullsized":"2023-12-20T04:40:59.956915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:10.439930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"60'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927990+0000","last_change":"2023-12-20T04:40:59.533939+0000","last_active":"2023-12-20T04:41:00.927990+0000","last_peered":"2023-12-20T04:41:00.927990+0000","last_clean":"2023-12-20T04:41:00.927990+0000","last_became_active":"2023-12-20T04:40:59.533674+0000","last_became_peered":"2023-12-20T04:40:59.533674+0000","last_unstale":"2023-12-20T04:41:00.927990+0000","last_undegraded":"2023-12-20T04:41:00.927990+0000","last_fullsized":"2023-12-20T04:41:00.927990+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:55:03.873417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.965057+0000","last_change":"2023-12-20T04:40:59.960645+0000","last_active":"2023-12-20T04:40:59.965057+0000","last_peered":"2023-12-20T04:40:59.965057+0000","last_clean":"2023-12-20T04:40:59.965057+0000","last_became_active":"2023-12-20T04:40:59.958925+0000","last_became_peered":"2023-12-20T04:40:59.958925+0000","last_unstale":"2023-12-20T04:40:59.965057+0000","last_undegraded":"2023-12-20T04:40:59.965057+0000","last_fullsized":"2023-12-20T04:40:59.965057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:54.303149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259847+0000","last_change":"2023-12-20T04:40:59.537954+0000","last_active":"2023-12-20T04:41:01.259847+0000","last_peered":"2023-12-20T04:41:01.259847+0000","last_clean":"2023-12-20T04:41:01.259847+0000","last_became_active":"2023-12-20T04:40:59.537602+0000","last_became_peered":"2023-12-20T04:40:59.537602+0000","last_unstale":"2023-12-20T04:41:01.259847+0000","last_undegraded":"2023-12-20T04:41:01.259847+0000","last_fullsized":"2023-12-20T04:41:01.259847+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:09.420717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957262+0000","last_change":"2023-12-20T04:40:59.949788+0000","last_active":"2023-12-20T04:40:59.957262+0000","last_peered":"2023-12-20T04:40:59.957262+0000","last_clean":"2023-12-20T04:40:59.957262+0000","last_became_active":"2023-12-20T04:40:59.949479+0000","last_became_peered":"2023-12-20T04:40:59.949479+0000","last_unstale":"2023-12-20T04:40:59.957262+0000","last_undegraded":"2023-12-20T04:40:59.957262+0000","last_fullsized":"2023-12-20T04:40:59.957262+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:12:52.553009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.936142+0000","last_change":"2023-12-20T04:40:59.545764+0000","last_active":"2023-12-20T04:40:59.936142+0000","last_peered":"2023-12-20T04:40:59.936142+0000","last_clean":"2023-12-20T04:40:59.936142+0000","last_became_active":"2023-12-20T04:40:59.545471+0000","last_became_peered":"2023-12-20T04:40:59.545471+0000","last_unstale":"2023-12-20T04:40:59.936142+0000","last_undegraded":"2023-12-20T04:40:59.936142+0000","last_fullsized":"2023-12-20T04:40:59.936142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:25:18.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259623+0000","last_change":"2023-12-20T04:40:59.937954+0000","last_active":"2023-12-20T04:41:01.259623+0000","last_peered":"2023-12-20T04:41:01.259623+0000","last_clean":"2023-12-20T04:41:01.259623+0000","last_became_active":"2023-12-20T04:40:59.937753+0000","last_became_peered":"2023-12-20T04:40:59.937753+0000","last_unstale":"2023-12-20T04:41:01.259623+0000","last_undegraded":"2023-12-20T04:41:01.259623+0000","last_fullsized":"2023-12-20T04:41:01.259623+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:21:09.542330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259747+0000","last_change":"2023-12-20T04:40:59.539018+0000","last_active":"2023-12-20T04:41:01.259747+0000","last_peered":"2023-12-20T04:41:01.259747+0000","last_clean":"2023-12-20T04:41:01.259747+0000","last_became_active":"2023-12-20T04:40:59.538738+0000","last_became_peered":"2023-12-20T04:40:59.538738+0000","last_unstale":"2023-12-20T04:41:01.259747+0000","last_undegraded":"2023-12-20T04:41:01.259747+0000","last_fullsized":"2023-12-20T04:41:01.259747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:52:06.343273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966164+0000","last_change":"2023-12-20T04:40:59.964614+0000","last_active":"2023-12-20T04:40:59.966164+0000","last_peered":"2023-12-20T04:40:59.966164+0000","last_clean":"2023-12-20T04:40:59.966164+0000","last_became_active":"2023-12-20T04:40:59.964159+0000","last_became_peered":"2023-12-20T04:40:59.964159+0000","last_unstale":"2023-12-20T04:40:59.966164+0000","last_undegraded":"2023-12-20T04:40:59.966164+0000","last_fullsized":"2023-12-20T04:40:59.966164+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:04.693874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"60'8","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.958852+0000","last_change":"2023-12-20T04:40:59.541090+0000","last_active":"2023-12-20T04:40:59.958852+0000","last_peered":"2023-12-20T04:40:59.958852+0000","last_clean":"2023-12-20T04:40:59.958852+0000","last_became_active":"2023-12-20T04:40:59.540807+0000","last_became_peered":"2023-12-20T04:40:59.540807+0000","last_unstale":"2023-12-20T04:40:59.958852+0000","last_undegraded":"2023-12-20T04:40:59.958852+0000","last_fullsized":"2023-12-20T04:40:59.958852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:37:48.689217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928002+0000","last_change":"2023-12-20T04:40:59.949117+0000","last_active":"2023-12-20T04:41:00.928002+0000","last_peered":"2023-12-20T04:41:00.928002+0000","last_clean":"2023-12-20T04:41:00.928002+0000","last_became_active":"2023-12-20T04:40:59.948845+0000","last_became_peered":"2023-12-20T04:40:59.948845+0000","last_unstale":"2023-12-20T04:41:00.928002+0000","last_undegraded":"2023-12-20T04:41:00.928002+0000","last_fullsized":"2023-12-20T04:41:00.928002+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:29:43.957431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928081+0000","last_change":"2023-12-20T04:40:59.539575+0000","last_active":"2023-12-20T04:41:00.928081+0000","last_peered":"2023-12-20T04:41:00.928081+0000","last_clean":"2023-12-20T04:41:00.928081+0000","last_became_active":"2023-12-20T04:40:59.539130+0000","last_became_peered":"2023-12-20T04:40:59.539130+0000","last_unstale":"2023-12-20T04:41:00.928081+0000","last_undegraded":"2023-12-20T04:41:00.928081+0000","last_fullsized":"2023-12-20T04:41:00.928081+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:20.780219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957405+0000","last_change":"2023-12-20T04:40:59.953804+0000","last_active":"2023-12-20T04:40:59.957405+0000","last_peered":"2023-12-20T04:40:59.957405+0000","last_clean":"2023-12-20T04:40:59.957405+0000","last_became_active":"2023-12-20T04:40:59.953548+0000","last_became_peered":"2023-12-20T04:40:59.953548+0000","last_unstale":"2023-12-20T04:40:59.957405+0000","last_undegraded":"2023-12-20T04:40:59.957405+0000","last_fullsized":"2023-12-20T04:40:59.957405+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:27.433632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928316+0000","last_change":"2023-12-20T04:40:59.542677+0000","last_active":"2023-12-20T04:41:00.928316+0000","last_peered":"2023-12-20T04:41:00.928316+0000","last_clean":"2023-12-20T04:41:00.928316+0000","last_became_active":"2023-12-20T04:40:59.542347+0000","last_became_peered":"2023-12-20T04:40:59.542347+0000","last_unstale":"2023-12-20T04:41:00.928316+0000","last_undegraded":"2023-12-20T04:41:00.928316+0000","last_fullsized":"2023-12-20T04:41:00.928316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:37:48.689217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.964878+0000","last_change":"2023-12-20T04:40:59.960527+0000","last_active":"2023-12-20T04:40:59.964878+0000","last_peered":"2023-12-20T04:40:59.964878+0000","last_clean":"2023-12-20T04:40:59.964878+0000","last_became_active":"2023-12-20T04:40:59.959321+0000","last_became_peered":"2023-12-20T04:40:59.959321+0000","last_unstale":"2023-12-20T04:40:59.964878+0000","last_undegraded":"2023-12-20T04:40:59.964878+0000","last_fullsized":"2023-12-20T04:40:59.964878+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:18.708635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.930074+0000","last_change":"2023-12-20T04:40:59.540201+0000","last_active":"2023-12-20T04:41:00.930074+0000","last_peered":"2023-12-20T04:41:00.930074+0000","last_clean":"2023-12-20T04:41:00.930074+0000","last_became_active":"2023-12-20T04:40:59.539901+0000","last_became_peered":"2023-12-20T04:40:59.539901+0000","last_unstale":"2023-12-20T04:41:00.930074+0000","last_undegraded":"2023-12-20T04:41:00.930074+0000","last_fullsized":"2023-12-20T04:41:00.930074+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:21:17.066394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966475+0000","last_change":"2023-12-20T04:40:59.962635+0000","last_active":"2023-12-20T04:40:59.966475+0000","last_peered":"2023-12-20T04:40:59.966475+0000","last_clean":"2023-12-20T04:40:59.966475+0000","last_became_active":"2023-12-20T04:40:59.962339+0000","last_became_peered":"2023-12-20T04:40:59.962339+0000","last_unstale":"2023-12-20T04:40:59.966475+0000","last_undegraded":"2023-12-20T04:40:59.966475+0000","last_fullsized":"2023-12-20T04:40:59.966475+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:29.949027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259955+0000","last_change":"2023-12-20T04:40:59.546841+0000","last_active":"2023-12-20T04:41:01.259955+0000","last_peered":"2023-12-20T04:41:01.259955+0000","last_clean":"2023-12-20T04:41:01.259955+0000","last_became_active":"2023-12-20T04:40:59.546550+0000","last_became_peered":"2023-12-20T04:40:59.546550+0000","last_unstale":"2023-12-20T04:41:01.259955+0000","last_undegraded":"2023-12-20T04:41:01.259955+0000","last_fullsized":"2023-12-20T04:41:01.259955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:50.221646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259798+0000","last_change":"2023-12-20T04:40:59.937996+0000","last_active":"2023-12-20T04:41:01.259798+0000","last_peered":"2023-12-20T04:41:01.259798+0000","last_clean":"2023-12-20T04:41:01.259798+0000","last_became_active":"2023-12-20T04:40:59.937730+0000","last_became_peered":"2023-12-20T04:40:59.937730+0000","last_unstale":"2023-12-20T04:41:01.259798+0000","last_undegraded":"2023-12-20T04:41:01.259798+0000","last_fullsized":"2023-12-20T04:41:01.259798+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:00:37.983783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.260039+0000","last_change":"2023-12-20T04:40:59.540776+0000","last_active":"2023-12-20T04:41:01.260039+0000","last_peered":"2023-12-20T04:41:01.260039+0000","last_clean":"2023-12-20T04:41:01.260039+0000","last_became_active":"2023-12-20T04:40:59.540500+0000","last_became_peered":"2023-12-20T04:40:59.540500+0000","last_unstale":"2023-12-20T04:41:01.260039+0000","last_undegraded":"2023-12-20T04:41:01.260039+0000","last_fullsized":"2023-12-20T04:41:01.260039+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:03.674661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.955870+0000","last_change":"2023-12-20T04:40:59.946041+0000","last_active":"2023-12-20T04:40:59.955870+0000","last_peered":"2023-12-20T04:40:59.955870+0000","last_clean":"2023-12-20T04:40:59.955870+0000","last_became_active":"2023-12-20T04:40:59.945794+0000","last_became_peered":"2023-12-20T04:40:59.945794+0000","last_unstale":"2023-12-20T04:40:59.955870+0000","last_undegraded":"2023-12-20T04:40:59.955870+0000","last_fullsized":"2023-12-20T04:40:59.955870+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:54.635898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929827+0000","last_change":"2023-12-20T04:40:59.546551+0000","last_active":"2023-12-20T04:41:00.929827+0000","last_peered":"2023-12-20T04:41:00.929827+0000","last_clean":"2023-12-20T04:41:00.929827+0000","last_became_active":"2023-12-20T04:40:59.546253+0000","last_became_peered":"2023-12-20T04:40:59.546253+0000","last_unstale":"2023-12-20T04:41:00.929827+0000","last_undegraded":"2023-12-20T04:41:00.929827+0000","last_fullsized":"2023-12-20T04:41:00.929827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:11:34.042110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927637+0000","last_change":"2023-12-20T04:40:59.950739+0000","last_active":"2023-12-20T04:41:00.927637+0000","last_peered":"2023-12-20T04:41:00.927637+0000","last_clean":"2023-12-20T04:41:00.927637+0000","last_became_active":"2023-12-20T04:40:59.950432+0000","last_became_peered":"2023-12-20T04:40:59.950432+0000","last_unstale":"2023-12-20T04:41:00.927637+0000","last_undegraded":"2023-12-20T04:41:00.927637+0000","last_fullsized":"2023-12-20T04:41:00.927637+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:29.949027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.933943+0000","last_change":"2023-12-20T04:40:59.548084+0000","last_active":"2023-12-20T04:40:59.933943+0000","last_peered":"2023-12-20T04:40:59.933943+0000","last_clean":"2023-12-20T04:40:59.933943+0000","last_became_active":"2023-12-20T04:40:59.547209+0000","last_became_peered":"2023-12-20T04:40:59.547209+0000","last_unstale":"2023-12-20T04:40:59.933943+0000","last_undegraded":"2023-12-20T04:40:59.933943+0000","last_fullsized":"2023-12-20T04:40:59.933943+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:21:17.066394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.953876+0000","last_change":"2023-12-20T04:40:59.943076+0000","last_active":"2023-12-20T04:40:59.953876+0000","last_peered":"2023-12-20T04:40:59.953876+0000","last_clean":"2023-12-20T04:40:59.953876+0000","last_became_active":"2023-12-20T04:40:59.942193+0000","last_became_peered":"2023-12-20T04:40:59.942193+0000","last_unstale":"2023-12-20T04:40:59.953876+0000","last_undegraded":"2023-12-20T04:40:59.953876+0000","last_fullsized":"2023-12-20T04:40:59.953876+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:25.915476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.935358+0000","last_change":"2023-12-20T04:40:59.543512+0000","last_active":"2023-12-20T04:40:59.935358+0000","last_peered":"2023-12-20T04:40:59.935358+0000","last_clean":"2023-12-20T04:40:59.935358+0000","last_became_active":"2023-12-20T04:40:59.542872+0000","last_became_peered":"2023-12-20T04:40:59.542872+0000","last_unstale":"2023-12-20T04:40:59.935358+0000","last_undegraded":"2023-12-20T04:40:59.935358+0000","last_fullsized":"2023-12-20T04:40:59.935358+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:53.616685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.965087+0000","last_change":"2023-12-20T04:40:59.960781+0000","last_active":"2023-12-20T04:40:59.965087+0000","last_peered":"2023-12-20T04:40:59.965087+0000","last_clean":"2023-12-20T04:40:59.965087+0000","last_became_active":"2023-12-20T04:40:59.959632+0000","last_became_peered":"2023-12-20T04:40:59.959632+0000","last_unstale":"2023-12-20T04:40:59.965087+0000","last_undegraded":"2023-12-20T04:40:59.965087+0000","last_fullsized":"2023-12-20T04:40:59.965087+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:29:43.957431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.260003+0000","last_change":"2023-12-20T04:40:59.545551+0000","last_active":"2023-12-20T04:41:01.260003+0000","last_peered":"2023-12-20T04:41:01.260003+0000","last_clean":"2023-12-20T04:41:01.260003+0000","last_became_active":"2023-12-20T04:40:59.545317+0000","last_became_peered":"2023-12-20T04:40:59.545317+0000","last_unstale":"2023-12-20T04:41:01.260003+0000","last_undegraded":"2023-12-20T04:41:01.260003+0000","last_fullsized":"2023-12-20T04:41:01.260003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:26.414419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957268+0000","last_change":"2023-12-20T04:40:59.949069+0000","last_active":"2023-12-20T04:40:59.957268+0000","last_peered":"2023-12-20T04:40:59.957268+0000","last_clean":"2023-12-20T04:40:59.957268+0000","last_became_active":"2023-12-20T04:40:59.948808+0000","last_became_peered":"2023-12-20T04:40:59.948808+0000","last_unstale":"2023-12-20T04:40:59.957268+0000","last_undegraded":"2023-12-20T04:40:59.957268+0000","last_fullsized":"2023-12-20T04:40:59.957268+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:02.479108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.936666+0000","last_change":"2023-12-20T04:40:59.549228+0000","last_active":"2023-12-20T04:40:59.936666+0000","last_peered":"2023-12-20T04:40:59.936666+0000","last_clean":"2023-12-20T04:40:59.936666+0000","last_became_active":"2023-12-20T04:40:59.548875+0000","last_became_peered":"2023-12-20T04:40:59.548875+0000","last_unstale":"2023-12-20T04:40:59.936666+0000","last_undegraded":"2023-12-20T04:40:59.936666+0000","last_fullsized":"2023-12-20T04:40:59.936666+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:52:06.756273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.956202+0000","last_change":"2023-12-20T04:40:59.947240+0000","last_active":"2023-12-20T04:40:59.956202+0000","last_peered":"2023-12-20T04:40:59.956202+0000","last_clean":"2023-12-20T04:40:59.956202+0000","last_became_active":"2023-12-20T04:40:59.946923+0000","last_became_peered":"2023-12-20T04:40:59.946923+0000","last_unstale":"2023-12-20T04:40:59.956202+0000","last_undegraded":"2023-12-20T04:40:59.956202+0000","last_fullsized":"2023-12-20T04:40:59.956202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:02.479108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.932915+0000","last_change":"2023-12-20T04:40:59.541887+0000","last_active":"2023-12-20T04:40:59.932915+0000","last_peered":"2023-12-20T04:40:59.932915+0000","last_clean":"2023-12-20T04:40:59.932915+0000","last_became_active":"2023-12-20T04:40:59.541348+0000","last_became_peered":"2023-12-20T04:40:59.541348+0000","last_unstale":"2023-12-20T04:40:59.932915+0000","last_undegraded":"2023-12-20T04:40:59.932915+0000","last_fullsized":"2023-12-20T04:40:59.932915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:21:50.352031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259891+0000","last_change":"2023-12-20T04:40:59.936829+0000","last_active":"2023-12-20T04:41:01.259891+0000","last_peered":"2023-12-20T04:41:01.259891+0000","last_clean":"2023-12-20T04:41:01.259891+0000","last_became_active":"2023-12-20T04:40:59.936519+0000","last_became_peered":"2023-12-20T04:40:59.936519+0000","last_unstale":"2023-12-20T04:41:01.259891+0000","last_undegraded":"2023-12-20T04:41:01.259891+0000","last_fullsized":"2023-12-20T04:41:01.259891+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:58:45.658383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.935962+0000","last_change":"2023-12-20T04:40:59.548395+0000","last_active":"2023-12-20T04:40:59.935962+0000","last_peered":"2023-12-20T04:40:59.935962+0000","last_clean":"2023-12-20T04:40:59.935962+0000","last_became_active":"2023-12-20T04:40:59.548134+0000","last_became_peered":"2023-12-20T04:40:59.548134+0000","last_unstale":"2023-12-20T04:40:59.935962+0000","last_undegraded":"2023-12-20T04:40:59.935962+0000","last_fullsized":"2023-12-20T04:40:59.935962+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:19:54.323865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259745+0000","last_change":"2023-12-20T04:40:59.938247+0000","last_active":"2023-12-20T04:41:01.259745+0000","last_peered":"2023-12-20T04:41:01.259745+0000","last_clean":"2023-12-20T04:41:01.259745+0000","last_became_active":"2023-12-20T04:40:59.938109+0000","last_became_peered":"2023-12-20T04:40:59.938109+0000","last_unstale":"2023-12-20T04:41:01.259745+0000","last_undegraded":"2023-12-20T04:41:01.259745+0000","last_fullsized":"2023-12-20T04:41:01.259745+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:33:20.043278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928103+0000","last_change":"2023-12-20T04:40:59.539669+0000","last_active":"2023-12-20T04:41:00.928103+0000","last_peered":"2023-12-20T04:41:00.928103+0000","last_clean":"2023-12-20T04:41:00.928103+0000","last_became_active":"2023-12-20T04:40:59.539036+0000","last_became_peered":"2023-12-20T04:40:59.539036+0000","last_unstale":"2023-12-20T04:41:00.928103+0000","last_undegraded":"2023-12-20T04:41:00.928103+0000","last_fullsized":"2023-12-20T04:41:00.928103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:53.616685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929770+0000","last_change":"2023-12-20T04:40:59.945678+0000","last_active":"2023-12-20T04:41:00.929770+0000","last_peered":"2023-12-20T04:41:00.929770+0000","last_clean":"2023-12-20T04:41:00.929770+0000","last_became_active":"2023-12-20T04:40:59.943177+0000","last_became_peered":"2023-12-20T04:40:59.943177+0000","last_unstale":"2023-12-20T04:41:00.929770+0000","last_undegraded":"2023-12-20T04:41:00.929770+0000","last_fullsized":"2023-12-20T04:41:00.929770+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:37:42.875580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.932854+0000","last_change":"2023-12-20T04:40:59.545744+0000","last_active":"2023-12-20T04:40:59.932854+0000","last_peered":"2023-12-20T04:40:59.932854+0000","last_clean":"2023-12-20T04:40:59.932854+0000","last_became_active":"2023-12-20T04:40:59.545462+0000","last_became_peered":"2023-12-20T04:40:59.545462+0000","last_unstale":"2023-12-20T04:40:59.932854+0000","last_undegraded":"2023-12-20T04:40:59.932854+0000","last_fullsized":"2023-12-20T04:40:59.932854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:33:19.014284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927938+0000","last_change":"2023-12-20T04:40:59.955425+0000","last_active":"2023-12-20T04:41:00.927938+0000","last_peered":"2023-12-20T04:41:00.927938+0000","last_clean":"2023-12-20T04:41:00.927938+0000","last_became_active":"2023-12-20T04:40:59.955120+0000","last_became_peered":"2023-12-20T04:40:59.955120+0000","last_unstale":"2023-12-20T04:41:00.927938+0000","last_undegraded":"2023-12-20T04:41:00.927938+0000","last_fullsized":"2023-12-20T04:41:00.927938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:02.479108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.937060+0000","last_change":"2023-12-20T04:40:59.537303+0000","last_active":"2023-12-20T04:40:59.937060+0000","last_peered":"2023-12-20T04:40:59.937060+0000","last_clean":"2023-12-20T04:40:59.937060+0000","last_became_active":"2023-12-20T04:40:59.537015+0000","last_became_peered":"2023-12-20T04:40:59.537015+0000","last_unstale":"2023-12-20T04:40:59.937060+0000","last_undegraded":"2023-12-20T04:40:59.937060+0000","last_fullsized":"2023-12-20T04:40:59.937060+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:43:20.804066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957401+0000","last_change":"2023-12-20T04:40:59.948696+0000","last_active":"2023-12-20T04:40:59.957401+0000","last_peered":"2023-12-20T04:40:59.957401+0000","last_clean":"2023-12-20T04:40:59.957401+0000","last_became_active":"2023-12-20T04:40:59.948430+0000","last_became_peered":"2023-12-20T04:40:59.948430+0000","last_unstale":"2023-12-20T04:40:59.957401+0000","last_undegraded":"2023-12-20T04:40:59.957401+0000","last_fullsized":"2023-12-20T04:40:59.957401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:34:27.599058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.930125+0000","last_change":"2023-12-20T04:40:59.539524+0000","last_active":"2023-12-20T04:41:00.930125+0000","last_peered":"2023-12-20T04:41:00.930125+0000","last_clean":"2023-12-20T04:41:00.930125+0000","last_became_active":"2023-12-20T04:40:59.539230+0000","last_became_peered":"2023-12-20T04:40:59.539230+0000","last_unstale":"2023-12-20T04:41:00.930125+0000","last_undegraded":"2023-12-20T04:41:00.930125+0000","last_fullsized":"2023-12-20T04:41:00.930125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:39:48.794770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"48'139","reported_seq":280,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.537704+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:40:59.537704+0000","last_peered":"2023-12-20T04:40:59.537704+0000","last_clean":"2023-12-20T04:40:59.537704+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:40:59.537704+0000","last_undegraded":"2023-12-20T04:40:59.537704+0000","last_fullsized":"2023-12-20T04:40:59.537704+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36424,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335416,"statfs":{"total":96636764160,"available":96599465984,"internally_reserved":0,"allocated":1310720,"data_stored":917904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72799999999999998,"15min":0.68999999999999995},"min":{"1min":0.372,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71999999999999997,"15min":0.70499999999999996},"min":{"1min":0.46800000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99299999999999999,"5min":1,"15min":1.115},"last":0.42499999999999999}]},{"osd":1,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.74099999999999999,"15min":0.71099999999999997},"min":{"1min":0.46100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":1.0529999999999999,"15min":1.2190000000000001},"last":0.443},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.74099999999999999,"15min":0.752},"min":{"1min":0.55200000000000005,"5min":0.46200000000000002,"15min":0.42999999999999999},"max":{"1min":1.032,"5min":1.032,"15min":2.649},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.498,"5min":0.48899999999999999,"15min":0.437},"max":{"1min":1.075,"5min":1.075,"15min":1.5960000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.753,"15min":0.74399999999999999},"min":{"1min":0.50600000000000001,"5min":0.505,"15min":0.45300000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.135},"last":0.55500000000000005}]},{"osd":3,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73999999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.016,"5min":1.016,"15min":1.641},"last":0.498},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76000000000000001,"15min":0.79000000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.49399999999999999},"max":{"1min":1.131,"5min":1.131,"15min":2.5990000000000002},"last":0.627}]},{"osd":4,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.61399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":0.753,"5min":0.98299999999999998,"15min":1.0109999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.60199999999999998,"15min":0.64300000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.242},"last":0.60199999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.627,"15min":0.66600000000000004},"min":{"1min":0.46899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.877,"5min":1.0449999999999999,"15min":1.0900000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65100000000000002,"15min":0.69099999999999995},"min":{"1min":0.47599999999999998,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":0.86399999999999999,"5min":1.052,"15min":1.071},"last":0.68600000000000005}]},{"osd":6,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.75},"min":{"1min":0.54900000000000004,"5min":0.47699999999999998,"15min":0.443},"max":{"1min":0.90700000000000003,"5min":1.1020000000000001,"15min":1.1279999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.71499999999999997},"min":{"1min":0.505,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.99199999999999999},"last":0.65700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":720896,"data_stored":328118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.72299999999999998,"15min":0.69699999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.27100000000000002},"max":{"1min":0.95899999999999996,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.96399999999999997,"15min":1.1910000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89500000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.748,"15min":0.71199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.997,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.73499999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.77000000000000002,"15min":0.749},"min":{"1min":0.47299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.026,"5min":1.216,"15min":1.216},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.748,"15min":0.71899999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.38300000000000001},"max":{"1min":0.95199999999999996,"5min":1.01,"15min":1.054},"last":0.625}]},{"osd":3,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.78000000000000003,"15min":0.74299999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.45600000000000002},"max":{"1min":0.99299999999999999,"5min":1.0600000000000001,"15min":1.109},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.73899999999999999,"15min":0.73699999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92000000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79300000000000004,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.501},{"interface":"front","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.65200000000000002},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":0.995,"15min":1.071},"last":0.77500000000000002}]},{"osd":5,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.624,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92600000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.754},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.70299999999999996},"min":{"1min":0.47399999999999998,"5min":0.45700000000000002,"15min":0.434},"max":{"1min":0.89700000000000002,"5min":1.306,"15min":1.306},"last":0.79400000000000004}]},{"osd":7,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70499999999999996,"15min":0.72999999999999998},"min":{"1min":0.504,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94499999999999995,"5min":0.99099999999999999,"15min":1.2270000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.73199999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96599999999999997,"5min":1.002,"15min":1.169},"last":0.81100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":716800,"data_stored":327516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70799999999999996,"15min":0.67100000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.29999999999999999},"max":{"1min":0.94999999999999996,"5min":1.006,"15min":1.05},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.34300000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.98599999999999999},"last":0.78400000000000003}]},{"osd":1,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":0.69199999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.441},"max":{"1min":1.044,"5min":1.046,"15min":1.0600000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73299999999999998,"15min":0.73599999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":1.024,"15min":1.1140000000000001},"last":0.83399999999999996}]},{"osd":2,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.53000000000000003,"5min":0.432,"15min":0.40500000000000003},"max":{"1min":0.93300000000000005,"5min":0.96299999999999997,"15min":1.153},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.47899999999999998},"max":{"1min":0.90300000000000002,"5min":0.95999999999999996,"15min":1.004},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47499999999999998},"max":{"1min":0.95999999999999996,"5min":1.018,"15min":1.758},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.77200000000000002,"15min":0.74299999999999999},"min":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.436},"max":{"1min":0.88400000000000001,"5min":1.0149999999999999,"15min":1.0469999999999999},"last":0.85399999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60899999999999999,"15min":0.626},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":0.92700000000000005,"15min":1.0229999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.623,"15min":0.64200000000000002},"min":{"1min":0.439,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84999999999999998,"5min":1.077,"15min":1.077},"last":0.61399999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68000000000000005,"15min":0.68700000000000006},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.39900000000000002},"max":{"1min":0.82499999999999996,"5min":1.056,"15min":1.0700000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63700000000000001,"15min":0.65300000000000002},"min":{"1min":0.42899999999999999,"5min":0.39400000000000002,"15min":0.38600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.70799999999999996},"min":{"1min":0.48999999999999999,"5min":0.442,"15min":0.42999999999999999},"max":{"1min":0.86799999999999999,"5min":1.141,"15min":1.141},"last":0.754},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.74399999999999999,"15min":0.755},"min":{"1min":0.54900000000000004,"5min":0.496,"15min":0.47899999999999998},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":1.119},"last":0.79700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317940,"statfs":{"total":96636764160,"available":96581570560,"internally_reserved":0,"allocated":724992,"data_stored":329010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.41699999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0740000000000001,"15min":1.5509999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.36199999999999999},"max":{"1min":1.625,"5min":1.625,"15min":1.633},"last":0.51300000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71699999999999997,"15min":0.70799999999999996},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88700000000000001,"5min":0.98799999999999999,"15min":1.798},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.751,"15min":0.747},"min":{"1min":0.45900000000000002,"5min":0.442,"15min":0.433},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.499}]},{"osd":2,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.43099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.52,"5min":1.52,"15min":1.728},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.55900000000000005,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.6619999999999999},"last":0.56899999999999995}]},{"osd":3,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73399999999999999,"15min":0.72299999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.40799999999999997},"max":{"1min":1.554,"5min":1.554,"15min":1.5760000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.75900000000000001,"15min":0.76600000000000001},"min":{"1min":0.57199999999999995,"5min":0.498,"15min":0.48299999999999998},"max":{"1min":0.91900000000000004,"5min":1.0620000000000001,"15min":1.7649999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.63800000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.65100000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64400000000000002,"15min":0.67600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85899999999999999,"5min":1.099,"15min":1.6739999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.71199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.41899999999999998}]},{"osd":7,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.70899999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73299999999999998,"15min":0.751},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97999999999999998,"5min":1.125,"15min":1.125},"last":0.47399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35852,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335988,"statfs":{"total":96636764160,"available":96600051712,"internally_reserved":0,"allocated":724992,"data_stored":328084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46100000000000002,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.26200000000000001,"15min":0.23200000000000001},"max":{"1min":0.875,"5min":0.875,"15min":4.3700000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.32500000000000001,"15min":0.28799999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":3.355},"last":0.30599999999999999}]},{"osd":1,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.52100000000000002},"min":{"1min":0.32000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.58499999999999996,"5min":0.70099999999999996,"15min":3.7160000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55600000000000005,"15min":0.58799999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.28399999999999997},"max":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":3.3109999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58599999999999997,"15min":0.63},"min":{"1min":0.37,"5min":0.318,"15min":0.318},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":4.0110000000000001},"last":0.44},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55900000000000005,"15min":0.66000000000000003},"min":{"1min":0.41199999999999998,"5min":0.34799999999999998,"15min":0.316},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":6.1399999999999997},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.48499999999999999},"max":{"1min":1.115,"5min":1.131,"15min":1.3089999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81299999999999994,"15min":0.87},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.198,"5min":1.4059999999999999,"15min":3.9510000000000001},"last":0.76800000000000002}]},{"osd":5,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81299999999999994,"15min":0.88300000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.46500000000000002},"max":{"1min":1.1000000000000001,"5min":1.3300000000000001,"15min":3.9830000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78900000000000003,"15min":0.874},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.38400000000000001},"max":{"1min":1.0609999999999999,"5min":1.252,"15min":4.6550000000000002},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.45600000000000002},"max":{"1min":1.002,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.82099999999999995,"15min":0.88},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.499},"max":{"1min":1.2270000000000001,"5min":1.2929999999999999,"15min":3.8900000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80400000000000005,"15min":0.88700000000000001},"min":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.49299999999999999},"max":{"1min":1.1319999999999999,"5min":1.1659999999999999,"15min":4.6970000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.55100000000000005},"max":{"1min":1.077,"5min":1.29,"15min":1.29},"last":0.69899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39964,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331876,"statfs":{"total":96636764160,"available":96595841024,"internally_reserved":0,"allocated":741376,"data_stored":335332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.52800000000000002,"5min":0.78900000000000003,"15min":0.89600000000000002},"last":0.309},{"interface":"front","average":{"1min":0.433,"5min":0.50600000000000001,"15min":0.47999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.26700000000000002},"max":{"1min":0.626,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.66700000000000004}]},{"osd":1,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.48099999999999998,"15min":0.46100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28899999999999998},"max":{"1min":0.64000000000000001,"5min":0.876,"15min":0.876},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.28100000000000003},"max":{"1min":0.629,"5min":0.88600000000000001,"15min":1.01},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.61199999999999999,"15min":0.58299999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.34999999999999998},"max":{"1min":0.78500000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64100000000000001,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.754,"5min":0.95599999999999996,"15min":1.0900000000000001},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.79500000000000004,"15min":0.78900000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.45200000000000001},"max":{"1min":0.92300000000000004,"5min":1.145,"15min":1.1799999999999999},"last":0.503},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41399999999999998},"max":{"1min":1.002,"5min":1.0760000000000001,"15min":1.2010000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.79300000000000004,"15min":0.78000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.48499999999999999},"max":{"1min":0.93000000000000005,"5min":1.1220000000000001,"15min":1.327},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.80100000000000005,"15min":0.80800000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.016,"5min":1.101,"15min":1.381},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.79500000000000004,"15min":0.79200000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":1.151,"15min":1.2809999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.80500000000000005,"15min":0.80000000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.46100000000000002},"max":{"1min":0.94599999999999995,"5min":1.091,"15min":1.153},"last":0.52000000000000002}]},{"osd":7,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.82299999999999995,"15min":0.82399999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95499999999999996,"5min":1.1499999999999999,"15min":1.3520000000000001},"last":0.626},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.86499999999999999,"15min":0.85399999999999998},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.504},"max":{"1min":0.97499999999999998,"5min":1.1220000000000001,"15min":1.401},"last":0.68000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":716800,"data_stored":327516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.44800000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57699999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.28599999999999998},"max":{"1min":0.60999999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.32800000000000001},"max":{"1min":0.81399999999999995,"5min":1.091,"15min":1.091},"last":0.442},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.33900000000000002},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.91300000000000003},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36599999999999999},"max":{"1min":0.73599999999999999,"5min":0.96499999999999997,"15min":1.016},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.54400000000000004,"15min":0.52100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64900000000000002,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.374}]},{"osd":4,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76100000000000001,"15min":0.77200000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.78000000000000003,"5min":1.27,"15min":1.4419999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88900000000000001,"5min":1.1160000000000001,"15min":1.3380000000000001},"last":0.73499999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.55400000000000005,"5min":0.54000000000000004,"15min":0.5},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.63},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.77000000000000002,"15min":0.79100000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.873,"5min":1.2110000000000001,"15min":1.3009999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.84199999999999997,"5min":1.194,"15min":1.415},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.252,"5min":1.252,"15min":1.268},"last":0.75900000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82499999999999996,"15min":0.83999999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.306,"5min":1.306,"15min":1.399},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94699999999999995,"5min":1.3440000000000001,"15min":1.373},"last":0.748}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1310720,"data_stored":917848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.46000000000000002,"15min":0.438},"min":{"1min":0.28499999999999998,"5min":0.218,"15min":0.218},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.60099999999999998,"15min":0.52800000000000002},"min":{"1min":0.35499999999999998,"5min":0.29699999999999999,"15min":0.26900000000000002},"max":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.54100000000000004,"15min":0.52100000000000002},"min":{"1min":0.35799999999999998,"5min":0.28899999999999998,"15min":0.28399999999999997},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.34000000000000002},"max":{"1min":0.77600000000000002,"5min":0.78000000000000003,"15min":1.2989999999999999},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55200000000000005,"15min":0.51500000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.29499999999999998},"max":{"1min":3.1000000000000001,"5min":3.1000000000000001,"15min":3.1000000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":1.103,"5min":0.69499999999999995,"15min":0.61099999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38200000000000001},"max":{"1min":10.49,"5min":10.49,"15min":10.49},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.81200000000000006,"15min":0.82299999999999995},"min":{"1min":0.58799999999999997,"5min":0.51100000000000001,"15min":0.45400000000000001},"max":{"1min":3.0659999999999998,"5min":3.0659999999999998,"15min":3.7469999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.84299999999999997,"15min":0.80100000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.2460000000000004,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.96699999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.81399999999999995,"15min":0.80500000000000005},"min":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.44700000000000001},"max":{"1min":0.996,"5min":1.3109999999999999,"15min":1.877},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.56200000000000006,"5min":0.496,"15min":0.496},"max":{"1min":1.151,"5min":1.2010000000000001,"15min":1.7450000000000001},"last":0.93999999999999995}]},{"osd":6,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.86199999999999999,"15min":0.82399999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.47299999999999998},"max":{"1min":5.2839999999999998,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81699999999999995,"15min":0.81899999999999995},"min":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.046,"5min":1.1479999999999999,"15min":1.492},"last":0.80000000000000004}]},{"osd":7,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82899999999999996,"15min":0.83699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0900000000000001,"5min":1.216,"15min":1.6539999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81799999999999995,"15min":0.82099999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.276,"15min":2.113},"last":0.91300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:41:04.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:41:04.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.217+0000 7fa90fa41640 1 Processor -- start 2023-12-20T04:41:04.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.217+0000 7fa90fa41640 1 -- start start 2023-12-20T04:41:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.217+0000 7fa90fa41640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa9080af5e0 0x7fa9080af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.217+0000 7fa90fa41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 0x7fa90815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.217+0000 7fa90fa41640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa90815b990 con 0x7fa90806ce60 2023-12-20T04:41:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.217+0000 7fa90fa41640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa90815bbd0 con 0x7fa9080af5e0 2023-12-20T04:41:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.217+0000 7fa90fa41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa90815beb0 con 0x7fa90815f110 2023-12-20T04:41:04.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.217+0000 7fa90dfde640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 0x7fa90815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.217+0000 7fa90d7dd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa9080af5e0 0x7fa9080af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.217+0000 7fa90dfde640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 0x7fa90815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45064/0 (socket says 172.21.15.32:45064) 2023-12-20T04:41:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.217+0000 7fa90dfde640 1 -- 172.21.15.32:0/467965208 learned_addr learned my addr 172.21.15.32:0/467965208 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa90dfde640 1 -- 172.21.15.32:0/467965208 >> v2:172.21.15.39:3300/0 conn(0x7fa9080af5e0 msgr2=0x7fa9080af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa90dfde640 1 --2- 172.21.15.32:0/467965208 >> v2:172.21.15.39:3300/0 conn(0x7fa9080af5e0 0x7fa9080af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa90dfde640 1 -- 172.21.15.32:0/467965208 >> v1:172.21.15.39:6789/0 conn(0x7fa90806ce60 legacy=0x7fa90815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa90dfde640 1 -- 172.21.15.32:0/467965208 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa90815e910 con 0x7fa90815f110 2023-12-20T04:41:04.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa8fffff640 1 -- 172.21.15.32:0/467965208 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3614984868 0 0) 0x7fa8f8001120 con 0x7fa90806ce60 2023-12-20T04:41:04.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa90d7dd640 1 --2- 172.21.15.32:0/467965208 >> v2:172.21.15.39:3300/0 conn(0x7fa9080af5e0 0x7fa9080af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:41:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa90dfde640 1 --2- 172.21.15.32:0/467965208 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 0x7fa90815c440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa900009e50 tx=0x7fa90001c9e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a64ca7cccc08de62 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa8fffff640 1 -- 172.21.15.32:0/467965208 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000a58d0 con 0x7fa90815f110 2023-12-20T04:41:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa8fffff640 1 -- 172.21.15.32:0/467965208 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9000a5a30 con 0x7fa90815f110 2023-12-20T04:41:04.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa8fffff640 1 -- 172.21.15.32:0/467965208 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9000a5dc0 con 0x7fa90815f110 2023-12-20T04:41:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa90fa41640 1 -- 172.21.15.32:0/467965208 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 msgr2=0x7fa90815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.221+0000 7fa90fa41640 1 --2- 172.21.15.32:0/467965208 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 0x7fa90815c440 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fa900009e50 tx=0x7fa90001c9e0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 -- 172.21.15.32:0/467965208 shutdown_connections 2023-12-20T04:41:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 --2- 172.21.15.32:0/467965208 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 0x7fa90815c440 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 --2- 172.21.15.32:0/467965208 >> v2:172.21.15.39:3300/0 conn(0x7fa9080af5e0 0x7fa9080af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 -- 172.21.15.32:0/467965208 >> 172.21.15.32:0/467965208 conn(0x7fa90806a320 msgr2=0x7fa9080ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 -- 172.21.15.32:0/467965208 shutdown_connections 2023-12-20T04:41:04.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 -- 172.21.15.32:0/467965208 wait complete. 2023-12-20T04:41:04.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 Processor -- start 2023-12-20T04:41:04.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 -- start start 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90806ce60 0x7fa90814c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080af5e0 0x7fa90814cc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 0x7fa90814d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080ce4c0 con 0x7fa9080af5e0 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9080ce1e0 con 0x7fa90806ce60 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90fa41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa908070260 con 0x7fa90815f110 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90cfdc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080af5e0 0x7fa90814cc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90cfdc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080af5e0 0x7fa90814cc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51462/0 (socket says 172.21.15.32:51462) 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90cfdc640 1 -- 172.21.15.32:0/2027303064 learned_addr learned my addr 172.21.15.32:0/2027303064 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90d7dd640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90806ce60 0x7fa90814c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90dfde640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 0x7fa90814d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90cfdc640 1 -- 172.21.15.32:0/2027303064 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 msgr2=0x7fa90814d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90cfdc640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 0x7fa90814d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90cfdc640 1 -- 172.21.15.32:0/2027303064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90806ce60 msgr2=0x7fa90814c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90cfdc640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90806ce60 0x7fa90814c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90cfdc640 1 -- 172.21.15.32:0/2027303064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9040093b0 con 0x7fa9080af5e0 2023-12-20T04:41:04.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.225+0000 7fa90cfdc640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080af5e0 0x7fa90814cc00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c750 tx=0x7fa8f800cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:04.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.229+0000 7fa8fdffb640 1 -- 172.21.15.32:0/2027303064 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f8005270 con 0x7fa9080af5e0 2023-12-20T04:41:04.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.229+0000 7fa8fdffb640 1 -- 172.21.15.32:0/2027303064 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8f80053f0 con 0x7fa9080af5e0 2023-12-20T04:41:04.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.229+0000 7fa8fdffb640 1 -- 172.21.15.32:0/2027303064 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8f800a880 con 0x7fa9080af5e0 2023-12-20T04:41:04.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.229+0000 7fa90fa41640 1 -- 172.21.15.32:0/2027303064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa900009b30 con 0x7fa9080af5e0 2023-12-20T04:41:04.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.229+0000 7fa90fa41640 1 -- 172.21.15.32:0/2027303064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa908070820 con 0x7fa9080af5e0 2023-12-20T04:41:04.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.229+0000 7fa90fa41640 1 -- 172.21.15.32:0/2027303064 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa90804e000 con 0x7fa9080af5e0 2023-12-20T04:41:04.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.229+0000 7fa8fdffb640 1 -- 172.21.15.32:0/2027303064 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa8f800a9e0 con 0x7fa9080af5e0 2023-12-20T04:41:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.229+0000 7fa8fdffb640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8ec065b80 0x7fa8ec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.229+0000 7fa8fdffb640 1 -- 172.21.15.32:0/2027303064 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa8f80826d0 con 0x7fa9080af5e0 2023-12-20T04:41:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.229+0000 7fa90d7dd640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8ec065b80 0x7fa8ec068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.233+0000 7fa90d7dd640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8ec065b80 0x7fa8ec068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa9080cf160 tx=0x7fa904006f90 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:04.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.233+0000 7fa8fdffb640 1 -- 172.21.15.32:0/2027303064 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa8f8050030 con 0x7fa9080af5e0 2023-12-20T04:41:04.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.389+0000 7fa90fa41640 1 -- 172.21.15.32:0/2027303064 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa90815b990 con 0x7fa8ec065b80 2023-12-20T04:41:04.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.393+0000 7fa8fdffb640 1 -- 172.21.15.32:0/2027303064 <== mgr.5664 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288499 (secure 0 0 0) 0x7fa90815b990 con 0x7fa8ec065b80 2023-12-20T04:41:04.397 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:41:04.399 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:41:04.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.397+0000 7fa90fa41640 1 -- 172.21.15.32:0/2027303064 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8ec065b80 msgr2=0x7fa8ec068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.397+0000 7fa90fa41640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8ec065b80 0x7fa8ec068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fa9080cf160 tx=0x7fa904006f90 comp rx=0 tx=0).stop 2023-12-20T04:41:04.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.397+0000 7fa90fa41640 1 -- 172.21.15.32:0/2027303064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080af5e0 msgr2=0x7fa90814cc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.397+0000 7fa90fa41640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080af5e0 0x7fa90814cc00 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7fa8f800c750 tx=0x7fa8f800cc10 comp rx=0 tx=0).stop 2023-12-20T04:41:04.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.397+0000 7fa90fa41640 1 -- 172.21.15.32:0/2027303064 shutdown_connections 2023-12-20T04:41:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.397+0000 7fa90fa41640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8ec065b80 0x7fa8ec068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.397+0000 7fa90fa41640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa90815f110 0x7fa90814d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.397+0000 7fa90fa41640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9080af5e0 0x7fa90814cc00 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.397+0000 7fa90fa41640 1 --2- 172.21.15.32:0/2027303064 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa90806ce60 0x7fa90814c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.397+0000 7fa90fa41640 1 -- 172.21.15.32:0/2027303064 >> 172.21.15.32:0/2027303064 conn(0x7fa90806a320 msgr2=0x7fa9080cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.401+0000 7fa90fa41640 1 -- 172.21.15.32:0/2027303064 shutdown_connections 2023-12-20T04:41:04.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.401+0000 7fa90fa41640 1 -- 172.21.15.32:0/2027303064 wait complete. 2023-12-20T04:41:04.418 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":25,"stamp":"2023-12-20T04:41:03.265840+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":243,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":177,"ondisk_log_size":177,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":124,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":297812,"kb_used_data":6804,"kb_used_omap":0,"kb_used_meta":290944,"kb_avail":754676908,"statfs":{"total":773094113280,"available":772789153792,"internally_reserved":0,"allocated":6967296,"data_stored":3811328,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":297926656},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.756120"},"pg_stats":[{"pgid":"7.3f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.961113+0000","last_change":"2023-12-20T04:40:59.950537+0000","last_active":"2023-12-20T04:40:59.961113+0000","last_peered":"2023-12-20T04:40:59.961113+0000","last_clean":"2023-12-20T04:40:59.961113+0000","last_became_active":"2023-12-20T04:40:59.950185+0000","last_became_peered":"2023-12-20T04:40:59.950185+0000","last_unstale":"2023-12-20T04:40:59.961113+0000","last_undegraded":"2023-12-20T04:40:59.961113+0000","last_fullsized":"2023-12-20T04:40:59.961113+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:52:07.362486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.930127+0000","last_change":"2023-12-20T04:40:59.953877+0000","last_active":"2023-12-20T04:41:00.930127+0000","last_peered":"2023-12-20T04:41:00.930127+0000","last_clean":"2023-12-20T04:41:00.930127+0000","last_became_active":"2023-12-20T04:40:59.953540+0000","last_became_peered":"2023-12-20T04:40:59.953540+0000","last_unstale":"2023-12-20T04:41:00.930127+0000","last_undegraded":"2023-12-20T04:41:00.930127+0000","last_fullsized":"2023-12-20T04:41:00.930127+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:07.685523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3d","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928093+0000","last_change":"2023-12-20T04:40:59.952341+0000","last_active":"2023-12-20T04:41:00.928093+0000","last_peered":"2023-12-20T04:41:00.928093+0000","last_clean":"2023-12-20T04:41:00.928093+0000","last_became_active":"2023-12-20T04:40:59.952037+0000","last_became_peered":"2023-12-20T04:40:59.952037+0000","last_unstale":"2023-12-20T04:41:00.928093+0000","last_undegraded":"2023-12-20T04:41:00.928093+0000","last_fullsized":"2023-12-20T04:41:00.928093+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:24.973589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.3c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.965928+0000","last_change":"2023-12-20T04:40:59.964203+0000","last_active":"2023-12-20T04:40:59.965928+0000","last_peered":"2023-12-20T04:40:59.965928+0000","last_clean":"2023-12-20T04:40:59.965928+0000","last_became_active":"2023-12-20T04:40:59.963916+0000","last_became_peered":"2023-12-20T04:40:59.963916+0000","last_unstale":"2023-12-20T04:40:59.965928+0000","last_undegraded":"2023-12-20T04:40:59.965928+0000","last_fullsized":"2023-12-20T04:40:59.965928+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:33:20.043278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.3b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929917+0000","last_change":"2023-12-20T04:40:59.954330+0000","last_active":"2023-12-20T04:41:00.929917+0000","last_peered":"2023-12-20T04:41:00.929917+0000","last_clean":"2023-12-20T04:41:00.929917+0000","last_became_active":"2023-12-20T04:40:59.954017+0000","last_became_peered":"2023-12-20T04:40:59.954017+0000","last_unstale":"2023-12-20T04:41:00.929917+0000","last_undegraded":"2023-12-20T04:41:00.929917+0000","last_fullsized":"2023-12-20T04:41:00.929917+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:24.973589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.3a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259696+0000","last_change":"2023-12-20T04:40:59.948940+0000","last_active":"2023-12-20T04:41:01.259696+0000","last_peered":"2023-12-20T04:41:01.259696+0000","last_clean":"2023-12-20T04:41:01.259696+0000","last_became_active":"2023-12-20T04:40:59.948668+0000","last_became_peered":"2023-12-20T04:40:59.948668+0000","last_unstale":"2023-12-20T04:41:01.259696+0000","last_undegraded":"2023-12-20T04:41:01.259696+0000","last_fullsized":"2023-12-20T04:41:01.259696+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:14:57.533936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.39","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.956104+0000","last_change":"2023-12-20T04:40:59.947725+0000","last_active":"2023-12-20T04:40:59.956104+0000","last_peered":"2023-12-20T04:40:59.956104+0000","last_clean":"2023-12-20T04:40:59.956104+0000","last_became_active":"2023-12-20T04:40:59.947430+0000","last_became_peered":"2023-12-20T04:40:59.947430+0000","last_unstale":"2023-12-20T04:40:59.956104+0000","last_undegraded":"2023-12-20T04:40:59.956104+0000","last_fullsized":"2023-12-20T04:40:59.956104+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:27.433632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.38","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966348+0000","last_change":"2023-12-20T04:40:59.965242+0000","last_active":"2023-12-20T04:40:59.966348+0000","last_peered":"2023-12-20T04:40:59.966348+0000","last_clean":"2023-12-20T04:40:59.966348+0000","last_became_active":"2023-12-20T04:40:59.964302+0000","last_became_peered":"2023-12-20T04:40:59.964302+0000","last_unstale":"2023-12-20T04:40:59.966348+0000","last_undegraded":"2023-12-20T04:40:59.966348+0000","last_fullsized":"2023-12-20T04:40:59.966348+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:36:49.912753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.37","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957043+0000","last_change":"2023-12-20T04:40:59.947035+0000","last_active":"2023-12-20T04:40:59.957043+0000","last_peered":"2023-12-20T04:40:59.957043+0000","last_clean":"2023-12-20T04:40:59.957043+0000","last_became_active":"2023-12-20T04:40:59.946725+0000","last_became_peered":"2023-12-20T04:40:59.946725+0000","last_unstale":"2023-12-20T04:40:59.957043+0000","last_undegraded":"2023-12-20T04:40:59.957043+0000","last_fullsized":"2023-12-20T04:40:59.957043+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:32:00.691348+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.36","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966256+0000","last_change":"2023-12-20T04:40:59.964374+0000","last_active":"2023-12-20T04:40:59.966256+0000","last_peered":"2023-12-20T04:40:59.966256+0000","last_clean":"2023-12-20T04:40:59.966256+0000","last_became_active":"2023-12-20T04:40:59.964095+0000","last_became_peered":"2023-12-20T04:40:59.964095+0000","last_unstale":"2023-12-20T04:40:59.966256+0000","last_undegraded":"2023-12-20T04:40:59.966256+0000","last_fullsized":"2023-12-20T04:40:59.966256+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:00:37.983783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.35","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.953303+0000","last_change":"2023-12-20T04:40:59.942545+0000","last_active":"2023-12-20T04:40:59.953303+0000","last_peered":"2023-12-20T04:40:59.953303+0000","last_clean":"2023-12-20T04:40:59.953303+0000","last_became_active":"2023-12-20T04:40:59.942205+0000","last_became_peered":"2023-12-20T04:40:59.942205+0000","last_unstale":"2023-12-20T04:40:59.953303+0000","last_undegraded":"2023-12-20T04:40:59.953303+0000","last_fullsized":"2023-12-20T04:40:59.953303+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:12:25.173510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.34","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929933+0000","last_change":"2023-12-20T04:40:59.949515+0000","last_active":"2023-12-20T04:41:00.929933+0000","last_peered":"2023-12-20T04:41:00.929933+0000","last_clean":"2023-12-20T04:41:00.929933+0000","last_became_active":"2023-12-20T04:40:59.948482+0000","last_became_peered":"2023-12-20T04:40:59.948482+0000","last_unstale":"2023-12-20T04:41:00.929933+0000","last_undegraded":"2023-12-20T04:41:00.929933+0000","last_fullsized":"2023-12-20T04:41:00.929933+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:11:05.165428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.33","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927806+0000","last_change":"2023-12-20T04:40:59.955753+0000","last_active":"2023-12-20T04:41:00.927806+0000","last_peered":"2023-12-20T04:41:00.927806+0000","last_clean":"2023-12-20T04:41:00.927806+0000","last_became_active":"2023-12-20T04:40:59.955493+0000","last_became_peered":"2023-12-20T04:40:59.955493+0000","last_unstale":"2023-12-20T04:41:00.927806+0000","last_undegraded":"2023-12-20T04:41:00.927806+0000","last_fullsized":"2023-12-20T04:41:00.927806+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:00:46.881125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.32","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.953199+0000","last_change":"2023-12-20T04:40:59.942919+0000","last_active":"2023-12-20T04:40:59.953199+0000","last_peered":"2023-12-20T04:40:59.953199+0000","last_clean":"2023-12-20T04:40:59.953199+0000","last_became_active":"2023-12-20T04:40:59.942609+0000","last_became_peered":"2023-12-20T04:40:59.942609+0000","last_unstale":"2023-12-20T04:40:59.953199+0000","last_undegraded":"2023-12-20T04:40:59.953199+0000","last_fullsized":"2023-12-20T04:40:59.953199+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:05.391543+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.31","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259640+0000","last_change":"2023-12-20T04:40:59.954208+0000","last_active":"2023-12-20T04:41:01.259640+0000","last_peered":"2023-12-20T04:41:01.259640+0000","last_clean":"2023-12-20T04:41:01.259640+0000","last_became_active":"2023-12-20T04:40:59.953953+0000","last_became_peered":"2023-12-20T04:40:59.953953+0000","last_unstale":"2023-12-20T04:41:01.259640+0000","last_undegraded":"2023-12-20T04:41:01.259640+0000","last_fullsized":"2023-12-20T04:41:01.259640+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:32:17.808232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.30","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.964769+0000","last_change":"2023-12-20T04:40:59.959481+0000","last_active":"2023-12-20T04:40:59.964769+0000","last_peered":"2023-12-20T04:40:59.964769+0000","last_clean":"2023-12-20T04:40:59.964769+0000","last_became_active":"2023-12-20T04:40:59.958981+0000","last_became_peered":"2023-12-20T04:40:59.958981+0000","last_unstale":"2023-12-20T04:40:59.964769+0000","last_undegraded":"2023-12-20T04:40:59.964769+0000","last_fullsized":"2023-12-20T04:40:59.964769+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:52:01.969606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2f","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928230+0000","last_change":"2023-12-20T04:40:59.953605+0000","last_active":"2023-12-20T04:41:00.928230+0000","last_peered":"2023-12-20T04:41:00.928230+0000","last_clean":"2023-12-20T04:41:00.928230+0000","last_became_active":"2023-12-20T04:40:59.953199+0000","last_became_peered":"2023-12-20T04:40:59.953199+0000","last_unstale":"2023-12-20T04:41:00.928230+0000","last_undegraded":"2023-12-20T04:41:00.928230+0000","last_fullsized":"2023-12-20T04:41:00.928230+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:27.433632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928193+0000","last_change":"2023-12-20T04:40:59.956790+0000","last_active":"2023-12-20T04:41:00.928193+0000","last_peered":"2023-12-20T04:41:00.928193+0000","last_clean":"2023-12-20T04:41:00.928193+0000","last_became_active":"2023-12-20T04:40:59.956516+0000","last_became_peered":"2023-12-20T04:40:59.956516+0000","last_unstale":"2023-12-20T04:41:00.928193+0000","last_undegraded":"2023-12-20T04:41:00.928193+0000","last_fullsized":"2023-12-20T04:41:00.928193+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:43:16.243501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.2d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966456+0000","last_change":"2023-12-20T04:40:59.962564+0000","last_active":"2023-12-20T04:40:59.966456+0000","last_peered":"2023-12-20T04:40:59.966456+0000","last_clean":"2023-12-20T04:40:59.966456+0000","last_became_active":"2023-12-20T04:40:59.962252+0000","last_became_peered":"2023-12-20T04:40:59.962252+0000","last_unstale":"2023-12-20T04:40:59.966456+0000","last_undegraded":"2023-12-20T04:40:59.966456+0000","last_fullsized":"2023-12-20T04:40:59.966456+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:14:36.769806+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.2c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927980+0000","last_change":"2023-12-20T04:40:59.951480+0000","last_active":"2023-12-20T04:41:00.927980+0000","last_peered":"2023-12-20T04:41:00.927980+0000","last_clean":"2023-12-20T04:41:00.927980+0000","last_became_active":"2023-12-20T04:40:59.951214+0000","last_became_peered":"2023-12-20T04:40:59.951214+0000","last_unstale":"2023-12-20T04:41:00.927980+0000","last_undegraded":"2023-12-20T04:41:00.927980+0000","last_fullsized":"2023-12-20T04:41:00.927980+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:33:20.033497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.2b","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259749+0000","last_change":"2023-12-20T04:40:59.968026+0000","last_active":"2023-12-20T04:41:01.259749+0000","last_peered":"2023-12-20T04:41:01.259749+0000","last_clean":"2023-12-20T04:41:01.259749+0000","last_became_active":"2023-12-20T04:40:59.967679+0000","last_became_peered":"2023-12-20T04:40:59.967679+0000","last_unstale":"2023-12-20T04:41:01.259749+0000","last_undegraded":"2023-12-20T04:41:01.259749+0000","last_fullsized":"2023-12-20T04:41:01.259749+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:01.174229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.2a","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.968504+0000","last_change":"2023-12-20T04:40:59.967936+0000","last_active":"2023-12-20T04:40:59.968504+0000","last_peered":"2023-12-20T04:40:59.968504+0000","last_clean":"2023-12-20T04:40:59.968504+0000","last_became_active":"2023-12-20T04:40:59.967638+0000","last_became_peered":"2023-12-20T04:40:59.967638+0000","last_unstale":"2023-12-20T04:40:59.968504+0000","last_undegraded":"2023-12-20T04:40:59.968504+0000","last_fullsized":"2023-12-20T04:40:59.968504+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:01.467255+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.29","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966404+0000","last_change":"2023-12-20T04:40:59.962694+0000","last_active":"2023-12-20T04:40:59.966404+0000","last_peered":"2023-12-20T04:40:59.966404+0000","last_clean":"2023-12-20T04:40:59.966404+0000","last_became_active":"2023-12-20T04:40:59.962343+0000","last_became_peered":"2023-12-20T04:40:59.962343+0000","last_unstale":"2023-12-20T04:40:59.966404+0000","last_undegraded":"2023-12-20T04:40:59.966404+0000","last_fullsized":"2023-12-20T04:40:59.966404+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:00:46.881125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.28","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.964792+0000","last_change":"2023-12-20T04:40:59.959782+0000","last_active":"2023-12-20T04:40:59.964792+0000","last_peered":"2023-12-20T04:40:59.964792+0000","last_clean":"2023-12-20T04:40:59.964792+0000","last_became_active":"2023-12-20T04:40:59.959426+0000","last_became_peered":"2023-12-20T04:40:59.959426+0000","last_unstale":"2023-12-20T04:40:59.964792+0000","last_undegraded":"2023-12-20T04:40:59.964792+0000","last_fullsized":"2023-12-20T04:40:59.964792+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:05:05.313155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.27","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.964849+0000","last_change":"2023-12-20T04:40:59.960365+0000","last_active":"2023-12-20T04:40:59.964849+0000","last_peered":"2023-12-20T04:40:59.964849+0000","last_clean":"2023-12-20T04:40:59.964849+0000","last_became_active":"2023-12-20T04:40:59.959117+0000","last_became_peered":"2023-12-20T04:40:59.959117+0000","last_unstale":"2023-12-20T04:40:59.964849+0000","last_undegraded":"2023-12-20T04:40:59.964849+0000","last_fullsized":"2023-12-20T04:40:59.964849+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:38:04.392962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.26","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927866+0000","last_change":"2023-12-20T04:40:59.955893+0000","last_active":"2023-12-20T04:41:00.927866+0000","last_peered":"2023-12-20T04:41:00.927866+0000","last_clean":"2023-12-20T04:41:00.927866+0000","last_became_active":"2023-12-20T04:40:59.955591+0000","last_became_peered":"2023-12-20T04:40:59.955591+0000","last_unstale":"2023-12-20T04:41:00.927866+0000","last_undegraded":"2023-12-20T04:41:00.927866+0000","last_fullsized":"2023-12-20T04:41:00.927866+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:09:00.207233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.25","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928098+0000","last_change":"2023-12-20T04:40:59.955297+0000","last_active":"2023-12-20T04:41:00.928098+0000","last_peered":"2023-12-20T04:41:00.928098+0000","last_clean":"2023-12-20T04:41:00.928098+0000","last_became_active":"2023-12-20T04:40:59.954028+0000","last_became_peered":"2023-12-20T04:40:59.954028+0000","last_unstale":"2023-12-20T04:41:00.928098+0000","last_undegraded":"2023-12-20T04:41:00.928098+0000","last_fullsized":"2023-12-20T04:41:00.928098+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:51.240859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.24","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929957+0000","last_change":"2023-12-20T04:40:59.943197+0000","last_active":"2023-12-20T04:41:00.929957+0000","last_peered":"2023-12-20T04:41:00.929957+0000","last_clean":"2023-12-20T04:41:00.929957+0000","last_became_active":"2023-12-20T04:40:59.942874+0000","last_became_peered":"2023-12-20T04:40:59.942874+0000","last_unstale":"2023-12-20T04:41:00.929957+0000","last_undegraded":"2023-12-20T04:41:00.929957+0000","last_fullsized":"2023-12-20T04:41:00.929957+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:16:42.925904+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.23","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.967289+0000","last_change":"2023-12-20T04:40:59.964866+0000","last_active":"2023-12-20T04:40:59.967289+0000","last_peered":"2023-12-20T04:40:59.967289+0000","last_clean":"2023-12-20T04:40:59.967289+0000","last_became_active":"2023-12-20T04:40:59.964568+0000","last_became_peered":"2023-12-20T04:40:59.964568+0000","last_unstale":"2023-12-20T04:40:59.967289+0000","last_undegraded":"2023-12-20T04:40:59.967289+0000","last_fullsized":"2023-12-20T04:40:59.967289+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:45.584995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"7.22","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927946+0000","last_change":"2023-12-20T04:40:59.954287+0000","last_active":"2023-12-20T04:41:00.927946+0000","last_peered":"2023-12-20T04:41:00.927946+0000","last_clean":"2023-12-20T04:41:00.927946+0000","last_became_active":"2023-12-20T04:40:59.953065+0000","last_became_peered":"2023-12-20T04:40:59.953065+0000","last_unstale":"2023-12-20T04:41:00.927946+0000","last_undegraded":"2023-12-20T04:41:00.927946+0000","last_fullsized":"2023-12-20T04:41:00.927946+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:52:07.362486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.21","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259709+0000","last_change":"2023-12-20T04:40:59.952866+0000","last_active":"2023-12-20T04:41:01.259709+0000","last_peered":"2023-12-20T04:41:01.259709+0000","last_clean":"2023-12-20T04:41:01.259709+0000","last_became_active":"2023-12-20T04:40:59.952032+0000","last_became_peered":"2023-12-20T04:40:59.952032+0000","last_unstale":"2023-12-20T04:41:01.259709+0000","last_undegraded":"2023-12-20T04:41:01.259709+0000","last_fullsized":"2023-12-20T04:41:01.259709+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:30:10.242145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.20","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927665+0000","last_change":"2023-12-20T04:40:59.940706+0000","last_active":"2023-12-20T04:41:00.927665+0000","last_peered":"2023-12-20T04:41:00.927665+0000","last_clean":"2023-12-20T04:41:00.927665+0000","last_became_active":"2023-12-20T04:40:59.940362+0000","last_became_peered":"2023-12-20T04:40:59.940362+0000","last_unstale":"2023-12-20T04:41:00.927665+0000","last_undegraded":"2023-12-20T04:41:00.927665+0000","last_fullsized":"2023-12-20T04:41:00.927665+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:05:05.313155+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.1e","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.955693+0000","last_change":"2023-12-20T04:40:59.947838+0000","last_active":"2023-12-20T04:40:59.955693+0000","last_peered":"2023-12-20T04:40:59.955693+0000","last_clean":"2023-12-20T04:40:59.955693+0000","last_became_active":"2023-12-20T04:40:59.947367+0000","last_became_peered":"2023-12-20T04:40:59.947367+0000","last_unstale":"2023-12-20T04:40:59.955693+0000","last_undegraded":"2023-12-20T04:40:59.955693+0000","last_fullsized":"2023-12-20T04:40:59.955693+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:25:19.139121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1f","version":"60'4","reported_seq":15,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.260064+0000","last_change":"2023-12-20T04:40:59.546885+0000","last_active":"2023-12-20T04:41:01.260064+0000","last_peered":"2023-12-20T04:41:01.260064+0000","last_clean":"2023-12-20T04:41:01.260064+0000","last_became_active":"2023-12-20T04:40:59.546555+0000","last_became_peered":"2023-12-20T04:40:59.546555+0000","last_unstale":"2023-12-20T04:41:01.260064+0000","last_undegraded":"2023-12-20T04:41:01.260064+0000","last_fullsized":"2023-12-20T04:41:01.260064+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:01.459895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966303+0000","last_change":"2023-12-20T04:40:59.962424+0000","last_active":"2023-12-20T04:40:59.966303+0000","last_peered":"2023-12-20T04:40:59.966303+0000","last_clean":"2023-12-20T04:40:59.966303+0000","last_became_active":"2023-12-20T04:40:59.962106+0000","last_became_peered":"2023-12-20T04:40:59.962106+0000","last_unstale":"2023-12-20T04:40:59.966303+0000","last_undegraded":"2023-12-20T04:40:59.966303+0000","last_fullsized":"2023-12-20T04:40:59.966303+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:33:20.033497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.1e","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928280+0000","last_change":"2023-12-20T04:40:59.545033+0000","last_active":"2023-12-20T04:41:00.928280+0000","last_peered":"2023-12-20T04:41:00.928280+0000","last_clean":"2023-12-20T04:41:00.928280+0000","last_became_active":"2023-12-20T04:40:59.544671+0000","last_became_peered":"2023-12-20T04:40:59.544671+0000","last_unstale":"2023-12-20T04:41:00.928280+0000","last_undegraded":"2023-12-20T04:41:00.928280+0000","last_fullsized":"2023-12-20T04:41:00.928280+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:25:18.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.1c","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.930032+0000","last_change":"2023-12-20T04:40:59.950545+0000","last_active":"2023-12-20T04:41:00.930032+0000","last_peered":"2023-12-20T04:41:00.930032+0000","last_clean":"2023-12-20T04:41:00.930032+0000","last_became_active":"2023-12-20T04:40:59.949051+0000","last_became_peered":"2023-12-20T04:40:59.949051+0000","last_unstale":"2023-12-20T04:41:00.930032+0000","last_undegraded":"2023-12-20T04:41:00.930032+0000","last_fullsized":"2023-12-20T04:41:00.930032+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:03:02.302456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1d","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.948084+0000","last_change":"2023-12-20T04:40:59.547531+0000","last_active":"2023-12-20T04:40:59.948084+0000","last_peered":"2023-12-20T04:40:59.948084+0000","last_clean":"2023-12-20T04:40:59.948084+0000","last_became_active":"2023-12-20T04:40:59.546128+0000","last_became_peered":"2023-12-20T04:40:59.546128+0000","last_unstale":"2023-12-20T04:40:59.948084+0000","last_undegraded":"2023-12-20T04:40:59.948084+0000","last_fullsized":"2023-12-20T04:40:59.948084+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:28.929814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.967608+0000","last_change":"2023-12-20T04:40:59.966631+0000","last_active":"2023-12-20T04:40:59.967608+0000","last_peered":"2023-12-20T04:40:59.967608+0000","last_clean":"2023-12-20T04:40:59.967608+0000","last_became_active":"2023-12-20T04:40:59.966323+0000","last_became_peered":"2023-12-20T04:40:59.966323+0000","last_unstale":"2023-12-20T04:40:59.967608+0000","last_undegraded":"2023-12-20T04:40:59.967608+0000","last_fullsized":"2023-12-20T04:40:59.967608+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:37:49.708430+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.1c","version":"60'2","reported_seq":12,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.951423+0000","last_change":"2023-12-20T04:40:59.549225+0000","last_active":"2023-12-20T04:40:59.951423+0000","last_peered":"2023-12-20T04:40:59.951423+0000","last_clean":"2023-12-20T04:40:59.951423+0000","last_became_active":"2023-12-20T04:40:59.548921+0000","last_became_peered":"2023-12-20T04:40:59.548921+0000","last_unstale":"2023-12-20T04:40:59.951423+0000","last_undegraded":"2023-12-20T04:40:59.951423+0000","last_fullsized":"2023-12-20T04:40:59.951423+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:37:48.689217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.1a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928017+0000","last_change":"2023-12-20T04:40:59.965437+0000","last_active":"2023-12-20T04:41:00.928017+0000","last_peered":"2023-12-20T04:41:00.928017+0000","last_clean":"2023-12-20T04:41:00.928017+0000","last_became_active":"2023-12-20T04:40:59.962778+0000","last_became_peered":"2023-12-20T04:40:59.962778+0000","last_unstale":"2023-12-20T04:41:00.928017+0000","last_undegraded":"2023-12-20T04:41:00.928017+0000","last_fullsized":"2023-12-20T04:41:00.928017+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:04.693874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.1b","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259851+0000","last_change":"2023-12-20T04:40:59.541677+0000","last_active":"2023-12-20T04:41:01.259851+0000","last_peered":"2023-12-20T04:41:01.259851+0000","last_clean":"2023-12-20T04:41:01.259851+0000","last_became_active":"2023-12-20T04:40:59.541386+0000","last_became_peered":"2023-12-20T04:40:59.541386+0000","last_unstale":"2023-12-20T04:41:01.259851+0000","last_undegraded":"2023-12-20T04:41:01.259851+0000","last_fullsized":"2023-12-20T04:41:01.259851+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:43:15.224288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.1b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957243+0000","last_change":"2023-12-20T04:40:59.949214+0000","last_active":"2023-12-20T04:40:59.957243+0000","last_peered":"2023-12-20T04:40:59.957243+0000","last_clean":"2023-12-20T04:40:59.957243+0000","last_became_active":"2023-12-20T04:40:59.948204+0000","last_became_peered":"2023-12-20T04:40:59.948204+0000","last_unstale":"2023-12-20T04:40:59.957243+0000","last_undegraded":"2023-12-20T04:40:59.957243+0000","last_fullsized":"2023-12-20T04:40:59.957243+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:11:54.734047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.1a","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928063+0000","last_change":"2023-12-20T04:40:59.542954+0000","last_active":"2023-12-20T04:41:00.928063+0000","last_peered":"2023-12-20T04:41:00.928063+0000","last_clean":"2023-12-20T04:41:00.928063+0000","last_became_active":"2023-12-20T04:40:59.541023+0000","last_became_peered":"2023-12-20T04:40:59.541023+0000","last_unstale":"2023-12-20T04:41:00.928063+0000","last_undegraded":"2023-12-20T04:41:00.928063+0000","last_fullsized":"2023-12-20T04:41:00.928063+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:21:17.066394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.18","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.965504+0000","last_change":"2023-12-20T04:40:59.964322+0000","last_active":"2023-12-20T04:40:59.965504+0000","last_peered":"2023-12-20T04:40:59.965504+0000","last_clean":"2023-12-20T04:40:59.965504+0000","last_became_active":"2023-12-20T04:40:59.964033+0000","last_became_peered":"2023-12-20T04:40:59.964033+0000","last_unstale":"2023-12-20T04:40:59.965504+0000","last_undegraded":"2023-12-20T04:40:59.965504+0000","last_fullsized":"2023-12-20T04:40:59.965504+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:11:42.857428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.19","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929798+0000","last_change":"2023-12-20T04:40:59.543696+0000","last_active":"2023-12-20T04:41:00.929798+0000","last_peered":"2023-12-20T04:41:00.929798+0000","last_clean":"2023-12-20T04:41:00.929798+0000","last_became_active":"2023-12-20T04:40:59.540121+0000","last_became_peered":"2023-12-20T04:40:59.540121+0000","last_unstale":"2023-12-20T04:41:00.929798+0000","last_undegraded":"2023-12-20T04:41:00.929798+0000","last_fullsized":"2023-12-20T04:41:00.929798+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:43:20.804066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.19","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966425+0000","last_change":"2023-12-20T04:40:59.962756+0000","last_active":"2023-12-20T04:40:59.966425+0000","last_peered":"2023-12-20T04:40:59.966425+0000","last_clean":"2023-12-20T04:40:59.966425+0000","last_became_active":"2023-12-20T04:40:59.962286+0000","last_became_peered":"2023-12-20T04:40:59.962286+0000","last_unstale":"2023-12-20T04:40:59.966425+0000","last_undegraded":"2023-12-20T04:40:59.966425+0000","last_fullsized":"2023-12-20T04:40:59.966425+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:09:00.207233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.18","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.936883+0000","last_change":"2023-12-20T04:40:59.534257+0000","last_active":"2023-12-20T04:40:59.936883+0000","last_peered":"2023-12-20T04:40:59.936883+0000","last_clean":"2023-12-20T04:40:59.936883+0000","last_became_active":"2023-12-20T04:40:59.533910+0000","last_became_peered":"2023-12-20T04:40:59.533910+0000","last_unstale":"2023-12-20T04:40:59.936883+0000","last_undegraded":"2023-12-20T04:40:59.936883+0000","last_fullsized":"2023-12-20T04:40:59.936883+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:25:18.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.16","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928264+0000","last_change":"2023-12-20T04:40:59.958070+0000","last_active":"2023-12-20T04:41:00.928264+0000","last_peered":"2023-12-20T04:41:00.928264+0000","last_clean":"2023-12-20T04:41:00.928264+0000","last_became_active":"2023-12-20T04:40:59.957793+0000","last_became_peered":"2023-12-20T04:40:59.957793+0000","last_unstale":"2023-12-20T04:41:00.928264+0000","last_undegraded":"2023-12-20T04:41:00.928264+0000","last_fullsized":"2023-12-20T04:41:00.928264+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:10.439930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.17","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928281+0000","last_change":"2023-12-20T04:40:59.537199+0000","last_active":"2023-12-20T04:41:00.928281+0000","last_peered":"2023-12-20T04:41:00.928281+0000","last_clean":"2023-12-20T04:41:00.928281+0000","last_became_active":"2023-12-20T04:40:59.536871+0000","last_became_peered":"2023-12-20T04:40:59.536871+0000","last_unstale":"2023-12-20T04:41:00.928281+0000","last_undegraded":"2023-12-20T04:41:00.928281+0000","last_fullsized":"2023-12-20T04:41:00.928281+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:39:48.794770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.17","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957153+0000","last_change":"2023-12-20T04:40:59.948780+0000","last_active":"2023-12-20T04:40:59.957153+0000","last_peered":"2023-12-20T04:40:59.957153+0000","last_clean":"2023-12-20T04:40:59.957153+0000","last_became_active":"2023-12-20T04:40:59.948494+0000","last_became_peered":"2023-12-20T04:40:59.948494+0000","last_unstale":"2023-12-20T04:40:59.957153+0000","last_undegraded":"2023-12-20T04:40:59.957153+0000","last_fullsized":"2023-12-20T04:40:59.957153+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:51.240859+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.16","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.937510+0000","last_change":"2023-12-20T04:40:59.546225+0000","last_active":"2023-12-20T04:40:59.937510+0000","last_peered":"2023-12-20T04:40:59.937510+0000","last_clean":"2023-12-20T04:40:59.937510+0000","last_became_active":"2023-12-20T04:40:59.545871+0000","last_became_peered":"2023-12-20T04:40:59.545871+0000","last_unstale":"2023-12-20T04:40:59.937510+0000","last_undegraded":"2023-12-20T04:40:59.937510+0000","last_fullsized":"2023-12-20T04:40:59.937510+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:53.616685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.14","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.260042+0000","last_change":"2023-12-20T04:40:59.941167+0000","last_active":"2023-12-20T04:41:01.260042+0000","last_peered":"2023-12-20T04:41:01.260042+0000","last_clean":"2023-12-20T04:41:01.260042+0000","last_became_active":"2023-12-20T04:40:59.940889+0000","last_became_peered":"2023-12-20T04:40:59.940889+0000","last_unstale":"2023-12-20T04:41:01.260042+0000","last_undegraded":"2023-12-20T04:41:01.260042+0000","last_fullsized":"2023-12-20T04:41:01.260042+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:39:33.817277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.15","version":"60'3","reported_seq":13,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.960337+0000","last_change":"2023-12-20T04:40:59.548265+0000","last_active":"2023-12-20T04:40:59.960337+0000","last_peered":"2023-12-20T04:40:59.960337+0000","last_clean":"2023-12-20T04:40:59.960337+0000","last_became_active":"2023-12-20T04:40:59.547975+0000","last_became_peered":"2023-12-20T04:40:59.547975+0000","last_unstale":"2023-12-20T04:40:59.960337+0000","last_undegraded":"2023-12-20T04:40:59.960337+0000","last_fullsized":"2023-12-20T04:40:59.960337+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:09:33.906588+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.15","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.956915+0000","last_change":"2023-12-20T04:40:59.947855+0000","last_active":"2023-12-20T04:40:59.956915+0000","last_peered":"2023-12-20T04:40:59.956915+0000","last_clean":"2023-12-20T04:40:59.956915+0000","last_became_active":"2023-12-20T04:40:59.947545+0000","last_became_peered":"2023-12-20T04:40:59.947545+0000","last_unstale":"2023-12-20T04:40:59.956915+0000","last_undegraded":"2023-12-20T04:40:59.956915+0000","last_fullsized":"2023-12-20T04:40:59.956915+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:10.439930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.14","version":"60'6","reported_seq":17,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927990+0000","last_change":"2023-12-20T04:40:59.533939+0000","last_active":"2023-12-20T04:41:00.927990+0000","last_peered":"2023-12-20T04:41:00.927990+0000","last_clean":"2023-12-20T04:41:00.927990+0000","last_became_active":"2023-12-20T04:40:59.533674+0000","last_became_peered":"2023-12-20T04:40:59.533674+0000","last_unstale":"2023-12-20T04:41:00.927990+0000","last_undegraded":"2023-12-20T04:41:00.927990+0000","last_fullsized":"2023-12-20T04:41:00.927990+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:55:03.873417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.12","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.965057+0000","last_change":"2023-12-20T04:40:59.960645+0000","last_active":"2023-12-20T04:40:59.965057+0000","last_peered":"2023-12-20T04:40:59.965057+0000","last_clean":"2023-12-20T04:40:59.965057+0000","last_became_active":"2023-12-20T04:40:59.958925+0000","last_became_peered":"2023-12-20T04:40:59.958925+0000","last_unstale":"2023-12-20T04:40:59.965057+0000","last_undegraded":"2023-12-20T04:40:59.965057+0000","last_fullsized":"2023-12-20T04:40:59.965057+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:54.303149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.13","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259847+0000","last_change":"2023-12-20T04:40:59.537954+0000","last_active":"2023-12-20T04:41:01.259847+0000","last_peered":"2023-12-20T04:41:01.259847+0000","last_clean":"2023-12-20T04:41:01.259847+0000","last_became_active":"2023-12-20T04:40:59.537602+0000","last_became_peered":"2023-12-20T04:40:59.537602+0000","last_unstale":"2023-12-20T04:41:01.259847+0000","last_undegraded":"2023-12-20T04:41:01.259847+0000","last_fullsized":"2023-12-20T04:41:01.259847+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:09.420717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.13","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957262+0000","last_change":"2023-12-20T04:40:59.949788+0000","last_active":"2023-12-20T04:40:59.957262+0000","last_peered":"2023-12-20T04:40:59.957262+0000","last_clean":"2023-12-20T04:40:59.957262+0000","last_became_active":"2023-12-20T04:40:59.949479+0000","last_became_peered":"2023-12-20T04:40:59.949479+0000","last_unstale":"2023-12-20T04:40:59.957262+0000","last_undegraded":"2023-12-20T04:40:59.957262+0000","last_fullsized":"2023-12-20T04:40:59.957262+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:12:52.553009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.12","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.936142+0000","last_change":"2023-12-20T04:40:59.545764+0000","last_active":"2023-12-20T04:40:59.936142+0000","last_peered":"2023-12-20T04:40:59.936142+0000","last_clean":"2023-12-20T04:40:59.936142+0000","last_became_active":"2023-12-20T04:40:59.545471+0000","last_became_peered":"2023-12-20T04:40:59.545471+0000","last_unstale":"2023-12-20T04:40:59.936142+0000","last_undegraded":"2023-12-20T04:40:59.936142+0000","last_fullsized":"2023-12-20T04:40:59.936142+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:25:18.119908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.10","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259623+0000","last_change":"2023-12-20T04:40:59.937954+0000","last_active":"2023-12-20T04:41:01.259623+0000","last_peered":"2023-12-20T04:41:01.259623+0000","last_clean":"2023-12-20T04:41:01.259623+0000","last_became_active":"2023-12-20T04:40:59.937753+0000","last_became_peered":"2023-12-20T04:40:59.937753+0000","last_unstale":"2023-12-20T04:41:01.259623+0000","last_undegraded":"2023-12-20T04:41:01.259623+0000","last_fullsized":"2023-12-20T04:41:01.259623+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:21:09.542330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.11","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259747+0000","last_change":"2023-12-20T04:40:59.539018+0000","last_active":"2023-12-20T04:41:01.259747+0000","last_peered":"2023-12-20T04:41:01.259747+0000","last_clean":"2023-12-20T04:41:01.259747+0000","last_became_active":"2023-12-20T04:40:59.538738+0000","last_became_peered":"2023-12-20T04:40:59.538738+0000","last_unstale":"2023-12-20T04:41:01.259747+0000","last_undegraded":"2023-12-20T04:41:01.259747+0000","last_fullsized":"2023-12-20T04:41:01.259747+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:52:06.343273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.11","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966164+0000","last_change":"2023-12-20T04:40:59.964614+0000","last_active":"2023-12-20T04:40:59.966164+0000","last_peered":"2023-12-20T04:40:59.966164+0000","last_clean":"2023-12-20T04:40:59.966164+0000","last_became_active":"2023-12-20T04:40:59.964159+0000","last_became_peered":"2023-12-20T04:40:59.964159+0000","last_unstale":"2023-12-20T04:40:59.966164+0000","last_undegraded":"2023-12-20T04:40:59.966164+0000","last_fullsized":"2023-12-20T04:40:59.966164+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:04.693874+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.10","version":"60'8","reported_seq":18,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.958852+0000","last_change":"2023-12-20T04:40:59.541090+0000","last_active":"2023-12-20T04:40:59.958852+0000","last_peered":"2023-12-20T04:40:59.958852+0000","last_clean":"2023-12-20T04:40:59.958852+0000","last_became_active":"2023-12-20T04:40:59.540807+0000","last_became_peered":"2023-12-20T04:40:59.540807+0000","last_unstale":"2023-12-20T04:40:59.958852+0000","last_undegraded":"2023-12-20T04:40:59.958852+0000","last_fullsized":"2023-12-20T04:40:59.958852+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:37:48.689217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.e","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928002+0000","last_change":"2023-12-20T04:40:59.949117+0000","last_active":"2023-12-20T04:41:00.928002+0000","last_peered":"2023-12-20T04:41:00.928002+0000","last_clean":"2023-12-20T04:41:00.928002+0000","last_became_active":"2023-12-20T04:40:59.948845+0000","last_became_peered":"2023-12-20T04:40:59.948845+0000","last_unstale":"2023-12-20T04:41:00.928002+0000","last_undegraded":"2023-12-20T04:41:00.928002+0000","last_fullsized":"2023-12-20T04:41:00.928002+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:29:43.957431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.f","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928081+0000","last_change":"2023-12-20T04:40:59.539575+0000","last_active":"2023-12-20T04:41:00.928081+0000","last_peered":"2023-12-20T04:41:00.928081+0000","last_clean":"2023-12-20T04:41:00.928081+0000","last_became_active":"2023-12-20T04:40:59.539130+0000","last_became_peered":"2023-12-20T04:40:59.539130+0000","last_unstale":"2023-12-20T04:41:00.928081+0000","last_undegraded":"2023-12-20T04:41:00.928081+0000","last_fullsized":"2023-12-20T04:41:00.928081+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:20.780219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"7.f","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957405+0000","last_change":"2023-12-20T04:40:59.953804+0000","last_active":"2023-12-20T04:40:59.957405+0000","last_peered":"2023-12-20T04:40:59.957405+0000","last_clean":"2023-12-20T04:40:59.957405+0000","last_became_active":"2023-12-20T04:40:59.953548+0000","last_became_peered":"2023-12-20T04:40:59.953548+0000","last_unstale":"2023-12-20T04:40:59.957405+0000","last_undegraded":"2023-12-20T04:40:59.957405+0000","last_fullsized":"2023-12-20T04:40:59.957405+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:27.433632+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.e","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928316+0000","last_change":"2023-12-20T04:40:59.542677+0000","last_active":"2023-12-20T04:41:00.928316+0000","last_peered":"2023-12-20T04:41:00.928316+0000","last_clean":"2023-12-20T04:41:00.928316+0000","last_became_active":"2023-12-20T04:40:59.542347+0000","last_became_peered":"2023-12-20T04:40:59.542347+0000","last_unstale":"2023-12-20T04:41:00.928316+0000","last_undegraded":"2023-12-20T04:41:00.928316+0000","last_fullsized":"2023-12-20T04:41:00.928316+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:37:48.689217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.c","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.964878+0000","last_change":"2023-12-20T04:40:59.960527+0000","last_active":"2023-12-20T04:40:59.964878+0000","last_peered":"2023-12-20T04:40:59.964878+0000","last_clean":"2023-12-20T04:40:59.964878+0000","last_became_active":"2023-12-20T04:40:59.959321+0000","last_became_peered":"2023-12-20T04:40:59.959321+0000","last_unstale":"2023-12-20T04:40:59.964878+0000","last_undegraded":"2023-12-20T04:40:59.964878+0000","last_fullsized":"2023-12-20T04:40:59.964878+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:41:18.708635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.d","version":"60'1","reported_seq":12,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.930074+0000","last_change":"2023-12-20T04:40:59.540201+0000","last_active":"2023-12-20T04:41:00.930074+0000","last_peered":"2023-12-20T04:41:00.930074+0000","last_clean":"2023-12-20T04:41:00.930074+0000","last_became_active":"2023-12-20T04:40:59.539901+0000","last_became_peered":"2023-12-20T04:40:59.539901+0000","last_unstale":"2023-12-20T04:41:00.930074+0000","last_undegraded":"2023-12-20T04:41:00.930074+0000","last_fullsized":"2023-12-20T04:41:00.930074+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:21:17.066394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.d","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.966475+0000","last_change":"2023-12-20T04:40:59.962635+0000","last_active":"2023-12-20T04:40:59.966475+0000","last_peered":"2023-12-20T04:40:59.966475+0000","last_clean":"2023-12-20T04:40:59.966475+0000","last_became_active":"2023-12-20T04:40:59.962339+0000","last_became_peered":"2023-12-20T04:40:59.962339+0000","last_unstale":"2023-12-20T04:40:59.966475+0000","last_undegraded":"2023-12-20T04:40:59.966475+0000","last_fullsized":"2023-12-20T04:40:59.966475+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:29.949027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.c","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259955+0000","last_change":"2023-12-20T04:40:59.546841+0000","last_active":"2023-12-20T04:41:01.259955+0000","last_peered":"2023-12-20T04:41:01.259955+0000","last_clean":"2023-12-20T04:41:01.259955+0000","last_became_active":"2023-12-20T04:40:59.546550+0000","last_became_peered":"2023-12-20T04:40:59.546550+0000","last_unstale":"2023-12-20T04:41:01.259955+0000","last_undegraded":"2023-12-20T04:41:01.259955+0000","last_fullsized":"2023-12-20T04:41:01.259955+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:50.221646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.a","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259798+0000","last_change":"2023-12-20T04:40:59.937996+0000","last_active":"2023-12-20T04:41:01.259798+0000","last_peered":"2023-12-20T04:41:01.259798+0000","last_clean":"2023-12-20T04:41:01.259798+0000","last_became_active":"2023-12-20T04:40:59.937730+0000","last_became_peered":"2023-12-20T04:40:59.937730+0000","last_unstale":"2023-12-20T04:41:01.259798+0000","last_undegraded":"2023-12-20T04:41:01.259798+0000","last_fullsized":"2023-12-20T04:41:01.259798+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:00:37.983783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.b","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.260039+0000","last_change":"2023-12-20T04:40:59.540776+0000","last_active":"2023-12-20T04:41:01.260039+0000","last_peered":"2023-12-20T04:41:01.260039+0000","last_clean":"2023-12-20T04:41:01.260039+0000","last_became_active":"2023-12-20T04:40:59.540500+0000","last_became_peered":"2023-12-20T04:40:59.540500+0000","last_unstale":"2023-12-20T04:41:01.260039+0000","last_undegraded":"2023-12-20T04:41:01.260039+0000","last_fullsized":"2023-12-20T04:41:01.260039+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:02:03.674661+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.b","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.955870+0000","last_change":"2023-12-20T04:40:59.946041+0000","last_active":"2023-12-20T04:40:59.955870+0000","last_peered":"2023-12-20T04:40:59.955870+0000","last_clean":"2023-12-20T04:40:59.955870+0000","last_became_active":"2023-12-20T04:40:59.945794+0000","last_became_peered":"2023-12-20T04:40:59.945794+0000","last_unstale":"2023-12-20T04:40:59.955870+0000","last_undegraded":"2023-12-20T04:40:59.955870+0000","last_fullsized":"2023-12-20T04:40:59.955870+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:54.635898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"6.a","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929827+0000","last_change":"2023-12-20T04:40:59.546551+0000","last_active":"2023-12-20T04:41:00.929827+0000","last_peered":"2023-12-20T04:41:00.929827+0000","last_clean":"2023-12-20T04:41:00.929827+0000","last_became_active":"2023-12-20T04:40:59.546253+0000","last_became_peered":"2023-12-20T04:40:59.546253+0000","last_unstale":"2023-12-20T04:41:00.929827+0000","last_undegraded":"2023-12-20T04:41:00.929827+0000","last_fullsized":"2023-12-20T04:41:00.929827+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:11:34.042110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"7.8","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927637+0000","last_change":"2023-12-20T04:40:59.950739+0000","last_active":"2023-12-20T04:41:00.927637+0000","last_peered":"2023-12-20T04:41:00.927637+0000","last_clean":"2023-12-20T04:41:00.927637+0000","last_became_active":"2023-12-20T04:40:59.950432+0000","last_became_peered":"2023-12-20T04:40:59.950432+0000","last_unstale":"2023-12-20T04:41:00.927637+0000","last_undegraded":"2023-12-20T04:41:00.927637+0000","last_fullsized":"2023-12-20T04:41:00.927637+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:29.949027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"6.9","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.933943+0000","last_change":"2023-12-20T04:40:59.548084+0000","last_active":"2023-12-20T04:40:59.933943+0000","last_peered":"2023-12-20T04:40:59.933943+0000","last_clean":"2023-12-20T04:40:59.933943+0000","last_became_active":"2023-12-20T04:40:59.547209+0000","last_became_peered":"2023-12-20T04:40:59.547209+0000","last_unstale":"2023-12-20T04:40:59.933943+0000","last_undegraded":"2023-12-20T04:40:59.933943+0000","last_fullsized":"2023-12-20T04:40:59.933943+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:21:17.066394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.9","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.953876+0000","last_change":"2023-12-20T04:40:59.943076+0000","last_active":"2023-12-20T04:40:59.953876+0000","last_peered":"2023-12-20T04:40:59.953876+0000","last_clean":"2023-12-20T04:40:59.953876+0000","last_became_active":"2023-12-20T04:40:59.942193+0000","last_became_peered":"2023-12-20T04:40:59.942193+0000","last_unstale":"2023-12-20T04:40:59.953876+0000","last_undegraded":"2023-12-20T04:40:59.953876+0000","last_fullsized":"2023-12-20T04:40:59.953876+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:25.915476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.8","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.935358+0000","last_change":"2023-12-20T04:40:59.543512+0000","last_active":"2023-12-20T04:40:59.935358+0000","last_peered":"2023-12-20T04:40:59.935358+0000","last_clean":"2023-12-20T04:40:59.935358+0000","last_became_active":"2023-12-20T04:40:59.542872+0000","last_became_peered":"2023-12-20T04:40:59.542872+0000","last_unstale":"2023-12-20T04:40:59.935358+0000","last_undegraded":"2023-12-20T04:40:59.935358+0000","last_fullsized":"2023-12-20T04:40:59.935358+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:53.616685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.7","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.965087+0000","last_change":"2023-12-20T04:40:59.960781+0000","last_active":"2023-12-20T04:40:59.965087+0000","last_peered":"2023-12-20T04:40:59.965087+0000","last_clean":"2023-12-20T04:40:59.965087+0000","last_became_active":"2023-12-20T04:40:59.959632+0000","last_became_peered":"2023-12-20T04:40:59.959632+0000","last_unstale":"2023-12-20T04:40:59.965087+0000","last_undegraded":"2023-12-20T04:40:59.965087+0000","last_fullsized":"2023-12-20T04:40:59.965087+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:29:43.957431+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"6.6","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.260003+0000","last_change":"2023-12-20T04:40:59.545551+0000","last_active":"2023-12-20T04:41:01.260003+0000","last_peered":"2023-12-20T04:41:01.260003+0000","last_clean":"2023-12-20T04:41:01.260003+0000","last_became_active":"2023-12-20T04:40:59.545317+0000","last_became_peered":"2023-12-20T04:40:59.545317+0000","last_unstale":"2023-12-20T04:41:01.260003+0000","last_undegraded":"2023-12-20T04:41:01.260003+0000","last_fullsized":"2023-12-20T04:41:01.260003+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:26.414419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"7.4","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957268+0000","last_change":"2023-12-20T04:40:59.949069+0000","last_active":"2023-12-20T04:40:59.957268+0000","last_peered":"2023-12-20T04:40:59.957268+0000","last_clean":"2023-12-20T04:40:59.957268+0000","last_became_active":"2023-12-20T04:40:59.948808+0000","last_became_peered":"2023-12-20T04:40:59.948808+0000","last_unstale":"2023-12-20T04:40:59.957268+0000","last_undegraded":"2023-12-20T04:40:59.957268+0000","last_fullsized":"2023-12-20T04:40:59.957268+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:02.479108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.5","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.936666+0000","last_change":"2023-12-20T04:40:59.549228+0000","last_active":"2023-12-20T04:40:59.936666+0000","last_peered":"2023-12-20T04:40:59.936666+0000","last_clean":"2023-12-20T04:40:59.936666+0000","last_became_active":"2023-12-20T04:40:59.548875+0000","last_became_peered":"2023-12-20T04:40:59.548875+0000","last_unstale":"2023-12-20T04:40:59.936666+0000","last_undegraded":"2023-12-20T04:40:59.936666+0000","last_fullsized":"2023-12-20T04:40:59.936666+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:52:06.756273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.5","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.956202+0000","last_change":"2023-12-20T04:40:59.947240+0000","last_active":"2023-12-20T04:40:59.956202+0000","last_peered":"2023-12-20T04:40:59.956202+0000","last_clean":"2023-12-20T04:40:59.956202+0000","last_became_active":"2023-12-20T04:40:59.946923+0000","last_became_peered":"2023-12-20T04:40:59.946923+0000","last_unstale":"2023-12-20T04:40:59.956202+0000","last_undegraded":"2023-12-20T04:40:59.956202+0000","last_fullsized":"2023-12-20T04:40:59.956202+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:02.479108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"6.4","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.932915+0000","last_change":"2023-12-20T04:40:59.541887+0000","last_active":"2023-12-20T04:40:59.932915+0000","last_peered":"2023-12-20T04:40:59.932915+0000","last_clean":"2023-12-20T04:40:59.932915+0000","last_became_active":"2023-12-20T04:40:59.541348+0000","last_became_peered":"2023-12-20T04:40:59.541348+0000","last_unstale":"2023-12-20T04:40:59.932915+0000","last_undegraded":"2023-12-20T04:40:59.932915+0000","last_fullsized":"2023-12-20T04:40:59.932915+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:21:50.352031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.2","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259891+0000","last_change":"2023-12-20T04:40:59.936829+0000","last_active":"2023-12-20T04:41:01.259891+0000","last_peered":"2023-12-20T04:41:01.259891+0000","last_clean":"2023-12-20T04:41:01.259891+0000","last_became_active":"2023-12-20T04:40:59.936519+0000","last_became_peered":"2023-12-20T04:40:59.936519+0000","last_unstale":"2023-12-20T04:41:01.259891+0000","last_undegraded":"2023-12-20T04:41:01.259891+0000","last_fullsized":"2023-12-20T04:41:01.259891+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:58:45.658383+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.3","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.935962+0000","last_change":"2023-12-20T04:40:59.548395+0000","last_active":"2023-12-20T04:40:59.935962+0000","last_peered":"2023-12-20T04:40:59.935962+0000","last_clean":"2023-12-20T04:40:59.935962+0000","last_became_active":"2023-12-20T04:40:59.548134+0000","last_became_peered":"2023-12-20T04:40:59.548134+0000","last_unstale":"2023-12-20T04:40:59.935962+0000","last_undegraded":"2023-12-20T04:40:59.935962+0000","last_fullsized":"2023-12-20T04:40:59.935962+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:19:54.323865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"7.3","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:01.259745+0000","last_change":"2023-12-20T04:40:59.938247+0000","last_active":"2023-12-20T04:41:01.259745+0000","last_peered":"2023-12-20T04:41:01.259745+0000","last_clean":"2023-12-20T04:41:01.259745+0000","last_became_active":"2023-12-20T04:40:59.938109+0000","last_became_peered":"2023-12-20T04:40:59.938109+0000","last_unstale":"2023-12-20T04:41:01.259745+0000","last_undegraded":"2023-12-20T04:41:01.259745+0000","last_fullsized":"2023-12-20T04:41:01.259745+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:33:20.043278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"6.2","version":"0'0","reported_seq":11,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.928103+0000","last_change":"2023-12-20T04:40:59.539669+0000","last_active":"2023-12-20T04:41:00.928103+0000","last_peered":"2023-12-20T04:41:00.928103+0000","last_clean":"2023-12-20T04:41:00.928103+0000","last_became_active":"2023-12-20T04:40:59.539036+0000","last_became_peered":"2023-12-20T04:40:59.539036+0000","last_unstale":"2023-12-20T04:41:00.928103+0000","last_undegraded":"2023-12-20T04:41:00.928103+0000","last_fullsized":"2023-12-20T04:41:00.928103+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:53.616685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"7.0","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.929770+0000","last_change":"2023-12-20T04:40:59.945678+0000","last_active":"2023-12-20T04:41:00.929770+0000","last_peered":"2023-12-20T04:41:00.929770+0000","last_clean":"2023-12-20T04:41:00.929770+0000","last_became_active":"2023-12-20T04:40:59.943177+0000","last_became_peered":"2023-12-20T04:40:59.943177+0000","last_unstale":"2023-12-20T04:41:00.929770+0000","last_undegraded":"2023-12-20T04:41:00.929770+0000","last_fullsized":"2023-12-20T04:41:00.929770+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:37:42.875580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"6.1","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.932854+0000","last_change":"2023-12-20T04:40:59.545744+0000","last_active":"2023-12-20T04:40:59.932854+0000","last_peered":"2023-12-20T04:40:59.932854+0000","last_clean":"2023-12-20T04:40:59.932854+0000","last_became_active":"2023-12-20T04:40:59.545462+0000","last_became_peered":"2023-12-20T04:40:59.545462+0000","last_unstale":"2023-12-20T04:40:59.932854+0000","last_undegraded":"2023-12-20T04:40:59.932854+0000","last_fullsized":"2023-12-20T04:40:59.932854+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:33:19.014284+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"7.1","version":"0'0","reported_seq":10,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.927938+0000","last_change":"2023-12-20T04:40:59.955425+0000","last_active":"2023-12-20T04:41:00.927938+0000","last_peered":"2023-12-20T04:41:00.927938+0000","last_clean":"2023-12-20T04:41:00.927938+0000","last_became_active":"2023-12-20T04:40:59.955120+0000","last_became_peered":"2023-12-20T04:40:59.955120+0000","last_unstale":"2023-12-20T04:41:00.927938+0000","last_undegraded":"2023-12-20T04:41:00.927938+0000","last_fullsized":"2023-12-20T04:41:00.927938+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:02.479108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"6.0","version":"0'0","reported_seq":10,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.937060+0000","last_change":"2023-12-20T04:40:59.537303+0000","last_active":"2023-12-20T04:40:59.937060+0000","last_peered":"2023-12-20T04:40:59.937060+0000","last_clean":"2023-12-20T04:40:59.937060+0000","last_became_active":"2023-12-20T04:40:59.537015+0000","last_became_peered":"2023-12-20T04:40:59.537015+0000","last_unstale":"2023-12-20T04:40:59.937060+0000","last_undegraded":"2023-12-20T04:40:59.937060+0000","last_fullsized":"2023-12-20T04:40:59.937060+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:43:20.804066+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"7.6","version":"0'0","reported_seq":9,"reported_epoch":60,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.957401+0000","last_change":"2023-12-20T04:40:59.948696+0000","last_active":"2023-12-20T04:40:59.957401+0000","last_peered":"2023-12-20T04:40:59.957401+0000","last_clean":"2023-12-20T04:40:59.957401+0000","last_became_active":"2023-12-20T04:40:59.948430+0000","last_became_peered":"2023-12-20T04:40:59.948430+0000","last_unstale":"2023-12-20T04:40:59.957401+0000","last_undegraded":"2023-12-20T04:40:59.957401+0000","last_fullsized":"2023-12-20T04:40:59.957401+0000","mapping_epoch":59,"log_start":"0'0","ondisk_log_start":"0'0","created":59,"last_epoch_clean":60,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:59.509564+0000","last_clean_scrub_stamp":"2023-12-20T04:40:59.509564+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:34:27.599058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"6.7","version":"60'2","reported_seq":13,"reported_epoch":61,"state":"active+clean","last_fresh":"2023-12-20T04:41:00.930125+0000","last_change":"2023-12-20T04:40:59.539524+0000","last_active":"2023-12-20T04:41:00.930125+0000","last_peered":"2023-12-20T04:41:00.930125+0000","last_clean":"2023-12-20T04:41:00.930125+0000","last_became_active":"2023-12-20T04:40:59.539230+0000","last_became_peered":"2023-12-20T04:40:59.539230+0000","last_unstale":"2023-12-20T04:41:00.930125+0000","last_undegraded":"2023-12-20T04:41:00.930125+0000","last_fullsized":"2023-12-20T04:41:00.930125+0000","mapping_epoch":58,"log_start":"0'0","ondisk_log_start":"0'0","created":58,"last_epoch_clean":59,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:40:58.490351+0000","last_clean_scrub_stamp":"2023-12-20T04:40:58.490351+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:39:48.794770+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"48'139","reported_seq":280,"reported_epoch":59,"state":"active+clean","last_fresh":"2023-12-20T04:40:59.537704+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:40:59.537704+0000","last_peered":"2023-12-20T04:40:59.537704+0000","last_clean":"2023-12-20T04:40:59.537704+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:40:59.537704+0000","last_undegraded":"2023-12-20T04:40:59.537704+0000","last_fullsized":"2023-12-20T04:40:59.537704+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":139,"log_dups_size":0,"ondisk_log_size":139,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":7,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":6,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":308,"num_read_kb":512,"num_write":199,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":139,"ondisk_log_size":139,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575000,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36424,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335416,"statfs":{"total":96636764160,"available":96599465984,"internally_reserved":0,"allocated":1310720,"data_stored":917904,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.72799999999999998,"15min":0.68999999999999995},"min":{"1min":0.372,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.58899999999999997},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.71999999999999997,"15min":0.70499999999999996},"min":{"1min":0.46800000000000003,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.99299999999999999,"5min":1,"15min":1.115},"last":0.42499999999999999}]},{"osd":1,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.74099999999999999,"15min":0.71099999999999997},"min":{"1min":0.46100000000000002,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98499999999999999,"5min":1.0529999999999999,"15min":1.2190000000000001},"last":0.443},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.74099999999999999,"15min":0.752},"min":{"1min":0.55200000000000005,"5min":0.46200000000000002,"15min":0.42999999999999999},"max":{"1min":1.032,"5min":1.032,"15min":2.649},"last":0.52000000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.72499999999999998,"15min":0.72699999999999998},"min":{"1min":0.498,"5min":0.48899999999999999,"15min":0.437},"max":{"1min":1.075,"5min":1.075,"15min":1.5960000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.753,"15min":0.74399999999999999},"min":{"1min":0.50600000000000001,"5min":0.505,"15min":0.45300000000000001},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.135},"last":0.55500000000000005}]},{"osd":3,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.70799999999999996,"5min":0.73199999999999998,"15min":0.73999999999999999},"min":{"1min":0.44,"5min":0.44,"15min":0.44},"max":{"1min":1.016,"5min":1.016,"15min":1.641},"last":0.498},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.76000000000000001,"15min":0.79000000000000004},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.49399999999999999},"max":{"1min":1.131,"5min":1.131,"15min":2.5990000000000002},"last":0.627}]},{"osd":4,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.58199999999999996,"15min":0.61399999999999999},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":0.753,"5min":0.98299999999999998,"15min":1.0109999999999999},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.60199999999999998,"15min":0.64300000000000002},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.242},"last":0.60199999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.627,"15min":0.66600000000000004},"min":{"1min":0.46899999999999997,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.877,"5min":1.0449999999999999,"15min":1.0900000000000001},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.65100000000000002,"15min":0.69099999999999995},"min":{"1min":0.47599999999999998,"5min":0.44800000000000001,"15min":0.432},"max":{"1min":0.86399999999999999,"5min":1.052,"15min":1.071},"last":0.68600000000000005}]},{"osd":6,"last update":"Wed Dec 20 04:40:41 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72199999999999998,"15min":0.75},"min":{"1min":0.54900000000000004,"5min":0.47699999999999998,"15min":0.443},"max":{"1min":0.90700000000000003,"5min":1.1020000000000001,"15min":1.1279999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.69599999999999995,"15min":0.71499999999999997},"min":{"1min":0.505,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":0.98199999999999998,"5min":0.98199999999999998,"15min":0.99199999999999999},"last":0.65700000000000003}]}]},{"osd":6,"up_from":13,"seq":55834575000,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35848,"kb_used_data":704,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335992,"statfs":{"total":96636764160,"available":96600055808,"internally_reserved":0,"allocated":720896,"data_stored":328118,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.72299999999999998,"15min":0.69699999999999995},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.27100000000000002},"max":{"1min":0.95899999999999996,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.64500000000000002,"5min":0.72399999999999998,"15min":0.69999999999999996},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":0.90600000000000003,"5min":0.96399999999999997,"15min":1.1910000000000001},"last":0.47799999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.69999999999999996,"15min":0.69499999999999995},"min":{"1min":0.34799999999999998,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.89500000000000002,"5min":0.97299999999999998,"15min":0.97299999999999998},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.748,"15min":0.71199999999999997},"min":{"1min":0.38800000000000001,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.997,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.73499999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70199999999999996,"5min":0.77000000000000002,"15min":0.749},"min":{"1min":0.47299999999999998,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.026,"5min":1.216,"15min":1.216},"last":0.54500000000000004},{"interface":"front","average":{"1min":0.67300000000000004,"5min":0.748,"15min":0.71899999999999997},"min":{"1min":0.48099999999999998,"5min":0.48099999999999998,"15min":0.38300000000000001},"max":{"1min":0.95199999999999996,"5min":1.01,"15min":1.054},"last":0.625}]},{"osd":3,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.78000000000000003,"15min":0.74299999999999999},"min":{"1min":0.51800000000000002,"5min":0.51800000000000002,"15min":0.45600000000000002},"max":{"1min":0.99299999999999999,"5min":1.0600000000000001,"15min":1.109},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.73899999999999999,"15min":0.73699999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92000000000000004,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60199999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.42399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.79300000000000004,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.501},{"interface":"front","average":{"1min":0.622,"5min":0.61799999999999999,"15min":0.65200000000000002},"min":{"1min":0.44500000000000001,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.876,"5min":0.995,"15min":1.071},"last":0.77500000000000002}]},{"osd":5,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.624,"15min":0.64800000000000002},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.92600000000000005,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.754},{"interface":"front","average":{"1min":0.67700000000000005,"5min":0.67900000000000005,"15min":0.70299999999999996},"min":{"1min":0.47399999999999998,"5min":0.45700000000000002,"15min":0.434},"max":{"1min":0.89700000000000002,"5min":1.306,"15min":1.306},"last":0.79400000000000004}]},{"osd":7,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.70499999999999996,"15min":0.72999999999999998},"min":{"1min":0.504,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.94499999999999995,"5min":0.99099999999999999,"15min":1.2270000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.70299999999999996,"15min":0.73199999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.96599999999999997,"5min":1.002,"15min":1.169},"last":0.81100000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":716800,"data_stored":327516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.70799999999999996,"15min":0.67100000000000004},"min":{"1min":0.33700000000000002,"5min":0.33700000000000002,"15min":0.29999999999999999},"max":{"1min":0.94999999999999996,"5min":1.006,"15min":1.05},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.72499999999999998,"15min":0.70699999999999996},"min":{"1min":0.439,"5min":0.439,"15min":0.34300000000000003},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":0.98599999999999999},"last":0.78400000000000003}]},{"osd":1,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66900000000000004,"5min":0.72699999999999998,"15min":0.69199999999999995},"min":{"1min":0.45500000000000002,"5min":0.45500000000000002,"15min":0.441},"max":{"1min":1.044,"5min":1.046,"15min":1.0600000000000001},"last":0.72499999999999998},{"interface":"front","average":{"1min":0.70399999999999996,"5min":0.73299999999999998,"15min":0.73599999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.91700000000000004,"5min":1.024,"15min":1.1140000000000001},"last":0.83399999999999996}]},{"osd":2,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.72799999999999998,"15min":0.71699999999999997},"min":{"1min":0.53000000000000003,"5min":0.432,"15min":0.40500000000000003},"max":{"1min":0.93300000000000005,"5min":0.96299999999999997,"15min":1.153},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.747,"15min":0.72299999999999998},"min":{"1min":0.52000000000000002,"5min":0.495,"15min":0.47899999999999998},"max":{"1min":0.90300000000000002,"5min":0.95999999999999996,"15min":1.004},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.74299999999999999,"15min":0.747},"min":{"1min":0.48799999999999999,"5min":0.48799999999999999,"15min":0.47499999999999998},"max":{"1min":0.95999999999999996,"5min":1.018,"15min":1.758},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.77200000000000002,"15min":0.74299999999999999},"min":{"1min":0.56200000000000006,"5min":0.55900000000000005,"15min":0.436},"max":{"1min":0.88400000000000001,"5min":1.0149999999999999,"15min":1.0469999999999999},"last":0.85399999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.57599999999999996,"5min":0.60899999999999999,"15min":0.626},"min":{"1min":0.41899999999999998,"5min":0.36899999999999999,"15min":0.35099999999999998},"max":{"1min":0.81499999999999995,"5min":0.92700000000000005,"15min":1.0229999999999999},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.623,"15min":0.64200000000000002},"min":{"1min":0.439,"5min":0.39600000000000002,"15min":0.39600000000000002},"max":{"1min":0.84999999999999998,"5min":1.077,"15min":1.077},"last":0.61399999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68000000000000005,"15min":0.68700000000000006},"min":{"1min":0.47799999999999998,"5min":0.46200000000000002,"15min":0.39900000000000002},"max":{"1min":0.82499999999999996,"5min":1.056,"15min":1.0700000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.63700000000000001,"15min":0.65300000000000002},"min":{"1min":0.42899999999999999,"5min":0.39400000000000002,"15min":0.38600000000000001},"max":{"1min":0.999,"5min":0.999,"15min":0.999},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68400000000000005,"15min":0.70799999999999996},"min":{"1min":0.48999999999999999,"5min":0.442,"15min":0.42999999999999999},"max":{"1min":0.86799999999999999,"5min":1.141,"15min":1.141},"last":0.754},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.74399999999999999,"15min":0.755},"min":{"1min":0.54900000000000004,"5min":0.496,"15min":0.47899999999999998},"max":{"1min":0.89800000000000002,"5min":0.99199999999999999,"15min":1.119},"last":0.79700000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575002,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":53900,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94317940,"statfs":{"total":96636764160,"available":96581570560,"internally_reserved":0,"allocated":724992,"data_stored":329010,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.41699999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0049999999999999,"5min":1.0740000000000001,"15min":1.5509999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.71899999999999997,"15min":0.70599999999999996},"min":{"1min":0.376,"5min":0.376,"15min":0.36199999999999999},"max":{"1min":1.625,"5min":1.625,"15min":1.633},"last":0.51300000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71699999999999997,"15min":0.70799999999999996},"min":{"1min":0.439,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.88700000000000001,"5min":0.98799999999999999,"15min":1.798},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.751,"15min":0.747},"min":{"1min":0.45900000000000002,"5min":0.442,"15min":0.433},"max":{"1min":2.2930000000000001,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.499}]},{"osd":2,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73399999999999999,"15min":0.72799999999999998},"min":{"1min":0.43099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.52,"5min":1.52,"15min":1.728},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.75700000000000001,"15min":0.753},"min":{"1min":0.55900000000000005,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":1.093,"5min":1.093,"15min":1.6619999999999999},"last":0.56899999999999995}]},{"osd":3,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70299999999999996,"5min":0.73399999999999999,"15min":0.72299999999999998},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.40799999999999997},"max":{"1min":1.554,"5min":1.554,"15min":1.5760000000000001},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.75900000000000001,"15min":0.76600000000000001},"min":{"1min":0.57199999999999995,"5min":0.498,"15min":0.48299999999999998},"max":{"1min":0.91900000000000004,"5min":1.0620000000000001,"15min":1.7649999999999999},"last":0.52500000000000002}]},{"osd":5,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.61399999999999999,"15min":0.63800000000000001},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":1.456,"5min":1.456,"15min":1.456},"last":0.38900000000000001},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.63600000000000001,"15min":0.65100000000000002},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.589,"5min":1.589,"15min":1.589},"last":0.43099999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.64400000000000002,"15min":0.67600000000000005},"min":{"1min":0.39200000000000002,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.85899999999999999,"5min":1.099,"15min":1.6739999999999999},"last":0.35999999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.69199999999999995,"15min":0.71199999999999997},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.0620000000000001,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.41899999999999998}]},{"osd":7,"last update":"Wed Dec 20 04:40:47 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.69799999999999995,"15min":0.70899999999999996},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.655,"5min":1.655,"15min":1.655},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73299999999999998,"15min":0.751},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.97999999999999998,"5min":1.125,"15min":1.125},"last":0.47399999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575001,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":35852,"kb_used_data":708,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335988,"statfs":{"total":96636764160,"available":96600051712,"internally_reserved":0,"allocated":724992,"data_stored":328084,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46100000000000002,"15min":0.53200000000000003},"min":{"1min":0.30399999999999999,"5min":0.26200000000000001,"15min":0.23200000000000001},"max":{"1min":0.875,"5min":0.875,"15min":4.3700000000000001},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.50600000000000001,"5min":0.53400000000000003,"15min":0.53900000000000003},"min":{"1min":0.32600000000000001,"5min":0.32500000000000001,"15min":0.28799999999999998},"max":{"1min":0.80800000000000005,"5min":0.80800000000000005,"15min":3.355},"last":0.30599999999999999}]},{"osd":1,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.47399999999999998,"15min":0.52100000000000002},"min":{"1min":0.32000000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.58499999999999996,"5min":0.70099999999999996,"15min":3.7160000000000002},"last":0.38200000000000001},{"interface":"front","average":{"1min":0.52700000000000002,"5min":0.55600000000000005,"15min":0.58799999999999997},"min":{"1min":0.316,"5min":0.316,"15min":0.28399999999999997},"max":{"1min":0.72599999999999998,"5min":0.79000000000000004,"15min":3.3109999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.58599999999999997,"15min":0.63},"min":{"1min":0.37,"5min":0.318,"15min":0.318},"max":{"1min":0.88700000000000001,"5min":0.88700000000000001,"15min":4.0110000000000001},"last":0.44},{"interface":"front","average":{"1min":0.56100000000000005,"5min":0.55900000000000005,"15min":0.66000000000000003},"min":{"1min":0.41199999999999998,"5min":0.34799999999999998,"15min":0.316},"max":{"1min":0.93300000000000005,"5min":0.93300000000000005,"15min":6.1399999999999997},"last":0.52800000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.79100000000000004,"15min":0.79100000000000004},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.48499999999999999},"max":{"1min":1.115,"5min":1.131,"15min":1.3089999999999999},"last":0.753},{"interface":"front","average":{"1min":0.79000000000000004,"5min":0.81299999999999994,"15min":0.87},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.198,"5min":1.4059999999999999,"15min":3.9510000000000001},"last":0.76800000000000002}]},{"osd":5,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.78300000000000003,"5min":0.81299999999999994,"15min":0.88300000000000001},"min":{"1min":0.54200000000000004,"5min":0.54200000000000004,"15min":0.46500000000000002},"max":{"1min":1.1000000000000001,"5min":1.3300000000000001,"15min":3.9830000000000001},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.78900000000000003,"15min":0.874},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.38400000000000001},"max":{"1min":1.0609999999999999,"5min":1.252,"15min":4.6550000000000002},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.76400000000000001,"5min":0.81299999999999994,"15min":0.81599999999999995},"min":{"1min":0.58199999999999996,"5min":0.56799999999999995,"15min":0.45600000000000002},"max":{"1min":1.002,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.82099999999999995,"15min":0.88},"min":{"1min":0.55000000000000004,"5min":0.55000000000000004,"15min":0.499},"max":{"1min":1.2270000000000001,"5min":1.2929999999999999,"15min":3.8900000000000001},"last":0.68100000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:40:45 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.80400000000000005,"15min":0.88700000000000001},"min":{"1min":0.61699999999999999,"5min":0.54000000000000004,"15min":0.49299999999999999},"max":{"1min":1.1319999999999999,"5min":1.1659999999999999,"15min":4.6970000000000001},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.55100000000000005},"max":{"1min":1.077,"5min":1.29,"15min":1.29},"last":0.69899999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575000,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":39964,"kb_used_data":724,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331876,"statfs":{"total":96636764160,"available":96595841024,"internally_reserved":0,"allocated":741376,"data_stored":335332,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.37,"5min":0.44800000000000001,"15min":0.442},"min":{"1min":0.216,"5min":0.216,"15min":0.216},"max":{"1min":0.52800000000000002,"5min":0.78900000000000003,"15min":0.89600000000000002},"last":0.309},{"interface":"front","average":{"1min":0.433,"5min":0.50600000000000001,"15min":0.47999999999999998},"min":{"1min":0.28499999999999998,"5min":0.28499999999999998,"15min":0.26700000000000002},"max":{"1min":0.626,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.66700000000000004}]},{"osd":1,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.42399999999999999,"5min":0.48099999999999998,"15min":0.46100000000000002},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28899999999999998},"max":{"1min":0.64000000000000001,"5min":0.876,"15min":0.876},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.46400000000000002,"5min":0.53600000000000003,"15min":0.52400000000000002},"min":{"1min":0.32000000000000001,"5min":0.32000000000000001,"15min":0.28100000000000003},"max":{"1min":0.629,"5min":0.88600000000000001,"15min":1.01},"last":0.35199999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.53700000000000003,"5min":0.61199999999999999,"15min":0.58299999999999996},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.34999999999999998},"max":{"1min":0.78500000000000003,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.64100000000000001,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.754,"5min":0.95599999999999996,"15min":1.0900000000000001},"last":0.58699999999999997}]},{"osd":4,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.79500000000000004,"15min":0.78900000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.45200000000000001},"max":{"1min":0.92300000000000004,"5min":1.145,"15min":1.1799999999999999},"last":0.503},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.77200000000000002,"15min":0.77600000000000002},"min":{"1min":0.44500000000000001,"5min":0.44500000000000001,"15min":0.41399999999999998},"max":{"1min":1.002,"5min":1.0760000000000001,"15min":1.2010000000000001},"last":0.73599999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.79300000000000004,"15min":0.78000000000000003},"min":{"1min":0.499,"5min":0.499,"15min":0.48499999999999999},"max":{"1min":0.93000000000000005,"5min":1.1220000000000001,"15min":1.327},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.80100000000000005,"15min":0.80800000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.016,"5min":1.101,"15min":1.381},"last":0.53300000000000003}]},{"osd":6,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.79500000000000004,"15min":0.79200000000000004},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.98399999999999999,"5min":1.151,"15min":1.2809999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.80500000000000005,"15min":0.80000000000000004},"min":{"1min":0.51000000000000001,"5min":0.51000000000000001,"15min":0.46100000000000002},"max":{"1min":0.94599999999999995,"5min":1.091,"15min":1.153},"last":0.52000000000000002}]},{"osd":7,"last update":"Wed Dec 20 04:40:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.82299999999999995,"15min":0.82399999999999995},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":0.95499999999999996,"5min":1.1499999999999999,"15min":1.3520000000000001},"last":0.626},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.86499999999999999,"15min":0.85399999999999998},"min":{"1min":0.64300000000000002,"5min":0.64300000000000002,"15min":0.504},"max":{"1min":0.97499999999999998,"5min":1.1220000000000001,"15min":1.401},"last":0.68000000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575002,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":31748,"kb_used_data":700,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94340092,"statfs":{"total":96636764160,"available":96604254208,"internally_reserved":0,"allocated":716800,"data_stored":327516,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.41099999999999998,"5min":0.44700000000000001,"15min":0.44800000000000001},"min":{"1min":0.22600000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57699999999999996,"5min":0.77800000000000002,"15min":0.77800000000000002},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.44400000000000001,"5min":0.49399999999999999,"15min":0.48599999999999999},"min":{"1min":0.315,"5min":0.29699999999999999,"15min":0.28599999999999998},"max":{"1min":0.60999999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.46000000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55300000000000005,"5min":0.52900000000000003,"15min":0.51500000000000001},"min":{"1min":0.38500000000000001,"5min":0.34699999999999998,"15min":0.32800000000000001},"max":{"1min":0.81399999999999995,"5min":1.091,"15min":1.091},"last":0.442},{"interface":"front","average":{"1min":0.46000000000000002,"5min":0.53600000000000003,"15min":0.53900000000000003},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.33900000000000002},"max":{"1min":0.63200000000000001,"5min":0.81599999999999995,"15min":0.91300000000000003},"last":0.48599999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.52600000000000002,"5min":0.60099999999999998,"15min":0.58699999999999997},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.36599999999999999},"max":{"1min":0.73599999999999999,"5min":0.96499999999999997,"15min":1.016},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.47899999999999998,"5min":0.54400000000000004,"15min":0.52100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.64900000000000002,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.374}]},{"osd":4,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.76100000000000001,"15min":0.77200000000000002},"min":{"1min":0.45000000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.78000000000000003,"5min":1.27,"15min":1.4419999999999999},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.68300000000000005,"5min":0.77100000000000002,"15min":0.78300000000000003},"min":{"1min":0.434,"5min":0.434,"15min":0.434},"max":{"1min":0.88900000000000001,"5min":1.1160000000000001,"15min":1.3380000000000001},"last":0.73499999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.55400000000000005,"5min":0.54000000000000004,"15min":0.5},"max":{"1min":1.3420000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.63},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.77000000000000002,"15min":0.79100000000000004},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.873,"5min":1.2110000000000001,"15min":1.3009999999999999},"last":0.71099999999999997}]},{"osd":6,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.78000000000000003,"15min":0.79200000000000004},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.84199999999999997,"5min":1.194,"15min":1.415},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.748,"5min":0.80200000000000005,"15min":0.79400000000000004},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.252,"5min":1.252,"15min":1.268},"last":0.75900000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.82499999999999996,"15min":0.83999999999999997},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.306,"5min":1.306,"15min":1.399},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.83599999999999997,"15min":0.84299999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":0.94699999999999995,"5min":1.3440000000000001,"15min":1.373},"last":0.748}]}]},{"osd":0,"up_from":13,"seq":55834575001,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32328,"kb_used_data":1280,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339512,"statfs":{"total":96636764160,"available":96603660288,"internally_reserved":0,"allocated":1310720,"data_stored":917848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55600000000000005,"5min":0.46000000000000002,"15min":0.438},"min":{"1min":0.28499999999999998,"5min":0.218,"15min":0.218},"max":{"1min":2.5640000000000001,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.94299999999999995,"5min":0.60099999999999998,"15min":0.52800000000000002},"min":{"1min":0.35499999999999998,"5min":0.29699999999999999,"15min":0.26900000000000002},"max":{"1min":7.8860000000000001,"5min":7.8860000000000001,"15min":7.8860000000000001},"last":0.46899999999999997}]},{"osd":2,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.54100000000000004,"15min":0.52100000000000002},"min":{"1min":0.35799999999999998,"5min":0.28899999999999998,"15min":0.28399999999999997},"max":{"1min":2.528,"5min":2.528,"15min":2.528},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.55400000000000005,"15min":0.55100000000000005},"min":{"1min":0.40600000000000003,"5min":0.36299999999999999,"15min":0.34000000000000002},"max":{"1min":0.77600000000000002,"5min":0.78000000000000003,"15min":1.2989999999999999},"last":0.56200000000000006}]},{"osd":3,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.55200000000000005,"15min":0.51500000000000001},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.29499999999999998},"max":{"1min":3.1000000000000001,"5min":3.1000000000000001,"15min":3.1000000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":1.103,"5min":0.69499999999999995,"15min":0.61099999999999999},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38200000000000001},"max":{"1min":10.49,"5min":10.49,"15min":10.49},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.91400000000000003,"5min":0.81200000000000006,"15min":0.82299999999999995},"min":{"1min":0.58799999999999997,"5min":0.51100000000000001,"15min":0.45400000000000001},"max":{"1min":3.0659999999999998,"5min":3.0659999999999998,"15min":3.7469999999999999},"last":0.81399999999999995},{"interface":"front","average":{"1min":1.0269999999999999,"5min":0.84299999999999997,"15min":0.80100000000000005},"min":{"1min":0.499,"5min":0.499,"15min":0.499},"max":{"1min":5.2460000000000004,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.96699999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.81399999999999995,"15min":0.80500000000000005},"min":{"1min":0.55500000000000005,"5min":0.54800000000000004,"15min":0.44700000000000001},"max":{"1min":0.996,"5min":1.3109999999999999,"15min":1.877},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.81000000000000005,"15min":0.79900000000000004},"min":{"1min":0.56200000000000006,"5min":0.496,"15min":0.496},"max":{"1min":1.151,"5min":1.2010000000000001,"15min":1.7450000000000001},"last":0.93999999999999995}]},{"osd":6,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":1.05,"5min":0.86199999999999999,"15min":0.82399999999999995},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.47299999999999998},"max":{"1min":5.2839999999999998,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.92600000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.81699999999999995,"15min":0.81899999999999995},"min":{"1min":0.51300000000000001,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.046,"5min":1.1479999999999999,"15min":1.492},"last":0.80000000000000004}]},{"osd":7,"last update":"Wed Dec 20 04:40:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.82899999999999996,"15min":0.83699999999999997},"min":{"1min":0.52800000000000002,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0900000000000001,"5min":1.216,"15min":1.6539999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.81799999999999995,"15min":0.82099999999999995},"min":{"1min":0.52900000000000003,"5min":0.52900000000000003,"15min":0.48799999999999999},"max":{"1min":0.93999999999999995,"5min":1.276,"15min":2.113},"last":0.91300000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":6,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":7,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:41:04.423 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T04:41:04.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T04:41:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.617+0000 7efdbac66640 1 Processor -- start 2023-12-20T04:41:04.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.617+0000 7efdbac66640 1 -- start start 2023-12-20T04:41:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.617+0000 7efdbac66640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdb415a940 0x7efdb40cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.617+0000 7efdbac66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40cb1c0 0x7efdb40cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.617+0000 7efdbac66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb40cc9d0 con 0x7efdb415ad20 2023-12-20T04:41:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.617+0000 7efdbac66640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb40cbbb0 con 0x7efdb415a940 2023-12-20T04:41:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.617+0000 7efdbac66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40c9b90 con 0x7efdb40cb1c0 2023-12-20T04:41:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.621+0000 7efdb8a02640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdb415a940 0x7efdb40cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.621+0000 7efdb8a02640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdb415a940 0x7efdb40cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51466/0 (socket says 172.21.15.32:51466) 2023-12-20T04:41:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.621+0000 7efdb8a02640 1 -- 172.21.15.32:0/229846052 learned_addr learned my addr 172.21.15.32:0/229846052 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:04.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.621+0000 7efdb9203640 1 --2- 172.21.15.32:0/229846052 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40cb1c0 0x7efdb40cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.621+0000 7efdb8a02640 1 -- 172.21.15.32:0/229846052 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40cb1c0 msgr2=0x7efdb40cb670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.621+0000 7efdb8a02640 1 --2- 172.21.15.32:0/229846052 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40cb1c0 0x7efdb40cb670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.621+0000 7efdb8a02640 1 -- 172.21.15.32:0/229846052 >> v1:172.21.15.39:6789/0 conn(0x7efdb415ad20 legacy=0x7efdb40ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.621+0000 7efdb8a02640 1 -- 172.21.15.32:0/229846052 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb40c9e70 con 0x7efdb415a940 2023-12-20T04:41:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.621+0000 7efdb8a02640 1 --2- 172.21.15.32:0/229846052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb415a940 0x7efdb40cc380 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efda8009870 tx=0x7efda801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3159524a78ee9ef4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.621+0000 7efdb37fe640 1 -- 172.21.15.32:0/229846052 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda80aa670 con 0x7efdb415a940 2023-12-20T04:41:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdb37fe640 1 -- 172.21.15.32:0/229846052 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda80aa7d0 con 0x7efdb415a940 2023-12-20T04:41:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdb37fe640 1 -- 172.21.15.32:0/229846052 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda80aab60 con 0x7efdb415a940 2023-12-20T04:41:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 -- 172.21.15.32:0/229846052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb415a940 msgr2=0x7efdb40cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 --2- 172.21.15.32:0/229846052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb415a940 0x7efdb40cc380 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7efda8009870 tx=0x7efda801c510 comp rx=0 tx=0).stop 2023-12-20T04:41:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 -- 172.21.15.32:0/229846052 shutdown_connections 2023-12-20T04:41:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 --2- 172.21.15.32:0/229846052 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40cb1c0 0x7efdb40cb670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 --2- 172.21.15.32:0/229846052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb415a940 0x7efdb40cc380 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 -- 172.21.15.32:0/229846052 >> 172.21.15.32:0/229846052 conn(0x7efdb406a4b0 msgr2=0x7efdb40b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 -- 172.21.15.32:0/229846052 shutdown_connections 2023-12-20T04:41:04.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 -- 172.21.15.32:0/229846052 wait complete. 2023-12-20T04:41:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 Processor -- start 2023-12-20T04:41:04.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 -- start start 2023-12-20T04:41:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb40cb1c0 0x7efdb414f260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb415a940 0x7efdb414f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb415ad20 0x7efdb40c6c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb414b9f0 con 0x7efdb415ad20 2023-12-20T04:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40ad8d0 con 0x7efdb40cb1c0 2023-12-20T04:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdbac66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40c8220 con 0x7efdb415a940 2023-12-20T04:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdb9203640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb415ad20 0x7efdb40c6c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdb9203640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb415ad20 0x7efdb40c6c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51478/0 (socket says 172.21.15.32:51478) 2023-12-20T04:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.625+0000 7efdb9203640 1 -- 172.21.15.32:0/1284714601 learned_addr learned my addr 172.21.15.32:0/1284714601 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb8a02640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb40cb1c0 0x7efdb414f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb3fff640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb415a940 0x7efdb414f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb9203640 1 -- 172.21.15.32:0/1284714601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb415a940 msgr2=0x7efdb414f7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb9203640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb415a940 0x7efdb414f7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb9203640 1 -- 172.21.15.32:0/1284714601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb40cb1c0 msgr2=0x7efdb414f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb9203640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb40cb1c0 0x7efdb414f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb9203640 1 -- 172.21.15.32:0/1284714601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda80094d0 con 0x7efdb415ad20 2023-12-20T04:41:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb9203640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb415ad20 0x7efdb40c6c30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efdac007e40 tx=0x7efdac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb17fa640 1 -- 172.21.15.32:0/1284714601 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac014620 con 0x7efdb415ad20 2023-12-20T04:41:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb17fa640 1 -- 172.21.15.32:0/1284714601 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdac00ee90 con 0x7efdb415ad20 2023-12-20T04:41:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb17fa640 1 -- 172.21.15.32:0/1284714601 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdac0149e0 con 0x7efdb415ad20 2023-12-20T04:41:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdbac66640 1 -- 172.21.15.32:0/1284714601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb40c7980 con 0x7efdb415ad20 2023-12-20T04:41:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdbac66640 1 -- 172.21.15.32:0/1284714601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb40c7f40 con 0x7efdb415ad20 2023-12-20T04:41:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdbac66640 1 -- 172.21.15.32:0/1284714601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb404e000 con 0x7efdb415ad20 2023-12-20T04:41:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.629+0000 7efdb17fa640 1 -- 172.21.15.32:0/1284714601 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7efdac014b40 con 0x7efdb415ad20 2023-12-20T04:41:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.633+0000 7efdb17fa640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd98065b80 0x7efd98068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.633+0000 7efdb17fa640 1 -- 172.21.15.32:0/1284714601 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efdac083ad0 con 0x7efdb415ad20 2023-12-20T04:41:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.633+0000 7efdb8a02640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd98065b80 0x7efd98068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.633+0000 7efdb8a02640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd98065b80 0x7efd98068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efda80ae000 tx=0x7efda80b5040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.637+0000 7efdb17fa640 1 -- 172.21.15.32:0/1284714601 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efdac051430 con 0x7efdb415ad20 2023-12-20T04:41:04.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.849+0000 7efdbac66640 1 -- 172.21.15.32:0/1284714601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7efdb415c4e0 con 0x7efdb415ad20 2023-12-20T04:41:04.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.853+0000 7efdb17fa640 1 -- 172.21.15.32:0/1284714601 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v16) v1 ==== 190+0+0 (secure 0 0 0) 0x7efdac054a50 con 0x7efdb415ad20 2023-12-20T04:41:04.854 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T04:41:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 -- 172.21.15.32:0/1284714601 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd98065b80 msgr2=0x7efd98068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd98065b80 0x7efd98068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efda80ae000 tx=0x7efda80b5040 comp rx=0 tx=0).stop 2023-12-20T04:41:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 -- 172.21.15.32:0/1284714601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb415ad20 msgr2=0x7efdb40c6c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb415ad20 0x7efdb40c6c30 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7efdac007e40 tx=0x7efdac00e4b0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 -- 172.21.15.32:0/1284714601 shutdown_connections 2023-12-20T04:41:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd98065b80 0x7efd98068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb415ad20 0x7efdb40c6c30 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb415a940 0x7efdb414f7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 --2- 172.21.15.32:0/1284714601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb40cb1c0 0x7efdb414f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 -- 172.21.15.32:0/1284714601 >> 172.21.15.32:0/1284714601 conn(0x7efdb406a4b0 msgr2=0x7efdb4154000 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 -- 172.21.15.32:0/1284714601 shutdown_connections 2023-12-20T04:41:04.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:04.857+0000 7efdbac66640 1 -- 172.21.15.32:0/1284714601 wait complete. 2023-12-20T04:41:04.875 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:41:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0ab597b640 1 Processor -- start 2023-12-20T04:41:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0ab597b640 1 -- start start 2023-12-20T04:41:05.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0ab597b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab015ce30 0x7f0ab0161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0ab597b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ab006deb0 0x7f0ab00cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0ab597b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0ab0071920 con 0x7f0ab0070950 2023-12-20T04:41:05.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0ab597b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab0071080 con 0x7f0ab006deb0 2023-12-20T04:41:05.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0ab597b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab0071360 con 0x7f0ab015ce30 2023-12-20T04:41:05.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0aaf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ab006deb0 0x7f0ab00cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0aae7fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0ab0070950 0x7f0ab006d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35848/0 (socket says 172.21.15.32:35848) 2023-12-20T04:41:05.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0aae7fc640 1 -- 172.21.15.32:0/1798539086 learned_addr learned my addr 172.21.15.32:0/1798539086 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:41:05.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0aaf7fe640 1 -- 172.21.15.32:0/1798539086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab015ce30 msgr2=0x7f0ab0161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:05.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0aaeffd640 1 --2- 172.21.15.32:0/1798539086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab015ce30 0x7f0ab0161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0aaf7fe640 1 --2- 172.21.15.32:0/1798539086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab015ce30 0x7f0ab0161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0aaf7fe640 1 -- 172.21.15.32:0/1798539086 >> v1:172.21.15.32:6789/0 conn(0x7f0ab0070950 legacy=0x7f0ab006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0aaf7fe640 1 -- 172.21.15.32:0/1798539086 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab0071640 con 0x7f0ab006deb0 2023-12-20T04:41:05.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0aadffb640 1 -- 172.21.15.32:0/1798539086 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2102250957 0 0) 0x7f0ab0071920 con 0x7f0ab0070950 2023-12-20T04:41:05.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.021+0000 7f0aaeffd640 1 --2- 172.21.15.32:0/1798539086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab015ce30 0x7f0ab0161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:41:05.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0aaf7fe640 1 --2- 172.21.15.32:0/1798539086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab006deb0 0x7f0ab00cd520 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009d90 tx=0x7f0aa801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eb7efede0f37c24 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:05.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0aadffb640 1 -- 172.21.15.32:0/1798539086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80a5810 con 0x7f0ab006deb0 2023-12-20T04:41:05.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0aadffb640 1 -- 172.21.15.32:0/1798539086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa80a5970 con 0x7f0ab006deb0 2023-12-20T04:41:05.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0aadffb640 1 -- 172.21.15.32:0/1798539086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa80a5d00 con 0x7f0ab006deb0 2023-12-20T04:41:05.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0ab597b640 1 -- 172.21.15.32:0/1798539086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab006deb0 msgr2=0x7f0ab00cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0ab597b640 1 --2- 172.21.15.32:0/1798539086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab006deb0 0x7f0ab00cd520 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x7f0aa8009d90 tx=0x7f0aa801c8c0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0ab597b640 1 -- 172.21.15.32:0/1798539086 shutdown_connections 2023-12-20T04:41:05.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0ab597b640 1 --2- 172.21.15.32:0/1798539086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab006deb0 0x7f0ab00cd520 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0ab597b640 1 --2- 172.21.15.32:0/1798539086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab015ce30 0x7f0ab0161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0ab597b640 1 -- 172.21.15.32:0/1798539086 >> 172.21.15.32:0/1798539086 conn(0x7f0ab00cba50 msgr2=0x7f0ab00cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:05.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0ab597b640 1 -- 172.21.15.32:0/1798539086 shutdown_connections 2023-12-20T04:41:05.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.025+0000 7f0ab597b640 1 -- 172.21.15.32:0/1798539086 wait complete. 2023-12-20T04:41:05.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0ab597b640 1 Processor -- start 2023-12-20T04:41:05.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0ab597b640 1 -- start start 2023-12-20T04:41:05.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0ab597b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab006deb0 0x7f0ab01529d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0ab597b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab0070950 0x7f0ab00c6c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0ab597b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab015ce30 0x7f0ab00ca780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0ab597b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab00c7d60 con 0x7f0ab006deb0 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0ab597b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab00c8040 con 0x7f0ab0070950 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0ab597b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab00c8320 con 0x7f0ab015ce30 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aaeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab006deb0 0x7f0ab01529d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aaeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab006deb0 0x7f0ab01529d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51496/0 (socket says 172.21.15.32:51496) 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aaeffd640 1 -- 172.21.15.32:0/51021295 learned_addr learned my addr 172.21.15.32:0/51021295 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aaf7fe640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab015ce30 0x7f0ab00ca780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aae7fc640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab0070950 0x7f0ab00c6c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aaeffd640 1 -- 172.21.15.32:0/51021295 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab015ce30 msgr2=0x7f0ab00ca780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aaeffd640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab015ce30 0x7f0ab00ca780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aaeffd640 1 -- 172.21.15.32:0/51021295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab0070950 msgr2=0x7f0ab00c6c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aaeffd640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab0070950 0x7f0ab00c6c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aaeffd640 1 -- 172.21.15.32:0/51021295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa8009a70 con 0x7f0ab006deb0 2023-12-20T04:41:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aae7fc640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab0070950 0x7f0ab00c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:41:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0aaeffd640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab006deb0 0x7f0ab01529d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000c7f0 tx=0x7f0aa000ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0a97fff640 1 -- 172.21.15.32:0/51021295 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0007820 con 0x7f0ab006deb0 2023-12-20T04:41:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0a97fff640 1 -- 172.21.15.32:0/51021295 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa0007980 con 0x7f0ab006deb0 2023-12-20T04:41:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0a97fff640 1 -- 172.21.15.32:0/51021295 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa0007d30 con 0x7f0ab006deb0 2023-12-20T04:41:05.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0ab597b640 1 -- 172.21.15.32:0/51021295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab00c8600 con 0x7f0ab006deb0 2023-12-20T04:41:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.029+0000 7f0ab597b640 1 -- 172.21.15.32:0/51021295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab00c7980 con 0x7f0ab006deb0 2023-12-20T04:41:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.033+0000 7f0ab597b640 1 -- 172.21.15.32:0/51021295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab004e000 con 0x7f0ab006deb0 2023-12-20T04:41:05.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.033+0000 7f0a97fff640 1 -- 172.21.15.32:0/51021295 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0aa00113f0 con 0x7f0ab006deb0 2023-12-20T04:41:05.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.033+0000 7f0a97fff640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a9c065b80 0x7f0a9c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.033+0000 7f0a97fff640 1 -- 172.21.15.32:0/51021295 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0aa0083730 con 0x7f0ab006deb0 2023-12-20T04:41:05.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.033+0000 7f0aae7fc640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a9c065b80 0x7f0a9c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.033+0000 7f0aae7fc640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a9c065b80 0x7f0a9c068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4007fd0 tx=0x7f0aa4007c20 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:05.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.037+0000 7f0a97fff640 1 -- 172.21.15.32:0/51021295 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0aa0050ee0 con 0x7f0ab006deb0 2023-12-20T04:41:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.225+0000 7f0ab597b640 1 -- 172.21.15.32:0/51021295 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ab00b4f50 con 0x7f0ab006deb0 2023-12-20T04:41:05.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0a97fff640 1 -- 172.21.15.32:0/51021295 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4700 (secure 0 0 0) 0x7f0aa0054500 con 0x7f0ab006deb0 2023-12-20T04:41:05.227 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:41:05.227 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/536564441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":536564441},{"type":"v1","addr":"172.21.15.32:6835","nonce":536564441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2898125535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2898125535},{"type":"v1","addr":"172.21.15.39:6835","nonce":2898125535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/319656564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":319656564},{"type":"v1","addr":"172.21.15.32:6837","nonce":319656564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:40:59.920384+0000","modified":"2023-12-20T04:41:02.943287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5895},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5895":{"gid":5895,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/192084837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":192084837},{"type":"v1","addr":"172.21.15.39:6834","nonce":192084837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-20T04:41:05.228 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2023-12-20T04:41:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 -- 172.21.15.32:0/51021295 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a9c065b80 msgr2=0x7f0a9c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a9c065b80 0x7f0a9c068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f0aa4007fd0 tx=0x7f0aa4007c20 comp rx=0 tx=0).stop 2023-12-20T04:41:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 -- 172.21.15.32:0/51021295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab006deb0 msgr2=0x7f0ab01529d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab006deb0 0x7f0ab01529d0 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f0aa000c7f0 tx=0x7f0aa000ccb0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 -- 172.21.15.32:0/51021295 shutdown_connections 2023-12-20T04:41:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0a9c065b80 0x7f0a9c068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab015ce30 0x7f0ab00ca780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab0070950 0x7f0ab00c6c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 --2- 172.21.15.32:0/51021295 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab006deb0 0x7f0ab01529d0 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 -- 172.21.15.32:0/51021295 >> 172.21.15.32:0/51021295 conn(0x7f0ab00cba50 msgr2=0x7f0ab015efc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 -- 172.21.15.32:0/51021295 shutdown_connections 2023-12-20T04:41:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.229+0000 7f0ab597b640 1 -- 172.21.15.32:0/51021295 wait complete. 2023-12-20T04:41:05.485 INFO:tasks.ceph.mds.b.smithi039.stderr:2023-12-20T04:41:05.486+0000 7fc016be3640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:41:05.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 24, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T04:40:59.920384+0000', 'modified': '2023-12-20T04:41:02.943287+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 5895}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_5895': {'gid': 5895, 'name': 'b', 'rank': 0, 'incarnation': 20, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6834/192084837', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 192084837}, {'type': 'v1', 'addr': '172.21.15.39:6834', 'nonce': 192084837}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [7], 'metadata_pool': 6, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T04:41:05.651 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T04:41:05.651 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2023-12-20T04:41:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd1e939640 1 Processor -- start 2023-12-20T04:41:05.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd1e939640 1 -- start start 2023-12-20T04:41:05.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd1e939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd181599e0 0x7fcd180aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd1e939640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcd180c90e0 0x7fcd180b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd1e939640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd180711d0 con 0x7fcd180b0b30 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd1e939640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd180c98c0 con 0x7fcd180c90e0 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd1e939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180c9ba0 con 0x7fcd181599e0 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd1ced6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcd180b0b30 0x7fcd1815d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35852/0 (socket says 172.21.15.32:35852) 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd1ced6640 1 -- 172.21.15.32:0/1145106108 learned_addr learned my addr 172.21.15.32:0/1145106108 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd177fe640 1 --2- 172.21.15.32:0/1145106108 >> v2:172.21.15.39:3300/0 conn(0x7fcd180c90e0 0x7fcd180b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd17fff640 1 --2- 172.21.15.32:0/1145106108 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd181599e0 0x7fcd180aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd16ffd640 1 -- 172.21.15.32:0/1145106108 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1171411389 0 0) 0x7fcd180711d0 con 0x7fcd180b0b30 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd16ffd640 1 -- 172.21.15.32:0/1145106108 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd00003530 con 0x7fcd180b0b30 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd177fe640 1 -- 172.21.15.32:0/1145106108 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd181599e0 msgr2=0x7fcd180aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd177fe640 1 --2- 172.21.15.32:0/1145106108 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd181599e0 0x7fcd180aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd177fe640 1 -- 172.21.15.32:0/1145106108 >> v1:172.21.15.32:6789/0 conn(0x7fcd180b0b30 legacy=0x7fcd1815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd177fe640 1 -- 172.21.15.32:0/1145106108 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd180c9e80 con 0x7fcd180c90e0 2023-12-20T04:41:05.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.801+0000 7fcd17fff640 1 --2- 172.21.15.32:0/1145106108 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd181599e0 0x7fcd180aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:41:05.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd177fe640 1 --2- 172.21.15.32:0/1145106108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd180c90e0 0x7fcd180b05f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0097c0 tx=0x7fcd0c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2174754a43af6b01 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd16ffd640 1 -- 172.21.15.32:0/1145106108 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0ab600 con 0x7fcd180c90e0 2023-12-20T04:41:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd16ffd640 1 -- 172.21.15.32:0/1145106108 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd0c0a7050 con 0x7fcd180c90e0 2023-12-20T04:41:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd16ffd640 1 -- 172.21.15.32:0/1145106108 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd0c0ab960 con 0x7fcd180c90e0 2023-12-20T04:41:05.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd1e939640 1 -- 172.21.15.32:0/1145106108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd180c90e0 msgr2=0x7fcd180b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd1e939640 1 --2- 172.21.15.32:0/1145106108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd180c90e0 0x7fcd180b05f0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7fcd0c0097c0 tx=0x7fcd0c01c450 comp rx=0 tx=0).stop 2023-12-20T04:41:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd1e939640 1 -- 172.21.15.32:0/1145106108 shutdown_connections 2023-12-20T04:41:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd1e939640 1 --2- 172.21.15.32:0/1145106108 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd180c90e0 0x7fcd180b05f0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd1e939640 1 --2- 172.21.15.32:0/1145106108 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd181599e0 0x7fcd180aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd1e939640 1 -- 172.21.15.32:0/1145106108 >> 172.21.15.32:0/1145106108 conn(0x7fcd1806a320 msgr2=0x7fcd180ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd1e939640 1 -- 172.21.15.32:0/1145106108 shutdown_connections 2023-12-20T04:41:05.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.805+0000 7fcd1e939640 1 -- 172.21.15.32:0/1145106108 wait complete. 2023-12-20T04:41:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1e939640 1 Processor -- start 2023-12-20T04:41:05.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1e939640 1 -- start start 2023-12-20T04:41:05.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1e939640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd180b0b30 0x7fcd18150c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1e939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd180c90e0 0x7fcd181526d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1e939640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd181599e0 0x7fcd18152c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1e939640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd181484e0 con 0x7fcd181599e0 2023-12-20T04:41:05.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1e939640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd180cd040 con 0x7fcd180b0b30 2023-12-20T04:41:05.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1e939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd18153150 con 0x7fcd180c90e0 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1ced6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd181599e0 0x7fcd18152c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1ced6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd181599e0 0x7fcd18152c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51512/0 (socket says 172.21.15.32:51512) 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1ced6640 1 -- 172.21.15.32:0/368572348 learned_addr learned my addr 172.21.15.32:0/368572348 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd17fff640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd180b0b30 0x7fcd18150c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd177fe640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd180c90e0 0x7fcd181526d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1ced6640 1 -- 172.21.15.32:0/368572348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd180c90e0 msgr2=0x7fcd181526d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd180c90e0 0x7fcd181526d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1ced6640 1 -- 172.21.15.32:0/368572348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd180b0b30 msgr2=0x7fcd18150c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd180b0b30 0x7fcd18150c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1ced6640 1 -- 172.21.15.32:0/368572348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd080093b0 con 0x7fcd181599e0 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd181599e0 0x7fcd18152c10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000cb70 tx=0x7fcd1000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd14ff9640 1 -- 172.21.15.32:0/368572348 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000ad40 con 0x7fcd181599e0 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd14ff9640 1 -- 172.21.15.32:0/368572348 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd10012400 con 0x7fcd181599e0 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd14ff9640 1 -- 172.21.15.32:0/368572348 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd1000aea0 con 0x7fcd181599e0 2023-12-20T04:41:05.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1e939640 1 -- 172.21.15.32:0/368572348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd0c009420 con 0x7fcd181599e0 2023-12-20T04:41:05.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.809+0000 7fcd1e939640 1 -- 172.21.15.32:0/368572348 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd1814ee20 con 0x7fcd181599e0 2023-12-20T04:41:05.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd14ff9640 1 -- 172.21.15.32:0/368572348 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcd10007070 con 0x7fcd181599e0 2023-12-20T04:41:05.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd14ff9640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd04065b80 0x7fcd04068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd14ff9640 1 -- 172.21.15.32:0/368572348 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd100833d0 con 0x7fcd181599e0 2023-12-20T04:41:05.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd17fff640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd04065b80 0x7fcd04068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd21c9b000 1 Processor -- start 2023-12-20T04:41:05.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd21c9b000 1 -- start start 2023-12-20T04:41:05.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd17fff640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd04065b80 0x7fcd04068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009bc0 tx=0x7fcd08008040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:05.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9f02d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d990 0x562a9efe2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9efe31c0 0x562a9efe35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9ee06c90 con 0x562a9f02d070 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9ee38220 con 0x562a9f02d990 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a9ee357f0 con 0x562a9efe31c0 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9f02d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9f02d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51524/0 (socket says 172.21.15.32:51524) 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd17fff640 1 -- 172.21.15.32:0/140741912 learned_addr learned my addr 172.21.15.32:0/140741912 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/140741912 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9efe31c0 0x562a9efe35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.813+0000 7fcd177fe640 1 --2- 172.21.15.32:0/140741912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d990 0x562a9efe2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd17fff640 1 -- 172.21.15.32:0/140741912 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9efe31c0 msgr2=0x562a9efe35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd17fff640 1 --2- 172.21.15.32:0/140741912 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9efe31c0 0x562a9efe35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd17fff640 1 -- 172.21.15.32:0/140741912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d990 msgr2=0x562a9efe2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd17fff640 1 --2- 172.21.15.32:0/140741912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d990 0x562a9efe2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd17fff640 1 -- 172.21.15.32:0/140741912 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a9eb2d890 con 0x562a9f02d070 2023-12-20T04:41:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd17fff640 1 --2- 172.21.15.32:0/140741912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9f02d450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcd08011e80 tx=0x7fcd08014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a5295bd7e80f695 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fccfdffb640 1 -- 172.21.15.32:0/140741912 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08015e50 con 0x562a9f02d070 2023-12-20T04:41:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fccfdffb640 1 -- 172.21.15.32:0/140741912 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd08021440 con 0x562a9f02d070 2023-12-20T04:41:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fccfdffb640 1 -- 172.21.15.32:0/140741912 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd080217b0 con 0x562a9f02d070 2023-12-20T04:41:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 -- 172.21.15.32:0/140741912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 msgr2=0x562a9f02d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/140741912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9f02d450 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7fcd08011e80 tx=0x7fcd08014000 comp rx=0 tx=0).stop 2023-12-20T04:41:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 -- 172.21.15.32:0/140741912 shutdown_connections 2023-12-20T04:41:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/140741912 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9efe31c0 0x562a9efe35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/140741912 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d990 0x562a9efe2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/140741912 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9f02d450 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 -- 172.21.15.32:0/140741912 >> 172.21.15.32:0/140741912 conn(0x562a9efe9950 msgr2=0x562a9efe1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:05.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 -- 172.21.15.32:0/140741912 shutdown_connections 2023-12-20T04:41:05.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 -- 172.21.15.32:0/140741912 wait complete. 2023-12-20T04:41:05.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 -- start start 2023-12-20T04:41:05.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 Processor -- start 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9efe31c0 0x562a9efe8740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9efef080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9f02d990 0x562a9efef5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9ee310e0 con 0x562a9f02d070 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9edd0310 con 0x562a9efe31c0 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a9edee770 con 0x562a9f02d990 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd21c9b000 10 client.0 _ll_register_callbacks cb 0x562a9efceb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9efef080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9efef080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51540/0 (socket says 172.21.15.32:51540) 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd177fe640 1 -- 172.21.15.32:0/16105473 learned_addr learned my addr 172.21.15.32:0/16105473 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9f02d990 0x562a9efef5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.817+0000 7fcd17fff640 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9efe31c0 0x562a9efe8740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd1ced6640 1 -- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9efe31c0 msgr2=0x562a9efe8740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9efe31c0 0x562a9efe8740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd1ced6640 1 -- 172.21.15.32:0/16105473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 msgr2=0x562a9efef080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9efef080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd1ced6640 1 -- 172.21.15.32:0/16105473 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd08011b60 con 0x562a9f02d990 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd177fe640 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9efef080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9f02d990 0x562a9efef5c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcd100816e0 tx=0x7fcd10058790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=62f21a68fd810b17 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fccf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.39:3301/0 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fccf6ffd640 1 -- 172.21.15.32:0/16105473 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10053d90 con 0x562a9f02d990 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fccf6ffd640 1 -- 172.21.15.32:0/16105473 <== mon.2 v2:172.21.15.39:3301/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd10053ef0 con 0x562a9f02d990 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 -- 172.21.15.32:0/16105473 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a9ee00c10 con 0x562a9f02d990 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fccf6ffd640 1 -- 172.21.15.32:0/16105473 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd100812b0 con 0x562a9f02d990 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 10 client.5471 fetch_fsmap learned FSMap version 24 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 -- 172.21.15.32:0/16105473 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x562a9edf6260 con 0x562a9f02d990 2023-12-20T04:41:05.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fccf6ffd640 1 -- 172.21.15.32:0/16105473 <== mon.2 v2:172.21.15.39:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562a9ee00c10 con 0x562a9f02d990 2023-12-20T04:41:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fccf6ffd640 1 -- 172.21.15.32:0/16105473 <== mon.2 v2:172.21.15.39:3301/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcd1005c3f0 con 0x562a9f02d990 2023-12-20T04:41:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 10 client.5471 fetch_fsmap finished waiting for FSMap version 24 2023-12-20T04:41:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 10 client.5471 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-20T04:41:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 20 client.5471 populate_metadata read hostname 'smithi032' 2023-12-20T04:41:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:6832/192084837,v1:172.21.15.39:6834/192084837] conn(0x562a9f0348d0 0x562a9f034cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 4 client.5471 mds_command: new command op to 5895 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T04:41:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 -- 172.21.15.32:0/16105473 --> [v2:172.21.15.39:6832/192084837,v1:172.21.15.39:6834/192084837] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x562a9ee2fa30 con 0x562a9f0348d0 2023-12-20T04:41:05.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd17fff640 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:6832/192084837,v1:172.21.15.39:6834/192084837] conn(0x562a9f0348d0 0x562a9f034cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd17fff640 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:6832/192084837,v1:172.21.15.39:6834/192084837] conn(0x562a9f0348d0 0x562a9f034cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdd2405475b5ac6b server_cookie=568941e040644bc2 in_seq=0 out_seq=0 2023-12-20T04:41:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fccf6ffd640 10 client.5471 ms_handle_connect on v2:172.21.15.39:6832/192084837 2023-12-20T04:41:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fccf6ffd640 1 -- 172.21.15.32:0/16105473 <== mds.0 v2:172.21.15.39:6832/192084837 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x562a9ee2fa30 con 0x562a9f0348d0 2023-12-20T04:41:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fccf6ffd640 10 client.5471 handle_command_reply: tid=0 2023-12-20T04:41:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 client.5471 shutdown 2023-12-20T04:41:05.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 -- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9f02d990 msgr2=0x562a9efef5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9f02d990 0x562a9efef5c0 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7fcd100816e0 tx=0x7fcd10058790 comp rx=0 tx=0).stop 2023-12-20T04:41:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 -- 172.21.15.32:0/16105473 shutdown_connections 2023-12-20T04:41:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:6832/192084837,v1:172.21.15.39:6834/192084837] conn(0x562a9f0348d0 0x562a9f034cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9f02d990 0x562a9efef5c0 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d070 0x562a9efef080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/16105473 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9efe31c0 0x562a9efe8740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fcd21c9b000 1 -- 172.21.15.32:0/16105473 >> 172.21.15.32:0/16105473 conn(0x562a9efe9950 msgr2=0x562a9efe1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.821+0000 7fccf6ffd640 0 client.5471 ms_handle_reset on v2:172.21.15.39:6832/192084837 2023-12-20T04:41:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.825+0000 7fcd21c9b000 1 -- 172.21.15.32:0/16105473 shutdown_connections 2023-12-20T04:41:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.825+0000 7fcd21c9b000 1 -- 172.21.15.32:0/16105473 wait complete. 2023-12-20T04:41:05.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.825+0000 7fcd21c9b000 20 client.5471 trim_cache size 0 max 16384 2023-12-20T04:41:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.837+0000 7fcd21c9b000 1 Processor -- start 2023-12-20T04:41:05.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.837+0000 7fcd21c9b000 1 -- start start 2023-12-20T04:41:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9efe31c0 0x562a9efeefd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d070 0x562a9efef510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d990 0x562a9f02b070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9eaee940 con 0x562a9f02d990 2023-12-20T04:41:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9efe6880 con 0x562a9f02d070 2023-12-20T04:41:05.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a9ee350e0 con 0x562a9efe31c0 2023-12-20T04:41:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd1ced6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d990 0x562a9f02b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd1ced6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d990 0x562a9f02b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51554/0 (socket says 172.21.15.32:51554) 2023-12-20T04:41:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd1ced6640 1 -- 172.21.15.32:0/676071895 learned_addr learned my addr 172.21.15.32:0/676071895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd17fff640 1 --2- 172.21.15.32:0/676071895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9efe31c0 0x562a9efeefd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd177fe640 1 --2- 172.21.15.32:0/676071895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d070 0x562a9efef510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd1ced6640 1 -- 172.21.15.32:0/676071895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9efe31c0 msgr2=0x562a9efeefd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/676071895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9efe31c0 0x562a9efeefd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd1ced6640 1 -- 172.21.15.32:0/676071895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d070 msgr2=0x562a9efef510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/676071895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d070 0x562a9efef510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd1ced6640 1 -- 172.21.15.32:0/676071895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x562a9efe60e0 con 0x562a9f02d990 2023-12-20T04:41:05.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/676071895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d990 0x562a9f02b070 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000cfd0 tx=0x7fcd10056000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=add3713672105cbf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:05.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fccfe7fc640 1 -- 172.21.15.32:0/676071895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10050be0 con 0x562a9f02d990 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fccfe7fc640 1 -- 172.21.15.32:0/676071895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd100548a0 con 0x562a9f02d990 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fccfe7fc640 1 -- 172.21.15.32:0/676071895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd10050fb0 con 0x562a9f02d990 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- 172.21.15.32:0/676071895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d990 msgr2=0x562a9f02b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/676071895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d990 0x562a9f02b070 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000cfd0 tx=0x7fcd10056000 comp rx=0 tx=0).stop 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- 172.21.15.32:0/676071895 shutdown_connections 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/676071895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9f02d990 0x562a9f02b070 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/676071895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d070 0x562a9efef510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/676071895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9efe31c0 0x562a9efeefd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- 172.21.15.32:0/676071895 >> 172.21.15.32:0/676071895 conn(0x562a9efe9950 msgr2=0x562a9efee8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- 172.21.15.32:0/676071895 shutdown_connections 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- 172.21.15.32:0/676071895 wait complete. 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- start start 2023-12-20T04:41:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 Processor -- start 2023-12-20T04:41:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9efe31c0 0x562a9f089450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d070 0x562a9f089990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9f02d990 0x562a9f03eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9ee2ff70 con 0x562a9efe31c0 2023-12-20T04:41:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x562a9efe5fa0 con 0x562a9f02d070 2023-12-20T04:41:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x562a9edf2ec0 con 0x562a9f02d990 2023-12-20T04:41:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd21c9b000 10 client.0 _ll_register_callbacks cb 0x562a9eddf640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:41:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9efe31c0 0x562a9f089450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9efe31c0 0x562a9f089450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51562/0 (socket says 172.21.15.32:51562) 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.841+0000 7fcd17fff640 1 -- 172.21.15.32:0/2776430828 learned_addr learned my addr 172.21.15.32:0/2776430828 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd177fe640 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d070 0x562a9f089990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd1ced6640 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9f02d990 0x562a9f03eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd17fff640 1 -- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9f02d990 msgr2=0x562a9f03eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd17fff640 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9f02d990 0x562a9f03eeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd17fff640 1 -- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d070 msgr2=0x562a9f089990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd17fff640 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d070 0x562a9f089990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd17fff640 1 -- 172.21.15.32:0/2776430828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x562a9edf6260 con 0x562a9efe31c0 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd17fff640 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9efe31c0 0x562a9f089450 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcd08006cc0 tx=0x7fcd080046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8be800ec29c3090 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fccf6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fccf6ffd640 1 -- 172.21.15.32:0/2776430828 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd08030040 con 0x562a9efe31c0 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 -- 172.21.15.32:0/2776430828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x562a9edf6e00 con 0x562a9efe31c0 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fccf6ffd640 1 -- 172.21.15.32:0/2776430828 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcd080373b0 con 0x562a9efe31c0 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fccf6ffd640 1 -- 172.21.15.32:0/2776430828 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=24) v2 ==== 24+0+0 (secure 0 0 0) 0x7fcd08037790 con 0x562a9efe31c0 2023-12-20T04:41:05.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 10 client.6048 fetch_fsmap learned FSMap version 24 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 -- 172.21.15.32:0/2776430828 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=24,monmap=2+}) v3 -- 0x562a9eddc3b0 con 0x562a9efe31c0 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fccf6ffd640 1 -- 172.21.15.32:0/2776430828 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x562a9edf6e00 con 0x562a9efe31c0 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fccf6ffd640 1 -- 172.21.15.32:0/2776430828 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 24) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fcd080378f0 con 0x562a9efe31c0 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 10 client.6048 fetch_fsmap finished waiting for FSMap version 24 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 10 client.6048 resolve_mds: resolved name 'b' to daemon mds.b 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 20 client.6048 populate_metadata read hostname 'smithi032' 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:6832/192084837,v1:172.21.15.39:6834/192084837] conn(0x562a9f03f840 0x562a9f03fc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 4 client.6048 mds_command: new command op to 5895 tid=0[{"prefix": "status"}] 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 -- 172.21.15.32:0/2776430828 --> [v2:172.21.15.39:6832/192084837,v1:172.21.15.39:6834/192084837] -- command(tid 0: {"prefix": "status"}) v1 -- 0x562a9ee2a160 con 0x562a9f03f840 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd177fe640 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:6832/192084837,v1:172.21.15.39:6834/192084837] conn(0x562a9f03f840 0x562a9f03fc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fccf6ffd640 10 client.6048 ms_handle_connect on v2:172.21.15.39:6832/192084837 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd177fe640 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:6832/192084837,v1:172.21.15.39:6834/192084837] conn(0x562a9f03f840 0x562a9f03fc20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3f10a953417770a0 server_cookie=5ce9a57d6b3a74ae in_seq=0 out_seq=0 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fccf6ffd640 1 -- 172.21.15.32:0/2776430828 <== mds.0 v2:172.21.15.39:6832/192084837 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x562a9ee2a160 con 0x562a9f03f840 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fccf6ffd640 10 client.6048 handle_command_reply: tid=0 2023-12-20T04:41:05.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 client.6048 shutdown 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 -- 172.21.15.32:0/2776430828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9efe31c0 msgr2=0x562a9f089450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9efe31c0 0x562a9f089450 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7fcd08006cc0 tx=0x7fcd080046d0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 -- 172.21.15.32:0/2776430828 shutdown_connections 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:6832/192084837,v1:172.21.15.39:6834/192084837] conn(0x562a9f03f840 0x562a9f03fc20 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x562a9f02d990 0x562a9f03eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x562a9f02d070 0x562a9f089990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 --2- 172.21.15.32:0/2776430828 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x562a9efe31c0 0x562a9f089450 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fccf6ffd640 0 client.6048 ms_handle_reset on v2:172.21.15.39:6832/192084837 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 -- 172.21.15.32:0/2776430828 >> 172.21.15.32:0/2776430828 conn(0x562a9efe9950 msgr2=0x562a9efee8c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 -- 172.21.15.32:0/2776430828 shutdown_connections 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 1 -- 172.21.15.32:0/2776430828 wait complete. 2023-12-20T04:41:05.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.845+0000 7fcd21c9b000 20 client.6048 trim_cache size 0 max 16384 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 5895, 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9271059519999998, 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 24, 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 61, 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 61, 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 11.164988513000001 2023-12-20T04:41:05.850 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:41:05.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 -- 172.21.15.32:0/368572348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd04065b80 msgr2=0x7fcd04068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd04065b80 0x7fcd04068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009bc0 tx=0x7fcd08008040 comp rx=0 tx=0).stop 2023-12-20T04:41:05.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 -- 172.21.15.32:0/368572348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd181599e0 msgr2=0x7fcd18152c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:05.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd181599e0 0x7fcd18152c10 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7fcd1000cb70 tx=0x7fcd1000a430 comp rx=0 tx=0).stop 2023-12-20T04:41:05.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 -- 172.21.15.32:0/368572348 shutdown_connections 2023-12-20T04:41:05.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd04065b80 0x7fcd04068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd181599e0 0x7fcd18152c10 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd180c90e0 0x7fcd181526d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 --2- 172.21.15.32:0/368572348 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd180b0b30 0x7fcd18150c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:05.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 -- 172.21.15.32:0/368572348 >> 172.21.15.32:0/368572348 conn(0x7fcd1806a320 msgr2=0x7fcd180ae970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:05.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 -- 172.21.15.32:0/368572348 shutdown_connections 2023-12-20T04:41:05.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:05.849+0000 7fcd1e939640 1 -- 172.21.15.32:0/368572348 wait complete. 2023-12-20T04:41:05.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T04:41:06.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.009+0000 7f6151fa2640 1 Processor -- start 2023-12-20T04:41:06.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.009+0000 7f6151fa2640 1 -- start start 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6151fa2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f614c06c8f0 0x7f614c160cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6151fa2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f614c15e130 0x7f614c0cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6151fa2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f614c0cba10 0x7f614c0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6151fa2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614c161290 con 0x7f614c06c8f0 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6151fa2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f614c161570 con 0x7f614c15e130 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6151fa2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c161850 con 0x7f614c0cba10 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6150fa0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f614c06c8f0 0x7f614c160cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6150fa0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f614c06c8f0 0x7f614c160cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51576/0 (socket says 172.21.15.32:51576) 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6150fa0640 1 -- 172.21.15.32:0/1645716448 learned_addr learned my addr 172.21.15.32:0/1645716448 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6150fa0640 1 -- 172.21.15.32:0/1645716448 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f614c0cba10 msgr2=0x7f614c0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f614bfff640 1 --2- 172.21.15.32:0/1645716448 >> v2:172.21.15.39:3300/0 conn(0x7f614c15e130 0x7f614c0cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:06.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f61517a1640 1 --2- 172.21.15.32:0/1645716448 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f614c0cba10 0x7f614c0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6150fa0640 1 --2- 172.21.15.32:0/1645716448 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f614c0cba10 0x7f614c0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6150fa0640 1 -- 172.21.15.32:0/1645716448 >> v2:172.21.15.39:3300/0 conn(0x7f614c15e130 msgr2=0x7f614c0cb4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6150fa0640 1 --2- 172.21.15.32:0/1645716448 >> v2:172.21.15.39:3300/0 conn(0x7f614c15e130 0x7f614c0cb4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f6150fa0640 1 -- 172.21.15.32:0/1645716448 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f614c0ce9d0 con 0x7f614c06c8f0 2023-12-20T04:41:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f614bfff640 1 --2- 172.21.15.32:0/1645716448 >> v2:172.21.15.39:3300/0 conn(0x7f614c15e130 0x7f614c0cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:41:06.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.013+0000 7f61517a1640 1 --2- 172.21.15.32:0/1645716448 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f614c0cba10 0x7f614c0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:41:06.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6150fa0640 1 --2- 172.21.15.32:0/1645716448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c06c8f0 0x7f614c160cb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f614c06abf0 tx=0x7f614001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7b6c33b20be69b0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:06.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f614b7fe640 1 -- 172.21.15.32:0/1645716448 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61400ab600 con 0x7f614c06c8f0 2023-12-20T04:41:06.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f614b7fe640 1 -- 172.21.15.32:0/1645716448 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61400a7050 con 0x7f614c06c8f0 2023-12-20T04:41:06.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f614b7fe640 1 -- 172.21.15.32:0/1645716448 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61400ab980 con 0x7f614c06c8f0 2023-12-20T04:41:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6151fa2640 1 -- 172.21.15.32:0/1645716448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c06c8f0 msgr2=0x7f614c160cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6151fa2640 1 --2- 172.21.15.32:0/1645716448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c06c8f0 0x7f614c160cb0 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f614c06abf0 tx=0x7f614001c500 comp rx=0 tx=0).stop 2023-12-20T04:41:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6151fa2640 1 -- 172.21.15.32:0/1645716448 shutdown_connections 2023-12-20T04:41:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6151fa2640 1 --2- 172.21.15.32:0/1645716448 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f614c0cba10 0x7f614c0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6151fa2640 1 --2- 172.21.15.32:0/1645716448 >> v2:172.21.15.39:3300/0 conn(0x7f614c15e130 0x7f614c0cb4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6151fa2640 1 --2- 172.21.15.32:0/1645716448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c06c8f0 0x7f614c160cb0 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:06.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6151fa2640 1 -- 172.21.15.32:0/1645716448 >> 172.21.15.32:0/1645716448 conn(0x7f614c06e280 msgr2=0x7f614c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6151fa2640 1 -- 172.21.15.32:0/1645716448 shutdown_connections 2023-12-20T04:41:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6151fa2640 1 -- 172.21.15.32:0/1645716448 wait complete. 2023-12-20T04:41:06.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6151fa2640 1 Processor -- start 2023-12-20T04:41:06.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.017+0000 7f6151fa2640 1 -- start start 2023-12-20T04:41:06.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f6151fa2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f614c06c8f0 0x7f614c1566c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:06.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f6151fa2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f614c0cba10 0x7f614c158240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:06.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f6151fa2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c15e130 0x7f614c15f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:06.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f6151fa2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0c5d90 con 0x7f614c15e130 2023-12-20T04:41:06.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f6151fa2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0c5110 con 0x7f614c0cba10 2023-12-20T04:41:06.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f6151fa2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f614c0c53f0 con 0x7f614c06c8f0 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61517a1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c15e130 0x7f614c15f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f6150fa0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f614c06c8f0 0x7f614c1566c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61517a1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c15e130 0x7f614c15f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51588/0 (socket says 172.21.15.32:51588) 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61517a1640 1 -- 172.21.15.32:0/1429987199 learned_addr learned my addr 172.21.15.32:0/1429987199 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f614bfff640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f614c0cba10 0x7f614c158240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61517a1640 1 -- 172.21.15.32:0/1429987199 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f614c06c8f0 msgr2=0x7f614c1566c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61517a1640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f614c06c8f0 0x7f614c1566c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61517a1640 1 -- 172.21.15.32:0/1429987199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f614c0cba10 msgr2=0x7f614c158240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61517a1640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f614c0cba10 0x7f614c158240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61517a1640 1 -- 172.21.15.32:0/1429987199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61400094d0 con 0x7f614c15e130 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61517a1640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c15e130 0x7f614c15f3b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f614400cea0 tx=0x7f614400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61497fa640 1 -- 172.21.15.32:0/1429987199 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6144003ba0 con 0x7f614c15e130 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61497fa640 1 -- 172.21.15.32:0/1429987199 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6144003d20 con 0x7f614c15e130 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f6151fa2640 1 -- 172.21.15.32:0/1429987199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f614c0c56d0 con 0x7f614c15e130 2023-12-20T04:41:06.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61497fa640 1 -- 172.21.15.32:0/1429987199 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f614400d880 con 0x7f614c15e130 2023-12-20T04:41:06.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f6151fa2640 1 -- 172.21.15.32:0/1429987199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f614c156c00 con 0x7f614c15e130 2023-12-20T04:41:06.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f6151fa2640 1 -- 172.21.15.32:0/1429987199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f614c074100 con 0x7f614c15e130 2023-12-20T04:41:06.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.021+0000 7f61497fa640 1 -- 172.21.15.32:0/1429987199 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f614400d9e0 con 0x7f614c15e130 2023-12-20T04:41:06.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.025+0000 7f61497fa640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6130065b80 0x7f6130068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:06.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.025+0000 7f61497fa640 1 -- 172.21.15.32:0/1429987199 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6144083ad0 con 0x7f614c15e130 2023-12-20T04:41:06.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.025+0000 7f6150fa0640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6130065b80 0x7f6130068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:06.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.025+0000 7f6150fa0640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6130065b80 0x7f6130068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f61400ad000 tx=0x7f6140008040 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:06.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.029+0000 7f61497fa640 1 -- 172.21.15.32:0/1429987199 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f61440514c0 con 0x7f614c15e130 2023-12-20T04:41:06.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.221+0000 7f6151fa2640 1 -- 172.21.15.32:0/1429987199 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f614c0b6790 con 0x7f614c15e130 2023-12-20T04:41:06.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.221+0000 7f61497fa640 1 -- 172.21.15.32:0/1429987199 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v24) v1 ==== 53+0+73 (secure 0 0 0) 0x7f6144054ae0 con 0x7f614c15e130 2023-12-20T04:41:06.222 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T04:41:06.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 -- 172.21.15.32:0/1429987199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6130065b80 msgr2=0x7f6130068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:06.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6130065b80 0x7f6130068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f61400ad000 tx=0x7f6140008040 comp rx=0 tx=0).stop 2023-12-20T04:41:06.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 -- 172.21.15.32:0/1429987199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c15e130 msgr2=0x7f614c15f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:06.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c15e130 0x7f614c15f3b0 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f614400cea0 tx=0x7f614400cf90 comp rx=0 tx=0).stop 2023-12-20T04:41:06.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 -- 172.21.15.32:0/1429987199 shutdown_connections 2023-12-20T04:41:06.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6130065b80 0x7f6130068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:06.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f614c15e130 0x7f614c15f3b0 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:06.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f614c0cba10 0x7f614c158240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:06.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 --2- 172.21.15.32:0/1429987199 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f614c06c8f0 0x7f614c1566c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:06.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 -- 172.21.15.32:0/1429987199 >> 172.21.15.32:0/1429987199 conn(0x7f614c06e280 msgr2=0x7f614c0b1b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:06.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 -- 172.21.15.32:0/1429987199 shutdown_connections 2023-12-20T04:41:06.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:06.225+0000 7f6151fa2640 1 -- 172.21.15.32:0/1429987199 wait complete. 2023-12-20T04:41:06.243 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T04:41:06.243 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T04:41:06.243 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T04:41:06.244 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T04:41:06.244 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T04:41:06.244 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T04:41:06.244 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T04:41:06.244 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T04:41:06.244 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:41:06.244 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:06.244 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T04:41:06.254 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41594sec preferred_lft 41594sec 2023-12-20T04:41:06.255 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:41:06.256 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T04:41:06.257 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:41:06.258 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:06.258 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T04:41:06.318 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T04:41:06.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:06.320 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T04:41:06.372 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T04:41:06.372 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T04:41:06.373 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T04:41:06.373 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T04:41:06.373 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T04:41:06.373 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T04:41:06.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:06.373 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:41:06.373 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:41:06.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T04:41:06.373 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T04:41:06.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T04:41:06.374 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:41:06.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:06.467 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T04:41:06.477 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T04:41:06.478 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T04:41:06.478 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:41:06.478 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:06.478 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T04:41:06.544 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:41:06.544 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T04:41:06.595 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:41:06.595 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:41:06.596 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:41:06.650 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T04:41:06.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:06.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:41:06.750 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:41:06.750 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:41:06.763 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:41:06.763 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:41:06.764 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T04:41:06.764 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T04:41:06.764 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T04:41:06.767 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T04:41:06.767 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:06.767 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:41:06.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:06.789 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:41:06.857 INFO:teuthology.orchestra.run.smithi032.stdout:client5929 2023-12-20T04:41:06.857 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:41:06.857 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:41:06.911 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T04:41:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.097+0000 7fe4958c4640 1 Processor -- start 2023-12-20T04:41:07.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.097+0000 7fe4958c4640 1 -- start start 2023-12-20T04:41:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe4958c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4900d2740 0x7fe4900cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe4958c4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe4900cfab0 0x7fe4900b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe4958c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4900cfec0 con 0x7fe49015cdb0 2023-12-20T04:41:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe4958c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49015d190 con 0x7fe4900d2740 2023-12-20T04:41:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe4958c4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4900d16a0 con 0x7fe4900cfab0 2023-12-20T04:41:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe4867fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4900d2740 0x7fe4900cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe48effd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe49015cdb0 0x7fe4900d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35866/0 (socket says 172.21.15.32:35866) 2023-12-20T04:41:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe48effd640 1 -- 172.21.15.32:0/3412777258 learned_addr learned my addr 172.21.15.32:0/3412777258 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:41:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe48f7fe640 1 --2- 172.21.15.32:0/3412777258 >> v2:172.21.15.39:3300/0 conn(0x7fe4900cfab0 0x7fe4900b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe4867fc640 1 -- 172.21.15.32:0/3412777258 >> v2:172.21.15.39:3300/0 conn(0x7fe4900cfab0 msgr2=0x7fe4900b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe4867fc640 1 --2- 172.21.15.32:0/3412777258 >> v2:172.21.15.39:3300/0 conn(0x7fe4900cfab0 0x7fe4900b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe4867fc640 1 -- 172.21.15.32:0/3412777258 >> v1:172.21.15.32:6789/0 conn(0x7fe49015cdb0 legacy=0x7fe4900d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe4867fc640 1 -- 172.21.15.32:0/3412777258 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4900d1980 con 0x7fe4900d2740 2023-12-20T04:41:07.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.101+0000 7fe48e7fc640 1 -- 172.21.15.32:0/3412777258 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 109131631 0 0) 0x7fe4900cfec0 con 0x7fe49015cdb0 2023-12-20T04:41:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe4867fc640 1 --2- 172.21.15.32:0/3412777258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900d2740 0x7fe4900cf570 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe49006c100 tx=0x7fe47801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90ebfcfaa6adb01b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe48e7fc640 1 -- 172.21.15.32:0/3412777258 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4780a8950 con 0x7fe4900d2740 2023-12-20T04:41:07.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe48e7fc640 1 -- 172.21.15.32:0/3412777258 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4780a8ab0 con 0x7fe4900d2740 2023-12-20T04:41:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe48e7fc640 1 -- 172.21.15.32:0/3412777258 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4780a8e40 con 0x7fe4900d2740 2023-12-20T04:41:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe4958c4640 1 -- 172.21.15.32:0/3412777258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900d2740 msgr2=0x7fe4900cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe4958c4640 1 --2- 172.21.15.32:0/3412777258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900d2740 0x7fe4900cf570 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7fe49006c100 tx=0x7fe47801ec50 comp rx=0 tx=0).stop 2023-12-20T04:41:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe4958c4640 1 -- 172.21.15.32:0/3412777258 shutdown_connections 2023-12-20T04:41:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe4958c4640 1 --2- 172.21.15.32:0/3412777258 >> v2:172.21.15.39:3300/0 conn(0x7fe4900cfab0 0x7fe4900b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe4958c4640 1 --2- 172.21.15.32:0/3412777258 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900d2740 0x7fe4900cf570 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe4958c4640 1 -- 172.21.15.32:0/3412777258 >> 172.21.15.32:0/3412777258 conn(0x7fe490065ee0 msgr2=0x7fe49006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe4958c4640 1 -- 172.21.15.32:0/3412777258 shutdown_connections 2023-12-20T04:41:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe4958c4640 1 -- 172.21.15.32:0/3412777258 wait complete. 2023-12-20T04:41:07.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe4958c4640 1 Processor -- start 2023-12-20T04:41:07.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.105+0000 7fe4958c4640 1 -- start start 2023-12-20T04:41:07.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4958c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe4900cfab0 0x7fe49015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4958c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900d2740 0x7fe49015dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4958c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe49015cdb0 0x7fe490158fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4958c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe490156420 con 0x7fe4900d2740 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4958c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe4901f2980 con 0x7fe49015cdb0 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4958c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4901560a0 con 0x7fe4900cfab0 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4867fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900d2740 0x7fe49015dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4867fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900d2740 0x7fe49015dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51608/0 (socket says 172.21.15.32:51608) 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4867fc640 1 -- 172.21.15.32:0/2265420911 learned_addr learned my addr 172.21.15.32:0/2265420911 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe48effd640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe4900cfab0 0x7fe49015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe48f7fe640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe49015cdb0 0x7fe490158fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4867fc640 1 -- 172.21.15.32:0/2265420911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe4900cfab0 msgr2=0x7fe49015d520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4867fc640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe4900cfab0 0x7fe49015d520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4867fc640 1 -- 172.21.15.32:0/2265420911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe49015cdb0 msgr2=0x7fe490158fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4867fc640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe49015cdb0 0x7fe490158fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4867fc640 1 -- 172.21.15.32:0/2265420911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe478009420 con 0x7fe4900d2740 2023-12-20T04:41:07.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4867fc640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900d2740 0x7fe49015dca0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe47801b860 tx=0x7fe47801d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe48cff9640 1 -- 172.21.15.32:0/2265420911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4780a8950 con 0x7fe4900d2740 2023-12-20T04:41:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe48cff9640 1 -- 172.21.15.32:0/2265420911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe478006e20 con 0x7fe4900d2740 2023-12-20T04:41:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4958c4640 1 -- 172.21.15.32:0/2265420911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49015e330 con 0x7fe4900d2740 2023-12-20T04:41:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe48cff9640 1 -- 172.21.15.32:0/2265420911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe478004230 con 0x7fe4900d2740 2023-12-20T04:41:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4958c4640 1 -- 172.21.15.32:0/2265420911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49015e8f0 con 0x7fe4900d2740 2023-12-20T04:41:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe4958c4640 1 -- 172.21.15.32:0/2265420911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe490156560 con 0x7fe4900d2740 2023-12-20T04:41:07.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.109+0000 7fe48cff9640 1 -- 172.21.15.32:0/2265420911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe478004390 con 0x7fe4900d2740 2023-12-20T04:41:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.113+0000 7fe48cff9640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe470065b80 0x7fe470068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.113+0000 7fe48cff9640 1 -- 172.21.15.32:0/2265420911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe4781215a0 con 0x7fe4900d2740 2023-12-20T04:41:07.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.113+0000 7fe48effd640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe470065b80 0x7fe470068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.113+0000 7fe48effd640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe470065b80 0x7fe470068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe480000f70 tx=0x7fe480007df0 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:07.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.117+0000 7fe48cff9640 1 -- 172.21.15.32:0/2265420911 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe490156560 con 0x7fe4900d2740 2023-12-20T04:41:07.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.297+0000 7fe4958c4640 1 -- 172.21.15.32:0/2265420911 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4900d1f40 con 0x7fe470065b80 2023-12-20T04:41:07.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.297+0000 7fe48cff9640 1 -- 172.21.15.32:0/2265420911 <== mgr.5664 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe4800072a0 con 0x7fe470065b80 2023-12-20T04:41:07.297 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:41:07.297 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:41:07.297 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T04:41:07.297 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:41:07.297 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:41:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.297+0000 7fe4958c4640 1 -- 172.21.15.32:0/2265420911 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe470065b80 msgr2=0x7fe470068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.297+0000 7fe4958c4640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe470065b80 0x7fe470068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fe480000f70 tx=0x7fe480007df0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.297+0000 7fe4958c4640 1 -- 172.21.15.32:0/2265420911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900d2740 msgr2=0x7fe49015dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.297+0000 7fe4958c4640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900d2740 0x7fe49015dca0 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7fe47801b860 tx=0x7fe47801d490 comp rx=0 tx=0).stop 2023-12-20T04:41:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.301+0000 7fe4958c4640 1 -- 172.21.15.32:0/2265420911 shutdown_connections 2023-12-20T04:41:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.301+0000 7fe4958c4640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe470065b80 0x7fe470068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.301+0000 7fe4958c4640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe49015cdb0 0x7fe490158fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.301+0000 7fe4958c4640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900d2740 0x7fe49015dca0 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.301+0000 7fe4958c4640 1 --2- 172.21.15.32:0/2265420911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe4900cfab0 0x7fe49015d520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.301+0000 7fe4958c4640 1 -- 172.21.15.32:0/2265420911 >> 172.21.15.32:0/2265420911 conn(0x7fe490065ee0 msgr2=0x7fe49015b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:07.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.301+0000 7fe4958c4640 1 -- 172.21.15.32:0/2265420911 shutdown_connections 2023-12-20T04:41:07.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.301+0000 7fe4958c4640 1 -- 172.21.15.32:0/2265420911 wait complete. 2023-12-20T04:41:07.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T04:41:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb91b841640 1 Processor -- start 2023-12-20T04:41:07.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb91b841640 1 -- start start 2023-12-20T04:41:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb91b841640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9140d3300 0x7fb9140d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb91b841640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb9140d1300 0x7fb9140b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb91b841640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9140d2650 con 0x7fb91415ce60 2023-12-20T04:41:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb91b841640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9140d1a40 con 0x7fb9140d1300 2023-12-20T04:41:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb91b841640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140d1d20 con 0x7fb9140d3300 2023-12-20T04:41:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb9195dd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb91415ce60 0x7fb9140d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35882/0 (socket says 172.21.15.32:35882) 2023-12-20T04:41:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb9195dd640 1 -- 172.21.15.32:0/227857406 learned_addr learned my addr 172.21.15.32:0/227857406 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:41:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb919dde640 1 --2- 172.21.15.32:0/227857406 >> v2:172.21.15.39:3300/0 conn(0x7fb9140d1300 0x7fb9140b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb918ddc640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9140d3300 0x7fb9140d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 195001463 0 0) 0x7fb9140d2650 con 0x7fb91415ce60 2023-12-20T04:41:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb900003530 con 0x7fb91415ce60 2023-12-20T04:41:07.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3854793744 0 0) 0x7fb900003530 con 0x7fb91415ce60 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9140d3300 msgr2=0x7fb9140d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9140d3300 0x7fb9140d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 >> v2:172.21.15.39:3300/0 conn(0x7fb9140d1300 msgr2=0x7fb9140b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 --2- 172.21.15.32:0/227857406 >> v2:172.21.15.39:3300/0 conn(0x7fb9140d1300 0x7fb9140b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9140d2000 con 0x7fb91415ce60 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fb910001680 con 0x7fb91415ce60 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 >> v1:172.21.15.32:6789/0 conn(0x7fb91415ce60 legacy=0x7fb9140d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb900003980 0x7fb900003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9140d3300 0x7fb9000047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb900004d10 0x7fb900005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140d1a40 con 0x7fb900004d10 2023-12-20T04:41:07.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9140d1d20 con 0x7fb900003980 2023-12-20T04:41:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.465+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb90000b9c0 con 0x7fb9140d3300 2023-12-20T04:41:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb919dde640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb900004d10 0x7fb900005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb9195dd640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb900003980 0x7fb900003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb918ddc640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9140d3300 0x7fb9000047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb919dde640 1 -- 172.21.15.32:0/227857406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9140d3300 msgr2=0x7fb9000047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb919dde640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9140d3300 0x7fb9000047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb919dde640 1 -- 172.21.15.32:0/227857406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb900003980 msgr2=0x7fb900003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb919dde640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb900003980 0x7fb900003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb919dde640 1 -- 172.21.15.32:0/227857406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb900003530 con 0x7fb900004d10 2023-12-20T04:41:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb919dde640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb900004d10 0x7fb900005190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009fc0 tx=0x7fb90c01cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d479ff05d8e15c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0a5cb0 con 0x7fb900004d10 2023-12-20T04:41:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c00f040 con 0x7fb900004d10 2023-12-20T04:41:07.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.469+0000 7fb90bfff640 1 -- 172.21.15.32:0/227857406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0ad680 con 0x7fb900004d10 2023-12-20T04:41:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 -- 172.21.15.32:0/227857406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb900004d10 msgr2=0x7fb900005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 --2- 172.21.15.32:0/227857406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb900004d10 0x7fb900005190 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7fb90c009fc0 tx=0x7fb90c01cca0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb919dde640 1 -- 172.21.15.32:0/227857406 reap_dead start 2023-12-20T04:41:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 -- 172.21.15.32:0/227857406 shutdown_connections 2023-12-20T04:41:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 -- 172.21.15.32:0/227857406 >> 172.21.15.32:0/227857406 conn(0x7fb91406b5e0 msgr2=0x7fb91406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:07.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 -- 172.21.15.32:0/227857406 shutdown_connections 2023-12-20T04:41:07.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 -- 172.21.15.32:0/227857406 wait complete. 2023-12-20T04:41:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 Processor -- start 2023-12-20T04:41:07.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 -- start start 2023-12-20T04:41:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb900004d10 0x7fb91414dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9141e48b0 0x7fb9141e2d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9141e3290 0x7fb9141e56b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb91414e190 con 0x7fb9141e3290 2023-12-20T04:41:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9141e4420 con 0x7fb9141e48b0 2023-12-20T04:41:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb91b841640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9141e3810 con 0x7fb900004d10 2023-12-20T04:41:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb919dde640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9141e3290 0x7fb9141e56b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb919dde640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9141e3290 0x7fb9141e56b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51618/0 (socket says 172.21.15.32:51618) 2023-12-20T04:41:07.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb919dde640 1 -- 172.21.15.32:0/2499550400 learned_addr learned my addr 172.21.15.32:0/2499550400 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb9195dd640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb900004d10 0x7fb91414dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.473+0000 7fb918ddc640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9141e48b0 0x7fb9141e2d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb919dde640 1 -- 172.21.15.32:0/2499550400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb900004d10 msgr2=0x7fb91414dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb919dde640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb900004d10 0x7fb91414dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb919dde640 1 -- 172.21.15.32:0/2499550400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9141e48b0 msgr2=0x7fb9141e2d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb919dde640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9141e48b0 0x7fb9141e2d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb919dde640 1 -- 172.21.15.32:0/2499550400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9141e3af0 con 0x7fb9141e3290 2023-12-20T04:41:07.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb919dde640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9141e3290 0x7fb9141e56b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0b0040 tx=0x7fb90c009750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:07.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb909ffb640 1 -- 172.21.15.32:0/2499550400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0039a0 con 0x7fb9141e3290 2023-12-20T04:41:07.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb909ffb640 1 -- 172.21.15.32:0/2499550400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb90c003b20 con 0x7fb9141e3290 2023-12-20T04:41:07.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb91b841640 1 -- 172.21.15.32:0/2499550400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9141e3dd0 con 0x7fb9141e3290 2023-12-20T04:41:07.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb909ffb640 1 -- 172.21.15.32:0/2499550400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb90c0ae890 con 0x7fb9141e3290 2023-12-20T04:41:07.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb91b841640 1 -- 172.21.15.32:0/2499550400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb91414e5e0 con 0x7fb9141e3290 2023-12-20T04:41:07.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb91b841640 1 -- 172.21.15.32:0/2499550400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb91404e000 con 0x7fb9141e3290 2023-12-20T04:41:07.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb909ffb640 1 -- 172.21.15.32:0/2499550400 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 18) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb90c0b4040 con 0x7fb9141e3290 2023-12-20T04:41:07.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb909ffb640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8f8065bb0 0x7fb8f8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:07.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.477+0000 7fb909ffb640 1 -- 172.21.15.32:0/2499550400 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb90c11d310 con 0x7fb9141e3290 2023-12-20T04:41:07.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.481+0000 7fb9195dd640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8f8065bb0 0x7fb8f8068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:07.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.481+0000 7fb9195dd640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8f8065bb0 0x7fb8f8068070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb910000c10 tx=0x7fb910007900 comp rx=0 tx=0).ready entity=mgr.5664 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:07.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.481+0000 7fb909ffb640 1 -- 172.21.15.32:0/2499550400 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb90c0ead20 con 0x7fb9141e3290 2023-12-20T04:41:07.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.677+0000 7fb91b841640 1 -- 172.21.15.32:0/2499550400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fb9141e4560 con 0x7fb9141e3290 2023-12-20T04:41:07.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.977+0000 7fb909ffb640 1 -- 172.21.15.32:0/2499550400 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 19) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fb90c0eb0f0 con 0x7fb9141e3290 2023-12-20T04:41:07.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.977+0000 7fb909ffb640 1 -- 172.21.15.32:0/2499550400 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) v1 ==== 92+0+0 (secure 0 0 0) 0x7fb90c0da7d0 con 0x7fb9141e3290 2023-12-20T04:41:07.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 -- 172.21.15.32:0/2499550400 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8f8065bb0 msgr2=0x7fb8f8068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8f8065bb0 0x7fb8f8068070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb910000c10 tx=0x7fb910007900 comp rx=0 tx=0).stop 2023-12-20T04:41:07.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 -- 172.21.15.32:0/2499550400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9141e3290 msgr2=0x7fb9141e56b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:07.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9141e3290 0x7fb9141e56b0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7fb90c0b0040 tx=0x7fb90c009750 comp rx=0 tx=0).stop 2023-12-20T04:41:07.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 -- 172.21.15.32:0/2499550400 shutdown_connections 2023-12-20T04:41:07.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8f8065bb0 0x7fb8f8068070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9141e3290 0x7fb9141e56b0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9141e48b0 0x7fb9141e2d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 --2- 172.21.15.32:0/2499550400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb900004d10 0x7fb91414dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:07.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 -- 172.21.15.32:0/2499550400 >> 172.21.15.32:0/2499550400 conn(0x7fb91406b5e0 msgr2=0x7fb9140d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:07.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 -- 172.21.15.32:0/2499550400 shutdown_connections 2023-12-20T04:41:07.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:07.981+0000 7fb91b841640 1 -- 172.21.15.32:0/2499550400 wait complete. 2023-12-20T04:41:08.001 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T04:41:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2c760640 1 Processor -- start 2023-12-20T04:41:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2c760640 1 -- start start 2023-12-20T04:41:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2c760640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0e2415abb0 0x7f0e240caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2c760640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e240b0cb0 0x7f0e2415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2c760640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e24070230 0x7f0e240706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2c760640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e2406e160 con 0x7f0e24070230 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2c760640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e24070be0 con 0x7f0e2415abb0 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2c760640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2415b6a0 con 0x7f0e240b0cb0 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2acfd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e24070230 0x7f0e240706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2a4fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0e2415abb0 0x7f0e240caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e29cfb640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e240b0cb0 0x7f0e2415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2acfd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0e24070230 0x7f0e240706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51626/0 (socket says 172.21.15.32:51626) 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2acfd640 1 -- 172.21.15.32:0/958316187 learned_addr learned my addr 172.21.15.32:0/958316187 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2acfd640 1 -- 172.21.15.32:0/958316187 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e240b0cb0 msgr2=0x7f0e2415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2acfd640 1 --2- 172.21.15.32:0/958316187 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e240b0cb0 0x7f0e2415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2acfd640 1 -- 172.21.15.32:0/958316187 >> v2:172.21.15.39:3300/0 conn(0x7f0e2415abb0 msgr2=0x7f0e240caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2acfd640 1 --2- 172.21.15.32:0/958316187 >> v2:172.21.15.39:3300/0 conn(0x7f0e2415abb0 0x7f0e240caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2acfd640 1 -- 172.21.15.32:0/958316187 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2415c070 con 0x7f0e24070230 2023-12-20T04:41:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.153+0000 7f0e2a4fc640 1 --2- 172.21.15.32:0/958316187 >> v2:172.21.15.39:3300/0 conn(0x7f0e2415abb0 0x7f0e240caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:41:08.155 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:08.158+0000 7f499c67d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:41:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.157+0000 7f0e2acfd640 1 --2- 172.21.15.32:0/958316187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24070230 0x7f0e240706a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0e240713f0 tx=0x7f0e1c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fc046d931d63a01 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e294fa640 1 -- 172.21.15.32:0/958316187 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0a5850 con 0x7f0e24070230 2023-12-20T04:41:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e294fa640 1 -- 172.21.15.32:0/958316187 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e1c0a59b0 con 0x7f0e24070230 2023-12-20T04:41:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e294fa640 1 -- 172.21.15.32:0/958316187 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e1c0a5d40 con 0x7f0e24070230 2023-12-20T04:41:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 -- 172.21.15.32:0/958316187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24070230 msgr2=0x7f0e240706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 --2- 172.21.15.32:0/958316187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24070230 0x7f0e240706a0 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f0e240713f0 tx=0x7f0e1c01c9d0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 -- 172.21.15.32:0/958316187 shutdown_connections 2023-12-20T04:41:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 --2- 172.21.15.32:0/958316187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24070230 0x7f0e240706a0 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 --2- 172.21.15.32:0/958316187 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e240b0cb0 0x7f0e2415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 --2- 172.21.15.32:0/958316187 >> v2:172.21.15.39:3300/0 conn(0x7f0e2415abb0 0x7f0e240caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 -- 172.21.15.32:0/958316187 >> 172.21.15.32:0/958316187 conn(0x7f0e2406a510 msgr2=0x7f0e240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 -- 172.21.15.32:0/958316187 shutdown_connections 2023-12-20T04:41:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 -- 172.21.15.32:0/958316187 wait complete. 2023-12-20T04:41:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 Processor -- start 2023-12-20T04:41:08.161 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f4b0ba31180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:41:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 -- start start 2023-12-20T04:41:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24070230 0x7f0e24148cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e240b0cb0 0x7f0e24135bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e2415abb0 0x7f0e24138660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e241526a0 con 0x7f0e24070230 2023-12-20T04:41:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e241511a0 con 0x7f0e2415abb0 2023-12-20T04:41:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.161+0000 7f0e2c760640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24138ba0 con 0x7f0e240b0cb0 2023-12-20T04:41:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2a4fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24070230 0x7f0e24148cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2a4fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24070230 0x7f0e24148cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51640/0 (socket says 172.21.15.32:51640) 2023-12-20T04:41:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2a4fc640 1 -- 172.21.15.32:0/901903461 learned_addr learned my addr 172.21.15.32:0/901903461 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e29cfb640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e240b0cb0 0x7f0e24135bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2acfd640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e2415abb0 0x7f0e24138660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2a4fc640 1 -- 172.21.15.32:0/901903461 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e240b0cb0 msgr2=0x7f0e24135bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2a4fc640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e240b0cb0 0x7f0e24135bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2a4fc640 1 -- 172.21.15.32:0/901903461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e2415abb0 msgr2=0x7f0e24138660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2a4fc640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e2415abb0 0x7f0e24138660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2a4fc640 1 -- 172.21.15.32:0/901903461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e20009420 con 0x7f0e24070230 2023-12-20T04:41:08.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2a4fc640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24070230 0x7f0e24148cc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000ebe0 tx=0x7f0e2000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e1affd640 1 -- 172.21.15.32:0/901903461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e2000eee0 con 0x7f0e24070230 2023-12-20T04:41:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e1affd640 1 -- 172.21.15.32:0/901903461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e2000cc30 con 0x7f0e24070230 2023-12-20T04:41:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2c760640 1 -- 172.21.15.32:0/901903461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e1c009b30 con 0x7f0e24070230 2023-12-20T04:41:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e1affd640 1 -- 172.21.15.32:0/901903461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e20013470 con 0x7f0e24070230 2023-12-20T04:41:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2c760640 1 -- 172.21.15.32:0/901903461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e241528d0 con 0x7f0e24070230 2023-12-20T04:41:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e2c760640 1 -- 172.21.15.32:0/901903461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2404e000 con 0x7f0e24070230 2023-12-20T04:41:08.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.165+0000 7f0e1affd640 1 -- 172.21.15.32:0/901903461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f0e200135d0 con 0x7f0e24070230 2023-12-20T04:41:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.169+0000 7f0e1affd640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e08065bd0 0x7f0e08068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.169+0000 7f0e1affd640 1 -- 172.21.15.32:0/901903461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0e20083640 con 0x7f0e24070230 2023-12-20T04:41:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.169+0000 7f0e29cfb640 1 -- 172.21.15.32:0/901903461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e08065bd0 msgr2=0x7f0e08068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:41:08.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.169+0000 7f0e29cfb640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e08065bd0 0x7f0e08068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:41:08.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.173+0000 7f0e1affd640 1 -- 172.21.15.32:0/901903461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0e20050f90 con 0x7f0e24070230 2023-12-20T04:41:08.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.333+0000 7f0e2c760640 1 -- 172.21.15.32:0/901903461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f0e24156ce0 con 0x7f0e24070230 2023-12-20T04:41:08.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.337+0000 7f0e1affd640 1 -- 172.21.15.32:0/901903461 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v17)=0 v17) v1 ==== 138+0+0 (secure 0 0 0) 0x7f0e200545b0 con 0x7f0e24070230 2023-12-20T04:41:08.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 -- 172.21.15.32:0/901903461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e08065bd0 msgr2=0x7f0e08068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:41:08.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e08065bd0 0x7f0e08068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 -- 172.21.15.32:0/901903461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24070230 msgr2=0x7f0e24148cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24070230 0x7f0e24148cc0 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f0e2000ebe0 tx=0x7f0e2000c520 comp rx=0 tx=0).stop 2023-12-20T04:41:08.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 -- 172.21.15.32:0/901903461 shutdown_connections 2023-12-20T04:41:08.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e08065bd0 0x7f0e08068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e2415abb0 0x7f0e24138660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e240b0cb0 0x7f0e24135bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 --2- 172.21.15.32:0/901903461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24070230 0x7f0e24148cc0 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 -- 172.21.15.32:0/901903461 >> 172.21.15.32:0/901903461 conn(0x7f0e2406a510 msgr2=0x7f0e240c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:08.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 -- 172.21.15.32:0/901903461 shutdown_connections 2023-12-20T04:41:08.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.341+0000 7f0e2c760640 1 -- 172.21.15.32:0/901903461 wait complete. 2023-12-20T04:41:08.361 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T04:41:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61d62fc640 1 Processor -- start 2023-12-20T04:41:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61d62fc640 1 -- start start 2023-12-20T04:41:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61d62fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61d015ce60 0x7f61d00d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61d62fc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d00d2650 con 0x7f61d00d3130 2023-12-20T04:41:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61d62fc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61d00d3570 con 0x7f61d00d14d0 2023-12-20T04:41:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61d62fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d00d1a40 con 0x7f61d015ce60 2023-12-20T04:41:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61cf7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f61d00d3130 0x7f61d00d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:35910/0 (socket says 172.21.15.32:35910) 2023-12-20T04:41:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61cf7fe640 1 -- 172.21.15.32:0/2513845509 learned_addr learned my addr 172.21.15.32:0/2513845509 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:41:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61cffff640 1 --2- 172.21.15.32:0/2513845509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61d015ce60 0x7f61d00d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61ceffd640 1 -- 172.21.15.32:0/2513845509 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1871202529 0 0) 0x7f61d00d2650 con 0x7f61d00d3130 2023-12-20T04:41:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61ceffd640 1 -- 172.21.15.32:0/2513845509 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61b8003530 con 0x7f61d00d3130 2023-12-20T04:41:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61ceffd640 1 -- 172.21.15.32:0/2513845509 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2539871573 0 0) 0x7f61d00d3570 con 0x7f61d00d14d0 2023-12-20T04:41:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61ceffd640 1 -- 172.21.15.32:0/2513845509 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d00d2650 con 0x7f61d00d14d0 2023-12-20T04:41:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61cffff640 1 -- 172.21.15.32:0/2513845509 >> v1:172.21.15.39:6789/0 conn(0x7f61d00d14d0 legacy=0x7f61d00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61cffff640 1 -- 172.21.15.32:0/2513845509 >> v1:172.21.15.32:6789/0 conn(0x7f61d00d3130 legacy=0x7f61d00d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.513+0000 7f61cffff640 1 -- 172.21.15.32:0/2513845509 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61d00d1d20 con 0x7f61d015ce60 2023-12-20T04:41:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.517+0000 7f61cffff640 1 --2- 172.21.15.32:0/2513845509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61d015ce60 0x7f61d00d2bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f61c00097f0 tx=0x7f61c001ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=35c1520af6de129e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.517+0000 7f61ceffd640 1 -- 172.21.15.32:0/2513845509 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00a89b0 con 0x7f61d015ce60 2023-12-20T04:41:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.517+0000 7f61ceffd640 1 -- 172.21.15.32:0/2513845509 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c00a8b10 con 0x7f61d015ce60 2023-12-20T04:41:08.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.517+0000 7f61ceffd640 1 -- 172.21.15.32:0/2513845509 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00a8ea0 con 0x7f61d015ce60 2023-12-20T04:41:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.517+0000 7f61d62fc640 1 -- 172.21.15.32:0/2513845509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61d015ce60 msgr2=0x7f61d00d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.517+0000 7f61d62fc640 1 --2- 172.21.15.32:0/2513845509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61d015ce60 0x7f61d00d2bf0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f61c00097f0 tx=0x7f61c001ecb0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.517+0000 7f61d62fc640 1 -- 172.21.15.32:0/2513845509 shutdown_connections 2023-12-20T04:41:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.517+0000 7f61d62fc640 1 --2- 172.21.15.32:0/2513845509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61d015ce60 0x7f61d00d2bf0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.517+0000 7f61d62fc640 1 -- 172.21.15.32:0/2513845509 >> 172.21.15.32:0/2513845509 conn(0x7f61d006b5e0 msgr2=0x7f61d006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.517+0000 7f61d62fc640 1 -- 172.21.15.32:0/2513845509 shutdown_connections 2023-12-20T04:41:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.517+0000 7f61d62fc640 1 -- 172.21.15.32:0/2513845509 wait complete. 2023-12-20T04:41:08.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61d62fc640 1 Processor -- start 2023-12-20T04:41:08.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61d62fc640 1 -- start start 2023-12-20T04:41:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61d62fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d00d14d0 0x7f61d01e7480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61d62fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61d00d3130 0x7f61d0150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61d62fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61d015ce60 0x7f61d01fa750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61d62fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d01511c0 con 0x7f61d015ce60 2023-12-20T04:41:08.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61d62fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61d01e5b90 con 0x7f61d00d14d0 2023-12-20T04:41:08.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61d62fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61d01fac90 con 0x7f61d00d3130 2023-12-20T04:41:08.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d00d14d0 0x7f61d01e7480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61cf7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61d00d3130 0x7f61d0150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61cffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d00d14d0 0x7f61d01e7480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:42682/0 (socket says 172.21.15.32:42682) 2023-12-20T04:41:08.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61cffff640 1 -- 172.21.15.32:0/1094500123 learned_addr learned my addr 172.21.15.32:0/1094500123 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:08.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61d4899640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61d015ce60 0x7f61d01fa750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61cffff640 1 -- 172.21.15.32:0/1094500123 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61d00d3130 msgr2=0x7f61d0150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61cffff640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61d00d3130 0x7f61d0150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61cffff640 1 -- 172.21.15.32:0/1094500123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61d015ce60 msgr2=0x7f61d01fa750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61cffff640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61d015ce60 0x7f61d01fa750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61cffff640 1 -- 172.21.15.32:0/1094500123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61c0009420 con 0x7f61d00d14d0 2023-12-20T04:41:08.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61cffff640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d00d14d0 0x7f61d01e7480 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f61c001b8c0 tx=0x7f61c001ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:08.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61ccff9640 1 -- 172.21.15.32:0/1094500123 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00b1cf0 con 0x7f61d00d14d0 2023-12-20T04:41:08.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61ccff9640 1 -- 172.21.15.32:0/1094500123 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61c00b1e70 con 0x7f61d00d14d0 2023-12-20T04:41:08.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61ccff9640 1 -- 172.21.15.32:0/1094500123 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61c00b9630 con 0x7f61d00d14d0 2023-12-20T04:41:08.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61d62fc640 1 -- 172.21.15.32:0/1094500123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61d01f6650 con 0x7f61d00d14d0 2023-12-20T04:41:08.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.521+0000 7f61d62fc640 1 -- 172.21.15.32:0/1094500123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61d01e60e0 con 0x7f61d00d14d0 2023-12-20T04:41:08.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.525+0000 7f61d62fc640 1 -- 172.21.15.32:0/1094500123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61d00d1fe0 con 0x7f61d00d14d0 2023-12-20T04:41:08.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.525+0000 7f61ccff9640 1 -- 172.21.15.32:0/1094500123 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 19) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f61c00b7040 con 0x7f61d00d14d0 2023-12-20T04:41:08.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.525+0000 7f61ccff9640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61bc065bd0 0x7f61bc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.525+0000 7f61ccff9640 1 -- 172.21.15.32:0/1094500123 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f61c0120b00 con 0x7f61d00d14d0 2023-12-20T04:41:08.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.525+0000 7f61cf7fe640 1 -- 172.21.15.32:0/1094500123 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61bc065bd0 msgr2=0x7f61bc068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:41:08.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.525+0000 7f61cf7fe640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61bc065bd0 0x7f61bc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:41:08.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.529+0000 7f61ccff9640 1 -- 172.21.15.32:0/1094500123 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f61d00d1fe0 con 0x7f61d00d14d0 2023-12-20T04:41:08.596 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:08.598+0000 7f499c67d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:41:08.614 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:08.613+0000 7f4b0ba31180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:41:08.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.689+0000 7f61d62fc640 1 -- 172.21.15.32:0/1094500123 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f61d00d1fe0 con 0x7f61d00d14d0 2023-12-20T04:41:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.697+0000 7f61ccff9640 1 -- 172.21.15.32:0/1094500123 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v18)=0 v18) v1 ==== 133+0+0 (secure 0 0 0) 0x7f61d00d1fe0 con 0x7f61d00d14d0 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 -- 172.21.15.32:0/1094500123 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61bc065bd0 msgr2=0x7f61bc068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61bc065bd0 0x7f61bc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 -- 172.21.15.32:0/1094500123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d00d14d0 msgr2=0x7f61d01e7480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d00d14d0 0x7f61d01e7480 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f61c001b8c0 tx=0x7f61c001ba50 comp rx=0 tx=0).stop 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 -- 172.21.15.32:0/1094500123 shutdown_connections 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61bc065bd0 0x7f61bc068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61d015ce60 0x7f61d01fa750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61d00d3130 0x7f61d0150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 --2- 172.21.15.32:0/1094500123 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61d00d14d0 0x7f61d01e7480 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 -- 172.21.15.32:0/1094500123 >> 172.21.15.32:0/1094500123 conn(0x7f61d006b5e0 msgr2=0x7f61d015bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 -- 172.21.15.32:0/1094500123 shutdown_connections 2023-12-20T04:41:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.701+0000 7f61d62fc640 1 -- 172.21.15.32:0/1094500123 wait complete. 2023-12-20T04:41:08.721 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:08.721 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-20T04:41:08.737 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-20T04:41:08.755 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:08.758+0000 7f499c67d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:41:08.776 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:08.777+0000 7f4b0ba31180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:41:08.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d74c4640 1 Processor -- start 2023-12-20T04:41:08.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d74c4640 1 -- start start 2023-12-20T04:41:08.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d74c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85d0157480 0x7f85d00c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d74c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85d00b0760 0x7f85d00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d74c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f85d00b1100 con 0x7f85d015d660 2023-12-20T04:41:08.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d74c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f85d00cde20 con 0x7f85d0157480 2023-12-20T04:41:08.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d74c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d00c8e30 con 0x7f85d00b0760 2023-12-20T04:41:08.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d5260640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85d0157480 0x7f85d00c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d5260640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f85d0157480 0x7f85d00c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36356/0 (socket says 172.21.15.32:36356) 2023-12-20T04:41:08.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d5260640 1 -- 172.21.15.32:0/810942626 learned_addr learned my addr 172.21.15.32:0/810942626 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:08.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d4a5f640 1 --2- 172.21.15.32:0/810942626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85d00b0760 0x7f85d00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d5260640 1 -- 172.21.15.32:0/810942626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85d00b0760 msgr2=0x7f85d00b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:08.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d5260640 1 --2- 172.21.15.32:0/810942626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85d00b0760 0x7f85d00b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d5260640 1 -- 172.21.15.32:0/810942626 >> v1:172.21.15.39:6789/0 conn(0x7f85d015d660 legacy=0x7f85d015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.925+0000 7f85d5260640 1 -- 172.21.15.32:0/810942626 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85d00ce070 con 0x7f85d0157480 2023-12-20T04:41:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85d5260640 1 --2- 172.21.15.32:0/810942626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85d0157480 0x7f85d00c9840 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009870 tx=0x7f85cc01e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6893f917f5e1ae server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85c7fff640 1 -- 172.21.15.32:0/810942626 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0ab660 con 0x7f85d0157480 2023-12-20T04:41:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85c7fff640 1 -- 172.21.15.32:0/810942626 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85cc0ab7c0 con 0x7f85d0157480 2023-12-20T04:41:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85c7fff640 1 -- 172.21.15.32:0/810942626 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85cc0abb50 con 0x7f85d0157480 2023-12-20T04:41:08.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85d74c4640 1 -- 172.21.15.32:0/810942626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85d0157480 msgr2=0x7f85d00c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85d74c4640 1 --2- 172.21.15.32:0/810942626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85d0157480 0x7f85d00c9840 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009870 tx=0x7f85cc01e690 comp rx=0 tx=0).stop 2023-12-20T04:41:08.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85d74c4640 1 -- 172.21.15.32:0/810942626 shutdown_connections 2023-12-20T04:41:08.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85d74c4640 1 --2- 172.21.15.32:0/810942626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85d00b0760 0x7f85d00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85d74c4640 1 --2- 172.21.15.32:0/810942626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85d0157480 0x7f85d00c9840 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85d74c4640 1 -- 172.21.15.32:0/810942626 >> 172.21.15.32:0/810942626 conn(0x7f85d006a510 msgr2=0x7f85d00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:08.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85d74c4640 1 -- 172.21.15.32:0/810942626 shutdown_connections 2023-12-20T04:41:08.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.929+0000 7f85d74c4640 1 -- 172.21.15.32:0/810942626 wait complete. 2023-12-20T04:41:08.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d74c4640 1 Processor -- start 2023-12-20T04:41:08.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d74c4640 1 -- start start 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d74c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85d00b0760 0x7f85d01595c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d74c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85d0157480 0x7f85d015b880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d74c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85d015d660 0x7f85d0162de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d74c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d0159b00 con 0x7f85d0157480 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d74c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f85d015bdc0 con 0x7f85d00b0760 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d74c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f85d00c5cd0 con 0x7f85d015d660 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d4a5f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85d0157480 0x7f85d015b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d4a5f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85d0157480 0x7f85d015b880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36372/0 (socket says 172.21.15.32:36372) 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d4a5f640 1 -- 172.21.15.32:0/1207490102 learned_addr learned my addr 172.21.15.32:0/1207490102 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d5260640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85d00b0760 0x7f85d01595c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d5a61640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85d015d660 0x7f85d0162de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d4a5f640 1 -- 172.21.15.32:0/1207490102 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85d015d660 msgr2=0x7f85d0162de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d4a5f640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85d015d660 0x7f85d0162de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d4a5f640 1 -- 172.21.15.32:0/1207490102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85d00b0760 msgr2=0x7f85d01595c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:08.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d4a5f640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85d00b0760 0x7f85d01595c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:08.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d4a5f640 1 -- 172.21.15.32:0/1207490102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85cc0094d0 con 0x7f85d0157480 2023-12-20T04:41:08.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d5a61640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85d015d660 0x7f85d0162de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:41:08.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d4a5f640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85d0157480 0x7f85d015b880 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f85c000d5f0 tx=0x7f85c000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:08.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85c5ffb640 1 -- 172.21.15.32:0/1207490102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c00125f0 con 0x7f85d0157480 2023-12-20T04:41:08.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85c5ffb640 1 -- 172.21.15.32:0/1207490102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85c0012750 con 0x7f85d0157480 2023-12-20T04:41:08.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85c5ffb640 1 -- 172.21.15.32:0/1207490102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85c0012ae0 con 0x7f85d0157480 2023-12-20T04:41:08.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d74c4640 1 -- 172.21.15.32:0/1207490102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f85d015bfb0 con 0x7f85d0157480 2023-12-20T04:41:08.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d74c4640 1 -- 172.21.15.32:0/1207490102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f85d00c5f50 con 0x7f85d0157480 2023-12-20T04:41:08.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.933+0000 7f85d74c4640 1 -- 172.21.15.32:0/1207490102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f85d004e000 con 0x7f85d0157480 2023-12-20T04:41:08.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.937+0000 7f85c5ffb640 1 -- 172.21.15.32:0/1207490102 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 19) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f85c0012c40 con 0x7f85d0157480 2023-12-20T04:41:08.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.937+0000 7f85c5ffb640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:08.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.937+0000 7f85d5260640 1 -- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 msgr2=0x7f85b4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:41:08.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.937+0000 7f85c5ffb640 1 -- 172.21.15.32:0/1207490102 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(61..61 src has 1..61) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f85c0083800 con 0x7f85d0157480 2023-12-20T04:41:08.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.937+0000 7f85d5260640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:41:08.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:08.941+0000 7f85c5ffb640 1 -- 172.21.15.32:0/1207490102 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f85c0051020 con 0x7f85d0157480 2023-12-20T04:41:09.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:09.121+0000 7f85d74c4640 1 -- 172.21.15.32:0/1207490102 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f85d015c460 con 0x7f85b4065bd0 2023-12-20T04:41:09.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:09.141+0000 7f85d5260640 1 -- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 msgr2=0x7f85b4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:41:09.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:09.141+0000 7f85d5260640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:41:09.265 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:09.266+0000 7f499c67d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:41:09.288 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:09.289+0000 7f4b0ba31180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:41:09.326 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:09.326+0000 7f499c67d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:41:09.350 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:09.349+0000 7f4b0ba31180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:41:09.385 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:09.386+0000 7f499c67d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:41:09.408 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:09.409+0000 7f4b0ba31180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:41:09.477 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:09.478+0000 7f499c67d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:41:09.500 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:09.501+0000 7f4b0ba31180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:41:09.537 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:09.538+0000 7f499c67d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:41:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:09.545+0000 7f85d5260640 1 -- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 msgr2=0x7f85b4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:41:09.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:09.545+0000 7f85d5260640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T04:41:09.560 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:09.561+0000 7f4b0ba31180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:41:09.625 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:09.626+0000 7f499c67d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:41:09.648 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:09.649+0000 7f4b0ba31180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:41:09.685 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:09.686+0000 7f499c67d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:41:09.708 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:09.709+0000 7f4b0ba31180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:41:09.836 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:09.838+0000 7f499c67d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:41:09.859 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:09.861+0000 7f4b0ba31180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:41:09.898 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:09.898+0000 7f499c67d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:41:09.921 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:09.921+0000 7f4b0ba31180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:41:10.149 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:41:10.149 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:41:10.149 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:41:10.158 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:10.158+0000 7f499c67d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:41:10.171 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:41:10.171 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:41:10.171 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:41:10.180 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:10.181+0000 7f4b0ba31180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:41:10.300 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:10.302+0000 7f499c67d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:41:10.323 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:10.321+0000 7f4b0ba31180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:41:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:10.349+0000 7f85d5260640 1 -- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 msgr2=0x7f85b4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:41:10.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:10.349+0000 7f85d5260640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T04:41:10.432 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:10.434+0000 7f499c67d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:41:10.454 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:10.453+0000 7f4b0ba31180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:41:10.506 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:10.506+0000 7f499c67d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:41:10.528 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:10.529+0000 7f4b0ba31180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:41:10.650 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:10.650+0000 7f499c67d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:41:10.671 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:10.673+0000 7f4b0ba31180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:41:11.220 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:11.222+0000 7f499c67d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:41:11.238 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:11.237+0000 7f4b0ba31180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:41:11.309 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:11.310+0000 7f499c67d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:41:11.327 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:11.329+0000 7f4b0ba31180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:41:11.466 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:11.466+0000 7f499c67d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:41:11.483 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:11.485+0000 7f4b0ba31180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:41:11.555 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:11.554+0000 7f499c67d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:41:11.570 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:11.569+0000 7f4b0ba31180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:41:11.891 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:11.894+0000 7f499c67d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:41:11.904 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:11.905+0000 7f4b0ba31180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:41:11.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:11.953+0000 7f85d5260640 1 -- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 msgr2=0x7f85b4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:41:11.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:11.953+0000 7f85d5260640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-20T04:41:11.954 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:41:11.954+0000 7f499c67d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:41:11.968 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:11.969+0000 7f4b0ba31180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:41:11.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:11.981+0000 7f85c5ffb640 1 -- 172.21.15.32:0/1207490102 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 20) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f85c0081d20 con 0x7f85d0157480 2023-12-20T04:41:11.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:11.981+0000 7f85c5ffb640 1 -- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 msgr2=0x7f85b4068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:41:11.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:11.981+0000 7f85c5ffb640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:12.099 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:12.101+0000 7f4a8e8a3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:41:12.099 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:12.101+0000 7f4a8e8a3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:41:12.099 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:12.101+0000 7f4a8e8a3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:41:12.099 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:12.101+0000 7f4a8e8a3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:41:12.099 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:12.101+0000 7f4a8e8a3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:41:12.100 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:12.101+0000 7f4a918a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:41:12.100 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:12.101+0000 7f4a918a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:41:12.100 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:12.101+0000 7f4a918a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:41:12.100 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:12.101+0000 7f4a918a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:41:12.100 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:41:12.101+0000 7f4a918a9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:41:12.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:12.985+0000 7f85c5ffb640 1 -- 172.21.15.32:0/1207490102 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 21) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f85c00513f0 con 0x7f85d0157480 2023-12-20T04:41:12.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:12.989+0000 7f85c5ffb640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:12.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:12.989+0000 7f85c5ffb640 1 -- 172.21.15.32:0/1207490102 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f85d015c460 con 0x7f85b4065bd0 2023-12-20T04:41:12.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:12.989+0000 7f85d5260640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:12.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:12.997+0000 7f85d5260640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0b0000 tx=0x7f85cc0a9000 comp rx=0 tx=0).ready entity=mgr.6081 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:13.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.061+0000 7f85c5ffb640 1 -- 172.21.15.32:0/1207490102 <== mgr.6081 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f85d015c460 con 0x7f85b4065bd0 2023-12-20T04:41:13.063 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path snap_test_dir1 2023-12-20T04:41:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 -- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 msgr2=0x7f85b4068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f85cc0b0000 tx=0x7f85cc0a9000 comp rx=0 tx=0).stop 2023-12-20T04:41:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 -- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85d0157480 msgr2=0x7f85d015b880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85d0157480 0x7f85d015b880 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f85c000d5f0 tx=0x7f85c000dab0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 -- 172.21.15.32:0/1207490102 shutdown_connections 2023-12-20T04:41:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f85b4065bd0 0x7f85b4068090 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f85d015d660 0x7f85d0162de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f85d0157480 0x7f85d015b880 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 --2- 172.21.15.32:0/1207490102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f85d00b0760 0x7f85d01595c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 -- 172.21.15.32:0/1207490102 >> 172.21.15.32:0/1207490102 conn(0x7f85d006a510 msgr2=0x7f85d006d810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 -- 172.21.15.32:0/1207490102 shutdown_connections 2023-12-20T04:41:13.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.065+0000 7f85d74c4640 1 -- 172.21.15.32:0/1207490102 wait complete. 2023-12-20T04:41:13.086 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-20T04:41:13.086 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 2M 2023-12-20T04:41:13.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.233+0000 7f51dd273640 1 Processor -- start 2023-12-20T04:41:13.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.233+0000 7f51dd273640 1 -- start start 2023-12-20T04:41:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51dd273640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51d8157480 0x7f51d80c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51dd273640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51d80cd4c0 0x7f51d80b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51dd273640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51d80b1100 con 0x7f51d80b0760 2023-12-20T04:41:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51dd273640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51d80cde20 con 0x7f51d80cd4c0 2023-12-20T04:41:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51dd273640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d80c8e30 con 0x7f51d8157480 2023-12-20T04:41:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51d77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51d80cd4c0 0x7f51d80b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51d77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51d80cd4c0 0x7f51d80b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36476/0 (socket says 172.21.15.32:36476) 2023-12-20T04:41:13.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51d77fe640 1 -- 172.21.15.32:0/614664219 learned_addr learned my addr 172.21.15.32:0/614664219 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51d6ffd640 1 --2- 172.21.15.32:0/614664219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51d8157480 0x7f51d80c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51d77fe640 1 -- 172.21.15.32:0/614664219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51d8157480 msgr2=0x7f51d80c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51d77fe640 1 --2- 172.21.15.32:0/614664219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51d8157480 0x7f51d80c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51d77fe640 1 -- 172.21.15.32:0/614664219 >> v1:172.21.15.39:6789/0 conn(0x7f51d80b0760 legacy=0x7f51d815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.237+0000 7f51d77fe640 1 -- 172.21.15.32:0/614664219 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d80ce070 con 0x7f51d80cd4c0 2023-12-20T04:41:13.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51d77fe640 1 --2- 172.21.15.32:0/614664219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d80cd4c0 0x7f51d80b0bc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009e50 tx=0x7f51d001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d27a15c35086064 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51d67fc640 1 -- 172.21.15.32:0/614664219 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00a5730 con 0x7f51d80cd4c0 2023-12-20T04:41:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51d67fc640 1 -- 172.21.15.32:0/614664219 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51d00a5890 con 0x7f51d80cd4c0 2023-12-20T04:41:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51d67fc640 1 -- 172.21.15.32:0/614664219 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51d00a5c20 con 0x7f51d80cd4c0 2023-12-20T04:41:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51dd273640 1 -- 172.21.15.32:0/614664219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d80cd4c0 msgr2=0x7f51d80b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51dd273640 1 --2- 172.21.15.32:0/614664219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d80cd4c0 0x7f51d80b0bc0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7f51d0009e50 tx=0x7f51d001c950 comp rx=0 tx=0).stop 2023-12-20T04:41:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51dd273640 1 -- 172.21.15.32:0/614664219 shutdown_connections 2023-12-20T04:41:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51dd273640 1 --2- 172.21.15.32:0/614664219 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d80cd4c0 0x7f51d80b0bc0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51dd273640 1 --2- 172.21.15.32:0/614664219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51d8157480 0x7f51d80c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51dd273640 1 -- 172.21.15.32:0/614664219 >> 172.21.15.32:0/614664219 conn(0x7f51d806a510 msgr2=0x7f51d80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51dd273640 1 -- 172.21.15.32:0/614664219 shutdown_connections 2023-12-20T04:41:13.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51dd273640 1 -- 172.21.15.32:0/614664219 wait complete. 2023-12-20T04:41:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51dd273640 1 Processor -- start 2023-12-20T04:41:13.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.241+0000 7f51dd273640 1 -- start start 2023-12-20T04:41:13.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51dd273640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d80b0760 0x7f51d814e0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51dd273640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51d80cd4c0 0x7f51d8155310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51dd273640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d8157480 0x7f51d8162b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51dd273640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d8146b50 con 0x7f51d80b0760 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51dd273640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51d814e5e0 con 0x7f51d8157480 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51dd273640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51d814e810 con 0x7f51d80cd4c0 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51d6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d80b0760 0x7f51d814e0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51d6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d80b0760 0x7f51d814e0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36484/0 (socket says 172.21.15.32:36484) 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51d6ffd640 1 -- 172.21.15.32:0/3614645007 learned_addr learned my addr 172.21.15.32:0/3614645007 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51cffff640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51d80cd4c0 0x7f51d8155310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51d77fe640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d8157480 0x7f51d8162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51d6ffd640 1 -- 172.21.15.32:0/3614645007 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51d80cd4c0 msgr2=0x7f51d8155310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51d6ffd640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51d80cd4c0 0x7f51d8155310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51d6ffd640 1 -- 172.21.15.32:0/3614645007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d8157480 msgr2=0x7f51d8162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51d6ffd640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d8157480 0x7f51d8162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51d6ffd640 1 -- 172.21.15.32:0/3614645007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51d0009b30 con 0x7f51d80b0760 2023-12-20T04:41:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51d6ffd640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d80b0760 0x7f51d814e0a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f51c800d800 tx=0x7f51c800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51cf7fe640 1 -- 172.21.15.32:0/3614645007 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c800b6c0 con 0x7f51d80b0760 2023-12-20T04:41:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51cf7fe640 1 -- 172.21.15.32:0/3614645007 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51c800b860 con 0x7f51d80b0760 2023-12-20T04:41:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51cf7fe640 1 -- 172.21.15.32:0/3614645007 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51c800bc10 con 0x7f51d80b0760 2023-12-20T04:41:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51dd273640 1 -- 172.21.15.32:0/3614645007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51d814eaf0 con 0x7f51d80b0760 2023-12-20T04:41:13.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51dd273640 1 -- 172.21.15.32:0/3614645007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51d814f0b0 con 0x7f51d80b0760 2023-12-20T04:41:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51dd273640 1 -- 172.21.15.32:0/3614645007 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51d804e000 con 0x7f51d80b0760 2023-12-20T04:41:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.245+0000 7f51cf7fe640 1 -- 172.21.15.32:0/3614645007 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f51c801e460 con 0x7f51d80b0760 2023-12-20T04:41:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.249+0000 7f51cf7fe640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51b8065900 0x7f51b8067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.249+0000 7f51cf7fe640 1 -- 172.21.15.32:0/3614645007 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f51c8083510 con 0x7f51d80b0760 2023-12-20T04:41:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.249+0000 7f51cffff640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51b8065900 0x7f51b8067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.249+0000 7f51cffff640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51b8065900 0x7f51b8067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f51d81502e0 tx=0x7f51c0009040 comp rx=0 tx=0).ready entity=mgr.6081 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:13.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.253+0000 7f51cf7fe640 1 -- 172.21.15.32:0/3614645007 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f51c8051090 con 0x7f51d80b0760 2023-12-20T04:41:13.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.429+0000 7f51dd273640 1 -- 172.21.15.32:0/3614645007 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "2M", "target": ["mon-mgr", ""]}) v1 -- 0x7f51d80c9360 con 0x7f51b8065900 2023-12-20T04:41:13.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.481+0000 7f51cf7fe640 1 -- 172.21.15.32:0/3614645007 <== mgr.6081 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f51c0001d60 con 0x7f51b8065900 2023-12-20T04:41:13.482 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path snap_test_dir1 2023-12-20T04:41:13.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 -- 172.21.15.32:0/3614645007 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51b8065900 msgr2=0x7f51b8067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51b8065900 0x7f51b8067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f51d81502e0 tx=0x7f51c0009040 comp rx=0 tx=0).stop 2023-12-20T04:41:13.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 -- 172.21.15.32:0/3614645007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d80b0760 msgr2=0x7f51d814e0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d80b0760 0x7f51d814e0a0 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7f51c800d800 tx=0x7f51c800dcc0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 -- 172.21.15.32:0/3614645007 shutdown_connections 2023-12-20T04:41:13.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51b8065900 0x7f51b8067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51d8157480 0x7f51d8162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51d80cd4c0 0x7f51d8155310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 --2- 172.21.15.32:0/3614645007 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51d80b0760 0x7f51d814e0a0 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 -- 172.21.15.32:0/3614645007 >> 172.21.15.32:0/3614645007 conn(0x7f51d806a510 msgr2=0x7f51d80cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:13.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 -- 172.21.15.32:0/3614645007 shutdown_connections 2023-12-20T04:41:13.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.485+0000 7f51dd273640 1 -- 172.21.15.32:0/3614645007 wait complete. 2023-12-20T04:41:13.503 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-20T04:41:13.504 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-20-04_42_00 in ~47s... 2023-12-20T04:41:13.504 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-20-04_43_00 in ~107s... 2023-12-20T04:41:13.504 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M', '2M'], expected_retention: [] 2023-12-20T04:41:13.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-20T04:41:13.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.649+0000 7fdb413c0640 1 Processor -- start 2023-12-20T04:41:13.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.649+0000 7fdb413c0640 1 -- start start 2023-12-20T04:41:13.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.649+0000 7fdb413c0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb3c15a9d0 0x7fdb3c0ce8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.649+0000 7fdb413c0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdb3c0b5e30 0x7fdb3c0b62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.649+0000 7fdb413c0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb3c0b2250 0x7fdb3c0b2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.649+0000 7fdb413c0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3c0b6820 con 0x7fdb3c15a9d0 2023-12-20T04:41:13.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.649+0000 7fdb413c0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb3c15adb0 con 0x7fdb3c0b5e30 2023-12-20T04:41:13.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.649+0000 7fdb413c0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c15b060 con 0x7fdb3c0b2250 2023-12-20T04:41:13.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.649+0000 7fdb3affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb3c15a9d0 0x7fdb3c0ce8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.649+0000 7fdb3affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb3c15a9d0 0x7fdb3c0ce8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36496/0 (socket says 172.21.15.32:36496) 2023-12-20T04:41:13.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.649+0000 7fdb3affd640 1 -- 172.21.15.32:0/153617145 learned_addr learned my addr 172.21.15.32:0/153617145 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:13.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb3a7fc640 1 --2- 172.21.15.32:0/153617145 >> v2:172.21.15.39:3300/0 conn(0x7fdb3c0b5e30 0x7fdb3c0b62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb3b7fe640 1 --2- 172.21.15.32:0/153617145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb3c0b2250 0x7fdb3c0b2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb3affd640 1 -- 172.21.15.32:0/153617145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb3c0b2250 msgr2=0x7fdb3c0b2630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:13.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb3affd640 1 --2- 172.21.15.32:0/153617145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb3c0b2250 0x7fdb3c0b2630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb3affd640 1 -- 172.21.15.32:0/153617145 >> v2:172.21.15.39:3300/0 conn(0x7fdb3c0b5e30 msgr2=0x7fdb3c0b62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:41:13.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb3affd640 1 --2- 172.21.15.32:0/153617145 >> v2:172.21.15.39:3300/0 conn(0x7fdb3c0b5e30 0x7fdb3c0b62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb3affd640 1 -- 172.21.15.32:0/153617145 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb3c15b340 con 0x7fdb3c15a9d0 2023-12-20T04:41:13.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb3a7fc640 1 --2- 172.21.15.32:0/153617145 >> v2:172.21.15.39:3300/0 conn(0x7fdb3c0b5e30 0x7fdb3c0b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:41:13.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb3affd640 1 --2- 172.21.15.32:0/153617145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb3c15a9d0 0x7fdb3c0ce8b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0ce330 tx=0x7fdb2c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f90965cfcbf6cb0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:13.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb39ffb640 1 -- 172.21.15.32:0/153617145 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0ac410 con 0x7fdb3c15a9d0 2023-12-20T04:41:13.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb39ffb640 1 -- 172.21.15.32:0/153617145 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c0ac570 con 0x7fdb3c15a9d0 2023-12-20T04:41:13.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.653+0000 7fdb39ffb640 1 -- 172.21.15.32:0/153617145 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0ac900 con 0x7fdb3c15a9d0 2023-12-20T04:41:13.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 -- 172.21.15.32:0/153617145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb3c15a9d0 msgr2=0x7fdb3c0ce8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 --2- 172.21.15.32:0/153617145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb3c15a9d0 0x7fdb3c0ce8b0 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c0ce330 tx=0x7fdb2c01c500 comp rx=0 tx=0).stop 2023-12-20T04:41:13.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 -- 172.21.15.32:0/153617145 shutdown_connections 2023-12-20T04:41:13.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 --2- 172.21.15.32:0/153617145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb3c0b2250 0x7fdb3c0b2630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 --2- 172.21.15.32:0/153617145 >> v2:172.21.15.39:3300/0 conn(0x7fdb3c0b5e30 0x7fdb3c0b62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 --2- 172.21.15.32:0/153617145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb3c15a9d0 0x7fdb3c0ce8b0 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 -- 172.21.15.32:0/153617145 >> 172.21.15.32:0/153617145 conn(0x7fdb3c0cd250 msgr2=0x7fdb3c0cd660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:13.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 -- 172.21.15.32:0/153617145 shutdown_connections 2023-12-20T04:41:13.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 -- 172.21.15.32:0/153617145 wait complete. 2023-12-20T04:41:13.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 Processor -- start 2023-12-20T04:41:13.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 -- start start 2023-12-20T04:41:13.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb3c0b2250 0x7fdb3c06b840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb3c0b5e30 0x7fdb3c1457a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb3c15a9d0 0x7fdb3c154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c153f30 con 0x7fdb3c0b2250 2023-12-20T04:41:13.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c157af0 con 0x7fdb3c0b5e30 2023-12-20T04:41:13.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb413c0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb3c06be90 con 0x7fdb3c15a9d0 2023-12-20T04:41:13.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb3affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb3c0b2250 0x7fdb3c06b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb3affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb3c0b2250 0x7fdb3c06b840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36512/0 (socket says 172.21.15.32:36512) 2023-12-20T04:41:13.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb3affd640 1 -- 172.21.15.32:0/3066665463 learned_addr learned my addr 172.21.15.32:0/3066665463 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:41:13.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb3a7fc640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb3c0b5e30 0x7fdb3c1457a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb3b7fe640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb3c15a9d0 0x7fdb3c154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb3affd640 1 -- 172.21.15.32:0/3066665463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb3c15a9d0 msgr2=0x7fdb3c154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb3affd640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb3c15a9d0 0x7fdb3c154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb3affd640 1 -- 172.21.15.32:0/3066665463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb3c0b5e30 msgr2=0x7fdb3c1457a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb3affd640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb3c0b5e30 0x7fdb3c1457a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.657+0000 7fdb3affd640 1 -- 172.21.15.32:0/3066665463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb2c0094d0 con 0x7fdb3c0b2250 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.661+0000 7fdb3affd640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb3c0b2250 0x7fdb3c06b840 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c01cda0 tx=0x7fdb2c0192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.661+0000 7fdb237fe640 1 -- 172.21.15.32:0/3066665463 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0ac410 con 0x7fdb3c0b2250 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.661+0000 7fdb237fe640 1 -- 172.21.15.32:0/3066665463 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb2c0b65f0 con 0x7fdb3c0b2250 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.661+0000 7fdb413c0640 1 -- 172.21.15.32:0/3066665463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb3c06c170 con 0x7fdb3c0b2250 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.661+0000 7fdb237fe640 1 -- 172.21.15.32:0/3066665463 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb2c0b69a0 con 0x7fdb3c0b2250 2023-12-20T04:41:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.661+0000 7fdb413c0640 1 -- 172.21.15.32:0/3066665463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb3c06c730 con 0x7fdb3c0b2250 2023-12-20T04:41:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.661+0000 7fdb413c0640 1 -- 172.21.15.32:0/3066665463 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb3c074100 con 0x7fdb3c0b2250 2023-12-20T04:41:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.661+0000 7fdb237fe640 1 -- 172.21.15.32:0/3066665463 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 21) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fdb2c0b6b00 con 0x7fdb3c0b2250 2023-12-20T04:41:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.661+0000 7fdb237fe640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb28065900 0x7fdb28067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:41:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.661+0000 7fdb237fe640 1 -- 172.21.15.32:0/3066665463 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(62..62 src has 1..62) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fdb2c11d890 con 0x7fdb3c0b2250 2023-12-20T04:41:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.661+0000 7fdb3a7fc640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb28065900 0x7fdb28067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:41:13.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.665+0000 7fdb3a7fc640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb28065900 0x7fdb28067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c1f8410 tx=0x7fdb30008040 comp rx=0 tx=0).ready entity=mgr.6081 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:41:13.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.665+0000 7fdb237fe640 1 -- 172.21.15.32:0/3066665463 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdb2c0eb410 con 0x7fdb3c0b2250 2023-12-20T04:41:13.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.845+0000 7fdb413c0640 1 -- 172.21.15.32:0/3066665463 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fdb3c0b2b70 con 0x7fdb28065900 2023-12-20T04:41:13.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb237fe640 1 -- 172.21.15.32:0/3066665463 <== mgr.6081 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+76 (secure 0 0 0) 0x7fdb300118b0 con 0x7fdb28065900 2023-12-20T04:41:13.852 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:41:13.852 INFO:teuthology.orchestra.run.smithi032.stdout:{"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-12-20T04:41:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 -- 172.21.15.32:0/3066665463 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb28065900 msgr2=0x7fdb28067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb28065900 0x7fdb28067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fdb3c1f8410 tx=0x7fdb30008040 comp rx=0 tx=0).stop 2023-12-20T04:41:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 -- 172.21.15.32:0/3066665463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb3c0b2250 msgr2=0x7fdb3c06b840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:41:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb3c0b2250 0x7fdb3c06b840 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x7fdb2c01cda0 tx=0x7fdb2c0192a0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 -- 172.21.15.32:0/3066665463 shutdown_connections 2023-12-20T04:41:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb28065900 0x7fdb28067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb3c15a9d0 0x7fdb3c154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb3c0b5e30 0x7fdb3c1457a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 --2- 172.21.15.32:0/3066665463 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb3c0b2250 0x7fdb3c06b840 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:41:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 -- 172.21.15.32:0/3066665463 >> 172.21.15.32:0/3066665463 conn(0x7fdb3c0cd250 msgr2=0x7fdb3c0caa80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:41:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 -- 172.21.15.32:0/3066665463 shutdown_connections 2023-12-20T04:41:13.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:41:13.853+0000 7fdb413c0640 1 -- 172.21.15.32:0/3066665463 wait complete. 2023-12-20T04:41:13.874 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M", "2M"], "retention": [{}, {}]} 2023-12-20T04:41:13.874 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M', '2M'], 'retention': [{}, {}]} 2023-12-20T04:41:13.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:13.874 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:13.885 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:13.885 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:13.885 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:14.887 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:14.887 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:14.900 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:14.900 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:14.900 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:15.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:15.901 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:15.914 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:15.914 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:15.915 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:16.915 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:16.916 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:16.928 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:16.928 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:16.928 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:17.929 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:17.929 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:17.942 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:17.942 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:17.942 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:18.943 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:18.943 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:18.955 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:18.956 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:18.956 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:19.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:19.957 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:19.970 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:19.970 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:19.970 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:20.971 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:20.971 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:20.984 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:20.984 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:20.984 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:21.985 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:21.985 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:21.998 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:21.998 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:21.998 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:23.000 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:23.000 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:23.013 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:23.013 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:23.013 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:24.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:24.014 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:24.028 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:24.028 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:24.028 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:25.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:25.029 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:25.042 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:25.042 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:25.042 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:26.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:26.042 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:26.055 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:26.055 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:26.055 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:27.055 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:27.056 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:27.068 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:27.069 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:27.069 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:28.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:28.070 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:28.082 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:28.082 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:28.082 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:29.083 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:29.084 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:29.096 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:29.096 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:29.096 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:30.097 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:30.098 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:30.110 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:30.111 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:30.111 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:30.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:41:30.882 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:41:31.112 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:31.112 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:31.124 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:31.125 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:31.125 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:32.125 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:32.126 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:32.138 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:32.138 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:32.138 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:33.139 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:33.139 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:33.151 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:33.152 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:33.152 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:34.152 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:34.153 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:34.165 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:34.165 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:34.165 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:35.166 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:35.167 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:35.179 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:35.179 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:35.179 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:36.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:36.180 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:36.193 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:36.193 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:36.193 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:37.193 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:37.194 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:37.206 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:37.206 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:37.206 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:38.207 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:38.207 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:38.220 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:38.220 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:38.220 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:39.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:39.221 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:39.233 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:39.234 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:39.234 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:40.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:40.234 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:40.247 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:40.247 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:40.247 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:41.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:41.248 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:41.261 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:41.261 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:41.261 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:42.262 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:42.262 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:42.275 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:42.275 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:42.275 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:43.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:43.277 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:43.289 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:43.290 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:43.290 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:44.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:44.291 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:44.303 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:44.303 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:44.303 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:45.304 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:45.305 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:45.317 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:45.317 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:45.317 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:46.318 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:46.319 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:46.331 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:46.331 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:46.332 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:47.332 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:47.332 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:47.346 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:47.346 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:47.346 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:48.348 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:48.348 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:48.360 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:48.361 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:48.361 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:49.362 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:49.362 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:49.375 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:49.375 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:49.375 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:50.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:50.376 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:50.389 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:50.389 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:50.389 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:51.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:51.390 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:51.402 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:51.402 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:51.403 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:52.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:52.403 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:52.416 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:52.416 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:52.416 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:53.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:53.417 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:53.429 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:53.430 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:53.430 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:54.432 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:54.432 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:54.445 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:54.445 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:54.445 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:55.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:55.446 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:55.459 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:55.459 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:55.459 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:56.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:56.460 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:56.473 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:56.474 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:56.474 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:57.475 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:57.475 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:57.487 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:57.487 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:57.487 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:58.488 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:58.488 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:58.501 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:58.501 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:58.501 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:41:59.502 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:41:59.503 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:41:59.515 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:41:59.515 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:41:59.516 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:00.493 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:42:00.492+0000 7f4a80887640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:42:00.494 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:42:00.492+0000 7f4a80887640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:42:00.494 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:42:00.492+0000 7f4a80887640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:42:00.494 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:42:00.492+0000 7f4a80887640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:42:00.494 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:42:00.492+0000 7f4a80887640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:42:00.516 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:00.516 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:00.527 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:00.528 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:00.528 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:00.528 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:00.528 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-20-04_42_00_UTC'] 2023-12-20T04:42:00.528 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703047273.5040474, now = 1703047320.528695, timo = 47 2023-12-20T04:42:00.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:42:00.946 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:42:01.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:01.529 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:01.540 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:01.541 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:01.541 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:01.541 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:02.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:02.542 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:02.553 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:02.555 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:02.555 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:02.555 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:03.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:03.556 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:03.568 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:03.569 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:03.569 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:03.569 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:04.570 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:04.570 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:04.581 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:04.582 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:04.582 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:04.583 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:05.583 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:05.583 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:05.595 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:05.596 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:05.596 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:05.596 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:06.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:06.597 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:06.609 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:06.610 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:06.610 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:06.610 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:07.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:07.611 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:07.624 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:07.625 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:07.625 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:07.625 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:08.626 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:08.626 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:08.637 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:08.638 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:08.638 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:08.638 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:09.639 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:09.639 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:09.650 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:09.651 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:09.651 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:09.651 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:10.652 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:10.652 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:10.664 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:10.665 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:10.665 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:10.666 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:11.667 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:11.667 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:11.679 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:11.679 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:11.680 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:11.680 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:12.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:12.681 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:12.693 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:12.694 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:12.694 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:12.694 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:13.695 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:13.695 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:13.707 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:13.708 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:13.708 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:13.708 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:14.709 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:14.709 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:14.721 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:14.722 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:14.722 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:14.722 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:15.723 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:15.723 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:15.735 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:15.736 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:15.736 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:15.736 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:16.737 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:16.737 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:16.748 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:16.750 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:16.750 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:16.750 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:17.751 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:17.751 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:17.763 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:17.764 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:17.764 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:17.764 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:18.764 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:18.765 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:18.776 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:18.777 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:18.777 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:18.777 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:19.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:19.778 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:19.790 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:19.791 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:19.791 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:19.791 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:20.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:20.792 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:20.804 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:20.805 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:20.806 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:20.806 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:21.806 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:21.807 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:21.818 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:21.819 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:21.819 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:21.819 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:22.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:22.820 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:22.833 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:22.834 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:22.834 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:22.834 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:23.835 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:23.836 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:23.848 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:23.849 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:23.849 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:23.849 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:24.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:24.850 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:24.862 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:24.863 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:24.863 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:24.863 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:25.863 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:25.864 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:25.875 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:25.876 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:25.876 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:25.876 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:26.877 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:26.878 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:26.889 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:26.890 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:26.890 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:26.890 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:27.891 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:27.891 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:27.903 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:27.904 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:27.904 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:27.904 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:28.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:28.905 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:28.917 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:28.918 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:28.918 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:28.918 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:29.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:29.919 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:29.930 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:29.931 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:29.931 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:29.931 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:30.932 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:30.933 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:30.944 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:30.945 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:30.945 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:30.945 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:30.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:42:31.002 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:42:31.946 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:31.946 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:31.958 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:31.959 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:31.959 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:31.959 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:32.960 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:32.960 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:32.972 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:32.973 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:32.973 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:32.973 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:33.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:33.974 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:33.985 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:33.987 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:33.987 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:33.987 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:34.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:34.989 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:35.000 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:35.002 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:35.002 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:35.002 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:36.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:36.003 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:36.014 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:36.015 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:36.015 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:36.015 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:37.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:37.016 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:37.027 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:37.028 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:37.029 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:37.029 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:38.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:38.030 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:38.041 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:38.042 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:38.042 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:38.042 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:39.044 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:39.044 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:39.055 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:39.056 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:39.056 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:39.056 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:40.057 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:40.058 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:40.069 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:40.071 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:40.071 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:40.071 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:41.071 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:41.071 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:41.082 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:41.083 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:41.083 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:41.084 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:42.084 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:42.085 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:42.097 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:42.098 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:42.098 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:42.098 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:43.099 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:43.100 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:43.111 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:43.112 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:43.112 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:43.113 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:44.114 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:44.114 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:44.124 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:44.125 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:44.125 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:44.126 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:45.126 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:45.126 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:45.138 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:45.139 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:45.140 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:45.140 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:46.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:46.141 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:46.152 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:46.153 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:46.153 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:46.153 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:47.154 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:47.154 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:47.166 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:47.167 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:47.167 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:47.167 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:48.167 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:48.167 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:48.179 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:48.180 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:48.180 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:48.180 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:49.181 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:49.181 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:49.193 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:49.194 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:49.194 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:49.194 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:50.195 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:50.195 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:50.206 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:50.207 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:50.207 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:50.207 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:51.208 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:51.208 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:51.220 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:51.221 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:51.221 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:51.221 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:52.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:52.222 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:52.234 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:52.234 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:52.235 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:52.235 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:53.235 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:53.235 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:53.247 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:53.248 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:53.248 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:53.248 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:54.249 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:54.250 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:54.261 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:54.262 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:54.262 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:54.262 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:55.263 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:55.264 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:55.275 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:55.276 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:55.276 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:55.276 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:56.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:56.277 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:56.288 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:56.289 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:56.290 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:56.290 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:57.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:57.290 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:57.301 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:57.302 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:57.302 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:57.302 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:58.303 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:58.303 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:58.315 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:58.316 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:58.316 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:58.316 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:42:59.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:42:59.316 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:42:59.328 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:42:59.329 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:42:59.329 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:42:59.329 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:43:00.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:00.330 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:43:00.342 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:43:00.343 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:43:00.343 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:43:00.343 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:43:01.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:43:01.080 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:43:01.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:01.344 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:43:01.359 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:43:01.359 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_43_00_UTC 2023-12-20T04:43:01.359 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_43_00_UTC', 'scheduled-2023-12-20-04_42_00_UTC'} 2023-12-20T04:43:01.359 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-20-04_43_00_UTC'} 2023-12-20T04:43:01.359 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:43:01.359 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-20-04_43_00_UTC'] 2023-12-20T04:43:01.359 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703047273.5040474, now = 1703047381.3598576, timo = 107 2023-12-20T04:43:02.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.519+0000 7ffbe6d6d640 1 Processor -- start 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.519+0000 7ffbe6d6d640 1 -- start start 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.519+0000 7ffbe6d6d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbe015ce60 0x7ffbe00d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.519+0000 7ffbe6d6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe00d2650 con 0x7ffbe00d14d0 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.519+0000 7ffbe6d6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffbe00d1a40 con 0x7ffbe00d3130 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.519+0000 7ffbe6d6d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffbe00d35a0 con 0x7ffbe015ce60 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.519+0000 7ffbe4b09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffbe015ce60 0x7ffbe00d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.519+0000 7ffbe530a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffbe00d14d0 0x7ffbe00b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49476/0 (socket says 172.21.15.32:49476) 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.519+0000 7ffbe530a640 1 -- 172.21.15.32:0/391641739 learned_addr learned my addr 172.21.15.32:0/391641739 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.523+0000 7ffbe4b09640 1 -- 172.21.15.32:0/391641739 >> v1:172.21.15.39:6789/0 conn(0x7ffbe00d3130 legacy=0x7ffbe00d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.523+0000 7ffbe4b09640 1 -- 172.21.15.32:0/391641739 >> v1:172.21.15.32:6789/0 conn(0x7ffbe00d14d0 legacy=0x7ffbe00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:02.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.523+0000 7ffbdf7fe640 1 -- 172.21.15.32:0/391641739 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214996895 0 0) 0x7ffbe00d2650 con 0x7ffbe00d14d0 2023-12-20T04:43:02.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.523+0000 7ffbe4b09640 1 -- 172.21.15.32:0/391641739 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbe00d1d20 con 0x7ffbe015ce60 2023-12-20T04:43:02.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.523+0000 7ffbe4b09640 1 --2- 172.21.15.32:0/391641739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe015ce60 0x7ffbe00d2bf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40097f0 tx=0x7ffbd401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b06455f65513fa9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:02.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.523+0000 7ffbdf7fe640 1 -- 172.21.15.32:0/391641739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd40a8970 con 0x7ffbe015ce60 2023-12-20T04:43:02.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.523+0000 7ffbdf7fe640 1 -- 172.21.15.32:0/391641739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd40a8ad0 con 0x7ffbe015ce60 2023-12-20T04:43:02.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.523+0000 7ffbdf7fe640 1 -- 172.21.15.32:0/391641739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd40a8e60 con 0x7ffbe015ce60 2023-12-20T04:43:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/391641739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe015ce60 msgr2=0x7ffbe00d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 --2- 172.21.15.32:0/391641739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe015ce60 0x7ffbe00d2bf0 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7ffbd40097f0 tx=0x7ffbd401ec70 comp rx=0 tx=0).stop 2023-12-20T04:43:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/391641739 shutdown_connections 2023-12-20T04:43:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 --2- 172.21.15.32:0/391641739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe015ce60 0x7ffbe00d2bf0 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/391641739 >> 172.21.15.32:0/391641739 conn(0x7ffbe006b5e0 msgr2=0x7ffbe006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/391641739 shutdown_connections 2023-12-20T04:43:02.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/391641739 wait complete. 2023-12-20T04:43:02.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 Processor -- start 2023-12-20T04:43:02.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 -- start start 2023-12-20T04:43:02.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe00d14d0 0x7ffbe0148fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:02.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffbe00d3130 0x7ffbe01e4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:02.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbe015ce60 0x7ffbe01e38d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:02.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe0149500 con 0x7ffbe00d14d0 2023-12-20T04:43:02.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe01f21a0 con 0x7ffbe015ce60 2023-12-20T04:43:02.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe6d6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffbe01561e0 con 0x7ffbe00d3130 2023-12-20T04:43:02.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe4b09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe00d14d0 0x7ffbe0148fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:02.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe4b09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe00d14d0 0x7ffbe0148fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56854/0 (socket says 172.21.15.32:56854) 2023-12-20T04:43:02.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe4b09640 1 -- 172.21.15.32:0/3444760269 learned_addr learned my addr 172.21.15.32:0/3444760269 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:02.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe530a640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbe015ce60 0x7ffbe01e38d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:02.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbdffff640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffbe00d3130 0x7ffbe01e4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:02.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe4b09640 1 -- 172.21.15.32:0/3444760269 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffbe00d3130 msgr2=0x7ffbe01e4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:02.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe4b09640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffbe00d3130 0x7ffbe01e4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:02.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe4b09640 1 -- 172.21.15.32:0/3444760269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbe015ce60 msgr2=0x7ffbe01e38d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:02.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe4b09640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbe015ce60 0x7ffbe01e38d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:02.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbe4b09640 1 -- 172.21.15.32:0/3444760269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffbd4009420 con 0x7ffbe00d14d0 2023-12-20T04:43:02.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.527+0000 7ffbdffff640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffbe00d3130 0x7ffbe01e4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:43:02.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.531+0000 7ffbe4b09640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe00d14d0 0x7ffbe0148fc0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ffbd401b880 tx=0x7ffbd401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:02.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.531+0000 7ffbdd7fa640 1 -- 172.21.15.32:0/3444760269 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd40a8970 con 0x7ffbe00d14d0 2023-12-20T04:43:02.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.531+0000 7ffbdd7fa640 1 -- 172.21.15.32:0/3444760269 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffbd40b9410 con 0x7ffbe00d14d0 2023-12-20T04:43:02.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.531+0000 7ffbdd7fa640 1 -- 172.21.15.32:0/3444760269 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffbd40b97a0 con 0x7ffbe00d14d0 2023-12-20T04:43:02.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.531+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/3444760269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffbe01564c0 con 0x7ffbe00d14d0 2023-12-20T04:43:02.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.531+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/3444760269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffbe0156a80 con 0x7ffbe00d14d0 2023-12-20T04:43:02.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.531+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/3444760269 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffbe004e000 con 0x7ffbe00d14d0 2023-12-20T04:43:02.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.531+0000 7ffbdd7fa640 1 -- 172.21.15.32:0/3444760269 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 25) v1 ==== 85359+0+0 (secure 0 0 0) 0x7ffbd40b7040 con 0x7ffbe00d14d0 2023-12-20T04:43:02.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.531+0000 7ffbdd7fa640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffbc4065c00 0x7ffbc40680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:02.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.531+0000 7ffbdd7fa640 1 -- 172.21.15.32:0/3444760269 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6111+0+0 (secure 0 0 0) 0x7ffbd41212b0 con 0x7ffbe00d14d0 2023-12-20T04:43:02.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.531+0000 7ffbdffff640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffbc4065c00 0x7ffbc40680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:02.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.535+0000 7ffbdffff640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffbc4065c00 0x7ffbc40680c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffbe01497e0 tx=0x7ffbcc009040 comp rx=0 tx=0).ready entity=mgr.6081 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:02.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.535+0000 7ffbdd7fa640 1 -- 172.21.15.32:0/3444760269 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffbd40ee940 con 0x7ffbe00d14d0 2023-12-20T04:43:02.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.715+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/3444760269 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7ffbe00d22e0 con 0x7ffbc4065c00 2023-12-20T04:43:02.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.787+0000 7ffbdd7fa640 1 -- 172.21.15.32:0/3444760269 <== mgr.6081 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ffbcc0013f0 con 0x7ffbc4065c00 2023-12-20T04:43:02.786 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path snap_test_dir1 2023-12-20T04:43:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/3444760269 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffbc4065c00 msgr2=0x7ffbc40680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffbc4065c00 0x7ffbc40680c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ffbe01497e0 tx=0x7ffbcc009040 comp rx=0 tx=0).stop 2023-12-20T04:43:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/3444760269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe00d14d0 msgr2=0x7ffbe0148fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe00d14d0 0x7ffbe0148fc0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7ffbd401b880 tx=0x7ffbd401ba10 comp rx=0 tx=0).stop 2023-12-20T04:43:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/3444760269 shutdown_connections 2023-12-20T04:43:02.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffbc4065c00 0x7ffbc40680c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffbe015ce60 0x7ffbe01e38d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffbe00d3130 0x7ffbe01e4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 --2- 172.21.15.32:0/3444760269 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffbe00d14d0 0x7ffbe0148fc0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/3444760269 >> 172.21.15.32:0/3444760269 conn(0x7ffbe006b5e0 msgr2=0x7ffbe006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/3444760269 shutdown_connections 2023-12-20T04:43:02.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:02.791+0000 7ffbe6d6d640 1 -- 172.21.15.32:0/3444760269 wait complete. 2023-12-20T04:43:02.807 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-20T04:43:02.807 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:02.807 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:43:02.817 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:43:02.817 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_43_00_UTC 2023-12-20T04:43:02.818 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_42_00_UTC 2023-12-20T04:43:02.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:02.818 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_42_00_UTC) 2023-12-20T04:43:02.880 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_43_00_UTC 2023-12-20T04:43:02.880 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:02.881 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_43_00_UTC) 2023-12-20T04:43:02.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:02.938 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-20T04:43:02.994 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T04:43:03.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.187+0000 7fb40da41640 1 Processor -- start 2023-12-20T04:43:03.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.187+0000 7fb40da41640 1 -- start start 2023-12-20T04:43:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.187+0000 7fb40da41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4081591e0 0x7fb4080caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.187+0000 7fb40da41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb40806cdf0 0x7fb408070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.187+0000 7fb40da41640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4080cc3b0 con 0x7fb408070770 2023-12-20T04:43:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.187+0000 7fb40da41640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb408070be0 con 0x7fb40806cdf0 2023-12-20T04:43:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.187+0000 7fb40da41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4080c8e30 con 0x7fb4081591e0 2023-12-20T04:43:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb3fffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb40806cdf0 0x7fb408070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb406ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4081591e0 0x7fb4080caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb3fffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb40806cdf0 0x7fb408070230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56862/0 (socket says 172.21.15.32:56862) 2023-12-20T04:43:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb3fffff640 1 -- 172.21.15.32:0/2217770837 learned_addr learned my addr 172.21.15.32:0/2217770837 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb3fffff640 1 -- 172.21.15.32:0/2217770837 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4081591e0 msgr2=0x7fb4080caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb3fffff640 1 --2- 172.21.15.32:0/2217770837 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4081591e0 0x7fb4080caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb4067fc640 1 -- 172.21.15.32:0/2217770837 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4262371702 0 0) 0x7fb4080cc3b0 con 0x7fb408070770 2023-12-20T04:43:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb3fffff640 1 -- 172.21.15.32:0/2217770837 >> v1:172.21.15.39:6789/0 conn(0x7fb408070770 legacy=0x7fb40815cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb3fffff640 1 -- 172.21.15.32:0/2217770837 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb40806d280 con 0x7fb40806cdf0 2023-12-20T04:43:03.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb3fffff640 1 --2- 172.21.15.32:0/2217770837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb40806cdf0 0x7fb408070230 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009870 tx=0x7fb3f001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae4f65d20410a4b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:03.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb4067fc640 1 -- 172.21.15.32:0/2217770837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00a7070 con 0x7fb40806cdf0 2023-12-20T04:43:03.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb4067fc640 1 -- 172.21.15.32:0/2217770837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f00ab430 con 0x7fb40806cdf0 2023-12-20T04:43:03.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.191+0000 7fb4067fc640 1 -- 172.21.15.32:0/2217770837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f00ab7c0 con 0x7fb40806cdf0 2023-12-20T04:43:03.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 -- 172.21.15.32:0/2217770837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb40806cdf0 msgr2=0x7fb408070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:03.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 --2- 172.21.15.32:0/2217770837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb40806cdf0 0x7fb408070230 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb3f0009870 tx=0x7fb3f001c4b0 comp rx=0 tx=0).stop 2023-12-20T04:43:03.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 -- 172.21.15.32:0/2217770837 shutdown_connections 2023-12-20T04:43:03.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 --2- 172.21.15.32:0/2217770837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb40806cdf0 0x7fb408070230 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:03.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 --2- 172.21.15.32:0/2217770837 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4081591e0 0x7fb4080caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:03.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 -- 172.21.15.32:0/2217770837 >> 172.21.15.32:0/2217770837 conn(0x7fb40806a510 msgr2=0x7fb4080ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:03.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 -- 172.21.15.32:0/2217770837 shutdown_connections 2023-12-20T04:43:03.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 -- 172.21.15.32:0/2217770837 wait complete. 2023-12-20T04:43:03.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 Processor -- start 2023-12-20T04:43:03.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 -- start start 2023-12-20T04:43:03.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb408070770 0x7fb408141690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4081591e0 0x7fb4081622d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb408162810 0x7fb408155760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4080c6900 con 0x7fb408162810 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb408162d90 con 0x7fb4081591e0 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb40da41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb408163070 con 0x7fb408070770 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb406ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb408070770 0x7fb408141690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb4077fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb408162810 0x7fb408155760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb406ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb408070770 0x7fb408141690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:51648/0 (socket says 172.21.15.32:51648) 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb406ffd640 1 -- 172.21.15.32:0/1244074969 learned_addr learned my addr 172.21.15.32:0/1244074969 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.195+0000 7fb3fffff640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4081591e0 0x7fb4081622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb406ffd640 1 -- 172.21.15.32:0/1244074969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4081591e0 msgr2=0x7fb4081622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb406ffd640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4081591e0 0x7fb4081622d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:03.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb406ffd640 1 -- 172.21.15.32:0/1244074969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb408162810 msgr2=0x7fb408155760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb406ffd640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb408162810 0x7fb408155760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb406ffd640 1 -- 172.21.15.32:0/1244074969 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3f00094d0 con 0x7fb408070770 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb406ffd640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb408070770 0x7fb408141690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800e6e0 tx=0x7fb3f800eba0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb404ff9640 1 -- 172.21.15.32:0/1244074969 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f800ca30 con 0x7fb408070770 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb404ff9640 1 -- 172.21.15.32:0/1244074969 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3f800cb90 con 0x7fb408070770 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb404ff9640 1 -- 172.21.15.32:0/1244074969 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3f80113f0 con 0x7fb408070770 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb40da41640 1 -- 172.21.15.32:0/1244074969 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4080c6b50 con 0x7fb408070770 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb40da41640 1 -- 172.21.15.32:0/1244074969 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb408163a40 con 0x7fb408070770 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb40da41640 1 -- 172.21.15.32:0/1244074969 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb408163230 con 0x7fb408070770 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb404ff9640 1 -- 172.21.15.32:0/1244074969 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 25) v1 ==== 85359+0+0 (secure 0 0 0) 0x7fb3f8011550 con 0x7fb408070770 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb404ff9640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb3e8065bd0 0x7fb3e8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb404ff9640 1 -- 172.21.15.32:0/1244074969 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fb3f8083360 con 0x7fb408070770 2023-12-20T04:43:03.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.199+0000 7fb3fffff640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb3e8065bd0 0x7fb3e8068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:03.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.203+0000 7fb3fffff640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb3e8065bd0 0x7fb3e8068090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00af000 tx=0x7fb3f00a7000 comp rx=0 tx=0).ready entity=mgr.6081 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:03.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.207+0000 7fb404ff9640 1 -- 172.21.15.32:0/1244074969 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb3f8050980 con 0x7fb408070770 2023-12-20T04:43:03.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:03.403+0000 7fb40da41640 1 -- 172.21.15.32:0/1244074969 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb40804e000 con 0x7fb408070770 2023-12-20T04:43:04.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.107+0000 7fb404ff9640 1 -- 172.21.15.32:0/1244074969 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fb3f8050d50 con 0x7fb408070770 2023-12-20T04:43:04.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.107+0000 7fb404ff9640 1 -- 172.21.15.32:0/1244074969 <== mon.2 v2:172.21.15.39:3301/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v26) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb3f807f650 con 0x7fb408070770 2023-12-20T04:43:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 -- 172.21.15.32:0/1244074969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb3e8065bd0 msgr2=0x7fb3e8068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb3e8065bd0 0x7fb3e8068090 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb3f00af000 tx=0x7fb3f00a7000 comp rx=0 tx=0).stop 2023-12-20T04:43:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 -- 172.21.15.32:0/1244074969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb408070770 msgr2=0x7fb408141690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb408070770 0x7fb408141690 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7fb3f800e6e0 tx=0x7fb3f800eba0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 -- 172.21.15.32:0/1244074969 shutdown_connections 2023-12-20T04:43:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb3e8065bd0 0x7fb3e8068090 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb408162810 0x7fb408155760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4081591e0 0x7fb4081622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 --2- 172.21.15.32:0/1244074969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb408070770 0x7fb408141690 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 -- 172.21.15.32:0/1244074969 >> 172.21.15.32:0/1244074969 conn(0x7fb40806a510 msgr2=0x7fb40814eab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 -- 172.21.15.32:0/1244074969 shutdown_connections 2023-12-20T04:43:04.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.111+0000 7fb40da41640 1 -- 172.21.15.32:0/1244074969 wait complete. 2023-12-20T04:43:04.132 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:43:04.132 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:43:04.254 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:43:04.281 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:04.283+0000 7fa2d0d99180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:43:04.284 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:04.284+0000 7fd5b4c2d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:43:04.349 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:43:04.349 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:43:04.349 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:04.361 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:43:04.361 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:04.412 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:04.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f690cce5640 1 Processor -- start 2023-12-20T04:43:04.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f690cce5640 1 -- start start 2023-12-20T04:43:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f690cce5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f690815ce60 0x7f69080c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f690cce5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69080713e0 0x7f6908071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f690cce5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6908071d80 con 0x7f6908160b30 2023-12-20T04:43:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f690cce5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69081616d0 con 0x7f69080713e0 2023-12-20T04:43:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f690cce5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69080c9a40 con 0x7f690815ce60 2023-12-20T04:43:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f68fffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69080713e0 0x7f6908071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f68fffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69080713e0 0x7f6908071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56888/0 (socket says 172.21.15.32:56888) 2023-12-20T04:43:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f68fffff640 1 -- 172.21.15.32:0/2115496827 learned_addr learned my addr 172.21.15.32:0/2115496827 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:04.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f690659c640 1 --2- 172.21.15.32:0/2115496827 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f690815ce60 0x7f69080c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:04.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f68fffff640 1 -- 172.21.15.32:0/2115496827 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f690815ce60 msgr2=0x7f69080c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:04.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f68fffff640 1 --2- 172.21.15.32:0/2115496827 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f690815ce60 0x7f69080c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f68fffff640 1 -- 172.21.15.32:0/2115496827 >> v1:172.21.15.32:6789/0 conn(0x7f6908160b30 legacy=0x7f6908160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:04.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.603+0000 7f68fffff640 1 -- 172.21.15.32:0/2115496827 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69080c9180 con 0x7f69080713e0 2023-12-20T04:43:04.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f68fffff640 1 --2- 172.21.15.32:0/2115496827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69080713e0 0x7f6908071840 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009870 tx=0x7f68f001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f711b5ff7b46ba7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:04.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f6905d9b640 1 -- 172.21.15.32:0/2115496827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f00a7070 con 0x7f69080713e0 2023-12-20T04:43:04.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f6905d9b640 1 -- 172.21.15.32:0/2115496827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68f00ab430 con 0x7f69080713e0 2023-12-20T04:43:04.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f6905d9b640 1 -- 172.21.15.32:0/2115496827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68f00ab7c0 con 0x7f69080713e0 2023-12-20T04:43:04.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f690cce5640 1 -- 172.21.15.32:0/2115496827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69080713e0 msgr2=0x7f6908071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:04.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f690cce5640 1 --2- 172.21.15.32:0/2115496827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69080713e0 0x7f6908071840 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f68f0009870 tx=0x7f68f001c540 comp rx=0 tx=0).stop 2023-12-20T04:43:04.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f690cce5640 1 -- 172.21.15.32:0/2115496827 shutdown_connections 2023-12-20T04:43:04.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f690cce5640 1 --2- 172.21.15.32:0/2115496827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69080713e0 0x7f6908071840 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f690cce5640 1 --2- 172.21.15.32:0/2115496827 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f690815ce60 0x7f69080c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f690cce5640 1 -- 172.21.15.32:0/2115496827 >> 172.21.15.32:0/2115496827 conn(0x7f690806a320 msgr2=0x7f69080ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:04.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f690cce5640 1 -- 172.21.15.32:0/2115496827 shutdown_connections 2023-12-20T04:43:04.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.607+0000 7f690cce5640 1 -- 172.21.15.32:0/2115496827 wait complete. 2023-12-20T04:43:04.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690cce5640 1 Processor -- start 2023-12-20T04:43:04.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690cce5640 1 -- start start 2023-12-20T04:43:04.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690cce5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f69080713e0 0x7f6908154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690cce5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690815ce60 0x7f69080c2090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690cce5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6908160b30 0x7f69080cced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690cce5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69080c28e0 con 0x7f6908160b30 2023-12-20T04:43:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690cce5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69080c2bc0 con 0x7f690815ce60 2023-12-20T04:43:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690cce5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69080c2ea0 con 0x7f69080713e0 2023-12-20T04:43:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f6906d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6908160b30 0x7f69080cced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690659c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f69080713e0 0x7f6908154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690659c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f69080713e0 0x7f6908154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:51700/0 (socket says 172.21.15.32:51700) 2023-12-20T04:43:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f68fffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690815ce60 0x7f69080c2090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f6906d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6908160b30 0x7f69080cced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56896/0 (socket says 172.21.15.32:56896) 2023-12-20T04:43:04.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690659c640 1 -- 172.21.15.32:0/598435430 learned_addr learned my addr 172.21.15.32:0/598435430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f6906d9d640 1 -- 172.21.15.32:0/598435430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f69080713e0 msgr2=0x7f6908154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f6906d9d640 1 --2- 172.21.15.32:0/598435430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f69080713e0 0x7f6908154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f6906d9d640 1 -- 172.21.15.32:0/598435430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690815ce60 msgr2=0x7f69080c2090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f6906d9d640 1 --2- 172.21.15.32:0/598435430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690815ce60 0x7f69080c2090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f6906d9d640 1 -- 172.21.15.32:0/598435430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68f00094d0 con 0x7f6908160b30 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f6906d9d640 1 --2- 172.21.15.32:0/598435430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6908160b30 0x7f69080cced0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f690000c700 tx=0x7f690000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f68feffd640 1 -- 172.21.15.32:0/598435430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900003b80 con 0x7f6908160b30 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f68feffd640 1 -- 172.21.15.32:0/598435430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6900003ce0 con 0x7f6908160b30 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f68feffd640 1 -- 172.21.15.32:0/598435430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6900005310 con 0x7f6908160b30 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690cce5640 1 -- 172.21.15.32:0/598435430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69080c3180 con 0x7f6908160b30 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690cce5640 1 -- 172.21.15.32:0/598435430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6908155270 con 0x7f6908160b30 2023-12-20T04:43:04.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.611+0000 7f690cce5640 1 -- 172.21.15.32:0/598435430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69080c6340 con 0x7f6908160b30 2023-12-20T04:43:04.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.615+0000 7f68feffd640 1 -- 172.21.15.32:0/598435430 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f6900005470 con 0x7f6908160b30 2023-12-20T04:43:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.615+0000 7f68feffd640 1 --2- 172.21.15.32:0/598435430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68e8065b80 0x7f68e8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.615+0000 7f68feffd640 1 -- 172.21.15.32:0/598435430 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f6900082fe0 con 0x7f6908160b30 2023-12-20T04:43:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.615+0000 7f690659c640 1 -- 172.21.15.32:0/598435430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68e8065b80 msgr2=0x7f68e8068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:04.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.615+0000 7f690659c640 1 --2- 172.21.15.32:0/598435430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68e8065b80 0x7f68e8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:04.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.623+0000 7f68feffd640 1 -- 172.21.15.32:0/598435430 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6900050800 con 0x7f6908160b30 2023-12-20T04:43:04.721 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:04.720+0000 7fd5b4c2d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:43:04.730 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:04.731+0000 7fa2d0d99180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:43:04.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.787+0000 7f690cce5640 1 -- 172.21.15.32:0/598435430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f690804e000 con 0x7f6908160b30 2023-12-20T04:43:04.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.787+0000 7f68feffd640 1 -- 172.21.15.32:0/598435430 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v64) v1 ==== 74+0+16577 (secure 0 0 0) 0x7f6900053e20 con 0x7f6908160b30 2023-12-20T04:43:04.787 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:04.787 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":64,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:42:13.074116+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:40:57.540206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-20T04:40:58.913164+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"61","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":5}]},{"pgid":"7.2a","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2279832503":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/1700967844":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/384810653":"2023-12-21T04:41:11.975357+0000","172.21.15.32:6832/38009":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/935462264":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/936743272":"2023-12-21T04:41:11.975357+0000","172.21.15.32:6833/38009":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/3773919514":"2023-12-21T04:41:11.975357+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 -- 172.21.15.32:0/598435430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68e8065b80 msgr2=0x7f68e8068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 --2- 172.21.15.32:0/598435430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68e8065b80 0x7f68e8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 -- 172.21.15.32:0/598435430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6908160b30 msgr2=0x7f69080cced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 --2- 172.21.15.32:0/598435430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6908160b30 0x7f69080cced0 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7f690000c700 tx=0x7f690000cbc0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 -- 172.21.15.32:0/598435430 shutdown_connections 2023-12-20T04:43:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 --2- 172.21.15.32:0/598435430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68e8065b80 0x7f68e8068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 --2- 172.21.15.32:0/598435430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6908160b30 0x7f69080cced0 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 --2- 172.21.15.32:0/598435430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f690815ce60 0x7f69080c2090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 --2- 172.21.15.32:0/598435430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f69080713e0 0x7f6908154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 -- 172.21.15.32:0/598435430 >> 172.21.15.32:0/598435430 conn(0x7f690806a320 msgr2=0x7f69080b2270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 -- 172.21.15.32:0/598435430 shutdown_connections 2023-12-20T04:43:04.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.791+0000 7f690cce5640 1 -- 172.21.15.32:0/598435430 wait complete. 2023-12-20T04:43:04.810 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:40:57.540206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:40:58.913164+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '61', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:43:04.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:43:04.878 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:04.876+0000 7fd5b4c2d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:43:04.894 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:04.895+0000 7fa2d0d99180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:43:04.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1c38f640 1 Processor -- start 2023-12-20T04:43:04.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1c38f640 1 -- start start 2023-12-20T04:43:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1c38f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ffb1415ce60 0x7ffb140d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1c38f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb140d14d0 0x7ffb140b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1c38f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb140d2650 con 0x7ffb140d3130 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1c38f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb140d35a0 con 0x7ffb1415ce60 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1c38f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140d1a40 con 0x7ffb140d14d0 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1a92c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb140d14d0 0x7ffb140b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1a12b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ffb1415ce60 0x7ffb140d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1992a640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ffb140d3130 0x7ffb140d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:42218/0 (socket says 172.21.15.32:42218) 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1992a640 1 -- 172.21.15.32:0/824486785 learned_addr learned my addr 172.21.15.32:0/824486785 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb19129640 1 -- 172.21.15.32:0/824486785 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 918245669 0 0) 0x7ffb140d2650 con 0x7ffb140d3130 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1a12b640 1 -- 172.21.15.32:0/824486785 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb140d14d0 msgr2=0x7ffb140b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1a12b640 1 --2- 172.21.15.32:0/824486785 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb140d14d0 0x7ffb140b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1a12b640 1 -- 172.21.15.32:0/824486785 >> v1:172.21.15.39:6789/0 conn(0x7ffb140d3130 legacy=0x7ffb140d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:04.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.963+0000 7ffb1a12b640 1 -- 172.21.15.32:0/824486785 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140d1d20 con 0x7ffb1415ce60 2023-12-20T04:43:04.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb1a12b640 1 --2- 172.21.15.32:0/824486785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb1415ce60 0x7ffb140d2bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffb100097c0 tx=0x7ffb1001ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3896bd1ffe5ac898 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb19129640 1 -- 172.21.15.32:0/824486785 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100a8970 con 0x7ffb1415ce60 2023-12-20T04:43:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb19129640 1 -- 172.21.15.32:0/824486785 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb100a8ad0 con 0x7ffb1415ce60 2023-12-20T04:43:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb19129640 1 -- 172.21.15.32:0/824486785 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100a8e60 con 0x7ffb1415ce60 2023-12-20T04:43:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb1c38f640 1 -- 172.21.15.32:0/824486785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb1415ce60 msgr2=0x7ffb140d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:04.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb1c38f640 1 --2- 172.21.15.32:0/824486785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb1415ce60 0x7ffb140d2bf0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7ffb100097c0 tx=0x7ffb1001ec70 comp rx=0 tx=0).stop 2023-12-20T04:43:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb1c38f640 1 -- 172.21.15.32:0/824486785 shutdown_connections 2023-12-20T04:43:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb1c38f640 1 --2- 172.21.15.32:0/824486785 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb140d14d0 0x7ffb140b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb1c38f640 1 --2- 172.21.15.32:0/824486785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb1415ce60 0x7ffb140d2bf0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb1c38f640 1 -- 172.21.15.32:0/824486785 >> 172.21.15.32:0/824486785 conn(0x7ffb1406b5e0 msgr2=0x7ffb1406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb1c38f640 1 -- 172.21.15.32:0/824486785 shutdown_connections 2023-12-20T04:43:04.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.967+0000 7ffb1c38f640 1 -- 172.21.15.32:0/824486785 wait complete. 2023-12-20T04:43:04.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1c38f640 1 Processor -- start 2023-12-20T04:43:04.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1c38f640 1 -- start start 2023-12-20T04:43:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1c38f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140d14d0 0x7ffb1415b070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1c38f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb140d3130 0x7ffb140b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1c38f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb1415ce60 0x7ffb1414e010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1c38f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140b63d0 con 0x7ffb140d14d0 2023-12-20T04:43:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1c38f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1414e550 con 0x7ffb1415ce60 2023-12-20T04:43:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1c38f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb140b6ea0 con 0x7ffb140d3130 2023-12-20T04:43:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1a12b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140d14d0 0x7ffb1415b070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:04.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1a12b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140d14d0 0x7ffb1415b070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56898/0 (socket says 172.21.15.32:56898) 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1a12b640 1 -- 172.21.15.32:0/674871323 learned_addr learned my addr 172.21.15.32:0/674871323 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1992a640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb140d3130 0x7ffb140b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1a92c640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb1415ce60 0x7ffb1414e010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1a12b640 1 -- 172.21.15.32:0/674871323 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb140d3130 msgr2=0x7ffb140b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1a12b640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb140d3130 0x7ffb140b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1a12b640 1 -- 172.21.15.32:0/674871323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb1415ce60 msgr2=0x7ffb1414e010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1a12b640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb1415ce60 0x7ffb1414e010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1a12b640 1 -- 172.21.15.32:0/674871323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb10009420 con 0x7ffb140d14d0 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1a92c640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb1415ce60 0x7ffb1414e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1a12b640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140d14d0 0x7ffb1415b070 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ffb100b3040 tx=0x7ffb1001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb0affd640 1 -- 172.21.15.32:0/674871323 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100b9630 con 0x7ffb140d14d0 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb0affd640 1 -- 172.21.15.32:0/674871323 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb100b97b0 con 0x7ffb140d14d0 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb1c38f640 1 -- 172.21.15.32:0/674871323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb140b6600 con 0x7ffb140d14d0 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.971+0000 7ffb0affd640 1 -- 172.21.15.32:0/674871323 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb100b9b60 con 0x7ffb140d14d0 2023-12-20T04:43:04.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.975+0000 7ffb1c38f640 1 -- 172.21.15.32:0/674871323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb140b6bc0 con 0x7ffb140d14d0 2023-12-20T04:43:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.975+0000 7ffb1c38f640 1 -- 172.21.15.32:0/674871323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1404e000 con 0x7ffb140d14d0 2023-12-20T04:43:04.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.975+0000 7ffb0affd640 1 -- 172.21.15.32:0/674871323 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7ffb100b7040 con 0x7ffb140d14d0 2023-12-20T04:43:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.975+0000 7ffb0affd640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffaf8065b80 0x7ffaf8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.975+0000 7ffb1992a640 1 -- 172.21.15.32:0/674871323 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffaf8065b80 msgr2=0x7ffaf8068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.975+0000 7ffb0affd640 1 -- 172.21.15.32:0/674871323 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(64..64 src has 1..64) v4 ==== 6111+0+0 (secure 0 0 0) 0x7ffb10121390 con 0x7ffb140d14d0 2023-12-20T04:43:04.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.975+0000 7ffb1992a640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffaf8065b80 0x7ffaf8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:04.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:04.979+0000 7ffb0affd640 1 -- 172.21.15.32:0/674871323 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffb100eea40 con 0x7ffb140d14d0 2023-12-20T04:43:05.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 -- 172.21.15.32:0/674871323 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ffb140d22b0 con 0x7ffb140d14d0 2023-12-20T04:43:05.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb0affd640 1 -- 172.21.15.32:0/674871323 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v65) v1 ==== 93+0+10 (secure 0 0 0) 0x7ffb100f2060 con 0x7ffb140d14d0 2023-12-20T04:43:05.141 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 -- 172.21.15.32:0/674871323 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffaf8065b80 msgr2=0x7ffaf8068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffaf8065b80 0x7ffaf8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 -- 172.21.15.32:0/674871323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140d14d0 msgr2=0x7ffb1415b070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140d14d0 0x7ffb1415b070 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7ffb100b3040 tx=0x7ffb1001ba10 comp rx=0 tx=0).stop 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 -- 172.21.15.32:0/674871323 shutdown_connections 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffaf8065b80 0x7ffaf8068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb1415ce60 0x7ffb1414e010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb140d3130 0x7ffb140b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 --2- 172.21.15.32:0/674871323 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140d14d0 0x7ffb1415b070 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 -- 172.21.15.32:0/674871323 >> 172.21.15.32:0/674871323 conn(0x7ffb1406b5e0 msgr2=0x7ffb1406b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 -- 172.21.15.32:0/674871323 shutdown_connections 2023-12-20T04:43:05.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.143+0000 7ffb1c38f640 1 -- 172.21.15.32:0/674871323 wait complete. 2023-12-20T04:43:05.163 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:43:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.315+0000 7f62b1d09640 1 Processor -- start 2023-12-20T04:43:05.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.315+0000 7f62b1d09640 1 -- start start 2023-12-20T04:43:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.315+0000 7f62b1d09640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f62ac15ce60 0x7f62ac0d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.315+0000 7f62b1d09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62ac0d3130 0x7f62ac0d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.315+0000 7f62b1d09640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62ac0d2650 con 0x7f62ac0d1300 2023-12-20T04:43:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.315+0000 7f62b1d09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ac0d35a0 con 0x7f62ac0d3130 2023-12-20T04:43:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.315+0000 7f62b1d09640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62ac0d1a40 con 0x7f62ac15ce60 2023-12-20T04:43:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.315+0000 7f62aaffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62ac0d3130 0x7f62ac0d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.315+0000 7f62aaffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62ac0d3130 0x7f62ac0d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56914/0 (socket says 172.21.15.32:56914) 2023-12-20T04:43:05.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.315+0000 7f62aaffd640 1 -- 172.21.15.32:0/2967871095 learned_addr learned my addr 172.21.15.32:0/2967871095 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.319+0000 7f62ab7fe640 1 --2- 172.21.15.32:0/2967871095 >> v2:172.21.15.39:3300/0 conn(0x7f62ac15ce60 0x7f62ac0d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.319+0000 7f62aaffd640 1 -- 172.21.15.32:0/2967871095 >> v2:172.21.15.39:3300/0 conn(0x7f62ac15ce60 msgr2=0x7f62ac0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.319+0000 7f62aaffd640 1 --2- 172.21.15.32:0/2967871095 >> v2:172.21.15.39:3300/0 conn(0x7f62ac15ce60 0x7f62ac0d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.319+0000 7f62aaffd640 1 -- 172.21.15.32:0/2967871095 >> v1:172.21.15.39:6789/0 conn(0x7f62ac0d1300 legacy=0x7f62ac0b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:05.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.319+0000 7f62aaffd640 1 -- 172.21.15.32:0/2967871095 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62ac0d1d20 con 0x7f62ac0d3130 2023-12-20T04:43:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.319+0000 7f62aaffd640 1 --2- 172.21.15.32:0/2967871095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ac0d3130 0x7f62ac0d0dc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009870 tx=0x7f62a001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da2bb0a36c082a0a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.319+0000 7f62aa7fc640 1 -- 172.21.15.32:0/2967871095 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a00a89f0 con 0x7f62ac0d3130 2023-12-20T04:43:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.319+0000 7f62aa7fc640 1 -- 172.21.15.32:0/2967871095 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a00a8b50 con 0x7f62ac0d3130 2023-12-20T04:43:05.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.319+0000 7f62aa7fc640 1 -- 172.21.15.32:0/2967871095 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a00b0440 con 0x7f62ac0d3130 2023-12-20T04:43:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 -- 172.21.15.32:0/2967871095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ac0d3130 msgr2=0x7f62ac0d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 --2- 172.21.15.32:0/2967871095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ac0d3130 0x7f62ac0d0dc0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7f62a0009870 tx=0x7f62a001ecf0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 -- 172.21.15.32:0/2967871095 shutdown_connections 2023-12-20T04:43:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 --2- 172.21.15.32:0/2967871095 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ac0d3130 0x7f62ac0d0dc0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 --2- 172.21.15.32:0/2967871095 >> v2:172.21.15.39:3300/0 conn(0x7f62ac15ce60 0x7f62ac0d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 -- 172.21.15.32:0/2967871095 >> 172.21.15.32:0/2967871095 conn(0x7f62ac06b5e0 msgr2=0x7f62ac06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 -- 172.21.15.32:0/2967871095 shutdown_connections 2023-12-20T04:43:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 -- 172.21.15.32:0/2967871095 wait complete. 2023-12-20T04:43:05.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 Processor -- start 2023-12-20T04:43:05.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 -- start start 2023-12-20T04:43:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62ac0d1300 0x7f62ac1f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0d3130 0x7f62ac0cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ac15ce60 0x7f62ac0b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0cbaa0 con 0x7f62ac15ce60 2023-12-20T04:43:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0ac2b0 con 0x7f62ac0d3130 2023-12-20T04:43:05.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62b1d09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62ac0cbec0 con 0x7f62ac0d1300 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62abfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ac15ce60 0x7f62ac0b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62ab7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62ac0d1300 0x7f62ac1f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62aaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0d3130 0x7f62ac0cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62aaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0d3130 0x7f62ac0cc820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:47876/0 (socket says 172.21.15.32:47876) 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62aaffd640 1 -- 172.21.15.32:0/938813577 learned_addr learned my addr 172.21.15.32:0/938813577 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62aaffd640 1 -- 172.21.15.32:0/938813577 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62ac0d1300 msgr2=0x7f62ac1f1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62aaffd640 1 --2- 172.21.15.32:0/938813577 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62ac0d1300 0x7f62ac1f1800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62aaffd640 1 -- 172.21.15.32:0/938813577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ac15ce60 msgr2=0x7f62ac0b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62aaffd640 1 --2- 172.21.15.32:0/938813577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ac15ce60 0x7f62ac0b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.323+0000 7f62aaffd640 1 -- 172.21.15.32:0/938813577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62a00094d0 con 0x7f62ac0d3130 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f62aaffd640 1 --2- 172.21.15.32:0/938813577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0d3130 0x7f62ac0cc820 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f62a00b3040 tx=0x7f62a001ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f6293fff640 1 -- 172.21.15.32:0/938813577 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a00be070 con 0x7f62ac0d3130 2023-12-20T04:43:05.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f6293fff640 1 -- 172.21.15.32:0/938813577 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62a00ba400 con 0x7f62ac0d3130 2023-12-20T04:43:05.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f62b1d09640 1 -- 172.21.15.32:0/938813577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62ac0cae20 con 0x7f62ac0d3130 2023-12-20T04:43:05.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f6293fff640 1 -- 172.21.15.32:0/938813577 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62a00ba7d0 con 0x7f62ac0d3130 2023-12-20T04:43:05.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f62b1d09640 1 -- 172.21.15.32:0/938813577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62ac0cb3e0 con 0x7f62ac0d3130 2023-12-20T04:43:05.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f62b1d09640 1 -- 172.21.15.32:0/938813577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62ac04e000 con 0x7f62ac0d3130 2023-12-20T04:43:05.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f6293fff640 1 -- 172.21.15.32:0/938813577 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f62a00c3410 con 0x7f62ac0d3130 2023-12-20T04:43:05.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f6293fff640 1 --2- 172.21.15.32:0/938813577 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6298065b80 0x7f6298068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f6293fff640 1 -- 172.21.15.32:0/938813577 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6199+0+0 (secure 0 0 0) 0x7f62a0120f20 con 0x7f62ac0d3130 2023-12-20T04:43:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f62ab7fe640 1 -- 172.21.15.32:0/938813577 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6298065b80 msgr2=0x7f6298068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:05.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.327+0000 7f62ab7fe640 1 --2- 172.21.15.32:0/938813577 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6298065b80 0x7f6298068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:05.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.335+0000 7f6293fff640 1 -- 172.21.15.32:0/938813577 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f62a00ee720 con 0x7f62ac0d3130 2023-12-20T04:43:05.389 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:05.388+0000 7fd5b4c2d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:05.427 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:05.427+0000 7fa2d0d99180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:05.450 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:05.452+0000 7fd5b4c2d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:43:05.490 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:05.491+0000 7fa2d0d99180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:43:05.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.495+0000 7f62b1d09640 1 -- 172.21.15.32:0/938813577 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f62ac0ab460 con 0x7f62ac0d3130 2023-12-20T04:43:05.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.495+0000 7f6293fff640 1 -- 172.21.15.32:0/938813577 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v65) v1 ==== 104+0+11 (secure 0 0 0) 0x7f62a00f1d40 con 0x7f62ac0d3130 2023-12-20T04:43:05.493 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:43:05.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.495+0000 7f62b1d09640 1 -- 172.21.15.32:0/938813577 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6298065b80 msgr2=0x7f6298068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:05.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.495+0000 7f62b1d09640 1 --2- 172.21.15.32:0/938813577 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6298065b80 0x7f6298068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.495+0000 7f62b1d09640 1 -- 172.21.15.32:0/938813577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0d3130 msgr2=0x7f62ac0cc820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:05.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.495+0000 7f62b1d09640 1 --2- 172.21.15.32:0/938813577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0d3130 0x7f62ac0cc820 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f62a00b3040 tx=0x7f62a001ba90 comp rx=0 tx=0).stop 2023-12-20T04:43:05.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.499+0000 7f62b1d09640 1 -- 172.21.15.32:0/938813577 shutdown_connections 2023-12-20T04:43:05.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.499+0000 7f62b1d09640 1 --2- 172.21.15.32:0/938813577 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6298065b80 0x7f6298068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.499+0000 7f62b1d09640 1 --2- 172.21.15.32:0/938813577 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62ac15ce60 0x7f62ac0b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.499+0000 7f62b1d09640 1 --2- 172.21.15.32:0/938813577 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62ac0d3130 0x7f62ac0cc820 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.499+0000 7f62b1d09640 1 --2- 172.21.15.32:0/938813577 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62ac0d1300 0x7f62ac1f1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.499+0000 7f62b1d09640 1 -- 172.21.15.32:0/938813577 >> 172.21.15.32:0/938813577 conn(0x7f62ac06b5e0 msgr2=0x7f62ac06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:05.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.499+0000 7f62b1d09640 1 -- 172.21.15.32:0/938813577 shutdown_connections 2023-12-20T04:43:05.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.499+0000 7f62b1d09640 1 -- 172.21.15.32:0/938813577 wait complete. 2023-12-20T04:43:05.509 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:05.508+0000 7fd5b4c2d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:43:05.515 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:43:05.550 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:05.551+0000 7fa2d0d99180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:43:05.600 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:05.600+0000 7fd5b4c2d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:43:05.644 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:05.647+0000 7fa2d0d99180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:43:05.661 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:05.660+0000 7fd5b4c2d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:43:05.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.667+0000 7f27f5339640 1 Processor -- start 2023-12-20T04:43:05.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.667+0000 7f27f5339640 1 -- start start 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.667+0000 7f27f5339640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27f015cb30 0x7f27f00b8300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.667+0000 7f27f5339640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f27f00cddc0 0x7f27f00d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.667+0000 7f27f5339640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27f00d0d40 con 0x7f27f01606b0 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.667+0000 7f27f5339640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f0160bb0 con 0x7f27f015cb30 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.667+0000 7f27f5339640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f0160e90 con 0x7f27f00cddc0 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27eeffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27f015cb30 0x7f27f00b8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27eeffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27f015cb30 0x7f27f00b8300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56926/0 (socket says 172.21.15.32:56926) 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27eeffd640 1 -- 172.21.15.32:0/3979400824 learned_addr learned my addr 172.21.15.32:0/3979400824 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27ef7fe640 1 --2- 172.21.15.32:0/3979400824 >> v2:172.21.15.39:3300/0 conn(0x7f27f00cddc0 0x7f27f00d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27eeffd640 1 -- 172.21.15.32:0/3979400824 >> v2:172.21.15.39:3300/0 conn(0x7f27f00cddc0 msgr2=0x7f27f00d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27eeffd640 1 --2- 172.21.15.32:0/3979400824 >> v2:172.21.15.39:3300/0 conn(0x7f27f00cddc0 0x7f27f00d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27eeffd640 1 -- 172.21.15.32:0/3979400824 >> v1:172.21.15.32:6789/0 conn(0x7f27f01606b0 legacy=0x7f27f00cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27eeffd640 1 -- 172.21.15.32:0/3979400824 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f0161170 con 0x7f27f015cb30 2023-12-20T04:43:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27eeffd640 1 --2- 172.21.15.32:0/3979400824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f015cb30 0x7f27f00b8300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f27f006c000 tx=0x7f27e001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9f7ae5f80099d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27ee7fc640 1 -- 172.21.15.32:0/3979400824 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00ab660 con 0x7f27f015cb30 2023-12-20T04:43:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27ee7fc640 1 -- 172.21.15.32:0/3979400824 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e00ab7c0 con 0x7f27f015cb30 2023-12-20T04:43:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.671+0000 7f27ee7fc640 1 -- 172.21.15.32:0/3979400824 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00abb50 con 0x7f27f015cb30 2023-12-20T04:43:05.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 -- 172.21.15.32:0/3979400824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f015cb30 msgr2=0x7f27f00b8300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 --2- 172.21.15.32:0/3979400824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f015cb30 0x7f27f00b8300 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7f27f006c000 tx=0x7f27e001c4d0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 -- 172.21.15.32:0/3979400824 shutdown_connections 2023-12-20T04:43:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 --2- 172.21.15.32:0/3979400824 >> v2:172.21.15.39:3300/0 conn(0x7f27f00cddc0 0x7f27f00d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 --2- 172.21.15.32:0/3979400824 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f015cb30 0x7f27f00b8300 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 -- 172.21.15.32:0/3979400824 >> 172.21.15.32:0/3979400824 conn(0x7f27f006af60 msgr2=0x7f27f006b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 -- 172.21.15.32:0/3979400824 shutdown_connections 2023-12-20T04:43:05.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 -- 172.21.15.32:0/3979400824 wait complete. 2023-12-20T04:43:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 Processor -- start 2023-12-20T04:43:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 -- start start 2023-12-20T04:43:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00cddc0 0x7f27f015e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27f015cb30 0x7f27f015a080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f01606b0 0x7f27f01499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f015e920 con 0x7f27f00cddc0 2023-12-20T04:43:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f015b490 con 0x7f27f015cb30 2023-12-20T04:43:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27f5339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f015a5c0 con 0x7f27f01606b0 2023-12-20T04:43:05.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27eeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00cddc0 0x7f27f015e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27eeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00cddc0 0x7f27f015e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56938/0 (socket says 172.21.15.32:56938) 2023-12-20T04:43:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27eeffd640 1 -- 172.21.15.32:0/1062756914 learned_addr learned my addr 172.21.15.32:0/1062756914 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.675+0000 7f27e7fff640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27f015cb30 0x7f27f015a080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27ef7fe640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f01606b0 0x7f27f01499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27eeffd640 1 -- 172.21.15.32:0/1062756914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f01606b0 msgr2=0x7f27f01499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27eeffd640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f01606b0 0x7f27f01499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27eeffd640 1 -- 172.21.15.32:0/1062756914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27f015cb30 msgr2=0x7f27f015a080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27eeffd640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27f015cb30 0x7f27f015a080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27eeffd640 1 -- 172.21.15.32:0/1062756914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e0009480 con 0x7f27f00cddc0 2023-12-20T04:43:05.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27eeffd640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00cddc0 0x7f27f015e3e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f27e00190e0 tx=0x7f27e0019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:05.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27e77fe640 1 -- 172.21.15.32:0/1062756914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e0005ba0 con 0x7f27f00cddc0 2023-12-20T04:43:05.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27e77fe640 1 -- 172.21.15.32:0/1062756914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e0005d00 con 0x7f27f00cddc0 2023-12-20T04:43:05.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27f5339640 1 -- 172.21.15.32:0/1062756914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f015a880 con 0x7f27f00cddc0 2023-12-20T04:43:05.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27e77fe640 1 -- 172.21.15.32:0/1062756914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00abb30 con 0x7f27f00cddc0 2023-12-20T04:43:05.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27f5339640 1 -- 172.21.15.32:0/1062756914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f015ae40 con 0x7f27f00cddc0 2023-12-20T04:43:05.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27f5339640 1 -- 172.21.15.32:0/1062756914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f0074100 con 0x7f27f00cddc0 2023-12-20T04:43:05.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27e77fe640 1 -- 172.21.15.32:0/1062756914 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f27e00a9070 con 0x7f27f00cddc0 2023-12-20T04:43:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27e77fe640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f27d0065b80 0x7f27d0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27e77fe640 1 -- 172.21.15.32:0/1062756914 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6199+0+0 (secure 0 0 0) 0x7f27e011d2f0 con 0x7f27f00cddc0 2023-12-20T04:43:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27e7fff640 1 -- 172.21.15.32:0/1062756914 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f27d0065b80 msgr2=0x7f27d0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:05.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.679+0000 7f27e7fff640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f27d0065b80 0x7f27d0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:05.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.683+0000 7f27e77fe640 1 -- 172.21.15.32:0/1062756914 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f27e00ea8b0 con 0x7f27f00cddc0 2023-12-20T04:43:05.707 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:05.711+0000 7fa2d0d99180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:43:05.753 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:05.752+0000 7fd5b4c2d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:43:05.797 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:05.799+0000 7fa2d0d99180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:43:05.812 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:05.812+0000 7fd5b4c2d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:43:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.843+0000 7f27f5339640 1 -- 172.21.15.32:0/1062756914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f27f00d14f0 con 0x7f27f00cddc0 2023-12-20T04:43:05.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.843+0000 7f27e77fe640 1 -- 172.21.15.32:0/1062756914 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v65) v1 ==== 100+0+11 (secure 0 0 0) 0x7f27e00bc0a0 con 0x7f27f00cddc0 2023-12-20T04:43:05.843 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:43:05.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 -- 172.21.15.32:0/1062756914 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f27d0065b80 msgr2=0x7f27d0068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f27d0065b80 0x7f27d0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 -- 172.21.15.32:0/1062756914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00cddc0 msgr2=0x7f27f015e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00cddc0 0x7f27f015e3e0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7f27e00190e0 tx=0x7f27e0019270 comp rx=0 tx=0).stop 2023-12-20T04:43:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 -- 172.21.15.32:0/1062756914 shutdown_connections 2023-12-20T04:43:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f27d0065b80 0x7f27d0068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f01606b0 0x7f27f01499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27f015cb30 0x7f27f015a080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 --2- 172.21.15.32:0/1062756914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00cddc0 0x7f27f015e3e0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 -- 172.21.15.32:0/1062756914 >> 172.21.15.32:0/1062756914 conn(0x7f27f006af60 msgr2=0x7f27f00b5ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 -- 172.21.15.32:0/1062756914 shutdown_connections 2023-12-20T04:43:05.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:05.847+0000 7f27f5339640 1 -- 172.21.15.32:0/1062756914 wait complete. 2023-12-20T04:43:05.858 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:05.859+0000 7fa2d0d99180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:43:05.865 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T04:43:05.963 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:05.964+0000 7fd5b4c2d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:43:06.016 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:06.019+0000 7fa2d0d99180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:43:06.025 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:06.024+0000 7fd5b4c2d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:43:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f9728b6d640 1 Processor -- start 2023-12-20T04:43:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f9728b6d640 1 -- start start 2023-12-20T04:43:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f9728b6d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9724071af0 0x7f97240c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f9728b6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97240c9a40 con 0x7f9724160b30 2023-12-20T04:43:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f9728b6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9724161660 con 0x7f972415ce60 2023-12-20T04:43:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f9728b6d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97240c9180 con 0x7f9724071af0 2023-12-20T04:43:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f9721d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9724071af0 0x7f97240c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f972259c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f972415ce60 0x7f97240713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:42246/0 (socket says 172.21.15.32:42246) 2023-12-20T04:43:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f9721d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9724071af0 0x7f97240c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56946/0 (socket says 172.21.15.32:56946) 2023-12-20T04:43:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f972259c640 1 -- 172.21.15.32:0/2869765327 learned_addr learned my addr 172.21.15.32:0/2869765327 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f9721d9b640 1 -- 172.21.15.32:0/2869765327 >> v1:172.21.15.39:6789/0 conn(0x7f972415ce60 legacy=0x7f97240713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f9721d9b640 1 -- 172.21.15.32:0/2869765327 >> v1:172.21.15.32:6789/0 conn(0x7f9724160b30 legacy=0x7f9724160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f9721d9b640 1 -- 172.21.15.32:0/2869765327 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97240c93f0 con 0x7f9724071af0 2023-12-20T04:43:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.059+0000 7f972159a640 1 -- 172.21.15.32:0/2869765327 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2473405845 0 0) 0x7f9724161660 con 0x7f972415ce60 2023-12-20T04:43:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.063+0000 7f9721d9b640 1 --2- 172.21.15.32:0/2869765327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9724071af0 0x7f97240c9ed0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f97180097c0 tx=0x7f971801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddfd72aadd09a52 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.063+0000 7f972159a640 1 -- 172.21.15.32:0/2869765327 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718004230 con 0x7f9724071af0 2023-12-20T04:43:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.063+0000 7f972159a640 1 -- 172.21.15.32:0/2869765327 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97180a7050 con 0x7f9724071af0 2023-12-20T04:43:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.063+0000 7f972159a640 1 -- 172.21.15.32:0/2869765327 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9718003620 con 0x7f9724071af0 2023-12-20T04:43:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.063+0000 7f9728b6d640 1 -- 172.21.15.32:0/2869765327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9724071af0 msgr2=0x7f97240c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.063+0000 7f9728b6d640 1 --2- 172.21.15.32:0/2869765327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9724071af0 0x7f97240c9ed0 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7f97180097c0 tx=0x7f971801c480 comp rx=0 tx=0).stop 2023-12-20T04:43:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.063+0000 7f9728b6d640 1 -- 172.21.15.32:0/2869765327 shutdown_connections 2023-12-20T04:43:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.063+0000 7f9728b6d640 1 --2- 172.21.15.32:0/2869765327 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9724071af0 0x7f97240c9ed0 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.063+0000 7f9728b6d640 1 -- 172.21.15.32:0/2869765327 >> 172.21.15.32:0/2869765327 conn(0x7f972406a320 msgr2=0x7f97240ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.063+0000 7f9728b6d640 1 -- 172.21.15.32:0/2869765327 shutdown_connections 2023-12-20T04:43:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.063+0000 7f9728b6d640 1 -- 172.21.15.32:0/2869765327 wait complete. 2023-12-20T04:43:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9728b6d640 1 Processor -- start 2023-12-20T04:43:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9728b6d640 1 -- start start 2023-12-20T04:43:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9728b6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9724071af0 0x7f97241507c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9728b6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f972415ce60 0x7f97240c3090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9728b6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9724160b30 0x7f97240c35d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9728b6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97241f97c0 con 0x7f9724160b30 2023-12-20T04:43:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9728b6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97240c3c00 con 0x7f9724071af0 2023-12-20T04:43:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9728b6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97241f8a00 con 0x7f972415ce60 2023-12-20T04:43:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f972259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9724071af0 0x7f97241507c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9722d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9724160b30 0x7f97240c35d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9721d9b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f972415ce60 0x7f97240c3090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f972259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9724071af0 0x7f97241507c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:47892/0 (socket says 172.21.15.32:47892) 2023-12-20T04:43:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f972259c640 1 -- 172.21.15.32:0/2668748018 learned_addr learned my addr 172.21.15.32:0/2668748018 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f972259c640 1 -- 172.21.15.32:0/2668748018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f972415ce60 msgr2=0x7f97240c3090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f972259c640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f972415ce60 0x7f97240c3090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f972259c640 1 -- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9724160b30 msgr2=0x7f97240c35d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f972259c640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9724160b30 0x7f97240c35d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f972259c640 1 -- 172.21.15.32:0/2668748018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9718009420 con 0x7f9724071af0 2023-12-20T04:43:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9722d9d640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9724160b30 0x7f97240c35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f972259c640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9724071af0 0x7f97241507c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f971400c450 tx=0x7f971400c910 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9712ffd640 1 -- 172.21.15.32:0/2668748018 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97140074e0 con 0x7f9724071af0 2023-12-20T04:43:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9712ffd640 1 -- 172.21.15.32:0/2668748018 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9714007640 con 0x7f9724071af0 2023-12-20T04:43:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9712ffd640 1 -- 172.21.15.32:0/2668748018 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9714004a40 con 0x7f9724071af0 2023-12-20T04:43:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9728b6d640 1 -- 172.21.15.32:0/2668748018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97241f8ce0 con 0x7f9724071af0 2023-12-20T04:43:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.067+0000 7f9728b6d640 1 -- 172.21.15.32:0/2668748018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97241f92a0 con 0x7f9724071af0 2023-12-20T04:43:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.071+0000 7f9728b6d640 1 -- 172.21.15.32:0/2668748018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97241f9490 con 0x7f9724071af0 2023-12-20T04:43:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.071+0000 7f9712ffd640 1 -- 172.21.15.32:0/2668748018 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f9714004ba0 con 0x7f9724071af0 2023-12-20T04:43:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.071+0000 7f9712ffd640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9704065b80 0x7f9704068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.071+0000 7f9712ffd640 1 -- 172.21.15.32:0/2668748018 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(65..65 src has 1..65) v4 ==== 6199+0+0 (secure 0 0 0) 0x7f97140825e0 con 0x7f9724071af0 2023-12-20T04:43:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.071+0000 7f9721d9b640 1 -- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9704065b80 msgr2=0x7f9704068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:06.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.071+0000 7f9721d9b640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9704065b80 0x7f9704068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:06.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.075+0000 7f9712ffd640 1 -- 172.21.15.32:0/2668748018 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f971404fd80 con 0x7f9724071af0 2023-12-20T04:43:06.082 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:06.083+0000 7fa2d0d99180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:43:06.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.275+0000 7f9721d9b640 1 -- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9704065b80 msgr2=0x7f9704068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:06.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.275+0000 7f9721d9b640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9704065b80 0x7f9704068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:43:06.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.279+0000 7f9728b6d640 1 -- 172.21.15.32:0/2668748018 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f972404e000 con 0x7f9724071af0 2023-12-20T04:43:06.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.279+0000 7f9712ffd640 1 -- 172.21.15.32:0/2668748018 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v66) v1 ==== 80+0+457 (secure 0 0 0) 0x7f97140533a0 con 0x7f9724071af0 2023-12-20T04:43:06.279 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2279832503 2023-12-21T04:41:11.975357+0000 2023-12-20T04:43:06.279 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1700967844 2023-12-21T04:41:11.975357+0000 2023-12-20T04:43:06.279 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/384810653 2023-12-21T04:41:11.975357+0000 2023-12-20T04:43:06.279 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/38009 2023-12-21T04:41:11.975357+0000 2023-12-20T04:43:06.279 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/935462264 2023-12-21T04:41:11.975357+0000 2023-12-20T04:43:06.280 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/936743272 2023-12-21T04:41:11.975357+0000 2023-12-20T04:43:06.280 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/38009 2023-12-21T04:41:11.975357+0000 2023-12-20T04:43:06.280 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3773919514 2023-12-21T04:41:11.975357+0000 2023-12-20T04:43:06.280 INFO:teuthology.orchestra.run.smithi032.stderr:listed 8 entries 2023-12-20T04:43:06.284 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:43:06.284 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:43:06.284 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:43:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.283+0000 7f9728b6d640 1 -- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9704065b80 msgr2=0x7f9704068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.283+0000 7f9728b6d640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9704065b80 0x7f9704068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.283+0000 7f9728b6d640 1 -- 172.21.15.32:0/2668748018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9724071af0 msgr2=0x7f97241507c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.283+0000 7f9728b6d640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9724071af0 0x7f97241507c0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f971400c450 tx=0x7f971400c910 comp rx=0 tx=0).stop 2023-12-20T04:43:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.287+0000 7f9728b6d640 1 -- 172.21.15.32:0/2668748018 shutdown_connections 2023-12-20T04:43:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.287+0000 7f9728b6d640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9704065b80 0x7f9704068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.287+0000 7f9728b6d640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9724160b30 0x7f97240c35d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.287+0000 7f9728b6d640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f972415ce60 0x7f97240c3090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.287+0000 7f9728b6d640 1 --2- 172.21.15.32:0/2668748018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9724071af0 0x7f97241507c0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.287+0000 7f9728b6d640 1 -- 172.21.15.32:0/2668748018 >> 172.21.15.32:0/2668748018 conn(0x7f972406a320 msgr2=0x7f97240713e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.287+0000 7f9728b6d640 1 -- 172.21.15.32:0/2668748018 shutdown_connections 2023-12-20T04:43:06.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.287+0000 7f9728b6d640 1 -- 172.21.15.32:0/2668748018 wait complete. 2023-12-20T04:43:06.291 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:06.292+0000 7fd5b4c2d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:43:06.315 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T04:43:06.315 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:06.315 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T04:43:06.343 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:43:06.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:06.344 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T04:43:06.344 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:43:06.345 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:43:06.345 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:43:06.353 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:06.355+0000 7fa2d0d99180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:43:06.354 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:43:06.354 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:06.363 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:43:06.363 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:43:06.363 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:06.404 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:43:06.427 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:43:06.428 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:06.434 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:06.432+0000 7fd5b4c2d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:06.437 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:43:06.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:06.497 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:06.499+0000 7fa2d0d99180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:06.565 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:06.564+0000 7fd5b4c2d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:06.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f626a1bc640 1 Processor -- start 2023-12-20T04:43:06.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f626a1bc640 1 -- start start 2023-12-20T04:43:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f626a1bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f62641599e0 0x7f62640aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f626a1bc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62640c90e0 0x7f62640b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f626a1bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62640b0b30 0x7f626415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f626a1bc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62640c98c0 con 0x7f62640c90e0 2023-12-20T04:43:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f626a1bc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62640c9ba0 con 0x7f62641599e0 2023-12-20T04:43:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f626a1bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62640c9e80 con 0x7f62640b0b30 2023-12-20T04:43:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f6262ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62640c90e0 0x7f62640b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f6262ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62640c90e0 0x7f62640b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56968/0 (socket says 172.21.15.32:56968) 2023-12-20T04:43:06.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f6262ffd640 1 -- 172.21.15.32:0/82358358 learned_addr learned my addr 172.21.15.32:0/82358358 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:06.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f6263fff640 1 --2- 172.21.15.32:0/82358358 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62640b0b30 0x7f626415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f62637fe640 1 --2- 172.21.15.32:0/82358358 >> v2:172.21.15.39:3300/0 conn(0x7f62641599e0 0x7f62640aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f6262ffd640 1 -- 172.21.15.32:0/82358358 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62640b0b30 msgr2=0x7f626415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:06.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f6262ffd640 1 --2- 172.21.15.32:0/82358358 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62640b0b30 0x7f626415d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f6262ffd640 1 -- 172.21.15.32:0/82358358 >> v2:172.21.15.39:3300/0 conn(0x7f62641599e0 msgr2=0x7f62640aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:06.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f6262ffd640 1 --2- 172.21.15.32:0/82358358 >> v2:172.21.15.39:3300/0 conn(0x7f62641599e0 0x7f62640aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f6262ffd640 1 -- 172.21.15.32:0/82358358 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62640ca160 con 0x7f62640c90e0 2023-12-20T04:43:06.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f62637fe640 1 --2- 172.21.15.32:0/82358358 >> v2:172.21.15.39:3300/0 conn(0x7f62641599e0 0x7f62640aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:06.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.595+0000 7f6263fff640 1 --2- 172.21.15.32:0/82358358 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62640b0b30 0x7f626415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:06.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f6262ffd640 1 --2- 172.21.15.32:0/82358358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62640c90e0 0x7f62640b05f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f626406e3c0 tx=0x7f625801e8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=329835b2837a944b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f62627fc640 1 -- 172.21.15.32:0/82358358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62580a7990 con 0x7f62640c90e0 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f62627fc640 1 -- 172.21.15.32:0/82358358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62580a7af0 con 0x7f62640c90e0 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f62627fc640 1 -- 172.21.15.32:0/82358358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62580a7e80 con 0x7f62640c90e0 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f626a1bc640 1 -- 172.21.15.32:0/82358358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62640c90e0 msgr2=0x7f62640b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f626a1bc640 1 --2- 172.21.15.32:0/82358358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62640c90e0 0x7f62640b05f0 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7f626406e3c0 tx=0x7f625801e8e0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f626a1bc640 1 -- 172.21.15.32:0/82358358 shutdown_connections 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f626a1bc640 1 --2- 172.21.15.32:0/82358358 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62640b0b30 0x7f626415d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f626a1bc640 1 --2- 172.21.15.32:0/82358358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62640c90e0 0x7f62640b05f0 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f626a1bc640 1 --2- 172.21.15.32:0/82358358 >> v2:172.21.15.39:3300/0 conn(0x7f62641599e0 0x7f62640aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f626a1bc640 1 -- 172.21.15.32:0/82358358 >> 172.21.15.32:0/82358358 conn(0x7f626406a320 msgr2=0x7f62640ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f626a1bc640 1 -- 172.21.15.32:0/82358358 shutdown_connections 2023-12-20T04:43:06.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.599+0000 7f626a1bc640 1 -- 172.21.15.32:0/82358358 wait complete. 2023-12-20T04:43:06.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f626a1bc640 1 Processor -- start 2023-12-20T04:43:06.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f626a1bc640 1 -- start start 2023-12-20T04:43:06.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f626a1bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62640b0b30 0x7f62640b3b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f626a1bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62640c90e0 0x7f62640c5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f626a1bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62641599e0 0x7f626414ef80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f626a1bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62640c5f50 con 0x7f62640c90e0 2023-12-20T04:43:06.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f626a1bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f626415a110 con 0x7f62640b0b30 2023-12-20T04:43:06.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f626a1bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f626415a3f0 con 0x7f62641599e0 2023-12-20T04:43:06.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f6262ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62640c90e0 0x7f62640c5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f6262ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62640c90e0 0x7f62640c5a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56972/0 (socket says 172.21.15.32:56972) 2023-12-20T04:43:06.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f6262ffd640 1 -- 172.21.15.32:0/259487433 learned_addr learned my addr 172.21.15.32:0/259487433 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:06.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f62637fe640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62640b0b30 0x7f62640b3b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.603+0000 7f6263fff640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62641599e0 0x7f626414ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f6262ffd640 1 -- 172.21.15.32:0/259487433 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62641599e0 msgr2=0x7f626414ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f6262ffd640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62641599e0 0x7f626414ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f6262ffd640 1 -- 172.21.15.32:0/259487433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62640b0b30 msgr2=0x7f62640b3b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f6262ffd640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62640b0b30 0x7f62640b3b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f6262ffd640 1 -- 172.21.15.32:0/259487433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f625800b770 con 0x7f62640c90e0 2023-12-20T04:43:06.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f6262ffd640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62640c90e0 0x7f62640c5a10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f625801cf90 tx=0x7f625801b680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:06.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f624bfff640 1 -- 172.21.15.32:0/259487433 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62580ba660 con 0x7f62640c90e0 2023-12-20T04:43:06.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f624bfff640 1 -- 172.21.15.32:0/259487433 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62580ba7c0 con 0x7f62640c90e0 2023-12-20T04:43:06.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f624bfff640 1 -- 172.21.15.32:0/259487433 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62580bab70 con 0x7f62640c90e0 2023-12-20T04:43:06.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f626a1bc640 1 -- 172.21.15.32:0/259487433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f626415a6d0 con 0x7f62640c90e0 2023-12-20T04:43:06.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f626a1bc640 1 -- 172.21.15.32:0/259487433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f626415ac90 con 0x7f62640c90e0 2023-12-20T04:43:06.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f626a1bc640 1 -- 172.21.15.32:0/259487433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f626415ae80 con 0x7f62640c90e0 2023-12-20T04:43:06.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f624bfff640 1 -- 172.21.15.32:0/259487433 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f62580c2360 con 0x7f62640c90e0 2023-12-20T04:43:06.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f624bfff640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6250065b80 0x7f6250068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f62637fe640 1 -- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6250065b80 msgr2=0x7f6250068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:06.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f624bfff640 1 -- 172.21.15.32:0/259487433 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f625811e3e0 con 0x7f62640c90e0 2023-12-20T04:43:06.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.607+0000 7f62637fe640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6250065b80 0x7f6250068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:06.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.615+0000 7f624bfff640 1 -- 172.21.15.32:0/259487433 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f62580ebac0 con 0x7f62640c90e0 2023-12-20T04:43:06.638 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:06.639+0000 7fa2d0d99180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:06.639 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:06.640+0000 7fd5b4c2d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:43:06.714 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:06.715+0000 7fa2d0d99180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:43:06.789 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:06.788+0000 7fd5b4c2d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.811+0000 7f62637fe640 1 -- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6250065b80 msgr2=0x7f6250068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:06.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.811+0000 7f62637fe640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6250065b80 0x7f6250068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:43:06.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.811+0000 7f626a1bc640 1 -- 172.21.15.32:0/259487433 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f626404e000 con 0x7f62640c90e0 2023-12-20T04:43:06.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f624bfff640 1 -- 172.21.15.32:0/259487433 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4700 (secure 0 0 0) 0x7f62580ef0e0 con 0x7f62640c90e0 2023-12-20T04:43:06.812 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:06.812 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/536564441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":536564441},{"type":"v1","addr":"172.21.15.32:6835","nonce":536564441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2898125535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2898125535},{"type":"v1","addr":"172.21.15.39:6835","nonce":2898125535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/319656564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":319656564},{"type":"v1","addr":"172.21.15.32:6837","nonce":319656564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:40:59.920384+0000","modified":"2023-12-20T04:41:02.943287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5895},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5895":{"gid":5895,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/192084837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":192084837},{"type":"v1","addr":"172.21.15.39:6834","nonce":192084837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-20T04:43:06.812 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2023-12-20T04:43:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 -- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6250065b80 msgr2=0x7f6250068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6250065b80 0x7f6250068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 -- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62640c90e0 msgr2=0x7f62640c5a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62640c90e0 0x7f62640c5a10 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f625801cf90 tx=0x7f625801b680 comp rx=0 tx=0).stop 2023-12-20T04:43:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 -- 172.21.15.32:0/259487433 shutdown_connections 2023-12-20T04:43:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6250065b80 0x7f6250068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62641599e0 0x7f626414ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62640c90e0 0x7f62640c5a10 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 --2- 172.21.15.32:0/259487433 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62640b0b30 0x7f62640b3b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 -- 172.21.15.32:0/259487433 >> 172.21.15.32:0/259487433 conn(0x7f626406a320 msgr2=0x7f6264156810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:06.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 -- 172.21.15.32:0/259487433 shutdown_connections 2023-12-20T04:43:06.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.815+0000 7f626a1bc640 1 -- 172.21.15.32:0/259487433 wait complete. 2023-12-20T04:43:06.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:06.864 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:06.867+0000 7fa2d0d99180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:06.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f681bb25640 1 Processor -- start 2023-12-20T04:43:06.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f681bb25640 1 -- start start 2023-12-20T04:43:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f681bb25640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6814164ef0 0x7f68140c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f681bb25640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68140ae3f0 con 0x7f68140ae010 2023-12-20T04:43:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f681bb25640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6814161800 con 0x7f681406cf10 2023-12-20T04:43:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f681bb25640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6814165380 con 0x7f6814164ef0 2023-12-20T04:43:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f68198c1640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f68140ae010 0x7f6814071470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49516/0 (socket says 172.21.15.32:49516) 2023-12-20T04:43:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f68198c1640 1 -- 172.21.15.32:0/2787516303 learned_addr learned my addr 172.21.15.32:0/2787516303 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:06.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f681a0c2640 1 --2- 172.21.15.32:0/2787516303 >> v2:172.21.15.39:3300/0 conn(0x7f6814164ef0 0x7f68140c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f68188bf640 1 -- 172.21.15.32:0/2787516303 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3501164313 0 0) 0x7f68140ae3f0 con 0x7f68140ae010 2023-12-20T04:43:06.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f68188bf640 1 -- 172.21.15.32:0/2787516303 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6800003530 con 0x7f68140ae010 2023-12-20T04:43:06.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f68188bf640 1 -- 172.21.15.32:0/2787516303 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1151180507 0 0) 0x7f6814161800 con 0x7f681406cf10 2023-12-20T04:43:06.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f68188bf640 1 -- 172.21.15.32:0/2787516303 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f68140ae3f0 con 0x7f681406cf10 2023-12-20T04:43:06.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f681a0c2640 1 -- 172.21.15.32:0/2787516303 >> v1:172.21.15.39:6789/0 conn(0x7f681406cf10 legacy=0x7f68140c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f681a0c2640 1 -- 172.21.15.32:0/2787516303 >> v1:172.21.15.32:6789/0 conn(0x7f68140ae010 legacy=0x7f6814071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.983+0000 7f681a0c2640 1 -- 172.21.15.32:0/2787516303 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6814165660 con 0x7f6814164ef0 2023-12-20T04:43:06.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.987+0000 7f681a0c2640 1 --2- 172.21.15.32:0/2787516303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6814164ef0 0x7f68140c9bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f680c009d90 tx=0x7f680c01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6c192c03a0ce3aa3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f68188bf640 1 -- 172.21.15.32:0/2787516303 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c0a5950 con 0x7f6814164ef0 2023-12-20T04:43:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f68188bf640 1 -- 172.21.15.32:0/2787516303 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f680c0a5ab0 con 0x7f6814164ef0 2023-12-20T04:43:06.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f68188bf640 1 -- 172.21.15.32:0/2787516303 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f680c0a5e40 con 0x7f6814164ef0 2023-12-20T04:43:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 -- 172.21.15.32:0/2787516303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6814164ef0 msgr2=0x7f68140c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 --2- 172.21.15.32:0/2787516303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6814164ef0 0x7f68140c9bb0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f680c009d90 tx=0x7f680c01c890 comp rx=0 tx=0).stop 2023-12-20T04:43:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 -- 172.21.15.32:0/2787516303 shutdown_connections 2023-12-20T04:43:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 --2- 172.21.15.32:0/2787516303 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6814164ef0 0x7f68140c9bb0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 -- 172.21.15.32:0/2787516303 >> 172.21.15.32:0/2787516303 conn(0x7f681406a320 msgr2=0x7f68140ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 -- 172.21.15.32:0/2787516303 shutdown_connections 2023-12-20T04:43:06.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 -- 172.21.15.32:0/2787516303 wait complete. 2023-12-20T04:43:06.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 Processor -- start 2023-12-20T04:43:06.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 -- start start 2023-12-20T04:43:06.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68140ae010 0x7f6814155410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6814155950 0x7f68140c4810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68140c4d50 0x7f68140cd340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6814152d90 con 0x7f6814155950 2023-12-20T04:43:06.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6814153070 con 0x7f68140ae010 2023-12-20T04:43:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.991+0000 7f681bb25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6814153350 con 0x7f68140c4d50 2023-12-20T04:43:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f68190c0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6814155950 0x7f68140c4810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f68198c1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68140ae010 0x7f6814155410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f68190c0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6814155950 0x7f68140c4810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56978/0 (socket says 172.21.15.32:56978) 2023-12-20T04:43:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f68190c0640 1 -- 172.21.15.32:0/1971697157 learned_addr learned my addr 172.21.15.32:0/1971697157 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f681a0c2640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68140c4d50 0x7f68140cd340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f68190c0640 1 -- 172.21.15.32:0/1971697157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68140c4d50 msgr2=0x7f68140cd340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f68190c0640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68140c4d50 0x7f68140cd340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f68190c0640 1 -- 172.21.15.32:0/1971697157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68140ae010 msgr2=0x7f6814155410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f68190c0640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68140ae010 0x7f6814155410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:06.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f68190c0640 1 -- 172.21.15.32:0/1971697157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f680c009a70 con 0x7f6814155950 2023-12-20T04:43:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f68190c0640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6814155950 0x7f68140c4810 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f680400c490 tx=0x7f680400c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f680a7fc640 1 -- 172.21.15.32:0/1971697157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68040052d0 con 0x7f6814155950 2023-12-20T04:43:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f680a7fc640 1 -- 172.21.15.32:0/1971697157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6804005430 con 0x7f6814155950 2023-12-20T04:43:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f680a7fc640 1 -- 172.21.15.32:0/1971697157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6804012870 con 0x7f6814155950 2023-12-20T04:43:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f681bb25640 1 -- 172.21.15.32:0/1971697157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68140b2070 con 0x7f6814155950 2023-12-20T04:43:06.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f681bb25640 1 -- 172.21.15.32:0/1971697157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f681414f3b0 con 0x7f6814155950 2023-12-20T04:43:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f681bb25640 1 -- 172.21.15.32:0/1971697157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f681404e000 con 0x7f6814155950 2023-12-20T04:43:06.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f680a7fc640 1 -- 172.21.15.32:0/1971697157 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f68040129d0 con 0x7f6814155950 2023-12-20T04:43:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f680a7fc640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f67f8065bb0 0x7f67f8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.995+0000 7f680a7fc640 1 -- 172.21.15.32:0/1971697157 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f6804082ff0 con 0x7f6814155950 2023-12-20T04:43:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.999+0000 7f68198c1640 1 -- 172.21.15.32:0/1971697157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f67f8065bb0 msgr2=0x7f67f8068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:06.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.999+0000 7f68198c1640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f67f8065bb0 0x7f67f8068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:06.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:06.999+0000 7f680a7fc640 1 -- 172.21.15.32:0/1971697157 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f68040506a0 con 0x7f6814155950 2023-12-20T04:43:07.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.167+0000 7f681bb25640 1 -- 172.21.15.32:0/1971697157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f6814152ed0 con 0x7f6814155950 2023-12-20T04:43:07.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.167+0000 7f680a7fc640 1 -- 172.21.15.32:0/1971697157 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v66) v1 ==== 74+0+16708 (secure 0 0 0) 0x7f6814152ed0 con 0x7f6814155950 2023-12-20T04:43:07.167 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:07.167 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":66,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:06.107534+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:40:57.540206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-20T04:40:58.913164+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":5}]},{"pgid":"7.2a","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2279832503":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/1700967844":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/384810653":"2023-12-21T04:41:11.975357+0000","172.21.15.32:6832/38009":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/935462264":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/936743272":"2023-12-21T04:41:11.975357+0000","172.21.15.32:6833/38009":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/3773919514":"2023-12-21T04:41:11.975357+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":7,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 -- 172.21.15.32:0/1971697157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f67f8065bb0 msgr2=0x7f67f8068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f67f8065bb0 0x7f67f8068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 -- 172.21.15.32:0/1971697157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6814155950 msgr2=0x7f68140c4810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6814155950 0x7f68140c4810 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f680400c490 tx=0x7f680400c950 comp rx=0 tx=0).stop 2023-12-20T04:43:07.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 -- 172.21.15.32:0/1971697157 shutdown_connections 2023-12-20T04:43:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f67f8065bb0 0x7f67f8068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68140c4d50 0x7f68140cd340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6814155950 0x7f68140c4810 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 --2- 172.21.15.32:0/1971697157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68140ae010 0x7f6814155410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 -- 172.21.15.32:0/1971697157 >> 172.21.15.32:0/1971697157 conn(0x7f681406a320 msgr2=0x7f6814073150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 -- 172.21.15.32:0/1971697157 shutdown_connections 2023-12-20T04:43:07.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.171+0000 7f681bb25640 1 -- 172.21.15.32:0/1971697157 wait complete. 2023-12-20T04:43:07.199 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 6, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:40:57.540206+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '60', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 7, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:40:58.913164+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '66', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 66, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:43:07.199 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:43:07.358 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:07.356+0000 7fd5b4c2d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:43:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.391+0000 7fd869c04640 1 Processor -- start 2023-12-20T04:43:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.391+0000 7fd869c04640 1 -- start start 2023-12-20T04:43:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd869c04640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8640cf3e0 0x7fd8640cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd869c04640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640d1580 0x7fd8640d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd869c04640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8640cfd80 con 0x7fd86415ccf0 2023-12-20T04:43:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd869c04640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8640ce2f0 con 0x7fd8640cf3e0 2023-12-20T04:43:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd869c04640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640d0b70 con 0x7fd8640d1580 2023-12-20T04:43:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd863fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8640cf3e0 0x7fd8640cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd863fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd8640cf3e0 0x7fd8640cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56990/0 (socket says 172.21.15.32:56990) 2023-12-20T04:43:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd863fff640 1 -- 172.21.15.32:0/4224158086 learned_addr learned my addr 172.21.15.32:0/4224158086 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd869403640 1 --2- 172.21.15.32:0/4224158086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640d1580 0x7fd8640d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd863fff640 1 -- 172.21.15.32:0/4224158086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640d1580 msgr2=0x7fd8640d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd863fff640 1 --2- 172.21.15.32:0/4224158086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640d1580 0x7fd8640d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd863fff640 1 -- 172.21.15.32:0/4224158086 >> v1:172.21.15.32:6789/0 conn(0x7fd86415ccf0 legacy=0x7fd8640cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.395+0000 7fd863fff640 1 -- 172.21.15.32:0/4224158086 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8640d10f0 con 0x7fd8640cf3e0 2023-12-20T04:43:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd863fff640 1 --2- 172.21.15.32:0/4224158086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8640cf3e0 0x7fd8640cf840 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd850009870 tx=0x7fd85001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1803ebc8f4ffe server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd8637fe640 1 -- 172.21.15.32:0/4224158086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500ab680 con 0x7fd8640cf3e0 2023-12-20T04:43:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd8637fe640 1 -- 172.21.15.32:0/4224158086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8500ab7e0 con 0x7fd8640cf3e0 2023-12-20T04:43:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd8637fe640 1 -- 172.21.15.32:0/4224158086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8500abb70 con 0x7fd8640cf3e0 2023-12-20T04:43:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd869c04640 1 -- 172.21.15.32:0/4224158086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8640cf3e0 msgr2=0x7fd8640cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd869c04640 1 --2- 172.21.15.32:0/4224158086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8640cf3e0 0x7fd8640cf840 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7fd850009870 tx=0x7fd85001c480 comp rx=0 tx=0).stop 2023-12-20T04:43:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd869c04640 1 -- 172.21.15.32:0/4224158086 shutdown_connections 2023-12-20T04:43:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd869c04640 1 --2- 172.21.15.32:0/4224158086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8640cf3e0 0x7fd8640cf840 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd869c04640 1 --2- 172.21.15.32:0/4224158086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640d1580 0x7fd8640d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd869c04640 1 -- 172.21.15.32:0/4224158086 >> 172.21.15.32:0/4224158086 conn(0x7fd8640658e0 msgr2=0x7fd864065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd869c04640 1 -- 172.21.15.32:0/4224158086 shutdown_connections 2023-12-20T04:43:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd869c04640 1 -- 172.21.15.32:0/4224158086 wait complete. 2023-12-20T04:43:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd869c04640 1 Processor -- start 2023-12-20T04:43:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.399+0000 7fd869c04640 1 -- start start 2023-12-20T04:43:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd869c04640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8640cf3e0 0x7fd8641558a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd869c04640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640d1580 0x7fd8640c5650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd869c04640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86415ccf0 0x7fd864150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd869c04640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640c9ae0 con 0x7fd8640cf3e0 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd869c04640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8641560f0 con 0x7fd86415ccf0 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd869c04640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640c9d80 con 0x7fd8640d1580 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd868c02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8640cf3e0 0x7fd8641558a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd868c02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8640cf3e0 0x7fd8641558a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57000/0 (socket says 172.21.15.32:57000) 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd868c02640 1 -- 172.21.15.32:0/2646718668 learned_addr learned my addr 172.21.15.32:0/2646718668 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd863fff640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640d1580 0x7fd8640c5650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd869403640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86415ccf0 0x7fd864150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd868c02640 1 -- 172.21.15.32:0/2646718668 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640d1580 msgr2=0x7fd8640c5650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd868c02640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640d1580 0x7fd8640c5650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd868c02640 1 -- 172.21.15.32:0/2646718668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86415ccf0 msgr2=0x7fd864150980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd868c02640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86415ccf0 0x7fd864150980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd868c02640 1 -- 172.21.15.32:0/2646718668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8500094d0 con 0x7fd8640cf3e0 2023-12-20T04:43:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd869403640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86415ccf0 0x7fd864150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd868c02640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8640cf3e0 0x7fd8641558a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd85800c500 tx=0x7fd85800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd8617fa640 1 -- 172.21.15.32:0/2646718668 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858005220 con 0x7fd8640cf3e0 2023-12-20T04:43:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd8617fa640 1 -- 172.21.15.32:0/2646718668 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd858005380 con 0x7fd8640cf3e0 2023-12-20T04:43:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd8617fa640 1 -- 172.21.15.32:0/2646718668 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd858007270 con 0x7fd8640cf3e0 2023-12-20T04:43:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd869c04640 1 -- 172.21.15.32:0/2646718668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8640ca060 con 0x7fd8640cf3e0 2023-12-20T04:43:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd869c04640 1 -- 172.21.15.32:0/2646718668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd864155de0 con 0x7fd8640cf3e0 2023-12-20T04:43:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd869c04640 1 -- 172.21.15.32:0/2646718668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8640ca580 con 0x7fd8640cf3e0 2023-12-20T04:43:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.403+0000 7fd8617fa640 1 -- 172.21.15.32:0/2646718668 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fd8580073d0 con 0x7fd8640cf3e0 2023-12-20T04:43:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.407+0000 7fd8617fa640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd848065bb0 0x7fd848068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.407+0000 7fd8617fa640 1 -- 172.21.15.32:0/2646718668 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fd858082750 con 0x7fd8640cf3e0 2023-12-20T04:43:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.407+0000 7fd863fff640 1 -- 172.21.15.32:0/2646718668 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd848065bb0 msgr2=0x7fd848068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.407+0000 7fd863fff640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd848065bb0 0x7fd848068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.411+0000 7fd8617fa640 1 -- 172.21.15.32:0/2646718668 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd8640ca580 con 0x7fd8640cf3e0 2023-12-20T04:43:07.446 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:07.448+0000 7fd5b4c2d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:43:07.464 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:07.467+0000 7fa2d0d99180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:43:07.556 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:07.559+0000 7fa2d0d99180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:43:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.571+0000 7fd869c04640 1 -- 172.21.15.32:0/2646718668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd8640ca580 con 0x7fd8640cf3e0 2023-12-20T04:43:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.571+0000 7fd8617fa640 1 -- 172.21.15.32:0/2646718668 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v66) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd8640ca580 con 0x7fd8640cf3e0 2023-12-20T04:43:07.569 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:43:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.571+0000 7fd869c04640 1 -- 172.21.15.32:0/2646718668 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd848065bb0 msgr2=0x7fd848068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.571+0000 7fd869c04640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd848065bb0 0x7fd848068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.571+0000 7fd869c04640 1 -- 172.21.15.32:0/2646718668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8640cf3e0 msgr2=0x7fd8641558a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.571+0000 7fd869c04640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8640cf3e0 0x7fd8641558a0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fd85800c500 tx=0x7fd85800c9c0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.575+0000 7fd869c04640 1 -- 172.21.15.32:0/2646718668 shutdown_connections 2023-12-20T04:43:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.575+0000 7fd869c04640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd848065bb0 0x7fd848068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.575+0000 7fd869c04640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd86415ccf0 0x7fd864150980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.575+0000 7fd869c04640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640d1580 0x7fd8640c5650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.575+0000 7fd869c04640 1 --2- 172.21.15.32:0/2646718668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8640cf3e0 0x7fd8641558a0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.575+0000 7fd869c04640 1 -- 172.21.15.32:0/2646718668 >> 172.21.15.32:0/2646718668 conn(0x7fd8640658e0 msgr2=0x7fd86415d710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.575+0000 7fd869c04640 1 -- 172.21.15.32:0/2646718668 shutdown_connections 2023-12-20T04:43:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.575+0000 7fd869c04640 1 -- 172.21.15.32:0/2646718668 wait complete. 2023-12-20T04:43:07.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:43:07.604 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:07.604+0000 7fd5b4c2d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:43:07.692 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:07.692+0000 7fd5b4c2d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:43:07.719 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:07.723+0000 7fa2d0d99180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:43:07.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.739+0000 7f722dab6640 1 Processor -- start 2023-12-20T04:43:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.739+0000 7f722dab6640 1 -- start start 2023-12-20T04:43:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f722dab6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f722815ce60 0x7f72280d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f722dab6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72280d3130 0x7f72280d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f722dab6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72280d1300 0x7f72280b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f722dab6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72280d35a0 con 0x7f72280d1300 2023-12-20T04:43:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f722dab6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72280d2650 con 0x7f722815ce60 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f722dab6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72280d1a40 con 0x7f72280d3130 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f7227fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72280d1300 0x7f72280b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f7227fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f72280d1300 0x7f72280b46e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57004/0 (socket says 172.21.15.32:57004) 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f7227fff640 1 -- 172.21.15.32:0/955881449 learned_addr learned my addr 172.21.15.32:0/955881449 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f72277fe640 1 --2- 172.21.15.32:0/955881449 >> v2:172.21.15.39:3300/0 conn(0x7f722815ce60 0x7f72280d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f7226ffd640 1 --2- 172.21.15.32:0/955881449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72280d3130 0x7f72280d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f7227fff640 1 -- 172.21.15.32:0/955881449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72280d3130 msgr2=0x7f72280d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f7227fff640 1 --2- 172.21.15.32:0/955881449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72280d3130 0x7f72280d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f7227fff640 1 -- 172.21.15.32:0/955881449 >> v2:172.21.15.39:3300/0 conn(0x7f722815ce60 msgr2=0x7f72280d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f7227fff640 1 --2- 172.21.15.32:0/955881449 >> v2:172.21.15.39:3300/0 conn(0x7f722815ce60 0x7f72280d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f7227fff640 1 -- 172.21.15.32:0/955881449 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72280d1d20 con 0x7f72280d1300 2023-12-20T04:43:07.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.743+0000 7f7226ffd640 1 --2- 172.21.15.32:0/955881449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72280d3130 0x7f72280d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f7227fff640 1 --2- 172.21.15.32:0/955881449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72280d1300 0x7f72280b46e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f72280b5820 tx=0x7f722001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae6e196ec2d9a58d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f72267fc640 1 -- 172.21.15.32:0/955881449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72200af400 con 0x7f72280d1300 2023-12-20T04:43:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f72267fc640 1 -- 172.21.15.32:0/955881449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72200ab050 con 0x7f72280d1300 2023-12-20T04:43:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f72267fc640 1 -- 172.21.15.32:0/955881449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72200af780 con 0x7f72280d1300 2023-12-20T04:43:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f722dab6640 1 -- 172.21.15.32:0/955881449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72280d1300 msgr2=0x7f72280b46e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:07.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f722dab6640 1 --2- 172.21.15.32:0/955881449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72280d1300 0x7f72280b46e0 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f72280b5820 tx=0x7f722001f1c0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f722dab6640 1 -- 172.21.15.32:0/955881449 shutdown_connections 2023-12-20T04:43:07.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f722dab6640 1 --2- 172.21.15.32:0/955881449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72280d1300 0x7f72280b46e0 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f722dab6640 1 --2- 172.21.15.32:0/955881449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72280d3130 0x7f72280d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f722dab6640 1 --2- 172.21.15.32:0/955881449 >> v2:172.21.15.39:3300/0 conn(0x7f722815ce60 0x7f72280d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f722dab6640 1 -- 172.21.15.32:0/955881449 >> 172.21.15.32:0/955881449 conn(0x7f722806b5e0 msgr2=0x7f722806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:07.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f722dab6640 1 -- 172.21.15.32:0/955881449 shutdown_connections 2023-12-20T04:43:07.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f722dab6640 1 -- 172.21.15.32:0/955881449 wait complete. 2023-12-20T04:43:07.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f722dab6640 1 Processor -- start 2023-12-20T04:43:07.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.747+0000 7f722dab6640 1 -- start start 2023-12-20T04:43:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f722dab6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72280d1300 0x7f722814bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f722dab6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72280d3130 0x7f722815f910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f722dab6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722815ce60 0x7f72280c7dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f722dab6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72280cae70 con 0x7f72280d3130 2023-12-20T04:43:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f722dab6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72280cb110 con 0x7f722815ce60 2023-12-20T04:43:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f722dab6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72280cb3f0 con 0x7f72280d1300 2023-12-20T04:43:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f7226ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72280d3130 0x7f722815f910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f7226ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72280d3130 0x7f722815f910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57020/0 (socket says 172.21.15.32:57020) 2023-12-20T04:43:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f7226ffd640 1 -- 172.21.15.32:0/3689953063 learned_addr learned my addr 172.21.15.32:0/3689953063 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f72277fe640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72280d1300 0x7f722814bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f7227fff640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722815ce60 0x7f72280c7dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f7226ffd640 1 -- 172.21.15.32:0/3689953063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72280d1300 msgr2=0x7f722814bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f7226ffd640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72280d1300 0x7f722814bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f7226ffd640 1 -- 172.21.15.32:0/3689953063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722815ce60 msgr2=0x7f72280c7dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f7226ffd640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722815ce60 0x7f72280c7dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f7226ffd640 1 -- 172.21.15.32:0/3689953063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7220009b30 con 0x7f72280d3130 2023-12-20T04:43:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f7226ffd640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72280d3130 0x7f722815f910 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f721c00e810 tx=0x7f721c00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f720ffff640 1 -- 172.21.15.32:0/3689953063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c004030 con 0x7f72280d3130 2023-12-20T04:43:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f720ffff640 1 -- 172.21.15.32:0/3689953063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c004190 con 0x7f72280d3130 2023-12-20T04:43:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f722dab6640 1 -- 172.21.15.32:0/3689953063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72280cb6d0 con 0x7f72280d3130 2023-12-20T04:43:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f720ffff640 1 -- 172.21.15.32:0/3689953063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c004520 con 0x7f72280d3130 2023-12-20T04:43:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f722dab6640 1 -- 172.21.15.32:0/3689953063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f722814e830 con 0x7f72280d3130 2023-12-20T04:43:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f722dab6640 1 -- 172.21.15.32:0/3689953063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f722804e000 con 0x7f72280d3130 2023-12-20T04:43:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.751+0000 7f720ffff640 1 -- 172.21.15.32:0/3689953063 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f721c00c6d0 con 0x7f72280d3130 2023-12-20T04:43:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.755+0000 7f720ffff640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7214065b80 0x7f7214068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.755+0000 7f720ffff640 1 -- 172.21.15.32:0/3689953063 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f721c0830f0 con 0x7f72280d3130 2023-12-20T04:43:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.755+0000 7f72277fe640 1 -- 172.21.15.32:0/3689953063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7214065b80 msgr2=0x7f7214068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.755+0000 7f72277fe640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7214065b80 0x7f7214068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:07.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.759+0000 7f720ffff640 1 -- 172.21.15.32:0/3689953063 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f721c050750 con 0x7f72280d3130 2023-12-20T04:43:07.812 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:07.815+0000 7fa2d0d99180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:43:07.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.915+0000 7f722dab6640 1 -- 172.21.15.32:0/3689953063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f72281e6ea0 con 0x7f72280d3130 2023-12-20T04:43:07.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f720ffff640 1 -- 172.21.15.32:0/3689953063 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v66) v1 ==== 104+0+11 (secure 0 0 0) 0x7f721c053d70 con 0x7f72280d3130 2023-12-20T04:43:07.915 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:43:07.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 -- 172.21.15.32:0/3689953063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7214065b80 msgr2=0x7f7214068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:07.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7214065b80 0x7f7214068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 -- 172.21.15.32:0/3689953063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72280d3130 msgr2=0x7f722815f910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:07.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72280d3130 0x7f722815f910 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f721c00e810 tx=0x7f721c00ecd0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 -- 172.21.15.32:0/3689953063 shutdown_connections 2023-12-20T04:43:07.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7214065b80 0x7f7214068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f722815ce60 0x7f72280c7dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72280d3130 0x7f722815f910 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 --2- 172.21.15.32:0/3689953063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72280d1300 0x7f722814bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:07.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 -- 172.21.15.32:0/3689953063 >> 172.21.15.32:0/3689953063 conn(0x7f722806b5e0 msgr2=0x7f72280b7d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:07.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 -- 172.21.15.32:0/3689953063 shutdown_connections 2023-12-20T04:43:07.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:07.919+0000 7f722dab6640 1 -- 172.21.15.32:0/3689953063 wait complete. 2023-12-20T04:43:07.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:43:08.030 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:08.028+0000 7fd5b4c2d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:43:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f249cfa6640 1 Processor -- start 2023-12-20T04:43:08.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f249cfa6640 1 -- start start 2023-12-20T04:43:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f249cfa6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24981591e0 0x7f24980caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f249cfa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f249806cdf0 0x7f2498070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f249cfa6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2498070770 0x7f249815cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f249cfa6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2498070be0 con 0x7f24981591e0 2023-12-20T04:43:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f249cfa6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24980cc3b0 con 0x7f2498070770 2023-12-20T04:43:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f249cfa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24980c8e30 con 0x7f249806cdf0 2023-12-20T04:43:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f249659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24981591e0 0x7f24980caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f249659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24981591e0 0x7f24980caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57024/0 (socket says 172.21.15.32:57024) 2023-12-20T04:43:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f249659c640 1 -- 172.21.15.32:0/1568816309 learned_addr learned my addr 172.21.15.32:0/1568816309 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f248ffff640 1 --2- 172.21.15.32:0/1568816309 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f249806cdf0 0x7f2498070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.091+0000 7f2496d9d640 1 --2- 172.21.15.32:0/1568816309 >> v2:172.21.15.39:3300/0 conn(0x7f2498070770 0x7f249815cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.095+0000 7f249659c640 1 -- 172.21.15.32:0/1568816309 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f249806cdf0 msgr2=0x7f2498070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.095+0000 7f249659c640 1 --2- 172.21.15.32:0/1568816309 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f249806cdf0 0x7f2498070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.095+0000 7f249659c640 1 -- 172.21.15.32:0/1568816309 >> v2:172.21.15.39:3300/0 conn(0x7f2498070770 msgr2=0x7f249815cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.095+0000 7f249659c640 1 --2- 172.21.15.32:0/1568816309 >> v2:172.21.15.39:3300/0 conn(0x7f2498070770 0x7f249815cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.095+0000 7f249659c640 1 -- 172.21.15.32:0/1568816309 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f249806d280 con 0x7f24981591e0 2023-12-20T04:43:08.094 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:08.092+0000 7fd5b4c2d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:43:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.095+0000 7f249659c640 1 --2- 172.21.15.32:0/1568816309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24981591e0 0x7f24980caa30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f249806fff0 tx=0x7f248801e920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0d8c61f8c0f2cca server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:08.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.095+0000 7f2495d9b640 1 -- 172.21.15.32:0/1568816309 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2488004230 con 0x7f24981591e0 2023-12-20T04:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.095+0000 7f2495d9b640 1 -- 172.21.15.32:0/1568816309 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24880ab070 con 0x7f24981591e0 2023-12-20T04:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.095+0000 7f2495d9b640 1 -- 172.21.15.32:0/1568816309 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24880a7730 con 0x7f24981591e0 2023-12-20T04:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 -- 172.21.15.32:0/1568816309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24981591e0 msgr2=0x7f24980caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 --2- 172.21.15.32:0/1568816309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24981591e0 0x7f24980caa30 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f249806fff0 tx=0x7f248801e920 comp rx=0 tx=0).stop 2023-12-20T04:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 -- 172.21.15.32:0/1568816309 shutdown_connections 2023-12-20T04:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 --2- 172.21.15.32:0/1568816309 >> v2:172.21.15.39:3300/0 conn(0x7f2498070770 0x7f249815cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 --2- 172.21.15.32:0/1568816309 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f249806cdf0 0x7f2498070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 --2- 172.21.15.32:0/1568816309 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24981591e0 0x7f24980caa30 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 -- 172.21.15.32:0/1568816309 >> 172.21.15.32:0/1568816309 conn(0x7f249806a510 msgr2=0x7f24980ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 -- 172.21.15.32:0/1568816309 shutdown_connections 2023-12-20T04:43:08.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 -- 172.21.15.32:0/1568816309 wait complete. 2023-12-20T04:43:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 Processor -- start 2023-12-20T04:43:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 -- start start 2023-12-20T04:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498070770 0x7f24980c7cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24981591e0 0x7f2498147380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24981480e0 0x7f249814aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2498155420 con 0x7f2498070770 2023-12-20T04:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24981478c0 con 0x7f24981591e0 2023-12-20T04:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249cfa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24980c8290 con 0x7f24981480e0 2023-12-20T04:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498070770 0x7f24980c7cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498070770 0x7f24980c7cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57032/0 (socket says 172.21.15.32:57032) 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.099+0000 7f249659c640 1 -- 172.21.15.32:0/628542642 learned_addr learned my addr 172.21.15.32:0/628542642 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f248ffff640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24981591e0 0x7f2498147380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f2496d9d640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24981480e0 0x7f249814aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f249659c640 1 -- 172.21.15.32:0/628542642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24981480e0 msgr2=0x7f249814aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f249659c640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24981480e0 0x7f249814aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f249659c640 1 -- 172.21.15.32:0/628542642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24981591e0 msgr2=0x7f2498147380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f249659c640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24981591e0 0x7f2498147380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f249659c640 1 -- 172.21.15.32:0/628542642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248800b780 con 0x7f2498070770 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f249659c640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498070770 0x7f24980c7cb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2488024ed0 tx=0x7f248801b6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f248effd640 1 -- 172.21.15.32:0/628542642 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24880a7940 con 0x7f2498070770 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f248effd640 1 -- 172.21.15.32:0/628542642 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24880a7aa0 con 0x7f2498070770 2023-12-20T04:43:08.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f248effd640 1 -- 172.21.15.32:0/628542642 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24880a7e30 con 0x7f2498070770 2023-12-20T04:43:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f249cfa6640 1 -- 172.21.15.32:0/628542642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24980c8570 con 0x7f2498070770 2023-12-20T04:43:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f249cfa6640 1 -- 172.21.15.32:0/628542642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2498155670 con 0x7f2498070770 2023-12-20T04:43:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f249cfa6640 1 -- 172.21.15.32:0/628542642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f249804e000 con 0x7f2498070770 2023-12-20T04:43:08.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f248effd640 1 -- 172.21.15.32:0/628542642 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 26) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f2488003620 con 0x7f2498070770 2023-12-20T04:43:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f248effd640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2478065b80 0x7f2478068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f248ffff640 1 -- 172.21.15.32:0/628542642 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2478065b80 msgr2=0x7f2478068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.103+0000 7f248effd640 1 -- 172.21.15.32:0/628542642 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(66..66 src has 1..66) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f248811d330 con 0x7f2498070770 2023-12-20T04:43:08.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.107+0000 7f248ffff640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2478065b80 0x7f2478068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:08.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.107+0000 7f248effd640 1 -- 172.21.15.32:0/628542642 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f24880eabc0 con 0x7f2498070770 2023-12-20T04:43:08.162 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.163+0000 7fa2d0d99180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:43:08.226 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.226+0000 7fa2d0d99180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:43:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.238+0000 7f248effd640 1 -- 172.21.15.32:0/628542642 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 27) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f24880eaf90 con 0x7f2498070770 2023-12-20T04:43:08.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.238+0000 7f248effd640 1 -- 172.21.15.32:0/628542642 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2478065b80 msgr2=0x7f2478068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:08.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.238+0000 7f248effd640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2478065b80 0x7f2478068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.278+0000 7f249cfa6640 1 -- 172.21.15.32:0/628542642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f249806d840 con 0x7f2498070770 2023-12-20T04:43:08.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.278+0000 7f248effd640 1 -- 172.21.15.32:0/628542642 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v67) v1 ==== 100+0+11 (secure 0 0 0) 0x7f24881168a0 con 0x7f2498070770 2023-12-20T04:43:08.279 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:43:08.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.282+0000 7f249cfa6640 1 -- 172.21.15.32:0/628542642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498070770 msgr2=0x7f24980c7cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.282+0000 7f249cfa6640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498070770 0x7f24980c7cb0 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f2488024ed0 tx=0x7f248801b6c0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.282+0000 7f249cfa6640 1 -- 172.21.15.32:0/628542642 shutdown_connections 2023-12-20T04:43:08.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.282+0000 7f249cfa6640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2478065b80 0x7f2478068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.282+0000 7f249cfa6640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24981480e0 0x7f249814aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.282+0000 7f249cfa6640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24981591e0 0x7f2498147380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.282+0000 7f249cfa6640 1 --2- 172.21.15.32:0/628542642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2498070770 0x7f24980c7cb0 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.282+0000 7f249cfa6640 1 -- 172.21.15.32:0/628542642 >> 172.21.15.32:0/628542642 conn(0x7f249806a510 msgr2=0x7f24980b3540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:08.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.282+0000 7f249cfa6640 1 -- 172.21.15.32:0/628542642 shutdown_connections 2023-12-20T04:43:08.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.282+0000 7f249cfa6640 1 -- 172.21.15.32:0/628542642 wait complete. 2023-12-20T04:43:08.301 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.302+0000 7fa25ad1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:08.301 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.302+0000 7fa25ad1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:08.302 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.302+0000 7fa25ad1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:08.302 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.302+0000 7fa25ad1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:08.302 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.302+0000 7fa25ad1b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:08.302 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.302+0000 7fa259518640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:08.302 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.302+0000 7fa259518640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:08.302 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.302+0000 7fa259518640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:08.302 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.302+0000 7fa259518640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:08.302 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:08.302+0000 7fa259518640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:08.306 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.450+0000 7fa72ca20640 1 Processor -- start 2023-12-20T04:43:08.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.450+0000 7fa72ca20640 1 -- start start 2023-12-20T04:43:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.450+0000 7fa72ca20640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa7281599e0 0x7fa7280aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.450+0000 7fa72ca20640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7280c98c0 con 0x7fa7280b0d00 2023-12-20T04:43:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.450+0000 7fa72ca20640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7280c9ba0 con 0x7fa7280c90e0 2023-12-20T04:43:08.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.450+0000 7fa72ca20640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7280c9e80 con 0x7fa7281599e0 2023-12-20T04:43:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.454+0000 7fa726d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa7280b0d00 0x7fa7280701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49532/0 (socket says 172.21.15.32:49532) 2023-12-20T04:43:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.454+0000 7fa726d9d640 1 -- 172.21.15.32:0/2101619673 learned_addr learned my addr 172.21.15.32:0/2101619673 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.454+0000 7fa72659c640 1 --2- 172.21.15.32:0/2101619673 >> v2:172.21.15.39:3300/0 conn(0x7fa7281599e0 0x7fa7280aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.454+0000 7fa725d9b640 1 -- 172.21.15.32:0/2101619673 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3957702890 0 0) 0x7fa7280c98c0 con 0x7fa7280b0d00 2023-12-20T04:43:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.454+0000 7fa725d9b640 1 -- 172.21.15.32:0/2101619673 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa714003530 con 0x7fa7280b0d00 2023-12-20T04:43:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.454+0000 7fa72659c640 1 -- 172.21.15.32:0/2101619673 >> v1:172.21.15.39:6789/0 conn(0x7fa7280c90e0 legacy=0x7fa7280b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.454+0000 7fa72659c640 1 -- 172.21.15.32:0/2101619673 >> v1:172.21.15.32:6789/0 conn(0x7fa7280b0d00 legacy=0x7fa7280701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.454+0000 7fa72659c640 1 -- 172.21.15.32:0/2101619673 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7280ca160 con 0x7fa7281599e0 2023-12-20T04:43:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.454+0000 7fa72659c640 1 --2- 172.21.15.32:0/2101619673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7281599e0 0x7fa7280aece0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa7180097a0 tx=0x7fa71801c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=336bfde8a02616c0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa725d9b640 1 -- 172.21.15.32:0/2101619673 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180ab600 con 0x7fa7281599e0 2023-12-20T04:43:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa725d9b640 1 -- 172.21.15.32:0/2101619673 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7180a7050 con 0x7fa7281599e0 2023-12-20T04:43:08.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa725d9b640 1 -- 172.21.15.32:0/2101619673 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180ab980 con 0x7fa7281599e0 2023-12-20T04:43:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 -- 172.21.15.32:0/2101619673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7281599e0 msgr2=0x7fa7280aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 --2- 172.21.15.32:0/2101619673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7281599e0 0x7fa7280aece0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa7180097a0 tx=0x7fa71801c430 comp rx=0 tx=0).stop 2023-12-20T04:43:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 -- 172.21.15.32:0/2101619673 shutdown_connections 2023-12-20T04:43:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 --2- 172.21.15.32:0/2101619673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7281599e0 0x7fa7280aece0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 -- 172.21.15.32:0/2101619673 >> 172.21.15.32:0/2101619673 conn(0x7fa72806a320 msgr2=0x7fa7280ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 -- 172.21.15.32:0/2101619673 shutdown_connections 2023-12-20T04:43:08.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 -- 172.21.15.32:0/2101619673 wait complete. 2023-12-20T04:43:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 Processor -- start 2023-12-20T04:43:08.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 -- start start 2023-12-20T04:43:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7280b0d00 0x7fa7280c3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7280c90e0 0x7fa7281426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7281599e0 0x7fa72815bed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72815ba40 con 0x7fa7280b0d00 2023-12-20T04:43:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa72815ae30 con 0x7fa7280c90e0 2023-12-20T04:43:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.458+0000 7fa72ca20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa72815b110 con 0x7fa7281599e0 2023-12-20T04:43:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7280b0d00 0x7fa7280c3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa71dd9b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7280c90e0 0x7fa7281426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7280b0d00 0x7fa7280c3f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57146/0 (socket says 172.21.15.32:57146) 2023-12-20T04:43:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72659c640 1 -- 172.21.15.32:0/3992919685 learned_addr learned my addr 172.21.15.32:0/3992919685 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:08.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa726d9d640 1 --2- 172.21.15.32:0/3992919685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7281599e0 0x7fa72815bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72659c640 1 -- 172.21.15.32:0/3992919685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7281599e0 msgr2=0x7fa72815bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72659c640 1 --2- 172.21.15.32:0/3992919685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7281599e0 0x7fa72815bed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72659c640 1 -- 172.21.15.32:0/3992919685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7280c90e0 msgr2=0x7fa7281426a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72659c640 1 --2- 172.21.15.32:0/3992919685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7280c90e0 0x7fa7281426a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72659c640 1 -- 172.21.15.32:0/3992919685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7180093b0 con 0x7fa7280b0d00 2023-12-20T04:43:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa71dd9b640 1 --2- 172.21.15.32:0/3992919685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7280c90e0 0x7fa7281426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:08.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72659c640 1 --2- 172.21.15.32:0/3992919685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7280b0d00 0x7fa7280c3f90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa718019040 tx=0x7fa7180191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:08.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa71f7fe640 1 -- 172.21.15.32:0/3992919685 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7180059a0 con 0x7fa7280b0d00 2023-12-20T04:43:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa71f7fe640 1 -- 172.21.15.32:0/3992919685 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa718005b00 con 0x7fa7280b0d00 2023-12-20T04:43:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa71f7fe640 1 -- 172.21.15.32:0/3992919685 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa718006020 con 0x7fa7280b0d00 2023-12-20T04:43:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72ca20640 1 -- 172.21.15.32:0/3992919685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa72815b3f0 con 0x7fa7280b0d00 2023-12-20T04:43:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72ca20640 1 -- 172.21.15.32:0/3992919685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7280c44d0 con 0x7fa7280b0d00 2023-12-20T04:43:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa72ca20640 1 -- 172.21.15.32:0/3992919685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7280c46c0 con 0x7fa7280b0d00 2023-12-20T04:43:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.462+0000 7fa71f7fe640 1 -- 172.21.15.32:0/3992919685 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 27) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fa7180b2040 con 0x7fa7280b0d00 2023-12-20T04:43:08.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.466+0000 7fa71f7fe640 1 -- 172.21.15.32:0/3992919685 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fa71811c080 con 0x7fa7280b0d00 2023-12-20T04:43:08.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.470+0000 7fa71f7fe640 1 -- 172.21.15.32:0/3992919685 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa7180e9d30 con 0x7fa7280b0d00 2023-12-20T04:43:08.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.658+0000 7fa72ca20640 1 -- 172.21.15.32:0/3992919685 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa72804e000 con 0x7fa7280b0d00 2023-12-20T04:43:08.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.658+0000 7fa71f7fe640 1 -- 172.21.15.32:0/3992919685 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4700 (secure 0 0 0) 0x7fa7180b0070 con 0x7fa7280b0d00 2023-12-20T04:43:08.657 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:08.658 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/536564441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":536564441},{"type":"v1","addr":"172.21.15.32:6835","nonce":536564441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2898125535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2898125535},{"type":"v1","addr":"172.21.15.39:6835","nonce":2898125535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/319656564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":319656564},{"type":"v1","addr":"172.21.15.32:6837","nonce":319656564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:40:59.920384+0000","modified":"2023-12-20T04:41:02.943287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5895},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5895":{"gid":5895,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/192084837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":192084837},{"type":"v1","addr":"172.21.15.39:6834","nonce":192084837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-20T04:43:08.658 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2023-12-20T04:43:08.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.658+0000 7fa72ca20640 1 -- 172.21.15.32:0/3992919685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7280b0d00 msgr2=0x7fa7280c3f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.658+0000 7fa72ca20640 1 --2- 172.21.15.32:0/3992919685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7280b0d00 0x7fa7280c3f90 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7fa718019040 tx=0x7fa7180191d0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.662+0000 7fa72ca20640 1 -- 172.21.15.32:0/3992919685 shutdown_connections 2023-12-20T04:43:08.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.662+0000 7fa72ca20640 1 --2- 172.21.15.32:0/3992919685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7281599e0 0x7fa72815bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.662+0000 7fa72ca20640 1 --2- 172.21.15.32:0/3992919685 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7280c90e0 0x7fa7281426a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.662+0000 7fa72ca20640 1 --2- 172.21.15.32:0/3992919685 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7280b0d00 0x7fa7280c3f90 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.662+0000 7fa72ca20640 1 -- 172.21.15.32:0/3992919685 >> 172.21.15.32:0/3992919685 conn(0x7fa72806a320 msgr2=0x7fa72814bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:08.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.662+0000 7fa72ca20640 1 -- 172.21.15.32:0/3992919685 shutdown_connections 2023-12-20T04:43:08.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.662+0000 7fa72ca20640 1 -- 172.21.15.32:0/3992919685 wait complete. 2023-12-20T04:43:08.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:08.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.822+0000 7f27f7ee5640 1 Processor -- start 2023-12-20T04:43:08.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.822+0000 7f27f7ee5640 1 -- start start 2023-12-20T04:43:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f7ee5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27f01599e0 0x7f27f00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f7ee5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f27f00c90e0 0x7f27f00b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f7ee5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f00b0b30 0x7f27f015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f7ee5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f00c98c0 con 0x7f27f01599e0 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f7ee5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27f00c9ba0 con 0x7f27f00c90e0 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f7ee5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00c9e80 con 0x7f27f00b0b30 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f5c81640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27f01599e0 0x7f27f00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f5c81640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27f01599e0 0x7f27f00aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36812/0 (socket says 172.21.15.32:36812) 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f5c81640 1 -- 172.21.15.32:0/3444006915 learned_addr learned my addr 172.21.15.32:0/3444006915 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f5480640 1 --2- 172.21.15.32:0/3444006915 >> v2:172.21.15.39:3300/0 conn(0x7f27f00c90e0 0x7f27f00b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f6482640 1 --2- 172.21.15.32:0/3444006915 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f00b0b30 0x7f27f015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f5c81640 1 -- 172.21.15.32:0/3444006915 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f00b0b30 msgr2=0x7f27f015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f5c81640 1 --2- 172.21.15.32:0/3444006915 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f00b0b30 0x7f27f015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f5c81640 1 -- 172.21.15.32:0/3444006915 >> v2:172.21.15.39:3300/0 conn(0x7f27f00c90e0 msgr2=0x7f27f00b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f5c81640 1 --2- 172.21.15.32:0/3444006915 >> v2:172.21.15.39:3300/0 conn(0x7f27f00c90e0 0x7f27f00b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.826+0000 7f27f5c81640 1 -- 172.21.15.32:0/3444006915 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27f00ca160 con 0x7f27f01599e0 2023-12-20T04:43:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f5c81640 1 --2- 172.21.15.32:0/3444006915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f01599e0 0x7f27f00aece0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f27f006ab20 tx=0x7f27ec01c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78f4265be635eee8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f4c7f640 1 -- 172.21.15.32:0/3444006915 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0ab600 con 0x7f27f01599e0 2023-12-20T04:43:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f4c7f640 1 -- 172.21.15.32:0/3444006915 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27ec0a7050 con 0x7f27f01599e0 2023-12-20T04:43:08.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f4c7f640 1 -- 172.21.15.32:0/3444006915 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27ec0ab980 con 0x7f27f01599e0 2023-12-20T04:43:08.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3444006915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f01599e0 msgr2=0x7f27f00aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 --2- 172.21.15.32:0/3444006915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f01599e0 0x7f27f00aece0 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f27f006ab20 tx=0x7f27ec01c210 comp rx=0 tx=0).stop 2023-12-20T04:43:08.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3444006915 shutdown_connections 2023-12-20T04:43:08.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 --2- 172.21.15.32:0/3444006915 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f00b0b30 0x7f27f015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 --2- 172.21.15.32:0/3444006915 >> v2:172.21.15.39:3300/0 conn(0x7f27f00c90e0 0x7f27f00b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 --2- 172.21.15.32:0/3444006915 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f01599e0 0x7f27f00aece0 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3444006915 >> 172.21.15.32:0/3444006915 conn(0x7f27f006a320 msgr2=0x7f27f00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:08.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3444006915 shutdown_connections 2023-12-20T04:43:08.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3444006915 wait complete. 2023-12-20T04:43:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 Processor -- start 2023-12-20T04:43:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 -- start start 2023-12-20T04:43:08.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f00b0b30 0x7f27f014c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00c90e0 0x7f27f00b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27f01599e0 0x7f27f00b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f0157010 con 0x7f27f00c90e0 2023-12-20T04:43:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00b1e00 con 0x7f27f01599e0 2023-12-20T04:43:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.830+0000 7f27f7ee5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27f00b20e0 con 0x7f27f00b0b30 2023-12-20T04:43:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f5480640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00c90e0 0x7f27f00b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f5480640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00c90e0 0x7f27f00b38a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36824/0 (socket says 172.21.15.32:36824) 2023-12-20T04:43:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f5480640 1 -- 172.21.15.32:0/3496266857 learned_addr learned my addr 172.21.15.32:0/3496266857 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:08.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f5c81640 1 --2- 172.21.15.32:0/3496266857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f00b0b30 0x7f27f014c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f6482640 1 --2- 172.21.15.32:0/3496266857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27f01599e0 0x7f27f00b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f5480640 1 -- 172.21.15.32:0/3496266857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f00b0b30 msgr2=0x7f27f014c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f5480640 1 --2- 172.21.15.32:0/3496266857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f00b0b30 0x7f27f014c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f5480640 1 -- 172.21.15.32:0/3496266857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27f01599e0 msgr2=0x7f27f00b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f5480640 1 --2- 172.21.15.32:0/3496266857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27f01599e0 0x7f27f00b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f5480640 1 -- 172.21.15.32:0/3496266857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec009480 con 0x7f27f00c90e0 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f5480640 1 --2- 172.21.15.32:0/3496266857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00c90e0 0x7f27f00b38a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f27e000d800 tx=0x7f27e000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27e67fc640 1 -- 172.21.15.32:0/3496266857 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e000b6c0 con 0x7f27f00c90e0 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27e67fc640 1 -- 172.21.15.32:0/3496266857 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e000b820 con 0x7f27f00c90e0 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3496266857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27f00b23c0 con 0x7f27f00c90e0 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27e67fc640 1 -- 172.21.15.32:0/3496266857 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e000bb90 con 0x7f27f00c90e0 2023-12-20T04:43:08.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3496266857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27f00b2980 con 0x7f27f00c90e0 2023-12-20T04:43:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3496266857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27f00b2b40 con 0x7f27f00c90e0 2023-12-20T04:43:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27e67fc640 1 -- 172.21.15.32:0/3496266857 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 27) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f27e000bcf0 con 0x7f27f00c90e0 2023-12-20T04:43:08.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.834+0000 7f27e67fc640 1 -- 172.21.15.32:0/3496266857 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f27e0083370 con 0x7f27f00c90e0 2023-12-20T04:43:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:08.838+0000 7f27e67fc640 1 -- 172.21.15.32:0/3496266857 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f27e0050e90 con 0x7f27f00c90e0 2023-12-20T04:43:09.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.002+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3496266857 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f27f004e000 con 0x7f27f00c90e0 2023-12-20T04:43:09.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.002+0000 7f27e67fc640 1 -- 172.21.15.32:0/3496266857 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17012 (secure 0 0 0) 0x7f27e00544b0 con 0x7f27f00c90e0 2023-12-20T04:43:09.001 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:09.001 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:08.233077+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:40:57.540206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-20T04:40:58.913164+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":5}]},{"pgid":"7.2a","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3952176094":"2023-12-21T04:43:08.233036+0000","172.21.15.32:0/4013312171":"2023-12-21T04:43:08.233036+0000","172.21.15.32:0/1898542955":"2023-12-21T04:43:08.233036+0000","172.21.15.32:0/1396114783":"2023-12-21T04:43:08.233036+0000","172.21.15.32:0/860082660":"2023-12-21T04:43:08.233036+0000","172.21.15.32:0/2279832503":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/1700967844":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/384810653":"2023-12-21T04:41:11.975357+0000","172.21.15.32:6832/38009":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/935462264":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/936743272":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/151493359":"2023-12-21T04:43:08.233036+0000","172.21.15.32:6833/38009":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/3773919514":"2023-12-21T04:41:11.975357+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.002+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3496266857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00c90e0 msgr2=0x7f27f00b38a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.002+0000 7f27f7ee5640 1 --2- 172.21.15.32:0/3496266857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00c90e0 0x7f27f00b38a0 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f27e000d800 tx=0x7f27e000dcc0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.002+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3496266857 shutdown_connections 2023-12-20T04:43:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.002+0000 7f27f7ee5640 1 --2- 172.21.15.32:0/3496266857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27f01599e0 0x7f27f00b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.002+0000 7f27f7ee5640 1 --2- 172.21.15.32:0/3496266857 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27f00c90e0 0x7f27f00b38a0 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.002+0000 7f27f7ee5640 1 --2- 172.21.15.32:0/3496266857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27f00b0b30 0x7f27f014c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.002+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3496266857 >> 172.21.15.32:0/3496266857 conn(0x7f27f006a320 msgr2=0x7f27f00ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.002+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3496266857 shutdown_connections 2023-12-20T04:43:09.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.002+0000 7f27f7ee5640 1 -- 172.21.15.32:0/3496266857 wait complete. 2023-12-20T04:43:09.023 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T04:43:09.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:09.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.170+0000 7fcffa76d640 1 Processor -- start 2023-12-20T04:43:09.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.170+0000 7fcffa76d640 1 -- start start 2023-12-20T04:43:09.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.170+0000 7fcffa76d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcff41599e0 0x7fcff40aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.170+0000 7fcffa76d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff40c98c0 con 0x7fcff40b0d00 2023-12-20T04:43:09.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.170+0000 7fcffa76d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcff40c9ba0 con 0x7fcff40c90e0 2023-12-20T04:43:09.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.170+0000 7fcffa76d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff40c9e80 con 0x7fcff41599e0 2023-12-20T04:43:09.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.174+0000 7fcff8d0a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcff40b0d00 0x7fcff40701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60516/0 (socket says 172.21.15.32:60516) 2023-12-20T04:43:09.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.174+0000 7fcff8d0a640 1 -- 172.21.15.32:0/1461619026 learned_addr learned my addr 172.21.15.32:0/1461619026 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:09.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.174+0000 7fcff3fff640 1 --2- 172.21.15.32:0/1461619026 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcff41599e0 0x7fcff40aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.174+0000 7fcff2ffd640 1 -- 172.21.15.32:0/1461619026 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4269580652 0 0) 0x7fcff40c98c0 con 0x7fcff40b0d00 2023-12-20T04:43:09.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.174+0000 7fcff2ffd640 1 -- 172.21.15.32:0/1461619026 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfdc005740 con 0x7fcff40b0d00 2023-12-20T04:43:09.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.174+0000 7fcff2ffd640 1 -- 172.21.15.32:0/1461619026 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1948427669 0 0) 0x7fcff40c9ba0 con 0x7fcff40c90e0 2023-12-20T04:43:09.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.174+0000 7fcff3fff640 1 -- 172.21.15.32:0/1461619026 >> v1:172.21.15.39:6789/0 conn(0x7fcff40c90e0 legacy=0x7fcff40b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.174+0000 7fcff3fff640 1 -- 172.21.15.32:0/1461619026 >> v1:172.21.15.32:6789/0 conn(0x7fcff40b0d00 legacy=0x7fcff40701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.174+0000 7fcff3fff640 1 -- 172.21.15.32:0/1461619026 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcff40ca160 con 0x7fcff41599e0 2023-12-20T04:43:09.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.174+0000 7fcff3fff640 1 --2- 172.21.15.32:0/1461619026 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcff41599e0 0x7fcff40aece0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40097d0 tx=0x7fcfe401c220 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3cb2049323cd8eb2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:09.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcff2ffd640 1 -- 172.21.15.32:0/1461619026 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfec001a00 con 0x7fcff41599e0 2023-12-20T04:43:09.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcff2ffd640 1 -- 172.21.15.32:0/1461619026 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe40a7050 con 0x7fcff41599e0 2023-12-20T04:43:09.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcff2ffd640 1 -- 172.21.15.32:0/1461619026 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe40ab620 con 0x7fcff41599e0 2023-12-20T04:43:09.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 -- 172.21.15.32:0/1461619026 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcff41599e0 msgr2=0x7fcff40aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 --2- 172.21.15.32:0/1461619026 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcff41599e0 0x7fcff40aece0 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40097d0 tx=0x7fcfe401c220 comp rx=0 tx=0).stop 2023-12-20T04:43:09.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 -- 172.21.15.32:0/1461619026 shutdown_connections 2023-12-20T04:43:09.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 --2- 172.21.15.32:0/1461619026 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcff41599e0 0x7fcff40aece0 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 -- 172.21.15.32:0/1461619026 >> 172.21.15.32:0/1461619026 conn(0x7fcff406a320 msgr2=0x7fcff40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:09.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 -- 172.21.15.32:0/1461619026 shutdown_connections 2023-12-20T04:43:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 -- 172.21.15.32:0/1461619026 wait complete. 2023-12-20T04:43:09.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 Processor -- start 2023-12-20T04:43:09.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 -- start start 2023-12-20T04:43:09.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff40b0d00 0x7fcff41f54f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcff40c90e0 0x7fcff4158cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcff41599e0 0x7fcff414a9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff41591f0 con 0x7fcff40b0d00 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcff41f5b20 con 0x7fcff41599e0 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcffa76d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcff41f5e00 con 0x7fcff40c90e0 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.178+0000 7fcff3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff40b0d00 0x7fcff41f54f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff40b0d00 0x7fcff41f54f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36838/0 (socket says 172.21.15.32:36838) 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff3fff640 1 -- 172.21.15.32:0/3293141944 learned_addr learned my addr 172.21.15.32:0/3293141944 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff8d0a640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcff41599e0 0x7fcff414a9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff37fe640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcff40c90e0 0x7fcff4158cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff3fff640 1 -- 172.21.15.32:0/3293141944 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcff40c90e0 msgr2=0x7fcff4158cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff3fff640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcff40c90e0 0x7fcff4158cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff3fff640 1 -- 172.21.15.32:0/3293141944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcff41599e0 msgr2=0x7fcff414a9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff3fff640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcff41599e0 0x7fcff414a9f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff3fff640 1 -- 172.21.15.32:0/3293141944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfe40093b0 con 0x7fcff40b0d00 2023-12-20T04:43:09.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff3fff640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff40b0d00 0x7fcff41f54f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40097d0 tx=0x7fcfe4018fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:09.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff0ff9640 1 -- 172.21.15.32:0/3293141944 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe40059a0 con 0x7fcff40b0d00 2023-12-20T04:43:09.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff0ff9640 1 -- 172.21.15.32:0/3293141944 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfe4005b20 con 0x7fcff40b0d00 2023-12-20T04:43:09.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcffa76d640 1 -- 172.21.15.32:0/3293141944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcff41f60e0 con 0x7fcff40b0d00 2023-12-20T04:43:09.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff0ff9640 1 -- 172.21.15.32:0/3293141944 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfe4006020 con 0x7fcff40b0d00 2023-12-20T04:43:09.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcffa76d640 1 -- 172.21.15.32:0/3293141944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcff41f8b60 con 0x7fcff40b0d00 2023-12-20T04:43:09.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcffa76d640 1 -- 172.21.15.32:0/3293141944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcff41f8d50 con 0x7fcff40b0d00 2023-12-20T04:43:09.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff0ff9640 1 -- 172.21.15.32:0/3293141944 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 27) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fcfe40b2040 con 0x7fcff40b0d00 2023-12-20T04:43:09.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.182+0000 7fcff0ff9640 1 -- 172.21.15.32:0/3293141944 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fcfe411cd50 con 0x7fcff40b0d00 2023-12-20T04:43:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.186+0000 7fcff0ff9640 1 -- 172.21.15.32:0/3293141944 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcfe40ea7c0 con 0x7fcff40b0d00 2023-12-20T04:43:09.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.246+0000 7fcff0ff9640 1 -- 172.21.15.32:0/3293141944 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 28) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fcfe40ea270 con 0x7fcff40b0d00 2023-12-20T04:43:09.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.246+0000 7fcff0ff9640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfe001c9b0 0x7fcfe006d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.250+0000 7fcff37fe640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfe001c9b0 0x7fcfe006d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.250+0000 7fcff37fe640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfe001c9b0 0x7fcfe006d210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcff4158520 tx=0x7fcfe8009040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.386+0000 7fcffa76d640 1 -- 172.21.15.32:0/3293141944 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcff41f5f40 con 0x7fcff40b0d00 2023-12-20T04:43:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.386+0000 7fcff0ff9640 1 -- 172.21.15.32:0/3293141944 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4700 (secure 0 0 0) 0x7fcff41f5f40 con 0x7fcff40b0d00 2023-12-20T04:43:09.385 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:09.385 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/536564441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":536564441},{"type":"v1","addr":"172.21.15.32:6835","nonce":536564441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2898125535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2898125535},{"type":"v1","addr":"172.21.15.39:6835","nonce":2898125535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/319656564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":319656564},{"type":"v1","addr":"172.21.15.32:6837","nonce":319656564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:40:59.920384+0000","modified":"2023-12-20T04:41:02.943287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5895},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5895":{"gid":5895,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/192084837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":192084837},{"type":"v1","addr":"172.21.15.39:6834","nonce":192084837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-20T04:43:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2023-12-20T04:43:09.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.386+0000 7fcffa76d640 1 -- 172.21.15.32:0/3293141944 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfe001c9b0 msgr2=0x7fcfe006d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.386+0000 7fcffa76d640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfe001c9b0 0x7fcfe006d210 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fcff4158520 tx=0x7fcfe8009040 comp rx=0 tx=0).stop 2023-12-20T04:43:09.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.386+0000 7fcffa76d640 1 -- 172.21.15.32:0/3293141944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff40b0d00 msgr2=0x7fcff41f54f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.386+0000 7fcffa76d640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff40b0d00 0x7fcff41f54f0 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7fcfe40097d0 tx=0x7fcfe4018fc0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.390+0000 7fcffa76d640 1 -- 172.21.15.32:0/3293141944 shutdown_connections 2023-12-20T04:43:09.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.390+0000 7fcffa76d640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfe001c9b0 0x7fcfe006d210 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.390+0000 7fcffa76d640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcff41599e0 0x7fcff414a9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.390+0000 7fcffa76d640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcff40c90e0 0x7fcff4158cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.390+0000 7fcffa76d640 1 --2- 172.21.15.32:0/3293141944 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcff40b0d00 0x7fcff41f54f0 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.390+0000 7fcffa76d640 1 -- 172.21.15.32:0/3293141944 >> 172.21.15.32:0/3293141944 conn(0x7fcff406a320 msgr2=0x7fcff414bc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:09.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.390+0000 7fcffa76d640 1 -- 172.21.15.32:0/3293141944 shutdown_connections 2023-12-20T04:43:09.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.390+0000 7fcffa76d640 1 -- 172.21.15.32:0/3293141944 wait complete. 2023-12-20T04:43:09.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.550+0000 7f41b6273640 1 Processor -- start 2023-12-20T04:43:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.550+0000 7f41b6273640 1 -- start start 2023-12-20T04:43:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b6273640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f41b015ce60 0x7f41b00c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b6273640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41b0161240 0x7f41b0071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b6273640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41b0071d80 con 0x7f41b00713e0 2023-12-20T04:43:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b6273640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b0161690 con 0x7f41b0161240 2023-12-20T04:43:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b6273640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41b00c9a40 con 0x7f41b015ce60 2023-12-20T04:43:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b4810640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41b0161240 0x7f41b0071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b4810640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f41b0161240 0x7f41b0071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36852/0 (socket says 172.21.15.32:36852) 2023-12-20T04:43:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b4810640 1 -- 172.21.15.32:0/428192313 learned_addr learned my addr 172.21.15.32:0/428192313 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41affff640 1 --2- 172.21.15.32:0/428192313 >> v2:172.21.15.39:3300/0 conn(0x7f41b015ce60 0x7f41b00c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b4810640 1 -- 172.21.15.32:0/428192313 >> v2:172.21.15.39:3300/0 conn(0x7f41b015ce60 msgr2=0x7f41b00c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b4810640 1 --2- 172.21.15.32:0/428192313 >> v2:172.21.15.39:3300/0 conn(0x7f41b015ce60 0x7f41b00c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b4810640 1 -- 172.21.15.32:0/428192313 >> v1:172.21.15.39:6789/0 conn(0x7f41b00713e0 legacy=0x7f41b0160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b4810640 1 -- 172.21.15.32:0/428192313 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41b00c9180 con 0x7f41b0161240 2023-12-20T04:43:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.554+0000 7f41b4810640 1 --2- 172.21.15.32:0/428192313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41b0161240 0x7f41b0071840 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f41b00cc230 tx=0x7f41a801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bb99e4b7fa8c26a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41aeffd640 1 -- 172.21.15.32:0/428192313 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80a5780 con 0x7f41b0161240 2023-12-20T04:43:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41aeffd640 1 -- 172.21.15.32:0/428192313 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a80a58e0 con 0x7f41b0161240 2023-12-20T04:43:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41aeffd640 1 -- 172.21.15.32:0/428192313 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80a5c70 con 0x7f41b0161240 2023-12-20T04:43:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 -- 172.21.15.32:0/428192313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41b0161240 msgr2=0x7f41b0071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 --2- 172.21.15.32:0/428192313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41b0161240 0x7f41b0071840 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f41b00cc230 tx=0x7f41a801c9d0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 -- 172.21.15.32:0/428192313 shutdown_connections 2023-12-20T04:43:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 --2- 172.21.15.32:0/428192313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41b0161240 0x7f41b0071840 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 --2- 172.21.15.32:0/428192313 >> v2:172.21.15.39:3300/0 conn(0x7f41b015ce60 0x7f41b00c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 -- 172.21.15.32:0/428192313 >> 172.21.15.32:0/428192313 conn(0x7f41b006a320 msgr2=0x7f41b00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 -- 172.21.15.32:0/428192313 shutdown_connections 2023-12-20T04:43:09.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 -- 172.21.15.32:0/428192313 wait complete. 2023-12-20T04:43:09.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 Processor -- start 2023-12-20T04:43:09.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 -- start start 2023-12-20T04:43:09.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41b00713e0 0x7f41b015fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b015ce60 0x7f41b015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41b0161240 0x7f41b0157110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00c8810 con 0x7f41b0161240 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41b01602b0 con 0x7f41b015ce60 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.558+0000 7f41b6273640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41b00c3140 con 0x7f41b00713e0 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b4810640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41b0161240 0x7f41b0157110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b4810640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41b0161240 0x7f41b0157110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36860/0 (socket says 172.21.15.32:36860) 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b4810640 1 -- 172.21.15.32:0/3919917042 learned_addr learned my addr 172.21.15.32:0/3919917042 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41affff640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41b00713e0 0x7f41b015fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41af7fe640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b015ce60 0x7f41b015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b4810640 1 -- 172.21.15.32:0/3919917042 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41b00713e0 msgr2=0x7f41b015fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b4810640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41b00713e0 0x7f41b015fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b4810640 1 -- 172.21.15.32:0/3919917042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b015ce60 msgr2=0x7f41b015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b4810640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b015ce60 0x7f41b015d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b4810640 1 -- 172.21.15.32:0/3919917042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a8009b30 con 0x7f41b0161240 2023-12-20T04:43:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41af7fe640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b015ce60 0x7f41b015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b4810640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41b0161240 0x7f41b0157110 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f41a80af040 tx=0x7f41a8019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41acff9640 1 -- 172.21.15.32:0/3919917042 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80b8640 con 0x7f41b0161240 2023-12-20T04:43:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41acff9640 1 -- 172.21.15.32:0/3919917042 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a80b87a0 con 0x7f41b0161240 2023-12-20T04:43:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b6273640 1 -- 172.21.15.32:0/3919917042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41b00c3420 con 0x7f41b0161240 2023-12-20T04:43:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41acff9640 1 -- 172.21.15.32:0/3919917042 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a80b8b10 con 0x7f41b0161240 2023-12-20T04:43:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b6273640 1 -- 172.21.15.32:0/3919917042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41b00c39e0 con 0x7f41b0161240 2023-12-20T04:43:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41b6273640 1 -- 172.21.15.32:0/3919917042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41b00c3bd0 con 0x7f41b0161240 2023-12-20T04:43:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.562+0000 7f41acff9640 1 -- 172.21.15.32:0/3919917042 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 28) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f41a80b8c70 con 0x7f41b0161240 2023-12-20T04:43:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.566+0000 7f41acff9640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f419c0658b0 0x7f419c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.566+0000 7f41acff9640 1 -- 172.21.15.32:0/3919917042 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f41a811e490 con 0x7f41b0161240 2023-12-20T04:43:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.566+0000 7f41affff640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f419c0658b0 0x7f419c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.566+0000 7f41affff640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f419c0658b0 0x7f419c067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009b40 tx=0x7f41a0006f90 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:09.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.570+0000 7f41acff9640 1 -- 172.21.15.32:0/3919917042 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f41a80ebf30 con 0x7f41b0161240 2023-12-20T04:43:09.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.758+0000 7f41b6273640 1 -- 172.21.15.32:0/3919917042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f41b004e000 con 0x7f41b0161240 2023-12-20T04:43:09.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41acff9640 1 -- 172.21.15.32:0/3919917042 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 24 v24) v1 ==== 94+0+4700 (secure 0 0 0) 0x7f41a8009510 con 0x7f41b0161240 2023-12-20T04:43:09.759 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:09.759 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":24,"default_fscid":3,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/536564441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":536564441},{"type":"v1","addr":"172.21.15.32:6835","nonce":536564441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2898125535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2898125535},{"type":"v1","addr":"172.21.15.39:6835","nonce":2898125535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/319656564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":319656564},{"type":"v1","addr":"172.21.15.32:6837","nonce":319656564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19}],"filesystems":[{"mdsmap":{"epoch":24,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:40:59.920384+0000","modified":"2023-12-20T04:41:02.943287+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":5895},"failed":[],"damaged":[],"stopped":[],"info":{"gid_5895":{"gid":5895,"name":"b","rank":0,"incarnation":20,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6834/192084837","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":192084837},{"type":"v1","addr":"172.21.15.39:6834","nonce":192084837}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[7],"metadata_pool":6,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":3}]} 2023-12-20T04:43:09.760 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 24 2023-12-20T04:43:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 -- 172.21.15.32:0/3919917042 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f419c0658b0 msgr2=0x7f419c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f419c0658b0 0x7f419c067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f41a0009b40 tx=0x7f41a0006f90 comp rx=0 tx=0).stop 2023-12-20T04:43:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 -- 172.21.15.32:0/3919917042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41b0161240 msgr2=0x7f41b0157110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41b0161240 0x7f41b0157110 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f41a80af040 tx=0x7f41a8019770 comp rx=0 tx=0).stop 2023-12-20T04:43:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 -- 172.21.15.32:0/3919917042 shutdown_connections 2023-12-20T04:43:09.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f419c0658b0 0x7f419c067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41b0161240 0x7f41b0157110 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41b015ce60 0x7f41b015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 --2- 172.21.15.32:0/3919917042 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41b00713e0 0x7f41b015fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 -- 172.21.15.32:0/3919917042 >> 172.21.15.32:0/3919917042 conn(0x7f41b006a320 msgr2=0x7f41b006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 -- 172.21.15.32:0/3919917042 shutdown_connections 2023-12-20T04:43:09.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.762+0000 7f41b6273640 1 -- 172.21.15.32:0/3919917042 wait complete. 2023-12-20T04:43:09.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:09.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d59c8c640 1 Processor -- start 2023-12-20T04:43:09.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d59c8c640 1 -- start start 2023-12-20T04:43:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d59c8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5d541599e0 0x7f5d540aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d59c8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540b0d00 0x7f5d540701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d59c8c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5d540c98c0 con 0x7f5d540c90e0 2023-12-20T04:43:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d59c8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5d540c9ba0 con 0x7f5d541599e0 2023-12-20T04:43:09.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d59c8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d540c9e80 con 0x7f5d540b0d00 2023-12-20T04:43:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d537fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5d541599e0 0x7f5d540aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d53fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540b0d00 0x7f5d540701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d52ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5d540c90e0 0x7f5d540b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:45272/0 (socket says 172.21.15.32:45272) 2023-12-20T04:43:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d52ffd640 1 -- 172.21.15.32:0/3636503993 learned_addr learned my addr 172.21.15.32:0/3636503993 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d537fe640 1 -- 172.21.15.32:0/3636503993 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540b0d00 msgr2=0x7f5d540701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d537fe640 1 --2- 172.21.15.32:0/3636503993 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540b0d00 0x7f5d540701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d537fe640 1 -- 172.21.15.32:0/3636503993 >> v1:172.21.15.39:6789/0 conn(0x7f5d540c90e0 legacy=0x7f5d540b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d537fe640 1 -- 172.21.15.32:0/3636503993 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d540ca160 con 0x7f5d541599e0 2023-12-20T04:43:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d53fff640 1 --2- 172.21.15.32:0/3636503993 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540b0d00 0x7f5d540701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:09.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.926+0000 7f5d527fc640 1 -- 172.21.15.32:0/3636503993 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3024953483 0 0) 0x7f5d540c98c0 con 0x7f5d540c90e0 2023-12-20T04:43:09.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d537fe640 1 --2- 172.21.15.32:0/3636503993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d541599e0 0x7f5d540aece0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5d440097a0 tx=0x7f5d4401c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=796676f4067f034d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:09.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d527fc640 1 -- 172.21.15.32:0/3636503993 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d440ac620 con 0x7f5d541599e0 2023-12-20T04:43:09.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d527fc640 1 -- 172.21.15.32:0/3636503993 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d440ac780 con 0x7f5d541599e0 2023-12-20T04:43:09.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d527fc640 1 -- 172.21.15.32:0/3636503993 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d440acb10 con 0x7f5d541599e0 2023-12-20T04:43:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d59c8c640 1 -- 172.21.15.32:0/3636503993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d541599e0 msgr2=0x7f5d540aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d59c8c640 1 --2- 172.21.15.32:0/3636503993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d541599e0 0x7f5d540aece0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f5d440097a0 tx=0x7f5d4401c3a0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d59c8c640 1 -- 172.21.15.32:0/3636503993 shutdown_connections 2023-12-20T04:43:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d59c8c640 1 --2- 172.21.15.32:0/3636503993 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540b0d00 0x7f5d540701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d59c8c640 1 --2- 172.21.15.32:0/3636503993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d541599e0 0x7f5d540aece0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d59c8c640 1 -- 172.21.15.32:0/3636503993 >> 172.21.15.32:0/3636503993 conn(0x7f5d5406a320 msgr2=0x7f5d540ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d59c8c640 1 -- 172.21.15.32:0/3636503993 shutdown_connections 2023-12-20T04:43:09.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.930+0000 7f5d59c8c640 1 -- 172.21.15.32:0/3636503993 wait complete. 2023-12-20T04:43:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d59c8c640 1 Processor -- start 2023-12-20T04:43:09.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d59c8c640 1 -- start start 2023-12-20T04:43:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d59c8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d540b0d00 0x7f5d54167200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d59c8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540c90e0 0x7f5d54147b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d59c8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d541599e0 0x7f5d54166ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d59c8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d541677e0 con 0x7f5d540b0d00 2023-12-20T04:43:09.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d59c8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5d54159190 con 0x7f5d541599e0 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d59c8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5d54158bd0 con 0x7f5d540c90e0 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d537fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d540b0d00 0x7f5d54167200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d537fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d540b0d00 0x7f5d54167200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36870/0 (socket says 172.21.15.32:36870) 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d537fe640 1 -- 172.21.15.32:0/4209518983 learned_addr learned my addr 172.21.15.32:0/4209518983 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d53fff640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d541599e0 0x7f5d54166ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d52ffd640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540c90e0 0x7f5d54147b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d537fe640 1 -- 172.21.15.32:0/4209518983 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540c90e0 msgr2=0x7f5d54147b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d537fe640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540c90e0 0x7f5d54147b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d537fe640 1 -- 172.21.15.32:0/4209518983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d541599e0 msgr2=0x7f5d54166ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d537fe640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d541599e0 0x7f5d54166ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d537fe640 1 -- 172.21.15.32:0/4209518983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5d4c009a70 con 0x7f5d540b0d00 2023-12-20T04:43:09.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d52ffd640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540c90e0 0x7f5d54147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:09.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d537fe640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d540b0d00 0x7f5d54167200 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5d4401aa50 tx=0x7f5d4401abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:09.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d3bfff640 1 -- 172.21.15.32:0/4209518983 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d440ac420 con 0x7f5d540b0d00 2023-12-20T04:43:09.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d3bfff640 1 -- 172.21.15.32:0/4209518983 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5d440ac5a0 con 0x7f5d540b0d00 2023-12-20T04:43:09.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d3bfff640 1 -- 172.21.15.32:0/4209518983 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5d44005be0 con 0x7f5d540b0d00 2023-12-20T04:43:09.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d59c8c640 1 -- 172.21.15.32:0/4209518983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5d440093b0 con 0x7f5d540b0d00 2023-12-20T04:43:09.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.934+0000 7f5d59c8c640 1 -- 172.21.15.32:0/4209518983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5d541665c0 con 0x7f5d540b0d00 2023-12-20T04:43:09.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.938+0000 7f5d59c8c640 1 -- 172.21.15.32:0/4209518983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5d54167020 con 0x7f5d540b0d00 2023-12-20T04:43:09.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.938+0000 7f5d3bfff640 1 -- 172.21.15.32:0/4209518983 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 28) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f5d440aa070 con 0x7f5d540b0d00 2023-12-20T04:43:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.938+0000 7f5d3bfff640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5d400658e0 0x7f5d40067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.938+0000 7f5d3bfff640 1 -- 172.21.15.32:0/4209518983 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f5d4411d710 con 0x7f5d540b0d00 2023-12-20T04:43:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.938+0000 7f5d52ffd640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5d400658e0 0x7f5d40067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:09.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.938+0000 7f5d52ffd640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5d400658e0 0x7f5d40067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5d48007fd0 tx=0x7f5d48007c20 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:09.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:09.942+0000 7f5d3bfff640 1 -- 172.21.15.32:0/4209518983 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5d440eb1b0 con 0x7f5d540b0d00 2023-12-20T04:43:10.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.102+0000 7f5d59c8c640 1 -- 172.21.15.32:0/4209518983 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5d5404e000 con 0x7f5d540b0d00 2023-12-20T04:43:10.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.102+0000 7f5d3bfff640 1 -- 172.21.15.32:0/4209518983 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v67) v1 ==== 74+0+17012 (secure 0 0 0) 0x7f5d440ee7d0 con 0x7f5d540b0d00 2023-12-20T04:43:10.101 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:10.101 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":67,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:08.233077+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":6,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:40:57.540206+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"60","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":7,"pool_name":"cephfs_data","create_time":"2023-12-20T04:40:58.913164+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"66","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":66,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"7.13","mappings":[{"from":0,"to":5}]},{"pgid":"7.2a","mappings":[{"from":0,"to":4}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3952176094":"2023-12-21T04:43:08.233036+0000","172.21.15.32:0/4013312171":"2023-12-21T04:43:08.233036+0000","172.21.15.32:0/1898542955":"2023-12-21T04:43:08.233036+0000","172.21.15.32:0/1396114783":"2023-12-21T04:43:08.233036+0000","172.21.15.32:0/860082660":"2023-12-21T04:43:08.233036+0000","172.21.15.32:0/2279832503":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/1700967844":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/384810653":"2023-12-21T04:41:11.975357+0000","172.21.15.32:6832/38009":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/935462264":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/936743272":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/151493359":"2023-12-21T04:43:08.233036+0000","172.21.15.32:6833/38009":"2023-12-21T04:41:11.975357+0000","172.21.15.32:0/3773919514":"2023-12-21T04:41:11.975357+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":7,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:10.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 -- 172.21.15.32:0/4209518983 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5d400658e0 msgr2=0x7f5d40067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:10.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5d400658e0 0x7f5d40067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f5d48007fd0 tx=0x7f5d48007c20 comp rx=0 tx=0).stop 2023-12-20T04:43:10.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 -- 172.21.15.32:0/4209518983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d540b0d00 msgr2=0x7f5d54167200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:10.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d540b0d00 0x7f5d54167200 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f5d4401aa50 tx=0x7f5d4401abe0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 -- 172.21.15.32:0/4209518983 shutdown_connections 2023-12-20T04:43:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5d400658e0 0x7f5d40067da0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5d541599e0 0x7f5d54166ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5d540c90e0 0x7f5d54147b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 --2- 172.21.15.32:0/4209518983 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5d540b0d00 0x7f5d54167200 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 -- 172.21.15.32:0/4209518983 >> 172.21.15.32:0/4209518983 conn(0x7f5d5406a320 msgr2=0x7f5d54071780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 -- 172.21.15.32:0/4209518983 shutdown_connections 2023-12-20T04:43:10.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.106+0000 7f5d59c8c640 1 -- 172.21.15.32:0/4209518983 wait complete. 2023-12-20T04:43:10.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T04:43:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9898bb0640 1 Processor -- start 2023-12-20T04:43:10.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9898bb0640 1 -- start start 2023-12-20T04:43:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9898bb0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98940b8480 0x7f989406af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9898bb0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f989406b460 0x7f9894160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9898bb0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9894161330 0x7f9894153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9898bb0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f989406b8c0 con 0x7f9894161330 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9898bb0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98940cd4d0 con 0x7f98940b8480 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9898bb0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98940ce4e0 con 0x7f989406b460 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9893fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9894161330 0x7f9894153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f98937fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98940b8480 0x7f989406af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9892ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f989406b460 0x7f9894160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9893fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9894161330 0x7f9894153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36884/0 (socket says 172.21.15.32:36884) 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9893fff640 1 -- 172.21.15.32:0/1153276566 learned_addr learned my addr 172.21.15.32:0/1153276566 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9893fff640 1 -- 172.21.15.32:0/1153276566 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f989406b460 msgr2=0x7f9894160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9893fff640 1 --2- 172.21.15.32:0/1153276566 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f989406b460 0x7f9894160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9893fff640 1 -- 172.21.15.32:0/1153276566 >> v2:172.21.15.39:3300/0 conn(0x7f98940b8480 msgr2=0x7f989406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9893fff640 1 --2- 172.21.15.32:0/1153276566 >> v2:172.21.15.39:3300/0 conn(0x7f98940b8480 0x7f989406af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.278+0000 7f9893fff640 1 -- 172.21.15.32:0/1153276566 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98940cd720 con 0x7f9894161330 2023-12-20T04:43:10.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f9893fff640 1 --2- 172.21.15.32:0/1153276566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9894161330 0x7f9894153610 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f989406bb70 tx=0x7f988c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb0fba6d1ef4b939 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:10.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f98927fc640 1 -- 172.21.15.32:0/1153276566 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0a5740 con 0x7f9894161330 2023-12-20T04:43:10.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f98927fc640 1 -- 172.21.15.32:0/1153276566 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988c0a58a0 con 0x7f9894161330 2023-12-20T04:43:10.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f98927fc640 1 -- 172.21.15.32:0/1153276566 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988c0a5c30 con 0x7f9894161330 2023-12-20T04:43:10.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f9898bb0640 1 -- 172.21.15.32:0/1153276566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9894161330 msgr2=0x7f9894153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:10.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f9898bb0640 1 --2- 172.21.15.32:0/1153276566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9894161330 0x7f9894153610 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f989406bb70 tx=0x7f988c01c9d0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f9898bb0640 1 -- 172.21.15.32:0/1153276566 shutdown_connections 2023-12-20T04:43:10.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f9898bb0640 1 --2- 172.21.15.32:0/1153276566 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9894161330 0x7f9894153610 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f9898bb0640 1 --2- 172.21.15.32:0/1153276566 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f989406b460 0x7f9894160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f9898bb0640 1 --2- 172.21.15.32:0/1153276566 >> v2:172.21.15.39:3300/0 conn(0x7f98940b8480 0x7f989406af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f9898bb0640 1 -- 172.21.15.32:0/1153276566 >> 172.21.15.32:0/1153276566 conn(0x7f98940d2a50 msgr2=0x7f98940d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f9898bb0640 1 -- 172.21.15.32:0/1153276566 shutdown_connections 2023-12-20T04:43:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.282+0000 7f9898bb0640 1 -- 172.21.15.32:0/1153276566 wait complete. 2023-12-20T04:43:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9898bb0640 1 Processor -- start 2023-12-20T04:43:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9898bb0640 1 -- start start 2023-12-20T04:43:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9898bb0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989406b460 0x7f989414cb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9898bb0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98940b8480 0x7f98940c4b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9898bb0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894161330 0x7f9894148790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9898bb0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98941539d0 con 0x7f989406b460 2023-12-20T04:43:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9898bb0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98941529a0 con 0x7f9894161330 2023-12-20T04:43:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9898bb0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9894152c10 con 0x7f98940b8480 2023-12-20T04:43:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f98937fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989406b460 0x7f989414cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f98937fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989406b460 0x7f989414cb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36890/0 (socket says 172.21.15.32:36890) 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f98937fe640 1 -- 172.21.15.32:0/3551802654 learned_addr learned my addr 172.21.15.32:0/3551802654 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9892ffd640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98940b8480 0x7f98940c4b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9893fff640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894161330 0x7f9894148790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f98937fe640 1 -- 172.21.15.32:0/3551802654 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98940b8480 msgr2=0x7f98940c4b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f98937fe640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98940b8480 0x7f98940c4b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f98937fe640 1 -- 172.21.15.32:0/3551802654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894161330 msgr2=0x7f9894148790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f98937fe640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894161330 0x7f9894148790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f98937fe640 1 -- 172.21.15.32:0/3551802654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988c009b30 con 0x7f989406b460 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f98937fe640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989406b460 0x7f989414cb30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f988400e960 tx=0x7f988400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f987bfff640 1 -- 172.21.15.32:0/3551802654 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9884009690 con 0x7f989406b460 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f987bfff640 1 -- 172.21.15.32:0/3551802654 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f988400ca00 con 0x7f989406b460 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f987bfff640 1 -- 172.21.15.32:0/3551802654 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f988400cd70 con 0x7f989406b460 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9898bb0640 1 -- 172.21.15.32:0/3551802654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9894152ef0 con 0x7f989406b460 2023-12-20T04:43:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.286+0000 7f9898bb0640 1 -- 172.21.15.32:0/3551802654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98941534b0 con 0x7f989406b460 2023-12-20T04:43:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.290+0000 7f9898bb0640 1 -- 172.21.15.32:0/3551802654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9894074100 con 0x7f989406b460 2023-12-20T04:43:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.290+0000 7f987bfff640 1 -- 172.21.15.32:0/3551802654 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 29) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f9884013470 con 0x7f989406b460 2023-12-20T04:43:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.290+0000 7f987bfff640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9880065af0 0x7f9880067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.290+0000 7f987bfff640 1 -- 172.21.15.32:0/3551802654 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(67..67 src has 1..67) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f98840836b0 con 0x7f989406b460 2023-12-20T04:43:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.290+0000 7f9892ffd640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9880065af0 0x7f9880067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.290+0000 7f9892ffd640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9880065af0 0x7f9880067fb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f989415dd10 tx=0x7f9888009190 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.294+0000 7f987bfff640 1 -- 172.21.15.32:0/3551802654 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9884050d80 con 0x7f989406b460 2023-12-20T04:43:10.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.490+0000 7f9898bb0640 1 -- 172.21.15.32:0/3551802654 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f98940d0a70 con 0x7f989406b460 2023-12-20T04:43:10.491 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T04:43:10.490+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T04:43:10.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.506+0000 7f987bfff640 1 -- 172.21.15.32:0/3551802654 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v25) v1 ==== 161+0+0 (secure 0 0 0) 0x7f98840543a0 con 0x7f989406b460 2023-12-20T04:43:10.505 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T04:43:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.506+0000 7f9898bb0640 1 -- 172.21.15.32:0/3551802654 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9880065af0 msgr2=0x7f9880067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.506+0000 7f9898bb0640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9880065af0 0x7f9880067fb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f989415dd10 tx=0x7f9888009190 comp rx=0 tx=0).stop 2023-12-20T04:43:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.506+0000 7f9898bb0640 1 -- 172.21.15.32:0/3551802654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989406b460 msgr2=0x7f989414cb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.506+0000 7f9898bb0640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989406b460 0x7f989414cb30 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f988400e960 tx=0x7f988400ee20 comp rx=0 tx=0).stop 2023-12-20T04:43:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.510+0000 7f9898bb0640 1 -- 172.21.15.32:0/3551802654 shutdown_connections 2023-12-20T04:43:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.510+0000 7f9898bb0640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9880065af0 0x7f9880067fb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.510+0000 7f9898bb0640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9894161330 0x7f9894148790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.510+0000 7f9898bb0640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98940b8480 0x7f98940c4b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.510+0000 7f9898bb0640 1 --2- 172.21.15.32:0/3551802654 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f989406b460 0x7f989414cb30 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.510+0000 7f9898bb0640 1 -- 172.21.15.32:0/3551802654 >> 172.21.15.32:0/3551802654 conn(0x7f98940d2a50 msgr2=0x7f98940b7150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.510+0000 7f9898bb0640 1 -- 172.21.15.32:0/3551802654 shutdown_connections 2023-12-20T04:43:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.510+0000 7f9898bb0640 1 -- 172.21.15.32:0/3551802654 wait complete. 2023-12-20T04:43:10.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T04:43:10.570 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T04:43:10.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe4a0939640 1 Processor -- start 2023-12-20T04:43:10.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe4a0939640 1 -- start start 2023-12-20T04:43:10.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe4a0939640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe49c0cb050 0x7fe49c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe4a0939640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe49c065d40 0x7fe49c161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe4a0939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe49c06ca60 0x7fe49c06cef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe4a0939640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c06d430 con 0x7fe49c065d40 2023-12-20T04:43:10.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe4a0939640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe49c06a1f0 con 0x7fe49c0cb050 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe4a0939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c161860 con 0x7fe49c06ca60 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe499d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe49c065d40 0x7fe49c161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe499d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe49c065d40 0x7fe49c161320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36932/0 (socket says 172.21.15.32:36932) 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe499d9b640 1 -- 172.21.15.32:0/3342682734 learned_addr learned my addr 172.21.15.32:0/3342682734 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe49ad9d640 1 --2- 172.21.15.32:0/3342682734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe49c06ca60 0x7fe49c06cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe49a59c640 1 --2- 172.21.15.32:0/3342682734 >> v2:172.21.15.39:3300/0 conn(0x7fe49c0cb050 0x7fe49c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe499d9b640 1 -- 172.21.15.32:0/3342682734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe49c06ca60 msgr2=0x7fe49c06cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe499d9b640 1 --2- 172.21.15.32:0/3342682734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe49c06ca60 0x7fe49c06cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe499d9b640 1 -- 172.21.15.32:0/3342682734 >> v2:172.21.15.39:3300/0 conn(0x7fe49c0cb050 msgr2=0x7fe49c0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe499d9b640 1 --2- 172.21.15.32:0/3342682734 >> v2:172.21.15.39:3300/0 conn(0x7fe49c0cb050 0x7fe49c0cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe499d9b640 1 -- 172.21.15.32:0/3342682734 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe49c06a440 con 0x7fe49c065d40 2023-12-20T04:43:10.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.674+0000 7fe49ad9d640 1 --2- 172.21.15.32:0/3342682734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe49c06ca60 0x7fe49c06cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:10.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe499d9b640 1 --2- 172.21.15.32:0/3342682734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe49c065d40 0x7fe49c161320 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe49c071ad0 tx=0x7fe49001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8199a0e193be362a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:10.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe49959a640 1 -- 172.21.15.32:0/3342682734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4900aa5f0 con 0x7fe49c065d40 2023-12-20T04:43:10.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe49959a640 1 -- 172.21.15.32:0/3342682734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4900aa750 con 0x7fe49c065d40 2023-12-20T04:43:10.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe49959a640 1 -- 172.21.15.32:0/3342682734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4900aaae0 con 0x7fe49c065d40 2023-12-20T04:43:10.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe4a0939640 1 -- 172.21.15.32:0/3342682734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe49c065d40 msgr2=0x7fe49c161320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:10.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe4a0939640 1 --2- 172.21.15.32:0/3342682734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe49c065d40 0x7fe49c161320 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7fe49c071ad0 tx=0x7fe49001c4d0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe4a0939640 1 -- 172.21.15.32:0/3342682734 shutdown_connections 2023-12-20T04:43:10.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe4a0939640 1 --2- 172.21.15.32:0/3342682734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe49c06ca60 0x7fe49c06cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe4a0939640 1 --2- 172.21.15.32:0/3342682734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe49c065d40 0x7fe49c161320 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe4a0939640 1 --2- 172.21.15.32:0/3342682734 >> v2:172.21.15.39:3300/0 conn(0x7fe49c0cb050 0x7fe49c0cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe4a0939640 1 -- 172.21.15.32:0/3342682734 >> 172.21.15.32:0/3342682734 conn(0x7fe49c0cf8d0 msgr2=0x7fe49c0cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:10.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe4a0939640 1 -- 172.21.15.32:0/3342682734 shutdown_connections 2023-12-20T04:43:10.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.678+0000 7fe4a0939640 1 -- 172.21.15.32:0/3342682734 wait complete. 2023-12-20T04:43:10.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe4a0939640 1 Processor -- start 2023-12-20T04:43:10.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe4a0939640 1 -- start start 2023-12-20T04:43:10.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe4a0939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe49c065d40 0x7fe49c0c55d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe4a0939640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe49c06ca60 0x7fe49c15e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe4a0939640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe49c0cb050 0x7fe49c15ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe4a0939640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0c5b10 con 0x7fe49c06ca60 2023-12-20T04:43:10.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe4a0939640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0c73e0 con 0x7fe49c0cb050 2023-12-20T04:43:10.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe4a0939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe49c0c76c0 con 0x7fe49c065d40 2023-12-20T04:43:10.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe499d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe49c06ca60 0x7fe49c15e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe499d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe49c06ca60 0x7fe49c15e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36934/0 (socket says 172.21.15.32:36934) 2023-12-20T04:43:10.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe499d9b640 1 -- 172.21.15.32:0/375813874 learned_addr learned my addr 172.21.15.32:0/375813874 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:10.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe49ad9d640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe49c0cb050 0x7fe49c15ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe49a59c640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe49c065d40 0x7fe49c0c55d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe499d9b640 1 -- 172.21.15.32:0/375813874 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe49c065d40 msgr2=0x7fe49c0c55d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe499d9b640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe49c065d40 0x7fe49c0c55d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe499d9b640 1 -- 172.21.15.32:0/375813874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe49c0cb050 msgr2=0x7fe49c15ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe499d9b640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe49c0cb050 0x7fe49c15ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe499d9b640 1 -- 172.21.15.32:0/375813874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4900094d0 con 0x7fe49c06ca60 2023-12-20T04:43:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe49ad9d640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe49c0cb050 0x7fe49c15ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe499d9b640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe49c06ca60 0x7fe49c15e9f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe49001ab80 tx=0x7fe49001ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe48affd640 1 -- 172.21.15.32:0/375813874 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4900b3800 con 0x7fe49c06ca60 2023-12-20T04:43:10.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe48affd640 1 -- 172.21.15.32:0/375813874 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4900b3980 con 0x7fe49c06ca60 2023-12-20T04:43:10.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe48affd640 1 -- 172.21.15.32:0/375813874 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4900b3d30 con 0x7fe49c06ca60 2023-12-20T04:43:10.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe4a0939640 1 -- 172.21.15.32:0/375813874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe49c0c79a0 con 0x7fe49c06ca60 2023-12-20T04:43:10.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe4a0939640 1 -- 172.21.15.32:0/375813874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe49c0c7f60 con 0x7fe49c06ca60 2023-12-20T04:43:10.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.682+0000 7fe4a0939640 1 -- 172.21.15.32:0/375813874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe49c04e000 con 0x7fe49c06ca60 2023-12-20T04:43:10.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.686+0000 7fe48affd640 1 -- 172.21.15.32:0/375813874 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 29) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fe4900b2380 con 0x7fe49c06ca60 2023-12-20T04:43:10.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.686+0000 7fe48affd640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065af0 0x7fe47c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:10.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.686+0000 7fe48affd640 1 -- 172.21.15.32:0/375813874 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6483+0+0 (secure 0 0 0) 0x7fe4900a7070 con 0x7fe49c06ca60 2023-12-20T04:43:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.686+0000 7fe49a59c640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065af0 0x7fe47c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:10.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.686+0000 7fe49a59c640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065af0 0x7fe47c067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0079e0 tx=0x7fe48c008040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:10.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.690+0000 7fe48affd640 1 -- 172.21.15.32:0/375813874 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe4900eb0c0 con 0x7fe49c06ca60 2023-12-20T04:43:10.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:10.878+0000 7fe4a0939640 1 -- 172.21.15.32:0/375813874 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fe49c06ace0 con 0x7fe49c06ca60 2023-12-20T04:43:11.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe48affd640 1 -- 172.21.15.32:0/375813874 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v26) v1 ==== 104+0+0 (secure 0 0 0) 0x7fe4900ee6e0 con 0x7fe49c06ca60 2023-12-20T04:43:11.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 -- 172.21.15.32:0/375813874 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065af0 msgr2=0x7fe47c067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065af0 0x7fe47c067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fe48c0079e0 tx=0x7fe48c008040 comp rx=0 tx=0).stop 2023-12-20T04:43:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 -- 172.21.15.32:0/375813874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe49c06ca60 msgr2=0x7fe49c15e9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe49c06ca60 0x7fe49c15e9f0 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7fe49001ab80 tx=0x7fe49001ad10 comp rx=0 tx=0).stop 2023-12-20T04:43:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 -- 172.21.15.32:0/375813874 shutdown_connections 2023-12-20T04:43:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065af0 0x7fe47c067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe49c0cb050 0x7fe49c15ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe49c06ca60 0x7fe49c15e9f0 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 --2- 172.21.15.32:0/375813874 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe49c065d40 0x7fe49c0c55d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 -- 172.21.15.32:0/375813874 >> 172.21.15.32:0/375813874 conn(0x7fe49c0cf8d0 msgr2=0x7fe49c15c5b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 -- 172.21.15.32:0/375813874 shutdown_connections 2023-12-20T04:43:11.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.510+0000 7fe4a0939640 1 -- 172.21.15.32:0/375813874 wait complete. 2023-12-20T04:43:11.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T04:43:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.686+0000 7f49df92c640 1 Processor -- start 2023-12-20T04:43:11.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.686+0000 7f49df92c640 1 -- start start 2023-12-20T04:43:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.690+0000 7f49df92c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f49d80ce0c0 0x7f49d816c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.690+0000 7f49df92c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d8161670 con 0x7f49d815b790 2023-12-20T04:43:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.690+0000 7f49df92c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49d8162690 con 0x7f49d8070940 2023-12-20T04:43:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.690+0000 7f49df92c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49d80ce6a0 con 0x7f49d80ce0c0 2023-12-20T04:43:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.690+0000 7f49ddec9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f49d80ce0c0 0x7f49d816c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:11.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.690+0000 7f49dd6c8640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f49d815b790 0x7f49d8070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60532/0 (socket says 172.21.15.32:60532) 2023-12-20T04:43:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.690+0000 7f49dd6c8640 1 -- 172.21.15.32:0/3398506502 learned_addr learned my addr 172.21.15.32:0/3398506502 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.690+0000 7f49ddec9640 1 -- 172.21.15.32:0/3398506502 >> v1:172.21.15.39:6789/0 conn(0x7f49d8070940 legacy=0x7f49d80cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.690+0000 7f49ddec9640 1 -- 172.21.15.32:0/3398506502 >> v1:172.21.15.32:6789/0 conn(0x7f49d815b790 legacy=0x7f49d8070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.690+0000 7f49ddec9640 1 -- 172.21.15.32:0/3398506502 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d815bb70 con 0x7f49d80ce0c0 2023-12-20T04:43:11.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.690+0000 7f49cffff640 1 -- 172.21.15.32:0/3398506502 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 301732221 0 0) 0x7f49d8161670 con 0x7f49d815b790 2023-12-20T04:43:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49ddec9640 1 --2- 172.21.15.32:0/3398506502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d80ce0c0 0x7f49d816c040 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009d90 tx=0x7f49d001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43af3ff23e629dfa server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49cffff640 1 -- 172.21.15.32:0/3398506502 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d00a59e0 con 0x7f49d80ce0c0 2023-12-20T04:43:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49cffff640 1 -- 172.21.15.32:0/3398506502 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49d00a5b40 con 0x7f49d80ce0c0 2023-12-20T04:43:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49cffff640 1 -- 172.21.15.32:0/3398506502 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49d00ad440 con 0x7f49d80ce0c0 2023-12-20T04:43:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 -- 172.21.15.32:0/3398506502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d80ce0c0 msgr2=0x7f49d816c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 --2- 172.21.15.32:0/3398506502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d80ce0c0 0x7f49d816c040 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f49d0009d90 tx=0x7f49d001c920 comp rx=0 tx=0).stop 2023-12-20T04:43:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 -- 172.21.15.32:0/3398506502 shutdown_connections 2023-12-20T04:43:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 --2- 172.21.15.32:0/3398506502 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d80ce0c0 0x7f49d816c040 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 -- 172.21.15.32:0/3398506502 >> 172.21.15.32:0/3398506502 conn(0x7f49d806a510 msgr2=0x7f49d80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 -- 172.21.15.32:0/3398506502 shutdown_connections 2023-12-20T04:43:11.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 -- 172.21.15.32:0/3398506502 wait complete. 2023-12-20T04:43:11.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 Processor -- start 2023-12-20T04:43:11.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 -- start start 2023-12-20T04:43:11.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49d8070940 0x7f49d815be30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:11.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d80ce0c0 0x7f49d8146c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:11.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49d815b790 0x7f49d8159f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:11.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d8162270 con 0x7f49d80ce0c0 2023-12-20T04:43:11.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49d815c370 con 0x7f49d8070940 2023-12-20T04:43:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.694+0000 7f49df92c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49d815cc80 con 0x7f49d815b790 2023-12-20T04:43:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49dcec7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d80ce0c0 0x7f49d8146c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49dcec7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d80ce0c0 0x7f49d8146c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36956/0 (socket says 172.21.15.32:36956) 2023-12-20T04:43:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49dcec7640 1 -- 172.21.15.32:0/2488524182 learned_addr learned my addr 172.21.15.32:0/2488524182 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49dd6c8640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49d8070940 0x7f49d815be30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49ddec9640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49d815b790 0x7f49d8159f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49dcec7640 1 -- 172.21.15.32:0/2488524182 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49d815b790 msgr2=0x7f49d8159f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49dcec7640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49d815b790 0x7f49d8159f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:11.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49dcec7640 1 -- 172.21.15.32:0/2488524182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49d8070940 msgr2=0x7f49d815be30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:11.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49dcec7640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49d8070940 0x7f49d815be30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:11.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49dcec7640 1 -- 172.21.15.32:0/2488524182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49d0009a70 con 0x7f49d80ce0c0 2023-12-20T04:43:11.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49dcec7640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d80ce0c0 0x7f49d8146c70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f49c800c530 tx=0x7f49c800c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:11.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49cdffb640 1 -- 172.21.15.32:0/2488524182 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8005220 con 0x7f49d80ce0c0 2023-12-20T04:43:11.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49cdffb640 1 -- 172.21.15.32:0/2488524182 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c8005380 con 0x7f49d80ce0c0 2023-12-20T04:43:11.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49cdffb640 1 -- 172.21.15.32:0/2488524182 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c8007270 con 0x7f49d80ce0c0 2023-12-20T04:43:11.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49df92c640 1 -- 172.21.15.32:0/2488524182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49d8159480 con 0x7f49d80ce0c0 2023-12-20T04:43:11.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49df92c640 1 -- 172.21.15.32:0/2488524182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49d8159a40 con 0x7f49d80ce0c0 2023-12-20T04:43:11.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49df92c640 1 -- 172.21.15.32:0/2488524182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f49d8159c60 con 0x7f49d80ce0c0 2023-12-20T04:43:11.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.698+0000 7f49cdffb640 1 -- 172.21.15.32:0/2488524182 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 29) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f49c80073d0 con 0x7f49d80ce0c0 2023-12-20T04:43:11.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.702+0000 7f49cdffb640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49bc065b20 0x7f49bc067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:11.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.702+0000 7f49cdffb640 1 -- 172.21.15.32:0/2488524182 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(68..68 src has 1..68) v4 ==== 6483+0+0 (secure 0 0 0) 0x7f49c8082e40 con 0x7f49d80ce0c0 2023-12-20T04:43:11.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.702+0000 7f49dd6c8640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49bc065b20 0x7f49bc067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.702+0000 7f49dd6c8640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49bc065b20 0x7f49bc067fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49d4000f70 tx=0x7f49d4009040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:11.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.706+0000 7f49cdffb640 1 -- 172.21.15.32:0/2488524182 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f49c8050490 con 0x7f49d80ce0c0 2023-12-20T04:43:11.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:11.862+0000 7f49df92c640 1 -- 172.21.15.32:0/2488524182 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f49d804e000 con 0x7f49d80ce0c0 2023-12-20T04:43:12.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.526+0000 7f49cdffb640 1 -- 172.21.15.32:0/2488524182 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f49c8010dc0 con 0x7f49d80ce0c0 2023-12-20T04:43:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49cdffb640 1 -- 172.21.15.32:0/2488524182 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v69) v1 ==== 181+0+0 (secure 0 0 0) 0x7f49c804cd70 con 0x7f49d80ce0c0 2023-12-20T04:43:12.528 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 -- 172.21.15.32:0/2488524182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49bc065b20 msgr2=0x7f49bc067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49bc065b20 0x7f49bc067fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f49d4000f70 tx=0x7f49d4009040 comp rx=0 tx=0).stop 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 -- 172.21.15.32:0/2488524182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d80ce0c0 msgr2=0x7f49d8146c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d80ce0c0 0x7f49d8146c70 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f49c800c530 tx=0x7f49c800c9f0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 -- 172.21.15.32:0/2488524182 shutdown_connections 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49bc065b20 0x7f49bc067fe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49d815b790 0x7f49d8159f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49d80ce0c0 0x7f49d8146c70 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 --2- 172.21.15.32:0/2488524182 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49d8070940 0x7f49d815be30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 -- 172.21.15.32:0/2488524182 >> 172.21.15.32:0/2488524182 conn(0x7f49d806a510 msgr2=0x7f49d81641f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 -- 172.21.15.32:0/2488524182 shutdown_connections 2023-12-20T04:43:12.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.530+0000 7f49df92c640 1 -- 172.21.15.32:0/2488524182 wait complete. 2023-12-20T04:43:12.561 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T04:43:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e2a62d640 1 Processor -- start 2023-12-20T04:43:12.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e2a62d640 1 -- start start 2023-12-20T04:43:12.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e2a62d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24158520 0x7f0e2406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:12.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e2a62d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0e240ca050 0x7f0e2415ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:12.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e2a62d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e240ca7f0 con 0x7f0e24070230 2023-12-20T04:43:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e2a62d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e240caad0 con 0x7f0e240ca050 2023-12-20T04:43:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e2a62d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e240cadb0 con 0x7f0e24158520 2023-12-20T04:43:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e23fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24158520 0x7f0e2406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e23fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24158520 0x7f0e2406dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:48438/0 (socket says 172.21.15.32:48438) 2023-12-20T04:43:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e23fff640 1 -- 172.21.15.32:0/705747758 learned_addr learned my addr 172.21.15.32:0/705747758 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e28bca640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0e24070230 0x7f0e2415b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:45294/0 (socket says 172.21.15.32:45294) 2023-12-20T04:43:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e237fe640 1 --2- 172.21.15.32:0/705747758 >> v2:172.21.15.39:3300/0 conn(0x7f0e240ca050 0x7f0e2415ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:12.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e23fff640 1 -- 172.21.15.32:0/705747758 >> v2:172.21.15.39:3300/0 conn(0x7f0e240ca050 msgr2=0x7f0e2415ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e23fff640 1 --2- 172.21.15.32:0/705747758 >> v2:172.21.15.39:3300/0 conn(0x7f0e240ca050 0x7f0e2415ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e23fff640 1 -- 172.21.15.32:0/705747758 >> v1:172.21.15.39:6789/0 conn(0x7f0e24070230 legacy=0x7f0e2415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e22ffd640 1 -- 172.21.15.32:0/705747758 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4072630978 0 0) 0x7f0e240ca7f0 con 0x7f0e24070230 2023-12-20T04:43:12.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.714+0000 7f0e23fff640 1 -- 172.21.15.32:0/705747758 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e2415e780 con 0x7f0e24158520 2023-12-20T04:43:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e23fff640 1 --2- 172.21.15.32:0/705747758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24158520 0x7f0e2406dcb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009870 tx=0x7f0e1401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=923c6a0dafbdf1f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e22ffd640 1 -- 172.21.15.32:0/705747758 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140a95f0 con 0x7f0e24158520 2023-12-20T04:43:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e22ffd640 1 -- 172.21.15.32:0/705747758 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e140a9750 con 0x7f0e24158520 2023-12-20T04:43:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e22ffd640 1 -- 172.21.15.32:0/705747758 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140a9ae0 con 0x7f0e24158520 2023-12-20T04:43:12.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e2a62d640 1 -- 172.21.15.32:0/705747758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24158520 msgr2=0x7f0e2406dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e2a62d640 1 --2- 172.21.15.32:0/705747758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24158520 0x7f0e2406dcb0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f0e14009870 tx=0x7f0e1401c510 comp rx=0 tx=0).stop 2023-12-20T04:43:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e2a62d640 1 -- 172.21.15.32:0/705747758 shutdown_connections 2023-12-20T04:43:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e2a62d640 1 --2- 172.21.15.32:0/705747758 >> v2:172.21.15.39:3300/0 conn(0x7f0e240ca050 0x7f0e2415ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e2a62d640 1 --2- 172.21.15.32:0/705747758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24158520 0x7f0e2406dcb0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e2a62d640 1 -- 172.21.15.32:0/705747758 >> 172.21.15.32:0/705747758 conn(0x7f0e2406a510 msgr2=0x7f0e240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e2a62d640 1 -- 172.21.15.32:0/705747758 shutdown_connections 2023-12-20T04:43:12.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.718+0000 7f0e2a62d640 1 -- 172.21.15.32:0/705747758 wait complete. 2023-12-20T04:43:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e2a62d640 1 Processor -- start 2023-12-20T04:43:12.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e2a62d640 1 -- start start 2023-12-20T04:43:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e2a62d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24070230 0x7f0e241542f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e2a62d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e240ca050 0x7f0e2415ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e2a62d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24158520 0x7f0e241461e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e2a62d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e2415d370 con 0x7f0e24158520 2023-12-20T04:43:12.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e2a62d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24154990 con 0x7f0e240ca050 2023-12-20T04:43:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e2a62d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e24154c70 con 0x7f0e24070230 2023-12-20T04:43:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e23fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24070230 0x7f0e241542f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e28bca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24158520 0x7f0e241461e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e23fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24070230 0x7f0e241542f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:48452/0 (socket says 172.21.15.32:48452) 2023-12-20T04:43:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e23fff640 1 -- 172.21.15.32:0/1059864429 learned_addr learned my addr 172.21.15.32:0/1059864429 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:12.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e237fe640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e240ca050 0x7f0e2415ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:12.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e23fff640 1 -- 172.21.15.32:0/1059864429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e240ca050 msgr2=0x7f0e2415ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e23fff640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e240ca050 0x7f0e2415ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e23fff640 1 -- 172.21.15.32:0/1059864429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24158520 msgr2=0x7f0e241461e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e23fff640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24158520 0x7f0e241461e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e23fff640 1 -- 172.21.15.32:0/1059864429 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e140094d0 con 0x7f0e24070230 2023-12-20T04:43:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e23fff640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24070230 0x7f0e241542f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e1401a140 tx=0x7f0e140192b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e20ff9640 1 -- 172.21.15.32:0/1059864429 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140ba410 con 0x7f0e24070230 2023-12-20T04:43:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e20ff9640 1 -- 172.21.15.32:0/1059864429 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0e140ba570 con 0x7f0e24070230 2023-12-20T04:43:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e20ff9640 1 -- 172.21.15.32:0/1059864429 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0e140ba920 con 0x7f0e24070230 2023-12-20T04:43:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e2a62d640 1 -- 172.21.15.32:0/1059864429 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e24154f50 con 0x7f0e24070230 2023-12-20T04:43:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.722+0000 7f0e2a62d640 1 -- 172.21.15.32:0/1059864429 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e24155510 con 0x7f0e24070230 2023-12-20T04:43:12.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.726+0000 7f0e2a62d640 1 -- 172.21.15.32:0/1059864429 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e2404e000 con 0x7f0e24070230 2023-12-20T04:43:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.726+0000 7f0e20ff9640 1 -- 172.21.15.32:0/1059864429 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0e140b13f0 con 0x7f0e24070230 2023-12-20T04:43:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.726+0000 7f0e20ff9640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e10065b80 0x7f0e10068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.726+0000 7f0e20ff9640 1 -- 172.21.15.32:0/1059864429 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(69..69 src has 1..69) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f0e1411d7e0 con 0x7f0e24070230 2023-12-20T04:43:12.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.726+0000 7f0e237fe640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e10065b80 0x7f0e10068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:12.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.726+0000 7f0e237fe640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e10065b80 0x7f0e10068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e240cbc80 tx=0x7f0e18009190 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:12.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.730+0000 7f0e20ff9640 1 -- 172.21.15.32:0/1059864429 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0e140eaf70 con 0x7f0e24070230 2023-12-20T04:43:12.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.886+0000 7f0e2a62d640 1 -- 172.21.15.32:0/1059864429 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f0e2415f230 con 0x7f0e24070230 2023-12-20T04:43:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.914+0000 7f0e20ff9640 1 -- 172.21.15.32:0/1059864429 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v70) v1 ==== 169+0+0 (secure 0 0 0) 0x7f0e140ee590 con 0x7f0e24070230 2023-12-20T04:43:12.915 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T04:43:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 -- 172.21.15.32:0/1059864429 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e10065b80 msgr2=0x7f0e10068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e10065b80 0x7f0e10068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0e240cbc80 tx=0x7f0e18009190 comp rx=0 tx=0).stop 2023-12-20T04:43:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 -- 172.21.15.32:0/1059864429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24070230 msgr2=0x7f0e241542f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24070230 0x7f0e241542f0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f0e1401a140 tx=0x7f0e140192b0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 -- 172.21.15.32:0/1059864429 shutdown_connections 2023-12-20T04:43:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0e10065b80 0x7f0e10068040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e24158520 0x7f0e241461e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e240ca050 0x7f0e2415ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 --2- 172.21.15.32:0/1059864429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e24070230 0x7f0e241542f0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 -- 172.21.15.32:0/1059864429 >> 172.21.15.32:0/1059864429 conn(0x7f0e2406a510 msgr2=0x7f0e240b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 -- 172.21.15.32:0/1059864429 shutdown_connections 2023-12-20T04:43:12.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:12.918+0000 7f0e2a62d640 1 -- 172.21.15.32:0/1059864429 wait complete. 2023-12-20T04:43:12.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T04:43:13.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c7812640 1 Processor -- start 2023-12-20T04:43:13.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c7812640 1 -- start start 2023-12-20T04:43:13.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c7812640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7c00d3300 0x7fb7c00d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c7812640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c00d2650 con 0x7fb7c015ce60 2023-12-20T04:43:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c7812640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7c00d1a40 con 0x7fb7c00d1300 2023-12-20T04:43:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c7812640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7c00d1d20 con 0x7fb7c00d3300 2023-12-20T04:43:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c4dad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7c00d3300 0x7fb7c00d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c4dad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7c00d3300 0x7fb7c00d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36978/0 (socket says 172.21.15.32:36978) 2023-12-20T04:43:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c4dad640 1 -- 172.21.15.32:0/293108752 learned_addr learned my addr 172.21.15.32:0/293108752 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c55ae640 1 --1- 172.21.15.32:0/293108752 >> v1:172.21.15.32:6789/0 conn(0x7fb7c015ce60 0x7fb7c00d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60548/0 (socket says 172.21.15.32:60548) 2023-12-20T04:43:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c4dad640 1 -- 172.21.15.32:0/293108752 >> v1:172.21.15.39:6789/0 conn(0x7fb7c00d1300 legacy=0x7fb7c00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c4dad640 1 -- 172.21.15.32:0/293108752 >> v1:172.21.15.32:6789/0 conn(0x7fb7c015ce60 legacy=0x7fb7c00d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.086+0000 7fb7c4dad640 1 -- 172.21.15.32:0/293108752 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7c00d2000 con 0x7fb7c00d3300 2023-12-20T04:43:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.090+0000 7fb7c4dad640 1 --2- 172.21.15.32:0/293108752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c00d3300 0x7fb7c00d0dc0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009870 tx=0x7fb7b001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9feeef5b7fcc7ae server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:13.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.090+0000 7fb7b7fff640 1 -- 172.21.15.32:0/293108752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00ac410 con 0x7fb7c00d3300 2023-12-20T04:43:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.090+0000 7fb7b7fff640 1 -- 172.21.15.32:0/293108752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7b00ac570 con 0x7fb7c00d3300 2023-12-20T04:43:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.090+0000 7fb7b7fff640 1 -- 172.21.15.32:0/293108752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7b00ac900 con 0x7fb7c00d3300 2023-12-20T04:43:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.090+0000 7fb7c7812640 1 -- 172.21.15.32:0/293108752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c00d3300 msgr2=0x7fb7c00d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.090+0000 7fb7c7812640 1 --2- 172.21.15.32:0/293108752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c00d3300 0x7fb7c00d0dc0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7fb7b0009870 tx=0x7fb7b001c500 comp rx=0 tx=0).stop 2023-12-20T04:43:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.090+0000 7fb7c7812640 1 -- 172.21.15.32:0/293108752 shutdown_connections 2023-12-20T04:43:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.090+0000 7fb7c7812640 1 --2- 172.21.15.32:0/293108752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c00d3300 0x7fb7c00d0dc0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.090+0000 7fb7c7812640 1 -- 172.21.15.32:0/293108752 >> 172.21.15.32:0/293108752 conn(0x7fb7c006b5e0 msgr2=0x7fb7c006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.090+0000 7fb7c7812640 1 -- 172.21.15.32:0/293108752 shutdown_connections 2023-12-20T04:43:13.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.090+0000 7fb7c7812640 1 -- 172.21.15.32:0/293108752 wait complete. 2023-12-20T04:43:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c7812640 1 Processor -- start 2023-12-20T04:43:13.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c7812640 1 -- start start 2023-12-20T04:43:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c7812640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c00d1300 0x7fb7c0154ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c7812640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c00d3300 0x7fb7c00b7340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c7812640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7c015ce60 0x7fb7c0141910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c7812640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c0153ab0 con 0x7fb7c00d1300 2023-12-20T04:43:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c7812640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00c7230 con 0x7fb7c00d3300 2023-12-20T04:43:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c7812640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7c00c7510 con 0x7fb7c015ce60 2023-12-20T04:43:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c55ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c00d1300 0x7fb7c0154ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c55ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c00d1300 0x7fb7c0154ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36994/0 (socket says 172.21.15.32:36994) 2023-12-20T04:43:13.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c55ae640 1 -- 172.21.15.32:0/1034944194 learned_addr learned my addr 172.21.15.32:0/1034944194 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c4dad640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c00d3300 0x7fb7c00b7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c55ae640 1 -- 172.21.15.32:0/1034944194 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7c015ce60 msgr2=0x7fb7c0141910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c55ae640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7c015ce60 0x7fb7c0141910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c55ae640 1 -- 172.21.15.32:0/1034944194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c00d3300 msgr2=0x7fb7c00b7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c55ae640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c00d3300 0x7fb7c00b7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c55ae640 1 -- 172.21.15.32:0/1034944194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7b00094d0 con 0x7fb7c00d1300 2023-12-20T04:43:13.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c4dad640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c00d3300 0x7fb7c00b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:43:13.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c55ae640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c00d1300 0x7fb7c0154ae0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00c430 tx=0x7fb7bc00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:13.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7b5ffb640 1 -- 172.21.15.32:0/1034944194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc0074e0 con 0x7fb7c00d1300 2023-12-20T04:43:13.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7b5ffb640 1 -- 172.21.15.32:0/1034944194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7bc007640 con 0x7fb7c00d1300 2023-12-20T04:43:13.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7b5ffb640 1 -- 172.21.15.32:0/1034944194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7bc004a10 con 0x7fb7c00d1300 2023-12-20T04:43:13.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c7812640 1 -- 172.21.15.32:0/1034944194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7c0153740 con 0x7fb7c00d1300 2023-12-20T04:43:13.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.094+0000 7fb7c7812640 1 -- 172.21.15.32:0/1034944194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7c00b7880 con 0x7fb7c00d1300 2023-12-20T04:43:13.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.098+0000 7fb7c7812640 1 -- 172.21.15.32:0/1034944194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7c00b7a70 con 0x7fb7c00d1300 2023-12-20T04:43:13.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.098+0000 7fb7b5ffb640 1 -- 172.21.15.32:0/1034944194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb7bc004b70 con 0x7fb7c00d1300 2023-12-20T04:43:13.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.098+0000 7fb7b5ffb640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb7a4065b80 0x7fb7a4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.098+0000 7fb7b5ffb640 1 -- 172.21.15.32:0/1034944194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb7bc082c90 con 0x7fb7c00d1300 2023-12-20T04:43:13.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.098+0000 7fb7c4dad640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb7a4065b80 0x7fb7a4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.098+0000 7fb7c4dad640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb7a4065b80 0x7fb7a4068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00ae000 tx=0x7fb7b00a6040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:13.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.102+0000 7fb7b5ffb640 1 -- 172.21.15.32:0/1034944194 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb7bc050650 con 0x7fb7c00d1300 2023-12-20T04:43:13.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.262+0000 7fb7c7812640 1 -- 172.21.15.32:0/1034944194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fb7c004e000 con 0x7fb7c00d1300 2023-12-20T04:43:13.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.266+0000 7fb7b5ffb640 1 -- 172.21.15.32:0/1034944194 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v19) v1 ==== 120+0+0 (secure 0 0 0) 0x7fb7bc053c70 con 0x7fb7c00d1300 2023-12-20T04:43:13.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 -- 172.21.15.32:0/1034944194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb7a4065b80 msgr2=0x7fb7a4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb7a4065b80 0x7fb7a4068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb7b00ae000 tx=0x7fb7b00a6040 comp rx=0 tx=0).stop 2023-12-20T04:43:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 -- 172.21.15.32:0/1034944194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c00d1300 msgr2=0x7fb7c0154ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c00d1300 0x7fb7c0154ae0 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7fb7bc00c430 tx=0x7fb7bc00c8f0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 -- 172.21.15.32:0/1034944194 shutdown_connections 2023-12-20T04:43:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb7a4065b80 0x7fb7a4068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7c015ce60 0x7fb7c0141910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7c00d3300 0x7fb7c00b7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 --2- 172.21.15.32:0/1034944194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7c00d1300 0x7fb7c0154ae0 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 -- 172.21.15.32:0/1034944194 >> 172.21.15.32:0/1034944194 conn(0x7fb7c006b5e0 msgr2=0x7fb7c00b3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 -- 172.21.15.32:0/1034944194 shutdown_connections 2023-12-20T04:43:13.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.270+0000 7fb7c7812640 1 -- 172.21.15.32:0/1034944194 wait complete. 2023-12-20T04:43:13.288 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T04:43:13.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.434+0000 7f530b2e5640 1 Processor -- start 2023-12-20T04:43:13.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.434+0000 7f530b2e5640 1 -- start start 2023-12-20T04:43:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f530b2e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5304164670 0x7f53040ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f530b2e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f530406da20 0x7f53040cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f530b2e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5304070230 0x7f5304070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f530b2e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53040cb0b0 con 0x7f530406da20 2023-12-20T04:43:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f530b2e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5304070ba0 con 0x7f5304164670 2023-12-20T04:43:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f530b2e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53040ca810 con 0x7f5304070230 2023-12-20T04:43:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f5308880640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f530406da20 0x7f53040cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f5308880640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f530406da20 0x7f53040cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37006/0 (socket says 172.21.15.32:37006) 2023-12-20T04:43:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f5308880640 1 -- 172.21.15.32:0/411940040 learned_addr learned my addr 172.21.15.32:0/411940040 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f5309081640 1 --2- 172.21.15.32:0/411940040 >> v2:172.21.15.39:3300/0 conn(0x7f5304164670 0x7f53040ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f5309882640 1 --2- 172.21.15.32:0/411940040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5304070230 0x7f5304070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f5308880640 1 -- 172.21.15.32:0/411940040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5304070230 msgr2=0x7f5304070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f5308880640 1 --2- 172.21.15.32:0/411940040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5304070230 0x7f5304070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f5308880640 1 -- 172.21.15.32:0/411940040 >> v2:172.21.15.39:3300/0 conn(0x7f5304164670 msgr2=0x7f53040ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f5308880640 1 --2- 172.21.15.32:0/411940040 >> v2:172.21.15.39:3300/0 conn(0x7f5304164670 0x7f53040ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f5308880640 1 -- 172.21.15.32:0/411940040 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53040caaf0 con 0x7f530406da20 2023-12-20T04:43:13.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.438+0000 7f5308880640 1 --2- 172.21.15.32:0/411940040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530406da20 0x7f53040cc3b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f530406e430 tx=0x7f52f401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d7c2babb03e1407 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:13.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f52fbfff640 1 -- 172.21.15.32:0/411940040 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40ab600 con 0x7f530406da20 2023-12-20T04:43:13.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f52fbfff640 1 -- 172.21.15.32:0/411940040 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f40a7050 con 0x7f530406da20 2023-12-20T04:43:13.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f52fbfff640 1 -- 172.21.15.32:0/411940040 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40ab980 con 0x7f530406da20 2023-12-20T04:43:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 -- 172.21.15.32:0/411940040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530406da20 msgr2=0x7f53040cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 --2- 172.21.15.32:0/411940040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530406da20 0x7f53040cc3b0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f530406e430 tx=0x7f52f401c3b0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 -- 172.21.15.32:0/411940040 shutdown_connections 2023-12-20T04:43:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 --2- 172.21.15.32:0/411940040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5304070230 0x7f5304070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 --2- 172.21.15.32:0/411940040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530406da20 0x7f53040cc3b0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 --2- 172.21.15.32:0/411940040 >> v2:172.21.15.39:3300/0 conn(0x7f5304164670 0x7f53040ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 -- 172.21.15.32:0/411940040 >> 172.21.15.32:0/411940040 conn(0x7f530406a510 msgr2=0x7f53040ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:13.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 -- 172.21.15.32:0/411940040 shutdown_connections 2023-12-20T04:43:13.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 -- 172.21.15.32:0/411940040 wait complete. 2023-12-20T04:43:13.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 Processor -- start 2023-12-20T04:43:13.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 -- start start 2023-12-20T04:43:13.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f530406da20 0x7f530415cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5304070230 0x7f53041547f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5304164670 0x7f5304154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5304155d60 con 0x7f5304070230 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5304156bc0 con 0x7f530406da20 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.442+0000 7f530b2e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5304157f30 con 0x7f5304164670 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f5308880640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5304070230 0x7f53041547f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f5308880640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5304070230 0x7f53041547f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37022/0 (socket says 172.21.15.32:37022) 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f5308880640 1 -- 172.21.15.32:0/200106082 learned_addr learned my addr 172.21.15.32:0/200106082 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f5309081640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f530406da20 0x7f530415cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f5309882640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5304164670 0x7f5304154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f5308880640 1 -- 172.21.15.32:0/200106082 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5304164670 msgr2=0x7f5304154d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f5308880640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5304164670 0x7f5304154d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f5308880640 1 -- 172.21.15.32:0/200106082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f530406da20 msgr2=0x7f530415cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f5308880640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f530406da20 0x7f530415cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f5308880640 1 -- 172.21.15.32:0/200106082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f40094d0 con 0x7f5304070230 2023-12-20T04:43:13.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f5308880640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5304070230 0x7f53041547f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f52f401aa60 tx=0x7f52f4019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:13.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f52f9ffb640 1 -- 172.21.15.32:0/200106082 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40b4620 con 0x7f5304070230 2023-12-20T04:43:13.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f52f9ffb640 1 -- 172.21.15.32:0/200106082 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f40b4780 con 0x7f5304070230 2023-12-20T04:43:13.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f52f9ffb640 1 -- 172.21.15.32:0/200106082 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f40b4b30 con 0x7f5304070230 2023-12-20T04:43:13.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f530b2e5640 1 -- 172.21.15.32:0/200106082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53040c82d0 con 0x7f5304070230 2023-12-20T04:43:13.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f530b2e5640 1 -- 172.21.15.32:0/200106082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5304155fe0 con 0x7f5304070230 2023-12-20T04:43:13.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f530b2e5640 1 -- 172.21.15.32:0/200106082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53041561d0 con 0x7f5304070230 2023-12-20T04:43:13.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.446+0000 7f52f9ffb640 1 -- 172.21.15.32:0/200106082 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f52f40b2070 con 0x7f5304070230 2023-12-20T04:43:13.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.450+0000 7f52f9ffb640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52e8065b80 0x7f52e8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.450+0000 7f52f9ffb640 1 -- 172.21.15.32:0/200106082 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f52f411ced0 con 0x7f5304070230 2023-12-20T04:43:13.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.450+0000 7f5309081640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52e8065b80 0x7f52e8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.450+0000 7f5309081640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52e8065b80 0x7f52e8068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5304157d90 tx=0x7f5300008040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:13.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.450+0000 7f52f9ffb640 1 -- 172.21.15.32:0/200106082 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f52f40ea790 con 0x7f5304070230 2023-12-20T04:43:13.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.610+0000 7f530b2e5640 1 -- 172.21.15.32:0/200106082 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f530404e000 con 0x7f5304070230 2023-12-20T04:43:13.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f52f9ffb640 1 -- 172.21.15.32:0/200106082 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v20) v1 ==== 115+0+0 (secure 0 0 0) 0x7f52f40eddb0 con 0x7f5304070230 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 -- 172.21.15.32:0/200106082 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52e8065b80 msgr2=0x7f52e8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52e8065b80 0x7f52e8068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5304157d90 tx=0x7f5300008040 comp rx=0 tx=0).stop 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 -- 172.21.15.32:0/200106082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5304070230 msgr2=0x7f53041547f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5304070230 0x7f53041547f0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f52f401aa60 tx=0x7f52f4019150 comp rx=0 tx=0).stop 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 -- 172.21.15.32:0/200106082 shutdown_connections 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52e8065b80 0x7f52e8068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5304164670 0x7f5304154d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5304070230 0x7f53041547f0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 --2- 172.21.15.32:0/200106082 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f530406da20 0x7f530415cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 -- 172.21.15.32:0/200106082 >> 172.21.15.32:0/200106082 conn(0x7f530406a510 msgr2=0x7f53040c4e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 -- 172.21.15.32:0/200106082 shutdown_connections 2023-12-20T04:43:13.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.618+0000 7f530b2e5640 1 -- 172.21.15.32:0/200106082 wait complete. 2023-12-20T04:43:13.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T04:43:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.782+0000 7fb03c360640 1 Processor -- start 2023-12-20T04:43:13.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.782+0000 7fb03c360640 1 -- start start 2023-12-20T04:43:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03c360640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb034157480 0x7fb0340c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03c360640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb0340cd4c0 0x7fb0340b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03c360640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0340b1100 con 0x7fb0340b0760 2023-12-20T04:43:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03c360640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0340cde20 con 0x7fb0340cd4c0 2023-12-20T04:43:13.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03c360640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0340c8e30 con 0x7fb034157480 2023-12-20T04:43:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03a0fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb034157480 0x7fb0340c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb0398fb640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb0340b0760 0x7fb03415d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:45306/0 (socket says 172.21.15.32:45306) 2023-12-20T04:43:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03a0fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb034157480 0x7fb0340c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:48480/0 (socket says 172.21.15.32:48480) 2023-12-20T04:43:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb0398fb640 1 -- 172.21.15.32:0/2284435263 learned_addr learned my addr 172.21.15.32:0/2284435263 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03a8fd640 1 --2- 172.21.15.32:0/2284435263 >> v2:172.21.15.39:3300/0 conn(0x7fb0340cd4c0 0x7fb0340b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03a0fc640 1 -- 172.21.15.32:0/2284435263 >> v2:172.21.15.39:3300/0 conn(0x7fb0340cd4c0 msgr2=0x7fb0340b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03a0fc640 1 --2- 172.21.15.32:0/2284435263 >> v2:172.21.15.39:3300/0 conn(0x7fb0340cd4c0 0x7fb0340b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03a0fc640 1 -- 172.21.15.32:0/2284435263 >> v1:172.21.15.39:6789/0 conn(0x7fb0340b0760 legacy=0x7fb03415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.786+0000 7fb03a0fc640 1 -- 172.21.15.32:0/2284435263 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0340ce070 con 0x7fb034157480 2023-12-20T04:43:13.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb03a0fc640 1 --2- 172.21.15.32:0/2284435263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb034157480 0x7fb0340c9840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb0300097c0 tx=0x7fb03001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e7af695bbba206bc server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:13.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb0390fa640 1 -- 172.21.15.32:0/2284435263 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0300ab600 con 0x7fb034157480 2023-12-20T04:43:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb0390fa640 1 -- 172.21.15.32:0/2284435263 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0300a7050 con 0x7fb034157480 2023-12-20T04:43:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb0390fa640 1 -- 172.21.15.32:0/2284435263 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0300ab980 con 0x7fb034157480 2023-12-20T04:43:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb03c360640 1 -- 172.21.15.32:0/2284435263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb034157480 msgr2=0x7fb0340c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb03c360640 1 --2- 172.21.15.32:0/2284435263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb034157480 0x7fb0340c9840 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb0300097c0 tx=0x7fb03001c3c0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb03c360640 1 -- 172.21.15.32:0/2284435263 shutdown_connections 2023-12-20T04:43:13.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb03c360640 1 --2- 172.21.15.32:0/2284435263 >> v2:172.21.15.39:3300/0 conn(0x7fb0340cd4c0 0x7fb0340b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb03c360640 1 --2- 172.21.15.32:0/2284435263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb034157480 0x7fb0340c9840 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb03c360640 1 -- 172.21.15.32:0/2284435263 >> 172.21.15.32:0/2284435263 conn(0x7fb03406a510 msgr2=0x7fb0340ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb03c360640 1 -- 172.21.15.32:0/2284435263 shutdown_connections 2023-12-20T04:43:13.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb03c360640 1 -- 172.21.15.32:0/2284435263 wait complete. 2023-12-20T04:43:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb03c360640 1 Processor -- start 2023-12-20T04:43:13.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.790+0000 7fb03c360640 1 -- start start 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03c360640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0340b0760 0x7fb03414fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03c360640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0340cd4c0 0x7fb03415d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03c360640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb034157480 0x7fb03414bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03c360640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb03415d5f0 con 0x7fb034157480 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03c360640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb034150760 con 0x7fb0340cd4c0 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03c360640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb034150480 con 0x7fb0340b0760 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03a8fd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb034157480 0x7fb03414bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03a8fd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb034157480 0x7fb03414bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37032/0 (socket says 172.21.15.32:37032) 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03a8fd640 1 -- 172.21.15.32:0/760930674 learned_addr learned my addr 172.21.15.32:0/760930674 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03a0fc640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0340b0760 0x7fb03414fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb0398fb640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0340cd4c0 0x7fb03415d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03a8fd640 1 -- 172.21.15.32:0/760930674 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0340b0760 msgr2=0x7fb03414fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03a8fd640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0340b0760 0x7fb03414fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03a8fd640 1 -- 172.21.15.32:0/760930674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0340cd4c0 msgr2=0x7fb03415d0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03a8fd640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0340cd4c0 0x7fb03415d0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03a8fd640 1 -- 172.21.15.32:0/760930674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb030009420 con 0x7fb034157480 2023-12-20T04:43:13.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03a8fd640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb034157480 0x7fb03414bf10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00ef90 tx=0x7fb02c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:13.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb02affd640 1 -- 172.21.15.32:0/760930674 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c009b20 con 0x7fb034157480 2023-12-20T04:43:13.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb02affd640 1 -- 172.21.15.32:0/760930674 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb02c00f040 con 0x7fb034157480 2023-12-20T04:43:13.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03c360640 1 -- 172.21.15.32:0/760930674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb034151030 con 0x7fb034157480 2023-12-20T04:43:13.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb02affd640 1 -- 172.21.15.32:0/760930674 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb02c0146c0 con 0x7fb034157480 2023-12-20T04:43:13.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03c360640 1 -- 172.21.15.32:0/760930674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0341515f0 con 0x7fb034157480 2023-12-20T04:43:13.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb03c360640 1 -- 172.21.15.32:0/760930674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb03404e000 con 0x7fb034157480 2023-12-20T04:43:13.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.794+0000 7fb02affd640 1 -- 172.21.15.32:0/760930674 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb02c014820 con 0x7fb034157480 2023-12-20T04:43:13.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.798+0000 7fb02affd640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb018065b80 0x7fb018068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:13.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.798+0000 7fb02affd640 1 -- 172.21.15.32:0/760930674 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fb02c083ff0 con 0x7fb034157480 2023-12-20T04:43:13.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.798+0000 7fb03a0fc640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb018065b80 0x7fb018068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:13.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.798+0000 7fb03a0fc640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb018065b80 0x7fb018068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb0300ad000 tx=0x7fb0300a6000 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:13.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.802+0000 7fb02affd640 1 -- 172.21.15.32:0/760930674 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb02c0518b0 con 0x7fb034157480 2023-12-20T04:43:13.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.970+0000 7fb03c360640 1 -- 172.21.15.32:0/760930674 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb034164340 con 0x7fb034157480 2023-12-20T04:43:13.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb02affd640 1 -- 172.21.15.32:0/760930674 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v21) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb02c0180a0 con 0x7fb034157480 2023-12-20T04:43:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 -- 172.21.15.32:0/760930674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb018065b80 msgr2=0x7fb018068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb018065b80 0x7fb018068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fb0300ad000 tx=0x7fb0300a6000 comp rx=0 tx=0).stop 2023-12-20T04:43:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 -- 172.21.15.32:0/760930674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb034157480 msgr2=0x7fb03414bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb034157480 0x7fb03414bf10 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7fb02c00ef90 tx=0x7fb02c00c490 comp rx=0 tx=0).stop 2023-12-20T04:43:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 -- 172.21.15.32:0/760930674 shutdown_connections 2023-12-20T04:43:13.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb018065b80 0x7fb018068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb034157480 0x7fb03414bf10 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0340cd4c0 0x7fb03415d0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 --2- 172.21.15.32:0/760930674 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0340b0760 0x7fb03414fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 -- 172.21.15.32:0/760930674 >> 172.21.15.32:0/760930674 conn(0x7fb03406a510 msgr2=0x7fb0340c3a20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 -- 172.21.15.32:0/760930674 shutdown_connections 2023-12-20T04:43:13.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:13.978+0000 7fb03c360640 1 -- 172.21.15.32:0/760930674 wait complete. 2023-12-20T04:43:13.997 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule' 2023-12-20T04:43:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c2b266640 1 Processor -- start 2023-12-20T04:43:14.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c2b266640 1 -- start start 2023-12-20T04:43:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c2b266640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c240ae010 0x7f1c24071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c2b266640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c2406cf10 0x7f1c240719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c2b266640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1c24164ef0 0x7f1c240701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c2b266640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c240ae3f0 con 0x7f1c240ae010 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c2b266640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c24161800 con 0x7f1c24164ef0 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c2b266640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24165380 con 0x7f1c2406cf10 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c29002640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c240ae010 0x7f1c24071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c29002640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c240ae010 0x7f1c24071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37042/0 (socket says 172.21.15.32:37042) 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c29002640 1 -- 172.21.15.32:0/1586658462 learned_addr learned my addr 172.21.15.32:0/1586658462 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c28801640 1 --2- 172.21.15.32:0/1586658462 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c2406cf10 0x7f1c240719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c29803640 1 --2- 172.21.15.32:0/1586658462 >> v2:172.21.15.39:3300/0 conn(0x7f1c24164ef0 0x7f1c240701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c29002640 1 -- 172.21.15.32:0/1586658462 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c2406cf10 msgr2=0x7f1c240719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c29002640 1 --2- 172.21.15.32:0/1586658462 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c2406cf10 0x7f1c240719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c29002640 1 -- 172.21.15.32:0/1586658462 >> v2:172.21.15.39:3300/0 conn(0x7f1c24164ef0 msgr2=0x7f1c240701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c29002640 1 --2- 172.21.15.32:0/1586658462 >> v2:172.21.15.39:3300/0 conn(0x7f1c24164ef0 0x7f1c240701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c29002640 1 -- 172.21.15.32:0/1586658462 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c24165660 con 0x7f1c240ae010 2023-12-20T04:43:14.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.142+0000 7f1c29803640 1 --2- 172.21.15.32:0/1586658462 >> v2:172.21.15.39:3300/0 conn(0x7f1c24164ef0 0x7f1c240701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:14.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.146+0000 7f1c29002640 1 --2- 172.21.15.32:0/1586658462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c240ae010 0x7f1c24071470 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1c240cab70 tx=0x7f1c2001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7fda076d86fbf42 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.146+0000 7f1c1bfff640 1 -- 172.21.15.32:0/1586658462 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200ab600 con 0x7f1c240ae010 2023-12-20T04:43:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.146+0000 7f1c1bfff640 1 -- 172.21.15.32:0/1586658462 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c200ab760 con 0x7f1c240ae010 2023-12-20T04:43:14.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.146+0000 7f1c1bfff640 1 -- 172.21.15.32:0/1586658462 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c200abaf0 con 0x7f1c240ae010 2023-12-20T04:43:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.146+0000 7f1c2b266640 1 -- 172.21.15.32:0/1586658462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c240ae010 msgr2=0x7f1c24071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.146+0000 7f1c2b266640 1 --2- 172.21.15.32:0/1586658462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c240ae010 0x7f1c24071470 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f1c240cab70 tx=0x7f1c2001c440 comp rx=0 tx=0).stop 2023-12-20T04:43:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.146+0000 7f1c2b266640 1 -- 172.21.15.32:0/1586658462 shutdown_connections 2023-12-20T04:43:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.146+0000 7f1c2b266640 1 --2- 172.21.15.32:0/1586658462 >> v2:172.21.15.39:3300/0 conn(0x7f1c24164ef0 0x7f1c240701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.146+0000 7f1c2b266640 1 --2- 172.21.15.32:0/1586658462 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c2406cf10 0x7f1c240719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.146+0000 7f1c2b266640 1 --2- 172.21.15.32:0/1586658462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c240ae010 0x7f1c24071470 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.146+0000 7f1c2b266640 1 -- 172.21.15.32:0/1586658462 >> 172.21.15.32:0/1586658462 conn(0x7f1c2406a320 msgr2=0x7f1c240ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:14.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c2b266640 1 -- 172.21.15.32:0/1586658462 shutdown_connections 2023-12-20T04:43:14.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c2b266640 1 -- 172.21.15.32:0/1586658462 wait complete. 2023-12-20T04:43:14.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c2b266640 1 Processor -- start 2023-12-20T04:43:14.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c2b266640 1 -- start start 2023-12-20T04:43:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c2b266640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c240ae010 0x7f1c24150c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c2b266640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c24163010 0x7f1c24163490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c2b266640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c24158c50 0x7f1c241590d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c2b266640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24164750 con 0x7f1c24158c50 2023-12-20T04:43:14.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c2b266640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24159610 con 0x7f1c240ae010 2023-12-20T04:43:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c2b266640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c24163b40 con 0x7f1c24163010 2023-12-20T04:43:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c29803640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c24158c50 0x7f1c241590d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c29803640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c24158c50 0x7f1c241590d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37056/0 (socket says 172.21.15.32:37056) 2023-12-20T04:43:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c29803640 1 -- 172.21.15.32:0/1971560852 learned_addr learned my addr 172.21.15.32:0/1971560852 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c28801640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c24163010 0x7f1c24163490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c29002640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c240ae010 0x7f1c24150c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c29803640 1 -- 172.21.15.32:0/1971560852 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c24163010 msgr2=0x7f1c24163490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c29803640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c24163010 0x7f1c24163490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c29803640 1 -- 172.21.15.32:0/1971560852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c240ae010 msgr2=0x7f1c24150c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c29803640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c240ae010 0x7f1c24150c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c29803640 1 -- 172.21.15.32:0/1971560852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c20009470 con 0x7f1c24158c50 2023-12-20T04:43:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.150+0000 7f1c29803640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c24158c50 0x7f1c241590d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c007df0 tx=0x7f1c1c00d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.154+0000 7f1c19ffb640 1 -- 172.21.15.32:0/1971560852 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0039a0 con 0x7f1c24158c50 2023-12-20T04:43:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.154+0000 7f1c19ffb640 1 -- 172.21.15.32:0/1971560852 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c00f040 con 0x7f1c24158c50 2023-12-20T04:43:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.154+0000 7f1c19ffb640 1 -- 172.21.15.32:0/1971560852 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c003d40 con 0x7f1c24158c50 2023-12-20T04:43:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.154+0000 7f1c2b266640 1 -- 172.21.15.32:0/1971560852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c24163e20 con 0x7f1c24158c50 2023-12-20T04:43:14.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.154+0000 7f1c2b266640 1 -- 172.21.15.32:0/1971560852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c241643e0 con 0x7f1c24158c50 2023-12-20T04:43:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.154+0000 7f1c2b266640 1 -- 172.21.15.32:0/1971560852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c241645a0 con 0x7f1c24158c50 2023-12-20T04:43:14.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.154+0000 7f1c19ffb640 1 -- 172.21.15.32:0/1971560852 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1c1c004030 con 0x7f1c24158c50 2023-12-20T04:43:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.154+0000 7f1c19ffb640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c08065b80 0x7f1c08068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.154+0000 7f1c19ffb640 1 -- 172.21.15.32:0/1971560852 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f1c1c0835c0 con 0x7f1c24158c50 2023-12-20T04:43:14.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.154+0000 7f1c29002640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c08065b80 0x7f1c08068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.158+0000 7f1c29002640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c08065b80 0x7f1c08068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1c200af000 tx=0x7f1c200b6040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:14.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.158+0000 7f1c19ffb640 1 -- 172.21.15.32:0/1971560852 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1c1c050e80 con 0x7f1c24158c50 2023-12-20T04:43:14.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.374+0000 7f1c2b266640 1 -- 172.21.15.32:0/1971560852 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]} v 0) v1 -- 0x7f1c2404e000 con 0x7f1c24158c50 2023-12-20T04:43:14.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.526+0000 7f1c19ffb640 1 -- 172.21.15.32:0/1971560852 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_multi_snap_schedule"]}]=0 v528) v1 ==== 154+0+0 (secure 0 0 0) 0x7f1c1c0544a0 con 0x7f1c24158c50 2023-12-20T04:43:14.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 -- 172.21.15.32:0/1971560852 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c08065b80 msgr2=0x7f1c08068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c08065b80 0x7f1c08068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f1c200af000 tx=0x7f1c200b6040 comp rx=0 tx=0).stop 2023-12-20T04:43:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 -- 172.21.15.32:0/1971560852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c24158c50 msgr2=0x7f1c241590d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c24158c50 0x7f1c241590d0 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c007df0 tx=0x7f1c1c00d4a0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 -- 172.21.15.32:0/1971560852 shutdown_connections 2023-12-20T04:43:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c08065b80 0x7f1c08068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c24158c50 0x7f1c241590d0 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c24163010 0x7f1c24163490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 --2- 172.21.15.32:0/1971560852 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c240ae010 0x7f1c24150c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 -- 172.21.15.32:0/1971560852 >> 172.21.15.32:0/1971560852 conn(0x7f1c2406a320 msgr2=0x7f1c2415b010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 -- 172.21.15.32:0/1971560852 shutdown_connections 2023-12-20T04:43:14.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.530+0000 7f1c2b266640 1 -- 172.21.15.32:0/1971560852 wait complete. 2023-12-20T04:43:14.547 INFO:tasks.cephfs_test_runner:Test exisitence of multiple scheduled snapshots ... ok 2023-12-20T04:43:14.548 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing non-existing snap schedules failure 2023-12-20T04:43:14.548 INFO:tasks.cephfs_test_runner:test_non_existent_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-20T04:43:14.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-12-20T04:43:14.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe847255640 1 Processor -- start 2023-12-20T04:43:14.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe847255640 1 -- start start 2023-12-20T04:43:14.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe847255640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe8401599e0 0x7fe8400aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe847255640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe8400c90e0 0x7fe8400b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe847255640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8400b0b30 0x7fe84015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe847255640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8400c98c0 con 0x7fe8400b0b30 2023-12-20T04:43:14.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe847255640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8400c9ba0 con 0x7fe8400c90e0 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe847255640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400c9e80 con 0x7fe8401599e0 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe8457f2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8400b0b30 0x7fe84015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe8457f2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe8400b0b30 0x7fe84015d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37070/0 (socket says 172.21.15.32:37070) 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe8457f2640 1 -- 172.21.15.32:0/1849552067 learned_addr learned my addr 172.21.15.32:0/1849552067 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe844ff1640 1 --2- 172.21.15.32:0/1849552067 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe8401599e0 0x7fe8400aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe83ffff640 1 --2- 172.21.15.32:0/1849552067 >> v2:172.21.15.39:3300/0 conn(0x7fe8400c90e0 0x7fe8400b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe8457f2640 1 -- 172.21.15.32:0/1849552067 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe8401599e0 msgr2=0x7fe8400aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe8457f2640 1 --2- 172.21.15.32:0/1849552067 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe8401599e0 0x7fe8400aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe8457f2640 1 -- 172.21.15.32:0/1849552067 >> v2:172.21.15.39:3300/0 conn(0x7fe8400c90e0 msgr2=0x7fe8400b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe8457f2640 1 --2- 172.21.15.32:0/1849552067 >> v2:172.21.15.39:3300/0 conn(0x7fe8400c90e0 0x7fe8400b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe8457f2640 1 -- 172.21.15.32:0/1849552067 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8400ca160 con 0x7fe8400b0b30 2023-12-20T04:43:14.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.694+0000 7fe83ffff640 1 --2- 172.21.15.32:0/1849552067 >> v2:172.21.15.39:3300/0 conn(0x7fe8400c90e0 0x7fe8400b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:14.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe8457f2640 1 --2- 172.21.15.32:0/1849552067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400b0b30 0x7fe84015d6e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe84006d6b0 tx=0x7fe83801c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f64f8c29e7bc12a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe83f7fe640 1 -- 172.21.15.32:0/1849552067 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8380a5580 con 0x7fe8400b0b30 2023-12-20T04:43:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe83f7fe640 1 -- 172.21.15.32:0/1849552067 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8380a56e0 con 0x7fe8400b0b30 2023-12-20T04:43:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe83f7fe640 1 -- 172.21.15.32:0/1849552067 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8380a5a70 con 0x7fe8400b0b30 2023-12-20T04:43:14.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe847255640 1 -- 172.21.15.32:0/1849552067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400b0b30 msgr2=0x7fe84015d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe847255640 1 --2- 172.21.15.32:0/1849552067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400b0b30 0x7fe84015d6e0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7fe84006d6b0 tx=0x7fe83801c6e0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe847255640 1 -- 172.21.15.32:0/1849552067 shutdown_connections 2023-12-20T04:43:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe847255640 1 --2- 172.21.15.32:0/1849552067 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400b0b30 0x7fe84015d6e0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe847255640 1 --2- 172.21.15.32:0/1849552067 >> v2:172.21.15.39:3300/0 conn(0x7fe8400c90e0 0x7fe8400b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe847255640 1 --2- 172.21.15.32:0/1849552067 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe8401599e0 0x7fe8400aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe847255640 1 -- 172.21.15.32:0/1849552067 >> 172.21.15.32:0/1849552067 conn(0x7fe84006a320 msgr2=0x7fe8400ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe847255640 1 -- 172.21.15.32:0/1849552067 shutdown_connections 2023-12-20T04:43:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.698+0000 7fe847255640 1 -- 172.21.15.32:0/1849552067 wait complete. 2023-12-20T04:43:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe847255640 1 Processor -- start 2023-12-20T04:43:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe847255640 1 -- start start 2023-12-20T04:43:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe847255640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400b0b30 0x7fe840163690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe847255640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe8400c90e0 0x7fe8400b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe847255640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8401599e0 0x7fe84014eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe847255640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8400b2e20 con 0x7fe8400b0b30 2023-12-20T04:43:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe847255640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe840150580 con 0x7fe8401599e0 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe847255640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe840150820 con 0x7fe8400c90e0 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe844ff1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400b0b30 0x7fe840163690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe844ff1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400b0b30 0x7fe840163690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37078/0 (socket says 172.21.15.32:37078) 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe844ff1640 1 -- 172.21.15.32:0/2652590524 learned_addr learned my addr 172.21.15.32:0/2652590524 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe83ffff640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe8400c90e0 0x7fe8400b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe8457f2640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8401599e0 0x7fe84014eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe844ff1640 1 -- 172.21.15.32:0/2652590524 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe8400c90e0 msgr2=0x7fe8400b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe844ff1640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe8400c90e0 0x7fe8400b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe844ff1640 1 -- 172.21.15.32:0/2652590524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8401599e0 msgr2=0x7fe84014eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe844ff1640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8401599e0 0x7fe84014eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe844ff1640 1 -- 172.21.15.32:0/2652590524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe838009b30 con 0x7fe8400b0b30 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe83ffff640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe8400c90e0 0x7fe8400b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe844ff1640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400b0b30 0x7fe840163690 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe83400e870 tx=0x7fe83400ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe83d7fa640 1 -- 172.21.15.32:0/2652590524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8340095d0 con 0x7fe8400b0b30 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe83d7fa640 1 -- 172.21.15.32:0/2652590524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe83400c940 con 0x7fe8400b0b30 2023-12-20T04:43:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe83d7fa640 1 -- 172.21.15.32:0/2652590524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe83400ccb0 con 0x7fe8400b0b30 2023-12-20T04:43:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe847255640 1 -- 172.21.15.32:0/2652590524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe840150b00 con 0x7fe8400b0b30 2023-12-20T04:43:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe847255640 1 -- 172.21.15.32:0/2652590524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe8400b3050 con 0x7fe8400b0b30 2023-12-20T04:43:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.702+0000 7fe847255640 1 -- 172.21.15.32:0/2652590524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe84004e000 con 0x7fe8400b0b30 2023-12-20T04:43:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.706+0000 7fe83d7fa640 1 -- 172.21.15.32:0/2652590524 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe834013470 con 0x7fe8400b0b30 2023-12-20T04:43:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.706+0000 7fe83d7fa640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe824065b80 0x7fe824068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.706+0000 7fe83d7fa640 1 -- 172.21.15.32:0/2652590524 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe8340836b0 con 0x7fe8400b0b30 2023-12-20T04:43:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.706+0000 7fe83ffff640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe824065b80 0x7fe824068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:14.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.706+0000 7fe83ffff640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe824065b80 0x7fe824068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe840156c80 tx=0x7fe82c008040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.710+0000 7fe83d7fa640 1 -- 172.21.15.32:0/2652590524 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe834051140 con 0x7fe8400b0b30 2023-12-20T04:43:14.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:14.922+0000 7fe847255640 1 -- 172.21.15.32:0/2652590524 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fe84015da00 con 0x7fe8400b0b30 2023-12-20T04:43:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe83d7fa640 1 -- 172.21.15.32:0/2652590524 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v529) v1 ==== 159+0+0 (secure 0 0 0) 0x7fe834054760 con 0x7fe8400b0b30 2023-12-20T04:43:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 -- 172.21.15.32:0/2652590524 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe824065b80 msgr2=0x7fe824068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe824065b80 0x7fe824068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fe840156c80 tx=0x7fe82c008040 comp rx=0 tx=0).stop 2023-12-20T04:43:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 -- 172.21.15.32:0/2652590524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400b0b30 msgr2=0x7fe840163690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:15.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400b0b30 0x7fe840163690 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7fe83400e870 tx=0x7fe83400ed30 comp rx=0 tx=0).stop 2023-12-20T04:43:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 -- 172.21.15.32:0/2652590524 shutdown_connections 2023-12-20T04:43:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe824065b80 0x7fe824068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe8401599e0 0x7fe84014eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe8400c90e0 0x7fe8400b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 --2- 172.21.15.32:0/2652590524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe8400b0b30 0x7fe840163690 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 -- 172.21.15.32:0/2652590524 >> 172.21.15.32:0/2652590524 conn(0x7fe84006a320 msgr2=0x7fe8400caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 -- 172.21.15.32:0/2652590524 shutdown_connections 2023-12-20T04:43:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.530+0000 7fe847255640 1 -- 172.21.15.32:0/2652590524 wait complete. 2023-12-20T04:43:15.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T04:43:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe957495640 1 Processor -- start 2023-12-20T04:43:15.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe957495640 1 -- start start 2023-12-20T04:43:15.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe957495640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe950157480 0x7fe9500c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:15.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe957495640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9500b0760 0x7fe9500b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:15.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe957495640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9500b1100 con 0x7fe95015d660 2023-12-20T04:43:15.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe957495640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9500cde20 con 0x7fe9500b0760 2023-12-20T04:43:15.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe957495640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9500c8e30 con 0x7fe950157480 2023-12-20T04:43:15.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe954a30640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9500b0760 0x7fe9500b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:15.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe954a30640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9500b0760 0x7fe9500b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37086/0 (socket says 172.21.15.32:37086) 2023-12-20T04:43:15.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe954a30640 1 -- 172.21.15.32:0/3476935189 learned_addr learned my addr 172.21.15.32:0/3476935189 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:15.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe955231640 1 --2- 172.21.15.32:0/3476935189 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe950157480 0x7fe9500c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:15.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe954a30640 1 -- 172.21.15.32:0/3476935189 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe950157480 msgr2=0x7fe9500c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:15.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe954a30640 1 --2- 172.21.15.32:0/3476935189 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe950157480 0x7fe9500c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe954a30640 1 -- 172.21.15.32:0/3476935189 >> v1:172.21.15.39:6789/0 conn(0x7fe95015d660 legacy=0x7fe95015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:15.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe954a30640 1 -- 172.21.15.32:0/3476935189 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9500ce070 con 0x7fe9500b0760 2023-12-20T04:43:15.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.702+0000 7fe955231640 1 --2- 172.21.15.32:0/3476935189 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe950157480 0x7fe9500c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:43:15.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe954a30640 1 --2- 172.21.15.32:0/3476935189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9500b0760 0x7fe9500b0bc0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe940009870 tx=0x7fe94001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e3fb10cd602f7e3e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:15.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe947fff640 1 -- 172.21.15.32:0/3476935189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9400ab680 con 0x7fe9500b0760 2023-12-20T04:43:15.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe947fff640 1 -- 172.21.15.32:0/3476935189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9400ab7e0 con 0x7fe9500b0760 2023-12-20T04:43:15.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe947fff640 1 -- 172.21.15.32:0/3476935189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9400abb70 con 0x7fe9500b0760 2023-12-20T04:43:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe957495640 1 -- 172.21.15.32:0/3476935189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9500b0760 msgr2=0x7fe9500b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe957495640 1 --2- 172.21.15.32:0/3476935189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9500b0760 0x7fe9500b0bc0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7fe940009870 tx=0x7fe94001c480 comp rx=0 tx=0).stop 2023-12-20T04:43:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe957495640 1 -- 172.21.15.32:0/3476935189 shutdown_connections 2023-12-20T04:43:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe957495640 1 --2- 172.21.15.32:0/3476935189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9500b0760 0x7fe9500b0bc0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe957495640 1 --2- 172.21.15.32:0/3476935189 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe950157480 0x7fe9500c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe957495640 1 -- 172.21.15.32:0/3476935189 >> 172.21.15.32:0/3476935189 conn(0x7fe95006a510 msgr2=0x7fe9500ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe957495640 1 -- 172.21.15.32:0/3476935189 shutdown_connections 2023-12-20T04:43:15.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.706+0000 7fe957495640 1 -- 172.21.15.32:0/3476935189 wait complete. 2023-12-20T04:43:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe957495640 1 Processor -- start 2023-12-20T04:43:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe957495640 1 -- start start 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe957495640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9500b0760 0x7fe950156830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe957495640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe950157480 0x7fe950140ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe957495640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95015d660 0x7fe95015ac10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe957495640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe950141530 con 0x7fe95015d660 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe957495640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95015bfe0 con 0x7fe9500b0760 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe957495640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95015b150 con 0x7fe950157480 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe955a32640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95015d660 0x7fe95015ac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe955a32640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95015d660 0x7fe95015ac10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37100/0 (socket says 172.21.15.32:37100) 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe955a32640 1 -- 172.21.15.32:0/1675403633 learned_addr learned my addr 172.21.15.32:0/1675403633 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe955231640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9500b0760 0x7fe950156830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe954a30640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe950157480 0x7fe950140ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe955a32640 1 -- 172.21.15.32:0/1675403633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe950157480 msgr2=0x7fe950140ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe955a32640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe950157480 0x7fe950140ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe955a32640 1 -- 172.21.15.32:0/1675403633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9500b0760 msgr2=0x7fe950156830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe955a32640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9500b0760 0x7fe950156830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe955a32640 1 -- 172.21.15.32:0/1675403633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9400094d0 con 0x7fe95015d660 2023-12-20T04:43:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe955a32640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95015d660 0x7fe95015ac10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c770 tx=0x7fe94800cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe945ffb640 1 -- 172.21.15.32:0/1675403633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe948003b80 con 0x7fe95015d660 2023-12-20T04:43:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe945ffb640 1 -- 172.21.15.32:0/1675403633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe948003ce0 con 0x7fe95015d660 2023-12-20T04:43:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe957495640 1 -- 172.21.15.32:0/1675403633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95015b3d0 con 0x7fe95015d660 2023-12-20T04:43:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe945ffb640 1 -- 172.21.15.32:0/1675403633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe94800aa90 con 0x7fe95015d660 2023-12-20T04:43:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.710+0000 7fe957495640 1 -- 172.21.15.32:0/1675403633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95015b990 con 0x7fe95015d660 2023-12-20T04:43:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.714+0000 7fe957495640 1 -- 172.21.15.32:0/1675403633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95015c170 con 0x7fe95015d660 2023-12-20T04:43:15.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.714+0000 7fe945ffb640 1 -- 172.21.15.32:0/1675403633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe94800abf0 con 0x7fe95015d660 2023-12-20T04:43:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.714+0000 7fe945ffb640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe934065bb0 0x7fe934068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.714+0000 7fe945ffb640 1 -- 172.21.15.32:0/1675403633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe948082fe0 con 0x7fe95015d660 2023-12-20T04:43:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.714+0000 7fe955231640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe934065bb0 0x7fe934068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.714+0000 7fe955231640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe934065bb0 0x7fe934068070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe950155880 tx=0x7fe94c008040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:15.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.718+0000 7fe945ffb640 1 -- 172.21.15.32:0/1675403633 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe948050920 con 0x7fe95015d660 2023-12-20T04:43:15.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.878+0000 7fe957495640 1 -- 172.21.15.32:0/1675403633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fe95004e000 con 0x7fe95015d660 2023-12-20T04:43:15.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.882+0000 7fe945ffb640 1 -- 172.21.15.32:0/1675403633 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v22)=0 v22) v1 ==== 122+0+0 (secure 0 0 0) 0x7fe948053f40 con 0x7fe95015d660 2023-12-20T04:43:15.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 -- 172.21.15.32:0/1675403633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe934065bb0 msgr2=0x7fe934068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:15.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe934065bb0 0x7fe934068070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fe950155880 tx=0x7fe94c008040 comp rx=0 tx=0).stop 2023-12-20T04:43:15.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 -- 172.21.15.32:0/1675403633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95015d660 msgr2=0x7fe95015ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:15.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95015d660 0x7fe95015ac10 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7fe94800c770 tx=0x7fe94800cc30 comp rx=0 tx=0).stop 2023-12-20T04:43:15.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 -- 172.21.15.32:0/1675403633 shutdown_connections 2023-12-20T04:43:15.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe934065bb0 0x7fe934068070 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95015d660 0x7fe95015ac10 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe950157480 0x7fe950140ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 --2- 172.21.15.32:0/1675403633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9500b0760 0x7fe950156830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:15.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 -- 172.21.15.32:0/1675403633 >> 172.21.15.32:0/1675403633 conn(0x7fe95006a510 msgr2=0x7fe950163700 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:15.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 -- 172.21.15.32:0/1675403633 shutdown_connections 2023-12-20T04:43:15.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:15.886+0000 7fe957495640 1 -- 172.21.15.32:0/1675403633 wait complete. 2023-12-20T04:43:15.902 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:43:15.902 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:43:15.982 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:43:16.069 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:43:16.069 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:43:16.069 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:16.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:43:16.079 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:43:16.079 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:16.088 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:43:16.088 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:43:16.089 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:16.135 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:43:16.135 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:43:16.135 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:16.145 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:43:16.145 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T04:43:16.145 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:16.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.322+0000 7f0a1d68c640 1 Processor -- start 2023-12-20T04:43:16.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.322+0000 7f0a1d68c640 1 -- start start 2023-12-20T04:43:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a1d68c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0a181599e0 0x7f0a180aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a1d68c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 0x7f0a180701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a1d68c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a180c98c0 con 0x7f0a180c90e0 2023-12-20T04:43:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a1d68c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a180c9ba0 con 0x7f0a181599e0 2023-12-20T04:43:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a1d68c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a180c9e80 con 0x7f0a180b0d00 2023-12-20T04:43:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 0x7f0a180701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:16.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 0x7f0a180701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:48538/0 (socket says 172.21.15.32:48538) 2023-12-20T04:43:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a16ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0a181599e0 0x7f0a180aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a177fe640 1 -- 172.21.15.32:0/408408120 learned_addr learned my addr 172.21.15.32:0/408408120 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a0ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0a180c90e0 0x7f0a180b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:45324/0 (socket says 172.21.15.32:45324) 2023-12-20T04:43:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a167fc640 1 -- 172.21.15.32:0/408408120 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566948133 0 0) 0x7f0a180c98c0 con 0x7f0a180c90e0 2023-12-20T04:43:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a167fc640 1 -- 172.21.15.32:0/408408120 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a04003530 con 0x7f0a180c90e0 2023-12-20T04:43:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a177fe640 1 -- 172.21.15.32:0/408408120 >> v2:172.21.15.39:3300/0 conn(0x7f0a181599e0 msgr2=0x7f0a180aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a177fe640 1 --2- 172.21.15.32:0/408408120 >> v2:172.21.15.39:3300/0 conn(0x7f0a181599e0 0x7f0a180aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a177fe640 1 -- 172.21.15.32:0/408408120 >> v1:172.21.15.39:6789/0 conn(0x7f0a180c90e0 legacy=0x7f0a180b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a177fe640 1 -- 172.21.15.32:0/408408120 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a180ca160 con 0x7f0a180b0d00 2023-12-20T04:43:16.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.326+0000 7f0a16ffd640 1 --2- 172.21.15.32:0/408408120 >> v2:172.21.15.39:3300/0 conn(0x7f0a181599e0 0x7f0a180aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:16.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a177fe640 1 --2- 172.21.15.32:0/408408120 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 0x7f0a180701c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000d150 tx=0x7f0a1001eb50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb068240448c29b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a167fc640 1 -- 172.21.15.32:0/408408120 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a1000b2d0 con 0x7f0a180b0d00 2023-12-20T04:43:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a167fc640 1 -- 172.21.15.32:0/408408120 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a1000b430 con 0x7f0a180b0d00 2023-12-20T04:43:16.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a167fc640 1 -- 172.21.15.32:0/408408120 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a100a7c60 con 0x7f0a180b0d00 2023-12-20T04:43:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a1d68c640 1 -- 172.21.15.32:0/408408120 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 msgr2=0x7f0a180701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a1d68c640 1 --2- 172.21.15.32:0/408408120 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 0x7f0a180701c0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f0a1000d150 tx=0x7f0a1001eb50 comp rx=0 tx=0).stop 2023-12-20T04:43:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a1d68c640 1 -- 172.21.15.32:0/408408120 shutdown_connections 2023-12-20T04:43:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a1d68c640 1 --2- 172.21.15.32:0/408408120 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 0x7f0a180701c0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a1d68c640 1 --2- 172.21.15.32:0/408408120 >> v2:172.21.15.39:3300/0 conn(0x7f0a181599e0 0x7f0a180aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a1d68c640 1 -- 172.21.15.32:0/408408120 >> 172.21.15.32:0/408408120 conn(0x7f0a1806a320 msgr2=0x7f0a180ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a1d68c640 1 -- 172.21.15.32:0/408408120 shutdown_connections 2023-12-20T04:43:16.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a1d68c640 1 -- 172.21.15.32:0/408408120 wait complete. 2023-12-20T04:43:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a1d68c640 1 Processor -- start 2023-12-20T04:43:16.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.330+0000 7f0a1d68c640 1 -- start start 2023-12-20T04:43:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a1d68c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 0x7f0a1814fe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a1d68c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a180c90e0 0x7f0a181524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a1d68c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a181599e0 0x7f0a1814acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a1d68c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a18152a00 con 0x7f0a181599e0 2023-12-20T04:43:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a1d68c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a18152c10 con 0x7f0a180c90e0 2023-12-20T04:43:16.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a1d68c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a18152ef0 con 0x7f0a180b0d00 2023-12-20T04:43:16.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a181599e0 0x7f0a1814acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a16ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 0x7f0a1814fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a0ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a180c90e0 0x7f0a181524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a16ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 0x7f0a1814fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:48544/0 (socket says 172.21.15.32:48544) 2023-12-20T04:43:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a16ffd640 1 -- 172.21.15.32:0/690719181 learned_addr learned my addr 172.21.15.32:0/690719181 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a181599e0 0x7f0a1814acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37116/0 (socket says 172.21.15.32:37116) 2023-12-20T04:43:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a177fe640 1 -- 172.21.15.32:0/690719181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 msgr2=0x7f0a1814fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a177fe640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 0x7f0a1814fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a177fe640 1 -- 172.21.15.32:0/690719181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a180c90e0 msgr2=0x7f0a181524c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:16.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a177fe640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a180c90e0 0x7f0a181524c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a177fe640 1 -- 172.21.15.32:0/690719181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a080093b0 con 0x7f0a181599e0 2023-12-20T04:43:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a177fe640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a181599e0 0x7f0a1814acb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0a100ad000 tx=0x7f0a1001b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a0f7fe640 1 -- 172.21.15.32:0/690719181 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10009800 con 0x7f0a181599e0 2023-12-20T04:43:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a0f7fe640 1 -- 172.21.15.32:0/690719181 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a10009980 con 0x7f0a181599e0 2023-12-20T04:43:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a0f7fe640 1 -- 172.21.15.32:0/690719181 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a10009d30 con 0x7f0a181599e0 2023-12-20T04:43:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a1d68c640 1 -- 172.21.15.32:0/690719181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a1000bd40 con 0x7f0a181599e0 2023-12-20T04:43:16.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a1d68c640 1 -- 172.21.15.32:0/690719181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a180abc60 con 0x7f0a181599e0 2023-12-20T04:43:16.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.334+0000 7f0a1d68c640 1 -- 172.21.15.32:0/690719181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a1804e000 con 0x7f0a181599e0 2023-12-20T04:43:16.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.338+0000 7f0a0f7fe640 1 -- 172.21.15.32:0/690719181 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0a100af3f0 con 0x7f0a181599e0 2023-12-20T04:43:16.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.338+0000 7f0a0f7fe640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09f8065bb0 0x7f09f8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:16.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.338+0000 7f0a0f7fe640 1 -- 172.21.15.32:0/690719181 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(70..70 src has 1..70) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f0a1011d9f0 con 0x7f0a181599e0 2023-12-20T04:43:16.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.338+0000 7f0a16ffd640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09f8065bb0 0x7f09f8068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:16.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.338+0000 7f0a16ffd640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09f8065bb0 0x7f09f8068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009dd0 tx=0x7f0a08006f90 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:16.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.342+0000 7f0a0f7fe640 1 -- 172.21.15.32:0/690719181 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0a100eb460 con 0x7f0a181599e0 2023-12-20T04:43:16.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.530+0000 7f0a1d68c640 1 -- 172.21.15.32:0/690719181 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a180cf550 con 0x7f0a181599e0 2023-12-20T04:43:16.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.530+0000 7f0a0f7fe640 1 -- 172.21.15.32:0/690719181 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 26 v26) v1 ==== 94+0+3523 (secure 0 0 0) 0x7f0a100bdde0 con 0x7f0a181599e0 2023-12-20T04:43:16.531 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:16.531 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":26,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5369,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/536564441","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":536564441},{"type":"v1","addr":"172.21.15.32:6835","nonce":536564441}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5381,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2898125535","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":2898125535},{"type":"v1","addr":"172.21.15.39:6835","nonce":2898125535}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":5892,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/319656564","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":319656564},{"type":"v1","addr":"172.21.15.32:6837","nonce":319656564}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":19},{"gid":6208,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1720091596","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1720091596},{"type":"v1","addr":"172.21.15.39:6834","nonce":1720091596}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":26}],"filesystems":[]} 2023-12-20T04:43:16.531 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 26 2023-12-20T04:43:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 -- 172.21.15.32:0/690719181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09f8065bb0 msgr2=0x7f09f8068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09f8065bb0 0x7f09f8068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f0a08009dd0 tx=0x7f0a08006f90 comp rx=0 tx=0).stop 2023-12-20T04:43:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 -- 172.21.15.32:0/690719181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a181599e0 msgr2=0x7f0a1814acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a181599e0 0x7f0a1814acb0 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f0a100ad000 tx=0x7f0a1001b8f0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 -- 172.21.15.32:0/690719181 shutdown_connections 2023-12-20T04:43:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09f8065bb0 0x7f09f8068070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a181599e0 0x7f0a1814acb0 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a180c90e0 0x7f0a181524c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 --2- 172.21.15.32:0/690719181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a180b0d00 0x7f0a1814fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 -- 172.21.15.32:0/690719181 >> 172.21.15.32:0/690719181 conn(0x7f0a1806a320 msgr2=0x7f0a18158cb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 -- 172.21.15.32:0/690719181 shutdown_connections 2023-12-20T04:43:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.534+0000 7f0a1d68c640 1 -- 172.21.15.32:0/690719181 wait complete. 2023-12-20T04:43:16.552 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T04:43:16.552 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T04:43:16.552 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T04:43:16.552 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:43:16.553 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T04:43:16.553 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T04:43:16.553 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T04:43:16.553 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:43:16.553 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T04:43:16.553 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T04:43:16.553 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T04:43:16.554 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:43:16.554 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T04:43:16.554 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T04:43:16.554 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T04:43:16.554 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:43:16.568 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T04:43:16.568 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T04:43:16.569 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T04:43:16.569 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T04:43:16.570 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T04:43:16.570 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T04:43:16.571 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T04:43:16.571 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T04:43:16.614 INFO:tasks.ceph.mds.a:Started 2023-12-20T04:43:16.614 DEBUG:teuthology.parallel:result is None 2023-12-20T04:43:16.616 INFO:tasks.ceph.mds.c:Started 2023-12-20T04:43:16.616 DEBUG:teuthology.parallel:result is None 2023-12-20T04:43:16.618 INFO:tasks.ceph.mds.b:Started 2023-12-20T04:43:16.618 DEBUG:teuthology.parallel:result is None 2023-12-20T04:43:16.620 INFO:tasks.ceph.mds.d:Started 2023-12-20T04:43:16.620 DEBUG:teuthology.parallel:result is None 2023-12-20T04:43:16.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T04:43:16.702 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T04:43:16.703 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T04:43:16.720 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T04:43:16.720 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T04:43:16.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.774+0000 7f0d18d25640 1 Processor -- start 2023-12-20T04:43:16.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.774+0000 7f0d18d25640 1 -- start start 2023-12-20T04:43:16.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.774+0000 7f0d18d25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140b0d00 0x7f0d14164a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.774+0000 7f0d18d25640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140711d0 con 0x7f0d141599e0 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.774+0000 7f0d18d25640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d140c98c0 con 0x7f0d140aece0 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.774+0000 7f0d18d25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d140c9ba0 con 0x7f0d140b0d00 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d1259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d141599e0 0x7f0d140c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60576/0 (socket says 172.21.15.32:60576) 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d1259c640 1 -- 172.21.15.32:0/2570632541 learned_addr learned my addr 172.21.15.32:0/2570632541 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d12d9d640 1 --2- 172.21.15.32:0/2570632541 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140b0d00 0x7f0d14164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d11d9b640 1 -- 172.21.15.32:0/2570632541 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2029235145 0 0) 0x7f0d140711d0 con 0x7f0d141599e0 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d11d9b640 1 -- 172.21.15.32:0/2570632541 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d00003530 con 0x7f0d141599e0 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d11d9b640 1 -- 172.21.15.32:0/2570632541 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4175525411 0 0) 0x7f0d140c98c0 con 0x7f0d140aece0 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d11d9b640 1 -- 172.21.15.32:0/2570632541 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0d140711d0 con 0x7f0d140aece0 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d12d9d640 1 -- 172.21.15.32:0/2570632541 >> v1:172.21.15.39:6789/0 conn(0x7f0d140aece0 legacy=0x7f0d140b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d12d9d640 1 -- 172.21.15.32:0/2570632541 >> v1:172.21.15.32:6789/0 conn(0x7f0d141599e0 legacy=0x7f0d140c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:16.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d12d9d640 1 -- 172.21.15.32:0/2570632541 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d140c9e80 con 0x7f0d140b0d00 2023-12-20T04:43:16.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d12d9d640 1 --2- 172.21.15.32:0/2570632541 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140b0d00 0x7f0d14164a70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009dc0 tx=0x7f0d0c01c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=aa4ed1c69023f2c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:16.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d11d9b640 1 -- 172.21.15.32:0/2570632541 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0a5790 con 0x7f0d140b0d00 2023-12-20T04:43:16.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d11d9b640 1 -- 172.21.15.32:0/2570632541 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d0c0a58f0 con 0x7f0d140b0d00 2023-12-20T04:43:16.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.778+0000 7f0d11d9b640 1 -- 172.21.15.32:0/2570632541 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d0c0a5c60 con 0x7f0d140b0d00 2023-12-20T04:43:16.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 -- 172.21.15.32:0/2570632541 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140b0d00 msgr2=0x7f0d14164a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:16.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 --2- 172.21.15.32:0/2570632541 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140b0d00 0x7f0d14164a70 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f0d0c009dc0 tx=0x7f0d0c01c8e0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 -- 172.21.15.32:0/2570632541 shutdown_connections 2023-12-20T04:43:16.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 --2- 172.21.15.32:0/2570632541 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140b0d00 0x7f0d14164a70 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 -- 172.21.15.32:0/2570632541 >> 172.21.15.32:0/2570632541 conn(0x7f0d1406a320 msgr2=0x7f0d140ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:16.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 -- 172.21.15.32:0/2570632541 shutdown_connections 2023-12-20T04:43:16.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 -- 172.21.15.32:0/2570632541 wait complete. 2023-12-20T04:43:16.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 Processor -- start 2023-12-20T04:43:16.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 -- start start 2023-12-20T04:43:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140aece0 0x7f0d14154630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d140b0d00 0x7f0d1415a100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d141599e0 0x7f0d1415a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14155d50 con 0x7f0d140b0d00 2023-12-20T04:43:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d1415d0a0 con 0x7f0d141599e0 2023-12-20T04:43:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d18d25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d14153b90 con 0x7f0d140aece0 2023-12-20T04:43:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d1259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140aece0 0x7f0d14154630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d0bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d140b0d00 0x7f0d1415a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d1259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140aece0 0x7f0d14154630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:48604/0 (socket says 172.21.15.32:48604) 2023-12-20T04:43:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d1259c640 1 -- 172.21.15.32:0/2358161904 learned_addr learned my addr 172.21.15.32:0/2358161904 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.782+0000 7f0d12d9d640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d141599e0 0x7f0d1415a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d1259c640 1 -- 172.21.15.32:0/2358161904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d141599e0 msgr2=0x7f0d1415a640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d1259c640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d141599e0 0x7f0d1415a640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d1259c640 1 -- 172.21.15.32:0/2358161904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d140b0d00 msgr2=0x7f0d1415a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d1259c640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d140b0d00 0x7f0d1415a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d1259c640 1 -- 172.21.15.32:0/2358161904 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d0c009a70 con 0x7f0d140aece0 2023-12-20T04:43:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d0bfff640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d140b0d00 0x7f0d1415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d1259c640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140aece0 0x7f0d14154630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400c700 tx=0x7f0d0400cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d0affd640 1 -- 172.21.15.32:0/2358161904 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d04007270 con 0x7f0d140aece0 2023-12-20T04:43:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d0affd640 1 -- 172.21.15.32:0/2358161904 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d040073d0 con 0x7f0d140aece0 2023-12-20T04:43:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d18d25640 1 -- 172.21.15.32:0/2358161904 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d1415cdc0 con 0x7f0d140aece0 2023-12-20T04:43:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d0affd640 1 -- 172.21.15.32:0/2358161904 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d040057b0 con 0x7f0d140aece0 2023-12-20T04:43:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d18d25640 1 -- 172.21.15.32:0/2358161904 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d14154110 con 0x7f0d140aece0 2023-12-20T04:43:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d18d25640 1 -- 172.21.15.32:0/2358161904 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d14154330 con 0x7f0d140aece0 2023-12-20T04:43:16.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d0affd640 1 -- 172.21.15.32:0/2358161904 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0d04005910 con 0x7f0d140aece0 2023-12-20T04:43:16.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d0affd640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cf4065b80 0x7f0cf4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:16.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d0affd640 1 -- 172.21.15.32:0/2358161904 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(74..74 src has 1..74) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f0d040827e0 con 0x7f0d140aece0 2023-12-20T04:43:16.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.786+0000 7f0d0bfff640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cf4065b80 0x7f0cf4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:16.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.790+0000 7f0d0bfff640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cf4065b80 0x7f0cf4068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d141f8870 tx=0x7f0cfc009040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:16.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.790+0000 7f0d0affd640 1 -- 172.21.15.32:0/2358161904 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0d040501d0 con 0x7f0d140aece0 2023-12-20T04:43:16.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:16.962+0000 7f0d18d25640 1 -- 172.21.15.32:0/2358161904 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0d1404e000 con 0x7f0d140aece0 2023-12-20T04:43:17.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d0affd640 1 -- 172.21.15.32:0/2358161904 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v75) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0d04017020 con 0x7f0d140aece0 2023-12-20T04:43:17.888 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T04:43:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 -- 172.21.15.32:0/2358161904 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cf4065b80 msgr2=0x7f0cf4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cf4065b80 0x7f0cf4068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0d141f8870 tx=0x7f0cfc009040 comp rx=0 tx=0).stop 2023-12-20T04:43:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 -- 172.21.15.32:0/2358161904 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140aece0 msgr2=0x7f0d14154630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140aece0 0x7f0d14154630 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f0d0400c700 tx=0x7f0d0400cbc0 comp rx=0 tx=0).stop 2023-12-20T04:43:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 -- 172.21.15.32:0/2358161904 shutdown_connections 2023-12-20T04:43:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cf4065b80 0x7f0cf4068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:17.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d141599e0 0x7f0d1415a640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:17.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d140b0d00 0x7f0d1415a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:17.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 --2- 172.21.15.32:0/2358161904 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d140aece0 0x7f0d14154630 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:17.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 -- 172.21.15.32:0/2358161904 >> 172.21.15.32:0/2358161904 conn(0x7f0d1406a320 msgr2=0x7f0d140b32d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:17.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 -- 172.21.15.32:0/2358161904 shutdown_connections 2023-12-20T04:43:17.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:17.890+0000 7f0d18d25640 1 -- 172.21.15.32:0/2358161904 wait complete. 2023-12-20T04:43:17.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T04:43:18.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f4339640 1 Processor -- start 2023-12-20T04:43:18.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f4339640 1 -- start start 2023-12-20T04:43:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f4339640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9ec1599e0 0x7fd9ec0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f4339640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd9ec0c90e0 0x7fd9ec0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f4339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9ec0b0b30 0x7fd9ec15d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f4339640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec0c98c0 con 0x7fd9ec1599e0 2023-12-20T04:43:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f4339640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9ec0c9ba0 con 0x7fd9ec0c90e0 2023-12-20T04:43:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f4339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0c9e80 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f20d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9ec1599e0 0x7fd9ec0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f20d5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9ec1599e0 0x7fd9ec0aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37148/0 (socket says 172.21.15.32:37148) 2023-12-20T04:43:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f20d5640 1 -- 172.21.15.32:0/2452718086 learned_addr learned my addr 172.21.15.32:0/2452718086 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f18d4640 1 --2- 172.21.15.32:0/2452718086 >> v2:172.21.15.39:3300/0 conn(0x7fd9ec0c90e0 0x7fd9ec0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f28d6640 1 --2- 172.21.15.32:0/2452718086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9ec0b0b30 0x7fd9ec15d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f20d5640 1 -- 172.21.15.32:0/2452718086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9ec0b0b30 msgr2=0x7fd9ec15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f20d5640 1 --2- 172.21.15.32:0/2452718086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9ec0b0b30 0x7fd9ec15d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f20d5640 1 -- 172.21.15.32:0/2452718086 >> v2:172.21.15.39:3300/0 conn(0x7fd9ec0c90e0 msgr2=0x7fd9ec0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f20d5640 1 --2- 172.21.15.32:0/2452718086 >> v2:172.21.15.39:3300/0 conn(0x7fd9ec0c90e0 0x7fd9ec0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.066+0000 7fd9f20d5640 1 -- 172.21.15.32:0/2452718086 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ec0ca160 con 0x7fd9ec1599e0 2023-12-20T04:43:18.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f20d5640 1 --2- 172.21.15.32:0/2452718086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ec1599e0 0x7fd9ec0aece0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec06ab20 tx=0x7fd9e801c210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f6e09ded4dc1fe86 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:18.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f10d3640 1 -- 172.21.15.32:0/2452718086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e80ab600 con 0x7fd9ec1599e0 2023-12-20T04:43:18.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f10d3640 1 -- 172.21.15.32:0/2452718086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e80a7050 con 0x7fd9ec1599e0 2023-12-20T04:43:18.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f10d3640 1 -- 172.21.15.32:0/2452718086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e80ab980 con 0x7fd9ec1599e0 2023-12-20T04:43:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f4339640 1 -- 172.21.15.32:0/2452718086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ec1599e0 msgr2=0x7fd9ec0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f4339640 1 --2- 172.21.15.32:0/2452718086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ec1599e0 0x7fd9ec0aece0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec06ab20 tx=0x7fd9e801c210 comp rx=0 tx=0).stop 2023-12-20T04:43:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f4339640 1 -- 172.21.15.32:0/2452718086 shutdown_connections 2023-12-20T04:43:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f4339640 1 --2- 172.21.15.32:0/2452718086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9ec0b0b30 0x7fd9ec15d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f4339640 1 --2- 172.21.15.32:0/2452718086 >> v2:172.21.15.39:3300/0 conn(0x7fd9ec0c90e0 0x7fd9ec0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f4339640 1 --2- 172.21.15.32:0/2452718086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ec1599e0 0x7fd9ec0aece0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f4339640 1 -- 172.21.15.32:0/2452718086 >> 172.21.15.32:0/2452718086 conn(0x7fd9ec06a320 msgr2=0x7fd9ec0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f4339640 1 -- 172.21.15.32:0/2452718086 shutdown_connections 2023-12-20T04:43:18.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.070+0000 7fd9f4339640 1 -- 172.21.15.32:0/2452718086 wait complete. 2023-12-20T04:43:18.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f4339640 1 Processor -- start 2023-12-20T04:43:18.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f4339640 1 -- start start 2023-12-20T04:43:18.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f4339640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ec0b0b30 0x7fd9ec14c780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f4339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9ec0c90e0 0x7fd9ec0b38a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f4339640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec1599e0 0x7fd9ec0b3de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f4339640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec157010 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f4339640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0b1e00 con 0x7fd9ec1599e0 2023-12-20T04:43:18.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f4339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9ec0b20e0 con 0x7fd9ec0c90e0 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f20d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ec0b0b30 0x7fd9ec14c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f20d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ec0b0b30 0x7fd9ec14c780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37160/0 (socket says 172.21.15.32:37160) 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f20d5640 1 -- 172.21.15.32:0/1247199744 learned_addr learned my addr 172.21.15.32:0/1247199744 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f18d4640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9ec0c90e0 0x7fd9ec0b38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f28d6640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec1599e0 0x7fd9ec0b3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f20d5640 1 -- 172.21.15.32:0/1247199744 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9ec0c90e0 msgr2=0x7fd9ec0b38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f20d5640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9ec0c90e0 0x7fd9ec0b38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f20d5640 1 -- 172.21.15.32:0/1247199744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec1599e0 msgr2=0x7fd9ec0b3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f20d5640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec1599e0 0x7fd9ec0b3de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f20d5640 1 -- 172.21.15.32:0/1247199744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9e8009480 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f20d5640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ec0b0b30 0x7fd9ec14c780 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd9e801a8c0 tx=0x7fd9e8018fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9e2ffd640 1 -- 172.21.15.32:0/1247199744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e80059a0 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9e2ffd640 1 -- 172.21.15.32:0/1247199744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9e8005b00 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9e2ffd640 1 -- 172.21.15.32:0/1247199744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9e8003970 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.074+0000 7fd9f4339640 1 -- 172.21.15.32:0/1247199744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9ec0b23c0 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.078+0000 7fd9f4339640 1 -- 172.21.15.32:0/1247199744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9ec0b2980 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.078+0000 7fd9f4339640 1 -- 172.21.15.32:0/1247199744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9ec0b2b70 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.078+0000 7fd9e2ffd640 1 -- 172.21.15.32:0/1247199744 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd9e8003ad0 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.078+0000 7fd9e2ffd640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9d0065b80 0x7fd9d0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.078+0000 7fd9e2ffd640 1 -- 172.21.15.32:0/1247199744 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fd9e811ce50 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.078+0000 7fd9f18d4640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9d0065b80 0x7fd9d0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.078+0000 7fd9f18d4640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9d0065b80 0x7fd9d0068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0b0890 tx=0x7fd9dc008040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:18.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.082+0000 7fd9e2ffd640 1 -- 172.21.15.32:0/1247199744 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd9e80ea9e0 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.302+0000 7fd9f4339640 1 -- 172.21.15.32:0/1247199744 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fd9ec04e000 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.302+0000 7fd9e2ffd640 1 -- 172.21.15.32:0/1247199744 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v16) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fd9e80b0090 con 0x7fd9ec0b0b30 2023-12-20T04:43:18.301 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:18.301 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:43:18.301 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T04:43:18.301 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.301 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T04:43:18.301 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:43:18.302 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.303 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:18.304 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.305 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.306 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.307 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:18.308 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:18.308 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.308 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.308 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T04:43:18.309 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:18.310 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.311 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:18.312 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.313 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T04:43:18.314 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:18.315 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:43:18.315 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:43:18.315 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:18.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.315 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:18.315 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T04:43:18.315 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:43:18.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.302+0000 7fd9f4339640 1 -- 172.21.15.32:0/1247199744 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9d0065b80 msgr2=0x7fd9d0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.302+0000 7fd9f4339640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9d0065b80 0x7fd9d0068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fd9ec0b0890 tx=0x7fd9dc008040 comp rx=0 tx=0).stop 2023-12-20T04:43:18.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.302+0000 7fd9f4339640 1 -- 172.21.15.32:0/1247199744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ec0b0b30 msgr2=0x7fd9ec14c780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.302+0000 7fd9f4339640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ec0b0b30 0x7fd9ec14c780 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7fd9e801a8c0 tx=0x7fd9e8018fb0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.302+0000 7fd9f4339640 1 -- 172.21.15.32:0/1247199744 shutdown_connections 2023-12-20T04:43:18.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.306+0000 7fd9f4339640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9d0065b80 0x7fd9d0068040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.306+0000 7fd9f4339640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9ec1599e0 0x7fd9ec0b3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.306+0000 7fd9f4339640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9ec0c90e0 0x7fd9ec0b38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.306+0000 7fd9f4339640 1 --2- 172.21.15.32:0/1247199744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9ec0b0b30 0x7fd9ec14c780 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.306+0000 7fd9f4339640 1 -- 172.21.15.32:0/1247199744 >> 172.21.15.32:0/1247199744 conn(0x7fd9ec06a320 msgr2=0x7fd9ec0ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:18.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.306+0000 7fd9f4339640 1 -- 172.21.15.32:0/1247199744 shutdown_connections 2023-12-20T04:43:18.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.306+0000 7fd9f4339640 1 -- 172.21.15.32:0/1247199744 wait complete. 2023-12-20T04:43:18.323 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:18.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.466+0000 7f51a2a1f640 1 Processor -- start 2023-12-20T04:43:18.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.466+0000 7f51a2a1f640 1 -- start start 2023-12-20T04:43:18.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a2a1f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f519c0af5e0 0x7f519c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a2a1f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f519c06ce60 0x7f519c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a2a1f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f519c15f110 0x7f519c15c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a2a1f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c15bbd0 con 0x7f519c15f110 2023-12-20T04:43:18.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a2a1f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f519c15beb0 con 0x7f519c06ce60 2023-12-20T04:43:18.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a2a1f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c15e910 con 0x7f519c0af5e0 2023-12-20T04:43:18.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a0fbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f519c15f110 0x7f519c15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a0fbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f519c15f110 0x7f519c15c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37176/0 (socket says 172.21.15.32:37176) 2023-12-20T04:43:18.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a0fbc640 1 -- 172.21.15.32:0/3826125238 learned_addr learned my addr 172.21.15.32:0/3826125238 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:18.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f519b7fe640 1 --2- 172.21.15.32:0/3826125238 >> v2:172.21.15.39:3300/0 conn(0x7f519c06ce60 0x7f519c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f519bfff640 1 --2- 172.21.15.32:0/3826125238 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f519c0af5e0 0x7f519c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a0fbc640 1 -- 172.21.15.32:0/3826125238 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f519c0af5e0 msgr2=0x7f519c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:18.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a0fbc640 1 --2- 172.21.15.32:0/3826125238 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f519c0af5e0 0x7f519c0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a0fbc640 1 -- 172.21.15.32:0/3826125238 >> v2:172.21.15.39:3300/0 conn(0x7f519c06ce60 msgr2=0x7f519c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:18.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a0fbc640 1 --2- 172.21.15.32:0/3826125238 >> v2:172.21.15.39:3300/0 conn(0x7f519c06ce60 0x7f519c15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a0fbc640 1 -- 172.21.15.32:0/3826125238 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f519c15ebf0 con 0x7f519c15f110 2023-12-20T04:43:18.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.470+0000 7f51a0fbc640 1 --2- 172.21.15.32:0/3826125238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f519c15f110 0x7f519c15c440 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f519c06ad00 tx=0x7f519401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79193d8d6bccf7dd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:18.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f519affd640 1 -- 172.21.15.32:0/3826125238 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940a57b0 con 0x7f519c15f110 2023-12-20T04:43:18.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f519affd640 1 -- 172.21.15.32:0/3826125238 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51940a5910 con 0x7f519c15f110 2023-12-20T04:43:18.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f519affd640 1 -- 172.21.15.32:0/3826125238 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51940a5ca0 con 0x7f519c15f110 2023-12-20T04:43:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 -- 172.21.15.32:0/3826125238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f519c15f110 msgr2=0x7f519c15c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 --2- 172.21.15.32:0/3826125238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f519c15f110 0x7f519c15c440 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7f519c06ad00 tx=0x7f519401c9d0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 -- 172.21.15.32:0/3826125238 shutdown_connections 2023-12-20T04:43:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 --2- 172.21.15.32:0/3826125238 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f519c15f110 0x7f519c15c440 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 --2- 172.21.15.32:0/3826125238 >> v2:172.21.15.39:3300/0 conn(0x7f519c06ce60 0x7f519c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 --2- 172.21.15.32:0/3826125238 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f519c0af5e0 0x7f519c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 -- 172.21.15.32:0/3826125238 >> 172.21.15.32:0/3826125238 conn(0x7f519c06a320 msgr2=0x7f519c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:18.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 -- 172.21.15.32:0/3826125238 shutdown_connections 2023-12-20T04:43:18.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 -- 172.21.15.32:0/3826125238 wait complete. 2023-12-20T04:43:18.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 Processor -- start 2023-12-20T04:43:18.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 -- start start 2023-12-20T04:43:18.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f519c06ce60 0x7f519c0b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519c0af5e0 0x7f519c15a630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f519c15f110 0x7f519c14ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c14de10 con 0x7f519c06ce60 2023-12-20T04:43:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f519c0b1fe0 con 0x7f519c0af5e0 2023-12-20T04:43:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a2a1f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f519c14eab0 con 0x7f519c15f110 2023-12-20T04:43:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f519bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f519c06ce60 0x7f519c0b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f519bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f519c06ce60 0x7f519c0b1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37178/0 (socket says 172.21.15.32:37178) 2023-12-20T04:43:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f519bfff640 1 -- 172.21.15.32:0/2518344650 learned_addr learned my addr 172.21.15.32:0/2518344650 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f519b7fe640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519c0af5e0 0x7f519c15a630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.474+0000 7f51a0fbc640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f519c15f110 0x7f519c14ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f519bfff640 1 -- 172.21.15.32:0/2518344650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f519c15f110 msgr2=0x7f519c14ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f519bfff640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f519c15f110 0x7f519c14ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f519bfff640 1 -- 172.21.15.32:0/2518344650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519c0af5e0 msgr2=0x7f519c15a630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f519bfff640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519c0af5e0 0x7f519c15a630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f519bfff640 1 -- 172.21.15.32:0/2518344650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5194009b30 con 0x7f519c06ce60 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f519b7fe640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519c0af5e0 0x7f519c15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f51a0fbc640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f519c15f110 0x7f519c14ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f519bfff640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f519c06ce60 0x7f519c0b1aa0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d800 tx=0x7f518c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f5198ff9640 1 -- 172.21.15.32:0/2518344650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c018070 con 0x7f519c06ce60 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f5198ff9640 1 -- 172.21.15.32:0/2518344650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f518c0059a0 con 0x7f519c06ce60 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f5198ff9640 1 -- 172.21.15.32:0/2518344650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f518c005b50 con 0x7f519c06ce60 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f51a2a1f640 1 -- 172.21.15.32:0/2518344650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f519c14ed90 con 0x7f519c06ce60 2023-12-20T04:43:18.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f51a2a1f640 1 -- 172.21.15.32:0/2518344650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f519c0b3750 con 0x7f519c06ce60 2023-12-20T04:43:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f51a2a1f640 1 -- 172.21.15.32:0/2518344650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f519c0b10f0 con 0x7f519c06ce60 2023-12-20T04:43:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f5198ff9640 1 -- 172.21.15.32:0/2518344650 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f518c005cb0 con 0x7f519c06ce60 2023-12-20T04:43:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f5198ff9640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065b80 0x7f5188068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f5198ff9640 1 -- 172.21.15.32:0/2518344650 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f518c082e90 con 0x7f519c06ce60 2023-12-20T04:43:18.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.478+0000 7f519b7fe640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065b80 0x7f5188068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.482+0000 7f519b7fe640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065b80 0x7f5188068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f519c151b90 tx=0x7f5190008040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:18.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.482+0000 7f5198ff9640 1 -- 172.21.15.32:0/2518344650 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f518c050a80 con 0x7f519c06ce60 2023-12-20T04:43:18.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.646+0000 7f51a2a1f640 1 -- 172.21.15.32:0/2518344650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f519c14efd0 con 0x7f519c06ce60 2023-12-20T04:43:18.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.646+0000 7f5198ff9640 1 -- 172.21.15.32:0/2518344650 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v75) v1 ==== 74+0+12341 (secure 0 0 0) 0x7f518c084020 con 0x7f519c06ce60 2023-12-20T04:43:18.645 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:18.646 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":75,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:17.884841+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":7,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":63,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":59,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:18.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 -- 172.21.15.32:0/2518344650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065b80 msgr2=0x7f5188068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065b80 0x7f5188068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f519c151b90 tx=0x7f5190008040 comp rx=0 tx=0).stop 2023-12-20T04:43:18.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 -- 172.21.15.32:0/2518344650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f519c06ce60 msgr2=0x7f519c0b1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f519c06ce60 0x7f519c0b1aa0 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7f518c00d800 tx=0x7f518c00dcc0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 -- 172.21.15.32:0/2518344650 shutdown_connections 2023-12-20T04:43:18.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5188065b80 0x7f5188068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f519c15f110 0x7f519c14ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f519c0af5e0 0x7f519c15a630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 --2- 172.21.15.32:0/2518344650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f519c06ce60 0x7f519c0b1aa0 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 -- 172.21.15.32:0/2518344650 >> 172.21.15.32:0/2518344650 conn(0x7f519c06a320 msgr2=0x7f519c15df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:18.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 -- 172.21.15.32:0/2518344650 shutdown_connections 2023-12-20T04:43:18.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.650+0000 7f51a2a1f640 1 -- 172.21.15.32:0/2518344650 wait complete. 2023-12-20T04:43:18.667 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:43:18.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:43:18.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.814+0000 7f67765b0640 1 Processor -- start 2023-12-20T04:43:18.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.814+0000 7f67765b0640 1 -- start start 2023-12-20T04:43:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f67765b0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f677015ccf0 0x7f67700cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f67765b0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67700d10f0 con 0x7f67700cfaf0 2023-12-20T04:43:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f67765b0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67700cdd30 con 0x7f67700cd7b0 2023-12-20T04:43:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f67765b0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67700ce2f0 con 0x7f677015ccf0 2023-12-20T04:43:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f67755ae640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f677015ccf0 0x7f67700cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f67755ae640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f677015ccf0 0x7f67700cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43334/0 (socket says 172.21.15.32:43334) 2023-12-20T04:43:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f67755ae640 1 -- 172.21.15.32:0/3359961762 learned_addr learned my addr 172.21.15.32:0/3359961762 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:18.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f6775daf640 1 --1- 172.21.15.32:0/3359961762 >> v1:172.21.15.32:6789/0 conn(0x7f67700cfaf0 0x7f67700d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54946/0 (socket says 172.21.15.32:54946) 2023-12-20T04:43:18.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f67755ae640 1 -- 172.21.15.32:0/3359961762 >> v1:172.21.15.39:6789/0 conn(0x7f67700cd7b0 legacy=0x7f67700cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:18.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f67755ae640 1 -- 172.21.15.32:0/3359961762 >> v1:172.21.15.32:6789/0 conn(0x7f67700cfaf0 legacy=0x7f67700d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f67755ae640 1 -- 172.21.15.32:0/3359961762 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67700ce010 con 0x7f677015ccf0 2023-12-20T04:43:18.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.818+0000 7f6767fff640 1 -- 172.21.15.32:0/3359961762 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4218289431 0 0) 0x7f67700d10f0 con 0x7f67700cfaf0 2023-12-20T04:43:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f67755ae640 1 --2- 172.21.15.32:0/3359961762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677015ccf0 0x7f67700cd270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f676c009870 tx=0x7f676c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=812e36687d3583c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:18.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f6767fff640 1 -- 172.21.15.32:0/3359961762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c0ab600 con 0x7f677015ccf0 2023-12-20T04:43:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f6767fff640 1 -- 172.21.15.32:0/3359961762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f676c0a7050 con 0x7f677015ccf0 2023-12-20T04:43:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f6767fff640 1 -- 172.21.15.32:0/3359961762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f676c0ab980 con 0x7f677015ccf0 2023-12-20T04:43:18.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f67765b0640 1 -- 172.21.15.32:0/3359961762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677015ccf0 msgr2=0x7f67700cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f67765b0640 1 --2- 172.21.15.32:0/3359961762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677015ccf0 0x7f67700cd270 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7f676c009870 tx=0x7f676c01c470 comp rx=0 tx=0).stop 2023-12-20T04:43:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f67765b0640 1 -- 172.21.15.32:0/3359961762 shutdown_connections 2023-12-20T04:43:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f67765b0640 1 --2- 172.21.15.32:0/3359961762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f677015ccf0 0x7f67700cd270 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f67765b0640 1 -- 172.21.15.32:0/3359961762 >> 172.21.15.32:0/3359961762 conn(0x7f67700658e0 msgr2=0x7f6770065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f67765b0640 1 -- 172.21.15.32:0/3359961762 shutdown_connections 2023-12-20T04:43:18.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f67765b0640 1 -- 172.21.15.32:0/3359961762 wait complete. 2023-12-20T04:43:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f67765b0640 1 Processor -- start 2023-12-20T04:43:18.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.822+0000 7f67765b0640 1 -- start start 2023-12-20T04:43:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f67765b0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67700cd7b0 0x7f67700c88f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f67765b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67700cfaf0 0x7f6770149700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f67765b0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f677015ccf0 0x7f67700c6f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f67765b0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67700c7850 con 0x7f67700cfaf0 2023-12-20T04:43:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f67765b0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67700c7b30 con 0x7f677015ccf0 2023-12-20T04:43:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f67765b0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67700c7e10 con 0x7f67700cd7b0 2023-12-20T04:43:18.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6774dad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67700cfaf0 0x7f6770149700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6774dad640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67700cfaf0 0x7f6770149700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43344/0 (socket says 172.21.15.32:43344) 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6774dad640 1 -- 172.21.15.32:0/869562772 learned_addr learned my addr 172.21.15.32:0/869562772 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f67755ae640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67700cd7b0 0x7f67700c88f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6775daf640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f677015ccf0 0x7f67700c6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6774dad640 1 -- 172.21.15.32:0/869562772 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67700cd7b0 msgr2=0x7f67700c88f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6774dad640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67700cd7b0 0x7f67700c88f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6774dad640 1 -- 172.21.15.32:0/869562772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f677015ccf0 msgr2=0x7f67700c6f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6774dad640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f677015ccf0 0x7f67700c6f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6774dad640 1 -- 172.21.15.32:0/869562772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f676c0094d0 con 0x7f67700cfaf0 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6775daf640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f677015ccf0 0x7f67700c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6774dad640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67700cfaf0 0x7f6770149700 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f676000c450 tx=0x7f676000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6765ffb640 1 -- 172.21.15.32:0/869562772 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760012830 con 0x7f67700cfaf0 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6765ffb640 1 -- 172.21.15.32:0/869562772 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67600129b0 con 0x7f67700cfaf0 2023-12-20T04:43:18.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f67765b0640 1 -- 172.21.15.32:0/869562772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67700c80f0 con 0x7f67700cfaf0 2023-12-20T04:43:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6765ffb640 1 -- 172.21.15.32:0/869562772 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6760012d60 con 0x7f67700cfaf0 2023-12-20T04:43:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f67765b0640 1 -- 172.21.15.32:0/869562772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6770149c40 con 0x7f67700cfaf0 2023-12-20T04:43:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f67765b0640 1 -- 172.21.15.32:0/869562772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6770074100 con 0x7f67700cfaf0 2023-12-20T04:43:18.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.826+0000 7f6765ffb640 1 -- 172.21.15.32:0/869562772 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f676001c460 con 0x7f67700cfaf0 2023-12-20T04:43:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.830+0000 7f6765ffb640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6754065b80 0x7f6754068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.830+0000 7f6765ffb640 1 -- 172.21.15.32:0/869562772 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f6760082db0 con 0x7f67700cfaf0 2023-12-20T04:43:18.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.830+0000 7f67755ae640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6754065b80 0x7f6754068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:18.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.830+0000 7f67755ae640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6754065b80 0x7f6754068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ad000 tx=0x7f676c0a4f90 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:18.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.834+0000 7f6765ffb640 1 -- 172.21.15.32:0/869562772 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f67600509b0 con 0x7f67700cfaf0 2023-12-20T04:43:18.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.990+0000 7f67765b0640 1 -- 172.21.15.32:0/869562772 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f67700b8380 con 0x7f67700cfaf0 2023-12-20T04:43:18.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.990+0000 7f6765ffb640 1 -- 172.21.15.32:0/869562772 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v75) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6760053fd0 con 0x7f67700cfaf0 2023-12-20T04:43:18.991 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:43:18.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 -- 172.21.15.32:0/869562772 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6754065b80 msgr2=0x7f6754068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6754065b80 0x7f6754068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f676c0ad000 tx=0x7f676c0a4f90 comp rx=0 tx=0).stop 2023-12-20T04:43:18.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 -- 172.21.15.32:0/869562772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67700cfaf0 msgr2=0x7f6770149700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:18.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67700cfaf0 0x7f6770149700 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7f676000c450 tx=0x7f676000c910 comp rx=0 tx=0).stop 2023-12-20T04:43:18.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 -- 172.21.15.32:0/869562772 shutdown_connections 2023-12-20T04:43:18.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6754065b80 0x7f6754068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f677015ccf0 0x7f67700c6f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67700cfaf0 0x7f6770149700 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 --2- 172.21.15.32:0/869562772 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67700cd7b0 0x7f67700c88f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:18.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 -- 172.21.15.32:0/869562772 >> 172.21.15.32:0/869562772 conn(0x7f67700658e0 msgr2=0x7f67700ca860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:18.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 -- 172.21.15.32:0/869562772 shutdown_connections 2023-12-20T04:43:18.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:18.994+0000 7f67765b0640 1 -- 172.21.15.32:0/869562772 wait complete. 2023-12-20T04:43:19.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T04:43:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.158+0000 7fba0fc2d640 1 Processor -- start 2023-12-20T04:43:19.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.158+0000 7fba0fc2d640 1 -- start start 2023-12-20T04:43:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.158+0000 7fba0fc2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 0x7fba080b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.158+0000 7fba0fc2d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fba080b0b30 0x7fba080701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.158+0000 7fba0fc2d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba080c98c0 con 0x7fba081599e0 2023-12-20T04:43:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.162+0000 7fba0fc2d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba080c9ba0 con 0x7fba080b0b30 2023-12-20T04:43:19.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.162+0000 7fba0fc2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba080c9e80 con 0x7fba080aece0 2023-12-20T04:43:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.162+0000 7fba0d1c8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 0x7fba080b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.162+0000 7fba0d1c8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 0x7fba080b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:41924/0 (socket says 172.21.15.32:41924) 2023-12-20T04:43:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.162+0000 7fba0d1c8640 1 -- 172.21.15.32:0/3741840371 learned_addr learned my addr 172.21.15.32:0/3741840371 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.162+0000 7fba0e1ca640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fba080b0b30 0x7fba080701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.162+0000 7fba0d1c8640 1 -- 172.21.15.32:0/3741840371 >> v2:172.21.15.39:3300/0 conn(0x7fba080b0b30 msgr2=0x7fba080701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.162+0000 7fba0d1c8640 1 --2- 172.21.15.32:0/3741840371 >> v2:172.21.15.39:3300/0 conn(0x7fba080b0b30 0x7fba080701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.162+0000 7fba0d1c8640 1 -- 172.21.15.32:0/3741840371 >> v1:172.21.15.39:6789/0 conn(0x7fba081599e0 legacy=0x7fba080c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:19.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.162+0000 7fba0d1c8640 1 -- 172.21.15.32:0/3741840371 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba080ca160 con 0x7fba080aece0 2023-12-20T04:43:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.162+0000 7fba0d1c8640 1 --2- 172.21.15.32:0/3741840371 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 0x7fba080b05f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009870 tx=0x7fb9f801c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e20a90113eca11e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0c9c7640 1 -- 172.21.15.32:0/3741840371 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80ab650 con 0x7fba080aece0 2023-12-20T04:43:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0c9c7640 1 -- 172.21.15.32:0/3741840371 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9f80ab7b0 con 0x7fba080aece0 2023-12-20T04:43:19.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0c9c7640 1 -- 172.21.15.32:0/3741840371 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80abb40 con 0x7fba080aece0 2023-12-20T04:43:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 -- 172.21.15.32:0/3741840371 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 msgr2=0x7fba080b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 --2- 172.21.15.32:0/3741840371 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 0x7fba080b05f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009870 tx=0x7fb9f801c540 comp rx=0 tx=0).stop 2023-12-20T04:43:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 -- 172.21.15.32:0/3741840371 shutdown_connections 2023-12-20T04:43:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 --2- 172.21.15.32:0/3741840371 >> v2:172.21.15.39:3300/0 conn(0x7fba080b0b30 0x7fba080701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 --2- 172.21.15.32:0/3741840371 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 0x7fba080b05f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 -- 172.21.15.32:0/3741840371 >> 172.21.15.32:0/3741840371 conn(0x7fba0806a320 msgr2=0x7fba080ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 -- 172.21.15.32:0/3741840371 shutdown_connections 2023-12-20T04:43:19.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 -- 172.21.15.32:0/3741840371 wait complete. 2023-12-20T04:43:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 Processor -- start 2023-12-20T04:43:19.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 -- start start 2023-12-20T04:43:19.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 0x7fba0814f3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba080b0b30 0x7fba081f62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba081599e0 0x7fba08158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0814f930 con 0x7fba080b0b30 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba0814fe30 con 0x7fba081599e0 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0fc2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba080c6200 con 0x7fba080aece0 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0d1c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba080b0b30 0x7fba081f62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0d1c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba080b0b30 0x7fba081f62b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43356/0 (socket says 172.21.15.32:43356) 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0d1c8640 1 -- 172.21.15.32:0/2334304181 learned_addr learned my addr 172.21.15.32:0/2334304181 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.166+0000 7fba0d9c9640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 0x7fba0814f3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fba0e1ca640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba081599e0 0x7fba08158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fba0d9c9640 1 -- 172.21.15.32:0/2334304181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba081599e0 msgr2=0x7fba08158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fba0d9c9640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba081599e0 0x7fba08158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fba0d9c9640 1 -- 172.21.15.32:0/2334304181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba080b0b30 msgr2=0x7fba081f62b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fba0d9c9640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba080b0b30 0x7fba081f62b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fba0d9c9640 1 -- 172.21.15.32:0/2334304181 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9f80094d0 con 0x7fba080aece0 2023-12-20T04:43:19.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fba0d1c8640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba080b0b30 0x7fba081f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fba0d9c9640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 0x7fba0814f3f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c390 tx=0x7fba0400c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fb9fe7fc640 1 -- 172.21.15.32:0/2334304181 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba040052d0 con 0x7fba080aece0 2023-12-20T04:43:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fb9fe7fc640 1 -- 172.21.15.32:0/2334304181 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba04005450 con 0x7fba080aece0 2023-12-20T04:43:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fba0fc2d640 1 -- 172.21.15.32:0/2334304181 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba080c64e0 con 0x7fba080aece0 2023-12-20T04:43:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fb9fe7fc640 1 -- 172.21.15.32:0/2334304181 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba04003890 con 0x7fba080aece0 2023-12-20T04:43:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fba0fc2d640 1 -- 172.21.15.32:0/2334304181 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba080c6aa0 con 0x7fba080aece0 2023-12-20T04:43:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fba0fc2d640 1 -- 172.21.15.32:0/2334304181 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba080cef50 con 0x7fba080aece0 2023-12-20T04:43:19.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.170+0000 7fb9fe7fc640 1 -- 172.21.15.32:0/2334304181 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fba040039f0 con 0x7fba080aece0 2023-12-20T04:43:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.174+0000 7fb9fe7fc640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9ec065bb0 0x7fb9ec068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.174+0000 7fb9fe7fc640 1 -- 172.21.15.32:0/2334304181 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fba04082410 con 0x7fba080aece0 2023-12-20T04:43:19.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.174+0000 7fba0d1c8640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9ec065bb0 0x7fb9ec068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.174+0000 7fba0d1c8640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9ec065bb0 0x7fb9ec068070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009870 tx=0x7fb9f80b6040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:19.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.178+0000 7fb9fe7fc640 1 -- 172.21.15.32:0/2334304181 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fba040500b0 con 0x7fba080aece0 2023-12-20T04:43:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.338+0000 7fba0fc2d640 1 -- 172.21.15.32:0/2334304181 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fba0804e000 con 0x7fba080aece0 2023-12-20T04:43:19.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.338+0000 7fb9fe7fc640 1 -- 172.21.15.32:0/2334304181 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v75) v1 ==== 84+0+65 (secure 0 0 0) 0x7fba04017060 con 0x7fba080aece0 2023-12-20T04:43:19.338 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:19.338 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:43:19.338 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:19.338 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T04:43:19.338 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T04:43:19.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:19.338 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 -- 172.21.15.32:0/2334304181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9ec065bb0 msgr2=0x7fb9ec068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9ec065bb0 0x7fb9ec068070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fb9f8009870 tx=0x7fb9f80b6040 comp rx=0 tx=0).stop 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 -- 172.21.15.32:0/2334304181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 msgr2=0x7fba0814f3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 0x7fba0814f3f0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fba0400c390 tx=0x7fba0400c850 comp rx=0 tx=0).stop 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 -- 172.21.15.32:0/2334304181 shutdown_connections 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb9ec065bb0 0x7fb9ec068070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba081599e0 0x7fba08158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba080b0b30 0x7fba081f62b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 --2- 172.21.15.32:0/2334304181 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba080aece0 0x7fba0814f3f0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 -- 172.21.15.32:0/2334304181 >> 172.21.15.32:0/2334304181 conn(0x7fba0806a320 msgr2=0x7fba080719d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 -- 172.21.15.32:0/2334304181 shutdown_connections 2023-12-20T04:43:19.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.342+0000 7fba0fc2d640 1 -- 172.21.15.32:0/2334304181 wait complete. 2023-12-20T04:43:19.359 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T04:43:19.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T04:43:19.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7157a20640 1 Processor -- start 2023-12-20T04:43:19.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7157a20640 1 -- start start 2023-12-20T04:43:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7157a20640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f715006dcb0 0x7f71500ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7157a20640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7150070230 0x7f71500706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7157a20640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f715006e100 con 0x7f7150158520 2023-12-20T04:43:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7157a20640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7150070be0 con 0x7f7150070230 2023-12-20T04:43:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7157a20640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71500ca590 con 0x7f715006dcb0 2023-12-20T04:43:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7155fbd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7150070230 0x7f71500706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f71557bc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7150158520 0x7f715015ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54958/0 (socket says 172.21.15.32:54958) 2023-12-20T04:43:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f71557bc640 1 -- 172.21.15.32:0/365347481 learned_addr learned my addr 172.21.15.32:0/365347481 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7154fbb640 1 --2- 172.21.15.32:0/365347481 >> v2:172.21.15.39:3300/0 conn(0x7f715006dcb0 0x7f71500ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7155fbd640 1 -- 172.21.15.32:0/365347481 >> v2:172.21.15.39:3300/0 conn(0x7f715006dcb0 msgr2=0x7f71500ca050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7155fbd640 1 --2- 172.21.15.32:0/365347481 >> v2:172.21.15.39:3300/0 conn(0x7f715006dcb0 0x7f71500ca050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7155fbd640 1 -- 172.21.15.32:0/365347481 >> v1:172.21.15.32:6789/0 conn(0x7f7150158520 legacy=0x7f715015ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7155fbd640 1 -- 172.21.15.32:0/365347481 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71500ca7f0 con 0x7f7150070230 2023-12-20T04:43:19.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7147fff640 1 -- 172.21.15.32:0/365347481 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214602747 0 0) 0x7f714c001120 con 0x7f7150158520 2023-12-20T04:43:19.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.510+0000 7f7154fbb640 1 --2- 172.21.15.32:0/365347481 >> v2:172.21.15.39:3300/0 conn(0x7f715006dcb0 0x7f71500ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:19.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7155fbd640 1 --2- 172.21.15.32:0/365347481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7150070230 0x7f71500706a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f71500cf830 tx=0x7f714801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67d2ae1651fd4719 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:19.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7147fff640 1 -- 172.21.15.32:0/365347481 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71480a5910 con 0x7f7150070230 2023-12-20T04:43:19.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7147fff640 1 -- 172.21.15.32:0/365347481 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71480a5a70 con 0x7f7150070230 2023-12-20T04:43:19.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7147fff640 1 -- 172.21.15.32:0/365347481 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71480a5e00 con 0x7f7150070230 2023-12-20T04:43:19.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7157a20640 1 -- 172.21.15.32:0/365347481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7150070230 msgr2=0x7f71500706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:19.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7157a20640 1 --2- 172.21.15.32:0/365347481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7150070230 0x7f71500706a0 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7f71500cf830 tx=0x7f714801c850 comp rx=0 tx=0).stop 2023-12-20T04:43:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7157a20640 1 -- 172.21.15.32:0/365347481 shutdown_connections 2023-12-20T04:43:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7157a20640 1 --2- 172.21.15.32:0/365347481 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7150070230 0x7f71500706a0 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7157a20640 1 --2- 172.21.15.32:0/365347481 >> v2:172.21.15.39:3300/0 conn(0x7f715006dcb0 0x7f71500ca050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7157a20640 1 -- 172.21.15.32:0/365347481 >> 172.21.15.32:0/365347481 conn(0x7f715006a510 msgr2=0x7f71500ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7157a20640 1 -- 172.21.15.32:0/365347481 shutdown_connections 2023-12-20T04:43:19.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.514+0000 7f7157a20640 1 -- 172.21.15.32:0/365347481 wait complete. 2023-12-20T04:43:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7157a20640 1 Processor -- start 2023-12-20T04:43:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7157a20640 1 -- start start 2023-12-20T04:43:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7157a20640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715006dcb0 0x7f715014d2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7157a20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7150070230 0x7f7150152810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7157a20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7150158520 0x7f71500cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7157a20640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f715014ef10 con 0x7f715006dcb0 2023-12-20T04:43:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7157a20640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7150153300 con 0x7f7150158520 2023-12-20T04:43:19.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7157a20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f715014f140 con 0x7f7150070230 2023-12-20T04:43:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f71557bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715006dcb0 0x7f715014d2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f71557bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715006dcb0 0x7f715014d2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43380/0 (socket says 172.21.15.32:43380) 2023-12-20T04:43:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f71557bc640 1 -- 172.21.15.32:0/678867190 learned_addr learned my addr 172.21.15.32:0/678867190 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7155fbd640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7150158520 0x7f71500cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7154fbb640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7150070230 0x7f7150152810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f71557bc640 1 -- 172.21.15.32:0/678867190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7150070230 msgr2=0x7f7150152810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f71557bc640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7150070230 0x7f7150152810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f71557bc640 1 -- 172.21.15.32:0/678867190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7150158520 msgr2=0x7f71500cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f71557bc640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7150158520 0x7f71500cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f71557bc640 1 -- 172.21.15.32:0/678867190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7148009a70 con 0x7f715006dcb0 2023-12-20T04:43:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f71557bc640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715006dcb0 0x7f715014d2e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f714c00c480 tx=0x7f714c00c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7145ffb640 1 -- 172.21.15.32:0/678867190 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c0074e0 con 0x7f715006dcb0 2023-12-20T04:43:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7145ffb640 1 -- 172.21.15.32:0/678867190 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f714c007660 con 0x7f715006dcb0 2023-12-20T04:43:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7145ffb640 1 -- 172.21.15.32:0/678867190 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f714c004a30 con 0x7f715006dcb0 2023-12-20T04:43:19.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7157a20640 1 -- 172.21.15.32:0/678867190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f715014f420 con 0x7f715006dcb0 2023-12-20T04:43:19.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7157a20640 1 -- 172.21.15.32:0/678867190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f715014f9e0 con 0x7f715006dcb0 2023-12-20T04:43:19.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.518+0000 7f7157a20640 1 -- 172.21.15.32:0/678867190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f715004e000 con 0x7f715006dcb0 2023-12-20T04:43:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.522+0000 7f7145ffb640 1 -- 172.21.15.32:0/678867190 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f714c004b90 con 0x7f715006dcb0 2023-12-20T04:43:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.522+0000 7f7145ffb640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7134065b80 0x7f7134068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.522+0000 7f7145ffb640 1 -- 172.21.15.32:0/678867190 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(75..75 src has 1..75) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f714c081e50 con 0x7f715006dcb0 2023-12-20T04:43:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.522+0000 7f7154fbb640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7134065b80 0x7f7134068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:19.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.522+0000 7f7154fbb640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7134065b80 0x7f7134068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7150157f40 tx=0x7f7140008040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:19.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.526+0000 7f7145ffb640 1 -- 172.21.15.32:0/678867190 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f714c04fb90 con 0x7f715006dcb0 2023-12-20T04:43:19.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:19.682+0000 7f7157a20640 1 -- 172.21.15.32:0/678867190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f715006e100 con 0x7f715006dcb0 2023-12-20T04:43:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.558+0000 7f7145ffb640 1 -- 172.21.15.32:0/678867190 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v76) v1 ==== 138+0+0 (secure 0 0 0) 0x7f714c018020 con 0x7f715006dcb0 2023-12-20T04:43:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T04:43:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 -- 172.21.15.32:0/678867190 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7134065b80 msgr2=0x7f7134068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7134065b80 0x7f7134068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f7150157f40 tx=0x7f7140008040 comp rx=0 tx=0).stop 2023-12-20T04:43:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 -- 172.21.15.32:0/678867190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715006dcb0 msgr2=0x7f715014d2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715006dcb0 0x7f715014d2e0 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f714c00c480 tx=0x7f714c00c940 comp rx=0 tx=0).stop 2023-12-20T04:43:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 -- 172.21.15.32:0/678867190 shutdown_connections 2023-12-20T04:43:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7134065b80 0x7f7134068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7150158520 0x7f71500cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7150070230 0x7f7150152810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 --2- 172.21.15.32:0/678867190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f715006dcb0 0x7f715014d2e0 secure :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f714c00c480 tx=0x7f714c00c940 comp rx=0 tx=0).stop 2023-12-20T04:43:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 -- 172.21.15.32:0/678867190 >> 172.21.15.32:0/678867190 conn(0x7f715006a510 msgr2=0x7f715015b8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 -- 172.21.15.32:0/678867190 shutdown_connections 2023-12-20T04:43:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.562+0000 7f7157a20640 1 -- 172.21.15.32:0/678867190 wait complete. 2023-12-20T04:43:20.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T04:43:20.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd51cd7640 1 Processor -- start 2023-12-20T04:43:20.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd51cd7640 1 -- start start 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd51cd7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbd4c0cd940 0x7fbd4c1606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd51cd7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 0x7fbd4c0d13a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd51cd7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd4c0d0c10 con 0x7fbd4c0b8410 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd51cd7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd4c160f80 con 0x7fbd4c0cd940 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd51cd7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c161240 con 0x7fbd4c0cdd20 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd50cd5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbd4c0cd940 0x7fbd4c1606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd514d6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 0x7fbd4c0d13a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd50cd5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbd4c0cd940 0x7fbd4c1606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:42430/0 (socket says 172.21.15.32:42430) 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd514d6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 0x7fbd4c0d13a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:41958/0 (socket says 172.21.15.32:41958) 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.742+0000 7fbd50cd5640 1 -- 172.21.15.32:0/1591966899 learned_addr learned my addr 172.21.15.32:0/1591966899 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.746+0000 7fbd514d6640 1 -- 172.21.15.32:0/1591966899 >> v2:172.21.15.39:3300/0 conn(0x7fbd4c0cd940 msgr2=0x7fbd4c1606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.746+0000 7fbd514d6640 1 --2- 172.21.15.32:0/1591966899 >> v2:172.21.15.39:3300/0 conn(0x7fbd4c0cd940 0x7fbd4c1606e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:20.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.746+0000 7fbd514d6640 1 -- 172.21.15.32:0/1591966899 >> v1:172.21.15.39:6789/0 conn(0x7fbd4c0b8410 legacy=0x7fbd4c15c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:20.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.746+0000 7fbd514d6640 1 -- 172.21.15.32:0/1591966899 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd4c0d0e80 con 0x7fbd4c0cdd20 2023-12-20T04:43:20.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.746+0000 7fbd50cd5640 1 --2- 172.21.15.32:0/1591966899 >> v2:172.21.15.39:3300/0 conn(0x7fbd4c0cd940 0x7fbd4c1606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:20.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.746+0000 7fbd514d6640 1 --2- 172.21.15.32:0/1591966899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 0x7fbd4c0d13a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009d90 tx=0x7fbd4401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=64b81af2d3039127 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:20.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.746+0000 7fbd4b7fe640 1 -- 172.21.15.32:0/1591966899 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd440a5a30 con 0x7fbd4c0cdd20 2023-12-20T04:43:20.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.746+0000 7fbd4b7fe640 1 -- 172.21.15.32:0/1591966899 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd440a5b90 con 0x7fbd4c0cdd20 2023-12-20T04:43:20.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.746+0000 7fbd4b7fe640 1 -- 172.21.15.32:0/1591966899 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd440ad440 con 0x7fbd4c0cdd20 2023-12-20T04:43:20.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1591966899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 msgr2=0x7fbd4c0d13a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:20.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 --2- 172.21.15.32:0/1591966899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 0x7fbd4c0d13a0 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7fbd44009d90 tx=0x7fbd4401c950 comp rx=0 tx=0).stop 2023-12-20T04:43:20.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1591966899 shutdown_connections 2023-12-20T04:43:20.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 --2- 172.21.15.32:0/1591966899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 0x7fbd4c0d13a0 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:20.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 --2- 172.21.15.32:0/1591966899 >> v2:172.21.15.39:3300/0 conn(0x7fbd4c0cd940 0x7fbd4c1606e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:20.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1591966899 >> 172.21.15.32:0/1591966899 conn(0x7fbd4c0658e0 msgr2=0x7fbd4c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:20.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1591966899 shutdown_connections 2023-12-20T04:43:20.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1591966899 wait complete. 2023-12-20T04:43:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 Processor -- start 2023-12-20T04:43:20.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 -- start start 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd4c0b8410 0x7fbd4c15a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd4c0cd940 0x7fbd4c0c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 0x7fbd4c1541b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c14f450 con 0x7fbd4c0cd940 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c0c5760 con 0x7fbd4c0b8410 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd51cd7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd4c14e7d0 con 0x7fbd4c0cdd20 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd4c0cd940 0x7fbd4c0c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd4bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd4c0cd940 0x7fbd4c0c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43390/0 (socket says 172.21.15.32:43390) 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd4bfff640 1 -- 172.21.15.32:0/1364176959 learned_addr learned my addr 172.21.15.32:0/1364176959 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd50cd5640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd4c0b8410 0x7fbd4c15a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd514d6640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 0x7fbd4c1541b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd4bfff640 1 -- 172.21.15.32:0/1364176959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 msgr2=0x7fbd4c1541b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd4bfff640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 0x7fbd4c1541b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:20.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd4bfff640 1 -- 172.21.15.32:0/1364176959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd4c0b8410 msgr2=0x7fbd4c15a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:20.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd4bfff640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd4c0b8410 0x7fbd4c15a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:20.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.750+0000 7fbd4bfff640 1 -- 172.21.15.32:0/1364176959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd400094d0 con 0x7fbd4c0cd940 2023-12-20T04:43:20.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.754+0000 7fbd4bfff640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd4c0cd940 0x7fbd4c0c5220 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800c500 tx=0x7fbd3800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:20.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.754+0000 7fbd497fa640 1 -- 172.21.15.32:0/1364176959 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38005220 con 0x7fbd4c0cd940 2023-12-20T04:43:20.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.754+0000 7fbd497fa640 1 -- 172.21.15.32:0/1364176959 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd38005380 con 0x7fbd4c0cd940 2023-12-20T04:43:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.754+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1364176959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd44009a70 con 0x7fbd4c0cd940 2023-12-20T04:43:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.754+0000 7fbd497fa640 1 -- 172.21.15.32:0/1364176959 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd38007270 con 0x7fbd4c0cd940 2023-12-20T04:43:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.754+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1364176959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd4c14ed90 con 0x7fbd4c0cd940 2023-12-20T04:43:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.754+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1364176959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd4c14f590 con 0x7fbd4c0cd940 2023-12-20T04:43:20.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.754+0000 7fbd497fa640 1 -- 172.21.15.32:0/1364176959 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbd380073d0 con 0x7fbd4c0cd940 2023-12-20T04:43:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.754+0000 7fbd497fa640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd30065b80 0x7fbd30068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:20.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.754+0000 7fbd497fa640 1 -- 172.21.15.32:0/1364176959 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(76..76 src has 1..76) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fbd38082960 con 0x7fbd4c0cd940 2023-12-20T04:43:20.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.754+0000 7fbd50cd5640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd30065b80 0x7fbd30068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:20.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.758+0000 7fbd50cd5640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd30065b80 0x7fbd30068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd40006cb0 tx=0x7fbd40009240 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:20.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.762+0000 7fbd497fa640 1 -- 172.21.15.32:0/1364176959 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbd4c14f590 con 0x7fbd4c0cd940 2023-12-20T04:43:20.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:20.918+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1364176959 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fbd4c14f590 con 0x7fbd4c0cd940 2023-12-20T04:43:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.562+0000 7fbd497fa640 1 -- 172.21.15.32:0/1364176959 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v77) v1 ==== 170+0+0 (secure 0 0 0) 0x7fbd4c14f590 con 0x7fbd4c0cd940 2023-12-20T04:43:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T04:43:21.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.566+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1364176959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd30065b80 msgr2=0x7fbd30068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:21.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.566+0000 7fbd51cd7640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd30065b80 0x7fbd30068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbd40006cb0 tx=0x7fbd40009240 comp rx=0 tx=0).stop 2023-12-20T04:43:21.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.566+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1364176959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd4c0cd940 msgr2=0x7fbd4c0c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:21.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.566+0000 7fbd51cd7640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd4c0cd940 0x7fbd4c0c5220 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7fbd3800c500 tx=0x7fbd3800c9c0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.570+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1364176959 shutdown_connections 2023-12-20T04:43:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.570+0000 7fbd51cd7640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd30065b80 0x7fbd30068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.570+0000 7fbd51cd7640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd4c0cdd20 0x7fbd4c1541b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.570+0000 7fbd51cd7640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd4c0cd940 0x7fbd4c0c5220 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.570+0000 7fbd51cd7640 1 --2- 172.21.15.32:0/1364176959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd4c0b8410 0x7fbd4c15a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.570+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1364176959 >> 172.21.15.32:0/1364176959 conn(0x7fbd4c0658e0 msgr2=0x7fbd4c0d29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.570+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1364176959 shutdown_connections 2023-12-20T04:43:21.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.574+0000 7fbd51cd7640 1 -- 172.21.15.32:0/1364176959 wait complete. 2023-12-20T04:43:21.605 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T04:43:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.754+0000 7f9a05220640 1 Processor -- start 2023-12-20T04:43:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.754+0000 7f9a05220640 1 -- start start 2023-12-20T04:43:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.754+0000 7f9a05220640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9a0015abb0 0x7f9a000caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.754+0000 7f9a05220640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a000b0cb0 0x7f9a0015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.754+0000 7f9a05220640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a0015b6a0 con 0x7f9a00070230 2023-12-20T04:43:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.754+0000 7f9a05220640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0006e160 con 0x7f9a000b0cb0 2023-12-20T04:43:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.754+0000 7f9a05220640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a0015c070 con 0x7f9a0015abb0 2023-12-20T04:43:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.754+0000 7f99f7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a000b0cb0 0x7f9a0015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.754+0000 7f99f7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a000b0cb0 0x7f9a0015bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43394/0 (socket says 172.21.15.32:43394) 2023-12-20T04:43:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.754+0000 7f99f7fff640 1 -- 172.21.15.32:0/1488791757 learned_addr learned my addr 172.21.15.32:0/1488791757 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:21.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.758+0000 7f99fed9d640 1 --2- 172.21.15.32:0/1488791757 >> v2:172.21.15.39:3300/0 conn(0x7f9a0015abb0 0x7f9a000caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:21.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.758+0000 7f99f7fff640 1 -- 172.21.15.32:0/1488791757 >> v2:172.21.15.39:3300/0 conn(0x7f9a0015abb0 msgr2=0x7f9a000caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:21.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.758+0000 7f99f7fff640 1 --2- 172.21.15.32:0/1488791757 >> v2:172.21.15.39:3300/0 conn(0x7f9a0015abb0 0x7f9a000caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.758+0000 7f99f7fff640 1 -- 172.21.15.32:0/1488791757 >> v1:172.21.15.39:6789/0 conn(0x7f9a00070230 legacy=0x7f9a0006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:21.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.758+0000 7f99f7fff640 1 -- 172.21.15.32:0/1488791757 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a0006d8c0 con 0x7f9a000b0cb0 2023-12-20T04:43:21.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.758+0000 7f99fed9d640 1 --2- 172.21.15.32:0/1488791757 >> v2:172.21.15.39:3300/0 conn(0x7f9a0015abb0 0x7f9a000caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:21.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.758+0000 7f99f7fff640 1 --2- 172.21.15.32:0/1488791757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b0cb0 0x7f9a0015bb30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9a000713f0 tx=0x7f99e801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d5634a2b7db1046 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:21.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.758+0000 7f99fe59c640 1 -- 172.21.15.32:0/1488791757 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80aa660 con 0x7f9a000b0cb0 2023-12-20T04:43:21.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.758+0000 7f99fe59c640 1 -- 172.21.15.32:0/1488791757 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e80aa7c0 con 0x7f9a000b0cb0 2023-12-20T04:43:21.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.758+0000 7f99fe59c640 1 -- 172.21.15.32:0/1488791757 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80aab50 con 0x7f9a000b0cb0 2023-12-20T04:43:21.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 -- 172.21.15.32:0/1488791757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b0cb0 msgr2=0x7f9a0015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:21.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 --2- 172.21.15.32:0/1488791757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b0cb0 0x7f9a0015bb30 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f9a000713f0 tx=0x7f99e801c500 comp rx=0 tx=0).stop 2023-12-20T04:43:21.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 -- 172.21.15.32:0/1488791757 shutdown_connections 2023-12-20T04:43:21.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 --2- 172.21.15.32:0/1488791757 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b0cb0 0x7f9a0015bb30 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 --2- 172.21.15.32:0/1488791757 >> v2:172.21.15.39:3300/0 conn(0x7f9a0015abb0 0x7f9a000caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 -- 172.21.15.32:0/1488791757 >> 172.21.15.32:0/1488791757 conn(0x7f9a0006a510 msgr2=0x7f9a000ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:21.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 -- 172.21.15.32:0/1488791757 shutdown_connections 2023-12-20T04:43:21.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 -- 172.21.15.32:0/1488791757 wait complete. 2023-12-20T04:43:21.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 Processor -- start 2023-12-20T04:43:21.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 -- start start 2023-12-20T04:43:21.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00070230 0x7f9a0013d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:21.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b0cb0 0x7f9a000c9070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:21.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a0015abb0 0x7f9a00158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:21.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a0013d790 con 0x7f9a000b0cb0 2023-12-20T04:43:21.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000c7fd0 con 0x7f9a00070230 2023-12-20T04:43:21.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f9a05220640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a000c82b0 con 0x7f9a0015abb0 2023-12-20T04:43:21.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f99f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b0cb0 0x7f9a000c9070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:21.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f99fed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00070230 0x7f9a0013d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:21.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f99f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b0cb0 0x7f9a000c9070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43404/0 (socket says 172.21.15.32:43404) 2023-12-20T04:43:21.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f99f7fff640 1 -- 172.21.15.32:0/4214390809 learned_addr learned my addr 172.21.15.32:0/4214390809 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:21.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f99ff59e640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a0015abb0 0x7f9a00158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f99f7fff640 1 -- 172.21.15.32:0/4214390809 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a0015abb0 msgr2=0x7f9a00158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f99f7fff640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a0015abb0 0x7f9a00158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f99f7fff640 1 -- 172.21.15.32:0/4214390809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00070230 msgr2=0x7f9a0013d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f99f7fff640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00070230 0x7f9a0013d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.762+0000 7f99f7fff640 1 -- 172.21.15.32:0/4214390809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99e80094d0 con 0x7f9a000b0cb0 2023-12-20T04:43:21.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.766+0000 7f99f7fff640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b0cb0 0x7f9a000c9070 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f99e8019110 tx=0x7f99e801ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:21.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.766+0000 7f99f77fe640 1 -- 172.21.15.32:0/4214390809 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80b3800 con 0x7f9a000b0cb0 2023-12-20T04:43:21.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.766+0000 7f99f77fe640 1 -- 172.21.15.32:0/4214390809 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99e80b3960 con 0x7f9a000b0cb0 2023-12-20T04:43:21.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.766+0000 7f9a05220640 1 -- 172.21.15.32:0/4214390809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a000c8590 con 0x7f9a000b0cb0 2023-12-20T04:43:21.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.766+0000 7f99f77fe640 1 -- 172.21.15.32:0/4214390809 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99e80b3cd0 con 0x7f9a000b0cb0 2023-12-20T04:43:21.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.766+0000 7f9a05220640 1 -- 172.21.15.32:0/4214390809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a000c8b50 con 0x7f9a000b0cb0 2023-12-20T04:43:21.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.766+0000 7f9a05220640 1 -- 172.21.15.32:0/4214390809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a0006acf0 con 0x7f9a000b0cb0 2023-12-20T04:43:21.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.766+0000 7f99f77fe640 1 -- 172.21.15.32:0/4214390809 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f99e80c63c0 con 0x7f9a000b0cb0 2023-12-20T04:43:21.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.770+0000 7f99f77fe640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f99e0065b80 0x7f99e0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:21.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.770+0000 7f99f77fe640 1 -- 172.21.15.32:0/4214390809 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(77..77 src has 1..77) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f99e80a6040 con 0x7f9a000b0cb0 2023-12-20T04:43:21.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.770+0000 7f99fed9d640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f99e0065b80 0x7f99e0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:21.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.770+0000 7f99fed9d640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f99e0065b80 0x7f99e0068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f99f00045f0 tx=0x7f99f0008040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:21.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.774+0000 7f99f77fe640 1 -- 172.21.15.32:0/4214390809 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f99e80eb900 con 0x7f9a000b0cb0 2023-12-20T04:43:21.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.966+0000 7f9a05220640 1 -- 172.21.15.32:0/4214390809 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f9a000c8d40 con 0x7f9a000b0cb0 2023-12-20T04:43:21.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.978+0000 7f99f77fe640 1 -- 172.21.15.32:0/4214390809 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 8 and data pool 9 v28) v1 ==== 172+0+0 (secure 0 0 0) 0x7f99e80eef20 con 0x7f9a000b0cb0 2023-12-20T04:43:21.980 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 8 and data pool 9 2023-12-20T04:43:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.982+0000 7f9a05220640 1 -- 172.21.15.32:0/4214390809 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f99e0065b80 msgr2=0x7f99e0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.982+0000 7f9a05220640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f99e0065b80 0x7f99e0068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f99f00045f0 tx=0x7f99f0008040 comp rx=0 tx=0).stop 2023-12-20T04:43:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.982+0000 7f9a05220640 1 -- 172.21.15.32:0/4214390809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b0cb0 msgr2=0x7f9a000c9070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:21.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.982+0000 7f9a05220640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b0cb0 0x7f9a000c9070 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f99e8019110 tx=0x7f99e801ad40 comp rx=0 tx=0).stop 2023-12-20T04:43:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.986+0000 7f9a05220640 1 -- 172.21.15.32:0/4214390809 shutdown_connections 2023-12-20T04:43:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.986+0000 7f9a05220640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f99e0065b80 0x7f99e0068040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.986+0000 7f9a05220640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a0015abb0 0x7f9a00158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.986+0000 7f9a05220640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a000b0cb0 0x7f9a000c9070 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.986+0000 7f9a05220640 1 --2- 172.21.15.32:0/4214390809 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a00070230 0x7f9a0013d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:21.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.986+0000 7f9a05220640 1 -- 172.21.15.32:0/4214390809 >> 172.21.15.32:0/4214390809 conn(0x7f9a0006a510 msgr2=0x7f9a000b29e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:21.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.990+0000 7f9a05220640 1 -- 172.21.15.32:0/4214390809 shutdown_connections 2023-12-20T04:43:21.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:21.994+0000 7f9a05220640 1 -- 172.21.15.32:0/4214390809 wait complete. 2023-12-20T04:43:22.025 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.170+0000 7fbf30466640 1 Processor -- start 2023-12-20T04:43:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.170+0000 7fbf30466640 1 -- start start 2023-12-20T04:43:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.170+0000 7fbf30466640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf280aece0 0x7fbf280b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.170+0000 7fbf30466640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbf280b0b30 0x7fbf280701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.170+0000 7fbf30466640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf280c98c0 con 0x7fbf281599e0 2023-12-20T04:43:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.170+0000 7fbf30466640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf280c9ba0 con 0x7fbf280aece0 2023-12-20T04:43:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.170+0000 7fbf30466640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf280c9e80 con 0x7fbf280b0b30 2023-12-20T04:43:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.170+0000 7fbf2da01640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf280aece0 0x7fbf280b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.174+0000 7fbf2da01640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbf280aece0 0x7fbf280b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43420/0 (socket says 172.21.15.32:43420) 2023-12-20T04:43:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.174+0000 7fbf2da01640 1 -- 172.21.15.32:0/3900670839 learned_addr learned my addr 172.21.15.32:0/3900670839 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.174+0000 7fbf2ea03640 1 --2- 172.21.15.32:0/3900670839 >> v2:172.21.15.39:3300/0 conn(0x7fbf280b0b30 0x7fbf280701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.174+0000 7fbf2da01640 1 -- 172.21.15.32:0/3900670839 >> v2:172.21.15.39:3300/0 conn(0x7fbf280b0b30 msgr2=0x7fbf280701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.174+0000 7fbf2da01640 1 --2- 172.21.15.32:0/3900670839 >> v2:172.21.15.39:3300/0 conn(0x7fbf280b0b30 0x7fbf280701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.174+0000 7fbf2da01640 1 -- 172.21.15.32:0/3900670839 >> v1:172.21.15.39:6789/0 conn(0x7fbf281599e0 legacy=0x7fbf280c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.174+0000 7fbf2da01640 1 -- 172.21.15.32:0/3900670839 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf280ca160 con 0x7fbf280aece0 2023-12-20T04:43:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.174+0000 7fbf2da01640 1 --2- 172.21.15.32:0/3900670839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf280aece0 0x7fbf280b05f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbf280b0060 tx=0x7fbf1801e710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9fa27422485934 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.174+0000 7fbf2d200640 1 -- 172.21.15.32:0/3900670839 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180aa650 con 0x7fbf280aece0 2023-12-20T04:43:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.174+0000 7fbf2d200640 1 -- 172.21.15.32:0/3900670839 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf180aa7b0 con 0x7fbf280aece0 2023-12-20T04:43:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.174+0000 7fbf2d200640 1 -- 172.21.15.32:0/3900670839 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180aab40 con 0x7fbf280aece0 2023-12-20T04:43:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 -- 172.21.15.32:0/3900670839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf280aece0 msgr2=0x7fbf280b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 --2- 172.21.15.32:0/3900670839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf280aece0 0x7fbf280b05f0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7fbf280b0060 tx=0x7fbf1801e710 comp rx=0 tx=0).stop 2023-12-20T04:43:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 -- 172.21.15.32:0/3900670839 shutdown_connections 2023-12-20T04:43:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 --2- 172.21.15.32:0/3900670839 >> v2:172.21.15.39:3300/0 conn(0x7fbf280b0b30 0x7fbf280701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 --2- 172.21.15.32:0/3900670839 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf280aece0 0x7fbf280b05f0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 -- 172.21.15.32:0/3900670839 >> 172.21.15.32:0/3900670839 conn(0x7fbf2806a320 msgr2=0x7fbf280ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 -- 172.21.15.32:0/3900670839 shutdown_connections 2023-12-20T04:43:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 -- 172.21.15.32:0/3900670839 wait complete. 2023-12-20T04:43:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 Processor -- start 2023-12-20T04:43:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 -- start start 2023-12-20T04:43:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf280aece0 0x7fbf2815ac70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf280b0b30 0x7fbf281f5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf281599e0 0x7fbf28152990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf281f6740 con 0x7fbf280b0b30 2023-12-20T04:43:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf281f5e40 con 0x7fbf281599e0 2023-12-20T04:43:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf30466640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf281f6120 con 0x7fbf280aece0 2023-12-20T04:43:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf2da01640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf280b0b30 0x7fbf281f5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf2da01640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf280b0b30 0x7fbf281f5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43428/0 (socket says 172.21.15.32:43428) 2023-12-20T04:43:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf2da01640 1 -- 172.21.15.32:0/1600488969 learned_addr learned my addr 172.21.15.32:0/1600488969 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf2e202640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf280aece0 0x7fbf2815ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf2ea03640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf281599e0 0x7fbf28152990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf2da01640 1 -- 172.21.15.32:0/1600488969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf280aece0 msgr2=0x7fbf2815ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf2da01640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf280aece0 0x7fbf2815ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf2da01640 1 -- 172.21.15.32:0/1600488969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf281599e0 msgr2=0x7fbf28152990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf2da01640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf281599e0 0x7fbf28152990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.178+0000 7fbf2da01640 1 -- 172.21.15.32:0/1600488969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf180094d0 con 0x7fbf280b0b30 2023-12-20T04:43:22.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.182+0000 7fbf2da01640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf280b0b30 0x7fbf281f5580 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf1801c340 tx=0x7fbf1801b4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:22.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.182+0000 7fbf1effd640 1 -- 172.21.15.32:0/1600488969 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180bb430 con 0x7fbf280b0b30 2023-12-20T04:43:22.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.182+0000 7fbf1effd640 1 -- 172.21.15.32:0/1600488969 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf180b6070 con 0x7fbf280b0b30 2023-12-20T04:43:22.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.182+0000 7fbf30466640 1 -- 172.21.15.32:0/1600488969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf281f6400 con 0x7fbf280b0b30 2023-12-20T04:43:22.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.182+0000 7fbf1effd640 1 -- 172.21.15.32:0/1600488969 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf180bb7b0 con 0x7fbf280b0b30 2023-12-20T04:43:22.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.182+0000 7fbf30466640 1 -- 172.21.15.32:0/1600488969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf281f5ac0 con 0x7fbf280b0b30 2023-12-20T04:43:22.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.182+0000 7fbf30466640 1 -- 172.21.15.32:0/1600488969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf2804e000 con 0x7fbf280b0b30 2023-12-20T04:43:22.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.182+0000 7fbf1effd640 1 -- 172.21.15.32:0/1600488969 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbf180b8030 con 0x7fbf280b0b30 2023-12-20T04:43:22.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.182+0000 7fbf1effd640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf0c065b80 0x7fbf0c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.182+0000 7fbf1effd640 1 -- 172.21.15.32:0/1600488969 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbf1811d7c0 con 0x7fbf280b0b30 2023-12-20T04:43:22.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.182+0000 7fbf2e202640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf0c065b80 0x7fbf0c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.186+0000 7fbf2e202640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf0c065b80 0x7fbf0c068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf24000fc0 tx=0x7fbf24009040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.186+0000 7fbf1effd640 1 -- 172.21.15.32:0/1600488969 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbf180eb110 con 0x7fbf280b0b30 2023-12-20T04:43:22.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.346+0000 7fbf30466640 1 -- 172.21.15.32:0/1600488969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbf281f6bb0 con 0x7fbf280b0b30 2023-12-20T04:43:22.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf1effd640 1 -- 172.21.15.32:0/1600488969 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16062 (secure 0 0 0) 0x7fbf180ee730 con 0x7fbf280b0b30 2023-12-20T04:43:22.347 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:22.347 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:21.971365+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:19.734786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:20.967556+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 -- 172.21.15.32:0/1600488969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf0c065b80 msgr2=0x7fbf0c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf0c065b80 0x7fbf0c068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fbf24000fc0 tx=0x7fbf24009040 comp rx=0 tx=0).stop 2023-12-20T04:43:22.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 -- 172.21.15.32:0/1600488969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf280b0b30 msgr2=0x7fbf281f5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf280b0b30 0x7fbf281f5580 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7fbf1801c340 tx=0x7fbf1801b4b0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 -- 172.21.15.32:0/1600488969 shutdown_connections 2023-12-20T04:43:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbf0c065b80 0x7fbf0c068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf281599e0 0x7fbf28152990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf280b0b30 0x7fbf281f5580 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 --2- 172.21.15.32:0/1600488969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf280aece0 0x7fbf2815ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 -- 172.21.15.32:0/1600488969 >> 172.21.15.32:0/1600488969 conn(0x7fbf2806a320 msgr2=0x7fbf28153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 -- 172.21.15.32:0/1600488969 shutdown_connections 2023-12-20T04:43:22.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.350+0000 7fbf30466640 1 -- 172.21.15.32:0/1600488969 wait complete. 2023-12-20T04:43:22.369 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:22.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb5a05f0640 1 Processor -- start 2023-12-20T04:43:22.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb5a05f0640 1 -- start start 2023-12-20T04:43:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb5a05f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb59815cdb0 0x7fb5980d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb5a05f0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb59815d190 con 0x7fb5980d2740 2023-12-20T04:43:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb5a05f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5980d16a0 con 0x7fb5980cfc80 2023-12-20T04:43:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb5a05f0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5980d1980 con 0x7fb59815cdb0 2023-12-20T04:43:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb59e38c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb59815cdb0 0x7fb5980d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb59e38c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb59815cdb0 0x7fb5980d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43438/0 (socket says 172.21.15.32:43438) 2023-12-20T04:43:22.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb59e38c640 1 -- 172.21.15.32:0/2005622296 learned_addr learned my addr 172.21.15.32:0/2005622296 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb59e38c640 1 -- 172.21.15.32:0/2005622296 >> v1:172.21.15.39:6789/0 conn(0x7fb5980cfc80 legacy=0x7fb5980b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb59e38c640 1 -- 172.21.15.32:0/2005622296 >> v1:172.21.15.32:6789/0 conn(0x7fb5980d2740 legacy=0x7fb5980cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.514+0000 7fb59e38c640 1 -- 172.21.15.32:0/2005622296 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5980d1c60 con 0x7fb59815cdb0 2023-12-20T04:43:22.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.518+0000 7fb59e38c640 1 --2- 172.21.15.32:0/2005622296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb59815cdb0 0x7fb5980d0be0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb594009870 tx=0x7fb59401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6238978959e89598 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:22.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.518+0000 7fb59d38a640 1 -- 172.21.15.32:0/2005622296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5940a89f0 con 0x7fb59815cdb0 2023-12-20T04:43:22.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.518+0000 7fb59d38a640 1 -- 172.21.15.32:0/2005622296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5940a8b50 con 0x7fb59815cdb0 2023-12-20T04:43:22.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.518+0000 7fb59d38a640 1 -- 172.21.15.32:0/2005622296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5940b0440 con 0x7fb59815cdb0 2023-12-20T04:43:22.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.518+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2005622296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb59815cdb0 msgr2=0x7fb5980d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.518+0000 7fb5a05f0640 1 --2- 172.21.15.32:0/2005622296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb59815cdb0 0x7fb5980d0be0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7fb594009870 tx=0x7fb59401ecf0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.518+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2005622296 shutdown_connections 2023-12-20T04:43:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.518+0000 7fb5a05f0640 1 --2- 172.21.15.32:0/2005622296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb59815cdb0 0x7fb5980d0be0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.518+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2005622296 >> 172.21.15.32:0/2005622296 conn(0x7fb598065ee0 msgr2=0x7fb59806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.518+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2005622296 shutdown_connections 2023-12-20T04:43:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.518+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2005622296 wait complete. 2023-12-20T04:43:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb5a05f0640 1 Processor -- start 2023-12-20T04:43:22.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb5a05f0640 1 -- start start 2023-12-20T04:43:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb5a05f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980cfc80 0x7fb598150fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb5a05f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5980d2740 0x7fb59814f240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb5a05f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb59815cdb0 0x7fb5980c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb5a05f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59815fc20 con 0x7fb5980cfc80 2023-12-20T04:43:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb5a05f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb59815f640 con 0x7fb5980d2740 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb5a05f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb59814e8e0 con 0x7fb59815cdb0 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59e38c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980cfc80 0x7fb598150fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59e38c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980cfc80 0x7fb598150fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43452/0 (socket says 172.21.15.32:43452) 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59e38c640 1 -- 172.21.15.32:0/2864464488 learned_addr learned my addr 172.21.15.32:0/2864464488 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59db8b640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5980d2740 0x7fb59814f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59eb8d640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb59815cdb0 0x7fb5980c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59e38c640 1 -- 172.21.15.32:0/2864464488 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb59815cdb0 msgr2=0x7fb5980c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59e38c640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb59815cdb0 0x7fb5980c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59e38c640 1 -- 172.21.15.32:0/2864464488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5980d2740 msgr2=0x7fb59814f240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59e38c640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5980d2740 0x7fb59814f240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59e38c640 1 -- 172.21.15.32:0/2864464488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5940094d0 con 0x7fb5980cfc80 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59eb8d640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb59815cdb0 0x7fb5980c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb59e38c640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980cfc80 0x7fb598150fe0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb59401b900 tx=0x7fb59401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb58effd640 1 -- 172.21.15.32:0/2864464488 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5940ba600 con 0x7fb5980cfc80 2023-12-20T04:43:22.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb58effd640 1 -- 172.21.15.32:0/2864464488 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5940ba780 con 0x7fb5980cfc80 2023-12-20T04:43:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb58effd640 1 -- 172.21.15.32:0/2864464488 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5940bab30 con 0x7fb5980cfc80 2023-12-20T04:43:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2864464488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb59815f8b0 con 0x7fb5980cfc80 2023-12-20T04:43:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.522+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2864464488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb59814ebc0 con 0x7fb5980cfc80 2023-12-20T04:43:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.526+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2864464488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb5980b6d40 con 0x7fb5980cfc80 2023-12-20T04:43:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.526+0000 7fb58effd640 1 -- 172.21.15.32:0/2864464488 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb5940bac90 con 0x7fb5980cfc80 2023-12-20T04:43:22.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.526+0000 7fb58effd640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb57c065bb0 0x7fb57c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.526+0000 7fb58effd640 1 -- 172.21.15.32:0/2864464488 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb594121950 con 0x7fb5980cfc80 2023-12-20T04:43:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.526+0000 7fb59db8b640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb57c065bb0 0x7fb57c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.526+0000 7fb59db8b640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb57c065bb0 0x7fb57c068070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb5980cdd50 tx=0x7fb588009040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.530+0000 7fb58effd640 1 -- 172.21.15.32:0/2864464488 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb5940ef280 con 0x7fb5980cfc80 2023-12-20T04:43:22.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2864464488 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5980b6d40 con 0x7fb5980cfc80 2023-12-20T04:43:22.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb58effd640 1 -- 172.21.15.32:0/2864464488 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v78) v1 ==== 74+0+16062 (secure 0 0 0) 0x7fb5940b5020 con 0x7fb5980cfc80 2023-12-20T04:43:22.689 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:22.689 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":78,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:21.971365+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:19.734786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:20.967556+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:22.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2864464488 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb57c065bb0 msgr2=0x7fb57c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb57c065bb0 0x7fb57c068070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb5980cdd50 tx=0x7fb588009040 comp rx=0 tx=0).stop 2023-12-20T04:43:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2864464488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980cfc80 msgr2=0x7fb598150fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980cfc80 0x7fb598150fe0 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7fb59401b900 tx=0x7fb59401ba90 comp rx=0 tx=0).stop 2023-12-20T04:43:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2864464488 shutdown_connections 2023-12-20T04:43:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb57c065bb0 0x7fb57c068070 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb59815cdb0 0x7fb5980c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5980d2740 0x7fb59814f240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 --2- 172.21.15.32:0/2864464488 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5980cfc80 0x7fb598150fe0 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2864464488 >> 172.21.15.32:0/2864464488 conn(0x7fb598065ee0 msgr2=0x7fb59814df60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2864464488 shutdown_connections 2023-12-20T04:43:22.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.690+0000 7fb5a05f0640 1 -- 172.21.15.32:0/2864464488 wait complete. 2023-12-20T04:43:22.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T04:43:22.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.858+0000 7f7b0630a640 1 Processor -- start 2023-12-20T04:43:22.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.858+0000 7f7b0630a640 1 -- start start 2023-12-20T04:43:22.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.858+0000 7f7b0630a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b0015abb0 0x7f7b000caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.858+0000 7f7b0630a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7b000b0cb0 0x7f7b0015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.858+0000 7f7b0630a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0015b6a0 con 0x7f7b00070230 2023-12-20T04:43:22.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.858+0000 7f7b0630a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b0006e160 con 0x7f7b000b0cb0 2023-12-20T04:43:22.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.858+0000 7f7b0630a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b0015c070 con 0x7f7b0015abb0 2023-12-20T04:43:22.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.858+0000 7f7aff7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7b000b0cb0 0x7f7b0015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.858+0000 7f7afffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b0015abb0 0x7f7b000caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.858+0000 7f7b048a7640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7b00070230 0x7f7b0006c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:53708/0 (socket says 172.21.15.32:53708) 2023-12-20T04:43:22.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.858+0000 7f7b048a7640 1 -- 172.21.15.32:0/3613378601 learned_addr learned my addr 172.21.15.32:0/3613378601 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:22.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.862+0000 7f7aff7fe640 1 -- 172.21.15.32:0/3613378601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b0015abb0 msgr2=0x7f7b000caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:22.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.862+0000 7f7aff7fe640 1 --2- 172.21.15.32:0/3613378601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b0015abb0 0x7f7b000caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.862+0000 7f7aff7fe640 1 -- 172.21.15.32:0/3613378601 >> v1:172.21.15.39:6789/0 conn(0x7f7b00070230 legacy=0x7f7b0006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.862+0000 7f7aff7fe640 1 -- 172.21.15.32:0/3613378601 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b0006d8c0 con 0x7f7b000b0cb0 2023-12-20T04:43:22.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.862+0000 7f7afffff640 1 --2- 172.21.15.32:0/3613378601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b0015abb0 0x7f7b000caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:22.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.862+0000 7f7afeffd640 1 -- 172.21.15.32:0/3613378601 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215538268 0 0) 0x7f7b0015b6a0 con 0x7f7b00070230 2023-12-20T04:43:22.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.862+0000 7f7aff7fe640 1 --2- 172.21.15.32:0/3613378601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b000b0cb0 0x7f7b0015bb30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7af40097c0 tx=0x7f7af401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a5ed432b5ff722d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:22.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.862+0000 7f7afeffd640 1 -- 172.21.15.32:0/3613378601 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40ab680 con 0x7f7b000b0cb0 2023-12-20T04:43:22.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.862+0000 7f7afeffd640 1 -- 172.21.15.32:0/3613378601 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af40ab7e0 con 0x7f7b000b0cb0 2023-12-20T04:43:22.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.862+0000 7f7afeffd640 1 -- 172.21.15.32:0/3613378601 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af40abb70 con 0x7f7b000b0cb0 2023-12-20T04:43:22.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 -- 172.21.15.32:0/3613378601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b000b0cb0 msgr2=0x7f7b0015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 --2- 172.21.15.32:0/3613378601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b000b0cb0 0x7f7b0015bb30 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f7af40097c0 tx=0x7f7af401c480 comp rx=0 tx=0).stop 2023-12-20T04:43:22.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 -- 172.21.15.32:0/3613378601 shutdown_connections 2023-12-20T04:43:22.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 --2- 172.21.15.32:0/3613378601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b000b0cb0 0x7f7b0015bb30 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 --2- 172.21.15.32:0/3613378601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b0015abb0 0x7f7b000caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 -- 172.21.15.32:0/3613378601 >> 172.21.15.32:0/3613378601 conn(0x7f7b0006a510 msgr2=0x7f7b000ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:22.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 -- 172.21.15.32:0/3613378601 shutdown_connections 2023-12-20T04:43:22.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 -- 172.21.15.32:0/3613378601 wait complete. 2023-12-20T04:43:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 Processor -- start 2023-12-20T04:43:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 -- start start 2023-12-20T04:43:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b00070230 0x7f7b0014fec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b000b0cb0 0x7f7b00150b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0015abb0 0x7f7b000c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b00150400 con 0x7f7b00070230 2023-12-20T04:43:22.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b00151490 con 0x7f7b0015abb0 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7b0630a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b00151770 con 0x7f7b000b0cb0 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7afffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b00070230 0x7f7b0014fec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7afffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b00070230 0x7f7b0014fec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43462/0 (socket says 172.21.15.32:43462) 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7afffff640 1 -- 172.21.15.32:0/3968442264 learned_addr learned my addr 172.21.15.32:0/3968442264 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7afffff640 1 -- 172.21.15.32:0/3968442264 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b000b0cb0 msgr2=0x7f7b00150b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7afffff640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b000b0cb0 0x7f7b00150b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7afffff640 1 -- 172.21.15.32:0/3968442264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0015abb0 msgr2=0x7f7b000c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7afffff640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0015abb0 0x7f7b000c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7afffff640 1 -- 172.21.15.32:0/3968442264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7af0009420 con 0x7f7b00070230 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.866+0000 7f7afffff640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b00070230 0x7f7b0014fec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7af000e980 tx=0x7f7af000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.870+0000 7f7afcff9640 1 -- 172.21.15.32:0/3968442264 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af000ca00 con 0x7f7b00070230 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.870+0000 7f7afcff9640 1 -- 172.21.15.32:0/3968442264 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af000cb60 con 0x7f7b00070230 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.870+0000 7f7afcff9640 1 -- 172.21.15.32:0/3968442264 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af0013470 con 0x7f7b00070230 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.870+0000 7f7b0630a640 1 -- 172.21.15.32:0/3968442264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af4009420 con 0x7f7b00070230 2023-12-20T04:43:22.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.870+0000 7f7b0630a640 1 -- 172.21.15.32:0/3968442264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b00151d30 con 0x7f7b00070230 2023-12-20T04:43:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.870+0000 7f7b0630a640 1 -- 172.21.15.32:0/3968442264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b0004e000 con 0x7f7b00070230 2023-12-20T04:43:22.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.870+0000 7f7afcff9640 1 -- 172.21.15.32:0/3968442264 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7af00135d0 con 0x7f7b00070230 2023-12-20T04:43:22.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.870+0000 7f7afcff9640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7aec065b80 0x7f7aec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.870+0000 7f7afcff9640 1 -- 172.21.15.32:0/3968442264 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(78..78 src has 1..78) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7af0083cd0 con 0x7f7b00070230 2023-12-20T04:43:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.874+0000 7f7aff7fe640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7aec065b80 0x7f7aec068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:22.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.874+0000 7f7aff7fe640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7aec065b80 0x7f7aec068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7af40af000 tx=0x7f7af4008040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:22.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:22.874+0000 7f7afcff9640 1 -- 172.21.15.32:0/3968442264 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7af0051660 con 0x7f7b00070230 2023-12-20T04:43:23.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.070+0000 7f7b0630a640 1 -- 172.21.15.32:0/3968442264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7b00071150 con 0x7f7b00070230 2023-12-20T04:43:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.982+0000 7f7afcff9640 1 -- 172.21.15.32:0/3968442264 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v30) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7af0054c80 con 0x7f7b00070230 2023-12-20T04:43:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.982+0000 7f7b0630a640 1 -- 172.21.15.32:0/3968442264 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7aec065b80 msgr2=0x7f7aec068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.982+0000 7f7b0630a640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7aec065b80 0x7f7aec068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7af40af000 tx=0x7f7af4008040 comp rx=0 tx=0).stop 2023-12-20T04:43:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.982+0000 7f7b0630a640 1 -- 172.21.15.32:0/3968442264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b00070230 msgr2=0x7f7b0014fec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.982+0000 7f7b0630a640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b00070230 0x7f7b0014fec0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f7af000e980 tx=0x7f7af000ee40 comp rx=0 tx=0).stop 2023-12-20T04:43:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.982+0000 7f7b0630a640 1 -- 172.21.15.32:0/3968442264 shutdown_connections 2023-12-20T04:43:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.982+0000 7f7b0630a640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7aec065b80 0x7f7aec068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.982+0000 7f7b0630a640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0015abb0 0x7f7b000c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.982+0000 7f7b0630a640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b000b0cb0 0x7f7b00150b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.982+0000 7f7b0630a640 1 --2- 172.21.15.32:0/3968442264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b00070230 0x7f7b0014fec0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.982+0000 7f7b0630a640 1 -- 172.21.15.32:0/3968442264 >> 172.21.15.32:0/3968442264 conn(0x7f7b0006a510 msgr2=0x7f7b00152fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.986+0000 7f7b0630a640 1 -- 172.21.15.32:0/3968442264 shutdown_connections 2023-12-20T04:43:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:23.986+0000 7f7b0630a640 1 -- 172.21.15.32:0/3968442264 wait complete. 2023-12-20T04:43:24.007 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T04:43:24.007 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T04:43:24.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.158+0000 7f6a5d6f2640 1 Processor -- start 2023-12-20T04:43:24.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.158+0000 7f6a5d6f2640 1 -- start start 2023-12-20T04:43:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a5d6f2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6a580b0cb0 0x7f6a580caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a5d6f2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6a58070230 0x7f6a58070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a5d6f2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a58070bc0 con 0x7f6a5815abb0 2023-12-20T04:43:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a5d6f2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5806e160 con 0x7f6a580b0cb0 2023-12-20T04:43:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a5d6f2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a5815b6a0 con 0x7f6a58070230 2023-12-20T04:43:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a4ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6a580b0cb0 0x7f6a580caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a4ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6a580b0cb0 0x7f6a580caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43474/0 (socket says 172.21.15.32:43474) 2023-12-20T04:43:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a4ffff640 1 -- 172.21.15.32:0/107517539 learned_addr learned my addr 172.21.15.32:0/107517539 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a577fe640 1 --2- 172.21.15.32:0/107517539 >> v2:172.21.15.39:3300/0 conn(0x7f6a58070230 0x7f6a58070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a4ffff640 1 -- 172.21.15.32:0/107517539 >> v2:172.21.15.39:3300/0 conn(0x7f6a58070230 msgr2=0x7f6a58070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a4ffff640 1 --2- 172.21.15.32:0/107517539 >> v2:172.21.15.39:3300/0 conn(0x7f6a58070230 0x7f6a58070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a4ffff640 1 -- 172.21.15.32:0/107517539 >> v1:172.21.15.39:6789/0 conn(0x7f6a5815abb0 legacy=0x7f6a5815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a4ffff640 1 -- 172.21.15.32:0/107517539 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a5806d8c0 con 0x7f6a580b0cb0 2023-12-20T04:43:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.162+0000 7f6a4ffff640 1 --2- 172.21.15.32:0/107517539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a580b0cb0 0x7f6a580caaf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6a580b0c40 tx=0x7f6a4001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d6f84b8f33b9a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a567fc640 1 -- 172.21.15.32:0/107517539 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a400aa620 con 0x7f6a580b0cb0 2023-12-20T04:43:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a567fc640 1 -- 172.21.15.32:0/107517539 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a400aa780 con 0x7f6a580b0cb0 2023-12-20T04:43:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a567fc640 1 -- 172.21.15.32:0/107517539 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a400aab10 con 0x7f6a580b0cb0 2023-12-20T04:43:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/107517539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a580b0cb0 msgr2=0x7f6a580caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 --2- 172.21.15.32:0/107517539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a580b0cb0 0x7f6a580caaf0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f6a580b0c40 tx=0x7f6a4001c4c0 comp rx=0 tx=0).stop 2023-12-20T04:43:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/107517539 shutdown_connections 2023-12-20T04:43:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 --2- 172.21.15.32:0/107517539 >> v2:172.21.15.39:3300/0 conn(0x7f6a58070230 0x7f6a58070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 --2- 172.21.15.32:0/107517539 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a580b0cb0 0x7f6a580caaf0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/107517539 >> 172.21.15.32:0/107517539 conn(0x7f6a5806a510 msgr2=0x7f6a580ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/107517539 shutdown_connections 2023-12-20T04:43:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/107517539 wait complete. 2023-12-20T04:43:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 Processor -- start 2023-12-20T04:43:24.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 -- start start 2023-12-20T04:43:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a58070230 0x7f6a5814ed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a580b0cb0 0x7f6a5814b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6a5815abb0 0x7f6a5814bb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5814e860 con 0x7f6a58070230 2023-12-20T04:43:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a58148a10 con 0x7f6a580b0cb0 2023-12-20T04:43:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a5d6f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a5814aa00 con 0x7f6a5815abb0 2023-12-20T04:43:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a56ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a58070230 0x7f6a5814ed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.166+0000 7f6a56ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a58070230 0x7f6a5814ed60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43482/0 (socket says 172.21.15.32:43482) 2023-12-20T04:43:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a56ffd640 1 -- 172.21.15.32:0/2403544329 learned_addr learned my addr 172.21.15.32:0/2403544329 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a577fe640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6a5815abb0 0x7f6a5814bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a4ffff640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a580b0cb0 0x7f6a5814b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a56ffd640 1 -- 172.21.15.32:0/2403544329 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6a5815abb0 msgr2=0x7f6a5814bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a56ffd640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6a5815abb0 0x7f6a5814bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a56ffd640 1 -- 172.21.15.32:0/2403544329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a580b0cb0 msgr2=0x7f6a5814b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a56ffd640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a580b0cb0 0x7f6a5814b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a56ffd640 1 -- 172.21.15.32:0/2403544329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a400094d0 con 0x7f6a58070230 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a56ffd640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a58070230 0x7f6a5814ed60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b410 tx=0x7f6a4800b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a4f7fe640 1 -- 172.21.15.32:0/2403544329 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48004020 con 0x7f6a58070230 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a4f7fe640 1 -- 172.21.15.32:0/2403544329 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a48004180 con 0x7f6a58070230 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a4f7fe640 1 -- 172.21.15.32:0/2403544329 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48011630 con 0x7f6a58070230 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/2403544329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a5814c170 con 0x7f6a58070230 2023-12-20T04:43:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/2403544329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a5814a720 con 0x7f6a58070230 2023-12-20T04:43:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/2403544329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a580cce50 con 0x7f6a58070230 2023-12-20T04:43:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a4f7fe640 1 -- 172.21.15.32:0/2403544329 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6a48011790 con 0x7f6a58070230 2023-12-20T04:43:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a4f7fe640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a38065bb0 0x7f6a38068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.170+0000 7f6a4f7fe640 1 -- 172.21.15.32:0/2403544329 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6a48082ff0 con 0x7f6a58070230 2023-12-20T04:43:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.174+0000 7f6a4ffff640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a38065bb0 0x7f6a38068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:24.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.174+0000 7f6a4ffff640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a38065bb0 0x7f6a38068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6a400ae000 tx=0x7f6a400a7000 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:24.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.178+0000 7f6a4f7fe640 1 -- 172.21.15.32:0/2403544329 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6a580cce50 con 0x7f6a58070230 2023-12-20T04:43:24.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.366+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/2403544329 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f6a580cce50 con 0x7f6a58070230 2023-12-20T04:43:24.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.982+0000 7f6a4f7fe640 1 -- 172.21.15.32:0/2403544329 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v31) v1 ==== 122+0+0 (secure 0 0 0) 0x7f6a580cce50 con 0x7f6a58070230 2023-12-20T04:43:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/2403544329 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a38065bb0 msgr2=0x7f6a38068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a38065bb0 0x7f6a38068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6a400ae000 tx=0x7f6a400a7000 comp rx=0 tx=0).stop 2023-12-20T04:43:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/2403544329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a58070230 msgr2=0x7f6a5814ed60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a58070230 0x7f6a5814ed60 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800b410 tx=0x7f6a4800b8d0 comp rx=0 tx=0).stop 2023-12-20T04:43:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/2403544329 shutdown_connections 2023-12-20T04:43:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a38065bb0 0x7f6a38068070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:24.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6a5815abb0 0x7f6a5814bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:24.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a580b0cb0 0x7f6a5814b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:24.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 --2- 172.21.15.32:0/2403544329 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a58070230 0x7f6a5814ed60 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:24.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/2403544329 >> 172.21.15.32:0/2403544329 conn(0x7f6a5806a510 msgr2=0x7f6a581543f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:24.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/2403544329 shutdown_connections 2023-12-20T04:43:24.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:24.986+0000 7f6a5d6f2640 1 -- 172.21.15.32:0/2403544329 wait complete. 2023-12-20T04:43:25.008 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T04:43:25.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f1139604640 1 Processor -- start 2023-12-20T04:43:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f1139604640 1 -- start start 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f1139604640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f113415cea0 0x7f11340cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f1139604640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f11340b84d0 0x7f11340cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f1139604640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f11340d0c90 con 0x7f113415cac0 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f1139604640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1134160f10 con 0x7f11340b84d0 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f1139604640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1134161110 con 0x7f113415cea0 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f1132ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f113415cac0 0x7f1134160800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54970/0 (socket says 172.21.15.32:54970) 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f1132ffd640 1 -- 172.21.15.32:0/1496860899 learned_addr learned my addr 172.21.15.32:0/1496860899 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965883309 0 0) 0x7f11340d0c90 con 0x7f113415cac0 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1120003530 con 0x7f113415cac0 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f112bfff640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f113415cea0 0x7f11340cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11337fe640 1 --2- 172.21.15.32:0/1496860899 >> v2:172.21.15.39:3300/0 conn(0x7f11340b84d0 0x7f11340cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1618941381 0 0) 0x7f1120003530 con 0x7f113415cac0 2023-12-20T04:43:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f113415cea0 msgr2=0x7f11340cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11327fc640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f113415cea0 0x7f11340cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 >> v2:172.21.15.39:3300/0 conn(0x7f11340b84d0 msgr2=0x7f11340cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11327fc640 1 --2- 172.21.15.32:0/1496860899 >> v2:172.21.15.39:3300/0 conn(0x7f11340b84d0 0x7f11340cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11341613f0 con 0x7f113415cac0 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f1124001680 con 0x7f113415cac0 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11337fe640 1 --2- 172.21.15.32:0/1496860899 >> v2:172.21.15.39:3300/0 conn(0x7f11340b84d0 0x7f11340cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 >> v1:172.21.15.32:6789/0 conn(0x7f113415cac0 legacy=0x7f1134160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.158+0000 7f11327fc640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1120003980 0x7f1120003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11327fc640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f113415cea0 0x7f11200047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11327fc640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 0x7f1120005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1134160f10 con 0x7f1120004d10 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1134161110 con 0x7f1120003980 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f112000b9c0 con 0x7f113415cea0 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11337fe640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 0x7f1120005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f1132ffd640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1120003980 0x7f1120003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f112bfff640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f113415cea0 0x7f11200047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11337fe640 1 -- 172.21.15.32:0/1496860899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f113415cea0 msgr2=0x7f11200047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11337fe640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f113415cea0 0x7f11200047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11337fe640 1 -- 172.21.15.32:0/1496860899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1120003980 msgr2=0x7f1120003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11337fe640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1120003980 0x7f1120003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11337fe640 1 -- 172.21.15.32:0/1496860899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1120003530 con 0x7f1120004d10 2023-12-20T04:43:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11337fe640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 0x7f1120005190 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f112c0073e0 tx=0x7f112c01cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=66d0fafea8da7ac6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c0a5e20 con 0x7f1120004d10 2023-12-20T04:43:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112c0ad440 con 0x7f1120004d10 2023-12-20T04:43:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.162+0000 7f11327fc640 1 -- 172.21.15.32:0/1496860899 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112c0ad7b0 con 0x7f1120004d10 2023-12-20T04:43:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 -- 172.21.15.32:0/1496860899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 msgr2=0x7f1120005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 --2- 172.21.15.32:0/1496860899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 0x7f1120005190 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f112c0073e0 tx=0x7f112c01cf20 comp rx=0 tx=0).stop 2023-12-20T04:43:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f11337fe640 1 -- 172.21.15.32:0/1496860899 reap_dead start 2023-12-20T04:43:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 -- 172.21.15.32:0/1496860899 shutdown_connections 2023-12-20T04:43:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 -- 172.21.15.32:0/1496860899 >> 172.21.15.32:0/1496860899 conn(0x7f113406afd0 msgr2=0x7f113406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 -- 172.21.15.32:0/1496860899 shutdown_connections 2023-12-20T04:43:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 -- 172.21.15.32:0/1496860899 wait complete. 2023-12-20T04:43:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 Processor -- start 2023-12-20T04:43:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 -- start start 2023-12-20T04:43:25.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 0x7f11340c9b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11340b5f60 0x7f11340d1a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f11340ca5c0 0x7f11340caa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11340b7030 con 0x7f1120004d10 2023-12-20T04:43:25.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11340cafb0 con 0x7f11340b5f60 2023-12-20T04:43:25.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1139604640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11340ca0a0 con 0x7f11340ca5c0 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1132ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 0x7f11340c9b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1132ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 0x7f11340c9b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43496/0 (socket says 172.21.15.32:43496) 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1132ffd640 1 -- 172.21.15.32:0/245751093 learned_addr learned my addr 172.21.15.32:0/245751093 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f112bfff640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11340b5f60 0x7f11340d1a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f11337fe640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f11340ca5c0 0x7f11340caa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1132ffd640 1 -- 172.21.15.32:0/245751093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f11340ca5c0 msgr2=0x7f11340caa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1132ffd640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f11340ca5c0 0x7f11340caa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1132ffd640 1 -- 172.21.15.32:0/245751093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11340b5f60 msgr2=0x7f11340d1a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1132ffd640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11340b5f60 0x7f11340d1a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.166+0000 7f1132ffd640 1 -- 172.21.15.32:0/245751093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1134146750 con 0x7f1120004d10 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.170+0000 7f1132ffd640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 0x7f11340c9b60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1124007370 tx=0x7f1124007770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.170+0000 7f112b7fe640 1 -- 172.21.15.32:0/245751093 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400f960 con 0x7f1120004d10 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.170+0000 7f112b7fe640 1 -- 172.21.15.32:0/245751093 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f112400fac0 con 0x7f1120004d10 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.170+0000 7f112b7fe640 1 -- 172.21.15.32:0/245751093 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f112400fe50 con 0x7f1120004d10 2023-12-20T04:43:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.170+0000 7f1139604640 1 -- 172.21.15.32:0/245751093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11340b6420 con 0x7f1120004d10 2023-12-20T04:43:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.170+0000 7f1139604640 1 -- 172.21.15.32:0/245751093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11340b69e0 con 0x7f1120004d10 2023-12-20T04:43:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.170+0000 7f1139604640 1 -- 172.21.15.32:0/245751093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1134074100 con 0x7f1120004d10 2023-12-20T04:43:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.170+0000 7f112b7fe640 1 -- 172.21.15.32:0/245751093 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f112401d460 con 0x7f1120004d10 2023-12-20T04:43:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.170+0000 7f112b7fe640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1114065b80 0x7f1114068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.170+0000 7f112b7fe640 1 -- 172.21.15.32:0/245751093 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f11240833f0 con 0x7f1120004d10 2023-12-20T04:43:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.174+0000 7f112bfff640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1114065b80 0x7f1114068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.174+0000 7f112bfff640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1114065b80 0x7f1114068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f113415af90 tx=0x7f111c009190 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:25.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.178+0000 7f112b7fe640 1 -- 172.21.15.32:0/245751093 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1124050c90 con 0x7f1120004d10 2023-12-20T04:43:25.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.366+0000 7f1139604640 1 -- 172.21.15.32:0/245751093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f11340d07a0 con 0x7f1120004d10 2023-12-20T04:43:25.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.366+0000 7f112b7fe640 1 -- 172.21.15.32:0/245751093 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4706 (secure 0 0 0) 0x7f11240542b0 con 0x7f1120004d10 2023-12-20T04:43:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:25.366 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5780,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4279891602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4279891602},{"type":"v1","addr":"172.21.15.39:6834","nonce":4279891602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6292,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3937321478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3937321478},{"type":"v1","addr":"172.21.15.39:6835","nonce":3937321478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/404004710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":404004710},{"type":"v1","addr":"172.21.15.32:6835","nonce":404004710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:21.971704+0000","modified":"2023-12-20T04:43:24.982163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3343723618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3343723618},{"type":"v1","addr":"172.21.15.32:6837","nonce":3343723618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-20T04:43:25.367 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2023-12-20T04:43:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 -- 172.21.15.32:0/245751093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1114065b80 msgr2=0x7f1114068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1114065b80 0x7f1114068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f113415af90 tx=0x7f111c009190 comp rx=0 tx=0).stop 2023-12-20T04:43:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 -- 172.21.15.32:0/245751093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 msgr2=0x7f11340c9b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 0x7f11340c9b60 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f1124007370 tx=0x7f1124007770 comp rx=0 tx=0).stop 2023-12-20T04:43:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 -- 172.21.15.32:0/245751093 shutdown_connections 2023-12-20T04:43:25.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1114065b80 0x7f1114068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f11340ca5c0 0x7f11340caa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f11340b5f60 0x7f11340d1a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 --2- 172.21.15.32:0/245751093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1120004d10 0x7f11340c9b60 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 -- 172.21.15.32:0/245751093 >> 172.21.15.32:0/245751093 conn(0x7f113406afd0 msgr2=0x7f1134154210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:25.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 -- 172.21.15.32:0/245751093 shutdown_connections 2023-12-20T04:43:25.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.370+0000 7f1139604640 1 -- 172.21.15.32:0/245751093 wait complete. 2023-12-20T04:43:25.388 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f15220b6640 1 Processor -- start 2023-12-20T04:43:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f15220b6640 1 -- start start 2023-12-20T04:43:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f15220b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f151c15cdb0 0x7f151c0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f15220b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f151c0cfc80 0x7f151c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f15220b6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f151c15d190 con 0x7f151c0d2740 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f15220b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f151c0d16a0 con 0x7f151c0cfc80 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f15220b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c0d1980 con 0x7f151c15cdb0 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f151affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f151c0d2740 0x7f151c0cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54976/0 (socket says 172.21.15.32:54976) 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f151affd640 1 -- 172.21.15.32:0/3303204112 learned_addr learned my addr 172.21.15.32:0/3303204112 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f151bfff640 1 --2- 172.21.15.32:0/3303204112 >> v2:172.21.15.32:3300/0 conn(0x7f151c0cfc80 0x7f151c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f151b7fe640 1 --2- 172.21.15.32:0/3303204112 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f151c15cdb0 0x7f151c0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f151a7fc640 1 -- 172.21.15.32:0/3303204112 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1197707688 0 0) 0x7f151c15d190 con 0x7f151c0d2740 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f151a7fc640 1 -- 172.21.15.32:0/3303204112 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1504003530 con 0x7f151c0d2740 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f151bfff640 1 -- 172.21.15.32:0/3303204112 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f151c15cdb0 msgr2=0x7f151c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f151bfff640 1 --2- 172.21.15.32:0/3303204112 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f151c15cdb0 0x7f151c0d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f151bfff640 1 -- 172.21.15.32:0/3303204112 >> v1:172.21.15.32:6789/0 conn(0x7f151c0d2740 legacy=0x7f151c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.534+0000 7f151bfff640 1 -- 172.21.15.32:0/3303204112 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f151c0d1c60 con 0x7f151c0cfc80 2023-12-20T04:43:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f151bfff640 1 --2- 172.21.15.32:0/3303204112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f151c0cfc80 0x7f151c0b5c90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1514009d90 tx=0x7f151401f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5285f33f66332a1a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f151a7fc640 1 -- 172.21.15.32:0/3303204112 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140af600 con 0x7f151c0cfc80 2023-12-20T04:43:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f151a7fc640 1 -- 172.21.15.32:0/3303204112 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15140ab050 con 0x7f151c0cfc80 2023-12-20T04:43:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f151a7fc640 1 -- 172.21.15.32:0/3303204112 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15140af960 con 0x7f151c0cfc80 2023-12-20T04:43:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f15220b6640 1 -- 172.21.15.32:0/3303204112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f151c0cfc80 msgr2=0x7f151c0b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f15220b6640 1 --2- 172.21.15.32:0/3303204112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f151c0cfc80 0x7f151c0b5c90 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f1514009d90 tx=0x7f151401f130 comp rx=0 tx=0).stop 2023-12-20T04:43:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f15220b6640 1 -- 172.21.15.32:0/3303204112 shutdown_connections 2023-12-20T04:43:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f15220b6640 1 --2- 172.21.15.32:0/3303204112 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f151c0cfc80 0x7f151c0b5c90 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f15220b6640 1 --2- 172.21.15.32:0/3303204112 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f151c15cdb0 0x7f151c0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f15220b6640 1 -- 172.21.15.32:0/3303204112 >> 172.21.15.32:0/3303204112 conn(0x7f151c065ee0 msgr2=0x7f151c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f15220b6640 1 -- 172.21.15.32:0/3303204112 shutdown_connections 2023-12-20T04:43:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f15220b6640 1 -- 172.21.15.32:0/3303204112 wait complete. 2023-12-20T04:43:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.538+0000 7f15220b6640 1 Processor -- start 2023-12-20T04:43:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f15220b6640 1 -- start start 2023-12-20T04:43:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f15220b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f151c0cfc80 0x7f151c14e160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f15220b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f151c0d2740 0x7f151c0c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f15220b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c15cdb0 0x7f151c155a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f15220b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c155590 con 0x7f151c0d2740 2023-12-20T04:43:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f15220b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f151c1600a0 con 0x7f151c15cdb0 2023-12-20T04:43:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f15220b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f151c154c60 con 0x7f151c0cfc80 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f151affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f151c0d2740 0x7f151c0c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f151affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f151c0d2740 0x7f151c0c8540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43510/0 (socket says 172.21.15.32:43510) 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f151affd640 1 -- 172.21.15.32:0/2221531832 learned_addr learned my addr 172.21.15.32:0/2221531832 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f151bfff640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c15cdb0 0x7f151c155a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f151b7fe640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f151c0cfc80 0x7f151c14e160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f151affd640 1 -- 172.21.15.32:0/2221531832 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f151c0cfc80 msgr2=0x7f151c14e160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f151affd640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f151c0cfc80 0x7f151c14e160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f151affd640 1 -- 172.21.15.32:0/2221531832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c15cdb0 msgr2=0x7f151c155a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f151affd640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c15cdb0 0x7f151c155a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f151affd640 1 -- 172.21.15.32:0/2221531832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1514009a70 con 0x7f151c0d2740 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f151affd640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f151c0d2740 0x7f151c0c8540 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f151000c450 tx=0x7f151000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f1503fff640 1 -- 172.21.15.32:0/2221531832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510005cf0 con 0x7f151c0d2740 2023-12-20T04:43:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f1503fff640 1 -- 172.21.15.32:0/2221531832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1510005e50 con 0x7f151c0d2740 2023-12-20T04:43:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f1503fff640 1 -- 172.21.15.32:0/2221531832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1510007290 con 0x7f151c0d2740 2023-12-20T04:43:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f15220b6640 1 -- 172.21.15.32:0/2221531832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f151c154f40 con 0x7f151c0d2740 2023-12-20T04:43:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f15220b6640 1 -- 172.21.15.32:0/2221531832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f151c14e6a0 con 0x7f151c0d2740 2023-12-20T04:43:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.542+0000 7f15220b6640 1 -- 172.21.15.32:0/2221531832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f151c04e000 con 0x7f151c0d2740 2023-12-20T04:43:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.546+0000 7f1503fff640 1 -- 172.21.15.32:0/2221531832 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f15100073f0 con 0x7f151c0d2740 2023-12-20T04:43:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.546+0000 7f1503fff640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1508065bb0 0x7f1508068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.546+0000 7f1503fff640 1 -- 172.21.15.32:0/2221531832 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f15100828f0 con 0x7f151c0d2740 2023-12-20T04:43:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.546+0000 7f151b7fe640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1508065bb0 0x7f1508068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.546+0000 7f151b7fe640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1508065bb0 0x7f1508068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f150c009ac0 tx=0x7f150c009190 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.550+0000 7f1503fff640 1 -- 172.21.15.32:0/2221531832 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1510050120 con 0x7f151c0d2740 2023-12-20T04:43:25.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.710+0000 7f15220b6640 1 -- 172.21.15.32:0/2221531832 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f151c06b8d0 con 0x7f151c0d2740 2023-12-20T04:43:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f1503fff640 1 -- 172.21.15.32:0/2221531832 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v79) v1 ==== 74+0+16049 (secure 0 0 0) 0x7f1510017030 con 0x7f151c0d2740 2023-12-20T04:43:25.711 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:25.711 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:22.974803+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:19.734786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:20.967556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 -- 172.21.15.32:0/2221531832 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1508065bb0 msgr2=0x7f1508068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1508065bb0 0x7f1508068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f150c009ac0 tx=0x7f150c009190 comp rx=0 tx=0).stop 2023-12-20T04:43:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 -- 172.21.15.32:0/2221531832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f151c0d2740 msgr2=0x7f151c0c8540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f151c0d2740 0x7f151c0c8540 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f151000c450 tx=0x7f151000c910 comp rx=0 tx=0).stop 2023-12-20T04:43:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 -- 172.21.15.32:0/2221531832 shutdown_connections 2023-12-20T04:43:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1508065bb0 0x7f1508068070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f151c15cdb0 0x7f151c155a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f151c0d2740 0x7f151c0c8540 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 --2- 172.21.15.32:0/2221531832 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f151c0cfc80 0x7f151c14e160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 -- 172.21.15.32:0/2221531832 >> 172.21.15.32:0/2221531832 conn(0x7f151c065ee0 msgr2=0x7f151c0d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 -- 172.21.15.32:0/2221531832 shutdown_connections 2023-12-20T04:43:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.714+0000 7f15220b6640 1 -- 172.21.15.32:0/2221531832 wait complete. 2023-12-20T04:43:25.733 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T04:43:25.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:43:25.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.878+0000 7efeff339640 1 Processor -- start 2023-12-20T04:43:25.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.878+0000 7efeff339640 1 -- start start 2023-12-20T04:43:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.878+0000 7efeff339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef815d080 0x7efef80cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.878+0000 7efeff339640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efef815de00 0x7efef806a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.878+0000 7efeff339640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efef815d970 con 0x7efef8069d20 2023-12-20T04:43:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.878+0000 7efeff339640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efef80ce740 con 0x7efef815de00 2023-12-20T04:43:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.878+0000 7efeff339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef806a640 con 0x7efef815d080 2023-12-20T04:43:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.878+0000 7efefd0d5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efef8069d20 0x7efef80cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54984/0 (socket says 172.21.15.32:54984) 2023-12-20T04:43:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.878+0000 7efefd0d5640 1 -- 172.21.15.32:0/2346387256 learned_addr learned my addr 172.21.15.32:0/2346387256 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efefd8d6640 1 --2- 172.21.15.32:0/2346387256 >> v2:172.21.15.39:3300/0 conn(0x7efef815de00 0x7efef806a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efefc8d4640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef815d080 0x7efef80cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2154427119 0 0) 0x7efef815d970 con 0x7efef8069d20 2023-12-20T04:43:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efee4003530 con 0x7efef8069d20 2023-12-20T04:43:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3596951767 0 0) 0x7efee4003530 con 0x7efef8069d20 2023-12-20T04:43:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef815d080 msgr2=0x7efef80cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:25.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef815d080 0x7efef80cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 >> v2:172.21.15.39:3300/0 conn(0x7efef815de00 msgr2=0x7efef806a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:25.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 --2- 172.21.15.32:0/2346387256 >> v2:172.21.15.39:3300/0 conn(0x7efef815de00 0x7efef806a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80ce180 con 0x7efef8069d20 2023-12-20T04:43:25.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7efef4001680 con 0x7efef8069d20 2023-12-20T04:43:25.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 >> v1:172.21.15.32:6789/0 conn(0x7efef8069d20 legacy=0x7efef80cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee4003980 0x7efee4003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee40047d0 0x7efee4004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef815d080 0x7efee4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80ce740 con 0x7efee40047d0 2023-12-20T04:43:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef806a640 con 0x7efee4003980 2023-12-20T04:43:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efee400b990 con 0x7efef815d080 2023-12-20T04:43:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efefc8d4640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee40047d0 0x7efee4004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efefd0d5640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee4003980 0x7efee4003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efefd8d6640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef815d080 0x7efee4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efefc8d4640 1 -- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef815d080 msgr2=0x7efee4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efefc8d4640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef815d080 0x7efee4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efefc8d4640 1 -- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee4003980 msgr2=0x7efee4003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efefc8d4640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee4003980 0x7efee4003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efefc8d4640 1 -- 172.21.15.32:0/2346387256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efee4003530 con 0x7efee40047d0 2023-12-20T04:43:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.882+0000 7efefd8d6640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef815d080 0x7efee4005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efefc8d4640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee40047d0 0x7efee4004c50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efee8009ae0 tx=0x7efee801ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce3ca6dd26eeb6eb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee80a5560 con 0x7efee40047d0 2023-12-20T04:43:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee80a56c0 con 0x7efee40047d0 2023-12-20T04:43:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efeeffff640 1 -- 172.21.15.32:0/2346387256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee80a5a70 con 0x7efee40047d0 2023-12-20T04:43:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efeff339640 1 -- 172.21.15.32:0/2346387256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee40047d0 msgr2=0x7efee4004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efeff339640 1 --2- 172.21.15.32:0/2346387256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efee40047d0 0x7efee4004c50 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7efee8009ae0 tx=0x7efee801ca50 comp rx=0 tx=0).stop 2023-12-20T04:43:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efefd8d6640 1 -- 172.21.15.32:0/2346387256 reap_dead start 2023-12-20T04:43:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efeff339640 1 -- 172.21.15.32:0/2346387256 shutdown_connections 2023-12-20T04:43:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efeff339640 1 -- 172.21.15.32:0/2346387256 >> 172.21.15.32:0/2346387256 conn(0x7efef80cc310 msgr2=0x7efef80cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:25.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efeff339640 1 -- 172.21.15.32:0/2346387256 shutdown_connections 2023-12-20T04:43:25.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efeff339640 1 -- 172.21.15.32:0/2346387256 wait complete. 2023-12-20T04:43:25.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efeff339640 1 Processor -- start 2023-12-20T04:43:25.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.886+0000 7efeff339640 1 -- start start 2023-12-20T04:43:25.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeff339640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee40047d0 0x7efef80b1610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeff339640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef80c4b20 0x7efef80b2e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeff339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef81f6bd0 0x7efef81f5f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeff339640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80b1b50 con 0x7efef80c4b20 2023-12-20T04:43:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeff339640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efef80b0640 con 0x7efee40047d0 2023-12-20T04:43:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeff339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efef80b3780 con 0x7efef81f6bd0 2023-12-20T04:43:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefc8d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef80c4b20 0x7efef80b2e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefc8d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef80c4b20 0x7efef80b2e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43532/0 (socket says 172.21.15.32:43532) 2023-12-20T04:43:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefc8d4640 1 -- 172.21.15.32:0/2641662765 learned_addr learned my addr 172.21.15.32:0/2641662765 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefd0d5640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee40047d0 0x7efef80b1610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefd8d6640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef81f6bd0 0x7efef81f5f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefc8d4640 1 -- 172.21.15.32:0/2641662765 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef81f6bd0 msgr2=0x7efef81f5f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefc8d4640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef81f6bd0 0x7efef81f5f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefc8d4640 1 -- 172.21.15.32:0/2641662765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee40047d0 msgr2=0x7efef80b1610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefc8d4640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee40047d0 0x7efef80b1610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefc8d4640 1 -- 172.21.15.32:0/2641662765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efef80b0850 con 0x7efef80c4b20 2023-12-20T04:43:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefd0d5640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee40047d0 0x7efef80b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efefc8d4640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef80c4b20 0x7efef80b2e30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efee801c880 tx=0x7efee4005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeedffb640 1 -- 172.21.15.32:0/2641662765 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8004220 con 0x7efef80c4b20 2023-12-20T04:43:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeedffb640 1 -- 172.21.15.32:0/2641662765 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efee80043c0 con 0x7efef80c4b20 2023-12-20T04:43:25.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeff339640 1 -- 172.21.15.32:0/2641662765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efef80b0b30 con 0x7efef80c4b20 2023-12-20T04:43:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeedffb640 1 -- 172.21.15.32:0/2641662765 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efee8003860 con 0x7efef80c4b20 2023-12-20T04:43:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeff339640 1 -- 172.21.15.32:0/2641662765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efef80b10f0 con 0x7efef80c4b20 2023-12-20T04:43:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeff339640 1 -- 172.21.15.32:0/2641662765 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efef804e000 con 0x7efef80c4b20 2023-12-20T04:43:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.890+0000 7efeedffb640 1 -- 172.21.15.32:0/2641662765 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7efee80b4040 con 0x7efef80c4b20 2023-12-20T04:43:25.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.894+0000 7efeedffb640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efedc065b80 0x7efedc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:25.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.894+0000 7efeedffb640 1 -- 172.21.15.32:0/2641662765 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7efee811d860 con 0x7efef80c4b20 2023-12-20T04:43:25.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.894+0000 7efefd0d5640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efedc065b80 0x7efedc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:25.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.894+0000 7efefd0d5640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efedc065b80 0x7efedc068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efef4001ec0 tx=0x7efef4009040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:25.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:25.898+0000 7efeedffb640 1 -- 172.21.15.32:0/2641662765 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efee80eb1b0 con 0x7efef80c4b20 2023-12-20T04:43:26.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.054+0000 7efeff339640 1 -- 172.21.15.32:0/2641662765 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efef815d600 con 0x7efedc065b80 2023-12-20T04:43:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.062+0000 7efeedffb640 1 -- 172.21.15.32:0/2641662765 <== mgr.6183 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+281564 (secure 0 0 0) 0x7efef4005720 con 0x7efedc065b80 2023-12-20T04:43:26.065 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:26.066 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:43:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 -- 172.21.15.32:0/2641662765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efedc065b80 msgr2=0x7efedc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efedc065b80 0x7efedc068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efef4001ec0 tx=0x7efef4009040 comp rx=0 tx=0).stop 2023-12-20T04:43:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 -- 172.21.15.32:0/2641662765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef80c4b20 msgr2=0x7efef80b2e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef80c4b20 0x7efef80b2e30 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7efee801c880 tx=0x7efee4005800 comp rx=0 tx=0).stop 2023-12-20T04:43:26.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 -- 172.21.15.32:0/2641662765 shutdown_connections 2023-12-20T04:43:26.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efedc065b80 0x7efedc068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efef81f6bd0 0x7efef81f5f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efef80c4b20 0x7efef80b2e30 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 --2- 172.21.15.32:0/2641662765 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efee40047d0 0x7efef80b1610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 -- 172.21.15.32:0/2641662765 >> 172.21.15.32:0/2641662765 conn(0x7efef80cc310 msgr2=0x7efef80ca160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:26.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 -- 172.21.15.32:0/2641662765 shutdown_connections 2023-12-20T04:43:26.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.066+0000 7efeff339640 1 -- 172.21.15.32:0/2641662765 wait complete. 2023-12-20T04:43:26.086 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-20T04:43:24.257468+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591006,"num_objects":11,"num_object_clones":0,"num_object_copies":22,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":11,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":257,"num_write_kb":2425,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":178,"ondisk_log_size":178,"up":104,"acting":104,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":57,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":5,"kb":754974720,"kb_used":325684,"kb_used_data":8436,"kb_used_omap":0,"kb_used_meta":317184,"kb_avail":754649036,"statfs":{"total":773094113280,"available":772760612864,"internally_reserved":0,"allocated":8638464,"data_stored":4646688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":324796416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":23,"apply_latency_ms":23,"commit_latency_ns":23000000,"apply_latency_ns":23000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000703"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553392+0000","last_change":"2023-12-20T04:43:22.004072+0000","last_active":"2023-12-20T04:43:23.553392+0000","last_peered":"2023-12-20T04:43:23.553392+0000","last_clean":"2023-12-20T04:43:23.553392+0000","last_became_active":"2023-12-20T04:43:22.003778+0000","last_became_peered":"2023-12-20T04:43:22.003778+0000","last_unstale":"2023-12-20T04:43:23.553392+0000","last_undegraded":"2023-12-20T04:43:23.553392+0000","last_fullsized":"2023-12-20T04:43:23.553392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:02.386672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924910+0000","last_change":"2023-12-20T04:43:21.999126+0000","last_active":"2023-12-20T04:43:23.924910+0000","last_peered":"2023-12-20T04:43:23.924910+0000","last_clean":"2023-12-20T04:43:23.924910+0000","last_became_active":"2023-12-20T04:43:21.998841+0000","last_became_peered":"2023-12-20T04:43:21.998841+0000","last_unstale":"2023-12-20T04:43:23.924910+0000","last_undegraded":"2023-12-20T04:43:23.924910+0000","last_fullsized":"2023-12-20T04:43:23.924910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:50:16.003698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553212+0000","last_change":"2023-12-20T04:43:21.990156+0000","last_active":"2023-12-20T04:43:23.553212+0000","last_peered":"2023-12-20T04:43:23.553212+0000","last_clean":"2023-12-20T04:43:23.553212+0000","last_became_active":"2023-12-20T04:43:21.989878+0000","last_became_peered":"2023-12-20T04:43:21.989878+0000","last_unstale":"2023-12-20T04:43:23.553212+0000","last_undegraded":"2023-12-20T04:43:23.553212+0000","last_fullsized":"2023-12-20T04:43:23.553212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:28:27.070336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924333+0000","last_change":"2023-12-20T04:43:22.000099+0000","last_active":"2023-12-20T04:43:23.924333+0000","last_peered":"2023-12-20T04:43:23.924333+0000","last_clean":"2023-12-20T04:43:23.924333+0000","last_became_active":"2023-12-20T04:43:21.999807+0000","last_became_peered":"2023-12-20T04:43:21.999807+0000","last_unstale":"2023-12-20T04:43:23.924333+0000","last_undegraded":"2023-12-20T04:43:23.924333+0000","last_fullsized":"2023-12-20T04:43:23.924333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:45:38.258467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924041+0000","last_change":"2023-12-20T04:43:21.995443+0000","last_active":"2023-12-20T04:43:23.924041+0000","last_peered":"2023-12-20T04:43:23.924041+0000","last_clean":"2023-12-20T04:43:23.924041+0000","last_became_active":"2023-12-20T04:43:21.995137+0000","last_became_peered":"2023-12-20T04:43:21.995137+0000","last_unstale":"2023-12-20T04:43:23.924041+0000","last_undegraded":"2023-12-20T04:43:23.924041+0000","last_fullsized":"2023-12-20T04:43:23.924041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:14.983424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983122+0000","last_change":"2023-12-20T04:43:22.002105+0000","last_active":"2023-12-20T04:43:22.983122+0000","last_peered":"2023-12-20T04:43:22.983122+0000","last_clean":"2023-12-20T04:43:22.983122+0000","last_became_active":"2023-12-20T04:43:22.001825+0000","last_became_peered":"2023-12-20T04:43:22.001825+0000","last_unstale":"2023-12-20T04:43:22.983122+0000","last_undegraded":"2023-12-20T04:43:22.983122+0000","last_fullsized":"2023-12-20T04:43:22.983122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:57:47.673127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982987+0000","last_change":"2023-12-20T04:43:22.000044+0000","last_active":"2023-12-20T04:43:22.982987+0000","last_peered":"2023-12-20T04:43:22.982987+0000","last_clean":"2023-12-20T04:43:22.982987+0000","last_became_active":"2023-12-20T04:43:21.999745+0000","last_became_peered":"2023-12-20T04:43:21.999745+0000","last_unstale":"2023-12-20T04:43:22.982987+0000","last_undegraded":"2023-12-20T04:43:22.982987+0000","last_fullsized":"2023-12-20T04:43:22.982987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:23.027733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924223+0000","last_change":"2023-12-20T04:43:22.001435+0000","last_active":"2023-12-20T04:43:23.924223+0000","last_peered":"2023-12-20T04:43:23.924223+0000","last_clean":"2023-12-20T04:43:23.924223+0000","last_became_active":"2023-12-20T04:43:22.001172+0000","last_became_peered":"2023-12-20T04:43:22.001172+0000","last_unstale":"2023-12-20T04:43:23.924223+0000","last_undegraded":"2023-12-20T04:43:23.924223+0000","last_fullsized":"2023-12-20T04:43:23.924223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:45:44.114176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924967+0000","last_change":"2023-12-20T04:43:22.003640+0000","last_active":"2023-12-20T04:43:23.924967+0000","last_peered":"2023-12-20T04:43:23.924967+0000","last_clean":"2023-12-20T04:43:23.924967+0000","last_became_active":"2023-12-20T04:43:22.003389+0000","last_became_peered":"2023-12-20T04:43:22.003389+0000","last_unstale":"2023-12-20T04:43:23.924967+0000","last_undegraded":"2023-12-20T04:43:23.924967+0000","last_fullsized":"2023-12-20T04:43:23.924967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:59:16.761895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553256+0000","last_change":"2023-12-20T04:43:21.996601+0000","last_active":"2023-12-20T04:43:23.553256+0000","last_peered":"2023-12-20T04:43:23.553256+0000","last_clean":"2023-12-20T04:43:23.553256+0000","last_became_active":"2023-12-20T04:43:21.993330+0000","last_became_peered":"2023-12-20T04:43:21.993330+0000","last_unstale":"2023-12-20T04:43:23.553256+0000","last_undegraded":"2023-12-20T04:43:23.553256+0000","last_fullsized":"2023-12-20T04:43:23.553256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:24:26.876505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924699+0000","last_change":"2023-12-20T04:43:22.005461+0000","last_active":"2023-12-20T04:43:23.924699+0000","last_peered":"2023-12-20T04:43:23.924699+0000","last_clean":"2023-12-20T04:43:23.924699+0000","last_became_active":"2023-12-20T04:43:22.005188+0000","last_became_peered":"2023-12-20T04:43:22.005188+0000","last_unstale":"2023-12-20T04:43:23.924699+0000","last_undegraded":"2023-12-20T04:43:23.924699+0000","last_fullsized":"2023-12-20T04:43:23.924699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:10:36.287963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.923924+0000","last_change":"2023-12-20T04:43:22.003002+0000","last_active":"2023-12-20T04:43:23.923924+0000","last_peered":"2023-12-20T04:43:23.923924+0000","last_clean":"2023-12-20T04:43:23.923924+0000","last_became_active":"2023-12-20T04:43:22.002687+0000","last_became_peered":"2023-12-20T04:43:22.002687+0000","last_unstale":"2023-12-20T04:43:23.923924+0000","last_undegraded":"2023-12-20T04:43:23.923924+0000","last_fullsized":"2023-12-20T04:43:23.923924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:22.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553458+0000","last_change":"2023-12-20T04:43:21.996101+0000","last_active":"2023-12-20T04:43:23.553458+0000","last_peered":"2023-12-20T04:43:23.553458+0000","last_clean":"2023-12-20T04:43:23.553458+0000","last_became_active":"2023-12-20T04:43:21.995817+0000","last_became_peered":"2023-12-20T04:43:21.995817+0000","last_unstale":"2023-12-20T04:43:23.553458+0000","last_undegraded":"2023-12-20T04:43:23.553458+0000","last_fullsized":"2023-12-20T04:43:23.553458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:59:23.231757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924759+0000","last_change":"2023-12-20T04:43:22.008701+0000","last_active":"2023-12-20T04:43:23.924759+0000","last_peered":"2023-12-20T04:43:23.924759+0000","last_clean":"2023-12-20T04:43:23.924759+0000","last_became_active":"2023-12-20T04:43:22.008420+0000","last_became_peered":"2023-12-20T04:43:22.008420+0000","last_unstale":"2023-12-20T04:43:23.924759+0000","last_undegraded":"2023-12-20T04:43:23.924759+0000","last_fullsized":"2023-12-20T04:43:23.924759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:12:28.334990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553213+0000","last_change":"2023-12-20T04:43:21.994859+0000","last_active":"2023-12-20T04:43:23.553213+0000","last_peered":"2023-12-20T04:43:23.553213+0000","last_clean":"2023-12-20T04:43:23.553213+0000","last_became_active":"2023-12-20T04:43:21.990501+0000","last_became_peered":"2023-12-20T04:43:21.990501+0000","last_unstale":"2023-12-20T04:43:23.553213+0000","last_undegraded":"2023-12-20T04:43:23.553213+0000","last_fullsized":"2023-12-20T04:43:23.553213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:18:47.468708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924542+0000","last_change":"2023-12-20T04:43:21.990922+0000","last_active":"2023-12-20T04:43:23.924542+0000","last_peered":"2023-12-20T04:43:23.924542+0000","last_clean":"2023-12-20T04:43:23.924542+0000","last_became_active":"2023-12-20T04:43:21.990665+0000","last_became_peered":"2023-12-20T04:43:21.990665+0000","last_unstale":"2023-12-20T04:43:23.924542+0000","last_undegraded":"2023-12-20T04:43:23.924542+0000","last_fullsized":"2023-12-20T04:43:23.924542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:37.847365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553099+0000","last_change":"2023-12-20T04:43:21.996074+0000","last_active":"2023-12-20T04:43:23.553099+0000","last_peered":"2023-12-20T04:43:23.553099+0000","last_clean":"2023-12-20T04:43:23.553099+0000","last_became_active":"2023-12-20T04:43:21.995790+0000","last_became_peered":"2023-12-20T04:43:21.995790+0000","last_unstale":"2023-12-20T04:43:23.553099+0000","last_undegraded":"2023-12-20T04:43:23.553099+0000","last_fullsized":"2023-12-20T04:43:23.553099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:45:36.939730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924534+0000","last_change":"2023-12-20T04:43:22.010168+0000","last_active":"2023-12-20T04:43:23.924534+0000","last_peered":"2023-12-20T04:43:23.924534+0000","last_clean":"2023-12-20T04:43:23.924534+0000","last_became_active":"2023-12-20T04:43:22.009235+0000","last_became_peered":"2023-12-20T04:43:22.009235+0000","last_unstale":"2023-12-20T04:43:23.924534+0000","last_undegraded":"2023-12-20T04:43:23.924534+0000","last_fullsized":"2023-12-20T04:43:23.924534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:18.578713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924240+0000","last_change":"2023-12-20T04:43:22.010640+0000","last_active":"2023-12-20T04:43:23.924240+0000","last_peered":"2023-12-20T04:43:23.924240+0000","last_clean":"2023-12-20T04:43:23.924240+0000","last_became_active":"2023-12-20T04:43:22.010344+0000","last_became_peered":"2023-12-20T04:43:22.010344+0000","last_unstale":"2023-12-20T04:43:23.924240+0000","last_undegraded":"2023-12-20T04:43:23.924240+0000","last_fullsized":"2023-12-20T04:43:23.924240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:28:50.727344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"78'4","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.925285+0000","last_change":"2023-12-20T04:43:21.587084+0000","last_active":"2023-12-20T04:43:23.925285+0000","last_peered":"2023-12-20T04:43:23.925285+0000","last_clean":"2023-12-20T04:43:23.925285+0000","last_became_active":"2023-12-20T04:43:21.586819+0000","last_became_peered":"2023-12-20T04:43:21.586819+0000","last_unstale":"2023-12-20T04:43:23.925285+0000","last_undegraded":"2023-12-20T04:43:23.925285+0000","last_fullsized":"2023-12-20T04:43:23.925285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:48:11.323426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983139+0000","last_change":"2023-12-20T04:43:22.007020+0000","last_active":"2023-12-20T04:43:22.983139+0000","last_peered":"2023-12-20T04:43:22.983139+0000","last_clean":"2023-12-20T04:43:22.983139+0000","last_became_active":"2023-12-20T04:43:22.004884+0000","last_became_peered":"2023-12-20T04:43:22.004884+0000","last_unstale":"2023-12-20T04:43:22.983139+0000","last_undegraded":"2023-12-20T04:43:22.983139+0000","last_fullsized":"2023-12-20T04:43:22.983139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:11:51.597335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"78'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.925154+0000","last_change":"2023-12-20T04:43:21.584516+0000","last_active":"2023-12-20T04:43:23.925154+0000","last_peered":"2023-12-20T04:43:23.925154+0000","last_clean":"2023-12-20T04:43:23.925154+0000","last_became_active":"2023-12-20T04:43:21.584196+0000","last_became_peered":"2023-12-20T04:43:21.584196+0000","last_unstale":"2023-12-20T04:43:23.925154+0000","last_undegraded":"2023-12-20T04:43:23.925154+0000","last_fullsized":"2023-12-20T04:43:23.925154+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:24.794750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924399+0000","last_change":"2023-12-20T04:43:21.999085+0000","last_active":"2023-12-20T04:43:23.924399+0000","last_peered":"2023-12-20T04:43:23.924399+0000","last_clean":"2023-12-20T04:43:23.924399+0000","last_became_active":"2023-12-20T04:43:21.998844+0000","last_became_peered":"2023-12-20T04:43:21.998844+0000","last_unstale":"2023-12-20T04:43:23.924399+0000","last_undegraded":"2023-12-20T04:43:23.924399+0000","last_fullsized":"2023-12-20T04:43:23.924399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:08.735681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"78'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924858+0000","last_change":"2023-12-20T04:43:21.586335+0000","last_active":"2023-12-20T04:43:23.924858+0000","last_peered":"2023-12-20T04:43:23.924858+0000","last_clean":"2023-12-20T04:43:23.924858+0000","last_became_active":"2023-12-20T04:43:21.586047+0000","last_became_peered":"2023-12-20T04:43:21.586047+0000","last_unstale":"2023-12-20T04:43:23.924858+0000","last_undegraded":"2023-12-20T04:43:23.924858+0000","last_fullsized":"2023-12-20T04:43:23.924858+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:14:56.160293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924758+0000","last_change":"2023-12-20T04:43:22.010407+0000","last_active":"2023-12-20T04:43:23.924758+0000","last_peered":"2023-12-20T04:43:23.924758+0000","last_clean":"2023-12-20T04:43:23.924758+0000","last_became_active":"2023-12-20T04:43:22.009339+0000","last_became_peered":"2023-12-20T04:43:22.009339+0000","last_unstale":"2023-12-20T04:43:23.924758+0000","last_undegraded":"2023-12-20T04:43:23.924758+0000","last_fullsized":"2023-12-20T04:43:23.924758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:56:34.953824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924380+0000","last_change":"2023-12-20T04:43:21.999575+0000","last_active":"2023-12-20T04:43:23.924380+0000","last_peered":"2023-12-20T04:43:23.924380+0000","last_clean":"2023-12-20T04:43:23.924380+0000","last_became_active":"2023-12-20T04:43:21.999330+0000","last_became_peered":"2023-12-20T04:43:21.999330+0000","last_unstale":"2023-12-20T04:43:23.924380+0000","last_undegraded":"2023-12-20T04:43:23.924380+0000","last_fullsized":"2023-12-20T04:43:23.924380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:45:42.523068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924185+0000","last_change":"2023-12-20T04:43:22.009978+0000","last_active":"2023-12-20T04:43:23.924185+0000","last_peered":"2023-12-20T04:43:23.924185+0000","last_clean":"2023-12-20T04:43:23.924185+0000","last_became_active":"2023-12-20T04:43:22.009709+0000","last_became_peered":"2023-12-20T04:43:22.009709+0000","last_unstale":"2023-12-20T04:43:23.924185+0000","last_undegraded":"2023-12-20T04:43:23.924185+0000","last_fullsized":"2023-12-20T04:43:23.924185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:57:20.248762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"78'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924466+0000","last_change":"2023-12-20T04:43:21.584942+0000","last_active":"2023-12-20T04:43:23.924466+0000","last_peered":"2023-12-20T04:43:23.924466+0000","last_clean":"2023-12-20T04:43:23.924466+0000","last_became_active":"2023-12-20T04:43:21.584665+0000","last_became_peered":"2023-12-20T04:43:21.584665+0000","last_unstale":"2023-12-20T04:43:23.924466+0000","last_undegraded":"2023-12-20T04:43:23.924466+0000","last_fullsized":"2023-12-20T04:43:23.924466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:29:14.050749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924199+0000","last_change":"2023-12-20T04:43:21.994327+0000","last_active":"2023-12-20T04:43:23.924199+0000","last_peered":"2023-12-20T04:43:23.924199+0000","last_clean":"2023-12-20T04:43:23.924199+0000","last_became_active":"2023-12-20T04:43:21.993655+0000","last_became_peered":"2023-12-20T04:43:21.993655+0000","last_unstale":"2023-12-20T04:43:23.924199+0000","last_undegraded":"2023-12-20T04:43:23.924199+0000","last_fullsized":"2023-12-20T04:43:23.924199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:08.735681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.925180+0000","last_change":"2023-12-20T04:43:21.587613+0000","last_active":"2023-12-20T04:43:23.925180+0000","last_peered":"2023-12-20T04:43:23.925180+0000","last_clean":"2023-12-20T04:43:23.925180+0000","last_became_active":"2023-12-20T04:43:21.587311+0000","last_became_peered":"2023-12-20T04:43:21.587311+0000","last_unstale":"2023-12-20T04:43:23.925180+0000","last_undegraded":"2023-12-20T04:43:23.925180+0000","last_fullsized":"2023-12-20T04:43:23.925180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:31:04.259225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982780+0000","last_change":"2023-12-20T04:43:21.998623+0000","last_active":"2023-12-20T04:43:22.982780+0000","last_peered":"2023-12-20T04:43:22.982780+0000","last_clean":"2023-12-20T04:43:22.982780+0000","last_became_active":"2023-12-20T04:43:21.998219+0000","last_became_peered":"2023-12-20T04:43:21.998219+0000","last_unstale":"2023-12-20T04:43:22.982780+0000","last_undegraded":"2023-12-20T04:43:22.982780+0000","last_fullsized":"2023-12-20T04:43:22.982780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:40:34.558866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.13","version":"78'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924358+0000","last_change":"2023-12-20T04:43:21.590148+0000","last_active":"2023-12-20T04:43:23.924358+0000","last_peered":"2023-12-20T04:43:23.924358+0000","last_clean":"2023-12-20T04:43:23.924358+0000","last_became_active":"2023-12-20T04:43:21.589854+0000","last_became_peered":"2023-12-20T04:43:21.589854+0000","last_unstale":"2023-12-20T04:43:23.924358+0000","last_undegraded":"2023-12-20T04:43:23.924358+0000","last_fullsized":"2023-12-20T04:43:23.924358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:17:46.717611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.f","version":"78'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.925157+0000","last_change":"2023-12-20T04:43:21.592864+0000","last_active":"2023-12-20T04:43:23.925157+0000","last_peered":"2023-12-20T04:43:23.925157+0000","last_clean":"2023-12-20T04:43:23.925157+0000","last_became_active":"2023-12-20T04:43:21.592590+0000","last_became_peered":"2023-12-20T04:43:21.592590+0000","last_unstale":"2023-12-20T04:43:23.925157+0000","last_undegraded":"2023-12-20T04:43:23.925157+0000","last_fullsized":"2023-12-20T04:43:23.925157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:32.934724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.e","version":"78'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924372+0000","last_change":"2023-12-20T04:43:21.586179+0000","last_active":"2023-12-20T04:43:23.924372+0000","last_peered":"2023-12-20T04:43:23.924372+0000","last_clean":"2023-12-20T04:43:23.924372+0000","last_became_active":"2023-12-20T04:43:21.585664+0000","last_became_peered":"2023-12-20T04:43:21.585664+0000","last_unstale":"2023-12-20T04:43:23.924372+0000","last_undegraded":"2023-12-20T04:43:23.924372+0000","last_fullsized":"2023-12-20T04:43:23.924372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:11:15.027492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924704+0000","last_change":"2023-12-20T04:43:21.992201+0000","last_active":"2023-12-20T04:43:23.924704+0000","last_peered":"2023-12-20T04:43:23.924704+0000","last_clean":"2023-12-20T04:43:23.924704+0000","last_became_active":"2023-12-20T04:43:21.991955+0000","last_became_peered":"2023-12-20T04:43:21.991955+0000","last_unstale":"2023-12-20T04:43:23.924704+0000","last_undegraded":"2023-12-20T04:43:23.924704+0000","last_fullsized":"2023-12-20T04:43:23.924704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:24:50.593990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924509+0000","last_change":"2023-12-20T04:43:21.585469+0000","last_active":"2023-12-20T04:43:23.924509+0000","last_peered":"2023-12-20T04:43:23.924509+0000","last_clean":"2023-12-20T04:43:23.924509+0000","last_became_active":"2023-12-20T04:43:21.585168+0000","last_became_peered":"2023-12-20T04:43:21.585168+0000","last_unstale":"2023-12-20T04:43:23.924509+0000","last_undegraded":"2023-12-20T04:43:23.924509+0000","last_fullsized":"2023-12-20T04:43:23.924509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:39:13.024438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.b","version":"78'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924642+0000","last_change":"2023-12-20T04:43:21.580313+0000","last_active":"2023-12-20T04:43:23.924642+0000","last_peered":"2023-12-20T04:43:23.924642+0000","last_clean":"2023-12-20T04:43:23.924642+0000","last_became_active":"2023-12-20T04:43:21.579994+0000","last_became_peered":"2023-12-20T04:43:21.579994+0000","last_unstale":"2023-12-20T04:43:23.924642+0000","last_undegraded":"2023-12-20T04:43:23.924642+0000","last_fullsized":"2023-12-20T04:43:23.924642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:11:18.990305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924483+0000","last_change":"2023-12-20T04:43:22.008471+0000","last_active":"2023-12-20T04:43:23.924483+0000","last_peered":"2023-12-20T04:43:23.924483+0000","last_clean":"2023-12-20T04:43:23.924483+0000","last_became_active":"2023-12-20T04:43:22.003050+0000","last_became_peered":"2023-12-20T04:43:22.003050+0000","last_unstale":"2023-12-20T04:43:23.924483+0000","last_undegraded":"2023-12-20T04:43:23.924483+0000","last_fullsized":"2023-12-20T04:43:23.924483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:18.582906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982602+0000","last_change":"2023-12-20T04:43:21.583671+0000","last_active":"2023-12-20T04:43:22.982602+0000","last_peered":"2023-12-20T04:43:22.982602+0000","last_clean":"2023-12-20T04:43:22.982602+0000","last_became_active":"2023-12-20T04:43:21.583405+0000","last_became_peered":"2023-12-20T04:43:21.583405+0000","last_unstale":"2023-12-20T04:43:22.982602+0000","last_undegraded":"2023-12-20T04:43:22.982602+0000","last_fullsized":"2023-12-20T04:43:22.982602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:21:38.152790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924004+0000","last_change":"2023-12-20T04:43:22.001495+0000","last_active":"2023-12-20T04:43:23.924004+0000","last_peered":"2023-12-20T04:43:23.924004+0000","last_clean":"2023-12-20T04:43:23.924004+0000","last_became_active":"2023-12-20T04:43:21.999535+0000","last_became_peered":"2023-12-20T04:43:21.999535+0000","last_unstale":"2023-12-20T04:43:23.924004+0000","last_undegraded":"2023-12-20T04:43:23.924004+0000","last_fullsized":"2023-12-20T04:43:23.924004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:12:54.915087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.7","version":"78'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983185+0000","last_change":"2023-12-20T04:43:21.586777+0000","last_active":"2023-12-20T04:43:22.983185+0000","last_peered":"2023-12-20T04:43:22.983185+0000","last_clean":"2023-12-20T04:43:22.983185+0000","last_became_active":"2023-12-20T04:43:21.586509+0000","last_became_peered":"2023-12-20T04:43:21.586509+0000","last_unstale":"2023-12-20T04:43:22.983185+0000","last_undegraded":"2023-12-20T04:43:22.983185+0000","last_fullsized":"2023-12-20T04:43:22.983185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:33:21.576017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924994+0000","last_change":"2023-12-20T04:43:21.588511+0000","last_active":"2023-12-20T04:43:23.924994+0000","last_peered":"2023-12-20T04:43:23.924994+0000","last_clean":"2023-12-20T04:43:23.924994+0000","last_became_active":"2023-12-20T04:43:21.588223+0000","last_became_peered":"2023-12-20T04:43:21.588223+0000","last_unstale":"2023-12-20T04:43:23.924994+0000","last_undegraded":"2023-12-20T04:43:23.924994+0000","last_fullsized":"2023-12-20T04:43:23.924994+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:40:15.874023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982789+0000","last_change":"2023-12-20T04:43:21.998457+0000","last_active":"2023-12-20T04:43:22.982789+0000","last_peered":"2023-12-20T04:43:22.982789+0000","last_clean":"2023-12-20T04:43:22.982789+0000","last_became_active":"2023-12-20T04:43:21.997835+0000","last_became_peered":"2023-12-20T04:43:21.997835+0000","last_unstale":"2023-12-20T04:43:22.982789+0000","last_undegraded":"2023-12-20T04:43:22.982789+0000","last_fullsized":"2023-12-20T04:43:22.982789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:27.937468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924883+0000","last_change":"2023-12-20T04:43:22.010202+0000","last_active":"2023-12-20T04:43:23.924883+0000","last_peered":"2023-12-20T04:43:23.924883+0000","last_clean":"2023-12-20T04:43:23.924883+0000","last_became_active":"2023-12-20T04:43:22.008971+0000","last_became_peered":"2023-12-20T04:43:22.008971+0000","last_unstale":"2023-12-20T04:43:23.924883+0000","last_undegraded":"2023-12-20T04:43:23.924883+0000","last_fullsized":"2023-12-20T04:43:23.924883+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:31:28.967106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924057+0000","last_change":"2023-12-20T04:43:21.996972+0000","last_active":"2023-12-20T04:43:23.924057+0000","last_peered":"2023-12-20T04:43:23.924057+0000","last_clean":"2023-12-20T04:43:23.924057+0000","last_became_active":"2023-12-20T04:43:21.996710+0000","last_became_peered":"2023-12-20T04:43:21.996710+0000","last_unstale":"2023-12-20T04:43:23.924057+0000","last_undegraded":"2023-12-20T04:43:23.924057+0000","last_fullsized":"2023-12-20T04:43:23.924057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:12:54.915087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982570+0000","last_change":"2023-12-20T04:43:21.577608+0000","last_active":"2023-12-20T04:43:22.982570+0000","last_peered":"2023-12-20T04:43:22.982570+0000","last_clean":"2023-12-20T04:43:22.982570+0000","last_became_active":"2023-12-20T04:43:21.577261+0000","last_became_peered":"2023-12-20T04:43:21.577261+0000","last_unstale":"2023-12-20T04:43:22.982570+0000","last_undegraded":"2023-12-20T04:43:22.982570+0000","last_fullsized":"2023-12-20T04:43:22.982570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:35:25.100304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:21.557238+0000","last_change":"2023-12-20T04:43:21.557238+0000","last_active":"2023-12-20T04:43:21.557238+0000","last_peered":"2023-12-20T04:43:21.557238+0000","last_clean":"2023-12-20T04:43:21.557238+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:21.557238+0000","last_undegraded":"2023-12-20T04:43:21.557238+0000","last_fullsized":"2023-12-20T04:43:21.557238+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982376+0000","last_change":"2023-12-20T04:43:21.996415+0000","last_active":"2023-12-20T04:43:22.982376+0000","last_peered":"2023-12-20T04:43:22.982376+0000","last_clean":"2023-12-20T04:43:22.982376+0000","last_became_active":"2023-12-20T04:43:21.995386+0000","last_became_peered":"2023-12-20T04:43:21.995386+0000","last_unstale":"2023-12-20T04:43:22.982376+0000","last_undegraded":"2023-12-20T04:43:22.982376+0000","last_fullsized":"2023-12-20T04:43:22.982376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:03:33.402892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982436+0000","last_change":"2023-12-20T04:43:21.590630+0000","last_active":"2023-12-20T04:43:22.982436+0000","last_peered":"2023-12-20T04:43:22.982436+0000","last_clean":"2023-12-20T04:43:22.982436+0000","last_became_active":"2023-12-20T04:43:21.590362+0000","last_became_peered":"2023-12-20T04:43:21.590362+0000","last_unstale":"2023-12-20T04:43:22.982436+0000","last_undegraded":"2023-12-20T04:43:22.982436+0000","last_fullsized":"2023-12-20T04:43:22.982436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:47:01.682508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924306+0000","last_change":"2023-12-20T04:43:21.994496+0000","last_active":"2023-12-20T04:43:23.924306+0000","last_peered":"2023-12-20T04:43:23.924306+0000","last_clean":"2023-12-20T04:43:23.924306+0000","last_became_active":"2023-12-20T04:43:21.994034+0000","last_became_peered":"2023-12-20T04:43:21.994034+0000","last_unstale":"2023-12-20T04:43:23.924306+0000","last_undegraded":"2023-12-20T04:43:23.924306+0000","last_fullsized":"2023-12-20T04:43:23.924306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:05:34.433133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924072+0000","last_change":"2023-12-20T04:43:21.580845+0000","last_active":"2023-12-20T04:43:23.924072+0000","last_peered":"2023-12-20T04:43:23.924072+0000","last_clean":"2023-12-20T04:43:23.924072+0000","last_became_active":"2023-12-20T04:43:21.580582+0000","last_became_peered":"2023-12-20T04:43:21.580582+0000","last_unstale":"2023-12-20T04:43:23.924072+0000","last_undegraded":"2023-12-20T04:43:23.924072+0000","last_fullsized":"2023-12-20T04:43:23.924072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:10.610546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.923973+0000","last_change":"2023-12-20T04:43:22.004878+0000","last_active":"2023-12-20T04:43:23.923973+0000","last_peered":"2023-12-20T04:43:23.923973+0000","last_clean":"2023-12-20T04:43:23.923973+0000","last_became_active":"2023-12-20T04:43:22.004628+0000","last_became_peered":"2023-12-20T04:43:22.004628+0000","last_unstale":"2023-12-20T04:43:23.923973+0000","last_undegraded":"2023-12-20T04:43:23.923973+0000","last_fullsized":"2023-12-20T04:43:23.923973+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:28:50.727344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:43:20.549706+0000","last_change":"2023-12-20T04:43:20.549706+0000","last_active":"2023-12-20T04:43:20.549706+0000","last_peered":"2023-12-20T04:43:20.549706+0000","last_clean":"2023-12-20T04:43:20.549706+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:43:20.549706+0000","last_undegraded":"2023-12-20T04:43:20.549706+0000","last_fullsized":"2023-12-20T04:43:20.549706+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"67'161","reported_seq":370,"reported_epoch":74,"state":"active+clean","last_fresh":"2023-12-20T04:43:16.748344+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:43:16.748344+0000","last_peered":"2023-12-20T04:43:16.748344+0000","last_clean":"2023-12-20T04:43:16.748344+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:43:16.748344+0000","last_undegraded":"2023-12-20T04:43:16.748344+0000","last_fullsized":"2023-12-20T04:43:16.748344+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":70,"acting":70,"num_store_stats":4},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":638,"num_objects":9,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":9,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":2348,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":17,"ondisk_log_size":17,"up":32,"acting":32,"num_store_stats":4},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575027,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36608,"kb_used_data":1464,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335232,"statfs":{"total":96636764160,"available":96599277568,"internally_reserved":0,"allocated":1499136,"data_stored":1012314,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.372,"15min":0.33000000000000002},"max":{"1min":1.048,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.64900000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.71399999999999997,"15min":0.71099999999999997},"min":{"1min":0.56399999999999995,"5min":0.39600000000000002,"15min":0.39400000000000002},"max":{"1min":0.89300000000000002,"5min":1,"15min":1.115},"last":0.77000000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73199999999999998,"15min":0.72299999999999998},"min":{"1min":0.55900000000000005,"5min":0.443,"15min":0.38100000000000001},"max":{"1min":1.0369999999999999,"5min":1.095,"15min":1.2190000000000001},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.745},"min":{"1min":0.51000000000000001,"5min":0.50800000000000001,"15min":0.42999999999999999},"max":{"1min":0.95099999999999996,"5min":1.032,"15min":2.649},"last":0.67900000000000005}]},{"osd":2,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.56200000000000006,"5min":0.48899999999999999,"15min":0.437},"max":{"1min":0.94099999999999995,"5min":1.075,"15min":1.5960000000000001},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.751,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.46899999999999997,"15min":0.45300000000000001},"max":{"1min":0.97999999999999998,"5min":1.1000000000000001,"15min":1.135},"last":0.85699999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.999,"5min":1.016,"15min":1.641},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77200000000000002,"15min":0.77900000000000003},"min":{"1min":0.59799999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93899999999999995,"5min":1.131,"15min":2.5990000000000002},"last":0.79900000000000004}]},{"osd":4,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45400000000000001,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":0.90700000000000003,"5min":0.98299999999999998,"15min":1.0109999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.627,"15min":0.63600000000000001},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":1.0469999999999999,"15min":1.242},"last":0.69299999999999995}]},{"osd":5,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.496,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.0900000000000001},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.442,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":1.052,"15min":1.071},"last":0.82299999999999995}]},{"osd":6,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.746},"min":{"1min":0.60699999999999998,"5min":0.53100000000000003,"15min":0.443},"max":{"1min":1.012,"5min":1.1020000000000001,"15min":1.1279999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72399999999999998,"15min":0.71299999999999997},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":1.0309999999999999,"5min":1.125,"15min":1.125},"last":0.628}]}]},{"osd":6,"up_from":13,"seq":55834575027,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36028,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335812,"statfs":{"total":96636764160,"available":96599871488,"internally_reserved":0,"allocated":905216,"data_stored":421946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.55600000000000005,"5min":0.33800000000000002,"15min":0.27100000000000002},"max":{"1min":0.92900000000000005,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.68000000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.71999999999999997,"15min":0.71699999999999997},"min":{"1min":0.626,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.1910000000000001},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69099999999999995,"15min":0.70199999999999996},"min":{"1min":0.55100000000000005,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.49199999999999999},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.57699999999999996,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.97799999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.61299999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.68400000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.006,"5min":1.216,"15min":1.216},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.38300000000000001},"max":{"1min":0.95599999999999996,"5min":1.0089999999999999,"15min":1.054},"last":0.66800000000000004}]},{"osd":3,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.78500000000000003,"15min":0.75900000000000001},"min":{"1min":0.68300000000000005,"5min":0.51800000000000002,"15min":0.45600000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.109},"last":0.71599999999999997},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96499999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.435}]},{"osd":4,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.624,"15min":0.621},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.36399999999999999},"max":{"1min":0.90300000000000002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":0.995,"15min":1.071},"last":0.78000000000000003}]},{"osd":5,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.51000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.64500000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.68999999999999995},"min":{"1min":0.54400000000000004,"5min":0.435,"15min":0.434},"max":{"1min":1,"5min":1.306,"15min":1.306},"last":0.76800000000000002}]},{"osd":7,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72599999999999998,"15min":0.72299999999999998},"min":{"1min":0.61899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.2270000000000001},"last":0.755},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58899999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":1.002,"15min":1.169},"last":0.70299999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575028,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36028,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335812,"statfs":{"total":96636764160,"available":96599871488,"internally_reserved":0,"allocated":905216,"data_stored":421946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.69699999999999995,"15min":0.68799999999999994},"min":{"1min":0.55200000000000005,"5min":0.33700000000000002,"15min":0.29999999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.05},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.755,"5min":0.71899999999999997,"15min":0.71399999999999997},"min":{"1min":0.55800000000000005,"5min":0.41599999999999998,"15min":0.34300000000000003},"max":{"1min":0.85899999999999999,"5min":0.95099999999999996,"15min":0.98599999999999999},"last":0.51700000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.71699999999999997,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.45500000000000002,"15min":0.441},"max":{"1min":0.92700000000000005,"5min":1.044,"15min":1.0600000000000001},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.73699999999999999},"min":{"1min":0.60299999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.1140000000000001},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.54000000000000004,"5min":0.432,"15min":0.40500000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.153},"last":0.66800000000000004},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.58599999999999997,"5min":0.48799999999999999,"15min":0.47899999999999998},"max":{"1min":0.91600000000000004,"5min":0.95999999999999996,"15min":1.004},"last":0.80600000000000005}]},{"osd":3,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74299999999999999,"15min":0.748},"min":{"1min":0.56299999999999994,"5min":0.48799999999999999,"15min":0.47499999999999998},"max":{"1min":0.94999999999999996,"5min":0.98099999999999998,"15min":1.758},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.749},"min":{"1min":0.62,"5min":0.46899999999999997,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":1.0469999999999999},"last":0.75600000000000001}]},{"osd":4,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.40600000000000003,"15min":0.35099999999999998},"max":{"1min":0.76300000000000001,"5min":0.97299999999999998,"15min":1.0229999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.39600000000000002},"max":{"1min":0.81000000000000005,"5min":0.98699999999999999,"15min":1.077},"last":0.78300000000000003}]},{"osd":6,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68100000000000005,"15min":0.68200000000000005},"min":{"1min":0.499,"5min":0.46400000000000002,"15min":0.39900000000000002},"max":{"1min":0.90000000000000002,"5min":0.94699999999999995,"15min":1.0700000000000001},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.38500000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86599999999999999,"5min":0.999,"15min":0.999},"last":0.751}]},{"osd":7,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.57599999999999996,"5min":0.48999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83899999999999997,"5min":1.0429999999999999,"15min":1.141},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.754,"15min":0.753},"min":{"1min":0.59699999999999998,"5min":0.54900000000000004,"15min":0.47899999999999998},"max":{"1min":0.95799999999999996,"5min":1.012,"15min":1.119},"last":0.81999999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575029,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":58172,"kb_used_data":884,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313668,"statfs":{"total":96636764160,"available":96577196032,"internally_reserved":0,"allocated":905216,"data_stored":421946,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0129999999999999,"5min":1.024,"15min":1.5509999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70599999999999996,"15min":0.70699999999999996},"min":{"1min":0.51900000000000002,"5min":0.376,"15min":0.36199999999999999},"max":{"1min":0.89500000000000002,"5min":1.625,"15min":1.633},"last":0.82399999999999995}]},{"osd":1,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.70699999999999996,"15min":0.71599999999999997},"min":{"1min":0.55200000000000005,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.95199999999999996,"5min":0.98799999999999999,"15min":1.798},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.752,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.036,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.78500000000000003}]},{"osd":2,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.66000000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0209999999999999,"5min":1.52,"15min":1.728},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.75,"15min":0.754},"min":{"1min":0.46899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":1.093,"15min":1.6619999999999999},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72899999999999998,"15min":0.72599999999999998},"min":{"1min":0.56699999999999995,"5min":0.41799999999999998,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.554,"15min":1.5760000000000001},"last":0.88600000000000001},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.63,"5min":0.498,"15min":0.48299999999999998},"max":{"1min":0.98399999999999999,"5min":1.0620000000000001,"15min":1.7649999999999999},"last":0.70399999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.627},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.91300000000000003,"5min":1.456,"15min":1.456},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.625,"5min":0.63400000000000001,"15min":0.63700000000000001},"min":{"1min":0.40200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96599999999999997,"5min":1.589,"15min":1.589},"last":0.97899999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65000000000000002,"15min":0.65900000000000003},"min":{"1min":0.47899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89200000000000002,"5min":1.099,"15min":1.6739999999999999},"last":0.754},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.70099999999999996},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96199999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.99099999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70099999999999996,"15min":0.69799999999999995},"min":{"1min":0.46700000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94099999999999995,"5min":1.655,"15min":1.655},"last":0.93200000000000005},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.432,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.052,"5min":1.125,"15min":1.125},"last":0.81000000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575029,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":950272,"data_stored":440861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.497},"min":{"1min":0.32900000000000001,"5min":0.26200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64300000000000002,"5min":0.875,"15min":4.3700000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.30599999999999999,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.80800000000000005,"15min":3.355},"last":0.377}]},{"osd":1,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.47599999999999998,"15min":0.498},"min":{"1min":0.38500000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":3.7160000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.42699999999999999,"5min":0.316,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.78100000000000003,"15min":3.3109999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.51600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":4.0110000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55800000000000005,"15min":0.60599999999999998},"min":{"1min":0.442,"5min":0.40500000000000003,"15min":0.316},"max":{"1min":0.79200000000000004,"5min":0.93300000000000005,"15min":6.1399999999999997},"last":0.34899999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.65800000000000003,"5min":0.55200000000000005,"15min":0.48499999999999999},"max":{"1min":0.98099999999999998,"5min":1.131,"15min":1.3089999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80300000000000005,"15min":0.83999999999999997},"min":{"1min":0.59499999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.198,"5min":1.4059999999999999,"15min":3.9510000000000001},"last":0.75800000000000001}]},{"osd":5,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81899999999999995,"15min":0.85099999999999998},"min":{"1min":0.69799999999999995,"5min":0.54200000000000004,"15min":0.46500000000000002},"max":{"1min":1.181,"5min":1.3300000000000001,"15min":3.9830000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.82999999999999996},"min":{"1min":0.59799999999999998,"5min":0.50800000000000001,"15min":0.38400000000000001},"max":{"1min":1.0489999999999999,"5min":1.252,"15min":4.6550000000000002},"last":0.85199999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78700000000000003,"15min":0.80800000000000005},"min":{"1min":0.57299999999999995,"5min":0.497,"15min":0.45600000000000002},"max":{"1min":0.98399999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81799999999999995,"15min":0.85099999999999998},"min":{"1min":0.64800000000000002,"5min":0.54500000000000004,"15min":0.499},"max":{"1min":1.133,"5min":1.2270000000000001,"15min":3.8900000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.84299999999999997},"min":{"1min":0.61099999999999999,"5min":0.51100000000000001,"15min":0.49299999999999999},"max":{"1min":1.0249999999999999,"5min":1.1319999999999999,"15min":4.6970000000000001},"last":0.745},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.81899999999999995,"15min":0.83199999999999996},"min":{"1min":0.63700000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.145,"5min":1.29,"15min":1.29},"last":0.86899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575028,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40168,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331672,"statfs":{"total":96636764160,"available":96595632128,"internally_reserved":0,"allocated":950272,"data_stored":440861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.433,"15min":0.439},"min":{"1min":0.32800000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.625,"5min":0.78900000000000003,"15min":0.89600000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.502,"15min":0.48899999999999999},"min":{"1min":0.39300000000000002,"5min":0.28499999999999998,"15min":0.26700000000000002},"max":{"1min":0.748,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.46899999999999997},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.876,"15min":0.876},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.28100000000000003},"max":{"1min":0.65900000000000003,"5min":0.88600000000000001,"15min":1.01},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.49299999999999999,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.49299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91600000000000004,"5min":0.95599999999999996,"15min":1.0900000000000001},"last":0.69299999999999995}]},{"osd":4,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78000000000000003,"15min":0.78700000000000003},"min":{"1min":0.69199999999999995,"5min":0.46500000000000002,"15min":0.45200000000000001},"max":{"1min":1.01,"5min":1.145,"15min":1.1799999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76600000000000001,"15min":0.77900000000000003},"min":{"1min":0.61399999999999999,"5min":0.44500000000000001,"15min":0.41399999999999998},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.2010000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.625,"5min":0.499,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":1.1220000000000001,"15min":1.327},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78700000000000003,"15min":0.80500000000000005},"min":{"1min":0.70999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0449999999999999,"15min":1.381},"last":0.88300000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.59399999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.995,"5min":1.151,"15min":1.2809999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78100000000000003,"15min":0.79800000000000004},"min":{"1min":0.65500000000000003,"5min":0.51000000000000001,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":1.0609999999999999,"15min":1.153},"last":0.82299999999999995}]},{"osd":7,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.81299999999999994,"15min":0.82499999999999996},"min":{"1min":0.72399999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.038,"5min":1.107,"15min":1.3520000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.73099999999999998,"5min":0.61699999999999999,"15min":0.504},"max":{"1min":1.0169999999999999,"5min":1.1220000000000001,"15min":1.401},"last":0.86399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575030,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50032,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321808,"statfs":{"total":96636764160,"available":96585531392,"internally_reserved":0,"allocated":958464,"data_stored":447195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28599999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48999999999999999},"min":{"1min":0.35899999999999999,"5min":0.29699999999999999,"15min":0.28599999999999998},"max":{"1min":0.73999999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.36099999999999999,"5min":0.34699999999999998,"15min":0.32800000000000001},"max":{"1min":0.627,"5min":1.091,"15min":1.091},"last":0.622},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.36399999999999999,"15min":0.33900000000000002},"max":{"1min":0.69899999999999995,"5min":0.85899999999999999,"15min":0.91300000000000003},"last":0.42999999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.40100000000000002,"15min":0.36599999999999999},"max":{"1min":0.77100000000000002,"5min":0.96499999999999997,"15min":1.016},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.40300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82399999999999995,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.53600000000000003}]},{"osd":4,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.63500000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93100000000000005,"5min":1.27,"15min":1.4419999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.63,"5min":0.434,"15min":0.434},"max":{"1min":0.98399999999999999,"5min":1.1160000000000001,"15min":1.3380000000000001},"last":0.76800000000000002}]},{"osd":5,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.5},"max":{"1min":1.0229999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76700000000000002,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":1.2450000000000001,"15min":1.3009999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.68400000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0449999999999999,"5min":1.194,"15min":1.415},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.58799999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1,"5min":1.252,"15min":1.268},"last":0.81399999999999995}]},{"osd":7,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.73699999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0580000000000001,"5min":1.306,"15min":1.399},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84199999999999997},"min":{"1min":0.66800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0349999999999999,"5min":1.3440000000000001,"15min":1.373},"last":0.79500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575029,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1564672,"data_stored":1039619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46899999999999997,"15min":0.44600000000000001},"min":{"1min":0.32800000000000001,"5min":0.28499999999999998,"15min":0.218},"max":{"1min":0.64200000000000002,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.254},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60999999999999999,"15min":0.53600000000000003},"min":{"1min":0.34999999999999998,"5min":0.313,"15min":0.26900000000000002},"max":{"1min":0.70999999999999996,"5min":7.8860000000000001,"15min":7.8860000000000001},"last":0.27600000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55200000000000005,"15min":0.52500000000000002},"min":{"1min":0.377,"5min":0.35799999999999998,"15min":0.28399999999999997},"max":{"1min":0.76300000000000001,"5min":2.528,"15min":2.528},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.38600000000000001,"15min":0.34000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":1.2989999999999999},"last":0.318}]},{"osd":3,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55700000000000005,"15min":0.52200000000000002},"min":{"1min":0.43099999999999999,"5min":0.35199999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":3.1000000000000001,"15min":3.1000000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.69599999999999995,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.38400000000000001,"15min":0.38200000000000001},"max":{"1min":0.749,"5min":10.49,"15min":10.49},"last":0.36599999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81899999999999995,"15min":0.80900000000000005},"min":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.45400000000000001},"max":{"1min":1.1499999999999999,"5min":3.0659999999999998,"15min":3.7469999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.85599999999999998,"15min":0.80800000000000005},"min":{"1min":0.61499999999999999,"5min":0.499,"15min":0.499},"max":{"1min":1.0129999999999999,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82199999999999995,"15min":0.80500000000000005},"min":{"1min":0.65800000000000003,"5min":0.54800000000000004,"15min":0.44700000000000001},"max":{"1min":1.0620000000000001,"5min":1.3109999999999999,"15min":1.877},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.82699999999999996,"15min":0.80400000000000005},"min":{"1min":0.64400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.1799999999999999,"5min":1.2010000000000001,"15min":1.7450000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.87,"15min":0.82599999999999996},"min":{"1min":0.64900000000000002,"5min":0.54700000000000004,"15min":0.47299999999999998},"max":{"1min":1.071,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.82199999999999995,"15min":0.81799999999999995},"min":{"1min":0.57099999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.306,"15min":1.492},"last":0.73499999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.66100000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.6539999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.83099999999999996,"15min":0.82099999999999995},"min":{"1min":0.71499999999999997,"5min":0.52900000000000003,"15min":0.48799999999999999},"max":{"1min":1.097,"5min":1.276,"15min":2.113},"last":0.66700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:43:26.093 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:43:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.270+0000 7fbd84e41640 1 Processor -- start 2023-12-20T04:43:26.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.270+0000 7fbd84e41640 1 -- start start 2023-12-20T04:43:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd84e41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd800ae010 0x7fbd80071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd84e41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd80164ef0 0x7fbd800719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:26.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd84e41640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd800ae3f0 con 0x7fbd8006cf10 2023-12-20T04:43:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd84e41640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd800ca4b0 con 0x7fbd80164ef0 2023-12-20T04:43:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd84e41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd80161800 con 0x7fbd800ae010 2023-12-20T04:43:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd7ed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd80164ef0 0x7fbd800719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd7ed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd80164ef0 0x7fbd800719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43538/0 (socket says 172.21.15.32:43538) 2023-12-20T04:43:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd7ed9d640 1 -- 172.21.15.32:0/1185423153 learned_addr learned my addr 172.21.15.32:0/1185423153 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd7e59c640 1 --2- 172.21.15.32:0/1185423153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd800ae010 0x7fbd80071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd7ed9d640 1 -- 172.21.15.32:0/1185423153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd800ae010 msgr2=0x7fbd80071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd7ed9d640 1 --2- 172.21.15.32:0/1185423153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd800ae010 0x7fbd80071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd7ed9d640 1 -- 172.21.15.32:0/1185423153 >> v1:172.21.15.39:6789/0 conn(0x7fbd8006cf10 legacy=0x7fbd800701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd7ed9d640 1 -- 172.21.15.32:0/1185423153 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd80165380 con 0x7fbd80164ef0 2023-12-20T04:43:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.274+0000 7fbd7ed9d640 1 --2- 172.21.15.32:0/1185423153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd80164ef0 0x7fbd800719b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009e50 tx=0x7fbd7801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fd9ff2a6297362b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:26.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd7d59a640 1 -- 172.21.15.32:0/1185423153 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd780a5810 con 0x7fbd80164ef0 2023-12-20T04:43:26.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd7d59a640 1 -- 172.21.15.32:0/1185423153 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd780a5970 con 0x7fbd80164ef0 2023-12-20T04:43:26.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd7d59a640 1 -- 172.21.15.32:0/1185423153 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd780a5d00 con 0x7fbd80164ef0 2023-12-20T04:43:26.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 -- 172.21.15.32:0/1185423153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd80164ef0 msgr2=0x7fbd800719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 --2- 172.21.15.32:0/1185423153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd80164ef0 0x7fbd800719b0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7fbd78009e50 tx=0x7fbd7801c950 comp rx=0 tx=0).stop 2023-12-20T04:43:26.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 -- 172.21.15.32:0/1185423153 shutdown_connections 2023-12-20T04:43:26.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 --2- 172.21.15.32:0/1185423153 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd80164ef0 0x7fbd800719b0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 --2- 172.21.15.32:0/1185423153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd800ae010 0x7fbd80071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 -- 172.21.15.32:0/1185423153 >> 172.21.15.32:0/1185423153 conn(0x7fbd8006a320 msgr2=0x7fbd800ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:26.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 -- 172.21.15.32:0/1185423153 shutdown_connections 2023-12-20T04:43:26.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 -- 172.21.15.32:0/1185423153 wait complete. 2023-12-20T04:43:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 Processor -- start 2023-12-20T04:43:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 -- start start 2023-12-20T04:43:26.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd800ae010 0x7fbd80159c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:26.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd80150a80 0x7fbd801604c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:26.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd8014ee80 0x7fbd8014f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:26.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8014f330 con 0x7fbd800ae010 2023-12-20T04:43:26.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd80159960 con 0x7fbd80150a80 2023-12-20T04:43:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd84e41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd8015a270 con 0x7fbd8014ee80 2023-12-20T04:43:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd7e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd800ae010 0x7fbd80159c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd7e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd800ae010 0x7fbd80159c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43540/0 (socket says 172.21.15.32:43540) 2023-12-20T04:43:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd7e59c640 1 -- 172.21.15.32:0/3563897875 learned_addr learned my addr 172.21.15.32:0/3563897875 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.278+0000 7fbd7dd9b640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd80150a80 0x7fbd801604c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd7ed9d640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd8014ee80 0x7fbd8014f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd7e59c640 1 -- 172.21.15.32:0/3563897875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd8014ee80 msgr2=0x7fbd8014f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd7e59c640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd8014ee80 0x7fbd8014f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd7e59c640 1 -- 172.21.15.32:0/3563897875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd80150a80 msgr2=0x7fbd801604c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd7e59c640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd80150a80 0x7fbd801604c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd7e59c640 1 -- 172.21.15.32:0/3563897875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd78009b30 con 0x7fbd800ae010 2023-12-20T04:43:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd7e59c640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd800ae010 0x7fbd80159c90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000d800 tx=0x7fbd7000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd6effd640 1 -- 172.21.15.32:0/3563897875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70018070 con 0x7fbd800ae010 2023-12-20T04:43:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd6effd640 1 -- 172.21.15.32:0/3563897875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd700059a0 con 0x7fbd800ae010 2023-12-20T04:43:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd84e41640 1 -- 172.21.15.32:0/3563897875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd801593a0 con 0x7fbd800ae010 2023-12-20T04:43:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd6effd640 1 -- 172.21.15.32:0/3563897875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd70005b50 con 0x7fbd800ae010 2023-12-20T04:43:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd84e41640 1 -- 172.21.15.32:0/3563897875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd8015ffa0 con 0x7fbd800ae010 2023-12-20T04:43:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd84e41640 1 -- 172.21.15.32:0/3563897875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd8004e000 con 0x7fbd800ae010 2023-12-20T04:43:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd6effd640 1 -- 172.21.15.32:0/3563897875 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbd70005cb0 con 0x7fbd800ae010 2023-12-20T04:43:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.282+0000 7fbd6effd640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd60065b80 0x7fbd60068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.286+0000 7fbd6effd640 1 -- 172.21.15.32:0/3563897875 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbd70083120 con 0x7fbd800ae010 2023-12-20T04:43:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.286+0000 7fbd7dd9b640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd60065b80 0x7fbd60068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.286+0000 7fbd7dd9b640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd60065b80 0x7fbd60068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd80159290 tx=0x7fbd74009040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.290+0000 7fbd6effd640 1 -- 172.21.15.32:0/3563897875 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbd700509e0 con 0x7fbd800ae010 2023-12-20T04:43:26.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.442+0000 7fbd84e41640 1 -- 172.21.15.32:0/3563897875 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbd80150e60 con 0x7fbd60065b80 2023-12-20T04:43:26.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.450+0000 7fbd6effd640 1 -- 172.21.15.32:0/3563897875 <== mgr.6183 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288539 (secure 0 0 0) 0x7fbd80150e60 con 0x7fbd60065b80 2023-12-20T04:43:26.452 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:26.454 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:43:26.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 -- 172.21.15.32:0/3563897875 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd60065b80 msgr2=0x7fbd60068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd60065b80 0x7fbd60068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbd80159290 tx=0x7fbd74009040 comp rx=0 tx=0).stop 2023-12-20T04:43:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 -- 172.21.15.32:0/3563897875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd800ae010 msgr2=0x7fbd80159c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd800ae010 0x7fbd80159c90 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7fbd7000d800 tx=0x7fbd7000dcc0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 -- 172.21.15.32:0/3563897875 shutdown_connections 2023-12-20T04:43:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd60065b80 0x7fbd60068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd8014ee80 0x7fbd8014f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd80150a80 0x7fbd801604c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 --2- 172.21.15.32:0/3563897875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd800ae010 0x7fbd80159c90 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 -- 172.21.15.32:0/3563897875 >> 172.21.15.32:0/3563897875 conn(0x7fbd8006a320 msgr2=0x7fbd8015aec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 -- 172.21.15.32:0/3563897875 shutdown_connections 2023-12-20T04:43:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.454+0000 7fbd84e41640 1 -- 172.21.15.32:0/3563897875 wait complete. 2023-12-20T04:43:26.474 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-20T04:43:26.257831+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":283,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":199,"ondisk_log_size":199,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":135,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":325904,"kb_used_data":8656,"kb_used_omap":0,"kb_used_meta":317184,"kb_avail":754648816,"statfs":{"total":773094113280,"available":772760387584,"internally_reserved":0,"allocated":8863744,"data_stored":4749700,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":324796416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.699643"},"pg_stats":[{"pgid":"9.3f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553392+0000","last_change":"2023-12-20T04:43:22.004072+0000","last_active":"2023-12-20T04:43:23.553392+0000","last_peered":"2023-12-20T04:43:23.553392+0000","last_clean":"2023-12-20T04:43:23.553392+0000","last_became_active":"2023-12-20T04:43:22.003778+0000","last_became_peered":"2023-12-20T04:43:22.003778+0000","last_unstale":"2023-12-20T04:43:23.553392+0000","last_undegraded":"2023-12-20T04:43:23.553392+0000","last_fullsized":"2023-12-20T04:43:23.553392+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:02.386672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924910+0000","last_change":"2023-12-20T04:43:21.999126+0000","last_active":"2023-12-20T04:43:23.924910+0000","last_peered":"2023-12-20T04:43:23.924910+0000","last_clean":"2023-12-20T04:43:23.924910+0000","last_became_active":"2023-12-20T04:43:21.998841+0000","last_became_peered":"2023-12-20T04:43:21.998841+0000","last_unstale":"2023-12-20T04:43:23.924910+0000","last_undegraded":"2023-12-20T04:43:23.924910+0000","last_fullsized":"2023-12-20T04:43:23.924910+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:50:16.003698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.3d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811444+0000","last_change":"2023-12-20T04:43:22.007986+0000","last_active":"2023-12-20T04:43:23.811444+0000","last_peered":"2023-12-20T04:43:23.811444+0000","last_clean":"2023-12-20T04:43:23.811444+0000","last_became_active":"2023-12-20T04:43:22.007647+0000","last_became_peered":"2023-12-20T04:43:22.007647+0000","last_unstale":"2023-12-20T04:43:23.811444+0000","last_undegraded":"2023-12-20T04:43:23.811444+0000","last_fullsized":"2023-12-20T04:43:23.811444+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:31:05.266757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.3c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983756+0000","last_change":"2023-12-20T04:43:21.998263+0000","last_active":"2023-12-20T04:43:22.983756+0000","last_peered":"2023-12-20T04:43:22.983756+0000","last_clean":"2023-12-20T04:43:22.983756+0000","last_became_active":"2023-12-20T04:43:21.997701+0000","last_became_peered":"2023-12-20T04:43:21.997701+0000","last_unstale":"2023-12-20T04:43:22.983756+0000","last_undegraded":"2023-12-20T04:43:22.983756+0000","last_fullsized":"2023-12-20T04:43:22.983756+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:40:16.881555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.3b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553212+0000","last_change":"2023-12-20T04:43:21.990156+0000","last_active":"2023-12-20T04:43:23.553212+0000","last_peered":"2023-12-20T04:43:23.553212+0000","last_clean":"2023-12-20T04:43:23.553212+0000","last_became_active":"2023-12-20T04:43:21.989878+0000","last_became_peered":"2023-12-20T04:43:21.989878+0000","last_unstale":"2023-12-20T04:43:23.553212+0000","last_undegraded":"2023-12-20T04:43:23.553212+0000","last_fullsized":"2023-12-20T04:43:23.553212+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:28:27.070336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.3a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924333+0000","last_change":"2023-12-20T04:43:22.000099+0000","last_active":"2023-12-20T04:43:23.924333+0000","last_peered":"2023-12-20T04:43:23.924333+0000","last_clean":"2023-12-20T04:43:23.924333+0000","last_became_active":"2023-12-20T04:43:21.999807+0000","last_became_peered":"2023-12-20T04:43:21.999807+0000","last_unstale":"2023-12-20T04:43:23.924333+0000","last_undegraded":"2023-12-20T04:43:23.924333+0000","last_fullsized":"2023-12-20T04:43:23.924333+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:45:38.258467+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.39","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983167+0000","last_change":"2023-12-20T04:43:21.990794+0000","last_active":"2023-12-20T04:43:22.983167+0000","last_peered":"2023-12-20T04:43:22.983167+0000","last_clean":"2023-12-20T04:43:22.983167+0000","last_became_active":"2023-12-20T04:43:21.990498+0000","last_became_peered":"2023-12-20T04:43:21.990498+0000","last_unstale":"2023-12-20T04:43:22.983167+0000","last_undegraded":"2023-12-20T04:43:22.983167+0000","last_fullsized":"2023-12-20T04:43:22.983167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:25.802282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.38","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924041+0000","last_change":"2023-12-20T04:43:21.995443+0000","last_active":"2023-12-20T04:43:23.924041+0000","last_peered":"2023-12-20T04:43:23.924041+0000","last_clean":"2023-12-20T04:43:23.924041+0000","last_became_active":"2023-12-20T04:43:21.995137+0000","last_became_peered":"2023-12-20T04:43:21.995137+0000","last_unstale":"2023-12-20T04:43:23.924041+0000","last_undegraded":"2023-12-20T04:43:23.924041+0000","last_fullsized":"2023-12-20T04:43:23.924041+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:14.983424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.37","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.984672+0000","last_change":"2023-12-20T04:43:22.000368+0000","last_active":"2023-12-20T04:43:22.984672+0000","last_peered":"2023-12-20T04:43:22.984672+0000","last_clean":"2023-12-20T04:43:22.984672+0000","last_became_active":"2023-12-20T04:43:22.000022+0000","last_became_peered":"2023-12-20T04:43:22.000022+0000","last_unstale":"2023-12-20T04:43:22.984672+0000","last_undegraded":"2023-12-20T04:43:22.984672+0000","last_fullsized":"2023-12-20T04:43:22.984672+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:33.942256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.36","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983122+0000","last_change":"2023-12-20T04:43:22.002105+0000","last_active":"2023-12-20T04:43:22.983122+0000","last_peered":"2023-12-20T04:43:22.983122+0000","last_clean":"2023-12-20T04:43:22.983122+0000","last_became_active":"2023-12-20T04:43:22.001825+0000","last_became_peered":"2023-12-20T04:43:22.001825+0000","last_unstale":"2023-12-20T04:43:22.983122+0000","last_undegraded":"2023-12-20T04:43:22.983122+0000","last_fullsized":"2023-12-20T04:43:22.983122+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:57:47.673127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.35","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982987+0000","last_change":"2023-12-20T04:43:22.000044+0000","last_active":"2023-12-20T04:43:22.982987+0000","last_peered":"2023-12-20T04:43:22.982987+0000","last_clean":"2023-12-20T04:43:22.982987+0000","last_became_active":"2023-12-20T04:43:21.999745+0000","last_became_peered":"2023-12-20T04:43:21.999745+0000","last_unstale":"2023-12-20T04:43:22.982987+0000","last_undegraded":"2023-12-20T04:43:22.982987+0000","last_fullsized":"2023-12-20T04:43:22.982987+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:23.027733+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.34","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983886+0000","last_change":"2023-12-20T04:43:22.004478+0000","last_active":"2023-12-20T04:43:22.983886+0000","last_peered":"2023-12-20T04:43:22.983886+0000","last_clean":"2023-12-20T04:43:22.983886+0000","last_became_active":"2023-12-20T04:43:22.004200+0000","last_became_peered":"2023-12-20T04:43:22.004200+0000","last_unstale":"2023-12-20T04:43:22.983886+0000","last_undegraded":"2023-12-20T04:43:22.983886+0000","last_fullsized":"2023-12-20T04:43:22.983886+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:48:12.330958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.33","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.923616+0000","last_change":"2023-12-20T04:43:22.012829+0000","last_active":"2023-12-20T04:43:23.923616+0000","last_peered":"2023-12-20T04:43:23.923616+0000","last_clean":"2023-12-20T04:43:23.923616+0000","last_became_active":"2023-12-20T04:43:22.012391+0000","last_became_peered":"2023-12-20T04:43:22.012391+0000","last_unstale":"2023-12-20T04:43:23.923616+0000","last_undegraded":"2023-12-20T04:43:23.923616+0000","last_fullsized":"2023-12-20T04:43:23.923616+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:21:39.160322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.32","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.984227+0000","last_change":"2023-12-20T04:43:21.995725+0000","last_active":"2023-12-20T04:43:22.984227+0000","last_peered":"2023-12-20T04:43:22.984227+0000","last_clean":"2023-12-20T04:43:22.984227+0000","last_became_active":"2023-12-20T04:43:21.995441+0000","last_became_peered":"2023-12-20T04:43:21.995441+0000","last_unstale":"2023-12-20T04:43:22.984227+0000","last_undegraded":"2023-12-20T04:43:22.984227+0000","last_fullsized":"2023-12-20T04:43:22.984227+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:17:47.725143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.31","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983555+0000","last_change":"2023-12-20T04:43:21.994127+0000","last_active":"2023-12-20T04:43:22.983555+0000","last_peered":"2023-12-20T04:43:22.983555+0000","last_clean":"2023-12-20T04:43:22.983555+0000","last_became_active":"2023-12-20T04:43:21.993792+0000","last_became_peered":"2023-12-20T04:43:21.993792+0000","last_unstale":"2023-12-20T04:43:22.983555+0000","last_undegraded":"2023-12-20T04:43:22.983555+0000","last_fullsized":"2023-12-20T04:43:22.983555+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:11:19.997837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.30","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924223+0000","last_change":"2023-12-20T04:43:22.001435+0000","last_active":"2023-12-20T04:43:23.924223+0000","last_peered":"2023-12-20T04:43:23.924223+0000","last_clean":"2023-12-20T04:43:23.924223+0000","last_became_active":"2023-12-20T04:43:22.001172+0000","last_became_peered":"2023-12-20T04:43:22.001172+0000","last_unstale":"2023-12-20T04:43:23.924223+0000","last_undegraded":"2023-12-20T04:43:23.924223+0000","last_fullsized":"2023-12-20T04:43:23.924223+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:45:44.114176+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.2f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924967+0000","last_change":"2023-12-20T04:43:22.003640+0000","last_active":"2023-12-20T04:43:23.924967+0000","last_peered":"2023-12-20T04:43:23.924967+0000","last_clean":"2023-12-20T04:43:23.924967+0000","last_became_active":"2023-12-20T04:43:22.003389+0000","last_became_peered":"2023-12-20T04:43:22.003389+0000","last_unstale":"2023-12-20T04:43:23.924967+0000","last_undegraded":"2023-12-20T04:43:23.924967+0000","last_fullsized":"2023-12-20T04:43:23.924967+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:59:16.761895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811640+0000","last_change":"2023-12-20T04:43:22.012521+0000","last_active":"2023-12-20T04:43:23.811640+0000","last_peered":"2023-12-20T04:43:23.811640+0000","last_clean":"2023-12-20T04:43:23.811640+0000","last_became_active":"2023-12-20T04:43:22.012168+0000","last_became_peered":"2023-12-20T04:43:22.012168+0000","last_unstale":"2023-12-20T04:43:23.811640+0000","last_undegraded":"2023-12-20T04:43:23.811640+0000","last_fullsized":"2023-12-20T04:43:23.811640+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:17:47.725143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553256+0000","last_change":"2023-12-20T04:43:21.996601+0000","last_active":"2023-12-20T04:43:23.553256+0000","last_peered":"2023-12-20T04:43:23.553256+0000","last_clean":"2023-12-20T04:43:23.553256+0000","last_became_active":"2023-12-20T04:43:21.993330+0000","last_became_peered":"2023-12-20T04:43:21.993330+0000","last_unstale":"2023-12-20T04:43:23.553256+0000","last_undegraded":"2023-12-20T04:43:23.553256+0000","last_fullsized":"2023-12-20T04:43:23.553256+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:24:26.876505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.2c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924699+0000","last_change":"2023-12-20T04:43:22.005461+0000","last_active":"2023-12-20T04:43:23.924699+0000","last_peered":"2023-12-20T04:43:23.924699+0000","last_clean":"2023-12-20T04:43:23.924699+0000","last_became_active":"2023-12-20T04:43:22.005188+0000","last_became_peered":"2023-12-20T04:43:22.005188+0000","last_unstale":"2023-12-20T04:43:23.924699+0000","last_undegraded":"2023-12-20T04:43:23.924699+0000","last_fullsized":"2023-12-20T04:43:23.924699+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:10:36.287963+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.2b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811144+0000","last_change":"2023-12-20T04:43:22.010594+0000","last_active":"2023-12-20T04:43:23.811144+0000","last_peered":"2023-12-20T04:43:23.811144+0000","last_clean":"2023-12-20T04:43:23.811144+0000","last_became_active":"2023-12-20T04:43:22.010251+0000","last_became_peered":"2023-12-20T04:43:22.010251+0000","last_unstale":"2023-12-20T04:43:23.811144+0000","last_undegraded":"2023-12-20T04:43:23.811144+0000","last_fullsized":"2023-12-20T04:43:23.811144+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:29:15.058280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.2a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.923924+0000","last_change":"2023-12-20T04:43:22.003002+0000","last_active":"2023-12-20T04:43:23.923924+0000","last_peered":"2023-12-20T04:43:23.923924+0000","last_clean":"2023-12-20T04:43:23.923924+0000","last_became_active":"2023-12-20T04:43:22.002687+0000","last_became_peered":"2023-12-20T04:43:22.002687+0000","last_unstale":"2023-12-20T04:43:23.923924+0000","last_undegraded":"2023-12-20T04:43:23.923924+0000","last_fullsized":"2023-12-20T04:43:23.923924+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:22.502676+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.29","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811171+0000","last_change":"2023-12-20T04:43:22.000763+0000","last_active":"2023-12-20T04:43:23.811171+0000","last_peered":"2023-12-20T04:43:23.811171+0000","last_clean":"2023-12-20T04:43:23.811171+0000","last_became_active":"2023-12-20T04:43:22.000442+0000","last_became_peered":"2023-12-20T04:43:22.000442+0000","last_unstale":"2023-12-20T04:43:23.811171+0000","last_undegraded":"2023-12-20T04:43:23.811171+0000","last_fullsized":"2023-12-20T04:43:23.811171+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:14:57.167825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.28","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.923554+0000","last_change":"2023-12-20T04:43:22.001998+0000","last_active":"2023-12-20T04:43:23.923554+0000","last_peered":"2023-12-20T04:43:23.923554+0000","last_clean":"2023-12-20T04:43:23.923554+0000","last_became_active":"2023-12-20T04:43:22.001645+0000","last_became_peered":"2023-12-20T04:43:22.001645+0000","last_unstale":"2023-12-20T04:43:23.923554+0000","last_undegraded":"2023-12-20T04:43:23.923554+0000","last_fullsized":"2023-12-20T04:43:23.923554+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:02:12.971111+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.27","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553458+0000","last_change":"2023-12-20T04:43:21.996101+0000","last_active":"2023-12-20T04:43:23.553458+0000","last_peered":"2023-12-20T04:43:23.553458+0000","last_clean":"2023-12-20T04:43:23.553458+0000","last_became_active":"2023-12-20T04:43:21.995817+0000","last_became_peered":"2023-12-20T04:43:21.995817+0000","last_unstale":"2023-12-20T04:43:23.553458+0000","last_undegraded":"2023-12-20T04:43:23.553458+0000","last_fullsized":"2023-12-20T04:43:23.553458+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:59:23.231757+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.26","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983950+0000","last_change":"2023-12-20T04:43:21.989135+0000","last_active":"2023-12-20T04:43:22.983950+0000","last_peered":"2023-12-20T04:43:22.983950+0000","last_clean":"2023-12-20T04:43:22.983950+0000","last_became_active":"2023-12-20T04:43:21.988789+0000","last_became_peered":"2023-12-20T04:43:21.988789+0000","last_unstale":"2023-12-20T04:43:22.983950+0000","last_undegraded":"2023-12-20T04:43:22.983950+0000","last_fullsized":"2023-12-20T04:43:22.983950+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:14:57.167825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.25","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924759+0000","last_change":"2023-12-20T04:43:22.008701+0000","last_active":"2023-12-20T04:43:23.924759+0000","last_peered":"2023-12-20T04:43:23.924759+0000","last_clean":"2023-12-20T04:43:23.924759+0000","last_became_active":"2023-12-20T04:43:22.008420+0000","last_became_peered":"2023-12-20T04:43:22.008420+0000","last_unstale":"2023-12-20T04:43:23.924759+0000","last_undegraded":"2023-12-20T04:43:23.924759+0000","last_fullsized":"2023-12-20T04:43:23.924759+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:12:28.334990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.24","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553213+0000","last_change":"2023-12-20T04:43:21.994859+0000","last_active":"2023-12-20T04:43:23.553213+0000","last_peered":"2023-12-20T04:43:23.553213+0000","last_clean":"2023-12-20T04:43:23.553213+0000","last_became_active":"2023-12-20T04:43:21.990501+0000","last_became_peered":"2023-12-20T04:43:21.990501+0000","last_unstale":"2023-12-20T04:43:23.553213+0000","last_undegraded":"2023-12-20T04:43:23.553213+0000","last_fullsized":"2023-12-20T04:43:23.553213+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:18:47.468708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.23","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924542+0000","last_change":"2023-12-20T04:43:21.990922+0000","last_active":"2023-12-20T04:43:23.924542+0000","last_peered":"2023-12-20T04:43:23.924542+0000","last_clean":"2023-12-20T04:43:23.924542+0000","last_became_active":"2023-12-20T04:43:21.990665+0000","last_became_peered":"2023-12-20T04:43:21.990665+0000","last_unstale":"2023-12-20T04:43:23.924542+0000","last_undegraded":"2023-12-20T04:43:23.924542+0000","last_fullsized":"2023-12-20T04:43:23.924542+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:42:37.847365+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.22","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811097+0000","last_change":"2023-12-20T04:43:22.008543+0000","last_active":"2023-12-20T04:43:23.811097+0000","last_peered":"2023-12-20T04:43:23.811097+0000","last_clean":"2023-12-20T04:43:23.811097+0000","last_became_active":"2023-12-20T04:43:22.008228+0000","last_became_peered":"2023-12-20T04:43:22.008228+0000","last_unstale":"2023-12-20T04:43:23.811097+0000","last_undegraded":"2023-12-20T04:43:23.811097+0000","last_fullsized":"2023-12-20T04:43:23.811097+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:11.618078+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.21","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.553099+0000","last_change":"2023-12-20T04:43:21.996074+0000","last_active":"2023-12-20T04:43:23.553099+0000","last_peered":"2023-12-20T04:43:23.553099+0000","last_clean":"2023-12-20T04:43:23.553099+0000","last_became_active":"2023-12-20T04:43:21.995790+0000","last_became_peered":"2023-12-20T04:43:21.995790+0000","last_unstale":"2023-12-20T04:43:23.553099+0000","last_undegraded":"2023-12-20T04:43:23.553099+0000","last_fullsized":"2023-12-20T04:43:23.553099+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:45:36.939730+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"9.20","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924534+0000","last_change":"2023-12-20T04:43:22.010168+0000","last_active":"2023-12-20T04:43:23.924534+0000","last_peered":"2023-12-20T04:43:23.924534+0000","last_clean":"2023-12-20T04:43:23.924534+0000","last_became_active":"2023-12-20T04:43:22.009235+0000","last_became_peered":"2023-12-20T04:43:22.009235+0000","last_unstale":"2023-12-20T04:43:23.924534+0000","last_undegraded":"2023-12-20T04:43:23.924534+0000","last_fullsized":"2023-12-20T04:43:23.924534+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:18.578713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924240+0000","last_change":"2023-12-20T04:43:22.010640+0000","last_active":"2023-12-20T04:43:23.924240+0000","last_peered":"2023-12-20T04:43:23.924240+0000","last_clean":"2023-12-20T04:43:23.924240+0000","last_became_active":"2023-12-20T04:43:22.010344+0000","last_became_peered":"2023-12-20T04:43:22.010344+0000","last_unstale":"2023-12-20T04:43:23.924240+0000","last_undegraded":"2023-12-20T04:43:23.924240+0000","last_fullsized":"2023-12-20T04:43:23.924240+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:28:50.727344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1f","version":"78'4","reported_seq":15,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.925285+0000","last_change":"2023-12-20T04:43:21.587084+0000","last_active":"2023-12-20T04:43:23.925285+0000","last_peered":"2023-12-20T04:43:23.925285+0000","last_clean":"2023-12-20T04:43:23.925285+0000","last_became_active":"2023-12-20T04:43:21.586819+0000","last_became_peered":"2023-12-20T04:43:21.586819+0000","last_unstale":"2023-12-20T04:43:23.925285+0000","last_undegraded":"2023-12-20T04:43:23.925285+0000","last_fullsized":"2023-12-20T04:43:23.925285+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:48:11.323426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983139+0000","last_change":"2023-12-20T04:43:22.007020+0000","last_active":"2023-12-20T04:43:22.983139+0000","last_peered":"2023-12-20T04:43:22.983139+0000","last_clean":"2023-12-20T04:43:22.983139+0000","last_became_active":"2023-12-20T04:43:22.004884+0000","last_became_peered":"2023-12-20T04:43:22.004884+0000","last_unstale":"2023-12-20T04:43:22.983139+0000","last_undegraded":"2023-12-20T04:43:22.983139+0000","last_fullsized":"2023-12-20T04:43:22.983139+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:11:51.597335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1e","version":"78'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.925154+0000","last_change":"2023-12-20T04:43:21.584516+0000","last_active":"2023-12-20T04:43:23.925154+0000","last_peered":"2023-12-20T04:43:23.925154+0000","last_clean":"2023-12-20T04:43:23.925154+0000","last_became_active":"2023-12-20T04:43:21.584196+0000","last_became_peered":"2023-12-20T04:43:21.584196+0000","last_unstale":"2023-12-20T04:43:23.925154+0000","last_undegraded":"2023-12-20T04:43:23.925154+0000","last_fullsized":"2023-12-20T04:43:23.925154+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:24.794750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.1c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924399+0000","last_change":"2023-12-20T04:43:21.999085+0000","last_active":"2023-12-20T04:43:23.924399+0000","last_peered":"2023-12-20T04:43:23.924399+0000","last_clean":"2023-12-20T04:43:23.924399+0000","last_became_active":"2023-12-20T04:43:21.998844+0000","last_became_peered":"2023-12-20T04:43:21.998844+0000","last_unstale":"2023-12-20T04:43:23.924399+0000","last_undegraded":"2023-12-20T04:43:23.924399+0000","last_fullsized":"2023-12-20T04:43:23.924399+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:08.735681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1d","version":"78'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924858+0000","last_change":"2023-12-20T04:43:21.586335+0000","last_active":"2023-12-20T04:43:23.924858+0000","last_peered":"2023-12-20T04:43:23.924858+0000","last_clean":"2023-12-20T04:43:23.924858+0000","last_became_active":"2023-12-20T04:43:21.586047+0000","last_became_peered":"2023-12-20T04:43:21.586047+0000","last_unstale":"2023-12-20T04:43:23.924858+0000","last_undegraded":"2023-12-20T04:43:23.924858+0000","last_fullsized":"2023-12-20T04:43:23.924858+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:14:56.160293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.1d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924758+0000","last_change":"2023-12-20T04:43:22.010407+0000","last_active":"2023-12-20T04:43:23.924758+0000","last_peered":"2023-12-20T04:43:23.924758+0000","last_clean":"2023-12-20T04:43:23.924758+0000","last_became_active":"2023-12-20T04:43:22.009339+0000","last_became_peered":"2023-12-20T04:43:22.009339+0000","last_unstale":"2023-12-20T04:43:23.924758+0000","last_undegraded":"2023-12-20T04:43:23.924758+0000","last_fullsized":"2023-12-20T04:43:23.924758+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:56:34.953824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.1c","version":"78'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.984038+0000","last_change":"2023-12-20T04:43:21.583452+0000","last_active":"2023-12-20T04:43:22.984038+0000","last_peered":"2023-12-20T04:43:22.984038+0000","last_clean":"2023-12-20T04:43:22.984038+0000","last_became_active":"2023-12-20T04:43:21.582543+0000","last_became_peered":"2023-12-20T04:43:21.582543+0000","last_unstale":"2023-12-20T04:43:22.984038+0000","last_undegraded":"2023-12-20T04:43:22.984038+0000","last_fullsized":"2023-12-20T04:43:22.984038+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:26.929937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.1a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924380+0000","last_change":"2023-12-20T04:43:21.999575+0000","last_active":"2023-12-20T04:43:23.924380+0000","last_peered":"2023-12-20T04:43:23.924380+0000","last_clean":"2023-12-20T04:43:23.924380+0000","last_became_active":"2023-12-20T04:43:21.999330+0000","last_became_peered":"2023-12-20T04:43:21.999330+0000","last_unstale":"2023-12-20T04:43:23.924380+0000","last_undegraded":"2023-12-20T04:43:23.924380+0000","last_fullsized":"2023-12-20T04:43:23.924380+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:45:42.523068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1b","version":"78'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.984360+0000","last_change":"2023-12-20T04:43:21.592168+0000","last_active":"2023-12-20T04:43:22.984360+0000","last_peered":"2023-12-20T04:43:22.984360+0000","last_clean":"2023-12-20T04:43:22.984360+0000","last_became_active":"2023-12-20T04:43:21.591845+0000","last_became_peered":"2023-12-20T04:43:21.591845+0000","last_unstale":"2023-12-20T04:43:22.984360+0000","last_undegraded":"2023-12-20T04:43:22.984360+0000","last_fullsized":"2023-12-20T04:43:22.984360+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:27:00.588443+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.1b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924185+0000","last_change":"2023-12-20T04:43:22.009978+0000","last_active":"2023-12-20T04:43:23.924185+0000","last_peered":"2023-12-20T04:43:23.924185+0000","last_clean":"2023-12-20T04:43:23.924185+0000","last_became_active":"2023-12-20T04:43:22.009709+0000","last_became_peered":"2023-12-20T04:43:22.009709+0000","last_unstale":"2023-12-20T04:43:23.924185+0000","last_undegraded":"2023-12-20T04:43:23.924185+0000","last_fullsized":"2023-12-20T04:43:23.924185+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:57:20.248762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.1a","version":"78'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924466+0000","last_change":"2023-12-20T04:43:21.584942+0000","last_active":"2023-12-20T04:43:23.924466+0000","last_peered":"2023-12-20T04:43:23.924466+0000","last_clean":"2023-12-20T04:43:23.924466+0000","last_became_active":"2023-12-20T04:43:21.584665+0000","last_became_peered":"2023-12-20T04:43:21.584665+0000","last_unstale":"2023-12-20T04:43:23.924466+0000","last_undegraded":"2023-12-20T04:43:23.924466+0000","last_fullsized":"2023-12-20T04:43:23.924466+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:29:14.050749+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.18","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.923514+0000","last_change":"2023-12-20T04:43:22.006131+0000","last_active":"2023-12-20T04:43:23.923514+0000","last_peered":"2023-12-20T04:43:23.923514+0000","last_clean":"2023-12-20T04:43:23.923514+0000","last_became_active":"2023-12-20T04:43:22.005853+0000","last_became_peered":"2023-12-20T04:43:22.005853+0000","last_unstale":"2023-12-20T04:43:23.923514+0000","last_undegraded":"2023-12-20T04:43:23.923514+0000","last_fullsized":"2023-12-20T04:43:23.923514+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:28:27.070336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"8.19","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983506+0000","last_change":"2023-12-20T04:43:21.585847+0000","last_active":"2023-12-20T04:43:22.983506+0000","last_peered":"2023-12-20T04:43:22.983506+0000","last_clean":"2023-12-20T04:43:22.983506+0000","last_became_active":"2023-12-20T04:43:21.585529+0000","last_became_peered":"2023-12-20T04:43:21.585529+0000","last_unstale":"2023-12-20T04:43:22.983506+0000","last_undegraded":"2023-12-20T04:43:22.983506+0000","last_fullsized":"2023-12-20T04:43:22.983506+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:57:46.665595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.19","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924199+0000","last_change":"2023-12-20T04:43:21.994327+0000","last_active":"2023-12-20T04:43:23.924199+0000","last_peered":"2023-12-20T04:43:23.924199+0000","last_clean":"2023-12-20T04:43:23.924199+0000","last_became_active":"2023-12-20T04:43:21.993655+0000","last_became_peered":"2023-12-20T04:43:21.993655+0000","last_unstale":"2023-12-20T04:43:23.924199+0000","last_undegraded":"2023-12-20T04:43:23.924199+0000","last_fullsized":"2023-12-20T04:43:23.924199+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:08.735681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.18","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983240+0000","last_change":"2023-12-20T04:43:21.585820+0000","last_active":"2023-12-20T04:43:22.983240+0000","last_peered":"2023-12-20T04:43:22.983240+0000","last_clean":"2023-12-20T04:43:22.983240+0000","last_became_active":"2023-12-20T04:43:21.585542+0000","last_became_peered":"2023-12-20T04:43:21.585542+0000","last_unstale":"2023-12-20T04:43:22.983240+0000","last_undegraded":"2023-12-20T04:43:22.983240+0000","last_fullsized":"2023-12-20T04:43:22.983240+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:18:40.781151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.16","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.984750+0000","last_change":"2023-12-20T04:43:21.994762+0000","last_active":"2023-12-20T04:43:22.984750+0000","last_peered":"2023-12-20T04:43:22.984750+0000","last_clean":"2023-12-20T04:43:22.984750+0000","last_became_active":"2023-12-20T04:43:21.994474+0000","last_became_peered":"2023-12-20T04:43:21.994474+0000","last_unstale":"2023-12-20T04:43:22.984750+0000","last_undegraded":"2023-12-20T04:43:22.984750+0000","last_fullsized":"2023-12-20T04:43:22.984750+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:29:15.058280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.17","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811704+0000","last_change":"2023-12-20T04:43:21.589184+0000","last_active":"2023-12-20T04:43:23.811704+0000","last_peered":"2023-12-20T04:43:23.811704+0000","last_clean":"2023-12-20T04:43:23.811704+0000","last_became_active":"2023-12-20T04:43:21.588854+0000","last_became_peered":"2023-12-20T04:43:21.588854+0000","last_unstale":"2023-12-20T04:43:23.811704+0000","last_undegraded":"2023-12-20T04:43:23.811704+0000","last_fullsized":"2023-12-20T04:43:23.811704+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:22.020202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.17","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811702+0000","last_change":"2023-12-20T04:43:22.013532+0000","last_active":"2023-12-20T04:43:23.811702+0000","last_peered":"2023-12-20T04:43:23.811702+0000","last_clean":"2023-12-20T04:43:23.811702+0000","last_became_active":"2023-12-20T04:43:22.013231+0000","last_became_peered":"2023-12-20T04:43:22.013231+0000","last_unstale":"2023-12-20T04:43:23.811702+0000","last_undegraded":"2023-12-20T04:43:23.811702+0000","last_fullsized":"2023-12-20T04:43:23.811702+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:11:19.997837+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.16","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.925180+0000","last_change":"2023-12-20T04:43:21.587613+0000","last_active":"2023-12-20T04:43:23.925180+0000","last_peered":"2023-12-20T04:43:23.925180+0000","last_clean":"2023-12-20T04:43:23.925180+0000","last_became_active":"2023-12-20T04:43:21.587311+0000","last_became_peered":"2023-12-20T04:43:21.587311+0000","last_unstale":"2023-12-20T04:43:23.925180+0000","last_undegraded":"2023-12-20T04:43:23.925180+0000","last_fullsized":"2023-12-20T04:43:23.925180+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:31:04.259225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.14","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982780+0000","last_change":"2023-12-20T04:43:21.998623+0000","last_active":"2023-12-20T04:43:22.982780+0000","last_peered":"2023-12-20T04:43:22.982780+0000","last_clean":"2023-12-20T04:43:22.982780+0000","last_became_active":"2023-12-20T04:43:21.998219+0000","last_became_peered":"2023-12-20T04:43:21.998219+0000","last_unstale":"2023-12-20T04:43:22.982780+0000","last_undegraded":"2023-12-20T04:43:22.982780+0000","last_fullsized":"2023-12-20T04:43:22.982780+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:40:34.558866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.15","version":"78'3","reported_seq":14,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811585+0000","last_change":"2023-12-20T04:43:21.586570+0000","last_active":"2023-12-20T04:43:23.811585+0000","last_peered":"2023-12-20T04:43:23.811585+0000","last_clean":"2023-12-20T04:43:23.811585+0000","last_became_active":"2023-12-20T04:43:21.586303+0000","last_became_peered":"2023-12-20T04:43:21.586303+0000","last_unstale":"2023-12-20T04:43:23.811585+0000","last_undegraded":"2023-12-20T04:43:23.811585+0000","last_fullsized":"2023-12-20T04:43:23.811585+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:40:33.551335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.15","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811466+0000","last_change":"2023-12-20T04:43:22.014144+0000","last_active":"2023-12-20T04:43:23.811466+0000","last_peered":"2023-12-20T04:43:23.811466+0000","last_clean":"2023-12-20T04:43:23.811466+0000","last_became_active":"2023-12-20T04:43:22.013363+0000","last_became_peered":"2023-12-20T04:43:22.013363+0000","last_unstale":"2023-12-20T04:43:23.811466+0000","last_undegraded":"2023-12-20T04:43:23.811466+0000","last_fullsized":"2023-12-20T04:43:23.811466+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:33.942256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.14","version":"78'6","reported_seq":17,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.984560+0000","last_change":"2023-12-20T04:43:21.590149+0000","last_active":"2023-12-20T04:43:22.984560+0000","last_peered":"2023-12-20T04:43:22.984560+0000","last_clean":"2023-12-20T04:43:22.984560+0000","last_became_active":"2023-12-20T04:43:21.589821+0000","last_became_peered":"2023-12-20T04:43:21.589821+0000","last_unstale":"2023-12-20T04:43:22.984560+0000","last_undegraded":"2023-12-20T04:43:22.984560+0000","last_fullsized":"2023-12-20T04:43:22.984560+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:51:34.676994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.12","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983435+0000","last_change":"2023-12-20T04:43:21.996556+0000","last_active":"2023-12-20T04:43:22.983435+0000","last_peered":"2023-12-20T04:43:22.983435+0000","last_clean":"2023-12-20T04:43:22.983435+0000","last_became_active":"2023-12-20T04:43:21.996072+0000","last_became_peered":"2023-12-20T04:43:21.996072+0000","last_unstale":"2023-12-20T04:43:22.983435+0000","last_undegraded":"2023-12-20T04:43:22.983435+0000","last_fullsized":"2023-12-20T04:43:22.983435+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:11:16.035024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.13","version":"78'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924358+0000","last_change":"2023-12-20T04:43:21.590148+0000","last_active":"2023-12-20T04:43:23.924358+0000","last_peered":"2023-12-20T04:43:23.924358+0000","last_clean":"2023-12-20T04:43:23.924358+0000","last_became_active":"2023-12-20T04:43:21.589854+0000","last_became_peered":"2023-12-20T04:43:21.589854+0000","last_unstale":"2023-12-20T04:43:23.924358+0000","last_undegraded":"2023-12-20T04:43:23.924358+0000","last_fullsized":"2023-12-20T04:43:23.924358+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:17:46.717611+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.13","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811093+0000","last_change":"2023-12-20T04:43:22.002621+0000","last_active":"2023-12-20T04:43:23.811093+0000","last_peered":"2023-12-20T04:43:23.811093+0000","last_clean":"2023-12-20T04:43:23.811093+0000","last_became_active":"2023-12-20T04:43:22.002187+0000","last_became_peered":"2023-12-20T04:43:22.002187+0000","last_unstale":"2023-12-20T04:43:23.811093+0000","last_undegraded":"2023-12-20T04:43:23.811093+0000","last_fullsized":"2023-12-20T04:43:23.811093+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:20:22.618097+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.12","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.923426+0000","last_change":"2023-12-20T04:43:21.577668+0000","last_active":"2023-12-20T04:43:23.923426+0000","last_peered":"2023-12-20T04:43:23.923426+0000","last_clean":"2023-12-20T04:43:23.923426+0000","last_became_active":"2023-12-20T04:43:21.577393+0000","last_became_peered":"2023-12-20T04:43:21.577393+0000","last_unstale":"2023-12-20T04:43:23.923426+0000","last_undegraded":"2023-12-20T04:43:23.923426+0000","last_fullsized":"2023-12-20T04:43:23.923426+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:13.343043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.10","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983135+0000","last_change":"2023-12-20T04:43:22.008403+0000","last_active":"2023-12-20T04:43:22.983135+0000","last_peered":"2023-12-20T04:43:22.983135+0000","last_clean":"2023-12-20T04:43:22.983135+0000","last_became_active":"2023-12-20T04:43:22.008103+0000","last_became_peered":"2023-12-20T04:43:22.008103+0000","last_unstale":"2023-12-20T04:43:22.983135+0000","last_undegraded":"2023-12-20T04:43:22.983135+0000","last_fullsized":"2023-12-20T04:43:22.983135+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:14:57.167825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"8.11","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983615+0000","last_change":"2023-12-20T04:43:21.583603+0000","last_active":"2023-12-20T04:43:22.983615+0000","last_peered":"2023-12-20T04:43:22.983615+0000","last_clean":"2023-12-20T04:43:22.983615+0000","last_became_active":"2023-12-20T04:43:21.582614+0000","last_became_peered":"2023-12-20T04:43:21.582614+0000","last_unstale":"2023-12-20T04:43:22.983615+0000","last_undegraded":"2023-12-20T04:43:22.983615+0000","last_fullsized":"2023-12-20T04:43:22.983615+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:30:22.020202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.11","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811318+0000","last_change":"2023-12-20T04:43:22.000737+0000","last_active":"2023-12-20T04:43:23.811318+0000","last_peered":"2023-12-20T04:43:23.811318+0000","last_clean":"2023-12-20T04:43:23.811318+0000","last_became_active":"2023-12-20T04:43:22.000364+0000","last_became_peered":"2023-12-20T04:43:22.000364+0000","last_unstale":"2023-12-20T04:43:23.811318+0000","last_undegraded":"2023-12-20T04:43:23.811318+0000","last_fullsized":"2023-12-20T04:43:23.811318+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:40:16.881555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.10","version":"78'8","reported_seq":19,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811289+0000","last_change":"2023-12-20T04:43:21.582273+0000","last_active":"2023-12-20T04:43:23.811289+0000","last_peered":"2023-12-20T04:43:23.811289+0000","last_clean":"2023-12-20T04:43:23.811289+0000","last_became_active":"2023-12-20T04:43:21.581917+0000","last_became_peered":"2023-12-20T04:43:21.581917+0000","last_unstale":"2023-12-20T04:43:23.811289+0000","last_undegraded":"2023-12-20T04:43:23.811289+0000","last_fullsized":"2023-12-20T04:43:23.811289+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:26.929937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"9.e","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811666+0000","last_change":"2023-12-20T04:43:22.010856+0000","last_active":"2023-12-20T04:43:23.811666+0000","last_peered":"2023-12-20T04:43:23.811666+0000","last_clean":"2023-12-20T04:43:23.811666+0000","last_became_active":"2023-12-20T04:43:22.010461+0000","last_became_peered":"2023-12-20T04:43:22.010461+0000","last_unstale":"2023-12-20T04:43:23.811666+0000","last_undegraded":"2023-12-20T04:43:23.811666+0000","last_fullsized":"2023-12-20T04:43:23.811666+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:46:45.293265+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.f","version":"78'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.925157+0000","last_change":"2023-12-20T04:43:21.592864+0000","last_active":"2023-12-20T04:43:23.925157+0000","last_peered":"2023-12-20T04:43:23.925157+0000","last_clean":"2023-12-20T04:43:23.925157+0000","last_became_active":"2023-12-20T04:43:21.592590+0000","last_became_peered":"2023-12-20T04:43:21.592590+0000","last_unstale":"2023-12-20T04:43:23.925157+0000","last_undegraded":"2023-12-20T04:43:23.925157+0000","last_fullsized":"2023-12-20T04:43:23.925157+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:32.934724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.f","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.984182+0000","last_change":"2023-12-20T04:43:21.996317+0000","last_active":"2023-12-20T04:43:22.984182+0000","last_peered":"2023-12-20T04:43:22.984182+0000","last_clean":"2023-12-20T04:43:22.984182+0000","last_became_active":"2023-12-20T04:43:21.995973+0000","last_became_peered":"2023-12-20T04:43:21.995973+0000","last_unstale":"2023-12-20T04:43:22.984182+0000","last_undegraded":"2023-12-20T04:43:22.984182+0000","last_fullsized":"2023-12-20T04:43:22.984182+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:49.408607+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.e","version":"78'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924372+0000","last_change":"2023-12-20T04:43:21.586179+0000","last_active":"2023-12-20T04:43:23.924372+0000","last_peered":"2023-12-20T04:43:23.924372+0000","last_clean":"2023-12-20T04:43:23.924372+0000","last_became_active":"2023-12-20T04:43:21.585664+0000","last_became_peered":"2023-12-20T04:43:21.585664+0000","last_unstale":"2023-12-20T04:43:23.924372+0000","last_undegraded":"2023-12-20T04:43:23.924372+0000","last_fullsized":"2023-12-20T04:43:23.924372+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:11:15.027492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.c","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924704+0000","last_change":"2023-12-20T04:43:21.992201+0000","last_active":"2023-12-20T04:43:23.924704+0000","last_peered":"2023-12-20T04:43:23.924704+0000","last_clean":"2023-12-20T04:43:23.924704+0000","last_became_active":"2023-12-20T04:43:21.991955+0000","last_became_peered":"2023-12-20T04:43:21.991955+0000","last_unstale":"2023-12-20T04:43:23.924704+0000","last_undegraded":"2023-12-20T04:43:23.924704+0000","last_fullsized":"2023-12-20T04:43:23.924704+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:24:50.593990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.d","version":"78'1","reported_seq":12,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.984098+0000","last_change":"2023-12-20T04:43:21.583073+0000","last_active":"2023-12-20T04:43:22.984098+0000","last_peered":"2023-12-20T04:43:22.984098+0000","last_clean":"2023-12-20T04:43:22.984098+0000","last_became_active":"2023-12-20T04:43:21.582632+0000","last_became_peered":"2023-12-20T04:43:21.582632+0000","last_unstale":"2023-12-20T04:43:22.984098+0000","last_undegraded":"2023-12-20T04:43:22.984098+0000","last_fullsized":"2023-12-20T04:43:22.984098+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:50:40.225267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.d","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811577+0000","last_change":"2023-12-20T04:43:22.014908+0000","last_active":"2023-12-20T04:43:23.811577+0000","last_peered":"2023-12-20T04:43:23.811577+0000","last_clean":"2023-12-20T04:43:23.811577+0000","last_became_active":"2023-12-20T04:43:22.014435+0000","last_became_peered":"2023-12-20T04:43:22.014435+0000","last_unstale":"2023-12-20T04:43:23.811577+0000","last_undegraded":"2023-12-20T04:43:23.811577+0000","last_fullsized":"2023-12-20T04:43:23.811577+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:11:16.035024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.c","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924509+0000","last_change":"2023-12-20T04:43:21.585469+0000","last_active":"2023-12-20T04:43:23.924509+0000","last_peered":"2023-12-20T04:43:23.924509+0000","last_clean":"2023-12-20T04:43:23.924509+0000","last_became_active":"2023-12-20T04:43:21.585168+0000","last_became_peered":"2023-12-20T04:43:21.585168+0000","last_unstale":"2023-12-20T04:43:23.924509+0000","last_undegraded":"2023-12-20T04:43:23.924509+0000","last_fullsized":"2023-12-20T04:43:23.924509+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:39:13.024438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.a","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983620+0000","last_change":"2023-12-20T04:43:21.996349+0000","last_active":"2023-12-20T04:43:22.983620+0000","last_peered":"2023-12-20T04:43:22.983620+0000","last_clean":"2023-12-20T04:43:22.983620+0000","last_became_active":"2023-12-20T04:43:21.995604+0000","last_became_peered":"2023-12-20T04:43:21.995604+0000","last_unstale":"2023-12-20T04:43:22.983620+0000","last_undegraded":"2023-12-20T04:43:22.983620+0000","last_fullsized":"2023-12-20T04:43:22.983620+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:39:14.031970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.b","version":"78'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924642+0000","last_change":"2023-12-20T04:43:21.580313+0000","last_active":"2023-12-20T04:43:23.924642+0000","last_peered":"2023-12-20T04:43:23.924642+0000","last_clean":"2023-12-20T04:43:23.924642+0000","last_became_active":"2023-12-20T04:43:21.579994+0000","last_became_peered":"2023-12-20T04:43:21.579994+0000","last_unstale":"2023-12-20T04:43:23.924642+0000","last_undegraded":"2023-12-20T04:43:23.924642+0000","last_fullsized":"2023-12-20T04:43:23.924642+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:11:18.990305+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.b","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924483+0000","last_change":"2023-12-20T04:43:22.008471+0000","last_active":"2023-12-20T04:43:23.924483+0000","last_peered":"2023-12-20T04:43:23.924483+0000","last_clean":"2023-12-20T04:43:23.924483+0000","last_became_active":"2023-12-20T04:43:22.003050+0000","last_became_peered":"2023-12-20T04:43:22.003050+0000","last_unstale":"2023-12-20T04:43:23.924483+0000","last_undegraded":"2023-12-20T04:43:23.924483+0000","last_fullsized":"2023-12-20T04:43:23.924483+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:18.582906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.a","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982602+0000","last_change":"2023-12-20T04:43:21.583671+0000","last_active":"2023-12-20T04:43:22.982602+0000","last_peered":"2023-12-20T04:43:22.982602+0000","last_clean":"2023-12-20T04:43:22.982602+0000","last_became_active":"2023-12-20T04:43:21.583405+0000","last_became_peered":"2023-12-20T04:43:21.583405+0000","last_unstale":"2023-12-20T04:43:22.982602+0000","last_undegraded":"2023-12-20T04:43:22.982602+0000","last_fullsized":"2023-12-20T04:43:22.982602+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:21:38.152790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.9","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924004+0000","last_change":"2023-12-20T04:43:22.001495+0000","last_active":"2023-12-20T04:43:23.924004+0000","last_peered":"2023-12-20T04:43:23.924004+0000","last_clean":"2023-12-20T04:43:23.924004+0000","last_became_active":"2023-12-20T04:43:21.999535+0000","last_became_peered":"2023-12-20T04:43:21.999535+0000","last_unstale":"2023-12-20T04:43:23.924004+0000","last_undegraded":"2023-12-20T04:43:23.924004+0000","last_fullsized":"2023-12-20T04:43:23.924004+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:12:54.915087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.8","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983498+0000","last_change":"2023-12-20T04:43:21.587671+0000","last_active":"2023-12-20T04:43:22.983498+0000","last_peered":"2023-12-20T04:43:22.983498+0000","last_clean":"2023-12-20T04:43:22.983498+0000","last_became_active":"2023-12-20T04:43:21.587385+0000","last_became_peered":"2023-12-20T04:43:21.587385+0000","last_unstale":"2023-12-20T04:43:22.983498+0000","last_undegraded":"2023-12-20T04:43:22.983498+0000","last_fullsized":"2023-12-20T04:43:22.983498+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:40:33.551335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.6","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.984167+0000","last_change":"2023-12-20T04:43:21.998495+0000","last_active":"2023-12-20T04:43:22.984167+0000","last_peered":"2023-12-20T04:43:22.984167+0000","last_clean":"2023-12-20T04:43:22.984167+0000","last_became_active":"2023-12-20T04:43:21.998159+0000","last_became_peered":"2023-12-20T04:43:21.998159+0000","last_unstale":"2023-12-20T04:43:22.984167+0000","last_undegraded":"2023-12-20T04:43:22.984167+0000","last_fullsized":"2023-12-20T04:43:22.984167+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:29:15.058280+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.7","version":"78'2","reported_seq":13,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983185+0000","last_change":"2023-12-20T04:43:21.586777+0000","last_active":"2023-12-20T04:43:22.983185+0000","last_peered":"2023-12-20T04:43:22.983185+0000","last_clean":"2023-12-20T04:43:22.983185+0000","last_became_active":"2023-12-20T04:43:21.586509+0000","last_became_peered":"2023-12-20T04:43:21.586509+0000","last_unstale":"2023-12-20T04:43:22.983185+0000","last_undegraded":"2023-12-20T04:43:22.983185+0000","last_fullsized":"2023-12-20T04:43:22.983185+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:33:21.576017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.7","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.984317+0000","last_change":"2023-12-20T04:43:21.999034+0000","last_active":"2023-12-20T04:43:22.984317+0000","last_peered":"2023-12-20T04:43:22.984317+0000","last_clean":"2023-12-20T04:43:22.984317+0000","last_became_active":"2023-12-20T04:43:21.998750+0000","last_became_peered":"2023-12-20T04:43:21.998750+0000","last_unstale":"2023-12-20T04:43:22.984317+0000","last_undegraded":"2023-12-20T04:43:22.984317+0000","last_fullsized":"2023-12-20T04:43:22.984317+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:12:07.109073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"8.6","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924994+0000","last_change":"2023-12-20T04:43:21.588511+0000","last_active":"2023-12-20T04:43:23.924994+0000","last_peered":"2023-12-20T04:43:23.924994+0000","last_clean":"2023-12-20T04:43:23.924994+0000","last_became_active":"2023-12-20T04:43:21.588223+0000","last_became_peered":"2023-12-20T04:43:21.588223+0000","last_unstale":"2023-12-20T04:43:23.924994+0000","last_undegraded":"2023-12-20T04:43:23.924994+0000","last_fullsized":"2023-12-20T04:43:23.924994+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:40:15.874023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"9.4","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982789+0000","last_change":"2023-12-20T04:43:21.998457+0000","last_active":"2023-12-20T04:43:22.982789+0000","last_peered":"2023-12-20T04:43:22.982789+0000","last_clean":"2023-12-20T04:43:22.982789+0000","last_became_active":"2023-12-20T04:43:21.997835+0000","last_became_peered":"2023-12-20T04:43:21.997835+0000","last_unstale":"2023-12-20T04:43:22.982789+0000","last_undegraded":"2023-12-20T04:43:22.982789+0000","last_fullsized":"2023-12-20T04:43:22.982789+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:27.937468+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.5","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.923631+0000","last_change":"2023-12-20T04:43:21.576897+0000","last_active":"2023-12-20T04:43:23.923631+0000","last_peered":"2023-12-20T04:43:23.923631+0000","last_clean":"2023-12-20T04:43:23.923631+0000","last_became_active":"2023-12-20T04:43:21.576339+0000","last_became_peered":"2023-12-20T04:43:21.576339+0000","last_unstale":"2023-12-20T04:43:23.923631+0000","last_undegraded":"2023-12-20T04:43:23.923631+0000","last_fullsized":"2023-12-20T04:43:23.923631+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:59:22.224225+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"9.5","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924883+0000","last_change":"2023-12-20T04:43:22.010202+0000","last_active":"2023-12-20T04:43:23.924883+0000","last_peered":"2023-12-20T04:43:23.924883+0000","last_clean":"2023-12-20T04:43:23.924883+0000","last_became_active":"2023-12-20T04:43:22.008971+0000","last_became_peered":"2023-12-20T04:43:22.008971+0000","last_unstale":"2023-12-20T04:43:23.924883+0000","last_undegraded":"2023-12-20T04:43:23.924883+0000","last_fullsized":"2023-12-20T04:43:23.924883+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:31:28.967106+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"8.4","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.983989+0000","last_change":"2023-12-20T04:43:21.584038+0000","last_active":"2023-12-20T04:43:22.983989+0000","last_peered":"2023-12-20T04:43:22.983989+0000","last_clean":"2023-12-20T04:43:22.983989+0000","last_became_active":"2023-12-20T04:43:21.583757+0000","last_became_peered":"2023-12-20T04:43:21.583757+0000","last_unstale":"2023-12-20T04:43:22.983989+0000","last_undegraded":"2023-12-20T04:43:22.983989+0000","last_fullsized":"2023-12-20T04:43:22.983989+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:11:50.589803+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"9.2","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924057+0000","last_change":"2023-12-20T04:43:21.996972+0000","last_active":"2023-12-20T04:43:23.924057+0000","last_peered":"2023-12-20T04:43:23.924057+0000","last_clean":"2023-12-20T04:43:23.924057+0000","last_became_active":"2023-12-20T04:43:21.996710+0000","last_became_peered":"2023-12-20T04:43:21.996710+0000","last_unstale":"2023-12-20T04:43:23.924057+0000","last_undegraded":"2023-12-20T04:43:23.924057+0000","last_fullsized":"2023-12-20T04:43:23.924057+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:12:54.915087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.3","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982570+0000","last_change":"2023-12-20T04:43:21.577608+0000","last_active":"2023-12-20T04:43:22.982570+0000","last_peered":"2023-12-20T04:43:22.982570+0000","last_clean":"2023-12-20T04:43:22.982570+0000","last_became_active":"2023-12-20T04:43:21.577261+0000","last_became_peered":"2023-12-20T04:43:21.577261+0000","last_unstale":"2023-12-20T04:43:22.982570+0000","last_undegraded":"2023-12-20T04:43:22.982570+0000","last_fullsized":"2023-12-20T04:43:22.982570+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:35:25.100304+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.3","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.811263+0000","last_change":"2023-12-20T04:43:22.005042+0000","last_active":"2023-12-20T04:43:23.811263+0000","last_peered":"2023-12-20T04:43:23.811263+0000","last_clean":"2023-12-20T04:43:23.811263+0000","last_became_active":"2023-12-20T04:43:22.004751+0000","last_became_peered":"2023-12-20T04:43:22.004751+0000","last_unstale":"2023-12-20T04:43:23.811263+0000","last_undegraded":"2023-12-20T04:43:23.811263+0000","last_fullsized":"2023-12-20T04:43:23.811263+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:25.802282+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"8.2","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.984227+0000","last_change":"2023-12-20T04:43:21.589202+0000","last_active":"2023-12-20T04:43:22.984227+0000","last_peered":"2023-12-20T04:43:22.984227+0000","last_clean":"2023-12-20T04:43:22.984227+0000","last_became_active":"2023-12-20T04:43:21.588863+0000","last_became_peered":"2023-12-20T04:43:21.588863+0000","last_unstale":"2023-12-20T04:43:22.984227+0000","last_undegraded":"2023-12-20T04:43:22.984227+0000","last_fullsized":"2023-12-20T04:43:22.984227+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:28:48.401075+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"9.0","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982376+0000","last_change":"2023-12-20T04:43:21.996415+0000","last_active":"2023-12-20T04:43:22.982376+0000","last_peered":"2023-12-20T04:43:22.982376+0000","last_clean":"2023-12-20T04:43:22.982376+0000","last_became_active":"2023-12-20T04:43:21.995386+0000","last_became_peered":"2023-12-20T04:43:21.995386+0000","last_unstale":"2023-12-20T04:43:22.982376+0000","last_undegraded":"2023-12-20T04:43:22.982376+0000","last_fullsized":"2023-12-20T04:43:22.982376+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:03:33.402892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"8.1","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:22.982436+0000","last_change":"2023-12-20T04:43:21.590630+0000","last_active":"2023-12-20T04:43:22.982436+0000","last_peered":"2023-12-20T04:43:22.982436+0000","last_clean":"2023-12-20T04:43:22.982436+0000","last_became_active":"2023-12-20T04:43:21.590362+0000","last_became_peered":"2023-12-20T04:43:21.590362+0000","last_unstale":"2023-12-20T04:43:22.982436+0000","last_undegraded":"2023-12-20T04:43:22.982436+0000","last_fullsized":"2023-12-20T04:43:22.982436+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:47:01.682508+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"9.1","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924306+0000","last_change":"2023-12-20T04:43:21.994496+0000","last_active":"2023-12-20T04:43:23.924306+0000","last_peered":"2023-12-20T04:43:23.924306+0000","last_clean":"2023-12-20T04:43:23.924306+0000","last_became_active":"2023-12-20T04:43:21.994034+0000","last_became_peered":"2023-12-20T04:43:21.994034+0000","last_unstale":"2023-12-20T04:43:23.924306+0000","last_undegraded":"2023-12-20T04:43:23.924306+0000","last_fullsized":"2023-12-20T04:43:23.924306+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:05:34.433133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.0","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.924072+0000","last_change":"2023-12-20T04:43:21.580845+0000","last_active":"2023-12-20T04:43:23.924072+0000","last_peered":"2023-12-20T04:43:23.924072+0000","last_clean":"2023-12-20T04:43:23.924072+0000","last_became_active":"2023-12-20T04:43:21.580582+0000","last_became_peered":"2023-12-20T04:43:21.580582+0000","last_unstale":"2023-12-20T04:43:23.924072+0000","last_undegraded":"2023-12-20T04:43:23.924072+0000","last_fullsized":"2023-12-20T04:43:23.924072+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:10.610546+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"9.8","version":"0'0","reported_seq":10,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.923973+0000","last_change":"2023-12-20T04:43:22.004878+0000","last_active":"2023-12-20T04:43:23.923973+0000","last_peered":"2023-12-20T04:43:23.923973+0000","last_clean":"2023-12-20T04:43:23.923973+0000","last_became_active":"2023-12-20T04:43:22.004628+0000","last_became_peered":"2023-12-20T04:43:22.004628+0000","last_unstale":"2023-12-20T04:43:23.923973+0000","last_undegraded":"2023-12-20T04:43:23.923973+0000","last_fullsized":"2023-12-20T04:43:23.923973+0000","mapping_epoch":77,"log_start":"0'0","ondisk_log_start":"0'0","created":77,"last_epoch_clean":78,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:21.557238+0000","last_clean_scrub_stamp":"2023-12-20T04:43:21.557238+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:28:50.727344+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"8.9","version":"0'0","reported_seq":11,"reported_epoch":79,"state":"active+clean","last_fresh":"2023-12-20T04:43:23.923492+0000","last_change":"2023-12-20T04:43:21.582751+0000","last_active":"2023-12-20T04:43:23.923492+0000","last_peered":"2023-12-20T04:43:23.923492+0000","last_clean":"2023-12-20T04:43:23.923492+0000","last_became_active":"2023-12-20T04:43:21.582107+0000","last_became_peered":"2023-12-20T04:43:21.582107+0000","last_unstale":"2023-12-20T04:43:23.923492+0000","last_undegraded":"2023-12-20T04:43:23.923492+0000","last_fullsized":"2023-12-20T04:43:23.923492+0000","mapping_epoch":76,"log_start":"0'0","ondisk_log_start":"0'0","created":76,"last_epoch_clean":77,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:20.549706+0000","last_clean_scrub_stamp":"2023-12-20T04:43:20.549706+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:14:04.533318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"67'161","reported_seq":374,"reported_epoch":78,"state":"active+clean","last_fresh":"2023-12-20T04:43:21.995445+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:43:21.995445+0000","last_peered":"2023-12-20T04:43:21.995445+0000","last_clean":"2023-12-20T04:43:21.995445+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:43:21.995445+0000","last_undegraded":"2023-12-20T04:43:21.995445+0000","last_fullsized":"2023-12-20T04:43:21.995445+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":161,"log_dups_size":0,"ondisk_log_size":161,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":9,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":8,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":500,"num_read_kb":924,"num_write":239,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":161,"ondisk_log_size":161,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575028,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36664,"kb_used_data":1520,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335176,"statfs":{"total":96636764160,"available":96599220224,"internally_reserved":0,"allocated":1556480,"data_stored":1037633,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.71999999999999997,"15min":0.70299999999999996},"min":{"1min":0.46600000000000003,"5min":0.372,"15min":0.33000000000000002},"max":{"1min":1.048,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.89800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.71399999999999997,"15min":0.71099999999999997},"min":{"1min":0.56399999999999995,"5min":0.39600000000000002,"15min":0.39400000000000002},"max":{"1min":0.89300000000000002,"5min":1,"15min":1.115},"last":0.80700000000000005}]},{"osd":1,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.73199999999999998,"15min":0.72299999999999998},"min":{"1min":0.55900000000000005,"5min":0.443,"15min":0.38100000000000001},"max":{"1min":1.0369999999999999,"5min":1.095,"15min":1.2190000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.745,"15min":0.745},"min":{"1min":0.51000000000000001,"5min":0.50800000000000001,"15min":0.42999999999999999},"max":{"1min":0.95099999999999996,"5min":1.032,"15min":2.649},"last":0.871}]},{"osd":2,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.71899999999999997,"15min":0.72299999999999998},"min":{"1min":0.56200000000000006,"5min":0.48899999999999999,"15min":0.437},"max":{"1min":0.94099999999999995,"5min":1.075,"15min":1.5960000000000001},"last":0.85099999999999998},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.751,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.46899999999999997,"15min":0.45300000000000001},"max":{"1min":0.97999999999999998,"5min":1.1000000000000001,"15min":1.135},"last":0.98999999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.72199999999999998,"15min":0.73499999999999999},"min":{"1min":0.50700000000000001,"5min":0.44,"15min":0.44},"max":{"1min":0.999,"5min":1.016,"15min":1.641},"last":0.76600000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77200000000000002,"15min":0.77900000000000003},"min":{"1min":0.59799999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93899999999999995,"5min":1.131,"15min":2.5990000000000002},"last":0.93500000000000005}]},{"osd":4,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.621,"5min":0.60599999999999998,"15min":0.60799999999999998},"min":{"1min":0.45400000000000001,"5min":0.39200000000000002,"15min":0.34599999999999997},"max":{"1min":0.90700000000000003,"5min":0.98299999999999998,"15min":1.0109999999999999},"last":0.55800000000000005},{"interface":"front","average":{"1min":0.63800000000000001,"5min":0.627,"15min":0.63600000000000001},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.879,"5min":1.0469999999999999,"15min":1.242},"last":0.67300000000000004}]},{"osd":5,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.66300000000000003,"15min":0.66400000000000003},"min":{"1min":0.496,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.089,"5min":1.089,"15min":1.0900000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.71299999999999997,"5min":0.68000000000000005,"15min":0.68700000000000006},"min":{"1min":0.53000000000000003,"5min":0.442,"15min":0.432},"max":{"1min":0.96999999999999997,"5min":1.052,"15min":1.071},"last":0.65400000000000003}]},{"osd":6,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.753,"15min":0.746},"min":{"1min":0.60699999999999998,"5min":0.53100000000000003,"15min":0.443},"max":{"1min":1.012,"5min":1.1020000000000001,"15min":1.1279999999999999},"last":0.94899999999999995},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.72399999999999998,"15min":0.71299999999999997},"min":{"1min":0.51700000000000002,"5min":0.48699999999999999,"15min":0.44400000000000001},"max":{"1min":1.0309999999999999,"5min":1.125,"15min":1.125},"last":0.62}]}]},{"osd":6,"up_from":13,"seq":55834575028,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36076,"kb_used_data":932,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335764,"statfs":{"total":96636764160,"available":96599822336,"internally_reserved":0,"allocated":954368,"data_stored":447105,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.55600000000000005,"5min":0.33800000000000002,"15min":0.27100000000000002},"max":{"1min":0.92900000000000005,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.71999999999999997,"15min":0.71699999999999997},"min":{"1min":0.626,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.1910000000000001},"last":0.877}]},{"osd":1,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69099999999999995,"15min":0.70199999999999996},"min":{"1min":0.55100000000000005,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.625},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.57699999999999996,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.97799999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.753}]},{"osd":2,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.68400000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.006,"5min":1.216,"15min":1.216},"last":0.80300000000000005},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.38300000000000001},"max":{"1min":0.95599999999999996,"5min":1.0089999999999999,"15min":1.054},"last":0.77500000000000002}]},{"osd":3,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.78500000000000003,"15min":0.75900000000000001},"min":{"1min":0.68300000000000005,"5min":0.51800000000000002,"15min":0.45600000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.109},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96499999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.65800000000000003}]},{"osd":4,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.624,"15min":0.621},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.36399999999999999},"max":{"1min":0.90300000000000002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.84899999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":0.995,"15min":1.071},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.51000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.42199999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.68999999999999995},"min":{"1min":0.54400000000000004,"5min":0.435,"15min":0.434},"max":{"1min":1,"5min":1.306,"15min":1.306},"last":0.58799999999999997}]},{"osd":7,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72599999999999998,"15min":0.72299999999999998},"min":{"1min":0.61899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.2270000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58899999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":1.002,"15min":1.169},"last":0.57199999999999995}]}]},{"osd":5,"up_from":13,"seq":55834575029,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36084,"kb_used_data":940,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335756,"statfs":{"total":96636764160,"available":96599814144,"internally_reserved":0,"allocated":962560,"data_stored":447757,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.69699999999999995,"15min":0.68799999999999994},"min":{"1min":0.55200000000000005,"5min":0.33700000000000002,"15min":0.29999999999999999},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.05},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.755,"5min":0.71899999999999997,"15min":0.71399999999999997},"min":{"1min":0.55800000000000005,"5min":0.41599999999999998,"15min":0.34300000000000003},"max":{"1min":0.85899999999999999,"5min":0.95099999999999996,"15min":0.98599999999999999},"last":0.51800000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.71699999999999997,"15min":0.70199999999999996},"min":{"1min":0.50700000000000001,"5min":0.45500000000000002,"15min":0.441},"max":{"1min":0.92700000000000005,"5min":1.044,"15min":1.0600000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.748,"5min":0.72399999999999998,"15min":0.73699999999999999},"min":{"1min":0.60299999999999998,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.96999999999999997,"5min":0.96999999999999997,"15min":1.1140000000000001},"last":0.71899999999999997}]},{"osd":2,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.73199999999999998,"15min":0.72699999999999998},"min":{"1min":0.54000000000000004,"5min":0.432,"15min":0.40500000000000003},"max":{"1min":0.94399999999999995,"5min":0.94399999999999995,"15min":1.153},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.73299999999999998,"15min":0.72699999999999998},"min":{"1min":0.58599999999999997,"5min":0.48799999999999999,"15min":0.47899999999999998},"max":{"1min":0.91600000000000004,"5min":0.95999999999999996,"15min":1.004},"last":0.70499999999999996}]},{"osd":3,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74299999999999999,"15min":0.748},"min":{"1min":0.56299999999999994,"5min":0.48799999999999999,"15min":0.47499999999999998},"max":{"1min":0.94999999999999996,"5min":0.98099999999999998,"15min":1.758},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.75600000000000001,"15min":0.749},"min":{"1min":0.62,"5min":0.46899999999999997,"15min":0.436},"max":{"1min":0.89800000000000002,"5min":0.98599999999999999,"15min":1.0469999999999999},"last":0.56499999999999995}]},{"osd":4,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.59799999999999998,"5min":0.60299999999999998,"15min":0.61799999999999999},"min":{"1min":0.433,"5min":0.40600000000000003,"15min":0.35099999999999998},"max":{"1min":0.76300000000000001,"5min":0.97299999999999998,"15min":1.0229999999999999},"last":0.63500000000000001},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.63600000000000001,"15min":0.63500000000000001},"min":{"1min":0.46600000000000003,"5min":0.40699999999999997,"15min":0.39600000000000002},"max":{"1min":0.81000000000000005,"5min":0.98699999999999999,"15min":1.077},"last":0.77300000000000002}]},{"osd":6,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.68100000000000005,"15min":0.68200000000000005},"min":{"1min":0.499,"5min":0.46400000000000002,"15min":0.39900000000000002},"max":{"1min":0.90000000000000002,"5min":0.94699999999999995,"15min":1.0700000000000001},"last":0.754},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64000000000000001,"15min":0.64300000000000002},"min":{"1min":0.38500000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.86599999999999999,"5min":0.999,"15min":0.999},"last":0.74099999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:42:47 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.68899999999999995,"15min":0.70099999999999996},"min":{"1min":0.57599999999999996,"5min":0.48999999999999999,"15min":0.42999999999999999},"max":{"1min":0.83899999999999997,"5min":1.0429999999999999,"15min":1.141},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.754,"15min":0.753},"min":{"1min":0.59699999999999998,"5min":0.54900000000000004,"15min":0.47899999999999998},"max":{"1min":0.95799999999999996,"5min":1.012,"15min":1.119},"last":0.80100000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575030,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58232,"kb_used_data":944,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313608,"statfs":{"total":96636764160,"available":96577134592,"internally_reserved":0,"allocated":966656,"data_stored":448669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69799999999999995,"5min":0.68700000000000006,"15min":0.69799999999999995},"min":{"1min":0.51700000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":1.0129999999999999,"5min":1.024,"15min":1.5509999999999999},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.70699999999999996,"5min":0.70599999999999996,"15min":0.70699999999999996},"min":{"1min":0.51900000000000002,"5min":0.376,"15min":0.36199999999999999},"max":{"1min":0.89500000000000002,"5min":1.625,"15min":1.633},"last":0.66200000000000003}]},{"osd":1,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.70699999999999996,"15min":0.71599999999999997},"min":{"1min":0.55200000000000005,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.95199999999999996,"5min":0.98799999999999999,"15min":1.798},"last":0.503},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.752,"15min":0.748},"min":{"1min":0.53700000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.036,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.749}]},{"osd":2,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.74099999999999999,"15min":0.73699999999999999},"min":{"1min":0.66000000000000003,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.0209999999999999,"5min":1.52,"15min":1.728},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.75,"15min":0.754},"min":{"1min":0.46899999999999997,"5min":0.38700000000000001,"15min":0.38700000000000001},"max":{"1min":0.91300000000000003,"5min":1.093,"15min":1.6619999999999999},"last":0.54500000000000004}]},{"osd":3,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.72899999999999998,"15min":0.72599999999999998},"min":{"1min":0.56699999999999995,"5min":0.41799999999999998,"15min":0.40799999999999997},"max":{"1min":1.004,"5min":1.554,"15min":1.5760000000000001},"last":0.60599999999999998},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76500000000000001,"15min":0.76800000000000002},"min":{"1min":0.63,"5min":0.498,"15min":0.48299999999999998},"max":{"1min":0.98399999999999999,"5min":1.0620000000000001,"15min":1.7649999999999999},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.623,"15min":0.627},"min":{"1min":0.44,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":0.91300000000000003,"5min":1.456,"15min":1.456},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.625,"5min":0.63400000000000001,"15min":0.63700000000000001},"min":{"1min":0.40200000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.96599999999999997,"5min":1.589,"15min":1.589},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.65000000000000002,"15min":0.65900000000000003},"min":{"1min":0.47899999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.89200000000000002,"5min":1.099,"15min":1.6739999999999999},"last":0.63300000000000001},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69699999999999995,"15min":0.70099999999999996},"min":{"1min":0.45000000000000001,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96199999999999997,"5min":1.0620000000000001,"15min":1.0620000000000001},"last":0.89400000000000002}]},{"osd":7,"last update":"Wed Dec 20 04:42:49 2023","interfaces":[{"interface":"back","average":{"1min":0.69499999999999995,"5min":0.70099999999999996,"15min":0.69799999999999995},"min":{"1min":0.46700000000000003,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.94099999999999995,"5min":1.655,"15min":1.655},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.76500000000000001,"5min":0.73799999999999999,"15min":0.74099999999999999},"min":{"1min":0.432,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.052,"5min":1.125,"15min":1.125},"last":0.57899999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575029,"num_pgs":6,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36072,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335768,"statfs":{"total":96636764160,"available":96599826432,"internally_reserved":0,"allocated":950272,"data_stored":440861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.48499999999999999,"5min":0.47299999999999998,"15min":0.497},"min":{"1min":0.32900000000000001,"5min":0.26200000000000001,"15min":0.23200000000000001},"max":{"1min":0.64300000000000002,"5min":0.875,"15min":4.3700000000000001},"last":0.30499999999999999},{"interface":"front","average":{"1min":0.53200000000000003,"5min":0.52600000000000002,"15min":0.53000000000000003},"min":{"1min":0.36299999999999999,"5min":0.30599999999999999,"15min":0.28799999999999998},"max":{"1min":0.72199999999999998,"5min":0.80800000000000005,"15min":3.355},"last":0.377}]},{"osd":1,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.51300000000000001,"5min":0.47599999999999998,"15min":0.498},"min":{"1min":0.38500000000000001,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.73999999999999999,"5min":0.73999999999999999,"15min":3.7160000000000002},"last":0.32900000000000001},{"interface":"front","average":{"1min":0.57299999999999995,"5min":0.55500000000000005,"15min":0.57099999999999995},"min":{"1min":0.42699999999999999,"5min":0.316,"15min":0.28399999999999997},"max":{"1min":0.74399999999999999,"5min":0.78100000000000003,"15min":3.3109999999999999},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.59099999999999997,"15min":0.60699999999999998},"min":{"1min":0.51600000000000001,"5min":0.318,"15min":0.318},"max":{"1min":0.80700000000000005,"5min":0.88700000000000001,"15min":4.0110000000000001},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.58299999999999996,"5min":0.55800000000000005,"15min":0.60599999999999998},"min":{"1min":0.442,"5min":0.40500000000000003,"15min":0.316},"max":{"1min":0.79200000000000004,"5min":0.93300000000000005,"15min":6.1399999999999997},"last":0.34899999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.79200000000000004},"min":{"1min":0.65800000000000003,"5min":0.55200000000000005,"15min":0.48499999999999999},"max":{"1min":0.98099999999999998,"5min":1.131,"15min":1.3089999999999999},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.80300000000000005,"15min":0.83999999999999997},"min":{"1min":0.59499999999999997,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":1.198,"5min":1.4059999999999999,"15min":3.9510000000000001},"last":0.75800000000000001}]},{"osd":5,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.85299999999999998,"5min":0.81899999999999995,"15min":0.85099999999999998},"min":{"1min":0.69799999999999995,"5min":0.54200000000000004,"15min":0.46500000000000002},"max":{"1min":1.181,"5min":1.3300000000000001,"15min":3.9830000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.80300000000000005,"5min":0.78800000000000003,"15min":0.82999999999999996},"min":{"1min":0.59799999999999998,"5min":0.50800000000000001,"15min":0.38400000000000001},"max":{"1min":1.0489999999999999,"5min":1.252,"15min":4.6550000000000002},"last":0.85199999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.78700000000000003,"15min":0.80800000000000005},"min":{"1min":0.57299999999999995,"5min":0.497,"15min":0.45600000000000002},"max":{"1min":0.98399999999999999,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81799999999999995,"15min":0.85099999999999998},"min":{"1min":0.64800000000000002,"5min":0.54500000000000004,"15min":0.499},"max":{"1min":1.133,"5min":1.2270000000000001,"15min":3.8900000000000001},"last":0.73599999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:42:48 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.79900000000000004,"15min":0.84299999999999997},"min":{"1min":0.61099999999999999,"5min":0.51100000000000001,"15min":0.49299999999999999},"max":{"1min":1.0249999999999999,"5min":1.1319999999999999,"15min":4.6970000000000001},"last":0.745},{"interface":"front","average":{"1min":0.86799999999999999,"5min":0.81899999999999995,"15min":0.83199999999999996},"min":{"1min":0.63700000000000001,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.145,"5min":1.29,"15min":1.29},"last":0.86899999999999999}]}]},{"osd":2,"up_from":13,"seq":55834575028,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40168,"kb_used_data":928,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331672,"statfs":{"total":96636764160,"available":96595632128,"internally_reserved":0,"allocated":950272,"data_stored":440861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.433,"15min":0.439},"min":{"1min":0.32800000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.625,"5min":0.78900000000000003,"15min":0.89600000000000002},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.502,"15min":0.48899999999999999},"min":{"1min":0.39300000000000002,"5min":0.28499999999999998,"15min":0.26700000000000002},"max":{"1min":0.748,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60099999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.46899999999999997},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.876,"15min":0.876},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.28100000000000003},"max":{"1min":0.65900000000000003,"5min":0.88600000000000001,"15min":1.01},"last":0.59099999999999997}]},{"osd":3,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.49299999999999999,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.49299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91600000000000004,"5min":0.95599999999999996,"15min":1.0900000000000001},"last":0.69299999999999995}]},{"osd":4,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78000000000000003,"15min":0.78700000000000003},"min":{"1min":0.69199999999999995,"5min":0.46500000000000002,"15min":0.45200000000000001},"max":{"1min":1.01,"5min":1.145,"15min":1.1799999999999999},"last":0.83499999999999996},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76600000000000001,"15min":0.77900000000000003},"min":{"1min":0.61399999999999999,"5min":0.44500000000000001,"15min":0.41399999999999998},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.2010000000000001},"last":0.90400000000000003}]},{"osd":5,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.625,"5min":0.499,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":1.1220000000000001,"15min":1.327},"last":0.94299999999999995},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78700000000000003,"15min":0.80500000000000005},"min":{"1min":0.70999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0449999999999999,"15min":1.381},"last":0.88300000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.59399999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.995,"5min":1.151,"15min":1.2809999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78100000000000003,"15min":0.79800000000000004},"min":{"1min":0.65500000000000003,"5min":0.51000000000000001,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":1.0609999999999999,"15min":1.153},"last":0.82299999999999995}]},{"osd":7,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.81299999999999994,"15min":0.82499999999999996},"min":{"1min":0.72399999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.038,"5min":1.107,"15min":1.3520000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.73099999999999998,"5min":0.61699999999999999,"15min":0.504},"max":{"1min":1.0169999999999999,"5min":1.1220000000000001,"15min":1.401},"last":0.86399999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575030,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50032,"kb_used_data":936,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321808,"statfs":{"total":96636764160,"available":96585531392,"internally_reserved":0,"allocated":958464,"data_stored":447195,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28599999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.57299999999999995},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48999999999999999},"min":{"1min":0.35899999999999999,"5min":0.29699999999999999,"15min":0.28599999999999998},"max":{"1min":0.73999999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.35299999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.36099999999999999,"5min":0.34699999999999998,"15min":0.32800000000000001},"max":{"1min":0.627,"5min":1.091,"15min":1.091},"last":0.622},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.36399999999999999,"15min":0.33900000000000002},"max":{"1min":0.69899999999999995,"5min":0.85899999999999999,"15min":0.91300000000000003},"last":0.42999999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.40100000000000002,"15min":0.36599999999999999},"max":{"1min":0.77100000000000002,"5min":0.96499999999999997,"15min":1.016},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.40300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82399999999999995,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.53600000000000003}]},{"osd":4,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.63500000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93100000000000005,"5min":1.27,"15min":1.4419999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.63,"5min":0.434,"15min":0.434},"max":{"1min":0.98399999999999999,"5min":1.1160000000000001,"15min":1.3380000000000001},"last":0.76800000000000002}]},{"osd":5,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.5},"max":{"1min":1.0229999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76700000000000002,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":1.2450000000000001,"15min":1.3009999999999999},"last":0.74099999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.68400000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0449999999999999,"5min":1.194,"15min":1.415},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.58799999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1,"5min":1.252,"15min":1.268},"last":0.81399999999999995}]},{"osd":7,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.73699999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0580000000000001,"5min":1.306,"15min":1.399},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84199999999999997},"min":{"1min":0.66800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0349999999999999,"5min":1.3440000000000001,"15min":1.373},"last":0.79500000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575029,"num_pgs":7,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32576,"kb_used_data":1528,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94339264,"statfs":{"total":96636764160,"available":96603406336,"internally_reserved":0,"allocated":1564672,"data_stored":1039619,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.46899999999999997,"15min":0.44600000000000001},"min":{"1min":0.32800000000000001,"5min":0.28499999999999998,"15min":0.218},"max":{"1min":0.64200000000000002,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.254},{"interface":"front","average":{"1min":0.56200000000000006,"5min":0.60999999999999999,"15min":0.53600000000000003},"min":{"1min":0.34999999999999998,"5min":0.313,"15min":0.26900000000000002},"max":{"1min":0.70999999999999996,"5min":7.8860000000000001,"15min":7.8860000000000001},"last":0.27600000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55800000000000005,"5min":0.55200000000000005,"15min":0.52500000000000002},"min":{"1min":0.377,"5min":0.35799999999999998,"15min":0.28399999999999997},"max":{"1min":0.76300000000000001,"5min":2.528,"15min":2.528},"last":0.29699999999999999},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.55100000000000005,"15min":0.55400000000000005},"min":{"1min":0.40699999999999997,"5min":0.38600000000000001,"15min":0.34000000000000002},"max":{"1min":0.83499999999999996,"5min":0.83499999999999996,"15min":1.2989999999999999},"last":0.318}]},{"osd":3,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.56899999999999995,"5min":0.55700000000000005,"15min":0.52200000000000002},"min":{"1min":0.43099999999999999,"5min":0.35199999999999998,"15min":0.29499999999999998},"max":{"1min":0.75600000000000001,"5min":3.1000000000000001,"15min":3.1000000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.69599999999999995,"15min":0.61899999999999999},"min":{"1min":0.47699999999999998,"5min":0.38400000000000001,"15min":0.38200000000000001},"max":{"1min":0.749,"5min":10.49,"15min":10.49},"last":0.36599999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81799999999999995,"5min":0.81899999999999995,"15min":0.80900000000000005},"min":{"1min":0.57399999999999995,"5min":0.51100000000000001,"15min":0.45400000000000001},"max":{"1min":1.1499999999999999,"5min":3.0659999999999998,"15min":3.7469999999999999},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.85599999999999998,"15min":0.80800000000000005},"min":{"1min":0.61499999999999999,"5min":0.499,"15min":0.499},"max":{"1min":1.0129999999999999,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.68400000000000005}]},{"osd":5,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.82199999999999995,"15min":0.80500000000000005},"min":{"1min":0.65800000000000003,"5min":0.54800000000000004,"15min":0.44700000000000001},"max":{"1min":1.0620000000000001,"5min":1.3109999999999999,"15min":1.877},"last":0.69699999999999995},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.82699999999999996,"15min":0.80400000000000005},"min":{"1min":0.64400000000000002,"5min":0.496,"15min":0.496},"max":{"1min":1.1799999999999999,"5min":1.2010000000000001,"15min":1.7450000000000001},"last":0.71299999999999997}]},{"osd":6,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.87,"15min":0.82599999999999996},"min":{"1min":0.64900000000000002,"5min":0.54700000000000004,"15min":0.47299999999999998},"max":{"1min":1.071,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.83799999999999997,"5min":0.82199999999999995,"15min":0.81799999999999995},"min":{"1min":0.57099999999999995,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.117,"5min":1.306,"15min":1.492},"last":0.73499999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:42:53 2023","interfaces":[{"interface":"back","average":{"1min":0.86299999999999999,"5min":0.83899999999999997,"15min":0.83699999999999997},"min":{"1min":0.66100000000000003,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.022,"5min":1.2549999999999999,"15min":1.6539999999999999},"last":0.58599999999999997},{"interface":"front","average":{"1min":0.878,"5min":0.83099999999999996,"15min":0.82099999999999995},"min":{"1min":0.71499999999999997,"5min":0.52900000000000003,"15min":0.48799999999999999},"max":{"1min":1.097,"5min":1.276,"15min":2.113},"last":0.66700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":8,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":9,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:43:26.480 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T04:43:26.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T04:43:26.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.670+0000 7fe9c4c73640 1 Processor -- start 2023-12-20T04:43:26.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.670+0000 7fe9c4c73640 1 -- start start 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.670+0000 7fe9c4c73640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe9c00cb8d0 0x7fe9c00cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.670+0000 7fe9c4c73640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c00cc9d0 con 0x7fe9c015ad20 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.670+0000 7fe9c4c73640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9c00c9b90 con 0x7fe9c015a940 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.670+0000 7fe9c4c73640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9c00c9e70 con 0x7fe9c00cb8d0 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bdd9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe9c015ad20 0x7fe9c00cb1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54986/0 (socket says 172.21.15.32:54986) 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bdd9b640 1 -- 172.21.15.32:0/3874506520 learned_addr learned my addr 172.21.15.32:0/3874506520 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bed9d640 1 --2- 172.21.15.32:0/3874506520 >> v2:172.21.15.39:3300/0 conn(0x7fe9c00cb8d0 0x7fe9c00cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bd59a640 1 -- 172.21.15.32:0/3874506520 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 316983394 0 0) 0x7fe9c00cc9d0 con 0x7fe9c015ad20 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bd59a640 1 -- 172.21.15.32:0/3874506520 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe9a8003530 con 0x7fe9c015ad20 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bed9d640 1 -- 172.21.15.32:0/3874506520 >> v1:172.21.15.39:6789/0 conn(0x7fe9c015a940 legacy=0x7fe9c00ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bd59a640 1 -- 172.21.15.32:0/3874506520 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3085740357 0 0) 0x7fe9c00c9b90 con 0x7fe9c015a940 2023-12-20T04:43:26.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bed9d640 1 -- 172.21.15.32:0/3874506520 >> v1:172.21.15.32:6789/0 conn(0x7fe9c015ad20 legacy=0x7fe9c00cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bed9d640 1 -- 172.21.15.32:0/3874506520 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9c00ca150 con 0x7fe9c00cb8d0 2023-12-20T04:43:26.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bed9d640 1 --2- 172.21.15.32:0/3874506520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c00cb8d0 0x7fe9c00cc380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009dc0 tx=0x7fe9b801c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=87952fe308a3573c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:26.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bd59a640 1 -- 172.21.15.32:0/3874506520 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80a5950 con 0x7fe9c00cb8d0 2023-12-20T04:43:26.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bd59a640 1 -- 172.21.15.32:0/3874506520 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b80a5ab0 con 0x7fe9c00cb8d0 2023-12-20T04:43:26.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.674+0000 7fe9bd59a640 1 -- 172.21.15.32:0/3874506520 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80a5e40 con 0x7fe9c00cb8d0 2023-12-20T04:43:26.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 -- 172.21.15.32:0/3874506520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c00cb8d0 msgr2=0x7fe9c00cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 --2- 172.21.15.32:0/3874506520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c00cb8d0 0x7fe9c00cc380 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fe9b8009dc0 tx=0x7fe9b801c870 comp rx=0 tx=0).stop 2023-12-20T04:43:26.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 -- 172.21.15.32:0/3874506520 shutdown_connections 2023-12-20T04:43:26.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 --2- 172.21.15.32:0/3874506520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c00cb8d0 0x7fe9c00cc380 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 -- 172.21.15.32:0/3874506520 >> 172.21.15.32:0/3874506520 conn(0x7fe9c006a4b0 msgr2=0x7fe9c00b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:26.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 -- 172.21.15.32:0/3874506520 shutdown_connections 2023-12-20T04:43:26.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 -- 172.21.15.32:0/3874506520 wait complete. 2023-12-20T04:43:26.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 Processor -- start 2023-12-20T04:43:26.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 -- start start 2023-12-20T04:43:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c00cb8d0 0x7fe9c014fad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe9c015a940 0x7fe9c014d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9c015ad20 0x7fe9c0151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0148de0 con 0x7fe9c015ad20 2023-12-20T04:43:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c01525e0 con 0x7fe9c00cb8d0 2023-12-20T04:43:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9c4c73640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9c0152a70 con 0x7fe9c015a940 2023-12-20T04:43:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9bed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9c015ad20 0x7fe9c0151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9bed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9c015ad20 0x7fe9c0151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43546/0 (socket says 172.21.15.32:43546) 2023-12-20T04:43:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.678+0000 7fe9bed9d640 1 -- 172.21.15.32:0/2414933388 learned_addr learned my addr 172.21.15.32:0/2414933388 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9bdd9b640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe9c015a940 0x7fe9c014d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9be59c640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c00cb8d0 0x7fe9c014fad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9bed9d640 1 -- 172.21.15.32:0/2414933388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe9c015a940 msgr2=0x7fe9c014d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9bed9d640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe9c015a940 0x7fe9c014d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9bed9d640 1 -- 172.21.15.32:0/2414933388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c00cb8d0 msgr2=0x7fe9c014fad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9bed9d640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c00cb8d0 0x7fe9c014fad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9bed9d640 1 -- 172.21.15.32:0/2414933388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9b8009a70 con 0x7fe9c015ad20 2023-12-20T04:43:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9bed9d640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9c015ad20 0x7fe9c0151180 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe9b801cd50 tx=0x7fe9b8019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9aeffd640 1 -- 172.21.15.32:0/2414933388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b80a5950 con 0x7fe9c015ad20 2023-12-20T04:43:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9aeffd640 1 -- 172.21.15.32:0/2414933388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9b8009450 con 0x7fe9c015ad20 2023-12-20T04:43:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9c4c73640 1 -- 172.21.15.32:0/2414933388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9c01519d0 con 0x7fe9c015ad20 2023-12-20T04:43:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9aeffd640 1 -- 172.21.15.32:0/2414933388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9b8003be0 con 0x7fe9c015ad20 2023-12-20T04:43:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9c4c73640 1 -- 172.21.15.32:0/2414933388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9c0151f90 con 0x7fe9c015ad20 2023-12-20T04:43:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9c4c73640 1 -- 172.21.15.32:0/2414933388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9c0152770 con 0x7fe9c015ad20 2023-12-20T04:43:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9aeffd640 1 -- 172.21.15.32:0/2414933388 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe9b8003d40 con 0x7fe9c015ad20 2023-12-20T04:43:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.682+0000 7fe9aeffd640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe9a0065b80 0x7fe9a0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.686+0000 7fe9aeffd640 1 -- 172.21.15.32:0/2414933388 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe9b811ee60 con 0x7fe9c015ad20 2023-12-20T04:43:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.686+0000 7fe9be59c640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe9a0065b80 0x7fe9a0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:26.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.686+0000 7fe9be59c640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe9a0065b80 0x7fe9a0068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0000f70 tx=0x7fe9b0009040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:26.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.690+0000 7fe9aeffd640 1 -- 172.21.15.32:0/2414933388 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe9b80ec7b0 con 0x7fe9c015ad20 2023-12-20T04:43:26.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.902+0000 7fe9c4c73640 1 -- 172.21.15.32:0/2414933388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe9c0152770 con 0x7fe9c015ad20 2023-12-20T04:43:26.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9aeffd640 1 -- 172.21.15.32:0/2414933388 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v17) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe9b80efdd0 con 0x7fe9c015ad20 2023-12-20T04:43:26.907 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T04:43:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 -- 172.21.15.32:0/2414933388 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe9a0065b80 msgr2=0x7fe9a0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe9a0065b80 0x7fe9a0068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe9b0000f70 tx=0x7fe9b0009040 comp rx=0 tx=0).stop 2023-12-20T04:43:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 -- 172.21.15.32:0/2414933388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9c015ad20 msgr2=0x7fe9c0151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9c015ad20 0x7fe9c0151180 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7fe9b801cd50 tx=0x7fe9b8019610 comp rx=0 tx=0).stop 2023-12-20T04:43:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 -- 172.21.15.32:0/2414933388 shutdown_connections 2023-12-20T04:43:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe9a0065b80 0x7fe9a0068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9c015ad20 0x7fe9c0151180 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe9c015a940 0x7fe9c014d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 --2- 172.21.15.32:0/2414933388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9c00cb8d0 0x7fe9c014fad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:26.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 -- 172.21.15.32:0/2414933388 >> 172.21.15.32:0/2414933388 conn(0x7fe9c006a4b0 msgr2=0x7fe9c00730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:26.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 -- 172.21.15.32:0/2414933388 shutdown_connections 2023-12-20T04:43:26.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:26.910+0000 7fe9c4c73640 1 -- 172.21.15.32:0/2414933388 wait complete. 2023-12-20T04:43:26.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:27.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81d0f0a640 1 Processor -- start 2023-12-20T04:43:27.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81d0f0a640 1 -- start start 2023-12-20T04:43:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81d0f0a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81cc1599e0 0x7f81cc0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81d0f0a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc0c90e0 0x7f81cc0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81d0f0a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81cc0711d0 con 0x7f81cc0b0b30 2023-12-20T04:43:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81d0f0a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81cc0c98c0 con 0x7f81cc1599e0 2023-12-20T04:43:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81d0f0a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0c9ba0 con 0x7f81cc0c90e0 2023-12-20T04:43:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81cad9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f81cc0b0b30 0x7f81cc15d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54988/0 (socket says 172.21.15.32:54988) 2023-12-20T04:43:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81cad9d640 1 -- 172.21.15.32:0/1930213979 learned_addr learned my addr 172.21.15.32:0/1930213979 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81ca59c640 1 --2- 172.21.15.32:0/1930213979 >> v2:172.21.15.39:3300/0 conn(0x7f81cc1599e0 0x7f81cc0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81c3fff640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc0c90e0 0x7f81cc0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344006391 0 0) 0x7f81cc0711d0 con 0x7f81cc0b0b30 2023-12-20T04:43:27.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81b8005740 con 0x7f81cc0b0b30 2023-12-20T04:43:27.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 867078223 0 0) 0x7f81b8005740 con 0x7f81cc0b0b30 2023-12-20T04:43:27.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.074+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc0c90e0 msgr2=0x7f81cc0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc0c90e0 0x7f81cc0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 >> v2:172.21.15.39:3300/0 conn(0x7f81cc1599e0 msgr2=0x7f81cc0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 --2- 172.21.15.32:0/1930213979 >> v2:172.21.15.39:3300/0 conn(0x7f81cc1599e0 0x7f81cc0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0c9e80 con 0x7f81cc0b0b30 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f81c4001a00 con 0x7f81cc0b0b30 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81ca59c640 1 --2- 172.21.15.32:0/1930213979 >> v2:172.21.15.39:3300/0 conn(0x7f81cc1599e0 0x7f81cc0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 >> v1:172.21.15.32:6789/0 conn(0x7f81cc0b0b30 legacy=0x7f81cc15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b8005b90 0x7f81b8005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b80069e0 0x7f81b8006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc0c90e0 0x7f81b80073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0c98c0 con 0x7f81b80069e0 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc0c9ba0 con 0x7f81b8005b90 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81b800dba0 con 0x7f81cc0c90e0 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c3fff640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b80069e0 0x7f81b8006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81ca59c640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b8005b90 0x7f81b8005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81cad9d640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc0c90e0 0x7f81b80073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c3fff640 1 -- 172.21.15.32:0/1930213979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc0c90e0 msgr2=0x7f81b80073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c3fff640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc0c90e0 0x7f81b80073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c3fff640 1 -- 172.21.15.32:0/1930213979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b8005b90 msgr2=0x7f81b8005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c3fff640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b8005b90 0x7f81b8005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c3fff640 1 -- 172.21.15.32:0/1930213979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81b8005740 con 0x7f81b80069e0 2023-12-20T04:43:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c3fff640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b80069e0 0x7f81b8006e60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009ae0 tx=0x7f81b401ca50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91170afe3b4b87ca server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.078+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b40a6460 con 0x7f81b80069e0 2023-12-20T04:43:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b40a65c0 con 0x7f81b80069e0 2023-12-20T04:43:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81c9d9b640 1 -- 172.21.15.32:0/1930213979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b40a6970 con 0x7f81b80069e0 2023-12-20T04:43:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1930213979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b80069e0 msgr2=0x7f81b8006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 --2- 172.21.15.32:0/1930213979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81b80069e0 0x7f81b8006e60 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009ae0 tx=0x7f81b401ca50 comp rx=0 tx=0).stop 2023-12-20T04:43:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81cad9d640 1 -- 172.21.15.32:0/1930213979 reap_dead start 2023-12-20T04:43:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1930213979 shutdown_connections 2023-12-20T04:43:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1930213979 >> 172.21.15.32:0/1930213979 conn(0x7f81cc06a320 msgr2=0x7f81cc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1930213979 shutdown_connections 2023-12-20T04:43:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1930213979 wait complete. 2023-12-20T04:43:27.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 Processor -- start 2023-12-20T04:43:27.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 -- start start 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b80069e0 0x7f81cc14ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15ae10 0x7f81cc15c7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc152ad0 0x7f81cc0ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc14f130 con 0x7f81cc15ae10 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc15b310 con 0x7f81b80069e0 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81d0f0a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc15b5f0 con 0x7f81cc152ad0 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15ae10 0x7f81cc15c7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15ae10 0x7f81cc15c7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43564/0 (socket says 172.21.15.32:43564) 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81c3fff640 1 -- 172.21.15.32:0/1605114093 learned_addr learned my addr 172.21.15.32:0/1605114093 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81ca59c640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b80069e0 0x7f81cc14ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.082+0000 7f81cad9d640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc152ad0 0x7f81cc0ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81c3fff640 1 -- 172.21.15.32:0/1605114093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc152ad0 msgr2=0x7f81cc0ad8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81c3fff640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc152ad0 0x7f81cc0ad8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81c3fff640 1 -- 172.21.15.32:0/1605114093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b80069e0 msgr2=0x7f81cc14ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81c3fff640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b80069e0 0x7f81cc14ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81c3fff640 1 -- 172.21.15.32:0/1605114093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc15b8d0 con 0x7f81cc15ae10 2023-12-20T04:43:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81c3fff640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15ae10 0x7f81cc15c7d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009ae0 tx=0x7f81b8007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81c2ffd640 1 -- 172.21.15.32:0/1605114093 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b40a6460 con 0x7f81cc15ae10 2023-12-20T04:43:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81c2ffd640 1 -- 172.21.15.32:0/1605114093 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81b4004220 con 0x7f81cc15ae10 2023-12-20T04:43:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1605114093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81cc14f130 con 0x7f81cc15ae10 2023-12-20T04:43:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81c2ffd640 1 -- 172.21.15.32:0/1605114093 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81b4003620 con 0x7f81cc15ae10 2023-12-20T04:43:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1605114093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc15cd10 con 0x7f81cc15ae10 2023-12-20T04:43:27.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1605114093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc04e000 con 0x7f81cc15ae10 2023-12-20T04:43:27.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.086+0000 7f81c2ffd640 1 -- 172.21.15.32:0/1605114093 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f81b4003780 con 0x7f81cc15ae10 2023-12-20T04:43:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.090+0000 7f81c2ffd640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81ac065b80 0x7f81ac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.090+0000 7f81c2ffd640 1 -- 172.21.15.32:0/1605114093 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81b411d780 con 0x7f81cc15ae10 2023-12-20T04:43:27.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.090+0000 7f81ca59c640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81ac065b80 0x7f81ac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.090+0000 7f81ca59c640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81ac065b80 0x7f81ac068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0cdb60 tx=0x7f81bc006f90 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:27.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.094+0000 7f81c2ffd640 1 -- 172.21.15.32:0/1605114093 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f81b40eaf20 con 0x7f81cc15ae10 2023-12-20T04:43:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.282+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1605114093 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81cc0cf550 con 0x7f81cc15ae10 2023-12-20T04:43:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.282+0000 7f81c2ffd640 1 -- 172.21.15.32:0/1605114093 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4706 (secure 0 0 0) 0x7f81b40ee540 con 0x7f81cc15ae10 2023-12-20T04:43:27.281 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:27.282 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5780,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4279891602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4279891602},{"type":"v1","addr":"172.21.15.39:6834","nonce":4279891602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6292,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3937321478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3937321478},{"type":"v1","addr":"172.21.15.39:6835","nonce":3937321478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/404004710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":404004710},{"type":"v1","addr":"172.21.15.32:6835","nonce":404004710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:21.971704+0000","modified":"2023-12-20T04:43:24.982163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3343723618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3343723618},{"type":"v1","addr":"172.21.15.32:6837","nonce":3343723618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-20T04:43:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2023-12-20T04:43:27.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1605114093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81ac065b80 msgr2=0x7f81ac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81ac065b80 0x7f81ac068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0cdb60 tx=0x7f81bc006f90 comp rx=0 tx=0).stop 2023-12-20T04:43:27.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1605114093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15ae10 msgr2=0x7f81cc15c7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15ae10 0x7f81cc15c7d0 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f81b4009ae0 tx=0x7f81b8007a10 comp rx=0 tx=0).stop 2023-12-20T04:43:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1605114093 shutdown_connections 2023-12-20T04:43:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81ac065b80 0x7f81ac068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc152ad0 0x7f81cc0ad8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc15ae10 0x7f81cc15c7d0 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 --2- 172.21.15.32:0/1605114093 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81b80069e0 0x7f81cc14ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1605114093 >> 172.21.15.32:0/1605114093 conn(0x7f81cc06a320 msgr2=0x7f81cc0ce230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1605114093 shutdown_connections 2023-12-20T04:43:27.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.286+0000 7f81d0f0a640 1 -- 172.21.15.32:0/1605114093 wait complete. 2023-12-20T04:43:27.548 INFO:tasks.ceph.mds.a.smithi032.stderr:2023-12-20T04:43:27.550+0000 7f54fcd15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:43:27.705 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 31, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T04:43:21.971704+0000', 'modified': '2023-12-20T04:43:24.982163+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6402}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6402': {'gid': 6402, 'name': 'a', 'rank': 0, 'incarnation': 28, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/3343723618', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 3343723618}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 3343723618}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [9], 'metadata_pool': 8, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T04:43:27.705 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T04:43:27.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-20T04:43:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.854+0000 7fbb41420640 1 Processor -- start 2023-12-20T04:43:27.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.854+0000 7fbb41420640 1 -- start start 2023-12-20T04:43:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.854+0000 7fbb41420640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbb3c06caf0 0x7fbb3c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.854+0000 7fbb41420640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c0ae140 con 0x7fbb3c15d9a0 2023-12-20T04:43:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.854+0000 7fbb41420640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb3c072170 con 0x7fbb3c1593c0 2023-12-20T04:43:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.854+0000 7fbb41420640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb3c0ae9e0 con 0x7fbb3c06caf0 2023-12-20T04:43:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.854+0000 7fbb33fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbb3c15d9a0 0x7fbb3c06c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54992/0 (socket says 172.21.15.32:54992) 2023-12-20T04:43:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.854+0000 7fbb33fff640 1 -- 172.21.15.32:0/3713064813 learned_addr learned my addr 172.21.15.32:0/3713064813 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.854+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/3713064813 >> v2:172.21.15.39:3300/0 conn(0x7fbb3c06caf0 0x7fbb3c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.858+0000 7fbb3a7fc640 1 -- 172.21.15.32:0/3713064813 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1104110569 0 0) 0x7fbb3c0ae140 con 0x7fbb3c15d9a0 2023-12-20T04:43:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.858+0000 7fbb3a7fc640 1 -- 172.21.15.32:0/3713064813 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb28003530 con 0x7fbb3c15d9a0 2023-12-20T04:43:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.858+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/3713064813 >> v1:172.21.15.39:6789/0 conn(0x7fbb3c1593c0 legacy=0x7fbb3c0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.858+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/3713064813 >> v1:172.21.15.32:6789/0 conn(0x7fbb3c15d9a0 legacy=0x7fbb3c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.858+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/3713064813 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb3c0ae390 con 0x7fbb3c06caf0 2023-12-20T04:43:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.858+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/3713064813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c06caf0 0x7fbb3c0705a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009d90 tx=0x7fbb3401c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=917b4db9a66cc4e4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.858+0000 7fbb3a7fc640 1 -- 172.21.15.32:0/3713064813 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb340a57a0 con 0x7fbb3c06caf0 2023-12-20T04:43:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.858+0000 7fbb3a7fc640 1 -- 172.21.15.32:0/3713064813 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb340a5900 con 0x7fbb3c06caf0 2023-12-20T04:43:27.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.858+0000 7fbb3a7fc640 1 -- 172.21.15.32:0/3713064813 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb340a5c90 con 0x7fbb3c06caf0 2023-12-20T04:43:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 -- 172.21.15.32:0/3713064813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c06caf0 msgr2=0x7fbb3c0705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 --2- 172.21.15.32:0/3713064813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c06caf0 0x7fbb3c0705a0 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7fbb34009d90 tx=0x7fbb3401c800 comp rx=0 tx=0).stop 2023-12-20T04:43:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 -- 172.21.15.32:0/3713064813 shutdown_connections 2023-12-20T04:43:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 --2- 172.21.15.32:0/3713064813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c06caf0 0x7fbb3c0705a0 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 -- 172.21.15.32:0/3713064813 >> 172.21.15.32:0/3713064813 conn(0x7fbb3c06a320 msgr2=0x7fbb3c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 -- 172.21.15.32:0/3713064813 shutdown_connections 2023-12-20T04:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 -- 172.21.15.32:0/3713064813 wait complete. 2023-12-20T04:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 Processor -- start 2023-12-20T04:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 -- start start 2023-12-20T04:43:27.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb3c06caf0 0x7fbb3c0cb960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3c1593c0 0x7fbb3c150180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15d9a0 0x7fbb3c14dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c0cbea0 con 0x7fbb3c1593c0 2023-12-20T04:43:27.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c14f0e0 con 0x7fbb3c15d9a0 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb41420640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb3c14f3c0 con 0x7fbb3c06caf0 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb33fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3c1593c0 0x7fbb3c150180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb33fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3c1593c0 0x7fbb3c150180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43572/0 (socket says 172.21.15.32:43572) 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb33fff640 1 -- 172.21.15.32:0/1919642298 learned_addr learned my addr 172.21.15.32:0/1919642298 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb33fff640 1 -- 172.21.15.32:0/1919642298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb3c06caf0 msgr2=0x7fbb3c0cb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb33fff640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb3c06caf0 0x7fbb3c0cb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb33fff640 1 -- 172.21.15.32:0/1919642298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15d9a0 msgr2=0x7fbb3c14dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb33fff640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15d9a0 0x7fbb3c14dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb33fff640 1 -- 172.21.15.32:0/1919642298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb34009a70 con 0x7fbb3c1593c0 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.862+0000 7fbb33fff640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3c1593c0 0x7fbb3c150180 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c4d0 tx=0x7fbb2400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.866+0000 7fbb337fe640 1 -- 172.21.15.32:0/1919642298 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24005cf0 con 0x7fbb3c1593c0 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.866+0000 7fbb337fe640 1 -- 172.21.15.32:0/1919642298 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb24005e50 con 0x7fbb3c1593c0 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.866+0000 7fbb337fe640 1 -- 172.21.15.32:0/1919642298 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb24007270 con 0x7fbb3c1593c0 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.866+0000 7fbb41420640 1 -- 172.21.15.32:0/1919642298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb3c14f6a0 con 0x7fbb3c1593c0 2023-12-20T04:43:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.866+0000 7fbb41420640 1 -- 172.21.15.32:0/1919642298 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c14fc60 con 0x7fbb3c1593c0 2023-12-20T04:43:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.866+0000 7fbb337fe640 1 -- 172.21.15.32:0/1919642298 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbb240073d0 con 0x7fbb3c1593c0 2023-12-20T04:43:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb337fe640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb1c065b80 0x7fbb1c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb337fe640 1 -- 172.21.15.32:0/1919642298 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb24082820 con 0x7fbb3c1593c0 2023-12-20T04:43:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 Processor -- start 2023-12-20T04:43:27.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3affd640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb1c065b80 0x7fbb1c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 -- start start 2023-12-20T04:43:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3affd640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb1c065b80 0x7fbb1c068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c000fc0 tx=0x7fbb2c0079b0 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88f7f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f990 0x55da88f34c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88f355a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da88d58c90 con 0x55da88f351c0 2023-12-20T04:43:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da88d8a220 con 0x55da88f7f070 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da88d877f0 con 0x55da88f7f990 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88f355a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88f355a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43582/0 (socket says 172.21.15.32:43582) 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/2051298954 learned_addr learned my addr 172.21.15.32:0/2051298954 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3affd640 1 --2- 172.21.15.32:0/2051298954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88f7f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb33fff640 1 --2- 172.21.15.32:0/2051298954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f990 0x55da88f34c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/2051298954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f990 msgr2=0x55da88f34c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/2051298954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f990 0x55da88f34c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/2051298954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 msgr2=0x55da88f7f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/2051298954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88f7f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/2051298954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da88a7f890 con 0x55da88f351c0 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3affd640 1 --2- 172.21.15.32:0/2051298954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88f7f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:27.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/2051298954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88f355a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbb34019410 tx=0x7fbb34004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6d0711422214834 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb30ff9640 1 -- 172.21.15.32:0/2051298954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb340bd070 con 0x55da88f351c0 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb30ff9640 1 -- 172.21.15.32:0/2051298954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb340a55a0 con 0x55da88f351c0 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb30ff9640 1 -- 172.21.15.32:0/2051298954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb340a5cb0 con 0x55da88f351c0 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 -- 172.21.15.32:0/2051298954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 msgr2=0x55da88f355a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 --2- 172.21.15.32:0/2051298954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88f355a0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fbb34019410 tx=0x7fbb34004a50 comp rx=0 tx=0).stop 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 -- 172.21.15.32:0/2051298954 shutdown_connections 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 --2- 172.21.15.32:0/2051298954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88f355a0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 --2- 172.21.15.32:0/2051298954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f990 0x55da88f34c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 --2- 172.21.15.32:0/2051298954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88f7f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 -- 172.21.15.32:0/2051298954 >> 172.21.15.32:0/2051298954 conn(0x55da88f3b950 msgr2=0x55da88f33560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 -- 172.21.15.32:0/2051298954 shutdown_connections 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.870+0000 7fbb44779000 1 -- 172.21.15.32:0/2051298954 wait complete. 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 -- start start 2023-12-20T04:43:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 Processor -- start 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f351c0 0x55da88f3a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f070 0x55da88f41080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88f415c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da88d830e0 con 0x55da88f7f990 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da88d22310 con 0x55da88f351c0 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da88d40770 con 0x55da88f7f070 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 10 client.0 _ll_register_callbacks cb 0x55da88f20b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb3b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88f415c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb3b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88f415c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43592/0 (socket says 172.21.15.32:43592) 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb3affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f351c0 0x55da88f3a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/3744880991 learned_addr learned my addr 172.21.15.32:0/3744880991 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb33fff640 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f070 0x55da88f41080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/3744880991 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f070 msgr2=0x55da88f41080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f070 0x55da88f41080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/3744880991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f351c0 msgr2=0x55da88f3a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f351c0 0x55da88f3a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/3744880991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb3c0ae390 con 0x55da88f7f990 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88f415c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbb340bb040 tx=0x7fbb3401b040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e145e3d5e1cdd56c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb19ffb640 1 -- 172.21.15.32:0/3744880991 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb340bd040 con 0x55da88f7f990 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 -- 172.21.15.32:0/3744880991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da88d52c10 con 0x55da88f7f990 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb19ffb640 1 -- 172.21.15.32:0/3744880991 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb340a6d80 con 0x55da88f7f990 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb19ffb640 1 -- 172.21.15.32:0/3744880991 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb340039a0 con 0x55da88f7f990 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 10 client.6531 fetch_fsmap learned FSMap version 31 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 -- 172.21.15.32:0/3744880991 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55da88d48260 con 0x55da88f7f990 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb19ffb640 1 -- 172.21.15.32:0/3744880991 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da88d52c10 con 0x55da88f7f990 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb19ffb640 1 -- 172.21.15.32:0/3744880991 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbb34003b00 con 0x55da88f7f990 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 10 client.6531 fetch_fsmap finished waiting for FSMap version 31 2023-12-20T04:43:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 10 client.6531 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T04:43:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 20 client.6531 populate_metadata read hostname 'smithi032' 2023-12-20T04:43:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.32:6836/3343723618,v1:172.21.15.32:6837/3343723618] conn(0x55da88f868d0 0x55da88f86cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 4 client.6531 mds_command: new command op to 6402 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T04:43:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb44779000 1 -- 172.21.15.32:0/3744880991 --> [v2:172.21.15.32:6836/3343723618,v1:172.21.15.32:6837/3343723618] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55da88d81a30 con 0x55da88f868d0 2023-12-20T04:43:27.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.874+0000 7fbb3affd640 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.32:6836/3343723618,v1:172.21.15.32:6837/3343723618] conn(0x55da88f868d0 0x55da88f86cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb3affd640 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.32:6836/3343723618,v1:172.21.15.32:6837/3343723618] conn(0x55da88f868d0 0x55da88f86cb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5030749479ab0947 server_cookie=34aefa9b44d2bcca in_seq=0 out_seq=0 2023-12-20T04:43:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb19ffb640 10 client.6531 ms_handle_connect on v2:172.21.15.32:6836/3343723618 2023-12-20T04:43:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb19ffb640 1 -- 172.21.15.32:0/3744880991 <== mds.0 v2:172.21.15.32:6836/3343723618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55da88d81a30 con 0x55da88f868d0 2023-12-20T04:43:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb19ffb640 10 client.6531 handle_command_reply: tid=0 2023-12-20T04:43:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 1 client.6531 shutdown 2023-12-20T04:43:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 1 -- 172.21.15.32:0/3744880991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 msgr2=0x55da88f415c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88f415c0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fbb340bb040 tx=0x7fbb3401b040 comp rx=0 tx=0).stop 2023-12-20T04:43:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 1 -- 172.21.15.32:0/3744880991 shutdown_connections 2023-12-20T04:43:27.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.32:6836/3343723618,v1:172.21.15.32:6837/3343723618] conn(0x55da88f868d0 0x55da88f86cb0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88f415c0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f070 0x55da88f41080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 1 --2- 172.21.15.32:0/3744880991 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f351c0 0x55da88f3a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb19ffb640 0 client.6531 ms_handle_reset on v2:172.21.15.32:6836/3343723618 2023-12-20T04:43:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 1 -- 172.21.15.32:0/3744880991 >> 172.21.15.32:0/3744880991 conn(0x55da88f3b950 msgr2=0x55da88f33c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 1 -- 172.21.15.32:0/3744880991 shutdown_connections 2023-12-20T04:43:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 1 -- 172.21.15.32:0/3744880991 wait complete. 2023-12-20T04:43:27.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.878+0000 7fbb44779000 20 client.6531 trim_cache size 0 max 16384 2023-12-20T04:43:27.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 Processor -- start 2023-12-20T04:43:27.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 -- start start 2023-12-20T04:43:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f351c0 0x55da88fd72f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88fd7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88fd7d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da88a40940 con 0x55da88f7f990 2023-12-20T04:43:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da88f3a770 con 0x55da88f7f070 2023-12-20T04:43:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da88d50a80 con 0x55da88f351c0 2023-12-20T04:43:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb3b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88fd7d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb3b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88fd7d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43604/0 (socket says 172.21.15.32:43604) 2023-12-20T04:43:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/1019951201 learned_addr learned my addr 172.21.15.32:0/1019951201 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb33fff640 1 --2- 172.21.15.32:0/1019951201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88fd7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb3affd640 1 --2- 172.21.15.32:0/1019951201 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f351c0 0x55da88fd72f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/1019951201 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f351c0 msgr2=0x55da88fd72f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/1019951201 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f351c0 0x55da88fd72f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/1019951201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 msgr2=0x55da88fd7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/1019951201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88fd7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb3b7fe640 1 -- 172.21.15.32:0/1019951201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55da88d2e3b0 con 0x55da88f7f990 2023-12-20T04:43:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/1019951201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88fd7d70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbb34004a90 tx=0x7fbb340be000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf17db3c72182631 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb317fa640 1 -- 172.21.15.32:0/1019951201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb340bd070 con 0x55da88f7f990 2023-12-20T04:43:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb317fa640 1 -- 172.21.15.32:0/1019951201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb34003ba0 con 0x55da88f7f990 2023-12-20T04:43:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb317fa640 1 -- 172.21.15.32:0/1019951201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb340a5cb0 con 0x55da88f7f990 2023-12-20T04:43:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 -- 172.21.15.32:0/1019951201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 msgr2=0x55da88fd7d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 --2- 172.21.15.32:0/1019951201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88fd7d70 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fbb34004a90 tx=0x7fbb340be000 comp rx=0 tx=0).stop 2023-12-20T04:43:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 -- 172.21.15.32:0/1019951201 shutdown_connections 2023-12-20T04:43:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 --2- 172.21.15.32:0/1019951201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f7f990 0x55da88fd7d70 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 --2- 172.21.15.32:0/1019951201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88fd7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 --2- 172.21.15.32:0/1019951201 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f351c0 0x55da88fd72f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 -- 172.21.15.32:0/1019951201 >> 172.21.15.32:0/1019951201 conn(0x55da88f3b950 msgr2=0x55da88f40f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 -- 172.21.15.32:0/1019951201 shutdown_connections 2023-12-20T04:43:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.894+0000 7fbb44779000 1 -- 172.21.15.32:0/1019951201 wait complete. 2023-12-20T04:43:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 -- start start 2023-12-20T04:43:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 Processor -- start 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88fbfb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88fc0090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f990 0x55da88fc05d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da88d808f0 con 0x55da88f351c0 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55da88d2b860 con 0x55da88f7f070 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55da88dc6ce0 con 0x55da88f7f990 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 10 client.0 _ll_register_callbacks cb 0x55da88d31640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb3affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88fbfb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb3affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88fbfb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43608/0 (socket says 172.21.15.32:43608) 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb3affd640 1 -- 172.21.15.32:0/593439042 learned_addr learned my addr 172.21.15.32:0/593439042 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb33fff640 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88fc0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f990 0x55da88fc05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb3affd640 1 -- 172.21.15.32:0/593439042 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f990 msgr2=0x55da88fc05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb3affd640 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f990 0x55da88fc05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb3affd640 1 -- 172.21.15.32:0/593439042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 msgr2=0x55da88fc0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb3affd640 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88fc0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb3affd640 1 -- 172.21.15.32:0/593439042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55da88d48260 con 0x55da88f351c0 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb3b7fe640 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f990 0x55da88fc05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb3affd640 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88fbfb50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c001f10 tx=0x7fbb2c0011f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c39236705a3a807 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb19ffb640 1 -- 172.21.15.32:0/593439042 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb2c025af0 con 0x55da88f351c0 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb19ffb640 1 -- 172.21.15.32:0/593439042 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbb2c025c50 con 0x55da88f351c0 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 -- 172.21.15.32:0/593439042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55da88d48e00 con 0x55da88f351c0 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb19ffb640 1 -- 172.21.15.32:0/593439042 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=31) v2 ==== 24+0+0 (secure 0 0 0) 0x7fbb2c025e10 con 0x55da88f351c0 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 10 client.6537 fetch_fsmap learned FSMap version 31 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 -- 172.21.15.32:0/593439042 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=31,monmap=2+}) v3 -- 0x55da88a7fcc0 con 0x55da88f351c0 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb19ffb640 1 -- 172.21.15.32:0/593439042 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55da88d48e00 con 0x55da88f351c0 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb19ffb640 1 -- 172.21.15.32:0/593439042 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 31) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fbb2c003650 con 0x55da88f351c0 2023-12-20T04:43:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 10 client.6537 fetch_fsmap finished waiting for FSMap version 31 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 10 client.6537 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 20 client.6537 populate_metadata read hostname 'smithi032' 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.32:6836/3343723618,v1:172.21.15.32:6837/3343723618] conn(0x55da88fc0b10 0x55da88fc0ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 4 client.6537 mds_command: new command op to 6402 tid=0[{"prefix": "status"}] 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb44779000 1 -- 172.21.15.32:0/593439042 --> [v2:172.21.15.32:6836/3343723618,v1:172.21.15.32:6837/3343723618] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55da88d7c160 con 0x55da88fc0b10 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb33fff640 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.32:6836/3343723618,v1:172.21.15.32:6837/3343723618] conn(0x55da88fc0b10 0x55da88fc0ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb33fff640 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.32:6836/3343723618,v1:172.21.15.32:6837/3343723618] conn(0x55da88fc0b10 0x55da88fc0ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=17b52d1b107434c6 server_cookie=3661e9eb5301dba8 in_seq=0 out_seq=0 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.898+0000 7fbb19ffb640 10 client.6537 ms_handle_connect on v2:172.21.15.32:6836/3343723618 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb19ffb640 1 -- 172.21.15.32:0/593439042 <== mds.0 v2:172.21.15.32:6836/3343723618 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+331 (crc 0 0 0) 0x55da88d7c160 con 0x55da88fc0b10 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb19ffb640 10 client.6537 handle_command_reply: tid=0 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 1 client.6537 shutdown 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 1 -- 172.21.15.32:0/593439042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 msgr2=0x55da88fbfb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88fbfb50 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c001f10 tx=0x7fbb2c0011f0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 1 -- 172.21.15.32:0/593439042 shutdown_connections 2023-12-20T04:43:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.32:6836/3343723618,v1:172.21.15.32:6837/3343723618] conn(0x55da88fc0b10 0x55da88fc0ef0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55da88f7f990 0x55da88fc05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55da88f7f070 0x55da88fc0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 1 --2- 172.21.15.32:0/593439042 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55da88f351c0 0x55da88fbfb50 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb19ffb640 0 client.6537 ms_handle_reset on v2:172.21.15.32:6836/3343723618 2023-12-20T04:43:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 1 -- 172.21.15.32:0/593439042 >> 172.21.15.32:0/593439042 conn(0x55da88f3b950 msgr2=0x55da88f7c060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 1 -- 172.21.15.32:0/593439042 shutdown_connections 2023-12-20T04:43:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 1 -- 172.21.15.32:0/593439042 wait complete. 2023-12-20T04:43:27.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb44779000 20 client.6537 trim_cache size 0 max 16384 2023-12-20T04:43:27.903 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:43:27.903 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T04:43:27.903 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T04:43:27.904 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6402, 2023-12-20T04:43:27.904 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T04:43:27.904 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T04:43:27.904 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T04:43:27.904 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9293011519999999, 2023-12-20T04:43:27.904 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 31, 2023-12-20T04:43:27.904 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 79, 2023-12-20T04:43:27.904 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 79, 2023-12-20T04:43:27.904 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 11.191232638000001 2023-12-20T04:43:27.904 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:43:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 -- 172.21.15.32:0/1919642298 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb1c065b80 msgr2=0x7fbb1c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb1c065b80 0x7fbb1c068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fbb2c000fc0 tx=0x7fbb2c0079b0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 -- 172.21.15.32:0/1919642298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3c1593c0 msgr2=0x7fbb3c150180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3c1593c0 0x7fbb3c150180 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7fbb2400c4d0 tx=0x7fbb2400c990 comp rx=0 tx=0).stop 2023-12-20T04:43:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 -- 172.21.15.32:0/1919642298 shutdown_connections 2023-12-20T04:43:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbb1c065b80 0x7fbb1c068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb3c15d9a0 0x7fbb3c14dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb3c1593c0 0x7fbb3c150180 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 --2- 172.21.15.32:0/1919642298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb3c06caf0 0x7fbb3c0cb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:27.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 -- 172.21.15.32:0/1919642298 >> 172.21.15.32:0/1919642298 conn(0x7fbb3c06a320 msgr2=0x7fbb3c06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:27.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 -- 172.21.15.32:0/1919642298 shutdown_connections 2023-12-20T04:43:27.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:27.902+0000 7fbb41420640 1 -- 172.21.15.32:0/1919642298 wait complete. 2023-12-20T04:43:27.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T04:43:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bfdd09640 1 Processor -- start 2023-12-20T04:43:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bfdd09640 1 -- start start 2023-12-20T04:43:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bfdd09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4bf806ce60 0x7f4bf815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bfdd09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4bf815f110 0x7f4bf815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bfdd09640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4bf815bbd0 con 0x7f4bf80af5e0 2023-12-20T04:43:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bfdd09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4bf815beb0 con 0x7f4bf815f110 2023-12-20T04:43:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bfdd09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf815e910 con 0x7f4bf806ce60 2023-12-20T04:43:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bf7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4bf815f110 0x7f4bf815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bf7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4bf815f110 0x7f4bf815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43622/0 (socket says 172.21.15.32:43622) 2023-12-20T04:43:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bf7fff640 1 -- 172.21.15.32:0/2052503313 learned_addr learned my addr 172.21.15.32:0/2052503313 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bf6ffd640 1 --2- 172.21.15.32:0/2052503313 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4bf806ce60 0x7f4bf815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:28.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bf7fff640 1 -- 172.21.15.32:0/2052503313 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4bf806ce60 msgr2=0x7f4bf815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:28.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.066+0000 7f4bf7fff640 1 --2- 172.21.15.32:0/2052503313 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4bf806ce60 0x7f4bf815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:28.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bf7fff640 1 -- 172.21.15.32:0/2052503313 >> v1:172.21.15.39:6789/0 conn(0x7f4bf80af5e0 legacy=0x7f4bf80af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:28.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bf7fff640 1 -- 172.21.15.32:0/2052503313 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf815ebf0 con 0x7f4bf815f110 2023-12-20T04:43:28.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bf67fc640 1 -- 172.21.15.32:0/2052503313 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 987200014 0 0) 0x7f4bf815bbd0 con 0x7f4bf80af5e0 2023-12-20T04:43:28.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bf7fff640 1 --2- 172.21.15.32:0/2052503313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf815f110 0x7f4bf815c440 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009e50 tx=0x7f4bf001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96c1df50495324f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bf67fc640 1 -- 172.21.15.32:0/2052503313 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf00a57e0 con 0x7f4bf815f110 2023-12-20T04:43:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bf67fc640 1 -- 172.21.15.32:0/2052503313 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bf00a5940 con 0x7f4bf815f110 2023-12-20T04:43:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bf67fc640 1 -- 172.21.15.32:0/2052503313 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bf00a5cd0 con 0x7f4bf815f110 2023-12-20T04:43:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bfdd09640 1 -- 172.21.15.32:0/2052503313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf815f110 msgr2=0x7f4bf815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bfdd09640 1 --2- 172.21.15.32:0/2052503313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf815f110 0x7f4bf815c440 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f4bf0009e50 tx=0x7f4bf001c980 comp rx=0 tx=0).stop 2023-12-20T04:43:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bfdd09640 1 -- 172.21.15.32:0/2052503313 shutdown_connections 2023-12-20T04:43:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bfdd09640 1 --2- 172.21.15.32:0/2052503313 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf815f110 0x7f4bf815c440 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bfdd09640 1 --2- 172.21.15.32:0/2052503313 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4bf806ce60 0x7f4bf815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bfdd09640 1 -- 172.21.15.32:0/2052503313 >> 172.21.15.32:0/2052503313 conn(0x7f4bf806a320 msgr2=0x7f4bf80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bfdd09640 1 -- 172.21.15.32:0/2052503313 shutdown_connections 2023-12-20T04:43:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.070+0000 7f4bfdd09640 1 -- 172.21.15.32:0/2052503313 wait complete. 2023-12-20T04:43:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bfdd09640 1 Processor -- start 2023-12-20T04:43:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bfdd09640 1 -- start start 2023-12-20T04:43:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bfdd09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bf806ce60 0x7f4bf8151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bfdd09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80af5e0 0x7f4bf8156d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bfdd09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4bf815f110 0x7f4bf81523c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bfdd09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf80c5020 con 0x7f4bf80af5e0 2023-12-20T04:43:28.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bfdd09640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf80c8570 con 0x7f4bf806ce60 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bfdd09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4bf80c7220 con 0x7f4bf815f110 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bf6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80af5e0 0x7f4bf8156d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bf6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80af5e0 0x7f4bf8156d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43624/0 (socket says 172.21.15.32:43624) 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bf6ffd640 1 -- 172.21.15.32:0/3370872679 learned_addr learned my addr 172.21.15.32:0/3370872679 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bf77fe640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bf806ce60 0x7f4bf8151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bf7fff640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4bf815f110 0x7f4bf81523c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bf6ffd640 1 -- 172.21.15.32:0/3370872679 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4bf815f110 msgr2=0x7f4bf81523c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bf6ffd640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4bf815f110 0x7f4bf81523c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bf6ffd640 1 -- 172.21.15.32:0/3370872679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bf806ce60 msgr2=0x7f4bf8151ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bf6ffd640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bf806ce60 0x7f4bf8151ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bf6ffd640 1 -- 172.21.15.32:0/3370872679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4bf0009b30 con 0x7f4bf80af5e0 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bf6ffd640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80af5e0 0x7f4bf8156d20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00e840 tx=0x7f4bec00ed00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:28.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bdffff640 1 -- 172.21.15.32:0/3370872679 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0094f0 con 0x7f4bf80af5e0 2023-12-20T04:43:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bdffff640 1 -- 172.21.15.32:0/3370872679 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4bec004020 con 0x7f4bf80af5e0 2023-12-20T04:43:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bdffff640 1 -- 172.21.15.32:0/3370872679 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4bec0043b0 con 0x7f4bf80af5e0 2023-12-20T04:43:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.074+0000 7f4bfdd09640 1 -- 172.21.15.32:0/3370872679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4bf8157260 con 0x7f4bf80af5e0 2023-12-20T04:43:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.078+0000 7f4bfdd09640 1 -- 172.21.15.32:0/3370872679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4bf80c74a0 con 0x7f4bf80af5e0 2023-12-20T04:43:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.078+0000 7f4bfdd09640 1 -- 172.21.15.32:0/3370872679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4bf804e000 con 0x7f4bf80af5e0 2023-12-20T04:43:28.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.078+0000 7f4bdffff640 1 -- 172.21.15.32:0/3370872679 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4bec00c940 con 0x7f4bf80af5e0 2023-12-20T04:43:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.078+0000 7f4bdffff640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4be4065bb0 0x7f4be4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.078+0000 7f4bdffff640 1 -- 172.21.15.32:0/3370872679 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4bec083360 con 0x7f4bf80af5e0 2023-12-20T04:43:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.078+0000 7f4bf77fe640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4be4065bb0 0x7f4be4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:28.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.078+0000 7f4bf77fe640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4be4065bb0 0x7f4be4068070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4be8001860 tx=0x7f4be8009040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:28.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.082+0000 7f4bdffff640 1 -- 172.21.15.32:0/3370872679 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4bec050c20 con 0x7f4bf80af5e0 2023-12-20T04:43:28.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 -- 172.21.15.32:0/3370872679 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f4bf81659c0 con 0x7f4bf80af5e0 2023-12-20T04:43:28.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bdffff640 1 -- 172.21.15.32:0/3370872679 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v31) v1 ==== 53+0+73 (secure 0 0 0) 0x7f4bec054240 con 0x7f4bf80af5e0 2023-12-20T04:43:28.273 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T04:43:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 -- 172.21.15.32:0/3370872679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4be4065bb0 msgr2=0x7f4be4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4be4065bb0 0x7f4be4068070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f4be8001860 tx=0x7f4be8009040 comp rx=0 tx=0).stop 2023-12-20T04:43:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 -- 172.21.15.32:0/3370872679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80af5e0 msgr2=0x7f4bf8156d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80af5e0 0x7f4bf8156d20 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7f4bec00e840 tx=0x7f4bec00ed00 comp rx=0 tx=0).stop 2023-12-20T04:43:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 -- 172.21.15.32:0/3370872679 shutdown_connections 2023-12-20T04:43:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4be4065bb0 0x7f4be4068070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4bf815f110 0x7f4bf81523c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:28.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4bf80af5e0 0x7f4bf8156d20 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:28.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 --2- 172.21.15.32:0/3370872679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4bf806ce60 0x7f4bf8151ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:28.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 -- 172.21.15.32:0/3370872679 >> 172.21.15.32:0/3370872679 conn(0x7f4bf806a320 msgr2=0x7f4bf81594c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:28.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 -- 172.21.15.32:0/3370872679 shutdown_connections 2023-12-20T04:43:28.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:28.274+0000 7f4bfdd09640 1 -- 172.21.15.32:0/3370872679 wait complete. 2023-12-20T04:43:28.295 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T04:43:28.295 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T04:43:28.295 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T04:43:28.295 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T04:43:28.295 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T04:43:28.295 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T04:43:28.295 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T04:43:28.295 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T04:43:28.295 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:43:28.296 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:28.296 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T04:43:28.306 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41451sec preferred_lft 41451sec 2023-12-20T04:43:28.307 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T04:43:28.308 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T04:43:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:43:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T04:43:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:43:28.309 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:43:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T04:43:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T04:43:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:43:28.309 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:43:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T04:43:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T04:43:28.309 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:43:28.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:28.310 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T04:43:28.374 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T04:43:28.376 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:28.376 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T04:43:28.429 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T04:43:28.429 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T04:43:28.429 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T04:43:28.429 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T04:43:28.429 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T04:43:28.429 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T04:43:28.430 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:28.430 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:43:28.430 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:43:28.430 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T04:43:28.430 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T04:43:28.430 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T04:43:28.430 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:43:28.523 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:28.524 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T04:43:28.533 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T04:43:28.534 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T04:43:28.534 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:43:28.534 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:28.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T04:43:28.600 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:43:28.600 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T04:43:28.655 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:43:28.656 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:43:28.656 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:43:28.706 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T04:43:28.707 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:28.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:43:28.803 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:43:28.803 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:43:28.816 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:43:28.816 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:43:28.816 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T04:43:28.816 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T04:43:28.816 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T04:43:28.820 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T04:43:28.820 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:28.820 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:43:28.842 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:28.842 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:43:28.901 INFO:teuthology.orchestra.run.smithi032.stdout:client6433 2023-12-20T04:43:28.902 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:43:28.902 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:28.952 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T04:43:29.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37eb441640 1 Processor -- start 2023-12-20T04:43:29.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37eb441640 1 -- start start 2023-12-20T04:43:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37eb441640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37e4164ef0 0x7f37e40c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37eb441640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e40ae3f0 con 0x7f37e40ae010 2023-12-20T04:43:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37eb441640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37e4161800 con 0x7f37e406cf10 2023-12-20T04:43:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37eb441640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37e4165380 con 0x7f37e4164ef0 2023-12-20T04:43:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37e91dd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f37e40ae010 0x7f37e4071470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54630/0 (socket says 172.21.15.32:54630) 2023-12-20T04:43:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37e91dd640 1 -- 172.21.15.32:0/587391908 learned_addr learned my addr 172.21.15.32:0/587391908 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37e99de640 1 --2- 172.21.15.32:0/587391908 >> v2:172.21.15.32:3300/0 conn(0x7f37e4164ef0 0x7f37e40c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37e99de640 1 -- 172.21.15.32:0/587391908 >> v1:172.21.15.39:6789/0 conn(0x7f37e406cf10 legacy=0x7f37e40c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37dbfff640 1 -- 172.21.15.32:0/587391908 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443066657 0 0) 0x7f37e40ae3f0 con 0x7f37e40ae010 2023-12-20T04:43:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37e99de640 1 -- 172.21.15.32:0/587391908 >> v1:172.21.15.32:6789/0 conn(0x7f37e40ae010 legacy=0x7f37e4071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:29.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.142+0000 7f37e99de640 1 -- 172.21.15.32:0/587391908 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37e4165660 con 0x7f37e4164ef0 2023-12-20T04:43:29.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.146+0000 7f37e99de640 1 --2- 172.21.15.32:0/587391908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e4164ef0 0x7f37e40c9bb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009d90 tx=0x7f37dc01c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4efc71af0326c134 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.146+0000 7f37dbfff640 1 -- 172.21.15.32:0/587391908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0a5910 con 0x7f37e4164ef0 2023-12-20T04:43:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.146+0000 7f37dbfff640 1 -- 172.21.15.32:0/587391908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37dc0a5a70 con 0x7f37e4164ef0 2023-12-20T04:43:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.146+0000 7f37dbfff640 1 -- 172.21.15.32:0/587391908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37dc0a5e00 con 0x7f37e4164ef0 2023-12-20T04:43:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.146+0000 7f37eb441640 1 -- 172.21.15.32:0/587391908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e4164ef0 msgr2=0x7f37e40c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.146+0000 7f37eb441640 1 --2- 172.21.15.32:0/587391908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e4164ef0 0x7f37e40c9bb0 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7f37dc009d90 tx=0x7f37dc01c850 comp rx=0 tx=0).stop 2023-12-20T04:43:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.146+0000 7f37eb441640 1 -- 172.21.15.32:0/587391908 shutdown_connections 2023-12-20T04:43:29.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.146+0000 7f37eb441640 1 --2- 172.21.15.32:0/587391908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e4164ef0 0x7f37e40c9bb0 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.146+0000 7f37eb441640 1 -- 172.21.15.32:0/587391908 >> 172.21.15.32:0/587391908 conn(0x7f37e406a320 msgr2=0x7f37e40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:29.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.146+0000 7f37eb441640 1 -- 172.21.15.32:0/587391908 shutdown_connections 2023-12-20T04:43:29.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.146+0000 7f37eb441640 1 -- 172.21.15.32:0/587391908 wait complete. 2023-12-20T04:43:29.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37eb441640 1 Processor -- start 2023-12-20T04:43:29.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37eb441640 1 -- start start 2023-12-20T04:43:29.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37eb441640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e40ae010 0x7f37e40c60b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37eb441640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e40c65f0 0x7f37e4158b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37eb441640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37e41590b0 0x7f37e40c7ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37eb441640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40c7710 con 0x7f37e40c65f0 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37eb441640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40c6b00 con 0x7f37e40ae010 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37eb441640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37e40c6de0 con 0x7f37e41590b0 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e89dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e40c65f0 0x7f37e4158b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e89dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e40c65f0 0x7f37e4158b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43198/0 (socket says 172.21.15.32:43198) 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e89dc640 1 -- 172.21.15.32:0/308142428 learned_addr learned my addr 172.21.15.32:0/308142428 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e99de640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37e41590b0 0x7f37e40c7ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e91dd640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e40ae010 0x7f37e40c60b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e89dc640 1 -- 172.21.15.32:0/308142428 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37e41590b0 msgr2=0x7f37e40c7ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e89dc640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37e41590b0 0x7f37e40c7ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e89dc640 1 -- 172.21.15.32:0/308142428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e40ae010 msgr2=0x7f37e40c60b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e89dc640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e40ae010 0x7f37e40c60b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e89dc640 1 -- 172.21.15.32:0/308142428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37dc009a70 con 0x7f37e40c65f0 2023-12-20T04:43:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e91dd640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e40ae010 0x7f37e40c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e99de640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37e41590b0 0x7f37e40c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37e89dc640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e40c65f0 0x7f37e4158b70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f37d400c450 tx=0x7f37d400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37d9ffb640 1 -- 172.21.15.32:0/308142428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4012830 con 0x7f37e40c65f0 2023-12-20T04:43:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37d9ffb640 1 -- 172.21.15.32:0/308142428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37d40129b0 con 0x7f37e40c65f0 2023-12-20T04:43:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37d9ffb640 1 -- 172.21.15.32:0/308142428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37d4012d60 con 0x7f37e40c65f0 2023-12-20T04:43:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37eb441640 1 -- 172.21.15.32:0/308142428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37e40c70c0 con 0x7f37e40c65f0 2023-12-20T04:43:29.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37eb441640 1 -- 172.21.15.32:0/308142428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37e415b040 con 0x7f37e40c65f0 2023-12-20T04:43:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.150+0000 7f37eb441640 1 -- 172.21.15.32:0/308142428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37e40c7880 con 0x7f37e40c65f0 2023-12-20T04:43:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.154+0000 7f37d9ffb640 1 -- 172.21.15.32:0/308142428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f37d401c460 con 0x7f37e40c65f0 2023-12-20T04:43:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.154+0000 7f37d9ffb640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37c8065bb0 0x7f37c8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.154+0000 7f37d9ffb640 1 -- 172.21.15.32:0/308142428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f37d4083120 con 0x7f37e40c65f0 2023-12-20T04:43:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.154+0000 7f37e91dd640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37c8065bb0 0x7f37c8068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.154+0000 7f37e91dd640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37c8065bb0 0x7f37c8068070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f37e40c5ea0 tx=0x7f37e0009040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:29.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.158+0000 7f37d9ffb640 1 -- 172.21.15.32:0/308142428 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f37d40509c0 con 0x7f37e40c65f0 2023-12-20T04:43:29.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37eb441640 1 -- 172.21.15.32:0/308142428 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f37e40af570 con 0x7f37c8065bb0 2023-12-20T04:43:29.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37d9ffb640 1 -- 172.21.15.32:0/308142428 <== mgr.6183 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f37e0001500 con 0x7f37c8065bb0 2023-12-20T04:43:29.337 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:43:29.337 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T04:43:29.338 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:29.338 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:43:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37eb441640 1 -- 172.21.15.32:0/308142428 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37c8065bb0 msgr2=0x7f37c8068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37eb441640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37c8065bb0 0x7f37c8068070 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f37e40c5ea0 tx=0x7f37e0009040 comp rx=0 tx=0).stop 2023-12-20T04:43:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37eb441640 1 -- 172.21.15.32:0/308142428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e40c65f0 msgr2=0x7f37e4158b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37eb441640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e40c65f0 0x7f37e4158b70 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7f37d400c450 tx=0x7f37d400c910 comp rx=0 tx=0).stop 2023-12-20T04:43:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37eb441640 1 -- 172.21.15.32:0/308142428 shutdown_connections 2023-12-20T04:43:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37eb441640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37c8065bb0 0x7f37c8068070 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37eb441640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37e41590b0 0x7f37e40c7ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37eb441640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37e40c65f0 0x7f37e4158b70 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37eb441640 1 --2- 172.21.15.32:0/308142428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37e40ae010 0x7f37e40c60b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.338+0000 7f37eb441640 1 -- 172.21.15.32:0/308142428 >> 172.21.15.32:0/308142428 conn(0x7f37e406a320 msgr2=0x7f37e406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:29.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.342+0000 7f37eb441640 1 -- 172.21.15.32:0/308142428 shutdown_connections 2023-12-20T04:43:29.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.342+0000 7f37eb441640 1 -- 172.21.15.32:0/308142428 wait complete. 2023-12-20T04:43:29.359 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T04:43:29.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.506+0000 7f263742d640 1 Processor -- start 2023-12-20T04:43:29.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.506+0000 7f263742d640 1 -- start start 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.506+0000 7f263742d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2630164670 0x7f26300ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.506+0000 7f263742d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f263006da20 0x7f26300cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.506+0000 7f263742d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2630070230 0x7f2630070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.506+0000 7f263742d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26300cb0b0 con 0x7f263006da20 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.506+0000 7f263742d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2630070ba0 con 0x7f2630070230 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.506+0000 7f263742d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26300ca810 con 0x7f2630164670 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26349c8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f263006da20 0x7f26300cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26349c8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f263006da20 0x7f26300cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43212/0 (socket says 172.21.15.32:43212) 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26349c8640 1 -- 172.21.15.32:0/3005114965 learned_addr learned my addr 172.21.15.32:0/3005114965 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26359ca640 1 --2- 172.21.15.32:0/3005114965 >> v2:172.21.15.39:3300/0 conn(0x7f2630070230 0x7f2630070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26351c9640 1 --2- 172.21.15.32:0/3005114965 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2630164670 0x7f26300ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26349c8640 1 -- 172.21.15.32:0/3005114965 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2630164670 msgr2=0x7f26300ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:29.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26349c8640 1 --2- 172.21.15.32:0/3005114965 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2630164670 0x7f26300ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26349c8640 1 -- 172.21.15.32:0/3005114965 >> v2:172.21.15.39:3300/0 conn(0x7f2630070230 msgr2=0x7f2630070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:29.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26349c8640 1 --2- 172.21.15.32:0/3005114965 >> v2:172.21.15.39:3300/0 conn(0x7f2630070230 0x7f2630070660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26349c8640 1 -- 172.21.15.32:0/3005114965 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26300caaf0 con 0x7f263006da20 2023-12-20T04:43:29.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26351c9640 1 --2- 172.21.15.32:0/3005114965 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2630164670 0x7f26300ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:43:29.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f26349c8640 1 --2- 172.21.15.32:0/3005114965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f263006da20 0x7f26300cc3b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f26300c8a00 tx=0x7f262001c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8cb051d864dbcdf0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:29.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f2627fff640 1 -- 172.21.15.32:0/3005114965 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26200ab600 con 0x7f263006da20 2023-12-20T04:43:29.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f2627fff640 1 -- 172.21.15.32:0/3005114965 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26200a7050 con 0x7f263006da20 2023-12-20T04:43:29.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.510+0000 7f2627fff640 1 -- 172.21.15.32:0/3005114965 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26200ab980 con 0x7f263006da20 2023-12-20T04:43:29.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 -- 172.21.15.32:0/3005114965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f263006da20 msgr2=0x7f26300cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:29.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 --2- 172.21.15.32:0/3005114965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f263006da20 0x7f26300cc3b0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f26300c8a00 tx=0x7f262001c3b0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 -- 172.21.15.32:0/3005114965 shutdown_connections 2023-12-20T04:43:29.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 --2- 172.21.15.32:0/3005114965 >> v2:172.21.15.39:3300/0 conn(0x7f2630070230 0x7f2630070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 --2- 172.21.15.32:0/3005114965 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f263006da20 0x7f26300cc3b0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 --2- 172.21.15.32:0/3005114965 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2630164670 0x7f26300ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 -- 172.21.15.32:0/3005114965 >> 172.21.15.32:0/3005114965 conn(0x7f263006a510 msgr2=0x7f26300ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:29.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 -- 172.21.15.32:0/3005114965 shutdown_connections 2023-12-20T04:43:29.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 -- 172.21.15.32:0/3005114965 wait complete. 2023-12-20T04:43:29.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 Processor -- start 2023-12-20T04:43:29.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 -- start start 2023-12-20T04:43:29.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f263006da20 0x7f2630155fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2630070230 0x7f263015a4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2630164670 0x7f263015aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263015eea0 con 0x7f2630164670 2023-12-20T04:43:29.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f263015e290 con 0x7f263006da20 2023-12-20T04:43:29.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f263742d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f263015e570 con 0x7f2630070230 2023-12-20T04:43:29.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f26359ca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2630164670 0x7f263015aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f26359ca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2630164670 0x7f263015aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43228/0 (socket says 172.21.15.32:43228) 2023-12-20T04:43:29.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f26359ca640 1 -- 172.21.15.32:0/1484648073 learned_addr learned my addr 172.21.15.32:0/1484648073 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f26351c9640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f263006da20 0x7f2630155fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.514+0000 7f26349c8640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2630070230 0x7f263015a4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f26359ca640 1 -- 172.21.15.32:0/1484648073 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2630070230 msgr2=0x7f263015a4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f26359ca640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2630070230 0x7f263015a4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f26359ca640 1 -- 172.21.15.32:0/1484648073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f263006da20 msgr2=0x7f2630155fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f26359ca640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f263006da20 0x7f2630155fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f26359ca640 1 -- 172.21.15.32:0/1484648073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26200094d0 con 0x7f2630164670 2023-12-20T04:43:29.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f26359ca640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2630164670 0x7f263015aa10 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2628007d20 tx=0x7f262800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:29.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f2625ffb640 1 -- 172.21.15.32:0/1484648073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628014420 con 0x7f2630164670 2023-12-20T04:43:29.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f2625ffb640 1 -- 172.21.15.32:0/1484648073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26280145a0 con 0x7f2630164670 2023-12-20T04:43:29.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f2625ffb640 1 -- 172.21.15.32:0/1484648073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2628014950 con 0x7f2630164670 2023-12-20T04:43:29.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f263742d640 1 -- 172.21.15.32:0/1484648073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f263015e850 con 0x7f2630164670 2023-12-20T04:43:29.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f263742d640 1 -- 172.21.15.32:0/1484648073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f263015d580 con 0x7f2630164670 2023-12-20T04:43:29.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f263742d640 1 -- 172.21.15.32:0/1484648073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f263004e000 con 0x7f2630164670 2023-12-20T04:43:29.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f2625ffb640 1 -- 172.21.15.32:0/1484648073 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 30) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2628014ab0 con 0x7f2630164670 2023-12-20T04:43:29.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f2625ffb640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2614065bb0 0x7f2614068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:29.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f2625ffb640 1 -- 172.21.15.32:0/1484648073 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f2628083a50 con 0x7f2630164670 2023-12-20T04:43:29.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.518+0000 7f26351c9640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2614065bb0 0x7f2614068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:29.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.522+0000 7f26351c9640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2614065bb0 0x7f2614068070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f262c00ab60 tx=0x7f262c008040 comp rx=0 tx=0).ready entity=mgr.6183 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:29.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.522+0000 7f2625ffb640 1 -- 172.21.15.32:0/1484648073 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f26280513b0 con 0x7f2630164670 2023-12-20T04:43:29.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:29.718+0000 7f263742d640 1 -- 172.21.15.32:0/1484648073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f263015a2b0 con 0x7f2630164670 2023-12-20T04:43:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.018+0000 7f2625ffb640 1 -- 172.21.15.32:0/1484648073 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 31) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f2628081f70 con 0x7f2630164670 2023-12-20T04:43:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.018+0000 7f2625ffb640 1 -- 172.21.15.32:0/1484648073 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v31) v1 ==== 92+0+0 (secure 0 0 0) 0x7f2628080b00 con 0x7f2630164670 2023-12-20T04:43:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 -- 172.21.15.32:0/1484648073 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2614065bb0 msgr2=0x7f2614068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2614065bb0 0x7f2614068070 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f262c00ab60 tx=0x7f262c008040 comp rx=0 tx=0).stop 2023-12-20T04:43:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 -- 172.21.15.32:0/1484648073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2630164670 msgr2=0x7f263015aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2630164670 0x7f263015aa10 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7f2628007d20 tx=0x7f262800e4b0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 -- 172.21.15.32:0/1484648073 shutdown_connections 2023-12-20T04:43:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2614065bb0 0x7f2614068070 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2630164670 0x7f263015aa10 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2630070230 0x7f263015a4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 --2- 172.21.15.32:0/1484648073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f263006da20 0x7f2630155fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 -- 172.21.15.32:0/1484648073 >> 172.21.15.32:0/1484648073 conn(0x7f263006a510 msgr2=0x7f26300ccc20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 -- 172.21.15.32:0/1484648073 shutdown_connections 2023-12-20T04:43:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.022+0000 7f263742d640 1 -- 172.21.15.32:0/1484648073 wait complete. 2023-12-20T04:43:30.050 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T04:43:30.196 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:30.196+0000 7feb9d9db180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:43:30.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc72d139640 1 Processor -- start 2023-12-20T04:43:30.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc72d139640 1 -- start start 2023-12-20T04:43:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc72d139640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7280d3300 0x7fc7280d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc72d139640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7280d2650 con 0x7fc72815ce60 2023-12-20T04:43:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc72d139640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7280d1a40 con 0x7fc7280d1300 2023-12-20T04:43:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc72d139640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7280d1d20 con 0x7fc7280d3300 2023-12-20T04:43:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc726d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc72815ce60 0x7fc7280d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54640/0 (socket says 172.21.15.32:54640) 2023-12-20T04:43:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc726d9d640 1 -- 172.21.15.32:0/4259592837 learned_addr learned my addr 172.21.15.32:0/4259592837 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:30.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc72659c640 1 --2- 172.21.15.32:0/4259592837 >> v2:172.21.15.39:3300/0 conn(0x7fc7280d3300 0x7fc7280d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc725d9b640 1 -- 172.21.15.32:0/4259592837 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193627693 0 0) 0x7fc7280d2650 con 0x7fc72815ce60 2023-12-20T04:43:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc725d9b640 1 -- 172.21.15.32:0/4259592837 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc710003530 con 0x7fc72815ce60 2023-12-20T04:43:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc72659c640 1 -- 172.21.15.32:0/4259592837 >> v1:172.21.15.39:6789/0 conn(0x7fc7280d1300 legacy=0x7fc7280b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc72659c640 1 -- 172.21.15.32:0/4259592837 >> v1:172.21.15.32:6789/0 conn(0x7fc72815ce60 legacy=0x7fc7280d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.202+0000 7fc72659c640 1 -- 172.21.15.32:0/4259592837 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7280d2000 con 0x7fc7280d3300 2023-12-20T04:43:30.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.206+0000 7fc72659c640 1 --2- 172.21.15.32:0/4259592837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280d3300 0x7fc7280d0dc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0097c0 tx=0x7fc71c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3253b4c861b59261 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:30.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.206+0000 7fc725d9b640 1 -- 172.21.15.32:0/4259592837 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0ac640 con 0x7fc7280d3300 2023-12-20T04:43:30.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.206+0000 7fc725d9b640 1 -- 172.21.15.32:0/4259592837 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c0ac7a0 con 0x7fc7280d3300 2023-12-20T04:43:30.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.206+0000 7fc725d9b640 1 -- 172.21.15.32:0/4259592837 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0acb10 con 0x7fc7280d3300 2023-12-20T04:43:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.206+0000 7fc72d139640 1 -- 172.21.15.32:0/4259592837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280d3300 msgr2=0x7fc7280d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.206+0000 7fc72d139640 1 --2- 172.21.15.32:0/4259592837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280d3300 0x7fc7280d0dc0 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0097c0 tx=0x7fc71c01c480 comp rx=0 tx=0).stop 2023-12-20T04:43:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.206+0000 7fc72d139640 1 -- 172.21.15.32:0/4259592837 shutdown_connections 2023-12-20T04:43:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.206+0000 7fc72d139640 1 --2- 172.21.15.32:0/4259592837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280d3300 0x7fc7280d0dc0 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.206+0000 7fc72d139640 1 -- 172.21.15.32:0/4259592837 >> 172.21.15.32:0/4259592837 conn(0x7fc72806b5e0 msgr2=0x7fc72806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.206+0000 7fc72d139640 1 -- 172.21.15.32:0/4259592837 shutdown_connections 2023-12-20T04:43:30.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.206+0000 7fc72d139640 1 -- 172.21.15.32:0/4259592837 wait complete. 2023-12-20T04:43:30.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72d139640 1 Processor -- start 2023-12-20T04:43:30.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72d139640 1 -- start start 2023-12-20T04:43:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72d139640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7280d1300 0x7fc728156b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72d139640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280d3300 0x7fc7280cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72d139640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc72815ce60 0x7fc7280cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72d139640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280cc2c0 con 0x7fc7280d1300 2023-12-20T04:43:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72d139640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7280cb510 con 0x7fc7280d3300 2023-12-20T04:43:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72d139640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7281572b0 con 0x7fc72815ce60 2023-12-20T04:43:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc726d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7280d1300 0x7fc728156b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280d3300 0x7fc7280cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc726d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7280d1300 0x7fc728156b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43252/0 (socket says 172.21.15.32:43252) 2023-12-20T04:43:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc726d9d640 1 -- 172.21.15.32:0/3588128888 learned_addr learned my addr 172.21.15.32:0/3588128888 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:30.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72759e640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc72815ce60 0x7fc7280cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72659c640 1 -- 172.21.15.32:0/3588128888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc72815ce60 msgr2=0x7fc7280cafd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72659c640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc72815ce60 0x7fc7280cafd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72659c640 1 -- 172.21.15.32:0/3588128888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7280d1300 msgr2=0x7fc728156b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72659c640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7280d1300 0x7fc728156b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72659c640 1 -- 172.21.15.32:0/3588128888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc71c009420 con 0x7fc7280d3300 2023-12-20T04:43:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72759e640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc72815ce60 0x7fc7280cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72659c640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280d3300 0x7fc7280cbd80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0ab000 tx=0x7fc71c019220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc70f7fe640 1 -- 172.21.15.32:0/3588128888 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c0acb10 con 0x7fc7280d3300 2023-12-20T04:43:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc70f7fe640 1 -- 172.21.15.32:0/3588128888 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc71c0acc70 con 0x7fc7280d3300 2023-12-20T04:43:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72d139640 1 -- 172.21.15.32:0/3588128888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7280cc720 con 0x7fc7280d3300 2023-12-20T04:43:30.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc70f7fe640 1 -- 172.21.15.32:0/3588128888 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc71c003620 con 0x7fc7280d3300 2023-12-20T04:43:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.210+0000 7fc72d139640 1 -- 172.21.15.32:0/3588128888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7280ccce0 con 0x7fc7280d3300 2023-12-20T04:43:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.214+0000 7fc72d139640 1 -- 172.21.15.32:0/3588128888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7280cced0 con 0x7fc7280d3300 2023-12-20T04:43:30.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.214+0000 7fc70f7fe640 1 -- 172.21.15.32:0/3588128888 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 31) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fc71c0b3040 con 0x7fc7280d3300 2023-12-20T04:43:30.212 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:30.214+0000 7f84ede2c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:43:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.214+0000 7fc70f7fe640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc714065bd0 0x7fc714068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.214+0000 7fc726d9d640 1 -- 172.21.15.32:0/3588128888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc714065bd0 msgr2=0x7fc714068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.214+0000 7fc70f7fe640 1 -- 172.21.15.32:0/3588128888 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc71c11c870 con 0x7fc7280d3300 2023-12-20T04:43:30.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.214+0000 7fc726d9d640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc714065bd0 0x7fc714068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:30.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.222+0000 7fc70f7fe640 1 -- 172.21.15.32:0/3588128888 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc71c0ea130 con 0x7fc7280d3300 2023-12-20T04:43:30.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.382+0000 7fc72d139640 1 -- 172.21.15.32:0/3588128888 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc72804e000 con 0x7fc7280d3300 2023-12-20T04:43:30.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.390+0000 7fc70f7fe640 1 -- 172.21.15.32:0/3588128888 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v23)=0 v23) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc71c0ed750 con 0x7fc7280d3300 2023-12-20T04:43:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 -- 172.21.15.32:0/3588128888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc714065bd0 msgr2=0x7fc714068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc714065bd0 0x7fc714068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 -- 172.21.15.32:0/3588128888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280d3300 msgr2=0x7fc7280cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280d3300 0x7fc7280cbd80 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc71c0ab000 tx=0x7fc71c019220 comp rx=0 tx=0).stop 2023-12-20T04:43:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 -- 172.21.15.32:0/3588128888 shutdown_connections 2023-12-20T04:43:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc714065bd0 0x7fc714068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc72815ce60 0x7fc7280cafd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7280d3300 0x7fc7280cbd80 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 --2- 172.21.15.32:0/3588128888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7280d1300 0x7fc728156b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 -- 172.21.15.32:0/3588128888 >> 172.21.15.32:0/3588128888 conn(0x7fc72806b5e0 msgr2=0x7fc7281581f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 -- 172.21.15.32:0/3588128888 shutdown_connections 2023-12-20T04:43:30.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.394+0000 7fc72d139640 1 -- 172.21.15.32:0/3588128888 wait complete. 2023-12-20T04:43:30.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T04:43:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f6adbc640 1 Processor -- start 2023-12-20T04:43:30.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f6adbc640 1 -- start start 2023-12-20T04:43:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f6adbc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5f6415ce60 0x7f5f640d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f6adbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5f640d3130 0x7f5f640d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f6adbc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5f640d2650 con 0x7f5f640d1300 2023-12-20T04:43:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f6adbc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f640d35a0 con 0x7f5f640d3130 2023-12-20T04:43:30.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f6adbc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5f640d1a40 con 0x7f5f6415ce60 2023-12-20T04:43:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f69359640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5f640d1300 0x7f5f640b46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54644/0 (socket says 172.21.15.32:54644) 2023-12-20T04:43:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f69359640 1 -- 172.21.15.32:0/1196589107 learned_addr learned my addr 172.21.15.32:0/1196589107 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f63fff640 1 --2- 172.21.15.32:0/1196589107 >> v2:172.21.15.32:3300/0 conn(0x7f5f640d3130 0x7f5f640d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f68b58640 1 --2- 172.21.15.32:0/1196589107 >> v2:172.21.15.39:3300/0 conn(0x7f5f6415ce60 0x7f5f640d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.562+0000 7f5f637fe640 1 -- 172.21.15.32:0/1196589107 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1257457088 0 0) 0x7f5f640d2650 con 0x7f5f640d1300 2023-12-20T04:43:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.566+0000 7f5f63fff640 1 -- 172.21.15.32:0/1196589107 >> v2:172.21.15.39:3300/0 conn(0x7f5f6415ce60 msgr2=0x7f5f640d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.566+0000 7f5f63fff640 1 --2- 172.21.15.32:0/1196589107 >> v2:172.21.15.39:3300/0 conn(0x7f5f6415ce60 0x7f5f640d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.566+0000 7f5f63fff640 1 -- 172.21.15.32:0/1196589107 >> v1:172.21.15.32:6789/0 conn(0x7f5f640d1300 legacy=0x7f5f640b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.566+0000 7f5f63fff640 1 -- 172.21.15.32:0/1196589107 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f640d1d20 con 0x7f5f640d3130 2023-12-20T04:43:30.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.566+0000 7f5f63fff640 1 --2- 172.21.15.32:0/1196589107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f640d3130 0x7f5f640d0dc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5f500097c0 tx=0x7f5f5001ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=466328a19675b1fb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.566+0000 7f5f637fe640 1 -- 172.21.15.32:0/1196589107 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f500a8930 con 0x7f5f640d3130 2023-12-20T04:43:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.566+0000 7f5f637fe640 1 -- 172.21.15.32:0/1196589107 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f500a8a90 con 0x7f5f640d3130 2023-12-20T04:43:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.566+0000 7f5f637fe640 1 -- 172.21.15.32:0/1196589107 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f500a8e20 con 0x7f5f640d3130 2023-12-20T04:43:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.566+0000 7f5f6adbc640 1 -- 172.21.15.32:0/1196589107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f640d3130 msgr2=0x7f5f640d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.566+0000 7f5f6adbc640 1 --2- 172.21.15.32:0/1196589107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f640d3130 0x7f5f640d0dc0 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7f5f500097c0 tx=0x7f5f5001ec30 comp rx=0 tx=0).stop 2023-12-20T04:43:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.566+0000 7f5f6adbc640 1 -- 172.21.15.32:0/1196589107 shutdown_connections 2023-12-20T04:43:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 --2- 172.21.15.32:0/1196589107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f640d3130 0x7f5f640d0dc0 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 --2- 172.21.15.32:0/1196589107 >> v2:172.21.15.39:3300/0 conn(0x7f5f6415ce60 0x7f5f640d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 -- 172.21.15.32:0/1196589107 >> 172.21.15.32:0/1196589107 conn(0x7f5f6406b5e0 msgr2=0x7f5f6406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 -- 172.21.15.32:0/1196589107 shutdown_connections 2023-12-20T04:43:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 -- 172.21.15.32:0/1196589107 wait complete. 2023-12-20T04:43:30.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 Processor -- start 2023-12-20T04:43:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 -- start start 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f640d1300 0x7f5f641e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5f640d3130 0x7f5f640cc0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f6415ce60 0x7f5f6414dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f641e5620 con 0x7f5f640d1300 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5f641e4a10 con 0x7f5f6415ce60 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f6adbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5f641e4cf0 con 0x7f5f640d3130 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f68b58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f640d1300 0x7f5f641e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f68b58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f640d1300 0x7f5f641e5ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43270/0 (socket says 172.21.15.32:43270) 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f68b58640 1 -- 172.21.15.32:0/3999422006 learned_addr learned my addr 172.21.15.32:0/3999422006 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f69359640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f6415ce60 0x7f5f6414dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f63fff640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5f640d3130 0x7f5f640cc0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f68b58640 1 -- 172.21.15.32:0/3999422006 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5f640d3130 msgr2=0x7f5f640cc0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f68b58640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5f640d3130 0x7f5f640cc0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f68b58640 1 -- 172.21.15.32:0/3999422006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f6415ce60 msgr2=0x7f5f6414dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f68b58640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f6415ce60 0x7f5f6414dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f68b58640 1 -- 172.21.15.32:0/3999422006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5f50009420 con 0x7f5f640d1300 2023-12-20T04:43:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f63fff640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5f640d3130 0x7f5f640cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.570+0000 7f5f68b58640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f640d1300 0x7f5f641e5ab0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800e870 tx=0x7f5f5800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.574+0000 7f5f617fa640 1 -- 172.21.15.32:0/3999422006 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f58009510 con 0x7f5f640d1300 2023-12-20T04:43:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.574+0000 7f5f617fa640 1 -- 172.21.15.32:0/3999422006 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5f5800eec0 con 0x7f5f640d1300 2023-12-20T04:43:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.574+0000 7f5f617fa640 1 -- 172.21.15.32:0/3999422006 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5f5800cb40 con 0x7f5f640d1300 2023-12-20T04:43:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.574+0000 7f5f6adbc640 1 -- 172.21.15.32:0/3999422006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5f641e4fd0 con 0x7f5f640d1300 2023-12-20T04:43:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.574+0000 7f5f6adbc640 1 -- 172.21.15.32:0/3999422006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5f6415a3c0 con 0x7f5f640d1300 2023-12-20T04:43:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.574+0000 7f5f6adbc640 1 -- 172.21.15.32:0/3999422006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5f6404e000 con 0x7f5f640d1300 2023-12-20T04:43:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.574+0000 7f5f617fa640 1 -- 172.21.15.32:0/3999422006 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f5f5800cca0 con 0x7f5f640d1300 2023-12-20T04:43:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.574+0000 7f5f617fa640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5f48065bd0 0x7f5f48068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.574+0000 7f5f63fff640 1 -- 172.21.15.32:0/3999422006 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5f48065bd0 msgr2=0x7f5f48068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.574+0000 7f5f617fa640 1 -- 172.21.15.32:0/3999422006 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5f58083360 con 0x7f5f640d1300 2023-12-20T04:43:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.574+0000 7f5f63fff640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5f48065bd0 0x7f5f48068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:30.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.586+0000 7f5f617fa640 1 -- 172.21.15.32:0/3999422006 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5f58050d30 con 0x7f5f640d1300 2023-12-20T04:43:30.641 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:30.640+0000 7feb9d9db180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:43:30.665 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:30.666+0000 7f84ede2c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:43:30.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.746+0000 7f5f6adbc640 1 -- 172.21.15.32:0/3999422006 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f5f641e33c0 con 0x7f5f640d1300 2023-12-20T04:43:30.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.750+0000 7f5f617fa640 1 -- 172.21.15.32:0/3999422006 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v24)=0 v24) v1 ==== 133+0+0 (secure 0 0 0) 0x7f5f58054350 con 0x7f5f640d1300 2023-12-20T04:43:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 -- 172.21.15.32:0/3999422006 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5f48065bd0 msgr2=0x7f5f48068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5f48065bd0 0x7f5f48068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 -- 172.21.15.32:0/3999422006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f640d1300 msgr2=0x7f5f641e5ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f640d1300 0x7f5f641e5ab0 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7f5f5800e870 tx=0x7f5f5800ec70 comp rx=0 tx=0).stop 2023-12-20T04:43:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 -- 172.21.15.32:0/3999422006 shutdown_connections 2023-12-20T04:43:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5f48065bd0 0x7f5f48068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5f6415ce60 0x7f5f6414dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5f640d3130 0x7f5f640cc0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 --2- 172.21.15.32:0/3999422006 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5f640d1300 0x7f5f641e5ab0 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 -- 172.21.15.32:0/3999422006 >> 172.21.15.32:0/3999422006 conn(0x7f5f6406b5e0 msgr2=0x7f5f640b7e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:30.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 -- 172.21.15.32:0/3999422006 shutdown_connections 2023-12-20T04:43:30.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.754+0000 7f5f6adbc640 1 -- 172.21.15.32:0/3999422006 wait complete. 2023-12-20T04:43:30.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:30.772 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-20T04:43:30.788 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-12-20T04:43:30.801 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:30.800+0000 7feb9d9db180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:43:30.825 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:30.826+0000 7f84ede2c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:43:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f07304d1640 1 Processor -- start 2023-12-20T04:43:30.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f07304d1640 1 -- start start 2023-12-20T04:43:30.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f07304d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07280d3300 0x7f07280d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f07304d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f07280d1300 0x7f07280b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f07304d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07280d2650 con 0x7f072815ce60 2023-12-20T04:43:30.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f07304d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07280d1a40 con 0x7f07280d1300 2023-12-20T04:43:30.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f07304d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07280d1d20 con 0x7f07280d3300 2023-12-20T04:43:30.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f072da6c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07280d3300 0x7f07280d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f072e26d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f072815ce60 0x7f07280d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:38092/0 (socket says 172.21.15.32:38092) 2023-12-20T04:43:30.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f072ea6e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f07280d1300 0x7f07280b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f072e26d640 1 -- 172.21.15.32:0/1008311383 learned_addr learned my addr 172.21.15.32:0/1008311383 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:30.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f072d26b640 1 -- 172.21.15.32:0/1008311383 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2602058434 0 0) 0x7f07280d2650 con 0x7f072815ce60 2023-12-20T04:43:30.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f072d26b640 1 -- 172.21.15.32:0/1008311383 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0714003530 con 0x7f072815ce60 2023-12-20T04:43:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f072da6c640 1 -- 172.21.15.32:0/1008311383 >> v2:172.21.15.39:3300/0 conn(0x7f07280d1300 msgr2=0x7f07280b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f072da6c640 1 --2- 172.21.15.32:0/1008311383 >> v2:172.21.15.39:3300/0 conn(0x7f07280d1300 0x7f07280b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f072da6c640 1 -- 172.21.15.32:0/1008311383 >> v1:172.21.15.39:6789/0 conn(0x7f072815ce60 legacy=0x7f07280d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.974+0000 7f072da6c640 1 -- 172.21.15.32:0/1008311383 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07280d2000 con 0x7f07280d3300 2023-12-20T04:43:30.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f072da6c640 1 --2- 172.21.15.32:0/1008311383 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07280d3300 0x7f07280d0dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f07180097c0 tx=0x7f071801ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b89c0a8d3697fef server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:30.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f072d26b640 1 -- 172.21.15.32:0/1008311383 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07180a8970 con 0x7f07280d3300 2023-12-20T04:43:30.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f072d26b640 1 -- 172.21.15.32:0/1008311383 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07180a8ad0 con 0x7f07280d3300 2023-12-20T04:43:30.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f072d26b640 1 -- 172.21.15.32:0/1008311383 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07180a8e60 con 0x7f07280d3300 2023-12-20T04:43:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f07304d1640 1 -- 172.21.15.32:0/1008311383 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07280d3300 msgr2=0x7f07280d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f07304d1640 1 --2- 172.21.15.32:0/1008311383 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07280d3300 0x7f07280d0dc0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f07180097c0 tx=0x7f071801ec70 comp rx=0 tx=0).stop 2023-12-20T04:43:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f07304d1640 1 -- 172.21.15.32:0/1008311383 shutdown_connections 2023-12-20T04:43:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f07304d1640 1 --2- 172.21.15.32:0/1008311383 >> v2:172.21.15.39:3300/0 conn(0x7f07280d1300 0x7f07280b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f07304d1640 1 --2- 172.21.15.32:0/1008311383 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07280d3300 0x7f07280d0dc0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f07304d1640 1 -- 172.21.15.32:0/1008311383 >> 172.21.15.32:0/1008311383 conn(0x7f072806b5e0 msgr2=0x7f072806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f07304d1640 1 -- 172.21.15.32:0/1008311383 shutdown_connections 2023-12-20T04:43:30.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.978+0000 7f07304d1640 1 -- 172.21.15.32:0/1008311383 wait complete. 2023-12-20T04:43:30.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f07304d1640 1 Processor -- start 2023-12-20T04:43:30.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f07304d1640 1 -- start start 2023-12-20T04:43:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f07304d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07280d1300 0x7f0728159ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f07304d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07280d3300 0x7f07281e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f07304d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f072815ce60 0x7f07280b2b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f07304d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07281e5b90 con 0x7f07280d3300 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f07304d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07281e4fa0 con 0x7f07280d1300 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f07304d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f072815a3e0 con 0x7f072815ce60 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f072da6c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07280d3300 0x7f07281e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f072da6c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07280d3300 0x7f07281e65e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43286/0 (socket says 172.21.15.32:43286) 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f072da6c640 1 -- 172.21.15.32:0/4109046560 learned_addr learned my addr 172.21.15.32:0/4109046560 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f072e26d640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07280d1300 0x7f0728159ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f072ea6e640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f072815ce60 0x7f07280b2b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f072da6c640 1 -- 172.21.15.32:0/4109046560 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f072815ce60 msgr2=0x7f07280b2b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f072da6c640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f072815ce60 0x7f07280b2b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f072da6c640 1 -- 172.21.15.32:0/4109046560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07280d1300 msgr2=0x7f0728159ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f072da6c640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07280d1300 0x7f0728159ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f072da6c640 1 -- 172.21.15.32:0/4109046560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0718009420 con 0x7f07280d3300 2023-12-20T04:43:30.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f072da6c640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07280d3300 0x7f07281e65e0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7f07180b3040 tx=0x7f071801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:30.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f071effd640 1 -- 172.21.15.32:0/4109046560 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07180b0d60 con 0x7f07280d3300 2023-12-20T04:43:30.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f071effd640 1 -- 172.21.15.32:0/4109046560 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07180b9430 con 0x7f07280d3300 2023-12-20T04:43:30.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f07304d1640 1 -- 172.21.15.32:0/4109046560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07281e51f0 con 0x7f07280d3300 2023-12-20T04:43:30.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f071effd640 1 -- 172.21.15.32:0/4109046560 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07180b97e0 con 0x7f07280d3300 2023-12-20T04:43:30.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.982+0000 7f07304d1640 1 -- 172.21.15.32:0/4109046560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07281e57b0 con 0x7f07280d3300 2023-12-20T04:43:30.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.986+0000 7f07304d1640 1 -- 172.21.15.32:0/4109046560 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07281e5cd0 con 0x7f07280d3300 2023-12-20T04:43:30.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.986+0000 7f071effd640 1 -- 172.21.15.32:0/4109046560 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 31) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f07180b7040 con 0x7f07280d3300 2023-12-20T04:43:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.986+0000 7f071effd640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 0x7f070c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.986+0000 7f072e26d640 1 -- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 msgr2=0x7f070c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.986+0000 7f071effd640 1 -- 172.21.15.32:0/4109046560 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(79..79 src has 1..79) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0718120cc0 con 0x7f07280d3300 2023-12-20T04:43:30.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.986+0000 7f072e26d640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 0x7f070c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:30.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:30.994+0000 7f071effd640 1 -- 172.21.15.32:0/4109046560 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f07281e5cd0 con 0x7f07280d3300 2023-12-20T04:43:31.143 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:43:31.148 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:43:31.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:31.174+0000 7f07304d1640 1 -- 172.21.15.32:0/4109046560 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f07280d1750 con 0x7f070c065bd0 2023-12-20T04:43:31.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:31.190+0000 7f072e26d640 1 -- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 msgr2=0x7f070c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:31.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:31.190+0000 7f072e26d640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 0x7f070c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:43:31.312 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:31.312+0000 7feb9d9db180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:31.345 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:31.346+0000 7f84ede2c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:31.374 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:31.372+0000 7feb9d9db180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:43:31.406 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:31.406+0000 7f84ede2c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:43:31.432 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:31.432+0000 7feb9d9db180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:43:31.465 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:31.466+0000 7f84ede2c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:43:31.524 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:31.524+0000 7feb9d9db180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:43:31.557 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:31.558+0000 7f84ede2c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:43:31.583 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:31.584+0000 7feb9d9db180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:43:31.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:31.594+0000 7f072e26d640 1 -- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 msgr2=0x7f070c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:31.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:31.594+0000 7f072e26d640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 0x7f070c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T04:43:31.616 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:31.618+0000 7f84ede2c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:43:31.671 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:31.672+0000 7feb9d9db180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:43:31.705 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:31.706+0000 7f84ede2c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:43:31.731 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:31.732+0000 7feb9d9db180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:43:31.764 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:31.766+0000 7f84ede2c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:43:31.881 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:31.880+0000 7feb9d9db180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:43:31.915 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:31.918+0000 7f84ede2c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:43:31.942 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:31.944+0000 7feb9d9db180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:43:31.977 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:31.978+0000 7f84ede2c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:43:32.193 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:43:32.193 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:43:32.193 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:43:32.201 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:32.200+0000 7feb9d9db180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:43:32.227 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:43:32.227 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:43:32.228 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:43:32.236 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:32.238+0000 7f84ede2c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:43:32.343 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:32.344+0000 7feb9d9db180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:32.378 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:32.378+0000 7f84ede2c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:32.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:32.398+0000 7f072e26d640 1 -- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 msgr2=0x7f070c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:32.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:32.398+0000 7f072e26d640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 0x7f070c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T04:43:32.474 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:32.476+0000 7feb9d9db180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:32.510 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:32.510+0000 7f84ede2c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:32.549 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:32.548+0000 7feb9d9db180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:43:32.584 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:32.586+0000 7f84ede2c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:43:32.691 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:32.692+0000 7feb9d9db180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:32.726 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:32.726+0000 7f84ede2c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:33.262 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:33.264+0000 7feb9d9db180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:43:33.297 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:33.298+0000 7f84ede2c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:43:33.351 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:33.352+0000 7feb9d9db180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:43:33.385 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:33.386+0000 7f84ede2c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:43:33.507 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:33.508+0000 7feb9d9db180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:43:33.543 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:33.542+0000 7f84ede2c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:43:33.595 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:33.596+0000 7feb9d9db180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:43:33.630 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:33.630+0000 7f84ede2c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:43:33.930 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:33.928+0000 7feb9d9db180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:43:33.965 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:33.966+0000 7f84ede2c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:43:33.993 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:33.992+0000 7feb9d9db180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:43:33.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:34.002+0000 7f072e26d640 1 -- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 msgr2=0x7f070c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:33.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:34.002+0000 7f072e26d640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 0x7f070c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-20T04:43:34.029 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:34.030+0000 7f84ede2c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:43:34.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:34.042+0000 7f071effd640 1 -- 172.21.15.32:0/4109046560 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 32) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f07180f80f0 con 0x7f07280d3300 2023-12-20T04:43:34.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:34.042+0000 7f071effd640 1 -- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 msgr2=0x7f070c068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:34.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:34.042+0000 7f071effd640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 0x7f070c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:34.155 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:34.154+0000 7f846fe60640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:34.155 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:34.154+0000 7f846fe60640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:34.155 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:34.154+0000 7f846fe60640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:34.155 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:34.154+0000 7f846fe60640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:34.155 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:34.154+0000 7f846fe60640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:34.155 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:34.154+0000 7f8472e66640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:34.156 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:34.154+0000 7f8472e66640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:34.156 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:34.154+0000 7f8472e66640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:34.156 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:34.158+0000 7f8472e66640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:34.156 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:34.158+0000 7f8472e66640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:35.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.070+0000 7f071effd640 1 -- 172.21.15.32:0/4109046560 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 33) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f07180eea80 con 0x7f07280d3300 2023-12-20T04:43:35.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.070+0000 7f071effd640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 0x7f070c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:35.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.070+0000 7f071effd640 1 -- 172.21.15.32:0/4109046560 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f070c033630 con 0x7f070c065bd0 2023-12-20T04:43:35.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.070+0000 7f072e26d640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 0x7f070c068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.074+0000 7f072e26d640 1 --2- 172.21.15.32:0/4109046560 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f070c065bd0 0x7f070c068090 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7f07240011f0 tx=0x7f0724007d40 comp rx=0 tx=0).ready entity=mgr.6573 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:35.084 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:35.086+0000 7f8479e74640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-20T04:43:35.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.086+0000 7f071effd640 1 -- 172.21.15.32:0/4109046560 <== mgr.6573 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7f070c033630 con 0x7f070c065bd0 2023-12-20T04:43:35.084 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-20T04:43:35.093 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-20T04:43:35.094 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:35.094 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-20T04:43:35.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T04:43:35.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.302+0000 7f8f35536640 1 Processor -- start 2023-12-20T04:43:35.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.302+0000 7f8f35536640 1 -- start start 2023-12-20T04:43:35.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.302+0000 7f8f35536640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 0x7f8f300b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:35.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.302+0000 7f8f35536640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f300b09b0 con 0x7f8f3015a920 2023-12-20T04:43:35.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.302+0000 7f8f35536640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f300c9ff0 con 0x7f8f300b16c0 2023-12-20T04:43:35.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.302+0000 7f8f35536640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f300b4510 con 0x7f8f300b0010 2023-12-20T04:43:35.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.302+0000 7f8f2ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8f3015a920 0x7f8f300ca690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54648/0 (socket says 172.21.15.32:54648) 2023-12-20T04:43:35.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.302+0000 7f8f2ffff640 1 -- 172.21.15.32:0/3645233429 learned_addr learned my addr 172.21.15.32:0/3645233429 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:35.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2f7fe640 1 --2- 172.21.15.32:0/3645233429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 0x7f8f300b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:35.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2effd640 1 -- 172.21.15.32:0/3645233429 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3227744036 0 0) 0x7f8f300b09b0 con 0x7f8f3015a920 2023-12-20T04:43:35.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2effd640 1 -- 172.21.15.32:0/3645233429 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f18003530 con 0x7f8f3015a920 2023-12-20T04:43:35.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2effd640 1 -- 172.21.15.32:0/3645233429 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2402624164 0 0) 0x7f8f300c9ff0 con 0x7f8f300b16c0 2023-12-20T04:43:35.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2effd640 1 -- 172.21.15.32:0/3645233429 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8f300b09b0 con 0x7f8f300b16c0 2023-12-20T04:43:35.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2f7fe640 1 -- 172.21.15.32:0/3645233429 >> v1:172.21.15.39:6789/0 conn(0x7f8f300b16c0 legacy=0x7f8f300b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:35.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2f7fe640 1 -- 172.21.15.32:0/3645233429 >> v1:172.21.15.32:6789/0 conn(0x7f8f3015a920 legacy=0x7f8f300ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:35.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2effd640 1 -- 172.21.15.32:0/3645233429 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1261588731 0 0) 0x7f8f18003530 con 0x7f8f3015a920 2023-12-20T04:43:35.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2f7fe640 1 -- 172.21.15.32:0/3645233429 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f300c93e0 con 0x7f8f300b0010 2023-12-20T04:43:35.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2f7fe640 1 --2- 172.21.15.32:0/3645233429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 0x7f8f300b0470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f240097f0 tx=0x7f8f2401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b06f72de08019057 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:35.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2effd640 1 -- 172.21.15.32:0/3645233429 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f20001680 con 0x7f8f300b0010 2023-12-20T04:43:35.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2effd640 1 -- 172.21.15.32:0/3645233429 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f240a7050 con 0x7f8f300b0010 2023-12-20T04:43:35.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.306+0000 7f8f2effd640 1 -- 172.21.15.32:0/3645233429 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f240ab600 con 0x7f8f300b0010 2023-12-20T04:43:35.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 -- 172.21.15.32:0/3645233429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 msgr2=0x7f8f300b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 --2- 172.21.15.32:0/3645233429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 0x7f8f300b0470 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f8f240097f0 tx=0x7f8f2401c3e0 comp rx=0 tx=0).stop 2023-12-20T04:43:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 -- 172.21.15.32:0/3645233429 shutdown_connections 2023-12-20T04:43:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 --2- 172.21.15.32:0/3645233429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 0x7f8f300b0470 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 -- 172.21.15.32:0/3645233429 >> 172.21.15.32:0/3645233429 conn(0x7f8f3006c060 msgr2=0x7f8f3006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 -- 172.21.15.32:0/3645233429 shutdown_connections 2023-12-20T04:43:35.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 -- 172.21.15.32:0/3645233429 wait complete. 2023-12-20T04:43:35.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 Processor -- start 2023-12-20T04:43:35.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 -- start start 2023-12-20T04:43:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 0x7f8f30164b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:35.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f300b16c0 0x7f8f30153b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f3015a920 0x7f8f301571b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f300c4bf0 con 0x7f8f300b16c0 2023-12-20T04:43:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f301540b0 con 0x7f8f3015a920 2023-12-20T04:43:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f35536640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f301576f0 con 0x7f8f300b0010 2023-12-20T04:43:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f2f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f300b16c0 0x7f8f30153b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f2f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f300b16c0 0x7f8f30153b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43436/0 (socket says 172.21.15.32:43436) 2023-12-20T04:43:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.310+0000 7f8f2f7fe640 1 -- 172.21.15.32:0/1231000954 learned_addr learned my addr 172.21.15.32:0/1231000954 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2ffff640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 0x7f8f30164b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:35.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f34d35640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f3015a920 0x7f8f301571b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2f7fe640 1 -- 172.21.15.32:0/1231000954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 msgr2=0x7f8f30164b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2f7fe640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 0x7f8f30164b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2f7fe640 1 -- 172.21.15.32:0/1231000954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f3015a920 msgr2=0x7f8f301571b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2f7fe640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f3015a920 0x7f8f301571b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2f7fe640 1 -- 172.21.15.32:0/1231000954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f24009420 con 0x7f8f300b16c0 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2ffff640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 0x7f8f30164b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f34d35640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f3015a920 0x7f8f301571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2f7fe640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f300b16c0 0x7f8f30153b70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8f240097f0 tx=0x7f8f24019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2cff9640 1 -- 172.21.15.32:0/1231000954 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f24003af0 con 0x7f8f300b16c0 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2cff9640 1 -- 172.21.15.32:0/1231000954 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f24003c50 con 0x7f8f300b16c0 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2cff9640 1 -- 172.21.15.32:0/1231000954 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f240b4400 con 0x7f8f300b16c0 2023-12-20T04:43:35.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f35536640 1 -- 172.21.15.32:0/1231000954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f300c3400 con 0x7f8f300b16c0 2023-12-20T04:43:35.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f35536640 1 -- 172.21.15.32:0/1231000954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f300c2db0 con 0x7f8f300b16c0 2023-12-20T04:43:35.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f35536640 1 -- 172.21.15.32:0/1231000954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f30074100 con 0x7f8f300b16c0 2023-12-20T04:43:35.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2cff9640 1 -- 172.21.15.32:0/1231000954 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 33) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f8f240b2040 con 0x7f8f300b16c0 2023-12-20T04:43:35.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2cff9640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f1c065900 0x7f8f1c067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:35.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.314+0000 7f8f2cff9640 1 -- 172.21.15.32:0/1231000954 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f8f2411be90 con 0x7f8f300b16c0 2023-12-20T04:43:35.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.318+0000 7f8f2ffff640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f1c065900 0x7f8f1c067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:35.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.318+0000 7f8f2ffff640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f1c065900 0x7f8f1c067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8f20001350 tx=0x7f8f20009040 comp rx=0 tx=0).ready entity=mgr.6573 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:35.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.322+0000 7f8f2cff9640 1 -- 172.21.15.32:0/1231000954 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8f240e9960 con 0x7f8f300b16c0 2023-12-20T04:43:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:35.518+0000 7f8f35536640 1 -- 172.21.15.32:0/1231000954 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f8f300b3140 con 0x7f8f300b16c0 2023-12-20T04:43:36.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.074+0000 7f8f2cff9640 1 -- 172.21.15.32:0/1231000954 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f8f240e9d30 con 0x7f8f300b16c0 2023-12-20T04:43:36.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f2cff9640 1 -- 172.21.15.32:0/1231000954 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v34) v1 ==== 93+0+0 (secure 0 0 0) 0x7f8f240ed3f0 con 0x7f8f300b16c0 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 -- 172.21.15.32:0/1231000954 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f1c065900 msgr2=0x7f8f1c067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f1c065900 0x7f8f1c067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8f20001350 tx=0x7f8f20009040 comp rx=0 tx=0).stop 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 -- 172.21.15.32:0/1231000954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f300b16c0 msgr2=0x7f8f30153b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f300b16c0 0x7f8f30153b70 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f8f240097f0 tx=0x7f8f24019180 comp rx=0 tx=0).stop 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 -- 172.21.15.32:0/1231000954 shutdown_connections 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f1c065900 0x7f8f1c067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f3015a920 0x7f8f301571b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f300b16c0 0x7f8f30153b70 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 --2- 172.21.15.32:0/1231000954 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f300b0010 0x7f8f30164b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 -- 172.21.15.32:0/1231000954 >> 172.21.15.32:0/1231000954 conn(0x7f8f3006c060 msgr2=0x7f8f300b3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 -- 172.21.15.32:0/1231000954 shutdown_connections 2023-12-20T04:43:36.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.078+0000 7f8f35536640 1 -- 172.21.15.32:0/1231000954 wait complete. 2023-12-20T04:43:36.104 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:43:36.104 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:43:36.167 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:43:36.245 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:43:36.245 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:43:36.245 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:36.252 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:36.252+0000 7f3c8b02d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:43:36.254 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:43:36.254 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:36.261 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:36.262+0000 7f8e47247180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:43:36.270 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2706841640 1 Processor -- start 2023-12-20T04:43:36.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2706841640 1 -- start start 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2706841640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f27000d2740 0x7f27000cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2706841640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27000cfab0 0x7f27000b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2706841640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27000cfec0 con 0x7f270015cdb0 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2706841640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f270015d190 con 0x7f27000cfab0 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2706841640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27000d16a0 con 0x7f27000d2740 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2704dde640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27000cfab0 0x7f27000b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2704dde640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27000cfab0 0x7f27000b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43454/0 (socket says 172.21.15.32:43454) 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f26ff7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f27000d2740 0x7f27000cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2704dde640 1 -- 172.21.15.32:0/2864997189 learned_addr learned my addr 172.21.15.32:0/2864997189 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2704dde640 1 -- 172.21.15.32:0/2864997189 >> v2:172.21.15.39:3300/0 conn(0x7f27000d2740 msgr2=0x7f27000cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2704dde640 1 --2- 172.21.15.32:0/2864997189 >> v2:172.21.15.39:3300/0 conn(0x7f27000d2740 0x7f27000cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2704dde640 1 -- 172.21.15.32:0/2864997189 >> v1:172.21.15.39:6789/0 conn(0x7f270015cdb0 legacy=0x7f27000d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.458+0000 7f2704dde640 1 -- 172.21.15.32:0/2864997189 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27000d1980 con 0x7f27000cfab0 2023-12-20T04:43:36.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f2704dde640 1 --2- 172.21.15.32:0/2864997189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27000cfab0 0x7f27000b5c90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f27000cd570 tx=0x7f26f801f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5e065f3870c6e59 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:36.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f26feffd640 1 -- 172.21.15.32:0/2864997189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f80af400 con 0x7f27000cfab0 2023-12-20T04:43:36.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f26feffd640 1 -- 172.21.15.32:0/2864997189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f80ab050 con 0x7f27000cfab0 2023-12-20T04:43:36.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f26feffd640 1 -- 172.21.15.32:0/2864997189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f80af780 con 0x7f27000cfab0 2023-12-20T04:43:36.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f2706841640 1 -- 172.21.15.32:0/2864997189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27000cfab0 msgr2=0x7f27000b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f2706841640 1 --2- 172.21.15.32:0/2864997189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27000cfab0 0x7f27000b5c90 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7f27000cd570 tx=0x7f26f801f1c0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f2706841640 1 -- 172.21.15.32:0/2864997189 shutdown_connections 2023-12-20T04:43:36.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f2706841640 1 --2- 172.21.15.32:0/2864997189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27000cfab0 0x7f27000b5c90 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f2706841640 1 --2- 172.21.15.32:0/2864997189 >> v2:172.21.15.39:3300/0 conn(0x7f27000d2740 0x7f27000cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f2706841640 1 -- 172.21.15.32:0/2864997189 >> 172.21.15.32:0/2864997189 conn(0x7f2700065ee0 msgr2=0x7f270006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:36.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f2706841640 1 -- 172.21.15.32:0/2864997189 shutdown_connections 2023-12-20T04:43:36.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.462+0000 7f2706841640 1 -- 172.21.15.32:0/2864997189 wait complete. 2023-12-20T04:43:36.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f2706841640 1 Processor -- start 2023-12-20T04:43:36.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f2706841640 1 -- start start 2023-12-20T04:43:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f2706841640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27000cfab0 0x7f270014ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f2706841640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000d2740 0x7f2700157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f2706841640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f270015cdb0 0x7f2700159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f2706841640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f270014b060 con 0x7f27000cfab0 2023-12-20T04:43:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f2706841640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2700156a10 con 0x7f27000d2740 2023-12-20T04:43:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f2706841640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27000c8920 con 0x7f270015cdb0 2023-12-20T04:43:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f26fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27000cfab0 0x7f270014ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f26ff7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000d2740 0x7f2700157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f26fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27000cfab0 0x7f270014ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43464/0 (socket says 172.21.15.32:43464) 2023-12-20T04:43:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f26fffff640 1 -- 172.21.15.32:0/3802229244 learned_addr learned my addr 172.21.15.32:0/3802229244 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f2704dde640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f270015cdb0 0x7f2700159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f26ff7fe640 1 -- 172.21.15.32:0/3802229244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f270015cdb0 msgr2=0x7f2700159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f26ff7fe640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f270015cdb0 0x7f2700159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f26ff7fe640 1 -- 172.21.15.32:0/3802229244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27000cfab0 msgr2=0x7f270014ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f26ff7fe640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27000cfab0 0x7f270014ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f26ff7fe640 1 -- 172.21.15.32:0/3802229244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26f8009b30 con 0x7f27000d2740 2023-12-20T04:43:36.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.466+0000 7f26ff7fe640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000d2740 0x7f2700157e40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f26f400e870 tx=0x7f26f400ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:36.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.470+0000 7f26fcff9640 1 -- 172.21.15.32:0/3802229244 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f40095d0 con 0x7f27000d2740 2023-12-20T04:43:36.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.470+0000 7f26fcff9640 1 -- 172.21.15.32:0/3802229244 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26f400c940 con 0x7f27000d2740 2023-12-20T04:43:36.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.470+0000 7f2706841640 1 -- 172.21.15.32:0/3802229244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2700156170 con 0x7f27000d2740 2023-12-20T04:43:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.470+0000 7f26fcff9640 1 -- 172.21.15.32:0/3802229244 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26f400ccb0 con 0x7f27000d2740 2023-12-20T04:43:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.470+0000 7f2706841640 1 -- 172.21.15.32:0/3802229244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2700156730 con 0x7f27000d2740 2023-12-20T04:43:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.470+0000 7f2706841640 1 -- 172.21.15.32:0/3802229244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f270004e000 con 0x7f27000d2740 2023-12-20T04:43:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.470+0000 7f26fcff9640 1 -- 172.21.15.32:0/3802229244 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f26f4013470 con 0x7f27000d2740 2023-12-20T04:43:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.470+0000 7f26fcff9640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26ec065b80 0x7f26ec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.470+0000 7f26fffff640 1 -- 172.21.15.32:0/3802229244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26ec065b80 msgr2=0x7f26ec068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:36.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.470+0000 7f26fcff9640 1 -- 172.21.15.32:0/3802229244 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f26f4083570 con 0x7f27000d2740 2023-12-20T04:43:36.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.470+0000 7f26fffff640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26ec065b80 0x7f26ec068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:36.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.474+0000 7f26fcff9640 1 -- 172.21.15.32:0/3802229244 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f26f4050d60 con 0x7f27000d2740 2023-12-20T04:43:36.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.634+0000 7f2706841640 1 -- 172.21.15.32:0/3802229244 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f270006b8d0 con 0x7f27000d2740 2023-12-20T04:43:36.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.634+0000 7f26fcff9640 1 -- 172.21.15.32:0/3802229244 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16540 (secure 0 0 0) 0x7f26f4054380 con 0x7f27000d2740 2023-12-20T04:43:36.635 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:36.635 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:34.035710+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:19.734786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:20.967556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2610637012":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2158383566":"2023-12-21T04:43:34.035676+0000","172.21.15.32:6833/38009":"2023-12-21T04:43:34.035676+0000","172.21.15.32:6832/38009":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2830398071":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/3257133891":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2848355101":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/1251934730":"2023-12-21T04:43:34.035676+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:36.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 -- 172.21.15.32:0/3802229244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26ec065b80 msgr2=0x7f26ec068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:36.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26ec065b80 0x7f26ec068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 -- 172.21.15.32:0/3802229244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000d2740 msgr2=0x7f2700157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000d2740 0x7f2700157e40 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f26f400e870 tx=0x7f26f400ed30 comp rx=0 tx=0).stop 2023-12-20T04:43:36.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 -- 172.21.15.32:0/3802229244 shutdown_connections 2023-12-20T04:43:36.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26ec065b80 0x7f26ec068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f270015cdb0 0x7f2700159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27000d2740 0x7f2700157e40 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 --2- 172.21.15.32:0/3802229244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27000cfab0 0x7f270014ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 -- 172.21.15.32:0/3802229244 >> 172.21.15.32:0/3802229244 conn(0x7f2700065ee0 msgr2=0x7f27000b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:36.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 -- 172.21.15.32:0/3802229244 shutdown_connections 2023-12-20T04:43:36.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.638+0000 7f2706841640 1 -- 172.21.15.32:0/3802229244 wait complete. 2023-12-20T04:43:36.658 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:43:19.734786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:43:20.967556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:43:36.658 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:43:36.689 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:36.688+0000 7f3c8b02d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:43:36.715 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:36.718+0000 7f8e47247180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:43:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.810+0000 7f8a4aa04640 1 Processor -- start 2023-12-20T04:43:36.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.810+0000 7f8a4aa04640 1 -- start start 2023-12-20T04:43:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4aa04640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a4415ccf0 0x7f8a440cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4aa04640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 0x7f8a440d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4aa04640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a440d10f0 con 0x7f8a440cd7b0 2023-12-20T04:43:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4aa04640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8a440ce2f0 con 0x7f8a4415ccf0 2023-12-20T04:43:36.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4aa04640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a440d0b70 con 0x7f8a440cfaf0 2023-12-20T04:43:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4a203640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 0x7f8a440d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a49a02640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8a4415ccf0 0x7f8a440cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4a203640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 0x7f8a440d1580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:51282/0 (socket says 172.21.15.32:51282) 2023-12-20T04:43:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4a203640 1 -- 172.21.15.32:0/3205818298 learned_addr learned my addr 172.21.15.32:0/3205818298 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4a203640 1 -- 172.21.15.32:0/3205818298 >> v2:172.21.15.32:3300/0 conn(0x7f8a4415ccf0 msgr2=0x7f8a440cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4a203640 1 --2- 172.21.15.32:0/3205818298 >> v2:172.21.15.32:3300/0 conn(0x7f8a4415ccf0 0x7f8a440cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4a203640 1 -- 172.21.15.32:0/3205818298 >> v1:172.21.15.39:6789/0 conn(0x7f8a440cd7b0 legacy=0x7f8a440cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.814+0000 7f8a4a203640 1 -- 172.21.15.32:0/3205818298 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a440cdd30 con 0x7f8a440cfaf0 2023-12-20T04:43:36.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a4a203640 1 --2- 172.21.15.32:0/3205818298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 0x7f8a440d1580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009e50 tx=0x7f8a3c01c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=39faf6eaaee65fb4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:36.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a48a00640 1 -- 172.21.15.32:0/3205818298 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0a5810 con 0x7f8a440cfaf0 2023-12-20T04:43:36.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a48a00640 1 -- 172.21.15.32:0/3205818298 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a3c0a5970 con 0x7f8a440cfaf0 2023-12-20T04:43:36.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a48a00640 1 -- 172.21.15.32:0/3205818298 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a3c0a5d00 con 0x7f8a440cfaf0 2023-12-20T04:43:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3205818298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 msgr2=0x7f8a440d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a4aa04640 1 --2- 172.21.15.32:0/3205818298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 0x7f8a440d1580 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8a3c009e50 tx=0x7f8a3c01c950 comp rx=0 tx=0).stop 2023-12-20T04:43:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3205818298 shutdown_connections 2023-12-20T04:43:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a4aa04640 1 --2- 172.21.15.32:0/3205818298 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 0x7f8a440d1580 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a4aa04640 1 --2- 172.21.15.32:0/3205818298 >> v2:172.21.15.32:3300/0 conn(0x7f8a4415ccf0 0x7f8a440cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3205818298 >> 172.21.15.32:0/3205818298 conn(0x7f8a440658e0 msgr2=0x7f8a44065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3205818298 shutdown_connections 2023-12-20T04:43:36.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3205818298 wait complete. 2023-12-20T04:43:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a4aa04640 1 Processor -- start 2023-12-20T04:43:36.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.818+0000 7f8a4aa04640 1 -- start start 2023-12-20T04:43:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a4aa04640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a440cd7b0 0x7f8a441496e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a4aa04640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 0x7f8a440b6d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a4aa04640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a4415ccf0 0x7f8a440b72d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a4aa04640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4414c740 con 0x7f8a440cd7b0 2023-12-20T04:43:36.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a4aa04640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a4414cb60 con 0x7f8a4415ccf0 2023-12-20T04:43:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a4aa04640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a441510e0 con 0x7f8a440cfaf0 2023-12-20T04:43:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a49a02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a440cd7b0 0x7f8a441496e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a49201640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 0x7f8a440b6d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a49a02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a440cd7b0 0x7f8a441496e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43484/0 (socket says 172.21.15.32:43484) 2023-12-20T04:43:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a49a02640 1 -- 172.21.15.32:0/3028816621 learned_addr learned my addr 172.21.15.32:0/3028816621 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a4a203640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a4415ccf0 0x7f8a440b72d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a49a02640 1 -- 172.21.15.32:0/3028816621 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 msgr2=0x7f8a440b6d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a49a02640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 0x7f8a440b6d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a49a02640 1 -- 172.21.15.32:0/3028816621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a4415ccf0 msgr2=0x7f8a440b72d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a49a02640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a4415ccf0 0x7f8a440b72d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a49a02640 1 -- 172.21.15.32:0/3028816621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a3c009b30 con 0x7f8a440cd7b0 2023-12-20T04:43:36.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a49a02640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a440cd7b0 0x7f8a441496e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000e930 tx=0x7f8a4000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:36.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a3a7fc640 1 -- 172.21.15.32:0/3028816621 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a40009690 con 0x7f8a440cd7b0 2023-12-20T04:43:36.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a3a7fc640 1 -- 172.21.15.32:0/3028816621 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a4000ca00 con 0x7f8a440cd7b0 2023-12-20T04:43:36.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3028816621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a44151380 con 0x7f8a440cd7b0 2023-12-20T04:43:36.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a3a7fc640 1 -- 172.21.15.32:0/3028816621 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a4000cd70 con 0x7f8a440cd7b0 2023-12-20T04:43:36.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3028816621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a4414c2f0 con 0x7f8a440cd7b0 2023-12-20T04:43:36.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.822+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3028816621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a44074100 con 0x7f8a440cd7b0 2023-12-20T04:43:36.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.826+0000 7f8a3a7fc640 1 -- 172.21.15.32:0/3028816621 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f8a40013470 con 0x7f8a440cd7b0 2023-12-20T04:43:36.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.826+0000 7f8a3a7fc640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a28065b80 0x7f8a28068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:36.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.826+0000 7f8a49201640 1 -- 172.21.15.32:0/3028816621 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a28065b80 msgr2=0x7f8a28068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:36.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.826+0000 7f8a3a7fc640 1 -- 172.21.15.32:0/3028816621 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f8a400835d0 con 0x7f8a440cd7b0 2023-12-20T04:43:36.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.826+0000 7f8a49201640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a28065b80 0x7f8a28068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:36.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.834+0000 7f8a3a7fc640 1 -- 172.21.15.32:0/3028816621 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8a40050e80 con 0x7f8a440cd7b0 2023-12-20T04:43:36.846 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:36.848+0000 7f3c8b02d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:43:36.883 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:36.886+0000 7f8e47247180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:43:36.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.994+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3028816621 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8a440d0720 con 0x7f8a440cd7b0 2023-12-20T04:43:36.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.994+0000 7f8a3a7fc640 1 -- 172.21.15.32:0/3028816621 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8a400544a0 con 0x7f8a440cd7b0 2023-12-20T04:43:36.993 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:43:36.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.994+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3028816621 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a28065b80 msgr2=0x7f8a28068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:36.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.994+0000 7f8a4aa04640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a28065b80 0x7f8a28068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.994+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3028816621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a440cd7b0 msgr2=0x7f8a441496e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:36.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.994+0000 7f8a4aa04640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a440cd7b0 0x7f8a441496e0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7f8a4000e930 tx=0x7f8a4000edf0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.998+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3028816621 shutdown_connections 2023-12-20T04:43:36.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.998+0000 7f8a4aa04640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a28065b80 0x7f8a28068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.998+0000 7f8a4aa04640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a4415ccf0 0x7f8a440b72d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.998+0000 7f8a4aa04640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a440cfaf0 0x7f8a440b6d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.998+0000 7f8a4aa04640 1 --2- 172.21.15.32:0/3028816621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a440cd7b0 0x7f8a441496e0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:36.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.998+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3028816621 >> 172.21.15.32:0/3028816621 conn(0x7f8a440658e0 msgr2=0x7f8a44065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:36.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.998+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3028816621 shutdown_connections 2023-12-20T04:43:36.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:36.998+0000 7f8a4aa04640 1 -- 172.21.15.32:0/3028816621 wait complete. 2023-12-20T04:43:37.016 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:43:37.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.166+0000 7fc888cfc640 1 Processor -- start 2023-12-20T04:43:37.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.166+0000 7fc888cfc640 1 -- start start 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.166+0000 7fc888cfc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc88415abb0 0x7fc8840caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.166+0000 7fc888cfc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc884070230 0x7fc8840706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.166+0000 7fc888cfc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc88415b6a0 con 0x7fc8840b0cb0 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.166+0000 7fc888cfc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc88406e160 con 0x7fc884070230 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.166+0000 7fc888cfc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc884070be0 con 0x7fc88415abb0 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.166+0000 7fc87bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc8840b0cb0 0x7fc88415bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54664/0 (socket says 172.21.15.32:54664) 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.166+0000 7fc87bfff640 1 -- 172.21.15.32:0/2276920448 learned_addr learned my addr 172.21.15.32:0/2276920448 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.166+0000 7fc882d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc884070230 0x7fc8840706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.166+0000 7fc88259c640 1 --2- 172.21.15.32:0/2276920448 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc88415abb0 0x7fc8840caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc88259c640 1 -- 172.21.15.32:0/2276920448 >> v2:172.21.15.32:3300/0 conn(0x7fc884070230 msgr2=0x7fc8840706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc88259c640 1 --2- 172.21.15.32:0/2276920448 >> v2:172.21.15.32:3300/0 conn(0x7fc884070230 0x7fc8840706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc88259c640 1 -- 172.21.15.32:0/2276920448 >> v1:172.21.15.32:6789/0 conn(0x7fc8840b0cb0 legacy=0x7fc88415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc881d9b640 1 -- 172.21.15.32:0/2276920448 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 965932769 0 0) 0x7fc88415b6a0 con 0x7fc8840b0cb0 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc88259c640 1 -- 172.21.15.32:0/2276920448 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc88406d8c0 con 0x7fc88415abb0 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc88259c640 1 --2- 172.21.15.32:0/2276920448 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc88415abb0 0x7fc8840caaf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc8740097c0 tx=0x7fc87401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28ea2b9127fc782e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc881d9b640 1 -- 172.21.15.32:0/2276920448 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8740ab680 con 0x7fc88415abb0 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc881d9b640 1 -- 172.21.15.32:0/2276920448 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8740ab7e0 con 0x7fc88415abb0 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc881d9b640 1 -- 172.21.15.32:0/2276920448 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8740abb70 con 0x7fc88415abb0 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc888cfc640 1 -- 172.21.15.32:0/2276920448 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc88415abb0 msgr2=0x7fc8840caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc888cfc640 1 --2- 172.21.15.32:0/2276920448 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc88415abb0 0x7fc8840caaf0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7fc8740097c0 tx=0x7fc87401c480 comp rx=0 tx=0).stop 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc888cfc640 1 -- 172.21.15.32:0/2276920448 shutdown_connections 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc888cfc640 1 --2- 172.21.15.32:0/2276920448 >> v2:172.21.15.32:3300/0 conn(0x7fc884070230 0x7fc8840706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc888cfc640 1 --2- 172.21.15.32:0/2276920448 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc88415abb0 0x7fc8840caaf0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.170+0000 7fc888cfc640 1 -- 172.21.15.32:0/2276920448 >> 172.21.15.32:0/2276920448 conn(0x7fc88406a510 msgr2=0x7fc8840ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc888cfc640 1 -- 172.21.15.32:0/2276920448 shutdown_connections 2023-12-20T04:43:37.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc888cfc640 1 -- 172.21.15.32:0/2276920448 wait complete. 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc888cfc640 1 Processor -- start 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc888cfc640 1 -- start start 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc888cfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc884070230 0x7fc884140940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc888cfc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8840b0cb0 0x7fc884152680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc888cfc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc88415abb0 0x7fc8840ccc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc888cfc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841488b0 con 0x7fc8840b0cb0 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc888cfc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc884140e80 con 0x7fc884070230 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc888cfc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8841521b0 con 0x7fc88415abb0 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8840b0cb0 0x7fc884152680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc87bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8840b0cb0 0x7fc884152680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43512/0 (socket says 172.21.15.32:43512) 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc87bfff640 1 -- 172.21.15.32:0/3604888077 learned_addr learned my addr 172.21.15.32:0/3604888077 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc882d9d640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc88415abb0 0x7fc8840ccc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc88259c640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc884070230 0x7fc884140940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc87bfff640 1 -- 172.21.15.32:0/3604888077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc88415abb0 msgr2=0x7fc8840ccc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc87bfff640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc88415abb0 0x7fc8840ccc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc87bfff640 1 -- 172.21.15.32:0/3604888077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc884070230 msgr2=0x7fc884140940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc87bfff640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc884070230 0x7fc884140940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.174+0000 7fc87bfff640 1 -- 172.21.15.32:0/3604888077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc874009420 con 0x7fc8840b0cb0 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc87bfff640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8840b0cb0 0x7fc884152680 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0011f0 tx=0x7fc86c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc87affd640 1 -- 172.21.15.32:0/3604888077 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c0074e0 con 0x7fc8840b0cb0 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc87affd640 1 -- 172.21.15.32:0/3604888077 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc86c007640 con 0x7fc8840b0cb0 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc87affd640 1 -- 172.21.15.32:0/3604888077 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc86c004a10 con 0x7fc8840b0cb0 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc888cfc640 1 -- 172.21.15.32:0/3604888077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc884148b00 con 0x7fc8840b0cb0 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc888cfc640 1 -- 172.21.15.32:0/3604888077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8841490c0 con 0x7fc8840b0cb0 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc888cfc640 1 -- 172.21.15.32:0/3604888077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8840b4950 con 0x7fc8840b0cb0 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc87affd640 1 -- 172.21.15.32:0/3604888077 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fc86c004b70 con 0x7fc8840b0cb0 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc87affd640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc864065b80 0x7fc864068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc87affd640 1 -- 172.21.15.32:0/3604888077 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fc86c0827e0 con 0x7fc8840b0cb0 2023-12-20T04:43:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc88259c640 1 -- 172.21.15.32:0/3604888077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc864065b80 msgr2=0x7fc864068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.178+0000 7fc88259c640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc864065b80 0x7fc864068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:37.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.182+0000 7fc87affd640 1 -- 172.21.15.32:0/3604888077 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc86c04feb0 con 0x7fc8840b0cb0 2023-12-20T04:43:37.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 -- 172.21.15.32:0/3604888077 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fc884152490 con 0x7fc8840b0cb0 2023-12-20T04:43:37.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc87affd640 1 -- 172.21.15.32:0/3604888077 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7fc86c0180a0 con 0x7fc8840b0cb0 2023-12-20T04:43:37.340 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:43:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 -- 172.21.15.32:0/3604888077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc864065b80 msgr2=0x7fc864068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc864065b80 0x7fc864068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 -- 172.21.15.32:0/3604888077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8840b0cb0 msgr2=0x7fc884152680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8840b0cb0 0x7fc884152680 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7fc86c0011f0 tx=0x7fc86c00c9b0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 -- 172.21.15.32:0/3604888077 shutdown_connections 2023-12-20T04:43:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc864065b80 0x7fc864068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc88415abb0 0x7fc8840ccc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8840b0cb0 0x7fc884152680 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 --2- 172.21.15.32:0/3604888077 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc884070230 0x7fc884140940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 -- 172.21.15.32:0/3604888077 >> 172.21.15.32:0/3604888077 conn(0x7fc88406a510 msgr2=0x7fc884149820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 -- 172.21.15.32:0/3604888077 shutdown_connections 2023-12-20T04:43:37.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.342+0000 7fc888cfc640 1 -- 172.21.15.32:0/3604888077 wait complete. 2023-12-20T04:43:37.351 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:37.352+0000 7f3c8b02d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:37.362 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:43:37.404 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:37.406+0000 7f8e47247180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:37.413 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:37.412+0000 7f3c8b02d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:43:37.467 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:37.470+0000 7f8e47247180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:43:37.471 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:37.472+0000 7f3c8b02d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:43:37.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.510+0000 7f37daef9640 1 Processor -- start 2023-12-20T04:43:37.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.510+0000 7f37daef9640 1 -- start start 2023-12-20T04:43:37.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.510+0000 7f37daef9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 0x7f37d40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.510+0000 7f37daef9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f37d4070230 0x7f37d40706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37daef9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37d415b6a0 con 0x7f37d415abb0 2023-12-20T04:43:37.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37daef9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37d406e160 con 0x7f37d4070230 2023-12-20T04:43:37.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37daef9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4070be0 con 0x7f37d40b0cb0 2023-12-20T04:43:37.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37d8c95640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f37d415abb0 0x7f37d415bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54668/0 (socket says 172.21.15.32:54668) 2023-12-20T04:43:37.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37d8c95640 1 -- 172.21.15.32:0/3730400176 learned_addr learned my addr 172.21.15.32:0/3730400176 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:37.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37d3fff640 1 --2- 172.21.15.32:0/3730400176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 0x7f37d40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37d9496640 1 --2- 172.21.15.32:0/3730400176 >> v2:172.21.15.39:3300/0 conn(0x7f37d4070230 0x7f37d40706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37d37fe640 1 -- 172.21.15.32:0/3730400176 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1370454619 0 0) 0x7f37d415b6a0 con 0x7f37d415abb0 2023-12-20T04:43:37.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37d37fe640 1 -- 172.21.15.32:0/3730400176 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f37c4003530 con 0x7f37d415abb0 2023-12-20T04:43:37.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37d3fff640 1 -- 172.21.15.32:0/3730400176 >> v2:172.21.15.39:3300/0 conn(0x7f37d4070230 msgr2=0x7f37d40706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:37.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37d3fff640 1 --2- 172.21.15.32:0/3730400176 >> v2:172.21.15.39:3300/0 conn(0x7f37d4070230 0x7f37d40706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37d3fff640 1 -- 172.21.15.32:0/3730400176 >> v1:172.21.15.32:6789/0 conn(0x7f37d415abb0 legacy=0x7f37d415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37d3fff640 1 -- 172.21.15.32:0/3730400176 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37d406d8c0 con 0x7f37d40b0cb0 2023-12-20T04:43:37.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.514+0000 7f37d9496640 1 --2- 172.21.15.32:0/3730400176 >> v2:172.21.15.39:3300/0 conn(0x7f37d4070230 0x7f37d40706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:37.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37d3fff640 1 --2- 172.21.15.32:0/3730400176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 0x7f37d40caaf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f37c00097c0 tx=0x7f37c001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebd1099ba25b1d79 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:37.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37d37fe640 1 -- 172.21.15.32:0/3730400176 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c00ab680 con 0x7f37d40b0cb0 2023-12-20T04:43:37.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37d37fe640 1 -- 172.21.15.32:0/3730400176 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c00ab7e0 con 0x7f37d40b0cb0 2023-12-20T04:43:37.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37d37fe640 1 -- 172.21.15.32:0/3730400176 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c00abb50 con 0x7f37d40b0cb0 2023-12-20T04:43:37.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37daef9640 1 -- 172.21.15.32:0/3730400176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 msgr2=0x7f37d40caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37daef9640 1 --2- 172.21.15.32:0/3730400176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 0x7f37d40caaf0 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f37c00097c0 tx=0x7f37c001c480 comp rx=0 tx=0).stop 2023-12-20T04:43:37.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37daef9640 1 -- 172.21.15.32:0/3730400176 shutdown_connections 2023-12-20T04:43:37.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37daef9640 1 --2- 172.21.15.32:0/3730400176 >> v2:172.21.15.39:3300/0 conn(0x7f37d4070230 0x7f37d40706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37daef9640 1 --2- 172.21.15.32:0/3730400176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 0x7f37d40caaf0 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37daef9640 1 -- 172.21.15.32:0/3730400176 >> 172.21.15.32:0/3730400176 conn(0x7f37d406a510 msgr2=0x7f37d40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:37.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37daef9640 1 -- 172.21.15.32:0/3730400176 shutdown_connections 2023-12-20T04:43:37.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.518+0000 7f37daef9640 1 -- 172.21.15.32:0/3730400176 wait complete. 2023-12-20T04:43:37.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37daef9640 1 Processor -- start 2023-12-20T04:43:37.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37daef9640 1 -- start start 2023-12-20T04:43:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37daef9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37d4070230 0x7f37d4159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37daef9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 0x7f37d4159bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37daef9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37d415abb0 0x7f37d414bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37daef9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4142b70 con 0x7f37d415abb0 2023-12-20T04:43:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37daef9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37d41425b0 con 0x7f37d4070230 2023-12-20T04:43:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37daef9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37d4142890 con 0x7f37d40b0cb0 2023-12-20T04:43:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d9496640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37d415abb0 0x7f37d414bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d3fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 0x7f37d4159bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d9496640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37d415abb0 0x7f37d414bb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43522/0 (socket says 172.21.15.32:43522) 2023-12-20T04:43:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d9496640 1 -- 172.21.15.32:0/696674076 learned_addr learned my addr 172.21.15.32:0/696674076 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:37.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d8c95640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37d4070230 0x7f37d4159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d3fff640 1 -- 172.21.15.32:0/696674076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37d4070230 msgr2=0x7f37d4159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d3fff640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37d4070230 0x7f37d4159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d3fff640 1 -- 172.21.15.32:0/696674076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37d415abb0 msgr2=0x7f37d414bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d3fff640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37d415abb0 0x7f37d414bb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d3fff640 1 -- 172.21.15.32:0/696674076 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37cc009a70 con 0x7f37d40b0cb0 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d9496640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37d415abb0 0x7f37d414bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d3fff640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 0x7f37d4159bb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f37c00ac000 tx=0x7f37c001acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d17fa640 1 -- 172.21.15.32:0/696674076 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c00bf610 con 0x7f37d40b0cb0 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d17fa640 1 -- 172.21.15.32:0/696674076 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37c00bf770 con 0x7f37d40b0cb0 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37daef9640 1 -- 172.21.15.32:0/696674076 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37c0009420 con 0x7f37d40b0cb0 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37d17fa640 1 -- 172.21.15.32:0/696674076 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37c00bfb00 con 0x7f37d40b0cb0 2023-12-20T04:43:37.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.522+0000 7f37daef9640 1 -- 172.21.15.32:0/696674076 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37d414b5f0 con 0x7f37d40b0cb0 2023-12-20T04:43:37.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.526+0000 7f37daef9640 1 -- 172.21.15.32:0/696674076 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f37d404e000 con 0x7f37d40b0cb0 2023-12-20T04:43:37.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.526+0000 7f37d17fa640 1 -- 172.21.15.32:0/696674076 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f37c00b5400 con 0x7f37d40b0cb0 2023-12-20T04:43:37.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.526+0000 7f37d17fa640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37b8065b80 0x7f37b8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.526+0000 7f37d17fa640 1 -- 172.21.15.32:0/696674076 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f37c011e360 con 0x7f37d40b0cb0 2023-12-20T04:43:37.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.526+0000 7f37d8c95640 1 -- 172.21.15.32:0/696674076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37b8065b80 msgr2=0x7f37b8068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:37.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.526+0000 7f37d8c95640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37b8065b80 0x7f37b8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:37.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.530+0000 7f37d17fa640 1 -- 172.21.15.32:0/696674076 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f37c00eba40 con 0x7f37d40b0cb0 2023-12-20T04:43:37.531 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:37.534+0000 7f8e47247180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:43:37.562 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:37.560+0000 7f3c8b02d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:43:37.623 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:37.624+0000 7f3c8b02d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:43:37.626 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:37.626+0000 7f8e47247180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:43:37.687 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:37.686+0000 7f8e47247180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:43:37.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.690+0000 7f37daef9640 1 -- 172.21.15.32:0/696674076 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f37d4141960 con 0x7f37d40b0cb0 2023-12-20T04:43:37.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.690+0000 7f37d17fa640 1 -- 172.21.15.32:0/696674076 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7f37c00ef060 con 0x7f37d40b0cb0 2023-12-20T04:43:37.691 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:43:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 -- 172.21.15.32:0/696674076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37b8065b80 msgr2=0x7f37b8068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37b8065b80 0x7f37b8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 -- 172.21.15.32:0/696674076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 msgr2=0x7f37d4159bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 0x7f37d4159bb0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f37c00ac000 tx=0x7f37c001acc0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 -- 172.21.15.32:0/696674076 shutdown_connections 2023-12-20T04:43:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f37b8065b80 0x7f37b8068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37d415abb0 0x7f37d414bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37d40b0cb0 0x7f37d4159bb0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 --2- 172.21.15.32:0/696674076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37d4070230 0x7f37d4159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 -- 172.21.15.32:0/696674076 >> 172.21.15.32:0/696674076 conn(0x7f37d406a510 msgr2=0x7f37d415bb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 -- 172.21.15.32:0/696674076 shutdown_connections 2023-12-20T04:43:37.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.694+0000 7f37daef9640 1 -- 172.21.15.32:0/696674076 wait complete. 2023-12-20T04:43:37.711 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:37.712+0000 7f3c8b02d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:43:37.714 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T04:43:37.770 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:37.768+0000 7f3c8b02d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:43:37.778 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:37.778+0000 7f8e47247180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:43:37.839 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:37.842+0000 7f8e47247180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:43:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f968a28c640 1 Processor -- start 2023-12-20T04:43:37.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f968a28c640 1 -- start start 2023-12-20T04:43:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f968a28c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f968406ce60 0x7f968415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f968a28c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968415bbd0 con 0x7f96840af5e0 2023-12-20T04:43:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f968a28c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f968415beb0 con 0x7f968415f110 2023-12-20T04:43:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f968a28c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f968415e910 con 0x7f968406ce60 2023-12-20T04:43:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f9683fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f96840af5e0 0x7f96840af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54678/0 (socket says 172.21.15.32:54678) 2023-12-20T04:43:37.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f9683fff640 1 -- 172.21.15.32:0/2697234337 learned_addr learned my addr 172.21.15.32:0/2697234337 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f96837fe640 1 --2- 172.21.15.32:0/2697234337 >> v2:172.21.15.39:3300/0 conn(0x7f968406ce60 0x7f968415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f9682ffd640 1 -- 172.21.15.32:0/2697234337 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2378631376 0 0) 0x7f968415bbd0 con 0x7f96840af5e0 2023-12-20T04:43:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f9682ffd640 1 -- 172.21.15.32:0/2697234337 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f966c003530 con 0x7f96840af5e0 2023-12-20T04:43:37.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f96837fe640 1 -- 172.21.15.32:0/2697234337 >> v1:172.21.15.39:6789/0 conn(0x7f968415f110 legacy=0x7f968415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f9682ffd640 1 -- 172.21.15.32:0/2697234337 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3621289909 0 0) 0x7f968415beb0 con 0x7f968415f110 2023-12-20T04:43:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f96837fe640 1 -- 172.21.15.32:0/2697234337 >> v1:172.21.15.32:6789/0 conn(0x7f96840af5e0 legacy=0x7f96840af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.902+0000 7f96837fe640 1 -- 172.21.15.32:0/2697234337 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f968415ebf0 con 0x7f968406ce60 2023-12-20T04:43:37.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.906+0000 7f96837fe640 1 --2- 172.21.15.32:0/2697234337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f968406ce60 0x7f968415b280 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f96780097c0 tx=0x7f967801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d93ae75fef707110 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.906+0000 7f9682ffd640 1 -- 172.21.15.32:0/2697234337 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96780a8050 con 0x7f968406ce60 2023-12-20T04:43:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.906+0000 7f9682ffd640 1 -- 172.21.15.32:0/2697234337 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96780ac5f0 con 0x7f968406ce60 2023-12-20T04:43:37.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.906+0000 7f9682ffd640 1 -- 172.21.15.32:0/2697234337 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f968415b9d0 con 0x7f968406ce60 2023-12-20T04:43:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.906+0000 7f968a28c640 1 -- 172.21.15.32:0/2697234337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f968406ce60 msgr2=0x7f968415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.906+0000 7f968a28c640 1 --2- 172.21.15.32:0/2697234337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f968406ce60 0x7f968415b280 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f96780097c0 tx=0x7f967801c3f0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.906+0000 7f968a28c640 1 -- 172.21.15.32:0/2697234337 shutdown_connections 2023-12-20T04:43:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.906+0000 7f968a28c640 1 --2- 172.21.15.32:0/2697234337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f968406ce60 0x7f968415b280 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.906+0000 7f968a28c640 1 -- 172.21.15.32:0/2697234337 >> 172.21.15.32:0/2697234337 conn(0x7f968406a320 msgr2=0x7f96840ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.906+0000 7f968a28c640 1 -- 172.21.15.32:0/2697234337 shutdown_connections 2023-12-20T04:43:37.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.906+0000 7f968a28c640 1 -- 172.21.15.32:0/2697234337 wait complete. 2023-12-20T04:43:37.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f968a28c640 1 Processor -- start 2023-12-20T04:43:37.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f968a28c640 1 -- start start 2023-12-20T04:43:37.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f968a28c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968406ce60 0x7f968414f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f968a28c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96840af5e0 0x7f968415cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f968a28c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f968415f110 0x7f968415d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f968a28c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9684070d00 con 0x7f968406ce60 2023-12-20T04:43:37.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f968a28c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96840cbf50 con 0x7f968415f110 2023-12-20T04:43:37.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f968a28c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f968414fe30 con 0x7f96840af5e0 2023-12-20T04:43:37.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9683fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968406ce60 0x7f968414f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9683fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968406ce60 0x7f968414f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43524/0 (socket says 172.21.15.32:43524) 2023-12-20T04:43:37.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9683fff640 1 -- 172.21.15.32:0/2576012420 learned_addr learned my addr 172.21.15.32:0/2576012420 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:37.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f96837fe640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96840af5e0 0x7f968415cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9688829640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f968415f110 0x7f968415d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:37.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9683fff640 1 -- 172.21.15.32:0/2576012420 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96840af5e0 msgr2=0x7f968415cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9683fff640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96840af5e0 0x7f968415cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9683fff640 1 -- 172.21.15.32:0/2576012420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f968415f110 msgr2=0x7f968415d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:37.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9683fff640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f968415f110 0x7f968415d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:37.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9683fff640 1 -- 172.21.15.32:0/2576012420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9678009420 con 0x7f968406ce60 2023-12-20T04:43:37.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9683fff640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968406ce60 0x7f968414f8f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f967400c730 tx=0x7f967400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9680ff9640 1 -- 172.21.15.32:0/2576012420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674007270 con 0x7f968406ce60 2023-12-20T04:43:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9680ff9640 1 -- 172.21.15.32:0/2576012420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96740073d0 con 0x7f968406ce60 2023-12-20T04:43:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f9680ff9640 1 -- 172.21.15.32:0/2576012420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9674001d50 con 0x7f968406ce60 2023-12-20T04:43:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f968a28c640 1 -- 172.21.15.32:0/2576012420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9684070360 con 0x7f968406ce60 2023-12-20T04:43:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.910+0000 7f968a28c640 1 -- 172.21.15.32:0/2576012420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9684070920 con 0x7f968406ce60 2023-12-20T04:43:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.914+0000 7f968a28c640 1 -- 172.21.15.32:0/2576012420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9684070e40 con 0x7f968406ce60 2023-12-20T04:43:37.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.914+0000 7f9680ff9640 1 -- 172.21.15.32:0/2576012420 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f96740057b0 con 0x7f968406ce60 2023-12-20T04:43:37.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.914+0000 7f9680ff9640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9670065b80 0x7f9670068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:37.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.914+0000 7f96837fe640 1 -- 172.21.15.32:0/2576012420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9670065b80 msgr2=0x7f9670068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:37.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.914+0000 7f9680ff9640 1 -- 172.21.15.32:0/2576012420 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f9674082910 con 0x7f968406ce60 2023-12-20T04:43:37.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.914+0000 7f96837fe640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9670065b80 0x7f9670068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:37.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:37.918+0000 7f9680ff9640 1 -- 172.21.15.32:0/2576012420 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9684070e40 con 0x7f968406ce60 2023-12-20T04:43:37.920 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:37.920+0000 7f3c8b02d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:43:37.982 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:37.980+0000 7f3c8b02d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:43:38.001 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:38.002+0000 7f8e47247180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:43:38.065 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:38.066+0000 7f8e47247180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:43:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.094+0000 7f968a28c640 1 -- 172.21.15.32:0/2576012420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f9684070e40 con 0x7f968406ce60 2023-12-20T04:43:38.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.094+0000 7f9680ff9640 1 -- 172.21.15.32:0/2576012420 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v80) v1 ==== 80+0+460 (secure 0 0 0) 0x7f9684070e40 con 0x7f968406ce60 2023-12-20T04:43:38.095 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2610637012 2023-12-21T04:43:34.035676+0000 2023-12-20T04:43:38.095 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2158383566 2023-12-21T04:43:34.035676+0000 2023-12-20T04:43:38.095 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/38009 2023-12-21T04:43:34.035676+0000 2023-12-20T04:43:38.095 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/38009 2023-12-21T04:43:34.035676+0000 2023-12-20T04:43:38.095 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2830398071 2023-12-21T04:43:34.035676+0000 2023-12-20T04:43:38.095 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3257133891 2023-12-21T04:43:34.035676+0000 2023-12-20T04:43:38.095 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2848355101 2023-12-21T04:43:34.035676+0000 2023-12-20T04:43:38.095 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1251934730 2023-12-21T04:43:34.035676+0000 2023-12-20T04:43:38.095 INFO:teuthology.orchestra.run.smithi032.stderr:listed 8 entries 2023-12-20T04:43:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 -- 172.21.15.32:0/2576012420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9670065b80 msgr2=0x7f9670068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9670065b80 0x7f9670068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 -- 172.21.15.32:0/2576012420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968406ce60 msgr2=0x7f968414f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968406ce60 0x7f968414f8f0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f967400c730 tx=0x7f967400cbf0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 -- 172.21.15.32:0/2576012420 shutdown_connections 2023-12-20T04:43:38.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9670065b80 0x7f9670068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f968415f110 0x7f968415d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96840af5e0 0x7f968415cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 --2- 172.21.15.32:0/2576012420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f968406ce60 0x7f968414f8f0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 -- 172.21.15.32:0/2576012420 >> 172.21.15.32:0/2576012420 conn(0x7f968406a320 msgr2=0x7f9684155ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:38.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 -- 172.21.15.32:0/2576012420 shutdown_connections 2023-12-20T04:43:38.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.098+0000 7f968a28c640 1 -- 172.21.15.32:0/2576012420 wait complete. 2023-12-20T04:43:38.117 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T04:43:38.117 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:38.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T04:43:38.189 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:43:38.189 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:43:38.189 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T04:43:38.236 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:43:38.237 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:43:38.237 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:43:38.245 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:38.244+0000 7f3c8b02d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:43:38.258 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:43:38.258 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:38.267 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:43:38.267 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:43:38.267 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:38.309 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:43:38.309 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:43:38.309 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:38.318 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:43:38.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:38.334 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:43:38.334 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:43:38.334 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:43:38.343 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:38.342+0000 7f8e47247180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:43:38.388 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:38.388+0000 7f3c8b02d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:38.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.478+0000 7fd37a77b640 1 Processor -- start 2023-12-20T04:43:38.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.478+0000 7fd37a77b640 1 -- start start 2023-12-20T04:43:38.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.478+0000 7fd37a77b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 0x7fd3740d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:38.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.478+0000 7fd37a77b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd37415d190 con 0x7fd3740cfc80 2023-12-20T04:43:38.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.478+0000 7fd37a77b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3740d16a0 con 0x7fd3740d2740 2023-12-20T04:43:38.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.478+0000 7fd37a77b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3740d1980 con 0x7fd37415cdb0 2023-12-20T04:43:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd378d18640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd3740cfc80 0x7fd3740b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54684/0 (socket says 172.21.15.32:54684) 2023-12-20T04:43:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd378d18640 1 -- 172.21.15.32:0/2499013656 learned_addr learned my addr 172.21.15.32:0/2499013656 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd373fff640 1 --2- 172.21.15.32:0/2499013656 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 0x7fd3740d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd3737fe640 1 --1- 172.21.15.32:0/2499013656 >> v1:172.21.15.39:6789/0 conn(0x7fd3740d2740 0x7fd3740cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:38136/0 (socket says 172.21.15.32:38136) 2023-12-20T04:43:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd373fff640 1 -- 172.21.15.32:0/2499013656 >> v1:172.21.15.39:6789/0 conn(0x7fd3740d2740 legacy=0x7fd3740cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd373fff640 1 -- 172.21.15.32:0/2499013656 >> v1:172.21.15.32:6789/0 conn(0x7fd3740cfc80 legacy=0x7fd3740b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd372ffd640 1 -- 172.21.15.32:0/2499013656 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1424533688 0 0) 0x7fd3740d16a0 con 0x7fd3740d2740 2023-12-20T04:43:38.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd373fff640 1 -- 172.21.15.32:0/2499013656 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3740d1c60 con 0x7fd37415cdb0 2023-12-20T04:43:38.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd373fff640 1 --2- 172.21.15.32:0/2499013656 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 0x7fd3740d0be0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3640097f0 tx=0x7fd36401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7ae455755dbd9ac4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:38.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd372ffd640 1 -- 172.21.15.32:0/2499013656 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3640ad680 con 0x7fd37415cdb0 2023-12-20T04:43:38.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd372ffd640 1 -- 172.21.15.32:0/2499013656 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3640ad7e0 con 0x7fd37415cdb0 2023-12-20T04:43:38.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.482+0000 7fd372ffd640 1 -- 172.21.15.32:0/2499013656 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3640adb70 con 0x7fd37415cdb0 2023-12-20T04:43:38.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 -- 172.21.15.32:0/2499013656 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 msgr2=0x7fd3740d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:38.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 --2- 172.21.15.32:0/2499013656 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 0x7fd3740d0be0 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7fd3640097f0 tx=0x7fd36401c3e0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 -- 172.21.15.32:0/2499013656 shutdown_connections 2023-12-20T04:43:38.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 --2- 172.21.15.32:0/2499013656 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 0x7fd3740d0be0 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 -- 172.21.15.32:0/2499013656 >> 172.21.15.32:0/2499013656 conn(0x7fd374065ee0 msgr2=0x7fd37406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:38.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 -- 172.21.15.32:0/2499013656 shutdown_connections 2023-12-20T04:43:38.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 -- 172.21.15.32:0/2499013656 wait complete. 2023-12-20T04:43:38.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 Processor -- start 2023-12-20T04:43:38.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 -- start start 2023-12-20T04:43:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3740cfc80 0x7fd374140e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740d2740 0x7fd37415d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 0x7fd3740cbe00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd37415e450 con 0x7fd3740d2740 2023-12-20T04:43:38.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd374141380 con 0x7fd3740cfc80 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd37a77b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3741f2b20 con 0x7fd37415cdb0 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd3737fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740d2740 0x7fd37415d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd3737fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740d2740 0x7fd37415d3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43528/0 (socket says 172.21.15.32:43528) 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd3737fe640 1 -- 172.21.15.32:0/382329398 learned_addr learned my addr 172.21.15.32:0/382329398 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.486+0000 7fd373fff640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3740cfc80 0x7fd374140e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd378d18640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 0x7fd3740cbe00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd3737fe640 1 -- 172.21.15.32:0/382329398 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 msgr2=0x7fd3740cbe00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd3737fe640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 0x7fd3740cbe00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd3737fe640 1 -- 172.21.15.32:0/382329398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3740cfc80 msgr2=0x7fd374140e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd3737fe640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3740cfc80 0x7fd374140e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd3737fe640 1 -- 172.21.15.32:0/382329398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd364009420 con 0x7fd3740d2740 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd378d18640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 0x7fd3740cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:38.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd3737fe640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740d2740 0x7fd37415d3d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd36800c520 tx=0x7fd36800c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:38.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd370ff9640 1 -- 172.21.15.32:0/382329398 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680074e0 con 0x7fd3740d2740 2023-12-20T04:43:38.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd370ff9640 1 -- 172.21.15.32:0/382329398 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd368007640 con 0x7fd3740d2740 2023-12-20T04:43:38.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd370ff9640 1 -- 172.21.15.32:0/382329398 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3680049f0 con 0x7fd3740d2740 2023-12-20T04:43:38.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd37a77b640 1 -- 172.21.15.32:0/382329398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd37415de90 con 0x7fd3740d2740 2023-12-20T04:43:38.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd37a77b640 1 -- 172.21.15.32:0/382329398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd37414b010 con 0x7fd3740d2740 2023-12-20T04:43:38.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd37a77b640 1 -- 172.21.15.32:0/382329398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd37414ae70 con 0x7fd3740d2740 2023-12-20T04:43:38.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd370ff9640 1 -- 172.21.15.32:0/382329398 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fd368004b50 con 0x7fd3740d2740 2023-12-20T04:43:38.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd370ff9640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd360065b80 0x7fd360068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:38.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd373fff640 1 -- 172.21.15.32:0/382329398 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd360065b80 msgr2=0x7fd360068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:38.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd370ff9640 1 -- 172.21.15.32:0/382329398 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fd368082550 con 0x7fd3740d2740 2023-12-20T04:43:38.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.490+0000 7fd373fff640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd360065b80 0x7fd360068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:38.492 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:38.494+0000 7f8e47247180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:38.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.494+0000 7fd370ff9640 1 -- 172.21.15.32:0/382329398 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd37414ae70 con 0x7fd3740d2740 2023-12-20T04:43:38.518 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:38.516+0000 7f3c8b02d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:38.592 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:38.592+0000 7f3c8b02d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:43:38.626 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:38.626+0000 7f8e47247180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:43:38.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 -- 172.21.15.32:0/382329398 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd37414ae70 con 0x7fd3740d2740 2023-12-20T04:43:38.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd370ff9640 1 -- 172.21.15.32:0/382329398 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4706 (secure 0 0 0) 0x7fd37414ae70 con 0x7fd3740d2740 2023-12-20T04:43:38.685 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:38.685 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5780,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4279891602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4279891602},{"type":"v1","addr":"172.21.15.39:6834","nonce":4279891602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6292,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3937321478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3937321478},{"type":"v1","addr":"172.21.15.39:6835","nonce":3937321478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/404004710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":404004710},{"type":"v1","addr":"172.21.15.32:6835","nonce":404004710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:21.971704+0000","modified":"2023-12-20T04:43:24.982163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3343723618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3343723618},{"type":"v1","addr":"172.21.15.32:6837","nonce":3343723618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-20T04:43:38.685 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2023-12-20T04:43:38.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 -- 172.21.15.32:0/382329398 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd360065b80 msgr2=0x7fd360068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:38.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd360065b80 0x7fd360068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 -- 172.21.15.32:0/382329398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740d2740 msgr2=0x7fd37415d3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:38.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740d2740 0x7fd37415d3d0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7fd36800c520 tx=0x7fd36800c9e0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 -- 172.21.15.32:0/382329398 shutdown_connections 2023-12-20T04:43:38.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd360065b80 0x7fd360068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd37415cdb0 0x7fd3740cbe00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3740d2740 0x7fd37415d3d0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 --2- 172.21.15.32:0/382329398 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3740cfc80 0x7fd374140e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 -- 172.21.15.32:0/382329398 >> 172.21.15.32:0/382329398 conn(0x7fd374065ee0 msgr2=0x7fd3740cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.686+0000 7fd37a77b640 1 -- 172.21.15.32:0/382329398 shutdown_connections 2023-12-20T04:43:38.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.690+0000 7fd37a77b640 1 -- 172.21.15.32:0/382329398 wait complete. 2023-12-20T04:43:38.703 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:38.706+0000 7f8e47247180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:43:38.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:38.734 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:38.732+0000 7f3c8b02d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:38.850 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:38.850+0000 7f8e47247180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:43:38.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746e3f3640 1 Processor -- start 2023-12-20T04:43:38.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746e3f3640 1 -- start start 2023-12-20T04:43:38.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746e3f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f746815d9a0 0x7f746806c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:38.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746e3f3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f746806c920 0x7f74680705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:38.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746e3f3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74680ae140 con 0x7f74681593c0 2023-12-20T04:43:38.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746e3f3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74680ae9e0 con 0x7f746806c920 2023-12-20T04:43:38.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746e3f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f746806cdb0 con 0x7f746815d9a0 2023-12-20T04:43:38.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f7467fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f74681593c0 0x7f74680afdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59268/0 (socket says 172.21.15.32:59268) 2023-12-20T04:43:38.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f7467fff640 1 -- 172.21.15.32:0/4067514008 learned_addr learned my addr 172.21.15.32:0/4067514008 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:38.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746c990640 1 --2- 172.21.15.32:0/4067514008 >> v2:172.21.15.39:3300/0 conn(0x7f746806c920 0x7f74680705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:38.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f74677fe640 1 --2- 172.21.15.32:0/4067514008 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f746815d9a0 0x7f746806c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:38.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f7466ffd640 1 -- 172.21.15.32:0/4067514008 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 481341778 0 0) 0x7f74680ae140 con 0x7f74681593c0 2023-12-20T04:43:38.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f7466ffd640 1 -- 172.21.15.32:0/4067514008 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7450003530 con 0x7f74681593c0 2023-12-20T04:43:38.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746c990640 1 -- 172.21.15.32:0/4067514008 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f746815d9a0 msgr2=0x7f746806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:38.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746c990640 1 --2- 172.21.15.32:0/4067514008 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f746815d9a0 0x7f746806c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746c990640 1 -- 172.21.15.32:0/4067514008 >> v1:172.21.15.32:6789/0 conn(0x7f74681593c0 legacy=0x7f74680afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:38.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.898+0000 7f746c990640 1 -- 172.21.15.32:0/4067514008 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7468072170 con 0x7f746806c920 2023-12-20T04:43:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f746c990640 1 --2- 172.21.15.32:0/4067514008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f746806c920 0x7f74680705a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7460009d90 tx=0x7f746001c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf271e6dc5e3e710 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:38.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f7466ffd640 1 -- 172.21.15.32:0/4067514008 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74600a5840 con 0x7f746806c920 2023-12-20T04:43:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f7466ffd640 1 -- 172.21.15.32:0/4067514008 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74600a59a0 con 0x7f746806c920 2023-12-20T04:43:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f7466ffd640 1 -- 172.21.15.32:0/4067514008 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74600a5d10 con 0x7f746806c920 2023-12-20T04:43:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f746e3f3640 1 -- 172.21.15.32:0/4067514008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f746806c920 msgr2=0x7f74680705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:38.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f746e3f3640 1 --2- 172.21.15.32:0/4067514008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f746806c920 0x7f74680705a0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f7460009d90 tx=0x7f746001c800 comp rx=0 tx=0).stop 2023-12-20T04:43:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f746e3f3640 1 -- 172.21.15.32:0/4067514008 shutdown_connections 2023-12-20T04:43:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f746e3f3640 1 --2- 172.21.15.32:0/4067514008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f746806c920 0x7f74680705a0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f746e3f3640 1 --2- 172.21.15.32:0/4067514008 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f746815d9a0 0x7f746806c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f746e3f3640 1 -- 172.21.15.32:0/4067514008 >> 172.21.15.32:0/4067514008 conn(0x7f746806a320 msgr2=0x7f74680ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f746e3f3640 1 -- 172.21.15.32:0/4067514008 shutdown_connections 2023-12-20T04:43:38.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.902+0000 7f746e3f3640 1 -- 172.21.15.32:0/4067514008 wait complete. 2023-12-20T04:43:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f746e3f3640 1 Processor -- start 2023-12-20T04:43:38.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f746e3f3640 1 -- start start 2023-12-20T04:43:38.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f746e3f3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f746806c920 0x7f7468155bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f746e3f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74681593c0 0x7f74681586d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f746e3f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f746815d9a0 0x7f74680b3c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f746e3f3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f746814c320 con 0x7f746806c920 2023-12-20T04:43:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f746e3f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7468158c10 con 0x7f74681593c0 2023-12-20T04:43:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f746e3f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7468156280 con 0x7f746815d9a0 2023-12-20T04:43:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7467fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f746806c920 0x7f7468155bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7467fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f746806c920 0x7f7468155bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44956/0 (socket says 172.21.15.32:44956) 2023-12-20T04:43:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7467fff640 1 -- 172.21.15.32:0/2702422863 learned_addr learned my addr 172.21.15.32:0/2702422863 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f74677fe640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74681593c0 0x7f74681586d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f746c990640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f746815d9a0 0x7f74680b3c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:38.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7467fff640 1 -- 172.21.15.32:0/2702422863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f746815d9a0 msgr2=0x7f74680b3c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7467fff640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f746815d9a0 0x7f74680b3c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7467fff640 1 -- 172.21.15.32:0/2702422863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74681593c0 msgr2=0x7f74681586d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7467fff640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74681593c0 0x7f74681586d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7467fff640 1 -- 172.21.15.32:0/2702422863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7460009a70 con 0x7f746806c920 2023-12-20T04:43:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7467fff640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f746806c920 0x7f7468155bf0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f745800c730 tx=0x7f745800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7464ff9640 1 -- 172.21.15.32:0/2702422863 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7458007270 con 0x7f746806c920 2023-12-20T04:43:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7464ff9640 1 -- 172.21.15.32:0/2702422863 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74580073d0 con 0x7f746806c920 2023-12-20T04:43:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f746e3f3640 1 -- 172.21.15.32:0/2702422863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f746814c570 con 0x7f746806c920 2023-12-20T04:43:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f7464ff9640 1 -- 172.21.15.32:0/2702422863 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7458001d50 con 0x7f746806c920 2023-12-20T04:43:38.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.906+0000 7f746e3f3640 1 -- 172.21.15.32:0/2702422863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f746814cb30 con 0x7f746806c920 2023-12-20T04:43:38.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.910+0000 7f746e3f3640 1 -- 172.21.15.32:0/2702422863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f746804e000 con 0x7f746806c920 2023-12-20T04:43:38.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.910+0000 7f7464ff9640 1 -- 172.21.15.32:0/2702422863 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f74580057b0 con 0x7f746806c920 2023-12-20T04:43:38.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.910+0000 7f7464ff9640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7454065b80 0x7f7454068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:38.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.910+0000 7f7464ff9640 1 -- 172.21.15.32:0/2702422863 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f7458082a20 con 0x7f746806c920 2023-12-20T04:43:38.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.910+0000 7f74677fe640 1 -- 172.21.15.32:0/2702422863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7454065b80 msgr2=0x7f7454068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:38.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.910+0000 7f74677fe640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7454065b80 0x7f7454068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:38.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:38.914+0000 7f7464ff9640 1 -- 172.21.15.32:0/2702422863 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f74580501a0 con 0x7f746806c920 2023-12-20T04:43:39.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.078+0000 7f746e3f3640 1 -- 172.21.15.32:0/2702422863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f74680ca5f0 con 0x7f746806c920 2023-12-20T04:43:39.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.078+0000 7f7464ff9640 1 -- 172.21.15.32:0/2702422863 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v80) v1 ==== 74+0+16540 (secure 0 0 0) 0x7f7458017020 con 0x7f746806c920 2023-12-20T04:43:39.079 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:39.079 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":80,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:34.035710+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:19.734786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:20.967556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2610637012":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2158383566":"2023-12-21T04:43:34.035676+0000","172.21.15.32:6833/38009":"2023-12-21T04:43:34.035676+0000","172.21.15.32:6832/38009":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2830398071":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/3257133891":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2848355101":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/1251934730":"2023-12-21T04:43:34.035676+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:39.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 -- 172.21.15.32:0/2702422863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7454065b80 msgr2=0x7f7454068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7454065b80 0x7f7454068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 -- 172.21.15.32:0/2702422863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f746806c920 msgr2=0x7f7468155bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f746806c920 0x7f7468155bf0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f745800c730 tx=0x7f745800cbf0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 -- 172.21.15.32:0/2702422863 shutdown_connections 2023-12-20T04:43:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7454065b80 0x7f7454068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f746815d9a0 0x7f74680b3c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74681593c0 0x7f74681586d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 --2- 172.21.15.32:0/2702422863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f746806c920 0x7f7468155bf0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 -- 172.21.15.32:0/2702422863 >> 172.21.15.32:0/2702422863 conn(0x7f746806a320 msgr2=0x7f7468070e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 -- 172.21.15.32:0/2702422863 shutdown_connections 2023-12-20T04:43:39.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.082+0000 7f746e3f3640 1 -- 172.21.15.32:0/2702422863 wait complete. 2023-12-20T04:43:39.103 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 8, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:43:19.734786+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '78', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 9, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:43:20.967556+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '79', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:43:39.104 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:43:39.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.254+0000 7f9730e2d640 1 Processor -- start 2023-12-20T04:43:39.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.254+0000 7f9730e2d640 1 -- start start 2023-12-20T04:43:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f9730e2d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f972c070940 0x7f972c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f9730e2d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f972c15b6a0 con 0x7f972c0b0cb0 2023-12-20T04:43:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f9730e2d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f972c06d8c0 con 0x7f972c15abb0 2023-12-20T04:43:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f9730e2d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f972c06e160 con 0x7f972c070940 2023-12-20T04:43:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f9729d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f972c0b0cb0 0x7f972c070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59280/0 (socket says 172.21.15.32:59280) 2023-12-20T04:43:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f9729d9b640 1 -- 172.21.15.32:0/2113491426 learned_addr learned my addr 172.21.15.32:0/2113491426 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f972ad9d640 1 --2- 172.21.15.32:0/2113491426 >> v2:172.21.15.39:3300/0 conn(0x7f972c070940 0x7f972c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f972959a640 1 -- 172.21.15.32:0/2113491426 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1359170121 0 0) 0x7f972c15b6a0 con 0x7f972c0b0cb0 2023-12-20T04:43:39.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f972959a640 1 -- 172.21.15.32:0/2113491426 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9714003530 con 0x7f972c0b0cb0 2023-12-20T04:43:39.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f972ad9d640 1 -- 172.21.15.32:0/2113491426 >> v1:172.21.15.39:6789/0 conn(0x7f972c15abb0 legacy=0x7f972c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f972959a640 1 -- 172.21.15.32:0/2113491426 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4005578587 0 0) 0x7f972c06d8c0 con 0x7f972c15abb0 2023-12-20T04:43:39.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f972ad9d640 1 -- 172.21.15.32:0/2113491426 >> v1:172.21.15.32:6789/0 conn(0x7f972c0b0cb0 legacy=0x7f972c070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.258+0000 7f972ad9d640 1 -- 172.21.15.32:0/2113491426 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f972c06dba0 con 0x7f972c070940 2023-12-20T04:43:39.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.262+0000 7f972ad9d640 1 --2- 172.21.15.32:0/2113491426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c070940 0x7f972c0caaf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9724009d90 tx=0x7f972401c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93843751cc895690 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:39.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.262+0000 7f972959a640 1 -- 172.21.15.32:0/2113491426 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97240a5a10 con 0x7f972c070940 2023-12-20T04:43:39.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.262+0000 7f972959a640 1 -- 172.21.15.32:0/2113491426 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97240a5b70 con 0x7f972c070940 2023-12-20T04:43:39.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.262+0000 7f972959a640 1 -- 172.21.15.32:0/2113491426 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97240ad440 con 0x7f972c070940 2023-12-20T04:43:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.262+0000 7f9730e2d640 1 -- 172.21.15.32:0/2113491426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c070940 msgr2=0x7f972c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.262+0000 7f9730e2d640 1 --2- 172.21.15.32:0/2113491426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c070940 0x7f972c0caaf0 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f9724009d90 tx=0x7f972401c950 comp rx=0 tx=0).stop 2023-12-20T04:43:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.262+0000 7f9730e2d640 1 -- 172.21.15.32:0/2113491426 shutdown_connections 2023-12-20T04:43:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.262+0000 7f9730e2d640 1 --2- 172.21.15.32:0/2113491426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c070940 0x7f972c0caaf0 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.262+0000 7f9730e2d640 1 -- 172.21.15.32:0/2113491426 >> 172.21.15.32:0/2113491426 conn(0x7f972c06a510 msgr2=0x7f972c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.262+0000 7f9730e2d640 1 -- 172.21.15.32:0/2113491426 shutdown_connections 2023-12-20T04:43:39.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.262+0000 7f9730e2d640 1 -- 172.21.15.32:0/2113491426 wait complete. 2023-12-20T04:43:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9730e2d640 1 Processor -- start 2023-12-20T04:43:39.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9730e2d640 1 -- start start 2023-12-20T04:43:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9730e2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c070940 0x7f972c148300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9730e2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f972c0b0cb0 0x7f972c14ca20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9730e2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c15abb0 0x7f972c14e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9730e2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c14e0b0 con 0x7f972c15abb0 2023-12-20T04:43:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9730e2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f972c148840 con 0x7f972c070940 2023-12-20T04:43:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9730e2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f972c14d4a0 con 0x7f972c0b0cb0 2023-12-20T04:43:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f972ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c15abb0 0x7f972c14e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9729d9b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f972c0b0cb0 0x7f972c14ca20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f972ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c15abb0 0x7f972c14e9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44958/0 (socket says 172.21.15.32:44958) 2023-12-20T04:43:39.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f972ad9d640 1 -- 172.21.15.32:0/1261671735 learned_addr learned my addr 172.21.15.32:0/1261671735 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:39.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f972a59c640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c070940 0x7f972c148300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9729d9b640 1 -- 172.21.15.32:0/1261671735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c070940 msgr2=0x7f972c148300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9729d9b640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c070940 0x7f972c148300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9729d9b640 1 -- 172.21.15.32:0/1261671735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c15abb0 msgr2=0x7f972c14e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9729d9b640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c15abb0 0x7f972c14e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f9729d9b640 1 -- 172.21.15.32:0/1261671735 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9724009a70 con 0x7f972c0b0cb0 2023-12-20T04:43:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.266+0000 7f972ad9d640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c15abb0 0x7f972c14e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f9729d9b640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f972c0b0cb0 0x7f972c14ca20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f972c0b1160 tx=0x7f972000c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f971affd640 1 -- 172.21.15.32:0/1261671735 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9720005cf0 con 0x7f972c0b0cb0 2023-12-20T04:43:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f971affd640 1 -- 172.21.15.32:0/1261671735 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9720005e50 con 0x7f972c0b0cb0 2023-12-20T04:43:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f9730e2d640 1 -- 172.21.15.32:0/1261671735 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f972c14d780 con 0x7f972c0b0cb0 2023-12-20T04:43:39.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f971affd640 1 -- 172.21.15.32:0/1261671735 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97200072f0 con 0x7f972c0b0cb0 2023-12-20T04:43:39.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f9730e2d640 1 -- 172.21.15.32:0/1261671735 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f972c14dd40 con 0x7f972c0b0cb0 2023-12-20T04:43:39.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f9730e2d640 1 -- 172.21.15.32:0/1261671735 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f972c14e260 con 0x7f972c0b0cb0 2023-12-20T04:43:39.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f971affd640 1 -- 172.21.15.32:0/1261671735 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f9720007450 con 0x7f972c0b0cb0 2023-12-20T04:43:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f971affd640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f970c065bb0 0x7f970c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f971affd640 1 -- 172.21.15.32:0/1261671735 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f97200828c0 con 0x7f972c0b0cb0 2023-12-20T04:43:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f972a59c640 1 -- 172.21.15.32:0/1261671735 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f970c065bb0 msgr2=0x7f970c068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:39.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.270+0000 7f972a59c640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f970c065bb0 0x7f970c068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:39.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.274+0000 7f971affd640 1 -- 172.21.15.32:0/1261671735 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f972c14e260 con 0x7f972c0b0cb0 2023-12-20T04:43:39.304 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:39.304+0000 7f3c8b02d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:43:39.392 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:39.392+0000 7f3c8b02d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:43:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.434+0000 7f9730e2d640 1 -- 172.21.15.32:0/1261671735 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f972c14e220 con 0x7f972c0b0cb0 2023-12-20T04:43:39.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.434+0000 7f971affd640 1 -- 172.21.15.32:0/1261671735 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v80) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9720017070 con 0x7f972c0b0cb0 2023-12-20T04:43:39.434 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:43:39.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 -- 172.21.15.32:0/1261671735 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f970c065bb0 msgr2=0x7f970c068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:39.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f970c065bb0 0x7f970c068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 -- 172.21.15.32:0/1261671735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f972c0b0cb0 msgr2=0x7f972c14ca20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f972c0b0cb0 0x7f972c14ca20 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f972c0b1160 tx=0x7f972000c890 comp rx=0 tx=0).stop 2023-12-20T04:43:39.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 -- 172.21.15.32:0/1261671735 shutdown_connections 2023-12-20T04:43:39.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f970c065bb0 0x7f970c068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f972c15abb0 0x7f972c14e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f972c0b0cb0 0x7f972c14ca20 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 --2- 172.21.15.32:0/1261671735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f972c070940 0x7f972c148300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 -- 172.21.15.32:0/1261671735 >> 172.21.15.32:0/1261671735 conn(0x7f972c06a510 msgr2=0x7f972c0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 -- 172.21.15.32:0/1261671735 shutdown_connections 2023-12-20T04:43:39.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.438+0000 7f9730e2d640 1 -- 172.21.15.32:0/1261671735 wait complete. 2023-12-20T04:43:39.441 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:39.442+0000 7f8e47247180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:43:39.456 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:43:39.533 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:39.534+0000 7f8e47247180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:43:39.548 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:39.548+0000 7f3c8b02d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:43:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.602+0000 7fdcefb39640 1 Processor -- start 2023-12-20T04:43:39.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.602+0000 7fdcefb39640 1 -- start start 2023-12-20T04:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdcefb39640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdce80d3300 0x7fdce80d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdcefb39640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce80d2650 con 0x7fdce815ce60 2023-12-20T04:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdcefb39640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdce80d1a40 con 0x7fdce80d1300 2023-12-20T04:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdcefb39640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdce80d1d20 con 0x7fdce80d3300 2023-12-20T04:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdced8d5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdce815ce60 0x7fdce80d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59296/0 (socket says 172.21.15.32:59296) 2023-12-20T04:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdced8d5640 1 -- 172.21.15.32:0/1133324608 learned_addr learned my addr 172.21.15.32:0/1133324608 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdced0d4640 1 --2- 172.21.15.32:0/1133324608 >> v2:172.21.15.39:3300/0 conn(0x7fdce80d3300 0x7fdce80d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdcec8d3640 1 -- 172.21.15.32:0/1133324608 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1764494250 0 0) 0x7fdce80d2650 con 0x7fdce815ce60 2023-12-20T04:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdcec8d3640 1 -- 172.21.15.32:0/1133324608 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdcd4003530 con 0x7fdce815ce60 2023-12-20T04:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdcec8d3640 1 -- 172.21.15.32:0/1133324608 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2416011015 0 0) 0x7fdce80d1a40 con 0x7fdce80d1300 2023-12-20T04:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdcec8d3640 1 -- 172.21.15.32:0/1133324608 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdce80d2650 con 0x7fdce80d1300 2023-12-20T04:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdced0d4640 1 -- 172.21.15.32:0/1133324608 >> v1:172.21.15.39:6789/0 conn(0x7fdce80d1300 legacy=0x7fdce80b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdced0d4640 1 -- 172.21.15.32:0/1133324608 >> v1:172.21.15.32:6789/0 conn(0x7fdce815ce60 legacy=0x7fdce80d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.606+0000 7fdced0d4640 1 -- 172.21.15.32:0/1133324608 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdce80d2000 con 0x7fdce80d3300 2023-12-20T04:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdced0d4640 1 --2- 172.21.15.32:0/1133324608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce80d3300 0x7fdce80d0dc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80097c0 tx=0x7fdcd801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b881ce73c22dd3fb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcec8d3640 1 -- 172.21.15.32:0/1133324608 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80ac640 con 0x7fdce80d3300 2023-12-20T04:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcec8d3640 1 -- 172.21.15.32:0/1133324608 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd80ac7a0 con 0x7fdce80d3300 2023-12-20T04:43:39.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcec8d3640 1 -- 172.21.15.32:0/1133324608 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80acb30 con 0x7fdce80d3300 2023-12-20T04:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcefb39640 1 -- 172.21.15.32:0/1133324608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce80d3300 msgr2=0x7fdce80d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcefb39640 1 --2- 172.21.15.32:0/1133324608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce80d3300 0x7fdce80d0dc0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80097c0 tx=0x7fdcd801c480 comp rx=0 tx=0).stop 2023-12-20T04:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcefb39640 1 -- 172.21.15.32:0/1133324608 shutdown_connections 2023-12-20T04:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcefb39640 1 --2- 172.21.15.32:0/1133324608 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce80d3300 0x7fdce80d0dc0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcefb39640 1 -- 172.21.15.32:0/1133324608 >> 172.21.15.32:0/1133324608 conn(0x7fdce806b5e0 msgr2=0x7fdce806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcefb39640 1 -- 172.21.15.32:0/1133324608 shutdown_connections 2023-12-20T04:43:39.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcefb39640 1 -- 172.21.15.32:0/1133324608 wait complete. 2023-12-20T04:43:39.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcefb39640 1 Processor -- start 2023-12-20T04:43:39.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.610+0000 7fdcefb39640 1 -- start start 2023-12-20T04:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcefb39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdce80d1300 0x7fdce80cc620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcefb39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce80d3300 0x7fdce80c7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcefb39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce815ce60 0x7fdce815a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcefb39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80c7a40 con 0x7fdce815ce60 2023-12-20T04:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcefb39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdce815b3f0 con 0x7fdce80d3300 2023-12-20T04:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcefb39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdce80c73f0 con 0x7fdce80d1300 2023-12-20T04:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcee0d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce815ce60 0x7fdce815a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdced0d4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce80d3300 0x7fdce80c7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdced8d5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdce80d1300 0x7fdce80cc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcee0d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce815ce60 0x7fdce815a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44962/0 (socket says 172.21.15.32:44962) 2023-12-20T04:43:39.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdced0d4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce80d3300 0x7fdce80c7ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:37840/0 (socket says 172.21.15.32:37840) 2023-12-20T04:43:39.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcee0d6640 1 -- 172.21.15.32:0/98285613 learned_addr learned my addr 172.21.15.32:0/98285613 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdced0d4640 1 -- 172.21.15.32:0/98285613 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdce80d1300 msgr2=0x7fdce80cc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdced0d4640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdce80d1300 0x7fdce80cc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdced0d4640 1 -- 172.21.15.32:0/98285613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce815ce60 msgr2=0x7fdce815a950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdced0d4640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce815ce60 0x7fdce815a950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdced0d4640 1 -- 172.21.15.32:0/98285613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdcd8009420 con 0x7fdce80d3300 2023-12-20T04:43:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcee0d6640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce815ce60 0x7fdce815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:39.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdced0d4640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce80d3300 0x7fdce80c7ed0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80ab000 tx=0x7fdcd8019220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:39.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcde7fc640 1 -- 172.21.15.32:0/98285613 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80b5610 con 0x7fdce80d3300 2023-12-20T04:43:39.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcde7fc640 1 -- 172.21.15.32:0/98285613 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdcd80b57b0 con 0x7fdce80d3300 2023-12-20T04:43:39.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcde7fc640 1 -- 172.21.15.32:0/98285613 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdcd80b5b60 con 0x7fdce80d3300 2023-12-20T04:43:39.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcefb39640 1 -- 172.21.15.32:0/98285613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdce80c76d0 con 0x7fdce80d3300 2023-12-20T04:43:39.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcefb39640 1 -- 172.21.15.32:0/98285613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdce80c8bd0 con 0x7fdce80d3300 2023-12-20T04:43:39.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.614+0000 7fdcefb39640 1 -- 172.21.15.32:0/98285613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdce80c7bb0 con 0x7fdce80d3300 2023-12-20T04:43:39.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.618+0000 7fdcde7fc640 1 -- 172.21.15.32:0/98285613 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fdcd80b3040 con 0x7fdce80d3300 2023-12-20T04:43:39.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.618+0000 7fdcde7fc640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdccc065b80 0x7fdccc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.618+0000 7fdced8d5640 1 -- 172.21.15.32:0/98285613 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdccc065b80 msgr2=0x7fdccc068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:39.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.618+0000 7fdcde7fc640 1 -- 172.21.15.32:0/98285613 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fdcd811d2a0 con 0x7fdce80d3300 2023-12-20T04:43:39.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.618+0000 7fdced8d5640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdccc065b80 0x7fdccc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:39.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.622+0000 7fdcde7fc640 1 -- 172.21.15.32:0/98285613 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdcd80eaaa0 con 0x7fdce80d3300 2023-12-20T04:43:39.637 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:39.636+0000 7f3c8b02d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:43:39.694 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:39.694+0000 7f8e47247180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:43:39.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.778+0000 7fdcefb39640 1 -- 172.21.15.32:0/98285613 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdce80c7530 con 0x7fdce80d3300 2023-12-20T04:43:39.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcde7fc640 1 -- 172.21.15.32:0/98285613 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v80) v1 ==== 104+0+11 (secure 0 0 0) 0x7fdce80c7530 con 0x7fdce80d3300 2023-12-20T04:43:39.779 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:43:39.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 -- 172.21.15.32:0/98285613 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdccc065b80 msgr2=0x7fdccc068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:39.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdccc065b80 0x7fdccc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 -- 172.21.15.32:0/98285613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce80d3300 msgr2=0x7fdce80c7ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce80d3300 0x7fdce80c7ed0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fdcd80ab000 tx=0x7fdcd8019220 comp rx=0 tx=0).stop 2023-12-20T04:43:39.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 -- 172.21.15.32:0/98285613 shutdown_connections 2023-12-20T04:43:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdccc065b80 0x7fdccc068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdce815ce60 0x7fdce815a950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdce80d3300 0x7fdce80c7ed0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 --2- 172.21.15.32:0/98285613 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdce80d1300 0x7fdce80cc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 -- 172.21.15.32:0/98285613 >> 172.21.15.32:0/98285613 conn(0x7fdce806b5e0 msgr2=0x7fdce8158390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 -- 172.21.15.32:0/98285613 shutdown_connections 2023-12-20T04:43:39.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.782+0000 7fdcefb39640 1 -- 172.21.15.32:0/98285613 wait complete. 2023-12-20T04:43:39.783 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:39.786+0000 7f8e47247180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:43:39.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:43:39.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec4cba2640 1 Processor -- start 2023-12-20T04:43:39.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec4cba2640 1 -- start start 2023-12-20T04:43:39.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec4cba2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4815ccf0 0x7fec480cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec4cba2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec480cd7b0 0x7fec480cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec4cba2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fec480cfd80 con 0x7fec480cf920 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec4cba2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fec480ce2f0 con 0x7fec480cd7b0 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec4cba2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec480d0b70 con 0x7fec4815ccf0 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec46ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fec480cd7b0 0x7fec480cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec47fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fec480cf920 0x7fec480b4490 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59310/0 (socket says 172.21.15.32:59310) 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec47fff640 1 -- 172.21.15.32:0/2834785807 learned_addr learned my addr 172.21.15.32:0/2834785807 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec477fe640 1 --2- 172.21.15.32:0/2834785807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4815ccf0 0x7fec480cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec477fe640 1 -- 172.21.15.32:0/2834785807 >> v2:172.21.15.32:3300/0 conn(0x7fec480cd7b0 msgr2=0x7fec480cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec477fe640 1 --2- 172.21.15.32:0/2834785807 >> v2:172.21.15.32:3300/0 conn(0x7fec480cd7b0 0x7fec480cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec477fe640 1 -- 172.21.15.32:0/2834785807 >> v1:172.21.15.32:6789/0 conn(0x7fec480cf920 legacy=0x7fec480b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec467fc640 1 -- 172.21.15.32:0/2834785807 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2113717245 0 0) 0x7fec480cfd80 con 0x7fec480cf920 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec477fe640 1 -- 172.21.15.32:0/2834785807 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec480d10f0 con 0x7fec4815ccf0 2023-12-20T04:43:39.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.950+0000 7fec46ffd640 1 --2- 172.21.15.32:0/2834785807 >> v2:172.21.15.32:3300/0 conn(0x7fec480cd7b0 0x7fec480cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.954+0000 7fec477fe640 1 --2- 172.21.15.32:0/2834785807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4815ccf0 0x7fec480cd270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fec380097c0 tx=0x7fec3801c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2f1b0f2cf6971da0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.954+0000 7fec467fc640 1 -- 172.21.15.32:0/2834785807 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380ab600 con 0x7fec4815ccf0 2023-12-20T04:43:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.954+0000 7fec467fc640 1 -- 172.21.15.32:0/2834785807 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec380a7050 con 0x7fec4815ccf0 2023-12-20T04:43:39.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.954+0000 7fec467fc640 1 -- 172.21.15.32:0/2834785807 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec380ab980 con 0x7fec4815ccf0 2023-12-20T04:43:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.954+0000 7fec4cba2640 1 -- 172.21.15.32:0/2834785807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4815ccf0 msgr2=0x7fec480cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.954+0000 7fec4cba2640 1 --2- 172.21.15.32:0/2834785807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4815ccf0 0x7fec480cd270 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fec380097c0 tx=0x7fec3801c3c0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.954+0000 7fec4cba2640 1 -- 172.21.15.32:0/2834785807 shutdown_connections 2023-12-20T04:43:39.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.954+0000 7fec4cba2640 1 --2- 172.21.15.32:0/2834785807 >> v2:172.21.15.32:3300/0 conn(0x7fec480cd7b0 0x7fec480cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.954+0000 7fec4cba2640 1 --2- 172.21.15.32:0/2834785807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4815ccf0 0x7fec480cd270 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.954+0000 7fec4cba2640 1 -- 172.21.15.32:0/2834785807 >> 172.21.15.32:0/2834785807 conn(0x7fec480658e0 msgr2=0x7fec48065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.954+0000 7fec4cba2640 1 -- 172.21.15.32:0/2834785807 shutdown_connections 2023-12-20T04:43:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec4cba2640 1 -- 172.21.15.32:0/2834785807 wait complete. 2023-12-20T04:43:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec4cba2640 1 Processor -- start 2023-12-20T04:43:39.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec4cba2640 1 -- start start 2023-12-20T04:43:39.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec4cba2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec480cd7b0 0x7fec480c6540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec4cba2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec480cf920 0x7fec48150a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec4cba2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4815ccf0 0x7fec48148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec4cba2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec480c6a80 con 0x7fec480cf920 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec4cba2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fec480d1b20 con 0x7fec480cd7b0 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec4cba2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fec480cc270 con 0x7fec4815ccf0 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec46ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec480cf920 0x7fec48150a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec46ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec480cf920 0x7fec48150a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44984/0 (socket says 172.21.15.32:44984) 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec46ffd640 1 -- 172.21.15.32:0/3689511013 learned_addr learned my addr 172.21.15.32:0/3689511013 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec477fe640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec480cd7b0 0x7fec480c6540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec47fff640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4815ccf0 0x7fec48148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec46ffd640 1 -- 172.21.15.32:0/3689511013 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4815ccf0 msgr2=0x7fec48148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec46ffd640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4815ccf0 0x7fec48148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec46ffd640 1 -- 172.21.15.32:0/3689511013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec480cd7b0 msgr2=0x7fec480c6540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:39.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec46ffd640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec480cd7b0 0x7fec480c6540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:39.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec46ffd640 1 -- 172.21.15.32:0/3689511013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fec3c009420 con 0x7fec480cf920 2023-12-20T04:43:39.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec477fe640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec480cd7b0 0x7fec480c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:39.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.958+0000 7fec46ffd640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec480cf920 0x7fec48150a40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00ebe0 tx=0x7fec3c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:39.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.962+0000 7fec2ffff640 1 -- 172.21.15.32:0/3689511013 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c00eee0 con 0x7fec480cf920 2023-12-20T04:43:39.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.962+0000 7fec2ffff640 1 -- 172.21.15.32:0/3689511013 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fec3c00cc30 con 0x7fec480cf920 2023-12-20T04:43:39.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.962+0000 7fec2ffff640 1 -- 172.21.15.32:0/3689511013 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fec3c013470 con 0x7fec480cf920 2023-12-20T04:43:39.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.962+0000 7fec4cba2640 1 -- 172.21.15.32:0/3689511013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fec38009420 con 0x7fec480cf920 2023-12-20T04:43:39.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.962+0000 7fec4cba2640 1 -- 172.21.15.32:0/3689511013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fec480cc530 con 0x7fec480cf920 2023-12-20T04:43:39.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.962+0000 7fec4cba2640 1 -- 172.21.15.32:0/3689511013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fec480ca4f0 con 0x7fec480cf920 2023-12-20T04:43:39.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.962+0000 7fec2ffff640 1 -- 172.21.15.32:0/3689511013 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 34) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fec3c0135d0 con 0x7fec480cf920 2023-12-20T04:43:39.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.962+0000 7fec2ffff640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec34065b80 0x7fec34068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:39.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.962+0000 7fec2ffff640 1 -- 172.21.15.32:0/3689511013 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(80..80 src has 1..80) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fec3c083740 con 0x7fec480cf920 2023-12-20T04:43:39.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.962+0000 7fec477fe640 1 -- 172.21.15.32:0/3689511013 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec34065b80 msgr2=0x7fec34068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:43:39.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.962+0000 7fec477fe640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec34065b80 0x7fec34068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:43:39.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:39.966+0000 7fec2ffff640 1 -- 172.21.15.32:0/3689511013 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fec3c050f30 con 0x7fec480cf920 2023-12-20T04:43:39.973 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:39.972+0000 7f3c8b02d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:43:40.035 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:43:40.036+0000 7f3c8b02d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:43:40.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.106+0000 7fec2ffff640 1 -- 172.21.15.32:0/3689511013 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 35) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fec3c051300 con 0x7fec480cf920 2023-12-20T04:43:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec4cba2640 1 -- 172.21.15.32:0/3689511013 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fec48074100 con 0x7fec480cf920 2023-12-20T04:43:40.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec2ffff640 1 -- 172.21.15.32:0/3689511013 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v80) v1 ==== 100+0+11 (secure 0 0 0) 0x7fec3c07f500 con 0x7fec480cf920 2023-12-20T04:43:40.141 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec4cba2640 1 -- 172.21.15.32:0/3689511013 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec34065b80 msgr2=0x7fec34068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec4cba2640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec34065b80 0x7fec34068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec4cba2640 1 -- 172.21.15.32:0/3689511013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec480cf920 msgr2=0x7fec48150a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec4cba2640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec480cf920 0x7fec48150a40 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7fec3c00ebe0 tx=0x7fec3c00c520 comp rx=0 tx=0).stop 2023-12-20T04:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec4cba2640 1 -- 172.21.15.32:0/3689511013 shutdown_connections 2023-12-20T04:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec4cba2640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fec34065b80 0x7fec34068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec4cba2640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fec4815ccf0 0x7fec48148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec4cba2640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fec480cf920 0x7fec48150a40 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec4cba2640 1 --2- 172.21.15.32:0/3689511013 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fec480cd7b0 0x7fec480c6540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.142+0000 7fec4cba2640 1 -- 172.21.15.32:0/3689511013 >> 172.21.15.32:0/3689511013 conn(0x7fec480658e0 msgr2=0x7fec48160b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:40.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.146+0000 7fec4cba2640 1 -- 172.21.15.32:0/3689511013 shutdown_connections 2023-12-20T04:43:40.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.146+0000 7fec4cba2640 1 -- 172.21.15.32:0/3689511013 wait complete. 2023-12-20T04:43:40.147 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.146+0000 7f8e47247180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:43:40.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:40.213 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.214+0000 7f8e47247180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:43:40.286 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.286+0000 7f8dcfabc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:40.287 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.286+0000 7f8dcfabc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:40.287 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.286+0000 7f8dcfabc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:40.287 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.286+0000 7f8dcfabc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:40.287 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.286+0000 7f8dcfabc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:40.287 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.290+0000 7f8dd2ac2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:40.287 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.290+0000 7f8dd2ac2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:40.287 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.290+0000 7f8dd2ac2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:40.287 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.290+0000 7f8dd2ac2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:40.288 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:40.290+0000 7f8dd2ac2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:43:40.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d8a55640 1 Processor -- start 2023-12-20T04:43:40.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d8a55640 1 -- start start 2023-12-20T04:43:40.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d8a55640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23d415cdb0 0x7f23d40d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:40.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d8a55640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23d40d2740 0x7f23d40cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:40.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d8a55640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23d40b6ca0 con 0x7f23d40d1120 2023-12-20T04:43:40.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d8a55640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23d415d190 con 0x7f23d415cdb0 2023-12-20T04:43:40.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d8a55640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d40d16a0 con 0x7f23d40d2740 2023-12-20T04:43:40.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23d415cdb0 0x7f23d40d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:40.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23d415cdb0 0x7f23d40d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45086/0 (socket says 172.21.15.32:45086) 2023-12-20T04:43:40.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d259c640 1 -- 172.21.15.32:0/2859370271 learned_addr learned my addr 172.21.15.32:0/2859370271 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:40.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23cbfff640 1 --2- 172.21.15.32:0/2859370271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23d40d2740 0x7f23d40cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:40.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d259c640 1 -- 172.21.15.32:0/2859370271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23d40d2740 msgr2=0x7f23d40cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:40.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d259c640 1 --2- 172.21.15.32:0/2859370271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23d40d2740 0x7f23d40cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d259c640 1 -- 172.21.15.32:0/2859370271 >> v1:172.21.15.39:6789/0 conn(0x7f23d40d1120 legacy=0x7f23d40b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:40.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.326+0000 7f23d259c640 1 -- 172.21.15.32:0/2859370271 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d40d1980 con 0x7f23d415cdb0 2023-12-20T04:43:40.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d259c640 1 --2- 172.21.15.32:0/2859370271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d415cdb0 0x7f23d40d0be0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009870 tx=0x7f23c401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=507d15674620d51 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:40.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d1d9b640 1 -- 172.21.15.32:0/2859370271 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40a89b0 con 0x7f23d415cdb0 2023-12-20T04:43:40.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d1d9b640 1 -- 172.21.15.32:0/2859370271 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23c40a8b10 con 0x7f23d415cdb0 2023-12-20T04:43:40.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d1d9b640 1 -- 172.21.15.32:0/2859370271 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23c40a8ea0 con 0x7f23d415cdb0 2023-12-20T04:43:40.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d8a55640 1 -- 172.21.15.32:0/2859370271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d415cdb0 msgr2=0x7f23d40d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:40.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d8a55640 1 --2- 172.21.15.32:0/2859370271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d415cdb0 0x7f23d40d0be0 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7f23c4009870 tx=0x7f23c401ecf0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d8a55640 1 -- 172.21.15.32:0/2859370271 shutdown_connections 2023-12-20T04:43:40.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d8a55640 1 --2- 172.21.15.32:0/2859370271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23d40d2740 0x7f23d40cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d8a55640 1 --2- 172.21.15.32:0/2859370271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d415cdb0 0x7f23d40d0be0 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d8a55640 1 -- 172.21.15.32:0/2859370271 >> 172.21.15.32:0/2859370271 conn(0x7f23d4065ee0 msgr2=0x7f23d406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:40.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d8a55640 1 -- 172.21.15.32:0/2859370271 shutdown_connections 2023-12-20T04:43:40.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.330+0000 7f23d8a55640 1 -- 172.21.15.32:0/2859370271 wait complete. 2023-12-20T04:43:40.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d8a55640 1 Processor -- start 2023-12-20T04:43:40.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d8a55640 1 -- start start 2023-12-20T04:43:40.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d8a55640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23d40d1120 0x7f23d40c8550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d8a55640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d40d2740 0x7f23d414f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d8a55640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d415cdb0 0x7f23d414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d8a55640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d414fdc0 con 0x7f23d415cdb0 2023-12-20T04:43:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d8a55640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23d414d200 con 0x7f23d40d2740 2023-12-20T04:43:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d8a55640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23d414c960 con 0x7f23d40d1120 2023-12-20T04:43:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23d40d1120 0x7f23d40c8550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d415cdb0 0x7f23d414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23cbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d40d2740 0x7f23d414f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23cbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d40d2740 0x7f23d414f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:37958/0 (socket says 172.21.15.32:37958) 2023-12-20T04:43:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23cbfff640 1 -- 172.21.15.32:0/4195369514 learned_addr learned my addr 172.21.15.32:0/4195369514 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:40.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d2d9d640 1 --2- 172.21.15.32:0/4195369514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d415cdb0 0x7f23d414c110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45100/0 (socket says 172.21.15.32:45100) 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23cbfff640 1 -- 172.21.15.32:0/4195369514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23d40d1120 msgr2=0x7f23d40c8550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23cbfff640 1 --2- 172.21.15.32:0/4195369514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23d40d1120 0x7f23d40c8550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23cbfff640 1 -- 172.21.15.32:0/4195369514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d415cdb0 msgr2=0x7f23d414c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23cbfff640 1 --2- 172.21.15.32:0/4195369514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d415cdb0 0x7f23d414c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23cbfff640 1 -- 172.21.15.32:0/4195369514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23c40094d0 con 0x7f23d40d2740 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d2d9d640 1 --2- 172.21.15.32:0/4195369514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d415cdb0 0x7f23d414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d259c640 1 --2- 172.21.15.32:0/4195369514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23d40d1120 0x7f23d40c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23cbfff640 1 --2- 172.21.15.32:0/4195369514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d40d2740 0x7f23d414f880 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00d800 tx=0x7f23bc00dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23caffd640 1 -- 172.21.15.32:0/4195369514 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc00b6c0 con 0x7f23d40d2740 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23caffd640 1 -- 172.21.15.32:0/4195369514 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23bc00b820 con 0x7f23d40d2740 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23caffd640 1 -- 172.21.15.32:0/4195369514 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23bc00bb90 con 0x7f23d40d2740 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d8a55640 1 -- 172.21.15.32:0/4195369514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23d414cc40 con 0x7f23d40d2740 2023-12-20T04:43:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.334+0000 7f23d8a55640 1 -- 172.21.15.32:0/4195369514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23d415f080 con 0x7f23d40d2740 2023-12-20T04:43:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.338+0000 7f23d8a55640 1 -- 172.21.15.32:0/4195369514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23d415f210 con 0x7f23d40d2740 2023-12-20T04:43:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.338+0000 7f23caffd640 1 -- 172.21.15.32:0/4195369514 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 36) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f23bc00bcf0 con 0x7f23d40d2740 2023-12-20T04:43:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.338+0000 7f23caffd640 1 -- 172.21.15.32:0/4195369514 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f23bc083370 con 0x7f23d40d2740 2023-12-20T04:43:40.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.342+0000 7f23caffd640 1 -- 172.21.15.32:0/4195369514 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f23bc050ee0 con 0x7f23d40d2740 2023-12-20T04:43:40.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.534+0000 7f23d8a55640 1 -- 172.21.15.32:0/4195369514 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f23d404e000 con 0x7f23d40d2740 2023-12-20T04:43:40.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.534+0000 7f23caffd640 1 -- 172.21.15.32:0/4195369514 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4706 (secure 0 0 0) 0x7f23bc054500 con 0x7f23d40d2740 2023-12-20T04:43:40.533 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:40.533 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5780,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4279891602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4279891602},{"type":"v1","addr":"172.21.15.39:6834","nonce":4279891602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6292,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3937321478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3937321478},{"type":"v1","addr":"172.21.15.39:6835","nonce":3937321478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/404004710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":404004710},{"type":"v1","addr":"172.21.15.32:6835","nonce":404004710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:21.971704+0000","modified":"2023-12-20T04:43:24.982163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3343723618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3343723618},{"type":"v1","addr":"172.21.15.32:6837","nonce":3343723618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-20T04:43:40.533 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2023-12-20T04:43:40.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.534+0000 7f23d8a55640 1 -- 172.21.15.32:0/4195369514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d40d2740 msgr2=0x7f23d414f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:40.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.534+0000 7f23d8a55640 1 --2- 172.21.15.32:0/4195369514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d40d2740 0x7f23d414f880 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f23bc00d800 tx=0x7f23bc00dcc0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.534+0000 7f23d8a55640 1 -- 172.21.15.32:0/4195369514 shutdown_connections 2023-12-20T04:43:40.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.534+0000 7f23d8a55640 1 --2- 172.21.15.32:0/4195369514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23d415cdb0 0x7f23d414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.534+0000 7f23d8a55640 1 --2- 172.21.15.32:0/4195369514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23d40d2740 0x7f23d414f880 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.534+0000 7f23d8a55640 1 --2- 172.21.15.32:0/4195369514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23d40d1120 0x7f23d40c8550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.534+0000 7f23d8a55640 1 -- 172.21.15.32:0/4195369514 >> 172.21.15.32:0/4195369514 conn(0x7f23d4065ee0 msgr2=0x7f23d4073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:40.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.534+0000 7f23d8a55640 1 -- 172.21.15.32:0/4195369514 shutdown_connections 2023-12-20T04:43:40.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.534+0000 7f23d8a55640 1 -- 172.21.15.32:0/4195369514 wait complete. 2023-12-20T04:43:40.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:40.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.742+0000 7f14ca6d1640 1 Processor -- start 2023-12-20T04:43:40.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.742+0000 7f14ca6d1640 1 -- start start 2023-12-20T04:43:40.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14ca6d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f14c415f110 0x7f14c415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:40.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14ca6d1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c415b990 con 0x7f14c40af5e0 2023-12-20T04:43:40.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14ca6d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14c415bbd0 con 0x7f14c406ce60 2023-12-20T04:43:40.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14ca6d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14c415beb0 con 0x7f14c415f110 2023-12-20T04:43:40.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14c3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f14c40af5e0 0x7f14c40af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59322/0 (socket says 172.21.15.32:59322) 2023-12-20T04:43:40.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14c3fff640 1 -- 172.21.15.32:0/832239460 learned_addr learned my addr 172.21.15.32:0/832239460 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:40.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14c8c6e640 1 --2- 172.21.15.32:0/832239460 >> v2:172.21.15.39:3300/0 conn(0x7f14c415f110 0x7f14c415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:40.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14c2ffd640 1 -- 172.21.15.32:0/832239460 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1816331288 0 0) 0x7f14b40012a0 con 0x7f14c40af5e0 2023-12-20T04:43:40.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14c2ffd640 1 -- 172.21.15.32:0/832239460 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14ac003530 con 0x7f14c40af5e0 2023-12-20T04:43:40.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14c2ffd640 1 -- 172.21.15.32:0/832239460 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1219029577 0 0) 0x7f14c415bbd0 con 0x7f14c406ce60 2023-12-20T04:43:40.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14c2ffd640 1 -- 172.21.15.32:0/832239460 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14b40012a0 con 0x7f14c406ce60 2023-12-20T04:43:40.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14c8c6e640 1 -- 172.21.15.32:0/832239460 >> v1:172.21.15.39:6789/0 conn(0x7f14c406ce60 legacy=0x7f14c415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:40.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14c8c6e640 1 -- 172.21.15.32:0/832239460 >> v1:172.21.15.32:6789/0 conn(0x7f14c40af5e0 legacy=0x7f14c40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:40.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.746+0000 7f14c8c6e640 1 -- 172.21.15.32:0/832239460 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14c415e910 con 0x7f14c415f110 2023-12-20T04:43:40.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14c8c6e640 1 --2- 172.21.15.32:0/832239460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14c415f110 0x7f14c415c440 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009d90 tx=0x7f14bc01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca4445d76795336 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:40.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14c2ffd640 1 -- 172.21.15.32:0/832239460 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc0a58f0 con 0x7f14c415f110 2023-12-20T04:43:40.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14c2ffd640 1 -- 172.21.15.32:0/832239460 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14bc0a5a50 con 0x7f14c415f110 2023-12-20T04:43:40.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14c2ffd640 1 -- 172.21.15.32:0/832239460 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14bc0a5de0 con 0x7f14c415f110 2023-12-20T04:43:40.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14ca6d1640 1 -- 172.21.15.32:0/832239460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14c415f110 msgr2=0x7f14c415c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:40.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14ca6d1640 1 --2- 172.21.15.32:0/832239460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14c415f110 0x7f14c415c440 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f14bc009d90 tx=0x7f14bc01c890 comp rx=0 tx=0).stop 2023-12-20T04:43:40.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14ca6d1640 1 -- 172.21.15.32:0/832239460 shutdown_connections 2023-12-20T04:43:40.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14ca6d1640 1 --2- 172.21.15.32:0/832239460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14c415f110 0x7f14c415c440 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14ca6d1640 1 -- 172.21.15.32:0/832239460 >> 172.21.15.32:0/832239460 conn(0x7f14c406a320 msgr2=0x7f14c40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:40.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14ca6d1640 1 -- 172.21.15.32:0/832239460 shutdown_connections 2023-12-20T04:43:40.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14ca6d1640 1 -- 172.21.15.32:0/832239460 wait complete. 2023-12-20T04:43:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14ca6d1640 1 Processor -- start 2023-12-20T04:43:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.750+0000 7f14ca6d1640 1 -- start start 2023-12-20T04:43:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14ca6d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c406ce60 0x7f14c40b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14ca6d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14c40af5e0 0x7f14c414ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14ca6d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14c415f110 0x7f14c414c6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14ca6d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c4165510 con 0x7f14c406ce60 2023-12-20T04:43:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14ca6d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14c414e1e0 con 0x7f14c40af5e0 2023-12-20T04:43:40.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14ca6d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14c414d3c0 con 0x7f14c415f110 2023-12-20T04:43:40.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c406ce60 0x7f14c40b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:40.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c406ce60 0x7f14c40b3cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45110/0 (socket says 172.21.15.32:45110) 2023-12-20T04:43:40.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c3fff640 1 -- 172.21.15.32:0/1729819183 learned_addr learned my addr 172.21.15.32:0/1729819183 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:40.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c37fe640 1 --2- 172.21.15.32:0/1729819183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14c40af5e0 0x7f14c414ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:40.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c8c6e640 1 --2- 172.21.15.32:0/1729819183 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14c415f110 0x7f14c414c6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:40.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c3fff640 1 -- 172.21.15.32:0/1729819183 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14c415f110 msgr2=0x7f14c414c6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:40.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c3fff640 1 --2- 172.21.15.32:0/1729819183 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14c415f110 0x7f14c414c6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c3fff640 1 -- 172.21.15.32:0/1729819183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14c40af5e0 msgr2=0x7f14c414ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:40.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c3fff640 1 --2- 172.21.15.32:0/1729819183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14c40af5e0 0x7f14c414ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c3fff640 1 -- 172.21.15.32:0/1729819183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14bc009a70 con 0x7f14c406ce60 2023-12-20T04:43:40.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c8c6e640 1 --2- 172.21.15.32:0/1729819183 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14c415f110 0x7f14c414c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:43:40.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c3fff640 1 --2- 172.21.15.32:0/1729819183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c406ce60 0x7f14c40b3cd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f14b4001ec0 tx=0x7f14b400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:40.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c0ff9640 1 -- 172.21.15.32:0/1729819183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b4007270 con 0x7f14c406ce60 2023-12-20T04:43:40.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c0ff9640 1 -- 172.21.15.32:0/1729819183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b40073d0 con 0x7f14c406ce60 2023-12-20T04:43:40.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c0ff9640 1 -- 172.21.15.32:0/1729819183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b40057b0 con 0x7f14c406ce60 2023-12-20T04:43:40.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14ca6d1640 1 -- 172.21.15.32:0/1729819183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14c414d660 con 0x7f14c406ce60 2023-12-20T04:43:40.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14ca6d1640 1 -- 172.21.15.32:0/1729819183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14c414dc20 con 0x7f14c406ce60 2023-12-20T04:43:40.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14ca6d1640 1 -- 172.21.15.32:0/1729819183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14c404e000 con 0x7f14c406ce60 2023-12-20T04:43:40.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.754+0000 7f14c0ff9640 1 -- 172.21.15.32:0/1729819183 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 36) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f14b4005910 con 0x7f14c406ce60 2023-12-20T04:43:40.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.758+0000 7f14c0ff9640 1 -- 172.21.15.32:0/1729819183 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f14b4082480 con 0x7f14c406ce60 2023-12-20T04:43:40.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.762+0000 7f14c0ff9640 1 -- 172.21.15.32:0/1729819183 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f14b404fff0 con 0x7f14c406ce60 2023-12-20T04:43:40.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.918+0000 7f14ca6d1640 1 -- 172.21.15.32:0/1729819183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14c415b990 con 0x7f14c406ce60 2023-12-20T04:43:40.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.922+0000 7f14c0ff9640 1 -- 172.21.15.32:0/1729819183 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16911 (secure 0 0 0) 0x7f14c415b990 con 0x7f14c406ce60 2023-12-20T04:43:40.920 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:40.921 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:40.220290+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:19.734786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:20.967556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3267047649":"2023-12-21T04:43:40.220238+0000","172.21.15.32:0/793620801":"2023-12-21T04:43:40.220238+0000","172.21.15.32:0/1397077541":"2023-12-21T04:43:40.220238+0000","172.21.15.32:0/2610637012":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2158383566":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/3862763229":"2023-12-21T04:43:40.220238+0000","172.21.15.32:6833/38009":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2038860058":"2023-12-21T04:43:40.220238+0000","172.21.15.32:6832/38009":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2830398071":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/3257133891":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/3556237801":"2023-12-21T04:43:40.220238+0000","172.21.15.32:0/2848355101":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/1251934730":"2023-12-21T04:43:34.035676+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:40.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.922+0000 7f14ca6d1640 1 -- 172.21.15.32:0/1729819183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c406ce60 msgr2=0x7f14c40b3cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:40.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.922+0000 7f14ca6d1640 1 --2- 172.21.15.32:0/1729819183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c406ce60 0x7f14c40b3cd0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7f14b4001ec0 tx=0x7f14b400cb10 comp rx=0 tx=0).stop 2023-12-20T04:43:40.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.922+0000 7f14ca6d1640 1 -- 172.21.15.32:0/1729819183 shutdown_connections 2023-12-20T04:43:40.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.922+0000 7f14ca6d1640 1 --2- 172.21.15.32:0/1729819183 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14c415f110 0x7f14c414c6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.922+0000 7f14ca6d1640 1 --2- 172.21.15.32:0/1729819183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14c40af5e0 0x7f14c414ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.922+0000 7f14ca6d1640 1 --2- 172.21.15.32:0/1729819183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14c406ce60 0x7f14c40b3cd0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:40.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.922+0000 7f14ca6d1640 1 -- 172.21.15.32:0/1729819183 >> 172.21.15.32:0/1729819183 conn(0x7f14c406a320 msgr2=0x7f14c40ccad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:40.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.922+0000 7f14ca6d1640 1 -- 172.21.15.32:0/1729819183 shutdown_connections 2023-12-20T04:43:40.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:40.922+0000 7f14ca6d1640 1 -- 172.21.15.32:0/1729819183 wait complete. 2023-12-20T04:43:40.942 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T04:43:40.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.086+0000 7f993928c640 1 Processor -- start 2023-12-20T04:43:41.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.086+0000 7f993928c640 1 -- start start 2023-12-20T04:43:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.086+0000 7f993928c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f993415ae80 0x7f99340cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.086+0000 7f993928c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99340ced30 con 0x7f99340afd50 2023-12-20T04:43:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.086+0000 7f993928c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f99340cf010 con 0x7f99340b2ae0 2023-12-20T04:43:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.086+0000 7f993928c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f99340ceb50 con 0x7f993415ae80 2023-12-20T04:43:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.090+0000 7f9932ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f993415ae80 0x7f99340cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.090+0000 7f9932ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f993415ae80 0x7f99340cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45120/0 (socket says 172.21.15.32:45120) 2023-12-20T04:43:41.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.090+0000 7f9932ffd640 1 -- 172.21.15.32:0/4176071482 learned_addr learned my addr 172.21.15.32:0/4176071482 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.090+0000 7f9932ffd640 1 -- 172.21.15.32:0/4176071482 >> v1:172.21.15.39:6789/0 conn(0x7f99340b2ae0 legacy=0x7f99340b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.090+0000 7f9932ffd640 1 -- 172.21.15.32:0/4176071482 >> v1:172.21.15.32:6789/0 conn(0x7f99340afd50 legacy=0x7f99340b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.090+0000 7f9932ffd640 1 -- 172.21.15.32:0/4176071482 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99340cf2f0 con 0x7f993415ae80 2023-12-20T04:43:41.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.090+0000 7f9932ffd640 1 --2- 172.21.15.32:0/4176071482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993415ae80 0x7f99340cfed0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9924009870 tx=0x7f992401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29a4c116d86f0204 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:41.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.090+0000 7f99327fc640 1 -- 172.21.15.32:0/4176071482 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240a89f0 con 0x7f993415ae80 2023-12-20T04:43:41.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.090+0000 7f99327fc640 1 -- 172.21.15.32:0/4176071482 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99240a8b50 con 0x7f993415ae80 2023-12-20T04:43:41.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.090+0000 7f99327fc640 1 -- 172.21.15.32:0/4176071482 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240b0440 con 0x7f993415ae80 2023-12-20T04:43:41.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 -- 172.21.15.32:0/4176071482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993415ae80 msgr2=0x7f99340cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 --2- 172.21.15.32:0/4176071482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993415ae80 0x7f99340cfed0 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7f9924009870 tx=0x7f992401ecf0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 -- 172.21.15.32:0/4176071482 shutdown_connections 2023-12-20T04:43:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 --2- 172.21.15.32:0/4176071482 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f993415ae80 0x7f99340cfed0 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 -- 172.21.15.32:0/4176071482 >> 172.21.15.32:0/4176071482 conn(0x7f99340cd3d0 msgr2=0x7f99340cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 -- 172.21.15.32:0/4176071482 shutdown_connections 2023-12-20T04:43:41.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 -- 172.21.15.32:0/4176071482 wait complete. 2023-12-20T04:43:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 Processor -- start 2023-12-20T04:43:41.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 -- start start 2023-12-20T04:43:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99340afd50 0x7f993406b9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99340b2ae0 0x7f9934156eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f993415ae80 0x7f99341587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f993406b590 con 0x7f99340afd50 2023-12-20T04:43:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99340af780 con 0x7f99340b2ae0 2023-12-20T04:43:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f993928c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99341573f0 con 0x7f993415ae80 2023-12-20T04:43:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f9932ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99340afd50 0x7f993406b9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f9932ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99340afd50 0x7f993406b9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45128/0 (socket says 172.21.15.32:45128) 2023-12-20T04:43:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f9932ffd640 1 -- 172.21.15.32:0/188174101 learned_addr learned my addr 172.21.15.32:0/188174101 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:41.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f992bfff640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99340b2ae0 0x7f9934156eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f99337fe640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f993415ae80 0x7f99341587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f9932ffd640 1 -- 172.21.15.32:0/188174101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f993415ae80 msgr2=0x7f99341587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f9932ffd640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f993415ae80 0x7f99341587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f9932ffd640 1 -- 172.21.15.32:0/188174101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99340b2ae0 msgr2=0x7f9934156eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f9932ffd640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99340b2ae0 0x7f9934156eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.094+0000 7f9932ffd640 1 -- 172.21.15.32:0/188174101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99240094d0 con 0x7f99340afd50 2023-12-20T04:43:41.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.098+0000 7f9932ffd640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99340afd50 0x7f993406b9b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f99240b3040 tx=0x7f992401ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.098+0000 7f992b7fe640 1 -- 172.21.15.32:0/188174101 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240b0e40 con 0x7f99340afd50 2023-12-20T04:43:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.098+0000 7f992b7fe640 1 -- 172.21.15.32:0/188174101 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99240bb470 con 0x7f99340afd50 2023-12-20T04:43:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.098+0000 7f992b7fe640 1 -- 172.21.15.32:0/188174101 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99240bb820 con 0x7f99340afd50 2023-12-20T04:43:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.098+0000 7f993928c640 1 -- 172.21.15.32:0/188174101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f99341f2db0 con 0x7f99340afd50 2023-12-20T04:43:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.098+0000 7f993928c640 1 -- 172.21.15.32:0/188174101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f993406aed0 con 0x7f99340afd50 2023-12-20T04:43:41.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.098+0000 7f993928c640 1 -- 172.21.15.32:0/188174101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f993406b800 con 0x7f99340afd50 2023-12-20T04:43:41.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.098+0000 7f992b7fe640 1 -- 172.21.15.32:0/188174101 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 36) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f99240b7070 con 0x7f99340afd50 2023-12-20T04:43:41.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.098+0000 7f992b7fe640 1 -- 172.21.15.32:0/188174101 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f9924121190 con 0x7f99340afd50 2023-12-20T04:43:41.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.106+0000 7f992b7fe640 1 -- 172.21.15.32:0/188174101 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f99240eed00 con 0x7f99340afd50 2023-12-20T04:43:41.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.230+0000 7f992b7fe640 1 -- 172.21.15.32:0/188174101 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 37) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f99240ee7b0 con 0x7f99340afd50 2023-12-20T04:43:41.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.230+0000 7f992b7fe640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f991401c9c0 0x7f991406d0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.234+0000 7f992bfff640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f991401c9c0 0x7f991406d0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.234+0000 7f992bfff640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f991401c9c0 0x7f991406d0f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f99340c70d0 tx=0x7f991c009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:41.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.298+0000 7f993928c640 1 -- 172.21.15.32:0/188174101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f99340b3870 con 0x7f99340afd50 2023-12-20T04:43:41.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.298+0000 7f992b7fe640 1 -- 172.21.15.32:0/188174101 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4706 (secure 0 0 0) 0x7f99240f4050 con 0x7f99340afd50 2023-12-20T04:43:41.299 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:41.300 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5780,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4279891602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4279891602},{"type":"v1","addr":"172.21.15.39:6834","nonce":4279891602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6292,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3937321478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3937321478},{"type":"v1","addr":"172.21.15.39:6835","nonce":3937321478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/404004710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":404004710},{"type":"v1","addr":"172.21.15.32:6835","nonce":404004710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:21.971704+0000","modified":"2023-12-20T04:43:24.982163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3343723618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3343723618},{"type":"v1","addr":"172.21.15.32:6837","nonce":3343723618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-20T04:43:41.300 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2023-12-20T04:43:41.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 -- 172.21.15.32:0/188174101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f991401c9c0 msgr2=0x7f991406d0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f991401c9c0 0x7f991406d0f0 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f99340c70d0 tx=0x7f991c009040 comp rx=0 tx=0).stop 2023-12-20T04:43:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 -- 172.21.15.32:0/188174101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99340afd50 msgr2=0x7f993406b9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99340afd50 0x7f993406b9b0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7f99240b3040 tx=0x7f992401ba90 comp rx=0 tx=0).stop 2023-12-20T04:43:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 -- 172.21.15.32:0/188174101 shutdown_connections 2023-12-20T04:43:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f991401c9c0 0x7f991406d0f0 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f993415ae80 0x7f99341587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99340b2ae0 0x7f9934156eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 --2- 172.21.15.32:0/188174101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99340afd50 0x7f993406b9b0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 -- 172.21.15.32:0/188174101 >> 172.21.15.32:0/188174101 conn(0x7f99340cd3d0 msgr2=0x7f99340d1040 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:41.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 -- 172.21.15.32:0/188174101 shutdown_connections 2023-12-20T04:43:41.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.302+0000 7f993928c640 1 -- 172.21.15.32:0/188174101 wait complete. 2023-12-20T04:43:41.320 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.462+0000 7f18ba68c640 1 Processor -- start 2023-12-20T04:43:41.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.462+0000 7f18ba68c640 1 -- start start 2023-12-20T04:43:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18ba68c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18b406ce60 0x7f18b415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18ba68c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b415bbd0 con 0x7f18b415f110 2023-12-20T04:43:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18ba68c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18b415beb0 con 0x7f18b40af5e0 2023-12-20T04:43:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18ba68c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18b415e910 con 0x7f18b406ce60 2023-12-20T04:43:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18b8c29640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f18b415f110 0x7f18b415c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59342/0 (socket says 172.21.15.32:59342) 2023-12-20T04:43:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18b8c29640 1 -- 172.21.15.32:0/1291400535 learned_addr learned my addr 172.21.15.32:0/1291400535 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:41.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18b37fe640 1 --2- 172.21.15.32:0/1291400535 >> v2:172.21.15.39:3300/0 conn(0x7f18b406ce60 0x7f18b415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18b2ffd640 1 -- 172.21.15.32:0/1291400535 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3852254692 0 0) 0x7f18b415bbd0 con 0x7f18b415f110 2023-12-20T04:43:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18b2ffd640 1 -- 172.21.15.32:0/1291400535 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f189c003530 con 0x7f18b415f110 2023-12-20T04:43:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18b37fe640 1 -- 172.21.15.32:0/1291400535 >> v1:172.21.15.39:6789/0 conn(0x7f18b40af5e0 legacy=0x7f18b40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18b37fe640 1 -- 172.21.15.32:0/1291400535 >> v1:172.21.15.32:6789/0 conn(0x7f18b415f110 legacy=0x7f18b415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.466+0000 7f18b37fe640 1 -- 172.21.15.32:0/1291400535 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18b415ebf0 con 0x7f18b406ce60 2023-12-20T04:43:41.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18b37fe640 1 --2- 172.21.15.32:0/1291400535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b406ce60 0x7f18b415b280 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f18a80097c0 tx=0x7f18a801c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9dce922fd2873f4a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18b2ffd640 1 -- 172.21.15.32:0/1291400535 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18a80a8050 con 0x7f18b406ce60 2023-12-20T04:43:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18b2ffd640 1 -- 172.21.15.32:0/1291400535 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18a80ac5f0 con 0x7f18b406ce60 2023-12-20T04:43:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18b2ffd640 1 -- 172.21.15.32:0/1291400535 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18b415b9d0 con 0x7f18b406ce60 2023-12-20T04:43:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18ba68c640 1 -- 172.21.15.32:0/1291400535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b406ce60 msgr2=0x7f18b415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18ba68c640 1 --2- 172.21.15.32:0/1291400535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b406ce60 0x7f18b415b280 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f18a80097c0 tx=0x7f18a801c3f0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18ba68c640 1 -- 172.21.15.32:0/1291400535 shutdown_connections 2023-12-20T04:43:41.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18ba68c640 1 --2- 172.21.15.32:0/1291400535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b406ce60 0x7f18b415b280 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18ba68c640 1 -- 172.21.15.32:0/1291400535 >> 172.21.15.32:0/1291400535 conn(0x7f18b406a320 msgr2=0x7f18b40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:41.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18ba68c640 1 -- 172.21.15.32:0/1291400535 shutdown_connections 2023-12-20T04:43:41.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18ba68c640 1 -- 172.21.15.32:0/1291400535 wait complete. 2023-12-20T04:43:41.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18ba68c640 1 Processor -- start 2023-12-20T04:43:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.470+0000 7f18ba68c640 1 -- start start 2023-12-20T04:43:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18ba68c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b406ce60 0x7f18b41508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18ba68c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18b40af5e0 0x7f18b40b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18ba68c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b415f110 0x7f18b4150250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18ba68c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b4165f60 con 0x7f18b415f110 2023-12-20T04:43:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18ba68c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18b4165c80 con 0x7f18b406ce60 2023-12-20T04:43:41.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18ba68c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18b4071660 con 0x7f18b40af5e0 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b8c29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b415f110 0x7f18b4150250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b8c29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b415f110 0x7f18b4150250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45130/0 (socket says 172.21.15.32:45130) 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b8c29640 1 -- 172.21.15.32:0/1665287812 learned_addr learned my addr 172.21.15.32:0/1665287812 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b37fe640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18b40af5e0 0x7f18b40b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b3fff640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b406ce60 0x7f18b41508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b8c29640 1 -- 172.21.15.32:0/1665287812 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18b40af5e0 msgr2=0x7f18b40b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b8c29640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18b40af5e0 0x7f18b40b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b8c29640 1 -- 172.21.15.32:0/1665287812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b406ce60 msgr2=0x7f18b41508d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b8c29640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b406ce60 0x7f18b41508d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b8c29640 1 -- 172.21.15.32:0/1665287812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18a8009420 con 0x7f18b415f110 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b3fff640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b406ce60 0x7f18b41508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b8c29640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b415f110 0x7f18b4150250 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00caa0 tx=0x7f18ac00cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b0ff9640 1 -- 172.21.15.32:0/1665287812 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac00ac00 con 0x7f18b415f110 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b0ff9640 1 -- 172.21.15.32:0/1665287812 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18ac00ad60 con 0x7f18b415f110 2023-12-20T04:43:41.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b0ff9640 1 -- 172.21.15.32:0/1665287812 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18ac012620 con 0x7f18b415f110 2023-12-20T04:43:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18ba68c640 1 -- 172.21.15.32:0/1665287812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18b4071940 con 0x7f18b415f110 2023-12-20T04:43:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18ba68c640 1 -- 172.21.15.32:0/1665287812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18b4072140 con 0x7f18b415f110 2023-12-20T04:43:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18ba68c640 1 -- 172.21.15.32:0/1665287812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18b40714c0 con 0x7f18b415f110 2023-12-20T04:43:41.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.474+0000 7f18b0ff9640 1 -- 172.21.15.32:0/1665287812 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f18ac007070 con 0x7f18b415f110 2023-12-20T04:43:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.478+0000 7f18b0ff9640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18a00658b0 0x7f18a0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.478+0000 7f18b0ff9640 1 -- 172.21.15.32:0/1665287812 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6311+0+0 (secure 0 0 0) 0x7f18ac082f30 con 0x7f18b415f110 2023-12-20T04:43:41.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.478+0000 7f18b3fff640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18a00658b0 0x7f18a0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.478+0000 7f18b3fff640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18a00658b0 0x7f18a0067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f18b4071410 tx=0x7f18a4009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.482+0000 7f18b0ff9640 1 -- 172.21.15.32:0/1665287812 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f18b40714c0 con 0x7f18b415f110 2023-12-20T04:43:41.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.670+0000 7f18ba68c640 1 -- 172.21.15.32:0/1665287812 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f18b40714c0 con 0x7f18b415f110 2023-12-20T04:43:41.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.670+0000 7f18b0ff9640 1 -- 172.21.15.32:0/1665287812 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 31 v31) v1 ==== 94+0+4706 (secure 0 0 0) 0x7f18b40714c0 con 0x7f18b415f110 2023-12-20T04:43:41.670 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:41.670 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":31,"default_fscid":4,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5780,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4279891602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4279891602},{"type":"v1","addr":"172.21.15.39:6834","nonce":4279891602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6292,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3937321478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3937321478},{"type":"v1","addr":"172.21.15.39:6835","nonce":3937321478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/404004710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":404004710},{"type":"v1","addr":"172.21.15.32:6835","nonce":404004710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27}],"filesystems":[{"mdsmap":{"epoch":31,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:21.971704+0000","modified":"2023-12-20T04:43:24.982163+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6402},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6402":{"gid":6402,"name":"a","rank":0,"incarnation":28,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3343723618","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3343723618},{"type":"v1","addr":"172.21.15.32:6837","nonce":3343723618}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[9],"metadata_pool":8,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":4}]} 2023-12-20T04:43:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 31 2023-12-20T04:43:41.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 -- 172.21.15.32:0/1665287812 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18a00658b0 msgr2=0x7f18a0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18a00658b0 0x7f18a0067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f18b4071410 tx=0x7f18a4009040 comp rx=0 tx=0).stop 2023-12-20T04:43:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 -- 172.21.15.32:0/1665287812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b415f110 msgr2=0x7f18b4150250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b415f110 0x7f18b4150250 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f18ac00caa0 tx=0x7f18ac00cf60 comp rx=0 tx=0).stop 2023-12-20T04:43:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 -- 172.21.15.32:0/1665287812 shutdown_connections 2023-12-20T04:43:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18a00658b0 0x7f18a0067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18b415f110 0x7f18b4150250 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18b40af5e0 0x7f18b40b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 --2- 172.21.15.32:0/1665287812 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18b406ce60 0x7f18b41508d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 -- 172.21.15.32:0/1665287812 >> 172.21.15.32:0/1665287812 conn(0x7f18b406a320 msgr2=0x7f18b415c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 -- 172.21.15.32:0/1665287812 shutdown_connections 2023-12-20T04:43:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.674+0000 7f18ba68c640 1 -- 172.21.15.32:0/1665287812 wait complete. 2023-12-20T04:43:41.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.834+0000 7fb55c2c4640 1 Processor -- start 2023-12-20T04:43:41.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.834+0000 7fb55c2c4640 1 -- start start 2023-12-20T04:43:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55c2c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5540b0d00 0x7fb554164a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55c2c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5540711d0 con 0x7fb5540aece0 2023-12-20T04:43:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55c2c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb5540c98c0 con 0x7fb5541599e0 2023-12-20T04:43:41.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55c2c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5540c9ba0 con 0x7fb5540b0d00 2023-12-20T04:43:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55985f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb5540aece0 0x7fb5540b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59358/0 (socket says 172.21.15.32:59358) 2023-12-20T04:43:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55985f640 1 -- 172.21.15.32:0/3000349640 learned_addr learned my addr 172.21.15.32:0/3000349640 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55a861640 1 --2- 172.21.15.32:0/3000349640 >> v2:172.21.15.32:3300/0 conn(0x7fb5540b0d00 0x7fb554164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55a861640 1 -- 172.21.15.32:0/3000349640 >> v1:172.21.15.39:6789/0 conn(0x7fb5541599e0 legacy=0x7fb5540c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55a861640 1 -- 172.21.15.32:0/3000349640 >> v1:172.21.15.32:6789/0 conn(0x7fb5540aece0 legacy=0x7fb5540b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55a861640 1 -- 172.21.15.32:0/3000349640 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5540c9e80 con 0x7fb5540b0d00 2023-12-20T04:43:41.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55905e640 1 -- 172.21.15.32:0/3000349640 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 857450020 0 0) 0x7fb5540c98c0 con 0x7fb5541599e0 2023-12-20T04:43:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.838+0000 7fb55a861640 1 --2- 172.21.15.32:0/3000349640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5540b0d00 0x7fb554164a70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009d90 tx=0x7fb54c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5dcda0ef56f6ca9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:41.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55905e640 1 -- 172.21.15.32:0/3000349640 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0a5900 con 0x7fb5540b0d00 2023-12-20T04:43:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55905e640 1 -- 172.21.15.32:0/3000349640 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c0a5a60 con 0x7fb5540b0d00 2023-12-20T04:43:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55905e640 1 -- 172.21.15.32:0/3000349640 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0a5df0 con 0x7fb5540b0d00 2023-12-20T04:43:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 -- 172.21.15.32:0/3000349640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5540b0d00 msgr2=0x7fb554164a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 --2- 172.21.15.32:0/3000349640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5540b0d00 0x7fb554164a70 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009d90 tx=0x7fb54c01c950 comp rx=0 tx=0).stop 2023-12-20T04:43:41.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 -- 172.21.15.32:0/3000349640 shutdown_connections 2023-12-20T04:43:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 --2- 172.21.15.32:0/3000349640 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5540b0d00 0x7fb554164a70 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 -- 172.21.15.32:0/3000349640 >> 172.21.15.32:0/3000349640 conn(0x7fb55406a320 msgr2=0x7fb5540ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 -- 172.21.15.32:0/3000349640 shutdown_connections 2023-12-20T04:43:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 -- 172.21.15.32:0/3000349640 wait complete. 2023-12-20T04:43:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 Processor -- start 2023-12-20T04:43:41.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 -- start start 2023-12-20T04:43:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5540aece0 0x7fb5541f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5540b0d00 0x7fb5541504e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5541599e0 0x7fb554152730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554151ed0 con 0x7fb5541599e0 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb554150e30 con 0x7fb5540b0d00 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55c2c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb554151110 con 0x7fb5540aece0 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55a861640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5541599e0 0x7fb554152730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55a861640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5541599e0 0x7fb554152730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45152/0 (socket says 172.21.15.32:45152) 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55a861640 1 -- 172.21.15.32:0/2401516819 learned_addr learned my addr 172.21.15.32:0/2401516819 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55985f640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5540b0d00 0x7fb5541504e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.842+0000 7fb55a060640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5540aece0 0x7fb5541f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb55a861640 1 -- 172.21.15.32:0/2401516819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5540aece0 msgr2=0x7fb5541f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb55a861640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5540aece0 0x7fb5541f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb55a861640 1 -- 172.21.15.32:0/2401516819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5540b0d00 msgr2=0x7fb5541504e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb55a861640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5540b0d00 0x7fb5541504e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb55a861640 1 -- 172.21.15.32:0/2401516819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54c009a70 con 0x7fb5541599e0 2023-12-20T04:43:41.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb55a861640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5541599e0 0x7fb554152730 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb54c01a580 tx=0x7fb54c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb54affd640 1 -- 172.21.15.32:0/2401516819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0ad950 con 0x7fb5541599e0 2023-12-20T04:43:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb54affd640 1 -- 172.21.15.32:0/2401516819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54c0adab0 con 0x7fb5541599e0 2023-12-20T04:43:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb54affd640 1 -- 172.21.15.32:0/2401516819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54c0ade40 con 0x7fb5541599e0 2023-12-20T04:43:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb55c2c4640 1 -- 172.21.15.32:0/2401516819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5541513f0 con 0x7fb5541599e0 2023-12-20T04:43:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb55c2c4640 1 -- 172.21.15.32:0/2401516819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5541519b0 con 0x7fb5541599e0 2023-12-20T04:43:41.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb55c2c4640 1 -- 172.21.15.32:0/2401516819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb554151ba0 con 0x7fb5541599e0 2023-12-20T04:43:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb54affd640 1 -- 172.21.15.32:0/2401516819 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fb54c0b4050 con 0x7fb5541599e0 2023-12-20T04:43:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb54affd640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb5380658b0 0x7fb538067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.846+0000 7fb54affd640 1 -- 172.21.15.32:0/2401516819 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fb54c11df20 con 0x7fb5541599e0 2023-12-20T04:43:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.850+0000 7fb55a060640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb5380658b0 0x7fb538067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:41.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.850+0000 7fb55a060640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb5380658b0 0x7fb538067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb5500017b0 tx=0x7fb550009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:41.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:41.850+0000 7fb54affd640 1 -- 172.21.15.32:0/2401516819 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb54c0eba30 con 0x7fb5541599e0 2023-12-20T04:43:42.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.010+0000 7fb55c2c4640 1 -- 172.21.15.32:0/2401516819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb55404e000 con 0x7fb5541599e0 2023-12-20T04:43:42.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.010+0000 7fb54affd640 1 -- 172.21.15.32:0/2401516819 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) v1 ==== 74+0+16911 (secure 0 0 0) 0x7fb54c0b2070 con 0x7fb5541599e0 2023-12-20T04:43:42.010 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:42.010 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:40.220290+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":8,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:19.734786+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"78","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":9,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:20.967556+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"79","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3267047649":"2023-12-21T04:43:40.220238+0000","172.21.15.32:0/793620801":"2023-12-21T04:43:40.220238+0000","172.21.15.32:0/1397077541":"2023-12-21T04:43:40.220238+0000","172.21.15.32:0/2610637012":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2158383566":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/3862763229":"2023-12-21T04:43:40.220238+0000","172.21.15.32:6833/38009":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2038860058":"2023-12-21T04:43:40.220238+0000","172.21.15.32:6832/38009":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/2830398071":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/3257133891":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/3556237801":"2023-12-21T04:43:40.220238+0000","172.21.15.32:0/2848355101":"2023-12-21T04:43:34.035676+0000","172.21.15.32:0/1251934730":"2023-12-21T04:43:34.035676+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 -- 172.21.15.32:0/2401516819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb5380658b0 msgr2=0x7fb538067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb5380658b0 0x7fb538067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb5500017b0 tx=0x7fb550009040 comp rx=0 tx=0).stop 2023-12-20T04:43:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 -- 172.21.15.32:0/2401516819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5541599e0 msgr2=0x7fb554152730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5541599e0 0x7fb554152730 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fb54c01a580 tx=0x7fb54c0196f0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 -- 172.21.15.32:0/2401516819 shutdown_connections 2023-12-20T04:43:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb5380658b0 0x7fb538067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5541599e0 0x7fb554152730 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb5540b0d00 0x7fb5541504e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 --2- 172.21.15.32:0/2401516819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5540aece0 0x7fb5541f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 -- 172.21.15.32:0/2401516819 >> 172.21.15.32:0/2401516819 conn(0x7fb55406a320 msgr2=0x7fb55406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 -- 172.21.15.32:0/2401516819 shutdown_connections 2023-12-20T04:43:42.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.014+0000 7fb55c2c4640 1 -- 172.21.15.32:0/2401516819 wait complete. 2023-12-20T04:43:42.032 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T04:43:42.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa649e2d640 1 Processor -- start 2023-12-20T04:43:42.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa649e2d640 1 -- start start 2023-12-20T04:43:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa649e2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64415ce30 0x7fa644161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa649e2d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa644070950 0x7fa64406d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa649e2d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa644071920 con 0x7fa64406dce0 2023-12-20T04:43:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa649e2d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa644071080 con 0x7fa644070950 2023-12-20T04:43:42.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa649e2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa644071360 con 0x7fa64415ce30 2023-12-20T04:43:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa6437fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64415ce30 0x7fa644161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa6437fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64415ce30 0x7fa644161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:42548/0 (socket says 172.21.15.32:42548) 2023-12-20T04:43:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa643fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fa64406dce0 0x7fa644069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:52600/0 (socket says 172.21.15.32:52600) 2023-12-20T04:43:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa6437fe640 1 -- 172.21.15.32:0/756651616 learned_addr learned my addr 172.21.15.32:0/756651616 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa642ffd640 1 --2- 172.21.15.32:0/756651616 >> v2:172.21.15.39:3300/0 conn(0x7fa644070950 0x7fa64406d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.178+0000 7fa6427fc640 1 -- 172.21.15.32:0/756651616 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1579608649 0 0) 0x7fa644071920 con 0x7fa64406dce0 2023-12-20T04:43:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa6427fc640 1 -- 172.21.15.32:0/756651616 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa62c003530 con 0x7fa64406dce0 2023-12-20T04:43:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa642ffd640 1 -- 172.21.15.32:0/756651616 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64415ce30 msgr2=0x7fa644161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa642ffd640 1 --2- 172.21.15.32:0/756651616 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64415ce30 0x7fa644161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa642ffd640 1 -- 172.21.15.32:0/756651616 >> v1:172.21.15.39:6789/0 conn(0x7fa64406dce0 legacy=0x7fa644069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa642ffd640 1 -- 172.21.15.32:0/756651616 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa644071640 con 0x7fa644070950 2023-12-20T04:43:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa6437fe640 1 --2- 172.21.15.32:0/756651616 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64415ce30 0x7fa644161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:42.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa642ffd640 1 --2- 172.21.15.32:0/756651616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa644070950 0x7fa64406d7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa6380097c0 tx=0x7fa63801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b926cc38dfbafe96 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa6427fc640 1 -- 172.21.15.32:0/756651616 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638004230 con 0x7fa644070950 2023-12-20T04:43:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa6427fc640 1 -- 172.21.15.32:0/756651616 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6380a7050 con 0x7fa644070950 2023-12-20T04:43:42.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa6427fc640 1 -- 172.21.15.32:0/756651616 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa638003620 con 0x7fa644070950 2023-12-20T04:43:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa649e2d640 1 -- 172.21.15.32:0/756651616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa644070950 msgr2=0x7fa64406d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa649e2d640 1 --2- 172.21.15.32:0/756651616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa644070950 0x7fa64406d7a0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7fa6380097c0 tx=0x7fa63801c480 comp rx=0 tx=0).stop 2023-12-20T04:43:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa649e2d640 1 -- 172.21.15.32:0/756651616 shutdown_connections 2023-12-20T04:43:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa649e2d640 1 --2- 172.21.15.32:0/756651616 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa644070950 0x7fa64406d7a0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa649e2d640 1 --2- 172.21.15.32:0/756651616 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64415ce30 0x7fa644161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa649e2d640 1 -- 172.21.15.32:0/756651616 >> 172.21.15.32:0/756651616 conn(0x7fa6440cba50 msgr2=0x7fa6440cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:42.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa649e2d640 1 -- 172.21.15.32:0/756651616 shutdown_connections 2023-12-20T04:43:42.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.182+0000 7fa649e2d640 1 -- 172.21.15.32:0/756651616 wait complete. 2023-12-20T04:43:42.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa649e2d640 1 Processor -- start 2023-12-20T04:43:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa649e2d640 1 -- start start 2023-12-20T04:43:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa649e2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64406dce0 0x7fa6440b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa649e2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa644070950 0x7fa6440ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa649e2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64415ce30 0x7fa6440c1c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa649e2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b40d0 con 0x7fa64415ce30 2023-12-20T04:43:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa649e2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b4560 con 0x7fa644070950 2023-12-20T04:43:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa649e2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6440b34c0 con 0x7fa64406dce0 2023-12-20T04:43:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa643fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64415ce30 0x7fa6440c1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa643fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64415ce30 0x7fa6440c1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45166/0 (socket says 172.21.15.32:45166) 2023-12-20T04:43:42.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa643fff640 1 -- 172.21.15.32:0/2784480257 learned_addr learned my addr 172.21.15.32:0/2784480257 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa6437fe640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64406dce0 0x7fa6440b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa642ffd640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa644070950 0x7fa6440ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa643fff640 1 -- 172.21.15.32:0/2784480257 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64406dce0 msgr2=0x7fa6440b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa643fff640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64406dce0 0x7fa6440b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa643fff640 1 -- 172.21.15.32:0/2784480257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa644070950 msgr2=0x7fa6440ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa643fff640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa644070950 0x7fa6440ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa643fff640 1 -- 172.21.15.32:0/2784480257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6340094d0 con 0x7fa64415ce30 2023-12-20T04:43:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa6437fe640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64406dce0 0x7fa6440b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa642ffd640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa644070950 0x7fa6440ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.186+0000 7fa643fff640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64415ce30 0x7fa6440c1c40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00c7c0 tx=0x7fa63c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.190+0000 7fa62bfff640 1 -- 172.21.15.32:0/2784480257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00a640 con 0x7fa64415ce30 2023-12-20T04:43:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.190+0000 7fa62bfff640 1 -- 172.21.15.32:0/2784480257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa63c00a7a0 con 0x7fa64415ce30 2023-12-20T04:43:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.190+0000 7fa62bfff640 1 -- 172.21.15.32:0/2784480257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa63c00ab50 con 0x7fa64415ce30 2023-12-20T04:43:42.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.190+0000 7fa649e2d640 1 -- 172.21.15.32:0/2784480257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa638009420 con 0x7fa64415ce30 2023-12-20T04:43:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.190+0000 7fa649e2d640 1 -- 172.21.15.32:0/2784480257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6440b3a80 con 0x7fa64415ce30 2023-12-20T04:43:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.190+0000 7fa649e2d640 1 -- 172.21.15.32:0/2784480257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6440b4240 con 0x7fa64415ce30 2023-12-20T04:43:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.190+0000 7fa62bfff640 1 -- 172.21.15.32:0/2784480257 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 37) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fa63c00acb0 con 0x7fa64415ce30 2023-12-20T04:43:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.190+0000 7fa62bfff640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa6300658b0 0x7fa630067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.190+0000 7fa62bfff640 1 -- 172.21.15.32:0/2784480257 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(81..81 src has 1..81) v4 ==== 6311+0+0 (secure 0 0 0) 0x7fa63c0828c0 con 0x7fa64415ce30 2023-12-20T04:43:42.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.190+0000 7fa6437fe640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa6300658b0 0x7fa630067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.190+0000 7fa6437fe640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa6300658b0 0x7fa630067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa634009c90 tx=0x7fa634008040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:42.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.194+0000 7fa62bfff640 1 -- 172.21.15.32:0/2784480257 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa63c050340 con 0x7fa64415ce30 2023-12-20T04:43:42.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.246+0000 7fa62bfff640 1 -- 172.21.15.32:0/2784480257 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fa63c050710 con 0x7fa64415ce30 2023-12-20T04:43:42.262 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:42.262+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:42.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.394+0000 7fa649e2d640 1 -- 172.21.15.32:0/2784480257 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fa64404e000 con 0x7fa64415ce30 2023-12-20T04:43:42.393 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T04:43:42.394+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T04:43:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.402+0000 7fa62bfff640 1 -- 172.21.15.32:0/2784480257 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v32) v1 ==== 161+0+0 (secure 0 0 0) 0x7fa63c06dea0 con 0x7fa64415ce30 2023-12-20T04:43:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T04:43:42.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 -- 172.21.15.32:0/2784480257 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa6300658b0 msgr2=0x7fa630067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa6300658b0 0x7fa630067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa634009c90 tx=0x7fa634008040 comp rx=0 tx=0).stop 2023-12-20T04:43:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 -- 172.21.15.32:0/2784480257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64415ce30 msgr2=0x7fa6440c1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64415ce30 0x7fa6440c1c40 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7fa63c00c7c0 tx=0x7fa63c00cc80 comp rx=0 tx=0).stop 2023-12-20T04:43:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 -- 172.21.15.32:0/2784480257 shutdown_connections 2023-12-20T04:43:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa6300658b0 0x7fa630067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa64415ce30 0x7fa6440c1c40 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa644070950 0x7fa6440ca8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 --2- 172.21.15.32:0/2784480257 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa64406dce0 0x7fa6440b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 -- 172.21.15.32:0/2784480257 >> 172.21.15.32:0/2784480257 conn(0x7fa6440cba50 msgr2=0x7fa64415e0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 -- 172.21.15.32:0/2784480257 shutdown_connections 2023-12-20T04:43:42.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.406+0000 7fa649e2d640 1 -- 172.21.15.32:0/2784480257 wait complete. 2023-12-20T04:43:42.410 INFO:tasks.ceph.mds.a.smithi032.stderr: -3785> 2023-12-20T04:43:27.550+0000 7f54fcd15640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:43:42.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T04:43:42.461 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T04:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.578+0000 7fa5f9df3640 1 Processor -- start 2023-12-20T04:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.578+0000 7fa5f9df3640 1 -- start start 2023-12-20T04:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f9df3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa5f40cd390 0x7fa5f40cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f9df3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5f40cd770 0x7fa5f41612b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f9df3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5f41617f0 con 0x7fa5f4071350 2023-12-20T04:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f9df3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f4070620 con 0x7fa5f40cd770 2023-12-20T04:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f9df3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5f40ccd70 con 0x7fa5f40cd390 2023-12-20T04:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5f40cd770 0x7fa5f41612b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5f40cd770 0x7fa5f41612b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45196/0 (socket says 172.21.15.32:45196) 2023-12-20T04:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f2ffd640 1 -- 172.21.15.32:0/2116039133 learned_addr learned my addr 172.21.15.32:0/2116039133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f37fe640 1 --2- 172.21.15.32:0/2116039133 >> v2:172.21.15.39:3300/0 conn(0x7fa5f40cd390 0x7fa5f40cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f2ffd640 1 -- 172.21.15.32:0/2116039133 >> v2:172.21.15.39:3300/0 conn(0x7fa5f40cd390 msgr2=0x7fa5f40cc790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f2ffd640 1 --2- 172.21.15.32:0/2116039133 >> v2:172.21.15.39:3300/0 conn(0x7fa5f40cd390 0x7fa5f40cc790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f2ffd640 1 -- 172.21.15.32:0/2116039133 >> v1:172.21.15.39:6789/0 conn(0x7fa5f4071350 legacy=0x7fa5f406d110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.582+0000 7fa5f2ffd640 1 -- 172.21.15.32:0/2116039133 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5f4070870 con 0x7fa5f40cd770 2023-12-20T04:43:42.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f2ffd640 1 --2- 172.21.15.32:0/2116039133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5f40cd770 0x7fa5f41612b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa5f406ad50 tx=0x7fa5e801c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ab18ecc33665a0f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f27fc640 1 -- 172.21.15.32:0/2116039133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80ac410 con 0x7fa5f40cd770 2023-12-20T04:43:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f27fc640 1 -- 172.21.15.32:0/2116039133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e80ac570 con 0x7fa5f40cd770 2023-12-20T04:43:42.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f27fc640 1 -- 172.21.15.32:0/2116039133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e80ac900 con 0x7fa5f40cd770 2023-12-20T04:43:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f9df3640 1 -- 172.21.15.32:0/2116039133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5f40cd770 msgr2=0x7fa5f41612b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f9df3640 1 --2- 172.21.15.32:0/2116039133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5f40cd770 0x7fa5f41612b0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7fa5f406ad50 tx=0x7fa5e801c3b0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f9df3640 1 -- 172.21.15.32:0/2116039133 shutdown_connections 2023-12-20T04:43:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f9df3640 1 --2- 172.21.15.32:0/2116039133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5f40cd770 0x7fa5f41612b0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f9df3640 1 --2- 172.21.15.32:0/2116039133 >> v2:172.21.15.39:3300/0 conn(0x7fa5f40cd390 0x7fa5f40cc790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f9df3640 1 -- 172.21.15.32:0/2116039133 >> 172.21.15.32:0/2116039133 conn(0x7fa5f40b06b0 msgr2=0x7fa5f40b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f9df3640 1 -- 172.21.15.32:0/2116039133 shutdown_connections 2023-12-20T04:43:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f9df3640 1 -- 172.21.15.32:0/2116039133 wait complete. 2023-12-20T04:43:42.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f9df3640 1 Processor -- start 2023-12-20T04:43:42.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.586+0000 7fa5f9df3640 1 -- start start 2023-12-20T04:43:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f9df3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5f4071350 0x7fa5f40ad230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f9df3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5f40cd390 0x7fa5f4153e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f9df3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5f40cd770 0x7fa5f40af100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f9df3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f40ad770 con 0x7fa5f4071350 2023-12-20T04:43:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f9df3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f4155a30 con 0x7fa5f40cd770 2023-12-20T04:43:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f9df3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5f41546b0 con 0x7fa5f40cd390 2023-12-20T04:43:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5f4071350 0x7fa5f40ad230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5f4071350 0x7fa5f40ad230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45202/0 (socket says 172.21.15.32:45202) 2023-12-20T04:43:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f37fe640 1 -- 172.21.15.32:0/3038863449 learned_addr learned my addr 172.21.15.32:0/3038863449 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:42.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f2ffd640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5f40cd390 0x7fa5f4153e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f3fff640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5f40cd770 0x7fa5f40af100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f37fe640 1 -- 172.21.15.32:0/3038863449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5f40cd390 msgr2=0x7fa5f4153e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f37fe640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5f40cd390 0x7fa5f4153e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f37fe640 1 -- 172.21.15.32:0/3038863449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5f40cd770 msgr2=0x7fa5f40af100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f37fe640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5f40cd770 0x7fa5f40af100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f37fe640 1 -- 172.21.15.32:0/3038863449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5e80094d0 con 0x7fa5f4071350 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f2ffd640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5f40cd390 0x7fa5f4153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f37fe640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5f4071350 0x7fa5f40ad230 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e930 tx=0x7fa5e400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5dbfff640 1 -- 172.21.15.32:0/3038863449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e4009690 con 0x7fa5f4071350 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5dbfff640 1 -- 172.21.15.32:0/3038863449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5e400ca00 con 0x7fa5f4071350 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5dbfff640 1 -- 172.21.15.32:0/3038863449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5e400cd70 con 0x7fa5f4071350 2023-12-20T04:43:42.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f9df3640 1 -- 172.21.15.32:0/3038863449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5f4154990 con 0x7fa5f4071350 2023-12-20T04:43:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f9df3640 1 -- 172.21.15.32:0/3038863449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5f4154f50 con 0x7fa5f4071350 2023-12-20T04:43:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.590+0000 7fa5f9df3640 1 -- 172.21.15.32:0/3038863449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5f404e000 con 0x7fa5f4071350 2023-12-20T04:43:42.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.594+0000 7fa5dbfff640 1 -- 172.21.15.32:0/3038863449 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fa5e4013470 con 0x7fa5f4071350 2023-12-20T04:43:42.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.594+0000 7fa5dbfff640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa5e0065af0 0x7fa5e0067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:42.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.594+0000 7fa5dbfff640 1 -- 172.21.15.32:0/3038863449 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6397+0+0 (secure 0 0 0) 0x7fa5e40836a0 con 0x7fa5f4071350 2023-12-20T04:43:42.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.594+0000 7fa5f2ffd640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa5e0065af0 0x7fa5e0067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:42.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.594+0000 7fa5f2ffd640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa5e0065af0 0x7fa5e0067fb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80ae000 tx=0x7fa5e8009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:42.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.598+0000 7fa5dbfff640 1 -- 172.21.15.32:0/3038863449 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa5e4050f30 con 0x7fa5f4071350 2023-12-20T04:43:42.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:42.802+0000 7fa5f9df3640 1 -- 172.21.15.32:0/3038863449 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa5f40b4b40 con 0x7fa5f4071350 2023-12-20T04:43:42.931 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:42.930+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:43.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5dbfff640 1 -- 172.21.15.32:0/3038863449 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7fa5e4054550 con 0x7fa5f4071350 2023-12-20T04:43:43.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 -- 172.21.15.32:0/3038863449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa5e0065af0 msgr2=0x7fa5e0067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:43.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa5e0065af0 0x7fa5e0067fb0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa5e80ae000 tx=0x7fa5e8009040 comp rx=0 tx=0).stop 2023-12-20T04:43:43.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 -- 172.21.15.32:0/3038863449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5f4071350 msgr2=0x7fa5f40ad230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:43.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5f4071350 0x7fa5f40ad230 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7fa5e400e930 tx=0x7fa5e400edf0 comp rx=0 tx=0).stop 2023-12-20T04:43:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 -- 172.21.15.32:0/3038863449 shutdown_connections 2023-12-20T04:43:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa5e0065af0 0x7fa5e0067fb0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5f40cd770 0x7fa5f40af100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5f40cd390 0x7fa5f4153e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 --2- 172.21.15.32:0/3038863449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5f4071350 0x7fa5f40ad230 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 -- 172.21.15.32:0/3038863449 >> 172.21.15.32:0/3038863449 conn(0x7fa5f40b06b0 msgr2=0x7fa5f406e050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 -- 172.21.15.32:0/3038863449 shutdown_connections 2023-12-20T04:43:43.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.406+0000 7fa5f9df3640 1 -- 172.21.15.32:0/3038863449 wait complete. 2023-12-20T04:43:43.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T04:43:43.602 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:43.602+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:43.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.618+0000 7f4a8bfce640 1 Processor -- start 2023-12-20T04:43:43.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.618+0000 7f4a8bfce640 1 -- start start 2023-12-20T04:43:43.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a8bfce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a84070940 0x7f4a841640d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:43.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a8bfce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a84159630 con 0x7f4a840ce290 2023-12-20T04:43:43.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a8bfce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a840c9480 con 0x7f4a84159250 2023-12-20T04:43:43.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a8bfce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8406cf90 con 0x7f4a84070940 2023-12-20T04:43:43.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a89569640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4a840ce290 0x7f4a84070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59362/0 (socket says 172.21.15.32:59362) 2023-12-20T04:43:43.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a89569640 1 -- 172.21.15.32:0/1762861553 learned_addr learned my addr 172.21.15.32:0/1762861553 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a8a56b640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a84070940 0x7f4a841640d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2626860387 0 0) 0x7f4a84159630 con 0x7f4a840ce290 2023-12-20T04:43:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a70003530 con 0x7f4a840ce290 2023-12-20T04:43:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513977688 0 0) 0x7f4a840c9480 con 0x7f4a84159250 2023-12-20T04:43:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4a84159630 con 0x7f4a84159250 2023-12-20T04:43:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3022588106 0 0) 0x7f4a70003530 con 0x7f4a840ce290 2023-12-20T04:43:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a84070940 msgr2=0x7f4a841640d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a84070940 0x7f4a841640d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 >> v1:172.21.15.39:6789/0 conn(0x7f4a84159250 legacy=0x7f4a840caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:43.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a8406d270 con 0x7f4a840ce290 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f4a74001680 con 0x7f4a840ce290 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 >> v1:172.21.15.32:6789/0 conn(0x7f4a840ce290 legacy=0x7f4a84070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a70003b40 0x7f4a70003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a700047d0 0x7f4a70004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a84070940 0x7f4a70005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a8406cf90 con 0x7f4a70003b40 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7000dc50 con 0x7f4a700047d0 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a7000de10 con 0x7f4a84070940 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a89d6a640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a70003b40 0x7f4a70003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a89569640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a700047d0 0x7f4a70004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a8a56b640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a84070940 0x7f4a70005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a89d6a640 1 -- 172.21.15.32:0/1762861553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a84070940 msgr2=0x7f4a70005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:43.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a89d6a640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a84070940 0x7f4a70005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a89d6a640 1 -- 172.21.15.32:0/1762861553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a700047d0 msgr2=0x7f4a70004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a89d6a640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a700047d0 0x7f4a70004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:43.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.622+0000 7f4a89d6a640 1 -- 172.21.15.32:0/1762861553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a70003530 con 0x7f4a70003b40 2023-12-20T04:43:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a89d6a640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a70003b40 0x7f4a70003f40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4a80007fd0 tx=0x7f4a8001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34db548883a90279 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a800052d0 con 0x7f4a70003b40 2023-12-20T04:43:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a80005450 con 0x7f4a70003b40 2023-12-20T04:43:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a88d68640 1 -- 172.21.15.32:0/1762861553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80003840 con 0x7f4a70003b40 2023-12-20T04:43:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a8bfce640 1 -- 172.21.15.32:0/1762861553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a70003b40 msgr2=0x7f4a70003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a8bfce640 1 --2- 172.21.15.32:0/1762861553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a70003b40 0x7f4a70003f40 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7f4a80007fd0 tx=0x7f4a8001c520 comp rx=0 tx=0).stop 2023-12-20T04:43:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a8a56b640 1 -- 172.21.15.32:0/1762861553 reap_dead start 2023-12-20T04:43:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a8bfce640 1 -- 172.21.15.32:0/1762861553 shutdown_connections 2023-12-20T04:43:43.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a8bfce640 1 -- 172.21.15.32:0/1762861553 >> 172.21.15.32:0/1762861553 conn(0x7f4a8406a510 msgr2=0x7f4a840ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a8bfce640 1 -- 172.21.15.32:0/1762861553 shutdown_connections 2023-12-20T04:43:43.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a8bfce640 1 -- 172.21.15.32:0/1762861553 wait complete. 2023-12-20T04:43:43.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.626+0000 7f4a8bfce640 1 Processor -- start 2023-12-20T04:43:43.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a8bfce640 1 -- start start 2023-12-20T04:43:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a8bfce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a70003b40 0x7f4a840c7310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a8bfce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8414eeb0 0x7f4a8414e680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a8bfce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a840c41a0 0x7f4a840c4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:43.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a8bfce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840c4b60 con 0x7f4a8414eeb0 2023-12-20T04:43:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a8bfce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840b2120 con 0x7f4a840c41a0 2023-12-20T04:43:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a8bfce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a840b1510 con 0x7f4a70003b40 2023-12-20T04:43:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89569640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8414eeb0 0x7f4a8414e680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89d6a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a70003b40 0x7f4a840c7310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89569640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8414eeb0 0x7f4a8414e680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45206/0 (socket says 172.21.15.32:45206) 2023-12-20T04:43:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89d6a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a70003b40 0x7f4a840c7310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:42604/0 (socket says 172.21.15.32:42604) 2023-12-20T04:43:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89569640 1 -- 172.21.15.32:0/2493931038 learned_addr learned my addr 172.21.15.32:0/2493931038 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:43.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a8a56b640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a840c41a0 0x7f4a840c4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89d6a640 1 -- 172.21.15.32:0/2493931038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a840c41a0 msgr2=0x7f4a840c4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89d6a640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a840c41a0 0x7f4a840c4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:43.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89d6a640 1 -- 172.21.15.32:0/2493931038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8414eeb0 msgr2=0x7f4a8414e680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89d6a640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8414eeb0 0x7f4a8414e680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89d6a640 1 -- 172.21.15.32:0/2493931038 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a840b17f0 con 0x7f4a70003b40 2023-12-20T04:43:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89569640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8414eeb0 0x7f4a8414e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a89d6a640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a70003b40 0x7f4a840c7310 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a700057b0 tx=0x7f4a70005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a7a7fc640 1 -- 172.21.15.32:0/2493931038 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a80005090 con 0x7f4a70003b40 2023-12-20T04:43:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a7a7fc640 1 -- 172.21.15.32:0/2493931038 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a800051f0 con 0x7f4a70003b40 2023-12-20T04:43:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a7a7fc640 1 -- 172.21.15.32:0/2493931038 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a800069e0 con 0x7f4a70003b40 2023-12-20T04:43:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a8bfce640 1 -- 172.21.15.32:0/2493931038 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a840b1ad0 con 0x7f4a70003b40 2023-12-20T04:43:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a8bfce640 1 -- 172.21.15.32:0/2493931038 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a8414ebc0 con 0x7f4a70003b40 2023-12-20T04:43:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.630+0000 7f4a8bfce640 1 -- 172.21.15.32:0/2493931038 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a8404e000 con 0x7f4a70003b40 2023-12-20T04:43:43.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.634+0000 7f4a7a7fc640 1 -- 172.21.15.32:0/2493931038 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f4a800aa070 con 0x7f4a70003b40 2023-12-20T04:43:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.634+0000 7f4a7a7fc640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4a68065af0 0x7f4a68067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.634+0000 7f4a7a7fc640 1 -- 172.21.15.32:0/2493931038 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(82..82 src has 1..82) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f4a8011c420 con 0x7f4a70003b40 2023-12-20T04:43:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.634+0000 7f4a89569640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4a68065af0 0x7f4a68067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:43.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.634+0000 7f4a89569640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4a68065af0 0x7f4a68067fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a840c7970 tx=0x7f4a74001e10 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:43.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.638+0000 7f4a7a7fc640 1 -- 172.21.15.32:0/2493931038 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4a800e9a70 con 0x7f4a70003b40 2023-12-20T04:43:43.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:43.806+0000 7f4a8bfce640 1 -- 172.21.15.32:0/2493931038 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f4a8406d820 con 0x7f4a70003b40 2023-12-20T04:43:44.280 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:44.282+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.410+0000 7f4a7a7fc640 1 -- 172.21.15.32:0/2493931038 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v83) v1 ==== 181+0+0 (secure 0 0 0) 0x7f4a800ed090 con 0x7f4a70003b40 2023-12-20T04:43:44.410 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T04:43:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 -- 172.21.15.32:0/2493931038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4a68065af0 msgr2=0x7f4a68067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4a68065af0 0x7f4a68067fb0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f4a840c7970 tx=0x7f4a74001e10 comp rx=0 tx=0).stop 2023-12-20T04:43:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 -- 172.21.15.32:0/2493931038 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a70003b40 msgr2=0x7f4a840c7310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a70003b40 0x7f4a840c7310 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f4a700057b0 tx=0x7f4a70005800 comp rx=0 tx=0).stop 2023-12-20T04:43:44.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 -- 172.21.15.32:0/2493931038 shutdown_connections 2023-12-20T04:43:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4a68065af0 0x7f4a68067fb0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a840c41a0 0x7f4a840c4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a8414eeb0 0x7f4a8414e680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 --2- 172.21.15.32:0/2493931038 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a70003b40 0x7f4a840c7310 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:44.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 -- 172.21.15.32:0/2493931038 >> 172.21.15.32:0/2493931038 conn(0x7f4a8406a510 msgr2=0x7f4a840cb260 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:44.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 -- 172.21.15.32:0/2493931038 shutdown_connections 2023-12-20T04:43:44.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.414+0000 7f4a8bfce640 1 -- 172.21.15.32:0/2493931038 wait complete. 2023-12-20T04:43:44.437 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T04:43:44.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.590+0000 7efdb359e640 1 Processor -- start 2023-12-20T04:43:44.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.590+0000 7efdb359e640 1 -- start start 2023-12-20T04:43:44.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.590+0000 7efdb359e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdb40cd940 0x7efdb41606e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:44.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.590+0000 7efdb359e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40b8410 0x7efdb415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:44.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.590+0000 7efdb359e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb40d0c10 con 0x7efdb40cdd20 2023-12-20T04:43:44.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.590+0000 7efdb359e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb4160f80 con 0x7efdb40cd940 2023-12-20T04:43:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.590+0000 7efdb359e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb4161240 con 0x7efdb40b8410 2023-12-20T04:43:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.590+0000 7efdb259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdb40cd940 0x7efdb41606e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.594+0000 7efdb259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdb40cd940 0x7efdb41606e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45210/0 (socket says 172.21.15.32:45210) 2023-12-20T04:43:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.594+0000 7efdb259c640 1 -- 172.21.15.32:0/2016300288 learned_addr learned my addr 172.21.15.32:0/2016300288 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.594+0000 7efdb1d9b640 1 --2- 172.21.15.32:0/2016300288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40b8410 0x7efdb415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.594+0000 7efdb259c640 1 -- 172.21.15.32:0/2016300288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40b8410 msgr2=0x7efdb415c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.594+0000 7efdb259c640 1 --2- 172.21.15.32:0/2016300288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40b8410 0x7efdb415c5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.594+0000 7efdb259c640 1 -- 172.21.15.32:0/2016300288 >> v1:172.21.15.32:6789/0 conn(0x7efdb40cdd20 legacy=0x7efdb40b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:44.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.594+0000 7efdb259c640 1 -- 172.21.15.32:0/2016300288 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb40d0e80 con 0x7efdb40cd940 2023-12-20T04:43:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.594+0000 7efdb259c640 1 --2- 172.21.15.32:0/2016300288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb40cd940 0x7efdb41606e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7efda4009870 tx=0x7efda401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c74310b4eb0c751 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.594+0000 7efdb159a640 1 -- 172.21.15.32:0/2016300288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda40a9620 con 0x7efdb40cd940 2023-12-20T04:43:44.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.594+0000 7efdb159a640 1 -- 172.21.15.32:0/2016300288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda40a9780 con 0x7efdb40cd940 2023-12-20T04:43:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.594+0000 7efdb159a640 1 -- 172.21.15.32:0/2016300288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda40a9b10 con 0x7efdb40cd940 2023-12-20T04:43:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 -- 172.21.15.32:0/2016300288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb40cd940 msgr2=0x7efdb41606e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:44.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 --2- 172.21.15.32:0/2016300288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb40cd940 0x7efdb41606e0 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7efda4009870 tx=0x7efda401c540 comp rx=0 tx=0).stop 2023-12-20T04:43:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 -- 172.21.15.32:0/2016300288 shutdown_connections 2023-12-20T04:43:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 --2- 172.21.15.32:0/2016300288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40b8410 0x7efdb415c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 --2- 172.21.15.32:0/2016300288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb40cd940 0x7efdb41606e0 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 -- 172.21.15.32:0/2016300288 >> 172.21.15.32:0/2016300288 conn(0x7efdb40658e0 msgr2=0x7efdb4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 -- 172.21.15.32:0/2016300288 shutdown_connections 2023-12-20T04:43:44.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 -- 172.21.15.32:0/2016300288 wait complete. 2023-12-20T04:43:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 Processor -- start 2023-12-20T04:43:44.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 -- start start 2023-12-20T04:43:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb40b8410 0x7efdb40cbbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb40cd940 0x7efdb40b73c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40cdd20 0x7efdb414fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40ccac0 con 0x7efdb40b8410 2023-12-20T04:43:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40cc500 con 0x7efdb40cd940 2023-12-20T04:43:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb359e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb40cc7e0 con 0x7efdb40cdd20 2023-12-20T04:43:44.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb40b8410 0x7efdb40cbbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb40b8410 0x7efdb40cbbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45216/0 (socket says 172.21.15.32:45216) 2023-12-20T04:43:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb259c640 1 -- 172.21.15.32:0/514564177 learned_addr learned my addr 172.21.15.32:0/514564177 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb1d9b640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb40cd940 0x7efdb40b73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.598+0000 7efdb2d9d640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40cdd20 0x7efdb414fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efdb259c640 1 -- 172.21.15.32:0/514564177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40cdd20 msgr2=0x7efdb414fc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efdb259c640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40cdd20 0x7efdb414fc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efdb259c640 1 -- 172.21.15.32:0/514564177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb40cd940 msgr2=0x7efdb40b73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efdb259c640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb40cd940 0x7efdb40b73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efdb259c640 1 -- 172.21.15.32:0/514564177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda40094d0 con 0x7efdb40b8410 2023-12-20T04:43:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efdb1d9b640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb40cd940 0x7efdb40b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:44.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efdb259c640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb40b8410 0x7efdb40cbbf0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7efda40ab040 tx=0x7efda401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efda2ffd640 1 -- 172.21.15.32:0/514564177 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda40a9e00 con 0x7efdb40b8410 2023-12-20T04:43:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efda2ffd640 1 -- 172.21.15.32:0/514564177 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda40bc450 con 0x7efdb40b8410 2023-12-20T04:43:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efda2ffd640 1 -- 172.21.15.32:0/514564177 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda40bc800 con 0x7efdb40b8410 2023-12-20T04:43:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efdb359e640 1 -- 172.21.15.32:0/514564177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb414ad40 con 0x7efdb40b8410 2023-12-20T04:43:44.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efdb359e640 1 -- 172.21.15.32:0/514564177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb40cc130 con 0x7efdb40b8410 2023-12-20T04:43:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efdb359e640 1 -- 172.21.15.32:0/514564177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb414aba0 con 0x7efdb40b8410 2023-12-20T04:43:44.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efda2ffd640 1 -- 172.21.15.32:0/514564177 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7efda40bc960 con 0x7efdb40b8410 2023-12-20T04:43:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.602+0000 7efda2ffd640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd94065af0 0x7efd94067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.606+0000 7efda2ffd640 1 -- 172.21.15.32:0/514564177 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(83..83 src has 1..83) v4 ==== 5959+0+0 (secure 0 0 0) 0x7efda411d890 con 0x7efdb40b8410 2023-12-20T04:43:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.606+0000 7efdb1d9b640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd94065af0 0x7efd94067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:44.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.606+0000 7efdb1d9b640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd94065af0 0x7efd94067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efdb41679e0 tx=0x7efda8009190 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:44.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.610+0000 7efda2ffd640 1 -- 172.21.15.32:0/514564177 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efdb414aba0 con 0x7efdb40b8410 2023-12-20T04:43:44.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:44.774+0000 7efdb359e640 1 -- 172.21.15.32:0/514564177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7efdb414aba0 con 0x7efdb40b8410 2023-12-20T04:43:44.965 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:44.966+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:45.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.410+0000 7efda2ffd640 1 -- 172.21.15.32:0/514564177 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v84) v1 ==== 169+0+0 (secure 0 0 0) 0x7efdb414aba0 con 0x7efdb40b8410 2023-12-20T04:43:45.411 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T04:43:45.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 -- 172.21.15.32:0/514564177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd94065af0 msgr2=0x7efd94067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd94065af0 0x7efd94067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7efdb41679e0 tx=0x7efda8009190 comp rx=0 tx=0).stop 2023-12-20T04:43:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 -- 172.21.15.32:0/514564177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb40b8410 msgr2=0x7efdb40cbbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb40b8410 0x7efdb40cbbf0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7efda40ab040 tx=0x7efda401ad80 comp rx=0 tx=0).stop 2023-12-20T04:43:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 -- 172.21.15.32:0/514564177 shutdown_connections 2023-12-20T04:43:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd94065af0 0x7efd94067fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb40cdd20 0x7efdb414fc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb40cd940 0x7efdb40b73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 --2- 172.21.15.32:0/514564177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb40b8410 0x7efdb40cbbf0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 -- 172.21.15.32:0/514564177 >> 172.21.15.32:0/514564177 conn(0x7efdb40658e0 msgr2=0x7efdb40d29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 -- 172.21.15.32:0/514564177 shutdown_connections 2023-12-20T04:43:45.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.414+0000 7efdb359e640 1 -- 172.21.15.32:0/514564177 wait complete. 2023-12-20T04:43:45.436 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T04:43:45.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.586+0000 7f6460811640 1 Processor -- start 2023-12-20T04:43:45.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.586+0000 7f6460811640 1 -- start start 2023-12-20T04:43:45.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.586+0000 7f6460811640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f645c159980 0x7f645c071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:45.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.586+0000 7f6460811640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c072330 con 0x7f645c0cdd10 2023-12-20T04:43:45.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.586+0000 7f6460811640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f645c159d60 con 0x7f645c0cb6b0 2023-12-20T04:43:45.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.586+0000 7f6460811640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f645c0cc290 con 0x7f645c159980 2023-12-20T04:43:45.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.590+0000 7f645a59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f645c0cdd10 0x7f645c0cafa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59372/0 (socket says 172.21.15.32:59372) 2023-12-20T04:43:45.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.590+0000 7f645a59c640 1 -- 172.21.15.32:0/664851888 learned_addr learned my addr 172.21.15.32:0/664851888 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:45.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.590+0000 7f645ad9d640 1 --2- 172.21.15.32:0/664851888 >> v2:172.21.15.39:3300/0 conn(0x7f645c159980 0x7f645c071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.590+0000 7f6459d9b640 1 -- 172.21.15.32:0/664851888 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3317764506 0 0) 0x7f645c072330 con 0x7f645c0cdd10 2023-12-20T04:43:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.590+0000 7f6459d9b640 1 -- 172.21.15.32:0/664851888 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6444003530 con 0x7f645c0cdd10 2023-12-20T04:43:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.590+0000 7f6459d9b640 1 -- 172.21.15.32:0/664851888 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 950624498 0 0) 0x7f645c159d60 con 0x7f645c0cb6b0 2023-12-20T04:43:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.590+0000 7f6459d9b640 1 -- 172.21.15.32:0/664851888 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f645c072330 con 0x7f645c0cb6b0 2023-12-20T04:43:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.590+0000 7f645ad9d640 1 -- 172.21.15.32:0/664851888 >> v1:172.21.15.39:6789/0 conn(0x7f645c0cb6b0 legacy=0x7f645c0c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.590+0000 7f645ad9d640 1 -- 172.21.15.32:0/664851888 >> v1:172.21.15.32:6789/0 conn(0x7f645c0cdd10 legacy=0x7f645c0cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.590+0000 7f645ad9d640 1 -- 172.21.15.32:0/664851888 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f645c0cc570 con 0x7f645c159980 2023-12-20T04:43:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.590+0000 7f645ad9d640 1 --2- 172.21.15.32:0/664851888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f645c159980 0x7f645c071df0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f644c0097c0 tx=0x7f644c01c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9749aaf06dc6e1bf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6459d9b640 1 -- 172.21.15.32:0/664851888 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c0ab600 con 0x7f645c159980 2023-12-20T04:43:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6459d9b640 1 -- 172.21.15.32:0/664851888 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c0a7050 con 0x7f645c159980 2023-12-20T04:43:45.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6459d9b640 1 -- 172.21.15.32:0/664851888 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c0ab980 con 0x7f645c159980 2023-12-20T04:43:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6460811640 1 -- 172.21.15.32:0/664851888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f645c159980 msgr2=0x7f645c071df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6460811640 1 --2- 172.21.15.32:0/664851888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f645c159980 0x7f645c071df0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f644c0097c0 tx=0x7f644c01c330 comp rx=0 tx=0).stop 2023-12-20T04:43:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6460811640 1 -- 172.21.15.32:0/664851888 shutdown_connections 2023-12-20T04:43:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6460811640 1 --2- 172.21.15.32:0/664851888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f645c159980 0x7f645c071df0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6460811640 1 -- 172.21.15.32:0/664851888 >> 172.21.15.32:0/664851888 conn(0x7f645c06c060 msgr2=0x7f645c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6460811640 1 -- 172.21.15.32:0/664851888 shutdown_connections 2023-12-20T04:43:45.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6460811640 1 -- 172.21.15.32:0/664851888 wait complete. 2023-12-20T04:43:45.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6460811640 1 Processor -- start 2023-12-20T04:43:45.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.594+0000 7f6460811640 1 -- start start 2023-12-20T04:43:45.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f6460811640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f645c0cb6b0 0x7f645c14b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:45.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f6460811640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f645c0cdd10 0x7f645c14a670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:45.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f6460811640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c159980 0x7f645c1499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:45.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f6460811640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c14abb0 con 0x7f645c159980 2023-12-20T04:43:45.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f6460811640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f645c15b6b0 con 0x7f645c0cdd10 2023-12-20T04:43:45.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f6460811640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f645c15b990 con 0x7f645c0cb6b0 2023-12-20T04:43:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f645ad9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f645c0cb6b0 0x7f645c14b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f645ad9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f645c0cb6b0 0x7f645c14b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:42636/0 (socket says 172.21.15.32:42636) 2023-12-20T04:43:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f645ad9d640 1 -- 172.21.15.32:0/619787033 learned_addr learned my addr 172.21.15.32:0/619787033 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f645b59e640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c159980 0x7f645c1499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f645a59c640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f645c0cdd10 0x7f645c14a670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f645ad9d640 1 -- 172.21.15.32:0/619787033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f645c0cdd10 msgr2=0x7f645c14a670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f645ad9d640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f645c0cdd10 0x7f645c14a670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f645ad9d640 1 -- 172.21.15.32:0/619787033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c159980 msgr2=0x7f645c1499d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f645ad9d640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c159980 0x7f645c1499d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f645ad9d640 1 -- 172.21.15.32:0/619787033 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f644c009420 con 0x7f645c0cb6b0 2023-12-20T04:43:45.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f645ad9d640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f645c0cb6b0 0x7f645c14b430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f644c01c810 tx=0x7f644c0190d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:45.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f64437fe640 1 -- 172.21.15.32:0/619787033 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c0abe50 con 0x7f645c0cb6b0 2023-12-20T04:43:45.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f64437fe640 1 -- 172.21.15.32:0/619787033 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f644c0b4400 con 0x7f645c0cb6b0 2023-12-20T04:43:45.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f64437fe640 1 -- 172.21.15.32:0/619787033 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f644c0b4770 con 0x7f645c0cb6b0 2023-12-20T04:43:45.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f6460811640 1 -- 172.21.15.32:0/619787033 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f645c15bc70 con 0x7f645c0cb6b0 2023-12-20T04:43:45.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f6460811640 1 -- 172.21.15.32:0/619787033 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f645c14a290 con 0x7f645c0cb6b0 2023-12-20T04:43:45.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f6460811640 1 -- 172.21.15.32:0/619787033 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f645c074100 con 0x7f645c0cb6b0 2023-12-20T04:43:45.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.598+0000 7f64437fe640 1 -- 172.21.15.32:0/619787033 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f644c0b2040 con 0x7f645c0cb6b0 2023-12-20T04:43:45.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.602+0000 7f64437fe640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6448065af0 0x7f6448067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:45.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.602+0000 7f64437fe640 1 -- 172.21.15.32:0/619787033 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f644c11cfa0 con 0x7f645c0cb6b0 2023-12-20T04:43:45.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.602+0000 7f645a59c640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6448065af0 0x7f6448067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:45.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.602+0000 7f645a59c640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6448065af0 0x7f6448067fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f645c1f89d0 tx=0x7f6450007d60 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:45.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.606+0000 7f64437fe640 1 -- 172.21.15.32:0/619787033 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f644c0ea8b0 con 0x7f645c0cb6b0 2023-12-20T04:43:45.650 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:45.650+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:45.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.766+0000 7f6460811640 1 -- 172.21.15.32:0/619787033 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f645c15e1e0 con 0x7f645c0cb6b0 2023-12-20T04:43:45.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.770+0000 7f64437fe640 1 -- 172.21.15.32:0/619787033 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v25) v1 ==== 120+0+0 (secure 0 0 0) 0x7f644c0b00b0 con 0x7f645c0cb6b0 2023-12-20T04:43:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.770+0000 7f6460811640 1 -- 172.21.15.32:0/619787033 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6448065af0 msgr2=0x7f6448067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.770+0000 7f6460811640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6448065af0 0x7f6448067fb0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f645c1f89d0 tx=0x7f6450007d60 comp rx=0 tx=0).stop 2023-12-20T04:43:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.770+0000 7f6460811640 1 -- 172.21.15.32:0/619787033 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f645c0cb6b0 msgr2=0x7f645c14b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.770+0000 7f6460811640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f645c0cb6b0 0x7f645c14b430 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f644c01c810 tx=0x7f644c0190d0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.774+0000 7f6460811640 1 -- 172.21.15.32:0/619787033 shutdown_connections 2023-12-20T04:43:45.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.774+0000 7f6460811640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6448065af0 0x7f6448067fb0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.774+0000 7f6460811640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f645c159980 0x7f645c1499d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.774+0000 7f6460811640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f645c0cdd10 0x7f645c14a670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.774+0000 7f6460811640 1 --2- 172.21.15.32:0/619787033 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f645c0cb6b0 0x7f645c14b430 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.774+0000 7f6460811640 1 -- 172.21.15.32:0/619787033 >> 172.21.15.32:0/619787033 conn(0x7f645c06c060 msgr2=0x7f645c159f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.774+0000 7f6460811640 1 -- 172.21.15.32:0/619787033 shutdown_connections 2023-12-20T04:43:45.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.774+0000 7f6460811640 1 -- 172.21.15.32:0/619787033 wait complete. 2023-12-20T04:43:45.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T04:43:45.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.946+0000 7f89f6666640 1 Processor -- start 2023-12-20T04:43:45.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.946+0000 7f89f6666640 1 -- start start 2023-12-20T04:43:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89f6666640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f89f00b1980 0x7f89f00cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89f6666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89f00cd8b0 0x7f89f00cec30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89f6666640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f89f015c4e0 con 0x7f89f015c100 2023-12-20T04:43:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89f6666640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f89f00ce810 con 0x7f89f00b1980 2023-12-20T04:43:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89f6666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00cdd60 con 0x7f89f00cd8b0 2023-12-20T04:43:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89effff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f89f015c100 0x7f89f00cb6b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59374/0 (socket says 172.21.15.32:59374) 2023-12-20T04:43:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89effff640 1 -- 172.21.15.32:0/2320794227 learned_addr learned my addr 172.21.15.32:0/2320794227 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:45.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89ef7fe640 1 --2- 172.21.15.32:0/2320794227 >> v2:172.21.15.32:3300/0 conn(0x7f89f00b1980 0x7f89f00cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89f4c03640 1 --2- 172.21.15.32:0/2320794227 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89f00cd8b0 0x7f89f00cec30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89ef7fe640 1 -- 172.21.15.32:0/2320794227 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89f00cd8b0 msgr2=0x7f89f00cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89ef7fe640 1 --2- 172.21.15.32:0/2320794227 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89f00cd8b0 0x7f89f00cec30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89ef7fe640 1 -- 172.21.15.32:0/2320794227 >> v1:172.21.15.32:6789/0 conn(0x7f89f015c100 legacy=0x7f89f00cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.950+0000 7f89ef7fe640 1 -- 172.21.15.32:0/2320794227 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89f00cc120 con 0x7f89f00b1980 2023-12-20T04:43:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89ef7fe640 1 --2- 172.21.15.32:0/2320794227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f00b1980 0x7f89f00cd370 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f89e40097c0 tx=0x7f89e401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=736c3a32bf12aa62 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89eeffd640 1 -- 172.21.15.32:0/2320794227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40ab650 con 0x7f89f00b1980 2023-12-20T04:43:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89eeffd640 1 -- 172.21.15.32:0/2320794227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e40ab7b0 con 0x7f89f00b1980 2023-12-20T04:43:45.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89eeffd640 1 -- 172.21.15.32:0/2320794227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40abb40 con 0x7f89f00b1980 2023-12-20T04:43:45.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89f6666640 1 -- 172.21.15.32:0/2320794227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f00b1980 msgr2=0x7f89f00cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89f6666640 1 --2- 172.21.15.32:0/2320794227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f00b1980 0x7f89f00cd370 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7f89e40097c0 tx=0x7f89e401c450 comp rx=0 tx=0).stop 2023-12-20T04:43:45.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89f6666640 1 -- 172.21.15.32:0/2320794227 shutdown_connections 2023-12-20T04:43:45.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89f6666640 1 --2- 172.21.15.32:0/2320794227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f00b1980 0x7f89f00cd370 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89f6666640 1 --2- 172.21.15.32:0/2320794227 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89f00cd8b0 0x7f89f00cec30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89f6666640 1 -- 172.21.15.32:0/2320794227 >> 172.21.15.32:0/2320794227 conn(0x7f89f0099d70 msgr2=0x7f89f00d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89f6666640 1 -- 172.21.15.32:0/2320794227 shutdown_connections 2023-12-20T04:43:45.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89f6666640 1 -- 172.21.15.32:0/2320794227 wait complete. 2023-12-20T04:43:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89f6666640 1 Processor -- start 2023-12-20T04:43:45.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.954+0000 7f89f6666640 1 -- start start 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89f6666640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89f00b1980 0x7f89f01f50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89f6666640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f00cd8b0 0x7f89f01f8e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89f6666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89f015c100 0x7f89f01f9350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89f6666640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00b55e0 con 0x7f89f00cd8b0 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89f6666640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00b58c0 con 0x7f89f00b1980 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89f6666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f89f00b5ba0 con 0x7f89f015c100 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f00cd8b0 0x7f89f01f8e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f00cd8b0 0x7f89f01f8e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45242/0 (socket says 172.21.15.32:45242) 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ef7fe640 1 -- 172.21.15.32:0/133645578 learned_addr learned my addr 172.21.15.32:0/133645578 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89effff640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89f00b1980 0x7f89f01f50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89f4c03640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89f015c100 0x7f89f01f9350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:45.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ef7fe640 1 -- 172.21.15.32:0/133645578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89f015c100 msgr2=0x7f89f01f9350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ef7fe640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89f015c100 0x7f89f01f9350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ef7fe640 1 -- 172.21.15.32:0/133645578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89f00b1980 msgr2=0x7f89f01f50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:45.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ef7fe640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89f00b1980 0x7f89f01f50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:45.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ef7fe640 1 -- 172.21.15.32:0/133645578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f89e4009420 con 0x7f89f00cd8b0 2023-12-20T04:43:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ef7fe640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f00cd8b0 0x7f89f01f8e10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f89e40ad040 tx=0x7f89e401ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ecff9640 1 -- 172.21.15.32:0/133645578 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40b5810 con 0x7f89f00cd8b0 2023-12-20T04:43:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89f6666640 1 -- 172.21.15.32:0/133645578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f89f00b5e80 con 0x7f89f00cd8b0 2023-12-20T04:43:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ecff9640 1 -- 172.21.15.32:0/133645578 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89e40b5970 con 0x7f89f00cd8b0 2023-12-20T04:43:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ecff9640 1 -- 172.21.15.32:0/133645578 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89e40b5d20 con 0x7f89f00cd8b0 2023-12-20T04:43:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89f6666640 1 -- 172.21.15.32:0/133645578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f89f00c90b0 con 0x7f89f00cd8b0 2023-12-20T04:43:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89f6666640 1 -- 172.21.15.32:0/133645578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f89f004e000 con 0x7f89f00cd8b0 2023-12-20T04:43:45.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.958+0000 7f89ecff9640 1 -- 172.21.15.32:0/133645578 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f89e40bf410 con 0x7f89f00cd8b0 2023-12-20T04:43:45.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.962+0000 7f89ecff9640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f89dc065af0 0x7f89dc067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:45.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.962+0000 7f89ecff9640 1 -- 172.21.15.32:0/133645578 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f89e411de90 con 0x7f89f00cd8b0 2023-12-20T04:43:45.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.962+0000 7f89effff640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f89dc065af0 0x7f89dc067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:45.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.962+0000 7f89effff640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f89dc065af0 0x7f89dc067fb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000f70 tx=0x7f89e0007e90 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:45.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:45.966+0000 7f89ecff9640 1 -- 172.21.15.32:0/133645578 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f89e40eba40 con 0x7f89f00cd8b0 2023-12-20T04:43:46.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.138+0000 7f89f6666640 1 -- 172.21.15.32:0/133645578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f89f008ab60 con 0x7f89f00cd8b0 2023-12-20T04:43:46.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.142+0000 7f89ecff9640 1 -- 172.21.15.32:0/133645578 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v26) v1 ==== 115+0+0 (secure 0 0 0) 0x7f89e40ab610 con 0x7f89f00cd8b0 2023-12-20T04:43:46.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 -- 172.21.15.32:0/133645578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f89dc065af0 msgr2=0x7f89dc067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f89dc065af0 0x7f89dc067fb0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f89e0000f70 tx=0x7f89e0007e90 comp rx=0 tx=0).stop 2023-12-20T04:43:46.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 -- 172.21.15.32:0/133645578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f00cd8b0 msgr2=0x7f89f01f8e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f00cd8b0 0x7f89f01f8e10 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7f89e40ad040 tx=0x7f89e401ac90 comp rx=0 tx=0).stop 2023-12-20T04:43:46.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 -- 172.21.15.32:0/133645578 shutdown_connections 2023-12-20T04:43:46.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f89dc065af0 0x7f89dc067fb0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f89f015c100 0x7f89f01f9350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f89f00cd8b0 0x7f89f01f8e10 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 --2- 172.21.15.32:0/133645578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f89f00b1980 0x7f89f01f50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 -- 172.21.15.32:0/133645578 >> 172.21.15.32:0/133645578 conn(0x7f89f0099d70 msgr2=0x7f89f00b2590 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 -- 172.21.15.32:0/133645578 shutdown_connections 2023-12-20T04:43:46.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.146+0000 7f89f6666640 1 -- 172.21.15.32:0/133645578 wait complete. 2023-12-20T04:43:46.165 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T04:43:46.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.314+0000 7f6c3d55a640 1 Processor -- start 2023-12-20T04:43:46.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.314+0000 7f6c3d55a640 1 -- start start 2023-12-20T04:43:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.314+0000 7f6c3d55a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6c380b0010 0x7f6c380b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.314+0000 7f6c3d55a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c380c9ff0 con 0x7f6c3815a920 2023-12-20T04:43:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.314+0000 7f6c3d55a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c380c93e0 con 0x7f6c380b16c0 2023-12-20T04:43:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.314+0000 7f6c3d55a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c380b4510 con 0x7f6c380b0010 2023-12-20T04:43:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.314+0000 7f6c37fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6c3815a920 0x7f6c380ca690 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59386/0 (socket says 172.21.15.32:59386) 2023-12-20T04:43:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.314+0000 7f6c37fff640 1 -- 172.21.15.32:0/765109392 learned_addr learned my addr 172.21.15.32:0/765109392 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:46.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c377fe640 1 --2- 172.21.15.32:0/765109392 >> v2:172.21.15.39:3300/0 conn(0x7f6c380b0010 0x7f6c380b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c36ffd640 1 -- 172.21.15.32:0/765109392 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2052969391 0 0) 0x7f6c380c9ff0 con 0x7f6c3815a920 2023-12-20T04:43:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c36ffd640 1 -- 172.21.15.32:0/765109392 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c20003530 con 0x7f6c3815a920 2023-12-20T04:43:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c36ffd640 1 -- 172.21.15.32:0/765109392 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 521989028 0 0) 0x7f6c380c93e0 con 0x7f6c380b16c0 2023-12-20T04:43:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c36ffd640 1 -- 172.21.15.32:0/765109392 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c380c9ff0 con 0x7f6c380b16c0 2023-12-20T04:43:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c377fe640 1 -- 172.21.15.32:0/765109392 >> v1:172.21.15.39:6789/0 conn(0x7f6c380b16c0 legacy=0x7f6c380b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c377fe640 1 -- 172.21.15.32:0/765109392 >> v1:172.21.15.32:6789/0 conn(0x7f6c3815a920 legacy=0x7f6c380ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c377fe640 1 -- 172.21.15.32:0/765109392 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c380c96c0 con 0x7f6c380b0010 2023-12-20T04:43:46.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c377fe640 1 --2- 172.21.15.32:0/765109392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c380b0010 0x7f6c380b04a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0097c0 tx=0x7f6c2c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7dfaa09c2aa44c35 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c36ffd640 1 -- 172.21.15.32:0/765109392 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c0ab600 con 0x7f6c380b0010 2023-12-20T04:43:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c36ffd640 1 -- 172.21.15.32:0/765109392 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c2c0a7050 con 0x7f6c380b0010 2023-12-20T04:43:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.318+0000 7f6c36ffd640 1 -- 172.21.15.32:0/765109392 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c2c0ab980 con 0x7f6c380b0010 2023-12-20T04:43:46.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 -- 172.21.15.32:0/765109392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c380b0010 msgr2=0x7f6c380b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 --2- 172.21.15.32:0/765109392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c380b0010 0x7f6c380b04a0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0097c0 tx=0x7f6c2c01c480 comp rx=0 tx=0).stop 2023-12-20T04:43:46.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 -- 172.21.15.32:0/765109392 shutdown_connections 2023-12-20T04:43:46.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 --2- 172.21.15.32:0/765109392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c380b0010 0x7f6c380b04a0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 -- 172.21.15.32:0/765109392 >> 172.21.15.32:0/765109392 conn(0x7f6c3806c060 msgr2=0x7f6c3806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:46.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 -- 172.21.15.32:0/765109392 shutdown_connections 2023-12-20T04:43:46.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 -- 172.21.15.32:0/765109392 wait complete. 2023-12-20T04:43:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 Processor -- start 2023-12-20T04:43:46.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 -- start start 2023-12-20T04:43:46.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c380b0010 0x7f6c380cd990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:46.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c380b16c0 0x7f6c380c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:46.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3815a920 0x7f6c38151130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:46.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380c2c40 con 0x7f6c380b0010 2023-12-20T04:43:46.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380c49e0 con 0x7f6c3815a920 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c3d55a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c380cded0 con 0x7f6c380b16c0 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c37fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c380b0010 0x7f6c380cd990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c37fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c380b0010 0x7f6c380cd990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45258/0 (socket says 172.21.15.32:45258) 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c37fff640 1 -- 172.21.15.32:0/740768978 learned_addr learned my addr 172.21.15.32:0/740768978 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.322+0000 7f6c377fe640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c380b16c0 0x7f6c380c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c3cd59640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3815a920 0x7f6c38151130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c37fff640 1 -- 172.21.15.32:0/740768978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c380b16c0 msgr2=0x7f6c380c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c37fff640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c380b16c0 0x7f6c380c20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c37fff640 1 -- 172.21.15.32:0/740768978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3815a920 msgr2=0x7f6c38151130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c37fff640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3815a920 0x7f6c38151130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c37fff640 1 -- 172.21.15.32:0/740768978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c2c009420 con 0x7f6c380b0010 2023-12-20T04:43:46.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c37fff640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c380b0010 0x7f6c380cd990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c500 tx=0x7f6c2800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:46.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c34ff9640 1 -- 172.21.15.32:0/740768978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28005cf0 con 0x7f6c380b0010 2023-12-20T04:43:46.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c34ff9640 1 -- 172.21.15.32:0/740768978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c28005e50 con 0x7f6c380b0010 2023-12-20T04:43:46.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c34ff9640 1 -- 172.21.15.32:0/740768978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c28007330 con 0x7f6c380b0010 2023-12-20T04:43:46.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c3d55a640 1 -- 172.21.15.32:0/740768978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c380c2960 con 0x7f6c380b0010 2023-12-20T04:43:46.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c3d55a640 1 -- 172.21.15.32:0/740768978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c380c4670 con 0x7f6c380b0010 2023-12-20T04:43:46.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c3d55a640 1 -- 172.21.15.32:0/740768978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c38074100 con 0x7f6c380b0010 2023-12-20T04:43:46.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.326+0000 7f6c34ff9640 1 -- 172.21.15.32:0/740768978 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f6c28007490 con 0x7f6c380b0010 2023-12-20T04:43:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.330+0000 7f6c34ff9640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c24065b20 0x7f6c24067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.330+0000 7f6c34ff9640 1 -- 172.21.15.32:0/740768978 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f6c28082770 con 0x7f6c380b0010 2023-12-20T04:43:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.330+0000 7f6c377fe640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c24065b20 0x7f6c24067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:46.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.330+0000 7f6c377fe640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c24065b20 0x7f6c24067fe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0ad000 tx=0x7f6c2c008040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:46.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.330+0000 7f6c34ff9640 1 -- 172.21.15.32:0/740768978 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c28050220 con 0x7f6c380b0010 2023-12-20T04:43:46.337 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:46.338+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:46.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.490+0000 7f6c3d55a640 1 -- 172.21.15.32:0/740768978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f6c380af5e0 con 0x7f6c380b0010 2023-12-20T04:43:46.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c34ff9640 1 -- 172.21.15.32:0/740768978 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v27) v1 ==== 104+0+0 (secure 0 0 0) 0x7f6c28053840 con 0x7f6c380b0010 2023-12-20T04:43:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 -- 172.21.15.32:0/740768978 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c24065b20 msgr2=0x7f6c24067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c24065b20 0x7f6c24067fe0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f6c2c0ad000 tx=0x7f6c2c008040 comp rx=0 tx=0).stop 2023-12-20T04:43:46.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 -- 172.21.15.32:0/740768978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c380b0010 msgr2=0x7f6c380cd990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c380b0010 0x7f6c380cd990 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f6c2800c500 tx=0x7f6c2800c9c0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 -- 172.21.15.32:0/740768978 shutdown_connections 2023-12-20T04:43:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c24065b20 0x7f6c24067fe0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c3815a920 0x7f6c38151130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c380b16c0 0x7f6c380c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 --2- 172.21.15.32:0/740768978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c380b0010 0x7f6c380cd990 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 -- 172.21.15.32:0/740768978 >> 172.21.15.32:0/740768978 conn(0x7f6c3806c060 msgr2=0x7f6c380ae9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 -- 172.21.15.32:0/740768978 shutdown_connections 2023-12-20T04:43:46.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.498+0000 7f6c3d55a640 1 -- 172.21.15.32:0/740768978 wait complete. 2023-12-20T04:43:46.518 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule' 2023-12-20T04:43:46.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7ab59e640 1 Processor -- start 2023-12-20T04:43:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7ab59e640 1 -- start start 2023-12-20T04:43:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7ab59e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe7ac15cd00 0x7fe7ac161450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7ab59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7ac0ccd10 0x7fe7ac06a770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7ab59e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7ac069d20 con 0x7fe7ac06c8f0 2023-12-20T04:43:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7ab59e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ac0cd560 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7ab59e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7ac0cd840 con 0x7fe7ac15cd00 2023-12-20T04:43:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7a9d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7ac0ccd10 0x7fe7ac06a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7a9d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7ac0ccd10 0x7fe7ac06a770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45262/0 (socket says 172.21.15.32:45262) 2023-12-20T04:43:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7a9d9b640 1 -- 172.21.15.32:0/1101488059 learned_addr learned my addr 172.21.15.32:0/1101488059 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7aa59c640 1 --2- 172.21.15.32:0/1101488059 >> v2:172.21.15.39:3300/0 conn(0x7fe7ac15cd00 0x7fe7ac161450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7a9d9b640 1 -- 172.21.15.32:0/1101488059 >> v2:172.21.15.39:3300/0 conn(0x7fe7ac15cd00 msgr2=0x7fe7ac161450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7a9d9b640 1 --2- 172.21.15.32:0/1101488059 >> v2:172.21.15.39:3300/0 conn(0x7fe7ac15cd00 0x7fe7ac161450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7a9d9b640 1 -- 172.21.15.32:0/1101488059 >> v1:172.21.15.39:6789/0 conn(0x7fe7ac06c8f0 legacy=0x7fe7ac0c95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.666+0000 7fe7a9d9b640 1 -- 172.21.15.32:0/1101488059 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7ac069f70 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7a9d9b640 1 --2- 172.21.15.32:0/1101488059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7ac0ccd10 0x7fe7ac06a770 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac06fef0 tx=0x7fe7a001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd343b0e2b54f115 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7a959a640 1 -- 172.21.15.32:0/1101488059 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00ab600 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7a959a640 1 -- 172.21.15.32:0/1101488059 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a00a7050 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7a959a640 1 -- 172.21.15.32:0/1101488059 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00ab980 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7ab59e640 1 -- 172.21.15.32:0/1101488059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7ac0ccd10 msgr2=0x7fe7ac06a770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7ab59e640 1 --2- 172.21.15.32:0/1101488059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7ac0ccd10 0x7fe7ac06a770 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7fe7ac06fef0 tx=0x7fe7a001c470 comp rx=0 tx=0).stop 2023-12-20T04:43:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7ab59e640 1 -- 172.21.15.32:0/1101488059 shutdown_connections 2023-12-20T04:43:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7ab59e640 1 --2- 172.21.15.32:0/1101488059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7ac0ccd10 0x7fe7ac06a770 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7ab59e640 1 --2- 172.21.15.32:0/1101488059 >> v2:172.21.15.39:3300/0 conn(0x7fe7ac15cd00 0x7fe7ac161450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7ab59e640 1 -- 172.21.15.32:0/1101488059 >> 172.21.15.32:0/1101488059 conn(0x7fe7ac0cee10 msgr2=0x7fe7ac0cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7ab59e640 1 -- 172.21.15.32:0/1101488059 shutdown_connections 2023-12-20T04:43:46.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.670+0000 7fe7ab59e640 1 -- 172.21.15.32:0/1101488059 wait complete. 2023-12-20T04:43:46.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7ab59e640 1 Processor -- start 2023-12-20T04:43:46.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7ab59e640 1 -- start start 2023-12-20T04:43:46.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7ab59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7ac06c8f0 0x7fe7ac158410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7ab59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7ac0ccd10 0x7fe7ac0c83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7ab59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ac15cd00 0x7fe7ac15d0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7ab59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0c77e0 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7ab59e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac158950 con 0x7fe7ac15cd00 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7ab59e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7ac0c68f0 con 0x7fe7ac06c8f0 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7a9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7ac0ccd10 0x7fe7ac0c83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7aa59c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7ac06c8f0 0x7fe7ac158410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7a9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7ac0ccd10 0x7fe7ac0c83f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45274/0 (socket says 172.21.15.32:45274) 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7a9d9b640 1 -- 172.21.15.32:0/2780615535 learned_addr learned my addr 172.21.15.32:0/2780615535 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7aad9d640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ac15cd00 0x7fe7ac15d0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7a9d9b640 1 -- 172.21.15.32:0/2780615535 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7ac06c8f0 msgr2=0x7fe7ac158410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7a9d9b640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7ac06c8f0 0x7fe7ac158410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7a9d9b640 1 -- 172.21.15.32:0/2780615535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ac15cd00 msgr2=0x7fe7ac15d0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7a9d9b640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ac15cd00 0x7fe7ac15d0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7a9d9b640 1 -- 172.21.15.32:0/2780615535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7a00094d0 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7a9d9b640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7ac0ccd10 0x7fe7ac0c83f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00ae040 tx=0x7fe7a0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe79affd640 1 -- 172.21.15.32:0/2780615535 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00abde0 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe79affd640 1 -- 172.21.15.32:0/2780615535 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7a00b4440 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7ab59e640 1 -- 172.21.15.32:0/2780615535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7ac06d290 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe79affd640 1 -- 172.21.15.32:0/2780615535 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7a00b47d0 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.674+0000 7fe7ab59e640 1 -- 172.21.15.32:0/2780615535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7ac0c6eb0 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.678+0000 7fe7ab59e640 1 -- 172.21.15.32:0/2780615535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7ac074100 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.678+0000 7fe79affd640 1 -- 172.21.15.32:0/2780615535 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fe7a00b2070 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.678+0000 7fe79affd640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe78c065af0 0x7fe78c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:46.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.678+0000 7fe79affd640 1 -- 172.21.15.32:0/2780615535 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5545+0+0 (secure 0 0 0) 0x7fe7a011cfa0 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.678+0000 7fe7aa59c640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe78c065af0 0x7fe78c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:46.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.678+0000 7fe7aa59c640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe78c065af0 0x7fe78c067fb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe79c007a60 tx=0x7fe79c008040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:46.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.682+0000 7fe79affd640 1 -- 172.21.15.32:0/2780615535 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe7a00ea8b0 con 0x7fe7ac0ccd10 2023-12-20T04:43:46.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:46.898+0000 7fe7ab59e640 1 -- 172.21.15.32:0/2780615535 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]} v 0) v1 -- 0x7fe7ac065ea0 con 0x7fe7ac0ccd10 2023-12-20T04:43:47.013 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:47.014+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:47.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.422+0000 7fe79affd640 1 -- 172.21.15.32:0/2780615535 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_schedule"]}]=0 v558) v1 ==== 156+0+0 (secure 0 0 0) 0x7fe7a00b00e0 con 0x7fe7ac0ccd10 2023-12-20T04:43:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 -- 172.21.15.32:0/2780615535 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe78c065af0 msgr2=0x7fe78c067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe78c065af0 0x7fe78c067fb0 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fe79c007a60 tx=0x7fe79c008040 comp rx=0 tx=0).stop 2023-12-20T04:43:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 -- 172.21.15.32:0/2780615535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7ac0ccd10 msgr2=0x7fe7ac0c83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7ac0ccd10 0x7fe7ac0c83f0 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7fe7a00ae040 tx=0x7fe7a0019210 comp rx=0 tx=0).stop 2023-12-20T04:43:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 -- 172.21.15.32:0/2780615535 shutdown_connections 2023-12-20T04:43:47.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe78c065af0 0x7fe78c067fb0 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7ac15cd00 0x7fe7ac15d0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7ac0ccd10 0x7fe7ac0c83f0 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 --2- 172.21.15.32:0/2780615535 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7ac06c8f0 0x7fe7ac158410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 -- 172.21.15.32:0/2780615535 >> 172.21.15.32:0/2780615535 conn(0x7fe7ac0cee10 msgr2=0x7fe7ac0ae770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 -- 172.21.15.32:0/2780615535 shutdown_connections 2023-12-20T04:43:47.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.426+0000 7fe7ab59e640 1 -- 172.21.15.32:0/2780615535 wait complete. 2023-12-20T04:43:47.445 INFO:tasks.cephfs_test_runner:Test listing non-existing snap schedules failure ... ok 2023-12-20T04:43:47.445 INFO:tasks.cephfs_test_runner:Starting test: test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules on a non-existing filesystem path failure 2023-12-20T04:43:47.445 INFO:tasks.cephfs_test_runner:test_non_existent_snap_schedule_list (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-20T04:43:47.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-12-20T04:43:47.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.594+0000 7f520ea6f640 1 Processor -- start 2023-12-20T04:43:47.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.594+0000 7f520ea6f640 1 -- start start 2023-12-20T04:43:47.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520ea6f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52080b8480 0x7f520806af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:47.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520ea6f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f520806b460 0x7f5208160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:47.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520ea6f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f520806b8c0 con 0x7f5208161330 2023-12-20T04:43:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520ea6f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52080cd4d0 con 0x7f520806b460 2023-12-20T04:43:47.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520ea6f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52080ce4e0 con 0x7f52080b8480 2023-12-20T04:43:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520da6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52080b8480 0x7f520806af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520d26c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f520806b460 0x7f5208160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520e26e640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f5208161330 0x7f5208153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:52650/0 (socket says 172.21.15.32:52650) 2023-12-20T04:43:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520da6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52080b8480 0x7f520806af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:42670/0 (socket says 172.21.15.32:42670) 2023-12-20T04:43:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520e26e640 1 -- 172.21.15.32:0/1979302523 learned_addr learned my addr 172.21.15.32:0/1979302523 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520da6d640 1 -- 172.21.15.32:0/1979302523 >> v2:172.21.15.39:3300/0 conn(0x7f520806b460 msgr2=0x7f5208160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520da6d640 1 --2- 172.21.15.32:0/1979302523 >> v2:172.21.15.39:3300/0 conn(0x7f520806b460 0x7f5208160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520da6d640 1 -- 172.21.15.32:0/1979302523 >> v1:172.21.15.39:6789/0 conn(0x7f5208161330 legacy=0x7f5208153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520da6d640 1 -- 172.21.15.32:0/1979302523 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52080cd720 con 0x7f52080b8480 2023-12-20T04:43:47.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.598+0000 7f520ca6b640 1 -- 172.21.15.32:0/1979302523 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 421609860 0 0) 0x7f520806b8c0 con 0x7f5208161330 2023-12-20T04:43:47.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520da6d640 1 --2- 172.21.15.32:0/1979302523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52080b8480 0x7f520806af20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f52040097c0 tx=0x7f520401c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=43d5b211f5e5fb62 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:47.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520ca6b640 1 -- 172.21.15.32:0/1979302523 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52040ab600 con 0x7f52080b8480 2023-12-20T04:43:47.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520ca6b640 1 -- 172.21.15.32:0/1979302523 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52040a7050 con 0x7f52080b8480 2023-12-20T04:43:47.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520ca6b640 1 -- 172.21.15.32:0/1979302523 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52040ab980 con 0x7f52080b8480 2023-12-20T04:43:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520ea6f640 1 -- 172.21.15.32:0/1979302523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52080b8480 msgr2=0x7f520806af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520ea6f640 1 --2- 172.21.15.32:0/1979302523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52080b8480 0x7f520806af20 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f52040097c0 tx=0x7f520401c480 comp rx=0 tx=0).stop 2023-12-20T04:43:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520ea6f640 1 -- 172.21.15.32:0/1979302523 shutdown_connections 2023-12-20T04:43:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520ea6f640 1 --2- 172.21.15.32:0/1979302523 >> v2:172.21.15.39:3300/0 conn(0x7f520806b460 0x7f5208160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520ea6f640 1 --2- 172.21.15.32:0/1979302523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52080b8480 0x7f520806af20 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520ea6f640 1 -- 172.21.15.32:0/1979302523 >> 172.21.15.32:0/1979302523 conn(0x7f52080d2a50 msgr2=0x7f52080d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520ea6f640 1 -- 172.21.15.32:0/1979302523 shutdown_connections 2023-12-20T04:43:47.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.602+0000 7f520ea6f640 1 -- 172.21.15.32:0/1979302523 wait complete. 2023-12-20T04:43:47.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520ea6f640 1 Processor -- start 2023-12-20T04:43:47.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520ea6f640 1 -- start start 2023-12-20T04:43:47.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520ea6f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f520806b460 0x7f5208167aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:47.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520ea6f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52080b8480 0x7f520814fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:47.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520ea6f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5208161330 0x7f520815f9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:47.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520ea6f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52080b6d80 con 0x7f520806b460 2023-12-20T04:43:47.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520ea6f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5208167fe0 con 0x7f52080b8480 2023-12-20T04:43:47.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520ea6f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5208168210 con 0x7f5208161330 2023-12-20T04:43:47.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520da6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f520806b460 0x7f5208167aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:47.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520da6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f520806b460 0x7f5208167aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45278/0 (socket says 172.21.15.32:45278) 2023-12-20T04:43:47.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520da6d640 1 -- 172.21.15.32:0/2794302770 learned_addr learned my addr 172.21.15.32:0/2794302770 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:47.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520d26c640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52080b8480 0x7f520814fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520e26e640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5208161330 0x7f520815f9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520da6d640 1 -- 172.21.15.32:0/2794302770 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5208161330 msgr2=0x7f520815f9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520da6d640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5208161330 0x7f520815f9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520da6d640 1 -- 172.21.15.32:0/2794302770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52080b8480 msgr2=0x7f520814fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520da6d640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52080b8480 0x7f520814fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520da6d640 1 -- 172.21.15.32:0/2794302770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5204009420 con 0x7f520806b460 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520da6d640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f520806b460 0x7f5208167aa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f520401cd20 tx=0x7f5204019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f51fe7fc640 1 -- 172.21.15.32:0/2794302770 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5204004020 con 0x7f520806b460 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f51fe7fc640 1 -- 172.21.15.32:0/2794302770 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5204004180 con 0x7f520806b460 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f51fe7fc640 1 -- 172.21.15.32:0/2794302770 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52040044f0 con 0x7f520806b460 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520ea6f640 1 -- 172.21.15.32:0/2794302770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52081684f0 con 0x7f520806b460 2023-12-20T04:43:47.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520ea6f640 1 -- 172.21.15.32:0/2794302770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5208168ab0 con 0x7f520806b460 2023-12-20T04:43:47.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.606+0000 7f520ea6f640 1 -- 172.21.15.32:0/2794302770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5208074100 con 0x7f520806b460 2023-12-20T04:43:47.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.610+0000 7f51fe7fc640 1 -- 172.21.15.32:0/2794302770 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f52040b2040 con 0x7f520806b460 2023-12-20T04:43:47.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.610+0000 7f51fe7fc640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51ec065af0 0x7f51ec067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:47.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.610+0000 7f51fe7fc640 1 -- 172.21.15.32:0/2794302770 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f520411c510 con 0x7f520806b460 2023-12-20T04:43:47.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.610+0000 7f520d26c640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51ec065af0 0x7f51ec067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:47.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.610+0000 7f520d26c640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51ec065af0 0x7f51ec067fb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f52080d2010 tx=0x7f51f8009190 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:47.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.614+0000 7f51fe7fc640 1 -- 172.21.15.32:0/2794302770 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f52040e9f20 con 0x7f520806b460 2023-12-20T04:43:47.706 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:47.706+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:47.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:47.826+0000 7f520ea6f640 1 -- 172.21.15.32:0/2794302770 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f52080d0a70 con 0x7f520806b460 2023-12-20T04:43:48.406 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:48.406+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:48.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f51fe7fc640 1 -- 172.21.15.32:0/2794302770 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v559) v1 ==== 169+0+0 (secure 0 0 0) 0x7f52040b00f0 con 0x7f520806b460 2023-12-20T04:43:48.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 -- 172.21.15.32:0/2794302770 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51ec065af0 msgr2=0x7f51ec067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:48.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51ec065af0 0x7f51ec067fb0 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f52080d2010 tx=0x7f51f8009190 comp rx=0 tx=0).stop 2023-12-20T04:43:48.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 -- 172.21.15.32:0/2794302770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f520806b460 msgr2=0x7f5208167aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:48.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f520806b460 0x7f5208167aa0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f520401cd20 tx=0x7f5204019220 comp rx=0 tx=0).stop 2023-12-20T04:43:48.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 -- 172.21.15.32:0/2794302770 shutdown_connections 2023-12-20T04:43:48.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f51ec065af0 0x7f51ec067fb0 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5208161330 0x7f520815f9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52080b8480 0x7f520814fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 --2- 172.21.15.32:0/2794302770 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f520806b460 0x7f5208167aa0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 -- 172.21.15.32:0/2794302770 >> 172.21.15.32:0/2794302770 conn(0x7f52080d2a50 msgr2=0x7f520815e620 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:48.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 -- 172.21.15.32:0/2794302770 shutdown_connections 2023-12-20T04:43:48.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.426+0000 7f520ea6f640 1 -- 172.21.15.32:0/2794302770 wait complete. 2023-12-20T04:43:48.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T04:43:48.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f451e998640 1 Processor -- start 2023-12-20T04:43:48.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f451e998640 1 -- start start 2023-12-20T04:43:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f451e998640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f451815abb0 0x7f45180caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f451e998640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45180b0cb0 0x7f451815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f451e998640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f451815b6a0 con 0x7f4518070230 2023-12-20T04:43:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f451e998640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f451806e160 con 0x7f451815abb0 2023-12-20T04:43:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f451e998640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451815c070 con 0x7f45180b0cb0 2023-12-20T04:43:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f451cf35640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4518070230 0x7f451806c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:59398/0 (socket says 172.21.15.32:59398) 2023-12-20T04:43:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f451cf35640 1 -- 172.21.15.32:0/1760460918 learned_addr learned my addr 172.21.15.32:0/1760460918 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f45177fe640 1 --2- 172.21.15.32:0/1760460918 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45180b0cb0 0x7f451815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:48.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f4517fff640 1 --2- 172.21.15.32:0/1760460918 >> v2:172.21.15.39:3300/0 conn(0x7f451815abb0 0x7f45180caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f4516ffd640 1 -- 172.21.15.32:0/1760460918 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4089292837 0 0) 0x7f451815b6a0 con 0x7f4518070230 2023-12-20T04:43:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f4517fff640 1 -- 172.21.15.32:0/1760460918 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45180b0cb0 msgr2=0x7f451815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f4517fff640 1 --2- 172.21.15.32:0/1760460918 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45180b0cb0 0x7f451815bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f4517fff640 1 -- 172.21.15.32:0/1760460918 >> v1:172.21.15.32:6789/0 conn(0x7f4518070230 legacy=0x7f451806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f4517fff640 1 -- 172.21.15.32:0/1760460918 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f451806d8c0 con 0x7f451815abb0 2023-12-20T04:43:48.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.602+0000 7f45177fe640 1 --2- 172.21.15.32:0/1760460918 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45180b0cb0 0x7f451815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f4517fff640 1 --2- 172.21.15.32:0/1760460918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f451815abb0 0x7f45180caaf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f45080097c0 tx=0x7f450801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7c7b49ec92796d71 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:48.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f4516ffd640 1 -- 172.21.15.32:0/1760460918 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080ab680 con 0x7f451815abb0 2023-12-20T04:43:48.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f4516ffd640 1 -- 172.21.15.32:0/1760460918 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45080ab7e0 con 0x7f451815abb0 2023-12-20T04:43:48.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f4516ffd640 1 -- 172.21.15.32:0/1760460918 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45080abb70 con 0x7f451815abb0 2023-12-20T04:43:48.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f451e998640 1 -- 172.21.15.32:0/1760460918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f451815abb0 msgr2=0x7f45180caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:48.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f451e998640 1 --2- 172.21.15.32:0/1760460918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f451815abb0 0x7f45180caaf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f45080097c0 tx=0x7f450801c480 comp rx=0 tx=0).stop 2023-12-20T04:43:48.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f451e998640 1 -- 172.21.15.32:0/1760460918 shutdown_connections 2023-12-20T04:43:48.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f451e998640 1 --2- 172.21.15.32:0/1760460918 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45180b0cb0 0x7f451815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f451e998640 1 --2- 172.21.15.32:0/1760460918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f451815abb0 0x7f45180caaf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f451e998640 1 -- 172.21.15.32:0/1760460918 >> 172.21.15.32:0/1760460918 conn(0x7f451806a510 msgr2=0x7f45180ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:48.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f451e998640 1 -- 172.21.15.32:0/1760460918 shutdown_connections 2023-12-20T04:43:48.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.606+0000 7f451e998640 1 -- 172.21.15.32:0/1760460918 wait complete. 2023-12-20T04:43:48.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451e998640 1 Processor -- start 2023-12-20T04:43:48.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451e998640 1 -- start start 2023-12-20T04:43:48.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451e998640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518070230 0x7f45180b3d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:48.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451e998640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45180b0cb0 0x7f451813bb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:48.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451e998640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f451815abb0 0x7f4518148b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:48.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451e998640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45180b4290 con 0x7f4518070230 2023-12-20T04:43:48.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451e998640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f451814c500 con 0x7f45180b0cb0 2023-12-20T04:43:48.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451e998640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f451814c7e0 con 0x7f451815abb0 2023-12-20T04:43:48.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4517fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518070230 0x7f45180b3d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:48.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4517fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518070230 0x7f45180b3d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45284/0 (socket says 172.21.15.32:45284) 2023-12-20T04:43:48.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4517fff640 1 -- 172.21.15.32:0/3915021905 learned_addr learned my addr 172.21.15.32:0/3915021905 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:48.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f45177fe640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45180b0cb0 0x7f451813bb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:48.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451cf35640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f451815abb0 0x7f4518148b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:48.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4517fff640 1 -- 172.21.15.32:0/3915021905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f451815abb0 msgr2=0x7f4518148b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:48.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4517fff640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f451815abb0 0x7f4518148b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4517fff640 1 -- 172.21.15.32:0/3915021905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45180b0cb0 msgr2=0x7f451813bb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:48.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4517fff640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45180b0cb0 0x7f451813bb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4517fff640 1 -- 172.21.15.32:0/3915021905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f450c009420 con 0x7f4518070230 2023-12-20T04:43:48.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451cf35640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f451815abb0 0x7f4518148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4517fff640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518070230 0x7f45180b3d50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f450801a0b0 tx=0x7f450801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4514ff9640 1 -- 172.21.15.32:0/3915021905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508004220 con 0x7f4518070230 2023-12-20T04:43:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4514ff9640 1 -- 172.21.15.32:0/3915021905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45080b9070 con 0x7f4518070230 2023-12-20T04:43:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f4514ff9640 1 -- 172.21.15.32:0/3915021905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4508003620 con 0x7f4518070230 2023-12-20T04:43:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451e998640 1 -- 172.21.15.32:0/3915021905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4508009420 con 0x7f4518070230 2023-12-20T04:43:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.610+0000 7f451e998640 1 -- 172.21.15.32:0/3915021905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f451814cda0 con 0x7f4518070230 2023-12-20T04:43:48.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.614+0000 7f451e998640 1 -- 172.21.15.32:0/3915021905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f451804e000 con 0x7f4518070230 2023-12-20T04:43:48.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.614+0000 7f4514ff9640 1 -- 172.21.15.32:0/3915021905 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f4508003780 con 0x7f4518070230 2023-12-20T04:43:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.614+0000 7f4514ff9640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4504065af0 0x7f4504067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.614+0000 7f4514ff9640 1 -- 172.21.15.32:0/3915021905 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f45080bb070 con 0x7f4518070230 2023-12-20T04:43:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.614+0000 7f45177fe640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4504065af0 0x7f4504067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:48.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.614+0000 7f45177fe640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4504065af0 0x7f4504067fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f451814dc80 tx=0x7f450c009190 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:48.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.618+0000 7f4514ff9640 1 -- 172.21.15.32:0/3915021905 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f45080eb810 con 0x7f4518070230 2023-12-20T04:43:48.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.782+0000 7f451e998640 1 -- 172.21.15.32:0/3915021905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f45180b4950 con 0x7f4518070230 2023-12-20T04:43:48.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f4514ff9640 1 -- 172.21.15.32:0/3915021905 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v28)=0 v28) v1 ==== 122+0+0 (secure 0 0 0) 0x7f45080c4020 con 0x7f4518070230 2023-12-20T04:43:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 -- 172.21.15.32:0/3915021905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4504065af0 msgr2=0x7f4504067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4504065af0 0x7f4504067fb0 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f451814dc80 tx=0x7f450c009190 comp rx=0 tx=0).stop 2023-12-20T04:43:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 -- 172.21.15.32:0/3915021905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518070230 msgr2=0x7f45180b3d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518070230 0x7f45180b3d50 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f450801a0b0 tx=0x7f450801acc0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 -- 172.21.15.32:0/3915021905 shutdown_connections 2023-12-20T04:43:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4504065af0 0x7f4504067fb0 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f451815abb0 0x7f4518148b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45180b0cb0 0x7f451813bb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 --2- 172.21.15.32:0/3915021905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4518070230 0x7f45180b3d50 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:48.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 -- 172.21.15.32:0/3915021905 >> 172.21.15.32:0/3915021905 conn(0x7f451806a510 msgr2=0x7f45180c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 -- 172.21.15.32:0/3915021905 shutdown_connections 2023-12-20T04:43:48.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:48.790+0000 7f451e998640 1 -- 172.21.15.32:0/3915021905 wait complete. 2023-12-20T04:43:48.810 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:43:48.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:43:48.897 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:43:49.004 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:43:49.005 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:43:49.005 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:49.014 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:43:49.014 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:43:49.014 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:49.024 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:43:49.024 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:43:49.024 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:49.063 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:43:49.063 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:43:49.064 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:43:49.078 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:43:49.079 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T04:43:49.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:49.102 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:49.102+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:49.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.246+0000 7ffaffad8640 1 Processor -- start 2023-12-20T04:43:49.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.246+0000 7ffaffad8640 1 -- start start 2023-12-20T04:43:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffaffad8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf80ae010 0x7ffaf8071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffaffad8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffaf806cf10 0x7ffaf80719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffaffad8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffaf80ae3f0 con 0x7ffaf8164ef0 2023-12-20T04:43:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffaffad8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffaf80ca4b0 con 0x7ffaf806cf10 2023-12-20T04:43:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffaffad8640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf8161800 con 0x7ffaf80ae010 2023-12-20T04:43:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafd073640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffaf806cf10 0x7ffaf80719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafd874640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf80ae010 0x7ffaf8071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafd073640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffaf806cf10 0x7ffaf80719b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46208/0 (socket says 172.21.15.32:46208) 2023-12-20T04:43:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafe075640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ffaf8164ef0 0x7ffaf80701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:58664/0 (socket says 172.21.15.32:58664) 2023-12-20T04:43:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafd073640 1 -- 172.21.15.32:0/3717446901 learned_addr learned my addr 172.21.15.32:0/3717446901 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafd073640 1 -- 172.21.15.32:0/3717446901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf80ae010 msgr2=0x7ffaf8071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafd073640 1 --2- 172.21.15.32:0/3717446901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf80ae010 0x7ffaf8071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafd073640 1 -- 172.21.15.32:0/3717446901 >> v1:172.21.15.39:6789/0 conn(0x7ffaf8164ef0 legacy=0x7ffaf80701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:49.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafc872640 1 -- 172.21.15.32:0/3717446901 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2584919930 0 0) 0x7ffaf80ae3f0 con 0x7ffaf8164ef0 2023-12-20T04:43:49.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafd073640 1 -- 172.21.15.32:0/3717446901 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffaf8165380 con 0x7ffaf806cf10 2023-12-20T04:43:49.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafd073640 1 --2- 172.21.15.32:0/3717446901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaf806cf10 0x7ffaf80719b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009870 tx=0x7ffae801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=626934b848651260 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.250+0000 7ffafc872640 1 -- 172.21.15.32:0/3717446901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae80ab680 con 0x7ffaf806cf10 2023-12-20T04:43:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffafc872640 1 -- 172.21.15.32:0/3717446901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae80ab7e0 con 0x7ffaf806cf10 2023-12-20T04:43:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffafc872640 1 -- 172.21.15.32:0/3717446901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae80abb70 con 0x7ffaf806cf10 2023-12-20T04:43:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 -- 172.21.15.32:0/3717446901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaf806cf10 msgr2=0x7ffaf80719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 --2- 172.21.15.32:0/3717446901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaf806cf10 0x7ffaf80719b0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7ffae8009870 tx=0x7ffae801c480 comp rx=0 tx=0).stop 2023-12-20T04:43:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 -- 172.21.15.32:0/3717446901 shutdown_connections 2023-12-20T04:43:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 --2- 172.21.15.32:0/3717446901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaf806cf10 0x7ffaf80719b0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 --2- 172.21.15.32:0/3717446901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf80ae010 0x7ffaf8071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 -- 172.21.15.32:0/3717446901 >> 172.21.15.32:0/3717446901 conn(0x7ffaf806a320 msgr2=0x7ffaf80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 -- 172.21.15.32:0/3717446901 shutdown_connections 2023-12-20T04:43:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 -- 172.21.15.32:0/3717446901 wait complete. 2023-12-20T04:43:49.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 Processor -- start 2023-12-20T04:43:49.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 -- start start 2023-12-20T04:43:49.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaf80ae010 0x7ffaf80b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:49.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf814ff50 0x7ffaf8158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaf8162c90 0x7ffaf814e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf81503d0 con 0x7ffaf8162c90 2023-12-20T04:43:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf8150df0 con 0x7ffaf80ae010 2023-12-20T04:43:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffaffad8640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffaf81508c0 con 0x7ffaf814ff50 2023-12-20T04:43:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffafe075640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaf8162c90 0x7ffaf814e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.254+0000 7ffafd073640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf814ff50 0x7ffaf8158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffafd874640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaf80ae010 0x7ffaf80b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffafd073640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf814ff50 0x7ffaf8158060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:39484/0 (socket says 172.21.15.32:39484) 2023-12-20T04:43:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffafd073640 1 -- 172.21.15.32:0/2375582549 learned_addr learned my addr 172.21.15.32:0/2375582549 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffafe075640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaf8162c90 0x7ffaf814e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46224/0 (socket says 172.21.15.32:46224) 2023-12-20T04:43:49.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffafd073640 1 -- 172.21.15.32:0/2375582549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaf80ae010 msgr2=0x7ffaf80b3ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffafd073640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaf80ae010 0x7ffaf80b3ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffafd073640 1 -- 172.21.15.32:0/2375582549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaf8162c90 msgr2=0x7ffaf814e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffafd073640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaf8162c90 0x7ffaf814e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffafd073640 1 -- 172.21.15.32:0/2375582549 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffae80094d0 con 0x7ffaf814ff50 2023-12-20T04:43:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffafd874640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaf80ae010 0x7ffaf80b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:49.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffafd073640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf814ff50 0x7ffaf8158060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffae801a0b0 tx=0x7ffae801acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffaee7fc640 1 -- 172.21.15.32:0/2375582549 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae80abd60 con 0x7ffaf814ff50 2023-12-20T04:43:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffaee7fc640 1 -- 172.21.15.32:0/2375582549 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffae80b8070 con 0x7ffaf814ff50 2023-12-20T04:43:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffaffad8640 1 -- 172.21.15.32:0/2375582549 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffaf81505e0 con 0x7ffaf814ff50 2023-12-20T04:43:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffaee7fc640 1 -- 172.21.15.32:0/2375582549 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffae8004220 con 0x7ffaf814ff50 2023-12-20T04:43:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffaffad8640 1 -- 172.21.15.32:0/2375582549 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffaf815f790 con 0x7ffaf814ff50 2023-12-20T04:43:49.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffaffad8640 1 -- 172.21.15.32:0/2375582549 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffaf804e000 con 0x7ffaf814ff50 2023-12-20T04:43:49.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.258+0000 7ffaee7fc640 1 -- 172.21.15.32:0/2375582549 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7ffae8004380 con 0x7ffaf814ff50 2023-12-20T04:43:49.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.262+0000 7ffaee7fc640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffadc065af0 0x7ffadc067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:49.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.262+0000 7ffaee7fc640 1 -- 172.21.15.32:0/2375582549 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(84..84 src has 1..84) v4 ==== 5545+0+0 (secure 0 0 0) 0x7ffae811da60 con 0x7ffaf814ff50 2023-12-20T04:43:49.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.262+0000 7ffafd874640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffadc065af0 0x7ffadc067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:49.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.262+0000 7ffafd874640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffadc065af0 0x7ffadc067fb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009f30 tx=0x7ffaf4006f90 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:49.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.262+0000 7ffaee7fc640 1 -- 172.21.15.32:0/2375582549 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffae80eb610 con 0x7ffaf814ff50 2023-12-20T04:43:49.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.462+0000 7ffaffad8640 1 -- 172.21.15.32:0/2375582549 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ffaf81585a0 con 0x7ffaf814ff50 2023-12-20T04:43:49.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.462+0000 7ffaee7fc640 1 -- 172.21.15.32:0/2375582549 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 33 v33) v1 ==== 94+0+3526 (secure 0 0 0) 0x7ffae80ba020 con 0x7ffaf814ff50 2023-12-20T04:43:49.462 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:49.462 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":33,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":5780,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/4279891602","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4279891602},{"type":"v1","addr":"172.21.15.39:6834","nonce":4279891602}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6292,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3937321478","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3937321478},{"type":"v1","addr":"172.21.15.39:6835","nonce":3937321478}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6393,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/404004710","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":404004710},{"type":"v1","addr":"172.21.15.32:6835","nonce":404004710}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":27},{"gid":6777,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2269137878","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2269137878},{"type":"v1","addr":"172.21.15.32:6837","nonce":2269137878}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":33}],"filesystems":[]} 2023-12-20T04:43:49.462 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 33 2023-12-20T04:43:49.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 -- 172.21.15.32:0/2375582549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffadc065af0 msgr2=0x7ffadc067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:49.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffadc065af0 0x7ffadc067fb0 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7ffaf4009f30 tx=0x7ffaf4006f90 comp rx=0 tx=0).stop 2023-12-20T04:43:49.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 -- 172.21.15.32:0/2375582549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf814ff50 msgr2=0x7ffaf8158060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:49.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf814ff50 0x7ffaf8158060 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7ffae801a0b0 tx=0x7ffae801acc0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 -- 172.21.15.32:0/2375582549 shutdown_connections 2023-12-20T04:43:49.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffadc065af0 0x7ffadc067fb0 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffaf8162c90 0x7ffaf814e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffaf814ff50 0x7ffaf8158060 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 --2- 172.21.15.32:0/2375582549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffaf80ae010 0x7ffaf80b3ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 -- 172.21.15.32:0/2375582549 >> 172.21.15.32:0/2375582549 conn(0x7ffaf806a320 msgr2=0x7ffaf815a6b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 -- 172.21.15.32:0/2375582549 shutdown_connections 2023-12-20T04:43:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.466+0000 7ffaffad8640 1 -- 172.21.15.32:0/2375582549 wait complete. 2023-12-20T04:43:49.484 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T04:43:49.484 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T04:43:49.484 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T04:43:49.485 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:43:49.485 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T04:43:49.485 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T04:43:49.485 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T04:43:49.485 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:43:49.485 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T04:43:49.485 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T04:43:49.486 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T04:43:49.486 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:43:49.486 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T04:43:49.486 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T04:43:49.487 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T04:43:49.487 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:43:49.497 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T04:43:49.498 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T04:43:49.498 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T04:43:49.499 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T04:43:49.500 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T04:43:49.500 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T04:43:49.501 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T04:43:49.501 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T04:43:49.542 INFO:tasks.ceph.mds.a:Started 2023-12-20T04:43:49.542 DEBUG:teuthology.parallel:result is None 2023-12-20T04:43:49.544 INFO:tasks.ceph.mds.c:Started 2023-12-20T04:43:49.544 DEBUG:teuthology.parallel:result is None 2023-12-20T04:43:49.549 INFO:tasks.ceph.mds.b:Started 2023-12-20T04:43:49.549 DEBUG:teuthology.parallel:result is None 2023-12-20T04:43:49.551 INFO:tasks.ceph.mds.d:Started 2023-12-20T04:43:49.552 DEBUG:teuthology.parallel:result is None 2023-12-20T04:43:49.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T04:43:49.634 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T04:43:49.635 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T04:43:49.643 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T04:43:49.643 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T04:43:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a5d198640 1 Processor -- start 2023-12-20T04:43:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a5d198640 1 -- start start 2023-12-20T04:43:49.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a5d198640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5806ce60 0x7f2a5815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:49.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a5d198640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5815bbd0 con 0x7f2a580af5e0 2023-12-20T04:43:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a5d198640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2a5815beb0 con 0x7f2a5815f110 2023-12-20T04:43:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a5d198640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a5815e910 con 0x7f2a5806ce60 2023-12-20T04:43:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a56d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2a580af5e0 0x7f2a580af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34858/0 (socket says 172.21.15.32:34858) 2023-12-20T04:43:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a56d9d640 1 -- 172.21.15.32:0/3455767467 learned_addr learned my addr 172.21.15.32:0/3455767467 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a5659c640 1 --2- 172.21.15.32:0/3455767467 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5806ce60 0x7f2a5815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a55d9b640 1 -- 172.21.15.32:0/3455767467 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 213838480 0 0) 0x7f2a5815bbd0 con 0x7f2a580af5e0 2023-12-20T04:43:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a55d9b640 1 -- 172.21.15.32:0/3455767467 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a40003530 con 0x7f2a580af5e0 2023-12-20T04:43:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a55d9b640 1 -- 172.21.15.32:0/3455767467 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 491749119 0 0) 0x7f2a5815beb0 con 0x7f2a5815f110 2023-12-20T04:43:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a55d9b640 1 -- 172.21.15.32:0/3455767467 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2a5815bbd0 con 0x7f2a5815f110 2023-12-20T04:43:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a5659c640 1 -- 172.21.15.32:0/3455767467 >> v1:172.21.15.39:6789/0 conn(0x7f2a5815f110 legacy=0x7f2a5815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a5659c640 1 -- 172.21.15.32:0/3455767467 >> v1:172.21.15.32:6789/0 conn(0x7f2a580af5e0 legacy=0x7f2a580af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.714+0000 7f2a5659c640 1 -- 172.21.15.32:0/3455767467 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a5815ebf0 con 0x7f2a5806ce60 2023-12-20T04:43:49.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.718+0000 7f2a5659c640 1 --2- 172.21.15.32:0/3455767467 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5806ce60 0x7f2a5815b280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0097f0 tx=0x7f2a4c01c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=afd72e87f105ccdf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:49.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.718+0000 7f2a55d9b640 1 -- 172.21.15.32:0/3455767467 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a5815b9d0 con 0x7f2a5806ce60 2023-12-20T04:43:49.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.718+0000 7f2a55d9b640 1 -- 172.21.15.32:0/3455767467 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a4c0ac630 con 0x7f2a5806ce60 2023-12-20T04:43:49.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.718+0000 7f2a55d9b640 1 -- 172.21.15.32:0/3455767467 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a4c0ac9c0 con 0x7f2a5806ce60 2023-12-20T04:43:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.718+0000 7f2a5d198640 1 -- 172.21.15.32:0/3455767467 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5806ce60 msgr2=0x7f2a5815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.718+0000 7f2a5d198640 1 --2- 172.21.15.32:0/3455767467 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5806ce60 0x7f2a5815b280 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0097f0 tx=0x7f2a4c01c430 comp rx=0 tx=0).stop 2023-12-20T04:43:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.718+0000 7f2a5d198640 1 -- 172.21.15.32:0/3455767467 shutdown_connections 2023-12-20T04:43:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.718+0000 7f2a5d198640 1 --2- 172.21.15.32:0/3455767467 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5806ce60 0x7f2a5815b280 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.718+0000 7f2a5d198640 1 -- 172.21.15.32:0/3455767467 >> 172.21.15.32:0/3455767467 conn(0x7f2a5806a320 msgr2=0x7f2a580ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.718+0000 7f2a5d198640 1 -- 172.21.15.32:0/3455767467 shutdown_connections 2023-12-20T04:43:49.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.718+0000 7f2a5d198640 1 -- 172.21.15.32:0/3455767467 wait complete. 2023-12-20T04:43:49.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5d198640 1 Processor -- start 2023-12-20T04:43:49.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5d198640 1 -- start start 2023-12-20T04:43:49.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5d198640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5806ce60 0x7f2a580c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:49.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5d198640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a580af5e0 0x7f2a580c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:49.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5d198640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5815f110 0x7f2a5815d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:49.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5d198640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580c8710 con 0x7f2a5806ce60 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5d198640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2a580c3d60 con 0x7f2a580af5e0 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5d198640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2a58153d20 con 0x7f2a5815f110 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a56d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5806ce60 0x7f2a580c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a56d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5806ce60 0x7f2a580c3820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46260/0 (socket says 172.21.15.32:46260) 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a56d9d640 1 -- 172.21.15.32:0/2291006194 learned_addr learned my addr 172.21.15.32:0/2291006194 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5659c640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a580af5e0 0x7f2a580c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5759e640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5815f110 0x7f2a5815d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a56d9d640 1 -- 172.21.15.32:0/2291006194 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5815f110 msgr2=0x7f2a5815d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a56d9d640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5815f110 0x7f2a5815d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a56d9d640 1 -- 172.21.15.32:0/2291006194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a580af5e0 msgr2=0x7f2a580c7cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a56d9d640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a580af5e0 0x7f2a580c7cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a56d9d640 1 -- 172.21.15.32:0/2291006194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a4c009420 con 0x7f2a5806ce60 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5659c640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a580af5e0 0x7f2a580c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:49.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5759e640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5815f110 0x7f2a5815d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a56d9d640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5806ce60 0x7f2a580c3820 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c730 tx=0x7f2a4800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/2291006194 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a48007270 con 0x7f2a5806ce60 2023-12-20T04:43:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/2291006194 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a480073d0 con 0x7f2a5806ce60 2023-12-20T04:43:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5d198640 1 -- 172.21.15.32:0/2291006194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2a58154000 con 0x7f2a5806ce60 2023-12-20T04:43:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/2291006194 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a480057b0 con 0x7f2a5806ce60 2023-12-20T04:43:49.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.722+0000 7f2a5d198640 1 -- 172.21.15.32:0/2291006194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2a581545c0 con 0x7f2a5806ce60 2023-12-20T04:43:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.726+0000 7f2a5d198640 1 -- 172.21.15.32:0/2291006194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2a5804e000 con 0x7f2a5806ce60 2023-12-20T04:43:49.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.726+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/2291006194 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f2a48005910 con 0x7f2a5806ce60 2023-12-20T04:43:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.726+0000 7f2a3f7fe640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2a44065af0 0x7f2a44067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.726+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/2291006194 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(88..88 src has 1..88) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f2a48082620 con 0x7f2a5806ce60 2023-12-20T04:43:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.726+0000 7f2a5659c640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2a44065af0 0x7f2a44067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:49.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.726+0000 7f2a5659c640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2a44065af0 0x7f2a44067fb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0ae000 tx=0x7f2a4c008040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:49.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.730+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/2291006194 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2a48050140 con 0x7f2a5806ce60 2023-12-20T04:43:49.823 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:49.822+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:49.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:49.902+0000 7f2a5d198640 1 -- 172.21.15.32:0/2291006194 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f2a580cea60 con 0x7f2a5806ce60 2023-12-20T04:43:50.524 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:50.526+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:50.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a3f7fe640 1 -- 172.21.15.32:0/2291006194 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v89) v1 ==== 97+0+0 (secure 0 0 0) 0x7f2a48017080 con 0x7f2a5806ce60 2023-12-20T04:43:50.712 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T04:43:50.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 -- 172.21.15.32:0/2291006194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2a44065af0 msgr2=0x7f2a44067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:50.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2a44065af0 0x7f2a44067fb0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f2a4c0ae000 tx=0x7f2a4c008040 comp rx=0 tx=0).stop 2023-12-20T04:43:50.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 -- 172.21.15.32:0/2291006194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5806ce60 msgr2=0x7f2a580c3820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:50.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5806ce60 0x7f2a580c3820 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7f2a4800c730 tx=0x7f2a4800cbf0 comp rx=0 tx=0).stop 2023-12-20T04:43:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 -- 172.21.15.32:0/2291006194 shutdown_connections 2023-12-20T04:43:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2a44065af0 0x7f2a44067fb0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2a5815f110 0x7f2a5815d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2a580af5e0 0x7f2a580c7cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 --2- 172.21.15.32:0/2291006194 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2a5806ce60 0x7f2a580c3820 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 -- 172.21.15.32:0/2291006194 >> 172.21.15.32:0/2291006194 conn(0x7f2a5806a320 msgr2=0x7f2a5815a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 -- 172.21.15.32:0/2291006194 shutdown_connections 2023-12-20T04:43:50.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.714+0000 7f2a5d198640 1 -- 172.21.15.32:0/2291006194 wait complete. 2023-12-20T04:43:50.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T04:43:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b3e295640 1 Processor -- start 2023-12-20T04:43:50.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b3e295640 1 -- start start 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b3e295640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b380af5e0 0x7f5b380af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b3e295640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5b3806ce60 0x7f5b3815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b3e295640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b3815bbd0 con 0x7f5b3815f110 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b3e295640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3815beb0 con 0x7f5b380af5e0 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b3e295640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b3815e910 con 0x7f5b3806ce60 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b37fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b380af5e0 0x7f5b380af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b37fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b380af5e0 0x7f5b380af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46264/0 (socket says 172.21.15.32:46264) 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b37fff640 1 -- 172.21.15.32:0/3213396435 learned_addr learned my addr 172.21.15.32:0/3213396435 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b377fe640 1 --2- 172.21.15.32:0/3213396435 >> v2:172.21.15.39:3300/0 conn(0x7f5b3806ce60 0x7f5b3815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b37fff640 1 -- 172.21.15.32:0/3213396435 >> v2:172.21.15.39:3300/0 conn(0x7f5b3806ce60 msgr2=0x7f5b3815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b37fff640 1 --2- 172.21.15.32:0/3213396435 >> v2:172.21.15.39:3300/0 conn(0x7f5b3806ce60 0x7f5b3815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:50.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b37fff640 1 -- 172.21.15.32:0/3213396435 >> v1:172.21.15.39:6789/0 conn(0x7f5b3815f110 legacy=0x7f5b3815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:50.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.882+0000 7f5b37fff640 1 -- 172.21.15.32:0/3213396435 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b3815ebf0 con 0x7f5b380af5e0 2023-12-20T04:43:50.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.886+0000 7f5b37fff640 1 --2- 172.21.15.32:0/3213396435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b380af5e0 0x7f5b380af9c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5b3806abe0 tx=0x7f5b2801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c1edf1d6d18f586 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:50.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.886+0000 7f5b36ffd640 1 -- 172.21.15.32:0/3213396435 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b280a6040 con 0x7f5b380af5e0 2023-12-20T04:43:50.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.886+0000 7f5b36ffd640 1 -- 172.21.15.32:0/3213396435 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b280aa600 con 0x7f5b380af5e0 2023-12-20T04:43:50.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.886+0000 7f5b36ffd640 1 -- 172.21.15.32:0/3213396435 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b280aa990 con 0x7f5b380af5e0 2023-12-20T04:43:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.886+0000 7f5b3e295640 1 -- 172.21.15.32:0/3213396435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b380af5e0 msgr2=0x7f5b380af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.886+0000 7f5b3e295640 1 --2- 172.21.15.32:0/3213396435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b380af5e0 0x7f5b380af9c0 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7f5b3806abe0 tx=0x7f5b2801c500 comp rx=0 tx=0).stop 2023-12-20T04:43:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.886+0000 7f5b3e295640 1 -- 172.21.15.32:0/3213396435 shutdown_connections 2023-12-20T04:43:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.886+0000 7f5b3e295640 1 --2- 172.21.15.32:0/3213396435 >> v2:172.21.15.39:3300/0 conn(0x7f5b3806ce60 0x7f5b3815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.886+0000 7f5b3e295640 1 --2- 172.21.15.32:0/3213396435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b380af5e0 0x7f5b380af9c0 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.886+0000 7f5b3e295640 1 -- 172.21.15.32:0/3213396435 >> 172.21.15.32:0/3213396435 conn(0x7f5b3806a320 msgr2=0x7f5b380ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:50.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3e295640 1 -- 172.21.15.32:0/3213396435 shutdown_connections 2023-12-20T04:43:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3e295640 1 -- 172.21.15.32:0/3213396435 wait complete. 2023-12-20T04:43:50.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3e295640 1 Processor -- start 2023-12-20T04:43:50.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3e295640 1 -- start start 2023-12-20T04:43:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3e295640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5b3806ce60 0x7f5b3814be50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3e295640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b380af5e0 0x7f5b38157610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3e295640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b3815f110 0x7f5b38157b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3e295640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b3815aa20 con 0x7f5b380af5e0 2023-12-20T04:43:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3e295640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b38150350 con 0x7f5b3815f110 2023-12-20T04:43:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3e295640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b38150630 con 0x7f5b3806ce60 2023-12-20T04:43:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b380af5e0 0x7f5b38157610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:50.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b380af5e0 0x7f5b38157610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46270/0 (socket says 172.21.15.32:46270) 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b377fe640 1 -- 172.21.15.32:0/2212062005 learned_addr learned my addr 172.21.15.32:0/2212062005 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b37fff640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5b3806ce60 0x7f5b3814be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3c832640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b3815f110 0x7f5b38157b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b377fe640 1 -- 172.21.15.32:0/2212062005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5b3806ce60 msgr2=0x7f5b3814be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b377fe640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5b3806ce60 0x7f5b3814be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b377fe640 1 -- 172.21.15.32:0/2212062005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b3815f110 msgr2=0x7f5b38157b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b377fe640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b3815f110 0x7f5b38157b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b377fe640 1 -- 172.21.15.32:0/2212062005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b28009480 con 0x7f5b380af5e0 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b37fff640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5b3806ce60 0x7f5b3814be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b3c832640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b3815f110 0x7f5b38157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.890+0000 7f5b377fe640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b380af5e0 0x7f5b38157610 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00e930 tx=0x7f5b2c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.894+0000 7f5b34ff9640 1 -- 172.21.15.32:0/2212062005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c009690 con 0x7f5b380af5e0 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.894+0000 7f5b34ff9640 1 -- 172.21.15.32:0/2212062005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b2c00ca00 con 0x7f5b380af5e0 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.894+0000 7f5b34ff9640 1 -- 172.21.15.32:0/2212062005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b2c00cd70 con 0x7f5b380af5e0 2023-12-20T04:43:50.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.894+0000 7f5b3e295640 1 -- 172.21.15.32:0/2212062005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b38150910 con 0x7f5b380af5e0 2023-12-20T04:43:50.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.894+0000 7f5b3e295640 1 -- 172.21.15.32:0/2212062005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b381523e0 con 0x7f5b380af5e0 2023-12-20T04:43:50.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.894+0000 7f5b3e295640 1 -- 172.21.15.32:0/2212062005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b3804e000 con 0x7f5b380af5e0 2023-12-20T04:43:50.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.894+0000 7f5b34ff9640 1 -- 172.21.15.32:0/2212062005 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f5b2c013470 con 0x7f5b380af5e0 2023-12-20T04:43:50.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.894+0000 7f5b34ff9640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5b24065af0 0x7f5b24067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:50.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.894+0000 7f5b34ff9640 1 -- 172.21.15.32:0/2212062005 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5b2c083110 con 0x7f5b380af5e0 2023-12-20T04:43:50.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.894+0000 7f5b37fff640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5b24065af0 0x7f5b24067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:50.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.898+0000 7f5b37fff640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5b24065af0 0x7f5b24067fb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5b280ae000 tx=0x7f5b28008040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:50.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:50.898+0000 7f5b34ff9640 1 -- 172.21.15.32:0/2212062005 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5b2c050f30 con 0x7f5b380af5e0 2023-12-20T04:43:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 -- 172.21.15.32:0/2212062005 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f5b380cf450 con 0x7f5b380af5e0 2023-12-20T04:43:51.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b34ff9640 1 -- 172.21.15.32:0/2212062005 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v17) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f5b2c054550 con 0x7f5b380af5e0 2023-12-20T04:43:51.120 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:51.120 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:43:51.120 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T04:43:51.121 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:51.122 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.123 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:51.124 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.125 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:51.126 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T04:43:51.127 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.128 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T04:43:51.129 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:43:51.130 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.131 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T04:43:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T04:43:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:43:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:43:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:43:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.132 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:43:51.133 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.134 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:51.134 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T04:43:51.134 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 -- 172.21.15.32:0/2212062005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5b24065af0 msgr2=0x7f5b24067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5b24065af0 0x7f5b24067fb0 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5b280ae000 tx=0x7f5b28008040 comp rx=0 tx=0).stop 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 -- 172.21.15.32:0/2212062005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b380af5e0 msgr2=0x7f5b38157610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b380af5e0 0x7f5b38157610 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x7f5b2c00e930 tx=0x7f5b2c00edf0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 -- 172.21.15.32:0/2212062005 shutdown_connections 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5b24065af0 0x7f5b24067fb0 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b3815f110 0x7f5b38157b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b380af5e0 0x7f5b38157610 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 --2- 172.21.15.32:0/2212062005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5b3806ce60 0x7f5b3814be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 -- 172.21.15.32:0/2212062005 >> 172.21.15.32:0/2212062005 conn(0x7f5b3806a320 msgr2=0x7f5b3806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 -- 172.21.15.32:0/2212062005 shutdown_connections 2023-12-20T04:43:51.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.122+0000 7f5b3e295640 1 -- 172.21.15.32:0/2212062005 wait complete. 2023-12-20T04:43:51.142 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:51.237 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:51.238+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:51.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.290+0000 7fbd76e04640 1 Processor -- start 2023-12-20T04:43:51.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.290+0000 7fbd76e04640 1 -- start start 2023-12-20T04:43:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd76e04640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbd7015ace0 0x7fbd700b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd76e04640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbd700709d0 0x7fbd700afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd76e04640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbd70070570 con 0x7fbd70070e30 2023-12-20T04:43:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd76e04640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd700cab80 con 0x7fbd700709d0 2023-12-20T04:43:51.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd76e04640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbd700c9c20 con 0x7fbd7015ace0 2023-12-20T04:43:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd75e02640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbd7015ace0 0x7fbd700b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd75e02640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbd7015ace0 0x7fbd700b1600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:35314/0 (socket says 172.21.15.32:35314) 2023-12-20T04:43:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd75e02640 1 -- 172.21.15.32:0/1815622700 learned_addr learned my addr 172.21.15.32:0/1815622700 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd75601640 1 --2- 172.21.15.32:0/1815622700 >> v2:172.21.15.32:3300/0 conn(0x7fbd700709d0 0x7fbd700afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd75e02640 1 -- 172.21.15.32:0/1815622700 >> v2:172.21.15.32:3300/0 conn(0x7fbd700709d0 msgr2=0x7fbd700afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd75e02640 1 --2- 172.21.15.32:0/1815622700 >> v2:172.21.15.32:3300/0 conn(0x7fbd700709d0 0x7fbd700afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd75e02640 1 -- 172.21.15.32:0/1815622700 >> v1:172.21.15.39:6789/0 conn(0x7fbd70070e30 legacy=0x7fbd7015e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd75e02640 1 -- 172.21.15.32:0/1815622700 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd700c9f00 con 0x7fbd7015ace0 2023-12-20T04:43:51.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.294+0000 7fbd74e00640 1 -- 172.21.15.32:0/1815622700 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1703834511 0 0) 0x7fbd70070570 con 0x7fbd70070e30 2023-12-20T04:43:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd75e02640 1 --2- 172.21.15.32:0/1815622700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7015ace0 0x7fbd700b1600 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbd700cc060 tx=0x7fbd6c01c4d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=327e3f9fb30db9b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd74e00640 1 -- 172.21.15.32:0/1815622700 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0ac640 con 0x7fbd7015ace0 2023-12-20T04:43:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd74e00640 1 -- 172.21.15.32:0/1815622700 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c0ac7a0 con 0x7fbd7015ace0 2023-12-20T04:43:51.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd74e00640 1 -- 172.21.15.32:0/1815622700 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0acb30 con 0x7fbd7015ace0 2023-12-20T04:43:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd76e04640 1 -- 172.21.15.32:0/1815622700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7015ace0 msgr2=0x7fbd700b1600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd76e04640 1 --2- 172.21.15.32:0/1815622700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7015ace0 0x7fbd700b1600 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7fbd700cc060 tx=0x7fbd6c01c4d0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd76e04640 1 -- 172.21.15.32:0/1815622700 shutdown_connections 2023-12-20T04:43:51.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd76e04640 1 --2- 172.21.15.32:0/1815622700 >> v2:172.21.15.32:3300/0 conn(0x7fbd700709d0 0x7fbd700afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd76e04640 1 --2- 172.21.15.32:0/1815622700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7015ace0 0x7fbd700b1600 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd76e04640 1 -- 172.21.15.32:0/1815622700 >> 172.21.15.32:0/1815622700 conn(0x7fbd7006c060 msgr2=0x7fbd7006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd76e04640 1 -- 172.21.15.32:0/1815622700 shutdown_connections 2023-12-20T04:43:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd76e04640 1 -- 172.21.15.32:0/1815622700 wait complete. 2023-12-20T04:43:51.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd76e04640 1 Processor -- start 2023-12-20T04:43:51.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.298+0000 7fbd76e04640 1 -- start start 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd76e04640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd700709d0 0x7fbd701553b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd76e04640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd70070e30 0x7fbd700b3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd76e04640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7015ace0 0x7fbd70147760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd76e04640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd70154f90 con 0x7fbd700709d0 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd76e04640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbd701561d0 con 0x7fbd7015ace0 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd76e04640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbd701564b0 con 0x7fbd70070e30 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd75e02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd700709d0 0x7fbd701553b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd75e02640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd700709d0 0x7fbd701553b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46294/0 (socket says 172.21.15.32:46294) 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd75e02640 1 -- 172.21.15.32:0/1946602901 learned_addr learned my addr 172.21.15.32:0/1946602901 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd75601640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd70070e30 0x7fbd700b3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd76603640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7015ace0 0x7fbd70147760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd75e02640 1 -- 172.21.15.32:0/1946602901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd70070e30 msgr2=0x7fbd700b3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd75e02640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd70070e30 0x7fbd700b3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd75e02640 1 -- 172.21.15.32:0/1946602901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7015ace0 msgr2=0x7fbd70147760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd75e02640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7015ace0 0x7fbd70147760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd75e02640 1 -- 172.21.15.32:0/1946602901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbd6c0094d0 con 0x7fbd700709d0 2023-12-20T04:43:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd75e02640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd700709d0 0x7fbd701553b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0190e0 tx=0x7fbd6c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:51.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd667fc640 1 -- 172.21.15.32:0/1946602901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0b7620 con 0x7fbd700709d0 2023-12-20T04:43:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd667fc640 1 -- 172.21.15.32:0/1946602901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbd6c0b7780 con 0x7fbd700709d0 2023-12-20T04:43:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd667fc640 1 -- 172.21.15.32:0/1946602901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbd6c0b7af0 con 0x7fbd700709d0 2023-12-20T04:43:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd76e04640 1 -- 172.21.15.32:0/1946602901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbd70154bb0 con 0x7fbd700709d0 2023-12-20T04:43:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd76e04640 1 -- 172.21.15.32:0/1946602901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbd70147ca0 con 0x7fbd700709d0 2023-12-20T04:43:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.302+0000 7fbd76e04640 1 -- 172.21.15.32:0/1946602901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbd70154a10 con 0x7fbd700709d0 2023-12-20T04:43:51.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.306+0000 7fbd667fc640 1 -- 172.21.15.32:0/1946602901 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fbd6c0b3040 con 0x7fbd700709d0 2023-12-20T04:43:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.306+0000 7fbd667fc640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd54065af0 0x7fbd54067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.306+0000 7fbd667fc640 1 -- 172.21.15.32:0/1946602901 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fbd6c11d590 con 0x7fbd700709d0 2023-12-20T04:43:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.306+0000 7fbd75601640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd54065af0 0x7fbd54067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.306+0000 7fbd75601640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd54065af0 0x7fbd54067fb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd70150f00 tx=0x7fbd60009190 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:51.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.310+0000 7fbd667fc640 1 -- 172.21.15.32:0/1946602901 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbd70154a10 con 0x7fbd700709d0 2023-12-20T04:43:51.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.474+0000 7fbd76e04640 1 -- 172.21.15.32:0/1946602901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbd70154a10 con 0x7fbd700709d0 2023-12-20T04:43:51.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.474+0000 7fbd667fc640 1 -- 172.21.15.32:0/1946602901 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) v1 ==== 74+0+12341 (secure 0 0 0) 0x7fbd70154a10 con 0x7fbd700709d0 2023-12-20T04:43:51.473 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:51.473 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:50.709262+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":9,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":77,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:51.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 -- 172.21.15.32:0/1946602901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd54065af0 msgr2=0x7fbd54067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd54065af0 0x7fbd54067fb0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbd70150f00 tx=0x7fbd60009190 comp rx=0 tx=0).stop 2023-12-20T04:43:51.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 -- 172.21.15.32:0/1946602901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd700709d0 msgr2=0x7fbd701553b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd700709d0 0x7fbd701553b0 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fbd6c0190e0 tx=0x7fbd6c01ad10 comp rx=0 tx=0).stop 2023-12-20T04:43:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 -- 172.21.15.32:0/1946602901 shutdown_connections 2023-12-20T04:43:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbd54065af0 0x7fbd54067fb0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbd7015ace0 0x7fbd70147760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbd70070e30 0x7fbd700b3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 --2- 172.21.15.32:0/1946602901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbd700709d0 0x7fbd701553b0 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 -- 172.21.15.32:0/1946602901 >> 172.21.15.32:0/1946602901 conn(0x7fbd7006c060 msgr2=0x7fbd70157310 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 -- 172.21.15.32:0/1946602901 shutdown_connections 2023-12-20T04:43:51.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.478+0000 7fbd76e04640 1 -- 172.21.15.32:0/1946602901 wait complete. 2023-12-20T04:43:51.495 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:43:51.496 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:43:51.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.646+0000 7f9b08bcb640 1 Processor -- start 2023-12-20T04:43:51.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.646+0000 7f9b08bcb640 1 -- start start 2023-12-20T04:43:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.646+0000 7f9b08bcb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9b040cf3e0 0x7f9b040cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.646+0000 7f9b08bcb640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9b040d1580 0x7f9b040d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.646+0000 7f9b08bcb640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9b040cfd80 con 0x7f9b0415ccf0 2023-12-20T04:43:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.646+0000 7f9b08bcb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9b040ce2f0 con 0x7f9b040cf3e0 2023-12-20T04:43:51.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.646+0000 7f9b08bcb640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b040d0b70 con 0x7f9b040d1580 2023-12-20T04:43:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b037fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9b0415ccf0 0x7f9b040cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34864/0 (socket says 172.21.15.32:34864) 2023-12-20T04:43:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b037fe640 1 -- 172.21.15.32:0/3966826154 learned_addr learned my addr 172.21.15.32:0/3966826154 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b03fff640 1 --2- 172.21.15.32:0/3966826154 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9b040d1580 0x7f9b040d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b02ffd640 1 --2- 172.21.15.32:0/3966826154 >> v2:172.21.15.39:3300/0 conn(0x7f9b040cf3e0 0x7f9b040cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b027fc640 1 -- 172.21.15.32:0/3966826154 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3720146995 0 0) 0x7f9b040cfd80 con 0x7f9b0415ccf0 2023-12-20T04:43:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b027fc640 1 -- 172.21.15.32:0/3966826154 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aec003530 con 0x7f9b0415ccf0 2023-12-20T04:43:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b02ffd640 1 -- 172.21.15.32:0/3966826154 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9b040d1580 msgr2=0x7f9b040d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b02ffd640 1 --2- 172.21.15.32:0/3966826154 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9b040d1580 0x7f9b040d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b02ffd640 1 -- 172.21.15.32:0/3966826154 >> v1:172.21.15.32:6789/0 conn(0x7f9b0415ccf0 legacy=0x7f9b040cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b02ffd640 1 -- 172.21.15.32:0/3966826154 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9b040d10f0 con 0x7f9b040cf3e0 2023-12-20T04:43:51.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b03fff640 1 --2- 172.21.15.32:0/3966826154 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9b040d1580 0x7f9b040d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:51.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b02ffd640 1 --2- 172.21.15.32:0/3966826154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b040cf3e0 0x7f9b040cf840 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9af80097c0 tx=0x7f9af801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=423845f57011e257 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:51.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b027fc640 1 -- 172.21.15.32:0/3966826154 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80ac620 con 0x7f9b040cf3e0 2023-12-20T04:43:51.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b027fc640 1 -- 172.21.15.32:0/3966826154 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af80ac780 con 0x7f9b040cf3e0 2023-12-20T04:43:51.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.650+0000 7f9b027fc640 1 -- 172.21.15.32:0/3966826154 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80acb10 con 0x7f9b040cf3e0 2023-12-20T04:43:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 -- 172.21.15.32:0/3966826154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b040cf3e0 msgr2=0x7f9b040cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 --2- 172.21.15.32:0/3966826154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b040cf3e0 0x7f9b040cf840 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f9af80097c0 tx=0x7f9af801c3c0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 -- 172.21.15.32:0/3966826154 shutdown_connections 2023-12-20T04:43:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 --2- 172.21.15.32:0/3966826154 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b040cf3e0 0x7f9b040cf840 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 --2- 172.21.15.32:0/3966826154 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9b040d1580 0x7f9b040d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 -- 172.21.15.32:0/3966826154 >> 172.21.15.32:0/3966826154 conn(0x7f9b040658e0 msgr2=0x7f9b04065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 -- 172.21.15.32:0/3966826154 shutdown_connections 2023-12-20T04:43:51.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 -- 172.21.15.32:0/3966826154 wait complete. 2023-12-20T04:43:51.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 Processor -- start 2023-12-20T04:43:51.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 -- start start 2023-12-20T04:43:51.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b040cf3e0 0x7f9b040c9d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b040d1580 0x7f9b041480d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9b0415ccf0 0x7f9b040c6e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0415d3b0 con 0x7f9b040d1580 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0415d690 con 0x7f9b040cf3e0 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b08bcb640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9b0415d970 con 0x7f9b0415ccf0 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b040d1580 0x7f9b041480d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b02ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b040d1580 0x7f9b041480d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46302/0 (socket says 172.21.15.32:46302) 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b02ffd640 1 -- 172.21.15.32:0/204374606 learned_addr learned my addr 172.21.15.32:0/204374606 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.654+0000 7f9b037fe640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b040cf3e0 0x7f9b040c9d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9b03fff640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9b0415ccf0 0x7f9b040c6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9b02ffd640 1 -- 172.21.15.32:0/204374606 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9b0415ccf0 msgr2=0x7f9b040c6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9b02ffd640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9b0415ccf0 0x7f9b040c6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9b02ffd640 1 -- 172.21.15.32:0/204374606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b040cf3e0 msgr2=0x7f9b040c9d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9b02ffd640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b040cf3e0 0x7f9b040c9d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9b02ffd640 1 -- 172.21.15.32:0/204374606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9afc009a70 con 0x7f9b040d1580 2023-12-20T04:43:51.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9b02ffd640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b040d1580 0x7f9b041480d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9af80af040 tx=0x7f9af8019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:51.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9aebfff640 1 -- 172.21.15.32:0/204374606 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80ac620 con 0x7f9b040d1580 2023-12-20T04:43:51.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9aebfff640 1 -- 172.21.15.32:0/204374606 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9af80b5410 con 0x7f9b040d1580 2023-12-20T04:43:51.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9b08bcb640 1 -- 172.21.15.32:0/204374606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9af8009420 con 0x7f9b040d1580 2023-12-20T04:43:51.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9aebfff640 1 -- 172.21.15.32:0/204374606 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9af80b57e0 con 0x7f9b040d1580 2023-12-20T04:43:51.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9b08bcb640 1 -- 172.21.15.32:0/204374606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9b0415ef80 con 0x7f9b040d1580 2023-12-20T04:43:51.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9b08bcb640 1 -- 172.21.15.32:0/204374606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9b04074100 con 0x7f9b040d1580 2023-12-20T04:43:51.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9aebfff640 1 -- 172.21.15.32:0/204374606 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f9af80b3040 con 0x7f9b040d1580 2023-12-20T04:43:51.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9aebfff640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9af0065af0 0x7f9af0067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.658+0000 7f9aebfff640 1 -- 172.21.15.32:0/204374606 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f9b040d2760 con 0x7f9b040d1580 2023-12-20T04:43:51.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.662+0000 7f9b037fe640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9af0065af0 0x7f9af0067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.662+0000 7f9b037fe640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9af0065af0 0x7f9af0067fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9af40011f0 tx=0x7f9af4007d40 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:51.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.662+0000 7f9aebfff640 1 -- 172.21.15.32:0/204374606 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9af80eab20 con 0x7f9b040d1580 2023-12-20T04:43:51.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.822+0000 7f9b08bcb640 1 -- 172.21.15.32:0/204374606 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f9b0415d4f0 con 0x7f9b040d1580 2023-12-20T04:43:51.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.822+0000 7f9aebfff640 1 -- 172.21.15.32:0/204374606 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) v1 ==== 93+0+10 (secure 0 0 0) 0x7f9b0415d4f0 con 0x7f9b040d1580 2023-12-20T04:43:51.822 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:43:51.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 -- 172.21.15.32:0/204374606 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9af0065af0 msgr2=0x7f9af0067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9af0065af0 0x7f9af0067fb0 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f9af40011f0 tx=0x7f9af4007d40 comp rx=0 tx=0).stop 2023-12-20T04:43:51.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 -- 172.21.15.32:0/204374606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b040d1580 msgr2=0x7f9b041480d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b040d1580 0x7f9b041480d0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f9af80af040 tx=0x7f9af8019160 comp rx=0 tx=0).stop 2023-12-20T04:43:51.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 -- 172.21.15.32:0/204374606 shutdown_connections 2023-12-20T04:43:51.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9af0065af0 0x7f9af0067fb0 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9b0415ccf0 0x7f9b040c6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9b040d1580 0x7f9b041480d0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 --2- 172.21.15.32:0/204374606 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9b040cf3e0 0x7f9b040c9d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 -- 172.21.15.32:0/204374606 >> 172.21.15.32:0/204374606 conn(0x7f9b040658e0 msgr2=0x7f9b040caf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:51.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 -- 172.21.15.32:0/204374606 shutdown_connections 2023-12-20T04:43:51.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.826+0000 7f9b08bcb640 1 -- 172.21.15.32:0/204374606 wait complete. 2023-12-20T04:43:51.844 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T04:43:51.946 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:51.946+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:51.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f2491325640 1 Processor -- start 2023-12-20T04:43:51.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f2491325640 1 -- start start 2023-12-20T04:43:51.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f2491325640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c1599e0 0x7f248c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f2491325640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f248c0c90e0 0x7f248c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:51.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f2491325640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f248c0711d0 con 0x7f248c0b0b30 2023-12-20T04:43:51.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f2491325640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f248c0c98c0 con 0x7f248c0c90e0 2023-12-20T04:43:51.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f2491325640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0c9ba0 con 0x7f248c1599e0 2023-12-20T04:43:51.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f2483fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f248c0c90e0 0x7f248c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f248affd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c1599e0 0x7f248c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:51.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f2483fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f248c0c90e0 0x7f248c0b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:35348/0 (socket says 172.21.15.32:35348) 2023-12-20T04:43:51.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f248affd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c1599e0 0x7f248c0aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:39578/0 (socket says 172.21.15.32:39578) 2023-12-20T04:43:51.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f2483fff640 1 -- 172.21.15.32:0/558412024 learned_addr learned my addr 172.21.15.32:0/558412024 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:51.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f248affd640 1 -- 172.21.15.32:0/558412024 >> v2:172.21.15.39:3300/0 conn(0x7f248c0c90e0 msgr2=0x7f248c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:51.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f248affd640 1 --2- 172.21.15.32:0/558412024 >> v2:172.21.15.39:3300/0 conn(0x7f248c0c90e0 0x7f248c0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f248affd640 1 -- 172.21.15.32:0/558412024 >> v1:172.21.15.39:6789/0 conn(0x7f248c0b0b30 legacy=0x7f248c15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f248affd640 1 -- 172.21.15.32:0/558412024 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f248c0c9e80 con 0x7f248c1599e0 2023-12-20T04:43:51.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.994+0000 7f2483fff640 1 --2- 172.21.15.32:0/558412024 >> v2:172.21.15.39:3300/0 conn(0x7f248c0c90e0 0x7f248c0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:51.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f248affd640 1 --2- 172.21.15.32:0/558412024 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c1599e0 0x7f248c0aece0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f247c0097a0 tx=0x7f247c01e640 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd567fbebe631dbd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:51.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f248a7fc640 1 -- 172.21.15.32:0/558412024 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c0ab630 con 0x7f248c1599e0 2023-12-20T04:43:51.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f248a7fc640 1 -- 172.21.15.32:0/558412024 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247c0ab790 con 0x7f248c1599e0 2023-12-20T04:43:51.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f248a7fc640 1 -- 172.21.15.32:0/558412024 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247c0abb20 con 0x7f248c1599e0 2023-12-20T04:43:51.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f2491325640 1 -- 172.21.15.32:0/558412024 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c1599e0 msgr2=0x7f248c0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:51.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f2491325640 1 --2- 172.21.15.32:0/558412024 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c1599e0 0x7f248c0aece0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f247c0097a0 tx=0x7f247c01e640 comp rx=0 tx=0).stop 2023-12-20T04:43:51.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f2491325640 1 -- 172.21.15.32:0/558412024 shutdown_connections 2023-12-20T04:43:51.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f2491325640 1 --2- 172.21.15.32:0/558412024 >> v2:172.21.15.39:3300/0 conn(0x7f248c0c90e0 0x7f248c0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:51.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f2491325640 1 --2- 172.21.15.32:0/558412024 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c1599e0 0x7f248c0aece0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f2491325640 1 -- 172.21.15.32:0/558412024 >> 172.21.15.32:0/558412024 conn(0x7f248c06a320 msgr2=0x7f248c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:52.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f2491325640 1 -- 172.21.15.32:0/558412024 shutdown_connections 2023-12-20T04:43:52.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:51.998+0000 7f2491325640 1 -- 172.21.15.32:0/558412024 wait complete. 2023-12-20T04:43:52.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2491325640 1 Processor -- start 2023-12-20T04:43:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2491325640 1 -- start start 2023-12-20T04:43:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2491325640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c0b0b30 0x7f248c15bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2491325640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f248c0c90e0 0x7f248c0c6360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2491325640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c1599e0 0x7f248c14e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:52.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2491325640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0c7830 con 0x7f248c0c90e0 2023-12-20T04:43:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2491325640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0c6bb0 con 0x7f248c1599e0 2023-12-20T04:43:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2491325640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f248c0c6e90 con 0x7f248c0b0b30 2023-12-20T04:43:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2483fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f248c0c90e0 0x7f248c0c6360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2483fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f248c0c90e0 0x7f248c0c6360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46308/0 (socket says 172.21.15.32:46308) 2023-12-20T04:43:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2483fff640 1 -- 172.21.15.32:0/3867381796 learned_addr learned my addr 172.21.15.32:0/3867381796 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f248b7fe640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c1599e0 0x7f248c14e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:52.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f248affd640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c0b0b30 0x7f248c15bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2483fff640 1 -- 172.21.15.32:0/3867381796 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c0b0b30 msgr2=0x7f248c15bd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2483fff640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c0b0b30 0x7f248c15bd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2483fff640 1 -- 172.21.15.32:0/3867381796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c1599e0 msgr2=0x7f248c14e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2483fff640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c1599e0 0x7f248c14e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2483fff640 1 -- 172.21.15.32:0/3867381796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24740094d0 con 0x7f248c0c90e0 2023-12-20T04:43:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f248b7fe640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c1599e0 0x7f248c14e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2483fff640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f248c0c90e0 0x7f248c0c6360 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f247400e9d0 tx=0x7f247400ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:52.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f24837fe640 1 -- 172.21.15.32:0/3867381796 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f247400ca60 con 0x7f248c0c90e0 2023-12-20T04:43:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f24837fe640 1 -- 172.21.15.32:0/3867381796 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f247400cbc0 con 0x7f248c0c90e0 2023-12-20T04:43:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2491325640 1 -- 172.21.15.32:0/3867381796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f247c0093b0 con 0x7f248c0c90e0 2023-12-20T04:43:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f24837fe640 1 -- 172.21.15.32:0/3867381796 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2474013470 con 0x7f248c0c90e0 2023-12-20T04:43:52.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.002+0000 7f2491325640 1 -- 172.21.15.32:0/3867381796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f248c0c7450 con 0x7f248c0c90e0 2023-12-20T04:43:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.006+0000 7f2491325640 1 -- 172.21.15.32:0/3867381796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f248c143660 con 0x7f248c0c90e0 2023-12-20T04:43:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.006+0000 7f24837fe640 1 -- 172.21.15.32:0/3867381796 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f24740135d0 con 0x7f248c0c90e0 2023-12-20T04:43:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.006+0000 7f24837fe640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f246c065af0 0x7f246c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:52.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.006+0000 7f24837fe640 1 -- 172.21.15.32:0/3867381796 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2474083110 con 0x7f248c0c90e0 2023-12-20T04:43:52.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.006+0000 7f248affd640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f246c065af0 0x7f246c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:52.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.006+0000 7f248affd640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f246c065af0 0x7f246c067fb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f247c0b0000 tx=0x7f247c0b5040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:52.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.010+0000 7f24837fe640 1 -- 172.21.15.32:0/3867381796 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f248c143660 con 0x7f248c0c90e0 2023-12-20T04:43:52.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 -- 172.21.15.32:0/3867381796 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f248c143660 con 0x7f248c0c90e0 2023-12-20T04:43:52.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f24837fe640 1 -- 172.21.15.32:0/3867381796 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v89) v1 ==== 84+0+65 (secure 0 0 0) 0x7f248c143660 con 0x7f248c0c90e0 2023-12-20T04:43:52.176 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:52.176 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:43:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:43:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T04:43:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T04:43:52.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:43:52.177 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:43:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 -- 172.21.15.32:0/3867381796 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f246c065af0 msgr2=0x7f246c067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f246c065af0 0x7f246c067fb0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f247c0b0000 tx=0x7f247c0b5040 comp rx=0 tx=0).stop 2023-12-20T04:43:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 -- 172.21.15.32:0/3867381796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f248c0c90e0 msgr2=0x7f248c0c6360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f248c0c90e0 0x7f248c0c6360 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7f247400e9d0 tx=0x7f247400ee90 comp rx=0 tx=0).stop 2023-12-20T04:43:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 -- 172.21.15.32:0/3867381796 shutdown_connections 2023-12-20T04:43:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f246c065af0 0x7f246c067fb0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f248c1599e0 0x7f248c14e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f248c0c90e0 0x7f248c0c6360 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 --2- 172.21.15.32:0/3867381796 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f248c0b0b30 0x7f248c15bd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 -- 172.21.15.32:0/3867381796 >> 172.21.15.32:0/3867381796 conn(0x7f248c06a320 msgr2=0x7f248c14fae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 -- 172.21.15.32:0/3867381796 shutdown_connections 2023-12-20T04:43:52.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.178+0000 7f2491325640 1 -- 172.21.15.32:0/3867381796 wait complete. 2023-12-20T04:43:52.198 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T04:43:52.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T04:43:52.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2eeb544640 1 Processor -- start 2023-12-20T04:43:52.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2eeb544640 1 -- start start 2023-12-20T04:43:52.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2eeb544640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ee415a810 0x7f2ee40b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:52.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2eeb544640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ee40701b0 0x7f2ee40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:52.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2eeb544640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2ee40caf10 0x7f2ee40cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:52.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2eeb544640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee4070b30 con 0x7f2ee40701b0 2023-12-20T04:43:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2eeb544640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ee40ca220 con 0x7f2ee40caf10 2023-12-20T04:43:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2eeb544640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40ca750 con 0x7f2ee415a810 2023-12-20T04:43:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2ee9d41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ee40701b0 0x7f2ee40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2ee9d41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ee40701b0 0x7f2ee40705f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46310/0 (socket says 172.21.15.32:46310) 2023-12-20T04:43:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2ee9d41640 1 -- 172.21.15.32:0/1110045157 learned_addr learned my addr 172.21.15.32:0/1110045157 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:52.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2eea542640 1 --2- 172.21.15.32:0/1110045157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ee415a810 0x7f2ee40b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:52.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2eead43640 1 --2- 172.21.15.32:0/1110045157 >> v2:172.21.15.39:3300/0 conn(0x7f2ee40caf10 0x7f2ee40cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:52.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2ee9d41640 1 -- 172.21.15.32:0/1110045157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ee415a810 msgr2=0x7f2ee40b13b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:52.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2ee9d41640 1 --2- 172.21.15.32:0/1110045157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ee415a810 0x7f2ee40b13b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2ee9d41640 1 -- 172.21.15.32:0/1110045157 >> v2:172.21.15.39:3300/0 conn(0x7f2ee40caf10 msgr2=0x7f2ee40cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:52.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2ee9d41640 1 --2- 172.21.15.32:0/1110045157 >> v2:172.21.15.39:3300/0 conn(0x7f2ee40caf10 0x7f2ee40cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.350+0000 7f2ee9d41640 1 -- 172.21.15.32:0/1110045157 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ee40cb8e0 con 0x7f2ee40701b0 2023-12-20T04:43:52.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2ee9d41640 1 --2- 172.21.15.32:0/1110045157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40701b0 0x7f2ee40705f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4071eb0 tx=0x7f2ed401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104133c89058e3c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:52.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2ee9540640 1 -- 172.21.15.32:0/1110045157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40ac440 con 0x7f2ee40701b0 2023-12-20T04:43:52.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2ee9540640 1 -- 172.21.15.32:0/1110045157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ed40ac5a0 con 0x7f2ee40701b0 2023-12-20T04:43:52.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2ee9540640 1 -- 172.21.15.32:0/1110045157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ed40ac930 con 0x7f2ee40701b0 2023-12-20T04:43:52.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2eeb544640 1 -- 172.21.15.32:0/1110045157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40701b0 msgr2=0x7f2ee40705f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:52.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2eeb544640 1 --2- 172.21.15.32:0/1110045157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40701b0 0x7f2ee40705f0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7f2ee4071eb0 tx=0x7f2ed401c470 comp rx=0 tx=0).stop 2023-12-20T04:43:52.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2eeb544640 1 -- 172.21.15.32:0/1110045157 shutdown_connections 2023-12-20T04:43:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2eeb544640 1 --2- 172.21.15.32:0/1110045157 >> v2:172.21.15.39:3300/0 conn(0x7f2ee40caf10 0x7f2ee40cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2eeb544640 1 --2- 172.21.15.32:0/1110045157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40701b0 0x7f2ee40705f0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2eeb544640 1 --2- 172.21.15.32:0/1110045157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ee415a810 0x7f2ee40b13b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2eeb544640 1 -- 172.21.15.32:0/1110045157 >> 172.21.15.32:0/1110045157 conn(0x7f2ee406a4e0 msgr2=0x7f2ee406bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2eeb544640 1 -- 172.21.15.32:0/1110045157 shutdown_connections 2023-12-20T04:43:52.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.354+0000 7f2eeb544640 1 -- 172.21.15.32:0/1110045157 wait complete. 2023-12-20T04:43:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eeb544640 1 Processor -- start 2023-12-20T04:43:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eeb544640 1 -- start start 2023-12-20T04:43:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eeb544640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40701b0 0x7f2ee40c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eeb544640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee40caf10 0x7f2ee40c7600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eeb544640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ee415a810 0x7f2ee415aef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:52.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eeb544640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40c8760 con 0x7f2ee40701b0 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eeb544640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee4155990 con 0x7f2ee40caf10 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eeb544640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ee40c7be0 con 0x7f2ee415a810 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eea542640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40701b0 0x7f2ee40c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eea542640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40701b0 0x7f2ee40c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46312/0 (socket says 172.21.15.32:46312) 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eea542640 1 -- 172.21.15.32:0/4139375102 learned_addr learned my addr 172.21.15.32:0/4139375102 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2ee9d41640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee40caf10 0x7f2ee40c7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eead43640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ee415a810 0x7f2ee415aef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eea542640 1 -- 172.21.15.32:0/4139375102 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ee415a810 msgr2=0x7f2ee415aef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eea542640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ee415a810 0x7f2ee415aef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eea542640 1 -- 172.21.15.32:0/4139375102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee40caf10 msgr2=0x7f2ee40c7600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eea542640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee40caf10 0x7f2ee40c7600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eea542640 1 -- 172.21.15.32:0/4139375102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ed40094d0 con 0x7f2ee40701b0 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eea542640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40701b0 0x7f2ee40c70c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000e870 tx=0x7f2ee000ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:52.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2edaffd640 1 -- 172.21.15.32:0/4139375102 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee0009510 con 0x7f2ee40701b0 2023-12-20T04:43:52.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2edaffd640 1 -- 172.21.15.32:0/4139375102 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ee000eec0 con 0x7f2ee40701b0 2023-12-20T04:43:52.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2eeb544640 1 -- 172.21.15.32:0/4139375102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ee40c7ec0 con 0x7f2ee40701b0 2023-12-20T04:43:52.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.358+0000 7f2edaffd640 1 -- 172.21.15.32:0/4139375102 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ee000cb40 con 0x7f2ee40701b0 2023-12-20T04:43:52.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.362+0000 7f2eeb544640 1 -- 172.21.15.32:0/4139375102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ee40c8480 con 0x7f2ee40701b0 2023-12-20T04:43:52.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.362+0000 7f2eeb544640 1 -- 172.21.15.32:0/4139375102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ee4074100 con 0x7f2ee40701b0 2023-12-20T04:43:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.362+0000 7f2edaffd640 1 -- 172.21.15.32:0/4139375102 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f2ee000cca0 con 0x7f2ee40701b0 2023-12-20T04:43:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.362+0000 7f2edaffd640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2ec8065af0 0x7f2ec8067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.362+0000 7f2edaffd640 1 -- 172.21.15.32:0/4139375102 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(89..89 src has 1..89) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f2ee0082e90 con 0x7f2ee40701b0 2023-12-20T04:43:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.362+0000 7f2ee9d41640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2ec8065af0 0x7f2ec8067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:52.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.362+0000 7f2ee9d41640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2ec8065af0 0x7f2ec8067fb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40ae000 tx=0x7f2ed4009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:52.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.366+0000 7f2edaffd640 1 -- 172.21.15.32:0/4139375102 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2ee0050bf0 con 0x7f2ee40701b0 2023-12-20T04:43:52.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:52.530+0000 7f2eeb544640 1 -- 172.21.15.32:0/4139375102 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f2ee40cd110 con 0x7f2ee40701b0 2023-12-20T04:43:52.653 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:52.654+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:53.372 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:53.374+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:53.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.454+0000 7f2edaffd640 1 -- 172.21.15.32:0/4139375102 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v90) v1 ==== 138+0+0 (secure 0 0 0) 0x7f2ee00170b0 con 0x7f2ee40701b0 2023-12-20T04:43:53.454 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T04:43:53.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 -- 172.21.15.32:0/4139375102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2ec8065af0 msgr2=0x7f2ec8067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:53.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2ec8065af0 0x7f2ec8067fb0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2ed40ae000 tx=0x7f2ed4009040 comp rx=0 tx=0).stop 2023-12-20T04:43:53.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 -- 172.21.15.32:0/4139375102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40701b0 msgr2=0x7f2ee40c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:53.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40701b0 0x7f2ee40c70c0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7f2ee000e870 tx=0x7f2ee000ec70 comp rx=0 tx=0).stop 2023-12-20T04:43:53.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 -- 172.21.15.32:0/4139375102 shutdown_connections 2023-12-20T04:43:53.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2ec8065af0 0x7f2ec8067fb0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:53.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ee415a810 0x7f2ee415aef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:53.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ee40caf10 0x7f2ee40c7600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:53.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 --2- 172.21.15.32:0/4139375102 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ee40701b0 0x7f2ee40c70c0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:53.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 -- 172.21.15.32:0/4139375102 >> 172.21.15.32:0/4139375102 conn(0x7f2ee406a4e0 msgr2=0x7f2ee40b24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:53.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 -- 172.21.15.32:0/4139375102 shutdown_connections 2023-12-20T04:43:53.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.458+0000 7f2eeb544640 1 -- 172.21.15.32:0/4139375102 wait complete. 2023-12-20T04:43:53.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T04:43:53.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc0c943640 1 Processor -- start 2023-12-20T04:43:53.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc0c943640 1 -- start start 2023-12-20T04:43:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc0c943640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc080b8410 0x7fdc0815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc0c943640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdc080cdd20 0x7fdc0815cb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc0c943640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc0815d050 con 0x7fdc080cd940 2023-12-20T04:43:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc0c943640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc080d0c10 con 0x7fdc080b8410 2023-12-20T04:43:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc0c943640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc08160f80 con 0x7fdc080cdd20 2023-12-20T04:43:53.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc06ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc080b8410 0x7fdc0815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc06ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc080b8410 0x7fdc0815c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46326/0 (socket says 172.21.15.32:46326) 2023-12-20T04:43:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc06ffd640 1 -- 172.21.15.32:0/4219265127 learned_addr learned my addr 172.21.15.32:0/4219265127 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc07fff640 1 --2- 172.21.15.32:0/4219265127 >> v2:172.21.15.39:3300/0 conn(0x7fdc080cdd20 0x7fdc0815cb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc06ffd640 1 -- 172.21.15.32:0/4219265127 >> v2:172.21.15.39:3300/0 conn(0x7fdc080cdd20 msgr2=0x7fdc0815cb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc06ffd640 1 --2- 172.21.15.32:0/4219265127 >> v2:172.21.15.39:3300/0 conn(0x7fdc080cdd20 0x7fdc0815cb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc06ffd640 1 -- 172.21.15.32:0/4219265127 >> v1:172.21.15.32:6789/0 conn(0x7fdc080cd940 legacy=0x7fdc081606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:53.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.638+0000 7fdc06ffd640 1 -- 172.21.15.32:0/4219265127 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc08161240 con 0x7fdc080b8410 2023-12-20T04:43:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc06ffd640 1 --2- 172.21.15.32:0/4219265127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc080b8410 0x7fdc0815c5d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdc080cef00 tx=0x7fdbfc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=173f8d143d909eed server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:53.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc067fc640 1 -- 172.21.15.32:0/4219265127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0aa650 con 0x7fdc080b8410 2023-12-20T04:43:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc067fc640 1 -- 172.21.15.32:0/4219265127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbfc0aa7b0 con 0x7fdc080b8410 2023-12-20T04:43:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc067fc640 1 -- 172.21.15.32:0/4219265127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbfc0aab40 con 0x7fdc080b8410 2023-12-20T04:43:53.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc0c943640 1 -- 172.21.15.32:0/4219265127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc080b8410 msgr2=0x7fdc0815c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc0c943640 1 --2- 172.21.15.32:0/4219265127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc080b8410 0x7fdc0815c5d0 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7fdc080cef00 tx=0x7fdbfc01c4d0 comp rx=0 tx=0).stop 2023-12-20T04:43:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc0c943640 1 -- 172.21.15.32:0/4219265127 shutdown_connections 2023-12-20T04:43:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc0c943640 1 --2- 172.21.15.32:0/4219265127 >> v2:172.21.15.39:3300/0 conn(0x7fdc080cdd20 0x7fdc0815cb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc0c943640 1 --2- 172.21.15.32:0/4219265127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc080b8410 0x7fdc0815c5d0 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc0c943640 1 -- 172.21.15.32:0/4219265127 >> 172.21.15.32:0/4219265127 conn(0x7fdc080658e0 msgr2=0x7fdc08065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc0c943640 1 -- 172.21.15.32:0/4219265127 shutdown_connections 2023-12-20T04:43:53.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.642+0000 7fdc0c943640 1 -- 172.21.15.32:0/4219265127 wait complete. 2023-12-20T04:43:53.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc0c943640 1 Processor -- start 2023-12-20T04:43:53.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc0c943640 1 -- start start 2023-12-20T04:43:53.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc0c943640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc080b8410 0x7fdc0815d820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc0c943640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc080cd940 0x7fdc08158060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc0c943640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc080cdd20 0x7fdc08153060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc0c943640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0815e0f0 con 0x7fdc080b8410 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc0c943640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0815e3d0 con 0x7fdc080cd940 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc0c943640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc0815e6b0 con 0x7fdc080cdd20 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc077fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc080b8410 0x7fdc0815d820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc077fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc080b8410 0x7fdc0815d820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46340/0 (socket says 172.21.15.32:46340) 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc077fe640 1 -- 172.21.15.32:0/862094960 learned_addr learned my addr 172.21.15.32:0/862094960 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc06ffd640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc080cd940 0x7fdc08158060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc07fff640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc080cdd20 0x7fdc08153060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc077fe640 1 -- 172.21.15.32:0/862094960 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc080cdd20 msgr2=0x7fdc08153060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:53.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc077fe640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc080cdd20 0x7fdc08153060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc077fe640 1 -- 172.21.15.32:0/862094960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc080cd940 msgr2=0x7fdc08158060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc077fe640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc080cd940 0x7fdc08158060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc077fe640 1 -- 172.21.15.32:0/862094960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdbfc0094d0 con 0x7fdc080b8410 2023-12-20T04:43:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc077fe640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc080b8410 0x7fdc0815d820 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800c500 tx=0x7fdbf800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdbeffff640 1 -- 172.21.15.32:0/862094960 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8005220 con 0x7fdc080b8410 2023-12-20T04:43:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdbeffff640 1 -- 172.21.15.32:0/862094960 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdbf8005380 con 0x7fdc080b8410 2023-12-20T04:43:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc0c943640 1 -- 172.21.15.32:0/862094960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc0815e990 con 0x7fdc080b8410 2023-12-20T04:43:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdbeffff640 1 -- 172.21.15.32:0/862094960 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdbf8007270 con 0x7fdc080b8410 2023-12-20T04:43:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.646+0000 7fdc0c943640 1 -- 172.21.15.32:0/862094960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc0815dd60 con 0x7fdc080b8410 2023-12-20T04:43:53.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.650+0000 7fdc0c943640 1 -- 172.21.15.32:0/862094960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc08074100 con 0x7fdc080b8410 2023-12-20T04:43:53.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.650+0000 7fdbeffff640 1 -- 172.21.15.32:0/862094960 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fdbf80073d0 con 0x7fdc080b8410 2023-12-20T04:43:53.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.650+0000 7fdbeffff640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbf4065af0 0x7fdbf4067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:53.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.650+0000 7fdbeffff640 1 -- 172.21.15.32:0/862094960 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(90..90 src has 1..90) v4 ==== 5227+0+0 (secure 0 0 0) 0x7fdbf80835f0 con 0x7fdc080b8410 2023-12-20T04:43:53.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.654+0000 7fdc06ffd640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbf4065af0 0x7fdbf4067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:53.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.654+0000 7fdc06ffd640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbf4065af0 0x7fdbf4067fb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0ae000 tx=0x7fdbfc0b5040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:53.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.654+0000 7fdbeffff640 1 -- 172.21.15.32:0/862094960 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdbf8051260 con 0x7fdc080b8410 2023-12-20T04:43:53.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:53.818+0000 7fdc0c943640 1 -- 172.21.15.32:0/862094960 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fdc08150480 con 0x7fdc080b8410 2023-12-20T04:43:54.098 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:54.098+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.470+0000 7fdbeffff640 1 -- 172.21.15.32:0/862094960 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v91) v1 ==== 170+0+0 (secure 0 0 0) 0x7fdbf8054880 con 0x7fdc080b8410 2023-12-20T04:43:54.468 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T04:43:54.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.474+0000 7fdc0c943640 1 -- 172.21.15.32:0/862094960 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbf4065af0 msgr2=0x7fdbf4067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.474+0000 7fdc0c943640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbf4065af0 0x7fdbf4067fb0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0ae000 tx=0x7fdbfc0b5040 comp rx=0 tx=0).stop 2023-12-20T04:43:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.474+0000 7fdc0c943640 1 -- 172.21.15.32:0/862094960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc080b8410 msgr2=0x7fdc0815d820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.474+0000 7fdc0c943640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc080b8410 0x7fdc0815d820 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800c500 tx=0x7fdbf800c9c0 comp rx=0 tx=0).stop 2023-12-20T04:43:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.474+0000 7fdc0c943640 1 -- 172.21.15.32:0/862094960 shutdown_connections 2023-12-20T04:43:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.474+0000 7fdc0c943640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdbf4065af0 0x7fdbf4067fb0 secure :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fdbfc0ae000 tx=0x7fdbfc0b5040 comp rx=0 tx=0).stop 2023-12-20T04:43:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.474+0000 7fdc0c943640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc080cdd20 0x7fdc08153060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.474+0000 7fdc0c943640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc080cd940 0x7fdc08158060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.474+0000 7fdc0c943640 1 --2- 172.21.15.32:0/862094960 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc080b8410 0x7fdc0815d820 secure :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7fdbf800c500 tx=0x7fdbf800c9c0 comp rx=0 tx=0).stop 2023-12-20T04:43:54.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.474+0000 7fdc0c943640 1 -- 172.21.15.32:0/862094960 >> 172.21.15.32:0/862094960 conn(0x7fdc080658e0 msgr2=0x7fdc0815b820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.478+0000 7fdc0c943640 1 -- 172.21.15.32:0/862094960 shutdown_connections 2023-12-20T04:43:54.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.478+0000 7fdc0c943640 1 -- 172.21.15.32:0/862094960 wait complete. 2023-12-20T04:43:54.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T04:43:54.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.658+0000 7fe961f6d640 1 Processor -- start 2023-12-20T04:43:54.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.658+0000 7fe961f6d640 1 -- start start 2023-12-20T04:43:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.658+0000 7fe961f6d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe95c065d40 0x7fe95c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.658+0000 7fe961f6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c06a1f0 con 0x7fe95c0cb050 2023-12-20T04:43:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.658+0000 7fe961f6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe95c06a440 con 0x7fe95c06d170 2023-12-20T04:43:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.658+0000 7fe961f6d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe95c06a720 con 0x7fe95c065d40 2023-12-20T04:43:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.662+0000 7fe95affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe95c065d40 0x7fe95c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.662+0000 7fe95bfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe95c06d170 0x7fe95c0c9720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:58712/0 (socket says 172.21.15.32:58712) 2023-12-20T04:43:54.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.662+0000 7fe95bfff640 1 -- 172.21.15.32:0/2541250085 learned_addr learned my addr 172.21.15.32:0/2541250085 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.662+0000 7fe95a7fc640 1 -- 172.21.15.32:0/2541250085 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2694896434 0 0) 0x7fe95c06a440 con 0x7fe95c06d170 2023-12-20T04:43:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.662+0000 7fe95a7fc640 1 -- 172.21.15.32:0/2541250085 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe944003530 con 0x7fe95c06d170 2023-12-20T04:43:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.662+0000 7fe95affd640 1 -- 172.21.15.32:0/2541250085 >> v1:172.21.15.39:6789/0 conn(0x7fe95c06d170 legacy=0x7fe95c0c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.662+0000 7fe95affd640 1 -- 172.21.15.32:0/2541250085 >> v1:172.21.15.32:6789/0 conn(0x7fe95c0cb050 legacy=0x7fe95c06ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:54.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.662+0000 7fe95affd640 1 -- 172.21.15.32:0/2541250085 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe95c06aa00 con 0x7fe95c065d40 2023-12-20T04:43:54.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.662+0000 7fe95affd640 1 --2- 172.21.15.32:0/2541250085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95c065d40 0x7fe95c0cb430 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe9500097c0 tx=0x7fe95001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2cd12119f88f227 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe95a7fc640 1 -- 172.21.15.32:0/2541250085 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9500ab650 con 0x7fe95c065d40 2023-12-20T04:43:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe95a7fc640 1 -- 172.21.15.32:0/2541250085 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9500ab7b0 con 0x7fe95c065d40 2023-12-20T04:43:54.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe95a7fc640 1 -- 172.21.15.32:0/2541250085 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9500abb40 con 0x7fe95c065d40 2023-12-20T04:43:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 -- 172.21.15.32:0/2541250085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95c065d40 msgr2=0x7fe95c0cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 --2- 172.21.15.32:0/2541250085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95c065d40 0x7fe95c0cb430 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7fe9500097c0 tx=0x7fe95001c450 comp rx=0 tx=0).stop 2023-12-20T04:43:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 -- 172.21.15.32:0/2541250085 shutdown_connections 2023-12-20T04:43:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 --2- 172.21.15.32:0/2541250085 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95c065d40 0x7fe95c0cb430 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 -- 172.21.15.32:0/2541250085 >> 172.21.15.32:0/2541250085 conn(0x7fe95c0cf8d0 msgr2=0x7fe95c0cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 -- 172.21.15.32:0/2541250085 shutdown_connections 2023-12-20T04:43:54.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 -- 172.21.15.32:0/2541250085 wait complete. 2023-12-20T04:43:54.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 Processor -- start 2023-12-20T04:43:54.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 -- start start 2023-12-20T04:43:54.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c065d40 0x7fe95c0c7670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:54.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe95c06d170 0x7fe95c14b5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:54.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95c0cb050 0x7fe95c153840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:54.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c71e0 con 0x7fe95c0cb050 2023-12-20T04:43:54.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c14bae0 con 0x7fe95c065d40 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe961f6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe95c0c6bd0 con 0x7fe95c06d170 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95c0cb050 0x7fe95c153840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95c0cb050 0x7fe95c153840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46360/0 (socket says 172.21.15.32:46360) 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe95bfff640 1 -- 172.21.15.32:0/3647559722 learned_addr learned my addr 172.21.15.32:0/3647559722 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe95b7fe640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c065d40 0x7fe95c0c7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.666+0000 7fe95affd640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe95c06d170 0x7fe95c14b5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe95bfff640 1 -- 172.21.15.32:0/3647559722 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe95c06d170 msgr2=0x7fe95c14b5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe95bfff640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe95c06d170 0x7fe95c14b5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe95bfff640 1 -- 172.21.15.32:0/3647559722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c065d40 msgr2=0x7fe95c0c7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe95bfff640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c065d40 0x7fe95c0c7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe95bfff640 1 -- 172.21.15.32:0/3647559722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe950009420 con 0x7fe95c0cb050 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe95b7fe640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c065d40 0x7fe95c0c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:54.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe95bfff640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95c0cb050 0x7fe95c153840 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe95400cb40 tx=0x7fe95400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe943fff640 1 -- 172.21.15.32:0/3647559722 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe95400aca0 con 0x7fe95c0cb050 2023-12-20T04:43:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe943fff640 1 -- 172.21.15.32:0/3647559722 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe95400ae00 con 0x7fe95c0cb050 2023-12-20T04:43:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe943fff640 1 -- 172.21.15.32:0/3647559722 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe954012620 con 0x7fe95c0cb050 2023-12-20T04:43:54.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe961f6d640 1 -- 172.21.15.32:0/3647559722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe95c15bb30 con 0x7fe95c0cb050 2023-12-20T04:43:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe961f6d640 1 -- 172.21.15.32:0/3647559722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe95c0c6e70 con 0x7fe95c0cb050 2023-12-20T04:43:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe961f6d640 1 -- 172.21.15.32:0/3647559722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe95c0c6a30 con 0x7fe95c0cb050 2023-12-20T04:43:54.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.670+0000 7fe943fff640 1 -- 172.21.15.32:0/3647559722 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fe954007070 con 0x7fe95c0cb050 2023-12-20T04:43:54.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.674+0000 7fe943fff640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe948065af0 0x7fe948067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.674+0000 7fe943fff640 1 -- 172.21.15.32:0/3647559722 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(91..91 src has 1..91) v4 ==== 5609+0+0 (secure 0 0 0) 0x7fe9540833e0 con 0x7fe95c0cb050 2023-12-20T04:43:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.674+0000 7fe95b7fe640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe948065af0 0x7fe948067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:54.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.674+0000 7fe95b7fe640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe948065af0 0x7fe948067fb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe94c007fd0 tx=0x7fe94c009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:54.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.674+0000 7fe943fff640 1 -- 172.21.15.32:0/3647559722 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe954050ec0 con 0x7fe95c0cb050 2023-12-20T04:43:54.836 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:54.838+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:54.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.870+0000 7fe961f6d640 1 -- 172.21.15.32:0/3647559722 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fe95c0c6a30 con 0x7fe95c0cb050 2023-12-20T04:43:54.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.878+0000 7fe943fff640 1 -- 172.21.15.32:0/3647559722 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 10 and data pool 11 v35) v1 ==== 174+0+0 (secure 0 0 0) 0x7fe95c0c6a30 con 0x7fe95c0cb050 2023-12-20T04:43:54.876 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 10 and data pool 11 2023-12-20T04:43:54.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.882+0000 7fe961f6d640 1 -- 172.21.15.32:0/3647559722 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe948065af0 msgr2=0x7fe948067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:54.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.882+0000 7fe961f6d640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe948065af0 0x7fe948067fb0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fe94c007fd0 tx=0x7fe94c009040 comp rx=0 tx=0).stop 2023-12-20T04:43:54.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.882+0000 7fe961f6d640 1 -- 172.21.15.32:0/3647559722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95c0cb050 msgr2=0x7fe95c153840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:54.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.882+0000 7fe961f6d640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95c0cb050 0x7fe95c153840 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7fe95400cb40 tx=0x7fe95400a060 comp rx=0 tx=0).stop 2023-12-20T04:43:54.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.886+0000 7fe961f6d640 1 -- 172.21.15.32:0/3647559722 shutdown_connections 2023-12-20T04:43:54.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.886+0000 7fe961f6d640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe948065af0 0x7fe948067fb0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:54.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.886+0000 7fe961f6d640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe95c0cb050 0x7fe95c153840 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:54.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.886+0000 7fe961f6d640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe95c06d170 0x7fe95c14b5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:54.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.886+0000 7fe961f6d640 1 --2- 172.21.15.32:0/3647559722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe95c065d40 0x7fe95c0c7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:54.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.886+0000 7fe961f6d640 1 -- 172.21.15.32:0/3647559722 >> 172.21.15.32:0/3647559722 conn(0x7fe95c0cf8d0 msgr2=0x7fe95c1581c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:54.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.886+0000 7fe961f6d640 1 -- 172.21.15.32:0/3647559722 shutdown_connections 2023-12-20T04:43:54.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:54.890+0000 7fe961f6d640 1 -- 172.21.15.32:0/3647559722 wait complete. 2023-12-20T04:43:54.922 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:55.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.070+0000 7f18921a6640 1 Processor -- start 2023-12-20T04:43:55.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.070+0000 7f18921a6640 1 -- start start 2023-12-20T04:43:55.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f18921a6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f188c15d9a0 0x7f188c06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f18921a6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f188c06c920 0x7f188c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f18921a6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f188c0ae140 con 0x7f188c1593c0 2023-12-20T04:43:55.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f18921a6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c0ae9e0 con 0x7f188c15d9a0 2023-12-20T04:43:55.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f18921a6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f188c06cd50 con 0x7f188c06c920 2023-12-20T04:43:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f188affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f188c15d9a0 0x7f188c06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f188affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f188c15d9a0 0x7f188c06c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46374/0 (socket says 172.21.15.32:46374) 2023-12-20T04:43:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f188affd640 1 -- 172.21.15.32:0/1038743590 learned_addr learned my addr 172.21.15.32:0/1038743590 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f188b7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f188c1593c0 0x7f188c0afdb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:58728/0 (socket says 172.21.15.32:58728) 2023-12-20T04:43:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f188bfff640 1 --2- 172.21.15.32:0/1038743590 >> v2:172.21.15.39:3300/0 conn(0x7f188c06c920 0x7f188c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f188affd640 1 -- 172.21.15.32:0/1038743590 >> v2:172.21.15.39:3300/0 conn(0x7f188c06c920 msgr2=0x7f188c0705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f188affd640 1 --2- 172.21.15.32:0/1038743590 >> v2:172.21.15.39:3300/0 conn(0x7f188c06c920 0x7f188c0705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f188affd640 1 -- 172.21.15.32:0/1038743590 >> v1:172.21.15.39:6789/0 conn(0x7f188c1593c0 legacy=0x7f188c0afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.074+0000 7f188affd640 1 -- 172.21.15.32:0/1038743590 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188c072170 con 0x7f188c15d9a0 2023-12-20T04:43:55.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f188affd640 1 --2- 172.21.15.32:0/1038743590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f188c15d9a0 0x7f188c06c3e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f188c070150 tx=0x7f188001ea80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cbdd771ac1fa5ec3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:55.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f188a7fc640 1 -- 172.21.15.32:0/1038743590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18800a7ad0 con 0x7f188c15d9a0 2023-12-20T04:43:55.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f188a7fc640 1 -- 172.21.15.32:0/1038743590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18800a7c30 con 0x7f188c15d9a0 2023-12-20T04:43:55.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f188a7fc640 1 -- 172.21.15.32:0/1038743590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18800af440 con 0x7f188c15d9a0 2023-12-20T04:43:55.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f18921a6640 1 -- 172.21.15.32:0/1038743590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f188c15d9a0 msgr2=0x7f188c06c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f18921a6640 1 --2- 172.21.15.32:0/1038743590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f188c15d9a0 0x7f188c06c3e0 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f188c070150 tx=0x7f188001ea80 comp rx=0 tx=0).stop 2023-12-20T04:43:55.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f18921a6640 1 -- 172.21.15.32:0/1038743590 shutdown_connections 2023-12-20T04:43:55.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f18921a6640 1 --2- 172.21.15.32:0/1038743590 >> v2:172.21.15.39:3300/0 conn(0x7f188c06c920 0x7f188c0705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f18921a6640 1 --2- 172.21.15.32:0/1038743590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f188c15d9a0 0x7f188c06c3e0 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f18921a6640 1 -- 172.21.15.32:0/1038743590 >> 172.21.15.32:0/1038743590 conn(0x7f188c06a320 msgr2=0x7f188c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:55.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f18921a6640 1 -- 172.21.15.32:0/1038743590 shutdown_connections 2023-12-20T04:43:55.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f18921a6640 1 -- 172.21.15.32:0/1038743590 wait complete. 2023-12-20T04:43:55.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f18921a6640 1 Processor -- start 2023-12-20T04:43:55.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.078+0000 7f18921a6640 1 -- start start 2023-12-20T04:43:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f18921a6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f188c06c920 0x7f188c14edb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f18921a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f188c1593c0 0x7f188c14e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f18921a6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f188c15d9a0 0x7f188c153e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f18921a6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0aca20 con 0x7f188c1593c0 2023-12-20T04:43:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f18921a6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f188c154350 con 0x7f188c15d9a0 2023-12-20T04:43:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f18921a6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f188c0ab980 con 0x7f188c06c920 2023-12-20T04:43:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f188affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f188c1593c0 0x7f188c14e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f188affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f188c1593c0 0x7f188c14e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46378/0 (socket says 172.21.15.32:46378) 2023-12-20T04:43:55.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f188affd640 1 -- 172.21.15.32:0/3148948822 learned_addr learned my addr 172.21.15.32:0/3148948822 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f188b7fe640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f188c06c920 0x7f188c14edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f188bfff640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f188c15d9a0 0x7f188c153e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f188affd640 1 -- 172.21.15.32:0/3148948822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f188c06c920 msgr2=0x7f188c14edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f188affd640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f188c06c920 0x7f188c14edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f188affd640 1 -- 172.21.15.32:0/3148948822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f188c15d9a0 msgr2=0x7f188c153e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f188affd640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f188c15d9a0 0x7f188c153e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f188affd640 1 -- 172.21.15.32:0/3148948822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f188000b770 con 0x7f188c1593c0 2023-12-20T04:43:55.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f188affd640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f188c1593c0 0x7f188c14e830 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f188001c6b0 tx=0x7f188001b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:55.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f1873fff640 1 -- 172.21.15.32:0/3148948822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18800ba660 con 0x7f188c1593c0 2023-12-20T04:43:55.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f1873fff640 1 -- 172.21.15.32:0/3148948822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18800ba7c0 con 0x7f188c1593c0 2023-12-20T04:43:55.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f1873fff640 1 -- 172.21.15.32:0/3148948822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18800bab70 con 0x7f188c1593c0 2023-12-20T04:43:55.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f18921a6640 1 -- 172.21.15.32:0/3148948822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f188c0abc60 con 0x7f188c1593c0 2023-12-20T04:43:55.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f18921a6640 1 -- 172.21.15.32:0/3148948822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f188c0ac220 con 0x7f188c1593c0 2023-12-20T04:43:55.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f18921a6640 1 -- 172.21.15.32:0/3148948822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f188c04e000 con 0x7f188c1593c0 2023-12-20T04:43:55.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.082+0000 7f1873fff640 1 -- 172.21.15.32:0/3148948822 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f18800c2360 con 0x7f188c1593c0 2023-12-20T04:43:55.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.086+0000 7f1873fff640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1878065af0 0x7f1878067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.086+0000 7f1873fff640 1 -- 172.21.15.32:0/3148948822 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f188011dfb0 con 0x7f188c1593c0 2023-12-20T04:43:55.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.086+0000 7f188b7fe640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1878065af0 0x7f1878067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.086+0000 7f188b7fe640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1878065af0 0x7f1878067fb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f187c000fc0 tx=0x7f187c000f30 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:55.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.090+0000 7f1873fff640 1 -- 172.21.15.32:0/3148948822 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f18800eba30 con 0x7f188c1593c0 2023-12-20T04:43:55.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.254+0000 7f18921a6640 1 -- 172.21.15.32:0/3148948822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f188c0acba0 con 0x7f188c1593c0 2023-12-20T04:43:55.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f1873fff640 1 -- 172.21.15.32:0/3148948822 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16065 (secure 0 0 0) 0x7f18800ef050 con 0x7f188c1593c0 2023-12-20T04:43:55.255 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:55.256 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:54.873834+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:52.582628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:53.871341+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 -- 172.21.15.32:0/3148948822 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1878065af0 msgr2=0x7f1878067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1878065af0 0x7f1878067fb0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f187c000fc0 tx=0x7f187c000f30 comp rx=0 tx=0).stop 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 -- 172.21.15.32:0/3148948822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f188c1593c0 msgr2=0x7f188c14e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f188c1593c0 0x7f188c14e830 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f188001c6b0 tx=0x7f188001b820 comp rx=0 tx=0).stop 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 -- 172.21.15.32:0/3148948822 shutdown_connections 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1878065af0 0x7f1878067fb0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f188c15d9a0 0x7f188c153e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f188c1593c0 0x7f188c14e830 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 --2- 172.21.15.32:0/3148948822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f188c06c920 0x7f188c14edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 -- 172.21.15.32:0/3148948822 >> 172.21.15.32:0/3148948822 conn(0x7f188c06a320 msgr2=0x7f188c0705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 -- 172.21.15.32:0/3148948822 shutdown_connections 2023-12-20T04:43:55.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.258+0000 7f18921a6640 1 -- 172.21.15.32:0/3148948822 wait complete. 2023-12-20T04:43:55.278 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:55.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd223020640 1 Processor -- start 2023-12-20T04:43:55.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd223020640 1 -- start start 2023-12-20T04:43:55.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd223020640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd21c0aece0 0x7fd21c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd223020640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd21c0b0b30 0x7fd21c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd223020640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd21c0c98c0 con 0x7fd21c1599e0 2023-12-20T04:43:55.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd223020640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd21c0c9ba0 con 0x7fd21c0aece0 2023-12-20T04:43:55.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd223020640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c0c9e80 con 0x7fd21c0b0b30 2023-12-20T04:43:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd21bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd21c0aece0 0x7fd21c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd220dbc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd21c1599e0 0x7fd21c0c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34890/0 (socket says 172.21.15.32:34890) 2023-12-20T04:43:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd220dbc640 1 -- 172.21.15.32:0/2960077822 learned_addr learned my addr 172.21.15.32:0/2960077822 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd2215bd640 1 --2- 172.21.15.32:0/2960077822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd21c0b0b30 0x7fd21c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd21bfff640 1 -- 172.21.15.32:0/2960077822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd21c0b0b30 msgr2=0x7fd21c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd21bfff640 1 --2- 172.21.15.32:0/2960077822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd21c0b0b30 0x7fd21c0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd21bfff640 1 -- 172.21.15.32:0/2960077822 >> v1:172.21.15.32:6789/0 conn(0x7fd21c1599e0 legacy=0x7fd21c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd21bfff640 1 -- 172.21.15.32:0/2960077822 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd21c0ca160 con 0x7fd21c0aece0 2023-12-20T04:43:55.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.426+0000 7fd21b7fe640 1 -- 172.21.15.32:0/2960077822 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3062701643 0 0) 0x7fd21c0c98c0 con 0x7fd21c1599e0 2023-12-20T04:43:55.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd21bfff640 1 --2- 172.21.15.32:0/2960077822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21c0aece0 0x7fd21c0b05f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd2080097c0 tx=0x7fd20801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ff4fe13fae8b83 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:55.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd21b7fe640 1 -- 172.21.15.32:0/2960077822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080ad640 con 0x7fd21c0aece0 2023-12-20T04:43:55.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd21b7fe640 1 -- 172.21.15.32:0/2960077822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2080ad7a0 con 0x7fd21c0aece0 2023-12-20T04:43:55.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd21b7fe640 1 -- 172.21.15.32:0/2960077822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080adb30 con 0x7fd21c0aece0 2023-12-20T04:43:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd223020640 1 -- 172.21.15.32:0/2960077822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21c0aece0 msgr2=0x7fd21c0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd223020640 1 --2- 172.21.15.32:0/2960077822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21c0aece0 0x7fd21c0b05f0 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7fd2080097c0 tx=0x7fd20801c480 comp rx=0 tx=0).stop 2023-12-20T04:43:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd223020640 1 -- 172.21.15.32:0/2960077822 shutdown_connections 2023-12-20T04:43:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd223020640 1 --2- 172.21.15.32:0/2960077822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21c0aece0 0x7fd21c0b05f0 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd223020640 1 --2- 172.21.15.32:0/2960077822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd21c0b0b30 0x7fd21c0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd223020640 1 -- 172.21.15.32:0/2960077822 >> 172.21.15.32:0/2960077822 conn(0x7fd21c06a320 msgr2=0x7fd21c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd223020640 1 -- 172.21.15.32:0/2960077822 shutdown_connections 2023-12-20T04:43:55.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.430+0000 7fd223020640 1 -- 172.21.15.32:0/2960077822 wait complete. 2023-12-20T04:43:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd223020640 1 Processor -- start 2023-12-20T04:43:55.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd223020640 1 -- start start 2023-12-20T04:43:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd223020640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd21c0aece0 0x7fd21c14d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd223020640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21c0b0b30 0x7fd21c15aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd223020640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd21c1599e0 0x7fd21c15c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd223020640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c14dda0 con 0x7fd21c0b0b30 2023-12-20T04:43:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd223020640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c0ac1c0 con 0x7fd21c0aece0 2023-12-20T04:43:55.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd223020640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd21c0ac460 con 0x7fd21c1599e0 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd21bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21c0b0b30 0x7fd21c15aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd21bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21c0b0b30 0x7fd21c15aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46398/0 (socket says 172.21.15.32:46398) 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd21bfff640 1 -- 172.21.15.32:0/3991375375 learned_addr learned my addr 172.21.15.32:0/3991375375 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd220dbc640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd21c0aece0 0x7fd21c14d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd2215bd640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd21c1599e0 0x7fd21c15c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd21bfff640 1 -- 172.21.15.32:0/3991375375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd21c1599e0 msgr2=0x7fd21c15c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd21bfff640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd21c1599e0 0x7fd21c15c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd21bfff640 1 -- 172.21.15.32:0/3991375375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd21c0aece0 msgr2=0x7fd21c14d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd21bfff640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd21c0aece0 0x7fd21c14d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd21bfff640 1 -- 172.21.15.32:0/3991375375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd208009420 con 0x7fd21c0b0b30 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd21bfff640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21c0b0b30 0x7fd21c15aa80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd20801a0b0 tx=0x7fd208019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd2197fa640 1 -- 172.21.15.32:0/3991375375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080b6610 con 0x7fd21c0b0b30 2023-12-20T04:43:55.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd2197fa640 1 -- 172.21.15.32:0/3991375375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2080a6040 con 0x7fd21c0b0b30 2023-12-20T04:43:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd223020640 1 -- 172.21.15.32:0/3991375375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd21c0ac740 con 0x7fd21c0b0b30 2023-12-20T04:43:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd2197fa640 1 -- 172.21.15.32:0/3991375375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2080b69b0 con 0x7fd21c0b0b30 2023-12-20T04:43:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd223020640 1 -- 172.21.15.32:0/3991375375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd21c0acd00 con 0x7fd21c0b0b30 2023-12-20T04:43:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.434+0000 7fd223020640 1 -- 172.21.15.32:0/3991375375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd21c06acf0 con 0x7fd21c0b0b30 2023-12-20T04:43:55.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.438+0000 7fd2197fa640 1 -- 172.21.15.32:0/3991375375 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fd2080b4040 con 0x7fd21c0b0b30 2023-12-20T04:43:55.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.438+0000 7fd2197fa640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd200065b20 0x7fd200067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.438+0000 7fd2197fa640 1 -- 172.21.15.32:0/3991375375 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fd20811d4e0 con 0x7fd21c0b0b30 2023-12-20T04:43:55.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.438+0000 7fd220dbc640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd200065b20 0x7fd200067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.438+0000 7fd220dbc640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd200065b20 0x7fd200067fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd210007fd0 tx=0x7fd210009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:55.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.442+0000 7fd2197fa640 1 -- 172.21.15.32:0/3991375375 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd2080eaef0 con 0x7fd21c0b0b30 2023-12-20T04:43:55.591 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:55.590+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:55.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.606+0000 7fd223020640 1 -- 172.21.15.32:0/3991375375 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd21c04e000 con 0x7fd21c0b0b30 2023-12-20T04:43:55.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.606+0000 7fd2197fa640 1 -- 172.21.15.32:0/3991375375 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v92) v1 ==== 74+0+16065 (secure 0 0 0) 0x7fd2080b2020 con 0x7fd21c0b0b30 2023-12-20T04:43:55.606 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:55.606 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":92,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:54.873834+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:52.582628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:53.871341+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 -- 172.21.15.32:0/3991375375 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd200065b20 msgr2=0x7fd200067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd200065b20 0x7fd200067fe0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd210007fd0 tx=0x7fd210009040 comp rx=0 tx=0).stop 2023-12-20T04:43:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 -- 172.21.15.32:0/3991375375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21c0b0b30 msgr2=0x7fd21c15aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21c0b0b30 0x7fd21c15aa80 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7fd20801a0b0 tx=0x7fd208019220 comp rx=0 tx=0).stop 2023-12-20T04:43:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 -- 172.21.15.32:0/3991375375 shutdown_connections 2023-12-20T04:43:55.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd200065b20 0x7fd200067fe0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd21c1599e0 0x7fd21c15c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd21c0b0b30 0x7fd21c15aa80 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 --2- 172.21.15.32:0/3991375375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd21c0aece0 0x7fd21c14d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 -- 172.21.15.32:0/3991375375 >> 172.21.15.32:0/3991375375 conn(0x7fd21c06a320 msgr2=0x7fd21c071320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 -- 172.21.15.32:0/3991375375 shutdown_connections 2023-12-20T04:43:55.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.610+0000 7fd223020640 1 -- 172.21.15.32:0/3991375375 wait complete. 2023-12-20T04:43:55.629 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T04:43:55.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.777+0000 7fbeca895640 1 Processor -- start 2023-12-20T04:43:55.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.777+0000 7fbeca895640 1 -- start start 2023-12-20T04:43:55.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbeca895640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbec41591e0 0x7fbec40caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbeca895640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbec4070940 0x7fbec40cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbeca895640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbec40cc3b0 con 0x7fbec406cdf0 2023-12-20T04:43:55.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbeca895640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbec40c8e30 con 0x7fbec4070940 2023-12-20T04:43:55.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbeca895640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec406d280 con 0x7fbec41591e0 2023-12-20T04:43:55.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbec37fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbec406cdf0 0x7fbec4070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34898/0 (socket says 172.21.15.32:34898) 2023-12-20T04:43:55.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbec37fe640 1 -- 172.21.15.32:0/632727641 learned_addr learned my addr 172.21.15.32:0/632727641 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:55.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbec8e32640 1 --2- 172.21.15.32:0/632727641 >> v2:172.21.15.32:3300/0 conn(0x7fbec4070940 0x7fbec40cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbec3fff640 1 --2- 172.21.15.32:0/632727641 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbec41591e0 0x7fbec40caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbec8e32640 1 -- 172.21.15.32:0/632727641 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbec41591e0 msgr2=0x7fbec40caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbec8e32640 1 --2- 172.21.15.32:0/632727641 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbec41591e0 0x7fbec40caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbec8e32640 1 -- 172.21.15.32:0/632727641 >> v1:172.21.15.32:6789/0 conn(0x7fbec406cdf0 legacy=0x7fbec4070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbec8e32640 1 -- 172.21.15.32:0/632727641 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbec40c90d0 con 0x7fbec4070940 2023-12-20T04:43:55.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.781+0000 7fbec2ffd640 1 -- 172.21.15.32:0/632727641 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4074972487 0 0) 0x7fbec40cc3b0 con 0x7fbec406cdf0 2023-12-20T04:43:55.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbec8e32640 1 --2- 172.21.15.32:0/632727641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbec4070940 0x7fbec40cc840 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009d90 tx=0x7fbebc01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c992c72c607283c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:55.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbec2ffd640 1 -- 172.21.15.32:0/632727641 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc0a58e0 con 0x7fbec4070940 2023-12-20T04:43:55.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbec2ffd640 1 -- 172.21.15.32:0/632727641 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbebc0a5a40 con 0x7fbec4070940 2023-12-20T04:43:55.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbec2ffd640 1 -- 172.21.15.32:0/632727641 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbebc0a5dd0 con 0x7fbec4070940 2023-12-20T04:43:55.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbeca895640 1 -- 172.21.15.32:0/632727641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbec4070940 msgr2=0x7fbec40cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbeca895640 1 --2- 172.21.15.32:0/632727641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbec4070940 0x7fbec40cc840 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7fbebc009d90 tx=0x7fbebc01c8c0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbeca895640 1 -- 172.21.15.32:0/632727641 shutdown_connections 2023-12-20T04:43:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbeca895640 1 --2- 172.21.15.32:0/632727641 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbec4070940 0x7fbec40cc840 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbeca895640 1 --2- 172.21.15.32:0/632727641 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbec41591e0 0x7fbec40caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbeca895640 1 -- 172.21.15.32:0/632727641 >> 172.21.15.32:0/632727641 conn(0x7fbec406a510 msgr2=0x7fbec40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbeca895640 1 -- 172.21.15.32:0/632727641 shutdown_connections 2023-12-20T04:43:55.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbeca895640 1 -- 172.21.15.32:0/632727641 wait complete. 2023-12-20T04:43:55.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbeca895640 1 Processor -- start 2023-12-20T04:43:55.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.785+0000 7fbeca895640 1 -- start start 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbeca895640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbec4070940 0x7fbec4143460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbeca895640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbec41591e0 0x7fbec40c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbeca895640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec40c58b0 0x7fbec4149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbeca895640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec414a850 con 0x7fbec4070940 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbeca895640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbec41439a0 con 0x7fbec40c58b0 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbeca895640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbec4147f50 con 0x7fbec41591e0 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbec4070940 0x7fbec4143460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbec4070940 0x7fbec4143460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46410/0 (socket says 172.21.15.32:46410) 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec3fff640 1 -- 172.21.15.32:0/1711805478 learned_addr learned my addr 172.21.15.32:0/1711805478 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec37fe640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbec41591e0 0x7fbec40c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec8e32640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec40c58b0 0x7fbec4149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec3fff640 1 -- 172.21.15.32:0/1711805478 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbec41591e0 msgr2=0x7fbec40c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec3fff640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbec41591e0 0x7fbec40c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec3fff640 1 -- 172.21.15.32:0/1711805478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec40c58b0 msgr2=0x7fbec4149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec3fff640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec40c58b0 0x7fbec4149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec3fff640 1 -- 172.21.15.32:0/1711805478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbebc009a70 con 0x7fbec4070940 2023-12-20T04:43:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec37fe640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbec41591e0 0x7fbec40c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec8e32640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec40c58b0 0x7fbec4149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec3fff640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbec4070940 0x7fbec4143460 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400d5f0 tx=0x7fbeb400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec0ff9640 1 -- 172.21.15.32:0/1711805478 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb40125f0 con 0x7fbec4070940 2023-12-20T04:43:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec0ff9640 1 -- 172.21.15.32:0/1711805478 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbeb4012750 con 0x7fbec4070940 2023-12-20T04:43:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec0ff9640 1 -- 172.21.15.32:0/1711805478 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbeb4012ae0 con 0x7fbec4070940 2023-12-20T04:43:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbeca895640 1 -- 172.21.15.32:0/1711805478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbec414aa30 con 0x7fbec4070940 2023-12-20T04:43:55.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbeca895640 1 -- 172.21.15.32:0/1711805478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbec414aff0 con 0x7fbec4070940 2023-12-20T04:43:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbeca895640 1 -- 172.21.15.32:0/1711805478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbec404e000 con 0x7fbec4070940 2023-12-20T04:43:55.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.789+0000 7fbec0ff9640 1 -- 172.21.15.32:0/1711805478 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fbeb4012c40 con 0x7fbec4070940 2023-12-20T04:43:55.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.793+0000 7fbec0ff9640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbeb0065af0 0x7fbeb0067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.793+0000 7fbec0ff9640 1 -- 172.21.15.32:0/1711805478 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(92..92 src has 1..92) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbeb40835e0 con 0x7fbec4070940 2023-12-20T04:43:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.793+0000 7fbec37fe640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbeb0065af0 0x7fbeb0067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:55.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.793+0000 7fbec37fe640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbeb0065af0 0x7fbeb0067fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8007fd0 tx=0x7fbeb8007c20 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:55.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.797+0000 7fbec0ff9640 1 -- 172.21.15.32:0/1711805478 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbeb4051040 con 0x7fbec4070940 2023-12-20T04:43:55.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:55.997+0000 7fbeca895640 1 -- 172.21.15.32:0/1711805478 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fbec40ccd50 con 0x7fbec4070940 2023-12-20T04:43:56.324 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:56.325+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:56.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.889+0000 7fbec0ff9640 1 -- 172.21.15.32:0/1711805478 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v38) v1 ==== 118+0+0 (secure 0 0 0) 0x7fbeb4007380 con 0x7fbec4070940 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 -- 172.21.15.32:0/1711805478 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbeb0065af0 msgr2=0x7fbeb0067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbeb0065af0 0x7fbeb0067fb0 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fbeb8007fd0 tx=0x7fbeb8007c20 comp rx=0 tx=0).stop 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 -- 172.21.15.32:0/1711805478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbec4070940 msgr2=0x7fbec4143460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbec4070940 0x7fbec4143460 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7fbeb400d5f0 tx=0x7fbeb400dab0 comp rx=0 tx=0).stop 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 -- 172.21.15.32:0/1711805478 shutdown_connections 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbeb0065af0 0x7fbeb0067fb0 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbec40c58b0 0x7fbec4149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbec41591e0 0x7fbec40c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 --2- 172.21.15.32:0/1711805478 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbec4070940 0x7fbec4143460 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 -- 172.21.15.32:0/1711805478 >> 172.21.15.32:0/1711805478 conn(0x7fbec406a510 msgr2=0x7fbec40c47e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 -- 172.21.15.32:0/1711805478 shutdown_connections 2023-12-20T04:43:56.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:56.893+0000 7fbeca895640 1 -- 172.21.15.32:0/1711805478 wait complete. 2023-12-20T04:43:56.914 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T04:43:56.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T04:43:57.058 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:57.057+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:57.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.061+0000 7f478721f640 1 Processor -- start 2023-12-20T04:43:57.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.061+0000 7f478721f640 1 -- start start 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f478721f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4780157480 0x7f47800c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f478721f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47800b0760 0x7f47800b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f478721f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47800b1100 con 0x7f478015d660 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f478721f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47800cde20 con 0x7f47800b0760 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f478721f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47800c8e30 con 0x7f4780157480 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f477ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47800b0760 0x7f47800b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f477ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47800b0760 0x7f47800b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46426/0 (socket says 172.21.15.32:46426) 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f477ffff640 1 -- 172.21.15.32:0/2240010540 learned_addr learned my addr 172.21.15.32:0/2240010540 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f4784fbb640 1 --2- 172.21.15.32:0/2240010540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4780157480 0x7f47800c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f477ffff640 1 -- 172.21.15.32:0/2240010540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4780157480 msgr2=0x7f47800c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f477ffff640 1 --2- 172.21.15.32:0/2240010540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4780157480 0x7f47800c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f477ffff640 1 -- 172.21.15.32:0/2240010540 >> v1:172.21.15.39:6789/0 conn(0x7f478015d660 legacy=0x7f478015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:57.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f477ffff640 1 -- 172.21.15.32:0/2240010540 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47800ce070 con 0x7f47800b0760 2023-12-20T04:43:57.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.065+0000 7f477ffff640 1 --2- 172.21.15.32:0/2240010540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47800b0760 0x7f47800b0bc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f476c009870 tx=0x7f476c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=124352c83ad2dbd8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f477f7fe640 1 -- 172.21.15.32:0/2240010540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c0ab680 con 0x7f47800b0760 2023-12-20T04:43:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f477f7fe640 1 -- 172.21.15.32:0/2240010540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476c0ab7e0 con 0x7f47800b0760 2023-12-20T04:43:57.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f477f7fe640 1 -- 172.21.15.32:0/2240010540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476c0abb70 con 0x7f47800b0760 2023-12-20T04:43:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 -- 172.21.15.32:0/2240010540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47800b0760 msgr2=0x7f47800b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 --2- 172.21.15.32:0/2240010540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47800b0760 0x7f47800b0bc0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f476c009870 tx=0x7f476c01c480 comp rx=0 tx=0).stop 2023-12-20T04:43:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 -- 172.21.15.32:0/2240010540 shutdown_connections 2023-12-20T04:43:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 --2- 172.21.15.32:0/2240010540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47800b0760 0x7f47800b0bc0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 --2- 172.21.15.32:0/2240010540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4780157480 0x7f47800c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 -- 172.21.15.32:0/2240010540 >> 172.21.15.32:0/2240010540 conn(0x7f478006a510 msgr2=0x7f47800ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 -- 172.21.15.32:0/2240010540 shutdown_connections 2023-12-20T04:43:57.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 -- 172.21.15.32:0/2240010540 wait complete. 2023-12-20T04:43:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 Processor -- start 2023-12-20T04:43:57.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 -- start start 2023-12-20T04:43:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47800b0760 0x7f478014f140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4780157480 0x7f4780162b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478015d660 0x7f47801630c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47800b39b0 con 0x7f47800b0760 2023-12-20T04:43:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f478015c1b0 con 0x7f478015d660 2023-12-20T04:43:57.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f478721f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478015c450 con 0x7f4780157480 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f4784fbb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47800b0760 0x7f478014f140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f4784fbb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47800b0760 0x7f478014f140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46430/0 (socket says 172.21.15.32:46430) 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.069+0000 7f4784fbb640 1 -- 172.21.15.32:0/469216283 learned_addr learned my addr 172.21.15.32:0/469216283 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f477ffff640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4780157480 0x7f4780162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f47857bc640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478015d660 0x7f47801630c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f4784fbb640 1 -- 172.21.15.32:0/469216283 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4780157480 msgr2=0x7f4780162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f4784fbb640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4780157480 0x7f4780162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f4784fbb640 1 -- 172.21.15.32:0/469216283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478015d660 msgr2=0x7f47801630c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f4784fbb640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478015d660 0x7f47801630c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f4784fbb640 1 -- 172.21.15.32:0/469216283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f476c0094d0 con 0x7f47800b0760 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f477ffff640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4780157480 0x7f4780162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f4784fbb640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47800b0760 0x7f478014f140 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f477400d5f0 tx=0x7f477400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f477d7fa640 1 -- 172.21.15.32:0/469216283 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47740125f0 con 0x7f47800b0760 2023-12-20T04:43:57.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f477d7fa640 1 -- 172.21.15.32:0/469216283 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4774012750 con 0x7f47800b0760 2023-12-20T04:43:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f478721f640 1 -- 172.21.15.32:0/469216283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f478015c730 con 0x7f47800b0760 2023-12-20T04:43:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f477d7fa640 1 -- 172.21.15.32:0/469216283 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4774012ae0 con 0x7f47800b0760 2023-12-20T04:43:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f478721f640 1 -- 172.21.15.32:0/469216283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f478015ccf0 con 0x7f47800b0760 2023-12-20T04:43:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f478721f640 1 -- 172.21.15.32:0/469216283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f478004e000 con 0x7f47800b0760 2023-12-20T04:43:57.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.073+0000 7f477d7fa640 1 -- 172.21.15.32:0/469216283 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f4774012c40 con 0x7f47800b0760 2023-12-20T04:43:57.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.077+0000 7f477d7fa640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4764065af0 0x7f4764067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:57.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.077+0000 7f477d7fa640 1 -- 172.21.15.32:0/469216283 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f4774083470 con 0x7f47800b0760 2023-12-20T04:43:57.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.077+0000 7f477ffff640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4764065af0 0x7f4764067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:57.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.077+0000 7f477ffff640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4764065af0 0x7f4764067fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f476c0af000 tx=0x7f476c0b6040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:57.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.081+0000 7f477d7fa640 1 -- 172.21.15.32:0/469216283 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4774050f80 con 0x7f47800b0760 2023-12-20T04:43:57.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.277+0000 7f478721f640 1 -- 172.21.15.32:0/469216283 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f47800cabf0 con 0x7f47800b0760 2023-12-20T04:43:57.786 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:57.785+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:57.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.893+0000 7f477d7fa640 1 -- 172.21.15.32:0/469216283 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v39) v1 ==== 122+0+0 (secure 0 0 0) 0x7f47740545a0 con 0x7f47800b0760 2023-12-20T04:43:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.893+0000 7f478721f640 1 -- 172.21.15.32:0/469216283 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4764065af0 msgr2=0x7f4764067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.893+0000 7f478721f640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4764065af0 0x7f4764067fb0 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f476c0af000 tx=0x7f476c0b6040 comp rx=0 tx=0).stop 2023-12-20T04:43:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.893+0000 7f478721f640 1 -- 172.21.15.32:0/469216283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47800b0760 msgr2=0x7f478014f140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.893+0000 7f478721f640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47800b0760 0x7f478014f140 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f477400d5f0 tx=0x7f477400dab0 comp rx=0 tx=0).stop 2023-12-20T04:43:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.897+0000 7f478721f640 1 -- 172.21.15.32:0/469216283 shutdown_connections 2023-12-20T04:43:57.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.897+0000 7f478721f640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4764065af0 0x7f4764067fb0 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.897+0000 7f478721f640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478015d660 0x7f47801630c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.897+0000 7f478721f640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4780157480 0x7f4780162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.897+0000 7f478721f640 1 --2- 172.21.15.32:0/469216283 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47800b0760 0x7f478014f140 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.897+0000 7f478721f640 1 -- 172.21.15.32:0/469216283 >> 172.21.15.32:0/469216283 conn(0x7f478006a510 msgr2=0x7f47800c4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.897+0000 7f478721f640 1 -- 172.21.15.32:0/469216283 shutdown_connections 2023-12-20T04:43:57.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:57.897+0000 7f478721f640 1 -- 172.21.15.32:0/469216283 wait complete. 2023-12-20T04:43:57.915 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T04:43:57.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:43:58.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c18da2640 1 Processor -- start 2023-12-20T04:43:58.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c18da2640 1 -- start start 2023-12-20T04:43:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c18da2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0c1406b630 0x7f0c14160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c18da2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c14161330 0x7f0c14153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c18da2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c140cd4d0 con 0x7f0c140b8480 2023-12-20T04:43:58.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c18da2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c140ce4e0 con 0x7f0c1406b630 2023-12-20T04:43:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c18da2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140cd720 con 0x7f0c14161330 2023-12-20T04:43:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c137fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0c140b8480 0x7f0c1406af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34912/0 (socket says 172.21.15.32:34912) 2023-12-20T04:43:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c137fe640 1 -- 172.21.15.32:0/1224928600 learned_addr learned my addr 172.21.15.32:0/1224928600 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c12ffd640 1 --2- 172.21.15.32:0/1224928600 >> v2:172.21.15.39:3300/0 conn(0x7f0c1406b630 0x7f0c14160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c13fff640 1 --2- 172.21.15.32:0/1224928600 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c14161330 0x7f0c14153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c127fc640 1 -- 172.21.15.32:0/1224928600 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 604415406 0 0) 0x7f0c140cd4d0 con 0x7f0c140b8480 2023-12-20T04:43:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c127fc640 1 -- 172.21.15.32:0/1224928600 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0bfc003530 con 0x7f0c140b8480 2023-12-20T04:43:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c12ffd640 1 -- 172.21.15.32:0/1224928600 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c14161330 msgr2=0x7f0c14153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c12ffd640 1 --2- 172.21.15.32:0/1224928600 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c14161330 0x7f0c14153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c12ffd640 1 -- 172.21.15.32:0/1224928600 >> v1:172.21.15.32:6789/0 conn(0x7f0c140b8480 legacy=0x7f0c1406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.065+0000 7f0c12ffd640 1 -- 172.21.15.32:0/1224928600 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140cda00 con 0x7f0c1406b630 2023-12-20T04:43:58.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c12ffd640 1 --2- 172.21.15.32:0/1224928600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1406b630 0x7f0c14160df0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097c0 tx=0x7f0c0801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=af61a9ab2c104d84 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:58.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c127fc640 1 -- 172.21.15.32:0/1224928600 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c080ab600 con 0x7f0c1406b630 2023-12-20T04:43:58.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c127fc640 1 -- 172.21.15.32:0/1224928600 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c080a7050 con 0x7f0c1406b630 2023-12-20T04:43:58.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c127fc640 1 -- 172.21.15.32:0/1224928600 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c080ab980 con 0x7f0c1406b630 2023-12-20T04:43:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c18da2640 1 -- 172.21.15.32:0/1224928600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1406b630 msgr2=0x7f0c14160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c18da2640 1 --2- 172.21.15.32:0/1224928600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1406b630 0x7f0c14160df0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7f0c080097c0 tx=0x7f0c0801c480 comp rx=0 tx=0).stop 2023-12-20T04:43:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c18da2640 1 -- 172.21.15.32:0/1224928600 shutdown_connections 2023-12-20T04:43:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c18da2640 1 --2- 172.21.15.32:0/1224928600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1406b630 0x7f0c14160df0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c18da2640 1 --2- 172.21.15.32:0/1224928600 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c14161330 0x7f0c14153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c18da2640 1 -- 172.21.15.32:0/1224928600 >> 172.21.15.32:0/1224928600 conn(0x7f0c140d2a50 msgr2=0x7f0c140d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c18da2640 1 -- 172.21.15.32:0/1224928600 shutdown_connections 2023-12-20T04:43:58.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.069+0000 7f0c18da2640 1 -- 172.21.15.32:0/1224928600 wait complete. 2023-12-20T04:43:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c18da2640 1 Processor -- start 2023-12-20T04:43:58.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c18da2640 1 -- start start 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c18da2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1406b630 0x7f0c140c5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c18da2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c140b8480 0x7f0c1414c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c18da2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c14161330 0x7f0c1414c650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c18da2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c1414e660 con 0x7f0c14161330 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c18da2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140c91d0 con 0x7f0c1406b630 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c18da2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14151960 con 0x7f0c140b8480 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c13fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c14161330 0x7f0c1414c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c137fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1406b630 0x7f0c140c5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c12ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c140b8480 0x7f0c1414c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c13fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c14161330 0x7f0c1414c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46436/0 (socket says 172.21.15.32:46436) 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c13fff640 1 -- 172.21.15.32:0/1531812241 learned_addr learned my addr 172.21.15.32:0/1531812241 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:58.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c13fff640 1 -- 172.21.15.32:0/1531812241 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c140b8480 msgr2=0x7f0c1414c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c13fff640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c140b8480 0x7f0c1414c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c13fff640 1 -- 172.21.15.32:0/1531812241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1406b630 msgr2=0x7f0c140c5520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c13fff640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1406b630 0x7f0c140c5520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c13fff640 1 -- 172.21.15.32:0/1531812241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c08009420 con 0x7f0c14161330 2023-12-20T04:43:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c13fff640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c14161330 0x7f0c1414c650 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00efc0 tx=0x7f0c0c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0bfbfff640 1 -- 172.21.15.32:0/1531812241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c009ba0 con 0x7f0c14161330 2023-12-20T04:43:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0bfbfff640 1 -- 172.21.15.32:0/1531812241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c0c00f040 con 0x7f0c14161330 2023-12-20T04:43:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c18da2640 1 -- 172.21.15.32:0/1531812241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c141513a0 con 0x7f0c14161330 2023-12-20T04:43:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0bfbfff640 1 -- 172.21.15.32:0/1531812241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c0c0146c0 con 0x7f0c14161330 2023-12-20T04:43:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.073+0000 7f0c18da2640 1 -- 172.21.15.32:0/1531812241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c1414dfa0 con 0x7f0c14161330 2023-12-20T04:43:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.077+0000 7f0c18da2640 1 -- 172.21.15.32:0/1531812241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1414e7a0 con 0x7f0c14161330 2023-12-20T04:43:58.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.077+0000 7f0bfbfff640 1 -- 172.21.15.32:0/1531812241 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f0c0c014820 con 0x7f0c14161330 2023-12-20T04:43:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.077+0000 7f0bfbfff640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c00065af0 0x7f0c00067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.077+0000 7f0bfbfff640 1 -- 172.21.15.32:0/1531812241 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f0c0c083d30 con 0x7f0c14161330 2023-12-20T04:43:58.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.077+0000 7f0c137fe640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c00065af0 0x7f0c00067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.077+0000 7f0c137fe640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c00065af0 0x7f0c00067fb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0c04007fd0 tx=0x7f0c04009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:58.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.081+0000 7f0bfbfff640 1 -- 172.21.15.32:0/1531812241 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0c1414e7a0 con 0x7f0c14161330 2023-12-20T04:43:58.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.273+0000 7f0c18da2640 1 -- 172.21.15.32:0/1531812241 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c1414e7a0 con 0x7f0c14161330 2023-12-20T04:43:58.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0bfbfff640 1 -- 172.21.15.32:0/1531812241 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4699 (secure 0 0 0) 0x7f0c1414e7a0 con 0x7f0c14161330 2023-12-20T04:43:58.276 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:58.276 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/655215798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":655215798},{"type":"v1","addr":"172.21.15.32:6837","nonce":655215798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/661544180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":661544180},{"type":"v1","addr":"172.21.15.39:6834","nonce":661544180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6855,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/833127896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":833127896},{"type":"v1","addr":"172.21.15.32:6836","nonce":833127896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:54.874151+0000","modified":"2023-12-20T04:43:57.893768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6864":{"gid":6864,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/979895154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":979895154},{"type":"v1","addr":"172.21.15.39:6835","nonce":979895154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-20T04:43:58.276 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2023-12-20T04:43:58.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 -- 172.21.15.32:0/1531812241 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c00065af0 msgr2=0x7f0c00067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c00065af0 0x7f0c00067fb0 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f0c04007fd0 tx=0x7f0c04009040 comp rx=0 tx=0).stop 2023-12-20T04:43:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 -- 172.21.15.32:0/1531812241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c14161330 msgr2=0x7f0c1414c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c14161330 0x7f0c1414c650 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f0c0c00efc0 tx=0x7f0c0c00c490 comp rx=0 tx=0).stop 2023-12-20T04:43:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 -- 172.21.15.32:0/1531812241 shutdown_connections 2023-12-20T04:43:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0c00065af0 0x7f0c00067fb0 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c14161330 0x7f0c1414c650 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c140b8480 0x7f0c1414c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 --2- 172.21.15.32:0/1531812241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c1406b630 0x7f0c140c5520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 -- 172.21.15.32:0/1531812241 >> 172.21.15.32:0/1531812241 conn(0x7f0c140d2a50 msgr2=0x7f0c140b6220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 -- 172.21.15.32:0/1531812241 shutdown_connections 2023-12-20T04:43:58.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.277+0000 7f0c18da2640 1 -- 172.21.15.32:0/1531812241 wait complete. 2023-12-20T04:43:58.298 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:43:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.445+0000 7fda0c9e9640 1 Processor -- start 2023-12-20T04:43:58.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.445+0000 7fda0c9e9640 1 -- start start 2023-12-20T04:43:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda0c9e9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda080b0d00 0x7fda08164a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda0c9e9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda080711d0 con 0x7fda081599e0 2023-12-20T04:43:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda0c9e9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fda080c98c0 con 0x7fda080aece0 2023-12-20T04:43:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda0c9e9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fda080c9ba0 con 0x7fda080b0d00 2023-12-20T04:43:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda06d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fda080b0d00 0x7fda08164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda0659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fda081599e0 0x7fda080c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:34916/0 (socket says 172.21.15.32:34916) 2023-12-20T04:43:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda0659c640 1 -- 172.21.15.32:0/3631105749 learned_addr learned my addr 172.21.15.32:0/3631105749 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda06d9d640 1 --2- 172.21.15.32:0/3631105749 >> v2:172.21.15.32:3300/0 conn(0x7fda080b0d00 0x7fda08164a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46440/0 (socket says 172.21.15.32:46440) 2023-12-20T04:43:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda06d9d640 1 -- 172.21.15.32:0/3631105749 >> v1:172.21.15.39:6789/0 conn(0x7fda080aece0 legacy=0x7fda080b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda06d9d640 1 -- 172.21.15.32:0/3631105749 >> v1:172.21.15.32:6789/0 conn(0x7fda081599e0 legacy=0x7fda080c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda06d9d640 1 -- 172.21.15.32:0/3631105749 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda080c9e80 con 0x7fda080b0d00 2023-12-20T04:43:58.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.449+0000 7fda0559a640 1 -- 172.21.15.32:0/3631105749 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4097213709 0 0) 0x7fda080c98c0 con 0x7fda080aece0 2023-12-20T04:43:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda06d9d640 1 --2- 172.21.15.32:0/3631105749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda080b0d00 0x7fda08164a70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fda0000bfa0 tx=0x7fda0001e9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24ba8be81384d209 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0559a640 1 -- 172.21.15.32:0/3631105749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda0000b2d0 con 0x7fda080b0d00 2023-12-20T04:43:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0559a640 1 -- 172.21.15.32:0/3631105749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda000a7620 con 0x7fda080b0d00 2023-12-20T04:43:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0559a640 1 -- 172.21.15.32:0/3631105749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda000a79b0 con 0x7fda080b0d00 2023-12-20T04:43:58.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0c9e9640 1 -- 172.21.15.32:0/3631105749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda080b0d00 msgr2=0x7fda08164a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0c9e9640 1 --2- 172.21.15.32:0/3631105749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda080b0d00 0x7fda08164a70 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7fda0000bfa0 tx=0x7fda0001e9e0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0c9e9640 1 -- 172.21.15.32:0/3631105749 shutdown_connections 2023-12-20T04:43:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0c9e9640 1 --2- 172.21.15.32:0/3631105749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda080b0d00 0x7fda08164a70 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0c9e9640 1 -- 172.21.15.32:0/3631105749 >> 172.21.15.32:0/3631105749 conn(0x7fda0806a320 msgr2=0x7fda080ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0c9e9640 1 -- 172.21.15.32:0/3631105749 shutdown_connections 2023-12-20T04:43:58.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0c9e9640 1 -- 172.21.15.32:0/3631105749 wait complete. 2023-12-20T04:43:58.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0c9e9640 1 Processor -- start 2023-12-20T04:43:58.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.453+0000 7fda0c9e9640 1 -- start start 2023-12-20T04:43:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda0c9e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fda080aece0 0x7fda08155f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda0c9e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080b0d00 0x7fda080c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda0c9e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda081599e0 0x7fda080c82e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda0c9e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda08155bc0 con 0x7fda081599e0 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda0c9e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fda0815ab00 con 0x7fda080b0d00 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda0c9e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fda0815ad50 con 0x7fda080aece0 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda06d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda081599e0 0x7fda080c82e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda06d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda081599e0 0x7fda080c82e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46446/0 (socket says 172.21.15.32:46446) 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda06d9d640 1 -- 172.21.15.32:0/280389395 learned_addr learned my addr 172.21.15.32:0/280389395 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda05d9b640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080b0d00 0x7fda080c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda0659c640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fda080aece0 0x7fda08155f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda06d9d640 1 -- 172.21.15.32:0/280389395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fda080aece0 msgr2=0x7fda08155f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda06d9d640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fda080aece0 0x7fda08155f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda06d9d640 1 -- 172.21.15.32:0/280389395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080b0d00 msgr2=0x7fda080c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda06d9d640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080b0d00 0x7fda080c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda06d9d640 1 -- 172.21.15.32:0/280389395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda0000bc80 con 0x7fda081599e0 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda0659c640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fda080aece0 0x7fda08155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:43:58.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda06d9d640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda081599e0 0x7fda080c82e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fda0001b5f0 tx=0x7fda0001b780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fd9f6ffd640 1 -- 172.21.15.32:0/280389395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00009540 con 0x7fda081599e0 2023-12-20T04:43:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fd9f6ffd640 1 -- 172.21.15.32:0/280389395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda000096c0 con 0x7fda081599e0 2023-12-20T04:43:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fd9f6ffd640 1 -- 172.21.15.32:0/280389395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda00009a70 con 0x7fda081599e0 2023-12-20T04:43:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda0c9e9640 1 -- 172.21.15.32:0/280389395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fda0815b030 con 0x7fda081599e0 2023-12-20T04:43:58.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda0c9e9640 1 -- 172.21.15.32:0/280389395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fda0815b5f0 con 0x7fda081599e0 2023-12-20T04:43:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.457+0000 7fda0c9e9640 1 -- 172.21.15.32:0/280389395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fda0804e000 con 0x7fda081599e0 2023-12-20T04:43:58.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.461+0000 7fd9f6ffd640 1 -- 172.21.15.32:0/280389395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fda00009bd0 con 0x7fda081599e0 2023-12-20T04:43:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.461+0000 7fd9f6ffd640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9e8065af0 0x7fd9e8067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.461+0000 7fd9f6ffd640 1 -- 172.21.15.32:0/280389395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fda0011e680 con 0x7fda081599e0 2023-12-20T04:43:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.461+0000 7fda0659c640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9e8065af0 0x7fd9e8067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.461+0000 7fda0659c640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9e8065af0 0x7fd9e8067fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8000f00 tx=0x7fd9f8009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:58.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.465+0000 7fd9f6ffd640 1 -- 172.21.15.32:0/280389395 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fda000ec100 con 0x7fda081599e0 2023-12-20T04:43:58.538 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:58.537+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:58.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.625+0000 7fda0c9e9640 1 -- 172.21.15.32:0/280389395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fda0804e000 con 0x7fda081599e0 2023-12-20T04:43:58.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fd9f6ffd640 1 -- 172.21.15.32:0/280389395 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) v1 ==== 74+0+16052 (secure 0 0 0) 0x7fda000ef720 con 0x7fda081599e0 2023-12-20T04:43:58.628 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:58.629 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":93,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:43:55.876479+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:52.582628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:53.871341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:43:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 -- 172.21.15.32:0/280389395 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9e8065af0 msgr2=0x7fd9e8067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9e8065af0 0x7fd9e8067fb0 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fd9f8000f00 tx=0x7fd9f8009040 comp rx=0 tx=0).stop 2023-12-20T04:43:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 -- 172.21.15.32:0/280389395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda081599e0 msgr2=0x7fda080c82e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda081599e0 0x7fda080c82e0 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7fda0001b5f0 tx=0x7fda0001b780 comp rx=0 tx=0).stop 2023-12-20T04:43:58.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 -- 172.21.15.32:0/280389395 shutdown_connections 2023-12-20T04:43:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9e8065af0 0x7fd9e8067fb0 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fda081599e0 0x7fda080c82e0 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fda080b0d00 0x7fda080c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 --2- 172.21.15.32:0/280389395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fda080aece0 0x7fda08155f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 -- 172.21.15.32:0/280389395 >> 172.21.15.32:0/280389395 conn(0x7fda0806a320 msgr2=0x7fda080c4b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 -- 172.21.15.32:0/280389395 shutdown_connections 2023-12-20T04:43:58.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.629+0000 7fda0c9e9640 1 -- 172.21.15.32:0/280389395 wait complete. 2023-12-20T04:43:58.650 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T04:43:58.650 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:43:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.797+0000 7fbe1d812640 1 Processor -- start 2023-12-20T04:43:58.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.797+0000 7fbe1d812640 1 -- start start 2023-12-20T04:43:58.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe1d812640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbe18071af0 0x7fbe180c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe1d812640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbe18160b30 0x7fbe18160fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe1d812640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbe180c9a40 con 0x7fbe1815ce60 2023-12-20T04:43:58.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe1d812640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbe180c9180 con 0x7fbe18071af0 2023-12-20T04:43:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe1d812640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe180c93f0 con 0x7fbe18160b30 2023-12-20T04:43:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbe18160b30 0x7fbe18160fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe0ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbe18071af0 0x7fbe180c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe0ffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbe18071af0 0x7fbe180c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:52832/0 (socket says 172.21.15.32:52832) 2023-12-20T04:43:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe0ffff640 1 -- 172.21.15.32:0/2175344578 learned_addr learned my addr 172.21.15.32:0/2175344578 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe0ffff640 1 -- 172.21.15.32:0/2175344578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbe18160b30 msgr2=0x7fbe18160fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe0ffff640 1 --2- 172.21.15.32:0/2175344578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbe18160b30 0x7fbe18160fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe0ffff640 1 -- 172.21.15.32:0/2175344578 >> v1:172.21.15.39:6789/0 conn(0x7fbe1815ce60 legacy=0x7fbe180713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe0ffff640 1 -- 172.21.15.32:0/2175344578 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe180c96d0 con 0x7fbe18071af0 2023-12-20T04:43:58.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.801+0000 7fbe0ffff640 1 --2- 172.21.15.32:0/2175344578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe18071af0 0x7fbe180c9ed0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009870 tx=0x7fbe0001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=51d3ccc689041f69 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:58.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe167fc640 1 -- 172.21.15.32:0/2175344578 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000ab650 con 0x7fbe18071af0 2023-12-20T04:43:58.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe167fc640 1 -- 172.21.15.32:0/2175344578 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe000ab7b0 con 0x7fbe18071af0 2023-12-20T04:43:58.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe167fc640 1 -- 172.21.15.32:0/2175344578 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe000abb40 con 0x7fbe18071af0 2023-12-20T04:43:58.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 -- 172.21.15.32:0/2175344578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe18071af0 msgr2=0x7fbe180c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 --2- 172.21.15.32:0/2175344578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe18071af0 0x7fbe180c9ed0 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7fbe00009870 tx=0x7fbe0001c540 comp rx=0 tx=0).stop 2023-12-20T04:43:58.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 -- 172.21.15.32:0/2175344578 shutdown_connections 2023-12-20T04:43:58.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 --2- 172.21.15.32:0/2175344578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbe18160b30 0x7fbe18160fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 --2- 172.21.15.32:0/2175344578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe18071af0 0x7fbe180c9ed0 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 -- 172.21.15.32:0/2175344578 >> 172.21.15.32:0/2175344578 conn(0x7fbe1806a320 msgr2=0x7fbe180ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:58.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 -- 172.21.15.32:0/2175344578 shutdown_connections 2023-12-20T04:43:58.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 -- 172.21.15.32:0/2175344578 wait complete. 2023-12-20T04:43:58.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 Processor -- start 2023-12-20T04:43:58.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 -- start start 2023-12-20T04:43:58.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbe18071af0 0x7fbe180c83c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe1815ce60 0x7fbe18155910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18160b30 0x7fbe18146a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe18155e50 con 0x7fbe18160b30 2023-12-20T04:43:58.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbe1815e360 con 0x7fbe1815ce60 2023-12-20T04:43:58.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe1d812640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbe18154b50 con 0x7fbe18071af0 2023-12-20T04:43:58.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18160b30 0x7fbe18146a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18160b30 0x7fbe18146a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42634/0 (socket says 172.21.15.32:42634) 2023-12-20T04:43:58.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe177fe640 1 -- 172.21.15.32:0/501862528 learned_addr learned my addr 172.21.15.32:0/501862528 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:58.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe0ffff640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe1815ce60 0x7fbe18155910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.805+0000 7fbe16ffd640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbe18071af0 0x7fbe180c83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe177fe640 1 -- 172.21.15.32:0/501862528 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbe18071af0 msgr2=0x7fbe180c83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe177fe640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbe18071af0 0x7fbe180c83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe177fe640 1 -- 172.21.15.32:0/501862528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe1815ce60 msgr2=0x7fbe18155910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe177fe640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe1815ce60 0x7fbe18155910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe177fe640 1 -- 172.21.15.32:0/501862528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbe000094d0 con 0x7fbe18160b30 2023-12-20T04:43:58.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe0ffff640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe1815ce60 0x7fbe18155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:58.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe177fe640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18160b30 0x7fbe18146a70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000efc0 tx=0x7fbe1000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:58.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe14ff9640 1 -- 172.21.15.32:0/501862528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe10009ba0 con 0x7fbe18160b30 2023-12-20T04:43:58.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe14ff9640 1 -- 172.21.15.32:0/501862528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbe1000f040 con 0x7fbe18160b30 2023-12-20T04:43:58.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe14ff9640 1 -- 172.21.15.32:0/501862528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbe100146c0 con 0x7fbe18160b30 2023-12-20T04:43:58.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe1d812640 1 -- 172.21.15.32:0/501862528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbe18154e30 con 0x7fbe18160b30 2023-12-20T04:43:58.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe1d812640 1 -- 172.21.15.32:0/501862528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbe181553f0 con 0x7fbe18160b30 2023-12-20T04:43:58.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe1d812640 1 -- 172.21.15.32:0/501862528 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbe181555e0 con 0x7fbe18160b30 2023-12-20T04:43:58.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.809+0000 7fbe14ff9640 1 -- 172.21.15.32:0/501862528 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fbe10014820 con 0x7fbe18160b30 2023-12-20T04:43:58.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.813+0000 7fbe14ff9640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdf8065b20 0x7fbdf8067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:58.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.813+0000 7fbe14ff9640 1 -- 172.21.15.32:0/501862528 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fbe10083ba0 con 0x7fbe18160b30 2023-12-20T04:43:58.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.813+0000 7fbe16ffd640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdf8065b20 0x7fbdf8067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:58.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.813+0000 7fbe16ffd640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdf8065b20 0x7fbdf8067fe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbe08000fc0 tx=0x7fbe08009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:58.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.813+0000 7fbe14ff9640 1 -- 172.21.15.32:0/501862528 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbe100515a0 con 0x7fbe18160b30 2023-12-20T04:43:58.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.981+0000 7fbe1d812640 1 -- 172.21.15.32:0/501862528 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbe180b1e60 con 0x7fbdf8065b20 2023-12-20T04:43:58.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.989+0000 7fbe14ff9640 1 -- 172.21.15.32:0/501862528 <== mgr.6642 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288913 (secure 0 0 0) 0x7fbe08001500 con 0x7fbdf8065b20 2023-12-20T04:43:58.993 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:58.995 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:43:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 -- 172.21.15.32:0/501862528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdf8065b20 msgr2=0x7fbdf8067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdf8065b20 0x7fbdf8067fe0 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fbe08000fc0 tx=0x7fbe08009040 comp rx=0 tx=0).stop 2023-12-20T04:43:58.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 -- 172.21.15.32:0/501862528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18160b30 msgr2=0x7fbe18146a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18160b30 0x7fbe18146a70 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fbe1000efc0 tx=0x7fbe1000c490 comp rx=0 tx=0).stop 2023-12-20T04:43:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 -- 172.21.15.32:0/501862528 shutdown_connections 2023-12-20T04:43:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdf8065b20 0x7fbdf8067fe0 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbe18160b30 0x7fbe18146a70 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbe1815ce60 0x7fbe18155910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 --2- 172.21.15.32:0/501862528 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbe18071af0 0x7fbe180c83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 -- 172.21.15.32:0/501862528 >> 172.21.15.32:0/501862528 conn(0x7fbe1806a320 msgr2=0x7fbe1806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 -- 172.21.15.32:0/501862528 shutdown_connections 2023-12-20T04:43:58.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:58.993+0000 7fbe1d812640 1 -- 172.21.15.32:0/501862528 wait complete. 2023-12-20T04:43:59.013 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-20T04:43:58.244994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":532,"num_read_kb":954,"num_write":285,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2191,"ondisk_log_size":2191,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":327312,"kb_used_data":10064,"kb_used_omap":0,"kb_used_meta":317184,"kb_avail":754647408,"statfs":{"total":773094113280,"available":772758945792,"internally_reserved":0,"allocated":10305536,"data_stored":5443068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":324796416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.787280"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884151+0000","last_change":"2023-12-20T04:43:54.909900+0000","last_active":"2023-12-20T04:43:55.884151+0000","last_peered":"2023-12-20T04:43:55.884151+0000","last_clean":"2023-12-20T04:43:55.884151+0000","last_became_active":"2023-12-20T04:43:54.909294+0000","last_became_peered":"2023-12-20T04:43:54.909294+0000","last_unstale":"2023-12-20T04:43:55.884151+0000","last_undegraded":"2023-12-20T04:43:55.884151+0000","last_fullsized":"2023-12-20T04:43:55.884151+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:00:19.171044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020869+0000","last_change":"2023-12-20T04:43:54.912545+0000","last_active":"2023-12-20T04:43:56.020869+0000","last_peered":"2023-12-20T04:43:56.020869+0000","last_clean":"2023-12-20T04:43:56.020869+0000","last_became_active":"2023-12-20T04:43:54.912250+0000","last_became_peered":"2023-12-20T04:43:54.912250+0000","last_unstale":"2023-12-20T04:43:56.020869+0000","last_undegraded":"2023-12-20T04:43:56.020869+0000","last_fullsized":"2023-12-20T04:43:56.020869+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:29.439450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021268+0000","last_change":"2023-12-20T04:43:54.891226+0000","last_active":"2023-12-20T04:43:56.021268+0000","last_peered":"2023-12-20T04:43:56.021268+0000","last_clean":"2023-12-20T04:43:56.021268+0000","last_became_active":"2023-12-20T04:43:54.890963+0000","last_became_peered":"2023-12-20T04:43:54.890963+0000","last_unstale":"2023-12-20T04:43:56.021268+0000","last_undegraded":"2023-12-20T04:43:56.021268+0000","last_fullsized":"2023-12-20T04:43:56.021268+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:25.438870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021038+0000","last_change":"2023-12-20T04:43:54.909066+0000","last_active":"2023-12-20T04:43:56.021038+0000","last_peered":"2023-12-20T04:43:56.021038+0000","last_clean":"2023-12-20T04:43:56.021038+0000","last_became_active":"2023-12-20T04:43:54.908464+0000","last_became_peered":"2023-12-20T04:43:54.908464+0000","last_unstale":"2023-12-20T04:43:56.021038+0000","last_undegraded":"2023-12-20T04:43:56.021038+0000","last_fullsized":"2023-12-20T04:43:56.021038+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:18.456750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021595+0000","last_change":"2023-12-20T04:43:54.895820+0000","last_active":"2023-12-20T04:43:56.021595+0000","last_peered":"2023-12-20T04:43:56.021595+0000","last_clean":"2023-12-20T04:43:56.021595+0000","last_became_active":"2023-12-20T04:43:54.895296+0000","last_became_peered":"2023-12-20T04:43:54.895296+0000","last_unstale":"2023-12-20T04:43:56.021595+0000","last_undegraded":"2023-12-20T04:43:56.021595+0000","last_fullsized":"2023-12-20T04:43:56.021595+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:57:53.149131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020355+0000","last_change":"2023-12-20T04:43:54.907933+0000","last_active":"2023-12-20T04:43:56.020355+0000","last_peered":"2023-12-20T04:43:56.020355+0000","last_clean":"2023-12-20T04:43:56.020355+0000","last_became_active":"2023-12-20T04:43:54.907658+0000","last_became_peered":"2023-12-20T04:43:54.907658+0000","last_unstale":"2023-12-20T04:43:56.020355+0000","last_undegraded":"2023-12-20T04:43:56.020355+0000","last_fullsized":"2023-12-20T04:43:56.020355+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:18.456750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881782+0000","last_change":"2023-12-20T04:43:54.895250+0000","last_active":"2023-12-20T04:43:55.881782+0000","last_peered":"2023-12-20T04:43:55.881782+0000","last_clean":"2023-12-20T04:43:55.881782+0000","last_became_active":"2023-12-20T04:43:54.894440+0000","last_became_peered":"2023-12-20T04:43:54.894440+0000","last_unstale":"2023-12-20T04:43:55.881782+0000","last_undegraded":"2023-12-20T04:43:55.881782+0000","last_fullsized":"2023-12-20T04:43:55.881782+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:19.780285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020357+0000","last_change":"2023-12-20T04:43:54.905941+0000","last_active":"2023-12-20T04:43:56.020357+0000","last_peered":"2023-12-20T04:43:56.020357+0000","last_clean":"2023-12-20T04:43:56.020357+0000","last_became_active":"2023-12-20T04:43:54.905487+0000","last_became_peered":"2023-12-20T04:43:54.905487+0000","last_unstale":"2023-12-20T04:43:56.020357+0000","last_undegraded":"2023-12-20T04:43:56.020357+0000","last_fullsized":"2023-12-20T04:43:56.020357+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:50:46.140658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021850+0000","last_change":"2023-12-20T04:43:54.899667+0000","last_active":"2023-12-20T04:43:56.021850+0000","last_peered":"2023-12-20T04:43:56.021850+0000","last_clean":"2023-12-20T04:43:56.021850+0000","last_became_active":"2023-12-20T04:43:54.899068+0000","last_became_peered":"2023-12-20T04:43:54.899068+0000","last_unstale":"2023-12-20T04:43:56.021850+0000","last_undegraded":"2023-12-20T04:43:56.021850+0000","last_fullsized":"2023-12-20T04:43:56.021850+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:50:48.904067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883980+0000","last_change":"2023-12-20T04:43:54.903659+0000","last_active":"2023-12-20T04:43:55.883980+0000","last_peered":"2023-12-20T04:43:55.883980+0000","last_clean":"2023-12-20T04:43:55.883980+0000","last_became_active":"2023-12-20T04:43:54.903372+0000","last_became_peered":"2023-12-20T04:43:54.903372+0000","last_unstale":"2023-12-20T04:43:55.883980+0000","last_undegraded":"2023-12-20T04:43:55.883980+0000","last_fullsized":"2023-12-20T04:43:55.883980+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:55.171922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884248+0000","last_change":"2023-12-20T04:43:54.901440+0000","last_active":"2023-12-20T04:43:55.884248+0000","last_peered":"2023-12-20T04:43:55.884248+0000","last_clean":"2023-12-20T04:43:55.884248+0000","last_became_active":"2023-12-20T04:43:54.900816+0000","last_became_peered":"2023-12-20T04:43:54.900816+0000","last_unstale":"2023-12-20T04:43:55.884248+0000","last_undegraded":"2023-12-20T04:43:55.884248+0000","last_fullsized":"2023-12-20T04:43:55.884248+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:28:37.106377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020635+0000","last_change":"2023-12-20T04:43:54.905361+0000","last_active":"2023-12-20T04:43:56.020635+0000","last_peered":"2023-12-20T04:43:56.020635+0000","last_clean":"2023-12-20T04:43:56.020635+0000","last_became_active":"2023-12-20T04:43:54.904897+0000","last_became_peered":"2023-12-20T04:43:54.904897+0000","last_unstale":"2023-12-20T04:43:56.020635+0000","last_undegraded":"2023-12-20T04:43:56.020635+0000","last_fullsized":"2023-12-20T04:43:56.020635+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:04:39.085502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880696+0000","last_change":"2023-12-20T04:43:54.909627+0000","last_active":"2023-12-20T04:43:55.880696+0000","last_peered":"2023-12-20T04:43:55.880696+0000","last_clean":"2023-12-20T04:43:55.880696+0000","last_became_active":"2023-12-20T04:43:54.909343+0000","last_became_peered":"2023-12-20T04:43:54.909343+0000","last_unstale":"2023-12-20T04:43:55.880696+0000","last_undegraded":"2023-12-20T04:43:55.880696+0000","last_fullsized":"2023-12-20T04:43:55.880696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:39:51.562870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021584+0000","last_change":"2023-12-20T04:43:54.903889+0000","last_active":"2023-12-20T04:43:56.021584+0000","last_peered":"2023-12-20T04:43:56.021584+0000","last_clean":"2023-12-20T04:43:56.021584+0000","last_became_active":"2023-12-20T04:43:54.901072+0000","last_became_peered":"2023-12-20T04:43:54.901072+0000","last_unstale":"2023-12-20T04:43:56.021584+0000","last_undegraded":"2023-12-20T04:43:56.021584+0000","last_fullsized":"2023-12-20T04:43:56.021584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:25:23.494360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880599+0000","last_change":"2023-12-20T04:43:54.895682+0000","last_active":"2023-12-20T04:43:55.880599+0000","last_peered":"2023-12-20T04:43:55.880599+0000","last_clean":"2023-12-20T04:43:55.880599+0000","last_became_active":"2023-12-20T04:43:54.895394+0000","last_became_peered":"2023-12-20T04:43:54.895394+0000","last_unstale":"2023-12-20T04:43:55.880599+0000","last_undegraded":"2023-12-20T04:43:55.880599+0000","last_fullsized":"2023-12-20T04:43:55.880599+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:20.099541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882912+0000","last_change":"2023-12-20T04:43:54.909324+0000","last_active":"2023-12-20T04:43:55.882912+0000","last_peered":"2023-12-20T04:43:55.882912+0000","last_clean":"2023-12-20T04:43:55.882912+0000","last_became_active":"2023-12-20T04:43:54.909009+0000","last_became_peered":"2023-12-20T04:43:54.909009+0000","last_unstale":"2023-12-20T04:43:55.882912+0000","last_undegraded":"2023-12-20T04:43:55.882912+0000","last_fullsized":"2023-12-20T04:43:55.882912+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:28:37.106377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021891+0000","last_change":"2023-12-20T04:43:54.899500+0000","last_active":"2023-12-20T04:43:56.021891+0000","last_peered":"2023-12-20T04:43:56.021891+0000","last_clean":"2023-12-20T04:43:56.021891+0000","last_became_active":"2023-12-20T04:43:54.898791+0000","last_became_peered":"2023-12-20T04:43:54.898791+0000","last_unstale":"2023-12-20T04:43:56.021891+0000","last_undegraded":"2023-12-20T04:43:56.021891+0000","last_fullsized":"2023-12-20T04:43:56.021891+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:43:10.747734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020696+0000","last_change":"2023-12-20T04:43:54.910720+0000","last_active":"2023-12-20T04:43:56.020696+0000","last_peered":"2023-12-20T04:43:56.020696+0000","last_clean":"2023-12-20T04:43:56.020696+0000","last_became_active":"2023-12-20T04:43:54.910443+0000","last_became_peered":"2023-12-20T04:43:54.910443+0000","last_unstale":"2023-12-20T04:43:56.020696+0000","last_undegraded":"2023-12-20T04:43:56.020696+0000","last_fullsized":"2023-12-20T04:43:56.020696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:29.439450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020528+0000","last_change":"2023-12-20T04:43:54.907136+0000","last_active":"2023-12-20T04:43:56.020528+0000","last_peered":"2023-12-20T04:43:56.020528+0000","last_clean":"2023-12-20T04:43:56.020528+0000","last_became_active":"2023-12-20T04:43:54.904857+0000","last_became_peered":"2023-12-20T04:43:54.904857+0000","last_unstale":"2023-12-20T04:43:56.020528+0000","last_undegraded":"2023-12-20T04:43:56.020528+0000","last_fullsized":"2023-12-20T04:43:56.020528+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:33:45.863254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880712+0000","last_change":"2023-12-20T04:43:54.897121+0000","last_active":"2023-12-20T04:43:55.880712+0000","last_peered":"2023-12-20T04:43:55.880712+0000","last_clean":"2023-12-20T04:43:55.880712+0000","last_became_active":"2023-12-20T04:43:54.896862+0000","last_became_peered":"2023-12-20T04:43:54.896862+0000","last_unstale":"2023-12-20T04:43:55.880712+0000","last_undegraded":"2023-12-20T04:43:55.880712+0000","last_fullsized":"2023-12-20T04:43:55.880712+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:54.224938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883413+0000","last_change":"2023-12-20T04:43:54.897408+0000","last_active":"2023-12-20T04:43:55.883413+0000","last_peered":"2023-12-20T04:43:55.883413+0000","last_clean":"2023-12-20T04:43:55.883413+0000","last_became_active":"2023-12-20T04:43:54.897062+0000","last_became_peered":"2023-12-20T04:43:54.897062+0000","last_unstale":"2023-12-20T04:43:55.883413+0000","last_undegraded":"2023-12-20T04:43:55.883413+0000","last_fullsized":"2023-12-20T04:43:55.883413+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:53.729698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883049+0000","last_change":"2023-12-20T04:43:54.903562+0000","last_active":"2023-12-20T04:43:55.883049+0000","last_peered":"2023-12-20T04:43:55.883049+0000","last_clean":"2023-12-20T04:43:55.883049+0000","last_became_active":"2023-12-20T04:43:54.903279+0000","last_became_peered":"2023-12-20T04:43:54.903279+0000","last_unstale":"2023-12-20T04:43:55.883049+0000","last_undegraded":"2023-12-20T04:43:55.883049+0000","last_fullsized":"2023-12-20T04:43:55.883049+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:56:23.814101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883854+0000","last_change":"2023-12-20T04:43:54.899617+0000","last_active":"2023-12-20T04:43:55.883854+0000","last_peered":"2023-12-20T04:43:55.883854+0000","last_clean":"2023-12-20T04:43:55.883854+0000","last_became_active":"2023-12-20T04:43:54.898947+0000","last_became_peered":"2023-12-20T04:43:54.898947+0000","last_unstale":"2023-12-20T04:43:55.883854+0000","last_undegraded":"2023-12-20T04:43:55.883854+0000","last_fullsized":"2023-12-20T04:43:55.883854+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:28.512003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883378+0000","last_change":"2023-12-20T04:43:54.913205+0000","last_active":"2023-12-20T04:43:55.883378+0000","last_peered":"2023-12-20T04:43:55.883378+0000","last_clean":"2023-12-20T04:43:55.883378+0000","last_became_active":"2023-12-20T04:43:54.912945+0000","last_became_peered":"2023-12-20T04:43:54.912945+0000","last_unstale":"2023-12-20T04:43:55.883378+0000","last_undegraded":"2023-12-20T04:43:55.883378+0000","last_fullsized":"2023-12-20T04:43:55.883378+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:29:39.148170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883744+0000","last_change":"2023-12-20T04:43:54.899556+0000","last_active":"2023-12-20T04:43:55.883744+0000","last_peered":"2023-12-20T04:43:55.883744+0000","last_clean":"2023-12-20T04:43:55.883744+0000","last_became_active":"2023-12-20T04:43:54.898140+0000","last_became_peered":"2023-12-20T04:43:54.898140+0000","last_unstale":"2023-12-20T04:43:55.883744+0000","last_undegraded":"2023-12-20T04:43:55.883744+0000","last_fullsized":"2023-12-20T04:43:55.883744+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:51:48.832259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882708+0000","last_change":"2023-12-20T04:43:54.914449+0000","last_active":"2023-12-20T04:43:55.882708+0000","last_peered":"2023-12-20T04:43:55.882708+0000","last_clean":"2023-12-20T04:43:55.882708+0000","last_became_active":"2023-12-20T04:43:54.913169+0000","last_became_peered":"2023-12-20T04:43:54.913169+0000","last_unstale":"2023-12-20T04:43:55.882708+0000","last_undegraded":"2023-12-20T04:43:55.882708+0000","last_fullsized":"2023-12-20T04:43:55.882708+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:40:28.475063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021356+0000","last_change":"2023-12-20T04:43:54.896212+0000","last_active":"2023-12-20T04:43:56.021356+0000","last_peered":"2023-12-20T04:43:56.021356+0000","last_clean":"2023-12-20T04:43:56.021356+0000","last_became_active":"2023-12-20T04:43:54.895973+0000","last_became_peered":"2023-12-20T04:43:54.895973+0000","last_unstale":"2023-12-20T04:43:56.021356+0000","last_undegraded":"2023-12-20T04:43:56.021356+0000","last_fullsized":"2023-12-20T04:43:56.021356+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:14:56.700329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883585+0000","last_change":"2023-12-20T04:43:54.904989+0000","last_active":"2023-12-20T04:43:55.883585+0000","last_peered":"2023-12-20T04:43:55.883585+0000","last_clean":"2023-12-20T04:43:55.883585+0000","last_became_active":"2023-12-20T04:43:54.904675+0000","last_became_peered":"2023-12-20T04:43:54.904675+0000","last_unstale":"2023-12-20T04:43:55.883585+0000","last_undegraded":"2023-12-20T04:43:55.883585+0000","last_fullsized":"2023-12-20T04:43:55.883585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:52.478483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020915+0000","last_change":"2023-12-20T04:43:54.889243+0000","last_active":"2023-12-20T04:43:56.020915+0000","last_peered":"2023-12-20T04:43:56.020915+0000","last_clean":"2023-12-20T04:43:56.020915+0000","last_became_active":"2023-12-20T04:43:54.888983+0000","last_became_peered":"2023-12-20T04:43:54.888983+0000","last_unstale":"2023-12-20T04:43:56.020915+0000","last_undegraded":"2023-12-20T04:43:56.020915+0000","last_fullsized":"2023-12-20T04:43:56.020915+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:36:39.445465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881847+0000","last_change":"2023-12-20T04:43:54.913726+0000","last_active":"2023-12-20T04:43:55.881847+0000","last_peered":"2023-12-20T04:43:55.881847+0000","last_clean":"2023-12-20T04:43:55.881847+0000","last_became_active":"2023-12-20T04:43:54.913471+0000","last_became_peered":"2023-12-20T04:43:54.913471+0000","last_unstale":"2023-12-20T04:43:55.881847+0000","last_undegraded":"2023-12-20T04:43:55.881847+0000","last_fullsized":"2023-12-20T04:43:55.881847+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:33:34.511010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881954+0000","last_change":"2023-12-20T04:43:54.904055+0000","last_active":"2023-12-20T04:43:55.881954+0000","last_peered":"2023-12-20T04:43:55.881954+0000","last_clean":"2023-12-20T04:43:55.881954+0000","last_became_active":"2023-12-20T04:43:54.903791+0000","last_became_peered":"2023-12-20T04:43:54.903791+0000","last_unstale":"2023-12-20T04:43:55.881954+0000","last_undegraded":"2023-12-20T04:43:55.881954+0000","last_fullsized":"2023-12-20T04:43:55.881954+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:19:35.337100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020359+0000","last_change":"2023-12-20T04:43:54.911172+0000","last_active":"2023-12-20T04:43:56.020359+0000","last_peered":"2023-12-20T04:43:56.020359+0000","last_clean":"2023-12-20T04:43:56.020359+0000","last_became_active":"2023-12-20T04:43:54.910873+0000","last_became_peered":"2023-12-20T04:43:54.910873+0000","last_unstale":"2023-12-20T04:43:56.020359+0000","last_undegraded":"2023-12-20T04:43:56.020359+0000","last_fullsized":"2023-12-20T04:43:56.020359+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:07:55.899832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884186+0000","last_change":"2023-12-20T04:43:54.914900+0000","last_active":"2023-12-20T04:43:55.884186+0000","last_peered":"2023-12-20T04:43:55.884186+0000","last_clean":"2023-12-20T04:43:55.884186+0000","last_became_active":"2023-12-20T04:43:54.913650+0000","last_became_peered":"2023-12-20T04:43:54.913650+0000","last_unstale":"2023-12-20T04:43:55.884186+0000","last_undegraded":"2023-12-20T04:43:55.884186+0000","last_fullsized":"2023-12-20T04:43:55.884186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:56.046081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"92'4","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884370+0000","last_change":"2023-12-20T04:43:54.486574+0000","last_active":"2023-12-20T04:43:55.884370+0000","last_peered":"2023-12-20T04:43:55.884370+0000","last_clean":"2023-12-20T04:43:55.884370+0000","last_became_active":"2023-12-20T04:43:54.486282+0000","last_became_peered":"2023-12-20T04:43:54.486282+0000","last_unstale":"2023-12-20T04:43:55.884370+0000","last_undegraded":"2023-12-20T04:43:55.884370+0000","last_fullsized":"2023-12-20T04:43:55.884370+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:07:02.481763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021925+0000","last_change":"2023-12-20T04:43:54.899335+0000","last_active":"2023-12-20T04:43:56.021925+0000","last_peered":"2023-12-20T04:43:56.021925+0000","last_clean":"2023-12-20T04:43:56.021925+0000","last_became_active":"2023-12-20T04:43:54.898507+0000","last_became_peered":"2023-12-20T04:43:54.898507+0000","last_unstale":"2023-12-20T04:43:56.021925+0000","last_undegraded":"2023-12-20T04:43:56.021925+0000","last_fullsized":"2023-12-20T04:43:56.021925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:47.883793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881043+0000","last_change":"2023-12-20T04:43:54.487030+0000","last_active":"2023-12-20T04:43:55.881043+0000","last_peered":"2023-12-20T04:43:55.881043+0000","last_clean":"2023-12-20T04:43:55.881043+0000","last_became_active":"2023-12-20T04:43:54.486749+0000","last_became_peered":"2023-12-20T04:43:54.486749+0000","last_unstale":"2023-12-20T04:43:55.881043+0000","last_undegraded":"2023-12-20T04:43:55.881043+0000","last_fullsized":"2023-12-20T04:43:55.881043+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:51.470068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020544+0000","last_change":"2023-12-20T04:43:54.909644+0000","last_active":"2023-12-20T04:43:56.020544+0000","last_peered":"2023-12-20T04:43:56.020544+0000","last_clean":"2023-12-20T04:43:56.020544+0000","last_became_active":"2023-12-20T04:43:54.909376+0000","last_became_peered":"2023-12-20T04:43:54.909376+0000","last_unstale":"2023-12-20T04:43:56.020544+0000","last_undegraded":"2023-12-20T04:43:56.020544+0000","last_fullsized":"2023-12-20T04:43:56.020544+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:53:32.413782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883879+0000","last_change":"2023-12-20T04:43:54.497844+0000","last_active":"2023-12-20T04:43:55.883879+0000","last_peered":"2023-12-20T04:43:55.883879+0000","last_clean":"2023-12-20T04:43:55.883879+0000","last_became_active":"2023-12-20T04:43:54.497548+0000","last_became_peered":"2023-12-20T04:43:54.497548+0000","last_unstale":"2023-12-20T04:43:55.883879+0000","last_undegraded":"2023-12-20T04:43:55.883879+0000","last_fullsized":"2023-12-20T04:43:55.883879+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:04:41.515933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884267+0000","last_change":"2023-12-20T04:43:54.901619+0000","last_active":"2023-12-20T04:43:55.884267+0000","last_peered":"2023-12-20T04:43:55.884267+0000","last_clean":"2023-12-20T04:43:55.884267+0000","last_became_active":"2023-12-20T04:43:54.901144+0000","last_became_peered":"2023-12-20T04:43:54.901144+0000","last_unstale":"2023-12-20T04:43:55.884267+0000","last_undegraded":"2023-12-20T04:43:55.884267+0000","last_fullsized":"2023-12-20T04:43:55.884267+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:51:48.832259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020640+0000","last_change":"2023-12-20T04:43:54.496861+0000","last_active":"2023-12-20T04:43:56.020640+0000","last_peered":"2023-12-20T04:43:56.020640+0000","last_clean":"2023-12-20T04:43:56.020640+0000","last_became_active":"2023-12-20T04:43:54.496464+0000","last_became_peered":"2023-12-20T04:43:54.496464+0000","last_unstale":"2023-12-20T04:43:56.020640+0000","last_undegraded":"2023-12-20T04:43:56.020640+0000","last_fullsized":"2023-12-20T04:43:56.020640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:06:06.325087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881940+0000","last_change":"2023-12-20T04:43:54.896052+0000","last_active":"2023-12-20T04:43:55.881940+0000","last_peered":"2023-12-20T04:43:55.881940+0000","last_clean":"2023-12-20T04:43:55.881940+0000","last_became_active":"2023-12-20T04:43:54.895763+0000","last_became_peered":"2023-12-20T04:43:54.895763+0000","last_unstale":"2023-12-20T04:43:55.881940+0000","last_undegraded":"2023-12-20T04:43:55.881940+0000","last_fullsized":"2023-12-20T04:43:55.881940+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:07:03.490178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020984+0000","last_change":"2023-12-20T04:43:54.494013+0000","last_active":"2023-12-20T04:43:56.020984+0000","last_peered":"2023-12-20T04:43:56.020984+0000","last_clean":"2023-12-20T04:43:56.020984+0000","last_became_active":"2023-12-20T04:43:54.493745+0000","last_became_peered":"2023-12-20T04:43:54.493745+0000","last_unstale":"2023-12-20T04:43:56.020984+0000","last_undegraded":"2023-12-20T04:43:56.020984+0000","last_fullsized":"2023-12-20T04:43:56.020984+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:53.216522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883545+0000","last_change":"2023-12-20T04:43:54.900003+0000","last_active":"2023-12-20T04:43:55.883545+0000","last_peered":"2023-12-20T04:43:55.883545+0000","last_clean":"2023-12-20T04:43:55.883545+0000","last_became_active":"2023-12-20T04:43:54.897336+0000","last_became_peered":"2023-12-20T04:43:54.897336+0000","last_unstale":"2023-12-20T04:43:55.883545+0000","last_undegraded":"2023-12-20T04:43:55.883545+0000","last_fullsized":"2023-12-20T04:43:55.883545+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:01.626753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020738+0000","last_change":"2023-12-20T04:43:54.497282+0000","last_active":"2023-12-20T04:43:56.020738+0000","last_peered":"2023-12-20T04:43:56.020738+0000","last_clean":"2023-12-20T04:43:56.020738+0000","last_became_active":"2023-12-20T04:43:54.496962+0000","last_became_peered":"2023-12-20T04:43:54.496962+0000","last_unstale":"2023-12-20T04:43:56.020738+0000","last_undegraded":"2023-12-20T04:43:56.020738+0000","last_fullsized":"2023-12-20T04:43:56.020738+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:25:52.397612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021050+0000","last_change":"2023-12-20T04:43:54.907240+0000","last_active":"2023-12-20T04:43:56.021050+0000","last_peered":"2023-12-20T04:43:56.021050+0000","last_clean":"2023-12-20T04:43:56.021050+0000","last_became_active":"2023-12-20T04:43:54.906932+0000","last_became_peered":"2023-12-20T04:43:54.906932+0000","last_unstale":"2023-12-20T04:43:56.021050+0000","last_undegraded":"2023-12-20T04:43:56.021050+0000","last_fullsized":"2023-12-20T04:43:56.021050+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:11:09.188333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020288+0000","last_change":"2023-12-20T04:43:54.492236+0000","last_active":"2023-12-20T04:43:56.020288+0000","last_peered":"2023-12-20T04:43:56.020288+0000","last_clean":"2023-12-20T04:43:56.020288+0000","last_became_active":"2023-12-20T04:43:54.491979+0000","last_became_peered":"2023-12-20T04:43:54.491979+0000","last_unstale":"2023-12-20T04:43:56.020288+0000","last_undegraded":"2023-12-20T04:43:56.020288+0000","last_fullsized":"2023-12-20T04:43:56.020288+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:43:09.739319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880694+0000","last_change":"2023-12-20T04:43:54.897155+0000","last_active":"2023-12-20T04:43:55.880694+0000","last_peered":"2023-12-20T04:43:55.880694+0000","last_clean":"2023-12-20T04:43:55.880694+0000","last_became_active":"2023-12-20T04:43:54.896907+0000","last_became_peered":"2023-12-20T04:43:54.896907+0000","last_unstale":"2023-12-20T04:43:55.880694+0000","last_undegraded":"2023-12-20T04:43:55.880694+0000","last_fullsized":"2023-12-20T04:43:55.880694+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:36.350327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021143+0000","last_change":"2023-12-20T04:43:54.484385+0000","last_active":"2023-12-20T04:43:56.021143+0000","last_peered":"2023-12-20T04:43:56.021143+0000","last_clean":"2023-12-20T04:43:56.021143+0000","last_became_active":"2023-12-20T04:43:54.484048+0000","last_became_peered":"2023-12-20T04:43:54.484048+0000","last_unstale":"2023-12-20T04:43:56.021143+0000","last_undegraded":"2023-12-20T04:43:56.021143+0000","last_fullsized":"2023-12-20T04:43:56.021143+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:50.474860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882661+0000","last_change":"2023-12-20T04:43:54.896351+0000","last_active":"2023-12-20T04:43:55.882661+0000","last_peered":"2023-12-20T04:43:55.882661+0000","last_clean":"2023-12-20T04:43:55.882661+0000","last_became_active":"2023-12-20T04:43:54.895371+0000","last_became_peered":"2023-12-20T04:43:54.895371+0000","last_unstale":"2023-12-20T04:43:55.882661+0000","last_undegraded":"2023-12-20T04:43:55.882661+0000","last_fullsized":"2023-12-20T04:43:55.882661+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:11:13.259566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883861+0000","last_change":"2023-12-20T04:43:54.490790+0000","last_active":"2023-12-20T04:43:55.883861+0000","last_peered":"2023-12-20T04:43:55.883861+0000","last_clean":"2023-12-20T04:43:55.883861+0000","last_became_active":"2023-12-20T04:43:54.490501+0000","last_became_peered":"2023-12-20T04:43:54.490501+0000","last_unstale":"2023-12-20T04:43:55.883861+0000","last_undegraded":"2023-12-20T04:43:55.883861+0000","last_fullsized":"2023-12-20T04:43:55.883861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:37:50.839793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884257+0000","last_change":"2023-12-20T04:43:54.912960+0000","last_active":"2023-12-20T04:43:55.884257+0000","last_peered":"2023-12-20T04:43:55.884257+0000","last_clean":"2023-12-20T04:43:55.884257+0000","last_became_active":"2023-12-20T04:43:54.912082+0000","last_became_peered":"2023-12-20T04:43:54.912082+0000","last_unstale":"2023-12-20T04:43:55.884257+0000","last_undegraded":"2023-12-20T04:43:55.884257+0000","last_fullsized":"2023-12-20T04:43:55.884257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:36.350327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020876+0000","last_change":"2023-12-20T04:43:54.496566+0000","last_active":"2023-12-20T04:43:56.020876+0000","last_peered":"2023-12-20T04:43:56.020876+0000","last_clean":"2023-12-20T04:43:56.020876+0000","last_became_active":"2023-12-20T04:43:54.496205+0000","last_became_peered":"2023-12-20T04:43:54.496205+0000","last_unstale":"2023-12-20T04:43:56.020876+0000","last_undegraded":"2023-12-20T04:43:56.020876+0000","last_fullsized":"2023-12-20T04:43:56.020876+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:40.627636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020586+0000","last_change":"2023-12-20T04:43:54.911648+0000","last_active":"2023-12-20T04:43:56.020586+0000","last_peered":"2023-12-20T04:43:56.020586+0000","last_clean":"2023-12-20T04:43:56.020586+0000","last_became_active":"2023-12-20T04:43:54.911277+0000","last_became_peered":"2023-12-20T04:43:54.911277+0000","last_unstale":"2023-12-20T04:43:56.020586+0000","last_undegraded":"2023-12-20T04:43:56.020586+0000","last_fullsized":"2023-12-20T04:43:56.020586+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:04:39.085502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"92'3","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021381+0000","last_change":"2023-12-20T04:43:54.493272+0000","last_active":"2023-12-20T04:43:56.021381+0000","last_peered":"2023-12-20T04:43:56.021381+0000","last_clean":"2023-12-20T04:43:56.021381+0000","last_became_active":"2023-12-20T04:43:54.492958+0000","last_became_peered":"2023-12-20T04:43:54.492958+0000","last_unstale":"2023-12-20T04:43:56.021381+0000","last_undegraded":"2023-12-20T04:43:56.021381+0000","last_fullsized":"2023-12-20T04:43:56.021381+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:39:50.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021657+0000","last_change":"2023-12-20T04:43:54.899165+0000","last_active":"2023-12-20T04:43:56.021657+0000","last_peered":"2023-12-20T04:43:56.021657+0000","last_clean":"2023-12-20T04:43:56.021657+0000","last_became_active":"2023-12-20T04:43:54.898895+0000","last_became_peered":"2023-12-20T04:43:54.898895+0000","last_unstale":"2023-12-20T04:43:56.021657+0000","last_undegraded":"2023-12-20T04:43:56.021657+0000","last_fullsized":"2023-12-20T04:43:56.021657+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:25:53.406028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"92'6","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020828+0000","last_change":"2023-12-20T04:43:54.487182+0000","last_active":"2023-12-20T04:43:56.020828+0000","last_peered":"2023-12-20T04:43:56.020828+0000","last_clean":"2023-12-20T04:43:56.020828+0000","last_became_active":"2023-12-20T04:43:54.486886+0000","last_became_peered":"2023-12-20T04:43:54.486886+0000","last_unstale":"2023-12-20T04:43:56.020828+0000","last_undegraded":"2023-12-20T04:43:56.020828+0000","last_fullsized":"2023-12-20T04:43:56.020828+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:25:22.485944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883843+0000","last_change":"2023-12-20T04:43:54.902851+0000","last_active":"2023-12-20T04:43:55.883843+0000","last_peered":"2023-12-20T04:43:55.883843+0000","last_clean":"2023-12-20T04:43:55.883843+0000","last_became_active":"2023-12-20T04:43:54.902569+0000","last_became_peered":"2023-12-20T04:43:54.902569+0000","last_unstale":"2023-12-20T04:43:55.883843+0000","last_undegraded":"2023-12-20T04:43:55.883843+0000","last_fullsized":"2023-12-20T04:43:55.883843+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:56:23.814101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020539+0000","last_change":"2023-12-20T04:43:54.483144+0000","last_active":"2023-12-20T04:43:56.020539+0000","last_peered":"2023-12-20T04:43:56.020539+0000","last_clean":"2023-12-20T04:43:56.020539+0000","last_became_active":"2023-12-20T04:43:54.482862+0000","last_became_peered":"2023-12-20T04:43:54.482862+0000","last_unstale":"2023-12-20T04:43:56.020539+0000","last_undegraded":"2023-12-20T04:43:56.020539+0000","last_fullsized":"2023-12-20T04:43:56.020539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:46.875378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882291+0000","last_change":"2023-12-20T04:43:54.899424+0000","last_active":"2023-12-20T04:43:55.882291+0000","last_peered":"2023-12-20T04:43:55.882291+0000","last_clean":"2023-12-20T04:43:55.882291+0000","last_became_active":"2023-12-20T04:43:54.898559+0000","last_became_peered":"2023-12-20T04:43:54.898559+0000","last_unstale":"2023-12-20T04:43:55.882291+0000","last_undegraded":"2023-12-20T04:43:55.882291+0000","last_fullsized":"2023-12-20T04:43:55.882291+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:26:29.783539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020829+0000","last_change":"2023-12-20T04:43:54.497321+0000","last_active":"2023-12-20T04:43:56.020829+0000","last_peered":"2023-12-20T04:43:56.020829+0000","last_clean":"2023-12-20T04:43:56.020829+0000","last_became_active":"2023-12-20T04:43:54.497051+0000","last_became_peered":"2023-12-20T04:43:54.497051+0000","last_unstale":"2023-12-20T04:43:56.020829+0000","last_undegraded":"2023-12-20T04:43:56.020829+0000","last_fullsized":"2023-12-20T04:43:56.020829+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:14.415022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880546+0000","last_change":"2023-12-20T04:43:54.906651+0000","last_active":"2023-12-20T04:43:55.880546+0000","last_peered":"2023-12-20T04:43:55.880546+0000","last_clean":"2023-12-20T04:43:55.880546+0000","last_became_active":"2023-12-20T04:43:54.906396+0000","last_became_peered":"2023-12-20T04:43:54.906396+0000","last_unstale":"2023-12-20T04:43:55.880546+0000","last_undegraded":"2023-12-20T04:43:55.880546+0000","last_fullsized":"2023-12-20T04:43:55.880546+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:20.532746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881860+0000","last_change":"2023-12-20T04:43:54.490358+0000","last_active":"2023-12-20T04:43:55.881860+0000","last_peered":"2023-12-20T04:43:55.881860+0000","last_clean":"2023-12-20T04:43:55.881860+0000","last_became_active":"2023-12-20T04:43:54.489860+0000","last_became_peered":"2023-12-20T04:43:54.489860+0000","last_unstale":"2023-12-20T04:43:55.881860+0000","last_undegraded":"2023-12-20T04:43:55.881860+0000","last_fullsized":"2023-12-20T04:43:55.881860+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:50:45.132243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021410+0000","last_change":"2023-12-20T04:43:54.895379+0000","last_active":"2023-12-20T04:43:56.021410+0000","last_peered":"2023-12-20T04:43:56.021410+0000","last_clean":"2023-12-20T04:43:56.021410+0000","last_became_active":"2023-12-20T04:43:54.895092+0000","last_became_peered":"2023-12-20T04:43:54.895092+0000","last_unstale":"2023-12-20T04:43:56.021410+0000","last_undegraded":"2023-12-20T04:43:56.021410+0000","last_fullsized":"2023-12-20T04:43:56.021410+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:55.403045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"92'8","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882904+0000","last_change":"2023-12-20T04:43:54.490638+0000","last_active":"2023-12-20T04:43:55.882904+0000","last_peered":"2023-12-20T04:43:55.882904+0000","last_clean":"2023-12-20T04:43:55.882904+0000","last_became_active":"2023-12-20T04:43:54.490095+0000","last_became_peered":"2023-12-20T04:43:54.490095+0000","last_unstale":"2023-12-20T04:43:55.882904+0000","last_undegraded":"2023-12-20T04:43:55.882904+0000","last_fullsized":"2023-12-20T04:43:55.882904+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:33:24.912541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883989+0000","last_change":"2023-12-20T04:43:54.891326+0000","last_active":"2023-12-20T04:43:55.883989+0000","last_peered":"2023-12-20T04:43:55.883989+0000","last_clean":"2023-12-20T04:43:55.883989+0000","last_became_active":"2023-12-20T04:43:54.891029+0000","last_became_peered":"2023-12-20T04:43:54.891029+0000","last_unstale":"2023-12-20T04:43:55.883989+0000","last_undegraded":"2023-12-20T04:43:55.883989+0000","last_fullsized":"2023-12-20T04:43:55.883989+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:42:10.876085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020693+0000","last_change":"2023-12-20T04:43:54.496914+0000","last_active":"2023-12-20T04:43:56.020693+0000","last_peered":"2023-12-20T04:43:56.020693+0000","last_clean":"2023-12-20T04:43:56.020693+0000","last_became_active":"2023-12-20T04:43:54.496523+0000","last_became_peered":"2023-12-20T04:43:54.496523+0000","last_unstale":"2023-12-20T04:43:56.020693+0000","last_undegraded":"2023-12-20T04:43:56.020693+0000","last_fullsized":"2023-12-20T04:43:56.020693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:01.919919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884371+0000","last_change":"2023-12-20T04:43:54.899487+0000","last_active":"2023-12-20T04:43:55.884371+0000","last_peered":"2023-12-20T04:43:55.884371+0000","last_clean":"2023-12-20T04:43:55.884371+0000","last_became_active":"2023-12-20T04:43:54.899244+0000","last_became_peered":"2023-12-20T04:43:54.899244+0000","last_unstale":"2023-12-20T04:43:55.884371+0000","last_undegraded":"2023-12-20T04:43:55.884371+0000","last_fullsized":"2023-12-20T04:43:55.884371+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:55.171922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884089+0000","last_change":"2023-12-20T04:43:54.497865+0000","last_active":"2023-12-20T04:43:55.884089+0000","last_peered":"2023-12-20T04:43:55.884089+0000","last_clean":"2023-12-20T04:43:55.884089+0000","last_became_active":"2023-12-20T04:43:54.497558+0000","last_became_peered":"2023-12-20T04:43:54.497558+0000","last_unstale":"2023-12-20T04:43:55.884089+0000","last_undegraded":"2023-12-20T04:43:55.884089+0000","last_fullsized":"2023-12-20T04:43:55.884089+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:58:33.383247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020734+0000","last_change":"2023-12-20T04:43:54.905117+0000","last_active":"2023-12-20T04:43:56.020734+0000","last_peered":"2023-12-20T04:43:56.020734+0000","last_clean":"2023-12-20T04:43:56.020734+0000","last_became_active":"2023-12-20T04:43:54.904809+0000","last_became_peered":"2023-12-20T04:43:54.904809+0000","last_unstale":"2023-12-20T04:43:56.020734+0000","last_undegraded":"2023-12-20T04:43:56.020734+0000","last_fullsized":"2023-12-20T04:43:56.020734+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:07:55.899832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020627+0000","last_change":"2023-12-20T04:43:54.500032+0000","last_active":"2023-12-20T04:43:56.020627+0000","last_peered":"2023-12-20T04:43:56.020627+0000","last_clean":"2023-12-20T04:43:56.020627+0000","last_became_active":"2023-12-20T04:43:54.499661+0000","last_became_peered":"2023-12-20T04:43:54.499661+0000","last_unstale":"2023-12-20T04:43:56.020627+0000","last_undegraded":"2023-12-20T04:43:56.020627+0000","last_fullsized":"2023-12-20T04:43:56.020627+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:00.859060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021785+0000","last_change":"2023-12-20T04:43:54.909151+0000","last_active":"2023-12-20T04:43:56.021785+0000","last_peered":"2023-12-20T04:43:56.021785+0000","last_clean":"2023-12-20T04:43:56.021785+0000","last_became_active":"2023-12-20T04:43:54.908881+0000","last_became_peered":"2023-12-20T04:43:54.908881+0000","last_unstale":"2023-12-20T04:43:56.021785+0000","last_undegraded":"2023-12-20T04:43:56.021785+0000","last_fullsized":"2023-12-20T04:43:56.021785+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:02.928334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884129+0000","last_change":"2023-12-20T04:43:54.492179+0000","last_active":"2023-12-20T04:43:55.884129+0000","last_peered":"2023-12-20T04:43:55.884129+0000","last_clean":"2023-12-20T04:43:55.884129+0000","last_became_active":"2023-12-20T04:43:54.491931+0000","last_became_peered":"2023-12-20T04:43:54.491931+0000","last_unstale":"2023-12-20T04:43:55.884129+0000","last_undegraded":"2023-12-20T04:43:55.884129+0000","last_fullsized":"2023-12-20T04:43:55.884129+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:19:49.788193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020898+0000","last_change":"2023-12-20T04:43:54.893670+0000","last_active":"2023-12-20T04:43:56.020898+0000","last_peered":"2023-12-20T04:43:56.020898+0000","last_clean":"2023-12-20T04:43:56.020898+0000","last_became_active":"2023-12-20T04:43:54.893375+0000","last_became_peered":"2023-12-20T04:43:54.893375+0000","last_unstale":"2023-12-20T04:43:56.020898+0000","last_undegraded":"2023-12-20T04:43:56.020898+0000","last_fullsized":"2023-12-20T04:43:56.020898+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:25:16.646926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020858+0000","last_change":"2023-12-20T04:43:54.496541+0000","last_active":"2023-12-20T04:43:56.020858+0000","last_peered":"2023-12-20T04:43:56.020858+0000","last_clean":"2023-12-20T04:43:56.020858+0000","last_became_active":"2023-12-20T04:43:54.496176+0000","last_became_peered":"2023-12-20T04:43:54.496176+0000","last_unstale":"2023-12-20T04:43:56.020858+0000","last_undegraded":"2023-12-20T04:43:56.020858+0000","last_fullsized":"2023-12-20T04:43:56.020858+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:41:23.641303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020526+0000","last_change":"2023-12-20T04:43:54.914363+0000","last_active":"2023-12-20T04:43:56.020526+0000","last_peered":"2023-12-20T04:43:56.020526+0000","last_clean":"2023-12-20T04:43:56.020526+0000","last_became_active":"2023-12-20T04:43:54.914068+0000","last_became_peered":"2023-12-20T04:43:54.914068+0000","last_unstale":"2023-12-20T04:43:56.020526+0000","last_undegraded":"2023-12-20T04:43:56.020526+0000","last_fullsized":"2023-12-20T04:43:56.020526+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:03.869454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881974+0000","last_change":"2023-12-20T04:43:54.487577+0000","last_active":"2023-12-20T04:43:55.881974+0000","last_peered":"2023-12-20T04:43:55.881974+0000","last_clean":"2023-12-20T04:43:55.881974+0000","last_became_active":"2023-12-20T04:43:54.487225+0000","last_became_peered":"2023-12-20T04:43:54.487225+0000","last_unstale":"2023-12-20T04:43:55.881974+0000","last_undegraded":"2023-12-20T04:43:55.881974+0000","last_fullsized":"2023-12-20T04:43:55.881974+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:33:44.854838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020695+0000","last_change":"2023-12-20T04:43:54.912286+0000","last_active":"2023-12-20T04:43:56.020695+0000","last_peered":"2023-12-20T04:43:56.020695+0000","last_clean":"2023-12-20T04:43:56.020695+0000","last_became_active":"2023-12-20T04:43:54.912017+0000","last_became_peered":"2023-12-20T04:43:54.912017+0000","last_unstale":"2023-12-20T04:43:56.020695+0000","last_undegraded":"2023-12-20T04:43:56.020695+0000","last_fullsized":"2023-12-20T04:43:56.020695+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:21.719783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880528+0000","last_change":"2023-12-20T04:43:54.485973+0000","last_active":"2023-12-20T04:43:55.880528+0000","last_peered":"2023-12-20T04:43:55.880528+0000","last_clean":"2023-12-20T04:43:55.880528+0000","last_became_active":"2023-12-20T04:43:54.485587+0000","last_became_peered":"2023-12-20T04:43:54.485587+0000","last_unstale":"2023-12-20T04:43:55.880528+0000","last_undegraded":"2023-12-20T04:43:55.880528+0000","last_fullsized":"2023-12-20T04:43:55.880528+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:54.163507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880961+0000","last_change":"2023-12-20T04:43:54.896336+0000","last_active":"2023-12-20T04:43:55.880961+0000","last_peered":"2023-12-20T04:43:55.880961+0000","last_clean":"2023-12-20T04:43:55.880961+0000","last_became_active":"2023-12-20T04:43:54.896033+0000","last_became_peered":"2023-12-20T04:43:54.896033+0000","last_unstale":"2023-12-20T04:43:55.880961+0000","last_undegraded":"2023-12-20T04:43:55.880961+0000","last_fullsized":"2023-12-20T04:43:55.880961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:10:23.142165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020744+0000","last_change":"2023-12-20T04:43:54.495784+0000","last_active":"2023-12-20T04:43:56.020744+0000","last_peered":"2023-12-20T04:43:56.020744+0000","last_clean":"2023-12-20T04:43:56.020744+0000","last_became_active":"2023-12-20T04:43:54.494894+0000","last_became_peered":"2023-12-20T04:43:54.494894+0000","last_unstale":"2023-12-20T04:43:56.020744+0000","last_undegraded":"2023-12-20T04:43:56.020744+0000","last_fullsized":"2023-12-20T04:43:56.020744+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:50.470667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882757+0000","last_change":"2023-12-20T04:43:54.893303+0000","last_active":"2023-12-20T04:43:55.882757+0000","last_peered":"2023-12-20T04:43:55.882757+0000","last_clean":"2023-12-20T04:43:55.882757+0000","last_became_active":"2023-12-20T04:43:54.893044+0000","last_became_peered":"2023-12-20T04:43:54.893044+0000","last_unstale":"2023-12-20T04:43:55.882757+0000","last_undegraded":"2023-12-20T04:43:55.882757+0000","last_fullsized":"2023-12-20T04:43:55.882757+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:29:44.222794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882746+0000","last_change":"2023-12-20T04:43:54.490980+0000","last_active":"2023-12-20T04:43:55.882746+0000","last_peered":"2023-12-20T04:43:55.882746+0000","last_clean":"2023-12-20T04:43:55.882746+0000","last_became_active":"2023-12-20T04:43:54.489980+0000","last_became_peered":"2023-12-20T04:43:54.489980+0000","last_unstale":"2023-12-20T04:43:55.882746+0000","last_undegraded":"2023-12-20T04:43:55.882746+0000","last_fullsized":"2023-12-20T04:43:55.882746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:32:30.324229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884071+0000","last_change":"2023-12-20T04:43:54.914434+0000","last_active":"2023-12-20T04:43:55.884071+0000","last_peered":"2023-12-20T04:43:55.884071+0000","last_clean":"2023-12-20T04:43:55.884071+0000","last_became_active":"2023-12-20T04:43:54.914174+0000","last_became_peered":"2023-12-20T04:43:54.914174+0000","last_unstale":"2023-12-20T04:43:55.884071+0000","last_undegraded":"2023-12-20T04:43:55.884071+0000","last_fullsized":"2023-12-20T04:43:55.884071+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:20.099541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021775+0000","last_change":"2023-12-20T04:43:54.488403+0000","last_active":"2023-12-20T04:43:56.021775+0000","last_peered":"2023-12-20T04:43:56.021775+0000","last_clean":"2023-12-20T04:43:56.021775+0000","last_became_active":"2023-12-20T04:43:54.488146+0000","last_became_peered":"2023-12-20T04:43:54.488146+0000","last_unstale":"2023-12-20T04:43:56.021775+0000","last_undegraded":"2023-12-20T04:43:56.021775+0000","last_fullsized":"2023-12-20T04:43:56.021775+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:46:16.006131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884340+0000","last_change":"2023-12-20T04:43:54.906949+0000","last_active":"2023-12-20T04:43:55.884340+0000","last_peered":"2023-12-20T04:43:55.884340+0000","last_clean":"2023-12-20T04:43:55.884340+0000","last_became_active":"2023-12-20T04:43:54.906694+0000","last_became_peered":"2023-12-20T04:43:54.906694+0000","last_unstale":"2023-12-20T04:43:55.884340+0000","last_undegraded":"2023-12-20T04:43:55.884340+0000","last_fullsized":"2023-12-20T04:43:55.884340+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:03:15.497172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880867+0000","last_change":"2023-12-20T04:43:54.490904+0000","last_active":"2023-12-20T04:43:55.880867+0000","last_peered":"2023-12-20T04:43:55.880867+0000","last_clean":"2023-12-20T04:43:55.880867+0000","last_became_active":"2023-12-20T04:43:54.490598+0000","last_became_peered":"2023-12-20T04:43:54.490598+0000","last_unstale":"2023-12-20T04:43:55.880867+0000","last_undegraded":"2023-12-20T04:43:55.880867+0000","last_fullsized":"2023-12-20T04:43:55.880867+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:51:47.823844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021599+0000","last_change":"2023-12-20T04:43:54.895461+0000","last_active":"2023-12-20T04:43:56.021599+0000","last_peered":"2023-12-20T04:43:56.021599+0000","last_clean":"2023-12-20T04:43:56.021599+0000","last_became_active":"2023-12-20T04:43:54.895162+0000","last_became_peered":"2023-12-20T04:43:54.895162+0000","last_unstale":"2023-12-20T04:43:56.021599+0000","last_undegraded":"2023-12-20T04:43:56.021599+0000","last_fullsized":"2023-12-20T04:43:56.021599+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:15.423438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021654+0000","last_change":"2023-12-20T04:43:54.488446+0000","last_active":"2023-12-20T04:43:56.021654+0000","last_peered":"2023-12-20T04:43:56.021654+0000","last_clean":"2023-12-20T04:43:56.021654+0000","last_became_active":"2023-12-20T04:43:54.487672+0000","last_became_peered":"2023-12-20T04:43:54.487672+0000","last_unstale":"2023-12-20T04:43:56.021654+0000","last_undegraded":"2023-12-20T04:43:56.021654+0000","last_fullsized":"2023-12-20T04:43:56.021654+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:15:47.225316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020639+0000","last_change":"2023-12-20T04:43:54.910186+0000","last_active":"2023-12-20T04:43:56.020639+0000","last_peered":"2023-12-20T04:43:56.020639+0000","last_clean":"2023-12-20T04:43:56.020639+0000","last_became_active":"2023-12-20T04:43:54.909930+0000","last_became_peered":"2023-12-20T04:43:54.909930+0000","last_unstale":"2023-12-20T04:43:56.020639+0000","last_undegraded":"2023-12-20T04:43:56.020639+0000","last_fullsized":"2023-12-20T04:43:56.020639+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:32:45.309358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020291+0000","last_change":"2023-12-20T04:43:54.492211+0000","last_active":"2023-12-20T04:43:56.020291+0000","last_peered":"2023-12-20T04:43:56.020291+0000","last_clean":"2023-12-20T04:43:56.020291+0000","last_became_active":"2023-12-20T04:43:54.491943+0000","last_became_peered":"2023-12-20T04:43:54.491943+0000","last_unstale":"2023-12-20T04:43:56.020291+0000","last_undegraded":"2023-12-20T04:43:56.020291+0000","last_fullsized":"2023-12-20T04:43:56.020291+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:50:47.895652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020328+0000","last_change":"2023-12-20T04:43:54.889864+0000","last_active":"2023-12-20T04:43:56.020328+0000","last_peered":"2023-12-20T04:43:56.020328+0000","last_clean":"2023-12-20T04:43:56.020328+0000","last_became_active":"2023-12-20T04:43:54.889445+0000","last_became_peered":"2023-12-20T04:43:54.889445+0000","last_unstale":"2023-12-20T04:43:56.020328+0000","last_undegraded":"2023-12-20T04:43:56.020328+0000","last_fullsized":"2023-12-20T04:43:56.020328+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:53:32.413782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882959+0000","last_change":"2023-12-20T04:43:54.491775+0000","last_active":"2023-12-20T04:43:55.882959+0000","last_peered":"2023-12-20T04:43:55.882959+0000","last_clean":"2023-12-20T04:43:55.882959+0000","last_became_active":"2023-12-20T04:43:54.490027+0000","last_became_peered":"2023-12-20T04:43:54.490027+0000","last_unstale":"2023-12-20T04:43:55.882959+0000","last_undegraded":"2023-12-20T04:43:55.882959+0000","last_fullsized":"2023-12-20T04:43:55.882959+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:07:11.323823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020242+0000","last_change":"2023-12-20T04:43:54.911274+0000","last_active":"2023-12-20T04:43:56.020242+0000","last_peered":"2023-12-20T04:43:56.020242+0000","last_clean":"2023-12-20T04:43:56.020242+0000","last_became_active":"2023-12-20T04:43:54.910998+0000","last_became_peered":"2023-12-20T04:43:54.910998+0000","last_unstale":"2023-12-20T04:43:56.020242+0000","last_undegraded":"2023-12-20T04:43:56.020242+0000","last_fullsized":"2023-12-20T04:43:56.020242+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:26:39.723104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020454+0000","last_change":"2023-12-20T04:43:54.496582+0000","last_active":"2023-12-20T04:43:56.020454+0000","last_peered":"2023-12-20T04:43:56.020454+0000","last_clean":"2023-12-20T04:43:56.020454+0000","last_became_active":"2023-12-20T04:43:54.496123+0000","last_became_peered":"2023-12-20T04:43:54.496123+0000","last_unstale":"2023-12-20T04:43:56.020454+0000","last_undegraded":"2023-12-20T04:43:56.020454+0000","last_fullsized":"2023-12-20T04:43:56.020454+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:29:22.619298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020450+0000","last_change":"2023-12-20T04:43:54.910411+0000","last_active":"2023-12-20T04:43:56.020450+0000","last_peered":"2023-12-20T04:43:56.020450+0000","last_clean":"2023-12-20T04:43:56.020450+0000","last_became_active":"2023-12-20T04:43:54.910125+0000","last_became_peered":"2023-12-20T04:43:54.910125+0000","last_unstale":"2023-12-20T04:43:56.020450+0000","last_undegraded":"2023-12-20T04:43:56.020450+0000","last_fullsized":"2023-12-20T04:43:56.020450+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:07:38.138240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020508+0000","last_change":"2023-12-20T04:43:54.495757+0000","last_active":"2023-12-20T04:43:56.020508+0000","last_peered":"2023-12-20T04:43:56.020508+0000","last_clean":"2023-12-20T04:43:56.020508+0000","last_became_active":"2023-12-20T04:43:54.495389+0000","last_became_peered":"2023-12-20T04:43:54.495389+0000","last_unstale":"2023-12-20T04:43:56.020508+0000","last_undegraded":"2023-12-20T04:43:56.020508+0000","last_fullsized":"2023-12-20T04:43:56.020508+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:57:06.845778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"93'2153","reported_seq":2403,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.232416+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:43:56.232416+0000","last_peered":"2023-12-20T04:43:56.232416+0000","last_clean":"2023-12-20T04:43:56.232416+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:43:56.232416+0000","last_undegraded":"2023-12-20T04:43:56.232416+0000","last_fullsized":"2023-12-20T04:43:56.232416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":2153,"log_dups_size":0,"ondisk_log_size":2153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":532,"num_read_kb":954,"num_write":241,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":532,"num_read_kb":954,"num_write":241,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2153,"ondisk_log_size":2153,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575034,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28648,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343192,"statfs":{"total":96636764160,"available":96607428608,"internally_reserved":0,"allocated":1736704,"data_stored":1124760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93899999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84499999999999997,"5min":1,"15min":1.115},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":1.095,"15min":1.2190000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91700000000000004,"5min":1.032,"15min":2.649},"last":0.44}]},{"osd":2,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.68999999999999995,"15min":0.71099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89300000000000002,"5min":1.075,"15min":1.5960000000000001},"last":0.379},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.74299999999999999,"15min":0.749},"min":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.45300000000000001},"max":{"1min":0.98999999999999999,"5min":1.1000000000000001,"15min":1.135},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.70099999999999996,"15min":0.72599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.641},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93500000000000005,"5min":1.131,"15min":2.5990000000000002},"last":0.61499999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34599999999999997},"max":{"1min":0.78500000000000003,"5min":0.90700000000000003,"15min":1.0109999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.624,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79500000000000004,"5min":1.0469999999999999,"15min":1.242},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.438,"5min":0.433,"15min":0.40400000000000003},"max":{"1min":0.86299999999999999,"5min":1.089,"15min":1.0900000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.432},"max":{"1min":0.83199999999999996,"5min":0.98799999999999999,"15min":1.071},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.443},"max":{"1min":0.99199999999999999,"5min":1.093,"15min":1.1279999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44400000000000001},"max":{"1min":0.96799999999999997,"5min":1.125,"15min":1.125},"last":0.46100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575034,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1142784,"data_stored":533868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.55600000000000005,"5min":0.33800000000000002,"15min":0.27100000000000002},"max":{"1min":0.92900000000000005,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.71999999999999997,"15min":0.71699999999999997},"min":{"1min":0.626,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.1910000000000001},"last":0.53700000000000003}]},{"osd":1,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69099999999999995,"15min":0.70199999999999996},"min":{"1min":0.55100000000000005,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.502},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.57699999999999996,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.97799999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.68400000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.006,"5min":1.216,"15min":1.216},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.38300000000000001},"max":{"1min":0.95599999999999996,"5min":1.0089999999999999,"15min":1.054},"last":0.58999999999999997}]},{"osd":3,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.78500000000000003,"15min":0.75900000000000001},"min":{"1min":0.68300000000000005,"5min":0.51800000000000002,"15min":0.45600000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.109},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96499999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60299999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.624,"15min":0.621},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.36399999999999999},"max":{"1min":0.90300000000000002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":0.995,"15min":1.071},"last":0.63200000000000001}]},{"osd":5,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.51000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.68999999999999995},"min":{"1min":0.54400000000000004,"5min":0.435,"15min":0.434},"max":{"1min":1,"5min":1.306,"15min":1.306},"last":0.61599999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72599999999999998,"15min":0.72299999999999998},"min":{"1min":0.61899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.2270000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58899999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":1.002,"15min":1.169},"last":0.55400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575035,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36256,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335584,"statfs":{"total":96636764160,"available":96599638016,"internally_reserved":0,"allocated":1138688,"data_stored":533866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68700000000000006,"15min":0.69299999999999995},"min":{"1min":0.36199999999999999,"5min":0.33700000000000002,"15min":0.29999999999999999},"max":{"1min":0.83099999999999996,"5min":0.97199999999999998,"15min":1.05},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68999999999999995,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.34300000000000003},"max":{"1min":0.877,"5min":0.95099999999999996,"15min":0.98599999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.441},"max":{"1min":0.88600000000000001,"5min":1.044,"15min":1.0600000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70999999999999996,"15min":0.73299999999999998},"min":{"1min":0.437,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95199999999999996,"5min":0.96999999999999997,"15min":1.1140000000000001},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.72699999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.94399999999999995,"15min":1.153},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71399999999999997,"15min":0.72299999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":1.004},"last":0.56499999999999995}]},{"osd":3,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92500000000000004,"5min":0.95999999999999996,"15min":1.758},"last":0.624},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73299999999999998,"15min":0.748},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90900000000000003,"5min":0.96599999999999997,"15min":1.0469999999999999},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.40600000000000003,"15min":0.35099999999999998},"max":{"1min":0.84799999999999998,"5min":0.97299999999999998,"15min":1.0229999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.39600000000000002},"max":{"1min":0.94299999999999995,"5min":0.98699999999999999,"15min":1.077},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68400000000000005,"15min":0.68000000000000005},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.39900000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":1.0700000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.87,"5min":0.999,"15min":0.999},"last":0.51400000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.42999999999999999},"max":{"1min":0.90000000000000002,"5min":1.0429999999999999,"15min":1.141},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.751},"min":{"1min":0.57599999999999996,"5min":0.54900000000000004,"15min":0.47899999999999998},"max":{"1min":0.92000000000000004,"5min":1.012,"15min":1.119},"last":0.5}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313452,"statfs":{"total":96636764160,"available":96576974848,"internally_reserved":0,"allocated":1126400,"data_stored":527546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":0.69799999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90900000000000003,"5min":1.024,"15min":1.5509999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68400000000000005,"15min":0.70499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92400000000000004,"5min":1.625,"15min":1.633},"last":0.47999999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89400000000000002,"5min":0.98799999999999999,"15min":1.798},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96399999999999997,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.55100000000000005}]},{"osd":2,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.98099999999999998,"5min":1.52,"15min":1.728},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.47999999999999998,"5min":0.46899999999999997,"15min":0.38700000000000001},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.6619999999999999},"last":0.505}]},{"osd":3,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":1.554,"15min":1.5760000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.94599999999999995,"5min":1.03,"15min":1.7649999999999999},"last":0.57399999999999995}]},{"osd":5,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63800000000000001,"15min":0.624},"min":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":1.1120000000000001,"5min":1.456,"15min":1.456},"last":0.436},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.077,"5min":1.589,"15min":1.589},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.47199999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.6739999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.47099999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.036,"5min":1.655,"15min":1.655},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.72099999999999997,"15min":0.73299999999999998},"min":{"1min":0.50700000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0029999999999999,"5min":1.052,"15min":1.125},"last":0.60899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575035,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1142784,"data_stored":533888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.26900000000000002,"5min":0.26200000000000001,"15min":0.23200000000000001},"max":{"1min":0.748,"5min":0.875,"15min":4.3700000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.499,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63100000000000001,"5min":0.80800000000000005,"15min":3.355},"last":0.28899999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.48499999999999999},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68200000000000005,"5min":0.73999999999999999,"15min":3.7160000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.53300000000000003,"15min":0.55700000000000005},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.28399999999999997},"max":{"1min":0.70199999999999996,"5min":0.74399999999999999,"15min":3.3109999999999999},"last":0.55600000000000005}]},{"osd":2,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.374,"5min":0.318,"15min":0.318},"max":{"1min":0.72799999999999998,"5min":0.88700000000000001,"15min":4.0110000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.316},"max":{"1min":0.81799999999999995,"5min":0.93300000000000005,"15min":6.1399999999999997},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.76800000000000002,"15min":0.78500000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99199999999999999,"5min":1.131,"15min":1.3089999999999999},"last":0.499},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78300000000000003,"15min":0.81799999999999995},"min":{"1min":0.52300000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93799999999999994,"5min":1.4059999999999999,"15min":3.9510000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80800000000000005,"15min":0.82899999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":1.0649999999999999,"5min":1.3300000000000001,"15min":3.9830000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.76300000000000001,"15min":0.80200000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.38400000000000001},"max":{"1min":0.88100000000000001,"5min":1.252,"15min":4.6550000000000002},"last":0.60099999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76000000000000001,"15min":0.79800000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.45600000000000002},"max":{"1min":0.95199999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.82999999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.499},"max":{"1min":1.0149999999999999,"5min":1.2270000000000001,"15min":3.8900000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.76900000000000002,"15min":0.81399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":1.1319999999999999,"15min":4.6970000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.79100000000000004,"15min":0.81999999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":1.29,"15min":1.29},"last":0.57899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40348,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331492,"statfs":{"total":96636764160,"available":96595447808,"internally_reserved":0,"allocated":1134592,"data_stored":529040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.433,"15min":0.439},"min":{"1min":0.32800000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.625,"5min":0.78900000000000003,"15min":0.89600000000000002},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.502,"15min":0.48899999999999999},"min":{"1min":0.39300000000000002,"5min":0.28499999999999998,"15min":0.26700000000000002},"max":{"1min":0.748,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.32500000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.46899999999999997},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.876,"15min":0.876},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.28100000000000003},"max":{"1min":0.65900000000000003,"5min":0.88600000000000001,"15min":1.01},"last":0.45700000000000002}]},{"osd":3,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.49299999999999999,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.504},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.49299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91600000000000004,"5min":0.95599999999999996,"15min":1.0900000000000001},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78000000000000003,"15min":0.78700000000000003},"min":{"1min":0.69199999999999995,"5min":0.46500000000000002,"15min":0.45200000000000001},"max":{"1min":1.01,"5min":1.145,"15min":1.1799999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76600000000000001,"15min":0.77900000000000003},"min":{"1min":0.61399999999999999,"5min":0.44500000000000001,"15min":0.41399999999999998},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.2010000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.625,"5min":0.499,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":1.1220000000000001,"15min":1.327},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78700000000000003,"15min":0.80500000000000005},"min":{"1min":0.70999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0449999999999999,"15min":1.381},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.59399999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.995,"5min":1.151,"15min":1.2809999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78100000000000003,"15min":0.79800000000000004},"min":{"1min":0.65500000000000003,"5min":0.51000000000000001,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":1.0609999999999999,"15min":1.153},"last":0.74199999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.81299999999999994,"15min":0.82499999999999996},"min":{"1min":0.72399999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.038,"5min":1.107,"15min":1.3520000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.73099999999999998,"5min":0.61699999999999999,"15min":0.504},"max":{"1min":1.0169999999999999,"5min":1.1220000000000001,"15min":1.401},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50224,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321616,"statfs":{"total":96636764160,"available":96585334784,"internally_reserved":0,"allocated":1155072,"data_stored":535956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28599999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48999999999999999},"min":{"1min":0.35899999999999999,"5min":0.29699999999999999,"15min":0.28599999999999998},"max":{"1min":0.73999999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.30499999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.36099999999999999,"5min":0.34699999999999998,"15min":0.32800000000000001},"max":{"1min":0.627,"5min":1.091,"15min":1.091},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.36399999999999999,"15min":0.33900000000000002},"max":{"1min":0.69899999999999995,"5min":0.85899999999999999,"15min":0.91300000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.40100000000000002,"15min":0.36599999999999999},"max":{"1min":0.77100000000000002,"5min":0.96499999999999997,"15min":1.016},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.40300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82399999999999995,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.42799999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.63500000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93100000000000005,"5min":1.27,"15min":1.4419999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.63,"5min":0.434,"15min":0.434},"max":{"1min":0.98399999999999999,"5min":1.1160000000000001,"15min":1.3380000000000001},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.5},"max":{"1min":1.0229999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.498},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76700000000000002,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":1.2450000000000001,"15min":1.3009999999999999},"last":0.48399999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.68400000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0449999999999999,"5min":1.194,"15min":1.415},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.58799999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1,"5min":1.252,"15min":1.268},"last":0.46700000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.73699999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0580000000000001,"5min":1.306,"15min":1.399},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84199999999999997},"min":{"1min":0.66800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0349999999999999,"5min":1.3440000000000001,"15min":1.373},"last":0.61499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40928,"kb_used_data":1688,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330912,"statfs":{"total":96636764160,"available":96594853888,"internally_reserved":0,"allocated":1728512,"data_stored":1124144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.46700000000000003,"15min":0.44600000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":0.83699999999999997,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.59599999999999997,"15min":0.53600000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.26900000000000002},"max":{"1min":0.70099999999999996,"5min":7.8860000000000001,"15min":7.8860000000000001},"last":0.64800000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.28399999999999997},"max":{"1min":0.75600000000000001,"5min":2.528,"15min":2.528},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71399999999999997,"5min":0.83499999999999996,"15min":1.2989999999999999},"last":0.44900000000000001}]},{"osd":3,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.52400000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29499999999999998},"max":{"1min":0.80000000000000004,"5min":3.1000000000000001,"15min":3.1000000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.68799999999999994,"15min":0.61899999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70899999999999996,"5min":10.49,"15min":10.49},"last":0.47199999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.78600000000000003,"15min":0.79300000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0780000000000001,"5min":3.0659999999999998,"15min":3.7469999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.83199999999999996,"15min":0.80300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.181,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.61099999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.79200000000000004,"15min":0.79800000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.44700000000000001},"max":{"1min":1.161,"5min":1.3109999999999999,"15min":1.877},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0349999999999999,"5min":1.2010000000000001,"15min":1.7450000000000001},"last":0.68799999999999994}]},{"osd":6,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.84499999999999997,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.47299999999999998},"max":{"1min":1.0069999999999999,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.52800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2150000000000001,"5min":1.306,"15min":1.492},"last":0.55200000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0740000000000001,"5min":1.2549999999999999,"15min":1.6539999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.81200000000000006,"15min":0.81299999999999994},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"max":{"1min":1.01,"5min":1.276,"15min":2.113},"last":0.48999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:43:59.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:43:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.205+0000 7fac5fec3640 1 Processor -- start 2023-12-20T04:43:59.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.205+0000 7fac5fec3640 1 -- start start 2023-12-20T04:43:59.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5fec3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fac5815cdb0 0x7fac580d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:59.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5fec3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac5815d190 con 0x7fac580cfc80 2023-12-20T04:43:59.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5fec3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac580d16a0 con 0x7fac580d2740 2023-12-20T04:43:59.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5fec3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fac580d1980 con 0x7fac5815cdb0 2023-12-20T04:43:59.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5e460640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fac580cfc80 0x7fac580b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39910/0 (socket says 172.21.15.32:39910) 2023-12-20T04:43:59.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5e460640 1 -- 172.21.15.32:0/2011459478 learned_addr learned my addr 172.21.15.32:0/2011459478 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:59.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5dc5f640 1 --2- 172.21.15.32:0/2011459478 >> v2:172.21.15.39:3300/0 conn(0x7fac5815cdb0 0x7fac580d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:59.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5cc5d640 1 -- 172.21.15.32:0/2011459478 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1681630150 0 0) 0x7fac5815d190 con 0x7fac580cfc80 2023-12-20T04:43:59.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5cc5d640 1 -- 172.21.15.32:0/2011459478 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac44003530 con 0x7fac580cfc80 2023-12-20T04:43:59.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5dc5f640 1 -- 172.21.15.32:0/2011459478 >> v1:172.21.15.39:6789/0 conn(0x7fac580d2740 legacy=0x7fac580cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5cc5d640 1 -- 172.21.15.32:0/2011459478 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1897405598 0 0) 0x7fac580d16a0 con 0x7fac580d2740 2023-12-20T04:43:59.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5dc5f640 1 -- 172.21.15.32:0/2011459478 >> v1:172.21.15.32:6789/0 conn(0x7fac580cfc80 legacy=0x7fac580b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5dc5f640 1 -- 172.21.15.32:0/2011459478 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac580d1c60 con 0x7fac5815cdb0 2023-12-20T04:43:59.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.209+0000 7fac5dc5f640 1 --2- 172.21.15.32:0/2011459478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac5815cdb0 0x7fac580d0be0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fac540097c0 tx=0x7fac5401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ee5d6382b104e2f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:59.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5cc5d640 1 -- 172.21.15.32:0/2011459478 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540a8970 con 0x7fac5815cdb0 2023-12-20T04:43:59.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5cc5d640 1 -- 172.21.15.32:0/2011459478 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac540a8ad0 con 0x7fac5815cdb0 2023-12-20T04:43:59.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5cc5d640 1 -- 172.21.15.32:0/2011459478 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac540a8e60 con 0x7fac5815cdb0 2023-12-20T04:43:59.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 -- 172.21.15.32:0/2011459478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac5815cdb0 msgr2=0x7fac580d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 --2- 172.21.15.32:0/2011459478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac5815cdb0 0x7fac580d0be0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7fac540097c0 tx=0x7fac5401ec70 comp rx=0 tx=0).stop 2023-12-20T04:43:59.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 -- 172.21.15.32:0/2011459478 shutdown_connections 2023-12-20T04:43:59.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 --2- 172.21.15.32:0/2011459478 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac5815cdb0 0x7fac580d0be0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 -- 172.21.15.32:0/2011459478 >> 172.21.15.32:0/2011459478 conn(0x7fac58065ee0 msgr2=0x7fac5806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:59.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 -- 172.21.15.32:0/2011459478 shutdown_connections 2023-12-20T04:43:59.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 -- 172.21.15.32:0/2011459478 wait complete. 2023-12-20T04:43:59.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 Processor -- start 2023-12-20T04:43:59.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 -- start start 2023-12-20T04:43:59.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac580cfc80 0x7fac581f5680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:59.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac580d2740 0x7fac580cbd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:59.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac5815cdb0 0x7fac580b7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:59.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac5814a5c0 con 0x7fac580d2740 2023-12-20T04:43:59.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac580cb8f0 con 0x7fac5815cdb0 2023-12-20T04:43:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.213+0000 7fac5fec3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac580ca840 con 0x7fac580cfc80 2023-12-20T04:43:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5d45e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac580d2740 0x7fac580cbd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5d45e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac580d2740 0x7fac580cbd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42646/0 (socket says 172.21.15.32:42646) 2023-12-20T04:43:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5d45e640 1 -- 172.21.15.32:0/3483999133 learned_addr learned my addr 172.21.15.32:0/3483999133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5e460640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac5815cdb0 0x7fac580b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5dc5f640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac580cfc80 0x7fac581f5680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5d45e640 1 -- 172.21.15.32:0/3483999133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac580cfc80 msgr2=0x7fac581f5680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5d45e640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac580cfc80 0x7fac581f5680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5d45e640 1 -- 172.21.15.32:0/3483999133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac5815cdb0 msgr2=0x7fac580b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5d45e640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac5815cdb0 0x7fac580b7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5d45e640 1 -- 172.21.15.32:0/3483999133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac54009420 con 0x7fac580d2740 2023-12-20T04:43:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5d45e640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac580d2740 0x7fac580cbd80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fac4800c500 tx=0x7fac4800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac4e7fc640 1 -- 172.21.15.32:0/3483999133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48003600 con 0x7fac580d2740 2023-12-20T04:43:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac4e7fc640 1 -- 172.21.15.32:0/3483999133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac48003760 con 0x7fac580d2740 2023-12-20T04:43:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac4e7fc640 1 -- 172.21.15.32:0/3483999133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac48003b10 con 0x7fac580d2740 2023-12-20T04:43:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5fec3640 1 -- 172.21.15.32:0/3483999133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac580cab20 con 0x7fac580d2740 2023-12-20T04:43:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5fec3640 1 -- 172.21.15.32:0/3483999133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac580cb580 con 0x7fac580d2740 2023-12-20T04:43:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac5fec3640 1 -- 172.21.15.32:0/3483999133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac5804e000 con 0x7fac580d2740 2023-12-20T04:43:59.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.217+0000 7fac4e7fc640 1 -- 172.21.15.32:0/3483999133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fac48003c70 con 0x7fac580d2740 2023-12-20T04:43:59.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.221+0000 7fac4e7fc640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac3c065b40 0x7fac3c068000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:59.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.221+0000 7fac4e7fc640 1 -- 172.21.15.32:0/3483999133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fac48082670 con 0x7fac580d2740 2023-12-20T04:43:59.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.221+0000 7fac5dc5f640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac3c065b40 0x7fac3c068000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:59.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.221+0000 7fac5dc5f640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac3c065b40 0x7fac3c068000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac540b2000 tx=0x7fac54008040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.225+0000 7fac4e7fc640 1 -- 172.21.15.32:0/3483999133 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fac480502c0 con 0x7fac580d2740 2023-12-20T04:43:59.295 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:43:59.293+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:43:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.385+0000 7fac5fec3640 1 -- 172.21.15.32:0/3483999133 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fac580d2220 con 0x7fac3c065b40 2023-12-20T04:43:59.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac4e7fc640 1 -- 172.21.15.32:0/3483999133 <== mgr.6642 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+288913 (secure 0 0 0) 0x7fac540c0090 con 0x7fac3c065b40 2023-12-20T04:43:59.394 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:43:59.395 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:43:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 -- 172.21.15.32:0/3483999133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac3c065b40 msgr2=0x7fac3c068000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac3c065b40 0x7fac3c068000 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7fac540b2000 tx=0x7fac54008040 comp rx=0 tx=0).stop 2023-12-20T04:43:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 -- 172.21.15.32:0/3483999133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac580d2740 msgr2=0x7fac580cbd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac580d2740 0x7fac580cbd80 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fac4800c500 tx=0x7fac4800c9c0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 -- 172.21.15.32:0/3483999133 shutdown_connections 2023-12-20T04:43:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fac3c065b40 0x7fac3c068000 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac5815cdb0 0x7fac580b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac580d2740 0x7fac580cbd80 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 --2- 172.21.15.32:0/3483999133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac580cfc80 0x7fac581f5680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 -- 172.21.15.32:0/3483999133 >> 172.21.15.32:0/3483999133 conn(0x7fac58065ee0 msgr2=0x7fac580cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 -- 172.21.15.32:0/3483999133 shutdown_connections 2023-12-20T04:43:59.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.393+0000 7fac5fec3640 1 -- 172.21.15.32:0/3483999133 wait complete. 2023-12-20T04:43:59.415 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-20T04:43:58.244994+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":532,"num_read_kb":954,"num_write":285,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2191,"ondisk_log_size":2191,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":106,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":327312,"kb_used_data":10064,"kb_used_omap":0,"kb_used_meta":317184,"kb_avail":754647408,"statfs":{"total":773094113280,"available":772758945792,"internally_reserved":0,"allocated":10305536,"data_stored":5443068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":324796416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":6,"num_read_kb":6,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.787280"},"pg_stats":[{"pgid":"11.3f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884151+0000","last_change":"2023-12-20T04:43:54.909900+0000","last_active":"2023-12-20T04:43:55.884151+0000","last_peered":"2023-12-20T04:43:55.884151+0000","last_clean":"2023-12-20T04:43:55.884151+0000","last_became_active":"2023-12-20T04:43:54.909294+0000","last_became_peered":"2023-12-20T04:43:54.909294+0000","last_unstale":"2023-12-20T04:43:55.884151+0000","last_undegraded":"2023-12-20T04:43:55.884151+0000","last_fullsized":"2023-12-20T04:43:55.884151+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:00:19.171044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.3e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020869+0000","last_change":"2023-12-20T04:43:54.912545+0000","last_active":"2023-12-20T04:43:56.020869+0000","last_peered":"2023-12-20T04:43:56.020869+0000","last_clean":"2023-12-20T04:43:56.020869+0000","last_became_active":"2023-12-20T04:43:54.912250+0000","last_became_peered":"2023-12-20T04:43:54.912250+0000","last_unstale":"2023-12-20T04:43:56.020869+0000","last_undegraded":"2023-12-20T04:43:56.020869+0000","last_fullsized":"2023-12-20T04:43:56.020869+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:29.439450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.3d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021268+0000","last_change":"2023-12-20T04:43:54.891226+0000","last_active":"2023-12-20T04:43:56.021268+0000","last_peered":"2023-12-20T04:43:56.021268+0000","last_clean":"2023-12-20T04:43:56.021268+0000","last_became_active":"2023-12-20T04:43:54.890963+0000","last_became_peered":"2023-12-20T04:43:54.890963+0000","last_unstale":"2023-12-20T04:43:56.021268+0000","last_undegraded":"2023-12-20T04:43:56.021268+0000","last_fullsized":"2023-12-20T04:43:56.021268+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:25.438870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.3c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021038+0000","last_change":"2023-12-20T04:43:54.909066+0000","last_active":"2023-12-20T04:43:56.021038+0000","last_peered":"2023-12-20T04:43:56.021038+0000","last_clean":"2023-12-20T04:43:56.021038+0000","last_became_active":"2023-12-20T04:43:54.908464+0000","last_became_peered":"2023-12-20T04:43:54.908464+0000","last_unstale":"2023-12-20T04:43:56.021038+0000","last_undegraded":"2023-12-20T04:43:56.021038+0000","last_fullsized":"2023-12-20T04:43:56.021038+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:18.456750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.3b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021595+0000","last_change":"2023-12-20T04:43:54.895820+0000","last_active":"2023-12-20T04:43:56.021595+0000","last_peered":"2023-12-20T04:43:56.021595+0000","last_clean":"2023-12-20T04:43:56.021595+0000","last_became_active":"2023-12-20T04:43:54.895296+0000","last_became_peered":"2023-12-20T04:43:54.895296+0000","last_unstale":"2023-12-20T04:43:56.021595+0000","last_undegraded":"2023-12-20T04:43:56.021595+0000","last_fullsized":"2023-12-20T04:43:56.021595+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:57:53.149131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020355+0000","last_change":"2023-12-20T04:43:54.907933+0000","last_active":"2023-12-20T04:43:56.020355+0000","last_peered":"2023-12-20T04:43:56.020355+0000","last_clean":"2023-12-20T04:43:56.020355+0000","last_became_active":"2023-12-20T04:43:54.907658+0000","last_became_peered":"2023-12-20T04:43:54.907658+0000","last_unstale":"2023-12-20T04:43:56.020355+0000","last_undegraded":"2023-12-20T04:43:56.020355+0000","last_fullsized":"2023-12-20T04:43:56.020355+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:10:18.456750+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.39","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881782+0000","last_change":"2023-12-20T04:43:54.895250+0000","last_active":"2023-12-20T04:43:55.881782+0000","last_peered":"2023-12-20T04:43:55.881782+0000","last_clean":"2023-12-20T04:43:55.881782+0000","last_became_active":"2023-12-20T04:43:54.894440+0000","last_became_peered":"2023-12-20T04:43:54.894440+0000","last_unstale":"2023-12-20T04:43:55.881782+0000","last_undegraded":"2023-12-20T04:43:55.881782+0000","last_fullsized":"2023-12-20T04:43:55.881782+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:06:19.780285+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.38","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020357+0000","last_change":"2023-12-20T04:43:54.905941+0000","last_active":"2023-12-20T04:43:56.020357+0000","last_peered":"2023-12-20T04:43:56.020357+0000","last_clean":"2023-12-20T04:43:56.020357+0000","last_became_active":"2023-12-20T04:43:54.905487+0000","last_became_peered":"2023-12-20T04:43:54.905487+0000","last_unstale":"2023-12-20T04:43:56.020357+0000","last_undegraded":"2023-12-20T04:43:56.020357+0000","last_fullsized":"2023-12-20T04:43:56.020357+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:50:46.140658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.37","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021850+0000","last_change":"2023-12-20T04:43:54.899667+0000","last_active":"2023-12-20T04:43:56.021850+0000","last_peered":"2023-12-20T04:43:56.021850+0000","last_clean":"2023-12-20T04:43:56.021850+0000","last_became_active":"2023-12-20T04:43:54.899068+0000","last_became_peered":"2023-12-20T04:43:54.899068+0000","last_unstale":"2023-12-20T04:43:56.021850+0000","last_undegraded":"2023-12-20T04:43:56.021850+0000","last_fullsized":"2023-12-20T04:43:56.021850+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:50:48.904067+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.36","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883980+0000","last_change":"2023-12-20T04:43:54.903659+0000","last_active":"2023-12-20T04:43:55.883980+0000","last_peered":"2023-12-20T04:43:55.883980+0000","last_clean":"2023-12-20T04:43:55.883980+0000","last_became_active":"2023-12-20T04:43:54.903372+0000","last_became_peered":"2023-12-20T04:43:54.903372+0000","last_unstale":"2023-12-20T04:43:55.883980+0000","last_undegraded":"2023-12-20T04:43:55.883980+0000","last_fullsized":"2023-12-20T04:43:55.883980+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:55.171922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.35","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884248+0000","last_change":"2023-12-20T04:43:54.901440+0000","last_active":"2023-12-20T04:43:55.884248+0000","last_peered":"2023-12-20T04:43:55.884248+0000","last_clean":"2023-12-20T04:43:55.884248+0000","last_became_active":"2023-12-20T04:43:54.900816+0000","last_became_peered":"2023-12-20T04:43:54.900816+0000","last_unstale":"2023-12-20T04:43:55.884248+0000","last_undegraded":"2023-12-20T04:43:55.884248+0000","last_fullsized":"2023-12-20T04:43:55.884248+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:28:37.106377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.34","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020635+0000","last_change":"2023-12-20T04:43:54.905361+0000","last_active":"2023-12-20T04:43:56.020635+0000","last_peered":"2023-12-20T04:43:56.020635+0000","last_clean":"2023-12-20T04:43:56.020635+0000","last_became_active":"2023-12-20T04:43:54.904897+0000","last_became_peered":"2023-12-20T04:43:54.904897+0000","last_unstale":"2023-12-20T04:43:56.020635+0000","last_undegraded":"2023-12-20T04:43:56.020635+0000","last_fullsized":"2023-12-20T04:43:56.020635+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:04:39.085502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.33","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880696+0000","last_change":"2023-12-20T04:43:54.909627+0000","last_active":"2023-12-20T04:43:55.880696+0000","last_peered":"2023-12-20T04:43:55.880696+0000","last_clean":"2023-12-20T04:43:55.880696+0000","last_became_active":"2023-12-20T04:43:54.909343+0000","last_became_peered":"2023-12-20T04:43:54.909343+0000","last_unstale":"2023-12-20T04:43:55.880696+0000","last_undegraded":"2023-12-20T04:43:55.880696+0000","last_fullsized":"2023-12-20T04:43:55.880696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:39:51.562870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.32","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021584+0000","last_change":"2023-12-20T04:43:54.903889+0000","last_active":"2023-12-20T04:43:56.021584+0000","last_peered":"2023-12-20T04:43:56.021584+0000","last_clean":"2023-12-20T04:43:56.021584+0000","last_became_active":"2023-12-20T04:43:54.901072+0000","last_became_peered":"2023-12-20T04:43:54.901072+0000","last_unstale":"2023-12-20T04:43:56.021584+0000","last_undegraded":"2023-12-20T04:43:56.021584+0000","last_fullsized":"2023-12-20T04:43:56.021584+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:25:23.494360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.31","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880599+0000","last_change":"2023-12-20T04:43:54.895682+0000","last_active":"2023-12-20T04:43:55.880599+0000","last_peered":"2023-12-20T04:43:55.880599+0000","last_clean":"2023-12-20T04:43:55.880599+0000","last_became_active":"2023-12-20T04:43:54.895394+0000","last_became_peered":"2023-12-20T04:43:54.895394+0000","last_unstale":"2023-12-20T04:43:55.880599+0000","last_undegraded":"2023-12-20T04:43:55.880599+0000","last_fullsized":"2023-12-20T04:43:55.880599+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:20.099541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.30","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882912+0000","last_change":"2023-12-20T04:43:54.909324+0000","last_active":"2023-12-20T04:43:55.882912+0000","last_peered":"2023-12-20T04:43:55.882912+0000","last_clean":"2023-12-20T04:43:55.882912+0000","last_became_active":"2023-12-20T04:43:54.909009+0000","last_became_peered":"2023-12-20T04:43:54.909009+0000","last_unstale":"2023-12-20T04:43:55.882912+0000","last_undegraded":"2023-12-20T04:43:55.882912+0000","last_fullsized":"2023-12-20T04:43:55.882912+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:28:37.106377+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021891+0000","last_change":"2023-12-20T04:43:54.899500+0000","last_active":"2023-12-20T04:43:56.021891+0000","last_peered":"2023-12-20T04:43:56.021891+0000","last_clean":"2023-12-20T04:43:56.021891+0000","last_became_active":"2023-12-20T04:43:54.898791+0000","last_became_peered":"2023-12-20T04:43:54.898791+0000","last_unstale":"2023-12-20T04:43:56.021891+0000","last_undegraded":"2023-12-20T04:43:56.021891+0000","last_fullsized":"2023-12-20T04:43:56.021891+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:43:10.747734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.2e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020696+0000","last_change":"2023-12-20T04:43:54.910720+0000","last_active":"2023-12-20T04:43:56.020696+0000","last_peered":"2023-12-20T04:43:56.020696+0000","last_clean":"2023-12-20T04:43:56.020696+0000","last_became_active":"2023-12-20T04:43:54.910443+0000","last_became_peered":"2023-12-20T04:43:54.910443+0000","last_unstale":"2023-12-20T04:43:56.020696+0000","last_undegraded":"2023-12-20T04:43:56.020696+0000","last_fullsized":"2023-12-20T04:43:56.020696+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:29.439450+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.2d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020528+0000","last_change":"2023-12-20T04:43:54.907136+0000","last_active":"2023-12-20T04:43:56.020528+0000","last_peered":"2023-12-20T04:43:56.020528+0000","last_clean":"2023-12-20T04:43:56.020528+0000","last_became_active":"2023-12-20T04:43:54.904857+0000","last_became_peered":"2023-12-20T04:43:54.904857+0000","last_unstale":"2023-12-20T04:43:56.020528+0000","last_undegraded":"2023-12-20T04:43:56.020528+0000","last_fullsized":"2023-12-20T04:43:56.020528+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:33:45.863254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.2c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880712+0000","last_change":"2023-12-20T04:43:54.897121+0000","last_active":"2023-12-20T04:43:55.880712+0000","last_peered":"2023-12-20T04:43:55.880712+0000","last_clean":"2023-12-20T04:43:55.880712+0000","last_became_active":"2023-12-20T04:43:54.896862+0000","last_became_peered":"2023-12-20T04:43:54.896862+0000","last_unstale":"2023-12-20T04:43:55.880712+0000","last_undegraded":"2023-12-20T04:43:55.880712+0000","last_fullsized":"2023-12-20T04:43:55.880712+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:54.224938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883413+0000","last_change":"2023-12-20T04:43:54.897408+0000","last_active":"2023-12-20T04:43:55.883413+0000","last_peered":"2023-12-20T04:43:55.883413+0000","last_clean":"2023-12-20T04:43:55.883413+0000","last_became_active":"2023-12-20T04:43:54.897062+0000","last_became_peered":"2023-12-20T04:43:54.897062+0000","last_unstale":"2023-12-20T04:43:55.883413+0000","last_undegraded":"2023-12-20T04:43:55.883413+0000","last_fullsized":"2023-12-20T04:43:55.883413+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:09:53.729698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.2a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883049+0000","last_change":"2023-12-20T04:43:54.903562+0000","last_active":"2023-12-20T04:43:55.883049+0000","last_peered":"2023-12-20T04:43:55.883049+0000","last_clean":"2023-12-20T04:43:55.883049+0000","last_became_active":"2023-12-20T04:43:54.903279+0000","last_became_peered":"2023-12-20T04:43:54.903279+0000","last_unstale":"2023-12-20T04:43:55.883049+0000","last_undegraded":"2023-12-20T04:43:55.883049+0000","last_fullsized":"2023-12-20T04:43:55.883049+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:56:23.814101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.29","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883854+0000","last_change":"2023-12-20T04:43:54.899617+0000","last_active":"2023-12-20T04:43:55.883854+0000","last_peered":"2023-12-20T04:43:55.883854+0000","last_clean":"2023-12-20T04:43:55.883854+0000","last_became_active":"2023-12-20T04:43:54.898947+0000","last_became_peered":"2023-12-20T04:43:54.898947+0000","last_unstale":"2023-12-20T04:43:55.883854+0000","last_undegraded":"2023-12-20T04:43:55.883854+0000","last_fullsized":"2023-12-20T04:43:55.883854+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:28.512003+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.28","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883378+0000","last_change":"2023-12-20T04:43:54.913205+0000","last_active":"2023-12-20T04:43:55.883378+0000","last_peered":"2023-12-20T04:43:55.883378+0000","last_clean":"2023-12-20T04:43:55.883378+0000","last_became_active":"2023-12-20T04:43:54.912945+0000","last_became_peered":"2023-12-20T04:43:54.912945+0000","last_unstale":"2023-12-20T04:43:55.883378+0000","last_undegraded":"2023-12-20T04:43:55.883378+0000","last_fullsized":"2023-12-20T04:43:55.883378+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:29:39.148170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.27","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883744+0000","last_change":"2023-12-20T04:43:54.899556+0000","last_active":"2023-12-20T04:43:55.883744+0000","last_peered":"2023-12-20T04:43:55.883744+0000","last_clean":"2023-12-20T04:43:55.883744+0000","last_became_active":"2023-12-20T04:43:54.898140+0000","last_became_peered":"2023-12-20T04:43:54.898140+0000","last_unstale":"2023-12-20T04:43:55.883744+0000","last_undegraded":"2023-12-20T04:43:55.883744+0000","last_fullsized":"2023-12-20T04:43:55.883744+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:51:48.832259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.26","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882708+0000","last_change":"2023-12-20T04:43:54.914449+0000","last_active":"2023-12-20T04:43:55.882708+0000","last_peered":"2023-12-20T04:43:55.882708+0000","last_clean":"2023-12-20T04:43:55.882708+0000","last_became_active":"2023-12-20T04:43:54.913169+0000","last_became_peered":"2023-12-20T04:43:54.913169+0000","last_unstale":"2023-12-20T04:43:55.882708+0000","last_undegraded":"2023-12-20T04:43:55.882708+0000","last_fullsized":"2023-12-20T04:43:55.882708+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:40:28.475063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.25","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021356+0000","last_change":"2023-12-20T04:43:54.896212+0000","last_active":"2023-12-20T04:43:56.021356+0000","last_peered":"2023-12-20T04:43:56.021356+0000","last_clean":"2023-12-20T04:43:56.021356+0000","last_became_active":"2023-12-20T04:43:54.895973+0000","last_became_peered":"2023-12-20T04:43:54.895973+0000","last_unstale":"2023-12-20T04:43:56.021356+0000","last_undegraded":"2023-12-20T04:43:56.021356+0000","last_fullsized":"2023-12-20T04:43:56.021356+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:14:56.700329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.24","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883585+0000","last_change":"2023-12-20T04:43:54.904989+0000","last_active":"2023-12-20T04:43:55.883585+0000","last_peered":"2023-12-20T04:43:55.883585+0000","last_clean":"2023-12-20T04:43:55.883585+0000","last_became_active":"2023-12-20T04:43:54.904675+0000","last_became_peered":"2023-12-20T04:43:54.904675+0000","last_unstale":"2023-12-20T04:43:55.883585+0000","last_undegraded":"2023-12-20T04:43:55.883585+0000","last_fullsized":"2023-12-20T04:43:55.883585+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:52.478483+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.23","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020915+0000","last_change":"2023-12-20T04:43:54.889243+0000","last_active":"2023-12-20T04:43:56.020915+0000","last_peered":"2023-12-20T04:43:56.020915+0000","last_clean":"2023-12-20T04:43:56.020915+0000","last_became_active":"2023-12-20T04:43:54.888983+0000","last_became_peered":"2023-12-20T04:43:54.888983+0000","last_unstale":"2023-12-20T04:43:56.020915+0000","last_undegraded":"2023-12-20T04:43:56.020915+0000","last_fullsized":"2023-12-20T04:43:56.020915+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:36:39.445465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.22","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881847+0000","last_change":"2023-12-20T04:43:54.913726+0000","last_active":"2023-12-20T04:43:55.881847+0000","last_peered":"2023-12-20T04:43:55.881847+0000","last_clean":"2023-12-20T04:43:55.881847+0000","last_became_active":"2023-12-20T04:43:54.913471+0000","last_became_peered":"2023-12-20T04:43:54.913471+0000","last_unstale":"2023-12-20T04:43:55.881847+0000","last_undegraded":"2023-12-20T04:43:55.881847+0000","last_fullsized":"2023-12-20T04:43:55.881847+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:33:34.511010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.21","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881954+0000","last_change":"2023-12-20T04:43:54.904055+0000","last_active":"2023-12-20T04:43:55.881954+0000","last_peered":"2023-12-20T04:43:55.881954+0000","last_clean":"2023-12-20T04:43:55.881954+0000","last_became_active":"2023-12-20T04:43:54.903791+0000","last_became_peered":"2023-12-20T04:43:54.903791+0000","last_unstale":"2023-12-20T04:43:55.881954+0000","last_undegraded":"2023-12-20T04:43:55.881954+0000","last_fullsized":"2023-12-20T04:43:55.881954+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:19:35.337100+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.20","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020359+0000","last_change":"2023-12-20T04:43:54.911172+0000","last_active":"2023-12-20T04:43:56.020359+0000","last_peered":"2023-12-20T04:43:56.020359+0000","last_clean":"2023-12-20T04:43:56.020359+0000","last_became_active":"2023-12-20T04:43:54.910873+0000","last_became_peered":"2023-12-20T04:43:54.910873+0000","last_unstale":"2023-12-20T04:43:56.020359+0000","last_undegraded":"2023-12-20T04:43:56.020359+0000","last_fullsized":"2023-12-20T04:43:56.020359+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:07:55.899832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884186+0000","last_change":"2023-12-20T04:43:54.914900+0000","last_active":"2023-12-20T04:43:55.884186+0000","last_peered":"2023-12-20T04:43:55.884186+0000","last_clean":"2023-12-20T04:43:55.884186+0000","last_became_active":"2023-12-20T04:43:54.913650+0000","last_became_peered":"2023-12-20T04:43:54.913650+0000","last_unstale":"2023-12-20T04:43:55.884186+0000","last_undegraded":"2023-12-20T04:43:55.884186+0000","last_fullsized":"2023-12-20T04:43:55.884186+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:56.046081+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1f","version":"92'4","reported_seq":15,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884370+0000","last_change":"2023-12-20T04:43:54.486574+0000","last_active":"2023-12-20T04:43:55.884370+0000","last_peered":"2023-12-20T04:43:55.884370+0000","last_clean":"2023-12-20T04:43:55.884370+0000","last_became_active":"2023-12-20T04:43:54.486282+0000","last_became_peered":"2023-12-20T04:43:54.486282+0000","last_unstale":"2023-12-20T04:43:55.884370+0000","last_undegraded":"2023-12-20T04:43:55.884370+0000","last_fullsized":"2023-12-20T04:43:55.884370+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:07:02.481763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.1f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021925+0000","last_change":"2023-12-20T04:43:54.899335+0000","last_active":"2023-12-20T04:43:56.021925+0000","last_peered":"2023-12-20T04:43:56.021925+0000","last_clean":"2023-12-20T04:43:56.021925+0000","last_became_active":"2023-12-20T04:43:54.898507+0000","last_became_peered":"2023-12-20T04:43:54.898507+0000","last_unstale":"2023-12-20T04:43:56.021925+0000","last_undegraded":"2023-12-20T04:43:56.021925+0000","last_fullsized":"2023-12-20T04:43:56.021925+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:47.883793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.1e","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881043+0000","last_change":"2023-12-20T04:43:54.487030+0000","last_active":"2023-12-20T04:43:55.881043+0000","last_peered":"2023-12-20T04:43:55.881043+0000","last_clean":"2023-12-20T04:43:55.881043+0000","last_became_active":"2023-12-20T04:43:54.486749+0000","last_became_peered":"2023-12-20T04:43:54.486749+0000","last_unstale":"2023-12-20T04:43:55.881043+0000","last_undegraded":"2023-12-20T04:43:55.881043+0000","last_fullsized":"2023-12-20T04:43:55.881043+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:51.470068+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.1c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020544+0000","last_change":"2023-12-20T04:43:54.909644+0000","last_active":"2023-12-20T04:43:56.020544+0000","last_peered":"2023-12-20T04:43:56.020544+0000","last_clean":"2023-12-20T04:43:56.020544+0000","last_became_active":"2023-12-20T04:43:54.909376+0000","last_became_peered":"2023-12-20T04:43:54.909376+0000","last_unstale":"2023-12-20T04:43:56.020544+0000","last_undegraded":"2023-12-20T04:43:56.020544+0000","last_fullsized":"2023-12-20T04:43:56.020544+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:53:32.413782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.1d","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883879+0000","last_change":"2023-12-20T04:43:54.497844+0000","last_active":"2023-12-20T04:43:55.883879+0000","last_peered":"2023-12-20T04:43:55.883879+0000","last_clean":"2023-12-20T04:43:55.883879+0000","last_became_active":"2023-12-20T04:43:54.497548+0000","last_became_peered":"2023-12-20T04:43:54.497548+0000","last_unstale":"2023-12-20T04:43:55.883879+0000","last_undegraded":"2023-12-20T04:43:55.883879+0000","last_fullsized":"2023-12-20T04:43:55.883879+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:04:41.515933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884267+0000","last_change":"2023-12-20T04:43:54.901619+0000","last_active":"2023-12-20T04:43:55.884267+0000","last_peered":"2023-12-20T04:43:55.884267+0000","last_clean":"2023-12-20T04:43:55.884267+0000","last_became_active":"2023-12-20T04:43:54.901144+0000","last_became_peered":"2023-12-20T04:43:54.901144+0000","last_unstale":"2023-12-20T04:43:55.884267+0000","last_undegraded":"2023-12-20T04:43:55.884267+0000","last_fullsized":"2023-12-20T04:43:55.884267+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:51:48.832259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.1c","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020640+0000","last_change":"2023-12-20T04:43:54.496861+0000","last_active":"2023-12-20T04:43:56.020640+0000","last_peered":"2023-12-20T04:43:56.020640+0000","last_clean":"2023-12-20T04:43:56.020640+0000","last_became_active":"2023-12-20T04:43:54.496464+0000","last_became_peered":"2023-12-20T04:43:54.496464+0000","last_unstale":"2023-12-20T04:43:56.020640+0000","last_undegraded":"2023-12-20T04:43:56.020640+0000","last_fullsized":"2023-12-20T04:43:56.020640+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:06:06.325087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.1a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881940+0000","last_change":"2023-12-20T04:43:54.896052+0000","last_active":"2023-12-20T04:43:55.881940+0000","last_peered":"2023-12-20T04:43:55.881940+0000","last_clean":"2023-12-20T04:43:55.881940+0000","last_became_active":"2023-12-20T04:43:54.895763+0000","last_became_peered":"2023-12-20T04:43:54.895763+0000","last_unstale":"2023-12-20T04:43:55.881940+0000","last_undegraded":"2023-12-20T04:43:55.881940+0000","last_fullsized":"2023-12-20T04:43:55.881940+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:07:03.490178+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.1b","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020984+0000","last_change":"2023-12-20T04:43:54.494013+0000","last_active":"2023-12-20T04:43:56.020984+0000","last_peered":"2023-12-20T04:43:56.020984+0000","last_clean":"2023-12-20T04:43:56.020984+0000","last_became_active":"2023-12-20T04:43:54.493745+0000","last_became_peered":"2023-12-20T04:43:54.493745+0000","last_unstale":"2023-12-20T04:43:56.020984+0000","last_undegraded":"2023-12-20T04:43:56.020984+0000","last_fullsized":"2023-12-20T04:43:56.020984+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:53.216522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.1b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883545+0000","last_change":"2023-12-20T04:43:54.900003+0000","last_active":"2023-12-20T04:43:55.883545+0000","last_peered":"2023-12-20T04:43:55.883545+0000","last_clean":"2023-12-20T04:43:55.883545+0000","last_became_active":"2023-12-20T04:43:54.897336+0000","last_became_peered":"2023-12-20T04:43:54.897336+0000","last_unstale":"2023-12-20T04:43:55.883545+0000","last_undegraded":"2023-12-20T04:43:55.883545+0000","last_fullsized":"2023-12-20T04:43:55.883545+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:01.626753+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.1a","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020738+0000","last_change":"2023-12-20T04:43:54.497282+0000","last_active":"2023-12-20T04:43:56.020738+0000","last_peered":"2023-12-20T04:43:56.020738+0000","last_clean":"2023-12-20T04:43:56.020738+0000","last_became_active":"2023-12-20T04:43:54.496962+0000","last_became_peered":"2023-12-20T04:43:54.496962+0000","last_unstale":"2023-12-20T04:43:56.020738+0000","last_undegraded":"2023-12-20T04:43:56.020738+0000","last_fullsized":"2023-12-20T04:43:56.020738+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:25:52.397612+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.18","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021050+0000","last_change":"2023-12-20T04:43:54.907240+0000","last_active":"2023-12-20T04:43:56.021050+0000","last_peered":"2023-12-20T04:43:56.021050+0000","last_clean":"2023-12-20T04:43:56.021050+0000","last_became_active":"2023-12-20T04:43:54.906932+0000","last_became_peered":"2023-12-20T04:43:54.906932+0000","last_unstale":"2023-12-20T04:43:56.021050+0000","last_undegraded":"2023-12-20T04:43:56.021050+0000","last_fullsized":"2023-12-20T04:43:56.021050+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:11:09.188333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.19","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020288+0000","last_change":"2023-12-20T04:43:54.492236+0000","last_active":"2023-12-20T04:43:56.020288+0000","last_peered":"2023-12-20T04:43:56.020288+0000","last_clean":"2023-12-20T04:43:56.020288+0000","last_became_active":"2023-12-20T04:43:54.491979+0000","last_became_peered":"2023-12-20T04:43:54.491979+0000","last_unstale":"2023-12-20T04:43:56.020288+0000","last_undegraded":"2023-12-20T04:43:56.020288+0000","last_fullsized":"2023-12-20T04:43:56.020288+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:43:09.739319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.19","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880694+0000","last_change":"2023-12-20T04:43:54.897155+0000","last_active":"2023-12-20T04:43:55.880694+0000","last_peered":"2023-12-20T04:43:55.880694+0000","last_clean":"2023-12-20T04:43:55.880694+0000","last_became_active":"2023-12-20T04:43:54.896907+0000","last_became_peered":"2023-12-20T04:43:54.896907+0000","last_unstale":"2023-12-20T04:43:55.880694+0000","last_undegraded":"2023-12-20T04:43:55.880694+0000","last_fullsized":"2023-12-20T04:43:55.880694+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:36.350327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.18","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021143+0000","last_change":"2023-12-20T04:43:54.484385+0000","last_active":"2023-12-20T04:43:56.021143+0000","last_peered":"2023-12-20T04:43:56.021143+0000","last_clean":"2023-12-20T04:43:56.021143+0000","last_became_active":"2023-12-20T04:43:54.484048+0000","last_became_peered":"2023-12-20T04:43:54.484048+0000","last_unstale":"2023-12-20T04:43:56.021143+0000","last_undegraded":"2023-12-20T04:43:56.021143+0000","last_fullsized":"2023-12-20T04:43:56.021143+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:50.474860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.16","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882661+0000","last_change":"2023-12-20T04:43:54.896351+0000","last_active":"2023-12-20T04:43:55.882661+0000","last_peered":"2023-12-20T04:43:55.882661+0000","last_clean":"2023-12-20T04:43:55.882661+0000","last_became_active":"2023-12-20T04:43:54.895371+0000","last_became_peered":"2023-12-20T04:43:54.895371+0000","last_unstale":"2023-12-20T04:43:55.882661+0000","last_undegraded":"2023-12-20T04:43:55.882661+0000","last_fullsized":"2023-12-20T04:43:55.882661+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:11:13.259566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.17","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883861+0000","last_change":"2023-12-20T04:43:54.490790+0000","last_active":"2023-12-20T04:43:55.883861+0000","last_peered":"2023-12-20T04:43:55.883861+0000","last_clean":"2023-12-20T04:43:55.883861+0000","last_became_active":"2023-12-20T04:43:54.490501+0000","last_became_peered":"2023-12-20T04:43:54.490501+0000","last_unstale":"2023-12-20T04:43:55.883861+0000","last_undegraded":"2023-12-20T04:43:55.883861+0000","last_fullsized":"2023-12-20T04:43:55.883861+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:37:50.839793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.17","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884257+0000","last_change":"2023-12-20T04:43:54.912960+0000","last_active":"2023-12-20T04:43:55.884257+0000","last_peered":"2023-12-20T04:43:55.884257+0000","last_clean":"2023-12-20T04:43:55.884257+0000","last_became_active":"2023-12-20T04:43:54.912082+0000","last_became_peered":"2023-12-20T04:43:54.912082+0000","last_unstale":"2023-12-20T04:43:55.884257+0000","last_undegraded":"2023-12-20T04:43:55.884257+0000","last_fullsized":"2023-12-20T04:43:55.884257+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:55:36.350327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.16","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020876+0000","last_change":"2023-12-20T04:43:54.496566+0000","last_active":"2023-12-20T04:43:56.020876+0000","last_peered":"2023-12-20T04:43:56.020876+0000","last_clean":"2023-12-20T04:43:56.020876+0000","last_became_active":"2023-12-20T04:43:54.496205+0000","last_became_peered":"2023-12-20T04:43:54.496205+0000","last_unstale":"2023-12-20T04:43:56.020876+0000","last_undegraded":"2023-12-20T04:43:56.020876+0000","last_fullsized":"2023-12-20T04:43:56.020876+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:40.627636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.14","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020586+0000","last_change":"2023-12-20T04:43:54.911648+0000","last_active":"2023-12-20T04:43:56.020586+0000","last_peered":"2023-12-20T04:43:56.020586+0000","last_clean":"2023-12-20T04:43:56.020586+0000","last_became_active":"2023-12-20T04:43:54.911277+0000","last_became_peered":"2023-12-20T04:43:54.911277+0000","last_unstale":"2023-12-20T04:43:56.020586+0000","last_undegraded":"2023-12-20T04:43:56.020586+0000","last_fullsized":"2023-12-20T04:43:56.020586+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:04:39.085502+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.15","version":"92'3","reported_seq":14,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021381+0000","last_change":"2023-12-20T04:43:54.493272+0000","last_active":"2023-12-20T04:43:56.021381+0000","last_peered":"2023-12-20T04:43:56.021381+0000","last_clean":"2023-12-20T04:43:56.021381+0000","last_became_active":"2023-12-20T04:43:54.492958+0000","last_became_peered":"2023-12-20T04:43:54.492958+0000","last_unstale":"2023-12-20T04:43:56.021381+0000","last_undegraded":"2023-12-20T04:43:56.021381+0000","last_fullsized":"2023-12-20T04:43:56.021381+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:39:50.554454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"11.15","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021657+0000","last_change":"2023-12-20T04:43:54.899165+0000","last_active":"2023-12-20T04:43:56.021657+0000","last_peered":"2023-12-20T04:43:56.021657+0000","last_clean":"2023-12-20T04:43:56.021657+0000","last_became_active":"2023-12-20T04:43:54.898895+0000","last_became_peered":"2023-12-20T04:43:54.898895+0000","last_unstale":"2023-12-20T04:43:56.021657+0000","last_undegraded":"2023-12-20T04:43:56.021657+0000","last_fullsized":"2023-12-20T04:43:56.021657+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:25:53.406028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.14","version":"92'6","reported_seq":17,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020828+0000","last_change":"2023-12-20T04:43:54.487182+0000","last_active":"2023-12-20T04:43:56.020828+0000","last_peered":"2023-12-20T04:43:56.020828+0000","last_clean":"2023-12-20T04:43:56.020828+0000","last_became_active":"2023-12-20T04:43:54.486886+0000","last_became_peered":"2023-12-20T04:43:54.486886+0000","last_unstale":"2023-12-20T04:43:56.020828+0000","last_undegraded":"2023-12-20T04:43:56.020828+0000","last_fullsized":"2023-12-20T04:43:56.020828+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:25:22.485944+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.12","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883843+0000","last_change":"2023-12-20T04:43:54.902851+0000","last_active":"2023-12-20T04:43:55.883843+0000","last_peered":"2023-12-20T04:43:55.883843+0000","last_clean":"2023-12-20T04:43:55.883843+0000","last_became_active":"2023-12-20T04:43:54.902569+0000","last_became_peered":"2023-12-20T04:43:54.902569+0000","last_unstale":"2023-12-20T04:43:55.883843+0000","last_undegraded":"2023-12-20T04:43:55.883843+0000","last_fullsized":"2023-12-20T04:43:55.883843+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:56:23.814101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.13","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020539+0000","last_change":"2023-12-20T04:43:54.483144+0000","last_active":"2023-12-20T04:43:56.020539+0000","last_peered":"2023-12-20T04:43:56.020539+0000","last_clean":"2023-12-20T04:43:56.020539+0000","last_became_active":"2023-12-20T04:43:54.482862+0000","last_became_peered":"2023-12-20T04:43:54.482862+0000","last_unstale":"2023-12-20T04:43:56.020539+0000","last_undegraded":"2023-12-20T04:43:56.020539+0000","last_fullsized":"2023-12-20T04:43:56.020539+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:46.875378+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.13","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882291+0000","last_change":"2023-12-20T04:43:54.899424+0000","last_active":"2023-12-20T04:43:55.882291+0000","last_peered":"2023-12-20T04:43:55.882291+0000","last_clean":"2023-12-20T04:43:55.882291+0000","last_became_active":"2023-12-20T04:43:54.898559+0000","last_became_peered":"2023-12-20T04:43:54.898559+0000","last_unstale":"2023-12-20T04:43:55.882291+0000","last_undegraded":"2023-12-20T04:43:55.882291+0000","last_fullsized":"2023-12-20T04:43:55.882291+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:26:29.783539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.12","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020829+0000","last_change":"2023-12-20T04:43:54.497321+0000","last_active":"2023-12-20T04:43:56.020829+0000","last_peered":"2023-12-20T04:43:56.020829+0000","last_clean":"2023-12-20T04:43:56.020829+0000","last_became_active":"2023-12-20T04:43:54.497051+0000","last_became_peered":"2023-12-20T04:43:54.497051+0000","last_unstale":"2023-12-20T04:43:56.020829+0000","last_undegraded":"2023-12-20T04:43:56.020829+0000","last_fullsized":"2023-12-20T04:43:56.020829+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:14.415022+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.10","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880546+0000","last_change":"2023-12-20T04:43:54.906651+0000","last_active":"2023-12-20T04:43:55.880546+0000","last_peered":"2023-12-20T04:43:55.880546+0000","last_clean":"2023-12-20T04:43:55.880546+0000","last_became_active":"2023-12-20T04:43:54.906396+0000","last_became_peered":"2023-12-20T04:43:54.906396+0000","last_unstale":"2023-12-20T04:43:55.880546+0000","last_undegraded":"2023-12-20T04:43:55.880546+0000","last_fullsized":"2023-12-20T04:43:55.880546+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:20.532746+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.11","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881860+0000","last_change":"2023-12-20T04:43:54.490358+0000","last_active":"2023-12-20T04:43:55.881860+0000","last_peered":"2023-12-20T04:43:55.881860+0000","last_clean":"2023-12-20T04:43:55.881860+0000","last_became_active":"2023-12-20T04:43:54.489860+0000","last_became_peered":"2023-12-20T04:43:54.489860+0000","last_unstale":"2023-12-20T04:43:55.881860+0000","last_undegraded":"2023-12-20T04:43:55.881860+0000","last_fullsized":"2023-12-20T04:43:55.881860+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:50:45.132243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.11","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021410+0000","last_change":"2023-12-20T04:43:54.895379+0000","last_active":"2023-12-20T04:43:56.021410+0000","last_peered":"2023-12-20T04:43:56.021410+0000","last_clean":"2023-12-20T04:43:56.021410+0000","last_became_active":"2023-12-20T04:43:54.895092+0000","last_became_peered":"2023-12-20T04:43:54.895092+0000","last_unstale":"2023-12-20T04:43:56.021410+0000","last_undegraded":"2023-12-20T04:43:56.021410+0000","last_fullsized":"2023-12-20T04:43:56.021410+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:55.403045+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.10","version":"92'8","reported_seq":19,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882904+0000","last_change":"2023-12-20T04:43:54.490638+0000","last_active":"2023-12-20T04:43:55.882904+0000","last_peered":"2023-12-20T04:43:55.882904+0000","last_clean":"2023-12-20T04:43:55.882904+0000","last_became_active":"2023-12-20T04:43:54.490095+0000","last_became_peered":"2023-12-20T04:43:54.490095+0000","last_unstale":"2023-12-20T04:43:55.882904+0000","last_undegraded":"2023-12-20T04:43:55.882904+0000","last_fullsized":"2023-12-20T04:43:55.882904+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:33:24.912541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.e","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.883989+0000","last_change":"2023-12-20T04:43:54.891326+0000","last_active":"2023-12-20T04:43:55.883989+0000","last_peered":"2023-12-20T04:43:55.883989+0000","last_clean":"2023-12-20T04:43:55.883989+0000","last_became_active":"2023-12-20T04:43:54.891029+0000","last_became_peered":"2023-12-20T04:43:54.891029+0000","last_unstale":"2023-12-20T04:43:55.883989+0000","last_undegraded":"2023-12-20T04:43:55.883989+0000","last_fullsized":"2023-12-20T04:43:55.883989+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:42:10.876085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"10.f","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020693+0000","last_change":"2023-12-20T04:43:54.496914+0000","last_active":"2023-12-20T04:43:56.020693+0000","last_peered":"2023-12-20T04:43:56.020693+0000","last_clean":"2023-12-20T04:43:56.020693+0000","last_became_active":"2023-12-20T04:43:54.496523+0000","last_became_peered":"2023-12-20T04:43:54.496523+0000","last_unstale":"2023-12-20T04:43:56.020693+0000","last_undegraded":"2023-12-20T04:43:56.020693+0000","last_fullsized":"2023-12-20T04:43:56.020693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:01.919919+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.f","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884371+0000","last_change":"2023-12-20T04:43:54.899487+0000","last_active":"2023-12-20T04:43:55.884371+0000","last_peered":"2023-12-20T04:43:55.884371+0000","last_clean":"2023-12-20T04:43:55.884371+0000","last_became_active":"2023-12-20T04:43:54.899244+0000","last_became_peered":"2023-12-20T04:43:54.899244+0000","last_unstale":"2023-12-20T04:43:55.884371+0000","last_undegraded":"2023-12-20T04:43:55.884371+0000","last_fullsized":"2023-12-20T04:43:55.884371+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:55.171922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.e","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884089+0000","last_change":"2023-12-20T04:43:54.497865+0000","last_active":"2023-12-20T04:43:55.884089+0000","last_peered":"2023-12-20T04:43:55.884089+0000","last_clean":"2023-12-20T04:43:55.884089+0000","last_became_active":"2023-12-20T04:43:54.497558+0000","last_became_peered":"2023-12-20T04:43:54.497558+0000","last_unstale":"2023-12-20T04:43:55.884089+0000","last_undegraded":"2023-12-20T04:43:55.884089+0000","last_fullsized":"2023-12-20T04:43:55.884089+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:58:33.383247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.c","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020734+0000","last_change":"2023-12-20T04:43:54.905117+0000","last_active":"2023-12-20T04:43:56.020734+0000","last_peered":"2023-12-20T04:43:56.020734+0000","last_clean":"2023-12-20T04:43:56.020734+0000","last_became_active":"2023-12-20T04:43:54.904809+0000","last_became_peered":"2023-12-20T04:43:54.904809+0000","last_unstale":"2023-12-20T04:43:56.020734+0000","last_undegraded":"2023-12-20T04:43:56.020734+0000","last_fullsized":"2023-12-20T04:43:56.020734+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:07:55.899832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.d","version":"92'1","reported_seq":12,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020627+0000","last_change":"2023-12-20T04:43:54.500032+0000","last_active":"2023-12-20T04:43:56.020627+0000","last_peered":"2023-12-20T04:43:56.020627+0000","last_clean":"2023-12-20T04:43:56.020627+0000","last_became_active":"2023-12-20T04:43:54.499661+0000","last_became_peered":"2023-12-20T04:43:54.499661+0000","last_unstale":"2023-12-20T04:43:56.020627+0000","last_undegraded":"2023-12-20T04:43:56.020627+0000","last_fullsized":"2023-12-20T04:43:56.020627+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:32:00.859060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.d","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021785+0000","last_change":"2023-12-20T04:43:54.909151+0000","last_active":"2023-12-20T04:43:56.021785+0000","last_peered":"2023-12-20T04:43:56.021785+0000","last_clean":"2023-12-20T04:43:56.021785+0000","last_became_active":"2023-12-20T04:43:54.908881+0000","last_became_peered":"2023-12-20T04:43:54.908881+0000","last_unstale":"2023-12-20T04:43:56.021785+0000","last_undegraded":"2023-12-20T04:43:56.021785+0000","last_fullsized":"2023-12-20T04:43:56.021785+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:02.928334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.c","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884129+0000","last_change":"2023-12-20T04:43:54.492179+0000","last_active":"2023-12-20T04:43:55.884129+0000","last_peered":"2023-12-20T04:43:55.884129+0000","last_clean":"2023-12-20T04:43:55.884129+0000","last_became_active":"2023-12-20T04:43:54.491931+0000","last_became_peered":"2023-12-20T04:43:54.491931+0000","last_unstale":"2023-12-20T04:43:55.884129+0000","last_undegraded":"2023-12-20T04:43:55.884129+0000","last_fullsized":"2023-12-20T04:43:55.884129+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:19:49.788193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"11.b","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020898+0000","last_change":"2023-12-20T04:43:54.893670+0000","last_active":"2023-12-20T04:43:56.020898+0000","last_peered":"2023-12-20T04:43:56.020898+0000","last_clean":"2023-12-20T04:43:56.020898+0000","last_became_active":"2023-12-20T04:43:54.893375+0000","last_became_peered":"2023-12-20T04:43:54.893375+0000","last_unstale":"2023-12-20T04:43:56.020898+0000","last_undegraded":"2023-12-20T04:43:56.020898+0000","last_fullsized":"2023-12-20T04:43:56.020898+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:25:16.646926+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.a","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020858+0000","last_change":"2023-12-20T04:43:54.496541+0000","last_active":"2023-12-20T04:43:56.020858+0000","last_peered":"2023-12-20T04:43:56.020858+0000","last_clean":"2023-12-20T04:43:56.020858+0000","last_became_active":"2023-12-20T04:43:54.496176+0000","last_became_peered":"2023-12-20T04:43:54.496176+0000","last_unstale":"2023-12-20T04:43:56.020858+0000","last_undegraded":"2023-12-20T04:43:56.020858+0000","last_fullsized":"2023-12-20T04:43:56.020858+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:41:23.641303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.8","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020526+0000","last_change":"2023-12-20T04:43:54.914363+0000","last_active":"2023-12-20T04:43:56.020526+0000","last_peered":"2023-12-20T04:43:56.020526+0000","last_clean":"2023-12-20T04:43:56.020526+0000","last_became_active":"2023-12-20T04:43:54.914068+0000","last_became_peered":"2023-12-20T04:43:54.914068+0000","last_unstale":"2023-12-20T04:43:56.020526+0000","last_undegraded":"2023-12-20T04:43:56.020526+0000","last_fullsized":"2023-12-20T04:43:56.020526+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:03.869454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.9","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.881974+0000","last_change":"2023-12-20T04:43:54.487577+0000","last_active":"2023-12-20T04:43:55.881974+0000","last_peered":"2023-12-20T04:43:55.881974+0000","last_clean":"2023-12-20T04:43:55.881974+0000","last_became_active":"2023-12-20T04:43:54.487225+0000","last_became_peered":"2023-12-20T04:43:54.487225+0000","last_unstale":"2023-12-20T04:43:55.881974+0000","last_undegraded":"2023-12-20T04:43:55.881974+0000","last_fullsized":"2023-12-20T04:43:55.881974+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:33:44.854838+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.9","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020695+0000","last_change":"2023-12-20T04:43:54.912286+0000","last_active":"2023-12-20T04:43:56.020695+0000","last_peered":"2023-12-20T04:43:56.020695+0000","last_clean":"2023-12-20T04:43:56.020695+0000","last_became_active":"2023-12-20T04:43:54.912017+0000","last_became_peered":"2023-12-20T04:43:54.912017+0000","last_unstale":"2023-12-20T04:43:56.020695+0000","last_undegraded":"2023-12-20T04:43:56.020695+0000","last_fullsized":"2023-12-20T04:43:56.020695+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:21.719783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.8","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880528+0000","last_change":"2023-12-20T04:43:54.485973+0000","last_active":"2023-12-20T04:43:55.880528+0000","last_peered":"2023-12-20T04:43:55.880528+0000","last_clean":"2023-12-20T04:43:55.880528+0000","last_became_active":"2023-12-20T04:43:54.485587+0000","last_became_peered":"2023-12-20T04:43:54.485587+0000","last_unstale":"2023-12-20T04:43:55.880528+0000","last_undegraded":"2023-12-20T04:43:55.880528+0000","last_fullsized":"2023-12-20T04:43:55.880528+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:54.163507+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.6","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880961+0000","last_change":"2023-12-20T04:43:54.896336+0000","last_active":"2023-12-20T04:43:55.880961+0000","last_peered":"2023-12-20T04:43:55.880961+0000","last_clean":"2023-12-20T04:43:55.880961+0000","last_became_active":"2023-12-20T04:43:54.896033+0000","last_became_peered":"2023-12-20T04:43:54.896033+0000","last_unstale":"2023-12-20T04:43:55.880961+0000","last_undegraded":"2023-12-20T04:43:55.880961+0000","last_fullsized":"2023-12-20T04:43:55.880961+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:10:23.142165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"10.7","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020744+0000","last_change":"2023-12-20T04:43:54.495784+0000","last_active":"2023-12-20T04:43:56.020744+0000","last_peered":"2023-12-20T04:43:56.020744+0000","last_clean":"2023-12-20T04:43:56.020744+0000","last_became_active":"2023-12-20T04:43:54.494894+0000","last_became_peered":"2023-12-20T04:43:54.494894+0000","last_unstale":"2023-12-20T04:43:56.020744+0000","last_undegraded":"2023-12-20T04:43:56.020744+0000","last_fullsized":"2023-12-20T04:43:56.020744+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:50.470667+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.7","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882757+0000","last_change":"2023-12-20T04:43:54.893303+0000","last_active":"2023-12-20T04:43:55.882757+0000","last_peered":"2023-12-20T04:43:55.882757+0000","last_clean":"2023-12-20T04:43:55.882757+0000","last_became_active":"2023-12-20T04:43:54.893044+0000","last_became_peered":"2023-12-20T04:43:54.893044+0000","last_unstale":"2023-12-20T04:43:55.882757+0000","last_undegraded":"2023-12-20T04:43:55.882757+0000","last_fullsized":"2023-12-20T04:43:55.882757+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:29:44.222794+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"10.6","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882746+0000","last_change":"2023-12-20T04:43:54.490980+0000","last_active":"2023-12-20T04:43:55.882746+0000","last_peered":"2023-12-20T04:43:55.882746+0000","last_clean":"2023-12-20T04:43:55.882746+0000","last_became_active":"2023-12-20T04:43:54.489980+0000","last_became_peered":"2023-12-20T04:43:54.489980+0000","last_unstale":"2023-12-20T04:43:55.882746+0000","last_undegraded":"2023-12-20T04:43:55.882746+0000","last_fullsized":"2023-12-20T04:43:55.882746+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:32:30.324229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"11.4","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884071+0000","last_change":"2023-12-20T04:43:54.914434+0000","last_active":"2023-12-20T04:43:55.884071+0000","last_peered":"2023-12-20T04:43:55.884071+0000","last_clean":"2023-12-20T04:43:55.884071+0000","last_became_active":"2023-12-20T04:43:54.914174+0000","last_became_peered":"2023-12-20T04:43:54.914174+0000","last_unstale":"2023-12-20T04:43:55.884071+0000","last_undegraded":"2023-12-20T04:43:55.884071+0000","last_fullsized":"2023-12-20T04:43:55.884071+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:20.099541+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.5","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021775+0000","last_change":"2023-12-20T04:43:54.488403+0000","last_active":"2023-12-20T04:43:56.021775+0000","last_peered":"2023-12-20T04:43:56.021775+0000","last_clean":"2023-12-20T04:43:56.021775+0000","last_became_active":"2023-12-20T04:43:54.488146+0000","last_became_peered":"2023-12-20T04:43:54.488146+0000","last_unstale":"2023-12-20T04:43:56.021775+0000","last_undegraded":"2023-12-20T04:43:56.021775+0000","last_fullsized":"2023-12-20T04:43:56.021775+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:46:16.006131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.5","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.884340+0000","last_change":"2023-12-20T04:43:54.906949+0000","last_active":"2023-12-20T04:43:55.884340+0000","last_peered":"2023-12-20T04:43:55.884340+0000","last_clean":"2023-12-20T04:43:55.884340+0000","last_became_active":"2023-12-20T04:43:54.906694+0000","last_became_peered":"2023-12-20T04:43:54.906694+0000","last_unstale":"2023-12-20T04:43:55.884340+0000","last_undegraded":"2023-12-20T04:43:55.884340+0000","last_fullsized":"2023-12-20T04:43:55.884340+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:03:15.497172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"10.4","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.880867+0000","last_change":"2023-12-20T04:43:54.490904+0000","last_active":"2023-12-20T04:43:55.880867+0000","last_peered":"2023-12-20T04:43:55.880867+0000","last_clean":"2023-12-20T04:43:55.880867+0000","last_became_active":"2023-12-20T04:43:54.490598+0000","last_became_peered":"2023-12-20T04:43:54.490598+0000","last_unstale":"2023-12-20T04:43:55.880867+0000","last_undegraded":"2023-12-20T04:43:55.880867+0000","last_fullsized":"2023-12-20T04:43:55.880867+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:51:47.823844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"11.2","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021599+0000","last_change":"2023-12-20T04:43:54.895461+0000","last_active":"2023-12-20T04:43:56.021599+0000","last_peered":"2023-12-20T04:43:56.021599+0000","last_clean":"2023-12-20T04:43:56.021599+0000","last_became_active":"2023-12-20T04:43:54.895162+0000","last_became_peered":"2023-12-20T04:43:54.895162+0000","last_unstale":"2023-12-20T04:43:56.021599+0000","last_undegraded":"2023-12-20T04:43:56.021599+0000","last_fullsized":"2023-12-20T04:43:56.021599+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:15.423438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"10.3","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.021654+0000","last_change":"2023-12-20T04:43:54.488446+0000","last_active":"2023-12-20T04:43:56.021654+0000","last_peered":"2023-12-20T04:43:56.021654+0000","last_clean":"2023-12-20T04:43:56.021654+0000","last_became_active":"2023-12-20T04:43:54.487672+0000","last_became_peered":"2023-12-20T04:43:54.487672+0000","last_unstale":"2023-12-20T04:43:56.021654+0000","last_undegraded":"2023-12-20T04:43:56.021654+0000","last_fullsized":"2023-12-20T04:43:56.021654+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:15:47.225316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"11.3","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020639+0000","last_change":"2023-12-20T04:43:54.910186+0000","last_active":"2023-12-20T04:43:56.020639+0000","last_peered":"2023-12-20T04:43:56.020639+0000","last_clean":"2023-12-20T04:43:56.020639+0000","last_became_active":"2023-12-20T04:43:54.909930+0000","last_became_peered":"2023-12-20T04:43:54.909930+0000","last_unstale":"2023-12-20T04:43:56.020639+0000","last_undegraded":"2023-12-20T04:43:56.020639+0000","last_fullsized":"2023-12-20T04:43:56.020639+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:32:45.309358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.2","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020291+0000","last_change":"2023-12-20T04:43:54.492211+0000","last_active":"2023-12-20T04:43:56.020291+0000","last_peered":"2023-12-20T04:43:56.020291+0000","last_clean":"2023-12-20T04:43:56.020291+0000","last_became_active":"2023-12-20T04:43:54.491943+0000","last_became_peered":"2023-12-20T04:43:54.491943+0000","last_unstale":"2023-12-20T04:43:56.020291+0000","last_undegraded":"2023-12-20T04:43:56.020291+0000","last_fullsized":"2023-12-20T04:43:56.020291+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:50:47.895652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"11.0","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020328+0000","last_change":"2023-12-20T04:43:54.889864+0000","last_active":"2023-12-20T04:43:56.020328+0000","last_peered":"2023-12-20T04:43:56.020328+0000","last_clean":"2023-12-20T04:43:56.020328+0000","last_became_active":"2023-12-20T04:43:54.889445+0000","last_became_peered":"2023-12-20T04:43:54.889445+0000","last_unstale":"2023-12-20T04:43:56.020328+0000","last_undegraded":"2023-12-20T04:43:56.020328+0000","last_fullsized":"2023-12-20T04:43:56.020328+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:53:32.413782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"10.1","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:55.882959+0000","last_change":"2023-12-20T04:43:54.491775+0000","last_active":"2023-12-20T04:43:55.882959+0000","last_peered":"2023-12-20T04:43:55.882959+0000","last_clean":"2023-12-20T04:43:55.882959+0000","last_became_active":"2023-12-20T04:43:54.490027+0000","last_became_peered":"2023-12-20T04:43:54.490027+0000","last_unstale":"2023-12-20T04:43:55.882959+0000","last_undegraded":"2023-12-20T04:43:55.882959+0000","last_fullsized":"2023-12-20T04:43:55.882959+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:07:11.323823+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"11.1","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020242+0000","last_change":"2023-12-20T04:43:54.911274+0000","last_active":"2023-12-20T04:43:56.020242+0000","last_peered":"2023-12-20T04:43:56.020242+0000","last_clean":"2023-12-20T04:43:56.020242+0000","last_became_active":"2023-12-20T04:43:54.910998+0000","last_became_peered":"2023-12-20T04:43:54.910998+0000","last_unstale":"2023-12-20T04:43:56.020242+0000","last_undegraded":"2023-12-20T04:43:56.020242+0000","last_fullsized":"2023-12-20T04:43:56.020242+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:26:39.723104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"10.0","version":"0'0","reported_seq":11,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020454+0000","last_change":"2023-12-20T04:43:54.496582+0000","last_active":"2023-12-20T04:43:56.020454+0000","last_peered":"2023-12-20T04:43:56.020454+0000","last_clean":"2023-12-20T04:43:56.020454+0000","last_became_active":"2023-12-20T04:43:54.496123+0000","last_became_peered":"2023-12-20T04:43:54.496123+0000","last_unstale":"2023-12-20T04:43:56.020454+0000","last_undegraded":"2023-12-20T04:43:56.020454+0000","last_fullsized":"2023-12-20T04:43:56.020454+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:29:22.619298+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"11.a","version":"0'0","reported_seq":10,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020450+0000","last_change":"2023-12-20T04:43:54.910411+0000","last_active":"2023-12-20T04:43:56.020450+0000","last_peered":"2023-12-20T04:43:56.020450+0000","last_clean":"2023-12-20T04:43:56.020450+0000","last_became_active":"2023-12-20T04:43:54.910125+0000","last_became_peered":"2023-12-20T04:43:54.910125+0000","last_unstale":"2023-12-20T04:43:56.020450+0000","last_undegraded":"2023-12-20T04:43:56.020450+0000","last_fullsized":"2023-12-20T04:43:56.020450+0000","mapping_epoch":91,"log_start":"0'0","ondisk_log_start":"0'0","created":91,"last_epoch_clean":92,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:54.457608+0000","last_clean_scrub_stamp":"2023-12-20T04:43:54.457608+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:07:38.138240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"10.b","version":"92'2","reported_seq":13,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.020508+0000","last_change":"2023-12-20T04:43:54.495757+0000","last_active":"2023-12-20T04:43:56.020508+0000","last_peered":"2023-12-20T04:43:56.020508+0000","last_clean":"2023-12-20T04:43:56.020508+0000","last_became_active":"2023-12-20T04:43:54.495389+0000","last_became_peered":"2023-12-20T04:43:54.495389+0000","last_unstale":"2023-12-20T04:43:56.020508+0000","last_undegraded":"2023-12-20T04:43:56.020508+0000","last_fullsized":"2023-12-20T04:43:56.020508+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":90,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:43:53.449192+0000","last_clean_scrub_stamp":"2023-12-20T04:43:53.449192+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:57:06.845778+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"93'2153","reported_seq":2403,"reported_epoch":93,"state":"active+clean","last_fresh":"2023-12-20T04:43:56.232416+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:43:56.232416+0000","last_peered":"2023-12-20T04:43:56.232416+0000","last_clean":"2023-12-20T04:43:56.232416+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:43:56.232416+0000","last_undegraded":"2023-12-20T04:43:56.232416+0000","last_fullsized":"2023-12-20T04:43:56.232416+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":2153,"log_dups_size":0,"ondisk_log_size":2153,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":532,"num_read_kb":954,"num_write":241,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":11,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":10,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":532,"num_read_kb":954,"num_write":241,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2153,"ondisk_log_size":2153,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575034,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28648,"kb_used_data":1696,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343192,"statfs":{"total":96636764160,"available":96607428608,"internally_reserved":0,"allocated":1736704,"data_stored":1124760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93899999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.29799999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84499999999999997,"5min":1,"15min":1.115},"last":0.47499999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":1.095,"15min":1.2190000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91700000000000004,"5min":1.032,"15min":2.649},"last":0.44}]},{"osd":2,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.68999999999999995,"15min":0.71099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89300000000000002,"5min":1.075,"15min":1.5960000000000001},"last":0.379},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.74299999999999999,"15min":0.749},"min":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.45300000000000001},"max":{"1min":0.98999999999999999,"5min":1.1000000000000001,"15min":1.135},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.70099999999999996,"15min":0.72599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.641},"last":0.42499999999999999},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93500000000000005,"5min":1.131,"15min":2.5990000000000002},"last":0.61499999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34599999999999997},"max":{"1min":0.78500000000000003,"5min":0.90700000000000003,"15min":1.0109999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.624,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79500000000000004,"5min":1.0469999999999999,"15min":1.242},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.438,"5min":0.433,"15min":0.40400000000000003},"max":{"1min":0.86299999999999999,"5min":1.089,"15min":1.0900000000000001},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.432},"max":{"1min":0.83199999999999996,"5min":0.98799999999999999,"15min":1.071},"last":0.53500000000000003}]},{"osd":6,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.443},"max":{"1min":0.99199999999999999,"5min":1.093,"15min":1.1279999999999999},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44400000000000001},"max":{"1min":0.96799999999999997,"5min":1.125,"15min":1.125},"last":0.46100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575034,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1142784,"data_stored":533868,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.71699999999999997,"15min":0.71499999999999997},"min":{"1min":0.55600000000000005,"5min":0.33800000000000002,"15min":0.27100000000000002},"max":{"1min":0.92900000000000005,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.57799999999999996},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.71999999999999997,"15min":0.71699999999999997},"min":{"1min":0.626,"5min":0.41299999999999998,"15min":0.33700000000000002},"max":{"1min":1.04,"5min":1.04,"15min":1.1910000000000001},"last":0.53700000000000003}]},{"osd":1,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.69099999999999995,"15min":0.70199999999999996},"min":{"1min":0.55100000000000005,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":0.502},{"interface":"front","average":{"1min":0.77400000000000002,"5min":0.73899999999999999,"15min":0.72599999999999998},"min":{"1min":0.57699999999999996,"5min":0.38800000000000001,"15min":0.36599999999999999},"max":{"1min":0.97799999999999998,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.38200000000000001}]},{"osd":2,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.76300000000000001,"15min":0.76100000000000001},"min":{"1min":0.68400000000000005,"5min":0.40899999999999997,"15min":0.40899999999999997},"max":{"1min":1.006,"5min":1.216,"15min":1.216},"last":0.52000000000000002},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73399999999999999,"15min":0.73099999999999998},"min":{"1min":0.55400000000000005,"5min":0.48099999999999998,"15min":0.38300000000000001},"max":{"1min":0.95599999999999996,"5min":1.0089999999999999,"15min":1.054},"last":0.58999999999999997}]},{"osd":3,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.83799999999999997,"5min":0.78500000000000003,"15min":0.75900000000000001},"min":{"1min":0.68300000000000005,"5min":0.51800000000000002,"15min":0.45600000000000002},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.109},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.72399999999999998,"15min":0.73699999999999999},"min":{"1min":0.50900000000000001,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.96499999999999997,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.60299999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.624,"15min":0.621},"min":{"1min":0.437,"5min":0.39900000000000002,"15min":0.36399999999999999},"max":{"1min":0.90300000000000002,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.63700000000000001,"15min":0.63700000000000001},"min":{"1min":0.46999999999999997,"5min":0.39100000000000001,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":0.995,"15min":1.071},"last":0.63200000000000001}]},{"osd":5,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.63800000000000001,"15min":0.63400000000000001},"min":{"1min":0.51000000000000001,"5min":0.39400000000000002,"15min":0.39400000000000002},"max":{"1min":0.871,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.42699999999999999},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69499999999999995,"15min":0.68999999999999995},"min":{"1min":0.54400000000000004,"5min":0.435,"15min":0.434},"max":{"1min":1,"5min":1.306,"15min":1.306},"last":0.61599999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:42:56 2023","interfaces":[{"interface":"back","average":{"1min":0.77200000000000002,"5min":0.72599999999999998,"15min":0.72299999999999998},"min":{"1min":0.61899999999999999,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":1.0129999999999999,"5min":1.0129999999999999,"15min":1.2270000000000001},"last":0.66600000000000004},{"interface":"front","average":{"1min":0.747,"5min":0.72199999999999998,"15min":0.71699999999999997},"min":{"1min":0.58899999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.90200000000000002,"5min":1.002,"15min":1.169},"last":0.55400000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575035,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36256,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335584,"statfs":{"total":96636764160,"available":96599638016,"internally_reserved":0,"allocated":1138688,"data_stored":533866,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68700000000000006,"15min":0.69299999999999995},"min":{"1min":0.36199999999999999,"5min":0.33700000000000002,"15min":0.29999999999999999},"max":{"1min":0.83099999999999996,"5min":0.97199999999999998,"15min":1.05},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68999999999999995,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.34300000000000003},"max":{"1min":0.877,"5min":0.95099999999999996,"15min":0.98599999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.441},"max":{"1min":0.88600000000000001,"5min":1.044,"15min":1.0600000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70999999999999996,"15min":0.73299999999999998},"min":{"1min":0.437,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95199999999999996,"5min":0.96999999999999997,"15min":1.1140000000000001},"last":0.53200000000000003}]},{"osd":2,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.72699999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.94399999999999995,"15min":1.153},"last":0.54700000000000004},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71399999999999997,"15min":0.72299999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":1.004},"last":0.56499999999999995}]},{"osd":3,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92500000000000004,"5min":0.95999999999999996,"15min":1.758},"last":0.624},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73299999999999998,"15min":0.748},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90900000000000003,"5min":0.96599999999999997,"15min":1.0469999999999999},"last":0.59599999999999997}]},{"osd":4,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.40600000000000003,"15min":0.35099999999999998},"max":{"1min":0.84799999999999998,"5min":0.97299999999999998,"15min":1.0229999999999999},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.39600000000000002},"max":{"1min":0.94299999999999995,"5min":0.98699999999999999,"15min":1.077},"last":0.65900000000000003}]},{"osd":6,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68400000000000005,"15min":0.68000000000000005},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.39900000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":1.0700000000000001},"last":0.47599999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.87,"5min":0.999,"15min":0.999},"last":0.51400000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.42999999999999999},"max":{"1min":0.90000000000000002,"5min":1.0429999999999999,"15min":1.141},"last":0.48399999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.751},"min":{"1min":0.57599999999999996,"5min":0.54900000000000004,"15min":0.47899999999999998},"max":{"1min":0.92000000000000004,"5min":1.012,"15min":1.119},"last":0.5}]}]},{"osd":4,"up_from":13,"seq":55834575036,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58388,"kb_used_data":1100,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313452,"statfs":{"total":96636764160,"available":96576974848,"internally_reserved":0,"allocated":1126400,"data_stored":527546,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":0.69799999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90900000000000003,"5min":1.024,"15min":1.5509999999999999},"last":0.28699999999999998},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68400000000000005,"15min":0.70499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92400000000000004,"5min":1.625,"15min":1.633},"last":0.47999999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89400000000000002,"5min":0.98799999999999999,"15min":1.798},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96399999999999997,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.55100000000000005}]},{"osd":2,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.98099999999999998,"5min":1.52,"15min":1.728},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.47999999999999998,"5min":0.46899999999999997,"15min":0.38700000000000001},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.6619999999999999},"last":0.505}]},{"osd":3,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":1.554,"15min":1.5760000000000001},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.94599999999999995,"5min":1.03,"15min":1.7649999999999999},"last":0.57399999999999995}]},{"osd":5,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63800000000000001,"15min":0.624},"min":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":1.1120000000000001,"5min":1.456,"15min":1.456},"last":0.436},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.077,"5min":1.589,"15min":1.589},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.47199999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.6739999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.58599999999999997}]},{"osd":7,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.47099999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.036,"5min":1.655,"15min":1.655},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.72099999999999997,"15min":0.73299999999999998},"min":{"1min":0.50700000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0029999999999999,"5min":1.052,"15min":1.125},"last":0.60899999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575035,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36260,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335580,"statfs":{"total":96636764160,"available":96599633920,"internally_reserved":0,"allocated":1142784,"data_stored":533888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.26900000000000002,"5min":0.26200000000000001,"15min":0.23200000000000001},"max":{"1min":0.748,"5min":0.875,"15min":4.3700000000000001},"last":0.38400000000000001},{"interface":"front","average":{"1min":0.44,"5min":0.499,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63100000000000001,"5min":0.80800000000000005,"15min":3.355},"last":0.28899999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.48499999999999999},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68200000000000005,"5min":0.73999999999999999,"15min":3.7160000000000002},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.53300000000000003,"15min":0.55700000000000005},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.28399999999999997},"max":{"1min":0.70199999999999996,"5min":0.74399999999999999,"15min":3.3109999999999999},"last":0.55600000000000005}]},{"osd":2,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.374,"5min":0.318,"15min":0.318},"max":{"1min":0.72799999999999998,"5min":0.88700000000000001,"15min":4.0110000000000001},"last":0.54300000000000004},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.316},"max":{"1min":0.81799999999999995,"5min":0.93300000000000005,"15min":6.1399999999999997},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.76800000000000002,"15min":0.78500000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99199999999999999,"5min":1.131,"15min":1.3089999999999999},"last":0.499},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78300000000000003,"15min":0.81799999999999995},"min":{"1min":0.52300000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93799999999999994,"5min":1.4059999999999999,"15min":3.9510000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80800000000000005,"15min":0.82899999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":1.0649999999999999,"5min":1.3300000000000001,"15min":3.9830000000000001},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.76300000000000001,"15min":0.80200000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.38400000000000001},"max":{"1min":0.88100000000000001,"5min":1.252,"15min":4.6550000000000002},"last":0.60099999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76000000000000001,"15min":0.79800000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.45600000000000002},"max":{"1min":0.95199999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.82999999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.499},"max":{"1min":1.0149999999999999,"5min":1.2270000000000001,"15min":3.8900000000000001},"last":0.65100000000000002}]},{"osd":7,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.76900000000000002,"15min":0.81399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":1.1319999999999999,"15min":4.6970000000000001},"last":0.52400000000000002},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.79100000000000004,"15min":0.81999999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":1.29,"15min":1.29},"last":0.57899999999999996}]}]},{"osd":2,"up_from":13,"seq":55834575034,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40348,"kb_used_data":1108,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331492,"statfs":{"total":96636764160,"available":96595447808,"internally_reserved":0,"allocated":1134592,"data_stored":529040,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.433,"15min":0.439},"min":{"1min":0.32800000000000001,"5min":0.216,"15min":0.216},"max":{"1min":0.625,"5min":0.78900000000000003,"15min":0.89600000000000002},"last":0.27700000000000002},{"interface":"front","average":{"1min":0.54400000000000004,"5min":0.502,"15min":0.48899999999999999},"min":{"1min":0.39300000000000002,"5min":0.28499999999999998,"15min":0.26700000000000002},"max":{"1min":0.748,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.32500000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.47799999999999998,"15min":0.46899999999999997},"min":{"1min":0.36499999999999999,"5min":0.30199999999999999,"15min":0.28899999999999998},"max":{"1min":0.64900000000000002,"5min":0.876,"15min":0.876},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.38800000000000001,"5min":0.32000000000000001,"15min":0.28100000000000003},"max":{"1min":0.65900000000000003,"5min":0.88600000000000001,"15min":1.01},"last":0.45700000000000002}]},{"osd":3,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.59999999999999998,"15min":0.59499999999999997},"min":{"1min":0.49299999999999999,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.89800000000000002,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.504},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.63500000000000001,"15min":0.63900000000000001},"min":{"1min":0.49299999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.91600000000000004,"5min":0.95599999999999996,"15min":1.0900000000000001},"last":0.60099999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81399999999999995,"5min":0.78000000000000003,"15min":0.78700000000000003},"min":{"1min":0.69199999999999995,"5min":0.46500000000000002,"15min":0.45200000000000001},"max":{"1min":1.01,"5min":1.145,"15min":1.1799999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76600000000000001,"15min":0.77900000000000003},"min":{"1min":0.61399999999999999,"5min":0.44500000000000001,"15min":0.41399999999999998},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.2010000000000001},"last":0.58599999999999997}]},{"osd":5,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.76300000000000001,"15min":0.77600000000000002},"min":{"1min":0.625,"5min":0.499,"15min":0.48499999999999999},"max":{"1min":0.97199999999999998,"5min":1.1220000000000001,"15min":1.327},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.78700000000000003,"15min":0.80500000000000005},"min":{"1min":0.70999999999999996,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.0249999999999999,"5min":1.0449999999999999,"15min":1.381},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.78800000000000003,"15min":0.79500000000000004},"min":{"1min":0.59399999999999997,"5min":0.436,"15min":0.436},"max":{"1min":0.995,"5min":1.151,"15min":1.2809999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.78100000000000003,"15min":0.79800000000000004},"min":{"1min":0.65500000000000003,"5min":0.51000000000000001,"15min":0.46100000000000002},"max":{"1min":0.91900000000000004,"5min":1.0609999999999999,"15min":1.153},"last":0.74199999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:42:58 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.81299999999999994,"15min":0.82499999999999996},"min":{"1min":0.72399999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.038,"5min":1.107,"15min":1.3520000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.85199999999999998,"5min":0.83499999999999996,"15min":0.85099999999999998},"min":{"1min":0.73099999999999998,"5min":0.61699999999999999,"15min":0.504},"max":{"1min":1.0169999999999999,"5min":1.1220000000000001,"15min":1.401},"last":0.68999999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575036,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50224,"kb_used_data":1128,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321616,"statfs":{"total":96636764160,"available":96585334784,"internally_reserved":0,"allocated":1155072,"data_stored":535956,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.44700000000000001,"15min":0.44700000000000001},"min":{"1min":0.28599999999999998,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.84499999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.38100000000000001},{"interface":"front","average":{"1min":0.51000000000000001,"5min":0.48599999999999999,"15min":0.48999999999999999},"min":{"1min":0.35899999999999999,"5min":0.29699999999999999,"15min":0.28599999999999998},"max":{"1min":0.73999999999999999,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.30499999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.502,"5min":0.52000000000000002,"15min":0.51600000000000001},"min":{"1min":0.36099999999999999,"5min":0.34699999999999998,"15min":0.32800000000000001},"max":{"1min":0.627,"5min":1.091,"15min":1.091},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.55000000000000004,"5min":0.53000000000000003,"15min":0.54300000000000004},"min":{"1min":0.434,"5min":0.36399999999999999,"15min":0.33900000000000002},"max":{"1min":0.69899999999999995,"5min":0.85899999999999999,"15min":0.91300000000000003},"last":0.40799999999999997}]},{"osd":3,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.59299999999999997,"15min":0.59199999999999997},"min":{"1min":0.502,"5min":0.40100000000000002,"15min":0.36599999999999999},"max":{"1min":0.77100000000000002,"5min":0.96499999999999997,"15min":1.016},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.53800000000000003,"5min":0.52900000000000003,"15min":0.52600000000000002},"min":{"1min":0.40300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.82399999999999995,"5min":1.2410000000000001,"15min":1.2410000000000001},"last":0.42799999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.75800000000000001,"15min":0.77300000000000002},"min":{"1min":0.63500000000000001,"5min":0.45000000000000001,"15min":0.45000000000000001},"max":{"1min":0.93100000000000005,"5min":1.27,"15min":1.4419999999999999},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.80200000000000005,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.63,"5min":0.434,"15min":0.434},"max":{"1min":0.98399999999999999,"5min":1.1160000000000001,"15min":1.3380000000000001},"last":0.51600000000000001}]},{"osd":5,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.78100000000000003,"15min":0.77600000000000002},"min":{"1min":0.60499999999999998,"5min":0.54500000000000004,"15min":0.5},"max":{"1min":1.0229999999999999,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.498},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.76700000000000002,"15min":0.79000000000000004},"min":{"1min":0.61099999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.88100000000000001,"5min":1.2450000000000001,"15min":1.3009999999999999},"last":0.48399999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.78500000000000003,"15min":0.79600000000000004},"min":{"1min":0.68400000000000005,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0449999999999999,"5min":1.194,"15min":1.415},"last":0.55500000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.78800000000000003,"15min":0.79200000000000004},"min":{"1min":0.58799999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1,"5min":1.252,"15min":1.268},"last":0.46700000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:43:01 2023","interfaces":[{"interface":"back","average":{"1min":0.85599999999999998,"5min":0.82599999999999996,"15min":0.83999999999999997},"min":{"1min":0.73699999999999999,"5min":0.56699999999999995,"15min":0.56699999999999995},"max":{"1min":1.0580000000000001,"5min":1.306,"15min":1.399},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.83499999999999996,"15min":0.84199999999999997},"min":{"1min":0.66800000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.0349999999999999,"5min":1.3440000000000001,"15min":1.373},"last":0.61499999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575035,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40928,"kb_used_data":1688,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94330912,"statfs":{"total":96636764160,"available":96594853888,"internally_reserved":0,"allocated":1728512,"data_stored":1124144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.46700000000000003,"15min":0.44600000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":0.83699999999999997,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.30399999999999999},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.59599999999999997,"15min":0.53600000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.26900000000000002},"max":{"1min":0.70099999999999996,"5min":7.8860000000000001,"15min":7.8860000000000001},"last":0.64800000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.28399999999999997},"max":{"1min":0.75600000000000001,"5min":2.528,"15min":2.528},"last":0.39900000000000002},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71399999999999997,"5min":0.83499999999999996,"15min":1.2989999999999999},"last":0.44900000000000001}]},{"osd":3,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.52400000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29499999999999998},"max":{"1min":0.80000000000000004,"5min":3.1000000000000001,"15min":3.1000000000000001},"last":0.35099999999999998},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.68799999999999994,"15min":0.61899999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70899999999999996,"5min":10.49,"15min":10.49},"last":0.47199999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.78600000000000003,"15min":0.79300000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0780000000000001,"5min":3.0659999999999998,"15min":3.7469999999999999},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.83199999999999996,"15min":0.80300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.181,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.61099999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.79200000000000004,"15min":0.79800000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.44700000000000001},"max":{"1min":1.161,"5min":1.3109999999999999,"15min":1.877},"last":0.51400000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0349999999999999,"5min":1.2010000000000001,"15min":1.7450000000000001},"last":0.68799999999999994}]},{"osd":6,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.84499999999999997,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.47299999999999998},"max":{"1min":1.0069999999999999,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.52800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2150000000000001,"5min":1.306,"15min":1.492},"last":0.55200000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0740000000000001,"5min":1.2549999999999999,"15min":1.6539999999999999},"last":0.71499999999999997},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.81200000000000006,"15min":0.81299999999999994},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"max":{"1min":1.01,"5min":1.276,"15min":2.113},"last":0.48999999999999999}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2180,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":92,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":10,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":11,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:43:59.420 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T04:43:59.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T04:43:59.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847ee95640 1 Processor -- start 2023-12-20T04:43:59.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847ee95640 1 -- start start 2023-12-20T04:43:59.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847ee95640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815a940 0x7f84780cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847ee95640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84780cb1c0 0x7f84780cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847ee95640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f84780cbb50 con 0x7f847815ad20 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847ee95640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f84780cc9d0 con 0x7f84780cb1c0 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847ee95640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84780c9b90 con 0x7f847815a940 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847d432640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84780cb1c0 0x7f84780cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f8477fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f847815ad20 0x7f84780ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:49196/0 (socket says 172.21.15.32:49196) 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847cc31640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815a940 0x7f84780cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f8477fff640 1 -- 172.21.15.32:0/136845268 learned_addr learned my addr 172.21.15.32:0/136845268 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847d432640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f84780cb1c0 0x7f84780cb610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42654/0 (socket says 172.21.15.32:42654) 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847d432640 1 -- 172.21.15.32:0/136845268 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815a940 msgr2=0x7f84780cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847d432640 1 --2- 172.21.15.32:0/136845268 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815a940 0x7f84780cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847d432640 1 -- 172.21.15.32:0/136845268 >> v1:172.21.15.39:6789/0 conn(0x7f847815ad20 legacy=0x7f84780ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f847d432640 1 -- 172.21.15.32:0/136845268 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f84780c9e70 con 0x7f84780cb1c0 2023-12-20T04:43:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.617+0000 7f84777fe640 1 -- 172.21.15.32:0/136845268 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3704059646 0 0) 0x7f8464001120 con 0x7f847815ad20 2023-12-20T04:43:59.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f847d432640 1 --2- 172.21.15.32:0/136845268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cb1c0 0x7f84780cb610 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8470009d90 tx=0x7f847001c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cda37da39a905f2f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:59.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f84777fe640 1 -- 172.21.15.32:0/136845268 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700a59e0 con 0x7f84780cb1c0 2023-12-20T04:43:59.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f84777fe640 1 -- 172.21.15.32:0/136845268 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84700a5b40 con 0x7f84780cb1c0 2023-12-20T04:43:59.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f84777fe640 1 -- 172.21.15.32:0/136845268 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700ad440 con 0x7f84780cb1c0 2023-12-20T04:43:59.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f847ee95640 1 -- 172.21.15.32:0/136845268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cb1c0 msgr2=0x7f84780cb610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f847ee95640 1 --2- 172.21.15.32:0/136845268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cb1c0 0x7f84780cb610 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7f8470009d90 tx=0x7f847001c920 comp rx=0 tx=0).stop 2023-12-20T04:43:59.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f847ee95640 1 -- 172.21.15.32:0/136845268 shutdown_connections 2023-12-20T04:43:59.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f847ee95640 1 --2- 172.21.15.32:0/136845268 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f84780cb1c0 0x7f84780cb610 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f847ee95640 1 --2- 172.21.15.32:0/136845268 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815a940 0x7f84780cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f847ee95640 1 -- 172.21.15.32:0/136845268 >> 172.21.15.32:0/136845268 conn(0x7f847806a4b0 msgr2=0x7f84780b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:59.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f847ee95640 1 -- 172.21.15.32:0/136845268 shutdown_connections 2023-12-20T04:43:59.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.621+0000 7f847ee95640 1 -- 172.21.15.32:0/136845268 wait complete. 2023-12-20T04:43:59.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847ee95640 1 Processor -- start 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847ee95640 1 -- start start 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847ee95640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780cb1c0 0x7f847813dc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847ee95640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815a940 0x7f847813e150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847ee95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847815ad20 0x7f84780c4010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847ee95640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f84780ab8d0 con 0x7f847815ad20 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847ee95640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f847813e9b0 con 0x7f84780cb1c0 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847ee95640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f84780c4bb0 con 0x7f847815a940 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847d432640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847815ad20 0x7f84780c4010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847cc31640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780cb1c0 0x7f847813dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f8477fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815a940 0x7f847813e150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847d432640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847815ad20 0x7f84780c4010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42658/0 (socket says 172.21.15.32:42658) 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847cc31640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780cb1c0 0x7f847813dc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:52868/0 (socket says 172.21.15.32:52868) 2023-12-20T04:43:59.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847d432640 1 -- 172.21.15.32:0/2661872910 learned_addr learned my addr 172.21.15.32:0/2661872910 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847d432640 1 -- 172.21.15.32:0/2661872910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815a940 msgr2=0x7f847813e150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847d432640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815a940 0x7f847813e150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847d432640 1 -- 172.21.15.32:0/2661872910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780cb1c0 msgr2=0x7f847813dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847d432640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780cb1c0 0x7f847813dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847d432640 1 -- 172.21.15.32:0/2661872910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8470009a70 con 0x7f847815ad20 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847cc31640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780cb1c0 0x7f847813dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847d432640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847815ad20 0x7f84780c4010 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f847001afd0 tx=0x7f84700196c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f84757fa640 1 -- 172.21.15.32:0/2661872910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700add90 con 0x7f847815ad20 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f84757fa640 1 -- 172.21.15.32:0/2661872910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8470009450 con 0x7f847815ad20 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f84757fa640 1 -- 172.21.15.32:0/2661872910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84700b9670 con 0x7f847815ad20 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847ee95640 1 -- 172.21.15.32:0/2661872910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f84780c4e90 con 0x7f847815ad20 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.625+0000 7f847ee95640 1 -- 172.21.15.32:0/2661872910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f84780c5450 con 0x7f847815ad20 2023-12-20T04:43:59.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.629+0000 7f847ee95640 1 -- 172.21.15.32:0/2661872910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f84780aba40 con 0x7f847815ad20 2023-12-20T04:43:59.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.629+0000 7f84757fa640 1 -- 172.21.15.32:0/2661872910 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f84700b97d0 con 0x7f847815ad20 2023-12-20T04:43:59.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.629+0000 7f84757fa640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f845c065af0 0x7f845c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:43:59.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.629+0000 7f84757fa640 1 -- 172.21.15.32:0/2661872910 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f847011e940 con 0x7f847815ad20 2023-12-20T04:43:59.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.629+0000 7f847cc31640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f845c065af0 0x7f845c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:43:59.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.629+0000 7f847cc31640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f845c065af0 0x7f845c067fb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f846c009f50 tx=0x7f846c009190 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:43:59.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.633+0000 7f84757fa640 1 -- 172.21.15.32:0/2661872910 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f84780aba40 con 0x7f847815ad20 2023-12-20T04:43:59.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.853+0000 7f847ee95640 1 -- 172.21.15.32:0/2661872910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f84780c5640 con 0x7f847815ad20 2023-12-20T04:43:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.857+0000 7f84757fa640 1 -- 172.21.15.32:0/2661872910 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v18) v1 ==== 190+0+0 (secure 0 0 0) 0x7f84700ec2c0 con 0x7f847815ad20 2023-12-20T04:43:59.857 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T04:43:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.857+0000 7f847ee95640 1 -- 172.21.15.32:0/2661872910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f845c065af0 msgr2=0x7f845c067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.857+0000 7f847ee95640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f845c065af0 0x7f845c067fb0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f846c009f50 tx=0x7f846c009190 comp rx=0 tx=0).stop 2023-12-20T04:43:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.857+0000 7f847ee95640 1 -- 172.21.15.32:0/2661872910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847815ad20 msgr2=0x7f84780c4010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:43:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.857+0000 7f847ee95640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847815ad20 0x7f84780c4010 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f847001afd0 tx=0x7f84700196c0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.861+0000 7f847ee95640 1 -- 172.21.15.32:0/2661872910 shutdown_connections 2023-12-20T04:43:59.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.861+0000 7f847ee95640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f845c065af0 0x7f845c067fb0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.861+0000 7f847ee95640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f847815ad20 0x7f84780c4010 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.861+0000 7f847ee95640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f847815a940 0x7f847813e150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.861+0000 7f847ee95640 1 --2- 172.21.15.32:0/2661872910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f84780cb1c0 0x7f847813dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:43:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.861+0000 7f847ee95640 1 -- 172.21.15.32:0/2661872910 >> 172.21.15.32:0/2661872910 conn(0x7f847806a4b0 msgr2=0x7f8478153e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:43:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.861+0000 7f847ee95640 1 -- 172.21.15.32:0/2661872910 shutdown_connections 2023-12-20T04:43:59.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:43:59.861+0000 7f847ee95640 1 -- 172.21.15.32:0/2661872910 wait complete. 2023-12-20T04:43:59.879 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:44:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49969be640 1 Processor -- start 2023-12-20T04:44:00.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49969be640 1 -- start start 2023-12-20T04:44:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49969be640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4990159ab0 0x7f49900714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49969be640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900b3e90 0x7f4990071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49969be640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4990071f40 con 0x7f49900cd2e0 2023-12-20T04:44:00.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49969be640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49900ca0b0 con 0x7f4990159ab0 2023-12-20T04:44:00.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49969be640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900ad7e0 con 0x7f49900b3e90 2023-12-20T04:44:00.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49959bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4990159ab0 0x7f49900714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49961bd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900b3e90 0x7f4990071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49961bd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900b3e90 0x7f4990071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:52836/0 (socket says 172.21.15.32:52836) 2023-12-20T04:44:00.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49959bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4990159ab0 0x7f49900714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:52880/0 (socket says 172.21.15.32:52880) 2023-12-20T04:44:00.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49961bd640 1 -- 172.21.15.32:0/2552828913 learned_addr learned my addr 172.21.15.32:0/2552828913 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:00.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49961bd640 1 -- 172.21.15.32:0/2552828913 >> v2:172.21.15.39:3300/0 conn(0x7f4990159ab0 msgr2=0x7f49900714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:00.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49961bd640 1 --2- 172.21.15.32:0/2552828913 >> v2:172.21.15.39:3300/0 conn(0x7f4990159ab0 0x7f49900714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49961bd640 1 -- 172.21.15.32:0/2552828913 >> v1:172.21.15.39:6789/0 conn(0x7f49900cd2e0 legacy=0x7f49900cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.033+0000 7f49961bd640 1 -- 172.21.15.32:0/2552828913 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49900ada80 con 0x7f49900b3e90 2023-12-20T04:44:00.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49961bd640 1 --2- 172.21.15.32:0/2552828913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900b3e90 0x7f4990071a00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4988009e50 tx=0x7f498801ebf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=33fec5b9e6d40e66 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:00.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49949ba640 1 -- 172.21.15.32:0/2552828913 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49880a79e0 con 0x7f49900b3e90 2023-12-20T04:44:00.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49949ba640 1 -- 172.21.15.32:0/2552828913 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49880a7b40 con 0x7f49900b3e90 2023-12-20T04:44:00.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49949ba640 1 -- 172.21.15.32:0/2552828913 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49880af3f0 con 0x7f49900b3e90 2023-12-20T04:44:00.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49969be640 1 -- 172.21.15.32:0/2552828913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900b3e90 msgr2=0x7f4990071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49969be640 1 --2- 172.21.15.32:0/2552828913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900b3e90 0x7f4990071a00 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f4988009e50 tx=0x7f498801ebf0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49969be640 1 -- 172.21.15.32:0/2552828913 shutdown_connections 2023-12-20T04:44:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49969be640 1 --2- 172.21.15.32:0/2552828913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900b3e90 0x7f4990071a00 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49969be640 1 --2- 172.21.15.32:0/2552828913 >> v2:172.21.15.39:3300/0 conn(0x7f4990159ab0 0x7f49900714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49969be640 1 -- 172.21.15.32:0/2552828913 >> 172.21.15.32:0/2552828913 conn(0x7f499006c060 msgr2=0x7f499006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49969be640 1 -- 172.21.15.32:0/2552828913 shutdown_connections 2023-12-20T04:44:00.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.037+0000 7f49969be640 1 -- 172.21.15.32:0/2552828913 wait complete. 2023-12-20T04:44:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49969be640 1 Processor -- start 2023-12-20T04:44:00.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49969be640 1 -- start start 2023-12-20T04:44:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49969be640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49900b3e90 0x7f499014b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49969be640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900cd2e0 0x7f49901f8e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49969be640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4990159ab0 0x7f49900c2080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49969be640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49900c6040 con 0x7f49900b3e90 2023-12-20T04:44:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49969be640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f499014bdf0 con 0x7f4990159ab0 2023-12-20T04:44:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49969be640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49900c5830 con 0x7f49900cd2e0 2023-12-20T04:44:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49959bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49900b3e90 0x7f499014b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49959bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49900b3e90 0x7f499014b8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42670/0 (socket says 172.21.15.32:42670) 2023-12-20T04:44:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49959bc640 1 -- 172.21.15.32:0/105968619 learned_addr learned my addr 172.21.15.32:0/105968619 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49961bd640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4990159ab0 0x7f49900c2080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49951bb640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900cd2e0 0x7f49901f8e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49959bc640 1 -- 172.21.15.32:0/105968619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900cd2e0 msgr2=0x7f49901f8e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49959bc640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900cd2e0 0x7f49901f8e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49959bc640 1 -- 172.21.15.32:0/105968619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4990159ab0 msgr2=0x7f49900c2080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49959bc640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4990159ab0 0x7f49900c2080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49959bc640 1 -- 172.21.15.32:0/105968619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4988009b30 con 0x7f49900b3e90 2023-12-20T04:44:00.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49959bc640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49900b3e90 0x7f499014b8b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f498c00e960 tx=0x7f498c00ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49867fc640 1 -- 172.21.15.32:0/105968619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c009690 con 0x7f49900b3e90 2023-12-20T04:44:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49867fc640 1 -- 172.21.15.32:0/105968619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f498c00c9a0 con 0x7f49900b3e90 2023-12-20T04:44:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49867fc640 1 -- 172.21.15.32:0/105968619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f498c00cd10 con 0x7f49900b3e90 2023-12-20T04:44:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49969be640 1 -- 172.21.15.32:0/105968619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f499015ba70 con 0x7f49900b3e90 2023-12-20T04:44:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49969be640 1 -- 172.21.15.32:0/105968619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f499015a3d0 con 0x7f49900b3e90 2023-12-20T04:44:00.043 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:44:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.041+0000 7f49969be640 1 -- 172.21.15.32:0/105968619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4990074100 con 0x7f49900b3e90 2023-12-20T04:44:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.045+0000 7f49867fc640 1 -- 172.21.15.32:0/105968619 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f498c013470 con 0x7f49900b3e90 2023-12-20T04:44:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.045+0000 7f49867fc640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4974065af0 0x7f4974067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.045+0000 7f49867fc640 1 -- 172.21.15.32:0/105968619 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f498c0832e0 con 0x7f49900b3e90 2023-12-20T04:44:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.045+0000 7f49951bb640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4974065af0 0x7f4974067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.045+0000 7f49951bb640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4974065af0 0x7f4974067fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49900ad040 tx=0x7f4980009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:00.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.049+0000 7f49867fc640 1 -- 172.21.15.32:0/105968619 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f498c050d60 con 0x7f49900b3e90 2023-12-20T04:44:00.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.237+0000 7f49969be640 1 -- 172.21.15.32:0/105968619 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f49900c8ef0 con 0x7f49900b3e90 2023-12-20T04:44:00.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.237+0000 7f49867fc640 1 -- 172.21.15.32:0/105968619 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4699 (secure 0 0 0) 0x7f498c054380 con 0x7f49900b3e90 2023-12-20T04:44:00.239 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:00.239 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/655215798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":655215798},{"type":"v1","addr":"172.21.15.32:6837","nonce":655215798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/661544180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":661544180},{"type":"v1","addr":"172.21.15.39:6834","nonce":661544180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6855,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/833127896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":833127896},{"type":"v1","addr":"172.21.15.32:6836","nonce":833127896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:54.874151+0000","modified":"2023-12-20T04:43:57.893768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6864":{"gid":6864,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/979895154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":979895154},{"type":"v1","addr":"172.21.15.39:6835","nonce":979895154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-20T04:44:00.239 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2023-12-20T04:44:00.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 -- 172.21.15.32:0/105968619 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4974065af0 msgr2=0x7f4974067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4974065af0 0x7f4974067fb0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f49900ad040 tx=0x7f4980009040 comp rx=0 tx=0).stop 2023-12-20T04:44:00.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 -- 172.21.15.32:0/105968619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49900b3e90 msgr2=0x7f499014b8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49900b3e90 0x7f499014b8b0 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7f498c00e960 tx=0x7f498c00ee20 comp rx=0 tx=0).stop 2023-12-20T04:44:00.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 -- 172.21.15.32:0/105968619 shutdown_connections 2023-12-20T04:44:00.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4974065af0 0x7f4974067fb0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4990159ab0 0x7f49900c2080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49900cd2e0 0x7f49901f8e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 --2- 172.21.15.32:0/105968619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49900b3e90 0x7f499014b8b0 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 -- 172.21.15.32:0/105968619 >> 172.21.15.32:0/105968619 conn(0x7f499006c060 msgr2=0x7f49900b2f20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:00.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 -- 172.21.15.32:0/105968619 shutdown_connections 2023-12-20T04:44:00.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.241+0000 7f49969be640 1 -- 172.21.15.32:0/105968619 wait complete. 2023-12-20T04:44:00.434 INFO:tasks.ceph.mds.d.smithi039.stderr:2023-12-20T04:44:00.435+0000 7f3593264640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:44:00.664 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 39, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T04:43:54.874151+0000', 'modified': '2023-12-20T04:43:57.893768+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 6864}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_6864': {'gid': 6864, 'name': 'd', 'rank': 0, 'incarnation': 35, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6835/979895154', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 979895154}, {'type': 'v1', 'addr': '172.21.15.39:6835', 'nonce': 979895154}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [11], 'metadata_pool': 10, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T04:44:00.665 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T04:44:00.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-20T04:44:00.791 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:00.793+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:44:00.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.813+0000 7f06c536d640 1 Processor -- start 2023-12-20T04:44:00.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.813+0000 7f06c536d640 1 -- start start 2023-12-20T04:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06c536d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06c00aece0 0x7f06c00b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06c536d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06c00b0b30 0x7f06c00701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06c536d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f06c00c98c0 con 0x7f06c01599e0 2023-12-20T04:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06c536d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f06c00c9ba0 con 0x7f06c00b0b30 2023-12-20T04:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06c536d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c00c9e80 con 0x7f06c00aece0 2023-12-20T04:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06bf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06c00b0b30 0x7f06c00701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06b7fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06c00aece0 0x7f06c00b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06bf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f06c00b0b30 0x7f06c00701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42674/0 (socket says 172.21.15.32:42674) 2023-12-20T04:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06bf7fe640 1 -- 172.21.15.32:0/370743842 learned_addr learned my addr 172.21.15.32:0/370743842 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06beffd640 1 --1- 172.21.15.32:0/370743842 >> v1:172.21.15.39:6789/0 conn(0x7f06c01599e0 0x7f06c00c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:49218/0 (socket says 172.21.15.32:49218) 2023-12-20T04:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06b7fff640 1 -- 172.21.15.32:0/370743842 >> v2:172.21.15.32:3300/0 conn(0x7f06c00b0b30 msgr2=0x7f06c00701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06b7fff640 1 --2- 172.21.15.32:0/370743842 >> v2:172.21.15.32:3300/0 conn(0x7f06c00b0b30 0x7f06c00701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06b7fff640 1 -- 172.21.15.32:0/370743842 >> v1:172.21.15.39:6789/0 conn(0x7f06c01599e0 legacy=0x7f06c00c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.817+0000 7f06b7fff640 1 -- 172.21.15.32:0/370743842 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06c00ca160 con 0x7f06c00aece0 2023-12-20T04:44:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06b7fff640 1 --2- 172.21.15.32:0/370743842 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06c00aece0 0x7f06c00b05f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06a80097c0 tx=0x7f06a801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d8dfcd0f36a3a3ee server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06be7fc640 1 -- 172.21.15.32:0/370743842 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a80ad640 con 0x7f06c00aece0 2023-12-20T04:44:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06be7fc640 1 -- 172.21.15.32:0/370743842 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a80ad7a0 con 0x7f06c00aece0 2023-12-20T04:44:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06be7fc640 1 -- 172.21.15.32:0/370743842 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a80adb30 con 0x7f06c00aece0 2023-12-20T04:44:00.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06c536d640 1 -- 172.21.15.32:0/370743842 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06c00aece0 msgr2=0x7f06c00b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06c536d640 1 --2- 172.21.15.32:0/370743842 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06c00aece0 0x7f06c00b05f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f06a80097c0 tx=0x7f06a801c480 comp rx=0 tx=0).stop 2023-12-20T04:44:00.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06c536d640 1 -- 172.21.15.32:0/370743842 shutdown_connections 2023-12-20T04:44:00.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06c536d640 1 --2- 172.21.15.32:0/370743842 >> v2:172.21.15.32:3300/0 conn(0x7f06c00b0b30 0x7f06c00701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06c536d640 1 --2- 172.21.15.32:0/370743842 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06c00aece0 0x7f06c00b05f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06c536d640 1 -- 172.21.15.32:0/370743842 >> 172.21.15.32:0/370743842 conn(0x7f06c006a320 msgr2=0x7f06c00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:00.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06c536d640 1 -- 172.21.15.32:0/370743842 shutdown_connections 2023-12-20T04:44:00.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06c536d640 1 -- 172.21.15.32:0/370743842 wait complete. 2023-12-20T04:44:00.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06c536d640 1 Processor -- start 2023-12-20T04:44:00.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06c536d640 1 -- start start 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.821+0000 7f06c536d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c00aece0 0x7f06c00cda60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06c536d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06c00b0b30 0x7f06c00cdfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06c536d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06c01599e0 0x7f06c0158030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06c536d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0152510 con 0x7f06c00aece0 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06c536d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0150680 con 0x7f06c00b0b30 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06c536d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f06c0150960 con 0x7f06c01599e0 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06beffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c00aece0 0x7f06c00cda60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06beffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c00aece0 0x7f06c00cda60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42682/0 (socket says 172.21.15.32:42682) 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06b7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06c00b0b30 0x7f06c00cdfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06beffd640 1 -- 172.21.15.32:0/876619756 learned_addr learned my addr 172.21.15.32:0/876619756 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06bf7fe640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06c01599e0 0x7f06c0158030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06b7fff640 1 -- 172.21.15.32:0/876619756 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06c01599e0 msgr2=0x7f06c0158030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06b7fff640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06c01599e0 0x7f06c0158030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06b7fff640 1 -- 172.21.15.32:0/876619756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c00aece0 msgr2=0x7f06c00cda60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06b7fff640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c00aece0 0x7f06c00cda60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06b7fff640 1 -- 172.21.15.32:0/876619756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f06a8009420 con 0x7f06c00b0b30 2023-12-20T04:44:00.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06beffd640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c00aece0 0x7f06c00cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:00.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06b7fff640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06c00b0b30 0x7f06c00cdfa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f06a80ac000 tx=0x7f06a8019220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:00.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06b77fe640 1 -- 172.21.15.32:0/876619756 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a80b6610 con 0x7f06c00b0b30 2023-12-20T04:44:00.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06b77fe640 1 -- 172.21.15.32:0/876619756 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a80a6040 con 0x7f06c00b0b30 2023-12-20T04:44:00.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06b77fe640 1 -- 172.21.15.32:0/876619756 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a80b69b0 con 0x7f06c00b0b30 2023-12-20T04:44:00.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06c536d640 1 -- 172.21.15.32:0/876619756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f06c0150c40 con 0x7f06c00b0b30 2023-12-20T04:44:00.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.825+0000 7f06c536d640 1 -- 172.21.15.32:0/876619756 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06c0151440 con 0x7f06c00b0b30 2023-12-20T04:44:00.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06b77fe640 1 -- 172.21.15.32:0/876619756 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f06a80b4040 con 0x7f06c00b0b30 2023-12-20T04:44:00.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06b77fe640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f06a0065b20 0x7f06a0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06b77fe640 1 -- 172.21.15.32:0/876619756 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06a811d240 con 0x7f06c00b0b30 2023-12-20T04:44:00.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06beffd640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f06a0065b20 0x7f06a0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06c86cb000 1 Processor -- start 2023-12-20T04:44:00.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06beffd640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f06a0065b20 0x7f06a0067fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f06b0000fc0 tx=0x7f06b0007f00 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:00.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06c86cb000 1 -- start start 2023-12-20T04:44:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4a2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4a2990 0x56512e457c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4581c0 0x56512e4585a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06c86cb000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56512e27bc90 con 0x56512e4581c0 2023-12-20T04:44:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06c86cb000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56512e2ad220 con 0x56512e4a2990 2023-12-20T04:44:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06c86cb000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x56512e2aa7f0 con 0x56512e4a2070 2023-12-20T04:44:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4581c0 0x56512e4585a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06beffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4a2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4581c0 0x56512e4585a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42688/0 (socket says 172.21.15.32:42688) 2023-12-20T04:44:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06bf7fe640 1 -- 172.21.15.32:0/3467081530 learned_addr learned my addr 172.21.15.32:0/3467081530 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:00.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06b7fff640 1 --2- 172.21.15.32:0/3467081530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4a2990 0x56512e457c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06bf7fe640 1 -- 172.21.15.32:0/3467081530 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 msgr2=0x56512e4a2450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06bf7fe640 1 --2- 172.21.15.32:0/3467081530 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4a2450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06bf7fe640 1 -- 172.21.15.32:0/3467081530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4a2990 msgr2=0x56512e457c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06bf7fe640 1 --2- 172.21.15.32:0/3467081530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4a2990 0x56512e457c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.829+0000 7f06bf7fe640 1 -- 172.21.15.32:0/3467081530 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56512dfa2890 con 0x56512e4581c0 2023-12-20T04:44:00.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06bf7fe640 1 --2- 172.21.15.32:0/3467081530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4581c0 0x56512e4585a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f06b8011580 tx=0x7f06b8009810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a4be83f71cc0aab server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:00.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06b4ff9640 1 -- 172.21.15.32:0/3467081530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b800e040 con 0x56512e4581c0 2023-12-20T04:44:00.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06b4ff9640 1 -- 172.21.15.32:0/3467081530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06b800f040 con 0x56512e4581c0 2023-12-20T04:44:00.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06b4ff9640 1 -- 172.21.15.32:0/3467081530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8015850 con 0x56512e4581c0 2023-12-20T04:44:00.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 -- 172.21.15.32:0/3467081530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4581c0 msgr2=0x56512e4585a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 --2- 172.21.15.32:0/3467081530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4581c0 0x56512e4585a0 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f06b8011580 tx=0x7f06b8009810 comp rx=0 tx=0).stop 2023-12-20T04:44:00.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 -- 172.21.15.32:0/3467081530 shutdown_connections 2023-12-20T04:44:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 --2- 172.21.15.32:0/3467081530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4581c0 0x56512e4585a0 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 --2- 172.21.15.32:0/3467081530 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4a2990 0x56512e457c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 --2- 172.21.15.32:0/3467081530 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4a2450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 -- 172.21.15.32:0/3467081530 >> 172.21.15.32:0/3467081530 conn(0x56512e45e950 msgr2=0x56512e456560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 -- 172.21.15.32:0/3467081530 shutdown_connections 2023-12-20T04:44:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 -- 172.21.15.32:0/3467081530 wait complete. 2023-12-20T04:44:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 -- start start 2023-12-20T04:44:00.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 Processor -- start 2023-12-20T04:44:00.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e45d720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2070 0x56512e464080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2990 0x56512e4645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56512e2a60e0 con 0x56512e4a2070 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56512e245310 con 0x56512e4581c0 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x56512e263770 con 0x56512e4a2990 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06c86cb000 10 client.0 _ll_register_callbacks cb 0x56512e443b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2070 0x56512e464080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2070 0x56512e464080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42692/0 (socket says 172.21.15.32:42692) 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06b7fff640 1 -- 172.21.15.32:0/2711974435 learned_addr learned my addr 172.21.15.32:0/2711974435 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06beffd640 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e45d720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06bf7fe640 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2990 0x56512e4645c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06b7fff640 1 -- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2990 msgr2=0x56512e4645c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06b7fff640 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2990 0x56512e4645c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06b7fff640 1 -- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 msgr2=0x56512e45d720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06b7fff640 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e45d720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.833+0000 7f06b7fff640 1 -- 172.21.15.32:0/2711974435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f06b8011230 con 0x56512e4a2070 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06b7fff640 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2070 0x56512e464080 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f06a80097c0 tx=0x7f06a80ee5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8290564599e04ed server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f069dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f069dffb640 1 -- 172.21.15.32:0/2711974435 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a80ee090 con 0x56512e4a2070 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f069dffb640 1 -- 172.21.15.32:0/2711974435 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06a80f68d0 con 0x56512e4a2070 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 -- 172.21.15.32:0/2711974435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56512e275c10 con 0x56512e4a2070 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f069dffb640 1 -- 172.21.15.32:0/2711974435 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06a80f6440 con 0x56512e4a2070 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 10 client.6990 fetch_fsmap learned FSMap version 39 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 -- 172.21.15.32:0/2711974435 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x56512e26b260 con 0x56512e4a2070 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f069dffb640 1 -- 172.21.15.32:0/2711974435 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56512e275c10 con 0x56512e4a2070 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f069dffb640 1 -- 172.21.15.32:0/2711974435 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f06a80b6b10 con 0x56512e4a2070 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 10 client.6990 fetch_fsmap finished waiting for FSMap version 39 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 10 client.6990 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-20T04:44:00.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 20 client.6990 populate_metadata read hostname 'smithi032' 2023-12-20T04:44:00.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:6832/979895154,v1:172.21.15.39:6835/979895154] conn(0x56512e4a98d0 0x56512e4a9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 4 client.6990 mds_command: new command op to 6864 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T04:44:00.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 -- 172.21.15.32:0/2711974435 --> [v2:172.21.15.39:6832/979895154,v1:172.21.15.39:6835/979895154] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x56512e2a4a30 con 0x56512e4a98d0 2023-12-20T04:44:00.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06beffd640 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:6832/979895154,v1:172.21.15.39:6835/979895154] conn(0x56512e4a98d0 0x56512e4a9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06beffd640 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:6832/979895154,v1:172.21.15.39:6835/979895154] conn(0x56512e4a98d0 0x56512e4a9cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5be55570eb077b1 server_cookie=d6a275372cf4b532 in_seq=0 out_seq=0 2023-12-20T04:44:00.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f069dffb640 10 client.6990 ms_handle_connect on v2:172.21.15.39:6832/979895154 2023-12-20T04:44:00.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f069dffb640 1 -- 172.21.15.32:0/2711974435 <== mds.0 v2:172.21.15.39:6832/979895154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x56512e2a4a30 con 0x56512e4a98d0 2023-12-20T04:44:00.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f069dffb640 10 client.6990 handle_command_reply: tid=0 2023-12-20T04:44:00.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 client.6990 shutdown 2023-12-20T04:44:00.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 -- 172.21.15.32:0/2711974435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2070 msgr2=0x56512e464080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2070 0x56512e464080 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f06a80097c0 tx=0x7f06a80ee5f0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 -- 172.21.15.32:0/2711974435 shutdown_connections 2023-12-20T04:44:00.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:6832/979895154,v1:172.21.15.39:6835/979895154] conn(0x56512e4a98d0 0x56512e4a9cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2990 0x56512e4645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2070 0x56512e464080 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 --2- 172.21.15.32:0/2711974435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e45d720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f069dffb640 0 client.6990 ms_handle_reset on v2:172.21.15.39:6832/979895154 2023-12-20T04:44:00.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 -- 172.21.15.32:0/2711974435 >> 172.21.15.32:0/2711974435 conn(0x56512e45e950 msgr2=0x56512e456c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:00.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 -- 172.21.15.32:0/2711974435 shutdown_connections 2023-12-20T04:44:00.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 1 -- 172.21.15.32:0/2711974435 wait complete. 2023-12-20T04:44:00.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.837+0000 7f06c86cb000 20 client.6990 trim_cache size 0 max 16384 2023-12-20T04:44:00.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06c86cb000 1 Processor -- start 2023-12-20T04:44:00.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06c86cb000 1 -- start start 2023-12-20T04:44:00.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e4fa2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4fa830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4fad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06c86cb000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56512df63940 con 0x56512e4a2990 2023-12-20T04:44:00.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06c86cb000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56512e2a34e0 con 0x56512e4581c0 2023-12-20T04:44:00.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06c86cb000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x56512e2a1a50 con 0x56512e4a2070 2023-12-20T04:44:00.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4fad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06beffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e4fa2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4fad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42700/0 (socket says 172.21.15.32:42700) 2023-12-20T04:44:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06bf7fe640 1 -- 172.21.15.32:0/1010981752 learned_addr learned my addr 172.21.15.32:0/1010981752 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.853+0000 7f06b7fff640 1 --2- 172.21.15.32:0/1010981752 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4fa830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06b7fff640 1 -- 172.21.15.32:0/1010981752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 msgr2=0x56512e4fa2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06b7fff640 1 --2- 172.21.15.32:0/1010981752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e4fa2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06b7fff640 1 -- 172.21.15.32:0/1010981752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 msgr2=0x56512e4fad70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06b7fff640 1 --2- 172.21.15.32:0/1010981752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4fad70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06b7fff640 1 -- 172.21.15.32:0/1010981752 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x56512e2513b0 con 0x56512e4a2070 2023-12-20T04:44:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06bf7fe640 1 --2- 172.21.15.32:0/1010981752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:00.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06b7fff640 1 --2- 172.21.15.32:0/1010981752 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4fa830 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f06a801c4c0 tx=0x7f06a80f0000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a5548bd80e02ab37 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:00.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06b57fa640 1 -- 172.21.15.32:0/1010981752 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a80eea20 con 0x56512e4a2070 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06b57fa640 1 -- 172.21.15.32:0/1010981752 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06a80ee4e0 con 0x56512e4a2070 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06b57fa640 1 -- 172.21.15.32:0/1010981752 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06a80f68d0 con 0x56512e4a2070 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 -- 172.21.15.32:0/1010981752 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 msgr2=0x56512e4fa830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 --2- 172.21.15.32:0/1010981752 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4fa830 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7f06a801c4c0 tx=0x7f06a80f0000 comp rx=0 tx=0).stop 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 -- 172.21.15.32:0/1010981752 shutdown_connections 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 --2- 172.21.15.32:0/1010981752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4fad70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 --2- 172.21.15.32:0/1010981752 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4fa830 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 --2- 172.21.15.32:0/1010981752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e4fa2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 -- 172.21.15.32:0/1010981752 >> 172.21.15.32:0/1010981752 conn(0x56512e45e950 msgr2=0x56512e463f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 -- 172.21.15.32:0/1010981752 shutdown_connections 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 -- 172.21.15.32:0/1010981752 wait complete. 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 -- start start 2023-12-20T04:44:00.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 Processor -- start 2023-12-20T04:44:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e4e5010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4fa500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4faa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x56512e24b570 con 0x56512e4a2990 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56512e45d770 con 0x56512e4581c0 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x56512e24e860 con 0x56512e4a2070 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06c86cb000 10 client.0 _ll_register_callbacks cb 0x56512e254640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4faa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4faa40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42714/0 (socket says 172.21.15.32:42714) 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06bf7fe640 1 -- 172.21.15.32:0/1926735120 learned_addr learned my addr 172.21.15.32:0/1926735120 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06beffd640 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e4e5010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06b7fff640 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4fa500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06bf7fe640 1 -- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 msgr2=0x56512e4fa500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06bf7fe640 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4fa500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06bf7fe640 1 -- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 msgr2=0x56512e4e5010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06bf7fe640 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e4e5010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06bf7fe640 1 -- 172.21.15.32:0/1926735120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x56512dfa2890 con 0x56512e4a2990 2023-12-20T04:44:00.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.857+0000 7f06b7fff640 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06bf7fe640 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4faa40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f06b8011ed0 tx=0x7f06b8004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc3cb83e3bd8aff7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f069dffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f069dffb640 1 -- 172.21.15.32:0/1926735120 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06b8023070 con 0x56512e4a2990 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f069dffb640 1 -- 172.21.15.32:0/1926735120 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f06b8004250 con 0x56512e4a2990 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 -- 172.21.15.32:0/1926735120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x56512e26be00 con 0x56512e4a2990 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f069dffb640 1 -- 172.21.15.32:0/1926735120 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=39) v2 ==== 24+0+0 (secure 0 0 0) 0x7f06b800e040 con 0x56512e4a2990 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 10 client.6996 fetch_fsmap learned FSMap version 39 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 -- 172.21.15.32:0/1926735120 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=39,monmap=2+}) v3 -- 0x7f06b8011230 con 0x56512e4a2990 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f069dffb640 1 -- 172.21.15.32:0/1926735120 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x56512e26be00 con 0x56512e4a2990 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f069dffb640 1 -- 172.21.15.32:0/1926735120 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 39) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f06b8004410 con 0x56512e4a2990 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 10 client.6996 fetch_fsmap finished waiting for FSMap version 39 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 10 client.6996 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 20 client.6996 populate_metadata read hostname 'smithi032' 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:6832/979895154,v1:172.21.15.39:6835/979895154] conn(0x56512e4faf80 0x56512e4fb360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 4 client.6996 mds_command: new command op to 6864 tid=0[{"prefix": "status"}] 2023-12-20T04:44:00.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 -- 172.21.15.32:0/1926735120 --> [v2:172.21.15.39:6832/979895154,v1:172.21.15.39:6835/979895154] -- command(tid 0: {"prefix": "status"}) v1 -- 0x56512e29f160 con 0x56512e4faf80 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06beffd640 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:6832/979895154,v1:172.21.15.39:6835/979895154] conn(0x56512e4faf80 0x56512e4fb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06beffd640 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:6832/979895154,v1:172.21.15.39:6835/979895154] conn(0x56512e4faf80 0x56512e4fb360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9d70b12d2d3fecb1 server_cookie=36e75c4f431e1919 in_seq=0 out_seq=0 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f069dffb640 10 client.6996 ms_handle_connect on v2:172.21.15.39:6832/979895154 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f069dffb640 1 -- 172.21.15.32:0/1926735120 <== mds.0 v2:172.21.15.39:6832/979895154 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+324 (crc 0 0 0) 0x56512e29f160 con 0x56512e4faf80 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f069dffb640 10 client.6996 handle_command_reply: tid=0 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 client.6996 shutdown 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 -- 172.21.15.32:0/1926735120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 msgr2=0x56512e4faa40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4faa40 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f06b8011ed0 tx=0x7f06b8004ac0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 -- 172.21.15.32:0/1926735120 shutdown_connections 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:6832/979895154,v1:172.21.15.39:6835/979895154] conn(0x56512e4faf80 0x56512e4fb360 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x56512e4a2990 0x56512e4faa40 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x56512e4a2070 0x56512e4fa500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 --2- 172.21.15.32:0/1926735120 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x56512e4581c0 0x56512e4e5010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f069dffb640 0 client.6996 ms_handle_reset on v2:172.21.15.39:6832/979895154 2023-12-20T04:44:00.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 -- 172.21.15.32:0/1926735120 >> 172.21.15.32:0/1926735120 conn(0x56512e45e950 msgr2=0x56512e49f060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:00.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 -- 172.21.15.32:0/1926735120 shutdown_connections 2023-12-20T04:44:00.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 1 -- 172.21.15.32:0/1926735120 wait complete. 2023-12-20T04:44:00.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c86cb000 20 client.6996 trim_cache size 0 max 16384 2023-12-20T04:44:00.866 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:44:00.866 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T04:44:00.866 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T04:44:00.866 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 6864, 2023-12-20T04:44:00.866 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T04:44:00.866 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T04:44:00.867 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T04:44:00.867 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9893034910000003, 2023-12-20T04:44:00.867 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 39, 2023-12-20T04:44:00.867 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 93, 2023-12-20T04:44:00.867 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 93, 2023-12-20T04:44:00.867 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 11.21432881 2023-12-20T04:44:00.867 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:44:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 -- 172.21.15.32:0/876619756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f06a0065b20 msgr2=0x7f06a0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f06a0065b20 0x7f06a0067fe0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f06b0000fc0 tx=0x7f06b0007f00 comp rx=0 tx=0).stop 2023-12-20T04:44:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 -- 172.21.15.32:0/876619756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06c00b0b30 msgr2=0x7f06c00cdfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06c00b0b30 0x7f06c00cdfa0 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f06a80ac000 tx=0x7f06a8019220 comp rx=0 tx=0).stop 2023-12-20T04:44:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 -- 172.21.15.32:0/876619756 shutdown_connections 2023-12-20T04:44:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f06a0065b20 0x7f06a0067fe0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f06c01599e0 0x7f06c0158030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f06c00b0b30 0x7f06c00cdfa0 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 --2- 172.21.15.32:0/876619756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f06c00aece0 0x7f06c00cda60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 -- 172.21.15.32:0/876619756 >> 172.21.15.32:0/876619756 conn(0x7f06c006a320 msgr2=0x7f06c006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:00.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 -- 172.21.15.32:0/876619756 shutdown_connections 2023-12-20T04:44:00.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:00.861+0000 7f06c536d640 1 -- 172.21.15.32:0/876619756 wait complete. 2023-12-20T04:44:00.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T04:44:01.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d8998640 1 Processor -- start 2023-12-20T04:44:01.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d8998640 1 -- start start 2023-12-20T04:44:01.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d8998640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1d415cdb0 0x7fa1d40d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:01.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d8998640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1d40d2740 0x7fa1d40cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:01.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d8998640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa1d40b6ca0 con 0x7fa1d40d1120 2023-12-20T04:44:01.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d8998640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa1d415d190 con 0x7fa1d40d2740 2023-12-20T04:44:01.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d8998640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d40d16a0 con 0x7fa1d415cdb0 2023-12-20T04:44:01.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1cbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa1d40d2740 0x7fa1d40cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:01.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1d415cdb0 0x7fa1d40d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:01.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d2d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa1d40d1120 0x7fa1d40b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39914/0 (socket says 172.21.15.32:39914) 2023-12-20T04:44:01.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d2d9d640 1 -- 172.21.15.32:0/3514296135 learned_addr learned my addr 172.21.15.32:0/3514296135 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:01.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1cbfff640 1 -- 172.21.15.32:0/3514296135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1d415cdb0 msgr2=0x7fa1d40d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:01.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1cbfff640 1 --2- 172.21.15.32:0/3514296135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1d415cdb0 0x7fa1d40d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:01.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1cbfff640 1 -- 172.21.15.32:0/3514296135 >> v1:172.21.15.32:6789/0 conn(0x7fa1d40d1120 legacy=0x7fa1d40b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:01.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1cbfff640 1 -- 172.21.15.32:0/3514296135 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1d40d1980 con 0x7fa1d40d2740 2023-12-20T04:44:01.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d259c640 1 --2- 172.21.15.32:0/3514296135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1d415cdb0 0x7fa1d40d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:01.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.029+0000 7fa1d1d9b640 1 -- 172.21.15.32:0/3514296135 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1602613166 0 0) 0x7fa1cc0014a0 con 0x7fa1d40d1120 2023-12-20T04:44:01.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1cbfff640 1 --2- 172.21.15.32:0/3514296135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d40d2740 0x7fa1d40cf570 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0097c0 tx=0x7fa1bc01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfe0a6f12c614c71 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:01.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1d1d9b640 1 -- 172.21.15.32:0/3514296135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0a8970 con 0x7fa1d40d2740 2023-12-20T04:44:01.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1d1d9b640 1 -- 172.21.15.32:0/3514296135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc0a8ad0 con 0x7fa1d40d2740 2023-12-20T04:44:01.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1d1d9b640 1 -- 172.21.15.32:0/3514296135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0a8e60 con 0x7fa1d40d2740 2023-12-20T04:44:01.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1d8998640 1 -- 172.21.15.32:0/3514296135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d40d2740 msgr2=0x7fa1d40cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:01.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1d8998640 1 --2- 172.21.15.32:0/3514296135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d40d2740 0x7fa1d40cf570 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc0097c0 tx=0x7fa1bc01ec70 comp rx=0 tx=0).stop 2023-12-20T04:44:01.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1d8998640 1 -- 172.21.15.32:0/3514296135 shutdown_connections 2023-12-20T04:44:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1d8998640 1 --2- 172.21.15.32:0/3514296135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d40d2740 0x7fa1d40cf570 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1d8998640 1 --2- 172.21.15.32:0/3514296135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1d415cdb0 0x7fa1d40d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1d8998640 1 -- 172.21.15.32:0/3514296135 >> 172.21.15.32:0/3514296135 conn(0x7fa1d4065ee0 msgr2=0x7fa1d406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1d8998640 1 -- 172.21.15.32:0/3514296135 shutdown_connections 2023-12-20T04:44:01.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.033+0000 7fa1d8998640 1 -- 172.21.15.32:0/3514296135 wait complete. 2023-12-20T04:44:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1d8998640 1 Processor -- start 2023-12-20T04:44:01.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1d8998640 1 -- start start 2023-12-20T04:44:01.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1d8998640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1d40d1120 0x7fa1d40c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:01.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1d8998640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d40d2740 0x7fa1d415f320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:01.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1d8998640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1d415cdb0 0x7fa1d40cbcb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:01.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1d8998640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d415d820 con 0x7fa1d40d2740 2023-12-20T04:44:01.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1d8998640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d415db00 con 0x7fa1d40d1120 2023-12-20T04:44:01.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1d8998640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa1d415dde0 con 0x7fa1d415cdb0 2023-12-20T04:44:01.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d40d2740 0x7fa1d415f320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:01.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1d259c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1d40d1120 0x7fa1d40c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:01.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d40d2740 0x7fa1d415f320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42738/0 (socket says 172.21.15.32:42738) 2023-12-20T04:44:01.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1cbfff640 1 -- 172.21.15.32:0/233257575 learned_addr learned my addr 172.21.15.32:0/233257575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1d2d9d640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1d415cdb0 0x7fa1d40cbcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1cbfff640 1 -- 172.21.15.32:0/233257575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1d415cdb0 msgr2=0x7fa1d40cbcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1cbfff640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1d415cdb0 0x7fa1d40cbcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1cbfff640 1 -- 172.21.15.32:0/233257575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1d40d1120 msgr2=0x7fa1d40c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1cbfff640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1d40d1120 0x7fa1d40c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:01.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1cbfff640 1 -- 172.21.15.32:0/233257575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa1c4009420 con 0x7fa1d40d2740 2023-12-20T04:44:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1cbfff640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d40d2740 0x7fa1d415f320 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc01c8a0 tx=0x7fa1bc01d4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1caffd640 1 -- 172.21.15.32:0/233257575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc0b1e00 con 0x7fa1d40d2740 2023-12-20T04:44:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1caffd640 1 -- 172.21.15.32:0/233257575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1bc006e20 con 0x7fa1d40d2740 2023-12-20T04:44:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1d8998640 1 -- 172.21.15.32:0/233257575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa1bc009420 con 0x7fa1d40d2740 2023-12-20T04:44:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.037+0000 7fa1caffd640 1 -- 172.21.15.32:0/233257575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1bc004230 con 0x7fa1d40d2740 2023-12-20T04:44:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.041+0000 7fa1d8998640 1 -- 172.21.15.32:0/233257575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa1d415e3a0 con 0x7fa1d40d2740 2023-12-20T04:44:01.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.041+0000 7fa1d8998640 1 -- 172.21.15.32:0/233257575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa1d404e000 con 0x7fa1d40d2740 2023-12-20T04:44:01.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.041+0000 7fa1caffd640 1 -- 172.21.15.32:0/233257575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fa1bc004390 con 0x7fa1d40d2740 2023-12-20T04:44:01.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.041+0000 7fa1caffd640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1b4065b20 0x7fa1b4067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:01.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.041+0000 7fa1caffd640 1 -- 172.21.15.32:0/233257575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa1bc120fc0 con 0x7fa1d40d2740 2023-12-20T04:44:01.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.041+0000 7fa1d259c640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1b4065b20 0x7fa1b4067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:01.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.041+0000 7fa1d259c640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1b4065b20 0x7fa1b4067fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009cf0 tx=0x7fa1c4009190 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:01.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.045+0000 7fa1caffd640 1 -- 172.21.15.32:0/233257575 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa1bc0eea60 con 0x7fa1d40d2740 2023-12-20T04:44:01.200 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:44:01.203 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:44:01.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.241+0000 7fa1d8998640 1 -- 172.21.15.32:0/233257575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fa1d406b8d0 con 0x7fa1d40d2740 2023-12-20T04:44:01.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.241+0000 7fa1caffd640 1 -- 172.21.15.32:0/233257575 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v39) v1 ==== 53+0+73 (secure 0 0 0) 0x7fa1bc0c2060 con 0x7fa1d40d2740 2023-12-20T04:44:01.242 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T04:44:01.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 -- 172.21.15.32:0/233257575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1b4065b20 msgr2=0x7fa1b4067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:01.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1b4065b20 0x7fa1b4067fe0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fa1c4009cf0 tx=0x7fa1c4009190 comp rx=0 tx=0).stop 2023-12-20T04:44:01.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 -- 172.21.15.32:0/233257575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d40d2740 msgr2=0x7fa1d415f320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:01.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d40d2740 0x7fa1d415f320 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7fa1bc01c8a0 tx=0x7fa1bc01d4b0 comp rx=0 tx=0).stop 2023-12-20T04:44:01.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 -- 172.21.15.32:0/233257575 shutdown_connections 2023-12-20T04:44:01.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa1b4065b20 0x7fa1b4067fe0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:01.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa1d415cdb0 0x7fa1d40cbcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:01.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa1d40d2740 0x7fa1d415f320 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:01.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 --2- 172.21.15.32:0/233257575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa1d40d1120 0x7fa1d40c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:01.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 -- 172.21.15.32:0/233257575 >> 172.21.15.32:0/233257575 conn(0x7fa1d4065ee0 msgr2=0x7fa1d40cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:01.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 -- 172.21.15.32:0/233257575 shutdown_connections 2023-12-20T04:44:01.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:01.245+0000 7fa1d8998640 1 -- 172.21.15.32:0/233257575 wait complete. 2023-12-20T04:44:01.264 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T04:44:01.264 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T04:44:01.264 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T04:44:01.264 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T04:44:01.264 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T04:44:01.264 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T04:44:01.264 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T04:44:01.264 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T04:44:01.264 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:44:01.264 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:01.264 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T04:44:01.320 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:44:01.320 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:44:01.320 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:44:01.320 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:01.320 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T04:44:01.320 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41418sec preferred_lft 41418sec 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T04:44:01.321 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:44:01.322 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T04:44:01.323 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T04:44:01.323 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:01.323 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:44:01.323 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T04:44:01.323 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T04:44:01.323 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:01.323 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:01.323 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T04:44:01.387 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T04:44:01.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:01.389 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T04:44:01.444 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T04:44:01.444 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T04:44:01.444 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T04:44:01.444 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T04:44:01.444 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T04:44:01.445 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T04:44:01.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:01.445 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:44:01.445 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:44:01.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T04:44:01.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T04:44:01.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T04:44:01.445 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:44:01.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:01.542 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T04:44:01.551 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T04:44:01.551 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T04:44:01.552 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:44:01.552 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:01.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T04:44:01.555 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:01.553+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:44:01.576 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:44:01.577 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T04:44:01.627 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:44:01.628 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:44:01.628 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:44:01.679 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T04:44:01.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:01.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:44:01.775 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:44:01.775 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:44:01.788 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:44:01.788 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:44:01.788 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T04:44:01.788 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T04:44:01.789 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T04:44:01.792 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T04:44:01.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:01.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:44:01.814 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:01.814 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:44:01.872 INFO:teuthology.orchestra.run.smithi032.stdout:client6913 2023-12-20T04:44:01.873 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:44:01.873 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:44:01.928 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T04:44:02.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe13d433640 1 Processor -- start 2023-12-20T04:44:02.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe13d433640 1 -- start start 2023-12-20T04:44:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe13d433640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1380b8480 0x7fe13806af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe13d433640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe13806b8c0 con 0x7fe13806b460 2023-12-20T04:44:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe13d433640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1380cd4d0 con 0x7fe138161500 2023-12-20T04:44:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe13d433640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1380ce4e0 con 0x7fe1380b8480 2023-12-20T04:44:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe1377fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe13806b460 0x7fe138160df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39918/0 (socket says 172.21.15.32:39918) 2023-12-20T04:44:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe1377fe640 1 -- 172.21.15.32:0/2974509175 learned_addr learned my addr 172.21.15.32:0/2974509175 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:02.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe137fff640 1 --2- 172.21.15.32:0/2974509175 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1380b8480 0x7fe13806af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe136ffd640 1 -- 172.21.15.32:0/2974509175 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2553997241 0 0) 0x7fe1380cd4d0 con 0x7fe138161500 2023-12-20T04:44:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe136ffd640 1 -- 172.21.15.32:0/2974509175 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe120003530 con 0x7fe138161500 2023-12-20T04:44:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe136ffd640 1 -- 172.21.15.32:0/2974509175 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 816787502 0 0) 0x7fe13806b8c0 con 0x7fe13806b460 2023-12-20T04:44:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe137fff640 1 -- 172.21.15.32:0/2974509175 >> v1:172.21.15.39:6789/0 conn(0x7fe138161500 legacy=0x7fe138153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe137fff640 1 -- 172.21.15.32:0/2974509175 >> v1:172.21.15.32:6789/0 conn(0x7fe13806b460 legacy=0x7fe138160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.121+0000 7fe137fff640 1 -- 172.21.15.32:0/2974509175 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1380cd720 con 0x7fe1380b8480 2023-12-20T04:44:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.125+0000 7fe137fff640 1 --2- 172.21.15.32:0/2974509175 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1380b8480 0x7fe13806af20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe1280097f0 tx=0x7fe12801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=812f2615f82052c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.125+0000 7fe136ffd640 1 -- 172.21.15.32:0/2974509175 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1280ab600 con 0x7fe1380b8480 2023-12-20T04:44:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.125+0000 7fe136ffd640 1 -- 172.21.15.32:0/2974509175 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1280a7050 con 0x7fe1380b8480 2023-12-20T04:44:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.125+0000 7fe136ffd640 1 -- 172.21.15.32:0/2974509175 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1280ab980 con 0x7fe1380b8480 2023-12-20T04:44:02.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.125+0000 7fe13d433640 1 -- 172.21.15.32:0/2974509175 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1380b8480 msgr2=0x7fe13806af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.125+0000 7fe13d433640 1 --2- 172.21.15.32:0/2974509175 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1380b8480 0x7fe13806af20 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fe1280097f0 tx=0x7fe12801c4c0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.125+0000 7fe13d433640 1 -- 172.21.15.32:0/2974509175 shutdown_connections 2023-12-20T04:44:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.125+0000 7fe13d433640 1 --2- 172.21.15.32:0/2974509175 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1380b8480 0x7fe13806af20 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.125+0000 7fe13d433640 1 -- 172.21.15.32:0/2974509175 >> 172.21.15.32:0/2974509175 conn(0x7fe1380d2a50 msgr2=0x7fe1380d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.125+0000 7fe13d433640 1 -- 172.21.15.32:0/2974509175 shutdown_connections 2023-12-20T04:44:02.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.125+0000 7fe13d433640 1 -- 172.21.15.32:0/2974509175 wait complete. 2023-12-20T04:44:02.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe13d433640 1 Processor -- start 2023-12-20T04:44:02.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe13d433640 1 -- start start 2023-12-20T04:44:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe13d433640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13806b460 0x7fe1380c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:02.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe13d433640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1380b8480 0x7fe13814d1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe13d433640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe138161500 0x7fe1380c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe13d433640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe138152ab0 con 0x7fe1380b8480 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe13d433640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe13814d6e0 con 0x7fe13806b460 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe13d433640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe138152110 con 0x7fe138161500 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe1377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1380b8480 0x7fe13814d1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe137fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13806b460 0x7fe1380c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe1377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1380b8480 0x7fe13814d1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42750/0 (socket says 172.21.15.32:42750) 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe137fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13806b460 0x7fe1380c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:52958/0 (socket says 172.21.15.32:52958) 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe1377fe640 1 -- 172.21.15.32:0/198706541 learned_addr learned my addr 172.21.15.32:0/198706541 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe13cc32640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe138161500 0x7fe1380c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe137fff640 1 -- 172.21.15.32:0/198706541 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe138161500 msgr2=0x7fe1380c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe137fff640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe138161500 0x7fe1380c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe137fff640 1 -- 172.21.15.32:0/198706541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1380b8480 msgr2=0x7fe13814d1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe137fff640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1380b8480 0x7fe13814d1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe137fff640 1 -- 172.21.15.32:0/198706541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe128009420 con 0x7fe13806b460 2023-12-20T04:44:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe1377fe640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1380b8480 0x7fe13814d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:02.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe137fff640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13806b460 0x7fe1380c9a90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe1280ab000 tx=0x7fe12801ad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe134ff9640 1 -- 172.21.15.32:0/198706541 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1280b7450 con 0x7fe13806b460 2023-12-20T04:44:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe134ff9640 1 -- 172.21.15.32:0/198706541 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1280b75b0 con 0x7fe13806b460 2023-12-20T04:44:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe134ff9640 1 -- 172.21.15.32:0/198706541 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1280b7940 con 0x7fe13806b460 2023-12-20T04:44:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe13d433640 1 -- 172.21.15.32:0/198706541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1381523f0 con 0x7fe13806b460 2023-12-20T04:44:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.129+0000 7fe13d433640 1 -- 172.21.15.32:0/198706541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe13814df70 con 0x7fe13806b460 2023-12-20T04:44:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.133+0000 7fe13d433640 1 -- 172.21.15.32:0/198706541 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe138152bf0 con 0x7fe13806b460 2023-12-20T04:44:02.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.133+0000 7fe134ff9640 1 -- 172.21.15.32:0/198706541 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fe1280b7aa0 con 0x7fe13806b460 2023-12-20T04:44:02.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.133+0000 7fe134ff9640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe124065af0 0x7fe124067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:02.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.133+0000 7fe134ff9640 1 -- 172.21.15.32:0/198706541 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe12811dc20 con 0x7fe13806b460 2023-12-20T04:44:02.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.133+0000 7fe1377fe640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe124065af0 0x7fe124067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:02.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.133+0000 7fe1377fe640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe124065af0 0x7fe124067fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe12c007b20 tx=0x7fe12c007a70 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:02.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.137+0000 7fe134ff9640 1 -- 172.21.15.32:0/198706541 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe12811de60 con 0x7fe13806b460 2023-12-20T04:44:02.321 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:02.321+0000 7f8de73eb640 -1 client.6714: SimpleRADOSStriper: lock: main.db: waiting for locks: lockers exclusive=1 tag= lockers=[client.6597:2ce667d1-ad1a-4873-a04e-ae7445737e9a:v1:172.21.15.32:0/2038860058] 2023-12-20T04:44:02.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.325+0000 7fe13d433640 1 -- 172.21.15.32:0/198706541 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fe1380cdfc0 con 0x7fe124065af0 2023-12-20T04:44:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.325+0000 7fe134ff9640 1 -- 172.21.15.32:0/198706541 <== mgr.6642 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe12c007360 con 0x7fe124065af0 2023-12-20T04:44:02.325 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:44:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T04:44:02.325 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:02.326 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 -- 172.21.15.32:0/198706541 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe124065af0 msgr2=0x7fe124067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe124065af0 0x7fe124067fb0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe12c007b20 tx=0x7fe12c007a70 comp rx=0 tx=0).stop 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 -- 172.21.15.32:0/198706541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13806b460 msgr2=0x7fe1380c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13806b460 0x7fe1380c9a90 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7fe1280ab000 tx=0x7fe12801ad00 comp rx=0 tx=0).stop 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 -- 172.21.15.32:0/198706541 shutdown_connections 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe124065af0 0x7fe124067fb0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe138161500 0x7fe1380c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1380b8480 0x7fe13814d1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 --2- 172.21.15.32:0/198706541 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe13806b460 0x7fe1380c9a90 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 -- 172.21.15.32:0/198706541 >> 172.21.15.32:0/198706541 conn(0x7fe1380d2a50 msgr2=0x7fe1380d17b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 -- 172.21.15.32:0/198706541 shutdown_connections 2023-12-20T04:44:02.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.329+0000 7fe13d433640 1 -- 172.21.15.32:0/198706541 wait complete. 2023-12-20T04:44:02.348 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T04:44:02.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.497+0000 7f9fcf59e640 1 Processor -- start 2023-12-20T04:44:02.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.497+0000 7f9fcf59e640 1 -- start start 2023-12-20T04:44:02.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.497+0000 7f9fcf59e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9fd0159980 0x7f9fd0071df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:02.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.497+0000 7f9fcf59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fd00cdd10 0x7f9fd00cafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:02.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.497+0000 7f9fcf59e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fd0072330 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.497+0000 7f9fcf59e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd00cb8f0 con 0x7f9fd00cdd10 2023-12-20T04:44:02.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.497+0000 7f9fcf59e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fd0159d60 con 0x7f9fd0159980 2023-12-20T04:44:02.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.497+0000 7f9fcdd9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fd00cdd10 0x7f9fd00cafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.497+0000 7f9fcdd9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fd00cdd10 0x7f9fd00cafa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42760/0 (socket says 172.21.15.32:42760) 2023-12-20T04:44:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.497+0000 7f9fcdd9b640 1 -- 172.21.15.32:0/1234834217 learned_addr learned my addr 172.21.15.32:0/1234834217 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.497+0000 7f9fce59c640 1 --2- 172.21.15.32:0/1234834217 >> v2:172.21.15.39:3300/0 conn(0x7f9fd0159980 0x7f9fd0071df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.501+0000 7f9fcdd9b640 1 -- 172.21.15.32:0/1234834217 >> v2:172.21.15.39:3300/0 conn(0x7f9fd0159980 msgr2=0x7f9fd0071df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.501+0000 7f9fcdd9b640 1 --2- 172.21.15.32:0/1234834217 >> v2:172.21.15.39:3300/0 conn(0x7f9fd0159980 0x7f9fd0071df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.501+0000 7f9fcdd9b640 1 -- 172.21.15.32:0/1234834217 >> v1:172.21.15.39:6789/0 conn(0x7f9fd00cb4e0 legacy=0x7f9fd015d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.501+0000 7f9fcdd9b640 1 -- 172.21.15.32:0/1234834217 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd00cc290 con 0x7f9fd00cdd10 2023-12-20T04:44:02.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.501+0000 7f9fcdd9b640 1 --2- 172.21.15.32:0/1234834217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd00cdd10 0x7f9fd00cafa0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9fd006cf10 tx=0x7f9fc401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35e53f13e38dbc5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:02.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.501+0000 7f9fcd59a640 1 -- 172.21.15.32:0/1234834217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40ac410 con 0x7f9fd00cdd10 2023-12-20T04:44:02.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.501+0000 7f9fcd59a640 1 -- 172.21.15.32:0/1234834217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc40ac570 con 0x7f9fd00cdd10 2023-12-20T04:44:02.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.501+0000 7f9fcd59a640 1 -- 172.21.15.32:0/1234834217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc40ac900 con 0x7f9fd00cdd10 2023-12-20T04:44:02.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 -- 172.21.15.32:0/1234834217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd00cdd10 msgr2=0x7f9fd00cafa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 --2- 172.21.15.32:0/1234834217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd00cdd10 0x7f9fd00cafa0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f9fd006cf10 tx=0x7f9fc401c3b0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 -- 172.21.15.32:0/1234834217 shutdown_connections 2023-12-20T04:44:02.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 --2- 172.21.15.32:0/1234834217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd00cdd10 0x7f9fd00cafa0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 --2- 172.21.15.32:0/1234834217 >> v2:172.21.15.39:3300/0 conn(0x7f9fd0159980 0x7f9fd0071df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 -- 172.21.15.32:0/1234834217 >> 172.21.15.32:0/1234834217 conn(0x7f9fd006c060 msgr2=0x7f9fd006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:02.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 -- 172.21.15.32:0/1234834217 shutdown_connections 2023-12-20T04:44:02.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 -- 172.21.15.32:0/1234834217 wait complete. 2023-12-20T04:44:02.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 Processor -- start 2023-12-20T04:44:02.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 -- start start 2023-12-20T04:44:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00cb4e0 0x7f9fd00ca260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:02.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd00cdd10 0x7f9fd01505e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fd0159980 0x7f9fd014cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd0152100 con 0x7f9fd00cdd10 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd0151c70 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcf59e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fd0151060 con 0x7f9fd0159980 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcdd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd00cdd10 0x7f9fd01505e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fce59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00cb4e0 0x7f9fd00ca260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcdd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd00cdd10 0x7f9fd01505e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42766/0 (socket says 172.21.15.32:42766) 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcdd9b640 1 -- 172.21.15.32:0/128111894 learned_addr learned my addr 172.21.15.32:0/128111894 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fced9d640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fd0159980 0x7f9fd014cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fce59c640 1 -- 172.21.15.32:0/128111894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fd0159980 msgr2=0x7f9fd014cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fce59c640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fd0159980 0x7f9fd014cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fce59c640 1 -- 172.21.15.32:0/128111894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd00cdd10 msgr2=0x7f9fd01505e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fce59c640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd00cdd10 0x7f9fd01505e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fce59c640 1 -- 172.21.15.32:0/128111894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fc40094d0 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fcdd9b640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd00cdd10 0x7f9fd01505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.505+0000 7f9fced9d640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fd0159980 0x7f9fd014cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.509+0000 7f9fce59c640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00cb4e0 0x7f9fd00ca260 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000e870 tx=0x7f9fc000ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.509+0000 7f9fbeffd640 1 -- 172.21.15.32:0/128111894 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000ee60 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.509+0000 7f9fbeffd640 1 -- 172.21.15.32:0/128111894 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fc000c940 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.509+0000 7f9fbeffd640 1 -- 172.21.15.32:0/128111894 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fc000ccd0 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.509+0000 7f9fcf59e640 1 -- 172.21.15.32:0/128111894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fd0151340 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.509+0000 7f9fcf59e640 1 -- 172.21.15.32:0/128111894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fd0151900 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.509+0000 7f9fcf59e640 1 -- 172.21.15.32:0/128111894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fd0074100 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.509+0000 7f9fbeffd640 1 -- 172.21.15.32:0/128111894 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 38) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f9fc0013470 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.509+0000 7f9fbeffd640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fb0065af0 0x7f9fb0067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.509+0000 7f9fbeffd640 1 -- 172.21.15.32:0/128111894 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fc00832b0 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.509+0000 7f9fcdd9b640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fb0065af0 0x7f9fb0067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:02.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.513+0000 7f9fcdd9b640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fb0065af0 0x7f9fb0067fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9fc401c3f0 tx=0x7f9fc4009040 comp rx=0 tx=0).ready entity=mgr.6642 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:02.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.513+0000 7f9fbeffd640 1 -- 172.21.15.32:0/128111894 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9fc0050bf0 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.717+0000 7f9fcf59e640 1 -- 172.21.15.32:0/128111894 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f9fd00cd390 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.937+0000 7f9fbeffd640 1 -- 172.21.15.32:0/128111894 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 39) v1 ==== 85291+0+0 (secure 0 0 0) 0x7f9fc0050fc0 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.937+0000 7f9fbeffd640 1 -- 172.21.15.32:0/128111894 <== mon.1 v2:172.21.15.39:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v39) v1 ==== 92+0+0 (secure 0 0 0) 0x7f9fc006e0e0 con 0x7f9fd00cb4e0 2023-12-20T04:44:02.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 -- 172.21.15.32:0/128111894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fb0065af0 msgr2=0x7f9fb0067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fb0065af0 0x7f9fb0067fb0 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f9fc401c3f0 tx=0x7f9fc4009040 comp rx=0 tx=0).stop 2023-12-20T04:44:02.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 -- 172.21.15.32:0/128111894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00cb4e0 msgr2=0x7f9fd00ca260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:02.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00cb4e0 0x7f9fd00ca260 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f9fc000e870 tx=0x7f9fc000ed30 comp rx=0 tx=0).stop 2023-12-20T04:44:02.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 -- 172.21.15.32:0/128111894 shutdown_connections 2023-12-20T04:44:02.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fb0065af0 0x7f9fb0067fb0 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fd0159980 0x7f9fd014cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fd00cdd10 0x7f9fd01505e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 --2- 172.21.15.32:0/128111894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fd00cb4e0 0x7f9fd00ca260 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:02.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 -- 172.21.15.32:0/128111894 >> 172.21.15.32:0/128111894 conn(0x7f9fd006c060 msgr2=0x7f9fd0165090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:02.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 -- 172.21.15.32:0/128111894 shutdown_connections 2023-12-20T04:44:02.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:02.941+0000 7f9fcf59e640 1 -- 172.21.15.32:0/128111894 wait complete. 2023-12-20T04:44:02.960 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T04:44:03.107 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:03.107+0000 7f8720d8c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:44:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f646856d640 1 Processor -- start 2023-12-20T04:44:03.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f646856d640 1 -- start start 2023-12-20T04:44:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f646856d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f64600b0760 0x7f64600b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f646856d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64600cc4b0 0x7f64600cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f646856d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64600b1100 con 0x7f6460157480 2023-12-20T04:44:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f646856d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64600c9f50 con 0x7f64600cc4b0 2023-12-20T04:44:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f646856d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64600cde20 con 0x7f64600b0760 2023-12-20T04:44:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f6466b0a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64600cc4b0 0x7f64600cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f6466309640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6460157480 0x7f64600c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39936/0 (socket says 172.21.15.32:39936) 2023-12-20T04:44:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f6466b0a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64600cc4b0 0x7f64600cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42796/0 (socket says 172.21.15.32:42796) 2023-12-20T04:44:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f6466309640 1 -- 172.21.15.32:0/467270262 learned_addr learned my addr 172.21.15.32:0/467270262 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:03.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f6465b08640 1 --2- 172.21.15.32:0/467270262 >> v2:172.21.15.39:3300/0 conn(0x7f64600b0760 0x7f64600b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f6466b0a640 1 -- 172.21.15.32:0/467270262 >> v2:172.21.15.39:3300/0 conn(0x7f64600b0760 msgr2=0x7f64600b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f6466b0a640 1 --2- 172.21.15.32:0/467270262 >> v2:172.21.15.39:3300/0 conn(0x7f64600b0760 0x7f64600b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f6466b0a640 1 -- 172.21.15.32:0/467270262 >> v1:172.21.15.32:6789/0 conn(0x7f6460157480 legacy=0x7f64600c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f6466b0a640 1 -- 172.21.15.32:0/467270262 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64600c8e30 con 0x7f64600cc4b0 2023-12-20T04:44:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f6465307640 1 -- 172.21.15.32:0/467270262 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3390789807 0 0) 0x7f64600b1100 con 0x7f6460157480 2023-12-20T04:44:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.113+0000 7f6465b08640 1 --2- 172.21.15.32:0/467270262 >> v2:172.21.15.39:3300/0 conn(0x7f64600b0760 0x7f64600b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:03.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f6466b0a640 1 --2- 172.21.15.32:0/467270262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64600cc4b0 0x7f64600cc8d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f646006e0d0 tx=0x7f645801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e396ba15ec0c00 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f6465307640 1 -- 172.21.15.32:0/467270262 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64580a5910 con 0x7f64600cc4b0 2023-12-20T04:44:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f6465307640 1 -- 172.21.15.32:0/467270262 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64580a5a70 con 0x7f64600cc4b0 2023-12-20T04:44:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f6465307640 1 -- 172.21.15.32:0/467270262 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64580a5e00 con 0x7f64600cc4b0 2023-12-20T04:44:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f646856d640 1 -- 172.21.15.32:0/467270262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64600cc4b0 msgr2=0x7f64600cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f646856d640 1 --2- 172.21.15.32:0/467270262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64600cc4b0 0x7f64600cc8d0 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f646006e0d0 tx=0x7f645801c850 comp rx=0 tx=0).stop 2023-12-20T04:44:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f646856d640 1 -- 172.21.15.32:0/467270262 shutdown_connections 2023-12-20T04:44:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f646856d640 1 --2- 172.21.15.32:0/467270262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64600cc4b0 0x7f64600cc8d0 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f646856d640 1 --2- 172.21.15.32:0/467270262 >> v2:172.21.15.39:3300/0 conn(0x7f64600b0760 0x7f64600b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f646856d640 1 -- 172.21.15.32:0/467270262 >> 172.21.15.32:0/467270262 conn(0x7f646006a510 msgr2=0x7f64600ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f646856d640 1 -- 172.21.15.32:0/467270262 shutdown_connections 2023-12-20T04:44:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f646856d640 1 -- 172.21.15.32:0/467270262 wait complete. 2023-12-20T04:44:03.123 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f90c7448180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:44:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f646856d640 1 Processor -- start 2023-12-20T04:44:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.121+0000 7f646856d640 1 -- start start 2023-12-20T04:44:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f646856d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64600b0760 0x7f6460150d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f646856d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64600cc4b0 0x7f6460158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f646856d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6460157480 0x7f64601587b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f646856d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460151eb0 con 0x7f6460157480 2023-12-20T04:44:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f646856d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6460154740 con 0x7f64600cc4b0 2023-12-20T04:44:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f646856d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6460158cf0 con 0x7f64600b0760 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6466b0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6460157480 0x7f64601587b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6465b08640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64600cc4b0 0x7f6460158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6466b0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6460157480 0x7f64601587b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42800/0 (socket says 172.21.15.32:42800) 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6466b0a640 1 -- 172.21.15.32:0/2473598785 learned_addr learned my addr 172.21.15.32:0/2473598785 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6466309640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64600b0760 0x7f6460150d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6466b0a640 1 -- 172.21.15.32:0/2473598785 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64600b0760 msgr2=0x7f6460150d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6466b0a640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64600b0760 0x7f6460150d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6466b0a640 1 -- 172.21.15.32:0/2473598785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64600cc4b0 msgr2=0x7f6460158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6466b0a640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64600cc4b0 0x7f6460158270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6466b0a640 1 -- 172.21.15.32:0/2473598785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6450009420 con 0x7f6460157480 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6465b08640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64600cc4b0 0x7f6460158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6466b0a640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6460157480 0x7f64601587b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f645801a480 tx=0x7f64580195f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6456ffd640 1 -- 172.21.15.32:0/2473598785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64580b9860 con 0x7f6460157480 2023-12-20T04:44:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6456ffd640 1 -- 172.21.15.32:0/2473598785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64580b99c0 con 0x7f6460157480 2023-12-20T04:44:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f646856d640 1 -- 172.21.15.32:0/2473598785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6458009a70 con 0x7f6460157480 2023-12-20T04:44:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f6456ffd640 1 -- 172.21.15.32:0/2473598785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64580b9d70 con 0x7f6460157480 2023-12-20T04:44:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f646856d640 1 -- 172.21.15.32:0/2473598785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6460159260 con 0x7f6460157480 2023-12-20T04:44:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.125+0000 7f646856d640 1 -- 172.21.15.32:0/2473598785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f646015adf0 con 0x7f6460157480 2023-12-20T04:44:03.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.129+0000 7f6456ffd640 1 -- 172.21.15.32:0/2473598785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 85291+0+0 (secure 0 0 0) 0x7f64580ad440 con 0x7f6460157480 2023-12-20T04:44:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.129+0000 7f6456ffd640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6444065b70 0x7f6444068030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.129+0000 7f6466309640 1 -- 172.21.15.32:0/2473598785 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6444065b70 msgr2=0x7f6444068030 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.129+0000 7f6456ffd640 1 -- 172.21.15.32:0/2473598785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f645811e3f0 con 0x7f6460157480 2023-12-20T04:44:03.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.129+0000 7f6466309640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6444065b70 0x7f6444068030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:03.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.133+0000 7f6456ffd640 1 -- 172.21.15.32:0/2473598785 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f646015adf0 con 0x7f6460157480 2023-12-20T04:44:03.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.293+0000 7f646856d640 1 -- 172.21.15.32:0/2473598785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f646015adf0 con 0x7f6460157480 2023-12-20T04:44:03.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f6456ffd640 1 -- 172.21.15.32:0/2473598785 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) v1 ==== 138+0+0 (secure 0 0 0) 0x7f646015adf0 con 0x7f6460157480 2023-12-20T04:44:03.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 -- 172.21.15.32:0/2473598785 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6444065b70 msgr2=0x7f6444068030 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:03.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6444065b70 0x7f6444068030 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 -- 172.21.15.32:0/2473598785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6460157480 msgr2=0x7f64601587b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6460157480 0x7f64601587b0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f645801a480 tx=0x7f64580195f0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 -- 172.21.15.32:0/2473598785 shutdown_connections 2023-12-20T04:44:03.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6444065b70 0x7f6444068030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6460157480 0x7f64601587b0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64600cc4b0 0x7f6460158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 --2- 172.21.15.32:0/2473598785 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64600b0760 0x7f6460150d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 -- 172.21.15.32:0/2473598785 >> 172.21.15.32:0/2473598785 conn(0x7f646006a510 msgr2=0x7f64600ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:03.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 -- 172.21.15.32:0/2473598785 shutdown_connections 2023-12-20T04:44:03.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.301+0000 7f646856d640 1 -- 172.21.15.32:0/2473598785 wait complete. 2023-12-20T04:44:03.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T04:44:03.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a4139640 1 Processor -- start 2023-12-20T04:44:03.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a4139640 1 -- start start 2023-12-20T04:44:03.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a4139640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c15ce60 0x7f549c0d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a4139640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f549c0d3130 0x7f549c0d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a4139640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f549c0d2650 con 0x7f549c0d1300 2023-12-20T04:44:03.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a4139640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f549c0d3570 con 0x7f549c0d3130 2023-12-20T04:44:03.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a4139640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0d1a40 con 0x7f549c15ce60 2023-12-20T04:44:03.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a16d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f549c0d3130 0x7f549c0d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a16d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f549c0d3130 0x7f549c0d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42816/0 (socket says 172.21.15.32:42816) 2023-12-20T04:44:03.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a16d4640 1 -- 172.21.15.32:0/1747778687 learned_addr learned my addr 172.21.15.32:0/1747778687 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:03.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a1ed5640 1 --2- 172.21.15.32:0/1747778687 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c15ce60 0x7f549c0d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a16d4640 1 -- 172.21.15.32:0/1747778687 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c15ce60 msgr2=0x7f549c0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:03.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a16d4640 1 --2- 172.21.15.32:0/1747778687 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c15ce60 0x7f549c0d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a16d4640 1 -- 172.21.15.32:0/1747778687 >> v1:172.21.15.39:6789/0 conn(0x7f549c0d1300 legacy=0x7f549c0b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a16d4640 1 -- 172.21.15.32:0/1747778687 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f549c0d1d20 con 0x7f549c0d3130 2023-12-20T04:44:03.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.477+0000 7f54a1ed5640 1 --2- 172.21.15.32:0/1747778687 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c15ce60 0x7f549c0d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:03.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a16d4640 1 --2- 172.21.15.32:0/1747778687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f549c0d3130 0x7f549c0d0dc0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f548c009870 tx=0x7f548c01ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e73fad8dee3b9463 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:03.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a0ed3640 1 -- 172.21.15.32:0/1747778687 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0a8a30 con 0x7f549c0d3130 2023-12-20T04:44:03.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a0ed3640 1 -- 172.21.15.32:0/1747778687 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c0a8b90 con 0x7f549c0d3130 2023-12-20T04:44:03.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a0ed3640 1 -- 172.21.15.32:0/1747778687 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0b0440 con 0x7f549c0d3130 2023-12-20T04:44:03.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a4139640 1 -- 172.21.15.32:0/1747778687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f549c0d3130 msgr2=0x7f549c0d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a4139640 1 --2- 172.21.15.32:0/1747778687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f549c0d3130 0x7f549c0d0dc0 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f548c009870 tx=0x7f548c01ed30 comp rx=0 tx=0).stop 2023-12-20T04:44:03.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a4139640 1 -- 172.21.15.32:0/1747778687 shutdown_connections 2023-12-20T04:44:03.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a4139640 1 --2- 172.21.15.32:0/1747778687 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f549c0d3130 0x7f549c0d0dc0 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a4139640 1 --2- 172.21.15.32:0/1747778687 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c15ce60 0x7f549c0d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a4139640 1 -- 172.21.15.32:0/1747778687 >> 172.21.15.32:0/1747778687 conn(0x7f549c06b5e0 msgr2=0x7f549c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:03.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a4139640 1 -- 172.21.15.32:0/1747778687 shutdown_connections 2023-12-20T04:44:03.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.481+0000 7f54a4139640 1 -- 172.21.15.32:0/1747778687 wait complete. 2023-12-20T04:44:03.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a4139640 1 Processor -- start 2023-12-20T04:44:03.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a4139640 1 -- start start 2023-12-20T04:44:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a4139640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c0d1300 0x7f549c0aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a4139640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f549c0d3130 0x7f549c151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a4139640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c15ce60 0x7f549c0ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a4139640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0aa500 con 0x7f549c0d3130 2023-12-20T04:44:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a4139640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f549c1516a0 con 0x7f549c15ce60 2023-12-20T04:44:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a4139640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f549c0a95a0 con 0x7f549c0d1300 2023-12-20T04:44:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a1ed5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c0d1300 0x7f549c0aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a16d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f549c0d3130 0x7f549c151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a1ed5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c0d1300 0x7f549c0aa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:52984/0 (socket says 172.21.15.32:52984) 2023-12-20T04:44:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a1ed5640 1 -- 172.21.15.32:0/2324772792 learned_addr learned my addr 172.21.15.32:0/2324772792 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a26d6640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c15ce60 0x7f549c0ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a16d4640 1 -- 172.21.15.32:0/2324772792 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c0d1300 msgr2=0x7f549c0aa920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a16d4640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c0d1300 0x7f549c0aa920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a16d4640 1 -- 172.21.15.32:0/2324772792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c15ce60 msgr2=0x7f549c0ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a16d4640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c15ce60 0x7f549c0ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a16d4640 1 -- 172.21.15.32:0/2324772792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f548c0094d0 con 0x7f549c0d3130 2023-12-20T04:44:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a1ed5640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c0d1300 0x7f549c0aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a16d4640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f549c0d3130 0x7f549c151160 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f549c0b5850 tx=0x7f548c01bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54927fc640 1 -- 172.21.15.32:0/2324772792 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0be070 con 0x7f549c0d3130 2023-12-20T04:44:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54927fc640 1 -- 172.21.15.32:0/2324772792 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f548c0ba400 con 0x7f549c0d3130 2023-12-20T04:44:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54927fc640 1 -- 172.21.15.32:0/2324772792 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f548c0ba7d0 con 0x7f549c0d3130 2023-12-20T04:44:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a4139640 1 -- 172.21.15.32:0/2324772792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f549c0a9880 con 0x7f549c0d3130 2023-12-20T04:44:03.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.485+0000 7f54a4139640 1 -- 172.21.15.32:0/2324772792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f549c0a9e40 con 0x7f549c0d3130 2023-12-20T04:44:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.489+0000 7f54a4139640 1 -- 172.21.15.32:0/2324772792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f549c04e000 con 0x7f549c0d3130 2023-12-20T04:44:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.489+0000 7f54927fc640 1 -- 172.21.15.32:0/2324772792 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 85291+0+0 (secure 0 0 0) 0x7f548c0c3410 con 0x7f549c0d3130 2023-12-20T04:44:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.489+0000 7f54927fc640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5480065b40 0x7f5480068000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.489+0000 7f54a1ed5640 1 -- 172.21.15.32:0/2324772792 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5480065b40 msgr2=0x7f5480068000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.489+0000 7f54927fc640 1 -- 172.21.15.32:0/2324772792 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f548c120c00 con 0x7f549c0d3130 2023-12-20T04:44:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.489+0000 7f54a1ed5640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5480065b40 0x7f5480068000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:03.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.493+0000 7f54927fc640 1 -- 172.21.15.32:0/2324772792 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f548c0ee610 con 0x7f549c0d3130 2023-12-20T04:44:03.548 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:03.547+0000 7f8720d8c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:44:03.591 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:03.589+0000 7f90c7448180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:44:03.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.653+0000 7f54a4139640 1 -- 172.21.15.32:0/2324772792 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f549c0b5910 con 0x7f549c0d3130 2023-12-20T04:44:03.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54927fc640 1 -- 172.21.15.32:0/2324772792 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) v1 ==== 133+0+0 (secure 0 0 0) 0x7f548c0f1c30 con 0x7f549c0d3130 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 -- 172.21.15.32:0/2324772792 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5480065b40 msgr2=0x7f5480068000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5480065b40 0x7f5480068000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 -- 172.21.15.32:0/2324772792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f549c0d3130 msgr2=0x7f549c151160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f549c0d3130 0x7f549c151160 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7f549c0b5850 tx=0x7f548c01bad0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 -- 172.21.15.32:0/2324772792 shutdown_connections 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5480065b40 0x7f5480068000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f549c15ce60 0x7f549c0ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f549c0d3130 0x7f549c151160 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 --2- 172.21.15.32:0/2324772792 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f549c0d1300 0x7f549c0aa920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 -- 172.21.15.32:0/2324772792 >> 172.21.15.32:0/2324772792 conn(0x7f549c06b5e0 msgr2=0x7f549c0b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 -- 172.21.15.32:0/2324772792 shutdown_connections 2023-12-20T04:44:03.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.661+0000 7f54a4139640 1 -- 172.21.15.32:0/2324772792 wait complete. 2023-12-20T04:44:03.682 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-12-20T04:44:03.707 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:03.707+0000 7f8720d8c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:44:03.753 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:03.753+0000 7f90c7448180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:44:03.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc660995640 1 Processor -- start 2023-12-20T04:44:03.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc660995640 1 -- start start 2023-12-20T04:44:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc660995640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc65c15cc30 0x7fc65c0b8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc660995640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc65c0d3060 0x7fc65c0b5af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc660995640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc65c0d2090 con 0x7fc65c0d0ca0 2023-12-20T04:44:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc660995640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65c15d010 con 0x7fc65c15cc30 2023-12-20T04:44:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc660995640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc65c0d1480 con 0x7fc65c0d3060 2023-12-20T04:44:03.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc65ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc65c15cc30 0x7fc65c0b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc65ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc65c15cc30 0x7fc65c0b8520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42838/0 (socket says 172.21.15.32:42838) 2023-12-20T04:44:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc65a59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc65c0d0ca0 0x7fc65c0d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39944/0 (socket says 172.21.15.32:39944) 2023-12-20T04:44:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc65ad9d640 1 -- 172.21.15.32:0/4040365371 learned_addr learned my addr 172.21.15.32:0/4040365371 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.829+0000 7fc65b59e640 1 --2- 172.21.15.32:0/4040365371 >> v2:172.21.15.39:3300/0 conn(0x7fc65c0d3060 0x7fc65c0b5af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.833+0000 7fc65ad9d640 1 -- 172.21.15.32:0/4040365371 >> v2:172.21.15.39:3300/0 conn(0x7fc65c0d3060 msgr2=0x7fc65c0b5af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.833+0000 7fc65ad9d640 1 --2- 172.21.15.32:0/4040365371 >> v2:172.21.15.39:3300/0 conn(0x7fc65c0d3060 0x7fc65c0b5af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.833+0000 7fc65ad9d640 1 -- 172.21.15.32:0/4040365371 >> v1:172.21.15.32:6789/0 conn(0x7fc65c0d0ca0 legacy=0x7fc65c0d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.833+0000 7fc659d9b640 1 -- 172.21.15.32:0/4040365371 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 311150856 0 0) 0x7fc65c0d2090 con 0x7fc65c0d0ca0 2023-12-20T04:44:03.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.833+0000 7fc65ad9d640 1 -- 172.21.15.32:0/4040365371 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc65c0d1760 con 0x7fc65c15cc30 2023-12-20T04:44:03.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.833+0000 7fc65ad9d640 1 --2- 172.21.15.32:0/4040365371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc65c15cc30 0x7fc65c0b8520 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0d2000 tx=0x7fc64c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7111d7ae5088602b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:03.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.833+0000 7fc659d9b640 1 -- 172.21.15.32:0/4040365371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0aa430 con 0x7fc65c15cc30 2023-12-20T04:44:03.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.833+0000 7fc659d9b640 1 -- 172.21.15.32:0/4040365371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc64c0aa590 con 0x7fc65c15cc30 2023-12-20T04:44:03.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.833+0000 7fc659d9b640 1 -- 172.21.15.32:0/4040365371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc64c0aa920 con 0x7fc65c15cc30 2023-12-20T04:44:03.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.833+0000 7fc660995640 1 -- 172.21.15.32:0/4040365371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc65c15cc30 msgr2=0x7fc65c0b8520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.833+0000 7fc660995640 1 --2- 172.21.15.32:0/4040365371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc65c15cc30 0x7fc65c0b8520 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc65c0d2000 tx=0x7fc64c01c4d0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 -- 172.21.15.32:0/4040365371 shutdown_connections 2023-12-20T04:44:03.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 --2- 172.21.15.32:0/4040365371 >> v2:172.21.15.39:3300/0 conn(0x7fc65c0d3060 0x7fc65c0b5af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 --2- 172.21.15.32:0/4040365371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc65c15cc30 0x7fc65c0b8520 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 -- 172.21.15.32:0/4040365371 >> 172.21.15.32:0/4040365371 conn(0x7fc65c065ce0 msgr2=0x7fc65c0660f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 -- 172.21.15.32:0/4040365371 shutdown_connections 2023-12-20T04:44:03.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 -- 172.21.15.32:0/4040365371 wait complete. 2023-12-20T04:44:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 Processor -- start 2023-12-20T04:44:03.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 -- start start 2023-12-20T04:44:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc65c0d0ca0 0x7fc65c14eae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc65c0d3060 0x7fc65c13fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc65c15cc30 0x7fc65c14db20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c140140 con 0x7fc65c0d3060 2023-12-20T04:44:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c1598e0 con 0x7fc65c15cc30 2023-12-20T04:44:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc660995640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc65c154890 con 0x7fc65c0d0ca0 2023-12-20T04:44:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc65a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc65c0d3060 0x7fc65c13fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc65a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc65c0d3060 0x7fc65c13fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42842/0 (socket says 172.21.15.32:42842) 2023-12-20T04:44:03.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc65a59c640 1 -- 172.21.15.32:0/1529857107 learned_addr learned my addr 172.21.15.32:0/1529857107 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc65ad9d640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc65c0d0ca0 0x7fc65c14eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc65b59e640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc65c15cc30 0x7fc65c14db20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc65a59c640 1 -- 172.21.15.32:0/1529857107 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc65c0d0ca0 msgr2=0x7fc65c14eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc65a59c640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc65c0d0ca0 0x7fc65c14eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc65a59c640 1 -- 172.21.15.32:0/1529857107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc65c15cc30 msgr2=0x7fc65c14db20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc65a59c640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc65c15cc30 0x7fc65c14db20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.837+0000 7fc65a59c640 1 -- 172.21.15.32:0/1529857107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc64c0094d0 con 0x7fc65c0d3060 2023-12-20T04:44:03.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.841+0000 7fc65a59c640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc65c0d3060 0x7fc65c13fc00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc6500011f0 tx=0x7fc65000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.841+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6500074e0 con 0x7fc65c0d3060 2023-12-20T04:44:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.841+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc650007640 con 0x7fc65c0d3060 2023-12-20T04:44:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.841+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc650004a10 con 0x7fc65c0d3060 2023-12-20T04:44:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.841+0000 7fc660995640 1 -- 172.21.15.32:0/1529857107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc65c159b30 con 0x7fc65c0d3060 2023-12-20T04:44:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.841+0000 7fc660995640 1 -- 172.21.15.32:0/1529857107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc65c15a0f0 con 0x7fc65c0d3060 2023-12-20T04:44:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.841+0000 7fc660995640 1 -- 172.21.15.32:0/1529857107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc65c0d2500 con 0x7fc65c0d3060 2023-12-20T04:44:03.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.841+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 39) v1 ==== 85291+0+0 (secure 0 0 0) 0x7fc650004b70 con 0x7fc65c0d3060 2023-12-20T04:44:03.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.845+0000 7fc6437fe640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 0x7fc648068000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:03.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.845+0000 7fc65ad9d640 1 -- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 msgr2=0x7fc648068000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:03.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.845+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(93..93 src has 1..93) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc650082490 con 0x7fc65c0d3060 2023-12-20T04:44:03.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.845+0000 7fc65ad9d640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 0x7fc648068000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:03.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:03.845+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc65c0d2500 con 0x7fc65c0d3060 2023-12-20T04:44:04.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:04.025+0000 7fc660995640 1 -- 172.21.15.32:0/1529857107 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc65c0d2500 con 0x7fc648065b40 2023-12-20T04:44:04.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:04.049+0000 7fc65ad9d640 1 -- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 msgr2=0x7fc648068000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:04.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:04.049+0000 7fc65ad9d640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 0x7fc648068000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:44:04.216 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:04.215+0000 7f8720d8c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:04.267 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:04.269+0000 7f90c7448180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:04.278 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:04.279+0000 7f8720d8c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:44:04.329 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:04.329+0000 7f90c7448180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:44:04.337 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:04.335+0000 7f8720d8c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:44:04.388 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:04.389+0000 7f90c7448180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:44:04.428 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:04.427+0000 7f8720d8c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:44:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:04.453+0000 7fc65ad9d640 1 -- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 msgr2=0x7fc648068000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:04.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:04.453+0000 7fc65ad9d640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 0x7fc648068000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T04:44:04.479 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:04.481+0000 7f90c7448180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:44:04.487 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:04.487+0000 7f8720d8c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:44:04.539 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:04.537+0000 7f90c7448180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:44:04.575 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:04.575+0000 7f8720d8c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:44:04.627 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:04.629+0000 7f90c7448180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:44:04.634 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:04.635+0000 7f8720d8c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:44:04.687 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:04.685+0000 7f90c7448180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:44:04.785 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:04.783+0000 7f8720d8c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:44:04.838 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:04.837+0000 7f90c7448180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:44:04.846 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:04.847+0000 7f8720d8c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:44:04.899 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:04.901+0000 7f90c7448180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:44:05.096 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:44:05.096 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:44:05.096 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:44:05.104 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:05.103+0000 7f8720d8c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:44:05.150 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:44:05.150 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:44:05.150 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:44:05.158 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:05.157+0000 7f90c7448180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:44:05.247 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:05.247+0000 7f8720d8c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:05.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:05.257+0000 7fc65ad9d640 1 -- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 msgr2=0x7fc648068000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:05.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:05.257+0000 7fc65ad9d640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 0x7fc648068000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T04:44:05.300 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:05.301+0000 7f90c7448180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:05.377 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:05.375+0000 7f8720d8c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:05.431 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:05.433+0000 7f90c7448180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:05.451 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:05.451+0000 7f8720d8c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:44:05.505 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:05.505+0000 7f90c7448180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:44:05.593 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:05.591+0000 7f8720d8c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:05.647 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:05.649+0000 7f90c7448180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:06.161 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:06.159+0000 7f8720d8c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:44:06.215 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:06.217+0000 7f90c7448180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:44:06.249 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:06.247+0000 7f8720d8c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:44:06.304 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:06.305+0000 7f90c7448180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:44:06.406 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:06.407+0000 7f8720d8c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:44:06.460 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:06.461+0000 7f90c7448180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:44:06.494 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:06.491+0000 7f8720d8c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:44:06.547 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:06.549+0000 7f90c7448180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:44:06.827 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:06.827+0000 7f8720d8c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:44:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:06.861+0000 7fc65ad9d640 1 -- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 msgr2=0x7fc648068000 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:06.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:06.861+0000 7fc65ad9d640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 0x7fc648068000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-20T04:44:06.881 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:06.881+0000 7f90c7448180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:44:06.890 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:06.891+0000 7f8720d8c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:44:06.946 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:06.945+0000 7f90c7448180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:44:06.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:06.953+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 40) v1 ==== 85291+0+0 (secure 0 0 0) 0x7fc650050270 con 0x7fc65c0d3060 2023-12-20T04:44:06.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:06.965+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 41) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fc650080450 con 0x7fc65c0d3060 2023-12-20T04:44:06.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:06.965+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 msgr2=0x7fc648068000 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:06.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:06.965+0000 7fc6437fe640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 0x7fc648068000 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:07.080 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:07.081+0000 7f9049652640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:07.080 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:07.081+0000 7f9049652640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:07.080 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:07.081+0000 7f9049652640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:07.081 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:07.081+0000 7f9049652640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:07.081 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:07.081+0000 7f9049652640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:07.081 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:07.081+0000 7f904be57640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:07.081 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:07.081+0000 7f904be57640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:07.081 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:07.081+0000 7f904be57640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:07.081 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:07.081+0000 7f904be57640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:07.081 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:07.081+0000 7f904be57640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:07.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:07.973+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 <== mon.0 v2:172.21.15.32:3300/0 9 ==== mgrmap(e 42) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fc65001ed50 con 0x7fc65c0d3060 2023-12-20T04:44:07.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:07.973+0000 7fc6437fe640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 0x7fc648067f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:07.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:07.973+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fc65c0d2500 con 0x7fc648065b40 2023-12-20T04:44:07.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:07.973+0000 7fc65ad9d640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 0x7fc648067f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:07.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:07.981+0000 7fc65ad9d640 1 --2- 172.21.15.32:0/1529857107 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc648065b40 0x7fc648067f80 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fc64c0ae000 tx=0x7fc64c0a7000 comp rx=0 tx=0).ready entity=mgr.7032 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:07.988 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:07.989+0000 7f9053666640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-20T04:44:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:07.989+0000 7fc6437fe640 1 -- 172.21.15.32:0/1529857107 <== mgr.7032 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7fc65c0d2500 con 0x7fc648065b40 2023-12-20T04:44:07.989 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-20T04:44:07.998 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-20T04:44:07.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T04:44:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18e2525640 1 Processor -- start 2023-12-20T04:44:08.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18e2525640 1 -- start start 2023-12-20T04:44:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18e2525640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18dc157480 0x7f18dc0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18e2525640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18dc0b0760 0x7f18dc0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18e2525640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18dc0b1100 con 0x7f18dc15d660 2023-12-20T04:44:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18e2525640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc0cde20 con 0x7f18dc157480 2023-12-20T04:44:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18e2525640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18dc0c8e30 con 0x7f18dc0b0760 2023-12-20T04:44:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18dbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18dc157480 0x7f18dc0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18dbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f18dc157480 0x7f18dc0c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42972/0 (socket says 172.21.15.32:42972) 2023-12-20T04:44:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18dbfff640 1 -- 172.21.15.32:0/1876912030 learned_addr learned my addr 172.21.15.32:0/1876912030 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:08.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18db7fe640 1 --2- 172.21.15.32:0/1876912030 >> v2:172.21.15.39:3300/0 conn(0x7f18dc0b0760 0x7f18dc0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18dbfff640 1 -- 172.21.15.32:0/1876912030 >> v2:172.21.15.39:3300/0 conn(0x7f18dc0b0760 msgr2=0x7f18dc0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18dbfff640 1 --2- 172.21.15.32:0/1876912030 >> v2:172.21.15.39:3300/0 conn(0x7f18dc0b0760 0x7f18dc0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18dbfff640 1 -- 172.21.15.32:0/1876912030 >> v1:172.21.15.39:6789/0 conn(0x7f18dc15d660 legacy=0x7f18dc15da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:08.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.153+0000 7f18dbfff640 1 -- 172.21.15.32:0/1876912030 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18dc0ce070 con 0x7f18dc157480 2023-12-20T04:44:08.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18dbfff640 1 --2- 172.21.15.32:0/1876912030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc157480 0x7f18dc0c9840 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f18dc06e0d0 tx=0x7f18cc01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17e36734c58c4e7a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18daffd640 1 -- 172.21.15.32:0/1876912030 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc0ab640 con 0x7f18dc157480 2023-12-20T04:44:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18daffd640 1 -- 172.21.15.32:0/1876912030 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18cc0a7070 con 0x7f18dc157480 2023-12-20T04:44:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18daffd640 1 -- 172.21.15.32:0/1876912030 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18cc0ab9c0 con 0x7f18dc157480 2023-12-20T04:44:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18e2525640 1 -- 172.21.15.32:0/1876912030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc157480 msgr2=0x7f18dc0c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:08.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18e2525640 1 --2- 172.21.15.32:0/1876912030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc157480 0x7f18dc0c9840 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f18dc06e0d0 tx=0x7f18cc01c440 comp rx=0 tx=0).stop 2023-12-20T04:44:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18e2525640 1 -- 172.21.15.32:0/1876912030 shutdown_connections 2023-12-20T04:44:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18e2525640 1 --2- 172.21.15.32:0/1876912030 >> v2:172.21.15.39:3300/0 conn(0x7f18dc0b0760 0x7f18dc0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18e2525640 1 --2- 172.21.15.32:0/1876912030 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc157480 0x7f18dc0c9840 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18e2525640 1 -- 172.21.15.32:0/1876912030 >> 172.21.15.32:0/1876912030 conn(0x7f18dc06a510 msgr2=0x7f18dc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18e2525640 1 -- 172.21.15.32:0/1876912030 shutdown_connections 2023-12-20T04:44:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18e2525640 1 -- 172.21.15.32:0/1876912030 wait complete. 2023-12-20T04:44:08.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.157+0000 7f18e2525640 1 Processor -- start 2023-12-20T04:44:08.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e2525640 1 -- start start 2023-12-20T04:44:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e2525640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0b0760 0x7f18dc0c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e2525640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18dc157480 0x7f18dc151cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:08.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e2525640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc15d660 0x7f18dc15b800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e2525640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc1f9970 con 0x7f18dc15d660 2023-12-20T04:44:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e2525640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc15c170 con 0x7f18dc0b0760 2023-12-20T04:44:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e2525640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18dc0b2be0 con 0x7f18dc157480 2023-12-20T04:44:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e0ac2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc15d660 0x7f18dc15b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e0ac2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc15d660 0x7f18dc15b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42978/0 (socket says 172.21.15.32:42978) 2023-12-20T04:44:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e0ac2640 1 -- 172.21.15.32:0/3492349456 learned_addr learned my addr 172.21.15.32:0/3492349456 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18db7fe640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18dc157480 0x7f18dc151cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:08.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18dbfff640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0b0760 0x7f18dc0c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e0ac2640 1 -- 172.21.15.32:0/3492349456 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18dc157480 msgr2=0x7f18dc151cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e0ac2640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18dc157480 0x7f18dc151cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e0ac2640 1 -- 172.21.15.32:0/3492349456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0b0760 msgr2=0x7f18dc0c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e0ac2640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0b0760 0x7f18dc0c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e0ac2640 1 -- 172.21.15.32:0/3492349456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18cc0094d0 con 0x7f18dc15d660 2023-12-20T04:44:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e0ac2640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc15d660 0x7f18dc15b800 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f18d400c770 tx=0x7f18d400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:08.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18d8ff9640 1 -- 172.21.15.32:0/3492349456 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d4003b80 con 0x7f18dc15d660 2023-12-20T04:44:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18d8ff9640 1 -- 172.21.15.32:0/3492349456 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18d4003ce0 con 0x7f18dc15d660 2023-12-20T04:44:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18d8ff9640 1 -- 172.21.15.32:0/3492349456 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18d400aa90 con 0x7f18dc15d660 2023-12-20T04:44:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e2525640 1 -- 172.21.15.32:0/3492349456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18dc0b2ec0 con 0x7f18dc15d660 2023-12-20T04:44:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e2525640 1 -- 172.21.15.32:0/3492349456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18dc1521f0 con 0x7f18dc15d660 2023-12-20T04:44:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.161+0000 7f18e2525640 1 -- 172.21.15.32:0/3492349456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f18dc04e000 con 0x7f18dc15d660 2023-12-20T04:44:08.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.165+0000 7f18d8ff9640 1 -- 172.21.15.32:0/3492349456 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 42) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f18d400abf0 con 0x7f18dc15d660 2023-12-20T04:44:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.165+0000 7f18d8ff9640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18c8065900 0x7f18c8067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.165+0000 7f18d8ff9640 1 -- 172.21.15.32:0/3492349456 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6010+0+0 (secure 0 0 0) 0x7f18d4082d10 con 0x7f18dc15d660 2023-12-20T04:44:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.165+0000 7f18dbfff640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18c8065900 0x7f18c8067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:08.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.165+0000 7f18dbfff640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18c8065900 0x7f18c8067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f18cc0af000 tx=0x7f18cc0b6040 comp rx=0 tx=0).ready entity=mgr.7032 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:08.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.169+0000 7f18d8ff9640 1 -- 172.21.15.32:0/3492349456 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f18d4088020 con 0x7f18dc15d660 2023-12-20T04:44:08.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.365+0000 7f18e2525640 1 -- 172.21.15.32:0/3492349456 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f18dc0b3370 con 0x7f18dc15d660 2023-12-20T04:44:08.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.985+0000 7f18d8ff9640 1 -- 172.21.15.32:0/3492349456 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 43) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f18d4050c90 con 0x7f18dc15d660 2023-12-20T04:44:08.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.985+0000 7f18d8ff9640 1 -- 172.21.15.32:0/3492349456 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v43) v1 ==== 93+0+0 (secure 0 0 0) 0x7f18d4052920 con 0x7f18dc15d660 2023-12-20T04:44:08.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 -- 172.21.15.32:0/3492349456 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18c8065900 msgr2=0x7f18c8067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:08.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18c8065900 0x7f18c8067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f18cc0af000 tx=0x7f18cc0b6040 comp rx=0 tx=0).stop 2023-12-20T04:44:08.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 -- 172.21.15.32:0/3492349456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc15d660 msgr2=0x7f18dc15b800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:08.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc15d660 0x7f18dc15b800 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f18d400c770 tx=0x7f18d400cc30 comp rx=0 tx=0).stop 2023-12-20T04:44:08.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 -- 172.21.15.32:0/3492349456 shutdown_connections 2023-12-20T04:44:08.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f18c8065900 0x7f18c8067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:08.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18dc15d660 0x7f18dc15b800 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:08.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18dc157480 0x7f18dc151cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:08.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 --2- 172.21.15.32:0/3492349456 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f18dc0b0760 0x7f18dc0c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:08.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 -- 172.21.15.32:0/3492349456 >> 172.21.15.32:0/3492349456 conn(0x7f18dc06a510 msgr2=0x7f18dc152580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:08.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 -- 172.21.15.32:0/3492349456 shutdown_connections 2023-12-20T04:44:08.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:08.989+0000 7f18e2525640 1 -- 172.21.15.32:0/3492349456 wait complete. 2023-12-20T04:44:09.012 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:44:09.012 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:44:09.138 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:44:09.162 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:09.163+0000 7fac0702d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:44:09.173 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:09.173+0000 7fa956f8e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:44:09.240 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:44:09.241 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:44:09.241 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:44:09.251 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:44:09.252 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:44:09.304 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:44:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.489+0000 7f1d4ada6640 1 Processor -- start 2023-12-20T04:44:09.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d4ada6640 1 -- start start 2023-12-20T04:44:09.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d4ada6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d4415cdb0 0x7f1d440d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:09.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d4ada6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d4415d190 con 0x7f1d440d2740 2023-12-20T04:44:09.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d4ada6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d440d16a0 con 0x7f1d440cfc80 2023-12-20T04:44:09.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d4ada6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d440d1980 con 0x7f1d4415cdb0 2023-12-20T04:44:09.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d43fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1d440d2740 0x7f1d440cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37842/0 (socket says 172.21.15.32:37842) 2023-12-20T04:44:09.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d48b42640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1d4415cdb0 0x7f1d440d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:09.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d43fff640 1 -- 172.21.15.32:0/3389284599 learned_addr learned my addr 172.21.15.32:0/3389284599 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:09.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d48b42640 1 -- 172.21.15.32:0/3389284599 >> v1:172.21.15.39:6789/0 conn(0x7f1d440cfc80 legacy=0x7f1d440b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:09.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d48b42640 1 -- 172.21.15.32:0/3389284599 >> v1:172.21.15.32:6789/0 conn(0x7f1d440d2740 legacy=0x7f1d440cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:09.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d437fe640 1 -- 172.21.15.32:0/3389284599 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3540659089 0 0) 0x7f1d4415d190 con 0x7f1d440d2740 2023-12-20T04:44:09.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.493+0000 7f1d48b42640 1 -- 172.21.15.32:0/3389284599 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d440d1c60 con 0x7f1d4415cdb0 2023-12-20T04:44:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.497+0000 7f1d48b42640 1 --2- 172.21.15.32:0/3389284599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d4415cdb0 0x7f1d440d0be0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1d380097c0 tx=0x7f1d3801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dec92dfbd8b2b3db server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.497+0000 7f1d437fe640 1 -- 172.21.15.32:0/3389284599 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d380a8970 con 0x7f1d4415cdb0 2023-12-20T04:44:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.497+0000 7f1d437fe640 1 -- 172.21.15.32:0/3389284599 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d380a8ad0 con 0x7f1d4415cdb0 2023-12-20T04:44:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.497+0000 7f1d437fe640 1 -- 172.21.15.32:0/3389284599 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d380a8e60 con 0x7f1d4415cdb0 2023-12-20T04:44:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.497+0000 7f1d4ada6640 1 -- 172.21.15.32:0/3389284599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d4415cdb0 msgr2=0x7f1d440d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.497+0000 7f1d4ada6640 1 --2- 172.21.15.32:0/3389284599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d4415cdb0 0x7f1d440d0be0 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f1d380097c0 tx=0x7f1d3801ec70 comp rx=0 tx=0).stop 2023-12-20T04:44:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.497+0000 7f1d4ada6640 1 -- 172.21.15.32:0/3389284599 shutdown_connections 2023-12-20T04:44:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.497+0000 7f1d4ada6640 1 --2- 172.21.15.32:0/3389284599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d4415cdb0 0x7f1d440d0be0 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.497+0000 7f1d4ada6640 1 -- 172.21.15.32:0/3389284599 >> 172.21.15.32:0/3389284599 conn(0x7f1d44065ee0 msgr2=0x7f1d4406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.497+0000 7f1d4ada6640 1 -- 172.21.15.32:0/3389284599 shutdown_connections 2023-12-20T04:44:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.497+0000 7f1d4ada6640 1 -- 172.21.15.32:0/3389284599 wait complete. 2023-12-20T04:44:09.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d4ada6640 1 Processor -- start 2023-12-20T04:44:09.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d4ada6640 1 -- start start 2023-12-20T04:44:09.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d4ada6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d440cfc80 0x7f1d441f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:09.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d4ada6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d440d2740 0x7f1d440cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:09.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d4ada6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d4415cdb0 0x7f1d4414b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:09.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d4ada6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440c9e60 con 0x7f1d440cfc80 2023-12-20T04:44:09.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d4ada6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440cc8f0 con 0x7f1d440d2740 2023-12-20T04:44:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d4ada6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d440cbda0 con 0x7f1d4415cdb0 2023-12-20T04:44:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d48b42640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d440cfc80 0x7f1d441f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d48b42640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d440cfc80 0x7f1d441f38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50520/0 (socket says 172.21.15.32:50520) 2023-12-20T04:44:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d48b42640 1 -- 172.21.15.32:0/271442899 learned_addr learned my addr 172.21.15.32:0/271442899 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d43fff640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d440d2740 0x7f1d440cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d49343640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d4415cdb0 0x7f1d4414b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d48b42640 1 -- 172.21.15.32:0/271442899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d4415cdb0 msgr2=0x7f1d4414b010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d48b42640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d4415cdb0 0x7f1d4414b010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d48b42640 1 -- 172.21.15.32:0/271442899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d440d2740 msgr2=0x7f1d440cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d48b42640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d440d2740 0x7f1d440cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d48b42640 1 -- 172.21.15.32:0/271442899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d38009420 con 0x7f1d440cfc80 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d43fff640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d440d2740 0x7f1d440cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d48b42640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d440cfc80 0x7f1d441f38e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1d380097c0 tx=0x7f1d3801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d417fa640 1 -- 172.21.15.32:0/271442899 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d380b9630 con 0x7f1d440cfc80 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d417fa640 1 -- 172.21.15.32:0/271442899 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d380b97b0 con 0x7f1d440cfc80 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d417fa640 1 -- 172.21.15.32:0/271442899 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d380b9b60 con 0x7f1d440cfc80 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d4ada6640 1 -- 172.21.15.32:0/271442899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d440cc050 con 0x7f1d440cfc80 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.501+0000 7f1d4ada6640 1 -- 172.21.15.32:0/271442899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d440cc610 con 0x7f1d440cfc80 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.505+0000 7f1d4ada6640 1 -- 172.21.15.32:0/271442899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d440c9fa0 con 0x7f1d440cfc80 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.505+0000 7f1d417fa640 1 -- 172.21.15.32:0/271442899 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 43) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f1d380b7040 con 0x7f1d440cfc80 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.505+0000 7f1d417fa640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d28065b80 0x7f1d28068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.505+0000 7f1d417fa640 1 -- 172.21.15.32:0/271442899 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6010+0+0 (secure 0 0 0) 0x7f1d38121460 con 0x7f1d440cfc80 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.505+0000 7f1d43fff640 1 -- 172.21.15.32:0/271442899 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d28065b80 msgr2=0x7f1d28068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:09.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.505+0000 7f1d43fff640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d28065b80 0x7f1d28068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:09.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.509+0000 7f1d417fa640 1 -- 172.21.15.32:0/271442899 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1d440c9fa0 con 0x7f1d440cfc80 2023-12-20T04:44:09.599 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:09.599+0000 7fac0702d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:44:09.619 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:09.621+0000 7fa956f8e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:44:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.673+0000 7f1d4ada6640 1 -- 172.21.15.32:0/271442899 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f1d440c9fa0 con 0x7f1d440cfc80 2023-12-20T04:44:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.673+0000 7f1d417fa640 1 -- 172.21.15.32:0/271442899 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f1d440c9fa0 con 0x7f1d440cfc80 2023-12-20T04:44:09.674 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:09.674 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":94,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:44:06.958760+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:52.582628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:53.871341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2058302965":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/1852394479":"2023-12-21T04:44:06.958717+0000","172.21.15.32:6833/38009":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/2860021829":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/3370166328":"2023-12-21T04:44:06.958717+0000","172.21.15.32:6832/38009":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/2187917072":"2023-12-21T04:44:06.958717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:44:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 -- 172.21.15.32:0/271442899 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d28065b80 msgr2=0x7f1d28068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d28065b80 0x7f1d28068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 -- 172.21.15.32:0/271442899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d440cfc80 msgr2=0x7f1d441f38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d440cfc80 0x7f1d441f38e0 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f1d380097c0 tx=0x7f1d3801ba10 comp rx=0 tx=0).stop 2023-12-20T04:44:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 -- 172.21.15.32:0/271442899 shutdown_connections 2023-12-20T04:44:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1d28065b80 0x7f1d28068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d4415cdb0 0x7f1d4414b010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d440d2740 0x7f1d440cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 --2- 172.21.15.32:0/271442899 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d440cfc80 0x7f1d441f38e0 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 -- 172.21.15.32:0/271442899 >> 172.21.15.32:0/271442899 conn(0x7f1d44065ee0 msgr2=0x7f1d4415ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 -- 172.21.15.32:0/271442899 shutdown_connections 2023-12-20T04:44:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.677+0000 7f1d4ada6640 1 -- 172.21.15.32:0/271442899 wait complete. 2023-12-20T04:44:09.697 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:43:52.582628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:43:53.871341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:44:09.697 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:44:09.755 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:09.755+0000 7fac0702d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:44:09.780 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:09.781+0000 7fa956f8e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:44:09.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.845+0000 7f8d8c42d640 1 Processor -- start 2023-12-20T04:44:09.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.845+0000 7f8d8c42d640 1 -- start start 2023-12-20T04:44:09.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.845+0000 7f8d8c42d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d8415ce50 0x7f8d8406b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:09.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.845+0000 7f8d8c42d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8d840cf7a0 0x7f8d840b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:09.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.845+0000 7f8d8c42d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8d840b0550 con 0x7f8d8406bb80 2023-12-20T04:44:09.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.845+0000 7f8d8c42d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d8406bfc0 con 0x7f8d8415ce50 2023-12-20T04:44:09.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.845+0000 7f8d8c42d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8d840aff20 con 0x7f8d840cf7a0 2023-12-20T04:44:09.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.845+0000 7f8d8a1c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d8415ce50 0x7f8d8406b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:09.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.845+0000 7f8d8a1c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8d8415ce50 0x7f8d8406b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50526/0 (socket says 172.21.15.32:50526) 2023-12-20T04:44:09.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.845+0000 7f8d8a1c9640 1 -- 172.21.15.32:0/2357488767 learned_addr learned my addr 172.21.15.32:0/2357488767 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:09.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.845+0000 7f8d8a9ca640 1 --2- 172.21.15.32:0/2357488767 >> v2:172.21.15.39:3300/0 conn(0x7f8d840cf7a0 0x7f8d840b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:09.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.849+0000 7f8d8a1c9640 1 -- 172.21.15.32:0/2357488767 >> v2:172.21.15.39:3300/0 conn(0x7f8d840cf7a0 msgr2=0x7f8d840b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:09.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.849+0000 7f8d8a1c9640 1 --2- 172.21.15.32:0/2357488767 >> v2:172.21.15.39:3300/0 conn(0x7f8d840cf7a0 0x7f8d840b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.849+0000 7f8d8a1c9640 1 -- 172.21.15.32:0/2357488767 >> v1:172.21.15.39:6789/0 conn(0x7f8d8406bb80 legacy=0x7f8d840cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:09.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.849+0000 7f8d8a1c9640 1 -- 172.21.15.32:0/2357488767 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d840cfdc0 con 0x7f8d8415ce50 2023-12-20T04:44:09.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.849+0000 7f8d8a1c9640 1 --2- 172.21.15.32:0/2357488767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d8415ce50 0x7f8d8406b640 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8d8406cff0 tx=0x7f8d8001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d6b686ef799924 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:09.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.849+0000 7f8d891c7640 1 -- 172.21.15.32:0/2357488767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800ac640 con 0x7f8d8415ce50 2023-12-20T04:44:09.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.849+0000 7f8d891c7640 1 -- 172.21.15.32:0/2357488767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d800ac7a0 con 0x7f8d8415ce50 2023-12-20T04:44:09.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.849+0000 7f8d891c7640 1 -- 172.21.15.32:0/2357488767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800acb30 con 0x7f8d8415ce50 2023-12-20T04:44:09.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.849+0000 7f8d8c42d640 1 -- 172.21.15.32:0/2357488767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d8415ce50 msgr2=0x7f8d8406b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:09.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.849+0000 7f8d8c42d640 1 --2- 172.21.15.32:0/2357488767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d8415ce50 0x7f8d8406b640 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f8d8406cff0 tx=0x7f8d8001c4d0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 -- 172.21.15.32:0/2357488767 shutdown_connections 2023-12-20T04:44:09.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 --2- 172.21.15.32:0/2357488767 >> v2:172.21.15.39:3300/0 conn(0x7f8d840cf7a0 0x7f8d840b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 --2- 172.21.15.32:0/2357488767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d8415ce50 0x7f8d8406b640 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 -- 172.21.15.32:0/2357488767 >> 172.21.15.32:0/2357488767 conn(0x7f8d84099d70 msgr2=0x7f8d840d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:09.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 -- 172.21.15.32:0/2357488767 shutdown_connections 2023-12-20T04:44:09.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 -- 172.21.15.32:0/2357488767 wait complete. 2023-12-20T04:44:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 Processor -- start 2023-12-20T04:44:09.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 -- start start 2023-12-20T04:44:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d8406bb80 0x7f8d841528d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d840cf7a0 0x7f8d841582d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8d8415ce50 0x7f8d840c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d840b3d90 con 0x7f8d840cf7a0 2023-12-20T04:44:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8d84157e40 con 0x7f8d8406bb80 2023-12-20T04:44:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8c42d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8d84152e10 con 0x7f8d8415ce50 2023-12-20T04:44:09.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8a1c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d8406bb80 0x7f8d841528d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d899c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d840cf7a0 0x7f8d841582d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8a1c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d8406bb80 0x7f8d841528d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:52862/0 (socket says 172.21.15.32:52862) 2023-12-20T04:44:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8a1c9640 1 -- 172.21.15.32:0/56106853 learned_addr learned my addr 172.21.15.32:0/56106853 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:09.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8a9ca640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8d8415ce50 0x7f8d840c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8a1c9640 1 -- 172.21.15.32:0/56106853 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8d8415ce50 msgr2=0x7f8d840c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8a1c9640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8d8415ce50 0x7f8d840c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8a1c9640 1 -- 172.21.15.32:0/56106853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d840cf7a0 msgr2=0x7f8d841582d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8a1c9640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d840cf7a0 0x7f8d841582d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.853+0000 7f8d8a1c9640 1 -- 172.21.15.32:0/56106853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d800094d0 con 0x7f8d8406bb80 2023-12-20T04:44:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d8a1c9640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d8406bb80 0x7f8d841528d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8d8001a100 tx=0x7f8d8001ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d7affd640 1 -- 172.21.15.32:0/56106853 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800b7650 con 0x7f8d8406bb80 2023-12-20T04:44:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d7affd640 1 -- 172.21.15.32:0/56106853 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d800b77b0 con 0x7f8d8406bb80 2023-12-20T04:44:09.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d8c42d640 1 -- 172.21.15.32:0/56106853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8d84158810 con 0x7f8d8406bb80 2023-12-20T04:44:09.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d7affd640 1 -- 172.21.15.32:0/56106853 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d800b7b20 con 0x7f8d8406bb80 2023-12-20T04:44:09.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d8c42d640 1 -- 172.21.15.32:0/56106853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8d841577f0 con 0x7f8d8406bb80 2023-12-20T04:44:09.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d8c42d640 1 -- 172.21.15.32:0/56106853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8d8404e000 con 0x7f8d8406bb80 2023-12-20T04:44:09.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d7affd640 1 -- 172.21.15.32:0/56106853 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 43) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f8d800b3070 con 0x7f8d8406bb80 2023-12-20T04:44:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d7affd640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d68065b80 0x7f8d68068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d7affd640 1 -- 172.21.15.32:0/56106853 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6010+0+0 (secure 0 0 0) 0x7f8d8011dd10 con 0x7f8d8406bb80 2023-12-20T04:44:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d899c8640 1 -- 172.21.15.32:0/56106853 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d68065b80 msgr2=0x7f8d68068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:09.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.857+0000 7f8d899c8640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d68065b80 0x7f8d68068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:09.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:09.861+0000 7f8d7affd640 1 -- 172.21.15.32:0/56106853 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8d800eb410 con 0x7f8d8406bb80 2023-12-20T04:44:10.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 -- 172.21.15.32:0/56106853 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f8d840ce120 con 0x7f8d8406bb80 2023-12-20T04:44:10.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d7affd640 1 -- 172.21.15.32:0/56106853 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7f8d800b10a0 con 0x7f8d8406bb80 2023-12-20T04:44:10.024 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:44:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 -- 172.21.15.32:0/56106853 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d68065b80 msgr2=0x7f8d68068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d68065b80 0x7f8d68068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 -- 172.21.15.32:0/56106853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d8406bb80 msgr2=0x7f8d841528d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d8406bb80 0x7f8d841528d0 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f8d8001a100 tx=0x7f8d8001ad10 comp rx=0 tx=0).stop 2023-12-20T04:44:10.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 -- 172.21.15.32:0/56106853 shutdown_connections 2023-12-20T04:44:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d68065b80 0x7f8d68068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8d8415ce50 0x7f8d840c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8d840cf7a0 0x7f8d841582d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 --2- 172.21.15.32:0/56106853 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8d8406bb80 0x7f8d841528d0 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 -- 172.21.15.32:0/56106853 >> 172.21.15.32:0/56106853 conn(0x7f8d84099d70 msgr2=0x7f8d8406e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 -- 172.21.15.32:0/56106853 shutdown_connections 2023-12-20T04:44:10.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.025+0000 7f8d8c42d640 1 -- 172.21.15.32:0/56106853 wait complete. 2023-12-20T04:44:10.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:44:10.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.193+0000 7f749d0f9640 1 Processor -- start 2023-12-20T04:44:10.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.193+0000 7f749d0f9640 1 -- start start 2023-12-20T04:44:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.193+0000 7f749d0f9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f749815ce60 0x7f74980d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.193+0000 7f749d0f9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f74980d3130 0x7f74980d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f749d0f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74980d2650 con 0x7f74980d1300 2023-12-20T04:44:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f749d0f9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74980d35a0 con 0x7f749815ce60 2023-12-20T04:44:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f749d0f9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74980d1a40 con 0x7f74980d3130 2023-12-20T04:44:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f7496d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f749815ce60 0x7f74980d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f749759e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f74980d1300 0x7f74980b46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37854/0 (socket says 172.21.15.32:37854) 2023-12-20T04:44:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f749759e640 1 -- 172.21.15.32:0/2921161778 learned_addr learned my addr 172.21.15.32:0/2921161778 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f749659c640 1 --2- 172.21.15.32:0/2921161778 >> v2:172.21.15.39:3300/0 conn(0x7f74980d3130 0x7f74980d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f7496d9d640 1 -- 172.21.15.32:0/2921161778 >> v2:172.21.15.39:3300/0 conn(0x7f74980d3130 msgr2=0x7f74980d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f7496d9d640 1 --2- 172.21.15.32:0/2921161778 >> v2:172.21.15.39:3300/0 conn(0x7f74980d3130 0x7f74980d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f7496d9d640 1 -- 172.21.15.32:0/2921161778 >> v1:172.21.15.32:6789/0 conn(0x7f74980d1300 legacy=0x7f74980b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f7496d9d640 1 -- 172.21.15.32:0/2921161778 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74980d1d20 con 0x7f749815ce60 2023-12-20T04:44:10.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f7495d9b640 1 -- 172.21.15.32:0/2921161778 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3340877562 0 0) 0x7f74980d2650 con 0x7f74980d1300 2023-12-20T04:44:10.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.197+0000 7f7496d9d640 1 --2- 172.21.15.32:0/2921161778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749815ce60 0x7f74980d2bf0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f74880097c0 tx=0x7f748801ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7884552e8e0e027 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f7495d9b640 1 -- 172.21.15.32:0/2921161778 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74880a8930 con 0x7f749815ce60 2023-12-20T04:44:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f7495d9b640 1 -- 172.21.15.32:0/2921161778 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74880a8a90 con 0x7f749815ce60 2023-12-20T04:44:10.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f7495d9b640 1 -- 172.21.15.32:0/2921161778 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74880a8e20 con 0x7f749815ce60 2023-12-20T04:44:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 -- 172.21.15.32:0/2921161778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749815ce60 msgr2=0x7f74980d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 --2- 172.21.15.32:0/2921161778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749815ce60 0x7f74980d2bf0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f74880097c0 tx=0x7f748801ec30 comp rx=0 tx=0).stop 2023-12-20T04:44:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 -- 172.21.15.32:0/2921161778 shutdown_connections 2023-12-20T04:44:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 --2- 172.21.15.32:0/2921161778 >> v2:172.21.15.39:3300/0 conn(0x7f74980d3130 0x7f74980d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 --2- 172.21.15.32:0/2921161778 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749815ce60 0x7f74980d2bf0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 -- 172.21.15.32:0/2921161778 >> 172.21.15.32:0/2921161778 conn(0x7f749806b5e0 msgr2=0x7f749806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 -- 172.21.15.32:0/2921161778 shutdown_connections 2023-12-20T04:44:10.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 -- 172.21.15.32:0/2921161778 wait complete. 2023-12-20T04:44:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 Processor -- start 2023-12-20T04:44:10.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 -- start start 2023-12-20T04:44:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74980d1300 0x7f7498154a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74980d3130 0x7f74981f1f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.201+0000 7f749d0f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749815ce60 0x7f7498158840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749d0f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980cbba0 con 0x7f749815ce60 2023-12-20T04:44:10.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749d0f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74980caf90 con 0x7f74980d1300 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749d0f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74980cb200 con 0x7f74980d3130 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749815ce60 0x7f7498158840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f7496d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74980d1300 0x7f7498154a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749815ce60 0x7f7498158840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50566/0 (socket says 172.21.15.32:50566) 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749759e640 1 -- 172.21.15.32:0/4018009797 learned_addr learned my addr 172.21.15.32:0/4018009797 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749659c640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74980d3130 0x7f74981f1f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749759e640 1 -- 172.21.15.32:0/4018009797 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74980d3130 msgr2=0x7f74981f1f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749759e640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74980d3130 0x7f74981f1f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749759e640 1 -- 172.21.15.32:0/4018009797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74980d1300 msgr2=0x7f7498154a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749759e640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74980d1300 0x7f7498154a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749759e640 1 -- 172.21.15.32:0/4018009797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7488009420 con 0x7f749815ce60 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f7496d9d640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74980d1300 0x7f7498154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749759e640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749815ce60 0x7f7498158840 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f749000c7b0 tx=0x7f749000cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:10.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f747f7fe640 1 -- 172.21.15.32:0/4018009797 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749000ce60 con 0x7f749815ce60 2023-12-20T04:44:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f747f7fe640 1 -- 172.21.15.32:0/4018009797 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f749000a640 con 0x7f749815ce60 2023-12-20T04:44:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749d0f9640 1 -- 172.21.15.32:0/4018009797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74980cb4e0 con 0x7f749815ce60 2023-12-20T04:44:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f747f7fe640 1 -- 172.21.15.32:0/4018009797 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f749000a9f0 con 0x7f749815ce60 2023-12-20T04:44:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749d0f9640 1 -- 172.21.15.32:0/4018009797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7498154fd0 con 0x7f749815ce60 2023-12-20T04:44:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f749d0f9640 1 -- 172.21.15.32:0/4018009797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f749804e000 con 0x7f749815ce60 2023-12-20T04:44:10.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.205+0000 7f747f7fe640 1 -- 172.21.15.32:0/4018009797 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 43) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f749000ab50 con 0x7f749815ce60 2023-12-20T04:44:10.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.209+0000 7f747f7fe640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7484065b80 0x7f7484068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.209+0000 7f747f7fe640 1 -- 172.21.15.32:0/4018009797 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6010+0+0 (secure 0 0 0) 0x7f7490082b80 con 0x7f749815ce60 2023-12-20T04:44:10.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.209+0000 7f7496d9d640 1 -- 172.21.15.32:0/4018009797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7484065b80 msgr2=0x7f7484068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:10.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.209+0000 7f7496d9d640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7484065b80 0x7f7484068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:10.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.213+0000 7f747f7fe640 1 -- 172.21.15.32:0/4018009797 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7490050440 con 0x7f749815ce60 2023-12-20T04:44:10.263 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:10.263+0000 7fac0702d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:10.303 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:10.301+0000 7fa956f8e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:10.325 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:10.323+0000 7fac0702d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:44:10.365 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:10.365+0000 7fa956f8e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:44:10.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.369+0000 7f749d0f9640 1 -- 172.21.15.32:0/4018009797 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f74980b5210 con 0x7f749815ce60 2023-12-20T04:44:10.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.369+0000 7f747f7fe640 1 -- 172.21.15.32:0/4018009797 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7f7490001770 con 0x7f749815ce60 2023-12-20T04:44:10.370 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:44:10.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 -- 172.21.15.32:0/4018009797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7484065b80 msgr2=0x7f7484068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:10.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7484065b80 0x7f7484068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 -- 172.21.15.32:0/4018009797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749815ce60 msgr2=0x7f7498158840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749815ce60 0x7f7498158840 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f749000c7b0 tx=0x7f749000cc70 comp rx=0 tx=0).stop 2023-12-20T04:44:10.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 -- 172.21.15.32:0/4018009797 shutdown_connections 2023-12-20T04:44:10.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7484065b80 0x7f7484068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f749815ce60 0x7f7498158840 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74980d3130 0x7f74981f1f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 --2- 172.21.15.32:0/4018009797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74980d1300 0x7f7498154a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 -- 172.21.15.32:0/4018009797 >> 172.21.15.32:0/4018009797 conn(0x7f749806b5e0 msgr2=0x7f74980b43c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:10.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 -- 172.21.15.32:0/4018009797 shutdown_connections 2023-12-20T04:44:10.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.373+0000 7f749d0f9640 1 -- 172.21.15.32:0/4018009797 wait complete. 2023-12-20T04:44:10.383 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:10.383+0000 7fac0702d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:44:10.392 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:44:10.427 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:10.429+0000 7fa956f8e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:44:10.474 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:10.475+0000 7fac0702d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:44:10.522 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:10.521+0000 7fa956f8e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:44:10.535 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:10.535+0000 7fac0702d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:44:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.545+0000 7f286d4df640 1 Processor -- start 2023-12-20T04:44:10.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.545+0000 7f286d4df640 1 -- start start 2023-12-20T04:44:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.545+0000 7f286d4df640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f286815ce50 0x7f286806b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.545+0000 7f286d4df640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28680cf7a0 0x7f28680b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.545+0000 7f286d4df640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28680b0550 con 0x7f286806bb80 2023-12-20T04:44:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.545+0000 7f286d4df640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286806bfc0 con 0x7f286815ce50 2023-12-20T04:44:10.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.545+0000 7f286d4df640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28680aff20 con 0x7f28680cf7a0 2023-12-20T04:44:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f2866ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f286815ce50 0x7f286806b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f28677fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28680cf7a0 0x7f28680b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f285ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f286806bb80 0x7f28680cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:51770/0 (socket says 172.21.15.32:51770) 2023-12-20T04:44:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f2866ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f286815ce50 0x7f286806b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:52884/0 (socket says 172.21.15.32:52884) 2023-12-20T04:44:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f285ffff640 1 -- 172.21.15.32:0/1798908450 learned_addr learned my addr 172.21.15.32:0/1798908450 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f28677fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28680cf7a0 0x7f28680b09e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:54828/0 (socket says 172.21.15.32:54828) 2023-12-20T04:44:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f2866ffd640 1 -- 172.21.15.32:0/1798908450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28680cf7a0 msgr2=0x7f28680b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f2866ffd640 1 --2- 172.21.15.32:0/1798908450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28680cf7a0 0x7f28680b09e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f2866ffd640 1 -- 172.21.15.32:0/1798908450 >> v1:172.21.15.39:6789/0 conn(0x7f286806bb80 legacy=0x7f28680cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f2866ffd640 1 -- 172.21.15.32:0/1798908450 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28680cfdc0 con 0x7f286815ce50 2023-12-20T04:44:10.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f28677fe640 1 --2- 172.21.15.32:0/1798908450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28680cf7a0 0x7f28680b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:10.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f28667fc640 1 -- 172.21.15.32:0/1798908450 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1347425740 0 0) 0x7f28680b0550 con 0x7f286806bb80 2023-12-20T04:44:10.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f2866ffd640 1 --2- 172.21.15.32:0/1798908450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f286815ce50 0x7f286806b640 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f28580097c0 tx=0x7f285801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9527f8fd91fa1651 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:10.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f28667fc640 1 -- 172.21.15.32:0/1798908450 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28580ad660 con 0x7f286815ce50 2023-12-20T04:44:10.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f28667fc640 1 -- 172.21.15.32:0/1798908450 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28580ad7c0 con 0x7f286815ce50 2023-12-20T04:44:10.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.549+0000 7f28667fc640 1 -- 172.21.15.32:0/1798908450 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28580adb50 con 0x7f286815ce50 2023-12-20T04:44:10.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 -- 172.21.15.32:0/1798908450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f286815ce50 msgr2=0x7f286806b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 --2- 172.21.15.32:0/1798908450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f286815ce50 0x7f286806b640 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f28580097c0 tx=0x7f285801c450 comp rx=0 tx=0).stop 2023-12-20T04:44:10.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 -- 172.21.15.32:0/1798908450 shutdown_connections 2023-12-20T04:44:10.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 --2- 172.21.15.32:0/1798908450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28680cf7a0 0x7f28680b09e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 --2- 172.21.15.32:0/1798908450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f286815ce50 0x7f286806b640 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 -- 172.21.15.32:0/1798908450 >> 172.21.15.32:0/1798908450 conn(0x7f2868099d70 msgr2=0x7f28680d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:10.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 -- 172.21.15.32:0/1798908450 shutdown_connections 2023-12-20T04:44:10.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 -- 172.21.15.32:0/1798908450 wait complete. 2023-12-20T04:44:10.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 Processor -- start 2023-12-20T04:44:10.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 -- start start 2023-12-20T04:44:10.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f286806bb80 0x7f286815a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cf7a0 0x7f286814ebc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f286815ce50 0x7f28680d0aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28680cb370 con 0x7f28680cf7a0 2023-12-20T04:44:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28680ca3d0 con 0x7f286815ce50 2023-12-20T04:44:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f286d4df640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286815aad0 con 0x7f286806bb80 2023-12-20T04:44:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f285ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cf7a0 0x7f286814ebc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f2866ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f286806bb80 0x7f286815a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f285ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cf7a0 0x7f286814ebc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50580/0 (socket says 172.21.15.32:50580) 2023-12-20T04:44:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f285ffff640 1 -- 172.21.15.32:0/1492075626 learned_addr learned my addr 172.21.15.32:0/1492075626 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:10.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f28677fe640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f286815ce50 0x7f28680d0aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f285ffff640 1 -- 172.21.15.32:0/1492075626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f286806bb80 msgr2=0x7f286815a3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f285ffff640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f286806bb80 0x7f286815a3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f285ffff640 1 -- 172.21.15.32:0/1492075626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f286815ce50 msgr2=0x7f28680d0aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.553+0000 7f285ffff640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f286815ce50 0x7f28680d0aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f285ffff640 1 -- 172.21.15.32:0/1492075626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2860009a70 con 0x7f28680cf7a0 2023-12-20T04:44:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f2866ffd640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f286806bb80 0x7f286815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:10.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f285ffff640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cf7a0 0x7f286814ebc0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f28500011f0 tx=0x7f285000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f285f7fe640 1 -- 172.21.15.32:0/1492075626 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28500074e0 con 0x7f28680cf7a0 2023-12-20T04:44:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f285f7fe640 1 -- 172.21.15.32:0/1492075626 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2850007640 con 0x7f28680cf7a0 2023-12-20T04:44:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f285f7fe640 1 -- 172.21.15.32:0/1492075626 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28500049f0 con 0x7f28680cf7a0 2023-12-20T04:44:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f286d4df640 1 -- 172.21.15.32:0/1492075626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2858009420 con 0x7f28680cf7a0 2023-12-20T04:44:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f286d4df640 1 -- 172.21.15.32:0/1492075626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f286815b090 con 0x7f28680cf7a0 2023-12-20T04:44:10.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f286d4df640 1 -- 172.21.15.32:0/1492075626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28680cb4e0 con 0x7f28680cf7a0 2023-12-20T04:44:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f285f7fe640 1 -- 172.21.15.32:0/1492075626 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 43) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f2850004b50 con 0x7f28680cf7a0 2023-12-20T04:44:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f285f7fe640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2848065b80 0x7f2848068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f285f7fe640 1 -- 172.21.15.32:0/1492075626 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6010+0+0 (secure 0 0 0) 0x7f28500831e0 con 0x7f28680cf7a0 2023-12-20T04:44:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f2866ffd640 1 -- 172.21.15.32:0/1492075626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2848065b80 msgr2=0x7f2848068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:10.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.557+0000 7f2866ffd640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2848065b80 0x7f2848068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:10.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.561+0000 7f285f7fe640 1 -- 172.21.15.32:0/1492075626 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f28680cb4e0 con 0x7f28680cf7a0 2023-12-20T04:44:10.586 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:10.585+0000 7fa956f8e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:44:10.624 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:10.623+0000 7fac0702d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:44:10.676 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:10.677+0000 7fa956f8e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:44:10.683 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:10.683+0000 7fac0702d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:44:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.725+0000 7f286d4df640 1 -- 172.21.15.32:0/1492075626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f28680cb4e0 con 0x7f28680cf7a0 2023-12-20T04:44:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.725+0000 7f285f7fe640 1 -- 172.21.15.32:0/1492075626 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) v1 ==== 100+0+11 (secure 0 0 0) 0x7f28680cb4e0 con 0x7f28680cf7a0 2023-12-20T04:44:10.726 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:44:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 -- 172.21.15.32:0/1492075626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2848065b80 msgr2=0x7f2848068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2848065b80 0x7f2848068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 -- 172.21.15.32:0/1492075626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cf7a0 msgr2=0x7f286814ebc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cf7a0 0x7f286814ebc0 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f28500011f0 tx=0x7f285000c8f0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 -- 172.21.15.32:0/1492075626 shutdown_connections 2023-12-20T04:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2848065b80 0x7f2848068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f286815ce50 0x7f28680d0aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cf7a0 0x7f286814ebc0 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 --2- 172.21.15.32:0/1492075626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f286806bb80 0x7f286815a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 -- 172.21.15.32:0/1492075626 >> 172.21.15.32:0/1492075626 conn(0x7f2868099d70 msgr2=0x7f2868158f70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 -- 172.21.15.32:0/1492075626 shutdown_connections 2023-12-20T04:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.729+0000 7f286d4df640 1 -- 172.21.15.32:0/1492075626 wait complete. 2023-12-20T04:44:10.740 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:10.741+0000 7fa956f8e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:44:10.757 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T04:44:10.833 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:10.831+0000 7fac0702d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:44:10.896 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:10.895+0000 7fac0702d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:44:10.901 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:10.901+0000 7fa956f8e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:44:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3eb5604640 1 Processor -- start 2023-12-20T04:44:10.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3eb5604640 1 -- start start 2023-12-20T04:44:10.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3eb5604640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3eb015cea0 0x7f3eb00cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3eb5604640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3eb00b84d0 0x7f3eb00cdca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3eb5604640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3eb00d0c90 con 0x7f3eb015cac0 2023-12-20T04:44:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3eb5604640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3eb0160f10 con 0x7f3eb015cea0 2023-12-20T04:44:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3eb5604640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb0161110 con 0x7f3eb00b84d0 2023-12-20T04:44:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3ea7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3eb015cea0 0x7f3eb00cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3ea7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3eb015cea0 0x7f3eb00cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50596/0 (socket says 172.21.15.32:50596) 2023-12-20T04:44:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3ea7fff640 1 -- 172.21.15.32:0/4228678468 learned_addr learned my addr 172.21.15.32:0/4228678468 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3eaf7fe640 1 --2- 172.21.15.32:0/4228678468 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3eb00b84d0 0x7f3eb00cdca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3ea7fff640 1 -- 172.21.15.32:0/4228678468 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3eb00b84d0 msgr2=0x7f3eb00cdca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3ea7fff640 1 --2- 172.21.15.32:0/4228678468 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3eb00b84d0 0x7f3eb00cdca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3ea7fff640 1 -- 172.21.15.32:0/4228678468 >> v1:172.21.15.39:6789/0 conn(0x7f3eb015cac0 legacy=0x7f3eb0160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.949+0000 7f3ea7fff640 1 -- 172.21.15.32:0/4228678468 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3eb01613f0 con 0x7f3eb015cea0 2023-12-20T04:44:10.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3ea7fff640 1 --2- 172.21.15.32:0/4228678468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eb015cea0 0x7f3eb00cd760 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009870 tx=0x7f3e9801cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7e3eca4c9fea71b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:10.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3eae7fc640 1 -- 172.21.15.32:0/4228678468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e980a95f0 con 0x7f3eb015cea0 2023-12-20T04:44:10.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3eae7fc640 1 -- 172.21.15.32:0/4228678468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e980a9750 con 0x7f3eb015cea0 2023-12-20T04:44:10.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3eae7fc640 1 -- 172.21.15.32:0/4228678468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e980a9ae0 con 0x7f3eb015cea0 2023-12-20T04:44:10.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3eb5604640 1 -- 172.21.15.32:0/4228678468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eb015cea0 msgr2=0x7f3eb00cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3eb5604640 1 --2- 172.21.15.32:0/4228678468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eb015cea0 0x7f3eb00cd760 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f3e98009870 tx=0x7f3e9801cf70 comp rx=0 tx=0).stop 2023-12-20T04:44:10.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3eb5604640 1 -- 172.21.15.32:0/4228678468 shutdown_connections 2023-12-20T04:44:10.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3eb5604640 1 --2- 172.21.15.32:0/4228678468 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3eb00b84d0 0x7f3eb00cdca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3eb5604640 1 --2- 172.21.15.32:0/4228678468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eb015cea0 0x7f3eb00cd760 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3eb5604640 1 -- 172.21.15.32:0/4228678468 >> 172.21.15.32:0/4228678468 conn(0x7f3eb006afd0 msgr2=0x7f3eb006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:10.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3eb5604640 1 -- 172.21.15.32:0/4228678468 shutdown_connections 2023-12-20T04:44:10.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.953+0000 7f3eb5604640 1 -- 172.21.15.32:0/4228678468 wait complete. 2023-12-20T04:44:10.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eb5604640 1 Processor -- start 2023-12-20T04:44:10.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eb5604640 1 -- start start 2023-12-20T04:44:10.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eb5604640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eb00b84d0 0x7f3eb0154820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eb5604640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eb015cac0 0x7f3eb00c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eb5604640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3eb015cea0 0x7f3eb00c6c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eb5604640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb014d5f0 con 0x7f3eb00b84d0 2023-12-20T04:44:10.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eb5604640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb014db70 con 0x7f3eb015cac0 2023-12-20T04:44:10.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eb5604640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3eb014f0e0 con 0x7f3eb015cea0 2023-12-20T04:44:10.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eaeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eb00b84d0 0x7f3eb0154820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eaeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eb00b84d0 0x7f3eb0154820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50612/0 (socket says 172.21.15.32:50612) 2023-12-20T04:44:10.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eaeffd640 1 -- 172.21.15.32:0/488545940 learned_addr learned my addr 172.21.15.32:0/488545940 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:10.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3ea7fff640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eb015cac0 0x7f3eb00c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eaf7fe640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3eb015cea0 0x7f3eb00c6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:10.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eaeffd640 1 -- 172.21.15.32:0/488545940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3eb015cea0 msgr2=0x7f3eb00c6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eaeffd640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3eb015cea0 0x7f3eb00c6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eaeffd640 1 -- 172.21.15.32:0/488545940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eb015cac0 msgr2=0x7f3eb00c6740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:10.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eaeffd640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eb015cac0 0x7f3eb00c6740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:10.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eaeffd640 1 -- 172.21.15.32:0/488545940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e980094d0 con 0x7f3eb00b84d0 2023-12-20T04:44:10.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eaeffd640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eb00b84d0 0x7f3eb0154820 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000c420 tx=0x7f3ea000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:10.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3ea77fe640 1 -- 172.21.15.32:0/488545940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea00052d0 con 0x7f3eb00b84d0 2023-12-20T04:44:10.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3ea77fe640 1 -- 172.21.15.32:0/488545940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ea0005430 con 0x7f3eb00b84d0 2023-12-20T04:44:10.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3ea77fe640 1 -- 172.21.15.32:0/488545940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0007260 con 0x7f3eb00b84d0 2023-12-20T04:44:10.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eb5604640 1 -- 172.21.15.32:0/488545940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3eb014d890 con 0x7f3eb00b84d0 2023-12-20T04:44:10.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.957+0000 7f3eb5604640 1 -- 172.21.15.32:0/488545940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3eb014ebc0 con 0x7f3eb00b84d0 2023-12-20T04:44:10.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.961+0000 7f3eb5604640 1 -- 172.21.15.32:0/488545940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3eb014edb0 con 0x7f3eb00b84d0 2023-12-20T04:44:10.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.961+0000 7f3ea77fe640 1 -- 172.21.15.32:0/488545940 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 43) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f3ea00073c0 con 0x7f3eb00b84d0 2023-12-20T04:44:10.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.961+0000 7f3ea77fe640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e90065b80 0x7f3e90068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:10.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.961+0000 7f3ea77fe640 1 -- 172.21.15.32:0/488545940 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6010+0+0 (secure 0 0 0) 0x7f3ea00827a0 con 0x7f3eb00b84d0 2023-12-20T04:44:10.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.961+0000 7f3ea7fff640 1 -- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e90065b80 msgr2=0x7f3e90068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:10.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.961+0000 7f3ea7fff640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e90065b80 0x7f3e90068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:10.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:10.965+0000 7f3ea77fe640 1 -- 172.21.15.32:0/488545940 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3ea004ffc0 con 0x7f3eb00b84d0 2023-12-20T04:44:10.967 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:10.969+0000 7fa956f8e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:44:11.149 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:44:11.149 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:44:11.149 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:44:11.158 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:11.159+0000 7fac0702d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:44:11.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.165+0000 7f3ea7fff640 1 -- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e90065b80 msgr2=0x7f3e90068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:11.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.165+0000 7f3ea7fff640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e90065b80 0x7f3e90068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:44:11.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.165+0000 7f3eb5604640 1 -- 172.21.15.32:0/488545940 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f3eb0074100 con 0x7f3eb00b84d0 2023-12-20T04:44:11.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.169+0000 7f3ea77fe640 1 -- 172.21.15.32:0/488545940 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 7 entries v94) v1 ==== 80+0+402 (secure 0 0 0) 0x7f3ea00535e0 con 0x7f3eb00b84d0 2023-12-20T04:44:11.170 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2058302965 2023-12-21T04:44:06.958717+0000 2023-12-20T04:44:11.170 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1852394479 2023-12-21T04:44:06.958717+0000 2023-12-20T04:44:11.170 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/38009 2023-12-21T04:44:06.958717+0000 2023-12-20T04:44:11.170 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2860021829 2023-12-21T04:44:06.958717+0000 2023-12-20T04:44:11.170 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3370166328 2023-12-21T04:44:06.958717+0000 2023-12-20T04:44:11.170 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/38009 2023-12-21T04:44:06.958717+0000 2023-12-20T04:44:11.170 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2187917072 2023-12-21T04:44:06.958717+0000 2023-12-20T04:44:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:listed 7 entries 2023-12-20T04:44:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.169+0000 7f3eb5604640 1 -- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e90065b80 msgr2=0x7f3e90068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.169+0000 7f3eb5604640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e90065b80 0x7f3e90068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.169+0000 7f3eb5604640 1 -- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eb00b84d0 msgr2=0x7f3eb0154820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.169+0000 7f3eb5604640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eb00b84d0 0x7f3eb0154820 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000c420 tx=0x7f3ea000c8e0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.169+0000 7f3eb5604640 1 -- 172.21.15.32:0/488545940 shutdown_connections 2023-12-20T04:44:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.169+0000 7f3eb5604640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e90065b80 0x7f3e90068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.169+0000 7f3eb5604640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3eb015cea0 0x7f3eb00c6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.169+0000 7f3eb5604640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3eb015cac0 0x7f3eb00c6740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.169+0000 7f3eb5604640 1 --2- 172.21.15.32:0/488545940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3eb00b84d0 0x7f3eb0154820 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.169+0000 7f3eb5604640 1 -- 172.21.15.32:0/488545940 >> 172.21.15.32:0/488545940 conn(0x7f3eb006afd0 msgr2=0x7f3eb0156bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.173+0000 7f3eb5604640 1 -- 172.21.15.32:0/488545940 shutdown_connections 2023-12-20T04:44:11.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.173+0000 7f3eb5604640 1 -- 172.21.15.32:0/488545940 wait complete. 2023-12-20T04:44:11.198 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T04:44:11.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:11.198 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T04:44:11.220 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:44:11.220 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:44:11.220 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:44:11.227 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:44:11.227 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:11.228 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T04:44:11.230 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:11.229+0000 7fa956f8e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:44:11.239 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:44:11.239 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:44:11.249 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:44:11.249 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:44:11.249 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:44:11.291 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:44:11.292 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:44:11.292 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:44:11.301 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:44:11.301 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:44:11.302 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:11.299+0000 7fac0702d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:11.374 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:11.373+0000 7fa956f8e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:11.431 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:11.431+0000 7fac0702d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:11.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.477+0000 7f66bbaf9640 1 Processor -- start 2023-12-20T04:44:11.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.477+0000 7f66bbaf9640 1 -- start start 2023-12-20T04:44:11.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66bbaf9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b40d2740 0x7f66b40cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66bbaf9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66b40cfab0 0x7f66b40b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66bbaf9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66b415d190 con 0x7f66b415cdb0 2023-12-20T04:44:11.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66bbaf9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66b40cff40 con 0x7f66b40cfab0 2023-12-20T04:44:11.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66bbaf9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40d16a0 con 0x7f66b40d2740 2023-12-20T04:44:11.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66b9895640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f66b415cdb0 0x7f66b40d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37868/0 (socket says 172.21.15.32:37868) 2023-12-20T04:44:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66b9895640 1 -- 172.21.15.32:0/248150503 learned_addr learned my addr 172.21.15.32:0/248150503 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66ba096640 1 --2- 172.21.15.32:0/248150503 >> v2:172.21.15.32:3300/0 conn(0x7f66b40cfab0 0x7f66b40b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66b9094640 1 --2- 172.21.15.32:0/248150503 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b40d2740 0x7f66b40cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66b8893640 1 -- 172.21.15.32:0/248150503 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3557032013 0 0) 0x7f66b415d190 con 0x7f66b415cdb0 2023-12-20T04:44:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66b8893640 1 -- 172.21.15.32:0/248150503 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f66a0003530 con 0x7f66b415cdb0 2023-12-20T04:44:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66ba096640 1 -- 172.21.15.32:0/248150503 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b40d2740 msgr2=0x7f66b40cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66ba096640 1 --2- 172.21.15.32:0/248150503 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b40d2740 0x7f66b40cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66ba096640 1 -- 172.21.15.32:0/248150503 >> v1:172.21.15.32:6789/0 conn(0x7f66b415cdb0 legacy=0x7f66b40d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:11.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.481+0000 7f66ba096640 1 -- 172.21.15.32:0/248150503 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66b40d1980 con 0x7f66b40cfab0 2023-12-20T04:44:11.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66ba096640 1 --2- 172.21.15.32:0/248150503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b40cfab0 0x7f66b40b5c90 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009d90 tx=0x7f66ac01f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dab0d4e7b425a75 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66b8893640 1 -- 172.21.15.32:0/248150503 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0af600 con 0x7f66b40cfab0 2023-12-20T04:44:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66b8893640 1 -- 172.21.15.32:0/248150503 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66ac0ab050 con 0x7f66b40cfab0 2023-12-20T04:44:11.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66b8893640 1 -- 172.21.15.32:0/248150503 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66ac0af960 con 0x7f66b40cfab0 2023-12-20T04:44:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66bbaf9640 1 -- 172.21.15.32:0/248150503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b40cfab0 msgr2=0x7f66b40b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66bbaf9640 1 --2- 172.21.15.32:0/248150503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b40cfab0 0x7f66b40b5c90 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f66ac009d90 tx=0x7f66ac01f130 comp rx=0 tx=0).stop 2023-12-20T04:44:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66bbaf9640 1 -- 172.21.15.32:0/248150503 shutdown_connections 2023-12-20T04:44:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66bbaf9640 1 --2- 172.21.15.32:0/248150503 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b40cfab0 0x7f66b40b5c90 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66bbaf9640 1 --2- 172.21.15.32:0/248150503 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b40d2740 0x7f66b40cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66bbaf9640 1 -- 172.21.15.32:0/248150503 >> 172.21.15.32:0/248150503 conn(0x7f66b4065ee0 msgr2=0x7f66b406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66bbaf9640 1 -- 172.21.15.32:0/248150503 shutdown_connections 2023-12-20T04:44:11.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66bbaf9640 1 -- 172.21.15.32:0/248150503 wait complete. 2023-12-20T04:44:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.485+0000 7f66bbaf9640 1 Processor -- start 2023-12-20T04:44:11.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66bbaf9640 1 -- start start 2023-12-20T04:44:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66bbaf9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66b40cfab0 0x7f66b41534d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66bbaf9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b40d2740 0x7f66b4150e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66bbaf9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b415cdb0 0x7f66b41513b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66bbaf9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b4157030 con 0x7f66b40d2740 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66bbaf9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40cca60 con 0x7f66b40cfab0 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66bbaf9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66b40cc4a0 con 0x7f66b415cdb0 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66b9094640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b40d2740 0x7f66b4150e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66b9094640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b40d2740 0x7f66b4150e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50628/0 (socket says 172.21.15.32:50628) 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66b9094640 1 -- 172.21.15.32:0/4234250761 learned_addr learned my addr 172.21.15.32:0/4234250761 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66ba096640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b415cdb0 0x7f66b41513b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66b9895640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66b40cfab0 0x7f66b41534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66b9094640 1 -- 172.21.15.32:0/4234250761 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b415cdb0 msgr2=0x7f66b41513b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66b9094640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b415cdb0 0x7f66b41513b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66b9094640 1 -- 172.21.15.32:0/4234250761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66b40cfab0 msgr2=0x7f66b41534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66b9094640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66b40cfab0 0x7f66b41534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66b9094640 1 -- 172.21.15.32:0/4234250761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66ac009a70 con 0x7f66b40d2740 2023-12-20T04:44:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66b9094640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b40d2740 0x7f66b4150e70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f66a400e930 tx=0x7f66a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66aa7fc640 1 -- 172.21.15.32:0/4234250761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a4009690 con 0x7f66b40d2740 2023-12-20T04:44:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66aa7fc640 1 -- 172.21.15.32:0/4234250761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66a400ca00 con 0x7f66b40d2740 2023-12-20T04:44:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66aa7fc640 1 -- 172.21.15.32:0/4234250761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66a400cd70 con 0x7f66b40d2740 2023-12-20T04:44:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66bbaf9640 1 -- 172.21.15.32:0/4234250761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66b40cc780 con 0x7f66b40d2740 2023-12-20T04:44:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66bbaf9640 1 -- 172.21.15.32:0/4234250761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66b4157450 con 0x7f66b40d2740 2023-12-20T04:44:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.489+0000 7f66bbaf9640 1 -- 172.21.15.32:0/4234250761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66b4156ac0 con 0x7f66b40d2740 2023-12-20T04:44:11.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.493+0000 7f66aa7fc640 1 -- 172.21.15.32:0/4234250761 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 43) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f66a4013470 con 0x7f66b40d2740 2023-12-20T04:44:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.493+0000 7f66aa7fc640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6698065b80 0x7f6698068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.493+0000 7f66aa7fc640 1 -- 172.21.15.32:0/4234250761 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6010+0+0 (secure 0 0 0) 0x7f66a4083690 con 0x7f66b40d2740 2023-12-20T04:44:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.493+0000 7f66b9895640 1 -- 172.21.15.32:0/4234250761 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6698065b80 msgr2=0x7f6698068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.493+0000 7f66b9895640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6698065b80 0x7f6698068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:11.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.501+0000 7f66aa7fc640 1 -- 172.21.15.32:0/4234250761 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f66a4050f70 con 0x7f66b40d2740 2023-12-20T04:44:11.505 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:11.503+0000 7fac0702d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:44:11.513 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:11.513+0000 7fa956f8e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:11.588 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:11.589+0000 7fa956f8e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:44:11.648 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:11.647+0000 7fac0702d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.689+0000 7f66bbaf9640 1 -- 172.21.15.32:0/4234250761 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f66b404e000 con 0x7f66b40d2740 2023-12-20T04:44:11.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.689+0000 7f66aa7fc640 1 -- 172.21.15.32:0/4234250761 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4699 (secure 0 0 0) 0x7f66a4054590 con 0x7f66b40d2740 2023-12-20T04:44:11.691 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:11.691 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/655215798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":655215798},{"type":"v1","addr":"172.21.15.32:6837","nonce":655215798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/661544180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":661544180},{"type":"v1","addr":"172.21.15.39:6834","nonce":661544180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6855,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/833127896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":833127896},{"type":"v1","addr":"172.21.15.32:6836","nonce":833127896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:54.874151+0000","modified":"2023-12-20T04:43:57.893768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6864":{"gid":6864,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/979895154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":979895154},{"type":"v1","addr":"172.21.15.39:6835","nonce":979895154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-20T04:44:11.691 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2023-12-20T04:44:11.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 -- 172.21.15.32:0/4234250761 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6698065b80 msgr2=0x7f6698068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:11.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6698065b80 0x7f6698068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 -- 172.21.15.32:0/4234250761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b40d2740 msgr2=0x7f66b4150e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:11.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b40d2740 0x7f66b4150e70 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f66a400e930 tx=0x7f66a400edf0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 -- 172.21.15.32:0/4234250761 shutdown_connections 2023-12-20T04:44:11.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6698065b80 0x7f6698068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66b415cdb0 0x7f66b41513b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66b40d2740 0x7f66b4150e70 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 --2- 172.21.15.32:0/4234250761 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66b40cfab0 0x7f66b41534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 -- 172.21.15.32:0/4234250761 >> 172.21.15.32:0/4234250761 conn(0x7f66b4065ee0 msgr2=0x7f66b40b74c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:11.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 -- 172.21.15.32:0/4234250761 shutdown_connections 2023-12-20T04:44:11.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.693+0000 7f66bbaf9640 1 -- 172.21.15.32:0/4234250761 wait complete. 2023-12-20T04:44:11.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:44:11.734 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:11.733+0000 7fa956f8e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:11.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.877+0000 7f711d125640 1 Processor -- start 2023-12-20T04:44:11.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.877+0000 7f711d125640 1 -- start start 2023-12-20T04:44:11.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.877+0000 7f711d125640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f711815cdb0 0x7f71180d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.877+0000 7f711d125640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71180cfc80 0x7f71180b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.877+0000 7f711d125640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f711815d190 con 0x7f71180d2740 2023-12-20T04:44:11.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.877+0000 7f711d125640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71180d16a0 con 0x7f71180cfc80 2023-12-20T04:44:11.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.877+0000 7f711d125640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71180d1980 con 0x7f711815cdb0 2023-12-20T04:44:11.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f711759e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71180cfc80 0x7f71180b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:11.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f7116d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f711815cdb0 0x7f71180d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:11.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f711759e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71180cfc80 0x7f71180b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:52926/0 (socket says 172.21.15.32:52926) 2023-12-20T04:44:11.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f711759e640 1 -- 172.21.15.32:0/489683404 learned_addr learned my addr 172.21.15.32:0/489683404 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:11.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f7116d9d640 1 -- 172.21.15.32:0/489683404 >> v2:172.21.15.39:3300/0 conn(0x7f71180cfc80 msgr2=0x7f71180b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:11.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f7116d9d640 1 --2- 172.21.15.32:0/489683404 >> v2:172.21.15.39:3300/0 conn(0x7f71180cfc80 0x7f71180b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f711659c640 1 -- 172.21.15.32:0/489683404 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 240299770 0 0) 0x7f711815d190 con 0x7f71180d2740 2023-12-20T04:44:11.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f7116d9d640 1 -- 172.21.15.32:0/489683404 >> v1:172.21.15.39:6789/0 conn(0x7f71180d2740 legacy=0x7f71180cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:11.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f7116d9d640 1 -- 172.21.15.32:0/489683404 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71180d1c60 con 0x7f711815cdb0 2023-12-20T04:44:11.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f711759e640 1 --2- 172.21.15.32:0/489683404 >> v2:172.21.15.39:3300/0 conn(0x7f71180cfc80 0x7f71180b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:11.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f7116d9d640 1 --2- 172.21.15.32:0/489683404 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f711815cdb0 0x7f71180d0be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f71080097c0 tx=0x7f710801ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca6dc6c6b0766902 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f711659c640 1 -- 172.21.15.32:0/489683404 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71080a8970 con 0x7f711815cdb0 2023-12-20T04:44:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f711659c640 1 -- 172.21.15.32:0/489683404 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71080a8ad0 con 0x7f711815cdb0 2023-12-20T04:44:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.881+0000 7f711659c640 1 -- 172.21.15.32:0/489683404 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71080a8e60 con 0x7f711815cdb0 2023-12-20T04:44:11.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 -- 172.21.15.32:0/489683404 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f711815cdb0 msgr2=0x7f71180d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 --2- 172.21.15.32:0/489683404 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f711815cdb0 0x7f71180d0be0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f71080097c0 tx=0x7f710801ec70 comp rx=0 tx=0).stop 2023-12-20T04:44:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 -- 172.21.15.32:0/489683404 shutdown_connections 2023-12-20T04:44:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 --2- 172.21.15.32:0/489683404 >> v2:172.21.15.39:3300/0 conn(0x7f71180cfc80 0x7f71180b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 --2- 172.21.15.32:0/489683404 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f711815cdb0 0x7f71180d0be0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 -- 172.21.15.32:0/489683404 >> 172.21.15.32:0/489683404 conn(0x7f7118065ee0 msgr2=0x7f711806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 -- 172.21.15.32:0/489683404 shutdown_connections 2023-12-20T04:44:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 -- 172.21.15.32:0/489683404 wait complete. 2023-12-20T04:44:11.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 Processor -- start 2023-12-20T04:44:11.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 -- start start 2023-12-20T04:44:11.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71180cfc80 0x7f711814e380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71180d2740 0x7f71180c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f711815cdb0 0x7f71180b6b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711815f2e0 con 0x7f71180cfc80 2023-12-20T04:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f711814e8c0 con 0x7f71180d2740 2023-12-20T04:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711d125640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71180c8ce0 con 0x7f711815cdb0 2023-12-20T04:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f7116d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71180cfc80 0x7f711814e380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f7116d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71180cfc80 0x7f711814e380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50632/0 (socket says 172.21.15.32:50632) 2023-12-20T04:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f7116d9d640 1 -- 172.21.15.32:0/3071140928 learned_addr learned my addr 172.21.15.32:0/3071140928 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f711759e640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f711815cdb0 0x7f71180b6b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.885+0000 7f710ffff640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71180d2740 0x7f71180c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f7116d9d640 1 -- 172.21.15.32:0/3071140928 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f711815cdb0 msgr2=0x7f71180b6b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f7116d9d640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f711815cdb0 0x7f71180b6b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f7116d9d640 1 -- 172.21.15.32:0/3071140928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71180d2740 msgr2=0x7f71180c87a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f7116d9d640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71180d2740 0x7f71180c87a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f7116d9d640 1 -- 172.21.15.32:0/3071140928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7110009b30 con 0x7f71180cfc80 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f7116d9d640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71180cfc80 0x7f711814e380 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f71080097c0 tx=0x7f710801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f710f7fe640 1 -- 172.21.15.32:0/3071140928 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71080bb640 con 0x7f71180cfc80 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f710f7fe640 1 -- 172.21.15.32:0/3071140928 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71080bb7c0 con 0x7f71180cfc80 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f710f7fe640 1 -- 172.21.15.32:0/3071140928 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71080bbb70 con 0x7f71180cfc80 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f711d125640 1 -- 172.21.15.32:0/3071140928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7108009420 con 0x7f71180cfc80 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f711d125640 1 -- 172.21.15.32:0/3071140928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f711815e9b0 con 0x7f71180cfc80 2023-12-20T04:44:11.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f711d125640 1 -- 172.21.15.32:0/3071140928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f711804e000 con 0x7f71180cfc80 2023-12-20T04:44:11.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.889+0000 7f710f7fe640 1 -- 172.21.15.32:0/3071140928 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 43) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f71080b7040 con 0x7f71180cfc80 2023-12-20T04:44:11.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.893+0000 7f710f7fe640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70f8065b80 0x7f70f8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:11.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.893+0000 7f710ffff640 1 -- 172.21.15.32:0/3071140928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70f8065b80 msgr2=0x7f70f8068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:11.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.893+0000 7f710f7fe640 1 -- 172.21.15.32:0/3071140928 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6010+0+0 (secure 0 0 0) 0x7f7108121690 con 0x7f71180cfc80 2023-12-20T04:44:11.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.893+0000 7f710ffff640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70f8065b80 0x7f70f8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:11.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:11.897+0000 7f710f7fe640 1 -- 172.21.15.32:0/3071140928 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f71080eeda0 con 0x7f71180cfc80 2023-12-20T04:44:12.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.057+0000 7f711d125640 1 -- 172.21.15.32:0/3071140928 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f711806b8d0 con 0x7f71180cfc80 2023-12-20T04:44:12.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.057+0000 7f710f7fe640 1 -- 172.21.15.32:0/3071140928 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) v1 ==== 74+0+16481 (secure 0 0 0) 0x7f71080f23c0 con 0x7f71180cfc80 2023-12-20T04:44:12.058 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:12.059 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":94,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:44:06.958760+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:52.582628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:53.871341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2058302965":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/1852394479":"2023-12-21T04:44:06.958717+0000","172.21.15.32:6833/38009":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/2860021829":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/3370166328":"2023-12-21T04:44:06.958717+0000","172.21.15.32:6832/38009":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/2187917072":"2023-12-21T04:44:06.958717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:44:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 -- 172.21.15.32:0/3071140928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70f8065b80 msgr2=0x7f70f8068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70f8065b80 0x7f70f8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 -- 172.21.15.32:0/3071140928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71180cfc80 msgr2=0x7f711814e380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71180cfc80 0x7f711814e380 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f71080097c0 tx=0x7f710801ba10 comp rx=0 tx=0).stop 2023-12-20T04:44:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 -- 172.21.15.32:0/3071140928 shutdown_connections 2023-12-20T04:44:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70f8065b80 0x7f70f8068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f711815cdb0 0x7f71180b6b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71180d2740 0x7f71180c87a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 --2- 172.21.15.32:0/3071140928 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71180cfc80 0x7f711814e380 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 -- 172.21.15.32:0/3071140928 >> 172.21.15.32:0/3071140928 conn(0x7f7118065ee0 msgr2=0x7f711815bfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 -- 172.21.15.32:0/3071140928 shutdown_connections 2023-12-20T04:44:12.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.061+0000 7f711d125640 1 -- 172.21.15.32:0/3071140928 wait complete. 2023-12-20T04:44:12.081 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 10, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:43:52.582628+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '92', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 11, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:43:53.871341+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '93', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:44:12.081 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:44:12.217 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:12.215+0000 7fac0702d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:44:12.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.229+0000 7fb7aadce640 1 Processor -- start 2023-12-20T04:44:12.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.229+0000 7fb7aadce640 1 -- start start 2023-12-20T04:44:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.229+0000 7fb7aadce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7a40b0cb0 0x7fb7a40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.229+0000 7fb7aadce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7a4070230 0x7fb7a40706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.229+0000 7fb7aadce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7a415b6a0 con 0x7fb7a415abb0 2023-12-20T04:44:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.229+0000 7fb7aadce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb7a406e160 con 0x7fb7a4070230 2023-12-20T04:44:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.229+0000 7fb7aadce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4070be0 con 0x7fb7a40b0cb0 2023-12-20T04:44:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a936b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb7a4070230 0x7fb7a40706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a8b6a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb7a415abb0 0x7fb7a415bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37876/0 (socket says 172.21.15.32:37876) 2023-12-20T04:44:12.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a8b6a640 1 -- 172.21.15.32:0/4153621299 learned_addr learned my addr 172.21.15.32:0/4153621299 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:12.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a3fff640 1 --2- 172.21.15.32:0/4153621299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7a40b0cb0 0x7fb7a40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a3fff640 1 -- 172.21.15.32:0/4153621299 >> v2:172.21.15.32:3300/0 conn(0x7fb7a4070230 msgr2=0x7fb7a40706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:12.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a3fff640 1 --2- 172.21.15.32:0/4153621299 >> v2:172.21.15.32:3300/0 conn(0x7fb7a4070230 0x7fb7a40706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a3fff640 1 -- 172.21.15.32:0/4153621299 >> v1:172.21.15.32:6789/0 conn(0x7fb7a415abb0 legacy=0x7fb7a415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a37fe640 1 -- 172.21.15.32:0/4153621299 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3038928293 0 0) 0x7fb7a415b6a0 con 0x7fb7a415abb0 2023-12-20T04:44:12.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a3fff640 1 -- 172.21.15.32:0/4153621299 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb7a406d8c0 con 0x7fb7a40b0cb0 2023-12-20T04:44:12.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a3fff640 1 --2- 172.21.15.32:0/4153621299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7a40b0cb0 0x7fb7a40caaf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb7900097c0 tx=0x7fb79001c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9722d9f8059f5042 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:12.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a37fe640 1 -- 172.21.15.32:0/4153621299 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7900ab680 con 0x7fb7a40b0cb0 2023-12-20T04:44:12.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a37fe640 1 -- 172.21.15.32:0/4153621299 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb7900ab7e0 con 0x7fb7a40b0cb0 2023-12-20T04:44:12.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.233+0000 7fb7a37fe640 1 -- 172.21.15.32:0/4153621299 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7900abb70 con 0x7fb7a40b0cb0 2023-12-20T04:44:12.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 -- 172.21.15.32:0/4153621299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7a40b0cb0 msgr2=0x7fb7a40caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 --2- 172.21.15.32:0/4153621299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7a40b0cb0 0x7fb7a40caaf0 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fb7900097c0 tx=0x7fb79001c480 comp rx=0 tx=0).stop 2023-12-20T04:44:12.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 -- 172.21.15.32:0/4153621299 shutdown_connections 2023-12-20T04:44:12.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 --2- 172.21.15.32:0/4153621299 >> v2:172.21.15.32:3300/0 conn(0x7fb7a4070230 0x7fb7a40706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 --2- 172.21.15.32:0/4153621299 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7a40b0cb0 0x7fb7a40caaf0 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 -- 172.21.15.32:0/4153621299 >> 172.21.15.32:0/4153621299 conn(0x7fb7a406a510 msgr2=0x7fb7a40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:12.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 -- 172.21.15.32:0/4153621299 shutdown_connections 2023-12-20T04:44:12.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 -- 172.21.15.32:0/4153621299 wait complete. 2023-12-20T04:44:12.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 Processor -- start 2023-12-20T04:44:12.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 -- start start 2023-12-20T04:44:12.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7a4070230 0x7fb7a414e870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7a40b0cb0 0x7fb7a413bac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a415abb0 0x7fb7a40cd350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4141290 con 0x7fb7a4070230 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4141570 con 0x7fb7a415abb0 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7aadce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb7a4141850 con 0x7fb7a40b0cb0 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7a8b6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7a4070230 0x7fb7a414e870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7a8b6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7a4070230 0x7fb7a414e870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50650/0 (socket says 172.21.15.32:50650) 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7a8b6a640 1 -- 172.21.15.32:0/3950905160 learned_addr learned my addr 172.21.15.32:0/3950905160 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.237+0000 7fb7a3fff640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7a40b0cb0 0x7fb7a413bac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a936b640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a415abb0 0x7fb7a40cd350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a8b6a640 1 -- 172.21.15.32:0/3950905160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7a40b0cb0 msgr2=0x7fb7a413bac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a8b6a640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7a40b0cb0 0x7fb7a413bac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a8b6a640 1 -- 172.21.15.32:0/3950905160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a415abb0 msgr2=0x7fb7a40cd350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a8b6a640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a415abb0 0x7fb7a40cd350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a8b6a640 1 -- 172.21.15.32:0/3950905160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb790009420 con 0x7fb7a4070230 2023-12-20T04:44:12.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a8b6a640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7a4070230 0x7fb7a414e870 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb79800c480 tx=0x7fb79800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a17fa640 1 -- 172.21.15.32:0/3950905160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb7980074e0 con 0x7fb7a4070230 2023-12-20T04:44:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a17fa640 1 -- 172.21.15.32:0/3950905160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb798007660 con 0x7fb7a4070230 2023-12-20T04:44:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7aadce640 1 -- 172.21.15.32:0/3950905160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb7a4141b30 con 0x7fb7a4070230 2023-12-20T04:44:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a17fa640 1 -- 172.21.15.32:0/3950905160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb798004a30 con 0x7fb7a4070230 2023-12-20T04:44:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7aadce640 1 -- 172.21.15.32:0/3950905160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7a414edb0 con 0x7fb7a4070230 2023-12-20T04:44:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7aadce640 1 -- 172.21.15.32:0/3950905160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb7a404e000 con 0x7fb7a4070230 2023-12-20T04:44:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a17fa640 1 -- 172.21.15.32:0/3950905160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 43) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fb798004b90 con 0x7fb7a4070230 2023-12-20T04:44:12.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a17fa640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb788065b80 0x7fb788068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a17fa640 1 -- 172.21.15.32:0/3950905160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6010+0+0 (secure 0 0 0) 0x7fb7980825f0 con 0x7fb7a4070230 2023-12-20T04:44:12.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a3fff640 1 -- 172.21.15.32:0/3950905160 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb788065b80 msgr2=0x7fb788068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:12.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.241+0000 7fb7a3fff640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb788065b80 0x7fb788068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:12.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.245+0000 7fb7a17fa640 1 -- 172.21.15.32:0/3950905160 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb79804fe10 con 0x7fb7a4070230 2023-12-20T04:44:12.305 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:12.303+0000 7fac0702d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:44:12.317 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:12.317+0000 7fa956f8e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:44:12.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.405+0000 7fb7aadce640 1 -- 172.21.15.32:0/3950905160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb7a4066e50 con 0x7fb7a4070230 2023-12-20T04:44:12.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.405+0000 7fb7a17fa640 1 -- 172.21.15.32:0/3950905160 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb798053430 con 0x7fb7a4070230 2023-12-20T04:44:12.406 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:44:12.406 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:12.405+0000 7fa956f8e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:44:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 -- 172.21.15.32:0/3950905160 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb788065b80 msgr2=0x7fb788068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb788065b80 0x7fb788068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 -- 172.21.15.32:0/3950905160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7a4070230 msgr2=0x7fb7a414e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7a4070230 0x7fb7a414e870 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7fb79800c480 tx=0x7fb79800c940 comp rx=0 tx=0).stop 2023-12-20T04:44:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 -- 172.21.15.32:0/3950905160 shutdown_connections 2023-12-20T04:44:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb788065b80 0x7fb788068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7a415abb0 0x7fb7a40cd350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7a40b0cb0 0x7fb7a413bac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 --2- 172.21.15.32:0/3950905160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb7a4070230 0x7fb7a414e870 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 -- 172.21.15.32:0/3950905160 >> 172.21.15.32:0/3950905160 conn(0x7fb7a406a510 msgr2=0x7fb7a40c4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 -- 172.21.15.32:0/3950905160 shutdown_connections 2023-12-20T04:44:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.409+0000 7fb7aadce640 1 -- 172.21.15.32:0/3950905160 wait complete. 2023-12-20T04:44:12.428 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:44:12.462 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:12.459+0000 7fac0702d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:44:12.549 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:12.547+0000 7fac0702d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:44:12.567 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:12.565+0000 7fa956f8e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:44:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee477f3640 1 Processor -- start 2023-12-20T04:44:12.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee477f3640 1 -- start start 2023-12-20T04:44:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee477f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee4015ce60 0x7fee400d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee477f3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee400d3130 0x7fee400d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee477f3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee400d2650 con 0x7fee400d1300 2023-12-20T04:44:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee477f3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee400d3570 con 0x7fee400d3130 2023-12-20T04:44:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee477f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee400d1a40 con 0x7fee4015ce60 2023-12-20T04:44:12.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee44d8e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fee400d3130 0x7fee400d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee45d90640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fee400d1300 0x7fee400b46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37890/0 (socket says 172.21.15.32:37890) 2023-12-20T04:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee45d90640 1 -- 172.21.15.32:0/1460061786 learned_addr learned my addr 172.21.15.32:0/1460061786 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee4558f640 1 --2- 172.21.15.32:0/1460061786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee4015ce60 0x7fee400d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee44d8e640 1 -- 172.21.15.32:0/1460061786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee4015ce60 msgr2=0x7fee400d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee44d8e640 1 --2- 172.21.15.32:0/1460061786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee4015ce60 0x7fee400d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee44d8e640 1 -- 172.21.15.32:0/1460061786 >> v1:172.21.15.32:6789/0 conn(0x7fee400d1300 legacy=0x7fee400b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.577+0000 7fee44d8e640 1 -- 172.21.15.32:0/1460061786 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee400d1d20 con 0x7fee400d3130 2023-12-20T04:44:12.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee44d8e640 1 --2- 172.21.15.32:0/1460061786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee400d3130 0x7fee400d0dc0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fee300097c0 tx=0x7fee3001ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1928c3d722003e41 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee37fff640 1 -- 172.21.15.32:0/1460061786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee300a8970 con 0x7fee400d3130 2023-12-20T04:44:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee37fff640 1 -- 172.21.15.32:0/1460061786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee300a8ad0 con 0x7fee400d3130 2023-12-20T04:44:12.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee37fff640 1 -- 172.21.15.32:0/1460061786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee300a8e60 con 0x7fee400d3130 2023-12-20T04:44:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee477f3640 1 -- 172.21.15.32:0/1460061786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee400d3130 msgr2=0x7fee400d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee477f3640 1 --2- 172.21.15.32:0/1460061786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee400d3130 0x7fee400d0dc0 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7fee300097c0 tx=0x7fee3001ec70 comp rx=0 tx=0).stop 2023-12-20T04:44:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee477f3640 1 -- 172.21.15.32:0/1460061786 shutdown_connections 2023-12-20T04:44:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee477f3640 1 --2- 172.21.15.32:0/1460061786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee400d3130 0x7fee400d0dc0 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee477f3640 1 --2- 172.21.15.32:0/1460061786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee4015ce60 0x7fee400d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee477f3640 1 -- 172.21.15.32:0/1460061786 >> 172.21.15.32:0/1460061786 conn(0x7fee4006b5e0 msgr2=0x7fee4006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee477f3640 1 -- 172.21.15.32:0/1460061786 shutdown_connections 2023-12-20T04:44:12.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.581+0000 7fee477f3640 1 -- 172.21.15.32:0/1460061786 wait complete. 2023-12-20T04:44:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee477f3640 1 Processor -- start 2023-12-20T04:44:12.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee477f3640 1 -- start start 2023-12-20T04:44:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee477f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee400d1300 0x7fee400aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee477f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee400d3130 0x7fee40151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee477f3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4015ce60 0x7fee400ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee477f3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee400aa500 con 0x7fee4015ce60 2023-12-20T04:44:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee477f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee401516a0 con 0x7fee400d1300 2023-12-20T04:44:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee477f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee400a95a0 con 0x7fee400d3130 2023-12-20T04:44:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee4558f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee400d1300 0x7fee400aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee44d8e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee400d3130 0x7fee40151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee45d90640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4015ce60 0x7fee400ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee44d8e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee400d3130 0x7fee40151160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:54938/0 (socket says 172.21.15.32:54938) 2023-12-20T04:44:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee4558f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee400d1300 0x7fee400aa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:52948/0 (socket says 172.21.15.32:52948) 2023-12-20T04:44:12.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee44d8e640 1 -- 172.21.15.32:0/1826704216 learned_addr learned my addr 172.21.15.32:0/1826704216 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee4558f640 1 -- 172.21.15.32:0/1826704216 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee400d3130 msgr2=0x7fee40151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee4558f640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee400d3130 0x7fee40151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee4558f640 1 -- 172.21.15.32:0/1826704216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4015ce60 msgr2=0x7fee400ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee4558f640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4015ce60 0x7fee400ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee4558f640 1 -- 172.21.15.32:0/1826704216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee30009420 con 0x7fee400d1300 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee44d8e640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee400d3130 0x7fee40151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee4558f640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee400d1300 0x7fee400aa920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00e870 tx=0x7fee3c00ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee35ffb640 1 -- 172.21.15.32:0/1826704216 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c00ee60 con 0x7fee400d1300 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee35ffb640 1 -- 172.21.15.32:0/1826704216 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee3c00c940 con 0x7fee400d1300 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee35ffb640 1 -- 172.21.15.32:0/1826704216 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3c00ccd0 con 0x7fee400d1300 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee477f3640 1 -- 172.21.15.32:0/1826704216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee400a9880 con 0x7fee400d1300 2023-12-20T04:44:12.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.585+0000 7fee477f3640 1 -- 172.21.15.32:0/1826704216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee400a9e40 con 0x7fee400d1300 2023-12-20T04:44:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.589+0000 7fee477f3640 1 -- 172.21.15.32:0/1826704216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee400aa770 con 0x7fee400d1300 2023-12-20T04:44:12.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.589+0000 7fee35ffb640 1 -- 172.21.15.32:0/1826704216 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 43) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fee3c013470 con 0x7fee400d1300 2023-12-20T04:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.589+0000 7fee35ffb640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee24065b80 0x7fee24068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.589+0000 7fee35ffb640 1 -- 172.21.15.32:0/1826704216 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6010+0+0 (secure 0 0 0) 0x7fee3c0833e0 con 0x7fee400d1300 2023-12-20T04:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.589+0000 7fee44d8e640 1 -- 172.21.15.32:0/1826704216 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee24065b80 msgr2=0x7fee24068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:12.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.589+0000 7fee44d8e640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee24065b80 0x7fee24068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:12.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.593+0000 7fee35ffb640 1 -- 172.21.15.32:0/1826704216 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee3c050c10 con 0x7fee400d1300 2023-12-20T04:44:12.659 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:12.657+0000 7fa956f8e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:44:12.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.749+0000 7fee477f3640 1 -- 172.21.15.32:0/1826704216 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee4004e000 con 0x7fee400d1300 2023-12-20T04:44:12.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.749+0000 7fee35ffb640 1 -- 172.21.15.32:0/1826704216 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee3c054230 con 0x7fee400d1300 2023-12-20T04:44:12.754 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 -- 172.21.15.32:0/1826704216 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee24065b80 msgr2=0x7fee24068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee24065b80 0x7fee24068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 -- 172.21.15.32:0/1826704216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee400d1300 msgr2=0x7fee400aa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee400d1300 0x7fee400aa920 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7fee3c00e870 tx=0x7fee3c00ed30 comp rx=0 tx=0).stop 2023-12-20T04:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 -- 172.21.15.32:0/1826704216 shutdown_connections 2023-12-20T04:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee24065b80 0x7fee24068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee4015ce60 0x7fee400ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee400d3130 0x7fee40151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 --2- 172.21.15.32:0/1826704216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee400d1300 0x7fee400aa920 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 -- 172.21.15.32:0/1826704216 >> 172.21.15.32:0/1826704216 conn(0x7fee4006b5e0 msgr2=0x7fee400b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:12.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 -- 172.21.15.32:0/1826704216 shutdown_connections 2023-12-20T04:44:12.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.753+0000 7fee477f3640 1 -- 172.21.15.32:0/1826704216 wait complete. 2023-12-20T04:44:12.773 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:44:12.883 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:12.883+0000 7fac0702d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:44:12.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb9ac4640 1 Processor -- start 2023-12-20T04:44:12.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb9ac4640 1 -- start start 2023-12-20T04:44:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb9ac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 0x7f1fb40d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb9ac4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1fb40d1300 0x7f1fb40b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb9ac4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb40d2650 con 0x7f1fb415ce60 2023-12-20T04:44:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb9ac4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb40d1a40 con 0x7f1fb40d1300 2023-12-20T04:44:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb9ac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40d1d20 con 0x7f1fb40d3300 2023-12-20T04:44:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb37fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1fb415ce60 0x7f1fb40d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37900/0 (socket says 172.21.15.32:37900) 2023-12-20T04:44:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb37fe640 1 -- 172.21.15.32:0/251709290 learned_addr learned my addr 172.21.15.32:0/251709290 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:12.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb3fff640 1 --2- 172.21.15.32:0/251709290 >> v2:172.21.15.32:3300/0 conn(0x7f1fb40d1300 0x7f1fb40b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb2ffd640 1 --2- 172.21.15.32:0/251709290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 0x7f1fb40d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb27fc640 1 -- 172.21.15.32:0/251709290 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 164305974 0 0) 0x7f1fb40d2650 con 0x7f1fb415ce60 2023-12-20T04:44:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb27fc640 1 -- 172.21.15.32:0/251709290 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1f9c003530 con 0x7f1fb415ce60 2023-12-20T04:44:12.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb3fff640 1 -- 172.21.15.32:0/251709290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 msgr2=0x7f1fb40d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb3fff640 1 --2- 172.21.15.32:0/251709290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 0x7f1fb40d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb3fff640 1 -- 172.21.15.32:0/251709290 >> v1:172.21.15.32:6789/0 conn(0x7f1fb415ce60 legacy=0x7f1fb40d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb3fff640 1 -- 172.21.15.32:0/251709290 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb40d2000 con 0x7f1fb40d1300 2023-12-20T04:44:12.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.921+0000 7f1fb2ffd640 1 --2- 172.21.15.32:0/251709290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 0x7f1fb40d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:12.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb3fff640 1 --2- 172.21.15.32:0/251709290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb40d1300 0x7f1fb40b5e90 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009d90 tx=0x7f1fac01f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172407a1a0523292 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb27fc640 1 -- 172.21.15.32:0/251709290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0af600 con 0x7f1fb40d1300 2023-12-20T04:44:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb27fc640 1 -- 172.21.15.32:0/251709290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fac0ab050 con 0x7f1fb40d1300 2023-12-20T04:44:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb27fc640 1 -- 172.21.15.32:0/251709290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fac0af960 con 0x7f1fb40d1300 2023-12-20T04:44:12.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/251709290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb40d1300 msgr2=0x7f1fb40b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb9ac4640 1 --2- 172.21.15.32:0/251709290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb40d1300 0x7f1fb40b5e90 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f1fac009d90 tx=0x7f1fac01f130 comp rx=0 tx=0).stop 2023-12-20T04:44:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/251709290 shutdown_connections 2023-12-20T04:44:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb9ac4640 1 --2- 172.21.15.32:0/251709290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb40d1300 0x7f1fb40b5e90 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb9ac4640 1 --2- 172.21.15.32:0/251709290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 0x7f1fb40d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/251709290 >> 172.21.15.32:0/251709290 conn(0x7f1fb406b5e0 msgr2=0x7f1fb406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/251709290 shutdown_connections 2023-12-20T04:44:12.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.925+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/251709290 wait complete. 2023-12-20T04:44:12.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb9ac4640 1 Processor -- start 2023-12-20T04:44:12.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb9ac4640 1 -- start start 2023-12-20T04:44:12.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb9ac4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb40d1300 0x7f1fb4158870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb9ac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 0x7f1fb41e5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb9ac4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb415ce60 0x7f1fb415aa70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb9ac4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb41e69f0 con 0x7f1fb415ce60 2023-12-20T04:44:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb9ac4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb4158db0 con 0x7f1fb40d1300 2023-12-20T04:44:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb9ac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb41e5e70 con 0x7f1fb40d3300 2023-12-20T04:44:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb415ce60 0x7f1fb415aa70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb415ce60 0x7f1fb415aa70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50700/0 (socket says 172.21.15.32:50700) 2023-12-20T04:44:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb3fff640 1 -- 172.21.15.32:0/3037245686 learned_addr learned my addr 172.21.15.32:0/3037245686 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:12.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb2ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 0x7f1fb41e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb37fe640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb40d1300 0x7f1fb4158870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb2ffd640 1 -- 172.21.15.32:0/3037245686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb40d1300 msgr2=0x7f1fb4158870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb2ffd640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb40d1300 0x7f1fb4158870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb2ffd640 1 -- 172.21.15.32:0/3037245686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb415ce60 msgr2=0x7f1fb415aa70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb2ffd640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb415ce60 0x7f1fb415aa70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb2ffd640 1 -- 172.21.15.32:0/3037245686 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8009420 con 0x7f1fb40d3300 2023-12-20T04:44:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb3fff640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb415ce60 0x7f1fb415aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1fb2ffd640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 0x7f1fb41e5470 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800e9b0 tx=0x7f1fa800ee70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.929+0000 7f1f9bfff640 1 -- 172.21.15.32:0/3037245686 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa800c9a0 con 0x7f1fb40d3300 2023-12-20T04:44:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.933+0000 7f1f9bfff640 1 -- 172.21.15.32:0/3037245686 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa800cb00 con 0x7f1fb40d3300 2023-12-20T04:44:12.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.933+0000 7f1f9bfff640 1 -- 172.21.15.32:0/3037245686 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa8013470 con 0x7f1fb40d3300 2023-12-20T04:44:12.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.933+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/3037245686 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fac009a70 con 0x7f1fb40d3300 2023-12-20T04:44:12.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.933+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/3037245686 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb41e6430 con 0x7f1fb40d3300 2023-12-20T04:44:12.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.933+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/3037245686 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb404e000 con 0x7f1fb40d3300 2023-12-20T04:44:12.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.933+0000 7f1f9bfff640 1 -- 172.21.15.32:0/3037245686 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 43) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f1fa80059a0 con 0x7f1fb40d3300 2023-12-20T04:44:12.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.933+0000 7f1f9bfff640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1fa0065b80 0x7f1fa0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:12.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.933+0000 7f1f9bfff640 1 -- 172.21.15.32:0/3037245686 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(94..94 src has 1..94) v4 ==== 6010+0+0 (secure 0 0 0) 0x7f1fa8083780 con 0x7f1fb40d3300 2023-12-20T04:44:12.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.933+0000 7f1fb37fe640 1 -- 172.21.15.32:0/3037245686 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1fa0065b80 msgr2=0x7f1fa0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:12.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.933+0000 7f1fb37fe640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1fa0065b80 0x7f1fa0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:12.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:12.937+0000 7f1f9bfff640 1 -- 172.21.15.32:0/3037245686 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1fa8051020 con 0x7f1fb40d3300 2023-12-20T04:44:12.948 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:12.947+0000 7fac0702d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:44:13.003 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.001+0000 7fa956f8e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:44:13.067 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.069+0000 7fa956f8e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:44:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.081+0000 7f1f9bfff640 1 -- 172.21.15.32:0/3037245686 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mgrmap(e 44) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f1fa80513f0 con 0x7f1fb40d3300 2023-12-20T04:44:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.081+0000 7f1f9bfff640 1 -- 172.21.15.32:0/3037245686 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1fa0065b80 msgr2=0x7f1fa0068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:13.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.081+0000 7f1f9bfff640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1fa0065b80 0x7f1fa0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/3037245686 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f1fb40b78e0 con 0x7f1fb40d3300 2023-12-20T04:44:13.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1f9bfff640 1 -- 172.21.15.32:0/3037245686 <== mon.2 v2:172.21.15.39:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v95) v1 ==== 100+0+11 (secure 0 0 0) 0x7f1fa807b020 con 0x7f1fb40d3300 2023-12-20T04:44:13.112 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:44:13.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/3037245686 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 msgr2=0x7f1fb41e5470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:13.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1fb9ac4640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 0x7f1fb41e5470 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7f1fa800e9b0 tx=0x7f1fa800ee70 comp rx=0 tx=0).stop 2023-12-20T04:44:13.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/3037245686 shutdown_connections 2023-12-20T04:44:13.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1fb9ac4640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1fa0065b80 0x7f1fa0068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1fb9ac4640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb415ce60 0x7f1fb415aa70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1fb9ac4640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb40d3300 0x7f1fb41e5470 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1fb9ac4640 1 --2- 172.21.15.32:0/3037245686 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb40d1300 0x7f1fb4158870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/3037245686 >> 172.21.15.32:0/3037245686 conn(0x7f1fb406b5e0 msgr2=0x7f1fb415c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:13.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/3037245686 shutdown_connections 2023-12-20T04:44:13.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.113+0000 7f1fb9ac4640 1 -- 172.21.15.32:0/3037245686 wait complete. 2023-12-20T04:44:13.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:44:13.145 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.141+0000 7fa8e2608640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:13.145 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.141+0000 7fa8e2608640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:13.145 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.141+0000 7fa8e2608640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:13.145 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.141+0000 7fa8e2608640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:13.145 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.141+0000 7fa8e2608640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:13.145 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.145+0000 7fa8de600640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:13.145 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.145+0000 7fa8de600640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:13.146 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.145+0000 7fa8de600640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:13.146 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.145+0000 7fa8de600640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:13.146 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:13.145+0000 7fa8de600640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:13.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.285+0000 7fad5d295640 1 Processor -- start 2023-12-20T04:44:13.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad5d295640 1 -- start start 2023-12-20T04:44:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad5d295640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fad5815cdb0 0x7fad580d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad5d295640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad580d2740 0x7fad580cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad5d295640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad580b6ca0 con 0x7fad580d1120 2023-12-20T04:44:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad5d295640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5815d190 con 0x7fad5815cdb0 2023-12-20T04:44:13.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad5d295640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad580d16a0 con 0x7fad580d2740 2023-12-20T04:44:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad4ffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad580d2740 0x7fad580cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad577fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fad580d1120 0x7fad580b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:51804/0 (socket says 172.21.15.32:51804) 2023-12-20T04:44:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad577fe640 1 -- 172.21.15.32:0/3286067147 learned_addr learned my addr 172.21.15.32:0/3286067147 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad56ffd640 1 --2- 172.21.15.32:0/3286067147 >> v2:172.21.15.39:3300/0 conn(0x7fad5815cdb0 0x7fad580d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad567fc640 1 -- 172.21.15.32:0/3286067147 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 659077370 0 0) 0x7fad500014a0 con 0x7fad580d1120 2023-12-20T04:44:13.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad567fc640 1 -- 172.21.15.32:0/3286067147 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fad44003530 con 0x7fad580d1120 2023-12-20T04:44:13.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad56ffd640 1 -- 172.21.15.32:0/3286067147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad580d2740 msgr2=0x7fad580cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:13.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad56ffd640 1 --2- 172.21.15.32:0/3286067147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad580d2740 0x7fad580cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad56ffd640 1 -- 172.21.15.32:0/3286067147 >> v1:172.21.15.39:6789/0 conn(0x7fad580d1120 legacy=0x7fad580b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:13.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad56ffd640 1 -- 172.21.15.32:0/3286067147 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad580d1980 con 0x7fad5815cdb0 2023-12-20T04:44:13.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.289+0000 7fad4ffff640 1 --2- 172.21.15.32:0/3286067147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad580d2740 0x7fad580cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:13.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad56ffd640 1 --2- 172.21.15.32:0/3286067147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad5815cdb0 0x7fad580d0be0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fad480097c0 tx=0x7fad4801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee377be54073c5ae server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:13.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad567fc640 1 -- 172.21.15.32:0/3286067147 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480a8970 con 0x7fad5815cdb0 2023-12-20T04:44:13.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad567fc640 1 -- 172.21.15.32:0/3286067147 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad480a8ad0 con 0x7fad5815cdb0 2023-12-20T04:44:13.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad567fc640 1 -- 172.21.15.32:0/3286067147 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480a8e60 con 0x7fad5815cdb0 2023-12-20T04:44:13.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad5d295640 1 -- 172.21.15.32:0/3286067147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad5815cdb0 msgr2=0x7fad580d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:13.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad5d295640 1 --2- 172.21.15.32:0/3286067147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad5815cdb0 0x7fad580d0be0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fad480097c0 tx=0x7fad4801ec70 comp rx=0 tx=0).stop 2023-12-20T04:44:13.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad5d295640 1 -- 172.21.15.32:0/3286067147 shutdown_connections 2023-12-20T04:44:13.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad5d295640 1 --2- 172.21.15.32:0/3286067147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad580d2740 0x7fad580cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad5d295640 1 --2- 172.21.15.32:0/3286067147 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad5815cdb0 0x7fad580d0be0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad5d295640 1 -- 172.21.15.32:0/3286067147 >> 172.21.15.32:0/3286067147 conn(0x7fad58065ee0 msgr2=0x7fad5806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad5d295640 1 -- 172.21.15.32:0/3286067147 shutdown_connections 2023-12-20T04:44:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad5d295640 1 -- 172.21.15.32:0/3286067147 wait complete. 2023-12-20T04:44:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad5d295640 1 Processor -- start 2023-12-20T04:44:13.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.293+0000 7fad5d295640 1 -- start start 2023-12-20T04:44:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad5d295640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d1120 0x7fad5814a210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad5d295640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad580d2740 0x7fad5815f4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad5d295640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad5815cdb0 0x7fad5815f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad5d295640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580cbad0 con 0x7fad580d1120 2023-12-20T04:44:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad5d295640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad5814e5f0 con 0x7fad580d2740 2023-12-20T04:44:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad5d295640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad5814a750 con 0x7fad5815cdb0 2023-12-20T04:44:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad56ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d1120 0x7fad5814a210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad56ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d1120 0x7fad5814a210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50792/0 (socket says 172.21.15.32:50792) 2023-12-20T04:44:13.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad56ffd640 1 -- 172.21.15.32:0/3977650754 learned_addr learned my addr 172.21.15.32:0/3977650754 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad577fe640 1 --2- 172.21.15.32:0/3977650754 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad5815cdb0 0x7fad5815f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad4ffff640 1 --2- 172.21.15.32:0/3977650754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad580d2740 0x7fad5815f4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad56ffd640 1 -- 172.21.15.32:0/3977650754 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad5815cdb0 msgr2=0x7fad5815f9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad56ffd640 1 --2- 172.21.15.32:0/3977650754 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad5815cdb0 0x7fad5815f9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad56ffd640 1 -- 172.21.15.32:0/3977650754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad580d2740 msgr2=0x7fad5815f4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad56ffd640 1 --2- 172.21.15.32:0/3977650754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad580d2740 0x7fad5815f4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad56ffd640 1 -- 172.21.15.32:0/3977650754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad40009420 con 0x7fad580d1120 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad577fe640 1 --2- 172.21.15.32:0/3977650754 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad5815cdb0 0x7fad5815f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad4ffff640 1 --2- 172.21.15.32:0/3977650754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad580d2740 0x7fad5815f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad56ffd640 1 --2- 172.21.15.32:0/3977650754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d1120 0x7fad5814a210 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fad480b3040 tx=0x7fad4801ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad4f7fe640 1 -- 172.21.15.32:0/3977650754 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480b0d60 con 0x7fad580d1120 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad4f7fe640 1 -- 172.21.15.32:0/3977650754 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad480b9430 con 0x7fad580d1120 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad5d295640 1 -- 172.21.15.32:0/3977650754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad48009420 con 0x7fad580d1120 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad4f7fe640 1 -- 172.21.15.32:0/3977650754 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad480b97e0 con 0x7fad580d1120 2023-12-20T04:44:13.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad5d295640 1 -- 172.21.15.32:0/3977650754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad580cab90 con 0x7fad580d1120 2023-12-20T04:44:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad5d295640 1 -- 172.21.15.32:0/3977650754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5804e000 con 0x7fad580d1120 2023-12-20T04:44:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.297+0000 7fad4f7fe640 1 -- 172.21.15.32:0/3977650754 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 44) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fad480b7040 con 0x7fad580d1120 2023-12-20T04:44:13.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.301+0000 7fad4f7fe640 1 -- 172.21.15.32:0/3977650754 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6268+0+0 (secure 0 0 0) 0x7fad48120a10 con 0x7fad580d1120 2023-12-20T04:44:13.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.305+0000 7fad4f7fe640 1 -- 172.21.15.32:0/3977650754 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fad480ee500 con 0x7fad580d1120 2023-12-20T04:44:13.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.493+0000 7fad5d295640 1 -- 172.21.15.32:0/3977650754 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fad5806b8d0 con 0x7fad580d1120 2023-12-20T04:44:13.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.493+0000 7fad4f7fe640 1 -- 172.21.15.32:0/3977650754 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4699 (secure 0 0 0) 0x7fad480b5090 con 0x7fad580d1120 2023-12-20T04:44:13.494 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:13.494 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/655215798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":655215798},{"type":"v1","addr":"172.21.15.32:6837","nonce":655215798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/661544180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":661544180},{"type":"v1","addr":"172.21.15.39:6834","nonce":661544180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6855,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/833127896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":833127896},{"type":"v1","addr":"172.21.15.32:6836","nonce":833127896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:54.874151+0000","modified":"2023-12-20T04:43:57.893768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6864":{"gid":6864,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/979895154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":979895154},{"type":"v1","addr":"172.21.15.39:6835","nonce":979895154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-20T04:44:13.494 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2023-12-20T04:44:13.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.497+0000 7fad5d295640 1 -- 172.21.15.32:0/3977650754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d1120 msgr2=0x7fad5814a210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:13.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.497+0000 7fad5d295640 1 --2- 172.21.15.32:0/3977650754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d1120 0x7fad5814a210 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7fad480b3040 tx=0x7fad4801ba10 comp rx=0 tx=0).stop 2023-12-20T04:44:13.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.497+0000 7fad5d295640 1 -- 172.21.15.32:0/3977650754 shutdown_connections 2023-12-20T04:44:13.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.497+0000 7fad5d295640 1 --2- 172.21.15.32:0/3977650754 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad5815cdb0 0x7fad5815f9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.497+0000 7fad5d295640 1 --2- 172.21.15.32:0/3977650754 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad580d2740 0x7fad5815f4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.497+0000 7fad5d295640 1 --2- 172.21.15.32:0/3977650754 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d1120 0x7fad5814a210 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.497+0000 7fad5d295640 1 -- 172.21.15.32:0/3977650754 >> 172.21.15.32:0/3977650754 conn(0x7fad58065ee0 msgr2=0x7fad580d33c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:13.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.497+0000 7fad5d295640 1 -- 172.21.15.32:0/3977650754 shutdown_connections 2023-12-20T04:44:13.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.497+0000 7fad5d295640 1 -- 172.21.15.32:0/3977650754 wait complete. 2023-12-20T04:44:13.515 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:44:13.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f2004996640 1 Processor -- start 2023-12-20T04:44:13.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f2004996640 1 -- start start 2023-12-20T04:44:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f2004996640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20000aece0 0x7f20000b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f2004996640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20000b0b30 0x7f20000701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f2004996640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20000c98c0 con 0x7f20001599e0 2023-12-20T04:44:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f2004996640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f20000c9ba0 con 0x7f20000aece0 2023-12-20T04:44:13.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f2004996640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20000c9e80 con 0x7f20000b0b30 2023-12-20T04:44:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f1ffdd9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f20000aece0 0x7f20000b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f1ffe59c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f20001599e0 0x7f20000c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:51808/0 (socket says 172.21.15.32:51808) 2023-12-20T04:44:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f1ffe59c640 1 -- 172.21.15.32:0/1629297170 learned_addr learned my addr 172.21.15.32:0/1629297170 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f1ffed9d640 1 --2- 172.21.15.32:0/1629297170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20000b0b30 0x7f20000701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f1ffd59a640 1 -- 172.21.15.32:0/1629297170 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1436088867 0 0) 0x7f20000c98c0 con 0x7f20001599e0 2023-12-20T04:44:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f1ffdd9b640 1 -- 172.21.15.32:0/1629297170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20000b0b30 msgr2=0x7f20000701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f1ffdd9b640 1 --2- 172.21.15.32:0/1629297170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20000b0b30 0x7f20000701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f1ffdd9b640 1 -- 172.21.15.32:0/1629297170 >> v1:172.21.15.39:6789/0 conn(0x7f20001599e0 legacy=0x7f20000c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.661+0000 7f1ffdd9b640 1 -- 172.21.15.32:0/1629297170 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20000ca160 con 0x7f20000aece0 2023-12-20T04:44:13.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f1ffdd9b640 1 --2- 172.21.15.32:0/1629297170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000aece0 0x7f20000b05f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40097c0 tx=0x7f1ff401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bba70fe73b70dd18 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:13.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f1ffd59a640 1 -- 172.21.15.32:0/1629297170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40ad640 con 0x7f20000aece0 2023-12-20T04:44:13.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f1ffd59a640 1 -- 172.21.15.32:0/1629297170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff40ad7a0 con 0x7f20000aece0 2023-12-20T04:44:13.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f1ffd59a640 1 -- 172.21.15.32:0/1629297170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff40adb30 con 0x7f20000aece0 2023-12-20T04:44:13.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f2004996640 1 -- 172.21.15.32:0/1629297170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000aece0 msgr2=0x7f20000b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:13.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f2004996640 1 --2- 172.21.15.32:0/1629297170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000aece0 0x7f20000b05f0 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f1ff40097c0 tx=0x7f1ff401c480 comp rx=0 tx=0).stop 2023-12-20T04:44:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f2004996640 1 -- 172.21.15.32:0/1629297170 shutdown_connections 2023-12-20T04:44:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f2004996640 1 --2- 172.21.15.32:0/1629297170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20000b0b30 0x7f20000701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f2004996640 1 --2- 172.21.15.32:0/1629297170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000aece0 0x7f20000b05f0 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f2004996640 1 -- 172.21.15.32:0/1629297170 >> 172.21.15.32:0/1629297170 conn(0x7f200006a320 msgr2=0x7f20000ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f2004996640 1 -- 172.21.15.32:0/1629297170 shutdown_connections 2023-12-20T04:44:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.665+0000 7f2004996640 1 -- 172.21.15.32:0/1629297170 wait complete. 2023-12-20T04:44:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f2004996640 1 Processor -- start 2023-12-20T04:44:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f2004996640 1 -- start start 2023-12-20T04:44:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f2004996640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20000aece0 0x7f200015b140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f2004996640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000b0b30 0x7f20001f6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f2004996640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001599e0 0x7f20001f6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f2004996640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20001505e0 con 0x7f20000b0b30 2023-12-20T04:44:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f2004996640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f20000c8a50 con 0x7f20001599e0 2023-12-20T04:44:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f2004996640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20000c81b0 con 0x7f20000aece0 2023-12-20T04:44:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1ffdd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000b0b30 0x7f20001f6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1ffdd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000b0b30 0x7f20001f6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50810/0 (socket says 172.21.15.32:50810) 2023-12-20T04:44:13.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1ffdd9b640 1 -- 172.21.15.32:0/1718134176 learned_addr learned my addr 172.21.15.32:0/1718134176 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1ffe59c640 1 --2- 172.21.15.32:0/1718134176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20000aece0 0x7f200015b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1ffed9d640 1 --2- 172.21.15.32:0/1718134176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001599e0 0x7f20001f6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1ffdd9b640 1 -- 172.21.15.32:0/1718134176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20000aece0 msgr2=0x7f200015b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1ffdd9b640 1 --2- 172.21.15.32:0/1718134176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20000aece0 0x7f200015b140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1ffdd9b640 1 -- 172.21.15.32:0/1718134176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001599e0 msgr2=0x7f20001f6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:13.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1ffdd9b640 1 --2- 172.21.15.32:0/1718134176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001599e0 0x7f20001f6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1ffdd9b640 1 -- 172.21.15.32:0/1718134176 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ff4009420 con 0x7f20000b0b30 2023-12-20T04:44:13.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1ffdd9b640 1 --2- 172.21.15.32:0/1718134176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000b0b30 0x7f20001f6900 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1ff401a0b0 tx=0x7f1ff4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:13.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1feeffd640 1 -- 172.21.15.32:0/1718134176 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4004220 con 0x7f20000b0b30 2023-12-20T04:44:13.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1feeffd640 1 -- 172.21.15.32:0/1718134176 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1ff40a6040 con 0x7f20000b0b30 2023-12-20T04:44:13.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f1feeffd640 1 -- 172.21.15.32:0/1718134176 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1ff4003620 con 0x7f20000b0b30 2023-12-20T04:44:13.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f2004996640 1 -- 172.21.15.32:0/1718134176 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f20000c8490 con 0x7f20000b0b30 2023-12-20T04:44:13.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f2004996640 1 -- 172.21.15.32:0/1718134176 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20000c59b0 con 0x7f20000b0b30 2023-12-20T04:44:13.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.669+0000 7f2004996640 1 -- 172.21.15.32:0/1718134176 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f200006acf0 con 0x7f20000b0b30 2023-12-20T04:44:13.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.673+0000 7f1feeffd640 1 -- 172.21.15.32:0/1718134176 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 44) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f1ff40b4040 con 0x7f20000b0b30 2023-12-20T04:44:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.673+0000 7f1feeffd640 1 -- 172.21.15.32:0/1718134176 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f1ff411c4f0 con 0x7f20000b0b30 2023-12-20T04:44:13.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.677+0000 7f1feeffd640 1 -- 172.21.15.32:0/1718134176 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1ff40ea090 con 0x7f20000b0b30 2023-12-20T04:44:13.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.837+0000 7f2004996640 1 -- 172.21.15.32:0/1718134176 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f20000c5810 con 0x7f20000b0b30 2023-12-20T04:44:13.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.837+0000 7f1feeffd640 1 -- 172.21.15.32:0/1718134176 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16853 (secure 0 0 0) 0x7f20000c5810 con 0x7f20000b0b30 2023-12-20T04:44:13.838 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:13.838 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":95,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:44:13.074292+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:52.582628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:53.871341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2756989681":"2023-12-21T04:44:13.074258+0000","172.21.15.32:0/1156261356":"2023-12-21T04:44:13.074258+0000","172.21.15.32:0/3141888936":"2023-12-21T04:44:13.074258+0000","172.21.15.32:0/4256985996":"2023-12-21T04:44:13.074258+0000","172.21.15.32:0/3308967759":"2023-12-21T04:44:13.074258+0000","172.21.15.32:0/2058302965":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/1852394479":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/1659600879":"2023-12-21T04:44:13.074258+0000","172.21.15.32:6833/38009":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/2860021829":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/3370166328":"2023-12-21T04:44:06.958717+0000","172.21.15.32:6832/38009":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/2187917072":"2023-12-21T04:44:06.958717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:44:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.841+0000 7f2004996640 1 -- 172.21.15.32:0/1718134176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000b0b30 msgr2=0x7f20001f6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.841+0000 7f2004996640 1 --2- 172.21.15.32:0/1718134176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000b0b30 0x7f20001f6900 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f1ff401a0b0 tx=0x7f1ff4019220 comp rx=0 tx=0).stop 2023-12-20T04:44:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.841+0000 7f2004996640 1 -- 172.21.15.32:0/1718134176 shutdown_connections 2023-12-20T04:44:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.841+0000 7f2004996640 1 --2- 172.21.15.32:0/1718134176 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20001599e0 0x7f20001f6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.841+0000 7f2004996640 1 --2- 172.21.15.32:0/1718134176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20000b0b30 0x7f20001f6900 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.841+0000 7f2004996640 1 --2- 172.21.15.32:0/1718134176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20000aece0 0x7f200015b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.841+0000 7f2004996640 1 -- 172.21.15.32:0/1718134176 >> 172.21.15.32:0/1718134176 conn(0x7f200006a320 msgr2=0x7f20000b3990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:13.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.841+0000 7f2004996640 1 -- 172.21.15.32:0/1718134176 shutdown_connections 2023-12-20T04:44:13.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:13.841+0000 7f2004996640 1 -- 172.21.15.32:0/1718134176 wait complete. 2023-12-20T04:44:13.859 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T04:44:13.860 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:44:14.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f5958939640 1 Processor -- start 2023-12-20T04:44:14.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f5958939640 1 -- start start 2023-12-20T04:44:14.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f5958939640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f595415cdb0 0x7f59540d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f5958939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59540d2740 0x7f59540cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f5958939640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59540b6ca0 con 0x7f59540d1120 2023-12-20T04:44:14.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f5958939640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f595415d190 con 0x7f595415cdb0 2023-12-20T04:44:14.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f5958939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59540d16a0 con 0x7f59540d2740 2023-12-20T04:44:14.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f595259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f595415cdb0 0x7f59540d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f595259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f595415cdb0 0x7f59540d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50824/0 (socket says 172.21.15.32:50824) 2023-12-20T04:44:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f595259c640 1 -- 172.21.15.32:0/3768610781 learned_addr learned my addr 172.21.15.32:0/3768610781 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f5951d9b640 1 --2- 172.21.15.32:0/3768610781 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59540d2740 0x7f59540cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f595259c640 1 -- 172.21.15.32:0/3768610781 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59540d2740 msgr2=0x7f59540cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f595259c640 1 --2- 172.21.15.32:0/3768610781 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59540d2740 0x7f59540cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f595259c640 1 -- 172.21.15.32:0/3768610781 >> v1:172.21.15.39:6789/0 conn(0x7f59540d1120 legacy=0x7f59540b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.005+0000 7f595259c640 1 -- 172.21.15.32:0/3768610781 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59540d1980 con 0x7f595415cdb0 2023-12-20T04:44:14.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f595259c640 1 --2- 172.21.15.32:0/3768610781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595415cdb0 0x7f59540d0be0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5944009870 tx=0x7f594401ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5975473e5f5cf5ed server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f595159a640 1 -- 172.21.15.32:0/3768610781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440a8a30 con 0x7f595415cdb0 2023-12-20T04:44:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f595159a640 1 -- 172.21.15.32:0/3768610781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59440a8b90 con 0x7f595415cdb0 2023-12-20T04:44:14.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f595159a640 1 -- 172.21.15.32:0/3768610781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59440b0440 con 0x7f595415cdb0 2023-12-20T04:44:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f5958939640 1 -- 172.21.15.32:0/3768610781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595415cdb0 msgr2=0x7f59540d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f5958939640 1 --2- 172.21.15.32:0/3768610781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595415cdb0 0x7f59540d0be0 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f5944009870 tx=0x7f594401ed30 comp rx=0 tx=0).stop 2023-12-20T04:44:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f5958939640 1 -- 172.21.15.32:0/3768610781 shutdown_connections 2023-12-20T04:44:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f5958939640 1 --2- 172.21.15.32:0/3768610781 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59540d2740 0x7f59540cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f5958939640 1 --2- 172.21.15.32:0/3768610781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f595415cdb0 0x7f59540d0be0 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f5958939640 1 -- 172.21.15.32:0/3768610781 >> 172.21.15.32:0/3768610781 conn(0x7f5954065ee0 msgr2=0x7f595406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f5958939640 1 -- 172.21.15.32:0/3768610781 shutdown_connections 2023-12-20T04:44:14.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.009+0000 7f5958939640 1 -- 172.21.15.32:0/3768610781 wait complete. 2023-12-20T04:44:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5958939640 1 Processor -- start 2023-12-20T04:44:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5958939640 1 -- start start 2023-12-20T04:44:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5958939640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59540d1120 0x7f59541f3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5958939640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59540d2740 0x7f595414c940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5958939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f595415cdb0 0x7f595414ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5958939640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595414d3c0 con 0x7f59540d2740 2023-12-20T04:44:14.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5958939640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f595414f610 con 0x7f59540d1120 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5958939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f595414f8f0 con 0x7f595415cdb0 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5951d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59540d2740 0x7f595414c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5951d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59540d2740 0x7f595414c940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50826/0 (socket says 172.21.15.32:50826) 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5951d9b640 1 -- 172.21.15.32:0/200837526 learned_addr learned my addr 172.21.15.32:0/200837526 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f595259c640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59540d1120 0x7f59541f3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5952d9d640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f595415cdb0 0x7f595414ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5951d9b640 1 -- 172.21.15.32:0/200837526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f595415cdb0 msgr2=0x7f595414ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5951d9b640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f595415cdb0 0x7f595414ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5951d9b640 1 -- 172.21.15.32:0/200837526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59540d1120 msgr2=0x7f59541f3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5951d9b640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59540d1120 0x7f59541f3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5951d9b640 1 -- 172.21.15.32:0/200837526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59440094d0 con 0x7f59540d2740 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f595259c640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59540d1120 0x7f59541f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:14.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5951d9b640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59540d2740 0x7f595414c940 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f594800e930 tx=0x7f594800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:14.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5942ffd640 1 -- 172.21.15.32:0/200837526 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5948009690 con 0x7f59540d2740 2023-12-20T04:44:14.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5942ffd640 1 -- 172.21.15.32:0/200837526 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f594800ca00 con 0x7f59540d2740 2023-12-20T04:44:14.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5942ffd640 1 -- 172.21.15.32:0/200837526 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f594800cd70 con 0x7f59540d2740 2023-12-20T04:44:14.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5958939640 1 -- 172.21.15.32:0/200837526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f595414fbd0 con 0x7f59540d2740 2023-12-20T04:44:14.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5958939640 1 -- 172.21.15.32:0/200837526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59540cb000 con 0x7f59540d2740 2023-12-20T04:44:14.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.013+0000 7f5958939640 1 -- 172.21.15.32:0/200837526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f595404e000 con 0x7f59540d2740 2023-12-20T04:44:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.017+0000 7f5942ffd640 1 -- 172.21.15.32:0/200837526 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 44) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f5948013470 con 0x7f59540d2740 2023-12-20T04:44:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.017+0000 7f5942ffd640 1 -- 172.21.15.32:0/200837526 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f59480830e0 con 0x7f59540d2740 2023-12-20T04:44:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.021+0000 7f5942ffd640 1 -- 172.21.15.32:0/200837526 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5948050d80 con 0x7f59540d2740 2023-12-20T04:44:14.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.081+0000 7f5942ffd640 1 -- 172.21.15.32:0/200837526 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 45) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f5948050830 con 0x7f59540d2740 2023-12-20T04:44:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.081+0000 7f5942ffd640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f593406b010 0x7f593406d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.085+0000 7f595259c640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f593406b010 0x7f593406d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.085+0000 7f595259c640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f593406b010 0x7f593406d400 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f59440b2000 tx=0x7f5944008040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:14.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 -- 172.21.15.32:0/200837526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f595406b8d0 con 0x7f59540d2740 2023-12-20T04:44:14.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5942ffd640 1 -- 172.21.15.32:0/200837526 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4699 (secure 0 0 0) 0x7f594806de40 con 0x7f59540d2740 2023-12-20T04:44:14.224 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:14.224 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/655215798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":655215798},{"type":"v1","addr":"172.21.15.32:6837","nonce":655215798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/661544180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":661544180},{"type":"v1","addr":"172.21.15.39:6834","nonce":661544180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6855,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/833127896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":833127896},{"type":"v1","addr":"172.21.15.32:6836","nonce":833127896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:54.874151+0000","modified":"2023-12-20T04:43:57.893768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6864":{"gid":6864,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/979895154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":979895154},{"type":"v1","addr":"172.21.15.39:6835","nonce":979895154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-20T04:44:14.225 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2023-12-20T04:44:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 -- 172.21.15.32:0/200837526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f593406b010 msgr2=0x7f593406d400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f593406b010 0x7f593406d400 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f59440b2000 tx=0x7f5944008040 comp rx=0 tx=0).stop 2023-12-20T04:44:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 -- 172.21.15.32:0/200837526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59540d2740 msgr2=0x7f595414c940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59540d2740 0x7f595414c940 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7f594800e930 tx=0x7f594800edf0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 -- 172.21.15.32:0/200837526 shutdown_connections 2023-12-20T04:44:14.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f593406b010 0x7f593406d400 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f595415cdb0 0x7f595414ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59540d2740 0x7f595414c940 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 --2- 172.21.15.32:0/200837526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59540d1120 0x7f59541f3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 -- 172.21.15.32:0/200837526 >> 172.21.15.32:0/200837526 conn(0x7f5954065ee0 msgr2=0x7f59540cdc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 -- 172.21.15.32:0/200837526 shutdown_connections 2023-12-20T04:44:14.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.225+0000 7f5958939640 1 -- 172.21.15.32:0/200837526 wait complete. 2023-12-20T04:44:14.246 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:44:14.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d8ce5640 1 Processor -- start 2023-12-20T04:44:14.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d8ce5640 1 -- start start 2023-12-20T04:44:14.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d8ce5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff8d415cdb0 0x7ff8d40d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d8ce5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8d40cfc80 0x7ff8d40b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d8ce5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8d415d190 con 0x7ff8d40d2740 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d8ce5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8d40d16a0 con 0x7ff8d415cdb0 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d8ce5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d40d1980 con 0x7ff8d40cfc80 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8cbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff8d40d2740 0x7ff8d40cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37908/0 (socket says 172.21.15.32:37908) 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8cbfff640 1 -- 172.21.15.32:0/3396669564 learned_addr learned my addr 172.21.15.32:0/3396669564 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d2d9d640 1 --2- 172.21.15.32:0/3396669564 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8d40cfc80 0x7ff8d40b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d259c640 1 --2- 172.21.15.32:0/3396669564 >> v2:172.21.15.39:3300/0 conn(0x7ff8d415cdb0 0x7ff8d40d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d1d9b640 1 -- 172.21.15.32:0/3396669564 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2885738172 0 0) 0x7ff8d415d190 con 0x7ff8d40d2740 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d1d9b640 1 -- 172.21.15.32:0/3396669564 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8c0003530 con 0x7ff8d40d2740 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d2d9d640 1 -- 172.21.15.32:0/3396669564 >> v2:172.21.15.39:3300/0 conn(0x7ff8d415cdb0 msgr2=0x7ff8d40d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d2d9d640 1 --2- 172.21.15.32:0/3396669564 >> v2:172.21.15.39:3300/0 conn(0x7ff8d415cdb0 0x7ff8d40d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d2d9d640 1 -- 172.21.15.32:0/3396669564 >> v1:172.21.15.32:6789/0 conn(0x7ff8d40d2740 legacy=0x7ff8d40cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.389+0000 7ff8d2d9d640 1 -- 172.21.15.32:0/3396669564 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8d40d1c60 con 0x7ff8d40cfc80 2023-12-20T04:44:14.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d2d9d640 1 --2- 172.21.15.32:0/3396669564 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8d40cfc80 0x7ff8d40b5c90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009d90 tx=0x7ff8cc01f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdfdfcd963d23a0a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:14.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d1d9b640 1 -- 172.21.15.32:0/3396669564 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc0af600 con 0x7ff8d40cfc80 2023-12-20T04:44:14.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d1d9b640 1 -- 172.21.15.32:0/3396669564 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8cc0ab050 con 0x7ff8d40cfc80 2023-12-20T04:44:14.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d1d9b640 1 -- 172.21.15.32:0/3396669564 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8cc0af960 con 0x7ff8d40cfc80 2023-12-20T04:44:14.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/3396669564 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8d40cfc80 msgr2=0x7ff8d40b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d8ce5640 1 --2- 172.21.15.32:0/3396669564 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8d40cfc80 0x7ff8d40b5c90 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7ff8cc009d90 tx=0x7ff8cc01f130 comp rx=0 tx=0).stop 2023-12-20T04:44:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/3396669564 shutdown_connections 2023-12-20T04:44:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d8ce5640 1 --2- 172.21.15.32:0/3396669564 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8d40cfc80 0x7ff8d40b5c90 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d8ce5640 1 --2- 172.21.15.32:0/3396669564 >> v2:172.21.15.39:3300/0 conn(0x7ff8d415cdb0 0x7ff8d40d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/3396669564 >> 172.21.15.32:0/3396669564 conn(0x7ff8d4065ee0 msgr2=0x7ff8d406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/3396669564 shutdown_connections 2023-12-20T04:44:14.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.393+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/3396669564 wait complete. 2023-12-20T04:44:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d8ce5640 1 Processor -- start 2023-12-20T04:44:14.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d8ce5640 1 -- start start 2023-12-20T04:44:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d8ce5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8d40cfc80 0x7ff8d4150fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d8ce5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8d40d2740 0x7ff8d41541c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d8ce5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d415cdb0 0x7ff8d4154700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d8ce5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d415e2c0 con 0x7ff8d40d2740 2023-12-20T04:44:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d8ce5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d415d3a0 con 0x7ff8d415cdb0 2023-12-20T04:44:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d8ce5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8d415d640 con 0x7ff8d40cfc80 2023-12-20T04:44:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8d40d2740 0x7ff8d41541c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8d40d2740 0x7ff8d41541c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50832/0 (socket says 172.21.15.32:50832) 2023-12-20T04:44:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8cbfff640 1 -- 172.21.15.32:0/1685930127 learned_addr learned my addr 172.21.15.32:0/1685930127 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:14.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d2d9d640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d415cdb0 0x7ff8d4154700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d259c640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8d40cfc80 0x7ff8d4150fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8cbfff640 1 -- 172.21.15.32:0/1685930127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8d40cfc80 msgr2=0x7ff8d4150fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8cbfff640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8d40cfc80 0x7ff8d4150fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8cbfff640 1 -- 172.21.15.32:0/1685930127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d415cdb0 msgr2=0x7ff8d4154700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8cbfff640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d415cdb0 0x7ff8d4154700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8cbfff640 1 -- 172.21.15.32:0/1685930127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc009a70 con 0x7ff8d40d2740 2023-12-20T04:44:14.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8cbfff640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8d40d2740 0x7ff8d41541c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00c450 tx=0x7ff8bc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:14.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8caffd640 1 -- 172.21.15.32:0/1685930127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc005cf0 con 0x7ff8d40d2740 2023-12-20T04:44:14.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8caffd640 1 -- 172.21.15.32:0/1685930127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc005e50 con 0x7ff8d40d2740 2023-12-20T04:44:14.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8caffd640 1 -- 172.21.15.32:0/1685930127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc007290 con 0x7ff8d40d2740 2023-12-20T04:44:14.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/1685930127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8d415d920 con 0x7ff8d40d2740 2023-12-20T04:44:14.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.397+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/1685930127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8d415dee0 con 0x7ff8d40d2740 2023-12-20T04:44:14.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.401+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/1685930127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8d415e400 con 0x7ff8d40d2740 2023-12-20T04:44:14.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.401+0000 7ff8caffd640 1 -- 172.21.15.32:0/1685930127 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 45) v1 ==== 85066+0+0 (secure 0 0 0) 0x7ff8bc0073f0 con 0x7ff8d40d2740 2023-12-20T04:44:14.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.401+0000 7ff8caffd640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff8b40658b0 0x7ff8b4067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.401+0000 7ff8caffd640 1 -- 172.21.15.32:0/1685930127 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6268+0+0 (secure 0 0 0) 0x7ff8bc0824f0 con 0x7ff8d40d2740 2023-12-20T04:44:14.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.401+0000 7ff8d259c640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff8b40658b0 0x7ff8b4067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.401+0000 7ff8d259c640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff8b40658b0 0x7ff8b4067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009ac0 tx=0x7ff8c4009190 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:14.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.405+0000 7ff8caffd640 1 -- 172.21.15.32:0/1685930127 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff8d415e400 con 0x7ff8d40d2740 2023-12-20T04:44:14.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/1685930127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff8d415e400 con 0x7ff8d40d2740 2023-12-20T04:44:14.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8caffd640 1 -- 172.21.15.32:0/1685930127 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 39 v39) v1 ==== 94+0+4699 (secure 0 0 0) 0x7ff8d415e400 con 0x7ff8d40d2740 2023-12-20T04:44:14.597 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:14.597 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":39,"default_fscid":5,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/655215798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":655215798},{"type":"v1","addr":"172.21.15.32:6837","nonce":655215798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/661544180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":661544180},{"type":"v1","addr":"172.21.15.39:6834","nonce":661544180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6855,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/833127896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":833127896},{"type":"v1","addr":"172.21.15.32:6836","nonce":833127896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34}],"filesystems":[{"mdsmap":{"epoch":39,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:43:54.874151+0000","modified":"2023-12-20T04:43:57.893768+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":6864},"failed":[],"damaged":[],"stopped":[],"info":{"gid_6864":{"gid":6864,"name":"d","rank":0,"incarnation":35,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6835/979895154","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":979895154},{"type":"v1","addr":"172.21.15.39:6835","nonce":979895154}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[11],"metadata_pool":10,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":5}]} 2023-12-20T04:44:14.597 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 39 2023-12-20T04:44:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/1685930127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff8b40658b0 msgr2=0x7ff8b4067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff8b40658b0 0x7ff8b4067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7ff8c4009ac0 tx=0x7ff8c4009190 comp rx=0 tx=0).stop 2023-12-20T04:44:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/1685930127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8d40d2740 msgr2=0x7ff8d41541c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8d40d2740 0x7ff8d41541c0 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc00c450 tx=0x7ff8bc00c910 comp rx=0 tx=0).stop 2023-12-20T04:44:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/1685930127 shutdown_connections 2023-12-20T04:44:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff8b40658b0 0x7ff8b4067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8d415cdb0 0x7ff8d4154700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8d40d2740 0x7ff8d41541c0 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 --2- 172.21.15.32:0/1685930127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8d40cfc80 0x7ff8d4150fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/1685930127 >> 172.21.15.32:0/1685930127 conn(0x7ff8d4065ee0 msgr2=0x7ff8d40d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/1685930127 shutdown_connections 2023-12-20T04:44:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.597+0000 7ff8d8ce5640 1 -- 172.21.15.32:0/1685930127 wait complete. 2023-12-20T04:44:14.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:44:14.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.761+0000 7efe91d6d640 1 Processor -- start 2023-12-20T04:44:14.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.761+0000 7efe91d6d640 1 -- start start 2023-12-20T04:44:14.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe91d6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe8c1599e0 0x7efe8c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe91d6d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe8c0b0d00 0x7efe8c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe91d6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe8c0c98c0 con 0x7efe8c0c90e0 2023-12-20T04:44:14.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe91d6d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe8c0c9ba0 con 0x7efe8c0b0d00 2023-12-20T04:44:14.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe91d6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c0c9e80 con 0x7efe8c1599e0 2023-12-20T04:44:14.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7efe8c0c90e0 0x7efe8c0b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37920/0 (socket says 172.21.15.32:37920) 2023-12-20T04:44:14.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8affd640 1 -- 172.21.15.32:0/2140367850 learned_addr learned my addr 172.21.15.32:0/2140367850 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8bfff640 1 --2- 172.21.15.32:0/2140367850 >> v2:172.21.15.32:3300/0 conn(0x7efe8c0b0d00 0x7efe8c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8b7fe640 1 --2- 172.21.15.32:0/2140367850 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe8c1599e0 0x7efe8c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8a7fc640 1 -- 172.21.15.32:0/2140367850 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1972575593 0 0) 0x7efe8c0c98c0 con 0x7efe8c0c90e0 2023-12-20T04:44:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8a7fc640 1 -- 172.21.15.32:0/2140367850 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7efe74003530 con 0x7efe8c0c90e0 2023-12-20T04:44:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8bfff640 1 -- 172.21.15.32:0/2140367850 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe8c1599e0 msgr2=0x7efe8c0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8bfff640 1 --2- 172.21.15.32:0/2140367850 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe8c1599e0 0x7efe8c0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8bfff640 1 -- 172.21.15.32:0/2140367850 >> v1:172.21.15.32:6789/0 conn(0x7efe8c0c90e0 legacy=0x7efe8c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8bfff640 1 -- 172.21.15.32:0/2140367850 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe8c0ca160 con 0x7efe8c0b0d00 2023-12-20T04:44:14.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8b7fe640 1 --2- 172.21.15.32:0/2140367850 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe8c1599e0 0x7efe8c0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.765+0000 7efe8bfff640 1 --2- 172.21.15.32:0/2140367850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe8c0b0d00 0x7efe8c0701c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7efe84009d90 tx=0x7efe8401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=739bf5f9b9f5c719 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe8a7fc640 1 -- 172.21.15.32:0/2140367850 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840a5970 con 0x7efe8c0b0d00 2023-12-20T04:44:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe8a7fc640 1 -- 172.21.15.32:0/2140367850 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe840a5ad0 con 0x7efe8c0b0d00 2023-12-20T04:44:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe8a7fc640 1 -- 172.21.15.32:0/2140367850 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840a5e60 con 0x7efe8c0b0d00 2023-12-20T04:44:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 -- 172.21.15.32:0/2140367850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe8c0b0d00 msgr2=0x7efe8c0701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 --2- 172.21.15.32:0/2140367850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe8c0b0d00 0x7efe8c0701c0 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7efe84009d90 tx=0x7efe8401c890 comp rx=0 tx=0).stop 2023-12-20T04:44:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 -- 172.21.15.32:0/2140367850 shutdown_connections 2023-12-20T04:44:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 --2- 172.21.15.32:0/2140367850 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe8c0b0d00 0x7efe8c0701c0 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 --2- 172.21.15.32:0/2140367850 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe8c1599e0 0x7efe8c0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 -- 172.21.15.32:0/2140367850 >> 172.21.15.32:0/2140367850 conn(0x7efe8c06a320 msgr2=0x7efe8c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:14.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 -- 172.21.15.32:0/2140367850 shutdown_connections 2023-12-20T04:44:14.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 -- 172.21.15.32:0/2140367850 wait complete. 2023-12-20T04:44:14.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 Processor -- start 2023-12-20T04:44:14.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 -- start start 2023-12-20T04:44:14.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe8c0b0d00 0x7efe8c149560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c0c90e0 0x7efe8c0c4b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe8c1599e0 0x7efe8c0cda70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c0ccf70 con 0x7efe8c1599e0 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c0c50c0 con 0x7efe8c0c90e0 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe91d6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe8c0cc9b0 con 0x7efe8c0b0d00 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe8bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe8c1599e0 0x7efe8c0cda70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe8bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe8c1599e0 0x7efe8c0cda70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50846/0 (socket says 172.21.15.32:50846) 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.769+0000 7efe8bfff640 1 -- 172.21.15.32:0/3218682086 learned_addr learned my addr 172.21.15.32:0/3218682086 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe8b7fe640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe8c0b0d00 0x7efe8c149560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe8affd640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c0c90e0 0x7efe8c0c4b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe8bfff640 1 -- 172.21.15.32:0/3218682086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe8c0b0d00 msgr2=0x7efe8c149560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe8bfff640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe8c0b0d00 0x7efe8c149560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe8bfff640 1 -- 172.21.15.32:0/3218682086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c0c90e0 msgr2=0x7efe8c0c4b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe8bfff640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c0c90e0 0x7efe8c0c4b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe8bfff640 1 -- 172.21.15.32:0/3218682086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe84009a70 con 0x7efe8c1599e0 2023-12-20T04:44:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe8bfff640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe8c1599e0 0x7efe8c0cda70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7efe8401af40 tx=0x7efe84019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe73fff640 1 -- 172.21.15.32:0/3218682086 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840add90 con 0x7efe8c1599e0 2023-12-20T04:44:14.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe73fff640 1 -- 172.21.15.32:0/3218682086 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efe84009450 con 0x7efe8c1599e0 2023-12-20T04:44:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe91d6d640 1 -- 172.21.15.32:0/3218682086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe8c0ccc90 con 0x7efe8c1599e0 2023-12-20T04:44:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe73fff640 1 -- 172.21.15.32:0/3218682086 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efe840b9670 con 0x7efe8c1599e0 2023-12-20T04:44:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe91d6d640 1 -- 172.21.15.32:0/3218682086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe8c0cd550 con 0x7efe8c1599e0 2023-12-20T04:44:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe91d6d640 1 -- 172.21.15.32:0/3218682086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe8c04e000 con 0x7efe8c1599e0 2023-12-20T04:44:14.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.773+0000 7efe73fff640 1 -- 172.21.15.32:0/3218682086 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 45) v1 ==== 85066+0+0 (secure 0 0 0) 0x7efe840b97d0 con 0x7efe8c1599e0 2023-12-20T04:44:14.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.777+0000 7efe73fff640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe780658e0 0x7efe78067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:14.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.777+0000 7efe73fff640 1 -- 172.21.15.32:0/3218682086 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6268+0+0 (secure 0 0 0) 0x7efe8411e500 con 0x7efe8c1599e0 2023-12-20T04:44:14.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.777+0000 7efe8b7fe640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe780658e0 0x7efe78067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:14.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.777+0000 7efe8b7fe640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe780658e0 0x7efe78067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0cc780 tx=0x7efe7c008040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.781+0000 7efe73fff640 1 -- 172.21.15.32:0/3218682086 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efe840ec080 con 0x7efe8c1599e0 2023-12-20T04:44:14.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.941+0000 7efe91d6d640 1 -- 172.21.15.32:0/3218682086 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7efe8c164850 con 0x7efe8c1599e0 2023-12-20T04:44:14.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.941+0000 7efe73fff640 1 -- 172.21.15.32:0/3218682086 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v95) v1 ==== 74+0+16853 (secure 0 0 0) 0x7efe840ef6a0 con 0x7efe8c1599e0 2023-12-20T04:44:14.942 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:14.942 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":95,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:44:13.074292+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":10,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:43:52.582628+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"92","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":11,"pool_name":"cephfs_data","create_time":"2023-12-20T04:43:53.871341+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"93","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2756989681":"2023-12-21T04:44:13.074258+0000","172.21.15.32:0/1156261356":"2023-12-21T04:44:13.074258+0000","172.21.15.32:0/3141888936":"2023-12-21T04:44:13.074258+0000","172.21.15.32:0/4256985996":"2023-12-21T04:44:13.074258+0000","172.21.15.32:0/3308967759":"2023-12-21T04:44:13.074258+0000","172.21.15.32:0/2058302965":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/1852394479":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/1659600879":"2023-12-21T04:44:13.074258+0000","172.21.15.32:6833/38009":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/2860021829":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/3370166328":"2023-12-21T04:44:06.958717+0000","172.21.15.32:6832/38009":"2023-12-21T04:44:06.958717+0000","172.21.15.32:0/2187917072":"2023-12-21T04:44:06.958717+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 -- 172.21.15.32:0/3218682086 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe780658e0 msgr2=0x7efe78067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe780658e0 0x7efe78067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7efe8c0cc780 tx=0x7efe7c008040 comp rx=0 tx=0).stop 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 -- 172.21.15.32:0/3218682086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe8c1599e0 msgr2=0x7efe8c0cda70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe8c1599e0 0x7efe8c0cda70 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7efe8401af40 tx=0x7efe84019630 comp rx=0 tx=0).stop 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 -- 172.21.15.32:0/3218682086 shutdown_connections 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efe780658e0 0x7efe78067da0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe8c1599e0 0x7efe8c0cda70 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe8c0c90e0 0x7efe8c0c4b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 --2- 172.21.15.32:0/3218682086 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe8c0b0d00 0x7efe8c149560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 -- 172.21.15.32:0/3218682086 >> 172.21.15.32:0/3218682086 conn(0x7efe8c06a320 msgr2=0x7efe8c153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 -- 172.21.15.32:0/3218682086 shutdown_connections 2023-12-20T04:44:14.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:14.945+0000 7efe91d6d640 1 -- 172.21.15.32:0/3218682086 wait complete. 2023-12-20T04:44:14.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T04:44:15.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97d63f3640 1 Processor -- start 2023-12-20T04:44:15.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97d63f3640 1 -- start start 2023-12-20T04:44:15.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97d63f3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97d00af5e0 0x7f97d00af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:15.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97d63f3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d015b990 con 0x7f97d015f110 2023-12-20T04:44:15.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97d63f3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97d015bbd0 con 0x7f97d006ce60 2023-12-20T04:44:15.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97d63f3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97d015beb0 con 0x7f97d00af5e0 2023-12-20T04:44:15.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97cffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f97d00af5e0 0x7f97d00af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:15.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97d4990640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f97d015f110 0x7f97d015c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37932/0 (socket says 172.21.15.32:37932) 2023-12-20T04:44:15.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97d4990640 1 -- 172.21.15.32:0/1219981063 learned_addr learned my addr 172.21.15.32:0/1219981063 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:15.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97cffff640 1 -- 172.21.15.32:0/1219981063 >> v1:172.21.15.39:6789/0 conn(0x7f97d006ce60 legacy=0x7f97d015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97cffff640 1 -- 172.21.15.32:0/1219981063 >> v1:172.21.15.32:6789/0 conn(0x7f97d015f110 legacy=0x7f97d015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97cffff640 1 -- 172.21.15.32:0/1219981063 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97d015e910 con 0x7f97d00af5e0 2023-12-20T04:44:15.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.117+0000 7f97ceffd640 1 -- 172.21.15.32:0/1219981063 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2189630382 0 0) 0x7f97c80014a0 con 0x7f97d015f110 2023-12-20T04:44:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.121+0000 7f97cffff640 1 --2- 172.21.15.32:0/1219981063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97d00af5e0 0x7f97d00af9c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f97c00097a0 tx=0x7f97c001c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f2512a85944f5fd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.121+0000 7f97ceffd640 1 -- 172.21.15.32:0/1219981063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00ab650 con 0x7f97d00af5e0 2023-12-20T04:44:15.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.121+0000 7f97ceffd640 1 -- 172.21.15.32:0/1219981063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c00ab7b0 con 0x7f97d00af5e0 2023-12-20T04:44:15.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.121+0000 7f97ceffd640 1 -- 172.21.15.32:0/1219981063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c00abb40 con 0x7f97d00af5e0 2023-12-20T04:44:15.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.121+0000 7f97d63f3640 1 -- 172.21.15.32:0/1219981063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97d00af5e0 msgr2=0x7f97d00af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.121+0000 7f97d63f3640 1 --2- 172.21.15.32:0/1219981063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97d00af5e0 0x7f97d00af9c0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f97c00097a0 tx=0x7f97c001c430 comp rx=0 tx=0).stop 2023-12-20T04:44:15.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.121+0000 7f97d63f3640 1 -- 172.21.15.32:0/1219981063 shutdown_connections 2023-12-20T04:44:15.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.121+0000 7f97d63f3640 1 --2- 172.21.15.32:0/1219981063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97d00af5e0 0x7f97d00af9c0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.121+0000 7f97d63f3640 1 -- 172.21.15.32:0/1219981063 >> 172.21.15.32:0/1219981063 conn(0x7f97d006a320 msgr2=0x7f97d00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:15.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.121+0000 7f97d63f3640 1 -- 172.21.15.32:0/1219981063 shutdown_connections 2023-12-20T04:44:15.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.121+0000 7f97d63f3640 1 -- 172.21.15.32:0/1219981063 wait complete. 2023-12-20T04:44:15.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d63f3640 1 Processor -- start 2023-12-20T04:44:15.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d63f3640 1 -- start start 2023-12-20T04:44:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d63f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97d006ce60 0x7f97d00b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d63f3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97d00af5e0 0x7f97d0150f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d63f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97d015f110 0x7f97d0158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d63f3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d00b20b0 con 0x7f97d00af5e0 2023-12-20T04:44:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d63f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97d00ce150 con 0x7f97d006ce60 2023-12-20T04:44:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d63f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97d00cc700 con 0x7f97d015f110 2023-12-20T04:44:15.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97cf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97d00af5e0 0x7f97d0150f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97cf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97d00af5e0 0x7f97d0150f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50874/0 (socket says 172.21.15.32:50874) 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97cf7fe640 1 -- 172.21.15.32:0/1960949288 learned_addr learned my addr 172.21.15.32:0/1960949288 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97cffff640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97d006ce60 0x7f97d00b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d4990640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97d015f110 0x7f97d0158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97cf7fe640 1 -- 172.21.15.32:0/1960949288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97d015f110 msgr2=0x7f97d0158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97cf7fe640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97d015f110 0x7f97d0158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97cf7fe640 1 -- 172.21.15.32:0/1960949288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97d006ce60 msgr2=0x7f97d00b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97cf7fe640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97d006ce60 0x7f97d00b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97cf7fe640 1 -- 172.21.15.32:0/1960949288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97c00093b0 con 0x7f97d00af5e0 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d4990640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97d015f110 0x7f97d0158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97cf7fe640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97d00af5e0 0x7f97d0150f10 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f97c400c500 tx=0x7f97c400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:15.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97ccff9640 1 -- 172.21.15.32:0/1960949288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4005220 con 0x7f97d00af5e0 2023-12-20T04:44:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97ccff9640 1 -- 172.21.15.32:0/1960949288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97c4005380 con 0x7f97d00af5e0 2023-12-20T04:44:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d63f3640 1 -- 172.21.15.32:0/1960949288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f97d00cc9e0 con 0x7f97d00af5e0 2023-12-20T04:44:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97ccff9640 1 -- 172.21.15.32:0/1960949288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97c4007270 con 0x7f97d00af5e0 2023-12-20T04:44:15.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.125+0000 7f97d63f3640 1 -- 172.21.15.32:0/1960949288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97d0071fa0 con 0x7f97d00af5e0 2023-12-20T04:44:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.129+0000 7f97d63f3640 1 -- 172.21.15.32:0/1960949288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97d00ccf00 con 0x7f97d00af5e0 2023-12-20T04:44:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.129+0000 7f97ccff9640 1 -- 172.21.15.32:0/1960949288 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 46) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f97c40073d0 con 0x7f97d00af5e0 2023-12-20T04:44:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.129+0000 7f97ccff9640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f97bc065af0 0x7f97bc067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.129+0000 7f97ccff9640 1 -- 172.21.15.32:0/1960949288 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(95..95 src has 1..95) v4 ==== 6268+0+0 (secure 0 0 0) 0x7f97c4082d20 con 0x7f97d00af5e0 2023-12-20T04:44:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.129+0000 7f97cffff640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f97bc065af0 0x7f97bc067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:15.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.129+0000 7f97cffff640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f97bc065af0 0x7f97bc067fb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f97c00af000 tx=0x7f97c00a7000 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:15.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.133+0000 7f97ccff9640 1 -- 172.21.15.32:0/1960949288 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f97d00ccf00 con 0x7f97d00af5e0 2023-12-20T04:44:15.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.325+0000 7f97d63f3640 1 -- 172.21.15.32:0/1960949288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f97d00ccf00 con 0x7f97d00af5e0 2023-12-20T04:44:15.327 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T04:44:15.325+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T04:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.341+0000 7f97ccff9640 1 -- 172.21.15.32:0/1960949288 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v40) v1 ==== 161+0+0 (secure 0 0 0) 0x7f97d00ccf00 con 0x7f97d00af5e0 2023-12-20T04:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 -- 172.21.15.32:0/1960949288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f97bc065af0 msgr2=0x7f97bc067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f97bc065af0 0x7f97bc067fb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f97c00af000 tx=0x7f97c00a7000 comp rx=0 tx=0).stop 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 -- 172.21.15.32:0/1960949288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97d00af5e0 msgr2=0x7f97d0150f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97d00af5e0 0x7f97d0150f10 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7f97c400c500 tx=0x7f97c400c9c0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 -- 172.21.15.32:0/1960949288 shutdown_connections 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f97bc065af0 0x7f97bc067fb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97d015f110 0x7f97d0158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97d00af5e0 0x7f97d0150f10 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 --2- 172.21.15.32:0/1960949288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97d006ce60 0x7f97d00b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 -- 172.21.15.32:0/1960949288 >> 172.21.15.32:0/1960949288 conn(0x7f97d006a320 msgr2=0x7f97d00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 -- 172.21.15.32:0/1960949288 shutdown_connections 2023-12-20T04:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.345+0000 7f97d63f3640 1 -- 172.21.15.32:0/1960949288 wait complete. 2023-12-20T04:44:15.350 INFO:tasks.ceph.mds.d.smithi039.stderr: -2496> 2023-12-20T04:44:00.435+0000 7f3593264640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:44:15.365 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T04:44:15.400 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T04:44:15.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.513+0000 7f899b59e640 1 Processor -- start 2023-12-20T04:44:15.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.513+0000 7f899b59e640 1 -- start start 2023-12-20T04:44:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.513+0000 7f899b59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899c0cfdd0 0x7f899c15d140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.513+0000 7f899b59e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899c15abc0 con 0x7f899c0b2240 2023-12-20T04:44:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.513+0000 7f899b59e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f899c160780 con 0x7f899c160f40 2023-12-20T04:44:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.513+0000 7f899b59e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c06bc80 con 0x7f899c0cfdd0 2023-12-20T04:44:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.517+0000 7f899a59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f899c0b2240 0x7f899c0b2620 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37938/0 (socket says 172.21.15.32:37938) 2023-12-20T04:44:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.517+0000 7f899a59c640 1 -- 172.21.15.32:0/704057517 learned_addr learned my addr 172.21.15.32:0/704057517 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.517+0000 7f8993fff640 1 --2- 172.21.15.32:0/704057517 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899c0cfdd0 0x7f899c15d140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.517+0000 7f8999d9b640 1 -- 172.21.15.32:0/704057517 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625067365 0 0) 0x7f899c15abc0 con 0x7f899c0b2240 2023-12-20T04:44:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.517+0000 7f8999d9b640 1 -- 172.21.15.32:0/704057517 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8988003530 con 0x7f899c0b2240 2023-12-20T04:44:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.517+0000 7f8999d9b640 1 -- 172.21.15.32:0/704057517 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1721200715 0 0) 0x7f899c160780 con 0x7f899c160f40 2023-12-20T04:44:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.517+0000 7f8999d9b640 1 -- 172.21.15.32:0/704057517 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f899c15abc0 con 0x7f899c160f40 2023-12-20T04:44:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.517+0000 7f8993fff640 1 -- 172.21.15.32:0/704057517 >> v1:172.21.15.39:6789/0 conn(0x7f899c160f40 legacy=0x7f899c0ce950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.517+0000 7f8993fff640 1 -- 172.21.15.32:0/704057517 >> v1:172.21.15.32:6789/0 conn(0x7f899c0b2240 legacy=0x7f899c0b2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.517+0000 7f8993fff640 1 -- 172.21.15.32:0/704057517 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f899c15da50 con 0x7f899c0cfdd0 2023-12-20T04:44:15.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.517+0000 7f8993fff640 1 --2- 172.21.15.32:0/704057517 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899c0cfdd0 0x7f899c15d140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f89840097f0 tx=0x7f898401ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5182389d329e91f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f8999d9b640 1 -- 172.21.15.32:0/704057517 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89840a89b0 con 0x7f899c0cfdd0 2023-12-20T04:44:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f8999d9b640 1 -- 172.21.15.32:0/704057517 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89840a8b10 con 0x7f899c0cfdd0 2023-12-20T04:44:15.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f8999d9b640 1 -- 172.21.15.32:0/704057517 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89840a8ea0 con 0x7f899c0cfdd0 2023-12-20T04:44:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 -- 172.21.15.32:0/704057517 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899c0cfdd0 msgr2=0x7f899c15d140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 --2- 172.21.15.32:0/704057517 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899c0cfdd0 0x7f899c15d140 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f89840097f0 tx=0x7f898401ecb0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 -- 172.21.15.32:0/704057517 shutdown_connections 2023-12-20T04:44:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 --2- 172.21.15.32:0/704057517 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899c0cfdd0 0x7f899c15d140 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 -- 172.21.15.32:0/704057517 >> 172.21.15.32:0/704057517 conn(0x7f899c0cd240 msgr2=0x7f899c0cd650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 -- 172.21.15.32:0/704057517 shutdown_connections 2023-12-20T04:44:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 -- 172.21.15.32:0/704057517 wait complete. 2023-12-20T04:44:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 Processor -- start 2023-12-20T04:44:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 -- start start 2023-12-20T04:44:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899c0b2240 0x7f899c1f19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c0cfdd0 0x7f899c15f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c160f40 0x7f899c15fc10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c157c80 con 0x7f899c160f40 2023-12-20T04:44:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f899c157f60 con 0x7f899c0cfdd0 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899b59e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f899c158240 con 0x7f899c0b2240 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c160f40 0x7f899c15fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c160f40 0x7f899c15fc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50912/0 (socket says 172.21.15.32:50912) 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899ad9d640 1 -- 172.21.15.32:0/2112456037 learned_addr learned my addr 172.21.15.32:0/2112456037 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f8993fff640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c0cfdd0 0x7f899c15f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.521+0000 7f899a59c640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899c0b2240 0x7f899c1f19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f899ad9d640 1 -- 172.21.15.32:0/2112456037 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899c0b2240 msgr2=0x7f899c1f19a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f899ad9d640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899c0b2240 0x7f899c1f19a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f899ad9d640 1 -- 172.21.15.32:0/2112456037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c0cfdd0 msgr2=0x7f899c15f6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f899ad9d640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c0cfdd0 0x7f899c15f6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f899ad9d640 1 -- 172.21.15.32:0/2112456037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8984009420 con 0x7f899c160f40 2023-12-20T04:44:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f899ad9d640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c160f40 0x7f899c15fc10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f899400c730 tx=0x7f899400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f8992ffd640 1 -- 172.21.15.32:0/2112456037 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8994003b80 con 0x7f899c160f40 2023-12-20T04:44:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f8992ffd640 1 -- 172.21.15.32:0/2112456037 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8994003ce0 con 0x7f899c160f40 2023-12-20T04:44:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f8992ffd640 1 -- 172.21.15.32:0/2112456037 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f899400aa90 con 0x7f899c160f40 2023-12-20T04:44:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f899b59e640 1 -- 172.21.15.32:0/2112456037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f899c158520 con 0x7f899c160f40 2023-12-20T04:44:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f899b59e640 1 -- 172.21.15.32:0/2112456037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f899c158a40 con 0x7f899c160f40 2023-12-20T04:44:15.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f899b59e640 1 -- 172.21.15.32:0/2112456037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f899c158710 con 0x7f899c160f40 2023-12-20T04:44:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f8992ffd640 1 -- 172.21.15.32:0/2112456037 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 46) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f899400abf0 con 0x7f899c160f40 2023-12-20T04:44:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f8992ffd640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065af0 0x7f897c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.525+0000 7f8992ffd640 1 -- 172.21.15.32:0/2112456037 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f8994083170 con 0x7f899c160f40 2023-12-20T04:44:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.529+0000 7f899a59c640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065af0 0x7f897c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:15.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.529+0000 7f899a59c640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065af0 0x7f897c067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f898c0011f0 tx=0x7f898c007d40 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.533+0000 7f8992ffd640 1 -- 172.21.15.32:0/2112456037 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8994050940 con 0x7f899c160f40 2023-12-20T04:44:15.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:15.721+0000 7f899b59e640 1 -- 172.21.15.32:0/2112456037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f899c157dc0 con 0x7f899c160f40 2023-12-20T04:44:16.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.345+0000 7f8992ffd640 1 -- 172.21.15.32:0/2112456037 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v41) v1 ==== 104+0+0 (secure 0 0 0) 0x7f899c157dc0 con 0x7f899c160f40 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 -- 172.21.15.32:0/2112456037 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065af0 msgr2=0x7f897c067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065af0 0x7f897c067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f898c0011f0 tx=0x7f898c007d40 comp rx=0 tx=0).stop 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 -- 172.21.15.32:0/2112456037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c160f40 msgr2=0x7f899c15fc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c160f40 0x7f899c15fc10 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7f899400c730 tx=0x7f899400cbf0 comp rx=0 tx=0).stop 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 -- 172.21.15.32:0/2112456037 shutdown_connections 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f897c065af0 0x7f897c067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f899c160f40 0x7f899c15fc10 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f899c0cfdd0 0x7f899c15f6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 --2- 172.21.15.32:0/2112456037 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f899c0b2240 0x7f899c1f19a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 -- 172.21.15.32:0/2112456037 >> 172.21.15.32:0/2112456037 conn(0x7f899c0cd240 msgr2=0x7f899c15b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 -- 172.21.15.32:0/2112456037 shutdown_connections 2023-12-20T04:44:16.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.349+0000 7f899b59e640 1 -- 172.21.15.32:0/2112456037 wait complete. 2023-12-20T04:44:16.371 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T04:44:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e89bc640 1 Processor -- start 2023-12-20T04:44:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e89bc640 1 -- start start 2023-12-20T04:44:16.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e89bc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f03e4070940 0x7f03e40ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:16.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e89bc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e406e100 con 0x7f03e406dcb0 2023-12-20T04:44:16.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e89bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f03e40ca590 con 0x7f03e4158520 2023-12-20T04:44:16.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e89bc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f03e40ca7f0 con 0x7f03e4070940 2023-12-20T04:44:16.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e1d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f03e406dcb0 0x7f03e4070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37946/0 (socket says 172.21.15.32:37946) 2023-12-20T04:44:16.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e1d9b640 1 -- 172.21.15.32:0/2767070570 learned_addr learned my addr 172.21.15.32:0/2767070570 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:16.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e2d9d640 1 --2- 172.21.15.32:0/2767070570 >> v2:172.21.15.32:3300/0 conn(0x7f03e4070940 0x7f03e40ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:16.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e2d9d640 1 -- 172.21.15.32:0/2767070570 >> v1:172.21.15.39:6789/0 conn(0x7f03e4158520 legacy=0x7f03e415ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:16.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e159a640 1 -- 172.21.15.32:0/2767070570 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2037808330 0 0) 0x7f03d8001120 con 0x7f03e406dcb0 2023-12-20T04:44:16.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e2d9d640 1 -- 172.21.15.32:0/2767070570 >> v1:172.21.15.32:6789/0 conn(0x7f03e406dcb0 legacy=0x7f03e4070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:16.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.525+0000 7f03e2d9d640 1 -- 172.21.15.32:0/2767070570 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03e40caad0 con 0x7f03e4070940 2023-12-20T04:44:16.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.529+0000 7f03e2d9d640 1 --2- 172.21.15.32:0/2767070570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03e4070940 0x7f03e40ca050 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f03e40ca720 tx=0x7f03dc01c870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad1a529701e1e366 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:16.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.529+0000 7f03e159a640 1 -- 172.21.15.32:0/2767070570 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0a5930 con 0x7f03e4070940 2023-12-20T04:44:16.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.529+0000 7f03e159a640 1 -- 172.21.15.32:0/2767070570 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc0a5a90 con 0x7f03e4070940 2023-12-20T04:44:16.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.529+0000 7f03e159a640 1 -- 172.21.15.32:0/2767070570 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0a5e20 con 0x7f03e4070940 2023-12-20T04:44:16.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.529+0000 7f03e89bc640 1 -- 172.21.15.32:0/2767070570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03e4070940 msgr2=0x7f03e40ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:16.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.529+0000 7f03e89bc640 1 --2- 172.21.15.32:0/2767070570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03e4070940 0x7f03e40ca050 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7f03e40ca720 tx=0x7f03dc01c870 comp rx=0 tx=0).stop 2023-12-20T04:44:16.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.529+0000 7f03e89bc640 1 -- 172.21.15.32:0/2767070570 shutdown_connections 2023-12-20T04:44:16.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.529+0000 7f03e89bc640 1 --2- 172.21.15.32:0/2767070570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03e4070940 0x7f03e40ca050 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:16.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.529+0000 7f03e89bc640 1 -- 172.21.15.32:0/2767070570 >> 172.21.15.32:0/2767070570 conn(0x7f03e406a510 msgr2=0x7f03e40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:16.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.529+0000 7f03e89bc640 1 -- 172.21.15.32:0/2767070570 shutdown_connections 2023-12-20T04:44:16.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.529+0000 7f03e89bc640 1 -- 172.21.15.32:0/2767070570 wait complete. 2023-12-20T04:44:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e89bc640 1 Processor -- start 2023-12-20T04:44:16.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e89bc640 1 -- start start 2023-12-20T04:44:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e89bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e406dcb0 0x7f03e414dac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e89bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f03e4070940 0x7f03e4156430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e89bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03e4158520 0x7f03e414ffd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e89bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e4157cd0 con 0x7f03e4158520 2023-12-20T04:44:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e89bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f03e414e000 con 0x7f03e406dcb0 2023-12-20T04:44:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e89bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f03e4156970 con 0x7f03e4070940 2023-12-20T04:44:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03e4158520 0x7f03e414ffd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03e4158520 0x7f03e414ffd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50920/0 (socket says 172.21.15.32:50920) 2023-12-20T04:44:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e2d9d640 1 -- 172.21.15.32:0/2726414542 learned_addr learned my addr 172.21.15.32:0/2726414542 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e259c640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e406dcb0 0x7f03e414dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e1d9b640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f03e4070940 0x7f03e4156430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e2d9d640 1 -- 172.21.15.32:0/2726414542 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f03e4070940 msgr2=0x7f03e4156430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e2d9d640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f03e4070940 0x7f03e4156430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e2d9d640 1 -- 172.21.15.32:0/2726414542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e406dcb0 msgr2=0x7f03e414dac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:16.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e2d9d640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e406dcb0 0x7f03e414dac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:16.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e2d9d640 1 -- 172.21.15.32:0/2726414542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03dc009a70 con 0x7f03e4158520 2023-12-20T04:44:16.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e2d9d640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03e4158520 0x7f03e414ffd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0b0040 tx=0x7f03dc019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:16.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03d2ffd640 1 -- 172.21.15.32:0/2726414542 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0adda0 con 0x7f03e4158520 2023-12-20T04:44:16.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03d2ffd640 1 -- 172.21.15.32:0/2726414542 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03dc0b9450 con 0x7f03e4158520 2023-12-20T04:44:16.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03d2ffd640 1 -- 172.21.15.32:0/2726414542 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03dc0b9800 con 0x7f03e4158520 2023-12-20T04:44:16.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e89bc640 1 -- 172.21.15.32:0/2726414542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f03e4150510 con 0x7f03e4158520 2023-12-20T04:44:16.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.533+0000 7f03e89bc640 1 -- 172.21.15.32:0/2726414542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f03e4156f10 con 0x7f03e4158520 2023-12-20T04:44:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.537+0000 7f03e89bc640 1 -- 172.21.15.32:0/2726414542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f03e4153af0 con 0x7f03e4158520 2023-12-20T04:44:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.537+0000 7f03d2ffd640 1 -- 172.21.15.32:0/2726414542 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 46) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f03dc0b9960 con 0x7f03e4158520 2023-12-20T04:44:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.537+0000 7f03d2ffd640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f03c4065af0 0x7f03c4067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.537+0000 7f03d2ffd640 1 -- 172.21.15.32:0/2726414542 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(96..96 src has 1..96) v4 ==== 6397+0+0 (secure 0 0 0) 0x7f03dc11ecf0 con 0x7f03e4158520 2023-12-20T04:44:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.537+0000 7f03e259c640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f03c4065af0 0x7f03c4067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:16.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.537+0000 7f03e259c640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f03c4065af0 0x7f03c4067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f03e414edd0 tx=0x7f03d4009040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:16.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.541+0000 7f03d2ffd640 1 -- 172.21.15.32:0/2726414542 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f03dc0ec570 con 0x7f03e4158520 2023-12-20T04:44:16.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:16.697+0000 7f03e89bc640 1 -- 172.21.15.32:0/2726414542 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f03e40b0a60 con 0x7f03e4158520 2023-12-20T04:44:17.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.365+0000 7f03d2ffd640 1 -- 172.21.15.32:0/2726414542 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f03dc0ec940 con 0x7f03e4158520 2023-12-20T04:44:17.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.365+0000 7f03d2ffd640 1 -- 172.21.15.32:0/2726414542 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v97) v1 ==== 181+0+0 (secure 0 0 0) 0x7f03dc0c4490 con 0x7f03e4158520 2023-12-20T04:44:17.366 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 -- 172.21.15.32:0/2726414542 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f03c4065af0 msgr2=0x7f03c4067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f03c4065af0 0x7f03c4067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f03e414edd0 tx=0x7f03d4009040 comp rx=0 tx=0).stop 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 -- 172.21.15.32:0/2726414542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03e4158520 msgr2=0x7f03e414ffd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03e4158520 0x7f03e414ffd0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7f03dc0b0040 tx=0x7f03dc019610 comp rx=0 tx=0).stop 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 -- 172.21.15.32:0/2726414542 shutdown_connections 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f03c4065af0 0x7f03c4067fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f03e4158520 0x7f03e414ffd0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f03e4070940 0x7f03e4156430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 --2- 172.21.15.32:0/2726414542 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f03e406dcb0 0x7f03e414dac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 -- 172.21.15.32:0/2726414542 >> 172.21.15.32:0/2726414542 conn(0x7f03e406a510 msgr2=0x7f03e40c4c50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 -- 172.21.15.32:0/2726414542 shutdown_connections 2023-12-20T04:44:17.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.369+0000 7f03e89bc640 1 -- 172.21.15.32:0/2726414542 wait complete. 2023-12-20T04:44:17.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T04:44:17.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f5af2c640 1 Processor -- start 2023-12-20T04:44:17.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f5af2c640 1 -- start start 2023-12-20T04:44:17.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f5af2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f54158520 0x7f3f5406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:17.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f5af2c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3f540ca050 0x7f3f5415ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:17.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f5af2c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f54070230 0x7f3f540706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:17.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f5af2c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f54070be0 con 0x7f3f54070230 2023-12-20T04:44:17.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f5af2c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f540ca7f0 con 0x7f3f540ca050 2023-12-20T04:44:17.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f5af2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f540caad0 con 0x7f3f54158520 2023-12-20T04:44:17.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f594c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f54070230 0x7f3f540706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:17.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f594c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f54070230 0x7f3f540706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50930/0 (socket says 172.21.15.32:50930) 2023-12-20T04:44:17.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f594c9640 1 -- 172.21.15.32:0/2709437855 learned_addr learned my addr 172.21.15.32:0/2709437855 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:17.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f58cc8640 1 --2- 172.21.15.32:0/2709437855 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f54158520 0x7f3f5406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:17.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f53fff640 1 --2- 172.21.15.32:0/2709437855 >> v2:172.21.15.39:3300/0 conn(0x7f3f540ca050 0x7f3f5415ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:17.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f594c9640 1 -- 172.21.15.32:0/2709437855 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f54158520 msgr2=0x7f3f5406dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:17.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f594c9640 1 --2- 172.21.15.32:0/2709437855 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f54158520 0x7f3f5406dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f594c9640 1 -- 172.21.15.32:0/2709437855 >> v2:172.21.15.39:3300/0 conn(0x7f3f540ca050 msgr2=0x7f3f5415ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:17.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f594c9640 1 --2- 172.21.15.32:0/2709437855 >> v2:172.21.15.39:3300/0 conn(0x7f3f540ca050 0x7f3f5415ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.545+0000 7f3f594c9640 1 -- 172.21.15.32:0/2709437855 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f540cadb0 con 0x7f3f54070230 2023-12-20T04:44:17.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f594c9640 1 --2- 172.21.15.32:0/2709437855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54070230 0x7f3f540706a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3f540716d0 tx=0x7f3f4c01c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c209fb25327e632 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:17.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f537fe640 1 -- 172.21.15.32:0/2709437855 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0a5820 con 0x7f3f54070230 2023-12-20T04:44:17.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f537fe640 1 -- 172.21.15.32:0/2709437855 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f4c0a5980 con 0x7f3f54070230 2023-12-20T04:44:17.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f537fe640 1 -- 172.21.15.32:0/2709437855 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f4c0a5d10 con 0x7f3f54070230 2023-12-20T04:44:17.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2709437855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54070230 msgr2=0x7f3f540706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:17.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f5af2c640 1 --2- 172.21.15.32:0/2709437855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54070230 0x7f3f540706a0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7f3f540716d0 tx=0x7f3f4c01c9a0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2709437855 shutdown_connections 2023-12-20T04:44:17.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f5af2c640 1 --2- 172.21.15.32:0/2709437855 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f54070230 0x7f3f540706a0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f5af2c640 1 --2- 172.21.15.32:0/2709437855 >> v2:172.21.15.39:3300/0 conn(0x7f3f540ca050 0x7f3f5415ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f5af2c640 1 --2- 172.21.15.32:0/2709437855 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f54158520 0x7f3f5406dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2709437855 >> 172.21.15.32:0/2709437855 conn(0x7f3f5406a510 msgr2=0x7f3f540ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:17.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2709437855 shutdown_connections 2023-12-20T04:44:17.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.549+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2709437855 wait complete. 2023-12-20T04:44:17.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f5af2c640 1 Processor -- start 2023-12-20T04:44:17.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f5af2c640 1 -- start start 2023-12-20T04:44:17.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f5af2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f54070230 0x7f3f5414eb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:17.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f5af2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540ca050 0x7f3f54151ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:17.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f5af2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f54158520 0x7f3f541504b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:17.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f5af2c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54158900 con 0x7f3f540ca050 2023-12-20T04:44:17.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f5af2c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f541521e0 con 0x7f3f54158520 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f5af2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f54158b50 con 0x7f3f54070230 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540ca050 0x7f3f54151ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540ca050 0x7f3f54151ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50944/0 (socket says 172.21.15.32:50944) 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f53fff640 1 -- 172.21.15.32:0/2325802484 learned_addr learned my addr 172.21.15.32:0/2325802484 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f58cc8640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f54070230 0x7f3f5414eb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f594c9640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f54158520 0x7f3f541504b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f53fff640 1 -- 172.21.15.32:0/2325802484 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f54070230 msgr2=0x7f3f5414eb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f53fff640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f54070230 0x7f3f5414eb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f53fff640 1 -- 172.21.15.32:0/2325802484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f54158520 msgr2=0x7f3f541504b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f53fff640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f54158520 0x7f3f541504b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f53fff640 1 -- 172.21.15.32:0/2325802484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f4c009b30 con 0x7f3f540ca050 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f594c9640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f54158520 0x7f3f541504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f53fff640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540ca050 0x7f3f54151ca0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000d800 tx=0x7f3f4000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:17.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f517fa640 1 -- 172.21.15.32:0/2325802484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f400096a0 con 0x7f3f540ca050 2023-12-20T04:44:17.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f517fa640 1 -- 172.21.15.32:0/2325802484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f40009800 con 0x7f3f540ca050 2023-12-20T04:44:17.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f517fa640 1 -- 172.21.15.32:0/2325802484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f40009b70 con 0x7f3f540ca050 2023-12-20T04:44:17.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.553+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2325802484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f54158e30 con 0x7f3f540ca050 2023-12-20T04:44:17.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.557+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2325802484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f541593f0 con 0x7f3f540ca050 2023-12-20T04:44:17.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.557+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2325802484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f54159580 con 0x7f3f540ca050 2023-12-20T04:44:17.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.557+0000 7f3f517fa640 1 -- 172.21.15.32:0/2325802484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3f40009cd0 con 0x7f3f540ca050 2023-12-20T04:44:17.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.557+0000 7f3f517fa640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f38065b80 0x7f3f38068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:17.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.557+0000 7f3f517fa640 1 -- 172.21.15.32:0/2325802484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(97..97 src has 1..97) v4 ==== 5959+0+0 (secure 0 0 0) 0x7f3f40083b80 con 0x7f3f540ca050 2023-12-20T04:44:17.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.557+0000 7f3f58cc8640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f38065b80 0x7f3f38068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:17.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.557+0000 7f3f58cc8640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f38065b80 0x7f3f38068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3f540715c0 tx=0x7f3f48009190 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:17.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.561+0000 7f3f517fa640 1 -- 172.21.15.32:0/2325802484 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3f40051290 con 0x7f3f540ca050 2023-12-20T04:44:17.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:17.725+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2325802484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f3f540b4740 con 0x7f3f540ca050 2023-12-20T04:44:18.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.361+0000 7f3f517fa640 1 -- 172.21.15.32:0/2325802484 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v98) v1 ==== 169+0+0 (secure 0 0 0) 0x7f3f40083e60 con 0x7f3f540ca050 2023-12-20T04:44:18.364 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T04:44:18.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2325802484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f38065b80 msgr2=0x7f3f38068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f38065b80 0x7f3f38068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3f540715c0 tx=0x7f3f48009190 comp rx=0 tx=0).stop 2023-12-20T04:44:18.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2325802484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540ca050 msgr2=0x7f3f54151ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540ca050 0x7f3f54151ca0 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7f3f4000d800 tx=0x7f3f4000dcc0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2325802484 shutdown_connections 2023-12-20T04:44:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f38065b80 0x7f3f38068040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f54158520 0x7f3f541504b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f540ca050 0x7f3f54151ca0 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 --2- 172.21.15.32:0/2325802484 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f54070230 0x7f3f5414eb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2325802484 >> 172.21.15.32:0/2325802484 conn(0x7f3f5406a510 msgr2=0x7f3f5415b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2325802484 shutdown_connections 2023-12-20T04:44:18.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.369+0000 7f3f5af2c640 1 -- 172.21.15.32:0/2325802484 wait complete. 2023-12-20T04:44:18.396 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T04:44:18.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.545+0000 7f92aaba6640 1 Processor -- start 2023-12-20T04:44:18.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.545+0000 7f92aaba6640 1 -- start start 2023-12-20T04:44:18.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.545+0000 7f92aaba6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92a4157480 0x7f92a40c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:18.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.545+0000 7f92aaba6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a40b0760 0x7f92a40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:18.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.545+0000 7f92aaba6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92a40b1100 con 0x7f92a415d660 2023-12-20T04:44:18.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.545+0000 7f92aaba6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92a40cde20 con 0x7f92a4157480 2023-12-20T04:44:18.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.545+0000 7f92aaba6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40c8e30 con 0x7f92a40b0760 2023-12-20T04:44:18.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a8942640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92a4157480 0x7f92a40c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:18.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a8942640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f92a4157480 0x7f92a40c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50956/0 (socket says 172.21.15.32:50956) 2023-12-20T04:44:18.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a9143640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92a415d660 0x7f92a415daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37960/0 (socket says 172.21.15.32:37960) 2023-12-20T04:44:18.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a8942640 1 -- 172.21.15.32:0/1361932769 learned_addr learned my addr 172.21.15.32:0/1361932769 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:18.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a3fff640 1 --2- 172.21.15.32:0/1361932769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a40b0760 0x7f92a40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:18.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a8942640 1 -- 172.21.15.32:0/1361932769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a40b0760 msgr2=0x7f92a40b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:18.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a8942640 1 --2- 172.21.15.32:0/1361932769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a40b0760 0x7f92a40b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a8942640 1 -- 172.21.15.32:0/1361932769 >> v1:172.21.15.32:6789/0 conn(0x7f92a415d660 legacy=0x7f92a415daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a8942640 1 -- 172.21.15.32:0/1361932769 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a40ce070 con 0x7f92a4157480 2023-12-20T04:44:18.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a37fe640 1 -- 172.21.15.32:0/1361932769 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2528847297 0 0) 0x7f92a40b1100 con 0x7f92a415d660 2023-12-20T04:44:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a8942640 1 --2- 172.21.15.32:0/1361932769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4157480 0x7f92a40c9840 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f929800bb10 tx=0x7f929801eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20cc821a21b18e75 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a37fe640 1 -- 172.21.15.32:0/1361932769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92980a7b20 con 0x7f92a4157480 2023-12-20T04:44:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a37fe640 1 -- 172.21.15.32:0/1361932769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92980a7c80 con 0x7f92a4157480 2023-12-20T04:44:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.549+0000 7f92a37fe640 1 -- 172.21.15.32:0/1361932769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92980af440 con 0x7f92a4157480 2023-12-20T04:44:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 -- 172.21.15.32:0/1361932769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4157480 msgr2=0x7f92a40c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 --2- 172.21.15.32:0/1361932769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4157480 0x7f92a40c9840 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7f929800bb10 tx=0x7f929801eb50 comp rx=0 tx=0).stop 2023-12-20T04:44:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 -- 172.21.15.32:0/1361932769 shutdown_connections 2023-12-20T04:44:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 --2- 172.21.15.32:0/1361932769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a40b0760 0x7f92a40b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 --2- 172.21.15.32:0/1361932769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4157480 0x7f92a40c9840 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 -- 172.21.15.32:0/1361932769 >> 172.21.15.32:0/1361932769 conn(0x7f92a406a510 msgr2=0x7f92a40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 -- 172.21.15.32:0/1361932769 shutdown_connections 2023-12-20T04:44:18.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 -- 172.21.15.32:0/1361932769 wait complete. 2023-12-20T04:44:18.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 Processor -- start 2023-12-20T04:44:18.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 -- start start 2023-12-20T04:44:18.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92a40b0760 0x7f92a4156930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:18.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4157480 0x7f92a40c3c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:18.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a415d660 0x7f92a406cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:18.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a41f5310 con 0x7f92a4157480 2023-12-20T04:44:18.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40c58b0 con 0x7f92a40b0760 2023-12-20T04:44:18.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92aaba6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a40c5b90 con 0x7f92a415d660 2023-12-20T04:44:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92a3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4157480 0x7f92a40c3c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92a3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4157480 0x7f92a40c3c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50960/0 (socket says 172.21.15.32:50960) 2023-12-20T04:44:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.553+0000 7f92a3fff640 1 -- 172.21.15.32:0/321573343 learned_addr learned my addr 172.21.15.32:0/321573343 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a8942640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92a40b0760 0x7f92a4156930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a3fff640 1 -- 172.21.15.32:0/321573343 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a415d660 msgr2=0x7f92a406cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a3fff640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a415d660 0x7f92a406cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a3fff640 1 -- 172.21.15.32:0/321573343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92a40b0760 msgr2=0x7f92a4156930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a3fff640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92a40b0760 0x7f92a4156930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a3fff640 1 -- 172.21.15.32:0/321573343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f929800b770 con 0x7f92a4157480 2023-12-20T04:44:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a8942640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92a40b0760 0x7f92a4156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:44:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a3fff640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4157480 0x7f92a40c3c00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f929000d5f0 tx=0x7f929000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a17fa640 1 -- 172.21.15.32:0/321573343 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92900125f0 con 0x7f92a4157480 2023-12-20T04:44:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a17fa640 1 -- 172.21.15.32:0/321573343 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9290012750 con 0x7f92a4157480 2023-12-20T04:44:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92aaba6640 1 -- 172.21.15.32:0/321573343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92a40c5e70 con 0x7f92a4157480 2023-12-20T04:44:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a17fa640 1 -- 172.21.15.32:0/321573343 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9290012ae0 con 0x7f92a4157480 2023-12-20T04:44:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92aaba6640 1 -- 172.21.15.32:0/321573343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92a40c6430 con 0x7f92a4157480 2023-12-20T04:44:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92aaba6640 1 -- 172.21.15.32:0/321573343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92a41f5480 con 0x7f92a4157480 2023-12-20T04:44:18.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a17fa640 1 -- 172.21.15.32:0/321573343 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9290012c40 con 0x7f92a4157480 2023-12-20T04:44:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a17fa640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9288065b80 0x7f9288068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a17fa640 1 -- 172.21.15.32:0/321573343 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f92900836a0 con 0x7f92a4157480 2023-12-20T04:44:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.557+0000 7f92a8942640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9288065b80 0x7f9288068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.561+0000 7f92a8942640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9288065b80 0x7f9288068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f92980b1000 tx=0x7f9298008040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.561+0000 7f92a17fa640 1 -- 172.21.15.32:0/321573343 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9290051080 con 0x7f92a4157480 2023-12-20T04:44:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.733+0000 7f92aaba6640 1 -- 172.21.15.32:0/321573343 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f92a404e000 con 0x7f92a4157480 2023-12-20T04:44:18.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.741+0000 7f92a17fa640 1 -- 172.21.15.32:0/321573343 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v31) v1 ==== 120+0+0 (secure 0 0 0) 0x7f9290007380 con 0x7f92a4157480 2023-12-20T04:44:18.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 -- 172.21.15.32:0/321573343 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9288065b80 msgr2=0x7f9288068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9288065b80 0x7f9288068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f92980b1000 tx=0x7f9298008040 comp rx=0 tx=0).stop 2023-12-20T04:44:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 -- 172.21.15.32:0/321573343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4157480 msgr2=0x7f92a40c3c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4157480 0x7f92a40c3c00 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f929000d5f0 tx=0x7f929000dab0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 -- 172.21.15.32:0/321573343 shutdown_connections 2023-12-20T04:44:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9288065b80 0x7f9288068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a415d660 0x7f92a406cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4157480 0x7f92a40c3c00 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 --2- 172.21.15.32:0/321573343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92a40b0760 0x7f92a4156930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 -- 172.21.15.32:0/321573343 >> 172.21.15.32:0/321573343 conn(0x7f92a406a510 msgr2=0x7f92a40c47a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 -- 172.21.15.32:0/321573343 shutdown_connections 2023-12-20T04:44:18.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.745+0000 7f92aaba6640 1 -- 172.21.15.32:0/321573343 wait complete. 2023-12-20T04:44:18.764 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T04:44:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.909+0000 7f1f81d6d640 1 Processor -- start 2023-12-20T04:44:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.909+0000 7f1f81d6d640 1 -- start start 2023-12-20T04:44:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.909+0000 7f1f81d6d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f7c070490 0x7f1f7c15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.909+0000 7f1f81d6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c06c480 con 0x7f1f7c06d240 2023-12-20T04:44:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.909+0000 7f1f81d6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f7c06c760 con 0x7f1f7c070870 2023-12-20T04:44:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.909+0000 7f1f81d6d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f7c06ca40 con 0x7f1f7c070490 2023-12-20T04:44:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f7c070490 0x7f1f7c15fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f7c070490 0x7f1f7c15fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38784/0 (socket says 172.21.15.32:38784) 2023-12-20T04:44:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7b7fe640 1 -- 172.21.15.32:0/3179263528 learned_addr learned my addr 172.21.15.32:0/3179263528 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:18.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7affd640 1 --1- 172.21.15.32:0/3179263528 >> v1:172.21.15.32:6789/0 conn(0x7f1f7c06d240 0x7f1f7c0ca1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54206/0 (socket says 172.21.15.32:54206) 2023-12-20T04:44:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7b7fe640 1 -- 172.21.15.32:0/3179263528 >> v1:172.21.15.39:6789/0 conn(0x7f1f7c070870 legacy=0x7f1f7c0cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7b7fe640 1 -- 172.21.15.32:0/3179263528 >> v1:172.21.15.32:6789/0 conn(0x7f1f7c06d240 legacy=0x7f1f7c0ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7b7fe640 1 -- 172.21.15.32:0/3179263528 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f7c06cd20 con 0x7f1f7c070490 2023-12-20T04:44:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7a7fc640 1 -- 172.21.15.32:0/3179263528 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3086473097 0 0) 0x7f1f7c06c480 con 0x7f1f7c06d240 2023-12-20T04:44:18.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7b7fe640 1 --2- 172.21.15.32:0/3179263528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f7c070490 0x7f1f7c15fcd0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009870 tx=0x7f1f6c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=830ec0f16f016aa3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7a7fc640 1 -- 172.21.15.32:0/3179263528 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c0ab680 con 0x7f1f7c070490 2023-12-20T04:44:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7a7fc640 1 -- 172.21.15.32:0/3179263528 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c0ab7e0 con 0x7f1f7c070490 2023-12-20T04:44:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.913+0000 7f1f7a7fc640 1 -- 172.21.15.32:0/3179263528 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c0abb70 con 0x7f1f7c070490 2023-12-20T04:44:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3179263528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f7c070490 msgr2=0x7f1f7c15fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 --2- 172.21.15.32:0/3179263528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f7c070490 0x7f1f7c15fcd0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009870 tx=0x7f1f6c01c480 comp rx=0 tx=0).stop 2023-12-20T04:44:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3179263528 shutdown_connections 2023-12-20T04:44:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 --2- 172.21.15.32:0/3179263528 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f7c070490 0x7f1f7c15fcd0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3179263528 >> 172.21.15.32:0/3179263528 conn(0x7f1f7c06a510 msgr2=0x7f1f7c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3179263528 shutdown_connections 2023-12-20T04:44:18.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3179263528 wait complete. 2023-12-20T04:44:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 Processor -- start 2023-12-20T04:44:18.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 -- start start 2023-12-20T04:44:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f7c06d240 0x7f1f7c1506d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f7c070490 0x7f1f7c1572e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f7c070870 0x7f1f7c0c4a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0c6480 con 0x7f1f7c070870 2023-12-20T04:44:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0c58b0 con 0x7f1f7c070490 2023-12-20T04:44:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f81d6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f7c0c5ae0 con 0x7f1f7c06d240 2023-12-20T04:44:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f7bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f7c070870 0x7f1f7c0c4a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f7b7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f7c06d240 0x7f1f7c1506d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f7affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f7c070490 0x7f1f7c1572e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f7b7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f7c06d240 0x7f1f7c1506d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:57410/0 (socket says 172.21.15.32:57410) 2023-12-20T04:44:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.917+0000 7f1f7b7fe640 1 -- 172.21.15.32:0/3948972319 learned_addr learned my addr 172.21.15.32:0/3948972319 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f7b7fe640 1 -- 172.21.15.32:0/3948972319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f7c070490 msgr2=0x7f1f7c1572e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f7b7fe640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f7c070490 0x7f1f7c1572e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f7b7fe640 1 -- 172.21.15.32:0/3948972319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f7c070870 msgr2=0x7f1f7c0c4a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f7b7fe640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f7c070870 0x7f1f7c0c4a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f7b7fe640 1 -- 172.21.15.32:0/3948972319 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f6c0094d0 con 0x7f1f7c06d240 2023-12-20T04:44:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f7bfff640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f7c070870 0x7f1f7c0c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:18.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f7b7fe640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f7c06d240 0x7f1f7c1506d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009870 tx=0x7f1f6c01acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f63fff640 1 -- 172.21.15.32:0/3948972319 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c0abea0 con 0x7f1f7c06d240 2023-12-20T04:44:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f63fff640 1 -- 172.21.15.32:0/3948972319 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f6c0b8070 con 0x7f1f7c06d240 2023-12-20T04:44:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f63fff640 1 -- 172.21.15.32:0/3948972319 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f6c004220 con 0x7f1f7c06d240 2023-12-20T04:44:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3948972319 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f7c0c5dc0 con 0x7f1f7c06d240 2023-12-20T04:44:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3948972319 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f7c0c4f80 con 0x7f1f7c06d240 2023-12-20T04:44:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3948972319 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f7c0c5110 con 0x7f1f7c06d240 2023-12-20T04:44:18.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f63fff640 1 -- 172.21.15.32:0/3948972319 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1f6c004380 con 0x7f1f7c06d240 2023-12-20T04:44:18.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f63fff640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1f68065b80 0x7f1f68068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:18.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.921+0000 7f1f63fff640 1 -- 172.21.15.32:0/3948972319 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f1f6c11d630 con 0x7f1f7c06d240 2023-12-20T04:44:18.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.925+0000 7f1f7affd640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1f68065b80 0x7f1f68068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:18.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.925+0000 7f1f7affd640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1f68065b80 0x7f1f68068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c14d9f0 tx=0x7f1f70007240 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:18.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:18.929+0000 7f1f63fff640 1 -- 172.21.15.32:0/3948972319 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1f6c0eafa0 con 0x7f1f7c06d240 2023-12-20T04:44:19.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.085+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3948972319 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f1f7c04e000 con 0x7f1f7c06d240 2023-12-20T04:44:19.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.093+0000 7f1f63fff640 1 -- 172.21.15.32:0/3948972319 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v32) v1 ==== 115+0+0 (secure 0 0 0) 0x7f1f6c0ee5c0 con 0x7f1f7c06d240 2023-12-20T04:44:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3948972319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1f68065b80 msgr2=0x7f1f68068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1f68065b80 0x7f1f68068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f1f7c14d9f0 tx=0x7f1f70007240 comp rx=0 tx=0).stop 2023-12-20T04:44:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3948972319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f7c06d240 msgr2=0x7f1f7c1506d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f7c06d240 0x7f1f7c1506d0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f1f6c009870 tx=0x7f1f6c01acc0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3948972319 shutdown_connections 2023-12-20T04:44:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1f68065b80 0x7f1f68068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f7c070870 0x7f1f7c0c4a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f7c070490 0x7f1f7c1572e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 --2- 172.21.15.32:0/3948972319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f7c06d240 0x7f1f7c1506d0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3948972319 >> 172.21.15.32:0/3948972319 conn(0x7f1f7c06a510 msgr2=0x7f1f7c157a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:19.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3948972319 shutdown_connections 2023-12-20T04:44:19.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.097+0000 7f1f81d6d640 1 -- 172.21.15.32:0/3948972319 wait complete. 2023-12-20T04:44:19.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T04:44:19.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff676925640 1 Processor -- start 2023-12-20T04:44:19.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff676925640 1 -- start start 2023-12-20T04:44:19.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff676925640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff670157480 0x7ff6700c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff676925640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6700b0760 0x7ff6700b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff676925640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6700b1100 con 0x7ff67015d660 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff676925640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6700cde20 con 0x7ff6700b0760 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff676925640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6700c8e30 con 0x7ff670157480 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff66f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6700b0760 0x7ff6700b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff66f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6700b0760 0x7ff6700b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38806/0 (socket says 172.21.15.32:38806) 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff66f7fe640 1 -- 172.21.15.32:0/2683624804 learned_addr learned my addr 172.21.15.32:0/2683624804 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff66ffff640 1 --2- 172.21.15.32:0/2683624804 >> v2:172.21.15.39:3300/0 conn(0x7ff670157480 0x7ff6700c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff66f7fe640 1 -- 172.21.15.32:0/2683624804 >> v2:172.21.15.39:3300/0 conn(0x7ff670157480 msgr2=0x7ff6700c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff66f7fe640 1 --2- 172.21.15.32:0/2683624804 >> v2:172.21.15.39:3300/0 conn(0x7ff670157480 0x7ff6700c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff66f7fe640 1 -- 172.21.15.32:0/2683624804 >> v1:172.21.15.39:6789/0 conn(0x7ff67015d660 legacy=0x7ff67015da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:19.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.261+0000 7ff66f7fe640 1 -- 172.21.15.32:0/2683624804 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6700ce070 con 0x7ff6700b0760 2023-12-20T04:44:19.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff66f7fe640 1 --2- 172.21.15.32:0/2683624804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6700b0760 0x7ff6700b0bc0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff6700cf410 tx=0x7ff66401c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c1389ef7d3cc25d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:19.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff66effd640 1 -- 172.21.15.32:0/2683624804 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640ab600 con 0x7ff6700b0760 2023-12-20T04:44:19.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff66effd640 1 -- 172.21.15.32:0/2683624804 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6640ab760 con 0x7ff6700b0760 2023-12-20T04:44:19.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff66effd640 1 -- 172.21.15.32:0/2683624804 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6640abaf0 con 0x7ff6700b0760 2023-12-20T04:44:19.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff676925640 1 -- 172.21.15.32:0/2683624804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6700b0760 msgr2=0x7ff6700b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:19.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff676925640 1 --2- 172.21.15.32:0/2683624804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6700b0760 0x7ff6700b0bc0 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7ff6700cf410 tx=0x7ff66401c400 comp rx=0 tx=0).stop 2023-12-20T04:44:19.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff676925640 1 -- 172.21.15.32:0/2683624804 shutdown_connections 2023-12-20T04:44:19.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff676925640 1 --2- 172.21.15.32:0/2683624804 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6700b0760 0x7ff6700b0bc0 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff676925640 1 --2- 172.21.15.32:0/2683624804 >> v2:172.21.15.39:3300/0 conn(0x7ff670157480 0x7ff6700c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff676925640 1 -- 172.21.15.32:0/2683624804 >> 172.21.15.32:0/2683624804 conn(0x7ff67006a510 msgr2=0x7ff6700ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:19.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff676925640 1 -- 172.21.15.32:0/2683624804 shutdown_connections 2023-12-20T04:44:19.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.265+0000 7ff676925640 1 -- 172.21.15.32:0/2683624804 wait complete. 2023-12-20T04:44:19.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff676925640 1 Processor -- start 2023-12-20T04:44:19.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff676925640 1 -- start start 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff676925640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6700b0760 0x7ff670155c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff676925640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670157480 0x7ff6701563e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff676925640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff67015d660 0x7ff6700c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff676925640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff670156160 con 0x7ff67015d660 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff676925640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700c5db0 con 0x7ff670157480 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff676925640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6700c6090 con 0x7ff6700b0760 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff674ec2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff67015d660 0x7ff6700c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff66f7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670157480 0x7ff6701563e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff674ec2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff67015d660 0x7ff6700c6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38816/0 (socket says 172.21.15.32:38816) 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff674ec2640 1 -- 172.21.15.32:0/2980204022 learned_addr learned my addr 172.21.15.32:0/2980204022 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff66ffff640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6700b0760 0x7ff670155c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff674ec2640 1 -- 172.21.15.32:0/2980204022 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6700b0760 msgr2=0x7ff670155c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:19.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff674ec2640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6700b0760 0x7ff670155c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff674ec2640 1 -- 172.21.15.32:0/2980204022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670157480 msgr2=0x7ff6701563e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:19.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff674ec2640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670157480 0x7ff6701563e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff674ec2640 1 -- 172.21.15.32:0/2980204022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6640094d0 con 0x7ff67015d660 2023-12-20T04:44:19.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff674ec2640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff67015d660 0x7ff6700c6e50 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff66800b970 tx=0x7ff66800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:19.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff66cff9640 1 -- 172.21.15.32:0/2980204022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff66800cb40 con 0x7ff67015d660 2023-12-20T04:44:19.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff66cff9640 1 -- 172.21.15.32:0/2980204022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff66800cca0 con 0x7ff67015d660 2023-12-20T04:44:19.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff66cff9640 1 -- 172.21.15.32:0/2980204022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff668014680 con 0x7ff67015d660 2023-12-20T04:44:19.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff676925640 1 -- 172.21.15.32:0/2980204022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6700c6370 con 0x7ff67015d660 2023-12-20T04:44:19.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.269+0000 7ff676925640 1 -- 172.21.15.32:0/2980204022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6700c6930 con 0x7ff67015d660 2023-12-20T04:44:19.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.273+0000 7ff676925640 1 -- 172.21.15.32:0/2980204022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6700c6b20 con 0x7ff67015d660 2023-12-20T04:44:19.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.273+0000 7ff66cff9640 1 -- 172.21.15.32:0/2980204022 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff66801d3b0 con 0x7ff67015d660 2023-12-20T04:44:19.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.273+0000 7ff66cff9640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff65c065b80 0x7ff65c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.273+0000 7ff66cff9640 1 -- 172.21.15.32:0/2980204022 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5545+0+0 (secure 0 0 0) 0x7ff6680830b0 con 0x7ff67015d660 2023-12-20T04:44:19.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.273+0000 7ff66ffff640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff65c065b80 0x7ff65c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.273+0000 7ff66ffff640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff65c065b80 0x7ff65c068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff660007a60 tx=0x7ff660008040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:19.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.277+0000 7ff66cff9640 1 -- 172.21.15.32:0/2980204022 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff668050a10 con 0x7ff67015d660 2023-12-20T04:44:19.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.437+0000 7ff676925640 1 -- 172.21.15.32:0/2980204022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7ff67004e000 con 0x7ff67015d660 2023-12-20T04:44:19.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff66cff9640 1 -- 172.21.15.32:0/2980204022 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v33) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff668054030 con 0x7ff67015d660 2023-12-20T04:44:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 -- 172.21.15.32:0/2980204022 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff65c065b80 msgr2=0x7ff65c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff65c065b80 0x7ff65c068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7ff660007a60 tx=0x7ff660008040 comp rx=0 tx=0).stop 2023-12-20T04:44:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 -- 172.21.15.32:0/2980204022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff67015d660 msgr2=0x7ff6700c6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff67015d660 0x7ff6700c6e50 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7ff66800b970 tx=0x7ff66800be30 comp rx=0 tx=0).stop 2023-12-20T04:44:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 -- 172.21.15.32:0/2980204022 shutdown_connections 2023-12-20T04:44:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff65c065b80 0x7ff65c068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff67015d660 0x7ff6700c6e50 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff670157480 0x7ff6701563e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 --2- 172.21.15.32:0/2980204022 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6700b0760 0x7ff670155c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 -- 172.21.15.32:0/2980204022 >> 172.21.15.32:0/2980204022 conn(0x7ff67006a510 msgr2=0x7ff6700ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 -- 172.21.15.32:0/2980204022 shutdown_connections 2023-12-20T04:44:19.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.441+0000 7ff676925640 1 -- 172.21.15.32:0/2980204022 wait complete. 2023-12-20T04:44:19.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list' 2023-12-20T04:44:19.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.605+0000 7f6980939640 1 Processor -- start 2023-12-20T04:44:19.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.605+0000 7f6980939640 1 -- start start 2023-12-20T04:44:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.605+0000 7f6980939640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f697c0aece0 0x7f697c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.605+0000 7f6980939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f697c0b0b30 0x7f697c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.605+0000 7f6980939640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f697c0c98c0 con 0x7f697c1599e0 2023-12-20T04:44:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.605+0000 7f6980939640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f697c0c9ba0 con 0x7f697c0aece0 2023-12-20T04:44:19.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.605+0000 7f6980939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0c9e80 con 0x7f697c0b0b30 2023-12-20T04:44:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f6979d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f697c0aece0 0x7f697c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f697a59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f697c1599e0 0x7f697c0c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54210/0 (socket says 172.21.15.32:54210) 2023-12-20T04:44:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f697a59c640 1 -- 172.21.15.32:0/2700972232 learned_addr learned my addr 172.21.15.32:0/2700972232 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f6979d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f697c0aece0 0x7f697c0b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38832/0 (socket says 172.21.15.32:38832) 2023-12-20T04:44:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f697ad9d640 1 --2- 172.21.15.32:0/2700972232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f697c0b0b30 0x7f697c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f6979d9b640 1 -- 172.21.15.32:0/2700972232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f697c0b0b30 msgr2=0x7f697c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f6979d9b640 1 --2- 172.21.15.32:0/2700972232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f697c0b0b30 0x7f697c0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f6979d9b640 1 -- 172.21.15.32:0/2700972232 >> v1:172.21.15.32:6789/0 conn(0x7f697c1599e0 legacy=0x7f697c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f6979d9b640 1 -- 172.21.15.32:0/2700972232 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f697c0ca160 con 0x7f697c0aece0 2023-12-20T04:44:19.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f697959a640 1 -- 172.21.15.32:0/2700972232 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3449904905 0 0) 0x7f697c0c98c0 con 0x7f697c1599e0 2023-12-20T04:44:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f6979d9b640 1 --2- 172.21.15.32:0/2700972232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c0aece0 0x7f697c0b05f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f69700097c0 tx=0x7f697001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82deca36d5e2a909 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f697959a640 1 -- 172.21.15.32:0/2700972232 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700ad640 con 0x7f697c0aece0 2023-12-20T04:44:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f697959a640 1 -- 172.21.15.32:0/2700972232 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69700ad7a0 con 0x7f697c0aece0 2023-12-20T04:44:19.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.609+0000 7f697959a640 1 -- 172.21.15.32:0/2700972232 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69700adb30 con 0x7f697c0aece0 2023-12-20T04:44:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 -- 172.21.15.32:0/2700972232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c0aece0 msgr2=0x7f697c0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 --2- 172.21.15.32:0/2700972232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c0aece0 0x7f697c0b05f0 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f69700097c0 tx=0x7f697001c480 comp rx=0 tx=0).stop 2023-12-20T04:44:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 -- 172.21.15.32:0/2700972232 shutdown_connections 2023-12-20T04:44:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 --2- 172.21.15.32:0/2700972232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c0aece0 0x7f697c0b05f0 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 --2- 172.21.15.32:0/2700972232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f697c0b0b30 0x7f697c0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 -- 172.21.15.32:0/2700972232 >> 172.21.15.32:0/2700972232 conn(0x7f697c06a320 msgr2=0x7f697c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 -- 172.21.15.32:0/2700972232 shutdown_connections 2023-12-20T04:44:19.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 -- 172.21.15.32:0/2700972232 wait complete. 2023-12-20T04:44:19.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 Processor -- start 2023-12-20T04:44:19.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 -- start start 2023-12-20T04:44:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f697c0aece0 0x7f697c0c4040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c0b0b30 0x7f697c155740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f697c1599e0 0x7f697c161a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c0c4580 con 0x7f697c0b0b30 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f697c14bea0 con 0x7f697c1599e0 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6980939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f697c155c80 con 0x7f697c0aece0 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6979d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c0b0b30 0x7f697c155740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6979d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c0b0b30 0x7f697c155740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38838/0 (socket says 172.21.15.32:38838) 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f6979d9b640 1 -- 172.21.15.32:0/3243801711 learned_addr learned my addr 172.21.15.32:0/3243801711 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.613+0000 7f697a59c640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f697c0aece0 0x7f697c0c4040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f697ad9d640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f697c1599e0 0x7f697c161a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f6979d9b640 1 -- 172.21.15.32:0/3243801711 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f697c0aece0 msgr2=0x7f697c0c4040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f6979d9b640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f697c0aece0 0x7f697c0c4040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f6979d9b640 1 -- 172.21.15.32:0/3243801711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f697c1599e0 msgr2=0x7f697c161a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f6979d9b640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f697c1599e0 0x7f697c161a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:19.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f6979d9b640 1 -- 172.21.15.32:0/3243801711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6970009420 con 0x7f697c0b0b30 2023-12-20T04:44:19.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f697a59c640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f697c0aece0 0x7f697c0c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:19.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f6979d9b640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c0b0b30 0x7f697c155740 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f697001a0b0 tx=0x7f6970019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f696affd640 1 -- 172.21.15.32:0/3243801711 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970004220 con 0x7f697c0b0b30 2023-12-20T04:44:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f696affd640 1 -- 172.21.15.32:0/3243801711 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69700a6040 con 0x7f697c0b0b30 2023-12-20T04:44:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f6980939640 1 -- 172.21.15.32:0/3243801711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f697c153dd0 con 0x7f697c0b0b30 2023-12-20T04:44:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f696affd640 1 -- 172.21.15.32:0/3243801711 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6970003620 con 0x7f697c0b0b30 2023-12-20T04:44:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f6980939640 1 -- 172.21.15.32:0/3243801711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f697c14c0f0 con 0x7f697c0b0b30 2023-12-20T04:44:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f6980939640 1 -- 172.21.15.32:0/3243801711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f697c1542f0 con 0x7f697c0b0b30 2023-12-20T04:44:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f696affd640 1 -- 172.21.15.32:0/3243801711 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f69700b4040 con 0x7f697c0b0b30 2023-12-20T04:44:19.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f696affd640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f695c065b80 0x7f695c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:19.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.617+0000 7f696affd640 1 -- 172.21.15.32:0/3243801711 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f697011cad0 con 0x7f697c0b0b30 2023-12-20T04:44:19.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.621+0000 7f697a59c640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f695c065b80 0x7f695c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:19.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.621+0000 7f697a59c640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f695c065b80 0x7f695c068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f696c0010f0 tx=0x7f696c009040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:19.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.621+0000 7f696affd640 1 -- 172.21.15.32:0/3243801711 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f69700ea2f0 con 0x7f697c0b0b30 2023-12-20T04:44:19.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:19.845+0000 7f6980939640 1 -- 172.21.15.32:0/3243801711 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]} v 0) v1 -- 0x7f697c04e000 con 0x7f697c0b0b30 2023-12-20T04:44:20.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.373+0000 7f696affd640 1 -- 172.21.15.32:0/3243801711 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_non_existent_snap_schedule_list"]}]=0 v588) v1 ==== 166+0+0 (secure 0 0 0) 0x7f69700b2030 con 0x7f697c0b0b30 2023-12-20T04:44:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 -- 172.21.15.32:0/3243801711 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f695c065b80 msgr2=0x7f695c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f695c065b80 0x7f695c068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f696c0010f0 tx=0x7f696c009040 comp rx=0 tx=0).stop 2023-12-20T04:44:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 -- 172.21.15.32:0/3243801711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c0b0b30 msgr2=0x7f697c155740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c0b0b30 0x7f697c155740 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f697001a0b0 tx=0x7f6970019220 comp rx=0 tx=0).stop 2023-12-20T04:44:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 -- 172.21.15.32:0/3243801711 shutdown_connections 2023-12-20T04:44:20.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f695c065b80 0x7f695c068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f697c1599e0 0x7f697c161a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f697c0b0b30 0x7f697c155740 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 --2- 172.21.15.32:0/3243801711 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f697c0aece0 0x7f697c0c4040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 -- 172.21.15.32:0/3243801711 >> 172.21.15.32:0/3243801711 conn(0x7f697c06a320 msgr2=0x7f697c0cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 -- 172.21.15.32:0/3243801711 shutdown_connections 2023-12-20T04:44:20.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.377+0000 7f6980939640 1 -- 172.21.15.32:0/3243801711 wait complete. 2023-12-20T04:44:20.401 INFO:tasks.cephfs_test_runner:Test listing snap schedules on a non-existing filesystem path failure ... ok 2023-12-20T04:44:20.401 INFO:tasks.cephfs_test_runner:Starting test: test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that a non-existent path leads to schedule deactivation after a few retries. 2023-12-20T04:44:20.401 INFO:tasks.cephfs_test_runner:test_schedule_auto_deactivation_for_non_existent_path (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-20T04:44:20.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-12-20T04:44:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.549+0000 7f00b16d8640 1 Processor -- start 2023-12-20T04:44:20.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00b16d8640 1 -- start start 2023-12-20T04:44:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00b16d8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00ac15cd70 0x7f00ac0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00b16d8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f00ac0b23d0 0x7f00ac0b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00b16d8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00ac0b2d70 con 0x7f00ac1608e0 2023-12-20T04:44:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00b16d8640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac1618b0 con 0x7f00ac15cd70 2023-12-20T04:44:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00b16d8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f00ac0ce6c0 con 0x7f00ac0b23d0 2023-12-20T04:44:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00aaffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00ac15cd70 0x7f00ac0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00aaffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f00ac15cd70 0x7f00ac0cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38842/0 (socket says 172.21.15.32:38842) 2023-12-20T04:44:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00aaffd640 1 -- 172.21.15.32:0/980696749 learned_addr learned my addr 172.21.15.32:0/980696749 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00a3fff640 1 --2- 172.21.15.32:0/980696749 >> v2:172.21.15.39:3300/0 conn(0x7f00ac0b23d0 0x7f00ac0b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00aaffd640 1 -- 172.21.15.32:0/980696749 >> v2:172.21.15.39:3300/0 conn(0x7f00ac0b23d0 msgr2=0x7f00ac0b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:20.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00aaffd640 1 --2- 172.21.15.32:0/980696749 >> v2:172.21.15.39:3300/0 conn(0x7f00ac0b23d0 0x7f00ac0b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:20.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00aaffd640 1 -- 172.21.15.32:0/980696749 >> v1:172.21.15.39:6789/0 conn(0x7f00ac1608e0 legacy=0x7f00ac160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:20.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.553+0000 7f00aaffd640 1 -- 172.21.15.32:0/980696749 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00ac1615d0 con 0x7f00ac15cd70 2023-12-20T04:44:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00aaffd640 1 --2- 172.21.15.32:0/980696749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac15cd70 0x7f00ac0cf160 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0d08b0 tx=0x7f009c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0236bec8a79f9ba server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00aa7fc640 1 -- 172.21.15.32:0/980696749 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c0aa430 con 0x7f00ac15cd70 2023-12-20T04:44:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00aa7fc640 1 -- 172.21.15.32:0/980696749 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f009c0aa590 con 0x7f00ac15cd70 2023-12-20T04:44:20.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00aa7fc640 1 -- 172.21.15.32:0/980696749 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f009c0aa920 con 0x7f00ac15cd70 2023-12-20T04:44:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00b16d8640 1 -- 172.21.15.32:0/980696749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac15cd70 msgr2=0x7f00ac0cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00b16d8640 1 --2- 172.21.15.32:0/980696749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac15cd70 0x7f00ac0cf160 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f00ac0d08b0 tx=0x7f009c01c4d0 comp rx=0 tx=0).stop 2023-12-20T04:44:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00b16d8640 1 -- 172.21.15.32:0/980696749 shutdown_connections 2023-12-20T04:44:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00b16d8640 1 --2- 172.21.15.32:0/980696749 >> v2:172.21.15.39:3300/0 conn(0x7f00ac0b23d0 0x7f00ac0b2830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00b16d8640 1 --2- 172.21.15.32:0/980696749 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac15cd70 0x7f00ac0cf160 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00b16d8640 1 -- 172.21.15.32:0/980696749 >> 172.21.15.32:0/980696749 conn(0x7f00ac0cd3d0 msgr2=0x7f00ac0cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00b16d8640 1 -- 172.21.15.32:0/980696749 shutdown_connections 2023-12-20T04:44:20.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00b16d8640 1 -- 172.21.15.32:0/980696749 wait complete. 2023-12-20T04:44:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00b16d8640 1 Processor -- start 2023-12-20T04:44:20.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.557+0000 7f00b16d8640 1 -- start start 2023-12-20T04:44:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00b16d8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac0b23d0 0x7f00ac1560c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00b16d8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f00ac15cd70 0x7f00ac0b1710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00b16d8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac1608e0 0x7f00ac0b1c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00b16d8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0c5f10 con 0x7f00ac1608e0 2023-12-20T04:44:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00b16d8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0c6330 con 0x7f00ac0b23d0 2023-12-20T04:44:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00b16d8640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f00ac0c5290 con 0x7f00ac15cd70 2023-12-20T04:44:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac1608e0 0x7f00ac0b1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:20.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00ab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac1608e0 0x7f00ac0b1c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38852/0 (socket says 172.21.15.32:38852) 2023-12-20T04:44:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00ab7fe640 1 -- 172.21.15.32:0/961457233 learned_addr learned my addr 172.21.15.32:0/961457233 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00aaffd640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac0b23d0 0x7f00ac1560c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:20.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00a3fff640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f00ac15cd70 0x7f00ac0b1710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00ab7fe640 1 -- 172.21.15.32:0/961457233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f00ac15cd70 msgr2=0x7f00ac0b1710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00ab7fe640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f00ac15cd70 0x7f00ac0b1710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00ab7fe640 1 -- 172.21.15.32:0/961457233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac0b23d0 msgr2=0x7f00ac1560c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00ab7fe640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac0b23d0 0x7f00ac1560c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00ab7fe640 1 -- 172.21.15.32:0/961457233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f009c0094d0 con 0x7f00ac1608e0 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00ab7fe640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac1608e0 0x7f00ac0b1c50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c730 tx=0x7f00a400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00a37fe640 1 -- 172.21.15.32:0/961457233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a4003b80 con 0x7f00ac1608e0 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00a37fe640 1 -- 172.21.15.32:0/961457233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00a4003ce0 con 0x7f00ac1608e0 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00a37fe640 1 -- 172.21.15.32:0/961457233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00a400aa30 con 0x7f00ac1608e0 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00b16d8640 1 -- 172.21.15.32:0/961457233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f00ac0c5570 con 0x7f00ac1608e0 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00b16d8640 1 -- 172.21.15.32:0/961457233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f00ac0c5b30 con 0x7f00ac1608e0 2023-12-20T04:44:20.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.561+0000 7f00b16d8640 1 -- 172.21.15.32:0/961457233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f00ac04e000 con 0x7f00ac1608e0 2023-12-20T04:44:20.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.565+0000 7f00a37fe640 1 -- 172.21.15.32:0/961457233 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f00a400ab90 con 0x7f00ac1608e0 2023-12-20T04:44:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.565+0000 7f00a37fe640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f008c065bb0 0x7f008c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.565+0000 7f00a37fe640 1 -- 172.21.15.32:0/961457233 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f00a4082e70 con 0x7f00ac1608e0 2023-12-20T04:44:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.565+0000 7f00aaffd640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f008c065bb0 0x7f008c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:20.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.565+0000 7f00aaffd640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f008c065bb0 0x7f008c068070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f009c0ae000 tx=0x7f009c0b5040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:20.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.569+0000 7f00a37fe640 1 -- 172.21.15.32:0/961457233 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f00a4050860 con 0x7f00ac1608e0 2023-12-20T04:44:20.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:20.781+0000 7f00b16d8640 1 -- 172.21.15.32:0/961457233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f00ac167a50 con 0x7f00ac1608e0 2023-12-20T04:44:21.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.381+0000 7f00a37fe640 1 -- 172.21.15.32:0/961457233 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v589) v1 ==== 186+0+0 (secure 0 0 0) 0x7f00a4053e80 con 0x7f00ac1608e0 2023-12-20T04:44:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 -- 172.21.15.32:0/961457233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f008c065bb0 msgr2=0x7f008c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f008c065bb0 0x7f008c068070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f009c0ae000 tx=0x7f009c0b5040 comp rx=0 tx=0).stop 2023-12-20T04:44:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 -- 172.21.15.32:0/961457233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac1608e0 msgr2=0x7f00ac0b1c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac1608e0 0x7f00ac0b1c50 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f00a400c730 tx=0x7f00a400cbf0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 -- 172.21.15.32:0/961457233 shutdown_connections 2023-12-20T04:44:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f008c065bb0 0x7f008c068070 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f00ac1608e0 0x7f00ac0b1c50 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f00ac15cd70 0x7f00ac0b1710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 --2- 172.21.15.32:0/961457233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00ac0b23d0 0x7f00ac1560c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 -- 172.21.15.32:0/961457233 >> 172.21.15.32:0/961457233 conn(0x7f00ac0cd3d0 msgr2=0x7f00ac0cb140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 -- 172.21.15.32:0/961457233 shutdown_connections 2023-12-20T04:44:21.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.385+0000 7f00b16d8640 1 -- 172.21.15.32:0/961457233 wait complete. 2023-12-20T04:44:21.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T04:44:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.557+0000 7f4f37139640 1 Processor -- start 2023-12-20T04:44:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.557+0000 7f4f37139640 1 -- start start 2023-12-20T04:44:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f37139640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f300b0cb0 0x7f4f300caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f37139640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3015b6a0 con 0x7f4f30070230 2023-12-20T04:44:21.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f37139640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f3006d8c0 con 0x7f4f3015abb0 2023-12-20T04:44:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f37139640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4f3006e160 con 0x7f4f300b0cb0 2023-12-20T04:44:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f2ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f300b0cb0 0x7f4f300caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f2ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4f300b0cb0 0x7f4f300caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38854/0 (socket says 172.21.15.32:38854) 2023-12-20T04:44:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f2ffff640 1 -- 172.21.15.32:0/468526113 learned_addr learned my addr 172.21.15.32:0/468526113 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f2ffff640 1 -- 172.21.15.32:0/468526113 >> v1:172.21.15.39:6789/0 conn(0x7f4f3015abb0 legacy=0x7f4f3015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f2ffff640 1 -- 172.21.15.32:0/468526113 >> v1:172.21.15.32:6789/0 conn(0x7f4f30070230 legacy=0x7f4f3006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:21.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f2ffff640 1 -- 172.21.15.32:0/468526113 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f3006dba0 con 0x7f4f300b0cb0 2023-12-20T04:44:21.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f2f7fe640 1 -- 172.21.15.32:0/468526113 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4149599589 0 0) 0x7f4f3015b6a0 con 0x7f4f30070230 2023-12-20T04:44:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.561+0000 7f4f2ffff640 1 --2- 172.21.15.32:0/468526113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f300b0cb0 0x7f4f300caaf0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009870 tx=0x7f4f1c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c2562bde47cecd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f2f7fe640 1 -- 172.21.15.32:0/468526113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0a9400 con 0x7f4f300b0cb0 2023-12-20T04:44:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f2f7fe640 1 -- 172.21.15.32:0/468526113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f1c0a9560 con 0x7f4f300b0cb0 2023-12-20T04:44:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f2f7fe640 1 -- 172.21.15.32:0/468526113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f1c0a98f0 con 0x7f4f300b0cb0 2023-12-20T04:44:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 -- 172.21.15.32:0/468526113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f300b0cb0 msgr2=0x7f4f300caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 --2- 172.21.15.32:0/468526113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f300b0cb0 0x7f4f300caaf0 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c009870 tx=0x7f4f1c01c540 comp rx=0 tx=0).stop 2023-12-20T04:44:21.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 -- 172.21.15.32:0/468526113 shutdown_connections 2023-12-20T04:44:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 --2- 172.21.15.32:0/468526113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f300b0cb0 0x7f4f300caaf0 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 -- 172.21.15.32:0/468526113 >> 172.21.15.32:0/468526113 conn(0x7f4f3006a510 msgr2=0x7f4f300ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 -- 172.21.15.32:0/468526113 shutdown_connections 2023-12-20T04:44:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 -- 172.21.15.32:0/468526113 wait complete. 2023-12-20T04:44:21.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 Processor -- start 2023-12-20T04:44:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 -- start start 2023-12-20T04:44:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f30070230 0x7f4f3014bf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f300b0cb0 0x7f4f30154410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f3015abb0 0x7f4f300c6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f300c73a0 con 0x7f4f30070230 2023-12-20T04:44:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f300c7680 con 0x7f4f300b0cb0 2023-12-20T04:44:21.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f37139640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f300c7960 con 0x7f4f3015abb0 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f34ed5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f30070230 0x7f4f3014bf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.565+0000 7f4f34ed5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f30070230 0x7f4f3014bf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38864/0 (socket says 172.21.15.32:38864) 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f34ed5640 1 -- 172.21.15.32:0/1749224710 learned_addr learned my addr 172.21.15.32:0/1749224710 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f2ffff640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f300b0cb0 0x7f4f30154410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f356d6640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f3015abb0 0x7f4f300c6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f34ed5640 1 -- 172.21.15.32:0/1749224710 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f3015abb0 msgr2=0x7f4f300c6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f34ed5640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f3015abb0 0x7f4f300c6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f34ed5640 1 -- 172.21.15.32:0/1749224710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f300b0cb0 msgr2=0x7f4f30154410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f34ed5640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f300b0cb0 0x7f4f30154410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f34ed5640 1 -- 172.21.15.32:0/1749224710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f1c0094d0 con 0x7f4f30070230 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f34ed5640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f30070230 0x7f4f3014bf10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c3c0 tx=0x7f4f2400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f2d7fa640 1 -- 172.21.15.32:0/1749224710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f240052d0 con 0x7f4f30070230 2023-12-20T04:44:21.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f2d7fa640 1 -- 172.21.15.32:0/1749224710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f24005430 con 0x7f4f30070230 2023-12-20T04:44:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f2d7fa640 1 -- 172.21.15.32:0/1749224710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f24003840 con 0x7f4f30070230 2023-12-20T04:44:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f37139640 1 -- 172.21.15.32:0/1749224710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f30157bc0 con 0x7f4f30070230 2023-12-20T04:44:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f37139640 1 -- 172.21.15.32:0/1749224710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f300c8660 con 0x7f4f30070230 2023-12-20T04:44:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f37139640 1 -- 172.21.15.32:0/1749224710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f3004e000 con 0x7f4f30070230 2023-12-20T04:44:21.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.569+0000 7f4f2d7fa640 1 -- 172.21.15.32:0/1749224710 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4f240039a0 con 0x7f4f30070230 2023-12-20T04:44:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.573+0000 7f4f2d7fa640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f14065b80 0x7f4f14068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.573+0000 7f4f2d7fa640 1 -- 172.21.15.32:0/1749224710 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f4f24082710 con 0x7f4f30070230 2023-12-20T04:44:21.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.573+0000 7f4f2ffff640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f14065b80 0x7f4f14068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:21.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.573+0000 7f4f2ffff640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f14065b80 0x7f4f14068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0ae000 tx=0x7f4f1c0b3040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:21.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.577+0000 7f4f2d7fa640 1 -- 172.21.15.32:0/1749224710 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4f24050100 con 0x7f4f30070230 2023-12-20T04:44:21.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.737+0000 7f4f37139640 1 -- 172.21.15.32:0/1749224710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f4f30150af0 con 0x7f4f30070230 2023-12-20T04:44:21.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f2d7fa640 1 -- 172.21.15.32:0/1749224710 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v34)=0 v34) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4f24053720 con 0x7f4f30070230 2023-12-20T04:44:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 -- 172.21.15.32:0/1749224710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f14065b80 msgr2=0x7f4f14068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f14065b80 0x7f4f14068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4f1c0ae000 tx=0x7f4f1c0b3040 comp rx=0 tx=0).stop 2023-12-20T04:44:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 -- 172.21.15.32:0/1749224710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f30070230 msgr2=0x7f4f3014bf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f30070230 0x7f4f3014bf10 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f4f2400c3c0 tx=0x7f4f2400c880 comp rx=0 tx=0).stop 2023-12-20T04:44:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 -- 172.21.15.32:0/1749224710 shutdown_connections 2023-12-20T04:44:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f14065b80 0x7f4f14068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f3015abb0 0x7f4f300c6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f300b0cb0 0x7f4f30154410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 --2- 172.21.15.32:0/1749224710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f30070230 0x7f4f3014bf10 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 -- 172.21.15.32:0/1749224710 >> 172.21.15.32:0/1749224710 conn(0x7f4f3006a510 msgr2=0x7f4f3014fd60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 -- 172.21.15.32:0/1749224710 shutdown_connections 2023-12-20T04:44:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:21.745+0000 7f4f37139640 1 -- 172.21.15.32:0/1749224710 wait complete. 2023-12-20T04:44:21.766 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:44:21.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:44:21.822 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:44:21.920 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:44:21.921 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:44:21.921 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:44:21.930 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:44:21.931 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:44:21.931 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:44:21.942 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:44:21.942 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:44:21.942 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:44:21.983 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:44:21.984 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:44:21.984 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:44:21.994 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:44:21.994 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T04:44:21.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:44:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.173+0000 7fc4ae939640 1 Processor -- start 2023-12-20T04:44:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.173+0000 7fc4ae939640 1 -- start start 2023-12-20T04:44:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4ae939640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc4a806cf10 0x7fc4a80c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4ae939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4a8164ef0 0x7fc4a80c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4ae939640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4a80ae3f0 con 0x7fc4a80ae010 2023-12-20T04:44:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4ae939640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4a8161800 con 0x7fc4a806cf10 2023-12-20T04:44:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4ae939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8165380 con 0x7fc4a8164ef0 2023-12-20T04:44:22.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4a77fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc4a806cf10 0x7fc4a80c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4aced6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4a8164ef0 0x7fc4a80c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4a77fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc4a806cf10 0x7fc4a80c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:49438/0 (socket says 172.21.15.32:49438) 2023-12-20T04:44:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4a77fe640 1 -- 172.21.15.32:0/296093329 learned_addr learned my addr 172.21.15.32:0/296093329 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4a77fe640 1 -- 172.21.15.32:0/296093329 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4a8164ef0 msgr2=0x7fc4a80c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4a77fe640 1 --2- 172.21.15.32:0/296093329 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4a8164ef0 0x7fc4a80c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4a77fe640 1 -- 172.21.15.32:0/296093329 >> v1:172.21.15.39:6789/0 conn(0x7fc4a80ae010 legacy=0x7fc4a8071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:22.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.177+0000 7fc4a77fe640 1 -- 172.21.15.32:0/296093329 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4a8165660 con 0x7fc4a806cf10 2023-12-20T04:44:22.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4a77fe640 1 --2- 172.21.15.32:0/296093329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a806cf10 0x7fc4a80c94a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009870 tx=0x7fc49c01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ee2eba8c42fb56eb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4a6ffd640 1 -- 172.21.15.32:0/296093329 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0ab680 con 0x7fc4a806cf10 2023-12-20T04:44:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4a6ffd640 1 -- 172.21.15.32:0/296093329 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc49c0ab7e0 con 0x7fc4a806cf10 2023-12-20T04:44:22.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4a6ffd640 1 -- 172.21.15.32:0/296093329 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc49c0abb70 con 0x7fc4a806cf10 2023-12-20T04:44:22.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4ae939640 1 -- 172.21.15.32:0/296093329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a806cf10 msgr2=0x7fc4a80c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:22.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4ae939640 1 --2- 172.21.15.32:0/296093329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a806cf10 0x7fc4a80c94a0 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7fc49c009870 tx=0x7fc49c01c480 comp rx=0 tx=0).stop 2023-12-20T04:44:22.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4ae939640 1 -- 172.21.15.32:0/296093329 shutdown_connections 2023-12-20T04:44:22.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4ae939640 1 --2- 172.21.15.32:0/296093329 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4a8164ef0 0x7fc4a80c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4ae939640 1 --2- 172.21.15.32:0/296093329 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a806cf10 0x7fc4a80c94a0 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4ae939640 1 -- 172.21.15.32:0/296093329 >> 172.21.15.32:0/296093329 conn(0x7fc4a806a320 msgr2=0x7fc4a80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:22.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4ae939640 1 -- 172.21.15.32:0/296093329 shutdown_connections 2023-12-20T04:44:22.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4ae939640 1 -- 172.21.15.32:0/296093329 wait complete. 2023-12-20T04:44:22.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4ae939640 1 Processor -- start 2023-12-20T04:44:22.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.181+0000 7fc4ae939640 1 -- start start 2023-12-20T04:44:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4ae939640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80ae010 0x7fc4a80b3df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4ae939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4a80c2110 0x7fc4a80c2590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4ae939640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a80c2ad0 0x7fc4a81549b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4ae939640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a80b4330 con 0x7fc4a80ae010 2023-12-20T04:44:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4ae939640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a80c2f50 con 0x7fc4a80c2ad0 2023-12-20T04:44:22.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4ae939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4a8153910 con 0x7fc4a80c2110 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80ae010 0x7fc4a80b3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80ae010 0x7fc4a80b3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38874/0 (socket says 172.21.15.32:38874) 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a7fff640 1 -- 172.21.15.32:0/3753016158 learned_addr learned my addr 172.21.15.32:0/3753016158 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a77fe640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4a80c2110 0x7fc4a80c2590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4aced6640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a80c2ad0 0x7fc4a81549b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a7fff640 1 -- 172.21.15.32:0/3753016158 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4a80c2110 msgr2=0x7fc4a80c2590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a7fff640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4a80c2110 0x7fc4a80c2590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a7fff640 1 -- 172.21.15.32:0/3753016158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a80c2ad0 msgr2=0x7fc4a81549b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a7fff640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a80c2ad0 0x7fc4a81549b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a7fff640 1 -- 172.21.15.32:0/3753016158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc49c0094d0 con 0x7fc4a80ae010 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a7fff640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80ae010 0x7fc4a80b3df0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc49800c480 tx=0x7fc49800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/3753016158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4980052d0 con 0x7fc4a80ae010 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/3753016158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc498005430 con 0x7fc4a80ae010 2023-12-20T04:44:22.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4ae939640 1 -- 172.21.15.32:0/3753016158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4a8153bf0 con 0x7fc4a80ae010 2023-12-20T04:44:22.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/3753016158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc498003890 con 0x7fc4a80ae010 2023-12-20T04:44:22.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4ae939640 1 -- 172.21.15.32:0/3753016158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4a81541b0 con 0x7fc4a80ae010 2023-12-20T04:44:22.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.185+0000 7fc4ae939640 1 -- 172.21.15.32:0/3753016158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4a804e000 con 0x7fc4a80ae010 2023-12-20T04:44:22.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.189+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/3753016158 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc4980039f0 con 0x7fc4a80ae010 2023-12-20T04:44:22.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.189+0000 7fc4a4ff9640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc494065b80 0x7fc494068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.189+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/3753016158 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(98..98 src has 1..98) v4 ==== 5545+0+0 (secure 0 0 0) 0x7fc498082d30 con 0x7fc4a80ae010 2023-12-20T04:44:22.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.189+0000 7fc4a77fe640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc494065b80 0x7fc494068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.189+0000 7fc4a77fe640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc494065b80 0x7fc494068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0af000 tx=0x7fc49c0b6040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:22.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.193+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/3753016158 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc498050720 con 0x7fc4a80ae010 2023-12-20T04:44:22.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.381+0000 7fc4ae939640 1 -- 172.21.15.32:0/3753016158 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc4a80cf450 con 0x7fc4a80ae010 2023-12-20T04:44:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4a4ff9640 1 -- 172.21.15.32:0/3753016158 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 41 v41) v1 ==== 94+0+3520 (secure 0 0 0) 0x7fc498053d40 con 0x7fc4a80ae010 2023-12-20T04:44:22.384 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:22.384 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":41,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6119,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/655215798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":655215798},{"type":"v1","addr":"172.21.15.32:6837","nonce":655215798}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6122,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/661544180","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":661544180},{"type":"v1","addr":"172.21.15.39:6834","nonce":661544180}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":6855,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/833127896","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":833127896},{"type":"v1","addr":"172.21.15.32:6836","nonce":833127896}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":34},{"gid":7141,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4052774079","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4052774079},{"type":"v1","addr":"172.21.15.39:6835","nonce":4052774079}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":41}],"filesystems":[]} 2023-12-20T04:44:22.384 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 41 2023-12-20T04:44:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 -- 172.21.15.32:0/3753016158 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc494065b80 msgr2=0x7fc494068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc494065b80 0x7fc494068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fc49c0af000 tx=0x7fc49c0b6040 comp rx=0 tx=0).stop 2023-12-20T04:44:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 -- 172.21.15.32:0/3753016158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80ae010 msgr2=0x7fc4a80b3df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80ae010 0x7fc4a80b3df0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fc49800c480 tx=0x7fc49800c940 comp rx=0 tx=0).stop 2023-12-20T04:44:22.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 -- 172.21.15.32:0/3753016158 shutdown_connections 2023-12-20T04:44:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc494065b80 0x7fc494068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4a80c2ad0 0x7fc4a81549b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4a80c2110 0x7fc4a80c2590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 --2- 172.21.15.32:0/3753016158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4a80ae010 0x7fc4a80b3df0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 -- 172.21.15.32:0/3753016158 >> 172.21.15.32:0/3753016158 conn(0x7fc4a806a320 msgr2=0x7fc4a80cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 -- 172.21.15.32:0/3753016158 shutdown_connections 2023-12-20T04:44:22.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.385+0000 7fc4ae939640 1 -- 172.21.15.32:0/3753016158 wait complete. 2023-12-20T04:44:22.406 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T04:44:22.406 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T04:44:22.406 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T04:44:22.406 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:44:22.407 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T04:44:22.407 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T04:44:22.407 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T04:44:22.407 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:44:22.407 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T04:44:22.407 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T04:44:22.407 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T04:44:22.408 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:44:22.408 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T04:44:22.408 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T04:44:22.408 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T04:44:22.408 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:44:22.419 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T04:44:22.419 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T04:44:22.420 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T04:44:22.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T04:44:22.425 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T04:44:22.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T04:44:22.426 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T04:44:22.427 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T04:44:22.466 INFO:tasks.ceph.mds.a:Started 2023-12-20T04:44:22.466 DEBUG:teuthology.parallel:result is None 2023-12-20T04:44:22.468 INFO:tasks.ceph.mds.c:Started 2023-12-20T04:44:22.468 DEBUG:teuthology.parallel:result is None 2023-12-20T04:44:22.473 INFO:tasks.ceph.mds.d:Started 2023-12-20T04:44:22.473 DEBUG:teuthology.parallel:result is None 2023-12-20T04:44:22.475 INFO:tasks.ceph.mds.b:Started 2023-12-20T04:44:22.475 DEBUG:teuthology.parallel:result is None 2023-12-20T04:44:22.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T04:44:22.557 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T04:44:22.557 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T04:44:22.572 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T04:44:22.576 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T04:44:22.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.629+0000 7f7581918640 1 Processor -- start 2023-12-20T04:44:22.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.629+0000 7f7581918640 1 -- start start 2023-12-20T04:44:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f7581918640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f757c069d20 0x7f757c0cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f7581918640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757c15de00 0x7f757c06a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f7581918640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f757c15d970 con 0x7f757c15d080 2023-12-20T04:44:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f7581918640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757c0ce740 con 0x7f757c069d20 2023-12-20T04:44:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f7581918640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c06a640 con 0x7f757c15de00 2023-12-20T04:44:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f757affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f757c069d20 0x7f757c0cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f7573fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f757c15d080 0x7f757c0cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54246/0 (socket says 172.21.15.32:54246) 2023-12-20T04:44:22.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f7573fff640 1 -- 172.21.15.32:0/3899513192 learned_addr learned my addr 172.21.15.32:0/3899513192 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f757b7fe640 1 --2- 172.21.15.32:0/3899513192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757c15de00 0x7f757c06a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f757affd640 1 -- 172.21.15.32:0/3899513192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757c15de00 msgr2=0x7f757c06a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f757affd640 1 --2- 172.21.15.32:0/3899513192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757c15de00 0x7f757c06a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f757affd640 1 -- 172.21.15.32:0/3899513192 >> v1:172.21.15.32:6789/0 conn(0x7f757c15d080 legacy=0x7f757c0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f757affd640 1 -- 172.21.15.32:0/3899513192 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f757c0ce180 con 0x7f757c069d20 2023-12-20T04:44:22.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.633+0000 7f757a7fc640 1 -- 172.21.15.32:0/3899513192 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1608717422 0 0) 0x7f757c15d970 con 0x7f757c15d080 2023-12-20T04:44:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f757affd640 1 --2- 172.21.15.32:0/3899513192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757c069d20 0x7f757c0cbbb0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f756c0097c0 tx=0x7f756c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13abbc4910fe1dee server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f757a7fc640 1 -- 172.21.15.32:0/3899513192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0ab650 con 0x7f757c069d20 2023-12-20T04:44:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f757a7fc640 1 -- 172.21.15.32:0/3899513192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c0ab7b0 con 0x7f757c069d20 2023-12-20T04:44:22.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f757a7fc640 1 -- 172.21.15.32:0/3899513192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0abb40 con 0x7f757c069d20 2023-12-20T04:44:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 -- 172.21.15.32:0/3899513192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757c069d20 msgr2=0x7f757c0cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 --2- 172.21.15.32:0/3899513192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757c069d20 0x7f757c0cbbb0 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f756c0097c0 tx=0x7f756c01c450 comp rx=0 tx=0).stop 2023-12-20T04:44:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 -- 172.21.15.32:0/3899513192 shutdown_connections 2023-12-20T04:44:22.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 --2- 172.21.15.32:0/3899513192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757c15de00 0x7f757c06a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 --2- 172.21.15.32:0/3899513192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757c069d20 0x7f757c0cbbb0 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 -- 172.21.15.32:0/3899513192 >> 172.21.15.32:0/3899513192 conn(0x7f757c0cc310 msgr2=0x7f757c0cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:22.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 -- 172.21.15.32:0/3899513192 shutdown_connections 2023-12-20T04:44:22.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 -- 172.21.15.32:0/3899513192 wait complete. 2023-12-20T04:44:22.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 Processor -- start 2023-12-20T04:44:22.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 -- start start 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f757c069d20 0x7f757c0c5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757c15d080 0x7f757c0cb540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757c15de00 0x7f757c150ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0b1d30 con 0x7f757c15de00 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0c62a0 con 0x7f757c069d20 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.637+0000 7f7581918640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f757c0b5250 con 0x7f757c15d080 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f757b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757c15de00 0x7f757c150ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f757b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757c15de00 0x7f757c150ea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38942/0 (socket says 172.21.15.32:38942) 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f757b7fe640 1 -- 172.21.15.32:0/1145218508 learned_addr learned my addr 172.21.15.32:0/1145218508 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f7573fff640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757c15d080 0x7f757c0cb540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f757affd640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f757c069d20 0x7f757c0c5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f757b7fe640 1 -- 172.21.15.32:0/1145218508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757c15d080 msgr2=0x7f757c0cb540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f757b7fe640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757c15d080 0x7f757c0cb540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f757b7fe640 1 -- 172.21.15.32:0/1145218508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f757c069d20 msgr2=0x7f757c0c5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f757b7fe640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f757c069d20 0x7f757c0c5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f757b7fe640 1 -- 172.21.15.32:0/1145218508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c009420 con 0x7f757c15de00 2023-12-20T04:44:22.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f757b7fe640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757c15de00 0x7f757c150ea0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f757400efc0 tx=0x7f757400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f7578ff9640 1 -- 172.21.15.32:0/1145218508 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7574009ba0 con 0x7f757c15de00 2023-12-20T04:44:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f7578ff9640 1 -- 172.21.15.32:0/1145218508 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757400f040 con 0x7f757c15de00 2023-12-20T04:44:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f7578ff9640 1 -- 172.21.15.32:0/1145218508 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75740146c0 con 0x7f757c15de00 2023-12-20T04:44:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f7581918640 1 -- 172.21.15.32:0/1145218508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757c15e390 con 0x7f757c15de00 2023-12-20T04:44:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f7581918640 1 -- 172.21.15.32:0/1145218508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f757c0b1990 con 0x7f757c15de00 2023-12-20T04:44:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f7581918640 1 -- 172.21.15.32:0/1145218508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f757c0b1e70 con 0x7f757c15de00 2023-12-20T04:44:22.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.641+0000 7f7578ff9640 1 -- 172.21.15.32:0/1145218508 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7574014820 con 0x7f757c15de00 2023-12-20T04:44:22.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.645+0000 7f7578ff9640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 0x7f755c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:22.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.645+0000 7f7578ff9640 1 -- 172.21.15.32:0/1145218508 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(102..102 src has 1..102) v4 ==== 5545+0+0 (secure 0 0 0) 0x7f7574083fe0 con 0x7f757c15de00 2023-12-20T04:44:22.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.645+0000 7f757affd640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 0x7f755c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:22.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.645+0000 7f757affd640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 0x7f755c068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f756c0af000 tx=0x7f756c0a7000 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:22.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.649+0000 7f7578ff9640 1 -- 172.21.15.32:0/1145218508 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f757c0b1e70 con 0x7f757c15de00 2023-12-20T04:44:22.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:22.817+0000 7f7581918640 1 -- 172.21.15.32:0/1145218508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f757c0b1e70 con 0x7f757c15de00 2023-12-20T04:44:23.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.621+0000 7f7578ff9640 1 -- 172.21.15.32:0/1145218508 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v103) v1 ==== 97+0+0 (secure 0 0 0) 0x7f757c0b1e70 con 0x7f757c15de00 2023-12-20T04:44:23.621 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 -- 172.21.15.32:0/1145218508 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 msgr2=0x7f755c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 0x7f755c068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f756c0af000 tx=0x7f756c0a7000 comp rx=0 tx=0).stop 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 -- 172.21.15.32:0/1145218508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757c15de00 msgr2=0x7f757c150ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757c15de00 0x7f757c150ea0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f757400efc0 tx=0x7f757400c490 comp rx=0 tx=0).stop 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 -- 172.21.15.32:0/1145218508 shutdown_connections 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 0x7f755c068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757c15de00 0x7f757c150ea0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757c15d080 0x7f757c0cb540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 --2- 172.21.15.32:0/1145218508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f757c069d20 0x7f757c0c5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 -- 172.21.15.32:0/1145218508 >> 172.21.15.32:0/1145218508 conn(0x7f757c0cc310 msgr2=0x7f757c15f640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 -- 172.21.15.32:0/1145218508 shutdown_connections 2023-12-20T04:44:23.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.625+0000 7f7581918640 1 -- 172.21.15.32:0/1145218508 wait complete. 2023-12-20T04:44:23.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T04:44:23.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb0093b3640 1 Processor -- start 2023-12-20T04:44:23.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb0093b3640 1 -- start start 2023-12-20T04:44:23.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb0093b3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb00415cea0 0x7fb0040cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb0093b3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0040d0c90 con 0x7fb00415cac0 2023-12-20T04:44:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb0093b3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb004161110 con 0x7fb0040b84d0 2023-12-20T04:44:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb0093b3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb004160f10 con 0x7fb00415cea0 2023-12-20T04:44:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb002ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb00415cac0 0x7fb004160800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54252/0 (socket says 172.21.15.32:54252) 2023-12-20T04:44:23.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb002ffd640 1 -- 172.21.15.32:0/260440736 learned_addr learned my addr 172.21.15.32:0/260440736 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb0027fc640 1 --2- 172.21.15.32:0/260440736 >> v2:172.21.15.39:3300/0 conn(0x7fb00415cea0 0x7fb0040cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4092652016 0 0) 0x7fb0040d0c90 con 0x7fb00415cac0 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fafec003530 con 0x7fb00415cac0 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 570073439 0 0) 0x7fb004161110 con 0x7fb0040b84d0 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0040d0c90 con 0x7fb0040b84d0 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2220588571 0 0) 0x7fafec003530 con 0x7fb00415cac0 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 >> v2:172.21.15.39:3300/0 conn(0x7fb00415cea0 msgr2=0x7fb0040cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 --2- 172.21.15.32:0/260440736 >> v2:172.21.15.39:3300/0 conn(0x7fb00415cea0 0x7fb0040cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 >> v1:172.21.15.39:6789/0 conn(0x7fb0040b84d0 legacy=0x7fb0040d1450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0041613f0 con 0x7fb00415cac0 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7faff4001680 con 0x7fb00415cac0 2023-12-20T04:44:23.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 >> v1:172.21.15.32:6789/0 conn(0x7fb00415cac0 legacy=0x7fb004160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafec003b10 0x7fafec003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafec0047d0 0x7fafec004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fafec005190 0x7fafec00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb004160f10 con 0x7fafec003b10 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fafec00c0c0 con 0x7fafec0047d0 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fafec00c280 con 0x7fafec005190 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb002ffd640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafec003b10 0x7fafec003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb0037fe640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fafec005190 0x7fafec00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb0027fc640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafec0047d0 0x7fafec004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb002ffd640 1 -- 172.21.15.32:0/260440736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fafec005190 msgr2=0x7fafec00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb002ffd640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fafec005190 0x7fafec00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:23.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb002ffd640 1 -- 172.21.15.32:0/260440736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafec0047d0 msgr2=0x7fafec004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:23.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb002ffd640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafec0047d0 0x7fafec004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:23.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb002ffd640 1 -- 172.21.15.32:0/260440736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fafec003530 con 0x7fafec003b10 2023-12-20T04:44:23.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb0037fe640 1 -- 172.21.15.32:0/260440736 reap_dead start 2023-12-20T04:44:23.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.797+0000 7fb0027fc640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafec0047d0 0x7fafec004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:23.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.801+0000 7fb002ffd640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafec003b10 0x7fafec003f10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7faff400c190 tx=0x7faff401cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bebaf89bcf4681ed server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:23.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.801+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff40058e0 con 0x7fafec003b10 2023-12-20T04:44:23.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.801+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faff4005a40 con 0x7fafec003b10 2023-12-20T04:44:23.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.801+0000 7fb001ffb640 1 -- 172.21.15.32:0/260440736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faff4005e50 con 0x7fafec003b10 2023-12-20T04:44:23.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.801+0000 7fb0093b3640 1 -- 172.21.15.32:0/260440736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafec003b10 msgr2=0x7fafec003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:23.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.801+0000 7fb0093b3640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafec003b10 0x7fafec003f10 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7faff400c190 tx=0x7faff401cb90 comp rx=0 tx=0).stop 2023-12-20T04:44:23.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.801+0000 7fb0093b3640 1 -- 172.21.15.32:0/260440736 shutdown_connections 2023-12-20T04:44:23.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.801+0000 7fb0093b3640 1 --2- 172.21.15.32:0/260440736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fafec003b10 0x7fafec003f10 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:23.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.801+0000 7fb0093b3640 1 -- 172.21.15.32:0/260440736 >> 172.21.15.32:0/260440736 conn(0x7fb00406afd0 msgr2=0x7fb00406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:23.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.801+0000 7fb0093b3640 1 -- 172.21.15.32:0/260440736 shutdown_connections 2023-12-20T04:44:23.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.801+0000 7fb0093b3640 1 -- 172.21.15.32:0/260440736 wait complete. 2023-12-20T04:44:23.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0093b3640 1 Processor -- start 2023-12-20T04:44:23.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0093b3640 1 -- start start 2023-12-20T04:44:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0093b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafec003b10 0x7fb0041f98b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0093b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0040c55b0 0x7fb0040c7870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0093b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb00415d280 0x7fb00415d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:23.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0093b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040c65c0 con 0x7fb00415d280 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0093b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb00415dc40 con 0x7fafec003b10 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0093b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0040c73e0 con 0x7fb0040c55b0 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0037fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb00415d280 0x7fb00415d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0037fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb00415d280 0x7fb00415d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38956/0 (socket says 172.21.15.32:38956) 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0037fe640 1 -- 172.21.15.32:0/1416302143 learned_addr learned my addr 172.21.15.32:0/1416302143 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb002ffd640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafec003b10 0x7fb0041f98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0027fc640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0040c55b0 0x7fb0040c7870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0037fe640 1 -- 172.21.15.32:0/1416302143 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0040c55b0 msgr2=0x7fb0040c7870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0037fe640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0040c55b0 0x7fb0040c7870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0037fe640 1 -- 172.21.15.32:0/1416302143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafec003b10 msgr2=0x7fb0041f98b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0037fe640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafec003b10 0x7fb0041f98b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0037fe640 1 -- 172.21.15.32:0/1416302143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0040c67d0 con 0x7fb00415d280 2023-12-20T04:44:23.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0037fe640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb00415d280 0x7fb00415d700 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7faffc007fd0 tx=0x7faffc00ff70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fafeb7fe640 1 -- 172.21.15.32:0/1416302143 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc006d70 con 0x7fb00415d280 2023-12-20T04:44:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fafeb7fe640 1 -- 172.21.15.32:0/1416302143 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faffc003980 con 0x7fb00415d280 2023-12-20T04:44:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fafeb7fe640 1 -- 172.21.15.32:0/1416302143 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faffc003d30 con 0x7fb00415d280 2023-12-20T04:44:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0093b3640 1 -- 172.21.15.32:0/1416302143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0040c6ab0 con 0x7fb00415d280 2023-12-20T04:44:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.805+0000 7fb0093b3640 1 -- 172.21.15.32:0/1416302143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0040c7070 con 0x7fb00415d280 2023-12-20T04:44:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.809+0000 7fb0093b3640 1 -- 172.21.15.32:0/1416302143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb004074100 con 0x7fb00415d280 2023-12-20T04:44:23.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.809+0000 7fafeb7fe640 1 -- 172.21.15.32:0/1416302143 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faffc00d740 con 0x7fb00415d280 2023-12-20T04:44:23.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.809+0000 7fafeb7fe640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faff0065b80 0x7faff0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:23.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.809+0000 7fafeb7fe640 1 -- 172.21.15.32:0/1416302143 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7faffc082190 con 0x7fb00415d280 2023-12-20T04:44:23.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.809+0000 7fb002ffd640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faff0065b80 0x7faff0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:23.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.809+0000 7fb002ffd640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faff0065b80 0x7faff0068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faff40aa000 tx=0x7faff4001990 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:23.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:23.813+0000 7fafeb7fe640 1 -- 172.21.15.32:0/1416302143 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faffc04fcf0 con 0x7fb00415d280 2023-12-20T04:44:24.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.025+0000 7fb0093b3640 1 -- 172.21.15.32:0/1416302143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fb0040d07a0 con 0x7fb00415d280 2023-12-20T04:44:24.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fafeb7fe640 1 -- 172.21.15.32:0/1416302143 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v19) v1 ==== 80+0+5304 (secure 0 0 0) 0x7faffc053310 con 0x7fb00415d280 2023-12-20T04:44:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:24.027 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:44:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T04:44:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T04:44:24.027 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:44:24.028 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T04:44:24.030 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T04:44:24.032 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.034 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.035 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:44:24.036 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T04:44:24.037 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:44:24.038 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T04:44:24.039 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T04:44:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:44:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:44:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:44:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:44:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:24.040 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T04:44:24.040 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:44:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 -- 172.21.15.32:0/1416302143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faff0065b80 msgr2=0x7faff0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faff0065b80 0x7faff0068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faff40aa000 tx=0x7faff4001990 comp rx=0 tx=0).stop 2023-12-20T04:44:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 -- 172.21.15.32:0/1416302143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb00415d280 msgr2=0x7fb00415d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb00415d280 0x7fb00415d700 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7faffc007fd0 tx=0x7faffc00ff70 comp rx=0 tx=0).stop 2023-12-20T04:44:24.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 -- 172.21.15.32:0/1416302143 shutdown_connections 2023-12-20T04:44:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faff0065b80 0x7faff0068040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb00415d280 0x7fb00415d700 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0040c55b0 0x7fb0040c7870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 --2- 172.21.15.32:0/1416302143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fafec003b10 0x7fb0041f98b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 -- 172.21.15.32:0/1416302143 >> 172.21.15.32:0/1416302143 conn(0x7fb00406afd0 msgr2=0x7fb0040b71e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 -- 172.21.15.32:0/1416302143 shutdown_connections 2023-12-20T04:44:24.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.029+0000 7fb0093b3640 1 -- 172.21.15.32:0/1416302143 wait complete. 2023-12-20T04:44:24.049 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:44:24.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f144ba20640 1 Processor -- start 2023-12-20T04:44:24.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f144ba20640 1 -- start start 2023-12-20T04:44:24.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f144ba20640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14440ae010 0x7f1444071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f144ba20640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14440ae3f0 con 0x7f1444164ef0 2023-12-20T04:44:24.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f144ba20640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14440ca4b0 con 0x7f144406cf10 2023-12-20T04:44:24.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f144ba20640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1444161800 con 0x7f14440ae010 2023-12-20T04:44:24.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f14497bc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14440ae010 0x7f1444071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f1449fbd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1444164ef0 0x7f14440708d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54256/0 (socket says 172.21.15.32:54256) 2023-12-20T04:44:24.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f1449fbd640 1 -- 172.21.15.32:0/796668972 learned_addr learned my addr 172.21.15.32:0/796668972 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:24.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f14497bc640 1 -- 172.21.15.32:0/796668972 >> v1:172.21.15.39:6789/0 conn(0x7f144406cf10 legacy=0x7f14440701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f14497bc640 1 -- 172.21.15.32:0/796668972 >> v1:172.21.15.32:6789/0 conn(0x7f1444164ef0 legacy=0x7f14440708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f14497bc640 1 -- 172.21.15.32:0/796668972 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1444070fe0 con 0x7f14440ae010 2023-12-20T04:44:24.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.193+0000 7f143bfff640 1 -- 172.21.15.32:0/796668972 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3275707233 0 0) 0x7f14440ae3f0 con 0x7f1444164ef0 2023-12-20T04:44:24.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.197+0000 7f14497bc640 1 --2- 172.21.15.32:0/796668972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14440ae010 0x7f1444071470 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f14400097a0 tx=0x7f144001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ce7b6b18d77e2484 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:24.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.197+0000 7f143bfff640 1 -- 172.21.15.32:0/796668972 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14400ac600 con 0x7f14440ae010 2023-12-20T04:44:24.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.197+0000 7f143bfff640 1 -- 172.21.15.32:0/796668972 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14400a8050 con 0x7f14440ae010 2023-12-20T04:44:24.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.197+0000 7f143bfff640 1 -- 172.21.15.32:0/796668972 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14400ac980 con 0x7f14440ae010 2023-12-20T04:44:24.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.197+0000 7f144ba20640 1 -- 172.21.15.32:0/796668972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14440ae010 msgr2=0x7f1444071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.197+0000 7f144ba20640 1 --2- 172.21.15.32:0/796668972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14440ae010 0x7f1444071470 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f14400097a0 tx=0x7f144001c3a0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.197+0000 7f144ba20640 1 -- 172.21.15.32:0/796668972 shutdown_connections 2023-12-20T04:44:24.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.197+0000 7f144ba20640 1 --2- 172.21.15.32:0/796668972 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14440ae010 0x7f1444071470 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.197+0000 7f144ba20640 1 -- 172.21.15.32:0/796668972 >> 172.21.15.32:0/796668972 conn(0x7f144406a320 msgr2=0x7f14440ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:24.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.197+0000 7f144ba20640 1 -- 172.21.15.32:0/796668972 shutdown_connections 2023-12-20T04:44:24.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.197+0000 7f144ba20640 1 -- 172.21.15.32:0/796668972 wait complete. 2023-12-20T04:44:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f144ba20640 1 Processor -- start 2023-12-20T04:44:24.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f144ba20640 1 -- start start 2023-12-20T04:44:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f144ba20640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14440ae010 0x7f14440c4e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f144ba20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14441646f0 0x7f14440c2290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f144ba20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14440b1e40 0x7f144415c670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f144ba20640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14440c4440 con 0x7f14440ae010 2023-12-20T04:44:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f144ba20640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1444153580 con 0x7f14440b1e40 2023-12-20T04:44:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f144ba20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14440c49c0 con 0x7f14441646f0 2023-12-20T04:44:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f14497bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14440ae010 0x7f14440c4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f14497bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14440ae010 0x7f14440c4e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38972/0 (socket says 172.21.15.32:38972) 2023-12-20T04:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f14497bc640 1 -- 172.21.15.32:0/2011689823 learned_addr learned my addr 172.21.15.32:0/2011689823 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f1449fbd640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14440b1e40 0x7f144415c670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f1448fbb640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14441646f0 0x7f14440c2290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f14497bc640 1 -- 172.21.15.32:0/2011689823 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14441646f0 msgr2=0x7f14440c2290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f14497bc640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14441646f0 0x7f14440c2290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f14497bc640 1 -- 172.21.15.32:0/2011689823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14440b1e40 msgr2=0x7f144415c670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f14497bc640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14440b1e40 0x7f144415c670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f14497bc640 1 -- 172.21.15.32:0/2011689823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14400093b0 con 0x7f14440ae010 2023-12-20T04:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f14497bc640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14440ae010 0x7f14440c4e50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1440018fb0 tx=0x7f144001abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f1439ffb640 1 -- 172.21.15.32:0/2011689823 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1440004220 con 0x7f14440ae010 2023-12-20T04:44:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f1439ffb640 1 -- 172.21.15.32:0/2011689823 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14400ace90 con 0x7f14440ae010 2023-12-20T04:44:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f1439ffb640 1 -- 172.21.15.32:0/2011689823 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14400059a0 con 0x7f14440ae010 2023-12-20T04:44:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.201+0000 7f144ba20640 1 -- 172.21.15.32:0/2011689823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14440c2990 con 0x7f14440ae010 2023-12-20T04:44:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.205+0000 7f144ba20640 1 -- 172.21.15.32:0/2011689823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14440c4650 con 0x7f14440ae010 2023-12-20T04:44:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.205+0000 7f144ba20640 1 -- 172.21.15.32:0/2011689823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14440c4810 con 0x7f14440ae010 2023-12-20T04:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.205+0000 7f1439ffb640 1 -- 172.21.15.32:0/2011689823 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f14400b3070 con 0x7f14440ae010 2023-12-20T04:44:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.205+0000 7f1439ffb640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065b80 0x7f1428068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.205+0000 7f1439ffb640 1 -- 172.21.15.32:0/2011689823 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f144011c7f0 con 0x7f14440ae010 2023-12-20T04:44:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.209+0000 7f1448fbb640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065b80 0x7f1428068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.209+0000 7f1448fbb640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065b80 0x7f1428068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1444158ea0 tx=0x7f1434009040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:24.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.209+0000 7f1439ffb640 1 -- 172.21.15.32:0/2011689823 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f14400ea250 con 0x7f14440ae010 2023-12-20T04:44:24.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.369+0000 7f144ba20640 1 -- 172.21.15.32:0/2011689823 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f14440b33a0 con 0x7f14440ae010 2023-12-20T04:44:24.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.369+0000 7f1439ffb640 1 -- 172.21.15.32:0/2011689823 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v103) v1 ==== 74+0+12343 (secure 0 0 0) 0x7f14400ed870 con 0x7f14440ae010 2023-12-20T04:44:24.369 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:24.370 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":103,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:44:23.621127+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":11,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":91,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:44:24.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 -- 172.21.15.32:0/2011689823 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065b80 msgr2=0x7f1428068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065b80 0x7f1428068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f1444158ea0 tx=0x7f1434009040 comp rx=0 tx=0).stop 2023-12-20T04:44:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 -- 172.21.15.32:0/2011689823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14440ae010 msgr2=0x7f14440c4e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14440ae010 0x7f14440c4e50 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f1440018fb0 tx=0x7f144001abe0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 -- 172.21.15.32:0/2011689823 shutdown_connections 2023-12-20T04:44:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1428065b80 0x7f1428068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14440b1e40 0x7f144415c670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14441646f0 0x7f14440c2290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 --2- 172.21.15.32:0/2011689823 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14440ae010 0x7f14440c4e50 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 -- 172.21.15.32:0/2011689823 >> 172.21.15.32:0/2011689823 conn(0x7f144406a320 msgr2=0x7f14440cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 -- 172.21.15.32:0/2011689823 shutdown_connections 2023-12-20T04:44:24.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.373+0000 7f144ba20640 1 -- 172.21.15.32:0/2011689823 wait complete. 2023-12-20T04:44:24.391 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:44:24.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:44:24.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.537+0000 7fb54e7ce640 1 Processor -- start 2023-12-20T04:44:24.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.537+0000 7fb54e7ce640 1 -- start start 2023-12-20T04:44:24.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.537+0000 7fb54e7ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5481591e0 0x7fb5480caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.537+0000 7fb54e7ce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb54806cdf0 0x7fb548070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb54e7ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb548070770 0x7fb54815cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb54e7ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb5480cc3b0 con 0x7fb5481591e0 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb54e7ce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb548070ba0 con 0x7fb54806cdf0 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb54e7ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480c8e30 con 0x7fb548070770 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb547fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5481591e0 0x7fb5480caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb547fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb5481591e0 0x7fb5480caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38986/0 (socket says 172.21.15.32:38986) 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb547fff640 1 -- 172.21.15.32:0/1745863560 learned_addr learned my addr 172.21.15.32:0/1745863560 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb547fff640 1 -- 172.21.15.32:0/1745863560 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb548070770 msgr2=0x7fb54815cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb547fff640 1 --2- 172.21.15.32:0/1745863560 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb548070770 0x7fb54815cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb547fff640 1 -- 172.21.15.32:0/1745863560 >> v2:172.21.15.39:3300/0 conn(0x7fb54806cdf0 msgr2=0x7fb548070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:24.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb547fff640 1 --2- 172.21.15.32:0/1745863560 >> v2:172.21.15.39:3300/0 conn(0x7fb54806cdf0 0x7fb548070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb547fff640 1 -- 172.21.15.32:0/1745863560 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb54806d280 con 0x7fb5481591e0 2023-12-20T04:44:24.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb547fff640 1 --2- 172.21.15.32:0/1745863560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5481591e0 0x7fb5480caa30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb54815de70 tx=0x7fb53801e690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe30ca86f1c176e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb546ffd640 1 -- 172.21.15.32:0/1745863560 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5380b2600 con 0x7fb5481591e0 2023-12-20T04:44:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb546ffd640 1 -- 172.21.15.32:0/1745863560 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5380b2760 con 0x7fb5481591e0 2023-12-20T04:44:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.541+0000 7fb546ffd640 1 -- 172.21.15.32:0/1745863560 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5380b2af0 con 0x7fb5481591e0 2023-12-20T04:44:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 -- 172.21.15.32:0/1745863560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5481591e0 msgr2=0x7fb5480caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 --2- 172.21.15.32:0/1745863560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5481591e0 0x7fb5480caa30 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7fb54815de70 tx=0x7fb53801e690 comp rx=0 tx=0).stop 2023-12-20T04:44:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 -- 172.21.15.32:0/1745863560 shutdown_connections 2023-12-20T04:44:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 --2- 172.21.15.32:0/1745863560 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb548070770 0x7fb54815cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 --2- 172.21.15.32:0/1745863560 >> v2:172.21.15.39:3300/0 conn(0x7fb54806cdf0 0x7fb548070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 --2- 172.21.15.32:0/1745863560 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb5481591e0 0x7fb5480caa30 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 -- 172.21.15.32:0/1745863560 >> 172.21.15.32:0/1745863560 conn(0x7fb54806a510 msgr2=0x7fb5480ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 -- 172.21.15.32:0/1745863560 shutdown_connections 2023-12-20T04:44:24.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 -- 172.21.15.32:0/1745863560 wait complete. 2023-12-20T04:44:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 Processor -- start 2023-12-20T04:44:24.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 -- start start 2023-12-20T04:44:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb548070770 0x7fb5481448c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5481591e0 0x7fb5480c8720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb548144f40 0x7fb5480b0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb54814e1b0 con 0x7fb548070770 2023-12-20T04:44:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb548146130 con 0x7fb548144f40 2023-12-20T04:44:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54e7ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5480c8c60 con 0x7fb5481591e0 2023-12-20T04:44:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb547fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb548070770 0x7fb5481448c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb547fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb548070770 0x7fb5481448c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38998/0 (socket says 172.21.15.32:38998) 2023-12-20T04:44:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb547fff640 1 -- 172.21.15.32:0/3115374255 learned_addr learned my addr 172.21.15.32:0/3115374255 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54cd6b640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb548144f40 0x7fb5480b0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb5477fe640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5481591e0 0x7fb5480c8720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb547fff640 1 -- 172.21.15.32:0/3115374255 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5481591e0 msgr2=0x7fb5480c8720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb547fff640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5481591e0 0x7fb5480c8720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb547fff640 1 -- 172.21.15.32:0/3115374255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb548144f40 msgr2=0x7fb5480b0630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb547fff640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb548144f40 0x7fb5480b0630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb547fff640 1 -- 172.21.15.32:0/3115374255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5380094d0 con 0x7fb548070770 2023-12-20T04:44:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb54cd6b640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb548144f40 0x7fb5480b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:44:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.545+0000 7fb5477fe640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5481591e0 0x7fb5480c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:44:24.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.549+0000 7fb547fff640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb548070770 0x7fb5481448c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb538024c40 tx=0x7fb53801b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.549+0000 7fb544ff9640 1 -- 172.21.15.32:0/3115374255 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5380b2ce0 con 0x7fb548070770 2023-12-20T04:44:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.549+0000 7fb544ff9640 1 -- 172.21.15.32:0/3115374255 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5380b2e40 con 0x7fb548070770 2023-12-20T04:44:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.549+0000 7fb544ff9640 1 -- 172.21.15.32:0/3115374255 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5380aa910 con 0x7fb548070770 2023-12-20T04:44:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.549+0000 7fb54e7ce640 1 -- 172.21.15.32:0/3115374255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb54814e400 con 0x7fb548070770 2023-12-20T04:44:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.549+0000 7fb54e7ce640 1 -- 172.21.15.32:0/3115374255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb548145dc0 con 0x7fb548070770 2023-12-20T04:44:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.549+0000 7fb54e7ce640 1 -- 172.21.15.32:0/3115374255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb548145f50 con 0x7fb548070770 2023-12-20T04:44:24.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.549+0000 7fb544ff9640 1 -- 172.21.15.32:0/3115374255 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb5380aaa70 con 0x7fb548070770 2023-12-20T04:44:24.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.549+0000 7fb544ff9640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb534065bb0 0x7fb534068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.549+0000 7fb544ff9640 1 -- 172.21.15.32:0/3115374255 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fb53811d490 con 0x7fb548070770 2023-12-20T04:44:24.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.549+0000 7fb5477fe640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb534065bb0 0x7fb534068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.553+0000 7fb5477fe640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb534065bb0 0x7fb534068070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb5480c38e0 tx=0x7fb53c008040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:24.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.553+0000 7fb544ff9640 1 -- 172.21.15.32:0/3115374255 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb5380eb0a0 con 0x7fb548070770 2023-12-20T04:44:24.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 -- 172.21.15.32:0/3115374255 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb54804e000 con 0x7fb548070770 2023-12-20T04:44:24.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb544ff9640 1 -- 172.21.15.32:0/3115374255 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v103) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb5380ee6c0 con 0x7fb548070770 2023-12-20T04:44:24.712 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 -- 172.21.15.32:0/3115374255 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb534065bb0 msgr2=0x7fb534068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb534065bb0 0x7fb534068070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fb5480c38e0 tx=0x7fb53c008040 comp rx=0 tx=0).stop 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 -- 172.21.15.32:0/3115374255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb548070770 msgr2=0x7fb5481448c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb548070770 0x7fb5481448c0 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7fb538024c40 tx=0x7fb53801b430 comp rx=0 tx=0).stop 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 -- 172.21.15.32:0/3115374255 shutdown_connections 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb534065bb0 0x7fb534068070 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb548144f40 0x7fb5480b0630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5481591e0 0x7fb5480c8720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 --2- 172.21.15.32:0/3115374255 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb548070770 0x7fb5481448c0 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 -- 172.21.15.32:0/3115374255 >> 172.21.15.32:0/3115374255 conn(0x7fb54806a510 msgr2=0x7fb5481480e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 -- 172.21.15.32:0/3115374255 shutdown_connections 2023-12-20T04:44:24.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.713+0000 7fb54e7ce640 1 -- 172.21.15.32:0/3115374255 wait complete. 2023-12-20T04:44:24.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T04:44:24.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e64796640 1 Processor -- start 2023-12-20T04:44:24.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e64796640 1 -- start start 2023-12-20T04:44:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e64796640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5e5c0b2ae0 0x7f5e5c0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e64796640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5e5c1608e0 0x7f5e5c160d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e64796640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5e5c0ce6c0 con 0x7f5e5c15cd70 2023-12-20T04:44:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e64796640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c1618b0 con 0x7f5e5c0b2ae0 2023-12-20T04:44:24.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e64796640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5e5c1615d0 con 0x7f5e5c1608e0 2023-12-20T04:44:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e62532640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5e5c15cd70 0x7f5e5c0b23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54262/0 (socket says 172.21.15.32:54262) 2023-12-20T04:44:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e62532640 1 -- 172.21.15.32:0/1127953678 learned_addr learned my addr 172.21.15.32:0/1127953678 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e61d31640 1 --2- 172.21.15.32:0/1127953678 >> v2:172.21.15.32:3300/0 conn(0x7f5e5c0b2ae0 0x7f5e5c0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e62d33640 1 --2- 172.21.15.32:0/1127953678 >> v2:172.21.15.39:3300/0 conn(0x7f5e5c1608e0 0x7f5e5c160d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e61d31640 1 -- 172.21.15.32:0/1127953678 >> v2:172.21.15.39:3300/0 conn(0x7f5e5c1608e0 msgr2=0x7f5e5c160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e61d31640 1 --2- 172.21.15.32:0/1127953678 >> v2:172.21.15.39:3300/0 conn(0x7f5e5c1608e0 0x7f5e5c160d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e61d31640 1 -- 172.21.15.32:0/1127953678 >> v1:172.21.15.32:6789/0 conn(0x7f5e5c15cd70 legacy=0x7f5e5c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e61d31640 1 -- 172.21.15.32:0/1127953678 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e5c0ce960 con 0x7f5e5c0b2ae0 2023-12-20T04:44:24.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.877+0000 7f5e61530640 1 -- 172.21.15.32:0/1127953678 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2639581320 0 0) 0x7f5e5c0ce6c0 con 0x7f5e5c15cd70 2023-12-20T04:44:24.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e61d31640 1 --2- 172.21.15.32:0/1127953678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5c0b2ae0 0x7f5e5c0cf160 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0cf6a0 tx=0x7f5e4c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beefa728d3d31de5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:24.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e61530640 1 -- 172.21.15.32:0/1127953678 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0ab5f0 con 0x7f5e5c0b2ae0 2023-12-20T04:44:24.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e61530640 1 -- 172.21.15.32:0/1127953678 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e4c0ab750 con 0x7f5e5c0b2ae0 2023-12-20T04:44:24.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e61530640 1 -- 172.21.15.32:0/1127953678 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e4c0abae0 con 0x7f5e5c0b2ae0 2023-12-20T04:44:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e64796640 1 -- 172.21.15.32:0/1127953678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5c0b2ae0 msgr2=0x7f5e5c0cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e64796640 1 --2- 172.21.15.32:0/1127953678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5c0b2ae0 0x7f5e5c0cf160 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f5e5c0cf6a0 tx=0x7f5e4c01c430 comp rx=0 tx=0).stop 2023-12-20T04:44:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e64796640 1 -- 172.21.15.32:0/1127953678 shutdown_connections 2023-12-20T04:44:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e64796640 1 --2- 172.21.15.32:0/1127953678 >> v2:172.21.15.39:3300/0 conn(0x7f5e5c1608e0 0x7f5e5c160d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e64796640 1 --2- 172.21.15.32:0/1127953678 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5c0b2ae0 0x7f5e5c0cf160 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e64796640 1 -- 172.21.15.32:0/1127953678 >> 172.21.15.32:0/1127953678 conn(0x7f5e5c0cd3d0 msgr2=0x7f5e5c0cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e64796640 1 -- 172.21.15.32:0/1127953678 shutdown_connections 2023-12-20T04:44:24.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.881+0000 7f5e64796640 1 -- 172.21.15.32:0/1127953678 wait complete. 2023-12-20T04:44:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e64796640 1 Processor -- start 2023-12-20T04:44:24.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e64796640 1 -- start start 2023-12-20T04:44:24.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e64796640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5e5c0b2ae0 0x7f5e5c0c7250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e64796640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e5c15cd70 0x7f5e5c14a820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e64796640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5c1608e0 0x7f5e5c1f8f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e64796640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c0b11d0 con 0x7f5e5c1608e0 2023-12-20T04:44:24.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e64796640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c15b170 con 0x7f5e5c15cd70 2023-12-20T04:44:24.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e64796640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5e5c15b450 con 0x7f5e5c0b2ae0 2023-12-20T04:44:24.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e62d33640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5c1608e0 0x7f5e5c1f8f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e62d33640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5c1608e0 0x7f5e5c1f8f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39020/0 (socket says 172.21.15.32:39020) 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e62d33640 1 -- 172.21.15.32:0/1664411751 learned_addr learned my addr 172.21.15.32:0/1664411751 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e62532640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5e5c0b2ae0 0x7f5e5c0c7250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e61d31640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e5c15cd70 0x7f5e5c14a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e62d33640 1 -- 172.21.15.32:0/1664411751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5e5c0b2ae0 msgr2=0x7f5e5c0c7250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e62d33640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5e5c0b2ae0 0x7f5e5c0c7250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e62d33640 1 -- 172.21.15.32:0/1664411751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e5c15cd70 msgr2=0x7f5e5c14a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e62d33640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e5c15cd70 0x7f5e5c14a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e62d33640 1 -- 172.21.15.32:0/1664411751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5e4c009420 con 0x7f5e5c1608e0 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e61d31640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e5c15cd70 0x7f5e5c14a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e62d33640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5c1608e0 0x7f5e5c1f8f70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400efc0 tx=0x7f5e5400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e52ffd640 1 -- 172.21.15.32:0/1664411751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e54009ba0 con 0x7f5e5c1608e0 2023-12-20T04:44:24.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e52ffd640 1 -- 172.21.15.32:0/1664411751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5e5400f040 con 0x7f5e5c1608e0 2023-12-20T04:44:24.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e64796640 1 -- 172.21.15.32:0/1664411751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5e5c0b0b80 con 0x7f5e5c1608e0 2023-12-20T04:44:24.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e52ffd640 1 -- 172.21.15.32:0/1664411751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5e540146c0 con 0x7f5e5c1608e0 2023-12-20T04:44:24.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e64796640 1 -- 172.21.15.32:0/1664411751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5e5c0c7790 con 0x7f5e5c1608e0 2023-12-20T04:44:24.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.885+0000 7f5e64796640 1 -- 172.21.15.32:0/1664411751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5e5c0b1340 con 0x7f5e5c1608e0 2023-12-20T04:44:24.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.889+0000 7f5e52ffd640 1 -- 172.21.15.32:0/1664411751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5e54014820 con 0x7f5e5c1608e0 2023-12-20T04:44:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.889+0000 7f5e52ffd640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5e40065b80 0x7f5e40068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.889+0000 7f5e52ffd640 1 -- 172.21.15.32:0/1664411751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5e54083c30 con 0x7f5e5c1608e0 2023-12-20T04:44:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.889+0000 7f5e62532640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5e40065b80 0x7f5e40068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:24.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.889+0000 7f5e62532640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5e40065b80 0x7f5e40068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e58000f70 tx=0x7f5e58007b60 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:24.893+0000 7f5e52ffd640 1 -- 172.21.15.32:0/1664411751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5e54051680 con 0x7f5e5c1608e0 2023-12-20T04:44:25.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 -- 172.21.15.32:0/1664411751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5e5c04e000 con 0x7f5e5c1608e0 2023-12-20T04:44:25.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e52ffd640 1 -- 172.21.15.32:0/1664411751 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v103) v1 ==== 84+0+65 (secure 0 0 0) 0x7f5e54018060 con 0x7f5e5c1608e0 2023-12-20T04:44:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:25.056 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:44:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T04:44:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T04:44:25.056 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:25.056 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 -- 172.21.15.32:0/1664411751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5e40065b80 msgr2=0x7f5e40068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5e40065b80 0x7f5e40068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5e58000f70 tx=0x7f5e58007b60 comp rx=0 tx=0).stop 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 -- 172.21.15.32:0/1664411751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5c1608e0 msgr2=0x7f5e5c1f8f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5c1608e0 0x7f5e5c1f8f70 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f5e5400efc0 tx=0x7f5e5400c490 comp rx=0 tx=0).stop 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 -- 172.21.15.32:0/1664411751 shutdown_connections 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5e40065b80 0x7f5e40068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5e5c1608e0 0x7f5e5c1f8f70 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5e5c15cd70 0x7f5e5c14a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 --2- 172.21.15.32:0/1664411751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5e5c0b2ae0 0x7f5e5c0c7250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 -- 172.21.15.32:0/1664411751 >> 172.21.15.32:0/1664411751 conn(0x7f5e5c0cd3d0 msgr2=0x7f5e5c0cc560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 -- 172.21.15.32:0/1664411751 shutdown_connections 2023-12-20T04:44:25.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.057+0000 7f5e64796640 1 -- 172.21.15.32:0/1664411751 wait complete. 2023-12-20T04:44:25.077 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T04:44:25.077 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T04:44:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3f4c3640 1 Processor -- start 2023-12-20T04:44:25.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3f4c3640 1 -- start start 2023-12-20T04:44:25.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3f4c3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed3815b790 0x7fed38070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3f4c3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fed380ce0c0 0x7fed3806c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3f4c3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fed38161670 con 0x7fed38070770 2023-12-20T04:44:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3f4c3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fed380ce6a0 con 0x7fed380ce0c0 2023-12-20T04:44:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3f4c3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed38162270 con 0x7fed3815b790 2023-12-20T04:44:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3ca5e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fed38070770 0x7fed380cd9b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54272/0 (socket says 172.21.15.32:54272) 2023-12-20T04:44:25.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3ca5e640 1 -- 172.21.15.32:0/883011600 learned_addr learned my addr 172.21.15.32:0/883011600 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3da60640 1 --2- 172.21.15.32:0/883011600 >> v2:172.21.15.39:3300/0 conn(0x7fed380ce0c0 0x7fed3806c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3d25f640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed3815b790 0x7fed38070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3044124315 0 0) 0x7fed38161670 con 0x7fed38070770 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fed24003530 con 0x7fed38070770 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2456770440 0 0) 0x7fed24003530 con 0x7fed38070770 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed3815b790 msgr2=0x7fed38070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed3815b790 0x7fed38070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 >> v2:172.21.15.39:3300/0 conn(0x7fed380ce0c0 msgr2=0x7fed3806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 --2- 172.21.15.32:0/883011600 >> v2:172.21.15.39:3300/0 conn(0x7fed380ce0c0 0x7fed3806c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed38162690 con 0x7fed38070770 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fed28001680 con 0x7fed38070770 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 >> v1:172.21.15.32:6789/0 conn(0x7fed38070770 legacy=0x7fed380cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed24003980 0x7fed24003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:25.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24004370 0x7fed24004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:25.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed24004ec0 0x7fed2400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:25.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed380ce6a0 con 0x7fed24004370 2023-12-20T04:44:25.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed38162270 con 0x7fed24004ec0 2023-12-20T04:44:25.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed2400bed0 con 0x7fed24003980 2023-12-20T04:44:25.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3ca5e640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24004370 0x7fed24004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:25.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.225+0000 7fed3d25f640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed24003980 0x7fed24003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:25.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed3da60640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed24004ec0 0x7fed2400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:25.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed3ca5e640 1 -- 172.21.15.32:0/883011600 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed24003980 msgr2=0x7fed24003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:25.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed3ca5e640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed24003980 0x7fed24003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:25.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed3ca5e640 1 -- 172.21.15.32:0/883011600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed24004ec0 msgr2=0x7fed2400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:25.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed3ca5e640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed24004ec0 0x7fed2400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed3ca5e640 1 -- 172.21.15.32:0/883011600 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed24003530 con 0x7fed24004370 2023-12-20T04:44:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed3ca5e640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24004370 0x7fed24004850 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fed2800c210 tx=0x7fed2801f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b14760c045273c2e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28005ae0 con 0x7fed24004370 2023-12-20T04:44:25.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed280ab070 con 0x7fed24004370 2023-12-20T04:44:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed2ffff640 1 -- 172.21.15.32:0/883011600 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28007070 con 0x7fed24004370 2023-12-20T04:44:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed3f4c3640 1 -- 172.21.15.32:0/883011600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24004370 msgr2=0x7fed24004850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:25.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed3f4c3640 1 --2- 172.21.15.32:0/883011600 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed24004370 0x7fed24004850 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7fed2800c210 tx=0x7fed2801f3b0 comp rx=0 tx=0).stop 2023-12-20T04:44:25.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.229+0000 7fed3da60640 1 -- 172.21.15.32:0/883011600 reap_dead start 2023-12-20T04:44:25.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 -- 172.21.15.32:0/883011600 shutdown_connections 2023-12-20T04:44:25.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 -- 172.21.15.32:0/883011600 >> 172.21.15.32:0/883011600 conn(0x7fed3806a510 msgr2=0x7fed380ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:25.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 -- 172.21.15.32:0/883011600 shutdown_connections 2023-12-20T04:44:25.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 -- 172.21.15.32:0/883011600 wait complete. 2023-12-20T04:44:25.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 Processor -- start 2023-12-20T04:44:25.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 -- start start 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed24004370 0x7fed3815cf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed3815e940 0x7fed38163890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed3815eda0 0x7fed38163dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed38155910 con 0x7fed3815e940 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fed3815c0f0 con 0x7fed24004370 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3f4c3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fed380c6ce0 con 0x7fed3815eda0 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3ca5e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed3815e940 0x7fed38163890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3ca5e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed3815e940 0x7fed38163890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39032/0 (socket says 172.21.15.32:39032) 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3ca5e640 1 -- 172.21.15.32:0/4217440101 learned_addr learned my addr 172.21.15.32:0/4217440101 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3d25f640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed24004370 0x7fed3815cf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3da60640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed3815eda0 0x7fed38163dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3ca5e640 1 -- 172.21.15.32:0/4217440101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed3815eda0 msgr2=0x7fed38163dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3ca5e640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed3815eda0 0x7fed38163dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:25.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3ca5e640 1 -- 172.21.15.32:0/4217440101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed24004370 msgr2=0x7fed3815cf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:25.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3ca5e640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed24004370 0x7fed3815cf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:25.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.233+0000 7fed3ca5e640 1 -- 172.21.15.32:0/4217440101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fed380c6fc0 con 0x7fed3815e940 2023-12-20T04:44:25.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.237+0000 7fed3ca5e640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed3815e940 0x7fed38163890 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fed2801f1e0 tx=0x7fed24005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:25.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.237+0000 7fed2dffb640 1 -- 172.21.15.32:0/4217440101 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed280b7070 con 0x7fed3815e940 2023-12-20T04:44:25.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.237+0000 7fed2dffb640 1 -- 172.21.15.32:0/4217440101 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fed280048b0 con 0x7fed3815e940 2023-12-20T04:44:25.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.237+0000 7fed3f4c3640 1 -- 172.21.15.32:0/4217440101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fed380c72a0 con 0x7fed3815e940 2023-12-20T04:44:25.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.237+0000 7fed2dffb640 1 -- 172.21.15.32:0/4217440101 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fed28004c40 con 0x7fed3815e940 2023-12-20T04:44:25.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.237+0000 7fed3f4c3640 1 -- 172.21.15.32:0/4217440101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fed3815c3d0 con 0x7fed3815e940 2023-12-20T04:44:25.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.237+0000 7fed3f4c3640 1 -- 172.21.15.32:0/4217440101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fed3804e000 con 0x7fed3815e940 2023-12-20T04:44:25.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.237+0000 7fed2dffb640 1 -- 172.21.15.32:0/4217440101 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fed280ad020 con 0x7fed3815e940 2023-12-20T04:44:25.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.237+0000 7fed2dffb640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed1c065bb0 0x7fed1c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:25.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.241+0000 7fed2dffb640 1 -- 172.21.15.32:0/4217440101 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(103..103 src has 1..103) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fed2811f4b0 con 0x7fed3815e940 2023-12-20T04:44:25.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.241+0000 7fed3d25f640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed1c065bb0 0x7fed1c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:25.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.241+0000 7fed3d25f640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed1c065bb0 0x7fed1c068070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fed3806e380 tx=0x7fed34009190 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:25.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.241+0000 7fed2dffb640 1 -- 172.21.15.32:0/4217440101 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fed280f09b0 con 0x7fed3815e940 2023-12-20T04:44:25.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:25.401+0000 7fed3f4c3640 1 -- 172.21.15.32:0/4217440101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fed380c7430 con 0x7fed3815e940 2023-12-20T04:44:26.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.405+0000 7fed2dffb640 1 -- 172.21.15.32:0/4217440101 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v104) v1 ==== 138+0+0 (secure 0 0 0) 0x7fed280f0540 con 0x7fed3815e940 2023-12-20T04:44:26.404 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T04:44:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 -- 172.21.15.32:0/4217440101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed1c065bb0 msgr2=0x7fed1c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed1c065bb0 0x7fed1c068070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fed3806e380 tx=0x7fed34009190 comp rx=0 tx=0).stop 2023-12-20T04:44:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 -- 172.21.15.32:0/4217440101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed3815e940 msgr2=0x7fed38163890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed3815e940 0x7fed38163890 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7fed2801f1e0 tx=0x7fed24005800 comp rx=0 tx=0).stop 2023-12-20T04:44:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 -- 172.21.15.32:0/4217440101 shutdown_connections 2023-12-20T04:44:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fed1c065bb0 0x7fed1c068070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fed3815eda0 0x7fed38163dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fed3815e940 0x7fed38163890 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 --2- 172.21.15.32:0/4217440101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fed24004370 0x7fed3815cf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:26.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 -- 172.21.15.32:0/4217440101 >> 172.21.15.32:0/4217440101 conn(0x7fed3806a510 msgr2=0x7fed380c4830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 -- 172.21.15.32:0/4217440101 shutdown_connections 2023-12-20T04:44:26.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.409+0000 7fed3f4c3640 1 -- 172.21.15.32:0/4217440101 wait complete. 2023-12-20T04:44:26.434 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T04:44:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.589+0000 7f492736d640 1 Processor -- start 2023-12-20T04:44:26.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.589+0000 7f492736d640 1 -- start start 2023-12-20T04:44:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f492736d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f49200cd750 0x7f49201655e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f492736d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49200cc710 con 0x7f492006d350 2023-12-20T04:44:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f492736d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49200cc990 con 0x7f49200adca0 2023-12-20T04:44:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f492736d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49200cdb50 con 0x7f49200cd750 2023-12-20T04:44:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f4924908640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f492006d350 0x7f4920164ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54286/0 (socket says 172.21.15.32:54286) 2023-12-20T04:44:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f4924908640 1 -- 172.21.15.32:0/3969736832 learned_addr learned my addr 172.21.15.32:0/3969736832 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:26.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f492590a640 1 --2- 172.21.15.32:0/3969736832 >> v2:172.21.15.32:3300/0 conn(0x7f49200cd750 0x7f49201655e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f4917fff640 1 -- 172.21.15.32:0/3969736832 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3277434185 0 0) 0x7f49200cc710 con 0x7f492006d350 2023-12-20T04:44:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f4917fff640 1 -- 172.21.15.32:0/3969736832 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f490c003530 con 0x7f492006d350 2023-12-20T04:44:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f492590a640 1 -- 172.21.15.32:0/3969736832 >> v1:172.21.15.39:6789/0 conn(0x7f49200adca0 legacy=0x7f492006cc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:26.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f492590a640 1 -- 172.21.15.32:0/3969736832 >> v1:172.21.15.32:6789/0 conn(0x7f492006d350 legacy=0x7f4920164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:26.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.593+0000 7f492590a640 1 -- 172.21.15.32:0/3969736832 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49200ccc70 con 0x7f49200cd750 2023-12-20T04:44:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f492590a640 1 --2- 172.21.15.32:0/3969736832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49200cd750 0x7f49201655e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4918009d90 tx=0x7f491801c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ead98c2a359fc52c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:26.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f4917fff640 1 -- 172.21.15.32:0/3969736832 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49180a5870 con 0x7f49200cd750 2023-12-20T04:44:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f4917fff640 1 -- 172.21.15.32:0/3969736832 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49180a59d0 con 0x7f49200cd750 2023-12-20T04:44:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f4917fff640 1 -- 172.21.15.32:0/3969736832 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49180a5d60 con 0x7f49200cd750 2023-12-20T04:44:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f492736d640 1 -- 172.21.15.32:0/3969736832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49200cd750 msgr2=0x7f49201655e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:26.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f492736d640 1 --2- 172.21.15.32:0/3969736832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49200cd750 0x7f49201655e0 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f4918009d90 tx=0x7f491801c8c0 comp rx=0 tx=0).stop 2023-12-20T04:44:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f492736d640 1 -- 172.21.15.32:0/3969736832 shutdown_connections 2023-12-20T04:44:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f492736d640 1 --2- 172.21.15.32:0/3969736832 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49200cd750 0x7f49201655e0 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f492736d640 1 -- 172.21.15.32:0/3969736832 >> 172.21.15.32:0/3969736832 conn(0x7f492006a4b0 msgr2=0x7f49200b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f492736d640 1 -- 172.21.15.32:0/3969736832 shutdown_connections 2023-12-20T04:44:26.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f492736d640 1 -- 172.21.15.32:0/3969736832 wait complete. 2023-12-20T04:44:26.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f492736d640 1 Processor -- start 2023-12-20T04:44:26.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.597+0000 7f492736d640 1 -- start start 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492736d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492006d350 0x7f4920160040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492736d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49200adca0 0x7f49200b1c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492736d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49200cd750 0x7f4920160620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492736d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49200b2380 con 0x7f49200cd750 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492736d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f492015df60 con 0x7f49200adca0 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492736d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f492015d670 con 0x7f492006d350 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492590a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49200cd750 0x7f4920160620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492590a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49200cd750 0x7f4920160620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39044/0 (socket says 172.21.15.32:39044) 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492590a640 1 -- 172.21.15.32:0/1944732475 learned_addr learned my addr 172.21.15.32:0/1944732475 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f4925109640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492006d350 0x7f4920160040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f4924908640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49200adca0 0x7f49200b1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:26.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492590a640 1 -- 172.21.15.32:0/1944732475 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492006d350 msgr2=0x7f4920160040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492590a640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492006d350 0x7f4920160040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492590a640 1 -- 172.21.15.32:0/1944732475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49200adca0 msgr2=0x7f49200b1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492590a640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49200adca0 0x7f49200b1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492590a640 1 -- 172.21.15.32:0/1944732475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4918009a70 con 0x7f49200cd750 2023-12-20T04:44:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492590a640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49200cd750 0x7f4920160620 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f49180b0040 tx=0x7f4918019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f4915ffb640 1 -- 172.21.15.32:0/1944732475 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49180ad850 con 0x7f49200cd750 2023-12-20T04:44:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f4915ffb640 1 -- 172.21.15.32:0/1944732475 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49180ad9b0 con 0x7f49200cd750 2023-12-20T04:44:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f4915ffb640 1 -- 172.21.15.32:0/1944732475 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49180add20 con 0x7f49200cd750 2023-12-20T04:44:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492736d640 1 -- 172.21.15.32:0/1944732475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f492014fc80 con 0x7f49200cd750 2023-12-20T04:44:26.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492736d640 1 -- 172.21.15.32:0/1944732475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f492015dbf0 con 0x7f49200cd750 2023-12-20T04:44:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.601+0000 7f492736d640 1 -- 172.21.15.32:0/1944732475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f492015e110 con 0x7f49200cd750 2023-12-20T04:44:26.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.605+0000 7f4915ffb640 1 -- 172.21.15.32:0/1944732475 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f49180b4050 con 0x7f49200cd750 2023-12-20T04:44:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.605+0000 7f4915ffb640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4904065b80 0x7f4904068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.605+0000 7f4915ffb640 1 -- 172.21.15.32:0/1944732475 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(104..104 src has 1..104) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f491811e0d0 con 0x7f49200cd750 2023-12-20T04:44:26.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.605+0000 7f4925109640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4904065b80 0x7f4904068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:26.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.605+0000 7f4925109640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4904065b80 0x7f4904068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f491c007fd0 tx=0x7f491c009040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:26.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.609+0000 7f4915ffb640 1 -- 172.21.15.32:0/1944732475 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f49180ebcf0 con 0x7f49200cd750 2023-12-20T04:44:26.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:26.769+0000 7f492736d640 1 -- 172.21.15.32:0/1944732475 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f492015e110 con 0x7f49200cd750 2023-12-20T04:44:27.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.409+0000 7f4915ffb640 1 -- 172.21.15.32:0/1944732475 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v105) v1 ==== 170+0+0 (secure 0 0 0) 0x7f492015e110 con 0x7f49200cd750 2023-12-20T04:44:27.410 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T04:44:27.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.421+0000 7f492736d640 1 -- 172.21.15.32:0/1944732475 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4904065b80 msgr2=0x7f4904068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:27.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.421+0000 7f492736d640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4904065b80 0x7f4904068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f491c007fd0 tx=0x7f491c009040 comp rx=0 tx=0).stop 2023-12-20T04:44:27.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.421+0000 7f492736d640 1 -- 172.21.15.32:0/1944732475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49200cd750 msgr2=0x7f4920160620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:27.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.421+0000 7f492736d640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49200cd750 0x7f4920160620 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f49180b0040 tx=0x7f4918019660 comp rx=0 tx=0).stop 2023-12-20T04:44:27.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.425+0000 7f492736d640 1 -- 172.21.15.32:0/1944732475 shutdown_connections 2023-12-20T04:44:27.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.425+0000 7f492736d640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4904065b80 0x7f4904068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.425+0000 7f492736d640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49200cd750 0x7f4920160620 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.425+0000 7f492736d640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49200adca0 0x7f49200b1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.425+0000 7f492736d640 1 --2- 172.21.15.32:0/1944732475 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f492006d350 0x7f4920160040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.425+0000 7f492736d640 1 -- 172.21.15.32:0/1944732475 >> 172.21.15.32:0/1944732475 conn(0x7f492006a4b0 msgr2=0x7f49200af630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:27.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.425+0000 7f492736d640 1 -- 172.21.15.32:0/1944732475 shutdown_connections 2023-12-20T04:44:27.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.425+0000 7f492736d640 1 -- 172.21.15.32:0/1944732475 wait complete. 2023-12-20T04:44:27.456 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T04:44:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.605+0000 7f992068c640 1 Processor -- start 2023-12-20T04:44:27.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.605+0000 7f992068c640 1 -- start start 2023-12-20T04:44:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f992068c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f991815abb0 0x7f99180caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f992068c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99180b0cb0 0x7f991815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f992068c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f991815b6a0 con 0x7f9918070230 2023-12-20T04:44:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f992068c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991806e160 con 0x7f99180b0cb0 2023-12-20T04:44:27.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f992068c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f991815c070 con 0x7f991815abb0 2023-12-20T04:44:27.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f991dc27640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99180b0cb0 0x7f991815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:27.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f991dc27640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f99180b0cb0 0x7f991815bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39052/0 (socket says 172.21.15.32:39052) 2023-12-20T04:44:27.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f991dc27640 1 -- 172.21.15.32:0/3841537054 learned_addr learned my addr 172.21.15.32:0/3841537054 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:27.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f991e428640 1 --2- 172.21.15.32:0/3841537054 >> v2:172.21.15.39:3300/0 conn(0x7f991815abb0 0x7f99180caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:27.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f991dc27640 1 -- 172.21.15.32:0/3841537054 >> v2:172.21.15.39:3300/0 conn(0x7f991815abb0 msgr2=0x7f99180caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:27.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f991dc27640 1 --2- 172.21.15.32:0/3841537054 >> v2:172.21.15.39:3300/0 conn(0x7f991815abb0 0x7f99180caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f991dc27640 1 -- 172.21.15.32:0/3841537054 >> v1:172.21.15.39:6789/0 conn(0x7f9918070230 legacy=0x7f991806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:27.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.609+0000 7f991dc27640 1 -- 172.21.15.32:0/3841537054 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f991806d8c0 con 0x7f99180b0cb0 2023-12-20T04:44:27.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f991dc27640 1 --2- 172.21.15.32:0/3841537054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99180b0cb0 0x7f991815bb30 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f99180713f0 tx=0x7f990801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c5941a73e67cac7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:27.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f991d426640 1 -- 172.21.15.32:0/3841537054 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99080aa660 con 0x7f99180b0cb0 2023-12-20T04:44:27.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f991d426640 1 -- 172.21.15.32:0/3841537054 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f99080aa7c0 con 0x7f99180b0cb0 2023-12-20T04:44:27.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f991d426640 1 -- 172.21.15.32:0/3841537054 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f99080aab50 con 0x7f99180b0cb0 2023-12-20T04:44:27.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f992068c640 1 -- 172.21.15.32:0/3841537054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99180b0cb0 msgr2=0x7f991815bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:27.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f992068c640 1 --2- 172.21.15.32:0/3841537054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99180b0cb0 0x7f991815bb30 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f99180713f0 tx=0x7f990801c500 comp rx=0 tx=0).stop 2023-12-20T04:44:27.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f992068c640 1 -- 172.21.15.32:0/3841537054 shutdown_connections 2023-12-20T04:44:27.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f992068c640 1 --2- 172.21.15.32:0/3841537054 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99180b0cb0 0x7f991815bb30 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f992068c640 1 --2- 172.21.15.32:0/3841537054 >> v2:172.21.15.39:3300/0 conn(0x7f991815abb0 0x7f99180caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f992068c640 1 -- 172.21.15.32:0/3841537054 >> 172.21.15.32:0/3841537054 conn(0x7f991806a510 msgr2=0x7f99180ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:27.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f992068c640 1 -- 172.21.15.32:0/3841537054 shutdown_connections 2023-12-20T04:44:27.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f992068c640 1 -- 172.21.15.32:0/3841537054 wait complete. 2023-12-20T04:44:27.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f992068c640 1 Processor -- start 2023-12-20T04:44:27.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.613+0000 7f992068c640 1 -- start start 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f992068c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9918070230 0x7f9918150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f992068c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99180b0cb0 0x7f99180a7720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f992068c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f991815abb0 0x7f991814f450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f992068c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9918158b60 con 0x7f9918070230 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f992068c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9918150300 con 0x7f99180b0cb0 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f992068c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9918152250 con 0x7f991815abb0 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f991e428640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9918070230 0x7f9918150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f991e428640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9918070230 0x7f9918150500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39054/0 (socket says 172.21.15.32:39054) 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f991e428640 1 -- 172.21.15.32:0/1029631890 learned_addr learned my addr 172.21.15.32:0/1029631890 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f991dc27640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99180b0cb0 0x7f99180a7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f991ec29640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f991815abb0 0x7f991814f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:27.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f991e428640 1 -- 172.21.15.32:0/1029631890 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f991815abb0 msgr2=0x7f991814f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f991e428640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f991815abb0 0x7f991814f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f991e428640 1 -- 172.21.15.32:0/1029631890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99180b0cb0 msgr2=0x7f99180a7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f991e428640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99180b0cb0 0x7f99180a7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f991e428640 1 -- 172.21.15.32:0/1029631890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f99080094d0 con 0x7f9918070230 2023-12-20T04:44:27.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f991e428640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9918070230 0x7f9918150500 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f991400d620 tx=0x7f991400dae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f990effd640 1 -- 172.21.15.32:0/1029631890 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991400b620 con 0x7f9918070230 2023-12-20T04:44:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f990effd640 1 -- 172.21.15.32:0/1029631890 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f991400b780 con 0x7f9918070230 2023-12-20T04:44:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f990effd640 1 -- 172.21.15.32:0/1029631890 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f991400bb10 con 0x7f9918070230 2023-12-20T04:44:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f992068c640 1 -- 172.21.15.32:0/1029631890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9918158230 con 0x7f9918070230 2023-12-20T04:44:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f992068c640 1 -- 172.21.15.32:0/1029631890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f99181587f0 con 0x7f9918070230 2023-12-20T04:44:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f992068c640 1 -- 172.21.15.32:0/1029631890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f991804e000 con 0x7f9918070230 2023-12-20T04:44:27.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.617+0000 7f990effd640 1 -- 172.21.15.32:0/1029631890 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f991400bc70 con 0x7f9918070230 2023-12-20T04:44:27.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.621+0000 7f990effd640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98fc065b80 0x7f98fc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.621+0000 7f990effd640 1 -- 172.21.15.32:0/1029631890 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(105..105 src has 1..105) v4 ==== 5609+0+0 (secure 0 0 0) 0x7f99140841b0 con 0x7f9918070230 2023-12-20T04:44:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.621+0000 7f991dc27640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98fc065b80 0x7f98fc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:27.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.621+0000 7f991dc27640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98fc065b80 0x7f98fc068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f99080ae000 tx=0x7f99080b5040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:27.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.625+0000 7f990effd640 1 -- 172.21.15.32:0/1029631890 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9914051b30 con 0x7f9918070230 2023-12-20T04:44:27.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.813+0000 7f992068c640 1 -- 172.21.15.32:0/1029631890 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f99180b1d10 con 0x7f9918070230 2023-12-20T04:44:27.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.821+0000 7f990effd640 1 -- 172.21.15.32:0/1029631890 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 12 and data pool 13 v43) v1 ==== 174+0+0 (secure 0 0 0) 0x7f9914007380 con 0x7f9918070230 2023-12-20T04:44:27.820 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 12 and data pool 13 2023-12-20T04:44:27.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 -- 172.21.15.32:0/1029631890 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98fc065b80 msgr2=0x7f98fc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:27.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98fc065b80 0x7f98fc068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f99080ae000 tx=0x7f99080b5040 comp rx=0 tx=0).stop 2023-12-20T04:44:27.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 -- 172.21.15.32:0/1029631890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9918070230 msgr2=0x7f9918150500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:27.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9918070230 0x7f9918150500 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f991400d620 tx=0x7f991400dae0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 -- 172.21.15.32:0/1029631890 shutdown_connections 2023-12-20T04:44:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98fc065b80 0x7f98fc068040 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f99080ae000 tx=0x7f99080b5040 comp rx=0 tx=0).stop 2023-12-20T04:44:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f991815abb0 0x7f991814f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f99180b0cb0 0x7f99180a7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 --2- 172.21.15.32:0/1029631890 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9918070230 0x7f9918150500 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 -- 172.21.15.32:0/1029631890 >> 172.21.15.32:0/1029631890 conn(0x7f991806a510 msgr2=0x7f9918159b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 -- 172.21.15.32:0/1029631890 shutdown_connections 2023-12-20T04:44:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:27.825+0000 7f992068c640 1 -- 172.21.15.32:0/1029631890 wait complete. 2023-12-20T04:44:27.860 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:44:28.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fed6d640 1 Processor -- start 2023-12-20T04:44:28.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fed6d640 1 -- start start 2023-12-20T04:44:28.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fed6d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4f80d2740 0x7fb4f80cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fed6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 0x7fb4f80b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fed6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4f80cff00 con 0x7fb4f815cdb0 2023-12-20T04:44:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fed6d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb4f815d190 con 0x7fb4f80d2740 2023-12-20T04:44:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fed6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f80d16a0 con 0x7fb4f80cfab0 2023-12-20T04:44:28.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fd30a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 0x7fb4f80b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4f7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4f80d2740 0x7fb4f80cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fd30a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 0x7fb4f80b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:57646/0 (socket says 172.21.15.32:57646) 2023-12-20T04:44:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fcb09640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb4f815cdb0 0x7fb4f80d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:48946/0 (socket says 172.21.15.32:48946) 2023-12-20T04:44:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fd30a640 1 -- 172.21.15.32:0/1010194250 learned_addr learned my addr 172.21.15.32:0/1010194250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4f77fe640 1 -- 172.21.15.32:0/1010194250 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427045369 0 0) 0x7fb4ec001120 con 0x7fb4f815cdb0 2023-12-20T04:44:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4f77fe640 1 -- 172.21.15.32:0/1010194250 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb4e8003530 con 0x7fb4f815cdb0 2023-12-20T04:44:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fd30a640 1 -- 172.21.15.32:0/1010194250 >> v2:172.21.15.39:3300/0 conn(0x7fb4f80d2740 msgr2=0x7fb4f80cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fd30a640 1 --2- 172.21.15.32:0/1010194250 >> v2:172.21.15.39:3300/0 conn(0x7fb4f80d2740 0x7fb4f80cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fd30a640 1 -- 172.21.15.32:0/1010194250 >> v1:172.21.15.39:6789/0 conn(0x7fb4f815cdb0 legacy=0x7fb4f80d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.005+0000 7fb4fd30a640 1 -- 172.21.15.32:0/1010194250 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f80d1980 con 0x7fb4f80cfab0 2023-12-20T04:44:28.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4fd30a640 1 --2- 172.21.15.32:0/1010194250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 0x7fb4f80b5c90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009e50 tx=0x7fb4f001f1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ef04e8549a76141 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4f77fe640 1 -- 172.21.15.32:0/1010194250 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f00af400 con 0x7fb4f80cfab0 2023-12-20T04:44:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4f77fe640 1 -- 172.21.15.32:0/1010194250 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f00ab050 con 0x7fb4f80cfab0 2023-12-20T04:44:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4f77fe640 1 -- 172.21.15.32:0/1010194250 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f00af780 con 0x7fb4f80cfab0 2023-12-20T04:44:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1010194250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 msgr2=0x7fb4f80b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4fed6d640 1 --2- 172.21.15.32:0/1010194250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 0x7fb4f80b5c90 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7fb4f0009e50 tx=0x7fb4f001f1f0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1010194250 shutdown_connections 2023-12-20T04:44:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4fed6d640 1 --2- 172.21.15.32:0/1010194250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 0x7fb4f80b5c90 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4fed6d640 1 --2- 172.21.15.32:0/1010194250 >> v2:172.21.15.39:3300/0 conn(0x7fb4f80d2740 0x7fb4f80cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1010194250 >> 172.21.15.32:0/1010194250 conn(0x7fb4f8065ee0 msgr2=0x7fb4f806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1010194250 shutdown_connections 2023-12-20T04:44:28.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.009+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1010194250 wait complete. 2023-12-20T04:44:28.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fed6d640 1 Processor -- start 2023-12-20T04:44:28.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fed6d640 1 -- start start 2023-12-20T04:44:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fed6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 0x7fb4f81e73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fed6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f80d2740 0x7fb4f81e8150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fed6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f815cdb0 0x7fb4f80c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fed6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81e6fb0 con 0x7fb4f815cdb0 2023-12-20T04:44:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fed6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81e6330 con 0x7fb4f80d2740 2023-12-20T04:44:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fed6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb4f81e6610 con 0x7fb4f80cfab0 2023-12-20T04:44:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fd30a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f815cdb0 0x7fb4f80c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fd30a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f815cdb0 0x7fb4f80c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39070/0 (socket says 172.21.15.32:39070) 2023-12-20T04:44:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fcb09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 0x7fb4f81e73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fd30a640 1 -- 172.21.15.32:0/1343368718 learned_addr learned my addr 172.21.15.32:0/1343368718 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4f7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f80d2740 0x7fb4f81e8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fd30a640 1 -- 172.21.15.32:0/1343368718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 msgr2=0x7fb4f81e73d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fd30a640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 0x7fb4f81e73d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fd30a640 1 -- 172.21.15.32:0/1343368718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f80d2740 msgr2=0x7fb4f81e8150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fd30a640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f80d2740 0x7fb4f81e8150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fd30a640 1 -- 172.21.15.32:0/1343368718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4f0009b30 con 0x7fb4f815cdb0 2023-12-20T04:44:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fd30a640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f815cdb0 0x7fb4f80c81d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb4f001ce20 tx=0x7fb4f001bf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4f57fa640 1 -- 172.21.15.32:0/1343368718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f00afc80 con 0x7fb4f815cdb0 2023-12-20T04:44:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4f57fa640 1 -- 172.21.15.32:0/1343368718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4f00afde0 con 0x7fb4f815cdb0 2023-12-20T04:44:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4f57fa640 1 -- 172.21.15.32:0/1343368718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4f00ba660 con 0x7fb4f815cdb0 2023-12-20T04:44:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.013+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1343368718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4f81e68f0 con 0x7fb4f815cdb0 2023-12-20T04:44:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.017+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1343368718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4f81e8690 con 0x7fb4f815cdb0 2023-12-20T04:44:28.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.017+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1343368718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4f81e8880 con 0x7fb4f815cdb0 2023-12-20T04:44:28.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.017+0000 7fb4f57fa640 1 -- 172.21.15.32:0/1343368718 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb4f00b6040 con 0x7fb4f815cdb0 2023-12-20T04:44:28.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.017+0000 7fb4f57fa640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb4dc065b80 0x7fb4dc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.017+0000 7fb4f57fa640 1 -- 172.21.15.32:0/1343368718 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb4f01213f0 con 0x7fb4f815cdb0 2023-12-20T04:44:28.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.017+0000 7fb4fcb09640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb4dc065b80 0x7fb4dc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.017+0000 7fb4fcb09640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb4dc065b80 0x7fb4dc068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec001380 tx=0x7fb4ec007bc0 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:28.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.021+0000 7fb4f57fa640 1 -- 172.21.15.32:0/1343368718 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb4f81e8880 con 0x7fb4f815cdb0 2023-12-20T04:44:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1343368718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb4f81e70f0 con 0x7fb4f815cdb0 2023-12-20T04:44:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4f57fa640 1 -- 172.21.15.32:0/1343368718 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16068 (secure 0 0 0) 0x7fb4f81e70f0 con 0x7fb4f815cdb0 2023-12-20T04:44:28.185 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:28.185 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":106,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:44:27.818073+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:44:25.451651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-20T04:44:26.819848+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1343368718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb4dc065b80 msgr2=0x7fb4dc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4fed6d640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb4dc065b80 0x7fb4dc068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fb4ec001380 tx=0x7fb4ec007bc0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1343368718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f815cdb0 msgr2=0x7fb4f80c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4fed6d640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f815cdb0 0x7fb4f80c81d0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fb4f001ce20 tx=0x7fb4f001bf90 comp rx=0 tx=0).stop 2023-12-20T04:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1343368718 shutdown_connections 2023-12-20T04:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4fed6d640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb4dc065b80 0x7fb4dc068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4fed6d640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4f815cdb0 0x7fb4f80c81d0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4fed6d640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4f80d2740 0x7fb4f81e8150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4fed6d640 1 --2- 172.21.15.32:0/1343368718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4f80cfab0 0x7fb4f81e73d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.185+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1343368718 >> 172.21.15.32:0/1343368718 conn(0x7fb4f8065ee0 msgr2=0x7fb4f80cbc40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.189+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1343368718 shutdown_connections 2023-12-20T04:44:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.189+0000 7fb4fed6d640 1 -- 172.21.15.32:0/1343368718 wait complete. 2023-12-20T04:44:28.207 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:44:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.349+0000 7f3dc6396640 1 Processor -- start 2023-12-20T04:44:28.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.349+0000 7f3dc6396640 1 -- start start 2023-12-20T04:44:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.349+0000 7f3dc6396640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc015ce30 0x7f3dc0161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.349+0000 7f3dc6396640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3dc0070950 0x7f3dc006d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dc6396640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3dc0071920 con 0x7f3dc006dce0 2023-12-20T04:44:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dc6396640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3dc0071080 con 0x7f3dc0070950 2023-12-20T04:44:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dc6396640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc0071360 con 0x7f3dc015ce30 2023-12-20T04:44:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dc4933640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3dc006dce0 0x7f3dc0069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54302/0 (socket says 172.21.15.32:54302) 2023-12-20T04:44:28.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dc4933640 1 -- 172.21.15.32:0/506859783 learned_addr learned my addr 172.21.15.32:0/506859783 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dbf7fe640 1 --2- 172.21.15.32:0/506859783 >> v2:172.21.15.39:3300/0 conn(0x7f3dc0070950 0x7f3dc006d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dbffff640 1 --2- 172.21.15.32:0/506859783 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc015ce30 0x7f3dc0161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dbeffd640 1 -- 172.21.15.32:0/506859783 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1672578585 0 0) 0x7f3dc0071920 con 0x7f3dc006dce0 2023-12-20T04:44:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dbeffd640 1 -- 172.21.15.32:0/506859783 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3da8003530 con 0x7f3dc006dce0 2023-12-20T04:44:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dbeffd640 1 -- 172.21.15.32:0/506859783 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3795257012 0 0) 0x7f3da8003530 con 0x7f3dc006dce0 2023-12-20T04:44:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dbf7fe640 1 -- 172.21.15.32:0/506859783 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc015ce30 msgr2=0x7f3dc0161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dbf7fe640 1 --2- 172.21.15.32:0/506859783 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc015ce30 0x7f3dc0161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dbf7fe640 1 -- 172.21.15.32:0/506859783 >> v1:172.21.15.32:6789/0 conn(0x7f3dc006dce0 legacy=0x7f3dc0069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dbf7fe640 1 -- 172.21.15.32:0/506859783 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3dc0071640 con 0x7f3dc0070950 2023-12-20T04:44:28.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.353+0000 7f3dbffff640 1 --2- 172.21.15.32:0/506859783 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc015ce30 0x7f3dc0161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dbf7fe640 1 --2- 172.21.15.32:0/506859783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc0070950 0x7f3dc006d7a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3db40097c0 tx=0x7f3db401c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ec339276aaf75c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dbeffd640 1 -- 172.21.15.32:0/506859783 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8001a00 con 0x7f3dc0070950 2023-12-20T04:44:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dbeffd640 1 -- 172.21.15.32:0/506859783 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db40a7050 con 0x7f3dc0070950 2023-12-20T04:44:28.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dbeffd640 1 -- 172.21.15.32:0/506859783 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db40ab620 con 0x7f3dc0070950 2023-12-20T04:44:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dc6396640 1 -- 172.21.15.32:0/506859783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc0070950 msgr2=0x7f3dc006d7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dc6396640 1 --2- 172.21.15.32:0/506859783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc0070950 0x7f3dc006d7a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7f3db40097c0 tx=0x7f3db401c3a0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dc6396640 1 -- 172.21.15.32:0/506859783 shutdown_connections 2023-12-20T04:44:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dc6396640 1 --2- 172.21.15.32:0/506859783 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc0070950 0x7f3dc006d7a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dc6396640 1 --2- 172.21.15.32:0/506859783 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc015ce30 0x7f3dc0161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dc6396640 1 -- 172.21.15.32:0/506859783 >> 172.21.15.32:0/506859783 conn(0x7f3dc00cba50 msgr2=0x7f3dc00cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dc6396640 1 -- 172.21.15.32:0/506859783 shutdown_connections 2023-12-20T04:44:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dc6396640 1 -- 172.21.15.32:0/506859783 wait complete. 2023-12-20T04:44:28.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dc6396640 1 Processor -- start 2023-12-20T04:44:28.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.357+0000 7f3dc6396640 1 -- start start 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc6396640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc006dce0 0x7f3dc00aeb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc6396640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc0070950 0x7f3dc00afb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc6396640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dc015ce30 0x7f3dc00c5220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc6396640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00b1a00 con 0x7f3dc015ce30 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc6396640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00b0080 con 0x7f3dc0070950 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc6396640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3dc00b02c0 con 0x7f3dc006dce0 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc4933640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dc015ce30 0x7f3dc00c5220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc4933640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dc015ce30 0x7f3dc00c5220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39076/0 (socket says 172.21.15.32:39076) 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc4933640 1 -- 172.21.15.32:0/135173617 learned_addr learned my addr 172.21.15.32:0/135173617 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dbf7fe640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc0070950 0x7f3dc00afb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dbffff640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc006dce0 0x7f3dc00aeb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc4933640 1 -- 172.21.15.32:0/135173617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc006dce0 msgr2=0x7f3dc00aeb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc4933640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc006dce0 0x7f3dc00aeb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc4933640 1 -- 172.21.15.32:0/135173617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc0070950 msgr2=0x7f3dc00afb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc4933640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc0070950 0x7f3dc00afb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc4933640 1 -- 172.21.15.32:0/135173617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3db0009420 con 0x7f3dc015ce30 2023-12-20T04:44:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dbf7fe640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc0070950 0x7f3dc00afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:44:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dbffff640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc006dce0 0x7f3dc00aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:44:28.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc4933640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dc015ce30 0x7f3dc00c5220 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3db8001840 tx=0x7f3db800a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dbcff9640 1 -- 172.21.15.32:0/135173617 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8012400 con 0x7f3dc015ce30 2023-12-20T04:44:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dbcff9640 1 -- 172.21.15.32:0/135173617 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3db8012560 con 0x7f3dc015ce30 2023-12-20T04:44:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc6396640 1 -- 172.21.15.32:0/135173617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3db4009420 con 0x7f3dc015ce30 2023-12-20T04:44:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dbcff9640 1 -- 172.21.15.32:0/135173617 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3db8012910 con 0x7f3dc015ce30 2023-12-20T04:44:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc6396640 1 -- 172.21.15.32:0/135173617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3dc00b0880 con 0x7f3dc015ce30 2023-12-20T04:44:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dc6396640 1 -- 172.21.15.32:0/135173617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3dc004e000 con 0x7f3dc015ce30 2023-12-20T04:44:28.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.361+0000 7f3dbcff9640 1 -- 172.21.15.32:0/135173617 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3db8012a70 con 0x7f3dc015ce30 2023-12-20T04:44:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.365+0000 7f3dbcff9640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dac065b80 0x7f3dac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.365+0000 7f3dbcff9640 1 -- 172.21.15.32:0/135173617 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3db8083ad0 con 0x7f3dc015ce30 2023-12-20T04:44:28.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.365+0000 7f3dbffff640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dac065b80 0x7f3dac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.365+0000 7f3dbffff640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dac065b80 0x7f3dac068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009bd0 tx=0x7f3db0008040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:28.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.369+0000 7f3dbcff9640 1 -- 172.21.15.32:0/135173617 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3db80512e0 con 0x7f3dc015ce30 2023-12-20T04:44:28.394 INFO:tasks.ceph.mds.d.smithi039.stderr:2023-12-20T04:44:28.395+0000 7ff9be2df640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:44:28.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.529+0000 7f3dc6396640 1 -- 172.21.15.32:0/135173617 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f3dc00b4f50 con 0x7f3dc015ce30 2023-12-20T04:44:28.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.529+0000 7f3dbcff9640 1 -- 172.21.15.32:0/135173617 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v106) v1 ==== 74+0+16068 (secure 0 0 0) 0x7f3db8054900 con 0x7f3dc015ce30 2023-12-20T04:44:28.530 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:28.531 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":106,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:44:27.818073+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:44:25.451651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-20T04:44:26.819848+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:44:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 -- 172.21.15.32:0/135173617 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dac065b80 msgr2=0x7f3dac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dac065b80 0x7f3dac068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f3db0009bd0 tx=0x7f3db0008040 comp rx=0 tx=0).stop 2023-12-20T04:44:28.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 -- 172.21.15.32:0/135173617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dc015ce30 msgr2=0x7f3dc00c5220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dc015ce30 0x7f3dc00c5220 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7f3db8001840 tx=0x7f3db800a430 comp rx=0 tx=0).stop 2023-12-20T04:44:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 -- 172.21.15.32:0/135173617 shutdown_connections 2023-12-20T04:44:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dac065b80 0x7f3dac068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3dc015ce30 0x7f3dc00c5220 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3dc0070950 0x7f3dc00afb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 --2- 172.21.15.32:0/135173617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3dc006dce0 0x7f3dc00aeb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 -- 172.21.15.32:0/135173617 >> 172.21.15.32:0/135173617 conn(0x7f3dc00cba50 msgr2=0x7f3dc015e090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 -- 172.21.15.32:0/135173617 shutdown_connections 2023-12-20T04:44:28.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.533+0000 7f3dc6396640 1 -- 172.21.15.32:0/135173617 wait complete. 2023-12-20T04:44:28.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T04:44:28.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fe2996640 1 Processor -- start 2023-12-20T04:44:28.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fe2996640 1 -- start start 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fe2996640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc157480 0x7f9fdc0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fe2996640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fdc0b0760 0x7f9fdc0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fe2996640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9fdc15d660 0x7f9fdc15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fe2996640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc0b1100 con 0x7f9fdc0b0760 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fe2996640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc0cde20 con 0x7f9fdc15d660 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fe2996640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0c8e30 con 0x7f9fdc157480 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fdb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fdc0b0760 0x7f9fdc0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fdb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fdc0b0760 0x7f9fdc0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39082/0 (socket says 172.21.15.32:39082) 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fdb7fe640 1 -- 172.21.15.32:0/3059552876 learned_addr learned my addr 172.21.15.32:0/3059552876 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fdbfff640 1 --2- 172.21.15.32:0/3059552876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc157480 0x7f9fdc0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fe0f33640 1 --2- 172.21.15.32:0/3059552876 >> v2:172.21.15.39:3300/0 conn(0x7f9fdc15d660 0x7f9fdc15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fdb7fe640 1 -- 172.21.15.32:0/3059552876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc157480 msgr2=0x7f9fdc0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:28.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fdb7fe640 1 --2- 172.21.15.32:0/3059552876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc157480 0x7f9fdc0c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fdb7fe640 1 -- 172.21.15.32:0/3059552876 >> v2:172.21.15.39:3300/0 conn(0x7f9fdc15d660 msgr2=0x7f9fdc15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fdb7fe640 1 --2- 172.21.15.32:0/3059552876 >> v2:172.21.15.39:3300/0 conn(0x7f9fdc15d660 0x7f9fdc15daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.701+0000 7f9fdb7fe640 1 -- 172.21.15.32:0/3059552876 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc0ce070 con 0x7f9fdc0b0760 2023-12-20T04:44:28.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fdb7fe640 1 --2- 172.21.15.32:0/3059552876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc0b0760 0x7f9fdc0b0bc0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0b49b0 tx=0x7f9fd001eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977f5deef7e4db07 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fdaffd640 1 -- 172.21.15.32:0/3059552876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd00a7aa0 con 0x7f9fdc0b0760 2023-12-20T04:44:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fdaffd640 1 -- 172.21.15.32:0/3059552876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd00a7c00 con 0x7f9fdc0b0760 2023-12-20T04:44:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fdaffd640 1 -- 172.21.15.32:0/3059552876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd00af440 con 0x7f9fdc0b0760 2023-12-20T04:44:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fe2996640 1 -- 172.21.15.32:0/3059552876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc0b0760 msgr2=0x7f9fdc0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fe2996640 1 --2- 172.21.15.32:0/3059552876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc0b0760 0x7f9fdc0b0bc0 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0b49b0 tx=0x7f9fd001eb10 comp rx=0 tx=0).stop 2023-12-20T04:44:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fe2996640 1 -- 172.21.15.32:0/3059552876 shutdown_connections 2023-12-20T04:44:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fe2996640 1 --2- 172.21.15.32:0/3059552876 >> v2:172.21.15.39:3300/0 conn(0x7f9fdc15d660 0x7f9fdc15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fe2996640 1 --2- 172.21.15.32:0/3059552876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc0b0760 0x7f9fdc0b0bc0 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fe2996640 1 --2- 172.21.15.32:0/3059552876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc157480 0x7f9fdc0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fe2996640 1 -- 172.21.15.32:0/3059552876 >> 172.21.15.32:0/3059552876 conn(0x7f9fdc06a510 msgr2=0x7f9fdc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:28.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fe2996640 1 -- 172.21.15.32:0/3059552876 shutdown_connections 2023-12-20T04:44:28.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.705+0000 7f9fe2996640 1 -- 172.21.15.32:0/3059552876 wait complete. 2023-12-20T04:44:28.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe2996640 1 Processor -- start 2023-12-20T04:44:28.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe2996640 1 -- start start 2023-12-20T04:44:28.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe2996640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fdc0b0760 0x7f9fdc0c6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe2996640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc157480 0x7f9fdc14e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe2996640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc15d660 0x7f9fdc0c7c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe2996640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0c6090 con 0x7f9fdc15d660 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe2996640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc156650 con 0x7f9fdc0b0760 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe2996640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0c5db0 con 0x7f9fdc157480 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe0f33640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc15d660 0x7f9fdc0c7c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fdbfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fdc0b0760 0x7f9fdc0c6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe0f33640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc15d660 0x7f9fdc0c7c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:39084/0 (socket says 172.21.15.32:39084) 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe0f33640 1 -- 172.21.15.32:0/137967306 learned_addr learned my addr 172.21.15.32:0/137967306 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fdb7fe640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc157480 0x7f9fdc14e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe0f33640 1 -- 172.21.15.32:0/137967306 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc157480 msgr2=0x7f9fdc14e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe0f33640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc157480 0x7f9fdc14e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe0f33640 1 -- 172.21.15.32:0/137967306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fdc0b0760 msgr2=0x7f9fdc0c6290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe0f33640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fdc0b0760 0x7f9fdc0c6290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:28.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe0f33640 1 -- 172.21.15.32:0/137967306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fd000b770 con 0x7f9fdc15d660 2023-12-20T04:44:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe0f33640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc15d660 0x7f9fdc0c7c90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009dd0 tx=0x7f9fd400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fd8ff9640 1 -- 172.21.15.32:0/137967306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4014480 con 0x7f9fdc15d660 2023-12-20T04:44:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fd8ff9640 1 -- 172.21.15.32:0/137967306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd400f040 con 0x7f9fdc15d660 2023-12-20T04:44:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe2996640 1 -- 172.21.15.32:0/137967306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc0b2b20 con 0x7f9fdc15d660 2023-12-20T04:44:28.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fd8ff9640 1 -- 172.21.15.32:0/137967306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4014840 con 0x7f9fdc15d660 2023-12-20T04:44:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.709+0000 7f9fe2996640 1 -- 172.21.15.32:0/137967306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc0b30e0 con 0x7f9fdc15d660 2023-12-20T04:44:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.713+0000 7f9fe2996640 1 -- 172.21.15.32:0/137967306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc0b2980 con 0x7f9fdc15d660 2023-12-20T04:44:28.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.713+0000 7f9fd8ff9640 1 -- 172.21.15.32:0/137967306 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9fd40149a0 con 0x7f9fdc15d660 2023-12-20T04:44:28.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.713+0000 7f9fd8ff9640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc8065b80 0x7f9fc8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:28.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.713+0000 7f9fd8ff9640 1 -- 172.21.15.32:0/137967306 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(106..106 src has 1..106) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f9fd4083e00 con 0x7f9fdc15d660 2023-12-20T04:44:28.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.713+0000 7f9fdbfff640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc8065b80 0x7f9fc8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:28.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.713+0000 7f9fdbfff640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc8065b80 0x7f9fc8068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009a40 tx=0x7f9fcc009190 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:28.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.717+0000 7f9fd8ff9640 1 -- 172.21.15.32:0/137967306 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9fdc0b2980 con 0x7f9fdc15d660 2023-12-20T04:44:28.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:28.913+0000 7f9fe2996640 1 -- 172.21.15.32:0/137967306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f9fdc0b2980 con 0x7f9fdc15d660 2023-12-20T04:44:29.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.829+0000 7f9fd8ff9640 1 -- 172.21.15.32:0/137967306 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v45) v1 ==== 118+0+0 (secure 0 0 0) 0x7f9fdc0b2980 con 0x7f9fdc15d660 2023-12-20T04:44:29.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 -- 172.21.15.32:0/137967306 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc8065b80 msgr2=0x7f9fc8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:29.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc8065b80 0x7f9fc8068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc009a40 tx=0x7f9fcc009190 comp rx=0 tx=0).stop 2023-12-20T04:44:29.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 -- 172.21.15.32:0/137967306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc15d660 msgr2=0x7f9fdc0c7c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc15d660 0x7f9fdc0c7c90 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009dd0 tx=0x7f9fd400c490 comp rx=0 tx=0).stop 2023-12-20T04:44:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 -- 172.21.15.32:0/137967306 shutdown_connections 2023-12-20T04:44:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc8065b80 0x7f9fc8068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc15d660 0x7f9fdc0c7c90 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc157480 0x7f9fdc14e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 --2- 172.21.15.32:0/137967306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fdc0b0760 0x7f9fdc0c6290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 -- 172.21.15.32:0/137967306 >> 172.21.15.32:0/137967306 conn(0x7f9fdc06a510 msgr2=0x7f9fdc06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 -- 172.21.15.32:0/137967306 shutdown_connections 2023-12-20T04:44:29.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:29.833+0000 7f9fe2996640 1 -- 172.21.15.32:0/137967306 wait complete. 2023-12-20T04:44:29.855 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T04:44:29.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T04:44:30.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366efe9640 1 Processor -- start 2023-12-20T04:44:30.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366efe9640 1 -- start start 2023-12-20T04:44:30.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366efe9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36680cb050 0x7f36680cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:30.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366efe9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366806a1f0 con 0x7f366806d170 2023-12-20T04:44:30.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366efe9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f366806a440 con 0x7f3668065d40 2023-12-20T04:44:30.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366efe9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f366806a720 con 0x7f36680cb050 2023-12-20T04:44:30.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366d586640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f366806d170 0x7f36680c9720 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49066/0 (socket says 172.21.15.32:49066) 2023-12-20T04:44:30.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366d586640 1 -- 172.21.15.32:0/2411720878 learned_addr learned my addr 172.21.15.32:0/2411720878 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:30.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366cd85640 1 --2- 172.21.15.32:0/2411720878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36680cb050 0x7f36680cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f36677fe640 1 -- 172.21.15.32:0/2411720878 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1454501181 0 0) 0x7f366806a1f0 con 0x7f366806d170 2023-12-20T04:44:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f36677fe640 1 -- 172.21.15.32:0/2411720878 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3658003530 con 0x7f366806d170 2023-12-20T04:44:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f36677fe640 1 -- 172.21.15.32:0/2411720878 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4044542675 0 0) 0x7f366806a440 con 0x7f3668065d40 2023-12-20T04:44:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f36677fe640 1 -- 172.21.15.32:0/2411720878 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f366806a1f0 con 0x7f3668065d40 2023-12-20T04:44:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366cd85640 1 -- 172.21.15.32:0/2411720878 >> v1:172.21.15.39:6789/0 conn(0x7f3668065d40 legacy=0x7f366806ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366cd85640 1 -- 172.21.15.32:0/2411720878 >> v1:172.21.15.32:6789/0 conn(0x7f366806d170 legacy=0x7f36680c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:30.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.009+0000 7f366cd85640 1 -- 172.21.15.32:0/2411720878 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f366806aa00 con 0x7f36680cb050 2023-12-20T04:44:30.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.013+0000 7f366cd85640 1 --2- 172.21.15.32:0/2411720878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36680cb050 0x7f36680cb430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f365c0097f0 tx=0x7f365c01c3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cb19cee7ea59e7e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:30.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.013+0000 7f36677fe640 1 -- 172.21.15.32:0/2411720878 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c0ab600 con 0x7f36680cb050 2023-12-20T04:44:30.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.013+0000 7f36677fe640 1 -- 172.21.15.32:0/2411720878 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f365c0a7050 con 0x7f36680cb050 2023-12-20T04:44:30.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.013+0000 7f36677fe640 1 -- 172.21.15.32:0/2411720878 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f365c0ab980 con 0x7f36680cb050 2023-12-20T04:44:30.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.013+0000 7f366efe9640 1 -- 172.21.15.32:0/2411720878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36680cb050 msgr2=0x7f36680cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:30.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.013+0000 7f366efe9640 1 --2- 172.21.15.32:0/2411720878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36680cb050 0x7f36680cb430 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f365c0097f0 tx=0x7f365c01c3b0 comp rx=0 tx=0).stop 2023-12-20T04:44:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.013+0000 7f366efe9640 1 -- 172.21.15.32:0/2411720878 shutdown_connections 2023-12-20T04:44:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.013+0000 7f366efe9640 1 --2- 172.21.15.32:0/2411720878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36680cb050 0x7f36680cb430 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.013+0000 7f366efe9640 1 -- 172.21.15.32:0/2411720878 >> 172.21.15.32:0/2411720878 conn(0x7f36680cf8d0 msgr2=0x7f36680cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.013+0000 7f366efe9640 1 -- 172.21.15.32:0/2411720878 shutdown_connections 2023-12-20T04:44:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.013+0000 7f366efe9640 1 -- 172.21.15.32:0/2411720878 wait complete. 2023-12-20T04:44:30.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366efe9640 1 Processor -- start 2023-12-20T04:44:30.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366efe9640 1 -- start start 2023-12-20T04:44:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366efe9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3668065d40 0x7f36681532d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366efe9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f366806d170 0x7f3668156ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366efe9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36680cb050 0x7f3668157400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366efe9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f366815a730 con 0x7f366806d170 2023-12-20T04:44:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366efe9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36681593f0 con 0x7f36680cb050 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366efe9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3668159690 con 0x7f3668065d40 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f3667fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f366806d170 0x7f3668156ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f3667fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f366806d170 0x7f3668156ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60476/0 (socket says 172.21.15.32:60476) 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f3667fff640 1 -- 172.21.15.32:0/1619245367 learned_addr learned my addr 172.21.15.32:0/1619245367 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366cd85640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3668065d40 0x7f36681532d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366d586640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36680cb050 0x7f3668157400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f3667fff640 1 -- 172.21.15.32:0/1619245367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3668065d40 msgr2=0x7f36681532d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f3667fff640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3668065d40 0x7f36681532d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f3667fff640 1 -- 172.21.15.32:0/1619245367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36680cb050 msgr2=0x7f3668157400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f3667fff640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36680cb050 0x7f3668157400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f3667fff640 1 -- 172.21.15.32:0/1619245367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f365c009420 con 0x7f366806d170 2023-12-20T04:44:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f3667fff640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f366806d170 0x7f3668156ec0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f365400c430 tx=0x7f365400c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f36657fa640 1 -- 172.21.15.32:0/1619245367 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36540052d0 con 0x7f366806d170 2023-12-20T04:44:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f36657fa640 1 -- 172.21.15.32:0/1619245367 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3654005450 con 0x7f366806d170 2023-12-20T04:44:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366efe9640 1 -- 172.21.15.32:0/1619245367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3668159970 con 0x7f366806d170 2023-12-20T04:44:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f36657fa640 1 -- 172.21.15.32:0/1619245367 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3654003840 con 0x7f366806d170 2023-12-20T04:44:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366efe9640 1 -- 172.21.15.32:0/1619245367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3668159f30 con 0x7f366806d170 2023-12-20T04:44:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.017+0000 7f366efe9640 1 -- 172.21.15.32:0/1619245367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f366804e000 con 0x7f366806d170 2023-12-20T04:44:30.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.021+0000 7f36657fa640 1 -- 172.21.15.32:0/1619245367 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f36540039a0 con 0x7f366806d170 2023-12-20T04:44:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.021+0000 7f36657fa640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f364c065bb0 0x7f364c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.021+0000 7f36657fa640 1 -- 172.21.15.32:0/1619245367 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f36540828d0 con 0x7f366806d170 2023-12-20T04:44:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.021+0000 7f366cd85640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f364c065bb0 0x7f364c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:30.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.021+0000 7f366cd85640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f364c065bb0 0x7f364c068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f365c0ad000 tx=0x7f365c0a6000 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.025+0000 7f36657fa640 1 -- 172.21.15.32:0/1619245367 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3654050190 con 0x7f366806d170 2023-12-20T04:44:30.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.217+0000 7f366efe9640 1 -- 172.21.15.32:0/1619245367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f366814bbb0 con 0x7f366806d170 2023-12-20T04:44:30.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.829+0000 7f36657fa640 1 -- 172.21.15.32:0/1619245367 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f36540170c0 con 0x7f366806d170 2023-12-20T04:44:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 -- 172.21.15.32:0/1619245367 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f364c065bb0 msgr2=0x7f364c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f364c065bb0 0x7f364c068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f365c0ad000 tx=0x7f365c0a6000 comp rx=0 tx=0).stop 2023-12-20T04:44:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 -- 172.21.15.32:0/1619245367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f366806d170 msgr2=0x7f3668156ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:30.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f366806d170 0x7f3668156ec0 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7f365400c430 tx=0x7f365400c8f0 comp rx=0 tx=0).stop 2023-12-20T04:44:30.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 -- 172.21.15.32:0/1619245367 shutdown_connections 2023-12-20T04:44:30.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f364c065bb0 0x7f364c068070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:30.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36680cb050 0x7f3668157400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:30.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f366806d170 0x7f3668156ec0 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:30.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 --2- 172.21.15.32:0/1619245367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3668065d40 0x7f36681532d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:30.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 -- 172.21.15.32:0/1619245367 >> 172.21.15.32:0/1619245367 conn(0x7f36680cf8d0 msgr2=0x7f366815fca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:30.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 -- 172.21.15.32:0/1619245367 shutdown_connections 2023-12-20T04:44:30.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:30.833+0000 7f366efe9640 1 -- 172.21.15.32:0/1619245367 wait complete. 2023-12-20T04:44:30.856 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T04:44:30.856 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:44:31.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effea6d7640 1 Processor -- start 2023-12-20T04:44:31.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effea6d7640 1 -- start start 2023-12-20T04:44:31.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effea6d7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7effe406b630 0x7effe4160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effea6d7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe40cd4d0 con 0x7effe40b8480 2023-12-20T04:44:31.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effea6d7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7effe40ce4e0 con 0x7effe4161330 2023-12-20T04:44:31.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effea6d7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7effe40cd720 con 0x7effe406b630 2023-12-20T04:44:31.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe96d5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7effe40b8480 0x7effe406af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49082/0 (socket says 172.21.15.32:49082) 2023-12-20T04:44:31.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe96d5640 1 -- 172.21.15.32:0/1713601226 learned_addr learned my addr 172.21.15.32:0/1713601226 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:31.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe8ed4640 1 --2- 172.21.15.32:0/1713601226 >> v2:172.21.15.39:3300/0 conn(0x7effe406b630 0x7effe4160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3547819182 0 0) 0x7effe40cd4d0 con 0x7effe40b8480 2023-12-20T04:44:31.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7effd0003530 con 0x7effe40b8480 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 777564473 0 0) 0x7effd0003530 con 0x7effe40b8480 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 >> v2:172.21.15.39:3300/0 conn(0x7effe406b630 msgr2=0x7effe4160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 --2- 172.21.15.32:0/1713601226 >> v2:172.21.15.39:3300/0 conn(0x7effe406b630 0x7effe4160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 >> v1:172.21.15.39:6789/0 conn(0x7effe4161330 legacy=0x7effe4153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe40cda00 con 0x7effe40b8480 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7effe0001680 con 0x7effe40b8480 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 >> v1:172.21.15.32:6789/0 conn(0x7effe40b8480 legacy=0x7effe406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7effd0003980 0x7effd0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd00047d0 0x7effd0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 0x7effd000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe40cd720 con 0x7effd0005190 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effd000c030 con 0x7effd00047d0 2023-12-20T04:44:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effd000c1f0 con 0x7effd0003980 2023-12-20T04:44:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe9ed6640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 0x7effd000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe8ed4640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd00047d0 0x7effd0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe96d5640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7effd0003980 0x7effd0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe9ed6640 1 -- 172.21.15.32:0/1713601226 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7effd0003980 msgr2=0x7effd0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe9ed6640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7effd0003980 0x7effd0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe9ed6640 1 -- 172.21.15.32:0/1713601226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd00047d0 msgr2=0x7effd0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe9ed6640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd00047d0 0x7effd0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe9ed6640 1 -- 172.21.15.32:0/1713601226 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effd0003530 con 0x7effd0005190 2023-12-20T04:44:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.005+0000 7effe8ed4640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effd00047d0 0x7effd0004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effe9ed6640 1 -- 172.21.15.32:0/1713601226 reap_dead start 2023-12-20T04:44:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effe9ed6640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 0x7effd000b990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7effdc001e70 tx=0x7effdc01c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14c6d640c11d88c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc003b80 con 0x7effd0005190 2023-12-20T04:44:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effdc003d00 con 0x7effd0005190 2023-12-20T04:44:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effdbfff640 1 -- 172.21.15.32:0/1713601226 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effdc005260 con 0x7effd0005190 2023-12-20T04:44:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effea6d7640 1 -- 172.21.15.32:0/1713601226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 msgr2=0x7effd000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effea6d7640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 0x7effd000b990 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7effdc001e70 tx=0x7effdc01c760 comp rx=0 tx=0).stop 2023-12-20T04:44:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effea6d7640 1 -- 172.21.15.32:0/1713601226 shutdown_connections 2023-12-20T04:44:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effea6d7640 1 --2- 172.21.15.32:0/1713601226 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 0x7effd000b990 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effea6d7640 1 -- 172.21.15.32:0/1713601226 >> 172.21.15.32:0/1713601226 conn(0x7effe40d2a50 msgr2=0x7effe40d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effea6d7640 1 -- 172.21.15.32:0/1713601226 shutdown_connections 2023-12-20T04:44:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.009+0000 7effea6d7640 1 -- 172.21.15.32:0/1713601226 wait complete. 2023-12-20T04:44:31.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effea6d7640 1 Processor -- start 2023-12-20T04:44:31.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effea6d7640 1 -- start start 2023-12-20T04:44:31.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effea6d7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 0x7effe40b4cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effea6d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effe414a690 0x7effe415e9f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effea6d7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7effe414ab10 0x7effe415ef30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effea6d7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe4148b70 con 0x7effd0005190 2023-12-20T04:44:31.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effea6d7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7effe414b670 con 0x7effe414a690 2023-12-20T04:44:31.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effea6d7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7effe40b5230 con 0x7effe414ab10 2023-12-20T04:44:31.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe96d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 0x7effe40b4cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe96d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 0x7effe40b4cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60490/0 (socket says 172.21.15.32:60490) 2023-12-20T04:44:31.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe96d5640 1 -- 172.21.15.32:0/4025359047 learned_addr learned my addr 172.21.15.32:0/4025359047 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe8ed4640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effe414a690 0x7effe415e9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe9ed6640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7effe414ab10 0x7effe415ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe96d5640 1 -- 172.21.15.32:0/4025359047 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7effe414ab10 msgr2=0x7effe415ef30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe96d5640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7effe414ab10 0x7effe415ef30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe96d5640 1 -- 172.21.15.32:0/4025359047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effe414a690 msgr2=0x7effe415e9f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe96d5640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effe414a690 0x7effe415e9f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe96d5640 1 -- 172.21.15.32:0/4025359047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7effe414b0b0 con 0x7effd0005190 2023-12-20T04:44:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe9ed6640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7effe414ab10 0x7effe415ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:44:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effe96d5640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 0x7effe40b4cf0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7effe00072c0 tx=0x7effe0007780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effd9ffb640 1 -- 172.21.15.32:0/4025359047 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe000f9c0 con 0x7effd0005190 2023-12-20T04:44:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effd9ffb640 1 -- 172.21.15.32:0/4025359047 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7effe000fb40 con 0x7effd0005190 2023-12-20T04:44:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effd9ffb640 1 -- 172.21.15.32:0/4025359047 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7effe00133f0 con 0x7effd0005190 2023-12-20T04:44:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effea6d7640 1 -- 172.21.15.32:0/4025359047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7effe414b390 con 0x7effd0005190 2023-12-20T04:44:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.013+0000 7effea6d7640 1 -- 172.21.15.32:0/4025359047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7effe415f6f0 con 0x7effd0005190 2023-12-20T04:44:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.017+0000 7effea6d7640 1 -- 172.21.15.32:0/4025359047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7effe415f910 con 0x7effd0005190 2023-12-20T04:44:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.017+0000 7effd9ffb640 1 -- 172.21.15.32:0/4025359047 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7effe001c3b0 con 0x7effd0005190 2023-12-20T04:44:31.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.017+0000 7effd9ffb640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7effc8065b80 0x7effc8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.017+0000 7effd9ffb640 1 -- 172.21.15.32:0/4025359047 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7effe0082c60 con 0x7effd0005190 2023-12-20T04:44:31.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.017+0000 7effe8ed4640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7effc8065b80 0x7effc8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.017+0000 7effe8ed4640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7effc8065b80 0x7effc8068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7effe40c5f10 tx=0x7effd4008040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:31.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.021+0000 7effd9ffb640 1 -- 172.21.15.32:0/4025359047 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7effe415f910 con 0x7effd0005190 2023-12-20T04:44:31.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.209+0000 7effea6d7640 1 -- 172.21.15.32:0/4025359047 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7effe415f910 con 0x7effd0005190 2023-12-20T04:44:31.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effd9ffb640 1 -- 172.21.15.32:0/4025359047 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4711 (secure 0 0 0) 0x7effe415f910 con 0x7effd0005190 2023-12-20T04:44:31.211 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:31.211 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1072056415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1072056415},{"type":"v1","addr":"172.21.15.39:6835","nonce":1072056415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3512326822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3512326822},{"type":"v1","addr":"172.21.15.32:6836","nonce":3512326822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2163601073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2163601073},{"type":"v1","addr":"172.21.15.32:6837","nonce":2163601073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:44:27.818499+0000","modified":"2023-12-20T04:44:30.831088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7359":{"gid":7359,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/2369052657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2369052657},{"type":"v1","addr":"172.21.15.39:6833","nonce":2369052657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-20T04:44:31.212 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 46 2023-12-20T04:44:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 -- 172.21.15.32:0/4025359047 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7effc8065b80 msgr2=0x7effc8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7effc8065b80 0x7effc8068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7effe40c5f10 tx=0x7effd4008040 comp rx=0 tx=0).stop 2023-12-20T04:44:31.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 -- 172.21.15.32:0/4025359047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 msgr2=0x7effe40b4cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 0x7effe40b4cf0 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7effe00072c0 tx=0x7effe0007780 comp rx=0 tx=0).stop 2023-12-20T04:44:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 -- 172.21.15.32:0/4025359047 shutdown_connections 2023-12-20T04:44:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7effc8065b80 0x7effc8068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7effe414ab10 0x7effe415ef30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7effe414a690 0x7effe415e9f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 --2- 172.21.15.32:0/4025359047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7effd0005190 0x7effe40b4cf0 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 -- 172.21.15.32:0/4025359047 >> 172.21.15.32:0/4025359047 conn(0x7effe40d2a50 msgr2=0x7effe415fe50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:31.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 -- 172.21.15.32:0/4025359047 shutdown_connections 2023-12-20T04:44:31.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.213+0000 7effea6d7640 1 -- 172.21.15.32:0/4025359047 wait complete. 2023-12-20T04:44:31.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:44:31.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:44:31.282 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:44:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.377+0000 7ff27d804640 1 Processor -- start 2023-12-20T04:44:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.377+0000 7ff27d804640 1 -- start start 2023-12-20T04:44:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.377+0000 7ff27d804640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 0x7ff278159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.377+0000 7ff27d804640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff2780cb090 0x7ff2780cb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.377+0000 7ff27d804640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff27806c180 con 0x7ff2780c8580 2023-12-20T04:44:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.377+0000 7ff27d804640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2780cba40 con 0x7ff2780cb090 2023-12-20T04:44:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.377+0000 7ff27d804640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff27806c460 con 0x7ff2781597e0 2023-12-20T04:44:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.377+0000 7ff276ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 0x7ff278159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.377+0000 7ff2777fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff2780cb090 0x7ff2780cb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.377+0000 7ff276ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 0x7ff278159bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:46948/0 (socket says 172.21.15.32:46948) 2023-12-20T04:44:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.377+0000 7ff276ffd640 1 -- 172.21.15.32:0/4146746801 learned_addr learned my addr 172.21.15.32:0/4146746801 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.381+0000 7ff276ffd640 1 -- 172.21.15.32:0/4146746801 >> v2:172.21.15.39:3300/0 conn(0x7ff2780cb090 msgr2=0x7ff2780cb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.381+0000 7ff276ffd640 1 --2- 172.21.15.32:0/4146746801 >> v2:172.21.15.39:3300/0 conn(0x7ff2780cb090 0x7ff2780cb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.381+0000 7ff276ffd640 1 -- 172.21.15.32:0/4146746801 >> v1:172.21.15.39:6789/0 conn(0x7ff2780c8580 legacy=0x7ff2780b0800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.381+0000 7ff276ffd640 1 -- 172.21.15.32:0/4146746801 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff27806c740 con 0x7ff2781597e0 2023-12-20T04:44:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.381+0000 7ff276ffd640 1 --2- 172.21.15.32:0/4146746801 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 0x7ff278159bc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff268009870 tx=0x7ff26801e750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a6a1451930fc7c56 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.381+0000 7ff2767fc640 1 -- 172.21.15.32:0/4146746801 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2680a8610 con 0x7ff2781597e0 2023-12-20T04:44:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.381+0000 7ff2767fc640 1 -- 172.21.15.32:0/4146746801 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2680a8770 con 0x7ff2781597e0 2023-12-20T04:44:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.381+0000 7ff2767fc640 1 -- 172.21.15.32:0/4146746801 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2680a8b00 con 0x7ff2781597e0 2023-12-20T04:44:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 -- 172.21.15.32:0/4146746801 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 msgr2=0x7ff278159bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 --2- 172.21.15.32:0/4146746801 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 0x7ff278159bc0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7ff268009870 tx=0x7ff26801e750 comp rx=0 tx=0).stop 2023-12-20T04:44:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 -- 172.21.15.32:0/4146746801 shutdown_connections 2023-12-20T04:44:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 --2- 172.21.15.32:0/4146746801 >> v2:172.21.15.39:3300/0 conn(0x7ff2780cb090 0x7ff2780cb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 --2- 172.21.15.32:0/4146746801 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 0x7ff278159bc0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 -- 172.21.15.32:0/4146746801 >> 172.21.15.32:0/4146746801 conn(0x7ff278069d20 msgr2=0x7ff27806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 -- 172.21.15.32:0/4146746801 shutdown_connections 2023-12-20T04:44:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 -- 172.21.15.32:0/4146746801 wait complete. 2023-12-20T04:44:31.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 Processor -- start 2023-12-20T04:44:31.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 -- start start 2023-12-20T04:44:31.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2780c8580 0x7ff27814a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2780cb090 0x7ff27814fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 0x7ff2781629b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff278152810 con 0x7ff2780cb090 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff27814ac30 con 0x7ff2780c8580 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff27d804640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff278150100 con 0x7ff2781597e0 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff276ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2780c8580 0x7ff27814a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff26ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2780cb090 0x7ff27814fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff276ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2780c8580 0x7ff27814a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:56724/0 (socket says 172.21.15.32:56724) 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff276ffd640 1 -- 172.21.15.32:0/2832895556 learned_addr learned my addr 172.21.15.32:0/2832895556 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff2777fe640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 0x7ff2781629b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff276ffd640 1 -- 172.21.15.32:0/2832895556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 msgr2=0x7ff2781629b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff276ffd640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 0x7ff2781629b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff276ffd640 1 -- 172.21.15.32:0/2832895556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2780cb090 msgr2=0x7ff27814fbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff276ffd640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2780cb090 0x7ff27814fbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.385+0000 7ff276ffd640 1 -- 172.21.15.32:0/2832895556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2680094d0 con 0x7ff2780c8580 2023-12-20T04:44:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.389+0000 7ff276ffd640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2780c8580 0x7ff27814a6f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff26801ce00 tx=0x7ff26801b4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.389+0000 7ff274ff9640 1 -- 172.21.15.32:0/2832895556 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2680b3860 con 0x7ff2780c8580 2023-12-20T04:44:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.389+0000 7ff274ff9640 1 -- 172.21.15.32:0/2832895556 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2680b39c0 con 0x7ff2780c8580 2023-12-20T04:44:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.389+0000 7ff274ff9640 1 -- 172.21.15.32:0/2832895556 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2680b3d70 con 0x7ff2780c8580 2023-12-20T04:44:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.389+0000 7ff27d804640 1 -- 172.21.15.32:0/2832895556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2781633c0 con 0x7ff2780c8580 2023-12-20T04:44:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.389+0000 7ff27d804640 1 -- 172.21.15.32:0/2832895556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff278163980 con 0x7ff2780c8580 2023-12-20T04:44:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.389+0000 7ff27d804640 1 -- 172.21.15.32:0/2832895556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2780caf00 con 0x7ff2780c8580 2023-12-20T04:44:31.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.389+0000 7ff274ff9640 1 -- 172.21.15.32:0/2832895556 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff2680c8430 con 0x7ff2780c8580 2023-12-20T04:44:31.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.389+0000 7ff274ff9640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff258065b80 0x7ff258068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.389+0000 7ff274ff9640 1 -- 172.21.15.32:0/2832895556 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7ff26811db50 con 0x7ff2780c8580 2023-12-20T04:44:31.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.389+0000 7ff26ffff640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff258065b80 0x7ff258068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.393+0000 7ff26ffff640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff258065b80 0x7ff258068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff260007fd0 tx=0x7ff260009040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:31.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.393+0000 7ff274ff9640 1 -- 172.21.15.32:0/2832895556 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff26811de60 con 0x7ff2780c8580 2023-12-20T04:44:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.553+0000 7ff27d804640 1 -- 172.21.15.32:0/2832895556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff278074100 con 0x7ff2780c8580 2023-12-20T04:44:31.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff274ff9640 1 -- 172.21.15.32:0/2832895556 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v107) v1 ==== 74+0+16055 (secure 0 0 0) 0x7ff2680ae090 con 0x7ff2780c8580 2023-12-20T04:44:31.555 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:31.555 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":107,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:44:28.820973+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:44:25.451651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-20T04:44:26.819848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:44:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 -- 172.21.15.32:0/2832895556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff258065b80 msgr2=0x7ff258068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff258065b80 0x7ff258068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff260007fd0 tx=0x7ff260009040 comp rx=0 tx=0).stop 2023-12-20T04:44:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 -- 172.21.15.32:0/2832895556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2780c8580 msgr2=0x7ff27814a6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2780c8580 0x7ff27814a6f0 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7ff26801ce00 tx=0x7ff26801b4f0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 -- 172.21.15.32:0/2832895556 shutdown_connections 2023-12-20T04:44:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff258065b80 0x7ff258068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2781597e0 0x7ff2781629b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2780cb090 0x7ff27814fbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 --2- 172.21.15.32:0/2832895556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2780c8580 0x7ff27814a6f0 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 -- 172.21.15.32:0/2832895556 >> 172.21.15.32:0/2832895556 conn(0x7ff278069d20 msgr2=0x7ff2780b28d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 -- 172.21.15.32:0/2832895556 shutdown_connections 2023-12-20T04:44:31.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.557+0000 7ff27d804640 1 -- 172.21.15.32:0/2832895556 wait complete. 2023-12-20T04:44:31.577 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T04:44:31.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:44:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d58c8c640 1 Processor -- start 2023-12-20T04:44:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d58c8c640 1 -- start start 2023-12-20T04:44:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d58c8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6d5406ce60 0x7f6d5415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d58c8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d5415f110 0x7f6d5415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d58c8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6d5415bbd0 con 0x7f6d540af5e0 2023-12-20T04:44:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d58c8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6d5415beb0 con 0x7f6d5406ce60 2023-12-20T04:44:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d58c8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d5415e910 con 0x7f6d5415f110 2023-12-20T04:44:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d5259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6d540af5e0 0x7f6d540af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49096/0 (socket says 172.21.15.32:49096) 2023-12-20T04:44:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d5259c640 1 -- 172.21.15.32:0/1435995977 learned_addr learned my addr 172.21.15.32:0/1435995977 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d4bfff640 1 --2- 172.21.15.32:0/1435995977 >> v2:172.21.15.32:3300/0 conn(0x7f6d5406ce60 0x7f6d5415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d52d9d640 1 --2- 172.21.15.32:0/1435995977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d5415f110 0x7f6d5415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d51d9b640 1 -- 172.21.15.32:0/1435995977 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4145414295 0 0) 0x7f6d5415bbd0 con 0x7f6d540af5e0 2023-12-20T04:44:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d4bfff640 1 -- 172.21.15.32:0/1435995977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d5415f110 msgr2=0x7f6d5415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d4bfff640 1 --2- 172.21.15.32:0/1435995977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d5415f110 0x7f6d5415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d4bfff640 1 -- 172.21.15.32:0/1435995977 >> v1:172.21.15.32:6789/0 conn(0x7f6d540af5e0 legacy=0x7f6d540af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d4bfff640 1 -- 172.21.15.32:0/1435995977 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d5415ebf0 con 0x7f6d5406ce60 2023-12-20T04:44:31.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.721+0000 7f6d52d9d640 1 --2- 172.21.15.32:0/1435995977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d5415f110 0x7f6d5415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:31.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d4bfff640 1 --2- 172.21.15.32:0/1435995977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5406ce60 0x7f6d5415b280 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0097c0 tx=0x7f6d3c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d03e2254fd8ad1aa server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:31.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d51d9b640 1 -- 172.21.15.32:0/1435995977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d5415b9d0 con 0x7f6d5406ce60 2023-12-20T04:44:31.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d51d9b640 1 -- 172.21.15.32:0/1435995977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3c0ad640 con 0x7f6d5406ce60 2023-12-20T04:44:31.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d51d9b640 1 -- 172.21.15.32:0/1435995977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c0ad9d0 con 0x7f6d5406ce60 2023-12-20T04:44:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d58c8c640 1 -- 172.21.15.32:0/1435995977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5406ce60 msgr2=0x7f6d5415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d58c8c640 1 --2- 172.21.15.32:0/1435995977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5406ce60 0x7f6d5415b280 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c0097c0 tx=0x7f6d3c01c3f0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d58c8c640 1 -- 172.21.15.32:0/1435995977 shutdown_connections 2023-12-20T04:44:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d58c8c640 1 --2- 172.21.15.32:0/1435995977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d5406ce60 0x7f6d5415b280 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d58c8c640 1 --2- 172.21.15.32:0/1435995977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d5415f110 0x7f6d5415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d58c8c640 1 -- 172.21.15.32:0/1435995977 >> 172.21.15.32:0/1435995977 conn(0x7f6d5406a320 msgr2=0x7f6d540ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:31.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d58c8c640 1 -- 172.21.15.32:0/1435995977 shutdown_connections 2023-12-20T04:44:31.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.725+0000 7f6d58c8c640 1 -- 172.21.15.32:0/1435995977 wait complete. 2023-12-20T04:44:31.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d58c8c640 1 Processor -- start 2023-12-20T04:44:31.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d58c8c640 1 -- start start 2023-12-20T04:44:31.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d58c8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d5406ce60 0x7f6d541482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d58c8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d540af5e0 0x7f6d54070d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d58c8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d5415f110 0x7f6d540cd650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d58c8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d540cc890 con 0x7f6d540af5e0 2023-12-20T04:44:31.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d58c8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6d540ccb70 con 0x7f6d5406ce60 2023-12-20T04:44:31.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d58c8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6d540cce50 con 0x7f6d5415f110 2023-12-20T04:44:31.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d540af5e0 0x7f6d54070d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d540af5e0 0x7f6d54070d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60524/0 (socket says 172.21.15.32:60524) 2023-12-20T04:44:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4bfff640 1 -- 172.21.15.32:0/3446798357 learned_addr learned my addr 172.21.15.32:0/3446798357 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d52d9d640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d5415f110 0x7f6d540cd650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d5259c640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d5406ce60 0x7f6d541482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4bfff640 1 -- 172.21.15.32:0/3446798357 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d5415f110 msgr2=0x7f6d540cd650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4bfff640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d5415f110 0x7f6d540cd650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4bfff640 1 -- 172.21.15.32:0/3446798357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d5406ce60 msgr2=0x7f6d541482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4bfff640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d5406ce60 0x7f6d541482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4bfff640 1 -- 172.21.15.32:0/3446798357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6d3c009420 con 0x7f6d540af5e0 2023-12-20T04:44:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4bfff640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d540af5e0 0x7f6d54070d40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c01a020 tx=0x7f6d3c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4affd640 1 -- 172.21.15.32:0/3446798357 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c004020 con 0x7f6d540af5e0 2023-12-20T04:44:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4affd640 1 -- 172.21.15.32:0/3446798357 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6d3c0a6040 con 0x7f6d540af5e0 2023-12-20T04:44:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d4affd640 1 -- 172.21.15.32:0/3446798357 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6d3c004380 con 0x7f6d540af5e0 2023-12-20T04:44:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d58c8c640 1 -- 172.21.15.32:0/3446798357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6d540cd130 con 0x7f6d540af5e0 2023-12-20T04:44:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.729+0000 7f6d58c8c640 1 -- 172.21.15.32:0/3446798357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6d54071730 con 0x7f6d540af5e0 2023-12-20T04:44:31.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.733+0000 7f6d58c8c640 1 -- 172.21.15.32:0/3446798357 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6d5404e000 con 0x7f6d540af5e0 2023-12-20T04:44:31.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.733+0000 7f6d4affd640 1 -- 172.21.15.32:0/3446798357 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6d3c0b4040 con 0x7f6d540af5e0 2023-12-20T04:44:31.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.733+0000 7f6d4affd640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d34065b80 0x7f6d34068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:31.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.733+0000 7f6d4affd640 1 -- 172.21.15.32:0/3446798357 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6d3c11d520 con 0x7f6d540af5e0 2023-12-20T04:44:31.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.733+0000 7f6d5259c640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d34065b80 0x7f6d34068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:31.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.733+0000 7f6d5259c640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d34065b80 0x7f6d34068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6d440010f0 tx=0x7f6d44009040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:31.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.737+0000 7f6d4affd640 1 -- 172.21.15.32:0/3446798357 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6d3c0eade0 con 0x7f6d540af5e0 2023-12-20T04:44:31.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.897+0000 7f6d58c8c640 1 -- 172.21.15.32:0/3446798357 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6d540ccce0 con 0x7f6d34065b80 2023-12-20T04:44:31.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.905+0000 7f6d4affd640 1 -- 172.21.15.32:0/3446798357 <== mgr.7113 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289174 (secure 0 0 0) 0x7f6d540ccce0 con 0x7f6d34065b80 2023-12-20T04:44:31.909 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:31.910 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:44:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 -- 172.21.15.32:0/3446798357 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d34065b80 msgr2=0x7f6d34068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d34065b80 0x7f6d34068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f6d440010f0 tx=0x7f6d44009040 comp rx=0 tx=0).stop 2023-12-20T04:44:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 -- 172.21.15.32:0/3446798357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d540af5e0 msgr2=0x7f6d54070d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d540af5e0 0x7f6d54070d40 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f6d3c01a020 tx=0x7f6d3c019190 comp rx=0 tx=0).stop 2023-12-20T04:44:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 -- 172.21.15.32:0/3446798357 shutdown_connections 2023-12-20T04:44:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6d34065b80 0x7f6d34068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6d5415f110 0x7f6d540cd650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6d540af5e0 0x7f6d54070d40 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 --2- 172.21.15.32:0/3446798357 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6d5406ce60 0x7f6d541482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 -- 172.21.15.32:0/3446798357 >> 172.21.15.32:0/3446798357 conn(0x7f6d5406a320 msgr2=0x7f6d540af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:31.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 -- 172.21.15.32:0/3446798357 shutdown_connections 2023-12-20T04:44:31.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:31.909+0000 7f6d58c8c640 1 -- 172.21.15.32:0/3446798357 wait complete. 2023-12-20T04:44:31.929 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-20T04:44:31.099803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":637,"num_read_kb":1171,"num_write":305,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3096,"ondisk_log_size":3096,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":320480,"kb_used_data":11424,"kb_used_omap":0,"kb_used_meta":308992,"kb_avail":754654240,"statfs":{"total":773094113280,"available":772765941760,"internally_reserved":0,"allocated":11698176,"data_stored":6110548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316407808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.694039"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909813+0000","last_change":"2023-12-20T04:44:27.855927+0000","last_active":"2023-12-20T04:44:28.909813+0000","last_peered":"2023-12-20T04:44:28.909813+0000","last_clean":"2023-12-20T04:44:28.909813+0000","last_became_active":"2023-12-20T04:44:27.855360+0000","last_became_peered":"2023-12-20T04:44:27.855360+0000","last_unstale":"2023-12-20T04:44:28.909813+0000","last_undegraded":"2023-12-20T04:44:28.909813+0000","last_fullsized":"2023-12-20T04:44:28.909813+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:01:46.648912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910343+0000","last_change":"2023-12-20T04:44:27.857546+0000","last_active":"2023-12-20T04:44:28.910343+0000","last_peered":"2023-12-20T04:44:28.910343+0000","last_clean":"2023-12-20T04:44:28.910343+0000","last_became_active":"2023-12-20T04:44:27.856534+0000","last_became_peered":"2023-12-20T04:44:27.856534+0000","last_unstale":"2023-12-20T04:44:28.910343+0000","last_undegraded":"2023-12-20T04:44:28.910343+0000","last_fullsized":"2023-12-20T04:44:28.910343+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:27:38.467452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909281+0000","last_change":"2023-12-20T04:44:27.863618+0000","last_active":"2023-12-20T04:44:28.909281+0000","last_peered":"2023-12-20T04:44:28.909281+0000","last_clean":"2023-12-20T04:44:28.909281+0000","last_became_active":"2023-12-20T04:44:27.863269+0000","last_became_peered":"2023-12-20T04:44:27.863269+0000","last_unstale":"2023-12-20T04:44:28.909281+0000","last_undegraded":"2023-12-20T04:44:28.909281+0000","last_fullsized":"2023-12-20T04:44:28.909281+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:46:48.669519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909729+0000","last_change":"2023-12-20T04:44:27.836048+0000","last_active":"2023-12-20T04:44:28.909729+0000","last_peered":"2023-12-20T04:44:28.909729+0000","last_clean":"2023-12-20T04:44:28.909729+0000","last_became_active":"2023-12-20T04:44:27.835175+0000","last_became_peered":"2023-12-20T04:44:27.835175+0000","last_unstale":"2023-12-20T04:44:28.909729+0000","last_undegraded":"2023-12-20T04:44:28.909729+0000","last_fullsized":"2023-12-20T04:44:28.909729+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:37:12.392685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827799+0000","last_change":"2023-12-20T04:44:27.843846+0000","last_active":"2023-12-20T04:44:28.827799+0000","last_peered":"2023-12-20T04:44:28.827799+0000","last_clean":"2023-12-20T04:44:28.827799+0000","last_became_active":"2023-12-20T04:44:27.843567+0000","last_became_peered":"2023-12-20T04:44:27.843567+0000","last_unstale":"2023-12-20T04:44:28.827799+0000","last_undegraded":"2023-12-20T04:44:28.827799+0000","last_fullsized":"2023-12-20T04:44:28.827799+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:24.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827418+0000","last_change":"2023-12-20T04:44:27.850953+0000","last_active":"2023-12-20T04:44:28.827418+0000","last_peered":"2023-12-20T04:44:28.827418+0000","last_clean":"2023-12-20T04:44:28.827418+0000","last_became_active":"2023-12-20T04:44:27.850598+0000","last_became_peered":"2023-12-20T04:44:27.850598+0000","last_unstale":"2023-12-20T04:44:28.827418+0000","last_undegraded":"2023-12-20T04:44:28.827418+0000","last_fullsized":"2023-12-20T04:44:28.827418+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:01:46.648912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908825+0000","last_change":"2023-12-20T04:44:27.854501+0000","last_active":"2023-12-20T04:44:28.908825+0000","last_peered":"2023-12-20T04:44:28.908825+0000","last_clean":"2023-12-20T04:44:28.908825+0000","last_became_active":"2023-12-20T04:44:27.854208+0000","last_became_peered":"2023-12-20T04:44:27.854208+0000","last_unstale":"2023-12-20T04:44:28.908825+0000","last_undegraded":"2023-12-20T04:44:28.908825+0000","last_fullsized":"2023-12-20T04:44:28.908825+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:08:11.085461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827098+0000","last_change":"2023-12-20T04:44:27.848635+0000","last_active":"2023-12-20T04:44:28.827098+0000","last_peered":"2023-12-20T04:44:28.827098+0000","last_clean":"2023-12-20T04:44:28.827098+0000","last_became_active":"2023-12-20T04:44:27.847032+0000","last_became_peered":"2023-12-20T04:44:27.847032+0000","last_unstale":"2023-12-20T04:44:28.827098+0000","last_undegraded":"2023-12-20T04:44:28.827098+0000","last_fullsized":"2023-12-20T04:44:28.827098+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:49.630920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827694+0000","last_change":"2023-12-20T04:44:27.851334+0000","last_active":"2023-12-20T04:44:28.827694+0000","last_peered":"2023-12-20T04:44:28.827694+0000","last_clean":"2023-12-20T04:44:28.827694+0000","last_became_active":"2023-12-20T04:44:27.849291+0000","last_became_peered":"2023-12-20T04:44:27.849291+0000","last_unstale":"2023-12-20T04:44:28.827694+0000","last_undegraded":"2023-12-20T04:44:28.827694+0000","last_fullsized":"2023-12-20T04:44:28.827694+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:45:05.584416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909835+0000","last_change":"2023-12-20T04:44:27.857058+0000","last_active":"2023-12-20T04:44:28.909835+0000","last_peered":"2023-12-20T04:44:28.909835+0000","last_clean":"2023-12-20T04:44:28.909835+0000","last_became_active":"2023-12-20T04:44:27.856717+0000","last_became_peered":"2023-12-20T04:44:27.856717+0000","last_unstale":"2023-12-20T04:44:28.909835+0000","last_undegraded":"2023-12-20T04:44:28.909835+0000","last_fullsized":"2023-12-20T04:44:28.909835+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:54:36.565658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910350+0000","last_change":"2023-12-20T04:44:27.851856+0000","last_active":"2023-12-20T04:44:28.910350+0000","last_peered":"2023-12-20T04:44:28.910350+0000","last_clean":"2023-12-20T04:44:28.910350+0000","last_became_active":"2023-12-20T04:44:27.851595+0000","last_became_peered":"2023-12-20T04:44:27.851595+0000","last_unstale":"2023-12-20T04:44:28.910350+0000","last_undegraded":"2023-12-20T04:44:28.910350+0000","last_fullsized":"2023-12-20T04:44:28.910350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:34:37.516180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909678+0000","last_change":"2023-12-20T04:44:27.855167+0000","last_active":"2023-12-20T04:44:28.909678+0000","last_peered":"2023-12-20T04:44:28.909678+0000","last_clean":"2023-12-20T04:44:28.909678+0000","last_became_active":"2023-12-20T04:44:27.854857+0000","last_became_peered":"2023-12-20T04:44:27.854857+0000","last_unstale":"2023-12-20T04:44:28.909678+0000","last_undegraded":"2023-12-20T04:44:28.909678+0000","last_fullsized":"2023-12-20T04:44:28.909678+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:47:21.553515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908998+0000","last_change":"2023-12-20T04:44:27.857790+0000","last_active":"2023-12-20T04:44:28.908998+0000","last_peered":"2023-12-20T04:44:28.908998+0000","last_clean":"2023-12-20T04:44:28.908998+0000","last_became_active":"2023-12-20T04:44:27.857483+0000","last_became_peered":"2023-12-20T04:44:27.857483+0000","last_unstale":"2023-12-20T04:44:28.908998+0000","last_undegraded":"2023-12-20T04:44:28.908998+0000","last_fullsized":"2023-12-20T04:44:28.908998+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:24.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910144+0000","last_change":"2023-12-20T04:44:27.840899+0000","last_active":"2023-12-20T04:44:28.910144+0000","last_peered":"2023-12-20T04:44:28.910144+0000","last_clean":"2023-12-20T04:44:28.910144+0000","last_became_active":"2023-12-20T04:44:27.840449+0000","last_became_peered":"2023-12-20T04:44:27.840449+0000","last_unstale":"2023-12-20T04:44:28.910144+0000","last_undegraded":"2023-12-20T04:44:28.910144+0000","last_fullsized":"2023-12-20T04:44:28.910144+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:36.816675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908756+0000","last_change":"2023-12-20T04:44:27.858294+0000","last_active":"2023-12-20T04:44:28.908756+0000","last_peered":"2023-12-20T04:44:28.908756+0000","last_clean":"2023-12-20T04:44:28.908756+0000","last_became_active":"2023-12-20T04:44:27.856272+0000","last_became_peered":"2023-12-20T04:44:27.856272+0000","last_unstale":"2023-12-20T04:44:28.908756+0000","last_undegraded":"2023-12-20T04:44:28.908756+0000","last_fullsized":"2023-12-20T04:44:28.908756+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:00:11.814269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827014+0000","last_change":"2023-12-20T04:44:27.851256+0000","last_active":"2023-12-20T04:44:28.827014+0000","last_peered":"2023-12-20T04:44:28.827014+0000","last_clean":"2023-12-20T04:44:28.827014+0000","last_became_active":"2023-12-20T04:44:27.850909+0000","last_became_peered":"2023-12-20T04:44:27.850909+0000","last_unstale":"2023-12-20T04:44:28.827014+0000","last_undegraded":"2023-12-20T04:44:28.827014+0000","last_fullsized":"2023-12-20T04:44:28.827014+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:22.692826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828375+0000","last_change":"2023-12-20T04:44:27.849744+0000","last_active":"2023-12-20T04:44:28.828375+0000","last_peered":"2023-12-20T04:44:28.828375+0000","last_clean":"2023-12-20T04:44:28.828375+0000","last_became_active":"2023-12-20T04:44:27.848693+0000","last_became_peered":"2023-12-20T04:44:27.848693+0000","last_unstale":"2023-12-20T04:44:28.828375+0000","last_undegraded":"2023-12-20T04:44:28.828375+0000","last_fullsized":"2023-12-20T04:44:28.828375+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:40:00.475342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909843+0000","last_change":"2023-12-20T04:44:27.857314+0000","last_active":"2023-12-20T04:44:28.909843+0000","last_peered":"2023-12-20T04:44:28.909843+0000","last_clean":"2023-12-20T04:44:28.909843+0000","last_became_active":"2023-12-20T04:44:27.857000+0000","last_became_peered":"2023-12-20T04:44:27.857000+0000","last_unstale":"2023-12-20T04:44:28.909843+0000","last_undegraded":"2023-12-20T04:44:28.909843+0000","last_fullsized":"2023-12-20T04:44:28.909843+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:16:05.460113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826283+0000","last_change":"2023-12-20T04:44:27.845451+0000","last_active":"2023-12-20T04:44:28.826283+0000","last_peered":"2023-12-20T04:44:28.826283+0000","last_clean":"2023-12-20T04:44:28.826283+0000","last_became_active":"2023-12-20T04:44:27.845184+0000","last_became_peered":"2023-12-20T04:44:27.845184+0000","last_unstale":"2023-12-20T04:44:28.826283+0000","last_undegraded":"2023-12-20T04:44:28.826283+0000","last_fullsized":"2023-12-20T04:44:28.826283+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:42:32.892908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909801+0000","last_change":"2023-12-20T04:44:27.845334+0000","last_active":"2023-12-20T04:44:28.909801+0000","last_peered":"2023-12-20T04:44:28.909801+0000","last_clean":"2023-12-20T04:44:28.909801+0000","last_became_active":"2023-12-20T04:44:27.843848+0000","last_became_peered":"2023-12-20T04:44:27.843848+0000","last_unstale":"2023-12-20T04:44:28.909801+0000","last_undegraded":"2023-12-20T04:44:28.909801+0000","last_fullsized":"2023-12-20T04:44:28.909801+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:58.386091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909668+0000","last_change":"2023-12-20T04:44:27.848203+0000","last_active":"2023-12-20T04:44:28.909668+0000","last_peered":"2023-12-20T04:44:28.909668+0000","last_clean":"2023-12-20T04:44:28.909668+0000","last_became_active":"2023-12-20T04:44:27.847931+0000","last_became_peered":"2023-12-20T04:44:27.847931+0000","last_unstale":"2023-12-20T04:44:28.909668+0000","last_undegraded":"2023-12-20T04:44:28.909668+0000","last_fullsized":"2023-12-20T04:44:28.909668+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:29.647550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909187+0000","last_change":"2023-12-20T04:44:27.853302+0000","last_active":"2023-12-20T04:44:28.909187+0000","last_peered":"2023-12-20T04:44:28.909187+0000","last_clean":"2023-12-20T04:44:28.909187+0000","last_became_active":"2023-12-20T04:44:27.853041+0000","last_became_peered":"2023-12-20T04:44:27.853041+0000","last_unstale":"2023-12-20T04:44:28.909187+0000","last_undegraded":"2023-12-20T04:44:28.909187+0000","last_fullsized":"2023-12-20T04:44:28.909187+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:36.231277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909260+0000","last_change":"2023-12-20T04:44:27.857085+0000","last_active":"2023-12-20T04:44:28.909260+0000","last_peered":"2023-12-20T04:44:28.909260+0000","last_clean":"2023-12-20T04:44:28.909260+0000","last_became_active":"2023-12-20T04:44:27.856051+0000","last_became_peered":"2023-12-20T04:44:27.856051+0000","last_unstale":"2023-12-20T04:44:28.909260+0000","last_undegraded":"2023-12-20T04:44:28.909260+0000","last_fullsized":"2023-12-20T04:44:28.909260+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:45:05.584416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908653+0000","last_change":"2023-12-20T04:44:27.850940+0000","last_active":"2023-12-20T04:44:28.908653+0000","last_peered":"2023-12-20T04:44:28.908653+0000","last_clean":"2023-12-20T04:44:28.908653+0000","last_became_active":"2023-12-20T04:44:27.850676+0000","last_became_peered":"2023-12-20T04:44:27.850676+0000","last_unstale":"2023-12-20T04:44:28.908653+0000","last_undegraded":"2023-12-20T04:44:28.908653+0000","last_fullsized":"2023-12-20T04:44:28.908653+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:03:11.664776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910446+0000","last_change":"2023-12-20T04:44:27.851616+0000","last_active":"2023-12-20T04:44:28.910446+0000","last_peered":"2023-12-20T04:44:28.910446+0000","last_clean":"2023-12-20T04:44:28.910446+0000","last_became_active":"2023-12-20T04:44:27.848794+0000","last_became_peered":"2023-12-20T04:44:27.848794+0000","last_unstale":"2023-12-20T04:44:28.910446+0000","last_undegraded":"2023-12-20T04:44:28.910446+0000","last_fullsized":"2023-12-20T04:44:28.910446+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:30:25.350251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909372+0000","last_change":"2023-12-20T04:44:27.858487+0000","last_active":"2023-12-20T04:44:28.909372+0000","last_peered":"2023-12-20T04:44:28.909372+0000","last_clean":"2023-12-20T04:44:28.909372+0000","last_became_active":"2023-12-20T04:44:27.857706+0000","last_became_peered":"2023-12-20T04:44:27.857706+0000","last_unstale":"2023-12-20T04:44:28.909372+0000","last_undegraded":"2023-12-20T04:44:28.909372+0000","last_fullsized":"2023-12-20T04:44:28.909372+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:47:46.040771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828232+0000","last_change":"2023-12-20T04:44:27.846014+0000","last_active":"2023-12-20T04:44:28.828232+0000","last_peered":"2023-12-20T04:44:28.828232+0000","last_clean":"2023-12-20T04:44:28.828232+0000","last_became_active":"2023-12-20T04:44:27.845729+0000","last_became_peered":"2023-12-20T04:44:27.845729+0000","last_unstale":"2023-12-20T04:44:28.828232+0000","last_undegraded":"2023-12-20T04:44:28.828232+0000","last_fullsized":"2023-12-20T04:44:28.828232+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:34.947486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909704+0000","last_change":"2023-12-20T04:44:27.855352+0000","last_active":"2023-12-20T04:44:28.909704+0000","last_peered":"2023-12-20T04:44:28.909704+0000","last_clean":"2023-12-20T04:44:28.909704+0000","last_became_active":"2023-12-20T04:44:27.855029+0000","last_became_peered":"2023-12-20T04:44:27.855029+0000","last_unstale":"2023-12-20T04:44:28.909704+0000","last_undegraded":"2023-12-20T04:44:28.909704+0000","last_fullsized":"2023-12-20T04:44:28.909704+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:49.630920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909250+0000","last_change":"2023-12-20T04:44:27.839468+0000","last_active":"2023-12-20T04:44:28.909250+0000","last_peered":"2023-12-20T04:44:28.909250+0000","last_clean":"2023-12-20T04:44:28.909250+0000","last_became_active":"2023-12-20T04:44:27.839194+0000","last_became_peered":"2023-12-20T04:44:27.839194+0000","last_unstale":"2023-12-20T04:44:28.909250+0000","last_undegraded":"2023-12-20T04:44:28.909250+0000","last_fullsized":"2023-12-20T04:44:28.909250+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:14:38.680960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909208+0000","last_change":"2023-12-20T04:44:27.846673+0000","last_active":"2023-12-20T04:44:28.909208+0000","last_peered":"2023-12-20T04:44:28.909208+0000","last_clean":"2023-12-20T04:44:28.909208+0000","last_became_active":"2023-12-20T04:44:27.845807+0000","last_became_peered":"2023-12-20T04:44:27.845807+0000","last_unstale":"2023-12-20T04:44:28.909208+0000","last_undegraded":"2023-12-20T04:44:28.909208+0000","last_fullsized":"2023-12-20T04:44:28.909208+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:04.915815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827968+0000","last_change":"2023-12-20T04:44:27.848267+0000","last_active":"2023-12-20T04:44:28.827968+0000","last_peered":"2023-12-20T04:44:28.827968+0000","last_clean":"2023-12-20T04:44:28.827968+0000","last_became_active":"2023-12-20T04:44:27.847933+0000","last_became_peered":"2023-12-20T04:44:27.847933+0000","last_unstale":"2023-12-20T04:44:28.827968+0000","last_undegraded":"2023-12-20T04:44:28.827968+0000","last_fullsized":"2023-12-20T04:44:28.827968+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:26.537288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909264+0000","last_change":"2023-12-20T04:44:27.853330+0000","last_active":"2023-12-20T04:44:28.909264+0000","last_peered":"2023-12-20T04:44:28.909264+0000","last_clean":"2023-12-20T04:44:28.909264+0000","last_became_active":"2023-12-20T04:44:27.852636+0000","last_became_peered":"2023-12-20T04:44:27.852636+0000","last_unstale":"2023-12-20T04:44:28.909264+0000","last_undegraded":"2023-12-20T04:44:28.909264+0000","last_fullsized":"2023-12-20T04:44:28.909264+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:27:38.467452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828389+0000","last_change":"2023-12-20T04:44:27.858425+0000","last_active":"2023-12-20T04:44:28.828389+0000","last_peered":"2023-12-20T04:44:28.828389+0000","last_clean":"2023-12-20T04:44:28.828389+0000","last_became_active":"2023-12-20T04:44:27.857979+0000","last_became_peered":"2023-12-20T04:44:27.857979+0000","last_unstale":"2023-12-20T04:44:28.828389+0000","last_undegraded":"2023-12-20T04:44:28.828389+0000","last_fullsized":"2023-12-20T04:44:28.828389+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:07.634419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"106'4","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910062+0000","last_change":"2023-12-20T04:44:27.435403+0000","last_active":"2023-12-20T04:44:28.910062+0000","last_peered":"2023-12-20T04:44:28.910062+0000","last_clean":"2023-12-20T04:44:28.910062+0000","last_became_active":"2023-12-20T04:44:27.435104+0000","last_became_peered":"2023-12-20T04:44:27.435104+0000","last_unstale":"2023-12-20T04:44:28.910062+0000","last_undegraded":"2023-12-20T04:44:28.910062+0000","last_fullsized":"2023-12-20T04:44:28.910062+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:59:16.483399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909986+0000","last_change":"2023-12-20T04:44:27.856588+0000","last_active":"2023-12-20T04:44:28.909986+0000","last_peered":"2023-12-20T04:44:28.909986+0000","last_clean":"2023-12-20T04:44:28.909986+0000","last_became_active":"2023-12-20T04:44:27.856274+0000","last_became_peered":"2023-12-20T04:44:27.856274+0000","last_unstale":"2023-12-20T04:44:28.909986+0000","last_undegraded":"2023-12-20T04:44:28.909986+0000","last_fullsized":"2023-12-20T04:44:28.909986+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:44.406191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828507+0000","last_change":"2023-12-20T04:44:27.435541+0000","last_active":"2023-12-20T04:44:28.828507+0000","last_peered":"2023-12-20T04:44:28.828507+0000","last_clean":"2023-12-20T04:44:28.828507+0000","last_became_active":"2023-12-20T04:44:27.435264+0000","last_became_peered":"2023-12-20T04:44:27.435264+0000","last_unstale":"2023-12-20T04:44:28.828507+0000","last_undegraded":"2023-12-20T04:44:28.828507+0000","last_fullsized":"2023-12-20T04:44:28.828507+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:45:04.581495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909807+0000","last_change":"2023-12-20T04:44:27.847375+0000","last_active":"2023-12-20T04:44:28.909807+0000","last_peered":"2023-12-20T04:44:28.909807+0000","last_clean":"2023-12-20T04:44:28.909807+0000","last_became_active":"2023-12-20T04:44:27.846856+0000","last_became_peered":"2023-12-20T04:44:27.846856+0000","last_unstale":"2023-12-20T04:44:28.909807+0000","last_undegraded":"2023-12-20T04:44:28.909807+0000","last_fullsized":"2023-12-20T04:44:28.909807+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:13:56.386220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826312+0000","last_change":"2023-12-20T04:44:27.428863+0000","last_active":"2023-12-20T04:44:28.826312+0000","last_peered":"2023-12-20T04:44:28.826312+0000","last_clean":"2023-12-20T04:44:28.826312+0000","last_became_active":"2023-12-20T04:44:27.428567+0000","last_became_peered":"2023-12-20T04:44:27.428567+0000","last_unstale":"2023-12-20T04:44:28.826312+0000","last_undegraded":"2023-12-20T04:44:28.826312+0000","last_fullsized":"2023-12-20T04:44:28.826312+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:39:59.472421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909332+0000","last_change":"2023-12-20T04:44:27.860735+0000","last_active":"2023-12-20T04:44:28.909332+0000","last_peered":"2023-12-20T04:44:28.909332+0000","last_clean":"2023-12-20T04:44:28.909332+0000","last_became_active":"2023-12-20T04:44:27.860431+0000","last_became_peered":"2023-12-20T04:44:27.860431+0000","last_unstale":"2023-12-20T04:44:28.909332+0000","last_undegraded":"2023-12-20T04:44:28.909332+0000","last_fullsized":"2023-12-20T04:44:28.909332+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:51:14.492093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826116+0000","last_change":"2023-12-20T04:44:27.431692+0000","last_active":"2023-12-20T04:44:28.826116+0000","last_peered":"2023-12-20T04:44:28.826116+0000","last_clean":"2023-12-20T04:44:28.826116+0000","last_became_active":"2023-12-20T04:44:27.429700+0000","last_became_peered":"2023-12-20T04:44:27.429700+0000","last_unstale":"2023-12-20T04:44:28.826116+0000","last_undegraded":"2023-12-20T04:44:28.826116+0000","last_fullsized":"2023-12-20T04:44:28.826116+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:47:45.037850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909400+0000","last_change":"2023-12-20T04:44:27.846467+0000","last_active":"2023-12-20T04:44:28.909400+0000","last_peered":"2023-12-20T04:44:28.909400+0000","last_clean":"2023-12-20T04:44:28.909400+0000","last_became_active":"2023-12-20T04:44:27.845424+0000","last_became_peered":"2023-12-20T04:44:27.845424+0000","last_unstale":"2023-12-20T04:44:28.909400+0000","last_undegraded":"2023-12-20T04:44:28.909400+0000","last_fullsized":"2023-12-20T04:44:28.909400+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:21:04.322894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828180+0000","last_change":"2023-12-20T04:44:27.438920+0000","last_active":"2023-12-20T04:44:28.828180+0000","last_peered":"2023-12-20T04:44:28.828180+0000","last_clean":"2023-12-20T04:44:28.828180+0000","last_became_active":"2023-12-20T04:44:27.438589+0000","last_became_peered":"2023-12-20T04:44:27.438589+0000","last_unstale":"2023-12-20T04:44:28.828180+0000","last_undegraded":"2023-12-20T04:44:28.828180+0000","last_fullsized":"2023-12-20T04:44:28.828180+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:13:55.383299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827909+0000","last_change":"2023-12-20T04:44:27.850552+0000","last_active":"2023-12-20T04:44:28.827909+0000","last_peered":"2023-12-20T04:44:28.827909+0000","last_clean":"2023-12-20T04:44:28.827909+0000","last_became_active":"2023-12-20T04:44:27.850234+0000","last_became_peered":"2023-12-20T04:44:27.850234+0000","last_unstale":"2023-12-20T04:44:28.827909+0000","last_undegraded":"2023-12-20T04:44:28.827909+0000","last_fullsized":"2023-12-20T04:44:28.827909+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:47:46.040771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827728+0000","last_change":"2023-12-20T04:44:27.437023+0000","last_active":"2023-12-20T04:44:28.827728+0000","last_peered":"2023-12-20T04:44:28.827728+0000","last_clean":"2023-12-20T04:44:28.827728+0000","last_became_active":"2023-12-20T04:44:27.436495+0000","last_became_peered":"2023-12-20T04:44:27.436495+0000","last_unstale":"2023-12-20T04:44:28.827728+0000","last_undegraded":"2023-12-20T04:44:28.827728+0000","last_fullsized":"2023-12-20T04:44:28.827728+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:48.627999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908948+0000","last_change":"2023-12-20T04:44:27.857393+0000","last_active":"2023-12-20T04:44:28.908948+0000","last_peered":"2023-12-20T04:44:28.908948+0000","last_clean":"2023-12-20T04:44:28.908948+0000","last_became_active":"2023-12-20T04:44:27.857086+0000","last_became_peered":"2023-12-20T04:44:27.857086+0000","last_unstale":"2023-12-20T04:44:28.908948+0000","last_undegraded":"2023-12-20T04:44:28.908948+0000","last_fullsized":"2023-12-20T04:44:28.908948+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:26.537288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.825765+0000","last_change":"2023-12-20T04:44:27.426439+0000","last_active":"2023-12-20T04:44:28.825765+0000","last_peered":"2023-12-20T04:44:28.825765+0000","last_clean":"2023-12-20T04:44:28.825765+0000","last_became_active":"2023-12-20T04:44:27.426120+0000","last_became_peered":"2023-12-20T04:44:27.426120+0000","last_unstale":"2023-12-20T04:44:28.825765+0000","last_undegraded":"2023-12-20T04:44:28.825765+0000","last_fullsized":"2023-12-20T04:44:28.825765+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:25.534367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909005+0000","last_change":"2023-12-20T04:44:27.858818+0000","last_active":"2023-12-20T04:44:28.909005+0000","last_peered":"2023-12-20T04:44:28.909005+0000","last_clean":"2023-12-20T04:44:28.909005+0000","last_became_active":"2023-12-20T04:44:27.858552+0000","last_became_peered":"2023-12-20T04:44:27.858552+0000","last_unstale":"2023-12-20T04:44:28.909005+0000","last_undegraded":"2023-12-20T04:44:28.909005+0000","last_fullsized":"2023-12-20T04:44:28.909005+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:32:02.736821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909086+0000","last_change":"2023-12-20T04:44:27.435217+0000","last_active":"2023-12-20T04:44:28.909086+0000","last_peered":"2023-12-20T04:44:28.909086+0000","last_clean":"2023-12-20T04:44:28.909086+0000","last_became_active":"2023-12-20T04:44:27.434970+0000","last_became_peered":"2023-12-20T04:44:27.434970+0000","last_unstale":"2023-12-20T04:44:28.909086+0000","last_undegraded":"2023-12-20T04:44:28.909086+0000","last_fullsized":"2023-12-20T04:44:28.909086+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:53.664083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909448+0000","last_change":"2023-12-20T04:44:27.840070+0000","last_active":"2023-12-20T04:44:28.909448+0000","last_peered":"2023-12-20T04:44:28.909448+0000","last_clean":"2023-12-20T04:44:28.909448+0000","last_became_active":"2023-12-20T04:44:27.839653+0000","last_became_peered":"2023-12-20T04:44:27.839653+0000","last_unstale":"2023-12-20T04:44:28.909448+0000","last_undegraded":"2023-12-20T04:44:28.909448+0000","last_fullsized":"2023-12-20T04:44:28.909448+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:47:27.704952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910089+0000","last_change":"2023-12-20T04:44:27.441770+0000","last_active":"2023-12-20T04:44:28.910089+0000","last_peered":"2023-12-20T04:44:28.910089+0000","last_clean":"2023-12-20T04:44:28.910089+0000","last_became_active":"2023-12-20T04:44:27.441398+0000","last_became_peered":"2023-12-20T04:44:27.441398+0000","last_unstale":"2023-12-20T04:44:28.910089+0000","last_undegraded":"2023-12-20T04:44:28.910089+0000","last_fullsized":"2023-12-20T04:44:28.910089+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:53:11.376149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827808+0000","last_change":"2023-12-20T04:44:27.855738+0000","last_active":"2023-12-20T04:44:28.827808+0000","last_peered":"2023-12-20T04:44:28.827808+0000","last_clean":"2023-12-20T04:44:28.827808+0000","last_became_active":"2023-12-20T04:44:27.855437+0000","last_became_peered":"2023-12-20T04:44:27.855437+0000","last_unstale":"2023-12-20T04:44:28.827808+0000","last_undegraded":"2023-12-20T04:44:28.827808+0000","last_fullsized":"2023-12-20T04:44:28.827808+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:54:36.565658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909885+0000","last_change":"2023-12-20T04:44:27.439468+0000","last_active":"2023-12-20T04:44:28.909885+0000","last_peered":"2023-12-20T04:44:28.909885+0000","last_clean":"2023-12-20T04:44:28.909885+0000","last_became_active":"2023-12-20T04:44:27.439125+0000","last_became_peered":"2023-12-20T04:44:27.439125+0000","last_unstale":"2023-12-20T04:44:28.909885+0000","last_undegraded":"2023-12-20T04:44:28.909885+0000","last_fullsized":"2023-12-20T04:44:28.909885+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:53.664083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826005+0000","last_change":"2023-12-20T04:44:27.857696+0000","last_active":"2023-12-20T04:44:28.826005+0000","last_peered":"2023-12-20T04:44:28.826005+0000","last_clean":"2023-12-20T04:44:28.826005+0000","last_became_active":"2023-12-20T04:44:27.857374+0000","last_became_peered":"2023-12-20T04:44:27.857374+0000","last_unstale":"2023-12-20T04:44:28.826005+0000","last_undegraded":"2023-12-20T04:44:28.826005+0000","last_fullsized":"2023-12-20T04:44:28.826005+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:02:33.755429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"106'3","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909927+0000","last_change":"2023-12-20T04:44:27.436072+0000","last_active":"2023-12-20T04:44:28.909927+0000","last_peered":"2023-12-20T04:44:28.909927+0000","last_clean":"2023-12-20T04:44:28.909927+0000","last_became_active":"2023-12-20T04:44:27.435773+0000","last_became_peered":"2023-12-20T04:44:27.435773+0000","last_unstale":"2023-12-20T04:44:28.909927+0000","last_undegraded":"2023-12-20T04:44:28.909927+0000","last_fullsized":"2023-12-20T04:44:28.909927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:33.813685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828323+0000","last_change":"2023-12-20T04:44:27.853454+0000","last_active":"2023-12-20T04:44:28.828323+0000","last_peered":"2023-12-20T04:44:28.828323+0000","last_clean":"2023-12-20T04:44:28.828323+0000","last_became_active":"2023-12-20T04:44:27.853147+0000","last_became_peered":"2023-12-20T04:44:27.853147+0000","last_unstale":"2023-12-20T04:44:28.828323+0000","last_undegraded":"2023-12-20T04:44:28.828323+0000","last_fullsized":"2023-12-20T04:44:28.828323+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:19:04.048295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"106'6","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827501+0000","last_change":"2023-12-20T04:44:27.435127+0000","last_active":"2023-12-20T04:44:28.827501+0000","last_peered":"2023-12-20T04:44:28.827501+0000","last_clean":"2023-12-20T04:44:28.827501+0000","last_became_active":"2023-12-20T04:44:27.434734+0000","last_became_peered":"2023-12-20T04:44:27.434734+0000","last_unstale":"2023-12-20T04:44:28.827501+0000","last_undegraded":"2023-12-20T04:44:28.827501+0000","last_fullsized":"2023-12-20T04:44:28.827501+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:59:36.745949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.825807+0000","last_change":"2023-12-20T04:44:27.846671+0000","last_active":"2023-12-20T04:44:28.825807+0000","last_peered":"2023-12-20T04:44:28.825807+0000","last_clean":"2023-12-20T04:44:28.825807+0000","last_became_active":"2023-12-20T04:44:27.845561+0000","last_became_peered":"2023-12-20T04:44:27.845561+0000","last_unstale":"2023-12-20T04:44:28.825807+0000","last_undegraded":"2023-12-20T04:44:28.825807+0000","last_fullsized":"2023-12-20T04:44:28.825807+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:59:33.634682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909381+0000","last_change":"2023-12-20T04:44:27.429846+0000","last_active":"2023-12-20T04:44:28.909381+0000","last_peered":"2023-12-20T04:44:28.909381+0000","last_clean":"2023-12-20T04:44:28.909381+0000","last_became_active":"2023-12-20T04:44:27.427964+0000","last_became_peered":"2023-12-20T04:44:27.427964+0000","last_unstale":"2023-12-20T04:44:28.909381+0000","last_undegraded":"2023-12-20T04:44:28.909381+0000","last_fullsized":"2023-12-20T04:44:28.909381+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:57.383170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909077+0000","last_change":"2023-12-20T04:44:27.856468+0000","last_active":"2023-12-20T04:44:28.909077+0000","last_peered":"2023-12-20T04:44:28.909077+0000","last_clean":"2023-12-20T04:44:28.909077+0000","last_became_active":"2023-12-20T04:44:27.856191+0000","last_became_peered":"2023-12-20T04:44:27.856191+0000","last_unstale":"2023-12-20T04:44:28.909077+0000","last_undegraded":"2023-12-20T04:44:28.909077+0000","last_fullsized":"2023-12-20T04:44:28.909077+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:40:00.475342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908941+0000","last_change":"2023-12-20T04:44:27.437001+0000","last_active":"2023-12-20T04:44:28.908941+0000","last_peered":"2023-12-20T04:44:28.908941+0000","last_clean":"2023-12-20T04:44:28.908941+0000","last_became_active":"2023-12-20T04:44:27.436661+0000","last_became_peered":"2023-12-20T04:44:27.436661+0000","last_unstale":"2023-12-20T04:44:28.908941+0000","last_undegraded":"2023-12-20T04:44:28.908941+0000","last_fullsized":"2023-12-20T04:44:28.908941+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:33.944565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909060+0000","last_change":"2023-12-20T04:44:27.839452+0000","last_active":"2023-12-20T04:44:28.909060+0000","last_peered":"2023-12-20T04:44:28.909060+0000","last_clean":"2023-12-20T04:44:28.909060+0000","last_became_active":"2023-12-20T04:44:27.839172+0000","last_became_peered":"2023-12-20T04:44:27.839172+0000","last_unstale":"2023-12-20T04:44:28.909060+0000","last_undegraded":"2023-12-20T04:44:28.909060+0000","last_fullsized":"2023-12-20T04:44:28.909060+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:59:37.748870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827120+0000","last_change":"2023-12-20T04:44:27.439496+0000","last_active":"2023-12-20T04:44:28.827120+0000","last_peered":"2023-12-20T04:44:28.827120+0000","last_clean":"2023-12-20T04:44:28.827120+0000","last_became_active":"2023-12-20T04:44:27.439116+0000","last_became_peered":"2023-12-20T04:44:27.439116+0000","last_unstale":"2023-12-20T04:44:28.827120+0000","last_undegraded":"2023-12-20T04:44:28.827120+0000","last_fullsized":"2023-12-20T04:44:28.827120+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:35.228356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908835+0000","last_change":"2023-12-20T04:44:27.856876+0000","last_active":"2023-12-20T04:44:28.908835+0000","last_peered":"2023-12-20T04:44:28.908835+0000","last_clean":"2023-12-20T04:44:28.908835+0000","last_became_active":"2023-12-20T04:44:27.856571+0000","last_became_peered":"2023-12-20T04:44:27.856571+0000","last_unstale":"2023-12-20T04:44:28.908835+0000","last_undegraded":"2023-12-20T04:44:28.908835+0000","last_fullsized":"2023-12-20T04:44:28.908835+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:06:30.990973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"106'8","reported_seq":19,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.825727+0000","last_change":"2023-12-20T04:44:27.435375+0000","last_active":"2023-12-20T04:44:28.825727+0000","last_peered":"2023-12-20T04:44:28.825727+0000","last_clean":"2023-12-20T04:44:28.825727+0000","last_became_active":"2023-12-20T04:44:27.435084+0000","last_became_peered":"2023-12-20T04:44:27.435084+0000","last_unstale":"2023-12-20T04:44:28.825727+0000","last_undegraded":"2023-12-20T04:44:28.825727+0000","last_fullsized":"2023-12-20T04:44:28.825727+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:51:13.489172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909433+0000","last_change":"2023-12-20T04:44:27.858556+0000","last_active":"2023-12-20T04:44:28.909433+0000","last_peered":"2023-12-20T04:44:28.909433+0000","last_clean":"2023-12-20T04:44:28.909433+0000","last_became_active":"2023-12-20T04:44:27.858213+0000","last_became_peered":"2023-12-20T04:44:27.858213+0000","last_unstale":"2023-12-20T04:44:28.909433+0000","last_undegraded":"2023-12-20T04:44:28.909433+0000","last_fullsized":"2023-12-20T04:44:28.909433+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:10:11.751473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910153+0000","last_change":"2023-12-20T04:44:27.440579+0000","last_active":"2023-12-20T04:44:28.910153+0000","last_peered":"2023-12-20T04:44:28.910153+0000","last_clean":"2023-12-20T04:44:28.910153+0000","last_became_active":"2023-12-20T04:44:27.438310+0000","last_became_peered":"2023-12-20T04:44:27.438310+0000","last_unstale":"2023-12-20T04:44:28.910153+0000","last_undegraded":"2023-12-20T04:44:28.910153+0000","last_fullsized":"2023-12-20T04:44:28.910153+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:16:04.457192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909989+0000","last_change":"2023-12-20T04:44:27.854848+0000","last_active":"2023-12-20T04:44:28.909989+0000","last_peered":"2023-12-20T04:44:28.909989+0000","last_clean":"2023-12-20T04:44:28.909989+0000","last_became_active":"2023-12-20T04:44:27.854536+0000","last_became_peered":"2023-12-20T04:44:27.854536+0000","last_unstale":"2023-12-20T04:44:28.909989+0000","last_undegraded":"2023-12-20T04:44:28.909989+0000","last_fullsized":"2023-12-20T04:44:28.909989+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:08:16.893096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826376+0000","last_change":"2023-12-20T04:44:27.436175+0000","last_active":"2023-12-20T04:44:28.826376+0000","last_peered":"2023-12-20T04:44:28.826376+0000","last_clean":"2023-12-20T04:44:28.826376+0000","last_became_active":"2023-12-20T04:44:27.435909+0000","last_became_peered":"2023-12-20T04:44:27.435909+0000","last_unstale":"2023-12-20T04:44:28.826376+0000","last_undegraded":"2023-12-20T04:44:28.826376+0000","last_fullsized":"2023-12-20T04:44:28.826376+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:42:16.707918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828405+0000","last_change":"2023-12-20T04:44:27.840420+0000","last_active":"2023-12-20T04:44:28.828405+0000","last_peered":"2023-12-20T04:44:28.828405+0000","last_clean":"2023-12-20T04:44:28.828405+0000","last_became_active":"2023-12-20T04:44:27.840101+0000","last_became_peered":"2023-12-20T04:44:27.840101+0000","last_unstale":"2023-12-20T04:44:28.828405+0000","last_undegraded":"2023-12-20T04:44:28.828405+0000","last_fullsized":"2023-12-20T04:44:28.828405+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:10:11.751473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826144+0000","last_change":"2023-12-20T04:44:27.435281+0000","last_active":"2023-12-20T04:44:28.826144+0000","last_peered":"2023-12-20T04:44:28.826144+0000","last_clean":"2023-12-20T04:44:28.826144+0000","last_became_active":"2023-12-20T04:44:27.434999+0000","last_became_peered":"2023-12-20T04:44:27.434999+0000","last_unstale":"2023-12-20T04:44:28.826144+0000","last_undegraded":"2023-12-20T04:44:28.826144+0000","last_fullsized":"2023-12-20T04:44:28.826144+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:23.833193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910203+0000","last_change":"2023-12-20T04:44:27.840722+0000","last_active":"2023-12-20T04:44:28.910203+0000","last_peered":"2023-12-20T04:44:28.910203+0000","last_clean":"2023-12-20T04:44:28.910203+0000","last_became_active":"2023-12-20T04:44:27.840072+0000","last_became_peered":"2023-12-20T04:44:27.840072+0000","last_unstale":"2023-12-20T04:44:28.910203+0000","last_undegraded":"2023-12-20T04:44:28.910203+0000","last_fullsized":"2023-12-20T04:44:28.910203+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:54.667004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909169+0000","last_change":"2023-12-20T04:44:27.437973+0000","last_active":"2023-12-20T04:44:28.909169+0000","last_peered":"2023-12-20T04:44:28.909169+0000","last_clean":"2023-12-20T04:44:28.909169+0000","last_became_active":"2023-12-20T04:44:27.437694+0000","last_became_peered":"2023-12-20T04:44:27.437694+0000","last_unstale":"2023-12-20T04:44:28.909169+0000","last_undegraded":"2023-12-20T04:44:28.909169+0000","last_fullsized":"2023-12-20T04:44:28.909169+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:21.689905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827751+0000","last_change":"2023-12-20T04:44:27.844180+0000","last_active":"2023-12-20T04:44:28.827751+0000","last_peered":"2023-12-20T04:44:28.827751+0000","last_clean":"2023-12-20T04:44:28.827751+0000","last_became_active":"2023-12-20T04:44:27.843861+0000","last_became_peered":"2023-12-20T04:44:27.843861+0000","last_unstale":"2023-12-20T04:44:28.827751+0000","last_undegraded":"2023-12-20T04:44:28.827751+0000","last_fullsized":"2023-12-20T04:44:28.827751+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:02:53.405091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910227+0000","last_change":"2023-12-20T04:44:27.433582+0000","last_active":"2023-12-20T04:44:28.910227+0000","last_peered":"2023-12-20T04:44:28.910227+0000","last_clean":"2023-12-20T04:44:28.910227+0000","last_became_active":"2023-12-20T04:44:27.432715+0000","last_became_peered":"2023-12-20T04:44:27.432715+0000","last_unstale":"2023-12-20T04:44:28.910227+0000","last_undegraded":"2023-12-20T04:44:28.910227+0000","last_fullsized":"2023-12-20T04:44:28.910227+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:10:55.086465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908693+0000","last_change":"2023-12-20T04:44:27.838723+0000","last_active":"2023-12-20T04:44:28.908693+0000","last_peered":"2023-12-20T04:44:28.908693+0000","last_clean":"2023-12-20T04:44:28.908693+0000","last_became_active":"2023-12-20T04:44:27.838472+0000","last_became_peered":"2023-12-20T04:44:27.838472+0000","last_unstale":"2023-12-20T04:44:28.908693+0000","last_undegraded":"2023-12-20T04:44:28.908693+0000","last_fullsized":"2023-12-20T04:44:28.908693+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:14:34.837571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909433+0000","last_change":"2023-12-20T04:44:27.436452+0000","last_active":"2023-12-20T04:44:28.909433+0000","last_peered":"2023-12-20T04:44:28.909433+0000","last_clean":"2023-12-20T04:44:28.909433+0000","last_became_active":"2023-12-20T04:44:27.436153+0000","last_became_peered":"2023-12-20T04:44:27.436153+0000","last_unstale":"2023-12-20T04:44:28.909433+0000","last_undegraded":"2023-12-20T04:44:28.909433+0000","last_fullsized":"2023-12-20T04:44:28.909433+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:19:26.169237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827018+0000","last_change":"2023-12-20T04:44:27.847933+0000","last_active":"2023-12-20T04:44:28.827018+0000","last_peered":"2023-12-20T04:44:28.827018+0000","last_clean":"2023-12-20T04:44:28.827018+0000","last_became_active":"2023-12-20T04:44:27.847474+0000","last_became_peered":"2023-12-20T04:44:27.847474+0000","last_unstale":"2023-12-20T04:44:28.827018+0000","last_undegraded":"2023-12-20T04:44:28.827018+0000","last_fullsized":"2023-12-20T04:44:28.827018+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:49:22.216892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908681+0000","last_change":"2023-12-20T04:44:27.434596+0000","last_active":"2023-12-20T04:44:28.908681+0000","last_peered":"2023-12-20T04:44:28.908681+0000","last_clean":"2023-12-20T04:44:28.908681+0000","last_became_active":"2023-12-20T04:44:27.433098+0000","last_became_peered":"2023-12-20T04:44:27.433098+0000","last_unstale":"2023-12-20T04:44:28.908681+0000","last_undegraded":"2023-12-20T04:44:28.908681+0000","last_fullsized":"2023-12-20T04:44:28.908681+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:19:03.045373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828464+0000","last_change":"2023-12-20T04:44:27.841449+0000","last_active":"2023-12-20T04:44:28.828464+0000","last_peered":"2023-12-20T04:44:28.828464+0000","last_clean":"2023-12-20T04:44:28.828464+0000","last_became_active":"2023-12-20T04:44:27.839703+0000","last_became_peered":"2023-12-20T04:44:27.839703+0000","last_unstale":"2023-12-20T04:44:28.828464+0000","last_undegraded":"2023-12-20T04:44:28.828464+0000","last_fullsized":"2023-12-20T04:44:28.828464+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:51:14.492093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910447+0000","last_change":"2023-12-20T04:44:27.435556+0000","last_active":"2023-12-20T04:44:28.910447+0000","last_peered":"2023-12-20T04:44:28.910447+0000","last_clean":"2023-12-20T04:44:28.910447+0000","last_became_active":"2023-12-20T04:44:27.435244+0000","last_became_peered":"2023-12-20T04:44:27.435244+0000","last_unstale":"2023-12-20T04:44:28.910447+0000","last_undegraded":"2023-12-20T04:44:28.910447+0000","last_fullsized":"2023-12-20T04:44:28.910447+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:28.644629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909596+0000","last_change":"2023-12-20T04:44:27.863560+0000","last_active":"2023-12-20T04:44:28.909596+0000","last_peered":"2023-12-20T04:44:28.909596+0000","last_clean":"2023-12-20T04:44:28.909596+0000","last_became_active":"2023-12-20T04:44:27.863208+0000","last_became_peered":"2023-12-20T04:44:27.863208+0000","last_unstale":"2023-12-20T04:44:28.909596+0000","last_undegraded":"2023-12-20T04:44:28.909596+0000","last_fullsized":"2023-12-20T04:44:28.909596+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:19:04.048295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910315+0000","last_change":"2023-12-20T04:44:27.428114+0000","last_active":"2023-12-20T04:44:28.910315+0000","last_peered":"2023-12-20T04:44:28.910315+0000","last_clean":"2023-12-20T04:44:28.910315+0000","last_became_active":"2023-12-20T04:44:27.427821+0000","last_became_peered":"2023-12-20T04:44:27.427821+0000","last_unstale":"2023-12-20T04:44:28.910315+0000","last_undegraded":"2023-12-20T04:44:28.910315+0000","last_fullsized":"2023-12-20T04:44:28.910315+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:13:19.646627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909823+0000","last_change":"2023-12-20T04:44:27.862284+0000","last_active":"2023-12-20T04:44:28.909823+0000","last_peered":"2023-12-20T04:44:28.909823+0000","last_clean":"2023-12-20T04:44:28.909823+0000","last_became_active":"2023-12-20T04:44:27.858651+0000","last_became_peered":"2023-12-20T04:44:27.858651+0000","last_unstale":"2023-12-20T04:44:28.909823+0000","last_undegraded":"2023-12-20T04:44:28.909823+0000","last_fullsized":"2023-12-20T04:44:28.909823+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:02:53.405091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909986+0000","last_change":"2023-12-20T04:44:27.435039+0000","last_active":"2023-12-20T04:44:28.909986+0000","last_peered":"2023-12-20T04:44:28.909986+0000","last_clean":"2023-12-20T04:44:28.909986+0000","last_became_active":"2023-12-20T04:44:27.433872+0000","last_became_peered":"2023-12-20T04:44:27.433872+0000","last_unstale":"2023-12-20T04:44:28.909986+0000","last_undegraded":"2023-12-20T04:44:28.909986+0000","last_fullsized":"2023-12-20T04:44:28.909986+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:35.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828463+0000","last_change":"2023-12-20T04:44:27.848367+0000","last_active":"2023-12-20T04:44:28.828463+0000","last_peered":"2023-12-20T04:44:28.828463+0000","last_clean":"2023-12-20T04:44:28.828463+0000","last_became_active":"2023-12-20T04:44:27.848080+0000","last_became_peered":"2023-12-20T04:44:27.848080+0000","last_unstale":"2023-12-20T04:44:28.828463+0000","last_undegraded":"2023-12-20T04:44:28.828463+0000","last_fullsized":"2023-12-20T04:44:28.828463+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:46:48.669519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909941+0000","last_change":"2023-12-20T04:44:27.442833+0000","last_active":"2023-12-20T04:44:28.909941+0000","last_peered":"2023-12-20T04:44:28.909941+0000","last_clean":"2023-12-20T04:44:28.909941+0000","last_became_active":"2023-12-20T04:44:27.441611+0000","last_became_peered":"2023-12-20T04:44:27.441611+0000","last_unstale":"2023-12-20T04:44:28.909941+0000","last_undegraded":"2023-12-20T04:44:28.909941+0000","last_fullsized":"2023-12-20T04:44:28.909941+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:14:33.834650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827560+0000","last_change":"2023-12-20T04:44:27.838999+0000","last_active":"2023-12-20T04:44:28.827560+0000","last_peered":"2023-12-20T04:44:28.827560+0000","last_clean":"2023-12-20T04:44:28.827560+0000","last_became_active":"2023-12-20T04:44:27.838683+0000","last_became_peered":"2023-12-20T04:44:27.838683+0000","last_unstale":"2023-12-20T04:44:28.827560+0000","last_undegraded":"2023-12-20T04:44:28.827560+0000","last_fullsized":"2023-12-20T04:44:28.827560+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:44.406191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826057+0000","last_change":"2023-12-20T04:44:27.437342+0000","last_active":"2023-12-20T04:44:28.826057+0000","last_peered":"2023-12-20T04:44:28.826057+0000","last_clean":"2023-12-20T04:44:28.826057+0000","last_became_active":"2023-12-20T04:44:27.437073+0000","last_became_peered":"2023-12-20T04:44:27.437073+0000","last_unstale":"2023-12-20T04:44:28.826057+0000","last_undegraded":"2023-12-20T04:44:28.826057+0000","last_fullsized":"2023-12-20T04:44:28.826057+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:10:10.748552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826013+0000","last_change":"2023-12-20T04:44:27.841942+0000","last_active":"2023-12-20T04:44:28.826013+0000","last_peered":"2023-12-20T04:44:28.826013+0000","last_clean":"2023-12-20T04:44:28.826013+0000","last_became_active":"2023-12-20T04:44:27.841691+0000","last_became_peered":"2023-12-20T04:44:27.841691+0000","last_unstale":"2023-12-20T04:44:28.826013+0000","last_undegraded":"2023-12-20T04:44:28.826013+0000","last_fullsized":"2023-12-20T04:44:28.826013+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:47:27.704952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827823+0000","last_change":"2023-12-20T04:44:27.436369+0000","last_active":"2023-12-20T04:44:28.827823+0000","last_peered":"2023-12-20T04:44:28.827823+0000","last_clean":"2023-12-20T04:44:28.827823+0000","last_became_active":"2023-12-20T04:44:27.432401+0000","last_became_peered":"2023-12-20T04:44:27.432401+0000","last_unstale":"2023-12-20T04:44:28.827823+0000","last_undegraded":"2023-12-20T04:44:28.827823+0000","last_fullsized":"2023-12-20T04:44:28.827823+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:43.403270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908955+0000","last_change":"2023-12-20T04:44:27.852121+0000","last_active":"2023-12-20T04:44:28.908955+0000","last_peered":"2023-12-20T04:44:28.908955+0000","last_clean":"2023-12-20T04:44:28.908955+0000","last_became_active":"2023-12-20T04:44:27.851797+0000","last_became_peered":"2023-12-20T04:44:27.851797+0000","last_unstale":"2023-12-20T04:44:28.908955+0000","last_undegraded":"2023-12-20T04:44:28.908955+0000","last_fullsized":"2023-12-20T04:44:28.908955+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:42:17.710839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909525+0000","last_change":"2023-12-20T04:44:27.444483+0000","last_active":"2023-12-20T04:44:28.909525+0000","last_peered":"2023-12-20T04:44:28.909525+0000","last_clean":"2023-12-20T04:44:28.909525+0000","last_became_active":"2023-12-20T04:44:27.444122+0000","last_became_peered":"2023-12-20T04:44:27.444122+0000","last_unstale":"2023-12-20T04:44:28.909525+0000","last_undegraded":"2023-12-20T04:44:28.909525+0000","last_fullsized":"2023-12-20T04:44:28.909525+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:03:10.661854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909514+0000","last_change":"2023-12-20T04:44:27.862352+0000","last_active":"2023-12-20T04:44:28.909514+0000","last_peered":"2023-12-20T04:44:28.909514+0000","last_clean":"2023-12-20T04:44:28.909514+0000","last_became_active":"2023-12-20T04:44:27.862068+0000","last_became_peered":"2023-12-20T04:44:27.862068+0000","last_unstale":"2023-12-20T04:44:28.909514+0000","last_undegraded":"2023-12-20T04:44:28.909514+0000","last_fullsized":"2023-12-20T04:44:28.909514+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:49:22.216892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827118+0000","last_change":"2023-12-20T04:44:27.443444+0000","last_active":"2023-12-20T04:44:28.827118+0000","last_peered":"2023-12-20T04:44:28.827118+0000","last_clean":"2023-12-20T04:44:28.827118+0000","last_became_active":"2023-12-20T04:44:27.443107+0000","last_became_peered":"2023-12-20T04:44:27.443107+0000","last_unstale":"2023-12-20T04:44:28.827118+0000","last_undegraded":"2023-12-20T04:44:28.827118+0000","last_fullsized":"2023-12-20T04:44:28.827118+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:49:21.213971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910179+0000","last_change":"2023-12-20T04:44:27.842597+0000","last_active":"2023-12-20T04:44:28.910179+0000","last_peered":"2023-12-20T04:44:28.910179+0000","last_clean":"2023-12-20T04:44:28.910179+0000","last_became_active":"2023-12-20T04:44:27.842332+0000","last_became_peered":"2023-12-20T04:44:27.842332+0000","last_unstale":"2023-12-20T04:44:28.910179+0000","last_undegraded":"2023-12-20T04:44:28.910179+0000","last_fullsized":"2023-12-20T04:44:28.910179+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:17:01.428136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909448+0000","last_change":"2023-12-20T04:44:27.437751+0000","last_active":"2023-12-20T04:44:28.909448+0000","last_peered":"2023-12-20T04:44:28.909448+0000","last_clean":"2023-12-20T04:44:28.909448+0000","last_became_active":"2023-12-20T04:44:27.437451+0000","last_became_peered":"2023-12-20T04:44:27.437451+0000","last_unstale":"2023-12-20T04:44:28.909448+0000","last_undegraded":"2023-12-20T04:44:28.909448+0000","last_fullsized":"2023-12-20T04:44:28.909448+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:48.627999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"96'3058","reported_seq":3359,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-20T04:44:27.839752+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:44:27.839752+0000","last_peered":"2023-12-20T04:44:27.839752+0000","last_clean":"2023-12-20T04:44:27.839752+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:44:27.839752+0000","last_undegraded":"2023-12-20T04:44:27.839752+0000","last_fullsized":"2023-12-20T04:44:27.839752+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3058,"log_dups_size":0,"ondisk_log_size":3058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":637,"num_read_kb":1171,"num_write":261,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":637,"num_read_kb":1171,"num_write":261,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3058,"ondisk_log_size":3058,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575040,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1900544,"data_stored":1204499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93899999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84499999999999997,"5min":1,"15min":1.115},"last":0.73799999999999999}]},{"osd":1,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":1.095,"15min":1.2190000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91700000000000004,"5min":1.032,"15min":2.649},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.68999999999999995,"15min":0.71099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89300000000000002,"5min":1.075,"15min":1.5960000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.74299999999999999,"15min":0.749},"min":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.45300000000000001},"max":{"1min":0.98999999999999999,"5min":1.1000000000000001,"15min":1.135},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.70099999999999996,"15min":0.72599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.641},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93500000000000005,"5min":1.131,"15min":2.5990000000000002},"last":0.88400000000000001}]},{"osd":4,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34599999999999997},"max":{"1min":0.78500000000000003,"5min":0.90700000000000003,"15min":1.0109999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.624,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79500000000000004,"5min":1.0469999999999999,"15min":1.242},"last":0.65800000000000003}]},{"osd":5,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.438,"5min":0.433,"15min":0.40400000000000003},"max":{"1min":0.86299999999999999,"5min":1.089,"15min":1.0900000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.432},"max":{"1min":0.83199999999999996,"5min":0.98799999999999999,"15min":1.071},"last":0.86499999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.443},"max":{"1min":0.99199999999999999,"5min":1.093,"15min":1.1279999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44400000000000001},"max":{"1min":0.96799999999999997,"5min":1.125,"15min":1.125},"last":0.82399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575040,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36428,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335412,"statfs":{"total":96636764160,"available":96599461888,"internally_reserved":0,"allocated":1314816,"data_stored":614755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.41399999999999998,"5min":0.33800000000000002,"15min":0.27100000000000002},"max":{"1min":1.1140000000000001,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.69699999999999995,"15min":0.71699999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.034,"5min":1.04,"15min":1.1910000000000001},"last":0.628}]},{"osd":1,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.66200000000000003,"15min":0.69899999999999995},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0129999999999999,"5min":1.016,"15min":1.016},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70999999999999996,"15min":0.72699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.36599999999999999},"max":{"1min":0.98899999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.36199999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.747,"15min":0.76100000000000001},"min":{"1min":0.49199999999999999,"5min":0.47299999999999998,"15min":0.40899999999999997},"max":{"1min":1.0580000000000001,"5min":1.216,"15min":1.216},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.38300000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.55900000000000005}]},{"osd":3,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.109},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.69799999999999995,"15min":0.73199999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97599999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.44}]},{"osd":4,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.47099999999999997,"5min":0.39900000000000002,"15min":0.36399999999999999},"max":{"1min":0.94099999999999995,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.63100000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.36899999999999999},"max":{"1min":0.93700000000000006,"5min":0.995,"15min":1.071},"last":0.51100000000000001}]},{"osd":5,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.76700000000000002,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.435,"15min":0.434},"max":{"1min":0.98899999999999999,"5min":1.306,"15min":1.306},"last":0.53900000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95499999999999996,"5min":1.0129999999999999,"15min":1.2270000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70499999999999996,"15min":0.70599999999999996},"min":{"1min":0.497,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92500000000000004,"5min":1.002,"15min":1.169},"last":0.57799999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575041,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36432,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335408,"statfs":{"total":96636764160,"available":96599457792,"internally_reserved":0,"allocated":1318912,"data_stored":615647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68700000000000006,"15min":0.69299999999999995},"min":{"1min":0.36199999999999999,"5min":0.33700000000000002,"15min":0.29999999999999999},"max":{"1min":0.83099999999999996,"5min":0.97199999999999998,"15min":1.05},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68999999999999995,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.34300000000000003},"max":{"1min":0.877,"5min":0.95099999999999996,"15min":0.98599999999999999},"last":0.64700000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.441},"max":{"1min":0.88600000000000001,"5min":1.044,"15min":1.0600000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70999999999999996,"15min":0.73299999999999998},"min":{"1min":0.437,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95199999999999996,"5min":0.96999999999999997,"15min":1.1140000000000001},"last":0.75600000000000001}]},{"osd":2,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.72699999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.94399999999999995,"15min":1.153},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71399999999999997,"15min":0.72299999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":1.004},"last":0.72399999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92500000000000004,"5min":0.95999999999999996,"15min":1.758},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73299999999999998,"15min":0.748},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90900000000000003,"5min":0.96599999999999997,"15min":1.0469999999999999},"last":0.78000000000000003}]},{"osd":4,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.40600000000000003,"15min":0.35099999999999998},"max":{"1min":0.84799999999999998,"5min":0.97299999999999998,"15min":1.0229999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.39600000000000002},"max":{"1min":0.94299999999999995,"5min":0.98699999999999999,"15min":1.077},"last":0.95099999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68400000000000005,"15min":0.68000000000000005},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.39900000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":1.0700000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.87,"5min":0.999,"15min":0.999},"last":0.81799999999999995}]},{"osd":7,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.42999999999999999},"max":{"1min":0.90000000000000002,"5min":1.0429999999999999,"15min":1.141},"last":1.006},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.751},"min":{"1min":0.57599999999999996,"5min":0.54900000000000004,"15min":0.47899999999999998},"max":{"1min":0.92000000000000004,"5min":1.012,"15min":1.119},"last":0.90700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58560,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313280,"statfs":{"total":96636764160,"available":96576798720,"internally_reserved":0,"allocated":1302528,"data_stored":614061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":0.69799999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90900000000000003,"5min":1.024,"15min":1.5509999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68400000000000005,"15min":0.70499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92400000000000004,"5min":1.625,"15min":1.633},"last":0.97699999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89400000000000002,"5min":0.98799999999999999,"15min":1.798},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96399999999999997,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.93999999999999995}]},{"osd":2,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.98099999999999998,"5min":1.52,"15min":1.728},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.47999999999999998,"5min":0.46899999999999997,"15min":0.38700000000000001},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.6619999999999999},"last":0.90700000000000003}]},{"osd":3,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":1.554,"15min":1.5760000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.94599999999999995,"5min":1.03,"15min":1.7649999999999999},"last":0.95099999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63800000000000001,"15min":0.624},"min":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":1.1120000000000001,"5min":1.456,"15min":1.456},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.077,"5min":1.589,"15min":1.589},"last":0.60499999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.47199999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.6739999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.622}]},{"osd":7,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.47099999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.036,"5min":1.655,"15min":1.655},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.72099999999999997,"15min":0.73299999999999998},"min":{"1min":0.50700000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0029999999999999,"5min":1.052,"15min":1.125},"last":0.80300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575041,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36440,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335400,"statfs":{"total":96636764160,"available":96599449600,"internally_reserved":0,"allocated":1327104,"data_stored":621925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.26900000000000002,"5min":0.26200000000000001,"15min":0.23200000000000001},"max":{"1min":0.748,"5min":0.875,"15min":4.3700000000000001},"last":0.435},{"interface":"front","average":{"1min":0.44,"5min":0.499,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63100000000000001,"5min":0.80800000000000005,"15min":3.355},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.48499999999999999},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68200000000000005,"5min":0.73999999999999999,"15min":3.7160000000000002},"last":0.307},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.53300000000000003,"15min":0.55700000000000005},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.28399999999999997},"max":{"1min":0.70199999999999996,"5min":0.74399999999999999,"15min":3.3109999999999999},"last":0.42599999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.374,"5min":0.318,"15min":0.318},"max":{"1min":0.72799999999999998,"5min":0.88700000000000001,"15min":4.0110000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.316},"max":{"1min":0.81799999999999995,"5min":0.93300000000000005,"15min":6.1399999999999997},"last":0.54200000000000004}]},{"osd":4,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.76800000000000002,"15min":0.78500000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99199999999999999,"5min":1.131,"15min":1.3089999999999999},"last":0.628},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78300000000000003,"15min":0.81799999999999995},"min":{"1min":0.52300000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93799999999999994,"5min":1.4059999999999999,"15min":3.9510000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80800000000000005,"15min":0.82899999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":1.0649999999999999,"5min":1.3300000000000001,"15min":3.9830000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.76300000000000001,"15min":0.80200000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.38400000000000001},"max":{"1min":0.88100000000000001,"5min":1.252,"15min":4.6550000000000002},"last":0.79300000000000004}]},{"osd":6,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76000000000000001,"15min":0.79800000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.45600000000000002},"max":{"1min":0.95199999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.82999999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.499},"max":{"1min":1.0149999999999999,"5min":1.2270000000000001,"15min":3.8900000000000001},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.76900000000000002,"15min":0.81399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":1.1319999999999999,"15min":4.6970000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.79100000000000004,"15min":0.81999999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":1.29,"15min":1.29},"last":0.57199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575040,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40516,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331324,"statfs":{"total":96636764160,"available":96595275776,"internally_reserved":0,"allocated":1306624,"data_stored":614643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.41499999999999998,"15min":0.437},"min":{"1min":0.27300000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.59199999999999997,"5min":0.78900000000000003,"15min":0.89600000000000002},"last":0.371},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.0589999999999999},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47499999999999998,"15min":0.47199999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67600000000000005,"5min":0.876,"15min":0.876},"last":0.313},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.32100000000000001,"5min":0.32000000000000001,"15min":0.28100000000000003},"max":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":1.01},"last":0.43099999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.39500000000000002,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.79800000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.624,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85199999999999998,"5min":0.91600000000000004,"15min":1.0900000000000001},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.78200000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.45200000000000001},"max":{"1min":1.016,"5min":1.145,"15min":1.1799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.753,"15min":0.77500000000000002},"min":{"1min":0.50700000000000001,"5min":0.44500000000000001,"15min":0.41399999999999998},"max":{"1min":0.91800000000000004,"5min":1.048,"15min":1.2010000000000001},"last":0.56599999999999995}]},{"osd":5,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.034,"5min":1.1220000000000001,"15min":1.327},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.53900000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98399999999999999,"5min":1.04,"15min":1.381},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":1.151,"15min":1.2809999999999999},"last":0.627},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76500000000000001,"15min":0.79400000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.46100000000000002},"max":{"1min":0.96899999999999997,"5min":1.0609999999999999,"15min":1.153},"last":0.61799999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.79600000000000004,"15min":0.81999999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98899999999999999,"5min":1.107,"15min":1.3520000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.81000000000000005,"15min":0.84699999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.504},"max":{"1min":0.97299999999999998,"5min":1.079,"15min":1.401},"last":0.59799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50380,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321460,"statfs":{"total":96636764160,"available":96585175040,"internally_reserved":0,"allocated":1314816,"data_stored":614255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.43099999999999999,"15min":0.44400000000000001},"min":{"1min":0.27300000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57299999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.49099999999999999},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.28599999999999998},"max":{"1min":0.75,"5min":0.76600000000000001,"15min":0.92100000000000004},"last":0.35599999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.5,"15min":0.51500000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63200000000000001,"5min":0.81399999999999995,"15min":1.091},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.51400000000000001,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.33900000000000002},"max":{"1min":0.84699999999999998,"5min":0.85899999999999999,"15min":0.91300000000000003},"last":0.40000000000000002}]},{"osd":3,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.36599999999999999},"max":{"1min":0.91000000000000003,"5min":0.93600000000000005,"15min":1.016},"last":0.442},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52800000000000002},"min":{"1min":0.38500000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70699999999999996,"5min":0.82399999999999995,"15min":1.2410000000000001},"last":0.41699999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.72099999999999997,"15min":0.76400000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92900000000000005,"5min":1.1579999999999999,"15min":1.4419999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75,"15min":0.78300000000000003},"min":{"1min":0.51600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.3380000000000001},"last":0.505}]},{"osd":5,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.754,"15min":0.77300000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94899999999999995,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73699999999999999,"15min":0.78400000000000003},"min":{"1min":0.48399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86699999999999999,"5min":1.2450000000000001,"15min":1.3009999999999999},"last":0.52500000000000002}]},{"osd":6,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.76300000000000001,"15min":0.79200000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.232,"5min":1.232,"15min":1.415},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.752,"15min":0.78600000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.89200000000000002,"5min":1.252,"15min":1.268},"last":0.45800000000000002}]},{"osd":7,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.79900000000000004,"15min":0.83299999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.028,"5min":1.306,"15min":1.399},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.79500000000000004,"15min":0.83299999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97899999999999998,"5min":1.0980000000000001,"15min":1.373},"last":0.58699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575041,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28820,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343020,"statfs":{"total":96636764160,"available":96607252480,"internally_reserved":0,"allocated":1912832,"data_stored":1210763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.46700000000000003,"15min":0.44600000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":0.83699999999999997,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.59599999999999997,"15min":0.53600000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.26900000000000002},"max":{"1min":0.70099999999999996,"5min":7.8860000000000001,"15min":7.8860000000000001},"last":0.376}]},{"osd":2,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.28399999999999997},"max":{"1min":0.75600000000000001,"5min":2.528,"15min":2.528},"last":0.441},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71399999999999997,"5min":0.83499999999999996,"15min":1.2989999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.52400000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29499999999999998},"max":{"1min":0.80000000000000004,"5min":3.1000000000000001,"15min":3.1000000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.68799999999999994,"15min":0.61899999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70899999999999996,"5min":10.49,"15min":10.49},"last":0.48699999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.78600000000000003,"15min":0.79300000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0780000000000001,"5min":3.0659999999999998,"15min":3.7469999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.83199999999999996,"15min":0.80300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.181,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.78900000000000003}]},{"osd":5,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.79200000000000004,"15min":0.79800000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.44700000000000001},"max":{"1min":1.161,"5min":1.3109999999999999,"15min":1.877},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0349999999999999,"5min":1.2010000000000001,"15min":1.7450000000000001},"last":0.81999999999999995}]},{"osd":6,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.84499999999999997,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.47299999999999998},"max":{"1min":1.0069999999999999,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.754},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.52800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2150000000000001,"5min":1.306,"15min":1.492},"last":0.84399999999999997}]},{"osd":7,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0740000000000001,"5min":1.2549999999999999,"15min":1.6539999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.81200000000000006,"15min":0.81299999999999994},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"max":{"1min":1.01,"5min":1.276,"15min":2.113},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:44:31.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:44:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.113+0000 7f10e7ef9640 1 Processor -- start 2023-12-20T04:44:32.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.113+0000 7f10e7ef9640 1 -- start start 2023-12-20T04:44:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e7ef9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e00ae010 0x7f10e0071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e7ef9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10e006cf10 0x7f10e00719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e7ef9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10e00ae3f0 con 0x7f10e0164ef0 2023-12-20T04:44:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e7ef9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10e00ca4b0 con 0x7f10e006cf10 2023-12-20T04:44:32.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e7ef9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e0161800 con 0x7f10e00ae010 2023-12-20T04:44:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e6496640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f10e0164ef0 0x7f10e00701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49098/0 (socket says 172.21.15.32:49098) 2023-12-20T04:44:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e6496640 1 -- 172.21.15.32:0/4059916088 learned_addr learned my addr 172.21.15.32:0/4059916088 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e5494640 1 --2- 172.21.15.32:0/4059916088 >> v2:172.21.15.32:3300/0 conn(0x7f10e006cf10 0x7f10e00719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e5c95640 1 --2- 172.21.15.32:0/4059916088 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e00ae010 0x7f10e0071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e5494640 1 -- 172.21.15.32:0/4059916088 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e00ae010 msgr2=0x7f10e0071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e5494640 1 --2- 172.21.15.32:0/4059916088 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e00ae010 0x7f10e0071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e5494640 1 -- 172.21.15.32:0/4059916088 >> v1:172.21.15.32:6789/0 conn(0x7f10e0164ef0 legacy=0x7f10e00701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e5494640 1 -- 172.21.15.32:0/4059916088 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10e0165380 con 0x7f10e006cf10 2023-12-20T04:44:32.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.117+0000 7f10e5c95640 1 --2- 172.21.15.32:0/4059916088 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e00ae010 0x7f10e0071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:32.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e5494640 1 --2- 172.21.15.32:0/4059916088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e006cf10 0x7f10e00719b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f10d00097c0 tx=0x7f10d001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bb9efb2e969a9e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:32.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e4c93640 1 -- 172.21.15.32:0/4059916088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d00ac620 con 0x7f10e006cf10 2023-12-20T04:44:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e4c93640 1 -- 172.21.15.32:0/4059916088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d00ac780 con 0x7f10e006cf10 2023-12-20T04:44:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e4c93640 1 -- 172.21.15.32:0/4059916088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d00acb10 con 0x7f10e006cf10 2023-12-20T04:44:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e7ef9640 1 -- 172.21.15.32:0/4059916088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e006cf10 msgr2=0x7f10e00719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e7ef9640 1 --2- 172.21.15.32:0/4059916088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e006cf10 0x7f10e00719b0 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7f10d00097c0 tx=0x7f10d001c3c0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e7ef9640 1 -- 172.21.15.32:0/4059916088 shutdown_connections 2023-12-20T04:44:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e7ef9640 1 --2- 172.21.15.32:0/4059916088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e006cf10 0x7f10e00719b0 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e7ef9640 1 --2- 172.21.15.32:0/4059916088 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e00ae010 0x7f10e0071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e7ef9640 1 -- 172.21.15.32:0/4059916088 >> 172.21.15.32:0/4059916088 conn(0x7f10e006a320 msgr2=0x7f10e00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e7ef9640 1 -- 172.21.15.32:0/4059916088 shutdown_connections 2023-12-20T04:44:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e7ef9640 1 -- 172.21.15.32:0/4059916088 wait complete. 2023-12-20T04:44:32.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e7ef9640 1 Processor -- start 2023-12-20T04:44:32.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.121+0000 7f10e7ef9640 1 -- start start 2023-12-20T04:44:32.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e7ef9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e00ae010 0x7f10e0153030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e7ef9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e0151fb0 0x7f10e0154a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e7ef9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e0154f70 0x7f10e0162060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e7ef9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e0155420 con 0x7f10e00ae010 2023-12-20T04:44:32.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e7ef9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10e0154350 con 0x7f10e0151fb0 2023-12-20T04:44:32.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e7ef9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10e0152430 con 0x7f10e0154f70 2023-12-20T04:44:32.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e5c95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e00ae010 0x7f10e0153030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e5c95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e00ae010 0x7f10e0153030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60548/0 (socket says 172.21.15.32:60548) 2023-12-20T04:44:32.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e5c95640 1 -- 172.21.15.32:0/1483428041 learned_addr learned my addr 172.21.15.32:0/1483428041 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:32.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e6496640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e0154f70 0x7f10e0162060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e5494640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e0151fb0 0x7f10e0154a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e5c95640 1 -- 172.21.15.32:0/1483428041 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e0154f70 msgr2=0x7f10e0162060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e5c95640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e0154f70 0x7f10e0162060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e5c95640 1 -- 172.21.15.32:0/1483428041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e0151fb0 msgr2=0x7f10e0154a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e5c95640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e0151fb0 0x7f10e0154a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e5c95640 1 -- 172.21.15.32:0/1483428041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d0009420 con 0x7f10e00ae010 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e6496640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e0154f70 0x7f10e0162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e5494640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e0151fb0 0x7f10e0154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e5c95640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e00ae010 0x7f10e0153030 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00e7e0 tx=0x7f10dc00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10d67fc640 1 -- 172.21.15.32:0/1483428041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc0059a0 con 0x7f10e00ae010 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10d67fc640 1 -- 172.21.15.32:0/1483428041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10dc005b00 con 0x7f10e00ae010 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10d67fc640 1 -- 172.21.15.32:0/1483428041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10dc005e90 con 0x7f10e00ae010 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e7ef9640 1 -- 172.21.15.32:0/1483428041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10e0153740 con 0x7f10e00ae010 2023-12-20T04:44:32.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e7ef9640 1 -- 172.21.15.32:0/1483428041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10e0153d00 con 0x7f10e00ae010 2023-12-20T04:44:32.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10e7ef9640 1 -- 172.21.15.32:0/1483428041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10e004e000 con 0x7f10e00ae010 2023-12-20T04:44:32.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.125+0000 7f10d67fc640 1 -- 172.21.15.32:0/1483428041 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f10dc004030 con 0x7f10e00ae010 2023-12-20T04:44:32.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.129+0000 7f10d67fc640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f10c4065bb0 0x7f10c4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.129+0000 7f10d67fc640 1 -- 172.21.15.32:0/1483428041 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f10dc0834e0 con 0x7f10e00ae010 2023-12-20T04:44:32.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.129+0000 7f10e5494640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f10c4065bb0 0x7f10c4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.129+0000 7f10e5494640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f10c4065bb0 0x7f10c4068070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10d00ae000 tx=0x7f10d0008040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:32.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.133+0000 7f10d67fc640 1 -- 172.21.15.32:0/1483428041 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f10dc050c70 con 0x7f10e00ae010 2023-12-20T04:44:32.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.285+0000 7f10e7ef9640 1 -- 172.21.15.32:0/1483428041 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f10e0165c20 con 0x7f10c4065bb0 2023-12-20T04:44:32.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.293+0000 7f10d67fc640 1 -- 172.21.15.32:0/1483428041 <== mgr.7113 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289174 (secure 0 0 0) 0x7f10d00bc090 con 0x7f10c4065bb0 2023-12-20T04:44:32.296 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:32.298 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:44:32.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 -- 172.21.15.32:0/1483428041 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f10c4065bb0 msgr2=0x7f10c4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f10c4065bb0 0x7f10c4068070 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f10d00ae000 tx=0x7f10d0008040 comp rx=0 tx=0).stop 2023-12-20T04:44:32.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 -- 172.21.15.32:0/1483428041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e00ae010 msgr2=0x7f10e0153030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e00ae010 0x7f10e0153030 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7f10dc00e7e0 tx=0x7f10dc00eca0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 -- 172.21.15.32:0/1483428041 shutdown_connections 2023-12-20T04:44:32.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f10c4065bb0 0x7f10c4068070 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10e0154f70 0x7f10e0162060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10e0151fb0 0x7f10e0154a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 --2- 172.21.15.32:0/1483428041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10e00ae010 0x7f10e0153030 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 -- 172.21.15.32:0/1483428041 >> 172.21.15.32:0/1483428041 conn(0x7f10e006a320 msgr2=0x7f10e00ccf30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:32.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 -- 172.21.15.32:0/1483428041 shutdown_connections 2023-12-20T04:44:32.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.297+0000 7f10e7ef9640 1 -- 172.21.15.32:0/1483428041 wait complete. 2023-12-20T04:44:32.316 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-20T04:44:31.099803+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":637,"num_read_kb":1171,"num_write":305,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3096,"ondisk_log_size":3096,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":169,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":320480,"kb_used_data":11424,"kb_used_omap":0,"kb_used_meta":308992,"kb_avail":754654240,"statfs":{"total":773094113280,"available":772765941760,"internally_reserved":0,"allocated":11698176,"data_stored":6110548,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316407808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.694039"},"pg_stats":[{"pgid":"13.3f","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909813+0000","last_change":"2023-12-20T04:44:27.855927+0000","last_active":"2023-12-20T04:44:28.909813+0000","last_peered":"2023-12-20T04:44:28.909813+0000","last_clean":"2023-12-20T04:44:28.909813+0000","last_became_active":"2023-12-20T04:44:27.855360+0000","last_became_peered":"2023-12-20T04:44:27.855360+0000","last_unstale":"2023-12-20T04:44:28.909813+0000","last_undegraded":"2023-12-20T04:44:28.909813+0000","last_fullsized":"2023-12-20T04:44:28.909813+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:01:46.648912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.3e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910343+0000","last_change":"2023-12-20T04:44:27.857546+0000","last_active":"2023-12-20T04:44:28.910343+0000","last_peered":"2023-12-20T04:44:28.910343+0000","last_clean":"2023-12-20T04:44:28.910343+0000","last_became_active":"2023-12-20T04:44:27.856534+0000","last_became_peered":"2023-12-20T04:44:27.856534+0000","last_unstale":"2023-12-20T04:44:28.910343+0000","last_undegraded":"2023-12-20T04:44:28.910343+0000","last_fullsized":"2023-12-20T04:44:28.910343+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:27:38.467452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.3d","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909281+0000","last_change":"2023-12-20T04:44:27.863618+0000","last_active":"2023-12-20T04:44:28.909281+0000","last_peered":"2023-12-20T04:44:28.909281+0000","last_clean":"2023-12-20T04:44:28.909281+0000","last_became_active":"2023-12-20T04:44:27.863269+0000","last_became_peered":"2023-12-20T04:44:27.863269+0000","last_unstale":"2023-12-20T04:44:28.909281+0000","last_undegraded":"2023-12-20T04:44:28.909281+0000","last_fullsized":"2023-12-20T04:44:28.909281+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:46:48.669519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.3c","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909729+0000","last_change":"2023-12-20T04:44:27.836048+0000","last_active":"2023-12-20T04:44:28.909729+0000","last_peered":"2023-12-20T04:44:28.909729+0000","last_clean":"2023-12-20T04:44:28.909729+0000","last_became_active":"2023-12-20T04:44:27.835175+0000","last_became_peered":"2023-12-20T04:44:27.835175+0000","last_unstale":"2023-12-20T04:44:28.909729+0000","last_undegraded":"2023-12-20T04:44:28.909729+0000","last_fullsized":"2023-12-20T04:44:28.909729+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:37:12.392685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.3b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827799+0000","last_change":"2023-12-20T04:44:27.843846+0000","last_active":"2023-12-20T04:44:28.827799+0000","last_peered":"2023-12-20T04:44:28.827799+0000","last_clean":"2023-12-20T04:44:28.827799+0000","last_became_active":"2023-12-20T04:44:27.843567+0000","last_became_peered":"2023-12-20T04:44:27.843567+0000","last_unstale":"2023-12-20T04:44:28.827799+0000","last_undegraded":"2023-12-20T04:44:28.827799+0000","last_fullsized":"2023-12-20T04:44:28.827799+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:24.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.3a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827418+0000","last_change":"2023-12-20T04:44:27.850953+0000","last_active":"2023-12-20T04:44:28.827418+0000","last_peered":"2023-12-20T04:44:28.827418+0000","last_clean":"2023-12-20T04:44:28.827418+0000","last_became_active":"2023-12-20T04:44:27.850598+0000","last_became_peered":"2023-12-20T04:44:27.850598+0000","last_unstale":"2023-12-20T04:44:28.827418+0000","last_undegraded":"2023-12-20T04:44:28.827418+0000","last_fullsized":"2023-12-20T04:44:28.827418+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:01:46.648912+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.39","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908825+0000","last_change":"2023-12-20T04:44:27.854501+0000","last_active":"2023-12-20T04:44:28.908825+0000","last_peered":"2023-12-20T04:44:28.908825+0000","last_clean":"2023-12-20T04:44:28.908825+0000","last_became_active":"2023-12-20T04:44:27.854208+0000","last_became_peered":"2023-12-20T04:44:27.854208+0000","last_unstale":"2023-12-20T04:44:28.908825+0000","last_undegraded":"2023-12-20T04:44:28.908825+0000","last_fullsized":"2023-12-20T04:44:28.908825+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:08:11.085461+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.38","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827098+0000","last_change":"2023-12-20T04:44:27.848635+0000","last_active":"2023-12-20T04:44:28.827098+0000","last_peered":"2023-12-20T04:44:28.827098+0000","last_clean":"2023-12-20T04:44:28.827098+0000","last_became_active":"2023-12-20T04:44:27.847032+0000","last_became_peered":"2023-12-20T04:44:27.847032+0000","last_unstale":"2023-12-20T04:44:28.827098+0000","last_undegraded":"2023-12-20T04:44:28.827098+0000","last_fullsized":"2023-12-20T04:44:28.827098+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:49.630920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.37","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827694+0000","last_change":"2023-12-20T04:44:27.851334+0000","last_active":"2023-12-20T04:44:28.827694+0000","last_peered":"2023-12-20T04:44:28.827694+0000","last_clean":"2023-12-20T04:44:28.827694+0000","last_became_active":"2023-12-20T04:44:27.849291+0000","last_became_peered":"2023-12-20T04:44:27.849291+0000","last_unstale":"2023-12-20T04:44:28.827694+0000","last_undegraded":"2023-12-20T04:44:28.827694+0000","last_fullsized":"2023-12-20T04:44:28.827694+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:45:05.584416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.36","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909835+0000","last_change":"2023-12-20T04:44:27.857058+0000","last_active":"2023-12-20T04:44:28.909835+0000","last_peered":"2023-12-20T04:44:28.909835+0000","last_clean":"2023-12-20T04:44:28.909835+0000","last_became_active":"2023-12-20T04:44:27.856717+0000","last_became_peered":"2023-12-20T04:44:27.856717+0000","last_unstale":"2023-12-20T04:44:28.909835+0000","last_undegraded":"2023-12-20T04:44:28.909835+0000","last_fullsized":"2023-12-20T04:44:28.909835+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:54:36.565658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.35","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910350+0000","last_change":"2023-12-20T04:44:27.851856+0000","last_active":"2023-12-20T04:44:28.910350+0000","last_peered":"2023-12-20T04:44:28.910350+0000","last_clean":"2023-12-20T04:44:28.910350+0000","last_became_active":"2023-12-20T04:44:27.851595+0000","last_became_peered":"2023-12-20T04:44:27.851595+0000","last_unstale":"2023-12-20T04:44:28.910350+0000","last_undegraded":"2023-12-20T04:44:28.910350+0000","last_fullsized":"2023-12-20T04:44:28.910350+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:34:37.516180+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.34","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909678+0000","last_change":"2023-12-20T04:44:27.855167+0000","last_active":"2023-12-20T04:44:28.909678+0000","last_peered":"2023-12-20T04:44:28.909678+0000","last_clean":"2023-12-20T04:44:28.909678+0000","last_became_active":"2023-12-20T04:44:27.854857+0000","last_became_peered":"2023-12-20T04:44:27.854857+0000","last_unstale":"2023-12-20T04:44:28.909678+0000","last_undegraded":"2023-12-20T04:44:28.909678+0000","last_fullsized":"2023-12-20T04:44:28.909678+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:47:21.553515+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.33","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908998+0000","last_change":"2023-12-20T04:44:27.857790+0000","last_active":"2023-12-20T04:44:28.908998+0000","last_peered":"2023-12-20T04:44:28.908998+0000","last_clean":"2023-12-20T04:44:28.908998+0000","last_became_active":"2023-12-20T04:44:27.857483+0000","last_became_peered":"2023-12-20T04:44:27.857483+0000","last_unstale":"2023-12-20T04:44:28.908998+0000","last_undegraded":"2023-12-20T04:44:28.908998+0000","last_fullsized":"2023-12-20T04:44:28.908998+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:24.836114+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.32","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910144+0000","last_change":"2023-12-20T04:44:27.840899+0000","last_active":"2023-12-20T04:44:28.910144+0000","last_peered":"2023-12-20T04:44:28.910144+0000","last_clean":"2023-12-20T04:44:28.910144+0000","last_became_active":"2023-12-20T04:44:27.840449+0000","last_became_peered":"2023-12-20T04:44:27.840449+0000","last_unstale":"2023-12-20T04:44:28.910144+0000","last_undegraded":"2023-12-20T04:44:28.910144+0000","last_fullsized":"2023-12-20T04:44:28.910144+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:36.816675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.31","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908756+0000","last_change":"2023-12-20T04:44:27.858294+0000","last_active":"2023-12-20T04:44:28.908756+0000","last_peered":"2023-12-20T04:44:28.908756+0000","last_clean":"2023-12-20T04:44:28.908756+0000","last_became_active":"2023-12-20T04:44:27.856272+0000","last_became_peered":"2023-12-20T04:44:27.856272+0000","last_unstale":"2023-12-20T04:44:28.908756+0000","last_undegraded":"2023-12-20T04:44:28.908756+0000","last_fullsized":"2023-12-20T04:44:28.908756+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:00:11.814269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.30","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827014+0000","last_change":"2023-12-20T04:44:27.851256+0000","last_active":"2023-12-20T04:44:28.827014+0000","last_peered":"2023-12-20T04:44:28.827014+0000","last_clean":"2023-12-20T04:44:28.827014+0000","last_became_active":"2023-12-20T04:44:27.850909+0000","last_became_peered":"2023-12-20T04:44:27.850909+0000","last_unstale":"2023-12-20T04:44:28.827014+0000","last_undegraded":"2023-12-20T04:44:28.827014+0000","last_fullsized":"2023-12-20T04:44:28.827014+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:22.692826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2f","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828375+0000","last_change":"2023-12-20T04:44:27.849744+0000","last_active":"2023-12-20T04:44:28.828375+0000","last_peered":"2023-12-20T04:44:28.828375+0000","last_clean":"2023-12-20T04:44:28.828375+0000","last_became_active":"2023-12-20T04:44:27.848693+0000","last_became_peered":"2023-12-20T04:44:27.848693+0000","last_unstale":"2023-12-20T04:44:28.828375+0000","last_undegraded":"2023-12-20T04:44:28.828375+0000","last_fullsized":"2023-12-20T04:44:28.828375+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:40:00.475342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.2e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909843+0000","last_change":"2023-12-20T04:44:27.857314+0000","last_active":"2023-12-20T04:44:28.909843+0000","last_peered":"2023-12-20T04:44:28.909843+0000","last_clean":"2023-12-20T04:44:28.909843+0000","last_became_active":"2023-12-20T04:44:27.857000+0000","last_became_peered":"2023-12-20T04:44:27.857000+0000","last_unstale":"2023-12-20T04:44:28.909843+0000","last_undegraded":"2023-12-20T04:44:28.909843+0000","last_fullsized":"2023-12-20T04:44:28.909843+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:16:05.460113+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2d","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826283+0000","last_change":"2023-12-20T04:44:27.845451+0000","last_active":"2023-12-20T04:44:28.826283+0000","last_peered":"2023-12-20T04:44:28.826283+0000","last_clean":"2023-12-20T04:44:28.826283+0000","last_became_active":"2023-12-20T04:44:27.845184+0000","last_became_peered":"2023-12-20T04:44:27.845184+0000","last_unstale":"2023-12-20T04:44:28.826283+0000","last_undegraded":"2023-12-20T04:44:28.826283+0000","last_fullsized":"2023-12-20T04:44:28.826283+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:42:32.892908+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.2c","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909801+0000","last_change":"2023-12-20T04:44:27.845334+0000","last_active":"2023-12-20T04:44:28.909801+0000","last_peered":"2023-12-20T04:44:28.909801+0000","last_clean":"2023-12-20T04:44:28.909801+0000","last_became_active":"2023-12-20T04:44:27.843848+0000","last_became_peered":"2023-12-20T04:44:27.843848+0000","last_unstale":"2023-12-20T04:44:28.909801+0000","last_undegraded":"2023-12-20T04:44:28.909801+0000","last_fullsized":"2023-12-20T04:44:28.909801+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:58.386091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909668+0000","last_change":"2023-12-20T04:44:27.848203+0000","last_active":"2023-12-20T04:44:28.909668+0000","last_peered":"2023-12-20T04:44:28.909668+0000","last_clean":"2023-12-20T04:44:28.909668+0000","last_became_active":"2023-12-20T04:44:27.847931+0000","last_became_peered":"2023-12-20T04:44:27.847931+0000","last_unstale":"2023-12-20T04:44:28.909668+0000","last_undegraded":"2023-12-20T04:44:28.909668+0000","last_fullsized":"2023-12-20T04:44:28.909668+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:29.647550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.2a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909187+0000","last_change":"2023-12-20T04:44:27.853302+0000","last_active":"2023-12-20T04:44:28.909187+0000","last_peered":"2023-12-20T04:44:28.909187+0000","last_clean":"2023-12-20T04:44:28.909187+0000","last_became_active":"2023-12-20T04:44:27.853041+0000","last_became_peered":"2023-12-20T04:44:27.853041+0000","last_unstale":"2023-12-20T04:44:28.909187+0000","last_undegraded":"2023-12-20T04:44:28.909187+0000","last_fullsized":"2023-12-20T04:44:28.909187+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:36.231277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.29","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909260+0000","last_change":"2023-12-20T04:44:27.857085+0000","last_active":"2023-12-20T04:44:28.909260+0000","last_peered":"2023-12-20T04:44:28.909260+0000","last_clean":"2023-12-20T04:44:28.909260+0000","last_became_active":"2023-12-20T04:44:27.856051+0000","last_became_peered":"2023-12-20T04:44:27.856051+0000","last_unstale":"2023-12-20T04:44:28.909260+0000","last_undegraded":"2023-12-20T04:44:28.909260+0000","last_fullsized":"2023-12-20T04:44:28.909260+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:45:05.584416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.28","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908653+0000","last_change":"2023-12-20T04:44:27.850940+0000","last_active":"2023-12-20T04:44:28.908653+0000","last_peered":"2023-12-20T04:44:28.908653+0000","last_clean":"2023-12-20T04:44:28.908653+0000","last_became_active":"2023-12-20T04:44:27.850676+0000","last_became_peered":"2023-12-20T04:44:27.850676+0000","last_unstale":"2023-12-20T04:44:28.908653+0000","last_undegraded":"2023-12-20T04:44:28.908653+0000","last_fullsized":"2023-12-20T04:44:28.908653+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:03:11.664776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.27","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910446+0000","last_change":"2023-12-20T04:44:27.851616+0000","last_active":"2023-12-20T04:44:28.910446+0000","last_peered":"2023-12-20T04:44:28.910446+0000","last_clean":"2023-12-20T04:44:28.910446+0000","last_became_active":"2023-12-20T04:44:27.848794+0000","last_became_peered":"2023-12-20T04:44:27.848794+0000","last_unstale":"2023-12-20T04:44:28.910446+0000","last_undegraded":"2023-12-20T04:44:28.910446+0000","last_fullsized":"2023-12-20T04:44:28.910446+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:30:25.350251+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.26","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909372+0000","last_change":"2023-12-20T04:44:27.858487+0000","last_active":"2023-12-20T04:44:28.909372+0000","last_peered":"2023-12-20T04:44:28.909372+0000","last_clean":"2023-12-20T04:44:28.909372+0000","last_became_active":"2023-12-20T04:44:27.857706+0000","last_became_peered":"2023-12-20T04:44:27.857706+0000","last_unstale":"2023-12-20T04:44:28.909372+0000","last_undegraded":"2023-12-20T04:44:28.909372+0000","last_fullsized":"2023-12-20T04:44:28.909372+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:47:46.040771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"13.25","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828232+0000","last_change":"2023-12-20T04:44:27.846014+0000","last_active":"2023-12-20T04:44:28.828232+0000","last_peered":"2023-12-20T04:44:28.828232+0000","last_clean":"2023-12-20T04:44:28.828232+0000","last_became_active":"2023-12-20T04:44:27.845729+0000","last_became_peered":"2023-12-20T04:44:27.845729+0000","last_unstale":"2023-12-20T04:44:28.828232+0000","last_undegraded":"2023-12-20T04:44:28.828232+0000","last_fullsized":"2023-12-20T04:44:28.828232+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:34.947486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.24","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909704+0000","last_change":"2023-12-20T04:44:27.855352+0000","last_active":"2023-12-20T04:44:28.909704+0000","last_peered":"2023-12-20T04:44:28.909704+0000","last_clean":"2023-12-20T04:44:28.909704+0000","last_became_active":"2023-12-20T04:44:27.855029+0000","last_became_peered":"2023-12-20T04:44:27.855029+0000","last_unstale":"2023-12-20T04:44:28.909704+0000","last_undegraded":"2023-12-20T04:44:28.909704+0000","last_fullsized":"2023-12-20T04:44:28.909704+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:49.630920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.23","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909250+0000","last_change":"2023-12-20T04:44:27.839468+0000","last_active":"2023-12-20T04:44:28.909250+0000","last_peered":"2023-12-20T04:44:28.909250+0000","last_clean":"2023-12-20T04:44:28.909250+0000","last_became_active":"2023-12-20T04:44:27.839194+0000","last_became_peered":"2023-12-20T04:44:27.839194+0000","last_unstale":"2023-12-20T04:44:28.909250+0000","last_undegraded":"2023-12-20T04:44:28.909250+0000","last_fullsized":"2023-12-20T04:44:28.909250+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:14:38.680960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.22","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909208+0000","last_change":"2023-12-20T04:44:27.846673+0000","last_active":"2023-12-20T04:44:28.909208+0000","last_peered":"2023-12-20T04:44:28.909208+0000","last_clean":"2023-12-20T04:44:28.909208+0000","last_became_active":"2023-12-20T04:44:27.845807+0000","last_became_peered":"2023-12-20T04:44:27.845807+0000","last_unstale":"2023-12-20T04:44:28.909208+0000","last_undegraded":"2023-12-20T04:44:28.909208+0000","last_fullsized":"2023-12-20T04:44:28.909208+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:46:04.915815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.21","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827968+0000","last_change":"2023-12-20T04:44:27.848267+0000","last_active":"2023-12-20T04:44:28.827968+0000","last_peered":"2023-12-20T04:44:28.827968+0000","last_clean":"2023-12-20T04:44:28.827968+0000","last_became_active":"2023-12-20T04:44:27.847933+0000","last_became_peered":"2023-12-20T04:44:27.847933+0000","last_unstale":"2023-12-20T04:44:28.827968+0000","last_undegraded":"2023-12-20T04:44:28.827968+0000","last_fullsized":"2023-12-20T04:44:28.827968+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:26.537288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.20","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909264+0000","last_change":"2023-12-20T04:44:27.853330+0000","last_active":"2023-12-20T04:44:28.909264+0000","last_peered":"2023-12-20T04:44:28.909264+0000","last_clean":"2023-12-20T04:44:28.909264+0000","last_became_active":"2023-12-20T04:44:27.852636+0000","last_became_peered":"2023-12-20T04:44:27.852636+0000","last_unstale":"2023-12-20T04:44:28.909264+0000","last_undegraded":"2023-12-20T04:44:28.909264+0000","last_fullsized":"2023-12-20T04:44:28.909264+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:27:38.467452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828389+0000","last_change":"2023-12-20T04:44:27.858425+0000","last_active":"2023-12-20T04:44:28.828389+0000","last_peered":"2023-12-20T04:44:28.828389+0000","last_clean":"2023-12-20T04:44:28.828389+0000","last_became_active":"2023-12-20T04:44:27.857979+0000","last_became_peered":"2023-12-20T04:44:27.857979+0000","last_unstale":"2023-12-20T04:44:28.828389+0000","last_undegraded":"2023-12-20T04:44:28.828389+0000","last_fullsized":"2023-12-20T04:44:28.828389+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:07.634419+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1f","version":"106'4","reported_seq":15,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910062+0000","last_change":"2023-12-20T04:44:27.435403+0000","last_active":"2023-12-20T04:44:28.910062+0000","last_peered":"2023-12-20T04:44:28.910062+0000","last_clean":"2023-12-20T04:44:28.910062+0000","last_became_active":"2023-12-20T04:44:27.435104+0000","last_became_peered":"2023-12-20T04:44:27.435104+0000","last_unstale":"2023-12-20T04:44:28.910062+0000","last_undegraded":"2023-12-20T04:44:28.910062+0000","last_fullsized":"2023-12-20T04:44:28.910062+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:59:16.483399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1f","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909986+0000","last_change":"2023-12-20T04:44:27.856588+0000","last_active":"2023-12-20T04:44:28.909986+0000","last_peered":"2023-12-20T04:44:28.909986+0000","last_clean":"2023-12-20T04:44:28.909986+0000","last_became_active":"2023-12-20T04:44:27.856274+0000","last_became_peered":"2023-12-20T04:44:27.856274+0000","last_unstale":"2023-12-20T04:44:28.909986+0000","last_undegraded":"2023-12-20T04:44:28.909986+0000","last_fullsized":"2023-12-20T04:44:28.909986+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:44.406191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1e","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828507+0000","last_change":"2023-12-20T04:44:27.435541+0000","last_active":"2023-12-20T04:44:28.828507+0000","last_peered":"2023-12-20T04:44:28.828507+0000","last_clean":"2023-12-20T04:44:28.828507+0000","last_became_active":"2023-12-20T04:44:27.435264+0000","last_became_peered":"2023-12-20T04:44:27.435264+0000","last_unstale":"2023-12-20T04:44:28.828507+0000","last_undegraded":"2023-12-20T04:44:28.828507+0000","last_fullsized":"2023-12-20T04:44:28.828507+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:45:04.581495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1c","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909807+0000","last_change":"2023-12-20T04:44:27.847375+0000","last_active":"2023-12-20T04:44:28.909807+0000","last_peered":"2023-12-20T04:44:28.909807+0000","last_clean":"2023-12-20T04:44:28.909807+0000","last_became_active":"2023-12-20T04:44:27.846856+0000","last_became_peered":"2023-12-20T04:44:27.846856+0000","last_unstale":"2023-12-20T04:44:28.909807+0000","last_undegraded":"2023-12-20T04:44:28.909807+0000","last_fullsized":"2023-12-20T04:44:28.909807+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:13:56.386220+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.1d","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826312+0000","last_change":"2023-12-20T04:44:27.428863+0000","last_active":"2023-12-20T04:44:28.826312+0000","last_peered":"2023-12-20T04:44:28.826312+0000","last_clean":"2023-12-20T04:44:28.826312+0000","last_became_active":"2023-12-20T04:44:27.428567+0000","last_became_peered":"2023-12-20T04:44:27.428567+0000","last_unstale":"2023-12-20T04:44:28.826312+0000","last_undegraded":"2023-12-20T04:44:28.826312+0000","last_fullsized":"2023-12-20T04:44:28.826312+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:39:59.472421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1d","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909332+0000","last_change":"2023-12-20T04:44:27.860735+0000","last_active":"2023-12-20T04:44:28.909332+0000","last_peered":"2023-12-20T04:44:28.909332+0000","last_clean":"2023-12-20T04:44:28.909332+0000","last_became_active":"2023-12-20T04:44:27.860431+0000","last_became_peered":"2023-12-20T04:44:27.860431+0000","last_unstale":"2023-12-20T04:44:28.909332+0000","last_undegraded":"2023-12-20T04:44:28.909332+0000","last_fullsized":"2023-12-20T04:44:28.909332+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:51:14.492093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1c","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826116+0000","last_change":"2023-12-20T04:44:27.431692+0000","last_active":"2023-12-20T04:44:28.826116+0000","last_peered":"2023-12-20T04:44:28.826116+0000","last_clean":"2023-12-20T04:44:28.826116+0000","last_became_active":"2023-12-20T04:44:27.429700+0000","last_became_peered":"2023-12-20T04:44:27.429700+0000","last_unstale":"2023-12-20T04:44:28.826116+0000","last_undegraded":"2023-12-20T04:44:28.826116+0000","last_fullsized":"2023-12-20T04:44:28.826116+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:47:45.037850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.1a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909400+0000","last_change":"2023-12-20T04:44:27.846467+0000","last_active":"2023-12-20T04:44:28.909400+0000","last_peered":"2023-12-20T04:44:28.909400+0000","last_clean":"2023-12-20T04:44:28.909400+0000","last_became_active":"2023-12-20T04:44:27.845424+0000","last_became_peered":"2023-12-20T04:44:27.845424+0000","last_unstale":"2023-12-20T04:44:28.909400+0000","last_undegraded":"2023-12-20T04:44:28.909400+0000","last_fullsized":"2023-12-20T04:44:28.909400+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:21:04.322894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.1b","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828180+0000","last_change":"2023-12-20T04:44:27.438920+0000","last_active":"2023-12-20T04:44:28.828180+0000","last_peered":"2023-12-20T04:44:28.828180+0000","last_clean":"2023-12-20T04:44:28.828180+0000","last_became_active":"2023-12-20T04:44:27.438589+0000","last_became_peered":"2023-12-20T04:44:27.438589+0000","last_unstale":"2023-12-20T04:44:28.828180+0000","last_undegraded":"2023-12-20T04:44:28.828180+0000","last_fullsized":"2023-12-20T04:44:28.828180+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:13:55.383299+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.1b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827909+0000","last_change":"2023-12-20T04:44:27.850552+0000","last_active":"2023-12-20T04:44:28.827909+0000","last_peered":"2023-12-20T04:44:28.827909+0000","last_clean":"2023-12-20T04:44:28.827909+0000","last_became_active":"2023-12-20T04:44:27.850234+0000","last_became_peered":"2023-12-20T04:44:27.850234+0000","last_unstale":"2023-12-20T04:44:28.827909+0000","last_undegraded":"2023-12-20T04:44:28.827909+0000","last_fullsized":"2023-12-20T04:44:28.827909+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:47:46.040771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.1a","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827728+0000","last_change":"2023-12-20T04:44:27.437023+0000","last_active":"2023-12-20T04:44:28.827728+0000","last_peered":"2023-12-20T04:44:28.827728+0000","last_clean":"2023-12-20T04:44:28.827728+0000","last_became_active":"2023-12-20T04:44:27.436495+0000","last_became_peered":"2023-12-20T04:44:27.436495+0000","last_unstale":"2023-12-20T04:44:28.827728+0000","last_undegraded":"2023-12-20T04:44:28.827728+0000","last_fullsized":"2023-12-20T04:44:28.827728+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:48.627999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.18","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908948+0000","last_change":"2023-12-20T04:44:27.857393+0000","last_active":"2023-12-20T04:44:28.908948+0000","last_peered":"2023-12-20T04:44:28.908948+0000","last_clean":"2023-12-20T04:44:28.908948+0000","last_became_active":"2023-12-20T04:44:27.857086+0000","last_became_peered":"2023-12-20T04:44:27.857086+0000","last_unstale":"2023-12-20T04:44:28.908948+0000","last_undegraded":"2023-12-20T04:44:28.908948+0000","last_fullsized":"2023-12-20T04:44:28.908948+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:26.537288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.19","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.825765+0000","last_change":"2023-12-20T04:44:27.426439+0000","last_active":"2023-12-20T04:44:28.825765+0000","last_peered":"2023-12-20T04:44:28.825765+0000","last_clean":"2023-12-20T04:44:28.825765+0000","last_became_active":"2023-12-20T04:44:27.426120+0000","last_became_peered":"2023-12-20T04:44:27.426120+0000","last_unstale":"2023-12-20T04:44:28.825765+0000","last_undegraded":"2023-12-20T04:44:28.825765+0000","last_fullsized":"2023-12-20T04:44:28.825765+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:25.534367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.19","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909005+0000","last_change":"2023-12-20T04:44:27.858818+0000","last_active":"2023-12-20T04:44:28.909005+0000","last_peered":"2023-12-20T04:44:28.909005+0000","last_clean":"2023-12-20T04:44:28.909005+0000","last_became_active":"2023-12-20T04:44:27.858552+0000","last_became_peered":"2023-12-20T04:44:27.858552+0000","last_unstale":"2023-12-20T04:44:28.909005+0000","last_undegraded":"2023-12-20T04:44:28.909005+0000","last_fullsized":"2023-12-20T04:44:28.909005+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:32:02.736821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.18","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909086+0000","last_change":"2023-12-20T04:44:27.435217+0000","last_active":"2023-12-20T04:44:28.909086+0000","last_peered":"2023-12-20T04:44:28.909086+0000","last_clean":"2023-12-20T04:44:28.909086+0000","last_became_active":"2023-12-20T04:44:27.434970+0000","last_became_peered":"2023-12-20T04:44:27.434970+0000","last_unstale":"2023-12-20T04:44:28.909086+0000","last_undegraded":"2023-12-20T04:44:28.909086+0000","last_fullsized":"2023-12-20T04:44:28.909086+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:53.664083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.16","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909448+0000","last_change":"2023-12-20T04:44:27.840070+0000","last_active":"2023-12-20T04:44:28.909448+0000","last_peered":"2023-12-20T04:44:28.909448+0000","last_clean":"2023-12-20T04:44:28.909448+0000","last_became_active":"2023-12-20T04:44:27.839653+0000","last_became_peered":"2023-12-20T04:44:27.839653+0000","last_unstale":"2023-12-20T04:44:28.909448+0000","last_undegraded":"2023-12-20T04:44:28.909448+0000","last_fullsized":"2023-12-20T04:44:28.909448+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:47:27.704952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"12.17","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910089+0000","last_change":"2023-12-20T04:44:27.441770+0000","last_active":"2023-12-20T04:44:28.910089+0000","last_peered":"2023-12-20T04:44:28.910089+0000","last_clean":"2023-12-20T04:44:28.910089+0000","last_became_active":"2023-12-20T04:44:27.441398+0000","last_became_peered":"2023-12-20T04:44:27.441398+0000","last_unstale":"2023-12-20T04:44:28.910089+0000","last_undegraded":"2023-12-20T04:44:28.910089+0000","last_fullsized":"2023-12-20T04:44:28.910089+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:53:11.376149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.17","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827808+0000","last_change":"2023-12-20T04:44:27.855738+0000","last_active":"2023-12-20T04:44:28.827808+0000","last_peered":"2023-12-20T04:44:28.827808+0000","last_clean":"2023-12-20T04:44:28.827808+0000","last_became_active":"2023-12-20T04:44:27.855437+0000","last_became_peered":"2023-12-20T04:44:27.855437+0000","last_unstale":"2023-12-20T04:44:28.827808+0000","last_undegraded":"2023-12-20T04:44:28.827808+0000","last_fullsized":"2023-12-20T04:44:28.827808+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:54:36.565658+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.16","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909885+0000","last_change":"2023-12-20T04:44:27.439468+0000","last_active":"2023-12-20T04:44:28.909885+0000","last_peered":"2023-12-20T04:44:28.909885+0000","last_clean":"2023-12-20T04:44:28.909885+0000","last_became_active":"2023-12-20T04:44:27.439125+0000","last_became_peered":"2023-12-20T04:44:27.439125+0000","last_unstale":"2023-12-20T04:44:28.909885+0000","last_undegraded":"2023-12-20T04:44:28.909885+0000","last_fullsized":"2023-12-20T04:44:28.909885+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:53.664083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.14","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826005+0000","last_change":"2023-12-20T04:44:27.857696+0000","last_active":"2023-12-20T04:44:28.826005+0000","last_peered":"2023-12-20T04:44:28.826005+0000","last_clean":"2023-12-20T04:44:28.826005+0000","last_became_active":"2023-12-20T04:44:27.857374+0000","last_became_peered":"2023-12-20T04:44:27.857374+0000","last_unstale":"2023-12-20T04:44:28.826005+0000","last_undegraded":"2023-12-20T04:44:28.826005+0000","last_fullsized":"2023-12-20T04:44:28.826005+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:02:33.755429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.15","version":"106'3","reported_seq":14,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909927+0000","last_change":"2023-12-20T04:44:27.436072+0000","last_active":"2023-12-20T04:44:28.909927+0000","last_peered":"2023-12-20T04:44:28.909927+0000","last_clean":"2023-12-20T04:44:28.909927+0000","last_became_active":"2023-12-20T04:44:27.435773+0000","last_became_peered":"2023-12-20T04:44:27.435773+0000","last_unstale":"2023-12-20T04:44:28.909927+0000","last_undegraded":"2023-12-20T04:44:28.909927+0000","last_fullsized":"2023-12-20T04:44:28.909927+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:33.813685+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.15","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828323+0000","last_change":"2023-12-20T04:44:27.853454+0000","last_active":"2023-12-20T04:44:28.828323+0000","last_peered":"2023-12-20T04:44:28.828323+0000","last_clean":"2023-12-20T04:44:28.828323+0000","last_became_active":"2023-12-20T04:44:27.853147+0000","last_became_peered":"2023-12-20T04:44:27.853147+0000","last_unstale":"2023-12-20T04:44:28.828323+0000","last_undegraded":"2023-12-20T04:44:28.828323+0000","last_fullsized":"2023-12-20T04:44:28.828323+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:19:04.048295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.14","version":"106'6","reported_seq":17,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827501+0000","last_change":"2023-12-20T04:44:27.435127+0000","last_active":"2023-12-20T04:44:28.827501+0000","last_peered":"2023-12-20T04:44:28.827501+0000","last_clean":"2023-12-20T04:44:28.827501+0000","last_became_active":"2023-12-20T04:44:27.434734+0000","last_became_peered":"2023-12-20T04:44:27.434734+0000","last_unstale":"2023-12-20T04:44:28.827501+0000","last_undegraded":"2023-12-20T04:44:28.827501+0000","last_fullsized":"2023-12-20T04:44:28.827501+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:59:36.745949+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.12","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.825807+0000","last_change":"2023-12-20T04:44:27.846671+0000","last_active":"2023-12-20T04:44:28.825807+0000","last_peered":"2023-12-20T04:44:28.825807+0000","last_clean":"2023-12-20T04:44:28.825807+0000","last_became_active":"2023-12-20T04:44:27.845561+0000","last_became_peered":"2023-12-20T04:44:27.845561+0000","last_unstale":"2023-12-20T04:44:28.825807+0000","last_undegraded":"2023-12-20T04:44:28.825807+0000","last_fullsized":"2023-12-20T04:44:28.825807+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:59:33.634682+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.13","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909381+0000","last_change":"2023-12-20T04:44:27.429846+0000","last_active":"2023-12-20T04:44:28.909381+0000","last_peered":"2023-12-20T04:44:28.909381+0000","last_clean":"2023-12-20T04:44:28.909381+0000","last_became_active":"2023-12-20T04:44:27.427964+0000","last_became_peered":"2023-12-20T04:44:27.427964+0000","last_unstale":"2023-12-20T04:44:28.909381+0000","last_undegraded":"2023-12-20T04:44:28.909381+0000","last_fullsized":"2023-12-20T04:44:28.909381+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:57.383170+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.13","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909077+0000","last_change":"2023-12-20T04:44:27.856468+0000","last_active":"2023-12-20T04:44:28.909077+0000","last_peered":"2023-12-20T04:44:28.909077+0000","last_clean":"2023-12-20T04:44:28.909077+0000","last_became_active":"2023-12-20T04:44:27.856191+0000","last_became_peered":"2023-12-20T04:44:27.856191+0000","last_unstale":"2023-12-20T04:44:28.909077+0000","last_undegraded":"2023-12-20T04:44:28.909077+0000","last_fullsized":"2023-12-20T04:44:28.909077+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:40:00.475342+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.12","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908941+0000","last_change":"2023-12-20T04:44:27.437001+0000","last_active":"2023-12-20T04:44:28.908941+0000","last_peered":"2023-12-20T04:44:28.908941+0000","last_clean":"2023-12-20T04:44:28.908941+0000","last_became_active":"2023-12-20T04:44:27.436661+0000","last_became_peered":"2023-12-20T04:44:27.436661+0000","last_unstale":"2023-12-20T04:44:28.908941+0000","last_undegraded":"2023-12-20T04:44:28.908941+0000","last_fullsized":"2023-12-20T04:44:28.908941+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:33.944565+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.10","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909060+0000","last_change":"2023-12-20T04:44:27.839452+0000","last_active":"2023-12-20T04:44:28.909060+0000","last_peered":"2023-12-20T04:44:28.909060+0000","last_clean":"2023-12-20T04:44:28.909060+0000","last_became_active":"2023-12-20T04:44:27.839172+0000","last_became_peered":"2023-12-20T04:44:27.839172+0000","last_unstale":"2023-12-20T04:44:28.909060+0000","last_undegraded":"2023-12-20T04:44:28.909060+0000","last_fullsized":"2023-12-20T04:44:28.909060+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:59:37.748870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.11","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827120+0000","last_change":"2023-12-20T04:44:27.439496+0000","last_active":"2023-12-20T04:44:28.827120+0000","last_peered":"2023-12-20T04:44:28.827120+0000","last_clean":"2023-12-20T04:44:28.827120+0000","last_became_active":"2023-12-20T04:44:27.439116+0000","last_became_peered":"2023-12-20T04:44:27.439116+0000","last_unstale":"2023-12-20T04:44:28.827120+0000","last_undegraded":"2023-12-20T04:44:28.827120+0000","last_fullsized":"2023-12-20T04:44:28.827120+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:35.228356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"13.11","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908835+0000","last_change":"2023-12-20T04:44:27.856876+0000","last_active":"2023-12-20T04:44:28.908835+0000","last_peered":"2023-12-20T04:44:28.908835+0000","last_clean":"2023-12-20T04:44:28.908835+0000","last_became_active":"2023-12-20T04:44:27.856571+0000","last_became_peered":"2023-12-20T04:44:27.856571+0000","last_unstale":"2023-12-20T04:44:28.908835+0000","last_undegraded":"2023-12-20T04:44:28.908835+0000","last_fullsized":"2023-12-20T04:44:28.908835+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:06:30.990973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.10","version":"106'8","reported_seq":19,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.825727+0000","last_change":"2023-12-20T04:44:27.435375+0000","last_active":"2023-12-20T04:44:28.825727+0000","last_peered":"2023-12-20T04:44:28.825727+0000","last_clean":"2023-12-20T04:44:28.825727+0000","last_became_active":"2023-12-20T04:44:27.435084+0000","last_became_peered":"2023-12-20T04:44:27.435084+0000","last_unstale":"2023-12-20T04:44:28.825727+0000","last_undegraded":"2023-12-20T04:44:28.825727+0000","last_fullsized":"2023-12-20T04:44:28.825727+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:51:13.489172+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.e","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909433+0000","last_change":"2023-12-20T04:44:27.858556+0000","last_active":"2023-12-20T04:44:28.909433+0000","last_peered":"2023-12-20T04:44:28.909433+0000","last_clean":"2023-12-20T04:44:28.909433+0000","last_became_active":"2023-12-20T04:44:27.858213+0000","last_became_peered":"2023-12-20T04:44:27.858213+0000","last_unstale":"2023-12-20T04:44:28.909433+0000","last_undegraded":"2023-12-20T04:44:28.909433+0000","last_fullsized":"2023-12-20T04:44:28.909433+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:10:11.751473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.f","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910153+0000","last_change":"2023-12-20T04:44:27.440579+0000","last_active":"2023-12-20T04:44:28.910153+0000","last_peered":"2023-12-20T04:44:28.910153+0000","last_clean":"2023-12-20T04:44:28.910153+0000","last_became_active":"2023-12-20T04:44:27.438310+0000","last_became_peered":"2023-12-20T04:44:27.438310+0000","last_unstale":"2023-12-20T04:44:28.910153+0000","last_undegraded":"2023-12-20T04:44:28.910153+0000","last_fullsized":"2023-12-20T04:44:28.910153+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:16:04.457192+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.f","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909989+0000","last_change":"2023-12-20T04:44:27.854848+0000","last_active":"2023-12-20T04:44:28.909989+0000","last_peered":"2023-12-20T04:44:28.909989+0000","last_clean":"2023-12-20T04:44:28.909989+0000","last_became_active":"2023-12-20T04:44:27.854536+0000","last_became_peered":"2023-12-20T04:44:27.854536+0000","last_unstale":"2023-12-20T04:44:28.909989+0000","last_undegraded":"2023-12-20T04:44:28.909989+0000","last_fullsized":"2023-12-20T04:44:28.909989+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:08:16.893096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.e","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826376+0000","last_change":"2023-12-20T04:44:27.436175+0000","last_active":"2023-12-20T04:44:28.826376+0000","last_peered":"2023-12-20T04:44:28.826376+0000","last_clean":"2023-12-20T04:44:28.826376+0000","last_became_active":"2023-12-20T04:44:27.435909+0000","last_became_peered":"2023-12-20T04:44:27.435909+0000","last_unstale":"2023-12-20T04:44:28.826376+0000","last_undegraded":"2023-12-20T04:44:28.826376+0000","last_fullsized":"2023-12-20T04:44:28.826376+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:42:16.707918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.d","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828405+0000","last_change":"2023-12-20T04:44:27.840420+0000","last_active":"2023-12-20T04:44:28.828405+0000","last_peered":"2023-12-20T04:44:28.828405+0000","last_clean":"2023-12-20T04:44:28.828405+0000","last_became_active":"2023-12-20T04:44:27.840101+0000","last_became_peered":"2023-12-20T04:44:27.840101+0000","last_unstale":"2023-12-20T04:44:28.828405+0000","last_undegraded":"2023-12-20T04:44:28.828405+0000","last_fullsized":"2023-12-20T04:44:28.828405+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:10:11.751473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.c","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826144+0000","last_change":"2023-12-20T04:44:27.435281+0000","last_active":"2023-12-20T04:44:28.826144+0000","last_peered":"2023-12-20T04:44:28.826144+0000","last_clean":"2023-12-20T04:44:28.826144+0000","last_became_active":"2023-12-20T04:44:27.434999+0000","last_became_peered":"2023-12-20T04:44:27.434999+0000","last_unstale":"2023-12-20T04:44:28.826144+0000","last_undegraded":"2023-12-20T04:44:28.826144+0000","last_fullsized":"2023-12-20T04:44:28.826144+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:23.833193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.a","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910203+0000","last_change":"2023-12-20T04:44:27.840722+0000","last_active":"2023-12-20T04:44:28.910203+0000","last_peered":"2023-12-20T04:44:28.910203+0000","last_clean":"2023-12-20T04:44:28.910203+0000","last_became_active":"2023-12-20T04:44:27.840072+0000","last_became_peered":"2023-12-20T04:44:27.840072+0000","last_unstale":"2023-12-20T04:44:28.910203+0000","last_undegraded":"2023-12-20T04:44:28.910203+0000","last_fullsized":"2023-12-20T04:44:28.910203+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:54.667004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.b","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909169+0000","last_change":"2023-12-20T04:44:27.437973+0000","last_active":"2023-12-20T04:44:28.909169+0000","last_peered":"2023-12-20T04:44:28.909169+0000","last_clean":"2023-12-20T04:44:28.909169+0000","last_became_active":"2023-12-20T04:44:27.437694+0000","last_became_peered":"2023-12-20T04:44:27.437694+0000","last_unstale":"2023-12-20T04:44:28.909169+0000","last_undegraded":"2023-12-20T04:44:28.909169+0000","last_fullsized":"2023-12-20T04:44:28.909169+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:19:21.689905+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.b","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827751+0000","last_change":"2023-12-20T04:44:27.844180+0000","last_active":"2023-12-20T04:44:28.827751+0000","last_peered":"2023-12-20T04:44:28.827751+0000","last_clean":"2023-12-20T04:44:28.827751+0000","last_became_active":"2023-12-20T04:44:27.843861+0000","last_became_peered":"2023-12-20T04:44:27.843861+0000","last_unstale":"2023-12-20T04:44:28.827751+0000","last_undegraded":"2023-12-20T04:44:28.827751+0000","last_fullsized":"2023-12-20T04:44:28.827751+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:02:53.405091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.a","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910227+0000","last_change":"2023-12-20T04:44:27.433582+0000","last_active":"2023-12-20T04:44:28.910227+0000","last_peered":"2023-12-20T04:44:28.910227+0000","last_clean":"2023-12-20T04:44:28.910227+0000","last_became_active":"2023-12-20T04:44:27.432715+0000","last_became_peered":"2023-12-20T04:44:27.432715+0000","last_unstale":"2023-12-20T04:44:28.910227+0000","last_undegraded":"2023-12-20T04:44:28.910227+0000","last_fullsized":"2023-12-20T04:44:28.910227+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:10:55.086465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.8","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908693+0000","last_change":"2023-12-20T04:44:27.838723+0000","last_active":"2023-12-20T04:44:28.908693+0000","last_peered":"2023-12-20T04:44:28.908693+0000","last_clean":"2023-12-20T04:44:28.908693+0000","last_became_active":"2023-12-20T04:44:27.838472+0000","last_became_peered":"2023-12-20T04:44:27.838472+0000","last_unstale":"2023-12-20T04:44:28.908693+0000","last_undegraded":"2023-12-20T04:44:28.908693+0000","last_fullsized":"2023-12-20T04:44:28.908693+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:14:34.837571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"12.9","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909433+0000","last_change":"2023-12-20T04:44:27.436452+0000","last_active":"2023-12-20T04:44:28.909433+0000","last_peered":"2023-12-20T04:44:28.909433+0000","last_clean":"2023-12-20T04:44:28.909433+0000","last_became_active":"2023-12-20T04:44:27.436153+0000","last_became_peered":"2023-12-20T04:44:27.436153+0000","last_unstale":"2023-12-20T04:44:28.909433+0000","last_undegraded":"2023-12-20T04:44:28.909433+0000","last_fullsized":"2023-12-20T04:44:28.909433+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:19:26.169237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"13.9","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827018+0000","last_change":"2023-12-20T04:44:27.847933+0000","last_active":"2023-12-20T04:44:28.827018+0000","last_peered":"2023-12-20T04:44:28.827018+0000","last_clean":"2023-12-20T04:44:28.827018+0000","last_became_active":"2023-12-20T04:44:27.847474+0000","last_became_peered":"2023-12-20T04:44:27.847474+0000","last_unstale":"2023-12-20T04:44:28.827018+0000","last_undegraded":"2023-12-20T04:44:28.827018+0000","last_fullsized":"2023-12-20T04:44:28.827018+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:49:22.216892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.8","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908681+0000","last_change":"2023-12-20T04:44:27.434596+0000","last_active":"2023-12-20T04:44:28.908681+0000","last_peered":"2023-12-20T04:44:28.908681+0000","last_clean":"2023-12-20T04:44:28.908681+0000","last_became_active":"2023-12-20T04:44:27.433098+0000","last_became_peered":"2023-12-20T04:44:27.433098+0000","last_unstale":"2023-12-20T04:44:28.908681+0000","last_undegraded":"2023-12-20T04:44:28.908681+0000","last_fullsized":"2023-12-20T04:44:28.908681+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:19:03.045373+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"13.6","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828464+0000","last_change":"2023-12-20T04:44:27.841449+0000","last_active":"2023-12-20T04:44:28.828464+0000","last_peered":"2023-12-20T04:44:28.828464+0000","last_clean":"2023-12-20T04:44:28.828464+0000","last_became_active":"2023-12-20T04:44:27.839703+0000","last_became_peered":"2023-12-20T04:44:27.839703+0000","last_unstale":"2023-12-20T04:44:28.828464+0000","last_undegraded":"2023-12-20T04:44:28.828464+0000","last_fullsized":"2023-12-20T04:44:28.828464+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:51:14.492093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.7","version":"106'2","reported_seq":13,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910447+0000","last_change":"2023-12-20T04:44:27.435556+0000","last_active":"2023-12-20T04:44:28.910447+0000","last_peered":"2023-12-20T04:44:28.910447+0000","last_clean":"2023-12-20T04:44:28.910447+0000","last_became_active":"2023-12-20T04:44:27.435244+0000","last_became_peered":"2023-12-20T04:44:27.435244+0000","last_unstale":"2023-12-20T04:44:28.910447+0000","last_undegraded":"2023-12-20T04:44:28.910447+0000","last_fullsized":"2023-12-20T04:44:28.910447+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:28.644629+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.7","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909596+0000","last_change":"2023-12-20T04:44:27.863560+0000","last_active":"2023-12-20T04:44:28.909596+0000","last_peered":"2023-12-20T04:44:28.909596+0000","last_clean":"2023-12-20T04:44:28.909596+0000","last_became_active":"2023-12-20T04:44:27.863208+0000","last_became_peered":"2023-12-20T04:44:27.863208+0000","last_unstale":"2023-12-20T04:44:28.909596+0000","last_undegraded":"2023-12-20T04:44:28.909596+0000","last_fullsized":"2023-12-20T04:44:28.909596+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:19:04.048295+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.6","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910315+0000","last_change":"2023-12-20T04:44:27.428114+0000","last_active":"2023-12-20T04:44:28.910315+0000","last_peered":"2023-12-20T04:44:28.910315+0000","last_clean":"2023-12-20T04:44:28.910315+0000","last_became_active":"2023-12-20T04:44:27.427821+0000","last_became_peered":"2023-12-20T04:44:27.427821+0000","last_unstale":"2023-12-20T04:44:28.910315+0000","last_undegraded":"2023-12-20T04:44:28.910315+0000","last_fullsized":"2023-12-20T04:44:28.910315+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:13:19.646627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"13.4","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909823+0000","last_change":"2023-12-20T04:44:27.862284+0000","last_active":"2023-12-20T04:44:28.909823+0000","last_peered":"2023-12-20T04:44:28.909823+0000","last_clean":"2023-12-20T04:44:28.909823+0000","last_became_active":"2023-12-20T04:44:27.858651+0000","last_became_peered":"2023-12-20T04:44:27.858651+0000","last_unstale":"2023-12-20T04:44:28.909823+0000","last_undegraded":"2023-12-20T04:44:28.909823+0000","last_fullsized":"2023-12-20T04:44:28.909823+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:02:53.405091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.5","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909986+0000","last_change":"2023-12-20T04:44:27.435039+0000","last_active":"2023-12-20T04:44:28.909986+0000","last_peered":"2023-12-20T04:44:28.909986+0000","last_clean":"2023-12-20T04:44:28.909986+0000","last_became_active":"2023-12-20T04:44:27.433872+0000","last_became_peered":"2023-12-20T04:44:27.433872+0000","last_unstale":"2023-12-20T04:44:28.909986+0000","last_undegraded":"2023-12-20T04:44:28.909986+0000","last_fullsized":"2023-12-20T04:44:28.909986+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:35.813754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.5","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.828463+0000","last_change":"2023-12-20T04:44:27.848367+0000","last_active":"2023-12-20T04:44:28.828463+0000","last_peered":"2023-12-20T04:44:28.828463+0000","last_clean":"2023-12-20T04:44:28.828463+0000","last_became_active":"2023-12-20T04:44:27.848080+0000","last_became_peered":"2023-12-20T04:44:27.848080+0000","last_unstale":"2023-12-20T04:44:28.828463+0000","last_undegraded":"2023-12-20T04:44:28.828463+0000","last_fullsized":"2023-12-20T04:44:28.828463+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:46:48.669519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"12.4","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909941+0000","last_change":"2023-12-20T04:44:27.442833+0000","last_active":"2023-12-20T04:44:28.909941+0000","last_peered":"2023-12-20T04:44:28.909941+0000","last_clean":"2023-12-20T04:44:28.909941+0000","last_became_active":"2023-12-20T04:44:27.441611+0000","last_became_peered":"2023-12-20T04:44:27.441611+0000","last_unstale":"2023-12-20T04:44:28.909941+0000","last_undegraded":"2023-12-20T04:44:28.909941+0000","last_fullsized":"2023-12-20T04:44:28.909941+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:14:33.834650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.2","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827560+0000","last_change":"2023-12-20T04:44:27.838999+0000","last_active":"2023-12-20T04:44:28.827560+0000","last_peered":"2023-12-20T04:44:28.827560+0000","last_clean":"2023-12-20T04:44:28.827560+0000","last_became_active":"2023-12-20T04:44:27.838683+0000","last_became_peered":"2023-12-20T04:44:27.838683+0000","last_unstale":"2023-12-20T04:44:28.827560+0000","last_undegraded":"2023-12-20T04:44:28.827560+0000","last_fullsized":"2023-12-20T04:44:28.827560+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:44.406191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"12.3","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826057+0000","last_change":"2023-12-20T04:44:27.437342+0000","last_active":"2023-12-20T04:44:28.826057+0000","last_peered":"2023-12-20T04:44:28.826057+0000","last_clean":"2023-12-20T04:44:28.826057+0000","last_became_active":"2023-12-20T04:44:27.437073+0000","last_became_peered":"2023-12-20T04:44:27.437073+0000","last_unstale":"2023-12-20T04:44:28.826057+0000","last_undegraded":"2023-12-20T04:44:28.826057+0000","last_fullsized":"2023-12-20T04:44:28.826057+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:10:10.748552+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"13.3","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.826013+0000","last_change":"2023-12-20T04:44:27.841942+0000","last_active":"2023-12-20T04:44:28.826013+0000","last_peered":"2023-12-20T04:44:28.826013+0000","last_clean":"2023-12-20T04:44:28.826013+0000","last_became_active":"2023-12-20T04:44:27.841691+0000","last_became_peered":"2023-12-20T04:44:27.841691+0000","last_unstale":"2023-12-20T04:44:28.826013+0000","last_undegraded":"2023-12-20T04:44:28.826013+0000","last_fullsized":"2023-12-20T04:44:28.826013+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:47:27.704952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"12.2","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827823+0000","last_change":"2023-12-20T04:44:27.436369+0000","last_active":"2023-12-20T04:44:28.827823+0000","last_peered":"2023-12-20T04:44:28.827823+0000","last_clean":"2023-12-20T04:44:28.827823+0000","last_became_active":"2023-12-20T04:44:27.432401+0000","last_became_peered":"2023-12-20T04:44:27.432401+0000","last_unstale":"2023-12-20T04:44:28.827823+0000","last_undegraded":"2023-12-20T04:44:28.827823+0000","last_fullsized":"2023-12-20T04:44:28.827823+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:43.403270+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.0","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.908955+0000","last_change":"2023-12-20T04:44:27.852121+0000","last_active":"2023-12-20T04:44:28.908955+0000","last_peered":"2023-12-20T04:44:28.908955+0000","last_clean":"2023-12-20T04:44:28.908955+0000","last_became_active":"2023-12-20T04:44:27.851797+0000","last_became_peered":"2023-12-20T04:44:27.851797+0000","last_unstale":"2023-12-20T04:44:28.908955+0000","last_undegraded":"2023-12-20T04:44:28.908955+0000","last_fullsized":"2023-12-20T04:44:28.908955+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:42:17.710839+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"12.1","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909525+0000","last_change":"2023-12-20T04:44:27.444483+0000","last_active":"2023-12-20T04:44:28.909525+0000","last_peered":"2023-12-20T04:44:28.909525+0000","last_clean":"2023-12-20T04:44:28.909525+0000","last_became_active":"2023-12-20T04:44:27.444122+0000","last_became_peered":"2023-12-20T04:44:27.444122+0000","last_unstale":"2023-12-20T04:44:28.909525+0000","last_undegraded":"2023-12-20T04:44:28.909525+0000","last_fullsized":"2023-12-20T04:44:28.909525+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:03:10.661854+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"13.1","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909514+0000","last_change":"2023-12-20T04:44:27.862352+0000","last_active":"2023-12-20T04:44:28.909514+0000","last_peered":"2023-12-20T04:44:28.909514+0000","last_clean":"2023-12-20T04:44:28.909514+0000","last_became_active":"2023-12-20T04:44:27.862068+0000","last_became_peered":"2023-12-20T04:44:27.862068+0000","last_unstale":"2023-12-20T04:44:28.909514+0000","last_undegraded":"2023-12-20T04:44:28.909514+0000","last_fullsized":"2023-12-20T04:44:28.909514+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:49:22.216892+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"12.0","version":"0'0","reported_seq":11,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.827118+0000","last_change":"2023-12-20T04:44:27.443444+0000","last_active":"2023-12-20T04:44:28.827118+0000","last_peered":"2023-12-20T04:44:28.827118+0000","last_clean":"2023-12-20T04:44:28.827118+0000","last_became_active":"2023-12-20T04:44:27.443107+0000","last_became_peered":"2023-12-20T04:44:27.443107+0000","last_unstale":"2023-12-20T04:44:28.827118+0000","last_undegraded":"2023-12-20T04:44:28.827118+0000","last_fullsized":"2023-12-20T04:44:28.827118+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:49:21.213971+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"13.c","version":"0'0","reported_seq":10,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.910179+0000","last_change":"2023-12-20T04:44:27.842597+0000","last_active":"2023-12-20T04:44:28.910179+0000","last_peered":"2023-12-20T04:44:28.910179+0000","last_clean":"2023-12-20T04:44:28.910179+0000","last_became_active":"2023-12-20T04:44:27.842332+0000","last_became_peered":"2023-12-20T04:44:27.842332+0000","last_unstale":"2023-12-20T04:44:28.910179+0000","last_undegraded":"2023-12-20T04:44:28.910179+0000","last_fullsized":"2023-12-20T04:44:28.910179+0000","mapping_epoch":105,"log_start":"0'0","ondisk_log_start":"0'0","created":105,"last_epoch_clean":106,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:27.404828+0000","last_clean_scrub_stamp":"2023-12-20T04:44:27.404828+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:17:01.428136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"12.d","version":"106'1","reported_seq":12,"reported_epoch":107,"state":"active+clean","last_fresh":"2023-12-20T04:44:28.909448+0000","last_change":"2023-12-20T04:44:27.437751+0000","last_active":"2023-12-20T04:44:28.909448+0000","last_peered":"2023-12-20T04:44:28.909448+0000","last_clean":"2023-12-20T04:44:28.909448+0000","last_became_active":"2023-12-20T04:44:27.437451+0000","last_became_peered":"2023-12-20T04:44:27.437451+0000","last_unstale":"2023-12-20T04:44:28.909448+0000","last_undegraded":"2023-12-20T04:44:28.909448+0000","last_fullsized":"2023-12-20T04:44:28.909448+0000","mapping_epoch":104,"log_start":"0'0","ondisk_log_start":"0'0","created":104,"last_epoch_clean":105,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:44:26.401907+0000","last_clean_scrub_stamp":"2023-12-20T04:44:26.401907+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:48.627999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"96'3058","reported_seq":3359,"reported_epoch":106,"state":"active+clean","last_fresh":"2023-12-20T04:44:27.839752+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:44:27.839752+0000","last_peered":"2023-12-20T04:44:27.839752+0000","last_clean":"2023-12-20T04:44:27.839752+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:44:27.839752+0000","last_undegraded":"2023-12-20T04:44:27.839752+0000","last_fullsized":"2023-12-20T04:44:27.839752+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3058,"log_dups_size":0,"ondisk_log_size":3058,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":637,"num_read_kb":1171,"num_write":261,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":13,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":12,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":637,"num_read_kb":1171,"num_write":261,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3058,"ondisk_log_size":3058,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575040,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":32904,"kb_used_data":1856,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338936,"statfs":{"total":96636764160,"available":96603070464,"internally_reserved":0,"allocated":1900544,"data_stored":1204499,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.61699999999999999,"5min":0.69499999999999995,"15min":0.70199999999999996},"min":{"1min":0.29799999999999999,"5min":0.29799999999999999,"15min":0.29799999999999999},"max":{"1min":0.93899999999999995,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.56200000000000006},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.69299999999999995,"15min":0.70699999999999996},"min":{"1min":0.36899999999999999,"5min":0.36899999999999999,"15min":0.36899999999999999},"max":{"1min":0.84499999999999997,"5min":1,"15min":1.115},"last":0.73799999999999999}]},{"osd":1,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.71299999999999997,"15min":0.72099999999999997},"min":{"1min":0.435,"5min":0.435,"15min":0.38100000000000001},"max":{"1min":0.89400000000000002,"5min":1.095,"15min":1.2190000000000001},"last":0.84299999999999997},{"interface":"front","average":{"1min":0.629,"5min":0.71799999999999997,"15min":0.73399999999999999},"min":{"1min":0.377,"5min":0.377,"15min":0.377},"max":{"1min":0.91700000000000004,"5min":1.032,"15min":2.649},"last":0.63500000000000001}]},{"osd":2,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.68999999999999995,"15min":0.71099999999999997},"min":{"1min":0.379,"5min":0.379,"15min":0.379},"max":{"1min":0.89300000000000002,"5min":1.075,"15min":1.5960000000000001},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.74299999999999999,"15min":0.749},"min":{"1min":0.49199999999999999,"5min":0.46899999999999997,"15min":0.45300000000000001},"max":{"1min":0.98999999999999999,"5min":1.1000000000000001,"15min":1.135},"last":0.70999999999999996}]},{"osd":3,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.624,"5min":0.70099999999999996,"15min":0.72599999999999998},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.03,"5min":1.03,"15min":1.641},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.67500000000000004,"5min":0.76000000000000001,"15min":0.76600000000000001},"min":{"1min":0.51900000000000002,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.93500000000000005,"5min":1.131,"15min":2.5990000000000002},"last":0.88400000000000001}]},{"osd":4,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.59799999999999998,"15min":0.59999999999999998},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.34599999999999997},"max":{"1min":0.78500000000000003,"5min":0.90700000000000003,"15min":1.0109999999999999},"last":0.57699999999999996},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.624,"15min":0.626},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.79500000000000004,"5min":1.0469999999999999,"15min":1.242},"last":0.65800000000000003}]},{"osd":5,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.66400000000000003,"15min":0.65800000000000003},"min":{"1min":0.438,"5min":0.433,"15min":0.40400000000000003},"max":{"1min":0.86299999999999999,"5min":1.089,"15min":1.0900000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.67400000000000004,"15min":0.67800000000000005},"min":{"1min":0.436,"5min":0.436,"15min":0.432},"max":{"1min":0.83199999999999996,"5min":0.98799999999999999,"15min":1.071},"last":0.86499999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:43:53 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.747,"15min":0.73899999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.443},"max":{"1min":0.99199999999999999,"5min":1.093,"15min":1.1279999999999999},"last":0.81100000000000005},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.70999999999999996,"15min":0.70399999999999996},"min":{"1min":0.46100000000000002,"5min":0.46100000000000002,"15min":0.44400000000000001},"max":{"1min":0.96799999999999997,"5min":1.125,"15min":1.125},"last":0.82399999999999995}]}]},{"osd":6,"up_from":13,"seq":55834575040,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36428,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335412,"statfs":{"total":96636764160,"available":96599461888,"internally_reserved":0,"allocated":1314816,"data_stored":614755,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.69499999999999995,"15min":0.71699999999999997},"min":{"1min":0.41399999999999998,"5min":0.33800000000000002,"15min":0.27100000000000002},"max":{"1min":1.1140000000000001,"5min":1.2649999999999999,"15min":1.2649999999999999},"last":0.40899999999999997},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.69699999999999995,"15min":0.71699999999999997},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.034,"5min":1.04,"15min":1.1910000000000001},"last":0.628}]},{"osd":1,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.66200000000000003,"15min":0.69899999999999995},"min":{"1min":0.36899999999999999,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":1.0129999999999999,"5min":1.016,"15min":1.016},"last":0.38600000000000001},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.70999999999999996,"15min":0.72699999999999998},"min":{"1min":0.372,"5min":0.372,"15min":0.36599999999999999},"max":{"1min":0.98899999999999999,"5min":1.1659999999999999,"15min":1.1659999999999999},"last":0.36199999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.747,"15min":0.76100000000000001},"min":{"1min":0.49199999999999999,"5min":0.47299999999999998,"15min":0.40899999999999997},"max":{"1min":1.0580000000000001,"5min":1.216,"15min":1.216},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.71599999999999997,"15min":0.73299999999999998},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.38300000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.55900000000000005}]},{"osd":3,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66200000000000003,"5min":0.753,"15min":0.75900000000000001},"min":{"1min":0.40300000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":1.093,"5min":1.093,"15min":1.109},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.69799999999999995,"15min":0.73199999999999998},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.97599999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.44}]},{"osd":4,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.65600000000000003,"5min":0.625,"15min":0.61899999999999999},"min":{"1min":0.47099999999999997,"5min":0.39900000000000002,"15min":0.36399999999999999},"max":{"1min":0.94099999999999995,"5min":1.1040000000000001,"15min":1.1040000000000001},"last":0.45800000000000002},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.64400000000000002,"15min":0.63100000000000001},"min":{"1min":0.378,"5min":0.378,"15min":0.36899999999999999},"max":{"1min":0.93700000000000006,"5min":0.995,"15min":1.071},"last":0.51100000000000001}]},{"osd":5,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56399999999999995,"5min":0.61699999999999999,"15min":0.622},"min":{"1min":0.307,"5min":0.307,"15min":0.307},"max":{"1min":0.76700000000000002,"5min":1.1519999999999999,"15min":1.1519999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.71099999999999997,"5min":0.69399999999999995,"15min":0.68500000000000005},"min":{"1min":0.53200000000000003,"5min":0.435,"15min":0.434},"max":{"1min":0.98899999999999999,"5min":1.306,"15min":1.306},"last":0.53900000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:44:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72199999999999998,"15min":0.71799999999999997},"min":{"1min":0.497,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.95499999999999996,"5min":1.0129999999999999,"15min":1.2270000000000001},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.70499999999999996,"15min":0.70599999999999996},"min":{"1min":0.497,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.92500000000000004,"5min":1.002,"15min":1.169},"last":0.57799999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575041,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36432,"kb_used_data":1288,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335408,"statfs":{"total":96636764160,"available":96599457792,"internally_reserved":0,"allocated":1318912,"data_stored":615647,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.68700000000000006,"15min":0.69299999999999995},"min":{"1min":0.36199999999999999,"5min":0.33700000000000002,"15min":0.29999999999999999},"max":{"1min":0.83099999999999996,"5min":0.97199999999999998,"15min":1.05},"last":0.44800000000000001},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.68999999999999995,"15min":0.70999999999999996},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.34300000000000003},"max":{"1min":0.877,"5min":0.95099999999999996,"15min":0.98599999999999999},"last":0.64700000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.60999999999999999,"5min":0.68999999999999995,"15min":0.69999999999999996},"min":{"1min":0.443,"5min":0.443,"15min":0.441},"max":{"1min":0.88600000000000001,"5min":1.044,"15min":1.0600000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.70999999999999996,"15min":0.73299999999999998},"min":{"1min":0.437,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.95199999999999996,"5min":0.96999999999999997,"15min":1.1140000000000001},"last":0.75600000000000001}]},{"osd":2,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.71799999999999997,"15min":0.72699999999999998},"min":{"1min":0.35699999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.93300000000000005,"5min":0.94399999999999995,"15min":1.153},"last":0.48499999999999999},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.71399999999999997,"15min":0.72299999999999998},"min":{"1min":0.44400000000000001,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.96899999999999997,"5min":0.96899999999999997,"15min":1.004},"last":0.72399999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73199999999999998,"15min":0.74399999999999999},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92500000000000004,"5min":0.95999999999999996,"15min":1.758},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.73299999999999998,"15min":0.748},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.90900000000000003,"5min":0.96599999999999997,"15min":1.0469999999999999},"last":0.78000000000000003}]},{"osd":4,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.60799999999999998,"15min":0.61499999999999999},"min":{"1min":0.42799999999999999,"5min":0.40600000000000003,"15min":0.35099999999999998},"max":{"1min":0.84799999999999998,"5min":0.97299999999999998,"15min":1.0229999999999999},"last":0.69199999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.63800000000000001,"15min":0.63100000000000001},"min":{"1min":0.47099999999999997,"5min":0.40699999999999997,"15min":0.39600000000000002},"max":{"1min":0.94299999999999995,"5min":0.98699999999999999,"15min":1.077},"last":0.95099999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.68899999999999995,"5min":0.68400000000000005,"15min":0.68000000000000005},"min":{"1min":0.47699999999999998,"5min":0.46400000000000002,"15min":0.39900000000000002},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":1.0700000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.61799999999999999,"5min":0.63500000000000001,"15min":0.63600000000000001},"min":{"1min":0.39900000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.87,"5min":0.999,"15min":0.999},"last":0.81799999999999995}]},{"osd":7,"last update":"Wed Dec 20 04:43:51 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.69099999999999995,"15min":0.69499999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.42999999999999999},"max":{"1min":0.90000000000000002,"5min":1.0429999999999999,"15min":1.141},"last":1.006},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.752,"15min":0.751},"min":{"1min":0.57599999999999996,"5min":0.54900000000000004,"15min":0.47899999999999998},"max":{"1min":0.92000000000000004,"5min":1.012,"15min":1.119},"last":0.90700000000000003}]}]},{"osd":4,"up_from":13,"seq":55834575042,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58560,"kb_used_data":1272,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313280,"statfs":{"total":96636764160,"available":96576798720,"internally_reserved":0,"allocated":1302528,"data_stored":614061,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.63200000000000001,"5min":0.67100000000000004,"15min":0.69799999999999995},"min":{"1min":0.28699999999999998,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.90900000000000003,"5min":1.024,"15min":1.5509999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.68400000000000005,"15min":0.70499999999999996},"min":{"1min":0.34000000000000002,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92400000000000004,"5min":1.625,"15min":1.633},"last":0.97699999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.60099999999999998,"5min":0.68400000000000005,"15min":0.71299999999999997},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.89400000000000002,"5min":0.98799999999999999,"15min":1.798},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.73899999999999999,"15min":0.745},"min":{"1min":0.45700000000000002,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":0.96399999999999997,"5min":2.2930000000000001,"15min":2.2930000000000001},"last":0.93999999999999995}]},{"osd":2,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.73599999999999999},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.98099999999999998,"5min":1.52,"15min":1.728},"last":0.89300000000000002},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.47999999999999998,"5min":0.46899999999999997,"15min":0.38700000000000001},"max":{"1min":1.0289999999999999,"5min":1.093,"15min":1.6619999999999999},"last":0.90700000000000003}]},{"osd":3,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.71399999999999997,"15min":0.72399999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.88600000000000001,"5min":1.554,"15min":1.5760000000000001},"last":0.86399999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.372,"5min":0.372,"15min":0.372},"max":{"1min":0.94599999999999995,"5min":1.03,"15min":1.7649999999999999},"last":0.95099999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.63800000000000001,"15min":0.624},"min":{"1min":0.36599999999999999,"5min":0.36099999999999999,"15min":0.34000000000000002},"max":{"1min":1.1120000000000001,"5min":1.456,"15min":1.456},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.64000000000000001,"15min":0.63100000000000001},"min":{"1min":0.42399999999999999,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.077,"5min":1.589,"15min":1.589},"last":0.60499999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.64700000000000002,"15min":0.65200000000000002},"min":{"1min":0.47199999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":1.006,"5min":1.006,"15min":1.6739999999999999},"last":0.70499999999999996},{"interface":"front","average":{"1min":0.72699999999999998,"5min":0.70299999999999996,"15min":0.69699999999999995},"min":{"1min":0.53500000000000003,"5min":0.41699999999999998,"15min":0.41699999999999998},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.622}]},{"osd":7,"last update":"Wed Dec 20 04:43:54 2023","interfaces":[{"interface":"back","average":{"1min":0.67100000000000004,"5min":0.69199999999999995,"15min":0.69099999999999995},"min":{"1min":0.47099999999999997,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":1.036,"5min":1.655,"15min":1.655},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.72099999999999997,"15min":0.73299999999999998},"min":{"1min":0.50700000000000001,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.0029999999999999,"5min":1.052,"15min":1.125},"last":0.80300000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575041,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36440,"kb_used_data":1296,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335400,"statfs":{"total":96636764160,"available":96599449600,"internally_reserved":0,"allocated":1327104,"data_stored":621925,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.42099999999999999,"5min":0.46000000000000002,"15min":0.47499999999999998},"min":{"1min":0.26900000000000002,"5min":0.26200000000000001,"15min":0.23200000000000001},"max":{"1min":0.748,"5min":0.875,"15min":4.3700000000000001},"last":0.435},{"interface":"front","average":{"1min":0.44,"5min":0.499,"15min":0.52000000000000002},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.63100000000000001,"5min":0.80800000000000005,"15min":3.355},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47199999999999998,"15min":0.48499999999999999},"min":{"1min":0.29299999999999998,"5min":0.27400000000000002,"15min":0.27400000000000002},"max":{"1min":0.68200000000000005,"5min":0.73999999999999999,"15min":3.7160000000000002},"last":0.307},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.53300000000000003,"15min":0.55700000000000005},"min":{"1min":0.35499999999999998,"5min":0.316,"15min":0.28399999999999997},"max":{"1min":0.70199999999999996,"5min":0.74399999999999999,"15min":3.3109999999999999},"last":0.42599999999999999}]},{"osd":2,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.53400000000000003,"5min":0.57099999999999995,"15min":0.59199999999999997},"min":{"1min":0.374,"5min":0.318,"15min":0.318},"max":{"1min":0.72799999999999998,"5min":0.88700000000000001,"15min":4.0110000000000001},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.52900000000000003,"5min":0.54800000000000004,"15min":0.57699999999999996},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.316},"max":{"1min":0.81799999999999995,"5min":0.93300000000000005,"15min":6.1399999999999997},"last":0.54200000000000004}]},{"osd":4,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.76800000000000002,"15min":0.78500000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99199999999999999,"5min":1.131,"15min":1.3089999999999999},"last":0.628},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.78300000000000003,"15min":0.81799999999999995},"min":{"1min":0.52300000000000002,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.93799999999999994,"5min":1.4059999999999999,"15min":3.9510000000000001},"last":0.60999999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.80800000000000005,"15min":0.82899999999999996},"min":{"1min":0.498,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":1.0649999999999999,"5min":1.3300000000000001,"15min":3.9830000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.76300000000000001,"15min":0.80200000000000005},"min":{"1min":0.503,"5min":0.503,"15min":0.38400000000000001},"max":{"1min":0.88100000000000001,"5min":1.252,"15min":4.6550000000000002},"last":0.79300000000000004}]},{"osd":6,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.76000000000000001,"15min":0.79800000000000004},"min":{"1min":0.49099999999999999,"5min":0.49099999999999999,"15min":0.45600000000000002},"max":{"1min":0.95199999999999996,"5min":1.2050000000000001,"15min":1.2050000000000001},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.79900000000000004,"15min":0.82999999999999996},"min":{"1min":0.53200000000000003,"5min":0.53200000000000003,"15min":0.499},"max":{"1min":1.0149999999999999,"5min":1.2270000000000001,"15min":3.8900000000000001},"last":0.56000000000000005}]},{"osd":7,"last update":"Wed Dec 20 04:43:49 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.76900000000000002,"15min":0.81399999999999995},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91700000000000004,"5min":1.1319999999999999,"15min":4.6970000000000001},"last":0.84199999999999997},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.79100000000000004,"15min":0.81999999999999995},"min":{"1min":0.45900000000000002,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.86899999999999999,"5min":1.29,"15min":1.29},"last":0.57199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575040,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40516,"kb_used_data":1276,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331324,"statfs":{"total":96636764160,"available":96595275776,"internally_reserved":0,"allocated":1306624,"data_stored":614643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.39600000000000002,"5min":0.41499999999999998,"15min":0.437},"min":{"1min":0.27300000000000002,"5min":0.216,"15min":0.216},"max":{"1min":0.59199999999999997,"5min":0.78900000000000003,"15min":0.89600000000000002},"last":0.371},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48899999999999999,"15min":0.49099999999999999},"min":{"1min":0.26400000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.92200000000000004,"5min":0.92200000000000004,"15min":1.0589999999999999},"last":0.33100000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.46100000000000002,"5min":0.47499999999999998,"15min":0.47199999999999998},"min":{"1min":0.28599999999999998,"5min":0.28599999999999998,"15min":0.28599999999999998},"max":{"1min":0.67600000000000005,"5min":0.876,"15min":0.876},"last":0.313},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.51400000000000001,"15min":0.52500000000000002},"min":{"1min":0.32100000000000001,"5min":0.32000000000000001,"15min":0.28100000000000003},"max":{"1min":0.81100000000000005,"5min":0.81399999999999995,"15min":1.01},"last":0.43099999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.56200000000000006,"5min":0.59099999999999997,"15min":0.59799999999999998},"min":{"1min":0.39500000000000002,"5min":0.373,"15min":0.34999999999999998},"max":{"1min":0.79800000000000004,"5min":0.93899999999999995,"15min":0.93899999999999995},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.60799999999999998,"5min":0.624,"15min":0.63900000000000001},"min":{"1min":0.434,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.85199999999999998,"5min":0.91600000000000004,"15min":1.0900000000000001},"last":0.48099999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.76400000000000001,"15min":0.78200000000000003},"min":{"1min":0.46500000000000002,"5min":0.46500000000000002,"15min":0.45200000000000001},"max":{"1min":1.016,"5min":1.145,"15min":1.1799999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.753,"15min":0.77500000000000002},"min":{"1min":0.50700000000000001,"5min":0.44500000000000001,"15min":0.41399999999999998},"max":{"1min":0.91800000000000004,"5min":1.048,"15min":1.2010000000000001},"last":0.56599999999999995}]},{"osd":5,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.74299999999999999,"15min":0.77300000000000002},"min":{"1min":0.47599999999999998,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.034,"5min":1.1220000000000001,"15min":1.327},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.76900000000000002,"15min":0.80000000000000004},"min":{"1min":0.53900000000000003,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.98399999999999999,"5min":1.04,"15min":1.381},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70999999999999996,"5min":0.77200000000000002,"15min":0.79000000000000004},"min":{"1min":0.47199999999999998,"5min":0.436,"15min":0.436},"max":{"1min":0.998,"5min":1.151,"15min":1.2809999999999999},"last":0.627},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76500000000000001,"15min":0.79400000000000004},"min":{"1min":0.502,"5min":0.502,"15min":0.46100000000000002},"max":{"1min":0.96899999999999997,"5min":1.0609999999999999,"15min":1.153},"last":0.61799999999999999}]},{"osd":7,"last update":"Wed Dec 20 04:44:01 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.79600000000000004,"15min":0.81999999999999995},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":0.98899999999999999,"5min":1.107,"15min":1.3520000000000001},"last":0.65500000000000003},{"interface":"front","average":{"1min":0.75700000000000001,"5min":0.81000000000000005,"15min":0.84699999999999998},"min":{"1min":0.52400000000000002,"5min":0.52400000000000002,"15min":0.504},"max":{"1min":0.97299999999999998,"5min":1.079,"15min":1.401},"last":0.59799999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575042,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50380,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321460,"statfs":{"total":96636764160,"available":96585175040,"internally_reserved":0,"allocated":1314816,"data_stored":614255,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.40200000000000002,"5min":0.43099999999999999,"15min":0.44400000000000001},"min":{"1min":0.27300000000000002,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.57299999999999995,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.23400000000000001},{"interface":"front","average":{"1min":0.46899999999999997,"5min":0.47499999999999998,"15min":0.49099999999999999},"min":{"1min":0.30499999999999999,"5min":0.29699999999999999,"15min":0.28599999999999998},"max":{"1min":0.75,"5min":0.76600000000000001,"15min":0.92100000000000004},"last":0.35599999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.46899999999999997,"5min":0.5,"15min":0.51500000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.63200000000000001,"5min":0.81399999999999995,"15min":1.091},"last":0.29199999999999998},{"interface":"front","average":{"1min":0.499,"5min":0.51400000000000001,"15min":0.54300000000000004},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.33900000000000002},"max":{"1min":0.84699999999999998,"5min":0.85899999999999999,"15min":0.91300000000000003},"last":0.40000000000000002}]},{"osd":3,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.58899999999999997,"5min":0.58199999999999996,"15min":0.59499999999999997},"min":{"1min":0.43099999999999999,"5min":0.40100000000000002,"15min":0.36599999999999999},"max":{"1min":0.91000000000000003,"5min":0.93600000000000005,"15min":1.016},"last":0.442},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.51100000000000001,"15min":0.52800000000000002},"min":{"1min":0.38500000000000001,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.70699999999999996,"5min":0.82399999999999995,"15min":1.2410000000000001},"last":0.41699999999999998}]},{"osd":4,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.72099999999999997,"15min":0.76400000000000001},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.92900000000000005,"5min":1.1579999999999999,"15min":1.4419999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75,"15min":0.78300000000000003},"min":{"1min":0.51600000000000001,"5min":0.434,"15min":0.434},"max":{"1min":1.1819999999999999,"5min":1.1819999999999999,"15min":1.3380000000000001},"last":0.505}]},{"osd":5,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.754,"15min":0.77300000000000002},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.94899999999999995,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.73699999999999999,"15min":0.78400000000000003},"min":{"1min":0.48399999999999999,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.86699999999999999,"5min":1.2450000000000001,"15min":1.3009999999999999},"last":0.52500000000000002}]},{"osd":6,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.76300000000000001,"15min":0.79200000000000004},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":1.232,"5min":1.232,"15min":1.415},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.752,"15min":0.78600000000000003},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.89200000000000002,"5min":1.252,"15min":1.268},"last":0.45800000000000002}]},{"osd":7,"last update":"Wed Dec 20 04:44:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.79900000000000004,"15min":0.83299999999999996},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.028,"5min":1.306,"15min":1.399},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.79500000000000004,"15min":0.83299999999999996},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.97899999999999998,"5min":1.0980000000000001,"15min":1.373},"last":0.58699999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575041,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28820,"kb_used_data":1868,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94343020,"statfs":{"total":96636764160,"available":96607252480,"internally_reserved":0,"allocated":1912832,"data_stored":1210763,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.46700000000000003,"15min":0.44600000000000001},"min":{"1min":0.23499999999999999,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":0.83699999999999997,"5min":2.5640000000000001,"15min":2.5640000000000001},"last":0.32200000000000001},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.59599999999999997,"15min":0.53600000000000003},"min":{"1min":0.27600000000000002,"5min":0.27600000000000002,"15min":0.26900000000000002},"max":{"1min":0.70099999999999996,"5min":7.8860000000000001,"15min":7.8860000000000001},"last":0.376}]},{"osd":2,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.44600000000000001,"5min":0.53700000000000003,"15min":0.52100000000000002},"min":{"1min":0.29699999999999999,"5min":0.29699999999999999,"15min":0.28399999999999997},"max":{"1min":0.75600000000000001,"5min":2.528,"15min":2.528},"last":0.441},{"interface":"front","average":{"1min":0.49099999999999999,"5min":0.54100000000000004,"15min":0.55100000000000005},"min":{"1min":0.318,"5min":0.318,"15min":0.318},"max":{"1min":0.71399999999999997,"5min":0.83499999999999996,"15min":1.2989999999999999},"last":0.41799999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.50800000000000001,"5min":0.54900000000000004,"15min":0.52400000000000002},"min":{"1min":0.34100000000000003,"5min":0.34100000000000003,"15min":0.29499999999999998},"max":{"1min":0.80000000000000004,"5min":3.1000000000000001,"15min":3.1000000000000001},"last":0.46100000000000002},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.68799999999999994,"15min":0.61899999999999999},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.70899999999999996,"5min":10.49,"15min":10.49},"last":0.48699999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.78600000000000003,"15min":0.79300000000000004},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0780000000000001,"5min":3.0659999999999998,"15min":3.7469999999999999},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.68200000000000005,"5min":0.83199999999999996,"15min":0.80300000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.181,"5min":5.2460000000000004,"15min":5.2460000000000004},"last":0.78900000000000003}]},{"osd":5,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.79200000000000004,"15min":0.79800000000000004},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.44700000000000001},"max":{"1min":1.161,"5min":1.3109999999999999,"15min":1.877},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.79100000000000004,"15min":0.79700000000000004},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.0349999999999999,"5min":1.2010000000000001,"15min":1.7450000000000001},"last":0.81999999999999995}]},{"osd":6,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.84499999999999997,"15min":0.81899999999999995},"min":{"1min":0.502,"5min":0.502,"15min":0.47299999999999998},"max":{"1min":1.0069999999999999,"5min":5.2839999999999998,"15min":5.2839999999999998},"last":0.754},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.80500000000000005,"15min":0.81299999999999994},"min":{"1min":0.52800000000000002,"5min":0.51000000000000001,"15min":0.51000000000000001},"max":{"1min":1.2150000000000001,"5min":1.306,"15min":1.492},"last":0.84399999999999997}]},{"osd":7,"last update":"Wed Dec 20 04:43:56 2023","interfaces":[{"interface":"back","average":{"1min":0.73399999999999999,"5min":0.81699999999999995,"15min":0.82899999999999996},"min":{"1min":0.55600000000000005,"5min":0.52800000000000002,"15min":0.52800000000000002},"max":{"1min":1.0740000000000001,"5min":1.2549999999999999,"15min":1.6539999999999999},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.81200000000000006,"15min":0.81299999999999994},"min":{"1min":0.48999999999999999,"5min":0.48999999999999999,"15min":0.48799999999999999},"max":{"1min":1.01,"5min":1.276,"15min":2.113},"last":0.76800000000000002}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":12,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":13,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:44:32.321 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T04:44:32.321 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T04:44:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2960b7b640 1 Processor -- start 2023-12-20T04:44:32.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2960b7b640 1 -- start start 2023-12-20T04:44:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2960b7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0adf00 0x7f295c0cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2960b7b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f295c06e150 0x7f295c0cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2960b7b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f295c159e60 con 0x7f295c0c4320 2023-12-20T04:44:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2960b7b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f295c0c3630 con 0x7f295c06e150 2023-12-20T04:44:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2960b7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c0ce380 con 0x7f295c0adf00 2023-12-20T04:44:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2953fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f295c0c4320 0x7f295c06da40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49112/0 (socket says 172.21.15.32:49112) 2023-12-20T04:44:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2953fff640 1 -- 172.21.15.32:0/1956029039 learned_addr learned my addr 172.21.15.32:0/1956029039 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f295a59c640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0adf00 0x7f295c0cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f295ad9d640 1 --2- 172.21.15.32:0/1956029039 >> v2:172.21.15.39:3300/0 conn(0x7f295c06e150 0x7f295c0cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3593019151 0 0) 0x7f295c159e60 con 0x7f295c0c4320 2023-12-20T04:44:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2948003530 con 0x7f295c0c4320 2023-12-20T04:44:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3909862322 0 0) 0x7f2948003530 con 0x7f295c0c4320 2023-12-20T04:44:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0adf00 msgr2=0x7f295c0cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0adf00 0x7f295c0cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 >> v2:172.21.15.39:3300/0 conn(0x7f295c06e150 msgr2=0x7f295c0cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 --2- 172.21.15.32:0/1956029039 >> v2:172.21.15.39:3300/0 conn(0x7f295c06e150 0x7f295c0cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c0c3840 con 0x7f295c0c4320 2023-12-20T04:44:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f2944001680 con 0x7f295c0c4320 2023-12-20T04:44:32.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f295ad9d640 1 --2- 172.21.15.32:0/1956029039 >> v2:172.21.15.39:3300/0 conn(0x7f295c06e150 0x7f295c0cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 >> v1:172.21.15.32:6789/0 conn(0x7f295c0c4320 legacy=0x7f295c06da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0adf00 0x7f2948003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 0x7f29480047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2948004d10 0x7f2948005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c0c3630 con 0x7f2948003f00 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c0ce380 con 0x7f2948004d10 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f294800b990 con 0x7f295c0adf00 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f2953fff640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 0x7f29480047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f295ad9d640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2948004d10 0x7f2948005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.509+0000 7f295a59c640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0adf00 0x7f2948003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2953fff640 1 -- 172.21.15.32:0/1956029039 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0adf00 msgr2=0x7f2948003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2953fff640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0adf00 0x7f2948003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2953fff640 1 -- 172.21.15.32:0/1956029039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2948004d10 msgr2=0x7f2948005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:32.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2953fff640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2948004d10 0x7f2948005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2953fff640 1 -- 172.21.15.32:0/1956029039 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2948003530 con 0x7f2948003f00 2023-12-20T04:44:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2953fff640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 0x7f29480047d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f294400c240 tx=0x7f294401f370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7674347fb4766ec0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944005ae0 con 0x7f2948003f00 2023-12-20T04:44:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29440ab070 con 0x7f2948003f00 2023-12-20T04:44:32.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2959d9b640 1 -- 172.21.15.32:0/1956029039 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2944007070 con 0x7f2948003f00 2023-12-20T04:44:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2960b7b640 1 -- 172.21.15.32:0/1956029039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 msgr2=0x7f29480047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2960b7b640 1 --2- 172.21.15.32:0/1956029039 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 0x7f29480047d0 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x7f294400c240 tx=0x7f294401f370 comp rx=0 tx=0).stop 2023-12-20T04:44:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f295ad9d640 1 -- 172.21.15.32:0/1956029039 reap_dead start 2023-12-20T04:44:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2960b7b640 1 -- 172.21.15.32:0/1956029039 shutdown_connections 2023-12-20T04:44:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2960b7b640 1 -- 172.21.15.32:0/1956029039 >> 172.21.15.32:0/1956029039 conn(0x7f295c06a4b0 msgr2=0x7f295c0b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.513+0000 7f2960b7b640 1 -- 172.21.15.32:0/1956029039 shutdown_connections 2023-12-20T04:44:32.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f2960b7b640 1 -- 172.21.15.32:0/1956029039 wait complete. 2023-12-20T04:44:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f2960b7b640 1 Processor -- start 2023-12-20T04:44:32.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f2960b7b640 1 -- start start 2023-12-20T04:44:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f2960b7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 0x7f295c0ab8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f2960b7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c0ad1a0 0x7f295c1f2570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f2960b7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0707e0 0x7f295c1f4f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f2960b7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c0c68d0 con 0x7f2948003f00 2023-12-20T04:44:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f2960b7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f295c0ad620 con 0x7f295c0ad1a0 2023-12-20T04:44:32.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f2960b7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f295c0abfd0 con 0x7f295c0707e0 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f295a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 0x7f295c0ab8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f295a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 0x7f295c0ab8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60558/0 (socket says 172.21.15.32:60558) 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f295a59c640 1 -- 172.21.15.32:0/2082243639 learned_addr learned my addr 172.21.15.32:0/2082243639 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f295ad9d640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0707e0 0x7f295c1f4f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f2953fff640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c0ad1a0 0x7f295c1f2570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f295a59c640 1 -- 172.21.15.32:0/2082243639 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0707e0 msgr2=0x7f295c1f4f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f295a59c640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0707e0 0x7f295c1f4f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f295a59c640 1 -- 172.21.15.32:0/2082243639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c0ad1a0 msgr2=0x7f295c1f2570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f295a59c640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c0ad1a0 0x7f295c1f2570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f295a59c640 1 -- 172.21.15.32:0/2082243639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f295c0ac2b0 con 0x7f2948003f00 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.517+0000 7f295a59c640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 0x7f295c0ab8d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f294c0119d0 tx=0x7f294c011e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.521+0000 7f2952ffd640 1 -- 172.21.15.32:0/2082243639 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c00fb50 con 0x7f2948003f00 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.521+0000 7f2952ffd640 1 -- 172.21.15.32:0/2082243639 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f294c00fcb0 con 0x7f2948003f00 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.521+0000 7f2960b7b640 1 -- 172.21.15.32:0/2082243639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f295c0ac590 con 0x7f2948003f00 2023-12-20T04:44:32.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.521+0000 7f2952ffd640 1 -- 172.21.15.32:0/2082243639 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f294c014440 con 0x7f2948003f00 2023-12-20T04:44:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.521+0000 7f2960b7b640 1 -- 172.21.15.32:0/2082243639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f295c1f2ab0 con 0x7f2948003f00 2023-12-20T04:44:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.521+0000 7f2960b7b640 1 -- 172.21.15.32:0/2082243639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f295c04e000 con 0x7f2948003f00 2023-12-20T04:44:32.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.521+0000 7f2952ffd640 1 -- 172.21.15.32:0/2082243639 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f294c01d400 con 0x7f2948003f00 2023-12-20T04:44:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.521+0000 7f2952ffd640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f293c065b80 0x7f293c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.521+0000 7f2952ffd640 1 -- 172.21.15.32:0/2082243639 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f294c083b70 con 0x7f2948003f00 2023-12-20T04:44:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.521+0000 7f2953fff640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f293c065b80 0x7f293c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.521+0000 7f2953fff640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f293c065b80 0x7f293c068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29440ae000 tx=0x7f2944001a80 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:32.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.525+0000 7f2952ffd640 1 -- 172.21.15.32:0/2082243639 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f294c0514d0 con 0x7f2948003f00 2023-12-20T04:44:32.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.737+0000 7f2960b7b640 1 -- 172.21.15.32:0/2082243639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f295c070060 con 0x7f2948003f00 2023-12-20T04:44:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.741+0000 7f2952ffd640 1 -- 172.21.15.32:0/2082243639 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v20) v1 ==== 190+0+0 (secure 0 0 0) 0x7f294c054af0 con 0x7f2948003f00 2023-12-20T04:44:32.743 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T04:44:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 -- 172.21.15.32:0/2082243639 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f293c065b80 msgr2=0x7f293c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f293c065b80 0x7f293c068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f29440ae000 tx=0x7f2944001a80 comp rx=0 tx=0).stop 2023-12-20T04:44:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 -- 172.21.15.32:0/2082243639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 msgr2=0x7f295c0ab8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 0x7f295c0ab8d0 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7f294c0119d0 tx=0x7f294c011e90 comp rx=0 tx=0).stop 2023-12-20T04:44:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 -- 172.21.15.32:0/2082243639 shutdown_connections 2023-12-20T04:44:32.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f293c065b80 0x7f293c068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f295c0707e0 0x7f295c1f4f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f295c0ad1a0 0x7f295c1f2570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 --2- 172.21.15.32:0/2082243639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2948003f00 0x7f295c0ab8d0 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 -- 172.21.15.32:0/2082243639 >> 172.21.15.32:0/2082243639 conn(0x7f295c06a4b0 msgr2=0x7f295c0c9f00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 -- 172.21.15.32:0/2082243639 shutdown_connections 2023-12-20T04:44:32.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.745+0000 7f2960b7b640 1 -- 172.21.15.32:0/2082243639 wait complete. 2023-12-20T04:44:32.765 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:44:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.909+0000 7f3e2982d640 1 Processor -- start 2023-12-20T04:44:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.909+0000 7f3e2982d640 1 -- start start 2023-12-20T04:44:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e2982d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e2406deb0 0x7f3e24161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e2982d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3e24070950 0x7f3e240cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e2982d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3e24071920 con 0x7f3e2415ce30 2023-12-20T04:44:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e2982d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e24070de0 con 0x7f3e2406deb0 2023-12-20T04:44:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e2982d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3e24071080 con 0x7f3e24070950 2023-12-20T04:44:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e22ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3e2415ce30 0x7f3e2406d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49128/0 (socket says 172.21.15.32:49128) 2023-12-20T04:44:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e1bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3e2406deb0 0x7f3e24161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e22ffd640 1 -- 172.21.15.32:0/2373835838 learned_addr learned my addr 172.21.15.32:0/2373835838 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e1bfff640 1 -- 172.21.15.32:0/2373835838 >> v2:172.21.15.39:3300/0 conn(0x7f3e24070950 msgr2=0x7f3e240cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e1bfff640 1 --2- 172.21.15.32:0/2373835838 >> v2:172.21.15.39:3300/0 conn(0x7f3e24070950 0x7f3e240cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e1bfff640 1 -- 172.21.15.32:0/2373835838 >> v1:172.21.15.32:6789/0 conn(0x7f3e2415ce30 legacy=0x7f3e2406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e1bfff640 1 -- 172.21.15.32:0/2373835838 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e24071360 con 0x7f3e2406deb0 2023-12-20T04:44:32.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.913+0000 7f3e227fc640 1 -- 172.21.15.32:0/2373835838 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2806976716 0 0) 0x7f3e24071920 con 0x7f3e2415ce30 2023-12-20T04:44:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e1bfff640 1 --2- 172.21.15.32:0/2373835838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2406deb0 0x7f3e24161560 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3e240c95b0 tx=0x7f3e0c01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=24458a5791e56c4c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e227fc640 1 -- 172.21.15.32:0/2373835838 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c004230 con 0x7f3e2406deb0 2023-12-20T04:44:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e227fc640 1 -- 172.21.15.32:0/2373835838 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c0a7050 con 0x7f3e2406deb0 2023-12-20T04:44:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e227fc640 1 -- 172.21.15.32:0/2373835838 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c003620 con 0x7f3e2406deb0 2023-12-20T04:44:32.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 -- 172.21.15.32:0/2373835838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2406deb0 msgr2=0x7f3e24161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 --2- 172.21.15.32:0/2373835838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2406deb0 0x7f3e24161560 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7f3e240c95b0 tx=0x7f3e0c01c460 comp rx=0 tx=0).stop 2023-12-20T04:44:32.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 -- 172.21.15.32:0/2373835838 shutdown_connections 2023-12-20T04:44:32.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 --2- 172.21.15.32:0/2373835838 >> v2:172.21.15.39:3300/0 conn(0x7f3e24070950 0x7f3e240cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 --2- 172.21.15.32:0/2373835838 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e2406deb0 0x7f3e24161560 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 -- 172.21.15.32:0/2373835838 >> 172.21.15.32:0/2373835838 conn(0x7f3e240cba50 msgr2=0x7f3e240cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:32.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 -- 172.21.15.32:0/2373835838 shutdown_connections 2023-12-20T04:44:32.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 -- 172.21.15.32:0/2373835838 wait complete. 2023-12-20T04:44:32.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 Processor -- start 2023-12-20T04:44:32.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 -- start start 2023-12-20T04:44:32.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e2406deb0 0x7f3e24154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e24070950 0x7f3e240c5580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e2415ce30 0x7f3e240c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e2409dcc0 con 0x7f3e24070950 2023-12-20T04:44:32.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240c5ac0 con 0x7f3e2415ce30 2023-12-20T04:44:32.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.917+0000 7f3e2982d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3e240c2ae0 con 0x7f3e2406deb0 2023-12-20T04:44:32.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e1bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e24070950 0x7f3e240c5580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e1bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e24070950 0x7f3e240c5580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60580/0 (socket says 172.21.15.32:60580) 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e1bfff640 1 -- 172.21.15.32:0/1356419801 learned_addr learned my addr 172.21.15.32:0/1356419801 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e237fe640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e2415ce30 0x7f3e240c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e22ffd640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e2406deb0 0x7f3e24154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e1bfff640 1 -- 172.21.15.32:0/1356419801 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e2406deb0 msgr2=0x7f3e24154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e1bfff640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e2406deb0 0x7f3e24154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e1bfff640 1 -- 172.21.15.32:0/1356419801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e2415ce30 msgr2=0x7f3e240c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e1bfff640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e2415ce30 0x7f3e240c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e1bfff640 1 -- 172.21.15.32:0/1356419801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3e0c009420 con 0x7f3e24070950 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e1bfff640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e24070950 0x7f3e240c5580 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c019070 tx=0x7f3e0c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e20ff9640 1 -- 172.21.15.32:0/1356419801 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c003cf0 con 0x7f3e24070950 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e20ff9640 1 -- 172.21.15.32:0/1356419801 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3e0c003e50 con 0x7f3e24070950 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e2982d640 1 -- 172.21.15.32:0/1356419801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3e240c2dc0 con 0x7f3e24070950 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e20ff9640 1 -- 172.21.15.32:0/1356419801 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3e0c0ab660 con 0x7f3e24070950 2023-12-20T04:44:32.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e2982d640 1 -- 172.21.15.32:0/1356419801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3e240c3380 con 0x7f3e24070950 2023-12-20T04:44:32.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.921+0000 7f3e2982d640 1 -- 172.21.15.32:0/1356419801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3e2404e000 con 0x7f3e24070950 2023-12-20T04:44:32.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.925+0000 7f3e20ff9640 1 -- 172.21.15.32:0/1356419801 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3e0c0b2040 con 0x7f3e24070950 2023-12-20T04:44:32.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.925+0000 7f3e20ff9640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e04065bb0 0x7f3e04068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:32.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.925+0000 7f3e20ff9640 1 -- 172.21.15.32:0/1356419801 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3e0c11df30 con 0x7f3e24070950 2023-12-20T04:44:32.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.925+0000 7f3e22ffd640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e04065bb0 0x7f3e04068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:32.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.925+0000 7f3e22ffd640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e04065bb0 0x7f3e04068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e14000f70 tx=0x7f3e14007c10 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:32.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:32.925+0000 7f3e20ff9640 1 -- 172.21.15.32:0/1356419801 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3e0c1490a0 con 0x7f3e24070950 2023-12-20T04:44:33.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e2982d640 1 -- 172.21.15.32:0/1356419801 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3e241603d0 con 0x7f3e24070950 2023-12-20T04:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e20ff9640 1 -- 172.21.15.32:0/1356419801 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f3e0c0b0090 con 0x7f3e24070950 2023-12-20T04:44:33.121 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:33.121 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1072056415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1072056415},{"type":"v1","addr":"172.21.15.39:6835","nonce":1072056415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3512326822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3512326822},{"type":"v1","addr":"172.21.15.32:6836","nonce":3512326822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2163601073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2163601073},{"type":"v1","addr":"172.21.15.32:6837","nonce":2163601073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:44:27.818499+0000","modified":"2023-12-20T04:44:30.831088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7359":{"gid":7359,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/2369052657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2369052657},{"type":"v1","addr":"172.21.15.39:6833","nonce":2369052657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-20T04:44:33.121 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 46 2023-12-20T04:44:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e2982d640 1 -- 172.21.15.32:0/1356419801 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e04065bb0 msgr2=0x7f3e04068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e2982d640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e04065bb0 0x7f3e04068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3e14000f70 tx=0x7f3e14007c10 comp rx=0 tx=0).stop 2023-12-20T04:44:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e2982d640 1 -- 172.21.15.32:0/1356419801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e24070950 msgr2=0x7f3e240c5580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e2982d640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e24070950 0x7f3e240c5580 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f3e0c019070 tx=0x7f3e0c019200 comp rx=0 tx=0).stop 2023-12-20T04:44:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e2982d640 1 -- 172.21.15.32:0/1356419801 shutdown_connections 2023-12-20T04:44:33.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e2982d640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3e04065bb0 0x7f3e04068070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e2982d640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3e2415ce30 0x7f3e240c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e2982d640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3e24070950 0x7f3e240c5580 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e2982d640 1 --2- 172.21.15.32:0/1356419801 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3e2406deb0 0x7f3e24154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.121+0000 7f3e2982d640 1 -- 172.21.15.32:0/1356419801 >> 172.21.15.32:0/1356419801 conn(0x7f3e240cba50 msgr2=0x7f3e2415e9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.125+0000 7f3e2982d640 1 -- 172.21.15.32:0/1356419801 shutdown_connections 2023-12-20T04:44:33.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.125+0000 7f3e2982d640 1 -- 172.21.15.32:0/1356419801 wait complete. 2023-12-20T04:44:33.546 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 46, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T04:44:27.818499+0000', 'modified': '2023-12-20T04:44:30.831088+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7359}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7359': {'gid': 7359, 'name': 'd', 'rank': 0, 'incarnation': 43, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.39:6833/2369052657', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 2369052657}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 2369052657}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [13], 'metadata_pool': 12, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T04:44:33.546 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T04:44:33.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-20T04:44:33.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.693+0000 7fb6d2e2d640 1 Processor -- start 2023-12-20T04:44:33.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6d2e2d640 1 -- start start 2023-12-20T04:44:33.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6d2e2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6cc1599e0 0x7fb6cc0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6d2e2d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb6cc0c90e0 0x7fb6cc0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6d2e2d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6cc0711d0 con 0x7fb6cc0b0b30 2023-12-20T04:44:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6d2e2d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6cc0c98c0 con 0x7fb6cc0c90e0 2023-12-20T04:44:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6d2e2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc0c9ba0 con 0x7fb6cc1599e0 2023-12-20T04:44:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6d13ca640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb6cc0b0b30 0x7fb6cc15d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49130/0 (socket says 172.21.15.32:49130) 2023-12-20T04:44:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6d13ca640 1 -- 172.21.15.32:0/4192530997 learned_addr learned my addr 172.21.15.32:0/4192530997 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:33.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/4192530997 >> v2:172.21.15.32:3300/0 conn(0x7fb6cc0c90e0 0x7fb6cc0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/4192530997 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6cc1599e0 0x7fb6cc0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6cb7fe640 1 -- 172.21.15.32:0/4192530997 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 874300494 0 0) 0x7fb6cc0711d0 con 0x7fb6cc0b0b30 2023-12-20T04:44:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6cb7fe640 1 -- 172.21.15.32:0/4192530997 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6bc003530 con 0x7fb6cc0b0b30 2023-12-20T04:44:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6cbfff640 1 -- 172.21.15.32:0/4192530997 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6cc1599e0 msgr2=0x7fb6cc0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/4192530997 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6cc1599e0 0x7fb6cc0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6cbfff640 1 -- 172.21.15.32:0/4192530997 >> v1:172.21.15.32:6789/0 conn(0x7fb6cc0b0b30 legacy=0x7fb6cc15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.697+0000 7fb6cbfff640 1 -- 172.21.15.32:0/4192530997 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6cc0c9e80 con 0x7fb6cc0c90e0 2023-12-20T04:44:33.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/4192530997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc0c90e0 0x7fb6cc0b05f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80097c0 tx=0x7fb6b801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a125fbbf04f49ed server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:33.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6cb7fe640 1 -- 172.21.15.32:0/4192530997 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b80ab600 con 0x7fb6cc0c90e0 2023-12-20T04:44:33.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6cb7fe640 1 -- 172.21.15.32:0/4192530997 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6b80a7050 con 0x7fb6cc0c90e0 2023-12-20T04:44:33.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6cb7fe640 1 -- 172.21.15.32:0/4192530997 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b80ab960 con 0x7fb6cc0c90e0 2023-12-20T04:44:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/4192530997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc0c90e0 msgr2=0x7fb6cc0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6d2e2d640 1 --2- 172.21.15.32:0/4192530997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc0c90e0 0x7fb6cc0b05f0 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fb6b80097c0 tx=0x7fb6b801c450 comp rx=0 tx=0).stop 2023-12-20T04:44:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/4192530997 shutdown_connections 2023-12-20T04:44:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6d2e2d640 1 --2- 172.21.15.32:0/4192530997 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc0c90e0 0x7fb6cc0b05f0 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6d2e2d640 1 --2- 172.21.15.32:0/4192530997 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6cc1599e0 0x7fb6cc0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/4192530997 >> 172.21.15.32:0/4192530997 conn(0x7fb6cc06a320 msgr2=0x7fb6cc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:33.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/4192530997 shutdown_connections 2023-12-20T04:44:33.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/4192530997 wait complete. 2023-12-20T04:44:33.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6d2e2d640 1 Processor -- start 2023-12-20T04:44:33.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.701+0000 7fb6d2e2d640 1 -- start start 2023-12-20T04:44:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d2e2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6cc0b0b30 0x7fb6cc0ce4e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d2e2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6cc0c90e0 0x7fb6cc147e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d2e2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc1599e0 0x7fb6cc0b35c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d2e2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc1589f0 con 0x7fb6cc1599e0 2023-12-20T04:44:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d2e2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc162ba0 con 0x7fb6cc0c90e0 2023-12-20T04:44:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d2e2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6cc158c90 con 0x7fb6cc0b0b30 2023-12-20T04:44:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d13ca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc1599e0 0x7fb6cc0b35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d13ca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc1599e0 0x7fb6cc0b35c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60598/0 (socket says 172.21.15.32:60598) 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d13ca640 1 -- 172.21.15.32:0/3819144183 learned_addr learned my addr 172.21.15.32:0/3819144183 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6cc0b0b30 0x7fb6cc0ce4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6cc0c90e0 0x7fb6cc147e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d13ca640 1 -- 172.21.15.32:0/3819144183 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6cc0b0b30 msgr2=0x7fb6cc0ce4e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d13ca640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6cc0b0b30 0x7fb6cc0ce4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d13ca640 1 -- 172.21.15.32:0/3819144183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6cc0c90e0 msgr2=0x7fb6cc147e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d13ca640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6cc0c90e0 0x7fb6cc147e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d13ca640 1 -- 172.21.15.32:0/3819144183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6b8009420 con 0x7fb6cc1599e0 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d13ca640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc1599e0 0x7fb6cc0b35c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400cb40 tx=0x7fb6c400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6c97fa640 1 -- 172.21.15.32:0/3819144183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c400aca0 con 0x7fb6cc1599e0 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6c97fa640 1 -- 172.21.15.32:0/3819144183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c400ae00 con 0x7fb6cc1599e0 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6c97fa640 1 -- 172.21.15.32:0/3819144183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c4012620 con 0x7fb6cc1599e0 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/3819144183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6cc162e40 con 0x7fb6cc1599e0 2023-12-20T04:44:33.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.705+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/3819144183 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6cc158f70 con 0x7fb6cc1599e0 2023-12-20T04:44:33.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6c97fa640 1 -- 172.21.15.32:0/3819144183 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb6c4007070 con 0x7fb6cc1599e0 2023-12-20T04:44:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6c97fa640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6b0065b80 0x7fb6b0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6c97fa640 1 -- 172.21.15.32:0/3819144183 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6c4083400 con 0x7fb6cc1599e0 2023-12-20T04:44:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6b0065b80 0x7fb6b0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d619f000 1 Processor -- start 2023-12-20T04:44:33.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6b0065b80 0x7fb6b0068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0006fd0 tx=0x7fb6c0008040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:33.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d619f000 1 -- start start 2023-12-20T04:44:33.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c070 0x564dd310c450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c990 0x564dd30c1c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd30c21c0 0x564dd30c25a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d619f000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dd2ee5c90 con 0x564dd310c990 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d619f000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dd2f17220 con 0x564dd310c070 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d619f000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dd2f147f0 con 0x564dd30c21c0 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c990 0x564dd30c1c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d0bc9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c070 0x564dd310c450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c990 0x564dd30c1c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60614/0 (socket says 172.21.15.32:60614) 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6cbfff640 1 -- 172.21.15.32:0/258203743 learned_addr learned my addr 172.21.15.32:0/258203743 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d13ca640 1 --2- 172.21.15.32:0/258203743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd30c21c0 0x564dd30c25a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d0bc9640 1 -- 172.21.15.32:0/258203743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd30c21c0 msgr2=0x564dd30c25a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/258203743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd30c21c0 0x564dd30c25a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d0bc9640 1 -- 172.21.15.32:0/258203743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c990 msgr2=0x564dd30c1c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:33.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/258203743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c990 0x564dd30c1c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d0bc9640 1 -- 172.21.15.32:0/258203743 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x564dd2c0c890 con 0x564dd310c070 2023-12-20T04:44:33.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/258203743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c990 0x564dd30c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:33.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d13ca640 1 --2- 172.21.15.32:0/258203743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd30c21c0 0x564dd30c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:33.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/258203743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c070 0x564dd310c450 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000f5b0 tx=0x7fb6c0012000 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2adbc1f957c96de2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:33.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6b67fc640 1 -- 172.21.15.32:0/258203743 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0024070 con 0x564dd310c070 2023-12-20T04:44:33.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.709+0000 7fb6b67fc640 1 -- 172.21.15.32:0/258203743 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c0019070 con 0x564dd310c070 2023-12-20T04:44:33.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6b67fc640 1 -- 172.21.15.32:0/258203743 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0014be0 con 0x564dd310c070 2023-12-20T04:44:33.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 -- 172.21.15.32:0/258203743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c070 msgr2=0x564dd310c450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 --2- 172.21.15.32:0/258203743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c070 0x564dd310c450 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7fb6c000f5b0 tx=0x7fb6c0012000 comp rx=0 tx=0).stop 2023-12-20T04:44:33.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 -- 172.21.15.32:0/258203743 shutdown_connections 2023-12-20T04:44:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 --2- 172.21.15.32:0/258203743 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd30c21c0 0x564dd30c25a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 --2- 172.21.15.32:0/258203743 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c990 0x564dd30c1c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 --2- 172.21.15.32:0/258203743 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c070 0x564dd310c450 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 -- 172.21.15.32:0/258203743 >> 172.21.15.32:0/258203743 conn(0x564dd30c8950 msgr2=0x564dd30c0560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 -- 172.21.15.32:0/258203743 shutdown_connections 2023-12-20T04:44:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 -- 172.21.15.32:0/258203743 wait complete. 2023-12-20T04:44:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 -- start start 2023-12-20T04:44:33.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 Processor -- start 2023-12-20T04:44:33.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd30c7780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c070 0x564dd30ce080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c990 0x564dd30ce5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dd2f100e0 con 0x564dd30c21c0 2023-12-20T04:44:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dd2eaf310 con 0x564dd310c990 2023-12-20T04:44:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dd2ecd770 con 0x564dd310c070 2023-12-20T04:44:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 10 client.0 _ll_register_callbacks cb 0x564dd30adb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:44:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d0bc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd30c7780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d0bc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd30c7780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60620/0 (socket says 172.21.15.32:60620) 2023-12-20T04:44:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d0bc9640 1 -- 172.21.15.32:0/3529286186 learned_addr learned my addr 172.21.15.32:0/3529286186 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c070 0x564dd30ce080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d13ca640 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c990 0x564dd30ce5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d0bc9640 1 -- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c070 msgr2=0x564dd30ce080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c070 0x564dd30ce080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d0bc9640 1 -- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c990 msgr2=0x564dd30ce5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c990 0x564dd30ce5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d0bc9640 1 -- 172.21.15.32:0/3529286186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6cc158f70 con 0x564dd30c21c0 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd30c7780 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x564dd2eb5a30 tx=0x7fb6c0004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d0707447cc293ea server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/3529286186 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c0024070 con 0x564dd30c21c0 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6d619f000 1 -- 172.21.15.32:0/3529286186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564dd2edfc10 con 0x564dd30c21c0 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.713+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/3529286186 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6c0005db0 con 0x564dd30c21c0 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/3529286186 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6c0006020 con 0x564dd30c21c0 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 10 client.7500 fetch_fsmap learned FSMap version 46 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 -- 172.21.15.32:0/3529286186 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x7fb6cc0c9e80 con 0x564dd30c21c0 2023-12-20T04:44:33.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/3529286186 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564dd2edfc10 con 0x564dd30c21c0 2023-12-20T04:44:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/3529286186 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb6c0006180 con 0x564dd30c21c0 2023-12-20T04:44:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 10 client.7500 fetch_fsmap finished waiting for FSMap version 46 2023-12-20T04:44:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 10 client.7500 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-20T04:44:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 20 client.7500 populate_metadata read hostname 'smithi032' 2023-12-20T04:44:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:6832/2369052657,v1:172.21.15.39:6833/2369052657] conn(0x564dd31138d0 0x564dd3113cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 4 client.7500 mds_command: new command op to 7359 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T04:44:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 -- 172.21.15.32:0/3529286186 --> [v2:172.21.15.39:6832/2369052657,v1:172.21.15.39:6833/2369052657] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x564dd2f0ea30 con 0x564dd31138d0 2023-12-20T04:44:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:6832/2369052657,v1:172.21.15.39:6833/2369052657] conn(0x564dd31138d0 0x564dd3113cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:6832/2369052657,v1:172.21.15.39:6833/2369052657] conn(0x564dd31138d0 0x564dd3113cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d2d2c7b24d2179d7 server_cookie=c471bfbc7e023bfb in_seq=0 out_seq=0 2023-12-20T04:44:33.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6ab7fe640 10 client.7500 ms_handle_connect on v2:172.21.15.39:6832/2369052657 2023-12-20T04:44:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/3529286186 <== mds.0 v2:172.21.15.39:6832/2369052657 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x564dd2f0ea30 con 0x564dd31138d0 2023-12-20T04:44:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6ab7fe640 10 client.7500 handle_command_reply: tid=0 2023-12-20T04:44:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 client.7500 shutdown 2023-12-20T04:44:33.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 -- 172.21.15.32:0/3529286186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 msgr2=0x564dd30c7780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd30c7780 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x564dd2eb5a30 tx=0x7fb6c0004740 comp rx=0 tx=0).stop 2023-12-20T04:44:33.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 -- 172.21.15.32:0/3529286186 shutdown_connections 2023-12-20T04:44:33.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:6832/2369052657,v1:172.21.15.39:6833/2369052657] conn(0x564dd31138d0 0x564dd3113cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c990 0x564dd30ce5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c070 0x564dd30ce080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 --2- 172.21.15.32:0/3529286186 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd30c7780 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6ab7fe640 0 client.7500 ms_handle_reset on v2:172.21.15.39:6832/2369052657 2023-12-20T04:44:33.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 -- 172.21.15.32:0/3529286186 >> 172.21.15.32:0/3529286186 conn(0x564dd30c8950 msgr2=0x564dd30c0c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:33.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 -- 172.21.15.32:0/3529286186 shutdown_connections 2023-12-20T04:44:33.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 1 -- 172.21.15.32:0/3529286186 wait complete. 2023-12-20T04:44:33.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.717+0000 7fb6d619f000 20 client.7500 trim_cache size 0 max 16384 2023-12-20T04:44:33.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d619f000 1 Processor -- start 2023-12-20T04:44:33.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d619f000 1 -- start start 2023-12-20T04:44:33.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd3163ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c070 0x564dd3164410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c990 0x564dd3164950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d619f000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dd2bcd940 con 0x564dd30c21c0 2023-12-20T04:44:33.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d619f000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dd2eb8860 con 0x564dd310c990 2023-12-20T04:44:33.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d619f000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dd30b99d0 con 0x564dd310c070 2023-12-20T04:44:33.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d0bc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd3163ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d0bc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd3163ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60632/0 (socket says 172.21.15.32:60632) 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d0bc9640 1 -- 172.21.15.32:0/814650985 learned_addr learned my addr 172.21.15.32:0/814650985 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d13ca640 1 --2- 172.21.15.32:0/814650985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c990 0x564dd3164950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/814650985 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c070 0x564dd3164410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d0bc9640 1 -- 172.21.15.32:0/814650985 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c070 msgr2=0x564dd3164410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/814650985 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c070 0x564dd3164410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d0bc9640 1 -- 172.21.15.32:0/814650985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c990 msgr2=0x564dd3164950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/814650985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c990 0x564dd3164950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.733+0000 7fb6d0bc9640 1 -- 172.21.15.32:0/814650985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6c000e040 con 0x564dd30c21c0 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/814650985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd3163ed0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0006b70 tx=0x7fb6c002a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97c2f28d207aaaf8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6b6ffd640 1 -- 172.21.15.32:0/814650985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c002a650 con 0x564dd30c21c0 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6b6ffd640 1 -- 172.21.15.32:0/814650985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6c0025040 con 0x564dd30c21c0 2023-12-20T04:44:33.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6b6ffd640 1 -- 172.21.15.32:0/814650985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6c002ab80 con 0x564dd30c21c0 2023-12-20T04:44:33.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 -- 172.21.15.32:0/814650985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 msgr2=0x564dd3163ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 --2- 172.21.15.32:0/814650985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd3163ed0 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0006b70 tx=0x7fb6c002a000 comp rx=0 tx=0).stop 2023-12-20T04:44:33.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 -- 172.21.15.32:0/814650985 shutdown_connections 2023-12-20T04:44:33.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 --2- 172.21.15.32:0/814650985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd310c990 0x564dd3164950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 --2- 172.21.15.32:0/814650985 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c070 0x564dd3164410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 --2- 172.21.15.32:0/814650985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd30c21c0 0x564dd3163ed0 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 -- 172.21.15.32:0/814650985 >> 172.21.15.32:0/814650985 conn(0x564dd30c8950 msgr2=0x564dd30cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:33.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 -- 172.21.15.32:0/814650985 shutdown_connections 2023-12-20T04:44:33.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 -- 172.21.15.32:0/814650985 wait complete. 2023-12-20T04:44:33.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 -- start start 2023-12-20T04:44:33.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 Processor -- start 2023-12-20T04:44:33.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd30c21c0 0x564dd314c940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c070 0x564dd314ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c990 0x564dd314d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dd30a5da0 con 0x564dd310c070 2023-12-20T04:44:33.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x564dd2f12c40 con 0x564dd30c21c0 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x564dd2f184b0 con 0x564dd310c990 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d619f000 10 client.0 _ll_register_callbacks cb 0x564dd2ebe640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c070 0x564dd314ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c070 0x564dd314ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60636/0 (socket says 172.21.15.32:60636) 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6cbfff640 1 -- 172.21.15.32:0/2570491287 learned_addr learned my addr 172.21.15.32:0/2570491287 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd30c21c0 0x564dd314c940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6d13ca640 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c990 0x564dd314d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6cbfff640 1 -- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c990 msgr2=0x564dd314d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c990 0x564dd314d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6cbfff640 1 -- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd30c21c0 msgr2=0x564dd314c940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd30c21c0 0x564dd314c940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6cbfff640 1 -- 172.21.15.32:0/2570491287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6cc0c9e80 con 0x564dd310c070 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6cbfff640 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c070 0x564dd314ce80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb6b801ab00 tx=0x7fb6b801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a134b0410f6ee856 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.737+0000 7fb6ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/2570491287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6b80b5a20 con 0x564dd310c070 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/2570491287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb6b80ba040 con 0x564dd310c070 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 -- 172.21.15.32:0/2570491287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x564dd2ed5e00 con 0x564dd310c070 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/2570491287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=46) v2 ==== 24+0+0 (secure 0 0 0) 0x7fb6b80be400 con 0x564dd310c070 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 10 client.7506 fetch_fsmap learned FSMap version 46 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 -- 172.21.15.32:0/2570491287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=46,monmap=2+}) v3 -- 0x564dd2ed5260 con 0x564dd310c070 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/2570491287 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x564dd2ed5e00 con 0x564dd310c070 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/2570491287 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 46) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fb6b80b5b80 con 0x564dd310c070 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 10 client.7506 fetch_fsmap finished waiting for FSMap version 46 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 10 client.7506 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-20T04:44:33.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 20 client.7506 populate_metadata read hostname 'smithi032' 2023-12-20T04:44:33.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:6832/2369052657,v1:172.21.15.39:6833/2369052657] conn(0x564dd314d900 0x564dd314dce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 4 client.7506 mds_command: new command op to 7359 tid=0[{"prefix": "status"}] 2023-12-20T04:44:33.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 -- 172.21.15.32:0/2570491287 --> [v2:172.21.15.39:6832/2369052657,v1:172.21.15.39:6833/2369052657] -- command(tid 0: {"prefix": "status"}) v1 -- 0x564dd2f09160 con 0x564dd314d900 2023-12-20T04:44:33.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:6832/2369052657,v1:172.21.15.39:6833/2369052657] conn(0x564dd314d900 0x564dd314dce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d0bc9640 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:6832/2369052657,v1:172.21.15.39:6833/2369052657] conn(0x564dd314d900 0x564dd314dce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=717a5378e614c89f server_cookie=e90340b68511d967 in_seq=0 out_seq=0 2023-12-20T04:44:33.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6ab7fe640 10 client.7506 ms_handle_connect on v2:172.21.15.39:6832/2369052657 2023-12-20T04:44:33.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6ab7fe640 1 -- 172.21.15.32:0/2570491287 <== mds.0 v2:172.21.15.39:6832/2369052657 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+326 (crc 0 0 0) 0x564dd2f09160 con 0x564dd314d900 2023-12-20T04:44:33.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6ab7fe640 10 client.7506 handle_command_reply: tid=0 2023-12-20T04:44:33.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 client.7506 shutdown 2023-12-20T04:44:33.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 -- 172.21.15.32:0/2570491287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c070 msgr2=0x564dd314ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c070 0x564dd314ce80 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7fb6b801ab00 tx=0x7fb6b801ac90 comp rx=0 tx=0).stop 2023-12-20T04:44:33.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 -- 172.21.15.32:0/2570491287 shutdown_connections 2023-12-20T04:44:33.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:6832/2369052657,v1:172.21.15.39:6833/2369052657] conn(0x564dd314d900 0x564dd314dce0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x564dd310c990 0x564dd314d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x564dd310c070 0x564dd314ce80 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 --2- 172.21.15.32:0/2570491287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x564dd30c21c0 0x564dd314c940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6ab7fe640 0 client.7506 ms_handle_reset on v2:172.21.15.39:6832/2369052657 2023-12-20T04:44:33.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 -- 172.21.15.32:0/2570491287 >> 172.21.15.32:0/2570491287 conn(0x564dd30c8950 msgr2=0x564dd3109060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:33.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 -- 172.21.15.32:0/2570491287 shutdown_connections 2023-12-20T04:44:33.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 1 -- 172.21.15.32:0/2570491287 wait complete. 2023-12-20T04:44:33.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d619f000 20 client.7506 trim_cache size 0 max 16384 2023-12-20T04:44:33.745 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:44:33.745 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T04:44:33.745 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T04:44:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 7359, 2023-12-20T04:44:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T04:44:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T04:44:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T04:44:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9245690120000001, 2023-12-20T04:44:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 46, 2023-12-20T04:44:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 107, 2023-12-20T04:44:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 107, 2023-12-20T04:44:33.746 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 11.16483113 2023-12-20T04:44:33.746 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:44:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/3819144183 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6b0065b80 msgr2=0x7fb6b0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6b0065b80 0x7fb6b0068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb6c0006fd0 tx=0x7fb6c0008040 comp rx=0 tx=0).stop 2023-12-20T04:44:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/3819144183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc1599e0 msgr2=0x7fb6cc0b35c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc1599e0 0x7fb6cc0b35c0 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7fb6c400cb40 tx=0x7fb6c400a060 comp rx=0 tx=0).stop 2023-12-20T04:44:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/3819144183 shutdown_connections 2023-12-20T04:44:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb6b0065b80 0x7fb6b0068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6cc1599e0 0x7fb6cc0b35c0 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6cc0c90e0 0x7fb6cc147e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 --2- 172.21.15.32:0/3819144183 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6cc0b0b30 0x7fb6cc0ce4e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/3819144183 >> 172.21.15.32:0/3819144183 conn(0x7fb6cc06a320 msgr2=0x7fb6cc071290 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:33.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/3819144183 shutdown_connections 2023-12-20T04:44:33.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.741+0000 7fb6d2e2d640 1 -- 172.21.15.32:0/3819144183 wait complete. 2023-12-20T04:44:33.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T04:44:33.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f53a6725640 1 Processor -- start 2023-12-20T04:44:33.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f53a6725640 1 -- start start 2023-12-20T04:44:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f53a6725640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53a006ce60 0x7f53a015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f53a6725640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f53a015f110 0x7f53a015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f53a6725640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53a015bbd0 con 0x7f53a00af5e0 2023-12-20T04:44:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f53a6725640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53a015beb0 con 0x7f53a015f110 2023-12-20T04:44:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f53a6725640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a015e910 con 0x7f53a006ce60 2023-12-20T04:44:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f539ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f53a00af5e0 0x7f53a00af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49138/0 (socket says 172.21.15.32:49138) 2023-12-20T04:44:33.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f539ffff640 1 -- 172.21.15.32:0/933364921 learned_addr learned my addr 172.21.15.32:0/933364921 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f53a4cc2640 1 --2- 172.21.15.32:0/933364921 >> v2:172.21.15.39:3300/0 conn(0x7f53a015f110 0x7f53a015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f539f7fe640 1 --2- 172.21.15.32:0/933364921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53a006ce60 0x7f53a015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f539effd640 1 -- 172.21.15.32:0/933364921 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 331753781 0 0) 0x7f53a015bbd0 con 0x7f53a00af5e0 2023-12-20T04:44:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.905+0000 7f539effd640 1 -- 172.21.15.32:0/933364921 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5388003530 con 0x7f53a00af5e0 2023-12-20T04:44:33.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f53a4cc2640 1 -- 172.21.15.32:0/933364921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53a006ce60 msgr2=0x7f53a015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f53a4cc2640 1 --2- 172.21.15.32:0/933364921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53a006ce60 0x7f53a015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f53a4cc2640 1 -- 172.21.15.32:0/933364921 >> v1:172.21.15.32:6789/0 conn(0x7f53a00af5e0 legacy=0x7f53a00af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f53a4cc2640 1 -- 172.21.15.32:0/933364921 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53a015ebf0 con 0x7f53a015f110 2023-12-20T04:44:33.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f539f7fe640 1 --2- 172.21.15.32:0/933364921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53a006ce60 0x7f53a015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f53a4cc2640 1 --2- 172.21.15.32:0/933364921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a015f110 0x7f53a015c440 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5398009d90 tx=0x7f539801c8c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dca26356dfc9aef4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f539effd640 1 -- 172.21.15.32:0/933364921 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53980a5920 con 0x7f53a015f110 2023-12-20T04:44:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f539effd640 1 -- 172.21.15.32:0/933364921 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53980a5a80 con 0x7f53a015f110 2023-12-20T04:44:33.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f539effd640 1 -- 172.21.15.32:0/933364921 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53980a5df0 con 0x7f53a015f110 2023-12-20T04:44:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f53a6725640 1 -- 172.21.15.32:0/933364921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a015f110 msgr2=0x7f53a015c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f53a6725640 1 --2- 172.21.15.32:0/933364921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a015f110 0x7f53a015c440 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f5398009d90 tx=0x7f539801c8c0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f53a6725640 1 -- 172.21.15.32:0/933364921 shutdown_connections 2023-12-20T04:44:33.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f53a6725640 1 --2- 172.21.15.32:0/933364921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a015f110 0x7f53a015c440 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f53a6725640 1 --2- 172.21.15.32:0/933364921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53a006ce60 0x7f53a015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.909+0000 7f53a6725640 1 -- 172.21.15.32:0/933364921 >> 172.21.15.32:0/933364921 conn(0x7f53a006a320 msgr2=0x7f53a00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f53a6725640 1 -- 172.21.15.32:0/933364921 shutdown_connections 2023-12-20T04:44:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f53a6725640 1 -- 172.21.15.32:0/933364921 wait complete. 2023-12-20T04:44:33.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f53a6725640 1 Processor -- start 2023-12-20T04:44:33.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f53a6725640 1 -- start start 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f53a6725640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53a006ce60 0x7f53a0154e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f53a6725640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a00af5e0 0x7f53a00c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f53a6725640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a015f110 0x7f53a015d0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f53a6725640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a0153320 con 0x7f53a00af5e0 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f53a6725640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00cd130 con 0x7f53a015f110 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f53a6725640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53a00ceca0 con 0x7f53a006ce60 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f539f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a00af5e0 0x7f53a00c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f539f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a00af5e0 0x7f53a00c4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60652/0 (socket says 172.21.15.32:60652) 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f539f7fe640 1 -- 172.21.15.32:0/599732536 learned_addr learned my addr 172.21.15.32:0/599732536 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f53a4cc2640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a015f110 0x7f53a015d0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f539ffff640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53a006ce60 0x7f53a0154e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f539f7fe640 1 -- 172.21.15.32:0/599732536 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53a006ce60 msgr2=0x7f53a0154e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f539f7fe640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53a006ce60 0x7f53a0154e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f539f7fe640 1 -- 172.21.15.32:0/599732536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a015f110 msgr2=0x7f53a015d0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f539f7fe640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a015f110 0x7f53a015d0c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.913+0000 7f539f7fe640 1 -- 172.21.15.32:0/599732536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5394009420 con 0x7f53a00af5e0 2023-12-20T04:44:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.917+0000 7f539f7fe640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a00af5e0 0x7f53a00c4510 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f539400ebb0 tx=0x7f539400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:33.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.917+0000 7f539cff9640 1 -- 172.21.15.32:0/599732536 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f539400eeb0 con 0x7f53a00af5e0 2023-12-20T04:44:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.917+0000 7f539cff9640 1 -- 172.21.15.32:0/599732536 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f539400cc30 con 0x7f53a00af5e0 2023-12-20T04:44:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.917+0000 7f53a6725640 1 -- 172.21.15.32:0/599732536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5398009a70 con 0x7f53a00af5e0 2023-12-20T04:44:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.917+0000 7f539cff9640 1 -- 172.21.15.32:0/599732536 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5394013470 con 0x7f53a00af5e0 2023-12-20T04:44:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.917+0000 7f53a6725640 1 -- 172.21.15.32:0/599732536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53a00ccae0 con 0x7f53a00af5e0 2023-12-20T04:44:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.917+0000 7f53a6725640 1 -- 172.21.15.32:0/599732536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f53a00cd2a0 con 0x7f53a00af5e0 2023-12-20T04:44:33.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.917+0000 7f539cff9640 1 -- 172.21.15.32:0/599732536 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f53940135d0 con 0x7f53a00af5e0 2023-12-20T04:44:33.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.917+0000 7f539cff9640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f538c065b80 0x7f538c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:33.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.917+0000 7f539cff9640 1 -- 172.21.15.32:0/599732536 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f5394083530 con 0x7f53a00af5e0 2023-12-20T04:44:33.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.917+0000 7f539ffff640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f538c065b80 0x7f538c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:33.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.921+0000 7f539ffff640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f538c065b80 0x7f538c068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5390001370 tx=0x7f5390009040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:33.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:33.921+0000 7f539cff9640 1 -- 172.21.15.32:0/599732536 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5394050e90 con 0x7f53a00af5e0 2023-12-20T04:44:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 -- 172.21.15.32:0/599732536 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f53a004e000 con 0x7f53a00af5e0 2023-12-20T04:44:34.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f539cff9640 1 -- 172.21.15.32:0/599732536 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v46) v1 ==== 53+0+73 (secure 0 0 0) 0x7f53940544b0 con 0x7f53a00af5e0 2023-12-20T04:44:34.116 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T04:44:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 -- 172.21.15.32:0/599732536 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f538c065b80 msgr2=0x7f538c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f538c065b80 0x7f538c068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f5390001370 tx=0x7f5390009040 comp rx=0 tx=0).stop 2023-12-20T04:44:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 -- 172.21.15.32:0/599732536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a00af5e0 msgr2=0x7f53a00c4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a00af5e0 0x7f53a00c4510 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f539400ebb0 tx=0x7f539400c520 comp rx=0 tx=0).stop 2023-12-20T04:44:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 -- 172.21.15.32:0/599732536 shutdown_connections 2023-12-20T04:44:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f538c065b80 0x7f538c068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53a015f110 0x7f53a015d0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53a00af5e0 0x7f53a00c4510 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 --2- 172.21.15.32:0/599732536 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53a006ce60 0x7f53a0154e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 -- 172.21.15.32:0/599732536 >> 172.21.15.32:0/599732536 conn(0x7f53a006a320 msgr2=0x7f53a0153840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 -- 172.21.15.32:0/599732536 shutdown_connections 2023-12-20T04:44:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.117+0000 7f53a6725640 1 -- 172.21.15.32:0/599732536 wait complete. 2023-12-20T04:44:34.137 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T04:44:34.137 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T04:44:34.137 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T04:44:34.137 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T04:44:34.137 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T04:44:34.137 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T04:44:34.137 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T04:44:34.137 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T04:44:34.138 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:44:34.138 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:34.138 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T04:44:34.148 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:44:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:44:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:44:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:34.148 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41386sec preferred_lft 41386sec 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:34.149 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T04:44:34.150 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:34.151 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:44:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T04:44:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T04:44:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:34.151 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:44:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T04:44:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T04:44:34.151 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:44:34.151 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:34.151 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T04:44:34.210 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T04:44:34.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:34.213 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T04:44:34.268 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T04:44:34.268 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T04:44:34.268 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T04:44:34.268 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T04:44:34.268 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T04:44:34.268 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T04:44:34.268 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:34.269 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:44:34.269 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:44:34.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T04:44:34.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T04:44:34.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T04:44:34.269 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:44:34.364 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:34.364 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T04:44:34.373 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T04:44:34.373 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T04:44:34.374 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:44:34.374 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:34.375 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T04:44:34.436 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:44:34.436 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T04:44:34.492 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:44:34.492 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:44:34.492 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:44:34.542 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T04:44:34.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:34.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:44:34.639 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:44:34.639 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:44:34.653 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:44:34.653 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:44:34.653 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T04:44:34.653 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T04:44:34.653 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T04:44:34.657 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T04:44:34.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:34.658 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:44:34.679 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:44:34.679 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:44:34.743 INFO:teuthology.orchestra.run.smithi032.stdout:client7515 2023-12-20T04:44:34.743 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:44:34.743 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:44:34.795 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T04:44:34.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.981+0000 7fcdfca20640 1 Processor -- start 2023-12-20T04:44:34.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.981+0000 7fcdfca20640 1 -- start start 2023-12-20T04:44:34.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdfca20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf8071af0 0x7fcdf80c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:34.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdfca20640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcdf8160b30 0x7fcdf8160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:34.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdfca20640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcdf80c9a40 con 0x7fcdf815ce60 2023-12-20T04:44:34.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdfca20640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcdf80c9180 con 0x7fcdf8160b30 2023-12-20T04:44:34.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdfca20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf80c93f0 con 0x7fcdf8071af0 2023-12-20T04:44:34.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdeffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf8071af0 0x7fcdf80c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdf659c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcdf815ce60 0x7fcdf80713e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:48448/0 (socket says 172.21.15.32:48448) 2023-12-20T04:44:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdf659c640 1 -- 172.21.15.32:0/4275647682 learned_addr learned my addr 172.21.15.32:0/4275647682 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdf6d9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcdf8160b30 0x7fcdf8160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdf5d9b640 1 -- 172.21.15.32:0/4275647682 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3280785091 0 0) 0x7fcdf80c9a40 con 0x7fcdf815ce60 2023-12-20T04:44:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdf5d9b640 1 -- 172.21.15.32:0/4275647682 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcde4003530 con 0x7fcdf815ce60 2023-12-20T04:44:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdf6d9d640 1 -- 172.21.15.32:0/4275647682 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf8071af0 msgr2=0x7fcdf80c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdf6d9d640 1 --2- 172.21.15.32:0/4275647682 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf8071af0 0x7fcdf80c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdf6d9d640 1 -- 172.21.15.32:0/4275647682 >> v1:172.21.15.39:6789/0 conn(0x7fcdf815ce60 legacy=0x7fcdf80713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdf6d9d640 1 -- 172.21.15.32:0/4275647682 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcdf80c96d0 con 0x7fcdf8160b30 2023-12-20T04:44:34.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.985+0000 7fcdeffff640 1 --2- 172.21.15.32:0/4275647682 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf8071af0 0x7fcdf80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:34.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdf6d9d640 1 --2- 172.21.15.32:0/4275647682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf8160b30 0x7fcdf8160fc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009d90 tx=0x7fcdf001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bf0705bccffa30c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:34.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdf5d9b640 1 -- 172.21.15.32:0/4275647682 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00a5920 con 0x7fcdf8160b30 2023-12-20T04:44:34.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdf5d9b640 1 -- 172.21.15.32:0/4275647682 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf00a5a80 con 0x7fcdf8160b30 2023-12-20T04:44:34.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdf5d9b640 1 -- 172.21.15.32:0/4275647682 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00a5e10 con 0x7fcdf8160b30 2023-12-20T04:44:34.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdfca20640 1 -- 172.21.15.32:0/4275647682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf8160b30 msgr2=0x7fcdf8160fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:34.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdfca20640 1 --2- 172.21.15.32:0/4275647682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf8160b30 0x7fcdf8160fc0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7fcdf0009d90 tx=0x7fcdf001c950 comp rx=0 tx=0).stop 2023-12-20T04:44:34.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdfca20640 1 -- 172.21.15.32:0/4275647682 shutdown_connections 2023-12-20T04:44:34.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdfca20640 1 --2- 172.21.15.32:0/4275647682 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf8160b30 0x7fcdf8160fc0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:34.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdfca20640 1 --2- 172.21.15.32:0/4275647682 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf8071af0 0x7fcdf80c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:34.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdfca20640 1 -- 172.21.15.32:0/4275647682 >> 172.21.15.32:0/4275647682 conn(0x7fcdf806a320 msgr2=0x7fcdf80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:34.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdfca20640 1 -- 172.21.15.32:0/4275647682 shutdown_connections 2023-12-20T04:44:34.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.989+0000 7fcdfca20640 1 -- 172.21.15.32:0/4275647682 wait complete. 2023-12-20T04:44:34.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdfca20640 1 Processor -- start 2023-12-20T04:44:34.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdfca20640 1 -- start start 2023-12-20T04:44:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdfca20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf8071af0 0x7fcdf80b2270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdfca20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf815ce60 0x7fcdf815e340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdfca20640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf8160b30 0x7fcdf8153e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdfca20640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf81f9c50 con 0x7fcdf8160b30 2023-12-20T04:44:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdfca20640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf815d580 con 0x7fcdf8071af0 2023-12-20T04:44:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdfca20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcdf815d860 con 0x7fcdf815ce60 2023-12-20T04:44:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdf6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf8160b30 0x7fcdf8153e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdf6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf8160b30 0x7fcdf8153e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60654/0 (socket says 172.21.15.32:60654) 2023-12-20T04:44:34.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdf6d9d640 1 -- 172.21.15.32:0/1774777986 learned_addr learned my addr 172.21.15.32:0/1774777986 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdeffff640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf815ce60 0x7fcdf815e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdf659c640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf8071af0 0x7fcdf80b2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdf6d9d640 1 -- 172.21.15.32:0/1774777986 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf815ce60 msgr2=0x7fcdf815e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdf6d9d640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf815ce60 0x7fcdf815e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdf6d9d640 1 -- 172.21.15.32:0/1774777986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf8071af0 msgr2=0x7fcdf80b2270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdf6d9d640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf8071af0 0x7fcdf80b2270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdf6d9d640 1 -- 172.21.15.32:0/1774777986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcde0009420 con 0x7fcdf8160b30 2023-12-20T04:44:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdf659c640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf8071af0 0x7fcdf80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:44:34.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdf6d9d640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf8160b30 0x7fcdf8153e60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fcdf001a580 tx=0x7fcdf00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:34.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdeeffd640 1 -- 172.21.15.32:0/1774777986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00bf070 con 0x7fcdf8160b30 2023-12-20T04:44:34.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdeeffd640 1 -- 172.21.15.32:0/1774777986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdf00aea30 con 0x7fcdf8160b30 2023-12-20T04:44:34.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdeeffd640 1 -- 172.21.15.32:0/1774777986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdf00aedc0 con 0x7fcdf8160b30 2023-12-20T04:44:34.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdfca20640 1 -- 172.21.15.32:0/1774777986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcdf0009a70 con 0x7fcdf8160b30 2023-12-20T04:44:34.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdfca20640 1 -- 172.21.15.32:0/1774777986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcdf8161680 con 0x7fcdf8160b30 2023-12-20T04:44:34.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.993+0000 7fcdfca20640 1 -- 172.21.15.32:0/1774777986 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcdf804e000 con 0x7fcdf8160b30 2023-12-20T04:44:34.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.997+0000 7fcdeeffd640 1 -- 172.21.15.32:0/1774777986 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcdf00b4050 con 0x7fcdf8160b30 2023-12-20T04:44:34.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.997+0000 7fcdeeffd640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcdd8065b80 0x7fcdd8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:34.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.997+0000 7fcdeeffd640 1 -- 172.21.15.32:0/1774777986 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fcdf011dc30 con 0x7fcdf8160b30 2023-12-20T04:44:34.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.997+0000 7fcdf659c640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcdd8065b80 0x7fcdd8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:34.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:34.997+0000 7fcdf659c640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcdd8065b80 0x7fcdd8068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcde80058e0 tx=0x7fcde8009040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.001+0000 7fcdeeffd640 1 -- 172.21.15.32:0/1774777986 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcdf00eb4d0 con 0x7fcdf8160b30 2023-12-20T04:44:35.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.181+0000 7fcdfca20640 1 -- 172.21.15.32:0/1774777986 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fcdf815d6f0 con 0x7fcdd8065b80 2023-12-20T04:44:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.181+0000 7fcdeeffd640 1 -- 172.21.15.32:0/1774777986 <== mgr.7113 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fcdf815d6f0 con 0x7fcdd8065b80 2023-12-20T04:44:35.182 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:44:35.182 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:44:35.182 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T04:44:35.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:44:35.182 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:44:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 -- 172.21.15.32:0/1774777986 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcdd8065b80 msgr2=0x7fcdd8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcdd8065b80 0x7fcdd8068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fcde80058e0 tx=0x7fcde8009040 comp rx=0 tx=0).stop 2023-12-20T04:44:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 -- 172.21.15.32:0/1774777986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf8160b30 msgr2=0x7fcdf8153e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:35.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf8160b30 0x7fcdf8153e60 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7fcdf001a580 tx=0x7fcdf00196f0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 -- 172.21.15.32:0/1774777986 shutdown_connections 2023-12-20T04:44:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcdd8065b80 0x7fcdd8068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcdf8160b30 0x7fcdf8153e60 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcdf815ce60 0x7fcdf815e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 --2- 172.21.15.32:0/1774777986 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcdf8071af0 0x7fcdf80b2270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 -- 172.21.15.32:0/1774777986 >> 172.21.15.32:0/1774777986 conn(0x7fcdf806a320 msgr2=0x7fcdf80b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 -- 172.21.15.32:0/1774777986 shutdown_connections 2023-12-20T04:44:35.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.185+0000 7fcdfca20640 1 -- 172.21.15.32:0/1774777986 wait complete. 2023-12-20T04:44:35.203 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T04:44:35.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa643cc4640 1 Processor -- start 2023-12-20T04:44:35.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa643cc4640 1 -- start start 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa643cc4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa63c15abb0 0x7fa63c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa643cc4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa63c0b0cb0 0x7fa63c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa643cc4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa63c070230 0x7fa63c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa643cc4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c06e160 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa643cc4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa63c070c00 con 0x7fa63c15abb0 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa643cc4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c15b6a0 con 0x7fa63c070230 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa64125f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa63c0b0cb0 0x7fa63c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa64125f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa63c0b0cb0 0x7fa63c15bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60660/0 (socket says 172.21.15.32:60660) 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa64125f640 1 -- 172.21.15.32:0/3118589221 learned_addr learned my addr 172.21.15.32:0/3118589221 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa64125f640 1 -- 172.21.15.32:0/3118589221 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa63c070230 msgr2=0x7fa63c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa64125f640 1 --2- 172.21.15.32:0/3118589221 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa63c070230 0x7fa63c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa64125f640 1 -- 172.21.15.32:0/3118589221 >> v2:172.21.15.39:3300/0 conn(0x7fa63c15abb0 msgr2=0x7fa63c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa64125f640 1 --2- 172.21.15.32:0/3118589221 >> v2:172.21.15.39:3300/0 conn(0x7fa63c15abb0 0x7fa63c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.349+0000 7fa64125f640 1 -- 172.21.15.32:0/3118589221 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa63c15c070 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa64125f640 1 --2- 172.21.15.32:0/3118589221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b0cb0 0x7fa63c15bb30 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0713f0 tx=0x7fa62c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4916ac23baa53e54 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:35.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa640a5e640 1 -- 172.21.15.32:0/3118589221 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0aa620 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa640a5e640 1 -- 172.21.15.32:0/3118589221 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62c0aa780 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa640a5e640 1 -- 172.21.15.32:0/3118589221 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0aab10 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa643cc4640 1 -- 172.21.15.32:0/3118589221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b0cb0 msgr2=0x7fa63c15bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa643cc4640 1 --2- 172.21.15.32:0/3118589221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b0cb0 0x7fa63c15bb30 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7fa63c0713f0 tx=0x7fa62c01c500 comp rx=0 tx=0).stop 2023-12-20T04:44:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa643cc4640 1 -- 172.21.15.32:0/3118589221 shutdown_connections 2023-12-20T04:44:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa643cc4640 1 --2- 172.21.15.32:0/3118589221 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa63c070230 0x7fa63c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa643cc4640 1 --2- 172.21.15.32:0/3118589221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b0cb0 0x7fa63c15bb30 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa643cc4640 1 --2- 172.21.15.32:0/3118589221 >> v2:172.21.15.39:3300/0 conn(0x7fa63c15abb0 0x7fa63c0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa643cc4640 1 -- 172.21.15.32:0/3118589221 >> 172.21.15.32:0/3118589221 conn(0x7fa63c06a510 msgr2=0x7fa63c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa643cc4640 1 -- 172.21.15.32:0/3118589221 shutdown_connections 2023-12-20T04:44:35.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.353+0000 7fa643cc4640 1 -- 172.21.15.32:0/3118589221 wait complete. 2023-12-20T04:44:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa643cc4640 1 Processor -- start 2023-12-20T04:44:35.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa643cc4640 1 -- start start 2023-12-20T04:44:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa643cc4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa63c070230 0x7fa63c1440d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:35.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa643cc4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b0cb0 0x7fa63c150500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa643cc4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa63c15abb0 0x7fa63c1598a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa643cc4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c0cd560 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa643cc4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c0cc950 con 0x7fa63c070230 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa643cc4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa63c0ccc30 con 0x7fa63c15abb0 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa64125f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b0cb0 0x7fa63c150500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa64125f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b0cb0 0x7fa63c150500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60668/0 (socket says 172.21.15.32:60668) 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa64125f640 1 -- 172.21.15.32:0/1704206206 learned_addr learned my addr 172.21.15.32:0/1704206206 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa641a60640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa63c070230 0x7fa63c1440d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa642261640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa63c15abb0 0x7fa63c1598a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa64125f640 1 -- 172.21.15.32:0/1704206206 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa63c15abb0 msgr2=0x7fa63c1598a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa64125f640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa63c15abb0 0x7fa63c1598a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa64125f640 1 -- 172.21.15.32:0/1704206206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa63c070230 msgr2=0x7fa63c1440d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa64125f640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa63c070230 0x7fa63c1440d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa64125f640 1 -- 172.21.15.32:0/1704206206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa62c0094d0 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa641a60640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa63c070230 0x7fa63c1440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:35.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa64125f640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b0cb0 0x7fa63c150500 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa62c01abb0 tx=0x7fa62c01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:35.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa6327fc640 1 -- 172.21.15.32:0/1704206206 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0bd5f0 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa6327fc640 1 -- 172.21.15.32:0/1704206206 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa62c0bd750 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa6327fc640 1 -- 172.21.15.32:0/1704206206 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa62c0bdb00 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa643cc4640 1 -- 172.21.15.32:0/1704206206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa63c0ccf10 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.357+0000 7fa643cc4640 1 -- 172.21.15.32:0/1704206206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa63c144610 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.361+0000 7fa643cc4640 1 -- 172.21.15.32:0/1704206206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa63c04e000 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.361+0000 7fa6327fc640 1 -- 172.21.15.32:0/1704206206 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 47) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa62c0b33f0 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.361+0000 7fa6327fc640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa620065b80 0x7fa620068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.361+0000 7fa6327fc640 1 -- 172.21.15.32:0/1704206206 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fa62c11df60 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.361+0000 7fa641a60640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa620065b80 0x7fa620068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:35.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.361+0000 7fa641a60640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa620065b80 0x7fa620068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa638006fd0 tx=0x7fa638008040 comp rx=0 tx=0).ready entity=mgr.7113 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:35.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.365+0000 7fa6327fc640 1 -- 172.21.15.32:0/1704206206 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa62c0eb8c0 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.565+0000 7fa643cc4640 1 -- 172.21.15.32:0/1704206206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fa63c151a00 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.961+0000 7fa6327fc640 1 -- 172.21.15.32:0/1704206206 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 48) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fa62c0ebc90 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.961+0000 7fa6327fc640 1 -- 172.21.15.32:0/1704206206 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v48) v1 ==== 92+0+0 (secure 0 0 0) 0x7fa62c0f2760 con 0x7fa63c0b0cb0 2023-12-20T04:44:35.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.961+0000 7fa643cc4640 1 -- 172.21.15.32:0/1704206206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa620065b80 msgr2=0x7fa620068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:35.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.961+0000 7fa643cc4640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa620065b80 0x7fa620068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa638006fd0 tx=0x7fa638008040 comp rx=0 tx=0).stop 2023-12-20T04:44:35.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.961+0000 7fa643cc4640 1 -- 172.21.15.32:0/1704206206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b0cb0 msgr2=0x7fa63c150500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:35.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.961+0000 7fa643cc4640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b0cb0 0x7fa63c150500 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7fa62c01abb0 tx=0x7fa62c01ad40 comp rx=0 tx=0).stop 2023-12-20T04:44:35.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.961+0000 7fa643cc4640 1 -- 172.21.15.32:0/1704206206 shutdown_connections 2023-12-20T04:44:35.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.961+0000 7fa643cc4640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa620065b80 0x7fa620068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.961+0000 7fa643cc4640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa63c15abb0 0x7fa63c1598a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.961+0000 7fa643cc4640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa63c0b0cb0 0x7fa63c150500 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.965+0000 7fa643cc4640 1 --2- 172.21.15.32:0/1704206206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa63c070230 0x7fa63c1440d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:35.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.965+0000 7fa643cc4640 1 -- 172.21.15.32:0/1704206206 >> 172.21.15.32:0/1704206206 conn(0x7fa63c06a510 msgr2=0x7fa63c0c4b00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:35.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.965+0000 7fa643cc4640 1 -- 172.21.15.32:0/1704206206 shutdown_connections 2023-12-20T04:44:35.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:35.965+0000 7fa643cc4640 1 -- 172.21.15.32:0/1704206206 wait complete. 2023-12-20T04:44:35.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T04:44:36.136 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:36.135+0000 7f2e0af4d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:44:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.149+0000 7f17abcf2640 1 Processor -- start 2023-12-20T04:44:36.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.149+0000 7f17abcf2640 1 -- start start 2023-12-20T04:44:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.149+0000 7f17abcf2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a406cdf0 0x7f17a4070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.149+0000 7f17abcf2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17a4070770 0x7f17a40cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.149+0000 7f17abcf2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a40cc3b0 con 0x7f17a41591e0 2023-12-20T04:44:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.149+0000 7f17abcf2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a4070be0 con 0x7f17a4070770 2023-12-20T04:44:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.149+0000 7f17abcf2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40c8e30 con 0x7f17a406cdf0 2023-12-20T04:44:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.149+0000 7f17a9a8e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f17a41591e0 0x7f17a40caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49146/0 (socket says 172.21.15.32:49146) 2023-12-20T04:44:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.149+0000 7f17a9a8e640 1 -- 172.21.15.32:0/2365978085 learned_addr learned my addr 172.21.15.32:0/2365978085 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.149+0000 7f17a928d640 1 --2- 172.21.15.32:0/2365978085 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a406cdf0 0x7f17a4070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.149+0000 7f17aa28f640 1 --2- 172.21.15.32:0/2365978085 >> v2:172.21.15.32:3300/0 conn(0x7f17a4070770 0x7f17a40cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.153+0000 7f17a8a8c640 1 -- 172.21.15.32:0/2365978085 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 433553441 0 0) 0x7f17a40cc3b0 con 0x7f17a41591e0 2023-12-20T04:44:36.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.153+0000 7f17a928d640 1 -- 172.21.15.32:0/2365978085 >> v2:172.21.15.32:3300/0 conn(0x7f17a4070770 msgr2=0x7f17a40cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:36.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.153+0000 7f17a928d640 1 --2- 172.21.15.32:0/2365978085 >> v2:172.21.15.32:3300/0 conn(0x7f17a4070770 0x7f17a40cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.153+0000 7f17a928d640 1 -- 172.21.15.32:0/2365978085 >> v1:172.21.15.32:6789/0 conn(0x7f17a41591e0 legacy=0x7f17a40caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.153+0000 7f17a928d640 1 -- 172.21.15.32:0/2365978085 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a406d280 con 0x7f17a406cdf0 2023-12-20T04:44:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.153+0000 7f17a928d640 1 --2- 172.21.15.32:0/2365978085 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a406cdf0 0x7f17a4070230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17940097c0 tx=0x7f179401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d9050373cc3acaa1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.153+0000 7f17a8a8c640 1 -- 172.21.15.32:0/2365978085 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17940ab600 con 0x7f17a406cdf0 2023-12-20T04:44:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.153+0000 7f17a8a8c640 1 -- 172.21.15.32:0/2365978085 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17940a7050 con 0x7f17a406cdf0 2023-12-20T04:44:36.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.153+0000 7f17a8a8c640 1 -- 172.21.15.32:0/2365978085 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17940ab980 con 0x7f17a406cdf0 2023-12-20T04:44:36.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.153+0000 7f17abcf2640 1 -- 172.21.15.32:0/2365978085 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a406cdf0 msgr2=0x7f17a4070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.153+0000 7f17abcf2640 1 --2- 172.21.15.32:0/2365978085 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a406cdf0 0x7f17a4070230 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f17940097c0 tx=0x7f179401c3f0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 -- 172.21.15.32:0/2365978085 shutdown_connections 2023-12-20T04:44:36.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 --2- 172.21.15.32:0/2365978085 >> v2:172.21.15.32:3300/0 conn(0x7f17a4070770 0x7f17a40cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 --2- 172.21.15.32:0/2365978085 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a406cdf0 0x7f17a4070230 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 -- 172.21.15.32:0/2365978085 >> 172.21.15.32:0/2365978085 conn(0x7f17a406a510 msgr2=0x7f17a40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:36.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 -- 172.21.15.32:0/2365978085 shutdown_connections 2023-12-20T04:44:36.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 -- 172.21.15.32:0/2365978085 wait complete. 2023-12-20T04:44:36.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 Processor -- start 2023-12-20T04:44:36.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 -- start start 2023-12-20T04:44:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17a4070770 0x7f17a41475a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a41591e0 0x7f17a4151830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4151d70 0x7f17a41577f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40c58b0 con 0x7f17a4070770 2023-12-20T04:44:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40c6430 con 0x7f17a41591e0 2023-12-20T04:44:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17abcf2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40c5b00 con 0x7f17a4151d70 2023-12-20T04:44:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17a9a8e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17a4070770 0x7f17a41475a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17a9a8e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17a4070770 0x7f17a41475a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60696/0 (socket says 172.21.15.32:60696) 2023-12-20T04:44:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17a9a8e640 1 -- 172.21.15.32:0/4152321675 learned_addr learned my addr 172.21.15.32:0/4152321675 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:36.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17a928d640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a41591e0 0x7f17a4151830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17aa28f640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4151d70 0x7f17a41577f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17a9a8e640 1 -- 172.21.15.32:0/4152321675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4151d70 msgr2=0x7f17a41577f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17a9a8e640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4151d70 0x7f17a41577f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17a9a8e640 1 -- 172.21.15.32:0/4152321675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a41591e0 msgr2=0x7f17a4151830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17a9a8e640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a41591e0 0x7f17a4151830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17a9a8e640 1 -- 172.21.15.32:0/4152321675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1794009420 con 0x7f17a4070770 2023-12-20T04:44:36.160 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f29f3448180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:44:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.157+0000 7f17a9a8e640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17a4070770 0x7f17a41475a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c4b0 tx=0x7f17a000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.161+0000 7f179a7fc640 1 -- 172.21.15.32:0/4152321675 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a00074e0 con 0x7f17a4070770 2023-12-20T04:44:36.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.161+0000 7f179a7fc640 1 -- 172.21.15.32:0/4152321675 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a0007660 con 0x7f17a4070770 2023-12-20T04:44:36.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.161+0000 7f179a7fc640 1 -- 172.21.15.32:0/4152321675 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0004a30 con 0x7f17a4070770 2023-12-20T04:44:36.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.161+0000 7f17abcf2640 1 -- 172.21.15.32:0/4152321675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a40c5de0 con 0x7f17a4070770 2023-12-20T04:44:36.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.161+0000 7f17abcf2640 1 -- 172.21.15.32:0/4152321675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a4147ae0 con 0x7f17a4070770 2023-12-20T04:44:36.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.161+0000 7f17abcf2640 1 -- 172.21.15.32:0/4152321675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a404e000 con 0x7f17a4070770 2023-12-20T04:44:36.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.161+0000 7f179a7fc640 1 -- 172.21.15.32:0/4152321675 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 48) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f17a0004b90 con 0x7f17a4070770 2023-12-20T04:44:36.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.161+0000 7f179a7fc640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f178806a010 0x7f178806c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.161+0000 7f179a7fc640 1 -- 172.21.15.32:0/4152321675 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f17a0082370 con 0x7f17a4070770 2023-12-20T04:44:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.161+0000 7f17a928d640 1 -- 172.21.15.32:0/4152321675 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f178806a010 msgr2=0x7f178806c4d0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:36.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.161+0000 7f17a928d640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f178806a010 0x7f178806c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:36.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.165+0000 7f179a7fc640 1 -- 172.21.15.32:0/4152321675 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f17a004fd60 con 0x7f17a4070770 2023-12-20T04:44:36.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.329+0000 7f17abcf2640 1 -- 172.21.15.32:0/4152321675 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f17a406d840 con 0x7f17a4070770 2023-12-20T04:44:36.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.333+0000 7f179a7fc640 1 -- 172.21.15.32:0/4152321675 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v35)=0 v35) v1 ==== 138+0+0 (secure 0 0 0) 0x7f17a0053380 con 0x7f17a4070770 2023-12-20T04:44:36.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 -- 172.21.15.32:0/4152321675 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f178806a010 msgr2=0x7f178806c4d0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:36.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f178806a010 0x7f178806c4d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 -- 172.21.15.32:0/4152321675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17a4070770 msgr2=0x7f17a41475a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17a4070770 0x7f17a41475a0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f17a000c4b0 tx=0x7f17a000c970 comp rx=0 tx=0).stop 2023-12-20T04:44:36.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 -- 172.21.15.32:0/4152321675 shutdown_connections 2023-12-20T04:44:36.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f178806a010 0x7f178806c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4151d70 0x7f17a41577f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a41591e0 0x7f17a4151830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 --2- 172.21.15.32:0/4152321675 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17a4070770 0x7f17a41475a0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 -- 172.21.15.32:0/4152321675 >> 172.21.15.32:0/4152321675 conn(0x7f17a406a510 msgr2=0x7f17a40cb340 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:36.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 -- 172.21.15.32:0/4152321675 shutdown_connections 2023-12-20T04:44:36.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.337+0000 7f17abcf2640 1 -- 172.21.15.32:0/4152321675 wait complete. 2023-12-20T04:44:36.356 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T04:44:36.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc93d48c640 1 Processor -- start 2023-12-20T04:44:36.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc93d48c640 1 -- start start 2023-12-20T04:44:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc93d48c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc93815ce60 0x7fc9380d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc93d48c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9380d2650 con 0x7fc9380d14d0 2023-12-20T04:44:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc93d48c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9380d1a40 con 0x7fc9380d3130 2023-12-20T04:44:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc93d48c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9380d35a0 con 0x7fc93815ce60 2023-12-20T04:44:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc936ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc93815ce60 0x7fc9380d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc9377fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc9380d14d0 0x7fc9380b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49162/0 (socket says 172.21.15.32:49162) 2023-12-20T04:44:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc936ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc93815ce60 0x7fc9380d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60702/0 (socket says 172.21.15.32:60702) 2023-12-20T04:44:36.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc9377fe640 1 -- 172.21.15.32:0/3491790438 learned_addr learned my addr 172.21.15.32:0/3491790438 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:44:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc936ffd640 1 -- 172.21.15.32:0/3491790438 >> v1:172.21.15.39:6789/0 conn(0x7fc9380d3130 legacy=0x7fc9380d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc936ffd640 1 -- 172.21.15.32:0/3491790438 >> v1:172.21.15.32:6789/0 conn(0x7fc9380d14d0 legacy=0x7fc9380b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc936ffd640 1 -- 172.21.15.32:0/3491790438 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9380d1d20 con 0x7fc93815ce60 2023-12-20T04:44:36.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.505+0000 7fc935ffb640 1 -- 172.21.15.32:0/3491790438 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3063171285 0 0) 0x7fc9380d1a40 con 0x7fc9380d3130 2023-12-20T04:44:36.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.509+0000 7fc936ffd640 1 --2- 172.21.15.32:0/3491790438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93815ce60 0x7fc9380d2bf0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc9280097f0 tx=0x7fc92801c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b34bf683b6d995 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.509+0000 7fc935ffb640 1 -- 172.21.15.32:0/3491790438 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9280ad680 con 0x7fc93815ce60 2023-12-20T04:44:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.509+0000 7fc935ffb640 1 -- 172.21.15.32:0/3491790438 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9280ad7e0 con 0x7fc93815ce60 2023-12-20T04:44:36.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.509+0000 7fc935ffb640 1 -- 172.21.15.32:0/3491790438 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9280adb70 con 0x7fc93815ce60 2023-12-20T04:44:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.509+0000 7fc93d48c640 1 -- 172.21.15.32:0/3491790438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93815ce60 msgr2=0x7fc9380d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.509+0000 7fc93d48c640 1 --2- 172.21.15.32:0/3491790438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93815ce60 0x7fc9380d2bf0 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7fc9280097f0 tx=0x7fc92801c3a0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.509+0000 7fc93d48c640 1 -- 172.21.15.32:0/3491790438 shutdown_connections 2023-12-20T04:44:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.509+0000 7fc93d48c640 1 --2- 172.21.15.32:0/3491790438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc93815ce60 0x7fc9380d2bf0 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.509+0000 7fc93d48c640 1 -- 172.21.15.32:0/3491790438 >> 172.21.15.32:0/3491790438 conn(0x7fc93806b5e0 msgr2=0x7fc93806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:36.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.509+0000 7fc93d48c640 1 -- 172.21.15.32:0/3491790438 shutdown_connections 2023-12-20T04:44:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.509+0000 7fc93d48c640 1 -- 172.21.15.32:0/3491790438 wait complete. 2023-12-20T04:44:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc93d48c640 1 Processor -- start 2023-12-20T04:44:36.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc93d48c640 1 -- start start 2023-12-20T04:44:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc93d48c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9380d14d0 0x7fc9380cc540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc93d48c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9380d3130 0x7fc9380cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc93d48c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc93815ce60 0x7fc938151d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc93d48c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380cb4a0 con 0x7fc9380d14d0 2023-12-20T04:44:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc93d48c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380cb780 con 0x7fc93815ce60 2023-12-20T04:44:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc93d48c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9380cba60 con 0x7fc9380d3130 2023-12-20T04:44:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc936ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9380d14d0 0x7fc9380cc540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc936ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9380d14d0 0x7fc9380cc540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60704/0 (socket says 172.21.15.32:60704) 2023-12-20T04:44:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc936ffd640 1 -- 172.21.15.32:0/3601730888 learned_addr learned my addr 172.21.15.32:0/3601730888 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc9377fe640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc93815ce60 0x7fc938151d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc9367fc640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9380d3130 0x7fc9380cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc936ffd640 1 -- 172.21.15.32:0/3601730888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9380d3130 msgr2=0x7fc9380cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc936ffd640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9380d3130 0x7fc9380cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc936ffd640 1 -- 172.21.15.32:0/3601730888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc93815ce60 msgr2=0x7fc938151d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc936ffd640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc93815ce60 0x7fc938151d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc936ffd640 1 -- 172.21.15.32:0/3601730888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc928009420 con 0x7fc9380d14d0 2023-12-20T04:44:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc936ffd640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9380d14d0 0x7fc9380cc540 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc928018fb0 tx=0x7fc92801abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc91f7fe640 1 -- 172.21.15.32:0/3601730888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9280b8830 con 0x7fc9380d14d0 2023-12-20T04:44:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc91f7fe640 1 -- 172.21.15.32:0/3601730888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9280b8990 con 0x7fc9380d14d0 2023-12-20T04:44:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc91f7fe640 1 -- 172.21.15.32:0/3601730888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9280b8d40 con 0x7fc9380d14d0 2023-12-20T04:44:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc93d48c640 1 -- 172.21.15.32:0/3601730888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9380cbd40 con 0x7fc9380d14d0 2023-12-20T04:44:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.513+0000 7fc93d48c640 1 -- 172.21.15.32:0/3601730888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9380cca80 con 0x7fc9380d14d0 2023-12-20T04:44:36.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.517+0000 7fc93d48c640 1 -- 172.21.15.32:0/3601730888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9380cd410 con 0x7fc9380d14d0 2023-12-20T04:44:36.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.517+0000 7fc91f7fe640 1 -- 172.21.15.32:0/3601730888 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 48) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fc9280b4040 con 0x7fc9380d14d0 2023-12-20T04:44:36.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.517+0000 7fc91f7fe640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc924065bd0 0x7fc924068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.517+0000 7fc91f7fe640 1 -- 172.21.15.32:0/3601730888 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fc92811dbe0 con 0x7fc9380d14d0 2023-12-20T04:44:36.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.517+0000 7fc9367fc640 1 -- 172.21.15.32:0/3601730888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc924065bd0 msgr2=0x7fc924068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:36.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.517+0000 7fc9367fc640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc924065bd0 0x7fc924068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:36.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.521+0000 7fc91f7fe640 1 -- 172.21.15.32:0/3601730888 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc9280eb430 con 0x7fc9380d14d0 2023-12-20T04:44:36.577 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:36.575+0000 7f2e0af4d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:44:36.609 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:36.609+0000 7f29f3448180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:44:36.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.681+0000 7fc93d48c640 1 -- 172.21.15.32:0/3601730888 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fc93804e000 con 0x7fc9380d14d0 2023-12-20T04:44:36.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc91f7fe640 1 -- 172.21.15.32:0/3601730888 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v36)=0 v36) v1 ==== 133+0+0 (secure 0 0 0) 0x7fc9280eea50 con 0x7fc9380d14d0 2023-12-20T04:44:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 -- 172.21.15.32:0/3601730888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc924065bd0 msgr2=0x7fc924068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc924065bd0 0x7fc924068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 -- 172.21.15.32:0/3601730888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9380d14d0 msgr2=0x7fc9380cc540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9380d14d0 0x7fc9380cc540 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7fc928018fb0 tx=0x7fc92801abe0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 -- 172.21.15.32:0/3601730888 shutdown_connections 2023-12-20T04:44:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc924065bd0 0x7fc924068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc93815ce60 0x7fc938151d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9380d3130 0x7fc9380cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 --2- 172.21.15.32:0/3601730888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9380d14d0 0x7fc9380cc540 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 -- 172.21.15.32:0/3601730888 >> 172.21.15.32:0/3601730888 conn(0x7fc93806b5e0 msgr2=0x7fc9380b7b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 -- 172.21.15.32:0/3601730888 shutdown_connections 2023-12-20T04:44:36.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.689+0000 7fc93d48c640 1 -- 172.21.15.32:0/3601730888 wait complete. 2023-12-20T04:44:36.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /bad-path 1M 2023-12-20T04:44:36.735 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:36.735+0000 7f2e0af4d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:44:36.771 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:36.769+0000 7f29f3448180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:44:36.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.857+0000 7f76132b6640 1 Processor -- start 2023-12-20T04:44:36.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f76132b6640 1 -- start start 2023-12-20T04:44:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f76132b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f760c15ce60 0x7f760c0d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f76132b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d3130 0x7f760c0d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f76132b6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f760c0d1300 0x7f760c0b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f76132b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760c0d35a0 con 0x7f760c15ce60 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f76132b6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f760c0d2650 con 0x7f760c0d1300 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f76132b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760c0d1a40 con 0x7f760c0d3130 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f7611052640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f760c15ce60 0x7f760c0d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f7611052640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f760c15ce60 0x7f760c0d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60710/0 (socket says 172.21.15.32:60710) 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f7611052640 1 -- 172.21.15.32:0/427359048 learned_addr learned my addr 172.21.15.32:0/427359048 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f7611853640 1 --2- 172.21.15.32:0/427359048 >> v2:172.21.15.39:3300/0 conn(0x7f760c0d1300 0x7f760c0b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f7610851640 1 --2- 172.21.15.32:0/427359048 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d3130 0x7f760c0d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f7611052640 1 -- 172.21.15.32:0/427359048 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d3130 msgr2=0x7f760c0d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f7611052640 1 --2- 172.21.15.32:0/427359048 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d3130 0x7f760c0d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f7611052640 1 -- 172.21.15.32:0/427359048 >> v2:172.21.15.39:3300/0 conn(0x7f760c0d1300 msgr2=0x7f760c0b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f7611052640 1 --2- 172.21.15.32:0/427359048 >> v2:172.21.15.39:3300/0 conn(0x7f760c0d1300 0x7f760c0b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f7611052640 1 -- 172.21.15.32:0/427359048 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f760c0d1d20 con 0x7f760c15ce60 2023-12-20T04:44:36.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.861+0000 7f7610851640 1 --2- 172.21.15.32:0/427359048 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d3130 0x7f760c0d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:44:36.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f7611052640 1 --2- 172.21.15.32:0/427359048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f760c15ce60 0x7f760c0d2bf0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f760c0cfed0 tx=0x7f760801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b001fec7e5f6bf8b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:36.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f7603fff640 1 -- 172.21.15.32:0/427359048 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76080a89b0 con 0x7f760c15ce60 2023-12-20T04:44:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f7603fff640 1 -- 172.21.15.32:0/427359048 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76080a8b10 con 0x7f760c15ce60 2023-12-20T04:44:36.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f7603fff640 1 -- 172.21.15.32:0/427359048 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76080a8ea0 con 0x7f760c15ce60 2023-12-20T04:44:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f76132b6640 1 -- 172.21.15.32:0/427359048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f760c15ce60 msgr2=0x7f760c0d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f76132b6640 1 --2- 172.21.15.32:0/427359048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f760c15ce60 0x7f760c0d2bf0 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f760c0cfed0 tx=0x7f760801ecf0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f76132b6640 1 -- 172.21.15.32:0/427359048 shutdown_connections 2023-12-20T04:44:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f76132b6640 1 --2- 172.21.15.32:0/427359048 >> v2:172.21.15.39:3300/0 conn(0x7f760c0d1300 0x7f760c0b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f76132b6640 1 --2- 172.21.15.32:0/427359048 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d3130 0x7f760c0d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f76132b6640 1 --2- 172.21.15.32:0/427359048 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f760c15ce60 0x7f760c0d2bf0 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f76132b6640 1 -- 172.21.15.32:0/427359048 >> 172.21.15.32:0/427359048 conn(0x7f760c06b5e0 msgr2=0x7f760c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f76132b6640 1 -- 172.21.15.32:0/427359048 shutdown_connections 2023-12-20T04:44:36.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f76132b6640 1 -- 172.21.15.32:0/427359048 wait complete. 2023-12-20T04:44:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.865+0000 7f76132b6640 1 Processor -- start 2023-12-20T04:44:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f76132b6640 1 -- start start 2023-12-20T04:44:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f76132b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d1300 0x7f760c151a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f76132b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f760c0d3130 0x7f760c15d240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f76132b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f760c15ce60 0x7f760c155130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f76132b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c15e170 con 0x7f760c15ce60 2023-12-20T04:44:36.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f76132b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f760c1e5770 con 0x7f760c0d3130 2023-12-20T04:44:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f76132b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f760c151f70 con 0x7f760c0d1300 2023-12-20T04:44:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7611853640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f760c15ce60 0x7f760c155130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7611052640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d1300 0x7f760c151a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7611853640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f760c15ce60 0x7f760c155130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60726/0 (socket says 172.21.15.32:60726) 2023-12-20T04:44:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7611853640 1 -- 172.21.15.32:0/3389863424 learned_addr learned my addr 172.21.15.32:0/3389863424 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7610851640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f760c0d3130 0x7f760c15d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7611853640 1 -- 172.21.15.32:0/3389863424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d1300 msgr2=0x7f760c151a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7611853640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d1300 0x7f760c151a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7611853640 1 -- 172.21.15.32:0/3389863424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f760c0d3130 msgr2=0x7f760c15d240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7611853640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f760c0d3130 0x7f760c15d240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7611853640 1 -- 172.21.15.32:0/3389863424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76080094d0 con 0x7f760c15ce60 2023-12-20T04:44:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7611052640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d1300 0x7f760c151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7611853640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f760c15ce60 0x7f760c155130 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f760400eed0 tx=0x7f760400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7601ffb640 1 -- 172.21.15.32:0/3389863424 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604010040 con 0x7f760c15ce60 2023-12-20T04:44:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7601ffb640 1 -- 172.21.15.32:0/3389863424 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f760400bd10 con 0x7f760c15ce60 2023-12-20T04:44:36.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f76132b6640 1 -- 172.21.15.32:0/3389863424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f760c15dbb0 con 0x7f760c15ce60 2023-12-20T04:44:36.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f7601ffb640 1 -- 172.21.15.32:0/3389863424 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7604014640 con 0x7f760c15ce60 2023-12-20T04:44:36.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f76132b6640 1 -- 172.21.15.32:0/3389863424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f760c1e5980 con 0x7f760c15ce60 2023-12-20T04:44:36.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.869+0000 7f76132b6640 1 -- 172.21.15.32:0/3389863424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f760c04e000 con 0x7f760c15ce60 2023-12-20T04:44:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.873+0000 7f7601ffb640 1 -- 172.21.15.32:0/3389863424 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 48) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f76040147a0 con 0x7f760c15ce60 2023-12-20T04:44:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.873+0000 7f7601ffb640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.873+0000 7f7601ffb640 1 -- 172.21.15.32:0/3389863424 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(107..107 src has 1..107) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f7604083ca0 con 0x7f760c15ce60 2023-12-20T04:44:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.873+0000 7f7611052640 1 -- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 msgr2=0x7f75f0068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:36.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.873+0000 7f7611052640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:44:36.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:36.877+0000 7f7601ffb640 1 -- 172.21.15.32:0/3389863424 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f76040514f0 con 0x7f760c15ce60 2023-12-20T04:44:37.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:37.053+0000 7f76132b6640 1 -- 172.21.15.32:0/3389863424 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f760c0d22e0 con 0x7f75f0065bd0 2023-12-20T04:44:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:37.077+0000 7f7611052640 1 -- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 msgr2=0x7f75f0068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:37.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:37.077+0000 7f7611052640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:44:37.244 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:37.243+0000 7f2e0af4d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:37.283 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:37.281+0000 7f29f3448180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:37.306 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:37.307+0000 7f2e0af4d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:44:37.344 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:37.345+0000 7f29f3448180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:44:37.365 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:37.363+0000 7f2e0af4d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:44:37.403 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:37.405+0000 7f29f3448180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:44:37.458 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:37.459+0000 7f2e0af4d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:44:37.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:37.481+0000 7f7611052640 1 -- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 msgr2=0x7f75f0068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:37.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:37.481+0000 7f7611052640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T04:44:37.495 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:37.493+0000 7f29f3448180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:44:37.517 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:37.515+0000 7f2e0af4d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:44:37.554 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:37.553+0000 7f29f3448180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:44:37.607 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:37.607+0000 7f2e0af4d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:44:37.643 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:37.645+0000 7f29f3448180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:44:37.666 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:37.667+0000 7f2e0af4d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:44:37.702 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:37.701+0000 7f29f3448180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:44:37.818 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:37.819+0000 7f2e0af4d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:44:37.853 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:37.853+0000 7f29f3448180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:44:37.880 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:37.879+0000 7f2e0af4d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:44:37.915 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:37.917+0000 7f29f3448180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:44:38.132 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:44:38.132 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:44:38.132 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:44:38.141 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:38.139+0000 7f2e0af4d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:44:38.167 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:44:38.167 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:44:38.167 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:44:38.175 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:38.177+0000 7f29f3448180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:44:38.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:38.285+0000 7f7611052640 1 -- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 msgr2=0x7f75f0068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:38.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:38.285+0000 7f7611052640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T04:44:38.284 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:38.283+0000 7f2e0af4d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:38.318 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:38.317+0000 7f29f3448180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:38.417 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:38.415+0000 7f2e0af4d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:38.450 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:38.449+0000 7f29f3448180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:44:38.492 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:38.491+0000 7f2e0af4d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:44:38.524 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:38.525+0000 7f29f3448180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:44:38.636 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:38.635+0000 7f2e0af4d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:38.667 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:38.669+0000 7f29f3448180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:44:39.206 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:39.207+0000 7f2e0af4d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:44:39.238 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:39.237+0000 7f29f3448180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:44:39.295 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:39.295+0000 7f2e0af4d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:44:39.327 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:39.329+0000 7f29f3448180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:44:39.453 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:39.451+0000 7f2e0af4d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:44:39.484 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:39.485+0000 7f29f3448180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:44:39.542 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:39.543+0000 7f2e0af4d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:44:39.572 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:39.573+0000 7f29f3448180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:44:39.878 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:39.879+0000 7f2e0af4d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:44:39.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:39.889+0000 7f7611052640 1 -- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 msgr2=0x7f75f0068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:44:39.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:39.889+0000 7f7611052640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-20T04:44:39.908 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:39.909+0000 7f29f3448180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:44:39.941 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:44:39.939+0000 7f2e0af4d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:44:39.972 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:39.973+0000 7f29f3448180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:44:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:39.981+0000 7f7601ffb640 1 -- 172.21.15.32:0/3389863424 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 49) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f76040821c0 con 0x7f760c15ce60 2023-12-20T04:44:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:39.981+0000 7f7601ffb640 1 -- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 msgr2=0x7f75f0068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:44:39.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:39.981+0000 7f7601ffb640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:40.107 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:40.105+0000 7f29788f3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:40.107 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:40.105+0000 7f29758ed640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:40.107 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:40.105+0000 7f29758ed640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:40.108 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:40.105+0000 7f29758ed640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:40.108 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:40.105+0000 7f29758ed640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:40.108 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:40.105+0000 7f29758ed640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:40.108 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:40.105+0000 7f29788f3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:40.108 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:40.105+0000 7f29788f3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:40.108 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:40.105+0000 7f29788f3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:40.108 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:44:40.105+0000 7f29788f3640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:44:41.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.037+0000 7f7601ffb640 1 -- 172.21.15.32:0/3389863424 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 50) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f76040518c0 con 0x7f760c15ce60 2023-12-20T04:44:41.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.037+0000 7f7601ffb640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:41.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.037+0000 7f7601ffb640 1 -- 172.21.15.32:0/3389863424 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/bad-path", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f75f0033630 con 0x7f75f0065bd0 2023-12-20T04:44:41.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.037+0000 7f7611052640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:41.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.041+0000 7f7611052640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f76080b2000 tx=0x7f76080aa070 comp rx=0 tx=0).ready entity=mgr.7536 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:41.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.105+0000 7f7601ffb640 1 -- 172.21.15.32:0/3389863424 <== mgr.7536 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+32 (secure 0 0 0) 0x7f75f0033630 con 0x7f75f0065bd0 2023-12-20T04:44:41.107 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /bad-path 2023-12-20T04:44:41.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 -- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 msgr2=0x7f75f0068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f76080b2000 tx=0x7f76080aa070 comp rx=0 tx=0).stop 2023-12-20T04:44:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 -- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f760c15ce60 msgr2=0x7f760c155130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f760c15ce60 0x7f760c155130 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f760400eed0 tx=0x7f760400b480 comp rx=0 tx=0).stop 2023-12-20T04:44:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 -- 172.21.15.32:0/3389863424 shutdown_connections 2023-12-20T04:44:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f75f0065bd0 0x7f75f0068090 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f760c15ce60 0x7f760c155130 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f760c0d3130 0x7f760c15d240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 --2- 172.21.15.32:0/3389863424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f760c0d1300 0x7f760c151a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 -- 172.21.15.32:0/3389863424 >> 172.21.15.32:0/3389863424 conn(0x7f760c06b5e0 msgr2=0x7f760c0cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 -- 172.21.15.32:0/3389863424 shutdown_connections 2023-12-20T04:44:41.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.109+0000 7f76132b6640 1 -- 172.21.15.32:0/3389863424 wait complete. 2023-12-20T04:44:41.131 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /bad-path 2023-12-20T04:44:41.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-12-20T04:44:41.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.277+0000 7f68c4ac3640 1 Processor -- start 2023-12-20T04:44:41.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.277+0000 7f68c4ac3640 1 -- start start 2023-12-20T04:44:41.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.277+0000 7f68c4ac3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68c00b0010 0x7f68c00b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:41.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.277+0000 7f68c4ac3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f68c00b16c0 0x7f68c00b1b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:41.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.277+0000 7f68c4ac3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f68c00b09b0 con 0x7f68c015a920 2023-12-20T04:44:41.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.277+0000 7f68c4ac3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f68c00b4510 con 0x7f68c00b16c0 2023-12-20T04:44:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.277+0000 7f68c4ac3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c00cada0 con 0x7f68c00b0010 2023-12-20T04:44:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68bffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f68c00b16c0 0x7f68c00b1b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68beffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68c00b0010 0x7f68c00b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:41.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68bffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f68c00b16c0 0x7f68c00b1b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:46060/0 (socket says 172.21.15.32:46060) 2023-12-20T04:44:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68bffff640 1 -- 172.21.15.32:0/3177103289 learned_addr learned my addr 172.21.15.32:0/3177103289 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68bffff640 1 -- 172.21.15.32:0/3177103289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68c00b0010 msgr2=0x7f68c00b0470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:44:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68bffff640 1 --2- 172.21.15.32:0/3177103289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68c00b0010 0x7f68c00b0470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68bffff640 1 -- 172.21.15.32:0/3177103289 >> v1:172.21.15.39:6789/0 conn(0x7f68c015a920 legacy=0x7f68c00ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68bffff640 1 -- 172.21.15.32:0/3177103289 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68c00c9ff0 con 0x7f68c00b16c0 2023-12-20T04:44:41.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68beffd640 1 --2- 172.21.15.32:0/3177103289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68c00b0010 0x7f68c00b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:44:41.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68bffff640 1 --2- 172.21.15.32:0/3177103289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c00b16c0 0x7f68c00b1b30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009e50 tx=0x7f68b801c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b50e418021a635e8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68be7fc640 1 -- 172.21.15.32:0/3177103289 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b80a58a0 con 0x7f68c00b16c0 2023-12-20T04:44:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68be7fc640 1 -- 172.21.15.32:0/3177103289 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b80a5a00 con 0x7f68c00b16c0 2023-12-20T04:44:41.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.281+0000 7f68be7fc640 1 -- 172.21.15.32:0/3177103289 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b80a5d90 con 0x7f68c00b16c0 2023-12-20T04:44:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3177103289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c00b16c0 msgr2=0x7f68c00b1b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 --2- 172.21.15.32:0/3177103289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c00b16c0 0x7f68c00b1b30 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7f68b8009e50 tx=0x7f68b801c9e0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3177103289 shutdown_connections 2023-12-20T04:44:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 --2- 172.21.15.32:0/3177103289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c00b16c0 0x7f68c00b1b30 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 --2- 172.21.15.32:0/3177103289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68c00b0010 0x7f68c00b0470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3177103289 >> 172.21.15.32:0/3177103289 conn(0x7f68c006c060 msgr2=0x7f68c006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3177103289 shutdown_connections 2023-12-20T04:44:41.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3177103289 wait complete. 2023-12-20T04:44:41.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 Processor -- start 2023-12-20T04:44:41.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 -- start start 2023-12-20T04:44:41.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68c00b0010 0x7f68c01502a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:41.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c00b16c0 0x7f68c0159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:41.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c015a920 0x7f68c00ae800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:41.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c014d7a0 con 0x7f68c00b16c0 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f68c00ccb30 con 0x7f68c015a920 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68c4ac3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f68c015a0d0 con 0x7f68c00b0010 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68beffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c00b16c0 0x7f68c0159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.285+0000 7f68beffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c00b16c0 0x7f68c0159610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34830/0 (socket says 172.21.15.32:34830) 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68beffd640 1 -- 172.21.15.32:0/3273357837 learned_addr learned my addr 172.21.15.32:0/3273357837 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68bffff640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c015a920 0x7f68c00ae800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68bf7fe640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68c00b0010 0x7f68c01502a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68beffd640 1 -- 172.21.15.32:0/3273357837 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68c00b0010 msgr2=0x7f68c01502a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68beffd640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68c00b0010 0x7f68c01502a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68beffd640 1 -- 172.21.15.32:0/3273357837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c015a920 msgr2=0x7f68c00ae800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68beffd640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c015a920 0x7f68c00ae800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68beffd640 1 -- 172.21.15.32:0/3273357837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f68b4009420 con 0x7f68c00b16c0 2023-12-20T04:44:41.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68beffd640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c00b16c0 0x7f68c0159610 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f68b400ebe0 tx=0x7f68b400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68a7fff640 1 -- 172.21.15.32:0/3273357837 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b400eee0 con 0x7f68c00b16c0 2023-12-20T04:44:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68a7fff640 1 -- 172.21.15.32:0/3273357837 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68b400cc30 con 0x7f68c00b16c0 2023-12-20T04:44:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68a7fff640 1 -- 172.21.15.32:0/3273357837 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68b4013470 con 0x7f68c00b16c0 2023-12-20T04:44:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3273357837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f68b8009b30 con 0x7f68c00b16c0 2023-12-20T04:44:41.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3273357837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f68c00cc4e0 con 0x7f68c00b16c0 2023-12-20T04:44:41.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3273357837 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f68c0074100 con 0x7f68c00b16c0 2023-12-20T04:44:41.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68a7fff640 1 -- 172.21.15.32:0/3273357837 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 50) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f68b40135d0 con 0x7f68c00b16c0 2023-12-20T04:44:41.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68a7fff640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68ac065930 0x7f68ac067df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:44:41.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.289+0000 7f68a7fff640 1 -- 172.21.15.32:0/3273357837 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(108..108 src has 1..108) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f68b4083160 con 0x7f68c00b16c0 2023-12-20T04:44:41.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.293+0000 7f68bf7fe640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68ac065930 0x7f68ac067df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:44:41.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.293+0000 7f68bf7fe640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68ac065930 0x7f68ac067df0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f68b0007fd0 tx=0x7f68b0009040 comp rx=0 tx=0).ready entity=mgr.7536 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:44:41.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.297+0000 7f68a7fff640 1 -- 172.21.15.32:0/3273357837 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f68b4050db0 con 0x7f68c00b16c0 2023-12-20T04:44:41.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.469+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3273357837 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f68c00c9c80 con 0x7f68ac065930 2023-12-20T04:44:41.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68a7fff640 1 -- 172.21.15.32:0/3273357837 <== mgr.7536 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+287 (secure 0 0 0) 0x7f68b0001450 con 0x7f68ac065930 2023-12-20T04:44:41.477 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:44:41.477 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:44:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3273357837 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68ac065930 msgr2=0x7f68ac067df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68ac065930 0x7f68ac067df0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f68b0007fd0 tx=0x7f68b0009040 comp rx=0 tx=0).stop 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3273357837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c00b16c0 msgr2=0x7f68c0159610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c00b16c0 0x7f68c0159610 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7f68b400ebe0 tx=0x7f68b400c520 comp rx=0 tx=0).stop 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3273357837 shutdown_connections 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f68ac065930 0x7f68ac067df0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f68c015a920 0x7f68c00ae800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f68c00b16c0 0x7f68c0159610 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 --2- 172.21.15.32:0/3273357837 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f68c00b0010 0x7f68c01502a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3273357837 >> 172.21.15.32:0/3273357837 conn(0x7f68c006c060 msgr2=0x7f68c0156f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3273357837 shutdown_connections 2023-12-20T04:44:41.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:44:41.477+0000 7f68c4ac3640 1 -- 172.21.15.32:0/3273357837 wait complete. 2023-12-20T04:44:41.498 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:44:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-20T04:45:00.120 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:00.120+0000 7f29680d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:00.120 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:00.120+0000 7f29680d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:00.120 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:00.120+0000 7f29680d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:00.120 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:00.120+0000 7f29680d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:00.121 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:00.120+0000 7f29680d2640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:01.328 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:45:01.332 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:45:31.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:45:31.405 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:45:41.500 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /bad-path 2023-12-20T04:45:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.660+0000 7f759d404640 1 Processor -- start 2023-12-20T04:45:41.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.660+0000 7f759d404640 1 -- start start 2023-12-20T04:45:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f759d404640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 0x7f7598164c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f759d404640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f75980b0750 0x7f75980cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f759d404640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75980cc8d0 con 0x7f75980701c0 2023-12-20T04:45:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f759d404640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75981656c0 con 0x7f75980b0750 2023-12-20T04:45:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f759d404640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75981659a0 con 0x7f7598071ea0 2023-12-20T04:45:41.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f7596ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 0x7f7598164c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f75977fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f75980b0750 0x7f75980cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f758ffff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f75980701c0 0x7f75980b0040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:40910/0 (socket says 172.21.15.32:40910) 2023-12-20T04:45:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f75977fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f75980b0750 0x7f75980cd300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:41350/0 (socket says 172.21.15.32:41350) 2023-12-20T04:45:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f7596ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 0x7f7598164c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45086/0 (socket says 172.21.15.32:45086) 2023-12-20T04:45:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f758ffff640 1 -- 172.21.15.32:0/1572516186 learned_addr learned my addr 172.21.15.32:0/1572516186 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:41.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f75967fc640 1 -- 172.21.15.32:0/1572516186 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2366346764 0 0) 0x7f75980cc8d0 con 0x7f75980701c0 2023-12-20T04:45:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f75967fc640 1 -- 172.21.15.32:0/1572516186 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7584003530 con 0x7f75980701c0 2023-12-20T04:45:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f7596ffd640 1 -- 172.21.15.32:0/1572516186 >> v2:172.21.15.39:3300/0 conn(0x7f75980b0750 msgr2=0x7f75980cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f7596ffd640 1 --2- 172.21.15.32:0/1572516186 >> v2:172.21.15.39:3300/0 conn(0x7f75980b0750 0x7f75980cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f7596ffd640 1 -- 172.21.15.32:0/1572516186 >> v1:172.21.15.39:6789/0 conn(0x7f75980701c0 legacy=0x7f75980b0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f7596ffd640 1 -- 172.21.15.32:0/1572516186 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75980ccb20 con 0x7f7598071ea0 2023-12-20T04:45:41.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.664+0000 7f75977fe640 1 --2- 172.21.15.32:0/1572516186 >> v2:172.21.15.39:3300/0 conn(0x7f75980b0750 0x7f75980cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f7596ffd640 1 --2- 172.21.15.32:0/1572516186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 0x7f7598164c00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f75880097c0 tx=0x7f758801c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8028861248544634 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f75967fc640 1 -- 172.21.15.32:0/1572516186 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75880ab670 con 0x7f7598071ea0 2023-12-20T04:45:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f75967fc640 1 -- 172.21.15.32:0/1572516186 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75880ab7d0 con 0x7f7598071ea0 2023-12-20T04:45:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f75967fc640 1 -- 172.21.15.32:0/1572516186 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75880abb60 con 0x7f7598071ea0 2023-12-20T04:45:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f759d404640 1 -- 172.21.15.32:0/1572516186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 msgr2=0x7f7598164c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:41.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f759d404640 1 --2- 172.21.15.32:0/1572516186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 0x7f7598164c00 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f75880097c0 tx=0x7f758801c450 comp rx=0 tx=0).stop 2023-12-20T04:45:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f759d404640 1 -- 172.21.15.32:0/1572516186 shutdown_connections 2023-12-20T04:45:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f759d404640 1 --2- 172.21.15.32:0/1572516186 >> v2:172.21.15.39:3300/0 conn(0x7f75980b0750 0x7f75980cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f759d404640 1 --2- 172.21.15.32:0/1572516186 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 0x7f7598164c00 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f759d404640 1 -- 172.21.15.32:0/1572516186 >> 172.21.15.32:0/1572516186 conn(0x7f759806bfe0 msgr2=0x7f759806c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f759d404640 1 -- 172.21.15.32:0/1572516186 shutdown_connections 2023-12-20T04:45:41.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f759d404640 1 -- 172.21.15.32:0/1572516186 wait complete. 2023-12-20T04:45:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f759d404640 1 Processor -- start 2023-12-20T04:45:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.668+0000 7f759d404640 1 -- start start 2023-12-20T04:45:41.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f759d404640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 0x7f7598157f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:41.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f759d404640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75980b0750 0x7f75980ae6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:41.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f759d404640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75980c2d70 0x7f75980aec20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:41.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f759d404640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75980c4fd0 con 0x7f75980b0750 2023-12-20T04:45:41.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f759d404640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7598159ec0 con 0x7f75980c2d70 2023-12-20T04:45:41.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f759d404640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7598158440 con 0x7f7598071ea0 2023-12-20T04:45:41.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75980b0750 0x7f75980ae6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:41.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75980b0750 0x7f75980ae6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53106/0 (socket says 172.21.15.32:53106) 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758ffff640 1 -- 172.21.15.32:0/3915682134 learned_addr learned my addr 172.21.15.32:0/3915682134 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f7596ffd640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 0x7f7598157f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f75977fe640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75980c2d70 0x7f75980aec20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758ffff640 1 -- 172.21.15.32:0/3915682134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 msgr2=0x7f7598157f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758ffff640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 0x7f7598157f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758ffff640 1 -- 172.21.15.32:0/3915682134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75980c2d70 msgr2=0x7f75980aec20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758ffff640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75980c2d70 0x7f75980aec20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758ffff640 1 -- 172.21.15.32:0/3915682134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7590009a70 con 0x7f75980b0750 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f75977fe640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75980c2d70 0x7f75980aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758ffff640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75980b0750 0x7f75980ae6e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f758000c4d0 tx=0x7f758000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758f7fe640 1 -- 172.21.15.32:0/3915682134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580005cf0 con 0x7f75980b0750 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758f7fe640 1 -- 172.21.15.32:0/3915682134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7580005e50 con 0x7f75980b0750 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f759d404640 1 -- 172.21.15.32:0/3915682134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7588009420 con 0x7f75980b0750 2023-12-20T04:45:41.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f758f7fe640 1 -- 172.21.15.32:0/3915682134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7580007270 con 0x7f75980b0750 2023-12-20T04:45:41.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f759d404640 1 -- 172.21.15.32:0/3915682134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75980c2290 con 0x7f75980b0750 2023-12-20T04:45:41.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.672+0000 7f759d404640 1 -- 172.21.15.32:0/3915682134 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7598074100 con 0x7f75980b0750 2023-12-20T04:45:41.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.676+0000 7f758f7fe640 1 -- 172.21.15.32:0/3915682134 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 53) v1 ==== 85461+0+0 (secure 0 0 0) 0x7f75800073d0 con 0x7f75980b0750 2023-12-20T04:45:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.676+0000 7f758f7fe640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7578065cf0 0x7f75780681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.676+0000 7f758f7fe640 1 -- 172.21.15.32:0/3915682134 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f7580082d70 con 0x7f75980b0750 2023-12-20T04:45:41.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.676+0000 7f7596ffd640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7578065cf0 0x7f75780681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:41.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.676+0000 7f7596ffd640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7578065cf0 0x7f75780681b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f75880af000 tx=0x7f75880a7000 comp rx=0 tx=0).ready entity=mgr.7536 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:41.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.680+0000 7f758f7fe640 1 -- 172.21.15.32:0/3915682134 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7580050250 con 0x7f75980b0750 2023-12-20T04:45:41.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.856+0000 7f759d404640 1 -- 172.21.15.32:0/3915682134 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/bad-path", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f75980c1960 con 0x7f7578065cf0 2023-12-20T04:45:41.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.860+0000 7f758f7fe640 1 -- 172.21.15.32:0/3915682134 <== mgr.7536 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+288 (secure 0 0 0) 0x7f75880ab940 con 0x7f7578065cf0 2023-12-20T04:45:41.858 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:41.859 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:44:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-12-20T04:45:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 -- 172.21.15.32:0/3915682134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7578065cf0 msgr2=0x7f75780681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7578065cf0 0x7f75780681b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f75880af000 tx=0x7f75880a7000 comp rx=0 tx=0).stop 2023-12-20T04:45:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 -- 172.21.15.32:0/3915682134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75980b0750 msgr2=0x7f75980ae6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75980b0750 0x7f75980ae6e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7f758000c4d0 tx=0x7f758000c990 comp rx=0 tx=0).stop 2023-12-20T04:45:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 -- 172.21.15.32:0/3915682134 shutdown_connections 2023-12-20T04:45:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7578065cf0 0x7f75780681b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:41.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75980c2d70 0x7f75980aec20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75980b0750 0x7f75980ae6e0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 --2- 172.21.15.32:0/3915682134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7598071ea0 0x7f7598157f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 -- 172.21.15.32:0/3915682134 >> 172.21.15.32:0/3915682134 conn(0x7f759806bfe0 msgr2=0x7f75981616d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 -- 172.21.15.32:0/3915682134 shutdown_connections 2023-12-20T04:45:41.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:41.864+0000 7f759d404640 1 -- 172.21.15.32:0/3915682134 wait complete. 2023-12-20T04:45:41.880 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/bad-path", "rel_path": "/bad-path", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:44:41", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": false}] 2023-12-20T04:45:41.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /bad-path 2023-12-20T04:45:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.028+0000 7f38f3ff3640 1 Processor -- start 2023-12-20T04:45:42.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f3ff3640 1 -- start start 2023-12-20T04:45:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f3ff3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38ec0b0bc0 0x7f38ec0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f3ff3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec0cde20 con 0x7f38ec0b0760 2023-12-20T04:45:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f3ff3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38ec0c8e30 con 0x7f38ec157480 2023-12-20T04:45:42.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f3ff3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38ec0c9f50 con 0x7f38ec0b0bc0 2023-12-20T04:45:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f158e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f38ec0b0760 0x7f38ec0cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54138/0 (socket says 172.21.15.32:54138) 2023-12-20T04:45:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f158e640 1 -- 172.21.15.32:0/2618879655 learned_addr learned my addr 172.21.15.32:0/2618879655 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f2590640 1 --2- 172.21.15.32:0/2618879655 >> v2:172.21.15.32:3300/0 conn(0x7f38ec0b0bc0 0x7f38ec0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f0d8d640 1 -- 172.21.15.32:0/2618879655 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3215999501 0 0) 0x7f38ec0cde20 con 0x7f38ec0b0760 2023-12-20T04:45:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f0d8d640 1 -- 172.21.15.32:0/2618879655 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f38d8005740 con 0x7f38ec0b0760 2023-12-20T04:45:42.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f2590640 1 -- 172.21.15.32:0/2618879655 >> v1:172.21.15.39:6789/0 conn(0x7f38ec157480 legacy=0x7f38ec0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f2590640 1 -- 172.21.15.32:0/2618879655 >> v1:172.21.15.32:6789/0 conn(0x7f38ec0b0760 legacy=0x7f38ec0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:42.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.032+0000 7f38f2590640 1 -- 172.21.15.32:0/2618879655 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38ec0ce070 con 0x7f38ec0b0bc0 2023-12-20T04:45:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f2590640 1 --2- 172.21.15.32:0/2618879655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec0b0bc0 0x7f38ec0ccbc0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009d90 tx=0x7f38e401c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=977b4bece180a06f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f0d8d640 1 -- 172.21.15.32:0/2618879655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e40a58d0 con 0x7f38ec0b0bc0 2023-12-20T04:45:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f0d8d640 1 -- 172.21.15.32:0/2618879655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e40a5a30 con 0x7f38ec0b0bc0 2023-12-20T04:45:42.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f0d8d640 1 -- 172.21.15.32:0/2618879655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e40a5dc0 con 0x7f38ec0b0bc0 2023-12-20T04:45:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2618879655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec0b0bc0 msgr2=0x7f38ec0ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f3ff3640 1 --2- 172.21.15.32:0/2618879655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec0b0bc0 0x7f38ec0ccbc0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x7f38e4009d90 tx=0x7f38e401c860 comp rx=0 tx=0).stop 2023-12-20T04:45:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2618879655 shutdown_connections 2023-12-20T04:45:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f3ff3640 1 --2- 172.21.15.32:0/2618879655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec0b0bc0 0x7f38ec0ccbc0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2618879655 >> 172.21.15.32:0/2618879655 conn(0x7f38ec06a510 msgr2=0x7f38ec0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2618879655 shutdown_connections 2023-12-20T04:45:42.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2618879655 wait complete. 2023-12-20T04:45:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f3ff3640 1 Processor -- start 2023-12-20T04:45:42.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.036+0000 7f38f3ff3640 1 -- start start 2023-12-20T04:45:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f3ff3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec0b0760 0x7f38ec0b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f3ff3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38ec0b0bc0 0x7f38ec1f49e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f3ff3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38ec157480 0x7f38ec0b4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f3ff3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec1f4f20 con 0x7f38ec0b0760 2023-12-20T04:45:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f3ff3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0b3d80 con 0x7f38ec0b0bc0 2023-12-20T04:45:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f3ff3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38ec0b4750 con 0x7f38ec157480 2023-12-20T04:45:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f1d8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec0b0760 0x7f38ec0b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:42.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f1d8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec0b0760 0x7f38ec0b18f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53130/0 (socket says 172.21.15.32:53130) 2023-12-20T04:45:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f1d8f640 1 -- 172.21.15.32:0/2613606166 learned_addr learned my addr 172.21.15.32:0/2613606166 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f158e640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38ec0b0bc0 0x7f38ec1f49e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f2590640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38ec157480 0x7f38ec0b4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f1d8f640 1 -- 172.21.15.32:0/2613606166 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38ec157480 msgr2=0x7f38ec0b4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f1d8f640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38ec157480 0x7f38ec0b4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f1d8f640 1 -- 172.21.15.32:0/2613606166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38ec0b0bc0 msgr2=0x7f38ec1f49e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f1d8f640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38ec0b0bc0 0x7f38ec1f49e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f1d8f640 1 -- 172.21.15.32:0/2613606166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e4009a70 con 0x7f38ec0b0760 2023-12-20T04:45:42.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f1d8f640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec0b0760 0x7f38ec0b18f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c3c0 tx=0x7f38e800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38e27fc640 1 -- 172.21.15.32:0/2613606166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80052d0 con 0x7f38ec0b0760 2023-12-20T04:45:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38e27fc640 1 -- 172.21.15.32:0/2613606166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e8005430 con 0x7f38ec0b0760 2023-12-20T04:45:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38e27fc640 1 -- 172.21.15.32:0/2613606166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e8003840 con 0x7f38ec0b0760 2023-12-20T04:45:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2613606166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38ec164160 con 0x7f38ec0b0760 2023-12-20T04:45:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2613606166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38ec156980 con 0x7f38ec0b0760 2023-12-20T04:45:42.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.040+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2613606166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38ec04e000 con 0x7f38ec0b0760 2023-12-20T04:45:42.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.044+0000 7f38e27fc640 1 -- 172.21.15.32:0/2613606166 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 53) v1 ==== 85461+0+0 (secure 0 0 0) 0x7f38e80039a0 con 0x7f38ec0b0760 2023-12-20T04:45:42.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.044+0000 7f38e27fc640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f38d0065cf0 0x7f38d00681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:42.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.044+0000 7f38e27fc640 1 -- 172.21.15.32:0/2613606166 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f38e8082ba0 con 0x7f38ec0b0760 2023-12-20T04:45:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.044+0000 7f38f158e640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f38d0065cf0 0x7f38d00681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:42.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.044+0000 7f38f158e640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f38d0065cf0 0x7f38d00681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38ec149110 tx=0x7f38dc009040 comp rx=0 tx=0).ready entity=mgr.7536 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:42.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.048+0000 7f38e27fc640 1 -- 172.21.15.32:0/2613606166 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f38e80501d0 con 0x7f38ec0b0760 2023-12-20T04:45:42.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.232+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2613606166 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/bad-path", "target": ["mon-mgr", ""]}) v1 -- 0x7f38ec0b0fe0 con 0x7f38d0065cf0 2023-12-20T04:45:42.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38e27fc640 1 -- 172.21.15.32:0/2613606166 <== mgr.7536 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+36 (secure 0 0 0) 0x7f38dc001400 con 0x7f38d0065cf0 2023-12-20T04:45:42.296 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /bad-path 2023-12-20T04:45:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2613606166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f38d0065cf0 msgr2=0x7f38d00681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f38d0065cf0 0x7f38d00681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38ec149110 tx=0x7f38dc009040 comp rx=0 tx=0).stop 2023-12-20T04:45:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2613606166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec0b0760 msgr2=0x7f38ec0b18f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec0b0760 0x7f38ec0b18f0 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f38e800c3c0 tx=0x7f38e800c880 comp rx=0 tx=0).stop 2023-12-20T04:45:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2613606166 shutdown_connections 2023-12-20T04:45:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f38d0065cf0 0x7f38d00681b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38ec157480 0x7f38ec0b4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38ec0b0bc0 0x7f38ec1f49e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 --2- 172.21.15.32:0/2613606166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38ec0b0760 0x7f38ec0b18f0 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2613606166 >> 172.21.15.32:0/2613606166 conn(0x7f38ec06a510 msgr2=0x7f38ec06c870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2613606166 shutdown_connections 2023-12-20T04:45:42.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.300+0000 7f38f3ff3640 1 -- 172.21.15.32:0/2613606166 wait complete. 2023-12-20T04:45:42.317 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /bad-path 2023-12-20T04:45:42.317 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T04:45:42.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb35da1f640 1 Processor -- start 2023-12-20T04:45:42.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb35da1f640 1 -- start start 2023-12-20T04:45:42.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb35da1f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb3580af400 0x7fb3580b1960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:42.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb35da1f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3580cb300 0x7fb3580b1ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb35da1f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3580ce7c0 con 0x7fb3580cdc60 2023-12-20T04:45:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb35da1f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3580af7e0 con 0x7fb3580af400 2023-12-20T04:45:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb35da1f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb358159cc0 con 0x7fb3580cb300 2023-12-20T04:45:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb3567fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3580cb300 0x7fb3580b1ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb356ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb3580af400 0x7fb3580b1960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb357fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fb3580cdc60 0x7fb3580ce0b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:40918/0 (socket says 172.21.15.32:40918) 2023-12-20T04:45:42.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb357fff640 1 -- 172.21.15.32:0/867944605 learned_addr learned my addr 172.21.15.32:0/867944605 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb355ffb640 1 -- 172.21.15.32:0/867944605 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3059448625 0 0) 0x7fb3580ce7c0 con 0x7fb3580cdc60 2023-12-20T04:45:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb355ffb640 1 -- 172.21.15.32:0/867944605 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb340003530 con 0x7fb3580cdc60 2023-12-20T04:45:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb3567fc640 1 -- 172.21.15.32:0/867944605 >> v2:172.21.15.39:3300/0 conn(0x7fb3580af400 msgr2=0x7fb3580b1960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb3567fc640 1 --2- 172.21.15.32:0/867944605 >> v2:172.21.15.39:3300/0 conn(0x7fb3580af400 0x7fb3580b1960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb3567fc640 1 -- 172.21.15.32:0/867944605 >> v1:172.21.15.39:6789/0 conn(0x7fb3580cdc60 legacy=0x7fb3580ce0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb3567fc640 1 -- 172.21.15.32:0/867944605 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3580cea30 con 0x7fb3580cb300 2023-12-20T04:45:42.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.468+0000 7fb356ffd640 1 --2- 172.21.15.32:0/867944605 >> v2:172.21.15.39:3300/0 conn(0x7fb3580af400 0x7fb3580b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:42.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb3567fc640 1 --2- 172.21.15.32:0/867944605 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3580cb300 0x7fb3580b1ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0097c0 tx=0x7fb34c01c450 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9d0354ba17545a03 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb355ffb640 1 -- 172.21.15.32:0/867944605 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c004230 con 0x7fb3580cb300 2023-12-20T04:45:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb355ffb640 1 -- 172.21.15.32:0/867944605 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34c0a7050 con 0x7fb3580cb300 2023-12-20T04:45:42.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb355ffb640 1 -- 172.21.15.32:0/867944605 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34c003620 con 0x7fb3580cb300 2023-12-20T04:45:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb35da1f640 1 -- 172.21.15.32:0/867944605 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3580cb300 msgr2=0x7fb3580b1ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb35da1f640 1 --2- 172.21.15.32:0/867944605 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3580cb300 0x7fb3580b1ea0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0097c0 tx=0x7fb34c01c450 comp rx=0 tx=0).stop 2023-12-20T04:45:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb35da1f640 1 -- 172.21.15.32:0/867944605 shutdown_connections 2023-12-20T04:45:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb35da1f640 1 --2- 172.21.15.32:0/867944605 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3580cb300 0x7fb3580b1ea0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb35da1f640 1 --2- 172.21.15.32:0/867944605 >> v2:172.21.15.39:3300/0 conn(0x7fb3580af400 0x7fb3580b1960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb35da1f640 1 -- 172.21.15.32:0/867944605 >> 172.21.15.32:0/867944605 conn(0x7fb358071aa0 msgr2=0x7fb3580cfee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb35da1f640 1 -- 172.21.15.32:0/867944605 shutdown_connections 2023-12-20T04:45:42.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.472+0000 7fb35da1f640 1 -- 172.21.15.32:0/867944605 wait complete. 2023-12-20T04:45:42.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb35da1f640 1 Processor -- start 2023-12-20T04:45:42.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb35da1f640 1 -- start start 2023-12-20T04:45:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb35da1f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3580af400 0x7fb358155270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb35da1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3580cb300 0x7fb3581f4790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb35da1f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3580cdc60 0x7fb3581f4cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb35da1f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35815bbe0 con 0x7fb3580af400 2023-12-20T04:45:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb35da1f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb35815ad40 con 0x7fb3580cb300 2023-12-20T04:45:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb35da1f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb35815be30 con 0x7fb3580cdc60 2023-12-20T04:45:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb356ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3580af400 0x7fb358155270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb356ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3580af400 0x7fb358155270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53146/0 (socket says 172.21.15.32:53146) 2023-12-20T04:45:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb356ffd640 1 -- 172.21.15.32:0/1233484583 learned_addr learned my addr 172.21.15.32:0/1233484583 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb3567fc640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3580cb300 0x7fb3581f4790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:42.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb357fff640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3580cdc60 0x7fb3581f4cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb356ffd640 1 -- 172.21.15.32:0/1233484583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3580cdc60 msgr2=0x7fb3581f4cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb356ffd640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3580cdc60 0x7fb3581f4cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb356ffd640 1 -- 172.21.15.32:0/1233484583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3580cb300 msgr2=0x7fb3581f4790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb356ffd640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3580cb300 0x7fb3581f4790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb356ffd640 1 -- 172.21.15.32:0/1233484583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb348009420 con 0x7fb3580af400 2023-12-20T04:45:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb3567fc640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3580cb300 0x7fb3581f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb356ffd640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3580af400 0x7fb358155270 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fb34800ebe0 tx=0x7fb34800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb33ffff640 1 -- 172.21.15.32:0/1233484583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb34800eee0 con 0x7fb3580af400 2023-12-20T04:45:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb33ffff640 1 -- 172.21.15.32:0/1233484583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb34800cc30 con 0x7fb3580af400 2023-12-20T04:45:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb33ffff640 1 -- 172.21.15.32:0/1233484583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb348013470 con 0x7fb3580af400 2023-12-20T04:45:42.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb35da1f640 1 -- 172.21.15.32:0/1233484583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb34c009420 con 0x7fb3580af400 2023-12-20T04:45:42.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.476+0000 7fb35da1f640 1 -- 172.21.15.32:0/1233484583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb35815c3f0 con 0x7fb3580af400 2023-12-20T04:45:42.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.480+0000 7fb35da1f640 1 -- 172.21.15.32:0/1233484583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3580caae0 con 0x7fb3580af400 2023-12-20T04:45:42.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.480+0000 7fb33ffff640 1 -- 172.21.15.32:0/1233484583 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 53) v1 ==== 85461+0+0 (secure 0 0 0) 0x7fb3480135d0 con 0x7fb3580af400 2023-12-20T04:45:42.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.480+0000 7fb33ffff640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb344065cf0 0x7fb3440681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:42.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.480+0000 7fb33ffff640 1 -- 172.21.15.32:0/1233484583 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fb348083b20 con 0x7fb3580af400 2023-12-20T04:45:42.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.480+0000 7fb3567fc640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb344065cf0 0x7fb3440681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:42.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.480+0000 7fb3567fc640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb344065cf0 0x7fb3440681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0097c0 tx=0x7fb34c0a6000 comp rx=0 tx=0).ready entity=mgr.7536 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:42.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.484+0000 7fb33ffff640 1 -- 172.21.15.32:0/1233484583 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb3580caae0 con 0x7fb3580af400 2023-12-20T04:45:42.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:42.680+0000 7fb35da1f640 1 -- 172.21.15.32:0/1233484583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fb3580caae0 con 0x7fb3580af400 2023-12-20T04:45:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.120+0000 7fb33ffff640 1 -- 172.21.15.32:0/1233484583 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 54) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fb3480513d0 con 0x7fb3580af400 2023-12-20T04:45:43.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.120+0000 7fb33ffff640 1 -- 172.21.15.32:0/1233484583 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v54) v1 ==== 93+0+0 (secure 0 0 0) 0x7fb34807fb40 con 0x7fb3580af400 2023-12-20T04:45:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 -- 172.21.15.32:0/1233484583 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb344065cf0 msgr2=0x7fb3440681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb344065cf0 0x7fb3440681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb34c0097c0 tx=0x7fb34c0a6000 comp rx=0 tx=0).stop 2023-12-20T04:45:43.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 -- 172.21.15.32:0/1233484583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3580af400 msgr2=0x7fb358155270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3580af400 0x7fb358155270 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7fb34800ebe0 tx=0x7fb34800c520 comp rx=0 tx=0).stop 2023-12-20T04:45:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 -- 172.21.15.32:0/1233484583 shutdown_connections 2023-12-20T04:45:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb344065cf0 0x7fb3440681b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3580cdc60 0x7fb3581f4cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3580cb300 0x7fb3581f4790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 --2- 172.21.15.32:0/1233484583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3580af400 0x7fb358155270 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 -- 172.21.15.32:0/1233484583 >> 172.21.15.32:0/1233484583 conn(0x7fb358071aa0 msgr2=0x7fb3580c6270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 -- 172.21.15.32:0/1233484583 shutdown_connections 2023-12-20T04:45:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.124+0000 7fb35da1f640 1 -- 172.21.15.32:0/1233484583 wait complete. 2023-12-20T04:45:43.139 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:45:43.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:45:43.230 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:45:43.294 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:43.298+0000 7fb0d0af0180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:45:43.299 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:43.304+0000 7f366ca48180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:45:43.320 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:45:43.320 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:45:43.320 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:45:43.329 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:45:43.330 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:45:43.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:45:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.568+0000 7f9be81f0640 1 Processor -- start 2023-12-20T04:45:43.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.568+0000 7f9be81f0640 1 -- start start 2023-12-20T04:45:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be81f0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9be015f110 0x7f9be015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be81f0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be015b990 con 0x7f9be006ce60 2023-12-20T04:45:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be81f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9be015bbd0 con 0x7f9be00af5e0 2023-12-20T04:45:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be81f0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9be015beb0 con 0x7f9be015f110 2023-12-20T04:45:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be578b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9be006ce60 0x7f9be015b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54152/0 (socket says 172.21.15.32:54152) 2023-12-20T04:45:43.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be578b640 1 -- 172.21.15.32:0/3490125263 learned_addr learned my addr 172.21.15.32:0/3490125263 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be678d640 1 --2- 172.21.15.32:0/3490125263 >> v2:172.21.15.39:3300/0 conn(0x7f9be015f110 0x7f9be015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:43.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be4f8a640 1 -- 172.21.15.32:0/3490125263 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2868280563 0 0) 0x7f9bd0001120 con 0x7f9be006ce60 2023-12-20T04:45:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be4f8a640 1 -- 172.21.15.32:0/3490125263 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bcc003530 con 0x7f9be006ce60 2023-12-20T04:45:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be4f8a640 1 -- 172.21.15.32:0/3490125263 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 982601591 0 0) 0x7f9be015bbd0 con 0x7f9be00af5e0 2023-12-20T04:45:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be4f8a640 1 -- 172.21.15.32:0/3490125263 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9bd0001120 con 0x7f9be00af5e0 2023-12-20T04:45:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be678d640 1 -- 172.21.15.32:0/3490125263 >> v1:172.21.15.39:6789/0 conn(0x7f9be00af5e0 legacy=0x7f9be00af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be678d640 1 -- 172.21.15.32:0/3490125263 >> v1:172.21.15.32:6789/0 conn(0x7f9be006ce60 legacy=0x7f9be015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.572+0000 7f9be678d640 1 -- 172.21.15.32:0/3490125263 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9be015e910 con 0x7f9be015f110 2023-12-20T04:45:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be678d640 1 --2- 172.21.15.32:0/3490125263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be015f110 0x7f9be015c440 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009d90 tx=0x7f9bd801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=78adab6ec0ef1f1e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be4f8a640 1 -- 172.21.15.32:0/3490125263 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80a58f0 con 0x7f9be015f110 2023-12-20T04:45:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be4f8a640 1 -- 172.21.15.32:0/3490125263 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd80a5a50 con 0x7f9be015f110 2023-12-20T04:45:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be4f8a640 1 -- 172.21.15.32:0/3490125263 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80a5de0 con 0x7f9be015f110 2023-12-20T04:45:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be81f0640 1 -- 172.21.15.32:0/3490125263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be015f110 msgr2=0x7f9be015c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be81f0640 1 --2- 172.21.15.32:0/3490125263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be015f110 0x7f9be015c440 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7f9bd8009d90 tx=0x7f9bd801c890 comp rx=0 tx=0).stop 2023-12-20T04:45:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be81f0640 1 -- 172.21.15.32:0/3490125263 shutdown_connections 2023-12-20T04:45:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be81f0640 1 --2- 172.21.15.32:0/3490125263 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be015f110 0x7f9be015c440 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be81f0640 1 -- 172.21.15.32:0/3490125263 >> 172.21.15.32:0/3490125263 conn(0x7f9be006a320 msgr2=0x7f9be00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be81f0640 1 -- 172.21.15.32:0/3490125263 shutdown_connections 2023-12-20T04:45:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be81f0640 1 -- 172.21.15.32:0/3490125263 wait complete. 2023-12-20T04:45:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be81f0640 1 Processor -- start 2023-12-20T04:45:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.576+0000 7f9be81f0640 1 -- start start 2023-12-20T04:45:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be81f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9be006ce60 0x7f9be01559d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be81f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9be00af5e0 0x7f9be0154570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be81f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be015f110 0x7f9be0153d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be81f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be0154dc0 con 0x7f9be00af5e0 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be81f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9be01550a0 con 0x7f9be015f110 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be81f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9be0155380 con 0x7f9be006ce60 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be578b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9be00af5e0 0x7f9be0154570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be678d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be015f110 0x7f9be0153d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be578b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9be00af5e0 0x7f9be0154570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53154/0 (socket says 172.21.15.32:53154) 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be578b640 1 -- 172.21.15.32:0/1687070889 learned_addr learned my addr 172.21.15.32:0/1687070889 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be5f8c640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9be006ce60 0x7f9be01559d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be678d640 1 -- 172.21.15.32:0/1687070889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9be006ce60 msgr2=0x7f9be01559d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be678d640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9be006ce60 0x7f9be01559d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be678d640 1 -- 172.21.15.32:0/1687070889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9be00af5e0 msgr2=0x7f9be0154570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be678d640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9be00af5e0 0x7f9be0154570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be678d640 1 -- 172.21.15.32:0/1687070889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd8009a70 con 0x7f9be015f110 2023-12-20T04:45:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be578b640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9be00af5e0 0x7f9be0154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be678d640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be015f110 0x7f9be0153d40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80ac000 tx=0x7f9bd8019630 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9bd67fc640 1 -- 172.21.15.32:0/1687070889 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80b9650 con 0x7f9be015f110 2023-12-20T04:45:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9bd67fc640 1 -- 172.21.15.32:0/1687070889 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bd80b97b0 con 0x7f9be015f110 2023-12-20T04:45:43.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9bd67fc640 1 -- 172.21.15.32:0/1687070889 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bd80b9b20 con 0x7f9be015f110 2023-12-20T04:45:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be81f0640 1 -- 172.21.15.32:0/1687070889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9be0155660 con 0x7f9be015f110 2023-12-20T04:45:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be81f0640 1 -- 172.21.15.32:0/1687070889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9be0154280 con 0x7f9be015f110 2023-12-20T04:45:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.580+0000 7f9be81f0640 1 -- 172.21.15.32:0/1687070889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9be004e000 con 0x7f9be015f110 2023-12-20T04:45:43.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.584+0000 7f9bd67fc640 1 -- 172.21.15.32:0/1687070889 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 54) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f9bd80b9c80 con 0x7f9be015f110 2023-12-20T04:45:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.584+0000 7f9bd67fc640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9bc4065ca0 0x7f9bc4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.584+0000 7f9bd67fc640 1 -- 172.21.15.32:0/1687070889 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f9bd811e7f0 con 0x7f9be015f110 2023-12-20T04:45:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.584+0000 7f9be5f8c640 1 -- 172.21.15.32:0/1687070889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9bc4065ca0 msgr2=0x7f9bc4068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:45:43.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.584+0000 7f9be5f8c640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9bc4065ca0 0x7f9bc4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:45:43.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.588+0000 7f9bd67fc640 1 -- 172.21.15.32:0/1687070889 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9bd80ebd60 con 0x7f9be015f110 2023-12-20T04:45:43.730 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:43.734+0000 7fb0d0af0180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:45:43.744 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:43.748+0000 7f366ca48180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:45:43.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.752+0000 7f9be81f0640 1 -- 172.21.15.32:0/1687070889 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9be00cf450 con 0x7f9be015f110 2023-12-20T04:45:43.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.752+0000 7f9bd67fc640 1 -- 172.21.15.32:0/1687070889 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16642 (secure 0 0 0) 0x7f9bd80ef380 con 0x7f9be015f110 2023-12-20T04:45:43.749 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:43.749 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":110,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:45:41.080658+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:44:25.451651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-20T04:44:26.819848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.4","mappings":[{"from":6,"to":3}]},{"pgid":"13.16","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1531123683":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3296285907":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3289362372":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/998392483":"2023-12-21T04:44:39.978724+0000","172.21.15.32:6833/38009":"2023-12-21T04:44:39.978724+0000","172.21.15.32:6832/38009":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3368879992":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/2997487195":"2023-12-21T04:44:39.978724+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:45:43.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 -- 172.21.15.32:0/1687070889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9bc4065ca0 msgr2=0x7f9bc4068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:45:43.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9bc4065ca0 0x7f9bc4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 -- 172.21.15.32:0/1687070889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be015f110 msgr2=0x7f9be0153d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be015f110 0x7f9be0153d40 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7f9bd80ac000 tx=0x7f9bd8019630 comp rx=0 tx=0).stop 2023-12-20T04:45:43.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 -- 172.21.15.32:0/1687070889 shutdown_connections 2023-12-20T04:45:43.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9bc4065ca0 0x7f9bc4068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9be015f110 0x7f9be0153d40 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9be00af5e0 0x7f9be0154570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 --2- 172.21.15.32:0/1687070889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9be006ce60 0x7f9be01559d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 -- 172.21.15.32:0/1687070889 >> 172.21.15.32:0/1687070889 conn(0x7f9be006a320 msgr2=0x7f9be0159be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:43.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 -- 172.21.15.32:0/1687070889 shutdown_connections 2023-12-20T04:45:43.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.756+0000 7f9be81f0640 1 -- 172.21.15.32:0/1687070889 wait complete. 2023-12-20T04:45:43.772 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:44:25.451651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:44:26.819848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:45:43.772 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:45:43.887 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:43.890+0000 7fb0d0af0180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:45:43.908 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:43.911+0000 7f366ca48180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:45:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.923+0000 7fcda172c640 1 Processor -- start 2023-12-20T04:45:43.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.923+0000 7fcda172c640 1 -- start start 2023-12-20T04:45:43.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcda172c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcd9c1606b0 0x7fcd9c0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcda172c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c0cdbf0 0x7fcd9c0d11d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcda172c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcd9c0ce040 con 0x7fcd9c15cb30 2023-12-20T04:45:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcda172c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcd9c0d0d40 con 0x7fcd9c1606b0 2023-12-20T04:45:43.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcda172c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c160bb0 con 0x7fcd9c0cdbf0 2023-12-20T04:45:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcd93fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcd9c1606b0 0x7fcd9c0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcd9b7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c0cdbf0 0x7fcd9c0d11d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcd9affd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fcd9c15cb30 0x7fcd9c0b8300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:40938/0 (socket says 172.21.15.32:40938) 2023-12-20T04:45:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcd9affd640 1 -- 172.21.15.32:0/2896297334 learned_addr learned my addr 172.21.15.32:0/2896297334 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcd9a7fc640 1 -- 172.21.15.32:0/2896297334 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3643803752 0 0) 0x7fcd8c001170 con 0x7fcd9c15cb30 2023-12-20T04:45:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcd9a7fc640 1 -- 172.21.15.32:0/2896297334 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcd88003530 con 0x7fcd9c15cb30 2023-12-20T04:45:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcd93fff640 1 -- 172.21.15.32:0/2896297334 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c0cdbf0 msgr2=0x7fcd9c0d11d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcd93fff640 1 --2- 172.21.15.32:0/2896297334 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c0cdbf0 0x7fcd9c0d11d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcd93fff640 1 -- 172.21.15.32:0/2896297334 >> v1:172.21.15.39:6789/0 conn(0x7fcd9c15cb30 legacy=0x7fcd9c0b8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcd93fff640 1 -- 172.21.15.32:0/2896297334 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd9c160e90 con 0x7fcd9c1606b0 2023-12-20T04:45:43.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.927+0000 7fcd9b7fe640 1 --2- 172.21.15.32:0/2896297334 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c0cdbf0 0x7fcd9c0d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcd93fff640 1 --2- 172.21.15.32:0/2896297334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c1606b0 0x7fcd9c0cd6b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcd840097c0 tx=0x7fcd8401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=acdade6cf718d101 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcd9a7fc640 1 -- 172.21.15.32:0/2896297334 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840ab680 con 0x7fcd9c1606b0 2023-12-20T04:45:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcd9a7fc640 1 -- 172.21.15.32:0/2896297334 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd840ab7e0 con 0x7fcd9c1606b0 2023-12-20T04:45:43.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcd9a7fc640 1 -- 172.21.15.32:0/2896297334 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd840abb70 con 0x7fcd9c1606b0 2023-12-20T04:45:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcda172c640 1 -- 172.21.15.32:0/2896297334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c1606b0 msgr2=0x7fcd9c0cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcda172c640 1 --2- 172.21.15.32:0/2896297334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c1606b0 0x7fcd9c0cd6b0 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fcd840097c0 tx=0x7fcd8401c480 comp rx=0 tx=0).stop 2023-12-20T04:45:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcda172c640 1 -- 172.21.15.32:0/2896297334 shutdown_connections 2023-12-20T04:45:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcda172c640 1 --2- 172.21.15.32:0/2896297334 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c0cdbf0 0x7fcd9c0d11d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcda172c640 1 --2- 172.21.15.32:0/2896297334 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c1606b0 0x7fcd9c0cd6b0 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcda172c640 1 -- 172.21.15.32:0/2896297334 >> 172.21.15.32:0/2896297334 conn(0x7fcd9c06af60 msgr2=0x7fcd9c06b370 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcda172c640 1 -- 172.21.15.32:0/2896297334 shutdown_connections 2023-12-20T04:45:43.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcda172c640 1 -- 172.21.15.32:0/2896297334 wait complete. 2023-12-20T04:45:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcda172c640 1 Processor -- start 2023-12-20T04:45:43.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.931+0000 7fcda172c640 1 -- start start 2023-12-20T04:45:43.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcda172c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9c0cdbf0 0x7fcd9c15dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcda172c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c15cb30 0x7fcd9c1f7ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcda172c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c1606b0 0x7fcd9c15d330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcda172c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c15d870 con 0x7fcd9c0cdbf0 2023-12-20T04:45:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcda172c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c15e270 con 0x7fcd9c15cb30 2023-12-20T04:45:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcda172c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcd9c15e550 con 0x7fcd9c1606b0 2023-12-20T04:45:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd9affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9c0cdbf0 0x7fcd9c15dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd9affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9c0cdbf0 0x7fcd9c15dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53168/0 (socket says 172.21.15.32:53168) 2023-12-20T04:45:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd9affd640 1 -- 172.21.15.32:0/4070782750 learned_addr learned my addr 172.21.15.32:0/4070782750 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd9b7fe640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c1606b0 0x7fcd9c15d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:43.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd93fff640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c15cb30 0x7fcd9c1f7ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:43.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd9affd640 1 -- 172.21.15.32:0/4070782750 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c1606b0 msgr2=0x7fcd9c15d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd9affd640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c1606b0 0x7fcd9c15d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd9affd640 1 -- 172.21.15.32:0/4070782750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c15cb30 msgr2=0x7fcd9c1f7ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:43.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd9affd640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c15cb30 0x7fcd9c1f7ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd9affd640 1 -- 172.21.15.32:0/4070782750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcd94009a70 con 0x7fcd9c0cdbf0 2023-12-20T04:45:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd93fff640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c15cb30 0x7fcd9c1f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd9affd640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9c0cdbf0 0x7fcd9c15dbe0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c700 tx=0x7fcd8c00cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd937fe640 1 -- 172.21.15.32:0/4070782750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c005d40 con 0x7fcd9c0cdbf0 2023-12-20T04:45:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd937fe640 1 -- 172.21.15.32:0/4070782750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcd8c0070d0 con 0x7fcd9c0cdbf0 2023-12-20T04:45:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcd937fe640 1 -- 172.21.15.32:0/4070782750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcd8c005ea0 con 0x7fcd9c0cdbf0 2023-12-20T04:45:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcda172c640 1 -- 172.21.15.32:0/4070782750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcd84009420 con 0x7fcd9c0cdbf0 2023-12-20T04:45:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcda172c640 1 -- 172.21.15.32:0/4070782750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcd9c0c78a0 con 0x7fcd9c0cdbf0 2023-12-20T04:45:43.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.935+0000 7fcda172c640 1 -- 172.21.15.32:0/4070782750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcd9c074100 con 0x7fcd9c0cdbf0 2023-12-20T04:45:43.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.939+0000 7fcd937fe640 1 -- 172.21.15.32:0/4070782750 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 54) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fcd8c0074a0 con 0x7fcd9c0cdbf0 2023-12-20T04:45:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.939+0000 7fcd937fe640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd7c065cd0 0x7fcd7c068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.939+0000 7fcd93fff640 1 -- 172.21.15.32:0/4070782750 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd7c065cd0 msgr2=0x7fcd7c068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:45:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.939+0000 7fcd937fe640 1 -- 172.21.15.32:0/4070782750 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fcd8c082bb0 con 0x7fcd9c0cdbf0 2023-12-20T04:45:43.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.939+0000 7fcd93fff640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd7c065cd0 0x7fcd7c068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:45:43.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:43.943+0000 7fcd937fe640 1 -- 172.21.15.32:0/4070782750 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcd8c050090 con 0x7fcd9c0cdbf0 2023-12-20T04:45:44.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.103+0000 7fcda172c640 1 -- 172.21.15.32:0/4070782750 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fcd9c0ce040 con 0x7fcd9c0cdbf0 2023-12-20T04:45:44.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.103+0000 7fcd937fe640 1 -- 172.21.15.32:0/4070782750 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7fcd9c0ce040 con 0x7fcd9c0cdbf0 2023-12-20T04:45:44.102 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:45:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 -- 172.21.15.32:0/4070782750 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd7c065cd0 msgr2=0x7fcd7c068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:45:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd7c065cd0 0x7fcd7c068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 -- 172.21.15.32:0/4070782750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9c0cdbf0 msgr2=0x7fcd9c15dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9c0cdbf0 0x7fcd9c15dbe0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7fcd8c00c700 tx=0x7fcd8c00cb00 comp rx=0 tx=0).stop 2023-12-20T04:45:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 -- 172.21.15.32:0/4070782750 shutdown_connections 2023-12-20T04:45:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcd7c065cd0 0x7fcd7c068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcd9c1606b0 0x7fcd9c15d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcd9c15cb30 0x7fcd9c1f7ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 --2- 172.21.15.32:0/4070782750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcd9c0cdbf0 0x7fcd9c15dbe0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 -- 172.21.15.32:0/4070782750 >> 172.21.15.32:0/4070782750 conn(0x7fcd9c06af60 msgr2=0x7fcd9c0d2920 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 -- 172.21.15.32:0/4070782750 shutdown_connections 2023-12-20T04:45:44.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.107+0000 7fcda172c640 1 -- 172.21.15.32:0/4070782750 wait complete. 2023-12-20T04:45:44.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:45:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.275+0000 7fcc200d1640 1 Processor -- start 2023-12-20T04:45:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.275+0000 7fcc200d1640 1 -- start start 2023-12-20T04:45:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.275+0000 7fcc200d1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc180d14d0 0x7fcc180b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.275+0000 7fcc200d1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc180d2650 con 0x7fcc1815ce60 2023-12-20T04:45:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.275+0000 7fcc200d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc180d1a40 con 0x7fcc180d3300 2023-12-20T04:45:44.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.275+0000 7fcc200d1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc180d1d20 con 0x7fcc180d14d0 2023-12-20T04:45:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.275+0000 7fcc1de6d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc1815ce60 0x7fcc180d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54162/0 (socket says 172.21.15.32:54162) 2023-12-20T04:45:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.275+0000 7fcc1de6d640 1 -- 172.21.15.32:0/255508111 learned_addr learned my addr 172.21.15.32:0/255508111 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.275+0000 7fcc1e66e640 1 --2- 172.21.15.32:0/255508111 >> v2:172.21.15.32:3300/0 conn(0x7fcc180d14d0 0x7fcc180b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:44.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.275+0000 7fcc1ce6b640 1 -- 172.21.15.32:0/255508111 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2340323993 0 0) 0x7fcc180d2650 con 0x7fcc1815ce60 2023-12-20T04:45:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc1ce6b640 1 -- 172.21.15.32:0/255508111 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcc04003530 con 0x7fcc1815ce60 2023-12-20T04:45:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc1ce6b640 1 -- 172.21.15.32:0/255508111 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1427352194 0 0) 0x7fcc180d1a40 con 0x7fcc180d3300 2023-12-20T04:45:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc1e66e640 1 -- 172.21.15.32:0/255508111 >> v1:172.21.15.39:6789/0 conn(0x7fcc180d3300 legacy=0x7fcc180d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc1e66e640 1 -- 172.21.15.32:0/255508111 >> v1:172.21.15.32:6789/0 conn(0x7fcc1815ce60 legacy=0x7fcc180d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc1e66e640 1 -- 172.21.15.32:0/255508111 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc180d2000 con 0x7fcc180d14d0 2023-12-20T04:45:44.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc1e66e640 1 --2- 172.21.15.32:0/255508111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc180d14d0 0x7fcc180b6ea0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009d90 tx=0x7fcc1001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e92592c39f0f3f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc1ce6b640 1 -- 172.21.15.32:0/255508111 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100af600 con 0x7fcc180d14d0 2023-12-20T04:45:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc1ce6b640 1 -- 172.21.15.32:0/255508111 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc100ab050 con 0x7fcc180d14d0 2023-12-20T04:45:44.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc1ce6b640 1 -- 172.21.15.32:0/255508111 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100af980 con 0x7fcc180d14d0 2023-12-20T04:45:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc200d1640 1 -- 172.21.15.32:0/255508111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc180d14d0 msgr2=0x7fcc180b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc200d1640 1 --2- 172.21.15.32:0/255508111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc180d14d0 0x7fcc180b6ea0 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fcc10009d90 tx=0x7fcc1001f130 comp rx=0 tx=0).stop 2023-12-20T04:45:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc200d1640 1 -- 172.21.15.32:0/255508111 shutdown_connections 2023-12-20T04:45:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc200d1640 1 --2- 172.21.15.32:0/255508111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc180d14d0 0x7fcc180b6ea0 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.279+0000 7fcc200d1640 1 -- 172.21.15.32:0/255508111 >> 172.21.15.32:0/255508111 conn(0x7fcc1806b5e0 msgr2=0x7fcc1806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc200d1640 1 -- 172.21.15.32:0/255508111 shutdown_connections 2023-12-20T04:45:44.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc200d1640 1 -- 172.21.15.32:0/255508111 wait complete. 2023-12-20T04:45:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc200d1640 1 Processor -- start 2023-12-20T04:45:44.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc200d1640 1 -- start start 2023-12-20T04:45:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc200d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc180d14d0 0x7fcc1814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:44.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc200d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc180d3300 0x7fcc180a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc200d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc1815ce60 0x7fcc181f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc200d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc180cbf30 con 0x7fcc180d14d0 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc200d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc180cb320 con 0x7fcc1815ce60 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc200d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc180cb600 con 0x7fcc180d3300 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1de6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc180d14d0 0x7fcc1814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1e66e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc1815ce60 0x7fcc181f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1de6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc180d14d0 0x7fcc1814f0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53186/0 (socket says 172.21.15.32:53186) 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1de6d640 1 -- 172.21.15.32:0/1675024347 learned_addr learned my addr 172.21.15.32:0/1675024347 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1d66c640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc180d3300 0x7fcc180a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1e66e640 1 -- 172.21.15.32:0/1675024347 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc180d3300 msgr2=0x7fcc180a95a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1e66e640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc180d3300 0x7fcc180a95a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1e66e640 1 -- 172.21.15.32:0/1675024347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc180d14d0 msgr2=0x7fcc1814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1e66e640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc180d14d0 0x7fcc1814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1e66e640 1 -- 172.21.15.32:0/1675024347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc10009a70 con 0x7fcc1815ce60 2023-12-20T04:45:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1de6d640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc180d14d0 0x7fcc1814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.283+0000 7fcc1e66e640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc1815ce60 0x7fcc181f1ef0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcc100af000 tx=0x7fcc1001bed0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:44.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.287+0000 7fcc0e7fc640 1 -- 172.21.15.32:0/1675024347 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100afe70 con 0x7fcc1815ce60 2023-12-20T04:45:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.287+0000 7fcc0e7fc640 1 -- 172.21.15.32:0/1675024347 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc10009450 con 0x7fcc1815ce60 2023-12-20T04:45:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.287+0000 7fcc0e7fc640 1 -- 172.21.15.32:0/1675024347 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc100b9610 con 0x7fcc1815ce60 2023-12-20T04:45:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.287+0000 7fcc200d1640 1 -- 172.21.15.32:0/1675024347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc180cb8e0 con 0x7fcc1815ce60 2023-12-20T04:45:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.287+0000 7fcc200d1640 1 -- 172.21.15.32:0/1675024347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc180cc3c0 con 0x7fcc1815ce60 2023-12-20T04:45:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.287+0000 7fcc200d1640 1 -- 172.21.15.32:0/1675024347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc1804e000 con 0x7fcc1815ce60 2023-12-20T04:45:44.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.287+0000 7fcc0e7fc640 1 -- 172.21.15.32:0/1675024347 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 54) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fcc100b6050 con 0x7fcc1815ce60 2023-12-20T04:45:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.287+0000 7fcc0e7fc640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcbfc065cf0 0x7fcbfc0681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.287+0000 7fcc0e7fc640 1 -- 172.21.15.32:0/1675024347 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fcc101219a0 con 0x7fcc1815ce60 2023-12-20T04:45:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.287+0000 7fcc1de6d640 1 -- 172.21.15.32:0/1675024347 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcbfc065cf0 msgr2=0x7fcbfc0681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:45:44.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.287+0000 7fcc1de6d640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcbfc065cf0 0x7fcbfc0681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:45:44.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.291+0000 7fcc0e7fc640 1 -- 172.21.15.32:0/1675024347 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcc100ef050 con 0x7fcc1815ce60 2023-12-20T04:45:44.393 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:44.398+0000 7fb0d0af0180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:45:44.428 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:44.431+0000 7f366ca48180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:45:44.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.451+0000 7fcc200d1640 1 -- 172.21.15.32:0/1675024347 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fcc1806bb80 con 0x7fcc1815ce60 2023-12-20T04:45:44.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.451+0000 7fcc0e7fc640 1 -- 172.21.15.32:0/1675024347 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7fcc100f2670 con 0x7fcc1815ce60 2023-12-20T04:45:44.449 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:45:44.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 -- 172.21.15.32:0/1675024347 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcbfc065cf0 msgr2=0x7fcbfc0681b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:45:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcbfc065cf0 0x7fcbfc0681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 -- 172.21.15.32:0/1675024347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc1815ce60 msgr2=0x7fcc181f1ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc1815ce60 0x7fcc181f1ef0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fcc100af000 tx=0x7fcc1001bed0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 -- 172.21.15.32:0/1675024347 shutdown_connections 2023-12-20T04:45:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcbfc065cf0 0x7fcbfc0681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc1815ce60 0x7fcc181f1ef0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc180d3300 0x7fcc180a95a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 --2- 172.21.15.32:0/1675024347 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc180d14d0 0x7fcc1814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 -- 172.21.15.32:0/1675024347 >> 172.21.15.32:0/1675024347 conn(0x7fcc1806b5e0 msgr2=0x7fcc18066030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 -- 172.21.15.32:0/1675024347 shutdown_connections 2023-12-20T04:45:44.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.455+0000 7fcc200d1640 1 -- 172.21.15.32:0/1675024347 wait complete. 2023-12-20T04:45:44.456 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:44.458+0000 7fb0d0af0180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:45:44.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:45:44.492 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:44.495+0000 7f366ca48180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:45:44.513 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:44.514+0000 7fb0d0af0180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:45:44.552 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:44.555+0000 7f366ca48180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:45:44.604 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:44.606+0000 7fb0d0af0180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:45:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.623+0000 7f8f2f8f9640 1 Processor -- start 2023-12-20T04:45:44.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.623+0000 7f8f2f8f9640 1 -- start start 2023-12-20T04:45:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.623+0000 7f8f2f8f9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8f2815ce60 0x7f8f280d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:44.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.623+0000 7f8f2f8f9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f280d14d0 0x7f8f280b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.623+0000 7f8f2f8f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f280d2650 con 0x7f8f280d3130 2023-12-20T04:45:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.623+0000 7f8f2f8f9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f280d35a0 con 0x7f8f280d14d0 2023-12-20T04:45:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.623+0000 7f8f2f8f9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f280d1a40 con 0x7f8f2815ce60 2023-12-20T04:45:44.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.623+0000 7f8f2de96640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f280d14d0 0x7f8f280b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.627+0000 7f8f2ce94640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8f280d3130 0x7f8f280d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54170/0 (socket says 172.21.15.32:54170) 2023-12-20T04:45:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.627+0000 7f8f2ce94640 1 -- 172.21.15.32:0/2812870524 learned_addr learned my addr 172.21.15.32:0/2812870524 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.627+0000 7f8f2d695640 1 --2- 172.21.15.32:0/2812870524 >> v2:172.21.15.39:3300/0 conn(0x7f8f2815ce60 0x7f8f280d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.627+0000 7f8f2de96640 1 -- 172.21.15.32:0/2812870524 >> v2:172.21.15.39:3300/0 conn(0x7f8f2815ce60 msgr2=0x7f8f280d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.627+0000 7f8f2de96640 1 --2- 172.21.15.32:0/2812870524 >> v2:172.21.15.39:3300/0 conn(0x7f8f2815ce60 0x7f8f280d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.627+0000 7f8f2de96640 1 -- 172.21.15.32:0/2812870524 >> v1:172.21.15.32:6789/0 conn(0x7f8f280d3130 legacy=0x7f8f280d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.627+0000 7f8f2de96640 1 -- 172.21.15.32:0/2812870524 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f280d1d20 con 0x7f8f280d14d0 2023-12-20T04:45:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.627+0000 7f8f1ffff640 1 -- 172.21.15.32:0/2812870524 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 483896987 0 0) 0x7f8f280d2650 con 0x7f8f280d3130 2023-12-20T04:45:44.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.627+0000 7f8f2d695640 1 --2- 172.21.15.32:0/2812870524 >> v2:172.21.15.39:3300/0 conn(0x7f8f2815ce60 0x7f8f280d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f2de96640 1 --2- 172.21.15.32:0/2812870524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f280d14d0 0x7f8f280b5e90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009d90 tx=0x7f8f2001f100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ec53ff254c00200 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f1ffff640 1 -- 172.21.15.32:0/2812870524 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f200af600 con 0x7f8f280d14d0 2023-12-20T04:45:44.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f1ffff640 1 -- 172.21.15.32:0/2812870524 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f200ab050 con 0x7f8f280d14d0 2023-12-20T04:45:44.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f1ffff640 1 -- 172.21.15.32:0/2812870524 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f200af980 con 0x7f8f280d14d0 2023-12-20T04:45:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2812870524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f280d14d0 msgr2=0x7f8f280b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f2f8f9640 1 --2- 172.21.15.32:0/2812870524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f280d14d0 0x7f8f280b5e90 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7f8f20009d90 tx=0x7f8f2001f100 comp rx=0 tx=0).stop 2023-12-20T04:45:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2812870524 shutdown_connections 2023-12-20T04:45:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f2f8f9640 1 --2- 172.21.15.32:0/2812870524 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f280d14d0 0x7f8f280b5e90 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f2f8f9640 1 --2- 172.21.15.32:0/2812870524 >> v2:172.21.15.39:3300/0 conn(0x7f8f2815ce60 0x7f8f280d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2812870524 >> 172.21.15.32:0/2812870524 conn(0x7f8f2806b5e0 msgr2=0x7f8f2806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2812870524 shutdown_connections 2023-12-20T04:45:44.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.631+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2812870524 wait complete. 2023-12-20T04:45:44.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2f8f9640 1 Processor -- start 2023-12-20T04:45:44.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2f8f9640 1 -- start start 2023-12-20T04:45:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2f8f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f280d14d0 0x7f8f2814df70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2f8f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f280d3130 0x7f8f2815ad70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2f8f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f2815ce60 0x7f8f2815b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2f8f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2814e4b0 con 0x7f8f280d3130 2023-12-20T04:45:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2f8f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2815f4a0 con 0x7f8f280d14d0 2023-12-20T04:45:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2f8f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f2815e260 con 0x7f8f2815ce60 2023-12-20T04:45:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2ce94640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f280d3130 0x7f8f2815ad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2d695640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f280d14d0 0x7f8f2814df70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2d695640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f280d14d0 0x7f8f2814df70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:41446/0 (socket says 172.21.15.32:41446) 2023-12-20T04:45:44.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2ce94640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f280d3130 0x7f8f2815ad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53204/0 (socket says 172.21.15.32:53204) 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2d695640 1 -- 172.21.15.32:0/2771745747 learned_addr learned my addr 172.21.15.32:0/2771745747 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2de96640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f2815ce60 0x7f8f2815b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2ce94640 1 -- 172.21.15.32:0/2771745747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f2815ce60 msgr2=0x7f8f2815b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2ce94640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f2815ce60 0x7f8f2815b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2ce94640 1 -- 172.21.15.32:0/2771745747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f280d14d0 msgr2=0x7f8f2814df70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2ce94640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f280d14d0 0x7f8f2814df70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2ce94640 1 -- 172.21.15.32:0/2771745747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f24009420 con 0x7f8f280d3130 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2de96640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f2815ce60 0x7f8f2815b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2d695640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f280d14d0 0x7f8f2814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2ce94640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f280d3130 0x7f8f2815ad70 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800c430 tx=0x7f8f1800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f1dffb640 1 -- 172.21.15.32:0/2771745747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f180074e0 con 0x7f8f280d3130 2023-12-20T04:45:44.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f1dffb640 1 -- 172.21.15.32:0/2771745747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f18007660 con 0x7f8f280d3130 2023-12-20T04:45:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f1dffb640 1 -- 172.21.15.32:0/2771745747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f18004a30 con 0x7f8f280d3130 2023-12-20T04:45:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2771745747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f20009a70 con 0x7f8f280d3130 2023-12-20T04:45:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.635+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2771745747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f2815fa20 con 0x7f8f280d3130 2023-12-20T04:45:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.639+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2771745747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f2804e000 con 0x7f8f280d3130 2023-12-20T04:45:44.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.639+0000 7f8f1dffb640 1 -- 172.21.15.32:0/2771745747 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 54) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f8f18004b90 con 0x7f8f280d3130 2023-12-20T04:45:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.639+0000 7f8f1dffb640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f0c065ca0 0x7f8f0c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.639+0000 7f8f1dffb640 1 -- 172.21.15.32:0/2771745747 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f8f18083510 con 0x7f8f280d3130 2023-12-20T04:45:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.639+0000 7f8f2d695640 1 -- 172.21.15.32:0/2771745747 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f0c065ca0 msgr2=0x7f8f0c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:45:44.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.639+0000 7f8f2d695640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f0c065ca0 0x7f8f0c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:45:44.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.643+0000 7f8f1dffb640 1 -- 172.21.15.32:0/2771745747 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8f18050a90 con 0x7f8f280d3130 2023-12-20T04:45:44.651 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:44.655+0000 7f366ca48180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:45:44.665 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:44.666+0000 7fb0d0af0180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:45:44.711 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:44.715+0000 7f366ca48180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:45:44.751 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:44.754+0000 7fb0d0af0180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:45:44.801 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:44.803+0000 7f366ca48180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:45:44.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2771745747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8f280b6a20 con 0x7f8f280d3130 2023-12-20T04:45:44.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f1dffb640 1 -- 172.21.15.32:0/2771745747 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8f180540b0 con 0x7f8f280d3130 2023-12-20T04:45:44.805 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:45:44.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2771745747 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f0c065ca0 msgr2=0x7f8f0c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:45:44.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f0c065ca0 0x7f8f0c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2771745747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f280d3130 msgr2=0x7f8f2815ad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f280d3130 0x7f8f2815ad70 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7f8f1800c430 tx=0x7f8f1800c8f0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2771745747 shutdown_connections 2023-12-20T04:45:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8f0c065ca0 0x7f8f0c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f2815ce60 0x7f8f2815b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f280d3130 0x7f8f2815ad70 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 --2- 172.21.15.32:0/2771745747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f280d14d0 0x7f8f2814df70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2771745747 >> 172.21.15.32:0/2771745747 conn(0x7f8f2806b5e0 msgr2=0x7f8f280730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2771745747 shutdown_connections 2023-12-20T04:45:44.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:44.807+0000 7f8f2f8f9640 1 -- 172.21.15.32:0/2771745747 wait complete. 2023-12-20T04:45:44.810 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:44.814+0000 7fb0d0af0180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:45:44.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T04:45:44.862 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:44.863+0000 7f366ca48180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:45:44.960 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:44.962+0000 7fb0d0af0180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:45:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.007+0000 7ff7bf4e5640 1 Processor -- start 2023-12-20T04:45:45.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bf4e5640 1 -- start start 2023-12-20T04:45:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bf4e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff7b80ae010 0x7ff7b8071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bf4e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7b8164ef0 0x7ff7b80719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bf4e5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7b80ae3f0 con 0x7ff7b806cf10 2023-12-20T04:45:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bf4e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b80ca4b0 con 0x7ff7b8164ef0 2023-12-20T04:45:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bf4e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7b8161800 con 0x7ff7b80ae010 2023-12-20T04:45:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bca80640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff7b806cf10 0x7ff7b80701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54176/0 (socket says 172.21.15.32:54176) 2023-12-20T04:45:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bca80640 1 -- 172.21.15.32:0/3761462257 learned_addr learned my addr 172.21.15.32:0/3761462257 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bda82640 1 --2- 172.21.15.32:0/3761462257 >> v2:172.21.15.32:3300/0 conn(0x7ff7b8164ef0 0x7ff7b80719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bd281640 1 --2- 172.21.15.32:0/3761462257 >> v2:172.21.15.39:3300/0 conn(0x7ff7b80ae010 0x7ff7b8071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bda82640 1 -- 172.21.15.32:0/3761462257 >> v2:172.21.15.39:3300/0 conn(0x7ff7b80ae010 msgr2=0x7ff7b8071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bda82640 1 --2- 172.21.15.32:0/3761462257 >> v2:172.21.15.39:3300/0 conn(0x7ff7b80ae010 0x7ff7b8071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bda82640 1 -- 172.21.15.32:0/3761462257 >> v1:172.21.15.32:6789/0 conn(0x7ff7b806cf10 legacy=0x7ff7b80701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7bda82640 1 -- 172.21.15.32:0/3761462257 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b8165380 con 0x7ff7b8164ef0 2023-12-20T04:45:45.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.011+0000 7ff7affff640 1 -- 172.21.15.32:0/3761462257 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 386609512 0 0) 0x7ff7b80ae3f0 con 0x7ff7b806cf10 2023-12-20T04:45:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7bda82640 1 --2- 172.21.15.32:0/3761462257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b8164ef0 0x7ff7b80719b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80cc230 tx=0x7ff7b001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bf3705e06e0f045 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7affff640 1 -- 172.21.15.32:0/3761462257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b00a5910 con 0x7ff7b8164ef0 2023-12-20T04:45:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7affff640 1 -- 172.21.15.32:0/3761462257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b00a5a70 con 0x7ff7b8164ef0 2023-12-20T04:45:45.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7affff640 1 -- 172.21.15.32:0/3761462257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b00a5e00 con 0x7ff7b8164ef0 2023-12-20T04:45:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/3761462257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b8164ef0 msgr2=0x7ff7b80719b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7bf4e5640 1 --2- 172.21.15.32:0/3761462257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b8164ef0 0x7ff7b80719b0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7ff7b80cc230 tx=0x7ff7b001c850 comp rx=0 tx=0).stop 2023-12-20T04:45:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/3761462257 shutdown_connections 2023-12-20T04:45:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7bf4e5640 1 --2- 172.21.15.32:0/3761462257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b8164ef0 0x7ff7b80719b0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7bf4e5640 1 --2- 172.21.15.32:0/3761462257 >> v2:172.21.15.39:3300/0 conn(0x7ff7b80ae010 0x7ff7b8071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/3761462257 >> 172.21.15.32:0/3761462257 conn(0x7ff7b806a320 msgr2=0x7ff7b80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/3761462257 shutdown_connections 2023-12-20T04:45:45.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/3761462257 wait complete. 2023-12-20T04:45:45.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.015+0000 7ff7bf4e5640 1 Processor -- start 2023-12-20T04:45:45.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bf4e5640 1 -- start start 2023-12-20T04:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bf4e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7b80ae010 0x7ff7b81fb160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bf4e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7b81560d0 0x7ff7b8159120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bf4e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b80c6920 0x7ff7b80c6db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bf4e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b8157430 con 0x7ff7b80c6920 2023-12-20T04:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bf4e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b81568b0 con 0x7ff7b80ae010 2023-12-20T04:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bf4e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7b8156b90 con 0x7ff7b81560d0 2023-12-20T04:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bd281640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7b80ae010 0x7ff7b81fb160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bca80640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7b81560d0 0x7ff7b8159120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bd281640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7b80ae010 0x7ff7b81fb160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:41452/0 (socket says 172.21.15.32:41452) 2023-12-20T04:45:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bda82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b80c6920 0x7ff7b80c6db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bd281640 1 -- 172.21.15.32:0/2868403847 learned_addr learned my addr 172.21.15.32:0/2868403847 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bd281640 1 -- 172.21.15.32:0/2868403847 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7b81560d0 msgr2=0x7ff7b8159120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bd281640 1 --2- 172.21.15.32:0/2868403847 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7b81560d0 0x7ff7b8159120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bd281640 1 -- 172.21.15.32:0/2868403847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b80c6920 msgr2=0x7ff7b80c6db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bd281640 1 --2- 172.21.15.32:0/2868403847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b80c6920 0x7ff7b80c6db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bd281640 1 -- 172.21.15.32:0/2868403847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7b0009a70 con 0x7ff7b80ae010 2023-12-20T04:45:45.018 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:45.019+0000 7f366ca48180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:45:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bd281640 1 --2- 172.21.15.32:0/2868403847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7b80ae010 0x7ff7b81fb160 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400e9b0 tx=0x7ff7b400ee70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7adffb640 1 -- 172.21.15.32:0/2868403847 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b4009670 con 0x7ff7b80ae010 2023-12-20T04:45:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7adffb640 1 -- 172.21.15.32:0/2868403847 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7b400c9a0 con 0x7ff7b80ae010 2023-12-20T04:45:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/2868403847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7b8156e70 con 0x7ff7b80ae010 2023-12-20T04:45:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7adffb640 1 -- 172.21.15.32:0/2868403847 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7b400cd10 con 0x7ff7b80ae010 2023-12-20T04:45:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/2868403847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7b8156550 con 0x7ff7b80ae010 2023-12-20T04:45:45.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.019+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/2868403847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff7b804e000 con 0x7ff7b80ae010 2023-12-20T04:45:45.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.023+0000 7ff7adffb640 1 -- 172.21.15.32:0/2868403847 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 54) v1 ==== 85444+0+0 (secure 0 0 0) 0x7ff7b4013470 con 0x7ff7b80ae010 2023-12-20T04:45:45.021 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:45.026+0000 7fb0d0af0180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:45:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.023+0000 7ff7adffb640 1 --2- 172.21.15.32:0/2868403847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff79c065cd0 0x7ff79c068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.023+0000 7ff7adffb640 1 -- 172.21.15.32:0/2868403847 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7ff7b40842c0 con 0x7ff7b80ae010 2023-12-20T04:45:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.023+0000 7ff7bca80640 1 -- 172.21.15.32:0/2868403847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff79c065cd0 msgr2=0x7ff79c068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:45:45.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.023+0000 7ff7bca80640 1 --2- 172.21.15.32:0/2868403847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff79c065cd0 0x7ff79c068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:45:45.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.027+0000 7ff7adffb640 1 -- 172.21.15.32:0/2868403847 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff7b40519b0 con 0x7ff7b80ae010 2023-12-20T04:45:45.081 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:45.083+0000 7f366ca48180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:45:45.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.199+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/2868403847 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff7b80cf450 con 0x7ff7b80ae010 2023-12-20T04:45:45.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.199+0000 7ff7adffb640 1 -- 172.21.15.32:0/2868403847 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v110) v1 ==== 80+0+459 (secure 0 0 0) 0x7ff7b4017080 con 0x7ff7b80ae010 2023-12-20T04:45:45.198 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1531123683 2023-12-21T04:44:39.978724+0000 2023-12-20T04:45:45.198 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3296285907 2023-12-21T04:44:39.978724+0000 2023-12-20T04:45:45.198 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3289362372 2023-12-21T04:44:39.978724+0000 2023-12-20T04:45:45.198 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/998392483 2023-12-21T04:44:39.978724+0000 2023-12-20T04:45:45.198 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/38009 2023-12-21T04:44:39.978724+0000 2023-12-20T04:45:45.198 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/38009 2023-12-21T04:44:39.978724+0000 2023-12-20T04:45:45.199 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3368879992 2023-12-21T04:44:39.978724+0000 2023-12-20T04:45:45.199 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2997487195 2023-12-21T04:44:39.978724+0000 2023-12-20T04:45:45.199 INFO:teuthology.orchestra.run.smithi032.stderr:listed 8 entries 2023-12-20T04:45:45.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/2868403847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff79c065cd0 msgr2=0x7ff79c068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:45:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 --2- 172.21.15.32:0/2868403847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff79c065cd0 0x7ff79c068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/2868403847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7b80ae010 msgr2=0x7ff7b81fb160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 --2- 172.21.15.32:0/2868403847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7b80ae010 0x7ff7b81fb160 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7ff7b400e9b0 tx=0x7ff7b400ee70 comp rx=0 tx=0).stop 2023-12-20T04:45:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/2868403847 shutdown_connections 2023-12-20T04:45:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 --2- 172.21.15.32:0/2868403847 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff79c065cd0 0x7ff79c068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 --2- 172.21.15.32:0/2868403847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7b80c6920 0x7ff7b80c6db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 --2- 172.21.15.32:0/2868403847 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7b81560d0 0x7ff7b8159120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 --2- 172.21.15.32:0/2868403847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7b80ae010 0x7ff7b81fb160 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/2868403847 >> 172.21.15.32:0/2868403847 conn(0x7ff7b806a320 msgr2=0x7ff7b806c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/2868403847 shutdown_connections 2023-12-20T04:45:45.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.203+0000 7ff7bf4e5640 1 -- 172.21.15.32:0/2868403847 wait complete. 2023-12-20T04:45:45.220 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T04:45:45.220 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:45:45.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T04:45:45.272 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:45:45.272 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:45:45.272 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:45:45.279 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:45:45.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:45:45.279 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T04:45:45.281 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:45.282+0000 7fb0d0af0180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:45:45.296 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:45:45.297 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:45:45.306 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:45:45.306 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:45:45.306 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:45:45.334 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:45:45.334 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:45:45.335 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:45:45.343 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:45:45.344 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:45:45.344 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:45:45.344 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:45.347+0000 7f366ca48180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:45:45.359 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:45:45.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:45:45.426 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:45.430+0000 7fb0d0af0180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:45:45.488 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:45.491+0000 7f366ca48180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:45:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.531+0000 7fa288af2640 1 Processor -- start 2023-12-20T04:45:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.531+0000 7fa288af2640 1 -- start start 2023-12-20T04:45:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa288af2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa2841593c0 0x7fa28415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa288af2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2840afdb0 0x7fa28406c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa288af2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa28406c920 0x7fa2840705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa288af2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa2840ae9e0 con 0x7fa2840afdb0 2023-12-20T04:45:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa288af2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa28406cd90 con 0x7fa2841593c0 2023-12-20T04:45:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa288af2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840ae140 con 0x7fa28406c920 2023-12-20T04:45:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa27bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2840afdb0 0x7fa28406c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa27bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa2840afdb0 0x7fa28406c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53244/0 (socket says 172.21.15.32:53244) 2023-12-20T04:45:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa27bfff640 1 -- 172.21.15.32:0/663481964 learned_addr learned my addr 172.21.15.32:0/663481964 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa28259c640 1 --2- 172.21.15.32:0/663481964 >> v2:172.21.15.39:3300/0 conn(0x7fa2841593c0 0x7fa28415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa282d9d640 1 --2- 172.21.15.32:0/663481964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa28406c920 0x7fa2840705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa27bfff640 1 -- 172.21.15.32:0/663481964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa28406c920 msgr2=0x7fa2840705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa27bfff640 1 --2- 172.21.15.32:0/663481964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa28406c920 0x7fa2840705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa27bfff640 1 -- 172.21.15.32:0/663481964 >> v2:172.21.15.39:3300/0 conn(0x7fa2841593c0 msgr2=0x7fa28415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa27bfff640 1 --2- 172.21.15.32:0/663481964 >> v2:172.21.15.39:3300/0 conn(0x7fa2841593c0 0x7fa28415d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.535+0000 7fa27bfff640 1 -- 172.21.15.32:0/663481964 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa28415dee0 con 0x7fa2840afdb0 2023-12-20T04:45:45.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa27bfff640 1 --2- 172.21.15.32:0/663481964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2840afdb0 0x7fa28406c3e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa28406ab00 tx=0x7fa26c01eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ce3d220ce63fc9a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:45.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa281d9b640 1 -- 172.21.15.32:0/663481964 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0a7b20 con 0x7fa2840afdb0 2023-12-20T04:45:45.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa281d9b640 1 -- 172.21.15.32:0/663481964 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa26c0a7c80 con 0x7fa2840afdb0 2023-12-20T04:45:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa281d9b640 1 -- 172.21.15.32:0/663481964 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa26c0af440 con 0x7fa2840afdb0 2023-12-20T04:45:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa288af2640 1 -- 172.21.15.32:0/663481964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2840afdb0 msgr2=0x7fa28406c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa288af2640 1 --2- 172.21.15.32:0/663481964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2840afdb0 0x7fa28406c3e0 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fa28406ab00 tx=0x7fa26c01eb10 comp rx=0 tx=0).stop 2023-12-20T04:45:45.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa288af2640 1 -- 172.21.15.32:0/663481964 shutdown_connections 2023-12-20T04:45:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa288af2640 1 --2- 172.21.15.32:0/663481964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa28406c920 0x7fa2840705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa288af2640 1 --2- 172.21.15.32:0/663481964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2840afdb0 0x7fa28406c3e0 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa288af2640 1 --2- 172.21.15.32:0/663481964 >> v2:172.21.15.39:3300/0 conn(0x7fa2841593c0 0x7fa28415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa288af2640 1 -- 172.21.15.32:0/663481964 >> 172.21.15.32:0/663481964 conn(0x7fa28406a320 msgr2=0x7fa2840ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa288af2640 1 -- 172.21.15.32:0/663481964 shutdown_connections 2023-12-20T04:45:45.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.539+0000 7fa288af2640 1 -- 172.21.15.32:0/663481964 wait complete. 2023-12-20T04:45:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa288af2640 1 Processor -- start 2023-12-20T04:45:45.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa288af2640 1 -- start start 2023-12-20T04:45:45.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa288af2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa28406c920 0x7fa2840abc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa288af2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2840afdb0 0x7fa2840c7220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa288af2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2841593c0 0x7fa28414ed10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa288af2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840ac1d0 con 0x7fa2841593c0 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa288af2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840ac480 con 0x7fa2840afdb0 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa288af2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2840ac760 con 0x7fa28406c920 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa282d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2841593c0 0x7fa28414ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa28259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa28406c920 0x7fa2840abc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa27bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2840afdb0 0x7fa2840c7220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa28259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa28406c920 0x7fa2840abc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45222/0 (socket says 172.21.15.32:45222) 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa28259c640 1 -- 172.21.15.32:0/3056316166 learned_addr learned my addr 172.21.15.32:0/3056316166 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa282d9d640 1 -- 172.21.15.32:0/3056316166 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa28406c920 msgr2=0x7fa2840abc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa282d9d640 1 --2- 172.21.15.32:0/3056316166 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa28406c920 0x7fa2840abc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa282d9d640 1 -- 172.21.15.32:0/3056316166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2840afdb0 msgr2=0x7fa2840c7220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa282d9d640 1 --2- 172.21.15.32:0/3056316166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2840afdb0 0x7fa2840c7220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa282d9d640 1 -- 172.21.15.32:0/3056316166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa26c00b770 con 0x7fa2841593c0 2023-12-20T04:45:45.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa282d9d640 1 --2- 172.21.15.32:0/3056316166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2841593c0 0x7fa28414ed10 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00eba0 tx=0x7fa27c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:45.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa27affd640 1 -- 172.21.15.32:0/3056316166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c00cd20 con 0x7fa2841593c0 2023-12-20T04:45:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa27affd640 1 -- 172.21.15.32:0/3056316166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa27c00f040 con 0x7fa2841593c0 2023-12-20T04:45:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa27affd640 1 -- 172.21.15.32:0/3056316166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa27c0146c0 con 0x7fa2841593c0 2023-12-20T04:45:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa288af2640 1 -- 172.21.15.32:0/3056316166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2840c6a20 con 0x7fa2841593c0 2023-12-20T04:45:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.543+0000 7fa288af2640 1 -- 172.21.15.32:0/3056316166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2840c7760 con 0x7fa2841593c0 2023-12-20T04:45:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.547+0000 7fa288af2640 1 -- 172.21.15.32:0/3056316166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa2840c6880 con 0x7fa2841593c0 2023-12-20T04:45:45.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.547+0000 7fa27affd640 1 -- 172.21.15.32:0/3056316166 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 54) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fa27c014820 con 0x7fa2841593c0 2023-12-20T04:45:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.547+0000 7fa27affd640 1 --2- 172.21.15.32:0/3056316166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa264065ca0 0x7fa264068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.547+0000 7fa28259c640 1 -- 172.21.15.32:0/3056316166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa264065ca0 msgr2=0x7fa264068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:45:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.547+0000 7fa27affd640 1 -- 172.21.15.32:0/3056316166 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fa27c084570 con 0x7fa2841593c0 2023-12-20T04:45:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.547+0000 7fa28259c640 1 --2- 172.21.15.32:0/3056316166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa264065ca0 0x7fa264068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:45:45.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.551+0000 7fa27affd640 1 -- 172.21.15.32:0/3056316166 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa2840c6880 con 0x7fa2841593c0 2023-12-20T04:45:45.556 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:45.558+0000 7fb0d0af0180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:45:45.625 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:45.627+0000 7f366ca48180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:45:45.630 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:45.634+0000 7fb0d0af0180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:45:45.700 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:45.703+0000 7f366ca48180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:45:45.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.743+0000 7fa288af2640 1 -- 172.21.15.32:0/3056316166 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa2840c6880 con 0x7fa2841593c0 2023-12-20T04:45:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.743+0000 7fa27affd640 1 -- 172.21.15.32:0/3056316166 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4711 (secure 0 0 0) 0x7fa2840c6880 con 0x7fa2841593c0 2023-12-20T04:45:45.742 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:45.742 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1072056415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1072056415},{"type":"v1","addr":"172.21.15.39:6835","nonce":1072056415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3512326822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3512326822},{"type":"v1","addr":"172.21.15.32:6836","nonce":3512326822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2163601073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2163601073},{"type":"v1","addr":"172.21.15.32:6837","nonce":2163601073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:44:27.818499+0000","modified":"2023-12-20T04:44:30.831088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7359":{"gid":7359,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/2369052657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2369052657},{"type":"v1","addr":"172.21.15.39:6833","nonce":2369052657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-20T04:45:45.742 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 46 2023-12-20T04:45:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 -- 172.21.15.32:0/3056316166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa264065ca0 msgr2=0x7fa264068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:45:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 --2- 172.21.15.32:0/3056316166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa264065ca0 0x7fa264068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 -- 172.21.15.32:0/3056316166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2841593c0 msgr2=0x7fa28414ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 --2- 172.21.15.32:0/3056316166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2841593c0 0x7fa28414ed10 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fa27c00eba0 tx=0x7fa27c00c490 comp rx=0 tx=0).stop 2023-12-20T04:45:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 -- 172.21.15.32:0/3056316166 shutdown_connections 2023-12-20T04:45:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 --2- 172.21.15.32:0/3056316166 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa264065ca0 0x7fa264068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 --2- 172.21.15.32:0/3056316166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa2841593c0 0x7fa28414ed10 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 --2- 172.21.15.32:0/3056316166 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa2840afdb0 0x7fa2840c7220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 --2- 172.21.15.32:0/3056316166 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa28406c920 0x7fa2840abc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 -- 172.21.15.32:0/3056316166 >> 172.21.15.32:0/3056316166 conn(0x7fa28406a320 msgr2=0x7fa2840c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 -- 172.21.15.32:0/3056316166 shutdown_connections 2023-12-20T04:45:45.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.747+0000 7fa288af2640 1 -- 172.21.15.32:0/3056316166 wait complete. 2023-12-20T04:45:45.765 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:45:45.772 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:45.774+0000 7fb0d0af0180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:45:45.846 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:45.847+0000 7f366ca48180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:45:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.911+0000 7fcf3ea55640 1 Processor -- start 2023-12-20T04:45:45.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.911+0000 7fcf3ea55640 1 -- start start 2023-12-20T04:45:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf3ea55640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf380af5e0 0x7fcf380af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf3ea55640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3815b990 con 0x7fcf3815f110 2023-12-20T04:45:45.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf3ea55640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf3815bbd0 con 0x7fcf3806ce60 2023-12-20T04:45:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf3ea55640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3815beb0 con 0x7fcf380af5e0 2023-12-20T04:45:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf3cff2640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcf3815f110 0x7fcf3815c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54178/0 (socket says 172.21.15.32:54178) 2023-12-20T04:45:45.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf3cff2640 1 -- 172.21.15.32:0/1815032768 learned_addr learned my addr 172.21.15.32:0/1815032768 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf37fff640 1 --2- 172.21.15.32:0/1815032768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf380af5e0 0x7fcf380af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf36ffd640 1 -- 172.21.15.32:0/1815032768 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149351620 0 0) 0x7fcf300014a0 con 0x7fcf3815f110 2023-12-20T04:45:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf36ffd640 1 -- 172.21.15.32:0/1815032768 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf20003530 con 0x7fcf3815f110 2023-12-20T04:45:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf36ffd640 1 -- 172.21.15.32:0/1815032768 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2383103635 0 0) 0x7fcf3815bbd0 con 0x7fcf3806ce60 2023-12-20T04:45:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf36ffd640 1 -- 172.21.15.32:0/1815032768 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcf300014a0 con 0x7fcf3806ce60 2023-12-20T04:45:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf37fff640 1 -- 172.21.15.32:0/1815032768 >> v1:172.21.15.39:6789/0 conn(0x7fcf3806ce60 legacy=0x7fcf3815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf37fff640 1 -- 172.21.15.32:0/1815032768 >> v1:172.21.15.32:6789/0 conn(0x7fcf3815f110 legacy=0x7fcf3815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.915+0000 7fcf37fff640 1 -- 172.21.15.32:0/1815032768 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf3815e910 con 0x7fcf380af5e0 2023-12-20T04:45:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf37fff640 1 --2- 172.21.15.32:0/1815032768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf380af5e0 0x7fcf380af9c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097d0 tx=0x7fcf2801c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bf6f3747bf72a45e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf36ffd640 1 -- 172.21.15.32:0/1815032768 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280aa5f0 con 0x7fcf380af5e0 2023-12-20T04:45:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf36ffd640 1 -- 172.21.15.32:0/1815032768 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf280aa750 con 0x7fcf380af5e0 2023-12-20T04:45:45.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf36ffd640 1 -- 172.21.15.32:0/1815032768 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf280aaac0 con 0x7fcf380af5e0 2023-12-20T04:45:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf3ea55640 1 -- 172.21.15.32:0/1815032768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf380af5e0 msgr2=0x7fcf380af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf3ea55640 1 --2- 172.21.15.32:0/1815032768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf380af5e0 0x7fcf380af9c0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fcf280097d0 tx=0x7fcf2801c470 comp rx=0 tx=0).stop 2023-12-20T04:45:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf3ea55640 1 -- 172.21.15.32:0/1815032768 shutdown_connections 2023-12-20T04:45:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf3ea55640 1 --2- 172.21.15.32:0/1815032768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf380af5e0 0x7fcf380af9c0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf3ea55640 1 -- 172.21.15.32:0/1815032768 >> 172.21.15.32:0/1815032768 conn(0x7fcf3806a320 msgr2=0x7fcf380ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf3ea55640 1 -- 172.21.15.32:0/1815032768 shutdown_connections 2023-12-20T04:45:45.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf3ea55640 1 -- 172.21.15.32:0/1815032768 wait complete. 2023-12-20T04:45:45.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf3ea55640 1 Processor -- start 2023-12-20T04:45:45.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.919+0000 7fcf3ea55640 1 -- start start 2023-12-20T04:45:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3ea55640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3806ce60 0x7fcf3814eed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3ea55640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf380af5e0 0x7fcf38153de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3ea55640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf3815f110 0x7fcf38158ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3ea55640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf38151bb0 con 0x7fcf380af5e0 2023-12-20T04:45:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3ea55640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf38070730 con 0x7fcf3806ce60 2023-12-20T04:45:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3ea55640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf38070a10 con 0x7fcf3815f110 2023-12-20T04:45:45.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf380af5e0 0x7fcf38153de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3cff2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf3815f110 0x7fcf38158ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf380af5e0 0x7fcf38153de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53258/0 (socket says 172.21.15.32:53258) 2023-12-20T04:45:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf37fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3806ce60 0x7fcf3814eed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:45.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf377fe640 1 -- 172.21.15.32:0/2035755125 learned_addr learned my addr 172.21.15.32:0/2035755125 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3cff2640 1 -- 172.21.15.32:0/2035755125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3806ce60 msgr2=0x7fcf3814eed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3cff2640 1 --2- 172.21.15.32:0/2035755125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3806ce60 0x7fcf3814eed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3cff2640 1 -- 172.21.15.32:0/2035755125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf380af5e0 msgr2=0x7fcf38153de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3cff2640 1 --2- 172.21.15.32:0/2035755125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf380af5e0 0x7fcf38153de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3cff2640 1 -- 172.21.15.32:0/2035755125 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf280093b0 con 0x7fcf3815f110 2023-12-20T04:45:45.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3cff2640 1 --2- 172.21.15.32:0/2035755125 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf3815f110 0x7fcf38158ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf30002240 tx=0x7fcf3000a430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf34ff9640 1 -- 172.21.15.32:0/2035755125 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000ad40 con 0x7fcf3815f110 2023-12-20T04:45:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf34ff9640 1 -- 172.21.15.32:0/2035755125 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf30012400 con 0x7fcf3815f110 2023-12-20T04:45:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf34ff9640 1 -- 172.21.15.32:0/2035755125 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf3000aea0 con 0x7fcf3815f110 2023-12-20T04:45:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3ea55640 1 -- 172.21.15.32:0/2035755125 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf38070cf0 con 0x7fcf3815f110 2023-12-20T04:45:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3ea55640 1 -- 172.21.15.32:0/2035755125 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3814f410 con 0x7fcf3815f110 2023-12-20T04:45:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf3ea55640 1 -- 172.21.15.32:0/2035755125 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf38151d20 con 0x7fcf3815f110 2023-12-20T04:45:45.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.923+0000 7fcf34ff9640 1 -- 172.21.15.32:0/2035755125 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 54) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fcf30007070 con 0x7fcf3815f110 2023-12-20T04:45:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.927+0000 7fcf34ff9640 1 --2- 172.21.15.32:0/2035755125 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf24065ca0 0x7fcf24068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.927+0000 7fcf37fff640 1 -- 172.21.15.32:0/2035755125 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf24065ca0 msgr2=0x7fcf24068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:45:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.927+0000 7fcf34ff9640 1 -- 172.21.15.32:0/2035755125 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fcf30083600 con 0x7fcf3815f110 2023-12-20T04:45:45.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.927+0000 7fcf37fff640 1 --2- 172.21.15.32:0/2035755125 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf24065ca0 0x7fcf24068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:45:45.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:45.931+0000 7fcf34ff9640 1 -- 172.21.15.32:0/2035755125 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcf30050ae0 con 0x7fcf3815f110 2023-12-20T04:45:46.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 -- 172.21.15.32:0/2035755125 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf3815b990 con 0x7fcf3815f110 2023-12-20T04:45:46.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf34ff9640 1 -- 172.21.15.32:0/2035755125 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v110) v1 ==== 74+0+16642 (secure 0 0 0) 0x7fcf3815b990 con 0x7fcf3815f110 2023-12-20T04:45:46.089 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:46.089 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":110,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:45:41.080658+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:44:25.451651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-20T04:44:26.819848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.4","mappings":[{"from":6,"to":3}]},{"pgid":"13.16","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1531123683":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3296285907":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3289362372":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/998392483":"2023-12-21T04:44:39.978724+0000","172.21.15.32:6833/38009":"2023-12-21T04:44:39.978724+0000","172.21.15.32:6832/38009":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3368879992":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/2997487195":"2023-12-21T04:44:39.978724+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:45:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 -- 172.21.15.32:0/2035755125 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf24065ca0 msgr2=0x7fcf24068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:45:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 --2- 172.21.15.32:0/2035755125 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf24065ca0 0x7fcf24068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 -- 172.21.15.32:0/2035755125 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf3815f110 msgr2=0x7fcf38158ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 --2- 172.21.15.32:0/2035755125 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf3815f110 0x7fcf38158ea0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7fcf30002240 tx=0x7fcf3000a430 comp rx=0 tx=0).stop 2023-12-20T04:45:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 -- 172.21.15.32:0/2035755125 shutdown_connections 2023-12-20T04:45:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 --2- 172.21.15.32:0/2035755125 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcf24065ca0 0x7fcf24068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 --2- 172.21.15.32:0/2035755125 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf3815f110 0x7fcf38158ea0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 --2- 172.21.15.32:0/2035755125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf380af5e0 0x7fcf38153de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 --2- 172.21.15.32:0/2035755125 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf3806ce60 0x7fcf3814eed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 -- 172.21.15.32:0/2035755125 >> 172.21.15.32:0/2035755125 conn(0x7fcf3806a320 msgr2=0x7fcf380ca5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:46.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 -- 172.21.15.32:0/2035755125 shutdown_connections 2023-12-20T04:45:46.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.091+0000 7fcf3ea55640 1 -- 172.21.15.32:0/2035755125 wait complete. 2023-12-20T04:45:46.111 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 12, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:44:25.451651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '106', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 13, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:44:26.819848+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:45:46.111 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:45:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.263+0000 7f154c96d640 1 Processor -- start 2023-12-20T04:45:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.263+0000 7f154c96d640 1 -- start start 2023-12-20T04:45:46.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.263+0000 7f154c96d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f154815ce60 0x7f15480d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.263+0000 7f154c96d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15480d14d0 0x7f15480b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.263+0000 7f154c96d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15480d2650 con 0x7f15480d3130 2023-12-20T04:45:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f154c96d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15480d35a0 con 0x7f154815ce60 2023-12-20T04:45:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f154c96d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15480d1a40 con 0x7f15480d14d0 2023-12-20T04:45:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f1546d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15480d14d0 0x7f15480b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f154659c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f154815ce60 0x7f15480d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f1546d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15480d14d0 0x7f15480b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45244/0 (socket says 172.21.15.32:45244) 2023-12-20T04:45:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f1546d9d640 1 -- 172.21.15.32:0/296292149 learned_addr learned my addr 172.21.15.32:0/296292149 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:46.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f154659c640 1 --2- 172.21.15.32:0/296292149 >> v2:172.21.15.39:3300/0 conn(0x7f154815ce60 0x7f15480d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:41486/0 (socket says 172.21.15.32:41486) 2023-12-20T04:45:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f1546d9d640 1 -- 172.21.15.32:0/296292149 >> v2:172.21.15.39:3300/0 conn(0x7f154815ce60 msgr2=0x7f15480d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f1546d9d640 1 --2- 172.21.15.32:0/296292149 >> v2:172.21.15.39:3300/0 conn(0x7f154815ce60 0x7f15480d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f1546d9d640 1 -- 172.21.15.32:0/296292149 >> v1:172.21.15.39:6789/0 conn(0x7f15480d3130 legacy=0x7f15480d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f1546d9d640 1 -- 172.21.15.32:0/296292149 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15480d1d20 con 0x7f15480d14d0 2023-12-20T04:45:46.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f1545d9b640 1 -- 172.21.15.32:0/296292149 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2272715940 0 0) 0x7f15480d2650 con 0x7f15480d3130 2023-12-20T04:45:46.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f1546d9d640 1 --2- 172.21.15.32:0/296292149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15480d14d0 0x7f15480b5e90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1540009e50 tx=0x7f154001f1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36fc0d17df8fcca3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:46.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.267+0000 7f1545d9b640 1 -- 172.21.15.32:0/296292149 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15400af400 con 0x7f15480d14d0 2023-12-20T04:45:46.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f1545d9b640 1 -- 172.21.15.32:0/296292149 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15400ab050 con 0x7f15480d14d0 2023-12-20T04:45:46.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f1545d9b640 1 -- 172.21.15.32:0/296292149 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15400af780 con 0x7f15480d14d0 2023-12-20T04:45:46.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 -- 172.21.15.32:0/296292149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15480d14d0 msgr2=0x7f15480b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 --2- 172.21.15.32:0/296292149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15480d14d0 0x7f15480b5e90 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f1540009e50 tx=0x7f154001f1f0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 -- 172.21.15.32:0/296292149 shutdown_connections 2023-12-20T04:45:46.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 --2- 172.21.15.32:0/296292149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15480d14d0 0x7f15480b5e90 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 --2- 172.21.15.32:0/296292149 >> v2:172.21.15.39:3300/0 conn(0x7f154815ce60 0x7f15480d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 -- 172.21.15.32:0/296292149 >> 172.21.15.32:0/296292149 conn(0x7f154806b5e0 msgr2=0x7f154806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:46.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 -- 172.21.15.32:0/296292149 shutdown_connections 2023-12-20T04:45:46.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 -- 172.21.15.32:0/296292149 wait complete. 2023-12-20T04:45:46.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 Processor -- start 2023-12-20T04:45:46.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 -- start start 2023-12-20T04:45:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15480d14d0 0x7f15481f5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15480d3130 0x7f154815ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f154815ce60 0x7f154815bd40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f154815a570 con 0x7f15480d14d0 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15481f64f0 con 0x7f15480d3130 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.271+0000 7f154c96d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1548153b10 con 0x7f154815ce60 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f154659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15480d14d0 0x7f15481f5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f1546d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f154815ce60 0x7f154815bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f1546d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f154815ce60 0x7f154815bd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45254/0 (socket says 172.21.15.32:45254) 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f1546d9d640 1 -- 172.21.15.32:0/3920210638 learned_addr learned my addr 172.21.15.32:0/3920210638 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f153ffff640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15480d3130 0x7f154815ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f1546d9d640 1 -- 172.21.15.32:0/3920210638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15480d3130 msgr2=0x7f154815ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f1546d9d640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15480d3130 0x7f154815ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f1546d9d640 1 -- 172.21.15.32:0/3920210638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15480d14d0 msgr2=0x7f15481f5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f1546d9d640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15480d14d0 0x7f15481f5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f1546d9d640 1 -- 172.21.15.32:0/3920210638 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1540009b30 con 0x7f154815ce60 2023-12-20T04:45:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f1546d9d640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f154815ce60 0x7f154815bd40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1540025f40 tx=0x7f154001bf90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:46.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f153effd640 1 -- 172.21.15.32:0/3920210638 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15400afce0 con 0x7f154815ce60 2023-12-20T04:45:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f153effd640 1 -- 172.21.15.32:0/3920210638 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15400afe40 con 0x7f154815ce60 2023-12-20T04:45:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f154c96d640 1 -- 172.21.15.32:0/3920210638 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15480cc6a0 con 0x7f154815ce60 2023-12-20T04:45:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f153effd640 1 -- 172.21.15.32:0/3920210638 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1540003bc0 con 0x7f154815ce60 2023-12-20T04:45:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f154c96d640 1 -- 172.21.15.32:0/3920210638 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15481531e0 con 0x7f154815ce60 2023-12-20T04:45:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f154c96d640 1 -- 172.21.15.32:0/3920210638 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1548153c80 con 0x7f154815ce60 2023-12-20T04:45:46.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.275+0000 7f153effd640 1 -- 172.21.15.32:0/3920210638 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 54) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f15400b6070 con 0x7f154815ce60 2023-12-20T04:45:46.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.279+0000 7f153effd640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1528065ca0 0x7f1528068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.279+0000 7f153effd640 1 -- 172.21.15.32:0/3920210638 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f1540120eb0 con 0x7f154815ce60 2023-12-20T04:45:46.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.279+0000 7f154659c640 1 -- 172.21.15.32:0/3920210638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1528065ca0 msgr2=0x7f1528068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:45:46.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.279+0000 7f154659c640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1528065ca0 0x7f1528068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:45:46.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.279+0000 7f153effd640 1 -- 172.21.15.32:0/3920210638 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f15400ee4d0 con 0x7f154815ce60 2023-12-20T04:45:46.341 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:46.342+0000 7fb0d0af0180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:45:46.428 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:46.430+0000 7fb0d0af0180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:45:46.428 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:46.431+0000 7f366ca48180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:45:46.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.435+0000 7f154c96d640 1 -- 172.21.15.32:0/3920210638 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f154804e000 con 0x7f154815ce60 2023-12-20T04:45:46.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.435+0000 7f153effd640 1 -- 172.21.15.32:0/3920210638 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v110) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1540009510 con 0x7f154815ce60 2023-12-20T04:45:46.435 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:45:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 -- 172.21.15.32:0/3920210638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1528065ca0 msgr2=0x7f1528068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:45:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1528065ca0 0x7f1528068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 -- 172.21.15.32:0/3920210638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f154815ce60 msgr2=0x7f154815bd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f154815ce60 0x7f154815bd40 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f1540025f40 tx=0x7f154001bf90 comp rx=0 tx=0).stop 2023-12-20T04:45:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 -- 172.21.15.32:0/3920210638 shutdown_connections 2023-12-20T04:45:46.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1528065ca0 0x7f1528068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f154815ce60 0x7f154815bd40 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15480d3130 0x7f154815ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 --2- 172.21.15.32:0/3920210638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15480d14d0 0x7f15481f5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 -- 172.21.15.32:0/3920210638 >> 172.21.15.32:0/3920210638 conn(0x7f154806b5e0 msgr2=0x7f154806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 -- 172.21.15.32:0/3920210638 shutdown_connections 2023-12-20T04:45:46.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.439+0000 7f154c96d640 1 -- 172.21.15.32:0/3920210638 wait complete. 2023-12-20T04:45:46.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:45:46.521 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:46.523+0000 7f366ca48180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:45:46.584 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:46.586+0000 7fb0d0af0180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:45:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.607+0000 7f8c3b398640 1 Processor -- start 2023-12-20T04:45:46.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.607+0000 7f8c3b398640 1 -- start start 2023-12-20T04:45:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.607+0000 7f8c3b398640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c3415ce60 0x7f8c340d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c3b398640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c340d2650 con 0x7f8c340d14d0 2023-12-20T04:45:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c3b398640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c340d3570 con 0x7f8c340d3130 2023-12-20T04:45:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c3b398640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c340d1a40 con 0x7f8c3415ce60 2023-12-20T04:45:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c39935640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8c340d14d0 0x7f8c340b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54188/0 (socket says 172.21.15.32:54188) 2023-12-20T04:45:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c39935640 1 -- 172.21.15.32:0/1602080286 learned_addr learned my addr 172.21.15.32:0/1602080286 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:46.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c39134640 1 --2- 172.21.15.32:0/1602080286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c3415ce60 0x7f8c340d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c2bfff640 1 -- 172.21.15.32:0/1602080286 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 644347246 0 0) 0x7f8c340d2650 con 0x7f8c340d14d0 2023-12-20T04:45:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c2bfff640 1 -- 172.21.15.32:0/1602080286 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c20003530 con 0x7f8c340d14d0 2023-12-20T04:45:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c2bfff640 1 -- 172.21.15.32:0/1602080286 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863673049 0 0) 0x7f8c340d3570 con 0x7f8c340d3130 2023-12-20T04:45:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c2bfff640 1 -- 172.21.15.32:0/1602080286 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8c340d2650 con 0x7f8c340d3130 2023-12-20T04:45:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c39134640 1 -- 172.21.15.32:0/1602080286 >> v1:172.21.15.39:6789/0 conn(0x7f8c340d3130 legacy=0x7f8c340d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c39134640 1 -- 172.21.15.32:0/1602080286 >> v1:172.21.15.32:6789/0 conn(0x7f8c340d14d0 legacy=0x7f8c340b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c39134640 1 -- 172.21.15.32:0/1602080286 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c340d1d20 con 0x7f8c3415ce60 2023-12-20T04:45:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.611+0000 7f8c39134640 1 --2- 172.21.15.32:0/1602080286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c3415ce60 0x7f8c340d2bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8c300097f0 tx=0x7f8c3001ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3f0c8d93a6146cc1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c2bfff640 1 -- 172.21.15.32:0/1602080286 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300a89b0 con 0x7f8c3415ce60 2023-12-20T04:45:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c2bfff640 1 -- 172.21.15.32:0/1602080286 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c300a8b10 con 0x7f8c3415ce60 2023-12-20T04:45:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c2bfff640 1 -- 172.21.15.32:0/1602080286 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300a8ea0 con 0x7f8c3415ce60 2023-12-20T04:45:46.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 -- 172.21.15.32:0/1602080286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c3415ce60 msgr2=0x7f8c340d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 --2- 172.21.15.32:0/1602080286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c3415ce60 0x7f8c340d2bf0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f8c300097f0 tx=0x7f8c3001ecb0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 -- 172.21.15.32:0/1602080286 shutdown_connections 2023-12-20T04:45:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 --2- 172.21.15.32:0/1602080286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c3415ce60 0x7f8c340d2bf0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 -- 172.21.15.32:0/1602080286 >> 172.21.15.32:0/1602080286 conn(0x7f8c3406b5e0 msgr2=0x7f8c3406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 -- 172.21.15.32:0/1602080286 shutdown_connections 2023-12-20T04:45:46.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 -- 172.21.15.32:0/1602080286 wait complete. 2023-12-20T04:45:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 Processor -- start 2023-12-20T04:45:46.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 -- start start 2023-12-20T04:45:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340d14d0 0x7f8c34152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c340d3130 0x7f8c34155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3415ce60 0x7f8c34154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c341e6be0 con 0x7f8c340d14d0 2023-12-20T04:45:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c34152ef0 con 0x7f8c3415ce60 2023-12-20T04:45:46.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c3b398640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c341e6520 con 0x7f8c340d3130 2023-12-20T04:45:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c39134640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340d14d0 0x7f8c34152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c39134640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340d14d0 0x7f8c34152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53270/0 (socket says 172.21.15.32:53270) 2023-12-20T04:45:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.615+0000 7f8c39134640 1 -- 172.21.15.32:0/325792700 learned_addr learned my addr 172.21.15.32:0/325792700 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c39935640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3415ce60 0x7f8c34154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c38933640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c340d3130 0x7f8c34155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c39134640 1 -- 172.21.15.32:0/325792700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c340d3130 msgr2=0x7f8c34155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c39134640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c340d3130 0x7f8c34155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c39134640 1 -- 172.21.15.32:0/325792700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3415ce60 msgr2=0x7f8c34154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c39134640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3415ce60 0x7f8c34154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c39134640 1 -- 172.21.15.32:0/325792700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c30009420 con 0x7f8c340d14d0 2023-12-20T04:45:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c39134640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340d14d0 0x7f8c34152910 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f8c3001b8c0 tx=0x7f8c3001ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c29ffb640 1 -- 172.21.15.32:0/325792700 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300b0d80 con 0x7f8c340d14d0 2023-12-20T04:45:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c29ffb640 1 -- 172.21.15.32:0/325792700 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c300b9430 con 0x7f8c340d14d0 2023-12-20T04:45:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c29ffb640 1 -- 172.21.15.32:0/325792700 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c300b97e0 con 0x7f8c340d14d0 2023-12-20T04:45:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c3b398640 1 -- 172.21.15.32:0/325792700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c341e6800 con 0x7f8c340d14d0 2023-12-20T04:45:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c3b398640 1 -- 172.21.15.32:0/325792700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c34155bd0 con 0x7f8c340d14d0 2023-12-20T04:45:46.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c3b398640 1 -- 172.21.15.32:0/325792700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c341e6d20 con 0x7f8c340d14d0 2023-12-20T04:45:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.619+0000 7f8c29ffb640 1 -- 172.21.15.32:0/325792700 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 54) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f8c300b7040 con 0x7f8c340d14d0 2023-12-20T04:45:46.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.623+0000 7f8c29ffb640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c18065ca0 0x7f8c18068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.623+0000 7f8c29ffb640 1 -- 172.21.15.32:0/325792700 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7f8c301210d0 con 0x7f8c340d14d0 2023-12-20T04:45:46.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.623+0000 7f8c38933640 1 -- 172.21.15.32:0/325792700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c18065ca0 msgr2=0x7f8c18068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:45:46.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.623+0000 7f8c38933640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c18065ca0 0x7f8c18068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:45:46.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.623+0000 7f8c29ffb640 1 -- 172.21.15.32:0/325792700 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8c300ee640 con 0x7f8c340d14d0 2023-12-20T04:45:46.672 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:46.674+0000 7fb0d0af0180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:45:46.685 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:46.687+0000 7f366ca48180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:45:46.774 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:46.775+0000 7f366ca48180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:45:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 -- 172.21.15.32:0/325792700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8c3404e000 con 0x7f8c340d14d0 2023-12-20T04:45:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c29ffb640 1 -- 172.21.15.32:0/325792700 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v110) v1 ==== 104+0+11 (secure 0 0 0) 0x7f8c300f1c60 con 0x7f8c340d14d0 2023-12-20T04:45:46.780 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 -- 172.21.15.32:0/325792700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c18065ca0 msgr2=0x7f8c18068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c18065ca0 0x7f8c18068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 -- 172.21.15.32:0/325792700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340d14d0 msgr2=0x7f8c34152910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340d14d0 0x7f8c34152910 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7f8c3001b8c0 tx=0x7f8c3001ba50 comp rx=0 tx=0).stop 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 -- 172.21.15.32:0/325792700 shutdown_connections 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c18065ca0 0x7f8c18068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c3415ce60 0x7f8c34154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c340d3130 0x7f8c34155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 --2- 172.21.15.32:0/325792700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c340d14d0 0x7f8c34152910 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 -- 172.21.15.32:0/325792700 >> 172.21.15.32:0/325792700 conn(0x7f8c3406b5e0 msgr2=0x7f8c34151fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 -- 172.21.15.32:0/325792700 shutdown_connections 2023-12-20T04:45:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.783+0000 7f8c3b398640 1 -- 172.21.15.32:0/325792700 wait complete. 2023-12-20T04:45:46.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:45:46.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.951+0000 7fdb955a2640 1 Processor -- start 2023-12-20T04:45:46.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.951+0000 7fdb955a2640 1 -- start start 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb955a2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb9006a430 0x7fdb900cd4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb955a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 0x7fdb900711b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb955a2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdb9015d3e0 con 0x7fdb9015d000 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb955a2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdb900ce330 con 0x7fdb9006a430 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb955a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb90161640 con 0x7fdb90072030 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb8f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb9006a430 0x7fdb900cd4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb8f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdb9006a430 0x7fdb900cd4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53280/0 (socket says 172.21.15.32:53280) 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb8f7fe640 1 -- 172.21.15.32:0/572953068 learned_addr learned my addr 172.21.15.32:0/572953068 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb8ffff640 1 --1- 172.21.15.32:0/572953068 >> v1:172.21.15.32:6789/0 conn(0x7fdb9015d000 0x7fdb90069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54198/0 (socket says 172.21.15.32:54198) 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb94da1640 1 --2- 172.21.15.32:0/572953068 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 0x7fdb900711b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb8f7fe640 1 -- 172.21.15.32:0/572953068 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 msgr2=0x7fdb900711b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:46.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb8f7fe640 1 --2- 172.21.15.32:0/572953068 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 0x7fdb900711b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb8f7fe640 1 -- 172.21.15.32:0/572953068 >> v1:172.21.15.32:6789/0 conn(0x7fdb9015d000 legacy=0x7fdb90069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb8f7fe640 1 -- 172.21.15.32:0/572953068 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb900cdd70 con 0x7fdb9006a430 2023-12-20T04:45:46.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb94da1640 1 --2- 172.21.15.32:0/572953068 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 0x7fdb900711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:46.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.955+0000 7fdb8f7fe640 1 --2- 172.21.15.32:0/572953068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9006a430 0x7fdb900cd4d0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009870 tx=0x7fdb8401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c22291e44ea92f87 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:46.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb8effd640 1 -- 172.21.15.32:0/572953068 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb840ab620 con 0x7fdb9006a430 2023-12-20T04:45:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb8effd640 1 -- 172.21.15.32:0/572953068 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb840ab780 con 0x7fdb9006a430 2023-12-20T04:45:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb8effd640 1 -- 172.21.15.32:0/572953068 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb840abb10 con 0x7fdb9006a430 2023-12-20T04:45:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 -- 172.21.15.32:0/572953068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9006a430 msgr2=0x7fdb900cd4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 --2- 172.21.15.32:0/572953068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9006a430 0x7fdb900cd4d0 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009870 tx=0x7fdb8401c480 comp rx=0 tx=0).stop 2023-12-20T04:45:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 -- 172.21.15.32:0/572953068 shutdown_connections 2023-12-20T04:45:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 --2- 172.21.15.32:0/572953068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9006a430 0x7fdb900cd4d0 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 --2- 172.21.15.32:0/572953068 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 0x7fdb900711b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 -- 172.21.15.32:0/572953068 >> 172.21.15.32:0/572953068 conn(0x7fdb9006d150 msgr2=0x7fdb9006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 -- 172.21.15.32:0/572953068 shutdown_connections 2023-12-20T04:45:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 -- 172.21.15.32:0/572953068 wait complete. 2023-12-20T04:45:46.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 Processor -- start 2023-12-20T04:45:46.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 -- start start 2023-12-20T04:45:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb9006a430 0x7fdb9014f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 0x7fdb9015beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9015d000 0x7fdb9015c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9014fa40 con 0x7fdb9015d000 2023-12-20T04:45:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdb90155350 con 0x7fdb9006a430 2023-12-20T04:45:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb955a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdb9015eb40 con 0x7fdb90072030 2023-12-20T04:45:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.959+0000 7fdb8f7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 0x7fdb9015beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb94da1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9015d000 0x7fdb9015c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8f7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 0x7fdb9015beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45284/0 (socket says 172.21.15.32:45284) 2023-12-20T04:45:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8f7fe640 1 -- 172.21.15.32:0/2656682609 learned_addr learned my addr 172.21.15.32:0/2656682609 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8ffff640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb9006a430 0x7fdb9014f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8f7fe640 1 -- 172.21.15.32:0/2656682609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb9006a430 msgr2=0x7fdb9014f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8f7fe640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb9006a430 0x7fdb9014f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8f7fe640 1 -- 172.21.15.32:0/2656682609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9015d000 msgr2=0x7fdb9015c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8f7fe640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9015d000 0x7fdb9015c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:46.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8f7fe640 1 -- 172.21.15.32:0/2656682609 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdb840094d0 con 0x7fdb90072030 2023-12-20T04:45:46.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8f7fe640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 0x7fdb9015beb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdb8401a0b0 tx=0x7fdb8401acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:46.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8cff9640 1 -- 172.21.15.32:0/2656682609 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb840be3f0 con 0x7fdb90072030 2023-12-20T04:45:46.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8cff9640 1 -- 172.21.15.32:0/2656682609 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdb840be550 con 0x7fdb90072030 2023-12-20T04:45:46.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb955a2640 1 -- 172.21.15.32:0/2656682609 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdb900d0390 con 0x7fdb90072030 2023-12-20T04:45:46.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8cff9640 1 -- 172.21.15.32:0/2656682609 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdb840be900 con 0x7fdb90072030 2023-12-20T04:45:46.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb955a2640 1 -- 172.21.15.32:0/2656682609 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdb90154a20 con 0x7fdb90072030 2023-12-20T04:45:46.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb955a2640 1 -- 172.21.15.32:0/2656682609 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdb90074100 con 0x7fdb90072030 2023-12-20T04:45:46.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8cff9640 1 -- 172.21.15.32:0/2656682609 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 54) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fdb840b43f0 con 0x7fdb90072030 2023-12-20T04:45:46.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.963+0000 7fdb8cff9640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb7c065ca0 0x7fdb7c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:46.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.967+0000 7fdb8ffff640 1 -- 172.21.15.32:0/2656682609 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb7c065ca0 msgr2=0x7fdb7c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:45:46.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.967+0000 7fdb8cff9640 1 -- 172.21.15.32:0/2656682609 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(110..110 src has 1..110) v4 ==== 6111+0+0 (secure 0 0 0) 0x7fdb8411d470 con 0x7fdb90072030 2023-12-20T04:45:46.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.967+0000 7fdb8ffff640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb7c065ca0 0x7fdb7c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:45:46.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:46.967+0000 7fdb8cff9640 1 -- 172.21.15.32:0/2656682609 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdb840ea960 con 0x7fdb90072030 2023-12-20T04:45:47.006 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:47.010+0000 7fb0d0af0180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:45:47.069 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:45:47.074+0000 7fb0d0af0180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:45:47.116 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.119+0000 7f366ca48180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:45:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.127+0000 7fdb955a2640 1 -- 172.21.15.32:0/2656682609 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdb900cdab0 con 0x7fdb90072030 2023-12-20T04:45:47.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.135+0000 7fdb8cff9640 1 -- 172.21.15.32:0/2656682609 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mgrmap(e 55) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fdb840ead30 con 0x7fdb90072030 2023-12-20T04:45:47.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.135+0000 7fdb8cff9640 1 -- 172.21.15.32:0/2656682609 <== mon.2 v2:172.21.15.39:3301/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v110) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdb84103310 con 0x7fdb90072030 2023-12-20T04:45:47.134 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:45:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 -- 172.21.15.32:0/2656682609 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb7c065ca0 msgr2=0x7fdb7c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:45:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb7c065ca0 0x7fdb7c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 -- 172.21.15.32:0/2656682609 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 msgr2=0x7fdb9015beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 0x7fdb9015beb0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fdb8401a0b0 tx=0x7fdb8401acc0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 -- 172.21.15.32:0/2656682609 shutdown_connections 2023-12-20T04:45:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdb7c065ca0 0x7fdb7c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdb9015d000 0x7fdb9015c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdb90072030 0x7fdb9015beb0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 --2- 172.21.15.32:0/2656682609 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdb9006a430 0x7fdb9014f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 -- 172.21.15.32:0/2656682609 >> 172.21.15.32:0/2656682609 conn(0x7fdb9006d150 msgr2=0x7fdb90159e90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 -- 172.21.15.32:0/2656682609 shutdown_connections 2023-12-20T04:45:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.139+0000 7fdb955a2640 1 -- 172.21.15.32:0/2656682609 wait complete. 2023-12-20T04:45:47.156 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:45:47.184 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.187+0000 7f366ca48180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:45:47.258 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.259+0000 7f35f4589640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:47.258 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.259+0000 7f35f4589640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:47.258 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.259+0000 7f35f4589640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:47.258 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.259+0000 7f35f4589640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:47.258 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.259+0000 7f35f4589640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:47.259 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.263+0000 7f35f6d8e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:47.259 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.263+0000 7f35f6d8e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:47.259 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.263+0000 7f35f6d8e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:47.259 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.263+0000 7f35f6d8e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:47.260 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:45:47.263+0000 7f35f6d8e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:45:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.315+0000 7ff18981f640 1 Processor -- start 2023-12-20T04:45:47.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.315+0000 7ff18981f640 1 -- start start 2023-12-20T04:45:47.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff18981f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1840cfc80 0x7ff184167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff18981f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1840b6ca0 con 0x7ff18415cdb0 2023-12-20T04:45:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff18981f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff18415d190 con 0x7ff1840d2740 2023-12-20T04:45:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff18981f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1840d16a0 con 0x7ff1840cfc80 2023-12-20T04:45:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff1837fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1840cfc80 0x7ff184167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff182ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff18415cdb0 0x7ff1840d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54214/0 (socket says 172.21.15.32:54214) 2023-12-20T04:45:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff182ffd640 1 -- 172.21.15.32:0/923896088 learned_addr learned my addr 172.21.15.32:0/923896088 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:47.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff17bfff640 1 --1- 172.21.15.32:0/923896088 >> v1:172.21.15.39:6789/0 conn(0x7ff1840d2740 0x7ff1840cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:40974/0 (socket says 172.21.15.32:40974) 2023-12-20T04:45:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff1837fe640 1 -- 172.21.15.32:0/923896088 >> v1:172.21.15.39:6789/0 conn(0x7ff1840d2740 legacy=0x7ff1840cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff1837fe640 1 -- 172.21.15.32:0/923896088 >> v1:172.21.15.32:6789/0 conn(0x7ff18415cdb0 legacy=0x7ff1840d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff1837fe640 1 -- 172.21.15.32:0/923896088 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1840d1980 con 0x7ff1840cfc80 2023-12-20T04:45:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.319+0000 7ff1827fc640 1 -- 172.21.15.32:0/923896088 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3249174663 0 0) 0x7ff174001120 con 0x7ff18415cdb0 2023-12-20T04:45:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff1837fe640 1 --2- 172.21.15.32:0/923896088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840cfc80 0x7ff184167910 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009d90 tx=0x7ff17c01f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c333d053f0984b5a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff1827fc640 1 -- 172.21.15.32:0/923896088 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0af600 con 0x7ff1840cfc80 2023-12-20T04:45:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff1827fc640 1 -- 172.21.15.32:0/923896088 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c0ab050 con 0x7ff1840cfc80 2023-12-20T04:45:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff1827fc640 1 -- 172.21.15.32:0/923896088 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0af980 con 0x7ff1840cfc80 2023-12-20T04:45:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 -- 172.21.15.32:0/923896088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840cfc80 msgr2=0x7ff184167910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 --2- 172.21.15.32:0/923896088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840cfc80 0x7ff184167910 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7ff17c009d90 tx=0x7ff17c01f130 comp rx=0 tx=0).stop 2023-12-20T04:45:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 -- 172.21.15.32:0/923896088 shutdown_connections 2023-12-20T04:45:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 --2- 172.21.15.32:0/923896088 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1840cfc80 0x7ff184167910 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 -- 172.21.15.32:0/923896088 >> 172.21.15.32:0/923896088 conn(0x7ff184065ee0 msgr2=0x7ff18406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 -- 172.21.15.32:0/923896088 shutdown_connections 2023-12-20T04:45:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 -- 172.21.15.32:0/923896088 wait complete. 2023-12-20T04:45:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 Processor -- start 2023-12-20T04:45:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 -- start start 2023-12-20T04:45:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff1840cfc80 0x7ff1840c7db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1840d2740 0x7ff184157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415cdb0 0x7ff184158380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff184155860 con 0x7ff18415cdb0 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff184155ff0 con 0x7ff1840d2740 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.323+0000 7ff18981f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1841562d0 con 0x7ff1840cfc80 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff1837fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415cdb0 0x7ff184158380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff1837fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415cdb0 0x7ff184158380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53422/0 (socket says 172.21.15.32:53422) 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff1837fe640 1 -- 172.21.15.32:0/3681514624 learned_addr learned my addr 172.21.15.32:0/3681514624 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff182ffd640 1 --2- 172.21.15.32:0/3681514624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff1840cfc80 0x7ff1840c7db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff17bfff640 1 --2- 172.21.15.32:0/3681514624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1840d2740 0x7ff184157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff1837fe640 1 -- 172.21.15.32:0/3681514624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff1840cfc80 msgr2=0x7ff1840c7db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff1837fe640 1 --2- 172.21.15.32:0/3681514624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff1840cfc80 0x7ff1840c7db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff1837fe640 1 -- 172.21.15.32:0/3681514624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1840d2740 msgr2=0x7ff184157e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff1837fe640 1 --2- 172.21.15.32:0/3681514624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1840d2740 0x7ff184157e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff1837fe640 1 -- 172.21.15.32:0/3681514624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff17c009a70 con 0x7ff18415cdb0 2023-12-20T04:45:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff1837fe640 1 --2- 172.21.15.32:0/3681514624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415cdb0 0x7ff184158380 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff17c01cd60 tx=0x7ff17c01bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff180ff9640 1 -- 172.21.15.32:0/3681514624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0b9610 con 0x7ff18415cdb0 2023-12-20T04:45:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff180ff9640 1 -- 172.21.15.32:0/3681514624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff17c0b9770 con 0x7ff18415cdb0 2023-12-20T04:45:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff180ff9640 1 -- 172.21.15.32:0/3681514624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff17c0b9b00 con 0x7ff18415cdb0 2023-12-20T04:45:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff18981f640 1 -- 172.21.15.32:0/3681514624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1841565b0 con 0x7ff18415cdb0 2023-12-20T04:45:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff18981f640 1 -- 172.21.15.32:0/3681514624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1840c82f0 con 0x7ff18415cdb0 2023-12-20T04:45:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff18981f640 1 -- 172.21.15.32:0/3681514624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff18404e000 con 0x7ff18415cdb0 2023-12-20T04:45:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.327+0000 7ff180ff9640 1 -- 172.21.15.32:0/3681514624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 56) v1 ==== 84939+0+0 (secure 0 0 0) 0x7ff17c0b6050 con 0x7ff18415cdb0 2023-12-20T04:45:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.331+0000 7ff180ff9640 1 -- 172.21.15.32:0/3681514624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6455+0+0 (secure 0 0 0) 0x7ff17c121540 con 0x7ff18415cdb0 2023-12-20T04:45:47.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.335+0000 7ff180ff9640 1 -- 172.21.15.32:0/3681514624 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff17c0ef0e0 con 0x7ff18415cdb0 2023-12-20T04:45:47.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.523+0000 7ff18981f640 1 -- 172.21.15.32:0/3681514624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff1840d21f0 con 0x7ff18415cdb0 2023-12-20T04:45:47.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.523+0000 7ff180ff9640 1 -- 172.21.15.32:0/3681514624 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4711 (secure 0 0 0) 0x7ff17c0b4090 con 0x7ff18415cdb0 2023-12-20T04:45:47.522 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:47.522 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1072056415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1072056415},{"type":"v1","addr":"172.21.15.39:6835","nonce":1072056415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3512326822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3512326822},{"type":"v1","addr":"172.21.15.32:6836","nonce":3512326822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2163601073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2163601073},{"type":"v1","addr":"172.21.15.32:6837","nonce":2163601073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:44:27.818499+0000","modified":"2023-12-20T04:44:30.831088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7359":{"gid":7359,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/2369052657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2369052657},{"type":"v1","addr":"172.21.15.39:6833","nonce":2369052657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-20T04:45:47.523 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 46 2023-12-20T04:45:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.527+0000 7ff18981f640 1 -- 172.21.15.32:0/3681514624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415cdb0 msgr2=0x7ff184158380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.527+0000 7ff18981f640 1 --2- 172.21.15.32:0/3681514624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415cdb0 0x7ff184158380 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7ff17c01cd60 tx=0x7ff17c01bed0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.527+0000 7ff18981f640 1 -- 172.21.15.32:0/3681514624 shutdown_connections 2023-12-20T04:45:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.527+0000 7ff18981f640 1 --2- 172.21.15.32:0/3681514624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff18415cdb0 0x7ff184158380 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.527+0000 7ff18981f640 1 --2- 172.21.15.32:0/3681514624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1840d2740 0x7ff184157e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.527+0000 7ff18981f640 1 --2- 172.21.15.32:0/3681514624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff1840cfc80 0x7ff1840c7db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.527+0000 7ff18981f640 1 -- 172.21.15.32:0/3681514624 >> 172.21.15.32:0/3681514624 conn(0x7ff184065ee0 msgr2=0x7ff1840d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.527+0000 7ff18981f640 1 -- 172.21.15.32:0/3681514624 shutdown_connections 2023-12-20T04:45:47.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.527+0000 7ff18981f640 1 -- 172.21.15.32:0/3681514624 wait complete. 2023-12-20T04:45:47.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:45:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.731+0000 7ff68b073640 1 Processor -- start 2023-12-20T04:45:47.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.731+0000 7ff68b073640 1 -- start start 2023-12-20T04:45:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.731+0000 7ff68b073640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6840aece0 0x7ff6840b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.731+0000 7ff68b073640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6840c98c0 con 0x7ff6840b0b30 2023-12-20T04:45:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff68b073640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6840c9ba0 con 0x7ff6841599e0 2023-12-20T04:45:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff68b073640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6840c9e80 con 0x7ff6840aece0 2023-12-20T04:45:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff683fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6840aece0 0x7ff6840b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff683fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6840aece0 0x7ff6840b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53428/0 (socket says 172.21.15.32:53428) 2023-12-20T04:45:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff683fff640 1 -- 172.21.15.32:0/1742956942 learned_addr learned my addr 172.21.15.32:0/1742956942 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:47.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff689610640 1 --1- 172.21.15.32:0/1742956942 >> v1:172.21.15.32:6789/0 conn(0x7ff6840b0b30 0x7ff6840701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54224/0 (socket says 172.21.15.32:54224) 2023-12-20T04:45:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff683fff640 1 -- 172.21.15.32:0/1742956942 >> v1:172.21.15.39:6789/0 conn(0x7ff6841599e0 legacy=0x7ff6840c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff683fff640 1 -- 172.21.15.32:0/1742956942 >> v1:172.21.15.32:6789/0 conn(0x7ff6840b0b30 legacy=0x7ff6840701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff683fff640 1 -- 172.21.15.32:0/1742956942 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6840ca160 con 0x7ff6840aece0 2023-12-20T04:45:47.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff6837fe640 1 -- 172.21.15.32:0/1742956942 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2409139442 0 0) 0x7ff6840c9ba0 con 0x7ff6841599e0 2023-12-20T04:45:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff683fff640 1 --2- 172.21.15.32:0/1742956942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6840aece0 0x7ff6840b05f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff670009870 tx=0x7ff67001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f30824a7eb205bfd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff6837fe640 1 -- 172.21.15.32:0/1742956942 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700ab650 con 0x7ff6840aece0 2023-12-20T04:45:47.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff6837fe640 1 -- 172.21.15.32:0/1742956942 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6700ab7b0 con 0x7ff6840aece0 2023-12-20T04:45:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.735+0000 7ff6837fe640 1 -- 172.21.15.32:0/1742956942 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700abb40 con 0x7ff6840aece0 2023-12-20T04:45:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 -- 172.21.15.32:0/1742956942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6840aece0 msgr2=0x7ff6840b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 --2- 172.21.15.32:0/1742956942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6840aece0 0x7ff6840b05f0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7ff670009870 tx=0x7ff67001c540 comp rx=0 tx=0).stop 2023-12-20T04:45:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 -- 172.21.15.32:0/1742956942 shutdown_connections 2023-12-20T04:45:47.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 --2- 172.21.15.32:0/1742956942 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6840aece0 0x7ff6840b05f0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 -- 172.21.15.32:0/1742956942 >> 172.21.15.32:0/1742956942 conn(0x7ff68406a320 msgr2=0x7ff6840ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 -- 172.21.15.32:0/1742956942 shutdown_connections 2023-12-20T04:45:47.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 -- 172.21.15.32:0/1742956942 wait complete. 2023-12-20T04:45:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 Processor -- start 2023-12-20T04:45:47.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 -- start start 2023-12-20T04:45:47.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6840aece0 0x7ff68406c640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6840b0b30 0x7ff68414ea20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6841599e0 0x7ff68414ef60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff684154370 con 0x7ff6840aece0 2023-12-20T04:45:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68414f4a0 con 0x7ff6840b0b30 2023-12-20T04:45:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff68b073640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6840c59d0 con 0x7ff6841599e0 2023-12-20T04:45:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff688e0f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6840aece0 0x7ff68406c640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff688e0f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6840aece0 0x7ff68406c640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53434/0 (socket says 172.21.15.32:53434) 2023-12-20T04:45:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff688e0f640 1 -- 172.21.15.32:0/902633127 learned_addr learned my addr 172.21.15.32:0/902633127 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:47.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff683fff640 1 --2- 172.21.15.32:0/902633127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6840b0b30 0x7ff68414ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.739+0000 7ff689610640 1 --2- 172.21.15.32:0/902633127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6841599e0 0x7ff68414ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff688e0f640 1 -- 172.21.15.32:0/902633127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6841599e0 msgr2=0x7ff68414ef60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff688e0f640 1 --2- 172.21.15.32:0/902633127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6841599e0 0x7ff68414ef60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff688e0f640 1 -- 172.21.15.32:0/902633127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6840b0b30 msgr2=0x7ff68414ea20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff688e0f640 1 --2- 172.21.15.32:0/902633127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6840b0b30 0x7ff68414ea20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff688e0f640 1 -- 172.21.15.32:0/902633127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6700094d0 con 0x7ff6840aece0 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff683fff640 1 --2- 172.21.15.32:0/902633127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6840b0b30 0x7ff68414ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff688e0f640 1 --2- 172.21.15.32:0/902633127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6840aece0 0x7ff68406c640 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff67800c3c0 tx=0x7ff67800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff6817fa640 1 -- 172.21.15.32:0/902633127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6780054e0 con 0x7ff6840aece0 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff6817fa640 1 -- 172.21.15.32:0/902633127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff678003650 con 0x7ff6840aece0 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff6817fa640 1 -- 172.21.15.32:0/902633127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678003a00 con 0x7ff6840aece0 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff68b073640 1 -- 172.21.15.32:0/902633127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6840c5cb0 con 0x7ff6840aece0 2023-12-20T04:45:47.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff68b073640 1 -- 172.21.15.32:0/902633127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6840c6270 con 0x7ff6840aece0 2023-12-20T04:45:47.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff68b073640 1 -- 172.21.15.32:0/902633127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68404e000 con 0x7ff6840aece0 2023-12-20T04:45:47.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.743+0000 7ff6817fa640 1 -- 172.21.15.32:0/902633127 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 56) v1 ==== 84939+0+0 (secure 0 0 0) 0x7ff678003b60 con 0x7ff6840aece0 2023-12-20T04:45:47.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.747+0000 7ff6817fa640 1 -- 172.21.15.32:0/902633127 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6455+0+0 (secure 0 0 0) 0x7ff6780824c0 con 0x7ff6840aece0 2023-12-20T04:45:47.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.751+0000 7ff6817fa640 1 -- 172.21.15.32:0/902633127 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff67804ffa0 con 0x7ff6840aece0 2023-12-20T04:45:47.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.915+0000 7ff68b073640 1 -- 172.21.15.32:0/902633127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7ff6840cf550 con 0x7ff6840aece0 2023-12-20T04:45:47.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.915+0000 7ff6817fa640 1 -- 172.21.15.32:0/902633127 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+17134 (secure 0 0 0) 0x7ff678017090 con 0x7ff6840aece0 2023-12-20T04:45:47.914 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:47.914 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:45:47.191931+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:44:25.451651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-20T04:44:26.819848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.4","mappings":[{"from":6,"to":3}]},{"pgid":"13.16","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/842509018":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/2758895268":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/2468150930":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/46584708":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/950991327":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/2883211641":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/3801303150":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/1531123683":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3296285907":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3289362372":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/998392483":"2023-12-21T04:44:39.978724+0000","172.21.15.32:6833/38009":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/1838475563":"2023-12-21T04:45:47.191868+0000","172.21.15.32:6832/38009":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3368879992":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/2997487195":"2023-12-21T04:44:39.978724+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:45:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.919+0000 7ff68b073640 1 -- 172.21.15.32:0/902633127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6840aece0 msgr2=0x7ff68406c640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.919+0000 7ff68b073640 1 --2- 172.21.15.32:0/902633127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6840aece0 0x7ff68406c640 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7ff67800c3c0 tx=0x7ff67800c880 comp rx=0 tx=0).stop 2023-12-20T04:45:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.919+0000 7ff68b073640 1 -- 172.21.15.32:0/902633127 shutdown_connections 2023-12-20T04:45:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.919+0000 7ff68b073640 1 --2- 172.21.15.32:0/902633127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6841599e0 0x7ff68414ef60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.919+0000 7ff68b073640 1 --2- 172.21.15.32:0/902633127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6840b0b30 0x7ff68414ea20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.919+0000 7ff68b073640 1 --2- 172.21.15.32:0/902633127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6840aece0 0x7ff68406c640 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.919+0000 7ff68b073640 1 -- 172.21.15.32:0/902633127 >> 172.21.15.32:0/902633127 conn(0x7ff68406a320 msgr2=0x7ff6840ce890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.919+0000 7ff68b073640 1 -- 172.21.15.32:0/902633127 shutdown_connections 2023-12-20T04:45:47.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:47.919+0000 7ff68b073640 1 -- 172.21.15.32:0/902633127 wait complete. 2023-12-20T04:45:47.936 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T04:45:47.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:45:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72aa666640 1 Processor -- start 2023-12-20T04:45:48.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72aa666640 1 -- start start 2023-12-20T04:45:48.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72aa666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72a41599e0 0x7f72a40aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72aa666640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f72a40c90e0 0x7f72a40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72aa666640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72a40711d0 con 0x7f72a40b0b30 2023-12-20T04:45:48.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72aa666640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72a40c98c0 con 0x7f72a40c90e0 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72aa666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a40c9ba0 con 0x7f72a41599e0 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72a8c03640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72a40b0b30 0x7f72a415d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54230/0 (socket says 172.21.15.32:54230) 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72a8c03640 1 -- 172.21.15.32:0/1405655101 learned_addr learned my addr 172.21.15.32:0/1405655101 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72a37fe640 1 --2- 172.21.15.32:0/1405655101 >> v2:172.21.15.39:3300/0 conn(0x7f72a40c90e0 0x7f72a40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72a3fff640 1 --2- 172.21.15.32:0/1405655101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72a41599e0 0x7f72a40aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72a2ffd640 1 -- 172.21.15.32:0/1405655101 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2229358100 0 0) 0x7f72a40711d0 con 0x7f72a40b0b30 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72a2ffd640 1 -- 172.21.15.32:0/1405655101 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f728c003530 con 0x7f72a40b0b30 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72a37fe640 1 -- 172.21.15.32:0/1405655101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72a41599e0 msgr2=0x7f72a40aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72a37fe640 1 --2- 172.21.15.32:0/1405655101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72a41599e0 0x7f72a40aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72a37fe640 1 -- 172.21.15.32:0/1405655101 >> v1:172.21.15.32:6789/0 conn(0x7f72a40b0b30 legacy=0x7f72a415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72a37fe640 1 -- 172.21.15.32:0/1405655101 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72a40c9e80 con 0x7f72a40c90e0 2023-12-20T04:45:48.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.083+0000 7f72a3fff640 1 --2- 172.21.15.32:0/1405655101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72a41599e0 0x7f72a40aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72a37fe640 1 --2- 172.21.15.32:0/1405655101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a40c90e0 0x7f72a40b05f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f72980097c0 tx=0x7f729801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7cd442b2337e6763 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72a2ffd640 1 -- 172.21.15.32:0/1405655101 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980ab600 con 0x7f72a40c90e0 2023-12-20T04:45:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72a2ffd640 1 -- 172.21.15.32:0/1405655101 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72980a7050 con 0x7f72a40c90e0 2023-12-20T04:45:48.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72a2ffd640 1 -- 172.21.15.32:0/1405655101 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980ab980 con 0x7f72a40c90e0 2023-12-20T04:45:48.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72aa666640 1 -- 172.21.15.32:0/1405655101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a40c90e0 msgr2=0x7f72a40b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72aa666640 1 --2- 172.21.15.32:0/1405655101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a40c90e0 0x7f72a40b05f0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7f72980097c0 tx=0x7f729801c450 comp rx=0 tx=0).stop 2023-12-20T04:45:48.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72aa666640 1 -- 172.21.15.32:0/1405655101 shutdown_connections 2023-12-20T04:45:48.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72aa666640 1 --2- 172.21.15.32:0/1405655101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a40c90e0 0x7f72a40b05f0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72aa666640 1 --2- 172.21.15.32:0/1405655101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72a41599e0 0x7f72a40aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72aa666640 1 -- 172.21.15.32:0/1405655101 >> 172.21.15.32:0/1405655101 conn(0x7f72a406a320 msgr2=0x7f72a40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:48.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72aa666640 1 -- 172.21.15.32:0/1405655101 shutdown_connections 2023-12-20T04:45:48.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.087+0000 7f72aa666640 1 -- 172.21.15.32:0/1405655101 wait complete. 2023-12-20T04:45:48.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72aa666640 1 Processor -- start 2023-12-20T04:45:48.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72aa666640 1 -- start start 2023-12-20T04:45:48.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72aa666640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a40b0b30 0x7f72a41f4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72aa666640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40c90e0 0x7f72a41633d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72aa666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72a41599e0 0x7f72a4163910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72aa666640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41474c0 con 0x7f72a40c90e0 2023-12-20T04:45:48.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72aa666640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72a41f5520 con 0x7f72a40b0b30 2023-12-20T04:45:48.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72aa666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72a414ac80 con 0x7f72a41599e0 2023-12-20T04:45:48.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40c90e0 0x7f72a41633d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40c90e0 0x7f72a41633d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53444/0 (socket says 172.21.15.32:53444) 2023-12-20T04:45:48.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a37fe640 1 -- 172.21.15.32:0/2474152728 learned_addr learned my addr 172.21.15.32:0/2474152728 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:48.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a8c03640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72a41599e0 0x7f72a4163910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a3fff640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a40b0b30 0x7f72a41f4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a37fe640 1 -- 172.21.15.32:0/2474152728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72a41599e0 msgr2=0x7f72a4163910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a37fe640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72a41599e0 0x7f72a4163910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a37fe640 1 -- 172.21.15.32:0/2474152728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a40b0b30 msgr2=0x7f72a41f4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a37fe640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a40b0b30 0x7f72a41f4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a37fe640 1 -- 172.21.15.32:0/2474152728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72940093b0 con 0x7f72a40c90e0 2023-12-20T04:45:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a37fe640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40c90e0 0x7f72a41633d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f72980097c0 tx=0x7f72980191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a0ff9640 1 -- 172.21.15.32:0/2474152728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980b4430 con 0x7f72a40c90e0 2023-12-20T04:45:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a0ff9640 1 -- 172.21.15.32:0/2474152728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72980b45b0 con 0x7f72a40c90e0 2023-12-20T04:45:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72aa666640 1 -- 172.21.15.32:0/2474152728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7298009420 con 0x7f72a40c90e0 2023-12-20T04:45:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72a0ff9640 1 -- 172.21.15.32:0/2474152728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72980b4960 con 0x7f72a40c90e0 2023-12-20T04:45:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.091+0000 7f72aa666640 1 -- 172.21.15.32:0/2474152728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72a414b240 con 0x7f72a40c90e0 2023-12-20T04:45:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.095+0000 7f72aa666640 1 -- 172.21.15.32:0/2474152728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72a40cef50 con 0x7f72a40c90e0 2023-12-20T04:45:48.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.095+0000 7f72a0ff9640 1 -- 172.21.15.32:0/2474152728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 56) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f72980b2040 con 0x7f72a40c90e0 2023-12-20T04:45:48.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.095+0000 7f72a0ff9640 1 -- 172.21.15.32:0/2474152728 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6455+0+0 (secure 0 0 0) 0x7f729811ccb0 con 0x7f72a40c90e0 2023-12-20T04:45:48.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.099+0000 7f72a0ff9640 1 -- 172.21.15.32:0/2474152728 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f72980ea790 con 0x7f72a40c90e0 2023-12-20T04:45:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.203+0000 7f72a0ff9640 1 -- 172.21.15.32:0/2474152728 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 57) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f72980ea240 con 0x7f72a40c90e0 2023-12-20T04:45:48.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.203+0000 7f72a0ff9640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f729006b240 0x7f729006d630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.207+0000 7f72a3fff640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f729006b240 0x7f729006d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.207+0000 7f72a3fff640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f729006b240 0x7f729006d630 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7294009c30 tx=0x7f7294006f90 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.299+0000 7f72aa666640 1 -- 172.21.15.32:0/2474152728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f72a404e000 con 0x7f72a40c90e0 2023-12-20T04:45:48.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.299+0000 7f72a0ff9640 1 -- 172.21.15.32:0/2474152728 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f72980f1960 con 0x7f72a40c90e0 2023-12-20T04:45:48.298 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:48.299 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1072056415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1072056415},{"type":"v1","addr":"172.21.15.39:6835","nonce":1072056415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3512326822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3512326822},{"type":"v1","addr":"172.21.15.32:6836","nonce":3512326822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2163601073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2163601073},{"type":"v1","addr":"172.21.15.32:6837","nonce":2163601073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:44:27.818499+0000","modified":"2023-12-20T04:44:30.831088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7359":{"gid":7359,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/2369052657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2369052657},{"type":"v1","addr":"172.21.15.39:6833","nonce":2369052657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-20T04:45:48.299 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 46 2023-12-20T04:45:48.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 -- 172.21.15.32:0/2474152728 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f729006b240 msgr2=0x7f729006d630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f729006b240 0x7f729006d630 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f7294009c30 tx=0x7f7294006f90 comp rx=0 tx=0).stop 2023-12-20T04:45:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 -- 172.21.15.32:0/2474152728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40c90e0 msgr2=0x7f72a41633d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40c90e0 0x7f72a41633d0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f72980097c0 tx=0x7f72980191f0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 -- 172.21.15.32:0/2474152728 shutdown_connections 2023-12-20T04:45:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f729006b240 0x7f729006d630 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72a41599e0 0x7f72a4163910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72a40c90e0 0x7f72a41633d0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 --2- 172.21.15.32:0/2474152728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f72a40b0b30 0x7f72a41f4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 -- 172.21.15.32:0/2474152728 >> 172.21.15.32:0/2474152728 conn(0x7f72a406a320 msgr2=0x7f72a406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:48.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 -- 172.21.15.32:0/2474152728 shutdown_connections 2023-12-20T04:45:48.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.303+0000 7f72aa666640 1 -- 172.21.15.32:0/2474152728 wait complete. 2023-12-20T04:45:48.321 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:45:48.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa947ee5640 1 Processor -- start 2023-12-20T04:45:48.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa947ee5640 1 -- start start 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa947ee5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9401599e0 0x7fa9400aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa947ee5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa9400c90e0 0x7fa9400b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa947ee5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9400b0b30 0x7fa94015d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa947ee5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9400c98c0 con 0x7fa9401599e0 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa947ee5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9400c9ba0 con 0x7fa9400c90e0 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa947ee5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9400c9e80 con 0x7fa9400b0b30 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa945c81640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9401599e0 0x7fa9400aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa945c81640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9401599e0 0x7fa9400aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53446/0 (socket says 172.21.15.32:53446) 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa945c81640 1 -- 172.21.15.32:0/3686187 learned_addr learned my addr 172.21.15.32:0/3686187 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa946482640 1 --2- 172.21.15.32:0/3686187 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9400b0b30 0x7fa94015d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa945c81640 1 -- 172.21.15.32:0/3686187 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9400b0b30 msgr2=0x7fa94015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa945480640 1 --2- 172.21.15.32:0/3686187 >> v2:172.21.15.39:3300/0 conn(0x7fa9400c90e0 0x7fa9400b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa945c81640 1 --2- 172.21.15.32:0/3686187 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9400b0b30 0x7fa94015d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa945c81640 1 -- 172.21.15.32:0/3686187 >> v2:172.21.15.39:3300/0 conn(0x7fa9400c90e0 msgr2=0x7fa9400b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa945c81640 1 --2- 172.21.15.32:0/3686187 >> v2:172.21.15.39:3300/0 conn(0x7fa9400c90e0 0x7fa9400b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa945c81640 1 -- 172.21.15.32:0/3686187 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9400ca160 con 0x7fa9401599e0 2023-12-20T04:45:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa946482640 1 --2- 172.21.15.32:0/3686187 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9400b0b30 0x7fa94015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:45:48.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.467+0000 7fa945480640 1 --2- 172.21.15.32:0/3686187 >> v2:172.21.15.39:3300/0 conn(0x7fa9400c90e0 0x7fa9400b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:45:48.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa945c81640 1 --2- 172.21.15.32:0/3686187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9401599e0 0x7fa9400aece0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa94006ab20 tx=0x7fa93c01e8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed790dcb971582c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa944c7f640 1 -- 172.21.15.32:0/3686187 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0a7970 con 0x7fa9401599e0 2023-12-20T04:45:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa944c7f640 1 -- 172.21.15.32:0/3686187 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c0a7ad0 con 0x7fa9401599e0 2023-12-20T04:45:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa944c7f640 1 -- 172.21.15.32:0/3686187 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0a7e60 con 0x7fa9401599e0 2023-12-20T04:45:48.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa947ee5640 1 -- 172.21.15.32:0/3686187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9401599e0 msgr2=0x7fa9400aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa947ee5640 1 --2- 172.21.15.32:0/3686187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9401599e0 0x7fa9400aece0 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7fa94006ab20 tx=0x7fa93c01e8c0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa947ee5640 1 -- 172.21.15.32:0/3686187 shutdown_connections 2023-12-20T04:45:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa947ee5640 1 --2- 172.21.15.32:0/3686187 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9400b0b30 0x7fa94015d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa947ee5640 1 --2- 172.21.15.32:0/3686187 >> v2:172.21.15.39:3300/0 conn(0x7fa9400c90e0 0x7fa9400b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa947ee5640 1 --2- 172.21.15.32:0/3686187 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9401599e0 0x7fa9400aece0 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa947ee5640 1 -- 172.21.15.32:0/3686187 >> 172.21.15.32:0/3686187 conn(0x7fa94006a320 msgr2=0x7fa9400ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa947ee5640 1 -- 172.21.15.32:0/3686187 shutdown_connections 2023-12-20T04:45:48.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.471+0000 7fa947ee5640 1 -- 172.21.15.32:0/3686187 wait complete. 2023-12-20T04:45:48.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa947ee5640 1 Processor -- start 2023-12-20T04:45:48.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa947ee5640 1 -- start start 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa947ee5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9400b0b30 0x7fa94014ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa947ee5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9400c90e0 0x7fa94015c740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa947ee5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9401599e0 0x7fa940148bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa947ee5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94015cc80 con 0x7fa9400c90e0 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa947ee5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94015b6a0 con 0x7fa9400b0b30 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa947ee5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94015b980 con 0x7fa9401599e0 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa945480640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9400c90e0 0x7fa94015c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa945480640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9400c90e0 0x7fa94015c740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53456/0 (socket says 172.21.15.32:53456) 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa945480640 1 -- 172.21.15.32:0/4208537358 learned_addr learned my addr 172.21.15.32:0/4208537358 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa945c81640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9400b0b30 0x7fa94014ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa946482640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9401599e0 0x7fa940148bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa945480640 1 -- 172.21.15.32:0/4208537358 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9401599e0 msgr2=0x7fa940148bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa945480640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9401599e0 0x7fa940148bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa945480640 1 -- 172.21.15.32:0/4208537358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9400b0b30 msgr2=0x7fa94014ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa945480640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9400b0b30 0x7fa94014ca60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa945480640 1 -- 172.21.15.32:0/4208537358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c00b720 con 0x7fa9400c90e0 2023-12-20T04:45:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa945480640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9400c90e0 0x7fa94015c740 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa93000c7c0 tx=0x7fa93000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:48.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa9367fc640 1 -- 172.21.15.32:0/4208537358 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa930007a30 con 0x7fa9400c90e0 2023-12-20T04:45:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa9367fc640 1 -- 172.21.15.32:0/4208537358 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa930007b90 con 0x7fa9400c90e0 2023-12-20T04:45:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa9367fc640 1 -- 172.21.15.32:0/4208537358 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93000ce70 con 0x7fa9400c90e0 2023-12-20T04:45:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa947ee5640 1 -- 172.21.15.32:0/4208537358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94015bc60 con 0x7fa9400c90e0 2023-12-20T04:45:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.475+0000 7fa947ee5640 1 -- 172.21.15.32:0/4208537358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94015c220 con 0x7fa9400c90e0 2023-12-20T04:45:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.479+0000 7fa947ee5640 1 -- 172.21.15.32:0/4208537358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94015c440 con 0x7fa9400c90e0 2023-12-20T04:45:48.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.479+0000 7fa9367fc640 1 -- 172.21.15.32:0/4208537358 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 57) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fa9300113f0 con 0x7fa9400c90e0 2023-12-20T04:45:48.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.479+0000 7fa9367fc640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa9240658b0 0x7fa924067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.479+0000 7fa9367fc640 1 -- 172.21.15.32:0/4208537358 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6455+0+0 (secure 0 0 0) 0x7fa9300834b0 con 0x7fa9400c90e0 2023-12-20T04:45:48.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.479+0000 7fa945c81640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa9240658b0 0x7fa924067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.479+0000 7fa945c81640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa9240658b0 0x7fa924067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0b1000 tx=0x7fa93c0a9070 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:48.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.483+0000 7fa9367fc640 1 -- 172.21.15.32:0/4208537358 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa930050ea0 con 0x7fa9400c90e0 2023-12-20T04:45:48.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 -- 172.21.15.32:0/4208537358 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa94004e000 con 0x7fa9400c90e0 2023-12-20T04:45:48.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa9367fc640 1 -- 172.21.15.32:0/4208537358 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 46 v46) v1 ==== 94+0+4711 (secure 0 0 0) 0x7fa9300544c0 con 0x7fa9400c90e0 2023-12-20T04:45:48.672 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:48.672 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":46,"default_fscid":6,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1072056415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1072056415},{"type":"v1","addr":"172.21.15.39:6835","nonce":1072056415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3512326822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3512326822},{"type":"v1","addr":"172.21.15.32:6836","nonce":3512326822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2163601073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2163601073},{"type":"v1","addr":"172.21.15.32:6837","nonce":2163601073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42}],"filesystems":[{"mdsmap":{"epoch":46,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:44:27.818499+0000","modified":"2023-12-20T04:44:30.831088+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7359},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7359":{"gid":7359,"name":"d","rank":0,"incarnation":43,"state":"up:active","state_seq":3,"addr":"172.21.15.39:6833/2369052657","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2369052657},{"type":"v1","addr":"172.21.15.39:6833","nonce":2369052657}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[13],"metadata_pool":12,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":6}]} 2023-12-20T04:45:48.672 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 46 2023-12-20T04:45:48.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 -- 172.21.15.32:0/4208537358 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa9240658b0 msgr2=0x7fa924067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa9240658b0 0x7fa924067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fa93c0b1000 tx=0x7fa93c0a9070 comp rx=0 tx=0).stop 2023-12-20T04:45:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 -- 172.21.15.32:0/4208537358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9400c90e0 msgr2=0x7fa94015c740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9400c90e0 0x7fa94015c740 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7fa93000c7c0 tx=0x7fa93000cc80 comp rx=0 tx=0).stop 2023-12-20T04:45:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 -- 172.21.15.32:0/4208537358 shutdown_connections 2023-12-20T04:45:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa9240658b0 0x7fa924067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9401599e0 0x7fa940148bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9400c90e0 0x7fa94015c740 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 --2- 172.21.15.32:0/4208537358 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9400b0b30 0x7fa94014ca60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 -- 172.21.15.32:0/4208537358 >> 172.21.15.32:0/4208537358 conn(0x7fa94006a320 msgr2=0x7fa940071240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 -- 172.21.15.32:0/4208537358 shutdown_connections 2023-12-20T04:45:48.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.675+0000 7fa947ee5640 1 -- 172.21.15.32:0/4208537358 wait complete. 2023-12-20T04:45:48.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:45:48.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f46680f2640 1 Processor -- start 2023-12-20T04:45:48.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f46680f2640 1 -- start start 2023-12-20T04:45:48.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f46680f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46601593c0 0x7f466015d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f46680f2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46600ae140 con 0x7f46600afdb0 2023-12-20T04:45:48.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f46680f2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f466015dee0 con 0x7f466006caf0 2023-12-20T04:45:48.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f46680f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46600ae9e0 con 0x7f46601593c0 2023-12-20T04:45:48.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f466568d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f46600afdb0 0x7f466006c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49186/0 (socket says 172.21.15.32:49186) 2023-12-20T04:45:48.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f466568d640 1 -- 172.21.15.32:0/3114443910 learned_addr learned my addr 172.21.15.32:0/3114443910 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f4665e8e640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46601593c0 0x7f466015d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 877118837 0 0) 0x7f46600ae140 con 0x7f46600afdb0 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f464c003450 con 0x7f46600afdb0 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2614377833 0 0) 0x7f464c003450 con 0x7f46600afdb0 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46601593c0 msgr2=0x7f466015d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f4664e8c640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46601593c0 0x7f466015d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 >> v1:172.21.15.39:6789/0 conn(0x7f466006caf0 legacy=0x7f46600705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4660072170 con 0x7f46600afdb0 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f4650001680 con 0x7f46600afdb0 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f4665e8e640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46601593c0 0x7f466015d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.839+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 >> v1:172.21.15.32:6789/0 conn(0x7f46600afdb0 legacy=0x7f466006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f4664e8c640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f464c003aa0 0x7f464c003e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f4664e8c640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f464c008860 0x7f464c00ac50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f4664e8c640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46601593c0 0x7f464c00d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c00dbc0 con 0x7f464c008860 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f464c00dd30 con 0x7f464c003aa0 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f464c00def0 con 0x7f46601593c0 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f466568d640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f464c008860 0x7f464c00ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f4665e8e640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f464c003aa0 0x7f464c003e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f466668f640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46601593c0 0x7f464c00d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f466568d640 1 -- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46601593c0 msgr2=0x7f464c00d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f466568d640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46601593c0 0x7f464c00d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f466568d640 1 -- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f464c003aa0 msgr2=0x7f464c003e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f466568d640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f464c003aa0 0x7f464c003e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f466568d640 1 -- 172.21.15.32:0/3114443910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f465c0093b0 con 0x7f464c008860 2023-12-20T04:45:48.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f466668f640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46601593c0 0x7f464c00d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f466568d640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f464c008860 0x7f464c00ac50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f465000c210 tx=0x7f465001cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e95ea6d564bf3bbb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46500a5b40 con 0x7f464c008860 2023-12-20T04:45:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46500a5cc0 con 0x7f464c008860 2023-12-20T04:45:48.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.843+0000 7f4664e8c640 1 -- 172.21.15.32:0/3114443910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4650005b20 con 0x7f464c008860 2023-12-20T04:45:48.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 -- 172.21.15.32:0/3114443910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f464c008860 msgr2=0x7f464c00ac50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 --2- 172.21.15.32:0/3114443910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f464c008860 0x7f464c00ac50 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f465000c210 tx=0x7f465001cc40 comp rx=0 tx=0).stop 2023-12-20T04:45:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f466668f640 1 -- 172.21.15.32:0/3114443910 reap_dead start 2023-12-20T04:45:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 -- 172.21.15.32:0/3114443910 shutdown_connections 2023-12-20T04:45:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 -- 172.21.15.32:0/3114443910 >> 172.21.15.32:0/3114443910 conn(0x7f466006a320 msgr2=0x7f46600ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 -- 172.21.15.32:0/3114443910 shutdown_connections 2023-12-20T04:45:48.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 -- 172.21.15.32:0/3114443910 wait complete. 2023-12-20T04:45:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 Processor -- start 2023-12-20T04:45:48.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 -- start start 2023-12-20T04:45:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f464c008860 0x7f466015c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46600c90e0 0x7f46600c9560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4660159c30 0x7f466015a0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46600c9aa0 con 0x7f4660159c30 2023-12-20T04:45:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46601443e0 con 0x7f464c008860 2023-12-20T04:45:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f46680f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f466015c960 con 0x7f46600c90e0 2023-12-20T04:45:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f466668f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4660159c30 0x7f466015a0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f466668f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4660159c30 0x7f466015a0b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42320/0 (socket says 172.21.15.32:42320) 2023-12-20T04:45:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f466668f640 1 -- 172.21.15.32:0/3736254836 learned_addr learned my addr 172.21.15.32:0/3736254836 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:48.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f466568d640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46600c90e0 0x7f46600c9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.847+0000 7f4665e8e640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f464c008860 0x7f466015c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f466668f640 1 -- 172.21.15.32:0/3736254836 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46600c90e0 msgr2=0x7f46600c9560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f466668f640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46600c90e0 0x7f46600c9560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f466668f640 1 -- 172.21.15.32:0/3736254836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f464c008860 msgr2=0x7f466015c420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:48.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f466668f640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f464c008860 0x7f466015c420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:48.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f466668f640 1 -- 172.21.15.32:0/3736254836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46601437d0 con 0x7f4660159c30 2023-12-20T04:45:48.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f466668f640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4660159c30 0x7f466015a0b0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f46600715b0 tx=0x7f4658010f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:48.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f46567fc640 1 -- 172.21.15.32:0/3736254836 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4658003d90 con 0x7f4660159c30 2023-12-20T04:45:48.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f46567fc640 1 -- 172.21.15.32:0/3736254836 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f465800d070 con 0x7f4660159c30 2023-12-20T04:45:48.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f46567fc640 1 -- 172.21.15.32:0/3736254836 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f465800ea30 con 0x7f4660159c30 2023-12-20T04:45:48.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f46680f2640 1 -- 172.21.15.32:0/3736254836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4660143ab0 con 0x7f4660159c30 2023-12-20T04:45:48.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f46680f2640 1 -- 172.21.15.32:0/3736254836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4660144070 con 0x7f4660159c30 2023-12-20T04:45:48.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f46680f2640 1 -- 172.21.15.32:0/3736254836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4660144200 con 0x7f4660159c30 2023-12-20T04:45:48.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f46567fc640 1 -- 172.21.15.32:0/3736254836 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 57) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f465800eb90 con 0x7f4660159c30 2023-12-20T04:45:48.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f46567fc640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46440658b0 0x7f4644067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:48.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f46567fc640 1 -- 172.21.15.32:0/3736254836 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6455+0+0 (secure 0 0 0) 0x7f46580861a0 con 0x7f4660159c30 2023-12-20T04:45:48.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.851+0000 7f4665e8e640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46440658b0 0x7f4644067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:48.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.855+0000 7f4665e8e640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46440658b0 0x7f4644067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f465c009c90 tx=0x7f465c008040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:48.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:48.855+0000 7f46567fc640 1 -- 172.21.15.32:0/3736254836 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4658053bb0 con 0x7f4660159c30 2023-12-20T04:45:49.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.015+0000 7f46680f2640 1 -- 172.21.15.32:0/3736254836 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f466004e000 con 0x7f4660159c30 2023-12-20T04:45:49.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.015+0000 7f46567fc640 1 -- 172.21.15.32:0/3736254836 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v111) v1 ==== 74+0+17134 (secure 0 0 0) 0x7f46580571d0 con 0x7f4660159c30 2023-12-20T04:45:49.014 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:49.014 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":111,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:45:47.191931+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":12,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:44:25.451651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"106","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":13,"pool_name":"cephfs_data","create_time":"2023-12-20T04:44:26.819848+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"13.4","mappings":[{"from":6,"to":3}]},{"pgid":"13.16","mappings":[{"from":1,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/842509018":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/2758895268":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/2468150930":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/46584708":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/950991327":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/2883211641":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/3801303150":"2023-12-21T04:45:47.191868+0000","172.21.15.32:0/1531123683":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3296285907":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3289362372":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/998392483":"2023-12-21T04:44:39.978724+0000","172.21.15.32:6833/38009":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/1838475563":"2023-12-21T04:45:47.191868+0000","172.21.15.32:6832/38009":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/3368879992":"2023-12-21T04:44:39.978724+0000","172.21.15.32:0/2997487195":"2023-12-21T04:44:39.978724+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:45:49.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 -- 172.21.15.32:0/3736254836 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46440658b0 msgr2=0x7f4644067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46440658b0 0x7f4644067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f465c009c90 tx=0x7f465c008040 comp rx=0 tx=0).stop 2023-12-20T04:45:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 -- 172.21.15.32:0/3736254836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4660159c30 msgr2=0x7f466015a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4660159c30 0x7f466015a0b0 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f46600715b0 tx=0x7f4658010f40 comp rx=0 tx=0).stop 2023-12-20T04:45:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 -- 172.21.15.32:0/3736254836 shutdown_connections 2023-12-20T04:45:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f46440658b0 0x7f4644067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4660159c30 0x7f466015a0b0 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46600c90e0 0x7f46600c9560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 --2- 172.21.15.32:0/3736254836 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f464c008860 0x7f466015c420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 -- 172.21.15.32:0/3736254836 >> 172.21.15.32:0/3736254836 conn(0x7f466006a320 msgr2=0x7f46600715e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 -- 172.21.15.32:0/3736254836 shutdown_connections 2023-12-20T04:45:49.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.019+0000 7f46680f2640 1 -- 172.21.15.32:0/3736254836 wait complete. 2023-12-20T04:45:49.035 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T04:45:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.183+0000 7fd6f1939640 1 Processor -- start 2023-12-20T04:45:49.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.183+0000 7fd6f1939640 1 -- start start 2023-12-20T04:45:49.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.183+0000 7fd6f1939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6ec0af5e0 0x7fd6ec0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.183+0000 7fd6f1939640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec15b990 con 0x7fd6ec15f110 2023-12-20T04:45:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.183+0000 7fd6f1939640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6ec15bbd0 con 0x7fd6ec06ce60 2023-12-20T04:45:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.183+0000 7fd6f1939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec15beb0 con 0x7fd6ec0af5e0 2023-12-20T04:45:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6eb7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd6ec15f110 0x7fd6ec15c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49194/0 (socket says 172.21.15.32:49194) 2023-12-20T04:45:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6eb7fe640 1 -- 172.21.15.32:0/3037351508 learned_addr learned my addr 172.21.15.32:0/3037351508 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6eaffd640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6ec0af5e0 0x7fd6ec0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 899316339 0 0) 0x7fd6e40014a0 con 0x7fd6ec15f110 2023-12-20T04:45:49.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6d8003530 con 0x7fd6ec15f110 2023-12-20T04:45:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4071644666 0 0) 0x7fd6ec15bbd0 con 0x7fd6ec06ce60 2023-12-20T04:45:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6e40014a0 con 0x7fd6ec06ce60 2023-12-20T04:45:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2335189886 0 0) 0x7fd6d8003530 con 0x7fd6ec15f110 2023-12-20T04:45:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6ec0af5e0 msgr2=0x7fd6ec0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6ec0af5e0 0x7fd6ec0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 >> v1:172.21.15.39:6789/0 conn(0x7fd6ec06ce60 legacy=0x7fd6ec15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec15e910 con 0x7fd6ec15f110 2023-12-20T04:45:49.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fd6e4001b90 con 0x7fd6ec15f110 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 >> v1:172.21.15.32:6789/0 conn(0x7fd6ec15f110 legacy=0x7fd6ec15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6d8003b40 0x7fd6d8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6d80047d0 0x7fd6d8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6d8005190 0x7fd6d800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec15beb0 con 0x7fd6d80047d0 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d800bf80 con 0x7fd6d8003b40 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6d800c140 con 0x7fd6d8005190 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6eaffd640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6ec0af5e0 0x7fd6ec0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6e3fff640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6d80047d0 0x7fd6d8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6eb7fe640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6d8005190 0x7fd6d800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6eaffd640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6d8003b40 0x7fd6d8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6e3fff640 1 -- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6d8005190 msgr2=0x7fd6d800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:49.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6e3fff640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6d8005190 0x7fd6d800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6e3fff640 1 -- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6d8003b40 msgr2=0x7fd6d8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:49.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6e3fff640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6d8003b40 0x7fd6d8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.187+0000 7fd6e3fff640 1 -- 172.21.15.32:0/3037351508 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6d8003530 con 0x7fd6d80047d0 2023-12-20T04:45:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6e3fff640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6d80047d0 0x7fd6d8004c50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec06d2a0 tx=0x7fd6d401ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5a3997e98e1cae server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:49.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40052d0 con 0x7fd6d80047d0 2023-12-20T04:45:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d4005450 con 0x7fd6d80047d0 2023-12-20T04:45:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6ea7fc640 1 -- 172.21.15.32:0/3037351508 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40a8910 con 0x7fd6d80047d0 2023-12-20T04:45:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6f1939640 1 -- 172.21.15.32:0/3037351508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6d80047d0 msgr2=0x7fd6d8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6f1939640 1 --2- 172.21.15.32:0/3037351508 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6d80047d0 0x7fd6d8004c50 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec06d2a0 tx=0x7fd6d401ecb0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6eb7fe640 1 -- 172.21.15.32:0/3037351508 reap_dead start 2023-12-20T04:45:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6f1939640 1 -- 172.21.15.32:0/3037351508 shutdown_connections 2023-12-20T04:45:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6f1939640 1 -- 172.21.15.32:0/3037351508 >> 172.21.15.32:0/3037351508 conn(0x7fd6ec06a320 msgr2=0x7fd6ec0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6f1939640 1 -- 172.21.15.32:0/3037351508 shutdown_connections 2023-12-20T04:45:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6f1939640 1 -- 172.21.15.32:0/3037351508 wait complete. 2023-12-20T04:45:49.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6f1939640 1 Processor -- start 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.191+0000 7fd6f1939640 1 -- start start 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6f1939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6d80047d0 0x7fd6ec1eeca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6f1939640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ec0b1aa0 0x7fd6ec152130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6f1939640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6ec1590b0 0x7fd6ec150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6f1939640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec152670 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6f1939640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec15c4f0 con 0x7fd6ec1590b0 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6f1939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6ec151eb0 con 0x7fd6d80047d0 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ec0b1aa0 0x7fd6ec152130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ec0b1aa0 0x7fd6ec152130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42336/0 (socket says 172.21.15.32:42336) 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e3fff640 1 -- 172.21.15.32:0/570923188 learned_addr learned my addr 172.21.15.32:0/570923188 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6eb7fe640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6ec1590b0 0x7fd6ec150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6eaffd640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6d80047d0 0x7fd6ec1eeca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e3fff640 1 -- 172.21.15.32:0/570923188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6d80047d0 msgr2=0x7fd6ec1eeca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e3fff640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6d80047d0 0x7fd6ec1eeca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e3fff640 1 -- 172.21.15.32:0/570923188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6ec1590b0 msgr2=0x7fd6ec150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e3fff640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6ec1590b0 0x7fd6ec150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e3fff640 1 -- 172.21.15.32:0/570923188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec0b1f20 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e3fff640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ec0b1aa0 0x7fd6ec152130 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4005040 tx=0x7fd6d8005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:49.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e8ff9640 1 -- 172.21.15.32:0/570923188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d40a8dd0 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e8ff9640 1 -- 172.21.15.32:0/570923188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6d40050d0 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e8ff9640 1 -- 172.21.15.32:0/570923188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6d4003600 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6f1939640 1 -- 172.21.15.32:0/570923188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6ec151930 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6f1939640 1 -- 172.21.15.32:0/570923188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6ec151bd0 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6f1939640 1 -- 172.21.15.32:0/570923188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6ec0b0b30 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.195+0000 7fd6e8ff9640 1 -- 172.21.15.32:0/570923188 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 57) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fd6d4003760 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.199+0000 7fd6e8ff9640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd6cc0658a0 0x7fd6cc067d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.199+0000 7fd6e8ff9640 1 -- 172.21.15.32:0/570923188 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(111..111 src has 1..111) v4 ==== 6455+0+0 (secure 0 0 0) 0x7fd6d411ff10 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.199+0000 7fd6eaffd640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd6cc0658a0 0x7fd6cc067d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.199+0000 7fd6eaffd640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd6cc0658a0 0x7fd6cc067d60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec06ac10 tx=0x7fd6dc008040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:49.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.203+0000 7fd6e8ff9640 1 -- 172.21.15.32:0/570923188 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd6d40ed850 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.215+0000 7fd6e8ff9640 1 -- 172.21.15.32:0/570923188 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 58) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fd6d40edc20 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.403+0000 7fd6f1939640 1 -- 172.21.15.32:0/570923188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd6ec0b0b30 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.401 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T04:45:49.403+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T04:45:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.415+0000 7fd6e8ff9640 1 -- 172.21.15.32:0/570923188 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v47) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd6d40f50d0 con 0x7fd6ec0b1aa0 2023-12-20T04:45:49.414 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T04:45:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 -- 172.21.15.32:0/570923188 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd6cc0658a0 msgr2=0x7fd6cc067d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd6cc0658a0 0x7fd6cc067d60 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec06ac10 tx=0x7fd6dc008040 comp rx=0 tx=0).stop 2023-12-20T04:45:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 -- 172.21.15.32:0/570923188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ec0b1aa0 msgr2=0x7fd6ec152130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ec0b1aa0 0x7fd6ec152130 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fd6d4005040 tx=0x7fd6d8005800 comp rx=0 tx=0).stop 2023-12-20T04:45:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 -- 172.21.15.32:0/570923188 shutdown_connections 2023-12-20T04:45:49.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd6cc0658a0 0x7fd6cc067d60 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6ec1590b0 0x7fd6ec150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6ec0b1aa0 0x7fd6ec152130 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 --2- 172.21.15.32:0/570923188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6d80047d0 0x7fd6ec1eeca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 -- 172.21.15.32:0/570923188 >> 172.21.15.32:0/570923188 conn(0x7fd6ec06a320 msgr2=0x7fd6ec157c90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 -- 172.21.15.32:0/570923188 shutdown_connections 2023-12-20T04:45:49.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.419+0000 7fd6f1939640 1 -- 172.21.15.32:0/570923188 wait complete. 2023-12-20T04:45:49.427 INFO:tasks.ceph.mds.d.smithi039.stderr: -7336> 2023-12-20T04:44:28.395+0000 7ff9be2df640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:45:49.435 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T04:45:49.478 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T04:45:49.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.583+0000 7f4f3e08c640 1 Processor -- start 2023-12-20T04:45:49.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.583+0000 7f4f3e08c640 1 -- start start 2023-12-20T04:45:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f3e08c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f38157480 0x7f4f380c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f3e08c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f380cde20 con 0x7f4f380b0760 2023-12-20T04:45:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f3e08c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4f380c8e30 con 0x7f4f380cd4c0 2023-12-20T04:45:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f3e08c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f3815e480 con 0x7f4f38157480 2023-12-20T04:45:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f36ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4f380b0760 0x7f4f3815d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49210/0 (socket says 172.21.15.32:49210) 2023-12-20T04:45:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f36ffd640 1 -- 172.21.15.32:0/4261288350 learned_addr learned my addr 172.21.15.32:0/4261288350 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f377fe640 1 --2- 172.21.15.32:0/4261288350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f38157480 0x7f4f380c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f367fc640 1 -- 172.21.15.32:0/4261288350 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1739615185 0 0) 0x7f4f380cde20 con 0x7f4f380b0760 2023-12-20T04:45:49.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f367fc640 1 -- 172.21.15.32:0/4261288350 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f20003530 con 0x7f4f380b0760 2023-12-20T04:45:49.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f367fc640 1 -- 172.21.15.32:0/4261288350 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1791424192 0 0) 0x7f4f380c8e30 con 0x7f4f380cd4c0 2023-12-20T04:45:49.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f367fc640 1 -- 172.21.15.32:0/4261288350 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4f380cde20 con 0x7f4f380cd4c0 2023-12-20T04:45:49.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f367fc640 1 -- 172.21.15.32:0/4261288350 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4241773188 0 0) 0x7f4f20003530 con 0x7f4f380b0760 2023-12-20T04:45:49.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f377fe640 1 -- 172.21.15.32:0/4261288350 >> v1:172.21.15.39:6789/0 conn(0x7f4f380cd4c0 legacy=0x7f4f3815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f377fe640 1 -- 172.21.15.32:0/4261288350 >> v1:172.21.15.32:6789/0 conn(0x7f4f380b0760 legacy=0x7f4f3815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.587+0000 7f4f377fe640 1 -- 172.21.15.32:0/4261288350 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f380ce070 con 0x7f4f38157480 2023-12-20T04:45:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f377fe640 1 --2- 172.21.15.32:0/4261288350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f38157480 0x7f4f380c9840 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4f280097f0 tx=0x7f4f2801c320 comp rx=0 tx=0).ready entity=mon.2 client_cookie=87bbb24d48814807 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f367fc640 1 -- 172.21.15.32:0/4261288350 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f2c001680 con 0x7f4f38157480 2023-12-20T04:45:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f367fc640 1 -- 172.21.15.32:0/4261288350 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f280a7050 con 0x7f4f38157480 2023-12-20T04:45:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f367fc640 1 -- 172.21.15.32:0/4261288350 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f280ab620 con 0x7f4f38157480 2023-12-20T04:45:49.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f3e08c640 1 -- 172.21.15.32:0/4261288350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f38157480 msgr2=0x7f4f380c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f3e08c640 1 --2- 172.21.15.32:0/4261288350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f38157480 0x7f4f380c9840 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f4f280097f0 tx=0x7f4f2801c320 comp rx=0 tx=0).stop 2023-12-20T04:45:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f3e08c640 1 -- 172.21.15.32:0/4261288350 shutdown_connections 2023-12-20T04:45:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f3e08c640 1 --2- 172.21.15.32:0/4261288350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f38157480 0x7f4f380c9840 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f3e08c640 1 -- 172.21.15.32:0/4261288350 >> 172.21.15.32:0/4261288350 conn(0x7f4f3806a510 msgr2=0x7f4f380ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f3e08c640 1 -- 172.21.15.32:0/4261288350 shutdown_connections 2023-12-20T04:45:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f3e08c640 1 -- 172.21.15.32:0/4261288350 wait complete. 2023-12-20T04:45:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f3e08c640 1 Processor -- start 2023-12-20T04:45:49.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.591+0000 7f4f3e08c640 1 -- start start 2023-12-20T04:45:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f3e08c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f380b0760 0x7f4f38142520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f3e08c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f380cd4c0 0x7f4f380c48d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f3e08c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f38157480 0x7f4f380ce9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f3e08c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380c78c0 con 0x7f4f38157480 2023-12-20T04:45:49.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f3e08c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380c6130 con 0x7f4f380b0760 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f3e08c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4f380c7590 con 0x7f4f380cd4c0 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f38157480 0x7f4f380ce9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f37fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f38157480 0x7f4f380ce9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42382/0 (socket says 172.21.15.32:42382) 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f37fff640 1 -- 172.21.15.32:0/860873698 learned_addr learned my addr 172.21.15.32:0/860873698 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f377fe640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f380b0760 0x7f4f38142520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f36ffd640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f380cd4c0 0x7f4f380c48d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f37fff640 1 -- 172.21.15.32:0/860873698 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f380cd4c0 msgr2=0x7f4f380c48d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f37fff640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f380cd4c0 0x7f4f380c48d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f37fff640 1 -- 172.21.15.32:0/860873698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f380b0760 msgr2=0x7f4f38142520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f37fff640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f380b0760 0x7f4f38142520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f37fff640 1 -- 172.21.15.32:0/860873698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4f28009420 con 0x7f4f38157480 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f37fff640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f38157480 0x7f4f380ce9d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c760 tx=0x7f4f3000cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:49.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f1ffff640 1 -- 172.21.15.32:0/860873698 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f30003b80 con 0x7f4f38157480 2023-12-20T04:45:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f1ffff640 1 -- 172.21.15.32:0/860873698 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4f30003ce0 con 0x7f4f38157480 2023-12-20T04:45:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f3e08c640 1 -- 172.21.15.32:0/860873698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4f380c4e10 con 0x7f4f38157480 2023-12-20T04:45:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f1ffff640 1 -- 172.21.15.32:0/860873698 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4f3000aa90 con 0x7f4f38157480 2023-12-20T04:45:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f3e08c640 1 -- 172.21.15.32:0/860873698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4f380c5b70 con 0x7f4f38157480 2023-12-20T04:45:49.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f3e08c640 1 -- 172.21.15.32:0/860873698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4f380c7a70 con 0x7f4f38157480 2023-12-20T04:45:49.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.595+0000 7f4f1ffff640 1 -- 172.21.15.32:0/860873698 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 58) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f4f3000abf0 con 0x7f4f38157480 2023-12-20T04:45:49.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.599+0000 7f4f1ffff640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f24065af0 0x7f4f24067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:49.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.599+0000 7f4f1ffff640 1 -- 172.21.15.32:0/860873698 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6541+0+0 (secure 0 0 0) 0x7f4f300832b0 con 0x7f4f38157480 2023-12-20T04:45:49.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.599+0000 7f4f377fe640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f24065af0 0x7f4f24067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:49.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.599+0000 7f4f377fe640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f24065af0 0x7f4f24067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f280ad000 tx=0x7f4f280a6000 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:49.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.599+0000 7f4f1ffff640 1 -- 172.21.15.32:0/860873698 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4f380c7a70 con 0x7f4f38157480 2023-12-20T04:45:49.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:49.791+0000 7f4f3e08c640 1 -- 172.21.15.32:0/860873698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f4f380c7a70 con 0x7f4f38157480 2023-12-20T04:45:50.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.419+0000 7f4f1ffff640 1 -- 172.21.15.32:0/860873698 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v48) v1 ==== 104+0+0 (secure 0 0 0) 0x7f4f380c7a70 con 0x7f4f38157480 2023-12-20T04:45:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.419+0000 7f4f3e08c640 1 -- 172.21.15.32:0/860873698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f24065af0 msgr2=0x7f4f24067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.419+0000 7f4f3e08c640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f24065af0 0x7f4f24067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f4f280ad000 tx=0x7f4f280a6000 comp rx=0 tx=0).stop 2023-12-20T04:45:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.419+0000 7f4f3e08c640 1 -- 172.21.15.32:0/860873698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f38157480 msgr2=0x7f4f380ce9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.419+0000 7f4f3e08c640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f38157480 0x7f4f380ce9d0 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7f4f3000c760 tx=0x7f4f3000cc20 comp rx=0 tx=0).stop 2023-12-20T04:45:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.419+0000 7f4f3e08c640 1 -- 172.21.15.32:0/860873698 shutdown_connections 2023-12-20T04:45:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.419+0000 7f4f3e08c640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4f24065af0 0x7f4f24067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.419+0000 7f4f3e08c640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4f38157480 0x7f4f380ce9d0 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.419+0000 7f4f3e08c640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4f380cd4c0 0x7f4f380c48d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.419+0000 7f4f3e08c640 1 --2- 172.21.15.32:0/860873698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4f380b0760 0x7f4f38142520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.419+0000 7f4f3e08c640 1 -- 172.21.15.32:0/860873698 >> 172.21.15.32:0/860873698 conn(0x7f4f3806a510 msgr2=0x7f4f3806d720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:50.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.423+0000 7f4f3e08c640 1 -- 172.21.15.32:0/860873698 shutdown_connections 2023-12-20T04:45:50.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.423+0000 7f4f3e08c640 1 -- 172.21.15.32:0/860873698 wait complete. 2023-12-20T04:45:50.439 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T04:45:50.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83fad8e640 1 Processor -- start 2023-12-20T04:45:50.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83fad8e640 1 -- start start 2023-12-20T04:45:50.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83fad8e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83f40b8410 0x7f83f415c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:50.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83fad8e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f40d0c10 con 0x7f83f40cd940 2023-12-20T04:45:50.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83fad8e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f4160f80 con 0x7f83f40cdd20 2023-12-20T04:45:50.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83fad8e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f4161240 con 0x7f83f40b8410 2023-12-20T04:45:50.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83f958b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83f40b8410 0x7f83f415c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:50.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83f958b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83f40b8410 0x7f83f415c5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42396/0 (socket says 172.21.15.32:42396) 2023-12-20T04:45:50.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83f958b640 1 -- 172.21.15.32:0/2326800449 learned_addr learned my addr 172.21.15.32:0/2326800449 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:50.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83f958b640 1 -- 172.21.15.32:0/2326800449 >> v1:172.21.15.39:6789/0 conn(0x7f83f40cdd20 legacy=0x7f83f40d13a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:50.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83f958b640 1 -- 172.21.15.32:0/2326800449 >> v1:172.21.15.32:6789/0 conn(0x7f83f40cd940 legacy=0x7f83f41606e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:50.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.591+0000 7f83f958b640 1 -- 172.21.15.32:0/2326800449 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f40d0e80 con 0x7f83f40b8410 2023-12-20T04:45:50.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83f958b640 1 --2- 172.21.15.32:0/2326800449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f40b8410 0x7f83f415c5d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009870 tx=0x7f83e401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f55d4584a78cb39e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:50.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83f8d8a640 1 -- 172.21.15.32:0/2326800449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40a9620 con 0x7f83f40b8410 2023-12-20T04:45:50.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83f8d8a640 1 -- 172.21.15.32:0/2326800449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e40a9780 con 0x7f83f40b8410 2023-12-20T04:45:50.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83f8d8a640 1 -- 172.21.15.32:0/2326800449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40a9b10 con 0x7f83f40b8410 2023-12-20T04:45:50.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83fad8e640 1 -- 172.21.15.32:0/2326800449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f40b8410 msgr2=0x7f83f415c5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:50.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83fad8e640 1 --2- 172.21.15.32:0/2326800449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f40b8410 0x7f83f415c5d0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f83e4009870 tx=0x7f83e401c540 comp rx=0 tx=0).stop 2023-12-20T04:45:50.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83fad8e640 1 -- 172.21.15.32:0/2326800449 shutdown_connections 2023-12-20T04:45:50.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83fad8e640 1 --2- 172.21.15.32:0/2326800449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f40b8410 0x7f83f415c5d0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:50.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83fad8e640 1 -- 172.21.15.32:0/2326800449 >> 172.21.15.32:0/2326800449 conn(0x7f83f40658e0 msgr2=0x7f83f4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:50.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83fad8e640 1 -- 172.21.15.32:0/2326800449 shutdown_connections 2023-12-20T04:45:50.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83fad8e640 1 -- 172.21.15.32:0/2326800449 wait complete. 2023-12-20T04:45:50.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83fad8e640 1 Processor -- start 2023-12-20T04:45:50.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.595+0000 7f83fad8e640 1 -- start start 2023-12-20T04:45:50.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83fad8e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40b8410 0x7f83f40c53b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:50.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83fad8e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f40cd940 0x7f83f4157eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:50.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83fad8e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f40cdd20 0x7f83f41574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:50.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83fad8e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f4158890 con 0x7f83f40cd940 2023-12-20T04:45:50.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83fad8e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f4157a20 con 0x7f83f40b8410 2023-12-20T04:45:50.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83fad8e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f415e950 con 0x7f83f40cdd20 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83f958b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f40cd940 0x7f83f4157eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83f958b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f40cd940 0x7f83f4157eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42408/0 (socket says 172.21.15.32:42408) 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83f958b640 1 -- 172.21.15.32:0/798259337 learned_addr learned my addr 172.21.15.32:0/798259337 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83f9d8c640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40b8410 0x7f83f40c53b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83fa58d640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f40cdd20 0x7f83f41574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83f958b640 1 -- 172.21.15.32:0/798259337 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f40cdd20 msgr2=0x7f83f41574e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83f958b640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f40cdd20 0x7f83f41574e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83f958b640 1 -- 172.21.15.32:0/798259337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40b8410 msgr2=0x7f83f40c53b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83f958b640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40b8410 0x7f83f40c53b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83f958b640 1 -- 172.21.15.32:0/798259337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e40094d0 con 0x7f83f40cd940 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83f9d8c640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40b8410 0x7f83f40c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83f958b640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f40cd940 0x7f83f4157eb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f83e40ab040 tx=0x7f83e401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:50.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83ea7fc640 1 -- 172.21.15.32:0/798259337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40bc450 con 0x7f83f40cd940 2023-12-20T04:45:50.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83ea7fc640 1 -- 172.21.15.32:0/798259337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e40bc5d0 con 0x7f83f40cd940 2023-12-20T04:45:50.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83fad8e640 1 -- 172.21.15.32:0/798259337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f41585b0 con 0x7f83f40cd940 2023-12-20T04:45:50.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83ea7fc640 1 -- 172.21.15.32:0/798259337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40bc980 con 0x7f83f40cd940 2023-12-20T04:45:50.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83fad8e640 1 -- 172.21.15.32:0/798259337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f415eeb0 con 0x7f83f40cd940 2023-12-20T04:45:50.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.599+0000 7f83fad8e640 1 -- 172.21.15.32:0/798259337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f4074100 con 0x7f83f40cd940 2023-12-20T04:45:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.603+0000 7f83ea7fc640 1 -- 172.21.15.32:0/798259337 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 58) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f83e40bcae0 con 0x7f83f40cd940 2023-12-20T04:45:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.603+0000 7f83ea7fc640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f83d8065af0 0x7f83d8067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.603+0000 7f83ea7fc640 1 -- 172.21.15.32:0/798259337 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(112..112 src has 1..112) v4 ==== 6541+0+0 (secure 0 0 0) 0x7f83e411def0 con 0x7f83f40cd940 2023-12-20T04:45:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.603+0000 7f83f9d8c640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f83d8065af0 0x7f83d8067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:50.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.603+0000 7f83f9d8c640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f83d8065af0 0x7f83d8067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f83f0000f70 tx=0x7f83f0009040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:50.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.607+0000 7f83ea7fc640 1 -- 172.21.15.32:0/798259337 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f83e40eb2e0 con 0x7f83f40cd940 2023-12-20T04:45:50.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:50.763+0000 7f83fad8e640 1 -- 172.21.15.32:0/798259337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f83f40b7410 con 0x7f83f40cd940 2023-12-20T04:45:51.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.439+0000 7f83ea7fc640 1 -- 172.21.15.32:0/798259337 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f83e40eb6b0 con 0x7f83f40cd940 2023-12-20T04:45:51.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.439+0000 7f83ea7fc640 1 -- 172.21.15.32:0/798259337 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v113) v1 ==== 181+0+0 (secure 0 0 0) 0x7f83e40bcf60 con 0x7f83f40cd940 2023-12-20T04:45:51.438 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T04:45:51.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 -- 172.21.15.32:0/798259337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f83d8065af0 msgr2=0x7f83d8067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f83d8065af0 0x7f83d8067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f83f0000f70 tx=0x7f83f0009040 comp rx=0 tx=0).stop 2023-12-20T04:45:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 -- 172.21.15.32:0/798259337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f40cd940 msgr2=0x7f83f4157eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f40cd940 0x7f83f4157eb0 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f83e40ab040 tx=0x7f83e401ad80 comp rx=0 tx=0).stop 2023-12-20T04:45:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 -- 172.21.15.32:0/798259337 shutdown_connections 2023-12-20T04:45:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f83d8065af0 0x7f83d8067fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f40cdd20 0x7f83f41574e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f40cd940 0x7f83f4157eb0 secure :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f83e40ab040 tx=0x7f83e401ad80 comp rx=0 tx=0).stop 2023-12-20T04:45:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 --2- 172.21.15.32:0/798259337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40b8410 0x7f83f40c53b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 -- 172.21.15.32:0/798259337 >> 172.21.15.32:0/798259337 conn(0x7f83f40658e0 msgr2=0x7f83f40b6970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 -- 172.21.15.32:0/798259337 shutdown_connections 2023-12-20T04:45:51.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.443+0000 7f83fad8e640 1 -- 172.21.15.32:0/798259337 wait complete. 2023-12-20T04:45:51.465 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T04:45:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe3e12640 1 Processor -- start 2023-12-20T04:45:51.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe3e12640 1 -- start start 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe3e12640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fdc070940 0x7f9fdc0cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe3e12640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9fdc0cdef0 0x7f9fdc06c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe3e12640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9fdc161670 con 0x7f9fdc15b790 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe3e12640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc0ce6a0 con 0x7f9fdc070940 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe3e12640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9fdc162270 con 0x7f9fdc0cdef0 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe13ad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fdc070940 0x7f9fdc0cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe1bae640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9fdc15b790 0x7f9fdc070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49234/0 (socket says 172.21.15.32:49234) 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe13ad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9fdc070940 0x7f9fdc0cd9b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42416/0 (socket says 172.21.15.32:42416) 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe1bae640 1 -- 172.21.15.32:0/1732319925 learned_addr learned my addr 172.21.15.32:0/1732319925 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe23af640 1 --2- 172.21.15.32:0/1732319925 >> v2:172.21.15.39:3300/0 conn(0x7f9fdc0cdef0 0x7f9fdc06c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe13ad640 1 -- 172.21.15.32:0/1732319925 >> v2:172.21.15.39:3300/0 conn(0x7f9fdc0cdef0 msgr2=0x7f9fdc06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe13ad640 1 --2- 172.21.15.32:0/1732319925 >> v2:172.21.15.39:3300/0 conn(0x7f9fdc0cdef0 0x7f9fdc06c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:51.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe13ad640 1 -- 172.21.15.32:0/1732319925 >> v1:172.21.15.32:6789/0 conn(0x7f9fdc15b790 legacy=0x7f9fdc070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe13ad640 1 -- 172.21.15.32:0/1732319925 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fdc162690 con 0x7f9fdc070940 2023-12-20T04:45:51.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.619+0000 7f9fe23af640 1 --2- 172.21.15.32:0/1732319925 >> v2:172.21.15.39:3300/0 conn(0x7f9fdc0cdef0 0x7f9fdc06c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe13ad640 1 --2- 172.21.15.32:0/1732319925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc070940 0x7f9fdc0cd9b0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0b49b0 tx=0x7f9fcc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c95be0f938b1d202 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe0bac640 1 -- 172.21.15.32:0/1732319925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0ab660 con 0x7f9fdc070940 2023-12-20T04:45:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe0bac640 1 -- 172.21.15.32:0/1732319925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fcc0ab7c0 con 0x7f9fdc070940 2023-12-20T04:45:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe0bac640 1 -- 172.21.15.32:0/1732319925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fcc0abb50 con 0x7f9fdc070940 2023-12-20T04:45:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1732319925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc070940 msgr2=0x7f9fdc0cd9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe3e12640 1 --2- 172.21.15.32:0/1732319925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc070940 0x7f9fdc0cd9b0 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f9fdc0b49b0 tx=0x7f9fcc01c460 comp rx=0 tx=0).stop 2023-12-20T04:45:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1732319925 shutdown_connections 2023-12-20T04:45:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe3e12640 1 --2- 172.21.15.32:0/1732319925 >> v2:172.21.15.39:3300/0 conn(0x7f9fdc0cdef0 0x7f9fdc06c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe3e12640 1 --2- 172.21.15.32:0/1732319925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc070940 0x7f9fdc0cd9b0 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1732319925 >> 172.21.15.32:0/1732319925 conn(0x7f9fdc06a510 msgr2=0x7f9fdc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1732319925 shutdown_connections 2023-12-20T04:45:51.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.623+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1732319925 wait complete. 2023-12-20T04:45:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe3e12640 1 Processor -- start 2023-12-20T04:45:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe3e12640 1 -- start start 2023-12-20T04:45:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe3e12640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc070940 0x7f9fdc0c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe3e12640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc0cdef0 0x7f9fdc0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe3e12640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fdc15b790 0x7f9fdc162e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe3e12640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0c6220 con 0x7f9fdc070940 2023-12-20T04:45:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe3e12640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0c6500 con 0x7f9fdc15b790 2023-12-20T04:45:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe3e12640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9fdc0c67e0 con 0x7f9fdc0cdef0 2023-12-20T04:45:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe1bae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc070940 0x7f9fdc0c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe1bae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc070940 0x7f9fdc0c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42420/0 (socket says 172.21.15.32:42420) 2023-12-20T04:45:51.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe1bae640 1 -- 172.21.15.32:0/1065208340 learned_addr learned my addr 172.21.15.32:0/1065208340 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe23af640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fdc15b790 0x7f9fdc162e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe13ad640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc0cdef0 0x7f9fdc0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe1bae640 1 -- 172.21.15.32:0/1065208340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc0cdef0 msgr2=0x7f9fdc0c72c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe1bae640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc0cdef0 0x7f9fdc0c72c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe1bae640 1 -- 172.21.15.32:0/1065208340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fdc15b790 msgr2=0x7f9fdc162e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe1bae640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fdc15b790 0x7f9fdc162e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:51.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe1bae640 1 -- 172.21.15.32:0/1065208340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9fcc009420 con 0x7f9fdc070940 2023-12-20T04:45:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe1bae640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc070940 0x7f9fdc0c58b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8007fd0 tx=0x7f9fd800c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fd27fc640 1 -- 172.21.15.32:0/1065208340 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd8005cf0 con 0x7f9fdc070940 2023-12-20T04:45:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fd27fc640 1 -- 172.21.15.32:0/1065208340 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9fd8005e70 con 0x7f9fdc070940 2023-12-20T04:45:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fd27fc640 1 -- 172.21.15.32:0/1065208340 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9fd80072f0 con 0x7f9fdc070940 2023-12-20T04:45:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1065208340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9fdc0c6ac0 con 0x7f9fdc070940 2023-12-20T04:45:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1065208340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9fdc0c5df0 con 0x7f9fdc070940 2023-12-20T04:45:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.627+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1065208340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9fdc04e000 con 0x7f9fdc070940 2023-12-20T04:45:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.631+0000 7f9fd27fc640 1 -- 172.21.15.32:0/1065208340 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9fd8007450 con 0x7f9fdc070940 2023-12-20T04:45:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.631+0000 7f9fd27fc640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc0065b80 0x7f9fc0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.631+0000 7f9fd27fc640 1 -- 172.21.15.32:0/1065208340 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(113..113 src has 1..113) v4 ==== 6103+0+0 (secure 0 0 0) 0x7f9fd8082a00 con 0x7f9fdc070940 2023-12-20T04:45:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.631+0000 7f9fe13ad640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc0065b80 0x7f9fc0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.631+0000 7f9fe13ad640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc0065b80 0x7f9fc0068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0af000 tx=0x7f9fcc0a7000 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.635+0000 7f9fd27fc640 1 -- 172.21.15.32:0/1065208340 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9fd8050130 con 0x7f9fdc070940 2023-12-20T04:45:51.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:51.795+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1065208340 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f9fdc15f070 con 0x7f9fdc070940 2023-12-20T04:45:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.435+0000 7f9fd27fc640 1 -- 172.21.15.32:0/1065208340 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v114) v1 ==== 169+0+0 (secure 0 0 0) 0x7f9fd80170d0 con 0x7f9fdc070940 2023-12-20T04:45:52.435 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T04:45:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.439+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1065208340 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc0065b80 msgr2=0x7f9fc0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.439+0000 7f9fe3e12640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc0065b80 0x7f9fc0068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f9fcc0af000 tx=0x7f9fcc0a7000 comp rx=0 tx=0).stop 2023-12-20T04:45:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.439+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1065208340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc070940 msgr2=0x7f9fdc0c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.439+0000 7f9fe3e12640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc070940 0x7f9fdc0c58b0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f9fd8007fd0 tx=0x7f9fd800c890 comp rx=0 tx=0).stop 2023-12-20T04:45:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.443+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1065208340 shutdown_connections 2023-12-20T04:45:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.443+0000 7f9fe3e12640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9fc0065b80 0x7f9fc0068040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.443+0000 7f9fe3e12640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9fdc15b790 0x7f9fdc162e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.443+0000 7f9fe3e12640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9fdc0cdef0 0x7f9fdc0c72c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.443+0000 7f9fe3e12640 1 --2- 172.21.15.32:0/1065208340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9fdc070940 0x7f9fdc0c58b0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.443+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1065208340 >> 172.21.15.32:0/1065208340 conn(0x7f9fdc06a510 msgr2=0x7f9fdc0c4c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.443+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1065208340 shutdown_connections 2023-12-20T04:45:52.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.443+0000 7f9fe3e12640 1 -- 172.21.15.32:0/1065208340 wait complete. 2023-12-20T04:45:52.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T04:45:52.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.623+0000 7f56ca0f2640 1 Processor -- start 2023-12-20T04:45:52.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.623+0000 7f56ca0f2640 1 -- start start 2023-12-20T04:45:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56ca0f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56c415abb0 0x7f56c40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56ca0f2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56c40b0cb0 0x7f56c415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56ca0f2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56c415b6a0 con 0x7f56c4070230 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56ca0f2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56c406e160 con 0x7f56c40b0cb0 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56ca0f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c415c070 con 0x7f56c415abb0 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56c2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56c40b0cb0 0x7f56c415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56c2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56c40b0cb0 0x7f56c415bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42436/0 (socket says 172.21.15.32:42436) 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56c2ffd640 1 -- 172.21.15.32:0/2153726429 learned_addr learned my addr 172.21.15.32:0/2153726429 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56c37fe640 1 --2- 172.21.15.32:0/2153726429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56c415abb0 0x7f56c40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56c2ffd640 1 -- 172.21.15.32:0/2153726429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56c415abb0 msgr2=0x7f56c40caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56c2ffd640 1 --2- 172.21.15.32:0/2153726429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56c415abb0 0x7f56c40caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56c2ffd640 1 -- 172.21.15.32:0/2153726429 >> v1:172.21.15.32:6789/0 conn(0x7f56c4070230 legacy=0x7f56c406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.627+0000 7f56c2ffd640 1 -- 172.21.15.32:0/2153726429 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c406d8c0 con 0x7f56c40b0cb0 2023-12-20T04:45:52.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56c2ffd640 1 --2- 172.21.15.32:0/2153726429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c40b0cb0 0x7f56c415bb30 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009870 tx=0x7f56b801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0ca5518840bdb4c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56c27fc640 1 -- 172.21.15.32:0/2153726429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80a9400 con 0x7f56c40b0cb0 2023-12-20T04:45:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56c27fc640 1 -- 172.21.15.32:0/2153726429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b80a9560 con 0x7f56c40b0cb0 2023-12-20T04:45:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56c27fc640 1 -- 172.21.15.32:0/2153726429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80a98f0 con 0x7f56c40b0cb0 2023-12-20T04:45:52.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56ca0f2640 1 -- 172.21.15.32:0/2153726429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c40b0cb0 msgr2=0x7f56c415bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56ca0f2640 1 --2- 172.21.15.32:0/2153726429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c40b0cb0 0x7f56c415bb30 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7f56b8009870 tx=0x7f56b801c540 comp rx=0 tx=0).stop 2023-12-20T04:45:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56ca0f2640 1 -- 172.21.15.32:0/2153726429 shutdown_connections 2023-12-20T04:45:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56ca0f2640 1 --2- 172.21.15.32:0/2153726429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c40b0cb0 0x7f56c415bb30 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56ca0f2640 1 --2- 172.21.15.32:0/2153726429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56c415abb0 0x7f56c40caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56ca0f2640 1 -- 172.21.15.32:0/2153726429 >> 172.21.15.32:0/2153726429 conn(0x7f56c406a510 msgr2=0x7f56c40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56ca0f2640 1 -- 172.21.15.32:0/2153726429 shutdown_connections 2023-12-20T04:45:52.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56ca0f2640 1 -- 172.21.15.32:0/2153726429 wait complete. 2023-12-20T04:45:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56ca0f2640 1 Processor -- start 2023-12-20T04:45:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.631+0000 7f56ca0f2640 1 -- start start 2023-12-20T04:45:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56ca0f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56c4070230 0x7f56c4149ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56ca0f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c40b0cb0 0x7f56c4143540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56ca0f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56c415abb0 0x7f56c40b2ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56ca0f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4144b30 con 0x7f56c40b0cb0 2023-12-20T04:45:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56ca0f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4144380 con 0x7f56c4070230 2023-12-20T04:45:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56ca0f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56c4144d80 con 0x7f56c415abb0 2023-12-20T04:45:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c40b0cb0 0x7f56c4143540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c40b0cb0 0x7f56c4143540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42444/0 (socket says 172.21.15.32:42444) 2023-12-20T04:45:52.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c2ffd640 1 -- 172.21.15.32:0/4231452293 learned_addr learned my addr 172.21.15.32:0/4231452293 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c3fff640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56c415abb0 0x7f56c40b2ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c37fe640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56c4070230 0x7f56c4149ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c2ffd640 1 -- 172.21.15.32:0/4231452293 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56c415abb0 msgr2=0x7f56c40b2ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c2ffd640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56c415abb0 0x7f56c40b2ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c2ffd640 1 -- 172.21.15.32:0/4231452293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56c4070230 msgr2=0x7f56c4149ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c2ffd640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56c4070230 0x7f56c4149ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c2ffd640 1 -- 172.21.15.32:0/4231452293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56b80094d0 con 0x7f56c40b0cb0 2023-12-20T04:45:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c37fe640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56c4070230 0x7f56c4149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:52.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56c2ffd640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c40b0cb0 0x7f56c4143540 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f56b801a170 tx=0x7f56b801ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56abfff640 1 -- 172.21.15.32:0/4231452293 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80a9400 con 0x7f56c40b0cb0 2023-12-20T04:45:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56abfff640 1 -- 172.21.15.32:0/4231452293 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56b80b1800 con 0x7f56c40b0cb0 2023-12-20T04:45:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56ca0f2640 1 -- 172.21.15.32:0/4231452293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56c4145060 con 0x7f56c40b0cb0 2023-12-20T04:45:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56abfff640 1 -- 172.21.15.32:0/4231452293 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56b80b1b70 con 0x7f56c40b0cb0 2023-12-20T04:45:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56ca0f2640 1 -- 172.21.15.32:0/4231452293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56c414a010 con 0x7f56c40b0cb0 2023-12-20T04:45:52.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56ca0f2640 1 -- 172.21.15.32:0/4231452293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56c415bd40 con 0x7f56c40b0cb0 2023-12-20T04:45:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.635+0000 7f56abfff640 1 -- 172.21.15.32:0/4231452293 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f56b80ba3a0 con 0x7f56c40b0cb0 2023-12-20T04:45:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.639+0000 7f56abfff640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f56b0065b80 0x7f56b0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.639+0000 7f56abfff640 1 -- 172.21.15.32:0/4231452293 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f56b811d720 con 0x7f56c40b0cb0 2023-12-20T04:45:52.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.639+0000 7f56c37fe640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f56b0065b80 0x7f56b0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.639+0000 7f56c37fe640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f56b0065b80 0x7f56b0068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f56b400b9f0 tx=0x7f56b4008040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:52.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.643+0000 7f56abfff640 1 -- 172.21.15.32:0/4231452293 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f56b80eb0f0 con 0x7f56c40b0cb0 2023-12-20T04:45:52.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.799+0000 7f56ca0f2640 1 -- 172.21.15.32:0/4231452293 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f56c404e000 con 0x7f56c40b0cb0 2023-12-20T04:45:52.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.803+0000 7f56abfff640 1 -- 172.21.15.32:0/4231452293 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v37) v1 ==== 120+0+0 (secure 0 0 0) 0x7f56b80a6070 con 0x7f56c40b0cb0 2023-12-20T04:45:52.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 -- 172.21.15.32:0/4231452293 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f56b0065b80 msgr2=0x7f56b0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f56b0065b80 0x7f56b0068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f56b400b9f0 tx=0x7f56b4008040 comp rx=0 tx=0).stop 2023-12-20T04:45:52.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 -- 172.21.15.32:0/4231452293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c40b0cb0 msgr2=0x7f56c4143540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c40b0cb0 0x7f56c4143540 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7f56b801a170 tx=0x7f56b801ad80 comp rx=0 tx=0).stop 2023-12-20T04:45:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 -- 172.21.15.32:0/4231452293 shutdown_connections 2023-12-20T04:45:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f56b0065b80 0x7f56b0068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56c415abb0 0x7f56c40b2ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56c40b0cb0 0x7f56c4143540 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 --2- 172.21.15.32:0/4231452293 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56c4070230 0x7f56c4149ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 -- 172.21.15.32:0/4231452293 >> 172.21.15.32:0/4231452293 conn(0x7f56c406a510 msgr2=0x7f56c414ecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 -- 172.21.15.32:0/4231452293 shutdown_connections 2023-12-20T04:45:52.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.807+0000 7f56ca0f2640 1 -- 172.21.15.32:0/4231452293 wait complete. 2023-12-20T04:45:52.825 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T04:45:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.971+0000 7f722b8fc640 1 Processor -- start 2023-12-20T04:45:52.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.971+0000 7f722b8fc640 1 -- start start 2023-12-20T04:45:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f722b8fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7224157480 0x7f72240c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f722b8fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72240cd4c0 0x7f72240b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f722b8fc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f72240b1100 con 0x7f72240b0760 2023-12-20T04:45:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f722b8fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72240cde20 con 0x7f7224157480 2023-12-20T04:45:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f722b8fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72240c8e30 con 0x7f72240cd4c0 2023-12-20T04:45:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f7228e97640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f72240b0760 0x7f722415d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49250/0 (socket says 172.21.15.32:49250) 2023-12-20T04:45:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f7228e97640 1 -- 172.21.15.32:0/3234904617 learned_addr learned my addr 172.21.15.32:0/3234904617 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:52.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f7229e99640 1 --2- 172.21.15.32:0/3234904617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72240cd4c0 0x7f72240b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f7229698640 1 --2- 172.21.15.32:0/3234904617 >> v2:172.21.15.39:3300/0 conn(0x7f7224157480 0x7f72240c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f721bfff640 1 -- 172.21.15.32:0/3234904617 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2731909597 0 0) 0x7f72240b1100 con 0x7f72240b0760 2023-12-20T04:45:52.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f721bfff640 1 -- 172.21.15.32:0/3234904617 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7210003530 con 0x7f72240b0760 2023-12-20T04:45:52.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f7229e99640 1 -- 172.21.15.32:0/3234904617 >> v2:172.21.15.39:3300/0 conn(0x7f7224157480 msgr2=0x7f72240c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:52.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f7229e99640 1 --2- 172.21.15.32:0/3234904617 >> v2:172.21.15.39:3300/0 conn(0x7f7224157480 0x7f72240c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f7229e99640 1 -- 172.21.15.32:0/3234904617 >> v1:172.21.15.32:6789/0 conn(0x7f72240b0760 legacy=0x7f722415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.975+0000 7f7229e99640 1 -- 172.21.15.32:0/3234904617 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72240ce070 con 0x7f72240cd4c0 2023-12-20T04:45:52.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f7229e99640 1 --2- 172.21.15.32:0/3234904617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72240cd4c0 0x7f72240b0bc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f721c009d90 tx=0x7f721c01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=128679472aeeb82d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:52.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f721bfff640 1 -- 172.21.15.32:0/3234904617 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0a5870 con 0x7f72240cd4c0 2023-12-20T04:45:52.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f721bfff640 1 -- 172.21.15.32:0/3234904617 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f721c0a59d0 con 0x7f72240cd4c0 2023-12-20T04:45:52.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f721bfff640 1 -- 172.21.15.32:0/3234904617 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f721c0a5d60 con 0x7f72240cd4c0 2023-12-20T04:45:52.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f722b8fc640 1 -- 172.21.15.32:0/3234904617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72240cd4c0 msgr2=0x7f72240b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f722b8fc640 1 --2- 172.21.15.32:0/3234904617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72240cd4c0 0x7f72240b0bc0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f721c009d90 tx=0x7f721c01c890 comp rx=0 tx=0).stop 2023-12-20T04:45:52.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f722b8fc640 1 -- 172.21.15.32:0/3234904617 shutdown_connections 2023-12-20T04:45:52.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f722b8fc640 1 --2- 172.21.15.32:0/3234904617 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72240cd4c0 0x7f72240b0bc0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f722b8fc640 1 --2- 172.21.15.32:0/3234904617 >> v2:172.21.15.39:3300/0 conn(0x7f7224157480 0x7f72240c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f722b8fc640 1 -- 172.21.15.32:0/3234904617 >> 172.21.15.32:0/3234904617 conn(0x7f722406a510 msgr2=0x7f72240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:52.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f722b8fc640 1 -- 172.21.15.32:0/3234904617 shutdown_connections 2023-12-20T04:45:52.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f722b8fc640 1 -- 172.21.15.32:0/3234904617 wait complete. 2023-12-20T04:45:52.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f722b8fc640 1 Processor -- start 2023-12-20T04:45:52.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.979+0000 7f722b8fc640 1 -- start start 2023-12-20T04:45:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f722b8fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72240b0760 0x7f7224155620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f722b8fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72240cd4c0 0x7f7224140210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f722b8fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7224157480 0x7f72240b2390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f722b8fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72240b37b0 con 0x7f72240cd4c0 2023-12-20T04:45:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f722b8fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7224140750 con 0x7f7224157480 2023-12-20T04:45:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f722b8fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72240b28d0 con 0x7f72240b0760 2023-12-20T04:45:52.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7228e97640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72240cd4c0 0x7f7224140210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7228e97640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72240cd4c0 0x7f7224140210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42452/0 (socket says 172.21.15.32:42452) 2023-12-20T04:45:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7228e97640 1 -- 172.21.15.32:0/2285793910 learned_addr learned my addr 172.21.15.32:0/2285793910 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7229e99640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7224157480 0x7f72240b2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7229698640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72240b0760 0x7f7224155620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7228e97640 1 -- 172.21.15.32:0/2285793910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72240b0760 msgr2=0x7f7224155620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7228e97640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72240b0760 0x7f7224155620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7228e97640 1 -- 172.21.15.32:0/2285793910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7224157480 msgr2=0x7f72240b2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7228e97640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7224157480 0x7f72240b2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7228e97640 1 -- 172.21.15.32:0/2285793910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f721c009a70 con 0x7f72240cd4c0 2023-12-20T04:45:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7228e97640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72240cd4c0 0x7f7224140210 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f721400c420 tx=0x7f721400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:52.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7219ffb640 1 -- 172.21.15.32:0/2285793910 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7214005cf0 con 0x7f72240cd4c0 2023-12-20T04:45:52.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7219ffb640 1 -- 172.21.15.32:0/2285793910 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7214005e50 con 0x7f72240cd4c0 2023-12-20T04:45:52.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f722b8fc640 1 -- 172.21.15.32:0/2285793910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72240b2ba0 con 0x7f72240cd4c0 2023-12-20T04:45:52.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f7219ffb640 1 -- 172.21.15.32:0/2285793910 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72140072d0 con 0x7f72240cd4c0 2023-12-20T04:45:52.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f722b8fc640 1 -- 172.21.15.32:0/2285793910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72240b3160 con 0x7f72240cd4c0 2023-12-20T04:45:52.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.983+0000 7f722b8fc640 1 -- 172.21.15.32:0/2285793910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72240b3960 con 0x7f72240cd4c0 2023-12-20T04:45:52.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.987+0000 7f7219ffb640 1 -- 172.21.15.32:0/2285793910 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7214007430 con 0x7f72240cd4c0 2023-12-20T04:45:52.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.987+0000 7f7219ffb640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7208065bb0 0x7f7208068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:52.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.987+0000 7f7219ffb640 1 -- 172.21.15.32:0/2285793910 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f72140827e0 con 0x7f72240cd4c0 2023-12-20T04:45:52.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.987+0000 7f7229698640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7208065bb0 0x7f7208068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:52.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.987+0000 7f7229698640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7208065bb0 0x7f7208068070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7220009b60 tx=0x7f7220009190 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:52.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:52.991+0000 7f7219ffb640 1 -- 172.21.15.32:0/2285793910 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f72240b3960 con 0x7f72240cd4c0 2023-12-20T04:45:53.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.163+0000 7f722b8fc640 1 -- 172.21.15.32:0/2285793910 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f722404e000 con 0x7f72240cd4c0 2023-12-20T04:45:53.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f7219ffb640 1 -- 172.21.15.32:0/2285793910 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v38) v1 ==== 115+0+0 (secure 0 0 0) 0x7f72140170b0 con 0x7f72240cd4c0 2023-12-20T04:45:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 -- 172.21.15.32:0/2285793910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7208065bb0 msgr2=0x7f7208068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7208065bb0 0x7f7208068070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f7220009b60 tx=0x7f7220009190 comp rx=0 tx=0).stop 2023-12-20T04:45:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 -- 172.21.15.32:0/2285793910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72240cd4c0 msgr2=0x7f7224140210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72240cd4c0 0x7f7224140210 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7f721400c420 tx=0x7f721400c8e0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 -- 172.21.15.32:0/2285793910 shutdown_connections 2023-12-20T04:45:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7208065bb0 0x7f7208068070 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7224157480 0x7f72240b2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f72240cd4c0 0x7f7224140210 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 --2- 172.21.15.32:0/2285793910 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f72240b0760 0x7f7224155620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 -- 172.21.15.32:0/2285793910 >> 172.21.15.32:0/2285793910 conn(0x7f722406a510 msgr2=0x7f72241564a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 -- 172.21.15.32:0/2285793910 shutdown_connections 2023-12-20T04:45:53.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.171+0000 7f722b8fc640 1 -- 172.21.15.32:0/2285793910 wait complete. 2023-12-20T04:45:53.190 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T04:45:53.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c8bbc640 1 Processor -- start 2023-12-20T04:45:53.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c8bbc640 1 -- start start 2023-12-20T04:45:53.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c8bbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64c40b0bc0 0x7f64c40ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:53.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c8bbc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c40cde20 con 0x7f64c40b0760 2023-12-20T04:45:53.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c8bbc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c40c8e30 con 0x7f64c4157480 2023-12-20T04:45:53.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c8bbc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c40c9f50 con 0x7f64c40b0bc0 2023-12-20T04:45:53.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c2d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64c40b0bc0 0x7f64c40ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:53.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c2d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64c40b0bc0 0x7f64c40ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42468/0 (socket says 172.21.15.32:42468) 2023-12-20T04:45:53.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c2d9d640 1 -- 172.21.15.32:0/66782834 learned_addr learned my addr 172.21.15.32:0/66782834 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:53.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c2d9d640 1 -- 172.21.15.32:0/66782834 >> v1:172.21.15.39:6789/0 conn(0x7f64c4157480 legacy=0x7f64c40c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c2d9d640 1 -- 172.21.15.32:0/66782834 >> v1:172.21.15.32:6789/0 conn(0x7f64c40b0760 legacy=0x7f64c40cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.339+0000 7f64c2d9d640 1 -- 172.21.15.32:0/66782834 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c40ce070 con 0x7f64c40b0bc0 2023-12-20T04:45:53.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.343+0000 7f64c2d9d640 1 --2- 172.21.15.32:0/66782834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c40b0bc0 0x7f64c40ccbc0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009e50 tx=0x7f64bc01eb30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f10576f077753c62 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:53.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.343+0000 7f64c1d9b640 1 -- 172.21.15.32:0/66782834 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc0a79b0 con 0x7f64c40b0bc0 2023-12-20T04:45:53.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.343+0000 7f64c1d9b640 1 -- 172.21.15.32:0/66782834 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc0a7b10 con 0x7f64c40b0bc0 2023-12-20T04:45:53.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.343+0000 7f64c1d9b640 1 -- 172.21.15.32:0/66782834 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc0a7ea0 con 0x7f64c40b0bc0 2023-12-20T04:45:53.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.343+0000 7f64c8bbc640 1 -- 172.21.15.32:0/66782834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c40b0bc0 msgr2=0x7f64c40ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.343+0000 7f64c8bbc640 1 --2- 172.21.15.32:0/66782834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c40b0bc0 0x7f64c40ccbc0 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f64bc009e50 tx=0x7f64bc01eb30 comp rx=0 tx=0).stop 2023-12-20T04:45:53.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.343+0000 7f64c8bbc640 1 -- 172.21.15.32:0/66782834 shutdown_connections 2023-12-20T04:45:53.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.343+0000 7f64c8bbc640 1 --2- 172.21.15.32:0/66782834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c40b0bc0 0x7f64c40ccbc0 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.343+0000 7f64c8bbc640 1 -- 172.21.15.32:0/66782834 >> 172.21.15.32:0/66782834 conn(0x7f64c406a510 msgr2=0x7f64c40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:53.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.343+0000 7f64c8bbc640 1 -- 172.21.15.32:0/66782834 shutdown_connections 2023-12-20T04:45:53.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.343+0000 7f64c8bbc640 1 -- 172.21.15.32:0/66782834 wait complete. 2023-12-20T04:45:53.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c8bbc640 1 Processor -- start 2023-12-20T04:45:53.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c8bbc640 1 -- start start 2023-12-20T04:45:53.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c8bbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c40b0760 0x7f64c4149c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:53.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c8bbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64c40b0bc0 0x7f64c40b3fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:53.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c8bbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c4157480 0x7f64c41f81a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:53.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c8bbc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c414dc40 con 0x7f64c4157480 2023-12-20T04:45:53.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c8bbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c4165560 con 0x7f64c40b0bc0 2023-12-20T04:45:53.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c8bbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c414a170 con 0x7f64c40b0760 2023-12-20T04:45:53.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c4157480 0x7f64c41f81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:53.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c4157480 0x7f64c41f81a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42476/0 (socket says 172.21.15.32:42476) 2023-12-20T04:45:53.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c2d9d640 1 -- 172.21.15.32:0/3100898205 learned_addr learned my addr 172.21.15.32:0/3100898205 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:53.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c259c640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c40b0760 0x7f64c4149c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:53.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64bbfff640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64c40b0bc0 0x7f64c40b3fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c2d9d640 1 -- 172.21.15.32:0/3100898205 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c40b0760 msgr2=0x7f64c4149c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c2d9d640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c40b0760 0x7f64c4149c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c2d9d640 1 -- 172.21.15.32:0/3100898205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64c40b0bc0 msgr2=0x7f64c40b3fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c2d9d640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64c40b0bc0 0x7f64c40b3fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c2d9d640 1 -- 172.21.15.32:0/3100898205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64bc009b30 con 0x7f64c4157480 2023-12-20T04:45:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c259c640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c40b0760 0x7f64c4149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:53.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c2d9d640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c4157480 0x7f64c41f81a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0b1040 tx=0x7f64bc01b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:53.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64baffd640 1 -- 172.21.15.32:0/3100898205 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc0a79b0 con 0x7f64c4157480 2023-12-20T04:45:53.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64baffd640 1 -- 172.21.15.32:0/3100898205 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc00e150 con 0x7f64c4157480 2023-12-20T04:45:53.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64baffd640 1 -- 172.21.15.32:0/3100898205 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc009510 con 0x7f64c4157480 2023-12-20T04:45:53.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c8bbc640 1 -- 172.21.15.32:0/3100898205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c4164950 con 0x7f64c4157480 2023-12-20T04:45:53.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c8bbc640 1 -- 172.21.15.32:0/3100898205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c4164f10 con 0x7f64c4157480 2023-12-20T04:45:53.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.347+0000 7f64c8bbc640 1 -- 172.21.15.32:0/3100898205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c4152920 con 0x7f64c4157480 2023-12-20T04:45:53.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.351+0000 7f64baffd640 1 -- 172.21.15.32:0/3100898205 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f64bc0c2430 con 0x7f64c4157480 2023-12-20T04:45:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.351+0000 7f64baffd640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f64a4065bb0 0x7f64a4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.351+0000 7f64baffd640 1 -- 172.21.15.32:0/3100898205 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f64bc11d5e0 con 0x7f64c4157480 2023-12-20T04:45:53.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.351+0000 7f64c259c640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f64a4065bb0 0x7f64a4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:53.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.351+0000 7f64c259c640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f64a4065bb0 0x7f64a4068070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64b4000f70 tx=0x7f64b4009040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:53.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.355+0000 7f64baffd640 1 -- 172.21.15.32:0/3100898205 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f64c4152920 con 0x7f64c4157480 2023-12-20T04:45:53.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.515+0000 7f64c8bbc640 1 -- 172.21.15.32:0/3100898205 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f64c4152920 con 0x7f64c4157480 2023-12-20T04:45:53.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64baffd640 1 -- 172.21.15.32:0/3100898205 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v39) v1 ==== 104+0+0 (secure 0 0 0) 0x7f64c4152920 con 0x7f64c4157480 2023-12-20T04:45:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 -- 172.21.15.32:0/3100898205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f64a4065bb0 msgr2=0x7f64a4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f64a4065bb0 0x7f64a4068070 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f64b4000f70 tx=0x7f64b4009040 comp rx=0 tx=0).stop 2023-12-20T04:45:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 -- 172.21.15.32:0/3100898205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c4157480 msgr2=0x7f64c41f81a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c4157480 0x7f64c41f81a0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f64bc0b1040 tx=0x7f64bc01b8d0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 -- 172.21.15.32:0/3100898205 shutdown_connections 2023-12-20T04:45:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f64a4065bb0 0x7f64a4068070 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c4157480 0x7f64c41f81a0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64c40b0bc0 0x7f64c40b3fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 --2- 172.21.15.32:0/3100898205 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c40b0760 0x7f64c4149c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 -- 172.21.15.32:0/3100898205 >> 172.21.15.32:0/3100898205 conn(0x7f64c406a510 msgr2=0x7f64c40c3de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:53.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 -- 172.21.15.32:0/3100898205 shutdown_connections 2023-12-20T04:45:53.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.523+0000 7f64c8bbc640 1 -- 172.21.15.32:0/3100898205 wait complete. 2023-12-20T04:45:53.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path' 2023-12-20T04:45:53.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.687+0000 7f1963cd1640 1 Processor -- start 2023-12-20T04:45:53.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.687+0000 7f1963cd1640 1 -- start start 2023-12-20T04:45:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.687+0000 7f1963cd1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f195c0aece0 0x7f195c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.687+0000 7f1963cd1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f195c0b0b30 0x7f195c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.687+0000 7f1963cd1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f195c0c98c0 con 0x7f195c1599e0 2023-12-20T04:45:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.687+0000 7f1963cd1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f195c0c9ba0 con 0x7f195c0b0b30 2023-12-20T04:45:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.687+0000 7f1963cd1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0c9e80 con 0x7f195c0aece0 2023-12-20T04:45:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.687+0000 7f1961a6d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f195c1599e0 0x7f195c0c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49274/0 (socket says 172.21.15.32:49274) 2023-12-20T04:45:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.687+0000 7f1961a6d640 1 -- 172.21.15.32:0/52761782 learned_addr learned my addr 172.21.15.32:0/52761782 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.687+0000 7f196226e640 1 --2- 172.21.15.32:0/52761782 >> v2:172.21.15.32:3300/0 conn(0x7f195c0b0b30 0x7f195c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.687+0000 7f196126c640 1 --2- 172.21.15.32:0/52761782 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f195c0aece0 0x7f195c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f196226e640 1 -- 172.21.15.32:0/52761782 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f195c0aece0 msgr2=0x7f195c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f196226e640 1 --2- 172.21.15.32:0/52761782 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f195c0aece0 0x7f195c0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f196226e640 1 -- 172.21.15.32:0/52761782 >> v1:172.21.15.32:6789/0 conn(0x7f195c1599e0 legacy=0x7f195c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f196226e640 1 -- 172.21.15.32:0/52761782 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f195c0ca160 con 0x7f195c0b0b30 2023-12-20T04:45:53.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f196126c640 1 --2- 172.21.15.32:0/52761782 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f195c0aece0 0x7f195c0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:53.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f196226e640 1 --2- 172.21.15.32:0/52761782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f195c0b0b30 0x7f195c0701c0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1954009d90 tx=0x7f195401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a996c2a12b2f60af server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:53.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f1960a6b640 1 -- 172.21.15.32:0/52761782 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19540a5920 con 0x7f195c0b0b30 2023-12-20T04:45:53.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f1960a6b640 1 -- 172.21.15.32:0/52761782 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19540a5a80 con 0x7f195c0b0b30 2023-12-20T04:45:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f1960a6b640 1 -- 172.21.15.32:0/52761782 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19540a5e10 con 0x7f195c0b0b30 2023-12-20T04:45:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f1963cd1640 1 -- 172.21.15.32:0/52761782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f195c0b0b30 msgr2=0x7f195c0701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f1963cd1640 1 --2- 172.21.15.32:0/52761782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f195c0b0b30 0x7f195c0701c0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f1954009d90 tx=0x7f195401c950 comp rx=0 tx=0).stop 2023-12-20T04:45:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f1963cd1640 1 -- 172.21.15.32:0/52761782 shutdown_connections 2023-12-20T04:45:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f1963cd1640 1 --2- 172.21.15.32:0/52761782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f195c0b0b30 0x7f195c0701c0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f1963cd1640 1 --2- 172.21.15.32:0/52761782 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f195c0aece0 0x7f195c0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f1963cd1640 1 -- 172.21.15.32:0/52761782 >> 172.21.15.32:0/52761782 conn(0x7f195c06a320 msgr2=0x7f195c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f1963cd1640 1 -- 172.21.15.32:0/52761782 shutdown_connections 2023-12-20T04:45:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.691+0000 7f1963cd1640 1 -- 172.21.15.32:0/52761782 wait complete. 2023-12-20T04:45:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f1963cd1640 1 Processor -- start 2023-12-20T04:45:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f1963cd1640 1 -- start start 2023-12-20T04:45:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f1963cd1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f195c0aece0 0x7f195c0c4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f1963cd1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f195c0b0b30 0x7f195c157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f1963cd1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f195c1599e0 0x7f195c1577c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f1963cd1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0aceb0 con 0x7f195c0b0b30 2023-12-20T04:45:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f1963cd1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f195c0acbd0 con 0x7f195c0aece0 2023-12-20T04:45:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f1963cd1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f195c15bd40 con 0x7f195c1599e0 2023-12-20T04:45:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f196126c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f195c0b0b30 0x7f195c157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f196126c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f195c0b0b30 0x7f195c157280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42488/0 (socket says 172.21.15.32:42488) 2023-12-20T04:45:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f196126c640 1 -- 172.21.15.32:0/1046353270 learned_addr learned my addr 172.21.15.32:0/1046353270 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f1961a6d640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f195c0aece0 0x7f195c0c4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f196226e640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f195c1599e0 0x7f195c1577c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f196126c640 1 -- 172.21.15.32:0/1046353270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f195c1599e0 msgr2=0x7f195c1577c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f196126c640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f195c1599e0 0x7f195c1577c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f196126c640 1 -- 172.21.15.32:0/1046353270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f195c0aece0 msgr2=0x7f195c0c4c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f196126c640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f195c0aece0 0x7f195c0c4c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f196126c640 1 -- 172.21.15.32:0/1046353270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1954009a70 con 0x7f195c0b0b30 2023-12-20T04:45:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f196126c640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f195c0b0b30 0x7f195c157280 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f194c00d800 tx=0x7f194c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f19527fc640 1 -- 172.21.15.32:0/1046353270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00b6c0 con 0x7f195c0b0b30 2023-12-20T04:45:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f19527fc640 1 -- 172.21.15.32:0/1046353270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f194c00b820 con 0x7f195c0b0b30 2023-12-20T04:45:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f19527fc640 1 -- 172.21.15.32:0/1046353270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f194c00bbd0 con 0x7f195c0b0b30 2023-12-20T04:45:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f1963cd1640 1 -- 172.21.15.32:0/1046353270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f195c15c020 con 0x7f195c0b0b30 2023-12-20T04:45:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.695+0000 7f1963cd1640 1 -- 172.21.15.32:0/1046353270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f195c1f5420 con 0x7f195c0b0b30 2023-12-20T04:45:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.699+0000 7f1963cd1640 1 -- 172.21.15.32:0/1046353270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f195c15bba0 con 0x7f195c0b0b30 2023-12-20T04:45:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.699+0000 7f19527fc640 1 -- 172.21.15.32:0/1046353270 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f194c00bd30 con 0x7f195c0b0b30 2023-12-20T04:45:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.699+0000 7f19527fc640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1940065b80 0x7f1940068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.699+0000 7f19527fc640 1 -- 172.21.15.32:0/1046353270 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f194c083700 con 0x7f195c0b0b30 2023-12-20T04:45:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.699+0000 7f1961a6d640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1940065b80 0x7f1940068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:53.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.699+0000 7f1961a6d640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1940065b80 0x7f1940068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f19580010f0 tx=0x7f1958007d10 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:53.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.703+0000 7f19527fc640 1 -- 172.21.15.32:0/1046353270 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f195c15bba0 con 0x7f195c0b0b30 2023-12-20T04:45:53.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:53.915+0000 7f1963cd1640 1 -- 172.21.15.32:0/1046353270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]} v 0) v1 -- 0x7f195c15bba0 con 0x7f195c0b0b30 2023-12-20T04:45:54.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.451+0000 7f19527fc640 1 -- 172.21.15.32:0/1046353270 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_schedule_auto_deactivation_for_non_existent_path"]}]=0 v651) v1 ==== 183+0+0 (secure 0 0 0) 0x7f195c15bba0 con 0x7f195c0b0b30 2023-12-20T04:45:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 -- 172.21.15.32:0/1046353270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1940065b80 msgr2=0x7f1940068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1940065b80 0x7f1940068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f19580010f0 tx=0x7f1958007d10 comp rx=0 tx=0).stop 2023-12-20T04:45:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 -- 172.21.15.32:0/1046353270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f195c0b0b30 msgr2=0x7f195c157280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f195c0b0b30 0x7f195c157280 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f194c00d800 tx=0x7f194c00dcc0 comp rx=0 tx=0).stop 2023-12-20T04:45:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 -- 172.21.15.32:0/1046353270 shutdown_connections 2023-12-20T04:45:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1940065b80 0x7f1940068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:54.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f195c1599e0 0x7f195c1577c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f195c0b0b30 0x7f195c157280 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 --2- 172.21.15.32:0/1046353270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f195c0aece0 0x7f195c0c4c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 -- 172.21.15.32:0/1046353270 >> 172.21.15.32:0/1046353270 conn(0x7f195c06a320 msgr2=0x7f195c06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 -- 172.21.15.32:0/1046353270 shutdown_connections 2023-12-20T04:45:54.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.455+0000 7f1963cd1640 1 -- 172.21.15.32:0/1046353270 wait complete. 2023-12-20T04:45:54.475 INFO:tasks.cephfs_test_runner:Test that a non-existent path leads to schedule deactivation after a few retries. ... ok 2023-12-20T04:45:54.475 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test existence of a scheduled snapshot 2023-12-20T04:45:54.475 INFO:tasks.cephfs_test_runner:test_snap_schedule (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-20T04:45:54.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-12-20T04:45:54.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d4055640 1 Processor -- start 2023-12-20T04:45:54.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d4055640 1 -- start start 2023-12-20T04:45:54.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d4055640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc15cd70 0x7fd1cc0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:54.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d4055640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd1cc160ff0 0x7fd1cc0b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:54.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d4055640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1cc0b2d70 con 0x7fd1cc0b23d0 2023-12-20T04:45:54.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d4055640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1cc1618b0 con 0x7fd1cc160ff0 2023-12-20T04:45:54.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d4055640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0ce6c0 con 0x7fd1cc15cd70 2023-12-20T04:45:54.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d15f0640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd1cc0b23d0 0x7fd1cc1608e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49276/0 (socket says 172.21.15.32:49276) 2023-12-20T04:45:54.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d15f0640 1 -- 172.21.15.32:0/1353993104 learned_addr learned my addr 172.21.15.32:0/1353993104 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d1df1640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc15cd70 0x7fd1cc0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d25f2640 1 --2- 172.21.15.32:0/1353993104 >> v2:172.21.15.39:3300/0 conn(0x7fd1cc160ff0 0x7fd1cc0b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3576495194 0 0) 0x7fd1cc0b2d70 con 0x7fd1cc0b23d0 2023-12-20T04:45:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd1b8003530 con 0x7fd1cc0b23d0 2023-12-20T04:45:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1783323814 0 0) 0x7fd1b8003530 con 0x7fd1cc0b23d0 2023-12-20T04:45:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc15cd70 msgr2=0x7fd1cc0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc15cd70 0x7fd1cc0cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 >> v2:172.21.15.39:3300/0 conn(0x7fd1cc160ff0 msgr2=0x7fd1cc0b2830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:54.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 --2- 172.21.15.32:0/1353993104 >> v2:172.21.15.39:3300/0 conn(0x7fd1cc160ff0 0x7fd1cc0b2830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc1615d0 con 0x7fd1cc0b23d0 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fd1bc001680 con 0x7fd1cc0b23d0 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 >> v1:172.21.15.32:6789/0 conn(0x7fd1cc0b23d0 legacy=0x7fd1cc1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b8003980 0x7fd1b8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1b80047d0 0x7fd1b8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc15cd70 0x7fd1b8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc1618b0 con 0x7fd1b8003980 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0ce6c0 con 0x7fd1b80047d0 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1b800b990 con 0x7fd1cc15cd70 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d1df1640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b8003980 0x7fd1b8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d25f2640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc15cd70 0x7fd1b8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d15f0640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1b80047d0 0x7fd1b8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d1df1640 1 -- 172.21.15.32:0/1353993104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc15cd70 msgr2=0x7fd1b8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d1df1640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc15cd70 0x7fd1b8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:54.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d1df1640 1 -- 172.21.15.32:0/1353993104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1b80047d0 msgr2=0x7fd1b8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:54.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d1df1640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1b80047d0 0x7fd1b8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:54.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.627+0000 7fd1d1df1640 1 -- 172.21.15.32:0/1353993104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1b8003530 con 0x7fd1b8003980 2023-12-20T04:45:54.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.631+0000 7fd1d1df1640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b8003980 0x7fd1b8003da0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009ae0 tx=0x7fd1c801ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8729af776714306 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:54.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.631+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80a6430 con 0x7fd1b8003980 2023-12-20T04:45:54.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.631+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c80a6590 con 0x7fd1b8003980 2023-12-20T04:45:54.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.631+0000 7fd1d0def640 1 -- 172.21.15.32:0/1353993104 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c80a6940 con 0x7fd1b8003980 2023-12-20T04:45:54.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.631+0000 7fd1d4055640 1 -- 172.21.15.32:0/1353993104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b8003980 msgr2=0x7fd1b8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:54.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.631+0000 7fd1d4055640 1 --2- 172.21.15.32:0/1353993104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1b8003980 0x7fd1b8003da0 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fd1c8009ae0 tx=0x7fd1c801ca20 comp rx=0 tx=0).stop 2023-12-20T04:45:54.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.631+0000 7fd1d25f2640 1 -- 172.21.15.32:0/1353993104 reap_dead start 2023-12-20T04:45:54.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.631+0000 7fd1d4055640 1 -- 172.21.15.32:0/1353993104 shutdown_connections 2023-12-20T04:45:54.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.631+0000 7fd1d4055640 1 -- 172.21.15.32:0/1353993104 >> 172.21.15.32:0/1353993104 conn(0x7fd1cc0cd3d0 msgr2=0x7fd1cc0cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:54.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.631+0000 7fd1d4055640 1 -- 172.21.15.32:0/1353993104 shutdown_connections 2023-12-20T04:45:54.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.631+0000 7fd1d4055640 1 -- 172.21.15.32:0/1353993104 wait complete. 2023-12-20T04:45:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d4055640 1 Processor -- start 2023-12-20T04:45:54.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d4055640 1 -- start start 2023-12-20T04:45:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d4055640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1b8003980 0x7fd1cc15adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:54.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d4055640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc0ac140 0x7fd1cc0ac5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d4055640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0ae2d0 0x7fd1cc0ae750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d4055640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0acb00 con 0x7fd1cc0ae2d0 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d4055640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0af8e0 con 0x7fd1b8003980 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d4055640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1cc0af040 con 0x7fd1cc0ac140 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d25f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0ae2d0 0x7fd1cc0ae750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d25f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0ae2d0 0x7fd1cc0ae750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42512/0 (socket says 172.21.15.32:42512) 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d25f2640 1 -- 172.21.15.32:0/2285225834 learned_addr learned my addr 172.21.15.32:0/2285225834 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d1df1640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1b8003980 0x7fd1cc15adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d15f0640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc0ac140 0x7fd1cc0ac5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d25f2640 1 -- 172.21.15.32:0/2285225834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc0ac140 msgr2=0x7fd1cc0ac5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d25f2640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc0ac140 0x7fd1cc0ac5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d25f2640 1 -- 172.21.15.32:0/2285225834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1b8003980 msgr2=0x7fd1cc15adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d25f2640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1b8003980 0x7fd1cc15adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d25f2640 1 -- 172.21.15.32:0/2285225834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1cc0af320 con 0x7fd1cc0ae2d0 2023-12-20T04:45:54.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d15f0640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc0ac140 0x7fd1cc0ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:54.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d25f2640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0ae2d0 0x7fd1cc0ae750 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0cf6a0 tx=0x7fd1c40115a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:54.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1c27fc640 1 -- 172.21.15.32:0/2285225834 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c400e070 con 0x7fd1cc0ae2d0 2023-12-20T04:45:54.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1c27fc640 1 -- 172.21.15.32:0/2285225834 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1c4011e10 con 0x7fd1cc0ae2d0 2023-12-20T04:45:54.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1c27fc640 1 -- 172.21.15.32:0/2285225834 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1c40039a0 con 0x7fd1cc0ae2d0 2023-12-20T04:45:54.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d4055640 1 -- 172.21.15.32:0/2285225834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1cc0af600 con 0x7fd1cc0ae2d0 2023-12-20T04:45:54.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.635+0000 7fd1d4055640 1 -- 172.21.15.32:0/2285225834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd1cc15b330 con 0x7fd1cc0ae2d0 2023-12-20T04:45:54.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.639+0000 7fd1d4055640 1 -- 172.21.15.32:0/2285225834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1cc04e000 con 0x7fd1cc0ae2d0 2023-12-20T04:45:54.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.639+0000 7fd1c27fc640 1 -- 172.21.15.32:0/2285225834 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd1c4003b00 con 0x7fd1cc0ae2d0 2023-12-20T04:45:54.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.639+0000 7fd1c27fc640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd1b0065b80 0x7fd1b0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:54.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.639+0000 7fd1c27fc640 1 -- 172.21.15.32:0/2285225834 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5631+0+0 (secure 0 0 0) 0x7fd1c4083a20 con 0x7fd1cc0ae2d0 2023-12-20T04:45:54.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.639+0000 7fd1d1df1640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd1b0065b80 0x7fd1b0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:54.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.639+0000 7fd1d1df1640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd1b0065b80 0x7fd1b0068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80ad000 tx=0x7fd1c80b3040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:54.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.643+0000 7fd1c27fc640 1 -- 172.21.15.32:0/2285225834 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd1c4051170 con 0x7fd1cc0ae2d0 2023-12-20T04:45:54.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:54.855+0000 7fd1d4055640 1 -- 172.21.15.32:0/2285225834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7fd1cc15deb0 con 0x7fd1cc0ae2d0 2023-12-20T04:45:55.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.459+0000 7fd1c27fc640 1 -- 172.21.15.32:0/2285225834 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v652) v1 ==== 151+0+0 (secure 0 0 0) 0x7fd1c4054790 con 0x7fd1cc0ae2d0 2023-12-20T04:45:55.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 -- 172.21.15.32:0/2285225834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd1b0065b80 msgr2=0x7fd1b0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:55.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd1b0065b80 0x7fd1b0068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fd1c80ad000 tx=0x7fd1c80b3040 comp rx=0 tx=0).stop 2023-12-20T04:45:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 -- 172.21.15.32:0/2285225834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0ae2d0 msgr2=0x7fd1cc0ae750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0ae2d0 0x7fd1cc0ae750 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fd1cc0cf6a0 tx=0x7fd1c40115a0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 -- 172.21.15.32:0/2285225834 shutdown_connections 2023-12-20T04:45:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd1b0065b80 0x7fd1b0068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1cc0ae2d0 0x7fd1cc0ae750 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1cc0ac140 0x7fd1cc0ac5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 --2- 172.21.15.32:0/2285225834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd1b8003980 0x7fd1cc15adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 -- 172.21.15.32:0/2285225834 >> 172.21.15.32:0/2285225834 conn(0x7fd1cc0cd3d0 msgr2=0x7fd1cc0cb470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 -- 172.21.15.32:0/2285225834 shutdown_connections 2023-12-20T04:45:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.463+0000 7fd1d4055640 1 -- 172.21.15.32:0/2285225834 wait complete. 2023-12-20T04:45:55.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T04:45:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.635+0000 7f866e125640 1 Processor -- start 2023-12-20T04:45:55.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.635+0000 7f866e125640 1 -- start start 2023-12-20T04:45:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.635+0000 7f866e125640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f866806cdf0 0x7f8668070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.635+0000 7f866e125640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8668070770 0x7f86680cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.635+0000 7f866e125640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86680cc3b0 con 0x7f86681591e0 2023-12-20T04:45:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.635+0000 7f866e125640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8668070ba0 con 0x7f866806cdf0 2023-12-20T04:45:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.635+0000 7f866e125640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86680c8e30 con 0x7f8668070770 2023-12-20T04:45:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.635+0000 7f86677fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f86681591e0 0x7f86680caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:49278/0 (socket says 172.21.15.32:49278) 2023-12-20T04:45:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.639+0000 7f86677fe640 1 -- 172.21.15.32:0/2623904412 learned_addr learned my addr 172.21.15.32:0/2623904412 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.639+0000 7f8666ffd640 1 --2- 172.21.15.32:0/2623904412 >> v2:172.21.15.32:3300/0 conn(0x7f866806cdf0 0x7f8668070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:55.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.639+0000 7f8667fff640 1 --2- 172.21.15.32:0/2623904412 >> v2:172.21.15.39:3300/0 conn(0x7f8668070770 0x7f86680cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.639+0000 7f8666ffd640 1 -- 172.21.15.32:0/2623904412 >> v2:172.21.15.39:3300/0 conn(0x7f8668070770 msgr2=0x7f86680cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.639+0000 7f8666ffd640 1 --2- 172.21.15.32:0/2623904412 >> v2:172.21.15.39:3300/0 conn(0x7f8668070770 0x7f86680cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.639+0000 7f8666ffd640 1 -- 172.21.15.32:0/2623904412 >> v1:172.21.15.32:6789/0 conn(0x7f86681591e0 legacy=0x7f86680caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:55.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.639+0000 7f8666ffd640 1 -- 172.21.15.32:0/2623904412 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f866806d280 con 0x7f866806cdf0 2023-12-20T04:45:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.639+0000 7f8666ffd640 1 --2- 172.21.15.32:0/2623904412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866806cdf0 0x7f8668070230 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f86680c5860 tx=0x7f865c01c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48b7935ef57b87a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.639+0000 7f86667fc640 1 -- 172.21.15.32:0/2623904412 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0ab600 con 0x7f866806cdf0 2023-12-20T04:45:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.639+0000 7f86667fc640 1 -- 172.21.15.32:0/2623904412 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f865c0a7050 con 0x7f866806cdf0 2023-12-20T04:45:55.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.639+0000 7f86667fc640 1 -- 172.21.15.32:0/2623904412 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f865c0ab980 con 0x7f866806cdf0 2023-12-20T04:45:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 -- 172.21.15.32:0/2623904412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866806cdf0 msgr2=0x7f8668070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 --2- 172.21.15.32:0/2623904412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866806cdf0 0x7f8668070230 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7f86680c5860 tx=0x7f865c01c3b0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 -- 172.21.15.32:0/2623904412 shutdown_connections 2023-12-20T04:45:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 --2- 172.21.15.32:0/2623904412 >> v2:172.21.15.39:3300/0 conn(0x7f8668070770 0x7f86680cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 --2- 172.21.15.32:0/2623904412 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f866806cdf0 0x7f8668070230 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 -- 172.21.15.32:0/2623904412 >> 172.21.15.32:0/2623904412 conn(0x7f866806a510 msgr2=0x7f86680ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 -- 172.21.15.32:0/2623904412 shutdown_connections 2023-12-20T04:45:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 -- 172.21.15.32:0/2623904412 wait complete. 2023-12-20T04:45:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 Processor -- start 2023-12-20T04:45:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 -- start start 2023-12-20T04:45:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8668070770 0x7f8668150150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86681591e0 0x7f86680c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8668148d30 0x7f8668142b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866814a5a0 con 0x7f8668148d30 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f866814a880 con 0x7f8668070770 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f866e125640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f866814ab60 con 0x7f86681591e0 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f8667fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8668148d30 0x7f8668142b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f8667fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8668148d30 0x7f8668142b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42540/0 (socket says 172.21.15.32:42540) 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f8667fff640 1 -- 172.21.15.32:0/749454515 learned_addr learned my addr 172.21.15.32:0/749454515 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f86677fe640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8668070770 0x7f8668150150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f8666ffd640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86681591e0 0x7f86680c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f8667fff640 1 -- 172.21.15.32:0/749454515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86681591e0 msgr2=0x7f86680c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f8667fff640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86681591e0 0x7f86680c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f8667fff640 1 -- 172.21.15.32:0/749454515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8668070770 msgr2=0x7f8668150150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f8667fff640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8668070770 0x7f8668150150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.643+0000 7f8667fff640 1 -- 172.21.15.32:0/749454515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f865c009420 con 0x7f8668148d30 2023-12-20T04:45:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.647+0000 7f8667fff640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8668148d30 0x7f8668142b70 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f866000efc0 tx=0x7f866000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.647+0000 7f864ffff640 1 -- 172.21.15.32:0/749454515 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660014480 con 0x7f8668148d30 2023-12-20T04:45:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.647+0000 7f864ffff640 1 -- 172.21.15.32:0/749454515 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f866000f040 con 0x7f8668148d30 2023-12-20T04:45:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.647+0000 7f864ffff640 1 -- 172.21.15.32:0/749454515 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8660014820 con 0x7f8668148d30 2023-12-20T04:45:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.647+0000 7f866e125640 1 -- 172.21.15.32:0/749454515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f866814ae40 con 0x7f8668148d30 2023-12-20T04:45:55.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.647+0000 7f866e125640 1 -- 172.21.15.32:0/749454515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8668150690 con 0x7f8668148d30 2023-12-20T04:45:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.647+0000 7f866e125640 1 -- 172.21.15.32:0/749454515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f866804e000 con 0x7f8668148d30 2023-12-20T04:45:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.647+0000 7f864ffff640 1 -- 172.21.15.32:0/749454515 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8660014980 con 0x7f8668148d30 2023-12-20T04:45:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.647+0000 7f864ffff640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8654065b80 0x7f8654068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.647+0000 7f864ffff640 1 -- 172.21.15.32:0/749454515 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f8660083fe0 con 0x7f8668148d30 2023-12-20T04:45:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.651+0000 7f86677fe640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8654065b80 0x7f8654068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.651+0000 7f86677fe640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8654065b80 0x7f8654068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8658005890 tx=0x7f8658009040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.651+0000 7f864ffff640 1 -- 172.21.15.32:0/749454515 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8660051730 con 0x7f8668148d30 2023-12-20T04:45:55.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.811+0000 7f866e125640 1 -- 172.21.15.32:0/749454515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f86681437c0 con 0x7f8668148d30 2023-12-20T04:45:55.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f864ffff640 1 -- 172.21.15.32:0/749454515 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v40)=0 v40) v1 ==== 122+0+0 (secure 0 0 0) 0x7f8660054d50 con 0x7f8668148d30 2023-12-20T04:45:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 -- 172.21.15.32:0/749454515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8654065b80 msgr2=0x7f8654068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8654065b80 0x7f8654068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f8658005890 tx=0x7f8658009040 comp rx=0 tx=0).stop 2023-12-20T04:45:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 -- 172.21.15.32:0/749454515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8668148d30 msgr2=0x7f8668142b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8668148d30 0x7f8668142b70 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7f866000efc0 tx=0x7f866000c490 comp rx=0 tx=0).stop 2023-12-20T04:45:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 -- 172.21.15.32:0/749454515 shutdown_connections 2023-12-20T04:45:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8654065b80 0x7f8654068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8668148d30 0x7f8668142b70 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86681591e0 0x7f86680c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 --2- 172.21.15.32:0/749454515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8668070770 0x7f8668150150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:55.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 -- 172.21.15.32:0/749454515 >> 172.21.15.32:0/749454515 conn(0x7f866806a510 msgr2=0x7f86680b0630 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:55.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 -- 172.21.15.32:0/749454515 shutdown_connections 2023-12-20T04:45:55.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:55.815+0000 7f866e125640 1 -- 172.21.15.32:0/749454515 wait complete. 2023-12-20T04:45:55.834 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:45:55.834 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:45:55.890 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:45:55.936 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:45:55.937 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:45:55.937 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:45:55.946 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:45:55.946 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:45:55.946 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:45:55.990 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:45:55.990 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:45:55.990 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:45:55.999 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:45:56.000 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:45:56.000 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:45:56.042 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:45:56.042 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T04:45:56.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:45:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.191+0000 7f220bdf3640 1 Processor -- start 2023-12-20T04:45:56.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.191+0000 7f220bdf3640 1 -- start start 2023-12-20T04:45:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.191+0000 7f220bdf3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22040ae010 0x7f2204071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.191+0000 7f220bdf3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f220406cf10 0x7f22040719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.191+0000 7f220bdf3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22040ae3f0 con 0x7f2204164ef0 2023-12-20T04:45:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.191+0000 7f220bdf3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22040ca4b0 con 0x7f22040ae010 2023-12-20T04:45:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.191+0000 7f220bdf3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2204161800 con 0x7f220406cf10 2023-12-20T04:45:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f2209b8f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22040ae010 0x7f2204071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:56.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f2209b8f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f22040ae010 0x7f2204071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42550/0 (socket says 172.21.15.32:42550) 2023-12-20T04:45:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f2209b8f640 1 -- 172.21.15.32:0/2105377814 learned_addr learned my addr 172.21.15.32:0/2105377814 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f220938e640 1 --2- 172.21.15.32:0/2105377814 >> v2:172.21.15.39:3300/0 conn(0x7f220406cf10 0x7f22040719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f2209b8f640 1 -- 172.21.15.32:0/2105377814 >> v2:172.21.15.39:3300/0 conn(0x7f220406cf10 msgr2=0x7f22040719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f2209b8f640 1 --2- 172.21.15.32:0/2105377814 >> v2:172.21.15.39:3300/0 conn(0x7f220406cf10 0x7f22040719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f2209b8f640 1 -- 172.21.15.32:0/2105377814 >> v1:172.21.15.32:6789/0 conn(0x7f2204164ef0 legacy=0x7f22040701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:56.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f2209b8f640 1 -- 172.21.15.32:0/2105377814 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2204165380 con 0x7f22040ae010 2023-12-20T04:45:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f2209b8f640 1 --2- 172.21.15.32:0/2105377814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22040ae010 0x7f2204071470 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f2204164e50 tx=0x7f220001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a68aa615172986e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f2208b8d640 1 -- 172.21.15.32:0/2105377814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22000ab640 con 0x7f22040ae010 2023-12-20T04:45:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f2208b8d640 1 -- 172.21.15.32:0/2105377814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22000a7070 con 0x7f22040ae010 2023-12-20T04:45:56.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.195+0000 7f2208b8d640 1 -- 172.21.15.32:0/2105377814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22000ab9c0 con 0x7f22040ae010 2023-12-20T04:45:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 -- 172.21.15.32:0/2105377814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22040ae010 msgr2=0x7f2204071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 --2- 172.21.15.32:0/2105377814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22040ae010 0x7f2204071470 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7f2204164e50 tx=0x7f220001c440 comp rx=0 tx=0).stop 2023-12-20T04:45:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 -- 172.21.15.32:0/2105377814 shutdown_connections 2023-12-20T04:45:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 --2- 172.21.15.32:0/2105377814 >> v2:172.21.15.39:3300/0 conn(0x7f220406cf10 0x7f22040719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 --2- 172.21.15.32:0/2105377814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22040ae010 0x7f2204071470 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 -- 172.21.15.32:0/2105377814 >> 172.21.15.32:0/2105377814 conn(0x7f220406a320 msgr2=0x7f22040ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:56.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 -- 172.21.15.32:0/2105377814 shutdown_connections 2023-12-20T04:45:56.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 -- 172.21.15.32:0/2105377814 wait complete. 2023-12-20T04:45:56.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 Processor -- start 2023-12-20T04:45:56.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 -- start start 2023-12-20T04:45:56.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22040ae010 0x7f22041619d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220415fde0 0x7f22041507a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2204163100 0x7f2204163580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204162c70 con 0x7f220415fde0 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2204162060 con 0x7f22040ae010 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220bdf3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2204162340 con 0x7f2204163100 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220938e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220415fde0 0x7f22041507a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220938e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220415fde0 0x7f22041507a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42556/0 (socket says 172.21.15.32:42556) 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220938e640 1 -- 172.21.15.32:0/2027828425 learned_addr learned my addr 172.21.15.32:0/2027828425 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f2209b8f640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22040ae010 0x7f22041619d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220a390640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2204163100 0x7f2204163580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220938e640 1 -- 172.21.15.32:0/2027828425 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2204163100 msgr2=0x7f2204163580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220938e640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2204163100 0x7f2204163580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220938e640 1 -- 172.21.15.32:0/2027828425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22040ae010 msgr2=0x7f22041619d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:56.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220938e640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22040ae010 0x7f22041619d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220938e640 1 -- 172.21.15.32:0/2027828425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2200009480 con 0x7f220415fde0 2023-12-20T04:45:56.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f2209b8f640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22040ae010 0x7f22041619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:56.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.199+0000 7f220a390640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2204163100 0x7f2204163580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:56.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.203+0000 7f220938e640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220415fde0 0x7f22041507a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f21f400d800 tx=0x7f21f400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:56.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.203+0000 7f21fa7fc640 1 -- 172.21.15.32:0/2027828425 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f400b6c0 con 0x7f220415fde0 2023-12-20T04:45:56.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.203+0000 7f21fa7fc640 1 -- 172.21.15.32:0/2027828425 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21f400b820 con 0x7f220415fde0 2023-12-20T04:45:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.203+0000 7f21fa7fc640 1 -- 172.21.15.32:0/2027828425 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21f400bb90 con 0x7f220415fde0 2023-12-20T04:45:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.203+0000 7f220bdf3640 1 -- 172.21.15.32:0/2027828425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2204162620 con 0x7f220415fde0 2023-12-20T04:45:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.203+0000 7f220bdf3640 1 -- 172.21.15.32:0/2027828425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2204151160 con 0x7f220415fde0 2023-12-20T04:45:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.203+0000 7f220bdf3640 1 -- 172.21.15.32:0/2027828425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f220404e000 con 0x7f220415fde0 2023-12-20T04:45:56.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.203+0000 7f21fa7fc640 1 -- 172.21.15.32:0/2027828425 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f21f400bcf0 con 0x7f220415fde0 2023-12-20T04:45:56.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.203+0000 7f21fa7fc640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f21e8065b80 0x7f21e8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:56.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.203+0000 7f21fa7fc640 1 -- 172.21.15.32:0/2027828425 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(114..114 src has 1..114) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f21f4083810 con 0x7f220415fde0 2023-12-20T04:45:56.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.203+0000 7f2209b8f640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f21e8065b80 0x7f21e8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:56.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.207+0000 7f2209b8f640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f21e8065b80 0x7f21e8068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f22000af000 tx=0x7f22000a7000 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:56.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.207+0000 7f21fa7fc640 1 -- 172.21.15.32:0/2027828425 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f21f4051110 con 0x7f220415fde0 2023-12-20T04:45:56.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 -- 172.21.15.32:0/2027828425 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2204160570 con 0x7f220415fde0 2023-12-20T04:45:56.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f21fa7fc640 1 -- 172.21.15.32:0/2027828425 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 48 v48) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f21f4054730 con 0x7f220415fde0 2023-12-20T04:45:56.397 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:56.397 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":48,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7222,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1072056415","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1072056415},{"type":"v1","addr":"172.21.15.39:6835","nonce":1072056415}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7350,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3512326822","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3512326822},{"type":"v1","addr":"172.21.15.32:6836","nonce":3512326822}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7353,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/2163601073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2163601073},{"type":"v1","addr":"172.21.15.32:6837","nonce":2163601073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":42},{"gid":7606,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/221071745","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":221071745},{"type":"v1","addr":"172.21.15.39:6833","nonce":221071745}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":48}],"filesystems":[]} 2023-12-20T04:45:56.397 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 48 2023-12-20T04:45:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 -- 172.21.15.32:0/2027828425 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f21e8065b80 msgr2=0x7f21e8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f21e8065b80 0x7f21e8068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f22000af000 tx=0x7f22000a7000 comp rx=0 tx=0).stop 2023-12-20T04:45:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 -- 172.21.15.32:0/2027828425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220415fde0 msgr2=0x7f22041507a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220415fde0 0x7f22041507a0 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7f21f400d800 tx=0x7f21f400dcc0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 -- 172.21.15.32:0/2027828425 shutdown_connections 2023-12-20T04:45:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f21e8065b80 0x7f21e8068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2204163100 0x7f2204163580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f220415fde0 0x7f22041507a0 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 --2- 172.21.15.32:0/2027828425 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22040ae010 0x7f22041619d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 -- 172.21.15.32:0/2027828425 >> 172.21.15.32:0/2027828425 conn(0x7f220406a320 msgr2=0x7f220415aea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:56.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 -- 172.21.15.32:0/2027828425 shutdown_connections 2023-12-20T04:45:56.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.399+0000 7f220bdf3640 1 -- 172.21.15.32:0/2027828425 wait complete. 2023-12-20T04:45:56.418 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T04:45:56.418 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T04:45:56.418 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T04:45:56.418 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:45:56.418 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T04:45:56.419 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T04:45:56.419 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T04:45:56.419 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:45:56.419 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T04:45:56.419 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T04:45:56.419 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T04:45:56.419 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:45:56.420 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T04:45:56.420 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T04:45:56.420 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T04:45:56.420 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:45:56.432 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T04:45:56.432 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T04:45:56.432 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T04:45:56.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T04:45:56.434 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T04:45:56.434 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T04:45:56.435 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T04:45:56.435 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T04:45:56.480 INFO:tasks.ceph.mds.d:Started 2023-12-20T04:45:56.480 DEBUG:teuthology.parallel:result is None 2023-12-20T04:45:56.481 INFO:tasks.ceph.mds.a:Started 2023-12-20T04:45:56.482 DEBUG:teuthology.parallel:result is None 2023-12-20T04:45:56.482 INFO:tasks.ceph.mds.b:Started 2023-12-20T04:45:56.482 DEBUG:teuthology.parallel:result is None 2023-12-20T04:45:56.483 INFO:tasks.ceph.mds.c:Started 2023-12-20T04:45:56.483 DEBUG:teuthology.parallel:result is None 2023-12-20T04:45:56.484 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T04:45:56.572 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T04:45:56.573 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T04:45:56.573 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T04:45:56.574 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T04:45:56.575 INFO:tasks.ceph.mds.b.smithi039.stderr:2023-12-20T04:45:56.578+0000 7fda9cf0e640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6833/0: (98) Address already in use 2023-12-20T04:45:56.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f5e6c4640 1 Processor -- start 2023-12-20T04:45:56.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f5e6c4640 1 -- start start 2023-12-20T04:45:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f5e6c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0f580aece0 0x7f0f580b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f5e6c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f580c98c0 con 0x7f0f581599e0 2023-12-20T04:45:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f5e6c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0f580c9ba0 con 0x7f0f580b0b30 2023-12-20T04:45:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f5e6c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0f580c9e80 con 0x7f0f580aece0 2023-12-20T04:45:56.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f577fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0f580aece0 0x7f0f580b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f577fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0f580aece0 0x7f0f580b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42592/0 (socket says 172.21.15.32:42592) 2023-12-20T04:45:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f577fe640 1 -- 172.21.15.32:0/1229870091 learned_addr learned my addr 172.21.15.32:0/1229870091 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f577fe640 1 -- 172.21.15.32:0/1229870091 >> v1:172.21.15.39:6789/0 conn(0x7f0f580b0b30 legacy=0x7f0f580701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f577fe640 1 -- 172.21.15.32:0/1229870091 >> v1:172.21.15.32:6789/0 conn(0x7f0f581599e0 legacy=0x7f0f580c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f577fe640 1 -- 172.21.15.32:0/1229870091 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f580ca160 con 0x7f0f580aece0 2023-12-20T04:45:56.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.639+0000 7f0f56ffd640 1 -- 172.21.15.32:0/1229870091 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1789610270 0 0) 0x7f0f580c98c0 con 0x7f0f581599e0 2023-12-20T04:45:56.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.643+0000 7f0f577fe640 1 --2- 172.21.15.32:0/1229870091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f580aece0 0x7f0f580b05f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009870 tx=0x7f0f4c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dee1defc0273a33 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:56.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.643+0000 7f0f56ffd640 1 -- 172.21.15.32:0/1229870091 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0ab650 con 0x7f0f580aece0 2023-12-20T04:45:56.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.643+0000 7f0f56ffd640 1 -- 172.21.15.32:0/1229870091 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f4c0ab7b0 con 0x7f0f580aece0 2023-12-20T04:45:56.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.643+0000 7f0f56ffd640 1 -- 172.21.15.32:0/1229870091 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f4c0abb40 con 0x7f0f580aece0 2023-12-20T04:45:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.643+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/1229870091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f580aece0 msgr2=0x7f0f580b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.643+0000 7f0f5e6c4640 1 --2- 172.21.15.32:0/1229870091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f580aece0 0x7f0f580b05f0 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c009870 tx=0x7f0f4c01c540 comp rx=0 tx=0).stop 2023-12-20T04:45:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.643+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/1229870091 shutdown_connections 2023-12-20T04:45:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.643+0000 7f0f5e6c4640 1 --2- 172.21.15.32:0/1229870091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f580aece0 0x7f0f580b05f0 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.643+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/1229870091 >> 172.21.15.32:0/1229870091 conn(0x7f0f5806a320 msgr2=0x7f0f580ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.643+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/1229870091 shutdown_connections 2023-12-20T04:45:56.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.643+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/1229870091 wait complete. 2023-12-20T04:45:56.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5e6c4640 1 Processor -- start 2023-12-20T04:45:56.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5e6c4640 1 -- start start 2023-12-20T04:45:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5e6c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f580aece0 0x7f0f58157640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5e6c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0f580b0b30 0x7f0f581f8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5e6c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f581599e0 0x7f0f58158850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5e6c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58151910 con 0x7f0f580aece0 2023-12-20T04:45:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5e6c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0f58157b80 con 0x7f0f581599e0 2023-12-20T04:45:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5e6c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0f581f93f0 con 0x7f0f580b0b30 2023-12-20T04:45:56.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f57fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f580aece0 0x7f0f58157640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f57fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f580aece0 0x7f0f58157640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42604/0 (socket says 172.21.15.32:42604) 2023-12-20T04:45:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f57fff640 1 -- 172.21.15.32:0/3691741046 learned_addr learned my addr 172.21.15.32:0/3691741046 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5cc61640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f581599e0 0x7f0f58158850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f577fe640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0f580b0b30 0x7f0f581f8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f57fff640 1 -- 172.21.15.32:0/3691741046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0f580b0b30 msgr2=0x7f0f581f8b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f57fff640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0f580b0b30 0x7f0f581f8b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f57fff640 1 -- 172.21.15.32:0/3691741046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f581599e0 msgr2=0x7f0f58158850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f57fff640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f581599e0 0x7f0f58158850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f57fff640 1 -- 172.21.15.32:0/3691741046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0f4c0094d0 con 0x7f0f580aece0 2023-12-20T04:45:56.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f57fff640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f580aece0 0x7f0f58157640 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c3c0 tx=0x7f0f4800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:56.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f54ff9640 1 -- 172.21.15.32:0/3691741046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480054e0 con 0x7f0f580aece0 2023-12-20T04:45:56.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f54ff9640 1 -- 172.21.15.32:0/3691741046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0f48003600 con 0x7f0f580aece0 2023-12-20T04:45:56.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f54ff9640 1 -- 172.21.15.32:0/3691741046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0f480039b0 con 0x7f0f580aece0 2023-12-20T04:45:56.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/3691741046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0f581f96d0 con 0x7f0f580aece0 2023-12-20T04:45:56.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/3691741046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0f58151530 con 0x7f0f580aece0 2023-12-20T04:45:56.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.647+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/3691741046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0f58151a50 con 0x7f0f580aece0 2023-12-20T04:45:56.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.651+0000 7f0f54ff9640 1 -- 172.21.15.32:0/3691741046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0f48003b10 con 0x7f0f580aece0 2023-12-20T04:45:56.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.651+0000 7f0f54ff9640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0f44065b80 0x7f0f44068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:56.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.651+0000 7f0f54ff9640 1 -- 172.21.15.32:0/3691741046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(118..118 src has 1..118) v4 ==== 5631+0+0 (secure 0 0 0) 0x7f0f48082760 con 0x7f0f580aece0 2023-12-20T04:45:56.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.651+0000 7f0f577fe640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0f44065b80 0x7f0f44068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:56.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.651+0000 7f0f577fe640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0f44065b80 0x7f0f44068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0af000 tx=0x7f0f4c0b6040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:56.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.655+0000 7f0f54ff9640 1 -- 172.21.15.32:0/3691741046 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0f58151a50 con 0x7f0f580aece0 2023-12-20T04:45:56.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:56.823+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/3691741046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f0f58151a50 con 0x7f0f580aece0 2023-12-20T04:45:57.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.619+0000 7f0f54ff9640 1 -- 172.21.15.32:0/3691741046 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v119) v1 ==== 97+0+0 (secure 0 0 0) 0x7f0f58151a50 con 0x7f0f580aece0 2023-12-20T04:45:57.618 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T04:45:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/3691741046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0f44065b80 msgr2=0x7f0f44068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0f44065b80 0x7f0f44068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0f4c0af000 tx=0x7f0f4c0b6040 comp rx=0 tx=0).stop 2023-12-20T04:45:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/3691741046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f580aece0 msgr2=0x7f0f58157640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f580aece0 0x7f0f58157640 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f0f4800c3c0 tx=0x7f0f4800c880 comp rx=0 tx=0).stop 2023-12-20T04:45:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/3691741046 shutdown_connections 2023-12-20T04:45:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0f44065b80 0x7f0f44068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0f581599e0 0x7f0f58158850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0f580b0b30 0x7f0f581f8b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:57.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 --2- 172.21.15.32:0/3691741046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0f580aece0 0x7f0f58157640 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:57.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/3691741046 >> 172.21.15.32:0/3691741046 conn(0x7f0f5806a320 msgr2=0x7f0f580cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:57.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/3691741046 shutdown_connections 2023-12-20T04:45:57.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.623+0000 7f0f5e6c4640 1 -- 172.21.15.32:0/3691741046 wait complete. 2023-12-20T04:45:57.642 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T04:45:57.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabba041640 1 Processor -- start 2023-12-20T04:45:57.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabba041640 1 -- start start 2023-12-20T04:45:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabba041640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabb4071af0 0x7fabb40c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabba041640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fabb4160b30 0x7fabb4160f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabba041640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabb40c9a40 con 0x7fabb415ce60 2023-12-20T04:45:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabba041640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb40c9180 con 0x7fabb4071af0 2023-12-20T04:45:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabba041640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabb40c93f0 con 0x7fabb4160b30 2023-12-20T04:45:57.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabb2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabb4071af0 0x7fabb40c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabb2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabb4071af0 0x7fabb40c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42618/0 (socket says 172.21.15.32:42618) 2023-12-20T04:45:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabb2ffd640 1 -- 172.21.15.32:0/186468520 learned_addr learned my addr 172.21.15.32:0/186468520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabb3fff640 1 --2- 172.21.15.32:0/186468520 >> v2:172.21.15.39:3300/0 conn(0x7fabb4160b30 0x7fabb4160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabb2ffd640 1 -- 172.21.15.32:0/186468520 >> v2:172.21.15.39:3300/0 conn(0x7fabb4160b30 msgr2=0x7fabb4160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabb2ffd640 1 --2- 172.21.15.32:0/186468520 >> v2:172.21.15.39:3300/0 conn(0x7fabb4160b30 0x7fabb4160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabb2ffd640 1 -- 172.21.15.32:0/186468520 >> v1:172.21.15.39:6789/0 conn(0x7fabb415ce60 legacy=0x7fabb40713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:57.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.791+0000 7fabb2ffd640 1 -- 172.21.15.32:0/186468520 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb40c96d0 con 0x7fabb4071af0 2023-12-20T04:45:57.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabb2ffd640 1 --2- 172.21.15.32:0/186468520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb4071af0 0x7fabb40c9ed0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fabb40b48e0 tx=0x7faba801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c113f3d705b2be8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:57.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabb27fc640 1 -- 172.21.15.32:0/186468520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80ab610 con 0x7fabb4071af0 2023-12-20T04:45:57.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabb27fc640 1 -- 172.21.15.32:0/186468520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faba80ab770 con 0x7fabb4071af0 2023-12-20T04:45:57.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabb27fc640 1 -- 172.21.15.32:0/186468520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faba80abb00 con 0x7fabb4071af0 2023-12-20T04:45:57.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabba041640 1 -- 172.21.15.32:0/186468520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb4071af0 msgr2=0x7fabb40c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:57.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabba041640 1 --2- 172.21.15.32:0/186468520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb4071af0 0x7fabb40c9ed0 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7fabb40b48e0 tx=0x7faba801c500 comp rx=0 tx=0).stop 2023-12-20T04:45:57.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabba041640 1 -- 172.21.15.32:0/186468520 shutdown_connections 2023-12-20T04:45:57.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabba041640 1 --2- 172.21.15.32:0/186468520 >> v2:172.21.15.39:3300/0 conn(0x7fabb4160b30 0x7fabb4160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:57.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabba041640 1 --2- 172.21.15.32:0/186468520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb4071af0 0x7fabb40c9ed0 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:57.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabba041640 1 -- 172.21.15.32:0/186468520 >> 172.21.15.32:0/186468520 conn(0x7fabb406a320 msgr2=0x7fabb40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:57.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabba041640 1 -- 172.21.15.32:0/186468520 shutdown_connections 2023-12-20T04:45:57.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.795+0000 7fabba041640 1 -- 172.21.15.32:0/186468520 wait complete. 2023-12-20T04:45:57.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabba041640 1 Processor -- start 2023-12-20T04:45:57.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabba041640 1 -- start start 2023-12-20T04:45:57.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabba041640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabb4071af0 0x7fabb40c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabba041640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb415ce60 0x7fabb40c2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabba041640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb4160b30 0x7fabb4145a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabba041640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40c81f0 con 0x7fabb4160b30 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabba041640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40c4300 con 0x7fabb415ce60 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabba041640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabb40c5d70 con 0x7fabb4071af0 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabb3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb4160b30 0x7fabb4145a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabb3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb4160b30 0x7fabb4145a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42620/0 (socket says 172.21.15.32:42620) 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabb3fff640 1 -- 172.21.15.32:0/2894600133 learned_addr learned my addr 172.21.15.32:0/2894600133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabb2ffd640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb415ce60 0x7fabb40c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabb37fe640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabb4071af0 0x7fabb40c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabb3fff640 1 -- 172.21.15.32:0/2894600133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabb4071af0 msgr2=0x7fabb40c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:57.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabb3fff640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabb4071af0 0x7fabb40c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabb3fff640 1 -- 172.21.15.32:0/2894600133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb415ce60 msgr2=0x7fabb40c2050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabb3fff640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb415ce60 0x7fabb40c2050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabb3fff640 1 -- 172.21.15.32:0/2894600133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faba80094d0 con 0x7fabb4160b30 2023-12-20T04:45:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabb3fff640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb4160b30 0x7fabb4145a30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fabac007e90 tx=0x7fabac00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fab9bfff640 1 -- 172.21.15.32:0/2894600133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac014420 con 0x7fabb4160b30 2023-12-20T04:45:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fab9bfff640 1 -- 172.21.15.32:0/2894600133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabac014580 con 0x7fabb4160b30 2023-12-20T04:45:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabba041640 1 -- 172.21.15.32:0/2894600133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabb40c8440 con 0x7fabb4160b30 2023-12-20T04:45:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fab9bfff640 1 -- 172.21.15.32:0/2894600133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabac014930 con 0x7fabb4160b30 2023-12-20T04:45:57.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.799+0000 7fabba041640 1 -- 172.21.15.32:0/2894600133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabb40c3f90 con 0x7fabb4160b30 2023-12-20T04:45:57.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.803+0000 7fabba041640 1 -- 172.21.15.32:0/2894600133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabb40c4470 con 0x7fabb4160b30 2023-12-20T04:45:57.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.803+0000 7fab9bfff640 1 -- 172.21.15.32:0/2894600133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fabac014a90 con 0x7fabb4160b30 2023-12-20T04:45:57.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.803+0000 7fab9bfff640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faba0065b80 0x7faba0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:57.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.803+0000 7fab9bfff640 1 -- 172.21.15.32:0/2894600133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4857+0+0 (secure 0 0 0) 0x7fabac083810 con 0x7fabb4160b30 2023-12-20T04:45:57.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.803+0000 7fabb37fe640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faba0065b80 0x7faba0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:57.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.803+0000 7fabb37fe640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faba0065b80 0x7faba0068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faba4000fc0 tx=0x7faba4009040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:57.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:57.807+0000 7fab9bfff640 1 -- 172.21.15.32:0/2894600133 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fabac051420 con 0x7fabb4160b30 2023-12-20T04:45:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.023+0000 7fabba041640 1 -- 172.21.15.32:0/2894600133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7fabb404e000 con 0x7fabb4160b30 2023-12-20T04:45:58.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.023+0000 7fab9bfff640 1 -- 172.21.15.32:0/2894600133 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v20) v1 ==== 80+0+5304 (secure 0 0 0) 0x7fabac054a40 con 0x7fabb4160b30 2023-12-20T04:45:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:58.021 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:45:58.021 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T04:45:58.022 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:45:58.023 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T04:45:58.024 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.025 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:45:58.026 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.027 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:45:58.028 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T04:45:58.029 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:45:58.030 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T04:45:58.031 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:45:58.032 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T04:45:58.033 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T04:45:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:45:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:45:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:45:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:45:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:58.034 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T04:45:58.034 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:45:58.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 -- 172.21.15.32:0/2894600133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faba0065b80 msgr2=0x7faba0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faba0065b80 0x7faba0068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7faba4000fc0 tx=0x7faba4009040 comp rx=0 tx=0).stop 2023-12-20T04:45:58.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 -- 172.21.15.32:0/2894600133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb4160b30 msgr2=0x7fabb4145a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb4160b30 0x7fabb4145a30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7fabac007e90 tx=0x7fabac00e4b0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 -- 172.21.15.32:0/2894600133 shutdown_connections 2023-12-20T04:45:58.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faba0065b80 0x7faba0068040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabb4160b30 0x7fabb4145a30 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabb415ce60 0x7fabb40c2050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 --2- 172.21.15.32:0/2894600133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabb4071af0 0x7fabb40c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 -- 172.21.15.32:0/2894600133 >> 172.21.15.32:0/2894600133 conn(0x7fabb406a320 msgr2=0x7fabb40cd140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:58.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 -- 172.21.15.32:0/2894600133 shutdown_connections 2023-12-20T04:45:58.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.027+0000 7fabba041640 1 -- 172.21.15.32:0/2894600133 wait complete. 2023-12-20T04:45:58.043 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:45:58.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.187+0000 7f75776e5640 1 Processor -- start 2023-12-20T04:45:58.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.187+0000 7f75776e5640 1 -- start start 2023-12-20T04:45:58.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f75776e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 0x7f75700c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f75776e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f75700713e0 0x7f7570071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f75776e5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7570071d80 con 0x7f7570160b30 2023-12-20T04:45:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f75776e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75701616d0 con 0x7f75700713e0 2023-12-20T04:45:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f75776e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75700c9a40 con 0x7f757015ce60 2023-12-20T04:45:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f7575481640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 0x7f75700c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f7574c80640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f75700713e0 0x7f7570071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f7575481640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 0x7f75700c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45580/0 (socket says 172.21.15.32:45580) 2023-12-20T04:45:58.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f7575481640 1 -- 172.21.15.32:0/2320573974 learned_addr learned my addr 172.21.15.32:0/2320573974 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:58.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f7575481640 1 -- 172.21.15.32:0/2320573974 >> v2:172.21.15.39:3300/0 conn(0x7f75700713e0 msgr2=0x7f7570071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:58.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f7575481640 1 --2- 172.21.15.32:0/2320573974 >> v2:172.21.15.39:3300/0 conn(0x7f75700713e0 0x7f7570071840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f7575481640 1 -- 172.21.15.32:0/2320573974 >> v1:172.21.15.39:6789/0 conn(0x7f7570160b30 legacy=0x7f7570160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.191+0000 7f7575481640 1 -- 172.21.15.32:0/2320573974 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75700c9180 con 0x7f757015ce60 2023-12-20T04:45:58.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f7575481640 1 --2- 172.21.15.32:0/2320573974 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 0x7f75700c9ed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f756c006fd0 tx=0x7f756c01c520 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ca47e2f541f16da server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:58.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f7567fff640 1 -- 172.21.15.32:0/2320573974 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0ab610 con 0x7f757015ce60 2023-12-20T04:45:58.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f7567fff640 1 -- 172.21.15.32:0/2320573974 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c0ab770 con 0x7f757015ce60 2023-12-20T04:45:58.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f7567fff640 1 -- 172.21.15.32:0/2320573974 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0abb00 con 0x7f757015ce60 2023-12-20T04:45:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f75776e5640 1 -- 172.21.15.32:0/2320573974 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 msgr2=0x7f75700c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f75776e5640 1 --2- 172.21.15.32:0/2320573974 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 0x7f75700c9ed0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7f756c006fd0 tx=0x7f756c01c520 comp rx=0 tx=0).stop 2023-12-20T04:45:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f75776e5640 1 -- 172.21.15.32:0/2320573974 shutdown_connections 2023-12-20T04:45:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f75776e5640 1 --2- 172.21.15.32:0/2320573974 >> v2:172.21.15.39:3300/0 conn(0x7f75700713e0 0x7f7570071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f75776e5640 1 --2- 172.21.15.32:0/2320573974 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 0x7f75700c9ed0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f75776e5640 1 -- 172.21.15.32:0/2320573974 >> 172.21.15.32:0/2320573974 conn(0x7f757006a320 msgr2=0x7f75700ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:58.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f75776e5640 1 -- 172.21.15.32:0/2320573974 shutdown_connections 2023-12-20T04:45:58.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f75776e5640 1 -- 172.21.15.32:0/2320573974 wait complete. 2023-12-20T04:45:58.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f75776e5640 1 Processor -- start 2023-12-20T04:45:58.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.195+0000 7f75776e5640 1 -- start start 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f75776e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75700713e0 0x7f75700c7b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f75776e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 0x7f75700caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f75776e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7570160b30 0x7f7570159cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f75776e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757015acc0 con 0x7f75700713e0 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f75776e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f757015a9e0 con 0x7f7570160b30 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f75776e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75700c70b0 con 0x7f757015ce60 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7575481640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75700713e0 0x7f75700c7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7575481640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75700713e0 0x7f75700c7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42634/0 (socket says 172.21.15.32:42634) 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7575481640 1 -- 172.21.15.32:0/3700552737 learned_addr learned my addr 172.21.15.32:0/3700552737 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7574c80640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 0x7f75700caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7575c82640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7570160b30 0x7f7570159cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7575481640 1 -- 172.21.15.32:0/3700552737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 msgr2=0x7f75700caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7575481640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 0x7f75700caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7575481640 1 -- 172.21.15.32:0/3700552737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7570160b30 msgr2=0x7f7570159cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7575481640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7570160b30 0x7f7570159cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7575481640 1 -- 172.21.15.32:0/3700552737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f756c009480 con 0x7f75700713e0 2023-12-20T04:45:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7575481640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75700713e0 0x7f75700c7b90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f756c01abd0 tx=0x7f756c0192c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7565ffb640 1 -- 172.21.15.32:0/3700552737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c005ba0 con 0x7f75700713e0 2023-12-20T04:45:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7565ffb640 1 -- 172.21.15.32:0/3700552737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f756c005d00 con 0x7f75700713e0 2023-12-20T04:45:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f75776e5640 1 -- 172.21.15.32:0/3700552737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75700c7390 con 0x7f75700713e0 2023-12-20T04:45:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f7565ffb640 1 -- 172.21.15.32:0/3700552737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f756c0abb00 con 0x7f75700713e0 2023-12-20T04:45:58.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f75776e5640 1 -- 172.21.15.32:0/3700552737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75700cb030 con 0x7f75700713e0 2023-12-20T04:45:58.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.199+0000 7f75776e5640 1 -- 172.21.15.32:0/3700552737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75700c6f10 con 0x7f75700713e0 2023-12-20T04:45:58.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.203+0000 7f7565ffb640 1 -- 172.21.15.32:0/3700552737 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f756c0abc60 con 0x7f75700713e0 2023-12-20T04:45:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.203+0000 7f7565ffb640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7554065b80 0x7f7554068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.203+0000 7f7565ffb640 1 -- 172.21.15.32:0/3700552737 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f756c11d720 con 0x7f75700713e0 2023-12-20T04:45:58.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.203+0000 7f7574c80640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7554065b80 0x7f7554068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.203+0000 7f7574c80640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7554065b80 0x7f7554068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f75700c4d30 tx=0x7f7560008040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:58.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.207+0000 7f7565ffb640 1 -- 172.21.15.32:0/3700552737 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f75700c6f10 con 0x7f75700713e0 2023-12-20T04:45:58.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f75776e5640 1 -- 172.21.15.32:0/3700552737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75700c6f10 con 0x7f75700713e0 2023-12-20T04:45:58.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f7565ffb640 1 -- 172.21.15.32:0/3700552737 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v119) v1 ==== 74+0+12351 (secure 0 0 0) 0x7f75700c6f10 con 0x7f75700713e0 2023-12-20T04:45:58.365 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:58.365 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":119,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:45:57.614594+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":13,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":109,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":105,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f75776e5640 1 -- 172.21.15.32:0/3700552737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7554065b80 msgr2=0x7f7554068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f75776e5640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7554065b80 0x7f7554068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f75700c4d30 tx=0x7f7560008040 comp rx=0 tx=0).stop 2023-12-20T04:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f75776e5640 1 -- 172.21.15.32:0/3700552737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75700713e0 msgr2=0x7f75700c7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f75776e5640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75700713e0 0x7f75700c7b90 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f756c01abd0 tx=0x7f756c0192c0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f75776e5640 1 -- 172.21.15.32:0/3700552737 shutdown_connections 2023-12-20T04:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f75776e5640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7554065b80 0x7f7554068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f75776e5640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7570160b30 0x7f7570159cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f75776e5640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f757015ce60 0x7f75700caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f75776e5640 1 --2- 172.21.15.32:0/3700552737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75700713e0 0x7f75700c7b90 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.367+0000 7f75776e5640 1 -- 172.21.15.32:0/3700552737 >> 172.21.15.32:0/3700552737 conn(0x7f757006a320 msgr2=0x7f7570160fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:58.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.371+0000 7f75776e5640 1 -- 172.21.15.32:0/3700552737 shutdown_connections 2023-12-20T04:45:58.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.371+0000 7f75776e5640 1 -- 172.21.15.32:0/3700552737 wait complete. 2023-12-20T04:45:58.386 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:45:58.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:45:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.535+0000 7f5ecf7e9640 1 Processor -- start 2023-12-20T04:45:58.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.535+0000 7f5ecf7e9640 1 -- start start 2023-12-20T04:45:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecf7e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 0x7f5ec80c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecf7e9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5ec80b0760 0x7f5ec80b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecf7e9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ec815d660 0x7f5ec815daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecf7e9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec80b1100 con 0x7f5ec815d660 2023-12-20T04:45:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecf7e9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5ec80cde20 con 0x7f5ec80b0760 2023-12-20T04:45:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecf7e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec80c8e30 con 0x7f5ec8157480 2023-12-20T04:45:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecdd86640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ec815d660 0x7f5ec815daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecdd86640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5ec815d660 0x7f5ec815daf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42648/0 (socket says 172.21.15.32:42648) 2023-12-20T04:45:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecdd86640 1 -- 172.21.15.32:0/38205083 learned_addr learned my addr 172.21.15.32:0/38205083 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5eccd84640 1 --2- 172.21.15.32:0/38205083 >> v2:172.21.15.39:3300/0 conn(0x7f5ec80b0760 0x7f5ec80b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecd585640 1 --2- 172.21.15.32:0/38205083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 0x7f5ec80c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecdd86640 1 -- 172.21.15.32:0/38205083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 msgr2=0x7f5ec80c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecdd86640 1 --2- 172.21.15.32:0/38205083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 0x7f5ec80c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecdd86640 1 -- 172.21.15.32:0/38205083 >> v2:172.21.15.39:3300/0 conn(0x7f5ec80b0760 msgr2=0x7f5ec80b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecdd86640 1 --2- 172.21.15.32:0/38205083 >> v2:172.21.15.39:3300/0 conn(0x7f5ec80b0760 0x7f5ec80b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecdd86640 1 -- 172.21.15.32:0/38205083 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec80ce070 con 0x7f5ec815d660 2023-12-20T04:45:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5eccd84640 1 --2- 172.21.15.32:0/38205083 >> v2:172.21.15.39:3300/0 conn(0x7f5ec80b0760 0x7f5ec80b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:58.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecd585640 1 --2- 172.21.15.32:0/38205083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 0x7f5ec80c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:58.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.539+0000 7f5ecdd86640 1 --2- 172.21.15.32:0/38205083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815d660 0x7f5ec815daf0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f5ec806e030 tx=0x7f5ec001c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdf80ff159e0005c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ebffff640 1 -- 172.21.15.32:0/38205083 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec00a5790 con 0x7f5ec815d660 2023-12-20T04:45:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ebffff640 1 -- 172.21.15.32:0/38205083 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec00a58f0 con 0x7f5ec815d660 2023-12-20T04:45:58.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ebffff640 1 -- 172.21.15.32:0/38205083 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec00a5c80 con 0x7f5ec815d660 2023-12-20T04:45:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/38205083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815d660 msgr2=0x7f5ec815daf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 --2- 172.21.15.32:0/38205083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815d660 0x7f5ec815daf0 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x7f5ec806e030 tx=0x7f5ec001c910 comp rx=0 tx=0).stop 2023-12-20T04:45:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/38205083 shutdown_connections 2023-12-20T04:45:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 --2- 172.21.15.32:0/38205083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec815d660 0x7f5ec815daf0 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 --2- 172.21.15.32:0/38205083 >> v2:172.21.15.39:3300/0 conn(0x7f5ec80b0760 0x7f5ec80b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 --2- 172.21.15.32:0/38205083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 0x7f5ec80c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/38205083 >> 172.21.15.32:0/38205083 conn(0x7f5ec806a510 msgr2=0x7f5ec80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/38205083 shutdown_connections 2023-12-20T04:45:58.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/38205083 wait complete. 2023-12-20T04:45:58.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 Processor -- start 2023-12-20T04:45:58.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 -- start start 2023-12-20T04:45:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80b0760 0x7f5ec8151030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 0x7f5ec814eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec815d660 0x7f5ec80b2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec8151570 con 0x7f5ec80b0760 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec8154830 con 0x7f5ec815d660 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.543+0000 7f5ecf7e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ec8154a80 con 0x7f5ec8157480 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecd585640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80b0760 0x7f5ec8151030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecd585640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80b0760 0x7f5ec8151030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42652/0 (socket says 172.21.15.32:42652) 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecd585640 1 -- 172.21.15.32:0/1108254150 learned_addr learned my addr 172.21.15.32:0/1108254150 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5eccd84640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 0x7f5ec814eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecdd86640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec815d660 0x7f5ec80b2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecd585640 1 -- 172.21.15.32:0/1108254150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 msgr2=0x7f5ec814eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecd585640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 0x7f5ec814eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecd585640 1 -- 172.21.15.32:0/1108254150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec815d660 msgr2=0x7f5ec80b2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecd585640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec815d660 0x7f5ec80b2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecd585640 1 -- 172.21.15.32:0/1108254150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ec0009b30 con 0x7f5ec80b0760 2023-12-20T04:45:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5eccd84640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 0x7f5ec814eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecd585640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80b0760 0x7f5ec8151030 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400d800 tx=0x7f5ec400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ebdffb640 1 -- 172.21.15.32:0/1108254150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec40096c0 con 0x7f5ec80b0760 2023-12-20T04:45:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ebdffb640 1 -- 172.21.15.32:0/1108254150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ec4009820 con 0x7f5ec80b0760 2023-12-20T04:45:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ebdffb640 1 -- 172.21.15.32:0/1108254150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ec4009bd0 con 0x7f5ec80b0760 2023-12-20T04:45:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/1108254150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ec8154d60 con 0x7f5ec80b0760 2023-12-20T04:45:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/1108254150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ec814f6c0 con 0x7f5ec80b0760 2023-12-20T04:45:58.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/1108254150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ec8150970 con 0x7f5ec80b0760 2023-12-20T04:45:58.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.547+0000 7f5ebdffb640 1 -- 172.21.15.32:0/1108254150 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5ec4009d30 con 0x7f5ec80b0760 2023-12-20T04:45:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.551+0000 7f5ebdffb640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5eac065b80 0x7f5eac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.551+0000 7f5ebdffb640 1 -- 172.21.15.32:0/1108254150 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f5ec4083600 con 0x7f5ec80b0760 2023-12-20T04:45:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.551+0000 7f5eccd84640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5eac065b80 0x7f5eac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.551+0000 7f5eccd84640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5eac065b80 0x7f5eac068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8156a60 tx=0x7f5eb8008040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:58.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.555+0000 7f5ebdffb640 1 -- 172.21.15.32:0/1108254150 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5ec8150970 con 0x7f5ec80b0760 2023-12-20T04:45:58.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.711+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/1108254150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f5ec8150970 con 0x7f5ec80b0760 2023-12-20T04:45:58.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.711+0000 7f5ebdffb640 1 -- 172.21.15.32:0/1108254150 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v119) v1 ==== 93+0+10 (secure 0 0 0) 0x7f5ec8150970 con 0x7f5ec80b0760 2023-12-20T04:45:58.710 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:45:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/1108254150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5eac065b80 msgr2=0x7f5eac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5eac065b80 0x7f5eac068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f5ec8156a60 tx=0x7f5eb8008040 comp rx=0 tx=0).stop 2023-12-20T04:45:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/1108254150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80b0760 msgr2=0x7f5ec8151030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80b0760 0x7f5ec8151030 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7f5ec400d800 tx=0x7f5ec400dcc0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/1108254150 shutdown_connections 2023-12-20T04:45:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5eac065b80 0x7f5eac068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ec815d660 0x7f5ec80b2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ec8157480 0x7f5ec814eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 --2- 172.21.15.32:0/1108254150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ec80b0760 0x7f5ec8151030 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/1108254150 >> 172.21.15.32:0/1108254150 conn(0x7f5ec806a510 msgr2=0x7f5ec815cf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:58.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/1108254150 shutdown_connections 2023-12-20T04:45:58.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.715+0000 7f5ecf7e9640 1 -- 172.21.15.32:0/1108254150 wait complete. 2023-12-20T04:45:58.731 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T04:45:58.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.875+0000 7f06330fc640 1 Processor -- start 2023-12-20T04:45:58.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f06330fc640 1 -- start start 2023-12-20T04:45:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f06330fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f062c0af5e0 0x7f062c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f06330fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f062c06ce60 0x7f062c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f06330fc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f062c15bbd0 con 0x7f062c15f110 2023-12-20T04:45:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f06330fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f062c15beb0 con 0x7f062c0af5e0 2023-12-20T04:45:58.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f06330fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062c15e910 con 0x7f062c06ce60 2023-12-20T04:45:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f0630e98640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f062c0af5e0 0x7f062c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f062bfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f062c06ce60 0x7f062c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f0630e98640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f062c0af5e0 0x7f062c0af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:44368/0 (socket says 172.21.15.32:44368) 2023-12-20T04:45:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f0630e98640 1 -- 172.21.15.32:0/1182652084 learned_addr learned my addr 172.21.15.32:0/1182652084 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f0631699640 1 --1- 172.21.15.32:0/1182652084 >> v1:172.21.15.39:6789/0 conn(0x7f062c15f110 0x7f062c15c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:43106/0 (socket says 172.21.15.32:43106) 2023-12-20T04:45:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f062bfff640 1 -- 172.21.15.32:0/1182652084 >> v2:172.21.15.39:3300/0 conn(0x7f062c0af5e0 msgr2=0x7f062c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f062bfff640 1 --2- 172.21.15.32:0/1182652084 >> v2:172.21.15.39:3300/0 conn(0x7f062c0af5e0 0x7f062c0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f062bfff640 1 -- 172.21.15.32:0/1182652084 >> v1:172.21.15.39:6789/0 conn(0x7f062c15f110 legacy=0x7f062c15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.879+0000 7f062bfff640 1 -- 172.21.15.32:0/1182652084 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f062c15ebf0 con 0x7f062c06ce60 2023-12-20T04:45:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f062bfff640 1 --2- 172.21.15.32:0/1182652084 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f062c06ce60 0x7f062c15b280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06180097c0 tx=0x7f061801c480 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3ccf2d321804a9c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:58.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f062b7fe640 1 -- 172.21.15.32:0/1182652084 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06180ac640 con 0x7f062c06ce60 2023-12-20T04:45:58.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f062b7fe640 1 -- 172.21.15.32:0/1182652084 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06180ac7a0 con 0x7f062c06ce60 2023-12-20T04:45:58.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f062b7fe640 1 -- 172.21.15.32:0/1182652084 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06180acb30 con 0x7f062c06ce60 2023-12-20T04:45:58.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f06330fc640 1 -- 172.21.15.32:0/1182652084 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f062c06ce60 msgr2=0x7f062c15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f06330fc640 1 --2- 172.21.15.32:0/1182652084 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f062c06ce60 0x7f062c15b280 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f06180097c0 tx=0x7f061801c480 comp rx=0 tx=0).stop 2023-12-20T04:45:58.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f06330fc640 1 -- 172.21.15.32:0/1182652084 shutdown_connections 2023-12-20T04:45:58.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f06330fc640 1 --2- 172.21.15.32:0/1182652084 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f062c06ce60 0x7f062c15b280 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f06330fc640 1 --2- 172.21.15.32:0/1182652084 >> v2:172.21.15.39:3300/0 conn(0x7f062c0af5e0 0x7f062c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f06330fc640 1 -- 172.21.15.32:0/1182652084 >> 172.21.15.32:0/1182652084 conn(0x7f062c06a320 msgr2=0x7f062c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:58.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f06330fc640 1 -- 172.21.15.32:0/1182652084 shutdown_connections 2023-12-20T04:45:58.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.883+0000 7f06330fc640 1 -- 172.21.15.32:0/1182652084 wait complete. 2023-12-20T04:45:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06330fc640 1 Processor -- start 2023-12-20T04:45:58.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06330fc640 1 -- start start 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06330fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f062c06ce60 0x7f062c15a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06330fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f062c0af5e0 0x7f062c14a2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06330fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f062c15f110 0x7f062c0cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06330fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c159040 con 0x7f062c0af5e0 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06330fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f062c159320 con 0x7f062c15f110 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06330fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f062c159600 con 0x7f062c06ce60 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f062bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f062c0af5e0 0x7f062c14a2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f062bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f062c0af5e0 0x7f062c14a2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37706/0 (socket says 172.21.15.32:37706) 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f0631699640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f062c15f110 0x7f062c0cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f062bfff640 1 -- 172.21.15.32:0/375808795 learned_addr learned my addr 172.21.15.32:0/375808795 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f0630e98640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f062c06ce60 0x7f062c15a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f062bfff640 1 -- 172.21.15.32:0/375808795 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f062c06ce60 msgr2=0x7f062c15a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f062bfff640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f062c06ce60 0x7f062c15a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f062bfff640 1 -- 172.21.15.32:0/375808795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f062c15f110 msgr2=0x7f062c0cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:58.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f062bfff640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f062c15f110 0x7f062c0cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:58.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f062bfff640 1 -- 172.21.15.32:0/375808795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0618009420 con 0x7f062c0af5e0 2023-12-20T04:45:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f062bfff640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f062c0af5e0 0x7f062c14a2d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f061801a0b0 tx=0x7f0618019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06297fa640 1 -- 172.21.15.32:0/375808795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06180b5600 con 0x7f062c0af5e0 2023-12-20T04:45:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06297fa640 1 -- 172.21.15.32:0/375808795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f06180b5760 con 0x7f062c0af5e0 2023-12-20T04:45:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06330fc640 1 -- 172.21.15.32:0/375808795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f062c1598e0 con 0x7f062c0af5e0 2023-12-20T04:45:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06297fa640 1 -- 172.21.15.32:0/375808795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f06180b5af0 con 0x7f062c0af5e0 2023-12-20T04:45:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.887+0000 7f06330fc640 1 -- 172.21.15.32:0/375808795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f062c15a620 con 0x7f062c0af5e0 2023-12-20T04:45:58.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.891+0000 7f06330fc640 1 -- 172.21.15.32:0/375808795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f062c159e00 con 0x7f062c0af5e0 2023-12-20T04:45:58.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.891+0000 7f06297fa640 1 -- 172.21.15.32:0/375808795 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f06180b3040 con 0x7f062c0af5e0 2023-12-20T04:45:58.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.891+0000 7f06297fa640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0610065b80 0x7f0610068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:58.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.891+0000 7f06297fa640 1 -- 172.21.15.32:0/375808795 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4857+0+0 (secure 0 0 0) 0x7f061811d0b0 con 0x7f062c0af5e0 2023-12-20T04:45:58.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.891+0000 7f0630e98640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0610065b80 0x7f0610068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:58.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.891+0000 7f0630e98640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0610065b80 0x7f0610068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0620006c70 tx=0x7f0620009210 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:58.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:58.895+0000 7f06297fa640 1 -- 172.21.15.32:0/375808795 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f062c159e00 con 0x7f062c0af5e0 2023-12-20T04:45:59.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.055+0000 7f06330fc640 1 -- 172.21.15.32:0/375808795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f062c159e00 con 0x7f062c0af5e0 2023-12-20T04:45:59.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.055+0000 7f06297fa640 1 -- 172.21.15.32:0/375808795 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v119) v1 ==== 84+0+65 (secure 0 0 0) 0x7f062c159e00 con 0x7f062c0af5e0 2023-12-20T04:45:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:45:59.053 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:45:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:45:59.053 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T04:45:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T04:45:59.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:45:59.054 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:45:59.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 -- 172.21.15.32:0/375808795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0610065b80 msgr2=0x7f0610068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0610065b80 0x7f0610068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f0620006c70 tx=0x7f0620009210 comp rx=0 tx=0).stop 2023-12-20T04:45:59.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 -- 172.21.15.32:0/375808795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f062c0af5e0 msgr2=0x7f062c14a2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f062c0af5e0 0x7f062c14a2d0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7f061801a0b0 tx=0x7f0618019220 comp rx=0 tx=0).stop 2023-12-20T04:45:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 -- 172.21.15.32:0/375808795 shutdown_connections 2023-12-20T04:45:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0610065b80 0x7f0610068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f062c15f110 0x7f062c0cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f062c0af5e0 0x7f062c14a2d0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 --2- 172.21.15.32:0/375808795 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f062c06ce60 0x7f062c15a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 -- 172.21.15.32:0/375808795 >> 172.21.15.32:0/375808795 conn(0x7f062c06a320 msgr2=0x7f062c15c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 -- 172.21.15.32:0/375808795 shutdown_connections 2023-12-20T04:45:59.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.059+0000 7f06330fc640 1 -- 172.21.15.32:0/375808795 wait complete. 2023-12-20T04:45:59.074 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T04:45:59.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T04:45:59.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.223+0000 7ff44ce66640 1 Processor -- start 2023-12-20T04:45:59.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.223+0000 7ff44ce66640 1 -- start start 2023-12-20T04:45:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.227+0000 7ff44ce66640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff448158520 0x7ff44806dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:59.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.227+0000 7ff44ce66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4480ca7f0 con 0x7ff4480ca050 2023-12-20T04:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.227+0000 7ff44ce66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff4480caad0 con 0x7ff448070230 2023-12-20T04:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.227+0000 7ff44ce66640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff4480cadb0 con 0x7ff448158520 2023-12-20T04:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.227+0000 7ff44659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff448158520 0x7ff44806dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.227+0000 7ff43ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff4480ca050 0x7ff44815ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36074/0 (socket says 172.21.15.32:36074) 2023-12-20T04:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.227+0000 7ff43ffff640 1 -- 172.21.15.32:0/2237598000 learned_addr learned my addr 172.21.15.32:0/2237598000 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.227+0000 7ff44659c640 1 -- 172.21.15.32:0/2237598000 >> v1:172.21.15.39:6789/0 conn(0x7ff448070230 legacy=0x7ff44815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.227+0000 7ff44659c640 1 -- 172.21.15.32:0/2237598000 >> v1:172.21.15.32:6789/0 conn(0x7ff4480ca050 legacy=0x7ff44815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.227+0000 7ff44659c640 1 -- 172.21.15.32:0/2237598000 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff44815e780 con 0x7ff448158520 2023-12-20T04:45:59.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44659c640 1 --2- 172.21.15.32:0/2237598000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff448158520 0x7ff44806dcb0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff4380097c0 tx=0x7ff43801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6f69ecd168f13e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:59.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff445d9b640 1 -- 172.21.15.32:0/2237598000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380ac640 con 0x7ff448158520 2023-12-20T04:45:59.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff445d9b640 1 -- 172.21.15.32:0/2237598000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4380ac7a0 con 0x7ff448158520 2023-12-20T04:45:59.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff445d9b640 1 -- 172.21.15.32:0/2237598000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380acb30 con 0x7ff448158520 2023-12-20T04:45:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 -- 172.21.15.32:0/2237598000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff448158520 msgr2=0x7ff44806dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 --2- 172.21.15.32:0/2237598000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff448158520 0x7ff44806dcb0 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7ff4380097c0 tx=0x7ff43801c3c0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 -- 172.21.15.32:0/2237598000 shutdown_connections 2023-12-20T04:45:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 --2- 172.21.15.32:0/2237598000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff448158520 0x7ff44806dcb0 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 -- 172.21.15.32:0/2237598000 >> 172.21.15.32:0/2237598000 conn(0x7ff44806a510 msgr2=0x7ff4480ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 -- 172.21.15.32:0/2237598000 shutdown_connections 2023-12-20T04:45:59.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 -- 172.21.15.32:0/2237598000 wait complete. 2023-12-20T04:45:59.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 Processor -- start 2023-12-20T04:45:59.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 -- start start 2023-12-20T04:45:59.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff448070230 0x7ff4481fdb00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:59.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4480ca050 0x7ff44815a1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:59.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.231+0000 7ff44ce66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff448158520 0x7ff4480aff30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:59.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44ce66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff4480c9880 con 0x7ff448070230 2023-12-20T04:45:59.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44ce66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff448159160 con 0x7ff4480ca050 2023-12-20T04:45:59.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44ce66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff448159420 con 0x7ff448158520 2023-12-20T04:45:59.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff448070230 0x7ff4481fdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff448070230 0x7ff4481fdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37730/0 (socket says 172.21.15.32:37730) 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44659c640 1 -- 172.21.15.32:0/2511749201 learned_addr learned my addr 172.21.15.32:0/2511749201 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff446d9d640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff448158520 0x7ff4480aff30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff43ffff640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4480ca050 0x7ff44815a1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44659c640 1 -- 172.21.15.32:0/2511749201 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff448158520 msgr2=0x7ff4480aff30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44659c640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff448158520 0x7ff4480aff30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44659c640 1 -- 172.21.15.32:0/2511749201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4480ca050 msgr2=0x7ff44815a1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44659c640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4480ca050 0x7ff44815a1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44659c640 1 -- 172.21.15.32:0/2511749201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff438009420 con 0x7ff448070230 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff43ffff640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4480ca050 0x7ff44815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44659c640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff448070230 0x7ff4481fdb00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff438018fd0 tx=0x7ff43801ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff43effd640 1 -- 172.21.15.32:0/2511749201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380b7860 con 0x7ff448070230 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff43effd640 1 -- 172.21.15.32:0/2511749201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff4380b79e0 con 0x7ff448070230 2023-12-20T04:45:59.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff43effd640 1 -- 172.21.15.32:0/2511749201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff4380b7d90 con 0x7ff448070230 2023-12-20T04:45:59.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44ce66640 1 -- 172.21.15.32:0/2511749201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff448070bd0 con 0x7ff448070230 2023-12-20T04:45:59.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44ce66640 1 -- 172.21.15.32:0/2511749201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff4481599e0 con 0x7ff448070230 2023-12-20T04:45:59.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff44ce66640 1 -- 172.21.15.32:0/2511749201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff44804e000 con 0x7ff448070230 2023-12-20T04:45:59.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.235+0000 7ff43effd640 1 -- 172.21.15.32:0/2511749201 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff4380b3070 con 0x7ff448070230 2023-12-20T04:45:59.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.239+0000 7ff43effd640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff428065b80 0x7ff428068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:59.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.239+0000 7ff43effd640 1 -- 172.21.15.32:0/2511749201 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(119..119 src has 1..119) v4 ==== 4857+0+0 (secure 0 0 0) 0x7ff43811dac0 con 0x7ff448070230 2023-12-20T04:45:59.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.239+0000 7ff43ffff640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff428065b80 0x7ff428068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:59.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.239+0000 7ff43ffff640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff428065b80 0x7ff428068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff430007fd0 tx=0x7ff430009040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:59.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.243+0000 7ff43effd640 1 -- 172.21.15.32:0/2511749201 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff4380eb6b0 con 0x7ff448070230 2023-12-20T04:45:59.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.403+0000 7ff44ce66640 1 -- 172.21.15.32:0/2511749201 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7ff4480b4490 con 0x7ff448070230 2023-12-20T04:45:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.483+0000 7ff43effd640 1 -- 172.21.15.32:0/2511749201 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v120) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff4380b10b0 con 0x7ff448070230 2023-12-20T04:45:59.480 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T04:45:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.483+0000 7ff44ce66640 1 -- 172.21.15.32:0/2511749201 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff428065b80 msgr2=0x7ff428068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.483+0000 7ff44ce66640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff428065b80 0x7ff428068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7ff430007fd0 tx=0x7ff430009040 comp rx=0 tx=0).stop 2023-12-20T04:45:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.483+0000 7ff44ce66640 1 -- 172.21.15.32:0/2511749201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff448070230 msgr2=0x7ff4481fdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.483+0000 7ff44ce66640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff448070230 0x7ff4481fdb00 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7ff438018fd0 tx=0x7ff43801ac00 comp rx=0 tx=0).stop 2023-12-20T04:45:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.487+0000 7ff44ce66640 1 -- 172.21.15.32:0/2511749201 shutdown_connections 2023-12-20T04:45:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.487+0000 7ff44ce66640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff428065b80 0x7ff428068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.487+0000 7ff44ce66640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff448158520 0x7ff4480aff30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.487+0000 7ff44ce66640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff4480ca050 0x7ff44815a1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.487+0000 7ff44ce66640 1 --2- 172.21.15.32:0/2511749201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff448070230 0x7ff4481fdb00 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.487+0000 7ff44ce66640 1 -- 172.21.15.32:0/2511749201 >> 172.21.15.32:0/2511749201 conn(0x7ff44806a510 msgr2=0x7ff4480c38d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.487+0000 7ff44ce66640 1 -- 172.21.15.32:0/2511749201 shutdown_connections 2023-12-20T04:45:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.487+0000 7ff44ce66640 1 -- 172.21.15.32:0/2511749201 wait complete. 2023-12-20T04:45:59.506 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T04:45:59.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c3978f640 1 Processor -- start 2023-12-20T04:45:59.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c3978f640 1 -- start start 2023-12-20T04:45:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c3978f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c34072040 0x7f1c340cafd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c3978f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c340cb510 0x7f1c340c8c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c3978f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c340cc190 con 0x7f1c34159940 2023-12-20T04:45:59.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c3978f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1c340cb9a0 con 0x7f1c34072040 2023-12-20T04:45:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c3978f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c34159d20 con 0x7f1c340cb510 2023-12-20T04:45:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c2bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c34072040 0x7f1c340cafd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c32ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1c34159940 0x7f1c340cdc60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36090/0 (socket says 172.21.15.32:36090) 2023-12-20T04:45:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c2bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1c34072040 0x7f1c340cafd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37734/0 (socket says 172.21.15.32:37734) 2023-12-20T04:45:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c32ffd640 1 -- 172.21.15.32:0/2315067310 learned_addr learned my addr 172.21.15.32:0/2315067310 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:45:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c337fe640 1 --2- 172.21.15.32:0/2315067310 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c340cb510 0x7f1c340c8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c2bfff640 1 -- 172.21.15.32:0/2315067310 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c340cb510 msgr2=0x7f1c340c8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:45:59.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c2bfff640 1 --2- 172.21.15.32:0/2315067310 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c340cb510 0x7f1c340c8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c2bfff640 1 -- 172.21.15.32:0/2315067310 >> v1:172.21.15.32:6789/0 conn(0x7f1c34159940 legacy=0x7f1c340cdc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.659+0000 7f1c2bfff640 1 -- 172.21.15.32:0/2315067310 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c340cc3e0 con 0x7f1c34072040 2023-12-20T04:45:59.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c2bfff640 1 --2- 172.21.15.32:0/2315067310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c34072040 0x7f1c340cafd0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097c0 tx=0x7f1c1c01c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7384ca8dd80af5c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:59.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c327fc640 1 -- 172.21.15.32:0/2315067310 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0ab600 con 0x7f1c34072040 2023-12-20T04:45:59.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c327fc640 1 -- 172.21.15.32:0/2315067310 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0a7050 con 0x7f1c34072040 2023-12-20T04:45:59.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c327fc640 1 -- 172.21.15.32:0/2315067310 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0ab980 con 0x7f1c34072040 2023-12-20T04:45:59.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c3978f640 1 -- 172.21.15.32:0/2315067310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c34072040 msgr2=0x7f1c340cafd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c3978f640 1 --2- 172.21.15.32:0/2315067310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c34072040 0x7f1c340cafd0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c0097c0 tx=0x7f1c1c01c3f0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c3978f640 1 -- 172.21.15.32:0/2315067310 shutdown_connections 2023-12-20T04:45:59.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c3978f640 1 --2- 172.21.15.32:0/2315067310 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c34072040 0x7f1c340cafd0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c3978f640 1 --2- 172.21.15.32:0/2315067310 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c340cb510 0x7f1c340c8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c3978f640 1 -- 172.21.15.32:0/2315067310 >> 172.21.15.32:0/2315067310 conn(0x7f1c3406bfe0 msgr2=0x7f1c3406c3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:45:59.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c3978f640 1 -- 172.21.15.32:0/2315067310 shutdown_connections 2023-12-20T04:45:59.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.663+0000 7f1c3978f640 1 -- 172.21.15.32:0/2315067310 wait complete. 2023-12-20T04:45:59.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c3978f640 1 Processor -- start 2023-12-20T04:45:59.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c3978f640 1 -- start start 2023-12-20T04:45:59.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c3978f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c34072040 0x7f1c340c7080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:59.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c3978f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c340cb510 0x7f1c34146b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:59.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c3978f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c34159940 0x7f1c3415a890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:59.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c3978f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c340c7690 con 0x7f1c340cb510 2023-12-20T04:45:59.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c3978f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c340c7970 con 0x7f1c34159940 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c3978f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c340c7c50 con 0x7f1c34072040 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c340cb510 0x7f1c34146b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c340cb510 0x7f1c34146b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37746/0 (socket says 172.21.15.32:37746) 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2bfff640 1 -- 172.21.15.32:0/3097855322 learned_addr learned my addr 172.21.15.32:0/3097855322 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c337fe640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c34159940 0x7f1c3415a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c32ffd640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c34072040 0x7f1c340c7080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2bfff640 1 -- 172.21.15.32:0/3097855322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c34072040 msgr2=0x7f1c340c7080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2bfff640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c34072040 0x7f1c340c7080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2bfff640 1 -- 172.21.15.32:0/3097855322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c34159940 msgr2=0x7f1c3415a890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2bfff640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c34159940 0x7f1c3415a890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2bfff640 1 -- 172.21.15.32:0/3097855322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c1c009420 con 0x7f1c340cb510 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c32ffd640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c34072040 0x7f1c340c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:45:59.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2bfff640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c340cb510 0x7f1c34146b60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c019000 tx=0x7f1c1c019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:59.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2b7fe640 1 -- 172.21.15.32:0/3097855322 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0b4600 con 0x7f1c340cb510 2023-12-20T04:45:59.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2b7fe640 1 -- 172.21.15.32:0/3097855322 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c1c0b4760 con 0x7f1c340cb510 2023-12-20T04:45:59.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c2b7fe640 1 -- 172.21.15.32:0/3097855322 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c1c0b4b10 con 0x7f1c340cb510 2023-12-20T04:45:59.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c3978f640 1 -- 172.21.15.32:0/3097855322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c340c7f30 con 0x7f1c340cb510 2023-12-20T04:45:59.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.667+0000 7f1c3978f640 1 -- 172.21.15.32:0/3097855322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c3415b5e0 con 0x7f1c340cb510 2023-12-20T04:45:59.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.671+0000 7f1c3978f640 1 -- 172.21.15.32:0/3097855322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c34074100 con 0x7f1c340cb510 2023-12-20T04:45:59.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.671+0000 7f1c2b7fe640 1 -- 172.21.15.32:0/3097855322 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1c1c0b2040 con 0x7f1c340cb510 2023-12-20T04:45:59.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.671+0000 7f1c2b7fe640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c14065b80 0x7f1c14068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:45:59.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.671+0000 7f1c2b7fe640 1 -- 172.21.15.32:0/3097855322 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(120..120 src has 1..120) v4 ==== 5227+0+0 (secure 0 0 0) 0x7f1c1c11d7a0 con 0x7f1c340cb510 2023-12-20T04:45:59.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.671+0000 7f1c32ffd640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c14065b80 0x7f1c14068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:45:59.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.671+0000 7f1c32ffd640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c14065b80 0x7f1c14068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1c340c5110 tx=0x7f1c24009040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:45:59.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.675+0000 7f1c2b7fe640 1 -- 172.21.15.32:0/3097855322 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1c1c0eb2b0 con 0x7f1c340cb510 2023-12-20T04:45:59.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:45:59.831+0000 7f1c3978f640 1 -- 172.21.15.32:0/3097855322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f1c34153a50 con 0x7f1c340cb510 2023-12-20T04:46:00.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.495+0000 7f1c2b7fe640 1 -- 172.21.15.32:0/3097855322 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v121) v1 ==== 170+0+0 (secure 0 0 0) 0x7f1c1c0ee8d0 con 0x7f1c340cb510 2023-12-20T04:46:00.491 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T04:46:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.507+0000 7f1c3978f640 1 -- 172.21.15.32:0/3097855322 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c14065b80 msgr2=0x7f1c14068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.507+0000 7f1c3978f640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c14065b80 0x7f1c14068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f1c340c5110 tx=0x7f1c24009040 comp rx=0 tx=0).stop 2023-12-20T04:46:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.507+0000 7f1c3978f640 1 -- 172.21.15.32:0/3097855322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c340cb510 msgr2=0x7f1c34146b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.507+0000 7f1c3978f640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c340cb510 0x7f1c34146b60 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f1c1c019000 tx=0x7f1c1c019190 comp rx=0 tx=0).stop 2023-12-20T04:46:00.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.507+0000 7f1c3978f640 1 -- 172.21.15.32:0/3097855322 shutdown_connections 2023-12-20T04:46:00.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.507+0000 7f1c3978f640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1c14065b80 0x7f1c14068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.507+0000 7f1c3978f640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c34159940 0x7f1c3415a890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.507+0000 7f1c3978f640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c340cb510 0x7f1c34146b60 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.507+0000 7f1c3978f640 1 --2- 172.21.15.32:0/3097855322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c34072040 0x7f1c340c7080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.507+0000 7f1c3978f640 1 -- 172.21.15.32:0/3097855322 >> 172.21.15.32:0/3097855322 conn(0x7f1c3406bfe0 msgr2=0x7f1c340730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.511+0000 7f1c3978f640 1 -- 172.21.15.32:0/3097855322 shutdown_connections 2023-12-20T04:46:00.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.511+0000 7f1c3978f640 1 -- 172.21.15.32:0/3097855322 wait complete. 2023-12-20T04:46:00.538 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T04:46:00.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.687+0000 7faf0c673640 1 Processor -- start 2023-12-20T04:46:00.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.687+0000 7faf0c673640 1 -- start start 2023-12-20T04:46:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0c673640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faf040cb050 0x7faf040cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0c673640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf0406d170 0x7faf04161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0c673640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf0406a1f0 con 0x7faf04065d40 2023-12-20T04:46:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0c673640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf04161860 con 0x7faf040cb050 2023-12-20T04:46:00.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0c673640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf0406a440 con 0x7faf0406d170 2023-12-20T04:46:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0a40f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faf040cb050 0x7faf040cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0ac10640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf0406d170 0x7faf04161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf09c0e640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7faf04065d40 0x7faf0406ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:43112/0 (socket says 172.21.15.32:43112) 2023-12-20T04:46:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf09c0e640 1 -- 172.21.15.32:0/3260891415 learned_addr learned my addr 172.21.15.32:0/3260891415 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:46:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0940d640 1 -- 172.21.15.32:0/3260891415 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035373728 0 0) 0x7faf0406a1f0 con 0x7faf04065d40 2023-12-20T04:46:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0a40f640 1 -- 172.21.15.32:0/3260891415 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf0406d170 msgr2=0x7faf04161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0a40f640 1 --2- 172.21.15.32:0/3260891415 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf0406d170 0x7faf04161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0a40f640 1 -- 172.21.15.32:0/3260891415 >> v1:172.21.15.39:6789/0 conn(0x7faf04065d40 legacy=0x7faf0406ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:00.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.691+0000 7faf0a40f640 1 -- 172.21.15.32:0/3260891415 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf0406a720 con 0x7faf040cb050 2023-12-20T04:46:00.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0a40f640 1 --2- 172.21.15.32:0/3260891415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf040cb050 0x7faf040cb430 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faf000097c0 tx=0x7faf0001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7d0155a551383acf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:00.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0940d640 1 -- 172.21.15.32:0/3260891415 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000ab650 con 0x7faf040cb050 2023-12-20T04:46:00.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0940d640 1 -- 172.21.15.32:0/3260891415 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf000ab7b0 con 0x7faf040cb050 2023-12-20T04:46:00.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0940d640 1 -- 172.21.15.32:0/3260891415 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000abb40 con 0x7faf040cb050 2023-12-20T04:46:00.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0c673640 1 -- 172.21.15.32:0/3260891415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf040cb050 msgr2=0x7faf040cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:00.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0c673640 1 --2- 172.21.15.32:0/3260891415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf040cb050 0x7faf040cb430 secure :-1 s=READY pgs=511 cs=0 l=1 rev1=1 crypto rx=0x7faf000097c0 tx=0x7faf0001c450 comp rx=0 tx=0).stop 2023-12-20T04:46:00.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0c673640 1 -- 172.21.15.32:0/3260891415 shutdown_connections 2023-12-20T04:46:00.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0c673640 1 --2- 172.21.15.32:0/3260891415 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf0406d170 0x7faf04161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0c673640 1 --2- 172.21.15.32:0/3260891415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf040cb050 0x7faf040cb430 unknown :-1 s=CLOSED pgs=511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0c673640 1 -- 172.21.15.32:0/3260891415 >> 172.21.15.32:0/3260891415 conn(0x7faf040cf8d0 msgr2=0x7faf040cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:00.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0c673640 1 -- 172.21.15.32:0/3260891415 shutdown_connections 2023-12-20T04:46:00.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0c673640 1 -- 172.21.15.32:0/3260891415 wait complete. 2023-12-20T04:46:00.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0c673640 1 Processor -- start 2023-12-20T04:46:00.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.695+0000 7faf0c673640 1 -- start start 2023-12-20T04:46:00.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0c673640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf04065d40 0x7faf040c70c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:00.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0c673640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0406d170 0x7faf04157ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:00.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0c673640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf040cb050 0x7faf0415b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:00.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0c673640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf041574c0 con 0x7faf04065d40 2023-12-20T04:46:00.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0c673640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf041583f0 con 0x7faf0406d170 2023-12-20T04:46:00.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0c673640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf04156b60 con 0x7faf040cb050 2023-12-20T04:46:00.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0a40f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf04065d40 0x7faf040c70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:00.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0a40f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf04065d40 0x7faf040c70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37760/0 (socket says 172.21.15.32:37760) 2023-12-20T04:46:00.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0a40f640 1 -- 172.21.15.32:0/3576293526 learned_addr learned my addr 172.21.15.32:0/3576293526 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0ac10640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf040cb050 0x7faf0415b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf09c0e640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0406d170 0x7faf04157ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0a40f640 1 -- 172.21.15.32:0/3576293526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf040cb050 msgr2=0x7faf0415b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0a40f640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf040cb050 0x7faf0415b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0a40f640 1 -- 172.21.15.32:0/3576293526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0406d170 msgr2=0x7faf04157ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0a40f640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0406d170 0x7faf04157ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0a40f640 1 -- 172.21.15.32:0/3576293526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf00009420 con 0x7faf04065d40 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0a40f640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf04065d40 0x7faf040c70c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7faf0001ab00 tx=0x7faf0001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faefaffd640 1 -- 172.21.15.32:0/3576293526 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000bf610 con 0x7faf04065d40 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faefaffd640 1 -- 172.21.15.32:0/3576293526 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf000bf790 con 0x7faf04065d40 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faefaffd640 1 -- 172.21.15.32:0/3576293526 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf000bfb40 con 0x7faf04065d40 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0c673640 1 -- 172.21.15.32:0/3576293526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf04156e00 con 0x7faf04065d40 2023-12-20T04:46:00.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0c673640 1 -- 172.21.15.32:0/3576293526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf041578e0 con 0x7faf04065d40 2023-12-20T04:46:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.699+0000 7faf0c673640 1 -- 172.21.15.32:0/3576293526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf04157730 con 0x7faf04065d40 2023-12-20T04:46:00.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.703+0000 7faefaffd640 1 -- 172.21.15.32:0/3576293526 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faf000b5610 con 0x7faf04065d40 2023-12-20T04:46:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.703+0000 7faefaffd640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faee8065b80 0x7faee8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.703+0000 7faefaffd640 1 -- 172.21.15.32:0/3576293526 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(121..121 src has 1..121) v4 ==== 5609+0+0 (secure 0 0 0) 0x7faf0011e7e0 con 0x7faf04065d40 2023-12-20T04:46:00.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.703+0000 7faf09c0e640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faee8065b80 0x7faee8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:00.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.703+0000 7faf09c0e640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faee8065b80 0x7faee8068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf040c7600 tx=0x7faef4007df0 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:00.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.707+0000 7faefaffd640 1 -- 172.21.15.32:0/3576293526 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf000ec0f0 con 0x7faf04065d40 2023-12-20T04:46:00.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.895+0000 7faf0c673640 1 -- 172.21.15.32:0/3576293526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7faf0404e000 con 0x7faf04065d40 2023-12-20T04:46:00.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.903+0000 7faefaffd640 1 -- 172.21.15.32:0/3576293526 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 14 and data pool 15 v50) v1 ==== 174+0+0 (secure 0 0 0) 0x7faf000ef710 con 0x7faf04065d40 2023-12-20T04:46:00.904 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 14 and data pool 15 2023-12-20T04:46:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.911+0000 7faf0c673640 1 -- 172.21.15.32:0/3576293526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faee8065b80 msgr2=0x7faee8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.911+0000 7faf0c673640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faee8065b80 0x7faee8068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7faf040c7600 tx=0x7faef4007df0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.911+0000 7faf0c673640 1 -- 172.21.15.32:0/3576293526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf04065d40 msgr2=0x7faf040c70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.911+0000 7faf0c673640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf04065d40 0x7faf040c70c0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7faf0001ab00 tx=0x7faf0001ac90 comp rx=0 tx=0).stop 2023-12-20T04:46:00.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.911+0000 7faf0c673640 1 -- 172.21.15.32:0/3576293526 shutdown_connections 2023-12-20T04:46:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.911+0000 7faf0c673640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faee8065b80 0x7faee8068040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.911+0000 7faf0c673640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf040cb050 0x7faf0415b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.911+0000 7faf0c673640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf0406d170 0x7faf04157ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.911+0000 7faf0c673640 1 --2- 172.21.15.32:0/3576293526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf04065d40 0x7faf040c70c0 secure :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7faf0001ab00 tx=0x7faf0001ac90 comp rx=0 tx=0).stop 2023-12-20T04:46:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.911+0000 7faf0c673640 1 -- 172.21.15.32:0/3576293526 >> 172.21.15.32:0/3576293526 conn(0x7faf040cf8d0 msgr2=0x7faf040ccb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.915+0000 7faf0c673640 1 -- 172.21.15.32:0/3576293526 shutdown_connections 2023-12-20T04:46:00.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:00.915+0000 7faf0c673640 1 -- 172.21.15.32:0/3576293526 wait complete. 2023-12-20T04:46:00.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:46:01.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad5dc66640 1 Processor -- start 2023-12-20T04:46:01.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad5dc66640 1 -- start start 2023-12-20T04:46:01.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad5dc66640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fad5815cdb0 0x7fad580d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad5dc66640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad580d2740 0x7fad580cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad5dc66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad580b6ca0 con 0x7fad580d1120 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad5dc66640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad5815d190 con 0x7fad580d2740 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad5dc66640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad580d16a0 con 0x7fad5815cdb0 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad56ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad580d2740 0x7fad580cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad56ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad580d2740 0x7fad580cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37772/0 (socket says 172.21.15.32:37772) 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad56ffd640 1 -- 172.21.15.32:0/4208472470 learned_addr learned my addr 172.21.15.32:0/4208472470 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad577fe640 1 --2- 172.21.15.32:0/4208472470 >> v2:172.21.15.39:3300/0 conn(0x7fad5815cdb0 0x7fad580d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad56ffd640 1 -- 172.21.15.32:0/4208472470 >> v2:172.21.15.39:3300/0 conn(0x7fad5815cdb0 msgr2=0x7fad580d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad56ffd640 1 --2- 172.21.15.32:0/4208472470 >> v2:172.21.15.39:3300/0 conn(0x7fad5815cdb0 0x7fad580d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad56ffd640 1 -- 172.21.15.32:0/4208472470 >> v1:172.21.15.32:6789/0 conn(0x7fad580d1120 legacy=0x7fad580b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad56ffd640 1 -- 172.21.15.32:0/4208472470 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad580d1980 con 0x7fad580d2740 2023-12-20T04:46:01.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.099+0000 7fad577fe640 1 --2- 172.21.15.32:0/4208472470 >> v2:172.21.15.39:3300/0 conn(0x7fad5815cdb0 0x7fad580d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:46:01.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad56ffd640 1 --2- 172.21.15.32:0/4208472470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d2740 0x7fad580cf570 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fad580667b0 tx=0x7fad4c01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a5353e7c0075c50 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:01.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad567fc640 1 -- 172.21.15.32:0/4208472470 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0a89f0 con 0x7fad580d2740 2023-12-20T04:46:01.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad567fc640 1 -- 172.21.15.32:0/4208472470 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4c0a8b50 con 0x7fad580d2740 2023-12-20T04:46:01.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad567fc640 1 -- 172.21.15.32:0/4208472470 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0b0440 con 0x7fad580d2740 2023-12-20T04:46:01.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad5dc66640 1 -- 172.21.15.32:0/4208472470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d2740 msgr2=0x7fad580cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad5dc66640 1 --2- 172.21.15.32:0/4208472470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d2740 0x7fad580cf570 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7fad580667b0 tx=0x7fad4c01ecf0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad5dc66640 1 -- 172.21.15.32:0/4208472470 shutdown_connections 2023-12-20T04:46:01.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad5dc66640 1 --2- 172.21.15.32:0/4208472470 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d2740 0x7fad580cf570 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad5dc66640 1 --2- 172.21.15.32:0/4208472470 >> v2:172.21.15.39:3300/0 conn(0x7fad5815cdb0 0x7fad580d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad5dc66640 1 -- 172.21.15.32:0/4208472470 >> 172.21.15.32:0/4208472470 conn(0x7fad58065ee0 msgr2=0x7fad5806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad5dc66640 1 -- 172.21.15.32:0/4208472470 shutdown_connections 2023-12-20T04:46:01.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad5dc66640 1 -- 172.21.15.32:0/4208472470 wait complete. 2023-12-20T04:46:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.103+0000 7fad5dc66640 1 Processor -- start 2023-12-20T04:46:01.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad5dc66640 1 -- start start 2023-12-20T04:46:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad5dc66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad580d1120 0x7fad5814db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad5dc66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d2740 0x7fad5815e0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad5dc66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad5815cdb0 0x7fad5815e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad5dc66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad580cb0c0 con 0x7fad580d2740 2023-12-20T04:46:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad5dc66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad58151cc0 con 0x7fad5815cdb0 2023-12-20T04:46:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad5dc66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad580cad20 con 0x7fad580d1120 2023-12-20T04:46:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad56ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d2740 0x7fad5815e0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad56ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d2740 0x7fad5815e0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37788/0 (socket says 172.21.15.32:37788) 2023-12-20T04:46:01.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad56ffd640 1 -- 172.21.15.32:0/3336313756 learned_addr learned my addr 172.21.15.32:0/3336313756 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad577fe640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad580d1120 0x7fad5814db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad57fff640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad5815cdb0 0x7fad5815e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad56ffd640 1 -- 172.21.15.32:0/3336313756 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad580d1120 msgr2=0x7fad5814db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad56ffd640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad580d1120 0x7fad5814db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad56ffd640 1 -- 172.21.15.32:0/3336313756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad5815cdb0 msgr2=0x7fad5815e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad56ffd640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad5815cdb0 0x7fad5815e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad56ffd640 1 -- 172.21.15.32:0/3336313756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad4c0094d0 con 0x7fad580d2740 2023-12-20T04:46:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad56ffd640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d2740 0x7fad5815e0c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fad4c01b900 tx=0x7fad4c01ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad3ffff640 1 -- 172.21.15.32:0/3336313756 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0ba600 con 0x7fad580d2740 2023-12-20T04:46:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad3ffff640 1 -- 172.21.15.32:0/3336313756 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad4c0ba780 con 0x7fad580d2740 2023-12-20T04:46:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad3ffff640 1 -- 172.21.15.32:0/3336313756 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad4c0bab30 con 0x7fad580d2740 2023-12-20T04:46:01.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad5dc66640 1 -- 172.21.15.32:0/3336313756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad5814e7e0 con 0x7fad580d2740 2023-12-20T04:46:01.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad5dc66640 1 -- 172.21.15.32:0/3336313756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad5814e070 con 0x7fad580d2740 2023-12-20T04:46:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.107+0000 7fad5dc66640 1 -- 172.21.15.32:0/3336313756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad5804e000 con 0x7fad580d2740 2023-12-20T04:46:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.111+0000 7fad3ffff640 1 -- 172.21.15.32:0/3336313756 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fad4c0bac90 con 0x7fad580d2740 2023-12-20T04:46:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.111+0000 7fad3ffff640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fad44065b80 0x7fad44068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.111+0000 7fad3ffff640 1 -- 172.21.15.32:0/3336313756 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fad4c121900 con 0x7fad580d2740 2023-12-20T04:46:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.111+0000 7fad577fe640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fad44065b80 0x7fad44068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.111+0000 7fad577fe640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fad44065b80 0x7fad44068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad580cc8d0 tx=0x7fad48008040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:01.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.115+0000 7fad3ffff640 1 -- 172.21.15.32:0/3336313756 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fad4c0ef1c0 con 0x7fad580d2740 2023-12-20T04:46:01.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.271+0000 7fad5dc66640 1 -- 172.21.15.32:0/3336313756 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fad5806b8d0 con 0x7fad580d2740 2023-12-20T04:46:01.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad3ffff640 1 -- 172.21.15.32:0/3336313756 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16016 (secure 0 0 0) 0x7fad4c0b5020 con 0x7fad580d2740 2023-12-20T04:46:01.272 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:46:01.272 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":122,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:46:00.898641+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:45:59.452651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-20T04:45:59.882509+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 -- 172.21.15.32:0/3336313756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fad44065b80 msgr2=0x7fad44068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fad44065b80 0x7fad44068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fad580cc8d0 tx=0x7fad48008040 comp rx=0 tx=0).stop 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 -- 172.21.15.32:0/3336313756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d2740 msgr2=0x7fad5815e0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d2740 0x7fad5815e0c0 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7fad4c01b900 tx=0x7fad4c01ba90 comp rx=0 tx=0).stop 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 -- 172.21.15.32:0/3336313756 shutdown_connections 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fad44065b80 0x7fad44068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad5815cdb0 0x7fad5815e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad580d2740 0x7fad5815e0c0 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 --2- 172.21.15.32:0/3336313756 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad580d1120 0x7fad5814db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 -- 172.21.15.32:0/3336313756 >> 172.21.15.32:0/3336313756 conn(0x7fad58065ee0 msgr2=0x7fad5815bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 -- 172.21.15.32:0/3336313756 shutdown_connections 2023-12-20T04:46:01.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.275+0000 7fad5dc66640 1 -- 172.21.15.32:0/3336313756 wait complete. 2023-12-20T04:46:01.293 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:46:01.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdae62b6640 1 Processor -- start 2023-12-20T04:46:01.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdae62b6640 1 -- start start 2023-12-20T04:46:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdae62b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdae01599e0 0x7fdae00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdae62b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdae00c90e0 0x7fdae00b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdae62b6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdae00711d0 con 0x7fdae00b0b30 2023-12-20T04:46:01.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdae62b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdae00c98c0 con 0x7fdae00c90e0 2023-12-20T04:46:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdae62b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae00c9ba0 con 0x7fdae01599e0 2023-12-20T04:46:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdadf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdae00c90e0 0x7fdae00b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdadf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdae00c90e0 0x7fdae00b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37794/0 (socket says 172.21.15.32:37794) 2023-12-20T04:46:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdadf7fe640 1 -- 172.21.15.32:0/1675933289 learned_addr learned my addr 172.21.15.32:0/1675933289 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdadffff640 1 --2- 172.21.15.32:0/1675933289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdae01599e0 0x7fdae00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdadf7fe640 1 -- 172.21.15.32:0/1675933289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdae01599e0 msgr2=0x7fdae00aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdadf7fe640 1 --2- 172.21.15.32:0/1675933289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdae01599e0 0x7fdae00aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdadf7fe640 1 -- 172.21.15.32:0/1675933289 >> v1:172.21.15.39:6789/0 conn(0x7fdae00b0b30 legacy=0x7fdae015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:01.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.439+0000 7fdadf7fe640 1 -- 172.21.15.32:0/1675933289 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdae00c9e80 con 0x7fdae00c90e0 2023-12-20T04:46:01.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdadf7fe640 1 --2- 172.21.15.32:0/1675933289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdae00c90e0 0x7fdae00b05f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009870 tx=0x7fdad401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9d5a4856af49a7d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdadeffd640 1 -- 172.21.15.32:0/1675933289 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad40a7070 con 0x7fdae00c90e0 2023-12-20T04:46:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdadeffd640 1 -- 172.21.15.32:0/1675933289 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad40ab430 con 0x7fdae00c90e0 2023-12-20T04:46:01.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdadeffd640 1 -- 172.21.15.32:0/1675933289 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad40ab7c0 con 0x7fdae00c90e0 2023-12-20T04:46:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdae62b6640 1 -- 172.21.15.32:0/1675933289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdae00c90e0 msgr2=0x7fdae00b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdae62b6640 1 --2- 172.21.15.32:0/1675933289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdae00c90e0 0x7fdae00b05f0 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fdad4009870 tx=0x7fdad401c510 comp rx=0 tx=0).stop 2023-12-20T04:46:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdae62b6640 1 -- 172.21.15.32:0/1675933289 shutdown_connections 2023-12-20T04:46:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdae62b6640 1 --2- 172.21.15.32:0/1675933289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdae00c90e0 0x7fdae00b05f0 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdae62b6640 1 --2- 172.21.15.32:0/1675933289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdae01599e0 0x7fdae00aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdae62b6640 1 -- 172.21.15.32:0/1675933289 >> 172.21.15.32:0/1675933289 conn(0x7fdae006a320 msgr2=0x7fdae00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdae62b6640 1 -- 172.21.15.32:0/1675933289 shutdown_connections 2023-12-20T04:46:01.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.443+0000 7fdae62b6640 1 -- 172.21.15.32:0/1675933289 wait complete. 2023-12-20T04:46:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdae62b6640 1 Processor -- start 2023-12-20T04:46:01.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdae62b6640 1 -- start start 2023-12-20T04:46:01.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdae62b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdae00b0b30 0x7fdae014bb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdae62b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdae00c90e0 0x7fdae00cd8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdae62b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdae01599e0 0x7fdae01585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdae62b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae00cd4a0 con 0x7fdae00b0b30 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdae62b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdae01408f0 con 0x7fdae00c90e0 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdae62b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdae014c240 con 0x7fdae01599e0 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdae00b0b30 0x7fdae014bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdae00b0b30 0x7fdae014bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37804/0 (socket says 172.21.15.32:37804) 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadffff640 1 -- 172.21.15.32:0/1171260718 learned_addr learned my addr 172.21.15.32:0/1171260718 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadf7fe640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdae00c90e0 0x7fdae00cd8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdae4853640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdae01599e0 0x7fdae01585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadffff640 1 -- 172.21.15.32:0/1171260718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdae01599e0 msgr2=0x7fdae01585f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadffff640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdae01599e0 0x7fdae01585f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadffff640 1 -- 172.21.15.32:0/1171260718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdae00c90e0 msgr2=0x7fdae00cd8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadffff640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdae00c90e0 0x7fdae00cd8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadffff640 1 -- 172.21.15.32:0/1171260718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdad40094d0 con 0x7fdae00b0b30 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadffff640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdae00b0b30 0x7fdae014bb40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fdad000d7d0 tx=0x7fdad000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:01.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadcff9640 1 -- 172.21.15.32:0/1171260718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad000b6c0 con 0x7fdae00b0b30 2023-12-20T04:46:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadcff9640 1 -- 172.21.15.32:0/1171260718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdad000b820 con 0x7fdae00b0b30 2023-12-20T04:46:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdae62b6640 1 -- 172.21.15.32:0/1171260718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdae013fb30 con 0x7fdae00b0b30 2023-12-20T04:46:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdadcff9640 1 -- 172.21.15.32:0/1171260718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdad000bbd0 con 0x7fdae00b0b30 2023-12-20T04:46:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.447+0000 7fdae62b6640 1 -- 172.21.15.32:0/1171260718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdae01400f0 con 0x7fdae00b0b30 2023-12-20T04:46:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.451+0000 7fdae62b6640 1 -- 172.21.15.32:0/1171260718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdae004e000 con 0x7fdae00b0b30 2023-12-20T04:46:01.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.451+0000 7fdadcff9640 1 -- 172.21.15.32:0/1171260718 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fdad000bd30 con 0x7fdae00b0b30 2023-12-20T04:46:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.451+0000 7fdadcff9640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdacc065b80 0x7fdacc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.451+0000 7fdadcff9640 1 -- 172.21.15.32:0/1171260718 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fdad0083760 con 0x7fdae00b0b30 2023-12-20T04:46:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.451+0000 7fdadf7fe640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdacc065b80 0x7fdacc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.451+0000 7fdadf7fe640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdacc065b80 0x7fdacc068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdad40af000 tx=0x7fdad4008040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:01.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.455+0000 7fdadcff9640 1 -- 172.21.15.32:0/1171260718 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdad0051020 con 0x7fdae00b0b30 2023-12-20T04:46:01.524 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:46:01.530 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:46:01.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.619+0000 7fdae62b6640 1 -- 172.21.15.32:0/1171260718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdae00cf550 con 0x7fdae00b0b30 2023-12-20T04:46:01.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.619+0000 7fdadcff9640 1 -- 172.21.15.32:0/1171260718 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v122) v1 ==== 74+0+16016 (secure 0 0 0) 0x7fdad0054640 con 0x7fdae00b0b30 2023-12-20T04:46:01.619 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:46:01.619 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":122,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:46:00.898641+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:45:59.452651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-20T04:45:59.882509+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:46:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 -- 172.21.15.32:0/1171260718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdacc065b80 msgr2=0x7fdacc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdacc065b80 0x7fdacc068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fdad40af000 tx=0x7fdad4008040 comp rx=0 tx=0).stop 2023-12-20T04:46:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 -- 172.21.15.32:0/1171260718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdae00b0b30 msgr2=0x7fdae014bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdae00b0b30 0x7fdae014bb40 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fdad000d7d0 tx=0x7fdad000dc90 comp rx=0 tx=0).stop 2023-12-20T04:46:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 -- 172.21.15.32:0/1171260718 shutdown_connections 2023-12-20T04:46:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fdacc065b80 0x7fdacc068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdae01599e0 0x7fdae01585f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdae00c90e0 0x7fdae00cd8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 --2- 172.21.15.32:0/1171260718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdae00b0b30 0x7fdae014bb40 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 -- 172.21.15.32:0/1171260718 >> 172.21.15.32:0/1171260718 conn(0x7fdae006a320 msgr2=0x7fdae014a3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 -- 172.21.15.32:0/1171260718 shutdown_connections 2023-12-20T04:46:01.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.623+0000 7fdae62b6640 1 -- 172.21.15.32:0/1171260718 wait complete. 2023-12-20T04:46:01.641 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T04:46:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3772996640 1 Processor -- start 2023-12-20T04:46:01.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3772996640 1 -- start start 2023-12-20T04:46:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3772996640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f376c157480 0x7f376c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3772996640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c0cd4c0 0x7f376c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3772996640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f376c0b1100 con 0x7f376c0b0760 2023-12-20T04:46:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3772996640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f376c0cde20 con 0x7f376c157480 2023-12-20T04:46:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3772996640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376c0c8e30 con 0x7f376c0cd4c0 2023-12-20T04:46:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3770f33640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c0cd4c0 0x7f376c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f376bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f376c157480 0x7f376c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3770f33640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c0cd4c0 0x7f376c0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:38198/0 (socket says 172.21.15.32:38198) 2023-12-20T04:46:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f376bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f376c157480 0x7f376c0c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:44448/0 (socket says 172.21.15.32:44448) 2023-12-20T04:46:01.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3770f33640 1 -- 172.21.15.32:0/4107244911 learned_addr learned my addr 172.21.15.32:0/4107244911 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3770f33640 1 -- 172.21.15.32:0/4107244911 >> v2:172.21.15.39:3300/0 conn(0x7f376c157480 msgr2=0x7f376c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3770f33640 1 --2- 172.21.15.32:0/4107244911 >> v2:172.21.15.39:3300/0 conn(0x7f376c157480 0x7f376c0c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3770f33640 1 -- 172.21.15.32:0/4107244911 >> v1:172.21.15.39:6789/0 conn(0x7f376c0b0760 legacy=0x7f376c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f376affd640 1 -- 172.21.15.32:0/4107244911 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1139377408 0 0) 0x7f376c0b1100 con 0x7f376c0b0760 2023-12-20T04:46:01.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.835+0000 7f3770f33640 1 -- 172.21.15.32:0/4107244911 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f376c0ce070 con 0x7f376c0cd4c0 2023-12-20T04:46:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f3770f33640 1 --2- 172.21.15.32:0/4107244911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c0cd4c0 0x7f376c0b0bc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3764009e50 tx=0x7f376401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d302c892eb749c6d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:01.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f376affd640 1 -- 172.21.15.32:0/4107244911 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37640a5730 con 0x7f376c0cd4c0 2023-12-20T04:46:01.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f376affd640 1 -- 172.21.15.32:0/4107244911 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37640a5890 con 0x7f376c0cd4c0 2023-12-20T04:46:01.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f376affd640 1 -- 172.21.15.32:0/4107244911 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37640a5c20 con 0x7f376c0cd4c0 2023-12-20T04:46:01.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f3772996640 1 -- 172.21.15.32:0/4107244911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c0cd4c0 msgr2=0x7f376c0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f3772996640 1 --2- 172.21.15.32:0/4107244911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c0cd4c0 0x7f376c0b0bc0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f3764009e50 tx=0x7f376401c950 comp rx=0 tx=0).stop 2023-12-20T04:46:01.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f3772996640 1 -- 172.21.15.32:0/4107244911 shutdown_connections 2023-12-20T04:46:01.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f3772996640 1 --2- 172.21.15.32:0/4107244911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c0cd4c0 0x7f376c0b0bc0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f3772996640 1 --2- 172.21.15.32:0/4107244911 >> v2:172.21.15.39:3300/0 conn(0x7f376c157480 0x7f376c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f3772996640 1 -- 172.21.15.32:0/4107244911 >> 172.21.15.32:0/4107244911 conn(0x7f376c06a510 msgr2=0x7f376c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:01.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f3772996640 1 -- 172.21.15.32:0/4107244911 shutdown_connections 2023-12-20T04:46:01.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.839+0000 7f3772996640 1 -- 172.21.15.32:0/4107244911 wait complete. 2023-12-20T04:46:01.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3772996640 1 Processor -- start 2023-12-20T04:46:01.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3772996640 1 -- start start 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3772996640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f376c0b0760 0x7f376c0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3772996640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f376c0cd4c0 0x7f376c0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3772996640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c157480 0x7f376c157b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3772996640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376c162c30 con 0x7f376c0cd4c0 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3772996640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f376c1634d0 con 0x7f376c0b0760 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3772996640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f376c158870 con 0x7f376c157480 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f376b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f376c0cd4c0 0x7f376c0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f376bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f376c0b0760 0x7f376c0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f376b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f376c0cd4c0 0x7f376c0c3ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37808/0 (socket says 172.21.15.32:37808) 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f376b7fe640 1 -- 172.21.15.32:0/569562044 learned_addr learned my addr 172.21.15.32:0/569562044 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3770f33640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c157480 0x7f376c157b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f376b7fe640 1 -- 172.21.15.32:0/569562044 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c157480 msgr2=0x7f376c157b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f376b7fe640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c157480 0x7f376c157b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f376b7fe640 1 -- 172.21.15.32:0/569562044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f376c0b0760 msgr2=0x7f376c0c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f376b7fe640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f376c0b0760 0x7f376c0c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f376b7fe640 1 -- 172.21.15.32:0/569562044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3764009b30 con 0x7f376c0cd4c0 2023-12-20T04:46:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f376b7fe640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f376c0cd4c0 0x7f376c0c3ba0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f376000c420 tx=0x7f376000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3768ff9640 1 -- 172.21.15.32:0/569562044 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37600052d0 con 0x7f376c0cd4c0 2023-12-20T04:46:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3768ff9640 1 -- 172.21.15.32:0/569562044 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3760005450 con 0x7f376c0cd4c0 2023-12-20T04:46:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3772996640 1 -- 172.21.15.32:0/569562044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f376c162e80 con 0x7f376c0cd4c0 2023-12-20T04:46:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3768ff9640 1 -- 172.21.15.32:0/569562044 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3760003860 con 0x7f376c0cd4c0 2023-12-20T04:46:01.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.843+0000 7f3772996640 1 -- 172.21.15.32:0/569562044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f376c0c6150 con 0x7f376c0cd4c0 2023-12-20T04:46:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.847+0000 7f3772996640 1 -- 172.21.15.32:0/569562044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f376c163660 con 0x7f376c0cd4c0 2023-12-20T04:46:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.847+0000 7f3768ff9640 1 -- 172.21.15.32:0/569562044 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f37600039c0 con 0x7f376c0cd4c0 2023-12-20T04:46:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.847+0000 7f3768ff9640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3758065b80 0x7f3758068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:01.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.847+0000 7f3768ff9640 1 -- 172.21.15.32:0/569562044 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(122..122 src has 1..122) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f3760082820 con 0x7f376c0cd4c0 2023-12-20T04:46:01.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.847+0000 7f376bfff640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3758065b80 0x7f3758068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:01.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.847+0000 7f376bfff640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3758065b80 0x7f3758068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f375c009b60 tx=0x7f375c009190 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:01.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:01.851+0000 7f3768ff9640 1 -- 172.21.15.32:0/569562044 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3760050170 con 0x7f376c0cd4c0 2023-12-20T04:46:02.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.047+0000 7f3772996640 1 -- 172.21.15.32:0/569562044 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f376c04e000 con 0x7f376c0cd4c0 2023-12-20T04:46:02.471 INFO:tasks.ceph.mds.a.smithi032.stderr:2023-12-20T04:46:02.475+0000 7f07c939d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:46:02.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.935+0000 7f3768ff9640 1 -- 172.21.15.32:0/569562044 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v53) v1 ==== 118+0+0 (secure 0 0 0) 0x7f3760053790 con 0x7f376c0cd4c0 2023-12-20T04:46:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 -- 172.21.15.32:0/569562044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3758065b80 msgr2=0x7f3758068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3758065b80 0x7f3758068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f375c009b60 tx=0x7f375c009190 comp rx=0 tx=0).stop 2023-12-20T04:46:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 -- 172.21.15.32:0/569562044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f376c0cd4c0 msgr2=0x7f376c0c3ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:02.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f376c0cd4c0 0x7f376c0c3ba0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7f376000c420 tx=0x7f376000c8e0 comp rx=0 tx=0).stop 2023-12-20T04:46:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 -- 172.21.15.32:0/569562044 shutdown_connections 2023-12-20T04:46:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3758065b80 0x7f3758068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f376c157480 0x7f376c157b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f376c0cd4c0 0x7f376c0c3ba0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 --2- 172.21.15.32:0/569562044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f376c0b0760 0x7f376c0c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 -- 172.21.15.32:0/569562044 >> 172.21.15.32:0/569562044 conn(0x7f376c06a510 msgr2=0x7f376c06c9d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 -- 172.21.15.32:0/569562044 shutdown_connections 2023-12-20T04:46:02.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:02.939+0000 7f3772996640 1 -- 172.21.15.32:0/569562044 wait complete. 2023-12-20T04:46:02.961 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T04:46:02.961 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T04:46:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.115+0000 7f63fd9db640 1 Processor -- start 2023-12-20T04:46:03.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.115+0000 7f63fd9db640 1 -- start start 2023-12-20T04:46:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.115+0000 7f63fd9db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63f815abb0 0x7f63f80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.115+0000 7f63fd9db640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63f8070230 0x7f63f80706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:03.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.115+0000 7f63fd9db640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63f815b6a0 con 0x7f63f80b0cb0 2023-12-20T04:46:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.115+0000 7f63fd9db640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63f806e160 con 0x7f63f8070230 2023-12-20T04:46:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.115+0000 7f63fd9db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f8070be0 con 0x7f63f815abb0 2023-12-20T04:46:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63f8070230 0x7f63f80706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63f8070230 0x7f63f80706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37812/0 (socket says 172.21.15.32:37812) 2023-12-20T04:46:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f77fe640 1 -- 172.21.15.32:0/1679279397 learned_addr learned my addr 172.21.15.32:0/1679279397 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f6ffd640 1 --2- 172.21.15.32:0/1679279397 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63f815abb0 0x7f63f80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f77fe640 1 -- 172.21.15.32:0/1679279397 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63f815abb0 msgr2=0x7f63f80caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f77fe640 1 --2- 172.21.15.32:0/1679279397 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63f815abb0 0x7f63f80caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f77fe640 1 -- 172.21.15.32:0/1679279397 >> v1:172.21.15.39:6789/0 conn(0x7f63f80b0cb0 legacy=0x7f63f815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:03.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f77fe640 1 -- 172.21.15.32:0/1679279397 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f806d8c0 con 0x7f63f8070230 2023-12-20T04:46:03.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f77fe640 1 --2- 172.21.15.32:0/1679279397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63f8070230 0x7f63f80706a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009e50 tx=0x7f63f001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbac3d22e3166968 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:03.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f67fc640 1 -- 172.21.15.32:0/1679279397 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00a58e0 con 0x7f63f8070230 2023-12-20T04:46:03.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f67fc640 1 -- 172.21.15.32:0/1679279397 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f00a5a40 con 0x7f63f8070230 2023-12-20T04:46:03.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.119+0000 7f63f67fc640 1 -- 172.21.15.32:0/1679279397 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00a5dd0 con 0x7f63f8070230 2023-12-20T04:46:03.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 -- 172.21.15.32:0/1679279397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63f8070230 msgr2=0x7f63f80706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:03.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 --2- 172.21.15.32:0/1679279397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63f8070230 0x7f63f80706a0 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7f63f0009e50 tx=0x7f63f001ca10 comp rx=0 tx=0).stop 2023-12-20T04:46:03.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 -- 172.21.15.32:0/1679279397 shutdown_connections 2023-12-20T04:46:03.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 --2- 172.21.15.32:0/1679279397 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63f8070230 0x7f63f80706a0 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:03.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 --2- 172.21.15.32:0/1679279397 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63f815abb0 0x7f63f80caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:03.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 -- 172.21.15.32:0/1679279397 >> 172.21.15.32:0/1679279397 conn(0x7f63f806a510 msgr2=0x7f63f80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:03.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 -- 172.21.15.32:0/1679279397 shutdown_connections 2023-12-20T04:46:03.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 -- 172.21.15.32:0/1679279397 wait complete. 2023-12-20T04:46:03.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 Processor -- start 2023-12-20T04:46:03.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 -- start start 2023-12-20T04:46:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63f8070230 0x7f63f8150310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63f80b0cb0 0x7f63f8135890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:03.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63f815abb0 0x7f63f80b3ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f814e250 con 0x7f63f815abb0 2023-12-20T04:46:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63f8139280 con 0x7f63f8070230 2023-12-20T04:46:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63fd9db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63f8138fa0 con 0x7f63f80b0cb0 2023-12-20T04:46:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63f77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63f815abb0 0x7f63f80b3ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63f77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63f815abb0 0x7f63f80b3ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37824/0 (socket says 172.21.15.32:37824) 2023-12-20T04:46:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63f77fe640 1 -- 172.21.15.32:0/3347605203 learned_addr learned my addr 172.21.15.32:0/3347605203 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63effff640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63f80b0cb0 0x7f63f8135890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:03.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.123+0000 7f63f6ffd640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63f8070230 0x7f63f8150310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:03.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f77fe640 1 -- 172.21.15.32:0/3347605203 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63f80b0cb0 msgr2=0x7f63f8135890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:03.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f77fe640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63f80b0cb0 0x7f63f8135890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:03.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f77fe640 1 -- 172.21.15.32:0/3347605203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63f8070230 msgr2=0x7f63f8150310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:03.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f77fe640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63f8070230 0x7f63f8150310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:03.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f77fe640 1 -- 172.21.15.32:0/3347605203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63f0009b30 con 0x7f63f815abb0 2023-12-20T04:46:03.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f77fe640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63f815abb0 0x7f63f80b3ac0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f63f001a640 tx=0x7f63f00197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:03.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f4ff9640 1 -- 172.21.15.32:0/3347605203 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00b8640 con 0x7f63f815abb0 2023-12-20T04:46:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f4ff9640 1 -- 172.21.15.32:0/3347605203 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63f00b87a0 con 0x7f63f815abb0 2023-12-20T04:46:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f4ff9640 1 -- 172.21.15.32:0/3347605203 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63f00b8b50 con 0x7f63f815abb0 2023-12-20T04:46:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63fd9db640 1 -- 172.21.15.32:0/3347605203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63f814f830 con 0x7f63f815abb0 2023-12-20T04:46:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63fd9db640 1 -- 172.21.15.32:0/3347605203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63f814fdf0 con 0x7f63f815abb0 2023-12-20T04:46:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63fd9db640 1 -- 172.21.15.32:0/3347605203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63f814f690 con 0x7f63f815abb0 2023-12-20T04:46:03.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f4ff9640 1 -- 172.21.15.32:0/3347605203 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f63f00b8cb0 con 0x7f63f815abb0 2023-12-20T04:46:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f4ff9640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63d8065b80 0x7f63d8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f4ff9640 1 -- 172.21.15.32:0/3347605203 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63f011e4b0 con 0x7f63f815abb0 2023-12-20T04:46:03.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.127+0000 7f63f6ffd640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63d8065b80 0x7f63d8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:03.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.131+0000 7f63f6ffd640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63d8065b80 0x7f63d8068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009af0 tx=0x7f63e8008040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:03.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.131+0000 7f63f4ff9640 1 -- 172.21.15.32:0/3347605203 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f63f814f690 con 0x7f63f815abb0 2023-12-20T04:46:03.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.339+0000 7f63fd9db640 1 -- 172.21.15.32:0/3347605203 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f63f814f690 con 0x7f63f815abb0 2023-12-20T04:46:03.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.971+0000 7f63f4ff9640 1 -- 172.21.15.32:0/3347605203 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v54) v1 ==== 122+0+0 (secure 0 0 0) 0x7f63f814f690 con 0x7f63f815abb0 2023-12-20T04:46:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 -- 172.21.15.32:0/3347605203 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63d8065b80 msgr2=0x7f63d8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63d8065b80 0x7f63d8068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f63e8009af0 tx=0x7f63e8008040 comp rx=0 tx=0).stop 2023-12-20T04:46:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 -- 172.21.15.32:0/3347605203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63f815abb0 msgr2=0x7f63f80b3ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63f815abb0 0x7f63f80b3ac0 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7f63f001a640 tx=0x7f63f00197b0 comp rx=0 tx=0).stop 2023-12-20T04:46:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 -- 172.21.15.32:0/3347605203 shutdown_connections 2023-12-20T04:46:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63d8065b80 0x7f63d8068040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63f815abb0 0x7f63f80b3ac0 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63f80b0cb0 0x7f63f8135890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 --2- 172.21.15.32:0/3347605203 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63f8070230 0x7f63f8150310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 -- 172.21.15.32:0/3347605203 >> 172.21.15.32:0/3347605203 conn(0x7f63f806a510 msgr2=0x7f63f80c46a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 -- 172.21.15.32:0/3347605203 shutdown_connections 2023-12-20T04:46:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:03.975+0000 7f63fd9db640 1 -- 172.21.15.32:0/3347605203 wait complete. 2023-12-20T04:46:03.996 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T04:46:03.997 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:46:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.147+0000 7f81ef59e640 1 Processor -- start 2023-12-20T04:46:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.147+0000 7f81ef59e640 1 -- start start 2023-12-20T04:46:04.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.147+0000 7f81ef59e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81f0161500 0x7f81f0153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.147+0000 7f81ef59e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f00cd4d0 con 0x7f81f006b630 2023-12-20T04:46:04.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.147+0000 7f81ef59e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81f00ce4e0 con 0x7f81f00b8480 2023-12-20T04:46:04.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.147+0000 7f81ef59e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81f00cd720 con 0x7f81f0161500 2023-12-20T04:46:04.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.147+0000 7f81edd9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f81f006b630 0x7f81f0160df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36110/0 (socket says 172.21.15.32:36110) 2023-12-20T04:46:04.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.147+0000 7f81edd9b640 1 -- 172.21.15.32:0/1664021723 learned_addr learned my addr 172.21.15.32:0/1664021723 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:46:04.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.151+0000 7f81eed9d640 1 --2- 172.21.15.32:0/1664021723 >> v2:172.21.15.39:3300/0 conn(0x7f81f0161500 0x7f81f0153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.151+0000 7f81ed59a640 1 -- 172.21.15.32:0/1664021723 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 914404544 0 0) 0x7f81f00cd4d0 con 0x7f81f006b630 2023-12-20T04:46:04.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.151+0000 7f81ed59a640 1 -- 172.21.15.32:0/1664021723 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81d8003530 con 0x7f81f006b630 2023-12-20T04:46:04.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.151+0000 7f81eed9d640 1 -- 172.21.15.32:0/1664021723 >> v1:172.21.15.39:6789/0 conn(0x7f81f00b8480 legacy=0x7f81f006af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.151+0000 7f81eed9d640 1 -- 172.21.15.32:0/1664021723 >> v1:172.21.15.32:6789/0 conn(0x7f81f006b630 legacy=0x7f81f0160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.151+0000 7f81eed9d640 1 -- 172.21.15.32:0/1664021723 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81f00cda00 con 0x7f81f0161500 2023-12-20T04:46:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.151+0000 7f81eed9d640 1 --2- 172.21.15.32:0/1664021723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f0161500 0x7f81f0153610 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009d90 tx=0x7f81e801c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b756a165d3a4a2bc server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.151+0000 7f81ed59a640 1 -- 172.21.15.32:0/1664021723 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e80a57f0 con 0x7f81f0161500 2023-12-20T04:46:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ed59a640 1 -- 172.21.15.32:0/1664021723 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e80a5950 con 0x7f81f0161500 2023-12-20T04:46:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ed59a640 1 -- 172.21.15.32:0/1664021723 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e80a5cc0 con 0x7f81f0161500 2023-12-20T04:46:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 -- 172.21.15.32:0/1664021723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f0161500 msgr2=0x7f81f0153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 --2- 172.21.15.32:0/1664021723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f0161500 0x7f81f0153610 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7f81e8009d90 tx=0x7f81e801c950 comp rx=0 tx=0).stop 2023-12-20T04:46:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 -- 172.21.15.32:0/1664021723 shutdown_connections 2023-12-20T04:46:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 --2- 172.21.15.32:0/1664021723 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f0161500 0x7f81f0153610 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 -- 172.21.15.32:0/1664021723 >> 172.21.15.32:0/1664021723 conn(0x7f81f00d2a50 msgr2=0x7f81f00d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 -- 172.21.15.32:0/1664021723 shutdown_connections 2023-12-20T04:46:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 -- 172.21.15.32:0/1664021723 wait complete. 2023-12-20T04:46:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 Processor -- start 2023-12-20T04:46:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 -- start start 2023-12-20T04:46:04.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f006b630 0x7f81f014db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f00b8480 0x7f81f015fb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f0161500 0x7f81f014af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f0153030 con 0x7f81f00b8480 2023-12-20T04:46:04.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81f014b460 con 0x7f81f006b630 2023-12-20T04:46:04.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81ef59e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81f0152b00 con 0x7f81f0161500 2023-12-20T04:46:04.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81edd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f00b8480 0x7f81f015fb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81edd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f00b8480 0x7f81f015fb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37828/0 (socket says 172.21.15.32:37828) 2023-12-20T04:46:04.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.155+0000 7f81edd9b640 1 -- 172.21.15.32:0/3989406319 learned_addr learned my addr 172.21.15.32:0/3989406319 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:04.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81ee59c640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f006b630 0x7f81f014db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81eed9d640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f0161500 0x7f81f014af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81edd9b640 1 -- 172.21.15.32:0/3989406319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f0161500 msgr2=0x7f81f014af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81edd9b640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f0161500 0x7f81f014af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81edd9b640 1 -- 172.21.15.32:0/3989406319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f006b630 msgr2=0x7f81f014db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81edd9b640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f006b630 0x7f81f014db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81edd9b640 1 -- 172.21.15.32:0/3989406319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81e8009a70 con 0x7f81f00b8480 2023-12-20T04:46:04.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81edd9b640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f00b8480 0x7f81f015fb40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f81e400c500 tx=0x7f81e400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:04.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81deffd640 1 -- 172.21.15.32:0/3989406319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e4005cf0 con 0x7f81f00b8480 2023-12-20T04:46:04.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81deffd640 1 -- 172.21.15.32:0/3989406319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81e4005e50 con 0x7f81f00b8480 2023-12-20T04:46:04.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81ef59e640 1 -- 172.21.15.32:0/3989406319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81f00c6ce0 con 0x7f81f00b8480 2023-12-20T04:46:04.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81deffd640 1 -- 172.21.15.32:0/3989406319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81e40072f0 con 0x7f81f00b8480 2023-12-20T04:46:04.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81ef59e640 1 -- 172.21.15.32:0/3989406319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81f00c6690 con 0x7f81f00b8480 2023-12-20T04:46:04.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81ef59e640 1 -- 172.21.15.32:0/3989406319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81f00c64f0 con 0x7f81f00b8480 2023-12-20T04:46:04.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.159+0000 7f81deffd640 1 -- 172.21.15.32:0/3989406319 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f81e4007450 con 0x7f81f00b8480 2023-12-20T04:46:04.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.163+0000 7f81deffd640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81d0065b80 0x7f81d0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.163+0000 7f81deffd640 1 -- 172.21.15.32:0/3989406319 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f81e4082900 con 0x7f81f00b8480 2023-12-20T04:46:04.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.163+0000 7f81ee59c640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81d0065b80 0x7f81d0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.163+0000 7f81ee59c640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81d0065b80 0x7f81d0068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f81f015cf80 tx=0x7f81e0009040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:04.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.167+0000 7f81deffd640 1 -- 172.21.15.32:0/3989406319 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f81f00c64f0 con 0x7f81f00b8480 2023-12-20T04:46:04.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.355+0000 7f81ef59e640 1 -- 172.21.15.32:0/3989406319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81f00c64f0 con 0x7f81f00b8480 2023-12-20T04:46:04.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81deffd640 1 -- 172.21.15.32:0/3989406319 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4705 (secure 0 0 0) 0x7f81f00c64f0 con 0x7f81f00b8480 2023-12-20T04:46:04.355 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:46:04.355 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/593676281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":593676281},{"type":"v1","addr":"172.21.15.39:6833","nonce":593676281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2437945959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2437945959},{"type":"v1","addr":"172.21.15.39:6835","nonce":2437945959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7681,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/430674440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":430674440},{"type":"v1","addr":"172.21.15.32:6837","nonce":430674440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:46:00.899089+0000","modified":"2023-12-20T04:46:03.967555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7881":{"gid":7881,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3259359483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3259359483},{"type":"v1","addr":"172.21.15.32:6836","nonce":3259359483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-20T04:46:04.355 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 54 2023-12-20T04:46:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 -- 172.21.15.32:0/3989406319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81d0065b80 msgr2=0x7f81d0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81d0065b80 0x7f81d0068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f81f015cf80 tx=0x7f81e0009040 comp rx=0 tx=0).stop 2023-12-20T04:46:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 -- 172.21.15.32:0/3989406319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f00b8480 msgr2=0x7f81f015fb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f00b8480 0x7f81f015fb40 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7f81e400c500 tx=0x7f81e400c9c0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 -- 172.21.15.32:0/3989406319 shutdown_connections 2023-12-20T04:46:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81d0065b80 0x7f81d0068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81f0161500 0x7f81f014af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81f00b8480 0x7f81f015fb40 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 --2- 172.21.15.32:0/3989406319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81f006b630 0x7f81f014db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 -- 172.21.15.32:0/3989406319 >> 172.21.15.32:0/3989406319 conn(0x7f81f00d2a50 msgr2=0x7f81f00d2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 -- 172.21.15.32:0/3989406319 shutdown_connections 2023-12-20T04:46:04.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.359+0000 7f81ef59e640 1 -- 172.21.15.32:0/3989406319 wait complete. 2023-12-20T04:46:04.376 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:46:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cfd09640 1 Processor -- start 2023-12-20T04:46:04.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cfd09640 1 -- start start 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cfd09640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8c815cdb0 0x7fb8c80d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cfd09640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb8c80d2740 0x7fb8c80cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cfd09640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8c80b6ca0 con 0x7fb8c80d1120 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cfd09640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c815d190 con 0x7fb8c815cdb0 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cfd09640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8c80d16a0 con 0x7fb8c80d2740 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cdaa5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8c815cdb0 0x7fb8c80d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cdaa5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8c815cdb0 0x7fb8c80d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37844/0 (socket says 172.21.15.32:37844) 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cdaa5640 1 -- 172.21.15.32:0/2289144485 learned_addr learned my addr 172.21.15.32:0/2289144485 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cd2a4640 1 --2- 172.21.15.32:0/2289144485 >> v2:172.21.15.39:3300/0 conn(0x7fb8c80d2740 0x7fb8c80cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cdaa5640 1 -- 172.21.15.32:0/2289144485 >> v2:172.21.15.39:3300/0 conn(0x7fb8c80d2740 msgr2=0x7fb8c80cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cdaa5640 1 --2- 172.21.15.32:0/2289144485 >> v2:172.21.15.39:3300/0 conn(0x7fb8c80d2740 0x7fb8c80cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cdaa5640 1 -- 172.21.15.32:0/2289144485 >> v1:172.21.15.39:6789/0 conn(0x7fb8c80d1120 legacy=0x7fb8c80b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.523+0000 7fb8cdaa5640 1 -- 172.21.15.32:0/2289144485 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c80d1980 con 0x7fb8c815cdb0 2023-12-20T04:46:04.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8cdaa5640 1 --2- 172.21.15.32:0/2289144485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c815cdb0 0x7fb8c80d0be0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80cd570 tx=0x7fb8c401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=772983e0b07e0484 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:04.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8ccaa3640 1 -- 172.21.15.32:0/2289144485 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40a89f0 con 0x7fb8c815cdb0 2023-12-20T04:46:04.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8ccaa3640 1 -- 172.21.15.32:0/2289144485 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c40a8b50 con 0x7fb8c815cdb0 2023-12-20T04:46:04.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8ccaa3640 1 -- 172.21.15.32:0/2289144485 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40b0440 con 0x7fb8c815cdb0 2023-12-20T04:46:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8cfd09640 1 -- 172.21.15.32:0/2289144485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c815cdb0 msgr2=0x7fb8c80d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8cfd09640 1 --2- 172.21.15.32:0/2289144485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c815cdb0 0x7fb8c80d0be0 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7fb8c80cd570 tx=0x7fb8c401ecf0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8cfd09640 1 -- 172.21.15.32:0/2289144485 shutdown_connections 2023-12-20T04:46:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8cfd09640 1 --2- 172.21.15.32:0/2289144485 >> v2:172.21.15.39:3300/0 conn(0x7fb8c80d2740 0x7fb8c80cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8cfd09640 1 --2- 172.21.15.32:0/2289144485 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c815cdb0 0x7fb8c80d0be0 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8cfd09640 1 -- 172.21.15.32:0/2289144485 >> 172.21.15.32:0/2289144485 conn(0x7fb8c8065ee0 msgr2=0x7fb8c806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8cfd09640 1 -- 172.21.15.32:0/2289144485 shutdown_connections 2023-12-20T04:46:04.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.527+0000 7fb8cfd09640 1 -- 172.21.15.32:0/2289144485 wait complete. 2023-12-20T04:46:04.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cfd09640 1 Processor -- start 2023-12-20T04:46:04.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cfd09640 1 -- start start 2023-12-20T04:46:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cfd09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c80d1120 0x7fb8c80c8300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cfd09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8c80d2740 0x7fb8c815f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cfd09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c815cdb0 0x7fb8c815f920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cfd09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c81e53d0 con 0x7fb8c815cdb0 2023-12-20T04:46:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cfd09640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c81e56b0 con 0x7fb8c80d1120 2023-12-20T04:46:04.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cfd09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8c81e5990 con 0x7fb8c80d2740 2023-12-20T04:46:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8ce2a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c815cdb0 0x7fb8c815f920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8ce2a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c815cdb0 0x7fb8c815f920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37852/0 (socket says 172.21.15.32:37852) 2023-12-20T04:46:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8ce2a6640 1 -- 172.21.15.32:0/685571889 learned_addr learned my addr 172.21.15.32:0/685571889 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cdaa5640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c80d1120 0x7fb8c80c8300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cd2a4640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8c80d2740 0x7fb8c815f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8ce2a6640 1 -- 172.21.15.32:0/685571889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8c80d2740 msgr2=0x7fb8c815f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8ce2a6640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8c80d2740 0x7fb8c815f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8ce2a6640 1 -- 172.21.15.32:0/685571889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c80d1120 msgr2=0x7fb8c80c8300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8ce2a6640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c80d1120 0x7fb8c80c8300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8ce2a6640 1 -- 172.21.15.32:0/685571889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c40094d0 con 0x7fb8c815cdb0 2023-12-20T04:46:04.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8ce2a6640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c815cdb0 0x7fb8c815f920 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c770 tx=0x7fb8c000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:04.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8be7fc640 1 -- 172.21.15.32:0/685571889 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c0003b80 con 0x7fb8c815cdb0 2023-12-20T04:46:04.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8be7fc640 1 -- 172.21.15.32:0/685571889 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c0003ce0 con 0x7fb8c815cdb0 2023-12-20T04:46:04.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cfd09640 1 -- 172.21.15.32:0/685571889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8c81e5c70 con 0x7fb8c815cdb0 2023-12-20T04:46:04.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8be7fc640 1 -- 172.21.15.32:0/685571889 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c000aa90 con 0x7fb8c815cdb0 2023-12-20T04:46:04.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cfd09640 1 -- 172.21.15.32:0/685571889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8c815e0b0 con 0x7fb8c815cdb0 2023-12-20T04:46:04.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.531+0000 7fb8cfd09640 1 -- 172.21.15.32:0/685571889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8c81e5eb0 con 0x7fb8c815cdb0 2023-12-20T04:46:04.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.535+0000 7fb8be7fc640 1 -- 172.21.15.32:0/685571889 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb8c000abf0 con 0x7fb8c815cdb0 2023-12-20T04:46:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.535+0000 7fb8be7fc640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8ac065b80 0x7fb8ac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.535+0000 7fb8be7fc640 1 -- 172.21.15.32:0/685571889 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fb8c0082f60 con 0x7fb8c815cdb0 2023-12-20T04:46:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.535+0000 7fb8cdaa5640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8ac065b80 0x7fb8ac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.535+0000 7fb8cdaa5640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8ac065b80 0x7fb8ac068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40b2000 tx=0x7fb8c4009040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:04.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.539+0000 7fb8be7fc640 1 -- 172.21.15.32:0/685571889 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb8c00507a0 con 0x7fb8c815cdb0 2023-12-20T04:46:04.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 -- 172.21.15.32:0/685571889 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8c80b6ca0 con 0x7fb8c815cdb0 2023-12-20T04:46:04.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8be7fc640 1 -- 172.21.15.32:0/685571889 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v123) v1 ==== 74+0+16003 (secure 0 0 0) 0x7fb8c80b6ca0 con 0x7fb8c815cdb0 2023-12-20T04:46:04.700 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:46:04.700 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":123,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:46:01.902714+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:45:59.452651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-20T04:45:59.882509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:46:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 -- 172.21.15.32:0/685571889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8ac065b80 msgr2=0x7fb8ac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8ac065b80 0x7fb8ac068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40b2000 tx=0x7fb8c4009040 comp rx=0 tx=0).stop 2023-12-20T04:46:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 -- 172.21.15.32:0/685571889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c815cdb0 msgr2=0x7fb8c815f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c815cdb0 0x7fb8c815f920 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fb8c000c770 tx=0x7fb8c000cc30 comp rx=0 tx=0).stop 2023-12-20T04:46:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 -- 172.21.15.32:0/685571889 shutdown_connections 2023-12-20T04:46:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb8ac065b80 0x7fb8ac068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8c815cdb0 0x7fb8c815f920 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8c80d2740 0x7fb8c815f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 --2- 172.21.15.32:0/685571889 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8c80d1120 0x7fb8c80c8300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 -- 172.21.15.32:0/685571889 >> 172.21.15.32:0/685571889 conn(0x7fb8c8065ee0 msgr2=0x7fb8c80d3030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 -- 172.21.15.32:0/685571889 shutdown_connections 2023-12-20T04:46:04.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.703+0000 7fb8cfd09640 1 -- 172.21.15.32:0/685571889 wait complete. 2023-12-20T04:46:04.722 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T04:46:04.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:46:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.867+0000 7f279f041640 1 Processor -- start 2023-12-20T04:46:04.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.867+0000 7f279f041640 1 -- start start 2023-12-20T04:46:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.867+0000 7f279f041640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f27981593c0 0x7f279815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.867+0000 7f279f041640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f279806caf0 0x7f27980705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.867+0000 7f279f041640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27980ae140 con 0x7f27980afdb0 2023-12-20T04:46:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.867+0000 7f279f041640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27980ae9e0 con 0x7f279806caf0 2023-12-20T04:46:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.867+0000 7f279f041640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f279815dee0 con 0x7f27981593c0 2023-12-20T04:46:04.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.867+0000 7f2797fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f27980afdb0 0x7f279806c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36112/0 (socket says 172.21.15.32:36112) 2023-12-20T04:46:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.867+0000 7f279d5de640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f279806caf0 0x7f27980705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.867+0000 7f2797fff640 1 -- 172.21.15.32:0/747630399 learned_addr learned my addr 172.21.15.32:0/747630399 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:46:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.867+0000 7f279cddd640 1 --2- 172.21.15.32:0/747630399 >> v2:172.21.15.39:3300/0 conn(0x7f27981593c0 0x7f279815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f27977fe640 1 -- 172.21.15.32:0/747630399 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 992420292 0 0) 0x7f27980ae140 con 0x7f27980afdb0 2023-12-20T04:46:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f279d5de640 1 -- 172.21.15.32:0/747630399 >> v2:172.21.15.39:3300/0 conn(0x7f27981593c0 msgr2=0x7f279815d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f279d5de640 1 --2- 172.21.15.32:0/747630399 >> v2:172.21.15.39:3300/0 conn(0x7f27981593c0 0x7f279815d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f279d5de640 1 -- 172.21.15.32:0/747630399 >> v1:172.21.15.32:6789/0 conn(0x7f27980afdb0 legacy=0x7f279806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f279d5de640 1 -- 172.21.15.32:0/747630399 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2798072170 con 0x7f279806caf0 2023-12-20T04:46:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f279d5de640 1 --2- 172.21.15.32:0/747630399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f279806caf0 0x7f27980705a0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f279806ab00 tx=0x7f279001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=785e95a0b1ea6741 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f27977fe640 1 -- 172.21.15.32:0/747630399 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27900a5780 con 0x7f279806caf0 2023-12-20T04:46:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f27977fe640 1 -- 172.21.15.32:0/747630399 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27900a58e0 con 0x7f279806caf0 2023-12-20T04:46:04.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f27977fe640 1 -- 172.21.15.32:0/747630399 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27900a5c70 con 0x7f279806caf0 2023-12-20T04:46:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f279f041640 1 -- 172.21.15.32:0/747630399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f279806caf0 msgr2=0x7f27980705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f279f041640 1 --2- 172.21.15.32:0/747630399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f279806caf0 0x7f27980705a0 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7f279806ab00 tx=0x7f279001c850 comp rx=0 tx=0).stop 2023-12-20T04:46:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.871+0000 7f279f041640 1 -- 172.21.15.32:0/747630399 shutdown_connections 2023-12-20T04:46:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 --2- 172.21.15.32:0/747630399 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f279806caf0 0x7f27980705a0 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 --2- 172.21.15.32:0/747630399 >> v2:172.21.15.39:3300/0 conn(0x7f27981593c0 0x7f279815d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 -- 172.21.15.32:0/747630399 >> 172.21.15.32:0/747630399 conn(0x7f279806a320 msgr2=0x7f27980ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 -- 172.21.15.32:0/747630399 shutdown_connections 2023-12-20T04:46:04.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 -- 172.21.15.32:0/747630399 wait complete. 2023-12-20T04:46:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 Processor -- start 2023-12-20T04:46:04.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 -- start start 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f279806caf0 0x7f27980c6c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27980afdb0 0x7f279815b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27981593c0 0x7f279815a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c5b60 con 0x7f279806caf0 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c5e40 con 0x7f27980afdb0 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279f041640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27980c6120 con 0x7f27981593c0 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279cddd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f279806caf0 0x7f27980c6c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279cddd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f279806caf0 0x7f27980c6c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37872/0 (socket says 172.21.15.32:37872) 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279cddd640 1 -- 172.21.15.32:0/2545018127 learned_addr learned my addr 172.21.15.32:0/2545018127 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279d5de640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27981593c0 0x7f279815a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f2797fff640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27980afdb0 0x7f279815b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279cddd640 1 -- 172.21.15.32:0/2545018127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27981593c0 msgr2=0x7f279815a3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279cddd640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27981593c0 0x7f279815a3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279cddd640 1 -- 172.21.15.32:0/2545018127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27980afdb0 msgr2=0x7f279815b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279cddd640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27980afdb0 0x7f279815b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279cddd640 1 -- 172.21.15.32:0/2545018127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2790009a70 con 0x7f279806caf0 2023-12-20T04:46:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f279cddd640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f279806caf0 0x7f27980c6c00 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f278c00e9b0 tx=0x7f278c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.875+0000 7f27957fa640 1 -- 172.21.15.32:0/2545018127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c009670 con 0x7f279806caf0 2023-12-20T04:46:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.879+0000 7f27957fa640 1 -- 172.21.15.32:0/2545018127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f278c00c9a0 con 0x7f279806caf0 2023-12-20T04:46:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.879+0000 7f27957fa640 1 -- 172.21.15.32:0/2545018127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f278c00cd10 con 0x7f279806caf0 2023-12-20T04:46:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.879+0000 7f279f041640 1 -- 172.21.15.32:0/2545018127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27980c6400 con 0x7f279806caf0 2023-12-20T04:46:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.879+0000 7f279f041640 1 -- 172.21.15.32:0/2545018127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27980c5830 con 0x7f279806caf0 2023-12-20T04:46:04.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.879+0000 7f279f041640 1 -- 172.21.15.32:0/2545018127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f279804e000 con 0x7f279806caf0 2023-12-20T04:46:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.879+0000 7f27957fa640 1 -- 172.21.15.32:0/2545018127 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f278c013470 con 0x7f279806caf0 2023-12-20T04:46:04.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.879+0000 7f27957fa640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f277c065b80 0x7f277c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.879+0000 7f27957fa640 1 -- 172.21.15.32:0/2545018127 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f278c083910 con 0x7f279806caf0 2023-12-20T04:46:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.879+0000 7f2797fff640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f277c065b80 0x7f277c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:04.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.879+0000 7f2797fff640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f277c065b80 0x7f277c068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f279815cae0 tx=0x7f2784007d60 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:04.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:04.883+0000 7f27957fa640 1 -- 172.21.15.32:0/2545018127 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f278c0511d0 con 0x7f279806caf0 2023-12-20T04:46:05.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.043+0000 7f279f041640 1 -- 172.21.15.32:0/2545018127 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f27981597a0 con 0x7f277c065b80 2023-12-20T04:46:05.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.051+0000 7f27957fa640 1 -- 172.21.15.32:0/2545018127 <== mgr.7465 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285495 (secure 0 0 0) 0x7f2784004c80 con 0x7f277c065b80 2023-12-20T04:46:05.053 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:46:05.054 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:46:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 -- 172.21.15.32:0/2545018127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f277c065b80 msgr2=0x7f277c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f277c065b80 0x7f277c068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f279815cae0 tx=0x7f2784007d60 comp rx=0 tx=0).stop 2023-12-20T04:46:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 -- 172.21.15.32:0/2545018127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f279806caf0 msgr2=0x7f27980c6c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f279806caf0 0x7f27980c6c00 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x7f278c00e9b0 tx=0x7f278c00ee70 comp rx=0 tx=0).stop 2023-12-20T04:46:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 -- 172.21.15.32:0/2545018127 shutdown_connections 2023-12-20T04:46:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f277c065b80 0x7f277c068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27981593c0 0x7f279815a3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27980afdb0 0x7f279815b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 --2- 172.21.15.32:0/2545018127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f279806caf0 0x7f27980c6c00 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 -- 172.21.15.32:0/2545018127 >> 172.21.15.32:0/2545018127 conn(0x7f279806a320 msgr2=0x7f27980ab8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 -- 172.21.15.32:0/2545018127 shutdown_connections 2023-12-20T04:46:05.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.055+0000 7f279f041640 1 -- 172.21.15.32:0/2545018127 wait complete. 2023-12-20T04:46:05.073 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-20T04:46:03.215390+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592548,"num_objects":19,"num_object_clones":0,"num_object_copies":38,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":19,"num_whiteouts":0,"num_read":829,"num_read_kb":1583,"num_write":339,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3112,"ondisk_log_size":3112,"up":144,"acting":144,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":7,"kb":754974720,"kb_used":321992,"kb_used_data":12936,"kb_used_omap":0,"kb_used_meta":308992,"kb_avail":754652728,"statfs":{"total":773094113280,"available":772764393472,"internally_reserved":0,"allocated":13246464,"data_stored":6875951,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316407808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2180,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000638"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.924349+0000","last_change":"2023-12-20T04:46:00.919376+0000","last_active":"2023-12-20T04:46:00.924349+0000","last_peered":"2023-12-20T04:46:00.924349+0000","last_clean":"2023-12-20T04:46:00.924349+0000","last_became_active":"2023-12-20T04:46:00.919120+0000","last_became_peered":"2023-12-20T04:46:00.919120+0000","last_unstale":"2023-12-20T04:46:00.924349+0000","last_undegraded":"2023-12-20T04:46:00.924349+0000","last_fullsized":"2023-12-20T04:46:00.924349+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:48:30.160568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913441+0000","last_change":"2023-12-20T04:46:00.927397+0000","last_active":"2023-12-20T04:46:01.913441+0000","last_peered":"2023-12-20T04:46:01.913441+0000","last_clean":"2023-12-20T04:46:01.913441+0000","last_became_active":"2023-12-20T04:46:00.919451+0000","last_became_peered":"2023-12-20T04:46:00.919451+0000","last_unstale":"2023-12-20T04:46:01.913441+0000","last_undegraded":"2023-12-20T04:46:01.913441+0000","last_fullsized":"2023-12-20T04:46:01.913441+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:57:30.367208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.924178+0000","last_change":"2023-12-20T04:46:00.921290+0000","last_active":"2023-12-20T04:46:00.924178+0000","last_peered":"2023-12-20T04:46:00.924178+0000","last_clean":"2023-12-20T04:46:00.924178+0000","last_became_active":"2023-12-20T04:46:00.920960+0000","last_became_peered":"2023-12-20T04:46:00.920960+0000","last_unstale":"2023-12-20T04:46:00.924178+0000","last_undegraded":"2023-12-20T04:46:00.924178+0000","last_fullsized":"2023-12-20T04:46:00.924178+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:13:17.257637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.943725+0000","last_change":"2023-12-20T04:46:00.940419+0000","last_active":"2023-12-20T04:46:00.943725+0000","last_peered":"2023-12-20T04:46:00.943725+0000","last_clean":"2023-12-20T04:46:00.943725+0000","last_became_active":"2023-12-20T04:46:00.940065+0000","last_became_peered":"2023-12-20T04:46:00.940065+0000","last_unstale":"2023-12-20T04:46:00.943725+0000","last_undegraded":"2023-12-20T04:46:00.943725+0000","last_fullsized":"2023-12-20T04:46:00.943725+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:08:58.476143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.912974+0000","last_change":"2023-12-20T04:46:00.921093+0000","last_active":"2023-12-20T04:46:01.912974+0000","last_peered":"2023-12-20T04:46:01.912974+0000","last_clean":"2023-12-20T04:46:01.912974+0000","last_became_active":"2023-12-20T04:46:00.920844+0000","last_became_peered":"2023-12-20T04:46:00.920844+0000","last_unstale":"2023-12-20T04:46:01.912974+0000","last_undegraded":"2023-12-20T04:46:01.912974+0000","last_fullsized":"2023-12-20T04:46:01.912974+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:07.906740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927375+0000","last_change":"2023-12-20T04:46:00.941968+0000","last_active":"2023-12-20T04:46:01.927375+0000","last_peered":"2023-12-20T04:46:01.927375+0000","last_clean":"2023-12-20T04:46:01.927375+0000","last_became_active":"2023-12-20T04:46:00.941617+0000","last_became_peered":"2023-12-20T04:46:00.941617+0000","last_unstale":"2023-12-20T04:46:01.927375+0000","last_undegraded":"2023-12-20T04:46:01.927375+0000","last_fullsized":"2023-12-20T04:46:01.927375+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:08:58.476143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.926863+0000","last_change":"2023-12-20T04:46:00.926790+0000","last_active":"2023-12-20T04:46:01.926863+0000","last_peered":"2023-12-20T04:46:01.926863+0000","last_clean":"2023-12-20T04:46:01.926863+0000","last_became_active":"2023-12-20T04:46:00.926451+0000","last_became_peered":"2023-12-20T04:46:00.926451+0000","last_unstale":"2023-12-20T04:46:01.926863+0000","last_undegraded":"2023-12-20T04:46:01.926863+0000","last_fullsized":"2023-12-20T04:46:01.926863+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:55.139256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.941940+0000","last_change":"2023-12-20T04:46:00.935206+0000","last_active":"2023-12-20T04:46:00.941940+0000","last_peered":"2023-12-20T04:46:00.941940+0000","last_clean":"2023-12-20T04:46:00.941940+0000","last_became_active":"2023-12-20T04:46:00.934874+0000","last_became_peered":"2023-12-20T04:46:00.934874+0000","last_unstale":"2023-12-20T04:46:00.941940+0000","last_undegraded":"2023-12-20T04:46:00.941940+0000","last_fullsized":"2023-12-20T04:46:00.941940+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:08:30.180394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.934860+0000","last_change":"2023-12-20T04:46:00.932291+0000","last_active":"2023-12-20T04:46:00.934860+0000","last_peered":"2023-12-20T04:46:00.934860+0000","last_clean":"2023-12-20T04:46:00.934860+0000","last_became_active":"2023-12-20T04:46:00.930497+0000","last_became_peered":"2023-12-20T04:46:00.930497+0000","last_unstale":"2023-12-20T04:46:00.934860+0000","last_undegraded":"2023-12-20T04:46:00.934860+0000","last_fullsized":"2023-12-20T04:46:00.934860+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:38:45.867057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.941979+0000","last_change":"2023-12-20T04:46:00.935957+0000","last_active":"2023-12-20T04:46:00.941979+0000","last_peered":"2023-12-20T04:46:00.941979+0000","last_clean":"2023-12-20T04:46:00.941979+0000","last_became_active":"2023-12-20T04:46:00.934688+0000","last_became_peered":"2023-12-20T04:46:00.934688+0000","last_unstale":"2023-12-20T04:46:00.941979+0000","last_undegraded":"2023-12-20T04:46:00.941979+0000","last_fullsized":"2023-12-20T04:46:00.941979+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:47:17.644936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913681+0000","last_change":"2023-12-20T04:46:00.935895+0000","last_active":"2023-12-20T04:46:01.913681+0000","last_peered":"2023-12-20T04:46:01.913681+0000","last_clean":"2023-12-20T04:46:01.913681+0000","last_became_active":"2023-12-20T04:46:00.935299+0000","last_became_peered":"2023-12-20T04:46:00.935299+0000","last_unstale":"2023-12-20T04:46:01.913681+0000","last_undegraded":"2023-12-20T04:46:01.913681+0000","last_fullsized":"2023-12-20T04:46:01.913681+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:30:30.148194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.943762+0000","last_change":"2023-12-20T04:46:00.940192+0000","last_active":"2023-12-20T04:46:00.943762+0000","last_peered":"2023-12-20T04:46:00.943762+0000","last_clean":"2023-12-20T04:46:00.943762+0000","last_became_active":"2023-12-20T04:46:00.939897+0000","last_became_peered":"2023-12-20T04:46:00.939897+0000","last_unstale":"2023-12-20T04:46:00.943762+0000","last_undegraded":"2023-12-20T04:46:00.943762+0000","last_fullsized":"2023-12-20T04:46:00.943762+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:07:45.553369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.931560+0000","last_change":"2023-12-20T04:46:00.926559+0000","last_active":"2023-12-20T04:46:00.931560+0000","last_peered":"2023-12-20T04:46:00.931560+0000","last_clean":"2023-12-20T04:46:00.931560+0000","last_became_active":"2023-12-20T04:46:00.926301+0000","last_became_peered":"2023-12-20T04:46:00.926301+0000","last_unstale":"2023-12-20T04:46:00.931560+0000","last_undegraded":"2023-12-20T04:46:00.931560+0000","last_fullsized":"2023-12-20T04:46:00.931560+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:01:24.712802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927005+0000","last_change":"2023-12-20T04:46:00.942197+0000","last_active":"2023-12-20T04:46:01.927005+0000","last_peered":"2023-12-20T04:46:01.927005+0000","last_clean":"2023-12-20T04:46:01.927005+0000","last_became_active":"2023-12-20T04:46:00.941909+0000","last_became_peered":"2023-12-20T04:46:00.941909+0000","last_unstale":"2023-12-20T04:46:01.927005+0000","last_undegraded":"2023-12-20T04:46:01.927005+0000","last_fullsized":"2023-12-20T04:46:01.927005+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:10:07.235864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.932713+0000","last_change":"2023-12-20T04:46:00.930113+0000","last_active":"2023-12-20T04:46:00.932713+0000","last_peered":"2023-12-20T04:46:00.932713+0000","last_clean":"2023-12-20T04:46:00.932713+0000","last_became_active":"2023-12-20T04:46:00.929881+0000","last_became_peered":"2023-12-20T04:46:00.929881+0000","last_unstale":"2023-12-20T04:46:00.932713+0000","last_undegraded":"2023-12-20T04:46:00.932713+0000","last_fullsized":"2023-12-20T04:46:00.932713+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:43.705186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.912978+0000","last_change":"2023-12-20T04:46:00.920294+0000","last_active":"2023-12-20T04:46:01.912978+0000","last_peered":"2023-12-20T04:46:01.912978+0000","last_clean":"2023-12-20T04:46:01.912978+0000","last_became_active":"2023-12-20T04:46:00.920001+0000","last_became_peered":"2023-12-20T04:46:00.920001+0000","last_unstale":"2023-12-20T04:46:01.912978+0000","last_undegraded":"2023-12-20T04:46:01.912978+0000","last_fullsized":"2023-12-20T04:46:01.912978+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:01:27.853191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913031+0000","last_change":"2023-12-20T04:46:00.918533+0000","last_active":"2023-12-20T04:46:01.913031+0000","last_peered":"2023-12-20T04:46:01.913031+0000","last_clean":"2023-12-20T04:46:01.913031+0000","last_became_active":"2023-12-20T04:46:00.918272+0000","last_became_peered":"2023-12-20T04:46:00.918272+0000","last_unstale":"2023-12-20T04:46:01.913031+0000","last_undegraded":"2023-12-20T04:46:01.913031+0000","last_fullsized":"2023-12-20T04:46:01.913031+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:08:15.729193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913439+0000","last_change":"2023-12-20T04:46:00.918837+0000","last_active":"2023-12-20T04:46:01.913439+0000","last_peered":"2023-12-20T04:46:01.913439+0000","last_clean":"2023-12-20T04:46:01.913439+0000","last_became_active":"2023-12-20T04:46:00.918585+0000","last_became_peered":"2023-12-20T04:46:00.918585+0000","last_unstale":"2023-12-20T04:46:01.913439+0000","last_undegraded":"2023-12-20T04:46:01.913439+0000","last_fullsized":"2023-12-20T04:46:01.913439+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:04:07.931455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913058+0000","last_change":"2023-12-20T04:46:00.928631+0000","last_active":"2023-12-20T04:46:01.913058+0000","last_peered":"2023-12-20T04:46:01.913058+0000","last_clean":"2023-12-20T04:46:01.913058+0000","last_became_active":"2023-12-20T04:46:00.919785+0000","last_became_peered":"2023-12-20T04:46:00.919785+0000","last_unstale":"2023-12-20T04:46:01.913058+0000","last_undegraded":"2023-12-20T04:46:01.913058+0000","last_fullsized":"2023-12-20T04:46:01.913058+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:55:56.657692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.932463+0000","last_change":"2023-12-20T04:46:00.927292+0000","last_active":"2023-12-20T04:46:00.932463+0000","last_peered":"2023-12-20T04:46:00.932463+0000","last_clean":"2023-12-20T04:46:00.932463+0000","last_became_active":"2023-12-20T04:46:00.926281+0000","last_became_peered":"2023-12-20T04:46:00.926281+0000","last_unstale":"2023-12-20T04:46:00.932463+0000","last_undegraded":"2023-12-20T04:46:00.932463+0000","last_fullsized":"2023-12-20T04:46:00.932463+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:24:50.810985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.942192+0000","last_change":"2023-12-20T04:46:00.937288+0000","last_active":"2023-12-20T04:46:00.942192+0000","last_peered":"2023-12-20T04:46:00.942192+0000","last_clean":"2023-12-20T04:46:00.942192+0000","last_became_active":"2023-12-20T04:46:00.934437+0000","last_became_peered":"2023-12-20T04:46:00.934437+0000","last_unstale":"2023-12-20T04:46:00.942192+0000","last_undegraded":"2023-12-20T04:46:00.942192+0000","last_fullsized":"2023-12-20T04:46:00.942192+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:13:54.533082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.916771+0000","last_change":"2023-12-20T04:46:00.915038+0000","last_active":"2023-12-20T04:46:00.916771+0000","last_peered":"2023-12-20T04:46:00.916771+0000","last_clean":"2023-12-20T04:46:00.916771+0000","last_became_active":"2023-12-20T04:46:00.914687+0000","last_became_peered":"2023-12-20T04:46:00.914687+0000","last_unstale":"2023-12-20T04:46:00.916771+0000","last_undegraded":"2023-12-20T04:46:00.916771+0000","last_fullsized":"2023-12-20T04:46:00.916771+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:53:55.732437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.933905+0000","last_change":"2023-12-20T04:46:00.931455+0000","last_active":"2023-12-20T04:46:00.933905+0000","last_peered":"2023-12-20T04:46:00.933905+0000","last_clean":"2023-12-20T04:46:00.933905+0000","last_became_active":"2023-12-20T04:46:00.931187+0000","last_became_peered":"2023-12-20T04:46:00.931187+0000","last_unstale":"2023-12-20T04:46:00.933905+0000","last_undegraded":"2023-12-20T04:46:00.933905+0000","last_fullsized":"2023-12-20T04:46:00.933905+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:35:04.809390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"122'4","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913713+0000","last_change":"2023-12-20T04:46:00.513411+0000","last_active":"2023-12-20T04:46:01.913713+0000","last_peered":"2023-12-20T04:46:01.913713+0000","last_clean":"2023-12-20T04:46:01.913713+0000","last_became_active":"2023-12-20T04:46:00.513137+0000","last_became_peered":"2023-12-20T04:46:00.513137+0000","last_unstale":"2023-12-20T04:46:01.913713+0000","last_undegraded":"2023-12-20T04:46:01.913713+0000","last_fullsized":"2023-12-20T04:46:01.913713+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:40:19.775654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.941868+0000","last_change":"2023-12-20T04:46:00.933940+0000","last_active":"2023-12-20T04:46:00.941868+0000","last_peered":"2023-12-20T04:46:00.941868+0000","last_clean":"2023-12-20T04:46:00.941868+0000","last_became_active":"2023-12-20T04:46:00.933244+0000","last_became_peered":"2023-12-20T04:46:00.933244+0000","last_unstale":"2023-12-20T04:46:00.941868+0000","last_undegraded":"2023-12-20T04:46:00.941868+0000","last_fullsized":"2023-12-20T04:46:00.941868+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:48:26.269219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:45:59.474887+0000","last_change":"2023-12-20T04:45:59.474887+0000","last_active":"2023-12-20T04:45:59.474887+0000","last_peered":"2023-12-20T04:45:59.474887+0000","last_clean":"2023-12-20T04:45:59.474887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:45:59.474887+0000","last_undegraded":"2023-12-20T04:45:59.474887+0000","last_fullsized":"2023-12-20T04:45:59.474887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.934794+0000","last_change":"2023-12-20T04:46:00.927129+0000","last_active":"2023-12-20T04:46:00.934794+0000","last_peered":"2023-12-20T04:46:00.934794+0000","last_clean":"2023-12-20T04:46:00.934794+0000","last_became_active":"2023-12-20T04:46:00.926798+0000","last_became_peered":"2023-12-20T04:46:00.926798+0000","last_unstale":"2023-12-20T04:46:00.934794+0000","last_undegraded":"2023-12-20T04:46:00.934794+0000","last_fullsized":"2023-12-20T04:46:00.934794+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:30:08.120496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913401+0000","last_change":"2023-12-20T04:46:00.510459+0000","last_active":"2023-12-20T04:46:01.913401+0000","last_peered":"2023-12-20T04:46:01.913401+0000","last_clean":"2023-12-20T04:46:01.913401+0000","last_became_active":"2023-12-20T04:46:00.510142+0000","last_became_peered":"2023-12-20T04:46:00.510142+0000","last_unstale":"2023-12-20T04:46:01.913401+0000","last_undegraded":"2023-12-20T04:46:01.913401+0000","last_fullsized":"2023-12-20T04:46:01.913401+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:54.131033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927248+0000","last_change":"2023-12-20T04:46:00.923438+0000","last_active":"2023-12-20T04:46:01.927248+0000","last_peered":"2023-12-20T04:46:01.927248+0000","last_clean":"2023-12-20T04:46:01.927248+0000","last_became_active":"2023-12-20T04:46:00.923158+0000","last_became_peered":"2023-12-20T04:46:00.923158+0000","last_unstale":"2023-12-20T04:46:01.927248+0000","last_undegraded":"2023-12-20T04:46:01.927248+0000","last_fullsized":"2023-12-20T04:46:01.927248+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:58:52.252540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913138+0000","last_change":"2023-12-20T04:46:00.510308+0000","last_active":"2023-12-20T04:46:01.913138+0000","last_peered":"2023-12-20T04:46:01.913138+0000","last_clean":"2023-12-20T04:46:01.913138+0000","last_became_active":"2023-12-20T04:46:00.509933+0000","last_became_peered":"2023-12-20T04:46:00.509933+0000","last_unstale":"2023-12-20T04:46:01.913138+0000","last_undegraded":"2023-12-20T04:46:01.913138+0000","last_fullsized":"2023-12-20T04:46:01.913138+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:46.493666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.930243+0000","last_change":"2023-12-20T04:46:00.925382+0000","last_active":"2023-12-20T04:46:01.930243+0000","last_peered":"2023-12-20T04:46:01.930243+0000","last_clean":"2023-12-20T04:46:01.930243+0000","last_became_active":"2023-12-20T04:46:00.925027+0000","last_became_peered":"2023-12-20T04:46:00.925027+0000","last_unstale":"2023-12-20T04:46:01.930243+0000","last_undegraded":"2023-12-20T04:46:01.930243+0000","last_fullsized":"2023-12-20T04:46:01.930243+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:47.193663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"122'1","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.930647+0000","last_change":"2023-12-20T04:46:00.512592+0000","last_active":"2023-12-20T04:46:00.930647+0000","last_peered":"2023-12-20T04:46:00.930647+0000","last_clean":"2023-12-20T04:46:00.930647+0000","last_became_active":"2023-12-20T04:46:00.512307+0000","last_became_peered":"2023-12-20T04:46:00.512307+0000","last_unstale":"2023-12-20T04:46:00.930647+0000","last_undegraded":"2023-12-20T04:46:00.930647+0000","last_fullsized":"2023-12-20T04:46:00.930647+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:01:23.704580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.1a","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.912978+0000","last_change":"2023-12-20T04:46:00.513258+0000","last_active":"2023-12-20T04:46:01.912978+0000","last_peered":"2023-12-20T04:46:01.912978+0000","last_clean":"2023-12-20T04:46:01.912978+0000","last_became_active":"2023-12-20T04:46:00.512979+0000","last_became_peered":"2023-12-20T04:46:00.512979+0000","last_unstale":"2023-12-20T04:46:01.912978+0000","last_undegraded":"2023-12-20T04:46:01.912978+0000","last_fullsized":"2023-12-20T04:46:01.912978+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:07:36.550183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.943628+0000","last_change":"2023-12-20T04:46:00.939877+0000","last_active":"2023-12-20T04:46:00.943628+0000","last_peered":"2023-12-20T04:46:00.943628+0000","last_clean":"2023-12-20T04:46:00.943628+0000","last_became_active":"2023-12-20T04:46:00.939074+0000","last_became_peered":"2023-12-20T04:46:00.939074+0000","last_unstale":"2023-12-20T04:46:00.943628+0000","last_undegraded":"2023-12-20T04:46:00.943628+0000","last_fullsized":"2023-12-20T04:46:00.943628+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:20.797046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:45:59.474887+0000","last_change":"2023-12-20T04:45:59.474887+0000","last_active":"2023-12-20T04:45:59.474887+0000","last_peered":"2023-12-20T04:45:59.474887+0000","last_clean":"2023-12-20T04:45:59.474887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:45:59.474887+0000","last_undegraded":"2023-12-20T04:45:59.474887+0000","last_fullsized":"2023-12-20T04:45:59.474887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.926662+0000","last_change":"2023-12-20T04:46:00.925542+0000","last_active":"2023-12-20T04:46:01.926662+0000","last_peered":"2023-12-20T04:46:01.926662+0000","last_clean":"2023-12-20T04:46:01.926662+0000","last_became_active":"2023-12-20T04:46:00.924882+0000","last_became_peered":"2023-12-20T04:46:00.924882+0000","last_unstale":"2023-12-20T04:46:01.926662+0000","last_undegraded":"2023-12-20T04:46:01.926662+0000","last_fullsized":"2023-12-20T04:46:01.926662+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:50:30.556188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.911178+0000","last_change":"2023-12-20T04:46:00.517062+0000","last_active":"2023-12-20T04:46:00.911178+0000","last_peered":"2023-12-20T04:46:00.911178+0000","last_clean":"2023-12-20T04:46:00.911178+0000","last_became_active":"2023-12-20T04:46:00.516766+0000","last_became_peered":"2023-12-20T04:46:00.516766+0000","last_unstale":"2023-12-20T04:46:00.911178+0000","last_undegraded":"2023-12-20T04:46:00.911178+0000","last_fullsized":"2023-12-20T04:46:00.911178+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:43.636745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913767+0000","last_change":"2023-12-20T04:46:00.927911+0000","last_active":"2023-12-20T04:46:01.913767+0000","last_peered":"2023-12-20T04:46:01.913767+0000","last_clean":"2023-12-20T04:46:01.913767+0000","last_became_active":"2023-12-20T04:46:00.927652+0000","last_became_peered":"2023-12-20T04:46:00.927652+0000","last_unstale":"2023-12-20T04:46:01.913767+0000","last_undegraded":"2023-12-20T04:46:01.913767+0000","last_fullsized":"2023-12-20T04:46:01.913767+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:20.797046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.912028+0000","last_change":"2023-12-20T04:46:00.514459+0000","last_active":"2023-12-20T04:46:00.912028+0000","last_peered":"2023-12-20T04:46:00.912028+0000","last_clean":"2023-12-20T04:46:00.912028+0000","last_became_active":"2023-12-20T04:46:00.514207+0000","last_became_peered":"2023-12-20T04:46:00.514207+0000","last_unstale":"2023-12-20T04:46:00.912028+0000","last_undegraded":"2023-12-20T04:46:00.912028+0000","last_fullsized":"2023-12-20T04:46:00.912028+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:41.589301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927378+0000","last_change":"2023-12-20T04:46:00.933228+0000","last_active":"2023-12-20T04:46:01.927378+0000","last_peered":"2023-12-20T04:46:01.927378+0000","last_clean":"2023-12-20T04:46:01.927378+0000","last_became_active":"2023-12-20T04:46:00.932863+0000","last_became_peered":"2023-12-20T04:46:00.932863+0000","last_unstale":"2023-12-20T04:46:01.927378+0000","last_undegraded":"2023-12-20T04:46:01.927378+0000","last_fullsized":"2023-12-20T04:46:01.927378+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:07:37.558405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.925839+0000","last_change":"2023-12-20T04:46:00.522428+0000","last_active":"2023-12-20T04:46:00.925839+0000","last_peered":"2023-12-20T04:46:00.925839+0000","last_clean":"2023-12-20T04:46:00.925839+0000","last_became_active":"2023-12-20T04:46:00.522091+0000","last_became_peered":"2023-12-20T04:46:00.522091+0000","last_unstale":"2023-12-20T04:46:00.925839+0000","last_undegraded":"2023-12-20T04:46:00.925839+0000","last_fullsized":"2023-12-20T04:46:00.925839+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:34.893107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:45:59.474887+0000","last_change":"2023-12-20T04:45:59.474887+0000","last_active":"2023-12-20T04:45:59.474887+0000","last_peered":"2023-12-20T04:45:59.474887+0000","last_clean":"2023-12-20T04:45:59.474887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:45:59.474887+0000","last_undegraded":"2023-12-20T04:45:59.474887+0000","last_fullsized":"2023-12-20T04:45:59.474887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.14","version":"122'6","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927119+0000","last_change":"2023-12-20T04:46:00.518446+0000","last_active":"2023-12-20T04:46:01.927119+0000","last_peered":"2023-12-20T04:46:01.927119+0000","last_clean":"2023-12-20T04:46:01.927119+0000","last_became_active":"2023-12-20T04:46:00.517918+0000","last_became_peered":"2023-12-20T04:46:00.517918+0000","last_unstale":"2023-12-20T04:46:01.927119+0000","last_undegraded":"2023-12-20T04:46:01.927119+0000","last_fullsized":"2023-12-20T04:46:01.927119+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:23.121194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.942284+0000","last_change":"2023-12-20T04:46:00.935394+0000","last_active":"2023-12-20T04:46:00.942284+0000","last_peered":"2023-12-20T04:46:00.942284+0000","last_clean":"2023-12-20T04:46:00.942284+0000","last_became_active":"2023-12-20T04:46:00.935140+0000","last_became_peered":"2023-12-20T04:46:00.935140+0000","last_unstale":"2023-12-20T04:46:00.942284+0000","last_undegraded":"2023-12-20T04:46:00.942284+0000","last_fullsized":"2023-12-20T04:46:00.942284+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:31:22.675050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"122'2","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.935354+0000","last_change":"2023-12-20T04:46:00.512630+0000","last_active":"2023-12-20T04:46:00.935354+0000","last_peered":"2023-12-20T04:46:00.935354+0000","last_clean":"2023-12-20T04:46:00.935354+0000","last_became_active":"2023-12-20T04:46:00.511966+0000","last_became_peered":"2023-12-20T04:46:00.511966+0000","last_unstale":"2023-12-20T04:46:00.935354+0000","last_undegraded":"2023-12-20T04:46:00.935354+0000","last_fullsized":"2023-12-20T04:46:00.935354+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:26:40.880550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.934960+0000","last_change":"2023-12-20T04:46:00.924622+0000","last_active":"2023-12-20T04:46:00.934960+0000","last_peered":"2023-12-20T04:46:00.934960+0000","last_clean":"2023-12-20T04:46:00.934960+0000","last_became_active":"2023-12-20T04:46:00.924327+0000","last_became_peered":"2023-12-20T04:46:00.924327+0000","last_unstale":"2023-12-20T04:46:00.934960+0000","last_undegraded":"2023-12-20T04:46:00.934960+0000","last_fullsized":"2023-12-20T04:46:00.934960+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:16.834340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.916241+0000","last_change":"2023-12-20T04:46:00.519325+0000","last_active":"2023-12-20T04:46:00.916241+0000","last_peered":"2023-12-20T04:46:00.916241+0000","last_clean":"2023-12-20T04:46:00.916241+0000","last_became_active":"2023-12-20T04:46:00.519061+0000","last_became_peered":"2023-12-20T04:46:00.519061+0000","last_unstale":"2023-12-20T04:46:00.916241+0000","last_undegraded":"2023-12-20T04:46:00.916241+0000","last_fullsized":"2023-12-20T04:46:00.916241+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:05:02.649351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.926641+0000","last_change":"2023-12-20T04:46:00.926925+0000","last_active":"2023-12-20T04:46:01.926641+0000","last_peered":"2023-12-20T04:46:01.926641+0000","last_clean":"2023-12-20T04:46:01.926641+0000","last_became_active":"2023-12-20T04:46:00.926578+0000","last_became_peered":"2023-12-20T04:46:00.926578+0000","last_unstale":"2023-12-20T04:46:01.926641+0000","last_undegraded":"2023-12-20T04:46:01.926641+0000","last_fullsized":"2023-12-20T04:46:01.926641+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:47.501889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.926724+0000","last_change":"2023-12-20T04:46:00.510988+0000","last_active":"2023-12-20T04:46:01.926724+0000","last_peered":"2023-12-20T04:46:01.926724+0000","last_clean":"2023-12-20T04:46:01.926724+0000","last_became_active":"2023-12-20T04:46:00.509913+0000","last_became_peered":"2023-12-20T04:46:00.509913+0000","last_unstale":"2023-12-20T04:46:01.926724+0000","last_undegraded":"2023-12-20T04:46:01.926724+0000","last_fullsized":"2023-12-20T04:46:01.926724+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:09:53.122786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.932160+0000","last_change":"2023-12-20T04:46:00.929177+0000","last_active":"2023-12-20T04:46:00.932160+0000","last_peered":"2023-12-20T04:46:00.932160+0000","last_clean":"2023-12-20T04:46:00.932160+0000","last_became_active":"2023-12-20T04:46:00.928804+0000","last_became_peered":"2023-12-20T04:46:00.928804+0000","last_unstale":"2023-12-20T04:46:00.932160+0000","last_undegraded":"2023-12-20T04:46:00.932160+0000","last_fullsized":"2023-12-20T04:46:00.932160+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:44.644967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"122'8","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.937158+0000","last_change":"2023-12-20T04:46:00.513860+0000","last_active":"2023-12-20T04:46:00.937158+0000","last_peered":"2023-12-20T04:46:00.937158+0000","last_clean":"2023-12-20T04:46:00.937158+0000","last_became_active":"2023-12-20T04:46:00.513571+0000","last_became_peered":"2023-12-20T04:46:00.513571+0000","last_unstale":"2023-12-20T04:46:00.937158+0000","last_undegraded":"2023-12-20T04:46:00.937158+0000","last_fullsized":"2023-12-20T04:46:00.937158+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:44.624915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.e","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927322+0000","last_change":"2023-12-20T04:46:00.517860+0000","last_active":"2023-12-20T04:46:01.927322+0000","last_peered":"2023-12-20T04:46:01.927322+0000","last_clean":"2023-12-20T04:46:01.927322+0000","last_became_active":"2023-12-20T04:46:00.517504+0000","last_became_peered":"2023-12-20T04:46:00.517504+0000","last_unstale":"2023-12-20T04:46:01.927322+0000","last_undegraded":"2023-12-20T04:46:01.927322+0000","last_fullsized":"2023-12-20T04:46:01.927322+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:23.637670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.930174+0000","last_change":"2023-12-20T04:46:00.915005+0000","last_active":"2023-12-20T04:46:01.930174+0000","last_peered":"2023-12-20T04:46:01.930174+0000","last_clean":"2023-12-20T04:46:01.930174+0000","last_became_active":"2023-12-20T04:46:00.913773+0000","last_became_peered":"2023-12-20T04:46:00.913773+0000","last_unstale":"2023-12-20T04:46:01.930174+0000","last_undegraded":"2023-12-20T04:46:01.930174+0000","last_fullsized":"2023-12-20T04:46:01.930174+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:53:55.732437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:45:59.474887+0000","last_change":"2023-12-20T04:45:59.474887+0000","last_active":"2023-12-20T04:45:59.474887+0000","last_peered":"2023-12-20T04:45:59.474887+0000","last_clean":"2023-12-20T04:45:59.474887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:45:59.474887+0000","last_undegraded":"2023-12-20T04:45:59.474887+0000","last_fullsized":"2023-12-20T04:45:59.474887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913179+0000","last_change":"2023-12-20T04:46:00.506704+0000","last_active":"2023-12-20T04:46:01.913179+0000","last_peered":"2023-12-20T04:46:01.913179+0000","last_clean":"2023-12-20T04:46:01.913179+0000","last_became_active":"2023-12-20T04:46:00.506380+0000","last_became_peered":"2023-12-20T04:46:00.506380+0000","last_unstale":"2023-12-20T04:46:01.913179+0000","last_undegraded":"2023-12-20T04:46:01.913179+0000","last_fullsized":"2023-12-20T04:46:01.913179+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:50:29.547966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.942843+0000","last_change":"2023-12-20T04:46:00.941229+0000","last_active":"2023-12-20T04:46:00.942843+0000","last_peered":"2023-12-20T04:46:00.942843+0000","last_clean":"2023-12-20T04:46:00.942843+0000","last_became_active":"2023-12-20T04:46:00.940970+0000","last_became_peered":"2023-12-20T04:46:00.940970+0000","last_unstale":"2023-12-20T04:46:00.942843+0000","last_undegraded":"2023-12-20T04:46:00.942843+0000","last_fullsized":"2023-12-20T04:46:00.942843+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:30.606203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913172+0000","last_change":"2023-12-20T04:46:00.510088+0000","last_active":"2023-12-20T04:46:01.913172+0000","last_peered":"2023-12-20T04:46:01.913172+0000","last_clean":"2023-12-20T04:46:01.913172+0000","last_became_active":"2023-12-20T04:46:00.509787+0000","last_became_peered":"2023-12-20T04:46:00.509787+0000","last_unstale":"2023-12-20T04:46:01.913172+0000","last_undegraded":"2023-12-20T04:46:01.913172+0000","last_fullsized":"2023-12-20T04:46:01.913172+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:34:05.851006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.942806+0000","last_change":"2023-12-20T04:46:00.940247+0000","last_active":"2023-12-20T04:46:00.942806+0000","last_peered":"2023-12-20T04:46:00.942806+0000","last_clean":"2023-12-20T04:46:00.942806+0000","last_became_active":"2023-12-20T04:46:00.939969+0000","last_became_peered":"2023-12-20T04:46:00.939969+0000","last_unstale":"2023-12-20T04:46:00.942806+0000","last_undegraded":"2023-12-20T04:46:00.942806+0000","last_fullsized":"2023-12-20T04:46:00.942806+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:26:35.868259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.930228+0000","last_change":"2023-12-20T04:46:00.515932+0000","last_active":"2023-12-20T04:46:01.930228+0000","last_peered":"2023-12-20T04:46:01.930228+0000","last_clean":"2023-12-20T04:46:01.930228+0000","last_became_active":"2023-12-20T04:46:00.515668+0000","last_became_peered":"2023-12-20T04:46:00.515668+0000","last_unstale":"2023-12-20T04:46:01.930228+0000","last_undegraded":"2023-12-20T04:46:01.930228+0000","last_fullsized":"2023-12-20T04:46:01.930228+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:29.597980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.937838+0000","last_change":"2023-12-20T04:46:00.934227+0000","last_active":"2023-12-20T04:46:00.937838+0000","last_peered":"2023-12-20T04:46:00.937838+0000","last_clean":"2023-12-20T04:46:00.937838+0000","last_became_active":"2023-12-20T04:46:00.933953+0000","last_became_peered":"2023-12-20T04:46:00.933953+0000","last_unstale":"2023-12-20T04:46:00.937838+0000","last_undegraded":"2023-12-20T04:46:00.937838+0000","last_fullsized":"2023-12-20T04:46:00.937838+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:04:55.770021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.930110+0000","last_change":"2023-12-20T04:46:00.517339+0000","last_active":"2023-12-20T04:46:01.930110+0000","last_peered":"2023-12-20T04:46:01.930110+0000","last_clean":"2023-12-20T04:46:01.930110+0000","last_became_active":"2023-12-20T04:46:00.516860+0000","last_became_peered":"2023-12-20T04:46:00.516860+0000","last_unstale":"2023-12-20T04:46:01.930110+0000","last_undegraded":"2023-12-20T04:46:01.930110+0000","last_fullsized":"2023-12-20T04:46:01.930110+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:32:34.843771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913155+0000","last_change":"2023-12-20T04:46:00.510020+0000","last_active":"2023-12-20T04:46:01.913155+0000","last_peered":"2023-12-20T04:46:01.913155+0000","last_clean":"2023-12-20T04:46:01.913155+0000","last_became_active":"2023-12-20T04:46:00.508089+0000","last_became_peered":"2023-12-20T04:46:00.508089+0000","last_unstale":"2023-12-20T04:46:01.913155+0000","last_undegraded":"2023-12-20T04:46:01.913155+0000","last_fullsized":"2023-12-20T04:46:01.913155+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:58:51.244318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913562+0000","last_change":"2023-12-20T04:46:00.920003+0000","last_active":"2023-12-20T04:46:01.913562+0000","last_peered":"2023-12-20T04:46:01.913562+0000","last_clean":"2023-12-20T04:46:01.913562+0000","last_became_active":"2023-12-20T04:46:00.919107+0000","last_became_peered":"2023-12-20T04:46:00.919107+0000","last_unstale":"2023-12-20T04:46:01.913562+0000","last_undegraded":"2023-12-20T04:46:01.913562+0000","last_fullsized":"2023-12-20T04:46:01.913562+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:30:20.820476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.930440+0000","last_change":"2023-12-20T04:46:00.510596+0000","last_active":"2023-12-20T04:46:01.930440+0000","last_peered":"2023-12-20T04:46:01.930440+0000","last_clean":"2023-12-20T04:46:01.930440+0000","last_became_active":"2023-12-20T04:46:00.509375+0000","last_became_peered":"2023-12-20T04:46:00.509375+0000","last_unstale":"2023-12-20T04:46:01.930440+0000","last_undegraded":"2023-12-20T04:46:01.930440+0000","last_fullsized":"2023-12-20T04:46:01.930440+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:52:12.651653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.931087+0000","last_change":"2023-12-20T04:46:00.924232+0000","last_active":"2023-12-20T04:46:00.931087+0000","last_peered":"2023-12-20T04:46:00.931087+0000","last_clean":"2023-12-20T04:46:00.931087+0000","last_became_active":"2023-12-20T04:46:00.923596+0000","last_became_peered":"2023-12-20T04:46:00.923596+0000","last_unstale":"2023-12-20T04:46:00.931087+0000","last_undegraded":"2023-12-20T04:46:00.931087+0000","last_fullsized":"2023-12-20T04:46:00.931087+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:08:50.075615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913630+0000","last_change":"2023-12-20T04:46:00.521573+0000","last_active":"2023-12-20T04:46:01.913630+0000","last_peered":"2023-12-20T04:46:01.913630+0000","last_clean":"2023-12-20T04:46:01.913630+0000","last_became_active":"2023-12-20T04:46:00.521000+0000","last_became_peered":"2023-12-20T04:46:00.521000+0000","last_unstale":"2023-12-20T04:46:01.913630+0000","last_undegraded":"2023-12-20T04:46:01.913630+0000","last_fullsized":"2023-12-20T04:46:01.913630+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:10:06.227641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.925376+0000","last_change":"2023-12-20T04:46:00.921513+0000","last_active":"2023-12-20T04:46:00.925376+0000","last_peered":"2023-12-20T04:46:00.925376+0000","last_clean":"2023-12-20T04:46:00.925376+0000","last_became_active":"2023-12-20T04:46:00.921258+0000","last_became_peered":"2023-12-20T04:46:00.921258+0000","last_unstale":"2023-12-20T04:46:00.925376+0000","last_undegraded":"2023-12-20T04:46:00.925376+0000","last_fullsized":"2023-12-20T04:46:00.925376+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:47.193663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913332+0000","last_change":"2023-12-20T04:46:00.507249+0000","last_active":"2023-12-20T04:46:01.913332+0000","last_peered":"2023-12-20T04:46:01.913332+0000","last_clean":"2023-12-20T04:46:01.913332+0000","last_became_active":"2023-12-20T04:46:00.506929+0000","last_became_peered":"2023-12-20T04:46:00.506929+0000","last_unstale":"2023-12-20T04:46:01.913332+0000","last_undegraded":"2023-12-20T04:46:01.913332+0000","last_fullsized":"2023-12-20T04:46:01.913332+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:11:16.258499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:45:59.474887+0000","last_change":"2023-12-20T04:45:59.474887+0000","last_active":"2023-12-20T04:45:59.474887+0000","last_peered":"2023-12-20T04:45:59.474887+0000","last_clean":"2023-12-20T04:45:59.474887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:45:59.474887+0000","last_undegraded":"2023-12-20T04:45:59.474887+0000","last_fullsized":"2023-12-20T04:45:59.474887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.932411+0000","last_change":"2023-12-20T04:46:00.926423+0000","last_active":"2023-12-20T04:46:00.932411+0000","last_peered":"2023-12-20T04:46:00.932411+0000","last_clean":"2023-12-20T04:46:00.932411+0000","last_became_active":"2023-12-20T04:46:00.926129+0000","last_became_peered":"2023-12-20T04:46:00.926129+0000","last_unstale":"2023-12-20T04:46:00.932411+0000","last_undegraded":"2023-12-20T04:46:00.932411+0000","last_fullsized":"2023-12-20T04:46:00.932411+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:55:48.120147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913265+0000","last_change":"2023-12-20T04:46:00.517528+0000","last_active":"2023-12-20T04:46:01.913265+0000","last_peered":"2023-12-20T04:46:01.913265+0000","last_clean":"2023-12-20T04:46:01.913265+0000","last_became_active":"2023-12-20T04:46:00.517243+0000","last_became_peered":"2023-12-20T04:46:00.517243+0000","last_unstale":"2023-12-20T04:46:01.913265+0000","last_undegraded":"2023-12-20T04:46:01.913265+0000","last_fullsized":"2023-12-20T04:46:01.913265+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:34.893107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.926885+0000","last_change":"2023-12-20T04:46:00.928554+0000","last_active":"2023-12-20T04:46:01.926885+0000","last_peered":"2023-12-20T04:46:01.926885+0000","last_clean":"2023-12-20T04:46:01.926885+0000","last_became_active":"2023-12-20T04:46:00.928283+0000","last_became_peered":"2023-12-20T04:46:00.928283+0000","last_unstale":"2023-12-20T04:46:01.926885+0000","last_undegraded":"2023-12-20T04:46:01.926885+0000","last_fullsized":"2023-12-20T04:46:01.926885+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:34:06.859228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:45:59.474887+0000","last_change":"2023-12-20T04:45:59.474887+0000","last_active":"2023-12-20T04:45:59.474887+0000","last_peered":"2023-12-20T04:45:59.474887+0000","last_clean":"2023-12-20T04:45:59.474887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:45:59.474887+0000","last_undegraded":"2023-12-20T04:45:59.474887+0000","last_fullsized":"2023-12-20T04:45:59.474887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.932098+0000","last_change":"2023-12-20T04:46:00.928204+0000","last_active":"2023-12-20T04:46:00.932098+0000","last_peered":"2023-12-20T04:46:00.932098+0000","last_clean":"2023-12-20T04:46:00.932098+0000","last_became_active":"2023-12-20T04:46:00.927898+0000","last_became_peered":"2023-12-20T04:46:00.927898+0000","last_unstale":"2023-12-20T04:46:00.932098+0000","last_undegraded":"2023-12-20T04:46:00.932098+0000","last_fullsized":"2023-12-20T04:46:00.932098+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:42.597523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.916582+0000","last_change":"2023-12-20T04:46:00.524160+0000","last_active":"2023-12-20T04:46:00.916582+0000","last_peered":"2023-12-20T04:46:00.916582+0000","last_clean":"2023-12-20T04:46:00.916582+0000","last_became_active":"2023-12-20T04:46:00.523650+0000","last_became_peered":"2023-12-20T04:46:00.523650+0000","last_unstale":"2023-12-20T04:46:00.916582+0000","last_undegraded":"2023-12-20T04:46:00.916582+0000","last_fullsized":"2023-12-20T04:46:00.916582+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:21.800243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:46:00.483109+0000","last_change":"2023-12-20T04:46:00.483109+0000","last_active":"2023-12-20T04:46:00.483109+0000","last_peered":"2023-12-20T04:46:00.483109+0000","last_clean":"2023-12-20T04:46:00.483109+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:46:00.483109+0000","last_undegraded":"2023-12-20T04:46:00.483109+0000","last_fullsized":"2023-12-20T04:46:00.483109+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.915854+0000","last_change":"2023-12-20T04:46:00.518737+0000","last_active":"2023-12-20T04:46:00.915854+0000","last_peered":"2023-12-20T04:46:00.915854+0000","last_clean":"2023-12-20T04:46:00.915854+0000","last_became_active":"2023-12-20T04:46:00.518466+0000","last_became_peered":"2023-12-20T04:46:00.518466+0000","last_unstale":"2023-12-20T04:46:00.915854+0000","last_undegraded":"2023-12-20T04:46:00.915854+0000","last_fullsized":"2023-12-20T04:46:00.915854+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:20:38.437718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.944012+0000","last_change":"2023-12-20T04:46:00.942173+0000","last_active":"2023-12-20T04:46:00.944012+0000","last_peered":"2023-12-20T04:46:00.944012+0000","last_clean":"2023-12-20T04:46:00.944012+0000","last_became_active":"2023-12-20T04:46:00.941922+0000","last_became_peered":"2023-12-20T04:46:00.941922+0000","last_unstale":"2023-12-20T04:46:00.944012+0000","last_undegraded":"2023-12-20T04:46:00.944012+0000","last_fullsized":"2023-12-20T04:46:00.944012+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:24:38.748808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T04:45:59.474887+0000","last_change":"2023-12-20T04:45:59.474887+0000","last_active":"2023-12-20T04:45:59.474887+0000","last_peered":"2023-12-20T04:45:59.474887+0000","last_clean":"2023-12-20T04:45:59.474887+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T04:45:59.474887+0000","last_undegraded":"2023-12-20T04:45:59.474887+0000","last_fullsized":"2023-12-20T04:45:59.474887+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"111'3080","reported_seq":3446,"reported_epoch":118,"state":"active+clean","last_fresh":"2023-12-20T04:45:56.616936+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:45:56.616936+0000","last_peered":"2023-12-20T04:45:56.616936+0000","last_clean":"2023-12-20T04:45:56.616936+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:45:56.616936+0000","last_undegraded":"2023-12-20T04:45:56.616936+0000","last_fullsized":"2023-12-20T04:45:56.616936+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3080,"log_dups_size":0,"ondisk_log_size":3080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":829,"num_read_kb":1583,"num_write":301,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":92,"acting":92,"num_store_stats":6},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2180,"num_objects":17,"num_object_clones":0,"num_object_copies":34,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":17,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":38,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":12,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":49152,"data_stored":3960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":32,"ondisk_log_size":32,"up":50,"acting":50,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":829,"num_read_kb":1583,"num_write":301,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3080,"ondisk_log_size":3080,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575057,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33052,"kb_used_data":2004,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338788,"statfs":{"total":96636764160,"available":96602918912,"internally_reserved":0,"allocated":2052096,"data_stored":1280286,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66600000000000004,"15min":0.70099999999999996},"min":{"1min":0.47599999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.87,"5min":1.0580000000000001,"15min":1.0600000000000001},"last":0.87},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.45700000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91500000000000004,"5min":0.93100000000000005,"15min":1.115},"last":0.51300000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68899999999999995,"15min":0.72199999999999998},"min":{"1min":0.47499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93000000000000005,"5min":1.095,"15min":1.095},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69499999999999995,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.377,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.95099999999999996,"15min":1.1080000000000001},"last":0.67500000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67800000000000005,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1.075,"15min":1.075},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.747},"min":{"1min":0.52800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90000000000000002,"5min":1.048,"15min":1.1000000000000001},"last":0.70399999999999996}]},{"osd":3,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.72099999999999997},"min":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96399999999999997,"5min":1.03,"15min":1.03},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73599999999999999,"15min":0.755},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96399999999999997,"5min":1.012,"15min":1.149},"last":0.66000000000000003}]},{"osd":4,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.34599999999999997},"max":{"1min":0.77100000000000002,"5min":0.90700000000000003,"15min":1.0109999999999999},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82199999999999995,"5min":0.97599999999999998,"15min":1.0469999999999999},"last":0.53200000000000003}]},{"osd":5,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40400000000000003},"max":{"1min":0.91800000000000004,"5min":1.089,"15min":1.0900000000000001},"last":0.91800000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68899999999999995,"15min":0.66800000000000004},"min":{"1min":0.48399999999999999,"5min":0.436,"15min":0.432},"max":{"1min":0.91500000000000004,"5min":0.98799999999999999,"15min":1.071},"last":0.63100000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.51000000000000001,"5min":0.46800000000000003,"15min":0.443},"max":{"1min":0.89200000000000002,"5min":1.093,"15min":1.1279999999999999},"last":0.55400000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.49399999999999999,"5min":0.46100000000000002,"15min":0.44400000000000001},"max":{"1min":0.91700000000000004,"5min":1.125,"15min":1.125},"last":0.72299999999999998}]}]},{"osd":6,"up_from":13,"seq":55834575057,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":36580,"kb_used_data":1436,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335260,"statfs":{"total":96636764160,"available":96599306240,"internally_reserved":0,"allocated":1470464,"data_stored":695605,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67600000000000005,"15min":0.72199999999999998},"min":{"1min":0.40899999999999997,"5min":0.33800000000000002,"15min":0.27100000000000002},"max":{"1min":0.86699999999999999,"5min":1.1140000000000001,"15min":1.2649999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68300000000000005,"15min":0.72199999999999998},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1910000000000001},"last":0.63100000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.65500000000000003,"15min":0.70299999999999996},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98599999999999999,"5min":1.016,"15min":1.016},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69099999999999995,"15min":0.72899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.76300000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.40899999999999997},"max":{"1min":1.0229999999999999,"5min":1.0580000000000001,"15min":1.216},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69599999999999995,"15min":0.73399999999999999},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.38300000000000001},"max":{"1min":1.0369999999999999,"5min":1.141,"15min":1.141},"last":0.71699999999999997}]},{"osd":3,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73799999999999999,"15min":0.76200000000000001},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":1.093,"15min":1.109},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.73399999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.73799999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.36399999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.1040000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.36899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93700000000000006,"15min":0.995},"last":0.84899999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.621,"15min":0.61599999999999999},"min":{"1min":0.44,"5min":0.307,"15min":0.307},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.1519999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.98599999999999999,"5min":1,"15min":1.306},"last":0.94799999999999995}]},{"osd":7,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71799999999999997,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.49099999999999999,"15min":0.40400000000000003},"max":{"1min":1.0069999999999999,"5min":1.0129999999999999,"15min":1.0149999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.47299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.872}]}]},{"osd":5,"up_from":13,"seq":55834575059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36632,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335208,"statfs":{"total":96636764160,"available":96599252992,"internally_reserved":0,"allocated":1523712,"data_stored":720842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68300000000000005,"15min":0.70299999999999996},"min":{"1min":0.60399999999999998,"5min":0.36199999999999999,"15min":0.29999999999999999},"max":{"1min":0.90300000000000002,"5min":0.97199999999999998,"15min":1.05},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.53400000000000003,"5min":0.35899999999999999,"15min":0.34300000000000003},"max":{"1min":0.96799999999999997,"5min":0.96899999999999997,"15min":0.98599999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.54900000000000004,"5min":0.443,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.98399999999999999,"15min":1.0600000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.70699999999999996,"15min":0.73299999999999998},"min":{"1min":0.57699999999999996,"5min":0.437,"15min":0.38400000000000001},"max":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":1.0309999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.71799999999999997,"15min":0.73199999999999998},"min":{"1min":0.52200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":1.014},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.004},"last":0.79100000000000004}]},{"osd":3,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73799999999999999,"15min":0.74299999999999999},"min":{"1min":0.61499999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92500000000000004,"5min":0.99199999999999999,"15min":1.018},"last":0.75},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.55400000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.002,"5min":1.008,"15min":1.0469999999999999},"last":0.76700000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.42099999999999999,"15min":0.35099999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64100000000000001,"15min":0.627},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.39600000000000002},"max":{"1min":0.89600000000000002,"5min":0.98699999999999999,"15min":1.077},"last":0.81000000000000005}]},{"osd":6,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.495,"5min":0.47599999999999998,"15min":0.39900000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":1.056},"last":0.871},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.45400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":0.999},"last":0.90800000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.442},"max":{"1min":0.89400000000000002,"5min":1.0429999999999999,"15min":1.141},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.754,"15min":0.751},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.47899999999999998},"max":{"1min":0.93500000000000005,"5min":1.012,"15min":1.0780000000000001},"last":0.92500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575060,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58792,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313048,"statfs":{"total":96636764160,"available":96576561152,"internally_reserved":0,"allocated":1540096,"data_stored":722460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.42999999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96899999999999997,"5min":1.024,"15min":1.5509999999999999},"last":0.503},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.41599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97799999999999998,"5min":0.98599999999999999,"15min":1.633},"last":0.70299999999999996}]},{"osd":1,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.68899999999999995,"15min":0.71899999999999997},"min":{"1min":0.52200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.798},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.58699999999999997,"5min":0.372,"15min":0.372},"max":{"1min":0.88400000000000001,"5min":1.036,"15min":2.2930000000000001},"last":0.77400000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72699999999999998,"15min":0.74299999999999999},"min":{"1min":0.61099999999999999,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.98399999999999999,"5min":1.0209999999999999,"15min":1.728},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.72499999999999998,"15min":0.748},"min":{"1min":0.5,"5min":0.46899999999999997,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":1.0289999999999999,"15min":1.6619999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70099999999999996,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98999999999999999,"5min":1.004,"15min":1.5760000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73599999999999999,"15min":0.76100000000000001},"min":{"1min":0.56899999999999995,"5min":0.372,"15min":0.372},"max":{"1min":0.95599999999999996,"5min":0.98399999999999999,"15min":1.7649999999999999},"last":0.746}]},{"osd":5,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.46899999999999997,"5min":0.36599999999999999,"15min":0.36099999999999999},"max":{"1min":0.85999999999999999,"5min":1.1120000000000001,"15min":1.456},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.623,"15min":0.621},"min":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.33800000000000002},"max":{"1min":0.97399999999999998,"5min":1.077,"15min":1.589},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.47599999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.998,"5min":1.006,"15min":1.6739999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47499999999999998,"5min":0.41899999999999998,"15min":0.41699999999999998},"max":{"1min":0.94099999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.65700000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.38400000000000001},"max":{"1min":0.91600000000000004,"5min":1.036,"15min":1.655},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.54400000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":1.052,"15min":1.125},"last":0.68100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36632,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335208,"statfs":{"total":96636764160,"available":96599252992,"internally_reserved":0,"allocated":1523712,"data_stored":716058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.436,"15min":0.45200000000000001},"min":{"1min":0.313,"5min":0.26100000000000001,"15min":0.23200000000000001},"max":{"1min":0.58999999999999997,"5min":0.81699999999999995,"15min":0.875},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.48599999999999999,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76300000000000001,"5min":0.77600000000000002,"15min":0.80800000000000005},"last":0.36099999999999999}]},{"osd":1,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46400000000000002,"15min":0.47099999999999997},"min":{"1min":0.36599999999999999,"5min":0.29299999999999998,"15min":0.27400000000000002},"max":{"1min":0.63800000000000001,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.54800000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.28399999999999997},"max":{"1min":0.67100000000000004,"5min":0.74399999999999999,"15min":0.88900000000000001},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56299999999999994,"15min":0.57899999999999996},"min":{"1min":0.439,"5min":0.374,"15min":0.318},"max":{"1min":0.69799999999999995,"5min":0.80700000000000005,"15min":0.88700000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.40500000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.93300000000000005},"last":0.433}]},{"osd":4,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.73999999999999999,"15min":0.77900000000000003},"min":{"1min":0.51800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0880000000000001,"5min":1.0960000000000001,"15min":1.131},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.75700000000000001,"15min":0.79700000000000004},"min":{"1min":0.51500000000000001,"5min":0.50700000000000001,"15min":0.41599999999999998},"max":{"1min":1.069,"5min":1.198,"15min":1.4059999999999999},"last":0.82699999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77300000000000002,"15min":0.80400000000000005},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":0.99099999999999999,"5min":1.181,"15min":1.3300000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.63800000000000001,"5min":0.503,"15min":0.38400000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.252},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.73499999999999999,"15min":0.78700000000000003},"min":{"1min":0.53800000000000003,"5min":0.49099999999999999,"15min":0.45600000000000002},"max":{"1min":1.016,"5min":1.0660000000000001,"15min":1.2050000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.77300000000000002,"15min":0.81100000000000005},"min":{"1min":0.58899999999999997,"5min":0.436,"15min":0.436},"max":{"1min":1.071,"5min":1.133,"15min":1.2929999999999999},"last":0.78400000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.74199999999999999,"15min":0.78600000000000003},"min":{"1min":0.67000000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0029999999999999,"5min":1.0249999999999999,"15min":1.1699999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.76300000000000001,"15min":0.81100000000000005},"min":{"1min":0.60599999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0629999999999999,"5min":1.145,"15min":1.29},"last":0.79800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575058,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40716,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331124,"statfs":{"total":96636764160,"available":96595070976,"internally_reserved":0,"allocated":1511424,"data_stored":714506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40999999999999998,"15min":0.438},"min":{"1min":0.28399999999999997,"5min":0.216,"15min":0.216},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46800000000000003,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79800000000000004,"5min":0.92200000000000004,"15min":1.0589999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45400000000000001,"15min":0.47199999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.876},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.5,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32000000000000001,"15min":0.28100000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.40400000000000003}]},{"osd":3,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69699999999999995,"5min":0.89800000000000002,"15min":0.93899999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76600000000000001,"5min":0.91600000000000004,"15min":0.95599999999999996},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73099999999999998,"15min":0.77500000000000002},"min":{"1min":0.495,"5min":0.46500000000000002,"15min":0.45200000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.1799999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73799999999999999,"15min":0.77500000000000002},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.55000000000000004}]},{"osd":5,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.77000000000000002},"min":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.751,"15min":0.79600000000000004},"min":{"1min":0.54400000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.56899999999999995}]},{"osd":6,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.752,"15min":0.78700000000000003},"min":{"1min":0.48799999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.79000000000000004},"min":{"1min":0.54500000000000004,"5min":0.502,"15min":0.46100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.147},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78000000000000003,"15min":0.81799999999999995},"min":{"1min":0.54500000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78400000000000003,"15min":0.84299999999999997},"min":{"1min":0.59399999999999997,"5min":0.52400000000000002,"15min":0.504},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.66100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575060,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50568,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321272,"statfs":{"total":96636764160,"available":96584982528,"internally_reserved":0,"allocated":1507328,"data_stored":714472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42099999999999999,"15min":0.441},"min":{"1min":0.23400000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59399999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47199999999999998,"15min":0.49099999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.92100000000000004},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67500000000000004,"5min":0.81399999999999995,"15min":1.091},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.502,"15min":0.54100000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71199999999999997,"5min":0.85899999999999999,"15min":0.875},"last":0.45300000000000001}]},{"osd":3,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.59499999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":0.93600000000000005,"15min":0.96499999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.52800000000000002},"min":{"1min":0.33300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63900000000000001,"5min":0.82399999999999995,"15min":1.2410000000000001},"last":0.436}]},{"osd":4,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70199999999999996,"15min":0.76000000000000001},"min":{"1min":0.45900000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0209999999999999,"5min":1.1579999999999999,"15min":1.27},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73499999999999999,"15min":0.78000000000000003},"min":{"1min":0.49399999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.0409999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73999999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.73799999999999999,"15min":0.77200000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0800000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72799999999999998,"15min":0.78100000000000003},"min":{"1min":0.52500000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0649999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.72299999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.74099999999999999,"15min":0.78900000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.95899999999999996,"5min":1.232,"15min":1.232},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73099999999999998,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":1.252,"15min":1.252},"last":0.75600000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78400000000000003,"15min":0.82799999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.002,"5min":1.306,"15min":1.306},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.77300000000000002,"15min":0.82799999999999996},"min":{"1min":0.54600000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91900000000000004,"5min":1.0860000000000001,"15min":1.3440000000000001},"last":0.76800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29020,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342820,"statfs":{"total":96636764160,"available":96607047680,"internally_reserved":0,"allocated":2117632,"data_stored":1311722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.43099999999999999,"15min":0.44800000000000001},"min":{"1min":0.26000000000000001,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":0.72299999999999998,"5min":0.83699999999999997,"15min":2.5640000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.496,"15min":0.53800000000000003},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63900000000000001,"5min":0.72499999999999998,"15min":7.8860000000000001},"last":0.433}]},{"osd":2,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.498,"15min":0.52100000000000002},"min":{"1min":0.33000000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68100000000000005,"5min":0.76300000000000001,"15min":2.528},"last":0.5},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53400000000000003,"15min":0.55200000000000005},"min":{"1min":0.37,"5min":0.318,"15min":0.318},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":1.2989999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.0350000000000001,"5min":3.0350000000000001,"15min":3.1000000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58099999999999996,"15min":0.626},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":10.49},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.75,"15min":0.79000000000000004},"min":{"1min":0.59799999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":3.0659999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.77600000000000002,"15min":0.80800000000000005},"min":{"1min":0.59899999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":5.2460000000000004},"last":0.65400000000000003}]},{"osd":5,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76700000000000002,"15min":0.79800000000000004},"min":{"1min":0.57799999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.073,"5min":1.1679999999999999,"15min":1.3109999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.624,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.76600000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.78500000000000003,"15min":0.82499999999999996},"min":{"1min":0.65200000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.345,"5min":1.345,"15min":5.2839999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.79200000000000004,"15min":0.81200000000000006},"min":{"1min":0.67300000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2529999999999999,"5min":1.306,"15min":1.306},"last":0.64000000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.80900000000000005,"15min":0.82899999999999996},"min":{"1min":0.69099999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1180000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.80100000000000005,"15min":0.81299999999999994},"min":{"1min":0.63300000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.276},"last":0.66700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:46:05.078 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:46:05.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.259+0000 7f05b802d640 1 Processor -- start 2023-12-20T04:46:05.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.259+0000 7f05b802d640 1 -- start start 2023-12-20T04:46:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b802d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05b00b0d00 0x7f05b0164a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:05.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b802d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b00711d0 con 0x7f05b01599e0 2023-12-20T04:46:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b802d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b00c98c0 con 0x7f05b00aece0 2023-12-20T04:46:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b802d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b00c9ba0 con 0x7f05b00b0d00 2023-12-20T04:46:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b65ca640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05b00b0d00 0x7f05b0164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b65ca640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f05b00b0d00 0x7f05b0164a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37888/0 (socket says 172.21.15.32:37888) 2023-12-20T04:46:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b65ca640 1 -- 172.21.15.32:0/2469888710 learned_addr learned my addr 172.21.15.32:0/2469888710 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b5dc9640 1 --1- 172.21.15.32:0/2469888710 >> v1:172.21.15.32:6789/0 conn(0x7f05b01599e0 0x7f05b00c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36116/0 (socket says 172.21.15.32:36116) 2023-12-20T04:46:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b65ca640 1 -- 172.21.15.32:0/2469888710 >> v1:172.21.15.39:6789/0 conn(0x7f05b00aece0 legacy=0x7f05b00b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b65ca640 1 -- 172.21.15.32:0/2469888710 >> v1:172.21.15.32:6789/0 conn(0x7f05b01599e0 legacy=0x7f05b00c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b65ca640 1 -- 172.21.15.32:0/2469888710 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b00c9e80 con 0x7f05b00b0d00 2023-12-20T04:46:05.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.263+0000 7f05b4dc7640 1 -- 172.21.15.32:0/2469888710 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1193543910 0 0) 0x7f05b00711d0 con 0x7f05b01599e0 2023-12-20T04:46:05.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b65ca640 1 --2- 172.21.15.32:0/2469888710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b00b0d00 0x7f05b0164a70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f05a800d150 tx=0x7f05a801ec10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c1da6b6d4e1f2bd7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:05.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b4dc7640 1 -- 172.21.15.32:0/2469888710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a800b2d0 con 0x7f05b00b0d00 2023-12-20T04:46:05.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b4dc7640 1 -- 172.21.15.32:0/2469888710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a800b430 con 0x7f05b00b0d00 2023-12-20T04:46:05.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b4dc7640 1 -- 172.21.15.32:0/2469888710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80a7af0 con 0x7f05b00b0d00 2023-12-20T04:46:05.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 -- 172.21.15.32:0/2469888710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b00b0d00 msgr2=0x7f05b0164a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 --2- 172.21.15.32:0/2469888710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b00b0d00 0x7f05b0164a70 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f05a800d150 tx=0x7f05a801ec10 comp rx=0 tx=0).stop 2023-12-20T04:46:05.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 -- 172.21.15.32:0/2469888710 shutdown_connections 2023-12-20T04:46:05.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 --2- 172.21.15.32:0/2469888710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b00b0d00 0x7f05b0164a70 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 -- 172.21.15.32:0/2469888710 >> 172.21.15.32:0/2469888710 conn(0x7f05b006a320 msgr2=0x7f05b00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:05.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 -- 172.21.15.32:0/2469888710 shutdown_connections 2023-12-20T04:46:05.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 -- 172.21.15.32:0/2469888710 wait complete. 2023-12-20T04:46:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 Processor -- start 2023-12-20T04:46:05.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 -- start start 2023-12-20T04:46:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b00aece0 0x7f05b014e830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b00b0d00 0x7f05b00c8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05b01599e0 0x7f05b00b2380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b00c6820 con 0x7f05b00aece0 2023-12-20T04:46:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b00c5c10 con 0x7f05b00b0d00 2023-12-20T04:46:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b802d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b00c5ef0 con 0x7f05b01599e0 2023-12-20T04:46:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.267+0000 7f05b5dc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b00aece0 0x7f05b014e830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b5dc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b00aece0 0x7f05b014e830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37900/0 (socket says 172.21.15.32:37900) 2023-12-20T04:46:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b5dc9640 1 -- 172.21.15.32:0/4282542699 learned_addr learned my addr 172.21.15.32:0/4282542699 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b55c8640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b00b0d00 0x7f05b00c8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:05.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b65ca640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05b01599e0 0x7f05b00b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b5dc9640 1 -- 172.21.15.32:0/4282542699 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05b01599e0 msgr2=0x7f05b00b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b5dc9640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05b01599e0 0x7f05b00b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b5dc9640 1 -- 172.21.15.32:0/4282542699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b00b0d00 msgr2=0x7f05b00c8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b5dc9640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b00b0d00 0x7f05b00c8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b5dc9640 1 -- 172.21.15.32:0/4282542699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a800bd40 con 0x7f05b00aece0 2023-12-20T04:46:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b5dc9640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b00aece0 0x7f05b014e830 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00c3f0 tx=0x7f05ac00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05a67fc640 1 -- 172.21.15.32:0/4282542699 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0074e0 con 0x7f05b00aece0 2023-12-20T04:46:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05a67fc640 1 -- 172.21.15.32:0/4282542699 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05ac007660 con 0x7f05b00aece0 2023-12-20T04:46:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b802d640 1 -- 172.21.15.32:0/4282542699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b00c61d0 con 0x7f05b00aece0 2023-12-20T04:46:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05a67fc640 1 -- 172.21.15.32:0/4282542699 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05ac0059d0 con 0x7f05b00aece0 2023-12-20T04:46:05.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b802d640 1 -- 172.21.15.32:0/4282542699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b014ed70 con 0x7f05b00aece0 2023-12-20T04:46:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05b802d640 1 -- 172.21.15.32:0/4282542699 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b004e000 con 0x7f05b00aece0 2023-12-20T04:46:05.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.271+0000 7f05a67fc640 1 -- 172.21.15.32:0/4282542699 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f05ac005b30 con 0x7f05b00aece0 2023-12-20T04:46:05.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.275+0000 7f05a67fc640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0594065b80 0x7f0594068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:05.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.275+0000 7f05a67fc640 1 -- 172.21.15.32:0/4282542699 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f05ac082930 con 0x7f05b00aece0 2023-12-20T04:46:05.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.275+0000 7f05b55c8640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0594065b80 0x7f0594068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:05.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.275+0000 7f05b55c8640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0594065b80 0x7f0594068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f05b00ae9e0 tx=0x7f05a0009040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:05.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.275+0000 7f05a67fc640 1 -- 172.21.15.32:0/4282542699 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f05ac0501f0 con 0x7f05b00aece0 2023-12-20T04:46:05.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.435+0000 7f05b802d640 1 -- 172.21.15.32:0/4282542699 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f05b0159dc0 con 0x7f0594065b80 2023-12-20T04:46:05.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.443+0000 7f05a67fc640 1 -- 172.21.15.32:0/4282542699 <== mgr.7465 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289367 (secure 0 0 0) 0x7f05a00014f0 con 0x7f0594065b80 2023-12-20T04:46:05.443 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:46:05.445 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:46:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 -- 172.21.15.32:0/4282542699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0594065b80 msgr2=0x7f0594068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0594065b80 0x7f0594068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f05b00ae9e0 tx=0x7f05a0009040 comp rx=0 tx=0).stop 2023-12-20T04:46:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 -- 172.21.15.32:0/4282542699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b00aece0 msgr2=0x7f05b014e830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b00aece0 0x7f05b014e830 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7f05ac00c3f0 tx=0x7f05ac00c8b0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 -- 172.21.15.32:0/4282542699 shutdown_connections 2023-12-20T04:46:05.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0594065b80 0x7f0594068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05b01599e0 0x7f05b00b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b00b0d00 0x7f05b00c8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 --2- 172.21.15.32:0/4282542699 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b00aece0 0x7f05b014e830 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 -- 172.21.15.32:0/4282542699 >> 172.21.15.32:0/4282542699 conn(0x7f05b006a320 msgr2=0x7f05b014f870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 -- 172.21.15.32:0/4282542699 shutdown_connections 2023-12-20T04:46:05.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.447+0000 7f05b802d640 1 -- 172.21.15.32:0/4282542699 wait complete. 2023-12-20T04:46:05.464 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-20T04:46:05.215742+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":829,"num_read_kb":1583,"num_write":345,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3118,"ondisk_log_size":3118,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":9,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":322092,"kb_used_data":13036,"kb_used_omap":0,"kb_used_meta":308992,"kb_avail":754652628,"statfs":{"total":773094113280,"available":772764291072,"internally_reserved":0,"allocated":13348864,"data_stored":6920136,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":316407808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":34,"apply_latency_ms":34,"commit_latency_ns":34000000,"apply_latency_ns":34000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.732778"},"pg_stats":[{"pgid":"15.3f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.924349+0000","last_change":"2023-12-20T04:46:00.919376+0000","last_active":"2023-12-20T04:46:00.924349+0000","last_peered":"2023-12-20T04:46:00.924349+0000","last_clean":"2023-12-20T04:46:00.924349+0000","last_became_active":"2023-12-20T04:46:00.919120+0000","last_became_peered":"2023-12-20T04:46:00.919120+0000","last_unstale":"2023-12-20T04:46:00.924349+0000","last_undegraded":"2023-12-20T04:46:00.924349+0000","last_fullsized":"2023-12-20T04:46:00.924349+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:48:30.160568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3e","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913441+0000","last_change":"2023-12-20T04:46:00.927397+0000","last_active":"2023-12-20T04:46:01.913441+0000","last_peered":"2023-12-20T04:46:01.913441+0000","last_clean":"2023-12-20T04:46:01.913441+0000","last_became_active":"2023-12-20T04:46:00.919451+0000","last_became_peered":"2023-12-20T04:46:00.919451+0000","last_unstale":"2023-12-20T04:46:01.913441+0000","last_undegraded":"2023-12-20T04:46:01.913441+0000","last_fullsized":"2023-12-20T04:46:01.913441+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:57:30.367208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.924178+0000","last_change":"2023-12-20T04:46:00.921290+0000","last_active":"2023-12-20T04:46:00.924178+0000","last_peered":"2023-12-20T04:46:00.924178+0000","last_clean":"2023-12-20T04:46:00.924178+0000","last_became_active":"2023-12-20T04:46:00.920960+0000","last_became_peered":"2023-12-20T04:46:00.920960+0000","last_unstale":"2023-12-20T04:46:00.924178+0000","last_undegraded":"2023-12-20T04:46:00.924178+0000","last_fullsized":"2023-12-20T04:46:00.924178+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:13:17.257637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.3c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.943725+0000","last_change":"2023-12-20T04:46:00.940419+0000","last_active":"2023-12-20T04:46:00.943725+0000","last_peered":"2023-12-20T04:46:00.943725+0000","last_clean":"2023-12-20T04:46:00.943725+0000","last_became_active":"2023-12-20T04:46:00.940065+0000","last_became_peered":"2023-12-20T04:46:00.940065+0000","last_unstale":"2023-12-20T04:46:00.943725+0000","last_undegraded":"2023-12-20T04:46:00.943725+0000","last_fullsized":"2023-12-20T04:46:00.943725+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:08:58.476143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.3b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.943002+0000","last_change":"2023-12-20T04:46:00.941690+0000","last_active":"2023-12-20T04:46:00.943002+0000","last_peered":"2023-12-20T04:46:00.943002+0000","last_clean":"2023-12-20T04:46:00.943002+0000","last_became_active":"2023-12-20T04:46:00.941363+0000","last_became_peered":"2023-12-20T04:46:00.941363+0000","last_unstale":"2023-12-20T04:46:00.943002+0000","last_undegraded":"2023-12-20T04:46:00.943002+0000","last_fullsized":"2023-12-20T04:46:00.943002+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:50:56.540048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.3a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.935755+0000","last_change":"2023-12-20T04:46:00.930820+0000","last_active":"2023-12-20T04:46:00.935755+0000","last_peered":"2023-12-20T04:46:00.935755+0000","last_clean":"2023-12-20T04:46:00.935755+0000","last_became_active":"2023-12-20T04:46:00.930542+0000","last_became_peered":"2023-12-20T04:46:00.930542+0000","last_unstale":"2023-12-20T04:46:00.935755+0000","last_undegraded":"2023-12-20T04:46:00.935755+0000","last_fullsized":"2023-12-20T04:46:00.935755+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:10.029821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.39","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.937532+0000","last_change":"2023-12-20T04:46:00.934470+0000","last_active":"2023-12-20T04:46:00.937532+0000","last_peered":"2023-12-20T04:46:00.937532+0000","last_clean":"2023-12-20T04:46:00.937532+0000","last_became_active":"2023-12-20T04:46:00.931697+0000","last_became_peered":"2023-12-20T04:46:00.931697+0000","last_unstale":"2023-12-20T04:46:00.937532+0000","last_undegraded":"2023-12-20T04:46:00.937532+0000","last_fullsized":"2023-12-20T04:46:00.937532+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:50:30.556188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.38","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.912974+0000","last_change":"2023-12-20T04:46:00.921093+0000","last_active":"2023-12-20T04:46:01.912974+0000","last_peered":"2023-12-20T04:46:01.912974+0000","last_clean":"2023-12-20T04:46:01.912974+0000","last_became_active":"2023-12-20T04:46:00.920844+0000","last_became_peered":"2023-12-20T04:46:00.920844+0000","last_unstale":"2023-12-20T04:46:01.912974+0000","last_undegraded":"2023-12-20T04:46:01.912974+0000","last_fullsized":"2023-12-20T04:46:01.912974+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:49:07.906740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.37","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927375+0000","last_change":"2023-12-20T04:46:00.941968+0000","last_active":"2023-12-20T04:46:01.927375+0000","last_peered":"2023-12-20T04:46:01.927375+0000","last_clean":"2023-12-20T04:46:01.927375+0000","last_became_active":"2023-12-20T04:46:00.941617+0000","last_became_peered":"2023-12-20T04:46:00.941617+0000","last_unstale":"2023-12-20T04:46:01.927375+0000","last_undegraded":"2023-12-20T04:46:01.927375+0000","last_fullsized":"2023-12-20T04:46:01.927375+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:08:58.476143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.36","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.928702+0000","last_change":"2023-12-20T04:46:00.922994+0000","last_active":"2023-12-20T04:46:00.928702+0000","last_peered":"2023-12-20T04:46:00.928702+0000","last_clean":"2023-12-20T04:46:00.928702+0000","last_became_active":"2023-12-20T04:46:00.922642+0000","last_became_peered":"2023-12-20T04:46:00.922642+0000","last_unstale":"2023-12-20T04:46:00.928702+0000","last_undegraded":"2023-12-20T04:46:00.928702+0000","last_fullsized":"2023-12-20T04:46:00.928702+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:54:39.495864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.35","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.937027+0000","last_change":"2023-12-20T04:46:00.932593+0000","last_active":"2023-12-20T04:46:00.937027+0000","last_peered":"2023-12-20T04:46:00.937027+0000","last_clean":"2023-12-20T04:46:00.937027+0000","last_became_active":"2023-12-20T04:46:00.932202+0000","last_became_peered":"2023-12-20T04:46:00.932202+0000","last_unstale":"2023-12-20T04:46:00.937027+0000","last_undegraded":"2023-12-20T04:46:00.937027+0000","last_fullsized":"2023-12-20T04:46:00.937027+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:47.501889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.34","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.926863+0000","last_change":"2023-12-20T04:46:00.926790+0000","last_active":"2023-12-20T04:46:01.926863+0000","last_peered":"2023-12-20T04:46:01.926863+0000","last_clean":"2023-12-20T04:46:01.926863+0000","last_became_active":"2023-12-20T04:46:00.926451+0000","last_became_peered":"2023-12-20T04:46:00.926451+0000","last_unstale":"2023-12-20T04:46:01.926863+0000","last_undegraded":"2023-12-20T04:46:01.926863+0000","last_fullsized":"2023-12-20T04:46:01.926863+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:55.139256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.33","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.941940+0000","last_change":"2023-12-20T04:46:00.935206+0000","last_active":"2023-12-20T04:46:00.941940+0000","last_peered":"2023-12-20T04:46:00.941940+0000","last_clean":"2023-12-20T04:46:00.941940+0000","last_became_active":"2023-12-20T04:46:00.934874+0000","last_became_peered":"2023-12-20T04:46:00.934874+0000","last_unstale":"2023-12-20T04:46:00.941940+0000","last_undegraded":"2023-12-20T04:46:00.941940+0000","last_fullsized":"2023-12-20T04:46:00.941940+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:08:30.180394+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.32","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.934860+0000","last_change":"2023-12-20T04:46:00.932291+0000","last_active":"2023-12-20T04:46:00.934860+0000","last_peered":"2023-12-20T04:46:00.934860+0000","last_clean":"2023-12-20T04:46:00.934860+0000","last_became_active":"2023-12-20T04:46:00.930497+0000","last_became_peered":"2023-12-20T04:46:00.930497+0000","last_unstale":"2023-12-20T04:46:00.934860+0000","last_undegraded":"2023-12-20T04:46:00.934860+0000","last_fullsized":"2023-12-20T04:46:00.934860+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:38:45.867057+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.31","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.941979+0000","last_change":"2023-12-20T04:46:00.935957+0000","last_active":"2023-12-20T04:46:00.941979+0000","last_peered":"2023-12-20T04:46:00.941979+0000","last_clean":"2023-12-20T04:46:00.941979+0000","last_became_active":"2023-12-20T04:46:00.934688+0000","last_became_peered":"2023-12-20T04:46:00.934688+0000","last_unstale":"2023-12-20T04:46:00.941979+0000","last_undegraded":"2023-12-20T04:46:00.941979+0000","last_fullsized":"2023-12-20T04:46:00.941979+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:47:17.644936+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.30","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.939881+0000","last_change":"2023-12-20T04:46:00.939213+0000","last_active":"2023-12-20T04:46:00.939881+0000","last_peered":"2023-12-20T04:46:00.939881+0000","last_clean":"2023-12-20T04:46:00.939881+0000","last_became_active":"2023-12-20T04:46:00.938918+0000","last_became_peered":"2023-12-20T04:46:00.938918+0000","last_unstale":"2023-12-20T04:46:00.939881+0000","last_undegraded":"2023-12-20T04:46:00.939881+0000","last_fullsized":"2023-12-20T04:46:00.939881+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:40:20.783877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.2f","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913681+0000","last_change":"2023-12-20T04:46:00.935895+0000","last_active":"2023-12-20T04:46:01.913681+0000","last_peered":"2023-12-20T04:46:01.913681+0000","last_clean":"2023-12-20T04:46:01.913681+0000","last_became_active":"2023-12-20T04:46:00.935299+0000","last_became_peered":"2023-12-20T04:46:00.935299+0000","last_unstale":"2023-12-20T04:46:01.913681+0000","last_undegraded":"2023-12-20T04:46:01.913681+0000","last_fullsized":"2023-12-20T04:46:01.913681+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:30:30.148194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.2e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.943762+0000","last_change":"2023-12-20T04:46:00.940192+0000","last_active":"2023-12-20T04:46:00.943762+0000","last_peered":"2023-12-20T04:46:00.943762+0000","last_clean":"2023-12-20T04:46:00.943762+0000","last_became_active":"2023-12-20T04:46:00.939897+0000","last_became_peered":"2023-12-20T04:46:00.939897+0000","last_unstale":"2023-12-20T04:46:00.943762+0000","last_undegraded":"2023-12-20T04:46:00.943762+0000","last_fullsized":"2023-12-20T04:46:00.943762+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:07:45.553369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.2d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.931560+0000","last_change":"2023-12-20T04:46:00.926559+0000","last_active":"2023-12-20T04:46:00.931560+0000","last_peered":"2023-12-20T04:46:00.931560+0000","last_clean":"2023-12-20T04:46:00.931560+0000","last_became_active":"2023-12-20T04:46:00.926301+0000","last_became_peered":"2023-12-20T04:46:00.926301+0000","last_unstale":"2023-12-20T04:46:00.931560+0000","last_undegraded":"2023-12-20T04:46:00.931560+0000","last_fullsized":"2023-12-20T04:46:00.931560+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:01:24.712802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927005+0000","last_change":"2023-12-20T04:46:00.942197+0000","last_active":"2023-12-20T04:46:01.927005+0000","last_peered":"2023-12-20T04:46:01.927005+0000","last_clean":"2023-12-20T04:46:01.927005+0000","last_became_active":"2023-12-20T04:46:00.941909+0000","last_became_peered":"2023-12-20T04:46:00.941909+0000","last_unstale":"2023-12-20T04:46:01.927005+0000","last_undegraded":"2023-12-20T04:46:01.927005+0000","last_fullsized":"2023-12-20T04:46:01.927005+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:10:07.235864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.2b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.932713+0000","last_change":"2023-12-20T04:46:00.930113+0000","last_active":"2023-12-20T04:46:00.932713+0000","last_peered":"2023-12-20T04:46:00.932713+0000","last_clean":"2023-12-20T04:46:00.932713+0000","last_became_active":"2023-12-20T04:46:00.929881+0000","last_became_peered":"2023-12-20T04:46:00.929881+0000","last_unstale":"2023-12-20T04:46:00.932713+0000","last_undegraded":"2023-12-20T04:46:00.932713+0000","last_fullsized":"2023-12-20T04:46:00.932713+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:43.705186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.2a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.912978+0000","last_change":"2023-12-20T04:46:00.920294+0000","last_active":"2023-12-20T04:46:01.912978+0000","last_peered":"2023-12-20T04:46:01.912978+0000","last_clean":"2023-12-20T04:46:01.912978+0000","last_became_active":"2023-12-20T04:46:00.920001+0000","last_became_peered":"2023-12-20T04:46:00.920001+0000","last_unstale":"2023-12-20T04:46:01.912978+0000","last_undegraded":"2023-12-20T04:46:01.912978+0000","last_fullsized":"2023-12-20T04:46:01.912978+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:01:27.853191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.29","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913031+0000","last_change":"2023-12-20T04:46:00.918533+0000","last_active":"2023-12-20T04:46:01.913031+0000","last_peered":"2023-12-20T04:46:01.913031+0000","last_clean":"2023-12-20T04:46:01.913031+0000","last_became_active":"2023-12-20T04:46:00.918272+0000","last_became_peered":"2023-12-20T04:46:00.918272+0000","last_unstale":"2023-12-20T04:46:01.913031+0000","last_undegraded":"2023-12-20T04:46:01.913031+0000","last_fullsized":"2023-12-20T04:46:01.913031+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:08:15.729193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.28","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.929525+0000","last_change":"2023-12-20T04:46:00.923879+0000","last_active":"2023-12-20T04:46:00.929525+0000","last_peered":"2023-12-20T04:46:00.929525+0000","last_clean":"2023-12-20T04:46:00.929525+0000","last_became_active":"2023-12-20T04:46:00.923606+0000","last_became_peered":"2023-12-20T04:46:00.923606+0000","last_unstale":"2023-12-20T04:46:00.929525+0000","last_undegraded":"2023-12-20T04:46:00.929525+0000","last_fullsized":"2023-12-20T04:46:00.929525+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:10:07.235864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.27","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.928008+0000","last_change":"2023-12-20T04:46:00.920889+0000","last_active":"2023-12-20T04:46:00.928008+0000","last_peered":"2023-12-20T04:46:00.928008+0000","last_clean":"2023-12-20T04:46:00.928008+0000","last_became_active":"2023-12-20T04:46:00.919900+0000","last_became_peered":"2023-12-20T04:46:00.919900+0000","last_unstale":"2023-12-20T04:46:00.928008+0000","last_undegraded":"2023-12-20T04:46:00.928008+0000","last_fullsized":"2023-12-20T04:46:00.928008+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:26.747072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.26","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913439+0000","last_change":"2023-12-20T04:46:00.918837+0000","last_active":"2023-12-20T04:46:01.913439+0000","last_peered":"2023-12-20T04:46:01.913439+0000","last_clean":"2023-12-20T04:46:01.913439+0000","last_became_active":"2023-12-20T04:46:00.918585+0000","last_became_peered":"2023-12-20T04:46:00.918585+0000","last_unstale":"2023-12-20T04:46:01.913439+0000","last_undegraded":"2023-12-20T04:46:01.913439+0000","last_fullsized":"2023-12-20T04:46:01.913439+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:04:07.931455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.25","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.937178+0000","last_change":"2023-12-20T04:46:00.935349+0000","last_active":"2023-12-20T04:46:00.937178+0000","last_peered":"2023-12-20T04:46:00.937178+0000","last_clean":"2023-12-20T04:46:00.937178+0000","last_became_active":"2023-12-20T04:46:00.932599+0000","last_became_peered":"2023-12-20T04:46:00.932599+0000","last_unstale":"2023-12-20T04:46:00.937178+0000","last_undegraded":"2023-12-20T04:46:00.937178+0000","last_fullsized":"2023-12-20T04:46:00.937178+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:11:17.266722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.24","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.941068+0000","last_change":"2023-12-20T04:46:00.940187+0000","last_active":"2023-12-20T04:46:00.941068+0000","last_peered":"2023-12-20T04:46:00.941068+0000","last_clean":"2023-12-20T04:46:00.941068+0000","last_became_active":"2023-12-20T04:46:00.939909+0000","last_became_peered":"2023-12-20T04:46:00.939909+0000","last_unstale":"2023-12-20T04:46:00.941068+0000","last_undegraded":"2023-12-20T04:46:00.941068+0000","last_fullsized":"2023-12-20T04:46:00.941068+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:44:51.858826+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.23","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913058+0000","last_change":"2023-12-20T04:46:00.928631+0000","last_active":"2023-12-20T04:46:01.913058+0000","last_peered":"2023-12-20T04:46:01.913058+0000","last_clean":"2023-12-20T04:46:01.913058+0000","last_became_active":"2023-12-20T04:46:00.919785+0000","last_became_peered":"2023-12-20T04:46:00.919785+0000","last_unstale":"2023-12-20T04:46:01.913058+0000","last_undegraded":"2023-12-20T04:46:01.913058+0000","last_fullsized":"2023-12-20T04:46:01.913058+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:55:56.657692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.22","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.932463+0000","last_change":"2023-12-20T04:46:00.927292+0000","last_active":"2023-12-20T04:46:00.932463+0000","last_peered":"2023-12-20T04:46:00.932463+0000","last_clean":"2023-12-20T04:46:00.932463+0000","last_became_active":"2023-12-20T04:46:00.926281+0000","last_became_peered":"2023-12-20T04:46:00.926281+0000","last_unstale":"2023-12-20T04:46:00.932463+0000","last_undegraded":"2023-12-20T04:46:00.932463+0000","last_fullsized":"2023-12-20T04:46:00.932463+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:24:50.810985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.21","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.942192+0000","last_change":"2023-12-20T04:46:00.937288+0000","last_active":"2023-12-20T04:46:00.942192+0000","last_peered":"2023-12-20T04:46:00.942192+0000","last_clean":"2023-12-20T04:46:00.942192+0000","last_became_active":"2023-12-20T04:46:00.934437+0000","last_became_peered":"2023-12-20T04:46:00.934437+0000","last_unstale":"2023-12-20T04:46:00.942192+0000","last_undegraded":"2023-12-20T04:46:00.942192+0000","last_fullsized":"2023-12-20T04:46:00.942192+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:13:54.533082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.20","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.916771+0000","last_change":"2023-12-20T04:46:00.915038+0000","last_active":"2023-12-20T04:46:00.916771+0000","last_peered":"2023-12-20T04:46:00.916771+0000","last_clean":"2023-12-20T04:46:00.916771+0000","last_became_active":"2023-12-20T04:46:00.914687+0000","last_became_peered":"2023-12-20T04:46:00.914687+0000","last_unstale":"2023-12-20T04:46:00.916771+0000","last_undegraded":"2023-12-20T04:46:00.916771+0000","last_fullsized":"2023-12-20T04:46:00.916771+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:53:55.732437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.933905+0000","last_change":"2023-12-20T04:46:00.931455+0000","last_active":"2023-12-20T04:46:00.933905+0000","last_peered":"2023-12-20T04:46:00.933905+0000","last_clean":"2023-12-20T04:46:00.933905+0000","last_became_active":"2023-12-20T04:46:00.931187+0000","last_became_peered":"2023-12-20T04:46:00.931187+0000","last_unstale":"2023-12-20T04:46:00.933905+0000","last_undegraded":"2023-12-20T04:46:00.933905+0000","last_fullsized":"2023-12-20T04:46:00.933905+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:35:04.809390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1f","version":"122'4","reported_seq":15,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913713+0000","last_change":"2023-12-20T04:46:00.513411+0000","last_active":"2023-12-20T04:46:01.913713+0000","last_peered":"2023-12-20T04:46:01.913713+0000","last_clean":"2023-12-20T04:46:01.913713+0000","last_became_active":"2023-12-20T04:46:00.513137+0000","last_became_peered":"2023-12-20T04:46:00.513137+0000","last_unstale":"2023-12-20T04:46:01.913713+0000","last_undegraded":"2023-12-20T04:46:01.913713+0000","last_fullsized":"2023-12-20T04:46:01.913713+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:40:19.775654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.941868+0000","last_change":"2023-12-20T04:46:00.933940+0000","last_active":"2023-12-20T04:46:00.941868+0000","last_peered":"2023-12-20T04:46:00.941868+0000","last_clean":"2023-12-20T04:46:00.941868+0000","last_became_active":"2023-12-20T04:46:00.933244+0000","last_became_peered":"2023-12-20T04:46:00.933244+0000","last_unstale":"2023-12-20T04:46:00.941868+0000","last_undegraded":"2023-12-20T04:46:00.941868+0000","last_fullsized":"2023-12-20T04:46:00.941868+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:48:26.269219+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.1e","version":"122'1","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.936516+0000","last_change":"2023-12-20T04:46:00.522048+0000","last_active":"2023-12-20T04:46:00.936516+0000","last_peered":"2023-12-20T04:46:00.936516+0000","last_clean":"2023-12-20T04:46:00.936516+0000","last_became_active":"2023-12-20T04:46:00.521791+0000","last_became_peered":"2023-12-20T04:46:00.521791+0000","last_unstale":"2023-12-20T04:46:00.936516+0000","last_undegraded":"2023-12-20T04:46:00.936516+0000","last_fullsized":"2023-12-20T04:46:00.936516+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:24:41.006760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.1c","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.934794+0000","last_change":"2023-12-20T04:46:00.927129+0000","last_active":"2023-12-20T04:46:00.934794+0000","last_peered":"2023-12-20T04:46:00.934794+0000","last_clean":"2023-12-20T04:46:00.934794+0000","last_became_active":"2023-12-20T04:46:00.926798+0000","last_became_peered":"2023-12-20T04:46:00.926798+0000","last_unstale":"2023-12-20T04:46:00.934794+0000","last_undegraded":"2023-12-20T04:46:00.934794+0000","last_fullsized":"2023-12-20T04:46:00.934794+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:30:08.120496+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.1d","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913401+0000","last_change":"2023-12-20T04:46:00.510459+0000","last_active":"2023-12-20T04:46:01.913401+0000","last_peered":"2023-12-20T04:46:01.913401+0000","last_clean":"2023-12-20T04:46:01.913401+0000","last_became_active":"2023-12-20T04:46:00.510142+0000","last_became_peered":"2023-12-20T04:46:00.510142+0000","last_unstale":"2023-12-20T04:46:01.913401+0000","last_undegraded":"2023-12-20T04:46:01.913401+0000","last_fullsized":"2023-12-20T04:46:01.913401+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:54.131033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1d","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927248+0000","last_change":"2023-12-20T04:46:00.923438+0000","last_active":"2023-12-20T04:46:01.927248+0000","last_peered":"2023-12-20T04:46:01.927248+0000","last_clean":"2023-12-20T04:46:01.927248+0000","last_became_active":"2023-12-20T04:46:00.923158+0000","last_became_peered":"2023-12-20T04:46:00.923158+0000","last_unstale":"2023-12-20T04:46:01.927248+0000","last_undegraded":"2023-12-20T04:46:01.927248+0000","last_fullsized":"2023-12-20T04:46:01.927248+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:58:52.252540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.1c","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913138+0000","last_change":"2023-12-20T04:46:00.510308+0000","last_active":"2023-12-20T04:46:01.913138+0000","last_peered":"2023-12-20T04:46:01.913138+0000","last_clean":"2023-12-20T04:46:01.913138+0000","last_became_active":"2023-12-20T04:46:00.509933+0000","last_became_peered":"2023-12-20T04:46:00.509933+0000","last_unstale":"2023-12-20T04:46:01.913138+0000","last_undegraded":"2023-12-20T04:46:01.913138+0000","last_fullsized":"2023-12-20T04:46:01.913138+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:46.493666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.1a","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.930243+0000","last_change":"2023-12-20T04:46:00.925382+0000","last_active":"2023-12-20T04:46:01.930243+0000","last_peered":"2023-12-20T04:46:01.930243+0000","last_clean":"2023-12-20T04:46:01.930243+0000","last_became_active":"2023-12-20T04:46:00.925027+0000","last_became_peered":"2023-12-20T04:46:00.925027+0000","last_unstale":"2023-12-20T04:46:01.930243+0000","last_undegraded":"2023-12-20T04:46:01.930243+0000","last_fullsized":"2023-12-20T04:46:01.930243+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:47.193663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.1b","version":"122'1","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.930647+0000","last_change":"2023-12-20T04:46:00.512592+0000","last_active":"2023-12-20T04:46:00.930647+0000","last_peered":"2023-12-20T04:46:00.930647+0000","last_clean":"2023-12-20T04:46:00.930647+0000","last_became_active":"2023-12-20T04:46:00.512307+0000","last_became_peered":"2023-12-20T04:46:00.512307+0000","last_unstale":"2023-12-20T04:46:00.930647+0000","last_undegraded":"2023-12-20T04:46:00.930647+0000","last_fullsized":"2023-12-20T04:46:00.930647+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:01:23.704580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.1b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.936231+0000","last_change":"2023-12-20T04:46:00.931078+0000","last_active":"2023-12-20T04:46:00.936231+0000","last_peered":"2023-12-20T04:46:00.936231+0000","last_clean":"2023-12-20T04:46:00.936231+0000","last_became_active":"2023-12-20T04:46:00.930822+0000","last_became_peered":"2023-12-20T04:46:00.930822+0000","last_unstale":"2023-12-20T04:46:00.936231+0000","last_undegraded":"2023-12-20T04:46:00.936231+0000","last_fullsized":"2023-12-20T04:46:00.936231+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:58:52.252540+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.1a","version":"122'1","reported_seq":12,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.912978+0000","last_change":"2023-12-20T04:46:00.513258+0000","last_active":"2023-12-20T04:46:01.912978+0000","last_peered":"2023-12-20T04:46:01.912978+0000","last_clean":"2023-12-20T04:46:01.912978+0000","last_became_active":"2023-12-20T04:46:00.512979+0000","last_became_peered":"2023-12-20T04:46:00.512979+0000","last_unstale":"2023-12-20T04:46:01.912978+0000","last_undegraded":"2023-12-20T04:46:01.912978+0000","last_fullsized":"2023-12-20T04:46:01.912978+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:07:36.550183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.18","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.943628+0000","last_change":"2023-12-20T04:46:00.939877+0000","last_active":"2023-12-20T04:46:00.943628+0000","last_peered":"2023-12-20T04:46:00.943628+0000","last_clean":"2023-12-20T04:46:00.943628+0000","last_became_active":"2023-12-20T04:46:00.939074+0000","last_became_peered":"2023-12-20T04:46:00.939074+0000","last_unstale":"2023-12-20T04:46:00.943628+0000","last_undegraded":"2023-12-20T04:46:00.943628+0000","last_fullsized":"2023-12-20T04:46:00.943628+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:20.797046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.19","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.908932+0000","last_change":"2023-12-20T04:46:00.519496+0000","last_active":"2023-12-20T04:46:00.908932+0000","last_peered":"2023-12-20T04:46:00.908932+0000","last_clean":"2023-12-20T04:46:00.908932+0000","last_became_active":"2023-12-20T04:46:00.519004+0000","last_became_peered":"2023-12-20T04:46:00.519004+0000","last_unstale":"2023-12-20T04:46:00.908932+0000","last_undegraded":"2023-12-20T04:46:00.908932+0000","last_fullsized":"2023-12-20T04:46:00.908932+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:32:34.843771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.19","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.926662+0000","last_change":"2023-12-20T04:46:00.925542+0000","last_active":"2023-12-20T04:46:01.926662+0000","last_peered":"2023-12-20T04:46:01.926662+0000","last_clean":"2023-12-20T04:46:01.926662+0000","last_became_active":"2023-12-20T04:46:00.924882+0000","last_became_peered":"2023-12-20T04:46:00.924882+0000","last_unstale":"2023-12-20T04:46:01.926662+0000","last_undegraded":"2023-12-20T04:46:01.926662+0000","last_fullsized":"2023-12-20T04:46:01.926662+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:50:30.556188+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.18","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.911178+0000","last_change":"2023-12-20T04:46:00.517062+0000","last_active":"2023-12-20T04:46:00.911178+0000","last_peered":"2023-12-20T04:46:00.911178+0000","last_clean":"2023-12-20T04:46:00.911178+0000","last_became_active":"2023-12-20T04:46:00.516766+0000","last_became_peered":"2023-12-20T04:46:00.516766+0000","last_unstale":"2023-12-20T04:46:00.911178+0000","last_undegraded":"2023-12-20T04:46:00.911178+0000","last_fullsized":"2023-12-20T04:46:00.911178+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:43.636745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.16","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913767+0000","last_change":"2023-12-20T04:46:00.927911+0000","last_active":"2023-12-20T04:46:01.913767+0000","last_peered":"2023-12-20T04:46:01.913767+0000","last_clean":"2023-12-20T04:46:01.913767+0000","last_became_active":"2023-12-20T04:46:00.927652+0000","last_became_peered":"2023-12-20T04:46:00.927652+0000","last_unstale":"2023-12-20T04:46:01.913767+0000","last_undegraded":"2023-12-20T04:46:01.913767+0000","last_fullsized":"2023-12-20T04:46:01.913767+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:20.797046+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.17","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.912028+0000","last_change":"2023-12-20T04:46:00.514459+0000","last_active":"2023-12-20T04:46:00.912028+0000","last_peered":"2023-12-20T04:46:00.912028+0000","last_clean":"2023-12-20T04:46:00.912028+0000","last_became_active":"2023-12-20T04:46:00.514207+0000","last_became_peered":"2023-12-20T04:46:00.514207+0000","last_unstale":"2023-12-20T04:46:00.912028+0000","last_undegraded":"2023-12-20T04:46:00.912028+0000","last_fullsized":"2023-12-20T04:46:00.912028+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:41.589301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.17","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927378+0000","last_change":"2023-12-20T04:46:00.933228+0000","last_active":"2023-12-20T04:46:01.927378+0000","last_peered":"2023-12-20T04:46:01.927378+0000","last_clean":"2023-12-20T04:46:01.927378+0000","last_became_active":"2023-12-20T04:46:00.932863+0000","last_became_peered":"2023-12-20T04:46:00.932863+0000","last_unstale":"2023-12-20T04:46:01.927378+0000","last_undegraded":"2023-12-20T04:46:01.927378+0000","last_fullsized":"2023-12-20T04:46:01.927378+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:07:37.558405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.16","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.925839+0000","last_change":"2023-12-20T04:46:00.522428+0000","last_active":"2023-12-20T04:46:00.925839+0000","last_peered":"2023-12-20T04:46:00.925839+0000","last_clean":"2023-12-20T04:46:00.925839+0000","last_became_active":"2023-12-20T04:46:00.522091+0000","last_became_peered":"2023-12-20T04:46:00.522091+0000","last_unstale":"2023-12-20T04:46:00.925839+0000","last_undegraded":"2023-12-20T04:46:00.925839+0000","last_fullsized":"2023-12-20T04:46:00.925839+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:34.893107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.14","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.919715+0000","last_change":"2023-12-20T04:46:00.913852+0000","last_active":"2023-12-20T04:46:00.919715+0000","last_peered":"2023-12-20T04:46:00.919715+0000","last_clean":"2023-12-20T04:46:00.919715+0000","last_became_active":"2023-12-20T04:46:00.913572+0000","last_became_peered":"2023-12-20T04:46:00.913572+0000","last_unstale":"2023-12-20T04:46:00.919715+0000","last_undegraded":"2023-12-20T04:46:00.919715+0000","last_fullsized":"2023-12-20T04:46:00.919715+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:24:42.014982+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.15","version":"122'3","reported_seq":13,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.936881+0000","last_change":"2023-12-20T04:46:00.517175+0000","last_active":"2023-12-20T04:46:00.936881+0000","last_peered":"2023-12-20T04:46:00.936881+0000","last_clean":"2023-12-20T04:46:00.936881+0000","last_became_active":"2023-12-20T04:46:00.516730+0000","last_became_peered":"2023-12-20T04:46:00.516730+0000","last_unstale":"2023-12-20T04:46:00.936881+0000","last_undegraded":"2023-12-20T04:46:00.936881+0000","last_fullsized":"2023-12-20T04:46:00.936881+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:33:08.960927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"15.15","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.942863+0000","last_change":"2023-12-20T04:46:00.941383+0000","last_active":"2023-12-20T04:46:00.942863+0000","last_peered":"2023-12-20T04:46:00.942863+0000","last_clean":"2023-12-20T04:46:00.942863+0000","last_became_active":"2023-12-20T04:46:00.941070+0000","last_became_peered":"2023-12-20T04:46:00.941070+0000","last_unstale":"2023-12-20T04:46:00.942863+0000","last_undegraded":"2023-12-20T04:46:00.942863+0000","last_fullsized":"2023-12-20T04:46:00.942863+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:07:37.558405+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.14","version":"122'6","reported_seq":17,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927119+0000","last_change":"2023-12-20T04:46:00.518446+0000","last_active":"2023-12-20T04:46:01.927119+0000","last_peered":"2023-12-20T04:46:01.927119+0000","last_clean":"2023-12-20T04:46:01.927119+0000","last_became_active":"2023-12-20T04:46:00.517918+0000","last_became_peered":"2023-12-20T04:46:00.517918+0000","last_unstale":"2023-12-20T04:46:01.927119+0000","last_undegraded":"2023-12-20T04:46:01.927119+0000","last_fullsized":"2023-12-20T04:46:01.927119+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:23.121194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.12","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.942284+0000","last_change":"2023-12-20T04:46:00.935394+0000","last_active":"2023-12-20T04:46:00.942284+0000","last_peered":"2023-12-20T04:46:00.942284+0000","last_clean":"2023-12-20T04:46:00.942284+0000","last_became_active":"2023-12-20T04:46:00.935140+0000","last_became_peered":"2023-12-20T04:46:00.935140+0000","last_unstale":"2023-12-20T04:46:00.942284+0000","last_undegraded":"2023-12-20T04:46:00.942284+0000","last_fullsized":"2023-12-20T04:46:00.942284+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:31:22.675050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"14.13","version":"122'2","reported_seq":12,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.935354+0000","last_change":"2023-12-20T04:46:00.512630+0000","last_active":"2023-12-20T04:46:00.935354+0000","last_peered":"2023-12-20T04:46:00.935354+0000","last_clean":"2023-12-20T04:46:00.935354+0000","last_became_active":"2023-12-20T04:46:00.511966+0000","last_became_peered":"2023-12-20T04:46:00.511966+0000","last_unstale":"2023-12-20T04:46:00.935354+0000","last_undegraded":"2023-12-20T04:46:00.935354+0000","last_fullsized":"2023-12-20T04:46:00.935354+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:26:40.880550+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.13","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.934960+0000","last_change":"2023-12-20T04:46:00.924622+0000","last_active":"2023-12-20T04:46:00.934960+0000","last_peered":"2023-12-20T04:46:00.934960+0000","last_clean":"2023-12-20T04:46:00.934960+0000","last_became_active":"2023-12-20T04:46:00.924327+0000","last_became_peered":"2023-12-20T04:46:00.924327+0000","last_unstale":"2023-12-20T04:46:00.934960+0000","last_undegraded":"2023-12-20T04:46:00.934960+0000","last_fullsized":"2023-12-20T04:46:00.934960+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:16.834340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.12","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.916241+0000","last_change":"2023-12-20T04:46:00.519325+0000","last_active":"2023-12-20T04:46:00.916241+0000","last_peered":"2023-12-20T04:46:00.916241+0000","last_clean":"2023-12-20T04:46:00.916241+0000","last_became_active":"2023-12-20T04:46:00.519061+0000","last_became_peered":"2023-12-20T04:46:00.519061+0000","last_unstale":"2023-12-20T04:46:00.916241+0000","last_undegraded":"2023-12-20T04:46:00.916241+0000","last_fullsized":"2023-12-20T04:46:00.916241+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:05:02.649351+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.10","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.926641+0000","last_change":"2023-12-20T04:46:00.926925+0000","last_active":"2023-12-20T04:46:01.926641+0000","last_peered":"2023-12-20T04:46:01.926641+0000","last_clean":"2023-12-20T04:46:01.926641+0000","last_became_active":"2023-12-20T04:46:00.926578+0000","last_became_peered":"2023-12-20T04:46:00.926578+0000","last_unstale":"2023-12-20T04:46:01.926641+0000","last_undegraded":"2023-12-20T04:46:01.926641+0000","last_fullsized":"2023-12-20T04:46:01.926641+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:47.501889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.11","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.926724+0000","last_change":"2023-12-20T04:46:00.510988+0000","last_active":"2023-12-20T04:46:01.926724+0000","last_peered":"2023-12-20T04:46:01.926724+0000","last_clean":"2023-12-20T04:46:01.926724+0000","last_became_active":"2023-12-20T04:46:00.509913+0000","last_became_peered":"2023-12-20T04:46:00.509913+0000","last_unstale":"2023-12-20T04:46:01.926724+0000","last_undegraded":"2023-12-20T04:46:01.926724+0000","last_fullsized":"2023-12-20T04:46:01.926724+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:09:53.122786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.11","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.932160+0000","last_change":"2023-12-20T04:46:00.929177+0000","last_active":"2023-12-20T04:46:00.932160+0000","last_peered":"2023-12-20T04:46:00.932160+0000","last_clean":"2023-12-20T04:46:00.932160+0000","last_became_active":"2023-12-20T04:46:00.928804+0000","last_became_peered":"2023-12-20T04:46:00.928804+0000","last_unstale":"2023-12-20T04:46:00.932160+0000","last_undegraded":"2023-12-20T04:46:00.932160+0000","last_fullsized":"2023-12-20T04:46:00.932160+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:44.644967+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.10","version":"122'8","reported_seq":18,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.937158+0000","last_change":"2023-12-20T04:46:00.513860+0000","last_active":"2023-12-20T04:46:00.937158+0000","last_peered":"2023-12-20T04:46:00.937158+0000","last_clean":"2023-12-20T04:46:00.937158+0000","last_became_active":"2023-12-20T04:46:00.513571+0000","last_became_peered":"2023-12-20T04:46:00.513571+0000","last_unstale":"2023-12-20T04:46:00.937158+0000","last_undegraded":"2023-12-20T04:46:00.937158+0000","last_fullsized":"2023-12-20T04:46:00.937158+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:44.624915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"15.f","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.932119+0000","last_change":"2023-12-20T04:46:00.926971+0000","last_active":"2023-12-20T04:46:00.932119+0000","last_peered":"2023-12-20T04:46:00.932119+0000","last_clean":"2023-12-20T04:46:00.932119+0000","last_became_active":"2023-12-20T04:46:00.926568+0000","last_became_peered":"2023-12-20T04:46:00.926568+0000","last_unstale":"2023-12-20T04:46:00.932119+0000","last_undegraded":"2023-12-20T04:46:00.932119+0000","last_fullsized":"2023-12-20T04:46:00.932119+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:02:50.897202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.e","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.927322+0000","last_change":"2023-12-20T04:46:00.517860+0000","last_active":"2023-12-20T04:46:01.927322+0000","last_peered":"2023-12-20T04:46:01.927322+0000","last_clean":"2023-12-20T04:46:01.927322+0000","last_became_active":"2023-12-20T04:46:00.517504+0000","last_became_peered":"2023-12-20T04:46:00.517504+0000","last_unstale":"2023-12-20T04:46:01.927322+0000","last_undegraded":"2023-12-20T04:46:01.927322+0000","last_fullsized":"2023-12-20T04:46:01.927322+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:23.637670+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"15.c","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.930174+0000","last_change":"2023-12-20T04:46:00.915005+0000","last_active":"2023-12-20T04:46:01.930174+0000","last_peered":"2023-12-20T04:46:01.930174+0000","last_clean":"2023-12-20T04:46:01.930174+0000","last_became_active":"2023-12-20T04:46:00.913773+0000","last_became_peered":"2023-12-20T04:46:00.913773+0000","last_unstale":"2023-12-20T04:46:01.930174+0000","last_undegraded":"2023-12-20T04:46:01.930174+0000","last_fullsized":"2023-12-20T04:46:01.930174+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:53:55.732437+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"14.d","version":"122'1","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.936864+0000","last_change":"2023-12-20T04:46:00.509594+0000","last_active":"2023-12-20T04:46:00.936864+0000","last_peered":"2023-12-20T04:46:00.936864+0000","last_clean":"2023-12-20T04:46:00.936864+0000","last_became_active":"2023-12-20T04:46:00.509291+0000","last_became_peered":"2023-12-20T04:46:00.509291+0000","last_unstale":"2023-12-20T04:46:00.936864+0000","last_undegraded":"2023-12-20T04:46:00.936864+0000","last_fullsized":"2023-12-20T04:46:00.936864+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:52:12.651653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.d","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.937044+0000","last_change":"2023-12-20T04:46:00.932818+0000","last_active":"2023-12-20T04:46:00.937044+0000","last_peered":"2023-12-20T04:46:00.937044+0000","last_clean":"2023-12-20T04:46:00.937044+0000","last_became_active":"2023-12-20T04:46:00.932274+0000","last_became_peered":"2023-12-20T04:46:00.932274+0000","last_unstale":"2023-12-20T04:46:00.937044+0000","last_undegraded":"2023-12-20T04:46:00.937044+0000","last_fullsized":"2023-12-20T04:46:00.937044+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:35.901329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"14.c","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913179+0000","last_change":"2023-12-20T04:46:00.506704+0000","last_active":"2023-12-20T04:46:01.913179+0000","last_peered":"2023-12-20T04:46:01.913179+0000","last_clean":"2023-12-20T04:46:01.913179+0000","last_became_active":"2023-12-20T04:46:00.506380+0000","last_became_peered":"2023-12-20T04:46:00.506380+0000","last_unstale":"2023-12-20T04:46:01.913179+0000","last_undegraded":"2023-12-20T04:46:01.913179+0000","last_fullsized":"2023-12-20T04:46:01.913179+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:50:29.547966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.a","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.942843+0000","last_change":"2023-12-20T04:46:00.941229+0000","last_active":"2023-12-20T04:46:00.942843+0000","last_peered":"2023-12-20T04:46:00.942843+0000","last_clean":"2023-12-20T04:46:00.942843+0000","last_became_active":"2023-12-20T04:46:00.940970+0000","last_became_peered":"2023-12-20T04:46:00.940970+0000","last_unstale":"2023-12-20T04:46:00.942843+0000","last_undegraded":"2023-12-20T04:46:00.942843+0000","last_fullsized":"2023-12-20T04:46:00.942843+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:30.606203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.b","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913172+0000","last_change":"2023-12-20T04:46:00.510088+0000","last_active":"2023-12-20T04:46:01.913172+0000","last_peered":"2023-12-20T04:46:01.913172+0000","last_clean":"2023-12-20T04:46:01.913172+0000","last_became_active":"2023-12-20T04:46:00.509787+0000","last_became_peered":"2023-12-20T04:46:00.509787+0000","last_unstale":"2023-12-20T04:46:01.913172+0000","last_undegraded":"2023-12-20T04:46:01.913172+0000","last_fullsized":"2023-12-20T04:46:01.913172+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:34:05.851006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.b","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.942806+0000","last_change":"2023-12-20T04:46:00.940247+0000","last_active":"2023-12-20T04:46:00.942806+0000","last_peered":"2023-12-20T04:46:00.942806+0000","last_clean":"2023-12-20T04:46:00.942806+0000","last_became_active":"2023-12-20T04:46:00.939969+0000","last_became_peered":"2023-12-20T04:46:00.939969+0000","last_unstale":"2023-12-20T04:46:00.942806+0000","last_undegraded":"2023-12-20T04:46:00.942806+0000","last_fullsized":"2023-12-20T04:46:00.942806+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:26:35.868259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.a","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.930228+0000","last_change":"2023-12-20T04:46:00.515932+0000","last_active":"2023-12-20T04:46:01.930228+0000","last_peered":"2023-12-20T04:46:01.930228+0000","last_clean":"2023-12-20T04:46:01.930228+0000","last_became_active":"2023-12-20T04:46:00.515668+0000","last_became_peered":"2023-12-20T04:46:00.515668+0000","last_unstale":"2023-12-20T04:46:01.930228+0000","last_undegraded":"2023-12-20T04:46:01.930228+0000","last_fullsized":"2023-12-20T04:46:01.930228+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:29.597980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.8","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.937838+0000","last_change":"2023-12-20T04:46:00.934227+0000","last_active":"2023-12-20T04:46:00.937838+0000","last_peered":"2023-12-20T04:46:00.937838+0000","last_clean":"2023-12-20T04:46:00.937838+0000","last_became_active":"2023-12-20T04:46:00.933953+0000","last_became_peered":"2023-12-20T04:46:00.933953+0000","last_unstale":"2023-12-20T04:46:00.937838+0000","last_undegraded":"2023-12-20T04:46:00.937838+0000","last_fullsized":"2023-12-20T04:46:00.937838+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:04:55.770021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.9","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.930110+0000","last_change":"2023-12-20T04:46:00.517339+0000","last_active":"2023-12-20T04:46:01.930110+0000","last_peered":"2023-12-20T04:46:01.930110+0000","last_clean":"2023-12-20T04:46:01.930110+0000","last_became_active":"2023-12-20T04:46:00.516860+0000","last_became_peered":"2023-12-20T04:46:00.516860+0000","last_unstale":"2023-12-20T04:46:01.930110+0000","last_undegraded":"2023-12-20T04:46:01.930110+0000","last_fullsized":"2023-12-20T04:46:01.930110+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:32:34.843771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.9","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.919751+0000","last_change":"2023-12-20T04:46:00.914217+0000","last_active":"2023-12-20T04:46:00.919751+0000","last_peered":"2023-12-20T04:46:00.919751+0000","last_clean":"2023-12-20T04:46:00.919751+0000","last_became_active":"2023-12-20T04:46:00.913941+0000","last_became_peered":"2023-12-20T04:46:00.913941+0000","last_unstale":"2023-12-20T04:46:00.919751+0000","last_undegraded":"2023-12-20T04:46:00.919751+0000","last_fullsized":"2023-12-20T04:46:00.919751+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:33:09.969149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.8","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913155+0000","last_change":"2023-12-20T04:46:00.510020+0000","last_active":"2023-12-20T04:46:01.913155+0000","last_peered":"2023-12-20T04:46:01.913155+0000","last_clean":"2023-12-20T04:46:01.913155+0000","last_became_active":"2023-12-20T04:46:00.508089+0000","last_became_peered":"2023-12-20T04:46:00.508089+0000","last_unstale":"2023-12-20T04:46:01.913155+0000","last_undegraded":"2023-12-20T04:46:01.913155+0000","last_fullsized":"2023-12-20T04:46:01.913155+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:58:51.244318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.6","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913562+0000","last_change":"2023-12-20T04:46:00.920003+0000","last_active":"2023-12-20T04:46:01.913562+0000","last_peered":"2023-12-20T04:46:01.913562+0000","last_clean":"2023-12-20T04:46:01.913562+0000","last_became_active":"2023-12-20T04:46:00.919107+0000","last_became_peered":"2023-12-20T04:46:00.919107+0000","last_unstale":"2023-12-20T04:46:01.913562+0000","last_undegraded":"2023-12-20T04:46:01.913562+0000","last_fullsized":"2023-12-20T04:46:01.913562+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:30:20.820476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"14.7","version":"122'2","reported_seq":13,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.930440+0000","last_change":"2023-12-20T04:46:00.510596+0000","last_active":"2023-12-20T04:46:01.930440+0000","last_peered":"2023-12-20T04:46:01.930440+0000","last_clean":"2023-12-20T04:46:01.930440+0000","last_became_active":"2023-12-20T04:46:00.509375+0000","last_became_peered":"2023-12-20T04:46:00.509375+0000","last_unstale":"2023-12-20T04:46:01.930440+0000","last_undegraded":"2023-12-20T04:46:01.930440+0000","last_fullsized":"2023-12-20T04:46:01.930440+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:52:12.651653+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"15.7","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.931087+0000","last_change":"2023-12-20T04:46:00.924232+0000","last_active":"2023-12-20T04:46:00.931087+0000","last_peered":"2023-12-20T04:46:00.931087+0000","last_clean":"2023-12-20T04:46:00.931087+0000","last_became_active":"2023-12-20T04:46:00.923596+0000","last_became_peered":"2023-12-20T04:46:00.923596+0000","last_unstale":"2023-12-20T04:46:00.931087+0000","last_undegraded":"2023-12-20T04:46:00.931087+0000","last_fullsized":"2023-12-20T04:46:00.931087+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:08:50.075615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.6","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913630+0000","last_change":"2023-12-20T04:46:00.521573+0000","last_active":"2023-12-20T04:46:01.913630+0000","last_peered":"2023-12-20T04:46:01.913630+0000","last_clean":"2023-12-20T04:46:01.913630+0000","last_became_active":"2023-12-20T04:46:00.521000+0000","last_became_peered":"2023-12-20T04:46:00.521000+0000","last_unstale":"2023-12-20T04:46:01.913630+0000","last_undegraded":"2023-12-20T04:46:01.913630+0000","last_fullsized":"2023-12-20T04:46:01.913630+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:10:06.227641+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.4","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.925376+0000","last_change":"2023-12-20T04:46:00.921513+0000","last_active":"2023-12-20T04:46:00.925376+0000","last_peered":"2023-12-20T04:46:00.925376+0000","last_clean":"2023-12-20T04:46:00.925376+0000","last_became_active":"2023-12-20T04:46:00.921258+0000","last_became_peered":"2023-12-20T04:46:00.921258+0000","last_unstale":"2023-12-20T04:46:00.925376+0000","last_undegraded":"2023-12-20T04:46:00.925376+0000","last_fullsized":"2023-12-20T04:46:00.925376+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:47.193663+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"14.5","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913332+0000","last_change":"2023-12-20T04:46:00.507249+0000","last_active":"2023-12-20T04:46:01.913332+0000","last_peered":"2023-12-20T04:46:01.913332+0000","last_clean":"2023-12-20T04:46:01.913332+0000","last_became_active":"2023-12-20T04:46:00.506929+0000","last_became_peered":"2023-12-20T04:46:00.506929+0000","last_unstale":"2023-12-20T04:46:01.913332+0000","last_undegraded":"2023-12-20T04:46:01.913332+0000","last_fullsized":"2023-12-20T04:46:01.913332+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:11:16.258499+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.5","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.939538+0000","last_change":"2023-12-20T04:46:00.936683+0000","last_active":"2023-12-20T04:46:00.939538+0000","last_peered":"2023-12-20T04:46:00.939538+0000","last_clean":"2023-12-20T04:46:00.939538+0000","last_became_active":"2023-12-20T04:46:00.935436+0000","last_became_peered":"2023-12-20T04:46:00.935436+0000","last_unstale":"2023-12-20T04:46:00.939538+0000","last_undegraded":"2023-12-20T04:46:00.939538+0000","last_fullsized":"2023-12-20T04:46:00.939538+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:25:56.571934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.4","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.909154+0000","last_change":"2023-12-20T04:46:00.514976+0000","last_active":"2023-12-20T04:46:00.909154+0000","last_peered":"2023-12-20T04:46:00.909154+0000","last_clean":"2023-12-20T04:46:00.909154+0000","last_became_active":"2023-12-20T04:46:00.514631+0000","last_became_peered":"2023-12-20T04:46:00.514631+0000","last_unstale":"2023-12-20T04:46:00.909154+0000","last_undegraded":"2023-12-20T04:46:00.909154+0000","last_fullsized":"2023-12-20T04:46:00.909154+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:08:29.597980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.2","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.932411+0000","last_change":"2023-12-20T04:46:00.926423+0000","last_active":"2023-12-20T04:46:00.932411+0000","last_peered":"2023-12-20T04:46:00.932411+0000","last_clean":"2023-12-20T04:46:00.932411+0000","last_became_active":"2023-12-20T04:46:00.926129+0000","last_became_peered":"2023-12-20T04:46:00.926129+0000","last_unstale":"2023-12-20T04:46:00.932411+0000","last_undegraded":"2023-12-20T04:46:00.932411+0000","last_fullsized":"2023-12-20T04:46:00.932411+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:55:48.120147+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.3","version":"0'0","reported_seq":11,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.913265+0000","last_change":"2023-12-20T04:46:00.517528+0000","last_active":"2023-12-20T04:46:01.913265+0000","last_peered":"2023-12-20T04:46:01.913265+0000","last_clean":"2023-12-20T04:46:01.913265+0000","last_became_active":"2023-12-20T04:46:00.517243+0000","last_became_peered":"2023-12-20T04:46:00.517243+0000","last_unstale":"2023-12-20T04:46:01.913265+0000","last_undegraded":"2023-12-20T04:46:01.913265+0000","last_fullsized":"2023-12-20T04:46:01.913265+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:34.893107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"15.3","version":"0'0","reported_seq":10,"reported_epoch":123,"state":"active+clean","last_fresh":"2023-12-20T04:46:01.926885+0000","last_change":"2023-12-20T04:46:00.928554+0000","last_active":"2023-12-20T04:46:01.926885+0000","last_peered":"2023-12-20T04:46:01.926885+0000","last_clean":"2023-12-20T04:46:01.926885+0000","last_became_active":"2023-12-20T04:46:00.928283+0000","last_became_peered":"2023-12-20T04:46:00.928283+0000","last_unstale":"2023-12-20T04:46:01.926885+0000","last_undegraded":"2023-12-20T04:46:01.926885+0000","last_fullsized":"2023-12-20T04:46:01.926885+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:34:06.859228+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"14.2","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.908881+0000","last_change":"2023-12-20T04:46:00.519750+0000","last_active":"2023-12-20T04:46:00.908881+0000","last_peered":"2023-12-20T04:46:00.908881+0000","last_clean":"2023-12-20T04:46:00.908881+0000","last_became_active":"2023-12-20T04:46:00.519468+0000","last_became_peered":"2023-12-20T04:46:00.519468+0000","last_unstale":"2023-12-20T04:46:00.908881+0000","last_undegraded":"2023-12-20T04:46:00.908881+0000","last_fullsized":"2023-12-20T04:46:00.908881+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:26:34.860037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"15.0","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.932098+0000","last_change":"2023-12-20T04:46:00.928204+0000","last_active":"2023-12-20T04:46:00.932098+0000","last_peered":"2023-12-20T04:46:00.932098+0000","last_clean":"2023-12-20T04:46:00.932098+0000","last_became_active":"2023-12-20T04:46:00.927898+0000","last_became_peered":"2023-12-20T04:46:00.927898+0000","last_unstale":"2023-12-20T04:46:00.932098+0000","last_undegraded":"2023-12-20T04:46:00.932098+0000","last_fullsized":"2023-12-20T04:46:00.932098+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:58:42.597523+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.1","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.916582+0000","last_change":"2023-12-20T04:46:00.524160+0000","last_active":"2023-12-20T04:46:00.916582+0000","last_peered":"2023-12-20T04:46:00.916582+0000","last_clean":"2023-12-20T04:46:00.916582+0000","last_became_active":"2023-12-20T04:46:00.523650+0000","last_became_peered":"2023-12-20T04:46:00.523650+0000","last_unstale":"2023-12-20T04:46:00.916582+0000","last_undegraded":"2023-12-20T04:46:00.916582+0000","last_fullsized":"2023-12-20T04:46:00.916582+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:21.800243+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"15.1","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.940371+0000","last_change":"2023-12-20T04:46:00.937998+0000","last_active":"2023-12-20T04:46:00.940371+0000","last_peered":"2023-12-20T04:46:00.940371+0000","last_clean":"2023-12-20T04:46:00.940371+0000","last_became_active":"2023-12-20T04:46:00.937707+0000","last_became_peered":"2023-12-20T04:46:00.937707+0000","last_unstale":"2023-12-20T04:46:00.940371+0000","last_undegraded":"2023-12-20T04:46:00.940371+0000","last_fullsized":"2023-12-20T04:46:00.940371+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:09:37.604790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"14.0","version":"0'0","reported_seq":10,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.915854+0000","last_change":"2023-12-20T04:46:00.518737+0000","last_active":"2023-12-20T04:46:00.915854+0000","last_peered":"2023-12-20T04:46:00.915854+0000","last_clean":"2023-12-20T04:46:00.915854+0000","last_became_active":"2023-12-20T04:46:00.518466+0000","last_became_peered":"2023-12-20T04:46:00.518466+0000","last_unstale":"2023-12-20T04:46:00.915854+0000","last_undegraded":"2023-12-20T04:46:00.915854+0000","last_fullsized":"2023-12-20T04:46:00.915854+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:20:38.437718+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"15.e","version":"0'0","reported_seq":9,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.944012+0000","last_change":"2023-12-20T04:46:00.942173+0000","last_active":"2023-12-20T04:46:00.944012+0000","last_peered":"2023-12-20T04:46:00.944012+0000","last_clean":"2023-12-20T04:46:00.944012+0000","last_became_active":"2023-12-20T04:46:00.941922+0000","last_became_peered":"2023-12-20T04:46:00.941922+0000","last_unstale":"2023-12-20T04:46:00.944012+0000","last_undegraded":"2023-12-20T04:46:00.944012+0000","last_fullsized":"2023-12-20T04:46:00.944012+0000","mapping_epoch":121,"log_start":"0'0","ondisk_log_start":"0'0","created":121,"last_epoch_clean":122,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:46:00.483109+0000","last_clean_scrub_stamp":"2023-12-20T04:46:00.483109+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:24:38.748808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"14.f","version":"122'1","reported_seq":11,"reported_epoch":122,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.944199+0000","last_change":"2023-12-20T04:46:00.521396+0000","last_active":"2023-12-20T04:46:00.944199+0000","last_peered":"2023-12-20T04:46:00.944199+0000","last_clean":"2023-12-20T04:46:00.944199+0000","last_became_active":"2023-12-20T04:46:00.521181+0000","last_became_peered":"2023-12-20T04:46:00.521181+0000","last_unstale":"2023-12-20T04:46:00.944199+0000","last_undegraded":"2023-12-20T04:46:00.944199+0000","last_fullsized":"2023-12-20T04:46:00.944199+0000","mapping_epoch":120,"log_start":"0'0","ondisk_log_start":"0'0","created":120,"last_epoch_clean":121,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:45:59.474887+0000","last_clean_scrub_stamp":"2023-12-20T04:45:59.474887+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:44:50.850604+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"111'3080","reported_seq":3449,"reported_epoch":121,"state":"active+clean","last_fresh":"2023-12-20T04:46:00.511521+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:46:00.511521+0000","last_peered":"2023-12-20T04:46:00.511521+0000","last_clean":"2023-12-20T04:46:00.511521+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:46:00.511521+0000","last_undegraded":"2023-12-20T04:46:00.511521+0000","last_fullsized":"2023-12-20T04:46:00.511521+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3080,"log_dups_size":0,"ondisk_log_size":3080,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":829,"num_read_kb":1583,"num_write":301,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":15,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":14,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":829,"num_read_kb":1583,"num_write":301,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3080,"ondisk_log_size":3080,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575058,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33108,"kb_used_data":2060,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338732,"statfs":{"total":96636764160,"available":96602861568,"internally_reserved":0,"allocated":2109440,"data_stored":1304952,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.68200000000000005,"5min":0.66600000000000004,"15min":0.70099999999999996},"min":{"1min":0.47599999999999998,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.87,"5min":1.0580000000000001,"15min":1.0600000000000001},"last":0.432},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.45700000000000002,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.91500000000000004,"5min":0.93100000000000005,"15min":1.115},"last":0.68000000000000005}]},{"osd":1,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.68899999999999995,"15min":0.72199999999999998},"min":{"1min":0.47499999999999998,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.93000000000000005,"5min":1.095,"15min":1.095},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.69999999999999996,"5min":0.69499999999999995,"15min":0.72699999999999998},"min":{"1min":0.503,"5min":0.377,"15min":0.377},"max":{"1min":0.85199999999999998,"5min":0.95099999999999996,"15min":1.1080000000000001},"last":0.625}]},{"osd":2,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.67800000000000005,"15min":0.70699999999999996},"min":{"1min":0.44800000000000001,"5min":0.379,"15min":0.379},"max":{"1min":0.95299999999999996,"5min":1.075,"15min":1.075},"last":0.56699999999999995},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72299999999999998,"15min":0.747},"min":{"1min":0.52800000000000002,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.90000000000000002,"5min":1.048,"15min":1.1000000000000001},"last":0.48399999999999999}]},{"osd":3,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.69599999999999995,"5min":0.68600000000000005,"15min":0.72099999999999997},"min":{"1min":0.44700000000000001,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.96399999999999997,"5min":1.03,"15min":1.03},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73599999999999999,"15min":0.755},"min":{"1min":0.50600000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.96399999999999997,"5min":1.012,"15min":1.149},"last":0.52600000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.57199999999999995,"5min":0.59299999999999997,"15min":0.58999999999999997},"min":{"1min":0.42699999999999999,"5min":0.37,"15min":0.34599999999999997},"max":{"1min":0.77100000000000002,"5min":0.90700000000000003,"15min":1.0109999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.624,"15min":0.61599999999999999},"min":{"1min":0.44400000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.82199999999999995,"5min":0.97599999999999998,"15min":1.0469999999999999},"last":0.69799999999999995}]},{"osd":5,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.66300000000000003,"15min":0.64500000000000002},"min":{"1min":0.40500000000000003,"5min":0.40500000000000003,"15min":0.40400000000000003},"max":{"1min":0.91800000000000004,"5min":1.089,"15min":1.0900000000000001},"last":0.64400000000000002},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.68899999999999995,"15min":0.66800000000000004},"min":{"1min":0.48399999999999999,"5min":0.436,"15min":0.432},"max":{"1min":0.91500000000000004,"5min":0.98799999999999999,"15min":1.071},"last":0.76300000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:45:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67600000000000005,"5min":0.73499999999999999,"15min":0.72699999999999998},"min":{"1min":0.51000000000000001,"5min":0.46800000000000003,"15min":0.443},"max":{"1min":0.89200000000000002,"5min":1.093,"15min":1.1279999999999999},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.70899999999999996,"15min":0.69899999999999995},"min":{"1min":0.49399999999999999,"5min":0.46100000000000002,"15min":0.44400000000000001},"max":{"1min":0.91700000000000004,"5min":1.125,"15min":1.125},"last":0.73899999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575058,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36624,"kb_used_data":1480,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335216,"statfs":{"total":96636764160,"available":96599261184,"internally_reserved":0,"allocated":1515520,"data_stored":715124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.67600000000000005,"15min":0.72199999999999998},"min":{"1min":0.40899999999999997,"5min":0.33800000000000002,"15min":0.27100000000000002},"max":{"1min":0.86699999999999999,"5min":1.1140000000000001,"15min":1.2649999999999999},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.68799999999999994,"5min":0.68300000000000005,"15min":0.72199999999999998},"min":{"1min":0.39300000000000002,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.1200000000000001,"5min":1.1200000000000001,"15min":1.1910000000000001},"last":0.63100000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.65500000000000003,"15min":0.70299999999999996},"min":{"1min":0.38600000000000001,"5min":0.34799999999999998,"15min":0.34799999999999998},"max":{"1min":0.98599999999999999,"5min":1.016,"15min":1.016},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.69099999999999995,"15min":0.72899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":1.1879999999999999,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.55500000000000005}]},{"osd":2,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.73099999999999998,"15min":0.76300000000000001},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.40899999999999997},"max":{"1min":1.0229999999999999,"5min":1.0580000000000001,"15min":1.216},"last":0.77100000000000002},{"interface":"front","average":{"1min":0.65700000000000003,"5min":0.69599999999999995,"15min":0.73399999999999999},"min":{"1min":0.42899999999999999,"5min":0.40400000000000003,"15min":0.38300000000000001},"max":{"1min":1.0369999999999999,"5min":1.141,"15min":1.141},"last":0.71699999999999997}]},{"osd":3,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.73799999999999999,"15min":0.76200000000000001},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96199999999999997,"5min":1.093,"15min":1.109},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.69399999999999995,"15min":0.73399999999999999},"min":{"1min":0.44,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.087,"5min":1.087,"15min":1.087},"last":0.73799999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.56000000000000005,"5min":0.60599999999999998,"15min":0.61099999999999999},"min":{"1min":0.379,"5min":0.379,"15min":0.36399999999999999},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.1040000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.64000000000000001,"15min":0.623},"min":{"1min":0.46000000000000002,"5min":0.378,"15min":0.36899999999999999},"max":{"1min":0.93500000000000005,"5min":0.93700000000000006,"15min":0.995},"last":0.84899999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.64400000000000002,"5min":0.621,"15min":0.61599999999999999},"min":{"1min":0.44,"5min":0.307,"15min":0.307},"max":{"1min":0.93799999999999994,"5min":0.93799999999999994,"15min":1.1519999999999999},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.68400000000000005,"15min":0.67600000000000005},"min":{"1min":0.42399999999999999,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.98599999999999999,"5min":1,"15min":1.306},"last":0.94799999999999995}]},{"osd":7,"last update":"Wed Dec 20 04:45:00 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71799999999999997,"15min":0.71099999999999997},"min":{"1min":0.498,"5min":0.49099999999999999,"15min":0.40400000000000003},"max":{"1min":1.0069999999999999,"5min":1.0129999999999999,"15min":1.0149999999999999},"last":0.89000000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.71499999999999997,"15min":0.69899999999999995},"min":{"1min":0.47299999999999998,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.054,"5min":1.054,"15min":1.054},"last":0.872}]}]},{"osd":5,"up_from":13,"seq":55834575059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36632,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335208,"statfs":{"total":96636764160,"available":96599252992,"internally_reserved":0,"allocated":1523712,"data_stored":720842,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.751,"5min":0.68300000000000005,"15min":0.70299999999999996},"min":{"1min":0.60399999999999998,"5min":0.36199999999999999,"15min":0.29999999999999999},"max":{"1min":0.90300000000000002,"5min":0.97199999999999998,"15min":1.05},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.68500000000000005,"15min":0.71099999999999997},"min":{"1min":0.53400000000000003,"5min":0.35899999999999999,"15min":0.34300000000000003},"max":{"1min":0.96799999999999997,"5min":0.96899999999999997,"15min":0.98599999999999999},"last":0.50800000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.68700000000000006,"15min":0.70399999999999996},"min":{"1min":0.54900000000000004,"5min":0.443,"15min":0.441},"max":{"1min":0.89000000000000001,"5min":0.98399999999999999,"15min":1.0600000000000001},"last":0.72999999999999998},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.70699999999999996,"15min":0.73299999999999998},"min":{"1min":0.57699999999999996,"5min":0.437,"15min":0.38400000000000001},"max":{"1min":0.95999999999999996,"5min":0.96999999999999997,"15min":1.0309999999999999},"last":0.69099999999999995}]},{"osd":2,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.71799999999999997,"15min":0.73199999999999998},"min":{"1min":0.52200000000000002,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":1.014},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.71499999999999997,"15min":0.72899999999999998},"min":{"1min":0.57499999999999996,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.004},"last":0.79100000000000004}]},{"osd":3,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.73799999999999999,"15min":0.74299999999999999},"min":{"1min":0.61499999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.92500000000000004,"5min":0.99199999999999999,"15min":1.018},"last":0.75},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.73599999999999999,"15min":0.754},"min":{"1min":0.55400000000000005,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.002,"5min":1.008,"15min":1.0469999999999999},"last":0.76700000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.62,"15min":0.61299999999999999},"min":{"1min":0.434,"5min":0.42099999999999999,"15min":0.35099999999999998},"max":{"1min":1.0069999999999999,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.63200000000000001,"5min":0.64100000000000001,"15min":0.627},"min":{"1min":0.47299999999999998,"5min":0.40300000000000002,"15min":0.39600000000000002},"max":{"1min":0.89600000000000002,"5min":0.98699999999999999,"15min":1.077},"last":0.81000000000000005}]},{"osd":6,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.71399999999999997,"5min":0.69299999999999995,"15min":0.68100000000000005},"min":{"1min":0.495,"5min":0.47599999999999998,"15min":0.39900000000000002},"max":{"1min":0.96599999999999997,"5min":0.96599999999999997,"15min":1.056},"last":0.871},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.63700000000000001,"15min":0.63100000000000001},"min":{"1min":0.45400000000000001,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":0.88100000000000001,"5min":0.96199999999999997,"15min":0.999},"last":0.90800000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.69499999999999995,"15min":0.68899999999999995},"min":{"1min":0.47799999999999998,"5min":0.45300000000000001,"15min":0.442},"max":{"1min":0.89400000000000002,"5min":1.0429999999999999,"15min":1.141},"last":0.88400000000000001},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.754,"15min":0.751},"min":{"1min":0.55700000000000005,"5min":0.48999999999999999,"15min":0.47899999999999998},"max":{"1min":0.93500000000000005,"5min":1.012,"15min":1.0780000000000001},"last":0.92500000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575060,"num_pgs":3,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58792,"kb_used_data":1504,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94313048,"statfs":{"total":96636764160,"available":96576561152,"internally_reserved":0,"allocated":1540096,"data_stored":722460,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.66300000000000003,"15min":0.69599999999999995},"min":{"1min":0.42999999999999999,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.96899999999999997,"5min":1.024,"15min":1.5509999999999999},"last":0.503},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.41599999999999998,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.97799999999999998,"5min":0.98599999999999999,"15min":1.633},"last":0.70299999999999996}]},{"osd":1,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.76300000000000001,"5min":0.68899999999999995,"15min":0.71899999999999997},"min":{"1min":0.52200000000000002,"5min":0.315,"15min":0.315},"max":{"1min":0.96099999999999997,"5min":0.96099999999999997,"15min":1.798},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.72799999999999998,"5min":0.71099999999999997,"15min":0.74199999999999999},"min":{"1min":0.58699999999999997,"5min":0.372,"15min":0.372},"max":{"1min":0.88400000000000001,"5min":1.036,"15min":2.2930000000000001},"last":0.77400000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.72699999999999998,"15min":0.74299999999999999},"min":{"1min":0.61099999999999999,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.98399999999999999,"5min":1.0209999999999999,"15min":1.728},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.748,"5min":0.72499999999999998,"15min":0.748},"min":{"1min":0.5,"5min":0.46899999999999997,"15min":0.38700000000000001},"max":{"1min":0.92300000000000004,"5min":1.0289999999999999,"15min":1.6619999999999999},"last":0.72699999999999998}]},{"osd":3,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.70099999999999996,"15min":0.72599999999999998},"min":{"1min":0.50900000000000001,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.98999999999999999,"5min":1.004,"15min":1.5760000000000001},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.73599999999999999,"15min":0.76100000000000001},"min":{"1min":0.56899999999999995,"5min":0.372,"15min":0.372},"max":{"1min":0.95599999999999996,"5min":0.98399999999999999,"15min":1.7649999999999999},"last":0.746}]},{"osd":5,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.63100000000000001,"15min":0.61299999999999999},"min":{"1min":0.46899999999999997,"5min":0.36599999999999999,"15min":0.36099999999999999},"max":{"1min":0.85999999999999999,"5min":1.1120000000000001,"15min":1.456},"last":0.57399999999999995},{"interface":"front","average":{"1min":0.64200000000000002,"5min":0.623,"15min":0.621},"min":{"1min":0.42499999999999999,"5min":0.39300000000000002,"15min":0.33800000000000002},"max":{"1min":0.97399999999999998,"5min":1.077,"15min":1.589},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.67400000000000004,"5min":0.66200000000000003,"15min":0.64900000000000002},"min":{"1min":0.47599999999999998,"5min":0.35999999999999999,"15min":0.35999999999999999},"max":{"1min":0.998,"5min":1.006,"15min":1.6739999999999999},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.69899999999999995,"15min":0.68899999999999995},"min":{"1min":0.47499999999999998,"5min":0.41899999999999998,"15min":0.41699999999999998},"max":{"1min":0.94099999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.65700000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:45:55 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.68300000000000005,"15min":0.68600000000000005},"min":{"1min":0.46500000000000002,"5min":0.40300000000000002,"15min":0.38400000000000001},"max":{"1min":0.91600000000000004,"5min":1.036,"15min":1.655},"last":0.79200000000000004},{"interface":"front","average":{"1min":0.749,"5min":0.71999999999999997,"15min":0.72899999999999998},"min":{"1min":0.54400000000000004,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":1.042,"5min":1.052,"15min":1.125},"last":0.68100000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575059,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36632,"kb_used_data":1488,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335208,"statfs":{"total":96636764160,"available":96599252992,"internally_reserved":0,"allocated":1523712,"data_stored":716058,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.41399999999999998,"5min":0.436,"15min":0.45200000000000001},"min":{"1min":0.313,"5min":0.26100000000000001,"15min":0.23200000000000001},"max":{"1min":0.58999999999999997,"5min":0.81699999999999995,"15min":0.875},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.495,"5min":0.48599999999999999,"15min":0.51200000000000001},"min":{"1min":0.30599999999999999,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.76300000000000001,"5min":0.77600000000000002,"15min":0.80800000000000005},"last":0.36099999999999999}]},{"osd":1,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.441,"5min":0.46400000000000002,"15min":0.47099999999999997},"min":{"1min":0.36599999999999999,"5min":0.29299999999999998,"15min":0.27400000000000002},"max":{"1min":0.63800000000000001,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.54600000000000004,"5min":0.52600000000000002,"15min":0.54800000000000004},"min":{"1min":0.39200000000000002,"5min":0.35499999999999998,"15min":0.28399999999999997},"max":{"1min":0.67100000000000004,"5min":0.74399999999999999,"15min":0.88900000000000001},"last":0.46700000000000003}]},{"osd":2,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.55400000000000005,"5min":0.56299999999999994,"15min":0.57899999999999996},"min":{"1min":0.439,"5min":0.374,"15min":0.318},"max":{"1min":0.69799999999999995,"5min":0.80700000000000005,"15min":0.88700000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.56000000000000005,"5min":0.53800000000000003,"15min":0.55000000000000004},"min":{"1min":0.40500000000000003,"5min":0.316,"15min":0.316},"max":{"1min":0.81299999999999994,"5min":0.81799999999999995,"15min":0.93300000000000005},"last":0.433}]},{"osd":4,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.73999999999999999,"15min":0.77900000000000003},"min":{"1min":0.51800000000000002,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.0880000000000001,"5min":1.0960000000000001,"15min":1.131},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.75700000000000001,"15min":0.79700000000000004},"min":{"1min":0.51500000000000001,"5min":0.50700000000000001,"15min":0.41599999999999998},"max":{"1min":1.069,"5min":1.198,"15min":1.4059999999999999},"last":0.82699999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.77300000000000002,"15min":0.80400000000000005},"min":{"1min":0.55000000000000004,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":0.99099999999999999,"5min":1.181,"15min":1.3300000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.747,"15min":0.77900000000000003},"min":{"1min":0.63800000000000001,"5min":0.503,"15min":0.38400000000000001},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.252},"last":0.67300000000000004}]},{"osd":6,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.73499999999999999,"15min":0.78700000000000003},"min":{"1min":0.53800000000000003,"5min":0.49099999999999999,"15min":0.45600000000000002},"max":{"1min":1.016,"5min":1.0660000000000001,"15min":1.2050000000000001},"last":0.69299999999999995},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.77300000000000002,"15min":0.81100000000000005},"min":{"1min":0.58899999999999997,"5min":0.436,"15min":0.436},"max":{"1min":1.071,"5min":1.133,"15min":1.2929999999999999},"last":0.78400000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:45:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82599999999999996,"5min":0.74199999999999999,"15min":0.78600000000000003},"min":{"1min":0.67000000000000004,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":1.0029999999999999,"5min":1.0249999999999999,"15min":1.1699999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.76300000000000001,"15min":0.81100000000000005},"min":{"1min":0.60599999999999998,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":1.0629999999999999,"5min":1.145,"15min":1.29},"last":0.79800000000000004}]}]},{"osd":2,"up_from":13,"seq":55834575058,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":40716,"kb_used_data":1476,"kb_used_omap":0,"kb_used_meta":39232,"kb_avail":94331124,"statfs":{"total":96636764160,"available":96595070976,"internally_reserved":0,"allocated":1511424,"data_stored":714506,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":40173568},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.40999999999999998,"15min":0.438},"min":{"1min":0.28399999999999997,"5min":0.216,"15min":0.216},"max":{"1min":0.86699999999999999,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.42799999999999999,"5min":0.46800000000000003,"15min":0.49099999999999999},"min":{"1min":0.30299999999999999,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.79800000000000004,"5min":0.92200000000000004,"15min":1.0589999999999999},"last":0.38500000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.42699999999999999,"5min":0.45400000000000001,"15min":0.47199999999999998},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.78100000000000003,"5min":0.78100000000000003,"15min":0.876},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.50800000000000001,"5min":0.5,"15min":0.52500000000000002},"min":{"1min":0.32300000000000001,"5min":0.32000000000000001,"15min":0.28100000000000003},"max":{"1min":1.1759999999999999,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.40400000000000003}]},{"osd":3,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.56499999999999995,"15min":0.59899999999999998},"min":{"1min":0.32600000000000001,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.69699999999999995,"5min":0.89800000000000002,"15min":0.93899999999999995},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.59399999999999997,"5min":0.60999999999999999,"15min":0.63900000000000001},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.76600000000000001,"5min":0.91600000000000004,"15min":0.95599999999999996},"last":0.49299999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.73099999999999998,"15min":0.77500000000000002},"min":{"1min":0.495,"5min":0.46500000000000002,"15min":0.45200000000000001},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.1799999999999999},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.73799999999999999,"15min":0.77500000000000002},"min":{"1min":0.55300000000000005,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":1.361,"5min":1.361,"15min":1.361},"last":0.55000000000000004}]},{"osd":5,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.71799999999999997,"15min":0.77000000000000002},"min":{"1min":0.51000000000000001,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":1.242,"5min":1.242,"15min":1.242},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.751,"15min":0.79600000000000004},"min":{"1min":0.54400000000000004,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":1.1779999999999999,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.56899999999999995}]},{"osd":6,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.752,"15min":0.78700000000000003},"min":{"1min":0.48799999999999999,"5min":0.436,"15min":0.436},"max":{"1min":1.323,"5min":1.323,"15min":1.323},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73699999999999999,"15min":0.79000000000000004},"min":{"1min":0.54500000000000004,"5min":0.502,"15min":0.46100000000000002},"max":{"1min":1.1060000000000001,"5min":1.1060000000000001,"15min":1.147},"last":0.63200000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:45:02 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78000000000000003,"15min":0.81799999999999995},"min":{"1min":0.54500000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.3919999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76800000000000002,"5min":0.78400000000000003,"15min":0.84299999999999997},"min":{"1min":0.59399999999999997,"5min":0.52400000000000002,"15min":0.504},"max":{"1min":1.3280000000000001,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.66100000000000003}]}]},{"osd":1,"up_from":13,"seq":55834575060,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50568,"kb_used_data":1472,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321272,"statfs":{"total":96636764160,"available":96584982528,"internally_reserved":0,"allocated":1507328,"data_stored":714472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42099999999999999,"15min":0.441},"min":{"1min":0.23400000000000001,"5min":0.22600000000000001,"15min":0.22600000000000001},"max":{"1min":0.59399999999999997,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.39800000000000002},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.47199999999999998,"15min":0.49099999999999999},"min":{"1min":0.26100000000000001,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.83799999999999997,"5min":0.83799999999999997,"15min":0.92100000000000004},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46000000000000002,"5min":0.495,"15min":0.51300000000000001},"min":{"1min":0.29199999999999998,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.67500000000000004,"5min":0.81399999999999995,"15min":1.091},"last":0.46600000000000003},{"interface":"front","average":{"1min":0.48399999999999999,"5min":0.502,"15min":0.54100000000000004},"min":{"1min":0.30299999999999999,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.71199999999999997,"5min":0.85899999999999999,"15min":0.875},"last":0.45300000000000001}]},{"osd":3,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.55500000000000005,"5min":0.57099999999999995,"15min":0.59499999999999997},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.36599999999999999},"max":{"1min":0.83899999999999997,"5min":0.93600000000000005,"15min":0.96499999999999997},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.504,"15min":0.52800000000000002},"min":{"1min":0.33300000000000002,"5min":0.29699999999999999,"15min":0.29699999999999999},"max":{"1min":0.63900000000000001,"5min":0.82399999999999995,"15min":1.2410000000000001},"last":0.436}]},{"osd":4,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.70199999999999996,"15min":0.76000000000000001},"min":{"1min":0.45900000000000002,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0209999999999999,"5min":1.1579999999999999,"15min":1.27},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.73499999999999999,"15min":0.78000000000000003},"min":{"1min":0.49399999999999999,"5min":0.434,"15min":0.434},"max":{"1min":1.0409999999999999,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.73999999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.73799999999999999,"15min":0.77200000000000002},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.0800000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.72799999999999998,"15min":0.78100000000000003},"min":{"1min":0.52500000000000002,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":1.0649999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.72299999999999998}]},{"osd":6,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.71199999999999997,"5min":0.74099999999999999,"15min":0.78900000000000003},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.95899999999999996,"5min":1.232,"15min":1.232},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.70599999999999996,"5min":0.73099999999999998,"15min":0.78200000000000003},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.99299999999999999,"5min":1.252,"15min":1.252},"last":0.75600000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:45:06 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.78400000000000003,"15min":0.82799999999999996},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.002,"5min":1.306,"15min":1.306},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.77300000000000002,"15min":0.82799999999999996},"min":{"1min":0.54600000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.91900000000000004,"5min":1.0860000000000001,"15min":1.3440000000000001},"last":0.76800000000000002}]}]},{"osd":0,"up_from":13,"seq":55834575059,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29020,"kb_used_data":2068,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342820,"statfs":{"total":96636764160,"available":96607047680,"internally_reserved":0,"allocated":2117632,"data_stored":1311722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.43099999999999999,"15min":0.44800000000000001},"min":{"1min":0.26000000000000001,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":0.72299999999999998,"5min":0.83699999999999997,"15min":2.5640000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.47599999999999998,"5min":0.496,"15min":0.53800000000000003},"min":{"1min":0.307,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.63900000000000001,"5min":0.72499999999999998,"15min":7.8860000000000001},"last":0.433}]},{"osd":2,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.49399999999999999,"5min":0.498,"15min":0.52100000000000002},"min":{"1min":0.33000000000000002,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.68100000000000005,"5min":0.76300000000000001,"15min":2.528},"last":0.5},{"interface":"front","average":{"1min":0.56699999999999995,"5min":0.53400000000000003,"15min":0.55200000000000005},"min":{"1min":0.37,"5min":0.318,"15min":0.318},"max":{"1min":0.86399999999999999,"5min":0.86399999999999999,"15min":1.2989999999999999},"last":0.58499999999999996}]},{"osd":3,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65400000000000003,"5min":0.54000000000000004,"15min":0.53600000000000003},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":3.0350000000000001,"5min":3.0350000000000001,"15min":3.1000000000000001},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.58199999999999996,"5min":0.58099999999999996,"15min":0.626},"min":{"1min":0.441,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.77000000000000002,"5min":0.84499999999999997,"15min":10.49},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.75,"15min":0.79000000000000004},"min":{"1min":0.59799999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.321,"5min":1.321,"15min":3.0659999999999998},"last":0.81200000000000006},{"interface":"front","average":{"1min":0.84099999999999997,"5min":0.77600000000000002,"15min":0.80800000000000005},"min":{"1min":0.59899999999999998,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.3080000000000001,"5min":1.3080000000000001,"15min":5.2460000000000004},"last":0.65400000000000003}]},{"osd":5,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.81000000000000005,"5min":0.76700000000000002,"15min":0.79800000000000004},"min":{"1min":0.57799999999999996,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.073,"5min":1.1679999999999999,"15min":1.3109999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.78400000000000003,"15min":0.79800000000000004},"min":{"1min":0.624,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.4319999999999999,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.76600000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85999999999999999,"5min":0.78500000000000003,"15min":0.82499999999999996},"min":{"1min":0.65200000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.345,"5min":1.345,"15min":5.2839999999999998},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.79200000000000004,"15min":0.81200000000000006},"min":{"1min":0.67300000000000004,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.2529999999999999,"5min":1.306,"15min":1.306},"last":0.64000000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:45:58 2023","interfaces":[{"interface":"back","average":{"1min":0.85899999999999999,"5min":0.80900000000000005,"15min":0.82899999999999996},"min":{"1min":0.69099999999999995,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.1180000000000001,"5min":1.2549999999999999,"15min":1.2549999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.83299999999999996,"5min":0.80100000000000005,"15min":0.81299999999999994},"min":{"1min":0.63300000000000001,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":1.1639999999999999,"5min":1.1639999999999999,"15min":1.276},"last":0.66700000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1586,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":14,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":15,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:46:05.469 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T04:46:05.469 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T04:46:05.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.659+0000 7fe23f68c640 1 Processor -- start 2023-12-20T04:46:05.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.659+0000 7fe23f68c640 1 -- start start 2023-12-20T04:46:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.659+0000 7fe23f68c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe23815ad20 0x7fe2380cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.659+0000 7fe23f68c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2380cb1c0 0x7fe2380cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:05.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.659+0000 7fe23f68c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2380cc9d0 con 0x7fe23815a940 2023-12-20T04:46:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.659+0000 7fe23f68c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2380cbbb0 con 0x7fe2380cb1c0 2023-12-20T04:46:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.659+0000 7fe23f68c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2380c9b90 con 0x7fe23815ad20 2023-12-20T04:46:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.659+0000 7fe23dc29640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2380cb1c0 0x7fe2380cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.659+0000 7fe23d428640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe23815a940 0x7fe2380ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36128/0 (socket says 172.21.15.32:36128) 2023-12-20T04:46:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.659+0000 7fe23d428640 1 -- 172.21.15.32:0/2916750621 learned_addr learned my addr 172.21.15.32:0/2916750621 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:46:05.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.659+0000 7fe23cc27640 1 --2- 172.21.15.32:0/2916750621 >> v2:172.21.15.39:3300/0 conn(0x7fe23815ad20 0x7fe2380cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe23dc29640 1 -- 172.21.15.32:0/2916750621 >> v2:172.21.15.39:3300/0 conn(0x7fe23815ad20 msgr2=0x7fe2380cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe23dc29640 1 --2- 172.21.15.32:0/2916750621 >> v2:172.21.15.39:3300/0 conn(0x7fe23815ad20 0x7fe2380cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe23dc29640 1 -- 172.21.15.32:0/2916750621 >> v1:172.21.15.32:6789/0 conn(0x7fe23815a940 legacy=0x7fe2380ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe22ffff640 1 -- 172.21.15.32:0/2916750621 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 425470174 0 0) 0x7fe2380cc9d0 con 0x7fe23815a940 2023-12-20T04:46:05.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe23dc29640 1 -- 172.21.15.32:0/2916750621 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2380c9e70 con 0x7fe2380cb1c0 2023-12-20T04:46:05.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe23dc29640 1 --2- 172.21.15.32:0/2916750621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2380cb1c0 0x7fe2380cb670 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe2380ce460 tx=0x7fe23001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab48148ae55bce10 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe22ffff640 1 -- 172.21.15.32:0/2916750621 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2300a59a0 con 0x7fe2380cb1c0 2023-12-20T04:46:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe22ffff640 1 -- 172.21.15.32:0/2916750621 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2300a5b00 con 0x7fe2380cb1c0 2023-12-20T04:46:05.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe22ffff640 1 -- 172.21.15.32:0/2916750621 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2300a5e90 con 0x7fe2380cb1c0 2023-12-20T04:46:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe23f68c640 1 -- 172.21.15.32:0/2916750621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2380cb1c0 msgr2=0x7fe2380cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe23f68c640 1 --2- 172.21.15.32:0/2916750621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2380cb1c0 0x7fe2380cb670 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fe2380ce460 tx=0x7fe23001c8e0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe23f68c640 1 -- 172.21.15.32:0/2916750621 shutdown_connections 2023-12-20T04:46:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe23f68c640 1 --2- 172.21.15.32:0/2916750621 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2380cb1c0 0x7fe2380cb670 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe23f68c640 1 --2- 172.21.15.32:0/2916750621 >> v2:172.21.15.39:3300/0 conn(0x7fe23815ad20 0x7fe2380cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.663+0000 7fe23f68c640 1 -- 172.21.15.32:0/2916750621 >> 172.21.15.32:0/2916750621 conn(0x7fe23806a4b0 msgr2=0x7fe2380b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23f68c640 1 -- 172.21.15.32:0/2916750621 shutdown_connections 2023-12-20T04:46:05.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23f68c640 1 -- 172.21.15.32:0/2916750621 wait complete. 2023-12-20T04:46:05.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23f68c640 1 Processor -- start 2023-12-20T04:46:05.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23f68c640 1 -- start start 2023-12-20T04:46:05.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23f68c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2380cb1c0 0x7fe2381494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:05.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23f68c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe23815a940 0x7fe2381461d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:05.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23f68c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe23815ad20 0x7fe238157150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:05.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23f68c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380ab970 con 0x7fe2380cb1c0 2023-12-20T04:46:05.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23f68c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380abc10 con 0x7fe23815ad20 2023-12-20T04:46:05.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23f68c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2380abef0 con 0x7fe23815a940 2023-12-20T04:46:05.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23d428640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2380cb1c0 0x7fe2381494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:05.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23d428640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2380cb1c0 0x7fe2381494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37912/0 (socket says 172.21.15.32:37912) 2023-12-20T04:46:05.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23d428640 1 -- 172.21.15.32:0/2549524862 learned_addr learned my addr 172.21.15.32:0/2549524862 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:05.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23dc29640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe23815ad20 0x7fe238157150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23cc27640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe23815a940 0x7fe2381461d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23d428640 1 -- 172.21.15.32:0/2549524862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe23815a940 msgr2=0x7fe2381461d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23d428640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe23815a940 0x7fe2381461d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23d428640 1 -- 172.21.15.32:0/2549524862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe23815ad20 msgr2=0x7fe238157150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23d428640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe23815ad20 0x7fe238157150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23d428640 1 -- 172.21.15.32:0/2549524862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe230009a70 con 0x7fe2380cb1c0 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23dc29640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe23815ad20 0x7fe238157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.667+0000 7fe23d428640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2380cb1c0 0x7fe2381494c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe2340011f0 tx=0x7fe23400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.671+0000 7fe22dffb640 1 -- 172.21.15.32:0/2549524862 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2340074e0 con 0x7fe2380cb1c0 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.671+0000 7fe22dffb640 1 -- 172.21.15.32:0/2549524862 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe234007640 con 0x7fe2380cb1c0 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.671+0000 7fe22dffb640 1 -- 172.21.15.32:0/2549524862 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe234004a10 con 0x7fe2380cb1c0 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.671+0000 7fe23f68c640 1 -- 172.21.15.32:0/2549524862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2380ac1d0 con 0x7fe2380cb1c0 2023-12-20T04:46:05.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.671+0000 7fe23f68c640 1 -- 172.21.15.32:0/2549524862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe238146710 con 0x7fe2380cb1c0 2023-12-20T04:46:05.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.671+0000 7fe23f68c640 1 -- 172.21.15.32:0/2549524862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe23804e000 con 0x7fe2380cb1c0 2023-12-20T04:46:05.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.671+0000 7fe22dffb640 1 -- 172.21.15.32:0/2549524862 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe234004b70 con 0x7fe2380cb1c0 2023-12-20T04:46:05.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.671+0000 7fe22dffb640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe21c065b80 0x7fe21c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:05.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.671+0000 7fe22dffb640 1 -- 172.21.15.32:0/2549524862 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7fe2340826a0 con 0x7fe2380cb1c0 2023-12-20T04:46:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.671+0000 7fe23cc27640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe21c065b80 0x7fe21c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:05.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.671+0000 7fe23cc27640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe21c065b80 0x7fe21c068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe23806e680 tx=0x7fe228008040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:05.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.675+0000 7fe22dffb640 1 -- 172.21.15.32:0/2549524862 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe23404feb0 con 0x7fe2380cb1c0 2023-12-20T04:46:05.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.887+0000 7fe23f68c640 1 -- 172.21.15.32:0/2549524862 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe2380adb30 con 0x7fe2380cb1c0 2023-12-20T04:46:05.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe22dffb640 1 -- 172.21.15.32:0/2549524862 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v21) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe2340180e0 con 0x7fe2380cb1c0 2023-12-20T04:46:05.891 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T04:46:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 -- 172.21.15.32:0/2549524862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe21c065b80 msgr2=0x7fe21c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe21c065b80 0x7fe21c068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe23806e680 tx=0x7fe228008040 comp rx=0 tx=0).stop 2023-12-20T04:46:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 -- 172.21.15.32:0/2549524862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2380cb1c0 msgr2=0x7fe2381494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2380cb1c0 0x7fe2381494c0 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fe2340011f0 tx=0x7fe23400c9b0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 -- 172.21.15.32:0/2549524862 shutdown_connections 2023-12-20T04:46:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe21c065b80 0x7fe21c068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe23815ad20 0x7fe238157150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe23815a940 0x7fe2381461d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 --2- 172.21.15.32:0/2549524862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2380cb1c0 0x7fe2381494c0 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:05.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 -- 172.21.15.32:0/2549524862 >> 172.21.15.32:0/2549524862 conn(0x7fe23806a4b0 msgr2=0x7fe238159ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:05.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 -- 172.21.15.32:0/2549524862 shutdown_connections 2023-12-20T04:46:05.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:05.895+0000 7fe23f68c640 1 -- 172.21.15.32:0/2549524862 wait complete. 2023-12-20T04:46:05.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:46:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63e63f0640 1 Processor -- start 2023-12-20T04:46:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63e63f0640 1 -- start start 2023-12-20T04:46:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63e63f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63e006ce60 0x7f63e015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63e63f0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e015bbd0 con 0x7f63e015f110 2023-12-20T04:46:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63e63f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63e015beb0 con 0x7f63e00af5e0 2023-12-20T04:46:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63e63f0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63e015e910 con 0x7f63e006ce60 2023-12-20T04:46:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63df7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63e006ce60 0x7f63e015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63df7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63e006ce60 0x7f63e015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37916/0 (socket says 172.21.15.32:37916) 2023-12-20T04:46:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63df7fe640 1 -- 172.21.15.32:0/4135427176 learned_addr learned my addr 172.21.15.32:0/4135427176 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63e498d640 1 --1- 172.21.15.32:0/4135427176 >> v1:172.21.15.32:6789/0 conn(0x7f63e015f110 0x7f63e015c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36140/0 (socket says 172.21.15.32:36140) 2023-12-20T04:46:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63df7fe640 1 -- 172.21.15.32:0/4135427176 >> v1:172.21.15.39:6789/0 conn(0x7f63e00af5e0 legacy=0x7f63e00af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63df7fe640 1 -- 172.21.15.32:0/4135427176 >> v1:172.21.15.32:6789/0 conn(0x7f63e015f110 legacy=0x7f63e015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.059+0000 7f63df7fe640 1 -- 172.21.15.32:0/4135427176 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63e015ebf0 con 0x7f63e006ce60 2023-12-20T04:46:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.063+0000 7f63df7fe640 1 --2- 172.21.15.32:0/4135427176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e006ce60 0x7f63e015b280 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009870 tx=0x7f63d401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=283d51aa8284875c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.063+0000 7f63deffd640 1 -- 172.21.15.32:0/4135427176 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e015b9d0 con 0x7f63e006ce60 2023-12-20T04:46:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.063+0000 7f63deffd640 1 -- 172.21.15.32:0/4135427176 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d40ab610 con 0x7f63e006ce60 2023-12-20T04:46:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.063+0000 7f63deffd640 1 -- 172.21.15.32:0/4135427176 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d40ab9a0 con 0x7f63e006ce60 2023-12-20T04:46:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.063+0000 7f63e63f0640 1 -- 172.21.15.32:0/4135427176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e006ce60 msgr2=0x7f63e015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.063+0000 7f63e63f0640 1 --2- 172.21.15.32:0/4135427176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e006ce60 0x7f63e015b280 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7f63d4009870 tx=0x7f63d401c470 comp rx=0 tx=0).stop 2023-12-20T04:46:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.063+0000 7f63e63f0640 1 -- 172.21.15.32:0/4135427176 shutdown_connections 2023-12-20T04:46:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.063+0000 7f63e63f0640 1 --2- 172.21.15.32:0/4135427176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e006ce60 0x7f63e015b280 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.063+0000 7f63e63f0640 1 -- 172.21.15.32:0/4135427176 >> 172.21.15.32:0/4135427176 conn(0x7f63e006a320 msgr2=0x7f63e00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.063+0000 7f63e63f0640 1 -- 172.21.15.32:0/4135427176 shutdown_connections 2023-12-20T04:46:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.063+0000 7f63e63f0640 1 -- 172.21.15.32:0/4135427176 wait complete. 2023-12-20T04:46:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e63f0640 1 Processor -- start 2023-12-20T04:46:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e63f0640 1 -- start start 2023-12-20T04:46:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e63f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e006ce60 0x7f63e014f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e63f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63e00af5e0 0x7f63e01566e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e63f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e015f110 0x7f63e0156c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e63f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e0071c20 con 0x7f63e015f110 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e63f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63e00712f0 con 0x7f63e006ce60 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e63f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63e0071560 con 0x7f63e00af5e0 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e498d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e015f110 0x7f63e0156c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e498d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e015f110 0x7f63e0156c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37922/0 (socket says 172.21.15.32:37922) 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e498d640 1 -- 172.21.15.32:0/910356627 learned_addr learned my addr 172.21.15.32:0/910356627 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63dffff640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e006ce60 0x7f63e014f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63df7fe640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63e00af5e0 0x7f63e01566e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e498d640 1 -- 172.21.15.32:0/910356627 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63e00af5e0 msgr2=0x7f63e01566e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e498d640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63e00af5e0 0x7f63e01566e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e498d640 1 -- 172.21.15.32:0/910356627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e006ce60 msgr2=0x7f63e014f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e498d640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e006ce60 0x7f63e014f940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e498d640 1 -- 172.21.15.32:0/910356627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63d40094d0 con 0x7f63e015f110 2023-12-20T04:46:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63df7fe640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63e00af5e0 0x7f63e01566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:46:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e498d640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e015f110 0x7f63e0156c20 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c730 tx=0x7f63d800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63dcff9640 1 -- 172.21.15.32:0/910356627 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d800ce40 con 0x7f63e015f110 2023-12-20T04:46:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63dcff9640 1 -- 172.21.15.32:0/910356627 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63d800a640 con 0x7f63e015f110 2023-12-20T04:46:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63dcff9640 1 -- 172.21.15.32:0/910356627 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63d800a9b0 con 0x7f63e015f110 2023-12-20T04:46:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e63f0640 1 -- 172.21.15.32:0/910356627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63e0071840 con 0x7f63e015f110 2023-12-20T04:46:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.067+0000 7f63e63f0640 1 -- 172.21.15.32:0/910356627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63e014fe80 con 0x7f63e015f110 2023-12-20T04:46:06.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.071+0000 7f63e63f0640 1 -- 172.21.15.32:0/910356627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63e0071d60 con 0x7f63e015f110 2023-12-20T04:46:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.071+0000 7f63dcff9640 1 -- 172.21.15.32:0/910356627 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f63d800ab10 con 0x7f63e015f110 2023-12-20T04:46:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.071+0000 7f63dcff9640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63cc065bb0 0x7f63cc068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.071+0000 7f63dcff9640 1 -- 172.21.15.32:0/910356627 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f63d8082a70 con 0x7f63e015f110 2023-12-20T04:46:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.071+0000 7f63dffff640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63cc065bb0 0x7f63cc068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.071+0000 7f63dffff640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63cc065bb0 0x7f63cc068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f63e00711e0 tx=0x7f63d0007e90 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:06.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.075+0000 7f63dcff9640 1 -- 172.21.15.32:0/910356627 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f63e0071d60 con 0x7f63e015f110 2023-12-20T04:46:06.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.267+0000 7f63e63f0640 1 -- 172.21.15.32:0/910356627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f63e0071d60 con 0x7f63e015f110 2023-12-20T04:46:06.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.267+0000 7f63dcff9640 1 -- 172.21.15.32:0/910356627 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4705 (secure 0 0 0) 0x7f63e0071d60 con 0x7f63e015f110 2023-12-20T04:46:06.266 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:46:06.266 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/593676281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":593676281},{"type":"v1","addr":"172.21.15.39:6833","nonce":593676281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2437945959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2437945959},{"type":"v1","addr":"172.21.15.39:6835","nonce":2437945959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7681,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/430674440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":430674440},{"type":"v1","addr":"172.21.15.32:6837","nonce":430674440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:46:00.899089+0000","modified":"2023-12-20T04:46:03.967555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7881":{"gid":7881,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3259359483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3259359483},{"type":"v1","addr":"172.21.15.32:6836","nonce":3259359483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-20T04:46:06.266 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 54 2023-12-20T04:46:06.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 -- 172.21.15.32:0/910356627 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63cc065bb0 msgr2=0x7f63cc068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63cc065bb0 0x7f63cc068070 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f63e00711e0 tx=0x7f63d0007e90 comp rx=0 tx=0).stop 2023-12-20T04:46:06.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 -- 172.21.15.32:0/910356627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e015f110 msgr2=0x7f63e0156c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e015f110 0x7f63e0156c20 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f63d800c730 tx=0x7f63d800cbf0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 -- 172.21.15.32:0/910356627 shutdown_connections 2023-12-20T04:46:06.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f63cc065bb0 0x7f63cc068070 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63e015f110 0x7f63e0156c20 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63e00af5e0 0x7f63e01566e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 --2- 172.21.15.32:0/910356627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63e006ce60 0x7f63e014f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 -- 172.21.15.32:0/910356627 >> 172.21.15.32:0/910356627 conn(0x7f63e006a320 msgr2=0x7f63e00cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:06.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 -- 172.21.15.32:0/910356627 shutdown_connections 2023-12-20T04:46:06.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.271+0000 7f63e63f0640 1 -- 172.21.15.32:0/910356627 wait complete. 2023-12-20T04:46:06.690 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 54, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T04:46:00.899089+0000', 'modified': '2023-12-20T04:46:03.967555+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 7881}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_7881': {'gid': 7881, 'name': 'a', 'rank': 0, 'incarnation': 50, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6836/3259359483', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 3259359483}, {'type': 'v1', 'addr': '172.21.15.32:6836', 'nonce': 3259359483}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [15], 'metadata_pool': 14, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T04:46:06.690 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T04:46:06.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-20T04:46:06.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.839+0000 7f6c5acfc640 1 Processor -- start 2023-12-20T04:46:06.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.839+0000 7f6c5acfc640 1 -- start start 2023-12-20T04:46:06.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.839+0000 7f6c5acfc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6c541599e0 0x7f6c540aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.839+0000 7f6c5acfc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c540c90e0 0x7f6c540b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c5acfc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c540711d0 con 0x7f6c540b0b30 2023-12-20T04:46:06.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c5acfc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c540c98c0 con 0x7f6c541599e0 2023-12-20T04:46:06.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c5acfc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540c9ba0 con 0x7f6c540c90e0 2023-12-20T04:46:06.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c59299640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6c540b0b30 0x7f6c5415d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36146/0 (socket says 172.21.15.32:36146) 2023-12-20T04:46:06.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c59299640 1 -- 172.21.15.32:0/415516747 learned_addr learned my addr 172.21.15.32:0/415516747 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:46:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c58a98640 1 --2- 172.21.15.32:0/415516747 >> v2:172.21.15.39:3300/0 conn(0x7f6c541599e0 0x7f6c540aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c53fff640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c540c90e0 0x7f6c540b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1989035294 0 0) 0x7f6c540711d0 con 0x7f6c540b0b30 2023-12-20T04:46:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c44003530 con 0x7f6c540b0b30 2023-12-20T04:46:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2728970681 0 0) 0x7f6c44003530 con 0x7f6c540b0b30 2023-12-20T04:46:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c540c90e0 msgr2=0x7f6c540b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c540c90e0 0x7f6c540b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 >> v2:172.21.15.39:3300/0 conn(0x7f6c541599e0 msgr2=0x7f6c540aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 --2- 172.21.15.32:0/415516747 >> v2:172.21.15.39:3300/0 conn(0x7f6c541599e0 0x7f6c540aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c540c9e80 con 0x7f6c540b0b30 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f6c4c001a00 con 0x7f6c540b0b30 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 >> v1:172.21.15.32:6789/0 conn(0x7f6c540b0b30 legacy=0x7f6c5415d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c44003980 0x7f6c44003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c440047d0 0x7f6c44004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c540c90e0 0x7f6c44005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540c98c0 con 0x7f6c44003980 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c540c9ba0 con 0x7f6c440047d0 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4400b990 con 0x7f6c540c90e0 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c58a98640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c44003980 0x7f6c44003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c53fff640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c440047d0 0x7f6c44004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c59299640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c540c90e0 0x7f6c44005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c58a98640 1 -- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c540c90e0 msgr2=0x7f6c44005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c58a98640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c540c90e0 0x7f6c44005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c58a98640 1 -- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c440047d0 msgr2=0x7f6c44004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c58a98640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c440047d0 0x7f6c44004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c58a98640 1 -- 172.21.15.32:0/415516747 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c44003530 con 0x7f6c44003980 2023-12-20T04:46:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.843+0000 7f6c59299640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c540c90e0 0x7f6c44005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:46:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c58a98640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c44003980 0x7f6c44003da0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009650 tx=0x7f6c4801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=63bda4fee04532c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480059a0 con 0x7f6c44003980 2023-12-20T04:46:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c48005b00 con 0x7f6c44003980 2023-12-20T04:46:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c537fe640 1 -- 172.21.15.32:0/415516747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480a7680 con 0x7f6c44003980 2023-12-20T04:46:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c5acfc640 1 -- 172.21.15.32:0/415516747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c44003980 msgr2=0x7f6c44003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c5acfc640 1 --2- 172.21.15.32:0/415516747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c44003980 0x7f6c44003da0 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f6c48009650 tx=0x7f6c4801ec70 comp rx=0 tx=0).stop 2023-12-20T04:46:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c59299640 1 -- 172.21.15.32:0/415516747 reap_dead start 2023-12-20T04:46:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c5acfc640 1 -- 172.21.15.32:0/415516747 shutdown_connections 2023-12-20T04:46:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c5acfc640 1 -- 172.21.15.32:0/415516747 >> 172.21.15.32:0/415516747 conn(0x7f6c5406a320 msgr2=0x7f6c540ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c5acfc640 1 -- 172.21.15.32:0/415516747 shutdown_connections 2023-12-20T04:46:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c5acfc640 1 -- 172.21.15.32:0/415516747 wait complete. 2023-12-20T04:46:06.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c5acfc640 1 Processor -- start 2023-12-20T04:46:06.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.847+0000 7f6c5acfc640 1 -- start start 2023-12-20T04:46:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c5acfc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c44003980 0x7f6c541f6130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c5acfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c540abf40 0x7f6c540ac3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c5acfc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c541577f0 0x7f6c54157c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c5acfc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541f6670 con 0x7f6c541577f0 2023-12-20T04:46:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c5acfc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541581b0 con 0x7f6c540abf40 2023-12-20T04:46:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c5acfc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c541f9b70 con 0x7f6c44003980 2023-12-20T04:46:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c59299640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c541577f0 0x7f6c54157c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c59299640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c541577f0 0x7f6c54157c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37932/0 (socket says 172.21.15.32:37932) 2023-12-20T04:46:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c59299640 1 -- 172.21.15.32:0/3685813149 learned_addr learned my addr 172.21.15.32:0/3685813149 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c53fff640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c540abf40 0x7f6c540ac3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c58a98640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c44003980 0x7f6c541f6130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c59299640 1 -- 172.21.15.32:0/3685813149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c44003980 msgr2=0x7f6c541f6130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c59299640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c44003980 0x7f6c541f6130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c59299640 1 -- 172.21.15.32:0/3685813149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c540abf40 msgr2=0x7f6c540ac3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c59299640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c540abf40 0x7f6c540ac3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c59299640 1 -- 172.21.15.32:0/3685813149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c541f8c80 con 0x7f6c541577f0 2023-12-20T04:46:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c59299640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c541577f0 0x7f6c54157c70 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6c540b4540 tx=0x7f6c4c00e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c517fa640 1 -- 172.21.15.32:0/3685813149 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c00eb90 con 0x7f6c541577f0 2023-12-20T04:46:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c517fa640 1 -- 172.21.15.32:0/3685813149 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4c00ed10 con 0x7f6c541577f0 2023-12-20T04:46:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c517fa640 1 -- 172.21.15.32:0/3685813149 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4c013660 con 0x7f6c541577f0 2023-12-20T04:46:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c5acfc640 1 -- 172.21.15.32:0/3685813149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c541f8f60 con 0x7f6c541577f0 2023-12-20T04:46:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.851+0000 7f6c5acfc640 1 -- 172.21.15.32:0/3685813149 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c541f9520 con 0x7f6c541577f0 2023-12-20T04:46:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c517fa640 1 -- 172.21.15.32:0/3685813149 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6c4c0137c0 con 0x7f6c541577f0 2023-12-20T04:46:06.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c517fa640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c38065bb0 0x7f6c38068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c517fa640 1 -- 172.21.15.32:0/3685813149 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c4c083610 con 0x7f6c541577f0 2023-12-20T04:46:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c58a98640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c38065bb0 0x7f6c38068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c5e059000 1 Processor -- start 2023-12-20T04:46:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c5e059000 1 -- start start 2023-12-20T04:46:06.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c58a98640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c38065bb0 0x7f6c38068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c480b0000 tx=0x7f6c480a9040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef2e450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e990 0x55c35eee3c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35eee45a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c5e059000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35ed07c90 con 0x55c35ef2e990 2023-12-20T04:46:06.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c5e059000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35ed39220 con 0x55c35eee41c0 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c5e059000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c35ed367f0 con 0x55c35ef2e070 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e990 0x55c35eee3c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e990 0x55c35eee3c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37948/0 (socket says 172.21.15.32:37948) 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c53fff640 1 -- 172.21.15.32:0/3567930655 learned_addr learned my addr 172.21.15.32:0/3567930655 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c58a98640 1 --2- 172.21.15.32:0/3567930655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef2e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c59299640 1 --2- 172.21.15.32:0/3567930655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35eee45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c53fff640 1 -- 172.21.15.32:0/3567930655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 msgr2=0x55c35ef2e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c53fff640 1 --2- 172.21.15.32:0/3567930655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef2e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c53fff640 1 -- 172.21.15.32:0/3567930655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 msgr2=0x55c35eee45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c53fff640 1 --2- 172.21.15.32:0/3567930655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35eee45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c53fff640 1 -- 172.21.15.32:0/3567930655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c35ea2e890 con 0x55c35ef2e990 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c53fff640 1 --2- 172.21.15.32:0/3567930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e990 0x55c35eee3c80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009cb0 tx=0x7f6c40014780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=274f40656add3976 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c3e7fc640 1 -- 172.21.15.32:0/3567930655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4001d440 con 0x55c35ef2e990 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c3e7fc640 1 -- 172.21.15.32:0/3567930655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c4001d5a0 con 0x55c35ef2e990 2023-12-20T04:46:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.855+0000 7f6c3e7fc640 1 -- 172.21.15.32:0/3567930655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c4001d910 con 0x55c35ef2e990 2023-12-20T04:46:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 -- 172.21.15.32:0/3567930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e990 msgr2=0x55c35eee3c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 --2- 172.21.15.32:0/3567930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e990 0x55c35eee3c80 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f6c40009cb0 tx=0x7f6c40014780 comp rx=0 tx=0).stop 2023-12-20T04:46:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 -- 172.21.15.32:0/3567930655 shutdown_connections 2023-12-20T04:46:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 --2- 172.21.15.32:0/3567930655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35eee45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 --2- 172.21.15.32:0/3567930655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e990 0x55c35eee3c80 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 --2- 172.21.15.32:0/3567930655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef2e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 -- 172.21.15.32:0/3567930655 >> 172.21.15.32:0/3567930655 conn(0x55c35eeea950 msgr2=0x55c35eee2560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 -- 172.21.15.32:0/3567930655 shutdown_connections 2023-12-20T04:46:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 -- 172.21.15.32:0/3567930655 wait complete. 2023-12-20T04:46:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 -- start start 2023-12-20T04:46:06.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 Processor -- start 2023-12-20T04:46:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35eee9720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e070 0x55c35eef0080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e990 0x55c35eef05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35ed320e0 con 0x55c35ef2e070 2023-12-20T04:46:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35ecd1310 con 0x55c35eee41c0 2023-12-20T04:46:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c35ecef770 con 0x55c35ef2e990 2023-12-20T04:46:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 10 client.0 _ll_register_callbacks cb 0x55c35eecfb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:46:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e070 0x55c35eef0080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e070 0x55c35eef0080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37952/0 (socket says 172.21.15.32:37952) 2023-12-20T04:46:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c53fff640 1 -- 172.21.15.32:0/2305424787 learned_addr learned my addr 172.21.15.32:0/2305424787 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:06.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c58a98640 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35eee9720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c59299640 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e990 0x55c35eef05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c53fff640 1 -- 172.21.15.32:0/2305424787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e990 msgr2=0x55c35eef05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c53fff640 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e990 0x55c35eef05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c53fff640 1 -- 172.21.15.32:0/2305424787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 msgr2=0x55c35eee9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c53fff640 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35eee9720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c53fff640 1 -- 172.21.15.32:0/2305424787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c400098e0 con 0x55c35ef2e070 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c59299640 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e990 0x55c35eef05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c53fff640 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e070 0x55c35eef0080 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f6c40020040 tx=0x7f6c400046d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93796c8643a49532 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c337fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c337fe640 1 -- 172.21.15.32:0/2305424787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40003a30 con 0x55c35ef2e070 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c337fe640 1 -- 172.21.15.32:0/2305424787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c40003b90 con 0x55c35ef2e070 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c5e059000 1 -- 172.21.15.32:0/2305424787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c35ed01c10 con 0x55c35ef2e070 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.859+0000 7f6c337fe640 1 -- 172.21.15.32:0/2305424787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c40003d50 con 0x55c35ef2e070 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 10 client.8022 fetch_fsmap learned FSMap version 54 2023-12-20T04:46:06.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 -- 172.21.15.32:0/2305424787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55c35ecf7260 con 0x55c35ef2e070 2023-12-20T04:46:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c337fe640 1 -- 172.21.15.32:0/2305424787 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c35ed01c10 con 0x55c35ef2e070 2023-12-20T04:46:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c337fe640 1 -- 172.21.15.32:0/2305424787 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 54) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6c40011040 con 0x55c35ef2e070 2023-12-20T04:46:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 10 client.8022 fetch_fsmap finished waiting for FSMap version 54 2023-12-20T04:46:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 10 client.8022 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T04:46:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 20 client.8022 populate_metadata read hostname 'smithi032' 2023-12-20T04:46:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.32:6834/3259359483,v1:172.21.15.32:6836/3259359483] conn(0x55c35ef358d0 0x55c35ef35cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 4 client.8022 mds_command: new command op to 7881 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T04:46:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 -- 172.21.15.32:0/2305424787 --> [v2:172.21.15.32:6834/3259359483,v1:172.21.15.32:6836/3259359483] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c35ed30a30 con 0x55c35ef358d0 2023-12-20T04:46:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c58a98640 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.32:6834/3259359483,v1:172.21.15.32:6836/3259359483] conn(0x55c35ef358d0 0x55c35ef35cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c58a98640 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.32:6834/3259359483,v1:172.21.15.32:6836/3259359483] conn(0x55c35ef358d0 0x55c35ef35cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aaf86f118ca8cc1f server_cookie=ac1b8542d5288c19 in_seq=0 out_seq=0 2023-12-20T04:46:06.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c337fe640 10 client.8022 ms_handle_connect on v2:172.21.15.32:6834/3259359483 2023-12-20T04:46:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c337fe640 1 -- 172.21.15.32:0/2305424787 <== mds.0 v2:172.21.15.32:6834/3259359483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c35ed30a30 con 0x55c35ef358d0 2023-12-20T04:46:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c337fe640 10 client.8022 handle_command_reply: tid=0 2023-12-20T04:46:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 client.8022 shutdown 2023-12-20T04:46:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 -- 172.21.15.32:0/2305424787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e070 msgr2=0x55c35eef0080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e070 0x55c35eef0080 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f6c40020040 tx=0x7f6c400046d0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 -- 172.21.15.32:0/2305424787 shutdown_connections 2023-12-20T04:46:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e990 0x55c35eef05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.32:6834/3259359483,v1:172.21.15.32:6836/3259359483] conn(0x55c35ef358d0 0x55c35ef35cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35ef2e070 0x55c35eef0080 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 --2- 172.21.15.32:0/2305424787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35eee9720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c337fe640 0 client.8022 ms_handle_reset on v2:172.21.15.32:6834/3259359483 2023-12-20T04:46:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 -- 172.21.15.32:0/2305424787 >> 172.21.15.32:0/2305424787 conn(0x55c35eeea950 msgr2=0x55c35eee2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 -- 172.21.15.32:0/2305424787 shutdown_connections 2023-12-20T04:46:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 1 -- 172.21.15.32:0/2305424787 wait complete. 2023-12-20T04:46:06.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.863+0000 7f6c5e059000 20 client.8022 trim_cache size 0 max 16384 2023-12-20T04:46:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c5e059000 1 Processor -- start 2023-12-20T04:46:06.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c5e059000 1 -- start start 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eeea950 0x55c35ef85ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eee41c0 0x55c35ef86410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef86950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c5e059000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35e9ef940 con 0x55c35eee41c0 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c5e059000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35eee6d70 con 0x55c35eeea950 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c5e059000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c35ed34c40 con 0x55c35ef2e070 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eee41c0 0x55c35ef86410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eee41c0 0x55c35ef86410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37960/0 (socket says 172.21.15.32:37960) 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c53fff640 1 -- 172.21.15.32:0/1001457979 learned_addr learned my addr 172.21.15.32:0/1001457979 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c58a98640 1 --2- 172.21.15.32:0/1001457979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eeea950 0x55c35ef85ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c59299640 1 --2- 172.21.15.32:0/1001457979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef86950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c53fff640 1 -- 172.21.15.32:0/1001457979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 msgr2=0x55c35ef86950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c53fff640 1 --2- 172.21.15.32:0/1001457979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef86950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c53fff640 1 -- 172.21.15.32:0/1001457979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eeea950 msgr2=0x55c35ef85ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c53fff640 1 --2- 172.21.15.32:0/1001457979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eeea950 0x55c35ef85ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c53fff640 1 -- 172.21.15.32:0/1001457979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c35ecdd3b0 con 0x55c35eee41c0 2023-12-20T04:46:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c53fff640 1 --2- 172.21.15.32:0/1001457979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eee41c0 0x55c35ef86410 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f6c40000c10 tx=0x7f6c40026000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0d0c549d1515c7c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c3effd640 1 -- 172.21.15.32:0/1001457979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40030d80 con 0x55c35eee41c0 2023-12-20T04:46:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c3effd640 1 -- 172.21.15.32:0/1001457979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c40030440 con 0x55c35eee41c0 2023-12-20T04:46:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.879+0000 7f6c3effd640 1 -- 172.21.15.32:0/1001457979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c400307d0 con 0x55c35eee41c0 2023-12-20T04:46:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 -- 172.21.15.32:0/1001457979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eee41c0 msgr2=0x55c35ef86410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 --2- 172.21.15.32:0/1001457979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eee41c0 0x55c35ef86410 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f6c40000c10 tx=0x7f6c40026000 comp rx=0 tx=0).stop 2023-12-20T04:46:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 -- 172.21.15.32:0/1001457979 shutdown_connections 2023-12-20T04:46:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 --2- 172.21.15.32:0/1001457979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef86950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 --2- 172.21.15.32:0/1001457979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eee41c0 0x55c35ef86410 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 --2- 172.21.15.32:0/1001457979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eeea950 0x55c35ef85ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 -- 172.21.15.32:0/1001457979 >> 172.21.15.32:0/1001457979 conn(0x55c35eee9d30 msgr2=0x55c35eeeff40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:06.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 -- 172.21.15.32:0/1001457979 shutdown_connections 2023-12-20T04:46:06.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 -- 172.21.15.32:0/1001457979 wait complete. 2023-12-20T04:46:06.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 -- start start 2023-12-20T04:46:06.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 Processor -- start 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35ef70f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eeea950 0x55c35ef860e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef86620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35eedb9d0 con 0x55c35eeea950 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c35ed271f0 con 0x55c35eee41c0 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c35ed30f70 con 0x55c35ef2e070 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 10 client.0 _ll_register_callbacks cb 0x55c35ece0640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eeea950 0x55c35ef860e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c53fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eeea950 0x55c35ef860e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37976/0 (socket says 172.21.15.32:37976) 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c53fff640 1 -- 172.21.15.32:0/137895588 learned_addr learned my addr 172.21.15.32:0/137895588 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c59299640 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef86620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c58a98640 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35ef70f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c53fff640 1 -- 172.21.15.32:0/137895588 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 msgr2=0x55c35ef86620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c53fff640 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef86620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c53fff640 1 -- 172.21.15.32:0/137895588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 msgr2=0x55c35ef70f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c53fff640 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35ef70f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c53fff640 1 -- 172.21.15.32:0/137895588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55c35ecf7260 con 0x55c35eeea950 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c58a98640 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35ef70f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c53fff640 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eeea950 0x55c35ef860e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f6c40014d00 tx=0x7f6c40009190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a03320e8f8a12e38 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c337fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c337fe640 1 -- 172.21.15.32:0/137895588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c40003c40 con 0x55c35eeea950 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c337fe640 1 -- 172.21.15.32:0/137895588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f6c40003da0 con 0x55c35eeea950 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 -- 172.21.15.32:0/137895588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c35ecf7e00 con 0x55c35eeea950 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c337fe640 1 -- 172.21.15.32:0/137895588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=54) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6c40006ba0 con 0x55c35eeea950 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 10 client.8028 fetch_fsmap learned FSMap version 54 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c5e059000 1 -- 172.21.15.32:0/137895588 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=54,monmap=2+}) v3 -- 0x55c35ea2ecc0 con 0x55c35eeea950 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.883+0000 7f6c337fe640 1 -- 172.21.15.32:0/137895588 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c35ecf7e00 con 0x55c35eeea950 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c337fe640 1 -- 172.21.15.32:0/137895588 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 54) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f6c40011040 con 0x55c35eeea950 2023-12-20T04:46:06.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 10 client.8028 fetch_fsmap finished waiting for FSMap version 54 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 10 client.8028 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 20 client.8028 populate_metadata read hostname 'smithi032' 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.32:6834/3259359483,v1:172.21.15.32:6836/3259359483] conn(0x55c35ef2e990 0x55c35ef86b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 4 client.8028 mds_command: new command op to 7881 tid=0[{"prefix": "status"}] 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 -- 172.21.15.32:0/137895588 --> [v2:172.21.15.32:6834/3259359483,v1:172.21.15.32:6836/3259359483] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c35ed2b160 con 0x55c35ef2e990 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c58a98640 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.32:6834/3259359483,v1:172.21.15.32:6836/3259359483] conn(0x55c35ef2e990 0x55c35ef86b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c58a98640 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.32:6834/3259359483,v1:172.21.15.32:6836/3259359483] conn(0x55c35ef2e990 0x55c35ef86b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=10c6c498900f3071 server_cookie=c8ac48e02b1b0f14 in_seq=0 out_seq=0 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c337fe640 10 client.8028 ms_handle_connect on v2:172.21.15.32:6834/3259359483 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c337fe640 1 -- 172.21.15.32:0/137895588 <== mds.0 v2:172.21.15.32:6834/3259359483 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55c35ed2b160 con 0x55c35ef2e990 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c337fe640 10 client.8028 handle_command_reply: tid=0 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 client.8028 shutdown 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 -- 172.21.15.32:0/137895588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eeea950 msgr2=0x55c35ef860e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eeea950 0x55c35ef860e0 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f6c40014d00 tx=0x7f6c40009190 comp rx=0 tx=0).stop 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 -- 172.21.15.32:0/137895588 shutdown_connections 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c35ef2e070 0x55c35ef86620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.32:6834/3259359483,v1:172.21.15.32:6836/3259359483] conn(0x55c35ef2e990 0x55c35ef86b60 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c35eeea950 0x55c35ef860e0 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c337fe640 0 client.8028 ms_handle_reset on v2:172.21.15.32:6834/3259359483 2023-12-20T04:46:06.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 --2- 172.21.15.32:0/137895588 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c35eee41c0 0x55c35ef70f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 -- 172.21.15.32:0/137895588 >> 172.21.15.32:0/137895588 conn(0x55c35eee9d30 msgr2=0x55c35ef2b060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:06.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 -- 172.21.15.32:0/137895588 shutdown_connections 2023-12-20T04:46:06.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 1 -- 172.21.15.32:0/137895588 wait complete. 2023-12-20T04:46:06.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5e059000 20 client.8028 trim_cache size 0 max 16384 2023-12-20T04:46:06.888 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:46:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T04:46:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T04:46:06.888 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 7881, 2023-12-20T04:46:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T04:46:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T04:46:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T04:46:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9863695979999996, 2023-12-20T04:46:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 54, 2023-12-20T04:46:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 123, 2023-12-20T04:46:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 123, 2023-12-20T04:46:06.889 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.307892950999999 2023-12-20T04:46:06.889 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:46:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 -- 172.21.15.32:0/3685813149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c38065bb0 msgr2=0x7f6c38068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c38065bb0 0x7f6c38068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f6c480b0000 tx=0x7f6c480a9040 comp rx=0 tx=0).stop 2023-12-20T04:46:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 -- 172.21.15.32:0/3685813149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c541577f0 msgr2=0x7f6c54157c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c541577f0 0x7f6c54157c70 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f6c540b4540 tx=0x7f6c4c00e530 comp rx=0 tx=0).stop 2023-12-20T04:46:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 -- 172.21.15.32:0/3685813149 shutdown_connections 2023-12-20T04:46:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c38065bb0 0x7f6c38068070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c541577f0 0x7f6c54157c70 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c540abf40 0x7f6c540ac3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 --2- 172.21.15.32:0/3685813149 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c44003980 0x7f6c541f6130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:06.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 -- 172.21.15.32:0/3685813149 >> 172.21.15.32:0/3685813149 conn(0x7f6c5406a320 msgr2=0x7f6c54154300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:06.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 -- 172.21.15.32:0/3685813149 shutdown_connections 2023-12-20T04:46:06.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:06.887+0000 7f6c5acfc640 1 -- 172.21.15.32:0/3685813149 wait complete. 2023-12-20T04:46:06.904 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T04:46:07.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f84746d1640 1 Processor -- start 2023-12-20T04:46:07.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f84746d1640 1 -- start start 2023-12-20T04:46:07.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f84746d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f846c15cdb0 0x7f846c0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:07.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f84746d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f846c0d2740 0x7f846c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:07.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f84746d1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f846c0d1120 0x7f846c0cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:07.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f84746d1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846c15d190 con 0x7f846c0d1120 2023-12-20T04:46:07.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f84746d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f846c0d16a0 con 0x7f846c15cdb0 2023-12-20T04:46:07.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f84746d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c0d1980 con 0x7f846c0d2740 2023-12-20T04:46:07.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f8472c6e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f846c0d1120 0x7f846c0cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:07.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f8472c6e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f846c0d1120 0x7f846c0cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37990/0 (socket says 172.21.15.32:37990) 2023-12-20T04:46:07.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f8472c6e640 1 -- 172.21.15.32:0/1194379254 learned_addr learned my addr 172.21.15.32:0/1194379254 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:07.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f8471c6c640 1 --2- 172.21.15.32:0/1194379254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f846c0d2740 0x7f846c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:07.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f847246d640 1 --2- 172.21.15.32:0/1194379254 >> v2:172.21.15.39:3300/0 conn(0x7f846c15cdb0 0x7f846c0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:07.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f8472c6e640 1 -- 172.21.15.32:0/1194379254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f846c0d2740 msgr2=0x7f846c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:07.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f8472c6e640 1 --2- 172.21.15.32:0/1194379254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f846c0d2740 0x7f846c0cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f8472c6e640 1 -- 172.21.15.32:0/1194379254 >> v2:172.21.15.39:3300/0 conn(0x7f846c15cdb0 msgr2=0x7f846c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:07.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f8472c6e640 1 --2- 172.21.15.32:0/1194379254 >> v2:172.21.15.39:3300/0 conn(0x7f846c15cdb0 0x7f846c0d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.051+0000 7f8472c6e640 1 -- 172.21.15.32:0/1194379254 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f846c0d1c60 con 0x7f846c0d1120 2023-12-20T04:46:07.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f8472c6e640 1 --2- 172.21.15.32:0/1194379254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f846c0d1120 0x7f846c0cfab0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f846c0667b0 tx=0x7f846401f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da0514261ecaf6c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:07.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f847146b640 1 -- 172.21.15.32:0/1194379254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84640af400 con 0x7f846c0d1120 2023-12-20T04:46:07.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f847146b640 1 -- 172.21.15.32:0/1194379254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f84640ab050 con 0x7f846c0d1120 2023-12-20T04:46:07.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f847146b640 1 -- 172.21.15.32:0/1194379254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84640af780 con 0x7f846c0d1120 2023-12-20T04:46:07.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f84746d1640 1 -- 172.21.15.32:0/1194379254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f846c0d1120 msgr2=0x7f846c0cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:07.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f84746d1640 1 --2- 172.21.15.32:0/1194379254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f846c0d1120 0x7f846c0cfab0 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f846c0667b0 tx=0x7f846401f1c0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f84746d1640 1 -- 172.21.15.32:0/1194379254 shutdown_connections 2023-12-20T04:46:07.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f84746d1640 1 --2- 172.21.15.32:0/1194379254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f846c0d1120 0x7f846c0cfab0 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f84746d1640 1 --2- 172.21.15.32:0/1194379254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f846c0d2740 0x7f846c0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f84746d1640 1 --2- 172.21.15.32:0/1194379254 >> v2:172.21.15.39:3300/0 conn(0x7f846c15cdb0 0x7f846c0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f84746d1640 1 -- 172.21.15.32:0/1194379254 >> 172.21.15.32:0/1194379254 conn(0x7f846c065ee0 msgr2=0x7f846c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:07.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f84746d1640 1 -- 172.21.15.32:0/1194379254 shutdown_connections 2023-12-20T04:46:07.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.055+0000 7f84746d1640 1 -- 172.21.15.32:0/1194379254 wait complete. 2023-12-20T04:46:07.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f84746d1640 1 Processor -- start 2023-12-20T04:46:07.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f84746d1640 1 -- start start 2023-12-20T04:46:07.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f84746d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846c0d1120 0x7f846c0c8540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f84746d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f846c0d2740 0x7f846c159e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f84746d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f846c15cdb0 0x7f846c15a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f84746d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c13ef70 con 0x7f846c15cdb0 2023-12-20T04:46:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f84746d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f846c15f710 con 0x7f846c0d1120 2023-12-20T04:46:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f84746d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f846c15f9f0 con 0x7f846c0d2740 2023-12-20T04:46:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8472c6e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f846c15cdb0 0x7f846c15a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8472c6e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f846c15cdb0 0x7f846c15a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38006/0 (socket says 172.21.15.32:38006) 2023-12-20T04:46:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f847246d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846c0d1120 0x7f846c0c8540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:07.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8472c6e640 1 -- 172.21.15.32:0/470007381 learned_addr learned my addr 172.21.15.32:0/470007381 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8471c6c640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f846c0d2740 0x7f846c159e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8472c6e640 1 -- 172.21.15.32:0/470007381 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f846c0d2740 msgr2=0x7f846c159e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8472c6e640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f846c0d2740 0x7f846c159e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8472c6e640 1 -- 172.21.15.32:0/470007381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846c0d1120 msgr2=0x7f846c0c8540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8472c6e640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846c0d1120 0x7f846c0c8540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8472c6e640 1 -- 172.21.15.32:0/470007381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8464009b30 con 0x7f846c15cdb0 2023-12-20T04:46:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8472c6e640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f846c15cdb0 0x7f846c15a390 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f84640b2040 tx=0x7f846401bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8462ffd640 1 -- 172.21.15.32:0/470007381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84640afde0 con 0x7f846c15cdb0 2023-12-20T04:46:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8462ffd640 1 -- 172.21.15.32:0/470007381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8464009510 con 0x7f846c15cdb0 2023-12-20T04:46:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f8462ffd640 1 -- 172.21.15.32:0/470007381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f84640ba660 con 0x7f846c15cdb0 2023-12-20T04:46:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f84746d1640 1 -- 172.21.15.32:0/470007381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f846c15fcd0 con 0x7f846c15cdb0 2023-12-20T04:46:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.059+0000 7f84746d1640 1 -- 172.21.15.32:0/470007381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f846c0c8a80 con 0x7f846c15cdb0 2023-12-20T04:46:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.063+0000 7f84746d1640 1 -- 172.21.15.32:0/470007381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f846c13f120 con 0x7f846c15cdb0 2023-12-20T04:46:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.063+0000 7f8462ffd640 1 -- 172.21.15.32:0/470007381 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f84640b6040 con 0x7f846c15cdb0 2023-12-20T04:46:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.063+0000 7f8462ffd640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8450065b80 0x7f8450068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:07.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.063+0000 7f8462ffd640 1 -- 172.21.15.32:0/470007381 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8464121600 con 0x7f846c15cdb0 2023-12-20T04:46:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.063+0000 7f847246d640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8450065b80 0x7f8450068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:07.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.063+0000 7f847246d640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8450065b80 0x7f8450068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8468007940 tx=0x7f8468008040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:07.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.067+0000 7f8462ffd640 1 -- 172.21.15.32:0/470007381 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f846c13f120 con 0x7f846c15cdb0 2023-12-20T04:46:07.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.259+0000 7f84746d1640 1 -- 172.21.15.32:0/470007381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f846c13f120 con 0x7f846c15cdb0 2023-12-20T04:46:07.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.259+0000 7f8462ffd640 1 -- 172.21.15.32:0/470007381 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v54) v1 ==== 53+0+73 (secure 0 0 0) 0x7f846c13f120 con 0x7f846c15cdb0 2023-12-20T04:46:07.257 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T04:46:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.259+0000 7f84746d1640 1 -- 172.21.15.32:0/470007381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8450065b80 msgr2=0x7f8450068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.259+0000 7f84746d1640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8450065b80 0x7f8450068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f8468007940 tx=0x7f8468008040 comp rx=0 tx=0).stop 2023-12-20T04:46:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.259+0000 7f84746d1640 1 -- 172.21.15.32:0/470007381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f846c15cdb0 msgr2=0x7f846c15a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.263+0000 7f84746d1640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f846c15cdb0 0x7f846c15a390 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f84640b2040 tx=0x7f846401bf60 comp rx=0 tx=0).stop 2023-12-20T04:46:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.263+0000 7f84746d1640 1 -- 172.21.15.32:0/470007381 shutdown_connections 2023-12-20T04:46:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.263+0000 7f84746d1640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8450065b80 0x7f8450068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.263+0000 7f84746d1640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f846c15cdb0 0x7f846c15a390 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.263+0000 7f84746d1640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f846c0d2740 0x7f846c159e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.263+0000 7f84746d1640 1 --2- 172.21.15.32:0/470007381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f846c0d1120 0x7f846c0c8540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:07.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.263+0000 7f84746d1640 1 -- 172.21.15.32:0/470007381 >> 172.21.15.32:0/470007381 conn(0x7f846c065ee0 msgr2=0x7f846c0b5b50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:07.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.263+0000 7f84746d1640 1 -- 172.21.15.32:0/470007381 shutdown_connections 2023-12-20T04:46:07.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:07.263+0000 7f84746d1640 1 -- 172.21.15.32:0/470007381 wait complete. 2023-12-20T04:46:07.278 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T04:46:07.278 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T04:46:07.278 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T04:46:07.278 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T04:46:07.278 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T04:46:07.279 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T04:46:07.279 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T04:46:07.279 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T04:46:07.279 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:46:07.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:07.279 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:46:07.290 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41293sec preferred_lft 41293sec 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:46:07.291 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:46:07.292 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:07.293 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T04:46:07.350 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T04:46:07.352 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:07.352 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T04:46:07.406 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T04:46:07.407 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T04:46:07.407 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T04:46:07.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T04:46:07.407 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T04:46:07.407 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T04:46:07.407 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:07.407 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:46:07.407 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:46:07.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T04:46:07.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T04:46:07.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T04:46:07.408 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:46:07.504 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:07.504 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T04:46:07.514 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T04:46:07.514 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T04:46:07.515 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:46:07.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:07.515 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T04:46:07.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:46:07.584 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T04:46:07.635 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:46:07.636 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:46:07.636 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:46:07.690 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T04:46:07.691 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:07.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:46:07.790 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:46:07.790 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:46:07.803 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:46:07.803 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:46:07.803 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T04:46:07.803 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T04:46:07.803 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T04:46:07.807 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T04:46:07.808 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:07.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:46:07.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:07.829 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:46:07.893 INFO:teuthology.orchestra.run.smithi032.stdout:client7828 2023-12-20T04:46:07.894 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:46:07.894 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:46:07.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T04:46:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.135+0000 7f8e04bb3640 1 Processor -- start 2023-12-20T04:46:08.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.135+0000 7f8e04bb3640 1 -- start start 2023-12-20T04:46:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8e04bb3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e000aece0 0x7f8e000b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8e04bb3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e000b0b30 0x7f8e000701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8e04bb3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e000c98c0 con 0x7f8e001599e0 2023-12-20T04:46:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8e04bb3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e000c9ba0 con 0x7f8e000aece0 2023-12-20T04:46:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8e04bb3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e000c9e80 con 0x7f8e000b0b30 2023-12-20T04:46:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8df7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e000aece0 0x7f8e000b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8df7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e000aece0 0x7f8e000b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38016/0 (socket says 172.21.15.32:38016) 2023-12-20T04:46:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8df7fff640 1 -- 172.21.15.32:0/2647182487 learned_addr learned my addr 172.21.15.32:0/2647182487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:08.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8dfe59c640 1 --1- 172.21.15.32:0/2647182487 >> v1:172.21.15.32:6789/0 conn(0x7f8e001599e0 0x7f8e000c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36154/0 (socket says 172.21.15.32:36154) 2023-12-20T04:46:08.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8dfed9d640 1 --2- 172.21.15.32:0/2647182487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e000b0b30 0x7f8e000701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8df7fff640 1 -- 172.21.15.32:0/2647182487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e000b0b30 msgr2=0x7f8e000701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:08.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8df7fff640 1 --2- 172.21.15.32:0/2647182487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e000b0b30 0x7f8e000701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8df7fff640 1 -- 172.21.15.32:0/2647182487 >> v1:172.21.15.32:6789/0 conn(0x7f8e001599e0 legacy=0x7f8e000c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:08.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8dfdd9b640 1 -- 172.21.15.32:0/2647182487 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3820789119 0 0) 0x7f8e000c98c0 con 0x7f8e001599e0 2023-12-20T04:46:08.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8df7fff640 1 -- 172.21.15.32:0/2647182487 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e000ca160 con 0x7f8e000aece0 2023-12-20T04:46:08.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.139+0000 7f8df7fff640 1 --2- 172.21.15.32:0/2647182487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e000aece0 0x7f8e000b05f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009870 tx=0x7f8de801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6124a686d5d0a244 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:08.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8dfdd9b640 1 -- 172.21.15.32:0/2647182487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de80ab650 con 0x7f8e000aece0 2023-12-20T04:46:08.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8dfdd9b640 1 -- 172.21.15.32:0/2647182487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8de80ab7b0 con 0x7f8e000aece0 2023-12-20T04:46:08.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8dfdd9b640 1 -- 172.21.15.32:0/2647182487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8de80abb40 con 0x7f8e000aece0 2023-12-20T04:46:08.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 -- 172.21.15.32:0/2647182487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e000aece0 msgr2=0x7f8e000b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:08.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 --2- 172.21.15.32:0/2647182487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e000aece0 0x7f8e000b05f0 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f8de8009870 tx=0x7f8de801c540 comp rx=0 tx=0).stop 2023-12-20T04:46:08.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 -- 172.21.15.32:0/2647182487 shutdown_connections 2023-12-20T04:46:08.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 --2- 172.21.15.32:0/2647182487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e000aece0 0x7f8e000b05f0 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 --2- 172.21.15.32:0/2647182487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e000b0b30 0x7f8e000701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 -- 172.21.15.32:0/2647182487 >> 172.21.15.32:0/2647182487 conn(0x7f8e0006a320 msgr2=0x7f8e000ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:08.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 -- 172.21.15.32:0/2647182487 shutdown_connections 2023-12-20T04:46:08.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 -- 172.21.15.32:0/2647182487 wait complete. 2023-12-20T04:46:08.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 Processor -- start 2023-12-20T04:46:08.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 -- start start 2023-12-20T04:46:08.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e000aece0 0x7f8e000c87a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e000b0b30 0x7f8e0015a140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e001599e0 0x7f8e001f5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e001f52b0 con 0x7f8e000aece0 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0015a680 con 0x7f8e001599e0 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8e04bb3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e0015a950 con 0x7f8e000b0b30 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8dfe59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e000aece0 0x7f8e000c87a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8dfe59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e000aece0 0x7f8e000c87a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38022/0 (socket says 172.21.15.32:38022) 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.143+0000 7f8dfe59c640 1 -- 172.21.15.32:0/3671009708 learned_addr learned my addr 172.21.15.32:0/3671009708 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8df7fff640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e000b0b30 0x7f8e0015a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8dfed9d640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e001599e0 0x7f8e001f5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8dfe59c640 1 -- 172.21.15.32:0/3671009708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e000b0b30 msgr2=0x7f8e0015a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8dfe59c640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e000b0b30 0x7f8e0015a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8dfe59c640 1 -- 172.21.15.32:0/3671009708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e001599e0 msgr2=0x7f8e001f5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:08.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8dfe59c640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e001599e0 0x7f8e001f5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8dfe59c640 1 -- 172.21.15.32:0/3671009708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8de80094d0 con 0x7f8e000aece0 2023-12-20T04:46:08.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8dfed9d640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e001599e0 0x7f8e001f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:46:08.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8dfe59c640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e000aece0 0x7f8e000c87a0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8df000c440 tx=0x7f8df000c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:08.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8df6ffd640 1 -- 172.21.15.32:0/3671009708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df0007480 con 0x7f8e000aece0 2023-12-20T04:46:08.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8df6ffd640 1 -- 172.21.15.32:0/3671009708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8df00075e0 con 0x7f8e000aece0 2023-12-20T04:46:08.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8e04bb3640 1 -- 172.21.15.32:0/3671009708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e0015ac30 con 0x7f8e000aece0 2023-12-20T04:46:08.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8df6ffd640 1 -- 172.21.15.32:0/3671009708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8df00059f0 con 0x7f8e000aece0 2023-12-20T04:46:08.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8e04bb3640 1 -- 172.21.15.32:0/3671009708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e001f54c0 con 0x7f8e000aece0 2023-12-20T04:46:08.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8e04bb3640 1 -- 172.21.15.32:0/3671009708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e0004e000 con 0x7f8e000aece0 2023-12-20T04:46:08.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8df6ffd640 1 -- 172.21.15.32:0/3671009708 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8df0005b50 con 0x7f8e000aece0 2023-12-20T04:46:08.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8df6ffd640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8de0065b80 0x7f8de0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.147+0000 7f8df6ffd640 1 -- 172.21.15.32:0/3671009708 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f8df0082eb0 con 0x7f8e000aece0 2023-12-20T04:46:08.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.151+0000 7f8df7fff640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8de0065b80 0x7f8de0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.151+0000 7f8df7fff640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8de0065b80 0x7f8de0068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8de80af000 tx=0x7f8de80a7000 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:08.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.151+0000 7f8df6ffd640 1 -- 172.21.15.32:0/3671009708 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8df00506c0 con 0x7f8e000aece0 2023-12-20T04:46:08.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.331+0000 7f8e04bb3640 1 -- 172.21.15.32:0/3671009708 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f8e00159dc0 con 0x7f8de0065b80 2023-12-20T04:46:08.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8df6ffd640 1 -- 172.21.15.32:0/3671009708 <== mgr.7465 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f8de80040c0 con 0x7f8de0065b80 2023-12-20T04:46:08.332 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:46:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:46:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T04:46:08.332 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:46:08.332 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:46:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 -- 172.21.15.32:0/3671009708 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8de0065b80 msgr2=0x7f8de0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8de0065b80 0x7f8de0068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f8de80af000 tx=0x7f8de80a7000 comp rx=0 tx=0).stop 2023-12-20T04:46:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 -- 172.21.15.32:0/3671009708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e000aece0 msgr2=0x7f8e000c87a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e000aece0 0x7f8e000c87a0 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f8df000c440 tx=0x7f8df000c900 comp rx=0 tx=0).stop 2023-12-20T04:46:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 -- 172.21.15.32:0/3671009708 shutdown_connections 2023-12-20T04:46:08.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8de0065b80 0x7f8de0068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e001599e0 0x7f8e001f5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e000b0b30 0x7f8e0015a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 --2- 172.21.15.32:0/3671009708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e000aece0 0x7f8e000c87a0 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 -- 172.21.15.32:0/3671009708 >> 172.21.15.32:0/3671009708 conn(0x7f8e0006a320 msgr2=0x7f8e000ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 -- 172.21.15.32:0/3671009708 shutdown_connections 2023-12-20T04:46:08.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.335+0000 7f8e04bb3640 1 -- 172.21.15.32:0/3671009708 wait complete. 2023-12-20T04:46:08.354 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T04:46:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f2909266640 1 Processor -- start 2023-12-20T04:46:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f2909266640 1 -- start start 2023-12-20T04:46:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f2909266640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f29040b0760 0x7f29040b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f2909266640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29040cc4b0 0x7f29040cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f2909266640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29040b1100 con 0x7f2904157480 2023-12-20T04:46:08.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f2909266640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29040c9f50 con 0x7f29040b0760 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f2909266640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29040cde20 con 0x7f29040cc4b0 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f2902ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2904157480 0x7f29040c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36170/0 (socket says 172.21.15.32:36170) 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f2902ffd640 1 -- 172.21.15.32:0/4064230094 learned_addr learned my addr 172.21.15.32:0/4064230094 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f29037fe640 1 --2- 172.21.15.32:0/4064230094 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29040cc4b0 0x7f29040cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f28fbfff640 1 --2- 172.21.15.32:0/4064230094 >> v2:172.21.15.39:3300/0 conn(0x7f29040b0760 0x7f29040b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f29027fc640 1 -- 172.21.15.32:0/4064230094 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3084541632 0 0) 0x7f29040b1100 con 0x7f2904157480 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f29027fc640 1 -- 172.21.15.32:0/4064230094 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28f0003530 con 0x7f2904157480 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f28fbfff640 1 -- 172.21.15.32:0/4064230094 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29040cc4b0 msgr2=0x7f29040cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f28fbfff640 1 --2- 172.21.15.32:0/4064230094 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29040cc4b0 0x7f29040cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f28fbfff640 1 -- 172.21.15.32:0/4064230094 >> v1:172.21.15.32:6789/0 conn(0x7f2904157480 legacy=0x7f29040c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f28fbfff640 1 -- 172.21.15.32:0/4064230094 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29040c8e30 con 0x7f29040b0760 2023-12-20T04:46:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.503+0000 7f29037fe640 1 --2- 172.21.15.32:0/4064230094 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29040cc4b0 0x7f29040cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:46:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f28fbfff640 1 --2- 172.21.15.32:0/4064230094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29040b0760 0x7f29040b0bc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0097c0 tx=0x7f28ec01e5d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=924d53505bfea862 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f29027fc640 1 -- 172.21.15.32:0/4064230094 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0b2600 con 0x7f29040b0760 2023-12-20T04:46:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f29027fc640 1 -- 172.21.15.32:0/4064230094 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28ec0b2760 con 0x7f29040b0760 2023-12-20T04:46:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f29027fc640 1 -- 172.21.15.32:0/4064230094 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28ec0b2af0 con 0x7f29040b0760 2023-12-20T04:46:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f2909266640 1 -- 172.21.15.32:0/4064230094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29040b0760 msgr2=0x7f29040b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f2909266640 1 --2- 172.21.15.32:0/4064230094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29040b0760 0x7f29040b0bc0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0097c0 tx=0x7f28ec01e5d0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f2909266640 1 -- 172.21.15.32:0/4064230094 shutdown_connections 2023-12-20T04:46:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f2909266640 1 --2- 172.21.15.32:0/4064230094 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29040b0760 0x7f29040b0bc0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f2909266640 1 --2- 172.21.15.32:0/4064230094 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29040cc4b0 0x7f29040cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f2909266640 1 -- 172.21.15.32:0/4064230094 >> 172.21.15.32:0/4064230094 conn(0x7f290406a510 msgr2=0x7f29040ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f2909266640 1 -- 172.21.15.32:0/4064230094 shutdown_connections 2023-12-20T04:46:08.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.507+0000 7f2909266640 1 -- 172.21.15.32:0/4064230094 wait complete. 2023-12-20T04:46:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2909266640 1 Processor -- start 2023-12-20T04:46:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2909266640 1 -- start start 2023-12-20T04:46:08.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2909266640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29040b0760 0x7f290415b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2909266640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29040cc4b0 0x7f290414e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2909266640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2904157480 0x7f290415d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2909266640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290414f820 con 0x7f29040b0760 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2909266640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290415bfb0 con 0x7f29040cc4b0 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2909266640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290414eef0 con 0x7f2904157480 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2902ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29040b0760 0x7f290415b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2902ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29040b0760 0x7f290415b780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:38038/0 (socket says 172.21.15.32:38038) 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2902ffd640 1 -- 172.21.15.32:0/2829396363 learned_addr learned my addr 172.21.15.32:0/2829396363 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f29037fe640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2904157480 0x7f290415d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f28fbfff640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29040cc4b0 0x7f290414e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2902ffd640 1 -- 172.21.15.32:0/2829396363 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2904157480 msgr2=0x7f290415d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:08.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2902ffd640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2904157480 0x7f290415d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2902ffd640 1 -- 172.21.15.32:0/2829396363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29040cc4b0 msgr2=0x7f290414e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:08.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2902ffd640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29040cc4b0 0x7f290414e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:08.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2902ffd640 1 -- 172.21.15.32:0/2829396363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc009a70 con 0x7f29040b0760 2023-12-20T04:46:08.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f29037fe640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2904157480 0x7f290415d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:46:08.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.511+0000 7f2902ffd640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29040b0760 0x7f290415b780 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f28f400c4d0 tx=0x7f28f400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.515+0000 7f28fb7fe640 1 -- 172.21.15.32:0/2829396363 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4005cf0 con 0x7f29040b0760 2023-12-20T04:46:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.515+0000 7f28fb7fe640 1 -- 172.21.15.32:0/2829396363 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28f4005e50 con 0x7f29040b0760 2023-12-20T04:46:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.515+0000 7f28fb7fe640 1 -- 172.21.15.32:0/2829396363 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28f4007270 con 0x7f29040b0760 2023-12-20T04:46:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.515+0000 7f2909266640 1 -- 172.21.15.32:0/2829396363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28ec009420 con 0x7f29040b0760 2023-12-20T04:46:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.515+0000 7f2909266640 1 -- 172.21.15.32:0/2829396363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290414f4b0 con 0x7f29040b0760 2023-12-20T04:46:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.515+0000 7f2909266640 1 -- 172.21.15.32:0/2829396363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290404e000 con 0x7f29040b0760 2023-12-20T04:46:08.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.515+0000 7f28fb7fe640 1 -- 172.21.15.32:0/2829396363 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 59) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f28f40073d0 con 0x7f29040b0760 2023-12-20T04:46:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.515+0000 7f28fb7fe640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28e4065b50 0x7f28e4068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.515+0000 7f28fb7fe640 1 -- 172.21.15.32:0/2829396363 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f28f40824d0 con 0x7f29040b0760 2023-12-20T04:46:08.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.515+0000 7f28fbfff640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28e4065b50 0x7f28e4068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:08.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.519+0000 7f28fbfff640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28e4065b50 0x7f28e4068010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0b1000 tx=0x7f28ec0b4040 comp rx=0 tx=0).ready entity=mgr.7465 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:08.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.523+0000 7f28fb7fe640 1 -- 172.21.15.32:0/2829396363 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f28f4053980 con 0x7f29040b0760 2023-12-20T04:46:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:08.719+0000 7f2909266640 1 -- 172.21.15.32:0/2829396363 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f29040c3790 con 0x7f29040b0760 2023-12-20T04:46:09.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.011+0000 7f28fb7fe640 1 -- 172.21.15.32:0/2829396363 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 60) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f28f40809f0 con 0x7f29040b0760 2023-12-20T04:46:09.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.011+0000 7f28fb7fe640 1 -- 172.21.15.32:0/2829396363 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v60) v1 ==== 92+0+0 (secure 0 0 0) 0x7f28f40718f0 con 0x7f29040b0760 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 -- 172.21.15.32:0/2829396363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28e4065b50 msgr2=0x7f28e4068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28e4065b50 0x7f28e4068010 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f28ec0b1000 tx=0x7f28ec0b4040 comp rx=0 tx=0).stop 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 -- 172.21.15.32:0/2829396363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29040b0760 msgr2=0x7f290415b780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29040b0760 0x7f290415b780 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f28f400c4d0 tx=0x7f28f400c990 comp rx=0 tx=0).stop 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 -- 172.21.15.32:0/2829396363 shutdown_connections 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28e4065b50 0x7f28e4068010 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2904157480 0x7f290415d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f29040cc4b0 0x7f290414e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 --2- 172.21.15.32:0/2829396363 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29040b0760 0x7f290415b780 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 -- 172.21.15.32:0/2829396363 >> 172.21.15.32:0/2829396363 conn(0x7f290406a510 msgr2=0x7f29040c3ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 -- 172.21.15.32:0/2829396363 shutdown_connections 2023-12-20T04:46:09.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.015+0000 7f2909266640 1 -- 172.21.15.32:0/2829396363 wait complete. 2023-12-20T04:46:09.034 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T04:46:09.186 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:09.190+0000 7fd8be255180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:46:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.187+0000 7faf00841640 1 Processor -- start 2023-12-20T04:46:09.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.187+0000 7faf00841640 1 -- start start 2023-12-20T04:46:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faf00841640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faefc0d14d0 0x7faefc0b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faf00841640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc0d2650 con 0x7faefc15ce60 2023-12-20T04:46:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faf00841640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faefc0d1a40 con 0x7faefc0d3300 2023-12-20T04:46:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faf00841640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faefc0d1d20 con 0x7faefc0d14d0 2023-12-20T04:46:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faefa59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faefc15ce60 0x7faefc0d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33520/0 (socket says 172.21.15.32:33520) 2023-12-20T04:46:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faefa59c640 1 -- 172.21.15.32:0/3407636101 learned_addr learned my addr 172.21.15.32:0/3407636101 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:46:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faefad9d640 1 --2- 172.21.15.32:0/3407636101 >> v2:172.21.15.39:3300/0 conn(0x7faefc0d14d0 0x7faefc0b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faef9d9b640 1 -- 172.21.15.32:0/3407636101 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865533526 0 0) 0x7faefc0d2650 con 0x7faefc15ce60 2023-12-20T04:46:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faef9d9b640 1 -- 172.21.15.32:0/3407636101 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faee8003530 con 0x7faefc15ce60 2023-12-20T04:46:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faefad9d640 1 -- 172.21.15.32:0/3407636101 >> v1:172.21.15.39:6789/0 conn(0x7faefc0d3300 legacy=0x7faefc0d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faefad9d640 1 -- 172.21.15.32:0/3407636101 >> v1:172.21.15.32:6789/0 conn(0x7faefc15ce60 legacy=0x7faefc0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.191+0000 7faefad9d640 1 -- 172.21.15.32:0/3407636101 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faefc0d2000 con 0x7faefc0d14d0 2023-12-20T04:46:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faefad9d640 1 --2- 172.21.15.32:0/3407636101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc0d14d0 0x7faefc0b6ea0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7faef4009d90 tx=0x7faef401f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=273209546f3d9b6e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:09.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faef9d9b640 1 -- 172.21.15.32:0/3407636101 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef40af600 con 0x7faefc0d14d0 2023-12-20T04:46:09.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faef9d9b640 1 -- 172.21.15.32:0/3407636101 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef40ab050 con 0x7faefc0d14d0 2023-12-20T04:46:09.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faef9d9b640 1 -- 172.21.15.32:0/3407636101 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef40af980 con 0x7faefc0d14d0 2023-12-20T04:46:09.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faf00841640 1 -- 172.21.15.32:0/3407636101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc0d14d0 msgr2=0x7faefc0b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faf00841640 1 --2- 172.21.15.32:0/3407636101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc0d14d0 0x7faefc0b6ea0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7faef4009d90 tx=0x7faef401f130 comp rx=0 tx=0).stop 2023-12-20T04:46:09.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faf00841640 1 -- 172.21.15.32:0/3407636101 shutdown_connections 2023-12-20T04:46:09.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faf00841640 1 --2- 172.21.15.32:0/3407636101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc0d14d0 0x7faefc0b6ea0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faf00841640 1 -- 172.21.15.32:0/3407636101 >> 172.21.15.32:0/3407636101 conn(0x7faefc06b5e0 msgr2=0x7faefc06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:09.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faf00841640 1 -- 172.21.15.32:0/3407636101 shutdown_connections 2023-12-20T04:46:09.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faf00841640 1 -- 172.21.15.32:0/3407636101 wait complete. 2023-12-20T04:46:09.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faf00841640 1 Processor -- start 2023-12-20T04:46:09.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.195+0000 7faf00841640 1 -- start start 2023-12-20T04:46:09.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faf00841640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc0d14d0 0x7faefc1f2ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faf00841640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faefc0d3300 0x7faefc14af20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faf00841640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faefc15ce60 0x7faefc15d5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faf00841640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc1f3410 con 0x7faefc15ce60 2023-12-20T04:46:09.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faf00841640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faefc0ca180 con 0x7faefc0d14d0 2023-12-20T04:46:09.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faf00841640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faefc160260 con 0x7faefc0d3300 2023-12-20T04:46:09.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faefad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faefc15ce60 0x7faefc15d5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faefad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faefc15ce60 0x7faefc15d5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34816/0 (socket says 172.21.15.32:34816) 2023-12-20T04:46:09.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faefad9d640 1 -- 172.21.15.32:0/2809065989 learned_addr learned my addr 172.21.15.32:0/2809065989 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:09.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faefa59c640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc0d14d0 0x7faefc1f2ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faef3fff640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faefc0d3300 0x7faefc14af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faefad9d640 1 -- 172.21.15.32:0/2809065989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faefc0d3300 msgr2=0x7faefc14af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faefad9d640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faefc0d3300 0x7faefc14af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faefad9d640 1 -- 172.21.15.32:0/2809065989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc0d14d0 msgr2=0x7faefc1f2ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faefad9d640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc0d14d0 0x7faefc1f2ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faefad9d640 1 -- 172.21.15.32:0/2809065989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faef4009a70 con 0x7faefc15ce60 2023-12-20T04:46:09.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faefad9d640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faefc15ce60 0x7faefc15d5d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7faef40b2040 tx=0x7faef401bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:09.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faef2ffd640 1 -- 172.21.15.32:0/2809065989 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef4009450 con 0x7faefc15ce60 2023-12-20T04:46:09.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faef2ffd640 1 -- 172.21.15.32:0/2809065989 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faef40b9410 con 0x7faefc15ce60 2023-12-20T04:46:09.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faf00841640 1 -- 172.21.15.32:0/2809065989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faefc0ca450 con 0x7faefc15ce60 2023-12-20T04:46:09.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faef2ffd640 1 -- 172.21.15.32:0/2809065989 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faef40b97c0 con 0x7faefc15ce60 2023-12-20T04:46:09.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faf00841640 1 -- 172.21.15.32:0/2809065989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faefc0caa10 con 0x7faefc15ce60 2023-12-20T04:46:09.199 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:09.199+0000 7f2033969180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:46:09.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.199+0000 7faf00841640 1 -- 172.21.15.32:0/2809065989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faefc0cac00 con 0x7faefc15ce60 2023-12-20T04:46:09.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.203+0000 7faef2ffd640 1 -- 172.21.15.32:0/2809065989 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 60) v1 ==== 85347+0+0 (secure 0 0 0) 0x7faef40b6050 con 0x7faefc15ce60 2023-12-20T04:46:09.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.203+0000 7faef2ffd640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faedc065bd0 0x7faedc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.203+0000 7faef2ffd640 1 -- 172.21.15.32:0/2809065989 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7faef4121940 con 0x7faefc15ce60 2023-12-20T04:46:09.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.203+0000 7faefa59c640 1 -- 172.21.15.32:0/2809065989 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faedc065bd0 msgr2=0x7faedc068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:46:09.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.203+0000 7faefa59c640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faedc065bd0 0x7faedc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:46:09.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.207+0000 7faef2ffd640 1 -- 172.21.15.32:0/2809065989 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faef40ef180 con 0x7faefc15ce60 2023-12-20T04:46:09.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.363+0000 7faf00841640 1 -- 172.21.15.32:0/2809065989 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7faefc0d22b0 con 0x7faefc15ce60 2023-12-20T04:46:09.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faef2ffd640 1 -- 172.21.15.32:0/2809065989 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v41)=0 v41) v1 ==== 138+0+0 (secure 0 0 0) 0x7faef40c90d0 con 0x7faefc15ce60 2023-12-20T04:46:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 -- 172.21.15.32:0/2809065989 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faedc065bd0 msgr2=0x7faedc068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:46:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faedc065bd0 0x7faedc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 -- 172.21.15.32:0/2809065989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faefc15ce60 msgr2=0x7faefc15d5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faefc15ce60 0x7faefc15d5d0 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7faef40b2040 tx=0x7faef401bed0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 -- 172.21.15.32:0/2809065989 shutdown_connections 2023-12-20T04:46:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faedc065bd0 0x7faedc068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faefc15ce60 0x7faefc15d5d0 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faefc0d3300 0x7faefc14af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 --2- 172.21.15.32:0/2809065989 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faefc0d14d0 0x7faefc1f2ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 -- 172.21.15.32:0/2809065989 >> 172.21.15.32:0/2809065989 conn(0x7faefc06b5e0 msgr2=0x7faefc066030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 -- 172.21.15.32:0/2809065989 shutdown_connections 2023-12-20T04:46:09.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.371+0000 7faf00841640 1 -- 172.21.15.32:0/2809065989 wait complete. 2023-12-20T04:46:09.391 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T04:46:09.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f191e4e5640 1 Processor -- start 2023-12-20T04:46:09.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f191e4e5640 1 -- start start 2023-12-20T04:46:09.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f191e4e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f191815ce60 0x7f19180d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f191e4e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19180d3130 0x7f19180d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f191e4e5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f19180d2650 con 0x7f19180d1300 2023-12-20T04:46:09.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f191e4e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19180d35a0 con 0x7f19180d3130 2023-12-20T04:46:09.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f191e4e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f19180d1a40 con 0x7f191815ce60 2023-12-20T04:46:09.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f19177fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19180d3130 0x7f19180d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f191ca82640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f19180d1300 0x7f19180b46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:33522/0 (socket says 172.21.15.32:33522) 2023-12-20T04:46:09.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f191ca82640 1 -- 172.21.15.32:0/3610845572 learned_addr learned my addr 172.21.15.32:0/3610845572 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:46:09.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f1917fff640 1 --2- 172.21.15.32:0/3610845572 >> v2:172.21.15.39:3300/0 conn(0x7f191815ce60 0x7f19180d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f19177fe640 1 -- 172.21.15.32:0/3610845572 >> v2:172.21.15.39:3300/0 conn(0x7f191815ce60 msgr2=0x7f19180d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f19177fe640 1 --2- 172.21.15.32:0/3610845572 >> v2:172.21.15.39:3300/0 conn(0x7f191815ce60 0x7f19180d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f19177fe640 1 -- 172.21.15.32:0/3610845572 >> v1:172.21.15.32:6789/0 conn(0x7f19180d1300 legacy=0x7f19180b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f19177fe640 1 -- 172.21.15.32:0/3610845572 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f19180d1d20 con 0x7f19180d3130 2023-12-20T04:46:09.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.543+0000 7f1916ffd640 1 -- 172.21.15.32:0/3610845572 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 120616090 0 0) 0x7f19180d2650 con 0x7f19180d1300 2023-12-20T04:46:09.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.547+0000 7f19177fe640 1 --2- 172.21.15.32:0/3610845572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180d3130 0x7f19180d0dc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f190c0097c0 tx=0x7f190c01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1253bb2224f6d247 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:09.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.547+0000 7f1916ffd640 1 -- 172.21.15.32:0/3610845572 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c0a8930 con 0x7f19180d3130 2023-12-20T04:46:09.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.547+0000 7f1916ffd640 1 -- 172.21.15.32:0/3610845572 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c0a8a90 con 0x7f19180d3130 2023-12-20T04:46:09.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.547+0000 7f1916ffd640 1 -- 172.21.15.32:0/3610845572 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c0a8e20 con 0x7f19180d3130 2023-12-20T04:46:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 -- 172.21.15.32:0/3610845572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180d3130 msgr2=0x7f19180d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 --2- 172.21.15.32:0/3610845572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180d3130 0x7f19180d0dc0 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x7f190c0097c0 tx=0x7f190c01ec30 comp rx=0 tx=0).stop 2023-12-20T04:46:09.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 -- 172.21.15.32:0/3610845572 shutdown_connections 2023-12-20T04:46:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 --2- 172.21.15.32:0/3610845572 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180d3130 0x7f19180d0dc0 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 --2- 172.21.15.32:0/3610845572 >> v2:172.21.15.39:3300/0 conn(0x7f191815ce60 0x7f19180d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 -- 172.21.15.32:0/3610845572 >> 172.21.15.32:0/3610845572 conn(0x7f191806b5e0 msgr2=0x7f191806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 -- 172.21.15.32:0/3610845572 shutdown_connections 2023-12-20T04:46:09.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 -- 172.21.15.32:0/3610845572 wait complete. 2023-12-20T04:46:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 Processor -- start 2023-12-20T04:46:09.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 -- start start 2023-12-20T04:46:09.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f19180d1300 0x7f191814c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180d3130 0x7f19180c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191815ce60 0x7f19180b3010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19180c7820 con 0x7f19180d3130 2023-12-20T04:46:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19180c6ba0 con 0x7f191815ce60 2023-12-20T04:46:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191e4e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19180c6e80 con 0x7f19180d1300 2023-12-20T04:46:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f19177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180d3130 0x7f19180c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f1917fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f19180d1300 0x7f191814c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f191ca82640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191815ce60 0x7f19180b3010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f19177fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180d3130 0x7f19180c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34842/0 (socket says 172.21.15.32:34842) 2023-12-20T04:46:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f1917fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f19180d1300 0x7f191814c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:34350/0 (socket says 172.21.15.32:34350) 2023-12-20T04:46:09.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.551+0000 7f19177fe640 1 -- 172.21.15.32:0/1204208076 learned_addr learned my addr 172.21.15.32:0/1204208076 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f19177fe640 1 -- 172.21.15.32:0/1204208076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f19180d1300 msgr2=0x7f191814c210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f19177fe640 1 --2- 172.21.15.32:0/1204208076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f19180d1300 0x7f191814c210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f19177fe640 1 -- 172.21.15.32:0/1204208076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191815ce60 msgr2=0x7f19180b3010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f19177fe640 1 --2- 172.21.15.32:0/1204208076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191815ce60 0x7f19180b3010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f19177fe640 1 -- 172.21.15.32:0/1204208076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f190c009420 con 0x7f19180d3130 2023-12-20T04:46:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f19177fe640 1 --2- 172.21.15.32:0/1204208076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180d3130 0x7f19180c6370 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f190c01b840 tx=0x7f190c01d470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f1914ff9640 1 -- 172.21.15.32:0/1204208076 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c0b1e00 con 0x7f19180d3130 2023-12-20T04:46:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f1914ff9640 1 -- 172.21.15.32:0/1204208076 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f190c006e20 con 0x7f19180d3130 2023-12-20T04:46:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f1914ff9640 1 -- 172.21.15.32:0/1204208076 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f190c004230 con 0x7f19180d3130 2023-12-20T04:46:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f191e4e5640 1 -- 172.21.15.32:0/1204208076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f19180c7160 con 0x7f19180d3130 2023-12-20T04:46:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f191e4e5640 1 -- 172.21.15.32:0/1204208076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19180c68b0 con 0x7f19180d3130 2023-12-20T04:46:09.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f191e4e5640 1 -- 172.21.15.32:0/1204208076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f19180c7960 con 0x7f19180d3130 2023-12-20T04:46:09.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.555+0000 7f1914ff9640 1 -- 172.21.15.32:0/1204208076 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 60) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f190c004390 con 0x7f19180d3130 2023-12-20T04:46:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.559+0000 7f1914ff9640 1 --2- 172.21.15.32:0/1204208076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1904065bd0 0x7f1904068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.559+0000 7f1914ff9640 1 -- 172.21.15.32:0/1204208076 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f190c120be0 con 0x7f19180d3130 2023-12-20T04:46:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.559+0000 7f1917fff640 1 -- 172.21.15.32:0/1204208076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1904065bd0 msgr2=0x7f1904068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:46:09.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.559+0000 7f1917fff640 1 --2- 172.21.15.32:0/1204208076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1904065bd0 0x7f1904068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:46:09.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.563+0000 7f1914ff9640 1 -- 172.21.15.32:0/1204208076 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f19180c7960 con 0x7f19180d3130 2023-12-20T04:46:09.622 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:09.626+0000 7fd8be255180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:46:09.651 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:09.651+0000 7f2033969180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:46:09.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.723+0000 7f191e4e5640 1 -- 172.21.15.32:0/1204208076 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f19180c7960 con 0x7f19180d3130 2023-12-20T04:46:09.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f1914ff9640 1 -- 172.21.15.32:0/1204208076 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v42)=0 v42) v1 ==== 133+0+0 (secure 0 0 0) 0x7f19180c7960 con 0x7f19180d3130 2023-12-20T04:46:09.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 -- 172.21.15.32:0/1204208076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1904065bd0 msgr2=0x7f1904068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:46:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 --2- 172.21.15.32:0/1204208076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1904065bd0 0x7f1904068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 -- 172.21.15.32:0/1204208076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180d3130 msgr2=0x7f19180c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 --2- 172.21.15.32:0/1204208076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180d3130 0x7f19180c6370 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f190c01b840 tx=0x7f190c01d470 comp rx=0 tx=0).stop 2023-12-20T04:46:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 -- 172.21.15.32:0/1204208076 shutdown_connections 2023-12-20T04:46:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 --2- 172.21.15.32:0/1204208076 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1904065bd0 0x7f1904068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 --2- 172.21.15.32:0/1204208076 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f191815ce60 0x7f19180b3010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 --2- 172.21.15.32:0/1204208076 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19180d3130 0x7f19180c6370 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 --2- 172.21.15.32:0/1204208076 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f19180d1300 0x7f191814c210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 -- 172.21.15.32:0/1204208076 >> 172.21.15.32:0/1204208076 conn(0x7f191806b5e0 msgr2=0x7f191806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 -- 172.21.15.32:0/1204208076 shutdown_connections 2023-12-20T04:46:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.731+0000 7f191e4e5640 1 -- 172.21.15.32:0/1204208076 wait complete. 2023-12-20T04:46:09.750 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:09.750 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-20T04:46:09.766 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-20T04:46:09.780 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:09.782+0000 7fd8be255180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:46:09.811 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:09.815+0000 7f2033969180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:46:09.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.955+0000 7f104a466640 1 Processor -- start 2023-12-20T04:46:09.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.955+0000 7f104a466640 1 -- start start 2023-12-20T04:46:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f104a466640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f104415ce60 0x7f10440d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f104a466640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f10440d3130 0x7f10440d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f104a466640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10440d1300 0x7f10440b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f104a466640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10440d35a0 con 0x7f104415ce60 2023-12-20T04:46:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f104a466640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10440d2650 con 0x7f10440d3130 2023-12-20T04:46:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f104a466640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10440d1a40 con 0x7f10440d1300 2023-12-20T04:46:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f1043fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f104415ce60 0x7f10440d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f1043fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f104415ce60 0x7f10440d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34846/0 (socket says 172.21.15.32:34846) 2023-12-20T04:46:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f1043fff640 1 -- 172.21.15.32:0/1334392417 learned_addr learned my addr 172.21.15.32:0/1334392417 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f10437fe640 1 --2- 172.21.15.32:0/1334392417 >> v2:172.21.15.39:3300/0 conn(0x7f10440d3130 0x7f10440d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f1048a03640 1 --2- 172.21.15.32:0/1334392417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10440d1300 0x7f10440b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f1043fff640 1 -- 172.21.15.32:0/1334392417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10440d1300 msgr2=0x7f10440b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f1043fff640 1 --2- 172.21.15.32:0/1334392417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10440d1300 0x7f10440b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f1043fff640 1 -- 172.21.15.32:0/1334392417 >> v2:172.21.15.39:3300/0 conn(0x7f10440d3130 msgr2=0x7f10440d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f1043fff640 1 --2- 172.21.15.32:0/1334392417 >> v2:172.21.15.39:3300/0 conn(0x7f10440d3130 0x7f10440d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f1043fff640 1 -- 172.21.15.32:0/1334392417 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10440d1d20 con 0x7f104415ce60 2023-12-20T04:46:09.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.959+0000 7f1048a03640 1 --2- 172.21.15.32:0/1334392417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10440d1300 0x7f10440b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:46:09.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f1043fff640 1 --2- 172.21.15.32:0/1334392417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f104415ce60 0x7f10440d2bf0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1044160a10 tx=0x7f103401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8eea3e47ad96f54e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f1042ffd640 1 -- 172.21.15.32:0/1334392417 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10340a89b0 con 0x7f104415ce60 2023-12-20T04:46:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f1042ffd640 1 -- 172.21.15.32:0/1334392417 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10340a8b10 con 0x7f104415ce60 2023-12-20T04:46:09.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f1042ffd640 1 -- 172.21.15.32:0/1334392417 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10340a8ea0 con 0x7f104415ce60 2023-12-20T04:46:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f104a466640 1 -- 172.21.15.32:0/1334392417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f104415ce60 msgr2=0x7f10440d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f104a466640 1 --2- 172.21.15.32:0/1334392417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f104415ce60 0x7f10440d2bf0 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f1044160a10 tx=0x7f103401ecf0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f104a466640 1 -- 172.21.15.32:0/1334392417 shutdown_connections 2023-12-20T04:46:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f104a466640 1 --2- 172.21.15.32:0/1334392417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10440d1300 0x7f10440b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f104a466640 1 --2- 172.21.15.32:0/1334392417 >> v2:172.21.15.39:3300/0 conn(0x7f10440d3130 0x7f10440d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f104a466640 1 --2- 172.21.15.32:0/1334392417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f104415ce60 0x7f10440d2bf0 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f104a466640 1 -- 172.21.15.32:0/1334392417 >> 172.21.15.32:0/1334392417 conn(0x7f104406b5e0 msgr2=0x7f104406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f104a466640 1 -- 172.21.15.32:0/1334392417 shutdown_connections 2023-12-20T04:46:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f104a466640 1 -- 172.21.15.32:0/1334392417 wait complete. 2023-12-20T04:46:09.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f104a466640 1 Processor -- start 2023-12-20T04:46:09.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.963+0000 7f104a466640 1 -- start start 2023-12-20T04:46:09.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f104a466640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10440d1300 0x7f10440cda50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f104a466640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10440d3130 0x7f10440cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f104a466640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f104415ce60 0x7f10440cb2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f104a466640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10440cb800 con 0x7f10440d1300 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f104a466640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10440cba00 con 0x7f10440d3130 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f104a466640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10440cbce0 con 0x7f104415ce60 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f1043fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10440d1300 0x7f10440cda50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f1043fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10440d1300 0x7f10440cda50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34852/0 (socket says 172.21.15.32:34852) 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f10437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10440d3130 0x7f10440cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f1043fff640 1 -- 172.21.15.32:0/1368653083 learned_addr learned my addr 172.21.15.32:0/1368653083 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f1048a03640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f104415ce60 0x7f10440cb2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f10437fe640 1 -- 172.21.15.32:0/1368653083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f104415ce60 msgr2=0x7f10440cb2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f10437fe640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f104415ce60 0x7f10440cb2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f10437fe640 1 -- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10440d1300 msgr2=0x7f10440cda50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f10437fe640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10440d1300 0x7f10440cda50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:09.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f10437fe640 1 -- 172.21.15.32:0/1368653083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c009a70 con 0x7f10440d3130 2023-12-20T04:46:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f10437fe640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10440d3130 0x7f10440cad80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f103800e870 tx=0x7f103800ed30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:09.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10380095d0 con 0x7f10440d3130 2023-12-20T04:46:09.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103800c940 con 0x7f10440d3130 2023-12-20T04:46:09.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f104a466640 1 -- 172.21.15.32:0/1368653083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10340094d0 con 0x7f10440d3130 2023-12-20T04:46:09.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103800ccb0 con 0x7f10440d3130 2023-12-20T04:46:09.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f104a466640 1 -- 172.21.15.32:0/1368653083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f104415d520 con 0x7f10440d3130 2023-12-20T04:46:09.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f104a466640 1 -- 172.21.15.32:0/1368653083 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f104404e000 con 0x7f10440d3130 2023-12-20T04:46:09.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.967+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 60) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f1038013470 con 0x7f10440d3130 2023-12-20T04:46:09.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.971+0000 7f1040ff9640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:09.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.971+0000 7f1043fff640 1 -- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 msgr2=0x7f1030068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:46:09.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.971+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(123..123 src has 1..123) v4 ==== 5709+0+0 (secure 0 0 0) 0x7f1038083580 con 0x7f10440d3130 2023-12-20T04:46:09.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.971+0000 7f1043fff640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:46:09.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:09.975+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1038050e40 con 0x7f10440d3130 2023-12-20T04:46:10.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:10.155+0000 7f104a466640 1 -- 172.21.15.32:0/1368653083 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f10440d22e0 con 0x7f1030065bd0 2023-12-20T04:46:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:10.175+0000 7f1043fff640 1 -- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 msgr2=0x7f1030068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:46:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:10.175+0000 7f1043fff640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:46:10.286 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:10.290+0000 7fd8be255180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:46:10.328 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:10.331+0000 7f2033969180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:46:10.348 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:10.350+0000 7fd8be255180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:46:10.390 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:10.391+0000 7f2033969180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:46:10.407 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:10.410+0000 7fd8be255180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:46:10.449 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:10.451+0000 7f2033969180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:46:10.498 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:10.502+0000 7fd8be255180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:46:10.541 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:10.543+0000 7f2033969180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:46:10.557 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:10.562+0000 7fd8be255180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:46:10.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:10.579+0000 7f1043fff640 1 -- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 msgr2=0x7f1030068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:46:10.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:10.579+0000 7f1043fff640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T04:46:10.601 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:10.603+0000 7f2033969180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:46:10.645 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:10.650+0000 7fd8be255180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:46:10.689 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:10.691+0000 7f2033969180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:46:10.704 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:10.706+0000 7fd8be255180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:46:10.748 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:10.751+0000 7f2033969180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:46:10.854 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:10.858+0000 7fd8be255180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:46:10.900 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:10.903+0000 7f2033969180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:46:10.916 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:10.918+0000 7fd8be255180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:46:10.962 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:10.963+0000 7f2033969180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:46:11.165 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:46:11.165 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:46:11.165 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:46:11.173 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:11.178+0000 7fd8be255180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:46:11.212 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:46:11.213 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:46:11.213 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:46:11.221 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:11.223+0000 7f2033969180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:46:11.316 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:11.318+0000 7fd8be255180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:46:11.364 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:11.367+0000 7f2033969180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:46:11.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:11.383+0000 7f1043fff640 1 -- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 msgr2=0x7f1030068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:46:11.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:11.383+0000 7f1043fff640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T04:46:11.447 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:11.450+0000 7fd8be255180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:46:11.496 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:11.499+0000 7f2033969180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:46:11.521 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:11.522+0000 7fd8be255180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:46:11.570 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:11.571+0000 7f2033969180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:46:11.662 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:11.666+0000 7fd8be255180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:46:11.713 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:11.715+0000 7f2033969180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:46:12.227 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:12.230+0000 7fd8be255180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:46:12.282 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:12.283+0000 7f2033969180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:46:12.315 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:12.318+0000 7fd8be255180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:46:12.371 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:12.375+0000 7f2033969180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:46:12.471 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:12.474+0000 7fd8be255180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:46:12.528 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:12.531+0000 7f2033969180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:46:12.559 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:12.562+0000 7fd8be255180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:46:12.617 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:12.619+0000 7f2033969180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:46:12.892 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:12.894+0000 7fd8be255180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:46:12.953 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:12.955+0000 7f2033969180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:46:12.954 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:46:12.958+0000 7fd8be255180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:46:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:12.987+0000 7f1043fff640 1 -- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 msgr2=0x7f1030068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:46:12.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:12.987+0000 7f1043fff640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-20T04:46:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:13.019+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mgrmap(e 61) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f1038051210 con 0x7f10440d3130 2023-12-20T04:46:13.025 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:13.027+0000 7f2033969180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:46:13.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:13.039+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 <== mon.1 v2:172.21.15.39:3300/0 8 ==== mgrmap(e 62) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f1038038f70 con 0x7f10440d3130 2023-12-20T04:46:13.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:13.039+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 msgr2=0x7f1030068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:46:13.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:13.039+0000 7f1040ff9640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:13.153 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:13.155+0000 7f1fb68d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:46:13.153 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:13.155+0000 7f1fb68d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:46:13.153 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:13.155+0000 7f1fb68d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:46:13.154 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:13.155+0000 7f1fb68d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:46:13.154 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:13.155+0000 7f1fb68d6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:46:13.154 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:13.155+0000 7f1fb98dc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:46:13.154 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:13.155+0000 7f1fb98dc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:46:13.154 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:13.155+0000 7f1fb98dc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:46:13.154 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:13.155+0000 7f1fb98dc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:46:13.154 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:46:13.155+0000 7f1fb98dc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:46:14.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.043+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 <== mon.1 v2:172.21.15.39:3300/0 9 ==== mgrmap(e 63) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f10380631e0 con 0x7f10440d3130 2023-12-20T04:46:14.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.043+0000 7f1040ff9640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:14.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.043+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f1030033920 con 0x7f1030065bd0 2023-12-20T04:46:14.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.047+0000 7f1043fff640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:14.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.051+0000 7f1043fff640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068010 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f103401ed30 tx=0x7f10340aa070 comp rx=0 tx=0).ready entity=mgr.8064 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:14.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.119+0000 7f1040ff9640 1 -- 172.21.15.32:0/1368653083 <== mgr.8064 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7f1030033920 con 0x7f1030065bd0 2023-12-20T04:46:14.117 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path snap_test_dir1 2023-12-20T04:46:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 -- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 msgr2=0x7f1030068010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068010 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f103401ed30 tx=0x7f10340aa070 comp rx=0 tx=0).stop 2023-12-20T04:46:14.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 -- 172.21.15.32:0/1368653083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10440d3130 msgr2=0x7f10440cad80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10440d3130 0x7f10440cad80 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f103800e870 tx=0x7f103800ed30 comp rx=0 tx=0).stop 2023-12-20T04:46:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 -- 172.21.15.32:0/1368653083 shutdown_connections 2023-12-20T04:46:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1030065bd0 0x7f1030068010 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f104415ce60 0x7f10440cb2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10440d3130 0x7f10440cad80 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 --2- 172.21.15.32:0/1368653083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10440d1300 0x7f10440cda50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 -- 172.21.15.32:0/1368653083 >> 172.21.15.32:0/1368653083 conn(0x7f104406b5e0 msgr2=0x7f10440b35b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 -- 172.21.15.32:0/1368653083 shutdown_connections 2023-12-20T04:46:14.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.123+0000 7f104a466640 1 -- 172.21.15.32:0/1368653083 wait complete. 2023-12-20T04:46:14.140 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-20T04:46:14.140 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-20-04_47_00 in ~46s... 2023-12-20T04:46:14.141 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-12-20T04:46:14.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-20T04:46:14.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.335+0000 7f8db4d11640 1 Processor -- start 2023-12-20T04:46:14.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.335+0000 7f8db4d11640 1 -- start start 2023-12-20T04:46:14.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.335+0000 7f8db4d11640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8db00b0cb0 0x7f8db00caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:14.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.335+0000 7f8db4d11640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8db0070230 0x7f8db0070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:14.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.335+0000 7f8db4d11640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8db0070bc0 con 0x7f8db015abb0 2023-12-20T04:46:14.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8db4d11640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db006e160 con 0x7f8db00b0cb0 2023-12-20T04:46:14.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8db4d11640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8db015b6a0 con 0x7f8db0070230 2023-12-20T04:46:14.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8da7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8db00b0cb0 0x7f8db00caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:14.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8da7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8db00b0cb0 0x7f8db00caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34986/0 (socket says 172.21.15.32:34986) 2023-12-20T04:46:14.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8da7fff640 1 -- 172.21.15.32:0/1922705681 learned_addr learned my addr 172.21.15.32:0/1922705681 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:14.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8daed9d640 1 --2- 172.21.15.32:0/1922705681 >> v2:172.21.15.39:3300/0 conn(0x7f8db0070230 0x7f8db0070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:14.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8da7fff640 1 -- 172.21.15.32:0/1922705681 >> v2:172.21.15.39:3300/0 conn(0x7f8db0070230 msgr2=0x7f8db0070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:46:14.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8da7fff640 1 --2- 172.21.15.32:0/1922705681 >> v2:172.21.15.39:3300/0 conn(0x7f8db0070230 0x7f8db0070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8da7fff640 1 -- 172.21.15.32:0/1922705681 >> v1:172.21.15.39:6789/0 conn(0x7f8db015abb0 legacy=0x7f8db015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:14.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8da7fff640 1 -- 172.21.15.32:0/1922705681 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8db006d8c0 con 0x7f8db00b0cb0 2023-12-20T04:46:14.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8da7fff640 1 --2- 172.21.15.32:0/1922705681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db00b0cb0 0x7f8db00caaf0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8db00b0c40 tx=0x7f8d9801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6bf049bfa93406d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:14.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8dadd9b640 1 -- 172.21.15.32:0/1922705681 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d980aa660 con 0x7f8db00b0cb0 2023-12-20T04:46:14.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8dadd9b640 1 -- 172.21.15.32:0/1922705681 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8d980aa7c0 con 0x7f8db00b0cb0 2023-12-20T04:46:14.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.339+0000 7f8dadd9b640 1 -- 172.21.15.32:0/1922705681 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8d980aab50 con 0x7f8db00b0cb0 2023-12-20T04:46:14.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 -- 172.21.15.32:0/1922705681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db00b0cb0 msgr2=0x7f8db00caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:14.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 --2- 172.21.15.32:0/1922705681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db00b0cb0 0x7f8db00caaf0 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f8db00b0c40 tx=0x7f8d9801c500 comp rx=0 tx=0).stop 2023-12-20T04:46:14.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 -- 172.21.15.32:0/1922705681 shutdown_connections 2023-12-20T04:46:14.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 --2- 172.21.15.32:0/1922705681 >> v2:172.21.15.39:3300/0 conn(0x7f8db0070230 0x7f8db0070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 --2- 172.21.15.32:0/1922705681 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db00b0cb0 0x7f8db00caaf0 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 -- 172.21.15.32:0/1922705681 >> 172.21.15.32:0/1922705681 conn(0x7f8db006a510 msgr2=0x7f8db00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:14.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 -- 172.21.15.32:0/1922705681 shutdown_connections 2023-12-20T04:46:14.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 -- 172.21.15.32:0/1922705681 wait complete. 2023-12-20T04:46:14.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 Processor -- start 2023-12-20T04:46:14.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 -- start start 2023-12-20T04:46:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db0070230 0x7f8db0159680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8db00b0cb0 0x7f8db0157970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db015abb0 0x7f8db00afe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db00b0350 con 0x7f8db015abb0 2023-12-20T04:46:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8db0158050 con 0x7f8db0070230 2023-12-20T04:46:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8db4d11640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8db0158330 con 0x7f8db00b0cb0 2023-12-20T04:46:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8daed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db015abb0 0x7f8db00afe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8daed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db015abb0 0x7f8db00afe10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34992/0 (socket says 172.21.15.32:34992) 2023-12-20T04:46:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8daed9d640 1 -- 172.21.15.32:0/2291397556 learned_addr learned my addr 172.21.15.32:0/2291397556 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:46:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8dae59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db0070230 0x7f8db0159680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:14.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.343+0000 7f8da7fff640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8db00b0cb0 0x7f8db0157970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:14.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8daed9d640 1 -- 172.21.15.32:0/2291397556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8db00b0cb0 msgr2=0x7f8db0157970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:14.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8daed9d640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8db00b0cb0 0x7f8db0157970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8daed9d640 1 -- 172.21.15.32:0/2291397556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db0070230 msgr2=0x7f8db0159680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:14.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8daed9d640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db0070230 0x7f8db0159680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8daed9d640 1 -- 172.21.15.32:0/2291397556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8d980094d0 con 0x7f8db015abb0 2023-12-20T04:46:14.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8dae59c640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db0070230 0x7f8db0159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:46:14.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8daed9d640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db015abb0 0x7f8db00afe10 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8da8007e40 tx=0x7f8da800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:14.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8da6ffd640 1 -- 172.21.15.32:0/2291397556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da8014620 con 0x7f8db015abb0 2023-12-20T04:46:14.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8da6ffd640 1 -- 172.21.15.32:0/2291397556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8da800ee90 con 0x7f8db015abb0 2023-12-20T04:46:14.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8da6ffd640 1 -- 172.21.15.32:0/2291397556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8da80149e0 con 0x7f8db015abb0 2023-12-20T04:46:14.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8db4d11640 1 -- 172.21.15.32:0/2291397556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8db0158610 con 0x7f8db015abb0 2023-12-20T04:46:14.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8db4d11640 1 -- 172.21.15.32:0/2291397556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8db00ccc70 con 0x7f8db015abb0 2023-12-20T04:46:14.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8db4d11640 1 -- 172.21.15.32:0/2291397556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8db004e000 con 0x7f8db015abb0 2023-12-20T04:46:14.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8da6ffd640 1 -- 172.21.15.32:0/2291397556 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 63) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f8da8014b40 con 0x7f8db015abb0 2023-12-20T04:46:14.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8da6ffd640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d90065900 0x7f8d90067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:46:14.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8da6ffd640 1 -- 172.21.15.32:0/2291397556 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f8da80838b0 con 0x7f8db015abb0 2023-12-20T04:46:14.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.347+0000 7f8dae59c640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d90065900 0x7f8d90067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:46:14.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.351+0000 7f8dae59c640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d90065900 0x7f8d90067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8da0007fd0 tx=0x7f8da0009040 comp rx=0 tx=0).ready entity=mgr.8064 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:46:14.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.355+0000 7f8da6ffd640 1 -- 172.21.15.32:0/2291397556 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8da8051430 con 0x7f8db015abb0 2023-12-20T04:46:14.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.531+0000 7f8db4d11640 1 -- 172.21.15.32:0/2291397556 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8db015b330 con 0x7f8d90065900 2023-12-20T04:46:14.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.535+0000 7f8da6ffd640 1 -- 172.21.15.32:0/2291397556 <== mgr.8064 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+66 (secure 0 0 0) 0x7f8da0001840 con 0x7f8d90065900 2023-12-20T04:46:14.534 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:46:14.534 INFO:teuthology.orchestra.run.smithi032.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-12-20T04:46:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 -- 172.21.15.32:0/2291397556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d90065900 msgr2=0x7f8d90067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d90065900 0x7f8d90067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8da0007fd0 tx=0x7f8da0009040 comp rx=0 tx=0).stop 2023-12-20T04:46:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 -- 172.21.15.32:0/2291397556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db015abb0 msgr2=0x7f8db00afe10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:46:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db015abb0 0x7f8db00afe10 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f8da8007e40 tx=0x7f8da800e4b0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 -- 172.21.15.32:0/2291397556 shutdown_connections 2023-12-20T04:46:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8d90065900 0x7f8d90067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8db015abb0 0x7f8db00afe10 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8db00b0cb0 0x7f8db0157970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 --2- 172.21.15.32:0/2291397556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8db0070230 0x7f8db0159680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:46:14.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 -- 172.21.15.32:0/2291397556 >> 172.21.15.32:0/2291397556 conn(0x7f8db006a510 msgr2=0x7f8db0159de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:46:14.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 -- 172.21.15.32:0/2291397556 shutdown_connections 2023-12-20T04:46:14.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:46:14.539+0000 7f8db4d11640 1 -- 172.21.15.32:0/2291397556 wait complete. 2023-12-20T04:46:14.555 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{}]} 2023-12-20T04:46:14.556 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{}]} 2023-12-20T04:46:14.556 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:14.556 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:14.567 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:14.567 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:14.567 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:15.568 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:15.568 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:15.580 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:15.580 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:15.580 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:16.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:16.582 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:16.594 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:16.594 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:16.594 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:17.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:17.595 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:17.607 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:17.607 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:17.608 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:18.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:18.609 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:18.621 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:18.621 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:18.621 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:19.622 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:19.623 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:19.635 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:19.635 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:19.635 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:20.636 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:20.636 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:20.648 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:20.648 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:20.648 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:21.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:21.650 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:21.662 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:21.662 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:21.662 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:22.663 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:22.664 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:22.676 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:22.676 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:22.676 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:23.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:23.677 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:23.689 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:23.689 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:23.689 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:24.690 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:24.690 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:24.702 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:24.702 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:24.702 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:25.703 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:25.703 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:25.716 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:25.716 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:25.716 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:26.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:26.717 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:26.729 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:26.729 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:26.730 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:27.731 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:27.731 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:27.744 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:27.744 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:27.744 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:28.744 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:28.744 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:28.757 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:28.757 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:28.757 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:29.758 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:29.758 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:29.771 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:29.771 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:29.771 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:30.772 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:30.772 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:30.784 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:30.784 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:30.784 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:31.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:46:31.592 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:46:31.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:31.785 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:31.797 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:31.797 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:31.797 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:32.800 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:32.801 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:32.813 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:32.813 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:32.813 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:33.815 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:33.815 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:33.830 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:33.830 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:33.830 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:34.831 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:34.831 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:34.843 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:34.843 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:34.844 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:35.844 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:35.844 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:35.857 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:35.857 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:35.857 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:36.858 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:36.859 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:36.871 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:36.871 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:36.871 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:37.872 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:37.872 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:37.884 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:37.884 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:37.884 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:38.885 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:38.885 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:38.897 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:38.897 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:38.898 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:39.899 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:39.900 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:39.912 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:39.912 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:39.912 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:40.913 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:40.913 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:40.926 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:40.927 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:40.927 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:41.927 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:41.928 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:41.940 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:41.940 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:41.940 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:42.941 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:42.941 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:42.953 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:42.953 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:42.953 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:43.954 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:43.954 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:43.967 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:43.967 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:43.967 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:44.968 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:44.968 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:44.980 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:44.981 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:44.981 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:45.981 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:45.982 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:45.994 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:45.994 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:45.994 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:46.995 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:46.995 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:47.008 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:47.008 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:47.008 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:48.009 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:48.009 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:48.021 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:48.021 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:48.021 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:49.022 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:49.022 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:49.034 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:49.035 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:49.035 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:50.036 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:50.036 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:50.048 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:50.048 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:50.048 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:51.048 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:51.049 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:51.061 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:51.061 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:51.062 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:52.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:52.065 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:52.077 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:52.077 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:52.078 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:53.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:53.079 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:53.091 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:53.092 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:53.092 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:54.092 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:54.092 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:54.105 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:54.105 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:54.105 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:55.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:55.106 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:55.119 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:55.120 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:55.120 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:56.121 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:56.121 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:56.133 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:56.133 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:56.133 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:57.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:57.134 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:57.146 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:57.146 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:57.146 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:58.147 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:58.148 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:58.160 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:58.161 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:58.161 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:46:59.162 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:46:59.162 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:46:59.175 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T04:46:59.175 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T04:46:59.175 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:47:00.128 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:00.130+0000 7f1fa90bb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:00.129 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:00.130+0000 7f1fa90bb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:00.129 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:00.130+0000 7f1fa90bb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:00.129 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:00.130+0000 7f1fa90bb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:00.129 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:00.130+0000 7f1fa90bb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:00.176 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:00.176 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:47:00.188 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_47_00_UTC 2023-12-20T04:47:00.190 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-04_47_00_UTC'} 2023-12-20T04:47:00.190 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-20-04_47_00_UTC'} 2023-12-20T04:47:00.190 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T04:47:00.190 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-20-04_47_00_UTC'] 2023-12-20T04:47:00.190 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703047574.1408436, now = 1703047620.1905324, timo = 46 2023-12-20T04:47:01.191 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-20T04:47:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.346+0000 7fe49677b640 1 Processor -- start 2023-12-20T04:47:01.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.346+0000 7fe49677b640 1 -- start start 2023-12-20T04:47:01.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.346+0000 7fe49677b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe4900b0760 0x7fe4900b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.346+0000 7fe49677b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4900cc4b0 0x7fe4900cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.346+0000 7fe49677b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe4900b1100 con 0x7fe490157480 2023-12-20T04:47:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.346+0000 7fe49677b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe4900c9f50 con 0x7fe4900cc4b0 2023-12-20T04:47:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.346+0000 7fe49677b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe4900cde20 con 0x7fe4900b0760 2023-12-20T04:47:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe494d18640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4900cc4b0 0x7fe4900cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe494d18640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe4900cc4b0 0x7fe4900cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48664/0 (socket says 172.21.15.32:48664) 2023-12-20T04:47:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe494d18640 1 -- 172.21.15.32:0/1105318761 learned_addr learned my addr 172.21.15.32:0/1105318761 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe48ffff640 1 --1- 172.21.15.32:0/1105318761 >> v1:172.21.15.32:6789/0 conn(0x7fe490157480 0x7fe4900c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40708/0 (socket says 172.21.15.32:40708) 2023-12-20T04:47:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe48f7fe640 1 --2- 172.21.15.32:0/1105318761 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe4900b0760 0x7fe4900b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe494d18640 1 -- 172.21.15.32:0/1105318761 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe4900b0760 msgr2=0x7fe4900b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:01.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe494d18640 1 --2- 172.21.15.32:0/1105318761 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe4900b0760 0x7fe4900b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe494d18640 1 -- 172.21.15.32:0/1105318761 >> v1:172.21.15.32:6789/0 conn(0x7fe490157480 legacy=0x7fe4900c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:01.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe494d18640 1 -- 172.21.15.32:0/1105318761 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe4900c8e30 con 0x7fe4900cc4b0 2023-12-20T04:47:01.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe494d18640 1 --2- 172.21.15.32:0/1105318761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900cc4b0 0x7fe4900cc940 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe488009e50 tx=0x7fe48801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bba2890e6929d11 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:01.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe48effd640 1 -- 172.21.15.32:0/1105318761 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4880a5810 con 0x7fe4900cc4b0 2023-12-20T04:47:01.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe48effd640 1 -- 172.21.15.32:0/1105318761 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe4880a5970 con 0x7fe4900cc4b0 2023-12-20T04:47:01.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.350+0000 7fe48effd640 1 -- 172.21.15.32:0/1105318761 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4880a5d00 con 0x7fe4900cc4b0 2023-12-20T04:47:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 -- 172.21.15.32:0/1105318761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900cc4b0 msgr2=0x7fe4900cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 --2- 172.21.15.32:0/1105318761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900cc4b0 0x7fe4900cc940 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7fe488009e50 tx=0x7fe48801c950 comp rx=0 tx=0).stop 2023-12-20T04:47:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 -- 172.21.15.32:0/1105318761 shutdown_connections 2023-12-20T04:47:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 --2- 172.21.15.32:0/1105318761 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900cc4b0 0x7fe4900cc940 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 --2- 172.21.15.32:0/1105318761 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe4900b0760 0x7fe4900b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 -- 172.21.15.32:0/1105318761 >> 172.21.15.32:0/1105318761 conn(0x7fe49006a510 msgr2=0x7fe4900ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 -- 172.21.15.32:0/1105318761 shutdown_connections 2023-12-20T04:47:01.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 -- 172.21.15.32:0/1105318761 wait complete. 2023-12-20T04:47:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 Processor -- start 2023-12-20T04:47:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 -- start start 2023-12-20T04:47:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900b0760 0x7fe4901468e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4900cc4b0 0x7fe490156820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe490157480 0x7fe490157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe490146e20 con 0x7fe4900b0760 2023-12-20T04:47:01.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe490156d60 con 0x7fe4900cc4b0 2023-12-20T04:47:01.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe49677b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe490166930 con 0x7fe490157480 2023-12-20T04:47:01.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe48ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900b0760 0x7fe4901468e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe48ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900b0760 0x7fe4901468e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48668/0 (socket says 172.21.15.32:48668) 2023-12-20T04:47:01.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe48ffff640 1 -- 172.21.15.32:0/2067826167 learned_addr learned my addr 172.21.15.32:0/2067826167 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:01.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe494d18640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe490157480 0x7fe490157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe48f7fe640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4900cc4b0 0x7fe490156820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe48ffff640 1 -- 172.21.15.32:0/2067826167 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe490157480 msgr2=0x7fe490157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe48ffff640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe490157480 0x7fe490157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe48ffff640 1 -- 172.21.15.32:0/2067826167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4900cc4b0 msgr2=0x7fe490156820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe48ffff640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4900cc4b0 0x7fe490156820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe48ffff640 1 -- 172.21.15.32:0/2067826167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe488009b30 con 0x7fe4900b0760 2023-12-20T04:47:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.354+0000 7fe48f7fe640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4900cc4b0 0x7fe490156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.358+0000 7fe48ffff640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900b0760 0x7fe4901468e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c420 tx=0x7fe48000c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.358+0000 7fe48cff9640 1 -- 172.21.15.32:0/2067826167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe4800074e0 con 0x7fe4900b0760 2023-12-20T04:47:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.358+0000 7fe48cff9640 1 -- 172.21.15.32:0/2067826167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe480007640 con 0x7fe4900b0760 2023-12-20T04:47:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.358+0000 7fe48cff9640 1 -- 172.21.15.32:0/2067826167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe480004a30 con 0x7fe4900b0760 2023-12-20T04:47:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.358+0000 7fe49677b640 1 -- 172.21.15.32:0/2067826167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe490166c10 con 0x7fe4900b0760 2023-12-20T04:47:01.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.358+0000 7fe49677b640 1 -- 172.21.15.32:0/2067826167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe4901685c0 con 0x7fe4900b0760 2023-12-20T04:47:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.358+0000 7fe49677b640 1 -- 172.21.15.32:0/2067826167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe490168750 con 0x7fe4900b0760 2023-12-20T04:47:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.358+0000 7fe48cff9640 1 -- 172.21.15.32:0/2067826167 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 66) v1 ==== 85461+0+0 (secure 0 0 0) 0x7fe480004b90 con 0x7fe4900b0760 2023-12-20T04:47:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.358+0000 7fe48cff9640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065cf0 0x7fe47c0681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.358+0000 7fe48cff9640 1 -- 172.21.15.32:0/2067826167 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fe480083180 con 0x7fe4900b0760 2023-12-20T04:47:01.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.358+0000 7fe48f7fe640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065cf0 0x7fe47c0681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.362+0000 7fe48f7fe640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065cf0 0x7fe47c0681b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe490070ae0 tx=0x7fe484008040 comp rx=0 tx=0).ready entity=mgr.8064 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:01.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.362+0000 7fe48cff9640 1 -- 172.21.15.32:0/2067826167 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe480050670 con 0x7fe4900b0760 2023-12-20T04:47:01.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.542+0000 7fe49677b640 1 -- 172.21.15.32:0/2067826167 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fe4900c9360 con 0x7fe47c065cf0 2023-12-20T04:47:01.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe48cff9640 1 -- 172.21.15.32:0/2067826167 <== mgr.8064 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fe48400b200 con 0x7fe47c065cf0 2023-12-20T04:47:01.608 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path snap_test_dir1 2023-12-20T04:47:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 -- 172.21.15.32:0/2067826167 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065cf0 msgr2=0x7fe47c0681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:01.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065cf0 0x7fe47c0681b0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fe490070ae0 tx=0x7fe484008040 comp rx=0 tx=0).stop 2023-12-20T04:47:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 -- 172.21.15.32:0/2067826167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900b0760 msgr2=0x7fe4901468e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900b0760 0x7fe4901468e0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7fe48000c420 tx=0x7fe48000c8e0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 -- 172.21.15.32:0/2067826167 shutdown_connections 2023-12-20T04:47:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe47c065cf0 0x7fe47c0681b0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe490157480 0x7fe490157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe4900cc4b0 0x7fe490156820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 --2- 172.21.15.32:0/2067826167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe4900b0760 0x7fe4901468e0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 -- 172.21.15.32:0/2067826167 >> 172.21.15.32:0/2067826167 conn(0x7fe49006a510 msgr2=0x7fe490164030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 -- 172.21.15.32:0/2067826167 shutdown_connections 2023-12-20T04:47:01.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.610+0000 7fe49677b640 1 -- 172.21.15.32:0/2067826167 wait complete. 2023-12-20T04:47:01.629 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-20T04:47:01.629 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:01.629 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T04:47:01.639 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_47_00_UTC 2023-12-20T04:47:01.640 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_47_00_UTC 2023-12-20T04:47:01.640 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:01.640 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_47_00_UTC) 2023-12-20T04:47:01.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:47:01.694 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:47:01.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:01.701 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-20T04:47:01.789 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T04:47:01.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa96376d640 1 Processor -- start 2023-12-20T04:47:01.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa96376d640 1 -- start start 2023-12-20T04:47:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa96376d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa95c0b0cb0 0x7fa95c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa96376d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 0x7fa95c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa96376d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa95c15b6a0 con 0x7fa95c15abb0 2023-12-20T04:47:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa96376d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa95c06e160 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa96376d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c070c00 con 0x7fa95c070230 2023-12-20T04:47:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa960d08640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa95c0b0cb0 0x7fa95c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa960d08640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa95c0b0cb0 0x7fa95c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48678/0 (socket says 172.21.15.32:48678) 2023-12-20T04:47:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa960d08640 1 -- 172.21.15.32:0/3062020111 learned_addr learned my addr 172.21.15.32:0/3062020111 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa961509640 1 --1- 172.21.15.32:0/3062020111 >> v1:172.21.15.32:6789/0 conn(0x7fa95c15abb0 0x7fa95c15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40718/0 (socket says 172.21.15.32:40718) 2023-12-20T04:47:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa961d0a640 1 --2- 172.21.15.32:0/3062020111 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 0x7fa95c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa960d08640 1 -- 172.21.15.32:0/3062020111 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 msgr2=0x7fa95c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa960d08640 1 --2- 172.21.15.32:0/3062020111 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 0x7fa95c0706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa960d08640 1 -- 172.21.15.32:0/3062020111 >> v1:172.21.15.32:6789/0 conn(0x7fa95c15abb0 legacy=0x7fa95c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa960d08640 1 -- 172.21.15.32:0/3062020111 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa95c06d8c0 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.978+0000 7fa961d0a640 1 --2- 172.21.15.32:0/3062020111 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 0x7fa95c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa960d08640 1 --2- 172.21.15.32:0/3062020111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa95c0b0cb0 0x7fa95c0caaf0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009870 tx=0x7fa94c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15c4da5c86a4cc38 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa953fff640 1 -- 172.21.15.32:0/3062020111 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0a9400 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa953fff640 1 -- 172.21.15.32:0/3062020111 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c0a9560 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa953fff640 1 -- 172.21.15.32:0/3062020111 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0a98f0 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa96376d640 1 -- 172.21.15.32:0/3062020111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa95c0b0cb0 msgr2=0x7fa95c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa96376d640 1 --2- 172.21.15.32:0/3062020111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa95c0b0cb0 0x7fa95c0caaf0 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fa94c009870 tx=0x7fa94c01c540 comp rx=0 tx=0).stop 2023-12-20T04:47:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa96376d640 1 -- 172.21.15.32:0/3062020111 shutdown_connections 2023-12-20T04:47:01.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa96376d640 1 --2- 172.21.15.32:0/3062020111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa95c0b0cb0 0x7fa95c0caaf0 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa96376d640 1 --2- 172.21.15.32:0/3062020111 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 0x7fa95c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa96376d640 1 -- 172.21.15.32:0/3062020111 >> 172.21.15.32:0/3062020111 conn(0x7fa95c06a510 msgr2=0x7fa95c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa96376d640 1 -- 172.21.15.32:0/3062020111 shutdown_connections 2023-12-20T04:47:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.982+0000 7fa96376d640 1 -- 172.21.15.32:0/3062020111 wait complete. 2023-12-20T04:47:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa96376d640 1 Processor -- start 2023-12-20T04:47:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa96376d640 1 -- start start 2023-12-20T04:47:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa96376d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 0x7fa95c138120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa96376d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa95c0b0cb0 0x7fa95c14e1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa96376d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa95c15abb0 0x7fa95c0c4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa96376d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c146290 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa96376d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c146570 con 0x7fa95c15abb0 2023-12-20T04:47:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa96376d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa95c146850 con 0x7fa95c070230 2023-12-20T04:47:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa960d08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa95c0b0cb0 0x7fa95c14e1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa960d08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa95c0b0cb0 0x7fa95c14e1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48680/0 (socket says 172.21.15.32:48680) 2023-12-20T04:47:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa960d08640 1 -- 172.21.15.32:0/441837406 learned_addr learned my addr 172.21.15.32:0/441837406 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa961d0a640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa95c15abb0 0x7fa95c0c4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa961509640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 0x7fa95c138120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa960d08640 1 -- 172.21.15.32:0/441837406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 msgr2=0x7fa95c138120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa960d08640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 0x7fa95c138120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa960d08640 1 -- 172.21.15.32:0/441837406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa95c15abb0 msgr2=0x7fa95c0c4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa960d08640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa95c15abb0 0x7fa95c0c4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa960d08640 1 -- 172.21.15.32:0/441837406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c0094d0 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa961d0a640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa95c15abb0 0x7fa95c0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa961509640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 0x7fa95c138120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa960d08640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa95c0b0cb0 0x7fa95c14e1a0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa94c01a170 tx=0x7fa94c01ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa951ffb640 1 -- 172.21.15.32:0/441837406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0a9400 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa951ffb640 1 -- 172.21.15.32:0/441837406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa94c0b1800 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa951ffb640 1 -- 172.21.15.32:0/441837406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa94c0b1b70 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa96376d640 1 -- 172.21.15.32:0/441837406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa95c146b30 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.986+0000 7fa96376d640 1 -- 172.21.15.32:0/441837406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa95c138660 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.990+0000 7fa96376d640 1 -- 172.21.15.32:0/441837406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa95c04e000 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.990+0000 7fa951ffb640 1 -- 172.21.15.32:0/441837406 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 66) v1 ==== 85461+0+0 (secure 0 0 0) 0x7fa94c0ba3a0 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.990+0000 7fa951ffb640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa940065cf0 0x7fa9400681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:01.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.990+0000 7fa951ffb640 1 -- 172.21.15.32:0/441837406 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fa94c11dc60 con 0x7fa95c0b0cb0 2023-12-20T04:47:01.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.990+0000 7fa961509640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa940065cf0 0x7fa9400681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:01.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.990+0000 7fa961509640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa940065cf0 0x7fa9400681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa958007fd0 tx=0x7fa958009040 comp rx=0 tx=0).ready entity=mgr.8064 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:01.994+0000 7fa951ffb640 1 -- 172.21.15.32:0/441837406 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa94c0eb160 con 0x7fa95c0b0cb0 2023-12-20T04:47:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:02.194+0000 7fa96376d640 1 -- 172.21.15.32:0/441837406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fa95c146fe0 con 0x7fa95c0b0cb0 2023-12-20T04:47:03.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.150+0000 7fa951ffb640 1 -- 172.21.15.32:0/441837406 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 67) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fa94c0eb530 con 0x7fa95c0b0cb0 2023-12-20T04:47:03.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.150+0000 7fa951ffb640 1 -- 172.21.15.32:0/441837406 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v67) v1 ==== 93+0+0 (secure 0 0 0) 0x7fa94c0eebf0 con 0x7fa95c0b0cb0 2023-12-20T04:47:03.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 -- 172.21.15.32:0/441837406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa940065cf0 msgr2=0x7fa9400681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:03.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa940065cf0 0x7fa9400681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fa958007fd0 tx=0x7fa958009040 comp rx=0 tx=0).stop 2023-12-20T04:47:03.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 -- 172.21.15.32:0/441837406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa95c0b0cb0 msgr2=0x7fa95c14e1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:03.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa95c0b0cb0 0x7fa95c14e1a0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fa94c01a170 tx=0x7fa94c01ad80 comp rx=0 tx=0).stop 2023-12-20T04:47:03.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 -- 172.21.15.32:0/441837406 shutdown_connections 2023-12-20T04:47:03.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa940065cf0 0x7fa9400681b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa95c15abb0 0x7fa95c0c4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa95c0b0cb0 0x7fa95c14e1a0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 --2- 172.21.15.32:0/441837406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa95c070230 0x7fa95c138120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 -- 172.21.15.32:0/441837406 >> 172.21.15.32:0/441837406 conn(0x7fa95c06a510 msgr2=0x7fa95c0b41a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 -- 172.21.15.32:0/441837406 shutdown_connections 2023-12-20T04:47:03.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.154+0000 7fa96376d640 1 -- 172.21.15.32:0/441837406 wait complete. 2023-12-20T04:47:03.177 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:47:03.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:47:03.273 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:47:03.330 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:03.333+0000 7f92b5f6e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:47:03.336 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:03.338+0000 7f4f30e0a180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:47:03.363 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:47:03.363 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:47:03.363 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:47:03.373 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:47:03.373 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:47:03.424 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:47:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f964042d640 1 Processor -- start 2023-12-20T04:47:03.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f964042d640 1 -- start start 2023-12-20T04:47:03.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f964042d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f963806ce60 0x7f963815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f964042d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f963815f110 0x7f963815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f964042d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f963815bbd0 con 0x7f96380af5e0 2023-12-20T04:47:03.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f964042d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963815beb0 con 0x7f963815f110 2023-12-20T04:47:03.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f964042d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f963815e910 con 0x7f963806ce60 2023-12-20T04:47:03.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f963e9ca640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f963815f110 0x7f963815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:03.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f963e9ca640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f963815f110 0x7f963815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48698/0 (socket says 172.21.15.32:48698) 2023-12-20T04:47:03.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f963e9ca640 1 -- 172.21.15.32:0/4050183445 learned_addr learned my addr 172.21.15.32:0/4050183445 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f963d9c8640 1 --2- 172.21.15.32:0/4050183445 >> v2:172.21.15.39:3300/0 conn(0x7f963806ce60 0x7f963815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f963e9ca640 1 -- 172.21.15.32:0/4050183445 >> v2:172.21.15.39:3300/0 conn(0x7f963806ce60 msgr2=0x7f963815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f963e9ca640 1 --2- 172.21.15.32:0/4050183445 >> v2:172.21.15.39:3300/0 conn(0x7f963806ce60 0x7f963815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f963e9ca640 1 -- 172.21.15.32:0/4050183445 >> v1:172.21.15.32:6789/0 conn(0x7f96380af5e0 legacy=0x7f96380af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:03.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.614+0000 7f963e9ca640 1 -- 172.21.15.32:0/4050183445 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f963815ebf0 con 0x7f963815f110 2023-12-20T04:47:03.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f963e9ca640 1 --2- 172.21.15.32:0/4050183445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963815f110 0x7f963815c440 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f963806abe0 tx=0x7f963001c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57a84c5575b731ba server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:03.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f963d1c7640 1 -- 172.21.15.32:0/4050183445 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f963815b9d0 con 0x7f963815f110 2023-12-20T04:47:03.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f963d1c7640 1 -- 172.21.15.32:0/4050183445 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96300a57a0 con 0x7f963815f110 2023-12-20T04:47:03.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f963d1c7640 1 -- 172.21.15.32:0/4050183445 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96300a5b30 con 0x7f963815f110 2023-12-20T04:47:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f964042d640 1 -- 172.21.15.32:0/4050183445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963815f110 msgr2=0x7f963815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f964042d640 1 --2- 172.21.15.32:0/4050183445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963815f110 0x7f963815c440 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f963806abe0 tx=0x7f963001c940 comp rx=0 tx=0).stop 2023-12-20T04:47:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f964042d640 1 -- 172.21.15.32:0/4050183445 shutdown_connections 2023-12-20T04:47:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f964042d640 1 --2- 172.21.15.32:0/4050183445 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963815f110 0x7f963815c440 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f964042d640 1 --2- 172.21.15.32:0/4050183445 >> v2:172.21.15.39:3300/0 conn(0x7f963806ce60 0x7f963815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f964042d640 1 -- 172.21.15.32:0/4050183445 >> 172.21.15.32:0/4050183445 conn(0x7f963806a320 msgr2=0x7f96380ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f964042d640 1 -- 172.21.15.32:0/4050183445 shutdown_connections 2023-12-20T04:47:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.618+0000 7f964042d640 1 -- 172.21.15.32:0/4050183445 wait complete. 2023-12-20T04:47:03.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f964042d640 1 Processor -- start 2023-12-20T04:47:03.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f964042d640 1 -- start start 2023-12-20T04:47:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f964042d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963806ce60 0x7f9638146a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f964042d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96380af5e0 0x7f96381554a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f964042d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f963815f110 0x7f96381559e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f964042d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963815a380 con 0x7f963806ce60 2023-12-20T04:47:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f964042d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f963815a660 con 0x7f963815f110 2023-12-20T04:47:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f964042d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f963815a940 con 0x7f96380af5e0 2023-12-20T04:47:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f963e1c9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963806ce60 0x7f9638146a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f963e1c9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963806ce60 0x7f9638146a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48700/0 (socket says 172.21.15.32:48700) 2023-12-20T04:47:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f963e1c9640 1 -- 172.21.15.32:0/1877079962 learned_addr learned my addr 172.21.15.32:0/1877079962 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f963e9ca640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f963815f110 0x7f96381559e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f963d9c8640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96380af5e0 0x7f96381554a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f963e1c9640 1 -- 172.21.15.32:0/1877079962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96380af5e0 msgr2=0x7f96381554a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f963e1c9640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96380af5e0 0x7f96381554a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f963e1c9640 1 -- 172.21.15.32:0/1877079962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f963815f110 msgr2=0x7f96381559e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f963e1c9640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f963815f110 0x7f96381559e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f963e1c9640 1 -- 172.21.15.32:0/1877079962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9630009b30 con 0x7f963806ce60 2023-12-20T04:47:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f963e1c9640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963806ce60 0x7f9638146a20 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f963400c450 tx=0x7f963400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f962effd640 1 -- 172.21.15.32:0/1877079962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9634005480 con 0x7f963806ce60 2023-12-20T04:47:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f962effd640 1 -- 172.21.15.32:0/1877079962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9634003600 con 0x7f963806ce60 2023-12-20T04:47:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f964042d640 1 -- 172.21.15.32:0/1877079962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f963815ac20 con 0x7f963806ce60 2023-12-20T04:47:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f962effd640 1 -- 172.21.15.32:0/1877079962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96340039b0 con 0x7f963806ce60 2023-12-20T04:47:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f964042d640 1 -- 172.21.15.32:0/1877079962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9638159630 con 0x7f963806ce60 2023-12-20T04:47:03.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.622+0000 7f964042d640 1 -- 172.21.15.32:0/1877079962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f963804e000 con 0x7f963806ce60 2023-12-20T04:47:03.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.626+0000 7f962effd640 1 -- 172.21.15.32:0/1877079962 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 67) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f9634003b10 con 0x7f963806ce60 2023-12-20T04:47:03.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.626+0000 7f962effd640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f961c065ca0 0x7f961c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.626+0000 7f962effd640 1 -- 172.21.15.32:0/1877079962 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f9634082ef0 con 0x7f963806ce60 2023-12-20T04:47:03.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.626+0000 7f963d9c8640 1 -- 172.21.15.32:0/1877079962 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f961c065ca0 msgr2=0x7f961c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:03.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.626+0000 7f963d9c8640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f961c065ca0 0x7f961c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:03.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.630+0000 7f962effd640 1 -- 172.21.15.32:0/1877079962 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9634050610 con 0x7f963806ce60 2023-12-20T04:47:03.765 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:03.769+0000 7f92b5f6e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:47:03.783 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:03.786+0000 7f4f30e0a180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:47:03.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.790+0000 7f964042d640 1 -- 172.21.15.32:0/1877079962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f96380cf450 con 0x7f963806ce60 2023-12-20T04:47:03.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.790+0000 7f962effd640 1 -- 172.21.15.32:0/1877079962 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16492 (secure 0 0 0) 0x7f9634053c30 con 0x7f963806ce60 2023-12-20T04:47:03.790 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:03.790 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":124,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:46:13.032320+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:45:59.452651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-20T04:45:59.882509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1806203372":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/878030299":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/3355189708":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/224603579":"2023-12-21T04:46:13.032274+0000","172.21.15.32:6833/38009":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/1563603368":"2023-12-21T04:46:13.032274+0000","172.21.15.32:6832/38009":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/1523085934":"2023-12-21T04:46:13.032274+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:47:03.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 -- 172.21.15.32:0/1877079962 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f961c065ca0 msgr2=0x7f961c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:03.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f961c065ca0 0x7f961c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 -- 172.21.15.32:0/1877079962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963806ce60 msgr2=0x7f9638146a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:03.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963806ce60 0x7f9638146a20 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f963400c450 tx=0x7f963400c910 comp rx=0 tx=0).stop 2023-12-20T04:47:03.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 -- 172.21.15.32:0/1877079962 shutdown_connections 2023-12-20T04:47:03.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f961c065ca0 0x7f961c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f963815f110 0x7f96381559e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96380af5e0 0x7f96381554a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 --2- 172.21.15.32:0/1877079962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f963806ce60 0x7f9638146a20 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 -- 172.21.15.32:0/1877079962 >> 172.21.15.32:0/1877079962 conn(0x7f963806a320 msgr2=0x7f9638157810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:03.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 -- 172.21.15.32:0/1877079962 shutdown_connections 2023-12-20T04:47:03.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.794+0000 7f964042d640 1 -- 172.21.15.32:0/1877079962 wait complete. 2023-12-20T04:47:03.812 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:45:59.452651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:45:59.882509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:47:03.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:47:03.923 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:03.925+0000 7f92b5f6e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:47:03.946 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:03.946+0000 7f4f30e0a180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:47:03.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.962+0000 7f26eb2f2640 1 Processor -- start 2023-12-20T04:47:03.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.962+0000 7f26eb2f2640 1 -- start start 2023-12-20T04:47:03.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.962+0000 7f26eb2f2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f26e415ce60 0x7f26e40d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.962+0000 7f26eb2f2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26e40d14d0 0x7f26e40b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.962+0000 7f26eb2f2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f26e40d2650 con 0x7f26e40d3130 2023-12-20T04:47:03.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.962+0000 7f26eb2f2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e40d35a0 con 0x7f26e40d14d0 2023-12-20T04:47:03.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.962+0000 7f26eb2f2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f26e40d1a40 con 0x7f26e415ce60 2023-12-20T04:47:03.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.962+0000 7f26e988f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26e40d14d0 0x7f26e40b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:03.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.962+0000 7f26e988f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f26e40d14d0 0x7f26e40b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48714/0 (socket says 172.21.15.32:48714) 2023-12-20T04:47:03.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.962+0000 7f26e988f640 1 -- 172.21.15.32:0/4071596570 learned_addr learned my addr 172.21.15.32:0/4071596570 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:03.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.966+0000 7f26e908e640 1 --2- 172.21.15.32:0/4071596570 >> v2:172.21.15.39:3300/0 conn(0x7f26e415ce60 0x7f26e40d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:03.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.966+0000 7f26e988f640 1 -- 172.21.15.32:0/4071596570 >> v2:172.21.15.39:3300/0 conn(0x7f26e415ce60 msgr2=0x7f26e40d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:03.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.966+0000 7f26e988f640 1 --2- 172.21.15.32:0/4071596570 >> v2:172.21.15.39:3300/0 conn(0x7f26e415ce60 0x7f26e40d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.966+0000 7f26e988f640 1 -- 172.21.15.32:0/4071596570 >> v1:172.21.15.39:6789/0 conn(0x7f26e40d3130 legacy=0x7f26e40d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:03.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.966+0000 7f26dbfff640 1 -- 172.21.15.32:0/4071596570 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3686794291 0 0) 0x7f26e40d2650 con 0x7f26e40d3130 2023-12-20T04:47:03.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.966+0000 7f26e988f640 1 -- 172.21.15.32:0/4071596570 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26e40d1d20 con 0x7f26e40d14d0 2023-12-20T04:47:03.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26e988f640 1 --2- 172.21.15.32:0/4071596570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26e40d14d0 0x7f26e40b5e90 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009e50 tx=0x7f26dc01f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d678fecfe984e82 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:03.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26dbfff640 1 -- 172.21.15.32:0/4071596570 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0af400 con 0x7f26e40d14d0 2023-12-20T04:47:03.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26dbfff640 1 -- 172.21.15.32:0/4071596570 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26dc0ab050 con 0x7f26e40d14d0 2023-12-20T04:47:03.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26dbfff640 1 -- 172.21.15.32:0/4071596570 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26dc0af780 con 0x7f26e40d14d0 2023-12-20T04:47:03.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26eb2f2640 1 -- 172.21.15.32:0/4071596570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26e40d14d0 msgr2=0x7f26e40b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:03.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26eb2f2640 1 --2- 172.21.15.32:0/4071596570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26e40d14d0 0x7f26e40b5e90 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f26dc009e50 tx=0x7f26dc01f1c0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26eb2f2640 1 -- 172.21.15.32:0/4071596570 shutdown_connections 2023-12-20T04:47:03.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26eb2f2640 1 --2- 172.21.15.32:0/4071596570 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26e40d14d0 0x7f26e40b5e90 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26eb2f2640 1 --2- 172.21.15.32:0/4071596570 >> v2:172.21.15.39:3300/0 conn(0x7f26e415ce60 0x7f26e40d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26eb2f2640 1 -- 172.21.15.32:0/4071596570 >> 172.21.15.32:0/4071596570 conn(0x7f26e406b5e0 msgr2=0x7f26e406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:03.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26eb2f2640 1 -- 172.21.15.32:0/4071596570 shutdown_connections 2023-12-20T04:47:03.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.970+0000 7f26eb2f2640 1 -- 172.21.15.32:0/4071596570 wait complete. 2023-12-20T04:47:03.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26eb2f2640 1 Processor -- start 2023-12-20T04:47:03.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26eb2f2640 1 -- start start 2023-12-20T04:47:03.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26eb2f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f26e40d14d0 0x7f26e414b390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26eb2f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26e40d3130 0x7f26e414d150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26eb2f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26e415ce60 0x7f26e414f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26eb2f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e414b8d0 con 0x7f26e415ce60 2023-12-20T04:47:03.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26eb2f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f26e414ccc0 con 0x7f26e40d3130 2023-12-20T04:47:03.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26eb2f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f26e4151ea0 con 0x7f26e40d14d0 2023-12-20T04:47:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e888d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26e40d3130 0x7f26e414d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e988f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26e415ce60 0x7f26e414f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e908e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f26e40d14d0 0x7f26e414b390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e908e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f26e40d14d0 0x7f26e414b390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:40144/0 (socket says 172.21.15.32:40144) 2023-12-20T04:47:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e908e640 1 -- 172.21.15.32:0/2267266628 learned_addr learned my addr 172.21.15.32:0/2267266628 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e908e640 1 -- 172.21.15.32:0/2267266628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26e40d3130 msgr2=0x7f26e414d150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e908e640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26e40d3130 0x7f26e414d150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e908e640 1 -- 172.21.15.32:0/2267266628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26e415ce60 msgr2=0x7f26e414f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e908e640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26e415ce60 0x7f26e414f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e908e640 1 -- 172.21.15.32:0/2267266628 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f26dc009b30 con 0x7f26e40d14d0 2023-12-20T04:47:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e988f640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26e415ce60 0x7f26e414f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26e908e640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f26e40d14d0 0x7f26e414b390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f26e000e870 tx=0x7f26e000ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26d9ffb640 1 -- 172.21.15.32:0/2267266628 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e0009510 con 0x7f26e40d14d0 2023-12-20T04:47:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26d9ffb640 1 -- 172.21.15.32:0/2267266628 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f26e000eec0 con 0x7f26e40d14d0 2023-12-20T04:47:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26d9ffb640 1 -- 172.21.15.32:0/2267266628 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f26e000cb40 con 0x7f26e40d14d0 2023-12-20T04:47:03.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26eb2f2640 1 -- 172.21.15.32:0/2267266628 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f26e414c0b0 con 0x7f26e40d14d0 2023-12-20T04:47:03.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.974+0000 7f26eb2f2640 1 -- 172.21.15.32:0/2267266628 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f26e414c670 con 0x7f26e40d14d0 2023-12-20T04:47:03.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.978+0000 7f26eb2f2640 1 -- 172.21.15.32:0/2267266628 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f26e404e000 con 0x7f26e40d14d0 2023-12-20T04:47:03.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.978+0000 7f26d9ffb640 1 -- 172.21.15.32:0/2267266628 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 67) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f26e000cca0 con 0x7f26e40d14d0 2023-12-20T04:47:03.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.978+0000 7f26d9ffb640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26c8065cd0 0x7f26c8068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:03.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.978+0000 7f26d9ffb640 1 -- 172.21.15.32:0/2267266628 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f26e0083710 con 0x7f26e40d14d0 2023-12-20T04:47:03.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.978+0000 7f26e888d640 1 -- 172.21.15.32:0/2267266628 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26c8065cd0 msgr2=0x7f26c8068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.978+0000 7f26e888d640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26c8065cd0 0x7f26c8068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:03.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:03.986+0000 7f26d9ffb640 1 -- 172.21.15.32:0/2267266628 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f26e0050cc0 con 0x7f26e40d14d0 2023-12-20T04:47:04.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.146+0000 7f26eb2f2640 1 -- 172.21.15.32:0/2267266628 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f26e40d22b0 con 0x7f26e40d14d0 2023-12-20T04:47:04.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.150+0000 7f26d9ffb640 1 -- 172.21.15.32:0/2267266628 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7f26e00542e0 con 0x7f26e40d14d0 2023-12-20T04:47:04.151 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:47:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 -- 172.21.15.32:0/2267266628 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26c8065cd0 msgr2=0x7f26c8068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26c8065cd0 0x7f26c8068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 -- 172.21.15.32:0/2267266628 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f26e40d14d0 msgr2=0x7f26e414b390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f26e40d14d0 0x7f26e414b390 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f26e000e870 tx=0x7f26e000ec70 comp rx=0 tx=0).stop 2023-12-20T04:47:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 -- 172.21.15.32:0/2267266628 shutdown_connections 2023-12-20T04:47:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f26c8065cd0 0x7f26c8068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f26e415ce60 0x7f26e414f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f26e40d3130 0x7f26e414d150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 --2- 172.21.15.32:0/2267266628 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f26e40d14d0 0x7f26e414b390 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 -- 172.21.15.32:0/2267266628 >> 172.21.15.32:0/2267266628 conn(0x7f26e406b5e0 msgr2=0x7f26e40b4240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:04.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 -- 172.21.15.32:0/2267266628 shutdown_connections 2023-12-20T04:47:04.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.154+0000 7f26eb2f2640 1 -- 172.21.15.32:0/2267266628 wait complete. 2023-12-20T04:47:04.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:47:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.322+0000 7fee843bc640 1 Processor -- start 2023-12-20T04:47:04.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.322+0000 7fee843bc640 1 -- start start 2023-12-20T04:47:04.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.322+0000 7fee843bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee7c15abb0 0x7fee7c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:04.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.322+0000 7fee843bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fee7c070230 0x7fee7c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:04.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.322+0000 7fee843bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee7c15b6a0 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.322+0000 7fee843bc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee7c06e160 con 0x7fee7c070230 2023-12-20T04:47:04.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.322+0000 7fee843bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c070be0 con 0x7fee7c15abb0 2023-12-20T04:47:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.322+0000 7fee82158640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee7c15abb0 0x7fee7c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.322+0000 7fee82959640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fee7c070230 0x7fee7c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.322+0000 7fee81957640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fee7c0b0cb0 0x7fee7c15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:60640/0 (socket says 172.21.15.32:60640) 2023-12-20T04:47:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.322+0000 7fee81957640 1 -- 172.21.15.32:0/3616567520 learned_addr learned my addr 172.21.15.32:0/3616567520 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:04.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.326+0000 7fee82959640 1 -- 172.21.15.32:0/3616567520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee7c15abb0 msgr2=0x7fee7c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.326+0000 7fee82959640 1 --2- 172.21.15.32:0/3616567520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee7c15abb0 0x7fee7c0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.326+0000 7fee82959640 1 -- 172.21.15.32:0/3616567520 >> v1:172.21.15.39:6789/0 conn(0x7fee7c0b0cb0 legacy=0x7fee7c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.326+0000 7fee82959640 1 -- 172.21.15.32:0/3616567520 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7c06d8c0 con 0x7fee7c070230 2023-12-20T04:47:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.326+0000 7fee81156640 1 -- 172.21.15.32:0/3616567520 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3919792982 0 0) 0x7fee7c15b6a0 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.326+0000 7fee82959640 1 --2- 172.21.15.32:0/3616567520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee7c070230 0x7fee7c0706a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fee74009d90 tx=0x7fee7401c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7dd66e3f1a933b2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.326+0000 7fee81156640 1 -- 172.21.15.32:0/3616567520 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee740a5a10 con 0x7fee7c070230 2023-12-20T04:47:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.326+0000 7fee81156640 1 -- 172.21.15.32:0/3616567520 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee740a5b70 con 0x7fee7c070230 2023-12-20T04:47:04.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.326+0000 7fee81156640 1 -- 172.21.15.32:0/3616567520 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee740ad440 con 0x7fee7c070230 2023-12-20T04:47:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 -- 172.21.15.32:0/3616567520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee7c070230 msgr2=0x7fee7c0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 --2- 172.21.15.32:0/3616567520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee7c070230 0x7fee7c0706a0 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7fee74009d90 tx=0x7fee7401c950 comp rx=0 tx=0).stop 2023-12-20T04:47:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 -- 172.21.15.32:0/3616567520 shutdown_connections 2023-12-20T04:47:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 --2- 172.21.15.32:0/3616567520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee7c070230 0x7fee7c0706a0 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 --2- 172.21.15.32:0/3616567520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee7c15abb0 0x7fee7c0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 -- 172.21.15.32:0/3616567520 >> 172.21.15.32:0/3616567520 conn(0x7fee7c06a510 msgr2=0x7fee7c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:04.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 -- 172.21.15.32:0/3616567520 shutdown_connections 2023-12-20T04:47:04.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 -- 172.21.15.32:0/3616567520 wait complete. 2023-12-20T04:47:04.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 Processor -- start 2023-12-20T04:47:04.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 -- start start 2023-12-20T04:47:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee7c070230 0x7fee7c148ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c0b0cb0 0x7fee7c149280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee7c15abb0 0x7fee7c137ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c150fc0 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c1512a0 con 0x7fee7c070230 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee843bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee7c151580 con 0x7fee7c15abb0 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee81957640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c0b0cb0 0x7fee7c149280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee82158640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee7c070230 0x7fee7c148ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee82959640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee7c15abb0 0x7fee7c137ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee81957640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c0b0cb0 0x7fee7c149280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48728/0 (socket says 172.21.15.32:48728) 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.330+0000 7fee81957640 1 -- 172.21.15.32:0/3460063811 learned_addr learned my addr 172.21.15.32:0/3460063811 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee81957640 1 -- 172.21.15.32:0/3460063811 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee7c15abb0 msgr2=0x7fee7c137ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee81957640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee7c15abb0 0x7fee7c137ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee81957640 1 -- 172.21.15.32:0/3460063811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee7c070230 msgr2=0x7fee7c148ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee81957640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee7c070230 0x7fee7c148ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee81957640 1 -- 172.21.15.32:0/3460063811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee74009a70 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee82158640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee7c070230 0x7fee7c148ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee81957640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c0b0cb0 0x7fee7c149280 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0011f0 tx=0x7fee6c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee72ffd640 1 -- 172.21.15.32:0/3460063811 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c0074e0 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee72ffd640 1 -- 172.21.15.32:0/3460063811 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee6c007640 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee72ffd640 1 -- 172.21.15.32:0/3460063811 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee6c004a10 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee843bc640 1 -- 172.21.15.32:0/3460063811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee7c151860 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee843bc640 1 -- 172.21.15.32:0/3460063811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee7c149010 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee843bc640 1 -- 172.21.15.32:0/3460063811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7c151d80 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee72ffd640 1 -- 172.21.15.32:0/3460063811 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 67) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fee6c004b70 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee72ffd640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee60065ca0 0x7fee60068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:04.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee72ffd640 1 -- 172.21.15.32:0/3460063811 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fee6c0828e0 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee82158640 1 -- 172.21.15.32:0/3460063811 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee60065ca0 msgr2=0x7fee60068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:04.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.334+0000 7fee82158640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee60065ca0 0x7fee60068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.338+0000 7fee72ffd640 1 -- 172.21.15.32:0/3460063811 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee7c151d80 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.431 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:04.433+0000 7f92b5f6e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:47:04.471 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:04.474+0000 7f4f30e0a180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:47:04.493 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:04.497+0000 7f92b5f6e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:47:04.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.498+0000 7fee843bc640 1 -- 172.21.15.32:0/3460063811 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fee7c151d80 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee72ffd640 1 -- 172.21.15.32:0/3460063811 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7fee7c151d80 con 0x7fee7c0b0cb0 2023-12-20T04:47:04.498 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:47:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 -- 172.21.15.32:0/3460063811 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee60065ca0 msgr2=0x7fee60068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee60065ca0 0x7fee60068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 -- 172.21.15.32:0/3460063811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c0b0cb0 msgr2=0x7fee7c149280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c0b0cb0 0x7fee7c149280 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7fee6c0011f0 tx=0x7fee6c00c9b0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 -- 172.21.15.32:0/3460063811 shutdown_connections 2023-12-20T04:47:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fee60065ca0 0x7fee60068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee7c15abb0 0x7fee7c137ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee7c0b0cb0 0x7fee7c149280 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 --2- 172.21.15.32:0/3460063811 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee7c070230 0x7fee7c148ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 -- 172.21.15.32:0/3460063811 >> 172.21.15.32:0/3460063811 conn(0x7fee7c06a510 msgr2=0x7fee7c156b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:04.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 -- 172.21.15.32:0/3460063811 shutdown_connections 2023-12-20T04:47:04.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.502+0000 7fee843bc640 1 -- 172.21.15.32:0/3460063811 wait complete. 2023-12-20T04:47:04.523 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:47:04.537 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:04.538+0000 7f4f30e0a180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:47:04.552 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:04.553+0000 7f92b5f6e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:47:04.598 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:04.598+0000 7f4f30e0a180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:47:04.643 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:04.645+0000 7f92b5f6e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:47:04.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.674+0000 7fb20282d640 1 Processor -- start 2023-12-20T04:47:04.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.674+0000 7fb20282d640 1 -- start start 2023-12-20T04:47:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.674+0000 7fb20282d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb1fc0d3300 0x7fb1fc0d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.674+0000 7fb20282d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc0d2650 con 0x7fb1fc0d1300 2023-12-20T04:47:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.674+0000 7fb20282d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1fc0d1a40 con 0x7fb1fc15ce60 2023-12-20T04:47:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.674+0000 7fb20282d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1fc0d1d20 con 0x7fb1fc0d3300 2023-12-20T04:47:04.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.674+0000 7fb200dca640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb1fc0d1300 0x7fb1fc0b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40754/0 (socket says 172.21.15.32:40754) 2023-12-20T04:47:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.674+0000 7fb200dca640 1 -- 172.21.15.32:0/1775050028 learned_addr learned my addr 172.21.15.32:0/1775050028 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.674+0000 7fb1fb7fe640 1 --2- 172.21.15.32:0/1775050028 >> v2:172.21.15.39:3300/0 conn(0x7fb1fc0d3300 0x7fb1fc0d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:04.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.678+0000 7fb1faffd640 1 -- 172.21.15.32:0/1775050028 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2953774471 0 0) 0x7fb1fc0d2650 con 0x7fb1fc0d1300 2023-12-20T04:47:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.678+0000 7fb1faffd640 1 -- 172.21.15.32:0/1775050028 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1e4003530 con 0x7fb1fc0d1300 2023-12-20T04:47:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.678+0000 7fb1faffd640 1 -- 172.21.15.32:0/1775050028 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3969293886 0 0) 0x7fb1fc0d1a40 con 0x7fb1fc15ce60 2023-12-20T04:47:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.678+0000 7fb1faffd640 1 -- 172.21.15.32:0/1775050028 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb1fc0d2650 con 0x7fb1fc15ce60 2023-12-20T04:47:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.678+0000 7fb1fb7fe640 1 -- 172.21.15.32:0/1775050028 >> v1:172.21.15.39:6789/0 conn(0x7fb1fc15ce60 legacy=0x7fb1fc0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.678+0000 7fb1fb7fe640 1 -- 172.21.15.32:0/1775050028 >> v1:172.21.15.32:6789/0 conn(0x7fb1fc0d1300 legacy=0x7fb1fc0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.678+0000 7fb1fb7fe640 1 -- 172.21.15.32:0/1775050028 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1fc0d2000 con 0x7fb1fc0d3300 2023-12-20T04:47:04.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.678+0000 7fb1fb7fe640 1 --2- 172.21.15.32:0/1775050028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1fc0d3300 0x7fb1fc0d0dc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00097c0 tx=0x7fb1f001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cd4d7d3cff7fcdb6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.678+0000 7fb1faffd640 1 -- 172.21.15.32:0/1775050028 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00ac640 con 0x7fb1fc0d3300 2023-12-20T04:47:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.678+0000 7fb1faffd640 1 -- 172.21.15.32:0/1775050028 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f00ac7a0 con 0x7fb1fc0d3300 2023-12-20T04:47:04.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.678+0000 7fb1faffd640 1 -- 172.21.15.32:0/1775050028 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00acb30 con 0x7fb1fc0d3300 2023-12-20T04:47:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 -- 172.21.15.32:0/1775050028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1fc0d3300 msgr2=0x7fb1fc0d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 --2- 172.21.15.32:0/1775050028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1fc0d3300 0x7fb1fc0d0dc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fb1f00097c0 tx=0x7fb1f001c480 comp rx=0 tx=0).stop 2023-12-20T04:47:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 -- 172.21.15.32:0/1775050028 shutdown_connections 2023-12-20T04:47:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 --2- 172.21.15.32:0/1775050028 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1fc0d3300 0x7fb1fc0d0dc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 -- 172.21.15.32:0/1775050028 >> 172.21.15.32:0/1775050028 conn(0x7fb1fc06b5e0 msgr2=0x7fb1fc06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 -- 172.21.15.32:0/1775050028 shutdown_connections 2023-12-20T04:47:04.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 -- 172.21.15.32:0/1775050028 wait complete. 2023-12-20T04:47:04.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 Processor -- start 2023-12-20T04:47:04.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 -- start start 2023-12-20T04:47:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1fc0d1300 0x7fb1fc14e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1fc0d3300 0x7fb1fc0b22a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1fc15ce60 0x7fb1fc0b27e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:04.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc157650 con 0x7fb1fc0d1300 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc1569d0 con 0x7fb1fc0d3300 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb20282d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1fc156cb0 con 0x7fb1fc15ce60 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb1fbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1fc0d1300 0x7fb1fc14e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb1fb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1fc0d3300 0x7fb1fc0b22a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb1fbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1fc0d1300 0x7fb1fc14e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48732/0 (socket says 172.21.15.32:48732) 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb1fbfff640 1 -- 172.21.15.32:0/3834570044 learned_addr learned my addr 172.21.15.32:0/3834570044 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.682+0000 7fb200dca640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1fc15ce60 0x7fb1fc0b27e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1fb7fe640 1 -- 172.21.15.32:0/3834570044 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1fc15ce60 msgr2=0x7fb1fc0b27e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1fb7fe640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1fc15ce60 0x7fb1fc0b27e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1fb7fe640 1 -- 172.21.15.32:0/3834570044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1fc0d1300 msgr2=0x7fb1fc14e2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1fb7fe640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1fc0d1300 0x7fb1fc14e2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1fb7fe640 1 -- 172.21.15.32:0/3834570044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1f0009420 con 0x7fb1fc0d3300 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1fbfff640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1fc0d1300 0x7fb1fc14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:04.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1fb7fe640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1fc0d3300 0x7fb1fc0b22a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0b5850 tx=0x7fb1f0019220 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:04.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1f8ff9640 1 -- 172.21.15.32:0/3834570044 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00b5610 con 0x7fb1fc0d3300 2023-12-20T04:47:04.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1f8ff9640 1 -- 172.21.15.32:0/3834570044 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1f00b57b0 con 0x7fb1fc0d3300 2023-12-20T04:47:04.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb20282d640 1 -- 172.21.15.32:0/3834570044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1fc156f90 con 0x7fb1fc0d3300 2023-12-20T04:47:04.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1f8ff9640 1 -- 172.21.15.32:0/3834570044 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1f00b5b60 con 0x7fb1fc0d3300 2023-12-20T04:47:04.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb20282d640 1 -- 172.21.15.32:0/3834570044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1fc14e7e0 con 0x7fb1fc0d3300 2023-12-20T04:47:04.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb20282d640 1 -- 172.21.15.32:0/3834570044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1fc04e000 con 0x7fb1fc0d3300 2023-12-20T04:47:04.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1f8ff9640 1 -- 172.21.15.32:0/3834570044 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 67) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fb1f00b3040 con 0x7fb1fc0d3300 2023-12-20T04:47:04.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1f8ff9640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1e8065ca0 0x7fb1e8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:04.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1f8ff9640 1 -- 172.21.15.32:0/3834570044 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fb1f011d660 con 0x7fb1fc0d3300 2023-12-20T04:47:04.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1fbfff640 1 -- 172.21.15.32:0/3834570044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1e8065ca0 msgr2=0x7fb1e8068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:04.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.686+0000 7fb1fbfff640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1e8065ca0 0x7fb1e8068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:04.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.694+0000 7fb1f8ff9640 1 -- 172.21.15.32:0/3834570044 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb1f00eac00 con 0x7fb1fc0d3300 2023-12-20T04:47:04.695 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:04.698+0000 7f4f30e0a180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:47:04.703 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:04.705+0000 7f92b5f6e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:47:04.756 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:04.758+0000 7f4f30e0a180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:47:04.792 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:04.793+0000 7f92b5f6e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:47:04.845 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:04.846+0000 7f4f30e0a180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:47:04.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.850+0000 7fb20282d640 1 -- 172.21.15.32:0/3834570044 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fb1fc06bd40 con 0x7fb1fc0d3300 2023-12-20T04:47:04.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.850+0000 7fb1f8ff9640 1 -- 172.21.15.32:0/3834570044 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v124) v1 ==== 100+0+11 (secure 0 0 0) 0x7fb1f00ee220 con 0x7fb1fc0d3300 2023-12-20T04:47:04.850 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:47:04.851 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:04.853+0000 7f92b5f6e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:47:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 -- 172.21.15.32:0/3834570044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1e8065ca0 msgr2=0x7fb1e8068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1e8065ca0 0x7fb1e8068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 -- 172.21.15.32:0/3834570044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1fc0d3300 msgr2=0x7fb1fc0b22a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1fc0d3300 0x7fb1fc0b22a0 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7fb1fc0b5850 tx=0x7fb1f0019220 comp rx=0 tx=0).stop 2023-12-20T04:47:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 -- 172.21.15.32:0/3834570044 shutdown_connections 2023-12-20T04:47:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb1e8065ca0 0x7fb1e8068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1fc15ce60 0x7fb1fc0b27e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1fc0d3300 0x7fb1fc0b22a0 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 --2- 172.21.15.32:0/3834570044 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1fc0d1300 0x7fb1fc14e2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 -- 172.21.15.32:0/3834570044 >> 172.21.15.32:0/3834570044 conn(0x7fb1fc06b5e0 msgr2=0x7fb1fc157c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 -- 172.21.15.32:0/3834570044 shutdown_connections 2023-12-20T04:47:04.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:04.854+0000 7fb20282d640 1 -- 172.21.15.32:0/3834570044 wait complete. 2023-12-20T04:47:04.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T04:47:04.907 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:04.910+0000 7f4f30e0a180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:47:05.005 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:05.009+0000 7f92b5f6e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:47:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc80562d640 1 Processor -- start 2023-12-20T04:47:05.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc80562d640 1 -- start start 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc80562d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc80015cac0 0x7fc800160800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc80562d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8000cde70 0x7fc8000d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc80562d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8000d0c90 con 0x7fc80015cea0 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc80562d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc800161110 con 0x7fc8000cde70 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc80562d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8001613f0 con 0x7fc80015cac0 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc7ff7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8000cde70 0x7fc8000d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc7ff7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8000cde70 0x7fc8000d1450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48746/0 (socket says 172.21.15.32:48746) 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc7ff7fe640 1 -- 172.21.15.32:0/2434545693 learned_addr learned my addr 172.21.15.32:0/2434545693 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc7feffd640 1 --2- 172.21.15.32:0/2434545693 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc80015cac0 0x7fc800160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc7ff7fe640 1 -- 172.21.15.32:0/2434545693 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc80015cac0 msgr2=0x7fc800160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc7ff7fe640 1 --2- 172.21.15.32:0/2434545693 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc80015cac0 0x7fc800160800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc7ff7fe640 1 -- 172.21.15.32:0/2434545693 >> v1:172.21.15.32:6789/0 conn(0x7fc80015cea0 legacy=0x7fc8000cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc7ff7fe640 1 -- 172.21.15.32:0/2434545693 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8000d0f30 con 0x7fc8000cde70 2023-12-20T04:47:05.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.058+0000 7fc7fdffb640 1 -- 172.21.15.32:0/2434545693 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 501614941 0 0) 0x7fc8000d0c90 con 0x7fc80015cea0 2023-12-20T04:47:05.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc7ff7fe640 1 --2- 172.21.15.32:0/2434545693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000cde70 0x7fc8000d1450 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009e50 tx=0x7fc7f801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1772efba980fa03 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:05.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc7fdffb640 1 -- 172.21.15.32:0/2434545693 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80a5900 con 0x7fc8000cde70 2023-12-20T04:47:05.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc7fdffb640 1 -- 172.21.15.32:0/2434545693 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f80a5a60 con 0x7fc8000cde70 2023-12-20T04:47:05.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc7fdffb640 1 -- 172.21.15.32:0/2434545693 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f80a5df0 con 0x7fc8000cde70 2023-12-20T04:47:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc80562d640 1 -- 172.21.15.32:0/2434545693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000cde70 msgr2=0x7fc8000d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc80562d640 1 --2- 172.21.15.32:0/2434545693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000cde70 0x7fc8000d1450 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7fc7f8009e50 tx=0x7fc7f801ca10 comp rx=0 tx=0).stop 2023-12-20T04:47:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc80562d640 1 -- 172.21.15.32:0/2434545693 shutdown_connections 2023-12-20T04:47:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc80562d640 1 --2- 172.21.15.32:0/2434545693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000cde70 0x7fc8000d1450 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc80562d640 1 --2- 172.21.15.32:0/2434545693 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc80015cac0 0x7fc800160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc80562d640 1 -- 172.21.15.32:0/2434545693 >> 172.21.15.32:0/2434545693 conn(0x7fc80006afd0 msgr2=0x7fc80006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc80562d640 1 -- 172.21.15.32:0/2434545693 shutdown_connections 2023-12-20T04:47:05.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.062+0000 7fc80562d640 1 -- 172.21.15.32:0/2434545693 wait complete. 2023-12-20T04:47:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc80562d640 1 Processor -- start 2023-12-20T04:47:05.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc80562d640 1 -- start start 2023-12-20T04:47:05.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc80562d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000cde70 0x7fc800156590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc80562d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80015cac0 0x7fc8000c5490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc80562d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc80015cea0 0x7fc80014ed90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc80562d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc80014de10 con 0x7fc8000cde70 2023-12-20T04:47:05.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc80562d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8001555c0 con 0x7fc80015cac0 2023-12-20T04:47:05.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc80562d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc80014dff0 con 0x7fc80015cea0 2023-12-20T04:47:05.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc7feffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000cde70 0x7fc800156590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc7feffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000cde70 0x7fc800156590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48758/0 (socket says 172.21.15.32:48758) 2023-12-20T04:47:05.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc7feffd640 1 -- 172.21.15.32:0/2324084316 learned_addr learned my addr 172.21.15.32:0/2324084316 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:05.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc7fe7fc640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80015cac0 0x7fc8000c5490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc7ff7fe640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc80015cea0 0x7fc80014ed90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.067 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:05.069+0000 7f92b5f6e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:47:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc7feffd640 1 -- 172.21.15.32:0/2324084316 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc80015cea0 msgr2=0x7fc80014ed90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc7feffd640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc80015cea0 0x7fc80014ed90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc7feffd640 1 -- 172.21.15.32:0/2324084316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80015cac0 msgr2=0x7fc8000c5490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.066+0000 7fc7feffd640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80015cac0 0x7fc8000c5490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc7feffd640 1 -- 172.21.15.32:0/2324084316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7f8009b30 con 0x7fc8000cde70 2023-12-20T04:47:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc7feffd640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000cde70 0x7fc800156590 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000e930 tx=0x7fc7f000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:05.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc7e77fe640 1 -- 172.21.15.32:0/2324084316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f0009690 con 0x7fc8000cde70 2023-12-20T04:47:05.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc7e77fe640 1 -- 172.21.15.32:0/2324084316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7f000ca00 con 0x7fc8000cde70 2023-12-20T04:47:05.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc7e77fe640 1 -- 172.21.15.32:0/2324084316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7f000cd70 con 0x7fc8000cde70 2023-12-20T04:47:05.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc80562d640 1 -- 172.21.15.32:0/2324084316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc80014e2d0 con 0x7fc8000cde70 2023-12-20T04:47:05.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc80562d640 1 -- 172.21.15.32:0/2324084316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc80014e890 con 0x7fc8000cde70 2023-12-20T04:47:05.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc80562d640 1 -- 172.21.15.32:0/2324084316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc800074100 con 0x7fc8000cde70 2023-12-20T04:47:05.069 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:05.070+0000 7f4f30e0a180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:47:05.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc7e77fe640 1 -- 172.21.15.32:0/2324084316 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 67) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fc7f0013470 con 0x7fc8000cde70 2023-12-20T04:47:05.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc7e77fe640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7ec065ca0 0x7fc7ec068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc7e77fe640 1 -- 172.21.15.32:0/2324084316 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fc7f00839a0 con 0x7fc8000cde70 2023-12-20T04:47:05.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc7fe7fc640 1 -- 172.21.15.32:0/2324084316 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7ec065ca0 msgr2=0x7fc7ec068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:05.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.070+0000 7fc7fe7fc640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7ec065ca0 0x7fc7ec068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:05.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.074+0000 7fc7e77fe640 1 -- 172.21.15.32:0/2324084316 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc7f0050f70 con 0x7fc8000cde70 2023-12-20T04:47:05.132 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:05.134+0000 7f4f30e0a180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:47:05.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.250+0000 7fc80562d640 1 -- 172.21.15.32:0/2324084316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc8001602d0 con 0x7fc8000cde70 2023-12-20T04:47:05.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.250+0000 7fc7e77fe640 1 -- 172.21.15.32:0/2324084316 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v124) v1 ==== 80+0+458 (secure 0 0 0) 0x7fc7f00211e0 con 0x7fc8000cde70 2023-12-20T04:47:05.249 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1806203372 2023-12-21T04:46:13.032274+0000 2023-12-20T04:47:05.249 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/878030299 2023-12-21T04:46:13.032274+0000 2023-12-20T04:47:05.250 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3355189708 2023-12-21T04:46:13.032274+0000 2023-12-20T04:47:05.250 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/224603579 2023-12-21T04:46:13.032274+0000 2023-12-20T04:47:05.250 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/38009 2023-12-21T04:46:13.032274+0000 2023-12-20T04:47:05.250 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1563603368 2023-12-21T04:46:13.032274+0000 2023-12-20T04:47:05.250 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/38009 2023-12-21T04:46:13.032274+0000 2023-12-20T04:47:05.250 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1523085934 2023-12-21T04:46:13.032274+0000 2023-12-20T04:47:05.250 INFO:teuthology.orchestra.run.smithi032.stderr:listed 8 entries 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 -- 172.21.15.32:0/2324084316 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7ec065ca0 msgr2=0x7fc7ec068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7ec065ca0 0x7fc7ec068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 -- 172.21.15.32:0/2324084316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000cde70 msgr2=0x7fc800156590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000cde70 0x7fc800156590 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7fc7f000e930 tx=0x7fc7f000edf0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 -- 172.21.15.32:0/2324084316 shutdown_connections 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc7ec065ca0 0x7fc7ec068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc80015cea0 0x7fc80014ed90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc80015cac0 0x7fc8000c5490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 --2- 172.21.15.32:0/2324084316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8000cde70 0x7fc800156590 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 -- 172.21.15.32:0/2324084316 >> 172.21.15.32:0/2324084316 conn(0x7fc80006afd0 msgr2=0x7fc8000cf360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 -- 172.21.15.32:0/2324084316 shutdown_connections 2023-12-20T04:47:05.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.254+0000 7fc80562d640 1 -- 172.21.15.32:0/2324084316 wait complete. 2023-12-20T04:47:05.271 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T04:47:05.271 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:05.271 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T04:47:05.318 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:47:05.318 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:47:05.318 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:47:05.327 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:05.329+0000 7f92b5f6e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:47:05.328 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T04:47:05.328 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:05.328 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T04:47:05.346 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:47:05.346 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:47:05.355 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:47:05.355 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:47:05.356 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:47:05.389 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:47:05.389 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:47:05.389 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:47:05.398 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:47:05.398 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:47:05.398 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:47:05.398 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:05.398+0000 7f4f30e0a180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:47:05.407 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:47:05.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:47:05.469 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:05.473+0000 7f92b5f6e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:47:05.542 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:05.542+0000 7f4f30e0a180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:47:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.586+0000 7f8a65198640 1 Processor -- start 2023-12-20T04:47:05.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.586+0000 7f8a65198640 1 -- start start 2023-12-20T04:47:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a65198640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a600cfc80 0x7f8a60167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a65198640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a600b6ca0 con 0x7f8a6015cdb0 2023-12-20T04:47:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a65198640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8a6015d190 con 0x7f8a600d2740 2023-12-20T04:47:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a65198640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a600d16a0 con 0x7f8a600cfc80 2023-12-20T04:47:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a5ed9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8a6015cdb0 0x7f8a600d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40778/0 (socket says 172.21.15.32:40778) 2023-12-20T04:47:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a5ed9d640 1 -- 172.21.15.32:0/326878966 learned_addr learned my addr 172.21.15.32:0/326878966 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a5f59e640 1 --2- 172.21.15.32:0/326878966 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a600cfc80 0x7f8a60167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a5dd9b640 1 -- 172.21.15.32:0/326878966 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2880867731 0 0) 0x7f8a50001120 con 0x7f8a6015cdb0 2023-12-20T04:47:05.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a5dd9b640 1 -- 172.21.15.32:0/326878966 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a48003530 con 0x7f8a6015cdb0 2023-12-20T04:47:05.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a5dd9b640 1 -- 172.21.15.32:0/326878966 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1349239657 0 0) 0x7f8a6015d190 con 0x7f8a600d2740 2023-12-20T04:47:05.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a5dd9b640 1 -- 172.21.15.32:0/326878966 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8a50001120 con 0x7f8a600d2740 2023-12-20T04:47:05.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a5f59e640 1 -- 172.21.15.32:0/326878966 >> v1:172.21.15.39:6789/0 conn(0x7f8a600d2740 legacy=0x7f8a600cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a5f59e640 1 -- 172.21.15.32:0/326878966 >> v1:172.21.15.32:6789/0 conn(0x7f8a6015cdb0 legacy=0x7f8a600d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.590+0000 7f8a5f59e640 1 -- 172.21.15.32:0/326878966 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a600d1980 con 0x7f8a600cfc80 2023-12-20T04:47:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a5f59e640 1 --2- 172.21.15.32:0/326878966 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a600cfc80 0x7f8a60167910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009dc0 tx=0x7f8a5801f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=20f441ea045670be server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a5dd9b640 1 -- 172.21.15.32:0/326878966 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a580af600 con 0x7f8a600cfc80 2023-12-20T04:47:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a5dd9b640 1 -- 172.21.15.32:0/326878966 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a580ab050 con 0x7f8a600cfc80 2023-12-20T04:47:05.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a5dd9b640 1 -- 172.21.15.32:0/326878966 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a580af980 con 0x7f8a600cfc80 2023-12-20T04:47:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a65198640 1 -- 172.21.15.32:0/326878966 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a600cfc80 msgr2=0x7f8a60167910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a65198640 1 --2- 172.21.15.32:0/326878966 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a600cfc80 0x7f8a60167910 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8a58009dc0 tx=0x7f8a5801f150 comp rx=0 tx=0).stop 2023-12-20T04:47:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a65198640 1 -- 172.21.15.32:0/326878966 shutdown_connections 2023-12-20T04:47:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a65198640 1 --2- 172.21.15.32:0/326878966 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a600cfc80 0x7f8a60167910 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a65198640 1 -- 172.21.15.32:0/326878966 >> 172.21.15.32:0/326878966 conn(0x7f8a60065ee0 msgr2=0x7f8a6006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a65198640 1 -- 172.21.15.32:0/326878966 shutdown_connections 2023-12-20T04:47:05.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a65198640 1 -- 172.21.15.32:0/326878966 wait complete. 2023-12-20T04:47:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a65198640 1 Processor -- start 2023-12-20T04:47:05.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.594+0000 7f8a65198640 1 -- start start 2023-12-20T04:47:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a65198640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a600cfc80 0x7f8a600cbde0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a65198640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a600d2740 0x7f8a60156680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a65198640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a6015cdb0 0x7f8a60156bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a65198640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6014e500 con 0x7f8a600d2740 2023-12-20T04:47:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a65198640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8a60157100 con 0x7f8a600cfc80 2023-12-20T04:47:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a65198640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8a6015dd60 con 0x7f8a6015cdb0 2023-12-20T04:47:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a600d2740 0x7f8a60156680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a600cfc80 0x7f8a600cbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5f59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a6015cdb0 0x7f8a60156bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5e59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a600d2740 0x7f8a60156680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48762/0 (socket says 172.21.15.32:48762) 2023-12-20T04:47:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5ed9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a600cfc80 0x7f8a600cbde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:51760/0 (socket says 172.21.15.32:51760) 2023-12-20T04:47:05.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5e59c640 1 -- 172.21.15.32:0/43229292 learned_addr learned my addr 172.21.15.32:0/43229292 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5ed9d640 1 -- 172.21.15.32:0/43229292 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a6015cdb0 msgr2=0x7f8a60156bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5ed9d640 1 --2- 172.21.15.32:0/43229292 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a6015cdb0 0x7f8a60156bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5ed9d640 1 -- 172.21.15.32:0/43229292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a600d2740 msgr2=0x7f8a60156680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5ed9d640 1 --2- 172.21.15.32:0/43229292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a600d2740 0x7f8a60156680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5ed9d640 1 -- 172.21.15.32:0/43229292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8a58009a70 con 0x7f8a600cfc80 2023-12-20T04:47:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a5ed9d640 1 --2- 172.21.15.32:0/43229292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a600cfc80 0x7f8a600cbde0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000c520 tx=0x7f8a5000c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a477fe640 1 -- 172.21.15.32:0/43229292 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50005cf0 con 0x7f8a600cfc80 2023-12-20T04:47:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a477fe640 1 -- 172.21.15.32:0/43229292 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8a50005e50 con 0x7f8a600cfc80 2023-12-20T04:47:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a65198640 1 -- 172.21.15.32:0/43229292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8a6015e040 con 0x7f8a600cfc80 2023-12-20T04:47:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a477fe640 1 -- 172.21.15.32:0/43229292 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8a50007290 con 0x7f8a600cfc80 2023-12-20T04:47:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a65198640 1 -- 172.21.15.32:0/43229292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8a6015e600 con 0x7f8a600cfc80 2023-12-20T04:47:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.598+0000 7f8a65198640 1 -- 172.21.15.32:0/43229292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8a6004e000 con 0x7f8a600cfc80 2023-12-20T04:47:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.602+0000 7f8a477fe640 1 -- 172.21.15.32:0/43229292 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 67) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f8a50007430 con 0x7f8a600cfc80 2023-12-20T04:47:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.602+0000 7f8a477fe640 1 --2- 172.21.15.32:0/43229292 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a4c065ca0 0x7f8a4c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.602+0000 7f8a5e59c640 1 -- 172.21.15.32:0/43229292 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a4c065ca0 msgr2=0x7f8a4c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.602+0000 7f8a477fe640 1 -- 172.21.15.32:0/43229292 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f8a50082bf0 con 0x7f8a600cfc80 2023-12-20T04:47:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.602+0000 7f8a5e59c640 1 --2- 172.21.15.32:0/43229292 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a4c065ca0 0x7f8a4c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:05.601 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:05.601+0000 7f92b5f6e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:47:05.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.606+0000 7f8a477fe640 1 -- 172.21.15.32:0/43229292 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8a500501c0 con 0x7f8a600cfc80 2023-12-20T04:47:05.674 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:05.677+0000 7f92b5f6e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:47:05.677 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:05.678+0000 7f4f30e0a180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:47:05.753 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:05.754+0000 7f4f30e0a180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:47:05.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.794+0000 7f8a65198640 1 -- 172.21.15.32:0/43229292 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8a600b6ca0 con 0x7f8a600cfc80 2023-12-20T04:47:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a477fe640 1 -- 172.21.15.32:0/43229292 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4705 (secure 0 0 0) 0x7f8a600b6ca0 con 0x7f8a600cfc80 2023-12-20T04:47:05.795 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:05.795 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/593676281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":593676281},{"type":"v1","addr":"172.21.15.39:6833","nonce":593676281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2437945959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2437945959},{"type":"v1","addr":"172.21.15.39:6835","nonce":2437945959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7681,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/430674440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":430674440},{"type":"v1","addr":"172.21.15.32:6837","nonce":430674440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:46:00.899089+0000","modified":"2023-12-20T04:46:03.967555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7881":{"gid":7881,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3259359483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3259359483},{"type":"v1","addr":"172.21.15.32:6836","nonce":3259359483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-20T04:47:05.795 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 54 2023-12-20T04:47:05.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 -- 172.21.15.32:0/43229292 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a4c065ca0 msgr2=0x7f8a4c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:05.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 --2- 172.21.15.32:0/43229292 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a4c065ca0 0x7f8a4c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 -- 172.21.15.32:0/43229292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a600cfc80 msgr2=0x7f8a600cbde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 --2- 172.21.15.32:0/43229292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a600cfc80 0x7f8a600cbde0 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f8a5000c520 tx=0x7f8a5000c9e0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 -- 172.21.15.32:0/43229292 shutdown_connections 2023-12-20T04:47:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 --2- 172.21.15.32:0/43229292 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8a4c065ca0 0x7f8a4c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 --2- 172.21.15.32:0/43229292 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8a6015cdb0 0x7f8a60156bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 --2- 172.21.15.32:0/43229292 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8a600d2740 0x7f8a60156680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 --2- 172.21.15.32:0/43229292 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8a600cfc80 0x7f8a600cbde0 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 -- 172.21.15.32:0/43229292 >> 172.21.15.32:0/43229292 conn(0x7f8a60065ee0 msgr2=0x7f8a6015ac70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 -- 172.21.15.32:0/43229292 shutdown_connections 2023-12-20T04:47:05.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.798+0000 7f8a65198640 1 -- 172.21.15.32:0/43229292 wait complete. 2023-12-20T04:47:05.817 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:05.817+0000 7f92b5f6e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:47:05.818 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:47:05.899 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:05.902+0000 7f4f30e0a180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:47:05.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.962+0000 7f9abe7bc640 1 Processor -- start 2023-12-20T04:47:05.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9abe7bc640 1 -- start start 2023-12-20T04:47:05.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9abe7bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9ab80d2740 0x7f9ab80cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9abe7bc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab815d190 con 0x7f9ab80cfab0 2023-12-20T04:47:05.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9abe7bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9ab80d16a0 con 0x7f9ab815cdb0 2023-12-20T04:47:05.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9abe7bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80cff40 con 0x7f9ab80d2740 2023-12-20T04:47:05.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9abcd59640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9ab80cfab0 0x7f9ab80b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40794/0 (socket says 172.21.15.32:40794) 2023-12-20T04:47:05.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9abcd59640 1 -- 172.21.15.32:0/1797487136 learned_addr learned my addr 172.21.15.32:0/1797487136 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:05.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9ab77fe640 1 --2- 172.21.15.32:0/1797487136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9ab80d2740 0x7f9ab80cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9ab6ffd640 1 -- 172.21.15.32:0/1797487136 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3404686668 0 0) 0x7f9ab80d16a0 con 0x7f9ab815cdb0 2023-12-20T04:47:05.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9ab6ffd640 1 -- 172.21.15.32:0/1797487136 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9aa0003530 con 0x7f9ab815cdb0 2023-12-20T04:47:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9ab6ffd640 1 -- 172.21.15.32:0/1797487136 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2966301128 0 0) 0x7f9ab815d190 con 0x7f9ab80cfab0 2023-12-20T04:47:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9ab77fe640 1 -- 172.21.15.32:0/1797487136 >> v1:172.21.15.39:6789/0 conn(0x7f9ab815cdb0 legacy=0x7f9ab80d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9ab77fe640 1 -- 172.21.15.32:0/1797487136 >> v1:172.21.15.32:6789/0 conn(0x7f9ab80cfab0 legacy=0x7f9ab80b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.966+0000 7f9ab77fe640 1 -- 172.21.15.32:0/1797487136 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ab80d1980 con 0x7f9ab80d2740 2023-12-20T04:47:05.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.970+0000 7f9ab77fe640 1 --2- 172.21.15.32:0/1797487136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9ab80d2740 0x7f9ab80cf570 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0097f0 tx=0x7f9aac01ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=29dbb38a06623220 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:05.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.970+0000 7f9ab6ffd640 1 -- 172.21.15.32:0/1797487136 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0a89b0 con 0x7f9ab80d2740 2023-12-20T04:47:05.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.970+0000 7f9ab6ffd640 1 -- 172.21.15.32:0/1797487136 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aac0a8b10 con 0x7f9ab80d2740 2023-12-20T04:47:05.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.970+0000 7f9ab6ffd640 1 -- 172.21.15.32:0/1797487136 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aac0a8ea0 con 0x7f9ab80d2740 2023-12-20T04:47:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 -- 172.21.15.32:0/1797487136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9ab80d2740 msgr2=0x7f9ab80cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 --2- 172.21.15.32:0/1797487136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9ab80d2740 0x7f9ab80cf570 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f9aac0097f0 tx=0x7f9aac01ecb0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 -- 172.21.15.32:0/1797487136 shutdown_connections 2023-12-20T04:47:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 --2- 172.21.15.32:0/1797487136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9ab80d2740 0x7f9ab80cf570 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 -- 172.21.15.32:0/1797487136 >> 172.21.15.32:0/1797487136 conn(0x7f9ab8065ee0 msgr2=0x7f9ab806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 -- 172.21.15.32:0/1797487136 shutdown_connections 2023-12-20T04:47:05.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 -- 172.21.15.32:0/1797487136 wait complete. 2023-12-20T04:47:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 Processor -- start 2023-12-20T04:47:05.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 -- start start 2023-12-20T04:47:05.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab80cfab0 0x7f9ab8150b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ab80d2740 0x7f9ab81f5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9ab815cdb0 0x7f9ab80cb360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab81577b0 con 0x7f9ab80cfab0 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab81f69e0 con 0x7f9ab80d2740 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abe7bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9ab80c9e60 con 0x7f9ab815cdb0 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9ab7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab80cfab0 0x7f9ab8150b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9ab7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab80cfab0 0x7f9ab8150b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48772/0 (socket says 172.21.15.32:48772) 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9ab7fff640 1 -- 172.21.15.32:0/535733296 learned_addr learned my addr 172.21.15.32:0/535733296 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9ab77fe640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ab80d2740 0x7f9ab81f5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9abcd59640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9ab815cdb0 0x7f9ab80cb360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9ab7fff640 1 -- 172.21.15.32:0/535733296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9ab815cdb0 msgr2=0x7f9ab80cb360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9ab7fff640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9ab815cdb0 0x7f9ab80cb360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9ab7fff640 1 -- 172.21.15.32:0/535733296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ab80d2740 msgr2=0x7f9ab81f5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:05.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9ab7fff640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ab80d2740 0x7f9ab81f5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.974+0000 7f9ab7fff640 1 -- 172.21.15.32:0/535733296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9aac009420 con 0x7f9ab80cfab0 2023-12-20T04:47:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9ab7fff640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab80cfab0 0x7f9ab8150b80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800c430 tx=0x7f9aa800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:05.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9ab4ff9640 1 -- 172.21.15.32:0/535733296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa80052d0 con 0x7f9ab80cfab0 2023-12-20T04:47:05.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9ab4ff9640 1 -- 172.21.15.32:0/535733296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9aa8005450 con 0x7f9ab80cfab0 2023-12-20T04:47:05.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9ab4ff9640 1 -- 172.21.15.32:0/535733296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9aa8003840 con 0x7f9ab80cfab0 2023-12-20T04:47:05.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9abe7bc640 1 -- 172.21.15.32:0/535733296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9ab81f6140 con 0x7f9ab80cfab0 2023-12-20T04:47:05.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9abe7bc640 1 -- 172.21.15.32:0/535733296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9ab81f6700 con 0x7f9ab80cfab0 2023-12-20T04:47:05.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9abe7bc640 1 -- 172.21.15.32:0/535733296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9ab804e000 con 0x7f9ab80cfab0 2023-12-20T04:47:05.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9ab4ff9640 1 -- 172.21.15.32:0/535733296 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 67) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f9aa80039a0 con 0x7f9ab80cfab0 2023-12-20T04:47:05.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9ab4ff9640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9aa4065ca0 0x7f9aa4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:05.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9ab4ff9640 1 -- 172.21.15.32:0/535733296 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f9aa8082c40 con 0x7f9ab80cfab0 2023-12-20T04:47:05.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9ab77fe640 1 -- 172.21.15.32:0/535733296 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9aa4065ca0 msgr2=0x7f9aa4068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:05.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.978+0000 7f9ab77fe640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9aa4065ca0 0x7f9aa4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:05.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:05.982+0000 7f9ab4ff9640 1 -- 172.21.15.32:0/535733296 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9aa80502b0 con 0x7f9ab80cfab0 2023-12-20T04:47:06.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.142+0000 7f9abe7bc640 1 -- 172.21.15.32:0/535733296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9ab806b8d0 con 0x7f9ab80cfab0 2023-12-20T04:47:06.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9ab4ff9640 1 -- 172.21.15.32:0/535733296 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v124) v1 ==== 74+0+16492 (secure 0 0 0) 0x7f9aa8017090 con 0x7f9ab80cfab0 2023-12-20T04:47:06.144 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:06.145 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":124,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:46:13.032320+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:45:59.452651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-20T04:45:59.882509+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"123","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1806203372":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/878030299":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/3355189708":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/224603579":"2023-12-21T04:46:13.032274+0000","172.21.15.32:6833/38009":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/1563603368":"2023-12-21T04:46:13.032274+0000","172.21.15.32:6832/38009":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/1523085934":"2023-12-21T04:46:13.032274+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:47:06.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 -- 172.21.15.32:0/535733296 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9aa4065ca0 msgr2=0x7f9aa4068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:06.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9aa4065ca0 0x7f9aa4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 -- 172.21.15.32:0/535733296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab80cfab0 msgr2=0x7f9ab8150b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:06.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab80cfab0 0x7f9ab8150b80 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7f9aa800c430 tx=0x7f9aa800c8f0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 -- 172.21.15.32:0/535733296 shutdown_connections 2023-12-20T04:47:06.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9aa4065ca0 0x7f9aa4068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9ab815cdb0 0x7f9ab80cb360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9ab80d2740 0x7f9ab81f5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 --2- 172.21.15.32:0/535733296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9ab80cfab0 0x7f9ab8150b80 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 -- 172.21.15.32:0/535733296 >> 172.21.15.32:0/535733296 conn(0x7f9ab8065ee0 msgr2=0x7f9ab80cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:06.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 -- 172.21.15.32:0/535733296 shutdown_connections 2023-12-20T04:47:06.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.146+0000 7f9abe7bc640 1 -- 172.21.15.32:0/535733296 wait complete. 2023-12-20T04:47:06.168 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 14, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:45:59.452651+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '122', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 15, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:45:59.882509+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '123', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:47:06.168 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:47:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc18c58f640 1 Processor -- start 2023-12-20T04:47:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc18c58f640 1 -- start start 2023-12-20T04:47:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc18c58f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc18415ce60 0x7fc1840d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc18c58f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc1840d3130 0x7fc1840d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc18c58f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1840d2650 con 0x7fc1840d1300 2023-12-20T04:47:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc18c58f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1840d3570 con 0x7fc1840d3130 2023-12-20T04:47:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc18c58f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840d1a40 con 0x7fc18415ce60 2023-12-20T04:47:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc18a32b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc18415ce60 0x7fc1840d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc189b2a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc1840d3130 0x7fc1840d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc18a32b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc18415ce60 0x7fc1840d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:40224/0 (socket says 172.21.15.32:40224) 2023-12-20T04:47:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc18a32b640 1 -- 172.21.15.32:0/3841648252 learned_addr learned my addr 172.21.15.32:0/3841648252 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.318+0000 7fc18ab2c640 1 --1- 172.21.15.32:0/3841648252 >> v1:172.21.15.39:6789/0 conn(0x7fc1840d1300 0x7fc1840b46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:60682/0 (socket says 172.21.15.32:60682) 2023-12-20T04:47:06.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.322+0000 7fc18a32b640 1 -- 172.21.15.32:0/3841648252 >> v2:172.21.15.39:3300/0 conn(0x7fc1840d3130 msgr2=0x7fc1840d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:06.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.322+0000 7fc18a32b640 1 --2- 172.21.15.32:0/3841648252 >> v2:172.21.15.39:3300/0 conn(0x7fc1840d3130 0x7fc1840d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.322+0000 7fc18a32b640 1 -- 172.21.15.32:0/3841648252 >> v1:172.21.15.39:6789/0 conn(0x7fc1840d1300 legacy=0x7fc1840b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:06.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.322+0000 7fc18a32b640 1 -- 172.21.15.32:0/3841648252 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1840d1d20 con 0x7fc18415ce60 2023-12-20T04:47:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.322+0000 7fc18a32b640 1 --2- 172.21.15.32:0/3841648252 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc18415ce60 0x7fc1840d2bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc180009870 tx=0x7fc18001ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c04292faa478019c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.322+0000 7fc189329640 1 -- 172.21.15.32:0/3841648252 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800a8a30 con 0x7fc18415ce60 2023-12-20T04:47:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.322+0000 7fc189329640 1 -- 172.21.15.32:0/3841648252 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1800a8b90 con 0x7fc18415ce60 2023-12-20T04:47:06.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.322+0000 7fc189329640 1 -- 172.21.15.32:0/3841648252 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1800b0440 con 0x7fc18415ce60 2023-12-20T04:47:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 -- 172.21.15.32:0/3841648252 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc18415ce60 msgr2=0x7fc1840d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 --2- 172.21.15.32:0/3841648252 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc18415ce60 0x7fc1840d2bf0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fc180009870 tx=0x7fc18001ed30 comp rx=0 tx=0).stop 2023-12-20T04:47:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 -- 172.21.15.32:0/3841648252 shutdown_connections 2023-12-20T04:47:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 --2- 172.21.15.32:0/3841648252 >> v2:172.21.15.39:3300/0 conn(0x7fc1840d3130 0x7fc1840d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 --2- 172.21.15.32:0/3841648252 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc18415ce60 0x7fc1840d2bf0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 -- 172.21.15.32:0/3841648252 >> 172.21.15.32:0/3841648252 conn(0x7fc18406b5e0 msgr2=0x7fc18406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 -- 172.21.15.32:0/3841648252 shutdown_connections 2023-12-20T04:47:06.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 -- 172.21.15.32:0/3841648252 wait complete. 2023-12-20T04:47:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 Processor -- start 2023-12-20T04:47:06.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 -- start start 2023-12-20T04:47:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1840d1300 0x7fc18414e030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1840d3130 0x7fc1840c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18415ce60 0x7fc1840b2f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840cb1e0 con 0x7fc18415ce60 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1841f1ed0 con 0x7fc1840d3130 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18c58f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1840ca660 con 0x7fc1840d1300 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18ab2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18415ce60 0x7fc1840b2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18ab2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18415ce60 0x7fc1840b2f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48776/0 (socket says 172.21.15.32:48776) 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18ab2c640 1 -- 172.21.15.32:0/2507409700 learned_addr learned my addr 172.21.15.32:0/2507409700 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc189b2a640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1840d3130 0x7fc1840c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.326+0000 7fc18a32b640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1840d1300 0x7fc18414e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc18ab2c640 1 -- 172.21.15.32:0/2507409700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1840d1300 msgr2=0x7fc18414e030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc18ab2c640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1840d1300 0x7fc18414e030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc18ab2c640 1 -- 172.21.15.32:0/2507409700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1840d3130 msgr2=0x7fc1840c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc18ab2c640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1840d3130 0x7fc1840c9d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc18ab2c640 1 -- 172.21.15.32:0/2507409700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1800094d0 con 0x7fc18415ce60 2023-12-20T04:47:06.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc18ab2c640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18415ce60 0x7fc1840b2f00 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c730 tx=0x7fc17c00cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:06.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc17affd640 1 -- 172.21.15.32:0/2507409700 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00ce40 con 0x7fc18415ce60 2023-12-20T04:47:06.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc17affd640 1 -- 172.21.15.32:0/2507409700 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc17c00a640 con 0x7fc18415ce60 2023-12-20T04:47:06.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc18c58f640 1 -- 172.21.15.32:0/2507409700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1840ca940 con 0x7fc18415ce60 2023-12-20T04:47:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc17affd640 1 -- 172.21.15.32:0/2507409700 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc17c00a9b0 con 0x7fc18415ce60 2023-12-20T04:47:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc18c58f640 1 -- 172.21.15.32:0/2507409700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1840caf00 con 0x7fc18415ce60 2023-12-20T04:47:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc18c58f640 1 -- 172.21.15.32:0/2507409700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc1841f2070 con 0x7fc18415ce60 2023-12-20T04:47:06.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc17affd640 1 -- 172.21.15.32:0/2507409700 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 67) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fc17c00ab10 con 0x7fc18415ce60 2023-12-20T04:47:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc17affd640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc168065ca0 0x7fc168068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc17affd640 1 -- 172.21.15.32:0/2507409700 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7fc17c082ea0 con 0x7fc18415ce60 2023-12-20T04:47:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc18a32b640 1 -- 172.21.15.32:0/2507409700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc168065ca0 msgr2=0x7fc168068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:06.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.330+0000 7fc18a32b640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc168065ca0 0x7fc168068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:06.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.334+0000 7fc17affd640 1 -- 172.21.15.32:0/2507409700 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc1841f2070 con 0x7fc18415ce60 2023-12-20T04:47:06.385 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:06.389+0000 7f92b5f6e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:47:06.473 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:06.477+0000 7f92b5f6e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:47:06.489 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:06.490+0000 7f4f30e0a180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:47:06.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 -- 172.21.15.32:0/2507409700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc1841f2070 con 0x7fc18415ce60 2023-12-20T04:47:06.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc17affd640 1 -- 172.21.15.32:0/2507409700 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v124) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc1841f2070 con 0x7fc18415ce60 2023-12-20T04:47:06.492 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:47:06.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 -- 172.21.15.32:0/2507409700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc168065ca0 msgr2=0x7fc168068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:06.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc168065ca0 0x7fc168068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 -- 172.21.15.32:0/2507409700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18415ce60 msgr2=0x7fc1840b2f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18415ce60 0x7fc1840b2f00 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7fc17c00c730 tx=0x7fc17c00cbf0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 -- 172.21.15.32:0/2507409700 shutdown_connections 2023-12-20T04:47:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc168065ca0 0x7fc168068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc18415ce60 0x7fc1840b2f00 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1840d3130 0x7fc1840c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 --2- 172.21.15.32:0/2507409700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1840d1300 0x7fc18414e030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 -- 172.21.15.32:0/2507409700 >> 172.21.15.32:0/2507409700 conn(0x7fc18406b5e0 msgr2=0x7fc18415c1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.494+0000 7fc18c58f640 1 -- 172.21.15.32:0/2507409700 shutdown_connections 2023-12-20T04:47:06.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.498+0000 7fc18c58f640 1 -- 172.21.15.32:0/2507409700 wait complete. 2023-12-20T04:47:06.514 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T04:47:06.580 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:06.582+0000 7f4f30e0a180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:47:06.629 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:06.633+0000 7f92b5f6e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:47:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f47758e5640 1 Processor -- start 2023-12-20T04:47:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f47758e5640 1 -- start start 2023-12-20T04:47:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f47758e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f477015ce60 0x7f47700d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f47758e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47700d3130 0x7f47700d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f47758e5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47700d2650 con 0x7f47700d1300 2023-12-20T04:47:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f47758e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47700d35a0 con 0x7f47700d3130 2023-12-20T04:47:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f47758e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47700d1a40 con 0x7f477015ce60 2023-12-20T04:47:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f4767fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47700d3130 0x7f47700d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f4767fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f47700d3130 0x7f47700d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48778/0 (socket says 172.21.15.32:48778) 2023-12-20T04:47:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f4767fff640 1 -- 172.21.15.32:0/1342514406 learned_addr learned my addr 172.21.15.32:0/1342514406 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f476effd640 1 --2- 172.21.15.32:0/1342514406 >> v2:172.21.15.39:3300/0 conn(0x7f477015ce60 0x7f47700d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f4767fff640 1 -- 172.21.15.32:0/1342514406 >> v2:172.21.15.39:3300/0 conn(0x7f477015ce60 msgr2=0x7f47700d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f4767fff640 1 --2- 172.21.15.32:0/1342514406 >> v2:172.21.15.39:3300/0 conn(0x7f477015ce60 0x7f47700d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f4767fff640 1 -- 172.21.15.32:0/1342514406 >> v1:172.21.15.39:6789/0 conn(0x7f47700d1300 legacy=0x7f47700b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f4767fff640 1 -- 172.21.15.32:0/1342514406 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47700d1d20 con 0x7f47700d3130 2023-12-20T04:47:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.666+0000 7f476effd640 1 --2- 172.21.15.32:0/1342514406 >> v2:172.21.15.39:3300/0 conn(0x7f477015ce60 0x7f47700d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f4767fff640 1 --2- 172.21.15.32:0/1342514406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47700d3130 0x7f47700d0dc0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f4758009870 tx=0x7f475801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fc4ed5674624dee server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:06.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f476e7fc640 1 -- 172.21.15.32:0/1342514406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580a89f0 con 0x7f47700d3130 2023-12-20T04:47:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f476e7fc640 1 -- 172.21.15.32:0/1342514406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47580a8b50 con 0x7f47700d3130 2023-12-20T04:47:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f476e7fc640 1 -- 172.21.15.32:0/1342514406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47580b0440 con 0x7f47700d3130 2023-12-20T04:47:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f47758e5640 1 -- 172.21.15.32:0/1342514406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47700d3130 msgr2=0x7f47700d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f47758e5640 1 --2- 172.21.15.32:0/1342514406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47700d3130 0x7f47700d0dc0 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7f4758009870 tx=0x7f475801ecf0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f47758e5640 1 -- 172.21.15.32:0/1342514406 shutdown_connections 2023-12-20T04:47:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f47758e5640 1 --2- 172.21.15.32:0/1342514406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47700d3130 0x7f47700d0dc0 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f47758e5640 1 --2- 172.21.15.32:0/1342514406 >> v2:172.21.15.39:3300/0 conn(0x7f477015ce60 0x7f47700d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f47758e5640 1 -- 172.21.15.32:0/1342514406 >> 172.21.15.32:0/1342514406 conn(0x7f477006b5e0 msgr2=0x7f477006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f47758e5640 1 -- 172.21.15.32:0/1342514406 shutdown_connections 2023-12-20T04:47:06.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.670+0000 7f47758e5640 1 -- 172.21.15.32:0/1342514406 wait complete. 2023-12-20T04:47:06.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f47758e5640 1 Processor -- start 2023-12-20T04:47:06.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f47758e5640 1 -- start start 2023-12-20T04:47:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f47758e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700d1300 0x7f47701f1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f47758e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47700d3130 0x7f47700cc820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f47758e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477015ce60 0x7f47700b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f47758e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47700cbaa0 con 0x7f477015ce60 2023-12-20T04:47:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f47758e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47700ac2b0 con 0x7f47700d1300 2023-12-20T04:47:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f47758e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47700cbec0 con 0x7f47700d3130 2023-12-20T04:47:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700d1300 0x7f47701f1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477015ce60 0x7f47700b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700d1300 0x7f47701f1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:51794/0 (socket says 172.21.15.32:51794) 2023-12-20T04:47:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476effd640 1 -- 172.21.15.32:0/3787026679 learned_addr learned my addr 172.21.15.32:0/3787026679 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:06.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f4767fff640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47700d3130 0x7f47700cc820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:06.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476effd640 1 -- 172.21.15.32:0/3787026679 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47700d3130 msgr2=0x7f47700cc820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476effd640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47700d3130 0x7f47700cc820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476effd640 1 -- 172.21.15.32:0/3787026679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477015ce60 msgr2=0x7f47700b6ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476effd640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477015ce60 0x7f47700b6ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476effd640 1 -- 172.21.15.32:0/3787026679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4760009420 con 0x7f47700d1300 2023-12-20T04:47:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476effd640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700d1300 0x7f47701f1800 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f476000e980 tx=0x7f476000ee40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476cff9640 1 -- 172.21.15.32:0/3787026679 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f476000ca00 con 0x7f47700d1300 2023-12-20T04:47:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476cff9640 1 -- 172.21.15.32:0/3787026679 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f476000cb60 con 0x7f47700d1300 2023-12-20T04:47:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f47758e5640 1 -- 172.21.15.32:0/3787026679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47580094d0 con 0x7f47700d1300 2023-12-20T04:47:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f476cff9640 1 -- 172.21.15.32:0/3787026679 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4760013470 con 0x7f47700d1300 2023-12-20T04:47:06.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.674+0000 7f47758e5640 1 -- 172.21.15.32:0/3787026679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47700cb100 con 0x7f47700d1300 2023-12-20T04:47:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.678+0000 7f47758e5640 1 -- 172.21.15.32:0/3787026679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47700cbd10 con 0x7f47700d1300 2023-12-20T04:47:06.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.678+0000 7f476cff9640 1 -- 172.21.15.32:0/3787026679 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 67) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f47600135d0 con 0x7f47700d1300 2023-12-20T04:47:06.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.678+0000 7f476cff9640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4750065ca0 0x7f4750068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:06.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.678+0000 7f4767fff640 1 -- 172.21.15.32:0/3787026679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4750065ca0 msgr2=0x7f4750068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:06.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.678+0000 7f476cff9640 1 -- 172.21.15.32:0/3787026679 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f47600839c0 con 0x7f47700d1300 2023-12-20T04:47:06.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.678+0000 7f4767fff640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4750065ca0 0x7f4750068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:06.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.686+0000 7f476cff9640 1 -- 172.21.15.32:0/3787026679 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4760050ee0 con 0x7f47700d1300 2023-12-20T04:47:06.721 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:06.725+0000 7f92b5f6e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:47:06.743 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:06.746+0000 7f4f30e0a180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:47:06.832 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:06.834+0000 7f4f30e0a180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:47:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.846+0000 7f47758e5640 1 -- 172.21.15.32:0/3787026679 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f477004e000 con 0x7f47700d1300 2023-12-20T04:47:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.846+0000 7f476cff9640 1 -- 172.21.15.32:0/3787026679 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v124) v1 ==== 104+0+11 (secure 0 0 0) 0x7f4760054500 con 0x7f47700d1300 2023-12-20T04:47:06.845 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T04:47:06.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 -- 172.21.15.32:0/3787026679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4750065ca0 msgr2=0x7f4750068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4750065ca0 0x7f4750068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 -- 172.21.15.32:0/3787026679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700d1300 msgr2=0x7f47701f1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700d1300 0x7f47701f1800 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7f476000e980 tx=0x7f476000ee40 comp rx=0 tx=0).stop 2023-12-20T04:47:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 -- 172.21.15.32:0/3787026679 shutdown_connections 2023-12-20T04:47:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4750065ca0 0x7f4750068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f477015ce60 0x7f47700b6ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47700d3130 0x7f47700cc820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 --2- 172.21.15.32:0/3787026679 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f47700d1300 0x7f47701f1800 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 -- 172.21.15.32:0/3787026679 >> 172.21.15.32:0/3787026679 conn(0x7f477006b5e0 msgr2=0x7f477006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 -- 172.21.15.32:0/3787026679 shutdown_connections 2023-12-20T04:47:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:06.850+0000 7f47758e5640 1 -- 172.21.15.32:0/3787026679 wait complete. 2023-12-20T04:47:06.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T04:47:07.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.014+0000 7f9e7fcf2640 1 Processor -- start 2023-12-20T04:47:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.014+0000 7f9e7fcf2640 1 -- start start 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7fcf2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e780b0bc0 0x7f9e780ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7fcf2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e780cde20 con 0x7f9e78157480 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7fcf2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e780c8e30 con 0x7f9e780b0760 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7fcf2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e780c9f50 con 0x7f9e780b0bc0 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7da8e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9e78157480 0x7f9e780c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40800/0 (socket says 172.21.15.32:40800) 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7da8e640 1 -- 172.21.15.32:0/3207067147 learned_addr learned my addr 172.21.15.32:0/3207067147 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7e28f640 1 --2- 172.21.15.32:0/3207067147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e780b0bc0 0x7f9e780ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7ca8c640 1 -- 172.21.15.32:0/3207067147 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1245149039 0 0) 0x7f9e780cde20 con 0x7f9e78157480 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7ca8c640 1 -- 172.21.15.32:0/3207067147 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e64005740 con 0x7f9e78157480 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7ca8c640 1 -- 172.21.15.32:0/3207067147 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3296395706 0 0) 0x7f9e780c8e30 con 0x7f9e780b0760 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7ca8c640 1 -- 172.21.15.32:0/3207067147 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9e780cde20 con 0x7f9e780b0760 2023-12-20T04:47:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7e28f640 1 -- 172.21.15.32:0/3207067147 >> v1:172.21.15.39:6789/0 conn(0x7f9e780b0760 legacy=0x7f9e780cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7ca8c640 1 -- 172.21.15.32:0/3207067147 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 454048129 0 0) 0x7f9e64005740 con 0x7f9e78157480 2023-12-20T04:47:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7e28f640 1 -- 172.21.15.32:0/3207067147 >> v1:172.21.15.32:6789/0 conn(0x7f9e78157480 legacy=0x7f9e780c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.018+0000 7f9e7e28f640 1 -- 172.21.15.32:0/3207067147 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e780ce070 con 0x7f9e780b0bc0 2023-12-20T04:47:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7e28f640 1 --2- 172.21.15.32:0/3207067147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e780b0bc0 0x7f9e780ccbc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009dc0 tx=0x7f9e7001c7a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=34064953c157e6f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7ca8c640 1 -- 172.21.15.32:0/3207067147 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74001680 con 0x7f9e780b0bc0 2023-12-20T04:47:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7ca8c640 1 -- 172.21.15.32:0/3207067147 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e700a5550 con 0x7f9e780b0bc0 2023-12-20T04:47:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7ca8c640 1 -- 172.21.15.32:0/3207067147 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e700a58e0 con 0x7f9e780b0bc0 2023-12-20T04:47:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/3207067147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e780b0bc0 msgr2=0x7f9e780ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7fcf2640 1 --2- 172.21.15.32:0/3207067147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e780b0bc0 0x7f9e780ccbc0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9e70009dc0 tx=0x7f9e7001c7a0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/3207067147 shutdown_connections 2023-12-20T04:47:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7fcf2640 1 --2- 172.21.15.32:0/3207067147 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e780b0bc0 0x7f9e780ccbc0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/3207067147 >> 172.21.15.32:0/3207067147 conn(0x7f9e7806a510 msgr2=0x7f9e780ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/3207067147 shutdown_connections 2023-12-20T04:47:07.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/3207067147 wait complete. 2023-12-20T04:47:07.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7fcf2640 1 Processor -- start 2023-12-20T04:47:07.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.022+0000 7f9e7fcf2640 1 -- start start 2023-12-20T04:47:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7fcf2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780b0760 0x7f9e781510f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7fcf2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e780b0bc0 0x7f9e780c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7fcf2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e78157480 0x7f9e7815b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7fcf2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7815d9f0 con 0x7f9e780b0760 2023-12-20T04:47:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7fcf2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7815d560 con 0x7f9e78157480 2023-12-20T04:47:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7fcf2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e7815cc30 con 0x7f9e780b0bc0 2023-12-20T04:47:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7da8e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780b0760 0x7f9e781510f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7da8e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780b0760 0x7f9e781510f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48800/0 (socket says 172.21.15.32:48800) 2023-12-20T04:47:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7da8e640 1 -- 172.21.15.32:0/1082794531 learned_addr learned my addr 172.21.15.32:0/1082794531 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7e28f640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e78157480 0x7f9e7815b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7d28d640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e780b0bc0 0x7f9e780c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7da8e640 1 -- 172.21.15.32:0/1082794531 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e780b0bc0 msgr2=0x7f9e780c6d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7da8e640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e780b0bc0 0x7f9e780c6d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7da8e640 1 -- 172.21.15.32:0/1082794531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e78157480 msgr2=0x7f9e7815b120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7da8e640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e78157480 0x7f9e7815b120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7da8e640 1 -- 172.21.15.32:0/1082794531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e70009a70 con 0x7f9e780b0760 2023-12-20T04:47:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7da8e640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780b0760 0x7f9e781510f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400c750 tx=0x7f9e7400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e6e7fc640 1 -- 172.21.15.32:0/1082794531 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74005af0 con 0x7f9e780b0760 2023-12-20T04:47:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e6e7fc640 1 -- 172.21.15.32:0/1082794531 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e74005c50 con 0x7f9e780b0760 2023-12-20T04:47:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/1082794531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e7815cf10 con 0x7f9e780b0760 2023-12-20T04:47:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e6e7fc640 1 -- 172.21.15.32:0/1082794531 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e74007070 con 0x7f9e780b0760 2023-12-20T04:47:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/1082794531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e7814fdd0 con 0x7f9e780b0760 2023-12-20T04:47:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.026+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/1082794531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e7815adf0 con 0x7f9e780b0760 2023-12-20T04:47:07.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.030+0000 7f9e6e7fc640 1 -- 172.21.15.32:0/1082794531 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 67) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f9e740071d0 con 0x7f9e780b0760 2023-12-20T04:47:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.030+0000 7f9e6e7fc640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9e5c065ca0 0x7f9e5c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.030+0000 7f9e7d28d640 1 -- 172.21.15.32:0/1082794531 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9e5c065ca0 msgr2=0x7f9e5c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.030+0000 7f9e6e7fc640 1 -- 172.21.15.32:0/1082794531 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(124..124 src has 1..124) v4 ==== 6053+0+0 (secure 0 0 0) 0x7f9e740832c0 con 0x7f9e780b0760 2023-12-20T04:47:07.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.030+0000 7f9e7d28d640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9e5c065ca0 0x7f9e5c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:07.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.034+0000 7f9e6e7fc640 1 -- 172.21.15.32:0/1082794531 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9e7815adf0 con 0x7f9e780b0760 2023-12-20T04:47:07.055 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:07.057+0000 7f92b5f6e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:47:07.118 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:07.121+0000 7f92b5f6e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:47:07.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.170+0000 7f9e6e7fc640 1 -- 172.21.15.32:0/1082794531 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 68) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f9e74050cd0 con 0x7f9e780b0760 2023-12-20T04:47:07.183 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.186+0000 7f4f30e0a180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:47:07.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.198+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/1082794531 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f9e7815adf0 con 0x7f9e780b0760 2023-12-20T04:47:07.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e6e7fc640 1 -- 172.21.15.32:0/1082794531 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v125) v1 ==== 100+0+11 (secure 0 0 0) 0x7f9e7815adf0 con 0x7f9e780b0760 2023-12-20T04:47:07.199 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T04:47:07.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/1082794531 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9e5c065ca0 msgr2=0x7f9e5c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:07.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9e5c065ca0 0x7f9e5c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/1082794531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780b0760 msgr2=0x7f9e781510f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780b0760 0x7f9e781510f0 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f9e7400c750 tx=0x7f9e7400cc10 comp rx=0 tx=0).stop 2023-12-20T04:47:07.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/1082794531 shutdown_connections 2023-12-20T04:47:07.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9e5c065ca0 0x7f9e5c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e78157480 0x7f9e7815b120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e780b0bc0 0x7f9e780c6d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 --2- 172.21.15.32:0/1082794531 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e780b0760 0x7f9e781510f0 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/1082794531 >> 172.21.15.32:0/1082794531 conn(0x7f9e7806a510 msgr2=0x7f9e780b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:07.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/1082794531 shutdown_connections 2023-12-20T04:47:07.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.202+0000 7f9e7fcf2640 1 -- 172.21.15.32:0/1082794531 wait complete. 2023-12-20T04:47:07.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:47:07.252 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.254+0000 7f4f30e0a180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:47:07.328 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.330+0000 7f4ebc581640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:07.329 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.330+0000 7f4ebc581640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:07.329 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.330+0000 7f4ebc581640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:07.329 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.330+0000 7f4ebc581640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:07.329 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.330+0000 7f4ebc581640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:07.329 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.330+0000 7f4eb957b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:07.329 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.330+0000 7f4eb957b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:07.329 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.330+0000 7f4eb957b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:07.330 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.330+0000 7f4eb957b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:07.330 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:07.330+0000 7f4eb957b640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:07.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.386+0000 7f35b528c640 1 Processor -- start 2023-12-20T04:47:07.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.386+0000 7f35b528c640 1 -- start start 2023-12-20T04:47:07.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.386+0000 7f35b528c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35b00aece0 0x7f35b00b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.386+0000 7f35b528c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b00c98c0 con 0x7f35b01599e0 2023-12-20T04:47:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.386+0000 7f35b528c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35b00c9ba0 con 0x7f35b00b0b30 2023-12-20T04:47:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.386+0000 7f35b528c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35b00c9e80 con 0x7f35b00aece0 2023-12-20T04:47:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35aeffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f35b01599e0 0x7f35b00c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40804/0 (socket says 172.21.15.32:40804) 2023-12-20T04:47:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35aeffd640 1 -- 172.21.15.32:0/2446440945 learned_addr learned my addr 172.21.15.32:0/2446440945 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35a7fff640 1 --2- 172.21.15.32:0/2446440945 >> v2:172.21.15.32:3300/0 conn(0x7f35b00aece0 0x7f35b00b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35ae7fc640 1 -- 172.21.15.32:0/2446440945 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1145052027 0 0) 0x7f35b00c98c0 con 0x7f35b01599e0 2023-12-20T04:47:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35ae7fc640 1 -- 172.21.15.32:0/2446440945 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f359c003530 con 0x7f35b01599e0 2023-12-20T04:47:07.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35ae7fc640 1 -- 172.21.15.32:0/2446440945 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 324523577 0 0) 0x7f35b00c9ba0 con 0x7f35b00b0b30 2023-12-20T04:47:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35a7fff640 1 -- 172.21.15.32:0/2446440945 >> v1:172.21.15.39:6789/0 conn(0x7f35b00b0b30 legacy=0x7f35b00701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35a7fff640 1 -- 172.21.15.32:0/2446440945 >> v1:172.21.15.32:6789/0 conn(0x7f35b01599e0 legacy=0x7f35b00c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35a7fff640 1 -- 172.21.15.32:0/2446440945 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35b00ca160 con 0x7f35b00aece0 2023-12-20T04:47:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35a7fff640 1 --2- 172.21.15.32:0/2446440945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b00aece0 0x7f35b00b05f0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f35980097c0 tx=0x7f359801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c92de3e6a4529a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35ae7fc640 1 -- 172.21.15.32:0/2446440945 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980a8050 con 0x7f35b00aece0 2023-12-20T04:47:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35ae7fc640 1 -- 172.21.15.32:0/2446440945 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35980ac5f0 con 0x7f35b00aece0 2023-12-20T04:47:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.390+0000 7f35ae7fc640 1 -- 172.21.15.32:0/2446440945 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35980ac960 con 0x7f35b00aece0 2023-12-20T04:47:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 -- 172.21.15.32:0/2446440945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b00aece0 msgr2=0x7f35b00b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 --2- 172.21.15.32:0/2446440945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b00aece0 0x7f35b00b05f0 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f35980097c0 tx=0x7f359801c480 comp rx=0 tx=0).stop 2023-12-20T04:47:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 -- 172.21.15.32:0/2446440945 shutdown_connections 2023-12-20T04:47:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 --2- 172.21.15.32:0/2446440945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b00aece0 0x7f35b00b05f0 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 -- 172.21.15.32:0/2446440945 >> 172.21.15.32:0/2446440945 conn(0x7f35b006a320 msgr2=0x7f35b00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 -- 172.21.15.32:0/2446440945 shutdown_connections 2023-12-20T04:47:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 -- 172.21.15.32:0/2446440945 wait complete. 2023-12-20T04:47:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 Processor -- start 2023-12-20T04:47:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 -- start start 2023-12-20T04:47:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b00aece0 0x7f35b015c860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35b00b0b30 0x7f35b00c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35b01599e0 0x7f35b00c5d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b01f8980 con 0x7f35b00aece0 2023-12-20T04:47:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35b015cda0 con 0x7f35b00b0b30 2023-12-20T04:47:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35b528c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35b00c63a0 con 0x7f35b01599e0 2023-12-20T04:47:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35aeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b00aece0 0x7f35b015c860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35aeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b00aece0 0x7f35b015c860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48914/0 (socket says 172.21.15.32:48914) 2023-12-20T04:47:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35aeffd640 1 -- 172.21.15.32:0/547265876 learned_addr learned my addr 172.21.15.32:0/547265876 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:07.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35a7fff640 1 --2- 172.21.15.32:0/547265876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35b00b0b30 0x7f35b00c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35af7fe640 1 --2- 172.21.15.32:0/547265876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35b01599e0 0x7f35b00c5d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35aeffd640 1 -- 172.21.15.32:0/547265876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35b01599e0 msgr2=0x7f35b00c5d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35aeffd640 1 --2- 172.21.15.32:0/547265876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35b01599e0 0x7f35b00c5d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35aeffd640 1 -- 172.21.15.32:0/547265876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35b00b0b30 msgr2=0x7f35b00c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35aeffd640 1 --2- 172.21.15.32:0/547265876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35b00b0b30 0x7f35b00c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35aeffd640 1 -- 172.21.15.32:0/547265876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3598009420 con 0x7f35b00aece0 2023-12-20T04:47:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.394+0000 7f35a7fff640 1 --2- 172.21.15.32:0/547265876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35b00b0b30 0x7f35b00c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.398+0000 7f35aeffd640 1 --2- 172.21.15.32:0/547265876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b00aece0 0x7f35b015c860 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f35a0001370 tx=0x7f35a000c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.398+0000 7f35a77fe640 1 -- 172.21.15.32:0/547265876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a00074e0 con 0x7f35b00aece0 2023-12-20T04:47:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.398+0000 7f35a77fe640 1 -- 172.21.15.32:0/547265876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35a0007660 con 0x7f35b00aece0 2023-12-20T04:47:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.398+0000 7f35a77fe640 1 -- 172.21.15.32:0/547265876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35a00059f0 con 0x7f35b00aece0 2023-12-20T04:47:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.398+0000 7f35b528c640 1 -- 172.21.15.32:0/547265876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35b01f8bd0 con 0x7f35b00aece0 2023-12-20T04:47:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.398+0000 7f35b528c640 1 -- 172.21.15.32:0/547265876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35b01f9190 con 0x7f35b00aece0 2023-12-20T04:47:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.398+0000 7f35b528c640 1 -- 172.21.15.32:0/547265876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35b004e000 con 0x7f35b00aece0 2023-12-20T04:47:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.398+0000 7f35a77fe640 1 -- 172.21.15.32:0/547265876 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 69) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f35a0005b50 con 0x7f35b00aece0 2023-12-20T04:47:07.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.402+0000 7f35a77fe640 1 -- 172.21.15.32:0/547265876 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f35a0082630 con 0x7f35b00aece0 2023-12-20T04:47:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.406+0000 7f35a77fe640 1 -- 172.21.15.32:0/547265876 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f35a004fff0 con 0x7f35b00aece0 2023-12-20T04:47:07.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.614+0000 7f35b528c640 1 -- 172.21.15.32:0/547265876 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f35b00cf550 con 0x7f35b00aece0 2023-12-20T04:47:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.614+0000 7f35a77fe640 1 -- 172.21.15.32:0/547265876 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4705 (secure 0 0 0) 0x7f35a0017020 con 0x7f35b00aece0 2023-12-20T04:47:07.614 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:07.614 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/593676281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":593676281},{"type":"v1","addr":"172.21.15.39:6833","nonce":593676281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2437945959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2437945959},{"type":"v1","addr":"172.21.15.39:6835","nonce":2437945959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7681,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/430674440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":430674440},{"type":"v1","addr":"172.21.15.32:6837","nonce":430674440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:46:00.899089+0000","modified":"2023-12-20T04:46:03.967555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7881":{"gid":7881,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3259359483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3259359483},{"type":"v1","addr":"172.21.15.32:6836","nonce":3259359483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-20T04:47:07.614 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 54 2023-12-20T04:47:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.618+0000 7f35b528c640 1 -- 172.21.15.32:0/547265876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b00aece0 msgr2=0x7f35b015c860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.618+0000 7f35b528c640 1 --2- 172.21.15.32:0/547265876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b00aece0 0x7f35b015c860 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f35a0001370 tx=0x7f35a000c890 comp rx=0 tx=0).stop 2023-12-20T04:47:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.618+0000 7f35b528c640 1 -- 172.21.15.32:0/547265876 shutdown_connections 2023-12-20T04:47:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.618+0000 7f35b528c640 1 --2- 172.21.15.32:0/547265876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35b01599e0 0x7f35b00c5d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.618+0000 7f35b528c640 1 --2- 172.21.15.32:0/547265876 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35b00b0b30 0x7f35b00c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.618+0000 7f35b528c640 1 --2- 172.21.15.32:0/547265876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35b00aece0 0x7f35b015c860 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.618+0000 7f35b528c640 1 -- 172.21.15.32:0/547265876 >> 172.21.15.32:0/547265876 conn(0x7f35b006a320 msgr2=0x7f35b006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.618+0000 7f35b528c640 1 -- 172.21.15.32:0/547265876 shutdown_connections 2023-12-20T04:47:07.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.618+0000 7f35b528c640 1 -- 172.21.15.32:0/547265876 wait complete. 2023-12-20T04:47:07.635 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:47:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.822+0000 7f43256b6640 1 Processor -- start 2023-12-20T04:47:07.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.822+0000 7f43256b6640 1 -- start start 2023-12-20T04:47:07.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.822+0000 7f43256b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f432015cd70 0x7f43200cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.822+0000 7f43256b6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f43200b23d0 0x7f43200b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.822+0000 7f43256b6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43200b2d70 con 0x7f43201608e0 2023-12-20T04:47:07.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.822+0000 7f43256b6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43201618b0 con 0x7f43200b23d0 2023-12-20T04:47:07.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.822+0000 7f43256b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43200ce6c0 con 0x7f432015cd70 2023-12-20T04:47:07.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.826+0000 7f4317fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f43200b23d0 0x7f43200b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.826+0000 7f431f7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f43201608e0 0x7f4320160d70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:60726/0 (socket says 172.21.15.32:60726) 2023-12-20T04:47:07.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.826+0000 7f431f7fe640 1 -- 172.21.15.32:0/1074651150 learned_addr learned my addr 172.21.15.32:0/1074651150 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:07.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.826+0000 7f431effd640 1 --2- 172.21.15.32:0/1074651150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f432015cd70 0x7f43200cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.826+0000 7f431e7fc640 1 -- 172.21.15.32:0/1074651150 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1730932685 0 0) 0x7f43200b2d70 con 0x7f43201608e0 2023-12-20T04:47:07.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.826+0000 7f431e7fc640 1 -- 172.21.15.32:0/1074651150 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f430c003530 con 0x7f43201608e0 2023-12-20T04:47:07.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.826+0000 7f4317fff640 1 -- 172.21.15.32:0/1074651150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f432015cd70 msgr2=0x7f43200cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:07.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.826+0000 7f4317fff640 1 --2- 172.21.15.32:0/1074651150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f432015cd70 0x7f43200cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.826+0000 7f4317fff640 1 -- 172.21.15.32:0/1074651150 >> v1:172.21.15.39:6789/0 conn(0x7f43201608e0 legacy=0x7f4320160d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.826+0000 7f4317fff640 1 -- 172.21.15.32:0/1074651150 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43201615d0 con 0x7f43200b23d0 2023-12-20T04:47:07.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.826+0000 7f431effd640 1 --2- 172.21.15.32:0/1074651150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f432015cd70 0x7f43200cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:07.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f4317fff640 1 --2- 172.21.15.32:0/1074651150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43200b23d0 0x7f43200b2830 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f43080097f0 tx=0x7f430801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=15c0100191b5bce5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:07.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f431e7fc640 1 -- 172.21.15.32:0/1074651150 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43080ab650 con 0x7f43200b23d0 2023-12-20T04:47:07.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f431e7fc640 1 -- 172.21.15.32:0/1074651150 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43080ab7b0 con 0x7f43200b23d0 2023-12-20T04:47:07.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f431e7fc640 1 -- 172.21.15.32:0/1074651150 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43080abb40 con 0x7f43200b23d0 2023-12-20T04:47:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f43256b6640 1 -- 172.21.15.32:0/1074651150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43200b23d0 msgr2=0x7f43200b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f43256b6640 1 --2- 172.21.15.32:0/1074651150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43200b23d0 0x7f43200b2830 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f43080097f0 tx=0x7f430801c450 comp rx=0 tx=0).stop 2023-12-20T04:47:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f43256b6640 1 -- 172.21.15.32:0/1074651150 shutdown_connections 2023-12-20T04:47:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f43256b6640 1 --2- 172.21.15.32:0/1074651150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43200b23d0 0x7f43200b2830 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f43256b6640 1 --2- 172.21.15.32:0/1074651150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f432015cd70 0x7f43200cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f43256b6640 1 -- 172.21.15.32:0/1074651150 >> 172.21.15.32:0/1074651150 conn(0x7f43200cd3d0 msgr2=0x7f43200cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f43256b6640 1 -- 172.21.15.32:0/1074651150 shutdown_connections 2023-12-20T04:47:07.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.830+0000 7f43256b6640 1 -- 172.21.15.32:0/1074651150 wait complete. 2023-12-20T04:47:07.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43256b6640 1 Processor -- start 2023-12-20T04:47:07.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43256b6640 1 -- start start 2023-12-20T04:47:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43256b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43200b23d0 0x7f432015fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43256b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f432015cd70 0x7f43200ca400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43256b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201608e0 0x7f43200b55f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43256b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432015f8f0 con 0x7f43201608e0 2023-12-20T04:47:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43256b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f432015ec70 con 0x7f43200b23d0 2023-12-20T04:47:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43256b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f432015ef50 con 0x7f432015cd70 2023-12-20T04:47:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43200b23d0 0x7f432015fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201608e0 0x7f43200b55f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f4317fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f432015cd70 0x7f43200ca400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43200b23d0 0x7f432015fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:51938/0 (socket says 172.21.15.32:51938) 2023-12-20T04:47:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201608e0 0x7f43200b55f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48918/0 (socket says 172.21.15.32:48918) 2023-12-20T04:47:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431effd640 1 -- 172.21.15.32:0/3479232306 learned_addr learned my addr 172.21.15.32:0/3479232306 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431f7fe640 1 -- 172.21.15.32:0/3479232306 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f432015cd70 msgr2=0x7f43200ca400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431f7fe640 1 --2- 172.21.15.32:0/3479232306 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f432015cd70 0x7f43200ca400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431f7fe640 1 -- 172.21.15.32:0/3479232306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43200b23d0 msgr2=0x7f432015fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431f7fe640 1 --2- 172.21.15.32:0/3479232306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43200b23d0 0x7f432015fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431f7fe640 1 -- 172.21.15.32:0/3479232306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310009420 con 0x7f43201608e0 2023-12-20T04:47:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431effd640 1 --2- 172.21.15.32:0/3479232306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43200b23d0 0x7f432015fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f431f7fe640 1 --2- 172.21.15.32:0/3479232306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201608e0 0x7f43200b55f0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f431800caa0 tx=0x7f431800cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:07.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43177fe640 1 -- 172.21.15.32:0/3479232306 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f431800ab40 con 0x7f43201608e0 2023-12-20T04:47:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43177fe640 1 -- 172.21.15.32:0/3479232306 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f431800aca0 con 0x7f43201608e0 2023-12-20T04:47:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43256b6640 1 -- 172.21.15.32:0/3479232306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4308009420 con 0x7f43201608e0 2023-12-20T04:47:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43177fe640 1 -- 172.21.15.32:0/3479232306 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4318012600 con 0x7f43201608e0 2023-12-20T04:47:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.834+0000 7f43256b6640 1 -- 172.21.15.32:0/3479232306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f432015f510 con 0x7f43201608e0 2023-12-20T04:47:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.838+0000 7f43256b6640 1 -- 172.21.15.32:0/3479232306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f432015fa30 con 0x7f43201608e0 2023-12-20T04:47:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.838+0000 7f43177fe640 1 -- 172.21.15.32:0/3479232306 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 69) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f4318007070 con 0x7f43201608e0 2023-12-20T04:47:07.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.838+0000 7f43177fe640 1 -- 172.21.15.32:0/3479232306 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f4318082d90 con 0x7f43201608e0 2023-12-20T04:47:07.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:07.842+0000 7f43177fe640 1 -- 172.21.15.32:0/3479232306 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f432015fa30 con 0x7f43201608e0 2023-12-20T04:47:08.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.002+0000 7f43256b6640 1 -- 172.21.15.32:0/3479232306 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f432015fa30 con 0x7f43201608e0 2023-12-20T04:47:08.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.002+0000 7f43177fe640 1 -- 172.21.15.32:0/3479232306 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+17051 (secure 0 0 0) 0x7f432015fa30 con 0x7f43201608e0 2023-12-20T04:47:08.002 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:08.002 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":126,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:47:07.259467+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:45:59.452651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-20T04:45:59.882509+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":125,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/807716600":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/80384475":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/2207630031":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/2387196534":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/2603374709":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/2613032803":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/423552018":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/1712648452":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/1806203372":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/878030299":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/3355189708":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/224603579":"2023-12-21T04:46:13.032274+0000","172.21.15.32:6833/38009":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/1563603368":"2023-12-21T04:46:13.032274+0000","172.21.15.32:6832/38009":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/1523085934":"2023-12-21T04:46:13.032274+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:47:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.006+0000 7f43256b6640 1 -- 172.21.15.32:0/3479232306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201608e0 msgr2=0x7f43200b55f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.006+0000 7f43256b6640 1 --2- 172.21.15.32:0/3479232306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201608e0 0x7f43200b55f0 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f431800caa0 tx=0x7f431800cea0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.006+0000 7f43256b6640 1 -- 172.21.15.32:0/3479232306 shutdown_connections 2023-12-20T04:47:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.006+0000 7f43256b6640 1 --2- 172.21.15.32:0/3479232306 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43201608e0 0x7f43200b55f0 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.006+0000 7f43256b6640 1 --2- 172.21.15.32:0/3479232306 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f432015cd70 0x7f43200ca400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.006+0000 7f43256b6640 1 --2- 172.21.15.32:0/3479232306 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43200b23d0 0x7f432015fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.006+0000 7f43256b6640 1 -- 172.21.15.32:0/3479232306 >> 172.21.15.32:0/3479232306 conn(0x7f43200cd3d0 msgr2=0x7f43200cb140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.006+0000 7f43256b6640 1 -- 172.21.15.32:0/3479232306 shutdown_connections 2023-12-20T04:47:08.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.006+0000 7f43256b6640 1 -- 172.21.15.32:0/3479232306 wait complete. 2023-12-20T04:47:08.023 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T04:47:08.023 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:47:08.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd284ae5640 1 Processor -- start 2023-12-20T04:47:08.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd284ae5640 1 -- start start 2023-12-20T04:47:08.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd284ae5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd2800d2740 0x7fd2800cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd284ae5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd2800cfab0 0x7fd2800b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd284ae5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd2800cfec0 con 0x7fd28015cdb0 2023-12-20T04:47:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd284ae5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd28015d190 con 0x7fd2800d2740 2023-12-20T04:47:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd284ae5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd2800d16a0 con 0x7fd2800cfab0 2023-12-20T04:47:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd27e59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd28015cdb0 0x7fd2800d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40806/0 (socket says 172.21.15.32:40806) 2023-12-20T04:47:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd27e59c640 1 -- 172.21.15.32:0/1558404183 learned_addr learned my addr 172.21.15.32:0/1558404183 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd277fff640 1 --2- 172.21.15.32:0/1558404183 >> v2:172.21.15.32:3300/0 conn(0x7fd2800d2740 0x7fd2800cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd27ed9d640 1 --2- 172.21.15.32:0/1558404183 >> v2:172.21.15.39:3300/0 conn(0x7fd2800cfab0 0x7fd2800b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd277fff640 1 -- 172.21.15.32:0/1558404183 >> v2:172.21.15.39:3300/0 conn(0x7fd2800cfab0 msgr2=0x7fd2800b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd277fff640 1 --2- 172.21.15.32:0/1558404183 >> v2:172.21.15.39:3300/0 conn(0x7fd2800cfab0 0x7fd2800b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd277fff640 1 -- 172.21.15.32:0/1558404183 >> v1:172.21.15.32:6789/0 conn(0x7fd28015cdb0 legacy=0x7fd2800d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.170+0000 7fd277fff640 1 -- 172.21.15.32:0/1558404183 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd2800d1980 con 0x7fd2800d2740 2023-12-20T04:47:08.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd277fff640 1 --2- 172.21.15.32:0/1558404183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800d2740 0x7fd2800cf570 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fd28006c100 tx=0x7fd26801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af1a9758b924c9e2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:08.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd27dd9b640 1 -- 172.21.15.32:0/1558404183 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2680a8950 con 0x7fd2800d2740 2023-12-20T04:47:08.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd27dd9b640 1 -- 172.21.15.32:0/1558404183 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd2680a8ab0 con 0x7fd2800d2740 2023-12-20T04:47:08.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd27dd9b640 1 -- 172.21.15.32:0/1558404183 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2680a8e40 con 0x7fd2800d2740 2023-12-20T04:47:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd284ae5640 1 -- 172.21.15.32:0/1558404183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800d2740 msgr2=0x7fd2800cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd284ae5640 1 --2- 172.21.15.32:0/1558404183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800d2740 0x7fd2800cf570 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7fd28006c100 tx=0x7fd26801ec50 comp rx=0 tx=0).stop 2023-12-20T04:47:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd284ae5640 1 -- 172.21.15.32:0/1558404183 shutdown_connections 2023-12-20T04:47:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd284ae5640 1 --2- 172.21.15.32:0/1558404183 >> v2:172.21.15.39:3300/0 conn(0x7fd2800cfab0 0x7fd2800b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd284ae5640 1 --2- 172.21.15.32:0/1558404183 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd2800d2740 0x7fd2800cf570 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd284ae5640 1 -- 172.21.15.32:0/1558404183 >> 172.21.15.32:0/1558404183 conn(0x7fd280065ee0 msgr2=0x7fd28006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd284ae5640 1 -- 172.21.15.32:0/1558404183 shutdown_connections 2023-12-20T04:47:08.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.174+0000 7fd284ae5640 1 -- 172.21.15.32:0/1558404183 wait complete. 2023-12-20T04:47:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd284ae5640 1 Processor -- start 2023-12-20T04:47:08.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd284ae5640 1 -- start start 2023-12-20T04:47:08.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd284ae5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd2800cfab0 0x7fd2801e5470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd284ae5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800d2740 0x7fd28015ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd284ae5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd28015cdb0 0x7fd28015f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd284ae5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28014f610 con 0x7fd28015cdb0 2023-12-20T04:47:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd284ae5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd28015fb90 con 0x7fd2800d2740 2023-12-20T04:47:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd284ae5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd28015fe70 con 0x7fd2800cfab0 2023-12-20T04:47:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd27ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd28015cdb0 0x7fd28015f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd27ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd28015cdb0 0x7fd28015f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48936/0 (socket says 172.21.15.32:48936) 2023-12-20T04:47:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd27ed9d640 1 -- 172.21.15.32:0/2635666851 learned_addr learned my addr 172.21.15.32:0/2635666851 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd277fff640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800d2740 0x7fd28015ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd27e59c640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd2800cfab0 0x7fd2801e5470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd27ed9d640 1 -- 172.21.15.32:0/2635666851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd2800cfab0 msgr2=0x7fd2801e5470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd27ed9d640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd2800cfab0 0x7fd2801e5470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd27ed9d640 1 -- 172.21.15.32:0/2635666851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800d2740 msgr2=0x7fd28015ef50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd27ed9d640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800d2740 0x7fd28015ef50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd27ed9d640 1 -- 172.21.15.32:0/2635666851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd268009420 con 0x7fd28015cdb0 2023-12-20T04:47:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd277fff640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800d2740 0x7fd28015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd27ed9d640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd28015cdb0 0x7fd28015f490 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd27800efc0 tx=0x7fd27800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd276ffd640 1 -- 172.21.15.32:0/2635666851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278014480 con 0x7fd28015cdb0 2023-12-20T04:47:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd276ffd640 1 -- 172.21.15.32:0/2635666851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd27800f040 con 0x7fd28015cdb0 2023-12-20T04:47:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd284ae5640 1 -- 172.21.15.32:0/2635666851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd280150a20 con 0x7fd28015cdb0 2023-12-20T04:47:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd276ffd640 1 -- 172.21.15.32:0/2635666851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd278014820 con 0x7fd28015cdb0 2023-12-20T04:47:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.178+0000 7fd284ae5640 1 -- 172.21.15.32:0/2635666851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd280150fe0 con 0x7fd28015cdb0 2023-12-20T04:47:08.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.182+0000 7fd284ae5640 1 -- 172.21.15.32:0/2635666851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd2801511d0 con 0x7fd28015cdb0 2023-12-20T04:47:08.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.182+0000 7fd276ffd640 1 -- 172.21.15.32:0/2635666851 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 69) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fd278014980 con 0x7fd28015cdb0 2023-12-20T04:47:08.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.182+0000 7fd276ffd640 1 -- 172.21.15.32:0/2635666851 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fd278083a70 con 0x7fd28015cdb0 2023-12-20T04:47:08.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.186+0000 7fd276ffd640 1 -- 172.21.15.32:0/2635666851 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd278051570 con 0x7fd28015cdb0 2023-12-20T04:47:08.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.270+0000 7fd276ffd640 1 -- 172.21.15.32:0/2635666851 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 70) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fd278051020 con 0x7fd28015cdb0 2023-12-20T04:47:08.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.270+0000 7fd276ffd640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd26001c9b0 0x7fd26006d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.270+0000 7fd27e59c640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd26001c9b0 0x7fd26006d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.278+0000 7fd27e59c640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd26001c9b0 0x7fd26006d1f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd270005890 tx=0x7fd270009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:08.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 -- 172.21.15.32:0/2635666851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fd28004e000 con 0x7fd28015cdb0 2023-12-20T04:47:08.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd276ffd640 1 -- 172.21.15.32:0/2635666851 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4705 (secure 0 0 0) 0x7fd2780815a0 con 0x7fd28015cdb0 2023-12-20T04:47:08.384 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:08.384 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/593676281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":593676281},{"type":"v1","addr":"172.21.15.39:6833","nonce":593676281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2437945959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2437945959},{"type":"v1","addr":"172.21.15.39:6835","nonce":2437945959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7681,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/430674440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":430674440},{"type":"v1","addr":"172.21.15.32:6837","nonce":430674440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:46:00.899089+0000","modified":"2023-12-20T04:46:03.967555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7881":{"gid":7881,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3259359483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3259359483},{"type":"v1","addr":"172.21.15.32:6836","nonce":3259359483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-20T04:47:08.385 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 54 2023-12-20T04:47:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 -- 172.21.15.32:0/2635666851 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd26001c9b0 msgr2=0x7fd26006d1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd26001c9b0 0x7fd26006d1f0 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fd270005890 tx=0x7fd270009040 comp rx=0 tx=0).stop 2023-12-20T04:47:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 -- 172.21.15.32:0/2635666851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd28015cdb0 msgr2=0x7fd28015f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd28015cdb0 0x7fd28015f490 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7fd27800efc0 tx=0x7fd27800c490 comp rx=0 tx=0).stop 2023-12-20T04:47:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 -- 172.21.15.32:0/2635666851 shutdown_connections 2023-12-20T04:47:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd26001c9b0 0x7fd26006d1f0 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd28015cdb0 0x7fd28015f490 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2800d2740 0x7fd28015ef50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 --2- 172.21.15.32:0/2635666851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd2800cfab0 0x7fd2801e5470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 -- 172.21.15.32:0/2635666851 >> 172.21.15.32:0/2635666851 conn(0x7fd280065ee0 msgr2=0x7fd28015a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 -- 172.21.15.32:0/2635666851 shutdown_connections 2023-12-20T04:47:08.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.386+0000 7fd284ae5640 1 -- 172.21.15.32:0/2635666851 wait complete. 2023-12-20T04:47:08.405 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:47:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fd2895640 1 Processor -- start 2023-12-20T04:47:08.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fd2895640 1 -- start start 2023-12-20T04:47:08.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fd2895640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc15cdb0 0x7f8fcc0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fd2895640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc15d190 con 0x7f8fcc0d2740 2023-12-20T04:47:08.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fd2895640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fcc0d16a0 con 0x7f8fcc0cfc80 2023-12-20T04:47:08.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fd2895640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0d1980 con 0x7f8fcc15cdb0 2023-12-20T04:47:08.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fcb7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8fcc0d2740 0x7f8fcc0cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40816/0 (socket says 172.21.15.32:40816) 2023-12-20T04:47:08.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fcb7fe640 1 -- 172.21.15.32:0/2315694748 learned_addr learned my addr 172.21.15.32:0/2315694748 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:08.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fcbfff640 1 --2- 172.21.15.32:0/2315694748 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc15cdb0 0x7f8fcc0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fcaffd640 1 -- 172.21.15.32:0/2315694748 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1164723711 0 0) 0x7f8fcc15d190 con 0x7f8fcc0d2740 2023-12-20T04:47:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fcaffd640 1 -- 172.21.15.32:0/2315694748 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb4003530 con 0x7f8fcc0d2740 2023-12-20T04:47:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fcaffd640 1 -- 172.21.15.32:0/2315694748 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1074580914 0 0) 0x7f8fcc0d16a0 con 0x7f8fcc0cfc80 2023-12-20T04:47:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fcbfff640 1 -- 172.21.15.32:0/2315694748 >> v1:172.21.15.39:6789/0 conn(0x7f8fcc0cfc80 legacy=0x7f8fcc0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fcbfff640 1 -- 172.21.15.32:0/2315694748 >> v1:172.21.15.32:6789/0 conn(0x7f8fcc0d2740 legacy=0x7f8fcc0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.550+0000 7f8fcbfff640 1 -- 172.21.15.32:0/2315694748 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fcc0d1c60 con 0x7f8fcc15cdb0 2023-12-20T04:47:08.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.554+0000 7f8fcbfff640 1 --2- 172.21.15.32:0/2315694748 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc15cdb0 0x7f8fcc0d0be0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0097f0 tx=0x7f8fbc01ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bbe705ec08d24826 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:08.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.554+0000 7f8fcaffd640 1 -- 172.21.15.32:0/2315694748 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0a89b0 con 0x7f8fcc15cdb0 2023-12-20T04:47:08.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.554+0000 7f8fcaffd640 1 -- 172.21.15.32:0/2315694748 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fbc0a8b10 con 0x7f8fcc15cdb0 2023-12-20T04:47:08.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.554+0000 7f8fcaffd640 1 -- 172.21.15.32:0/2315694748 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fbc0a8e80 con 0x7f8fcc15cdb0 2023-12-20T04:47:08.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.554+0000 7f8fd2895640 1 -- 172.21.15.32:0/2315694748 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc15cdb0 msgr2=0x7f8fcc0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.554+0000 7f8fd2895640 1 --2- 172.21.15.32:0/2315694748 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc15cdb0 0x7f8fcc0d0be0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0097f0 tx=0x7f8fbc01ecb0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 -- 172.21.15.32:0/2315694748 shutdown_connections 2023-12-20T04:47:08.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 --2- 172.21.15.32:0/2315694748 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc15cdb0 0x7f8fcc0d0be0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 -- 172.21.15.32:0/2315694748 >> 172.21.15.32:0/2315694748 conn(0x7f8fcc065ee0 msgr2=0x7f8fcc06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:08.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 -- 172.21.15.32:0/2315694748 shutdown_connections 2023-12-20T04:47:08.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 -- 172.21.15.32:0/2315694748 wait complete. 2023-12-20T04:47:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 Processor -- start 2023-12-20T04:47:08.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 -- start start 2023-12-20T04:47:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc0cfc80 0x7f8fcc1f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc0d2740 0x7f8fcc0c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fcc15cdb0 0x7f8fcc1587a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0cb290 con 0x7f8fcc0d2740 2023-12-20T04:47:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0cc8a0 con 0x7f8fcc15cdb0 2023-12-20T04:47:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd2895640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fcc0ca610 con 0x7f8fcc0cfc80 2023-12-20T04:47:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fcb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc0d2740 0x7f8fcc0c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fcb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc0d2740 0x7f8fcc0c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:48948/0 (socket says 172.21.15.32:48948) 2023-12-20T04:47:08.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fcb7fe640 1 -- 172.21.15.32:0/1313901491 learned_addr learned my addr 172.21.15.32:0/1313901491 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:08.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fcb7fe640 1 -- 172.21.15.32:0/1313901491 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc0cfc80 msgr2=0x7f8fcc1f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fcbfff640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc0cfc80 0x7f8fcc1f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd0e32640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fcc15cdb0 0x7f8fcc1587a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fcb7fe640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc0cfc80 0x7f8fcc1f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fcb7fe640 1 -- 172.21.15.32:0/1313901491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fcc15cdb0 msgr2=0x7f8fcc1587a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fcb7fe640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fcc15cdb0 0x7f8fcc1587a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fcb7fe640 1 -- 172.21.15.32:0/1313901491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fbc009420 con 0x7f8fcc0d2740 2023-12-20T04:47:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fcbfff640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc0cfc80 0x7f8fcc1f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.558+0000 7f8fd0e32640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fcc15cdb0 0x7f8fcc1587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fcb7fe640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc0d2740 0x7f8fcc0c9c30 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c450 tx=0x7f8fc000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fc8ff9640 1 -- 172.21.15.32:0/1313901491 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0005cf0 con 0x7f8fcc0d2740 2023-12-20T04:47:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fc8ff9640 1 -- 172.21.15.32:0/1313901491 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc0005e50 con 0x7f8fcc0d2740 2023-12-20T04:47:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fc8ff9640 1 -- 172.21.15.32:0/1313901491 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc0007290 con 0x7f8fcc0d2740 2023-12-20T04:47:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fd2895640 1 -- 172.21.15.32:0/1313901491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fcc0ca8f0 con 0x7f8fcc0d2740 2023-12-20T04:47:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fd2895640 1 -- 172.21.15.32:0/1313901491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fcc0caeb0 con 0x7f8fcc0d2740 2023-12-20T04:47:08.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fd2895640 1 -- 172.21.15.32:0/1313901491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fcc04e000 con 0x7f8fcc0d2740 2023-12-20T04:47:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fc8ff9640 1 -- 172.21.15.32:0/1313901491 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 70) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f8fc00073f0 con 0x7f8fcc0d2740 2023-12-20T04:47:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fc8ff9640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fb80658b0 0x7f8fb8067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fc8ff9640 1 -- 172.21.15.32:0/1313901491 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f8fc0082740 con 0x7f8fcc0d2740 2023-12-20T04:47:08.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fcbfff640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fb80658b0 0x7f8fb8067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.562+0000 7f8fcbfff640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fb80658b0 0x7f8fb8067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0b2000 tx=0x7f8fbc0aa070 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:08.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.566+0000 7f8fc8ff9640 1 -- 172.21.15.32:0/1313901491 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8fc00500a0 con 0x7f8fcc0d2740 2023-12-20T04:47:08.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.758+0000 7f8fd2895640 1 -- 172.21.15.32:0/1313901491 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8fcc06b8d0 con 0x7f8fcc0d2740 2023-12-20T04:47:08.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.758+0000 7f8fc8ff9640 1 -- 172.21.15.32:0/1313901491 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 54 v54) v1 ==== 94+0+4705 (secure 0 0 0) 0x7f8fc0017020 con 0x7f8fcc0d2740 2023-12-20T04:47:08.757 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:08.757 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":54,"default_fscid":7,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/593676281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":593676281},{"type":"v1","addr":"172.21.15.39:6833","nonce":593676281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2437945959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2437945959},{"type":"v1","addr":"172.21.15.39:6835","nonce":2437945959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7681,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/430674440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":430674440},{"type":"v1","addr":"172.21.15.32:6837","nonce":430674440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49}],"filesystems":[{"mdsmap":{"epoch":54,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:46:00.899089+0000","modified":"2023-12-20T04:46:03.967555+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":7881},"failed":[],"damaged":[],"stopped":[],"info":{"gid_7881":{"gid":7881,"name":"a","rank":0,"incarnation":50,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6836/3259359483","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3259359483},{"type":"v1","addr":"172.21.15.32:6836","nonce":3259359483}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[15],"metadata_pool":14,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":7}]} 2023-12-20T04:47:08.758 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 54 2023-12-20T04:47:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 -- 172.21.15.32:0/1313901491 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fb80658b0 msgr2=0x7f8fb8067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fb80658b0 0x7f8fb8067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f8fbc0b2000 tx=0x7f8fbc0aa070 comp rx=0 tx=0).stop 2023-12-20T04:47:08.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 -- 172.21.15.32:0/1313901491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc0d2740 msgr2=0x7f8fcc0c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc0d2740 0x7f8fcc0c9c30 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f8fc000c450 tx=0x7f8fc000c910 comp rx=0 tx=0).stop 2023-12-20T04:47:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 -- 172.21.15.32:0/1313901491 shutdown_connections 2023-12-20T04:47:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fb80658b0 0x7f8fb8067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fcc15cdb0 0x7f8fcc1587a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fcc0d2740 0x7f8fcc0c9c30 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 --2- 172.21.15.32:0/1313901491 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fcc0cfc80 0x7f8fcc1f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 -- 172.21.15.32:0/1313901491 >> 172.21.15.32:0/1313901491 conn(0x7f8fcc065ee0 msgr2=0x7f8fcc159c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 -- 172.21.15.32:0/1313901491 shutdown_connections 2023-12-20T04:47:08.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.762+0000 7f8fd2895640 1 -- 172.21.15.32:0/1313901491 wait complete. 2023-12-20T04:47:08.779 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:47:08.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.922+0000 7fc4c61f0640 1 Processor -- start 2023-12-20T04:47:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.922+0000 7fc4c61f0640 1 -- start start 2023-12-20T04:47:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.922+0000 7fc4c61f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c006ce60 0x7fc4c015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.922+0000 7fc4c61f0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc4c015f110 0x7fc4c015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.922+0000 7fc4c61f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c015bbd0 con 0x7fc4c00af5e0 2023-12-20T04:47:08.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.922+0000 7fc4c61f0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c015beb0 con 0x7fc4c015f110 2023-12-20T04:47:08.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.922+0000 7fc4c61f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c015e910 con 0x7fc4c006ce60 2023-12-20T04:47:08.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.926+0000 7fc4beffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c006ce60 0x7fc4c015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.926+0000 7fc4bffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc4c015f110 0x7fc4c015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.926+0000 7fc4bf7fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc4c00af5e0 0x7fc4c00af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:40790/0 (socket says 172.21.15.32:40790) 2023-12-20T04:47:08.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.926+0000 7fc4bf7fe640 1 -- 172.21.15.32:0/1712482762 learned_addr learned my addr 172.21.15.32:0/1712482762 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:08.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.926+0000 7fc4beffd640 1 -- 172.21.15.32:0/1712482762 >> v2:172.21.15.39:3300/0 conn(0x7fc4c015f110 msgr2=0x7fc4c015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:08.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.926+0000 7fc4beffd640 1 --2- 172.21.15.32:0/1712482762 >> v2:172.21.15.39:3300/0 conn(0x7fc4c015f110 0x7fc4c015c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.926+0000 7fc4beffd640 1 -- 172.21.15.32:0/1712482762 >> v1:172.21.15.39:6789/0 conn(0x7fc4c00af5e0 legacy=0x7fc4c00af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.926+0000 7fc4beffd640 1 -- 172.21.15.32:0/1712482762 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c015ebf0 con 0x7fc4c006ce60 2023-12-20T04:47:08.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.926+0000 7fc4be7fc640 1 -- 172.21.15.32:0/1712482762 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1383695035 0 0) 0x7fc4c015bbd0 con 0x7fc4c00af5e0 2023-12-20T04:47:08.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4beffd640 1 --2- 172.21.15.32:0/1712482762 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c006ce60 0x7fc4c015b280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40097c0 tx=0x7fc4b401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=6ac20a38a7539395 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:08.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4be7fc640 1 -- 172.21.15.32:0/1712482762 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4c015b9d0 con 0x7fc4c006ce60 2023-12-20T04:47:08.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4be7fc640 1 -- 172.21.15.32:0/1712482762 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b40ad640 con 0x7fc4c006ce60 2023-12-20T04:47:08.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4be7fc640 1 -- 172.21.15.32:0/1712482762 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40ad9d0 con 0x7fc4c006ce60 2023-12-20T04:47:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1712482762 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c006ce60 msgr2=0x7fc4c015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 --2- 172.21.15.32:0/1712482762 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c006ce60 0x7fc4c015b280 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fc4b40097c0 tx=0x7fc4b401c3f0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1712482762 shutdown_connections 2023-12-20T04:47:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 --2- 172.21.15.32:0/1712482762 >> v2:172.21.15.39:3300/0 conn(0x7fc4c015f110 0x7fc4c015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 --2- 172.21.15.32:0/1712482762 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c006ce60 0x7fc4c015b280 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1712482762 >> 172.21.15.32:0/1712482762 conn(0x7fc4c006a320 msgr2=0x7fc4c00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1712482762 shutdown_connections 2023-12-20T04:47:08.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1712482762 wait complete. 2023-12-20T04:47:08.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 Processor -- start 2023-12-20T04:47:08.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 -- start start 2023-12-20T04:47:08.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c006ce60 0x7fc4c00c7f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c00af5e0 0x7fc4c0157630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015f110 0x7fc4c00c7790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0152360 con 0x7fc4c015f110 2023-12-20T04:47:08.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0157b70 con 0x7fc4c00af5e0 2023-12-20T04:47:08.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.930+0000 7fc4c61f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c0151a00 con 0x7fc4c006ce60 2023-12-20T04:47:08.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015f110 0x7fc4c00c7790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4bffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015f110 0x7fc4c00c7790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56864/0 (socket says 172.21.15.32:56864) 2023-12-20T04:47:08.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4bffff640 1 -- 172.21.15.32:0/1766932078 learned_addr learned my addr 172.21.15.32:0/1766932078 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:08.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4beffd640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c00af5e0 0x7fc4c0157630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4bf7fe640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c006ce60 0x7fc4c00c7f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4bffff640 1 -- 172.21.15.32:0/1766932078 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c006ce60 msgr2=0x7fc4c00c7f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4bffff640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c006ce60 0x7fc4c00c7f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4bffff640 1 -- 172.21.15.32:0/1766932078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c00af5e0 msgr2=0x7fc4c0157630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:08.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4bffff640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c00af5e0 0x7fc4c0157630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:08.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4bffff640 1 -- 172.21.15.32:0/1766932078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b4009420 con 0x7fc4c015f110 2023-12-20T04:47:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4bffff640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015f110 0x7fc4c00c7790 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800efc0 tx=0x7fc4b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4a7fff640 1 -- 172.21.15.32:0/1766932078 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b8009ba0 con 0x7fc4c015f110 2023-12-20T04:47:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4a7fff640 1 -- 172.21.15.32:0/1766932078 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b800f040 con 0x7fc4c015f110 2023-12-20T04:47:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1766932078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c0151ca0 con 0x7fc4c015f110 2023-12-20T04:47:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4a7fff640 1 -- 172.21.15.32:0/1766932078 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b80146c0 con 0x7fc4c015f110 2023-12-20T04:47:08.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1766932078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c00c8600 con 0x7fc4c015f110 2023-12-20T04:47:08.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1766932078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c00c5830 con 0x7fc4c015f110 2023-12-20T04:47:08.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.934+0000 7fc4a7fff640 1 -- 172.21.15.32:0/1766932078 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 70) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fc4b8014820 con 0x7fc4c015f110 2023-12-20T04:47:08.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.938+0000 7fc4a7fff640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4ac0658b0 0x7fc4ac067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:08.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.938+0000 7fc4a7fff640 1 -- 172.21.15.32:0/1766932078 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fc4b8083ac0 con 0x7fc4c015f110 2023-12-20T04:47:08.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.938+0000 7fc4bf7fe640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4ac0658b0 0x7fc4ac067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:08.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.938+0000 7fc4bf7fe640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4ac0658b0 0x7fc4ac067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0000fc0 tx=0x7fc4b0007d30 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:08.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:08.942+0000 7fc4a7fff640 1 -- 172.21.15.32:0/1766932078 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc4c00c5830 con 0x7fc4c015f110 2023-12-20T04:47:09.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.098+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1766932078 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4c00c5830 con 0x7fc4c015f110 2023-12-20T04:47:09.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.098+0000 7fc4a7fff640 1 -- 172.21.15.32:0/1766932078 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v126) v1 ==== 74+0+17051 (secure 0 0 0) 0x7fc4c00c5830 con 0x7fc4c015f110 2023-12-20T04:47:09.098 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:09.098 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":126,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:47:07.259467+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":14,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:45:59.452651+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"122","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":15,"pool_name":"cephfs_data","create_time":"2023-12-20T04:45:59.882509+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"125","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":3,"snap_epoch":125,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/807716600":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/80384475":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/2207630031":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/2387196534":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/2603374709":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/2613032803":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/423552018":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/1712648452":"2023-12-21T04:47:07.259432+0000","172.21.15.32:0/1806203372":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/878030299":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/3355189708":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/224603579":"2023-12-21T04:46:13.032274+0000","172.21.15.32:6833/38009":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/1563603368":"2023-12-21T04:46:13.032274+0000","172.21.15.32:6832/38009":"2023-12-21T04:46:13.032274+0000","172.21.15.32:0/1523085934":"2023-12-21T04:46:13.032274+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:47:09.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1766932078 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4ac0658b0 msgr2=0x7fc4ac067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4ac0658b0 0x7fc4ac067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fc4b0000fc0 tx=0x7fc4b0007d30 comp rx=0 tx=0).stop 2023-12-20T04:47:09.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1766932078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015f110 msgr2=0x7fc4c00c7790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015f110 0x7fc4c00c7790 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7fc4b800efc0 tx=0x7fc4b800c490 comp rx=0 tx=0).stop 2023-12-20T04:47:09.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1766932078 shutdown_connections 2023-12-20T04:47:09.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4ac0658b0 0x7fc4ac067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c015f110 0x7fc4c00c7790 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c00af5e0 0x7fc4c0157630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 --2- 172.21.15.32:0/1766932078 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c006ce60 0x7fc4c00c7f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1766932078 >> 172.21.15.32:0/1766932078 conn(0x7fc4c006a320 msgr2=0x7fc4c015a0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1766932078 shutdown_connections 2023-12-20T04:47:09.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.102+0000 7fc4c61f0640 1 -- 172.21.15.32:0/1766932078 wait complete. 2023-12-20T04:47:09.120 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T04:47:09.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f253d263640 1 Processor -- start 2023-12-20T04:47:09.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f253d263640 1 -- start start 2023-12-20T04:47:09.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f253d263640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f253815bd80 0x7f253806a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f253d263640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25380af780 con 0x7f25381586a0 2023-12-20T04:47:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f253d263640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2538158a80 con 0x7f25380ccd80 2023-12-20T04:47:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f253d263640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25380af2b0 con 0x7f253815bd80 2023-12-20T04:47:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f2536ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f25381586a0 0x7f25380aeba0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57034/0 (socket says 172.21.15.32:57034) 2023-12-20T04:47:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f2536ffd640 1 -- 172.21.15.32:0/1105657464 learned_addr learned my addr 172.21.15.32:0/1105657464 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:09.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f25367fc640 1 --2- 172.21.15.32:0/1105657464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f253815bd80 0x7f253806a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:09.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f25367fc640 1 -- 172.21.15.32:0/1105657464 >> v1:172.21.15.39:6789/0 conn(0x7f25380ccd80 legacy=0x7f253815e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f25367fc640 1 -- 172.21.15.32:0/1105657464 >> v1:172.21.15.32:6789/0 conn(0x7f25381586a0 legacy=0x7f25380aeba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.270+0000 7f25367fc640 1 -- 172.21.15.32:0/1105657464 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f25380af4a0 con 0x7f253815bd80 2023-12-20T04:47:09.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.278+0000 7f25367fc640 1 --2- 172.21.15.32:0/1105657464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f253815bd80 0x7f253806a7c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f252c0097f0 tx=0x7f252c01c490 comp rx=0 tx=0).ready entity=mon.2 client_cookie=277136f7d5db35cc server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:09.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.278+0000 7f2535ffb640 1 -- 172.21.15.32:0/1105657464 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c0aa5f0 con 0x7f253815bd80 2023-12-20T04:47:09.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.278+0000 7f2535ffb640 1 -- 172.21.15.32:0/1105657464 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f252c0aa750 con 0x7f253815bd80 2023-12-20T04:47:09.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.278+0000 7f2535ffb640 1 -- 172.21.15.32:0/1105657464 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f252c0aaae0 con 0x7f253815bd80 2023-12-20T04:47:09.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.278+0000 7f253d263640 1 -- 172.21.15.32:0/1105657464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f253815bd80 msgr2=0x7f253806a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.278+0000 7f253d263640 1 --2- 172.21.15.32:0/1105657464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f253815bd80 0x7f253806a7c0 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f252c0097f0 tx=0x7f252c01c490 comp rx=0 tx=0).stop 2023-12-20T04:47:09.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.278+0000 7f253d263640 1 -- 172.21.15.32:0/1105657464 shutdown_connections 2023-12-20T04:47:09.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.278+0000 7f253d263640 1 --2- 172.21.15.32:0/1105657464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f253815bd80 0x7f253806a7c0 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.278+0000 7f253d263640 1 -- 172.21.15.32:0/1105657464 >> 172.21.15.32:0/1105657464 conn(0x7f2538071960 msgr2=0x7f2538071d70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:09.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.278+0000 7f253d263640 1 -- 172.21.15.32:0/1105657464 shutdown_connections 2023-12-20T04:47:09.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.278+0000 7f253d263640 1 -- 172.21.15.32:0/1105657464 wait complete. 2023-12-20T04:47:09.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f253d263640 1 Processor -- start 2023-12-20T04:47:09.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f253d263640 1 -- start start 2023-12-20T04:47:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f253d263640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f25380ccd80 0x7f2538149cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:09.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f253d263640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25381586a0 0x7f2538154130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f253d263640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253815bd80 0x7f2538154670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f253d263640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253814a1f0 con 0x7f253815bd80 2023-12-20T04:47:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f253d263640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f253815c230 con 0x7f25381586a0 2023-12-20T04:47:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f253d263640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f253815cdb0 con 0x7f25380ccd80 2023-12-20T04:47:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f2537fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253815bd80 0x7f2538154670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f2536ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f25380ccd80 0x7f2538149cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f25367fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25381586a0 0x7f2538154130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f2537fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253815bd80 0x7f2538154670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56868/0 (socket says 172.21.15.32:56868) 2023-12-20T04:47:09.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f2537fff640 1 -- 172.21.15.32:0/3495923816 learned_addr learned my addr 172.21.15.32:0/3495923816 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f2537fff640 1 -- 172.21.15.32:0/3495923816 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f25380ccd80 msgr2=0x7f2538149cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f2537fff640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f25380ccd80 0x7f2538149cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f2537fff640 1 -- 172.21.15.32:0/3495923816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25381586a0 msgr2=0x7f2538154130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f2537fff640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25381586a0 0x7f2538154130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f2537fff640 1 -- 172.21.15.32:0/3495923816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f252c009420 con 0x7f253815bd80 2023-12-20T04:47:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f2536ffd640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f25380ccd80 0x7f2538149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f2537fff640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253815bd80 0x7f2538154670 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f253000c6f0 tx=0x7f253000cbb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f251f7fe640 1 -- 172.21.15.32:0/3495923816 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530003980 con 0x7f253815bd80 2023-12-20T04:47:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f251f7fe640 1 -- 172.21.15.32:0/3495923816 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2530003ae0 con 0x7f253815bd80 2023-12-20T04:47:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f251f7fe640 1 -- 172.21.15.32:0/3495923816 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2530003e90 con 0x7f253815bd80 2023-12-20T04:47:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f253d263640 1 -- 172.21.15.32:0/3495923816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f253815c480 con 0x7f253815bd80 2023-12-20T04:47:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.282+0000 7f253d263640 1 -- 172.21.15.32:0/3495923816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f253815ca40 con 0x7f253815bd80 2023-12-20T04:47:09.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.286+0000 7f253d263640 1 -- 172.21.15.32:0/3495923816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f253804e000 con 0x7f253815bd80 2023-12-20T04:47:09.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.286+0000 7f251f7fe640 1 -- 172.21.15.32:0/3495923816 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 71) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f253000a850 con 0x7f253815bd80 2023-12-20T04:47:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.286+0000 7f251f7fe640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2524065af0 0x7f2524067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.286+0000 7f251f7fe640 1 -- 172.21.15.32:0/3495923816 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(126..126 src has 1..126) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f25300830f0 con 0x7f253815bd80 2023-12-20T04:47:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.286+0000 7f2536ffd640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2524065af0 0x7f2524067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:09.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.286+0000 7f2536ffd640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2524065af0 0x7f2524067fb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2528005890 tx=0x7f2528009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:09.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.290+0000 7f251f7fe640 1 -- 172.21.15.32:0/3495923816 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2530050830 con 0x7f253815bd80 2023-12-20T04:47:09.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.486+0000 7f253d263640 1 -- 172.21.15.32:0/3495923816 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f253815cef0 con 0x7f253815bd80 2023-12-20T04:47:09.487 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T04:47:09.490+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T04:47:09.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.498+0000 7f251f7fe640 1 -- 172.21.15.32:0/3495923816 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v55) v1 ==== 161+0+0 (secure 0 0 0) 0x7f2530053e50 con 0x7f253815bd80 2023-12-20T04:47:09.496 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T04:47:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 -- 172.21.15.32:0/3495923816 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2524065af0 msgr2=0x7f2524067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2524065af0 0x7f2524067fb0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f2528005890 tx=0x7f2528009040 comp rx=0 tx=0).stop 2023-12-20T04:47:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 -- 172.21.15.32:0/3495923816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253815bd80 msgr2=0x7f2538154670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253815bd80 0x7f2538154670 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f253000c6f0 tx=0x7f253000cbb0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 -- 172.21.15.32:0/3495923816 shutdown_connections 2023-12-20T04:47:09.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2524065af0 0x7f2524067fb0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f253815bd80 0x7f2538154670 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25381586a0 0x7f2538154130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 --2- 172.21.15.32:0/3495923816 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f25380ccd80 0x7f2538149cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 -- 172.21.15.32:0/3495923816 >> 172.21.15.32:0/3495923816 conn(0x7f2538071960 msgr2=0x7f2538071d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 -- 172.21.15.32:0/3495923816 shutdown_connections 2023-12-20T04:47:09.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.502+0000 7f253d263640 1 -- 172.21.15.32:0/3495923816 wait complete. 2023-12-20T04:47:09.507 INFO:tasks.ceph.mds.a.smithi032.stderr: -9666> 2023-12-20T04:46:02.475+0000 7f07c939d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:47:09.519 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T04:47:09.561 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T04:47:09.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.666+0000 7f52d8d0a640 1 Processor -- start 2023-12-20T04:47:09.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.666+0000 7f52d8d0a640 1 -- start start 2023-12-20T04:47:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.670+0000 7f52d8d0a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52d415abb0 0x7f52d40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.670+0000 7f52d8d0a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d415b6a0 con 0x7f52d4070230 2023-12-20T04:47:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.670+0000 7f52d8d0a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52d406d8c0 con 0x7f52d40b0cb0 2023-12-20T04:47:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.670+0000 7f52d8d0a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52d406e160 con 0x7f52d415abb0 2023-12-20T04:47:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.670+0000 7f52d259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f52d415abb0 0x7f52d40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.670+0000 7f52d2d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f52d4070230 0x7f52d406c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57054/0 (socket says 172.21.15.32:57054) 2023-12-20T04:47:09.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.670+0000 7f52d2d9d640 1 -- 172.21.15.32:0/591374340 learned_addr learned my addr 172.21.15.32:0/591374340 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.670+0000 7f52d259c640 1 -- 172.21.15.32:0/591374340 >> v1:172.21.15.39:6789/0 conn(0x7f52d40b0cb0 legacy=0x7f52d415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.670+0000 7f52d259c640 1 -- 172.21.15.32:0/591374340 >> v1:172.21.15.32:6789/0 conn(0x7f52d4070230 legacy=0x7f52d406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.670+0000 7f52d259c640 1 -- 172.21.15.32:0/591374340 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52d406dba0 con 0x7f52d415abb0 2023-12-20T04:47:09.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d259c640 1 --2- 172.21.15.32:0/591374340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d415abb0 0x7f52d40caaf0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f52c40097c0 tx=0x7f52c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8739846c0565e178 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:09.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d1d9b640 1 -- 172.21.15.32:0/591374340 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40ab680 con 0x7f52d415abb0 2023-12-20T04:47:09.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d1d9b640 1 -- 172.21.15.32:0/591374340 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c40ab7e0 con 0x7f52d415abb0 2023-12-20T04:47:09.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d1d9b640 1 -- 172.21.15.32:0/591374340 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40abb70 con 0x7f52d415abb0 2023-12-20T04:47:09.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 -- 172.21.15.32:0/591374340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d415abb0 msgr2=0x7f52d40caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 --2- 172.21.15.32:0/591374340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d415abb0 0x7f52d40caaf0 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f52c40097c0 tx=0x7f52c401c480 comp rx=0 tx=0).stop 2023-12-20T04:47:09.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 -- 172.21.15.32:0/591374340 shutdown_connections 2023-12-20T04:47:09.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 --2- 172.21.15.32:0/591374340 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d415abb0 0x7f52d40caaf0 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 -- 172.21.15.32:0/591374340 >> 172.21.15.32:0/591374340 conn(0x7f52d406a510 msgr2=0x7f52d40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:09.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 -- 172.21.15.32:0/591374340 shutdown_connections 2023-12-20T04:47:09.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 -- 172.21.15.32:0/591374340 wait complete. 2023-12-20T04:47:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 Processor -- start 2023-12-20T04:47:09.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 -- start start 2023-12-20T04:47:09.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d4070230 0x7f52d4147f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52d40b0cb0 0x7f52d40b3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d415abb0 0x7f52d414def0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d414a1c0 con 0x7f52d4070230 2023-12-20T04:47:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52d41487a0 con 0x7f52d415abb0 2023-12-20T04:47:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.674+0000 7f52d8d0a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52d414a3b0 con 0x7f52d40b0cb0 2023-12-20T04:47:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d4070230 0x7f52d4147f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d4070230 0x7f52d4147f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56920/0 (socket says 172.21.15.32:56920) 2023-12-20T04:47:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d259c640 1 -- 172.21.15.32:0/3161677465 learned_addr learned my addr 172.21.15.32:0/3161677465 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d2d9d640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d415abb0 0x7f52d414def0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:09.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52c9d9b640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52d40b0cb0 0x7f52d40b3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d259c640 1 -- 172.21.15.32:0/3161677465 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52d40b0cb0 msgr2=0x7f52d40b3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d259c640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52d40b0cb0 0x7f52d40b3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d259c640 1 -- 172.21.15.32:0/3161677465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d415abb0 msgr2=0x7f52d414def0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d259c640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d415abb0 0x7f52d414def0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d259c640 1 -- 172.21.15.32:0/3161677465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52c4009420 con 0x7f52d4070230 2023-12-20T04:47:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d2d9d640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d415abb0 0x7f52d414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d259c640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d4070230 0x7f52d4147f00 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f52c40ad040 tx=0x7f52c401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52cb7fe640 1 -- 172.21.15.32:0/3161677465 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40b5810 con 0x7f52d4070230 2023-12-20T04:47:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52cb7fe640 1 -- 172.21.15.32:0/3161677465 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52c40b5990 con 0x7f52d4070230 2023-12-20T04:47:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d8d0a640 1 -- 172.21.15.32:0/3161677465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52d414d410 con 0x7f52d4070230 2023-12-20T04:47:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52cb7fe640 1 -- 172.21.15.32:0/3161677465 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52c40b5d40 con 0x7f52d4070230 2023-12-20T04:47:09.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d8d0a640 1 -- 172.21.15.32:0/3161677465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52d414d9d0 con 0x7f52d4070230 2023-12-20T04:47:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52d8d0a640 1 -- 172.21.15.32:0/3161677465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52d414dbc0 con 0x7f52d4070230 2023-12-20T04:47:09.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.678+0000 7f52cb7fe640 1 -- 172.21.15.32:0/3161677465 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 71) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f52c40c83f0 con 0x7f52d4070230 2023-12-20T04:47:09.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.682+0000 7f52cb7fe640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52b4065af0 0x7f52b4067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:09.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.682+0000 7f52cb7fe640 1 -- 172.21.15.32:0/3161677465 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f52c411e550 con 0x7f52d4070230 2023-12-20T04:47:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.682+0000 7f52c9d9b640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52b4065af0 0x7f52b4067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:09.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.682+0000 7f52c9d9b640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52b4065af0 0x7f52b4067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f52d41494f0 tx=0x7f52bc009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:09.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.686+0000 7f52cb7fe640 1 -- 172.21.15.32:0/3161677465 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f52c40ebaa0 con 0x7f52d4070230 2023-12-20T04:47:09.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:09.878+0000 7f52d8d0a640 1 -- 172.21.15.32:0/3161677465 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f52d404e000 con 0x7f52d4070230 2023-12-20T04:47:10.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52cb7fe640 1 -- 172.21.15.32:0/3161677465 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v56) v1 ==== 104+0+0 (secure 0 0 0) 0x7f52c40ef0c0 con 0x7f52d4070230 2023-12-20T04:47:10.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 -- 172.21.15.32:0/3161677465 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52b4065af0 msgr2=0x7f52b4067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:10.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52b4065af0 0x7f52b4067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f52d41494f0 tx=0x7f52bc009040 comp rx=0 tx=0).stop 2023-12-20T04:47:10.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 -- 172.21.15.32:0/3161677465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d4070230 msgr2=0x7f52d4147f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:10.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d4070230 0x7f52d4147f00 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f52c40ad040 tx=0x7f52c401acc0 comp rx=0 tx=0).stop 2023-12-20T04:47:10.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 -- 172.21.15.32:0/3161677465 shutdown_connections 2023-12-20T04:47:10.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52b4065af0 0x7f52b4067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:10.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52d415abb0 0x7f52d414def0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:10.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52d40b0cb0 0x7f52d40b3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:10.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 --2- 172.21.15.32:0/3161677465 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52d4070230 0x7f52d4147f00 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:10.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 -- 172.21.15.32:0/3161677465 >> 172.21.15.32:0/3161677465 conn(0x7f52d406a510 msgr2=0x7f52d40cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:10.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 -- 172.21.15.32:0/3161677465 shutdown_connections 2023-12-20T04:47:10.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.502+0000 7f52d8d0a640 1 -- 172.21.15.32:0/3161677465 wait complete. 2023-12-20T04:47:10.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T04:47:10.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.722+0000 7f30c01f0640 1 Processor -- start 2023-12-20T04:47:10.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.722+0000 7f30c01f0640 1 -- start start 2023-12-20T04:47:10.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30c01f0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f30b8159af0 0x7f30b80b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:10.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30c01f0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b806dd50 con 0x7f30b80b0f00 2023-12-20T04:47:10.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30c01f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f30b80cc940 con 0x7f30b806d2d0 2023-12-20T04:47:10.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30c01f0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30b80ccc20 con 0x7f30b8159af0 2023-12-20T04:47:10.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30be78d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f30b80b0f00 0x7f30b80cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57066/0 (socket says 172.21.15.32:57066) 2023-12-20T04:47:10.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30be78d640 1 -- 172.21.15.32:0/154931430 learned_addr learned my addr 172.21.15.32:0/154931430 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bdf8c640 1 --2- 172.21.15.32:0/154931430 >> v2:172.21.15.39:3300/0 conn(0x7f30b8159af0 0x7f30b80b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1376745335 0 0) 0x7f30b806dd50 con 0x7f30b80b0f00 2023-12-20T04:47:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f30a4005740 con 0x7f30b80b0f00 2023-12-20T04:47:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 96588528 0 0) 0x7f30a4005740 con 0x7f30b80b0f00 2023-12-20T04:47:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 >> v2:172.21.15.39:3300/0 conn(0x7f30b8159af0 msgr2=0x7f30b80b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 --2- 172.21.15.32:0/154931430 >> v2:172.21.15.39:3300/0 conn(0x7f30b8159af0 0x7f30b80b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 >> v1:172.21.15.39:6789/0 conn(0x7f30b806d2d0 legacy=0x7f30b80b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80ccf00 con 0x7f30b80b0f00 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f30b0001a00 con 0x7f30b80b0f00 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 >> v1:172.21.15.32:6789/0 conn(0x7f30b80b0f00 legacy=0x7f30b80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 --2- 172.21.15.32:0/154931430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30a4005b90 0x7f30a4005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 --2- 172.21.15.32:0/154931430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30a40069e0 0x7f30a4006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 --2- 172.21.15.32:0/154931430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 0x7f30a400dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b80ccc20 con 0x7f30a40073a0 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30a400e240 con 0x7f30a4005b90 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30a400e400 con 0x7f30a40069e0 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30be78d640 1 --2- 172.21.15.32:0/154931430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 0x7f30a400dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bdf8c640 1 --2- 172.21.15.32:0/154931430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30a4005b90 0x7f30a4005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30bd78b640 1 --2- 172.21.15.32:0/154931430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30a40069e0 0x7f30a4006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30be78d640 1 -- 172.21.15.32:0/154931430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30a40069e0 msgr2=0x7f30a4006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30be78d640 1 --2- 172.21.15.32:0/154931430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30a40069e0 0x7f30a4006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30be78d640 1 -- 172.21.15.32:0/154931430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30a4005b90 msgr2=0x7f30a4005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30be78d640 1 --2- 172.21.15.32:0/154931430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30a4005b90 0x7f30a4005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.726+0000 7f30be78d640 1 -- 172.21.15.32:0/154931430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30a4005740 con 0x7f30a40073a0 2023-12-20T04:47:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30be78d640 1 -- 172.21.15.32:0/154931430 reap_dead start 2023-12-20T04:47:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30be78d640 1 --2- 172.21.15.32:0/154931430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 0x7f30a400dba0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f30b000a830 tx=0x7f30b001d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e576ea1f2e314778 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b00072d0 con 0x7f30a40073a0 2023-12-20T04:47:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b00a8070 con 0x7f30a40073a0 2023-12-20T04:47:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30bcf8a640 1 -- 172.21.15.32:0/154931430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b00076b0 con 0x7f30a40073a0 2023-12-20T04:47:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30c01f0640 1 -- 172.21.15.32:0/154931430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 msgr2=0x7f30a400dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30c01f0640 1 --2- 172.21.15.32:0/154931430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 0x7f30a400dba0 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f30b000a830 tx=0x7f30b001d340 comp rx=0 tx=0).stop 2023-12-20T04:47:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30c01f0640 1 -- 172.21.15.32:0/154931430 shutdown_connections 2023-12-20T04:47:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30c01f0640 1 --2- 172.21.15.32:0/154931430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 0x7f30a400dba0 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30c01f0640 1 -- 172.21.15.32:0/154931430 >> 172.21.15.32:0/154931430 conn(0x7f30b806a510 msgr2=0x7f30b80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30c01f0640 1 -- 172.21.15.32:0/154931430 shutdown_connections 2023-12-20T04:47:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.730+0000 7f30c01f0640 1 -- 172.21.15.32:0/154931430 wait complete. 2023-12-20T04:47:10.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30c01f0640 1 Processor -- start 2023-12-20T04:47:10.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30c01f0640 1 -- start start 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30c01f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 0x7f30b80c67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30c01f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b80c58b0 0x7f30b8155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30c01f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30b8156360 0x7f30b80c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30c01f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8151370 con 0x7f30a40073a0 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30c01f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8150e90 con 0x7f30b80c58b0 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30c01f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30b8151020 con 0x7f30b8156360 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30bdf8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 0x7f30b80c67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30bdf8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 0x7f30b80c67e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56928/0 (socket says 172.21.15.32:56928) 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30bdf8c640 1 -- 172.21.15.32:0/3279710080 learned_addr learned my addr 172.21.15.32:0/3279710080 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30bd78b640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b80c58b0 0x7f30b8155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30be78d640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30b8156360 0x7f30b80c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30bdf8c640 1 -- 172.21.15.32:0/3279710080 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30b8156360 msgr2=0x7f30b80c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:10.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30bdf8c640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30b8156360 0x7f30b80c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:10.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30bdf8c640 1 -- 172.21.15.32:0/3279710080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b80c58b0 msgr2=0x7f30b8155e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:10.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30bdf8c640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b80c58b0 0x7f30b8155e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:10.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30bdf8c640 1 -- 172.21.15.32:0/3279710080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30b80c6d20 con 0x7f30a40073a0 2023-12-20T04:47:10.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30bdf8c640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 0x7f30b80c67e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f30b4011960 tx=0x7f30b4011e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:10.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30ae7fc640 1 -- 172.21.15.32:0/3279710080 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b400faf0 con 0x7f30a40073a0 2023-12-20T04:47:10.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30ae7fc640 1 -- 172.21.15.32:0/3279710080 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30b400fc70 con 0x7f30a40073a0 2023-12-20T04:47:10.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30ae7fc640 1 -- 172.21.15.32:0/3279710080 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30b4014440 con 0x7f30a40073a0 2023-12-20T04:47:10.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30c01f0640 1 -- 172.21.15.32:0/3279710080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30b80c7870 con 0x7f30a40073a0 2023-12-20T04:47:10.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.734+0000 7f30c01f0640 1 -- 172.21.15.32:0/3279710080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30b80c7e30 con 0x7f30a40073a0 2023-12-20T04:47:10.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.738+0000 7f30c01f0640 1 -- 172.21.15.32:0/3279710080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30b804e000 con 0x7f30a40073a0 2023-12-20T04:47:10.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.738+0000 7f30ae7fc640 1 -- 172.21.15.32:0/3279710080 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 71) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f30b40145a0 con 0x7f30a40073a0 2023-12-20T04:47:10.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.738+0000 7f30ae7fc640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f309c065af0 0x7f309c067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:10.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.738+0000 7f30ae7fc640 1 -- 172.21.15.32:0/3279710080 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(127..127 src has 1..127) v4 ==== 6511+0+0 (secure 0 0 0) 0x7f30b4083e40 con 0x7f30a40073a0 2023-12-20T04:47:10.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.738+0000 7f30bd78b640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f309c065af0 0x7f309c067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:10.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.738+0000 7f30bd78b640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f309c065af0 0x7f309c067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f30a8000c10 tx=0x7f30a8009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:10.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.742+0000 7f30ae7fc640 1 -- 172.21.15.32:0/3279710080 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f30b40513f0 con 0x7f30a40073a0 2023-12-20T04:47:10.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:10.898+0000 7f30c01f0640 1 -- 172.21.15.32:0/3279710080 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f30b80c9470 con 0x7f30a40073a0 2023-12-20T04:47:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.514+0000 7f30ae7fc640 1 -- 172.21.15.32:0/3279710080 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f30b40517c0 con 0x7f30a40073a0 2023-12-20T04:47:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.514+0000 7f30ae7fc640 1 -- 172.21.15.32:0/3279710080 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v128) v1 ==== 181+0+0 (secure 0 0 0) 0x7f30b404cd70 con 0x7f30a40073a0 2023-12-20T04:47:11.514 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T04:47:11.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 -- 172.21.15.32:0/3279710080 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f309c065af0 msgr2=0x7f309c067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:11.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f309c065af0 0x7f309c067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f30a8000c10 tx=0x7f30a8009040 comp rx=0 tx=0).stop 2023-12-20T04:47:11.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 -- 172.21.15.32:0/3279710080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 msgr2=0x7f30b80c67e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 0x7f30b80c67e0 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f30b4011960 tx=0x7f30b4011e20 comp rx=0 tx=0).stop 2023-12-20T04:47:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 -- 172.21.15.32:0/3279710080 shutdown_connections 2023-12-20T04:47:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f309c065af0 0x7f309c067fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30b8156360 0x7f30b80c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30b80c58b0 0x7f30b8155e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 --2- 172.21.15.32:0/3279710080 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30a40073a0 0x7f30b80c67e0 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 -- 172.21.15.32:0/3279710080 >> 172.21.15.32:0/3279710080 conn(0x7f30b806a510 msgr2=0x7f30b8158c40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 -- 172.21.15.32:0/3279710080 shutdown_connections 2023-12-20T04:47:11.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.518+0000 7f30c01f0640 1 -- 172.21.15.32:0/3279710080 wait complete. 2023-12-20T04:47:11.541 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T04:47:11.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f932e109640 1 Processor -- start 2023-12-20T04:47:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f932e109640 1 -- start start 2023-12-20T04:47:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f932e109640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9328158520 0x7f932806dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:11.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f932e109640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f93280ca050 0x7f932815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:11.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f932e109640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93280ca7f0 con 0x7f9328070230 2023-12-20T04:47:11.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f932e109640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93280caad0 con 0x7f9328158520 2023-12-20T04:47:11.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f932e109640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93280cadb0 con 0x7f93280ca050 2023-12-20T04:47:11.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f93277fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9328158520 0x7f932806dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f93277fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9328158520 0x7f932806dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56932/0 (socket says 172.21.15.32:56932) 2023-12-20T04:47:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f93277fe640 1 -- 172.21.15.32:0/178210182 learned_addr learned my addr 172.21.15.32:0/178210182 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f9326ffd640 1 --2- 172.21.15.32:0/178210182 >> v2:172.21.15.39:3300/0 conn(0x7f93280ca050 0x7f932815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f93277fe640 1 -- 172.21.15.32:0/178210182 >> v2:172.21.15.39:3300/0 conn(0x7f93280ca050 msgr2=0x7f932815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f93277fe640 1 --2- 172.21.15.32:0/178210182 >> v2:172.21.15.39:3300/0 conn(0x7f93280ca050 0x7f932815ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f93277fe640 1 -- 172.21.15.32:0/178210182 >> v1:172.21.15.32:6789/0 conn(0x7f9328070230 legacy=0x7f932815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f93277fe640 1 -- 172.21.15.32:0/178210182 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f932815e780 con 0x7f9328158520 2023-12-20T04:47:11.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.702+0000 7f93267fc640 1 -- 172.21.15.32:0/178210182 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1105430822 0 0) 0x7f93280ca7f0 con 0x7f9328070230 2023-12-20T04:47:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f93277fe640 1 --2- 172.21.15.32:0/178210182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9328158520 0x7f932806dcb0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f93280cf830 tx=0x7f931801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10a0147b38c12f42 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f93267fc640 1 -- 172.21.15.32:0/178210182 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93180aa650 con 0x7f9328158520 2023-12-20T04:47:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f93267fc640 1 -- 172.21.15.32:0/178210182 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93180aa7b0 con 0x7f9328158520 2023-12-20T04:47:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f93267fc640 1 -- 172.21.15.32:0/178210182 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93180aab40 con 0x7f9328158520 2023-12-20T04:47:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f932e109640 1 -- 172.21.15.32:0/178210182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9328158520 msgr2=0x7f932806dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f932e109640 1 --2- 172.21.15.32:0/178210182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9328158520 0x7f932806dcb0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7f93280cf830 tx=0x7f931801c4d0 comp rx=0 tx=0).stop 2023-12-20T04:47:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f932e109640 1 -- 172.21.15.32:0/178210182 shutdown_connections 2023-12-20T04:47:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f932e109640 1 --2- 172.21.15.32:0/178210182 >> v2:172.21.15.39:3300/0 conn(0x7f93280ca050 0x7f932815ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f932e109640 1 --2- 172.21.15.32:0/178210182 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9328158520 0x7f932806dcb0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f932e109640 1 -- 172.21.15.32:0/178210182 >> 172.21.15.32:0/178210182 conn(0x7f932806a510 msgr2=0x7f93280ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f932e109640 1 -- 172.21.15.32:0/178210182 shutdown_connections 2023-12-20T04:47:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.706+0000 7f932e109640 1 -- 172.21.15.32:0/178210182 wait complete. 2023-12-20T04:47:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f932e109640 1 Processor -- start 2023-12-20T04:47:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f932e109640 1 -- start start 2023-12-20T04:47:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f932e109640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328070230 0x7f9328155660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f932e109640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93280ca050 0x7f93280cc6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f932e109640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9328158520 0x7f93281519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f932e109640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280cd140 con 0x7f9328158520 2023-12-20T04:47:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f932e109640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93280cd420 con 0x7f9328070230 2023-12-20T04:47:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f932e109640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93280cd700 con 0x7f93280ca050 2023-12-20T04:47:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f9327fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9328158520 0x7f93281519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f9327fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9328158520 0x7f93281519c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56948/0 (socket says 172.21.15.32:56948) 2023-12-20T04:47:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f9327fff640 1 -- 172.21.15.32:0/976919484 learned_addr learned my addr 172.21.15.32:0/976919484 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f9326ffd640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93280ca050 0x7f93280cc6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f93277fe640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328070230 0x7f9328155660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f9327fff640 1 -- 172.21.15.32:0/976919484 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93280ca050 msgr2=0x7f93280cc6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f9327fff640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93280ca050 0x7f93280cc6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f9327fff640 1 -- 172.21.15.32:0/976919484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328070230 msgr2=0x7f9328155660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f9327fff640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328070230 0x7f9328155660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f9327fff640 1 -- 172.21.15.32:0/976919484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93180094d0 con 0x7f9328158520 2023-12-20T04:47:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f9327fff640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9328158520 0x7f93281519c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f9320001570 tx=0x7f932000cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f930ffff640 1 -- 172.21.15.32:0/976919484 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320003db0 con 0x7f9328158520 2023-12-20T04:47:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f930ffff640 1 -- 172.21.15.32:0/976919484 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9320005070 con 0x7f9328158520 2023-12-20T04:47:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f930ffff640 1 -- 172.21.15.32:0/976919484 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9320005420 con 0x7f9328158520 2023-12-20T04:47:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f932e109640 1 -- 172.21.15.32:0/976919484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93280cd9e0 con 0x7f9328158520 2023-12-20T04:47:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f932e109640 1 -- 172.21.15.32:0/976919484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9328152870 con 0x7f9328158520 2023-12-20T04:47:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.710+0000 7f932e109640 1 -- 172.21.15.32:0/976919484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93280cdf00 con 0x7f9328158520 2023-12-20T04:47:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.714+0000 7f930ffff640 1 -- 172.21.15.32:0/976919484 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9320005580 con 0x7f9328158520 2023-12-20T04:47:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.714+0000 7f930ffff640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9314065b80 0x7f9314068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.714+0000 7f930ffff640 1 -- 172.21.15.32:0/976919484 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(128..128 src has 1..128) v4 ==== 6073+0+0 (secure 0 0 0) 0x7f9320082e00 con 0x7f9328158520 2023-12-20T04:47:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.714+0000 7f93277fe640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9314065b80 0x7f9314068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:11.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.714+0000 7f93277fe640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9314065b80 0x7f9314068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f93180ae000 tx=0x7f93180a7000 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.718+0000 7f930ffff640 1 -- 172.21.15.32:0/976919484 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f93280cdf00 con 0x7f9328158520 2023-12-20T04:47:11.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:11.874+0000 7f932e109640 1 -- 172.21.15.32:0/976919484 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f93280cdf00 con 0x7f9328158520 2023-12-20T04:47:12.512 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T04:47:12.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.514+0000 7f930ffff640 1 -- 172.21.15.32:0/976919484 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v129) v1 ==== 169+0+0 (secure 0 0 0) 0x7f93280cdf00 con 0x7f9328158520 2023-12-20T04:47:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 -- 172.21.15.32:0/976919484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9314065b80 msgr2=0x7f9314068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9314065b80 0x7f9314068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f93180ae000 tx=0x7f93180a7000 comp rx=0 tx=0).stop 2023-12-20T04:47:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 -- 172.21.15.32:0/976919484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9328158520 msgr2=0x7f93281519c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:12.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9328158520 0x7f93281519c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7f9320001570 tx=0x7f932000cd20 comp rx=0 tx=0).stop 2023-12-20T04:47:12.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 -- 172.21.15.32:0/976919484 shutdown_connections 2023-12-20T04:47:12.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9314065b80 0x7f9314068040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9328158520 0x7f93281519c0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93280ca050 0x7f93280cc6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 --2- 172.21.15.32:0/976919484 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9328070230 0x7f9328155660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 -- 172.21.15.32:0/976919484 >> 172.21.15.32:0/976919484 conn(0x7f932806a510 msgr2=0x7f9328164e20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 -- 172.21.15.32:0/976919484 shutdown_connections 2023-12-20T04:47:12.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.518+0000 7f932e109640 1 -- 172.21.15.32:0/976919484 wait complete. 2023-12-20T04:47:12.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T04:47:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.694+0000 7f65cc0c4640 1 Processor -- start 2023-12-20T04:47:12.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.694+0000 7f65cc0c4640 1 -- start start 2023-12-20T04:47:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65cc0c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c4157480 0x7f65c40c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65cc0c4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f65c40cd4c0 0x7f65c40b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65cc0c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65c40b1100 con 0x7f65c40b0760 2023-12-20T04:47:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65cc0c4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65c40cde20 con 0x7f65c40cd4c0 2023-12-20T04:47:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65cc0c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c40c8e30 con 0x7f65c4157480 2023-12-20T04:47:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65c9e60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c4157480 0x7f65c40c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65c965f640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f65c40b0760 0x7f65c415d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:40816/0 (socket says 172.21.15.32:40816) 2023-12-20T04:47:12.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65c9e60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c4157480 0x7f65c40c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:37836/0 (socket says 172.21.15.32:37836) 2023-12-20T04:47:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65c965f640 1 -- 172.21.15.32:0/2253637190 learned_addr learned my addr 172.21.15.32:0/2253637190 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65ca661640 1 --2- 172.21.15.32:0/2253637190 >> v2:172.21.15.39:3300/0 conn(0x7f65c40cd4c0 0x7f65c40b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65c8e5e640 1 -- 172.21.15.32:0/2253637190 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2779658409 0 0) 0x7f65c40b1100 con 0x7f65c40b0760 2023-12-20T04:47:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65c8e5e640 1 -- 172.21.15.32:0/2253637190 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f65b0003530 con 0x7f65c40b0760 2023-12-20T04:47:12.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65ca661640 1 -- 172.21.15.32:0/2253637190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c4157480 msgr2=0x7f65c40c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65ca661640 1 --2- 172.21.15.32:0/2253637190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c4157480 0x7f65c40c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65ca661640 1 -- 172.21.15.32:0/2253637190 >> v1:172.21.15.39:6789/0 conn(0x7f65c40b0760 legacy=0x7f65c415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65ca661640 1 -- 172.21.15.32:0/2253637190 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c40ce070 con 0x7f65c40cd4c0 2023-12-20T04:47:12.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.698+0000 7f65c9e60640 1 --2- 172.21.15.32:0/2253637190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c4157480 0x7f65c40c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65ca661640 1 --2- 172.21.15.32:0/2253637190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c40cd4c0 0x7f65c40b0bc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009d90 tx=0x7f65bc01c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a71b648b52616d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65c8e5e640 1 -- 172.21.15.32:0/2253637190 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc0a5870 con 0x7f65c40cd4c0 2023-12-20T04:47:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65c8e5e640 1 -- 172.21.15.32:0/2253637190 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65bc0a59d0 con 0x7f65c40cd4c0 2023-12-20T04:47:12.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65c8e5e640 1 -- 172.21.15.32:0/2253637190 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65bc0a5d60 con 0x7f65c40cd4c0 2023-12-20T04:47:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 -- 172.21.15.32:0/2253637190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c40cd4c0 msgr2=0x7f65c40b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 --2- 172.21.15.32:0/2253637190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c40cd4c0 0x7f65c40b0bc0 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f65bc009d90 tx=0x7f65bc01c890 comp rx=0 tx=0).stop 2023-12-20T04:47:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 -- 172.21.15.32:0/2253637190 shutdown_connections 2023-12-20T04:47:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 --2- 172.21.15.32:0/2253637190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c40cd4c0 0x7f65c40b0bc0 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 --2- 172.21.15.32:0/2253637190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c4157480 0x7f65c40c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 -- 172.21.15.32:0/2253637190 >> 172.21.15.32:0/2253637190 conn(0x7f65c406a510 msgr2=0x7f65c40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 -- 172.21.15.32:0/2253637190 shutdown_connections 2023-12-20T04:47:12.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 -- 172.21.15.32:0/2253637190 wait complete. 2023-12-20T04:47:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 Processor -- start 2023-12-20T04:47:12.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 -- start start 2023-12-20T04:47:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c40b0760 0x7f65c413a220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c40cd4c0 0x7f65c414df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c4157480 0x7f65c414f030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c40c8710 con 0x7f65c40b0760 2023-12-20T04:47:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65c40c7e70 con 0x7f65c4157480 2023-12-20T04:47:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.702+0000 7f65cc0c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65c40c8150 con 0x7f65c40cd4c0 2023-12-20T04:47:12.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65c9e60640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c40b0760 0x7f65c413a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65c9e60640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c40b0760 0x7f65c413a220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56950/0 (socket says 172.21.15.32:56950) 2023-12-20T04:47:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65c9e60640 1 -- 172.21.15.32:0/1560317556 learned_addr learned my addr 172.21.15.32:0/1560317556 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65ca661640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c4157480 0x7f65c414f030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65c965f640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c40cd4c0 0x7f65c414df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65c9e60640 1 -- 172.21.15.32:0/1560317556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c40cd4c0 msgr2=0x7f65c414df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65c9e60640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c40cd4c0 0x7f65c414df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65c9e60640 1 -- 172.21.15.32:0/1560317556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c4157480 msgr2=0x7f65c414f030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65c9e60640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c4157480 0x7f65c414f030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65c9e60640 1 -- 172.21.15.32:0/1560317556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65c000b6c0 con 0x7f65c40b0760 2023-12-20T04:47:12.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65c965f640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c40cd4c0 0x7f65c414df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:12.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65c9e60640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c40b0760 0x7f65c413a220 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f65c000ed40 tx=0x7f65c000d720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:12.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65ba7fc640 1 -- 172.21.15.32:0/1560317556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0010850 con 0x7f65c40b0760 2023-12-20T04:47:12.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65ba7fc640 1 -- 172.21.15.32:0/1560317556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f65c00109b0 con 0x7f65c40b0760 2023-12-20T04:47:12.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65cc0c4640 1 -- 172.21.15.32:0/1560317556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65bc009a70 con 0x7f65c40b0760 2023-12-20T04:47:12.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65ba7fc640 1 -- 172.21.15.32:0/1560317556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f65c0010d20 con 0x7f65c40b0760 2023-12-20T04:47:12.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65cc0c4640 1 -- 172.21.15.32:0/1560317556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65c4150020 con 0x7f65c40b0760 2023-12-20T04:47:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65cc0c4640 1 -- 172.21.15.32:0/1560317556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65c404e000 con 0x7f65c40b0760 2023-12-20T04:47:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65ba7fc640 1 -- 172.21.15.32:0/1560317556 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f65c000f3d0 con 0x7f65c40b0760 2023-12-20T04:47:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65ba7fc640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65a8065b80 0x7f65a8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.706+0000 7f65ba7fc640 1 -- 172.21.15.32:0/1560317556 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f65c0083e90 con 0x7f65c40b0760 2023-12-20T04:47:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.710+0000 7f65c965f640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65a8065b80 0x7f65a8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:12.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.710+0000 7f65c965f640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65a8065b80 0x7f65a8068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f65c414e9f0 tx=0x7f65b4009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:12.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.710+0000 7f65ba7fc640 1 -- 172.21.15.32:0/1560317556 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f65c00515c0 con 0x7f65c40b0760 2023-12-20T04:47:12.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.870+0000 7f65cc0c4640 1 -- 172.21.15.32:0/1560317556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f65c40b39e0 con 0x7f65c40b0760 2023-12-20T04:47:12.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.874+0000 7f65ba7fc640 1 -- 172.21.15.32:0/1560317556 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v43) v1 ==== 120+0+0 (secure 0 0 0) 0x7f65c0054be0 con 0x7f65c40b0760 2023-12-20T04:47:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 -- 172.21.15.32:0/1560317556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65a8065b80 msgr2=0x7f65a8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65a8065b80 0x7f65a8068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f65c414e9f0 tx=0x7f65b4009040 comp rx=0 tx=0).stop 2023-12-20T04:47:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 -- 172.21.15.32:0/1560317556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c40b0760 msgr2=0x7f65c413a220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c40b0760 0x7f65c413a220 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7f65c000ed40 tx=0x7f65c000d720 comp rx=0 tx=0).stop 2023-12-20T04:47:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 -- 172.21.15.32:0/1560317556 shutdown_connections 2023-12-20T04:47:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f65a8065b80 0x7f65a8068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65c4157480 0x7f65c414f030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65c40cd4c0 0x7f65c414df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 --2- 172.21.15.32:0/1560317556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65c40b0760 0x7f65c413a220 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:12.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 -- 172.21.15.32:0/1560317556 >> 172.21.15.32:0/1560317556 conn(0x7f65c406a510 msgr2=0x7f65c40c4d80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 -- 172.21.15.32:0/1560317556 shutdown_connections 2023-12-20T04:47:12.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:12.878+0000 7f65cc0c4640 1 -- 172.21.15.32:0/1560317556 wait complete. 2023-12-20T04:47:12.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T04:47:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff7a0c95640 1 Processor -- start 2023-12-20T04:47:13.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff7a0c95640 1 -- start start 2023-12-20T04:47:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff7a0c95640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff79c15abb0 0x7ff79c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff7a0c95640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff79c0b0cb0 0x7ff79c15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff7a0c95640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff79c15b6a0 con 0x7ff79c070230 2023-12-20T04:47:13.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff7a0c95640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff79c06e160 con 0x7ff79c15abb0 2023-12-20T04:47:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff7a0c95640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c15c070 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff79a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff79c15abb0 0x7ff79c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff79ad9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff79c070230 0x7ff79c06c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57092/0 (socket says 172.21.15.32:57092) 2023-12-20T04:47:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff79ad9d640 1 -- 172.21.15.32:0/3493358123 learned_addr learned my addr 172.21.15.32:0/3493358123 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:13.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff793fff640 1 --2- 172.21.15.32:0/3493358123 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff79c0b0cb0 0x7ff79c15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff79a59c640 1 -- 172.21.15.32:0/3493358123 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff79c0b0cb0 msgr2=0x7ff79c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff79a59c640 1 --2- 172.21.15.32:0/3493358123 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff79c0b0cb0 0x7ff79c15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff79a59c640 1 -- 172.21.15.32:0/3493358123 >> v1:172.21.15.32:6789/0 conn(0x7ff79c070230 legacy=0x7ff79c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff79a59c640 1 -- 172.21.15.32:0/3493358123 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff79c06d8c0 con 0x7ff79c15abb0 2023-12-20T04:47:13.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.058+0000 7ff799d9b640 1 -- 172.21.15.32:0/3493358123 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3131430136 0 0) 0x7ff79c15b6a0 con 0x7ff79c070230 2023-12-20T04:47:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff79a59c640 1 --2- 172.21.15.32:0/3493358123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff79c15abb0 0x7ff79c0caaf0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0097c0 tx=0x7ff78c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=beac00d722295591 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff799d9b640 1 -- 172.21.15.32:0/3493358123 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c0ab680 con 0x7ff79c15abb0 2023-12-20T04:47:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff799d9b640 1 -- 172.21.15.32:0/3493358123 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78c0ab7e0 con 0x7ff79c15abb0 2023-12-20T04:47:13.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff799d9b640 1 -- 172.21.15.32:0/3493358123 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78c0abb70 con 0x7ff79c15abb0 2023-12-20T04:47:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff7a0c95640 1 -- 172.21.15.32:0/3493358123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff79c15abb0 msgr2=0x7ff79c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff7a0c95640 1 --2- 172.21.15.32:0/3493358123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff79c15abb0 0x7ff79c0caaf0 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0097c0 tx=0x7ff78c01c480 comp rx=0 tx=0).stop 2023-12-20T04:47:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff7a0c95640 1 -- 172.21.15.32:0/3493358123 shutdown_connections 2023-12-20T04:47:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff7a0c95640 1 --2- 172.21.15.32:0/3493358123 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff79c0b0cb0 0x7ff79c15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff7a0c95640 1 --2- 172.21.15.32:0/3493358123 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff79c15abb0 0x7ff79c0caaf0 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff7a0c95640 1 -- 172.21.15.32:0/3493358123 >> 172.21.15.32:0/3493358123 conn(0x7ff79c06a510 msgr2=0x7ff79c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff7a0c95640 1 -- 172.21.15.32:0/3493358123 shutdown_connections 2023-12-20T04:47:13.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.062+0000 7ff7a0c95640 1 -- 172.21.15.32:0/3493358123 wait complete. 2023-12-20T04:47:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff7a0c95640 1 Processor -- start 2023-12-20T04:47:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff7a0c95640 1 -- start start 2023-12-20T04:47:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff7a0c95640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79c070230 0x7ff79c1f8f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff7a0c95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff79c0b0cb0 0x7ff79c1f94a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff7a0c95640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff79c15abb0 0x7ff79c15cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff7a0c95640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c163890 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff7a0c95640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c15d880 con 0x7ff79c070230 2023-12-20T04:47:13.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff7a0c95640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff79c15db60 con 0x7ff79c15abb0 2023-12-20T04:47:13.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff793fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff79c0b0cb0 0x7ff79c1f94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff793fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff79c0b0cb0 0x7ff79c1f94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56960/0 (socket says 172.21.15.32:56960) 2023-12-20T04:47:13.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff793fff640 1 -- 172.21.15.32:0/1584782132 learned_addr learned my addr 172.21.15.32:0/1584782132 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:13.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff79a59c640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79c070230 0x7ff79c1f8f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff79ad9d640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff79c15abb0 0x7ff79c15cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff793fff640 1 -- 172.21.15.32:0/1584782132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff79c15abb0 msgr2=0x7ff79c15cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff793fff640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff79c15abb0 0x7ff79c15cbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff793fff640 1 -- 172.21.15.32:0/1584782132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79c070230 msgr2=0x7ff79c1f8f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff793fff640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79c070230 0x7ff79c1f8f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff793fff640 1 -- 172.21.15.32:0/1584782132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff78c009420 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff793fff640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff79c0b0cb0 0x7ff79c1f94a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff78400e930 tx=0x7ff78400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff792ffd640 1 -- 172.21.15.32:0/1584782132 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff784009690 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff792ffd640 1 -- 172.21.15.32:0/1584782132 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff78400ca00 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff792ffd640 1 -- 172.21.15.32:0/1584782132 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff78400cd70 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff7a0c95640 1 -- 172.21.15.32:0/1584782132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff79c163240 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.066+0000 7ff7a0c95640 1 -- 172.21.15.32:0/1584782132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff79c15d100 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.070+0000 7ff7a0c95640 1 -- 172.21.15.32:0/1584782132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff79c04e000 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.070+0000 7ff792ffd640 1 -- 172.21.15.32:0/1584782132 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff784013470 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.070+0000 7ff792ffd640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff77c065b80 0x7ff77c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.070+0000 7ff792ffd640 1 -- 172.21.15.32:0/1584782132 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5659+0+0 (secure 0 0 0) 0x7ff784083550 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.070+0000 7ff79a59c640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff77c065b80 0x7ff77c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.070+0000 7ff79a59c640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff77c065b80 0x7ff77c068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0af000 tx=0x7ff78c0a7000 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:13.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.074+0000 7ff792ffd640 1 -- 172.21.15.32:0/1584782132 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff784050f00 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.242+0000 7ff7a0c95640 1 -- 172.21.15.32:0/1584782132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ff79c0cf2f0 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.246+0000 7ff792ffd640 1 -- 172.21.15.32:0/1584782132 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v44) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff784054520 con 0x7ff79c0b0cb0 2023-12-20T04:47:13.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 -- 172.21.15.32:0/1584782132 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff77c065b80 msgr2=0x7ff77c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff77c065b80 0x7ff77c068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff78c0af000 tx=0x7ff78c0a7000 comp rx=0 tx=0).stop 2023-12-20T04:47:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 -- 172.21.15.32:0/1584782132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff79c0b0cb0 msgr2=0x7ff79c1f94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff79c0b0cb0 0x7ff79c1f94a0 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7ff78400e930 tx=0x7ff78400edf0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 -- 172.21.15.32:0/1584782132 shutdown_connections 2023-12-20T04:47:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff77c065b80 0x7ff77c068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff79c15abb0 0x7ff79c15cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff79c0b0cb0 0x7ff79c1f94a0 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 --2- 172.21.15.32:0/1584782132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff79c070230 0x7ff79c1f8f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 -- 172.21.15.32:0/1584782132 >> 172.21.15.32:0/1584782132 conn(0x7ff79c06a510 msgr2=0x7ff79c0b1190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 -- 172.21.15.32:0/1584782132 shutdown_connections 2023-12-20T04:47:13.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.250+0000 7ff7a0c95640 1 -- 172.21.15.32:0/1584782132 wait complete. 2023-12-20T04:47:13.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T04:47:13.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.414+0000 7fcfde81f640 1 Processor -- start 2023-12-20T04:47:13.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfde81f640 1 -- start start 2023-12-20T04:47:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfde81f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcfd815abb0 0x7fcfd80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfde81f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd815b6a0 con 0x7fcfd8070230 2023-12-20T04:47:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfde81f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcfd806d8c0 con 0x7fcfd80b0cb0 2023-12-20T04:47:13.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfde81f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd806e160 con 0x7fcfd815abb0 2023-12-20T04:47:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfdcdbc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcfd8070230 0x7fcfd806c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57096/0 (socket says 172.21.15.32:57096) 2023-12-20T04:47:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfdcdbc640 1 -- 172.21.15.32:0/1147935 learned_addr learned my addr 172.21.15.32:0/1147935 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfd7fff640 1 --2- 172.21.15.32:0/1147935 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcfd815abb0 0x7fcfd80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfd6ffd640 1 -- 172.21.15.32:0/1147935 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2700741876 0 0) 0x7fcfd815b6a0 con 0x7fcfd8070230 2023-12-20T04:47:13.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfd6ffd640 1 -- 172.21.15.32:0/1147935 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcfc0003530 con 0x7fcfd8070230 2023-12-20T04:47:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfd6ffd640 1 -- 172.21.15.32:0/1147935 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3794667918 0 0) 0x7fcfd806d8c0 con 0x7fcfd80b0cb0 2023-12-20T04:47:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfd7fff640 1 -- 172.21.15.32:0/1147935 >> v1:172.21.15.39:6789/0 conn(0x7fcfd80b0cb0 legacy=0x7fcfd815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfd7fff640 1 -- 172.21.15.32:0/1147935 >> v1:172.21.15.32:6789/0 conn(0x7fcfd8070230 legacy=0x7fcfd806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.418+0000 7fcfd7fff640 1 -- 172.21.15.32:0/1147935 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfd806dba0 con 0x7fcfd815abb0 2023-12-20T04:47:13.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfd7fff640 1 --2- 172.21.15.32:0/1147935 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcfd815abb0 0x7fcfd80caaf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80097f0 tx=0x7fcfc801c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3082cb6873e5fac4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfd6ffd640 1 -- 172.21.15.32:0/1147935 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80aa620 con 0x7fcfd815abb0 2023-12-20T04:47:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfd6ffd640 1 -- 172.21.15.32:0/1147935 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfc80aa780 con 0x7fcfd815abb0 2023-12-20T04:47:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfd6ffd640 1 -- 172.21.15.32:0/1147935 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfc80aaaf0 con 0x7fcfd815abb0 2023-12-20T04:47:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfde81f640 1 -- 172.21.15.32:0/1147935 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcfd815abb0 msgr2=0x7fcfd80caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfde81f640 1 --2- 172.21.15.32:0/1147935 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcfd815abb0 0x7fcfd80caaf0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80097f0 tx=0x7fcfc801c4c0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfde81f640 1 -- 172.21.15.32:0/1147935 shutdown_connections 2023-12-20T04:47:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfde81f640 1 --2- 172.21.15.32:0/1147935 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcfd815abb0 0x7fcfd80caaf0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfde81f640 1 -- 172.21.15.32:0/1147935 >> 172.21.15.32:0/1147935 conn(0x7fcfd806a510 msgr2=0x7fcfd80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfde81f640 1 -- 172.21.15.32:0/1147935 shutdown_connections 2023-12-20T04:47:13.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfde81f640 1 -- 172.21.15.32:0/1147935 wait complete. 2023-12-20T04:47:13.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.422+0000 7fcfde81f640 1 Processor -- start 2023-12-20T04:47:13.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfde81f640 1 -- start start 2023-12-20T04:47:13.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfde81f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfd8070230 0x7fcfd80c9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfde81f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcfd80b0cb0 0x7fcfd8158880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfde81f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfd815abb0 0x7fcfd80cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfde81f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd81577e0 con 0x7fcfd815abb0 2023-12-20T04:47:13.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfde81f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8157ac0 con 0x7fcfd8070230 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfde81f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcfd8157da0 con 0x7fcfd80b0cb0 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfdcdbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfd815abb0 0x7fcfd80cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfdcdbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfd815abb0 0x7fcfd80cb3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56972/0 (socket says 172.21.15.32:56972) 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfd7fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfd8070230 0x7fcfd80c9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfdcdbc640 1 -- 172.21.15.32:0/514280964 learned_addr learned my addr 172.21.15.32:0/514280964 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfd77fe640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcfd80b0cb0 0x7fcfd8158880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfdcdbc640 1 -- 172.21.15.32:0/514280964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcfd80b0cb0 msgr2=0x7fcfd8158880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfdcdbc640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcfd80b0cb0 0x7fcfd8158880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfdcdbc640 1 -- 172.21.15.32:0/514280964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfd8070230 msgr2=0x7fcfd80c9160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfdcdbc640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfd8070230 0x7fcfd80c9160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfdcdbc640 1 -- 172.21.15.32:0/514280964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcfc8009420 con 0x7fcfd815abb0 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfd7fff640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfd8070230 0x7fcfd80c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:13.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfdcdbc640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfd815abb0 0x7fcfd80cb3b0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000cb40 tx=0x7fcfd000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfd4ff9640 1 -- 172.21.15.32:0/514280964 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd000aca0 con 0x7fcfd815abb0 2023-12-20T04:47:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfd4ff9640 1 -- 172.21.15.32:0/514280964 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcfd000ae00 con 0x7fcfd815abb0 2023-12-20T04:47:13.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfde81f640 1 -- 172.21.15.32:0/514280964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcfd8158080 con 0x7fcfd815abb0 2023-12-20T04:47:13.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfd4ff9640 1 -- 172.21.15.32:0/514280964 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcfd0012620 con 0x7fcfd815abb0 2023-12-20T04:47:13.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfde81f640 1 -- 172.21.15.32:0/514280964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcfd80c96a0 con 0x7fcfd815abb0 2023-12-20T04:47:13.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.426+0000 7fcfde81f640 1 -- 172.21.15.32:0/514280964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcfd804e000 con 0x7fcfd815abb0 2023-12-20T04:47:13.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.430+0000 7fcfd4ff9640 1 -- 172.21.15.32:0/514280964 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcfd0007070 con 0x7fcfd815abb0 2023-12-20T04:47:13.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.430+0000 7fcfd4ff9640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfc4065b80 0x7fcfc4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.430+0000 7fcfd4ff9640 1 -- 172.21.15.32:0/514280964 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5659+0+0 (secure 0 0 0) 0x7fcfd0083280 con 0x7fcfd815abb0 2023-12-20T04:47:13.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.430+0000 7fcfd7fff640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfc4065b80 0x7fcfc4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.430+0000 7fcfd7fff640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfc4065b80 0x7fcfc4068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80ae000 tx=0x7fcfc80b5040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:13.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.434+0000 7fcfd4ff9640 1 -- 172.21.15.32:0/514280964 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcfd0050c00 con 0x7fcfd815abb0 2023-12-20T04:47:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.594+0000 7fcfde81f640 1 -- 172.21.15.32:0/514280964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fcfd806c690 con 0x7fcfd815abb0 2023-12-20T04:47:13.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfd4ff9640 1 -- 172.21.15.32:0/514280964 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v45) v1 ==== 104+0+0 (secure 0 0 0) 0x7fcfd0054220 con 0x7fcfd815abb0 2023-12-20T04:47:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 -- 172.21.15.32:0/514280964 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfc4065b80 msgr2=0x7fcfc4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfc4065b80 0x7fcfc4068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fcfc80ae000 tx=0x7fcfc80b5040 comp rx=0 tx=0).stop 2023-12-20T04:47:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 -- 172.21.15.32:0/514280964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfd815abb0 msgr2=0x7fcfd80cb3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfd815abb0 0x7fcfd80cb3b0 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7fcfd000cb40 tx=0x7fcfd000a060 comp rx=0 tx=0).stop 2023-12-20T04:47:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 -- 172.21.15.32:0/514280964 shutdown_connections 2023-12-20T04:47:13.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fcfc4065b80 0x7fcfc4068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcfd815abb0 0x7fcfd80cb3b0 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcfd80b0cb0 0x7fcfd8158880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 --2- 172.21.15.32:0/514280964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcfd8070230 0x7fcfd80c9160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 -- 172.21.15.32:0/514280964 >> 172.21.15.32:0/514280964 conn(0x7fcfd806a510 msgr2=0x7fcfd8159dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:13.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 -- 172.21.15.32:0/514280964 shutdown_connections 2023-12-20T04:47:13.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.598+0000 7fcfde81f640 1 -- 172.21.15.32:0/514280964 wait complete. 2023-12-20T04:47:13.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule' 2023-12-20T04:47:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.762+0000 7f9052efc640 1 Processor -- start 2023-12-20T04:47:13.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.762+0000 7f9052efc640 1 -- start start 2023-12-20T04:47:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9052efc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f904c15aa50 0x7f904c0716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9052efc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f904c06b160 con 0x7f904c06d840 2023-12-20T04:47:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9052efc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f904c06a550 con 0x7f904c15ae30 2023-12-20T04:47:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9052efc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f904c06a830 con 0x7f904c15aa50 2023-12-20T04:47:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9051499640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f904c06d840 0x7f904c06dcb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57102/0 (socket says 172.21.15.32:57102) 2023-12-20T04:47:13.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9051499640 1 -- 172.21.15.32:0/2464975593 learned_addr learned my addr 172.21.15.32:0/2464975593 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:13.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9050c98640 1 --2- 172.21.15.32:0/2464975593 >> v2:172.21.15.39:3300/0 conn(0x7f904c15aa50 0x7f904c0716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4280002314 0 0) 0x7f904c06b160 con 0x7f904c06d840 2023-12-20T04:47:13.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f903c003530 con 0x7f904c06d840 2023-12-20T04:47:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1903199166 0 0) 0x7f904c06a550 con 0x7f904c15ae30 2023-12-20T04:47:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f904c06b160 con 0x7f904c15ae30 2023-12-20T04:47:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 400073975 0 0) 0x7f903c003530 con 0x7f904c06d840 2023-12-20T04:47:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 >> v2:172.21.15.39:3300/0 conn(0x7f904c15aa50 msgr2=0x7f904c0716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 --2- 172.21.15.32:0/2464975593 >> v2:172.21.15.39:3300/0 conn(0x7f904c15aa50 0x7f904c0716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 >> v1:172.21.15.39:6789/0 conn(0x7f904c15ae30 legacy=0x7f904c069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c06ab10 con 0x7f904c06d840 2023-12-20T04:47:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f9044001980 con 0x7f904c06d840 2023-12-20T04:47:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 >> v1:172.21.15.32:6789/0 conn(0x7f904c06d840 legacy=0x7f904c06dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 --2- 172.21.15.32:0/2464975593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 0x7f903c003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 --2- 172.21.15.32:0/2464975593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c0047d0 0x7f903c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 --2- 172.21.15.32:0/2464975593 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f903c005190 0x7f903c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c06a830 con 0x7f903c003b60 2023-12-20T04:47:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f903c00c060 con 0x7f903c0047d0 2023-12-20T04:47:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f903c00c220 con 0x7f903c005190 2023-12-20T04:47:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9050c98640 1 --2- 172.21.15.32:0/2464975593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 0x7f903c003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f904bfff640 1 --2- 172.21.15.32:0/2464975593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c0047d0 0x7f903c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9051499640 1 --2- 172.21.15.32:0/2464975593 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f903c005190 0x7f903c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9050c98640 1 -- 172.21.15.32:0/2464975593 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f903c005190 msgr2=0x7f903c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9050c98640 1 --2- 172.21.15.32:0/2464975593 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f903c005190 0x7f903c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9050c98640 1 -- 172.21.15.32:0/2464975593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c0047d0 msgr2=0x7f903c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9050c98640 1 --2- 172.21.15.32:0/2464975593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f903c0047d0 0x7f903c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9050c98640 1 -- 172.21.15.32:0/2464975593 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f903c003530 con 0x7f903c003b60 2023-12-20T04:47:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.766+0000 7f9051499640 1 -- 172.21.15.32:0/2464975593 reap_dead start 2023-12-20T04:47:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.770+0000 7f9050c98640 1 --2- 172.21.15.32:0/2464975593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 0x7f903c003f60 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f9040009ae0 tx=0x7f904001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40c50a710d4e20b3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.770+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400a6510 con 0x7f903c003b60 2023-12-20T04:47:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.770+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90400a6670 con 0x7f903c003b60 2023-12-20T04:47:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.770+0000 7f904b7fe640 1 -- 172.21.15.32:0/2464975593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400a6a20 con 0x7f903c003b60 2023-12-20T04:47:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.770+0000 7f9052efc640 1 -- 172.21.15.32:0/2464975593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 msgr2=0x7f903c003f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.770+0000 7f9052efc640 1 --2- 172.21.15.32:0/2464975593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 0x7f903c003f60 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f9040009ae0 tx=0x7f904001ca20 comp rx=0 tx=0).stop 2023-12-20T04:47:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.770+0000 7f9052efc640 1 -- 172.21.15.32:0/2464975593 shutdown_connections 2023-12-20T04:47:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.770+0000 7f9052efc640 1 --2- 172.21.15.32:0/2464975593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 0x7f903c003f60 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.770+0000 7f9052efc640 1 -- 172.21.15.32:0/2464975593 >> 172.21.15.32:0/2464975593 conn(0x7f904c0cba50 msgr2=0x7f904c0cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.770+0000 7f9052efc640 1 -- 172.21.15.32:0/2464975593 shutdown_connections 2023-12-20T04:47:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.770+0000 7f9052efc640 1 -- 172.21.15.32:0/2464975593 wait complete. 2023-12-20T04:47:13.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9052efc640 1 Processor -- start 2023-12-20T04:47:13.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9052efc640 1 -- start start 2023-12-20T04:47:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9052efc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 0x7f904c0af5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9052efc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f904c0b35d0 0x7f904c1597e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9052efc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f904c0b0f30 0x7f904c0b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9052efc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c0b4630 con 0x7f903c003b60 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9052efc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f904c0c38e0 con 0x7f904c0b0f30 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9052efc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f904c0b3d90 con 0x7f904c0b35d0 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9050c98640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 0x7f904c0af5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9050c98640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 0x7f904c0af5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:56992/0 (socket says 172.21.15.32:56992) 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9050c98640 1 -- 172.21.15.32:0/2666450878 learned_addr learned my addr 172.21.15.32:0/2666450878 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f904bfff640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f904c0b35d0 0x7f904c1597e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9051499640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f904c0b0f30 0x7f904c0b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9050c98640 1 -- 172.21.15.32:0/2666450878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f904c0b35d0 msgr2=0x7f904c1597e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9050c98640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f904c0b35d0 0x7f904c1597e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9050c98640 1 -- 172.21.15.32:0/2666450878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f904c0b0f30 msgr2=0x7f904c0b1390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9050c98640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f904c0b0f30 0x7f904c0b1390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9050c98640 1 -- 172.21.15.32:0/2666450878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f904c0b4070 con 0x7f903c003b60 2023-12-20T04:47:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9050c98640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 0x7f904c0af5b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f904001c850 tx=0x7f903c005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f90497fa640 1 -- 172.21.15.32:0/2666450878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f904000f080 con 0x7f903c003b60 2023-12-20T04:47:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f90497fa640 1 -- 172.21.15.32:0/2666450878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f90400a67e0 con 0x7f903c003b60 2023-12-20T04:47:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f90497fa640 1 -- 172.21.15.32:0/2666450878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f90400a6b90 con 0x7f903c003b60 2023-12-20T04:47:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9052efc640 1 -- 172.21.15.32:0/2666450878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f904c0b4350 con 0x7f903c003b60 2023-12-20T04:47:13.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9052efc640 1 -- 172.21.15.32:0/2666450878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f904c0b3a50 con 0x7f903c003b60 2023-12-20T04:47:13.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.774+0000 7f9052efc640 1 -- 172.21.15.32:0/2666450878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f904c0c36b0 con 0x7f903c003b60 2023-12-20T04:47:13.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.778+0000 7f90497fa640 1 -- 172.21.15.32:0/2666450878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f90400a6cf0 con 0x7f903c003b60 2023-12-20T04:47:13.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.778+0000 7f90497fa640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9030065b80 0x7f9030068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:13.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.778+0000 7f90497fa640 1 -- 172.21.15.32:0/2666450878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f90400aa070 con 0x7f903c003b60 2023-12-20T04:47:13.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.778+0000 7f904bfff640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9030065b80 0x7f9030068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.778+0000 7f904bfff640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9030065b80 0x7f9030068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f904c0c7050 tx=0x7f9038009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.782+0000 7f90497fa640 1 -- 172.21.15.32:0/2666450878 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f904c0c36b0 con 0x7f903c003b60 2023-12-20T04:47:13.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:13.998+0000 7f9052efc640 1 -- 172.21.15.32:0/2666450878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]} v 0) v1 -- 0x7f904c0c36b0 con 0x7f903c003b60 2023-12-20T04:47:14.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f90497fa640 1 -- 172.21.15.32:0/2666450878 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule"]}]=0 v706) v1 ==== 148+0+0 (secure 0 0 0) 0x7f904c0c36b0 con 0x7f903c003b60 2023-12-20T04:47:14.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 -- 172.21.15.32:0/2666450878 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9030065b80 msgr2=0x7f9030068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:14.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9030065b80 0x7f9030068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f904c0c7050 tx=0x7f9038009040 comp rx=0 tx=0).stop 2023-12-20T04:47:14.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 -- 172.21.15.32:0/2666450878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 msgr2=0x7f904c0af5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:14.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 0x7f904c0af5b0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f904001c850 tx=0x7f903c005800 comp rx=0 tx=0).stop 2023-12-20T04:47:14.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 -- 172.21.15.32:0/2666450878 shutdown_connections 2023-12-20T04:47:14.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9030065b80 0x7f9030068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:14.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f904c0b0f30 0x7f904c0b1390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:14.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f904c0b35d0 0x7f904c1597e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:14.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 --2- 172.21.15.32:0/2666450878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f903c003b60 0x7f904c0af5b0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:14.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 -- 172.21.15.32:0/2666450878 >> 172.21.15.32:0/2666450878 conn(0x7f904c0cba50 msgr2=0x7f904c0c62d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:14.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 -- 172.21.15.32:0/2666450878 shutdown_connections 2023-12-20T04:47:14.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.526+0000 7f9052efc640 1 -- 172.21.15.32:0/2666450878 wait complete. 2023-12-20T04:47:14.549 INFO:tasks.cephfs_test_runner:Test existence of a scheduled snapshot ... ok 2023-12-20T04:47:14.549 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that number of snaps retained are as per user spec. 2023-12-20T04:47:14.549 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_number_of_snaps_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-20T04:47:14.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-12-20T04:47:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.698+0000 7f149b055640 1 Processor -- start 2023-12-20T04:47:14.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.698+0000 7f149b055640 1 -- start start 2023-12-20T04:47:14.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.698+0000 7f149b055640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 0x7f14940caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:14.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.698+0000 7f149b055640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f149406d990 0x7f149415a870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:14.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.698+0000 7f149b055640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14940b1ff0 con 0x7f14940700a0 2023-12-20T04:47:14.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.698+0000 7f149b055640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14940b1190 con 0x7f149406d990 2023-12-20T04:47:14.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.698+0000 7f149b055640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14940c88e0 con 0x7f14940b0db0 2023-12-20T04:47:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.698+0000 7f14995f2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f149406d990 0x7f149415a870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.698+0000 7f1493fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f14940700a0 0x7f149406d280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57108/0 (socket says 172.21.15.32:57108) 2023-12-20T04:47:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.698+0000 7f1493fff640 1 -- 172.21.15.32:0/1052325745 learned_addr learned my addr 172.21.15.32:0/1052325745 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.702+0000 7f1498df1640 1 --2- 172.21.15.32:0/1052325745 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 0x7f14940caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:14.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.702+0000 7f14995f2640 1 -- 172.21.15.32:0/1052325745 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 msgr2=0x7f14940caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.702+0000 7f14995f2640 1 --2- 172.21.15.32:0/1052325745 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 0x7f14940caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.702+0000 7f14995f2640 1 -- 172.21.15.32:0/1052325745 >> v1:172.21.15.32:6789/0 conn(0x7f14940700a0 legacy=0x7f149406d280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.702+0000 7f14995f2640 1 -- 172.21.15.32:0/1052325745 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14940b2240 con 0x7f149406d990 2023-12-20T04:47:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.702+0000 7f14937fe640 1 -- 172.21.15.32:0/1052325745 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 262114889 0 0) 0x7f14940b1ff0 con 0x7f14940700a0 2023-12-20T04:47:14.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.702+0000 7f1498df1640 1 --2- 172.21.15.32:0/1052325745 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 0x7f14940caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:14.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.702+0000 7f14995f2640 1 --2- 172.21.15.32:0/1052325745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149406d990 0x7f149415a870 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f148c009d90 tx=0x7f148c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=721ba26f201d7e60 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.702+0000 7f14937fe640 1 -- 172.21.15.32:0/1052325745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0a5860 con 0x7f149406d990 2023-12-20T04:47:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.702+0000 7f14937fe640 1 -- 172.21.15.32:0/1052325745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f148c0a59c0 con 0x7f149406d990 2023-12-20T04:47:14.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.702+0000 7f14937fe640 1 -- 172.21.15.32:0/1052325745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f148c0a5d50 con 0x7f149406d990 2023-12-20T04:47:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 -- 172.21.15.32:0/1052325745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149406d990 msgr2=0x7f149415a870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 --2- 172.21.15.32:0/1052325745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149406d990 0x7f149415a870 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f148c009d90 tx=0x7f148c01c950 comp rx=0 tx=0).stop 2023-12-20T04:47:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 -- 172.21.15.32:0/1052325745 shutdown_connections 2023-12-20T04:47:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 --2- 172.21.15.32:0/1052325745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149406d990 0x7f149415a870 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 --2- 172.21.15.32:0/1052325745 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 0x7f14940caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 -- 172.21.15.32:0/1052325745 >> 172.21.15.32:0/1052325745 conn(0x7f14940ce650 msgr2=0x7f14940cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 -- 172.21.15.32:0/1052325745 shutdown_connections 2023-12-20T04:47:14.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 -- 172.21.15.32:0/1052325745 wait complete. 2023-12-20T04:47:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 Processor -- start 2023-12-20T04:47:14.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 -- start start 2023-12-20T04:47:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149406d990 0x7f149414fdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14940700a0 0x7f14940c4230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 0x7f14940c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14940c4770 con 0x7f149406d990 2023-12-20T04:47:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14941540a0 con 0x7f14940700a0 2023-12-20T04:47:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f149b055640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14940c57c0 con 0x7f14940b0db0 2023-12-20T04:47:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f1498df1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149406d990 0x7f149414fdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f1498df1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149406d990 0x7f149414fdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57022/0 (socket says 172.21.15.32:57022) 2023-12-20T04:47:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f1498df1640 1 -- 172.21.15.32:0/104797538 learned_addr learned my addr 172.21.15.32:0/104797538 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:14.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f1493fff640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14940700a0 0x7f14940c4230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f14995f2640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 0x7f14940c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f1498df1640 1 -- 172.21.15.32:0/104797538 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 msgr2=0x7f14940c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f1498df1640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 0x7f14940c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f1498df1640 1 -- 172.21.15.32:0/104797538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14940700a0 msgr2=0x7f14940c4230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f1498df1640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14940700a0 0x7f14940c4230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f1498df1640 1 -- 172.21.15.32:0/104797538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f148c009a70 con 0x7f149406d990 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f14995f2640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 0x7f14940c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.706+0000 7f1493fff640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14940700a0 0x7f14940c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.710+0000 7f1498df1640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149406d990 0x7f149414fdc0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f148800c7f0 tx=0x7f148800ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.710+0000 7f14917fa640 1 -- 172.21.15.32:0/104797538 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488007820 con 0x7f149406d990 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.710+0000 7f14917fa640 1 -- 172.21.15.32:0/104797538 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1488007980 con 0x7f149406d990 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.710+0000 7f14917fa640 1 -- 172.21.15.32:0/104797538 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1488007d30 con 0x7f149406d990 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.710+0000 7f149b055640 1 -- 172.21.15.32:0/104797538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14940c5aa0 con 0x7f149406d990 2023-12-20T04:47:14.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.710+0000 7f149b055640 1 -- 172.21.15.32:0/104797538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14940c5d80 con 0x7f149406d990 2023-12-20T04:47:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.710+0000 7f149b055640 1 -- 172.21.15.32:0/104797538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1494074100 con 0x7f149406d990 2023-12-20T04:47:14.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.710+0000 7f14917fa640 1 -- 172.21.15.32:0/104797538 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f14880113f0 con 0x7f149406d990 2023-12-20T04:47:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.710+0000 7f14917fa640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1478065b80 0x7f1478068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.710+0000 7f14917fa640 1 -- 172.21.15.32:0/104797538 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f1488083760 con 0x7f149406d990 2023-12-20T04:47:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.710+0000 7f1493fff640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1478065b80 0x7f1478068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:14.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.714+0000 7f1493fff640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1478065b80 0x7f1478068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1494155760 tx=0x7f1480009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:14.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.714+0000 7f14917fa640 1 -- 172.21.15.32:0/104797538 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1488050f20 con 0x7f149406d990 2023-12-20T04:47:14.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:14.926+0000 7f149b055640 1 -- 172.21.15.32:0/104797538 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f14940c8ca0 con 0x7f149406d990 2023-12-20T04:47:15.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f14917fa640 1 -- 172.21.15.32:0/104797538 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v707) v1 ==== 181+0+0 (secure 0 0 0) 0x7f1488054540 con 0x7f149406d990 2023-12-20T04:47:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 -- 172.21.15.32:0/104797538 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1478065b80 msgr2=0x7f1478068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1478065b80 0x7f1478068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1494155760 tx=0x7f1480009040 comp rx=0 tx=0).stop 2023-12-20T04:47:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 -- 172.21.15.32:0/104797538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149406d990 msgr2=0x7f149414fdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149406d990 0x7f149414fdc0 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f148800c7f0 tx=0x7f148800ccb0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 -- 172.21.15.32:0/104797538 shutdown_connections 2023-12-20T04:47:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1478065b80 0x7f1478068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14940b0db0 0x7f14940c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14940700a0 0x7f14940c4230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 --2- 172.21.15.32:0/104797538 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f149406d990 0x7f149414fdc0 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 -- 172.21.15.32:0/104797538 >> 172.21.15.32:0/104797538 conn(0x7f14940ce650 msgr2=0x7f149415b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:15.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 -- 172.21.15.32:0/104797538 shutdown_connections 2023-12-20T04:47:15.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.534+0000 7f149b055640 1 -- 172.21.15.32:0/104797538 wait complete. 2023-12-20T04:47:15.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T04:47:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7274473640 1 Processor -- start 2023-12-20T04:47:15.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7274473640 1 -- start start 2023-12-20T04:47:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7274473640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f726c157480 0x7f726c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7274473640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f726c0b0760 0x7f726c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7274473640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f726c0b1100 con 0x7f726c15d660 2023-12-20T04:47:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7274473640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f726c0cde20 con 0x7f726c0b0760 2023-12-20T04:47:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7274473640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c0c8e30 con 0x7f726c157480 2023-12-20T04:47:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7271a0e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f726c0b0760 0x7f726c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:15.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7271a0e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f726c0b0760 0x7f726c0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57032/0 (socket says 172.21.15.32:57032) 2023-12-20T04:47:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7271a0e640 1 -- 172.21.15.32:0/3803410375 learned_addr learned my addr 172.21.15.32:0/3803410375 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f727220f640 1 --2- 172.21.15.32:0/3803410375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f726c157480 0x7f726c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7271a0e640 1 -- 172.21.15.32:0/3803410375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f726c157480 msgr2=0x7f726c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7271a0e640 1 --2- 172.21.15.32:0/3803410375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f726c157480 0x7f726c0c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7271a0e640 1 -- 172.21.15.32:0/3803410375 >> v1:172.21.15.39:6789/0 conn(0x7f726c15d660 legacy=0x7f726c15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:15.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.710+0000 7f7271a0e640 1 -- 172.21.15.32:0/3803410375 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f726c0ce070 con 0x7f726c0b0760 2023-12-20T04:47:15.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f7271a0e640 1 --2- 172.21.15.32:0/3803410375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c0b0760 0x7f726c0b0bc0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f725c009870 tx=0x7f725c01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ab724ded100905 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f727120d640 1 -- 172.21.15.32:0/3803410375 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0ab680 con 0x7f726c0b0760 2023-12-20T04:47:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f727120d640 1 -- 172.21.15.32:0/3803410375 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f725c0ab7e0 con 0x7f726c0b0760 2023-12-20T04:47:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f727120d640 1 -- 172.21.15.32:0/3803410375 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f725c0abb70 con 0x7f726c0b0760 2023-12-20T04:47:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f7274473640 1 -- 172.21.15.32:0/3803410375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c0b0760 msgr2=0x7f726c0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:15.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f7274473640 1 --2- 172.21.15.32:0/3803410375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c0b0760 0x7f726c0b0bc0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f725c009870 tx=0x7f725c01c480 comp rx=0 tx=0).stop 2023-12-20T04:47:15.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f7274473640 1 -- 172.21.15.32:0/3803410375 shutdown_connections 2023-12-20T04:47:15.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f7274473640 1 --2- 172.21.15.32:0/3803410375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c0b0760 0x7f726c0b0bc0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f7274473640 1 --2- 172.21.15.32:0/3803410375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f726c157480 0x7f726c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f7274473640 1 -- 172.21.15.32:0/3803410375 >> 172.21.15.32:0/3803410375 conn(0x7f726c06a510 msgr2=0x7f726c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:15.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f7274473640 1 -- 172.21.15.32:0/3803410375 shutdown_connections 2023-12-20T04:47:15.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.714+0000 7f7274473640 1 -- 172.21.15.32:0/3803410375 wait complete. 2023-12-20T04:47:15.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7274473640 1 Processor -- start 2023-12-20T04:47:15.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7274473640 1 -- start start 2023-12-20T04:47:15.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7274473640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c0b0760 0x7f726c0c8230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:15.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7274473640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f726c157480 0x7f726c0c3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:15.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7274473640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f726c15d660 0x7f726c0c3ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:15.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7274473640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c0c8770 con 0x7f726c0b0760 2023-12-20T04:47:15.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7274473640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f726c15ac80 con 0x7f726c157480 2023-12-20T04:47:15.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7274473640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f726c15af60 con 0x7f726c15d660 2023-12-20T04:47:15.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f727220f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c0b0760 0x7f726c0c8230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f727220f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c0b0760 0x7f726c0c8230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57034/0 (socket says 172.21.15.32:57034) 2023-12-20T04:47:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f727220f640 1 -- 172.21.15.32:0/3334118462 learned_addr learned my addr 172.21.15.32:0/3334118462 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7271a0e640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f726c157480 0x7f726c0c3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7272a10640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f726c15d660 0x7f726c0c3ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f727220f640 1 -- 172.21.15.32:0/3334118462 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f726c15d660 msgr2=0x7f726c0c3ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:15.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f727220f640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f726c15d660 0x7f726c0c3ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f727220f640 1 -- 172.21.15.32:0/3334118462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f726c157480 msgr2=0x7f726c0c3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:15.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f727220f640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f726c157480 0x7f726c0c3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f727220f640 1 -- 172.21.15.32:0/3334118462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f725c0094d0 con 0x7f726c0b0760 2023-12-20T04:47:15.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f727220f640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c0b0760 0x7f726c0c8230 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f726800d5f0 tx=0x7f726800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:15.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7262ffd640 1 -- 172.21.15.32:0/3334118462 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72680125f0 con 0x7f726c0b0760 2023-12-20T04:47:15.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7262ffd640 1 -- 172.21.15.32:0/3334118462 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7268012750 con 0x7f726c0b0760 2023-12-20T04:47:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7274473640 1 -- 172.21.15.32:0/3334118462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f726c157b00 con 0x7f726c0b0760 2023-12-20T04:47:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7262ffd640 1 -- 172.21.15.32:0/3334118462 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7268012b00 con 0x7f726c0b0760 2023-12-20T04:47:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.718+0000 7f7274473640 1 -- 172.21.15.32:0/3334118462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f726c1580c0 con 0x7f726c0b0760 2023-12-20T04:47:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.722+0000 7f7274473640 1 -- 172.21.15.32:0/3334118462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f726c1582b0 con 0x7f726c0b0760 2023-12-20T04:47:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.722+0000 7f7262ffd640 1 -- 172.21.15.32:0/3334118462 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7268012c60 con 0x7f726c0b0760 2023-12-20T04:47:15.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.722+0000 7f7262ffd640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7250065b80 0x7f7250068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.722+0000 7f7262ffd640 1 -- 172.21.15.32:0/3334118462 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f72680837b0 con 0x7f726c0b0760 2023-12-20T04:47:15.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.722+0000 7f7271a0e640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7250065b80 0x7f7250068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:15.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.722+0000 7f7271a0e640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7250065b80 0x7f7250068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f725c0af000 tx=0x7f725c0b6040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:15.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.726+0000 7f7262ffd640 1 -- 172.21.15.32:0/3334118462 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7268051080 con 0x7f726c0b0760 2023-12-20T04:47:15.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.886+0000 7f7274473640 1 -- 172.21.15.32:0/3334118462 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f726c04e000 con 0x7f726c0b0760 2023-12-20T04:47:15.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7262ffd640 1 -- 172.21.15.32:0/3334118462 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v46)=0 v46) v1 ==== 122+0+0 (secure 0 0 0) 0x7f72680546a0 con 0x7f726c0b0760 2023-12-20T04:47:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 -- 172.21.15.32:0/3334118462 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7250065b80 msgr2=0x7f7250068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7250065b80 0x7f7250068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f725c0af000 tx=0x7f725c0b6040 comp rx=0 tx=0).stop 2023-12-20T04:47:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 -- 172.21.15.32:0/3334118462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c0b0760 msgr2=0x7f726c0c8230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c0b0760 0x7f726c0c8230 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f726800d5f0 tx=0x7f726800dab0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 -- 172.21.15.32:0/3334118462 shutdown_connections 2023-12-20T04:47:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7250065b80 0x7f7250068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f726c15d660 0x7f726c0c3ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f726c157480 0x7f726c0c3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 --2- 172.21.15.32:0/3334118462 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f726c0b0760 0x7f726c0c8230 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 -- 172.21.15.32:0/3334118462 >> 172.21.15.32:0/3334118462 conn(0x7f726c06a510 msgr2=0x7f726c0cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 -- 172.21.15.32:0/3334118462 shutdown_connections 2023-12-20T04:47:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:15.890+0000 7f7274473640 1 -- 172.21.15.32:0/3334118462 wait complete. 2023-12-20T04:47:15.909 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T04:47:15.910 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:47:16.005 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T04:47:16.056 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T04:47:16.056 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:47:16.056 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:47:16.066 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:47:16.066 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:47:16.066 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:47:16.076 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:47:16.076 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:47:16.076 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:47:16.118 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T04:47:16.119 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T04:47:16.119 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:47:16.128 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T04:47:16.128 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T04:47:16.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:47:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.306+0000 7f6a648e5640 1 Processor -- start 2023-12-20T04:47:16.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.306+0000 7f6a648e5640 1 -- start start 2023-12-20T04:47:16.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a648e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6a6015f110 0x7f6a6015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a648e5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6015b990 con 0x7f6a600af5e0 2023-12-20T04:47:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a648e5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6a6015bbd0 con 0x7f6a6006ce60 2023-12-20T04:47:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a648e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6a6015beb0 con 0x7f6a6015f110 2023-12-20T04:47:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a5ed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6a6015f110 0x7f6a6015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a5e59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6a600af5e0 0x7f6a600af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57116/0 (socket says 172.21.15.32:57116) 2023-12-20T04:47:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a5e59c640 1 -- 172.21.15.32:0/3044312467 learned_addr learned my addr 172.21.15.32:0/3044312467 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a5ed9d640 1 -- 172.21.15.32:0/3044312467 >> v1:172.21.15.39:6789/0 conn(0x7f6a6006ce60 legacy=0x7f6a6015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a5ed9d640 1 -- 172.21.15.32:0/3044312467 >> v1:172.21.15.32:6789/0 conn(0x7f6a600af5e0 legacy=0x7f6a600af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a5dd9b640 1 -- 172.21.15.32:0/3044312467 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1610042726 0 0) 0x7f6a500012a0 con 0x7f6a600af5e0 2023-12-20T04:47:16.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a5ed9d640 1 -- 172.21.15.32:0/3044312467 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a6015e910 con 0x7f6a6015f110 2023-12-20T04:47:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.310+0000 7f6a5ed9d640 1 --2- 172.21.15.32:0/3044312467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6015f110 0x7f6a6015c440 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a5801c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7d2d822a7a02a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a5dd9b640 1 -- 172.21.15.32:0/3044312467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580a58b0 con 0x7f6a6015f110 2023-12-20T04:47:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a5dd9b640 1 -- 172.21.15.32:0/3044312467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a580a5a10 con 0x7f6a6015f110 2023-12-20T04:47:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a5dd9b640 1 -- 172.21.15.32:0/3044312467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a580a5da0 con 0x7f6a6015f110 2023-12-20T04:47:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 -- 172.21.15.32:0/3044312467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6015f110 msgr2=0x7f6a6015c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:16.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 --2- 172.21.15.32:0/3044312467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6015f110 0x7f6a6015c440 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f6a58009d90 tx=0x7f6a5801c850 comp rx=0 tx=0).stop 2023-12-20T04:47:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 -- 172.21.15.32:0/3044312467 shutdown_connections 2023-12-20T04:47:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 --2- 172.21.15.32:0/3044312467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a6015f110 0x7f6a6015c440 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 -- 172.21.15.32:0/3044312467 >> 172.21.15.32:0/3044312467 conn(0x7f6a6006a320 msgr2=0x7f6a600ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 -- 172.21.15.32:0/3044312467 shutdown_connections 2023-12-20T04:47:16.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 -- 172.21.15.32:0/3044312467 wait complete. 2023-12-20T04:47:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 Processor -- start 2023-12-20T04:47:16.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 -- start start 2023-12-20T04:47:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6006ce60 0x7f6a6014cc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a600af5e0 0x7f6a60149070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6a6015f110 0x7f6a60151680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a601495b0 con 0x7f6a600af5e0 2023-12-20T04:47:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6a60151bc0 con 0x7f6a6006ce60 2023-12-20T04:47:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.314+0000 7f6a648e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6a6014de80 con 0x7f6a6015f110 2023-12-20T04:47:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a55d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a600af5e0 0x7f6a60149070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a55d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a600af5e0 0x7f6a60149070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57058/0 (socket says 172.21.15.32:57058) 2023-12-20T04:47:16.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a55d9b640 1 -- 172.21.15.32:0/3151368200 learned_addr learned my addr 172.21.15.32:0/3151368200 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a5e59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6006ce60 0x7f6a6014cc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a5ed9d640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6a6015f110 0x7f6a60151680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a55d9b640 1 -- 172.21.15.32:0/3151368200 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6a6015f110 msgr2=0x7f6a60151680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a55d9b640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6a6015f110 0x7f6a60151680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a55d9b640 1 -- 172.21.15.32:0/3151368200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6006ce60 msgr2=0x7f6a6014cc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a55d9b640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6006ce60 0x7f6a6014cc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a55d9b640 1 -- 172.21.15.32:0/3151368200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6a58009a70 con 0x7f6a600af5e0 2023-12-20T04:47:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a55d9b640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a600af5e0 0x7f6a60149070 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c450 tx=0x7f6a4800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a577fe640 1 -- 172.21.15.32:0/3151368200 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48012830 con 0x7f6a600af5e0 2023-12-20T04:47:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a577fe640 1 -- 172.21.15.32:0/3151368200 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6a480129b0 con 0x7f6a600af5e0 2023-12-20T04:47:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a577fe640 1 -- 172.21.15.32:0/3151368200 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6a48012d60 con 0x7f6a600af5e0 2023-12-20T04:47:16.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a648e5640 1 -- 172.21.15.32:0/3151368200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6a6014d300 con 0x7f6a600af5e0 2023-12-20T04:47:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a648e5640 1 -- 172.21.15.32:0/3151368200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6a6014d8c0 con 0x7f6a600af5e0 2023-12-20T04:47:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a648e5640 1 -- 172.21.15.32:0/3151368200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6a6004e000 con 0x7f6a600af5e0 2023-12-20T04:47:16.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.318+0000 7f6a577fe640 1 -- 172.21.15.32:0/3151368200 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6a4801c460 con 0x7f6a600af5e0 2023-12-20T04:47:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.322+0000 7f6a577fe640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a40065b80 0x7f6a40068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.322+0000 7f6a577fe640 1 -- 172.21.15.32:0/3151368200 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(129..129 src has 1..129) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f6a48082ff0 con 0x7f6a600af5e0 2023-12-20T04:47:16.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.322+0000 7f6a5e59c640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a40065b80 0x7f6a40068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:16.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.322+0000 7f6a5e59c640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a40065b80 0x7f6a40068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6a50007fd0 tx=0x7f6a50009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:16.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.326+0000 7f6a577fe640 1 -- 172.21.15.32:0/3151368200 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6a480b40a0 con 0x7f6a600af5e0 2023-12-20T04:47:16.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.514+0000 7f6a648e5640 1 -- 172.21.15.32:0/3151368200 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6a600cf450 con 0x7f6a600af5e0 2023-12-20T04:47:16.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a577fe640 1 -- 172.21.15.32:0/3151368200 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 56 v56) v1 ==== 94+0+3520 (secure 0 0 0) 0x7f6a480509b0 con 0x7f6a600af5e0 2023-12-20T04:47:16.515 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:16.515 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":56,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":6869,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/593676281","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":593676281},{"type":"v1","addr":"172.21.15.39:6833","nonce":593676281}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":6878,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/2437945959","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":2437945959},{"type":"v1","addr":"172.21.15.39:6835","nonce":2437945959}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":7681,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/430674440","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":430674440},{"type":"v1","addr":"172.21.15.32:6837","nonce":430674440}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":49},{"gid":8307,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/860287533","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":860287533},{"type":"v1","addr":"172.21.15.32:6836","nonce":860287533}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":56}],"filesystems":[]} 2023-12-20T04:47:16.516 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 56 2023-12-20T04:47:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 -- 172.21.15.32:0/3151368200 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a40065b80 msgr2=0x7f6a40068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a40065b80 0x7f6a40068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6a50007fd0 tx=0x7f6a50009040 comp rx=0 tx=0).stop 2023-12-20T04:47:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 -- 172.21.15.32:0/3151368200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a600af5e0 msgr2=0x7f6a60149070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a600af5e0 0x7f6a60149070 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f6a4800c450 tx=0x7f6a4800c910 comp rx=0 tx=0).stop 2023-12-20T04:47:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 -- 172.21.15.32:0/3151368200 shutdown_connections 2023-12-20T04:47:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6a40065b80 0x7f6a40068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6a6015f110 0x7f6a60151680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6a600af5e0 0x7f6a60149070 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 --2- 172.21.15.32:0/3151368200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6a6006ce60 0x7f6a6014cc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 -- 172.21.15.32:0/3151368200 >> 172.21.15.32:0/3151368200 conn(0x7f6a6006a320 msgr2=0x7f6a601575c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 -- 172.21.15.32:0/3151368200 shutdown_connections 2023-12-20T04:47:16.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.518+0000 7f6a648e5640 1 -- 172.21.15.32:0/3151368200 wait complete. 2023-12-20T04:47:16.537 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T04:47:16.537 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T04:47:16.538 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T04:47:16.538 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:47:16.538 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T04:47:16.538 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T04:47:16.538 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T04:47:16.538 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:47:16.538 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T04:47:16.539 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T04:47:16.539 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T04:47:16.539 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:47:16.539 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T04:47:16.539 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T04:47:16.540 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T04:47:16.540 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T04:47:16.550 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T04:47:16.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T04:47:16.552 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T04:47:16.552 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T04:47:16.561 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T04:47:16.561 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T04:47:16.562 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T04:47:16.562 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T04:47:16.601 INFO:tasks.ceph.mds.a:Started 2023-12-20T04:47:16.601 DEBUG:teuthology.parallel:result is None 2023-12-20T04:47:16.604 INFO:tasks.ceph.mds.c:Started 2023-12-20T04:47:16.604 DEBUG:teuthology.parallel:result is None 2023-12-20T04:47:16.611 INFO:tasks.ceph.mds.d:Started 2023-12-20T04:47:16.612 DEBUG:teuthology.parallel:result is None 2023-12-20T04:47:16.614 INFO:tasks.ceph.mds.b:Started 2023-12-20T04:47:16.614 DEBUG:teuthology.parallel:result is None 2023-12-20T04:47:16.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T04:47:16.690 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T04:47:16.691 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T04:47:16.713 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T04:47:16.714 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T04:47:16.715 INFO:tasks.ceph.mds.d.smithi039.stderr:2023-12-20T04:47:16.717+0000 7f70d956a640 -1 PosixStack listen unable to listen on v2:0.0.0.0:6832/0: (98) Address already in use 2023-12-20T04:47:16.718 INFO:tasks.ceph.mds.b.smithi039.stderr:2023-12-20T04:47:16.721+0000 7fb86e779640 -1 PosixStack listen unable to listen on v1:0.0.0.0:6834/0: (98) Address already in use 2023-12-20T04:47:16.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b2b39640 1 Processor -- start 2023-12-20T04:47:16.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b2b39640 1 -- start start 2023-12-20T04:47:16.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b2b39640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f41ac069d20 0x7f41ac0cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:16.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b2b39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41ac15d080 0x7f41ac0cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:16.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b2b39640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41ac15d970 con 0x7f41ac15de00 2023-12-20T04:47:16.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b2b39640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41ac0ce740 con 0x7f41ac069d20 2023-12-20T04:47:16.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b2b39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac0cdf00 con 0x7f41ac15d080 2023-12-20T04:47:16.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41abfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41ac15d080 0x7f41ac0cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:16.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b08d5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f41ac069d20 0x7f41ac0cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:16.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41abfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41ac15d080 0x7f41ac0cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:37982/0 (socket says 172.21.15.32:37982) 2023-12-20T04:47:16.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b08d5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f41ac069d20 0x7f41ac0cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:42932/0 (socket says 172.21.15.32:42932) 2023-12-20T04:47:16.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41abfff640 1 -- 172.21.15.32:0/3928066468 learned_addr learned my addr 172.21.15.32:0/3928066468 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:16.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b08d5640 1 -- 172.21.15.32:0/3928066468 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41ac15d080 msgr2=0x7f41ac0cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:16.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b08d5640 1 --2- 172.21.15.32:0/3928066468 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41ac15d080 0x7f41ac0cd9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b08d5640 1 -- 172.21.15.32:0/3928066468 >> v1:172.21.15.39:6789/0 conn(0x7f41ac15de00 legacy=0x7f41ac06a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:16.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41b08d5640 1 -- 172.21.15.32:0/3928066468 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41ac0ce180 con 0x7f41ac069d20 2023-12-20T04:47:16.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.770+0000 7f41ab7fe640 1 -- 172.21.15.32:0/3928066468 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3076356016 0 0) 0x7f41ac15d970 con 0x7f41ac15de00 2023-12-20T04:47:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41b08d5640 1 --2- 172.21.15.32:0/3928066468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41ac069d20 0x7f41ac0cbbb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f41a00097c0 tx=0x7f41a001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c5f54805bcc7cfd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:16.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41ab7fe640 1 -- 172.21.15.32:0/3928066468 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a00ab650 con 0x7f41ac069d20 2023-12-20T04:47:16.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41ab7fe640 1 -- 172.21.15.32:0/3928066468 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a00ab7b0 con 0x7f41ac069d20 2023-12-20T04:47:16.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41ab7fe640 1 -- 172.21.15.32:0/3928066468 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a00abb40 con 0x7f41ac069d20 2023-12-20T04:47:16.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41b2b39640 1 -- 172.21.15.32:0/3928066468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41ac069d20 msgr2=0x7f41ac0cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:16.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41b2b39640 1 --2- 172.21.15.32:0/3928066468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41ac069d20 0x7f41ac0cbbb0 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f41a00097c0 tx=0x7f41a001c450 comp rx=0 tx=0).stop 2023-12-20T04:47:16.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41b2b39640 1 -- 172.21.15.32:0/3928066468 shutdown_connections 2023-12-20T04:47:16.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41b2b39640 1 --2- 172.21.15.32:0/3928066468 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41ac15d080 0x7f41ac0cd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41b2b39640 1 --2- 172.21.15.32:0/3928066468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41ac069d20 0x7f41ac0cbbb0 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41b2b39640 1 -- 172.21.15.32:0/3928066468 >> 172.21.15.32:0/3928066468 conn(0x7f41ac0cc310 msgr2=0x7f41ac0cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:16.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41b2b39640 1 -- 172.21.15.32:0/3928066468 shutdown_connections 2023-12-20T04:47:16.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.774+0000 7f41b2b39640 1 -- 172.21.15.32:0/3928066468 wait complete. 2023-12-20T04:47:16.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b2b39640 1 Processor -- start 2023-12-20T04:47:16.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b2b39640 1 -- start start 2023-12-20T04:47:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b2b39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41ac069d20 0x7f41ac15c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b2b39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41ac15d080 0x7f41ac0c6960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b2b39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41ac15de00 0x7f41ac0b1b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b2b39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac148b50 con 0x7f41ac15d080 2023-12-20T04:47:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b2b39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac0b03d0 con 0x7f41ac069d20 2023-12-20T04:47:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b2b39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41ac0b06b0 con 0x7f41ac15de00 2023-12-20T04:47:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41abfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41ac15d080 0x7f41ac0c6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:16.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b08d5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41ac069d20 0x7f41ac15c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41abfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41ac15d080 0x7f41ac0c6960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57110/0 (socket says 172.21.15.32:57110) 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41abfff640 1 -- 172.21.15.32:0/2339430698 learned_addr learned my addr 172.21.15.32:0/2339430698 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b10d6640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41ac15de00 0x7f41ac0b1b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41abfff640 1 -- 172.21.15.32:0/2339430698 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41ac15de00 msgr2=0x7f41ac0b1b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41abfff640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41ac15de00 0x7f41ac0b1b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41abfff640 1 -- 172.21.15.32:0/2339430698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41ac069d20 msgr2=0x7f41ac15c200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41abfff640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41ac069d20 0x7f41ac15c200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41abfff640 1 -- 172.21.15.32:0/2339430698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a0009420 con 0x7f41ac15d080 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b08d5640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41ac069d20 0x7f41ac15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41abfff640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41ac15d080 0x7f41ac0c6960 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f419800e890 tx=0x7f419800ed50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41a97fa640 1 -- 172.21.15.32:0/2339430698 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4198009620 con 0x7f41ac15d080 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41a97fa640 1 -- 172.21.15.32:0/2339430698 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f419800c940 con 0x7f41ac15d080 2023-12-20T04:47:16.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41a97fa640 1 -- 172.21.15.32:0/2339430698 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f419800ccf0 con 0x7f41ac15d080 2023-12-20T04:47:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.778+0000 7f41b2b39640 1 -- 172.21.15.32:0/2339430698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41ac0c43d0 con 0x7f41ac15d080 2023-12-20T04:47:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.782+0000 7f41b2b39640 1 -- 172.21.15.32:0/2339430698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41ac0c4990 con 0x7f41ac15d080 2023-12-20T04:47:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.782+0000 7f41b2b39640 1 -- 172.21.15.32:0/2339430698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41ac0c4230 con 0x7f41ac15d080 2023-12-20T04:47:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.782+0000 7f41a97fa640 1 -- 172.21.15.32:0/2339430698 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4198013470 con 0x7f41ac15d080 2023-12-20T04:47:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.782+0000 7f41a97fa640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4190065b80 0x7f4190068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.782+0000 7f41a97fa640 1 -- 172.21.15.32:0/2339430698 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(133..133 src has 1..133) v4 ==== 5659+0+0 (secure 0 0 0) 0x7f4198083370 con 0x7f41ac15d080 2023-12-20T04:47:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.782+0000 7f41b08d5640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4190065b80 0x7f4190068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:16.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.782+0000 7f41b08d5640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4190065b80 0x7f4190068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f41a00af000 tx=0x7f41a00a7000 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:16.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.786+0000 7f41a97fa640 1 -- 172.21.15.32:0/2339430698 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f41ac0c4230 con 0x7f41ac15d080 2023-12-20T04:47:16.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:16.954+0000 7f41b2b39640 1 -- 172.21.15.32:0/2339430698 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f41ac0c4230 con 0x7f41ac15d080 2023-12-20T04:47:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.762+0000 7f41a97fa640 1 -- 172.21.15.32:0/2339430698 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v134) v1 ==== 97+0+0 (secure 0 0 0) 0x7f41ac0c4230 con 0x7f41ac15d080 2023-12-20T04:47:17.761 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T04:47:17.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 -- 172.21.15.32:0/2339430698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4190065b80 msgr2=0x7f4190068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:17.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4190065b80 0x7f4190068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f41a00af000 tx=0x7f41a00a7000 comp rx=0 tx=0).stop 2023-12-20T04:47:17.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 -- 172.21.15.32:0/2339430698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41ac15d080 msgr2=0x7f41ac0c6960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:17.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41ac15d080 0x7f41ac0c6960 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f419800e890 tx=0x7f419800ed50 comp rx=0 tx=0).stop 2023-12-20T04:47:17.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 -- 172.21.15.32:0/2339430698 shutdown_connections 2023-12-20T04:47:17.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4190065b80 0x7f4190068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:17.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41ac15de00 0x7f41ac0b1b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:17.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41ac15d080 0x7f41ac0c6960 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:17.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 --2- 172.21.15.32:0/2339430698 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41ac069d20 0x7f41ac15c200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:17.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 -- 172.21.15.32:0/2339430698 >> 172.21.15.32:0/2339430698 conn(0x7f41ac0cc310 msgr2=0x7f41ac156240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:17.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 -- 172.21.15.32:0/2339430698 shutdown_connections 2023-12-20T04:47:17.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.766+0000 7f41b2b39640 1 -- 172.21.15.32:0/2339430698 wait complete. 2023-12-20T04:47:17.787 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T04:47:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.938+0000 7f45329c0640 1 Processor -- start 2023-12-20T04:47:17.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.938+0000 7f45329c0640 1 -- start start 2023-12-20T04:47:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.938+0000 7f45329c0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f452c0aece0 0x7f452c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.938+0000 7f45329c0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452c0b0b30 0x7f452c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.938+0000 7f45329c0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f452c0c98c0 con 0x7f452c1599e0 2023-12-20T04:47:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.938+0000 7f45329c0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452c0c9ba0 con 0x7f452c0aece0 2023-12-20T04:47:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.938+0000 7f45329c0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c0c9e80 con 0x7f452c0b0b30 2023-12-20T04:47:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.938+0000 7f452b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f452c0aece0 0x7f452c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:17.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.938+0000 7f452bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f452c1599e0 0x7f452c0c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:57150/0 (socket says 172.21.15.32:57150) 2023-12-20T04:47:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.938+0000 7f452bfff640 1 -- 172.21.15.32:0/293940642 learned_addr learned my addr 172.21.15.32:0/293940642 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f4530f5d640 1 --2- 172.21.15.32:0/293940642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452c0b0b30 0x7f452c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:17.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f452affd640 1 -- 172.21.15.32:0/293940642 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3182660273 0 0) 0x7f452c0c98c0 con 0x7f452c1599e0 2023-12-20T04:47:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f452affd640 1 -- 172.21.15.32:0/293940642 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4514003530 con 0x7f452c1599e0 2023-12-20T04:47:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f452b7fe640 1 -- 172.21.15.32:0/293940642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452c0b0b30 msgr2=0x7f452c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f452b7fe640 1 --2- 172.21.15.32:0/293940642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452c0b0b30 0x7f452c0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f452b7fe640 1 -- 172.21.15.32:0/293940642 >> v1:172.21.15.32:6789/0 conn(0x7f452c1599e0 legacy=0x7f452c0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:17.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f452b7fe640 1 -- 172.21.15.32:0/293940642 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f452c0ca160 con 0x7f452c0aece0 2023-12-20T04:47:17.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f452b7fe640 1 --2- 172.21.15.32:0/293940642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f452c0aece0 0x7f452c0b05f0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f45200097c0 tx=0x7f452001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45876119dfd36bc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:17.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f452affd640 1 -- 172.21.15.32:0/293940642 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200ad640 con 0x7f452c0aece0 2023-12-20T04:47:17.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f452affd640 1 -- 172.21.15.32:0/293940642 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45200ad7a0 con 0x7f452c0aece0 2023-12-20T04:47:17.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f452affd640 1 -- 172.21.15.32:0/293940642 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200adb30 con 0x7f452c0aece0 2023-12-20T04:47:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f45329c0640 1 -- 172.21.15.32:0/293940642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f452c0aece0 msgr2=0x7f452c0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f45329c0640 1 --2- 172.21.15.32:0/293940642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f452c0aece0 0x7f452c0b05f0 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f45200097c0 tx=0x7f452001c480 comp rx=0 tx=0).stop 2023-12-20T04:47:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f45329c0640 1 -- 172.21.15.32:0/293940642 shutdown_connections 2023-12-20T04:47:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f45329c0640 1 --2- 172.21.15.32:0/293940642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f452c0aece0 0x7f452c0b05f0 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f45329c0640 1 --2- 172.21.15.32:0/293940642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452c0b0b30 0x7f452c0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.942+0000 7f45329c0640 1 -- 172.21.15.32:0/293940642 >> 172.21.15.32:0/293940642 conn(0x7f452c06a320 msgr2=0x7f452c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f45329c0640 1 -- 172.21.15.32:0/293940642 shutdown_connections 2023-12-20T04:47:17.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f45329c0640 1 -- 172.21.15.32:0/293940642 wait complete. 2023-12-20T04:47:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f45329c0640 1 Processor -- start 2023-12-20T04:47:17.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f45329c0640 1 -- start start 2023-12-20T04:47:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f45329c0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f452c0aece0 0x7f452c14fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:17.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f45329c0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f452c0b0b30 0x7f452c14c550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f45329c0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452c1599e0 0x7f452c154ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f45329c0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c150150 con 0x7f452c0b0b30 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f45329c0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452c153a60 con 0x7f452c0aece0 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f45329c0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f452c153d40 con 0x7f452c1599e0 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f452b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f452c0b0b30 0x7f452c14c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f452b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f452c0b0b30 0x7f452c14c550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57122/0 (socket says 172.21.15.32:57122) 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f452b7fe640 1 -- 172.21.15.32:0/4232675202 learned_addr learned my addr 172.21.15.32:0/4232675202 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f4530f5d640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452c1599e0 0x7f452c154ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f452bfff640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f452c0aece0 0x7f452c14fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f452b7fe640 1 -- 172.21.15.32:0/4232675202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452c1599e0 msgr2=0x7f452c154ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f452b7fe640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452c1599e0 0x7f452c154ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f452b7fe640 1 -- 172.21.15.32:0/4232675202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f452c0aece0 msgr2=0x7f452c14fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:17.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f452b7fe640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f452c0aece0 0x7f452c14fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f452b7fe640 1 -- 172.21.15.32:0/4232675202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4520009420 con 0x7f452c0b0b30 2023-12-20T04:47:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.946+0000 7f452b7fe640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f452c0b0b30 0x7f452c14c550 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f452001a0b0 tx=0x7f4520019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.950+0000 7f4528ff9640 1 -- 172.21.15.32:0/4232675202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200b6610 con 0x7f452c0b0b30 2023-12-20T04:47:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.950+0000 7f4528ff9640 1 -- 172.21.15.32:0/4232675202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45200a6040 con 0x7f452c0b0b30 2023-12-20T04:47:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.950+0000 7f4528ff9640 1 -- 172.21.15.32:0/4232675202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45200b6990 con 0x7f452c0b0b30 2023-12-20T04:47:17.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.950+0000 7f45329c0640 1 -- 172.21.15.32:0/4232675202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f452c154020 con 0x7f452c0b0b30 2023-12-20T04:47:17.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.950+0000 7f45329c0640 1 -- 172.21.15.32:0/4232675202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452c14ca90 con 0x7f452c0b0b30 2023-12-20T04:47:17.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.950+0000 7f45329c0640 1 -- 172.21.15.32:0/4232675202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f452c0ce2d0 con 0x7f452c0b0b30 2023-12-20T04:47:17.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.950+0000 7f4528ff9640 1 -- 172.21.15.32:0/4232675202 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f45200b4040 con 0x7f452c0b0b30 2023-12-20T04:47:17.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.950+0000 7f4528ff9640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4518065b80 0x7f4518068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:17.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.950+0000 7f4528ff9640 1 -- 172.21.15.32:0/4232675202 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f452011d1e0 con 0x7f452c0b0b30 2023-12-20T04:47:17.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.950+0000 7f452bfff640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4518065b80 0x7f4518068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:17.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.954+0000 7f452bfff640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4518065b80 0x7f4518068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f451c007fd0 tx=0x7f451c009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:17.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:17.954+0000 7f4528ff9640 1 -- 172.21.15.32:0/4232675202 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f45200eadc0 con 0x7f452c0b0b30 2023-12-20T04:47:18.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f45329c0640 1 -- 172.21.15.32:0/4232675202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f452c04e000 con 0x7f452c0b0b30 2023-12-20T04:47:18.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f4528ff9640 1 -- 172.21.15.32:0/4232675202 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v21) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f452011be60 con 0x7f452c0b0b30 2023-12-20T04:47:18.168 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.169 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.170 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:47:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:47:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:47:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T04:47:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T04:47:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:47:18.171 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.172 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:47:18.173 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T04:47:18.174 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.175 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.176 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T04:47:18.177 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T04:47:18.178 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T04:47:18.179 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.180 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T04:47:18.181 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:18.182 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T04:47:18.182 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:47:18.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f45329c0640 1 -- 172.21.15.32:0/4232675202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4518065b80 msgr2=0x7f4518068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f45329c0640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4518065b80 0x7f4518068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f451c007fd0 tx=0x7f451c009040 comp rx=0 tx=0).stop 2023-12-20T04:47:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f45329c0640 1 -- 172.21.15.32:0/4232675202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f452c0b0b30 msgr2=0x7f452c14c550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f45329c0640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f452c0b0b30 0x7f452c14c550 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f452001a0b0 tx=0x7f4520019220 comp rx=0 tx=0).stop 2023-12-20T04:47:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f45329c0640 1 -- 172.21.15.32:0/4232675202 shutdown_connections 2023-12-20T04:47:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f45329c0640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4518065b80 0x7f4518068040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f45329c0640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452c1599e0 0x7f452c154ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f45329c0640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f452c0b0b30 0x7f452c14c550 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f45329c0640 1 --2- 172.21.15.32:0/4232675202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f452c0aece0 0x7f452c14fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.170+0000 7f45329c0640 1 -- 172.21.15.32:0/4232675202 >> 172.21.15.32:0/4232675202 conn(0x7f452c06a320 msgr2=0x7f452c06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.174+0000 7f45329c0640 1 -- 172.21.15.32:0/4232675202 shutdown_connections 2023-12-20T04:47:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.174+0000 7f45329c0640 1 -- 172.21.15.32:0/4232675202 wait complete. 2023-12-20T04:47:18.190 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:47:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.334+0000 7f7873ad1640 1 Processor -- start 2023-12-20T04:47:18.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.334+0000 7f7873ad1640 1 -- start start 2023-12-20T04:47:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.334+0000 7f7873ad1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c0af5e0 0x7f786c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.334+0000 7f7873ad1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f786c15f110 0x7f786c15c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.334+0000 7f7873ad1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f786c15b990 con 0x7f786c06ce60 2023-12-20T04:47:18.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.334+0000 7f7873ad1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f786c15bbd0 con 0x7f786c15f110 2023-12-20T04:47:18.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.334+0000 7f7873ad1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c15beb0 con 0x7f786c0af5e0 2023-12-20T04:47:18.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787186d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c0af5e0 0x7f786c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787206e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f786c15f110 0x7f786c15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787106c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f786c06ce60 0x7f786c15b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:40866/0 (socket says 172.21.15.32:40866) 2023-12-20T04:47:18.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787186d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c0af5e0 0x7f786c0af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:38026/0 (socket says 172.21.15.32:38026) 2023-12-20T04:47:18.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787206e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f786c15f110 0x7f786c15c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:42966/0 (socket says 172.21.15.32:42966) 2023-12-20T04:47:18.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787106c640 1 -- 172.21.15.32:0/2049765774 learned_addr learned my addr 172.21.15.32:0/2049765774 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:18.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787186d640 1 -- 172.21.15.32:0/2049765774 >> v2:172.21.15.39:3300/0 conn(0x7f786c15f110 msgr2=0x7f786c15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:18.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787186d640 1 --2- 172.21.15.32:0/2049765774 >> v2:172.21.15.39:3300/0 conn(0x7f786c15f110 0x7f786c15c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787186d640 1 -- 172.21.15.32:0/2049765774 >> v1:172.21.15.39:6789/0 conn(0x7f786c06ce60 legacy=0x7f786c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787186d640 1 -- 172.21.15.32:0/2049765774 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f786c15e910 con 0x7f786c0af5e0 2023-12-20T04:47:18.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787086b640 1 -- 172.21.15.32:0/2049765774 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 70813266 0 0) 0x7f785c001120 con 0x7f786c06ce60 2023-12-20T04:47:18.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787206e640 1 --2- 172.21.15.32:0/2049765774 >> v2:172.21.15.39:3300/0 conn(0x7f786c15f110 0x7f786c15c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:18.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.338+0000 7f787186d640 1 --2- 172.21.15.32:0/2049765774 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c0af5e0 0x7f786c0af9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f78680097a0 tx=0x7f786801c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=94ffd0fd79e68628 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:18.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f787086b640 1 -- 172.21.15.32:0/2049765774 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680ab650 con 0x7f786c0af5e0 2023-12-20T04:47:18.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f787086b640 1 -- 172.21.15.32:0/2049765774 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f78680ab7b0 con 0x7f786c0af5e0 2023-12-20T04:47:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f787086b640 1 -- 172.21.15.32:0/2049765774 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f78680abb40 con 0x7f786c0af5e0 2023-12-20T04:47:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 -- 172.21.15.32:0/2049765774 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c0af5e0 msgr2=0x7f786c0af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 --2- 172.21.15.32:0/2049765774 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c0af5e0 0x7f786c0af9c0 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f78680097a0 tx=0x7f786801c430 comp rx=0 tx=0).stop 2023-12-20T04:47:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 -- 172.21.15.32:0/2049765774 shutdown_connections 2023-12-20T04:47:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 --2- 172.21.15.32:0/2049765774 >> v2:172.21.15.39:3300/0 conn(0x7f786c15f110 0x7f786c15c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 --2- 172.21.15.32:0/2049765774 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c0af5e0 0x7f786c0af9c0 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 -- 172.21.15.32:0/2049765774 >> 172.21.15.32:0/2049765774 conn(0x7f786c06a320 msgr2=0x7f786c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 -- 172.21.15.32:0/2049765774 shutdown_connections 2023-12-20T04:47:18.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 -- 172.21.15.32:0/2049765774 wait complete. 2023-12-20T04:47:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 Processor -- start 2023-12-20T04:47:18.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 -- start start 2023-12-20T04:47:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f786c06ce60 0x7f786c1f67e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f786c0af5e0 0x7f786c156c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c15f110 0x7f786c14da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c071bf0 con 0x7f786c0af5e0 2023-12-20T04:47:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f786c1f6d20 con 0x7f786c06ce60 2023-12-20T04:47:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f7873ad1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f786c14f910 con 0x7f786c15f110 2023-12-20T04:47:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f787106c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f786c0af5e0 0x7f786c156c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f787106c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f786c0af5e0 0x7f786c156c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57134/0 (socket says 172.21.15.32:57134) 2023-12-20T04:47:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.342+0000 7f787106c640 1 -- 172.21.15.32:0/3385316893 learned_addr learned my addr 172.21.15.32:0/3385316893 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:18.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f787186d640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f786c06ce60 0x7f786c1f67e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f787206e640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c15f110 0x7f786c14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f787106c640 1 -- 172.21.15.32:0/3385316893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c15f110 msgr2=0x7f786c14da80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f787106c640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c15f110 0x7f786c14da80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f787106c640 1 -- 172.21.15.32:0/3385316893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f786c06ce60 msgr2=0x7f786c1f67e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f787106c640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f786c06ce60 0x7f786c1f67e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f787106c640 1 -- 172.21.15.32:0/3385316893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7864009a70 con 0x7f786c0af5e0 2023-12-20T04:47:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f787206e640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c15f110 0x7f786c14da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f787186d640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f786c06ce60 0x7f786c1f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:18.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f787106c640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f786c0af5e0 0x7f786c156c10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c7b0 tx=0x7f785c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f78627fc640 1 -- 172.21.15.32:0/3385316893 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c0072d0 con 0x7f786c0af5e0 2023-12-20T04:47:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f78627fc640 1 -- 172.21.15.32:0/3385316893 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f785c007450 con 0x7f786c0af5e0 2023-12-20T04:47:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f78627fc640 1 -- 172.21.15.32:0/3385316893 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f785c0047f0 con 0x7f786c0af5e0 2023-12-20T04:47:18.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f7873ad1640 1 -- 172.21.15.32:0/3385316893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f78680093b0 con 0x7f786c0af5e0 2023-12-20T04:47:18.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f7873ad1640 1 -- 172.21.15.32:0/3385316893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f786c0ce500 con 0x7f786c0af5e0 2023-12-20T04:47:18.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f7873ad1640 1 -- 172.21.15.32:0/3385316893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f786c071da0 con 0x7f786c0af5e0 2023-12-20T04:47:18.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.346+0000 7f78627fc640 1 -- 172.21.15.32:0/3385316893 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f785c004950 con 0x7f786c0af5e0 2023-12-20T04:47:18.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.350+0000 7f78627fc640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7850065b80 0x7f7850068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.350+0000 7f78627fc640 1 -- 172.21.15.32:0/3385316893 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f785c081fe0 con 0x7f786c0af5e0 2023-12-20T04:47:18.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.350+0000 7f787186d640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7850065b80 0x7f7850068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.350+0000 7f787186d640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7850065b80 0x7f7850068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78680af000 tx=0x7f78680a7000 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:18.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.354+0000 7f78627fc640 1 -- 172.21.15.32:0/3385316893 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f786c071da0 con 0x7f786c0af5e0 2023-12-20T04:47:18.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.510+0000 7f7873ad1640 1 -- 172.21.15.32:0/3385316893 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f786c15b990 con 0x7f786c0af5e0 2023-12-20T04:47:18.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f78627fc640 1 -- 172.21.15.32:0/3385316893 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v134) v1 ==== 74+0+12395 (secure 0 0 0) 0x7f786c15b990 con 0x7f786c0af5e0 2023-12-20T04:47:18.511 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:18.511 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":134,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:47:17.755631+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":15,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":121,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:47:18.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 -- 172.21.15.32:0/3385316893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7850065b80 msgr2=0x7f7850068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7850065b80 0x7f7850068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f78680af000 tx=0x7f78680a7000 comp rx=0 tx=0).stop 2023-12-20T04:47:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 -- 172.21.15.32:0/3385316893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f786c0af5e0 msgr2=0x7f786c156c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f786c0af5e0 0x7f786c156c10 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f785c00c7b0 tx=0x7f785c00cc70 comp rx=0 tx=0).stop 2023-12-20T04:47:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 -- 172.21.15.32:0/3385316893 shutdown_connections 2023-12-20T04:47:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7850065b80 0x7f7850068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f786c15f110 0x7f786c14da80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f786c0af5e0 0x7f786c156c10 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 --2- 172.21.15.32:0/3385316893 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f786c06ce60 0x7f786c1f67e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 -- 172.21.15.32:0/3385316893 >> 172.21.15.32:0/3385316893 conn(0x7f786c06a320 msgr2=0x7f786c164ca0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 -- 172.21.15.32:0/3385316893 shutdown_connections 2023-12-20T04:47:18.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.514+0000 7f7873ad1640 1 -- 172.21.15.32:0/3385316893 wait complete. 2023-12-20T04:47:18.532 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T04:47:18.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T04:47:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.682+0000 7f023e760640 1 Processor -- start 2023-12-20T04:47:18.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.682+0000 7f023e760640 1 -- start start 2023-12-20T04:47:18.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.682+0000 7f023e760640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f023815abb0 0x7f02380caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.682+0000 7f023e760640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0238070230 0x7f02380706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.682+0000 7f023e760640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f023815b6a0 con 0x7f02380b0cb0 2023-12-20T04:47:18.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.682+0000 7f023e760640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f023806e160 con 0x7f0238070230 2023-12-20T04:47:18.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.682+0000 7f023e760640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0238070be0 con 0x7f023815abb0 2023-12-20T04:47:18.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f023ccfd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0238070230 0x7f02380706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f0237fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f023815abb0 0x7f02380caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f02377fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f02380b0cb0 0x7f023815bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:40868/0 (socket says 172.21.15.32:40868) 2023-12-20T04:47:18.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f02377fe640 1 -- 172.21.15.32:0/2310710988 learned_addr learned my addr 172.21.15.32:0/2310710988 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:18.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f0236ffd640 1 -- 172.21.15.32:0/2310710988 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 340054474 0 0) 0x7f023815b6a0 con 0x7f02380b0cb0 2023-12-20T04:47:18.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f0236ffd640 1 -- 172.21.15.32:0/2310710988 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0220003530 con 0x7f02380b0cb0 2023-12-20T04:47:18.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f023ccfd640 1 -- 172.21.15.32:0/2310710988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f023815abb0 msgr2=0x7f02380caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:18.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f023ccfd640 1 --2- 172.21.15.32:0/2310710988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f023815abb0 0x7f02380caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f023ccfd640 1 -- 172.21.15.32:0/2310710988 >> v1:172.21.15.39:6789/0 conn(0x7f02380b0cb0 legacy=0x7f023815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f023ccfd640 1 -- 172.21.15.32:0/2310710988 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f023806d8c0 con 0x7f0238070230 2023-12-20T04:47:18.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f0237fff640 1 --2- 172.21.15.32:0/2310710988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f023815abb0 0x7f02380caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:18.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f023ccfd640 1 --2- 172.21.15.32:0/2310710988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0238070230 0x7f02380706a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0230009d90 tx=0x7f023001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=115718031476cd4e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:18.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f0236ffd640 1 -- 172.21.15.32:0/2310710988 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02300a5a10 con 0x7f0238070230 2023-12-20T04:47:18.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f0236ffd640 1 -- 172.21.15.32:0/2310710988 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02300a5b70 con 0x7f0238070230 2023-12-20T04:47:18.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.686+0000 7f0236ffd640 1 -- 172.21.15.32:0/2310710988 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02300ad440 con 0x7f0238070230 2023-12-20T04:47:18.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 -- 172.21.15.32:0/2310710988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0238070230 msgr2=0x7f02380706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 --2- 172.21.15.32:0/2310710988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0238070230 0x7f02380706a0 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f0230009d90 tx=0x7f023001c950 comp rx=0 tx=0).stop 2023-12-20T04:47:18.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 -- 172.21.15.32:0/2310710988 shutdown_connections 2023-12-20T04:47:18.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 --2- 172.21.15.32:0/2310710988 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0238070230 0x7f02380706a0 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 --2- 172.21.15.32:0/2310710988 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f023815abb0 0x7f02380caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 -- 172.21.15.32:0/2310710988 >> 172.21.15.32:0/2310710988 conn(0x7f023806a510 msgr2=0x7f02380ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:18.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 -- 172.21.15.32:0/2310710988 shutdown_connections 2023-12-20T04:47:18.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 -- 172.21.15.32:0/2310710988 wait complete. 2023-12-20T04:47:18.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 Processor -- start 2023-12-20T04:47:18.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 -- start start 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0238070230 0x7f0238142e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02380b0cb0 0x7f023813c580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f023815abb0 0x7f0238153310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f023813cac0 con 0x7f0238070230 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0238140f20 con 0x7f023815abb0 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023e760640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0238141200 con 0x7f02380b0cb0 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f0237fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0238070230 0x7f0238142e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f0237fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0238070230 0x7f0238142e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57150/0 (socket says 172.21.15.32:57150) 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f0237fff640 1 -- 172.21.15.32:0/2592080913 learned_addr learned my addr 172.21.15.32:0/2592080913 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f023ccfd640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f023815abb0 0x7f0238153310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.690+0000 7f02377fe640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02380b0cb0 0x7f023813c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0237fff640 1 -- 172.21.15.32:0/2592080913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02380b0cb0 msgr2=0x7f023813c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0237fff640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02380b0cb0 0x7f023813c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0237fff640 1 -- 172.21.15.32:0/2592080913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f023815abb0 msgr2=0x7f0238153310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0237fff640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f023815abb0 0x7f0238153310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0237fff640 1 -- 172.21.15.32:0/2592080913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0228009420 con 0x7f0238070230 2023-12-20T04:47:18.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0237fff640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0238070230 0x7f0238142e40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f022800e980 tx=0x7f022800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:18.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0234ff9640 1 -- 172.21.15.32:0/2592080913 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f022800ca00 con 0x7f0238070230 2023-12-20T04:47:18.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0234ff9640 1 -- 172.21.15.32:0/2592080913 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f022800cb60 con 0x7f0238070230 2023-12-20T04:47:18.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0234ff9640 1 -- 172.21.15.32:0/2592080913 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0228013470 con 0x7f0238070230 2023-12-20T04:47:18.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f023e760640 1 -- 172.21.15.32:0/2592080913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0230009a70 con 0x7f0238070230 2023-12-20T04:47:18.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f023e760640 1 -- 172.21.15.32:0/2592080913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0238148ff0 con 0x7f0238070230 2023-12-20T04:47:18.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f023e760640 1 -- 172.21.15.32:0/2592080913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0238148e50 con 0x7f0238070230 2023-12-20T04:47:18.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0234ff9640 1 -- 172.21.15.32:0/2592080913 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f02280135d0 con 0x7f0238070230 2023-12-20T04:47:18.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0234ff9640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0224065b80 0x7f0224068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:18.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f0234ff9640 1 -- 172.21.15.32:0/2592080913 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f02280831e0 con 0x7f0238070230 2023-12-20T04:47:18.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.694+0000 7f02377fe640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0224065b80 0x7f0224068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:18.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.698+0000 7f02377fe640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0224065b80 0x7f0224068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0238145260 tx=0x7f022c007d40 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:18.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.698+0000 7f0234ff9640 1 -- 172.21.15.32:0/2592080913 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0238148e50 con 0x7f0238070230 2023-12-20T04:47:18.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f023e760640 1 -- 172.21.15.32:0/2592080913 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f0238148e50 con 0x7f0238070230 2023-12-20T04:47:18.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f0234ff9640 1 -- 172.21.15.32:0/2592080913 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v134) v1 ==== 93+0+10 (secure 0 0 0) 0x7f0238148e50 con 0x7f0238070230 2023-12-20T04:47:18.856 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T04:47:18.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f023e760640 1 -- 172.21.15.32:0/2592080913 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0224065b80 msgr2=0x7f0224068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f023e760640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0224065b80 0x7f0224068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f0238145260 tx=0x7f022c007d40 comp rx=0 tx=0).stop 2023-12-20T04:47:18.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f023e760640 1 -- 172.21.15.32:0/2592080913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0238070230 msgr2=0x7f0238142e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:18.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f023e760640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0238070230 0x7f0238142e40 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f022800e980 tx=0x7f022800ee40 comp rx=0 tx=0).stop 2023-12-20T04:47:18.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f023e760640 1 -- 172.21.15.32:0/2592080913 shutdown_connections 2023-12-20T04:47:18.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f023e760640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0224065b80 0x7f0224068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f023e760640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f023815abb0 0x7f0238153310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f023e760640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02380b0cb0 0x7f023813c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f023e760640 1 --2- 172.21.15.32:0/2592080913 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0238070230 0x7f0238142e40 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:18.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.858+0000 7f023e760640 1 -- 172.21.15.32:0/2592080913 >> 172.21.15.32:0/2592080913 conn(0x7f023806a510 msgr2=0x7f0238154a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:18.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.862+0000 7f023e760640 1 -- 172.21.15.32:0/2592080913 shutdown_connections 2023-12-20T04:47:18.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:18.862+0000 7f023e760640 1 -- 172.21.15.32:0/2592080913 wait complete. 2023-12-20T04:47:18.878 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T04:47:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.022+0000 7f5877a95640 1 Processor -- start 2023-12-20T04:47:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.022+0000 7f5877a95640 1 -- start start 2023-12-20T04:47:19.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5877a95640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f58700af5e0 0x7f58700af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5877a95640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587015b990 con 0x7f587006ce60 2023-12-20T04:47:19.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5877a95640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f587015bbd0 con 0x7f587015f110 2023-12-20T04:47:19.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5877a95640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f587015beb0 con 0x7f58700af5e0 2023-12-20T04:47:19.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5875030640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f587006ce60 0x7f587015b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43884/0 (socket says 172.21.15.32:43884) 2023-12-20T04:47:19.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5875030640 1 -- 172.21.15.32:0/621489188 learned_addr learned my addr 172.21.15.32:0/621489188 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5875831640 1 --2- 172.21.15.32:0/621489188 >> v2:172.21.15.39:3300/0 conn(0x7f58700af5e0 0x7f58700af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 179332398 0 0) 0x7f5860001120 con 0x7f587006ce60 2023-12-20T04:47:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f585c003530 con 0x7f587006ce60 2023-12-20T04:47:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1163274982 0 0) 0x7f585c003530 con 0x7f587006ce60 2023-12-20T04:47:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 >> v2:172.21.15.39:3300/0 conn(0x7f58700af5e0 msgr2=0x7f58700af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 --2- 172.21.15.32:0/621489188 >> v2:172.21.15.39:3300/0 conn(0x7f58700af5e0 0x7f58700af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 >> v1:172.21.15.39:6789/0 conn(0x7f587015f110 legacy=0x7f587015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f587015e910 con 0x7f587006ce60 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f5860001810 con 0x7f587006ce60 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 >> v1:172.21.15.32:6789/0 conn(0x7f587006ce60 legacy=0x7f587015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 --2- 172.21.15.32:0/621489188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f585c003980 0x7f585c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 --2- 172.21.15.32:0/621489188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f585c0047d0 0x7f585c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 --2- 172.21.15.32:0/621489188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f585c005190 0x7f585c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f587015beb0 con 0x7f585c005190 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f585c00c030 con 0x7f585c0047d0 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f585c00c1f0 con 0x7f585c003980 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5876032640 1 --2- 172.21.15.32:0/621489188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f585c005190 0x7f585c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5875030640 1 --2- 172.21.15.32:0/621489188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f585c0047d0 0x7f585c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5875831640 1 --2- 172.21.15.32:0/621489188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f585c003980 0x7f585c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5876032640 1 -- 172.21.15.32:0/621489188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f585c003980 msgr2=0x7f585c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5876032640 1 --2- 172.21.15.32:0/621489188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f585c003980 0x7f585c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5876032640 1 -- 172.21.15.32:0/621489188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f585c0047d0 msgr2=0x7f585c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:19.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5876032640 1 --2- 172.21.15.32:0/621489188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f585c0047d0 0x7f585c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.026+0000 7f5876032640 1 -- 172.21.15.32:0/621489188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f585c003530 con 0x7f585c005190 2023-12-20T04:47:19.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f5876032640 1 -- 172.21.15.32:0/621489188 reap_dead start 2023-12-20T04:47:19.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f5876032640 1 --2- 172.21.15.32:0/621489188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f585c005190 0x7f585c00b990 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f5868001e70 tx=0x7f586801c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30bdee0847b5dd41 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:19.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868003b80 con 0x7f585c005190 2023-12-20T04:47:19.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5868003d00 con 0x7f585c005190 2023-12-20T04:47:19.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f587482f640 1 -- 172.21.15.32:0/621489188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5868005260 con 0x7f585c005190 2023-12-20T04:47:19.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f5877a95640 1 -- 172.21.15.32:0/621489188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f585c005190 msgr2=0x7f585c00b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f5877a95640 1 --2- 172.21.15.32:0/621489188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f585c005190 0x7f585c00b990 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7f5868001e70 tx=0x7f586801c760 comp rx=0 tx=0).stop 2023-12-20T04:47:19.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f5877a95640 1 -- 172.21.15.32:0/621489188 shutdown_connections 2023-12-20T04:47:19.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f5877a95640 1 --2- 172.21.15.32:0/621489188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f585c005190 0x7f585c00b990 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f5877a95640 1 -- 172.21.15.32:0/621489188 >> 172.21.15.32:0/621489188 conn(0x7f587006a320 msgr2=0x7f58700ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:19.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f5877a95640 1 -- 172.21.15.32:0/621489188 shutdown_connections 2023-12-20T04:47:19.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.030+0000 7f5877a95640 1 -- 172.21.15.32:0/621489188 wait complete. 2023-12-20T04:47:19.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5877a95640 1 Processor -- start 2023-12-20T04:47:19.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5877a95640 1 -- start start 2023-12-20T04:47:19.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5877a95640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f585c005190 0x7f5870151740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5877a95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587015dee0 0x7f5870071210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5877a95640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f587015e360 0x7f5870071750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5877a95640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5870159b60 con 0x7f587015dee0 2023-12-20T04:47:19.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5877a95640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5870158f90 con 0x7f585c005190 2023-12-20T04:47:19.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5877a95640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5870159230 con 0x7f587015e360 2023-12-20T04:47:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5875030640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587015dee0 0x7f5870071210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5875030640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587015dee0 0x7f5870071210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34944/0 (socket says 172.21.15.32:34944) 2023-12-20T04:47:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5875030640 1 -- 172.21.15.32:0/3925194312 learned_addr learned my addr 172.21.15.32:0/3925194312 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5876032640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f587015e360 0x7f5870071750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5875831640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f585c005190 0x7f5870151740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5875030640 1 -- 172.21.15.32:0/3925194312 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f587015e360 msgr2=0x7f5870071750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5875030640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f587015e360 0x7f5870071750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5875030640 1 -- 172.21.15.32:0/3925194312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f585c005190 msgr2=0x7f5870151740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5875030640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f585c005190 0x7f5870151740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5875030640 1 -- 172.21.15.32:0/3925194312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5870159510 con 0x7f587015dee0 2023-12-20T04:47:19.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5875831640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f585c005190 0x7f5870151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:19.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5875030640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587015dee0 0x7f5870071210 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f586000e7a0 tx=0x7f586000ec60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:19.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f58667fc640 1 -- 172.21.15.32:0/3925194312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586000c8d0 con 0x7f587015dee0 2023-12-20T04:47:19.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f58667fc640 1 -- 172.21.15.32:0/3925194312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f586000ca30 con 0x7f587015dee0 2023-12-20T04:47:19.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f58667fc640 1 -- 172.21.15.32:0/3925194312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f586000cdc0 con 0x7f587015dee0 2023-12-20T04:47:19.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5877a95640 1 -- 172.21.15.32:0/3925194312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58701597f0 con 0x7f587015dee0 2023-12-20T04:47:19.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5877a95640 1 -- 172.21.15.32:0/3925194312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f587015c310 con 0x7f587015dee0 2023-12-20T04:47:19.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.034+0000 7f5877a95640 1 -- 172.21.15.32:0/3925194312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5870159cd0 con 0x7f587015dee0 2023-12-20T04:47:19.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.038+0000 7f58667fc640 1 -- 172.21.15.32:0/3925194312 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f586001d460 con 0x7f587015dee0 2023-12-20T04:47:19.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.038+0000 7f58667fc640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5854065b80 0x7f5854068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.038+0000 7f58667fc640 1 -- 172.21.15.32:0/3925194312 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f5860082e70 con 0x7f587015dee0 2023-12-20T04:47:19.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.038+0000 7f5875831640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5854065b80 0x7f5854068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.038+0000 7f5875831640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5854065b80 0x7f5854068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5870071dd0 tx=0x7f586c008040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:19.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.042+0000 7f58667fc640 1 -- 172.21.15.32:0/3925194312 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5860050a50 con 0x7f587015dee0 2023-12-20T04:47:19.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.202+0000 7f5877a95640 1 -- 172.21.15.32:0/3925194312 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f587015b990 con 0x7f587015dee0 2023-12-20T04:47:19.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.202+0000 7f58667fc640 1 -- 172.21.15.32:0/3925194312 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v134) v1 ==== 84+0+65 (secure 0 0 0) 0x7f587015b990 con 0x7f587015dee0 2023-12-20T04:47:19.202 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:19.202 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:47:19.202 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:19.202 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T04:47:19.202 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T04:47:19.202 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:19.202 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 -- 172.21.15.32:0/3925194312 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5854065b80 msgr2=0x7f5854068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5854065b80 0x7f5854068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f5870071dd0 tx=0x7f586c008040 comp rx=0 tx=0).stop 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 -- 172.21.15.32:0/3925194312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587015dee0 msgr2=0x7f5870071210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587015dee0 0x7f5870071210 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7f586000e7a0 tx=0x7f586000ec60 comp rx=0 tx=0).stop 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 -- 172.21.15.32:0/3925194312 shutdown_connections 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5854065b80 0x7f5854068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f587015e360 0x7f5870071750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f587015dee0 0x7f5870071210 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 --2- 172.21.15.32:0/3925194312 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f585c005190 0x7f5870151740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 -- 172.21.15.32:0/3925194312 >> 172.21.15.32:0/3925194312 conn(0x7f587006a320 msgr2=0x7f58700cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 -- 172.21.15.32:0/3925194312 shutdown_connections 2023-12-20T04:47:19.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.206+0000 7f5877a95640 1 -- 172.21.15.32:0/3925194312 wait complete. 2023-12-20T04:47:19.223 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T04:47:19.223 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T04:47:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.370+0000 7f290dff3640 1 Processor -- start 2023-12-20T04:47:19.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.370+0000 7f290dff3640 1 -- start start 2023-12-20T04:47:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f290dff3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290806dcb0 0x7f29080ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f290dff3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290806e100 con 0x7f2908070230 2023-12-20T04:47:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f290dff3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29080ca590 con 0x7f2908158520 2023-12-20T04:47:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f290dff3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29080ca7f0 con 0x7f290806dcb0 2023-12-20T04:47:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f2907fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2908070230 0x7f290815b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43890/0 (socket says 172.21.15.32:43890) 2023-12-20T04:47:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f2907fff640 1 -- 172.21.15.32:0/2807891632 learned_addr learned my addr 172.21.15.32:0/2807891632 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:19.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f2906ffd640 1 --2- 172.21.15.32:0/2807891632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290806dcb0 0x7f29080ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f29067fc640 1 -- 172.21.15.32:0/2807891632 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1863043114 0 0) 0x7f29000014a0 con 0x7f2908070230 2023-12-20T04:47:19.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f29067fc640 1 -- 172.21.15.32:0/2807891632 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f28f0003530 con 0x7f2908070230 2023-12-20T04:47:19.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f29067fc640 1 -- 172.21.15.32:0/2807891632 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 440213381 0 0) 0x7f29080ca590 con 0x7f2908158520 2023-12-20T04:47:19.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f2906ffd640 1 -- 172.21.15.32:0/2807891632 >> v1:172.21.15.39:6789/0 conn(0x7f2908158520 legacy=0x7f290815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f2906ffd640 1 -- 172.21.15.32:0/2807891632 >> v1:172.21.15.32:6789/0 conn(0x7f2908070230 legacy=0x7f290815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.374+0000 7f2906ffd640 1 -- 172.21.15.32:0/2807891632 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29080caad0 con 0x7f290806dcb0 2023-12-20T04:47:19.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f2906ffd640 1 --2- 172.21.15.32:0/2807891632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290806dcb0 0x7f29080ca050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0097f0 tx=0x7f28fc01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=21a38b22697943a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f29067fc640 1 -- 172.21.15.32:0/2807891632 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0ab600 con 0x7f290806dcb0 2023-12-20T04:47:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f29067fc640 1 -- 172.21.15.32:0/2807891632 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc0ab760 con 0x7f290806dcb0 2023-12-20T04:47:19.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f29067fc640 1 -- 172.21.15.32:0/2807891632 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0abaf0 con 0x7f290806dcb0 2023-12-20T04:47:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f290dff3640 1 -- 172.21.15.32:0/2807891632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290806dcb0 msgr2=0x7f29080ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f290dff3640 1 --2- 172.21.15.32:0/2807891632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290806dcb0 0x7f29080ca050 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0097f0 tx=0x7f28fc01c400 comp rx=0 tx=0).stop 2023-12-20T04:47:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f290dff3640 1 -- 172.21.15.32:0/2807891632 shutdown_connections 2023-12-20T04:47:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f290dff3640 1 --2- 172.21.15.32:0/2807891632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290806dcb0 0x7f29080ca050 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f290dff3640 1 -- 172.21.15.32:0/2807891632 >> 172.21.15.32:0/2807891632 conn(0x7f290806a510 msgr2=0x7f29080ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f290dff3640 1 -- 172.21.15.32:0/2807891632 shutdown_connections 2023-12-20T04:47:19.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f290dff3640 1 -- 172.21.15.32:0/2807891632 wait complete. 2023-12-20T04:47:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f290dff3640 1 Processor -- start 2023-12-20T04:47:19.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.378+0000 7f290dff3640 1 -- start start 2023-12-20T04:47:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f290dff3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290806dcb0 0x7f290813d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f290dff3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2908070230 0x7f29081592c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f290dff3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2908158520 0x7f290815c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f290dff3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290815a890 con 0x7f2908070230 2023-12-20T04:47:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f290dff3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290813d790 con 0x7f2908158520 2023-12-20T04:47:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f290dff3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2908159c10 con 0x7f290806dcb0 2023-12-20T04:47:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f2906ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2908070230 0x7f29081592c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f2906ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2908070230 0x7f29081592c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34954/0 (socket says 172.21.15.32:34954) 2023-12-20T04:47:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f2906ffd640 1 -- 172.21.15.32:0/3667092818 learned_addr learned my addr 172.21.15.32:0/3667092818 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:19.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f29077fe640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290806dcb0 0x7f290813d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f2907fff640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2908158520 0x7f290815c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f2906ffd640 1 -- 172.21.15.32:0/3667092818 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290806dcb0 msgr2=0x7f290813d250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f2906ffd640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290806dcb0 0x7f290813d250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f2906ffd640 1 -- 172.21.15.32:0/3667092818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2908158520 msgr2=0x7f290815c220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f2906ffd640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2908158520 0x7f290815c220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f2906ffd640 1 -- 172.21.15.32:0/3667092818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28fc009420 con 0x7f2908070230 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f2906ffd640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2908070230 0x7f29081592c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0097f0 tx=0x7f28fc01ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f28effff640 1 -- 172.21.15.32:0/3667092818 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0bf610 con 0x7f2908070230 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f28effff640 1 -- 172.21.15.32:0/3667092818 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc0bf790 con 0x7f2908070230 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f28effff640 1 -- 172.21.15.32:0/3667092818 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc0bfb40 con 0x7f2908070230 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f290dff3640 1 -- 172.21.15.32:0/3667092818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2908159ef0 con 0x7f2908070230 2023-12-20T04:47:19.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f290dff3640 1 -- 172.21.15.32:0/3667092818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290815a4b0 con 0x7f2908070230 2023-12-20T04:47:19.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f290dff3640 1 -- 172.21.15.32:0/3667092818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290815a9d0 con 0x7f2908070230 2023-12-20T04:47:19.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.382+0000 7f28effff640 1 -- 172.21.15.32:0/3667092818 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f28fc0bfca0 con 0x7f2908070230 2023-12-20T04:47:19.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.386+0000 7f28effff640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28f4065bb0 0x7f28f4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.386+0000 7f28effff640 1 -- 172.21.15.32:0/3667092818 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(134..134 src has 1..134) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f28fc11df40 con 0x7f2908070230 2023-12-20T04:47:19.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.386+0000 7f29077fe640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28f4065bb0 0x7f28f4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.386+0000 7f29077fe640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28f4065bb0 0x7f28f4068070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28f8007fd0 tx=0x7f28f8009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:19.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.390+0000 7f28effff640 1 -- 172.21.15.32:0/3667092818 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f290815a9d0 con 0x7f2908070230 2023-12-20T04:47:19.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.546+0000 7f290dff3640 1 -- 172.21.15.32:0/3667092818 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f290815a9d0 con 0x7f2908070230 2023-12-20T04:47:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.646+0000 7f28effff640 1 -- 172.21.15.32:0/3667092818 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v135) v1 ==== 138+0+0 (secure 0 0 0) 0x7f290815a9d0 con 0x7f2908070230 2023-12-20T04:47:19.646 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T04:47:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 -- 172.21.15.32:0/3667092818 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28f4065bb0 msgr2=0x7f28f4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28f4065bb0 0x7f28f4068070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f28f8007fd0 tx=0x7f28f8009040 comp rx=0 tx=0).stop 2023-12-20T04:47:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 -- 172.21.15.32:0/3667092818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2908070230 msgr2=0x7f29081592c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2908070230 0x7f29081592c0 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7f28fc0097f0 tx=0x7f28fc01ac40 comp rx=0 tx=0).stop 2023-12-20T04:47:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 -- 172.21.15.32:0/3667092818 shutdown_connections 2023-12-20T04:47:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f28f4065bb0 0x7f28f4068070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2908158520 0x7f290815c220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2908070230 0x7f29081592c0 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 --2- 172.21.15.32:0/3667092818 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290806dcb0 0x7f290813d250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 -- 172.21.15.32:0/3667092818 >> 172.21.15.32:0/3667092818 conn(0x7f290806a510 msgr2=0x7f29080c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 -- 172.21.15.32:0/3667092818 shutdown_connections 2023-12-20T04:47:19.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.650+0000 7f290dff3640 1 -- 172.21.15.32:0/3667092818 wait complete. 2023-12-20T04:47:19.673 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T04:47:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918f517640 1 Processor -- start 2023-12-20T04:47:19.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918f517640 1 -- start start 2023-12-20T04:47:19.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918f517640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880adca0 0x7f91880cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918f517640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91880cc710 con 0x7f918806cc40 2023-12-20T04:47:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918f517640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91880cc990 con 0x7f91881655e0 2023-12-20T04:47:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918f517640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9188165a70 con 0x7f91880adca0 2023-12-20T04:47:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918cab2640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f918806cc40 0x7f9188164ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43894/0 (socket says 172.21.15.32:43894) 2023-12-20T04:47:19.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918cab2640 1 -- 172.21.15.32:0/2468694463 learned_addr learned my addr 172.21.15.32:0/2468694463 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918d2b3640 1 --2- 172.21.15.32:0/2468694463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880adca0 0x7f91880cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f917ffff640 1 -- 172.21.15.32:0/2468694463 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1751985255 0 0) 0x7f91880cc710 con 0x7f918806cc40 2023-12-20T04:47:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f917ffff640 1 -- 172.21.15.32:0/2468694463 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9174003530 con 0x7f918806cc40 2023-12-20T04:47:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f917ffff640 1 -- 172.21.15.32:0/2468694463 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3513662412 0 0) 0x7f91880cc990 con 0x7f91881655e0 2023-12-20T04:47:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f917ffff640 1 -- 172.21.15.32:0/2468694463 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91880cc710 con 0x7f91881655e0 2023-12-20T04:47:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f917ffff640 1 -- 172.21.15.32:0/2468694463 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 483191161 0 0) 0x7f9174003530 con 0x7f918806cc40 2023-12-20T04:47:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918d2b3640 1 -- 172.21.15.32:0/2468694463 >> v1:172.21.15.39:6789/0 conn(0x7f91881655e0 legacy=0x7f91880c3b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918d2b3640 1 -- 172.21.15.32:0/2468694463 >> v1:172.21.15.32:6789/0 conn(0x7f918806cc40 legacy=0x7f9188164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.826+0000 7f918d2b3640 1 -- 172.21.15.32:0/2468694463 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880ccc70 con 0x7f91880adca0 2023-12-20T04:47:19.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.830+0000 7f918d2b3640 1 --2- 172.21.15.32:0/2468694463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880adca0 0x7f91880cd750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f91840097f0 tx=0x7f91880c4a70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f7d5bd2e039b09a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.830+0000 7f917ffff640 1 -- 172.21.15.32:0/2468694463 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9178001680 con 0x7f91880adca0 2023-12-20T04:47:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.830+0000 7f917ffff640 1 -- 172.21.15.32:0/2468694463 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91840a7050 con 0x7f91880adca0 2023-12-20T04:47:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.830+0000 7f917ffff640 1 -- 172.21.15.32:0/2468694463 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91840ab620 con 0x7f91880adca0 2023-12-20T04:47:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.830+0000 7f918f517640 1 -- 172.21.15.32:0/2468694463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880adca0 msgr2=0x7f91880cd750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.830+0000 7f918f517640 1 --2- 172.21.15.32:0/2468694463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880adca0 0x7f91880cd750 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f91840097f0 tx=0x7f91880c4a70 comp rx=0 tx=0).stop 2023-12-20T04:47:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.830+0000 7f918f517640 1 -- 172.21.15.32:0/2468694463 shutdown_connections 2023-12-20T04:47:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.830+0000 7f918f517640 1 --2- 172.21.15.32:0/2468694463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880adca0 0x7f91880cd750 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.830+0000 7f918f517640 1 -- 172.21.15.32:0/2468694463 >> 172.21.15.32:0/2468694463 conn(0x7f918806a4b0 msgr2=0x7f91880b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.830+0000 7f918f517640 1 -- 172.21.15.32:0/2468694463 shutdown_connections 2023-12-20T04:47:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.830+0000 7f918f517640 1 -- 172.21.15.32:0/2468694463 wait complete. 2023-12-20T04:47:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918f517640 1 Processor -- start 2023-12-20T04:47:19.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918f517640 1 -- start start 2023-12-20T04:47:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918f517640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918806cc40 0x7f91880c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918f517640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880adca0 0x7f91880b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918f517640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91881655e0 0x7f91880b2b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918f517640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880c84e0 con 0x7f91881655e0 2023-12-20T04:47:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918f517640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880b17f0 con 0x7f91880adca0 2023-12-20T04:47:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918f517640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91880b1a90 con 0x7f918806cc40 2023-12-20T04:47:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918dab4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91881655e0 0x7f91880b2b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918d2b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918806cc40 0x7f91880c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918cab2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880adca0 0x7f91880b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918dab4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91881655e0 0x7f91880b2b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34964/0 (socket says 172.21.15.32:34964) 2023-12-20T04:47:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918dab4640 1 -- 172.21.15.32:0/1775909460 learned_addr learned my addr 172.21.15.32:0/1775909460 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918dab4640 1 -- 172.21.15.32:0/1775909460 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918806cc40 msgr2=0x7f91880c7fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918dab4640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918806cc40 0x7f91880c7fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918dab4640 1 -- 172.21.15.32:0/1775909460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880adca0 msgr2=0x7f91880b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918dab4640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880adca0 0x7f91880b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918dab4640 1 -- 172.21.15.32:0/1775909460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9184009420 con 0x7f91881655e0 2023-12-20T04:47:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918d2b3640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918806cc40 0x7f91880c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918cab2640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880adca0 0x7f91880b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918dab4640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91881655e0 0x7f91880b2b30 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f918000c730 tx=0x7f918000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f917dffb640 1 -- 172.21.15.32:0/1775909460 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9180003b80 con 0x7f91881655e0 2023-12-20T04:47:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f917dffb640 1 -- 172.21.15.32:0/1775909460 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9180003ce0 con 0x7f91881655e0 2023-12-20T04:47:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f917dffb640 1 -- 172.21.15.32:0/1775909460 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f918000aa90 con 0x7f91881655e0 2023-12-20T04:47:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918f517640 1 -- 172.21.15.32:0/1775909460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91880b1d70 con 0x7f91881655e0 2023-12-20T04:47:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.834+0000 7f918f517640 1 -- 172.21.15.32:0/1775909460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91880b2330 con 0x7f91881655e0 2023-12-20T04:47:19.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.838+0000 7f918f517640 1 -- 172.21.15.32:0/1775909460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91880c8bc0 con 0x7f91881655e0 2023-12-20T04:47:19.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.838+0000 7f917dffb640 1 -- 172.21.15.32:0/1775909460 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f918000abf0 con 0x7f91881655e0 2023-12-20T04:47:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.838+0000 7f917dffb640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f916c065bb0 0x7f916c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.838+0000 7f917dffb640 1 -- 172.21.15.32:0/1775909460 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(135..135 src has 1..135) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f9180083830 con 0x7f91881655e0 2023-12-20T04:47:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.838+0000 7f918d2b3640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f916c065bb0 0x7f916c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.842+0000 7f918d2b3640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f916c065bb0 0x7f916c068070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f91840ad000 tx=0x7f9184004740 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:19.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:19.842+0000 7f917dffb640 1 -- 172.21.15.32:0/1775909460 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9180051430 con 0x7f91881655e0 2023-12-20T04:47:20.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.002+0000 7f918f517640 1 -- 172.21.15.32:0/1775909460 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f918804e000 con 0x7f91881655e0 2023-12-20T04:47:20.650 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T04:47:20.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.650+0000 7f917dffb640 1 -- 172.21.15.32:0/1775909460 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v136) v1 ==== 170+0+0 (secure 0 0 0) 0x7f9180054a50 con 0x7f91881655e0 2023-12-20T04:47:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.654+0000 7f918f517640 1 -- 172.21.15.32:0/1775909460 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f916c065bb0 msgr2=0x7f916c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.654+0000 7f918f517640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f916c065bb0 0x7f916c068070 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f91840ad000 tx=0x7f9184004740 comp rx=0 tx=0).stop 2023-12-20T04:47:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.654+0000 7f918f517640 1 -- 172.21.15.32:0/1775909460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91881655e0 msgr2=0x7f91880b2b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:20.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.654+0000 7f918f517640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91881655e0 0x7f91880b2b30 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7f918000c730 tx=0x7f918000cbf0 comp rx=0 tx=0).stop 2023-12-20T04:47:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.658+0000 7f918f517640 1 -- 172.21.15.32:0/1775909460 shutdown_connections 2023-12-20T04:47:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.658+0000 7f918f517640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f916c065bb0 0x7f916c068070 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.658+0000 7f918f517640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91881655e0 0x7f91880b2b30 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.658+0000 7f918f517640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880adca0 0x7f91880b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.658+0000 7f918f517640 1 --2- 172.21.15.32:0/1775909460 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918806cc40 0x7f91880c7fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.658+0000 7f918f517640 1 -- 172.21.15.32:0/1775909460 >> 172.21.15.32:0/1775909460 conn(0x7f918806a4b0 msgr2=0x7f91880730e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.658+0000 7f918f517640 1 -- 172.21.15.32:0/1775909460 shutdown_connections 2023-12-20T04:47:20.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.662+0000 7f918f517640 1 -- 172.21.15.32:0/1775909460 wait complete. 2023-12-20T04:47:20.692 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T04:47:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc84c5c0640 1 Processor -- start 2023-12-20T04:47:20.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc84c5c0640 1 -- start start 2023-12-20T04:47:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc84c5c0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8440cd770 0x7fc8440cc790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc84c5c0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc8441612b0 0x7fc84415dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc84c5c0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc844070620 con 0x7fc8440cd390 2023-12-20T04:47:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc84c5c0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc84415e160 con 0x7fc8440cd770 2023-12-20T04:47:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc84c5c0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8440ccd70 con 0x7fc8441612b0 2023-12-20T04:47:20.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc849b5b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8440cd770 0x7fc8440cc790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc849b5b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8440cd770 0x7fc8440cc790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34972/0 (socket says 172.21.15.32:34972) 2023-12-20T04:47:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc849b5b640 1 -- 172.21.15.32:0/3731769760 learned_addr learned my addr 172.21.15.32:0/3731769760 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc84ab5d640 1 --2- 172.21.15.32:0/3731769760 >> v2:172.21.15.39:3300/0 conn(0x7fc8441612b0 0x7fc84415dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc849b5b640 1 -- 172.21.15.32:0/3731769760 >> v2:172.21.15.39:3300/0 conn(0x7fc8441612b0 msgr2=0x7fc84415dc20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc849b5b640 1 --2- 172.21.15.32:0/3731769760 >> v2:172.21.15.39:3300/0 conn(0x7fc8441612b0 0x7fc84415dc20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc849b5b640 1 -- 172.21.15.32:0/3731769760 >> v1:172.21.15.39:6789/0 conn(0x7fc8440cd390 legacy=0x7fc844071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:20.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.842+0000 7fc849b5b640 1 -- 172.21.15.32:0/3731769760 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc844070870 con 0x7fc8440cd770 2023-12-20T04:47:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc849b5b640 1 --2- 172.21.15.32:0/3731769760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8440cd770 0x7fc8440cc790 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc8440b16f0 tx=0x7fc83401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=415093af1cde872e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84935a640 1 -- 172.21.15.32:0/3731769760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8340ac410 con 0x7fc8440cd770 2023-12-20T04:47:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84935a640 1 -- 172.21.15.32:0/3731769760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8340ac570 con 0x7fc8440cd770 2023-12-20T04:47:20.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84935a640 1 -- 172.21.15.32:0/3731769760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8340ac900 con 0x7fc8440cd770 2023-12-20T04:47:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84c5c0640 1 -- 172.21.15.32:0/3731769760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8440cd770 msgr2=0x7fc8440cc790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84c5c0640 1 --2- 172.21.15.32:0/3731769760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8440cd770 0x7fc8440cc790 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fc8440b16f0 tx=0x7fc83401c3b0 comp rx=0 tx=0).stop 2023-12-20T04:47:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84c5c0640 1 -- 172.21.15.32:0/3731769760 shutdown_connections 2023-12-20T04:47:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84c5c0640 1 --2- 172.21.15.32:0/3731769760 >> v2:172.21.15.39:3300/0 conn(0x7fc8441612b0 0x7fc84415dc20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84c5c0640 1 --2- 172.21.15.32:0/3731769760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8440cd770 0x7fc8440cc790 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:20.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84c5c0640 1 -- 172.21.15.32:0/3731769760 >> 172.21.15.32:0/3731769760 conn(0x7fc8440b06b0 msgr2=0x7fc8440b0ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84c5c0640 1 -- 172.21.15.32:0/3731769760 shutdown_connections 2023-12-20T04:47:20.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84c5c0640 1 -- 172.21.15.32:0/3731769760 wait complete. 2023-12-20T04:47:20.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.846+0000 7fc84c5c0640 1 Processor -- start 2023-12-20T04:47:20.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84c5c0640 1 -- start start 2023-12-20T04:47:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84c5c0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8440cd390 0x7fc844157e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84c5c0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8440cd770 0x7fc844157850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84c5c0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8441612b0 0x7fc8440caa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84c5c0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8440b17e0 con 0x7fc8440cd390 2023-12-20T04:47:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84c5c0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc84415bfc0 con 0x7fc8441612b0 2023-12-20T04:47:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84c5c0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc844156a90 con 0x7fc8440cd770 2023-12-20T04:47:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84a35c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8440cd390 0x7fc844157e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84a35c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8440cd390 0x7fc844157e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34976/0 (socket says 172.21.15.32:34976) 2023-12-20T04:47:20.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84a35c640 1 -- 172.21.15.32:0/2070395626 learned_addr learned my addr 172.21.15.32:0/2070395626 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84ab5d640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8441612b0 0x7fc8440caa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc849b5b640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8440cd770 0x7fc844157850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84a35c640 1 -- 172.21.15.32:0/2070395626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8440cd770 msgr2=0x7fc844157850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84a35c640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8440cd770 0x7fc844157850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84a35c640 1 -- 172.21.15.32:0/2070395626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8441612b0 msgr2=0x7fc8440caa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84a35c640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8441612b0 0x7fc8440caa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84a35c640 1 -- 172.21.15.32:0/2070395626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8340094d0 con 0x7fc8440cd390 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84a35c640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8440cd390 0x7fc844157e30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc84000c500 tx=0x7fc84000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc83affd640 1 -- 172.21.15.32:0/2070395626 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840005220 con 0x7fc8440cd390 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc83affd640 1 -- 172.21.15.32:0/2070395626 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc840005380 con 0x7fc8440cd390 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc83affd640 1 -- 172.21.15.32:0/2070395626 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc840007270 con 0x7fc8440cd390 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84c5c0640 1 -- 172.21.15.32:0/2070395626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc844156d70 con 0x7fc8440cd390 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84c5c0640 1 -- 172.21.15.32:0/2070395626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc844157330 con 0x7fc8440cd390 2023-12-20T04:47:20.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.850+0000 7fc84c5c0640 1 -- 172.21.15.32:0/2070395626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc84404e000 con 0x7fc8440cd390 2023-12-20T04:47:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.854+0000 7fc83affd640 1 -- 172.21.15.32:0/2070395626 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc8400073d0 con 0x7fc8440cd390 2023-12-20T04:47:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.854+0000 7fc83affd640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc828065b80 0x7fc828068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.854+0000 7fc83affd640 1 -- 172.21.15.32:0/2070395626 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(136..136 src has 1..136) v4 ==== 5637+0+0 (secure 0 0 0) 0x7fc8400838e0 con 0x7fc8440cd390 2023-12-20T04:47:20.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.854+0000 7fc849b5b640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc828065b80 0x7fc828068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:20.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.854+0000 7fc849b5b640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc828065b80 0x7fc828068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc8340ae000 tx=0x7fc834009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:20.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:20.858+0000 7fc83affd640 1 -- 172.21.15.32:0/2070395626 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc840051270 con 0x7fc8440cd390 2023-12-20T04:47:21.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.046+0000 7fc84c5c0640 1 -- 172.21.15.32:0/2070395626 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fc8440c8f70 con 0x7fc8440cd390 2023-12-20T04:47:21.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.054+0000 7fc83affd640 1 -- 172.21.15.32:0/2070395626 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 16 and data pool 17 v58) v1 ==== 174+0+0 (secure 0 0 0) 0x7fc840054890 con 0x7fc8440cd390 2023-12-20T04:47:21.054 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 16 and data pool 17 2023-12-20T04:47:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.058+0000 7fc84c5c0640 1 -- 172.21.15.32:0/2070395626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc828065b80 msgr2=0x7fc828068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.058+0000 7fc84c5c0640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc828065b80 0x7fc828068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc8340ae000 tx=0x7fc834009040 comp rx=0 tx=0).stop 2023-12-20T04:47:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.058+0000 7fc84c5c0640 1 -- 172.21.15.32:0/2070395626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8440cd390 msgr2=0x7fc844157e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.058+0000 7fc84c5c0640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8440cd390 0x7fc844157e30 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc84000c500 tx=0x7fc84000c9c0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.058+0000 7fc84c5c0640 1 -- 172.21.15.32:0/2070395626 shutdown_connections 2023-12-20T04:47:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.058+0000 7fc84c5c0640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc828065b80 0x7fc828068040 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fc8340ae000 tx=0x7fc834009040 comp rx=0 tx=0).stop 2023-12-20T04:47:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.058+0000 7fc84c5c0640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8441612b0 0x7fc8440caa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.058+0000 7fc84c5c0640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8440cd770 0x7fc844157850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.058+0000 7fc84c5c0640 1 --2- 172.21.15.32:0/2070395626 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8440cd390 0x7fc844157e30 secure :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fc84000c500 tx=0x7fc84000c9c0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.058+0000 7fc84c5c0640 1 -- 172.21.15.32:0/2070395626 >> 172.21.15.32:0/2070395626 conn(0x7fc8440b06b0 msgr2=0x7fc84415f810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:21.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.062+0000 7fc84c5c0640 1 -- 172.21.15.32:0/2070395626 shutdown_connections 2023-12-20T04:47:21.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.066+0000 7fc84c5c0640 1 -- 172.21.15.32:0/2070395626 wait complete. 2023-12-20T04:47:21.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:47:21.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.246+0000 7f71fb073640 1 Processor -- start 2023-12-20T04:47:21.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.246+0000 7f71fb073640 1 -- start start 2023-12-20T04:47:21.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.246+0000 7f71fb073640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71f406ce60 0x7f71f415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.246+0000 7f71fb073640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71f415f110 0x7f71f415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.246+0000 7f71fb073640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f71f415bbd0 con 0x7f71f40af5e0 2023-12-20T04:47:21.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.246+0000 7f71fb073640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71f415beb0 con 0x7f71f406ce60 2023-12-20T04:47:21.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.246+0000 7f71fb073640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f415e910 con 0x7f71f415f110 2023-12-20T04:47:21.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71f406ce60 0x7f71f415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f8e0f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f71f40af5e0 0x7f71f40af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43908/0 (socket says 172.21.15.32:43908) 2023-12-20T04:47:21.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f8e0f640 1 -- 172.21.15.32:0/3334904863 learned_addr learned my addr 172.21.15.32:0/3334904863 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:21.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f9610640 1 --2- 172.21.15.32:0/3334904863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71f415f110 0x7f71f415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f3fff640 1 -- 172.21.15.32:0/3334904863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71f415f110 msgr2=0x7f71f415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:21.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f3fff640 1 --2- 172.21.15.32:0/3334904863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71f415f110 0x7f71f415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f3fff640 1 -- 172.21.15.32:0/3334904863 >> v1:172.21.15.32:6789/0 conn(0x7f71f40af5e0 legacy=0x7f71f40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f3fff640 1 -- 172.21.15.32:0/3334904863 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71f415ebf0 con 0x7f71f406ce60 2023-12-20T04:47:21.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f37fe640 1 -- 172.21.15.32:0/3334904863 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 865692850 0 0) 0x7f71f415bbd0 con 0x7f71f40af5e0 2023-12-20T04:47:21.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f3fff640 1 --2- 172.21.15.32:0/3334904863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f406ce60 0x7f71f415b280 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f71e00097c0 tx=0x7f71e001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa62d4a7d365eae1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:21.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f37fe640 1 -- 172.21.15.32:0/3334904863 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71f415b9d0 con 0x7f71f406ce60 2023-12-20T04:47:21.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f37fe640 1 -- 172.21.15.32:0/3334904863 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e00ad640 con 0x7f71f406ce60 2023-12-20T04:47:21.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.250+0000 7f71f37fe640 1 -- 172.21.15.32:0/3334904863 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e00ad9d0 con 0x7f71f406ce60 2023-12-20T04:47:21.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 -- 172.21.15.32:0/3334904863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f406ce60 msgr2=0x7f71f415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 --2- 172.21.15.32:0/3334904863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f406ce60 0x7f71f415b280 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7f71e00097c0 tx=0x7f71e001c3f0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 -- 172.21.15.32:0/3334904863 shutdown_connections 2023-12-20T04:47:21.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 --2- 172.21.15.32:0/3334904863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f406ce60 0x7f71f415b280 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 --2- 172.21.15.32:0/3334904863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71f415f110 0x7f71f415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 -- 172.21.15.32:0/3334904863 >> 172.21.15.32:0/3334904863 conn(0x7f71f406a320 msgr2=0x7f71f40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:21.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 -- 172.21.15.32:0/3334904863 shutdown_connections 2023-12-20T04:47:21.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 -- 172.21.15.32:0/3334904863 wait complete. 2023-12-20T04:47:21.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 Processor -- start 2023-12-20T04:47:21.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 -- start start 2023-12-20T04:47:21.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f406ce60 0x7f71f40cb710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f40af5e0 0x7f71f4141670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71f415f110 0x7f71f40b3760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c7770 con 0x7f71f40af5e0 2023-12-20T04:47:21.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c79e0 con 0x7f71f406ce60 2023-12-20T04:47:21.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71fb073640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71f40c7cc0 con 0x7f71f415f110 2023-12-20T04:47:21.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f40af5e0 0x7f71f4141670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f40af5e0 0x7f71f4141670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34988/0 (socket says 172.21.15.32:34988) 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f3fff640 1 -- 172.21.15.32:0/397799014 learned_addr learned my addr 172.21.15.32:0/397799014 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f9610640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71f415f110 0x7f71f40b3760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f8e0f640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f406ce60 0x7f71f40cb710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f3fff640 1 -- 172.21.15.32:0/397799014 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71f415f110 msgr2=0x7f71f40b3760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f3fff640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71f415f110 0x7f71f40b3760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f3fff640 1 -- 172.21.15.32:0/397799014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f406ce60 msgr2=0x7f71f40cb710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f3fff640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f406ce60 0x7f71f40cb710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f3fff640 1 -- 172.21.15.32:0/397799014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71e0009420 con 0x7f71f40af5e0 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f9610640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71f415f110 0x7f71f40b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.254+0000 7f71f8e0f640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f406ce60 0x7f71f40cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.258+0000 7f71f3fff640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f40af5e0 0x7f71f4141670 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f71e001a020 tx=0x7f71e001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.258+0000 7f71f17fa640 1 -- 172.21.15.32:0/397799014 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0006e20 con 0x7f71f40af5e0 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.258+0000 7f71f17fa640 1 -- 172.21.15.32:0/397799014 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71e00a6070 con 0x7f71f40af5e0 2023-12-20T04:47:21.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.258+0000 7f71fb073640 1 -- 172.21.15.32:0/397799014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71f40c7fa0 con 0x7f71f40af5e0 2023-12-20T04:47:21.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.258+0000 7f71f17fa640 1 -- 172.21.15.32:0/397799014 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71e0004220 con 0x7f71f40af5e0 2023-12-20T04:47:21.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.258+0000 7f71fb073640 1 -- 172.21.15.32:0/397799014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71f4141bb0 con 0x7f71f40af5e0 2023-12-20T04:47:21.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.258+0000 7f71fb073640 1 -- 172.21.15.32:0/397799014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71f404e000 con 0x7f71f40af5e0 2023-12-20T04:47:21.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.258+0000 7f71f17fa640 1 -- 172.21.15.32:0/397799014 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f71e0004380 con 0x7f71f40af5e0 2023-12-20T04:47:21.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.258+0000 7f71f17fa640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71d8065b80 0x7f71d8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.258+0000 7f71f17fa640 1 -- 172.21.15.32:0/397799014 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f71e011d9e0 con 0x7f71f40af5e0 2023-12-20T04:47:21.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.258+0000 7f71f8e0f640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71d8065b80 0x7f71d8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.262+0000 7f71f8e0f640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71d8065b80 0x7f71d8068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f71e8007fd0 tx=0x7f71e8009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:21.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.262+0000 7f71f17fa640 1 -- 172.21.15.32:0/397799014 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f71e00eb300 con 0x7f71f40af5e0 2023-12-20T04:47:21.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.422+0000 7f71fb073640 1 -- 172.21.15.32:0/397799014 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f71f40cf450 con 0x7f71f40af5e0 2023-12-20T04:47:21.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71f17fa640 1 -- 172.21.15.32:0/397799014 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16068 (secure 0 0 0) 0x7f71e00b2090 con 0x7f71f40af5e0 2023-12-20T04:47:21.424 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:21.424 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":137,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:47:21.050468+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:47:19.596882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-20T04:47:20.055023+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 -- 172.21.15.32:0/397799014 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71d8065b80 msgr2=0x7f71d8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71d8065b80 0x7f71d8068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f71e8007fd0 tx=0x7f71e8009040 comp rx=0 tx=0).stop 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 -- 172.21.15.32:0/397799014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f40af5e0 msgr2=0x7f71f4141670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f40af5e0 0x7f71f4141670 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7f71e001a020 tx=0x7f71e001ac30 comp rx=0 tx=0).stop 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 -- 172.21.15.32:0/397799014 shutdown_connections 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f71d8065b80 0x7f71d8068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71f415f110 0x7f71f40b3760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71f40af5e0 0x7f71f4141670 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 --2- 172.21.15.32:0/397799014 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71f406ce60 0x7f71f40cb710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 -- 172.21.15.32:0/397799014 >> 172.21.15.32:0/397799014 conn(0x7f71f406a320 msgr2=0x7f71f40af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 -- 172.21.15.32:0/397799014 shutdown_connections 2023-12-20T04:47:21.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.426+0000 7f71fb073640 1 -- 172.21.15.32:0/397799014 wait complete. 2023-12-20T04:47:21.445 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:47:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.590+0000 7f0ccef25640 1 Processor -- start 2023-12-20T04:47:21.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.590+0000 7f0ccef25640 1 -- start start 2023-12-20T04:47:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.590+0000 7f0ccef25640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0cc80af5e0 0x7f0cc80af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.590+0000 7f0ccef25640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0cc806ce60 0x7f0cc815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.590+0000 7f0ccef25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0cc815f110 0x7f0cc815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.590+0000 7f0ccef25640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc815bbd0 con 0x7f0cc80af5e0 2023-12-20T04:47:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.590+0000 7f0ccef25640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0cc815beb0 con 0x7f0cc806ce60 2023-12-20T04:47:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.590+0000 7f0ccef25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc815e910 con 0x7f0cc815f110 2023-12-20T04:47:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.590+0000 7f0ccccc1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0cc80af5e0 0x7f0cc80af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0ccccc1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0cc80af5e0 0x7f0cc80af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34994/0 (socket says 172.21.15.32:34994) 2023-12-20T04:47:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0cc7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0cc806ce60 0x7f0cc815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0ccccc1640 1 -- 172.21.15.32:0/1700854087 learned_addr learned my addr 172.21.15.32:0/1700854087 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0ccd4c2640 1 --2- 172.21.15.32:0/1700854087 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0cc815f110 0x7f0cc815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0ccd4c2640 1 -- 172.21.15.32:0/1700854087 >> v2:172.21.15.39:3300/0 conn(0x7f0cc806ce60 msgr2=0x7f0cc815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0ccd4c2640 1 --2- 172.21.15.32:0/1700854087 >> v2:172.21.15.39:3300/0 conn(0x7f0cc806ce60 0x7f0cc815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0ccd4c2640 1 -- 172.21.15.32:0/1700854087 >> v2:172.21.15.32:3300/0 conn(0x7f0cc80af5e0 msgr2=0x7f0cc80af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0ccd4c2640 1 --2- 172.21.15.32:0/1700854087 >> v2:172.21.15.32:3300/0 conn(0x7f0cc80af5e0 0x7f0cc80af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0ccd4c2640 1 -- 172.21.15.32:0/1700854087 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cc815ebf0 con 0x7f0cc815f110 2023-12-20T04:47:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0cc7fff640 1 --2- 172.21.15.32:0/1700854087 >> v2:172.21.15.39:3300/0 conn(0x7f0cc806ce60 0x7f0cc815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0ccccc1640 1 --2- 172.21.15.32:0/1700854087 >> v2:172.21.15.32:3300/0 conn(0x7f0cc80af5e0 0x7f0cc80af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:21.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0ccd4c2640 1 --2- 172.21.15.32:0/1700854087 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0cc815f110 0x7f0cc815c440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0cc806abe0 tx=0x7f0cc001c910 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dd6283b767e46d1e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:21.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0cc77fe640 1 -- 172.21.15.32:0/1700854087 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00a58f0 con 0x7f0cc815f110 2023-12-20T04:47:21.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0cc77fe640 1 -- 172.21.15.32:0/1700854087 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc00a5a50 con 0x7f0cc815f110 2023-12-20T04:47:21.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.594+0000 7f0cc77fe640 1 -- 172.21.15.32:0/1700854087 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00a5de0 con 0x7f0cc815f110 2023-12-20T04:47:21.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 -- 172.21.15.32:0/1700854087 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0cc815f110 msgr2=0x7f0cc815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 --2- 172.21.15.32:0/1700854087 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0cc815f110 0x7f0cc815c440 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7f0cc806abe0 tx=0x7f0cc001c910 comp rx=0 tx=0).stop 2023-12-20T04:47:21.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 -- 172.21.15.32:0/1700854087 shutdown_connections 2023-12-20T04:47:21.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 --2- 172.21.15.32:0/1700854087 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0cc815f110 0x7f0cc815c440 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 --2- 172.21.15.32:0/1700854087 >> v2:172.21.15.39:3300/0 conn(0x7f0cc806ce60 0x7f0cc815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 --2- 172.21.15.32:0/1700854087 >> v2:172.21.15.32:3300/0 conn(0x7f0cc80af5e0 0x7f0cc80af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 -- 172.21.15.32:0/1700854087 >> 172.21.15.32:0/1700854087 conn(0x7f0cc806a320 msgr2=0x7f0cc80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:21.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 -- 172.21.15.32:0/1700854087 shutdown_connections 2023-12-20T04:47:21.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 -- 172.21.15.32:0/1700854087 wait complete. 2023-12-20T04:47:21.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 Processor -- start 2023-12-20T04:47:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 -- start start 2023-12-20T04:47:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0cc806ce60 0x7f0cc8071c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc80af5e0 0x7f0cc80cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc815f110 0x7f0cc814de60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc8159b00 con 0x7f0cc815f110 2023-12-20T04:47:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc8158ec0 con 0x7f0cc80af5e0 2023-12-20T04:47:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccef25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0cc8159160 con 0x7f0cc806ce60 2023-12-20T04:47:21.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccd4c2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc815f110 0x7f0cc814de60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccd4c2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc815f110 0x7f0cc814de60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34996/0 (socket says 172.21.15.32:34996) 2023-12-20T04:47:21.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.598+0000 7f0ccd4c2640 1 -- 172.21.15.32:0/1813582052 learned_addr learned my addr 172.21.15.32:0/1813582052 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:21.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0ccccc1640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0cc806ce60 0x7f0cc8071c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0cc7fff640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc80af5e0 0x7f0cc80cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0ccd4c2640 1 -- 172.21.15.32:0/1813582052 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0cc806ce60 msgr2=0x7f0cc8071c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0ccd4c2640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0cc806ce60 0x7f0cc8071c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0ccd4c2640 1 -- 172.21.15.32:0/1813582052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc80af5e0 msgr2=0x7f0cc80cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0ccd4c2640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc80af5e0 0x7f0cc80cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0ccd4c2640 1 -- 172.21.15.32:0/1813582052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cb4009420 con 0x7f0cc815f110 2023-12-20T04:47:21.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0ccd4c2640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc815f110 0x7f0cc814de60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0cc001d1b0 tx=0x7f0cc00196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:21.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0cc57fa640 1 -- 172.21.15.32:0/1813582052 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00b9650 con 0x7f0cc815f110 2023-12-20T04:47:21.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0cc57fa640 1 -- 172.21.15.32:0/1813582052 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cc00b97b0 con 0x7f0cc815f110 2023-12-20T04:47:21.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0cc57fa640 1 -- 172.21.15.32:0/1813582052 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cc00b9b60 con 0x7f0cc815f110 2023-12-20T04:47:21.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0ccef25640 1 -- 172.21.15.32:0/1813582052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0cbc009480 con 0x7f0cc815f110 2023-12-20T04:47:21.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0ccef25640 1 -- 172.21.15.32:0/1813582052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0cc0009a70 con 0x7f0cc815f110 2023-12-20T04:47:21.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0ccef25640 1 -- 172.21.15.32:0/1813582052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0cc8159c40 con 0x7f0cc815f110 2023-12-20T04:47:21.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0cc57fa640 1 -- 172.21.15.32:0/1813582052 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0cc00b9cc0 con 0x7f0cc815f110 2023-12-20T04:47:21.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0cc57fa640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cac065b80 0x7f0cac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.602+0000 7f0cc57fa640 1 -- 172.21.15.32:0/1813582052 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0cc011e6b0 con 0x7f0cc815f110 2023-12-20T04:47:21.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.606+0000 7f0ccccc1640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cac065b80 0x7f0cac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.606+0000 7f0ccccc1640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cac065b80 0x7f0cac068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc006fd0 tx=0x7f0cbc008040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:21.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.606+0000 7f0cc57fa640 1 -- 172.21.15.32:0/1813582052 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0cc8159c40 con 0x7f0cc815f110 2023-12-20T04:47:21.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.766+0000 7f0ccef25640 1 -- 172.21.15.32:0/1813582052 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0cc8159c40 con 0x7f0cc815f110 2023-12-20T04:47:21.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0cc57fa640 1 -- 172.21.15.32:0/1813582052 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v137) v1 ==== 74+0+16068 (secure 0 0 0) 0x7f0cc8159c40 con 0x7f0cc815f110 2023-12-20T04:47:21.767 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:21.767 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":137,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:47:21.050468+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:47:19.596882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-20T04:47:20.055023+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 -- 172.21.15.32:0/1813582052 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cac065b80 msgr2=0x7f0cac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cac065b80 0x7f0cac068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f0cbc006fd0 tx=0x7f0cbc008040 comp rx=0 tx=0).stop 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 -- 172.21.15.32:0/1813582052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc815f110 msgr2=0x7f0cc814de60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc815f110 0x7f0cc814de60 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7f0cc001d1b0 tx=0x7f0cc00196b0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 -- 172.21.15.32:0/1813582052 shutdown_connections 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cac065b80 0x7f0cac068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0cc815f110 0x7f0cc814de60 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0cc80af5e0 0x7f0cc80cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 --2- 172.21.15.32:0/1813582052 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0cc806ce60 0x7f0cc8071c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 -- 172.21.15.32:0/1813582052 >> 172.21.15.32:0/1813582052 conn(0x7f0cc806a320 msgr2=0x7f0cc80b3cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 -- 172.21.15.32:0/1813582052 shutdown_connections 2023-12-20T04:47:21.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.770+0000 7f0ccef25640 1 -- 172.21.15.32:0/1813582052 wait complete. 2023-12-20T04:47:21.789 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T04:47:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.938+0000 7f4e117e9640 1 Processor -- start 2023-12-20T04:47:21.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.938+0000 7f4e117e9640 1 -- start start 2023-12-20T04:47:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.938+0000 7f4e117e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e0c1591e0 0x7f4e0c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.938+0000 7f4e117e9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e0c070940 0x7f4e0c0cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.938+0000 7f4e117e9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e0c0cc3b0 con 0x7f4e0c06cdf0 2023-12-20T04:47:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.938+0000 7f4e117e9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e0c0c8e30 con 0x7f4e0c070940 2023-12-20T04:47:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.938+0000 7f4e117e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c06d280 con 0x7f4e0c1591e0 2023-12-20T04:47:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.938+0000 7f4e03fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e0c06cdf0 0x7f4e0c070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43920/0 (socket says 172.21.15.32:43920) 2023-12-20T04:47:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.938+0000 7f4e03fff640 1 -- 172.21.15.32:0/1406977020 learned_addr learned my addr 172.21.15.32:0/1406977020 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0b7fe640 1 --2- 172.21.15.32:0/1406977020 >> v2:172.21.15.32:3300/0 conn(0x7f4e0c070940 0x7f4e0c0cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0affd640 1 --2- 172.21.15.32:0/1406977020 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e0c1591e0 0x7f4e0c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0a7fc640 1 -- 172.21.15.32:0/1406977020 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2228525119 0 0) 0x7f4e0c0cc3b0 con 0x7f4e0c06cdf0 2023-12-20T04:47:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0a7fc640 1 -- 172.21.15.32:0/1406977020 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4df8003450 con 0x7f4e0c06cdf0 2023-12-20T04:47:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0b7fe640 1 -- 172.21.15.32:0/1406977020 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e0c1591e0 msgr2=0x7f4e0c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0b7fe640 1 --2- 172.21.15.32:0/1406977020 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e0c1591e0 0x7f4e0c0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0b7fe640 1 -- 172.21.15.32:0/1406977020 >> v1:172.21.15.32:6789/0 conn(0x7f4e0c06cdf0 legacy=0x7f4e0c070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0b7fe640 1 -- 172.21.15.32:0/1406977020 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e0c0c90d0 con 0x7f4e0c070940 2023-12-20T04:47:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0b7fe640 1 --2- 172.21.15.32:0/1406977020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c070940 0x7f4e0c0cc840 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009d90 tx=0x7f4e0401c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0b259c1d780011c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0a7fc640 1 -- 172.21.15.32:0/1406977020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e040a58e0 con 0x7f4e0c070940 2023-12-20T04:47:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0a7fc640 1 -- 172.21.15.32:0/1406977020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e040a5a40 con 0x7f4e0c070940 2023-12-20T04:47:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.942+0000 7f4e0a7fc640 1 -- 172.21.15.32:0/1406977020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e040a5dd0 con 0x7f4e0c070940 2023-12-20T04:47:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 -- 172.21.15.32:0/1406977020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c070940 msgr2=0x7f4e0c0cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 --2- 172.21.15.32:0/1406977020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c070940 0x7f4e0c0cc840 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f4e04009d90 tx=0x7f4e0401c8c0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 -- 172.21.15.32:0/1406977020 shutdown_connections 2023-12-20T04:47:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 --2- 172.21.15.32:0/1406977020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c070940 0x7f4e0c0cc840 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 --2- 172.21.15.32:0/1406977020 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e0c1591e0 0x7f4e0c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 -- 172.21.15.32:0/1406977020 >> 172.21.15.32:0/1406977020 conn(0x7f4e0c06a510 msgr2=0x7f4e0c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 -- 172.21.15.32:0/1406977020 shutdown_connections 2023-12-20T04:47:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 -- 172.21.15.32:0/1406977020 wait complete. 2023-12-20T04:47:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 Processor -- start 2023-12-20T04:47:21.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 -- start start 2023-12-20T04:47:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e0c070940 0x7f4e0c144e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1591e0 0x7f4e0c0c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e0c0c69f0 0x7f4e0c0c7860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0c5950 con 0x7f4e0c1591e0 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0c5c30 con 0x7f4e0c070940 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e117e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e0c0c5f10 con 0x7f4e0c0c69f0 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1591e0 0x7f4e0c0c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e03fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1591e0 0x7f4e0c0c7320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35014/0 (socket says 172.21.15.32:35014) 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e03fff640 1 -- 172.21.15.32:0/4256896464 learned_addr learned my addr 172.21.15.32:0/4256896464 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e0affd640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e0c070940 0x7f4e0c144e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e0b7fe640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e0c0c69f0 0x7f4e0c0c7860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e03fff640 1 -- 172.21.15.32:0/4256896464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e0c0c69f0 msgr2=0x7f4e0c0c7860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e03fff640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e0c0c69f0 0x7f4e0c0c7860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e03fff640 1 -- 172.21.15.32:0/4256896464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e0c070940 msgr2=0x7f4e0c144e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e03fff640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e0c070940 0x7f4e0c144e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e03fff640 1 -- 172.21.15.32:0/4256896464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e04009a70 con 0x7f4e0c1591e0 2023-12-20T04:47:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.946+0000 7f4e0b7fe640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e0c0c69f0 0x7f4e0c0c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.950+0000 7f4e03fff640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1591e0 0x7f4e0c0c7320 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f4df400c450 tx=0x7f4df400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.950+0000 7f4e037fe640 1 -- 172.21.15.32:0/4256896464 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4005cf0 con 0x7f4e0c1591e0 2023-12-20T04:47:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.950+0000 7f4e037fe640 1 -- 172.21.15.32:0/4256896464 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4df4005e50 con 0x7f4e0c1591e0 2023-12-20T04:47:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.950+0000 7f4e037fe640 1 -- 172.21.15.32:0/4256896464 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4df4007290 con 0x7f4e0c1591e0 2023-12-20T04:47:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.950+0000 7f4e117e9640 1 -- 172.21.15.32:0/4256896464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e0c0c61f0 con 0x7f4e0c1591e0 2023-12-20T04:47:21.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.950+0000 7f4e117e9640 1 -- 172.21.15.32:0/4256896464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e0c145350 con 0x7f4e0c1591e0 2023-12-20T04:47:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.950+0000 7f4e117e9640 1 -- 172.21.15.32:0/4256896464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e0c04e000 con 0x7f4e0c1591e0 2023-12-20T04:47:21.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.950+0000 7f4e037fe640 1 -- 172.21.15.32:0/4256896464 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4df40073f0 con 0x7f4e0c1591e0 2023-12-20T04:47:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.950+0000 7f4e037fe640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4dec065b80 0x7f4dec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.950+0000 7f4e037fe640 1 -- 172.21.15.32:0/4256896464 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(137..137 src has 1..137) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4df4082900 con 0x7f4e0c1591e0 2023-12-20T04:47:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.950+0000 7f4e0affd640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4dec065b80 0x7f4dec068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.954+0000 7f4e0affd640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4dec065b80 0x7f4dec068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009ac0 tx=0x7f4dfc009190 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:21.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:21.954+0000 7f4e037fe640 1 -- 172.21.15.32:0/4256896464 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4df40501b0 con 0x7f4e0c1591e0 2023-12-20T04:47:22.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:22.146+0000 7f4e117e9640 1 -- 172.21.15.32:0/4256896464 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f4e0c06d840 con 0x7f4e0c1591e0 2023-12-20T04:47:22.558 INFO:tasks.ceph.mds.c.smithi032.stderr:2023-12-20T04:47:22.558+0000 7f731ebe8640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T04:47:23.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e037fe640 1 -- 172.21.15.32:0/4256896464 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v61) v1 ==== 118+0+0 (secure 0 0 0) 0x7f4df4017030 con 0x7f4e0c1591e0 2023-12-20T04:47:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 -- 172.21.15.32:0/4256896464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4dec065b80 msgr2=0x7f4dec068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4dec065b80 0x7f4dec068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f4dfc009ac0 tx=0x7f4dfc009190 comp rx=0 tx=0).stop 2023-12-20T04:47:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 -- 172.21.15.32:0/4256896464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1591e0 msgr2=0x7f4e0c0c7320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1591e0 0x7f4e0c0c7320 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f4df400c450 tx=0x7f4df400c910 comp rx=0 tx=0).stop 2023-12-20T04:47:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 -- 172.21.15.32:0/4256896464 shutdown_connections 2023-12-20T04:47:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4dec065b80 0x7f4dec068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e0c0c69f0 0x7f4e0c0c7860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e0c1591e0 0x7f4e0c0c7320 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 --2- 172.21.15.32:0/4256896464 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e0c070940 0x7f4e0c144e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 -- 172.21.15.32:0/4256896464 >> 172.21.15.32:0/4256896464 conn(0x7f4e0c06a510 msgr2=0x7f4e0c0cd190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:23.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 -- 172.21.15.32:0/4256896464 shutdown_connections 2023-12-20T04:47:23.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.070+0000 7f4e117e9640 1 -- 172.21.15.32:0/4256896464 wait complete. 2023-12-20T04:47:23.093 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T04:47:23.094 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T04:47:23.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.246+0000 7ff067960640 1 Processor -- start 2023-12-20T04:47:23.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.246+0000 7ff067960640 1 -- start start 2023-12-20T04:47:23.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff067960640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff06015abb0 0x7ff0600caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:23.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff067960640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff060070230 0x7ff0600706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff067960640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff06015b6a0 con 0x7ff0600b0cb0 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff067960640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff06006e160 con 0x7ff06015abb0 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff067960640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff060070c00 con 0x7ff060070230 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff064efb640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0600b0cb0 0x7ff06015bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43932/0 (socket says 172.21.15.32:43932) 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff064efb640 1 -- 172.21.15.32:0/2788934992 learned_addr learned my addr 172.21.15.32:0/2788934992 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff065efd640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff060070230 0x7ff0600706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff0656fc640 1 --2- 172.21.15.32:0/2788934992 >> v2:172.21.15.39:3300/0 conn(0x7ff06015abb0 0x7ff0600caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2146303477 0 0) 0x7ff06015b6a0 con 0x7ff0600b0cb0 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff04c003530 con 0x7ff0600b0cb0 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2960632889 0 0) 0x7ff04c003530 con 0x7ff0600b0cb0 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff060070230 msgr2=0x7ff0600706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff060070230 0x7ff0600706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 >> v2:172.21.15.39:3300/0 conn(0x7ff06015abb0 msgr2=0x7ff0600caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 --2- 172.21.15.32:0/2788934992 >> v2:172.21.15.39:3300/0 conn(0x7ff06015abb0 0x7ff0600caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff06006d8c0 con 0x7ff0600b0cb0 2023-12-20T04:47:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7ff050001680 con 0x7ff0600b0cb0 2023-12-20T04:47:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 >> v1:172.21.15.32:6789/0 conn(0x7ff0600b0cb0 legacy=0x7ff06015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff060070230 0x7ff04c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c003f00 0x7ff04c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff04c004d10 0x7ff04c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff06006e160 con 0x7ff04c003f00 2023-12-20T04:47:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff060070c00 con 0x7ff04c004d10 2023-12-20T04:47:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff04c00b990 con 0x7ff060070230 2023-12-20T04:47:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff064efb640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c003f00 0x7ff04c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff0656fc640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff060070230 0x7ff04c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff065efd640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff04c004d10 0x7ff04c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff064efb640 1 -- 172.21.15.32:0/2788934992 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff060070230 msgr2=0x7ff04c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff064efb640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff060070230 0x7ff04c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff064efb640 1 -- 172.21.15.32:0/2788934992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff04c004d10 msgr2=0x7ff04c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff064efb640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff04c004d10 0x7ff04c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:23.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.250+0000 7ff064efb640 1 -- 172.21.15.32:0/2788934992 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff04c003530 con 0x7ff04c003f00 2023-12-20T04:47:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff064efb640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c003f00 0x7ff04c0047d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c210 tx=0x7ff05001f330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=311f9b815fb9bfd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050005ae0 con 0x7ff04c003f00 2023-12-20T04:47:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0500ab070 con 0x7ff04c003f00 2023-12-20T04:47:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff057fff640 1 -- 172.21.15.32:0/2788934992 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff050007070 con 0x7ff04c003f00 2023-12-20T04:47:23.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff067960640 1 -- 172.21.15.32:0/2788934992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c003f00 msgr2=0x7ff04c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff067960640 1 --2- 172.21.15.32:0/2788934992 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff04c003f00 0x7ff04c0047d0 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7ff05000c210 tx=0x7ff05001f330 comp rx=0 tx=0).stop 2023-12-20T04:47:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff065efd640 1 -- 172.21.15.32:0/2788934992 reap_dead start 2023-12-20T04:47:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff067960640 1 -- 172.21.15.32:0/2788934992 shutdown_connections 2023-12-20T04:47:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff067960640 1 -- 172.21.15.32:0/2788934992 >> 172.21.15.32:0/2788934992 conn(0x7ff06006a510 msgr2=0x7ff0600ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff067960640 1 -- 172.21.15.32:0/2788934992 shutdown_connections 2023-12-20T04:47:23.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff067960640 1 -- 172.21.15.32:0/2788934992 wait complete. 2023-12-20T04:47:23.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff067960640 1 Processor -- start 2023-12-20T04:47:23.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.254+0000 7ff067960640 1 -- start start 2023-12-20T04:47:23.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff067960640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff04c003f00 0x7ff060158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:23.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff067960640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0600a97b0 0x7ff0600a9c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:23.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff067960640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0600a74c0 0x7ff0600a7940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:23.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff067960640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0600a9140 con 0x7ff0600a74c0 2023-12-20T04:47:23.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff067960640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0600a7e80 con 0x7ff0600a97b0 2023-12-20T04:47:23.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff067960640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff060159ec0 con 0x7ff04c003f00 2023-12-20T04:47:23.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff065efd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0600a74c0 0x7ff0600a7940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:23.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff0656fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff04c003f00 0x7ff060158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:23.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff065efd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0600a74c0 0x7ff0600a7940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35024/0 (socket says 172.21.15.32:35024) 2023-12-20T04:47:23.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff065efd640 1 -- 172.21.15.32:0/4127302861 learned_addr learned my addr 172.21.15.32:0/4127302861 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:23.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff064efb640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0600a97b0 0x7ff0600a9c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff065efd640 1 -- 172.21.15.32:0/4127302861 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff04c003f00 msgr2=0x7ff060158d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff065efd640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff04c003f00 0x7ff060158d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff065efd640 1 -- 172.21.15.32:0/4127302861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0600a97b0 msgr2=0x7ff0600a9c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff065efd640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0600a97b0 0x7ff0600a9c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff065efd640 1 -- 172.21.15.32:0/4127302861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0600aa150 con 0x7ff0600a74c0 2023-12-20T04:47:23.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff065efd640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0600a74c0 0x7ff0600a7940 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7ff058011bf0 tx=0x7ff05800f1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff055ffb640 1 -- 172.21.15.32:0/4127302861 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05800e040 con 0x7ff0600a74c0 2023-12-20T04:47:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff055ffb640 1 -- 172.21.15.32:0/4127302861 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff05800f040 con 0x7ff0600a74c0 2023-12-20T04:47:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff067960640 1 -- 172.21.15.32:0/4127302861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0600a8cb0 con 0x7ff0600a74c0 2023-12-20T04:47:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff055ffb640 1 -- 172.21.15.32:0/4127302861 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058015430 con 0x7ff0600a74c0 2023-12-20T04:47:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff067960640 1 -- 172.21.15.32:0/4127302861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0600a8380 con 0x7ff0600a74c0 2023-12-20T04:47:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.258+0000 7ff067960640 1 -- 172.21.15.32:0/4127302861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff06004e000 con 0x7ff0600a74c0 2023-12-20T04:47:23.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.262+0000 7ff055ffb640 1 -- 172.21.15.32:0/4127302861 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff058015590 con 0x7ff0600a74c0 2023-12-20T04:47:23.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.262+0000 7ff055ffb640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff044065bb0 0x7ff044068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:23.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.262+0000 7ff055ffb640 1 -- 172.21.15.32:0/4127302861 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff058083910 con 0x7ff0600a74c0 2023-12-20T04:47:23.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.262+0000 7ff0656fc640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff044065bb0 0x7ff044068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:23.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.262+0000 7ff0656fc640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff044065bb0 0x7ff044068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009be0 tx=0x7ff05c009190 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:23.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.266+0000 7ff055ffb640 1 -- 172.21.15.32:0/4127302861 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff058051090 con 0x7ff0600a74c0 2023-12-20T04:47:23.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:23.458+0000 7ff067960640 1 -- 172.21.15.32:0/4127302861 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7ff0600ce880 con 0x7ff0600a74c0 2023-12-20T04:47:24.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff055ffb640 1 -- 172.21.15.32:0/4127302861 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v62) v1 ==== 122+0+0 (secure 0 0 0) 0x7ff0580546b0 con 0x7ff0600a74c0 2023-12-20T04:47:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 -- 172.21.15.32:0/4127302861 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff044065bb0 msgr2=0x7ff044068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff044065bb0 0x7ff044068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7ff05c009be0 tx=0x7ff05c009190 comp rx=0 tx=0).stop 2023-12-20T04:47:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 -- 172.21.15.32:0/4127302861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0600a74c0 msgr2=0x7ff0600a7940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0600a74c0 0x7ff0600a7940 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7ff058011bf0 tx=0x7ff05800f1b0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 -- 172.21.15.32:0/4127302861 shutdown_connections 2023-12-20T04:47:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff044065bb0 0x7ff044068070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0600a74c0 0x7ff0600a7940 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0600a97b0 0x7ff0600a9c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 --2- 172.21.15.32:0/4127302861 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff04c003f00 0x7ff060158d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 -- 172.21.15.32:0/4127302861 >> 172.21.15.32:0/4127302861 conn(0x7ff06006a510 msgr2=0x7ff0600cb830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:24.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 -- 172.21.15.32:0/4127302861 shutdown_connections 2023-12-20T04:47:24.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.078+0000 7ff067960640 1 -- 172.21.15.32:0/4127302861 wait complete. 2023-12-20T04:47:24.101 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T04:47:24.101 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:47:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.250+0000 7f17ab663640 1 Processor -- start 2023-12-20T04:47:24.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.250+0000 7f17ab663640 1 -- start start 2023-12-20T04:47:24.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.250+0000 7f17ab663640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f17a40ae010 0x7f17a4071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.250+0000 7f17ab663640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4164ef0 0x7f17a40719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.250+0000 7f17ab663640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17a40ae3f0 con 0x7f17a406cf10 2023-12-20T04:47:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.250+0000 7f17ab663640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17a40ca4b0 con 0x7f17a40ae010 2023-12-20T04:47:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.250+0000 7f17ab663640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a4161800 con 0x7f17a4164ef0 2023-12-20T04:47:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.250+0000 7f17a8bfe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f17a406cf10 0x7f17a40701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43936/0 (socket says 172.21.15.32:43936) 2023-12-20T04:47:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.250+0000 7f17a8bfe640 1 -- 172.21.15.32:0/3657817392 learned_addr learned my addr 172.21.15.32:0/3657817392 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a9c00640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4164ef0 0x7f17a40719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a93ff640 1 --2- 172.21.15.32:0/3657817392 >> v2:172.21.15.39:3300/0 conn(0x7f17a40ae010 0x7f17a4071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1481000602 0 0) 0x7f17a40ae3f0 con 0x7f17a406cf10 2023-12-20T04:47:24.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1790003530 con 0x7f17a406cf10 2023-12-20T04:47:24.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3970306702 0 0) 0x7f1790003530 con 0x7f17a406cf10 2023-12-20T04:47:24.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4164ef0 msgr2=0x7f17a40719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:24.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4164ef0 0x7f17a40719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 >> v2:172.21.15.39:3300/0 conn(0x7f17a40ae010 msgr2=0x7f17a4071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:24.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 --2- 172.21.15.32:0/3657817392 >> v2:172.21.15.39:3300/0 conn(0x7f17a40ae010 0x7f17a4071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a4165380 con 0x7f17a406cf10 2023-12-20T04:47:24.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f1794001680 con 0x7f17a406cf10 2023-12-20T04:47:24.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 >> v1:172.21.15.32:6789/0 conn(0x7f17a406cf10 legacy=0x7f17a40701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 0x7f1790003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17900047d0 0x7f1790004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1790005190 0x7f179000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a40ca4b0 con 0x7f1790003980 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a4161800 con 0x7f17900047d0 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1790005680 con 0x7f1790005190 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a93ff640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 0x7f1790003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a8bfe640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17900047d0 0x7f1790004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a9c00640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1790005190 0x7f179000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a93ff640 1 -- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1790005190 msgr2=0x7f179000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a93ff640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1790005190 0x7f179000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a93ff640 1 -- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17900047d0 msgr2=0x7f1790004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a93ff640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17900047d0 0x7f1790004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a93ff640 1 -- 172.21.15.32:0/3657817392 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1790003530 con 0x7f1790003980 2023-12-20T04:47:24.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a8bfe640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17900047d0 0x7f1790004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:24.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.254+0000 7f17a9c00640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1790005190 0x7f179000b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f17a93ff640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 0x7f1790003da0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009ac0 tx=0x7f17a001ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=809bc0b4e0a4613 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a00059a0 con 0x7f1790003980 2023-12-20T04:47:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a0005b00 con 0x7f1790003980 2023-12-20T04:47:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f179bfff640 1 -- 172.21.15.32:0/3657817392 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a00a5610 con 0x7f1790003980 2023-12-20T04:47:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f17ab663640 1 -- 172.21.15.32:0/3657817392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 msgr2=0x7f1790003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f17ab663640 1 --2- 172.21.15.32:0/3657817392 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 0x7f1790003da0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7f17a0009ac0 tx=0x7f17a001ca40 comp rx=0 tx=0).stop 2023-12-20T04:47:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f17a9c00640 1 -- 172.21.15.32:0/3657817392 reap_dead start 2023-12-20T04:47:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f17ab663640 1 -- 172.21.15.32:0/3657817392 shutdown_connections 2023-12-20T04:47:24.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f17ab663640 1 -- 172.21.15.32:0/3657817392 >> 172.21.15.32:0/3657817392 conn(0x7f17a406a320 msgr2=0x7f17a40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f17ab663640 1 -- 172.21.15.32:0/3657817392 shutdown_connections 2023-12-20T04:47:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f17ab663640 1 -- 172.21.15.32:0/3657817392 wait complete. 2023-12-20T04:47:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f17ab663640 1 Processor -- start 2023-12-20T04:47:24.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.258+0000 7f17ab663640 1 -- start start 2023-12-20T04:47:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17ab663640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 0x7f17a4155d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17ab663640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a40c28b0 0x7f17a4144df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17ab663640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4142350 0x7f17a415c7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17ab663640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a41562b0 con 0x7f1790003980 2023-12-20T04:47:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17ab663640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17a4145330 con 0x7f17a40c28b0 2023-12-20T04:47:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17ab663640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17a41637a0 con 0x7f17a4142350 2023-12-20T04:47:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a93ff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 0x7f17a4155d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a93ff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 0x7f17a4155d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35046/0 (socket says 172.21.15.32:35046) 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a93ff640 1 -- 172.21.15.32:0/2781470905 learned_addr learned my addr 172.21.15.32:0/2781470905 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a8bfe640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a40c28b0 0x7f17a4144df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a9c00640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4142350 0x7f17a415c7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a93ff640 1 -- 172.21.15.32:0/2781470905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4142350 msgr2=0x7f17a415c7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a93ff640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4142350 0x7f17a415c7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a93ff640 1 -- 172.21.15.32:0/2781470905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a40c28b0 msgr2=0x7f17a4144df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a93ff640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a40c28b0 0x7f17a4144df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a93ff640 1 -- 172.21.15.32:0/2781470905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17a4161ac0 con 0x7f1790003980 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a8bfe640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a40c28b0 0x7f17a4144df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a9c00640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4142350 0x7f17a415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17a93ff640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 0x7f17a4155d70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f17a001c870 tx=0x7f1790005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f1799ffb640 1 -- 172.21.15.32:0/2781470905 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a00a5d30 con 0x7f1790003980 2023-12-20T04:47:24.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f1799ffb640 1 -- 172.21.15.32:0/2781470905 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17a0005c70 con 0x7f1790003980 2023-12-20T04:47:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17ab663640 1 -- 172.21.15.32:0/2781470905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17a4161da0 con 0x7f1790003980 2023-12-20T04:47:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f1799ffb640 1 -- 172.21.15.32:0/2781470905 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17a0004020 con 0x7f1790003980 2023-12-20T04:47:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17ab663640 1 -- 172.21.15.32:0/2781470905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17a4162080 con 0x7f1790003980 2023-12-20T04:47:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f17ab663640 1 -- 172.21.15.32:0/2781470905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f17a4163600 con 0x7f1790003980 2023-12-20T04:47:24.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.262+0000 7f1799ffb640 1 -- 172.21.15.32:0/2781470905 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f17a0004180 con 0x7f1790003980 2023-12-20T04:47:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.266+0000 7f1799ffb640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1788065bb0 0x7f1788068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.266+0000 7f1799ffb640 1 -- 172.21.15.32:0/2781470905 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f17a011da60 con 0x7f1790003980 2023-12-20T04:47:24.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.266+0000 7f17a8bfe640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1788065bb0 0x7f1788068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.266+0000 7f17a8bfe640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1788065bb0 0x7f1788068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17a406cc30 tx=0x7f1794000ee0 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:24.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.270+0000 7f1799ffb640 1 -- 172.21.15.32:0/2781470905 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f17a4163600 con 0x7f1790003980 2023-12-20T04:47:24.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 -- 172.21.15.32:0/2781470905 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f17a4163600 con 0x7f1790003980 2023-12-20T04:47:24.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f1799ffb640 1 -- 172.21.15.32:0/2781470905 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4708 (secure 0 0 0) 0x7f17a4163600 con 0x7f1790003980 2023-12-20T04:47:24.460 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:24.460 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3306270605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3306270605},{"type":"v1","addr":"172.21.15.39:6834","nonce":3306270605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4000562557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4000562557},{"type":"v1","addr":"172.21.15.39:6835","nonce":4000562557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/310576469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":310576469},{"type":"v1","addr":"172.21.15.32:6835","nonce":310576469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:47:21.050837+0000","modified":"2023-12-20T04:47:24.070982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8397},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8397":{"gid":8397,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3314020360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3314020360},{"type":"v1","addr":"172.21.15.32:6837","nonce":3314020360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-20T04:47:24.461 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2023-12-20T04:47:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 -- 172.21.15.32:0/2781470905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1788065bb0 msgr2=0x7f1788068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1788065bb0 0x7f1788068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f17a406cc30 tx=0x7f1794000ee0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 -- 172.21.15.32:0/2781470905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 msgr2=0x7f17a4155d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 0x7f17a4155d70 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7f17a001c870 tx=0x7f1790005800 comp rx=0 tx=0).stop 2023-12-20T04:47:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 -- 172.21.15.32:0/2781470905 shutdown_connections 2023-12-20T04:47:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1788065bb0 0x7f1788068070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17a4142350 0x7f17a415c7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17a40c28b0 0x7f17a4144df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 --2- 172.21.15.32:0/2781470905 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1790003980 0x7f17a4155d70 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 -- 172.21.15.32:0/2781470905 >> 172.21.15.32:0/2781470905 conn(0x7f17a406a320 msgr2=0x7f17a40cc760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 -- 172.21.15.32:0/2781470905 shutdown_connections 2023-12-20T04:47:24.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.462+0000 7f17ab663640 1 -- 172.21.15.32:0/2781470905 wait complete. 2023-12-20T04:47:24.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T04:47:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f112a72c640 1 Processor -- start 2023-12-20T04:47:24.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f112a72c640 1 -- start start 2023-12-20T04:47:24.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f112a72c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11240af5e0 0x7f11240af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f112a72c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112415b990 con 0x7f112415f110 2023-12-20T04:47:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f112a72c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f112415bbd0 con 0x7f112406ce60 2023-12-20T04:47:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f112a72c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f112415beb0 con 0x7f11240af5e0 2023-12-20T04:47:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f1123fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f11240af5e0 0x7f11240af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f1128cc9640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f112415f110 0x7f112415c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43940/0 (socket says 172.21.15.32:43940) 2023-12-20T04:47:24.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f1128cc9640 1 -- 172.21.15.32:0/1626279629 learned_addr learned my addr 172.21.15.32:0/1626279629 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:24.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f1122ffd640 1 -- 172.21.15.32:0/1626279629 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4215605157 0 0) 0x7f111c0014a0 con 0x7f112415f110 2023-12-20T04:47:24.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f1122ffd640 1 -- 172.21.15.32:0/1626279629 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f110c003530 con 0x7f112415f110 2023-12-20T04:47:24.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f1123fff640 1 -- 172.21.15.32:0/1626279629 >> v1:172.21.15.39:6789/0 conn(0x7f112406ce60 legacy=0x7f112415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f1123fff640 1 -- 172.21.15.32:0/1626279629 >> v1:172.21.15.32:6789/0 conn(0x7f112415f110 legacy=0x7f112415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.626+0000 7f1123fff640 1 -- 172.21.15.32:0/1626279629 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f112415e910 con 0x7f11240af5e0 2023-12-20T04:47:24.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.630+0000 7f1123fff640 1 --2- 172.21.15.32:0/1626279629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11240af5e0 0x7f11240af9c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f11140097a0 tx=0x7f111401c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462c413eee2f67ca server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:24.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.630+0000 7f1122ffd640 1 -- 172.21.15.32:0/1626279629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11140ab650 con 0x7f11240af5e0 2023-12-20T04:47:24.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.630+0000 7f1122ffd640 1 -- 172.21.15.32:0/1626279629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f11140ab7b0 con 0x7f11240af5e0 2023-12-20T04:47:24.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.630+0000 7f1122ffd640 1 -- 172.21.15.32:0/1626279629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f11140abb40 con 0x7f11240af5e0 2023-12-20T04:47:24.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.630+0000 7f112a72c640 1 -- 172.21.15.32:0/1626279629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11240af5e0 msgr2=0x7f11240af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.630+0000 7f112a72c640 1 --2- 172.21.15.32:0/1626279629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11240af5e0 0x7f11240af9c0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7f11140097a0 tx=0x7f111401c430 comp rx=0 tx=0).stop 2023-12-20T04:47:24.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.630+0000 7f112a72c640 1 -- 172.21.15.32:0/1626279629 shutdown_connections 2023-12-20T04:47:24.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.630+0000 7f112a72c640 1 --2- 172.21.15.32:0/1626279629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11240af5e0 0x7f11240af9c0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.630+0000 7f112a72c640 1 -- 172.21.15.32:0/1626279629 >> 172.21.15.32:0/1626279629 conn(0x7f112406a320 msgr2=0x7f11240ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.630+0000 7f112a72c640 1 -- 172.21.15.32:0/1626279629 shutdown_connections 2023-12-20T04:47:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.630+0000 7f112a72c640 1 -- 172.21.15.32:0/1626279629 wait complete. 2023-12-20T04:47:24.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f112a72c640 1 Processor -- start 2023-12-20T04:47:24.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f112a72c640 1 -- start start 2023-12-20T04:47:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f112a72c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f112406ce60 0x7f11240b35b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f112a72c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11240af5e0 0x7f112414e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f112a72c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f112415f110 0x7f1124163fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f112a72c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11240b3af0 con 0x7f11240af5e0 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f112a72c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f11240b2550 con 0x7f112415f110 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f112a72c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f11240b27f0 con 0x7f112406ce60 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f11237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11240af5e0 0x7f112414e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f11237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11240af5e0 0x7f112414e9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35060/0 (socket says 172.21.15.32:35060) 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f11237fe640 1 -- 172.21.15.32:0/583551834 learned_addr learned my addr 172.21.15.32:0/583551834 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f11237fe640 1 -- 172.21.15.32:0/583551834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f112406ce60 msgr2=0x7f11240b35b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f11237fe640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f112406ce60 0x7f11240b35b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f11237fe640 1 -- 172.21.15.32:0/583551834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f112415f110 msgr2=0x7f1124163fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f11237fe640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f112415f110 0x7f1124163fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f11237fe640 1 -- 172.21.15.32:0/583551834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f11140093b0 con 0x7f11240af5e0 2023-12-20T04:47:24.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f11237fe640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11240af5e0 0x7f112414e9a0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f111800c500 tx=0x7f111800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f1120ff9640 1 -- 172.21.15.32:0/583551834 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118005220 con 0x7f11240af5e0 2023-12-20T04:47:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f1120ff9640 1 -- 172.21.15.32:0/583551834 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1118005380 con 0x7f11240af5e0 2023-12-20T04:47:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f1120ff9640 1 -- 172.21.15.32:0/583551834 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1118007270 con 0x7f11240af5e0 2023-12-20T04:47:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f112a72c640 1 -- 172.21.15.32:0/583551834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f11240b2ad0 con 0x7f11240af5e0 2023-12-20T04:47:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.634+0000 7f112a72c640 1 -- 172.21.15.32:0/583551834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f11240b3090 con 0x7f11240af5e0 2023-12-20T04:47:24.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.638+0000 7f112a72c640 1 -- 172.21.15.32:0/583551834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f112404e000 con 0x7f11240af5e0 2023-12-20T04:47:24.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.638+0000 7f1120ff9640 1 -- 172.21.15.32:0/583551834 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f11180073d0 con 0x7f11240af5e0 2023-12-20T04:47:24.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.638+0000 7f1120ff9640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1110065b80 0x7f1110068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.638+0000 7f1120ff9640 1 -- 172.21.15.32:0/583551834 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1118082cf0 con 0x7f11240af5e0 2023-12-20T04:47:24.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.638+0000 7f1123fff640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1110065b80 0x7f1110068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.638+0000 7f1123fff640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1110065b80 0x7f1110068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f11140af000 tx=0x7f1114008040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:24.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.642+0000 7f1120ff9640 1 -- 172.21.15.32:0/583551834 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f11180504f0 con 0x7f11240af5e0 2023-12-20T04:47:24.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.798+0000 7f112a72c640 1 -- 172.21.15.32:0/583551834 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f112415c160 con 0x7f11240af5e0 2023-12-20T04:47:24.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f1120ff9640 1 -- 172.21.15.32:0/583551834 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v138) v1 ==== 74+0+16055 (secure 0 0 0) 0x7f1118053b10 con 0x7f11240af5e0 2023-12-20T04:47:24.799 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:24.799 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":138,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T04:47:22.053797+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:47:19.596882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-20T04:47:20.055023+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"138","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T04:47:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 -- 172.21.15.32:0/583551834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1110065b80 msgr2=0x7f1110068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1110065b80 0x7f1110068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f11140af000 tx=0x7f1114008040 comp rx=0 tx=0).stop 2023-12-20T04:47:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 -- 172.21.15.32:0/583551834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11240af5e0 msgr2=0x7f112414e9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11240af5e0 0x7f112414e9a0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7f111800c500 tx=0x7f111800c9c0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 -- 172.21.15.32:0/583551834 shutdown_connections 2023-12-20T04:47:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1110065b80 0x7f1110068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f112415f110 0x7f1124163fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f11240af5e0 0x7f112414e9a0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 --2- 172.21.15.32:0/583551834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f112406ce60 0x7f11240b35b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 -- 172.21.15.32:0/583551834 >> 172.21.15.32:0/583551834 conn(0x7f112406a320 msgr2=0x7f11240ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 -- 172.21.15.32:0/583551834 shutdown_connections 2023-12-20T04:47:24.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.802+0000 7f112a72c640 1 -- 172.21.15.32:0/583551834 wait complete. 2023-12-20T04:47:24.821 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T04:47:24.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:47:24.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0e0d798640 1 Processor -- start 2023-12-20T04:47:24.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0e0d798640 1 -- start start 2023-12-20T04:47:24.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0e0d798640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0e0806deb0 0x7f0e08161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0e0d798640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e08070950 0x7f0e080cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0e0d798640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0e08071920 con 0x7f0e0815ce30 2023-12-20T04:47:24.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0e0d798640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0e08070de0 con 0x7f0e0806deb0 2023-12-20T04:47:24.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0e0d798640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e08071080 con 0x7f0e08070950 2023-12-20T04:47:24.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0dfffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0e0806deb0 0x7f0e08161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0e077fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e08070950 0x7f0e080cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0dfffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0e0806deb0 0x7f0e08161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:58276/0 (socket says 172.21.15.32:58276) 2023-12-20T04:47:24.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0dfffff640 1 -- 172.21.15.32:0/772029858 learned_addr learned my addr 172.21.15.32:0/772029858 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:24.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.966+0000 7f0e06ffd640 1 --1- 172.21.15.32:0/772029858 >> v1:172.21.15.39:6789/0 conn(0x7f0e0815ce30 0x7f0e0806d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:36946/0 (socket says 172.21.15.32:36946) 2023-12-20T04:47:24.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.970+0000 7f0dfffff640 1 -- 172.21.15.32:0/772029858 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e08070950 msgr2=0x7f0e080cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:24.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.970+0000 7f0dfffff640 1 --2- 172.21.15.32:0/772029858 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e08070950 0x7f0e080cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.970+0000 7f0dfffff640 1 -- 172.21.15.32:0/772029858 >> v1:172.21.15.39:6789/0 conn(0x7f0e0815ce30 legacy=0x7f0e0806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.970+0000 7f0dfffff640 1 -- 172.21.15.32:0/772029858 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0e08071360 con 0x7f0e0806deb0 2023-12-20T04:47:24.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.970+0000 7f0e067fc640 1 -- 172.21.15.32:0/772029858 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 228344122 0 0) 0x7f0e08071920 con 0x7f0e0815ce30 2023-12-20T04:47:24.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.970+0000 7f0dfffff640 1 --2- 172.21.15.32:0/772029858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0806deb0 0x7f0e08161560 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009870 tx=0x7f0df001c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=60de4400bd0d9540 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:24.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.970+0000 7f0e067fc640 1 -- 172.21.15.32:0/772029858 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df00a7070 con 0x7f0e0806deb0 2023-12-20T04:47:24.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.970+0000 7f0e067fc640 1 -- 172.21.15.32:0/772029858 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df00ab430 con 0x7f0e0806deb0 2023-12-20T04:47:24.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.970+0000 7f0e067fc640 1 -- 172.21.15.32:0/772029858 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df00ab7c0 con 0x7f0e0806deb0 2023-12-20T04:47:24.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.970+0000 7f0e0d798640 1 -- 172.21.15.32:0/772029858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0806deb0 msgr2=0x7f0e08161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.970+0000 7f0e0d798640 1 --2- 172.21.15.32:0/772029858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0806deb0 0x7f0e08161560 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f0df0009870 tx=0x7f0df001c540 comp rx=0 tx=0).stop 2023-12-20T04:47:24.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 -- 172.21.15.32:0/772029858 shutdown_connections 2023-12-20T04:47:24.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 --2- 172.21.15.32:0/772029858 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e08070950 0x7f0e080cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 --2- 172.21.15.32:0/772029858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0806deb0 0x7f0e08161560 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 -- 172.21.15.32:0/772029858 >> 172.21.15.32:0/772029858 conn(0x7f0e080cba50 msgr2=0x7f0e080cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:24.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 -- 172.21.15.32:0/772029858 shutdown_connections 2023-12-20T04:47:24.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 -- 172.21.15.32:0/772029858 wait complete. 2023-12-20T04:47:24.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 Processor -- start 2023-12-20T04:47:24.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 -- start start 2023-12-20T04:47:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0806deb0 0x7f0e08153ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08070950 0x7f0e0815fe60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e0815ce30 0x7f0e081576f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080c3110 con 0x7f0e08070950 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0e08155ee0 con 0x7f0e0806deb0 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e0d798640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0e080c2ae0 con 0x7f0e0815ce30 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08070950 0x7f0e0815fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0dfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08070950 0x7f0e0815fe60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35064/0 (socket says 172.21.15.32:35064) 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0dfffff640 1 -- 172.21.15.32:0/2714840074 learned_addr learned my addr 172.21.15.32:0/2714840074 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e06ffd640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0806deb0 0x7f0e08153ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e077fe640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e0815ce30 0x7f0e081576f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0dfffff640 1 -- 172.21.15.32:0/2714840074 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e0815ce30 msgr2=0x7f0e081576f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0dfffff640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e0815ce30 0x7f0e081576f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0dfffff640 1 -- 172.21.15.32:0/2714840074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0806deb0 msgr2=0x7f0e08153ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0dfffff640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0806deb0 0x7f0e08153ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:24.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0dfffff640 1 -- 172.21.15.32:0/2714840074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0df00094d0 con 0x7f0e08070950 2023-12-20T04:47:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.974+0000 7f0e077fe640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e0815ce30 0x7f0e081576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.978+0000 7f0dfffff640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08070950 0x7f0e0815fe60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0df001abf0 tx=0x7f0df00192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.978+0000 7f0dff7fe640 1 -- 172.21.15.32:0/2714840074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df00b4620 con 0x7f0e08070950 2023-12-20T04:47:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.978+0000 7f0dff7fe640 1 -- 172.21.15.32:0/2714840074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0df00b4780 con 0x7f0e08070950 2023-12-20T04:47:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.978+0000 7f0e0d798640 1 -- 172.21.15.32:0/2714840074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0e08154400 con 0x7f0e08070950 2023-12-20T04:47:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.978+0000 7f0dff7fe640 1 -- 172.21.15.32:0/2714840074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0df00b4b30 con 0x7f0e08070950 2023-12-20T04:47:24.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.978+0000 7f0e0d798640 1 -- 172.21.15.32:0/2714840074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0e08155880 con 0x7f0e08070950 2023-12-20T04:47:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.978+0000 7f0e0d798640 1 -- 172.21.15.32:0/2714840074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0e0804e000 con 0x7f0e08070950 2023-12-20T04:47:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.978+0000 7f0dff7fe640 1 -- 172.21.15.32:0/2714840074 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0df00b4c90 con 0x7f0e08070950 2023-12-20T04:47:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.978+0000 7f0dff7fe640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0de8065b80 0x7f0de8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.978+0000 7f0dff7fe640 1 -- 172.21.15.32:0/2714840074 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f0df011d750 con 0x7f0e08070950 2023-12-20T04:47:24.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.978+0000 7f0e06ffd640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0de8065b80 0x7f0de8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:24.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.982+0000 7f0e06ffd640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0de8065b80 0x7f0de8068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0df8000f70 tx=0x7f0df8007c10 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:24.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:24.982+0000 7f0dff7fe640 1 -- 172.21.15.32:0/2714840074 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0df00eaec0 con 0x7f0e08070950 2023-12-20T04:47:25.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.142+0000 7f0e0d798640 1 -- 172.21.15.32:0/2714840074 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f0e080ce270 con 0x7f0de8065b80 2023-12-20T04:47:25.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.150+0000 7f0dff7fe640 1 -- 172.21.15.32:0/2714840074 <== mgr.8172 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289314 (secure 0 0 0) 0x7f0df8007270 con 0x7f0de8065b80 2023-12-20T04:47:25.153 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:25.154 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:47:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 -- 172.21.15.32:0/2714840074 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0de8065b80 msgr2=0x7f0de8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0de8065b80 0x7f0de8068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f0df8000f70 tx=0x7f0df8007c10 comp rx=0 tx=0).stop 2023-12-20T04:47:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 -- 172.21.15.32:0/2714840074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08070950 msgr2=0x7f0e0815fe60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08070950 0x7f0e0815fe60 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7f0df001abf0 tx=0x7f0df00192e0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 -- 172.21.15.32:0/2714840074 shutdown_connections 2023-12-20T04:47:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0de8065b80 0x7f0de8068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0e0815ce30 0x7f0e081576f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0e08070950 0x7f0e0815fe60 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 --2- 172.21.15.32:0/2714840074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0e0806deb0 0x7f0e08153ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 -- 172.21.15.32:0/2714840074 >> 172.21.15.32:0/2714840074 conn(0x7f0e080cba50 msgr2=0x7f0e0806ac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 -- 172.21.15.32:0/2714840074 shutdown_connections 2023-12-20T04:47:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.154+0000 7f0e0d798640 1 -- 172.21.15.32:0/2714840074 wait complete. 2023-12-20T04:47:25.172 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-20T04:47:23.284420+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1021,"num_read_kb":1995,"num_write":385,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3140,"ondisk_log_size":3140,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":341692,"kb_used_data":14588,"kb_used_omap":0,"kb_used_meta":327040,"kb_avail":754633028,"statfs":{"total":773094113280,"available":772744220672,"internally_reserved":0,"allocated":14938112,"data_stored":7702164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":334888960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000725"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350904+0000","last_change":"2023-12-20T04:47:21.091423+0000","last_active":"2023-12-20T04:47:22.350904+0000","last_peered":"2023-12-20T04:47:22.350904+0000","last_clean":"2023-12-20T04:47:22.350904+0000","last_became_active":"2023-12-20T04:47:21.091106+0000","last_became_peered":"2023-12-20T04:47:21.091106+0000","last_unstale":"2023-12-20T04:47:22.350904+0000","last_undegraded":"2023-12-20T04:47:22.350904+0000","last_fullsized":"2023-12-20T04:47:22.350904+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:46:22.464171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066783+0000","last_change":"2023-12-20T04:47:21.075045+0000","last_active":"2023-12-20T04:47:22.066783+0000","last_peered":"2023-12-20T04:47:22.066783+0000","last_clean":"2023-12-20T04:47:22.066783+0000","last_became_active":"2023-12-20T04:47:21.074774+0000","last_became_peered":"2023-12-20T04:47:21.074774+0000","last_unstale":"2023-12-20T04:47:22.066783+0000","last_undegraded":"2023-12-20T04:47:22.066783+0000","last_fullsized":"2023-12-20T04:47:22.066783+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:24.251906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517125+0000","last_change":"2023-12-20T04:47:21.069629+0000","last_active":"2023-12-20T04:47:22.517125+0000","last_peered":"2023-12-20T04:47:22.517125+0000","last_clean":"2023-12-20T04:47:22.517125+0000","last_became_active":"2023-12-20T04:47:21.069171+0000","last_became_peered":"2023-12-20T04:47:21.069171+0000","last_unstale":"2023-12-20T04:47:22.517125+0000","last_undegraded":"2023-12-20T04:47:22.517125+0000","last_fullsized":"2023-12-20T04:47:22.517125+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:24.251906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350602+0000","last_change":"2023-12-20T04:47:21.075343+0000","last_active":"2023-12-20T04:47:22.350602+0000","last_peered":"2023-12-20T04:47:22.350602+0000","last_clean":"2023-12-20T04:47:22.350602+0000","last_became_active":"2023-12-20T04:47:21.075077+0000","last_became_peered":"2023-12-20T04:47:21.075077+0000","last_unstale":"2023-12-20T04:47:22.350602+0000","last_undegraded":"2023-12-20T04:47:22.350602+0000","last_fullsized":"2023-12-20T04:47:22.350602+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:23:26.602240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.514978+0000","last_change":"2023-12-20T04:47:21.076280+0000","last_active":"2023-12-20T04:47:22.514978+0000","last_peered":"2023-12-20T04:47:22.514978+0000","last_clean":"2023-12-20T04:47:22.514978+0000","last_became_active":"2023-12-20T04:47:21.076035+0000","last_became_peered":"2023-12-20T04:47:21.076035+0000","last_unstale":"2023-12-20T04:47:22.514978+0000","last_undegraded":"2023-12-20T04:47:22.514978+0000","last_fullsized":"2023-12-20T04:47:22.514978+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:02.256326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063682+0000","last_change":"2023-12-20T04:47:21.079746+0000","last_active":"2023-12-20T04:47:22.063682+0000","last_peered":"2023-12-20T04:47:22.063682+0000","last_clean":"2023-12-20T04:47:22.063682+0000","last_became_active":"2023-12-20T04:47:21.077814+0000","last_became_peered":"2023-12-20T04:47:21.077814+0000","last_unstale":"2023-12-20T04:47:22.063682+0000","last_undegraded":"2023-12-20T04:47:22.063682+0000","last_fullsized":"2023-12-20T04:47:22.063682+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:01:58.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067161+0000","last_change":"2023-12-20T04:47:21.081421+0000","last_active":"2023-12-20T04:47:22.067161+0000","last_peered":"2023-12-20T04:47:22.067161+0000","last_clean":"2023-12-20T04:47:22.067161+0000","last_became_active":"2023-12-20T04:47:21.081082+0000","last_became_peered":"2023-12-20T04:47:21.081082+0000","last_unstale":"2023-12-20T04:47:22.067161+0000","last_undegraded":"2023-12-20T04:47:22.067161+0000","last_fullsized":"2023-12-20T04:47:22.067161+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:36:31.212164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517430+0000","last_change":"2023-12-20T04:47:21.079984+0000","last_active":"2023-12-20T04:47:22.517430+0000","last_peered":"2023-12-20T04:47:22.517430+0000","last_clean":"2023-12-20T04:47:22.517430+0000","last_became_active":"2023-12-20T04:47:21.078965+0000","last_became_peered":"2023-12-20T04:47:21.078965+0000","last_unstale":"2023-12-20T04:47:22.517430+0000","last_undegraded":"2023-12-20T04:47:22.517430+0000","last_fullsized":"2023-12-20T04:47:22.517430+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:14:04.540171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350916+0000","last_change":"2023-12-20T04:47:21.086683+0000","last_active":"2023-12-20T04:47:22.350916+0000","last_peered":"2023-12-20T04:47:22.350916+0000","last_clean":"2023-12-20T04:47:22.350916+0000","last_became_active":"2023-12-20T04:47:21.086380+0000","last_became_peered":"2023-12-20T04:47:21.086380+0000","last_unstale":"2023-12-20T04:47:22.350916+0000","last_undegraded":"2023-12-20T04:47:22.350916+0000","last_fullsized":"2023-12-20T04:47:22.350916+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:58.717666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350761+0000","last_change":"2023-12-20T04:47:21.089785+0000","last_active":"2023-12-20T04:47:22.350761+0000","last_peered":"2023-12-20T04:47:22.350761+0000","last_clean":"2023-12-20T04:47:22.350761+0000","last_became_active":"2023-12-20T04:47:21.089533+0000","last_became_peered":"2023-12-20T04:47:21.089533+0000","last_unstale":"2023-12-20T04:47:22.350761+0000","last_undegraded":"2023-12-20T04:47:22.350761+0000","last_fullsized":"2023-12-20T04:47:22.350761+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:56:36.089809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066512+0000","last_change":"2023-12-20T04:47:21.087132+0000","last_active":"2023-12-20T04:47:22.066512+0000","last_peered":"2023-12-20T04:47:22.066512+0000","last_clean":"2023-12-20T04:47:22.066512+0000","last_became_active":"2023-12-20T04:47:21.086798+0000","last_became_peered":"2023-12-20T04:47:21.086798+0000","last_unstale":"2023-12-20T04:47:22.066512+0000","last_undegraded":"2023-12-20T04:47:22.066512+0000","last_fullsized":"2023-12-20T04:47:22.066512+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:33:53.116054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350677+0000","last_change":"2023-12-20T04:47:21.092487+0000","last_active":"2023-12-20T04:47:22.350677+0000","last_peered":"2023-12-20T04:47:22.350677+0000","last_clean":"2023-12-20T04:47:22.350677+0000","last_became_active":"2023-12-20T04:47:21.092186+0000","last_became_peered":"2023-12-20T04:47:21.092186+0000","last_unstale":"2023-12-20T04:47:22.350677+0000","last_undegraded":"2023-12-20T04:47:22.350677+0000","last_fullsized":"2023-12-20T04:47:22.350677+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:40.036506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517007+0000","last_change":"2023-12-20T04:47:21.077812+0000","last_active":"2023-12-20T04:47:22.517007+0000","last_peered":"2023-12-20T04:47:22.517007+0000","last_clean":"2023-12-20T04:47:22.517007+0000","last_became_active":"2023-12-20T04:47:21.076837+0000","last_became_peered":"2023-12-20T04:47:21.076837+0000","last_unstale":"2023-12-20T04:47:22.517007+0000","last_undegraded":"2023-12-20T04:47:22.517007+0000","last_fullsized":"2023-12-20T04:47:22.517007+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:34:04.890556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.516778+0000","last_change":"2023-12-20T04:47:21.083103+0000","last_active":"2023-12-20T04:47:22.516778+0000","last_peered":"2023-12-20T04:47:22.516778+0000","last_clean":"2023-12-20T04:47:22.516778+0000","last_became_active":"2023-12-20T04:47:21.082803+0000","last_became_peered":"2023-12-20T04:47:21.082803+0000","last_unstale":"2023-12-20T04:47:22.516778+0000","last_undegraded":"2023-12-20T04:47:22.516778+0000","last_fullsized":"2023-12-20T04:47:22.516778+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:05:19.725201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063600+0000","last_change":"2023-12-20T04:47:21.080822+0000","last_active":"2023-12-20T04:47:22.063600+0000","last_peered":"2023-12-20T04:47:22.063600+0000","last_clean":"2023-12-20T04:47:22.063600+0000","last_became_active":"2023-12-20T04:47:21.079908+0000","last_became_peered":"2023-12-20T04:47:21.079908+0000","last_unstale":"2023-12-20T04:47:22.063600+0000","last_undegraded":"2023-12-20T04:47:22.063600+0000","last_fullsized":"2023-12-20T04:47:22.063600+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:32:59.997762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350361+0000","last_change":"2023-12-20T04:47:21.087903+0000","last_active":"2023-12-20T04:47:22.350361+0000","last_peered":"2023-12-20T04:47:22.350361+0000","last_clean":"2023-12-20T04:47:22.350361+0000","last_became_active":"2023-12-20T04:47:21.087650+0000","last_became_peered":"2023-12-20T04:47:21.087650+0000","last_unstale":"2023-12-20T04:47:22.350361+0000","last_undegraded":"2023-12-20T04:47:22.350361+0000","last_fullsized":"2023-12-20T04:47:22.350361+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:00:12.243151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066720+0000","last_change":"2023-12-20T04:47:21.087850+0000","last_active":"2023-12-20T04:47:22.066720+0000","last_peered":"2023-12-20T04:47:22.066720+0000","last_clean":"2023-12-20T04:47:22.066720+0000","last_became_active":"2023-12-20T04:47:21.086811+0000","last_became_peered":"2023-12-20T04:47:21.086811+0000","last_unstale":"2023-12-20T04:47:22.066720+0000","last_undegraded":"2023-12-20T04:47:22.066720+0000","last_fullsized":"2023-12-20T04:47:22.066720+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:35.008516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.516872+0000","last_change":"2023-12-20T04:47:21.088414+0000","last_active":"2023-12-20T04:47:22.516872+0000","last_peered":"2023-12-20T04:47:22.516872+0000","last_clean":"2023-12-20T04:47:22.516872+0000","last_became_active":"2023-12-20T04:47:21.087622+0000","last_became_peered":"2023-12-20T04:47:21.087622+0000","last_unstale":"2023-12-20T04:47:22.516872+0000","last_undegraded":"2023-12-20T04:47:22.516872+0000","last_fullsized":"2023-12-20T04:47:22.516872+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:52:31.943799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517204+0000","last_change":"2023-12-20T04:47:21.083411+0000","last_active":"2023-12-20T04:47:22.517204+0000","last_peered":"2023-12-20T04:47:22.517204+0000","last_clean":"2023-12-20T04:47:22.517204+0000","last_became_active":"2023-12-20T04:47:21.082408+0000","last_became_peered":"2023-12-20T04:47:21.082408+0000","last_unstale":"2023-12-20T04:47:22.517204+0000","last_undegraded":"2023-12-20T04:47:22.517204+0000","last_fullsized":"2023-12-20T04:47:22.517204+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:40:35.208765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066484+0000","last_change":"2023-12-20T04:47:21.071406+0000","last_active":"2023-12-20T04:47:22.066484+0000","last_peered":"2023-12-20T04:47:22.066484+0000","last_clean":"2023-12-20T04:47:22.066484+0000","last_became_active":"2023-12-20T04:47:21.071095+0000","last_became_peered":"2023-12-20T04:47:21.071095+0000","last_unstale":"2023-12-20T04:47:22.066484+0000","last_undegraded":"2023-12-20T04:47:22.066484+0000","last_fullsized":"2023-12-20T04:47:22.066484+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:36:24.652104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067046+0000","last_change":"2023-12-20T04:47:21.077702+0000","last_active":"2023-12-20T04:47:22.067046+0000","last_peered":"2023-12-20T04:47:22.067046+0000","last_clean":"2023-12-20T04:47:22.067046+0000","last_became_active":"2023-12-20T04:47:21.075896+0000","last_became_peered":"2023-12-20T04:47:21.075896+0000","last_unstale":"2023-12-20T04:47:22.067046+0000","last_undegraded":"2023-12-20T04:47:22.067046+0000","last_fullsized":"2023-12-20T04:47:22.067046+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:22.066012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515466+0000","last_change":"2023-12-20T04:47:21.083596+0000","last_active":"2023-12-20T04:47:22.515466+0000","last_peered":"2023-12-20T04:47:22.515466+0000","last_clean":"2023-12-20T04:47:22.515466+0000","last_became_active":"2023-12-20T04:47:21.083346+0000","last_became_peered":"2023-12-20T04:47:21.083346+0000","last_unstale":"2023-12-20T04:47:22.515466+0000","last_undegraded":"2023-12-20T04:47:22.515466+0000","last_fullsized":"2023-12-20T04:47:22.515466+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:20:30.137015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066204+0000","last_change":"2023-12-20T04:47:21.072642+0000","last_active":"2023-12-20T04:47:22.066204+0000","last_peered":"2023-12-20T04:47:22.066204+0000","last_clean":"2023-12-20T04:47:22.066204+0000","last_became_active":"2023-12-20T04:47:21.071745+0000","last_became_peered":"2023-12-20T04:47:21.071745+0000","last_unstale":"2023-12-20T04:47:22.066204+0000","last_undegraded":"2023-12-20T04:47:22.066204+0000","last_fullsized":"2023-12-20T04:47:22.066204+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:04.754646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515142+0000","last_change":"2023-12-20T04:47:21.071373+0000","last_active":"2023-12-20T04:47:22.515142+0000","last_peered":"2023-12-20T04:47:22.515142+0000","last_clean":"2023-12-20T04:47:22.515142+0000","last_became_active":"2023-12-20T04:47:21.070805+0000","last_became_peered":"2023-12-20T04:47:21.070805+0000","last_unstale":"2023-12-20T04:47:22.515142+0000","last_undegraded":"2023-12-20T04:47:22.515142+0000","last_fullsized":"2023-12-20T04:47:22.515142+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:57.798722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066836+0000","last_change":"2023-12-20T04:47:21.087422+0000","last_active":"2023-12-20T04:47:22.066836+0000","last_peered":"2023-12-20T04:47:22.066836+0000","last_clean":"2023-12-20T04:47:22.066836+0000","last_became_active":"2023-12-20T04:47:21.086945+0000","last_became_peered":"2023-12-20T04:47:21.086945+0000","last_unstale":"2023-12-20T04:47:22.066836+0000","last_undegraded":"2023-12-20T04:47:22.066836+0000","last_fullsized":"2023-12-20T04:47:22.066836+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:40.721761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350787+0000","last_change":"2023-12-20T04:47:21.086979+0000","last_active":"2023-12-20T04:47:22.350787+0000","last_peered":"2023-12-20T04:47:22.350787+0000","last_clean":"2023-12-20T04:47:22.350787+0000","last_became_active":"2023-12-20T04:47:21.079415+0000","last_became_peered":"2023-12-20T04:47:21.079415+0000","last_unstale":"2023-12-20T04:47:22.350787+0000","last_undegraded":"2023-12-20T04:47:22.350787+0000","last_fullsized":"2023-12-20T04:47:22.350787+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:51:20.521357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.064095+0000","last_change":"2023-12-20T04:47:21.084937+0000","last_active":"2023-12-20T04:47:22.064095+0000","last_peered":"2023-12-20T04:47:22.064095+0000","last_clean":"2023-12-20T04:47:22.064095+0000","last_became_active":"2023-12-20T04:47:21.084486+0000","last_became_peered":"2023-12-20T04:47:21.084486+0000","last_unstale":"2023-12-20T04:47:22.064095+0000","last_undegraded":"2023-12-20T04:47:22.064095+0000","last_fullsized":"2023-12-20T04:47:22.064095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:57:47.330544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515724+0000","last_change":"2023-12-20T04:47:21.084096+0000","last_active":"2023-12-20T04:47:22.515724+0000","last_peered":"2023-12-20T04:47:22.515724+0000","last_clean":"2023-12-20T04:47:22.515724+0000","last_became_active":"2023-12-20T04:47:21.083160+0000","last_became_peered":"2023-12-20T04:47:21.083160+0000","last_unstale":"2023-12-20T04:47:22.515724+0000","last_undegraded":"2023-12-20T04:47:22.515724+0000","last_fullsized":"2023-12-20T04:47:22.515724+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:29:20.024881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066190+0000","last_change":"2023-12-20T04:47:21.073672+0000","last_active":"2023-12-20T04:47:22.066190+0000","last_peered":"2023-12-20T04:47:22.066190+0000","last_clean":"2023-12-20T04:47:22.066190+0000","last_became_active":"2023-12-20T04:47:21.071261+0000","last_became_peered":"2023-12-20T04:47:21.071261+0000","last_unstale":"2023-12-20T04:47:22.066190+0000","last_undegraded":"2023-12-20T04:47:22.066190+0000","last_fullsized":"2023-12-20T04:47:22.066190+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:18:33.360364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067124+0000","last_change":"2023-12-20T04:47:21.079530+0000","last_active":"2023-12-20T04:47:22.067124+0000","last_peered":"2023-12-20T04:47:22.067124+0000","last_clean":"2023-12-20T04:47:22.067124+0000","last_became_active":"2023-12-20T04:47:21.079239+0000","last_became_peered":"2023-12-20T04:47:21.079239+0000","last_unstale":"2023-12-20T04:47:22.067124+0000","last_undegraded":"2023-12-20T04:47:22.067124+0000","last_fullsized":"2023-12-20T04:47:22.067124+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:10:55.384589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350457+0000","last_change":"2023-12-20T04:47:21.077303+0000","last_active":"2023-12-20T04:47:22.350457+0000","last_peered":"2023-12-20T04:47:22.350457+0000","last_clean":"2023-12-20T04:47:22.350457+0000","last_became_active":"2023-12-20T04:47:21.074740+0000","last_became_peered":"2023-12-20T04:47:21.074740+0000","last_unstale":"2023-12-20T04:47:22.350457+0000","last_undegraded":"2023-12-20T04:47:22.350457+0000","last_fullsized":"2023-12-20T04:47:22.350457+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:02:05.096070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063599+0000","last_change":"2023-12-20T04:47:21.080479+0000","last_active":"2023-12-20T04:47:22.063599+0000","last_peered":"2023-12-20T04:47:22.063599+0000","last_clean":"2023-12-20T04:47:22.063599+0000","last_became_active":"2023-12-20T04:47:21.079140+0000","last_became_peered":"2023-12-20T04:47:21.079140+0000","last_unstale":"2023-12-20T04:47:22.063599+0000","last_undegraded":"2023-12-20T04:47:22.063599+0000","last_fullsized":"2023-12-20T04:47:22.063599+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:10:55.384589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.064177+0000","last_change":"2023-12-20T04:47:21.080796+0000","last_active":"2023-12-20T04:47:22.064177+0000","last_peered":"2023-12-20T04:47:22.064177+0000","last_clean":"2023-12-20T04:47:22.064177+0000","last_became_active":"2023-12-20T04:47:21.080543+0000","last_became_peered":"2023-12-20T04:47:21.080543+0000","last_unstale":"2023-12-20T04:47:22.064177+0000","last_undegraded":"2023-12-20T04:47:22.064177+0000","last_fullsized":"2023-12-20T04:47:22.064177+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:12.552047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"137'4","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067209+0000","last_change":"2023-12-20T04:47:20.672320+0000","last_active":"2023-12-20T04:47:22.067209+0000","last_peered":"2023-12-20T04:47:22.067209+0000","last_clean":"2023-12-20T04:47:22.067209+0000","last_became_active":"2023-12-20T04:47:20.672052+0000","last_became_peered":"2023-12-20T04:47:20.672052+0000","last_unstale":"2023-12-20T04:47:22.067209+0000","last_undegraded":"2023-12-20T04:47:22.067209+0000","last_fullsized":"2023-12-20T04:47:22.067209+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:25:19.317934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.516979+0000","last_change":"2023-12-20T04:47:21.086129+0000","last_active":"2023-12-20T04:47:22.516979+0000","last_peered":"2023-12-20T04:47:22.516979+0000","last_clean":"2023-12-20T04:47:22.516979+0000","last_became_active":"2023-12-20T04:47:21.085835+0000","last_became_peered":"2023-12-20T04:47:21.085835+0000","last_unstale":"2023-12-20T04:47:22.516979+0000","last_undegraded":"2023-12-20T04:47:22.516979+0000","last_fullsized":"2023-12-20T04:47:22.516979+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:47:50.903316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"137'1","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517248+0000","last_change":"2023-12-20T04:47:20.676411+0000","last_active":"2023-12-20T04:47:22.517248+0000","last_peered":"2023-12-20T04:47:22.517248+0000","last_clean":"2023-12-20T04:47:22.517248+0000","last_became_active":"2023-12-20T04:47:20.676027+0000","last_became_peered":"2023-12-20T04:47:20.676027+0000","last_unstale":"2023-12-20T04:47:22.517248+0000","last_undegraded":"2023-12-20T04:47:22.517248+0000","last_fullsized":"2023-12-20T04:47:22.517248+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:57:14.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.064298+0000","last_change":"2023-12-20T04:47:21.085293+0000","last_active":"2023-12-20T04:47:22.064298+0000","last_peered":"2023-12-20T04:47:22.064298+0000","last_clean":"2023-12-20T04:47:22.064298+0000","last_became_active":"2023-12-20T04:47:21.085037+0000","last_became_peered":"2023-12-20T04:47:21.085037+0000","last_unstale":"2023-12-20T04:47:22.064298+0000","last_undegraded":"2023-12-20T04:47:22.064298+0000","last_fullsized":"2023-12-20T04:47:22.064298+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:44:38.501754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515270+0000","last_change":"2023-12-20T04:47:20.673680+0000","last_active":"2023-12-20T04:47:22.515270+0000","last_peered":"2023-12-20T04:47:22.515270+0000","last_clean":"2023-12-20T04:47:22.515270+0000","last_became_active":"2023-12-20T04:47:20.673096+0000","last_became_peered":"2023-12-20T04:47:20.673096+0000","last_unstale":"2023-12-20T04:47:22.515270+0000","last_undegraded":"2023-12-20T04:47:22.515270+0000","last_fullsized":"2023-12-20T04:47:22.515270+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:03.093126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517241+0000","last_change":"2023-12-20T04:47:21.077215+0000","last_active":"2023-12-20T04:47:22.517241+0000","last_peered":"2023-12-20T04:47:22.517241+0000","last_clean":"2023-12-20T04:47:22.517241+0000","last_became_active":"2023-12-20T04:47:21.076833+0000","last_became_peered":"2023-12-20T04:47:21.076833+0000","last_unstale":"2023-12-20T04:47:22.517241+0000","last_undegraded":"2023-12-20T04:47:22.517241+0000","last_fullsized":"2023-12-20T04:47:22.517241+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:57.570829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515012+0000","last_change":"2023-12-20T04:47:20.673930+0000","last_active":"2023-12-20T04:47:22.515012+0000","last_peered":"2023-12-20T04:47:22.515012+0000","last_clean":"2023-12-20T04:47:22.515012+0000","last_became_active":"2023-12-20T04:47:20.673625+0000","last_became_peered":"2023-12-20T04:47:20.673625+0000","last_unstale":"2023-12-20T04:47:22.515012+0000","last_undegraded":"2023-12-20T04:47:22.515012+0000","last_fullsized":"2023-12-20T04:47:22.515012+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:10:27.154267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350470+0000","last_change":"2023-12-20T04:47:21.085541+0000","last_active":"2023-12-20T04:47:22.350470+0000","last_peered":"2023-12-20T04:47:22.350470+0000","last_clean":"2023-12-20T04:47:22.350470+0000","last_became_active":"2023-12-20T04:47:21.085274+0000","last_became_peered":"2023-12-20T04:47:21.085274+0000","last_unstale":"2023-12-20T04:47:22.350470+0000","last_undegraded":"2023-12-20T04:47:22.350470+0000","last_fullsized":"2023-12-20T04:47:22.350470+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:28.137951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"137'1","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067271+0000","last_change":"2023-12-20T04:47:20.665305+0000","last_active":"2023-12-20T04:47:22.067271+0000","last_peered":"2023-12-20T04:47:22.067271+0000","last_clean":"2023-12-20T04:47:22.067271+0000","last_became_active":"2023-12-20T04:47:20.664920+0000","last_became_peered":"2023-12-20T04:47:20.664920+0000","last_unstale":"2023-12-20T04:47:22.067271+0000","last_undegraded":"2023-12-20T04:47:22.067271+0000","last_fullsized":"2023-12-20T04:47:22.067271+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:54:01.405424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066427+0000","last_change":"2023-12-20T04:47:21.081180+0000","last_active":"2023-12-20T04:47:22.066427+0000","last_peered":"2023-12-20T04:47:22.066427+0000","last_clean":"2023-12-20T04:47:22.066427+0000","last_became_active":"2023-12-20T04:47:21.080689+0000","last_became_peered":"2023-12-20T04:47:21.080689+0000","last_unstale":"2023-12-20T04:47:22.066427+0000","last_undegraded":"2023-12-20T04:47:22.066427+0000","last_fullsized":"2023-12-20T04:47:22.066427+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:09.861807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"137'1","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515555+0000","last_change":"2023-12-20T04:47:20.669215+0000","last_active":"2023-12-20T04:47:22.515555+0000","last_peered":"2023-12-20T04:47:22.515555+0000","last_clean":"2023-12-20T04:47:22.515555+0000","last_became_active":"2023-12-20T04:47:20.668956+0000","last_became_peered":"2023-12-20T04:47:20.668956+0000","last_unstale":"2023-12-20T04:47:22.515555+0000","last_undegraded":"2023-12-20T04:47:22.515555+0000","last_fullsized":"2023-12-20T04:47:22.515555+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:43:16.354141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350395+0000","last_change":"2023-12-20T04:47:21.079411+0000","last_active":"2023-12-20T04:47:22.350395+0000","last_peered":"2023-12-20T04:47:22.350395+0000","last_clean":"2023-12-20T04:47:22.350395+0000","last_became_active":"2023-12-20T04:47:21.079103+0000","last_became_peered":"2023-12-20T04:47:21.079103+0000","last_unstale":"2023-12-20T04:47:22.350395+0000","last_undegraded":"2023-12-20T04:47:22.350395+0000","last_fullsized":"2023-12-20T04:47:22.350395+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:39:08.904834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066316+0000","last_change":"2023-12-20T04:47:20.667597+0000","last_active":"2023-12-20T04:47:22.066316+0000","last_peered":"2023-12-20T04:47:22.066316+0000","last_clean":"2023-12-20T04:47:22.066316+0000","last_became_active":"2023-12-20T04:47:20.667338+0000","last_became_peered":"2023-12-20T04:47:20.667338+0000","last_unstale":"2023-12-20T04:47:22.066316+0000","last_undegraded":"2023-12-20T04:47:22.066316+0000","last_fullsized":"2023-12-20T04:47:22.066316+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:30:10.422004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067030+0000","last_change":"2023-12-20T04:47:21.081796+0000","last_active":"2023-12-20T04:47:22.067030+0000","last_peered":"2023-12-20T04:47:22.067030+0000","last_clean":"2023-12-20T04:47:22.067030+0000","last_became_active":"2023-12-20T04:47:21.081521+0000","last_became_peered":"2023-12-20T04:47:21.081521+0000","last_unstale":"2023-12-20T04:47:22.067030+0000","last_undegraded":"2023-12-20T04:47:22.067030+0000","last_fullsized":"2023-12-20T04:47:22.067030+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:00:32.666079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350465+0000","last_change":"2023-12-20T04:47:20.670726+0000","last_active":"2023-12-20T04:47:22.350465+0000","last_peered":"2023-12-20T04:47:22.350465+0000","last_clean":"2023-12-20T04:47:22.350465+0000","last_became_active":"2023-12-20T04:47:20.670464+0000","last_became_peered":"2023-12-20T04:47:20.670464+0000","last_unstale":"2023-12-20T04:47:22.350465+0000","last_undegraded":"2023-12-20T04:47:22.350465+0000","last_fullsized":"2023-12-20T04:47:22.350465+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:05:27.098396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067385+0000","last_change":"2023-12-20T04:47:21.074982+0000","last_active":"2023-12-20T04:47:22.067385+0000","last_peered":"2023-12-20T04:47:22.067385+0000","last_clean":"2023-12-20T04:47:22.067385+0000","last_became_active":"2023-12-20T04:47:21.074655+0000","last_became_peered":"2023-12-20T04:47:21.074655+0000","last_unstale":"2023-12-20T04:47:22.067385+0000","last_undegraded":"2023-12-20T04:47:22.067385+0000","last_fullsized":"2023-12-20T04:47:22.067385+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:12.552047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515243+0000","last_change":"2023-12-20T04:47:20.673320+0000","last_active":"2023-12-20T04:47:22.515243+0000","last_peered":"2023-12-20T04:47:22.515243+0000","last_clean":"2023-12-20T04:47:22.515243+0000","last_became_active":"2023-12-20T04:47:20.673019+0000","last_became_peered":"2023-12-20T04:47:20.673019+0000","last_unstale":"2023-12-20T04:47:22.515243+0000","last_undegraded":"2023-12-20T04:47:22.515243+0000","last_fullsized":"2023-12-20T04:47:22.515243+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:25:19.317934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517020+0000","last_change":"2023-12-20T04:47:21.096934+0000","last_active":"2023-12-20T04:47:22.517020+0000","last_peered":"2023-12-20T04:47:22.517020+0000","last_clean":"2023-12-20T04:47:22.517020+0000","last_became_active":"2023-12-20T04:47:21.096663+0000","last_became_peered":"2023-12-20T04:47:21.096663+0000","last_unstale":"2023-12-20T04:47:22.517020+0000","last_undegraded":"2023-12-20T04:47:22.517020+0000","last_fullsized":"2023-12-20T04:47:22.517020+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:57:15.777946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515639+0000","last_change":"2023-12-20T04:47:20.671827+0000","last_active":"2023-12-20T04:47:22.515639+0000","last_peered":"2023-12-20T04:47:22.515639+0000","last_clean":"2023-12-20T04:47:22.515639+0000","last_became_active":"2023-12-20T04:47:20.671540+0000","last_became_peered":"2023-12-20T04:47:20.671540+0000","last_unstale":"2023-12-20T04:47:22.515639+0000","last_undegraded":"2023-12-20T04:47:22.515639+0000","last_fullsized":"2023-12-20T04:47:22.515639+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:56.568492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515747+0000","last_change":"2023-12-20T04:47:21.067337+0000","last_active":"2023-12-20T04:47:22.515747+0000","last_peered":"2023-12-20T04:47:22.515747+0000","last_clean":"2023-12-20T04:47:22.515747+0000","last_became_active":"2023-12-20T04:47:21.067077+0000","last_became_peered":"2023-12-20T04:47:21.067077+0000","last_unstale":"2023-12-20T04:47:22.515747+0000","last_undegraded":"2023-12-20T04:47:22.515747+0000","last_fullsized":"2023-12-20T04:47:22.515747+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:16.089644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"137'3","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350773+0000","last_change":"2023-12-20T04:47:20.680376+0000","last_active":"2023-12-20T04:47:22.350773+0000","last_peered":"2023-12-20T04:47:22.350773+0000","last_clean":"2023-12-20T04:47:22.350773+0000","last_became_active":"2023-12-20T04:47:20.679529+0000","last_became_peered":"2023-12-20T04:47:20.679529+0000","last_unstale":"2023-12-20T04:47:22.350773+0000","last_undegraded":"2023-12-20T04:47:22.350773+0000","last_fullsized":"2023-12-20T04:47:22.350773+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:56.796386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067530+0000","last_change":"2023-12-20T04:47:21.071747+0000","last_active":"2023-12-20T04:47:22.067530+0000","last_peered":"2023-12-20T04:47:22.067530+0000","last_clean":"2023-12-20T04:47:22.067530+0000","last_became_active":"2023-12-20T04:47:21.070530+0000","last_became_peered":"2023-12-20T04:47:21.070530+0000","last_unstale":"2023-12-20T04:47:22.067530+0000","last_undegraded":"2023-12-20T04:47:22.067530+0000","last_fullsized":"2023-12-20T04:47:22.067530+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:01:58.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"137'6","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350892+0000","last_change":"2023-12-20T04:47:20.671461+0000","last_active":"2023-12-20T04:47:22.350892+0000","last_peered":"2023-12-20T04:47:22.350892+0000","last_clean":"2023-12-20T04:47:22.350892+0000","last_became_active":"2023-12-20T04:47:20.670521+0000","last_became_peered":"2023-12-20T04:47:20.670521+0000","last_unstale":"2023-12-20T04:47:22.350892+0000","last_undegraded":"2023-12-20T04:47:22.350892+0000","last_fullsized":"2023-12-20T04:47:22.350892+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:06.195631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515510+0000","last_change":"2023-12-20T04:47:21.066579+0000","last_active":"2023-12-20T04:47:22.515510+0000","last_peered":"2023-12-20T04:47:22.515510+0000","last_clean":"2023-12-20T04:47:22.515510+0000","last_became_active":"2023-12-20T04:47:21.066321+0000","last_became_peered":"2023-12-20T04:47:21.066321+0000","last_unstale":"2023-12-20T04:47:22.515510+0000","last_undegraded":"2023-12-20T04:47:22.515510+0000","last_fullsized":"2023-12-20T04:47:22.515510+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:37:07.023828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063900+0000","last_change":"2023-12-20T04:47:20.671126+0000","last_active":"2023-12-20T04:47:22.063900+0000","last_peered":"2023-12-20T04:47:22.063900+0000","last_clean":"2023-12-20T04:47:22.063900+0000","last_became_active":"2023-12-20T04:47:20.670856+0000","last_became_peered":"2023-12-20T04:47:20.670856+0000","last_unstale":"2023-12-20T04:47:22.063900+0000","last_undegraded":"2023-12-20T04:47:22.063900+0000","last_fullsized":"2023-12-20T04:47:22.063900+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:59.615775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515534+0000","last_change":"2023-12-20T04:47:21.071478+0000","last_active":"2023-12-20T04:47:22.515534+0000","last_peered":"2023-12-20T04:47:22.515534+0000","last_clean":"2023-12-20T04:47:22.515534+0000","last_became_active":"2023-12-20T04:47:21.070759+0000","last_became_peered":"2023-12-20T04:47:21.070759+0000","last_unstale":"2023-12-20T04:47:22.515534+0000","last_undegraded":"2023-12-20T04:47:22.515534+0000","last_fullsized":"2023-12-20T04:47:22.515534+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:07.197968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517394+0000","last_change":"2023-12-20T04:47:20.681539+0000","last_active":"2023-12-20T04:47:22.517394+0000","last_peered":"2023-12-20T04:47:22.517394+0000","last_clean":"2023-12-20T04:47:22.517394+0000","last_became_active":"2023-12-20T04:47:20.681330+0000","last_became_peered":"2023-12-20T04:47:20.681330+0000","last_unstale":"2023-12-20T04:47:22.517394+0000","last_undegraded":"2023-12-20T04:47:22.517394+0000","last_fullsized":"2023-12-20T04:47:22.517394+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:44:37.499418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515598+0000","last_change":"2023-12-20T04:47:21.068586+0000","last_active":"2023-12-20T04:47:22.515598+0000","last_peered":"2023-12-20T04:47:22.515598+0000","last_clean":"2023-12-20T04:47:22.515598+0000","last_became_active":"2023-12-20T04:47:21.065948+0000","last_became_peered":"2023-12-20T04:47:21.065948+0000","last_unstale":"2023-12-20T04:47:22.515598+0000","last_undegraded":"2023-12-20T04:47:22.515598+0000","last_fullsized":"2023-12-20T04:47:22.515598+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:36:24.652104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"137'8","reported_seq":19,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.514841+0000","last_change":"2023-12-20T04:47:20.679901+0000","last_active":"2023-12-20T04:47:22.514841+0000","last_peered":"2023-12-20T04:47:22.514841+0000","last_clean":"2023-12-20T04:47:22.514841+0000","last_became_active":"2023-12-20T04:47:20.679640+0000","last_became_peered":"2023-12-20T04:47:20.679640+0000","last_unstale":"2023-12-20T04:47:22.514841+0000","last_undegraded":"2023-12-20T04:47:22.514841+0000","last_fullsized":"2023-12-20T04:47:22.514841+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:13:33.535191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515423+0000","last_change":"2023-12-20T04:47:21.080324+0000","last_active":"2023-12-20T04:47:22.515423+0000","last_peered":"2023-12-20T04:47:22.515423+0000","last_clean":"2023-12-20T04:47:22.515423+0000","last_became_active":"2023-12-20T04:47:21.080056+0000","last_became_peered":"2023-12-20T04:47:21.080056+0000","last_unstale":"2023-12-20T04:47:22.515423+0000","last_undegraded":"2023-12-20T04:47:22.515423+0000","last_fullsized":"2023-12-20T04:47:22.515423+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:37:19.231725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"137'1","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.351055+0000","last_change":"2023-12-20T04:47:20.674266+0000","last_active":"2023-12-20T04:47:22.351055+0000","last_peered":"2023-12-20T04:47:22.351055+0000","last_clean":"2023-12-20T04:47:22.351055+0000","last_became_active":"2023-12-20T04:47:20.673457+0000","last_became_peered":"2023-12-20T04:47:20.673457+0000","last_unstale":"2023-12-20T04:47:22.351055+0000","last_undegraded":"2023-12-20T04:47:22.351055+0000","last_fullsized":"2023-12-20T04:47:22.351055+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:18:32.358027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.064193+0000","last_change":"2023-12-20T04:47:21.090277+0000","last_active":"2023-12-20T04:47:22.064193+0000","last_peered":"2023-12-20T04:47:22.064193+0000","last_clean":"2023-12-20T04:47:22.064193+0000","last_became_active":"2023-12-20T04:47:21.089945+0000","last_became_peered":"2023-12-20T04:47:21.089945+0000","last_unstale":"2023-12-20T04:47:22.064193+0000","last_undegraded":"2023-12-20T04:47:22.064193+0000","last_fullsized":"2023-12-20T04:47:22.064193+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:10:25.971082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515506+0000","last_change":"2023-12-20T04:47:20.670643+0000","last_active":"2023-12-20T04:47:22.515506+0000","last_peered":"2023-12-20T04:47:22.515506+0000","last_clean":"2023-12-20T04:47:22.515506+0000","last_became_active":"2023-12-20T04:47:20.670364+0000","last_became_peered":"2023-12-20T04:47:20.670364+0000","last_unstale":"2023-12-20T04:47:22.515506+0000","last_undegraded":"2023-12-20T04:47:22.515506+0000","last_fullsized":"2023-12-20T04:47:22.515506+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:56:44.521740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517204+0000","last_change":"2023-12-20T04:47:21.082753+0000","last_active":"2023-12-20T04:47:22.517204+0000","last_peered":"2023-12-20T04:47:22.517204+0000","last_clean":"2023-12-20T04:47:22.517204+0000","last_became_active":"2023-12-20T04:47:21.082438+0000","last_became_peered":"2023-12-20T04:47:21.082438+0000","last_unstale":"2023-12-20T04:47:22.517204+0000","last_undegraded":"2023-12-20T04:47:22.517204+0000","last_fullsized":"2023-12-20T04:47:22.517204+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:33:53.116054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"137'1","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517231+0000","last_change":"2023-12-20T04:47:20.674831+0000","last_active":"2023-12-20T04:47:22.517231+0000","last_peered":"2023-12-20T04:47:22.517231+0000","last_clean":"2023-12-20T04:47:22.517231+0000","last_became_active":"2023-12-20T04:47:20.674527+0000","last_became_peered":"2023-12-20T04:47:20.674527+0000","last_unstale":"2023-12-20T04:47:22.517231+0000","last_undegraded":"2023-12-20T04:47:22.517231+0000","last_fullsized":"2023-12-20T04:47:22.517231+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:57:46.328208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517317+0000","last_change":"2023-12-20T04:47:21.075450+0000","last_active":"2023-12-20T04:47:22.517317+0000","last_peered":"2023-12-20T04:47:22.517317+0000","last_clean":"2023-12-20T04:47:22.517317+0000","last_became_active":"2023-12-20T04:47:21.074991+0000","last_became_peered":"2023-12-20T04:47:21.074991+0000","last_unstale":"2023-12-20T04:47:22.517317+0000","last_undegraded":"2023-12-20T04:47:22.517317+0000","last_fullsized":"2023-12-20T04:47:22.517317+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:43:17.356477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067443+0000","last_change":"2023-12-20T04:47:20.674932+0000","last_active":"2023-12-20T04:47:22.067443+0000","last_peered":"2023-12-20T04:47:22.067443+0000","last_clean":"2023-12-20T04:47:22.067443+0000","last_became_active":"2023-12-20T04:47:20.674611+0000","last_became_peered":"2023-12-20T04:47:20.674611+0000","last_unstale":"2023-12-20T04:47:22.067443+0000","last_undegraded":"2023-12-20T04:47:22.067443+0000","last_fullsized":"2023-12-20T04:47:22.067443+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:59.615775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.516890+0000","last_change":"2023-12-20T04:47:21.069399+0000","last_active":"2023-12-20T04:47:22.516890+0000","last_peered":"2023-12-20T04:47:22.516890+0000","last_clean":"2023-12-20T04:47:22.516890+0000","last_became_active":"2023-12-20T04:47:21.069097+0000","last_became_peered":"2023-12-20T04:47:21.069097+0000","last_unstale":"2023-12-20T04:47:22.516890+0000","last_undegraded":"2023-12-20T04:47:22.516890+0000","last_fullsized":"2023-12-20T04:47:22.516890+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:09.861807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063916+0000","last_change":"2023-12-20T04:47:20.675234+0000","last_active":"2023-12-20T04:47:22.063916+0000","last_peered":"2023-12-20T04:47:22.063916+0000","last_clean":"2023-12-20T04:47:22.063916+0000","last_became_active":"2023-12-20T04:47:20.674918+0000","last_became_peered":"2023-12-20T04:47:20.674918+0000","last_unstale":"2023-12-20T04:47:22.063916+0000","last_undegraded":"2023-12-20T04:47:22.063916+0000","last_fullsized":"2023-12-20T04:47:22.063916+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:25:19.317934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350528+0000","last_change":"2023-12-20T04:47:21.069723+0000","last_active":"2023-12-20T04:47:22.350528+0000","last_peered":"2023-12-20T04:47:22.350528+0000","last_clean":"2023-12-20T04:47:22.350528+0000","last_became_active":"2023-12-20T04:47:21.069463+0000","last_became_peered":"2023-12-20T04:47:21.069463+0000","last_unstale":"2023-12-20T04:47:22.350528+0000","last_undegraded":"2023-12-20T04:47:22.350528+0000","last_fullsized":"2023-12-20T04:47:22.350528+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:08.698549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.514846+0000","last_change":"2023-12-20T04:47:20.673163+0000","last_active":"2023-12-20T04:47:22.514846+0000","last_peered":"2023-12-20T04:47:22.514846+0000","last_clean":"2023-12-20T04:47:22.514846+0000","last_became_active":"2023-12-20T04:47:20.671975+0000","last_became_peered":"2023-12-20T04:47:20.671975+0000","last_unstale":"2023-12-20T04:47:22.514846+0000","last_undegraded":"2023-12-20T04:47:22.514846+0000","last_fullsized":"2023-12-20T04:47:22.514846+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:45:27.642645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517468+0000","last_change":"2023-12-20T04:47:21.077384+0000","last_active":"2023-12-20T04:47:22.517468+0000","last_peered":"2023-12-20T04:47:22.517468+0000","last_clean":"2023-12-20T04:47:22.517468+0000","last_became_active":"2023-12-20T04:47:21.077111+0000","last_became_peered":"2023-12-20T04:47:21.077111+0000","last_unstale":"2023-12-20T04:47:22.517468+0000","last_undegraded":"2023-12-20T04:47:22.517468+0000","last_fullsized":"2023-12-20T04:47:22.517468+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:42.621356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063612+0000","last_change":"2023-12-20T04:47:20.668523+0000","last_active":"2023-12-20T04:47:22.063612+0000","last_peered":"2023-12-20T04:47:22.063612+0000","last_clean":"2023-12-20T04:47:22.063612+0000","last_became_active":"2023-12-20T04:47:20.668229+0000","last_became_peered":"2023-12-20T04:47:20.668229+0000","last_unstale":"2023-12-20T04:47:22.063612+0000","last_undegraded":"2023-12-20T04:47:22.063612+0000","last_fullsized":"2023-12-20T04:47:22.063612+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:13:33.535191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515318+0000","last_change":"2023-12-20T04:47:21.072261+0000","last_active":"2023-12-20T04:47:22.515318+0000","last_peered":"2023-12-20T04:47:22.515318+0000","last_clean":"2023-12-20T04:47:22.515318+0000","last_became_active":"2023-12-20T04:47:21.069346+0000","last_became_peered":"2023-12-20T04:47:21.069346+0000","last_unstale":"2023-12-20T04:47:22.515318+0000","last_undegraded":"2023-12-20T04:47:22.515318+0000","last_fullsized":"2023-12-20T04:47:22.515318+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:53:13.541202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350489+0000","last_change":"2023-12-20T04:47:20.678668+0000","last_active":"2023-12-20T04:47:22.350489+0000","last_peered":"2023-12-20T04:47:22.350489+0000","last_clean":"2023-12-20T04:47:22.350489+0000","last_became_active":"2023-12-20T04:47:20.678366+0000","last_became_peered":"2023-12-20T04:47:20.678366+0000","last_unstale":"2023-12-20T04:47:22.350489+0000","last_undegraded":"2023-12-20T04:47:22.350489+0000","last_fullsized":"2023-12-20T04:47:22.350489+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:37:06.021492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.516992+0000","last_change":"2023-12-20T04:47:21.097056+0000","last_active":"2023-12-20T04:47:22.516992+0000","last_peered":"2023-12-20T04:47:22.516992+0000","last_clean":"2023-12-20T04:47:22.516992+0000","last_became_active":"2023-12-20T04:47:21.096783+0000","last_became_peered":"2023-12-20T04:47:21.096783+0000","last_unstale":"2023-12-20T04:47:22.516992+0000","last_undegraded":"2023-12-20T04:47:22.516992+0000","last_fullsized":"2023-12-20T04:47:22.516992+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:34:15.389835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515679+0000","last_change":"2023-12-20T04:47:20.676685+0000","last_active":"2023-12-20T04:47:22.515679+0000","last_peered":"2023-12-20T04:47:22.515679+0000","last_clean":"2023-12-20T04:47:22.515679+0000","last_became_active":"2023-12-20T04:47:20.676395+0000","last_became_peered":"2023-12-20T04:47:20.676395+0000","last_unstale":"2023-12-20T04:47:22.515679+0000","last_undegraded":"2023-12-20T04:47:22.515679+0000","last_fullsized":"2023-12-20T04:47:22.515679+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:14:03.537835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350980+0000","last_change":"2023-12-20T04:47:21.073764+0000","last_active":"2023-12-20T04:47:22.350980+0000","last_peered":"2023-12-20T04:47:22.350980+0000","last_clean":"2023-12-20T04:47:22.350980+0000","last_became_active":"2023-12-20T04:47:21.073455+0000","last_became_peered":"2023-12-20T04:47:21.073455+0000","last_unstale":"2023-12-20T04:47:22.350980+0000","last_undegraded":"2023-12-20T04:47:22.350980+0000","last_fullsized":"2023-12-20T04:47:22.350980+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:38.925591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517136+0000","last_change":"2023-12-20T04:47:20.672782+0000","last_active":"2023-12-20T04:47:22.517136+0000","last_peered":"2023-12-20T04:47:22.517136+0000","last_clean":"2023-12-20T04:47:22.517136+0000","last_became_active":"2023-12-20T04:47:20.672477+0000","last_became_peered":"2023-12-20T04:47:20.672477+0000","last_unstale":"2023-12-20T04:47:22.517136+0000","last_undegraded":"2023-12-20T04:47:22.517136+0000","last_fullsized":"2023-12-20T04:47:22.517136+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:32:58.995426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350857+0000","last_change":"2023-12-20T04:47:21.079477+0000","last_active":"2023-12-20T04:47:22.350857+0000","last_peered":"2023-12-20T04:47:22.350857+0000","last_clean":"2023-12-20T04:47:22.350857+0000","last_became_active":"2023-12-20T04:47:21.079200+0000","last_became_peered":"2023-12-20T04:47:21.079200+0000","last_unstale":"2023-12-20T04:47:22.350857+0000","last_undegraded":"2023-12-20T04:47:22.350857+0000","last_fullsized":"2023-12-20T04:47:22.350857+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:17:51.266380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350834+0000","last_change":"2023-12-20T04:47:20.678925+0000","last_active":"2023-12-20T04:47:22.350834+0000","last_peered":"2023-12-20T04:47:22.350834+0000","last_clean":"2023-12-20T04:47:22.350834+0000","last_became_active":"2023-12-20T04:47:20.678234+0000","last_became_peered":"2023-12-20T04:47:20.678234+0000","last_unstale":"2023-12-20T04:47:22.350834+0000","last_undegraded":"2023-12-20T04:47:22.350834+0000","last_fullsized":"2023-12-20T04:47:22.350834+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:50:29.834279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517128+0000","last_change":"2023-12-20T04:47:21.094972+0000","last_active":"2023-12-20T04:47:22.517128+0000","last_peered":"2023-12-20T04:47:22.517128+0000","last_clean":"2023-12-20T04:47:22.517128+0000","last_became_active":"2023-12-20T04:47:21.094696+0000","last_became_peered":"2023-12-20T04:47:21.094696+0000","last_unstale":"2023-12-20T04:47:22.517128+0000","last_undegraded":"2023-12-20T04:47:22.517128+0000","last_fullsized":"2023-12-20T04:47:22.517128+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:35:04.579810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515830+0000","last_change":"2023-12-20T04:47:20.672063+0000","last_active":"2023-12-20T04:47:22.515830+0000","last_peered":"2023-12-20T04:47:22.515830+0000","last_clean":"2023-12-20T04:47:22.515830+0000","last_became_active":"2023-12-20T04:47:20.671626+0000","last_became_peered":"2023-12-20T04:47:20.671626+0000","last_unstale":"2023-12-20T04:47:22.515830+0000","last_undegraded":"2023-12-20T04:47:22.515830+0000","last_fullsized":"2023-12-20T04:47:22.515830+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:28:24.189539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066170+0000","last_change":"2023-12-20T04:47:21.069595+0000","last_active":"2023-12-20T04:47:22.066170+0000","last_peered":"2023-12-20T04:47:22.066170+0000","last_clean":"2023-12-20T04:47:22.066170+0000","last_became_active":"2023-12-20T04:47:21.069307+0000","last_became_peered":"2023-12-20T04:47:21.069307+0000","last_unstale":"2023-12-20T04:47:22.066170+0000","last_undegraded":"2023-12-20T04:47:22.066170+0000","last_fullsized":"2023-12-20T04:47:22.066170+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:40:35.208765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067432+0000","last_change":"2023-12-20T04:47:20.669617+0000","last_active":"2023-12-20T04:47:22.067432+0000","last_peered":"2023-12-20T04:47:22.067432+0000","last_clean":"2023-12-20T04:47:22.067432+0000","last_became_active":"2023-12-20T04:47:20.669305+0000","last_became_peered":"2023-12-20T04:47:20.669305+0000","last_unstale":"2023-12-20T04:47:22.067432+0000","last_undegraded":"2023-12-20T04:47:22.067432+0000","last_fullsized":"2023-12-20T04:47:22.067432+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:45:27.642645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066949+0000","last_change":"2023-12-20T04:47:21.075733+0000","last_active":"2023-12-20T04:47:22.066949+0000","last_peered":"2023-12-20T04:47:22.066949+0000","last_clean":"2023-12-20T04:47:22.066949+0000","last_became_active":"2023-12-20T04:47:21.075451+0000","last_became_peered":"2023-12-20T04:47:21.075451+0000","last_unstale":"2023-12-20T04:47:22.066949+0000","last_undegraded":"2023-12-20T04:47:22.066949+0000","last_fullsized":"2023-12-20T04:47:22.066949+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:32.472254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066254+0000","last_change":"2023-12-20T04:47:20.669633+0000","last_active":"2023-12-20T04:47:22.066254+0000","last_peered":"2023-12-20T04:47:22.066254+0000","last_clean":"2023-12-20T04:47:22.066254+0000","last_became_active":"2023-12-20T04:47:20.669368+0000","last_became_peered":"2023-12-20T04:47:20.669368+0000","last_unstale":"2023-12-20T04:47:22.066254+0000","last_undegraded":"2023-12-20T04:47:22.066254+0000","last_fullsized":"2023-12-20T04:47:22.066254+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:10.755454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517496+0000","last_change":"2023-12-20T04:47:21.087552+0000","last_active":"2023-12-20T04:47:22.517496+0000","last_peered":"2023-12-20T04:47:22.517496+0000","last_clean":"2023-12-20T04:47:22.517496+0000","last_became_active":"2023-12-20T04:47:21.087267+0000","last_became_peered":"2023-12-20T04:47:21.087267+0000","last_unstale":"2023-12-20T04:47:22.517496+0000","last_undegraded":"2023-12-20T04:47:22.517496+0000","last_fullsized":"2023-12-20T04:47:22.517496+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:11:44.960894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.514887+0000","last_change":"2023-12-20T04:47:20.672325+0000","last_active":"2023-12-20T04:47:22.514887+0000","last_peered":"2023-12-20T04:47:22.514887+0000","last_clean":"2023-12-20T04:47:22.514887+0000","last_became_active":"2023-12-20T04:47:20.671957+0000","last_became_peered":"2023-12-20T04:47:20.671957+0000","last_unstale":"2023-12-20T04:47:22.514887+0000","last_undegraded":"2023-12-20T04:47:22.514887+0000","last_fullsized":"2023-12-20T04:47:22.514887+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:54:01.405424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350555+0000","last_change":"2023-12-20T04:47:21.086535+0000","last_active":"2023-12-20T04:47:22.350555+0000","last_peered":"2023-12-20T04:47:22.350555+0000","last_clean":"2023-12-20T04:47:22.350555+0000","last_became_active":"2023-12-20T04:47:21.086252+0000","last_became_peered":"2023-12-20T04:47:21.086252+0000","last_unstale":"2023-12-20T04:47:22.350555+0000","last_undegraded":"2023-12-20T04:47:22.350555+0000","last_fullsized":"2023-12-20T04:47:22.350555+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:44.596329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350577+0000","last_change":"2023-12-20T04:47:20.678744+0000","last_active":"2023-12-20T04:47:22.350577+0000","last_peered":"2023-12-20T04:47:22.350577+0000","last_clean":"2023-12-20T04:47:22.350577+0000","last_became_active":"2023-12-20T04:47:20.678492+0000","last_became_peered":"2023-12-20T04:47:20.678492+0000","last_unstale":"2023-12-20T04:47:22.350577+0000","last_undegraded":"2023-12-20T04:47:22.350577+0000","last_fullsized":"2023-12-20T04:47:22.350577+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:15.087308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515243+0000","last_change":"2023-12-20T04:47:21.073845+0000","last_active":"2023-12-20T04:47:22.515243+0000","last_peered":"2023-12-20T04:47:22.515243+0000","last_clean":"2023-12-20T04:47:22.515243+0000","last_became_active":"2023-12-20T04:47:21.073426+0000","last_became_peered":"2023-12-20T04:47:21.073426+0000","last_unstale":"2023-12-20T04:47:22.515243+0000","last_undegraded":"2023-12-20T04:47:22.515243+0000","last_fullsized":"2023-12-20T04:47:22.515243+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:50:30.836615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067269+0000","last_change":"2023-12-20T04:47:20.673484+0000","last_active":"2023-12-20T04:47:22.067269+0000","last_peered":"2023-12-20T04:47:22.067269+0000","last_clean":"2023-12-20T04:47:22.067269+0000","last_became_active":"2023-12-20T04:47:20.673216+0000","last_became_peered":"2023-12-20T04:47:20.673216+0000","last_unstale":"2023-12-20T04:47:22.067269+0000","last_undegraded":"2023-12-20T04:47:22.067269+0000","last_fullsized":"2023-12-20T04:47:22.067269+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:13:33.535191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"126'3102","reported_seq":3541,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-20T04:47:21.066883+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:47:21.066883+0000","last_peered":"2023-12-20T04:47:21.066883+0000","last_clean":"2023-12-20T04:47:21.066883+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:47:21.066883+0000","last_undegraded":"2023-12-20T04:47:21.066883+0000","last_fullsized":"2023-12-20T04:47:21.066883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3102,"log_dups_size":0,"ondisk_log_size":3102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1021,"num_read_kb":1995,"num_write":341,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1021,"num_read_kb":1995,"num_write":341,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3102,"ondisk_log_size":3102,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575073,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2260,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2314240,"data_stored":1406363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.70299999999999996},"min":{"1min":0.42099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.0600000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.69099999999999995,"15min":0.71099999999999997},"min":{"1min":0.47599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.59799999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70399999999999996,"15min":0.72699999999999998},"min":{"1min":0.59499999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.70699999999999996,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.046,"5min":1.046,"15min":1.1080000000000001},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.68700000000000006,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.46899999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.88400000000000001}]},{"osd":3,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.70299999999999996,"15min":0.72399999999999998},"min":{"1min":0.495,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.74299999999999999,"15min":0.75900000000000001},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.47599999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.76900000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.45900000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":1.0109999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.45200000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.0469999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66300000000000003,"15min":0.65200000000000002},"min":{"1min":0.51700000000000002,"5min":0.40500000000000003,"15min":0.40400000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.44,"5min":0.436,"15min":0.432},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.071},"last":0.89600000000000002}]},{"osd":6,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.61499999999999999,"5min":0.46800000000000003,"15min":0.443},"max":{"1min":1.026,"5min":1.026,"15min":1.1279999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.70899999999999996,"15min":0.70599999999999996},"min":{"1min":0.55700000000000005,"5min":0.46100000000000002,"15min":0.44400000000000001},"max":{"1min":0.97899999999999998,"5min":1.0309999999999999,"15min":1.125},"last":0.86699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575073,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36808,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335032,"statfs":{"total":96636764160,"available":96599072768,"internally_reserved":0,"allocated":1703936,"data_stored":809121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69599999999999995,"15min":0.71499999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.27100000000000002},"max":{"1min":0.996,"5min":1.1140000000000001,"15min":1.2649999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69999999999999996,"15min":0.71599999999999997},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92000000000000004,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67700000000000005,"15min":0.69999999999999996},"min":{"1min":0.42599999999999999,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.93300000000000005,"5min":1.016,"15min":1.016},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.54400000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95999999999999996,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.72699999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.747,"15min":0.76100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.40899999999999997},"max":{"1min":0.90700000000000003,"5min":1.0580000000000001,"15min":1.216},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70899999999999996,"15min":0.73299999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93100000000000005,"5min":1.141,"15min":1.141},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.53400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96899999999999997,"5min":1.093,"15min":1.093},"last":0.746},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93100000000000005,"5min":1.087,"15min":1.087},"last":0.63100000000000001}]},{"osd":4,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.379,"15min":0.36399999999999999},"max":{"1min":0.81999999999999995,"5min":0.97099999999999997,"15min":1.1040000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.66300000000000003,"15min":0.624},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.36899999999999999},"max":{"1min":0.82499999999999996,"5min":0.98499999999999999,"15min":0.995},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.41399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.874,"5min":0.93799999999999994,"15min":1.1519999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70099999999999996,"15min":0.67700000000000005},"min":{"1min":0.46300000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92300000000000004,"5min":1.0169999999999999,"15min":1.306},"last":0.58299999999999996}]},{"osd":7,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.71799999999999997,"15min":0.70599999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":1.0129999999999999,"15min":1.0149999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71899999999999997,"15min":0.69799999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84599999999999997,"5min":1.131,"15min":1.131},"last":0.85499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575074,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1720320,"data_stored":815983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.42199999999999999,"5min":0.36199999999999999,"15min":0.29999999999999999},"max":{"1min":0.996,"5min":0.996,"15min":1.05},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.34300000000000003},"max":{"1min":0.83799999999999997,"5min":0.96899999999999997,"15min":0.98599999999999999},"last":0.72699999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.443,"15min":0.441},"max":{"1min":0.91300000000000003,"5min":0.98399999999999999,"15min":1.0600000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.72999999999999998},"min":{"1min":0.54300000000000004,"5min":0.437,"15min":0.38400000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0309999999999999},"last":0.66900000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.71399999999999997}]},{"osd":3,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.54700000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.73799999999999999,"15min":0.755},"min":{"1min":0.47499999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92400000000000004,"5min":1.008,"15min":1.0469999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.44800000000000001,"5min":0.42099999999999999,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.63,"15min":0.629},"min":{"1min":0.43099999999999999,"5min":0.40300000000000002,"15min":0.39600000000000002},"max":{"1min":0.81000000000000005,"5min":0.95099999999999996,"15min":1.077},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90600000000000003,"5min":0.96599999999999997,"15min":1.056},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63,"15min":0.63},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.999},"last":0.76500000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.46899999999999997,"5min":0.45300000000000001,"15min":0.442},"max":{"1min":0.88400000000000001,"5min":1.006,"15min":1.141},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.55000000000000004,"5min":0.48999999999999999,"15min":0.47899999999999998},"max":{"1min":0.92500000000000004,"5min":0.95799999999999996,"15min":1.0780000000000001},"last":0.73999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575075,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58960,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312880,"statfs":{"total":96636764160,"available":96576389120,"internally_reserved":0,"allocated":1712128,"data_stored":815379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.38300000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.93400000000000005,"5min":1.0129999999999999,"15min":1.5509999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67500000000000004,"15min":0.70299999999999996},"min":{"1min":0.36699999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92100000000000004,"5min":0.97799999999999998,"15min":1.633},"last":0.64200000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69199999999999995,"15min":0.71299999999999997},"min":{"1min":0.42599999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.91300000000000003,"5min":0.96099999999999997,"15min":1.798},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70999999999999996,"15min":0.73799999999999999},"min":{"1min":0.42499999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.89100000000000001,"5min":1.036,"15min":2.2930000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72299999999999998,"15min":0.73999999999999999},"min":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.93999999999999995,"5min":1.0209999999999999,"15min":1.728},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71699999999999997,"15min":0.74199999999999999},"min":{"1min":0.503,"5min":0.46899999999999997,"15min":0.38700000000000001},"max":{"1min":0.96299999999999997,"5min":1.0289999999999999,"15min":1.6619999999999999},"last":0.75900000000000001}]},{"osd":3,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.72199999999999998},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85199999999999998,"5min":1.004,"15min":1.5760000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.997,"5min":0.997,"15min":1.7649999999999999},"last":0.74299999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.39900000000000002,"5min":0.36599999999999999,"15min":0.36099999999999999},"max":{"1min":0.75700000000000001,"5min":1.1120000000000001,"15min":1.456},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39700000000000002,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":1.077,"15min":1.589},"last":0.44400000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.35999999999999999},"max":{"1min":0.71699999999999997,"5min":1.006,"15min":1.6739999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.41999999999999998,"15min":0.41699999999999998},"max":{"1min":0.81499999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67700000000000005,"15min":0.68400000000000005},"min":{"1min":0.47199999999999998,"5min":0.46500000000000002,"15min":0.38400000000000001},"max":{"1min":0.86399999999999999,"5min":1.036,"15min":1.655},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":0.88,"5min":1.052,"15min":1.125},"last":0.68600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575074,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36832,"kb_used_data":1688,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335008,"statfs":{"total":96636764160,"available":96599048192,"internally_reserved":0,"allocated":1728512,"data_stored":817013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.45100000000000001},"min":{"1min":0.32300000000000001,"5min":0.26100000000000001,"15min":0.23200000000000001},"max":{"1min":0.61099999999999999,"5min":0.748,"15min":0.875},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.48299999999999998,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77300000000000002,"5min":0.77600000000000002,"15min":0.80800000000000005},"last":0.41299999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.29299999999999998,"15min":0.27400000000000002},"max":{"1min":0.56799999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.435},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.54800000000000004},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.28399999999999997},"max":{"1min":0.65900000000000003,"5min":0.74399999999999999,"15min":0.88900000000000001},"last":0.375}]},{"osd":2,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56100000000000005,"15min":0.57599999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.318},"max":{"1min":0.74199999999999999,"5min":0.80700000000000005,"15min":0.88700000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.316,"15min":0.316},"max":{"1min":0.77000000000000002,"5min":0.81799999999999995,"15min":0.93300000000000005},"last":0.51800000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73099999999999998,"15min":0.77400000000000002},"min":{"1min":0.59399999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.0880000000000001,"15min":1.131},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75600000000000001,"15min":0.79200000000000004},"min":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.41599999999999998},"max":{"1min":0.998,"5min":1.198,"15min":1.4059999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.80000000000000004},"min":{"1min":0.60599999999999998,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":1.002,"5min":1.181,"15min":1.3300000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75,"15min":0.77700000000000002},"min":{"1min":0.56899999999999995,"5min":0.503,"15min":0.38400000000000001},"max":{"1min":0.97799999999999998,"5min":1.1060000000000001,"15min":1.252},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.746,"15min":0.78600000000000003},"min":{"1min":0.61799999999999999,"5min":0.49099999999999999,"15min":0.45600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.2050000000000001},"last":0.999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76800000000000002,"15min":0.80400000000000005},"min":{"1min":0.56699999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.052,"5min":1.133,"15min":1.2929999999999999},"last":0.82499999999999996}]},{"osd":7,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.751,"15min":0.78600000000000003},"min":{"1min":0.65000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96399999999999997,"5min":1.0249999999999999,"15min":1.1699999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76900000000000002,"15min":0.80800000000000005},"min":{"1min":0.622,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95599999999999996,"5min":1.145,"15min":1.29},"last":0.80200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575073,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58960,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312880,"statfs":{"total":96636764160,"available":96576389120,"internally_reserved":0,"allocated":1712128,"data_stored":815379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.432,"15min":0.44},"min":{"1min":0.307,"5min":0.26200000000000001,"15min":0.216},"max":{"1min":0.64400000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.48599999999999999},"min":{"1min":0.32500000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60399999999999998,"5min":0.92200000000000004,"15min":1.0589999999999999},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45600000000000002,"15min":0.46999999999999997},"min":{"1min":0.33100000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64400000000000002,"5min":0.78100000000000003,"15min":0.876},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.505,"15min":0.52200000000000002},"min":{"1min":0.36199999999999999,"5min":0.32100000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.44}]},{"osd":3,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57499999999999996,"15min":0.59799999999999998},"min":{"1min":0.49199999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76800000000000002,"5min":0.89800000000000002,"15min":0.93899999999999995},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.625,"15min":0.63800000000000001},"min":{"1min":0.47299999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95599999999999996},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74299999999999999,"15min":0.77200000000000002},"min":{"1min":0.56000000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.046,"5min":1.0760000000000001,"15min":1.145},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91600000000000004,"5min":1.361,"15min":1.361},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.76200000000000001},"min":{"1min":0.53100000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94499999999999995,"5min":1.242,"15min":1.242},"last":0.75},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.755,"15min":0.79000000000000004},"min":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48499999999999999},"max":{"1min":1.0980000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.878}]},{"osd":6,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75900000000000001,"15min":0.78400000000000003},"min":{"1min":0.502,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97299999999999998,"5min":1.323,"15min":1.323},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.74299999999999999,"15min":0.78100000000000003},"min":{"1min":0.58999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90200000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.84299999999999997}]},{"osd":7,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77100000000000002,"15min":0.80300000000000005},"min":{"1min":0.54800000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0669999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.79700000000000004,"15min":0.83699999999999997},"min":{"1min":0.64500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.079,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.871}]}]},{"osd":1,"up_from":13,"seq":55834575075,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50776,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321064,"statfs":{"total":96636764160,"available":96584769536,"internally_reserved":0,"allocated":1720320,"data_stored":815503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.43099999999999999,"15min":0.439},"min":{"1min":0.30099999999999999,"5min":0.23400000000000001,"15min":0.22600000000000001},"max":{"1min":0.67300000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48999999999999999,"15min":0.49299999999999999},"min":{"1min":0.34899999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.621,"5min":0.86199999999999999,"15min":0.92100000000000004},"last":0.83299999999999996}]},{"osd":2,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.51100000000000001},"min":{"1min":0.39400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69599999999999995,"5min":0.72799999999999998,"15min":1.091},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77600000000000002,"5min":0.84699999999999998,"15min":0.875},"last":0.66900000000000004}]},{"osd":3,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.74299999999999999,"5min":0.91000000000000003,"15min":0.96499999999999997},"last":0.62},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51400000000000001,"15min":0.52700000000000002},"min":{"1min":0.36599999999999999,"5min":0.33300000000000002,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.82399999999999995,"15min":1.2410000000000001},"last":0.59299999999999997}]},{"osd":4,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.75600000000000001},"min":{"1min":0.53400000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89400000000000002,"5min":1.036,"15min":1.27},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.77900000000000003},"min":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":0.93000000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.748,"15min":0.77200000000000002},"min":{"1min":0.54600000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.3420000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.53700000000000003,"5min":0.48399999999999999,"15min":0.41299999999999998},"max":{"1min":1.0449999999999999,"5min":1.129,"15min":1.2450000000000001},"last":0.94999999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.52300000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.044,"5min":1.232,"15min":1.232},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.74199999999999999,"15min":0.77900000000000003},"min":{"1min":0.621,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90900000000000003,"5min":1.0289999999999999,"15min":1.252},"last":0.87}]},{"osd":7,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.80000000000000004,"15min":0.82599999999999996},"min":{"1min":0.58299999999999996,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.306},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.79600000000000004,"15min":0.83099999999999996},"min":{"1min":0.61199999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0700000000000001,"5min":1.071,"15min":1.3440000000000001},"last":1.006}]}]},{"osd":0,"up_from":13,"seq":55834575074,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29224,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342616,"statfs":{"total":96636764160,"available":96606838784,"internally_reserved":0,"allocated":2326528,"data_stored":1407423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.28299999999999997,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":0.68300000000000005,"5min":0.83699999999999997,"15min":2.5640000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.53400000000000003},"min":{"1min":0.31900000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":7.8860000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.503,"15min":0.51600000000000001},"min":{"1min":0.41499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71499999999999997,"5min":0.76300000000000001,"15min":2.528},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54500000000000004,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.441}]},{"osd":3,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.53500000000000003},"min":{"1min":0.432,"5min":0.309,"15min":0.309},"max":{"1min":1.127,"5min":3.0350000000000001,"15min":3.1000000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58699999999999997,"15min":0.623},"min":{"1min":0.46300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79300000000000004,"5min":0.80300000000000005,"15min":10.49},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75600000000000001,"15min":0.78400000000000003},"min":{"1min":0.505,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0109999999999999,"5min":1.321,"15min":3.0659999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78000000000000003,"15min":0.80600000000000005},"min":{"1min":0.54400000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.165,"5min":1.3080000000000001,"15min":5.2460000000000004},"last":0.95399999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.77200000000000002,"15min":0.79300000000000004},"min":{"1min":0.58799999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.3109999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78500000000000003,"15min":0.79500000000000004},"min":{"1min":0.53400000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2130000000000001,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.98699999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78600000000000003,"15min":0.81699999999999995},"min":{"1min":0.51900000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0569999999999999,"5min":1.345,"15min":5.2839999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79200000000000004,"15min":0.80600000000000005},"min":{"1min":0.55800000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.111,"5min":1.2529999999999999,"15min":1.306},"last":0.78800000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.81399999999999995,"15min":0.82299999999999995},"min":{"1min":0.60499999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.752},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.878}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:47:25.178 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T04:47:25.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.358+0000 7f430cbb3640 1 Processor -- start 2023-12-20T04:47:25.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.358+0000 7f430cbb3640 1 -- start start 2023-12-20T04:47:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f430cbb3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f43080af5e0 0x7f43080af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f430cbb3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f430806ce60 0x7f430815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f430cbb3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f430815f110 0x7f430815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f430cbb3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430815bbd0 con 0x7f430806ce60 2023-12-20T04:47:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f430cbb3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f430815beb0 con 0x7f43080af5e0 2023-12-20T04:47:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f430cbb3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f430815e910 con 0x7f430815f110 2023-12-20T04:47:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f42fffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f430806ce60 0x7f430815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f42fffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f430806ce60 0x7f430815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35076/0 (socket says 172.21.15.32:35076) 2023-12-20T04:47:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f42fffff640 1 -- 172.21.15.32:0/2532792926 learned_addr learned my addr 172.21.15.32:0/2532792926 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:25.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f430659c640 1 --2- 172.21.15.32:0/2532792926 >> v2:172.21.15.39:3300/0 conn(0x7f43080af5e0 0x7f43080af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f4306d9d640 1 --2- 172.21.15.32:0/2532792926 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f430815f110 0x7f430815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f42fffff640 1 -- 172.21.15.32:0/2532792926 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f430815f110 msgr2=0x7f430815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f42fffff640 1 --2- 172.21.15.32:0/2532792926 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f430815f110 0x7f430815c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f42fffff640 1 -- 172.21.15.32:0/2532792926 >> v2:172.21.15.39:3300/0 conn(0x7f43080af5e0 msgr2=0x7f43080af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f42fffff640 1 --2- 172.21.15.32:0/2532792926 >> v2:172.21.15.39:3300/0 conn(0x7f43080af5e0 0x7f43080af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f42fffff640 1 -- 172.21.15.32:0/2532792926 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f430815ebf0 con 0x7f430806ce60 2023-12-20T04:47:25.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.362+0000 7f4306d9d640 1 --2- 172.21.15.32:0/2532792926 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f430815f110 0x7f430815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:25.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f42fffff640 1 --2- 172.21.15.32:0/2532792926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f430806ce60 0x7f430815b280 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f430806ac10 tx=0x7f42f001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa46e7eba130a114 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:25.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f4305d9b640 1 -- 172.21.15.32:0/2532792926 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f00ab620 con 0x7f430806ce60 2023-12-20T04:47:25.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f4305d9b640 1 -- 172.21.15.32:0/2532792926 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f00ab780 con 0x7f430806ce60 2023-12-20T04:47:25.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f4305d9b640 1 -- 172.21.15.32:0/2532792926 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f00abb10 con 0x7f430806ce60 2023-12-20T04:47:25.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f430cbb3640 1 -- 172.21.15.32:0/2532792926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f430806ce60 msgr2=0x7f430815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f430cbb3640 1 --2- 172.21.15.32:0/2532792926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f430806ce60 0x7f430815b280 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7f430806ac10 tx=0x7f42f001c500 comp rx=0 tx=0).stop 2023-12-20T04:47:25.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f430cbb3640 1 -- 172.21.15.32:0/2532792926 shutdown_connections 2023-12-20T04:47:25.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f430cbb3640 1 --2- 172.21.15.32:0/2532792926 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f430815f110 0x7f430815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f430cbb3640 1 --2- 172.21.15.32:0/2532792926 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f430806ce60 0x7f430815b280 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f430cbb3640 1 --2- 172.21.15.32:0/2532792926 >> v2:172.21.15.39:3300/0 conn(0x7f43080af5e0 0x7f43080af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f430cbb3640 1 -- 172.21.15.32:0/2532792926 >> 172.21.15.32:0/2532792926 conn(0x7f430806a320 msgr2=0x7f43080ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:25.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f430cbb3640 1 -- 172.21.15.32:0/2532792926 shutdown_connections 2023-12-20T04:47:25.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f430cbb3640 1 -- 172.21.15.32:0/2532792926 wait complete. 2023-12-20T04:47:25.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f430cbb3640 1 Processor -- start 2023-12-20T04:47:25.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.366+0000 7f430cbb3640 1 -- start start 2023-12-20T04:47:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430cbb3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f430806ce60 0x7f430814ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430cbb3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43080af5e0 0x7f4308153080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430cbb3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f430815f110 0x7f43080b36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430cbb3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43080b4260 con 0x7f430806ce60 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430cbb3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43081565c0 con 0x7f43080af5e0 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430cbb3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43081568a0 con 0x7f430815f110 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f430806ce60 0x7f430814ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f430806ce60 0x7f430814ea10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35092/0 (socket says 172.21.15.32:35092) 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430659c640 1 -- 172.21.15.32:0/280453177 learned_addr learned my addr 172.21.15.32:0/280453177 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f42fffff640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43080af5e0 0x7f4308153080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f4306d9d640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f430815f110 0x7f43080b36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430659c640 1 -- 172.21.15.32:0/280453177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f430815f110 msgr2=0x7f43080b36c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430659c640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f430815f110 0x7f43080b36c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430659c640 1 -- 172.21.15.32:0/280453177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43080af5e0 msgr2=0x7f4308153080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430659c640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43080af5e0 0x7f4308153080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430659c640 1 -- 172.21.15.32:0/280453177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42f00094d0 con 0x7f430806ce60 2023-12-20T04:47:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430659c640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f430806ce60 0x7f430814ea10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f42f800d800 tx=0x7f42f800dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:25.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f42feffd640 1 -- 172.21.15.32:0/280453177 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f80096a0 con 0x7f430806ce60 2023-12-20T04:47:25.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f42feffd640 1 -- 172.21.15.32:0/280453177 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42f8009800 con 0x7f430806ce60 2023-12-20T04:47:25.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430cbb3640 1 -- 172.21.15.32:0/280453177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4308156b80 con 0x7f430806ce60 2023-12-20T04:47:25.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f42feffd640 1 -- 172.21.15.32:0/280453177 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42f8009b70 con 0x7f430806ce60 2023-12-20T04:47:25.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430cbb3640 1 -- 172.21.15.32:0/280453177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f430814ae10 con 0x7f430806ce60 2023-12-20T04:47:25.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f430cbb3640 1 -- 172.21.15.32:0/280453177 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f430804e000 con 0x7f430806ce60 2023-12-20T04:47:25.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.370+0000 7f42feffd640 1 -- 172.21.15.32:0/280453177 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f42f8009cd0 con 0x7f430806ce60 2023-12-20T04:47:25.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.374+0000 7f42feffd640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42e8065b80 0x7f42e8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.374+0000 7f42feffd640 1 -- 172.21.15.32:0/280453177 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f42f8083aa0 con 0x7f430806ce60 2023-12-20T04:47:25.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.374+0000 7f42fffff640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42e8065b80 0x7f42e8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.374+0000 7f42fffff640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42e8065b80 0x7f42e8068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f42f00af000 tx=0x7f42f00a7000 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:25.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.378+0000 7f42feffd640 1 -- 172.21.15.32:0/280453177 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f42f80513c0 con 0x7f430806ce60 2023-12-20T04:47:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.542+0000 7f430cbb3640 1 -- 172.21.15.32:0/280453177 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f430815b7c0 con 0x7f42e8065b80 2023-12-20T04:47:25.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.550+0000 7f42feffd640 1 -- 172.21.15.32:0/280453177 <== mgr.8172 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289314 (secure 0 0 0) 0x7f42f00ab8f0 con 0x7f42e8065b80 2023-12-20T04:47:25.551 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:25.553 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T04:47:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 -- 172.21.15.32:0/280453177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42e8065b80 msgr2=0x7f42e8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42e8065b80 0x7f42e8068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f42f00af000 tx=0x7f42f00a7000 comp rx=0 tx=0).stop 2023-12-20T04:47:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 -- 172.21.15.32:0/280453177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f430806ce60 msgr2=0x7f430814ea10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f430806ce60 0x7f430814ea10 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7f42f800d800 tx=0x7f42f800dcc0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 -- 172.21.15.32:0/280453177 shutdown_connections 2023-12-20T04:47:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42e8065b80 0x7f42e8068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f430815f110 0x7f43080b36c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43080af5e0 0x7f4308153080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 --2- 172.21.15.32:0/280453177 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f430806ce60 0x7f430814ea10 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 -- 172.21.15.32:0/280453177 >> 172.21.15.32:0/280453177 conn(0x7f430806a320 msgr2=0x7f4308155510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:25.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 -- 172.21.15.32:0/280453177 shutdown_connections 2023-12-20T04:47:25.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.554+0000 7f430cbb3640 1 -- 172.21.15.32:0/280453177 wait complete. 2023-12-20T04:47:25.572 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-20T04:47:25.284848+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1021,"num_read_kb":1995,"num_write":385,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3140,"ondisk_log_size":3140,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":130,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":341692,"kb_used_data":14588,"kb_used_omap":0,"kb_used_meta":327040,"kb_avail":754633028,"statfs":{"total":773094113280,"available":772744220672,"internally_reserved":0,"allocated":14938112,"data_stored":7702164,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":334888960},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":30,"apply_latency_ms":30,"commit_latency_ns":30000000,"apply_latency_ns":30000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.634711"},"pg_stats":[{"pgid":"17.3f","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350904+0000","last_change":"2023-12-20T04:47:21.091423+0000","last_active":"2023-12-20T04:47:22.350904+0000","last_peered":"2023-12-20T04:47:22.350904+0000","last_clean":"2023-12-20T04:47:22.350904+0000","last_became_active":"2023-12-20T04:47:21.091106+0000","last_became_peered":"2023-12-20T04:47:21.091106+0000","last_unstale":"2023-12-20T04:47:22.350904+0000","last_undegraded":"2023-12-20T04:47:22.350904+0000","last_fullsized":"2023-12-20T04:47:22.350904+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:46:22.464171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3e","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066783+0000","last_change":"2023-12-20T04:47:21.075045+0000","last_active":"2023-12-20T04:47:22.066783+0000","last_peered":"2023-12-20T04:47:22.066783+0000","last_clean":"2023-12-20T04:47:22.066783+0000","last_became_active":"2023-12-20T04:47:21.074774+0000","last_became_peered":"2023-12-20T04:47:21.074774+0000","last_unstale":"2023-12-20T04:47:22.066783+0000","last_undegraded":"2023-12-20T04:47:22.066783+0000","last_fullsized":"2023-12-20T04:47:22.066783+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:24.251906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.3d","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517125+0000","last_change":"2023-12-20T04:47:21.069629+0000","last_active":"2023-12-20T04:47:22.517125+0000","last_peered":"2023-12-20T04:47:22.517125+0000","last_clean":"2023-12-20T04:47:22.517125+0000","last_became_active":"2023-12-20T04:47:21.069171+0000","last_became_peered":"2023-12-20T04:47:21.069171+0000","last_unstale":"2023-12-20T04:47:22.517125+0000","last_undegraded":"2023-12-20T04:47:22.517125+0000","last_fullsized":"2023-12-20T04:47:22.517125+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:30:24.251906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.3c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350602+0000","last_change":"2023-12-20T04:47:21.075343+0000","last_active":"2023-12-20T04:47:22.350602+0000","last_peered":"2023-12-20T04:47:22.350602+0000","last_clean":"2023-12-20T04:47:22.350602+0000","last_became_active":"2023-12-20T04:47:21.075077+0000","last_became_peered":"2023-12-20T04:47:21.075077+0000","last_unstale":"2023-12-20T04:47:22.350602+0000","last_undegraded":"2023-12-20T04:47:22.350602+0000","last_fullsized":"2023-12-20T04:47:22.350602+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:23:26.602240+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.3b","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.514978+0000","last_change":"2023-12-20T04:47:21.076280+0000","last_active":"2023-12-20T04:47:22.514978+0000","last_peered":"2023-12-20T04:47:22.514978+0000","last_clean":"2023-12-20T04:47:22.514978+0000","last_became_active":"2023-12-20T04:47:21.076035+0000","last_became_peered":"2023-12-20T04:47:21.076035+0000","last_unstale":"2023-12-20T04:47:22.514978+0000","last_undegraded":"2023-12-20T04:47:22.514978+0000","last_fullsized":"2023-12-20T04:47:22.514978+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:02.256326+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.3a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063682+0000","last_change":"2023-12-20T04:47:21.079746+0000","last_active":"2023-12-20T04:47:22.063682+0000","last_peered":"2023-12-20T04:47:22.063682+0000","last_clean":"2023-12-20T04:47:22.063682+0000","last_became_active":"2023-12-20T04:47:21.077814+0000","last_became_peered":"2023-12-20T04:47:21.077814+0000","last_unstale":"2023-12-20T04:47:22.063682+0000","last_undegraded":"2023-12-20T04:47:22.063682+0000","last_fullsized":"2023-12-20T04:47:22.063682+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:01:58.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.39","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067161+0000","last_change":"2023-12-20T04:47:21.081421+0000","last_active":"2023-12-20T04:47:22.067161+0000","last_peered":"2023-12-20T04:47:22.067161+0000","last_clean":"2023-12-20T04:47:22.067161+0000","last_became_active":"2023-12-20T04:47:21.081082+0000","last_became_peered":"2023-12-20T04:47:21.081082+0000","last_unstale":"2023-12-20T04:47:22.067161+0000","last_undegraded":"2023-12-20T04:47:22.067161+0000","last_fullsized":"2023-12-20T04:47:22.067161+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:36:31.212164+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.38","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517430+0000","last_change":"2023-12-20T04:47:21.079984+0000","last_active":"2023-12-20T04:47:22.517430+0000","last_peered":"2023-12-20T04:47:22.517430+0000","last_clean":"2023-12-20T04:47:22.517430+0000","last_became_active":"2023-12-20T04:47:21.078965+0000","last_became_peered":"2023-12-20T04:47:21.078965+0000","last_unstale":"2023-12-20T04:47:22.517430+0000","last_undegraded":"2023-12-20T04:47:22.517430+0000","last_fullsized":"2023-12-20T04:47:22.517430+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:14:04.540171+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.37","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350916+0000","last_change":"2023-12-20T04:47:21.086683+0000","last_active":"2023-12-20T04:47:22.350916+0000","last_peered":"2023-12-20T04:47:22.350916+0000","last_clean":"2023-12-20T04:47:22.350916+0000","last_became_active":"2023-12-20T04:47:21.086380+0000","last_became_peered":"2023-12-20T04:47:21.086380+0000","last_unstale":"2023-12-20T04:47:22.350916+0000","last_undegraded":"2023-12-20T04:47:22.350916+0000","last_fullsized":"2023-12-20T04:47:22.350916+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:58.717666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.36","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350761+0000","last_change":"2023-12-20T04:47:21.089785+0000","last_active":"2023-12-20T04:47:22.350761+0000","last_peered":"2023-12-20T04:47:22.350761+0000","last_clean":"2023-12-20T04:47:22.350761+0000","last_became_active":"2023-12-20T04:47:21.089533+0000","last_became_peered":"2023-12-20T04:47:21.089533+0000","last_unstale":"2023-12-20T04:47:22.350761+0000","last_undegraded":"2023-12-20T04:47:22.350761+0000","last_fullsized":"2023-12-20T04:47:22.350761+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:56:36.089809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.35","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066512+0000","last_change":"2023-12-20T04:47:21.087132+0000","last_active":"2023-12-20T04:47:22.066512+0000","last_peered":"2023-12-20T04:47:22.066512+0000","last_clean":"2023-12-20T04:47:22.066512+0000","last_became_active":"2023-12-20T04:47:21.086798+0000","last_became_peered":"2023-12-20T04:47:21.086798+0000","last_unstale":"2023-12-20T04:47:22.066512+0000","last_undegraded":"2023-12-20T04:47:22.066512+0000","last_fullsized":"2023-12-20T04:47:22.066512+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:33:53.116054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.34","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350677+0000","last_change":"2023-12-20T04:47:21.092487+0000","last_active":"2023-12-20T04:47:22.350677+0000","last_peered":"2023-12-20T04:47:22.350677+0000","last_clean":"2023-12-20T04:47:22.350677+0000","last_became_active":"2023-12-20T04:47:21.092186+0000","last_became_peered":"2023-12-20T04:47:21.092186+0000","last_unstale":"2023-12-20T04:47:22.350677+0000","last_undegraded":"2023-12-20T04:47:22.350677+0000","last_fullsized":"2023-12-20T04:47:22.350677+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:40.036506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.33","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517007+0000","last_change":"2023-12-20T04:47:21.077812+0000","last_active":"2023-12-20T04:47:22.517007+0000","last_peered":"2023-12-20T04:47:22.517007+0000","last_clean":"2023-12-20T04:47:22.517007+0000","last_became_active":"2023-12-20T04:47:21.076837+0000","last_became_peered":"2023-12-20T04:47:21.076837+0000","last_unstale":"2023-12-20T04:47:22.517007+0000","last_undegraded":"2023-12-20T04:47:22.517007+0000","last_fullsized":"2023-12-20T04:47:22.517007+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:34:04.890556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.32","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.516778+0000","last_change":"2023-12-20T04:47:21.083103+0000","last_active":"2023-12-20T04:47:22.516778+0000","last_peered":"2023-12-20T04:47:22.516778+0000","last_clean":"2023-12-20T04:47:22.516778+0000","last_became_active":"2023-12-20T04:47:21.082803+0000","last_became_peered":"2023-12-20T04:47:21.082803+0000","last_unstale":"2023-12-20T04:47:22.516778+0000","last_undegraded":"2023-12-20T04:47:22.516778+0000","last_fullsized":"2023-12-20T04:47:22.516778+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:05:19.725201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.31","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063600+0000","last_change":"2023-12-20T04:47:21.080822+0000","last_active":"2023-12-20T04:47:22.063600+0000","last_peered":"2023-12-20T04:47:22.063600+0000","last_clean":"2023-12-20T04:47:22.063600+0000","last_became_active":"2023-12-20T04:47:21.079908+0000","last_became_peered":"2023-12-20T04:47:21.079908+0000","last_unstale":"2023-12-20T04:47:22.063600+0000","last_undegraded":"2023-12-20T04:47:22.063600+0000","last_fullsized":"2023-12-20T04:47:22.063600+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:32:59.997762+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.30","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350361+0000","last_change":"2023-12-20T04:47:21.087903+0000","last_active":"2023-12-20T04:47:22.350361+0000","last_peered":"2023-12-20T04:47:22.350361+0000","last_clean":"2023-12-20T04:47:22.350361+0000","last_became_active":"2023-12-20T04:47:21.087650+0000","last_became_peered":"2023-12-20T04:47:21.087650+0000","last_unstale":"2023-12-20T04:47:22.350361+0000","last_undegraded":"2023-12-20T04:47:22.350361+0000","last_fullsized":"2023-12-20T04:47:22.350361+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:00:12.243151+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.2f","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066720+0000","last_change":"2023-12-20T04:47:21.087850+0000","last_active":"2023-12-20T04:47:22.066720+0000","last_peered":"2023-12-20T04:47:22.066720+0000","last_clean":"2023-12-20T04:47:22.066720+0000","last_became_active":"2023-12-20T04:47:21.086811+0000","last_became_peered":"2023-12-20T04:47:21.086811+0000","last_unstale":"2023-12-20T04:47:22.066720+0000","last_undegraded":"2023-12-20T04:47:22.066720+0000","last_fullsized":"2023-12-20T04:47:22.066720+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:35.008516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2e","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.516872+0000","last_change":"2023-12-20T04:47:21.088414+0000","last_active":"2023-12-20T04:47:22.516872+0000","last_peered":"2023-12-20T04:47:22.516872+0000","last_clean":"2023-12-20T04:47:22.516872+0000","last_became_active":"2023-12-20T04:47:21.087622+0000","last_became_peered":"2023-12-20T04:47:21.087622+0000","last_unstale":"2023-12-20T04:47:22.516872+0000","last_undegraded":"2023-12-20T04:47:22.516872+0000","last_fullsized":"2023-12-20T04:47:22.516872+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:52:31.943799+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.2d","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517204+0000","last_change":"2023-12-20T04:47:21.083411+0000","last_active":"2023-12-20T04:47:22.517204+0000","last_peered":"2023-12-20T04:47:22.517204+0000","last_clean":"2023-12-20T04:47:22.517204+0000","last_became_active":"2023-12-20T04:47:21.082408+0000","last_became_peered":"2023-12-20T04:47:21.082408+0000","last_unstale":"2023-12-20T04:47:22.517204+0000","last_undegraded":"2023-12-20T04:47:22.517204+0000","last_fullsized":"2023-12-20T04:47:22.517204+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:40:35.208765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.2c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066484+0000","last_change":"2023-12-20T04:47:21.071406+0000","last_active":"2023-12-20T04:47:22.066484+0000","last_peered":"2023-12-20T04:47:22.066484+0000","last_clean":"2023-12-20T04:47:22.066484+0000","last_became_active":"2023-12-20T04:47:21.071095+0000","last_became_peered":"2023-12-20T04:47:21.071095+0000","last_unstale":"2023-12-20T04:47:22.066484+0000","last_undegraded":"2023-12-20T04:47:22.066484+0000","last_fullsized":"2023-12-20T04:47:22.066484+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:36:24.652104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2b","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067046+0000","last_change":"2023-12-20T04:47:21.077702+0000","last_active":"2023-12-20T04:47:22.067046+0000","last_peered":"2023-12-20T04:47:22.067046+0000","last_clean":"2023-12-20T04:47:22.067046+0000","last_became_active":"2023-12-20T04:47:21.075896+0000","last_became_peered":"2023-12-20T04:47:21.075896+0000","last_unstale":"2023-12-20T04:47:22.067046+0000","last_undegraded":"2023-12-20T04:47:22.067046+0000","last_fullsized":"2023-12-20T04:47:22.067046+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:13:22.066012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.2a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515466+0000","last_change":"2023-12-20T04:47:21.083596+0000","last_active":"2023-12-20T04:47:22.515466+0000","last_peered":"2023-12-20T04:47:22.515466+0000","last_clean":"2023-12-20T04:47:22.515466+0000","last_became_active":"2023-12-20T04:47:21.083346+0000","last_became_peered":"2023-12-20T04:47:21.083346+0000","last_unstale":"2023-12-20T04:47:22.515466+0000","last_undegraded":"2023-12-20T04:47:22.515466+0000","last_fullsized":"2023-12-20T04:47:22.515466+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:20:30.137015+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.29","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066204+0000","last_change":"2023-12-20T04:47:21.072642+0000","last_active":"2023-12-20T04:47:22.066204+0000","last_peered":"2023-12-20T04:47:22.066204+0000","last_clean":"2023-12-20T04:47:22.066204+0000","last_became_active":"2023-12-20T04:47:21.071745+0000","last_became_peered":"2023-12-20T04:47:21.071745+0000","last_unstale":"2023-12-20T04:47:22.066204+0000","last_undegraded":"2023-12-20T04:47:22.066204+0000","last_fullsized":"2023-12-20T04:47:22.066204+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:04.754646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.28","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515142+0000","last_change":"2023-12-20T04:47:21.071373+0000","last_active":"2023-12-20T04:47:22.515142+0000","last_peered":"2023-12-20T04:47:22.515142+0000","last_clean":"2023-12-20T04:47:22.515142+0000","last_became_active":"2023-12-20T04:47:21.070805+0000","last_became_peered":"2023-12-20T04:47:21.070805+0000","last_unstale":"2023-12-20T04:47:22.515142+0000","last_undegraded":"2023-12-20T04:47:22.515142+0000","last_fullsized":"2023-12-20T04:47:22.515142+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:57.798722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.27","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066836+0000","last_change":"2023-12-20T04:47:21.087422+0000","last_active":"2023-12-20T04:47:22.066836+0000","last_peered":"2023-12-20T04:47:22.066836+0000","last_clean":"2023-12-20T04:47:22.066836+0000","last_became_active":"2023-12-20T04:47:21.086945+0000","last_became_peered":"2023-12-20T04:47:21.086945+0000","last_unstale":"2023-12-20T04:47:22.066836+0000","last_undegraded":"2023-12-20T04:47:22.066836+0000","last_fullsized":"2023-12-20T04:47:22.066836+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:40.721761+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.26","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350787+0000","last_change":"2023-12-20T04:47:21.086979+0000","last_active":"2023-12-20T04:47:22.350787+0000","last_peered":"2023-12-20T04:47:22.350787+0000","last_clean":"2023-12-20T04:47:22.350787+0000","last_became_active":"2023-12-20T04:47:21.079415+0000","last_became_peered":"2023-12-20T04:47:21.079415+0000","last_unstale":"2023-12-20T04:47:22.350787+0000","last_undegraded":"2023-12-20T04:47:22.350787+0000","last_fullsized":"2023-12-20T04:47:22.350787+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:51:20.521357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.25","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.064095+0000","last_change":"2023-12-20T04:47:21.084937+0000","last_active":"2023-12-20T04:47:22.064095+0000","last_peered":"2023-12-20T04:47:22.064095+0000","last_clean":"2023-12-20T04:47:22.064095+0000","last_became_active":"2023-12-20T04:47:21.084486+0000","last_became_peered":"2023-12-20T04:47:21.084486+0000","last_unstale":"2023-12-20T04:47:22.064095+0000","last_undegraded":"2023-12-20T04:47:22.064095+0000","last_fullsized":"2023-12-20T04:47:22.064095+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:57:47.330544+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.24","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515724+0000","last_change":"2023-12-20T04:47:21.084096+0000","last_active":"2023-12-20T04:47:22.515724+0000","last_peered":"2023-12-20T04:47:22.515724+0000","last_clean":"2023-12-20T04:47:22.515724+0000","last_became_active":"2023-12-20T04:47:21.083160+0000","last_became_peered":"2023-12-20T04:47:21.083160+0000","last_unstale":"2023-12-20T04:47:22.515724+0000","last_undegraded":"2023-12-20T04:47:22.515724+0000","last_fullsized":"2023-12-20T04:47:22.515724+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:29:20.024881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.23","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066190+0000","last_change":"2023-12-20T04:47:21.073672+0000","last_active":"2023-12-20T04:47:22.066190+0000","last_peered":"2023-12-20T04:47:22.066190+0000","last_clean":"2023-12-20T04:47:22.066190+0000","last_became_active":"2023-12-20T04:47:21.071261+0000","last_became_peered":"2023-12-20T04:47:21.071261+0000","last_unstale":"2023-12-20T04:47:22.066190+0000","last_undegraded":"2023-12-20T04:47:22.066190+0000","last_fullsized":"2023-12-20T04:47:22.066190+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:18:33.360364+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.22","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067124+0000","last_change":"2023-12-20T04:47:21.079530+0000","last_active":"2023-12-20T04:47:22.067124+0000","last_peered":"2023-12-20T04:47:22.067124+0000","last_clean":"2023-12-20T04:47:22.067124+0000","last_became_active":"2023-12-20T04:47:21.079239+0000","last_became_peered":"2023-12-20T04:47:21.079239+0000","last_unstale":"2023-12-20T04:47:22.067124+0000","last_undegraded":"2023-12-20T04:47:22.067124+0000","last_fullsized":"2023-12-20T04:47:22.067124+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:10:55.384589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.21","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350457+0000","last_change":"2023-12-20T04:47:21.077303+0000","last_active":"2023-12-20T04:47:22.350457+0000","last_peered":"2023-12-20T04:47:22.350457+0000","last_clean":"2023-12-20T04:47:22.350457+0000","last_became_active":"2023-12-20T04:47:21.074740+0000","last_became_peered":"2023-12-20T04:47:21.074740+0000","last_unstale":"2023-12-20T04:47:22.350457+0000","last_undegraded":"2023-12-20T04:47:22.350457+0000","last_fullsized":"2023-12-20T04:47:22.350457+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:02:05.096070+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.20","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063599+0000","last_change":"2023-12-20T04:47:21.080479+0000","last_active":"2023-12-20T04:47:22.063599+0000","last_peered":"2023-12-20T04:47:22.063599+0000","last_clean":"2023-12-20T04:47:22.063599+0000","last_became_active":"2023-12-20T04:47:21.079140+0000","last_became_peered":"2023-12-20T04:47:21.079140+0000","last_unstale":"2023-12-20T04:47:22.063599+0000","last_undegraded":"2023-12-20T04:47:22.063599+0000","last_fullsized":"2023-12-20T04:47:22.063599+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:10:55.384589+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.1e","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.064177+0000","last_change":"2023-12-20T04:47:21.080796+0000","last_active":"2023-12-20T04:47:22.064177+0000","last_peered":"2023-12-20T04:47:22.064177+0000","last_clean":"2023-12-20T04:47:22.064177+0000","last_became_active":"2023-12-20T04:47:21.080543+0000","last_became_peered":"2023-12-20T04:47:21.080543+0000","last_unstale":"2023-12-20T04:47:22.064177+0000","last_undegraded":"2023-12-20T04:47:22.064177+0000","last_fullsized":"2023-12-20T04:47:22.064177+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:12.552047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1f","version":"137'4","reported_seq":15,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067209+0000","last_change":"2023-12-20T04:47:20.672320+0000","last_active":"2023-12-20T04:47:22.067209+0000","last_peered":"2023-12-20T04:47:22.067209+0000","last_clean":"2023-12-20T04:47:22.067209+0000","last_became_active":"2023-12-20T04:47:20.672052+0000","last_became_peered":"2023-12-20T04:47:20.672052+0000","last_unstale":"2023-12-20T04:47:22.067209+0000","last_undegraded":"2023-12-20T04:47:22.067209+0000","last_fullsized":"2023-12-20T04:47:22.067209+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:25:19.317934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1f","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.516979+0000","last_change":"2023-12-20T04:47:21.086129+0000","last_active":"2023-12-20T04:47:22.516979+0000","last_peered":"2023-12-20T04:47:22.516979+0000","last_clean":"2023-12-20T04:47:22.516979+0000","last_became_active":"2023-12-20T04:47:21.085835+0000","last_became_peered":"2023-12-20T04:47:21.085835+0000","last_unstale":"2023-12-20T04:47:22.516979+0000","last_undegraded":"2023-12-20T04:47:22.516979+0000","last_fullsized":"2023-12-20T04:47:22.516979+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:47:50.903316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.1e","version":"137'1","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517248+0000","last_change":"2023-12-20T04:47:20.676411+0000","last_active":"2023-12-20T04:47:22.517248+0000","last_peered":"2023-12-20T04:47:22.517248+0000","last_clean":"2023-12-20T04:47:22.517248+0000","last_became_active":"2023-12-20T04:47:20.676027+0000","last_became_peered":"2023-12-20T04:47:20.676027+0000","last_unstale":"2023-12-20T04:47:22.517248+0000","last_undegraded":"2023-12-20T04:47:22.517248+0000","last_fullsized":"2023-12-20T04:47:22.517248+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:57:14.775610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"17.1c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.064298+0000","last_change":"2023-12-20T04:47:21.085293+0000","last_active":"2023-12-20T04:47:22.064298+0000","last_peered":"2023-12-20T04:47:22.064298+0000","last_clean":"2023-12-20T04:47:22.064298+0000","last_became_active":"2023-12-20T04:47:21.085037+0000","last_became_peered":"2023-12-20T04:47:21.085037+0000","last_unstale":"2023-12-20T04:47:22.064298+0000","last_undegraded":"2023-12-20T04:47:22.064298+0000","last_fullsized":"2023-12-20T04:47:22.064298+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:44:38.501754+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.1d","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515270+0000","last_change":"2023-12-20T04:47:20.673680+0000","last_active":"2023-12-20T04:47:22.515270+0000","last_peered":"2023-12-20T04:47:22.515270+0000","last_clean":"2023-12-20T04:47:22.515270+0000","last_became_active":"2023-12-20T04:47:20.673096+0000","last_became_peered":"2023-12-20T04:47:20.673096+0000","last_unstale":"2023-12-20T04:47:22.515270+0000","last_undegraded":"2023-12-20T04:47:22.515270+0000","last_fullsized":"2023-12-20T04:47:22.515270+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:03.093126+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1d","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517241+0000","last_change":"2023-12-20T04:47:21.077215+0000","last_active":"2023-12-20T04:47:22.517241+0000","last_peered":"2023-12-20T04:47:22.517241+0000","last_clean":"2023-12-20T04:47:22.517241+0000","last_became_active":"2023-12-20T04:47:21.076833+0000","last_became_peered":"2023-12-20T04:47:21.076833+0000","last_unstale":"2023-12-20T04:47:22.517241+0000","last_undegraded":"2023-12-20T04:47:22.517241+0000","last_fullsized":"2023-12-20T04:47:22.517241+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:57.570829+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1c","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515012+0000","last_change":"2023-12-20T04:47:20.673930+0000","last_active":"2023-12-20T04:47:22.515012+0000","last_peered":"2023-12-20T04:47:22.515012+0000","last_clean":"2023-12-20T04:47:22.515012+0000","last_became_active":"2023-12-20T04:47:20.673625+0000","last_became_peered":"2023-12-20T04:47:20.673625+0000","last_unstale":"2023-12-20T04:47:22.515012+0000","last_undegraded":"2023-12-20T04:47:22.515012+0000","last_fullsized":"2023-12-20T04:47:22.515012+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:10:27.154267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350470+0000","last_change":"2023-12-20T04:47:21.085541+0000","last_active":"2023-12-20T04:47:22.350470+0000","last_peered":"2023-12-20T04:47:22.350470+0000","last_clean":"2023-12-20T04:47:22.350470+0000","last_became_active":"2023-12-20T04:47:21.085274+0000","last_became_peered":"2023-12-20T04:47:21.085274+0000","last_unstale":"2023-12-20T04:47:22.350470+0000","last_undegraded":"2023-12-20T04:47:22.350470+0000","last_fullsized":"2023-12-20T04:47:22.350470+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:28.137951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.1b","version":"137'1","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067271+0000","last_change":"2023-12-20T04:47:20.665305+0000","last_active":"2023-12-20T04:47:22.067271+0000","last_peered":"2023-12-20T04:47:22.067271+0000","last_clean":"2023-12-20T04:47:22.067271+0000","last_became_active":"2023-12-20T04:47:20.664920+0000","last_became_peered":"2023-12-20T04:47:20.664920+0000","last_unstale":"2023-12-20T04:47:22.067271+0000","last_undegraded":"2023-12-20T04:47:22.067271+0000","last_fullsized":"2023-12-20T04:47:22.067271+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:54:01.405424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.1b","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066427+0000","last_change":"2023-12-20T04:47:21.081180+0000","last_active":"2023-12-20T04:47:22.066427+0000","last_peered":"2023-12-20T04:47:22.066427+0000","last_clean":"2023-12-20T04:47:22.066427+0000","last_became_active":"2023-12-20T04:47:21.080689+0000","last_became_peered":"2023-12-20T04:47:21.080689+0000","last_unstale":"2023-12-20T04:47:22.066427+0000","last_undegraded":"2023-12-20T04:47:22.066427+0000","last_fullsized":"2023-12-20T04:47:22.066427+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:09.861807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.1a","version":"137'1","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515555+0000","last_change":"2023-12-20T04:47:20.669215+0000","last_active":"2023-12-20T04:47:22.515555+0000","last_peered":"2023-12-20T04:47:22.515555+0000","last_clean":"2023-12-20T04:47:22.515555+0000","last_became_active":"2023-12-20T04:47:20.668956+0000","last_became_peered":"2023-12-20T04:47:20.668956+0000","last_unstale":"2023-12-20T04:47:22.515555+0000","last_undegraded":"2023-12-20T04:47:22.515555+0000","last_fullsized":"2023-12-20T04:47:22.515555+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:43:16.354141+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.18","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350395+0000","last_change":"2023-12-20T04:47:21.079411+0000","last_active":"2023-12-20T04:47:22.350395+0000","last_peered":"2023-12-20T04:47:22.350395+0000","last_clean":"2023-12-20T04:47:22.350395+0000","last_became_active":"2023-12-20T04:47:21.079103+0000","last_became_peered":"2023-12-20T04:47:21.079103+0000","last_unstale":"2023-12-20T04:47:22.350395+0000","last_undegraded":"2023-12-20T04:47:22.350395+0000","last_fullsized":"2023-12-20T04:47:22.350395+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:39:08.904834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.19","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066316+0000","last_change":"2023-12-20T04:47:20.667597+0000","last_active":"2023-12-20T04:47:22.066316+0000","last_peered":"2023-12-20T04:47:22.066316+0000","last_clean":"2023-12-20T04:47:22.066316+0000","last_became_active":"2023-12-20T04:47:20.667338+0000","last_became_peered":"2023-12-20T04:47:20.667338+0000","last_unstale":"2023-12-20T04:47:22.066316+0000","last_undegraded":"2023-12-20T04:47:22.066316+0000","last_fullsized":"2023-12-20T04:47:22.066316+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:30:10.422004+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.19","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067030+0000","last_change":"2023-12-20T04:47:21.081796+0000","last_active":"2023-12-20T04:47:22.067030+0000","last_peered":"2023-12-20T04:47:22.067030+0000","last_clean":"2023-12-20T04:47:22.067030+0000","last_became_active":"2023-12-20T04:47:21.081521+0000","last_became_peered":"2023-12-20T04:47:21.081521+0000","last_unstale":"2023-12-20T04:47:22.067030+0000","last_undegraded":"2023-12-20T04:47:22.067030+0000","last_fullsized":"2023-12-20T04:47:22.067030+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:00:32.666079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.18","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350465+0000","last_change":"2023-12-20T04:47:20.670726+0000","last_active":"2023-12-20T04:47:22.350465+0000","last_peered":"2023-12-20T04:47:22.350465+0000","last_clean":"2023-12-20T04:47:22.350465+0000","last_became_active":"2023-12-20T04:47:20.670464+0000","last_became_peered":"2023-12-20T04:47:20.670464+0000","last_unstale":"2023-12-20T04:47:22.350465+0000","last_undegraded":"2023-12-20T04:47:22.350465+0000","last_fullsized":"2023-12-20T04:47:22.350465+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:05:27.098396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.16","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067385+0000","last_change":"2023-12-20T04:47:21.074982+0000","last_active":"2023-12-20T04:47:22.067385+0000","last_peered":"2023-12-20T04:47:22.067385+0000","last_clean":"2023-12-20T04:47:22.067385+0000","last_became_active":"2023-12-20T04:47:21.074655+0000","last_became_peered":"2023-12-20T04:47:21.074655+0000","last_unstale":"2023-12-20T04:47:22.067385+0000","last_undegraded":"2023-12-20T04:47:22.067385+0000","last_fullsized":"2023-12-20T04:47:22.067385+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:12.552047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.17","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515243+0000","last_change":"2023-12-20T04:47:20.673320+0000","last_active":"2023-12-20T04:47:22.515243+0000","last_peered":"2023-12-20T04:47:22.515243+0000","last_clean":"2023-12-20T04:47:22.515243+0000","last_became_active":"2023-12-20T04:47:20.673019+0000","last_became_peered":"2023-12-20T04:47:20.673019+0000","last_unstale":"2023-12-20T04:47:22.515243+0000","last_undegraded":"2023-12-20T04:47:22.515243+0000","last_fullsized":"2023-12-20T04:47:22.515243+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:25:19.317934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.17","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517020+0000","last_change":"2023-12-20T04:47:21.096934+0000","last_active":"2023-12-20T04:47:22.517020+0000","last_peered":"2023-12-20T04:47:22.517020+0000","last_clean":"2023-12-20T04:47:22.517020+0000","last_became_active":"2023-12-20T04:47:21.096663+0000","last_became_peered":"2023-12-20T04:47:21.096663+0000","last_unstale":"2023-12-20T04:47:22.517020+0000","last_undegraded":"2023-12-20T04:47:22.517020+0000","last_fullsized":"2023-12-20T04:47:22.517020+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:57:15.777946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.16","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515639+0000","last_change":"2023-12-20T04:47:20.671827+0000","last_active":"2023-12-20T04:47:22.515639+0000","last_peered":"2023-12-20T04:47:22.515639+0000","last_clean":"2023-12-20T04:47:22.515639+0000","last_became_active":"2023-12-20T04:47:20.671540+0000","last_became_peered":"2023-12-20T04:47:20.671540+0000","last_unstale":"2023-12-20T04:47:22.515639+0000","last_undegraded":"2023-12-20T04:47:22.515639+0000","last_fullsized":"2023-12-20T04:47:22.515639+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:19:56.568492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.14","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515747+0000","last_change":"2023-12-20T04:47:21.067337+0000","last_active":"2023-12-20T04:47:22.515747+0000","last_peered":"2023-12-20T04:47:22.515747+0000","last_clean":"2023-12-20T04:47:22.515747+0000","last_became_active":"2023-12-20T04:47:21.067077+0000","last_became_peered":"2023-12-20T04:47:21.067077+0000","last_unstale":"2023-12-20T04:47:22.515747+0000","last_undegraded":"2023-12-20T04:47:22.515747+0000","last_fullsized":"2023-12-20T04:47:22.515747+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:16.089644+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.15","version":"137'3","reported_seq":14,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350773+0000","last_change":"2023-12-20T04:47:20.680376+0000","last_active":"2023-12-20T04:47:22.350773+0000","last_peered":"2023-12-20T04:47:22.350773+0000","last_clean":"2023-12-20T04:47:22.350773+0000","last_became_active":"2023-12-20T04:47:20.679529+0000","last_became_peered":"2023-12-20T04:47:20.679529+0000","last_unstale":"2023-12-20T04:47:22.350773+0000","last_undegraded":"2023-12-20T04:47:22.350773+0000","last_fullsized":"2023-12-20T04:47:22.350773+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:56.796386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.15","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067530+0000","last_change":"2023-12-20T04:47:21.071747+0000","last_active":"2023-12-20T04:47:22.067530+0000","last_peered":"2023-12-20T04:47:22.067530+0000","last_clean":"2023-12-20T04:47:22.067530+0000","last_became_active":"2023-12-20T04:47:21.070530+0000","last_became_peered":"2023-12-20T04:47:21.070530+0000","last_unstale":"2023-12-20T04:47:22.067530+0000","last_undegraded":"2023-12-20T04:47:22.067530+0000","last_fullsized":"2023-12-20T04:47:22.067530+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:01:58.495054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"16.14","version":"137'6","reported_seq":17,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350892+0000","last_change":"2023-12-20T04:47:20.671461+0000","last_active":"2023-12-20T04:47:22.350892+0000","last_peered":"2023-12-20T04:47:22.350892+0000","last_clean":"2023-12-20T04:47:22.350892+0000","last_became_active":"2023-12-20T04:47:20.670521+0000","last_became_peered":"2023-12-20T04:47:20.670521+0000","last_unstale":"2023-12-20T04:47:22.350892+0000","last_undegraded":"2023-12-20T04:47:22.350892+0000","last_fullsized":"2023-12-20T04:47:22.350892+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:06.195631+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.12","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515510+0000","last_change":"2023-12-20T04:47:21.066579+0000","last_active":"2023-12-20T04:47:22.515510+0000","last_peered":"2023-12-20T04:47:22.515510+0000","last_clean":"2023-12-20T04:47:22.515510+0000","last_became_active":"2023-12-20T04:47:21.066321+0000","last_became_peered":"2023-12-20T04:47:21.066321+0000","last_unstale":"2023-12-20T04:47:22.515510+0000","last_undegraded":"2023-12-20T04:47:22.515510+0000","last_fullsized":"2023-12-20T04:47:22.515510+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:37:07.023828+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.13","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063900+0000","last_change":"2023-12-20T04:47:20.671126+0000","last_active":"2023-12-20T04:47:22.063900+0000","last_peered":"2023-12-20T04:47:22.063900+0000","last_clean":"2023-12-20T04:47:22.063900+0000","last_became_active":"2023-12-20T04:47:20.670856+0000","last_became_peered":"2023-12-20T04:47:20.670856+0000","last_unstale":"2023-12-20T04:47:22.063900+0000","last_undegraded":"2023-12-20T04:47:22.063900+0000","last_fullsized":"2023-12-20T04:47:22.063900+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:59.615775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.13","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515534+0000","last_change":"2023-12-20T04:47:21.071478+0000","last_active":"2023-12-20T04:47:22.515534+0000","last_peered":"2023-12-20T04:47:22.515534+0000","last_clean":"2023-12-20T04:47:22.515534+0000","last_became_active":"2023-12-20T04:47:21.070759+0000","last_became_peered":"2023-12-20T04:47:21.070759+0000","last_unstale":"2023-12-20T04:47:22.515534+0000","last_undegraded":"2023-12-20T04:47:22.515534+0000","last_fullsized":"2023-12-20T04:47:22.515534+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:07.197968+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.12","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517394+0000","last_change":"2023-12-20T04:47:20.681539+0000","last_active":"2023-12-20T04:47:22.517394+0000","last_peered":"2023-12-20T04:47:22.517394+0000","last_clean":"2023-12-20T04:47:22.517394+0000","last_became_active":"2023-12-20T04:47:20.681330+0000","last_became_peered":"2023-12-20T04:47:20.681330+0000","last_unstale":"2023-12-20T04:47:22.517394+0000","last_undegraded":"2023-12-20T04:47:22.517394+0000","last_fullsized":"2023-12-20T04:47:22.517394+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:44:37.499418+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.11","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515598+0000","last_change":"2023-12-20T04:47:21.068586+0000","last_active":"2023-12-20T04:47:22.515598+0000","last_peered":"2023-12-20T04:47:22.515598+0000","last_clean":"2023-12-20T04:47:22.515598+0000","last_became_active":"2023-12-20T04:47:21.065948+0000","last_became_peered":"2023-12-20T04:47:21.065948+0000","last_unstale":"2023-12-20T04:47:22.515598+0000","last_undegraded":"2023-12-20T04:47:22.515598+0000","last_fullsized":"2023-12-20T04:47:22.515598+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:36:24.652104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.10","version":"137'8","reported_seq":19,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.514841+0000","last_change":"2023-12-20T04:47:20.679901+0000","last_active":"2023-12-20T04:47:22.514841+0000","last_peered":"2023-12-20T04:47:22.514841+0000","last_clean":"2023-12-20T04:47:22.514841+0000","last_became_active":"2023-12-20T04:47:20.679640+0000","last_became_peered":"2023-12-20T04:47:20.679640+0000","last_unstale":"2023-12-20T04:47:22.514841+0000","last_undegraded":"2023-12-20T04:47:22.514841+0000","last_fullsized":"2023-12-20T04:47:22.514841+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:13:33.535191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.e","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515423+0000","last_change":"2023-12-20T04:47:21.080324+0000","last_active":"2023-12-20T04:47:22.515423+0000","last_peered":"2023-12-20T04:47:22.515423+0000","last_clean":"2023-12-20T04:47:22.515423+0000","last_became_active":"2023-12-20T04:47:21.080056+0000","last_became_peered":"2023-12-20T04:47:21.080056+0000","last_unstale":"2023-12-20T04:47:22.515423+0000","last_undegraded":"2023-12-20T04:47:22.515423+0000","last_fullsized":"2023-12-20T04:47:22.515423+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:37:19.231725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"16.f","version":"137'1","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.351055+0000","last_change":"2023-12-20T04:47:20.674266+0000","last_active":"2023-12-20T04:47:22.351055+0000","last_peered":"2023-12-20T04:47:22.351055+0000","last_clean":"2023-12-20T04:47:22.351055+0000","last_became_active":"2023-12-20T04:47:20.673457+0000","last_became_peered":"2023-12-20T04:47:20.673457+0000","last_unstale":"2023-12-20T04:47:22.351055+0000","last_undegraded":"2023-12-20T04:47:22.351055+0000","last_fullsized":"2023-12-20T04:47:22.351055+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:18:32.358027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.f","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.064193+0000","last_change":"2023-12-20T04:47:21.090277+0000","last_active":"2023-12-20T04:47:22.064193+0000","last_peered":"2023-12-20T04:47:22.064193+0000","last_clean":"2023-12-20T04:47:22.064193+0000","last_became_active":"2023-12-20T04:47:21.089945+0000","last_became_peered":"2023-12-20T04:47:21.089945+0000","last_unstale":"2023-12-20T04:47:22.064193+0000","last_undegraded":"2023-12-20T04:47:22.064193+0000","last_fullsized":"2023-12-20T04:47:22.064193+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:10:25.971082+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"16.e","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515506+0000","last_change":"2023-12-20T04:47:20.670643+0000","last_active":"2023-12-20T04:47:22.515506+0000","last_peered":"2023-12-20T04:47:22.515506+0000","last_clean":"2023-12-20T04:47:22.515506+0000","last_became_active":"2023-12-20T04:47:20.670364+0000","last_became_peered":"2023-12-20T04:47:20.670364+0000","last_unstale":"2023-12-20T04:47:22.515506+0000","last_undegraded":"2023-12-20T04:47:22.515506+0000","last_fullsized":"2023-12-20T04:47:22.515506+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:56:44.521740+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.c","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517204+0000","last_change":"2023-12-20T04:47:21.082753+0000","last_active":"2023-12-20T04:47:22.517204+0000","last_peered":"2023-12-20T04:47:22.517204+0000","last_clean":"2023-12-20T04:47:22.517204+0000","last_became_active":"2023-12-20T04:47:21.082438+0000","last_became_peered":"2023-12-20T04:47:21.082438+0000","last_unstale":"2023-12-20T04:47:22.517204+0000","last_undegraded":"2023-12-20T04:47:22.517204+0000","last_fullsized":"2023-12-20T04:47:22.517204+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:33:53.116054+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.d","version":"137'1","reported_seq":12,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517231+0000","last_change":"2023-12-20T04:47:20.674831+0000","last_active":"2023-12-20T04:47:22.517231+0000","last_peered":"2023-12-20T04:47:22.517231+0000","last_clean":"2023-12-20T04:47:22.517231+0000","last_became_active":"2023-12-20T04:47:20.674527+0000","last_became_peered":"2023-12-20T04:47:20.674527+0000","last_unstale":"2023-12-20T04:47:22.517231+0000","last_undegraded":"2023-12-20T04:47:22.517231+0000","last_fullsized":"2023-12-20T04:47:22.517231+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:57:46.328208+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.d","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517317+0000","last_change":"2023-12-20T04:47:21.075450+0000","last_active":"2023-12-20T04:47:22.517317+0000","last_peered":"2023-12-20T04:47:22.517317+0000","last_clean":"2023-12-20T04:47:22.517317+0000","last_became_active":"2023-12-20T04:47:21.074991+0000","last_became_peered":"2023-12-20T04:47:21.074991+0000","last_unstale":"2023-12-20T04:47:22.517317+0000","last_undegraded":"2023-12-20T04:47:22.517317+0000","last_fullsized":"2023-12-20T04:47:22.517317+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:43:17.356477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.c","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067443+0000","last_change":"2023-12-20T04:47:20.674932+0000","last_active":"2023-12-20T04:47:22.067443+0000","last_peered":"2023-12-20T04:47:22.067443+0000","last_clean":"2023-12-20T04:47:22.067443+0000","last_became_active":"2023-12-20T04:47:20.674611+0000","last_became_peered":"2023-12-20T04:47:20.674611+0000","last_unstale":"2023-12-20T04:47:22.067443+0000","last_undegraded":"2023-12-20T04:47:22.067443+0000","last_fullsized":"2023-12-20T04:47:22.067443+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:59.615775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.a","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.516890+0000","last_change":"2023-12-20T04:47:21.069399+0000","last_active":"2023-12-20T04:47:22.516890+0000","last_peered":"2023-12-20T04:47:22.516890+0000","last_clean":"2023-12-20T04:47:22.516890+0000","last_became_active":"2023-12-20T04:47:21.069097+0000","last_became_peered":"2023-12-20T04:47:21.069097+0000","last_unstale":"2023-12-20T04:47:22.516890+0000","last_undegraded":"2023-12-20T04:47:22.516890+0000","last_fullsized":"2023-12-20T04:47:22.516890+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:09.861807+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.b","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063916+0000","last_change":"2023-12-20T04:47:20.675234+0000","last_active":"2023-12-20T04:47:22.063916+0000","last_peered":"2023-12-20T04:47:22.063916+0000","last_clean":"2023-12-20T04:47:22.063916+0000","last_became_active":"2023-12-20T04:47:20.674918+0000","last_became_peered":"2023-12-20T04:47:20.674918+0000","last_unstale":"2023-12-20T04:47:22.063916+0000","last_undegraded":"2023-12-20T04:47:22.063916+0000","last_fullsized":"2023-12-20T04:47:22.063916+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:25:19.317934+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.b","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350528+0000","last_change":"2023-12-20T04:47:21.069723+0000","last_active":"2023-12-20T04:47:22.350528+0000","last_peered":"2023-12-20T04:47:22.350528+0000","last_clean":"2023-12-20T04:47:22.350528+0000","last_became_active":"2023-12-20T04:47:21.069463+0000","last_became_peered":"2023-12-20T04:47:21.069463+0000","last_unstale":"2023-12-20T04:47:22.350528+0000","last_undegraded":"2023-12-20T04:47:22.350528+0000","last_fullsized":"2023-12-20T04:47:22.350528+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:08.698549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.a","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.514846+0000","last_change":"2023-12-20T04:47:20.673163+0000","last_active":"2023-12-20T04:47:22.514846+0000","last_peered":"2023-12-20T04:47:22.514846+0000","last_clean":"2023-12-20T04:47:22.514846+0000","last_became_active":"2023-12-20T04:47:20.671975+0000","last_became_peered":"2023-12-20T04:47:20.671975+0000","last_unstale":"2023-12-20T04:47:22.514846+0000","last_undegraded":"2023-12-20T04:47:22.514846+0000","last_fullsized":"2023-12-20T04:47:22.514846+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:45:27.642645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.8","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517468+0000","last_change":"2023-12-20T04:47:21.077384+0000","last_active":"2023-12-20T04:47:22.517468+0000","last_peered":"2023-12-20T04:47:22.517468+0000","last_clean":"2023-12-20T04:47:22.517468+0000","last_became_active":"2023-12-20T04:47:21.077111+0000","last_became_peered":"2023-12-20T04:47:21.077111+0000","last_unstale":"2023-12-20T04:47:22.517468+0000","last_undegraded":"2023-12-20T04:47:22.517468+0000","last_fullsized":"2023-12-20T04:47:22.517468+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:42.621356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.9","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.063612+0000","last_change":"2023-12-20T04:47:20.668523+0000","last_active":"2023-12-20T04:47:22.063612+0000","last_peered":"2023-12-20T04:47:22.063612+0000","last_clean":"2023-12-20T04:47:22.063612+0000","last_became_active":"2023-12-20T04:47:20.668229+0000","last_became_peered":"2023-12-20T04:47:20.668229+0000","last_unstale":"2023-12-20T04:47:22.063612+0000","last_undegraded":"2023-12-20T04:47:22.063612+0000","last_fullsized":"2023-12-20T04:47:22.063612+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:13:33.535191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"17.9","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515318+0000","last_change":"2023-12-20T04:47:21.072261+0000","last_active":"2023-12-20T04:47:22.515318+0000","last_peered":"2023-12-20T04:47:22.515318+0000","last_clean":"2023-12-20T04:47:22.515318+0000","last_became_active":"2023-12-20T04:47:21.069346+0000","last_became_peered":"2023-12-20T04:47:21.069346+0000","last_unstale":"2023-12-20T04:47:22.515318+0000","last_undegraded":"2023-12-20T04:47:22.515318+0000","last_fullsized":"2023-12-20T04:47:22.515318+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:53:13.541202+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.8","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350489+0000","last_change":"2023-12-20T04:47:20.678668+0000","last_active":"2023-12-20T04:47:22.350489+0000","last_peered":"2023-12-20T04:47:22.350489+0000","last_clean":"2023-12-20T04:47:22.350489+0000","last_became_active":"2023-12-20T04:47:20.678366+0000","last_became_peered":"2023-12-20T04:47:20.678366+0000","last_unstale":"2023-12-20T04:47:22.350489+0000","last_undegraded":"2023-12-20T04:47:22.350489+0000","last_fullsized":"2023-12-20T04:47:22.350489+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:37:06.021492+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.6","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.516992+0000","last_change":"2023-12-20T04:47:21.097056+0000","last_active":"2023-12-20T04:47:22.516992+0000","last_peered":"2023-12-20T04:47:22.516992+0000","last_clean":"2023-12-20T04:47:22.516992+0000","last_became_active":"2023-12-20T04:47:21.096783+0000","last_became_peered":"2023-12-20T04:47:21.096783+0000","last_unstale":"2023-12-20T04:47:22.516992+0000","last_undegraded":"2023-12-20T04:47:22.516992+0000","last_fullsized":"2023-12-20T04:47:22.516992+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:34:15.389835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.7","version":"137'2","reported_seq":13,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515679+0000","last_change":"2023-12-20T04:47:20.676685+0000","last_active":"2023-12-20T04:47:22.515679+0000","last_peered":"2023-12-20T04:47:22.515679+0000","last_clean":"2023-12-20T04:47:22.515679+0000","last_became_active":"2023-12-20T04:47:20.676395+0000","last_became_peered":"2023-12-20T04:47:20.676395+0000","last_unstale":"2023-12-20T04:47:22.515679+0000","last_undegraded":"2023-12-20T04:47:22.515679+0000","last_fullsized":"2023-12-20T04:47:22.515679+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:14:03.537835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.7","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350980+0000","last_change":"2023-12-20T04:47:21.073764+0000","last_active":"2023-12-20T04:47:22.350980+0000","last_peered":"2023-12-20T04:47:22.350980+0000","last_clean":"2023-12-20T04:47:22.350980+0000","last_became_active":"2023-12-20T04:47:21.073455+0000","last_became_peered":"2023-12-20T04:47:21.073455+0000","last_unstale":"2023-12-20T04:47:22.350980+0000","last_undegraded":"2023-12-20T04:47:22.350980+0000","last_fullsized":"2023-12-20T04:47:22.350980+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:38.925591+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.6","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517136+0000","last_change":"2023-12-20T04:47:20.672782+0000","last_active":"2023-12-20T04:47:22.517136+0000","last_peered":"2023-12-20T04:47:22.517136+0000","last_clean":"2023-12-20T04:47:22.517136+0000","last_became_active":"2023-12-20T04:47:20.672477+0000","last_became_peered":"2023-12-20T04:47:20.672477+0000","last_unstale":"2023-12-20T04:47:22.517136+0000","last_undegraded":"2023-12-20T04:47:22.517136+0000","last_fullsized":"2023-12-20T04:47:22.517136+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:32:58.995426+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"17.4","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350857+0000","last_change":"2023-12-20T04:47:21.079477+0000","last_active":"2023-12-20T04:47:22.350857+0000","last_peered":"2023-12-20T04:47:22.350857+0000","last_clean":"2023-12-20T04:47:22.350857+0000","last_became_active":"2023-12-20T04:47:21.079200+0000","last_became_peered":"2023-12-20T04:47:21.079200+0000","last_unstale":"2023-12-20T04:47:22.350857+0000","last_undegraded":"2023-12-20T04:47:22.350857+0000","last_fullsized":"2023-12-20T04:47:22.350857+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:17:51.266380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.5","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350834+0000","last_change":"2023-12-20T04:47:20.678925+0000","last_active":"2023-12-20T04:47:22.350834+0000","last_peered":"2023-12-20T04:47:22.350834+0000","last_clean":"2023-12-20T04:47:22.350834+0000","last_became_active":"2023-12-20T04:47:20.678234+0000","last_became_peered":"2023-12-20T04:47:20.678234+0000","last_unstale":"2023-12-20T04:47:22.350834+0000","last_undegraded":"2023-12-20T04:47:22.350834+0000","last_fullsized":"2023-12-20T04:47:22.350834+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:50:29.834279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.5","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517128+0000","last_change":"2023-12-20T04:47:21.094972+0000","last_active":"2023-12-20T04:47:22.517128+0000","last_peered":"2023-12-20T04:47:22.517128+0000","last_clean":"2023-12-20T04:47:22.517128+0000","last_became_active":"2023-12-20T04:47:21.094696+0000","last_became_peered":"2023-12-20T04:47:21.094696+0000","last_unstale":"2023-12-20T04:47:22.517128+0000","last_undegraded":"2023-12-20T04:47:22.517128+0000","last_fullsized":"2023-12-20T04:47:22.517128+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:35:04.579810+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"16.4","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515830+0000","last_change":"2023-12-20T04:47:20.672063+0000","last_active":"2023-12-20T04:47:22.515830+0000","last_peered":"2023-12-20T04:47:22.515830+0000","last_clean":"2023-12-20T04:47:22.515830+0000","last_became_active":"2023-12-20T04:47:20.671626+0000","last_became_peered":"2023-12-20T04:47:20.671626+0000","last_unstale":"2023-12-20T04:47:22.515830+0000","last_undegraded":"2023-12-20T04:47:22.515830+0000","last_fullsized":"2023-12-20T04:47:22.515830+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:28:24.189539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"17.2","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066170+0000","last_change":"2023-12-20T04:47:21.069595+0000","last_active":"2023-12-20T04:47:22.066170+0000","last_peered":"2023-12-20T04:47:22.066170+0000","last_clean":"2023-12-20T04:47:22.066170+0000","last_became_active":"2023-12-20T04:47:21.069307+0000","last_became_peered":"2023-12-20T04:47:21.069307+0000","last_unstale":"2023-12-20T04:47:22.066170+0000","last_undegraded":"2023-12-20T04:47:22.066170+0000","last_fullsized":"2023-12-20T04:47:22.066170+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:40:35.208765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.3","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067432+0000","last_change":"2023-12-20T04:47:20.669617+0000","last_active":"2023-12-20T04:47:22.067432+0000","last_peered":"2023-12-20T04:47:22.067432+0000","last_clean":"2023-12-20T04:47:22.067432+0000","last_became_active":"2023-12-20T04:47:20.669305+0000","last_became_peered":"2023-12-20T04:47:20.669305+0000","last_unstale":"2023-12-20T04:47:22.067432+0000","last_undegraded":"2023-12-20T04:47:22.067432+0000","last_fullsized":"2023-12-20T04:47:22.067432+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:45:27.642645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"17.3","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066949+0000","last_change":"2023-12-20T04:47:21.075733+0000","last_active":"2023-12-20T04:47:22.066949+0000","last_peered":"2023-12-20T04:47:22.066949+0000","last_clean":"2023-12-20T04:47:22.066949+0000","last_became_active":"2023-12-20T04:47:21.075451+0000","last_became_peered":"2023-12-20T04:47:21.075451+0000","last_unstale":"2023-12-20T04:47:22.066949+0000","last_undegraded":"2023-12-20T04:47:22.066949+0000","last_fullsized":"2023-12-20T04:47:22.066949+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T04:53:32.472254+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"16.2","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.066254+0000","last_change":"2023-12-20T04:47:20.669633+0000","last_active":"2023-12-20T04:47:22.066254+0000","last_peered":"2023-12-20T04:47:22.066254+0000","last_clean":"2023-12-20T04:47:22.066254+0000","last_became_active":"2023-12-20T04:47:20.669368+0000","last_became_peered":"2023-12-20T04:47:20.669368+0000","last_unstale":"2023-12-20T04:47:22.066254+0000","last_undegraded":"2023-12-20T04:47:22.066254+0000","last_fullsized":"2023-12-20T04:47:22.066254+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:36:10.755454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"17.0","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.517496+0000","last_change":"2023-12-20T04:47:21.087552+0000","last_active":"2023-12-20T04:47:22.517496+0000","last_peered":"2023-12-20T04:47:22.517496+0000","last_clean":"2023-12-20T04:47:22.517496+0000","last_became_active":"2023-12-20T04:47:21.087267+0000","last_became_peered":"2023-12-20T04:47:21.087267+0000","last_unstale":"2023-12-20T04:47:22.517496+0000","last_undegraded":"2023-12-20T04:47:22.517496+0000","last_fullsized":"2023-12-20T04:47:22.517496+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:11:44.960894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"16.1","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.514887+0000","last_change":"2023-12-20T04:47:20.672325+0000","last_active":"2023-12-20T04:47:22.514887+0000","last_peered":"2023-12-20T04:47:22.514887+0000","last_clean":"2023-12-20T04:47:22.514887+0000","last_became_active":"2023-12-20T04:47:20.671957+0000","last_became_peered":"2023-12-20T04:47:20.671957+0000","last_unstale":"2023-12-20T04:47:22.514887+0000","last_undegraded":"2023-12-20T04:47:22.514887+0000","last_fullsized":"2023-12-20T04:47:22.514887+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:54:01.405424+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"17.1","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350555+0000","last_change":"2023-12-20T04:47:21.086535+0000","last_active":"2023-12-20T04:47:22.350555+0000","last_peered":"2023-12-20T04:47:22.350555+0000","last_clean":"2023-12-20T04:47:22.350555+0000","last_became_active":"2023-12-20T04:47:21.086252+0000","last_became_peered":"2023-12-20T04:47:21.086252+0000","last_unstale":"2023-12-20T04:47:22.350555+0000","last_undegraded":"2023-12-20T04:47:22.350555+0000","last_fullsized":"2023-12-20T04:47:22.350555+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:44.596329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"16.0","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.350577+0000","last_change":"2023-12-20T04:47:20.678744+0000","last_active":"2023-12-20T04:47:22.350577+0000","last_peered":"2023-12-20T04:47:22.350577+0000","last_clean":"2023-12-20T04:47:22.350577+0000","last_became_active":"2023-12-20T04:47:20.678492+0000","last_became_peered":"2023-12-20T04:47:20.678492+0000","last_unstale":"2023-12-20T04:47:22.350577+0000","last_undegraded":"2023-12-20T04:47:22.350577+0000","last_fullsized":"2023-12-20T04:47:22.350577+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:15.087308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"17.10","version":"0'0","reported_seq":10,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.515243+0000","last_change":"2023-12-20T04:47:21.073845+0000","last_active":"2023-12-20T04:47:22.515243+0000","last_peered":"2023-12-20T04:47:22.515243+0000","last_clean":"2023-12-20T04:47:22.515243+0000","last_became_active":"2023-12-20T04:47:21.073426+0000","last_became_peered":"2023-12-20T04:47:21.073426+0000","last_unstale":"2023-12-20T04:47:22.515243+0000","last_undegraded":"2023-12-20T04:47:22.515243+0000","last_fullsized":"2023-12-20T04:47:22.515243+0000","mapping_epoch":136,"log_start":"0'0","ondisk_log_start":"0'0","created":136,"last_epoch_clean":137,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:20.649446+0000","last_clean_scrub_stamp":"2023-12-20T04:47:20.649446+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:50:30.836615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"16.11","version":"0'0","reported_seq":11,"reported_epoch":138,"state":"active+clean","last_fresh":"2023-12-20T04:47:22.067269+0000","last_change":"2023-12-20T04:47:20.673484+0000","last_active":"2023-12-20T04:47:22.067269+0000","last_peered":"2023-12-20T04:47:22.067269+0000","last_clean":"2023-12-20T04:47:22.067269+0000","last_became_active":"2023-12-20T04:47:20.673216+0000","last_became_peered":"2023-12-20T04:47:20.673216+0000","last_unstale":"2023-12-20T04:47:22.067269+0000","last_undegraded":"2023-12-20T04:47:22.067269+0000","last_fullsized":"2023-12-20T04:47:22.067269+0000","mapping_epoch":135,"log_start":"0'0","ondisk_log_start":"0'0","created":135,"last_epoch_clean":136,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:47:19.647110+0000","last_clean_scrub_stamp":"2023-12-20T04:47:19.647110+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:13:33.535191+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"126'3102","reported_seq":3541,"reported_epoch":137,"state":"active+clean","last_fresh":"2023-12-20T04:47:21.066883+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T04:47:21.066883+0000","last_peered":"2023-12-20T04:47:21.066883+0000","last_clean":"2023-12-20T04:47:21.066883+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T04:47:21.066883+0000","last_undegraded":"2023-12-20T04:47:21.066883+0000","last_fullsized":"2023-12-20T04:47:21.066883+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3102,"log_dups_size":0,"ondisk_log_size":3102,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1021,"num_read_kb":1995,"num_write":341,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":17,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":16,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1021,"num_read_kb":1995,"num_write":341,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3102,"ondisk_log_size":3102,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575073,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33308,"kb_used_data":2260,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338532,"statfs":{"total":96636764160,"available":96602656768,"internally_reserved":0,"allocated":2314240,"data_stored":1406363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.67100000000000004,"15min":0.70299999999999996},"min":{"1min":0.42099999999999999,"5min":0.29599999999999999,"15min":0.29599999999999999},"max":{"1min":0.98999999999999999,"5min":1.048,"15min":1.0600000000000001},"last":0.70599999999999996},{"interface":"front","average":{"1min":0.747,"5min":0.69099999999999995,"15min":0.71099999999999997},"min":{"1min":0.47599999999999998,"5min":0.36699999999999999,"15min":0.36699999999999999},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.1259999999999999},"last":0.59799999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.70399999999999996,"15min":0.72699999999999998},"min":{"1min":0.59499999999999997,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":1.1000000000000001,"5min":1.1000000000000001,"15min":1.1000000000000001},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.70699999999999996,"15min":0.73299999999999998},"min":{"1min":0.53600000000000003,"5min":0.377,"15min":0.377},"max":{"1min":1.046,"5min":1.046,"15min":1.1080000000000001},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.68700000000000006,"15min":0.71099999999999997},"min":{"1min":0.45000000000000001,"5min":0.379,"15min":0.379},"max":{"1min":1.1479999999999999,"5min":1.1479999999999999,"15min":1.1479999999999999},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.73699999999999999,"15min":0.75},"min":{"1min":0.48399999999999999,"5min":0.48399999999999999,"15min":0.46899999999999997},"max":{"1min":1.169,"5min":1.169,"15min":1.169},"last":0.88400000000000001}]},{"osd":3,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.70299999999999996,"15min":0.72399999999999998},"min":{"1min":0.495,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.74299999999999999,"15min":0.75900000000000001},"min":{"1min":0.52600000000000002,"5min":0.50600000000000001,"15min":0.47599999999999998},"max":{"1min":1.218,"5min":1.218,"15min":1.218},"last":0.76900000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.59699999999999998,"15min":0.59399999999999997},"min":{"1min":0.45900000000000002,"5min":0.37,"15min":0.37},"max":{"1min":0.95099999999999996,"5min":0.95099999999999996,"15min":1.0109999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.61899999999999999,"15min":0.62},"min":{"1min":0.45200000000000001,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.96799999999999997,"5min":0.96799999999999997,"15min":1.0469999999999999},"last":0.79500000000000004}]},{"osd":5,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.66300000000000003,"15min":0.65200000000000002},"min":{"1min":0.51700000000000002,"5min":0.40500000000000003,"15min":0.40400000000000003},"max":{"1min":1.093,"5min":1.093,"15min":1.093},"last":0.91700000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.68799999999999994,"15min":0.67500000000000004},"min":{"1min":0.44,"5min":0.436,"15min":0.432},"max":{"1min":1.0680000000000001,"5min":1.0680000000000001,"15min":1.071},"last":0.89600000000000002}]},{"osd":6,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.73799999999999999,"15min":0.73199999999999998},"min":{"1min":0.61499999999999999,"5min":0.46800000000000003,"15min":0.443},"max":{"1min":1.026,"5min":1.026,"15min":1.1279999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.70899999999999996,"15min":0.70599999999999996},"min":{"1min":0.55700000000000005,"5min":0.46100000000000002,"15min":0.44400000000000001},"max":{"1min":0.97899999999999998,"5min":1.0309999999999999,"15min":1.125},"last":0.86699999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575073,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36808,"kb_used_data":1664,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335032,"statfs":{"total":96636764160,"available":96599072768,"internally_reserved":0,"allocated":1703936,"data_stored":809121,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.69599999999999995,"15min":0.71499999999999997},"min":{"1min":0.39400000000000002,"5min":0.39400000000000002,"15min":0.27100000000000002},"max":{"1min":0.996,"5min":1.1140000000000001,"15min":1.2649999999999999},"last":0.77300000000000002},{"interface":"front","average":{"1min":0.69299999999999995,"5min":0.69999999999999996,"15min":0.71599999999999997},"min":{"1min":0.36799999999999999,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.92000000000000004,"5min":1.1200000000000001,"15min":1.1200000000000001},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.67700000000000005,"15min":0.69999999999999996},"min":{"1min":0.42599999999999999,"5min":0.36899999999999999,"15min":0.34799999999999998},"max":{"1min":0.93300000000000005,"5min":1.016,"15min":1.016},"last":0.79600000000000004},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.70899999999999996,"15min":0.72699999999999998},"min":{"1min":0.54400000000000004,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.95999999999999996,"5min":1.1879999999999999,"15min":1.1879999999999999},"last":0.72699999999999998}]},{"osd":2,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.71499999999999997,"5min":0.747,"15min":0.76100000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.40899999999999997},"max":{"1min":0.90700000000000003,"5min":1.0580000000000001,"15min":1.216},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.70899999999999996,"15min":0.73299999999999998},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":0.93100000000000005,"5min":1.141,"15min":1.141},"last":0.68000000000000005}]},{"osd":3,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.74099999999999999,"15min":0.75900000000000001},"min":{"1min":0.53400000000000003,"5min":0.40300000000000002,"15min":0.40300000000000002},"max":{"1min":0.96899999999999997,"5min":1.093,"15min":1.093},"last":0.746},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.71599999999999997,"15min":0.73199999999999998},"min":{"1min":0.48299999999999998,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.93100000000000005,"5min":1.087,"15min":1.087},"last":0.63100000000000001}]},{"osd":4,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.624,"15min":0.61299999999999999},"min":{"1min":0.42099999999999999,"5min":0.379,"15min":0.36399999999999999},"max":{"1min":0.81999999999999995,"5min":0.97099999999999997,"15min":1.1040000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.61599999999999999,"5min":0.66300000000000003,"15min":0.624},"min":{"1min":0.40799999999999997,"5min":0.378,"15min":0.36899999999999999},"max":{"1min":0.82499999999999996,"5min":0.98499999999999999,"15min":0.995},"last":0.60399999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.61099999999999999,"5min":0.63100000000000001,"15min":0.61899999999999999},"min":{"1min":0.41399999999999998,"5min":0.307,"15min":0.307},"max":{"1min":0.874,"5min":0.93799999999999994,"15min":1.1519999999999999},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.66300000000000003,"5min":0.70099999999999996,"15min":0.67700000000000005},"min":{"1min":0.46300000000000002,"5min":0.42399999999999999,"15min":0.42399999999999999},"max":{"1min":0.92300000000000004,"5min":1.0169999999999999,"15min":1.306},"last":0.58299999999999996}]},{"osd":7,"last update":"Wed Dec 20 04:47:08 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.71799999999999997,"15min":0.70599999999999996},"min":{"1min":0.47799999999999998,"5min":0.47799999999999998,"15min":0.40400000000000003},"max":{"1min":0.86099999999999999,"5min":1.0129999999999999,"15min":1.0149999999999999},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.71899999999999997,"15min":0.69799999999999995},"min":{"1min":0.38400000000000001,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":0.84599999999999997,"5min":1.131,"15min":1.131},"last":0.85499999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575074,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36824,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335016,"statfs":{"total":96636764160,"available":96599056384,"internally_reserved":0,"allocated":1720320,"data_stored":815983,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.69799999999999995,"15min":0.70399999999999996},"min":{"1min":0.42199999999999999,"5min":0.36199999999999999,"15min":0.29999999999999999},"max":{"1min":0.996,"5min":0.996,"15min":1.05},"last":0.59299999999999997},{"interface":"front","average":{"1min":0.63300000000000001,"5min":0.67500000000000004,"15min":0.70699999999999996},"min":{"1min":0.374,"5min":0.35899999999999999,"15min":0.34300000000000003},"max":{"1min":0.83799999999999997,"5min":0.96899999999999997,"15min":0.98599999999999999},"last":0.72699999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.69499999999999995,"15min":0.70699999999999996},"min":{"1min":0.48499999999999999,"5min":0.443,"15min":0.441},"max":{"1min":0.91300000000000003,"5min":0.98399999999999999,"15min":1.0600000000000001},"last":0.64100000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.70799999999999996,"15min":0.72999999999999998},"min":{"1min":0.54300000000000004,"5min":0.437,"15min":0.38400000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.0309999999999999},"last":0.66900000000000004}]},{"osd":2,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.71999999999999997,"15min":0.73199999999999998},"min":{"1min":0.41599999999999998,"5min":0.35699999999999998,"15min":0.35699999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.72099999999999997,"15min":0.73099999999999998},"min":{"1min":0.45500000000000002,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.0469999999999999,"5min":1.0469999999999999,"15min":1.0469999999999999},"last":0.71399999999999997}]},{"osd":3,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.54700000000000004,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0780000000000001,"5min":1.0780000000000001,"15min":1.0780000000000001},"last":0.51800000000000002},{"interface":"front","average":{"1min":0.745,"5min":0.73799999999999999,"15min":0.755},"min":{"1min":0.47499999999999998,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.92400000000000004,"5min":1.008,"15min":1.0469999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.61799999999999999,"15min":0.61399999999999999},"min":{"1min":0.44800000000000001,"5min":0.42099999999999999,"15min":0.36899999999999999},"max":{"1min":0.89500000000000002,"5min":1.0069999999999999,"15min":1.0069999999999999},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.60699999999999998,"5min":0.63,"15min":0.629},"min":{"1min":0.43099999999999999,"5min":0.40300000000000002,"15min":0.39600000000000002},"max":{"1min":0.81000000000000005,"5min":0.95099999999999996,"15min":1.077},"last":0.58099999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.68600000000000005,"15min":0.68000000000000005},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90600000000000003,"5min":0.96599999999999997,"15min":1.056},"last":0.69099999999999995},{"interface":"front","average":{"1min":0.61299999999999999,"5min":0.63,"15min":0.63},"min":{"1min":0.36299999999999999,"5min":0.36299999999999999,"15min":0.36299999999999999},"max":{"1min":0.90800000000000003,"5min":0.90800000000000003,"15min":0.999},"last":0.76500000000000001}]},{"osd":7,"last update":"Wed Dec 20 04:47:01 2023","interfaces":[{"interface":"back","average":{"1min":0.65200000000000002,"5min":0.67800000000000005,"15min":0.68799999999999994},"min":{"1min":0.46899999999999997,"5min":0.45300000000000001,"15min":0.442},"max":{"1min":0.88400000000000001,"5min":1.006,"15min":1.141},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.746,"15min":0.75},"min":{"1min":0.55000000000000004,"5min":0.48999999999999999,"15min":0.47899999999999998},"max":{"1min":0.92500000000000004,"5min":0.95799999999999996,"15min":1.0780000000000001},"last":0.73999999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575075,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58960,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312880,"statfs":{"total":96636764160,"available":96576389120,"internally_reserved":0,"allocated":1712128,"data_stored":815379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.65900000000000003,"15min":0.69099999999999995},"min":{"1min":0.38300000000000001,"5min":0.28699999999999998,"15min":0.28699999999999998},"max":{"1min":0.93400000000000005,"5min":1.0129999999999999,"15min":1.5509999999999999},"last":0.66500000000000004},{"interface":"front","average":{"1min":0.66800000000000004,"5min":0.67500000000000004,"15min":0.70299999999999996},"min":{"1min":0.36699999999999999,"5min":0.34000000000000002,"15min":0.34000000000000002},"max":{"1min":0.92100000000000004,"5min":0.97799999999999998,"15min":1.633},"last":0.64200000000000002}]},{"osd":1,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68400000000000005,"5min":0.69199999999999995,"15min":0.71299999999999997},"min":{"1min":0.42599999999999999,"5min":0.315,"15min":0.315},"max":{"1min":0.91300000000000003,"5min":0.96099999999999997,"15min":1.798},"last":0.81000000000000005},{"interface":"front","average":{"1min":0.70899999999999996,"5min":0.70999999999999996,"15min":0.73799999999999999},"min":{"1min":0.42499999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.89100000000000001,"5min":1.036,"15min":2.2930000000000001},"last":0.70799999999999996}]},{"osd":2,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.70099999999999996,"5min":0.72299999999999998,"15min":0.73999999999999999},"min":{"1min":0.46700000000000003,"5min":0.40500000000000003,"15min":0.36199999999999999},"max":{"1min":0.93999999999999995,"5min":1.0209999999999999,"15min":1.728},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.71699999999999997,"15min":0.74199999999999999},"min":{"1min":0.503,"5min":0.46899999999999997,"15min":0.38700000000000001},"max":{"1min":0.96299999999999997,"5min":1.0289999999999999,"15min":1.6619999999999999},"last":0.75900000000000001}]},{"osd":3,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.68100000000000005,"5min":0.69699999999999995,"15min":0.72199999999999998},"min":{"1min":0.41199999999999998,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.85199999999999998,"5min":1.004,"15min":1.5760000000000001},"last":0.72099999999999997},{"interface":"front","average":{"1min":0.71499999999999997,"5min":0.73099999999999998,"15min":0.754},"min":{"1min":0.48999999999999999,"5min":0.372,"15min":0.372},"max":{"1min":0.997,"5min":0.997,"15min":1.7649999999999999},"last":0.74299999999999999}]},{"osd":5,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.57399999999999995,"5min":0.62,"15min":0.60999999999999999},"min":{"1min":0.39900000000000002,"5min":0.36599999999999999,"15min":0.36099999999999999},"max":{"1min":0.75700000000000001,"5min":1.1120000000000001,"15min":1.456},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.56899999999999995,"5min":0.61299999999999999,"15min":0.61899999999999999},"min":{"1min":0.39900000000000002,"5min":0.39700000000000002,"15min":0.33800000000000002},"max":{"1min":0.754,"5min":1.077,"15min":1.589},"last":0.44400000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.58799999999999997,"5min":0.65500000000000003,"15min":0.64400000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.35999999999999999},"max":{"1min":0.71699999999999997,"5min":1.006,"15min":1.6739999999999999},"last":0.48799999999999999},{"interface":"front","average":{"1min":0.629,"5min":0.68400000000000005,"15min":0.68400000000000005},"min":{"1min":0.47599999999999998,"5min":0.41999999999999998,"15min":0.41699999999999998},"max":{"1min":0.81499999999999995,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.59399999999999997}]},{"osd":7,"last update":"Wed Dec 20 04:46:58 2023","interfaces":[{"interface":"back","average":{"1min":0.66100000000000003,"5min":0.67700000000000005,"15min":0.68400000000000005},"min":{"1min":0.47199999999999998,"5min":0.46500000000000002,"15min":0.38400000000000001},"max":{"1min":0.86399999999999999,"5min":1.036,"15min":1.655},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.68000000000000005,"5min":0.71599999999999997,"15min":0.72299999999999998},"min":{"1min":0.503,"5min":0.432,"15min":0.42399999999999999},"max":{"1min":0.88,"5min":1.052,"15min":1.125},"last":0.68600000000000005}]}]},{"osd":3,"up_from":13,"seq":55834575074,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":36832,"kb_used_data":1688,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94335008,"statfs":{"total":96636764160,"available":96599048192,"internally_reserved":0,"allocated":1728512,"data_stored":817013,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.44400000000000001,"5min":0.42799999999999999,"15min":0.45100000000000001},"min":{"1min":0.32300000000000001,"5min":0.26100000000000001,"15min":0.23200000000000001},"max":{"1min":0.61099999999999999,"5min":0.748,"15min":0.875},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.497,"5min":0.48299999999999998,"15min":0.50900000000000001},"min":{"1min":0.32200000000000001,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.77300000000000002,"5min":0.77600000000000002,"15min":0.80800000000000005},"last":0.41299999999999998}]},{"osd":1,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.46100000000000002,"15min":0.46800000000000003},"min":{"1min":0.35299999999999998,"5min":0.29299999999999998,"15min":0.27400000000000002},"max":{"1min":0.56799999999999995,"5min":0.73999999999999999,"15min":0.73999999999999999},"last":0.435},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.52600000000000002,"15min":0.54800000000000004},"min":{"1min":0.41099999999999998,"5min":0.35499999999999998,"15min":0.28399999999999997},"max":{"1min":0.65900000000000003,"5min":0.74399999999999999,"15min":0.88900000000000001},"last":0.375}]},{"osd":2,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.55700000000000005,"5min":0.56100000000000005,"15min":0.57599999999999996},"min":{"1min":0.34599999999999997,"5min":0.34599999999999997,"15min":0.318},"max":{"1min":0.74199999999999999,"5min":0.80700000000000005,"15min":0.88700000000000001},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.55600000000000005,"5min":0.54500000000000004,"15min":0.55100000000000005},"min":{"1min":0.433,"5min":0.316,"15min":0.316},"max":{"1min":0.77000000000000002,"5min":0.81799999999999995,"15min":0.93300000000000005},"last":0.51800000000000002}]},{"osd":4,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.73099999999999998,"15min":0.77400000000000002},"min":{"1min":0.59399999999999997,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":1.016,"5min":1.0880000000000001,"15min":1.131},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.77700000000000002,"5min":0.75600000000000001,"15min":0.79200000000000004},"min":{"1min":0.61099999999999999,"5min":0.50700000000000001,"15min":0.41599999999999998},"max":{"1min":0.998,"5min":1.198,"15min":1.4059999999999999},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.78500000000000003,"5min":0.76800000000000002,"15min":0.80000000000000004},"min":{"1min":0.60599999999999998,"5min":0.498,"15min":0.46500000000000002},"max":{"1min":1.002,"5min":1.181,"15min":1.3300000000000001},"last":0.94099999999999995},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.75,"15min":0.77700000000000002},"min":{"1min":0.56899999999999995,"5min":0.503,"15min":0.38400000000000001},"max":{"1min":0.97799999999999998,"5min":1.1060000000000001,"15min":1.252},"last":0.76500000000000001}]},{"osd":6,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.746,"15min":0.78600000000000003},"min":{"1min":0.61799999999999999,"5min":0.49099999999999999,"15min":0.45600000000000002},"max":{"1min":1.046,"5min":1.046,"15min":1.2050000000000001},"last":0.999},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.76800000000000002,"15min":0.80400000000000005},"min":{"1min":0.56699999999999995,"5min":0.436,"15min":0.436},"max":{"1min":1.052,"5min":1.133,"15min":1.2929999999999999},"last":0.82499999999999996}]},{"osd":7,"last update":"Wed Dec 20 04:46:53 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.751,"15min":0.78600000000000003},"min":{"1min":0.65000000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.96399999999999997,"5min":1.0249999999999999,"15min":1.1699999999999999},"last":0.92200000000000004},{"interface":"front","average":{"1min":0.80100000000000005,"5min":0.76900000000000002,"15min":0.80800000000000005},"min":{"1min":0.622,"5min":0.45900000000000002,"15min":0.45900000000000002},"max":{"1min":0.95599999999999996,"5min":1.145,"15min":1.29},"last":0.80200000000000005}]}]},{"osd":2,"up_from":13,"seq":55834575073,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":58960,"kb_used_data":1672,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312880,"statfs":{"total":96636764160,"available":96576389120,"internally_reserved":0,"allocated":1712128,"data_stored":815379,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.45000000000000001,"5min":0.432,"15min":0.44},"min":{"1min":0.307,"5min":0.26200000000000001,"15min":0.216},"max":{"1min":0.64400000000000002,"5min":0.86699999999999999,"15min":0.86699999999999999},"last":0.41699999999999998},{"interface":"front","average":{"1min":0.46300000000000002,"5min":0.46999999999999997,"15min":0.48599999999999999},"min":{"1min":0.32500000000000001,"5min":0.26400000000000001,"15min":0.26400000000000001},"max":{"1min":0.60399999999999998,"5min":0.92200000000000004,"15min":1.0589999999999999},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.46400000000000002,"5min":0.45600000000000002,"15min":0.46999999999999997},"min":{"1min":0.33100000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":0.64400000000000002,"5min":0.78100000000000003,"15min":0.876},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.53000000000000003,"5min":0.505,"15min":0.52200000000000002},"min":{"1min":0.36199999999999999,"5min":0.32100000000000001,"15min":0.32000000000000001},"max":{"1min":0.79300000000000004,"5min":1.1759999999999999,"15min":1.1759999999999999},"last":0.44}]},{"osd":3,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.57499999999999996,"15min":0.59799999999999998},"min":{"1min":0.49199999999999999,"5min":0.32600000000000001,"15min":0.32600000000000001},"max":{"1min":0.76800000000000002,"5min":0.89800000000000002,"15min":0.93899999999999995},"last":0.59099999999999997},{"interface":"front","average":{"1min":0.65400000000000003,"5min":0.625,"15min":0.63800000000000001},"min":{"1min":0.47299999999999998,"5min":0.40799999999999997,"15min":0.40799999999999997},"max":{"1min":0.95299999999999996,"5min":0.95299999999999996,"15min":0.95599999999999996},"last":0.54800000000000004}]},{"osd":4,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.74299999999999999,"15min":0.77200000000000002},"min":{"1min":0.56000000000000005,"5min":0.46500000000000002,"15min":0.46500000000000002},"max":{"1min":1.046,"5min":1.0760000000000001,"15min":1.145},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.73299999999999998,"15min":0.76400000000000001},"min":{"1min":0.45900000000000002,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91600000000000004,"5min":1.361,"15min":1.361},"last":0.85799999999999998}]},{"osd":5,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72899999999999998,"15min":0.76200000000000001},"min":{"1min":0.53100000000000003,"5min":0.47599999999999998,"15min":0.47599999999999998},"max":{"1min":0.94499999999999995,"5min":1.242,"15min":1.242},"last":0.75},{"interface":"front","average":{"1min":0.79100000000000004,"5min":0.755,"15min":0.79000000000000004},"min":{"1min":0.57699999999999996,"5min":0.495,"15min":0.48499999999999999},"max":{"1min":1.0980000000000001,"5min":1.1779999999999999,"15min":1.1779999999999999},"last":0.878}]},{"osd":6,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75900000000000001,"15min":0.78400000000000003},"min":{"1min":0.502,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.97299999999999998,"5min":1.323,"15min":1.323},"last":0.91000000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.74299999999999999,"15min":0.78100000000000003},"min":{"1min":0.58999999999999997,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.90200000000000002,"5min":1.1060000000000001,"15min":1.1060000000000001},"last":0.84299999999999997}]},{"osd":7,"last update":"Wed Dec 20 04:47:06 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.77100000000000002,"15min":0.80300000000000005},"min":{"1min":0.54800000000000004,"5min":0.44600000000000001,"15min":0.44600000000000001},"max":{"1min":1.0669999999999999,"5min":1.3919999999999999,"15min":1.3919999999999999},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.79700000000000004,"15min":0.83699999999999997},"min":{"1min":0.64500000000000002,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.079,"5min":1.3280000000000001,"15min":1.3280000000000001},"last":0.871}]}]},{"osd":1,"up_from":13,"seq":55834575075,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":50776,"kb_used_data":1680,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94321064,"statfs":{"total":96636764160,"available":96584769536,"internally_reserved":0,"allocated":1720320,"data_stored":815503,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.43099999999999999,"15min":0.439},"min":{"1min":0.30099999999999999,"5min":0.23400000000000001,"15min":0.22600000000000001},"max":{"1min":0.67300000000000004,"5min":0.84499999999999997,"15min":0.84499999999999997},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.47199999999999998,"5min":0.48999999999999999,"15min":0.49299999999999999},"min":{"1min":0.34899999999999998,"5min":0.26100000000000001,"15min":0.26100000000000001},"max":{"1min":0.621,"5min":0.86199999999999999,"15min":0.92100000000000004},"last":0.83299999999999996}]},{"osd":2,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.50600000000000001,"5min":0.49299999999999999,"15min":0.51100000000000001},"min":{"1min":0.39400000000000002,"5min":0.29199999999999998,"15min":0.29199999999999998},"max":{"1min":0.69599999999999995,"5min":0.72799999999999998,"15min":1.091},"last":0.70199999999999996},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.53700000000000003,"15min":0.54500000000000004},"min":{"1min":0.41099999999999998,"5min":0.30299999999999999,"15min":0.30299999999999999},"max":{"1min":0.77600000000000002,"5min":0.84699999999999998,"15min":0.875},"last":0.66900000000000004}]},{"osd":3,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.56100000000000005,"5min":0.58199999999999996,"15min":0.59299999999999997},"min":{"1min":0.45900000000000002,"5min":0.39700000000000002,"15min":0.39700000000000002},"max":{"1min":0.74299999999999999,"5min":0.91000000000000003,"15min":0.96499999999999997},"last":0.62},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.51400000000000001,"15min":0.52700000000000002},"min":{"1min":0.36599999999999999,"5min":0.33300000000000002,"15min":0.29699999999999999},"max":{"1min":0.67600000000000005,"5min":0.82399999999999995,"15min":1.2410000000000001},"last":0.59299999999999997}]},{"osd":4,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.72099999999999997,"5min":0.72099999999999997,"15min":0.75600000000000001},"min":{"1min":0.53400000000000003,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.89400000000000002,"5min":1.036,"15min":1.27},"last":0.85399999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.77200000000000002,"15min":0.77900000000000003},"min":{"1min":0.54200000000000004,"5min":0.49399999999999999,"15min":0.434},"max":{"1min":0.93000000000000005,"5min":1.1819999999999999,"15min":1.1819999999999999},"last":0.93500000000000005}]},{"osd":5,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.755,"5min":0.748,"15min":0.77200000000000002},"min":{"1min":0.54600000000000004,"5min":0.49399999999999999,"15min":0.49399999999999999},"max":{"1min":1.089,"5min":1.089,"15min":1.3420000000000001},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.53700000000000003,"5min":0.48399999999999999,"15min":0.41299999999999998},"max":{"1min":1.0449999999999999,"5min":1.129,"15min":1.2450000000000001},"last":0.94999999999999996}]},{"osd":6,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.76900000000000002,"15min":0.78500000000000003},"min":{"1min":0.52300000000000002,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":1.044,"5min":1.232,"15min":1.232},"last":0.98599999999999999},{"interface":"front","average":{"1min":0.745,"5min":0.74199999999999999,"15min":0.77900000000000003},"min":{"1min":0.621,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.90900000000000003,"5min":1.0289999999999999,"15min":1.252},"last":0.87}]},{"osd":7,"last update":"Wed Dec 20 04:47:09 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.80000000000000004,"15min":0.82599999999999996},"min":{"1min":0.58299999999999996,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.306},"last":0.96799999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.79600000000000004,"15min":0.83099999999999996},"min":{"1min":0.61199999999999999,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0700000000000001,"5min":1.071,"15min":1.3440000000000001},"last":1.006}]}]},{"osd":0,"up_from":13,"seq":55834575074,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29224,"kb_used_data":2272,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342616,"statfs":{"total":96636764160,"available":96606838784,"internally_reserved":0,"allocated":2326528,"data_stored":1407423,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.48299999999999998,"5min":0.44,"15min":0.44600000000000001},"min":{"1min":0.28299999999999997,"5min":0.23499999999999999,"15min":0.218},"max":{"1min":0.68300000000000005,"5min":0.83699999999999997,"15min":2.5640000000000001},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.51200000000000001,"5min":0.5,"15min":0.53400000000000003},"min":{"1min":0.31900000000000001,"5min":0.26800000000000002,"15min":0.26800000000000002},"max":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":7.8860000000000001},"last":0.40000000000000002}]},{"osd":2,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.53000000000000003,"5min":0.503,"15min":0.51600000000000001},"min":{"1min":0.41499999999999998,"5min":0.28299999999999997,"15min":0.28299999999999997},"max":{"1min":0.71499999999999997,"5min":0.76300000000000001,"15min":2.528},"last":0.41499999999999998},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.54500000000000004,"15min":0.55000000000000004},"min":{"1min":0.39700000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.82299999999999995,"5min":0.86399999999999999,"15min":0.86399999999999999},"last":0.441}]},{"osd":3,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.55400000000000005,"15min":0.53500000000000003},"min":{"1min":0.432,"5min":0.309,"15min":0.309},"max":{"1min":1.127,"5min":3.0350000000000001,"15min":3.1000000000000001},"last":0.34699999999999998},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58699999999999997,"15min":0.623},"min":{"1min":0.46300000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79300000000000004,"5min":0.80300000000000005,"15min":10.49},"last":0.48999999999999999}]},{"osd":4,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.78900000000000003,"5min":0.75600000000000001,"15min":0.78400000000000003},"min":{"1min":0.505,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0109999999999999,"5min":1.321,"15min":3.0659999999999998},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.78000000000000003,"15min":0.80600000000000005},"min":{"1min":0.54400000000000004,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.165,"5min":1.3080000000000001,"15min":5.2460000000000004},"last":0.95399999999999996}]},{"osd":5,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.77200000000000002,"15min":0.79300000000000004},"min":{"1min":0.58799999999999997,"5min":0.48399999999999999,"15min":0.48399999999999999},"max":{"1min":1.2509999999999999,"5min":1.2509999999999999,"15min":1.3109999999999999},"last":0.82399999999999995},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.78500000000000003,"15min":0.79500000000000004},"min":{"1min":0.53400000000000003,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":1.2130000000000001,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.98699999999999999}]},{"osd":6,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.78600000000000003,"15min":0.81699999999999995},"min":{"1min":0.51900000000000002,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.0569999999999999,"5min":1.345,"15min":5.2839999999999998},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.79200000000000004,"15min":0.80600000000000005},"min":{"1min":0.55800000000000005,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.111,"5min":1.2529999999999999,"15min":1.306},"last":0.78800000000000003}]},{"osd":7,"last update":"Wed Dec 20 04:46:59 2023","interfaces":[{"interface":"back","average":{"1min":0.87,"5min":0.81399999999999995,"15min":0.82299999999999995},"min":{"1min":0.60499999999999998,"5min":0.44400000000000001,"15min":0.44400000000000001},"max":{"1min":1.3009999999999999,"5min":1.3009999999999999,"15min":1.3009999999999999},"last":0.752},{"interface":"front","average":{"1min":0.84999999999999998,"5min":0.80800000000000005,"15min":0.81200000000000006},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.2829999999999999,"5min":1.2829999999999999,"15min":1.2829999999999999},"last":0.878}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1676,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":124,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":16,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":17,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T04:47:25.577 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T04:47:25.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T04:47:25.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.766+0000 7fc2c750b640 1 Processor -- start 2023-12-20T04:47:25.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.766+0000 7fc2c750b640 1 -- start start 2023-12-20T04:47:25.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c750b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2c00b1cd0 0x7fc2c00ac360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c750b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c00b2180 con 0x7fc2c015ccc0 2023-12-20T04:47:25.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c750b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2c015d0a0 con 0x7fc2c006a430 2023-12-20T04:47:25.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c750b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c0160b80 con 0x7fc2c00b1cd0 2023-12-20T04:47:25.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c6509640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc2c015ccc0 0x7fc2c0069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43946/0 (socket says 172.21.15.32:43946) 2023-12-20T04:47:25.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c6509640 1 -- 172.21.15.32:0/3752669947 learned_addr learned my addr 172.21.15.32:0/3752669947 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:25.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c6d0a640 1 --2- 172.21.15.32:0/3752669947 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2c00b1cd0 0x7fc2c00ac360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c5507640 1 -- 172.21.15.32:0/3752669947 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494750503 0 0) 0x7fc2c00b2180 con 0x7fc2c015ccc0 2023-12-20T04:47:25.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c5507640 1 -- 172.21.15.32:0/3752669947 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2ac003530 con 0x7fc2c015ccc0 2023-12-20T04:47:25.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c5507640 1 -- 172.21.15.32:0/3752669947 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2633854814 0 0) 0x7fc2c015d0a0 con 0x7fc2c006a430 2023-12-20T04:47:25.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c5507640 1 -- 172.21.15.32:0/3752669947 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc2c00b2180 con 0x7fc2c006a430 2023-12-20T04:47:25.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c6d0a640 1 -- 172.21.15.32:0/3752669947 >> v1:172.21.15.39:6789/0 conn(0x7fc2c006a430 legacy=0x7fc2c00abc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c5507640 1 -- 172.21.15.32:0/3752669947 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 775522385 0 0) 0x7fc2ac003530 con 0x7fc2c015ccc0 2023-12-20T04:47:25.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c6d0a640 1 -- 172.21.15.32:0/3752669947 >> v1:172.21.15.32:6789/0 conn(0x7fc2c015ccc0 legacy=0x7fc2c0069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.770+0000 7fc2c6d0a640 1 -- 172.21.15.32:0/3752669947 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2c0160e60 con 0x7fc2c00b1cd0 2023-12-20T04:47:25.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c6d0a640 1 --2- 172.21.15.32:0/3752669947 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2c00b1cd0 0x7fc2c00ac360 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009dc0 tx=0x7fc2b801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d14e1c967938ae30 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:25.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c5507640 1 -- 172.21.15.32:0/3752669947 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc001680 con 0x7fc2c00b1cd0 2023-12-20T04:47:25.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c5507640 1 -- 172.21.15.32:0/3752669947 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2b80a56d0 con 0x7fc2c00b1cd0 2023-12-20T04:47:25.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c5507640 1 -- 172.21.15.32:0/3752669947 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2b80a5a60 con 0x7fc2c00b1cd0 2023-12-20T04:47:25.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c750b640 1 -- 172.21.15.32:0/3752669947 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2c00b1cd0 msgr2=0x7fc2c00ac360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c750b640 1 --2- 172.21.15.32:0/3752669947 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2c00b1cd0 0x7fc2c00ac360 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7fc2b8009dc0 tx=0x7fc2b801c890 comp rx=0 tx=0).stop 2023-12-20T04:47:25.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c750b640 1 -- 172.21.15.32:0/3752669947 shutdown_connections 2023-12-20T04:47:25.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c750b640 1 --2- 172.21.15.32:0/3752669947 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2c00b1cd0 0x7fc2c00ac360 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c750b640 1 -- 172.21.15.32:0/3752669947 >> 172.21.15.32:0/3752669947 conn(0x7fc2c006d150 msgr2=0x7fc2c006d560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:25.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c750b640 1 -- 172.21.15.32:0/3752669947 shutdown_connections 2023-12-20T04:47:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c750b640 1 -- 172.21.15.32:0/3752669947 wait complete. 2023-12-20T04:47:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c750b640 1 Processor -- start 2023-12-20T04:47:25.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.774+0000 7fc2c750b640 1 -- start start 2023-12-20T04:47:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c750b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c006a430 0x7fc2c00c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c750b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2c00b1cd0 0x7fc2c014a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c750b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2c015ccc0 0x7fc2c00b4900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c750b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c00c4fa0 con 0x7fc2c006a430 2023-12-20T04:47:25.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c750b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c01527c0 con 0x7fc2c00b1cd0 2023-12-20T04:47:25.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c750b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2c00c51b0 con 0x7fc2c015ccc0 2023-12-20T04:47:25.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c6509640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c006a430 0x7fc2c00c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c6509640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c006a430 0x7fc2c00c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35098/0 (socket says 172.21.15.32:35098) 2023-12-20T04:47:25.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c6509640 1 -- 172.21.15.32:0/2802182929 learned_addr learned my addr 172.21.15.32:0/2802182929 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:25.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c6d0a640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2c015ccc0 0x7fc2c00b4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c5d08640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2c00b1cd0 0x7fc2c014a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c6509640 1 -- 172.21.15.32:0/2802182929 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2c015ccc0 msgr2=0x7fc2c00b4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c6509640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2c015ccc0 0x7fc2c00b4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c6509640 1 -- 172.21.15.32:0/2802182929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2c00b1cd0 msgr2=0x7fc2c014a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c6509640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2c00b1cd0 0x7fc2c014a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:25.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c6509640 1 -- 172.21.15.32:0/2802182929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2b8009a70 con 0x7fc2c006a430 2023-12-20T04:47:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c6509640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c006a430 0x7fc2c00c4a60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c750 tx=0x7fc2bc00cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2b6ffd640 1 -- 172.21.15.32:0/2802182929 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc005af0 con 0x7fc2c006a430 2023-12-20T04:47:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2b6ffd640 1 -- 172.21.15.32:0/2802182929 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2bc005c50 con 0x7fc2c006a430 2023-12-20T04:47:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2b6ffd640 1 -- 172.21.15.32:0/2802182929 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2bc007070 con 0x7fc2c006a430 2023-12-20T04:47:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c750b640 1 -- 172.21.15.32:0/2802182929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2c00c5490 con 0x7fc2c006a430 2023-12-20T04:47:25.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c750b640 1 -- 172.21.15.32:0/2802182929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2c00c5770 con 0x7fc2c006a430 2023-12-20T04:47:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2c750b640 1 -- 172.21.15.32:0/2802182929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2c0074100 con 0x7fc2c006a430 2023-12-20T04:47:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.778+0000 7fc2b6ffd640 1 -- 172.21.15.32:0/2802182929 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc2bc0071d0 con 0x7fc2c006a430 2023-12-20T04:47:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.782+0000 7fc2b6ffd640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2a4065bb0 0x7fc2a4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:25.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.782+0000 7fc2b6ffd640 1 -- 172.21.15.32:0/2802182929 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc2bc083020 con 0x7fc2c006a430 2023-12-20T04:47:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.782+0000 7fc2c5d08640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2a4065bb0 0x7fc2a4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:25.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.782+0000 7fc2c5d08640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2a4065bb0 0x7fc2a4068070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00c3750 tx=0x7fc2b0009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:25.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.786+0000 7fc2b6ffd640 1 -- 172.21.15.32:0/2802182929 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc2bc0508c0 con 0x7fc2c006a430 2023-12-20T04:47:25.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:25.998+0000 7fc2c750b640 1 -- 172.21.15.32:0/2802182929 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fc2c00b1350 con 0x7fc2c006a430 2023-12-20T04:47:26.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.002+0000 7fc2b6ffd640 1 -- 172.21.15.32:0/2802182929 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v22) v1 ==== 190+0+0 (secure 0 0 0) 0x7fc2bc053ee0 con 0x7fc2c006a430 2023-12-20T04:47:26.000 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T04:47:26.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.002+0000 7fc2c750b640 1 -- 172.21.15.32:0/2802182929 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2a4065bb0 msgr2=0x7fc2a4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.002+0000 7fc2c750b640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2a4065bb0 0x7fc2a4068070 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fc2c00c3750 tx=0x7fc2b0009040 comp rx=0 tx=0).stop 2023-12-20T04:47:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.002+0000 7fc2c750b640 1 -- 172.21.15.32:0/2802182929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c006a430 msgr2=0x7fc2c00c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.002+0000 7fc2c750b640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c006a430 0x7fc2c00c4a60 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fc2bc00c750 tx=0x7fc2bc00cc10 comp rx=0 tx=0).stop 2023-12-20T04:47:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.002+0000 7fc2c750b640 1 -- 172.21.15.32:0/2802182929 shutdown_connections 2023-12-20T04:47:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.002+0000 7fc2c750b640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc2a4065bb0 0x7fc2a4068070 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.002+0000 7fc2c750b640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2c015ccc0 0x7fc2c00b4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.002+0000 7fc2c750b640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2c00b1cd0 0x7fc2c014a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.002+0000 7fc2c750b640 1 --2- 172.21.15.32:0/2802182929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2c006a430 0x7fc2c00c4a60 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.002+0000 7fc2c750b640 1 -- 172.21.15.32:0/2802182929 >> 172.21.15.32:0/2802182929 conn(0x7fc2c006d150 msgr2=0x7fc2c00cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.006+0000 7fc2c750b640 1 -- 172.21.15.32:0/2802182929 shutdown_connections 2023-12-20T04:47:26.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.006+0000 7fc2c750b640 1 -- 172.21.15.32:0/2802182929 wait complete. 2023-12-20T04:47:26.022 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T04:47:26.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff2797a2640 1 Processor -- start 2023-12-20T04:47:26.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff2797a2640 1 -- start start 2023-12-20T04:47:26.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff2797a2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff27406b630 0x7ff274160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff2797a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff274161330 0x7ff274153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff2797a2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2740cd4d0 con 0x7ff2740b8480 2023-12-20T04:47:26.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff2797a2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2740ce4e0 con 0x7ff27406b630 2023-12-20T04:47:26.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff2797a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740cd720 con 0x7ff274161330 2023-12-20T04:47:26.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff273fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff2740b8480 0x7ff27406af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43958/0 (socket says 172.21.15.32:43958) 2023-12-20T04:47:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff273fff640 1 -- 172.21.15.32:0/1704380402 learned_addr learned my addr 172.21.15.32:0/1704380402 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff2737fe640 1 --2- 172.21.15.32:0/1704380402 >> v2:172.21.15.39:3300/0 conn(0x7ff27406b630 0x7ff274160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff278fa1640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff274161330 0x7ff274153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3770430335 0 0) 0x7ff2740cd4d0 con 0x7ff2740b8480 2023-12-20T04:47:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.170+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff25c003530 con 0x7ff2740b8480 2023-12-20T04:47:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3626448884 0 0) 0x7ff25c003530 con 0x7ff2740b8480 2023-12-20T04:47:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff274161330 msgr2=0x7ff274153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff274161330 0x7ff274153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 >> v2:172.21.15.39:3300/0 conn(0x7ff27406b630 msgr2=0x7ff274160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 --2- 172.21.15.32:0/1704380402 >> v2:172.21.15.39:3300/0 conn(0x7ff27406b630 0x7ff274160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2740cda00 con 0x7ff2740b8480 2023-12-20T04:47:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7ff264001680 con 0x7ff2740b8480 2023-12-20T04:47:26.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 >> v1:172.21.15.32:6789/0 conn(0x7ff2740b8480 legacy=0x7ff27406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff25c003980 0x7ff25c003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 0x7ff25c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff25c005190 0x7ff25c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740ce4e0 con 0x7ff25c0047d0 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2740cd720 con 0x7ff25c003980 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff25c00bed0 con 0x7ff25c005190 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff278fa1640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff274161330 0x7ff274153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff2737fe640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 0x7ff25c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff273fff640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff25c003980 0x7ff25c003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff278fa1640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff25c005190 0x7ff25c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff2737fe640 1 -- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff25c005190 msgr2=0x7ff25c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff2737fe640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff25c005190 0x7ff25c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff2737fe640 1 -- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff25c003980 msgr2=0x7ff25c003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff2737fe640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff25c003980 0x7ff25c003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff2737fe640 1 -- 172.21.15.32:0/1704380402 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff25c003530 con 0x7ff25c0047d0 2023-12-20T04:47:26.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff278fa1640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff25c005190 0x7ff25c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff2737fe640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 0x7ff25c004c50 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff268009b10 tx=0x7ff26801ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838625d9aa278fd3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.174+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2680a5680 con 0x7ff25c0047d0 2023-12-20T04:47:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2680a57e0 con 0x7ff25c0047d0 2023-12-20T04:47:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff272ffd640 1 -- 172.21.15.32:0/1704380402 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2680a5bd0 con 0x7ff25c0047d0 2023-12-20T04:47:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 -- 172.21.15.32:0/1704380402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 msgr2=0x7ff25c004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 --2- 172.21.15.32:0/1704380402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 0x7ff25c004c50 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7ff268009b10 tx=0x7ff26801ca90 comp rx=0 tx=0).stop 2023-12-20T04:47:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff278fa1640 1 -- 172.21.15.32:0/1704380402 reap_dead start 2023-12-20T04:47:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 -- 172.21.15.32:0/1704380402 shutdown_connections 2023-12-20T04:47:26.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 -- 172.21.15.32:0/1704380402 >> 172.21.15.32:0/1704380402 conn(0x7ff2740d2a50 msgr2=0x7ff2740d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 -- 172.21.15.32:0/1704380402 shutdown_connections 2023-12-20T04:47:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 -- 172.21.15.32:0/1704380402 wait complete. 2023-12-20T04:47:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 Processor -- start 2023-12-20T04:47:26.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 -- start start 2023-12-20T04:47:26.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 0x7ff2740cd090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff274153ae0 0x7ff274151050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff274154860 0x7ff274154ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff274151590 con 0x7ff25c0047d0 2023-12-20T04:47:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff274152a80 con 0x7ff274153ae0 2023-12-20T04:47:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff2797a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff274155220 con 0x7ff274154860 2023-12-20T04:47:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff273fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 0x7ff2740cd090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff273fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 0x7ff2740cd090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35114/0 (socket says 172.21.15.32:35114) 2023-12-20T04:47:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.178+0000 7ff273fff640 1 -- 172.21.15.32:0/2663002209 learned_addr learned my addr 172.21.15.32:0/2663002209 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff278fa1640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff274154860 0x7ff274154ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff2737fe640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff274153ae0 0x7ff274151050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff273fff640 1 -- 172.21.15.32:0/2663002209 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff274154860 msgr2=0x7ff274154ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff273fff640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff274154860 0x7ff274154ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff273fff640 1 -- 172.21.15.32:0/2663002209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff274153ae0 msgr2=0x7ff274151050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff273fff640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff274153ae0 0x7ff274151050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff273fff640 1 -- 172.21.15.32:0/2663002209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2741538a0 con 0x7ff25c0047d0 2023-12-20T04:47:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff2737fe640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff274153ae0 0x7ff274151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff278fa1640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff274154860 0x7ff274154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff273fff640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 0x7ff2740cd090 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff264012510 tx=0x7ff2640129d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff270ff9640 1 -- 172.21.15.32:0/2663002209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264014750 con 0x7ff25c0047d0 2023-12-20T04:47:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff270ff9640 1 -- 172.21.15.32:0/2663002209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2640148b0 con 0x7ff25c0047d0 2023-12-20T04:47:26.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff270ff9640 1 -- 172.21.15.32:0/2663002209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff264014c60 con 0x7ff25c0047d0 2023-12-20T04:47:26.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff2797a2640 1 -- 172.21.15.32:0/2663002209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff274152c90 con 0x7ff25c0047d0 2023-12-20T04:47:26.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff2797a2640 1 -- 172.21.15.32:0/2663002209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff274153250 con 0x7ff25c0047d0 2023-12-20T04:47:26.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff2797a2640 1 -- 172.21.15.32:0/2663002209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff274074100 con 0x7ff25c0047d0 2023-12-20T04:47:26.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.182+0000 7ff270ff9640 1 -- 172.21.15.32:0/2663002209 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff2640153e0 con 0x7ff25c0047d0 2023-12-20T04:47:26.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.186+0000 7ff270ff9640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff260065b80 0x7ff260068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.186+0000 7ff270ff9640 1 -- 172.21.15.32:0/2663002209 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff264085be0 con 0x7ff25c0047d0 2023-12-20T04:47:26.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.186+0000 7ff2737fe640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff260065b80 0x7ff260068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.186+0000 7ff2737fe640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff260065b80 0x7ff260068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff2680af000 tx=0x7ff2680046d0 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:26.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.190+0000 7ff270ff9640 1 -- 172.21.15.32:0/2663002209 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff264053520 con 0x7ff25c0047d0 2023-12-20T04:47:26.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.378+0000 7ff2797a2640 1 -- 172.21.15.32:0/2663002209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2740d1bf0 con 0x7ff25c0047d0 2023-12-20T04:47:26.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.378+0000 7ff270ff9640 1 -- 172.21.15.32:0/2663002209 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4708 (secure 0 0 0) 0x7ff26401c090 con 0x7ff25c0047d0 2023-12-20T04:47:26.378 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:26.378 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3306270605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3306270605},{"type":"v1","addr":"172.21.15.39:6834","nonce":3306270605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4000562557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4000562557},{"type":"v1","addr":"172.21.15.39:6835","nonce":4000562557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/310576469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":310576469},{"type":"v1","addr":"172.21.15.32:6835","nonce":310576469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:47:21.050837+0000","modified":"2023-12-20T04:47:24.070982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8397},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8397":{"gid":8397,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3314020360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3314020360},{"type":"v1","addr":"172.21.15.32:6837","nonce":3314020360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-20T04:47:26.378 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2023-12-20T04:47:26.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 -- 172.21.15.32:0/2663002209 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff260065b80 msgr2=0x7ff260068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff260065b80 0x7ff260068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7ff2680af000 tx=0x7ff2680046d0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 -- 172.21.15.32:0/2663002209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 msgr2=0x7ff2740cd090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 0x7ff2740cd090 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7ff264012510 tx=0x7ff2640129d0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 -- 172.21.15.32:0/2663002209 shutdown_connections 2023-12-20T04:47:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff260065b80 0x7ff260068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff274154860 0x7ff274154ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff274153ae0 0x7ff274151050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 --2- 172.21.15.32:0/2663002209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff25c0047d0 0x7ff2740cd090 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 -- 172.21.15.32:0/2663002209 >> 172.21.15.32:0/2663002209 conn(0x7ff2740d2a50 msgr2=0x7ff274151710 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 -- 172.21.15.32:0/2663002209 shutdown_connections 2023-12-20T04:47:26.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.382+0000 7ff2797a2640 1 -- 172.21.15.32:0/2663002209 wait complete. 2023-12-20T04:47:26.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 62, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T04:47:21.050837+0000', 'modified': '2023-12-20T04:47:24.070982+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8397}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8397': {'gid': 8397, 'name': 'c', 'rank': 0, 'incarnation': 58, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/3314020360', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 3314020360}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 3314020360}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [17], 'metadata_pool': 16, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T04:47:26.802 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T04:47:26.802 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-20T04:47:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff804739640 1 Processor -- start 2023-12-20T04:47:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff804739640 1 -- start start 2023-12-20T04:47:26.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff804739640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7fc0aece0 0x7ff7fc0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff804739640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff7fc0b0b30 0x7ff7fc0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff804739640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7fc0c98c0 con 0x7ff7fc1599e0 2023-12-20T04:47:26.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff804739640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7fc0c9ba0 con 0x7ff7fc0aece0 2023-12-20T04:47:26.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff804739640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7fc0c9e80 con 0x7ff7fc0b0b30 2023-12-20T04:47:26.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff801cd4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7fc0aece0 0x7ff7fc0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff801cd4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7fc0aece0 0x7ff7fc0b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35128/0 (socket says 172.21.15.32:35128) 2023-12-20T04:47:26.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff801cd4640 1 -- 172.21.15.32:0/2421218781 learned_addr learned my addr 172.21.15.32:0/2421218781 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:26.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff802cd6640 1 --2- 172.21.15.32:0/2421218781 >> v2:172.21.15.39:3300/0 conn(0x7ff7fc0b0b30 0x7ff7fc0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff801cd4640 1 -- 172.21.15.32:0/2421218781 >> v2:172.21.15.39:3300/0 conn(0x7ff7fc0b0b30 msgr2=0x7ff7fc0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff801cd4640 1 --2- 172.21.15.32:0/2421218781 >> v2:172.21.15.39:3300/0 conn(0x7ff7fc0b0b30 0x7ff7fc0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff801cd4640 1 -- 172.21.15.32:0/2421218781 >> v1:172.21.15.39:6789/0 conn(0x7ff7fc1599e0 legacy=0x7ff7fc0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.954+0000 7ff801cd4640 1 -- 172.21.15.32:0/2421218781 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7fc0ca160 con 0x7ff7fc0aece0 2023-12-20T04:47:26.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff801cd4640 1 --2- 172.21.15.32:0/2421218781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7fc0aece0 0x7ff7fc0b05f0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0b0060 tx=0x7ff7ec01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14a69526367a2777 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:26.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff8014d3640 1 -- 172.21.15.32:0/2421218781 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0ab610 con 0x7ff7fc0aece0 2023-12-20T04:47:26.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff8014d3640 1 -- 172.21.15.32:0/2421218781 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7ec0ab770 con 0x7ff7fc0aece0 2023-12-20T04:47:26.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff8014d3640 1 -- 172.21.15.32:0/2421218781 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0abb00 con 0x7ff7fc0aece0 2023-12-20T04:47:26.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff804739640 1 -- 172.21.15.32:0/2421218781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7fc0aece0 msgr2=0x7ff7fc0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff804739640 1 --2- 172.21.15.32:0/2421218781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7fc0aece0 0x7ff7fc0b05f0 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7ff7fc0b0060 tx=0x7ff7ec01c500 comp rx=0 tx=0).stop 2023-12-20T04:47:26.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff804739640 1 -- 172.21.15.32:0/2421218781 shutdown_connections 2023-12-20T04:47:26.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff804739640 1 --2- 172.21.15.32:0/2421218781 >> v2:172.21.15.39:3300/0 conn(0x7ff7fc0b0b30 0x7ff7fc0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff804739640 1 --2- 172.21.15.32:0/2421218781 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7fc0aece0 0x7ff7fc0b05f0 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff804739640 1 -- 172.21.15.32:0/2421218781 >> 172.21.15.32:0/2421218781 conn(0x7ff7fc06a320 msgr2=0x7ff7fc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:26.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff804739640 1 -- 172.21.15.32:0/2421218781 shutdown_connections 2023-12-20T04:47:26.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.958+0000 7ff804739640 1 -- 172.21.15.32:0/2421218781 wait complete. 2023-12-20T04:47:26.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff804739640 1 Processor -- start 2023-12-20T04:47:26.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff804739640 1 -- start start 2023-12-20T04:47:26.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff804739640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7fc0aece0 0x7ff7fc0acf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff804739640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7fc0b0b30 0x7ff7fc0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff804739640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7fc1599e0 0x7ff7fc0cdbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff804739640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0ae340 con 0x7ff7fc0aece0 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff804739640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0ad7c0 con 0x7ff7fc0b0b30 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff804739640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff7fc0adaa0 con 0x7ff7fc1599e0 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff8024d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7fc0aece0 0x7ff7fc0acf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff8024d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7fc0aece0 0x7ff7fc0acf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35130/0 (socket says 172.21.15.32:35130) 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff8024d5640 1 -- 172.21.15.32:0/1788710734 learned_addr learned my addr 172.21.15.32:0/1788710734 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff802cd6640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7fc1599e0 0x7ff7fc0cdbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff801cd4640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7fc0b0b30 0x7ff7fc0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff8024d5640 1 -- 172.21.15.32:0/1788710734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7fc1599e0 msgr2=0x7ff7fc0cdbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff8024d5640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7fc1599e0 0x7ff7fc0cdbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff8024d5640 1 -- 172.21.15.32:0/1788710734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7fc0b0b30 msgr2=0x7ff7fc0cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff8024d5640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7fc0b0b30 0x7ff7fc0cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff8024d5640 1 -- 172.21.15.32:0/1788710734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7ec0094d0 con 0x7ff7fc0aece0 2023-12-20T04:47:26.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff802cd6640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7fc1599e0 0x7ff7fc0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:26.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff8024d5640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7fc0aece0 0x7ff7fc0acf00 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800c400 tx=0x7ff7f800c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:26.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff7f2ffd640 1 -- 172.21.15.32:0/1788710734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f80052d0 con 0x7ff7fc0aece0 2023-12-20T04:47:26.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff7f2ffd640 1 -- 172.21.15.32:0/1788710734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f8005430 con 0x7ff7fc0aece0 2023-12-20T04:47:26.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff7f2ffd640 1 -- 172.21.15.32:0/1788710734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8003890 con 0x7ff7fc0aece0 2023-12-20T04:47:26.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff804739640 1 -- 172.21.15.32:0/1788710734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff7fc0add80 con 0x7ff7fc0aece0 2023-12-20T04:47:26.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.962+0000 7ff804739640 1 -- 172.21.15.32:0/1788710734 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc0ad440 con 0x7ff7fc0aece0 2023-12-20T04:47:26.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.966+0000 7ff7f2ffd640 1 -- 172.21.15.32:0/1788710734 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff7f80039f0 con 0x7ff7fc0aece0 2023-12-20T04:47:26.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.966+0000 7ff7f2ffd640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff7e0065b80 0x7ff7e0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.966+0000 7ff7f2ffd640 1 -- 172.21.15.32:0/1788710734 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff7f80827d0 con 0x7ff7fc0aece0 2023-12-20T04:47:26.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.966+0000 7ff807aa1000 1 Processor -- start 2023-12-20T04:47:26.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.966+0000 7ff801cd4640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff7e0065b80 0x7ff7e0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.966+0000 7ff807aa1000 1 -- start start 2023-12-20T04:47:26.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.966+0000 7ff801cd4640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff7e0065b80 0x7ff7e0068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0af000 tx=0x7ff7ec0b6040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3662450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662990 0x55a2f3617c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f36181c0 0x55a2f36185a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f343bc90 con 0x55a2f3662070 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f346d220 con 0x55a2f36181c0 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2f346a7f0 con 0x55a2f3662990 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff8024d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3662450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff8024d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3662450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35140/0 (socket says 172.21.15.32:35140) 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff8024d5640 1 -- 172.21.15.32:0/4075613103 learned_addr learned my addr 172.21.15.32:0/4075613103 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff802cd6640 1 --2- 172.21.15.32:0/4075613103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f36181c0 0x55a2f36185a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff801cd4640 1 --2- 172.21.15.32:0/4075613103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662990 0x55a2f3617c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff8024d5640 1 -- 172.21.15.32:0/4075613103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662990 msgr2=0x55a2f3617c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff8024d5640 1 --2- 172.21.15.32:0/4075613103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662990 0x55a2f3617c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff8024d5640 1 -- 172.21.15.32:0/4075613103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f36181c0 msgr2=0x55a2f36185a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff8024d5640 1 --2- 172.21.15.32:0/4075613103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f36181c0 0x55a2f36185a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff8024d5640 1 -- 172.21.15.32:0/4075613103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2f3162890 con 0x55a2f3662070 2023-12-20T04:47:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff802cd6640 1 --2- 172.21.15.32:0/4075613103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f36181c0 0x55a2f36185a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff8024d5640 1 --2- 172.21.15.32:0/4075613103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3662450 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8007fd0 tx=0x7ff7f8053f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e944b1a094c43b4e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff7dffff640 1 -- 172.21.15.32:0/4075613103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8080cd0 con 0x55a2f3662070 2023-12-20T04:47:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff7dffff640 1 -- 172.21.15.32:0/4075613103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f8080e50 con 0x55a2f3662070 2023-12-20T04:47:26.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff7dffff640 1 -- 172.21.15.32:0/4075613103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f805b900 con 0x55a2f3662070 2023-12-20T04:47:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 -- 172.21.15.32:0/4075613103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 msgr2=0x55a2f3662450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 --2- 172.21.15.32:0/4075613103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3662450 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8007fd0 tx=0x7ff7f8053f20 comp rx=0 tx=0).stop 2023-12-20T04:47:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 -- 172.21.15.32:0/4075613103 shutdown_connections 2023-12-20T04:47:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 --2- 172.21.15.32:0/4075613103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f36181c0 0x55a2f36185a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 --2- 172.21.15.32:0/4075613103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662990 0x55a2f3617c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 --2- 172.21.15.32:0/4075613103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3662450 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 -- 172.21.15.32:0/4075613103 >> 172.21.15.32:0/4075613103 conn(0x55a2f361e950 msgr2=0x55a2f3616560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 -- 172.21.15.32:0/4075613103 shutdown_connections 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 -- 172.21.15.32:0/4075613103 wait complete. 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 -- start start 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.970+0000 7ff807aa1000 1 Processor -- start 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f36181c0 0x55a2f361d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3624080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662990 0x55a2f36245c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f34660e0 con 0x55a2f3662070 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f3405310 con 0x55a2f36181c0 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2f3423770 con 0x55a2f3662990 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 10 client.0 _ll_register_callbacks cb 0x55a2f3603b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:47:26.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff801cd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3624080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff801cd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3624080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35150/0 (socket says 172.21.15.32:35150) 2023-12-20T04:47:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff801cd4640 1 -- 172.21.15.32:0/2697539319 learned_addr learned my addr 172.21.15.32:0/2697539319 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff8024d5640 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f36181c0 0x55a2f361d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff802cd6640 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662990 0x55a2f36245c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff801cd4640 1 -- 172.21.15.32:0/2697539319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662990 msgr2=0x55a2f36245c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff801cd4640 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662990 0x55a2f36245c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff801cd4640 1 -- 172.21.15.32:0/2697539319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f36181c0 msgr2=0x55a2f361d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff801cd4640 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f36181c0 0x55a2f361d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff801cd4640 1 -- 172.21.15.32:0/2697539319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff7fc0ad440 con 0x55a2f3662070 2023-12-20T04:47:26.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff801cd4640 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3624080 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec01a130 tx=0x7ff7ec01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9075a42bf5431f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff7dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2697539319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7ec0c5070 con 0x55a2f3662070 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2697539319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff7ec0be3f0 con 0x55a2f3662070 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 1 -- 172.21.15.32:0/2697539319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2f3435c10 con 0x55a2f3662070 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2697539319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7ec0be5b0 con 0x55a2f3662070 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 10 client.8526 fetch_fsmap learned FSMap version 62 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 1 -- 172.21.15.32:0/2697539319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55a2f342b260 con 0x55a2f3662070 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2697539319 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a2f3435c10 con 0x55a2f3662070 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2697539319 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff7ec0be710 con 0x55a2f3662070 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 10 client.8526 fetch_fsmap finished waiting for FSMap version 62 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 10 client.8526 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-20T04:47:26.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 20 client.8526 populate_metadata read hostname 'smithi032' 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.32:6836/3314020360,v1:172.21.15.32:6837/3314020360] conn(0x55a2f36698d0 0x55a2f3669cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 4 client.8526 mds_command: new command op to 8397 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 1 -- 172.21.15.32:0/2697539319 --> [v2:172.21.15.32:6836/3314020360,v1:172.21.15.32:6837/3314020360] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55a2f3464a30 con 0x55a2f36698d0 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff8024d5640 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.32:6836/3314020360,v1:172.21.15.32:6837/3314020360] conn(0x55a2f36698d0 0x55a2f3669cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff8024d5640 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.32:6836/3314020360,v1:172.21.15.32:6837/3314020360] conn(0x55a2f36698d0 0x55a2f3669cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=81e9aa761fc7f8e9 server_cookie=8400e0d59b509129 in_seq=0 out_seq=0 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff7dd7fa640 10 client.8526 ms_handle_connect on v2:172.21.15.32:6836/3314020360 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2697539319 <== mds.0 v2:172.21.15.32:6836/3314020360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55a2f3464a30 con 0x55a2f36698d0 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff7dd7fa640 10 client.8526 handle_command_reply: tid=0 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.974+0000 7ff807aa1000 1 client.8526 shutdown 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff807aa1000 1 -- 172.21.15.32:0/2697539319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 msgr2=0x55a2f3624080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3624080 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec01a130 tx=0x7ff7ec01ad40 comp rx=0 tx=0).stop 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff807aa1000 1 -- 172.21.15.32:0/2697539319 shutdown_connections 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.32:6836/3314020360,v1:172.21.15.32:6837/3314020360] conn(0x55a2f36698d0 0x55a2f3669cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662990 0x55a2f36245c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f3662070 0x55a2f3624080 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2697539319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f36181c0 0x55a2f361d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff7dd7fa640 0 client.8526 ms_handle_reset on v2:172.21.15.32:6836/3314020360 2023-12-20T04:47:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff807aa1000 1 -- 172.21.15.32:0/2697539319 >> 172.21.15.32:0/2697539319 conn(0x55a2f361e950 msgr2=0x55a2f3616c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff807aa1000 1 -- 172.21.15.32:0/2697539319 shutdown_connections 2023-12-20T04:47:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff807aa1000 1 -- 172.21.15.32:0/2697539319 wait complete. 2023-12-20T04:47:26.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.978+0000 7ff807aa1000 20 client.8526 trim_cache size 0 max 16384 2023-12-20T04:47:26.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 Processor -- start 2023-12-20T04:47:26.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 -- start start 2023-12-20T04:47:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662070 0x55a2f36b9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f3662990 0x55a2f36ba320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f3123940 con 0x55a2f36181c0 2023-12-20T04:47:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f3668fd0 con 0x55a2f3662990 2023-12-20T04:47:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2f340e860 con 0x55a2f3662070 2023-12-20T04:47:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff8024d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff8024d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b98a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35160/0 (socket says 172.21.15.32:35160) 2023-12-20T04:47:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff8024d5640 1 -- 172.21.15.32:0/3770025388 learned_addr learned my addr 172.21.15.32:0/3770025388 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff801cd4640 1 --2- 172.21.15.32:0/3770025388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662070 0x55a2f36b9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff802cd6640 1 --2- 172.21.15.32:0/3770025388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f3662990 0x55a2f36ba320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff8024d5640 1 -- 172.21.15.32:0/3770025388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662070 msgr2=0x55a2f36b9de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff8024d5640 1 --2- 172.21.15.32:0/3770025388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662070 0x55a2f36b9de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff8024d5640 1 -- 172.21.15.32:0/3770025388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f3662990 msgr2=0x55a2f36ba320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff8024d5640 1 --2- 172.21.15.32:0/3770025388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f3662990 0x55a2f36ba320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff8024d5640 1 -- 172.21.15.32:0/3770025388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55a2f34113b0 con 0x55a2f36181c0 2023-12-20T04:47:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff8024d5640 1 --2- 172.21.15.32:0/3770025388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b98a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8053df0 tx=0x7ff7f8053d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df03b830de57c669 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff7f0ff9640 1 -- 172.21.15.32:0/3770025388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8050570 con 0x55a2f36181c0 2023-12-20T04:47:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff7f0ff9640 1 -- 172.21.15.32:0/3770025388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7f80506d0 con 0x55a2f36181c0 2023-12-20T04:47:26.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff7f0ff9640 1 -- 172.21.15.32:0/3770025388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8080ea0 con 0x55a2f36181c0 2023-12-20T04:47:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 -- 172.21.15.32:0/3770025388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 msgr2=0x55a2f36b98a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 --2- 172.21.15.32:0/3770025388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b98a0 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7ff7f8053df0 tx=0x7ff7f8053d60 comp rx=0 tx=0).stop 2023-12-20T04:47:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 -- 172.21.15.32:0/3770025388 shutdown_connections 2023-12-20T04:47:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 --2- 172.21.15.32:0/3770025388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f3662990 0x55a2f36ba320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 --2- 172.21.15.32:0/3770025388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662070 0x55a2f36b9de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 --2- 172.21.15.32:0/3770025388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b98a0 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 -- 172.21.15.32:0/3770025388 >> 172.21.15.32:0/3770025388 conn(0x55a2f361e950 msgr2=0x55a2f36238c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 -- 172.21.15.32:0/3770025388 shutdown_connections 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 -- 172.21.15.32:0/3770025388 wait complete. 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 -- start start 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.994+0000 7ff807aa1000 1 Processor -- start 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662070 0x55a2f36b9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f3662990 0x55a2f36bab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f345b1f0 con 0x55a2f36181c0 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55a2f3433a80 con 0x55a2f3662990 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55a2f340ce10 con 0x55a2f3662070 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 10 client.0 _ll_register_callbacks cb 0x55a2f3414640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff8024d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff802cd6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f3662990 0x55a2f36bab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff8024d5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b9ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35176/0 (socket says 172.21.15.32:35176) 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff8024d5640 1 -- 172.21.15.32:0/2015096819 learned_addr learned my addr 172.21.15.32:0/2015096819 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff801cd4640 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662070 0x55a2f36b9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff8024d5640 1 -- 172.21.15.32:0/2015096819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662070 msgr2=0x55a2f36b9ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff8024d5640 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662070 0x55a2f36b9ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff8024d5640 1 -- 172.21.15.32:0/2015096819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f3662990 msgr2=0x55a2f36bab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff8024d5640 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f3662990 0x55a2f36bab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff8024d5640 1 -- 172.21.15.32:0/2015096819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55a2f3162890 con 0x55a2f36181c0 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff8024d5640 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b9ab0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800cdc0 tx=0x7ff7f801e070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aade39649b9dc102 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff7dd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2015096819 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7f8050570 con 0x55a2f36181c0 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2015096819 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff7f806b1e0 con 0x55a2f36181c0 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 -- 172.21.15.32:0/2015096819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55a2f342be00 con 0x55a2f36181c0 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2015096819 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=62) v2 ==== 24+0+0 (secure 0 0 0) 0x7ff7f8080ea0 con 0x55a2f36181c0 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 10 client.8532 fetch_fsmap learned FSMap version 62 2023-12-20T04:47:26.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 -- 172.21.15.32:0/2015096819 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=62,monmap=2+}) v3 -- 0x55a2f3162cc0 con 0x55a2f36181c0 2023-12-20T04:47:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2015096819 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55a2f342be00 con 0x55a2f36181c0 2023-12-20T04:47:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2015096819 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 62) v1 ==== 3114+0+0 (secure 0 0 0) 0x7ff7f805b900 con 0x55a2f36181c0 2023-12-20T04:47:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 10 client.8532 fetch_fsmap finished waiting for FSMap version 62 2023-12-20T04:47:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 10 client.8532 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-20T04:47:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 20 client.8532 populate_metadata read hostname 'smithi032' 2023-12-20T04:47:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.32:6836/3314020360,v1:172.21.15.32:6837/3314020360] conn(0x55a2f36b94c0 0x55a2f36bb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 4 client.8532 mds_command: new command op to 8397 tid=0[{"prefix": "status"}] 2023-12-20T04:47:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 -- 172.21.15.32:0/2015096819 --> [v2:172.21.15.32:6836/3314020360,v1:172.21.15.32:6837/3314020360] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55a2f345f160 con 0x55a2f36b94c0 2023-12-20T04:47:27.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff801cd4640 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.32:6836/3314020360,v1:172.21.15.32:6837/3314020360] conn(0x55a2f36b94c0 0x55a2f36bb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff801cd4640 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.32:6836/3314020360,v1:172.21.15.32:6837/3314020360] conn(0x55a2f36b94c0 0x55a2f36bb090 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce19ef57d8fc9b56 server_cookie=7aad78133a987638 in_seq=0 out_seq=0 2023-12-20T04:47:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff7dd7fa640 10 client.8532 ms_handle_connect on v2:172.21.15.32:6836/3314020360 2023-12-20T04:47:27.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff7dd7fa640 1 -- 172.21.15.32:0/2015096819 <== mds.0 v2:172.21.15.32:6836/3314020360 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x55a2f345f160 con 0x55a2f36b94c0 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff7dd7fa640 10 client.8532 handle_command_reply: tid=0 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 client.8532 shutdown 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 -- 172.21.15.32:0/2015096819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 msgr2=0x55a2f36b9ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b9ab0 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800cdc0 tx=0x7ff7f801e070 comp rx=0 tx=0).stop 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 -- 172.21.15.32:0/2015096819 shutdown_connections 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.32:6836/3314020360,v1:172.21.15.32:6837/3314020360] conn(0x55a2f36b94c0 0x55a2f36bb090 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55a2f3662990 0x55a2f36bab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55a2f3662070 0x55a2f36b9ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 --2- 172.21.15.32:0/2015096819 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55a2f36181c0 0x55a2f36b9ab0 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff7dd7fa640 0 client.8532 ms_handle_reset on v2:172.21.15.32:6836/3314020360 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:26.998+0000 7ff807aa1000 1 -- 172.21.15.32:0/2015096819 >> 172.21.15.32:0/2015096819 conn(0x55a2f361e950 msgr2=0x55a2f3623fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff807aa1000 1 -- 172.21.15.32:0/2015096819 shutdown_connections 2023-12-20T04:47:27.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff807aa1000 1 -- 172.21.15.32:0/2015096819 wait complete. 2023-12-20T04:47:27.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff807aa1000 20 client.8532 trim_cache size 0 max 16384 2023-12-20T04:47:27.003 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T04:47:27.003 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T04:47:27.003 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T04:47:27.003 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 8397, 2023-12-20T04:47:27.003 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T04:47:27.003 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T04:47:27.003 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T04:47:27.003 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9498005029999996, 2023-12-20T04:47:27.003 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 62, 2023-12-20T04:47:27.003 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 138, 2023-12-20T04:47:27.004 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 138, 2023-12-20T04:47:27.004 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.302276915 2023-12-20T04:47:27.004 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T04:47:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 -- 172.21.15.32:0/1788710734 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff7e0065b80 msgr2=0x7ff7e0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff7e0065b80 0x7ff7e0068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7ff7ec0af000 tx=0x7ff7ec0b6040 comp rx=0 tx=0).stop 2023-12-20T04:47:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 -- 172.21.15.32:0/1788710734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7fc0aece0 msgr2=0x7ff7fc0acf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7fc0aece0 0x7ff7fc0acf00 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7ff7f800c400 tx=0x7ff7f800c8c0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 -- 172.21.15.32:0/1788710734 shutdown_connections 2023-12-20T04:47:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff7e0065b80 0x7ff7e0068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7fc1599e0 0x7ff7fc0cdbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff7fc0b0b30 0x7ff7fc0cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 --2- 172.21.15.32:0/1788710734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7fc0aece0 0x7ff7fc0acf00 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 -- 172.21.15.32:0/1788710734 >> 172.21.15.32:0/1788710734 conn(0x7ff7fc06a320 msgr2=0x7ff7fc153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:27.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 -- 172.21.15.32:0/1788710734 shutdown_connections 2023-12-20T04:47:27.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.002+0000 7ff804739640 1 -- 172.21.15.32:0/1788710734 wait complete. 2023-12-20T04:47:27.018 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T04:47:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.162+0000 7fc99d760640 1 Processor -- start 2023-12-20T04:47:27.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.162+0000 7fc99d760640 1 -- start start 2023-12-20T04:47:27.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.166+0000 7fc99d760640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc99815ce30 0x7fc998161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.166+0000 7fc99d760640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc998071920 con 0x7fc998070950 2023-12-20T04:47:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.166+0000 7fc99d760640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc998071080 con 0x7fc99806deb0 2023-12-20T04:47:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.166+0000 7fc99d760640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc998071360 con 0x7fc99815ce30 2023-12-20T04:47:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.166+0000 7fc98ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc998070950 0x7fc99806d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43960/0 (socket says 172.21.15.32:43960) 2023-12-20T04:47:27.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.166+0000 7fc98ffff640 1 -- 172.21.15.32:0/1574272984 learned_addr learned my addr 172.21.15.32:0/1574272984 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:27.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.166+0000 7fc996ffd640 1 --2- 172.21.15.32:0/1574272984 >> v2:172.21.15.32:3300/0 conn(0x7fc99815ce30 0x7fc998161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:27.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.166+0000 7fc996ffd640 1 -- 172.21.15.32:0/1574272984 >> v1:172.21.15.39:6789/0 conn(0x7fc99806deb0 legacy=0x7fc9980cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:27.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.166+0000 7fc996ffd640 1 -- 172.21.15.32:0/1574272984 >> v1:172.21.15.32:6789/0 conn(0x7fc998070950 legacy=0x7fc99806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:27.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.166+0000 7fc996ffd640 1 -- 172.21.15.32:0/1574272984 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc998071640 con 0x7fc99815ce30 2023-12-20T04:47:27.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.166+0000 7fc9967fc640 1 -- 172.21.15.32:0/1574272984 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3798086070 0 0) 0x7fc998071920 con 0x7fc998070950 2023-12-20T04:47:27.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc996ffd640 1 --2- 172.21.15.32:0/1574272984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc99815ce30 0x7fc998161560 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc9880097c0 tx=0x7fc98801c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2ea511fdf423c16 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:27.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc9967fc640 1 -- 172.21.15.32:0/1574272984 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9880ab600 con 0x7fc99815ce30 2023-12-20T04:47:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc9967fc640 1 -- 172.21.15.32:0/1574272984 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9880a7050 con 0x7fc99815ce30 2023-12-20T04:47:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc9967fc640 1 -- 172.21.15.32:0/1574272984 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9880ab980 con 0x7fc99815ce30 2023-12-20T04:47:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 -- 172.21.15.32:0/1574272984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc99815ce30 msgr2=0x7fc998161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 --2- 172.21.15.32:0/1574272984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc99815ce30 0x7fc998161560 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fc9880097c0 tx=0x7fc98801c3f0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 -- 172.21.15.32:0/1574272984 shutdown_connections 2023-12-20T04:47:27.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 --2- 172.21.15.32:0/1574272984 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc99815ce30 0x7fc998161560 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 -- 172.21.15.32:0/1574272984 >> 172.21.15.32:0/1574272984 conn(0x7fc9980cba50 msgr2=0x7fc9980cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:27.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 -- 172.21.15.32:0/1574272984 shutdown_connections 2023-12-20T04:47:27.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 -- 172.21.15.32:0/1574272984 wait complete. 2023-12-20T04:47:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 Processor -- start 2023-12-20T04:47:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 -- start start 2023-12-20T04:47:27.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc99806deb0 0x7fc998158b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc998070950 0x7fc998156db0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc99815ce30 0x7fc9980c0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980c0940 con 0x7fc998070950 2023-12-20T04:47:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9980c05d0 con 0x7fc99815ce30 2023-12-20T04:47:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc99d760640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc998159a80 con 0x7fc99806deb0 2023-12-20T04:47:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc998070950 0x7fc998156db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:27.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc98ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc998070950 0x7fc998156db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35194/0 (socket says 172.21.15.32:35194) 2023-12-20T04:47:27.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc98ffff640 1 -- 172.21.15.32:0/3493491004 learned_addr learned my addr 172.21.15.32:0/3493491004 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:27.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc996ffd640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc99806deb0 0x7fc998158b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:27.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.170+0000 7fc9977fe640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc99815ce30 0x7fc9980c0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:27.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98ffff640 1 -- 172.21.15.32:0/3493491004 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc99806deb0 msgr2=0x7fc998158b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:27.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98ffff640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc99806deb0 0x7fc998158b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98ffff640 1 -- 172.21.15.32:0/3493491004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc99815ce30 msgr2=0x7fc9980c0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98ffff640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc99815ce30 0x7fc9980c0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98ffff640 1 -- 172.21.15.32:0/3493491004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc988009420 con 0x7fc998070950 2023-12-20T04:47:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98ffff640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc998070950 0x7fc998156db0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc98000c4d0 tx=0x7fc98000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98f7fe640 1 -- 172.21.15.32:0/3493491004 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9800074e0 con 0x7fc998070950 2023-12-20T04:47:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98f7fe640 1 -- 172.21.15.32:0/3493491004 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc980007640 con 0x7fc998070950 2023-12-20T04:47:27.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98f7fe640 1 -- 172.21.15.32:0/3493491004 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc980004a30 con 0x7fc998070950 2023-12-20T04:47:27.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc99d760640 1 -- 172.21.15.32:0/3493491004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc998159d60 con 0x7fc998070950 2023-12-20T04:47:27.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc99d760640 1 -- 172.21.15.32:0/3493491004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9981572f0 con 0x7fc998070950 2023-12-20T04:47:27.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc99d760640 1 -- 172.21.15.32:0/3493491004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc99804e000 con 0x7fc998070950 2023-12-20T04:47:27.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98f7fe640 1 -- 172.21.15.32:0/3493491004 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc980004b90 con 0x7fc998070950 2023-12-20T04:47:27.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98f7fe640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc978065b80 0x7fc978068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:27.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc98f7fe640 1 -- 172.21.15.32:0/3493491004 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc980082450 con 0x7fc998070950 2023-12-20T04:47:27.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.174+0000 7fc996ffd640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc978065b80 0x7fc978068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:27.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.178+0000 7fc996ffd640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc978065b80 0x7fc978068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc9880ad000 tx=0x7fc9880a6000 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:27.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.178+0000 7fc98f7fe640 1 -- 172.21.15.32:0/3493491004 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc98004fe20 con 0x7fc998070950 2023-12-20T04:47:27.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.370+0000 7fc99d760640 1 -- 172.21.15.32:0/3493491004 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7fc9980b4f50 con 0x7fc998070950 2023-12-20T04:47:27.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.370+0000 7fc98f7fe640 1 -- 172.21.15.32:0/3493491004 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v62) v1 ==== 53+0+73 (secure 0 0 0) 0x7fc980018070 con 0x7fc998070950 2023-12-20T04:47:27.370 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T04:47:27.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 -- 172.21.15.32:0/3493491004 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc978065b80 msgr2=0x7fc978068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:27.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc978065b80 0x7fc978068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc9880ad000 tx=0x7fc9880a6000 comp rx=0 tx=0).stop 2023-12-20T04:47:27.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 -- 172.21.15.32:0/3493491004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc998070950 msgr2=0x7fc998156db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:27.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc998070950 0x7fc998156db0 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fc98000c4d0 tx=0x7fc98000c990 comp rx=0 tx=0).stop 2023-12-20T04:47:27.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 -- 172.21.15.32:0/3493491004 shutdown_connections 2023-12-20T04:47:27.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc978065b80 0x7fc978068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc99815ce30 0x7fc9980c0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc998070950 0x7fc998156db0 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 --2- 172.21.15.32:0/3493491004 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc99806deb0 0x7fc998158b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:27.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 -- 172.21.15.32:0/3493491004 >> 172.21.15.32:0/3493491004 conn(0x7fc9980cba50 msgr2=0x7fc998160e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:27.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 -- 172.21.15.32:0/3493491004 shutdown_connections 2023-12-20T04:47:27.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:27.374+0000 7fc99d760640 1 -- 172.21.15.32:0/3493491004 wait complete. 2023-12-20T04:47:27.392 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T04:47:27.392 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T04:47:27.392 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T04:47:27.392 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T04:47:27.392 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T04:47:27.392 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T04:47:27.392 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T04:47:27.392 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T04:47:27.392 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:47:27.393 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:27.393 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T04:47:27.403 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T04:47:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T04:47:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T04:47:27.403 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 41212sec preferred_lft 41212sec 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T04:47:27.404 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T04:47:27.405 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:47:27.406 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:47:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T04:47:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T04:47:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:47:27.406 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T04:47:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T04:47:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T04:47:27.406 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T04:47:27.406 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:27.406 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T04:47:27.465 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T04:47:27.467 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:27.467 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T04:47:27.520 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T04:47:27.520 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T04:47:27.520 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T04:47:27.520 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T04:47:27.520 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T04:47:27.520 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T04:47:27.520 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:27.520 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T04:47:27.521 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T04:47:27.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T04:47:27.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T04:47:27.521 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T04:47:27.521 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T04:47:27.619 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:27.619 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T04:47:27.632 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T04:47:27.632 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T04:47:27.633 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:47:27.633 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:27.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T04:47:27.702 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:47:27.702 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T04:47:27.760 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T04:47:27.760 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T04:47:27.760 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:47:27.812 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T04:47:27.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:27.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:47:27.905 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:47:27.905 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:47:27.918 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T04:47:27.918 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T04:47:27.918 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T04:47:27.919 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T04:47:27.919 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T04:47:27.923 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T04:47:27.923 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:27.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T04:47:27.944 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:27.945 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T04:47:28.003 INFO:teuthology.orchestra.run.smithi032.stdout:client8299 2023-12-20T04:47:28.003 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T04:47:28.003 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T04:47:28.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T04:47:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.242+0000 7f1e4c56d640 1 Processor -- start 2023-12-20T04:47:28.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.242+0000 7f1e4c56d640 1 -- start start 2023-12-20T04:47:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e4c56d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e440aece0 0x7f1e440b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e4c56d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1e440b0b30 0x7f1e440701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e4c56d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e440c98c0 con 0x7f1e441599e0 2023-12-20T04:47:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e4c56d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e440c9ba0 con 0x7f1e440aece0 2023-12-20T04:47:28.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e4c56d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e440c9e80 con 0x7f1e440b0b30 2023-12-20T04:47:28.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e49b08640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e440aece0 0x7f1e440b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e4a309640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1e441599e0 0x7f1e440c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43962/0 (socket says 172.21.15.32:43962) 2023-12-20T04:47:28.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e4a309640 1 -- 172.21.15.32:0/1549347359 learned_addr learned my addr 172.21.15.32:0/1549347359 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:28.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e4ab0a640 1 --2- 172.21.15.32:0/1549347359 >> v2:172.21.15.39:3300/0 conn(0x7f1e440b0b30 0x7f1e440701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e49b08640 1 -- 172.21.15.32:0/1549347359 >> v2:172.21.15.39:3300/0 conn(0x7f1e440b0b30 msgr2=0x7f1e440701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:28.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e49b08640 1 --2- 172.21.15.32:0/1549347359 >> v2:172.21.15.39:3300/0 conn(0x7f1e440b0b30 0x7f1e440701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e49b08640 1 -- 172.21.15.32:0/1549347359 >> v1:172.21.15.32:6789/0 conn(0x7f1e441599e0 legacy=0x7f1e440c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:28.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e49b08640 1 -- 172.21.15.32:0/1549347359 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e440ca160 con 0x7f1e440aece0 2023-12-20T04:47:28.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.246+0000 7f1e49307640 1 -- 172.21.15.32:0/1549347359 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547231314 0 0) 0x7f1e440c98c0 con 0x7f1e441599e0 2023-12-20T04:47:28.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e49b08640 1 --2- 172.21.15.32:0/1549347359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e440aece0 0x7f1e440b05f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f1e440b0060 tx=0x7f1e3401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d40e312c8f47665e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:28.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e49307640 1 -- 172.21.15.32:0/1549347359 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340ab600 con 0x7f1e440aece0 2023-12-20T04:47:28.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e49307640 1 -- 172.21.15.32:0/1549347359 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e340a7050 con 0x7f1e440aece0 2023-12-20T04:47:28.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e49307640 1 -- 172.21.15.32:0/1549347359 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e340ab980 con 0x7f1e440aece0 2023-12-20T04:47:28.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1549347359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e440aece0 msgr2=0x7f1e440b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:28.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e4c56d640 1 --2- 172.21.15.32:0/1549347359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e440aece0 0x7f1e440b05f0 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f1e440b0060 tx=0x7f1e3401c460 comp rx=0 tx=0).stop 2023-12-20T04:47:28.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1549347359 shutdown_connections 2023-12-20T04:47:28.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e4c56d640 1 --2- 172.21.15.32:0/1549347359 >> v2:172.21.15.39:3300/0 conn(0x7f1e440b0b30 0x7f1e440701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e4c56d640 1 --2- 172.21.15.32:0/1549347359 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e440aece0 0x7f1e440b05f0 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1549347359 >> 172.21.15.32:0/1549347359 conn(0x7f1e4406a320 msgr2=0x7f1e440ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:28.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1549347359 shutdown_connections 2023-12-20T04:47:28.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1549347359 wait complete. 2023-12-20T04:47:28.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e4c56d640 1 Processor -- start 2023-12-20T04:47:28.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.250+0000 7f1e4c56d640 1 -- start start 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4c56d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e440aece0 0x7f1e440c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4c56d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e440b0b30 0x7f1e4414fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4c56d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e441599e0 0x7f1e440c7f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4c56d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e44151da0 con 0x7f1e441599e0 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4c56d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4415c010 con 0x7f1e440b0b30 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4c56d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e4415c2f0 con 0x7f1e440aece0 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4ab0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e441599e0 0x7f1e440c7f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4ab0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e441599e0 0x7f1e440c7f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35208/0 (socket says 172.21.15.32:35208) 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4ab0a640 1 -- 172.21.15.32:0/1264375916 learned_addr learned my addr 172.21.15.32:0/1264375916 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e49b08640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e440b0b30 0x7f1e4414fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4a309640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e440aece0 0x7f1e440c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4ab0a640 1 -- 172.21.15.32:0/1264375916 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e440aece0 msgr2=0x7f1e440c5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4ab0a640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e440aece0 0x7f1e440c5a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4ab0a640 1 -- 172.21.15.32:0/1264375916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e440b0b30 msgr2=0x7f1e4414fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4ab0a640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e440b0b30 0x7f1e4414fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4ab0a640 1 -- 172.21.15.32:0/1264375916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e34009420 con 0x7f1e441599e0 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4a309640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e440aece0 0x7f1e440c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e49b08640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e440b0b30 0x7f1e4414fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4ab0a640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e441599e0 0x7f1e440c7f60 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00efc0 tx=0x7f1e3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e3affd640 1 -- 172.21.15.32:0/1264375916 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c014480 con 0x7f1e441599e0 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e3affd640 1 -- 172.21.15.32:0/1264375916 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e3c00f040 con 0x7f1e441599e0 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1264375916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e4415c5d0 con 0x7f1e441599e0 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e3affd640 1 -- 172.21.15.32:0/1264375916 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e3c014820 con 0x7f1e441599e0 2023-12-20T04:47:28.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1264375916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e441500b0 con 0x7f1e441599e0 2023-12-20T04:47:28.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1264375916 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e4406acf0 con 0x7f1e441599e0 2023-12-20T04:47:28.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.254+0000 7f1e3affd640 1 -- 172.21.15.32:0/1264375916 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1e3c014980 con 0x7f1e441599e0 2023-12-20T04:47:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.258+0000 7f1e3affd640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1e28065b80 0x7f1e28068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.258+0000 7f1e3affd640 1 -- 172.21.15.32:0/1264375916 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f1e3c083fc0 con 0x7f1e441599e0 2023-12-20T04:47:28.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.258+0000 7f1e4a309640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1e28065b80 0x7f1e28068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.258+0000 7f1e4a309640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1e28065b80 0x7f1e28068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e400016c0 tx=0x7f1e40009040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:28.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.262+0000 7f1e3affd640 1 -- 172.21.15.32:0/1264375916 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1e3c051730 con 0x7f1e441599e0 2023-12-20T04:47:28.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.442+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1264375916 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f1e4404e000 con 0x7f1e28065b80 2023-12-20T04:47:28.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.442+0000 7f1e3affd640 1 -- 172.21.15.32:0/1264375916 <== mgr.8172 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f1e4404e000 con 0x7f1e28065b80 2023-12-20T04:47:28.442 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T04:47:28.442 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T04:47:28.442 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T04:47:28.442 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T04:47:28.442 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T04:47:28.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1264375916 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1e28065b80 msgr2=0x7f1e28068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:28.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1e28065b80 0x7f1e28068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f1e400016c0 tx=0x7f1e40009040 comp rx=0 tx=0).stop 2023-12-20T04:47:28.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1264375916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e441599e0 msgr2=0x7f1e440c7f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:28.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e441599e0 0x7f1e440c7f60 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c00efc0 tx=0x7f1e3c00c490 comp rx=0 tx=0).stop 2023-12-20T04:47:28.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1264375916 shutdown_connections 2023-12-20T04:47:28.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1e28065b80 0x7f1e28068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e441599e0 0x7f1e440c7f60 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e440b0b30 0x7f1e4414fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 --2- 172.21.15.32:0/1264375916 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e440aece0 0x7f1e440c5a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1264375916 >> 172.21.15.32:0/1264375916 conn(0x7f1e4406a320 msgr2=0x7f1e4406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:28.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1264375916 shutdown_connections 2023-12-20T04:47:28.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.446+0000 7f1e4c56d640 1 -- 172.21.15.32:0/1264375916 wait complete. 2023-12-20T04:47:28.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T04:47:28.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.610+0000 7f5528a03640 1 Processor -- start 2023-12-20T04:47:28.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.610+0000 7f5528a03640 1 -- start start 2023-12-20T04:47:28.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.610+0000 7f5528a03640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f552415cc30 0x7f55240b8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.610+0000 7f5528a03640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55240d0ca0 0x7f55240d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.610+0000 7f5528a03640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55240b6b00 con 0x7f55240d2e90 2023-12-20T04:47:28.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.610+0000 7f5528a03640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55240d3320 con 0x7f55240d0ca0 2023-12-20T04:47:28.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.610+0000 7f5528a03640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552415d010 con 0x7f552415cc30 2023-12-20T04:47:28.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.610+0000 7f552259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f55240d0ca0 0x7f55240d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.610+0000 7f552359e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f55240d2e90 0x7f55240b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43966/0 (socket says 172.21.15.32:43966) 2023-12-20T04:47:28.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.610+0000 7f552359e640 1 -- 172.21.15.32:0/3085154222 learned_addr learned my addr 172.21.15.32:0/3085154222 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:28.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f5522d9d640 1 --2- 172.21.15.32:0/3085154222 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f552415cc30 0x7f55240b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f552259c640 1 -- 172.21.15.32:0/3085154222 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f552415cc30 msgr2=0x7f55240b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:28.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f552259c640 1 --2- 172.21.15.32:0/3085154222 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f552415cc30 0x7f55240b8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f552259c640 1 -- 172.21.15.32:0/3085154222 >> v1:172.21.15.32:6789/0 conn(0x7f55240d2e90 legacy=0x7f55240b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:28.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f552259c640 1 -- 172.21.15.32:0/3085154222 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55240d2090 con 0x7f55240d0ca0 2023-12-20T04:47:28.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f5521d9b640 1 -- 172.21.15.32:0/3085154222 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1147484256 0 0) 0x7f551c0014a0 con 0x7f55240d2e90 2023-12-20T04:47:28.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f552259c640 1 --2- 172.21.15.32:0/3085154222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55240d0ca0 0x7f55240d2950 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f55180097f0 tx=0x7f551801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efe83e1c30ee1050 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:28.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f5521d9b640 1 -- 172.21.15.32:0/3085154222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55180ab600 con 0x7f55240d0ca0 2023-12-20T04:47:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f5521d9b640 1 -- 172.21.15.32:0/3085154222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55180a7050 con 0x7f55240d0ca0 2023-12-20T04:47:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f5521d9b640 1 -- 172.21.15.32:0/3085154222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55180ab980 con 0x7f55240d0ca0 2023-12-20T04:47:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f5528a03640 1 -- 172.21.15.32:0/3085154222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55240d0ca0 msgr2=0x7f55240d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.614+0000 7f5528a03640 1 --2- 172.21.15.32:0/3085154222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55240d0ca0 0x7f55240d2950 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f55180097f0 tx=0x7f551801c450 comp rx=0 tx=0).stop 2023-12-20T04:47:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 -- 172.21.15.32:0/3085154222 shutdown_connections 2023-12-20T04:47:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 --2- 172.21.15.32:0/3085154222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55240d0ca0 0x7f55240d2950 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 --2- 172.21.15.32:0/3085154222 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f552415cc30 0x7f55240b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 -- 172.21.15.32:0/3085154222 >> 172.21.15.32:0/3085154222 conn(0x7f5524065ce0 msgr2=0x7f55240660f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 -- 172.21.15.32:0/3085154222 shutdown_connections 2023-12-20T04:47:28.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 -- 172.21.15.32:0/3085154222 wait complete. 2023-12-20T04:47:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 Processor -- start 2023-12-20T04:47:28.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 -- start start 2023-12-20T04:47:28.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55240d0ca0 0x7f5524157710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55240d2e90 0x7f55241585f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f552415cc30 0x7f552415efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5524158b30 con 0x7f55240d2e90 2023-12-20T04:47:28.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f552415ab90 con 0x7f55240d0ca0 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5528a03640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f552415d550 con 0x7f552415cc30 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f552259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55240d2e90 0x7f55241585f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f552259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55240d2e90 0x7f55241585f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35230/0 (socket says 172.21.15.32:35230) 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f552259c640 1 -- 172.21.15.32:0/911534710 learned_addr learned my addr 172.21.15.32:0/911534710 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f552359e640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f552415cc30 0x7f552415efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f5522d9d640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55240d0ca0 0x7f5524157710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f552259c640 1 -- 172.21.15.32:0/911534710 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f552415cc30 msgr2=0x7f552415efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f552259c640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f552415cc30 0x7f552415efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f552259c640 1 -- 172.21.15.32:0/911534710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55240d0ca0 msgr2=0x7f5524157710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f552259c640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55240d0ca0 0x7f5524157710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.618+0000 7f552259c640 1 -- 172.21.15.32:0/911534710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5518009420 con 0x7f55240d2e90 2023-12-20T04:47:28.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.622+0000 7f552259c640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55240d2e90 0x7f55241585f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f5518009740 tx=0x7f55180191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:28.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.622+0000 7f550b7fe640 1 -- 172.21.15.32:0/911534710 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518004220 con 0x7f55240d2e90 2023-12-20T04:47:28.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.622+0000 7f550b7fe640 1 -- 172.21.15.32:0/911534710 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5518004380 con 0x7f55240d2e90 2023-12-20T04:47:28.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.622+0000 7f550b7fe640 1 -- 172.21.15.32:0/911534710 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5518003820 con 0x7f55240d2e90 2023-12-20T04:47:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.622+0000 7f5528a03640 1 -- 172.21.15.32:0/911534710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5524148e30 con 0x7f55240d2e90 2023-12-20T04:47:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.622+0000 7f5528a03640 1 -- 172.21.15.32:0/911534710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5524149080 con 0x7f55240d2e90 2023-12-20T04:47:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.622+0000 7f5528a03640 1 -- 172.21.15.32:0/911534710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5524074100 con 0x7f55240d2e90 2023-12-20T04:47:28.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.622+0000 7f550b7fe640 1 -- 172.21.15.32:0/911534710 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 72) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f55180b2040 con 0x7f55240d2e90 2023-12-20T04:47:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.622+0000 7f550b7fe640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5510065b80 0x7f5510068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.626+0000 7f550b7fe640 1 -- 172.21.15.32:0/911534710 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f551811d270 con 0x7f55240d2e90 2023-12-20T04:47:28.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.626+0000 7f5522d9d640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5510065b80 0x7f5510068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:28.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.626+0000 7f5522d9d640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5510065b80 0x7f5510068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f551400b9d0 tx=0x7f5514008040 comp rx=0 tx=0).ready entity=mgr.8172 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:28.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.630+0000 7f550b7fe640 1 -- 172.21.15.32:0/911534710 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f55180eaa60 con 0x7f55240d2e90 2023-12-20T04:47:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:28.826+0000 7f5528a03640 1 -- 172.21.15.32:0/911534710 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f5524155e70 con 0x7f55240d2e90 2023-12-20T04:47:29.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.102+0000 7f550b7fe640 1 -- 172.21.15.32:0/911534710 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 73) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f55180eae30 con 0x7f55240d2e90 2023-12-20T04:47:29.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f550b7fe640 1 -- 172.21.15.32:0/911534710 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v73) v1 ==== 92+0+0 (secure 0 0 0) 0x7f5518102150 con 0x7f55240d2e90 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 -- 172.21.15.32:0/911534710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5510065b80 msgr2=0x7f5510068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5510065b80 0x7f5510068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f551400b9d0 tx=0x7f5514008040 comp rx=0 tx=0).stop 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 -- 172.21.15.32:0/911534710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55240d2e90 msgr2=0x7f55241585f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55240d2e90 0x7f55241585f0 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f5518009740 tx=0x7f55180191f0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 -- 172.21.15.32:0/911534710 shutdown_connections 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5510065b80 0x7f5510068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f552415cc30 0x7f552415efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55240d2e90 0x7f55241585f0 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 --2- 172.21.15.32:0/911534710 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55240d0ca0 0x7f5524157710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 -- 172.21.15.32:0/911534710 >> 172.21.15.32:0/911534710 conn(0x7f5524065ce0 msgr2=0x7f55240b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 -- 172.21.15.32:0/911534710 shutdown_connections 2023-12-20T04:47:29.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.106+0000 7f5528a03640 1 -- 172.21.15.32:0/911534710 wait complete. 2023-12-20T04:47:29.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T04:47:29.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.278+0000 7f8fcfe66640 1 Processor -- start 2023-12-20T04:47:29.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.278+0000 7f8fcfe66640 1 -- start start 2023-12-20T04:47:29.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fcfe66640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8fc8070230 0x7f8fc8070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:29.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fcfe66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc80cb0b0 con 0x7f8fc8164670 2023-12-20T04:47:29.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fcfe66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8fc8070b50 con 0x7f8fc80ca1a0 2023-12-20T04:47:29.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fcfe66640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8fc80ca630 con 0x7f8fc8070230 2023-12-20T04:47:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fcdc02640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8fc8164670 0x7f8fc806da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53130/0 (socket says 172.21.15.32:53130) 2023-12-20T04:47:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fcd401640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8fc80ca1a0 0x7f8fc80cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:58496/0 (socket says 172.21.15.32:58496) 2023-12-20T04:47:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fcdc02640 1 -- 172.21.15.32:0/2709890400 learned_addr learned my addr 172.21.15.32:0/2709890400 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fce403640 1 --2- 172.21.15.32:0/2709890400 >> v2:172.21.15.32:3300/0 conn(0x7f8fc8070230 0x7f8fc8070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fccc00640 1 -- 172.21.15.32:0/2709890400 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3571999216 0 0) 0x7f8fc8070b50 con 0x7f8fc80ca1a0 2023-12-20T04:47:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fccc00640 1 -- 172.21.15.32:0/2709890400 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fb4003530 con 0x7f8fc80ca1a0 2023-12-20T04:47:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fccc00640 1 -- 172.21.15.32:0/2709890400 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2427213076 0 0) 0x7f8fc80cb0b0 con 0x7f8fc8164670 2023-12-20T04:47:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fccc00640 1 -- 172.21.15.32:0/2709890400 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8fc8070b50 con 0x7f8fc8164670 2023-12-20T04:47:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fce403640 1 -- 172.21.15.32:0/2709890400 >> v1:172.21.15.39:6789/0 conn(0x7f8fc80ca1a0 legacy=0x7f8fc80cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fce403640 1 -- 172.21.15.32:0/2709890400 >> v1:172.21.15.32:6789/0 conn(0x7f8fc8164670 legacy=0x7f8fc806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.282+0000 7f8fce403640 1 -- 172.21.15.32:0/2709890400 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc80ca810 con 0x7f8fc8070230 2023-12-20T04:47:29.281 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:29.285+0000 7f0a24c2e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:47:29.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.286+0000 7f8fce403640 1 --2- 172.21.15.32:0/2709890400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc8070230 0x7f8fc8070610 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009d90 tx=0x7f8fc001c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a9edfd23e89e8c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:29.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.286+0000 7f8fccc00640 1 -- 172.21.15.32:0/2709890400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00a5800 con 0x7f8fc8070230 2023-12-20T04:47:29.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.286+0000 7f8fccc00640 1 -- 172.21.15.32:0/2709890400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc00a5960 con 0x7f8fc8070230 2023-12-20T04:47:29.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.286+0000 7f8fccc00640 1 -- 172.21.15.32:0/2709890400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc00a5cf0 con 0x7f8fc8070230 2023-12-20T04:47:29.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.286+0000 7f8fcfe66640 1 -- 172.21.15.32:0/2709890400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc8070230 msgr2=0x7f8fc8070610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.286+0000 7f8fcfe66640 1 --2- 172.21.15.32:0/2709890400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc8070230 0x7f8fc8070610 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f8fc0009d90 tx=0x7f8fc001c830 comp rx=0 tx=0).stop 2023-12-20T04:47:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.286+0000 7f8fcfe66640 1 -- 172.21.15.32:0/2709890400 shutdown_connections 2023-12-20T04:47:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.286+0000 7f8fcfe66640 1 --2- 172.21.15.32:0/2709890400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc8070230 0x7f8fc8070610 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.286+0000 7f8fcfe66640 1 -- 172.21.15.32:0/2709890400 >> 172.21.15.32:0/2709890400 conn(0x7f8fc806a510 msgr2=0x7f8fc80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.286+0000 7f8fcfe66640 1 -- 172.21.15.32:0/2709890400 shutdown_connections 2023-12-20T04:47:29.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.286+0000 7f8fcfe66640 1 -- 172.21.15.32:0/2709890400 wait complete. 2023-12-20T04:47:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcfe66640 1 Processor -- start 2023-12-20T04:47:29.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcfe66640 1 -- start start 2023-12-20T04:47:29.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcfe66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fc8070230 0x7f8fc815df40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:29.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcfe66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fc80ca1a0 0x7f8fc81499a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:29.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcfe66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc8164670 0x7f8fc80cd170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:29.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcfe66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc80b14c0 con 0x7f8fc8164670 2023-12-20T04:47:29.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcfe66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc814ca90 con 0x7f8fc80ca1a0 2023-12-20T04:47:29.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcfe66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8fc815e480 con 0x7f8fc8070230 2023-12-20T04:47:29.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fce403640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc8164670 0x7f8fc80cd170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:29.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcdc02640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fc8070230 0x7f8fc815df40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:29.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcd401640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fc80ca1a0 0x7f8fc81499a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:29.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcdc02640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fc8070230 0x7f8fc815df40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:51836/0 (socket says 172.21.15.32:51836) 2023-12-20T04:47:29.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcdc02640 1 -- 172.21.15.32:0/3828547134 learned_addr learned my addr 172.21.15.32:0/3828547134 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:29.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcdc02640 1 -- 172.21.15.32:0/3828547134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fc80ca1a0 msgr2=0x7f8fc81499a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcdc02640 1 --2- 172.21.15.32:0/3828547134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fc80ca1a0 0x7f8fc81499a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcdc02640 1 -- 172.21.15.32:0/3828547134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc8164670 msgr2=0x7f8fc80cd170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcdc02640 1 --2- 172.21.15.32:0/3828547134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc8164670 0x7f8fc80cd170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcdc02640 1 -- 172.21.15.32:0/3828547134 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8fc0009a70 con 0x7f8fc8070230 2023-12-20T04:47:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcdc02640 1 --2- 172.21.15.32:0/3828547134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fc8070230 0x7f8fc815df40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40011f0 tx=0x7f8fc400c960 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fbe7fc640 1 -- 172.21.15.32:0/3828547134 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4005cf0 con 0x7f8fc8070230 2023-12-20T04:47:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fbe7fc640 1 -- 172.21.15.32:0/3828547134 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8fc4005e50 con 0x7f8fc8070230 2023-12-20T04:47:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fbe7fc640 1 -- 172.21.15.32:0/3828547134 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8fc4007270 con 0x7f8fc8070230 2023-12-20T04:47:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcfe66640 1 -- 172.21.15.32:0/3828547134 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8fc80b3590 con 0x7f8fc8070230 2023-12-20T04:47:29.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.290+0000 7f8fcfe66640 1 -- 172.21.15.32:0/3828547134 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8fc80b19f0 con 0x7f8fc8070230 2023-12-20T04:47:29.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.294+0000 7f8fcfe66640 1 -- 172.21.15.32:0/3828547134 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8fc804e000 con 0x7f8fc8070230 2023-12-20T04:47:29.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.294+0000 7f8fbe7fc640 1 -- 172.21.15.32:0/3828547134 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 73) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f8fc40073d0 con 0x7f8fc8070230 2023-12-20T04:47:29.292 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:29.294+0000 7f1a7a42f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T04:47:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.294+0000 7f8fbe7fc640 1 --2- 172.21.15.32:0/3828547134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fac065c00 0x7f8fac0680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.294+0000 7f8fcd401640 1 -- 172.21.15.32:0/3828547134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fac065c00 msgr2=0x7f8fac0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.294+0000 7f8fbe7fc640 1 -- 172.21.15.32:0/3828547134 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8fc4082820 con 0x7f8fc8070230 2023-12-20T04:47:29.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.294+0000 7f8fcd401640 1 --2- 172.21.15.32:0/3828547134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fac065c00 0x7f8fac0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:29.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.298+0000 7f8fbe7fc640 1 -- 172.21.15.32:0/3828547134 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8fc4050150 con 0x7f8fc8070230 2023-12-20T04:47:29.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.458+0000 7f8fcfe66640 1 -- 172.21.15.32:0/3828547134 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f8fc80cf4d0 con 0x7f8fc8070230 2023-12-20T04:47:29.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fbe7fc640 1 -- 172.21.15.32:0/3828547134 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v47)=0 v47) v1 ==== 138+0+0 (secure 0 0 0) 0x7f8fc4017020 con 0x7f8fc8070230 2023-12-20T04:47:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 -- 172.21.15.32:0/3828547134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fac065c00 msgr2=0x7f8fac0680c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:29.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 --2- 172.21.15.32:0/3828547134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fac065c00 0x7f8fac0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 -- 172.21.15.32:0/3828547134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fc8070230 msgr2=0x7f8fc815df40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 --2- 172.21.15.32:0/3828547134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fc8070230 0x7f8fc815df40 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f8fc40011f0 tx=0x7f8fc400c960 comp rx=0 tx=0).stop 2023-12-20T04:47:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 -- 172.21.15.32:0/3828547134 shutdown_connections 2023-12-20T04:47:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 --2- 172.21.15.32:0/3828547134 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8fac065c00 0x7f8fac0680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 --2- 172.21.15.32:0/3828547134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8fc8164670 0x7f8fc80cd170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 --2- 172.21.15.32:0/3828547134 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8fc80ca1a0 0x7f8fc81499a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 --2- 172.21.15.32:0/3828547134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8fc8070230 0x7f8fc815df40 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 -- 172.21.15.32:0/3828547134 >> 172.21.15.32:0/3828547134 conn(0x7f8fc806a510 msgr2=0x7f8fc81632d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 -- 172.21.15.32:0/3828547134 shutdown_connections 2023-12-20T04:47:29.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.466+0000 7f8fcfe66640 1 -- 172.21.15.32:0/3828547134 wait complete. 2023-12-20T04:47:29.487 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T04:47:29.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3ff39640 1 Processor -- start 2023-12-20T04:47:29.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3ff39640 1 -- start start 2023-12-20T04:47:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3ff39640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ffb380cb050 0x7ffb380cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3ff39640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb38065d40 0x7ffb38161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3ff39640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb3806a1f0 con 0x7ffb3806ca60 2023-12-20T04:47:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3ff39640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb38161860 con 0x7ffb38065d40 2023-12-20T04:47:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3ff39640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb3806a440 con 0x7ffb380cb050 2023-12-20T04:47:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3d4d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb38065d40 0x7ffb38161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:29.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3d4d4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb38065d40 0x7ffb38161320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59438/0 (socket says 172.21.15.32:59438) 2023-12-20T04:47:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3d4d4640 1 -- 172.21.15.32:0/1307299886 learned_addr learned my addr 172.21.15.32:0/1307299886 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3dcd5640 1 --2- 172.21.15.32:0/1307299886 >> v2:172.21.15.39:3300/0 conn(0x7ffb380cb050 0x7ffb380cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3d4d4640 1 -- 172.21.15.32:0/1307299886 >> v2:172.21.15.39:3300/0 conn(0x7ffb380cb050 msgr2=0x7ffb380cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3d4d4640 1 --2- 172.21.15.32:0/1307299886 >> v2:172.21.15.39:3300/0 conn(0x7ffb380cb050 0x7ffb380cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3d4d4640 1 -- 172.21.15.32:0/1307299886 >> v1:172.21.15.39:6789/0 conn(0x7ffb3806ca60 legacy=0x7ffb38070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.638+0000 7ffb3d4d4640 1 -- 172.21.15.32:0/1307299886 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb3806a720 con 0x7ffb38065d40 2023-12-20T04:47:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3d4d4640 1 --2- 172.21.15.32:0/1307299886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb38065d40 0x7ffb38161320 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7ffb38071ad0 tx=0x7ffb2801c590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a88da07cb212d392 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3ccd3640 1 -- 172.21.15.32:0/1307299886 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280ac600 con 0x7ffb38065d40 2023-12-20T04:47:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3ccd3640 1 -- 172.21.15.32:0/1307299886 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb280ac760 con 0x7ffb38065d40 2023-12-20T04:47:29.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3ccd3640 1 -- 172.21.15.32:0/1307299886 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb280acaf0 con 0x7ffb38065d40 2023-12-20T04:47:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1307299886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb38065d40 msgr2=0x7ffb38161320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3ff39640 1 --2- 172.21.15.32:0/1307299886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb38065d40 0x7ffb38161320 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7ffb38071ad0 tx=0x7ffb2801c590 comp rx=0 tx=0).stop 2023-12-20T04:47:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1307299886 shutdown_connections 2023-12-20T04:47:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3ff39640 1 --2- 172.21.15.32:0/1307299886 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb38065d40 0x7ffb38161320 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3ff39640 1 --2- 172.21.15.32:0/1307299886 >> v2:172.21.15.39:3300/0 conn(0x7ffb380cb050 0x7ffb380cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1307299886 >> 172.21.15.32:0/1307299886 conn(0x7ffb380cf8d0 msgr2=0x7ffb380cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:29.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1307299886 shutdown_connections 2023-12-20T04:47:29.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.642+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1307299886 wait complete. 2023-12-20T04:47:29.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3ff39640 1 Processor -- start 2023-12-20T04:47:29.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3ff39640 1 -- start start 2023-12-20T04:47:29.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3ff39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb38065d40 0x7ffb38156ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3ff39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb3806ca60 0x7ffb38153980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3ff39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb380cb050 0x7ffb38159940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3ff39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb38156ab0 con 0x7ffb380cb050 2023-12-20T04:47:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3ff39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb38153ff0 con 0x7ffb38065d40 2023-12-20T04:47:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3ff39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb381563f0 con 0x7ffb3806ca60 2023-12-20T04:47:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3e4d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb380cb050 0x7ffb38159940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3e4d6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb380cb050 0x7ffb38159940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59454/0 (socket says 172.21.15.32:59454) 2023-12-20T04:47:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3e4d6640 1 -- 172.21.15.32:0/1152898787 learned_addr learned my addr 172.21.15.32:0/1152898787 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3dcd5640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb38065d40 0x7ffb38156ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3d4d4640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb3806ca60 0x7ffb38153980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:29.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3e4d6640 1 -- 172.21.15.32:0/1152898787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb3806ca60 msgr2=0x7ffb38153980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3e4d6640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb3806ca60 0x7ffb38153980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3e4d6640 1 -- 172.21.15.32:0/1152898787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb38065d40 msgr2=0x7ffb38156ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3e4d6640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb38065d40 0x7ffb38156ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3e4d6640 1 -- 172.21.15.32:0/1152898787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb280094d0 con 0x7ffb380cb050 2023-12-20T04:47:29.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3e4d6640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb380cb050 0x7ffb38159940 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000c770 tx=0x7ffb3000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:29.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb2e7fc640 1 -- 172.21.15.32:0/1152898787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb30003b80 con 0x7ffb380cb050 2023-12-20T04:47:29.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb2e7fc640 1 -- 172.21.15.32:0/1152898787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb30003ce0 con 0x7ffb380cb050 2023-12-20T04:47:29.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb2e7fc640 1 -- 172.21.15.32:0/1152898787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb3000aa90 con 0x7ffb380cb050 2023-12-20T04:47:29.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1152898787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb381566d0 con 0x7ffb380cb050 2023-12-20T04:47:29.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.646+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1152898787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb381542d0 con 0x7ffb380cb050 2023-12-20T04:47:29.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.650+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1152898787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb38156bf0 con 0x7ffb380cb050 2023-12-20T04:47:29.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.650+0000 7ffb2e7fc640 1 -- 172.21.15.32:0/1152898787 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 73) v1 ==== 85347+0+0 (secure 0 0 0) 0x7ffb3000abf0 con 0x7ffb380cb050 2023-12-20T04:47:29.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.650+0000 7ffb2e7fc640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffb1c065bd0 0x7ffb1c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:29.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.650+0000 7ffb3dcd5640 1 -- 172.21.15.32:0/1152898787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffb1c065bd0 msgr2=0x7ffb1c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:29.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.650+0000 7ffb2e7fc640 1 -- 172.21.15.32:0/1152898787 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ffb30083080 con 0x7ffb380cb050 2023-12-20T04:47:29.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.650+0000 7ffb3dcd5640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffb1c065bd0 0x7ffb1c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:29.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.654+0000 7ffb2e7fc640 1 -- 172.21.15.32:0/1152898787 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffb300b40a0 con 0x7ffb380cb050 2023-12-20T04:47:29.717 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:29.721+0000 7f0a24c2e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:47:29.742 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:29.742+0000 7f1a7a42f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T04:47:29.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.814+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1152898787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7ffb3804e000 con 0x7ffb380cb050 2023-12-20T04:47:29.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb2e7fc640 1 -- 172.21.15.32:0/1152898787 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v48)=0 v48) v1 ==== 133+0+0 (secure 0 0 0) 0x7ffb38156bf0 con 0x7ffb380cb050 2023-12-20T04:47:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1152898787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffb1c065bd0 msgr2=0x7ffb1c068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffb1c065bd0 0x7ffb1c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1152898787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb380cb050 msgr2=0x7ffb38159940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb380cb050 0x7ffb38159940 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7ffb3000c770 tx=0x7ffb3000cc30 comp rx=0 tx=0).stop 2023-12-20T04:47:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1152898787 shutdown_connections 2023-12-20T04:47:29.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ffb1c065bd0 0x7ffb1c068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb380cb050 0x7ffb38159940 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb3806ca60 0x7ffb38153980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 --2- 172.21.15.32:0/1152898787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb38065d40 0x7ffb38156ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1152898787 >> 172.21.15.32:0/1152898787 conn(0x7ffb380cf8d0 msgr2=0x7ffb38155ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1152898787 shutdown_connections 2023-12-20T04:47:29.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:29.822+0000 7ffb3ff39640 1 -- 172.21.15.32:0/1152898787 wait complete. 2023-12-20T04:47:29.841 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T04:47:29.841 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-20T04:47:29.856 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1 1M 2023-12-20T04:47:29.874 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:29.877+0000 7f0a24c2e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:47:29.901 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:29.902+0000 7f1a7a42f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T04:47:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.046+0000 7f8dd6125640 1 Processor -- start 2023-12-20T04:47:30.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.046+0000 7f8dd6125640 1 -- start start 2023-12-20T04:47:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.046+0000 7f8dd6125640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8dd015ce60 0x7f8dd00d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.046+0000 7f8dd6125640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d14d0 0x7f8dd00b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.046+0000 7f8dd6125640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8dd00d2650 con 0x7f8dd00d3130 2023-12-20T04:47:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.046+0000 7f8dd6125640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8dd00d35a0 con 0x7f8dd015ce60 2023-12-20T04:47:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.046+0000 7f8dd6125640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd00d1a40 con 0x7f8dd00d14d0 2023-12-20T04:47:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.050+0000 7f8dcffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d14d0 0x7f8dd00b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.050+0000 7f8dceffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f8dd00d3130 0x7f8dd00d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:58510/0 (socket says 172.21.15.32:58510) 2023-12-20T04:47:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.050+0000 7f8dceffd640 1 -- 172.21.15.32:0/1797183530 learned_addr learned my addr 172.21.15.32:0/1797183530 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.050+0000 7f8dcf7fe640 1 --2- 172.21.15.32:0/1797183530 >> v2:172.21.15.39:3300/0 conn(0x7f8dd015ce60 0x7f8dd00d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.050+0000 7f8dcffff640 1 -- 172.21.15.32:0/1797183530 >> v2:172.21.15.39:3300/0 conn(0x7f8dd015ce60 msgr2=0x7f8dd00d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:30.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.050+0000 7f8dcffff640 1 --2- 172.21.15.32:0/1797183530 >> v2:172.21.15.39:3300/0 conn(0x7f8dd015ce60 0x7f8dd00d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:30.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.050+0000 7f8dce7fc640 1 -- 172.21.15.32:0/1797183530 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008251941 0 0) 0x7f8dd00d2650 con 0x7f8dd00d3130 2023-12-20T04:47:30.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.050+0000 7f8dcffff640 1 -- 172.21.15.32:0/1797183530 >> v1:172.21.15.39:6789/0 conn(0x7f8dd00d3130 legacy=0x7f8dd00d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:30.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.050+0000 7f8dcffff640 1 -- 172.21.15.32:0/1797183530 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dd00d1d20 con 0x7f8dd00d14d0 2023-12-20T04:47:30.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.050+0000 7f8dcf7fe640 1 --2- 172.21.15.32:0/1797183530 >> v2:172.21.15.39:3300/0 conn(0x7f8dd015ce60 0x7f8dd00d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dcffff640 1 --2- 172.21.15.32:0/1797183530 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d14d0 0x7f8dd00b5e90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009d90 tx=0x7f8dc801f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3af931559047c301 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dce7fc640 1 -- 172.21.15.32:0/1797183530 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80af600 con 0x7f8dd00d14d0 2023-12-20T04:47:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dce7fc640 1 -- 172.21.15.32:0/1797183530 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc80ab050 con 0x7f8dd00d14d0 2023-12-20T04:47:30.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dce7fc640 1 -- 172.21.15.32:0/1797183530 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc80af980 con 0x7f8dd00d14d0 2023-12-20T04:47:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dd6125640 1 -- 172.21.15.32:0/1797183530 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d14d0 msgr2=0x7f8dd00b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dd6125640 1 --2- 172.21.15.32:0/1797183530 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d14d0 0x7f8dd00b5e90 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f8dc8009d90 tx=0x7f8dc801f130 comp rx=0 tx=0).stop 2023-12-20T04:47:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dd6125640 1 -- 172.21.15.32:0/1797183530 shutdown_connections 2023-12-20T04:47:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dd6125640 1 --2- 172.21.15.32:0/1797183530 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d14d0 0x7f8dd00b5e90 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dd6125640 1 --2- 172.21.15.32:0/1797183530 >> v2:172.21.15.39:3300/0 conn(0x7f8dd015ce60 0x7f8dd00d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:30.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dd6125640 1 -- 172.21.15.32:0/1797183530 >> 172.21.15.32:0/1797183530 conn(0x7f8dd006b5e0 msgr2=0x7f8dd006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dd6125640 1 -- 172.21.15.32:0/1797183530 shutdown_connections 2023-12-20T04:47:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.054+0000 7f8dd6125640 1 -- 172.21.15.32:0/1797183530 wait complete. 2023-12-20T04:47:30.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dd6125640 1 Processor -- start 2023-12-20T04:47:30.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dd6125640 1 -- start start 2023-12-20T04:47:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dd6125640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dd00d14d0 0x7f8dd0153cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dd6125640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d3130 0x7f8dd01f5f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:30.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dd6125640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd015ce60 0x7f8dd015bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dd6125640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd01f2f20 con 0x7f8dd00d14d0 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dd6125640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd01f64b0 con 0x7f8dd015ce60 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dd6125640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8dd0154580 con 0x7f8dd00d3130 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dcf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dd00d14d0 0x7f8dd0153cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dcf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dd00d14d0 0x7f8dd0153cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59460/0 (socket says 172.21.15.32:59460) 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dcf7fe640 1 -- 172.21.15.32:0/2066826746 learned_addr learned my addr 172.21.15.32:0/2066826746 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dceffd640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d3130 0x7f8dd01f5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dcffff640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd015ce60 0x7f8dd015bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dcf7fe640 1 -- 172.21.15.32:0/2066826746 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d3130 msgr2=0x7f8dd01f5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dcf7fe640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d3130 0x7f8dd01f5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dcf7fe640 1 -- 172.21.15.32:0/2066826746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd015ce60 msgr2=0x7f8dd015bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dcf7fe640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd015ce60 0x7f8dd015bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dcf7fe640 1 -- 172.21.15.32:0/2066826746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8dc0009420 con 0x7f8dd00d14d0 2023-12-20T04:47:30.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dceffd640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d3130 0x7f8dd01f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:30.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dcf7fe640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dd00d14d0 0x7f8dd0153cb0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000e980 tx=0x7f8dc000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:30.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8db7fff640 1 -- 172.21.15.32:0/2066826746 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc000c9a0 con 0x7f8dd00d14d0 2023-12-20T04:47:30.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8db7fff640 1 -- 172.21.15.32:0/2066826746 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8dc000cb00 con 0x7f8dd00d14d0 2023-12-20T04:47:30.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dd6125640 1 -- 172.21.15.32:0/2066826746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8dc8009a70 con 0x7f8dd00d14d0 2023-12-20T04:47:30.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8db7fff640 1 -- 172.21.15.32:0/2066826746 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8dc000ce90 con 0x7f8dd00d14d0 2023-12-20T04:47:30.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.058+0000 7f8dd6125640 1 -- 172.21.15.32:0/2066826746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8dd01f2160 con 0x7f8dd00d14d0 2023-12-20T04:47:30.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.062+0000 7f8dd6125640 1 -- 172.21.15.32:0/2066826746 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8dd004e000 con 0x7f8dd00d14d0 2023-12-20T04:47:30.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.062+0000 7f8db7fff640 1 -- 172.21.15.32:0/2066826746 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 73) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f8dc0013470 con 0x7f8dd00d14d0 2023-12-20T04:47:30.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.062+0000 7f8db7fff640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:30.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.062+0000 7f8db7fff640 1 -- 172.21.15.32:0/2066826746 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(138..138 src has 1..138) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8dc0083930 con 0x7f8dd00d14d0 2023-12-20T04:47:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.062+0000 7f8dceffd640 1 -- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 msgr2=0x7f8dbc068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:30.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.062+0000 7f8dceffd640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T04:47:30.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.066+0000 7f8db7fff640 1 -- 172.21.15.32:0/2066826746 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8dc00511d0 con 0x7f8dd00d14d0 2023-12-20T04:47:30.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.242+0000 7f8dd6125640 1 -- 172.21.15.32:0/2066826746 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dd00d22e0 con 0x7f8dbc065bd0 2023-12-20T04:47:30.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.266+0000 7f8dceffd640 1 -- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 msgr2=0x7f8dbc068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:30.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.266+0000 7f8dceffd640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T04:47:30.380 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:30.381+0000 7f0a24c2e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:47:30.417 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:30.418+0000 7f1a7a42f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T04:47:30.441 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:30.445+0000 7f0a24c2e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:47:30.478 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:30.482+0000 7f1a7a42f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T04:47:30.499 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:30.501+0000 7f0a24c2e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:47:30.537 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:30.538+0000 7f1a7a42f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T04:47:30.590 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:30.593+0000 7f0a24c2e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:47:30.628 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:30.630+0000 7f1a7a42f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T04:47:30.649 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:30.653+0000 7f0a24c2e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:47:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.670+0000 7f8dceffd640 1 -- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 msgr2=0x7f8dbc068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:30.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:30.670+0000 7f8dceffd640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T04:47:30.688 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:30.690+0000 7f1a7a42f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T04:47:30.737 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:30.741+0000 7f0a24c2e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:47:30.776 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:30.778+0000 7f1a7a42f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T04:47:30.796 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:30.801+0000 7f0a24c2e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:47:30.835 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:30.838+0000 7f1a7a42f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T04:47:30.947 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:30.949+0000 7f0a24c2e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:47:30.986 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:30.986+0000 7f1a7a42f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T04:47:31.008 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:31.009+0000 7f0a24c2e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:47:31.048 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:31.050+0000 7f1a7a42f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T04:47:31.257 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:47:31.258 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:47:31.258 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:47:31.266 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:31.269+0000 7f0a24c2e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:47:31.298 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T04:47:31.299 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T04:47:31.299 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T04:47:31.307 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:31.310+0000 7f1a7a42f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T04:47:31.408 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:31.409+0000 7f0a24c2e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:47:31.450 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:31.450+0000 7f1a7a42f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:47:31.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:31.474+0000 7f8dceffd640 1 -- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 msgr2=0x7f8dbc068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:31.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:31.474+0000 7f8dceffd640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T04:47:31.539 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:31.541+0000 7f0a24c2e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:47:31.581 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:31.582+0000 7f1a7a42f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T04:47:31.613 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:31.617+0000 7f0a24c2e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:47:31.656 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:31.658+0000 7f1a7a42f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T04:47:31.750 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:47:31.754 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:47:31.758 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:31.757+0000 7f0a24c2e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:47:31.800 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:31.802+0000 7f1a7a42f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T04:47:32.322 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:32.325+0000 7f0a24c2e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:47:32.371 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:32.374+0000 7f1a7a42f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T04:47:32.410 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:32.413+0000 7f0a24c2e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:47:32.460 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:32.462+0000 7f1a7a42f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T04:47:32.568 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:32.569+0000 7f0a24c2e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:47:32.617 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:32.618+0000 7f1a7a42f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T04:47:32.657 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:32.657+0000 7f0a24c2e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:47:32.706 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:32.706+0000 7f1a7a42f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T04:47:32.989 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:32.993+0000 7f0a24c2e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:47:33.040 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.042+0000 7f1a7a42f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T04:47:33.053 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T04:47:33.057+0000 7f0a24c2e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:47:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:33.078+0000 7f8dceffd640 1 -- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 msgr2=0x7f8dbc068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T04:47:33.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:33.078+0000 7f8dceffd640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-20T04:47:33.105 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.106+0000 7f1a7a42f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T04:47:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:33.118+0000 7f8db7fff640 1 -- 172.21.15.32:0/2066826746 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 74) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f8dc00515a0 con 0x7f8dd00d14d0 2023-12-20T04:47:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:33.118+0000 7f8db7fff640 1 -- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 msgr2=0x7f8dbc068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T04:47:33.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:33.118+0000 7f8db7fff640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:33.240 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.242+0000 7f19faf20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:33.240 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.242+0000 7f19faf20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:33.240 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.242+0000 7f19faf20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:33.240 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.242+0000 7f19faf20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:33.240 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.242+0000 7f19faf20640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:33.240 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.242+0000 7f19fef28640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:33.240 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.242+0000 7f19fef28640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:33.240 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.242+0000 7f19fef28640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:33.241 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.242+0000 7f19fef28640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:33.241 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:47:33.242+0000 7f19fef28640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:47:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.122+0000 7f8db7fff640 1 -- 172.21.15.32:0/2066826746 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 75) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f8dc0052c40 con 0x7f8dd00d14d0 2023-12-20T04:47:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.126+0000 7f8db7fff640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.126+0000 7f8db7fff640 1 -- 172.21.15.32:0/2066826746 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f8dbc033610 con 0x7f8dbc065bd0 2023-12-20T04:47:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.126+0000 7f8dceffd640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.134+0000 7f8dceffd640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00b7cd0 tx=0x7f8dc4007e50 comp rx=0 tx=0).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:34.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.198+0000 7f8db7fff640 1 -- 172.21.15.32:0/2066826746 <== mgr.8314 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+38 (secure 0 0 0) 0x7f8dbc033610 con 0x7f8dbc065bd0 2023-12-20T04:47:34.196 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1 2023-12-20T04:47:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 -- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 msgr2=0x7f8dbc068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f8dd00b7cd0 tx=0x7f8dc4007e50 comp rx=0 tx=0).stop 2023-12-20T04:47:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 -- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dd00d14d0 msgr2=0x7f8dd0153cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dd00d14d0 0x7f8dd0153cb0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f8dc000e980 tx=0x7f8dc000ee40 comp rx=0 tx=0).stop 2023-12-20T04:47:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 -- 172.21.15.32:0/2066826746 shutdown_connections 2023-12-20T04:47:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8dbc065bd0 0x7f8dbc068090 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8dd015ce60 0x7f8dd015bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8dd00d3130 0x7f8dd01f5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 --2- 172.21.15.32:0/2066826746 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8dd00d14d0 0x7f8dd0153cb0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 -- 172.21.15.32:0/2066826746 >> 172.21.15.32:0/2066826746 conn(0x7f8dd006b5e0 msgr2=0x7f8dd01e5750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 -- 172.21.15.32:0/2066826746 shutdown_connections 2023-12-20T04:47:34.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.202+0000 7f8dd6125640 1 -- 172.21.15.32:0/2066826746 wait complete. 2023-12-20T04:47:34.220 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1 2023-12-20T04:47:34.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add /snap_test_dir1 55n 2023-12-20T04:47:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cbbae5640 1 Processor -- start 2023-12-20T04:47:34.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cbbae5640 1 -- start start 2023-12-20T04:47:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cbbae5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8cb415b790 0x7f8cb4070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cbbae5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8cb4070770 0x7f8cb40cd9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cbbae5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8cb4161670 con 0x7f8cb40cdef0 2023-12-20T04:47:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cbbae5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8cb40ce6a0 con 0x7f8cb415b790 2023-12-20T04:47:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cbbae5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb4162270 con 0x7f8cb4070770 2023-12-20T04:47:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cb9881640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8cb415b790 0x7f8cb4070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cb9881640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8cb415b790 0x7f8cb4070230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59594/0 (socket says 172.21.15.32:59594) 2023-12-20T04:47:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cb9881640 1 -- 172.21.15.32:0/3102438451 learned_addr learned my addr 172.21.15.32:0/3102438451 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cba082640 1 --1- 172.21.15.32:0/3102438451 >> v1:172.21.15.32:6789/0 conn(0x7f8cb40cdef0 0x7f8cb4164cb0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53134/0 (socket says 172.21.15.32:53134) 2023-12-20T04:47:34.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cb9080640 1 --2- 172.21.15.32:0/3102438451 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8cb4070770 0x7f8cb40cd9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cb9881640 1 -- 172.21.15.32:0/3102438451 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8cb4070770 msgr2=0x7f8cb40cd9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T04:47:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cb9881640 1 --2- 172.21.15.32:0/3102438451 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8cb4070770 0x7f8cb40cd9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cb9881640 1 -- 172.21.15.32:0/3102438451 >> v1:172.21.15.32:6789/0 conn(0x7f8cb40cdef0 legacy=0x7f8cb4164cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cb9881640 1 -- 172.21.15.32:0/3102438451 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb4162690 con 0x7f8cb415b790 2023-12-20T04:47:34.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.370+0000 7f8cb887f640 1 -- 172.21.15.32:0/3102438451 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2319948940 0 0) 0x7f8cb4161670 con 0x7f8cb40cdef0 2023-12-20T04:47:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cb9881640 1 --2- 172.21.15.32:0/3102438451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb415b790 0x7f8cb4070230 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009870 tx=0x7f8cb001c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a853d6bb8f21004 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cb887f640 1 -- 172.21.15.32:0/3102438451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0004230 con 0x7f8cb415b790 2023-12-20T04:47:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cb887f640 1 -- 172.21.15.32:0/3102438451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb00a7050 con 0x7f8cb415b790 2023-12-20T04:47:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cb887f640 1 -- 172.21.15.32:0/3102438451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0003620 con 0x7f8cb415b790 2023-12-20T04:47:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cbbae5640 1 -- 172.21.15.32:0/3102438451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb415b790 msgr2=0x7f8cb4070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cbbae5640 1 --2- 172.21.15.32:0/3102438451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb415b790 0x7f8cb4070230 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7f8cb0009870 tx=0x7f8cb001c540 comp rx=0 tx=0).stop 2023-12-20T04:47:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cbbae5640 1 -- 172.21.15.32:0/3102438451 shutdown_connections 2023-12-20T04:47:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cbbae5640 1 --2- 172.21.15.32:0/3102438451 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8cb4070770 0x7f8cb40cd9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cbbae5640 1 --2- 172.21.15.32:0/3102438451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb415b790 0x7f8cb4070230 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cbbae5640 1 -- 172.21.15.32:0/3102438451 >> 172.21.15.32:0/3102438451 conn(0x7f8cb406a510 msgr2=0x7f8cb40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cbbae5640 1 -- 172.21.15.32:0/3102438451 shutdown_connections 2023-12-20T04:47:34.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.374+0000 7f8cbbae5640 1 -- 172.21.15.32:0/3102438451 wait complete. 2023-12-20T04:47:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cbbae5640 1 Processor -- start 2023-12-20T04:47:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cbbae5640 1 -- start start 2023-12-20T04:47:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cbbae5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb4070770 0x7f8cb4156120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cbbae5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb40cdef0 0x7f8cb41568f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cbbae5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8cb415b790 0x7f8cb40cc7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cbbae5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb414cf10 con 0x7f8cb4070770 2023-12-20T04:47:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cbbae5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb4154720 con 0x7f8cb40cdef0 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cbbae5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8cb41549c0 con 0x7f8cb415b790 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cb9881640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb4070770 0x7f8cb4156120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cb9881640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb4070770 0x7f8cb4156120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59604/0 (socket says 172.21.15.32:59604) 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cb9881640 1 -- 172.21.15.32:0/99764063 learned_addr learned my addr 172.21.15.32:0/99764063 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cba082640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8cb415b790 0x7f8cb40cc7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cb9080640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb40cdef0 0x7f8cb41568f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cb9881640 1 -- 172.21.15.32:0/99764063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8cb415b790 msgr2=0x7f8cb40cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cb9881640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8cb415b790 0x7f8cb40cc7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cb9881640 1 -- 172.21.15.32:0/99764063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb40cdef0 msgr2=0x7f8cb41568f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cb9881640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb40cdef0 0x7f8cb41568f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cb9881640 1 -- 172.21.15.32:0/99764063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8cb00094d0 con 0x7f8cb4070770 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cba082640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8cb415b790 0x7f8cb40cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T04:47:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cb9881640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb4070770 0x7f8cb4156120 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00ae040 tx=0x7f8cb00192e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8caa7fc640 1 -- 172.21.15.32:0/99764063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb0003b70 con 0x7f8cb4070770 2023-12-20T04:47:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8caa7fc640 1 -- 172.21.15.32:0/99764063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8cb0003cd0 con 0x7f8cb4070770 2023-12-20T04:47:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8caa7fc640 1 -- 172.21.15.32:0/99764063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8cb00ab680 con 0x7f8cb4070770 2023-12-20T04:47:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cbbae5640 1 -- 172.21.15.32:0/99764063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8cb4154ca0 con 0x7f8cb4070770 2023-12-20T04:47:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.378+0000 7f8cbbae5640 1 -- 172.21.15.32:0/99764063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8cb4156e30 con 0x7f8cb4070770 2023-12-20T04:47:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.382+0000 7f8cbbae5640 1 -- 172.21.15.32:0/99764063 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8cb404e000 con 0x7f8cb4070770 2023-12-20T04:47:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.382+0000 7f8caa7fc640 1 -- 172.21.15.32:0/99764063 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 75) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f8cb00ab7e0 con 0x7f8cb4070770 2023-12-20T04:47:34.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.382+0000 7f8caa7fc640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c98065900 0x7f8c98067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.382+0000 7f8caa7fc640 1 -- 172.21.15.32:0/99764063 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f8cb011d760 con 0x7f8cb4070770 2023-12-20T04:47:34.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.382+0000 7f8cb9080640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c98065900 0x7f8c98067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.382+0000 7f8cb9080640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c98065900 0x7f8c98067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8cb415ab90 tx=0x7f8ca4009190 comp rx=0 tx=0).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:34.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.386+0000 7f8caa7fc640 1 -- 172.21.15.32:0/99764063 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8cb00eb2c0 con 0x7f8cb4070770 2023-12-20T04:47:34.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.566+0000 7f8cbbae5640 1 -- 172.21.15.32:0/99764063 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "/snap_test_dir1", "retention_spec_or_period": "55n", "target": ["mon-mgr", ""]}) v1 -- 0x7f8cb4164640 con 0x7f8c98065900 2023-12-20T04:47:34.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8caa7fc640 1 -- 172.21.15.32:0/99764063 <== mgr.8314 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+40 (secure 0 0 0) 0x7f8cb4164640 con 0x7f8c98065900 2023-12-20T04:47:34.627 INFO:teuthology.orchestra.run.smithi032.stdout:Retention added to path /snap_test_dir1 2023-12-20T04:47:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 -- 172.21.15.32:0/99764063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c98065900 msgr2=0x7f8c98067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c98065900 0x7f8c98067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f8cb415ab90 tx=0x7f8ca4009190 comp rx=0 tx=0).stop 2023-12-20T04:47:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 -- 172.21.15.32:0/99764063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb4070770 msgr2=0x7f8cb4156120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb4070770 0x7f8cb4156120 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7f8cb00ae040 tx=0x7f8cb00192e0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 -- 172.21.15.32:0/99764063 shutdown_connections 2023-12-20T04:47:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8c98065900 0x7f8c98067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8cb415b790 0x7f8cb40cc7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8cb40cdef0 0x7f8cb41568f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 --2- 172.21.15.32:0/99764063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8cb4070770 0x7f8cb4156120 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 -- 172.21.15.32:0/99764063 >> 172.21.15.32:0/99764063 conn(0x7f8cb406a510 msgr2=0x7f8cb4157a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 -- 172.21.15.32:0/99764063 shutdown_connections 2023-12-20T04:47:34.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.630+0000 7f8cbbae5640 1 -- 172.21.15.32:0/99764063 wait complete. 2023-12-20T04:47:34.648 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path /snap_test_dir1 2023-12-20T04:47:34.649 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [] 2023-12-20T04:47:34.649 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json /snap_test_dir1 2023-12-20T04:47:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.794+0000 7fe16002d640 1 Processor -- start 2023-12-20T04:47:34.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.794+0000 7fe16002d640 1 -- start start 2023-12-20T04:47:34.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.798+0000 7fe16002d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe1580ca1a0 0x7fe1580cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.798+0000 7fe16002d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1580cb0b0 con 0x7fe158164670 2023-12-20T04:47:34.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.798+0000 7fe16002d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1580ca810 con 0x7fe158070230 2023-12-20T04:47:34.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.798+0000 7fe16002d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1580ca630 con 0x7fe1580ca1a0 2023-12-20T04:47:34.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.798+0000 7fe15ddc9640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe158164670 0x7fe15806da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53150/0 (socket says 172.21.15.32:53150) 2023-12-20T04:47:34.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.798+0000 7fe15ddc9640 1 -- 172.21.15.32:0/1384463940 learned_addr learned my addr 172.21.15.32:0/1384463940 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T04:47:34.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.798+0000 7fe15d5c8640 1 --2- 172.21.15.32:0/1384463940 >> v2:172.21.15.32:3300/0 conn(0x7fe1580ca1a0 0x7fe1580cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.798+0000 7fe15d5c8640 1 -- 172.21.15.32:0/1384463940 >> v1:172.21.15.39:6789/0 conn(0x7fe158070230 legacy=0x7fe158070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.798+0000 7fe15d5c8640 1 -- 172.21.15.32:0/1384463940 >> v1:172.21.15.32:6789/0 conn(0x7fe158164670 legacy=0x7fe15806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.798+0000 7fe15d5c8640 1 -- 172.21.15.32:0/1384463940 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580caaf0 con 0x7fe1580ca1a0 2023-12-20T04:47:34.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.798+0000 7fe15cdc7640 1 -- 172.21.15.32:0/1384463940 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2152041169 0 0) 0x7fe1580cb0b0 con 0x7fe158164670 2023-12-20T04:47:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe15d5c8640 1 --2- 172.21.15.32:0/1384463940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1580ca1a0 0x7fe1580cc3b0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fe1480097c0 tx=0x7fe14801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c3ebaaec3abef66 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe15cdc7640 1 -- 172.21.15.32:0/1384463940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480ad660 con 0x7fe1580ca1a0 2023-12-20T04:47:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe15cdc7640 1 -- 172.21.15.32:0/1384463940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1480ad7c0 con 0x7fe1580ca1a0 2023-12-20T04:47:34.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe15cdc7640 1 -- 172.21.15.32:0/1384463940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480adb50 con 0x7fe1580ca1a0 2023-12-20T04:47:34.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe16002d640 1 -- 172.21.15.32:0/1384463940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1580ca1a0 msgr2=0x7fe1580cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe16002d640 1 --2- 172.21.15.32:0/1384463940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1580ca1a0 0x7fe1580cc3b0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fe1480097c0 tx=0x7fe14801c360 comp rx=0 tx=0).stop 2023-12-20T04:47:34.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe16002d640 1 -- 172.21.15.32:0/1384463940 shutdown_connections 2023-12-20T04:47:34.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe16002d640 1 --2- 172.21.15.32:0/1384463940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1580ca1a0 0x7fe1580cc3b0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe16002d640 1 -- 172.21.15.32:0/1384463940 >> 172.21.15.32:0/1384463940 conn(0x7fe15806a510 msgr2=0x7fe1580ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:34.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe16002d640 1 -- 172.21.15.32:0/1384463940 shutdown_connections 2023-12-20T04:47:34.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe16002d640 1 -- 172.21.15.32:0/1384463940 wait complete. 2023-12-20T04:47:34.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe16002d640 1 Processor -- start 2023-12-20T04:47:34.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.802+0000 7fe16002d640 1 -- start start 2023-12-20T04:47:34.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe16002d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe158070230 0x7fe1580c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe16002d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1580ca1a0 0x7fe15815ef50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe16002d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe158164670 0x7fe158157bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe16002d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe158158750 con 0x7fe158164670 2023-12-20T04:47:34.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe16002d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15815b980 con 0x7fe1580ca1a0 2023-12-20T04:47:34.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe16002d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe158160ee0 con 0x7fe158070230 2023-12-20T04:47:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15ddc9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe158070230 0x7fe1580c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15e5ca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe158164670 0x7fe158157bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15d5c8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1580ca1a0 0x7fe15815ef50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15d5c8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1580ca1a0 0x7fe15815ef50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:60346/0 (socket says 172.21.15.32:60346) 2023-12-20T04:47:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15d5c8640 1 -- 172.21.15.32:0/4069593029 learned_addr learned my addr 172.21.15.32:0/4069593029 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T04:47:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15ddc9640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe158070230 0x7fe1580c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:52030/0 (socket says 172.21.15.32:52030) 2023-12-20T04:47:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15d5c8640 1 -- 172.21.15.32:0/4069593029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe158070230 msgr2=0x7fe1580c8220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15d5c8640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe158070230 0x7fe1580c8220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15d5c8640 1 -- 172.21.15.32:0/4069593029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe158164670 msgr2=0x7fe158157bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15d5c8640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe158164670 0x7fe158157bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15d5c8640 1 -- 172.21.15.32:0/4069593029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe148009420 con 0x7fe1580ca1a0 2023-12-20T04:47:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15e5ca640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe158164670 0x7fe158157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15ddc9640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe158070230 0x7fe1580c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T04:47:34.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe15d5c8640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1580ca1a0 0x7fe15815ef50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe1480b0040 tx=0x7fe148019100 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:34.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe14e7fc640 1 -- 172.21.15.32:0/4069593029 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480b6820 con 0x7fe1580ca1a0 2023-12-20T04:47:34.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe14e7fc640 1 -- 172.21.15.32:0/4069593029 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1480b69c0 con 0x7fe1580ca1a0 2023-12-20T04:47:34.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe14e7fc640 1 -- 172.21.15.32:0/4069593029 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480b6d70 con 0x7fe1580ca1a0 2023-12-20T04:47:34.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe16002d640 1 -- 172.21.15.32:0/4069593029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1581616a0 con 0x7fe1580ca1a0 2023-12-20T04:47:34.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe16002d640 1 -- 172.21.15.32:0/4069593029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15815e1b0 con 0x7fe1580ca1a0 2023-12-20T04:47:34.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.806+0000 7fe16002d640 1 -- 172.21.15.32:0/4069593029 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15815e3a0 con 0x7fe1580ca1a0 2023-12-20T04:47:34.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.810+0000 7fe14e7fc640 1 -- 172.21.15.32:0/4069593029 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 75) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fe1480b4040 con 0x7fe1580ca1a0 2023-12-20T04:47:34.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.810+0000 7fe14e7fc640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe13c065900 0x7fe13c067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T04:47:34.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.810+0000 7fe14e7fc640 1 -- 172.21.15.32:0/4069593029 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(139..139 src has 1..139) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fe14811d3f0 con 0x7fe1580ca1a0 2023-12-20T04:47:34.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.810+0000 7fe15ddc9640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe13c065900 0x7fe13c067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T04:47:34.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.810+0000 7fe15ddc9640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe13c065900 0x7fe13c067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe154007bc0 tx=0x7fe154007b10 comp rx=0 tx=0).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T04:47:34.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.814+0000 7fe14e7fc640 1 -- 172.21.15.32:0/4069593029 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe1480eaf50 con 0x7fe1580ca1a0 2023-12-20T04:47:34.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.990+0000 7fe16002d640 1 -- 172.21.15.32:0/4069593029 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fe158164a50 con 0x7fe13c065900 2023-12-20T04:47:34.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:34.998+0000 7fe14e7fc640 1 -- 172.21.15.32:0/4069593029 <== mgr.8314 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+74 (secure 0 0 0) 0x7fe154004b50 con 0x7fe13c065900 2023-12-20T04:47:34.997 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T04:47:34.997 INFO:teuthology.orchestra.run.smithi032.stdout:{"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-12-20T04:47:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 -- 172.21.15.32:0/4069593029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe13c065900 msgr2=0x7fe13c067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe13c065900 0x7fe13c067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe154007bc0 tx=0x7fe154007b10 comp rx=0 tx=0).stop 2023-12-20T04:47:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 -- 172.21.15.32:0/4069593029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1580ca1a0 msgr2=0x7fe15815ef50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T04:47:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1580ca1a0 0x7fe15815ef50 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fe1480b0040 tx=0x7fe148019100 comp rx=0 tx=0).stop 2023-12-20T04:47:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 -- 172.21.15.32:0/4069593029 shutdown_connections 2023-12-20T04:47:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe13c065900 0x7fe13c067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe158164670 0x7fe158157bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1580ca1a0 0x7fe15815ef50 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 --2- 172.21.15.32:0/4069593029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe158070230 0x7fe1580c8220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T04:47:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 -- 172.21.15.32:0/4069593029 >> 172.21.15.32:0/4069593029 conn(0x7fe15806a510 msgr2=0x7fe158162860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T04:47:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 -- 172.21.15.32:0/4069593029 shutdown_connections 2023-12-20T04:47:35.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T04:47:35.002+0000 7fe16002d640 1 -- 172.21.15.32:0/4069593029 wait complete. 2023-12-20T04:47:35.019 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "/snap_test_dir1", "schedule": ["1M"], "retention": [{"n": 55}]} 2023-12-20T04:47:35.019 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': '/snap_test_dir1', 'schedule': ['1M'], 'retention': [{'n': 55}]} 2023-12-20T04:48:00.641 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:48:00.641+0000 7f19edf06640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:48:00.641 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:48:00.641+0000 7f19edf06640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:48:00.641 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:48:00.641+0000 7f19edf06640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:48:00.641 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:48:00.641+0000 7f19edf06640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:48:00.641 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T04:48:00.641+0000 7f19edf06640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T04:48:01.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:48:01.930 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:48:31.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:48:31.992 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:49:02.044 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:49:02.050 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:49:32.108 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:49:32.113 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:50:02.181 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:50:02.186 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:50:32.261 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:50:32.265 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:51:02.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:51:02.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:51:32.397 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:51:32.402 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:52:02.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:52:02.467 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:52:32.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:52:32.532 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:53:02.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:53:02.595 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:53:32.655 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:53:32.659 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:54:02.836 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:54:02.840 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:54:32.892 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:54:32.896 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:55:02.988 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:55:02.993 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:55:33.063 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:55:33.067 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:56:03.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:56:03.130 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:56:33.190 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:56:33.194 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:57:03.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:57:03.259 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:57:33.318 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:57:33.323 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:58:03.382 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:58:03.388 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:58:33.441 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:58:33.447 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:59:03.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:59:03.505 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:59:33.566 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T04:59:33.570 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:00:03.643 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:00:03.648 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:00:33.844 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:00:33.849 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:01:03.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:01:03.928 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:01:33.982 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:01:33.987 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:02:04.046 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:02:04.051 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:02:34.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:02:34.141 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:03:04.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:03:04.208 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:03:34.260 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:03:34.274 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:04:04.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:04:04.333 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:04:34.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:04:34.418 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:05:04.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:05:04.490 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:05:34.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:05:34.570 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:06:04.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:06:04.625 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:06:34.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:06:34.682 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:07:04.741 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:07:04.746 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:07:34.806 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:07:34.811 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:08:04.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:08:04.885 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:08:34.942 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:08:34.947 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:09:05.030 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:09:05.036 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:09:35.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:09:35.102 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:10:05.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:10:05.176 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:10:35.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:10:35.281 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:11:05.337 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:11:05.342 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:11:35.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:11:35.403 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:12:05.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:12:05.471 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:12:35.528 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:12:35.532 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:13:05.594 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:13:05.599 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:13:35.658 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:13:35.662 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:14:05.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:14:05.718 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:14:35.770 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:14:35.774 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:15:05.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:15:05.848 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:15:35.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:15:35.919 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:16:05.979 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:16:05.984 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:16:36.038 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:16:36.044 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:17:06.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:17:06.104 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:17:36.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:17:36.167 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:18:06.235 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:18:06.240 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:18:36.290 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:18:36.296 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:19:06.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:19:06.361 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:19:36.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:19:36.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:20:06.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:20:06.499 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:20:36.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:20:36.576 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:21:06.632 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:21:06.639 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:21:36.707 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:21:36.722 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:22:06.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:22:06.793 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:22:36.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:22:36.866 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:23:06.919 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:23:06.924 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:23:36.983 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:23:36.989 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:24:07.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:24:07.052 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:24:37.103 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:24:37.108 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:25:07.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:25:07.176 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:25:37.240 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:25:37.247 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:26:07.303 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:26:07.308 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:26:37.366 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:26:37.370 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:27:07.425 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:27:07.430 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:27:37.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:27:37.495 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:28:07.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:28:07.563 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:28:37.620 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:28:37.625 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:29:07.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:29:07.686 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:29:37.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:29:37.745 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:30:07.820 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:30:07.826 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:30:38.019 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:30:38.027 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:31:08.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:31:08.093 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:31:38.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:31:38.154 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:32:08.213 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:32:08.218 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:32:38.269 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:32:38.275 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:33:08.334 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:33:08.340 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:33:38.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:33:38.403 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:34:08.454 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:34:08.460 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:34:38.518 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:34:38.523 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:35:08.608 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:35:08.613 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:35:38.689 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:35:38.697 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:36:08.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:36:08.751 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:36:38.811 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:36:38.818 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:37:08.999 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:37:09.005 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:37:39.067 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:37:39.075 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:38:09.135 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:38:09.140 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:38:39.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:38:39.213 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:39:09.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:39:09.277 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:39:39.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:39:39.341 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:40:09.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:40:09.402 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:40:39.482 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:40:39.486 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:41:09.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:41:09.544 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:41:39.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:41:39.608 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:42:09.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:42:09.676 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:42:39.732 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:42:39.738 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:43:09.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:43:09.823 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:43:16.020 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:16.020 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T05:43:16.036 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_49_00_UTC 2023-12-20T05:43:16.037 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_50_00_UTC 2023-12-20T05:43:16.037 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_51_00_UTC 2023-12-20T05:43:16.037 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_52_00_UTC 2023-12-20T05:43:16.037 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_53_00_UTC 2023-12-20T05:43:16.037 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_54_00_UTC 2023-12-20T05:43:16.037 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_55_00_UTC 2023-12-20T05:43:16.037 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_56_00_UTC 2023-12-20T05:43:16.037 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_57_00_UTC 2023-12-20T05:43:16.037 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_58_00_UTC 2023-12-20T05:43:16.037 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_59_00_UTC 2023-12-20T05:43:16.037 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_00_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_01_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_02_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_03_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_04_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_05_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_06_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_07_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_08_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_09_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_10_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_11_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_12_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_13_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_14_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_15_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_16_00_UTC 2023-12-20T05:43:16.038 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_17_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_18_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_19_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_20_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_21_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_22_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_23_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_24_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_25_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_26_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_27_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_28_00_UTC 2023-12-20T05:43:16.039 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_29_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_30_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_31_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_32_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_33_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_34_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_35_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_36_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_37_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_38_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_39_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_40_00_UTC 2023-12-20T05:43:16.040 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_41_00_UTC 2023-12-20T05:43:16.041 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_42_00_UTC 2023-12-20T05:43:16.041 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_43_00_UTC 2023-12-20T05:43:16.041 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-04_49_00_UTC', 'scheduled-2023-12-20-04_50_00_UTC', 'scheduled-2023-12-20-04_51_00_UTC', 'scheduled-2023-12-20-04_52_00_UTC', 'scheduled-2023-12-20-04_53_00_UTC', 'scheduled-2023-12-20-04_54_00_UTC', 'scheduled-2023-12-20-04_55_00_UTC', 'scheduled-2023-12-20-04_56_00_UTC', 'scheduled-2023-12-20-04_57_00_UTC', 'scheduled-2023-12-20-04_58_00_UTC', 'scheduled-2023-12-20-04_59_00_UTC', 'scheduled-2023-12-20-05_00_00_UTC', 'scheduled-2023-12-20-05_01_00_UTC', 'scheduled-2023-12-20-05_02_00_UTC', 'scheduled-2023-12-20-05_03_00_UTC', 'scheduled-2023-12-20-05_04_00_UTC', 'scheduled-2023-12-20-05_05_00_UTC', 'scheduled-2023-12-20-05_06_00_UTC', 'scheduled-2023-12-20-05_07_00_UTC', 'scheduled-2023-12-20-05_08_00_UTC', 'scheduled-2023-12-20-05_09_00_UTC', 'scheduled-2023-12-20-05_10_00_UTC', 'scheduled-2023-12-20-05_11_00_UTC', 'scheduled-2023-12-20-05_12_00_UTC', 'scheduled-2023-12-20-05_13_00_UTC', 'scheduled-2023-12-20-05_14_00_UTC', 'scheduled-2023-12-20-05_15_00_UTC', 'scheduled-2023-12-20-05_16_00_UTC', 'scheduled-2023-12-20-05_17_00_UTC', 'scheduled-2023-12-20-05_18_00_UTC', 'scheduled-2023-12-20-05_19_00_UTC', 'scheduled-2023-12-20-05_20_00_UTC', 'scheduled-2023-12-20-05_21_00_UTC', 'scheduled-2023-12-20-05_22_00_UTC', 'scheduled-2023-12-20-05_23_00_UTC', 'scheduled-2023-12-20-05_24_00_UTC', 'scheduled-2023-12-20-05_25_00_UTC', 'scheduled-2023-12-20-05_26_00_UTC', 'scheduled-2023-12-20-05_27_00_UTC', 'scheduled-2023-12-20-05_28_00_UTC', 'scheduled-2023-12-20-05_29_00_UTC', 'scheduled-2023-12-20-05_30_00_UTC', 'scheduled-2023-12-20-05_31_00_UTC', 'scheduled-2023-12-20-05_32_00_UTC', 'scheduled-2023-12-20-05_33_00_UTC', 'scheduled-2023-12-20-05_34_00_UTC', 'scheduled-2023-12-20-05_35_00_UTC', 'scheduled-2023-12-20-05_36_00_UTC', 'scheduled-2023-12-20-05_37_00_UTC', 'scheduled-2023-12-20-05_38_00_UTC', 'scheduled-2023-12-20-05_39_00_UTC', 'scheduled-2023-12-20-05_40_00_UTC', 'scheduled-2023-12-20-05_41_00_UTC', 'scheduled-2023-12-20-05_42_00_UTC', 'scheduled-2023-12-20-05_43_00_UTC'] 2023-12-20T05:43:16.042 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1 2023-12-20T05:43:16.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef759e640 1 Processor -- start 2023-12-20T05:43:16.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef759e640 1 -- start start 2023-12-20T05:43:16.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feef80b0720 0x7feef80cc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef759e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef80cc160 con 0x7feef80c8be0 2023-12-20T05:43:16.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef759e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feef80c9e50 con 0x7feef8159100 2023-12-20T05:43:16.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef759e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feef80ca130 con 0x7feef80b0720 2023-12-20T05:43:16.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef5d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feef80b0720 0x7feef80cc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef6d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feef80c8be0 0x7feef80ccdf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48444/0 (socket says 172.21.15.32:48444) 2023-12-20T05:43:16.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef6d9d640 1 -- 172.21.15.32:0/2238659005 learned_addr learned my addr 172.21.15.32:0/2238659005 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:16.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef5d9b640 1 -- 172.21.15.32:0/2238659005 >> v1:172.21.15.39:6789/0 conn(0x7feef8159100 legacy=0x7feef80b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:16.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef5d9b640 1 -- 172.21.15.32:0/2238659005 >> v1:172.21.15.32:6789/0 conn(0x7feef80c8be0 legacy=0x7feef80ccdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef559a640 1 -- 172.21.15.32:0/2238659005 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1267975035 0 0) 0x7feef80cc160 con 0x7feef80c8be0 2023-12-20T05:43:16.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.237+0000 7feef5d9b640 1 -- 172.21.15.32:0/2238659005 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feef80ca410 con 0x7feef80b0720 2023-12-20T05:43:16.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.241+0000 7feef5d9b640 1 --2- 172.21.15.32:0/2238659005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feef80b0720 0x7feef80cc8b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7feeec0097c0 tx=0x7feeec01c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c60b7c059fcc05b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:16.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.241+0000 7feef559a640 1 -- 172.21.15.32:0/2238659005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0ab600 con 0x7feef80b0720 2023-12-20T05:43:16.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.241+0000 7feef559a640 1 -- 172.21.15.32:0/2238659005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec0a7050 con 0x7feef80b0720 2023-12-20T05:43:16.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.241+0000 7feef559a640 1 -- 172.21.15.32:0/2238659005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0ab980 con 0x7feef80b0720 2023-12-20T05:43:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.241+0000 7feef759e640 1 -- 172.21.15.32:0/2238659005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feef80b0720 msgr2=0x7feef80cc8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.241+0000 7feef759e640 1 --2- 172.21.15.32:0/2238659005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feef80b0720 0x7feef80cc8b0 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7feeec0097c0 tx=0x7feeec01c440 comp rx=0 tx=0).stop 2023-12-20T05:43:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.241+0000 7feef759e640 1 -- 172.21.15.32:0/2238659005 shutdown_connections 2023-12-20T05:43:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.241+0000 7feef759e640 1 --2- 172.21.15.32:0/2238659005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feef80b0720 0x7feef80cc8b0 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.241+0000 7feef759e640 1 -- 172.21.15.32:0/2238659005 >> 172.21.15.32:0/2238659005 conn(0x7feef806c060 msgr2=0x7feef806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.241+0000 7feef759e640 1 -- 172.21.15.32:0/2238659005 shutdown_connections 2023-12-20T05:43:16.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.241+0000 7feef759e640 1 -- 172.21.15.32:0/2238659005 wait complete. 2023-12-20T05:43:16.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef759e640 1 Processor -- start 2023-12-20T05:43:16.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef759e640 1 -- start start 2023-12-20T05:43:16.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feef80b0720 0x7feef80ae090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feef80c8be0 0x7feef80c48a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef759e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feef8159100 0x7feef81503b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef759e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef80c4de0 con 0x7feef80b0720 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef759e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feef81508f0 con 0x7feef80c8be0 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef759e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feef80ae770 con 0x7feef8159100 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feef80b0720 0x7feef80ae090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feef80b0720 0x7feef80ae090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35234/0 (socket says 172.21.15.32:35234) 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef659c640 1 -- 172.21.15.32:0/3098949222 learned_addr learned my addr 172.21.15.32:0/3098949222 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef6d9d640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feef8159100 0x7feef81503b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef5d9b640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feef80c8be0 0x7feef80c48a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef659c640 1 -- 172.21.15.32:0/3098949222 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feef8159100 msgr2=0x7feef81503b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef659c640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feef8159100 0x7feef81503b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef659c640 1 -- 172.21.15.32:0/3098949222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feef80c8be0 msgr2=0x7feef80c48a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef659c640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feef80c8be0 0x7feef80c48a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef659c640 1 -- 172.21.15.32:0/3098949222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feeec009420 con 0x7feef80b0720 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef5d9b640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feef80c8be0 0x7feef80c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef659c640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feef80b0720 0x7feef80ae090 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7feee800b440 tx=0x7feee800b900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feee6ffd640 1 -- 172.21.15.32:0/3098949222 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8011890 con 0x7feef80b0720 2023-12-20T05:43:16.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feee6ffd640 1 -- 172.21.15.32:0/3098949222 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feee80119f0 con 0x7feef80b0720 2023-12-20T05:43:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef759e640 1 -- 172.21.15.32:0/3098949222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feef80aea50 con 0x7feef80b0720 2023-12-20T05:43:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feee6ffd640 1 -- 172.21.15.32:0/3098949222 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feee8011da0 con 0x7feef80b0720 2023-12-20T05:43:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.245+0000 7feef759e640 1 -- 172.21.15.32:0/3098949222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feef80af010 con 0x7feef80b0720 2023-12-20T05:43:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.249+0000 7feef759e640 1 -- 172.21.15.32:0/3098949222 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feef80cd4b0 con 0x7feef80b0720 2023-12-20T05:43:16.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.249+0000 7feee6ffd640 1 -- 172.21.15.32:0/3098949222 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 79) v1 ==== 85359+0+0 (secure 0 0 0) 0x7feee801a400 con 0x7feef80b0720 2023-12-20T05:43:16.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.249+0000 7feee6ffd640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feed8065bd0 0x7feed8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.249+0000 7feee6ffd640 1 -- 172.21.15.32:0/3098949222 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6183+0+0 (secure 0 0 0) 0x7feee8083960 con 0x7feef80b0720 2023-12-20T05:43:16.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.249+0000 7feef5d9b640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feed8065bd0 0x7feed8068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.249+0000 7feef5d9b640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feed8065bd0 0x7feed8068090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feeec0ad000 tx=0x7feeec0a6000 comp rx=0 tx=0).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:16.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.253+0000 7feee6ffd640 1 -- 172.21.15.32:0/3098949222 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feee8050f90 con 0x7feef80b0720 2023-12-20T05:43:16.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.429+0000 7feef759e640 1 -- 172.21.15.32:0/3098949222 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7feef8074100 con 0x7feed8065bd0 2023-12-20T05:43:16.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feee6ffd640 1 -- 172.21.15.32:0/3098949222 <== mgr.8314 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+358 (secure 0 0 0) 0x7feef8074100 con 0x7feed8065bd0 2023-12-20T05:43:16.441 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:16.441 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:47:34", "first": "2023-12-20T04:48:00", "last": "2023-12-20T05:43:00", "last_pruned": "2023-12-20T05:43:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-12-20T05:43:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 -- 172.21.15.32:0/3098949222 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feed8065bd0 msgr2=0x7feed8068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feed8065bd0 0x7feed8068090 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7feeec0ad000 tx=0x7feeec0a6000 comp rx=0 tx=0).stop 2023-12-20T05:43:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 -- 172.21.15.32:0/3098949222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feef80b0720 msgr2=0x7feef80ae090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feef80b0720 0x7feef80ae090 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7feee800b440 tx=0x7feee800b900 comp rx=0 tx=0).stop 2023-12-20T05:43:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 -- 172.21.15.32:0/3098949222 shutdown_connections 2023-12-20T05:43:16.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feed8065bd0 0x7feed8068090 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feef8159100 0x7feef81503b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feef80c8be0 0x7feef80c48a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 --2- 172.21.15.32:0/3098949222 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feef80b0720 0x7feef80ae090 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 -- 172.21.15.32:0/3098949222 >> 172.21.15.32:0/3098949222 conn(0x7feef806c060 msgr2=0x7feef80cede0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 -- 172.21.15.32:0/3098949222 shutdown_connections 2023-12-20T05:43:16.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.441+0000 7feef759e640 1 -- 172.21.15.32:0/3098949222 wait complete. 2023-12-20T05:43:16.462 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1", "rel_path": "/snap_test_dir1", "schedule": "1M", "retention": {"n": 55}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T04:47:34", "first": "2023-12-20T04:48:00", "last": "2023-12-20T05:43:00", "last_pruned": "2023-12-20T05:43:00", "created_count": 56, "pruned_count": 1, "active": true}] 2023-12-20T05:43:16.462 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1', 'rel_path': '/snap_test_dir1', 'schedule': '1M', 'retention': {'n': 55}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T04:47:34', 'first': '2023-12-20T04:48:00', 'last': '2023-12-20T05:43:00', 'last_pruned': '2023-12-20T05:43:00', 'created_count': 56, 'pruned_count': 1, 'active': True} 2023-12-20T05:43:16.462 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 55 2023-12-20T05:43:16.462 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 56 2023-12-20T05:43:16.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1 2023-12-20T05:43:16.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.605+0000 7f92bb0f2640 1 Processor -- start 2023-12-20T05:43:16.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92bb0f2640 1 -- start start 2023-12-20T05:43:16.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92bb0f2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f92b415abb0 0x7f92b40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92bb0f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b40b0cb0 0x7f92b415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92bb0f2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b415b6a0 con 0x7f92b4070230 2023-12-20T05:43:16.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92bb0f2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92b406e160 con 0x7f92b415abb0 2023-12-20T05:43:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92bb0f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b415c070 con 0x7f92b40b0cb0 2023-12-20T05:43:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b968f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92b4070230 0x7f92b406c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48454/0 (socket says 172.21.15.32:48454) 2023-12-20T05:43:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b968f640 1 -- 172.21.15.32:0/3435780160 learned_addr learned my addr 172.21.15.32:0/3435780160 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b3fff640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b40b0cb0 0x7f92b415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b8e8e640 1 --2- 172.21.15.32:0/3435780160 >> v2:172.21.15.39:3300/0 conn(0x7f92b415abb0 0x7f92b40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3154395159 0 0) 0x7f92b415b6a0 con 0x7f92b4070230 2023-12-20T05:43:16.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92a4003530 con 0x7f92b4070230 2023-12-20T05:43:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2612996798 0 0) 0x7f92a4003530 con 0x7f92b4070230 2023-12-20T05:43:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b40b0cb0 msgr2=0x7f92b415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b40b0cb0 0x7f92b415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 >> v2:172.21.15.39:3300/0 conn(0x7f92b415abb0 msgr2=0x7f92b40caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 --2- 172.21.15.32:0/3435780160 >> v2:172.21.15.39:3300/0 conn(0x7f92b415abb0 0x7f92b40caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b406d8c0 con 0x7f92b4070230 2023-12-20T05:43:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f92ac001a00 con 0x7f92b4070230 2023-12-20T05:43:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b8e8e640 1 --2- 172.21.15.32:0/3435780160 >> v2:172.21.15.39:3300/0 conn(0x7f92b415abb0 0x7f92b40caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:43:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 >> v1:172.21.15.32:6789/0 conn(0x7f92b4070230 legacy=0x7f92b406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b40b0cb0 0x7f92a4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4003f00 0x7f92a40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92a4004d10 0x7f92a4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b406e160 con 0x7f92a4003f00 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b415c070 con 0x7f92a4004d10 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92a400b990 con 0x7f92b40b0cb0 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b3fff640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4003f00 0x7f92a40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b8e8e640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b40b0cb0 0x7f92a4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b968f640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92a4004d10 0x7f92a4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b3fff640 1 -- 172.21.15.32:0/3435780160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b40b0cb0 msgr2=0x7f92a4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b3fff640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b40b0cb0 0x7f92a4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b3fff640 1 -- 172.21.15.32:0/3435780160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92a4004d10 msgr2=0x7f92a4005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b3fff640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92a4004d10 0x7f92a4005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.609+0000 7f92b3fff640 1 -- 172.21.15.32:0/3435780160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92a4003530 con 0x7f92a4003f00 2023-12-20T05:43:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.613+0000 7f92b3fff640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4003f00 0x7f92a40047d0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009ae0 tx=0x7f92a001c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ac7bad6d25a1f66 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.613+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a00a5530 con 0x7f92a4003f00 2023-12-20T05:43:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.613+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92a00a5690 con 0x7f92a4003f00 2023-12-20T05:43:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.613+0000 7f92b37fe640 1 -- 172.21.15.32:0/3435780160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92a00a5a40 con 0x7f92a4003f00 2023-12-20T05:43:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.613+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3435780160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4003f00 msgr2=0x7f92a40047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.613+0000 7f92bb0f2640 1 --2- 172.21.15.32:0/3435780160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92a4003f00 0x7f92a40047d0 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7f92a0009ae0 tx=0x7f92a001c9f0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.613+0000 7f92b968f640 1 -- 172.21.15.32:0/3435780160 reap_dead start 2023-12-20T05:43:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.613+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3435780160 shutdown_connections 2023-12-20T05:43:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.613+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3435780160 >> 172.21.15.32:0/3435780160 conn(0x7f92b406a510 msgr2=0x7f92b40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:16.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.613+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3435780160 shutdown_connections 2023-12-20T05:43:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.613+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3435780160 wait complete. 2023-12-20T05:43:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92bb0f2640 1 Processor -- start 2023-12-20T05:43:16.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92bb0f2640 1 -- start start 2023-12-20T05:43:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92bb0f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a4003f00 0x7f92b41597a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92bb0f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b40aafc0 0x7f92b40a97c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92bb0f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b414f0d0 0x7f92b4158ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92bb0f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b4159ce0 con 0x7f92b414f0d0 2023-12-20T05:43:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92bb0f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40a8720 con 0x7f92b40aafc0 2023-12-20T05:43:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92bb0f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b40a8a00 con 0x7f92a4003f00 2023-12-20T05:43:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b968f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b414f0d0 0x7f92b4158ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b968f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b414f0d0 0x7f92b4158ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35246/0 (socket says 172.21.15.32:35246) 2023-12-20T05:43:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b968f640 1 -- 172.21.15.32:0/3170639933 learned_addr learned my addr 172.21.15.32:0/3170639933 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b3fff640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b40aafc0 0x7f92b40a97c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b8e8e640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a4003f00 0x7f92b41597a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b968f640 1 -- 172.21.15.32:0/3170639933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a4003f00 msgr2=0x7f92b41597a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b968f640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a4003f00 0x7f92b41597a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b968f640 1 -- 172.21.15.32:0/3170639933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b40aafc0 msgr2=0x7f92b40a97c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b968f640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b40aafc0 0x7f92b40a97c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b968f640 1 -- 172.21.15.32:0/3170639933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b40a8ce0 con 0x7f92b414f0d0 2023-12-20T05:43:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b968f640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b414f0d0 0x7f92b4158ae0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f92b40701e0 tx=0x7f92ac0115c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b17fa640 1 -- 172.21.15.32:0/3170639933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac011e20 con 0x7f92b414f0d0 2023-12-20T05:43:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b17fa640 1 -- 172.21.15.32:0/3170639933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac004b70 con 0x7f92b414f0d0 2023-12-20T05:43:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92b17fa640 1 -- 172.21.15.32:0/3170639933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac006070 con 0x7f92b414f0d0 2023-12-20T05:43:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3170639933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b40a8fc0 con 0x7f92b414f0d0 2023-12-20T05:43:16.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.617+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3170639933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b40a9d00 con 0x7f92b414f0d0 2023-12-20T05:43:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.621+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3170639933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b404e000 con 0x7f92b414f0d0 2023-12-20T05:43:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.621+0000 7f92b17fa640 1 -- 172.21.15.32:0/3170639933 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 79) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f92ac0061d0 con 0x7f92b414f0d0 2023-12-20T05:43:16.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.621+0000 7f92b17fa640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9298065bd0 0x7f9298068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.621+0000 7f92b17fa640 1 -- 172.21.15.32:0/3170639933 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f92ac086590 con 0x7f92b414f0d0 2023-12-20T05:43:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.621+0000 7f92b8e8e640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9298065bd0 0x7f9298068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:16.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.621+0000 7f92b8e8e640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9298065bd0 0x7f9298068090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009de0 tx=0x7f92a8009190 comp rx=0 tx=0).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:16.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.625+0000 7f92b17fa640 1 -- 172.21.15.32:0/3170639933 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f92ac053be0 con 0x7f92b414f0d0 2023-12-20T05:43:16.802 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.801+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3170639933 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f92b406de80 con 0x7f9298065bd0 2023-12-20T05:43:16.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92b17fa640 1 -- 172.21.15.32:0/3170639933 <== mgr.8314 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+42 (secure 0 0 0) 0x7f92a8019070 con 0x7f9298065bd0 2023-12-20T05:43:16.881 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3170639933 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9298065bd0 msgr2=0x7f9298068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9298065bd0 0x7f9298068090 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f92a8009de0 tx=0x7f92a8009190 comp rx=0 tx=0).stop 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3170639933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b414f0d0 msgr2=0x7f92b4158ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b414f0d0 0x7f92b4158ae0 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7f92b40701e0 tx=0x7f92ac0115c0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3170639933 shutdown_connections 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9298065bd0 0x7f9298068090 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b414f0d0 0x7f92b4158ae0 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b40aafc0 0x7f92b40a97c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 --2- 172.21.15.32:0/3170639933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92a4003f00 0x7f92b41597a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3170639933 >> 172.21.15.32:0/3170639933 conn(0x7f92b406a510 msgr2=0x7f92b40b3970 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3170639933 shutdown_connections 2023-12-20T05:43:16.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:16.881+0000 7f92bb0f2640 1 -- 172.21.15.32:0/3170639933 wait complete. 2023-12-20T05:43:16.902 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1 2023-12-20T05:43:16.902 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:16.902 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T05:43:16.913 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_49_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_50_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_51_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_52_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_53_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_54_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_55_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_56_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_57_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_58_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-04_59_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_00_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_01_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_02_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_03_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_04_00_UTC 2023-12-20T05:43:16.914 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_05_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_06_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_07_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_08_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_09_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_10_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_11_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_12_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_13_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_14_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_15_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_16_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_17_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_18_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_19_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_20_00_UTC 2023-12-20T05:43:16.915 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_21_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_22_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_23_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_24_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_25_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_26_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_27_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_28_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_29_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_30_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_31_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_32_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_33_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_34_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_35_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_36_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_37_00_UTC 2023-12-20T05:43:16.916 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_38_00_UTC 2023-12-20T05:43:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_39_00_UTC 2023-12-20T05:43:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_40_00_UTC 2023-12-20T05:43:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_41_00_UTC 2023-12-20T05:43:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_42_00_UTC 2023-12-20T05:43:16.917 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_43_00_UTC 2023-12-20T05:43:16.918 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_49_00_UTC 2023-12-20T05:43:16.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:16.918 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_49_00_UTC) 2023-12-20T05:43:16.974 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_50_00_UTC 2023-12-20T05:43:16.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:16.974 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_50_00_UTC) 2023-12-20T05:43:17.029 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_51_00_UTC 2023-12-20T05:43:17.029 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.029 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_51_00_UTC) 2023-12-20T05:43:17.085 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_52_00_UTC 2023-12-20T05:43:17.085 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.086 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_52_00_UTC) 2023-12-20T05:43:17.141 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_53_00_UTC 2023-12-20T05:43:17.141 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.141 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_53_00_UTC) 2023-12-20T05:43:17.197 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_54_00_UTC 2023-12-20T05:43:17.197 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.198 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_54_00_UTC) 2023-12-20T05:43:17.257 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_55_00_UTC 2023-12-20T05:43:17.257 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.257 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_55_00_UTC) 2023-12-20T05:43:17.317 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_56_00_UTC 2023-12-20T05:43:17.317 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.317 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_56_00_UTC) 2023-12-20T05:43:17.373 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_57_00_UTC 2023-12-20T05:43:17.373 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.373 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_57_00_UTC) 2023-12-20T05:43:17.429 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_58_00_UTC 2023-12-20T05:43:17.429 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.429 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_58_00_UTC) 2023-12-20T05:43:17.485 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-04_59_00_UTC 2023-12-20T05:43:17.485 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.485 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-04_59_00_UTC) 2023-12-20T05:43:17.540 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_00_00_UTC 2023-12-20T05:43:17.541 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.541 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_00_00_UTC) 2023-12-20T05:43:17.597 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_01_00_UTC 2023-12-20T05:43:17.597 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.597 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_01_00_UTC) 2023-12-20T05:43:17.657 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_02_00_UTC 2023-12-20T05:43:17.657 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.657 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_02_00_UTC) 2023-12-20T05:43:17.712 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_03_00_UTC 2023-12-20T05:43:17.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.713 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_03_00_UTC) 2023-12-20T05:43:17.769 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_04_00_UTC 2023-12-20T05:43:17.769 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.769 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_04_00_UTC) 2023-12-20T05:43:17.826 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_05_00_UTC 2023-12-20T05:43:17.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.826 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_05_00_UTC) 2023-12-20T05:43:17.881 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_06_00_UTC 2023-12-20T05:43:17.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.882 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_06_00_UTC) 2023-12-20T05:43:17.937 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_07_00_UTC 2023-12-20T05:43:17.937 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.937 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_07_00_UTC) 2023-12-20T05:43:17.993 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_08_00_UTC 2023-12-20T05:43:17.993 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:17.993 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_08_00_UTC) 2023-12-20T05:43:18.050 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_09_00_UTC 2023-12-20T05:43:18.050 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.050 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_09_00_UTC) 2023-12-20T05:43:18.106 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_10_00_UTC 2023-12-20T05:43:18.106 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.106 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_10_00_UTC) 2023-12-20T05:43:18.161 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_11_00_UTC 2023-12-20T05:43:18.161 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.162 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_11_00_UTC) 2023-12-20T05:43:18.217 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_12_00_UTC 2023-12-20T05:43:18.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.217 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_12_00_UTC) 2023-12-20T05:43:18.277 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_13_00_UTC 2023-12-20T05:43:18.277 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.277 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_13_00_UTC) 2023-12-20T05:43:18.333 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_14_00_UTC 2023-12-20T05:43:18.333 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.333 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_14_00_UTC) 2023-12-20T05:43:18.389 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_15_00_UTC 2023-12-20T05:43:18.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.389 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_15_00_UTC) 2023-12-20T05:43:18.445 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_16_00_UTC 2023-12-20T05:43:18.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.445 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_16_00_UTC) 2023-12-20T05:43:18.500 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_17_00_UTC 2023-12-20T05:43:18.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.501 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_17_00_UTC) 2023-12-20T05:43:18.561 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_18_00_UTC 2023-12-20T05:43:18.561 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.561 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_18_00_UTC) 2023-12-20T05:43:18.617 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_19_00_UTC 2023-12-20T05:43:18.617 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.617 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_19_00_UTC) 2023-12-20T05:43:18.677 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_20_00_UTC 2023-12-20T05:43:18.677 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.677 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_20_00_UTC) 2023-12-20T05:43:18.733 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_21_00_UTC 2023-12-20T05:43:18.733 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.734 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_21_00_UTC) 2023-12-20T05:43:18.789 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_22_00_UTC 2023-12-20T05:43:18.789 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.789 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_22_00_UTC) 2023-12-20T05:43:18.845 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_23_00_UTC 2023-12-20T05:43:18.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.845 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_23_00_UTC) 2023-12-20T05:43:18.901 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_24_00_UTC 2023-12-20T05:43:18.901 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.901 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_24_00_UTC) 2023-12-20T05:43:18.957 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_25_00_UTC 2023-12-20T05:43:18.957 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:18.957 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_25_00_UTC) 2023-12-20T05:43:19.017 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_26_00_UTC 2023-12-20T05:43:19.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.017 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_26_00_UTC) 2023-12-20T05:43:19.073 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_27_00_UTC 2023-12-20T05:43:19.073 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.074 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_27_00_UTC) 2023-12-20T05:43:19.134 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_28_00_UTC 2023-12-20T05:43:19.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.134 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_28_00_UTC) 2023-12-20T05:43:19.194 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_29_00_UTC 2023-12-20T05:43:19.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.195 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_29_00_UTC) 2023-12-20T05:43:19.249 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_30_00_UTC 2023-12-20T05:43:19.250 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.250 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_30_00_UTC) 2023-12-20T05:43:19.309 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_31_00_UTC 2023-12-20T05:43:19.309 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.309 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_31_00_UTC) 2023-12-20T05:43:19.366 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_32_00_UTC 2023-12-20T05:43:19.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.366 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_32_00_UTC) 2023-12-20T05:43:19.425 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_33_00_UTC 2023-12-20T05:43:19.425 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.425 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_33_00_UTC) 2023-12-20T05:43:19.482 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_34_00_UTC 2023-12-20T05:43:19.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.482 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_34_00_UTC) 2023-12-20T05:43:19.537 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_35_00_UTC 2023-12-20T05:43:19.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.538 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_35_00_UTC) 2023-12-20T05:43:19.594 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_36_00_UTC 2023-12-20T05:43:19.594 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.594 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_36_00_UTC) 2023-12-20T05:43:19.650 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_37_00_UTC 2023-12-20T05:43:19.650 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.650 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_37_00_UTC) 2023-12-20T05:43:19.706 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_38_00_UTC 2023-12-20T05:43:19.706 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.706 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_38_00_UTC) 2023-12-20T05:43:19.762 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_39_00_UTC 2023-12-20T05:43:19.762 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.762 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_39_00_UTC) 2023-12-20T05:43:19.817 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_40_00_UTC 2023-12-20T05:43:19.818 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.818 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_40_00_UTC) 2023-12-20T05:43:19.874 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_41_00_UTC 2023-12-20T05:43:19.874 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.874 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_41_00_UTC) 2023-12-20T05:43:19.930 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_42_00_UTC 2023-12-20T05:43:19.930 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.930 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_42_00_UTC) 2023-12-20T05:43:19.986 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-05_43_00_UTC 2023-12-20T05:43:19.986 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:19.986 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-05_43_00_UTC) 2023-12-20T05:43:20.042 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:20.043 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-20T05:43:20.096 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T05:43:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f46a552c640 1 Processor -- start 2023-12-20T05:43:20.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f46a552c640 1 -- start start 2023-12-20T05:43:20.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f46a552c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a00b0760 0x7f46a00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f46a552c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46a00cc4b0 0x7f46a00cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f46a552c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f46a00b1100 con 0x7f46a0157480 2023-12-20T05:43:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f46a552c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f46a00c9f50 con 0x7f46a00b0760 2023-12-20T05:43:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f46a552c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a00cde20 con 0x7f46a00cc4b0 2023-12-20T05:43:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f4697fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a00b0760 0x7f46a00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f4697fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f46a00b0760 0x7f46a00b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49994/0 (socket says 172.21.15.32:49994) 2023-12-20T05:43:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f4697fff640 1 -- 172.21.15.32:0/1471025509 learned_addr learned my addr 172.21.15.32:0/1471025509 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f469f7fe640 1 --2- 172.21.15.32:0/1471025509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46a00cc4b0 0x7f46a00cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f4697fff640 1 -- 172.21.15.32:0/1471025509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46a00cc4b0 msgr2=0x7f46a00cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:20.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f4697fff640 1 --2- 172.21.15.32:0/1471025509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46a00cc4b0 0x7f46a00cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:20.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f4697fff640 1 -- 172.21.15.32:0/1471025509 >> v1:172.21.15.39:6789/0 conn(0x7f46a0157480 legacy=0x7f46a00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:20.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.285+0000 7f4697fff640 1 -- 172.21.15.32:0/1471025509 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46a00c8e30 con 0x7f46a00b0760 2023-12-20T05:43:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f4697fff640 1 --2- 172.21.15.32:0/1471025509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00b0760 0x7f46a00b0bc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f4688009870 tx=0x7f468801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16dcfb4130737e8a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f469e7fc640 1 -- 172.21.15.32:0/1471025509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46880ab680 con 0x7f46a00b0760 2023-12-20T05:43:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f469e7fc640 1 -- 172.21.15.32:0/1471025509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46880ab7e0 con 0x7f46a00b0760 2023-12-20T05:43:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f469e7fc640 1 -- 172.21.15.32:0/1471025509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46880abb70 con 0x7f46a00b0760 2023-12-20T05:43:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f46a552c640 1 -- 172.21.15.32:0/1471025509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00b0760 msgr2=0x7f46a00b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f46a552c640 1 --2- 172.21.15.32:0/1471025509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00b0760 0x7f46a00b0bc0 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7f4688009870 tx=0x7f468801c480 comp rx=0 tx=0).stop 2023-12-20T05:43:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f46a552c640 1 -- 172.21.15.32:0/1471025509 shutdown_connections 2023-12-20T05:43:20.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f46a552c640 1 --2- 172.21.15.32:0/1471025509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46a00cc4b0 0x7f46a00cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f46a552c640 1 --2- 172.21.15.32:0/1471025509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00b0760 0x7f46a00b0bc0 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f46a552c640 1 -- 172.21.15.32:0/1471025509 >> 172.21.15.32:0/1471025509 conn(0x7f46a006a510 msgr2=0x7f46a00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f46a552c640 1 -- 172.21.15.32:0/1471025509 shutdown_connections 2023-12-20T05:43:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.289+0000 7f46a552c640 1 -- 172.21.15.32:0/1471025509 wait complete. 2023-12-20T05:43:20.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46a552c640 1 Processor -- start 2023-12-20T05:43:20.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46a552c640 1 -- start start 2023-12-20T05:43:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46a552c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46a00b0760 0x7f46a00c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46a552c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00cc4b0 0x7f46a014d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:20.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46a552c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46a0157480 0x7f46a0158d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46a552c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a014e7b0 con 0x7f46a00cc4b0 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46a552c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f46a0166bc0 con 0x7f46a0157480 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46a552c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f46a006cb60 con 0x7f46a00b0760 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f4697fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00cc4b0 0x7f46a014d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f4697fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00cc4b0 0x7f46a014d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50004/0 (socket says 172.21.15.32:50004) 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f4697fff640 1 -- 172.21.15.32:0/2472668119 learned_addr learned my addr 172.21.15.32:0/2472668119 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f469f7fe640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46a0157480 0x7f46a0158d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f469effd640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46a00b0760 0x7f46a00c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f4697fff640 1 -- 172.21.15.32:0/2472668119 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46a00b0760 msgr2=0x7f46a00c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f4697fff640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46a00b0760 0x7f46a00c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f4697fff640 1 -- 172.21.15.32:0/2472668119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46a0157480 msgr2=0x7f46a0158d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:20.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f4697fff640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46a0157480 0x7f46a0158d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:20.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f4697fff640 1 -- 172.21.15.32:0/2472668119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f46880094d0 con 0x7f46a00cc4b0 2023-12-20T05:43:20.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f4697fff640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00cc4b0 0x7f46a014d9c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f46880ad040 tx=0x7f468801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:20.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46977fe640 1 -- 172.21.15.32:0/2472668119 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46880b4800 con 0x7f46a00cc4b0 2023-12-20T05:43:20.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46977fe640 1 -- 172.21.15.32:0/2472668119 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f46880b4980 con 0x7f46a00cc4b0 2023-12-20T05:43:20.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46a552c640 1 -- 172.21.15.32:0/2472668119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f46a006ce40 con 0x7f46a00cc4b0 2023-12-20T05:43:20.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46977fe640 1 -- 172.21.15.32:0/2472668119 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f46880b4d30 con 0x7f46a00cc4b0 2023-12-20T05:43:20.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.293+0000 7f46a552c640 1 -- 172.21.15.32:0/2472668119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f46a006d120 con 0x7f46a00cc4b0 2023-12-20T05:43:20.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.297+0000 7f46a552c640 1 -- 172.21.15.32:0/2472668119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f46a0166d30 con 0x7f46a00cc4b0 2023-12-20T05:43:20.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.297+0000 7f46977fe640 1 -- 172.21.15.32:0/2472668119 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 79) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f46880b3380 con 0x7f46a00cc4b0 2023-12-20T05:43:20.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.297+0000 7f46977fe640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4680065bd0 0x7f4680068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:20.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.297+0000 7f46977fe640 1 -- 172.21.15.32:0/2472668119 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6183+0+0 (secure 0 0 0) 0x7f468811df60 con 0x7f46a00cc4b0 2023-12-20T05:43:20.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.297+0000 7f469effd640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4680065bd0 0x7f4680068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:20.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.297+0000 7f469effd640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4680065bd0 0x7f4680068090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4690007fd0 tx=0x7f4690009040 comp rx=0 tx=0).ready entity=mgr.8314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.301+0000 7f46977fe640 1 -- 172.21.15.32:0/2472668119 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f46880eb650 con 0x7f46a00cc4b0 2023-12-20T05:43:20.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:20.493+0000 7f46a552c640 1 -- 172.21.15.32:0/2472668119 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f46a00c9050 con 0x7f46a00cc4b0 2023-12-20T05:43:21.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.109+0000 7f46977fe640 1 -- 172.21.15.32:0/2472668119 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 80) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f46880f5020 con 0x7f46a00cc4b0 2023-12-20T05:43:21.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.109+0000 7f46977fe640 1 -- 172.21.15.32:0/2472668119 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v80) v1 ==== 93+0+0 (secure 0 0 0) 0x7f46880c05b0 con 0x7f46a00cc4b0 2023-12-20T05:43:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 -- 172.21.15.32:0/2472668119 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4680065bd0 msgr2=0x7f4680068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4680065bd0 0x7f4680068090 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f4690007fd0 tx=0x7f4690009040 comp rx=0 tx=0).stop 2023-12-20T05:43:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 -- 172.21.15.32:0/2472668119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00cc4b0 msgr2=0x7f46a014d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00cc4b0 0x7f46a014d9c0 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x7f46880ad040 tx=0x7f468801acc0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 -- 172.21.15.32:0/2472668119 shutdown_connections 2023-12-20T05:43:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4680065bd0 0x7f4680068090 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f46a0157480 0x7f46a0158d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f46a00cc4b0 0x7f46a014d9c0 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 --2- 172.21.15.32:0/2472668119 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f46a00b0760 0x7f46a00c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 -- 172.21.15.32:0/2472668119 >> 172.21.15.32:0/2472668119 conn(0x7f46a006a510 msgr2=0x7f46a00b4010 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 -- 172.21.15.32:0/2472668119 shutdown_connections 2023-12-20T05:43:21.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.113+0000 7f46a552c640 1 -- 172.21.15.32:0/2472668119 wait complete. 2023-12-20T05:43:21.138 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T05:43:21.138 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T05:43:21.246 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T05:43:21.289 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:21.284+0000 7f076ca4d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T05:43:21.301 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:21.301+0000 7f9a7a1e8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T05:43:21.332 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T05:43:21.332 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:43:21.332 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:43:21.342 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:43:21.343 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:43:21.393 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:43:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb264cd7640 1 Processor -- start 2023-12-20T05:43:21.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb264cd7640 1 -- start start 2023-12-20T05:43:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb264cd7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb2600b8480 0x7fb26006af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb264cd7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 0x7fb260160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:21.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb264cd7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb26006b8c0 con 0x7fb260161330 2023-12-20T05:43:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb264cd7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2600cd4d0 con 0x7fb2600b8480 2023-12-20T05:43:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb264cd7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2600ce4e0 con 0x7fb26006b460 2023-12-20T05:43:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb25ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb260161330 0x7fb260153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51348/0 (socket says 172.21.15.32:51348) 2023-12-20T05:43:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb25ffff640 1 -- 172.21.15.32:0/3709659953 learned_addr learned my addr 172.21.15.32:0/3709659953 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb25f7fe640 1 --2- 172.21.15.32:0/3709659953 >> v2:172.21.15.39:3300/0 conn(0x7fb2600b8480 0x7fb26006af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb25effd640 1 --2- 172.21.15.32:0/3709659953 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 0x7fb260160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb25f7fe640 1 -- 172.21.15.32:0/3709659953 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 msgr2=0x7fb260160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb25e7fc640 1 -- 172.21.15.32:0/3709659953 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2763273992 0 0) 0x7fb26006b8c0 con 0x7fb260161330 2023-12-20T05:43:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb25f7fe640 1 --2- 172.21.15.32:0/3709659953 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 0x7fb260160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb25f7fe640 1 -- 172.21.15.32:0/3709659953 >> v1:172.21.15.32:6789/0 conn(0x7fb260161330 legacy=0x7fb260153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb25f7fe640 1 -- 172.21.15.32:0/3709659953 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2600cd720 con 0x7fb2600b8480 2023-12-20T05:43:21.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.581+0000 7fb25effd640 1 --2- 172.21.15.32:0/3709659953 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 0x7fb260160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:43:21.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb25f7fe640 1 --2- 172.21.15.32:0/3709659953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2600b8480 0x7fb26006af20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb2500097c0 tx=0x7fb25001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bcdd533e5995da7c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:21.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb25e7fc640 1 -- 172.21.15.32:0/3709659953 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2500ab600 con 0x7fb2600b8480 2023-12-20T05:43:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb25e7fc640 1 -- 172.21.15.32:0/3709659953 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2500a7050 con 0x7fb2600b8480 2023-12-20T05:43:21.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb25e7fc640 1 -- 172.21.15.32:0/3709659953 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2500ab980 con 0x7fb2600b8480 2023-12-20T05:43:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb264cd7640 1 -- 172.21.15.32:0/3709659953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2600b8480 msgr2=0x7fb26006af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb264cd7640 1 --2- 172.21.15.32:0/3709659953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2600b8480 0x7fb26006af20 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fb2500097c0 tx=0x7fb25001c480 comp rx=0 tx=0).stop 2023-12-20T05:43:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb264cd7640 1 -- 172.21.15.32:0/3709659953 shutdown_connections 2023-12-20T05:43:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb264cd7640 1 --2- 172.21.15.32:0/3709659953 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 0x7fb260160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb264cd7640 1 --2- 172.21.15.32:0/3709659953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2600b8480 0x7fb26006af20 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb264cd7640 1 -- 172.21.15.32:0/3709659953 >> 172.21.15.32:0/3709659953 conn(0x7fb2600d2a50 msgr2=0x7fb2600d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb264cd7640 1 -- 172.21.15.32:0/3709659953 shutdown_connections 2023-12-20T05:43:21.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.585+0000 7fb264cd7640 1 -- 172.21.15.32:0/3709659953 wait complete. 2023-12-20T05:43:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb264cd7640 1 Processor -- start 2023-12-20T05:43:21.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb264cd7640 1 -- start start 2023-12-20T05:43:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb264cd7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 0x7fb260148de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb264cd7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2600b8480 0x7fb260149320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb264cd7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb260161330 0x7fb2601517a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb264cd7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb26014b2b0 con 0x7fb260161330 2023-12-20T05:43:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb264cd7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb260150a60 con 0x7fb2600b8480 2023-12-20T05:43:21.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb264cd7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb260150d40 con 0x7fb26006b460 2023-12-20T05:43:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb25f7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 0x7fb260148de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb25ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb260161330 0x7fb2601517a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb25effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2600b8480 0x7fb260149320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb25f7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 0x7fb260148de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:60590/0 (socket says 172.21.15.32:60590) 2023-12-20T05:43:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb25f7fe640 1 -- 172.21.15.32:0/3205905205 learned_addr learned my addr 172.21.15.32:0/3205905205 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb25f7fe640 1 -- 172.21.15.32:0/3205905205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2600b8480 msgr2=0x7fb260149320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb25f7fe640 1 --2- 172.21.15.32:0/3205905205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2600b8480 0x7fb260149320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb25f7fe640 1 -- 172.21.15.32:0/3205905205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb260161330 msgr2=0x7fb2601517a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb25f7fe640 1 --2- 172.21.15.32:0/3205905205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb260161330 0x7fb2601517a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb25f7fe640 1 -- 172.21.15.32:0/3205905205 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb254009420 con 0x7fb26006b460 2023-12-20T05:43:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb25f7fe640 1 --2- 172.21.15.32:0/3205905205 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 0x7fb260148de0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb25001cd20 tx=0x7fb25001acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb247fff640 1 -- 172.21.15.32:0/3205905205 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2500b7450 con 0x7fb26006b460 2023-12-20T05:43:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb247fff640 1 -- 172.21.15.32:0/3205905205 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2500b75d0 con 0x7fb26006b460 2023-12-20T05:43:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb247fff640 1 -- 172.21.15.32:0/3205905205 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2500b7980 con 0x7fb26006b460 2023-12-20T05:43:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb264cd7640 1 -- 172.21.15.32:0/3205905205 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb250009420 con 0x7fb26006b460 2023-12-20T05:43:21.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.589+0000 7fb264cd7640 1 -- 172.21.15.32:0/3205905205 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb260149c10 con 0x7fb26006b460 2023-12-20T05:43:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.593+0000 7fb264cd7640 1 -- 172.21.15.32:0/3205905205 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb260074100 con 0x7fb26006b460 2023-12-20T05:43:21.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.593+0000 7fb247fff640 1 -- 172.21.15.32:0/3205905205 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 80) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fb2500b7ae0 con 0x7fb26006b460 2023-12-20T05:43:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.593+0000 7fb247fff640 1 --2- 172.21.15.32:0/3205905205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb24c065b80 0x7fb24c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.593+0000 7fb25effd640 1 -- 172.21.15.32:0/3205905205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb24c065b80 msgr2=0x7fb24c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.593+0000 7fb247fff640 1 -- 172.21.15.32:0/3205905205 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fb25011de40 con 0x7fb26006b460 2023-12-20T05:43:21.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.593+0000 7fb25effd640 1 --2- 172.21.15.32:0/3205905205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb24c065b80 0x7fb24c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:21.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.597+0000 7fb247fff640 1 -- 172.21.15.32:0/3205905205 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb2500eb410 con 0x7fb26006b460 2023-12-20T05:43:21.729 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:21.724+0000 7f076ca4d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T05:43:21.748 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:21.749+0000 7f9a7a1e8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T05:43:21.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 -- 172.21.15.32:0/3205905205 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2600d0a70 con 0x7fb26006b460 2023-12-20T05:43:21.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb247fff640 1 -- 172.21.15.32:0/3205905205 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v143) v1 ==== 74+0+16734 (secure 0 0 0) 0x7fb2500eea30 con 0x7fb26006b460 2023-12-20T05:43:21.762 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:21.762 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":143,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:43:14.076646+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:47:19.596882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-20T04:47:20.055023+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"142","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":58,"snap_epoch":142,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":2}]},{"pgid":"17.17","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/801772685":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/2511943559":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/3621453434":"2023-12-21T04:47:33.113748+0000","172.21.15.32:6833/38009":"2023-12-21T04:47:33.113748+0000","172.21.15.32:6832/38009":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/1810110767":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/2135559470":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/1663137028":"2023-12-21T04:47:33.113748+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[{"pool":17,"snaps":[{"begin":2,"length":1},{"begin":58,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 -- 172.21.15.32:0/3205905205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb24c065b80 msgr2=0x7fb24c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 --2- 172.21.15.32:0/3205905205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb24c065b80 0x7fb24c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 -- 172.21.15.32:0/3205905205 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 msgr2=0x7fb260148de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 --2- 172.21.15.32:0/3205905205 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 0x7fb260148de0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fb25001cd20 tx=0x7fb25001acc0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 -- 172.21.15.32:0/3205905205 shutdown_connections 2023-12-20T05:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 --2- 172.21.15.32:0/3205905205 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb24c065b80 0x7fb24c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 --2- 172.21.15.32:0/3205905205 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb260161330 0x7fb2601517a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 --2- 172.21.15.32:0/3205905205 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2600b8480 0x7fb260149320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 --2- 172.21.15.32:0/3205905205 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb26006b460 0x7fb260148de0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 -- 172.21.15.32:0/3205905205 >> 172.21.15.32:0/3205905205 conn(0x7fb2600d2a50 msgr2=0x7fb2600d1e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:21.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 -- 172.21.15.32:0/3205905205 shutdown_connections 2023-12-20T05:43:21.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.761+0000 7fb264cd7640 1 -- 172.21.15.32:0/3205905205 wait complete. 2023-12-20T05:43:21.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:47:19.596882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:47:20.055023+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '142', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 58, 'snap_epoch': 142, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T05:43:21.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T05:43:21.886 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:21.884+0000 7f076ca4d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T05:43:21.912 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:21.913+0000 7f9a7a1e8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T05:43:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd6a06d1640 1 Processor -- start 2023-12-20T05:43:21.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd6a06d1640 1 -- start start 2023-12-20T05:43:21.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd6a06d1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd69815ce60 0x7fd6980d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:21.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd6a06d1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980d2650 con 0x7fd6980d14d0 2023-12-20T05:43:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd6a06d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6980d1a40 con 0x7fd6980d3130 2023-12-20T05:43:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd6a06d1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd6980d35a0 con 0x7fd69815ce60 2023-12-20T05:43:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd69ec6e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd6980d14d0 0x7fd6980b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51354/0 (socket says 172.21.15.32:51354) 2023-12-20T05:43:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd69ec6e640 1 -- 172.21.15.32:0/2839655220 learned_addr learned my addr 172.21.15.32:0/2839655220 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd69e46d640 1 --2- 172.21.15.32:0/2839655220 >> v2:172.21.15.32:3300/0 conn(0x7fd69815ce60 0x7fd6980d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd69e46d640 1 -- 172.21.15.32:0/2839655220 >> v1:172.21.15.39:6789/0 conn(0x7fd6980d3130 legacy=0x7fd6980d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd69e46d640 1 -- 172.21.15.32:0/2839655220 >> v1:172.21.15.32:6789/0 conn(0x7fd6980d14d0 legacy=0x7fd6980b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd69e46d640 1 -- 172.21.15.32:0/2839655220 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6980d1d20 con 0x7fd69815ce60 2023-12-20T05:43:21.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.937+0000 7fd69d46b640 1 -- 172.21.15.32:0/2839655220 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4144743143 0 0) 0x7fd6980d2650 con 0x7fd6980d14d0 2023-12-20T05:43:21.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.941+0000 7fd69e46d640 1 --2- 172.21.15.32:0/2839655220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69815ce60 0x7fd6980d2bf0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fd6940097f0 tx=0x7fd69401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72538079c6e51a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.941+0000 7fd69d46b640 1 -- 172.21.15.32:0/2839655220 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6940a8970 con 0x7fd69815ce60 2023-12-20T05:43:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.941+0000 7fd69d46b640 1 -- 172.21.15.32:0/2839655220 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6940a8ad0 con 0x7fd69815ce60 2023-12-20T05:43:21.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.941+0000 7fd69d46b640 1 -- 172.21.15.32:0/2839655220 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6940a8e60 con 0x7fd69815ce60 2023-12-20T05:43:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.941+0000 7fd6a06d1640 1 -- 172.21.15.32:0/2839655220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69815ce60 msgr2=0x7fd6980d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.941+0000 7fd6a06d1640 1 --2- 172.21.15.32:0/2839655220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69815ce60 0x7fd6980d2bf0 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7fd6940097f0 tx=0x7fd69401ec70 comp rx=0 tx=0).stop 2023-12-20T05:43:21.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.941+0000 7fd6a06d1640 1 -- 172.21.15.32:0/2839655220 shutdown_connections 2023-12-20T05:43:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.941+0000 7fd6a06d1640 1 --2- 172.21.15.32:0/2839655220 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd69815ce60 0x7fd6980d2bf0 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.941+0000 7fd6a06d1640 1 -- 172.21.15.32:0/2839655220 >> 172.21.15.32:0/2839655220 conn(0x7fd69806b5e0 msgr2=0x7fd69806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.941+0000 7fd6a06d1640 1 -- 172.21.15.32:0/2839655220 shutdown_connections 2023-12-20T05:43:21.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.941+0000 7fd6a06d1640 1 -- 172.21.15.32:0/2839655220 wait complete. 2023-12-20T05:43:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd6a06d1640 1 Processor -- start 2023-12-20T05:43:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd6a06d1640 1 -- start start 2023-12-20T05:43:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd6a06d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6980d14d0 0x7fd69814e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd6a06d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6980d3130 0x7fd6981e4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd6a06d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd69815ce60 0x7fd6981e6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd6a06d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6981e6080 con 0x7fd6980d3130 2023-12-20T05:43:21.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd6a06d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd69814e770 con 0x7fd69815ce60 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd6a06d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6981e4fe0 con 0x7fd6980d14d0 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd69dc6c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6980d3130 0x7fd6981e4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd69dc6c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6980d3130 0x7fd6981e4590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50038/0 (socket says 172.21.15.32:50038) 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd69dc6c640 1 -- 172.21.15.32:0/352496365 learned_addr learned my addr 172.21.15.32:0/352496365 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd69e46d640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6980d14d0 0x7fd69814e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd69ec6e640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd69815ce60 0x7fd6981e6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd69dc6c640 1 -- 172.21.15.32:0/352496365 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6980d14d0 msgr2=0x7fd69814e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd69dc6c640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6980d14d0 0x7fd69814e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd69dc6c640 1 -- 172.21.15.32:0/352496365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd69815ce60 msgr2=0x7fd6981e6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd69dc6c640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd69815ce60 0x7fd6981e6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd69dc6c640 1 -- 172.21.15.32:0/352496365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd694009420 con 0x7fd6980d3130 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd69dc6c640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6980d3130 0x7fd6981e4590 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c500 tx=0x7fd68800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:21.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd68effd640 1 -- 172.21.15.32:0/352496365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688005220 con 0x7fd6980d3130 2023-12-20T05:43:21.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd68effd640 1 -- 172.21.15.32:0/352496365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd688005380 con 0x7fd6980d3130 2023-12-20T05:43:21.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd6a06d1640 1 -- 172.21.15.32:0/352496365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6981e52c0 con 0x7fd6980d3130 2023-12-20T05:43:21.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd68effd640 1 -- 172.21.15.32:0/352496365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd688007270 con 0x7fd6980d3130 2023-12-20T05:43:21.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.945+0000 7fd6a06d1640 1 -- 172.21.15.32:0/352496365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6981e5880 con 0x7fd6980d3130 2023-12-20T05:43:21.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.949+0000 7fd6a06d1640 1 -- 172.21.15.32:0/352496365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd69804e000 con 0x7fd6980d3130 2023-12-20T05:43:21.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.949+0000 7fd68effd640 1 -- 172.21.15.32:0/352496365 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 80) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fd6880073d0 con 0x7fd6980d3130 2023-12-20T05:43:21.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.949+0000 7fd68effd640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd67c065b80 0x7fd67c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:21.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.949+0000 7fd68effd640 1 -- 172.21.15.32:0/352496365 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(143..143 src has 1..143) v4 ==== 6183+0+0 (secure 0 0 0) 0x7fd688083ab0 con 0x7fd6980d3130 2023-12-20T05:43:21.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.949+0000 7fd69e46d640 1 -- 172.21.15.32:0/352496365 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd67c065b80 msgr2=0x7fd67c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:21.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.949+0000 7fd69e46d640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd67c065b80 0x7fd67c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:21.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:21.953+0000 7fd68effd640 1 -- 172.21.15.32:0/352496365 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd688051220 con 0x7fd6980d3130 2023-12-20T05:43:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.117+0000 7fd6a06d1640 1 -- 172.21.15.32:0/352496365 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd6981e36a0 con 0x7fd6980d3130 2023-12-20T05:43:22.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.117+0000 7fd68effd640 1 -- 172.21.15.32:0/352496365 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v144) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd688054840 con 0x7fd6980d3130 2023-12-20T05:43:22.118 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T05:43:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 -- 172.21.15.32:0/352496365 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd67c065b80 msgr2=0x7fd67c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd67c065b80 0x7fd67c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 -- 172.21.15.32:0/352496365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6980d3130 msgr2=0x7fd6981e4590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6980d3130 0x7fd6981e4590 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7fd68800c500 tx=0x7fd68800c9c0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 -- 172.21.15.32:0/352496365 shutdown_connections 2023-12-20T05:43:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd67c065b80 0x7fd67c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd69815ce60 0x7fd6981e6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6980d3130 0x7fd6981e4590 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 --2- 172.21.15.32:0/352496365 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6980d14d0 0x7fd69814e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 -- 172.21.15.32:0/352496365 >> 172.21.15.32:0/352496365 conn(0x7fd69806b5e0 msgr2=0x7fd6980b2ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:22.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 -- 172.21.15.32:0/352496365 shutdown_connections 2023-12-20T05:43:22.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.121+0000 7fd6a06d1640 1 -- 172.21.15.32:0/352496365 wait complete. 2023-12-20T05:43:22.157 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T05:43:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.393+0000 7fa8389f3640 1 Processor -- start 2023-12-20T05:43:22.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.393+0000 7fa8389f3640 1 -- start start 2023-12-20T05:43:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.397+0000 7fa8389f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa83415abb0 0x7fa8340caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.397+0000 7fa8389f3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83415b6a0 con 0x7fa8340b0cb0 2023-12-20T05:43:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.397+0000 7fa8389f3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa83406d8c0 con 0x7fa834070230 2023-12-20T05:43:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.397+0000 7fa8389f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83406e160 con 0x7fa83415abb0 2023-12-20T05:43:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.397+0000 7fa831d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa8340b0cb0 0x7fa83415bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51366/0 (socket says 172.21.15.32:51366) 2023-12-20T05:43:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.397+0000 7fa831d9b640 1 -- 172.21.15.32:0/4031481030 learned_addr learned my addr 172.21.15.32:0/4031481030 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.397+0000 7fa83259c640 1 --2- 172.21.15.32:0/4031481030 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa83415abb0 0x7fa8340caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:22.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.397+0000 7fa83259c640 1 -- 172.21.15.32:0/4031481030 >> v1:172.21.15.39:6789/0 conn(0x7fa834070230 legacy=0x7fa83406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:22.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.397+0000 7fa83259c640 1 -- 172.21.15.32:0/4031481030 >> v1:172.21.15.32:6789/0 conn(0x7fa8340b0cb0 legacy=0x7fa83415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:22.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.397+0000 7fa83159a640 1 -- 172.21.15.32:0/4031481030 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2863995605 0 0) 0x7fa83406d8c0 con 0x7fa834070230 2023-12-20T05:43:22.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.397+0000 7fa83259c640 1 -- 172.21.15.32:0/4031481030 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa83406dba0 con 0x7fa83415abb0 2023-12-20T05:43:22.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa83259c640 1 --2- 172.21.15.32:0/4031481030 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa83415abb0 0x7fa8340caaf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8240097f0 tx=0x7fa82401c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e96d179ccca1aa4f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:22.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa83159a640 1 -- 172.21.15.32:0/4031481030 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8240ab600 con 0x7fa83415abb0 2023-12-20T05:43:22.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa83159a640 1 -- 172.21.15.32:0/4031481030 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8240a7050 con 0x7fa83415abb0 2023-12-20T05:43:22.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa83159a640 1 -- 172.21.15.32:0/4031481030 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8240ab980 con 0x7fa83415abb0 2023-12-20T05:43:22.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa8389f3640 1 -- 172.21.15.32:0/4031481030 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa83415abb0 msgr2=0x7fa8340caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa8389f3640 1 --2- 172.21.15.32:0/4031481030 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa83415abb0 0x7fa8340caaf0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fa8240097f0 tx=0x7fa82401c350 comp rx=0 tx=0).stop 2023-12-20T05:43:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa8389f3640 1 -- 172.21.15.32:0/4031481030 shutdown_connections 2023-12-20T05:43:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa8389f3640 1 --2- 172.21.15.32:0/4031481030 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa83415abb0 0x7fa8340caaf0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa8389f3640 1 -- 172.21.15.32:0/4031481030 >> 172.21.15.32:0/4031481030 conn(0x7fa83406a510 msgr2=0x7fa8340ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa8389f3640 1 -- 172.21.15.32:0/4031481030 shutdown_connections 2023-12-20T05:43:22.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa8389f3640 1 -- 172.21.15.32:0/4031481030 wait complete. 2023-12-20T05:43:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa8389f3640 1 Processor -- start 2023-12-20T05:43:22.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.401+0000 7fa8389f3640 1 -- start start 2023-12-20T05:43:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa8389f3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa834070230 0x7fa8341475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa8389f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8340b0cb0 0x7fa834144be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa8389f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa83415abb0 0x7fa8341f5e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa8389f3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa834156f80 con 0x7fa834070230 2023-12-20T05:43:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa8389f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa834147af0 con 0x7fa8340b0cb0 2023-12-20T05:43:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa8389f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa83414bac0 con 0x7fa83415abb0 2023-12-20T05:43:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa83259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa834070230 0x7fa8341475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa83259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa834070230 0x7fa8341475b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50046/0 (socket says 172.21.15.32:50046) 2023-12-20T05:43:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa83259c640 1 -- 172.21.15.32:0/1087978062 learned_addr learned my addr 172.21.15.32:0/1087978062 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:22.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa832d9d640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa83415abb0 0x7fa8341f5e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa831d9b640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8340b0cb0 0x7fa834144be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa83259c640 1 -- 172.21.15.32:0/1087978062 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa83415abb0 msgr2=0x7fa8341f5e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa83259c640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa83415abb0 0x7fa8341f5e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa83259c640 1 -- 172.21.15.32:0/1087978062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8340b0cb0 msgr2=0x7fa834144be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa83259c640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8340b0cb0 0x7fa834144be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa83259c640 1 -- 172.21.15.32:0/1087978062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa824009420 con 0x7fa834070230 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa831d9b640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8340b0cb0 0x7fa834144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa83259c640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa834070230 0x7fa8341475b0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa8240097f0 tx=0x7fa8240190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa822ffd640 1 -- 172.21.15.32:0/1087978062 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8240b5940 con 0x7fa834070230 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa822ffd640 1 -- 172.21.15.32:0/1087978062 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8240b5ac0 con 0x7fa834070230 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa822ffd640 1 -- 172.21.15.32:0/1087978062 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8240b5e70 con 0x7fa834070230 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa8389f3640 1 -- 172.21.15.32:0/1087978062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8341f5980 con 0x7fa834070230 2023-12-20T05:43:22.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa8389f3640 1 -- 172.21.15.32:0/1087978062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa834148000 con 0x7fa834070230 2023-12-20T05:43:22.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa8389f3640 1 -- 172.21.15.32:0/1087978062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa83404e000 con 0x7fa834070230 2023-12-20T05:43:22.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.405+0000 7fa822ffd640 1 -- 172.21.15.32:0/1087978062 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 80) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fa8240b2040 con 0x7fa834070230 2023-12-20T05:43:22.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.409+0000 7fa822ffd640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa814065b80 0x7fa814068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:22.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.409+0000 7fa822ffd640 1 -- 172.21.15.32:0/1087978062 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6227+0+0 (secure 0 0 0) 0x7fa82411dfb0 con 0x7fa834070230 2023-12-20T05:43:22.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.409+0000 7fa831d9b640 1 -- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa814065b80 msgr2=0x7fa814068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:22.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.409+0000 7fa831d9b640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa814065b80 0x7fa814068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:22.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.417+0000 7fa822ffd640 1 -- 172.21.15.32:0/1087978062 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa8240eb420 con 0x7fa834070230 2023-12-20T05:43:22.485 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:22.484+0000 7f076ca4d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T05:43:22.554 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:22.552+0000 7f076ca4d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T05:43:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.613+0000 7fa831d9b640 1 -- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa814065b80 msgr2=0x7fa814068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:22.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.613+0000 7fa831d9b640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa814065b80 0x7fa814068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T05:43:22.622 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:22.620+0000 7f076ca4d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T05:43:22.713 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:22.713+0000 7f9a7a1e8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T05:43:22.732 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:22.728+0000 7f076ca4d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T05:43:22.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.761+0000 7fa8389f3640 1 -- 172.21.15.32:0/1087978062 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa8340b1a40 con 0x7fa834070230 2023-12-20T05:43:22.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.765+0000 7fa822ffd640 1 -- 172.21.15.32:0/1087978062 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v144) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa8240eea40 con 0x7fa834070230 2023-12-20T05:43:22.766 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T05:43:22.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 -- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa814065b80 msgr2=0x7fa814068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:22.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa814065b80 0x7fa814068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 -- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa834070230 msgr2=0x7fa8341475b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:22.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa834070230 0x7fa8341475b0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa8240097f0 tx=0x7fa8240190f0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 -- 172.21.15.32:0/1087978062 shutdown_connections 2023-12-20T05:43:22.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa814065b80 0x7fa814068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa83415abb0 0x7fa8341f5e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8340b0cb0 0x7fa834144be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 --2- 172.21.15.32:0/1087978062 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa834070230 0x7fa8341475b0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:22.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 -- 172.21.15.32:0/1087978062 >> 172.21.15.32:0/1087978062 conn(0x7fa83406a510 msgr2=0x7fa8340cc3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:22.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 -- 172.21.15.32:0/1087978062 shutdown_connections 2023-12-20T05:43:22.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:22.769+0000 7fa8389f3640 1 -- 172.21.15.32:0/1087978062 wait complete. 2023-12-20T05:43:22.808 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T05:43:22.808 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:22.804+0000 7f076ca4d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T05:43:22.835 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:22.833+0000 7f9a7a1e8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T05:43:22.917 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:22.916+0000 7f076ca4d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T05:43:22.937 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:22.937+0000 7f9a7a1e8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T05:43:22.976 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:22.972+0000 7f076ca4d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T05:43:23.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f9480f2d640 1 Processor -- start 2023-12-20T05:43:23.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f9480f2d640 1 -- start start 2023-12-20T05:43:23.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f9480f2d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f947c15ccf0 0x7f947c0cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f9480f2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c0cfaf0 0x7f947c0d1580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f9480f2d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f947c0d10f0 con 0x7f947c0cd7b0 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f9480f2d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f947c0ce2f0 con 0x7f947c15ccf0 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f9480f2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0d0b70 con 0x7f947c0cfaf0 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f947affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f947c0cd7b0 0x7f947c0cf3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51374/0 (socket says 172.21.15.32:51374) 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f947affd640 1 -- 172.21.15.32:0/145524141 learned_addr learned my addr 172.21.15.32:0/145524141 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f947bfff640 1 --2- 172.21.15.32:0/145524141 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c0cfaf0 0x7f947c0d1580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f947b7fe640 1 --2- 172.21.15.32:0/145524141 >> v2:172.21.15.39:3300/0 conn(0x7f947c15ccf0 0x7f947c0cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f947a7fc640 1 -- 172.21.15.32:0/145524141 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3597202487 0 0) 0x7f947c0d10f0 con 0x7f947c0cd7b0 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f947a7fc640 1 -- 172.21.15.32:0/145524141 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9464003530 con 0x7f947c0cd7b0 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f947bfff640 1 -- 172.21.15.32:0/145524141 >> v2:172.21.15.39:3300/0 conn(0x7f947c15ccf0 msgr2=0x7f947c0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f947bfff640 1 --2- 172.21.15.32:0/145524141 >> v2:172.21.15.39:3300/0 conn(0x7f947c15ccf0 0x7f947c0cd270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f947bfff640 1 -- 172.21.15.32:0/145524141 >> v1:172.21.15.32:6789/0 conn(0x7f947c0cd7b0 legacy=0x7f947c0cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f947bfff640 1 -- 172.21.15.32:0/145524141 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f947c0cdd30 con 0x7f947c0cfaf0 2023-12-20T05:43:23.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.013+0000 7f947b7fe640 1 --2- 172.21.15.32:0/145524141 >> v2:172.21.15.39:3300/0 conn(0x7f947c15ccf0 0x7f947c0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:43:23.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f947bfff640 1 --2- 172.21.15.32:0/145524141 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c0cfaf0 0x7f947c0d1580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9474009d90 tx=0x7f947401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d29c1962c2206824 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:23.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f947a7fc640 1 -- 172.21.15.32:0/145524141 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94740a5950 con 0x7f947c0cfaf0 2023-12-20T05:43:23.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f947a7fc640 1 -- 172.21.15.32:0/145524141 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94740a5ab0 con 0x7f947c0cfaf0 2023-12-20T05:43:23.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f947a7fc640 1 -- 172.21.15.32:0/145524141 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94740a5e40 con 0x7f947c0cfaf0 2023-12-20T05:43:23.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f9480f2d640 1 -- 172.21.15.32:0/145524141 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c0cfaf0 msgr2=0x7f947c0d1580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f9480f2d640 1 --2- 172.21.15.32:0/145524141 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c0cfaf0 0x7f947c0d1580 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f9474009d90 tx=0x7f947401c890 comp rx=0 tx=0).stop 2023-12-20T05:43:23.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f9480f2d640 1 -- 172.21.15.32:0/145524141 shutdown_connections 2023-12-20T05:43:23.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f9480f2d640 1 --2- 172.21.15.32:0/145524141 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c0cfaf0 0x7f947c0d1580 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f9480f2d640 1 --2- 172.21.15.32:0/145524141 >> v2:172.21.15.39:3300/0 conn(0x7f947c15ccf0 0x7f947c0cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f9480f2d640 1 -- 172.21.15.32:0/145524141 >> 172.21.15.32:0/145524141 conn(0x7f947c0658e0 msgr2=0x7f947c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:23.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f9480f2d640 1 -- 172.21.15.32:0/145524141 shutdown_connections 2023-12-20T05:43:23.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.017+0000 7f9480f2d640 1 -- 172.21.15.32:0/145524141 wait complete. 2023-12-20T05:43:23.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9480f2d640 1 Processor -- start 2023-12-20T05:43:23.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9480f2d640 1 -- start start 2023-12-20T05:43:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9480f2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f947c0cd7b0 0x7f947c1412f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9480f2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f947c0cfaf0 0x7f947c151a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9480f2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c15ccf0 0x7f947c151fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9480f2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0b2510 con 0x7f947c0cd7b0 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9480f2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0b27f0 con 0x7f947c0cfaf0 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9480f2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f947c0b2ad0 con 0x7f947c15ccf0 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f947c0cd7b0 0x7f947c1412f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947bfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c15ccf0 0x7f947c151fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f947c0cfaf0 0x7f947c151a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f947c0cd7b0 0x7f947c1412f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50060/0 (socket says 172.21.15.32:50060) 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947b7fe640 1 -- 172.21.15.32:0/1389448862 learned_addr learned my addr 172.21.15.32:0/1389448862 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947bfff640 1 -- 172.21.15.32:0/1389448862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f947c0cfaf0 msgr2=0x7f947c151a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947bfff640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f947c0cfaf0 0x7f947c151a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947bfff640 1 -- 172.21.15.32:0/1389448862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f947c0cd7b0 msgr2=0x7f947c1412f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947bfff640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f947c0cd7b0 0x7f947c1412f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947bfff640 1 -- 172.21.15.32:0/1389448862 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f946c009420 con 0x7f947c15ccf0 2023-12-20T05:43:23.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947affd640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f947c0cfaf0 0x7f947c151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:43:23.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f947bfff640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c15ccf0 0x7f947c151fd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f94740ac000 tx=0x7f9474019630 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:23.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9463fff640 1 -- 172.21.15.32:0/1389448862 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94740bf070 con 0x7f947c15ccf0 2023-12-20T05:43:23.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9463fff640 1 -- 172.21.15.32:0/1389448862 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94740aeab0 con 0x7f947c15ccf0 2023-12-20T05:43:23.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9463fff640 1 -- 172.21.15.32:0/1389448862 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94740aee60 con 0x7f947c15ccf0 2023-12-20T05:43:23.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9480f2d640 1 -- 172.21.15.32:0/1389448862 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9474009a70 con 0x7f947c15ccf0 2023-12-20T05:43:23.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.021+0000 7f9480f2d640 1 -- 172.21.15.32:0/1389448862 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f947c0b3090 con 0x7f947c15ccf0 2023-12-20T05:43:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.025+0000 7f9480f2d640 1 -- 172.21.15.32:0/1389448862 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f947c0b3280 con 0x7f947c15ccf0 2023-12-20T05:43:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.025+0000 7f9463fff640 1 -- 172.21.15.32:0/1389448862 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 80) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f94740b4050 con 0x7f947c15ccf0 2023-12-20T05:43:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.025+0000 7f9463fff640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9468065b80 0x7f9468068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.025+0000 7f9463fff640 1 -- 172.21.15.32:0/1389448862 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(144..144 src has 1..144) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f947411dd40 con 0x7f947c15ccf0 2023-12-20T05:43:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.025+0000 7f947b7fe640 1 -- 172.21.15.32:0/1389448862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9468065b80 msgr2=0x7f9468068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:23.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.025+0000 7f947b7fe640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9468065b80 0x7f9468068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.029+0000 7f9463fff640 1 -- 172.21.15.32:0/1389448862 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f94740eb510 con 0x7f947c15ccf0 2023-12-20T05:43:23.032 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:23.033+0000 7f9a7a1e8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T05:43:23.108 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:23.109+0000 7f9a7a1e8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T05:43:23.198 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:23.196+0000 7f076ca4d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T05:43:23.227 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:23.229+0000 7f9a7a1e8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T05:43:23.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.229+0000 7f947b7fe640 1 -- 172.21.15.32:0/1389448862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9468065b80 msgr2=0x7f9468068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:23.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.229+0000 7f947b7fe640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9468065b80 0x7f9468068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T05:43:23.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.245+0000 7f9480f2d640 1 -- 172.21.15.32:0/1389448862 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f947c074100 con 0x7f947c15ccf0 2023-12-20T05:43:23.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.245+0000 7f9463fff640 1 -- 172.21.15.32:0/1389448862 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7f94740eeb30 con 0x7f947c15ccf0 2023-12-20T05:43:23.247 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T05:43:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 -- 172.21.15.32:0/1389448862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9468065b80 msgr2=0x7f9468068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9468065b80 0x7f9468068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 -- 172.21.15.32:0/1389448862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c15ccf0 msgr2=0x7f947c151fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c15ccf0 0x7f947c151fd0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7f94740ac000 tx=0x7f9474019630 comp rx=0 tx=0).stop 2023-12-20T05:43:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 -- 172.21.15.32:0/1389448862 shutdown_connections 2023-12-20T05:43:23.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9468065b80 0x7f9468068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f947c15ccf0 0x7f947c151fd0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f947c0cfaf0 0x7f947c151a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 --2- 172.21.15.32:0/1389448862 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f947c0cd7b0 0x7f947c1412f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 -- 172.21.15.32:0/1389448862 >> 172.21.15.32:0/1389448862 conn(0x7f947c0658e0 msgr2=0x7f947c0c8d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 -- 172.21.15.32:0/1389448862 shutdown_connections 2023-12-20T05:43:23.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.249+0000 7f9480f2d640 1 -- 172.21.15.32:0/1389448862 wait complete. 2023-12-20T05:43:23.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T05:43:23.304 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:23.305+0000 7f9a7a1e8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T05:43:23.318 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:23.312+0000 7f076ca4d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T05:43:23.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.493+0000 7f153359e640 1 Processor -- start 2023-12-20T05:43:23.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.493+0000 7f153359e640 1 -- start start 2023-12-20T05:43:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.493+0000 7f153359e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15340b8480 0x7f153406af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.493+0000 7f153359e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15340cd4d0 con 0x7f1534161500 2023-12-20T05:43:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.493+0000 7f153359e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15340ce4e0 con 0x7f153406b460 2023-12-20T05:43:23.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.493+0000 7f153359e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1534161930 con 0x7f15340b8480 2023-12-20T05:43:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.493+0000 7f153259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15340b8480 0x7f153406af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.493+0000 7f1532d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1534161500 0x7f1534153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51376/0 (socket says 172.21.15.32:51376) 2023-12-20T05:43:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.493+0000 7f1532d9d640 1 -- 172.21.15.32:0/2100629963 learned_addr learned my addr 172.21.15.32:0/2100629963 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.497+0000 7f153259c640 1 -- 172.21.15.32:0/2100629963 >> v1:172.21.15.39:6789/0 conn(0x7f153406b460 legacy=0x7f1534160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.497+0000 7f153259c640 1 -- 172.21.15.32:0/2100629963 >> v1:172.21.15.32:6789/0 conn(0x7f1534161500 legacy=0x7f1534153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.497+0000 7f153159a640 1 -- 172.21.15.32:0/2100629963 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1645783321 0 0) 0x7f15340cd4d0 con 0x7f1534161500 2023-12-20T05:43:23.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.497+0000 7f153259c640 1 -- 172.21.15.32:0/2100629963 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15340cd720 con 0x7f15340b8480 2023-12-20T05:43:23.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.501+0000 7f153259c640 1 --2- 172.21.15.32:0/2100629963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15340b8480 0x7f153406af20 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f15240097c0 tx=0x7f152401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ada2c604fc648bbe server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.501+0000 7f153159a640 1 -- 172.21.15.32:0/2100629963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15240ab600 con 0x7f15340b8480 2023-12-20T05:43:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.501+0000 7f153159a640 1 -- 172.21.15.32:0/2100629963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15240a7050 con 0x7f15340b8480 2023-12-20T05:43:23.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.501+0000 7f153159a640 1 -- 172.21.15.32:0/2100629963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15240ab980 con 0x7f15340b8480 2023-12-20T05:43:23.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.501+0000 7f153359e640 1 -- 172.21.15.32:0/2100629963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15340b8480 msgr2=0x7f153406af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.501+0000 7f153359e640 1 --2- 172.21.15.32:0/2100629963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15340b8480 0x7f153406af20 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7f15240097c0 tx=0x7f152401c480 comp rx=0 tx=0).stop 2023-12-20T05:43:23.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.501+0000 7f153359e640 1 -- 172.21.15.32:0/2100629963 shutdown_connections 2023-12-20T05:43:23.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.501+0000 7f153359e640 1 --2- 172.21.15.32:0/2100629963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15340b8480 0x7f153406af20 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.501+0000 7f153359e640 1 -- 172.21.15.32:0/2100629963 >> 172.21.15.32:0/2100629963 conn(0x7f15340d2a50 msgr2=0x7f15340d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:23.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.501+0000 7f153359e640 1 -- 172.21.15.32:0/2100629963 shutdown_connections 2023-12-20T05:43:23.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.501+0000 7f153359e640 1 -- 172.21.15.32:0/2100629963 wait complete. 2023-12-20T05:43:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f153359e640 1 Processor -- start 2023-12-20T05:43:23.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f153359e640 1 -- start start 2023-12-20T05:43:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f153359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f153406b460 0x7f1534152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f153359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15340b8480 0x7f15340c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f153359e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1534161500 0x7f1534152f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f153359e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15340c7dd0 con 0x7f15340b8480 2023-12-20T05:43:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f153359e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15340c71c0 con 0x7f153406b460 2023-12-20T05:43:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f153359e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15340c74a0 con 0x7f1534161500 2023-12-20T05:43:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1531d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15340b8480 0x7f15340c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1532d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1534161500 0x7f1534152f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1531d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15340b8480 0x7f15340c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50076/0 (socket says 172.21.15.32:50076) 2023-12-20T05:43:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1531d9b640 1 -- 172.21.15.32:0/916290936 learned_addr learned my addr 172.21.15.32:0/916290936 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:23.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f153259c640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f153406b460 0x7f1534152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:23.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1531d9b640 1 -- 172.21.15.32:0/916290936 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1534161500 msgr2=0x7f1534152f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1531d9b640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1534161500 0x7f1534152f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1531d9b640 1 -- 172.21.15.32:0/916290936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f153406b460 msgr2=0x7f1534152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1531d9b640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f153406b460 0x7f1534152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1531d9b640 1 -- 172.21.15.32:0/916290936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1524009420 con 0x7f15340b8480 2023-12-20T05:43:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1531d9b640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15340b8480 0x7f15340c8260 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f152800c500 tx=0x7f152800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1522ffd640 1 -- 172.21.15.32:0/916290936 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528005220 con 0x7f15340b8480 2023-12-20T05:43:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1522ffd640 1 -- 172.21.15.32:0/916290936 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1528005380 con 0x7f15340b8480 2023-12-20T05:43:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f1522ffd640 1 -- 172.21.15.32:0/916290936 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1528007270 con 0x7f15340b8480 2023-12-20T05:43:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f153359e640 1 -- 172.21.15.32:0/916290936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15340c7780 con 0x7f15340b8480 2023-12-20T05:43:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.505+0000 7f153359e640 1 -- 172.21.15.32:0/916290936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15340c87a0 con 0x7f15340b8480 2023-12-20T05:43:23.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.509+0000 7f153359e640 1 -- 172.21.15.32:0/916290936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15340c7f40 con 0x7f15340b8480 2023-12-20T05:43:23.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.509+0000 7f1522ffd640 1 -- 172.21.15.32:0/916290936 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 80) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f15280073d0 con 0x7f15340b8480 2023-12-20T05:43:23.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.509+0000 7f1522ffd640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1514065b80 0x7f1514068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.510 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:23.509+0000 7f9a7a1e8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T05:43:23.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.509+0000 7f153259c640 1 -- 172.21.15.32:0/916290936 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1514065b80 msgr2=0x7f1514068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:23.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.509+0000 7f1522ffd640 1 -- 172.21.15.32:0/916290936 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f1528082d90 con 0x7f15340b8480 2023-12-20T05:43:23.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.509+0000 7f153259c640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1514065b80 0x7f1514068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:23.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.513+0000 7f1522ffd640 1 -- 172.21.15.32:0/916290936 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1528050450 con 0x7f15340b8480 2023-12-20T05:43:23.576 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:23.577+0000 7f9a7a1e8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T05:43:23.651 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T05:43:23.651 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T05:43:23.651 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T05:43:23.660 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:23.656+0000 7f076ca4d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T05:43:23.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.689+0000 7f153359e640 1 -- 172.21.15.32:0/916290936 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f1534074100 con 0x7f15340b8480 2023-12-20T05:43:23.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.689+0000 7f1522ffd640 1 -- 172.21.15.32:0/916290936 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v145) v1 ==== 80+0+459 (secure 0 0 0) 0x7f1528053a70 con 0x7f15340b8480 2023-12-20T05:43:23.690 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/801772685 2023-12-21T04:47:33.113748+0000 2023-12-20T05:43:23.690 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2511943559 2023-12-21T04:47:33.113748+0000 2023-12-20T05:43:23.690 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3621453434 2023-12-21T04:47:33.113748+0000 2023-12-20T05:43:23.690 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/38009 2023-12-21T04:47:33.113748+0000 2023-12-20T05:43:23.690 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/38009 2023-12-21T04:47:33.113748+0000 2023-12-20T05:43:23.690 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1810110767 2023-12-21T04:47:33.113748+0000 2023-12-20T05:43:23.691 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2135559470 2023-12-21T04:47:33.113748+0000 2023-12-20T05:43:23.691 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1663137028 2023-12-21T04:47:33.113748+0000 2023-12-20T05:43:23.691 INFO:teuthology.orchestra.run.smithi032.stderr:listed 8 entries 2023-12-20T05:43:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 -- 172.21.15.32:0/916290936 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1514065b80 msgr2=0x7f1514068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1514065b80 0x7f1514068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 -- 172.21.15.32:0/916290936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15340b8480 msgr2=0x7f15340c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15340b8480 0x7f15340c8260 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7f152800c500 tx=0x7f152800c9c0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 -- 172.21.15.32:0/916290936 shutdown_connections 2023-12-20T05:43:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1514065b80 0x7f1514068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1534161500 0x7f1534152f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15340b8480 0x7f15340c8260 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 --2- 172.21.15.32:0/916290936 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f153406b460 0x7f1534152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 -- 172.21.15.32:0/916290936 >> 172.21.15.32:0/916290936 conn(0x7f15340d2a50 msgr2=0x7f15340cf460 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 -- 172.21.15.32:0/916290936 shutdown_connections 2023-12-20T05:43:23.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.693+0000 7f153359e640 1 -- 172.21.15.32:0/916290936 wait complete. 2023-12-20T05:43:23.712 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T05:43:23.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:23.712 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T05:43:23.761 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T05:43:23.761 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:23.761 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T05:43:23.779 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T05:43:23.780 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:43:23.790 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T05:43:23.790 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:43:23.790 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:43:23.804 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:23.800+0000 7f076ca4d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:43:23.830 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T05:43:23.830 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T05:43:23.830 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T05:43:23.832 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T05:43:23.832 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T05:43:23.832 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:43:23.839 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:23.837+0000 7f9a7a1e8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T05:43:23.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T05:43:23.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:43:23.937 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:23.932+0000 7f076ca4d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T05:43:23.988 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f9a7a1e8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:43:23.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f5504517640 1 Processor -- start 2023-12-20T05:43:23.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f5504517640 1 -- start start 2023-12-20T05:43:23.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f5504517640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc15cdb0 0x7f54fc0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f5504517640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f54fc0d2740 0x7f54fc0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:23.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f5504517640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f54fc0b6ca0 con 0x7f54fc0d1120 2023-12-20T05:43:23.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f5504517640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f54fc15d190 con 0x7f54fc0d2740 2023-12-20T05:43:23.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f5504517640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc0d16a0 con 0x7f54fc15cdb0 2023-12-20T05:43:23.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f5501ab2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f54fc0d2740 0x7f54fc0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:23.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f55022b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc15cdb0 0x7f54fc0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:23.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f55022b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc15cdb0 0x7f54fc0d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:60664/0 (socket says 172.21.15.32:60664) 2023-12-20T05:43:23.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f55022b3640 1 -- 172.21.15.32:0/976097029 learned_addr learned my addr 172.21.15.32:0/976097029 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:23.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.989+0000 7f5502ab4640 1 --1- 172.21.15.32:0/976097029 >> v1:172.21.15.39:6789/0 conn(0x7f54fc0d1120 0x7f54fc0b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:48570/0 (socket says 172.21.15.32:48570) 2023-12-20T05:43:23.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.993+0000 7f55012b1640 1 -- 172.21.15.32:0/976097029 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580077460 0 0) 0x7f54f40014a0 con 0x7f54fc0d1120 2023-12-20T05:43:23.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.993+0000 7f55022b3640 1 -- 172.21.15.32:0/976097029 >> v2:172.21.15.39:3300/0 conn(0x7f54fc0d2740 msgr2=0x7f54fc0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:23.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.993+0000 7f55022b3640 1 --2- 172.21.15.32:0/976097029 >> v2:172.21.15.39:3300/0 conn(0x7f54fc0d2740 0x7f54fc0cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.993+0000 7f55022b3640 1 -- 172.21.15.32:0/976097029 >> v1:172.21.15.39:6789/0 conn(0x7f54fc0d1120 legacy=0x7f54fc0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.993+0000 7f55022b3640 1 -- 172.21.15.32:0/976097029 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54fc0d1980 con 0x7f54fc15cdb0 2023-12-20T05:43:23.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.993+0000 7f55022b3640 1 --2- 172.21.15.32:0/976097029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc15cdb0 0x7f54fc0d0be0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009870 tx=0x7f54f801ed30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=97552512d097d744 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:23.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.993+0000 7f55012b1640 1 -- 172.21.15.32:0/976097029 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80a8a30 con 0x7f54fc15cdb0 2023-12-20T05:43:23.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.993+0000 7f55012b1640 1 -- 172.21.15.32:0/976097029 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f80a8b90 con 0x7f54fc15cdb0 2023-12-20T05:43:23.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.993+0000 7f55012b1640 1 -- 172.21.15.32:0/976097029 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80b0440 con 0x7f54fc15cdb0 2023-12-20T05:43:23.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 -- 172.21.15.32:0/976097029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc15cdb0 msgr2=0x7f54fc0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:23.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 --2- 172.21.15.32:0/976097029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc15cdb0 0x7f54fc0d0be0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f54f8009870 tx=0x7f54f801ed30 comp rx=0 tx=0).stop 2023-12-20T05:43:23.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 -- 172.21.15.32:0/976097029 shutdown_connections 2023-12-20T05:43:23.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 --2- 172.21.15.32:0/976097029 >> v2:172.21.15.39:3300/0 conn(0x7f54fc0d2740 0x7f54fc0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 --2- 172.21.15.32:0/976097029 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc15cdb0 0x7f54fc0d0be0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:23.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 -- 172.21.15.32:0/976097029 >> 172.21.15.32:0/976097029 conn(0x7f54fc065ee0 msgr2=0x7f54fc06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:23.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 -- 172.21.15.32:0/976097029 shutdown_connections 2023-12-20T05:43:23.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 -- 172.21.15.32:0/976097029 wait complete. 2023-12-20T05:43:23.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 Processor -- start 2023-12-20T05:43:23.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 -- start start 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54fc0d1120 0x7f54fc15fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc0d2740 0x7f54fc15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc15cdb0 0x7f54fc14c9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc15ed30 con 0x7f54fc0d1120 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc15dc90 con 0x7f54fc15cdb0 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5504517640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f54fc15df70 con 0x7f54fc0d2740 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f55022b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54fc0d1120 0x7f54fc15fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f55022b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54fc0d1120 0x7f54fc15fd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50080/0 (socket says 172.21.15.32:50080) 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f55022b3640 1 -- 172.21.15.32:0/1100983049 learned_addr learned my addr 172.21.15.32:0/1100983049 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5501ab2640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc0d2740 0x7f54fc15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:23.997+0000 7f5502ab4640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc15cdb0 0x7f54fc14c9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f55022b3640 1 -- 172.21.15.32:0/1100983049 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc0d2740 msgr2=0x7f54fc15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f55022b3640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc0d2740 0x7f54fc15d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f55022b3640 1 -- 172.21.15.32:0/1100983049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc15cdb0 msgr2=0x7f54fc14c9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f55022b3640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc15cdb0 0x7f54fc14c9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f55022b3640 1 -- 172.21.15.32:0/1100983049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f54f80094d0 con 0x7f54fc0d1120 2023-12-20T05:43:24.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f55022b3640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54fc0d1120 0x7f54fc15fd30 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f54f80b3040 tx=0x7f54f801bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:24.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f54f2ffd640 1 -- 172.21.15.32:0/1100983049 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80b9620 con 0x7f54fc0d1120 2023-12-20T05:43:24.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f54f2ffd640 1 -- 172.21.15.32:0/1100983049 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f54f80b97c0 con 0x7f54fc0d1120 2023-12-20T05:43:24.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f54f2ffd640 1 -- 172.21.15.32:0/1100983049 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f54f80b9b70 con 0x7f54fc0d1120 2023-12-20T05:43:24.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f5504517640 1 -- 172.21.15.32:0/1100983049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f54fc15e250 con 0x7f54fc0d1120 2023-12-20T05:43:24.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f5504517640 1 -- 172.21.15.32:0/1100983049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f54fc15e810 con 0x7f54fc0d1120 2023-12-20T05:43:24.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f5504517640 1 -- 172.21.15.32:0/1100983049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f54fc15ea00 con 0x7f54fc0d1120 2023-12-20T05:43:24.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f54f2ffd640 1 -- 172.21.15.32:0/1100983049 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 80) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f54f80b7070 con 0x7f54fc0d1120 2023-12-20T05:43:24.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f54f2ffd640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f54e0065b80 0x7f54e0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f54f2ffd640 1 -- 172.21.15.32:0/1100983049 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f54f8121400 con 0x7f54fc0d1120 2023-12-20T05:43:24.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f5501ab2640 1 -- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f54e0065b80 msgr2=0x7f54e0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:24.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.001+0000 7f5501ab2640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f54e0065b80 0x7f54e0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.005+0000 7f54f2ffd640 1 -- 172.21.15.32:0/1100983049 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f54f80eeab0 con 0x7f54fc0d1120 2023-12-20T05:43:24.011 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:24.008+0000 7f076ca4d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T05:43:24.125 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:24.125+0000 7f9a7a1e8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T05:43:24.165 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:24.160+0000 7f076ca4d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:43:24.202 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:24.201+0000 7f9a7a1e8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T05:43:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.205+0000 7f5501ab2640 1 -- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f54e0065b80 msgr2=0x7f54e0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.205+0000 7f5501ab2640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f54e0065b80 0x7f54e0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T05:43:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.205+0000 7f5504517640 1 -- 172.21.15.32:0/1100983049 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f54fc04e000 con 0x7f54fc0d1120 2023-12-20T05:43:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.205+0000 7f54f2ffd640 1 -- 172.21.15.32:0/1100983049 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4708 (secure 0 0 0) 0x7f54f80f20d0 con 0x7f54fc0d1120 2023-12-20T05:43:24.206 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:24.206 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3306270605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3306270605},{"type":"v1","addr":"172.21.15.39:6834","nonce":3306270605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4000562557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4000562557},{"type":"v1","addr":"172.21.15.39:6835","nonce":4000562557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/310576469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":310576469},{"type":"v1","addr":"172.21.15.32:6835","nonce":310576469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:47:21.050837+0000","modified":"2023-12-20T04:47:24.070982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8397},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8397":{"gid":8397,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3314020360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3314020360},{"type":"v1","addr":"172.21.15.32:6837","nonce":3314020360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-20T05:43:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 -- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f54e0065b80 msgr2=0x7f54e0068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f54e0065b80 0x7f54e0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 -- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54fc0d1120 msgr2=0x7f54fc15fd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54fc0d1120 0x7f54fc15fd30 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7f54f80b3040 tx=0x7f54f801bad0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 -- 172.21.15.32:0/1100983049 shutdown_connections 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f54e0065b80 0x7f54e0068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f54fc15cdb0 0x7f54fc14c9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f54fc0d2740 0x7f54fc15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 --2- 172.21.15.32:0/1100983049 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f54fc0d1120 0x7f54fc15fd30 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 -- 172.21.15.32:0/1100983049 >> 172.21.15.32:0/1100983049 conn(0x7f54fc065ee0 msgr2=0x7f54fc15b0d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 -- 172.21.15.32:0/1100983049 shutdown_connections 2023-12-20T05:43:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.209+0000 7f5504517640 1 -- 172.21.15.32:0/1100983049 wait complete. 2023-12-20T05:43:24.237 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:43:24.360 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:24.361+0000 7f9a7a1e8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:43:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2088b39640 1 Processor -- start 2023-12-20T05:43:24.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2088b39640 1 -- start start 2023-12-20T05:43:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2088b39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20840cfc80 0x7f2084167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2088b39640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f20840b6ca0 con 0x7f20840d2740 2023-12-20T05:43:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2088b39640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f208415d190 con 0x7f208415cdb0 2023-12-20T05:43:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2088b39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f20840d16a0 con 0x7f20840cfc80 2023-12-20T05:43:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2081d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f20840d2740 0x7f20840cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51378/0 (socket says 172.21.15.32:51378) 2023-12-20T05:43:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2081d9b640 1 -- 172.21.15.32:0/449330414 learned_addr learned my addr 172.21.15.32:0/449330414 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:24.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2082d9d640 1 --2- 172.21.15.32:0/449330414 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20840cfc80 0x7f2084167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f208159a640 1 -- 172.21.15.32:0/449330414 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3322273825 0 0) 0x7f2078001120 con 0x7f20840d2740 2023-12-20T05:43:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2082d9d640 1 -- 172.21.15.32:0/449330414 >> v1:172.21.15.39:6789/0 conn(0x7f208415cdb0 legacy=0x7f20840d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2082d9d640 1 -- 172.21.15.32:0/449330414 >> v1:172.21.15.32:6789/0 conn(0x7f20840d2740 legacy=0x7f20840cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.381+0000 7f2082d9d640 1 -- 172.21.15.32:0/449330414 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f20840d1980 con 0x7f20840cfc80 2023-12-20T05:43:24.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.385+0000 7f2082d9d640 1 --2- 172.21.15.32:0/449330414 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20840cfc80 0x7f2084167910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f207c009dc0 tx=0x7f207c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a52569f3e20af179 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.385+0000 7f208159a640 1 -- 172.21.15.32:0/449330414 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c0a5910 con 0x7f20840cfc80 2023-12-20T05:43:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.385+0000 7f208159a640 1 -- 172.21.15.32:0/449330414 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207c0a5a70 con 0x7f20840cfc80 2023-12-20T05:43:24.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.385+0000 7f208159a640 1 -- 172.21.15.32:0/449330414 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f207c0a5e00 con 0x7f20840cfc80 2023-12-20T05:43:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.385+0000 7f2088b39640 1 -- 172.21.15.32:0/449330414 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20840cfc80 msgr2=0x7f2084167910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.385+0000 7f2088b39640 1 --2- 172.21.15.32:0/449330414 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20840cfc80 0x7f2084167910 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f207c009dc0 tx=0x7f207c01c8b0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.385+0000 7f2088b39640 1 -- 172.21.15.32:0/449330414 shutdown_connections 2023-12-20T05:43:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.385+0000 7f2088b39640 1 --2- 172.21.15.32:0/449330414 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f20840cfc80 0x7f2084167910 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.385+0000 7f2088b39640 1 -- 172.21.15.32:0/449330414 >> 172.21.15.32:0/449330414 conn(0x7f2084065ee0 msgr2=0x7f208406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.385+0000 7f2088b39640 1 -- 172.21.15.32:0/449330414 shutdown_connections 2023-12-20T05:43:24.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.385+0000 7f2088b39640 1 -- 172.21.15.32:0/449330414 wait complete. 2023-12-20T05:43:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f2088b39640 1 Processor -- start 2023-12-20T05:43:24.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f2088b39640 1 -- start start 2023-12-20T05:43:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f2088b39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20840cfc80 0x7f208415fb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f2088b39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20840d2740 0x7f2084150a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f2088b39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f208415cdb0 0x7f20840c7b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f2088b39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208415eaf0 con 0x7f20840cfc80 2023-12-20T05:43:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f2088b39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f208415edd0 con 0x7f20840d2740 2023-12-20T05:43:24.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f2088b39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f208415f0b0 con 0x7f208415cdb0 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f208259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20840cfc80 0x7f208415fb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f208259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20840cfc80 0x7f208415fb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50082/0 (socket says 172.21.15.32:50082) 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f208259c640 1 -- 172.21.15.32:0/615279431 learned_addr learned my addr 172.21.15.32:0/615279431 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f2081d9b640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20840d2740 0x7f2084150a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f2082d9d640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f208415cdb0 0x7f20840c7b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f208259c640 1 -- 172.21.15.32:0/615279431 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f208415cdb0 msgr2=0x7f20840c7b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f208259c640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f208415cdb0 0x7f20840c7b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f208259c640 1 -- 172.21.15.32:0/615279431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20840d2740 msgr2=0x7f2084150a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f208259c640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20840d2740 0x7f2084150a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f208259c640 1 -- 172.21.15.32:0/615279431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f207c009a70 con 0x7f20840cfc80 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f208259c640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20840cfc80 0x7f208415fb90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2074009e40 tx=0x7f207400c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.389+0000 7f2072ffd640 1 -- 172.21.15.32:0/615279431 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074018070 con 0x7f20840cfc80 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.393+0000 7f2072ffd640 1 -- 172.21.15.32:0/615279431 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f207400cdb0 con 0x7f20840cfc80 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.393+0000 7f2088b39640 1 -- 172.21.15.32:0/615279431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f208415f390 con 0x7f20840cfc80 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.393+0000 7f2072ffd640 1 -- 172.21.15.32:0/615279431 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2074004260 con 0x7f20840cfc80 2023-12-20T05:43:24.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.393+0000 7f2088b39640 1 -- 172.21.15.32:0/615279431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f20840c8090 con 0x7f20840cfc80 2023-12-20T05:43:24.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.393+0000 7f2088b39640 1 -- 172.21.15.32:0/615279431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f208404e000 con 0x7f20840cfc80 2023-12-20T05:43:24.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.393+0000 7f2072ffd640 1 -- 172.21.15.32:0/615279431 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 80) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f20740087f0 con 0x7f20840cfc80 2023-12-20T05:43:24.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.393+0000 7f2072ffd640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2064065b80 0x7f2064068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.393+0000 7f2072ffd640 1 -- 172.21.15.32:0/615279431 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f2074085610 con 0x7f20840cfc80 2023-12-20T05:43:24.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.393+0000 7f2081d9b640 1 -- 172.21.15.32:0/615279431 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2064065b80 msgr2=0x7f2064068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:24.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.393+0000 7f2081d9b640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2064065b80 0x7f2064068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:24.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.397+0000 7f2072ffd640 1 -- 172.21.15.32:0/615279431 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2074052d50 con 0x7f20840cfc80 2023-12-20T05:43:24.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.557+0000 7f2088b39640 1 -- 172.21.15.32:0/615279431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f208406b8d0 con 0x7f20840cfc80 2023-12-20T05:43:24.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2072ffd640 1 -- 172.21.15.32:0/615279431 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v145) v1 ==== 74+0+16808 (secure 0 0 0) 0x7f2074056370 con 0x7f20840cfc80 2023-12-20T05:43:24.560 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:24.560 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":145,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:43:23.114326+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:47:19.596882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-20T04:47:20.055023+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":145,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":2}]},{"pgid":"17.17","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/801772685":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/2511943559":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/3621453434":"2023-12-21T04:47:33.113748+0000","172.21.15.32:6833/38009":"2023-12-21T04:47:33.113748+0000","172.21.15.32:6832/38009":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/1810110767":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/2135559470":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/1663137028":"2023-12-21T04:47:33.113748+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[{"pool":17,"snaps":[{"begin":57,"length":1},{"begin":113,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:43:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 -- 172.21.15.32:0/615279431 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2064065b80 msgr2=0x7f2064068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2064065b80 0x7f2064068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 -- 172.21.15.32:0/615279431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20840cfc80 msgr2=0x7f208415fb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20840cfc80 0x7f208415fb90 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f2074009e40 tx=0x7f207400c430 comp rx=0 tx=0).stop 2023-12-20T05:43:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 -- 172.21.15.32:0/615279431 shutdown_connections 2023-12-20T05:43:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2064065b80 0x7f2064068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f208415cdb0 0x7f20840c7b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f20840d2740 0x7f2084150a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 --2- 172.21.15.32:0/615279431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f20840cfc80 0x7f208415fb90 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 -- 172.21.15.32:0/615279431 >> 172.21.15.32:0/615279431 conn(0x7f2084065ee0 msgr2=0x7f208415b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 -- 172.21.15.32:0/615279431 shutdown_connections 2023-12-20T05:43:24.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.561+0000 7f2088b39640 1 -- 172.21.15.32:0/615279431 wait complete. 2023-12-20T05:43:24.583 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 16, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T04:47:19.596882+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '137', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 17, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T04:47:20.055023+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '145', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 113, 'snap_epoch': 145, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T05:43:24.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T05:43:24.731 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:24.728+0000 7f076ca4d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T05:43:24.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91c29a5640 1 Processor -- start 2023-12-20T05:43:24.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91c29a5640 1 -- start start 2023-12-20T05:43:24.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91c29a5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91bc0b0760 0x7f91bc0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91c29a5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91bc0b1100 con 0x7f91bc157480 2023-12-20T05:43:24.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91c29a5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91bc0cde20 con 0x7f91bc0cc4b0 2023-12-20T05:43:24.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91c29a5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0c9f50 con 0x7f91bc0b0760 2023-12-20T05:43:24.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91bbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f91bc157480 0x7f91bc0c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51382/0 (socket says 172.21.15.32:51382) 2023-12-20T05:43:24.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91bbfff640 1 -- 172.21.15.32:0/796457518 learned_addr learned my addr 172.21.15.32:0/796457518 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:24.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91bb7fe640 1 --2- 172.21.15.32:0/796457518 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91bc0b0760 0x7f91bc0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:24.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91baffd640 1 -- 172.21.15.32:0/796457518 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1580420523 0 0) 0x7f91bc0b1100 con 0x7f91bc157480 2023-12-20T05:43:24.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91baffd640 1 -- 172.21.15.32:0/796457518 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91a4003530 con 0x7f91bc157480 2023-12-20T05:43:24.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91baffd640 1 -- 172.21.15.32:0/796457518 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2008661132 0 0) 0x7f91bc0cde20 con 0x7f91bc0cc4b0 2023-12-20T05:43:24.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91baffd640 1 -- 172.21.15.32:0/796457518 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f91bc0b1100 con 0x7f91bc0cc4b0 2023-12-20T05:43:24.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91bb7fe640 1 -- 172.21.15.32:0/796457518 >> v1:172.21.15.39:6789/0 conn(0x7f91bc0cc4b0 legacy=0x7f91bc0cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91bb7fe640 1 -- 172.21.15.32:0/796457518 >> v1:172.21.15.32:6789/0 conn(0x7f91bc157480 legacy=0x7f91bc0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.733+0000 7f91bb7fe640 1 -- 172.21.15.32:0/796457518 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91bc0c8e30 con 0x7f91bc0b0760 2023-12-20T05:43:24.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.737+0000 7f91bb7fe640 1 --2- 172.21.15.32:0/796457518 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91bc0b0760 0x7f91bc0b0bc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f91b00097f0 tx=0x7f91b001c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8b7a82cc825acf3a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:24.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.737+0000 7f91baffd640 1 -- 172.21.15.32:0/796457518 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b00ab600 con 0x7f91bc0b0760 2023-12-20T05:43:24.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.737+0000 7f91baffd640 1 -- 172.21.15.32:0/796457518 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91b00ab760 con 0x7f91bc0b0760 2023-12-20T05:43:24.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.737+0000 7f91baffd640 1 -- 172.21.15.32:0/796457518 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91b00abad0 con 0x7f91bc0b0760 2023-12-20T05:43:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.737+0000 7f91c29a5640 1 -- 172.21.15.32:0/796457518 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91bc0b0760 msgr2=0x7f91bc0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.737+0000 7f91c29a5640 1 --2- 172.21.15.32:0/796457518 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91bc0b0760 0x7f91bc0b0bc0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f91b00097f0 tx=0x7f91b001c400 comp rx=0 tx=0).stop 2023-12-20T05:43:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.737+0000 7f91c29a5640 1 -- 172.21.15.32:0/796457518 shutdown_connections 2023-12-20T05:43:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.737+0000 7f91c29a5640 1 --2- 172.21.15.32:0/796457518 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91bc0b0760 0x7f91bc0b0bc0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.737+0000 7f91c29a5640 1 -- 172.21.15.32:0/796457518 >> 172.21.15.32:0/796457518 conn(0x7f91bc06a510 msgr2=0x7f91bc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.737+0000 7f91c29a5640 1 -- 172.21.15.32:0/796457518 shutdown_connections 2023-12-20T05:43:24.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.737+0000 7f91c29a5640 1 -- 172.21.15.32:0/796457518 wait complete. 2023-12-20T05:43:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91c29a5640 1 Processor -- start 2023-12-20T05:43:24.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91c29a5640 1 -- start start 2023-12-20T05:43:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91c29a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc0b0760 0x7f91bc1499d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91c29a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91bc0cc4b0 0x7f91bc0b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91c29a5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91bc157480 0x7f91bc0b3e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91c29a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0c5a50 con 0x7f91bc0b0760 2023-12-20T05:43:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91c29a5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0c5d30 con 0x7f91bc0cc4b0 2023-12-20T05:43:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91c29a5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91bc0c6010 con 0x7f91bc157480 2023-12-20T05:43:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc0b0760 0x7f91bc1499d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91c0f42640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91bc157480 0x7f91bc0b3e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91bb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91bc0cc4b0 0x7f91bc0b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:24.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc0b0760 0x7f91bc1499d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50092/0 (socket says 172.21.15.32:50092) 2023-12-20T05:43:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91bbfff640 1 -- 172.21.15.32:0/3656302615 learned_addr learned my addr 172.21.15.32:0/3656302615 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91bbfff640 1 -- 172.21.15.32:0/3656302615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91bc157480 msgr2=0x7f91bc0b3e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91bbfff640 1 --2- 172.21.15.32:0/3656302615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91bc157480 0x7f91bc0b3e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91bbfff640 1 -- 172.21.15.32:0/3656302615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91bc0cc4b0 msgr2=0x7f91bc0b3940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91bbfff640 1 --2- 172.21.15.32:0/3656302615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91bc0cc4b0 0x7f91bc0b3940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91bbfff640 1 -- 172.21.15.32:0/3656302615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91b0009420 con 0x7f91bc0b0760 2023-12-20T05:43:24.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.741+0000 7f91bbfff640 1 --2- 172.21.15.32:0/3656302615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc0b0760 0x7f91bc1499d0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00c4d0 tx=0x7f91ac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.745+0000 7f91b8ff9640 1 -- 172.21.15.32:0/3656302615 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac005cf0 con 0x7f91bc0b0760 2023-12-20T05:43:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.745+0000 7f91b8ff9640 1 -- 172.21.15.32:0/3656302615 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91ac005e50 con 0x7f91bc0b0760 2023-12-20T05:43:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.745+0000 7f91b8ff9640 1 -- 172.21.15.32:0/3656302615 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91ac007270 con 0x7f91bc0b0760 2023-12-20T05:43:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.745+0000 7f91c29a5640 1 -- 172.21.15.32:0/3656302615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91bc0c62f0 con 0x7f91bc0b0760 2023-12-20T05:43:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.745+0000 7f91c29a5640 1 -- 172.21.15.32:0/3656302615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91bc149f10 con 0x7f91bc0b0760 2023-12-20T05:43:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.745+0000 7f91c29a5640 1 -- 172.21.15.32:0/3656302615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f91bc04e000 con 0x7f91bc0b0760 2023-12-20T05:43:24.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.745+0000 7f91b8ff9640 1 -- 172.21.15.32:0/3656302615 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 80) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f91ac0073d0 con 0x7f91bc0b0760 2023-12-20T05:43:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.745+0000 7f91b8ff9640 1 --2- 172.21.15.32:0/3656302615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f91a8065b80 0x7f91a8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.749+0000 7f91b8ff9640 1 -- 172.21.15.32:0/3656302615 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f91ac082ae0 con 0x7f91bc0b0760 2023-12-20T05:43:24.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.749+0000 7f91bb7fe640 1 -- 172.21.15.32:0/3656302615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f91a8065b80 msgr2=0x7f91a8068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:24.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.749+0000 7f91bb7fe640 1 --2- 172.21.15.32:0/3656302615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f91a8065b80 0x7f91a8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:24.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.749+0000 7f91b8ff9640 1 -- 172.21.15.32:0/3656302615 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f91ac050190 con 0x7f91bc0b0760 2023-12-20T05:43:24.820 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:24.816+0000 7f076ca4d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T05:43:24.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.909+0000 7f91c29a5640 1 -- 172.21.15.32:0/3656302615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f91bc0c67a0 con 0x7f91bc0b0760 2023-12-20T05:43:24.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.909+0000 7f91b8ff9640 1 -- 172.21.15.32:0/3656302615 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v145) v1 ==== 93+0+10 (secure 0 0 0) 0x7f91ac017020 con 0x7f91bc0b0760 2023-12-20T05:43:24.910 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T05:43:24.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 -- 172.21.15.32:0/3656302615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f91a8065b80 msgr2=0x7f91a8068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 --2- 172.21.15.32:0/3656302615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f91a8065b80 0x7f91a8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 -- 172.21.15.32:0/3656302615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc0b0760 msgr2=0x7f91bc1499d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 --2- 172.21.15.32:0/3656302615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc0b0760 0x7f91bc1499d0 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f91ac00c4d0 tx=0x7f91ac00c990 comp rx=0 tx=0).stop 2023-12-20T05:43:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 -- 172.21.15.32:0/3656302615 shutdown_connections 2023-12-20T05:43:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 --2- 172.21.15.32:0/3656302615 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f91a8065b80 0x7f91a8068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 --2- 172.21.15.32:0/3656302615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91bc157480 0x7f91bc0b3e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 --2- 172.21.15.32:0/3656302615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91bc0cc4b0 0x7f91bc0b3940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 --2- 172.21.15.32:0/3656302615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91bc0b0760 0x7f91bc1499d0 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 -- 172.21.15.32:0/3656302615 >> 172.21.15.32:0/3656302615 conn(0x7f91bc06a510 msgr2=0x7f91bc0c4ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 -- 172.21.15.32:0/3656302615 shutdown_connections 2023-12-20T05:43:24.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:24.913+0000 7f91c29a5640 1 -- 172.21.15.32:0/3656302615 wait complete. 2023-12-20T05:43:24.932 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T05:43:24.950 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:24.949+0000 7f9a7a1e8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T05:43:24.976 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:24.972+0000 7f076ca4d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T05:43:25.042 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.041+0000 7f9a7a1e8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T05:43:25.064 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:25.060+0000 7f076ca4d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T05:43:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.081+0000 7fe73d0e5640 1 Processor -- start 2023-12-20T05:43:25.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.081+0000 7fe73d0e5640 1 -- start start 2023-12-20T05:43:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73d0e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe7380d3300 0x7fe7380d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73d0e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7380d1300 0x7fe7380b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73d0e5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe7380d2650 con 0x7fe73815ce60 2023-12-20T05:43:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73d0e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7380d1a40 con 0x7fe7380d1300 2023-12-20T05:43:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73d0e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe7380d1d20 con 0x7fe7380d3300 2023-12-20T05:43:25.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7380d1300 0x7fe7380b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe7380d1300 0x7fe7380b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50100/0 (socket says 172.21.15.32:50100) 2023-12-20T05:43:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73759e640 1 -- 172.21.15.32:0/3730092783 learned_addr learned my addr 172.21.15.32:0/3730092783 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe72ffff640 1 --2- 172.21.15.32:0/3730092783 >> v2:172.21.15.39:3300/0 conn(0x7fe7380d3300 0x7fe7380d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73759e640 1 -- 172.21.15.32:0/3730092783 >> v2:172.21.15.39:3300/0 conn(0x7fe7380d3300 msgr2=0x7fe7380d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73759e640 1 --2- 172.21.15.32:0/3730092783 >> v2:172.21.15.39:3300/0 conn(0x7fe7380d3300 0x7fe7380d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73759e640 1 -- 172.21.15.32:0/3730092783 >> v1:172.21.15.39:6789/0 conn(0x7fe73815ce60 legacy=0x7fe7380d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.085+0000 7fe73759e640 1 -- 172.21.15.32:0/3730092783 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe7380d2000 con 0x7fe7380d1300 2023-12-20T05:43:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73759e640 1 --2- 172.21.15.32:0/3730092783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380d1300 0x7fe7380b5e90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe7380cd240 tx=0x7fe73001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8b396b22a095fb0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73659c640 1 -- 172.21.15.32:0/3730092783 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7300a56c0 con 0x7fe7380d1300 2023-12-20T05:43:25.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73659c640 1 -- 172.21.15.32:0/3730092783 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe7300a5820 con 0x7fe7380d1300 2023-12-20T05:43:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73659c640 1 -- 172.21.15.32:0/3730092783 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe7300a5bb0 con 0x7fe7380d1300 2023-12-20T05:43:25.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73d0e5640 1 -- 172.21.15.32:0/3730092783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380d1300 msgr2=0x7fe7380b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73d0e5640 1 --2- 172.21.15.32:0/3730092783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380d1300 0x7fe7380b5e90 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7fe7380cd240 tx=0x7fe73001c9d0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73d0e5640 1 -- 172.21.15.32:0/3730092783 shutdown_connections 2023-12-20T05:43:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73d0e5640 1 --2- 172.21.15.32:0/3730092783 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe7380d1300 0x7fe7380b5e90 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73d0e5640 1 --2- 172.21.15.32:0/3730092783 >> v2:172.21.15.39:3300/0 conn(0x7fe7380d3300 0x7fe7380d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73d0e5640 1 -- 172.21.15.32:0/3730092783 >> 172.21.15.32:0/3730092783 conn(0x7fe73806b5e0 msgr2=0x7fe73806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73d0e5640 1 -- 172.21.15.32:0/3730092783 shutdown_connections 2023-12-20T05:43:25.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.089+0000 7fe73d0e5640 1 -- 172.21.15.32:0/3730092783 wait complete. 2023-12-20T05:43:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73d0e5640 1 Processor -- start 2023-12-20T05:43:25.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73d0e5640 1 -- start start 2023-12-20T05:43:25.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73d0e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7380d1300 0x7fe73815c120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73d0e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7380d3300 0x7fe738155df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73d0e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73815ce60 0x7fe738158b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73d0e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe738167610 con 0x7fe73815ce60 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73d0e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe7380c9340 con 0x7fe7380d3300 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73d0e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe7380c8730 con 0x7fe7380d1300 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73815ce60 0x7fe738158b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe72ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7380d3300 0x7fe738155df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73815ce60 0x7fe738158b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50114/0 (socket says 172.21.15.32:50114) 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73759e640 1 -- 172.21.15.32:0/152987993 learned_addr learned my addr 172.21.15.32:0/152987993 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe736d9d640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7380d1300 0x7fe73815c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe72ffff640 1 -- 172.21.15.32:0/152987993 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7380d1300 msgr2=0x7fe73815c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe72ffff640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7380d1300 0x7fe73815c120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe72ffff640 1 -- 172.21.15.32:0/152987993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73815ce60 msgr2=0x7fe738158b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe72ffff640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73815ce60 0x7fe738158b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe72ffff640 1 -- 172.21.15.32:0/152987993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe730009b30 con 0x7fe7380d3300 2023-12-20T05:43:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe72ffff640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7380d3300 0x7fe738155df0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe72000e840 tx=0x7fe72000ed00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe72f7fe640 1 -- 172.21.15.32:0/152987993 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72000c940 con 0x7fe7380d3300 2023-12-20T05:43:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe72f7fe640 1 -- 172.21.15.32:0/152987993 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe72000caa0 con 0x7fe7380d3300 2023-12-20T05:43:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe73d0e5640 1 -- 172.21.15.32:0/152987993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe7380c8a10 con 0x7fe7380d3300 2023-12-20T05:43:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.093+0000 7fe72f7fe640 1 -- 172.21.15.32:0/152987993 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe72000ce10 con 0x7fe7380d3300 2023-12-20T05:43:25.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.097+0000 7fe73d0e5640 1 -- 172.21.15.32:0/152987993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe7380c8fd0 con 0x7fe7380d3300 2023-12-20T05:43:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.097+0000 7fe73d0e5640 1 -- 172.21.15.32:0/152987993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe7380c94f0 con 0x7fe7380d3300 2023-12-20T05:43:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.097+0000 7fe72f7fe640 1 -- 172.21.15.32:0/152987993 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 80) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fe720013470 con 0x7fe7380d3300 2023-12-20T05:43:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.097+0000 7fe72f7fe640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe718065bb0 0x7fe718068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.097+0000 7fe736d9d640 1 -- 172.21.15.32:0/152987993 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe718065bb0 msgr2=0x7fe718068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.097+0000 7fe72f7fe640 1 -- 172.21.15.32:0/152987993 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6227+0+0 (secure 0 0 0) 0x7fe720083620 con 0x7fe7380d3300 2023-12-20T05:43:25.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.097+0000 7fe736d9d640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe718065bb0 0x7fe718068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:25.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.101+0000 7fe72f7fe640 1 -- 172.21.15.32:0/152987993 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe7380c94f0 con 0x7fe7380d3300 2023-12-20T05:43:25.208 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.209+0000 7f9a7a1e8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T05:43:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.261+0000 7fe73d0e5640 1 -- 172.21.15.32:0/152987993 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe7380c94f0 con 0x7fe7380d3300 2023-12-20T05:43:25.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.261+0000 7fe72f7fe640 1 -- 172.21.15.32:0/152987993 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v145) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe7380c94f0 con 0x7fe7380d3300 2023-12-20T05:43:25.264 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T05:43:25.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 -- 172.21.15.32:0/152987993 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe718065bb0 msgr2=0x7fe718068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:25.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe718065bb0 0x7fe718068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 -- 172.21.15.32:0/152987993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7380d3300 msgr2=0x7fe738155df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7380d3300 0x7fe738155df0 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7fe72000e840 tx=0x7fe72000ed00 comp rx=0 tx=0).stop 2023-12-20T05:43:25.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 -- 172.21.15.32:0/152987993 shutdown_connections 2023-12-20T05:43:25.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe718065bb0 0x7fe718068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe73815ce60 0x7fe738158b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe7380d3300 0x7fe738155df0 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 --2- 172.21.15.32:0/152987993 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe7380d1300 0x7fe73815c120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 -- 172.21.15.32:0/152987993 >> 172.21.15.32:0/152987993 conn(0x7fe73806b5e0 msgr2=0x7fe7380d2bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:25.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 -- 172.21.15.32:0/152987993 shutdown_connections 2023-12-20T05:43:25.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.265+0000 7fe73d0e5640 1 -- 172.21.15.32:0/152987993 wait complete. 2023-12-20T05:43:25.285 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T05:43:25.299 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.297+0000 7f9a7a1e8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T05:43:25.401 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:25.396+0000 7f076ca4d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T05:43:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f751496d640 1 Processor -- start 2023-12-20T05:43:25.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f751496d640 1 -- start start 2023-12-20T05:43:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f751496d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f751015cef0 0x7f7510161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f751496d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75100cde70 0x7f75100ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f751496d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75100cdbd0 con 0x7f7510071760 2023-12-20T05:43:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f751496d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75100cd7b0 con 0x7f75100cde70 2023-12-20T05:43:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f751496d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7510071be0 con 0x7f751015cef0 2023-12-20T05:43:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f750ed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75100cde70 0x7f75100ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f750ed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f75100cde70 0x7f75100ce280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50116/0 (socket says 172.21.15.32:50116) 2023-12-20T05:43:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f750ed9d640 1 -- 172.21.15.32:0/2460262439 learned_addr learned my addr 172.21.15.32:0/2460262439 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f750e59c640 1 --2- 172.21.15.32:0/2460262439 >> v2:172.21.15.39:3300/0 conn(0x7f751015cef0 0x7f7510161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f750ed9d640 1 -- 172.21.15.32:0/2460262439 >> v2:172.21.15.39:3300/0 conn(0x7f751015cef0 msgr2=0x7f7510161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f750ed9d640 1 --2- 172.21.15.32:0/2460262439 >> v2:172.21.15.39:3300/0 conn(0x7f751015cef0 0x7f7510161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f750ed9d640 1 -- 172.21.15.32:0/2460262439 >> v1:172.21.15.39:6789/0 conn(0x7f7510071760 legacy=0x7f75100ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.449+0000 7f750ed9d640 1 -- 172.21.15.32:0/2460262439 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75100cd460 con 0x7f75100cde70 2023-12-20T05:43:25.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f750ed9d640 1 --2- 172.21.15.32:0/2460262439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100cde70 0x7f75100ce280 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f75100cf8b0 tx=0x7f750801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1fa424442439d2d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:25.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f750d59a640 1 -- 172.21.15.32:0/2460262439 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75080a56f0 con 0x7f75100cde70 2023-12-20T05:43:25.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f750d59a640 1 -- 172.21.15.32:0/2460262439 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75080a5850 con 0x7f75100cde70 2023-12-20T05:43:25.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f750d59a640 1 -- 172.21.15.32:0/2460262439 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75080a5be0 con 0x7f75100cde70 2023-12-20T05:43:25.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f751496d640 1 -- 172.21.15.32:0/2460262439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100cde70 msgr2=0x7f75100ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f751496d640 1 --2- 172.21.15.32:0/2460262439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100cde70 0x7f75100ce280 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f75100cf8b0 tx=0x7f750801c940 comp rx=0 tx=0).stop 2023-12-20T05:43:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f751496d640 1 -- 172.21.15.32:0/2460262439 shutdown_connections 2023-12-20T05:43:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f751496d640 1 --2- 172.21.15.32:0/2460262439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100cde70 0x7f75100ce280 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f751496d640 1 --2- 172.21.15.32:0/2460262439 >> v2:172.21.15.39:3300/0 conn(0x7f751015cef0 0x7f7510161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f751496d640 1 -- 172.21.15.32:0/2460262439 >> 172.21.15.32:0/2460262439 conn(0x7f751006b230 msgr2=0x7f751006b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f751496d640 1 -- 172.21.15.32:0/2460262439 shutdown_connections 2023-12-20T05:43:25.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.453+0000 7f751496d640 1 -- 172.21.15.32:0/2460262439 wait complete. 2023-12-20T05:43:25.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f751496d640 1 Processor -- start 2023-12-20T05:43:25.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f751496d640 1 -- start start 2023-12-20T05:43:25.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f751496d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510071760 0x7f7510154a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f751496d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100cde70 0x7f75100ad2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f751496d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f751015cef0 0x7f75100c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f751496d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7510154f80 con 0x7f75100cde70 2023-12-20T05:43:25.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f751496d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75100b4df0 con 0x7f7510071760 2023-12-20T05:43:25.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f751496d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75100b24b0 con 0x7f751015cef0 2023-12-20T05:43:25.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f750dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100cde70 0x7f75100ad2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f750ed9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f751015cef0 0x7f75100c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f750dd9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100cde70 0x7f75100ad2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50130/0 (socket says 172.21.15.32:50130) 2023-12-20T05:43:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f750dd9b640 1 -- 172.21.15.32:0/3029411586 learned_addr learned my addr 172.21.15.32:0/3029411586 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f750e59c640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510071760 0x7f7510154a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f750dd9b640 1 -- 172.21.15.32:0/3029411586 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f751015cef0 msgr2=0x7f75100c6aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f750dd9b640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f751015cef0 0x7f75100c6aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f750dd9b640 1 -- 172.21.15.32:0/3029411586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510071760 msgr2=0x7f7510154a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f750dd9b640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510071760 0x7f7510154a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f750dd9b640 1 -- 172.21.15.32:0/3029411586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7508009b30 con 0x7f75100cde70 2023-12-20T05:43:25.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f750dd9b640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100cde70 0x7f75100ad2b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f750400c3c0 tx=0x7f750400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f74feffd640 1 -- 172.21.15.32:0/3029411586 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75040052d0 con 0x7f75100cde70 2023-12-20T05:43:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f74feffd640 1 -- 172.21.15.32:0/3029411586 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7504005430 con 0x7f75100cde70 2023-12-20T05:43:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f751496d640 1 -- 172.21.15.32:0/3029411586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f75100b2760 con 0x7f75100cde70 2023-12-20T05:43:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f74feffd640 1 -- 172.21.15.32:0/3029411586 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7504003840 con 0x7f75100cde70 2023-12-20T05:43:25.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.457+0000 7f751496d640 1 -- 172.21.15.32:0/3029411586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75100b2fa0 con 0x7f75100cde70 2023-12-20T05:43:25.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.461+0000 7f751496d640 1 -- 172.21.15.32:0/3029411586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f751004e000 con 0x7f75100cde70 2023-12-20T05:43:25.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.461+0000 7f74feffd640 1 -- 172.21.15.32:0/3029411586 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 80) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f75040039a0 con 0x7f75100cde70 2023-12-20T05:43:25.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.461+0000 7f74feffd640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74f0065b80 0x7f74f0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.461+0000 7f74feffd640 1 -- 172.21.15.32:0/3029411586 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(145..145 src has 1..145) v4 ==== 6227+0+0 (secure 0 0 0) 0x7f7504082ad0 con 0x7f75100cde70 2023-12-20T05:43:25.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.461+0000 7f750e59c640 1 -- 172.21.15.32:0/3029411586 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74f0065b80 msgr2=0x7f74f0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:25.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.461+0000 7f750e59c640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74f0065b80 0x7f74f0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:25.465 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:25.460+0000 7f076ca4d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T05:43:25.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.469+0000 7f74feffd640 1 -- 172.21.15.32:0/3029411586 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7504050100 con 0x7f75100cde70 2023-12-20T05:43:25.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.629+0000 7f751496d640 1 -- 172.21.15.32:0/3029411586 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f7510158970 con 0x7f75100cde70 2023-12-20T05:43:25.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.629+0000 7f74feffd640 1 -- 172.21.15.32:0/3029411586 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v145) v1 ==== 100+0+11 (secure 0 0 0) 0x7f7504017020 con 0x7f75100cde70 2023-12-20T05:43:25.632 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T05:43:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 -- 172.21.15.32:0/3029411586 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74f0065b80 msgr2=0x7f74f0068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74f0065b80 0x7f74f0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 -- 172.21.15.32:0/3029411586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100cde70 msgr2=0x7f75100ad2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100cde70 0x7f75100ad2b0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f750400c3c0 tx=0x7f750400c880 comp rx=0 tx=0).stop 2023-12-20T05:43:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 -- 172.21.15.32:0/3029411586 shutdown_connections 2023-12-20T05:43:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74f0065b80 0x7f74f0068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f751015cef0 0x7f75100c6aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f75100cde70 0x7f75100ad2b0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 --2- 172.21.15.32:0/3029411586 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7510071760 0x7f7510154a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 -- 172.21.15.32:0/3029411586 >> 172.21.15.32:0/3029411586 conn(0x7f751006b230 msgr2=0x7f75100b0210 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 -- 172.21.15.32:0/3029411586 shutdown_connections 2023-12-20T05:43:25.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.633+0000 7f751496d640 1 -- 172.21.15.32:0/3029411586 wait complete. 2023-12-20T05:43:25.650 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.649+0000 7f9a7a1e8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T05:43:25.652 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:43:25.716 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.717+0000 7f9a7a1e8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T05:43:25.798 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.797+0000 7f9a01157640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:25.798 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.797+0000 7f9a01157640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:25.798 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.797+0000 7f9a01157640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:25.798 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.797+0000 7f9a01157640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:25.799 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.797+0000 7f9a01157640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:25.799 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.797+0000 7f9a0415d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:25.799 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.797+0000 7f9a0415d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:25.799 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.797+0000 7f9a0415d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:25.799 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.797+0000 7f9a0415d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:25.799 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:25.797+0000 7f9a0415d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:25.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.817+0000 7fdec3ac4640 1 Processor -- start 2023-12-20T05:43:25.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.817+0000 7fdec3ac4640 1 -- start start 2023-12-20T05:43:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec3ac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdebc15cdb0 0x7fdebc0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec3ac4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdebc0d2740 0x7fdebc0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec3ac4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdebc0b6ca0 con 0x7fdebc0d1120 2023-12-20T05:43:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec3ac4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdebc15d190 con 0x7fdebc0d2740 2023-12-20T05:43:25.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec3ac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc0d16a0 con 0x7fdebc15cdb0 2023-12-20T05:43:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec105f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdebc0d2740 0x7fdebc0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec2061640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdebc0d1120 0x7fdebc0b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51392/0 (socket says 172.21.15.32:51392) 2023-12-20T05:43:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec2061640 1 -- 172.21.15.32:0/376529151 learned_addr learned my addr 172.21.15.32:0/376529151 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec1860640 1 --2- 172.21.15.32:0/376529151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdebc15cdb0 0x7fdebc0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec105f640 1 -- 172.21.15.32:0/376529151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdebc15cdb0 msgr2=0x7fdebc0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec105f640 1 --2- 172.21.15.32:0/376529151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdebc15cdb0 0x7fdebc0d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec105f640 1 -- 172.21.15.32:0/376529151 >> v1:172.21.15.32:6789/0 conn(0x7fdebc0d1120 legacy=0x7fdebc0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec105f640 1 -- 172.21.15.32:0/376529151 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdebc0d1980 con 0x7fdebc0d2740 2023-12-20T05:43:25.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec085e640 1 -- 172.21.15.32:0/376529151 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3617592448 0 0) 0x7fdeb40014a0 con 0x7fdebc0d1120 2023-12-20T05:43:25.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.821+0000 7fdec105f640 1 --2- 172.21.15.32:0/376529151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdebc0d2740 0x7fdebc0cf570 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0097c0 tx=0x7fdeac01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2aeb3cccc8548db server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:25.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec085e640 1 -- 172.21.15.32:0/376529151 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0a8970 con 0x7fdebc0d2740 2023-12-20T05:43:25.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec085e640 1 -- 172.21.15.32:0/376529151 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeac0a8ad0 con 0x7fdebc0d2740 2023-12-20T05:43:25.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec085e640 1 -- 172.21.15.32:0/376529151 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeac0a8e60 con 0x7fdebc0d2740 2023-12-20T05:43:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 -- 172.21.15.32:0/376529151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdebc0d2740 msgr2=0x7fdebc0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 --2- 172.21.15.32:0/376529151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdebc0d2740 0x7fdebc0cf570 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7fdeac0097c0 tx=0x7fdeac01ec70 comp rx=0 tx=0).stop 2023-12-20T05:43:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 -- 172.21.15.32:0/376529151 shutdown_connections 2023-12-20T05:43:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 --2- 172.21.15.32:0/376529151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdebc0d2740 0x7fdebc0cf570 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 --2- 172.21.15.32:0/376529151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdebc15cdb0 0x7fdebc0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 -- 172.21.15.32:0/376529151 >> 172.21.15.32:0/376529151 conn(0x7fdebc065ee0 msgr2=0x7fdebc06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:25.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 -- 172.21.15.32:0/376529151 shutdown_connections 2023-12-20T05:43:25.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 -- 172.21.15.32:0/376529151 wait complete. 2023-12-20T05:43:25.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 Processor -- start 2023-12-20T05:43:25.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 -- start start 2023-12-20T05:43:25.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdebc0d1120 0x7fdebc1509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdebc0d2740 0x7fdebc15d2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdebc15cdb0 0x7fdebc15f980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15dbd0 con 0x7fdebc0d1120 2023-12-20T05:43:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15deb0 con 0x7fdebc15cdb0 2023-12-20T05:43:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.825+0000 7fdec3ac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdebc15e190 con 0x7fdebc0d2740 2023-12-20T05:43:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec1860640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdebc0d1120 0x7fdebc1509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec1860640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdebc0d1120 0x7fdebc1509d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50252/0 (socket says 172.21.15.32:50252) 2023-12-20T05:43:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec1860640 1 -- 172.21.15.32:0/4122273807 learned_addr learned my addr 172.21.15.32:0/4122273807 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec105f640 1 --2- 172.21.15.32:0/4122273807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdebc0d2740 0x7fdebc15d2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec2061640 1 --2- 172.21.15.32:0/4122273807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdebc15cdb0 0x7fdebc15f980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec1860640 1 -- 172.21.15.32:0/4122273807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdebc0d2740 msgr2=0x7fdebc15d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec1860640 1 --2- 172.21.15.32:0/4122273807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdebc0d2740 0x7fdebc15d2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec1860640 1 -- 172.21.15.32:0/4122273807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdebc15cdb0 msgr2=0x7fdebc15f980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec1860640 1 --2- 172.21.15.32:0/4122273807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdebc15cdb0 0x7fdebc15f980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec1860640 1 -- 172.21.15.32:0/4122273807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdeac009420 con 0x7fdebc0d1120 2023-12-20T05:43:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec1860640 1 --2- 172.21.15.32:0/4122273807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdebc0d1120 0x7fdebc1509d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fdeb800e930 tx=0x7fdeb800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdeb27fc640 1 -- 172.21.15.32:0/4122273807 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb8009690 con 0x7fdebc0d1120 2023-12-20T05:43:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdeb27fc640 1 -- 172.21.15.32:0/4122273807 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdeb800ca00 con 0x7fdebc0d1120 2023-12-20T05:43:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdeb27fc640 1 -- 172.21.15.32:0/4122273807 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdeb800cd70 con 0x7fdebc0d1120 2023-12-20T05:43:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec3ac4640 1 -- 172.21.15.32:0/4122273807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdebc15e470 con 0x7fdebc0d1120 2023-12-20T05:43:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec3ac4640 1 -- 172.21.15.32:0/4122273807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdebc150f10 con 0x7fdebc0d1120 2023-12-20T05:43:25.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdec3ac4640 1 -- 172.21.15.32:0/4122273807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdebc04e000 con 0x7fdebc0d1120 2023-12-20T05:43:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.829+0000 7fdeb27fc640 1 -- 172.21.15.32:0/4122273807 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 81) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fdeb8013470 con 0x7fdebc0d1120 2023-12-20T05:43:25.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.833+0000 7fdeb27fc640 1 -- 172.21.15.32:0/4122273807 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fdeb80830d0 con 0x7fdebc0d1120 2023-12-20T05:43:25.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:25.837+0000 7fdeb27fc640 1 -- 172.21.15.32:0/4122273807 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdeb8050d70 con 0x7fdebc0d1120 2023-12-20T05:43:26.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.029+0000 7fdec3ac4640 1 -- 172.21.15.32:0/4122273807 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdebc06b8d0 con 0x7fdebc0d1120 2023-12-20T05:43:26.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.029+0000 7fdeb27fc640 1 -- 172.21.15.32:0/4122273807 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4708 (secure 0 0 0) 0x7fdeb8017090 con 0x7fdebc0d1120 2023-12-20T05:43:26.030 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:26.031 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3306270605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3306270605},{"type":"v1","addr":"172.21.15.39:6834","nonce":3306270605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4000562557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4000562557},{"type":"v1","addr":"172.21.15.39:6835","nonce":4000562557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/310576469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":310576469},{"type":"v1","addr":"172.21.15.32:6835","nonce":310576469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:47:21.050837+0000","modified":"2023-12-20T04:47:24.070982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8397},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8397":{"gid":8397,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3314020360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3314020360},{"type":"v1","addr":"172.21.15.32:6837","nonce":3314020360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-20T05:43:26.032 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2023-12-20T05:43:26.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.029+0000 7fdec3ac4640 1 -- 172.21.15.32:0/4122273807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdebc0d1120 msgr2=0x7fdebc1509d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.029+0000 7fdec3ac4640 1 --2- 172.21.15.32:0/4122273807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdebc0d1120 0x7fdebc1509d0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7fdeb800e930 tx=0x7fdeb800edf0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.029+0000 7fdec3ac4640 1 -- 172.21.15.32:0/4122273807 shutdown_connections 2023-12-20T05:43:26.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.029+0000 7fdec3ac4640 1 --2- 172.21.15.32:0/4122273807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdebc15cdb0 0x7fdebc15f980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.029+0000 7fdec3ac4640 1 --2- 172.21.15.32:0/4122273807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdebc0d2740 0x7fdebc15d2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.029+0000 7fdec3ac4640 1 --2- 172.21.15.32:0/4122273807 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdebc0d1120 0x7fdebc1509d0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.029+0000 7fdec3ac4640 1 -- 172.21.15.32:0/4122273807 >> 172.21.15.32:0/4122273807 conn(0x7fdebc065ee0 msgr2=0x7fdebc155060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:26.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.033+0000 7fdec3ac4640 1 -- 172.21.15.32:0/4122273807 shutdown_connections 2023-12-20T05:43:26.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.033+0000 7fdec3ac4640 1 -- 172.21.15.32:0/4122273807 wait complete. 2023-12-20T05:43:26.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:43:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.193+0000 7fcf06bce640 1 Processor -- start 2023-12-20T05:43:26.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.193+0000 7fcf06bce640 1 -- start start 2023-12-20T05:43:26.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcf06bce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcf0015ce60 0x7fcf000c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcf06bce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 0x7fcf00071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcf06bce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf00071d80 con 0x7fcf00160b30 2023-12-20T05:43:26.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcf06bce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf001616d0 con 0x7fcf0015ce60 2023-12-20T05:43:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcf06bce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000c9a40 con 0x7fcf000713e0 2023-12-20T05:43:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcf0496a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcf0015ce60 0x7fcf000c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcefffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 0x7fcf00071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcefffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 0x7fcf00071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:60872/0 (socket says 172.21.15.32:60872) 2023-12-20T05:43:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcefffff640 1 -- 172.21.15.32:0/361515000 learned_addr learned my addr 172.21.15.32:0/361515000 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcefffff640 1 -- 172.21.15.32:0/361515000 >> v2:172.21.15.39:3300/0 conn(0x7fcf0015ce60 msgr2=0x7fcf000c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcefffff640 1 --2- 172.21.15.32:0/361515000 >> v2:172.21.15.39:3300/0 conn(0x7fcf0015ce60 0x7fcf000c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcefffff640 1 -- 172.21.15.32:0/361515000 >> v1:172.21.15.39:6789/0 conn(0x7fcf00160b30 legacy=0x7fcf00160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.197+0000 7fcefffff640 1 -- 172.21.15.32:0/361515000 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf000c9180 con 0x7fcf000713e0 2023-12-20T05:43:26.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fcefffff640 1 --2- 172.21.15.32:0/361515000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 0x7fcf00071840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fceec009870 tx=0x7fceec01c540 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e4eba198bfe80da server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:26.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fceff7fe640 1 -- 172.21.15.32:0/361515000 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0a7070 con 0x7fcf000713e0 2023-12-20T05:43:26.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fceff7fe640 1 -- 172.21.15.32:0/361515000 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec0ab430 con 0x7fcf000713e0 2023-12-20T05:43:26.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fceff7fe640 1 -- 172.21.15.32:0/361515000 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0ab7c0 con 0x7fcf000713e0 2023-12-20T05:43:26.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fcf06bce640 1 -- 172.21.15.32:0/361515000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 msgr2=0x7fcf00071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fcf06bce640 1 --2- 172.21.15.32:0/361515000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 0x7fcf00071840 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fceec009870 tx=0x7fceec01c540 comp rx=0 tx=0).stop 2023-12-20T05:43:26.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fcf06bce640 1 -- 172.21.15.32:0/361515000 shutdown_connections 2023-12-20T05:43:26.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fcf06bce640 1 --2- 172.21.15.32:0/361515000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 0x7fcf00071840 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fcf06bce640 1 --2- 172.21.15.32:0/361515000 >> v2:172.21.15.39:3300/0 conn(0x7fcf0015ce60 0x7fcf000c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fcf06bce640 1 -- 172.21.15.32:0/361515000 >> 172.21.15.32:0/361515000 conn(0x7fcf0006a320 msgr2=0x7fcf000ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:26.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fcf06bce640 1 -- 172.21.15.32:0/361515000 shutdown_connections 2023-12-20T05:43:26.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fcf06bce640 1 -- 172.21.15.32:0/361515000 wait complete. 2023-12-20T05:43:26.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fcf06bce640 1 Processor -- start 2023-12-20T05:43:26.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.201+0000 7fcf06bce640 1 -- start start 2023-12-20T05:43:26.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf06bce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 0x7fcf00156040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf06bce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0015ce60 0x7fcf000c81b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf06bce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00160b30 0x7fcf0015d3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf06bce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000c3540 con 0x7fcf00160b30 2023-12-20T05:43:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf06bce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000c3820 con 0x7fcf0015ce60 2023-12-20T05:43:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf06bce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf000c3b00 con 0x7fcf000713e0 2023-12-20T05:43:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf0516b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00160b30 0x7fcf0015d3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf0516b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00160b30 0x7fcf0015d3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50258/0 (socket says 172.21.15.32:50258) 2023-12-20T05:43:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf0516b640 1 -- 172.21.15.32:0/3215393375 learned_addr learned my addr 172.21.15.32:0/3215393375 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf0496a640 1 --2- 172.21.15.32:0/3215393375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 0x7fcf00156040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcefffff640 1 --2- 172.21.15.32:0/3215393375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0015ce60 0x7fcf000c81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcefffff640 1 -- 172.21.15.32:0/3215393375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 msgr2=0x7fcf00156040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcefffff640 1 --2- 172.21.15.32:0/3215393375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 0x7fcf00156040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcefffff640 1 -- 172.21.15.32:0/3215393375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00160b30 msgr2=0x7fcf0015d3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcefffff640 1 --2- 172.21.15.32:0/3215393375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00160b30 0x7fcf0015d3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcefffff640 1 -- 172.21.15.32:0/3215393375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fceec0094d0 con 0x7fcf0015ce60 2023-12-20T05:43:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcefffff640 1 --2- 172.21.15.32:0/3215393375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0015ce60 0x7fcf000c81b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fceec0ac000 tx=0x7fceec0192e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcefd7fa640 1 -- 172.21.15.32:0/3215393375 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0b4630 con 0x7fcf0015ce60 2023-12-20T05:43:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcefd7fa640 1 -- 172.21.15.32:0/3215393375 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fceec0b4790 con 0x7fcf0015ce60 2023-12-20T05:43:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcefd7fa640 1 -- 172.21.15.32:0/3215393375 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fceec0b4b40 con 0x7fcf0015ce60 2023-12-20T05:43:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf06bce640 1 -- 172.21.15.32:0/3215393375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf000c3de0 con 0x7fcf0015ce60 2023-12-20T05:43:26.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf06bce640 1 -- 172.21.15.32:0/3215393375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf00156580 con 0x7fcf0015ce60 2023-12-20T05:43:26.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcf06bce640 1 -- 172.21.15.32:0/3215393375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf0004e000 con 0x7fcf0015ce60 2023-12-20T05:43:26.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.205+0000 7fcefd7fa640 1 -- 172.21.15.32:0/3215393375 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 81) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fceec0bf3f0 con 0x7fcf0015ce60 2023-12-20T05:43:26.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.209+0000 7fcefd7fa640 1 -- 172.21.15.32:0/3215393375 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fceec11c530 con 0x7fcf0015ce60 2023-12-20T05:43:26.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.213+0000 7fcefd7fa640 1 -- 172.21.15.32:0/3215393375 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fceec0ea020 con 0x7fcf0015ce60 2023-12-20T05:43:26.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.369+0000 7fcf06bce640 1 -- 172.21.15.32:0/3215393375 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf000c3960 con 0x7fcf0015ce60 2023-12-20T05:43:26.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.373+0000 7fcefd7fa640 1 -- 172.21.15.32:0/3215393375 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+17109 (secure 0 0 0) 0x7fcf000c3960 con 0x7fcf0015ce60 2023-12-20T05:43:26.372 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:26.372 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":146,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:43:25.722250+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:47:19.596882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-20T04:47:20.055023+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":145,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":2}]},{"pgid":"17.17","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/568130455":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/2815557802":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/1186578222":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/801772685":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/2494305942":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/2892107659":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/2511943559":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/3621453434":"2023-12-21T04:47:33.113748+0000","172.21.15.32:6833/38009":"2023-12-21T04:47:33.113748+0000","172.21.15.32:6832/38009":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/1810110767":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/2135559470":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/3483848649":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/1663137028":"2023-12-21T04:47:33.113748+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:43:26.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.373+0000 7fcf06bce640 1 -- 172.21.15.32:0/3215393375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0015ce60 msgr2=0x7fcf000c81b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.373+0000 7fcf06bce640 1 --2- 172.21.15.32:0/3215393375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0015ce60 0x7fcf000c81b0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7fceec0ac000 tx=0x7fceec0192e0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.373+0000 7fcf06bce640 1 -- 172.21.15.32:0/3215393375 shutdown_connections 2023-12-20T05:43:26.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.373+0000 7fcf06bce640 1 --2- 172.21.15.32:0/3215393375 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf00160b30 0x7fcf0015d3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.373+0000 7fcf06bce640 1 --2- 172.21.15.32:0/3215393375 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf0015ce60 0x7fcf000c81b0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.373+0000 7fcf06bce640 1 --2- 172.21.15.32:0/3215393375 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf000713e0 0x7fcf00156040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.373+0000 7fcf06bce640 1 -- 172.21.15.32:0/3215393375 >> 172.21.15.32:0/3215393375 conn(0x7fcf0006a320 msgr2=0x7fcf000cdb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:26.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.373+0000 7fcf06bce640 1 -- 172.21.15.32:0/3215393375 shutdown_connections 2023-12-20T05:43:26.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.373+0000 7fcf06bce640 1 -- 172.21.15.32:0/3215393375 wait complete. 2023-12-20T05:43:26.393 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T05:43:26.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:43:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.537+0000 7fc922811640 1 Processor -- start 2023-12-20T05:43:26.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.537+0000 7fc922811640 1 -- start start 2023-12-20T05:43:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.537+0000 7fc922811640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc91c1599e0 0x7fc91c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.537+0000 7fc922811640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91c0c98c0 con 0x7fc91c0c90e0 2023-12-20T05:43:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.537+0000 7fc922811640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91c0c9ba0 con 0x7fc91c0b0d00 2023-12-20T05:43:26.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.537+0000 7fc922811640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91c0c9e80 con 0x7fc91c1599e0 2023-12-20T05:43:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.537+0000 7fc91bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc91c1599e0 0x7fc91c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.537+0000 7fc91b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc91c0c90e0 0x7fc91c0b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51396/0 (socket says 172.21.15.32:51396) 2023-12-20T05:43:26.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.537+0000 7fc91b7fe640 1 -- 172.21.15.32:0/3628080956 learned_addr learned my addr 172.21.15.32:0/3628080956 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.541+0000 7fc91bfff640 1 -- 172.21.15.32:0/3628080956 >> v1:172.21.15.39:6789/0 conn(0x7fc91c0b0d00 legacy=0x7fc91c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.541+0000 7fc91bfff640 1 -- 172.21.15.32:0/3628080956 >> v1:172.21.15.32:6789/0 conn(0x7fc91c0c90e0 legacy=0x7fc91c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.541+0000 7fc91bfff640 1 -- 172.21.15.32:0/3628080956 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91c0ca160 con 0x7fc91c1599e0 2023-12-20T05:43:26.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.541+0000 7fc91affd640 1 -- 172.21.15.32:0/3628080956 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 317179459 0 0) 0x7fc91c0c98c0 con 0x7fc91c0c90e0 2023-12-20T05:43:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.541+0000 7fc91bfff640 1 --2- 172.21.15.32:0/3628080956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91c1599e0 0x7fc91c0aece0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0097a0 tx=0x7fc90c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=194e86e37fb9c3df server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.541+0000 7fc91affd640 1 -- 172.21.15.32:0/3628080956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c0ab600 con 0x7fc91c1599e0 2023-12-20T05:43:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.541+0000 7fc91affd640 1 -- 172.21.15.32:0/3628080956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c0a7050 con 0x7fc91c1599e0 2023-12-20T05:43:26.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.541+0000 7fc91affd640 1 -- 172.21.15.32:0/3628080956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c0ab980 con 0x7fc91c1599e0 2023-12-20T05:43:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 -- 172.21.15.32:0/3628080956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91c1599e0 msgr2=0x7fc91c0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 --2- 172.21.15.32:0/3628080956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91c1599e0 0x7fc91c0aece0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0097a0 tx=0x7fc90c01c430 comp rx=0 tx=0).stop 2023-12-20T05:43:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 -- 172.21.15.32:0/3628080956 shutdown_connections 2023-12-20T05:43:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 --2- 172.21.15.32:0/3628080956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91c1599e0 0x7fc91c0aece0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 -- 172.21.15.32:0/3628080956 >> 172.21.15.32:0/3628080956 conn(0x7fc91c06a320 msgr2=0x7fc91c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 -- 172.21.15.32:0/3628080956 shutdown_connections 2023-12-20T05:43:26.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 -- 172.21.15.32:0/3628080956 wait complete. 2023-12-20T05:43:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 Processor -- start 2023-12-20T05:43:26.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 -- start start 2023-12-20T05:43:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91c0b0d00 0x7fc91c0ce060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c0c90e0 0x7fc91c14a390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc91c1599e0 0x7fc91c15b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c1401e0 con 0x7fc91c0b0d00 2023-12-20T05:43:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c14bc60 con 0x7fc91c0c90e0 2023-12-20T05:43:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc922811640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91c1495d0 con 0x7fc91c1599e0 2023-12-20T05:43:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc91bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91c0b0d00 0x7fc91c0ce060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc91bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91c0b0d00 0x7fc91c0ce060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50266/0 (socket says 172.21.15.32:50266) 2023-12-20T05:43:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc91bfff640 1 -- 172.21.15.32:0/3046864661 learned_addr learned my addr 172.21.15.32:0/3046864661 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc91b7fe640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c0c90e0 0x7fc91c14a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc920dae640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc91c1599e0 0x7fc91c15b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc91bfff640 1 -- 172.21.15.32:0/3046864661 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc91c1599e0 msgr2=0x7fc91c15b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc91bfff640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc91c1599e0 0x7fc91c15b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc91bfff640 1 -- 172.21.15.32:0/3046864661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c0c90e0 msgr2=0x7fc91c14a390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc91bfff640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c0c90e0 0x7fc91c14a390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.545+0000 7fc91bfff640 1 -- 172.21.15.32:0/3046864661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc90c0093b0 con 0x7fc91c0b0d00 2023-12-20T05:43:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.549+0000 7fc91bfff640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91c0b0d00 0x7fc91c0ce060 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc90c019040 tx=0x7fc90c0191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.549+0000 7fc918ff9640 1 -- 172.21.15.32:0/3046864661 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c005ba0 con 0x7fc91c0b0d00 2023-12-20T05:43:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.549+0000 7fc918ff9640 1 -- 172.21.15.32:0/3046864661 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c005d00 con 0x7fc91c0b0d00 2023-12-20T05:43:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.549+0000 7fc918ff9640 1 -- 172.21.15.32:0/3046864661 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c006260 con 0x7fc91c0b0d00 2023-12-20T05:43:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.549+0000 7fc922811640 1 -- 172.21.15.32:0/3046864661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc91c1498b0 con 0x7fc91c0b0d00 2023-12-20T05:43:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.549+0000 7fc922811640 1 -- 172.21.15.32:0/3046864661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91c149e70 con 0x7fc91c0b0d00 2023-12-20T05:43:26.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.549+0000 7fc922811640 1 -- 172.21.15.32:0/3046864661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91c04e000 con 0x7fc91c0b0d00 2023-12-20T05:43:26.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.549+0000 7fc918ff9640 1 -- 172.21.15.32:0/3046864661 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 81) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fc90c0b2040 con 0x7fc91c0b0d00 2023-12-20T05:43:26.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.549+0000 7fc918ff9640 1 -- 172.21.15.32:0/3046864661 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fc90c11c3f0 con 0x7fc91c0b0d00 2023-12-20T05:43:26.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.557+0000 7fc918ff9640 1 -- 172.21.15.32:0/3046864661 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc90c0e9ee0 con 0x7fc91c0b0d00 2023-12-20T05:43:26.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.733+0000 7fc918ff9640 1 -- 172.21.15.32:0/3046864661 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 82) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fc90c0e9990 con 0x7fc91c0b0d00 2023-12-20T05:43:26.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.733+0000 7fc918ff9640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc90806b290 0x7fc90806d680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.733+0000 7fc91b7fe640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc90806b290 0x7fc90806d680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.733+0000 7fc91b7fe640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc90806b290 0x7fc90806d680 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc91c1fd670 tx=0x7fc910007240 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:26.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.753+0000 7fc922811640 1 -- 172.21.15.32:0/3046864661 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc91c0cf550 con 0x7fc91c0b0d00 2023-12-20T05:43:26.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.753+0000 7fc918ff9640 1 -- 172.21.15.32:0/3046864661 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4708 (secure 0 0 0) 0x7fc90c0ba090 con 0x7fc91c0b0d00 2023-12-20T05:43:26.754 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:26.754 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3306270605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3306270605},{"type":"v1","addr":"172.21.15.39:6834","nonce":3306270605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4000562557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4000562557},{"type":"v1","addr":"172.21.15.39:6835","nonce":4000562557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/310576469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":310576469},{"type":"v1","addr":"172.21.15.32:6835","nonce":310576469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:47:21.050837+0000","modified":"2023-12-20T04:47:24.070982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8397},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8397":{"gid":8397,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3314020360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3314020360},{"type":"v1","addr":"172.21.15.32:6837","nonce":3314020360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-20T05:43:26.755 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2023-12-20T05:43:26.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 -- 172.21.15.32:0/3046864661 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc90806b290 msgr2=0x7fc90806d680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc90806b290 0x7fc90806d680 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7fc91c1fd670 tx=0x7fc910007240 comp rx=0 tx=0).stop 2023-12-20T05:43:26.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 -- 172.21.15.32:0/3046864661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91c0b0d00 msgr2=0x7fc91c0ce060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91c0b0d00 0x7fc91c0ce060 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7fc90c019040 tx=0x7fc90c0191d0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 -- 172.21.15.32:0/3046864661 shutdown_connections 2023-12-20T05:43:26.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc90806b290 0x7fc90806d680 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc91c1599e0 0x7fc91c15b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91c0c90e0 0x7fc91c14a390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 --2- 172.21.15.32:0/3046864661 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc91c0b0d00 0x7fc91c0ce060 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 -- 172.21.15.32:0/3046864661 >> 172.21.15.32:0/3046864661 conn(0x7fc91c06a320 msgr2=0x7fc91c0c4ac0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:26.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 -- 172.21.15.32:0/3046864661 shutdown_connections 2023-12-20T05:43:26.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.757+0000 7fc922811640 1 -- 172.21.15.32:0/3046864661 wait complete. 2023-12-20T05:43:26.776 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:43:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d92f2640 1 Processor -- start 2023-12-20T05:43:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d92f2640 1 -- start start 2023-12-20T05:43:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d92f2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5d40c8580 0x7fe5d40b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d92f2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d406c180 con 0x7fe5d41597e0 2023-12-20T05:43:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d92f2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d406c460 con 0x7fe5d40cb090 2023-12-20T05:43:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d92f2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d406c740 con 0x7fe5d40c8580 2023-12-20T05:43:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d2ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe5d41597e0 0x7fe5d4159bc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51410/0 (socket says 172.21.15.32:51410) 2023-12-20T05:43:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d2ffd640 1 -- 172.21.15.32:0/3580742888 learned_addr learned my addr 172.21.15.32:0/3580742888 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d27fc640 1 --2- 172.21.15.32:0/3580742888 >> v2:172.21.15.32:3300/0 conn(0x7fe5d40c8580 0x7fe5d40b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d27fc640 1 -- 172.21.15.32:0/3580742888 >> v1:172.21.15.39:6789/0 conn(0x7fe5d40cb090 legacy=0x7fe5d415d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d27fc640 1 -- 172.21.15.32:0/3580742888 >> v1:172.21.15.32:6789/0 conn(0x7fe5d41597e0 legacy=0x7fe5d4159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.921+0000 7fe5d27fc640 1 -- 172.21.15.32:0/3580742888 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d406ca20 con 0x7fe5d40c8580 2023-12-20T05:43:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.925+0000 7fe5d27fc640 1 --2- 172.21.15.32:0/3580742888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40c8580 0x7fe5d40b0800 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80097c0 tx=0x7fe5c801c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b766b85e940c7021 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.925+0000 7fe5d1ffb640 1 -- 172.21.15.32:0/3580742888 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80ab670 con 0x7fe5d40c8580 2023-12-20T05:43:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.925+0000 7fe5d1ffb640 1 -- 172.21.15.32:0/3580742888 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c80ab7d0 con 0x7fe5d40c8580 2023-12-20T05:43:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.925+0000 7fe5d1ffb640 1 -- 172.21.15.32:0/3580742888 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80abb60 con 0x7fe5d40c8580 2023-12-20T05:43:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.925+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3580742888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40c8580 msgr2=0x7fe5d40b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.925+0000 7fe5d92f2640 1 --2- 172.21.15.32:0/3580742888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40c8580 0x7fe5d40b0800 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80097c0 tx=0x7fe5c801c450 comp rx=0 tx=0).stop 2023-12-20T05:43:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.925+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3580742888 shutdown_connections 2023-12-20T05:43:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.925+0000 7fe5d92f2640 1 --2- 172.21.15.32:0/3580742888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40c8580 0x7fe5d40b0800 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.925+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3580742888 >> 172.21.15.32:0/3580742888 conn(0x7fe5d4069d20 msgr2=0x7fe5d406a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.925+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3580742888 shutdown_connections 2023-12-20T05:43:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.925+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3580742888 wait complete. 2023-12-20T05:43:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d92f2640 1 Processor -- start 2023-12-20T05:43:26.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d92f2640 1 -- start start 2023-12-20T05:43:26.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d92f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40c8580 0x7fe5d4157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d92f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d40cb090 0x7fe5d414f9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d92f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d41597e0 0x7fe5d414eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d92f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4150570 con 0x7fe5d40c8580 2023-12-20T05:43:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d92f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4150df0 con 0x7fe5d40cb090 2023-12-20T05:43:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d92f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4152760 con 0x7fe5d41597e0 2023-12-20T05:43:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40c8580 0x7fe5d4157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40c8580 0x7fe5d4157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50284/0 (socket says 172.21.15.32:50284) 2023-12-20T05:43:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d2ffd640 1 -- 172.21.15.32:0/3307062976 learned_addr learned my addr 172.21.15.32:0/3307062976 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d27fc640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d40cb090 0x7fe5d414f9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d37fe640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d41597e0 0x7fe5d414eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d2ffd640 1 -- 172.21.15.32:0/3307062976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d41597e0 msgr2=0x7fe5d414eda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d2ffd640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d41597e0 0x7fe5d414eda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d2ffd640 1 -- 172.21.15.32:0/3307062976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d40cb090 msgr2=0x7fe5d414f9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d2ffd640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d40cb090 0x7fe5d414f9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d2ffd640 1 -- 172.21.15.32:0/3307062976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c8009420 con 0x7fe5d40c8580 2023-12-20T05:43:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d27fc640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d40cb090 0x7fe5d414f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d37fe640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d41597e0 0x7fe5d414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d2ffd640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40c8580 0x7fe5d4157a80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c400 tx=0x7fe5c400c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5bb7fe640 1 -- 172.21.15.32:0/3307062976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c4005cf0 con 0x7fe5d40c8580 2023-12-20T05:43:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5bb7fe640 1 -- 172.21.15.32:0/3307062976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c4005e50 con 0x7fe5d40c8580 2023-12-20T05:43:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3307062976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d40cba30 con 0x7fe5d40c8580 2023-12-20T05:43:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5bb7fe640 1 -- 172.21.15.32:0/3307062976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c40072d0 con 0x7fe5d40c8580 2023-12-20T05:43:26.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3307062976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d4150780 con 0x7fe5d40c8580 2023-12-20T05:43:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.929+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3307062976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d40caf00 con 0x7fe5d40c8580 2023-12-20T05:43:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.933+0000 7fe5bb7fe640 1 -- 172.21.15.32:0/3307062976 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 82) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fe5c4007430 con 0x7fe5d40c8580 2023-12-20T05:43:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.933+0000 7fe5bb7fe640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c00658b0 0x7fe5c0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.933+0000 7fe5bb7fe640 1 -- 172.21.15.32:0/3307062976 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6441+0+0 (secure 0 0 0) 0x7fe5c4082640 con 0x7fe5d40c8580 2023-12-20T05:43:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.933+0000 7fe5d27fc640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c00658b0 0x7fe5c0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:26.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.933+0000 7fe5d27fc640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c00658b0 0x7fe5c0067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80af000 tx=0x7fe5c8008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:26.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:26.937+0000 7fe5bb7fe640 1 -- 172.21.15.32:0/3307062976 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe5c4050040 con 0x7fe5d40c8580 2023-12-20T05:43:27.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.125+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3307062976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe5d4074100 con 0x7fe5d40c8580 2023-12-20T05:43:27.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.125+0000 7fe5bb7fe640 1 -- 172.21.15.32:0/3307062976 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 62 v62) v1 ==== 94+0+4708 (secure 0 0 0) 0x7fe5c4053660 con 0x7fe5d40c8580 2023-12-20T05:43:27.126 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:27.126 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":62,"default_fscid":8,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3306270605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3306270605},{"type":"v1","addr":"172.21.15.39:6834","nonce":3306270605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4000562557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4000562557},{"type":"v1","addr":"172.21.15.39:6835","nonce":4000562557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/310576469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":310576469},{"type":"v1","addr":"172.21.15.32:6835","nonce":310576469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[{"mdsmap":{"epoch":62,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T04:47:21.050837+0000","modified":"2023-12-20T04:47:24.070982+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8397},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8397":{"gid":8397,"name":"c","rank":0,"incarnation":58,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3314020360","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3314020360},{"type":"v1","addr":"172.21.15.32:6837","nonce":3314020360}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[17],"metadata_pool":16,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":8}]} 2023-12-20T05:43:27.127 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 62 2023-12-20T05:43:27.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3307062976 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c00658b0 msgr2=0x7fe5c0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c00658b0 0x7fe5c0067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fe5c80af000 tx=0x7fe5c8008040 comp rx=0 tx=0).stop 2023-12-20T05:43:27.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3307062976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40c8580 msgr2=0x7fe5d4157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40c8580 0x7fe5d4157a80 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7fe5c400c400 tx=0x7fe5c400c8c0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3307062976 shutdown_connections 2023-12-20T05:43:27.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5c00658b0 0x7fe5c0067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d41597e0 0x7fe5d414eda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d40cb090 0x7fe5d414f9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 --2- 172.21.15.32:0/3307062976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40c8580 0x7fe5d4157a80 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3307062976 >> 172.21.15.32:0/3307062976 conn(0x7fe5d4069d20 msgr2=0x7fe5d4159bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:27.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3307062976 shutdown_connections 2023-12-20T05:43:27.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.129+0000 7fe5d92f2640 1 -- 172.21.15.32:0/3307062976 wait complete. 2023-12-20T05:43:27.148 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:43:27.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939f36d640 1 Processor -- start 2023-12-20T05:43:27.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939f36d640 1 -- start start 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939f36d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93980b2ae0 0x7f93980cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939f36d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f93980afd50 0x7f93980b3b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939f36d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93980ced30 con 0x7f939815ae80 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939f36d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93980ceb50 con 0x7f93980b2ae0 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939f36d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93980d0410 con 0x7f93980afd50 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939c908640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93980b2ae0 0x7f93980cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939c908640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93980b2ae0 0x7f93980cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50294/0 (socket says 172.21.15.32:50294) 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939c908640 1 -- 172.21.15.32:0/3578425289 learned_addr learned my addr 172.21.15.32:0/3578425289 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939d90a640 1 --2- 172.21.15.32:0/3578425289 >> v2:172.21.15.39:3300/0 conn(0x7f93980afd50 0x7f93980b3b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939c908640 1 -- 172.21.15.32:0/3578425289 >> v2:172.21.15.39:3300/0 conn(0x7f93980afd50 msgr2=0x7f93980b3b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939c908640 1 --2- 172.21.15.32:0/3578425289 >> v2:172.21.15.39:3300/0 conn(0x7f93980afd50 0x7f93980b3b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939c908640 1 -- 172.21.15.32:0/3578425289 >> v1:172.21.15.32:6789/0 conn(0x7f939815ae80 legacy=0x7f93980b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.293+0000 7f939c908640 1 -- 172.21.15.32:0/3578425289 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93980cf010 con 0x7f93980b2ae0 2023-12-20T05:43:27.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f939c908640 1 --2- 172.21.15.32:0/3578425289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980b2ae0 0x7f93980cfed0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f93980b30b0 tx=0x7f938801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c7d4ddbbb112901 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:27.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f938ffff640 1 -- 172.21.15.32:0/3578425289 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93880a89f0 con 0x7f93980b2ae0 2023-12-20T05:43:27.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f938ffff640 1 -- 172.21.15.32:0/3578425289 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93880a8b50 con 0x7f93980b2ae0 2023-12-20T05:43:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f938ffff640 1 -- 172.21.15.32:0/3578425289 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93880b0440 con 0x7f93980b2ae0 2023-12-20T05:43:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f939f36d640 1 -- 172.21.15.32:0/3578425289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980b2ae0 msgr2=0x7f93980cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f939f36d640 1 --2- 172.21.15.32:0/3578425289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980b2ae0 0x7f93980cfed0 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f93980b30b0 tx=0x7f938801ecf0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f939f36d640 1 -- 172.21.15.32:0/3578425289 shutdown_connections 2023-12-20T05:43:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f939f36d640 1 --2- 172.21.15.32:0/3578425289 >> v2:172.21.15.39:3300/0 conn(0x7f93980afd50 0x7f93980b3b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f939f36d640 1 --2- 172.21.15.32:0/3578425289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980b2ae0 0x7f93980cfed0 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f939f36d640 1 -- 172.21.15.32:0/3578425289 >> 172.21.15.32:0/3578425289 conn(0x7f93980cd3d0 msgr2=0x7f93980cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f939f36d640 1 -- 172.21.15.32:0/3578425289 shutdown_connections 2023-12-20T05:43:27.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.297+0000 7f939f36d640 1 -- 172.21.15.32:0/3578425289 wait complete. 2023-12-20T05:43:27.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939f36d640 1 Processor -- start 2023-12-20T05:43:27.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939f36d640 1 -- start start 2023-12-20T05:43:27.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939f36d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980afd50 0x7f939815dd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939f36d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93980b2ae0 0x7f93980c8430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939f36d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f939815ae80 0x7f93980c8970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939f36d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f939814df70 con 0x7f93980afd50 2023-12-20T05:43:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939f36d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93980ca760 con 0x7f93980b2ae0 2023-12-20T05:43:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939f36d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f939814e390 con 0x7f939815ae80 2023-12-20T05:43:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939d109640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980afd50 0x7f939815dd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939d109640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980afd50 0x7f939815dd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50302/0 (socket says 172.21.15.32:50302) 2023-12-20T05:43:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939d109640 1 -- 172.21.15.32:0/3816084012 learned_addr learned my addr 172.21.15.32:0/3816084012 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939c908640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93980b2ae0 0x7f93980c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939d90a640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f939815ae80 0x7f93980c8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939d109640 1 -- 172.21.15.32:0/3816084012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f939815ae80 msgr2=0x7f93980c8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939d109640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f939815ae80 0x7f93980c8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939d109640 1 -- 172.21.15.32:0/3816084012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93980b2ae0 msgr2=0x7f93980c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939d109640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93980b2ae0 0x7f93980c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939d109640 1 -- 172.21.15.32:0/3816084012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93880094d0 con 0x7f93980afd50 2023-12-20T05:43:27.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939d90a640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f939815ae80 0x7f93980c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:27.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939d109640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980afd50 0x7f939815dd30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f939400c370 tx=0x7f939400c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:27.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f938dffb640 1 -- 172.21.15.32:0/3816084012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93940052d0 con 0x7f93980afd50 2023-12-20T05:43:27.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f938dffb640 1 -- 172.21.15.32:0/3816084012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9394005430 con 0x7f93980afd50 2023-12-20T05:43:27.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f938dffb640 1 -- 172.21.15.32:0/3816084012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9394007260 con 0x7f93980afd50 2023-12-20T05:43:27.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939f36d640 1 -- 172.21.15.32:0/3816084012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f939815e270 con 0x7f93980afd50 2023-12-20T05:43:27.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939f36d640 1 -- 172.21.15.32:0/3816084012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f939814d5d0 con 0x7f93980afd50 2023-12-20T05:43:27.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.301+0000 7f939f36d640 1 -- 172.21.15.32:0/3816084012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93980b3880 con 0x7f93980afd50 2023-12-20T05:43:27.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.305+0000 7f938dffb640 1 -- 172.21.15.32:0/3816084012 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 82) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f93940073c0 con 0x7f93980afd50 2023-12-20T05:43:27.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.305+0000 7f938dffb640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f937c0658e0 0x7f937c067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.305+0000 7f938dffb640 1 -- 172.21.15.32:0/3816084012 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f93940823a0 con 0x7f93980afd50 2023-12-20T05:43:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.305+0000 7f939c908640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f937c0658e0 0x7f937c067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.305+0000 7f939c908640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f937c0658e0 0x7f937c067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f93880b2000 tx=0x7f93880aa070 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:27.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.309+0000 7f938dffb640 1 -- 172.21.15.32:0/3816084012 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f93980b3880 con 0x7f93980afd50 2023-12-20T05:43:27.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.465+0000 7f939f36d640 1 -- 172.21.15.32:0/3816084012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f93980b3880 con 0x7f93980afd50 2023-12-20T05:43:27.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f938dffb640 1 -- 172.21.15.32:0/3816084012 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v146) v1 ==== 74+0+17109 (secure 0 0 0) 0x7f93980b3880 con 0x7f93980afd50 2023-12-20T05:43:27.468 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:27.468 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":146,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:43:25.722250+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":16,"pool_name":"cephfs_metadata","create_time":"2023-12-20T04:47:19.596882+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"137","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":17,"pool_name":"cephfs_data","create_time":"2023-12-20T04:47:20.055023+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"145","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":113,"snap_epoch":145,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"17.f","mappings":[{"from":0,"to":2}]},{"pgid":"17.17","mappings":[{"from":0,"to":5}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/568130455":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/2815557802":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/1186578222":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/801772685":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/2494305942":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/2892107659":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/2511943559":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/3621453434":"2023-12-21T04:47:33.113748+0000","172.21.15.32:6833/38009":"2023-12-21T04:47:33.113748+0000","172.21.15.32:6832/38009":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/1810110767":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/2135559470":"2023-12-21T04:47:33.113748+0000","172.21.15.32:0/3483848649":"2023-12-21T05:43:25.722217+0000","172.21.15.32:0/1663137028":"2023-12-21T04:47:33.113748+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":17,"snaps":[{"begin":3,"length":55},{"begin":59,"length":55}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:43:27.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 -- 172.21.15.32:0/3816084012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f937c0658e0 msgr2=0x7f937c067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f937c0658e0 0x7f937c067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f93880b2000 tx=0x7f93880aa070 comp rx=0 tx=0).stop 2023-12-20T05:43:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 -- 172.21.15.32:0/3816084012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980afd50 msgr2=0x7f939815dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980afd50 0x7f939815dd30 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f939400c370 tx=0x7f939400c830 comp rx=0 tx=0).stop 2023-12-20T05:43:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 -- 172.21.15.32:0/3816084012 shutdown_connections 2023-12-20T05:43:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f937c0658e0 0x7f937c067da0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f939815ae80 0x7f93980c8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93980b2ae0 0x7f93980c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 --2- 172.21.15.32:0/3816084012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93980afd50 0x7f939815dd30 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 -- 172.21.15.32:0/3816084012 >> 172.21.15.32:0/3816084012 conn(0x7f93980cd3d0 msgr2=0x7f9398150140 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 -- 172.21.15.32:0/3816084012 shutdown_connections 2023-12-20T05:43:27.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.469+0000 7f939f36d640 1 -- 172.21.15.32:0/3816084012 wait complete. 2023-12-20T05:43:27.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T05:43:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.633+0000 7f812996d640 1 Processor -- start 2023-12-20T05:43:27.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.633+0000 7f812996d640 1 -- start start 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.633+0000 7f812996d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81240aece0 0x7f81240b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.633+0000 7f812996d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81240b0b30 0x7f81240701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.633+0000 7f812996d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81240c98c0 con 0x7f81241599e0 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.633+0000 7f812996d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81240c9ba0 con 0x7f81240b0b30 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.633+0000 7f812996d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81240c9e80 con 0x7f81240aece0 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.633+0000 7f81237fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81240b0b30 0x7f81240701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.637+0000 7f81237fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81240b0b30 0x7f81240701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50318/0 (socket says 172.21.15.32:50318) 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.637+0000 7f81237fe640 1 -- 172.21.15.32:0/2991896751 learned_addr learned my addr 172.21.15.32:0/2991896751 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.637+0000 7f811bfff640 1 --2- 172.21.15.32:0/2991896751 >> v2:172.21.15.39:3300/0 conn(0x7f81240aece0 0x7f81240b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.637+0000 7f81237fe640 1 -- 172.21.15.32:0/2991896751 >> v2:172.21.15.39:3300/0 conn(0x7f81240aece0 msgr2=0x7f81240b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.637+0000 7f81237fe640 1 --2- 172.21.15.32:0/2991896751 >> v2:172.21.15.39:3300/0 conn(0x7f81240aece0 0x7f81240b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.637+0000 7f81237fe640 1 -- 172.21.15.32:0/2991896751 >> v1:172.21.15.39:6789/0 conn(0x7f81241599e0 legacy=0x7f81240c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.637+0000 7f81237fe640 1 -- 172.21.15.32:0/2991896751 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81240ca160 con 0x7f81240b0b30 2023-12-20T05:43:27.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.637+0000 7f81237fe640 1 --2- 172.21.15.32:0/2991896751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81240b0b30 0x7f81240701c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f81240cf520 tx=0x7f811c01f640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb37a66798401f4b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:27.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.637+0000 7f81227fc640 1 -- 172.21.15.32:0/2991896751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c0a79c0 con 0x7f81240b0b30 2023-12-20T05:43:27.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.637+0000 7f81227fc640 1 -- 172.21.15.32:0/2991896751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c0a7b20 con 0x7f81240b0b30 2023-12-20T05:43:27.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.637+0000 7f81227fc640 1 -- 172.21.15.32:0/2991896751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c0af3f0 con 0x7f81240b0b30 2023-12-20T05:43:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 -- 172.21.15.32:0/2991896751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81240b0b30 msgr2=0x7f81240701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 --2- 172.21.15.32:0/2991896751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81240b0b30 0x7f81240701c0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f81240cf520 tx=0x7f811c01f640 comp rx=0 tx=0).stop 2023-12-20T05:43:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 -- 172.21.15.32:0/2991896751 shutdown_connections 2023-12-20T05:43:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 --2- 172.21.15.32:0/2991896751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81240b0b30 0x7f81240701c0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 --2- 172.21.15.32:0/2991896751 >> v2:172.21.15.39:3300/0 conn(0x7f81240aece0 0x7f81240b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 -- 172.21.15.32:0/2991896751 >> 172.21.15.32:0/2991896751 conn(0x7f812406a320 msgr2=0x7f81240ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 -- 172.21.15.32:0/2991896751 shutdown_connections 2023-12-20T05:43:27.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 -- 172.21.15.32:0/2991896751 wait complete. 2023-12-20T05:43:27.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 Processor -- start 2023-12-20T05:43:27.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 -- start start 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81240aece0 0x7f812414c1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81240b0b30 0x7f81241623c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81241599e0 0x7f8124163f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81240ccc60 con 0x7f81241599e0 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81241f9690 con 0x7f81240aece0 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f812996d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f812414c730 con 0x7f81240b0b30 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f81237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81241599e0 0x7f8124163f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f81237fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81241599e0 0x7f8124163f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50328/0 (socket says 172.21.15.32:50328) 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f81237fe640 1 -- 172.21.15.32:0/2677508684 learned_addr learned my addr 172.21.15.32:0/2677508684 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f8122ffd640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81240aece0 0x7f812414c1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f811bfff640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81240b0b30 0x7f81241623c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f81237fe640 1 -- 172.21.15.32:0/2677508684 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81240b0b30 msgr2=0x7f81241623c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f81237fe640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81240b0b30 0x7f81241623c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f81237fe640 1 -- 172.21.15.32:0/2677508684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81240aece0 msgr2=0x7f812414c1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f81237fe640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81240aece0 0x7f812414c1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f81237fe640 1 -- 172.21.15.32:0/2677508684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f811c009b30 con 0x7f81241599e0 2023-12-20T05:43:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.641+0000 7f8122ffd640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81240aece0 0x7f812414c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f81237fe640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81241599e0 0x7f8124163f20 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f811c0b1040 tx=0x7f811c01b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f8120ff9640 1 -- 172.21.15.32:0/2677508684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c0a79c0 con 0x7f81241599e0 2023-12-20T05:43:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f8120ff9640 1 -- 172.21.15.32:0/2677508684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f811c00e150 con 0x7f81241599e0 2023-12-20T05:43:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f8120ff9640 1 -- 172.21.15.32:0/2677508684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f811c007dd0 con 0x7f81241599e0 2023-12-20T05:43:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f812996d640 1 -- 172.21.15.32:0/2677508684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81240cc6a0 con 0x7f81241599e0 2023-12-20T05:43:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f812996d640 1 -- 172.21.15.32:0/2677508684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81241f9040 con 0x7f81241599e0 2023-12-20T05:43:27.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f812996d640 1 -- 172.21.15.32:0/2677508684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81241f9800 con 0x7f81241599e0 2023-12-20T05:43:27.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f8120ff9640 1 -- 172.21.15.32:0/2677508684 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 82) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f811c0c2450 con 0x7f81241599e0 2023-12-20T05:43:27.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f8120ff9640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81040658b0 0x7f8104067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:27.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f8120ff9640 1 -- 172.21.15.32:0/2677508684 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(146..146 src has 1..146) v4 ==== 6441+0+0 (secure 0 0 0) 0x7f811c11d630 con 0x7f81241599e0 2023-12-20T05:43:27.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f8122ffd640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81040658b0 0x7f8104067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:27.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.645+0000 7f8122ffd640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81040658b0 0x7f8104067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8124155df0 tx=0x7f8114009040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:27.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.649+0000 7f8120ff9640 1 -- 172.21.15.32:0/2677508684 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f811c0eaf80 con 0x7f81241599e0 2023-12-20T05:43:27.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.841+0000 7f812996d640 1 -- 172.21.15.32:0/2677508684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f812404e000 con 0x7f81241599e0 2023-12-20T05:43:27.844 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T05:43:27.845+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T05:43:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.857+0000 7f8120ff9640 1 -- 172.21.15.32:0/2677508684 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v63) v1 ==== 161+0+0 (secure 0 0 0) 0x7f811c009510 con 0x7f81241599e0 2023-12-20T05:43:27.858 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T05:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 -- 172.21.15.32:0/2677508684 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81040658b0 msgr2=0x7f8104067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81040658b0 0x7f8104067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f8124155df0 tx=0x7f8114009040 comp rx=0 tx=0).stop 2023-12-20T05:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 -- 172.21.15.32:0/2677508684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81241599e0 msgr2=0x7f8124163f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81241599e0 0x7f8124163f20 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f811c0b1040 tx=0x7f811c01b980 comp rx=0 tx=0).stop 2023-12-20T05:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 -- 172.21.15.32:0/2677508684 shutdown_connections 2023-12-20T05:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81040658b0 0x7f8104067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81241599e0 0x7f8124163f20 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81240b0b30 0x7f81241623c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 --2- 172.21.15.32:0/2677508684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81240aece0 0x7f812414c1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 -- 172.21.15.32:0/2677508684 >> 172.21.15.32:0/2677508684 conn(0x7f812406a320 msgr2=0x7f8124158a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 -- 172.21.15.32:0/2677508684 shutdown_connections 2023-12-20T05:43:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:27.861+0000 7f812996d640 1 -- 172.21.15.32:0/2677508684 wait complete. 2023-12-20T05:43:27.882 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T05:43:27.926 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T05:43:28.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.029+0000 7ff63d8f9640 1 Processor -- start 2023-12-20T05:43:28.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.029+0000 7ff63d8f9640 1 -- start start 2023-12-20T05:43:28.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.033+0000 7ff63d8f9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff638157480 0x7ff6380c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:28.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.033+0000 7ff63d8f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6380cde20 con 0x7ff6380cd4c0 2023-12-20T05:43:28.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.033+0000 7ff63d8f9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6380c8e30 con 0x7ff6380b0760 2023-12-20T05:43:28.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.033+0000 7ff63d8f9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff63815e480 con 0x7ff638157480 2023-12-20T05:43:28.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.033+0000 7ff636ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff638157480 0x7ff6380c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:28.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.033+0000 7ff6377fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff6380cd4c0 0x7ff63815dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51436/0 (socket says 172.21.15.32:51436) 2023-12-20T05:43:28.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.033+0000 7ff6377fe640 1 -- 172.21.15.32:0/34513142 learned_addr learned my addr 172.21.15.32:0/34513142 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:28.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.033+0000 7ff636ffd640 1 -- 172.21.15.32:0/34513142 >> v1:172.21.15.39:6789/0 conn(0x7ff6380b0760 legacy=0x7ff63815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:28.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.033+0000 7ff636ffd640 1 -- 172.21.15.32:0/34513142 >> v1:172.21.15.32:6789/0 conn(0x7ff6380cd4c0 legacy=0x7ff63815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:28.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.033+0000 7ff636ffd640 1 -- 172.21.15.32:0/34513142 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6380ce070 con 0x7ff638157480 2023-12-20T05:43:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff636ffd640 1 --2- 172.21.15.32:0/34513142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff638157480 0x7ff6380c9840 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff6280097c0 tx=0x7ff62801c280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfaa6e91bddbd00f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff6367fc640 1 -- 172.21.15.32:0/34513142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280ab600 con 0x7ff638157480 2023-12-20T05:43:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff6367fc640 1 -- 172.21.15.32:0/34513142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6280a7050 con 0x7ff638157480 2023-12-20T05:43:28.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff6367fc640 1 -- 172.21.15.32:0/34513142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6280ab980 con 0x7ff638157480 2023-12-20T05:43:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff63d8f9640 1 -- 172.21.15.32:0/34513142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff638157480 msgr2=0x7ff6380c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff63d8f9640 1 --2- 172.21.15.32:0/34513142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff638157480 0x7ff6380c9840 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7ff6280097c0 tx=0x7ff62801c280 comp rx=0 tx=0).stop 2023-12-20T05:43:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff63d8f9640 1 -- 172.21.15.32:0/34513142 shutdown_connections 2023-12-20T05:43:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff63d8f9640 1 --2- 172.21.15.32:0/34513142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff638157480 0x7ff6380c9840 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff63d8f9640 1 -- 172.21.15.32:0/34513142 >> 172.21.15.32:0/34513142 conn(0x7ff63806a510 msgr2=0x7ff6380ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff63d8f9640 1 -- 172.21.15.32:0/34513142 shutdown_connections 2023-12-20T05:43:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff63d8f9640 1 -- 172.21.15.32:0/34513142 wait complete. 2023-12-20T05:43:28.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff63d8f9640 1 Processor -- start 2023-12-20T05:43:28.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.037+0000 7ff63d8f9640 1 -- start start 2023-12-20T05:43:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff63d8f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6380b0760 0x7ff6380c86b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff63d8f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380cd4c0 0x7ff63814ca40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff63d8f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff638157480 0x7ff6380b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff63d8f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff63814cf80 con 0x7ff6380cd4c0 2023-12-20T05:43:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff63d8f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6380b1e70 con 0x7ff638157480 2023-12-20T05:43:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff63d8f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff63814e520 con 0x7ff6380b0760 2023-12-20T05:43:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff62ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380cd4c0 0x7ff63814ca40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff62ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380cd4c0 0x7ff63814ca40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50364/0 (socket says 172.21.15.32:50364) 2023-12-20T05:43:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff62ffff640 1 -- 172.21.15.32:0/2630280308 learned_addr learned my addr 172.21.15.32:0/2630280308 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:28.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff636ffd640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6380b0760 0x7ff6380c86b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff6377fe640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff638157480 0x7ff6380b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff62ffff640 1 -- 172.21.15.32:0/2630280308 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6380b0760 msgr2=0x7ff6380c86b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff62ffff640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6380b0760 0x7ff6380c86b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff62ffff640 1 -- 172.21.15.32:0/2630280308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff638157480 msgr2=0x7ff6380b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff62ffff640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff638157480 0x7ff6380b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff62ffff640 1 -- 172.21.15.32:0/2630280308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff628009420 con 0x7ff6380cd4c0 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff62ffff640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380cd4c0 0x7ff63814ca40 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff62000c3c0 tx=0x7ff62000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff634ff9640 1 -- 172.21.15.32:0/2630280308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6200052d0 con 0x7ff6380cd4c0 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff634ff9640 1 -- 172.21.15.32:0/2630280308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff620005430 con 0x7ff6380cd4c0 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff634ff9640 1 -- 172.21.15.32:0/2630280308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff620003840 con 0x7ff6380cd4c0 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff63d8f9640 1 -- 172.21.15.32:0/2630280308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff63814e7e0 con 0x7ff6380cd4c0 2023-12-20T05:43:28.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff63d8f9640 1 -- 172.21.15.32:0/2630280308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff63814eda0 con 0x7ff6380cd4c0 2023-12-20T05:43:28.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff63d8f9640 1 -- 172.21.15.32:0/2630280308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff63804e000 con 0x7ff6380cd4c0 2023-12-20T05:43:28.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.041+0000 7ff634ff9640 1 -- 172.21.15.32:0/2630280308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 82) v1 ==== 85066+0+0 (secure 0 0 0) 0x7ff6200039a0 con 0x7ff6380cd4c0 2023-12-20T05:43:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.045+0000 7ff634ff9640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6180658b0 0x7ff618067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.045+0000 7ff634ff9640 1 -- 172.21.15.32:0/2630280308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(147..147 src has 1..147) v4 ==== 6527+0+0 (secure 0 0 0) 0x7ff620082580 con 0x7ff6380cd4c0 2023-12-20T05:43:28.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.045+0000 7ff636ffd640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6180658b0 0x7ff618067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:28.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.045+0000 7ff636ffd640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6180658b0 0x7ff618067d70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff6280ad000 tx=0x7ff628008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:28.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.049+0000 7ff634ff9640 1 -- 172.21.15.32:0/2630280308 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff620050000 con 0x7ff6380cd4c0 2023-12-20T05:43:28.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.157+0000 7ff634ff9640 1 -- 172.21.15.32:0/2630280308 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 83) v1 ==== 85274+0+0 (secure 0 0 0) 0x7ff6200503d0 con 0x7ff6380cd4c0 2023-12-20T05:43:28.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.241+0000 7ff63d8f9640 1 -- 172.21.15.32:0/2630280308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ff6380ce320 con 0x7ff6380cd4c0 2023-12-20T05:43:28.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff634ff9640 1 -- 172.21.15.32:0/2630280308 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v64) v1 ==== 104+0+0 (secure 0 0 0) 0x7ff62006c920 con 0x7ff6380cd4c0 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff63d8f9640 1 -- 172.21.15.32:0/2630280308 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6180658b0 msgr2=0x7ff618067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff63d8f9640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6180658b0 0x7ff618067d70 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7ff6280ad000 tx=0x7ff628008040 comp rx=0 tx=0).stop 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff63d8f9640 1 -- 172.21.15.32:0/2630280308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380cd4c0 msgr2=0x7ff63814ca40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff63d8f9640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380cd4c0 0x7ff63814ca40 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7ff62000c3c0 tx=0x7ff62000c880 comp rx=0 tx=0).stop 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff63d8f9640 1 -- 172.21.15.32:0/2630280308 shutdown_connections 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff63d8f9640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff6180658b0 0x7ff618067d70 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff63d8f9640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff638157480 0x7ff6380b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff63d8f9640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6380cd4c0 0x7ff63814ca40 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff63d8f9640 1 --2- 172.21.15.32:0/2630280308 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6380b0760 0x7ff6380c86b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff63d8f9640 1 -- 172.21.15.32:0/2630280308 >> 172.21.15.32:0/2630280308 conn(0x7ff63806a510 msgr2=0x7ff6381566c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.861+0000 7ff63d8f9640 1 -- 172.21.15.32:0/2630280308 shutdown_connections 2023-12-20T05:43:28.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:28.865+0000 7ff63d8f9640 1 -- 172.21.15.32:0/2630280308 wait complete. 2023-12-20T05:43:28.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T05:43:29.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.077+0000 7f930a37b640 1 Processor -- start 2023-12-20T05:43:29.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.077+0000 7f930a37b640 1 -- start start 2023-12-20T05:43:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.077+0000 7f930a37b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9304159af0 0x7f93040b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.077+0000 7f930a37b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f930406d2d0 0x7f930406d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.077+0000 7f930a37b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f930406dd50 con 0x7f93040b07f0 2023-12-20T05:43:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.077+0000 7f930a37b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93040cc940 con 0x7f9304159af0 2023-12-20T05:43:29.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.077+0000 7f930a37b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93040ccc20 con 0x7f930406d2d0 2023-12-20T05:43:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.077+0000 7f9303fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9304159af0 0x7f93040b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.077+0000 7f9303fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9304159af0 0x7f93040b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43606/0 (socket says 172.21.15.32:43606) 2023-12-20T05:43:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.077+0000 7f9303fff640 1 -- 172.21.15.32:0/4017640373 learned_addr learned my addr 172.21.15.32:0/4017640373 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.077+0000 7f93037fe640 1 --2- 172.21.15.32:0/4017640373 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f930406d2d0 0x7f930406d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.081+0000 7f9303fff640 1 -- 172.21.15.32:0/4017640373 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f930406d2d0 msgr2=0x7f930406d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.081+0000 7f9303fff640 1 --2- 172.21.15.32:0/4017640373 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f930406d2d0 0x7f930406d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.081+0000 7f9303fff640 1 -- 172.21.15.32:0/4017640373 >> v1:172.21.15.39:6789/0 conn(0x7f93040b07f0 legacy=0x7f93040cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:29.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.081+0000 7f9303fff640 1 -- 172.21.15.32:0/4017640373 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93040ccf00 con 0x7f9304159af0 2023-12-20T05:43:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.081+0000 7f9303fff640 1 --2- 172.21.15.32:0/4017640373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9304159af0 0x7f93040b1930 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009870 tx=0x7f92f401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2bf62befade02ef server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:29.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f9302ffd640 1 -- 172.21.15.32:0/4017640373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f40a95f0 con 0x7f9304159af0 2023-12-20T05:43:29.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f9302ffd640 1 -- 172.21.15.32:0/4017640373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f40a9750 con 0x7f9304159af0 2023-12-20T05:43:29.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f9302ffd640 1 -- 172.21.15.32:0/4017640373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f40a9ae0 con 0x7f9304159af0 2023-12-20T05:43:29.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f930a37b640 1 -- 172.21.15.32:0/4017640373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9304159af0 msgr2=0x7f93040b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:29.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f930a37b640 1 --2- 172.21.15.32:0/4017640373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9304159af0 0x7f93040b1930 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f92f4009870 tx=0x7f92f401c510 comp rx=0 tx=0).stop 2023-12-20T05:43:29.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f930a37b640 1 -- 172.21.15.32:0/4017640373 shutdown_connections 2023-12-20T05:43:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f930a37b640 1 --2- 172.21.15.32:0/4017640373 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f930406d2d0 0x7f930406d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f930a37b640 1 --2- 172.21.15.32:0/4017640373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9304159af0 0x7f93040b1930 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f930a37b640 1 -- 172.21.15.32:0/4017640373 >> 172.21.15.32:0/4017640373 conn(0x7f930406a510 msgr2=0x7f93040ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f930a37b640 1 -- 172.21.15.32:0/4017640373 shutdown_connections 2023-12-20T05:43:29.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f930a37b640 1 -- 172.21.15.32:0/4017640373 wait complete. 2023-12-20T05:43:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f930a37b640 1 Processor -- start 2023-12-20T05:43:29.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.085+0000 7f930a37b640 1 -- start start 2023-12-20T05:43:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f930a37b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930406d2d0 0x7f93041420c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f930a37b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93040b07f0 0x7f9304156cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f930a37b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9304159af0 0x7f9304146170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:29.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f930a37b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9304142600 con 0x7f930406d2d0 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f930a37b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f930414fdc0 con 0x7f93040b07f0 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f930a37b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93041500a0 con 0x7f9304159af0 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9303fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930406d2d0 0x7f93041420c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9303fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930406d2d0 0x7f93041420c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43616/0 (socket says 172.21.15.32:43616) 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9303fff640 1 -- 172.21.15.32:0/2609624751 learned_addr learned my addr 172.21.15.32:0/2609624751 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f93037fe640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93040b07f0 0x7f9304156cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9308918640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9304159af0 0x7f9304146170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9303fff640 1 -- 172.21.15.32:0/2609624751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9304159af0 msgr2=0x7f9304146170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9303fff640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9304159af0 0x7f9304146170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9303fff640 1 -- 172.21.15.32:0/2609624751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93040b07f0 msgr2=0x7f9304156cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9303fff640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93040b07f0 0x7f9304156cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9303fff640 1 -- 172.21.15.32:0/2609624751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92f40094d0 con 0x7f930406d2d0 2023-12-20T05:43:29.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9303fff640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930406d2d0 0x7f93041420c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f92f401abc0 tx=0x7f92f40192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9300ff9640 1 -- 172.21.15.32:0/2609624751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f40a95f0 con 0x7f930406d2d0 2023-12-20T05:43:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9300ff9640 1 -- 172.21.15.32:0/2609624751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92f40b1800 con 0x7f930406d2d0 2023-12-20T05:43:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f930a37b640 1 -- 172.21.15.32:0/2609624751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93040b1190 con 0x7f930406d2d0 2023-12-20T05:43:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f9300ff9640 1 -- 172.21.15.32:0/2609624751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92f40b1bb0 con 0x7f930406d2d0 2023-12-20T05:43:29.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.089+0000 7f930a37b640 1 -- 172.21.15.32:0/2609624751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9304150660 con 0x7f930406d2d0 2023-12-20T05:43:29.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.093+0000 7f930a37b640 1 -- 172.21.15.32:0/2609624751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9304150850 con 0x7f930406d2d0 2023-12-20T05:43:29.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.093+0000 7f9300ff9640 1 -- 172.21.15.32:0/2609624751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 83) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f92f40b1d10 con 0x7f930406d2d0 2023-12-20T05:43:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.093+0000 7f9300ff9640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f92f0065b20 0x7f92f0067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.093+0000 7f9300ff9640 1 -- 172.21.15.32:0/2609624751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(148..148 src has 1..148) v4 ==== 6527+0+0 (secure 0 0 0) 0x7f92f411dfe0 con 0x7f930406d2d0 2023-12-20T05:43:29.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.093+0000 7f93037fe640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f92f0065b20 0x7f92f0067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:29.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.093+0000 7f93037fe640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f92f0065b20 0x7f92f0067fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f93041678b0 tx=0x7f92f8008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:29.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.097+0000 7f9300ff9640 1 -- 172.21.15.32:0/2609624751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f92f40eb580 con 0x7f930406d2d0 2023-12-20T05:43:29.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:29.256+0000 7f930a37b640 1 -- 172.21.15.32:0/2609624751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f930404e000 con 0x7f930406d2d0 2023-12-20T05:43:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.168+0000 7f9300ff9640 1 -- 172.21.15.32:0/2609624751 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f92f40eb950 con 0x7f930406d2d0 2023-12-20T05:43:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.168+0000 7f9300ff9640 1 -- 172.21.15.32:0/2609624751 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v149) v1 ==== 181+0+0 (secure 0 0 0) 0x7f92f40eb580 con 0x7f930406d2d0 2023-12-20T05:43:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T05:43:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 -- 172.21.15.32:0/2609624751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f92f0065b20 msgr2=0x7f92f0067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f92f0065b20 0x7f92f0067fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f93041678b0 tx=0x7f92f8008040 comp rx=0 tx=0).stop 2023-12-20T05:43:30.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 -- 172.21.15.32:0/2609624751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930406d2d0 msgr2=0x7f93041420c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930406d2d0 0x7f93041420c0 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f92f401abc0 tx=0x7f92f40192b0 comp rx=0 tx=0).stop 2023-12-20T05:43:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 -- 172.21.15.32:0/2609624751 shutdown_connections 2023-12-20T05:43:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f92f0065b20 0x7f92f0067fe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9304159af0 0x7f9304146170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93040b07f0 0x7f9304156cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 --2- 172.21.15.32:0/2609624751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f930406d2d0 0x7f93041420c0 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 -- 172.21.15.32:0/2609624751 >> 172.21.15.32:0/2609624751 conn(0x7f930406a510 msgr2=0x7f930406caf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 -- 172.21.15.32:0/2609624751 shutdown_connections 2023-12-20T05:43:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.172+0000 7f930a37b640 1 -- 172.21.15.32:0/2609624751 wait complete. 2023-12-20T05:43:30.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T05:43:30.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.352+0000 7f35a0e8c640 1 Processor -- start 2023-12-20T05:43:30.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.352+0000 7f35a0e8c640 1 -- start start 2023-12-20T05:43:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f35a0e8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f359c070940 0x7f359c0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f35a0e8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c06e100 con 0x7f359c06dcb0 2023-12-20T05:43:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f35a0e8c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f359c0ca590 con 0x7f359c158520 2023-12-20T05:43:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f35a0e8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f359c0ca7f0 con 0x7f359c070940 2023-12-20T05:43:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f3599d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f359c06dcb0 0x7f359c070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32858/0 (socket says 172.21.15.32:32858) 2023-12-20T05:43:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f3599d9b640 1 -- 172.21.15.32:0/666794926 learned_addr learned my addr 172.21.15.32:0/666794926 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f359ad9d640 1 --2- 172.21.15.32:0/666794926 >> v2:172.21.15.39:3300/0 conn(0x7f359c070940 0x7f359c0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f359959a640 1 -- 172.21.15.32:0/666794926 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1566631613 0 0) 0x7f3590001120 con 0x7f359c06dcb0 2023-12-20T05:43:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f359959a640 1 -- 172.21.15.32:0/666794926 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3584003530 con 0x7f359c06dcb0 2023-12-20T05:43:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f359959a640 1 -- 172.21.15.32:0/666794926 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 885853252 0 0) 0x7f3584003530 con 0x7f359c06dcb0 2023-12-20T05:43:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f359ad9d640 1 -- 172.21.15.32:0/666794926 >> v1:172.21.15.39:6789/0 conn(0x7f359c158520 legacy=0x7f359c15ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f359ad9d640 1 -- 172.21.15.32:0/666794926 >> v1:172.21.15.32:6789/0 conn(0x7f359c06dcb0 legacy=0x7f359c070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:30.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.356+0000 7f359ad9d640 1 -- 172.21.15.32:0/666794926 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f359c0caad0 con 0x7f359c070940 2023-12-20T05:43:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.360+0000 7f359ad9d640 1 --2- 172.21.15.32:0/666794926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f359c070940 0x7f359c0ca050 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f359c0ca720 tx=0x7f359401c790 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c55b09fea2949be server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.360+0000 7f359959a640 1 -- 172.21.15.32:0/666794926 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3590001810 con 0x7f359c070940 2023-12-20T05:43:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.360+0000 7f359959a640 1 -- 172.21.15.32:0/666794926 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35940a55d0 con 0x7f359c070940 2023-12-20T05:43:30.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.360+0000 7f359959a640 1 -- 172.21.15.32:0/666794926 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35940a5960 con 0x7f359c070940 2023-12-20T05:43:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.360+0000 7f35a0e8c640 1 -- 172.21.15.32:0/666794926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f359c070940 msgr2=0x7f359c0ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.360+0000 7f35a0e8c640 1 --2- 172.21.15.32:0/666794926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f359c070940 0x7f359c0ca050 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7f359c0ca720 tx=0x7f359401c790 comp rx=0 tx=0).stop 2023-12-20T05:43:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.360+0000 7f35a0e8c640 1 -- 172.21.15.32:0/666794926 shutdown_connections 2023-12-20T05:43:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.360+0000 7f35a0e8c640 1 --2- 172.21.15.32:0/666794926 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f359c070940 0x7f359c0ca050 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.360+0000 7f35a0e8c640 1 -- 172.21.15.32:0/666794926 >> 172.21.15.32:0/666794926 conn(0x7f359c06a510 msgr2=0x7f359c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.360+0000 7f35a0e8c640 1 -- 172.21.15.32:0/666794926 shutdown_connections 2023-12-20T05:43:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.360+0000 7f35a0e8c640 1 -- 172.21.15.32:0/666794926 wait complete. 2023-12-20T05:43:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f35a0e8c640 1 Processor -- start 2023-12-20T05:43:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f35a0e8c640 1 -- start start 2023-12-20T05:43:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f35a0e8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f359c06dcb0 0x7f359c13a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f35a0e8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c070940 0x7f359c13ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f35a0e8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f359c158520 0x7f359c0c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f35a0e8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c156790 con 0x7f359c070940 2023-12-20T05:43:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f35a0e8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f359c13e400 con 0x7f359c06dcb0 2023-12-20T05:43:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f35a0e8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f359c1f9130 con 0x7f359c158520 2023-12-20T05:43:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f3599d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c070940 0x7f359c13ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f359a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f359c06dcb0 0x7f359c13a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f359ad9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f359c158520 0x7f359c0c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f359a59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f359c06dcb0 0x7f359c13a8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:46714/0 (socket says 172.21.15.32:46714) 2023-12-20T05:43:30.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f3599d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c070940 0x7f359c13ae20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43620/0 (socket says 172.21.15.32:43620) 2023-12-20T05:43:30.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f359a59c640 1 -- 172.21.15.32:0/782997031 learned_addr learned my addr 172.21.15.32:0/782997031 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f359a59c640 1 -- 172.21.15.32:0/782997031 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f359c158520 msgr2=0x7f359c0c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f359a59c640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f359c158520 0x7f359c0c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f359a59c640 1 -- 172.21.15.32:0/782997031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c070940 msgr2=0x7f359c13ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f359a59c640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c070940 0x7f359c13ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f359a59c640 1 -- 172.21.15.32:0/782997031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3594009a70 con 0x7f359c06dcb0 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f3599d9b640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c070940 0x7f359c13ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f359a59c640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f359c06dcb0 0x7f359c13a8e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f358c00c3c0 tx=0x7f358c00c880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f358affd640 1 -- 172.21.15.32:0/782997031 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c0052d0 con 0x7f359c06dcb0 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f358affd640 1 -- 172.21.15.32:0/782997031 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f358c005430 con 0x7f359c06dcb0 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f358affd640 1 -- 172.21.15.32:0/782997031 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f358c003840 con 0x7f359c06dcb0 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f35a0e8c640 1 -- 172.21.15.32:0/782997031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f359c13d640 con 0x7f359c06dcb0 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f35a0e8c640 1 -- 172.21.15.32:0/782997031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f359c13dee0 con 0x7f359c06dcb0 2023-12-20T05:43:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.364+0000 7f35a0e8c640 1 -- 172.21.15.32:0/782997031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f359c13da40 con 0x7f359c06dcb0 2023-12-20T05:43:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.368+0000 7f358affd640 1 -- 172.21.15.32:0/782997031 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f358c0039a0 con 0x7f359c06dcb0 2023-12-20T05:43:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.368+0000 7f358affd640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f357c065b80 0x7f357c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.368+0000 7f358affd640 1 -- 172.21.15.32:0/782997031 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(149..149 src has 1..149) v4 ==== 6045+0+0 (secure 0 0 0) 0x7f358c082cb0 con 0x7f359c06dcb0 2023-12-20T05:43:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.368+0000 7f3599d9b640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f357c065b80 0x7f357c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.368+0000 7f3599d9b640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f357c065b80 0x7f357c068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3590001820 tx=0x7f3590001770 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.372+0000 7f358affd640 1 -- 172.21.15.32:0/782997031 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f358c0505b0 con 0x7f359c06dcb0 2023-12-20T05:43:30.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:30.528+0000 7f35a0e8c640 1 -- 172.21.15.32:0/782997031 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f359c13d7d0 con 0x7f359c06dcb0 2023-12-20T05:43:31.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.192+0000 7f358affd640 1 -- 172.21.15.32:0/782997031 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v150) v1 ==== 169+0+0 (secure 0 0 0) 0x7f358c053bd0 con 0x7f359c06dcb0 2023-12-20T05:43:31.195 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T05:43:31.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 -- 172.21.15.32:0/782997031 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f357c065b80 msgr2=0x7f357c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f357c065b80 0x7f357c068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f3590001820 tx=0x7f3590001770 comp rx=0 tx=0).stop 2023-12-20T05:43:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 -- 172.21.15.32:0/782997031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f359c06dcb0 msgr2=0x7f359c13a8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f359c06dcb0 0x7f359c13a8e0 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7f358c00c3c0 tx=0x7f358c00c880 comp rx=0 tx=0).stop 2023-12-20T05:43:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 -- 172.21.15.32:0/782997031 shutdown_connections 2023-12-20T05:43:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f357c065b80 0x7f357c068040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f359c158520 0x7f359c0c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f359c070940 0x7f359c13ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 --2- 172.21.15.32:0/782997031 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f359c06dcb0 0x7f359c13a8e0 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 -- 172.21.15.32:0/782997031 >> 172.21.15.32:0/782997031 conn(0x7f359c06a510 msgr2=0x7f359c1553b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 -- 172.21.15.32:0/782997031 shutdown_connections 2023-12-20T05:43:31.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.196+0000 7f35a0e8c640 1 -- 172.21.15.32:0/782997031 wait complete. 2023-12-20T05:43:31.220 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T05:43:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.368+0000 7f7e42e2d640 1 Processor -- start 2023-12-20T05:43:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.368+0000 7f7e42e2d640 1 -- start start 2023-12-20T05:43:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.368+0000 7f7e42e2d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7e3c0b0760 0x7f7e3c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.368+0000 7f7e42e2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e3c0cc4b0 0x7f7e3c0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.368+0000 7f7e42e2d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e3c0b1100 con 0x7f7e3c157480 2023-12-20T05:43:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.368+0000 7f7e42e2d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e3c0c9f50 con 0x7f7e3c0b0760 2023-12-20T05:43:31.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.368+0000 7f7e42e2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0cde20 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.372+0000 7f7e3bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7e3c0b0760 0x7f7e3c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.372+0000 7f7e413ca640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e3c0cc4b0 0x7f7e3c0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.372+0000 7f7e3bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7e3c0b0760 0x7f7e3c0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:46728/0 (socket says 172.21.15.32:46728) 2023-12-20T05:43:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.372+0000 7f7e3bfff640 1 -- 172.21.15.32:0/1096634959 learned_addr learned my addr 172.21.15.32:0/1096634959 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.372+0000 7f7e3bfff640 1 -- 172.21.15.32:0/1096634959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e3c0cc4b0 msgr2=0x7f7e3c0cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.372+0000 7f7e3bfff640 1 --2- 172.21.15.32:0/1096634959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e3c0cc4b0 0x7f7e3c0cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.372+0000 7f7e3bfff640 1 -- 172.21.15.32:0/1096634959 >> v1:172.21.15.39:6789/0 conn(0x7f7e3c157480 legacy=0x7f7e3c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.372+0000 7f7e3bfff640 1 -- 172.21.15.32:0/1096634959 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e3c0c8e30 con 0x7f7e3c0b0760 2023-12-20T05:43:31.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.372+0000 7f7e3bfff640 1 --2- 172.21.15.32:0/1096634959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c0b0760 0x7f7e3c0b0bc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009870 tx=0x7f7e2801e690 comp rx=0 tx=0).ready entity=mon.1 client_cookie=340b7573e86b0837 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e3b7fe640 1 -- 172.21.15.32:0/1096634959 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280a8660 con 0x7f7e3c0b0760 2023-12-20T05:43:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e3b7fe640 1 -- 172.21.15.32:0/1096634959 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e280a87c0 con 0x7f7e3c0b0760 2023-12-20T05:43:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e3b7fe640 1 -- 172.21.15.32:0/1096634959 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280a8b50 con 0x7f7e3c0b0760 2023-12-20T05:43:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 -- 172.21.15.32:0/1096634959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c0b0760 msgr2=0x7f7e3c0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 --2- 172.21.15.32:0/1096634959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c0b0760 0x7f7e3c0b0bc0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f7e28009870 tx=0x7f7e2801e690 comp rx=0 tx=0).stop 2023-12-20T05:43:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 -- 172.21.15.32:0/1096634959 shutdown_connections 2023-12-20T05:43:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 --2- 172.21.15.32:0/1096634959 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e3c0cc4b0 0x7f7e3c0cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 --2- 172.21.15.32:0/1096634959 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c0b0760 0x7f7e3c0b0bc0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 -- 172.21.15.32:0/1096634959 >> 172.21.15.32:0/1096634959 conn(0x7f7e3c06a510 msgr2=0x7f7e3c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 -- 172.21.15.32:0/1096634959 shutdown_connections 2023-12-20T05:43:31.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 -- 172.21.15.32:0/1096634959 wait complete. 2023-12-20T05:43:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 Processor -- start 2023-12-20T05:43:31.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 -- start start 2023-12-20T05:43:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c0b0760 0x7f7e3c14e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3c0cc4b0 0x7f7e3c1591a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e3c157480 0x7f7e3c163bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c14e700 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c158a00 con 0x7f7e3c0b0760 2023-12-20T05:43:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.376+0000 7f7e42e2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e3c0b2e10 con 0x7f7e3c157480 2023-12-20T05:43:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3c0cc4b0 0x7f7e3c1591a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e3bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3c0cc4b0 0x7f7e3c1591a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43630/0 (socket says 172.21.15.32:43630) 2023-12-20T05:43:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e3bfff640 1 -- 172.21.15.32:0/30755001 learned_addr learned my addr 172.21.15.32:0/30755001 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e40bc9640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c0b0760 0x7f7e3c14e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e413ca640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e3c157480 0x7f7e3c163bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e3bfff640 1 -- 172.21.15.32:0/30755001 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e3c157480 msgr2=0x7f7e3c163bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e3bfff640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e3c157480 0x7f7e3c163bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e3bfff640 1 -- 172.21.15.32:0/30755001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c0b0760 msgr2=0x7f7e3c14e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e3bfff640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c0b0760 0x7f7e3c14e1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e3bfff640 1 -- 172.21.15.32:0/30755001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e280094d0 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e3bfff640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3c0cc4b0 0x7f7e3c1591a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7e28024c80 tx=0x7f7e2801b430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e397fa640 1 -- 172.21.15.32:0/30755001 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280b3650 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e397fa640 1 -- 172.21.15.32:0/30755001 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e280b37b0 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e397fa640 1 -- 172.21.15.32:0/30755001 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e280b3b40 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e42e2d640 1 -- 172.21.15.32:0/30755001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e3c0b3060 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e42e2d640 1 -- 172.21.15.32:0/30755001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e3c0b3620 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e42e2d640 1 -- 172.21.15.32:0/30755001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e3c04e000 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e397fa640 1 -- 172.21.15.32:0/30755001 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7e280b3ca0 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.380+0000 7f7e397fa640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e20065b80 0x7f7e20068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.384+0000 7f7e397fa640 1 -- 172.21.15.32:0/30755001 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f7e2811d670 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.384+0000 7f7e40bc9640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e20065b80 0x7f7e20068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.384+0000 7f7e40bc9640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e20065b80 0x7f7e20068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7e30007fd0 tx=0x7f7e30009040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:31.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.388+0000 7f7e397fa640 1 -- 172.21.15.32:0/30755001 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7e280eaeb0 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.544+0000 7f7e42e2d640 1 -- 172.21.15.32:0/30755001 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f7e3c15e240 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.548+0000 7f7e397fa640 1 -- 172.21.15.32:0/30755001 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v49) v1 ==== 120+0+0 (secure 0 0 0) 0x7f7e280ee4d0 con 0x7f7e3c0cc4b0 2023-12-20T05:43:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 -- 172.21.15.32:0/30755001 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e20065b80 msgr2=0x7f7e20068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e20065b80 0x7f7e20068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f7e30007fd0 tx=0x7f7e30009040 comp rx=0 tx=0).stop 2023-12-20T05:43:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 -- 172.21.15.32:0/30755001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3c0cc4b0 msgr2=0x7f7e3c1591a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3c0cc4b0 0x7f7e3c1591a0 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f7e28024c80 tx=0x7f7e2801b430 comp rx=0 tx=0).stop 2023-12-20T05:43:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 -- 172.21.15.32:0/30755001 shutdown_connections 2023-12-20T05:43:31.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e20065b80 0x7f7e20068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e3c157480 0x7f7e3c163bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e3c0cc4b0 0x7f7e3c1591a0 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 --2- 172.21.15.32:0/30755001 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e3c0b0760 0x7f7e3c14e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 -- 172.21.15.32:0/30755001 >> 172.21.15.32:0/30755001 conn(0x7f7e3c06a510 msgr2=0x7f7e3c0c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 -- 172.21.15.32:0/30755001 shutdown_connections 2023-12-20T05:43:31.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.552+0000 7f7e42e2d640 1 -- 172.21.15.32:0/30755001 wait complete. 2023-12-20T05:43:31.573 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T05:43:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.716+0000 7f5aa52d1640 1 Processor -- start 2023-12-20T05:43:31.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.716+0000 7f5aa52d1640 1 -- start start 2023-12-20T05:43:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5aa52d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5aa00b0cb0 0x7f5aa00caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5aa52d1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5aa0070230 0x7f5aa0070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5aa52d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5aa0070bc0 con 0x7f5aa015abb0 2023-12-20T05:43:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5aa52d1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa006e160 con 0x7f5aa0070230 2023-12-20T05:43:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5aa52d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5aa015b6a0 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5a9f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5aa0070230 0x7f5aa0070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5a9f7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5aa0070230 0x7f5aa0070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43646/0 (socket says 172.21.15.32:43646) 2023-12-20T05:43:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5a9f7fe640 1 -- 172.21.15.32:0/1588089717 learned_addr learned my addr 172.21.15.32:0/1588089717 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5a9e7fc640 1 --2- 172.21.15.32:0/1588089717 >> v2:172.21.15.39:3300/0 conn(0x7f5aa00b0cb0 0x7f5aa00caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5a9f7fe640 1 -- 172.21.15.32:0/1588089717 >> v2:172.21.15.39:3300/0 conn(0x7f5aa00b0cb0 msgr2=0x7f5aa00caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5a9f7fe640 1 --2- 172.21.15.32:0/1588089717 >> v2:172.21.15.39:3300/0 conn(0x7f5aa00b0cb0 0x7f5aa00caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5a9f7fe640 1 -- 172.21.15.32:0/1588089717 >> v1:172.21.15.39:6789/0 conn(0x7f5aa015abb0 legacy=0x7f5aa015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.720+0000 7f5a9f7fe640 1 -- 172.21.15.32:0/1588089717 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5aa006d8c0 con 0x7f5aa0070230 2023-12-20T05:43:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5a9f7fe640 1 --2- 172.21.15.32:0/1588089717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa0070230 0x7f5aa0070680 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00b4470 tx=0x7f5a9801c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4007abec70f614 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5a9dffb640 1 -- 172.21.15.32:0/1588089717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a980a5850 con 0x7f5aa0070230 2023-12-20T05:43:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5a9dffb640 1 -- 172.21.15.32:0/1588089717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a980a59b0 con 0x7f5aa0070230 2023-12-20T05:43:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5a9dffb640 1 -- 172.21.15.32:0/1588089717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a980a5d40 con 0x7f5aa0070230 2023-12-20T05:43:31.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5aa52d1640 1 -- 172.21.15.32:0/1588089717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa0070230 msgr2=0x7f5aa0070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5aa52d1640 1 --2- 172.21.15.32:0/1588089717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa0070230 0x7f5aa0070680 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f5aa00b4470 tx=0x7f5a9801c990 comp rx=0 tx=0).stop 2023-12-20T05:43:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5aa52d1640 1 -- 172.21.15.32:0/1588089717 shutdown_connections 2023-12-20T05:43:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5aa52d1640 1 --2- 172.21.15.32:0/1588089717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa0070230 0x7f5aa0070680 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5aa52d1640 1 --2- 172.21.15.32:0/1588089717 >> v2:172.21.15.39:3300/0 conn(0x7f5aa00b0cb0 0x7f5aa00caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5aa52d1640 1 -- 172.21.15.32:0/1588089717 >> 172.21.15.32:0/1588089717 conn(0x7f5aa006a510 msgr2=0x7f5aa00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5aa52d1640 1 -- 172.21.15.32:0/1588089717 shutdown_connections 2023-12-20T05:43:31.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5aa52d1640 1 -- 172.21.15.32:0/1588089717 wait complete. 2023-12-20T05:43:31.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5aa52d1640 1 Processor -- start 2023-12-20T05:43:31.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.724+0000 7f5aa52d1640 1 -- start start 2023-12-20T05:43:31.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5aa52d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aa0070230 0x7f5aa01523b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5aa52d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa00b0cb0 0x7f5aa01500f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5aa52d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5aa015abb0 0x7f5aa00c8fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5aa52d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00cc8a0 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5aa52d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00ccaf0 con 0x7f5aa0070230 2023-12-20T05:43:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5aa52d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5aa00ccdd0 con 0x7f5aa015abb0 2023-12-20T05:43:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa00b0cb0 0x7f5aa01500f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9effd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aa0070230 0x7f5aa01523b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9e7fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa00b0cb0 0x7f5aa01500f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43660/0 (socket says 172.21.15.32:43660) 2023-12-20T05:43:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9e7fc640 1 -- 172.21.15.32:0/3672045921 learned_addr learned my addr 172.21.15.32:0/3672045921 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9f7fe640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5aa015abb0 0x7f5aa00c8fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9e7fc640 1 -- 172.21.15.32:0/3672045921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5aa015abb0 msgr2=0x7f5aa00c8fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9e7fc640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5aa015abb0 0x7f5aa00c8fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9e7fc640 1 -- 172.21.15.32:0/3672045921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aa0070230 msgr2=0x7f5aa01523b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9e7fc640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aa0070230 0x7f5aa01523b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9e7fc640 1 -- 172.21.15.32:0/3672045921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5a98009b30 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9f7fe640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5aa015abb0 0x7f5aa00c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a9e7fc640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa00b0cb0 0x7f5aa01500f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400d7d0 tx=0x7f5a9400dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a877fe640 1 -- 172.21.15.32:0/3672045921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a94018070 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a877fe640 1 -- 172.21.15.32:0/3672045921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5a94004020 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5aa52d1640 1 -- 172.21.15.32:0/3672045921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5aa00cd0b0 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a877fe640 1 -- 172.21.15.32:0/3672045921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5a940041d0 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5aa52d1640 1 -- 172.21.15.32:0/3672045921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5aa0150630 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5aa52d1640 1 -- 172.21.15.32:0/3672045921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5aa004e000 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.728+0000 7f5a877fe640 1 -- 172.21.15.32:0/3672045921 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5a94004330 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.732+0000 7f5a877fe640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5a8c065b80 0x7f5a8c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:31.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.732+0000 7f5a877fe640 1 -- 172.21.15.32:0/3672045921 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5a94083030 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.732+0000 7f5a9effd640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5a8c065b80 0x7f5a8c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:31.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.732+0000 7f5a9effd640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5a8c065b80 0x7f5a8c068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5a90007fd0 tx=0x7f5a90007e40 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:31.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.736+0000 7f5a877fe640 1 -- 172.21.15.32:0/3672045921 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5a940509f0 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.904+0000 7f5aa52d1640 1 -- 172.21.15.32:0/3672045921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f5aa00b38f0 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5a877fe640 1 -- 172.21.15.32:0/3672045921 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v50) v1 ==== 115+0+0 (secure 0 0 0) 0x7f5a94054010 con 0x7f5aa00b0cb0 2023-12-20T05:43:31.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 -- 172.21.15.32:0/3672045921 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5a8c065b80 msgr2=0x7f5a8c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5a8c065b80 0x7f5a8c068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f5a90007fd0 tx=0x7f5a90007e40 comp rx=0 tx=0).stop 2023-12-20T05:43:31.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 -- 172.21.15.32:0/3672045921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa00b0cb0 msgr2=0x7f5aa01500f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:31.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa00b0cb0 0x7f5aa01500f0 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f5a9400d7d0 tx=0x7f5a9400dc90 comp rx=0 tx=0).stop 2023-12-20T05:43:31.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 -- 172.21.15.32:0/3672045921 shutdown_connections 2023-12-20T05:43:31.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5a8c065b80 0x7f5a8c068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5aa015abb0 0x7f5aa00c8fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5aa00b0cb0 0x7f5aa01500f0 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 --2- 172.21.15.32:0/3672045921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5aa0070230 0x7f5aa01523b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 -- 172.21.15.32:0/3672045921 >> 172.21.15.32:0/3672045921 conn(0x7f5aa006a510 msgr2=0x7f5aa00b40f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 -- 172.21.15.32:0/3672045921 shutdown_connections 2023-12-20T05:43:31.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:31.912+0000 7f5aa52d1640 1 -- 172.21.15.32:0/3672045921 wait complete. 2023-12-20T05:43:31.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T05:43:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f532afce640 1 Processor -- start 2023-12-20T05:43:32.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f532afce640 1 -- start start 2023-12-20T05:43:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f532afce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53240b0760 0x7f53240b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f532afce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53240cc4b0 0x7f53240cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f532afce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53240b1100 con 0x7f5324157480 2023-12-20T05:43:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f532afce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53240c9f50 con 0x7f53240b0760 2023-12-20T05:43:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f532afce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53240cde20 con 0x7f53240cc4b0 2023-12-20T05:43:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f5328d6a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5324157480 0x7f53240c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32862/0 (socket says 172.21.15.32:32862) 2023-12-20T05:43:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f5328d6a640 1 -- 172.21.15.32:0/2138557059 learned_addr learned my addr 172.21.15.32:0/2138557059 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f5323fff640 1 --2- 172.21.15.32:0/2138557059 >> v2:172.21.15.32:3300/0 conn(0x7f53240b0760 0x7f53240b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f532956b640 1 --2- 172.21.15.32:0/2138557059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53240cc4b0 0x7f53240cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f5323fff640 1 -- 172.21.15.32:0/2138557059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53240cc4b0 msgr2=0x7f53240cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f5323fff640 1 --2- 172.21.15.32:0/2138557059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53240cc4b0 0x7f53240cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f5323fff640 1 -- 172.21.15.32:0/2138557059 >> v1:172.21.15.32:6789/0 conn(0x7f5324157480 legacy=0x7f53240c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f5323fff640 1 -- 172.21.15.32:0/2138557059 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53240c8e30 con 0x7f53240b0760 2023-12-20T05:43:32.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f53237fe640 1 -- 172.21.15.32:0/2138557059 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3991874039 0 0) 0x7f53240b1100 con 0x7f5324157480 2023-12-20T05:43:32.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.080+0000 7f532956b640 1 --2- 172.21.15.32:0/2138557059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53240cc4b0 0x7f53240cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:32.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f5323fff640 1 --2- 172.21.15.32:0/2138557059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53240b0760 0x7f53240b0bc0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f53100097c0 tx=0x7f531001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381767d430e67dc3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:32.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f53237fe640 1 -- 172.21.15.32:0/2138557059 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53100ac620 con 0x7f53240b0760 2023-12-20T05:43:32.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f53237fe640 1 -- 172.21.15.32:0/2138557059 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f53100ac780 con 0x7f53240b0760 2023-12-20T05:43:32.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f53237fe640 1 -- 172.21.15.32:0/2138557059 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f53100acb10 con 0x7f53240b0760 2023-12-20T05:43:32.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f532afce640 1 -- 172.21.15.32:0/2138557059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53240b0760 msgr2=0x7f53240b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:32.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f532afce640 1 --2- 172.21.15.32:0/2138557059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53240b0760 0x7f53240b0bc0 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f53100097c0 tx=0x7f531001c3c0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f532afce640 1 -- 172.21.15.32:0/2138557059 shutdown_connections 2023-12-20T05:43:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f532afce640 1 --2- 172.21.15.32:0/2138557059 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53240b0760 0x7f53240b0bc0 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f532afce640 1 --2- 172.21.15.32:0/2138557059 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53240cc4b0 0x7f53240cc920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f532afce640 1 -- 172.21.15.32:0/2138557059 >> 172.21.15.32:0/2138557059 conn(0x7f532406a510 msgr2=0x7f53240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f532afce640 1 -- 172.21.15.32:0/2138557059 shutdown_connections 2023-12-20T05:43:32.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.084+0000 7f532afce640 1 -- 172.21.15.32:0/2138557059 wait complete. 2023-12-20T05:43:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532afce640 1 Processor -- start 2023-12-20T05:43:32.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532afce640 1 -- start start 2023-12-20T05:43:32.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532afce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53240b0760 0x7f53240b2820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532afce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53240cc4b0 0x7f53240b38b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532afce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5324157480 0x7f5324162b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532afce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5324151e30 con 0x7f53240b0760 2023-12-20T05:43:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532afce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53240b1a60 con 0x7f53240cc4b0 2023-12-20T05:43:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532afce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53240b1d40 con 0x7f5324157480 2023-12-20T05:43:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f5328d6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53240b0760 0x7f53240b2820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f5328d6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53240b0760 0x7f53240b2820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43674/0 (socket says 172.21.15.32:43674) 2023-12-20T05:43:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f5328d6a640 1 -- 172.21.15.32:0/663401575 learned_addr learned my addr 172.21.15.32:0/663401575 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:32.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f5323fff640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53240cc4b0 0x7f53240b38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532956b640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5324157480 0x7f5324162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f5328d6a640 1 -- 172.21.15.32:0/663401575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5324157480 msgr2=0x7f5324162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f5328d6a640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5324157480 0x7f5324162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f5328d6a640 1 -- 172.21.15.32:0/663401575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53240cc4b0 msgr2=0x7f53240b38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f5328d6a640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53240cc4b0 0x7f53240b38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f5328d6a640 1 -- 172.21.15.32:0/663401575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5310009420 con 0x7f53240b0760 2023-12-20T05:43:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532956b640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5324157480 0x7f5324162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f5328d6a640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53240b0760 0x7f53240b2820 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f531800c550 tx=0x7f531800ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f53217fa640 1 -- 172.21.15.32:0/663401575 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318005cf0 con 0x7f53240b0760 2023-12-20T05:43:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f53217fa640 1 -- 172.21.15.32:0/663401575 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5318005e70 con 0x7f53240b0760 2023-12-20T05:43:32.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f53217fa640 1 -- 172.21.15.32:0/663401575 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5318007310 con 0x7f53240b0760 2023-12-20T05:43:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532afce640 1 -- 172.21.15.32:0/663401575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53240b2020 con 0x7f53240b0760 2023-12-20T05:43:32.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532afce640 1 -- 172.21.15.32:0/663401575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53240b3df0 con 0x7f53240b0760 2023-12-20T05:43:32.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.088+0000 7f532afce640 1 -- 172.21.15.32:0/663401575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f532404e000 con 0x7f53240b0760 2023-12-20T05:43:32.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.092+0000 7f53217fa640 1 -- 172.21.15.32:0/663401575 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5318007470 con 0x7f53240b0760 2023-12-20T05:43:32.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.092+0000 7f53217fa640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5308065b80 0x7f5308068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.092+0000 7f53217fa640 1 -- 172.21.15.32:0/663401575 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f53180827c0 con 0x7f53240b0760 2023-12-20T05:43:32.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.092+0000 7f5323fff640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5308065b80 0x7f5308068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.092+0000 7f5323fff640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5308065b80 0x7f5308068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f53100ae000 tx=0x7f5310008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:32.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.096+0000 7f53217fa640 1 -- 172.21.15.32:0/663401575 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5318050230 con 0x7f53240b0760 2023-12-20T05:43:32.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.252+0000 7f532afce640 1 -- 172.21.15.32:0/663401575 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f53240b24d0 con 0x7f53240b0760 2023-12-20T05:43:32.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f53217fa640 1 -- 172.21.15.32:0/663401575 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v51) v1 ==== 104+0+0 (secure 0 0 0) 0x7f5318053850 con 0x7f53240b0760 2023-12-20T05:43:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 -- 172.21.15.32:0/663401575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5308065b80 msgr2=0x7f5308068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5308065b80 0x7f5308068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f53100ae000 tx=0x7f5310008040 comp rx=0 tx=0).stop 2023-12-20T05:43:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 -- 172.21.15.32:0/663401575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53240b0760 msgr2=0x7f53240b2820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53240b0760 0x7f53240b2820 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f531800c550 tx=0x7f531800ca10 comp rx=0 tx=0).stop 2023-12-20T05:43:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 -- 172.21.15.32:0/663401575 shutdown_connections 2023-12-20T05:43:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5308065b80 0x7f5308068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5324157480 0x7f5324162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53240cc4b0 0x7f53240b38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 --2- 172.21.15.32:0/663401575 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f53240b0760 0x7f53240b2820 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 -- 172.21.15.32:0/663401575 >> 172.21.15.32:0/663401575 conn(0x7f532406a510 msgr2=0x7f53240c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 -- 172.21.15.32:0/663401575 shutdown_connections 2023-12-20T05:43:32.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.260+0000 7f532afce640 1 -- 172.21.15.32:0/663401575 wait complete. 2023-12-20T05:43:32.282 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention' 2023-12-20T05:43:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.424+0000 7f74cfc20640 1 Processor -- start 2023-12-20T05:43:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.424+0000 7f74cfc20640 1 -- start start 2023-12-20T05:43:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cfc20640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f74c80aece0 0x7f74c80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cfc20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80b0b30 0x7f74c80701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cfc20640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74c80c98c0 con 0x7f74c81599e0 2023-12-20T05:43:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cfc20640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74c80c9ba0 con 0x7f74c80aece0 2023-12-20T05:43:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cfc20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80c9e80 con 0x7f74c80b0b30 2023-12-20T05:43:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cd9bc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f74c81599e0 0x7f74c80c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32872/0 (socket says 172.21.15.32:32872) 2023-12-20T05:43:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cd9bc640 1 -- 172.21.15.32:0/838982767 learned_addr learned my addr 172.21.15.32:0/838982767 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74ce1bd640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80b0b30 0x7f74c80701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cd1bb640 1 --2- 172.21.15.32:0/838982767 >> v2:172.21.15.39:3300/0 conn(0x7f74c80aece0 0x7f74c80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1323935983 0 0) 0x7f74c80c98c0 con 0x7f74c81599e0 2023-12-20T05:43:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74b4003530 con 0x7f74c81599e0 2023-12-20T05:43:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 123491498 0 0) 0x7f74b4003530 con 0x7f74c81599e0 2023-12-20T05:43:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80b0b30 msgr2=0x7f74c80701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80b0b30 0x7f74c80701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 >> v2:172.21.15.39:3300/0 conn(0x7f74c80aece0 msgr2=0x7f74c80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 --2- 172.21.15.32:0/838982767 >> v2:172.21.15.39:3300/0 conn(0x7f74c80aece0 0x7f74c80b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c80ca160 con 0x7f74c81599e0 2023-12-20T05:43:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f74c4001800 con 0x7f74c81599e0 2023-12-20T05:43:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 >> v1:172.21.15.32:6789/0 conn(0x7f74c81599e0 legacy=0x7f74c80c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80b0b30 0x7f74b4003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74b4003f00 0x7f74b40047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b4004d10 0x7f74b4005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80c9ba0 con 0x7f74b4004d10 2023-12-20T05:43:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80c9e80 con 0x7f74b4003f00 2023-12-20T05:43:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74b400b990 con 0x7f74c80b0b30 2023-12-20T05:43:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cd9bc640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80b0b30 0x7f74b4003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74cd1bb640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74b4003f00 0x7f74b40047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74ce1bd640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b4004d10 0x7f74b4005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74ce1bd640 1 -- 172.21.15.32:0/838982767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80b0b30 msgr2=0x7f74b4003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74ce1bd640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c80b0b30 0x7f74b4003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74ce1bd640 1 -- 172.21.15.32:0/838982767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74b4003f00 msgr2=0x7f74b40047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74ce1bd640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74b4003f00 0x7f74b40047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.428+0000 7f74ce1bd640 1 -- 172.21.15.32:0/838982767 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74b4003530 con 0x7f74b4004d10 2023-12-20T05:43:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.432+0000 7f74ce1bd640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b4004d10 0x7f74b4005190 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009fc0 tx=0x7f74c001cc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d851ba565ddfc6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.432+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00a5c90 con 0x7f74b4004d10 2023-12-20T05:43:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.432+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74c000f040 con 0x7f74b4004d10 2023-12-20T05:43:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.432+0000 7f74cc9ba640 1 -- 172.21.15.32:0/838982767 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74c00ad440 con 0x7f74b4004d10 2023-12-20T05:43:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.432+0000 7f74cfc20640 1 -- 172.21.15.32:0/838982767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b4004d10 msgr2=0x7f74b4005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.432+0000 7f74cfc20640 1 --2- 172.21.15.32:0/838982767 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74b4004d10 0x7f74b4005190 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f74c0009fc0 tx=0x7f74c001cc20 comp rx=0 tx=0).stop 2023-12-20T05:43:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.432+0000 7f74ce1bd640 1 -- 172.21.15.32:0/838982767 reap_dead start 2023-12-20T05:43:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.432+0000 7f74cfc20640 1 -- 172.21.15.32:0/838982767 shutdown_connections 2023-12-20T05:43:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.432+0000 7f74cfc20640 1 -- 172.21.15.32:0/838982767 >> 172.21.15.32:0/838982767 conn(0x7f74c806a320 msgr2=0x7f74c80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.432+0000 7f74cfc20640 1 -- 172.21.15.32:0/838982767 shutdown_connections 2023-12-20T05:43:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.432+0000 7f74cfc20640 1 -- 172.21.15.32:0/838982767 wait complete. 2023-12-20T05:43:32.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cfc20640 1 Processor -- start 2023-12-20T05:43:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cfc20640 1 -- start start 2023-12-20T05:43:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cfc20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74b4004d10 0x7f74c815cad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cfc20640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80c8170 0x7f74c81e5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cfc20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c81e64f0 0x7f74c80c6310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cfc20640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81e4980 con 0x7f74c80c8170 2023-12-20T05:43:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cfc20640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74c80c85f0 con 0x7f74b4004d10 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cfc20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74c81e3930 con 0x7f74c81e64f0 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cd1bb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80c8170 0x7f74c81e5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cd1bb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80c8170 0x7f74c81e5fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43694/0 (socket says 172.21.15.32:43694) 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cd1bb640 1 -- 172.21.15.32:0/2570728800 learned_addr learned my addr 172.21.15.32:0/2570728800 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cd9bc640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74b4004d10 0x7f74c815cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74ce1bd640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c81e64f0 0x7f74c80c6310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cd1bb640 1 -- 172.21.15.32:0/2570728800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c81e64f0 msgr2=0x7f74c80c6310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cd1bb640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c81e64f0 0x7f74c80c6310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cd1bb640 1 -- 172.21.15.32:0/2570728800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74b4004d10 msgr2=0x7f74c815cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cd1bb640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74b4004d10 0x7f74c815cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cd1bb640 1 -- 172.21.15.32:0/2570728800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74c81e3c10 con 0x7f74c80c8170 2023-12-20T05:43:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cd1bb640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80c8170 0x7f74c81e5fb0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f74b8011aa0 tx=0x7f74b8011f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74be7fc640 1 -- 172.21.15.32:0/2570728800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b800fb80 con 0x7f74c80c8170 2023-12-20T05:43:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74be7fc640 1 -- 172.21.15.32:0/2570728800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74b800fce0 con 0x7f74c80c8170 2023-12-20T05:43:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74be7fc640 1 -- 172.21.15.32:0/2570728800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74b8014680 con 0x7f74c80c8170 2023-12-20T05:43:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cfc20640 1 -- 172.21.15.32:0/2570728800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74c80c5830 con 0x7f74c80c8170 2023-12-20T05:43:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cfc20640 1 -- 172.21.15.32:0/2570728800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74c80c5df0 con 0x7f74c80c8170 2023-12-20T05:43:32.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.436+0000 7f74cfc20640 1 -- 172.21.15.32:0/2570728800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74c804e000 con 0x7f74c80c8170 2023-12-20T05:43:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.440+0000 7f74be7fc640 1 -- 172.21.15.32:0/2570728800 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f74b80147e0 con 0x7f74c80c8170 2023-12-20T05:43:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.440+0000 7f74be7fc640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74ac065b80 0x7f74ac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.440+0000 7f74be7fc640 1 -- 172.21.15.32:0/2570728800 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f74b8083ad0 con 0x7f74c80c8170 2023-12-20T05:43:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.440+0000 7f74cd9bc640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74ac065b80 0x7f74ac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:32.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.440+0000 7f74cd9bc640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74ac065b80 0x7f74ac068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f74c4000f00 tx=0x7f74c4009040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:32.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.444+0000 7f74be7fc640 1 -- 172.21.15.32:0/2570728800 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f74b80514b0 con 0x7f74c80c8170 2023-12-20T05:43:32.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:32.656+0000 7f74cfc20640 1 -- 172.21.15.32:0/2570728800 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]} v 0) v1 -- 0x7f74c80cf550 con 0x7f74c80c8170 2023-12-20T05:43:33.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.188+0000 7f74be7fc640 1 -- 172.21.15.32:0/2570728800 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_for_number_of_snaps_retention"]}]=0 v2416) v1 ==== 178+0+0 (secure 0 0 0) 0x7f74b8054ad0 con 0x7f74c80c8170 2023-12-20T05:43:33.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 -- 172.21.15.32:0/2570728800 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74ac065b80 msgr2=0x7f74ac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74ac065b80 0x7f74ac068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f74c4000f00 tx=0x7f74c4009040 comp rx=0 tx=0).stop 2023-12-20T05:43:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 -- 172.21.15.32:0/2570728800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80c8170 msgr2=0x7f74c81e5fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80c8170 0x7f74c81e5fb0 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f74b8011aa0 tx=0x7f74b8011f60 comp rx=0 tx=0).stop 2023-12-20T05:43:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 -- 172.21.15.32:0/2570728800 shutdown_connections 2023-12-20T05:43:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f74ac065b80 0x7f74ac068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74c81e64f0 0x7f74c80c6310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74c80c8170 0x7f74c81e5fb0 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 --2- 172.21.15.32:0/2570728800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74b4004d10 0x7f74c815cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:33.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 -- 172.21.15.32:0/2570728800 >> 172.21.15.32:0/2570728800 conn(0x7f74c806a320 msgr2=0x7f74c80c49e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 -- 172.21.15.32:0/2570728800 shutdown_connections 2023-12-20T05:43:33.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.192+0000 7f74cfc20640 1 -- 172.21.15.32:0/2570728800 wait complete. 2023-12-20T05:43:33.215 INFO:tasks.cephfs_test_runner:Test that number of snaps retained are as per user spec. ... ok 2023-12-20T05:43:33.216 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test listing snap schedules post removal of a schedule 2023-12-20T05:43:33.216 INFO:tasks.cephfs_test_runner:test_snap_schedule_list_post_schedule_remove (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-20T05:43:33.216 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-12-20T05:43:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.360+0000 7fbaa1d52640 1 Processor -- start 2023-12-20T05:43:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.360+0000 7fbaa1d52640 1 -- start start 2023-12-20T05:43:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.360+0000 7fbaa1d52640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c06ce60 0x7fba9c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.360+0000 7fbaa1d52640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9c15bbd0 con 0x7fba9c0af5e0 2023-12-20T05:43:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.360+0000 7fbaa1d52640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9c15beb0 con 0x7fba9c15f110 2023-12-20T05:43:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.360+0000 7fbaa1d52640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c15e910 con 0x7fba9c06ce60 2023-12-20T05:43:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba9c0af5e0 0x7fba9c0af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32874/0 (socket says 172.21.15.32:32874) 2023-12-20T05:43:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9b7fe640 1 -- 172.21.15.32:0/2149417923 learned_addr learned my addr 172.21.15.32:0/2149417923 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:33.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9affd640 1 --2- 172.21.15.32:0/2149417923 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c06ce60 0x7fba9c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:33.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9a7fc640 1 -- 172.21.15.32:0/2149417923 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3546394969 0 0) 0x7fba9c15bbd0 con 0x7fba9c0af5e0 2023-12-20T05:43:33.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9a7fc640 1 -- 172.21.15.32:0/2149417923 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba84003530 con 0x7fba9c0af5e0 2023-12-20T05:43:33.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9a7fc640 1 -- 172.21.15.32:0/2149417923 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3198968943 0 0) 0x7fba9c15beb0 con 0x7fba9c15f110 2023-12-20T05:43:33.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9a7fc640 1 -- 172.21.15.32:0/2149417923 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba9c15bbd0 con 0x7fba9c15f110 2023-12-20T05:43:33.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9affd640 1 -- 172.21.15.32:0/2149417923 >> v1:172.21.15.39:6789/0 conn(0x7fba9c15f110 legacy=0x7fba9c15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:33.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9affd640 1 -- 172.21.15.32:0/2149417923 >> v1:172.21.15.32:6789/0 conn(0x7fba9c0af5e0 legacy=0x7fba9c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:33.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9affd640 1 -- 172.21.15.32:0/2149417923 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9c15ebf0 con 0x7fba9c06ce60 2023-12-20T05:43:33.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9affd640 1 --2- 172.21.15.32:0/2149417923 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c06ce60 0x7fba9c15b280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba900097f0 tx=0x7fba9001c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ddaa478cb5d9499f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:33.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.364+0000 7fba9a7fc640 1 -- 172.21.15.32:0/2149417923 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba9c15b9d0 con 0x7fba9c06ce60 2023-12-20T05:43:33.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fba9a7fc640 1 -- 172.21.15.32:0/2149417923 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba900ac630 con 0x7fba9c06ce60 2023-12-20T05:43:33.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fba9a7fc640 1 -- 172.21.15.32:0/2149417923 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba900ac9c0 con 0x7fba9c06ce60 2023-12-20T05:43:33.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 -- 172.21.15.32:0/2149417923 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c06ce60 msgr2=0x7fba9c15b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:33.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 --2- 172.21.15.32:0/2149417923 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c06ce60 0x7fba9c15b280 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fba900097f0 tx=0x7fba9001c430 comp rx=0 tx=0).stop 2023-12-20T05:43:33.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 -- 172.21.15.32:0/2149417923 shutdown_connections 2023-12-20T05:43:33.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 --2- 172.21.15.32:0/2149417923 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c06ce60 0x7fba9c15b280 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:33.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 -- 172.21.15.32:0/2149417923 >> 172.21.15.32:0/2149417923 conn(0x7fba9c06a320 msgr2=0x7fba9c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:33.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 -- 172.21.15.32:0/2149417923 shutdown_connections 2023-12-20T05:43:33.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 -- 172.21.15.32:0/2149417923 wait complete. 2023-12-20T05:43:33.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 Processor -- start 2023-12-20T05:43:33.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 -- start start 2023-12-20T05:43:33.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c06ce60 0x7fba9c0c3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:33.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c0af5e0 0x7fba9c0c7cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:33.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba9c15f110 0x7fba9c15d3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:33.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c0c8710 con 0x7fba9c0af5e0 2023-12-20T05:43:33.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c0c3d60 con 0x7fba9c15f110 2023-12-20T05:43:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fbaa1d52640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9c153d20 con 0x7fba9c06ce60 2023-12-20T05:43:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fba9affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c0af5e0 0x7fba9c0c7cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fba9affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c0af5e0 0x7fba9c0c7cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43708/0 (socket says 172.21.15.32:43708) 2023-12-20T05:43:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fba9affd640 1 -- 172.21.15.32:0/4237797321 learned_addr learned my addr 172.21.15.32:0/4237797321 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fba9b7fe640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c06ce60 0x7fba9c0c3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.368+0000 7fba9bfff640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba9c15f110 0x7fba9c15d3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba9affd640 1 -- 172.21.15.32:0/4237797321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c06ce60 msgr2=0x7fba9c0c3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba9affd640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c06ce60 0x7fba9c0c3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:33.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba9affd640 1 -- 172.21.15.32:0/4237797321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba9c15f110 msgr2=0x7fba9c15d3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba9affd640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba9c15f110 0x7fba9c15d3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba9affd640 1 -- 172.21.15.32:0/4237797321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba90009420 con 0x7fba9c0af5e0 2023-12-20T05:43:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba9affd640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c0af5e0 0x7fba9c0c7cc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fba900097f0 tx=0x7fba900191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba83fff640 1 -- 172.21.15.32:0/4237797321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba900ace30 con 0x7fba9c0af5e0 2023-12-20T05:43:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba83fff640 1 -- 172.21.15.32:0/4237797321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba900b5620 con 0x7fba9c0af5e0 2023-12-20T05:43:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba83fff640 1 -- 172.21.15.32:0/4237797321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba900b59d0 con 0x7fba9c0af5e0 2023-12-20T05:43:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fbaa1d52640 1 -- 172.21.15.32:0/4237797321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba9c154000 con 0x7fba9c0af5e0 2023-12-20T05:43:33.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fbaa1d52640 1 -- 172.21.15.32:0/4237797321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba9c1545c0 con 0x7fba9c0af5e0 2023-12-20T05:43:33.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fbaa1d52640 1 -- 172.21.15.32:0/4237797321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba9c04e000 con 0x7fba9c0af5e0 2023-12-20T05:43:33.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba83fff640 1 -- 172.21.15.32:0/4237797321 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fba900b3040 con 0x7fba9c0af5e0 2023-12-20T05:43:33.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba83fff640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba88065b80 0x7fba88068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:33.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba83fff640 1 -- 172.21.15.32:0/4237797321 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fba9011d4c0 con 0x7fba9c0af5e0 2023-12-20T05:43:33.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.372+0000 7fba9b7fe640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba88065b80 0x7fba88068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:33.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.376+0000 7fba9b7fe640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba88065b80 0x7fba88068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fba8c001010 tx=0x7fba8c009040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:33.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.376+0000 7fba83fff640 1 -- 172.21.15.32:0/4237797321 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fba900eae30 con 0x7fba9c0af5e0 2023-12-20T05:43:33.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:33.592+0000 7fbaa1d52640 1 -- 172.21.15.32:0/4237797321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7fba9c0cf450 con 0x7fba9c0af5e0 2023-12-20T05:43:34.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.196+0000 7fba83fff640 1 -- 172.21.15.32:0/4237797321 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2417) v1 ==== 177+0+0 (secure 0 0 0) 0x7fba900ee450 con 0x7fba9c0af5e0 2023-12-20T05:43:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 -- 172.21.15.32:0/4237797321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba88065b80 msgr2=0x7fba88068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba88065b80 0x7fba88068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fba8c001010 tx=0x7fba8c009040 comp rx=0 tx=0).stop 2023-12-20T05:43:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 -- 172.21.15.32:0/4237797321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c0af5e0 msgr2=0x7fba9c0c7cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c0af5e0 0x7fba9c0c7cc0 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7fba900097f0 tx=0x7fba900191d0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 -- 172.21.15.32:0/4237797321 shutdown_connections 2023-12-20T05:43:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba88065b80 0x7fba88068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba9c15f110 0x7fba9c15d3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9c0af5e0 0x7fba9c0c7cc0 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 --2- 172.21.15.32:0/4237797321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9c06ce60 0x7fba9c0c3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 -- 172.21.15.32:0/4237797321 >> 172.21.15.32:0/4237797321 conn(0x7fba9c06a320 msgr2=0x7fba9c15a2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 -- 172.21.15.32:0/4237797321 shutdown_connections 2023-12-20T05:43:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.200+0000 7fbaa1d52640 1 -- 172.21.15.32:0/4237797321 wait complete. 2023-12-20T05:43:34.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T05:43:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.372+0000 7fc14e7bc640 1 Processor -- start 2023-12-20T05:43:34.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.372+0000 7fc14e7bc640 1 -- start start 2023-12-20T05:43:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.372+0000 7fc14e7bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1481603e0 0x7fc1480ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.372+0000 7fc14e7bc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc14806c480 con 0x7fc148070490 2023-12-20T05:43:34.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.372+0000 7fc14e7bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc14806c760 con 0x7fc14806d240 2023-12-20T05:43:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.372+0000 7fc14e7bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc14806ca40 con 0x7fc1481603e0 2023-12-20T05:43:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.372+0000 7fc147fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc148070490 0x7fc14815fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32886/0 (socket says 172.21.15.32:32886) 2023-12-20T05:43:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.372+0000 7fc147fff640 1 -- 172.21.15.32:0/622911856 learned_addr learned my addr 172.21.15.32:0/622911856 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.372+0000 7fc14cd59640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1481603e0 0x7fc1480ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.372+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1915418619 0 0) 0x7fc14806c480 con 0x7fc148070490 2023-12-20T05:43:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.372+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc130003530 con 0x7fc148070490 2023-12-20T05:43:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 614961853 0 0) 0x7fc14806c760 con 0x7fc14806d240 2023-12-20T05:43:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc14806c480 con 0x7fc14806d240 2023-12-20T05:43:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 326126177 0 0) 0x7fc130003530 con 0x7fc148070490 2023-12-20T05:43:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1481603e0 msgr2=0x7fc1480ca8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1481603e0 0x7fc1480ca8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 >> v1:172.21.15.39:6789/0 conn(0x7fc14806d240 legacy=0x7fc1480ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc14806cd20 con 0x7fc148070490 2023-12-20T05:43:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fc138001680 con 0x7fc148070490 2023-12-20T05:43:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 >> v1:172.21.15.32:6789/0 conn(0x7fc148070490 legacy=0x7fc14815fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 0x7fc130003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1481603e0 0x7fc1300047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc130004d10 0x7fc130005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc14806ca40 con 0x7fc130003b40 2023-12-20T05:43:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc13000ba40 con 0x7fc130004d10 2023-12-20T05:43:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc13000bbb0 con 0x7fc1481603e0 2023-12-20T05:43:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc147fff640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 0x7fc130003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc14cd59640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc130004d10 0x7fc130005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc1477fe640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1481603e0 0x7fc1300047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc147fff640 1 -- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1481603e0 msgr2=0x7fc1300047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc147fff640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1481603e0 0x7fc1300047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc147fff640 1 -- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc130004d10 msgr2=0x7fc130005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc147fff640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc130004d10 0x7fc130005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc147fff640 1 -- 172.21.15.32:0/622911856 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc130003530 con 0x7fc130003b40 2023-12-20T05:43:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.376+0000 7fc14cd59640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc130004d10 0x7fc130005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc147fff640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 0x7fc130003f40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c160 tx=0x7fc13801cb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f079f5a26bf6f147 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1380058e0 con 0x7fc130003b40 2023-12-20T05:43:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc138005a60 con 0x7fc130003b40 2023-12-20T05:43:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc146ffd640 1 -- 172.21.15.32:0/622911856 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138005e10 con 0x7fc130003b40 2023-12-20T05:43:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 -- 172.21.15.32:0/622911856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 msgr2=0x7fc130003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 --2- 172.21.15.32:0/622911856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 0x7fc130003f40 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7fc13800c160 tx=0x7fc13801cb00 comp rx=0 tx=0).stop 2023-12-20T05:43:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14cd59640 1 -- 172.21.15.32:0/622911856 reap_dead start 2023-12-20T05:43:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 -- 172.21.15.32:0/622911856 shutdown_connections 2023-12-20T05:43:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 -- 172.21.15.32:0/622911856 >> 172.21.15.32:0/622911856 conn(0x7fc14806a510 msgr2=0x7fc1480ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 -- 172.21.15.32:0/622911856 shutdown_connections 2023-12-20T05:43:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 -- 172.21.15.32:0/622911856 wait complete. 2023-12-20T05:43:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 Processor -- start 2023-12-20T05:43:34.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 -- start start 2023-12-20T05:43:34.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 0x7fc148151620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:34.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1480c7810 0x7fc148151e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:34.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1480c7c90 0x7fc148152390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480c6590 con 0x7fc130003b40 2023-12-20T05:43:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480c6770 con 0x7fc1480c7810 2023-12-20T05:43:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.380+0000 7fc14e7bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc1480c6a50 con 0x7fc1480c7c90 2023-12-20T05:43:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc147fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 0x7fc148151620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc147fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 0x7fc148151620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43718/0 (socket says 172.21.15.32:43718) 2023-12-20T05:43:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc147fff640 1 -- 172.21.15.32:0/3450732406 learned_addr learned my addr 172.21.15.32:0/3450732406 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc1477fe640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1480c7810 0x7fc148151e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc147fff640 1 -- 172.21.15.32:0/3450732406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1480c7c90 msgr2=0x7fc148152390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc14cd59640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1480c7c90 0x7fc148152390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc147fff640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1480c7c90 0x7fc148152390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc147fff640 1 -- 172.21.15.32:0/3450732406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1480c7810 msgr2=0x7fc148151e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc147fff640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1480c7810 0x7fc148151e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc147fff640 1 -- 172.21.15.32:0/3450732406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1480c6d30 con 0x7fc130003b40 2023-12-20T05:43:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc14cd59640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1480c7c90 0x7fc148152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc1477fe640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1480c7810 0x7fc148151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc147fff640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 0x7fc148151620 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc1380ab040 tx=0x7fc130005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc144ff9640 1 -- 172.21.15.32:0/3450732406 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc138001be0 con 0x7fc130003b40 2023-12-20T05:43:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc144ff9640 1 -- 172.21.15.32:0/3450732406 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc138001480 con 0x7fc130003b40 2023-12-20T05:43:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc144ff9640 1 -- 172.21.15.32:0/3450732406 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1380a5a20 con 0x7fc130003b40 2023-12-20T05:43:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc14e7bc640 1 -- 172.21.15.32:0/3450732406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc1480c7010 con 0x7fc130003b40 2023-12-20T05:43:34.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc14e7bc640 1 -- 172.21.15.32:0/3450732406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc14815b9f0 con 0x7fc130003b40 2023-12-20T05:43:34.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc14e7bc640 1 -- 172.21.15.32:0/3450732406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc14804e000 con 0x7fc130003b40 2023-12-20T05:43:34.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.384+0000 7fc144ff9640 1 -- 172.21.15.32:0/3450732406 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc1380a5b80 con 0x7fc130003b40 2023-12-20T05:43:34.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.388+0000 7fc144ff9640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc134065bb0 0x7fc134068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:34.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.388+0000 7fc144ff9640 1 -- 172.21.15.32:0/3450732406 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc13811cef0 con 0x7fc130003b40 2023-12-20T05:43:34.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.388+0000 7fc1477fe640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc134065bb0 0x7fc134068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:34.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.388+0000 7fc1477fe640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc134065bb0 0x7fc134068070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc13c007af0 tx=0x7fc13c009040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:34.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.392+0000 7fc144ff9640 1 -- 172.21.15.32:0/3450732406 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc1380ea8b0 con 0x7fc130003b40 2023-12-20T05:43:34.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.548+0000 7fc14e7bc640 1 -- 172.21.15.32:0/3450732406 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc1480cf4d0 con 0x7fc130003b40 2023-12-20T05:43:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.552+0000 7fc144ff9640 1 -- 172.21.15.32:0/3450732406 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v52)=0 v52) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc1380eded0 con 0x7fc130003b40 2023-12-20T05:43:34.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 -- 172.21.15.32:0/3450732406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc134065bb0 msgr2=0x7fc134068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:34.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc134065bb0 0x7fc134068070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fc13c007af0 tx=0x7fc13c009040 comp rx=0 tx=0).stop 2023-12-20T05:43:34.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 -- 172.21.15.32:0/3450732406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 msgr2=0x7fc148151620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:34.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 0x7fc148151620 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7fc1380ab040 tx=0x7fc130005800 comp rx=0 tx=0).stop 2023-12-20T05:43:34.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 -- 172.21.15.32:0/3450732406 shutdown_connections 2023-12-20T05:43:34.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc134065bb0 0x7fc134068070 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1480c7c90 0x7fc148152390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1480c7810 0x7fc148151e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 --2- 172.21.15.32:0/3450732406 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc130003b40 0x7fc148151620 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:34.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 -- 172.21.15.32:0/3450732406 >> 172.21.15.32:0/3450732406 conn(0x7fc14806a510 msgr2=0x7fc1480b2c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:34.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 -- 172.21.15.32:0/3450732406 shutdown_connections 2023-12-20T05:43:34.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:34.556+0000 7fc14e7bc640 1 -- 172.21.15.32:0/3450732406 wait complete. 2023-12-20T05:43:34.577 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T05:43:34.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T05:43:34.650 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T05:43:34.764 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T05:43:34.765 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:43:34.765 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:43:34.775 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T05:43:34.775 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T05:43:34.775 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:43:34.786 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T05:43:34.786 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:43:34.786 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:43:34.828 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T05:43:34.828 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T05:43:34.828 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:43:34.843 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T05:43:34.843 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T05:43:34.843 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:43:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.012+0000 7f483190a640 1 Processor -- start 2023-12-20T05:43:35.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.012+0000 7f483190a640 1 -- start start 2023-12-20T05:43:35.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f483190a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f482c1599e0 0x7f482c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:35.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f483190a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0c98c0 con 0x7f482c0c90e0 2023-12-20T05:43:35.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f483190a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f482c0c9ba0 con 0x7f482c0b0d00 2023-12-20T05:43:35.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f483190a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f482c0c9e80 con 0x7f482c1599e0 2023-12-20T05:43:35.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f4823fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f482c0c90e0 0x7f482c0b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:32888/0 (socket says 172.21.15.32:32888) 2023-12-20T05:43:35.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f4823fff640 1 -- 172.21.15.32:0/2277812841 learned_addr learned my addr 172.21.15.32:0/2277812841 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:35.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482affd640 1 --2- 172.21.15.32:0/2277812841 >> v2:172.21.15.39:3300/0 conn(0x7f482c1599e0 0x7f482c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:35.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2984944541 0 0) 0x7f482c0c98c0 con 0x7f482c0c90e0 2023-12-20T05:43:35.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4818003530 con 0x7f482c0c90e0 2023-12-20T05:43:35.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4062663415 0 0) 0x7f4818003530 con 0x7f482c0c90e0 2023-12-20T05:43:35.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 >> v2:172.21.15.39:3300/0 conn(0x7f482c1599e0 msgr2=0x7f482c0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:35.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 --2- 172.21.15.32:0/2277812841 >> v2:172.21.15.39:3300/0 conn(0x7f482c1599e0 0x7f482c0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:35.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 >> v1:172.21.15.39:6789/0 conn(0x7f482c0b0d00 legacy=0x7f482c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:35.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0ca160 con 0x7f482c0c90e0 2023-12-20T05:43:35.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f4814001680 con 0x7f482c0c90e0 2023-12-20T05:43:35.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 >> v1:172.21.15.32:6789/0 conn(0x7f482c0c90e0 legacy=0x7f482c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:35.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4818003980 0x7f4818003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:35.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48180047d0 0x7f4818004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:35.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4818005190 0x7f481800dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:35.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0c9e80 con 0x7f4818005190 2023-12-20T05:43:35.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f481800e240 con 0x7f48180047d0 2023-12-20T05:43:35.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f481800e400 con 0x7f4818003980 2023-12-20T05:43:35.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482b7fe640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4818005190 0x7f481800dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:35.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482affd640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4818003980 0x7f4818003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:35.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f4823fff640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48180047d0 0x7f4818004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:35.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482b7fe640 1 -- 172.21.15.32:0/2277812841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4818003980 msgr2=0x7f4818003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:35.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482b7fe640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4818003980 0x7f4818003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:35.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482b7fe640 1 -- 172.21.15.32:0/2277812841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48180047d0 msgr2=0x7f4818004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:35.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482b7fe640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48180047d0 0x7f4818004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:35.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f482b7fe640 1 -- 172.21.15.32:0/2277812841 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4818003530 con 0x7f4818005190 2023-12-20T05:43:35.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.016+0000 7f4823fff640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48180047d0 0x7f4818004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:35.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f482b7fe640 1 -- 172.21.15.32:0/2277812841 reap_dead start 2023-12-20T05:43:35.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f482b7fe640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4818005190 0x7f481800dba0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4824001e70 tx=0x7f482401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=73567436947739ce server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:35.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824003b80 con 0x7f4818005190 2023-12-20T05:43:35.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824003d20 con 0x7f4818005190 2023-12-20T05:43:35.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f482a7fc640 1 -- 172.21.15.32:0/2277812841 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824005260 con 0x7f4818005190 2023-12-20T05:43:35.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f483190a640 1 -- 172.21.15.32:0/2277812841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4818005190 msgr2=0x7f481800dba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f483190a640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4818005190 0x7f481800dba0 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f4824001e70 tx=0x7f482401c760 comp rx=0 tx=0).stop 2023-12-20T05:43:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f483190a640 1 -- 172.21.15.32:0/2277812841 shutdown_connections 2023-12-20T05:43:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f483190a640 1 --2- 172.21.15.32:0/2277812841 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4818005190 0x7f481800dba0 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f483190a640 1 -- 172.21.15.32:0/2277812841 >> 172.21.15.32:0/2277812841 conn(0x7f482c06a320 msgr2=0x7f482c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f483190a640 1 -- 172.21.15.32:0/2277812841 shutdown_connections 2023-12-20T05:43:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.020+0000 7f483190a640 1 -- 172.21.15.32:0/2277812841 wait complete. 2023-12-20T05:43:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f483190a640 1 Processor -- start 2023-12-20T05:43:35.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f483190a640 1 -- start start 2023-12-20T05:43:35.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f483190a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4818005190 0x7f482c1e59c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:35.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f483190a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f482c1485e0 0x7f482c0ade00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:35.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f483190a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c148a60 0x7f482c0ae340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:35.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f483190a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0ae880 con 0x7f482c148a60 2023-12-20T05:43:35.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f483190a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0ad0f0 con 0x7f4818005190 2023-12-20T05:43:35.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f483190a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482c0ac6c0 con 0x7f482c1485e0 2023-12-20T05:43:35.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f482b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c148a60 0x7f482c0ae340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:35.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f482b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c148a60 0x7f482c0ae340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43742/0 (socket says 172.21.15.32:43742) 2023-12-20T05:43:35.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f482b7fe640 1 -- 172.21.15.32:0/2817778694 learned_addr learned my addr 172.21.15.32:0/2817778694 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:35.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f482affd640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4818005190 0x7f482c1e59c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:35.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f4823fff640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f482c1485e0 0x7f482c0ade00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:35.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f482b7fe640 1 -- 172.21.15.32:0/2817778694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f482c1485e0 msgr2=0x7f482c0ade00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:35.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f482b7fe640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f482c1485e0 0x7f482c0ade00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f482b7fe640 1 -- 172.21.15.32:0/2817778694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4818005190 msgr2=0x7f482c1e59c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f482b7fe640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4818005190 0x7f482c1e59c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f482b7fe640 1 -- 172.21.15.32:0/2817778694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f482c0ad340 con 0x7f482c148a60 2023-12-20T05:43:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f482b7fe640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c148a60 0x7f482c0ae340 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f482401cc40 tx=0x7f4818005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f4828ff9640 1 -- 172.21.15.32:0/2817778694 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48240ac070 con 0x7f482c148a60 2023-12-20T05:43:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f4828ff9640 1 -- 172.21.15.32:0/2817778694 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4824007dc0 con 0x7f482c148a60 2023-12-20T05:43:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f4828ff9640 1 -- 172.21.15.32:0/2817778694 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4824006d60 con 0x7f482c148a60 2023-12-20T05:43:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f483190a640 1 -- 172.21.15.32:0/2817778694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f482c0ad620 con 0x7f482c148a60 2023-12-20T05:43:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f483190a640 1 -- 172.21.15.32:0/2817778694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f482c1e5f00 con 0x7f482c148a60 2023-12-20T05:43:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.024+0000 7f483190a640 1 -- 172.21.15.32:0/2817778694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f482c04e000 con 0x7f482c148a60 2023-12-20T05:43:35.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.028+0000 7f4828ff9640 1 -- 172.21.15.32:0/2817778694 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4824007030 con 0x7f482c148a60 2023-12-20T05:43:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.028+0000 7f4828ff9640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f480c065bb0 0x7f480c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:35.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.028+0000 7f4828ff9640 1 -- 172.21.15.32:0/2817778694 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(150..150 src has 1..150) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f482411bff0 con 0x7f482c148a60 2023-12-20T05:43:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.028+0000 7f482affd640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f480c065bb0 0x7f480c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:35.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.028+0000 7f482affd640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f480c065bb0 0x7f480c068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f481c009b40 tx=0x7f481c006f90 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:35.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.032+0000 7f4828ff9640 1 -- 172.21.15.32:0/2817778694 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f48240e98b0 con 0x7f482c148a60 2023-12-20T05:43:35.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 -- 172.21.15.32:0/2817778694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f482c0cf550 con 0x7f482c148a60 2023-12-20T05:43:35.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f4828ff9640 1 -- 172.21.15.32:0/2817778694 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 64 v64) v1 ==== 94+0+3526 (secure 0 0 0) 0x7f48240b5020 con 0x7f482c148a60 2023-12-20T05:43:35.229 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:35.230 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":64,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":7565,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3433304126","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3433304126},{"type":"v1","addr":"172.21.15.32:6837","nonce":3433304126}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":64},{"gid":8164,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3306270605","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3306270605},{"type":"v1","addr":"172.21.15.39:6834","nonce":3306270605}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8167,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4000562557","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":4000562557},{"type":"v1","addr":"172.21.15.39:6835","nonce":4000562557}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57},{"gid":8394,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/310576469","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":310576469},{"type":"v1","addr":"172.21.15.32:6835","nonce":310576469}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":57}],"filesystems":[]} 2023-12-20T05:43:35.230 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 64 2023-12-20T05:43:35.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 -- 172.21.15.32:0/2817778694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f480c065bb0 msgr2=0x7f480c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:35.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f480c065bb0 0x7f480c068070 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f481c009b40 tx=0x7f481c006f90 comp rx=0 tx=0).stop 2023-12-20T05:43:35.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 -- 172.21.15.32:0/2817778694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c148a60 msgr2=0x7f482c0ae340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:35.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c148a60 0x7f482c0ae340 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f482401cc40 tx=0x7f4818005800 comp rx=0 tx=0).stop 2023-12-20T05:43:35.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 -- 172.21.15.32:0/2817778694 shutdown_connections 2023-12-20T05:43:35.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f480c065bb0 0x7f480c068070 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:35.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f482c148a60 0x7f482c0ae340 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:35.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f482c1485e0 0x7f482c0ade00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:35.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 --2- 172.21.15.32:0/2817778694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4818005190 0x7f482c1e59c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:35.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 -- 172.21.15.32:0/2817778694 >> 172.21.15.32:0/2817778694 conn(0x7f482c06a320 msgr2=0x7f482c158d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:35.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 -- 172.21.15.32:0/2817778694 shutdown_connections 2023-12-20T05:43:35.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:35.228+0000 7f483190a640 1 -- 172.21.15.32:0/2817778694 wait complete. 2023-12-20T05:43:35.251 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T05:43:35.251 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T05:43:35.252 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T05:43:35.252 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T05:43:35.252 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T05:43:35.252 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T05:43:35.253 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T05:43:35.253 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T05:43:35.253 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T05:43:35.253 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T05:43:35.253 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T05:43:35.254 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T05:43:35.254 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T05:43:35.254 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T05:43:35.254 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T05:43:35.254 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T05:43:35.268 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T05:43:35.268 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T05:43:35.272 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T05:43:35.272 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T05:43:35.274 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T05:43:35.274 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T05:43:35.318 INFO:tasks.ceph.mds.c:Started 2023-12-20T05:43:35.319 DEBUG:teuthology.parallel:result is None 2023-12-20T05:43:35.324 INFO:tasks.ceph.mds.b:Started 2023-12-20T05:43:35.325 DEBUG:teuthology.parallel:result is None 2023-12-20T05:43:35.327 INFO:tasks.ceph.mds.d:Started 2023-12-20T05:43:35.327 DEBUG:teuthology.parallel:result is None 2023-12-20T05:43:35.406 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T05:43:35.422 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T05:43:35.424 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T05:43:39.885 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:43:39.890 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:43:41.267 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T05:43:41.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T05:43:41.272 INFO:tasks.ceph.mds.a:Started 2023-12-20T05:43:41.273 DEBUG:teuthology.parallel:result is None 2023-12-20T05:43:41.273 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T05:43:41.370 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T05:43:41.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.432+0000 7f22a759e640 1 Processor -- start 2023-12-20T05:43:41.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.432+0000 7f22a759e640 1 -- start start 2023-12-20T05:43:41.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.432+0000 7f22a759e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f22a8159ab0 0x7f22a80714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:41.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.432+0000 7f22a759e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a80cd2e0 0x7f22a8071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:41.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.432+0000 7f22a759e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f22a8071f40 con 0x7f22a80b3e90 2023-12-20T05:43:41.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.432+0000 7f22a759e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22a80ca0b0 con 0x7f22a8159ab0 2023-12-20T05:43:41.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.432+0000 7f22a759e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80ad7e0 con 0x7f22a80cd2e0 2023-12-20T05:43:41.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.432+0000 7f229ffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a80cd2e0 0x7f22a8071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:41.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.432+0000 7f229ffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a80cd2e0 0x7f22a8071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:46544/0 (socket says 172.21.15.32:46544) 2023-12-20T05:43:41.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.432+0000 7f229ffff640 1 -- 172.21.15.32:0/1993479877 learned_addr learned my addr 172.21.15.32:0/1993479877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:41.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.436+0000 7f22a659c640 1 --2- 172.21.15.32:0/1993479877 >> v2:172.21.15.39:3300/0 conn(0x7f22a8159ab0 0x7f22a80714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:41.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.436+0000 7f229ffff640 1 -- 172.21.15.32:0/1993479877 >> v2:172.21.15.39:3300/0 conn(0x7f22a8159ab0 msgr2=0x7f22a80714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:41.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.436+0000 7f229ffff640 1 --2- 172.21.15.32:0/1993479877 >> v2:172.21.15.39:3300/0 conn(0x7f22a8159ab0 0x7f22a80714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:41.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.436+0000 7f229ffff640 1 -- 172.21.15.32:0/1993479877 >> v1:172.21.15.39:6789/0 conn(0x7f22a80b3e90 legacy=0x7f22a80cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:41.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.436+0000 7f229ffff640 1 -- 172.21.15.32:0/1993479877 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22a80ada80 con 0x7f22a80cd2e0 2023-12-20T05:43:41.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.436+0000 7f22a5d9b640 1 -- 172.21.15.32:0/1993479877 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4214528946 0 0) 0x7f22a8071f40 con 0x7f22a80b3e90 2023-12-20T05:43:41.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.436+0000 7f229ffff640 1 --2- 172.21.15.32:0/1993479877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a80cd2e0 0x7f22a8071a00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2290009870 tx=0x7f229001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=de74f167a55974bd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:41.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.436+0000 7f22a5d9b640 1 -- 172.21.15.32:0/1993479877 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22900a7070 con 0x7f22a80cd2e0 2023-12-20T05:43:41.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.436+0000 7f22a5d9b640 1 -- 172.21.15.32:0/1993479877 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22900ab430 con 0x7f22a80cd2e0 2023-12-20T05:43:41.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.436+0000 7f22a5d9b640 1 -- 172.21.15.32:0/1993479877 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22900ab7c0 con 0x7f22a80cd2e0 2023-12-20T05:43:41.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 -- 172.21.15.32:0/1993479877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a80cd2e0 msgr2=0x7f22a8071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:41.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 --2- 172.21.15.32:0/1993479877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a80cd2e0 0x7f22a8071a00 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f2290009870 tx=0x7f229001c510 comp rx=0 tx=0).stop 2023-12-20T05:43:41.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 -- 172.21.15.32:0/1993479877 shutdown_connections 2023-12-20T05:43:41.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 --2- 172.21.15.32:0/1993479877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a80cd2e0 0x7f22a8071a00 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:41.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 --2- 172.21.15.32:0/1993479877 >> v2:172.21.15.39:3300/0 conn(0x7f22a8159ab0 0x7f22a80714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:41.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 -- 172.21.15.32:0/1993479877 >> 172.21.15.32:0/1993479877 conn(0x7f22a806c060 msgr2=0x7f22a806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:41.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 -- 172.21.15.32:0/1993479877 shutdown_connections 2023-12-20T05:43:41.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 -- 172.21.15.32:0/1993479877 wait complete. 2023-12-20T05:43:41.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 Processor -- start 2023-12-20T05:43:41.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 -- start start 2023-12-20T05:43:41.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80b3e90 0x7f22a80c5810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80cd2e0 0x7f22a8148910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a8159ab0 0x7f22a80c7180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80ae4e0 con 0x7f22a80b3e90 2023-12-20T05:43:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80aeaa0 con 0x7f22a80cd2e0 2023-12-20T05:43:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a759e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22a80c5d50 con 0x7f22a8159ab0 2023-12-20T05:43:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80b3e90 0x7f22a80c5810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80b3e90 0x7f22a80c5810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60792/0 (socket says 172.21.15.32:60792) 2023-12-20T05:43:41.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a659c640 1 -- 172.21.15.32:0/1037415515 learned_addr learned my addr 172.21.15.32:0/1037415515 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f229ffff640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80cd2e0 0x7f22a8148910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a6d9d640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a8159ab0 0x7f22a80c7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a659c640 1 -- 172.21.15.32:0/1037415515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a8159ab0 msgr2=0x7f22a80c7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a659c640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a8159ab0 0x7f22a80c7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a659c640 1 -- 172.21.15.32:0/1037415515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80cd2e0 msgr2=0x7f22a8148910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a659c640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80cd2e0 0x7f22a8148910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a659c640 1 -- 172.21.15.32:0/1037415515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22900094d0 con 0x7f22a80b3e90 2023-12-20T05:43:41.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.440+0000 7f22a6d9d640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a8159ab0 0x7f22a80c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.444+0000 7f22a659c640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80b3e90 0x7f22a80c5810 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f229800e8a0 tx=0x7f229800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.444+0000 7f229effd640 1 -- 172.21.15.32:0/1037415515 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2298009510 con 0x7f22a80b3e90 2023-12-20T05:43:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.444+0000 7f229effd640 1 -- 172.21.15.32:0/1037415515 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f229800c940 con 0x7f22a80b3e90 2023-12-20T05:43:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.444+0000 7f229effd640 1 -- 172.21.15.32:0/1037415515 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f229800ccb0 con 0x7f22a80b3e90 2023-12-20T05:43:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.444+0000 7f22a759e640 1 -- 172.21.15.32:0/1037415515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22a815c7c0 con 0x7f22a80b3e90 2023-12-20T05:43:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.444+0000 7f22a759e640 1 -- 172.21.15.32:0/1037415515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f22a80ae730 con 0x7f22a80b3e90 2023-12-20T05:43:41.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.444+0000 7f22a759e640 1 -- 172.21.15.32:0/1037415515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f22a8074100 con 0x7f22a80b3e90 2023-12-20T05:43:41.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.444+0000 7f229effd640 1 -- 172.21.15.32:0/1037415515 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2298013470 con 0x7f22a80b3e90 2023-12-20T05:43:41.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.444+0000 7f229effd640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2288065b80 0x7f2288068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:41.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.444+0000 7f229effd640 1 -- 172.21.15.32:0/1037415515 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(154..154 src has 1..154) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2298083300 con 0x7f22a80b3e90 2023-12-20T05:43:41.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.444+0000 7f229ffff640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2288065b80 0x7f2288068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:41.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.448+0000 7f229ffff640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2288065b80 0x7f2288068040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f22900af000 tx=0x7f2290008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:41.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.448+0000 7f229effd640 1 -- 172.21.15.32:0/1037415515 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2298050cd0 con 0x7f22a80b3e90 2023-12-20T05:43:41.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:41.624+0000 7f22a759e640 1 -- 172.21.15.32:0/1037415515 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f22a8070dc0 con 0x7f22a80b3e90 2023-12-20T05:43:42.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.396+0000 7f229effd640 1 -- 172.21.15.32:0/1037415515 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v155) v1 ==== 97+0+0 (secure 0 0 0) 0x7f22980542f0 con 0x7f22a80b3e90 2023-12-20T05:43:42.398 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T05:43:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 -- 172.21.15.32:0/1037415515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2288065b80 msgr2=0x7f2288068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2288065b80 0x7f2288068040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f22900af000 tx=0x7f2290008040 comp rx=0 tx=0).stop 2023-12-20T05:43:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 -- 172.21.15.32:0/1037415515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80b3e90 msgr2=0x7f22a80c5810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80b3e90 0x7f22a80c5810 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7f229800e8a0 tx=0x7f229800eca0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 -- 172.21.15.32:0/1037415515 shutdown_connections 2023-12-20T05:43:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2288065b80 0x7f2288068040 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22a8159ab0 0x7f22a80c7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f22a80cd2e0 0x7f22a8148910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 --2- 172.21.15.32:0/1037415515 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f22a80b3e90 0x7f22a80c5810 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 -- 172.21.15.32:0/1037415515 >> 172.21.15.32:0/1037415515 conn(0x7f22a806c060 msgr2=0x7f22a80cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 -- 172.21.15.32:0/1037415515 shutdown_connections 2023-12-20T05:43:42.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.400+0000 7f22a759e640 1 -- 172.21.15.32:0/1037415515 wait complete. 2023-12-20T05:43:42.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T05:43:42.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b0df6d640 1 Processor -- start 2023-12-20T05:43:42.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b0df6d640 1 -- start start 2023-12-20T05:43:42.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b0df6d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7b0815cdb0 0x7f7b080d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b0df6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b080cfc80 0x7f7b080b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b0df6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7b0815d190 con 0x7f7b080d2740 2023-12-20T05:43:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b0df6d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7b080d16a0 con 0x7f7b0815cdb0 2023-12-20T05:43:42.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b0df6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080d1980 con 0x7f7b080cfc80 2023-12-20T05:43:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b077fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7b0815cdb0 0x7f7b080d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b080cfc80 0x7f7b080b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b06ffd640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7b080d2740 0x7f7b080cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:36826/0 (socket says 172.21.15.32:36826) 2023-12-20T05:43:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b06ffd640 1 -- 172.21.15.32:0/388453770 learned_addr learned my addr 172.21.15.32:0/388453770 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b077fe640 1 -- 172.21.15.32:0/388453770 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b080cfc80 msgr2=0x7f7b080b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b077fe640 1 --2- 172.21.15.32:0/388453770 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b080cfc80 0x7f7b080b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b077fe640 1 -- 172.21.15.32:0/388453770 >> v1:172.21.15.39:6789/0 conn(0x7f7b080d2740 legacy=0x7f7b080cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b077fe640 1 -- 172.21.15.32:0/388453770 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b080d1c60 con 0x7f7b0815cdb0 2023-12-20T05:43:42.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b067fc640 1 -- 172.21.15.32:0/388453770 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3304647160 0 0) 0x7f7b0815d190 con 0x7f7b080d2740 2023-12-20T05:43:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.568+0000 7f7b07fff640 1 --2- 172.21.15.32:0/388453770 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b080cfc80 0x7f7b080b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:43:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b077fe640 1 --2- 172.21.15.32:0/388453770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0815cdb0 0x7f7b080d0be0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7af80097c0 tx=0x7f7af801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b56aba9d0e5b0fa9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b067fc640 1 -- 172.21.15.32:0/388453770 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af80a8970 con 0x7f7b0815cdb0 2023-12-20T05:43:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b067fc640 1 -- 172.21.15.32:0/388453770 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7af80a8ad0 con 0x7f7b0815cdb0 2023-12-20T05:43:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b067fc640 1 -- 172.21.15.32:0/388453770 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7af80a8e60 con 0x7f7b0815cdb0 2023-12-20T05:43:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b0df6d640 1 -- 172.21.15.32:0/388453770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0815cdb0 msgr2=0x7f7b080d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b0df6d640 1 --2- 172.21.15.32:0/388453770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0815cdb0 0x7f7b080d0be0 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f7af80097c0 tx=0x7f7af801ec70 comp rx=0 tx=0).stop 2023-12-20T05:43:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b0df6d640 1 -- 172.21.15.32:0/388453770 shutdown_connections 2023-12-20T05:43:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b0df6d640 1 --2- 172.21.15.32:0/388453770 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b080cfc80 0x7f7b080b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b0df6d640 1 --2- 172.21.15.32:0/388453770 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b0815cdb0 0x7f7b080d0be0 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b0df6d640 1 -- 172.21.15.32:0/388453770 >> 172.21.15.32:0/388453770 conn(0x7f7b08065ee0 msgr2=0x7f7b0806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b0df6d640 1 -- 172.21.15.32:0/388453770 shutdown_connections 2023-12-20T05:43:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.572+0000 7f7b0df6d640 1 -- 172.21.15.32:0/388453770 wait complete. 2023-12-20T05:43:42.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b0df6d640 1 Processor -- start 2023-12-20T05:43:42.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b0df6d640 1 -- start start 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b0df6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b080cfc80 0x7f7b080c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b0df6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b080d2740 0x7f7b080cbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b0df6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b0815cdb0 0x7f7b081f4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b0df6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b081f6a30 con 0x7f7b0815cdb0 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b0df6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080ca210 con 0x7f7b080cfc80 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b0df6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7b080ca4f0 con 0x7f7b080d2740 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b0815cdb0 0x7f7b081f4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b07fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b0815cdb0 0x7f7b081f4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60800/0 (socket says 172.21.15.32:60800) 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b07fff640 1 -- 172.21.15.32:0/1036530479 learned_addr learned my addr 172.21.15.32:0/1036530479 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b077fe640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b080cfc80 0x7f7b080c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b06ffd640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b080d2740 0x7f7b080cbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b07fff640 1 -- 172.21.15.32:0/1036530479 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b080d2740 msgr2=0x7f7b080cbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b07fff640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b080d2740 0x7f7b080cbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b07fff640 1 -- 172.21.15.32:0/1036530479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b080cfc80 msgr2=0x7f7b080c9c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b07fff640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b080cfc80 0x7f7b080c9c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b07fff640 1 -- 172.21.15.32:0/1036530479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7b00009a70 con 0x7f7b0815cdb0 2023-12-20T05:43:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b07fff640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b0815cdb0 0x7f7b081f4560 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000efb0 tx=0x7f7b0000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7aeffff640 1 -- 172.21.15.32:0/1036530479 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b00009d50 con 0x7f7b0815cdb0 2023-12-20T05:43:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7aeffff640 1 -- 172.21.15.32:0/1036530479 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7b0000f040 con 0x7f7b0815cdb0 2023-12-20T05:43:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7aeffff640 1 -- 172.21.15.32:0/1036530479 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7b000146a0 con 0x7f7b0815cdb0 2023-12-20T05:43:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b0df6d640 1 -- 172.21.15.32:0/1036530479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7af8009420 con 0x7f7b0815cdb0 2023-12-20T05:43:42.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.576+0000 7f7b0df6d640 1 -- 172.21.15.32:0/1036530479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7b080caab0 con 0x7f7b0815cdb0 2023-12-20T05:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.580+0000 7f7b0df6d640 1 -- 172.21.15.32:0/1036530479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7b081f6c00 con 0x7f7b0815cdb0 2023-12-20T05:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.580+0000 7f7aeffff640 1 -- 172.21.15.32:0/1036530479 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7b00014800 con 0x7f7b0815cdb0 2023-12-20T05:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.580+0000 7f7aeffff640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7af4065bb0 0x7f7af4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.580+0000 7f7aeffff640 1 -- 172.21.15.32:0/1036530479 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f7b00084080 con 0x7f7b0815cdb0 2023-12-20T05:43:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.580+0000 7f7b077fe640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7af4065bb0 0x7f7af4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.580+0000 7f7b077fe640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7af4065bb0 0x7f7af4068070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7af80b2000 tx=0x7f7af8008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:42.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.584+0000 7f7aeffff640 1 -- 172.21.15.32:0/1036530479 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7b000b50a0 con 0x7f7b0815cdb0 2023-12-20T05:43:42.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 -- 172.21.15.32:0/1036530479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f7b0804e000 con 0x7f7b0815cdb0 2023-12-20T05:43:42.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7aeffff640 1 -- 172.21.15.32:0/1036530479 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v41) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f7b081f6c00 con 0x7f7b0815cdb0 2023-12-20T05:43:42.796 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:42.796 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T05:43:42.796 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T05:43:42.796 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.796 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T05:43:42.796 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.797 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.798 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T05:43:42.799 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.800 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T05:43:42.801 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.802 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T05:43:42.803 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.804 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T05:43:42.805 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T05:43:42.806 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T05:43:42.807 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.808 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T05:43:42.809 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T05:43:42.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 -- 172.21.15.32:0/1036530479 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7af4065bb0 msgr2=0x7f7af4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:42.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7af4065bb0 0x7f7af4068070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f7af80b2000 tx=0x7f7af8008040 comp rx=0 tx=0).stop 2023-12-20T05:43:42.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 -- 172.21.15.32:0/1036530479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b0815cdb0 msgr2=0x7f7b081f4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:42.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b0815cdb0 0x7f7b081f4560 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7f7b0000efb0 tx=0x7f7b0000c490 comp rx=0 tx=0).stop 2023-12-20T05:43:42.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 -- 172.21.15.32:0/1036530479 shutdown_connections 2023-12-20T05:43:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7af4065bb0 0x7f7af4068070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7b0815cdb0 0x7f7b081f4560 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7b080d2740 0x7f7b080cbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 --2- 172.21.15.32:0/1036530479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7b080cfc80 0x7f7b080c9c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 -- 172.21.15.32:0/1036530479 >> 172.21.15.32:0/1036530479 conn(0x7f7b08065ee0 msgr2=0x7f7b080cf570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 -- 172.21.15.32:0/1036530479 shutdown_connections 2023-12-20T05:43:42.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.796+0000 7f7b0df6d640 1 -- 172.21.15.32:0/1036530479 wait complete. 2023-12-20T05:43:42.818 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:43:42.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f954e9f0640 1 Processor -- start 2023-12-20T05:43:42.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f954e9f0640 1 -- start start 2023-12-20T05:43:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f954e9f0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f95481599e0 0x7f95480aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f954e9f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95480c90e0 0x7f95480b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f954e9f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95480711d0 con 0x7f95480b0b30 2023-12-20T05:43:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f954e9f0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95480c98c0 con 0x7f95480c90e0 2023-12-20T05:43:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f954e9f0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95480c9ba0 con 0x7f95481599e0 2023-12-20T05:43:42.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f95477fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95480c90e0 0x7f95480b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f95477fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95480c90e0 0x7f95480b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60806/0 (socket says 172.21.15.32:60806) 2023-12-20T05:43:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f95477fe640 1 -- 172.21.15.32:0/1485961173 learned_addr learned my addr 172.21.15.32:0/1485961173 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f9547fff640 1 --2- 172.21.15.32:0/1485961173 >> v2:172.21.15.39:3300/0 conn(0x7f95481599e0 0x7f95480aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f95477fe640 1 -- 172.21.15.32:0/1485961173 >> v2:172.21.15.39:3300/0 conn(0x7f95481599e0 msgr2=0x7f95480aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f95477fe640 1 --2- 172.21.15.32:0/1485961173 >> v2:172.21.15.39:3300/0 conn(0x7f95481599e0 0x7f95480aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f95477fe640 1 -- 172.21.15.32:0/1485961173 >> v1:172.21.15.39:6789/0 conn(0x7f95480b0b30 legacy=0x7f954815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f95477fe640 1 -- 172.21.15.32:0/1485961173 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95480c9e80 con 0x7f95480c90e0 2023-12-20T05:43:42.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.960+0000 7f9547fff640 1 --2- 172.21.15.32:0/1485961173 >> v2:172.21.15.39:3300/0 conn(0x7f95481599e0 0x7f95480aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:42.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f95477fe640 1 --2- 172.21.15.32:0/1485961173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95480c90e0 0x7f95480b05f0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f954806e3c0 tx=0x7f953c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c7f1900468ae7b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:42.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f9546ffd640 1 -- 172.21.15.32:0/1485961173 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0ac410 con 0x7f95480c90e0 2023-12-20T05:43:42.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f9546ffd640 1 -- 172.21.15.32:0/1485961173 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c0ac570 con 0x7f95480c90e0 2023-12-20T05:43:42.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f9546ffd640 1 -- 172.21.15.32:0/1485961173 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0ac900 con 0x7f95480c90e0 2023-12-20T05:43:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f954e9f0640 1 -- 172.21.15.32:0/1485961173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95480c90e0 msgr2=0x7f95480b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f954e9f0640 1 --2- 172.21.15.32:0/1485961173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95480c90e0 0x7f95480b05f0 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7f954806e3c0 tx=0x7f953c01c490 comp rx=0 tx=0).stop 2023-12-20T05:43:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f954e9f0640 1 -- 172.21.15.32:0/1485961173 shutdown_connections 2023-12-20T05:43:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f954e9f0640 1 --2- 172.21.15.32:0/1485961173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95480c90e0 0x7f95480b05f0 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f954e9f0640 1 --2- 172.21.15.32:0/1485961173 >> v2:172.21.15.39:3300/0 conn(0x7f95481599e0 0x7f95480aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f954e9f0640 1 -- 172.21.15.32:0/1485961173 >> 172.21.15.32:0/1485961173 conn(0x7f954806a320 msgr2=0x7f95480ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f954e9f0640 1 -- 172.21.15.32:0/1485961173 shutdown_connections 2023-12-20T05:43:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.964+0000 7f954e9f0640 1 -- 172.21.15.32:0/1485961173 wait complete. 2023-12-20T05:43:42.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f954e9f0640 1 Processor -- start 2023-12-20T05:43:42.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f954e9f0640 1 -- start start 2023-12-20T05:43:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f954e9f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95480b0b30 0x7f954814fb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f954e9f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95480c90e0 0x7f95480b1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f954e9f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95481599e0 0x7f95480b2110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f954e9f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95481641c0 con 0x7f95480c90e0 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f954e9f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95481635b0 con 0x7f95481599e0 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f954e9f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9548163890 con 0x7f95480b0b30 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f95477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95480c90e0 0x7f95480b1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f95477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95480c90e0 0x7f95480b1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60816/0 (socket says 172.21.15.32:60816) 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f95477fe640 1 -- 172.21.15.32:0/2969462794 learned_addr learned my addr 172.21.15.32:0/2969462794 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f9547fff640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95480b0b30 0x7f954814fb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f954cf8d640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95481599e0 0x7f95480b2110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f95477fe640 1 -- 172.21.15.32:0/2969462794 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95480b0b30 msgr2=0x7f954814fb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f95477fe640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95480b0b30 0x7f954814fb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f95477fe640 1 -- 172.21.15.32:0/2969462794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95481599e0 msgr2=0x7f95480b2110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:42.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f95477fe640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95481599e0 0x7f95480b2110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f95477fe640 1 -- 172.21.15.32:0/2969462794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f953c0094d0 con 0x7f95480c90e0 2023-12-20T05:43:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f9547fff640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95480b0b30 0x7f954814fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f95477fe640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95480c90e0 0x7f95480b1bd0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f953c0af040 tx=0x7f953c019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f9544ff9640 1 -- 172.21.15.32:0/2969462794 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0ac410 con 0x7f95480c90e0 2023-12-20T05:43:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f9544ff9640 1 -- 172.21.15.32:0/2969462794 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f953c0b65f0 con 0x7f95480c90e0 2023-12-20T05:43:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f9544ff9640 1 -- 172.21.15.32:0/2969462794 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f953c0b69a0 con 0x7f95480c90e0 2023-12-20T05:43:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.968+0000 7f954e9f0640 1 -- 172.21.15.32:0/2969462794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9548163b70 con 0x7f95480c90e0 2023-12-20T05:43:42.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.972+0000 7f954e9f0640 1 -- 172.21.15.32:0/2969462794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95481500b0 con 0x7f95480c90e0 2023-12-20T05:43:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.972+0000 7f954e9f0640 1 -- 172.21.15.32:0/2969462794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f954804e000 con 0x7f95480c90e0 2023-12-20T05:43:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.972+0000 7f9544ff9640 1 -- 172.21.15.32:0/2969462794 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f953c0b6b00 con 0x7f95480c90e0 2023-12-20T05:43:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.972+0000 7f9544ff9640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9534065b80 0x7f9534068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.972+0000 7f9544ff9640 1 -- 172.21.15.32:0/2969462794 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f953c11d9e0 con 0x7f95480c90e0 2023-12-20T05:43:42.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.972+0000 7f9547fff640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9534065b80 0x7f9534068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:42.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.972+0000 7f9547fff640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9534065b80 0x7f9534068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f953800a990 tx=0x7f9538008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:42.976+0000 7f9544ff9640 1 -- 172.21.15.32:0/2969462794 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f953c0eb520 con 0x7f95480c90e0 2023-12-20T05:43:43.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.144+0000 7f954e9f0640 1 -- 172.21.15.32:0/2969462794 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f95480cf550 con 0x7f95480c90e0 2023-12-20T05:43:43.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f9544ff9640 1 -- 172.21.15.32:0/2969462794 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v155) v1 ==== 74+0+12395 (secure 0 0 0) 0x7f953c0eeb40 con 0x7f95480c90e0 2023-12-20T05:43:43.148 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:43.149 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":155,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:43:42.391059+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":17,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":140,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":136,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:43:43.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 -- 172.21.15.32:0/2969462794 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9534065b80 msgr2=0x7f9534068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9534065b80 0x7f9534068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f953800a990 tx=0x7f9538008040 comp rx=0 tx=0).stop 2023-12-20T05:43:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 -- 172.21.15.32:0/2969462794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95480c90e0 msgr2=0x7f95480b1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95480c90e0 0x7f95480b1bd0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7f953c0af040 tx=0x7f953c019230 comp rx=0 tx=0).stop 2023-12-20T05:43:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 -- 172.21.15.32:0/2969462794 shutdown_connections 2023-12-20T05:43:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9534065b80 0x7f9534068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95481599e0 0x7f95480b2110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95480c90e0 0x7f95480b1bd0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 --2- 172.21.15.32:0/2969462794 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95480b0b30 0x7f954814fb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 -- 172.21.15.32:0/2969462794 >> 172.21.15.32:0/2969462794 conn(0x7f954806a320 msgr2=0x7f95481524f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 -- 172.21.15.32:0/2969462794 shutdown_connections 2023-12-20T05:43:43.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.148+0000 7f954e9f0640 1 -- 172.21.15.32:0/2969462794 wait complete. 2023-12-20T05:43:43.170 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T05:43:43.170 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T05:43:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd759af9640 1 Processor -- start 2023-12-20T05:43:43.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd759af9640 1 -- start start 2023-12-20T05:43:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd759af9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7540cb050 0x7fd7540cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd759af9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd754065d40 0x7fd754161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd759af9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd75406ca60 0x7fd75406cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd759af9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd75406d450 con 0x7fd754065d40 2023-12-20T05:43:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd759af9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd75406a1f0 con 0x7fd75406ca60 2023-12-20T05:43:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd759af9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd754161860 con 0x7fd7540cb050 2023-12-20T05:43:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd752ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd754065d40 0x7fd754161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd752ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd754065d40 0x7fd754161320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60826/0 (socket says 172.21.15.32:60826) 2023-12-20T05:43:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd752ffd640 1 -- 172.21.15.32:0/3738497647 learned_addr learned my addr 172.21.15.32:0/3738497647 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd7537fe640 1 --2- 172.21.15.32:0/3738497647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7540cb050 0x7fd7540cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd752ffd640 1 -- 172.21.15.32:0/3738497647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7540cb050 msgr2=0x7fd7540cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd753fff640 1 --2- 172.21.15.32:0/3738497647 >> v2:172.21.15.39:3300/0 conn(0x7fd75406ca60 0x7fd75406cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd752ffd640 1 --2- 172.21.15.32:0/3738497647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7540cb050 0x7fd7540cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd752ffd640 1 -- 172.21.15.32:0/3738497647 >> v2:172.21.15.39:3300/0 conn(0x7fd75406ca60 msgr2=0x7fd75406cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd752ffd640 1 --2- 172.21.15.32:0/3738497647 >> v2:172.21.15.39:3300/0 conn(0x7fd75406ca60 0x7fd75406cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd752ffd640 1 -- 172.21.15.32:0/3738497647 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd75406a440 con 0x7fd754065d40 2023-12-20T05:43:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd753fff640 1 --2- 172.21.15.32:0/3738497647 >> v2:172.21.15.39:3300/0 conn(0x7fd75406ca60 0x7fd75406cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:43.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.316+0000 7fd7537fe640 1 --2- 172.21.15.32:0/3738497647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7540cb050 0x7fd7540cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd752ffd640 1 --2- 172.21.15.32:0/3738497647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd754065d40 0x7fd754161320 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fd7540ce6c0 tx=0x7fd74801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f59e08b4e215fb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd7527fc640 1 -- 172.21.15.32:0/3738497647 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7480aa5f0 con 0x7fd754065d40 2023-12-20T05:43:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd7527fc640 1 -- 172.21.15.32:0/3738497647 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7480aa750 con 0x7fd754065d40 2023-12-20T05:43:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd7527fc640 1 -- 172.21.15.32:0/3738497647 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7480aaae0 con 0x7fd754065d40 2023-12-20T05:43:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd759af9640 1 -- 172.21.15.32:0/3738497647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd754065d40 msgr2=0x7fd754161320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd759af9640 1 --2- 172.21.15.32:0/3738497647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd754065d40 0x7fd754161320 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fd7540ce6c0 tx=0x7fd74801c4d0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd759af9640 1 -- 172.21.15.32:0/3738497647 shutdown_connections 2023-12-20T05:43:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd759af9640 1 --2- 172.21.15.32:0/3738497647 >> v2:172.21.15.39:3300/0 conn(0x7fd75406ca60 0x7fd75406cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd759af9640 1 --2- 172.21.15.32:0/3738497647 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd754065d40 0x7fd754161320 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd759af9640 1 --2- 172.21.15.32:0/3738497647 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7540cb050 0x7fd7540cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd759af9640 1 -- 172.21.15.32:0/3738497647 >> 172.21.15.32:0/3738497647 conn(0x7fd7540cf8d0 msgr2=0x7fd7540cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd759af9640 1 -- 172.21.15.32:0/3738497647 shutdown_connections 2023-12-20T05:43:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.320+0000 7fd759af9640 1 -- 172.21.15.32:0/3738497647 wait complete. 2023-12-20T05:43:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd759af9640 1 Processor -- start 2023-12-20T05:43:43.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd759af9640 1 -- start start 2023-12-20T05:43:43.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd759af9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd754065d40 0x7fd75414e260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd759af9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd75406ca60 0x7fd754157980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd759af9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7540cb050 0x7fd754157ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd759af9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75414e960 con 0x7fd75406ca60 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd759af9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd75414ec40 con 0x7fd754065d40 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd759af9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd75414ef20 con 0x7fd7540cb050 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd752ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd75406ca60 0x7fd754157980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd752ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd75406ca60 0x7fd754157980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60828/0 (socket says 172.21.15.32:60828) 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd752ffd640 1 -- 172.21.15.32:0/2299693321 learned_addr learned my addr 172.21.15.32:0/2299693321 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd7537fe640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd754065d40 0x7fd75414e260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd753fff640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7540cb050 0x7fd754157ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd752ffd640 1 -- 172.21.15.32:0/2299693321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7540cb050 msgr2=0x7fd754157ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd752ffd640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7540cb050 0x7fd754157ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd752ffd640 1 -- 172.21.15.32:0/2299693321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd754065d40 msgr2=0x7fd75414e260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd752ffd640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd754065d40 0x7fd75414e260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd752ffd640 1 -- 172.21.15.32:0/2299693321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7480094d0 con 0x7fd75406ca60 2023-12-20T05:43:43.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd7537fe640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd754065d40 0x7fd75414e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd752ffd640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd75406ca60 0x7fd754157980 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fd74801ab80 tx=0x7fd74801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd73bfff640 1 -- 172.21.15.32:0/2299693321 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7480b3800 con 0x7fd75406ca60 2023-12-20T05:43:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd73bfff640 1 -- 172.21.15.32:0/2299693321 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7480b3980 con 0x7fd75406ca60 2023-12-20T05:43:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd73bfff640 1 -- 172.21.15.32:0/2299693321 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7480b3d30 con 0x7fd75406ca60 2023-12-20T05:43:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd759af9640 1 -- 172.21.15.32:0/2299693321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd75414f200 con 0x7fd75406ca60 2023-12-20T05:43:43.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.324+0000 7fd759af9640 1 -- 172.21.15.32:0/2299693321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd75415a0e0 con 0x7fd75406ca60 2023-12-20T05:43:43.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.328+0000 7fd759af9640 1 -- 172.21.15.32:0/2299693321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7541501f0 con 0x7fd75406ca60 2023-12-20T05:43:43.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.328+0000 7fd73bfff640 1 -- 172.21.15.32:0/2299693321 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd7480b2380 con 0x7fd75406ca60 2023-12-20T05:43:43.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.328+0000 7fd73bfff640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd740065b80 0x7fd740068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.328+0000 7fd73bfff640 1 -- 172.21.15.32:0/2299693321 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fd7480a7070 con 0x7fd75406ca60 2023-12-20T05:43:43.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.328+0000 7fd7537fe640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd740065b80 0x7fd740068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.328+0000 7fd7537fe640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd740065b80 0x7fd740068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd744006fd0 tx=0x7fd744008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:43.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.332+0000 7fd73bfff640 1 -- 172.21.15.32:0/2299693321 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd7541501f0 con 0x7fd75406ca60 2023-12-20T05:43:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.488+0000 7fd759af9640 1 -- 172.21.15.32:0/2299693321 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fd7541501f0 con 0x7fd75406ca60 2023-12-20T05:43:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.488+0000 7fd73bfff640 1 -- 172.21.15.32:0/2299693321 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v155) v1 ==== 93+0+10 (secure 0 0 0) 0x7fd7541501f0 con 0x7fd75406ca60 2023-12-20T05:43:43.491 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 -- 172.21.15.32:0/2299693321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd740065b80 msgr2=0x7fd740068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd740065b80 0x7fd740068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fd744006fd0 tx=0x7fd744008040 comp rx=0 tx=0).stop 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 -- 172.21.15.32:0/2299693321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd75406ca60 msgr2=0x7fd754157980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd75406ca60 0x7fd754157980 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fd74801ab80 tx=0x7fd74801ad10 comp rx=0 tx=0).stop 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 -- 172.21.15.32:0/2299693321 shutdown_connections 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd740065b80 0x7fd740068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7540cb050 0x7fd754157ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd75406ca60 0x7fd754157980 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 --2- 172.21.15.32:0/2299693321 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd754065d40 0x7fd75414e260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 -- 172.21.15.32:0/2299693321 >> 172.21.15.32:0/2299693321 conn(0x7fd7540cf8d0 msgr2=0x7fd7540cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 -- 172.21.15.32:0/2299693321 shutdown_connections 2023-12-20T05:43:43.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.492+0000 7fd759af9640 1 -- 172.21.15.32:0/2299693321 wait complete. 2023-12-20T05:43:43.513 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T05:43:43.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.656+0000 7faf31a66640 1 Processor -- start 2023-12-20T05:43:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.656+0000 7faf31a66640 1 -- start start 2023-12-20T05:43:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.656+0000 7faf31a66640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf2c15cdb0 0x7faf2c0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.656+0000 7faf31a66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2c15d190 con 0x7faf2c0d2740 2023-12-20T05:43:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.656+0000 7faf31a66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faf2c0d16a0 con 0x7faf2c0cfc80 2023-12-20T05:43:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.656+0000 7faf31a66640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faf2c0d1980 con 0x7faf2c15cdb0 2023-12-20T05:43:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.656+0000 7faf2b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf2c15cdb0 0x7faf2c0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.656+0000 7faf2affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7faf2c0d2740 0x7faf2c0cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47774/0 (socket says 172.21.15.32:47774) 2023-12-20T05:43:43.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.656+0000 7faf2b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faf2c15cdb0 0x7faf2c0d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60832/0 (socket says 172.21.15.32:60832) 2023-12-20T05:43:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.656+0000 7faf2affd640 1 -- 172.21.15.32:0/1620539956 learned_addr learned my addr 172.21.15.32:0/1620539956 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.660+0000 7faf2b7fe640 1 -- 172.21.15.32:0/1620539956 >> v1:172.21.15.39:6789/0 conn(0x7faf2c0cfc80 legacy=0x7faf2c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.660+0000 7faf2b7fe640 1 -- 172.21.15.32:0/1620539956 >> v1:172.21.15.32:6789/0 conn(0x7faf2c0d2740 legacy=0x7faf2c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.660+0000 7faf2b7fe640 1 -- 172.21.15.32:0/1620539956 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf2c0d1c60 con 0x7faf2c15cdb0 2023-12-20T05:43:43.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.660+0000 7faf2a7fc640 1 -- 172.21.15.32:0/1620539956 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3860157546 0 0) 0x7faf2c15d190 con 0x7faf2c0d2740 2023-12-20T05:43:43.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.660+0000 7faf2b7fe640 1 --2- 172.21.15.32:0/1620539956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c15cdb0 0x7faf2c0d0be0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0097c0 tx=0x7faf1c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737dff63dc1dbd9e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.660+0000 7faf2a7fc640 1 -- 172.21.15.32:0/1620539956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c0a8970 con 0x7faf2c15cdb0 2023-12-20T05:43:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.660+0000 7faf2a7fc640 1 -- 172.21.15.32:0/1620539956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf1c0a8ad0 con 0x7faf2c15cdb0 2023-12-20T05:43:43.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.660+0000 7faf2a7fc640 1 -- 172.21.15.32:0/1620539956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf1c0a8e60 con 0x7faf2c15cdb0 2023-12-20T05:43:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.660+0000 7faf31a66640 1 -- 172.21.15.32:0/1620539956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c15cdb0 msgr2=0x7faf2c0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.660+0000 7faf31a66640 1 --2- 172.21.15.32:0/1620539956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c15cdb0 0x7faf2c0d0be0 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0097c0 tx=0x7faf1c01ec70 comp rx=0 tx=0).stop 2023-12-20T05:43:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 -- 172.21.15.32:0/1620539956 shutdown_connections 2023-12-20T05:43:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 --2- 172.21.15.32:0/1620539956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c15cdb0 0x7faf2c0d0be0 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 -- 172.21.15.32:0/1620539956 >> 172.21.15.32:0/1620539956 conn(0x7faf2c065ee0 msgr2=0x7faf2c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 -- 172.21.15.32:0/1620539956 shutdown_connections 2023-12-20T05:43:43.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 -- 172.21.15.32:0/1620539956 wait complete. 2023-12-20T05:43:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 Processor -- start 2023-12-20T05:43:43.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 -- start start 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf2c0cfc80 0x7faf2c1f38e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf2c0d2740 0x7faf2c0cb860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c15cdb0 0x7faf2c14b010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0c9e60 con 0x7faf2c15cdb0 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0cc8f0 con 0x7faf2c0cfc80 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf31a66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faf2c0cbda0 con 0x7faf2c0d2740 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf2bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c15cdb0 0x7faf2c14b010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf2bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c15cdb0 0x7faf2c14b010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60840/0 (socket says 172.21.15.32:60840) 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf2bfff640 1 -- 172.21.15.32:0/2254155694 learned_addr learned my addr 172.21.15.32:0/2254155694 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf2affd640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf2c0d2740 0x7faf2c0cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf2b7fe640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf2c0cfc80 0x7faf2c1f38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf2bfff640 1 -- 172.21.15.32:0/2254155694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf2c0d2740 msgr2=0x7faf2c0cb860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf2bfff640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf2c0d2740 0x7faf2c0cb860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf2bfff640 1 -- 172.21.15.32:0/2254155694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf2c0cfc80 msgr2=0x7faf2c1f38e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf2bfff640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf2c0cfc80 0x7faf2c1f38e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.664+0000 7faf2bfff640 1 -- 172.21.15.32:0/2254155694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faf1c009420 con 0x7faf2c15cdb0 2023-12-20T05:43:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.668+0000 7faf2bfff640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c15cdb0 0x7faf2c14b010 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7faf2400c7a0 tx=0x7faf2400cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.668+0000 7faf13fff640 1 -- 172.21.15.32:0/2254155694 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf24003b80 con 0x7faf2c15cdb0 2023-12-20T05:43:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.668+0000 7faf13fff640 1 -- 172.21.15.32:0/2254155694 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faf24003ce0 con 0x7faf2c15cdb0 2023-12-20T05:43:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.668+0000 7faf13fff640 1 -- 172.21.15.32:0/2254155694 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faf2400aa90 con 0x7faf2c15cdb0 2023-12-20T05:43:43.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.668+0000 7faf31a66640 1 -- 172.21.15.32:0/2254155694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faf2c0cc050 con 0x7faf2c15cdb0 2023-12-20T05:43:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.668+0000 7faf31a66640 1 -- 172.21.15.32:0/2254155694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faf2c0cc610 con 0x7faf2c15cdb0 2023-12-20T05:43:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.668+0000 7faf31a66640 1 -- 172.21.15.32:0/2254155694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faf2c0c9fa0 con 0x7faf2c15cdb0 2023-12-20T05:43:43.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.668+0000 7faf13fff640 1 -- 172.21.15.32:0/2254155694 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7faf2400abf0 con 0x7faf2c15cdb0 2023-12-20T05:43:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.668+0000 7faf13fff640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf18065b80 0x7faf18068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.668+0000 7faf13fff640 1 -- 172.21.15.32:0/2254155694 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 4885+0+0 (secure 0 0 0) 0x7faf24082e30 con 0x7faf2c15cdb0 2023-12-20T05:43:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.668+0000 7faf2b7fe640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf18065b80 0x7faf18068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:43.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.672+0000 7faf2b7fe640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf18065b80 0x7faf18068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0b2000 tx=0x7faf1c008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:43.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.672+0000 7faf13fff640 1 -- 172.21.15.32:0/2254155694 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faf2c0c9fa0 con 0x7faf2c15cdb0 2023-12-20T05:43:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 -- 172.21.15.32:0/2254155694 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7faf2c0c9fa0 con 0x7faf2c15cdb0 2023-12-20T05:43:43.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf13fff640 1 -- 172.21.15.32:0/2254155694 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v155) v1 ==== 84+0+65 (secure 0 0 0) 0x7faf2c0c9fa0 con 0x7faf2c15cdb0 2023-12-20T05:43:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:43.836 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T05:43:43.836 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T05:43:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T05:43:43.837 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:43.837 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T05:43:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 -- 172.21.15.32:0/2254155694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf18065b80 msgr2=0x7faf18068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf18065b80 0x7faf18068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7faf1c0b2000 tx=0x7faf1c008040 comp rx=0 tx=0).stop 2023-12-20T05:43:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 -- 172.21.15.32:0/2254155694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c15cdb0 msgr2=0x7faf2c14b010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c15cdb0 0x7faf2c14b010 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x7faf2400c7a0 tx=0x7faf2400cc60 comp rx=0 tx=0).stop 2023-12-20T05:43:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 -- 172.21.15.32:0/2254155694 shutdown_connections 2023-12-20T05:43:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7faf18065b80 0x7faf18068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faf2c15cdb0 0x7faf2c14b010 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faf2c0d2740 0x7faf2c0cb860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 --2- 172.21.15.32:0/2254155694 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faf2c0cfc80 0x7faf2c1f38e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 -- 172.21.15.32:0/2254155694 >> 172.21.15.32:0/2254155694 conn(0x7faf2c065ee0 msgr2=0x7faf2c15ac50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:43.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 -- 172.21.15.32:0/2254155694 shutdown_connections 2023-12-20T05:43:43.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:43.836+0000 7faf31a66640 1 -- 172.21.15.32:0/2254155694 wait complete. 2023-12-20T05:43:43.858 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T05:43:43.858 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T05:43:44.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3dfb0b6640 1 Processor -- start 2023-12-20T05:43:44.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3dfb0b6640 1 -- start start 2023-12-20T05:43:44.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3dfb0b6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3df4158520 0x7f3df406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:44.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3dfb0b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3df40ca050 0x7f3df415ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:44.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3dfb0b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3df4070230 0x7f3df40706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:44.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3dfb0b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df4070be0 con 0x7f3df40ca050 2023-12-20T05:43:44.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3dfb0b6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3df40ca7f0 con 0x7f3df4158520 2023-12-20T05:43:44.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3dfb0b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40caad0 con 0x7f3df4070230 2023-12-20T05:43:44.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3df3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3df40ca050 0x7f3df415ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:44.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3df3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3df40ca050 0x7f3df415ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60848/0 (socket says 172.21.15.32:60848) 2023-12-20T05:43:44.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3df3fff640 1 -- 172.21.15.32:0/1659592708 learned_addr learned my addr 172.21.15.32:0/1659592708 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3df9653640 1 --2- 172.21.15.32:0/1659592708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3df4070230 0x7f3df40706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.004+0000 7f3df8e52640 1 --2- 172.21.15.32:0/1659592708 >> v2:172.21.15.39:3300/0 conn(0x7f3df4158520 0x7f3df406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3df3fff640 1 -- 172.21.15.32:0/1659592708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3df4070230 msgr2=0x7f3df40706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:44.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3df3fff640 1 --2- 172.21.15.32:0/1659592708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3df4070230 0x7f3df40706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3df3fff640 1 -- 172.21.15.32:0/1659592708 >> v2:172.21.15.39:3300/0 conn(0x7f3df4158520 msgr2=0x7f3df406dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3df3fff640 1 --2- 172.21.15.32:0/1659592708 >> v2:172.21.15.39:3300/0 conn(0x7f3df4158520 0x7f3df406dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:44.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3df3fff640 1 -- 172.21.15.32:0/1659592708 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3df40cadb0 con 0x7f3df40ca050 2023-12-20T05:43:44.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3df3fff640 1 --2- 172.21.15.32:0/1659592708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df40ca050 0x7f3df415ef80 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f3df40cf010 tx=0x7f3de001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88f3fd7a713f4cef server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3df37fe640 1 -- 172.21.15.32:0/1659592708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de00aa5f0 con 0x7f3df40ca050 2023-12-20T05:43:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3df37fe640 1 -- 172.21.15.32:0/1659592708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de00aa750 con 0x7f3df40ca050 2023-12-20T05:43:44.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3df37fe640 1 -- 172.21.15.32:0/1659592708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de00aaae0 con 0x7f3df40ca050 2023-12-20T05:43:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/1659592708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df40ca050 msgr2=0x7f3df415ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3dfb0b6640 1 --2- 172.21.15.32:0/1659592708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df40ca050 0x7f3df415ef80 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f3df40cf010 tx=0x7f3de001c4d0 comp rx=0 tx=0).stop 2023-12-20T05:43:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/1659592708 shutdown_connections 2023-12-20T05:43:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3dfb0b6640 1 --2- 172.21.15.32:0/1659592708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3df4070230 0x7f3df40706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3dfb0b6640 1 --2- 172.21.15.32:0/1659592708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df40ca050 0x7f3df415ef80 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3dfb0b6640 1 --2- 172.21.15.32:0/1659592708 >> v2:172.21.15.39:3300/0 conn(0x7f3df4158520 0x7f3df406dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.008+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/1659592708 >> 172.21.15.32:0/1659592708 conn(0x7f3df406a510 msgr2=0x7f3df40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/1659592708 shutdown_connections 2023-12-20T05:43:44.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/1659592708 wait complete. 2023-12-20T05:43:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3dfb0b6640 1 Processor -- start 2023-12-20T05:43:44.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3dfb0b6640 1 -- start start 2023-12-20T05:43:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3dfb0b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4070230 0x7f3df4152980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3dfb0b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3df40ca050 0x7f3df40c5f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3dfb0b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3df4158520 0x7f3df40c6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3dfb0b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df4154cb0 con 0x7f3df4070230 2023-12-20T05:43:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3dfb0b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3df4158c60 con 0x7f3df40ca050 2023-12-20T05:43:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3dfb0b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3df40c9510 con 0x7f3df4158520 2023-12-20T05:43:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df8e52640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4070230 0x7f3df4152980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df8e52640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4070230 0x7f3df4152980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60858/0 (socket says 172.21.15.32:60858) 2023-12-20T05:43:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df8e52640 1 -- 172.21.15.32:0/2807267430 learned_addr learned my addr 172.21.15.32:0/2807267430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df9653640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3df4158520 0x7f3df40c6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:44.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df3fff640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3df40ca050 0x7f3df40c5f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:44.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df8e52640 1 -- 172.21.15.32:0/2807267430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3df4158520 msgr2=0x7f3df40c6450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:44.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df8e52640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3df4158520 0x7f3df40c6450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:44.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df8e52640 1 -- 172.21.15.32:0/2807267430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3df40ca050 msgr2=0x7f3df40c5f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:44.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df8e52640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3df40ca050 0x7f3df40c5f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:44.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df8e52640 1 -- 172.21.15.32:0/2807267430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3de00094d0 con 0x7f3df4070230 2023-12-20T05:43:44.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df9653640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3df4158520 0x7f3df40c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:44.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df8e52640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4070230 0x7f3df4152980 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f3de800d5f0 tx=0x7f3de800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:44.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.012+0000 7f3df17fa640 1 -- 172.21.15.32:0/2807267430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de80125f0 con 0x7f3df4070230 2023-12-20T05:43:44.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.016+0000 7f3df17fa640 1 -- 172.21.15.32:0/2807267430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3de8012750 con 0x7f3df4070230 2023-12-20T05:43:44.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.016+0000 7f3df17fa640 1 -- 172.21.15.32:0/2807267430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3de8012ae0 con 0x7f3df4070230 2023-12-20T05:43:44.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.016+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/2807267430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3df4158eb0 con 0x7f3df4070230 2023-12-20T05:43:44.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.016+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/2807267430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3df4159470 con 0x7f3df4070230 2023-12-20T05:43:44.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.016+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/2807267430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3df404e000 con 0x7f3df4070230 2023-12-20T05:43:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.016+0000 7f3df17fa640 1 -- 172.21.15.32:0/2807267430 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3de8012c40 con 0x7f3df4070230 2023-12-20T05:43:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.016+0000 7f3df17fa640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd8065bd0 0x7f3dd8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.016+0000 7f3df17fa640 1 -- 172.21.15.32:0/2807267430 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(155..155 src has 1..155) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f3de80833d0 con 0x7f3df4070230 2023-12-20T05:43:44.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.016+0000 7f3df3fff640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd8065bd0 0x7f3dd8068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:44.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.016+0000 7f3df3fff640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd8065bd0 0x7f3dd8068090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3de00ae000 tx=0x7f3de00b5040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:44.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.020+0000 7f3df17fa640 1 -- 172.21.15.32:0/2807267430 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3de8051020 con 0x7f3df4070230 2023-12-20T05:43:44.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:44.180+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/2807267430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f3df40c4a40 con 0x7f3df4070230 2023-12-20T05:43:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.088+0000 7f3df17fa640 1 -- 172.21.15.32:0/2807267430 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v156) v1 ==== 138+0+0 (secure 0 0 0) 0x7f3de8007380 con 0x7f3df4070230 2023-12-20T05:43:45.090 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T05:43:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/2807267430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd8065bd0 msgr2=0x7f3dd8068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd8065bd0 0x7f3dd8068090 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f3de00ae000 tx=0x7f3de00b5040 comp rx=0 tx=0).stop 2023-12-20T05:43:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/2807267430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4070230 msgr2=0x7f3df4152980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:45.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4070230 0x7f3df4152980 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f3de800d5f0 tx=0x7f3de800dab0 comp rx=0 tx=0).stop 2023-12-20T05:43:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/2807267430 shutdown_connections 2023-12-20T05:43:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3dd8065bd0 0x7f3dd8068090 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3df4158520 0x7f3df40c6450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:45.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3df40ca050 0x7f3df40c5f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:45.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 --2- 172.21.15.32:0/2807267430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3df4070230 0x7f3df4152980 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:45.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/2807267430 >> 172.21.15.32:0/2807267430 conn(0x7f3df406a510 msgr2=0x7f3df40cec60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:45.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/2807267430 shutdown_connections 2023-12-20T05:43:45.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.092+0000 7f3dfb0b6640 1 -- 172.21.15.32:0/2807267430 wait complete. 2023-12-20T05:43:45.123 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T05:43:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.276+0000 7fa2572fc640 1 Processor -- start 2023-12-20T05:43:45.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.276+0000 7fa2572fc640 1 -- start start 2023-12-20T05:43:45.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa2572fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa2500adca0 0x7fa2500cd750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:45.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa2572fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2501655e0 0x7fa25006d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:45.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa2572fc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa25006d5e0 con 0x7fa25006cc40 2023-12-20T05:43:45.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa2572fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa250165a50 con 0x7fa2500adca0 2023-12-20T05:43:45.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa2572fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500cc710 con 0x7fa2501655e0 2023-12-20T05:43:45.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa254897640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa25006cc40 0x7fa250164ed0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47776/0 (socket says 172.21.15.32:47776) 2023-12-20T05:43:45.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa254897640 1 -- 172.21.15.32:0/2232449437 learned_addr learned my addr 172.21.15.32:0/2232449437 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:45.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3108942682 0 0) 0x7fa25006d5e0 con 0x7fa25006cc40 2023-12-20T05:43:45.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa255098640 1 --2- 172.21.15.32:0/2232449437 >> v2:172.21.15.39:3300/0 conn(0x7fa2500adca0 0x7fa2500cd750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:45.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa255899640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2501655e0 0x7fa25006d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:45.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa23c003530 con 0x7fa25006cc40 2023-12-20T05:43:45.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3276399542 0 0) 0x7fa23c003530 con 0x7fa25006cc40 2023-12-20T05:43:45.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2501655e0 msgr2=0x7fa25006d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:45.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2501655e0 0x7fa25006d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:45.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 >> v2:172.21.15.39:3300/0 conn(0x7fa2500adca0 msgr2=0x7fa2500cd750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:45.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 --2- 172.21.15.32:0/2232449437 >> v2:172.21.15.39:3300/0 conn(0x7fa2500adca0 0x7fa2500cd750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:45.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2500cc990 con 0x7fa25006cc40 2023-12-20T05:43:45.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fa240001680 con 0x7fa25006cc40 2023-12-20T05:43:45.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 >> v1:172.21.15.32:6789/0 conn(0x7fa25006cc40 legacy=0x7fa250164ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:45.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2501655e0 0x7fa23c003980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:45.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 0x7fa23c0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:45.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa23c004d10 0x7fa23c005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:45.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa250165a50 con 0x7fa23c003f00 2023-12-20T05:43:45.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2500cc710 con 0x7fa23c004d10 2023-12-20T05:43:45.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa23c00b9e0 con 0x7fa2501655e0 2023-12-20T05:43:45.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa254897640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 0x7fa23c0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:45.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa255098640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2501655e0 0x7fa23c003980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:45.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa255899640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa23c004d10 0x7fa23c005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:45.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa254897640 1 -- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2501655e0 msgr2=0x7fa23c003980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:45.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa254897640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2501655e0 0x7fa23c003980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa254897640 1 -- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa23c004d10 msgr2=0x7fa23c005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa254897640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa23c004d10 0x7fa23c005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa254897640 1 -- 172.21.15.32:0/2232449437 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa23c003530 con 0x7fa23c003f00 2023-12-20T05:43:45.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.280+0000 7fa255098640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2501655e0 0x7fa23c003980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:43:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.284+0000 7fa254897640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 0x7fa23c0047d0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa24000c270 tx=0x7fa24001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b0de1c76c8dbe2a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.284+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa2400058e0 con 0x7fa23c003f00 2023-12-20T05:43:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.284+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa240005a40 con 0x7fa23c003f00 2023-12-20T05:43:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.284+0000 7fa247fff640 1 -- 172.21.15.32:0/2232449437 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa240005e70 con 0x7fa23c003f00 2023-12-20T05:43:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.284+0000 7fa2572fc640 1 -- 172.21.15.32:0/2232449437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 msgr2=0x7fa23c0047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:45.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.284+0000 7fa2572fc640 1 --2- 172.21.15.32:0/2232449437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 0x7fa23c0047d0 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7fa24000c270 tx=0x7fa24001cc60 comp rx=0 tx=0).stop 2023-12-20T05:43:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.284+0000 7fa255899640 1 -- 172.21.15.32:0/2232449437 reap_dead start 2023-12-20T05:43:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.284+0000 7fa2572fc640 1 -- 172.21.15.32:0/2232449437 shutdown_connections 2023-12-20T05:43:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.284+0000 7fa2572fc640 1 -- 172.21.15.32:0/2232449437 >> 172.21.15.32:0/2232449437 conn(0x7fa25006a4b0 msgr2=0x7fa2500b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.284+0000 7fa2572fc640 1 -- 172.21.15.32:0/2232449437 shutdown_connections 2023-12-20T05:43:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.284+0000 7fa2572fc640 1 -- 172.21.15.32:0/2232449437 wait complete. 2023-12-20T05:43:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa2572fc640 1 Processor -- start 2023-12-20T05:43:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa2572fc640 1 -- start start 2023-12-20T05:43:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa2572fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 0x7fa2501fd370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa2572fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa250158130 0x7fa2501f9520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:45.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa2572fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2500cb400 0x7fa2500b12b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa2572fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2501fa450 con 0x7fa23c003f00 2023-12-20T05:43:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa2572fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa2501f9e90 con 0x7fa250158130 2023-12-20T05:43:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa2572fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa2501fa170 con 0x7fa2500cb400 2023-12-20T05:43:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa255098640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 0x7fa2501fd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa255098640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 0x7fa2501fd370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60878/0 (socket says 172.21.15.32:60878) 2023-12-20T05:43:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa255098640 1 -- 172.21.15.32:0/1850464266 learned_addr learned my addr 172.21.15.32:0/1850464266 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa254897640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa250158130 0x7fa2501f9520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa255899640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2500cb400 0x7fa2500b12b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:45.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa255098640 1 -- 172.21.15.32:0/1850464266 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2500cb400 msgr2=0x7fa2500b12b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa255098640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2500cb400 0x7fa2500b12b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa255098640 1 -- 172.21.15.32:0/1850464266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa250158130 msgr2=0x7fa2501f9520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa255098640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa250158130 0x7fa2501f9520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa255098640 1 -- 172.21.15.32:0/1850464266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa2501fcb70 con 0x7fa23c003f00 2023-12-20T05:43:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa255098640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 0x7fa2501fd370 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa24c011ca0 tx=0x7fa24c00f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa245ffb640 1 -- 172.21.15.32:0/1850464266 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c00fd20 con 0x7fa23c003f00 2023-12-20T05:43:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa245ffb640 1 -- 172.21.15.32:0/1850464266 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa24c00fe80 con 0x7fa23c003f00 2023-12-20T05:43:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa245ffb640 1 -- 172.21.15.32:0/1850464266 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa24c004260 con 0x7fa23c003f00 2023-12-20T05:43:45.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa2572fc640 1 -- 172.21.15.32:0/1850464266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa2501fce50 con 0x7fa23c003f00 2023-12-20T05:43:45.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.288+0000 7fa2572fc640 1 -- 172.21.15.32:0/1850464266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa2501f9a60 con 0x7fa23c003f00 2023-12-20T05:43:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.292+0000 7fa2572fc640 1 -- 172.21.15.32:0/1850464266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa25004e000 con 0x7fa23c003f00 2023-12-20T05:43:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.292+0000 7fa245ffb640 1 -- 172.21.15.32:0/1850464266 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa24c0043c0 con 0x7fa23c003f00 2023-12-20T05:43:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.292+0000 7fa245ffb640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa234065b80 0x7fa234068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.292+0000 7fa245ffb640 1 -- 172.21.15.32:0/1850464266 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(156..156 src has 1..156) v4 ==== 5255+0+0 (secure 0 0 0) 0x7fa24c083c60 con 0x7fa23c003f00 2023-12-20T05:43:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.292+0000 7fa254897640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa234065b80 0x7fa234068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:45.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.292+0000 7fa254897640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa234065b80 0x7fa234068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa2400aa000 tx=0x7fa240001a80 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:45.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.296+0000 7fa245ffb640 1 -- 172.21.15.32:0/1850464266 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa24c051860 con 0x7fa23c003f00 2023-12-20T05:43:45.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:45.452+0000 7fa2572fc640 1 -- 172.21.15.32:0/1850464266 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa2501fcfe0 con 0x7fa23c003f00 2023-12-20T05:43:46.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.092+0000 7fa245ffb640 1 -- 172.21.15.32:0/1850464266 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v157) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa24c054e80 con 0x7fa23c003f00 2023-12-20T05:43:46.096 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T05:43:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.104+0000 7fa2572fc640 1 -- 172.21.15.32:0/1850464266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa234065b80 msgr2=0x7fa234068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.104+0000 7fa2572fc640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa234065b80 0x7fa234068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fa2400aa000 tx=0x7fa240001a80 comp rx=0 tx=0).stop 2023-12-20T05:43:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.104+0000 7fa2572fc640 1 -- 172.21.15.32:0/1850464266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 msgr2=0x7fa2501fd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.104+0000 7fa2572fc640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 0x7fa2501fd370 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7fa24c011ca0 tx=0x7fa24c00f610 comp rx=0 tx=0).stop 2023-12-20T05:43:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.104+0000 7fa2572fc640 1 -- 172.21.15.32:0/1850464266 shutdown_connections 2023-12-20T05:43:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.104+0000 7fa2572fc640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa234065b80 0x7fa234068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.104+0000 7fa2572fc640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa2500cb400 0x7fa2500b12b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.104+0000 7fa2572fc640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa250158130 0x7fa2501f9520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.104+0000 7fa2572fc640 1 --2- 172.21.15.32:0/1850464266 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa23c003f00 0x7fa2501fd370 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.104+0000 7fa2572fc640 1 -- 172.21.15.32:0/1850464266 >> 172.21.15.32:0/1850464266 conn(0x7fa25006a4b0 msgr2=0x7fa2500aea30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.108+0000 7fa2572fc640 1 -- 172.21.15.32:0/1850464266 shutdown_connections 2023-12-20T05:43:46.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.108+0000 7fa2572fc640 1 -- 172.21.15.32:0/1850464266 wait complete. 2023-12-20T05:43:46.140 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T05:43:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f8951ba2640 1 Processor -- start 2023-12-20T05:43:46.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f8951ba2640 1 -- start start 2023-12-20T05:43:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f8951ba2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f894c0affe0 0x7f894c071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:46.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f8951ba2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f894c0cdce0 0x7f894c0cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f8951ba2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f894c071820 con 0x7f894c0b13b0 2023-12-20T05:43:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f8951ba2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894c0723a0 con 0x7f894c0cdce0 2023-12-20T05:43:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f8951ba2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f894c071b00 con 0x7f894c0affe0 2023-12-20T05:43:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f89513a1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f894c0cdce0 0x7f894c0cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f89513a1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f894c0cdce0 0x7f894c0cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60894/0 (socket says 172.21.15.32:60894) 2023-12-20T05:43:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f89513a1640 1 -- 172.21.15.32:0/1784142244 learned_addr learned my addr 172.21.15.32:0/1784142244 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f8950ba0640 1 --2- 172.21.15.32:0/1784142244 >> v2:172.21.15.39:3300/0 conn(0x7f894c0affe0 0x7f894c071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f89513a1640 1 -- 172.21.15.32:0/1784142244 >> v2:172.21.15.39:3300/0 conn(0x7f894c0affe0 msgr2=0x7f894c071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:46.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f89513a1640 1 --2- 172.21.15.32:0/1784142244 >> v2:172.21.15.39:3300/0 conn(0x7f894c0affe0 0x7f894c071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f89513a1640 1 -- 172.21.15.32:0/1784142244 >> v1:172.21.15.32:6789/0 conn(0x7f894c0b13b0 legacy=0x7f894c0b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f894b7fe640 1 -- 172.21.15.32:0/1784142244 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 979177031 0 0) 0x7f894c071820 con 0x7f894c0b13b0 2023-12-20T05:43:46.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.288+0000 7f89513a1640 1 -- 172.21.15.32:0/1784142244 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f894c071de0 con 0x7f894c0cdce0 2023-12-20T05:43:46.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f89513a1640 1 --2- 172.21.15.32:0/1784142244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c0cdce0 0x7f894c0cc510 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8944009e50 tx=0x7f894401c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=891650448b889976 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:46.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f894b7fe640 1 -- 172.21.15.32:0/1784142244 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89440a5690 con 0x7f894c0cdce0 2023-12-20T05:43:46.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f894b7fe640 1 -- 172.21.15.32:0/1784142244 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f89440a57f0 con 0x7f894c0cdce0 2023-12-20T05:43:46.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f894b7fe640 1 -- 172.21.15.32:0/1784142244 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89440a5b80 con 0x7f894c0cdce0 2023-12-20T05:43:46.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f8951ba2640 1 -- 172.21.15.32:0/1784142244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c0cdce0 msgr2=0x7f894c0cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f8951ba2640 1 --2- 172.21.15.32:0/1784142244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c0cdce0 0x7f894c0cc510 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f8944009e50 tx=0x7f894401c9d0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f8951ba2640 1 -- 172.21.15.32:0/1784142244 shutdown_connections 2023-12-20T05:43:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f8951ba2640 1 --2- 172.21.15.32:0/1784142244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c0cdce0 0x7f894c0cc510 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f8951ba2640 1 --2- 172.21.15.32:0/1784142244 >> v2:172.21.15.39:3300/0 conn(0x7f894c0affe0 0x7f894c071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f8951ba2640 1 -- 172.21.15.32:0/1784142244 >> 172.21.15.32:0/1784142244 conn(0x7f894c06a4e0 msgr2=0x7f894c06bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f8951ba2640 1 -- 172.21.15.32:0/1784142244 shutdown_connections 2023-12-20T05:43:46.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.292+0000 7f8951ba2640 1 -- 172.21.15.32:0/1784142244 wait complete. 2023-12-20T05:43:46.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f8951ba2640 1 Processor -- start 2023-12-20T05:43:46.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f8951ba2640 1 -- start start 2023-12-20T05:43:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f8951ba2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f894c0affe0 0x7f894c154a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f8951ba2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f894c0b13b0 0x7f894c154f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f8951ba2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c0cdce0 0x7f894c1f5420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f8951ba2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c154600 con 0x7f894c0cdce0 2023-12-20T05:43:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f8951ba2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f894c153980 con 0x7f894c0b13b0 2023-12-20T05:43:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f8951ba2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f894c153c60 con 0x7f894c0affe0 2023-12-20T05:43:46.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89513a1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c0cdce0 0x7f894c1f5420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89513a1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c0cdce0 0x7f894c1f5420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60910/0 (socket says 172.21.15.32:60910) 2023-12-20T05:43:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f8950ba0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f894c0affe0 0x7f894c154a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89513a1640 1 -- 172.21.15.32:0/1257904987 learned_addr learned my addr 172.21.15.32:0/1257904987 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f894bfff640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f894c0b13b0 0x7f894c154f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89513a1640 1 -- 172.21.15.32:0/1257904987 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f894c0affe0 msgr2=0x7f894c154a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89513a1640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f894c0affe0 0x7f894c154a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89513a1640 1 -- 172.21.15.32:0/1257904987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f894c0b13b0 msgr2=0x7f894c154f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89513a1640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f894c0b13b0 0x7f894c154f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89513a1640 1 -- 172.21.15.32:0/1257904987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8944009b30 con 0x7f894c0cdce0 2023-12-20T05:43:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89513a1640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c0cdce0 0x7f894c1f5420 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f89440195e0 tx=0x7f8944019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89497fa640 1 -- 172.21.15.32:0/1257904987 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89440bd070 con 0x7f894c0cdce0 2023-12-20T05:43:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89497fa640 1 -- 172.21.15.32:0/1257904987 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8944009510 con 0x7f894c0cdce0 2023-12-20T05:43:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f89497fa640 1 -- 172.21.15.32:0/1257904987 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f89440a5690 con 0x7f894c0cdce0 2023-12-20T05:43:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f8951ba2640 1 -- 172.21.15.32:0/1257904987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f894c153f40 con 0x7f894c0cdce0 2023-12-20T05:43:46.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.296+0000 7f8951ba2640 1 -- 172.21.15.32:0/1257904987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f894c151e40 con 0x7f894c0cdce0 2023-12-20T05:43:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.300+0000 7f8951ba2640 1 -- 172.21.15.32:0/1257904987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f894c152030 con 0x7f894c0cdce0 2023-12-20T05:43:46.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.300+0000 7f89497fa640 1 -- 172.21.15.32:0/1257904987 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f89440b88a0 con 0x7f894c0cdce0 2023-12-20T05:43:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.300+0000 7f89497fa640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8930065bb0 0x7f8930068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.300+0000 7f89497fa640 1 -- 172.21.15.32:0/1257904987 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(157..157 src has 1..157) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f894411e8c0 con 0x7f894c0cdce0 2023-12-20T05:43:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.304+0000 7f8950ba0640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8930065bb0 0x7f8930068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:46.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.304+0000 7f8950ba0640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8930065bb0 0x7f8930068070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f894c158620 tx=0x7f8940009190 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:46.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.304+0000 7f89497fa640 1 -- 172.21.15.32:0/1257904987 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f89440ec370 con 0x7f894c0cdce0 2023-12-20T05:43:46.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.500+0000 7f8951ba2640 1 -- 172.21.15.32:0/1257904987 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f894c154740 con 0x7f894c0cdce0 2023-12-20T05:43:46.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.508+0000 7f89497fa640 1 -- 172.21.15.32:0/1257904987 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 18 and data pool 19 v67) v1 ==== 174+0+0 (secure 0 0 0) 0x7f894c154740 con 0x7f894c0cdce0 2023-12-20T05:43:46.510 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 18 and data pool 19 2023-12-20T05:43:46.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.512+0000 7f8951ba2640 1 -- 172.21.15.32:0/1257904987 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8930065bb0 msgr2=0x7f8930068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.512+0000 7f8951ba2640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8930065bb0 0x7f8930068070 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f894c158620 tx=0x7f8940009190 comp rx=0 tx=0).stop 2023-12-20T05:43:46.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.512+0000 7f8951ba2640 1 -- 172.21.15.32:0/1257904987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c0cdce0 msgr2=0x7f894c1f5420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.512+0000 7f8951ba2640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c0cdce0 0x7f894c1f5420 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f89440195e0 tx=0x7f8944019770 comp rx=0 tx=0).stop 2023-12-20T05:43:46.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.512+0000 7f8951ba2640 1 -- 172.21.15.32:0/1257904987 shutdown_connections 2023-12-20T05:43:46.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.512+0000 7f8951ba2640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8930065bb0 0x7f8930068070 secure :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f894c158620 tx=0x7f8940009190 comp rx=0 tx=0).stop 2023-12-20T05:43:46.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.512+0000 7f8951ba2640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f894c0cdce0 0x7f894c1f5420 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.512+0000 7f8951ba2640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f894c0b13b0 0x7f894c154f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.512+0000 7f8951ba2640 1 --2- 172.21.15.32:0/1257904987 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f894c0affe0 0x7f894c154a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.512+0000 7f8951ba2640 1 -- 172.21.15.32:0/1257904987 >> 172.21.15.32:0/1257904987 conn(0x7f894c06a4e0 msgr2=0x7f894c0adfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:46.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.516+0000 7f8951ba2640 1 -- 172.21.15.32:0/1257904987 shutdown_connections 2023-12-20T05:43:46.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.516+0000 7f8951ba2640 1 -- 172.21.15.32:0/1257904987 wait complete. 2023-12-20T05:43:46.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:43:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f65f0640 1 Processor -- start 2023-12-20T05:43:46.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f65f0640 1 -- start start 2023-12-20T05:43:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f65f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8f006caf0 0x7fa8f00705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f65f0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f00ae140 con 0x7fa8f015d9a0 2023-12-20T05:43:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f65f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa8f0072170 con 0x7fa8f01593c0 2023-12-20T05:43:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f65f0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa8f00ae9e0 con 0x7fa8f006caf0 2023-12-20T05:43:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f4b8d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8f006caf0 0x7fa8f00705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f4b8d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa8f006caf0 0x7fa8f00705a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60918/0 (socket says 172.21.15.32:60918) 2023-12-20T05:43:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f4b8d640 1 -- 172.21.15.32:0/61579628 learned_addr learned my addr 172.21.15.32:0/61579628 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:46.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8ef7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa8f015d9a0 0x7fa8f006c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47782/0 (socket says 172.21.15.32:47782) 2023-12-20T05:43:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f4b8d640 1 -- 172.21.15.32:0/61579628 >> v1:172.21.15.39:6789/0 conn(0x7fa8f01593c0 legacy=0x7fa8f00afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f4b8d640 1 -- 172.21.15.32:0/61579628 >> v1:172.21.15.32:6789/0 conn(0x7fa8f015d9a0 legacy=0x7fa8f006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.696+0000 7fa8f4b8d640 1 -- 172.21.15.32:0/61579628 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8f00ae390 con 0x7fa8f006caf0 2023-12-20T05:43:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.700+0000 7fa8f4b8d640 1 --2- 172.21.15.32:0/61579628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f006caf0 0x7fa8f00705a0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800d150 tx=0x7fa8e801eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=faaba55d4b4078ca server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:46.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.700+0000 7fa8eeffd640 1 -- 172.21.15.32:0/61579628 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80a7a70 con 0x7fa8f006caf0 2023-12-20T05:43:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.700+0000 7fa8eeffd640 1 -- 172.21.15.32:0/61579628 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e80ab070 con 0x7fa8f006caf0 2023-12-20T05:43:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.700+0000 7fa8eeffd640 1 -- 172.21.15.32:0/61579628 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e80a7df0 con 0x7fa8f006caf0 2023-12-20T05:43:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.700+0000 7fa8f65f0640 1 -- 172.21.15.32:0/61579628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f006caf0 msgr2=0x7fa8f00705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.700+0000 7fa8f65f0640 1 --2- 172.21.15.32:0/61579628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f006caf0 0x7fa8f00705a0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7fa8e800d150 tx=0x7fa8e801eac0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.700+0000 7fa8f65f0640 1 -- 172.21.15.32:0/61579628 shutdown_connections 2023-12-20T05:43:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.700+0000 7fa8f65f0640 1 --2- 172.21.15.32:0/61579628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f006caf0 0x7fa8f00705a0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.700+0000 7fa8f65f0640 1 -- 172.21.15.32:0/61579628 >> 172.21.15.32:0/61579628 conn(0x7fa8f006a320 msgr2=0x7fa8f00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.700+0000 7fa8f65f0640 1 -- 172.21.15.32:0/61579628 shutdown_connections 2023-12-20T05:43:46.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.700+0000 7fa8f65f0640 1 -- 172.21.15.32:0/61579628 wait complete. 2023-12-20T05:43:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f65f0640 1 Processor -- start 2023-12-20T05:43:46.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f65f0640 1 -- start start 2023-12-20T05:43:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f65f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f006caf0 0x7fa8f0148670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f65f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f01593c0 0x7fa8f014b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f65f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa8f015d9a0 0x7fa8f0153d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:46.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f65f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f014da70 con 0x7fa8f006caf0 2023-12-20T05:43:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f65f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f00c5be0 con 0x7fa8f01593c0 2023-12-20T05:43:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f65f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa8f00c5ec0 con 0x7fa8f015d9a0 2023-12-20T05:43:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f006caf0 0x7fa8f0148670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8effff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f006caf0 0x7fa8f0148670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60930/0 (socket says 172.21.15.32:60930) 2023-12-20T05:43:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8effff640 1 -- 172.21.15.32:0/213754227 learned_addr learned my addr 172.21.15.32:0/213754227 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8ef7fe640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f01593c0 0x7fa8f014b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f4b8d640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa8f015d9a0 0x7fa8f0153d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8effff640 1 -- 172.21.15.32:0/213754227 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa8f015d9a0 msgr2=0x7fa8f0153d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8effff640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa8f015d9a0 0x7fa8f0153d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8effff640 1 -- 172.21.15.32:0/213754227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f01593c0 msgr2=0x7fa8f014b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8effff640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f01593c0 0x7fa8f014b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8effff640 1 -- 172.21.15.32:0/213754227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa8e800bd40 con 0x7fa8f006caf0 2023-12-20T05:43:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8ef7fe640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f01593c0 0x7fa8f014b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f4b8d640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa8f015d9a0 0x7fa8f0153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8effff640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f006caf0 0x7fa8f0148670 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000c400 tx=0x7fa8e000c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8ecff9640 1 -- 172.21.15.32:0/213754227 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e00052d0 con 0x7fa8f006caf0 2023-12-20T05:43:46.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8ecff9640 1 -- 172.21.15.32:0/213754227 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa8e0005450 con 0x7fa8f006caf0 2023-12-20T05:43:46.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8ecff9640 1 -- 172.21.15.32:0/213754227 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa8e0003890 con 0x7fa8f006caf0 2023-12-20T05:43:46.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f65f0640 1 -- 172.21.15.32:0/213754227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa8f00c61a0 con 0x7fa8f006caf0 2023-12-20T05:43:46.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.704+0000 7fa8f65f0640 1 -- 172.21.15.32:0/213754227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa8f0148bb0 con 0x7fa8f006caf0 2023-12-20T05:43:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.708+0000 7fa8f65f0640 1 -- 172.21.15.32:0/213754227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa8f00c5d20 con 0x7fa8f006caf0 2023-12-20T05:43:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.708+0000 7fa8ecff9640 1 -- 172.21.15.32:0/213754227 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa8e00039f0 con 0x7fa8f006caf0 2023-12-20T05:43:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.708+0000 7fa8ecff9640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8dc065b80 0x7fa8dc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.708+0000 7fa8ecff9640 1 -- 172.21.15.32:0/213754227 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa8e0082900 con 0x7fa8f006caf0 2023-12-20T05:43:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.708+0000 7fa8ef7fe640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8dc065b80 0x7fa8dc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:46.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.708+0000 7fa8ef7fe640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8dc065b80 0x7fa8dc068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0154930 tx=0x7fa8e4009040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:46.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.712+0000 7fa8ecff9640 1 -- 172.21.15.32:0/213754227 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa8f00c5d20 con 0x7fa8f006caf0 2023-12-20T05:43:46.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.872+0000 7fa8f65f0640 1 -- 172.21.15.32:0/213754227 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa8f00c5d20 con 0x7fa8f006caf0 2023-12-20T05:43:46.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.872+0000 7fa8ecff9640 1 -- 172.21.15.32:0/213754227 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16120 (secure 0 0 0) 0x7fa8f00c5d20 con 0x7fa8f006caf0 2023-12-20T05:43:46.875 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:46.875 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":158,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:43:46.504917+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:43:44.235578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-20T05:43:45.508783+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:43:46.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 -- 172.21.15.32:0/213754227 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8dc065b80 msgr2=0x7fa8dc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8dc065b80 0x7fa8dc068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fa8f0154930 tx=0x7fa8e4009040 comp rx=0 tx=0).stop 2023-12-20T05:43:46.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 -- 172.21.15.32:0/213754227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f006caf0 msgr2=0x7fa8f0148670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:46.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f006caf0 0x7fa8f0148670 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7fa8e000c400 tx=0x7fa8e000c8c0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 -- 172.21.15.32:0/213754227 shutdown_connections 2023-12-20T05:43:46.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa8dc065b80 0x7fa8dc068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa8f015d9a0 0x7fa8f0153d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa8f01593c0 0x7fa8f014b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 --2- 172.21.15.32:0/213754227 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa8f006caf0 0x7fa8f0148670 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:46.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 -- 172.21.15.32:0/213754227 >> 172.21.15.32:0/213754227 conn(0x7fa8f006a320 msgr2=0x7fa8f00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:46.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 -- 172.21.15.32:0/213754227 shutdown_connections 2023-12-20T05:43:46.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:46.876+0000 7fa8f65f0640 1 -- 172.21.15.32:0/213754227 wait complete. 2023-12-20T05:43:46.897 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:43:47.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757fa95640 1 Processor -- start 2023-12-20T05:43:47.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757fa95640 1 -- start start 2023-12-20T05:43:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757fa95640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f757815cdb0 0x7f75780d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757fa95640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f75780d2740 0x7f75780cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757fa95640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f75780b6ca0 con 0x7f75780d1120 2023-12-20T05:43:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757fa95640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f757815d190 con 0x7f757815cdb0 2023-12-20T05:43:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757fa95640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f75780d16a0 con 0x7f75780d2740 2023-12-20T05:43:47.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757d831640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f757815cdb0 0x7f75780d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757e032640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f75780d1120 0x7f75780b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47798/0 (socket says 172.21.15.32:47798) 2023-12-20T05:43:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757e032640 1 -- 172.21.15.32:0/4075473414 learned_addr learned my addr 172.21.15.32:0/4075473414 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757d030640 1 --2- 172.21.15.32:0/4075473414 >> v2:172.21.15.39:3300/0 conn(0x7f75780d2740 0x7f75780cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757d831640 1 -- 172.21.15.32:0/4075473414 >> v2:172.21.15.39:3300/0 conn(0x7f75780d2740 msgr2=0x7f75780cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757d831640 1 --2- 172.21.15.32:0/4075473414 >> v2:172.21.15.39:3300/0 conn(0x7f75780d2740 0x7f75780cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757d831640 1 -- 172.21.15.32:0/4075473414 >> v1:172.21.15.32:6789/0 conn(0x7f75780d1120 legacy=0x7f75780b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757d831640 1 -- 172.21.15.32:0/4075473414 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f75780d1980 con 0x7f757815cdb0 2023-12-20T05:43:47.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.040+0000 7f757c82f640 1 -- 172.21.15.32:0/4075473414 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1726481719 0 0) 0x7f75700014a0 con 0x7f75780d1120 2023-12-20T05:43:47.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757d831640 1 --2- 172.21.15.32:0/4075473414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757815cdb0 0x7f75780d0be0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f75780cd570 tx=0x7f757401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36acd2213d7c9d91 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757c82f640 1 -- 172.21.15.32:0/4075473414 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75740a8950 con 0x7f757815cdb0 2023-12-20T05:43:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757c82f640 1 -- 172.21.15.32:0/4075473414 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f75740a8ab0 con 0x7f757815cdb0 2023-12-20T05:43:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757c82f640 1 -- 172.21.15.32:0/4075473414 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f75740a8e40 con 0x7f757815cdb0 2023-12-20T05:43:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757fa95640 1 -- 172.21.15.32:0/4075473414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757815cdb0 msgr2=0x7f75780d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757fa95640 1 --2- 172.21.15.32:0/4075473414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757815cdb0 0x7f75780d0be0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7f75780cd570 tx=0x7f757401ec50 comp rx=0 tx=0).stop 2023-12-20T05:43:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757fa95640 1 -- 172.21.15.32:0/4075473414 shutdown_connections 2023-12-20T05:43:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757fa95640 1 --2- 172.21.15.32:0/4075473414 >> v2:172.21.15.39:3300/0 conn(0x7f75780d2740 0x7f75780cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757fa95640 1 --2- 172.21.15.32:0/4075473414 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757815cdb0 0x7f75780d0be0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757fa95640 1 -- 172.21.15.32:0/4075473414 >> 172.21.15.32:0/4075473414 conn(0x7f7578065ee0 msgr2=0x7f757806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757fa95640 1 -- 172.21.15.32:0/4075473414 shutdown_connections 2023-12-20T05:43:47.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.044+0000 7f757fa95640 1 -- 172.21.15.32:0/4075473414 wait complete. 2023-12-20T05:43:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757fa95640 1 Processor -- start 2023-12-20T05:43:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757fa95640 1 -- start start 2023-12-20T05:43:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757fa95640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f75780d1120 0x7f757814f470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757fa95640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75780d2740 0x7f757815e910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757fa95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757815cdb0 0x7f757815ee50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757fa95640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f75781511a0 con 0x7f757815cdb0 2023-12-20T05:43:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757fa95640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7578150e50 con 0x7f75780d2740 2023-12-20T05:43:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757fa95640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f75780c90c0 con 0x7f75780d1120 2023-12-20T05:43:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757e032640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757815cdb0 0x7f757815ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757e032640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757815cdb0 0x7f757815ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60944/0 (socket says 172.21.15.32:60944) 2023-12-20T05:43:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757e032640 1 -- 172.21.15.32:0/1704081514 learned_addr learned my addr 172.21.15.32:0/1704081514 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:47.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757d831640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f75780d1120 0x7f757814f470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757d030640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75780d2740 0x7f757815e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757e032640 1 -- 172.21.15.32:0/1704081514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f75780d1120 msgr2=0x7f757814f470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757e032640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f75780d1120 0x7f757814f470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757e032640 1 -- 172.21.15.32:0/1704081514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75780d2740 msgr2=0x7f757815e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757e032640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75780d2740 0x7f757815e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757e032640 1 -- 172.21.15.32:0/1704081514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7574009420 con 0x7f757815cdb0 2023-12-20T05:43:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757e032640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757815cdb0 0x7f757815ee50 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f757000c7e0 tx=0x7f757000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:47.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f756e7fc640 1 -- 172.21.15.32:0/1704081514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000a640 con 0x7f757815cdb0 2023-12-20T05:43:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f756e7fc640 1 -- 172.21.15.32:0/1704081514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f757000a7c0 con 0x7f757815cdb0 2023-12-20T05:43:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f756e7fc640 1 -- 172.21.15.32:0/1704081514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f757000abb0 con 0x7f757815cdb0 2023-12-20T05:43:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757fa95640 1 -- 172.21.15.32:0/1704081514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f757814f9b0 con 0x7f757815cdb0 2023-12-20T05:43:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757fa95640 1 -- 172.21.15.32:0/1704081514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f75781e88f0 con 0x7f757815cdb0 2023-12-20T05:43:47.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.048+0000 7f757fa95640 1 -- 172.21.15.32:0/1704081514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f75781512e0 con 0x7f757815cdb0 2023-12-20T05:43:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.052+0000 7f756e7fc640 1 -- 172.21.15.32:0/1704081514 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f757000ad10 con 0x7f757815cdb0 2023-12-20T05:43:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.052+0000 7f756e7fc640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 0x7f755c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.052+0000 7f756e7fc640 1 -- 172.21.15.32:0/1704081514 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f75700826d0 con 0x7f757815cdb0 2023-12-20T05:43:47.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.052+0000 7f757d831640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 0x7f755c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.052+0000 7f757d831640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 0x7f755c068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75740b2000 tx=0x7f7574008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:47.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.056+0000 7f756e7fc640 1 -- 172.21.15.32:0/1704081514 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f75781512e0 con 0x7f757815cdb0 2023-12-20T05:43:47.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.212+0000 7f757fa95640 1 -- 172.21.15.32:0/1704081514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f75780b6ca0 con 0x7f757815cdb0 2023-12-20T05:43:47.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f756e7fc640 1 -- 172.21.15.32:0/1704081514 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v158) v1 ==== 74+0+16120 (secure 0 0 0) 0x7f75780b6ca0 con 0x7f757815cdb0 2023-12-20T05:43:47.216 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:47.216 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":158,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:43:46.504917+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:43:44.235578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-20T05:43:45.508783+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:43:47.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 -- 172.21.15.32:0/1704081514 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 msgr2=0x7f755c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 0x7f755c068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f75740b2000 tx=0x7f7574008040 comp rx=0 tx=0).stop 2023-12-20T05:43:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 -- 172.21.15.32:0/1704081514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757815cdb0 msgr2=0x7f757815ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757815cdb0 0x7f757815ee50 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7f757000c7e0 tx=0x7f757000cca0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 -- 172.21.15.32:0/1704081514 shutdown_connections 2023-12-20T05:43:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f755c065b80 0x7f755c068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f757815cdb0 0x7f757815ee50 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f75780d2740 0x7f757815e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 --2- 172.21.15.32:0/1704081514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f75780d1120 0x7f757814f470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 -- 172.21.15.32:0/1704081514 >> 172.21.15.32:0/1704081514 conn(0x7f7578065ee0 msgr2=0x7f75780cc300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:47.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 -- 172.21.15.32:0/1704081514 shutdown_connections 2023-12-20T05:43:47.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.216+0000 7f757fa95640 1 -- 172.21.15.32:0/1704081514 wait complete. 2023-12-20T05:43:47.238 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T05:43:47.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba9e466640 1 Processor -- start 2023-12-20T05:43:47.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba9e466640 1 -- start start 2023-12-20T05:43:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba9e466640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9806d240 0x7fba980ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba9e466640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fba981603e0 0x7fba980ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba9e466640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba9806c480 con 0x7fba98070490 2023-12-20T05:43:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba9e466640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba9806c760 con 0x7fba981603e0 2023-12-20T05:43:47.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba9e466640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba9806ca40 con 0x7fba9806d240 2023-12-20T05:43:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba8ffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9806d240 0x7fba980ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba97fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fba98070490 0x7fba9815fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:36868/0 (socket says 172.21.15.32:36868) 2023-12-20T05:43:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba97fff640 1 -- 172.21.15.32:0/3047841065 learned_addr learned my addr 172.21.15.32:0/3047841065 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:47.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba8ffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9806d240 0x7fba980ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:46670/0 (socket says 172.21.15.32:46670) 2023-12-20T05:43:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba9ca03640 1 --2- 172.21.15.32:0/3047841065 >> v2:172.21.15.39:3300/0 conn(0x7fba981603e0 0x7fba980ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba8ffff640 1 -- 172.21.15.32:0/3047841065 >> v2:172.21.15.39:3300/0 conn(0x7fba981603e0 msgr2=0x7fba980ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba8ffff640 1 --2- 172.21.15.32:0/3047841065 >> v2:172.21.15.39:3300/0 conn(0x7fba981603e0 0x7fba980ca6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba8ffff640 1 -- 172.21.15.32:0/3047841065 >> v1:172.21.15.39:6789/0 conn(0x7fba98070490 legacy=0x7fba9815fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:47.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.384+0000 7fba8ffff640 1 -- 172.21.15.32:0/3047841065 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba9806cd20 con 0x7fba9806d240 2023-12-20T05:43:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba8ffff640 1 --2- 172.21.15.32:0/3047841065 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9806d240 0x7fba980ca1a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fba800097c0 tx=0x7fba8001c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1974a4ab8bfbb7d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba977fe640 1 -- 172.21.15.32:0/3047841065 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba800ab600 con 0x7fba9806d240 2023-12-20T05:43:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba977fe640 1 -- 172.21.15.32:0/3047841065 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba800a7050 con 0x7fba9806d240 2023-12-20T05:43:47.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba977fe640 1 -- 172.21.15.32:0/3047841065 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba800ab980 con 0x7fba9806d240 2023-12-20T05:43:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba9e466640 1 -- 172.21.15.32:0/3047841065 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9806d240 msgr2=0x7fba980ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba9e466640 1 --2- 172.21.15.32:0/3047841065 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9806d240 0x7fba980ca1a0 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fba800097c0 tx=0x7fba8001c3c0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba9e466640 1 -- 172.21.15.32:0/3047841065 shutdown_connections 2023-12-20T05:43:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba9e466640 1 --2- 172.21.15.32:0/3047841065 >> v2:172.21.15.39:3300/0 conn(0x7fba981603e0 0x7fba980ca6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba9e466640 1 --2- 172.21.15.32:0/3047841065 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba9806d240 0x7fba980ca1a0 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba9e466640 1 -- 172.21.15.32:0/3047841065 >> 172.21.15.32:0/3047841065 conn(0x7fba9806a510 msgr2=0x7fba980ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba9e466640 1 -- 172.21.15.32:0/3047841065 shutdown_connections 2023-12-20T05:43:47.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.388+0000 7fba9e466640 1 -- 172.21.15.32:0/3047841065 wait complete. 2023-12-20T05:43:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba9e466640 1 Processor -- start 2023-12-20T05:43:47.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba9e466640 1 -- start start 2023-12-20T05:43:47.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba9e466640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9806d240 0x7fba981443e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba9e466640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba98070490 0x7fba98154500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba9e466640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba981603e0 0x7fba981564a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba9e466640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba98144920 con 0x7fba9806d240 2023-12-20T05:43:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba9e466640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba98154ea0 con 0x7fba981603e0 2023-12-20T05:43:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba9e466640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba98155180 con 0x7fba98070490 2023-12-20T05:43:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba97fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9806d240 0x7fba981443e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba97fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9806d240 0x7fba981443e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60950/0 (socket says 172.21.15.32:60950) 2023-12-20T05:43:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba97fff640 1 -- 172.21.15.32:0/1839026290 learned_addr learned my addr 172.21.15.32:0/1839026290 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba8ffff640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba98070490 0x7fba98154500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba9ca03640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba981603e0 0x7fba981564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba97fff640 1 -- 172.21.15.32:0/1839026290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba98070490 msgr2=0x7fba98154500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba97fff640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba98070490 0x7fba98154500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba97fff640 1 -- 172.21.15.32:0/1839026290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba981603e0 msgr2=0x7fba981564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:47.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba97fff640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba981603e0 0x7fba981564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:47.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba97fff640 1 -- 172.21.15.32:0/1839026290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba80009420 con 0x7fba9806d240 2023-12-20T05:43:47.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba97fff640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9806d240 0x7fba981443e0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fba8800c500 tx=0x7fba8800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:47.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba957fa640 1 -- 172.21.15.32:0/1839026290 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba88005cf0 con 0x7fba9806d240 2023-12-20T05:43:47.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba957fa640 1 -- 172.21.15.32:0/1839026290 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba88005e50 con 0x7fba9806d240 2023-12-20T05:43:47.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba957fa640 1 -- 172.21.15.32:0/1839026290 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba880072f0 con 0x7fba9806d240 2023-12-20T05:43:47.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba9e466640 1 -- 172.21.15.32:0/1839026290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba98155460 con 0x7fba9806d240 2023-12-20T05:43:47.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.392+0000 7fba9e466640 1 -- 172.21.15.32:0/1839026290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba98155a20 con 0x7fba9806d240 2023-12-20T05:43:47.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.396+0000 7fba9e466640 1 -- 172.21.15.32:0/1839026290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba98155c10 con 0x7fba9806d240 2023-12-20T05:43:47.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.396+0000 7fba957fa640 1 -- 172.21.15.32:0/1839026290 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fba88007450 con 0x7fba9806d240 2023-12-20T05:43:47.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.396+0000 7fba957fa640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba78065b80 0x7fba78068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:47.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.396+0000 7fba957fa640 1 -- 172.21.15.32:0/1839026290 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(158..158 src has 1..158) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fba88082820 con 0x7fba9806d240 2023-12-20T05:43:47.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.396+0000 7fba8ffff640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba78065b80 0x7fba78068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:47.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.396+0000 7fba8ffff640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba78065b80 0x7fba78068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fba800ad000 tx=0x7fba80008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:47.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.400+0000 7fba957fa640 1 -- 172.21.15.32:0/1839026290 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fba88050150 con 0x7fba9806d240 2023-12-20T05:43:47.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:47.600+0000 7fba9e466640 1 -- 172.21.15.32:0/1839026290 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fba9804e000 con 0x7fba9806d240 2023-12-20T05:43:48.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.524+0000 7fba957fa640 1 -- 172.21.15.32:0/1839026290 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v70) v1 ==== 118+0+0 (secure 0 0 0) 0x7fba880170b0 con 0x7fba9806d240 2023-12-20T05:43:48.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 -- 172.21.15.32:0/1839026290 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba78065b80 msgr2=0x7fba78068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:48.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba78065b80 0x7fba78068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fba800ad000 tx=0x7fba80008040 comp rx=0 tx=0).stop 2023-12-20T05:43:48.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 -- 172.21.15.32:0/1839026290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9806d240 msgr2=0x7fba981443e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:48.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9806d240 0x7fba981443e0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fba8800c500 tx=0x7fba8800c9c0 comp rx=0 tx=0).stop 2023-12-20T05:43:48.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 -- 172.21.15.32:0/1839026290 shutdown_connections 2023-12-20T05:43:48.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba78065b80 0x7fba78068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:48.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba981603e0 0x7fba981564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:48.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba98070490 0x7fba98154500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:48.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 --2- 172.21.15.32:0/1839026290 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba9806d240 0x7fba981443e0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:48.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 -- 172.21.15.32:0/1839026290 >> 172.21.15.32:0/1839026290 conn(0x7fba9806a510 msgr2=0x7fba980b2cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:48.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 -- 172.21.15.32:0/1839026290 shutdown_connections 2023-12-20T05:43:48.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.528+0000 7fba9e466640 1 -- 172.21.15.32:0/1839026290 wait complete. 2023-12-20T05:43:48.552 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T05:43:48.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T05:43:48.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8c32c640 1 Processor -- start 2023-12-20T05:43:48.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8c32c640 1 -- start start 2023-12-20T05:43:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8c32c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d8415ce60 0x7f4d840d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8c32c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4d840d3130 0x7f4d840d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8c32c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4d840d2650 con 0x7f4d840d1300 2023-12-20T05:43:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8c32c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4d840d35a0 con 0x7f4d8415ce60 2023-12-20T05:43:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8c32c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840d1a40 con 0x7f4d840d3130 2023-12-20T05:43:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8a0c8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d8415ce60 0x7f4d840d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:48.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8a0c8640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4d8415ce60 0x7f4d840d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60958/0 (socket says 172.21.15.32:60958) 2023-12-20T05:43:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8a0c8640 1 -- 172.21.15.32:0/3528524387 learned_addr learned my addr 172.21.15.32:0/3528524387 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d898c7640 1 --2- 172.21.15.32:0/3528524387 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4d840d3130 0x7f4d840d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8a0c8640 1 -- 172.21.15.32:0/3528524387 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4d840d3130 msgr2=0x7f4d840d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8a0c8640 1 --2- 172.21.15.32:0/3528524387 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4d840d3130 0x7f4d840d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8a0c8640 1 -- 172.21.15.32:0/3528524387 >> v1:172.21.15.39:6789/0 conn(0x7f4d840d1300 legacy=0x7f4d840b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:48.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.704+0000 7f4d8a0c8640 1 -- 172.21.15.32:0/3528524387 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d840d1d20 con 0x7f4d8415ce60 2023-12-20T05:43:48.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d8a0c8640 1 --2- 172.21.15.32:0/3528524387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d8415ce60 0x7f4d840d2bf0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009870 tx=0x7f4d8001ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=659abfeb8ef2ad0a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:48.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d890c6640 1 -- 172.21.15.32:0/3528524387 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d800a8a30 con 0x7f4d8415ce60 2023-12-20T05:43:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d890c6640 1 -- 172.21.15.32:0/3528524387 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d800a8b90 con 0x7f4d8415ce60 2023-12-20T05:43:48.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d890c6640 1 -- 172.21.15.32:0/3528524387 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d800b0440 con 0x7f4d8415ce60 2023-12-20T05:43:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3528524387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d8415ce60 msgr2=0x7f4d840d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d8c32c640 1 --2- 172.21.15.32:0/3528524387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d8415ce60 0x7f4d840d2bf0 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7f4d80009870 tx=0x7f4d8001ed30 comp rx=0 tx=0).stop 2023-12-20T05:43:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3528524387 shutdown_connections 2023-12-20T05:43:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d8c32c640 1 --2- 172.21.15.32:0/3528524387 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4d840d3130 0x7f4d840d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d8c32c640 1 --2- 172.21.15.32:0/3528524387 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d8415ce60 0x7f4d840d2bf0 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:48.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3528524387 >> 172.21.15.32:0/3528524387 conn(0x7f4d8406b5e0 msgr2=0x7f4d8406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:48.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3528524387 shutdown_connections 2023-12-20T05:43:48.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.708+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3528524387 wait complete. 2023-12-20T05:43:48.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8c32c640 1 Processor -- start 2023-12-20T05:43:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8c32c640 1 -- start start 2023-12-20T05:43:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8c32c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d840d1300 0x7f4d841f2330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8c32c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d840d3130 0x7f4d841f51a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8c32c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4d8415ce60 0x7f4d840c9d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8c32c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d841f2870 con 0x7f4d840d3130 2023-12-20T05:43:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8c32c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4d841f56e0 con 0x7f4d840d1300 2023-12-20T05:43:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8c32c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4d840ca8a0 con 0x7f4d8415ce60 2023-12-20T05:43:48.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d898c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d840d3130 0x7f4d841f51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d898c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d840d3130 0x7f4d841f51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60970/0 (socket says 172.21.15.32:60970) 2023-12-20T05:43:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d898c7640 1 -- 172.21.15.32:0/3723076522 learned_addr learned my addr 172.21.15.32:0/3723076522 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d898c7640 1 -- 172.21.15.32:0/3723076522 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4d8415ce60 msgr2=0x7f4d840c9d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d898c7640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4d8415ce60 0x7f4d840c9d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8a0c8640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d840d1300 0x7f4d841f2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d898c7640 1 -- 172.21.15.32:0/3723076522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d840d1300 msgr2=0x7f4d841f2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d898c7640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d840d1300 0x7f4d841f2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:48.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d898c7640 1 -- 172.21.15.32:0/3723076522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4d800094d0 con 0x7f4d840d3130 2023-12-20T05:43:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8a0c8640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d840d1300 0x7f4d841f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d898c7640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d840d3130 0x7f4d841f51a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400e960 tx=0x7f4d7400ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d7affd640 1 -- 172.21.15.32:0/3723076522 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d74009690 con 0x7f4d840d3130 2023-12-20T05:43:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d7affd640 1 -- 172.21.15.32:0/3723076522 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4d7400ca00 con 0x7f4d840d3130 2023-12-20T05:43:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3723076522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4d840cab80 con 0x7f4d840d3130 2023-12-20T05:43:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d7affd640 1 -- 172.21.15.32:0/3723076522 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4d7400cd70 con 0x7f4d840d3130 2023-12-20T05:43:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.712+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3723076522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4d840cbde0 con 0x7f4d840d3130 2023-12-20T05:43:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.716+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3723076522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4d840cb960 con 0x7f4d840d3130 2023-12-20T05:43:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.716+0000 7f4d7affd640 1 -- 172.21.15.32:0/3723076522 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4d74013470 con 0x7f4d840d3130 2023-12-20T05:43:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.716+0000 7f4d7affd640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4d68065bb0 0x7f4d68068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.716+0000 7f4d7affd640 1 -- 172.21.15.32:0/3723076522 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4d74083660 con 0x7f4d840d3130 2023-12-20T05:43:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.716+0000 7f4d8a0c8640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4d68065bb0 0x7f4d68068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:48.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.716+0000 7f4d8a0c8640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4d68065bb0 0x7f4d68068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4d800b2000 tx=0x7f4d800aa070 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:48.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.720+0000 7f4d7affd640 1 -- 172.21.15.32:0/3723076522 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4d840cb960 con 0x7f4d840d3130 2023-12-20T05:43:48.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:48.912+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3723076522 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f4d840cb960 con 0x7f4d840d3130 2023-12-20T05:43:49.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d7affd640 1 -- 172.21.15.32:0/3723076522 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v71) v1 ==== 122+0+0 (secure 0 0 0) 0x7f4d840cb960 con 0x7f4d840d3130 2023-12-20T05:43:49.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3723076522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4d68065bb0 msgr2=0x7f4d68068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4d68065bb0 0x7f4d68068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4d800b2000 tx=0x7f4d800aa070 comp rx=0 tx=0).stop 2023-12-20T05:43:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3723076522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d840d3130 msgr2=0x7f4d841f51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d840d3130 0x7f4d841f51a0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7f4d7400e960 tx=0x7f4d7400ee20 comp rx=0 tx=0).stop 2023-12-20T05:43:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3723076522 shutdown_connections 2023-12-20T05:43:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4d68065bb0 0x7f4d68068070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4d8415ce60 0x7f4d840c9d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4d840d3130 0x7f4d841f51a0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 --2- 172.21.15.32:0/3723076522 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4d840d1300 0x7f4d841f2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:49.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3723076522 >> 172.21.15.32:0/3723076522 conn(0x7f4d8406b5e0 msgr2=0x7f4d840b1f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:49.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3723076522 shutdown_connections 2023-12-20T05:43:49.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.528+0000 7f4d8c32c640 1 -- 172.21.15.32:0/3723076522 wait complete. 2023-12-20T05:43:49.552 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T05:43:49.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:43:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.696+0000 7f6e8ffce640 1 Processor -- start 2023-12-20T05:43:49.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.696+0000 7f6e8ffce640 1 -- start start 2023-12-20T05:43:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.700+0000 7f6e8ffce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e880aece0 0x7f6e880b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.700+0000 7f6e8ffce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e880c98c0 con 0x7f6e881599e0 2023-12-20T05:43:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.700+0000 7f6e8ffce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6e880c9ba0 con 0x7f6e880b0b30 2023-12-20T05:43:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.700+0000 7f6e8ffce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6e880c9e80 con 0x7f6e880aece0 2023-12-20T05:43:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.700+0000 7f6e8d569640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e880aece0 0x7f6e880b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:49.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.700+0000 7f6e8d569640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6e880aece0 0x7f6e880b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33606/0 (socket says 172.21.15.32:33606) 2023-12-20T05:43:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.700+0000 7f6e8d569640 1 -- 172.21.15.32:0/3891420038 learned_addr learned my addr 172.21.15.32:0/3891420038 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.700+0000 7f6e8d569640 1 -- 172.21.15.32:0/3891420038 >> v1:172.21.15.39:6789/0 conn(0x7f6e880b0b30 legacy=0x7f6e880701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.700+0000 7f6e8d569640 1 -- 172.21.15.32:0/3891420038 >> v1:172.21.15.32:6789/0 conn(0x7f6e881599e0 legacy=0x7f6e880c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.700+0000 7f6e8d569640 1 -- 172.21.15.32:0/3891420038 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e880ca160 con 0x7f6e880aece0 2023-12-20T05:43:49.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.700+0000 7f6e8cd68640 1 -- 172.21.15.32:0/3891420038 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3099886916 0 0) 0x7f6e880c98c0 con 0x7f6e881599e0 2023-12-20T05:43:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8d569640 1 --2- 172.21.15.32:0/3891420038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e880aece0 0x7f6e880b05f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009870 tx=0x7f6e7801cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d276cfaaa29faf28 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8cd68640 1 -- 172.21.15.32:0/3891420038 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e780ab610 con 0x7f6e880aece0 2023-12-20T05:43:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8cd68640 1 -- 172.21.15.32:0/3891420038 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e780ab770 con 0x7f6e880aece0 2023-12-20T05:43:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8cd68640 1 -- 172.21.15.32:0/3891420038 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e780abb00 con 0x7f6e880aece0 2023-12-20T05:43:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 -- 172.21.15.32:0/3891420038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e880aece0 msgr2=0x7f6e880b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 --2- 172.21.15.32:0/3891420038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e880aece0 0x7f6e880b05f0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f6e78009870 tx=0x7f6e7801cf60 comp rx=0 tx=0).stop 2023-12-20T05:43:49.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 -- 172.21.15.32:0/3891420038 shutdown_connections 2023-12-20T05:43:49.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 --2- 172.21.15.32:0/3891420038 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e880aece0 0x7f6e880b05f0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:49.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 -- 172.21.15.32:0/3891420038 >> 172.21.15.32:0/3891420038 conn(0x7f6e8806a320 msgr2=0x7f6e880ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:49.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 -- 172.21.15.32:0/3891420038 shutdown_connections 2023-12-20T05:43:49.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 -- 172.21.15.32:0/3891420038 wait complete. 2023-12-20T05:43:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 Processor -- start 2023-12-20T05:43:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 -- start start 2023-12-20T05:43:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e880aece0 0x7f6e88152e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e880b0b30 0x7f6e880b1e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e881599e0 0x7f6e88148180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e881474c0 con 0x7f6e880aece0 2023-12-20T05:43:49.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8814e8f0 con 0x7f6e880b0b30 2023-12-20T05:43:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.704+0000 7f6e8ffce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6e8814ebd0 con 0x7f6e881599e0 2023-12-20T05:43:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8dd6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e880aece0 0x7f6e88152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8dd6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e880aece0 0x7f6e88152e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33610/0 (socket says 172.21.15.32:33610) 2023-12-20T05:43:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8dd6a640 1 -- 172.21.15.32:0/2996124002 learned_addr learned my addr 172.21.15.32:0/2996124002 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8dd6a640 1 -- 172.21.15.32:0/2996124002 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e881599e0 msgr2=0x7f6e88148180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8e56b640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e881599e0 0x7f6e88148180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:49.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8d569640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e880b0b30 0x7f6e880b1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8dd6a640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e881599e0 0x7f6e88148180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8dd6a640 1 -- 172.21.15.32:0/2996124002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e880b0b30 msgr2=0x7f6e880b1e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8dd6a640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e880b0b30 0x7f6e880b1e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8dd6a640 1 -- 172.21.15.32:0/2996124002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6e780094d0 con 0x7f6e880aece0 2023-12-20T05:43:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8d569640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e880b0b30 0x7f6e880b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8e56b640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e881599e0 0x7f6e88148180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8dd6a640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e880aece0 0x7f6e88152e80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c480 tx=0x7f6e8400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e7e7fc640 1 -- 172.21.15.32:0/2996124002 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84005480 con 0x7f6e880aece0 2023-12-20T05:43:49.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e7e7fc640 1 -- 172.21.15.32:0/2996124002 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6e84003600 con 0x7f6e880aece0 2023-12-20T05:43:49.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e7e7fc640 1 -- 172.21.15.32:0/2996124002 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6e84003a30 con 0x7f6e880aece0 2023-12-20T05:43:49.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8ffce640 1 -- 172.21.15.32:0/2996124002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6e881476a0 con 0x7f6e880aece0 2023-12-20T05:43:49.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8ffce640 1 -- 172.21.15.32:0/2996124002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6e88147c60 con 0x7f6e880aece0 2023-12-20T05:43:49.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e8ffce640 1 -- 172.21.15.32:0/2996124002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6e8814ed40 con 0x7f6e880aece0 2023-12-20T05:43:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e7e7fc640 1 -- 172.21.15.32:0/2996124002 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6e84003b90 con 0x7f6e880aece0 2023-12-20T05:43:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.708+0000 7f6e7e7fc640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e6c065b80 0x7f6e6c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.712+0000 7f6e7e7fc640 1 -- 172.21.15.32:0/2996124002 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6e84082520 con 0x7f6e880aece0 2023-12-20T05:43:49.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.712+0000 7f6e8d569640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e6c065b80 0x7f6e6c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:49.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.712+0000 7f6e8d569640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e6c065b80 0x7f6e6c068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6e780af000 tx=0x7f6e78008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:49.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.716+0000 7f6e7e7fc640 1 -- 172.21.15.32:0/2996124002 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6e8404fe40 con 0x7f6e880aece0 2023-12-20T05:43:49.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 -- 172.21.15.32:0/2996124002 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6e8804e000 con 0x7f6e880aece0 2023-12-20T05:43:49.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e7e7fc640 1 -- 172.21.15.32:0/2996124002 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f6e84053460 con 0x7f6e880aece0 2023-12-20T05:43:49.909 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:49.909 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2062259714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2062259714},{"type":"v1","addr":"172.21.15.39:6834","nonce":2062259714}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4210073096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":4210073096},{"type":"v1","addr":"172.21.15.39:6835","nonce":4210073096}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8913,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3547769022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3547769022},{"type":"v1","addr":"172.21.15.32:6835","nonce":3547769022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:43:46.505246+0000","modified":"2023-12-20T05:43:49.528294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8928":{"gid":8928,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3239941411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3239941411},{"type":"v1","addr":"172.21.15.32:6837","nonce":3239941411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-20T05:43:49.910 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 71 2023-12-20T05:43:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 -- 172.21.15.32:0/2996124002 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e6c065b80 msgr2=0x7f6e6c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:49.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e6c065b80 0x7f6e6c068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f6e780af000 tx=0x7f6e78008040 comp rx=0 tx=0).stop 2023-12-20T05:43:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 -- 172.21.15.32:0/2996124002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e880aece0 msgr2=0x7f6e88152e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e880aece0 0x7f6e88152e80 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f6e8400c480 tx=0x7f6e8400c940 comp rx=0 tx=0).stop 2023-12-20T05:43:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 -- 172.21.15.32:0/2996124002 shutdown_connections 2023-12-20T05:43:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6e6c065b80 0x7f6e6c068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6e881599e0 0x7f6e88148180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6e880b0b30 0x7f6e880b1e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 --2- 172.21.15.32:0/2996124002 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6e880aece0 0x7f6e88152e80 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 -- 172.21.15.32:0/2996124002 >> 172.21.15.32:0/2996124002 conn(0x7f6e8806a320 msgr2=0x7f6e880cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:49.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 -- 172.21.15.32:0/2996124002 shutdown_connections 2023-12-20T05:43:49.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:49.908+0000 7f6e8ffce640 1 -- 172.21.15.32:0/2996124002 wait complete. 2023-12-20T05:43:49.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:43:50.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.072+0000 7fb556996640 1 Processor -- start 2023-12-20T05:43:50.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.072+0000 7fb556996640 1 -- start start 2023-12-20T05:43:50.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb556996640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb5500af5e0 0x7fb5500af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb556996640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb55006ce60 0x7fb55015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb556996640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb55015f110 0x7fb55015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb556996640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55015bbd0 con 0x7fb55015f110 2023-12-20T05:43:50.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb556996640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb55015beb0 con 0x7fb5500af5e0 2023-12-20T05:43:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb556996640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb55015e910 con 0x7fb55006ce60 2023-12-20T05:43:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb554f33640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb55015f110 0x7fb55015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb554f33640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb55015f110 0x7fb55015c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33616/0 (socket says 172.21.15.32:33616) 2023-12-20T05:43:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb554f33640 1 -- 172.21.15.32:0/579780619 learned_addr learned my addr 172.21.15.32:0/579780619 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb54ffff640 1 --2- 172.21.15.32:0/579780619 >> v2:172.21.15.39:3300/0 conn(0x7fb5500af5e0 0x7fb5500af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb54f7fe640 1 --2- 172.21.15.32:0/579780619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb55006ce60 0x7fb55015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb554f33640 1 -- 172.21.15.32:0/579780619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb55006ce60 msgr2=0x7fb55015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb554f33640 1 --2- 172.21.15.32:0/579780619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb55006ce60 0x7fb55015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb554f33640 1 -- 172.21.15.32:0/579780619 >> v2:172.21.15.39:3300/0 conn(0x7fb5500af5e0 msgr2=0x7fb5500af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb554f33640 1 --2- 172.21.15.32:0/579780619 >> v2:172.21.15.39:3300/0 conn(0x7fb5500af5e0 0x7fb5500af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.076+0000 7fb554f33640 1 -- 172.21.15.32:0/579780619 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb55015ebf0 con 0x7fb55015f110 2023-12-20T05:43:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb554f33640 1 --2- 172.21.15.32:0/579780619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb55015f110 0x7fb55015c440 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb55006ac10 tx=0x7fb54801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7587053010e5e5f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb54effd640 1 -- 172.21.15.32:0/579780619 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5480a57b0 con 0x7fb55015f110 2023-12-20T05:43:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb54effd640 1 -- 172.21.15.32:0/579780619 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5480a5910 con 0x7fb55015f110 2023-12-20T05:43:50.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb54effd640 1 -- 172.21.15.32:0/579780619 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5480a5ca0 con 0x7fb55015f110 2023-12-20T05:43:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb556996640 1 -- 172.21.15.32:0/579780619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb55015f110 msgr2=0x7fb55015c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb556996640 1 --2- 172.21.15.32:0/579780619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb55015f110 0x7fb55015c440 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7fb55006ac10 tx=0x7fb54801c9d0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb556996640 1 -- 172.21.15.32:0/579780619 shutdown_connections 2023-12-20T05:43:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb556996640 1 --2- 172.21.15.32:0/579780619 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb55015f110 0x7fb55015c440 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb556996640 1 --2- 172.21.15.32:0/579780619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb55006ce60 0x7fb55015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb556996640 1 --2- 172.21.15.32:0/579780619 >> v2:172.21.15.39:3300/0 conn(0x7fb5500af5e0 0x7fb5500af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb556996640 1 -- 172.21.15.32:0/579780619 >> 172.21.15.32:0/579780619 conn(0x7fb55006a320 msgr2=0x7fb5500ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb556996640 1 -- 172.21.15.32:0/579780619 shutdown_connections 2023-12-20T05:43:50.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb556996640 1 -- 172.21.15.32:0/579780619 wait complete. 2023-12-20T05:43:50.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.080+0000 7fb556996640 1 Processor -- start 2023-12-20T05:43:50.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb556996640 1 -- start start 2023-12-20T05:43:50.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb556996640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb55006ce60 0x7fb5501f9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb556996640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5500af5e0 0x7fb5500c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb556996640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb55015f110 0x7fb5500c7f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb556996640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5500c5830 con 0x7fb55006ce60 2023-12-20T05:43:50.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb556996640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb5501fa1f0 con 0x7fb55015f110 2023-12-20T05:43:50.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb556996640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb5500c43e0 con 0x7fb5500af5e0 2023-12-20T05:43:50.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb55006ce60 0x7fb5501f9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb55006ce60 0x7fb5501f9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33618/0 (socket says 172.21.15.32:33618) 2023-12-20T05:43:50.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54ffff640 1 -- 172.21.15.32:0/1859457955 learned_addr learned my addr 172.21.15.32:0/1859457955 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:50.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54f7fe640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5500af5e0 0x7fb5500c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54ffff640 1 -- 172.21.15.32:0/1859457955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5500af5e0 msgr2=0x7fb5500c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb554f33640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb55015f110 0x7fb5500c7f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54ffff640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5500af5e0 0x7fb5500c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54ffff640 1 -- 172.21.15.32:0/1859457955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb55015f110 msgr2=0x7fb5500c7f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54ffff640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb55015f110 0x7fb5500c7f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54ffff640 1 -- 172.21.15.32:0/1859457955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb548009b30 con 0x7fb55006ce60 2023-12-20T05:43:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54f7fe640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5500af5e0 0x7fb5500c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:50.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54ffff640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb55006ce60 0x7fb5501f9cb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb54000e9b0 tx=0x7fb54000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54cff9640 1 -- 172.21.15.32:0/1859457955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb540009670 con 0x7fb55006ce60 2023-12-20T05:43:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54cff9640 1 -- 172.21.15.32:0/1859457955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb54000c9a0 con 0x7fb55006ce60 2023-12-20T05:43:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb54cff9640 1 -- 172.21.15.32:0/1859457955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb54000cd10 con 0x7fb55006ce60 2023-12-20T05:43:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb556996640 1 -- 172.21.15.32:0/1859457955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb5500c5a10 con 0x7fb55006ce60 2023-12-20T05:43:50.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb556996640 1 -- 172.21.15.32:0/1859457955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb5500c7730 con 0x7fb55006ce60 2023-12-20T05:43:50.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.084+0000 7fb556996640 1 -- 172.21.15.32:0/1859457955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb55004e000 con 0x7fb55006ce60 2023-12-20T05:43:50.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.088+0000 7fb54cff9640 1 -- 172.21.15.32:0/1859457955 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb540013470 con 0x7fb55006ce60 2023-12-20T05:43:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.088+0000 7fb54cff9640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb53c065b80 0x7fb53c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.088+0000 7fb54cff9640 1 -- 172.21.15.32:0/1859457955 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fb5400839e0 con 0x7fb55006ce60 2023-12-20T05:43:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.088+0000 7fb54f7fe640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb53c065b80 0x7fb53c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.088+0000 7fb54f7fe640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb53c065b80 0x7fb53c068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5500c7120 tx=0x7fb544008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:50.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.092+0000 7fb54cff9640 1 -- 172.21.15.32:0/1859457955 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb5400511d0 con 0x7fb55006ce60 2023-12-20T05:43:50.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 -- 172.21.15.32:0/1859457955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb5500afce0 con 0x7fb55006ce60 2023-12-20T05:43:50.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb54cff9640 1 -- 172.21.15.32:0/1859457955 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v159) v1 ==== 74+0+16107 (secure 0 0 0) 0x7fb540021090 con 0x7fb55006ce60 2023-12-20T05:43:50.253 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:50.253 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":159,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:43:47.508394+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:43:44.235578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-20T05:43:45.508783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:43:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 -- 172.21.15.32:0/1859457955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb53c065b80 msgr2=0x7fb53c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb53c065b80 0x7fb53c068040 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fb5500c7120 tx=0x7fb544008040 comp rx=0 tx=0).stop 2023-12-20T05:43:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 -- 172.21.15.32:0/1859457955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb55006ce60 msgr2=0x7fb5501f9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb55006ce60 0x7fb5501f9cb0 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7fb54000e9b0 tx=0x7fb54000ee70 comp rx=0 tx=0).stop 2023-12-20T05:43:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 -- 172.21.15.32:0/1859457955 shutdown_connections 2023-12-20T05:43:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fb53c065b80 0x7fb53c068040 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb55015f110 0x7fb5500c7f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb5500af5e0 0x7fb5500c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 --2- 172.21.15.32:0/1859457955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb55006ce60 0x7fb5501f9cb0 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 -- 172.21.15.32:0/1859457955 >> 172.21.15.32:0/1859457955 conn(0x7fb55006a320 msgr2=0x7fb5500ca910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 -- 172.21.15.32:0/1859457955 shutdown_connections 2023-12-20T05:43:50.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.252+0000 7fb556996640 1 -- 172.21.15.32:0/1859457955 wait complete. 2023-12-20T05:43:50.275 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T05:43:50.276 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T05:43:50.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdc0596640 1 Processor -- start 2023-12-20T05:43:50.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdc0596640 1 -- start start 2023-12-20T05:43:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdc0596640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb80af5e0 0x7efdb80af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdc0596640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdb806ce60 0x7efdb815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdc0596640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efdb815bbd0 con 0x7efdb815f110 2023-12-20T05:43:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdc0596640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efdb815beb0 con 0x7efdb806ce60 2023-12-20T05:43:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdc0596640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb815e910 con 0x7efdb80af5e0 2023-12-20T05:43:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdbdb31640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdb806ce60 0x7efdb815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdbdb31640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efdb806ce60 0x7efdb815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33628/0 (socket says 172.21.15.32:33628) 2023-12-20T05:43:50.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdbdb31640 1 -- 172.21.15.32:0/1438356564 learned_addr learned my addr 172.21.15.32:0/1438356564 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:50.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdbe332640 1 --2- 172.21.15.32:0/1438356564 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb80af5e0 0x7efdb80af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdbdb31640 1 -- 172.21.15.32:0/1438356564 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb80af5e0 msgr2=0x7efdb80af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:50.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdbdb31640 1 --2- 172.21.15.32:0/1438356564 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb80af5e0 0x7efdb80af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdbdb31640 1 -- 172.21.15.32:0/1438356564 >> v1:172.21.15.39:6789/0 conn(0x7efdb815f110 legacy=0x7efdb815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.420+0000 7efdbdb31640 1 -- 172.21.15.32:0/1438356564 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdb815ebf0 con 0x7efdb806ce60 2023-12-20T05:43:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdbdb31640 1 --2- 172.21.15.32:0/1438356564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb806ce60 0x7efdb815b280 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efda8009870 tx=0x7efda801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=912b2a308d1363f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdbd330640 1 -- 172.21.15.32:0/1438356564 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda80a6040 con 0x7efdb806ce60 2023-12-20T05:43:50.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdbd330640 1 -- 172.21.15.32:0/1438356564 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efda80aa600 con 0x7efdb806ce60 2023-12-20T05:43:50.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdbd330640 1 -- 172.21.15.32:0/1438356564 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efda80aa990 con 0x7efdb806ce60 2023-12-20T05:43:50.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdc0596640 1 -- 172.21.15.32:0/1438356564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb806ce60 msgr2=0x7efdb815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdc0596640 1 --2- 172.21.15.32:0/1438356564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb806ce60 0x7efdb815b280 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7efda8009870 tx=0x7efda801c540 comp rx=0 tx=0).stop 2023-12-20T05:43:50.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdc0596640 1 -- 172.21.15.32:0/1438356564 shutdown_connections 2023-12-20T05:43:50.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdc0596640 1 --2- 172.21.15.32:0/1438356564 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb806ce60 0x7efdb815b280 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdc0596640 1 --2- 172.21.15.32:0/1438356564 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb80af5e0 0x7efdb80af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdc0596640 1 -- 172.21.15.32:0/1438356564 >> 172.21.15.32:0/1438356564 conn(0x7efdb806a320 msgr2=0x7efdb80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:50.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdc0596640 1 -- 172.21.15.32:0/1438356564 shutdown_connections 2023-12-20T05:43:50.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.424+0000 7efdc0596640 1 -- 172.21.15.32:0/1438356564 wait complete. 2023-12-20T05:43:50.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdc0596640 1 Processor -- start 2023-12-20T05:43:50.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdc0596640 1 -- start start 2023-12-20T05:43:50.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdc0596640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb806ce60 0x7efdb8151260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdc0596640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb80af5e0 0x7efdb8070b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdc0596640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb815f110 0x7efdb8164ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdc0596640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80b3260 con 0x7efdb815f110 2023-12-20T05:43:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdc0596640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80b2650 con 0x7efdb80af5e0 2023-12-20T05:43:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdc0596640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efdb80b2930 con 0x7efdb806ce60 2023-12-20T05:43:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdbeb33640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb815f110 0x7efdb8164ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdbeb33640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb815f110 0x7efdb8164ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33638/0 (socket says 172.21.15.32:33638) 2023-12-20T05:43:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdbeb33640 1 -- 172.21.15.32:0/3566760718 learned_addr learned my addr 172.21.15.32:0/3566760718 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdbe332640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb806ce60 0x7efdb8151260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdbdb31640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb80af5e0 0x7efdb8070b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdbeb33640 1 -- 172.21.15.32:0/3566760718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb806ce60 msgr2=0x7efdb8151260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdbeb33640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb806ce60 0x7efdb8151260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdbeb33640 1 -- 172.21.15.32:0/3566760718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb80af5e0 msgr2=0x7efdb8070b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdbeb33640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb80af5e0 0x7efdb8070b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdbeb33640 1 -- 172.21.15.32:0/3566760718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efda80094d0 con 0x7efdb815f110 2023-12-20T05:43:50.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdbeb33640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb815f110 0x7efdb8164ee0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c730 tx=0x7efdb000cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdaeffd640 1 -- 172.21.15.32:0/3566760718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb0003b80 con 0x7efdb815f110 2023-12-20T05:43:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdaeffd640 1 -- 172.21.15.32:0/3566760718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdb0003ce0 con 0x7efdb815f110 2023-12-20T05:43:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdaeffd640 1 -- 172.21.15.32:0/3566760718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdb000aa30 con 0x7efdb815f110 2023-12-20T05:43:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdc0596640 1 -- 172.21.15.32:0/3566760718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efdb80b2c10 con 0x7efdb815f110 2023-12-20T05:43:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdc0596640 1 -- 172.21.15.32:0/3566760718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efdb8072160 con 0x7efdb815f110 2023-12-20T05:43:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.428+0000 7efdc0596640 1 -- 172.21.15.32:0/3566760718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efdb80b33d0 con 0x7efdb815f110 2023-12-20T05:43:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.432+0000 7efdaeffd640 1 -- 172.21.15.32:0/3566760718 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7efdb000ab90 con 0x7efdb815f110 2023-12-20T05:43:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.432+0000 7efdaeffd640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd9c065b80 0x7efd9c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.432+0000 7efdaeffd640 1 -- 172.21.15.32:0/3566760718 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7efdb0082f80 con 0x7efdb815f110 2023-12-20T05:43:50.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.432+0000 7efdbe332640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd9c065b80 0x7efd9c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.432+0000 7efdbe332640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd9c065b80 0x7efd9c068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efdb4006fd0 tx=0x7efdb4008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.436+0000 7efdaeffd640 1 -- 172.21.15.32:0/3566760718 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efdb00b40a0 con 0x7efdb815f110 2023-12-20T05:43:50.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.592+0000 7efdc0596640 1 -- 172.21.15.32:0/3566760718 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7efdb8158af0 con 0x7efd9c065b80 2023-12-20T05:43:50.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.604+0000 7efdaeffd640 1 -- 172.21.15.32:0/3566760718 <== mgr.8691 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289398 (secure 0 0 0) 0x7efdb40059a0 con 0x7efd9c065b80 2023-12-20T05:43:50.609 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:50.612 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T05:43:50.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 -- 172.21.15.32:0/3566760718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd9c065b80 msgr2=0x7efd9c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd9c065b80 0x7efd9c068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7efdb4006fd0 tx=0x7efdb4008040 comp rx=0 tx=0).stop 2023-12-20T05:43:50.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 -- 172.21.15.32:0/3566760718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb815f110 msgr2=0x7efdb8164ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb815f110 0x7efdb8164ee0 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7efdb000c730 tx=0x7efdb000cbf0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 -- 172.21.15.32:0/3566760718 shutdown_connections 2023-12-20T05:43:50.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7efd9c065b80 0x7efd9c068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efdb815f110 0x7efdb8164ee0 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efdb80af5e0 0x7efdb8070b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 --2- 172.21.15.32:0/3566760718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efdb806ce60 0x7efdb8151260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 -- 172.21.15.32:0/3566760718 >> 172.21.15.32:0/3566760718 conn(0x7efdb806a320 msgr2=0x7efdb80c49b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:50.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 -- 172.21.15.32:0/3566760718 shutdown_connections 2023-12-20T05:43:50.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.608+0000 7efdc0596640 1 -- 172.21.15.32:0/3566760718 wait complete. 2023-12-20T05:43:50.628 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-12-20T05:43:49.747641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1213,"num_read_kb":2407,"num_write":425,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3162,"ondisk_log_size":3162,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":367320,"kb_used_data":16408,"kb_used_omap":0,"kb_used_meta":350848,"kb_avail":754607400,"statfs":{"total":773094113280,"available":772717977600,"internally_reserved":0,"allocated":16801792,"data_stored":8657934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":359268352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.655901"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542762+0000","last_change":"2023-12-20T05:43:46.539209+0000","last_active":"2023-12-20T05:43:46.542762+0000","last_peered":"2023-12-20T05:43:46.542762+0000","last_clean":"2023-12-20T05:43:46.542762+0000","last_became_active":"2023-12-20T05:43:46.537002+0000","last_became_peered":"2023-12-20T05:43:46.537002+0000","last_unstale":"2023-12-20T05:43:46.542762+0000","last_undegraded":"2023-12-20T05:43:46.542762+0000","last_fullsized":"2023-12-20T05:43:46.542762+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:43:16.474732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545093+0000","last_change":"2023-12-20T05:43:46.543541+0000","last_active":"2023-12-20T05:43:46.545093+0000","last_peered":"2023-12-20T05:43:46.545093+0000","last_clean":"2023-12-20T05:43:46.545093+0000","last_became_active":"2023-12-20T05:43:46.543218+0000","last_became_peered":"2023-12-20T05:43:46.543218+0000","last_unstale":"2023-12-20T05:43:46.545093+0000","last_undegraded":"2023-12-20T05:43:46.545093+0000","last_fullsized":"2023-12-20T05:43:46.545093+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:40:42.139649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545229+0000","last_change":"2023-12-20T05:43:46.540512+0000","last_active":"2023-12-20T05:43:46.545229+0000","last_peered":"2023-12-20T05:43:46.545229+0000","last_clean":"2023-12-20T05:43:46.545229+0000","last_became_active":"2023-12-20T05:43:46.540239+0000","last_became_peered":"2023-12-20T05:43:46.540239+0000","last_unstale":"2023-12-20T05:43:46.545229+0000","last_undegraded":"2023-12-20T05:43:46.545229+0000","last_fullsized":"2023-12-20T05:43:46.545229+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:42.083442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.539665+0000","last_change":"2023-12-20T05:43:46.530287+0000","last_active":"2023-12-20T05:43:46.539665+0000","last_peered":"2023-12-20T05:43:46.539665+0000","last_clean":"2023-12-20T05:43:46.539665+0000","last_became_active":"2023-12-20T05:43:46.529997+0000","last_became_peered":"2023-12-20T05:43:46.529997+0000","last_unstale":"2023-12-20T05:43:46.539665+0000","last_undegraded":"2023-12-20T05:43:46.539665+0000","last_fullsized":"2023-12-20T05:43:46.539665+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:37:46.626012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542585+0000","last_change":"2023-12-20T05:43:46.533306+0000","last_active":"2023-12-20T05:43:46.542585+0000","last_peered":"2023-12-20T05:43:46.542585+0000","last_clean":"2023-12-20T05:43:46.542585+0000","last_became_active":"2023-12-20T05:43:46.532672+0000","last_became_peered":"2023-12-20T05:43:46.532672+0000","last_unstale":"2023-12-20T05:43:46.542585+0000","last_undegraded":"2023-12-20T05:43:46.542585+0000","last_fullsized":"2023-12-20T05:43:46.542585+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:58:35.971693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.544861+0000","last_change":"2023-12-20T05:43:46.540210+0000","last_active":"2023-12-20T05:43:46.544861+0000","last_peered":"2023-12-20T05:43:46.544861+0000","last_clean":"2023-12-20T05:43:46.544861+0000","last_became_active":"2023-12-20T05:43:46.539831+0000","last_became_peered":"2023-12-20T05:43:46.539831+0000","last_unstale":"2023-12-20T05:43:46.544861+0000","last_undegraded":"2023-12-20T05:43:46.544861+0000","last_fullsized":"2023-12-20T05:43:46.544861+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:48:16.672093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.550222+0000","last_change":"2023-12-20T05:43:46.549425+0000","last_active":"2023-12-20T05:43:46.550222+0000","last_peered":"2023-12-20T05:43:46.550222+0000","last_clean":"2023-12-20T05:43:46.550222+0000","last_became_active":"2023-12-20T05:43:46.549132+0000","last_became_peered":"2023-12-20T05:43:46.549132+0000","last_unstale":"2023-12-20T05:43:46.550222+0000","last_undegraded":"2023-12-20T05:43:46.550222+0000","last_fullsized":"2023-12-20T05:43:46.550222+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:54.706797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.551653+0000","last_change":"2023-12-20T05:43:46.548423+0000","last_active":"2023-12-20T05:43:46.551653+0000","last_peered":"2023-12-20T05:43:46.551653+0000","last_clean":"2023-12-20T05:43:46.551653+0000","last_became_active":"2023-12-20T05:43:46.547993+0000","last_became_peered":"2023-12-20T05:43:46.547993+0000","last_unstale":"2023-12-20T05:43:46.551653+0000","last_undegraded":"2023-12-20T05:43:46.551653+0000","last_fullsized":"2023-12-20T05:43:46.551653+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:56.569699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.548332+0000","last_change":"2023-12-20T05:43:46.544259+0000","last_active":"2023-12-20T05:43:46.548332+0000","last_peered":"2023-12-20T05:43:46.548332+0000","last_clean":"2023-12-20T05:43:46.548332+0000","last_became_active":"2023-12-20T05:43:46.543983+0000","last_became_peered":"2023-12-20T05:43:46.543983+0000","last_unstale":"2023-12-20T05:43:46.548332+0000","last_undegraded":"2023-12-20T05:43:46.548332+0000","last_fullsized":"2023-12-20T05:43:46.548332+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:47:52.037903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.539075+0000","last_change":"2023-12-20T05:43:46.537962+0000","last_active":"2023-12-20T05:43:46.539075+0000","last_peered":"2023-12-20T05:43:46.539075+0000","last_clean":"2023-12-20T05:43:46.539075+0000","last_became_active":"2023-12-20T05:43:46.536562+0000","last_became_peered":"2023-12-20T05:43:46.536562+0000","last_unstale":"2023-12-20T05:43:46.539075+0000","last_undegraded":"2023-12-20T05:43:46.539075+0000","last_fullsized":"2023-12-20T05:43:46.539075+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:44:43.156025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.535050+0000","last_change":"2023-12-20T05:43:46.526104+0000","last_active":"2023-12-20T05:43:46.535050+0000","last_peered":"2023-12-20T05:43:46.535050+0000","last_clean":"2023-12-20T05:43:46.535050+0000","last_became_active":"2023-12-20T05:43:46.525849+0000","last_became_peered":"2023-12-20T05:43:46.525849+0000","last_unstale":"2023-12-20T05:43:46.535050+0000","last_undegraded":"2023-12-20T05:43:46.535050+0000","last_fullsized":"2023-12-20T05:43:46.535050+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:21.854766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.551817+0000","last_change":"2023-12-20T05:43:46.549260+0000","last_active":"2023-12-20T05:43:46.551817+0000","last_peered":"2023-12-20T05:43:46.551817+0000","last_clean":"2023-12-20T05:43:46.551817+0000","last_became_active":"2023-12-20T05:43:46.548892+0000","last_became_peered":"2023-12-20T05:43:46.548892+0000","last_unstale":"2023-12-20T05:43:46.551817+0000","last_undegraded":"2023-12-20T05:43:46.551817+0000","last_fullsized":"2023-12-20T05:43:46.551817+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:08:27.340043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545459+0000","last_change":"2023-12-20T05:43:46.544121+0000","last_active":"2023-12-20T05:43:46.545459+0000","last_peered":"2023-12-20T05:43:46.545459+0000","last_clean":"2023-12-20T05:43:46.545459+0000","last_became_active":"2023-12-20T05:43:46.543467+0000","last_became_peered":"2023-12-20T05:43:46.543467+0000","last_unstale":"2023-12-20T05:43:46.545459+0000","last_undegraded":"2023-12-20T05:43:46.545459+0000","last_fullsized":"2023-12-20T05:43:46.545459+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:01:47.687795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.531707+0000","last_change":"2023-12-20T05:43:46.528316+0000","last_active":"2023-12-20T05:43:46.531707+0000","last_peered":"2023-12-20T05:43:46.531707+0000","last_clean":"2023-12-20T05:43:46.531707+0000","last_became_active":"2023-12-20T05:43:46.528011+0000","last_became_peered":"2023-12-20T05:43:46.528011+0000","last_unstale":"2023-12-20T05:43:46.531707+0000","last_undegraded":"2023-12-20T05:43:46.531707+0000","last_fullsized":"2023-12-20T05:43:46.531707+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:46.567327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.550279+0000","last_change":"2023-12-20T05:43:46.549107+0000","last_active":"2023-12-20T05:43:46.550279+0000","last_peered":"2023-12-20T05:43:46.550279+0000","last_clean":"2023-12-20T05:43:46.550279+0000","last_became_active":"2023-12-20T05:43:46.548833+0000","last_became_peered":"2023-12-20T05:43:46.548833+0000","last_unstale":"2023-12-20T05:43:46.550279+0000","last_undegraded":"2023-12-20T05:43:46.550279+0000","last_fullsized":"2023-12-20T05:43:46.550279+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:55:41.406815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.540590+0000","last_change":"2023-12-20T05:43:46.535359+0000","last_active":"2023-12-20T05:43:46.540590+0000","last_peered":"2023-12-20T05:43:46.540590+0000","last_clean":"2023-12-20T05:43:46.540590+0000","last_became_active":"2023-12-20T05:43:46.532959+0000","last_became_peered":"2023-12-20T05:43:46.532959+0000","last_unstale":"2023-12-20T05:43:46.540590+0000","last_undegraded":"2023-12-20T05:43:46.540590+0000","last_fullsized":"2023-12-20T05:43:46.540590+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:06:33.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538820+0000","last_change":"2023-12-20T05:43:46.532100+0000","last_active":"2023-12-20T05:43:46.538820+0000","last_peered":"2023-12-20T05:43:46.538820+0000","last_clean":"2023-12-20T05:43:46.538820+0000","last_became_active":"2023-12-20T05:43:46.531820+0000","last_became_peered":"2023-12-20T05:43:46.531820+0000","last_unstale":"2023-12-20T05:43:46.538820+0000","last_undegraded":"2023-12-20T05:43:46.538820+0000","last_fullsized":"2023-12-20T05:43:46.538820+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:24:17.148686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.543158+0000","last_change":"2023-12-20T05:43:46.541857+0000","last_active":"2023-12-20T05:43:46.543158+0000","last_peered":"2023-12-20T05:43:46.543158+0000","last_clean":"2023-12-20T05:43:46.543158+0000","last_became_active":"2023-12-20T05:43:46.541554+0000","last_became_peered":"2023-12-20T05:43:46.541554+0000","last_unstale":"2023-12-20T05:43:46.543158+0000","last_undegraded":"2023-12-20T05:43:46.543158+0000","last_fullsized":"2023-12-20T05:43:46.543158+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:26:51.798310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.543202+0000","last_change":"2023-12-20T05:43:46.537502+0000","last_active":"2023-12-20T05:43:46.543202+0000","last_peered":"2023-12-20T05:43:46.543202+0000","last_clean":"2023-12-20T05:43:46.543202+0000","last_became_active":"2023-12-20T05:43:46.534874+0000","last_became_peered":"2023-12-20T05:43:46.534874+0000","last_unstale":"2023-12-20T05:43:46.543202+0000","last_undegraded":"2023-12-20T05:43:46.543202+0000","last_fullsized":"2023-12-20T05:43:46.543202+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:11.087920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.530353+0000","last_change":"2023-12-20T05:43:46.526615+0000","last_active":"2023-12-20T05:43:46.530353+0000","last_peered":"2023-12-20T05:43:46.530353+0000","last_clean":"2023-12-20T05:43:46.530353+0000","last_became_active":"2023-12-20T05:43:46.526331+0000","last_became_peered":"2023-12-20T05:43:46.526331+0000","last_unstale":"2023-12-20T05:43:46.530353+0000","last_undegraded":"2023-12-20T05:43:46.530353+0000","last_fullsized":"2023-12-20T05:43:46.530353+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:57:56.867666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542608+0000","last_change":"2023-12-20T05:43:46.533508+0000","last_active":"2023-12-20T05:43:46.542608+0000","last_peered":"2023-12-20T05:43:46.542608+0000","last_clean":"2023-12-20T05:43:46.542608+0000","last_became_active":"2023-12-20T05:43:46.533031+0000","last_became_peered":"2023-12-20T05:43:46.533031+0000","last_unstale":"2023-12-20T05:43:46.542608+0000","last_undegraded":"2023-12-20T05:43:46.542608+0000","last_fullsized":"2023-12-20T05:43:46.542608+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:19:45.792519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.534505+0000","last_change":"2023-12-20T05:43:46.525527+0000","last_active":"2023-12-20T05:43:46.534505+0000","last_peered":"2023-12-20T05:43:46.534505+0000","last_clean":"2023-12-20T05:43:46.534505+0000","last_became_active":"2023-12-20T05:43:46.525199+0000","last_became_peered":"2023-12-20T05:43:46.525199+0000","last_unstale":"2023-12-20T05:43:46.534505+0000","last_undegraded":"2023-12-20T05:43:46.534505+0000","last_fullsized":"2023-12-20T05:43:46.534505+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:21:16.795497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.548347+0000","last_change":"2023-12-20T05:43:46.543802+0000","last_active":"2023-12-20T05:43:46.548347+0000","last_peered":"2023-12-20T05:43:46.548347+0000","last_clean":"2023-12-20T05:43:46.548347+0000","last_became_active":"2023-12-20T05:43:46.543390+0000","last_became_peered":"2023-12-20T05:43:46.543390+0000","last_unstale":"2023-12-20T05:43:46.548347+0000","last_undegraded":"2023-12-20T05:43:46.548347+0000","last_fullsized":"2023-12-20T05:43:46.548347+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:20:55.122432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.533736+0000","last_change":"2023-12-20T05:43:46.527292+0000","last_active":"2023-12-20T05:43:46.533736+0000","last_peered":"2023-12-20T05:43:46.533736+0000","last_clean":"2023-12-20T05:43:46.533736+0000","last_became_active":"2023-12-20T05:43:46.527009+0000","last_became_peered":"2023-12-20T05:43:46.527009+0000","last_unstale":"2023-12-20T05:43:46.533736+0000","last_undegraded":"2023-12-20T05:43:46.533736+0000","last_fullsized":"2023-12-20T05:43:46.533736+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:28:52.808329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538951+0000","last_change":"2023-12-20T05:43:46.536485+0000","last_active":"2023-12-20T05:43:46.538951+0000","last_peered":"2023-12-20T05:43:46.538951+0000","last_clean":"2023-12-20T05:43:46.538951+0000","last_became_active":"2023-12-20T05:43:46.536227+0000","last_became_peered":"2023-12-20T05:43:46.536227+0000","last_unstale":"2023-12-20T05:43:46.538951+0000","last_undegraded":"2023-12-20T05:43:46.538951+0000","last_fullsized":"2023-12-20T05:43:46.538951+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:15:17.571592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.534254+0000","last_change":"2023-12-20T05:43:46.532421+0000","last_active":"2023-12-20T05:43:46.534254+0000","last_peered":"2023-12-20T05:43:46.534254+0000","last_clean":"2023-12-20T05:43:46.534254+0000","last_became_active":"2023-12-20T05:43:46.530999+0000","last_became_peered":"2023-12-20T05:43:46.530999+0000","last_unstale":"2023-12-20T05:43:46.534254+0000","last_undegraded":"2023-12-20T05:43:46.534254+0000","last_fullsized":"2023-12-20T05:43:46.534254+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:29:31.310692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542598+0000","last_change":"2023-12-20T05:43:46.539025+0000","last_active":"2023-12-20T05:43:46.542598+0000","last_peered":"2023-12-20T05:43:46.542598+0000","last_clean":"2023-12-20T05:43:46.542598+0000","last_became_active":"2023-12-20T05:43:46.537890+0000","last_became_peered":"2023-12-20T05:43:46.537890+0000","last_unstale":"2023-12-20T05:43:46.542598+0000","last_undegraded":"2023-12-20T05:43:46.542598+0000","last_fullsized":"2023-12-20T05:43:46.542598+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:02:46.647862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.525454+0000","last_change":"2023-12-20T05:43:46.520782+0000","last_active":"2023-12-20T05:43:46.525454+0000","last_peered":"2023-12-20T05:43:46.525454+0000","last_clean":"2023-12-20T05:43:46.525454+0000","last_became_active":"2023-12-20T05:43:46.520462+0000","last_became_peered":"2023-12-20T05:43:46.520462+0000","last_unstale":"2023-12-20T05:43:46.525454+0000","last_undegraded":"2023-12-20T05:43:46.525454+0000","last_fullsized":"2023-12-20T05:43:46.525454+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:56.569699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.547340+0000","last_change":"2023-12-20T05:43:46.546781+0000","last_active":"2023-12-20T05:43:46.547340+0000","last_peered":"2023-12-20T05:43:46.547340+0000","last_clean":"2023-12-20T05:43:46.547340+0000","last_became_active":"2023-12-20T05:43:46.546477+0000","last_became_peered":"2023-12-20T05:43:46.546477+0000","last_unstale":"2023-12-20T05:43:46.547340+0000","last_undegraded":"2023-12-20T05:43:46.547340+0000","last_fullsized":"2023-12-20T05:43:46.547340+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:21:31.569553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.549714+0000","last_change":"2023-12-20T05:43:46.548897+0000","last_active":"2023-12-20T05:43:46.549714+0000","last_peered":"2023-12-20T05:43:46.549714+0000","last_clean":"2023-12-20T05:43:46.549714+0000","last_became_active":"2023-12-20T05:43:46.548613+0000","last_became_peered":"2023-12-20T05:43:46.548613+0000","last_unstale":"2023-12-20T05:43:46.549714+0000","last_undegraded":"2023-12-20T05:43:46.549714+0000","last_fullsized":"2023-12-20T05:43:46.549714+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:28.105274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545108+0000","last_change":"2023-12-20T05:43:46.536793+0000","last_active":"2023-12-20T05:43:46.545108+0000","last_peered":"2023-12-20T05:43:46.545108+0000","last_clean":"2023-12-20T05:43:46.545108+0000","last_became_active":"2023-12-20T05:43:46.536520+0000","last_became_peered":"2023-12-20T05:43:46.536520+0000","last_unstale":"2023-12-20T05:43:46.545108+0000","last_undegraded":"2023-12-20T05:43:46.545108+0000","last_fullsized":"2023-12-20T05:43:46.545108+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:42:15.654028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538071+0000","last_change":"2023-12-20T05:43:46.531443+0000","last_active":"2023-12-20T05:43:46.538071+0000","last_peered":"2023-12-20T05:43:46.538071+0000","last_clean":"2023-12-20T05:43:46.538071+0000","last_became_active":"2023-12-20T05:43:46.531105+0000","last_became_peered":"2023-12-20T05:43:46.531105+0000","last_unstale":"2023-12-20T05:43:46.538071+0000","last_undegraded":"2023-12-20T05:43:46.538071+0000","last_fullsized":"2023-12-20T05:43:46.538071+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:13:51.694337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.530123+0000","last_change":"2023-12-20T05:43:46.524217+0000","last_active":"2023-12-20T05:43:46.530123+0000","last_peered":"2023-12-20T05:43:46.530123+0000","last_clean":"2023-12-20T05:43:46.530123+0000","last_became_active":"2023-12-20T05:43:46.523954+0000","last_became_peered":"2023-12-20T05:43:46.523954+0000","last_unstale":"2023-12-20T05:43:46.530123+0000","last_undegraded":"2023-12-20T05:43:46.530123+0000","last_fullsized":"2023-12-20T05:43:46.530123+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:50.494257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"158'4","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.544012+0000","last_change":"2023-12-20T05:43:46.114332+0000","last_active":"2023-12-20T05:43:46.544012+0000","last_peered":"2023-12-20T05:43:46.544012+0000","last_clean":"2023-12-20T05:43:46.544012+0000","last_became_active":"2023-12-20T05:43:46.114032+0000","last_became_peered":"2023-12-20T05:43:46.114032+0000","last_unstale":"2023-12-20T05:43:46.544012+0000","last_undegraded":"2023-12-20T05:43:46.544012+0000","last_fullsized":"2023-12-20T05:43:46.544012+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:01:34.225881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.547970+0000","last_change":"2023-12-20T05:43:46.542973+0000","last_active":"2023-12-20T05:43:46.547970+0000","last_peered":"2023-12-20T05:43:46.547970+0000","last_clean":"2023-12-20T05:43:46.547970+0000","last_became_active":"2023-12-20T05:43:46.542177+0000","last_became_peered":"2023-12-20T05:43:46.542177+0000","last_unstale":"2023-12-20T05:43:46.547970+0000","last_undegraded":"2023-12-20T05:43:46.547970+0000","last_fullsized":"2023-12-20T05:43:46.547970+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:21.430825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"158'1","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.539453+0000","last_change":"2023-12-20T05:43:46.119329+0000","last_active":"2023-12-20T05:43:46.539453+0000","last_peered":"2023-12-20T05:43:46.539453+0000","last_clean":"2023-12-20T05:43:46.539453+0000","last_became_active":"2023-12-20T05:43:46.119034+0000","last_became_peered":"2023-12-20T05:43:46.119034+0000","last_unstale":"2023-12-20T05:43:46.539453+0000","last_undegraded":"2023-12-20T05:43:46.539453+0000","last_fullsized":"2023-12-20T05:43:46.539453+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:01:34.225881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542895+0000","last_change":"2023-12-20T05:43:46.534616+0000","last_active":"2023-12-20T05:43:46.542895+0000","last_peered":"2023-12-20T05:43:46.542895+0000","last_clean":"2023-12-20T05:43:46.542895+0000","last_became_active":"2023-12-20T05:43:46.534337+0000","last_became_peered":"2023-12-20T05:43:46.534337+0000","last_unstale":"2023-12-20T05:43:46.542895+0000","last_undegraded":"2023-12-20T05:43:46.542895+0000","last_fullsized":"2023-12-20T05:43:46.542895+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:20:15.408520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542493+0000","last_change":"2023-12-20T05:43:46.116964+0000","last_active":"2023-12-20T05:43:46.542493+0000","last_peered":"2023-12-20T05:43:46.542493+0000","last_clean":"2023-12-20T05:43:46.542493+0000","last_became_active":"2023-12-20T05:43:46.114237+0000","last_became_peered":"2023-12-20T05:43:46.114237+0000","last_unstale":"2023-12-20T05:43:46.542493+0000","last_undegraded":"2023-12-20T05:43:46.542493+0000","last_fullsized":"2023-12-20T05:43:46.542493+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:37:47.327725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545452+0000","last_change":"2023-12-20T05:43:46.532968+0000","last_active":"2023-12-20T05:43:46.545452+0000","last_peered":"2023-12-20T05:43:46.545452+0000","last_clean":"2023-12-20T05:43:46.545452+0000","last_became_active":"2023-12-20T05:43:46.532716+0000","last_became_peered":"2023-12-20T05:43:46.532716+0000","last_unstale":"2023-12-20T05:43:46.545452+0000","last_undegraded":"2023-12-20T05:43:46.545452+0000","last_fullsized":"2023-12-20T05:43:46.545452+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:39:20.211360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.543608+0000","last_change":"2023-12-20T05:43:46.115735+0000","last_active":"2023-12-20T05:43:46.543608+0000","last_peered":"2023-12-20T05:43:46.543608+0000","last_clean":"2023-12-20T05:43:46.543608+0000","last_became_active":"2023-12-20T05:43:46.115438+0000","last_became_peered":"2023-12-20T05:43:46.115438+0000","last_unstale":"2023-12-20T05:43:46.543608+0000","last_undegraded":"2023-12-20T05:43:46.543608+0000","last_fullsized":"2023-12-20T05:43:46.543608+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:02:45.645010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.540383+0000","last_change":"2023-12-20T05:43:46.534307+0000","last_active":"2023-12-20T05:43:46.540383+0000","last_peered":"2023-12-20T05:43:46.540383+0000","last_clean":"2023-12-20T05:43:46.540383+0000","last_became_active":"2023-12-20T05:43:46.534056+0000","last_became_peered":"2023-12-20T05:43:46.534056+0000","last_unstale":"2023-12-20T05:43:46.540383+0000","last_undegraded":"2023-12-20T05:43:46.540383+0000","last_fullsized":"2023-12-20T05:43:46.540383+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:37:28.001287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"158'1","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.536962+0000","last_change":"2023-12-20T05:43:46.114483+0000","last_active":"2023-12-20T05:43:46.536962+0000","last_peered":"2023-12-20T05:43:46.536962+0000","last_clean":"2023-12-20T05:43:46.536962+0000","last_became_active":"2023-12-20T05:43:46.114199+0000","last_became_peered":"2023-12-20T05:43:46.114199+0000","last_unstale":"2023-12-20T05:43:46.536962+0000","last_undegraded":"2023-12-20T05:43:46.536962+0000","last_fullsized":"2023-12-20T05:43:46.536962+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:51.446095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.541268+0000","last_change":"2023-12-20T05:43:46.538206+0000","last_active":"2023-12-20T05:43:46.541268+0000","last_peered":"2023-12-20T05:43:46.541268+0000","last_clean":"2023-12-20T05:43:46.541268+0000","last_became_active":"2023-12-20T05:43:46.537820+0000","last_became_peered":"2023-12-20T05:43:46.537820+0000","last_unstale":"2023-12-20T05:43:46.541268+0000","last_undegraded":"2023-12-20T05:43:46.541268+0000","last_fullsized":"2023-12-20T05:43:46.541268+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:15:33.061793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"158'1","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.537634+0000","last_change":"2023-12-20T05:43:46.115868+0000","last_active":"2023-12-20T05:43:46.537634+0000","last_peered":"2023-12-20T05:43:46.537634+0000","last_clean":"2023-12-20T05:43:46.537634+0000","last_became_active":"2023-12-20T05:43:46.115606+0000","last_became_peered":"2023-12-20T05:43:46.115606+0000","last_unstale":"2023-12-20T05:43:46.537634+0000","last_undegraded":"2023-12-20T05:43:46.537634+0000","last_fullsized":"2023-12-20T05:43:46.537634+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:43:59.588695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.547384+0000","last_change":"2023-12-20T05:43:46.541044+0000","last_active":"2023-12-20T05:43:46.547384+0000","last_peered":"2023-12-20T05:43:46.547384+0000","last_clean":"2023-12-20T05:43:46.547384+0000","last_became_active":"2023-12-20T05:43:46.540765+0000","last_became_peered":"2023-12-20T05:43:46.540765+0000","last_unstale":"2023-12-20T05:43:46.547384+0000","last_undegraded":"2023-12-20T05:43:46.547384+0000","last_fullsized":"2023-12-20T05:43:46.547384+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:15.079889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.522054+0000","last_change":"2023-12-20T05:43:46.108835+0000","last_active":"2023-12-20T05:43:46.522054+0000","last_peered":"2023-12-20T05:43:46.522054+0000","last_clean":"2023-12-20T05:43:46.522054+0000","last_became_active":"2023-12-20T05:43:46.108514+0000","last_became_peered":"2023-12-20T05:43:46.108514+0000","last_unstale":"2023-12-20T05:43:46.522054+0000","last_undegraded":"2023-12-20T05:43:46.522054+0000","last_fullsized":"2023-12-20T05:43:46.522054+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:57:30.268487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.526636+0000","last_change":"2023-12-20T05:43:46.524801+0000","last_active":"2023-12-20T05:43:46.526636+0000","last_peered":"2023-12-20T05:43:46.526636+0000","last_clean":"2023-12-20T05:43:46.526636+0000","last_became_active":"2023-12-20T05:43:46.524489+0000","last_became_peered":"2023-12-20T05:43:46.524489+0000","last_unstale":"2023-12-20T05:43:46.526636+0000","last_undegraded":"2023-12-20T05:43:46.526636+0000","last_fullsized":"2023-12-20T05:43:46.526636+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:42:15.654028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.515492+0000","last_change":"2023-12-20T05:43:46.121725+0000","last_active":"2023-12-20T05:43:46.515492+0000","last_peered":"2023-12-20T05:43:46.515492+0000","last_clean":"2023-12-20T05:43:46.515492+0000","last_became_active":"2023-12-20T05:43:46.121428+0000","last_became_peered":"2023-12-20T05:43:46.121428+0000","last_unstale":"2023-12-20T05:43:46.515492+0000","last_undegraded":"2023-12-20T05:43:46.515492+0000","last_fullsized":"2023-12-20T05:43:46.515492+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:37:45.623160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538438+0000","last_change":"2023-12-20T05:43:46.528228+0000","last_active":"2023-12-20T05:43:46.538438+0000","last_peered":"2023-12-20T05:43:46.538438+0000","last_clean":"2023-12-20T05:43:46.538438+0000","last_became_active":"2023-12-20T05:43:46.527432+0000","last_became_peered":"2023-12-20T05:43:46.527432+0000","last_unstale":"2023-12-20T05:43:46.538438+0000","last_undegraded":"2023-12-20T05:43:46.538438+0000","last_fullsized":"2023-12-20T05:43:46.538438+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:32:18.577407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.517436+0000","last_change":"2023-12-20T05:43:46.118424+0000","last_active":"2023-12-20T05:43:46.517436+0000","last_peered":"2023-12-20T05:43:46.517436+0000","last_clean":"2023-12-20T05:43:46.517436+0000","last_became_active":"2023-12-20T05:43:46.118119+0000","last_became_peered":"2023-12-20T05:43:46.118119+0000","last_unstale":"2023-12-20T05:43:46.517436+0000","last_undegraded":"2023-12-20T05:43:46.517436+0000","last_fullsized":"2023-12-20T05:43:46.517436+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:44:42.153173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.524910+0000","last_change":"2023-12-20T05:43:46.521313+0000","last_active":"2023-12-20T05:43:46.524910+0000","last_peered":"2023-12-20T05:43:46.524910+0000","last_clean":"2023-12-20T05:43:46.524910+0000","last_became_active":"2023-12-20T05:43:46.521014+0000","last_became_peered":"2023-12-20T05:43:46.521014+0000","last_unstale":"2023-12-20T05:43:46.524910+0000","last_undegraded":"2023-12-20T05:43:46.524910+0000","last_fullsized":"2023-12-20T05:43:46.524910+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:42:15.654028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.516505+0000","last_change":"2023-12-20T05:43:46.126253+0000","last_active":"2023-12-20T05:43:46.516505+0000","last_peered":"2023-12-20T05:43:46.516505+0000","last_clean":"2023-12-20T05:43:46.516505+0000","last_became_active":"2023-12-20T05:43:46.125916+0000","last_became_peered":"2023-12-20T05:43:46.125916+0000","last_unstale":"2023-12-20T05:43:46.516505+0000","last_undegraded":"2023-12-20T05:43:46.516505+0000","last_fullsized":"2023-12-20T05:43:46.516505+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:42:29.151990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542915+0000","last_change":"2023-12-20T05:43:46.535612+0000","last_active":"2023-12-20T05:43:46.542915+0000","last_peered":"2023-12-20T05:43:46.542915+0000","last_clean":"2023-12-20T05:43:46.542915+0000","last_became_active":"2023-12-20T05:43:46.535086+0000","last_became_peered":"2023-12-20T05:43:46.535086+0000","last_unstale":"2023-12-20T05:43:46.542915+0000","last_undegraded":"2023-12-20T05:43:46.542915+0000","last_fullsized":"2023-12-20T05:43:46.542915+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:35.414585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"158'3","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538386+0000","last_change":"2023-12-20T05:43:46.114674+0000","last_active":"2023-12-20T05:43:46.538386+0000","last_peered":"2023-12-20T05:43:46.538386+0000","last_clean":"2023-12-20T05:43:46.538386+0000","last_became_active":"2023-12-20T05:43:46.114403+0000","last_became_peered":"2023-12-20T05:43:46.114403+0000","last_unstale":"2023-12-20T05:43:46.538386+0000","last_undegraded":"2023-12-20T05:43:46.538386+0000","last_fullsized":"2023-12-20T05:43:46.538386+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:37.148286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545472+0000","last_change":"2023-12-20T05:43:46.541901+0000","last_active":"2023-12-20T05:43:46.545472+0000","last_peered":"2023-12-20T05:43:46.545472+0000","last_clean":"2023-12-20T05:43:46.545472+0000","last_became_active":"2023-12-20T05:43:46.541617+0000","last_became_peered":"2023-12-20T05:43:46.541617+0000","last_unstale":"2023-12-20T05:43:46.545472+0000","last_undegraded":"2023-12-20T05:43:46.545472+0000","last_fullsized":"2023-12-20T05:43:46.545472+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:57:21.808421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"158'6","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.553908+0000","last_change":"2023-12-20T05:43:46.121257+0000","last_active":"2023-12-20T05:43:46.553908+0000","last_peered":"2023-12-20T05:43:46.553908+0000","last_clean":"2023-12-20T05:43:46.553908+0000","last_became_active":"2023-12-20T05:43:46.120975+0000","last_became_peered":"2023-12-20T05:43:46.120975+0000","last_unstale":"2023-12-20T05:43:46.553908+0000","last_undegraded":"2023-12-20T05:43:46.553908+0000","last_fullsized":"2023-12-20T05:43:46.553908+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:21:30.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.537847+0000","last_change":"2023-12-20T05:43:46.528104+0000","last_active":"2023-12-20T05:43:46.537847+0000","last_peered":"2023-12-20T05:43:46.537847+0000","last_clean":"2023-12-20T05:43:46.537847+0000","last_became_active":"2023-12-20T05:43:46.527854+0000","last_became_peered":"2023-12-20T05:43:46.527854+0000","last_unstale":"2023-12-20T05:43:46.537847+0000","last_undegraded":"2023-12-20T05:43:46.537847+0000","last_fullsized":"2023-12-20T05:43:46.537847+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:46:50.112808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.511044+0000","last_change":"2023-12-20T05:43:46.113071+0000","last_active":"2023-12-20T05:43:46.511044+0000","last_peered":"2023-12-20T05:43:46.511044+0000","last_clean":"2023-12-20T05:43:46.511044+0000","last_became_active":"2023-12-20T05:43:46.112791+0000","last_became_peered":"2023-12-20T05:43:46.112791+0000","last_unstale":"2023-12-20T05:43:46.511044+0000","last_undegraded":"2023-12-20T05:43:46.511044+0000","last_fullsized":"2023-12-20T05:43:46.511044+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:31.016776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.553477+0000","last_change":"2023-12-20T05:43:46.552804+0000","last_active":"2023-12-20T05:43:46.553477+0000","last_peered":"2023-12-20T05:43:46.553477+0000","last_clean":"2023-12-20T05:43:46.553477+0000","last_became_active":"2023-12-20T05:43:46.552479+0000","last_became_peered":"2023-12-20T05:43:46.552479+0000","last_unstale":"2023-12-20T05:43:46.553477+0000","last_undegraded":"2023-12-20T05:43:46.553477+0000","last_fullsized":"2023-12-20T05:43:46.553477+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:28:41.860606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.516133+0000","last_change":"2023-12-20T05:43:46.120127+0000","last_active":"2023-12-20T05:43:46.516133+0000","last_peered":"2023-12-20T05:43:46.516133+0000","last_clean":"2023-12-20T05:43:46.516133+0000","last_became_active":"2023-12-20T05:43:46.119571+0000","last_became_peered":"2023-12-20T05:43:46.119571+0000","last_unstale":"2023-12-20T05:43:46.516133+0000","last_undegraded":"2023-12-20T05:43:46.516133+0000","last_fullsized":"2023-12-20T05:43:46.516133+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:21:30.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545804+0000","last_change":"2023-12-20T05:43:46.540846+0000","last_active":"2023-12-20T05:43:46.545804+0000","last_peered":"2023-12-20T05:43:46.545804+0000","last_clean":"2023-12-20T05:43:46.545804+0000","last_became_active":"2023-12-20T05:43:46.539573+0000","last_became_peered":"2023-12-20T05:43:46.539573+0000","last_unstale":"2023-12-20T05:43:46.545804+0000","last_undegraded":"2023-12-20T05:43:46.545804+0000","last_fullsized":"2023-12-20T05:43:46.545804+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:46.567327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"158'8","reported_seq":18,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.553020+0000","last_change":"2023-12-20T05:43:46.119425+0000","last_active":"2023-12-20T05:43:46.553020+0000","last_peered":"2023-12-20T05:43:46.553020+0000","last_clean":"2023-12-20T05:43:46.553020+0000","last_became_active":"2023-12-20T05:43:46.118802+0000","last_became_peered":"2023-12-20T05:43:46.118802+0000","last_unstale":"2023-12-20T05:43:46.553020+0000","last_undegraded":"2023-12-20T05:43:46.553020+0000","last_fullsized":"2023-12-20T05:43:46.553020+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:01:46.684943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.547209+0000","last_change":"2023-12-20T05:43:46.541550+0000","last_active":"2023-12-20T05:43:46.547209+0000","last_peered":"2023-12-20T05:43:46.547209+0000","last_clean":"2023-12-20T05:43:46.547209+0000","last_became_active":"2023-12-20T05:43:46.541278+0000","last_became_peered":"2023-12-20T05:43:46.541278+0000","last_unstale":"2023-12-20T05:43:46.547209+0000","last_undegraded":"2023-12-20T05:43:46.547209+0000","last_fullsized":"2023-12-20T05:43:46.547209+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:15:17.571592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"158'1","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545025+0000","last_change":"2023-12-20T05:43:46.120827+0000","last_active":"2023-12-20T05:43:46.545025+0000","last_peered":"2023-12-20T05:43:46.545025+0000","last_clean":"2023-12-20T05:43:46.545025+0000","last_became_active":"2023-12-20T05:43:46.120538+0000","last_became_peered":"2023-12-20T05:43:46.120538+0000","last_unstale":"2023-12-20T05:43:46.545025+0000","last_undegraded":"2023-12-20T05:43:46.545025+0000","last_fullsized":"2023-12-20T05:43:46.545025+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:20:54.119580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.531978+0000","last_change":"2023-12-20T05:43:46.528569+0000","last_active":"2023-12-20T05:43:46.531978+0000","last_peered":"2023-12-20T05:43:46.531978+0000","last_clean":"2023-12-20T05:43:46.531978+0000","last_became_active":"2023-12-20T05:43:46.528243+0000","last_became_peered":"2023-12-20T05:43:46.528243+0000","last_unstale":"2023-12-20T05:43:46.531978+0000","last_undegraded":"2023-12-20T05:43:46.531978+0000","last_fullsized":"2023-12-20T05:43:46.531978+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:15:17.571592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.547814+0000","last_change":"2023-12-20T05:43:46.126934+0000","last_active":"2023-12-20T05:43:46.547814+0000","last_peered":"2023-12-20T05:43:46.547814+0000","last_clean":"2023-12-20T05:43:46.547814+0000","last_became_active":"2023-12-20T05:43:46.126625+0000","last_became_peered":"2023-12-20T05:43:46.126625+0000","last_unstale":"2023-12-20T05:43:46.547814+0000","last_undegraded":"2023-12-20T05:43:46.547814+0000","last_fullsized":"2023-12-20T05:43:46.547814+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:53:59.899060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.523892+0000","last_change":"2023-12-20T05:43:46.522829+0000","last_active":"2023-12-20T05:43:46.523892+0000","last_peered":"2023-12-20T05:43:46.523892+0000","last_clean":"2023-12-20T05:43:46.523892+0000","last_became_active":"2023-12-20T05:43:46.522103+0000","last_became_peered":"2023-12-20T05:43:46.522103+0000","last_unstale":"2023-12-20T05:43:46.523892+0000","last_undegraded":"2023-12-20T05:43:46.523892+0000","last_fullsized":"2023-12-20T05:43:46.523892+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:21:16.244410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"158'1","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.537461+0000","last_change":"2023-12-20T05:43:46.111276+0000","last_active":"2023-12-20T05:43:46.537461+0000","last_peered":"2023-12-20T05:43:46.537461+0000","last_clean":"2023-12-20T05:43:46.537461+0000","last_became_active":"2023-12-20T05:43:46.110933+0000","last_became_peered":"2023-12-20T05:43:46.110933+0000","last_unstale":"2023-12-20T05:43:46.537461+0000","last_undegraded":"2023-12-20T05:43:46.537461+0000","last_fullsized":"2023-12-20T05:43:46.537461+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:57:20.805569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.526469+0000","last_change":"2023-12-20T05:43:46.524339+0000","last_active":"2023-12-20T05:43:46.526469+0000","last_peered":"2023-12-20T05:43:46.526469+0000","last_clean":"2023-12-20T05:43:46.526469+0000","last_became_active":"2023-12-20T05:43:46.524070+0000","last_became_peered":"2023-12-20T05:43:46.524070+0000","last_unstale":"2023-12-20T05:43:46.526469+0000","last_undegraded":"2023-12-20T05:43:46.526469+0000","last_fullsized":"2023-12-20T05:43:46.526469+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:56.569699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.516904+0000","last_change":"2023-12-20T05:43:46.118736+0000","last_active":"2023-12-20T05:43:46.516904+0000","last_peered":"2023-12-20T05:43:46.516904+0000","last_clean":"2023-12-20T05:43:46.516904+0000","last_became_active":"2023-12-20T05:43:46.118349+0000","last_became_peered":"2023-12-20T05:43:46.118349+0000","last_unstale":"2023-12-20T05:43:46.516904+0000","last_undegraded":"2023-12-20T05:43:46.516904+0000","last_fullsized":"2023-12-20T05:43:46.516904+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:06:23.077181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545549+0000","last_change":"2023-12-20T05:43:46.544192+0000","last_active":"2023-12-20T05:43:46.545549+0000","last_peered":"2023-12-20T05:43:46.545549+0000","last_clean":"2023-12-20T05:43:46.545549+0000","last_became_active":"2023-12-20T05:43:46.543504+0000","last_became_peered":"2023-12-20T05:43:46.543504+0000","last_unstale":"2023-12-20T05:43:46.545549+0000","last_undegraded":"2023-12-20T05:43:46.545549+0000","last_fullsized":"2023-12-20T05:43:46.545549+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:35:19.638668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538161+0000","last_change":"2023-12-20T05:43:46.119209+0000","last_active":"2023-12-20T05:43:46.538161+0000","last_peered":"2023-12-20T05:43:46.538161+0000","last_clean":"2023-12-20T05:43:46.538161+0000","last_became_active":"2023-12-20T05:43:46.117369+0000","last_became_peered":"2023-12-20T05:43:46.117369+0000","last_unstale":"2023-12-20T05:43:46.538161+0000","last_undegraded":"2023-12-20T05:43:46.538161+0000","last_fullsized":"2023-12-20T05:43:46.538161+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:20.427973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.546887+0000","last_change":"2023-12-20T05:43:46.545720+0000","last_active":"2023-12-20T05:43:46.546887+0000","last_peered":"2023-12-20T05:43:46.546887+0000","last_clean":"2023-12-20T05:43:46.546887+0000","last_became_active":"2023-12-20T05:43:46.545328+0000","last_became_peered":"2023-12-20T05:43:46.545328+0000","last_unstale":"2023-12-20T05:43:46.546887+0000","last_undegraded":"2023-12-20T05:43:46.546887+0000","last_fullsized":"2023-12-20T05:43:46.546887+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:12:29.110689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.515258+0000","last_change":"2023-12-20T05:43:46.126273+0000","last_active":"2023-12-20T05:43:46.515258+0000","last_peered":"2023-12-20T05:43:46.515258+0000","last_clean":"2023-12-20T05:43:46.515258+0000","last_became_active":"2023-12-20T05:43:46.125888+0000","last_became_peered":"2023-12-20T05:43:46.125888+0000","last_unstale":"2023-12-20T05:43:46.515258+0000","last_undegraded":"2023-12-20T05:43:46.515258+0000","last_fullsized":"2023-12-20T05:43:46.515258+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:55:22.693101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.525197+0000","last_change":"2023-12-20T05:43:46.523070+0000","last_active":"2023-12-20T05:43:46.525197+0000","last_peered":"2023-12-20T05:43:46.525197+0000","last_clean":"2023-12-20T05:43:46.525197+0000","last_became_active":"2023-12-20T05:43:46.522763+0000","last_became_peered":"2023-12-20T05:43:46.522763+0000","last_unstale":"2023-12-20T05:43:46.525197+0000","last_undegraded":"2023-12-20T05:43:46.525197+0000","last_fullsized":"2023-12-20T05:43:46.525197+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:47:52.037903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.514907+0000","last_change":"2023-12-20T05:43:46.122985+0000","last_active":"2023-12-20T05:43:46.514907+0000","last_peered":"2023-12-20T05:43:46.514907+0000","last_clean":"2023-12-20T05:43:46.514907+0000","last_became_active":"2023-12-20T05:43:46.121903+0000","last_became_peered":"2023-12-20T05:43:46.121903+0000","last_unstale":"2023-12-20T05:43:46.514907+0000","last_undegraded":"2023-12-20T05:43:46.514907+0000","last_fullsized":"2023-12-20T05:43:46.514907+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:48:57.395143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545125+0000","last_change":"2023-12-20T05:43:46.537360+0000","last_active":"2023-12-20T05:43:46.545125+0000","last_peered":"2023-12-20T05:43:46.545125+0000","last_clean":"2023-12-20T05:43:46.545125+0000","last_became_active":"2023-12-20T05:43:46.537083+0000","last_became_peered":"2023-12-20T05:43:46.537083+0000","last_unstale":"2023-12-20T05:43:46.545125+0000","last_undegraded":"2023-12-20T05:43:46.545125+0000","last_fullsized":"2023-12-20T05:43:46.545125+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:28:41.860606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.519208+0000","last_change":"2023-12-20T05:43:46.122667+0000","last_active":"2023-12-20T05:43:46.519208+0000","last_peered":"2023-12-20T05:43:46.519208+0000","last_clean":"2023-12-20T05:43:46.519208+0000","last_became_active":"2023-12-20T05:43:46.122357+0000","last_became_peered":"2023-12-20T05:43:46.122357+0000","last_unstale":"2023-12-20T05:43:46.519208+0000","last_undegraded":"2023-12-20T05:43:46.519208+0000","last_fullsized":"2023-12-20T05:43:46.519208+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:46.278593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.537692+0000","last_change":"2023-12-20T05:43:46.530254+0000","last_active":"2023-12-20T05:43:46.537692+0000","last_peered":"2023-12-20T05:43:46.537692+0000","last_clean":"2023-12-20T05:43:46.537692+0000","last_became_active":"2023-12-20T05:43:46.529971+0000","last_became_peered":"2023-12-20T05:43:46.529971+0000","last_unstale":"2023-12-20T05:43:46.537692+0000","last_undegraded":"2023-12-20T05:43:46.537692+0000","last_fullsized":"2023-12-20T05:43:46.537692+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:48:58.397995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.539153+0000","last_change":"2023-12-20T05:43:46.123658+0000","last_active":"2023-12-20T05:43:46.539153+0000","last_peered":"2023-12-20T05:43:46.539153+0000","last_clean":"2023-12-20T05:43:46.539153+0000","last_became_active":"2023-12-20T05:43:46.123314+0000","last_became_peered":"2023-12-20T05:43:46.123314+0000","last_unstale":"2023-12-20T05:43:46.539153+0000","last_undegraded":"2023-12-20T05:43:46.539153+0000","last_fullsized":"2023-12-20T05:43:46.539153+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:41:06.068217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.550288+0000","last_change":"2023-12-20T05:43:46.549246+0000","last_active":"2023-12-20T05:43:46.550288+0000","last_peered":"2023-12-20T05:43:46.550288+0000","last_clean":"2023-12-20T05:43:46.550288+0000","last_became_active":"2023-12-20T05:43:46.549014+0000","last_became_peered":"2023-12-20T05:43:46.549014+0000","last_unstale":"2023-12-20T05:43:46.550288+0000","last_undegraded":"2023-12-20T05:43:46.550288+0000","last_fullsized":"2023-12-20T05:43:46.550288+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:00:55.245435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.522907+0000","last_change":"2023-12-20T05:43:46.113428+0000","last_active":"2023-12-20T05:43:46.522907+0000","last_peered":"2023-12-20T05:43:46.522907+0000","last_clean":"2023-12-20T05:43:46.522907+0000","last_became_active":"2023-12-20T05:43:46.113096+0000","last_became_peered":"2023-12-20T05:43:46.113096+0000","last_unstale":"2023-12-20T05:43:46.522907+0000","last_undegraded":"2023-12-20T05:43:46.522907+0000","last_fullsized":"2023-12-20T05:43:46.522907+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:37.148286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.534189+0000","last_change":"2023-12-20T05:43:46.529405+0000","last_active":"2023-12-20T05:43:46.534189+0000","last_peered":"2023-12-20T05:43:46.534189+0000","last_clean":"2023-12-20T05:43:46.534189+0000","last_became_active":"2023-12-20T05:43:46.529148+0000","last_became_peered":"2023-12-20T05:43:46.529148+0000","last_unstale":"2023-12-20T05:43:46.534189+0000","last_undegraded":"2023-12-20T05:43:46.534189+0000","last_fullsized":"2023-12-20T05:43:46.534189+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:15.079889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.519015+0000","last_change":"2023-12-20T05:43:46.124867+0000","last_active":"2023-12-20T05:43:46.519015+0000","last_peered":"2023-12-20T05:43:46.519015+0000","last_clean":"2023-12-20T05:43:46.519015+0000","last_became_active":"2023-12-20T05:43:46.124521+0000","last_became_peered":"2023-12-20T05:43:46.124521+0000","last_unstale":"2023-12-20T05:43:46.519015+0000","last_undegraded":"2023-12-20T05:43:46.519015+0000","last_fullsized":"2023-12-20T05:43:46.519015+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:43:15.471880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.544450+0000","last_change":"2023-12-20T05:43:46.539564+0000","last_active":"2023-12-20T05:43:46.544450+0000","last_peered":"2023-12-20T05:43:46.544450+0000","last_clean":"2023-12-20T05:43:46.544450+0000","last_became_active":"2023-12-20T05:43:46.539299+0000","last_became_peered":"2023-12-20T05:43:46.539299+0000","last_unstale":"2023-12-20T05:43:46.544450+0000","last_undegraded":"2023-12-20T05:43:46.544450+0000","last_fullsized":"2023-12-20T05:43:46.544450+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:14:48.759628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.519285+0000","last_change":"2023-12-20T05:43:46.115765+0000","last_active":"2023-12-20T05:43:46.519285+0000","last_peered":"2023-12-20T05:43:46.519285+0000","last_clean":"2023-12-20T05:43:46.519285+0000","last_became_active":"2023-12-20T05:43:46.115487+0000","last_became_peered":"2023-12-20T05:43:46.115487+0000","last_unstale":"2023-12-20T05:43:46.519285+0000","last_undegraded":"2023-12-20T05:43:46.519285+0000","last_fullsized":"2023-12-20T05:43:46.519285+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:37:29.678269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.543367+0000","last_change":"2023-12-20T05:43:46.534683+0000","last_active":"2023-12-20T05:43:46.543367+0000","last_peered":"2023-12-20T05:43:46.543367+0000","last_clean":"2023-12-20T05:43:46.543367+0000","last_became_active":"2023-12-20T05:43:46.534354+0000","last_became_peered":"2023-12-20T05:43:46.534354+0000","last_unstale":"2023-12-20T05:43:46.543367+0000","last_undegraded":"2023-12-20T05:43:46.543367+0000","last_fullsized":"2023-12-20T05:43:46.543367+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:05:29.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.515801+0000","last_change":"2023-12-20T05:43:46.119875+0000","last_active":"2023-12-20T05:43:46.515801+0000","last_peered":"2023-12-20T05:43:46.515801+0000","last_clean":"2023-12-20T05:43:46.515801+0000","last_became_active":"2023-12-20T05:43:46.119594+0000","last_became_peered":"2023-12-20T05:43:46.119594+0000","last_unstale":"2023-12-20T05:43:46.515801+0000","last_undegraded":"2023-12-20T05:43:46.515801+0000","last_fullsized":"2023-12-20T05:43:46.515801+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:01:46.684943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.526213+0000","last_change":"2023-12-20T05:43:46.524269+0000","last_active":"2023-12-20T05:43:46.526213+0000","last_peered":"2023-12-20T05:43:46.526213+0000","last_clean":"2023-12-20T05:43:46.526213+0000","last_became_active":"2023-12-20T05:43:46.523979+0000","last_became_peered":"2023-12-20T05:43:46.523979+0000","last_unstale":"2023-12-20T05:43:46.526213+0000","last_undegraded":"2023-12-20T05:43:46.526213+0000","last_fullsized":"2023-12-20T05:43:46.526213+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:15.079889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.519487+0000","last_change":"2023-12-20T05:43:46.114892+0000","last_active":"2023-12-20T05:43:46.519487+0000","last_peered":"2023-12-20T05:43:46.519487+0000","last_clean":"2023-12-20T05:43:46.519487+0000","last_became_active":"2023-12-20T05:43:46.113045+0000","last_became_peered":"2023-12-20T05:43:46.113045+0000","last_unstale":"2023-12-20T05:43:46.519487+0000","last_undegraded":"2023-12-20T05:43:46.519487+0000","last_fullsized":"2023-12-20T05:43:46.519487+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:13.245566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.553252+0000","last_change":"2023-12-20T05:43:46.552510+0000","last_active":"2023-12-20T05:43:46.553252+0000","last_peered":"2023-12-20T05:43:46.553252+0000","last_clean":"2023-12-20T05:43:46.553252+0000","last_became_active":"2023-12-20T05:43:46.552184+0000","last_became_peered":"2023-12-20T05:43:46.552184+0000","last_unstale":"2023-12-20T05:43:46.553252+0000","last_undegraded":"2023-12-20T05:43:46.553252+0000","last_fullsized":"2023-12-20T05:43:46.553252+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:34:19.412615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.515579+0000","last_change":"2023-12-20T05:43:46.123211+0000","last_active":"2023-12-20T05:43:46.515579+0000","last_peered":"2023-12-20T05:43:46.515579+0000","last_clean":"2023-12-20T05:43:46.515579+0000","last_became_active":"2023-12-20T05:43:46.122598+0000","last_became_peered":"2023-12-20T05:43:46.122598+0000","last_unstale":"2023-12-20T05:43:46.515579+0000","last_undegraded":"2023-12-20T05:43:46.515579+0000","last_fullsized":"2023-12-20T05:43:46.515579+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:07.874108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.532414+0000","last_change":"2023-12-20T05:43:46.530064+0000","last_active":"2023-12-20T05:43:46.532414+0000","last_peered":"2023-12-20T05:43:46.532414+0000","last_clean":"2023-12-20T05:43:46.532414+0000","last_became_active":"2023-12-20T05:43:46.529778+0000","last_became_peered":"2023-12-20T05:43:46.529778+0000","last_unstale":"2023-12-20T05:43:46.532414+0000","last_undegraded":"2023-12-20T05:43:46.532414+0000","last_fullsized":"2023-12-20T05:43:46.532414+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:08:27.340043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.519297+0000","last_change":"2023-12-20T05:43:46.115882+0000","last_active":"2023-12-20T05:43:46.519297+0000","last_peered":"2023-12-20T05:43:46.519297+0000","last_clean":"2023-12-20T05:43:46.519297+0000","last_became_active":"2023-12-20T05:43:46.115604+0000","last_became_peered":"2023-12-20T05:43:46.115604+0000","last_unstale":"2023-12-20T05:43:46.519297+0000","last_undegraded":"2023-12-20T05:43:46.519297+0000","last_fullsized":"2023-12-20T05:43:46.519297+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:51:48.971451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.543349+0000","last_change":"2023-12-20T05:43:46.529405+0000","last_active":"2023-12-20T05:43:46.543349+0000","last_peered":"2023-12-20T05:43:46.543349+0000","last_clean":"2023-12-20T05:43:46.543349+0000","last_became_active":"2023-12-20T05:43:46.529139+0000","last_became_peered":"2023-12-20T05:43:46.529139+0000","last_unstale":"2023-12-20T05:43:46.543349+0000","last_undegraded":"2023-12-20T05:43:46.543349+0000","last_fullsized":"2023-12-20T05:43:46.543349+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:19.676103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542456+0000","last_change":"2023-12-20T05:43:46.115728+0000","last_active":"2023-12-20T05:43:46.542456+0000","last_peered":"2023-12-20T05:43:46.542456+0000","last_clean":"2023-12-20T05:43:46.542456+0000","last_became_active":"2023-12-20T05:43:46.115359+0000","last_became_peered":"2023-12-20T05:43:46.115359+0000","last_unstale":"2023-12-20T05:43:46.542456+0000","last_undegraded":"2023-12-20T05:43:46.542456+0000","last_fullsized":"2023-12-20T05:43:46.542456+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:41.080590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"146'3124","reported_seq":3666,"reported_epoch":157,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.105331+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T05:43:46.105331+0000","last_peered":"2023-12-20T05:43:46.105331+0000","last_clean":"2023-12-20T05:43:46.105331+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T05:43:46.105331+0000","last_undegraded":"2023-12-20T05:43:46.105331+0000","last_fullsized":"2023-12-20T05:43:46.105331+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3124,"log_dups_size":0,"ondisk_log_size":3124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1213,"num_read_kb":2407,"num_write":381,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1213,"num_read_kb":2407,"num_write":381,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3124,"ondisk_log_size":3124,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575749,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33524,"kb_used_data":2476,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338316,"statfs":{"total":96636764160,"available":96602435584,"internally_reserved":0,"allocated":2535424,"data_stored":1520208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.48699999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93500000000000005,"5min":1.133,"15min":1.133},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75700000000000001,"15min":0.73999999999999999},"min":{"1min":0.52500000000000002,"5min":0.45500000000000002,"15min":0.40300000000000002},"max":{"1min":0.93000000000000005,"5min":0.95599999999999996,"15min":1.077},"last":0.82599999999999996}]},{"osd":1,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.57499999999999996,"5min":0.48099999999999998,"15min":0.45200000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.131},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.59299999999999997,"5min":0.53700000000000003,"15min":0.47699999999999998},"max":{"1min":0.92100000000000004,"5min":1.0129999999999999,"15min":1.056},"last":0.752}]},{"osd":2,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.56599999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"max":{"1min":0.85799999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.54800000000000004,"5min":0.52200000000000002,"15min":0.501},"max":{"1min":0.97999999999999998,"5min":1.099,"15min":2.2160000000000002},"last":0.84399999999999997}]},{"osd":3,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.75800000000000001,"15min":0.752},"min":{"1min":0.59699999999999998,"5min":0.52100000000000002,"15min":0.46899999999999997},"max":{"1min":0.91000000000000003,"5min":0.98999999999999999,"15min":2.6829999999999998},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.62,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":2.1259999999999999},"last":0.77300000000000002}]},{"osd":4,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.60099999999999998},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.8180000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40200000000000002},"max":{"1min":0.71799999999999997,"5min":1.01,"15min":2.7370000000000001},"last":0.50900000000000001}]},{"osd":5,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96299999999999997,"5min":0.97399999999999998,"15min":2.27},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.501,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92800000000000005,"5min":1.0029999999999999,"15min":1.0700000000000001},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.747,"15min":0.746},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.42699999999999999},"max":{"1min":1.002,"5min":1.071,"15min":1.1699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.70899999999999996},"min":{"1min":0.50800000000000001,"5min":0.45400000000000001,"15min":0.45200000000000001},"max":{"1min":0.95899999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.61199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575749,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1941504,"data_stored":929840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.52100000000000002,"5min":0.45700000000000002,"15min":0.36899999999999999},"max":{"1min":0.99299999999999999,"5min":1,"15min":1.093},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.755,"15min":0.755},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.101},"last":0.80400000000000005}]},{"osd":1,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.753,"15min":0.752},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.42699999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.03},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.48699999999999999,"15min":0.46200000000000002},"max":{"1min":0.93700000000000006,"5min":1.0149999999999999,"15min":1.038},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.77600000000000002,"15min":0.78100000000000003},"min":{"1min":0.63900000000000001,"5min":0.48099999999999998,"15min":0.47999999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.0960000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78600000000000003,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99099999999999999,"5min":1.0369999999999999,"15min":1.083},"last":0.752}]},{"osd":3,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.503},"max":{"1min":0.97499999999999998,"5min":1.036,"15min":1.0629999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77500000000000002,"15min":0.78000000000000003},"min":{"1min":0.58299999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.93500000000000005,"5min":1.0169999999999999,"15min":1.1020000000000001},"last":0.78100000000000003}]},{"osd":4,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.40799999999999997,"15min":0.36599999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.48199999999999998,"5min":0.46200000000000002,"15min":0.40999999999999998},"max":{"1min":0.89800000000000002,"5min":1.048,"15min":1.048},"last":0.85199999999999998}]},{"osd":5,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.47599999999999998,"5min":0.42799999999999999,"15min":0.40200000000000002},"max":{"1min":0.999,"5min":0.999,"15min":1.0429999999999999},"last":0.879},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.50700000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95699999999999996,"5min":1.0109999999999999,"15min":1.0509999999999999},"last":0.81799999999999995}]},{"osd":7,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.45500000000000002},"max":{"1min":0.98699999999999999,"5min":1.095,"15min":1.095},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.49099999999999999},"max":{"1min":0.97599999999999998,"5min":1.0049999999999999,"15min":1.105},"last":0.83499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28860,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342980,"statfs":{"total":96636764160,"available":96607211520,"internally_reserved":0,"allocated":1953792,"data_stored":936143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":0.76300000000000001},"min":{"1min":0.54300000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97399999999999998,"5min":1.145,"15min":1.145},"last":0.66200000000000003}]},{"osd":1,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76200000000000001,"15min":0.751},"min":{"1min":0.496,"5min":0.44900000000000001,"15min":0.42799999999999999},"max":{"1min":0.92800000000000005,"5min":1.016,"15min":1.016},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.45000000000000001},"max":{"1min":1.0009999999999999,"5min":1.022,"15min":1.0569999999999999},"last":0.80600000000000005}]},{"osd":2,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.77100000000000002},"min":{"1min":0.55700000000000005,"5min":0.48199999999999998,"15min":0.44700000000000001},"max":{"1min":1.091,"5min":1.204,"15min":1.204},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76900000000000002,"15min":0.76400000000000001},"min":{"1min":0.58499999999999996,"5min":0.497,"15min":0.49399999999999999},"max":{"1min":0.94599999999999995,"5min":1.052,"15min":1.052},"last":0.82399999999999995}]},{"osd":3,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79400000000000004,"15min":0.78400000000000003},"min":{"1min":0.56399999999999995,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":1.0369999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.80900000000000005,"15min":0.78900000000000003},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.53300000000000003},"max":{"1min":1.0800000000000001,"5min":1.107,"15min":1.107},"last":0.68400000000000005}]},{"osd":4,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65300000000000002,"15min":0.63600000000000001},"min":{"1min":0.499,"5min":0.436,"15min":0.41199999999999998},"max":{"1min":0.99099999999999999,"5min":1.167,"15min":1.167},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97899999999999998,"5min":1.008,"15min":1.008},"last":0.70199999999999996}]},{"osd":6,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.71299999999999997,"15min":0.69299999999999995},"min":{"1min":0.53400000000000003,"5min":0.48899999999999999,"15min":0.435},"max":{"1min":0.95899999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65600000000000003,"15min":0.63500000000000001},"min":{"1min":0.52100000000000002,"5min":0.43099999999999999,"15min":0.40999999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.085},"last":0.76800000000000002}]},{"osd":7,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71899999999999997,"15min":0.70299999999999996},"min":{"1min":0.51900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98699999999999999,"5min":1.073,"15min":1.1020000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.78800000000000003,"15min":0.76100000000000001},"min":{"1min":0.70799999999999996,"5min":0.52200000000000002,"15min":0.48199999999999998},"max":{"1min":1.0229999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.85199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59196,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312644,"statfs":{"total":96636764160,"available":96576147456,"internally_reserved":0,"allocated":1953792,"data_stored":936143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34399999999999997},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.75,"15min":0.73799999999999999},"min":{"1min":0.57299999999999995,"5min":0.47599999999999998,"15min":0.35699999999999998},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.0249999999999999,"5min":1.077,"15min":1.137},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.75,"15min":0.75700000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.72599999999999998}]},{"osd":2,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.76500000000000001,"15min":0.76600000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.624},{"interface":"front","average":{"1min":0.749,"5min":0.78100000000000003,"15min":0.77800000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.51900000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.69599999999999995}]},{"osd":3,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.45500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.79800000000000004,"15min":0.79200000000000004},"min":{"1min":0.58699999999999997,"5min":0.51500000000000001,"15min":0.47999999999999998},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.70999999999999996}]},{"osd":5,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.895,"5min":2.895,"15min":2.895},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.629,"15min":0.621},"min":{"1min":0.41199999999999998,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.53200000000000003}]},{"osd":6,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.68700000000000006,"15min":0.65800000000000003},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.40100000000000002},"max":{"1min":3.8279999999999998,"5min":3.8279999999999998,"15min":3.8279999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.70499999999999996,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.44700000000000001,"15min":0.436},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.77800000000000002,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.50800000000000001,"15min":0.47099999999999997},"max":{"1min":3.8690000000000002,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":0.76600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63312,"kb_used_data":1928,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308528,"statfs":{"total":96636764160,"available":96571932672,"internally_reserved":0,"allocated":1974272,"data_stored":943106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47199999999999998,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.28299999999999997},"max":{"1min":0.61599999999999999,"5min":0.75800000000000001,"15min":0.88800000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.36499999999999999,"5min":0.308,"15min":0.29799999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.876},"last":0.317}]},{"osd":1,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47699999999999998,"15min":0.47199999999999998},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.32800000000000001},"max":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.81499999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.40000000000000002,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.879},"last":0.39700000000000002}]},{"osd":2,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40400000000000003},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":0.88600000000000001},"last":0.622},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.46200000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.93000000000000005},"last":0.60899999999999999}]},{"osd":4,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82199999999999995,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.56000000000000005,"15min":0.55200000000000005},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.246},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83399999999999996,"15min":0.82399999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.59799999999999998}]},{"osd":5,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.61299999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.214,"5min":1.214,"15min":1.353},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.71899999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.65300000000000002}]},{"osd":6,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.53200000000000003},"max":{"1min":1.278,"5min":1.278,"15min":1.3799999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.498}]},{"osd":7,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.624,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.56699999999999995},"max":{"1min":1.1160000000000001,"5min":1.2470000000000001,"15min":1.323},"last":0.63400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575749,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46908,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324932,"statfs":{"total":96636764160,"available":96588730368,"internally_reserved":0,"allocated":1953792,"data_stored":936143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.89700000000000002},"last":0.68300000000000005}]},{"osd":1,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32700000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69099999999999995,"15min":0.68400000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.442},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.71199999999999997}]},{"osd":4,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.83799999999999997,"15min":0.81399999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.50900000000000001},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.48199999999999998},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.93100000000000005}]},{"osd":5,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.83599999999999997,"15min":0.82399999999999995},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.85299999999999998,"15min":0.83499999999999996},"min":{"1min":0.623,"5min":0.58699999999999997,"15min":0.57999999999999996},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85999999999999999,"15min":0.83299999999999996},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.86799999999999999,"15min":0.84199999999999997},"min":{"1min":0.58999999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.90000000000000002}]},{"osd":7,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.876,"15min":0.86399999999999999},"min":{"1min":0.60299999999999998,"5min":0.54800000000000004,"15min":0.51800000000000002},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91600000000000004,"15min":0.89400000000000002},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.94399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575751,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":1953792,"data_stored":936143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47999999999999998,"15min":0.46500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.29499999999999998},"max":{"1min":0.69699999999999995,"5min":0.754,"15min":0.96499999999999997},"last":0.372},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.34899999999999998,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.72599999999999998,"5min":0.76200000000000001,"15min":0.98799999999999999},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54000000000000004,"15min":0.53300000000000003},"min":{"1min":0.40100000000000002,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.91000000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.38800000000000001,"15min":0.35399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.93799999999999994},"last":0.39100000000000001}]},{"osd":3,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65200000000000002,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.442,"15min":0.42599999999999999},"max":{"1min":0.77400000000000002,"5min":0.876,"15min":0.94899999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.55400000000000005},"min":{"1min":0.40200000000000002,"5min":0.38900000000000001,"15min":0.35399999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82999999999999996,"15min":0.80500000000000005},"min":{"1min":0.624,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0329999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.84899999999999998,"15min":0.82499999999999996},"min":{"1min":0.64600000000000002,"5min":0.57599999999999996,"15min":0.56200000000000006},"max":{"1min":1.097,"5min":1.294,"15min":1.294},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.84899999999999998,"15min":0.81000000000000005},"min":{"1min":0.66200000000000003,"5min":0.56899999999999995,"15min":0.54000000000000004},"max":{"1min":0.96999999999999997,"5min":1.268,"15min":1.2709999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.86399999999999999,"15min":0.82199999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.54500000000000004},"max":{"1min":1.0609999999999999,"5min":1.274,"15min":1.274},"last":0.77100000000000002}]},{"osd":6,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.87,"15min":0.84099999999999997},"min":{"1min":0.71499999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1040000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.84799999999999998,"15min":0.82399999999999995},"min":{"1min":0.63400000000000001,"5min":0.627,"15min":0.51900000000000002},"max":{"1min":1.089,"5min":1.1459999999999999,"15min":1.284},"last":0.86199999999999999}]},{"osd":7,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89800000000000002,"15min":0.86899999999999999},"min":{"1min":0.67000000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1839999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91500000000000004,"15min":0.88100000000000001},"min":{"1min":0.68999999999999995,"5min":0.67500000000000004,"15min":0.55900000000000005},"max":{"1min":1.1519999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.74099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575750,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51572,"kb_used_data":2476,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320268,"statfs":{"total":96636764160,"available":96583954432,"internally_reserved":0,"allocated":2535424,"data_stored":1520208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.88100000000000001},"last":0.372},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.35799999999999998,"5min":0.35399999999999998,"15min":0.35199999999999998},"max":{"1min":0.78000000000000003,"5min":0.874,"15min":1.3859999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77200000000000002,"5min":0.79600000000000004,"15min":1.369},"last":0.56499999999999995}]},{"osd":3,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71599999999999997,"5min":0.84699999999999998,"15min":1.2709999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":2.016},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.56200000000000006,"5min":0.52800000000000002,"15min":0.47799999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.294},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.63200000000000001,"5min":0.58099999999999996,"15min":0.52100000000000002},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.4530000000000001},"last":0.69699999999999995}]},{"osd":5,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80200000000000005,"15min":0.82199999999999995},"min":{"1min":0.59199999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.177,"5min":1.238,"15min":1.6080000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81699999999999995,"15min":0.81899999999999995},"min":{"1min":0.60999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.129,"5min":1.22,"15min":1.3200000000000001},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.621,"5min":0.499,"15min":0.499},"max":{"1min":1.256,"5min":1.256,"15min":1.665},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55100000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1439999999999999,"5min":1.167,"15min":1.5660000000000001},"last":0.84699999999999998}]},{"osd":7,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.63200000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.212,"5min":1.222,"15min":1.4670000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.198,"5min":1.2649999999999999,"15min":1.415},"last":0.83399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T05:43:50.634 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T05:43:50.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.816+0000 7f865359e640 1 Processor -- start 2023-12-20T05:43:50.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.816+0000 7f865359e640 1 -- start start 2023-12-20T05:43:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.816+0000 7f865359e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f865406c8f0 0x7f865406cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.816+0000 7f865359e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8654069d20 con 0x7f8654161450 2023-12-20T05:43:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.816+0000 7f865359e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f865406ae80 con 0x7f865415cd00 2023-12-20T05:43:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.816+0000 7f865359e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f865406d2a0 con 0x7f865406c8f0 2023-12-20T05:43:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.816+0000 7f864bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8654161450 0x7f865406a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43024/0 (socket says 172.21.15.32:43024) 2023-12-20T05:43:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.816+0000 7f864bfff640 1 -- 172.21.15.32:0/4263445858 learned_addr learned my addr 172.21.15.32:0/4263445858 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.820+0000 7f8652d9d640 1 --2- 172.21.15.32:0/4263445858 >> v2:172.21.15.32:3300/0 conn(0x7f865406c8f0 0x7f865406cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.820+0000 7f8652d9d640 1 -- 172.21.15.32:0/4263445858 >> v1:172.21.15.39:6789/0 conn(0x7f865415cd00 legacy=0x7f86540ccd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.820+0000 7f8652d9d640 1 -- 172.21.15.32:0/4263445858 >> v1:172.21.15.32:6789/0 conn(0x7f8654161450 legacy=0x7f865406a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.820+0000 7f8652d9d640 1 -- 172.21.15.32:0/4263445858 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86541618b0 con 0x7f865406c8f0 2023-12-20T05:43:50.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.820+0000 7f8651d9b640 1 -- 172.21.15.32:0/4263445858 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 309180872 0 0) 0x7f8654069d20 con 0x7f8654161450 2023-12-20T05:43:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.820+0000 7f8652d9d640 1 --2- 172.21.15.32:0/4263445858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865406c8f0 0x7f865406cd60 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f864c009d90 tx=0x7f864c01c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a4ee1d5b85f8f90 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.820+0000 7f8651d9b640 1 -- 172.21.15.32:0/4263445858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c0a5870 con 0x7f865406c8f0 2023-12-20T05:43:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.820+0000 7f8651d9b640 1 -- 172.21.15.32:0/4263445858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f864c0a59d0 con 0x7f865406c8f0 2023-12-20T05:43:50.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.820+0000 7f8651d9b640 1 -- 172.21.15.32:0/4263445858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f864c0a5d60 con 0x7f865406c8f0 2023-12-20T05:43:50.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 -- 172.21.15.32:0/4263445858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865406c8f0 msgr2=0x7f865406cd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 --2- 172.21.15.32:0/4263445858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865406c8f0 0x7f865406cd60 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7f864c009d90 tx=0x7f864c01c8c0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 -- 172.21.15.32:0/4263445858 shutdown_connections 2023-12-20T05:43:50.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 --2- 172.21.15.32:0/4263445858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865406c8f0 0x7f865406cd60 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 -- 172.21.15.32:0/4263445858 >> 172.21.15.32:0/4263445858 conn(0x7f86540cee10 msgr2=0x7f86540cf220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:50.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 -- 172.21.15.32:0/4263445858 shutdown_connections 2023-12-20T05:43:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 -- 172.21.15.32:0/4263445858 wait complete. 2023-12-20T05:43:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 Processor -- start 2023-12-20T05:43:50.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 -- start start 2023-12-20T05:43:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f865406c8f0 0x7f86540c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865415cd00 0x7f86540ac930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8654161450 0x7f86540ace70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8654154740 con 0x7f865415cd00 2023-12-20T05:43:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86540c2540 con 0x7f8654161450 2023-12-20T05:43:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865359e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86541574c0 con 0x7f865406c8f0 2023-12-20T05:43:50.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f864bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865415cd00 0x7f86540ac930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f864bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865415cd00 0x7f86540ac930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33646/0 (socket says 172.21.15.32:33646) 2023-12-20T05:43:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f864bfff640 1 -- 172.21.15.32:0/4092959157 learned_addr learned my addr 172.21.15.32:0/4092959157 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f865259c640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f865406c8f0 0x7f86540c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f8652d9d640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8654161450 0x7f86540ace70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f864bfff640 1 -- 172.21.15.32:0/4092959157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f865406c8f0 msgr2=0x7f86540c1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f864bfff640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f865406c8f0 0x7f86540c1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f864bfff640 1 -- 172.21.15.32:0/4092959157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8654161450 msgr2=0x7f86540ace70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f864bfff640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8654161450 0x7f86540ace70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.824+0000 7f864bfff640 1 -- 172.21.15.32:0/4092959157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f864c009a70 con 0x7f865415cd00 2023-12-20T05:43:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.828+0000 7f864bfff640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865415cd00 0x7f86540ac930 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f863c00c4a0 tx=0x7f863c00c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:50.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.828+0000 7f864affd640 1 -- 172.21.15.32:0/4092959157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c0074e0 con 0x7f865415cd00 2023-12-20T05:43:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.828+0000 7f864affd640 1 -- 172.21.15.32:0/4092959157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f863c007640 con 0x7f865415cd00 2023-12-20T05:43:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.828+0000 7f865359e640 1 -- 172.21.15.32:0/4092959157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f865415de60 con 0x7f865415cd00 2023-12-20T05:43:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.828+0000 7f864affd640 1 -- 172.21.15.32:0/4092959157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f863c004a30 con 0x7f865415cd00 2023-12-20T05:43:50.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.828+0000 7f865359e640 1 -- 172.21.15.32:0/4092959157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f865415e420 con 0x7f865415cd00 2023-12-20T05:43:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.828+0000 7f865359e640 1 -- 172.21.15.32:0/4092959157 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86541548f0 con 0x7f865415cd00 2023-12-20T05:43:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.828+0000 7f864affd640 1 -- 172.21.15.32:0/4092959157 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f863c004b90 con 0x7f865415cd00 2023-12-20T05:43:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.828+0000 7f864affd640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8634065b80 0x7f8634068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.828+0000 7f864affd640 1 -- 172.21.15.32:0/4092959157 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f863c082870 con 0x7f865415cd00 2023-12-20T05:43:50.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.828+0000 7f865259c640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8634065b80 0x7f8634068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:50.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.832+0000 7f865259c640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8634065b80 0x7f8634068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f865415d740 tx=0x7f8644009040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:50.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.832+0000 7f864affd640 1 -- 172.21.15.32:0/4092959157 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f86541548f0 con 0x7f865415cd00 2023-12-20T05:43:50.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:50.992+0000 7f865359e640 1 -- 172.21.15.32:0/4092959157 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f865406a250 con 0x7f8634065b80 2023-12-20T05:43:51.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.000+0000 7f864affd640 1 -- 172.21.15.32:0/4092959157 <== mgr.8691 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289398 (secure 0 0 0) 0x7f86440050d0 con 0x7f8634065b80 2023-12-20T05:43:51.004 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:51.006 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T05:43:51.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 -- 172.21.15.32:0/4092959157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8634065b80 msgr2=0x7f8634068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8634065b80 0x7f8634068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f865415d740 tx=0x7f8644009040 comp rx=0 tx=0).stop 2023-12-20T05:43:51.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 -- 172.21.15.32:0/4092959157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865415cd00 msgr2=0x7f86540ac930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865415cd00 0x7f86540ac930 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7f863c00c4a0 tx=0x7f863c00c960 comp rx=0 tx=0).stop 2023-12-20T05:43:51.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 -- 172.21.15.32:0/4092959157 shutdown_connections 2023-12-20T05:43:51.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8634065b80 0x7f8634068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8654161450 0x7f86540ace70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f865415cd00 0x7f86540ac930 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 --2- 172.21.15.32:0/4092959157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f865406c8f0 0x7f86540c1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 -- 172.21.15.32:0/4092959157 >> 172.21.15.32:0/4092959157 conn(0x7f86540cee10 msgr2=0x7f8654070750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:51.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 -- 172.21.15.32:0/4092959157 shutdown_connections 2023-12-20T05:43:51.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.004+0000 7f865359e640 1 -- 172.21.15.32:0/4092959157 wait complete. 2023-12-20T05:43:51.024 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-12-20T05:43:49.747641+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1213,"num_read_kb":2407,"num_write":425,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3162,"ondisk_log_size":3162,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":49,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":367320,"kb_used_data":16408,"kb_used_omap":0,"kb_used_meta":350848,"kb_avail":754607400,"statfs":{"total":773094113280,"available":772717977600,"internally_reserved":0,"allocated":16801792,"data_stored":8657934,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":359268352},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.655901"},"pg_stats":[{"pgid":"19.3f","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542762+0000","last_change":"2023-12-20T05:43:46.539209+0000","last_active":"2023-12-20T05:43:46.542762+0000","last_peered":"2023-12-20T05:43:46.542762+0000","last_clean":"2023-12-20T05:43:46.542762+0000","last_became_active":"2023-12-20T05:43:46.537002+0000","last_became_peered":"2023-12-20T05:43:46.537002+0000","last_unstale":"2023-12-20T05:43:46.542762+0000","last_undegraded":"2023-12-20T05:43:46.542762+0000","last_fullsized":"2023-12-20T05:43:46.542762+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:43:16.474732+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3e","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545093+0000","last_change":"2023-12-20T05:43:46.543541+0000","last_active":"2023-12-20T05:43:46.545093+0000","last_peered":"2023-12-20T05:43:46.545093+0000","last_clean":"2023-12-20T05:43:46.545093+0000","last_became_active":"2023-12-20T05:43:46.543218+0000","last_became_peered":"2023-12-20T05:43:46.543218+0000","last_unstale":"2023-12-20T05:43:46.545093+0000","last_undegraded":"2023-12-20T05:43:46.545093+0000","last_fullsized":"2023-12-20T05:43:46.545093+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:40:42.139649+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3d","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545229+0000","last_change":"2023-12-20T05:43:46.540512+0000","last_active":"2023-12-20T05:43:46.545229+0000","last_peered":"2023-12-20T05:43:46.545229+0000","last_clean":"2023-12-20T05:43:46.545229+0000","last_became_active":"2023-12-20T05:43:46.540239+0000","last_became_peered":"2023-12-20T05:43:46.540239+0000","last_unstale":"2023-12-20T05:43:46.545229+0000","last_undegraded":"2023-12-20T05:43:46.545229+0000","last_fullsized":"2023-12-20T05:43:46.545229+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:42.083442+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.3c","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.539665+0000","last_change":"2023-12-20T05:43:46.530287+0000","last_active":"2023-12-20T05:43:46.539665+0000","last_peered":"2023-12-20T05:43:46.539665+0000","last_clean":"2023-12-20T05:43:46.539665+0000","last_became_active":"2023-12-20T05:43:46.529997+0000","last_became_peered":"2023-12-20T05:43:46.529997+0000","last_unstale":"2023-12-20T05:43:46.539665+0000","last_undegraded":"2023-12-20T05:43:46.539665+0000","last_fullsized":"2023-12-20T05:43:46.539665+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:37:46.626012+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.3b","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542585+0000","last_change":"2023-12-20T05:43:46.533306+0000","last_active":"2023-12-20T05:43:46.542585+0000","last_peered":"2023-12-20T05:43:46.542585+0000","last_clean":"2023-12-20T05:43:46.542585+0000","last_became_active":"2023-12-20T05:43:46.532672+0000","last_became_peered":"2023-12-20T05:43:46.532672+0000","last_unstale":"2023-12-20T05:43:46.542585+0000","last_undegraded":"2023-12-20T05:43:46.542585+0000","last_fullsized":"2023-12-20T05:43:46.542585+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:58:35.971693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.3a","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.544861+0000","last_change":"2023-12-20T05:43:46.540210+0000","last_active":"2023-12-20T05:43:46.544861+0000","last_peered":"2023-12-20T05:43:46.544861+0000","last_clean":"2023-12-20T05:43:46.544861+0000","last_became_active":"2023-12-20T05:43:46.539831+0000","last_became_peered":"2023-12-20T05:43:46.539831+0000","last_unstale":"2023-12-20T05:43:46.544861+0000","last_undegraded":"2023-12-20T05:43:46.544861+0000","last_fullsized":"2023-12-20T05:43:46.544861+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:48:16.672093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.39","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.550222+0000","last_change":"2023-12-20T05:43:46.549425+0000","last_active":"2023-12-20T05:43:46.550222+0000","last_peered":"2023-12-20T05:43:46.550222+0000","last_clean":"2023-12-20T05:43:46.550222+0000","last_became_active":"2023-12-20T05:43:46.549132+0000","last_became_peered":"2023-12-20T05:43:46.549132+0000","last_unstale":"2023-12-20T05:43:46.550222+0000","last_undegraded":"2023-12-20T05:43:46.550222+0000","last_fullsized":"2023-12-20T05:43:46.550222+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:54.706797+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.38","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.551653+0000","last_change":"2023-12-20T05:43:46.548423+0000","last_active":"2023-12-20T05:43:46.551653+0000","last_peered":"2023-12-20T05:43:46.551653+0000","last_clean":"2023-12-20T05:43:46.551653+0000","last_became_active":"2023-12-20T05:43:46.547993+0000","last_became_peered":"2023-12-20T05:43:46.547993+0000","last_unstale":"2023-12-20T05:43:46.551653+0000","last_undegraded":"2023-12-20T05:43:46.551653+0000","last_fullsized":"2023-12-20T05:43:46.551653+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:56.569699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.37","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.548332+0000","last_change":"2023-12-20T05:43:46.544259+0000","last_active":"2023-12-20T05:43:46.548332+0000","last_peered":"2023-12-20T05:43:46.548332+0000","last_clean":"2023-12-20T05:43:46.548332+0000","last_became_active":"2023-12-20T05:43:46.543983+0000","last_became_peered":"2023-12-20T05:43:46.543983+0000","last_unstale":"2023-12-20T05:43:46.548332+0000","last_undegraded":"2023-12-20T05:43:46.548332+0000","last_fullsized":"2023-12-20T05:43:46.548332+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:47:52.037903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.36","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.539075+0000","last_change":"2023-12-20T05:43:46.537962+0000","last_active":"2023-12-20T05:43:46.539075+0000","last_peered":"2023-12-20T05:43:46.539075+0000","last_clean":"2023-12-20T05:43:46.539075+0000","last_became_active":"2023-12-20T05:43:46.536562+0000","last_became_peered":"2023-12-20T05:43:46.536562+0000","last_unstale":"2023-12-20T05:43:46.539075+0000","last_undegraded":"2023-12-20T05:43:46.539075+0000","last_fullsized":"2023-12-20T05:43:46.539075+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:44:43.156025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.35","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.535050+0000","last_change":"2023-12-20T05:43:46.526104+0000","last_active":"2023-12-20T05:43:46.535050+0000","last_peered":"2023-12-20T05:43:46.535050+0000","last_clean":"2023-12-20T05:43:46.535050+0000","last_became_active":"2023-12-20T05:43:46.525849+0000","last_became_peered":"2023-12-20T05:43:46.525849+0000","last_unstale":"2023-12-20T05:43:46.535050+0000","last_undegraded":"2023-12-20T05:43:46.535050+0000","last_fullsized":"2023-12-20T05:43:46.535050+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:21.854766+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.34","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.551817+0000","last_change":"2023-12-20T05:43:46.549260+0000","last_active":"2023-12-20T05:43:46.551817+0000","last_peered":"2023-12-20T05:43:46.551817+0000","last_clean":"2023-12-20T05:43:46.551817+0000","last_became_active":"2023-12-20T05:43:46.548892+0000","last_became_peered":"2023-12-20T05:43:46.548892+0000","last_unstale":"2023-12-20T05:43:46.551817+0000","last_undegraded":"2023-12-20T05:43:46.551817+0000","last_fullsized":"2023-12-20T05:43:46.551817+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:08:27.340043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.33","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545459+0000","last_change":"2023-12-20T05:43:46.544121+0000","last_active":"2023-12-20T05:43:46.545459+0000","last_peered":"2023-12-20T05:43:46.545459+0000","last_clean":"2023-12-20T05:43:46.545459+0000","last_became_active":"2023-12-20T05:43:46.543467+0000","last_became_peered":"2023-12-20T05:43:46.543467+0000","last_unstale":"2023-12-20T05:43:46.545459+0000","last_undegraded":"2023-12-20T05:43:46.545459+0000","last_fullsized":"2023-12-20T05:43:46.545459+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:01:47.687795+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.32","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.531707+0000","last_change":"2023-12-20T05:43:46.528316+0000","last_active":"2023-12-20T05:43:46.531707+0000","last_peered":"2023-12-20T05:43:46.531707+0000","last_clean":"2023-12-20T05:43:46.531707+0000","last_became_active":"2023-12-20T05:43:46.528011+0000","last_became_peered":"2023-12-20T05:43:46.528011+0000","last_unstale":"2023-12-20T05:43:46.531707+0000","last_undegraded":"2023-12-20T05:43:46.531707+0000","last_fullsized":"2023-12-20T05:43:46.531707+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:46.567327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.31","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.550279+0000","last_change":"2023-12-20T05:43:46.549107+0000","last_active":"2023-12-20T05:43:46.550279+0000","last_peered":"2023-12-20T05:43:46.550279+0000","last_clean":"2023-12-20T05:43:46.550279+0000","last_became_active":"2023-12-20T05:43:46.548833+0000","last_became_peered":"2023-12-20T05:43:46.548833+0000","last_unstale":"2023-12-20T05:43:46.550279+0000","last_undegraded":"2023-12-20T05:43:46.550279+0000","last_fullsized":"2023-12-20T05:43:46.550279+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:55:41.406815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.30","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.540590+0000","last_change":"2023-12-20T05:43:46.535359+0000","last_active":"2023-12-20T05:43:46.540590+0000","last_peered":"2023-12-20T05:43:46.540590+0000","last_clean":"2023-12-20T05:43:46.540590+0000","last_became_active":"2023-12-20T05:43:46.532959+0000","last_became_peered":"2023-12-20T05:43:46.532959+0000","last_unstale":"2023-12-20T05:43:46.540590+0000","last_undegraded":"2023-12-20T05:43:46.540590+0000","last_fullsized":"2023-12-20T05:43:46.540590+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:06:33.665981+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2f","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538820+0000","last_change":"2023-12-20T05:43:46.532100+0000","last_active":"2023-12-20T05:43:46.538820+0000","last_peered":"2023-12-20T05:43:46.538820+0000","last_clean":"2023-12-20T05:43:46.538820+0000","last_became_active":"2023-12-20T05:43:46.531820+0000","last_became_peered":"2023-12-20T05:43:46.531820+0000","last_unstale":"2023-12-20T05:43:46.538820+0000","last_undegraded":"2023-12-20T05:43:46.538820+0000","last_fullsized":"2023-12-20T05:43:46.538820+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:24:17.148686+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.2e","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.543158+0000","last_change":"2023-12-20T05:43:46.541857+0000","last_active":"2023-12-20T05:43:46.543158+0000","last_peered":"2023-12-20T05:43:46.543158+0000","last_clean":"2023-12-20T05:43:46.543158+0000","last_became_active":"2023-12-20T05:43:46.541554+0000","last_became_peered":"2023-12-20T05:43:46.541554+0000","last_unstale":"2023-12-20T05:43:46.543158+0000","last_undegraded":"2023-12-20T05:43:46.543158+0000","last_fullsized":"2023-12-20T05:43:46.543158+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:26:51.798310+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2d","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.543202+0000","last_change":"2023-12-20T05:43:46.537502+0000","last_active":"2023-12-20T05:43:46.543202+0000","last_peered":"2023-12-20T05:43:46.543202+0000","last_clean":"2023-12-20T05:43:46.543202+0000","last_became_active":"2023-12-20T05:43:46.534874+0000","last_became_peered":"2023-12-20T05:43:46.534874+0000","last_unstale":"2023-12-20T05:43:46.543202+0000","last_undegraded":"2023-12-20T05:43:46.543202+0000","last_fullsized":"2023-12-20T05:43:46.543202+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:11.087920+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2c","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.530353+0000","last_change":"2023-12-20T05:43:46.526615+0000","last_active":"2023-12-20T05:43:46.530353+0000","last_peered":"2023-12-20T05:43:46.530353+0000","last_clean":"2023-12-20T05:43:46.530353+0000","last_became_active":"2023-12-20T05:43:46.526331+0000","last_became_peered":"2023-12-20T05:43:46.526331+0000","last_unstale":"2023-12-20T05:43:46.530353+0000","last_undegraded":"2023-12-20T05:43:46.530353+0000","last_fullsized":"2023-12-20T05:43:46.530353+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:57:56.867666+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.2b","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542608+0000","last_change":"2023-12-20T05:43:46.533508+0000","last_active":"2023-12-20T05:43:46.542608+0000","last_peered":"2023-12-20T05:43:46.542608+0000","last_clean":"2023-12-20T05:43:46.542608+0000","last_became_active":"2023-12-20T05:43:46.533031+0000","last_became_peered":"2023-12-20T05:43:46.533031+0000","last_unstale":"2023-12-20T05:43:46.542608+0000","last_undegraded":"2023-12-20T05:43:46.542608+0000","last_fullsized":"2023-12-20T05:43:46.542608+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:19:45.792519+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.2a","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.534505+0000","last_change":"2023-12-20T05:43:46.525527+0000","last_active":"2023-12-20T05:43:46.534505+0000","last_peered":"2023-12-20T05:43:46.534505+0000","last_clean":"2023-12-20T05:43:46.534505+0000","last_became_active":"2023-12-20T05:43:46.525199+0000","last_became_peered":"2023-12-20T05:43:46.525199+0000","last_unstale":"2023-12-20T05:43:46.534505+0000","last_undegraded":"2023-12-20T05:43:46.534505+0000","last_fullsized":"2023-12-20T05:43:46.534505+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:21:16.795497+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.29","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.548347+0000","last_change":"2023-12-20T05:43:46.543802+0000","last_active":"2023-12-20T05:43:46.548347+0000","last_peered":"2023-12-20T05:43:46.548347+0000","last_clean":"2023-12-20T05:43:46.548347+0000","last_became_active":"2023-12-20T05:43:46.543390+0000","last_became_peered":"2023-12-20T05:43:46.543390+0000","last_unstale":"2023-12-20T05:43:46.548347+0000","last_undegraded":"2023-12-20T05:43:46.548347+0000","last_fullsized":"2023-12-20T05:43:46.548347+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:20:55.122432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.28","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.533736+0000","last_change":"2023-12-20T05:43:46.527292+0000","last_active":"2023-12-20T05:43:46.533736+0000","last_peered":"2023-12-20T05:43:46.533736+0000","last_clean":"2023-12-20T05:43:46.533736+0000","last_became_active":"2023-12-20T05:43:46.527009+0000","last_became_peered":"2023-12-20T05:43:46.527009+0000","last_unstale":"2023-12-20T05:43:46.533736+0000","last_undegraded":"2023-12-20T05:43:46.533736+0000","last_fullsized":"2023-12-20T05:43:46.533736+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:28:52.808329+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.27","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538951+0000","last_change":"2023-12-20T05:43:46.536485+0000","last_active":"2023-12-20T05:43:46.538951+0000","last_peered":"2023-12-20T05:43:46.538951+0000","last_clean":"2023-12-20T05:43:46.538951+0000","last_became_active":"2023-12-20T05:43:46.536227+0000","last_became_peered":"2023-12-20T05:43:46.536227+0000","last_unstale":"2023-12-20T05:43:46.538951+0000","last_undegraded":"2023-12-20T05:43:46.538951+0000","last_fullsized":"2023-12-20T05:43:46.538951+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:15:17.571592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.26","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.534254+0000","last_change":"2023-12-20T05:43:46.532421+0000","last_active":"2023-12-20T05:43:46.534254+0000","last_peered":"2023-12-20T05:43:46.534254+0000","last_clean":"2023-12-20T05:43:46.534254+0000","last_became_active":"2023-12-20T05:43:46.530999+0000","last_became_peered":"2023-12-20T05:43:46.530999+0000","last_unstale":"2023-12-20T05:43:46.534254+0000","last_undegraded":"2023-12-20T05:43:46.534254+0000","last_fullsized":"2023-12-20T05:43:46.534254+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:29:31.310692+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.25","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542598+0000","last_change":"2023-12-20T05:43:46.539025+0000","last_active":"2023-12-20T05:43:46.542598+0000","last_peered":"2023-12-20T05:43:46.542598+0000","last_clean":"2023-12-20T05:43:46.542598+0000","last_became_active":"2023-12-20T05:43:46.537890+0000","last_became_peered":"2023-12-20T05:43:46.537890+0000","last_unstale":"2023-12-20T05:43:46.542598+0000","last_undegraded":"2023-12-20T05:43:46.542598+0000","last_fullsized":"2023-12-20T05:43:46.542598+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:02:46.647862+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.24","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.525454+0000","last_change":"2023-12-20T05:43:46.520782+0000","last_active":"2023-12-20T05:43:46.525454+0000","last_peered":"2023-12-20T05:43:46.525454+0000","last_clean":"2023-12-20T05:43:46.525454+0000","last_became_active":"2023-12-20T05:43:46.520462+0000","last_became_peered":"2023-12-20T05:43:46.520462+0000","last_unstale":"2023-12-20T05:43:46.525454+0000","last_undegraded":"2023-12-20T05:43:46.525454+0000","last_fullsized":"2023-12-20T05:43:46.525454+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:56.569699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.23","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.547340+0000","last_change":"2023-12-20T05:43:46.546781+0000","last_active":"2023-12-20T05:43:46.547340+0000","last_peered":"2023-12-20T05:43:46.547340+0000","last_clean":"2023-12-20T05:43:46.547340+0000","last_became_active":"2023-12-20T05:43:46.546477+0000","last_became_peered":"2023-12-20T05:43:46.546477+0000","last_unstale":"2023-12-20T05:43:46.547340+0000","last_undegraded":"2023-12-20T05:43:46.547340+0000","last_fullsized":"2023-12-20T05:43:46.547340+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:21:31.569553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.22","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.549714+0000","last_change":"2023-12-20T05:43:46.548897+0000","last_active":"2023-12-20T05:43:46.549714+0000","last_peered":"2023-12-20T05:43:46.549714+0000","last_clean":"2023-12-20T05:43:46.549714+0000","last_became_active":"2023-12-20T05:43:46.548613+0000","last_became_peered":"2023-12-20T05:43:46.548613+0000","last_unstale":"2023-12-20T05:43:46.549714+0000","last_undegraded":"2023-12-20T05:43:46.549714+0000","last_fullsized":"2023-12-20T05:43:46.549714+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:06:28.105274+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.21","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545108+0000","last_change":"2023-12-20T05:43:46.536793+0000","last_active":"2023-12-20T05:43:46.545108+0000","last_peered":"2023-12-20T05:43:46.545108+0000","last_clean":"2023-12-20T05:43:46.545108+0000","last_became_active":"2023-12-20T05:43:46.536520+0000","last_became_peered":"2023-12-20T05:43:46.536520+0000","last_unstale":"2023-12-20T05:43:46.545108+0000","last_undegraded":"2023-12-20T05:43:46.545108+0000","last_fullsized":"2023-12-20T05:43:46.545108+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:42:15.654028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.20","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538071+0000","last_change":"2023-12-20T05:43:46.531443+0000","last_active":"2023-12-20T05:43:46.538071+0000","last_peered":"2023-12-20T05:43:46.538071+0000","last_clean":"2023-12-20T05:43:46.538071+0000","last_became_active":"2023-12-20T05:43:46.531105+0000","last_became_peered":"2023-12-20T05:43:46.531105+0000","last_unstale":"2023-12-20T05:43:46.538071+0000","last_undegraded":"2023-12-20T05:43:46.538071+0000","last_fullsized":"2023-12-20T05:43:46.538071+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:13:51.694337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.1e","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.530123+0000","last_change":"2023-12-20T05:43:46.524217+0000","last_active":"2023-12-20T05:43:46.530123+0000","last_peered":"2023-12-20T05:43:46.530123+0000","last_clean":"2023-12-20T05:43:46.530123+0000","last_became_active":"2023-12-20T05:43:46.523954+0000","last_became_peered":"2023-12-20T05:43:46.523954+0000","last_unstale":"2023-12-20T05:43:46.530123+0000","last_undegraded":"2023-12-20T05:43:46.530123+0000","last_fullsized":"2023-12-20T05:43:46.530123+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:50.494257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1f","version":"158'4","reported_seq":14,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.544012+0000","last_change":"2023-12-20T05:43:46.114332+0000","last_active":"2023-12-20T05:43:46.544012+0000","last_peered":"2023-12-20T05:43:46.544012+0000","last_clean":"2023-12-20T05:43:46.544012+0000","last_became_active":"2023-12-20T05:43:46.114032+0000","last_became_peered":"2023-12-20T05:43:46.114032+0000","last_unstale":"2023-12-20T05:43:46.544012+0000","last_undegraded":"2023-12-20T05:43:46.544012+0000","last_fullsized":"2023-12-20T05:43:46.544012+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:01:34.225881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.1f","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.547970+0000","last_change":"2023-12-20T05:43:46.542973+0000","last_active":"2023-12-20T05:43:46.547970+0000","last_peered":"2023-12-20T05:43:46.547970+0000","last_clean":"2023-12-20T05:43:46.547970+0000","last_became_active":"2023-12-20T05:43:46.542177+0000","last_became_peered":"2023-12-20T05:43:46.542177+0000","last_unstale":"2023-12-20T05:43:46.547970+0000","last_undegraded":"2023-12-20T05:43:46.547970+0000","last_fullsized":"2023-12-20T05:43:46.547970+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:21.430825+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.1e","version":"158'1","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.539453+0000","last_change":"2023-12-20T05:43:46.119329+0000","last_active":"2023-12-20T05:43:46.539453+0000","last_peered":"2023-12-20T05:43:46.539453+0000","last_clean":"2023-12-20T05:43:46.539453+0000","last_became_active":"2023-12-20T05:43:46.119034+0000","last_became_peered":"2023-12-20T05:43:46.119034+0000","last_unstale":"2023-12-20T05:43:46.539453+0000","last_undegraded":"2023-12-20T05:43:46.539453+0000","last_fullsized":"2023-12-20T05:43:46.539453+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:01:34.225881+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.1c","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542895+0000","last_change":"2023-12-20T05:43:46.534616+0000","last_active":"2023-12-20T05:43:46.542895+0000","last_peered":"2023-12-20T05:43:46.542895+0000","last_clean":"2023-12-20T05:43:46.542895+0000","last_became_active":"2023-12-20T05:43:46.534337+0000","last_became_peered":"2023-12-20T05:43:46.534337+0000","last_unstale":"2023-12-20T05:43:46.542895+0000","last_undegraded":"2023-12-20T05:43:46.542895+0000","last_fullsized":"2023-12-20T05:43:46.542895+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:20:15.408520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1d","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542493+0000","last_change":"2023-12-20T05:43:46.116964+0000","last_active":"2023-12-20T05:43:46.542493+0000","last_peered":"2023-12-20T05:43:46.542493+0000","last_clean":"2023-12-20T05:43:46.542493+0000","last_became_active":"2023-12-20T05:43:46.114237+0000","last_became_peered":"2023-12-20T05:43:46.114237+0000","last_unstale":"2023-12-20T05:43:46.542493+0000","last_undegraded":"2023-12-20T05:43:46.542493+0000","last_fullsized":"2023-12-20T05:43:46.542493+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:37:47.327725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.1d","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545452+0000","last_change":"2023-12-20T05:43:46.532968+0000","last_active":"2023-12-20T05:43:46.545452+0000","last_peered":"2023-12-20T05:43:46.545452+0000","last_clean":"2023-12-20T05:43:46.545452+0000","last_became_active":"2023-12-20T05:43:46.532716+0000","last_became_peered":"2023-12-20T05:43:46.532716+0000","last_unstale":"2023-12-20T05:43:46.545452+0000","last_undegraded":"2023-12-20T05:43:46.545452+0000","last_fullsized":"2023-12-20T05:43:46.545452+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:39:20.211360+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.1c","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.543608+0000","last_change":"2023-12-20T05:43:46.115735+0000","last_active":"2023-12-20T05:43:46.543608+0000","last_peered":"2023-12-20T05:43:46.543608+0000","last_clean":"2023-12-20T05:43:46.543608+0000","last_became_active":"2023-12-20T05:43:46.115438+0000","last_became_peered":"2023-12-20T05:43:46.115438+0000","last_unstale":"2023-12-20T05:43:46.543608+0000","last_undegraded":"2023-12-20T05:43:46.543608+0000","last_fullsized":"2023-12-20T05:43:46.543608+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:02:45.645010+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1a","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.540383+0000","last_change":"2023-12-20T05:43:46.534307+0000","last_active":"2023-12-20T05:43:46.540383+0000","last_peered":"2023-12-20T05:43:46.540383+0000","last_clean":"2023-12-20T05:43:46.540383+0000","last_became_active":"2023-12-20T05:43:46.534056+0000","last_became_peered":"2023-12-20T05:43:46.534056+0000","last_unstale":"2023-12-20T05:43:46.540383+0000","last_undegraded":"2023-12-20T05:43:46.540383+0000","last_fullsized":"2023-12-20T05:43:46.540383+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:37:28.001287+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1b","version":"158'1","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.536962+0000","last_change":"2023-12-20T05:43:46.114483+0000","last_active":"2023-12-20T05:43:46.536962+0000","last_peered":"2023-12-20T05:43:46.536962+0000","last_clean":"2023-12-20T05:43:46.536962+0000","last_became_active":"2023-12-20T05:43:46.114199+0000","last_became_peered":"2023-12-20T05:43:46.114199+0000","last_unstale":"2023-12-20T05:43:46.536962+0000","last_undegraded":"2023-12-20T05:43:46.536962+0000","last_fullsized":"2023-12-20T05:43:46.536962+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:51.446095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.1b","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.541268+0000","last_change":"2023-12-20T05:43:46.538206+0000","last_active":"2023-12-20T05:43:46.541268+0000","last_peered":"2023-12-20T05:43:46.541268+0000","last_clean":"2023-12-20T05:43:46.541268+0000","last_became_active":"2023-12-20T05:43:46.537820+0000","last_became_peered":"2023-12-20T05:43:46.537820+0000","last_unstale":"2023-12-20T05:43:46.541268+0000","last_undegraded":"2023-12-20T05:43:46.541268+0000","last_fullsized":"2023-12-20T05:43:46.541268+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:15:33.061793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.1a","version":"158'1","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.537634+0000","last_change":"2023-12-20T05:43:46.115868+0000","last_active":"2023-12-20T05:43:46.537634+0000","last_peered":"2023-12-20T05:43:46.537634+0000","last_clean":"2023-12-20T05:43:46.537634+0000","last_became_active":"2023-12-20T05:43:46.115606+0000","last_became_peered":"2023-12-20T05:43:46.115606+0000","last_unstale":"2023-12-20T05:43:46.537634+0000","last_undegraded":"2023-12-20T05:43:46.537634+0000","last_fullsized":"2023-12-20T05:43:46.537634+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:43:59.588695+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.18","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.547384+0000","last_change":"2023-12-20T05:43:46.541044+0000","last_active":"2023-12-20T05:43:46.547384+0000","last_peered":"2023-12-20T05:43:46.547384+0000","last_clean":"2023-12-20T05:43:46.547384+0000","last_became_active":"2023-12-20T05:43:46.540765+0000","last_became_peered":"2023-12-20T05:43:46.540765+0000","last_unstale":"2023-12-20T05:43:46.547384+0000","last_undegraded":"2023-12-20T05:43:46.547384+0000","last_fullsized":"2023-12-20T05:43:46.547384+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:15.079889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.19","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.522054+0000","last_change":"2023-12-20T05:43:46.108835+0000","last_active":"2023-12-20T05:43:46.522054+0000","last_peered":"2023-12-20T05:43:46.522054+0000","last_clean":"2023-12-20T05:43:46.522054+0000","last_became_active":"2023-12-20T05:43:46.108514+0000","last_became_peered":"2023-12-20T05:43:46.108514+0000","last_unstale":"2023-12-20T05:43:46.522054+0000","last_undegraded":"2023-12-20T05:43:46.522054+0000","last_fullsized":"2023-12-20T05:43:46.522054+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:57:30.268487+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.19","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.526636+0000","last_change":"2023-12-20T05:43:46.524801+0000","last_active":"2023-12-20T05:43:46.526636+0000","last_peered":"2023-12-20T05:43:46.526636+0000","last_clean":"2023-12-20T05:43:46.526636+0000","last_became_active":"2023-12-20T05:43:46.524489+0000","last_became_peered":"2023-12-20T05:43:46.524489+0000","last_unstale":"2023-12-20T05:43:46.526636+0000","last_undegraded":"2023-12-20T05:43:46.526636+0000","last_fullsized":"2023-12-20T05:43:46.526636+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:42:15.654028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.18","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.515492+0000","last_change":"2023-12-20T05:43:46.121725+0000","last_active":"2023-12-20T05:43:46.515492+0000","last_peered":"2023-12-20T05:43:46.515492+0000","last_clean":"2023-12-20T05:43:46.515492+0000","last_became_active":"2023-12-20T05:43:46.121428+0000","last_became_peered":"2023-12-20T05:43:46.121428+0000","last_unstale":"2023-12-20T05:43:46.515492+0000","last_undegraded":"2023-12-20T05:43:46.515492+0000","last_fullsized":"2023-12-20T05:43:46.515492+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:37:45.623160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.16","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538438+0000","last_change":"2023-12-20T05:43:46.528228+0000","last_active":"2023-12-20T05:43:46.538438+0000","last_peered":"2023-12-20T05:43:46.538438+0000","last_clean":"2023-12-20T05:43:46.538438+0000","last_became_active":"2023-12-20T05:43:46.527432+0000","last_became_peered":"2023-12-20T05:43:46.527432+0000","last_unstale":"2023-12-20T05:43:46.538438+0000","last_undegraded":"2023-12-20T05:43:46.538438+0000","last_fullsized":"2023-12-20T05:43:46.538438+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:32:18.577407+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.17","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.517436+0000","last_change":"2023-12-20T05:43:46.118424+0000","last_active":"2023-12-20T05:43:46.517436+0000","last_peered":"2023-12-20T05:43:46.517436+0000","last_clean":"2023-12-20T05:43:46.517436+0000","last_became_active":"2023-12-20T05:43:46.118119+0000","last_became_peered":"2023-12-20T05:43:46.118119+0000","last_unstale":"2023-12-20T05:43:46.517436+0000","last_undegraded":"2023-12-20T05:43:46.517436+0000","last_fullsized":"2023-12-20T05:43:46.517436+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:44:42.153173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.17","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.524910+0000","last_change":"2023-12-20T05:43:46.521313+0000","last_active":"2023-12-20T05:43:46.524910+0000","last_peered":"2023-12-20T05:43:46.524910+0000","last_clean":"2023-12-20T05:43:46.524910+0000","last_became_active":"2023-12-20T05:43:46.521014+0000","last_became_peered":"2023-12-20T05:43:46.521014+0000","last_unstale":"2023-12-20T05:43:46.524910+0000","last_undegraded":"2023-12-20T05:43:46.524910+0000","last_fullsized":"2023-12-20T05:43:46.524910+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:42:15.654028+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.16","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.516505+0000","last_change":"2023-12-20T05:43:46.126253+0000","last_active":"2023-12-20T05:43:46.516505+0000","last_peered":"2023-12-20T05:43:46.516505+0000","last_clean":"2023-12-20T05:43:46.516505+0000","last_became_active":"2023-12-20T05:43:46.125916+0000","last_became_peered":"2023-12-20T05:43:46.125916+0000","last_unstale":"2023-12-20T05:43:46.516505+0000","last_undegraded":"2023-12-20T05:43:46.516505+0000","last_fullsized":"2023-12-20T05:43:46.516505+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:42:29.151990+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.14","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542915+0000","last_change":"2023-12-20T05:43:46.535612+0000","last_active":"2023-12-20T05:43:46.542915+0000","last_peered":"2023-12-20T05:43:46.542915+0000","last_clean":"2023-12-20T05:43:46.542915+0000","last_became_active":"2023-12-20T05:43:46.535086+0000","last_became_peered":"2023-12-20T05:43:46.535086+0000","last_unstale":"2023-12-20T05:43:46.542915+0000","last_undegraded":"2023-12-20T05:43:46.542915+0000","last_fullsized":"2023-12-20T05:43:46.542915+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:39:35.414585+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.15","version":"158'3","reported_seq":13,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538386+0000","last_change":"2023-12-20T05:43:46.114674+0000","last_active":"2023-12-20T05:43:46.538386+0000","last_peered":"2023-12-20T05:43:46.538386+0000","last_clean":"2023-12-20T05:43:46.538386+0000","last_became_active":"2023-12-20T05:43:46.114403+0000","last_became_peered":"2023-12-20T05:43:46.114403+0000","last_unstale":"2023-12-20T05:43:46.538386+0000","last_undegraded":"2023-12-20T05:43:46.538386+0000","last_fullsized":"2023-12-20T05:43:46.538386+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:37.148286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.15","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545472+0000","last_change":"2023-12-20T05:43:46.541901+0000","last_active":"2023-12-20T05:43:46.545472+0000","last_peered":"2023-12-20T05:43:46.545472+0000","last_clean":"2023-12-20T05:43:46.545472+0000","last_became_active":"2023-12-20T05:43:46.541617+0000","last_became_peered":"2023-12-20T05:43:46.541617+0000","last_unstale":"2023-12-20T05:43:46.545472+0000","last_undegraded":"2023-12-20T05:43:46.545472+0000","last_fullsized":"2023-12-20T05:43:46.545472+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:57:21.808421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.14","version":"158'6","reported_seq":16,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.553908+0000","last_change":"2023-12-20T05:43:46.121257+0000","last_active":"2023-12-20T05:43:46.553908+0000","last_peered":"2023-12-20T05:43:46.553908+0000","last_clean":"2023-12-20T05:43:46.553908+0000","last_became_active":"2023-12-20T05:43:46.120975+0000","last_became_peered":"2023-12-20T05:43:46.120975+0000","last_unstale":"2023-12-20T05:43:46.553908+0000","last_undegraded":"2023-12-20T05:43:46.553908+0000","last_fullsized":"2023-12-20T05:43:46.553908+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:21:30.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.13","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.537847+0000","last_change":"2023-12-20T05:43:46.528104+0000","last_active":"2023-12-20T05:43:46.537847+0000","last_peered":"2023-12-20T05:43:46.537847+0000","last_clean":"2023-12-20T05:43:46.537847+0000","last_became_active":"2023-12-20T05:43:46.527854+0000","last_became_peered":"2023-12-20T05:43:46.527854+0000","last_unstale":"2023-12-20T05:43:46.537847+0000","last_undegraded":"2023-12-20T05:43:46.537847+0000","last_fullsized":"2023-12-20T05:43:46.537847+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:46:50.112808+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.12","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.511044+0000","last_change":"2023-12-20T05:43:46.113071+0000","last_active":"2023-12-20T05:43:46.511044+0000","last_peered":"2023-12-20T05:43:46.511044+0000","last_clean":"2023-12-20T05:43:46.511044+0000","last_became_active":"2023-12-20T05:43:46.112791+0000","last_became_peered":"2023-12-20T05:43:46.112791+0000","last_unstale":"2023-12-20T05:43:46.511044+0000","last_undegraded":"2023-12-20T05:43:46.511044+0000","last_fullsized":"2023-12-20T05:43:46.511044+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:31.016776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.10","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.553477+0000","last_change":"2023-12-20T05:43:46.552804+0000","last_active":"2023-12-20T05:43:46.553477+0000","last_peered":"2023-12-20T05:43:46.553477+0000","last_clean":"2023-12-20T05:43:46.553477+0000","last_became_active":"2023-12-20T05:43:46.552479+0000","last_became_peered":"2023-12-20T05:43:46.552479+0000","last_unstale":"2023-12-20T05:43:46.553477+0000","last_undegraded":"2023-12-20T05:43:46.553477+0000","last_fullsized":"2023-12-20T05:43:46.553477+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:28:41.860606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.11","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.516133+0000","last_change":"2023-12-20T05:43:46.120127+0000","last_active":"2023-12-20T05:43:46.516133+0000","last_peered":"2023-12-20T05:43:46.516133+0000","last_clean":"2023-12-20T05:43:46.516133+0000","last_became_active":"2023-12-20T05:43:46.119571+0000","last_became_peered":"2023-12-20T05:43:46.119571+0000","last_unstale":"2023-12-20T05:43:46.516133+0000","last_undegraded":"2023-12-20T05:43:46.516133+0000","last_fullsized":"2023-12-20T05:43:46.516133+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:21:30.566702+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.11","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545804+0000","last_change":"2023-12-20T05:43:46.540846+0000","last_active":"2023-12-20T05:43:46.545804+0000","last_peered":"2023-12-20T05:43:46.545804+0000","last_clean":"2023-12-20T05:43:46.545804+0000","last_became_active":"2023-12-20T05:43:46.539573+0000","last_became_peered":"2023-12-20T05:43:46.539573+0000","last_unstale":"2023-12-20T05:43:46.545804+0000","last_undegraded":"2023-12-20T05:43:46.545804+0000","last_fullsized":"2023-12-20T05:43:46.545804+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:19:46.567327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.10","version":"158'8","reported_seq":18,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.553020+0000","last_change":"2023-12-20T05:43:46.119425+0000","last_active":"2023-12-20T05:43:46.553020+0000","last_peered":"2023-12-20T05:43:46.553020+0000","last_clean":"2023-12-20T05:43:46.553020+0000","last_became_active":"2023-12-20T05:43:46.118802+0000","last_became_peered":"2023-12-20T05:43:46.118802+0000","last_unstale":"2023-12-20T05:43:46.553020+0000","last_undegraded":"2023-12-20T05:43:46.553020+0000","last_fullsized":"2023-12-20T05:43:46.553020+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:01:46.684943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"19.e","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.547209+0000","last_change":"2023-12-20T05:43:46.541550+0000","last_active":"2023-12-20T05:43:46.547209+0000","last_peered":"2023-12-20T05:43:46.547209+0000","last_clean":"2023-12-20T05:43:46.547209+0000","last_became_active":"2023-12-20T05:43:46.541278+0000","last_became_peered":"2023-12-20T05:43:46.541278+0000","last_unstale":"2023-12-20T05:43:46.547209+0000","last_undegraded":"2023-12-20T05:43:46.547209+0000","last_fullsized":"2023-12-20T05:43:46.547209+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:15:17.571592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"18.f","version":"158'1","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545025+0000","last_change":"2023-12-20T05:43:46.120827+0000","last_active":"2023-12-20T05:43:46.545025+0000","last_peered":"2023-12-20T05:43:46.545025+0000","last_clean":"2023-12-20T05:43:46.545025+0000","last_became_active":"2023-12-20T05:43:46.120538+0000","last_became_peered":"2023-12-20T05:43:46.120538+0000","last_unstale":"2023-12-20T05:43:46.545025+0000","last_undegraded":"2023-12-20T05:43:46.545025+0000","last_fullsized":"2023-12-20T05:43:46.545025+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:20:54.119580+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.f","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.531978+0000","last_change":"2023-12-20T05:43:46.528569+0000","last_active":"2023-12-20T05:43:46.531978+0000","last_peered":"2023-12-20T05:43:46.531978+0000","last_clean":"2023-12-20T05:43:46.531978+0000","last_became_active":"2023-12-20T05:43:46.528243+0000","last_became_peered":"2023-12-20T05:43:46.528243+0000","last_unstale":"2023-12-20T05:43:46.531978+0000","last_undegraded":"2023-12-20T05:43:46.531978+0000","last_fullsized":"2023-12-20T05:43:46.531978+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:15:17.571592+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.e","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.547814+0000","last_change":"2023-12-20T05:43:46.126934+0000","last_active":"2023-12-20T05:43:46.547814+0000","last_peered":"2023-12-20T05:43:46.547814+0000","last_clean":"2023-12-20T05:43:46.547814+0000","last_became_active":"2023-12-20T05:43:46.126625+0000","last_became_peered":"2023-12-20T05:43:46.126625+0000","last_unstale":"2023-12-20T05:43:46.547814+0000","last_undegraded":"2023-12-20T05:43:46.547814+0000","last_fullsized":"2023-12-20T05:43:46.547814+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:53:59.899060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.c","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.523892+0000","last_change":"2023-12-20T05:43:46.522829+0000","last_active":"2023-12-20T05:43:46.523892+0000","last_peered":"2023-12-20T05:43:46.523892+0000","last_clean":"2023-12-20T05:43:46.523892+0000","last_became_active":"2023-12-20T05:43:46.522103+0000","last_became_peered":"2023-12-20T05:43:46.522103+0000","last_unstale":"2023-12-20T05:43:46.523892+0000","last_undegraded":"2023-12-20T05:43:46.523892+0000","last_fullsized":"2023-12-20T05:43:46.523892+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:21:16.244410+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.d","version":"158'1","reported_seq":11,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.537461+0000","last_change":"2023-12-20T05:43:46.111276+0000","last_active":"2023-12-20T05:43:46.537461+0000","last_peered":"2023-12-20T05:43:46.537461+0000","last_clean":"2023-12-20T05:43:46.537461+0000","last_became_active":"2023-12-20T05:43:46.110933+0000","last_became_peered":"2023-12-20T05:43:46.110933+0000","last_unstale":"2023-12-20T05:43:46.537461+0000","last_undegraded":"2023-12-20T05:43:46.537461+0000","last_fullsized":"2023-12-20T05:43:46.537461+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:57:20.805569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"19.d","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.526469+0000","last_change":"2023-12-20T05:43:46.524339+0000","last_active":"2023-12-20T05:43:46.526469+0000","last_peered":"2023-12-20T05:43:46.526469+0000","last_clean":"2023-12-20T05:43:46.526469+0000","last_became_active":"2023-12-20T05:43:46.524070+0000","last_became_peered":"2023-12-20T05:43:46.524070+0000","last_unstale":"2023-12-20T05:43:46.526469+0000","last_undegraded":"2023-12-20T05:43:46.526469+0000","last_fullsized":"2023-12-20T05:43:46.526469+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:56.569699+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.c","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.516904+0000","last_change":"2023-12-20T05:43:46.118736+0000","last_active":"2023-12-20T05:43:46.516904+0000","last_peered":"2023-12-20T05:43:46.516904+0000","last_clean":"2023-12-20T05:43:46.516904+0000","last_became_active":"2023-12-20T05:43:46.118349+0000","last_became_peered":"2023-12-20T05:43:46.118349+0000","last_unstale":"2023-12-20T05:43:46.516904+0000","last_undegraded":"2023-12-20T05:43:46.516904+0000","last_fullsized":"2023-12-20T05:43:46.516904+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:06:23.077181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.a","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545549+0000","last_change":"2023-12-20T05:43:46.544192+0000","last_active":"2023-12-20T05:43:46.545549+0000","last_peered":"2023-12-20T05:43:46.545549+0000","last_clean":"2023-12-20T05:43:46.545549+0000","last_became_active":"2023-12-20T05:43:46.543504+0000","last_became_peered":"2023-12-20T05:43:46.543504+0000","last_unstale":"2023-12-20T05:43:46.545549+0000","last_undegraded":"2023-12-20T05:43:46.545549+0000","last_fullsized":"2023-12-20T05:43:46.545549+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:35:19.638668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.b","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.538161+0000","last_change":"2023-12-20T05:43:46.119209+0000","last_active":"2023-12-20T05:43:46.538161+0000","last_peered":"2023-12-20T05:43:46.538161+0000","last_clean":"2023-12-20T05:43:46.538161+0000","last_became_active":"2023-12-20T05:43:46.117369+0000","last_became_peered":"2023-12-20T05:43:46.117369+0000","last_unstale":"2023-12-20T05:43:46.538161+0000","last_undegraded":"2023-12-20T05:43:46.538161+0000","last_fullsized":"2023-12-20T05:43:46.538161+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:20.427973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"19.b","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.546887+0000","last_change":"2023-12-20T05:43:46.545720+0000","last_active":"2023-12-20T05:43:46.546887+0000","last_peered":"2023-12-20T05:43:46.546887+0000","last_clean":"2023-12-20T05:43:46.546887+0000","last_became_active":"2023-12-20T05:43:46.545328+0000","last_became_peered":"2023-12-20T05:43:46.545328+0000","last_unstale":"2023-12-20T05:43:46.546887+0000","last_undegraded":"2023-12-20T05:43:46.546887+0000","last_fullsized":"2023-12-20T05:43:46.546887+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:12:29.110689+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.a","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.515258+0000","last_change":"2023-12-20T05:43:46.126273+0000","last_active":"2023-12-20T05:43:46.515258+0000","last_peered":"2023-12-20T05:43:46.515258+0000","last_clean":"2023-12-20T05:43:46.515258+0000","last_became_active":"2023-12-20T05:43:46.125888+0000","last_became_peered":"2023-12-20T05:43:46.125888+0000","last_unstale":"2023-12-20T05:43:46.515258+0000","last_undegraded":"2023-12-20T05:43:46.515258+0000","last_fullsized":"2023-12-20T05:43:46.515258+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:55:22.693101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.8","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.525197+0000","last_change":"2023-12-20T05:43:46.523070+0000","last_active":"2023-12-20T05:43:46.525197+0000","last_peered":"2023-12-20T05:43:46.525197+0000","last_clean":"2023-12-20T05:43:46.525197+0000","last_became_active":"2023-12-20T05:43:46.522763+0000","last_became_peered":"2023-12-20T05:43:46.522763+0000","last_unstale":"2023-12-20T05:43:46.525197+0000","last_undegraded":"2023-12-20T05:43:46.525197+0000","last_fullsized":"2023-12-20T05:43:46.525197+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:47:52.037903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.9","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.514907+0000","last_change":"2023-12-20T05:43:46.122985+0000","last_active":"2023-12-20T05:43:46.514907+0000","last_peered":"2023-12-20T05:43:46.514907+0000","last_clean":"2023-12-20T05:43:46.514907+0000","last_became_active":"2023-12-20T05:43:46.121903+0000","last_became_peered":"2023-12-20T05:43:46.121903+0000","last_unstale":"2023-12-20T05:43:46.514907+0000","last_undegraded":"2023-12-20T05:43:46.514907+0000","last_fullsized":"2023-12-20T05:43:46.514907+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:48:57.395143+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.9","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.545125+0000","last_change":"2023-12-20T05:43:46.537360+0000","last_active":"2023-12-20T05:43:46.545125+0000","last_peered":"2023-12-20T05:43:46.545125+0000","last_clean":"2023-12-20T05:43:46.545125+0000","last_became_active":"2023-12-20T05:43:46.537083+0000","last_became_peered":"2023-12-20T05:43:46.537083+0000","last_unstale":"2023-12-20T05:43:46.545125+0000","last_undegraded":"2023-12-20T05:43:46.545125+0000","last_fullsized":"2023-12-20T05:43:46.545125+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:28:41.860606+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.8","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.519208+0000","last_change":"2023-12-20T05:43:46.122667+0000","last_active":"2023-12-20T05:43:46.519208+0000","last_peered":"2023-12-20T05:43:46.519208+0000","last_clean":"2023-12-20T05:43:46.519208+0000","last_became_active":"2023-12-20T05:43:46.122357+0000","last_became_peered":"2023-12-20T05:43:46.122357+0000","last_unstale":"2023-12-20T05:43:46.519208+0000","last_undegraded":"2023-12-20T05:43:46.519208+0000","last_fullsized":"2023-12-20T05:43:46.519208+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:46.278593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.6","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.537692+0000","last_change":"2023-12-20T05:43:46.530254+0000","last_active":"2023-12-20T05:43:46.537692+0000","last_peered":"2023-12-20T05:43:46.537692+0000","last_clean":"2023-12-20T05:43:46.537692+0000","last_became_active":"2023-12-20T05:43:46.529971+0000","last_became_peered":"2023-12-20T05:43:46.529971+0000","last_unstale":"2023-12-20T05:43:46.537692+0000","last_undegraded":"2023-12-20T05:43:46.537692+0000","last_fullsized":"2023-12-20T05:43:46.537692+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:48:58.397995+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.7","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.539153+0000","last_change":"2023-12-20T05:43:46.123658+0000","last_active":"2023-12-20T05:43:46.539153+0000","last_peered":"2023-12-20T05:43:46.539153+0000","last_clean":"2023-12-20T05:43:46.539153+0000","last_became_active":"2023-12-20T05:43:46.123314+0000","last_became_peered":"2023-12-20T05:43:46.123314+0000","last_unstale":"2023-12-20T05:43:46.539153+0000","last_undegraded":"2023-12-20T05:43:46.539153+0000","last_fullsized":"2023-12-20T05:43:46.539153+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:41:06.068217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"19.7","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.550288+0000","last_change":"2023-12-20T05:43:46.549246+0000","last_active":"2023-12-20T05:43:46.550288+0000","last_peered":"2023-12-20T05:43:46.550288+0000","last_clean":"2023-12-20T05:43:46.550288+0000","last_became_active":"2023-12-20T05:43:46.549014+0000","last_became_peered":"2023-12-20T05:43:46.549014+0000","last_unstale":"2023-12-20T05:43:46.550288+0000","last_undegraded":"2023-12-20T05:43:46.550288+0000","last_fullsized":"2023-12-20T05:43:46.550288+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:00:55.245435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"18.6","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.522907+0000","last_change":"2023-12-20T05:43:46.113428+0000","last_active":"2023-12-20T05:43:46.522907+0000","last_peered":"2023-12-20T05:43:46.522907+0000","last_clean":"2023-12-20T05:43:46.522907+0000","last_became_active":"2023-12-20T05:43:46.113096+0000","last_became_peered":"2023-12-20T05:43:46.113096+0000","last_unstale":"2023-12-20T05:43:46.522907+0000","last_undegraded":"2023-12-20T05:43:46.522907+0000","last_fullsized":"2023-12-20T05:43:46.522907+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:37.148286+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.4","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.534189+0000","last_change":"2023-12-20T05:43:46.529405+0000","last_active":"2023-12-20T05:43:46.534189+0000","last_peered":"2023-12-20T05:43:46.534189+0000","last_clean":"2023-12-20T05:43:46.534189+0000","last_became_active":"2023-12-20T05:43:46.529148+0000","last_became_peered":"2023-12-20T05:43:46.529148+0000","last_unstale":"2023-12-20T05:43:46.534189+0000","last_undegraded":"2023-12-20T05:43:46.534189+0000","last_fullsized":"2023-12-20T05:43:46.534189+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:15.079889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.5","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.519015+0000","last_change":"2023-12-20T05:43:46.124867+0000","last_active":"2023-12-20T05:43:46.519015+0000","last_peered":"2023-12-20T05:43:46.519015+0000","last_clean":"2023-12-20T05:43:46.519015+0000","last_became_active":"2023-12-20T05:43:46.124521+0000","last_became_peered":"2023-12-20T05:43:46.124521+0000","last_unstale":"2023-12-20T05:43:46.519015+0000","last_undegraded":"2023-12-20T05:43:46.519015+0000","last_fullsized":"2023-12-20T05:43:46.519015+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:43:15.471880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.5","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.544450+0000","last_change":"2023-12-20T05:43:46.539564+0000","last_active":"2023-12-20T05:43:46.544450+0000","last_peered":"2023-12-20T05:43:46.544450+0000","last_clean":"2023-12-20T05:43:46.544450+0000","last_became_active":"2023-12-20T05:43:46.539299+0000","last_became_peered":"2023-12-20T05:43:46.539299+0000","last_unstale":"2023-12-20T05:43:46.544450+0000","last_undegraded":"2023-12-20T05:43:46.544450+0000","last_fullsized":"2023-12-20T05:43:46.544450+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:14:48.759628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"18.4","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.519285+0000","last_change":"2023-12-20T05:43:46.115765+0000","last_active":"2023-12-20T05:43:46.519285+0000","last_peered":"2023-12-20T05:43:46.519285+0000","last_clean":"2023-12-20T05:43:46.519285+0000","last_became_active":"2023-12-20T05:43:46.115487+0000","last_became_peered":"2023-12-20T05:43:46.115487+0000","last_unstale":"2023-12-20T05:43:46.519285+0000","last_undegraded":"2023-12-20T05:43:46.519285+0000","last_fullsized":"2023-12-20T05:43:46.519285+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:37:29.678269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.2","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.543367+0000","last_change":"2023-12-20T05:43:46.534683+0000","last_active":"2023-12-20T05:43:46.543367+0000","last_peered":"2023-12-20T05:43:46.543367+0000","last_clean":"2023-12-20T05:43:46.543367+0000","last_became_active":"2023-12-20T05:43:46.534354+0000","last_became_peered":"2023-12-20T05:43:46.534354+0000","last_unstale":"2023-12-20T05:43:46.543367+0000","last_undegraded":"2023-12-20T05:43:46.543367+0000","last_fullsized":"2023-12-20T05:43:46.543367+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:05:29.056372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.3","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.515801+0000","last_change":"2023-12-20T05:43:46.119875+0000","last_active":"2023-12-20T05:43:46.515801+0000","last_peered":"2023-12-20T05:43:46.515801+0000","last_clean":"2023-12-20T05:43:46.515801+0000","last_became_active":"2023-12-20T05:43:46.119594+0000","last_became_peered":"2023-12-20T05:43:46.119594+0000","last_unstale":"2023-12-20T05:43:46.515801+0000","last_undegraded":"2023-12-20T05:43:46.515801+0000","last_fullsized":"2023-12-20T05:43:46.515801+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:01:46.684943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"19.3","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.526213+0000","last_change":"2023-12-20T05:43:46.524269+0000","last_active":"2023-12-20T05:43:46.526213+0000","last_peered":"2023-12-20T05:43:46.526213+0000","last_clean":"2023-12-20T05:43:46.526213+0000","last_became_active":"2023-12-20T05:43:46.523979+0000","last_became_peered":"2023-12-20T05:43:46.523979+0000","last_unstale":"2023-12-20T05:43:46.526213+0000","last_undegraded":"2023-12-20T05:43:46.526213+0000","last_fullsized":"2023-12-20T05:43:46.526213+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:20:15.079889+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"18.2","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.519487+0000","last_change":"2023-12-20T05:43:46.114892+0000","last_active":"2023-12-20T05:43:46.519487+0000","last_peered":"2023-12-20T05:43:46.519487+0000","last_clean":"2023-12-20T05:43:46.519487+0000","last_became_active":"2023-12-20T05:43:46.113045+0000","last_became_peered":"2023-12-20T05:43:46.113045+0000","last_unstale":"2023-12-20T05:43:46.519487+0000","last_undegraded":"2023-12-20T05:43:46.519487+0000","last_fullsized":"2023-12-20T05:43:46.519487+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:13.245566+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"19.0","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.553252+0000","last_change":"2023-12-20T05:43:46.552510+0000","last_active":"2023-12-20T05:43:46.553252+0000","last_peered":"2023-12-20T05:43:46.553252+0000","last_clean":"2023-12-20T05:43:46.553252+0000","last_became_active":"2023-12-20T05:43:46.552184+0000","last_became_peered":"2023-12-20T05:43:46.552184+0000","last_unstale":"2023-12-20T05:43:46.553252+0000","last_undegraded":"2023-12-20T05:43:46.553252+0000","last_fullsized":"2023-12-20T05:43:46.553252+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:34:19.412615+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"18.1","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.515579+0000","last_change":"2023-12-20T05:43:46.123211+0000","last_active":"2023-12-20T05:43:46.515579+0000","last_peered":"2023-12-20T05:43:46.515579+0000","last_clean":"2023-12-20T05:43:46.515579+0000","last_became_active":"2023-12-20T05:43:46.122598+0000","last_became_peered":"2023-12-20T05:43:46.122598+0000","last_unstale":"2023-12-20T05:43:46.515579+0000","last_undegraded":"2023-12-20T05:43:46.515579+0000","last_fullsized":"2023-12-20T05:43:46.515579+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:07.874108+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"19.1","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.532414+0000","last_change":"2023-12-20T05:43:46.530064+0000","last_active":"2023-12-20T05:43:46.532414+0000","last_peered":"2023-12-20T05:43:46.532414+0000","last_clean":"2023-12-20T05:43:46.532414+0000","last_became_active":"2023-12-20T05:43:46.529778+0000","last_became_peered":"2023-12-20T05:43:46.529778+0000","last_unstale":"2023-12-20T05:43:46.532414+0000","last_undegraded":"2023-12-20T05:43:46.532414+0000","last_fullsized":"2023-12-20T05:43:46.532414+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:08:27.340043+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"18.0","version":"0'0","reported_seq":10,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.519297+0000","last_change":"2023-12-20T05:43:46.115882+0000","last_active":"2023-12-20T05:43:46.519297+0000","last_peered":"2023-12-20T05:43:46.519297+0000","last_clean":"2023-12-20T05:43:46.519297+0000","last_became_active":"2023-12-20T05:43:46.115604+0000","last_became_peered":"2023-12-20T05:43:46.115604+0000","last_unstale":"2023-12-20T05:43:46.519297+0000","last_undegraded":"2023-12-20T05:43:46.519297+0000","last_fullsized":"2023-12-20T05:43:46.519297+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:51:48.971451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"19.12","version":"0'0","reported_seq":9,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.543349+0000","last_change":"2023-12-20T05:43:46.529405+0000","last_active":"2023-12-20T05:43:46.543349+0000","last_peered":"2023-12-20T05:43:46.543349+0000","last_clean":"2023-12-20T05:43:46.543349+0000","last_became_active":"2023-12-20T05:43:46.529139+0000","last_became_peered":"2023-12-20T05:43:46.529139+0000","last_unstale":"2023-12-20T05:43:46.543349+0000","last_undegraded":"2023-12-20T05:43:46.543349+0000","last_fullsized":"2023-12-20T05:43:46.543349+0000","mapping_epoch":157,"log_start":"0'0","ondisk_log_start":"0'0","created":157,"last_epoch_clean":158,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:46.090734+0000","last_clean_scrub_stamp":"2023-12-20T05:43:46.090734+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:19.676103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"18.13","version":"158'2","reported_seq":12,"reported_epoch":158,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.542456+0000","last_change":"2023-12-20T05:43:46.115728+0000","last_active":"2023-12-20T05:43:46.542456+0000","last_peered":"2023-12-20T05:43:46.542456+0000","last_clean":"2023-12-20T05:43:46.542456+0000","last_became_active":"2023-12-20T05:43:46.115359+0000","last_became_peered":"2023-12-20T05:43:46.115359+0000","last_unstale":"2023-12-20T05:43:46.542456+0000","last_undegraded":"2023-12-20T05:43:46.542456+0000","last_fullsized":"2023-12-20T05:43:46.542456+0000","mapping_epoch":156,"log_start":"0'0","ondisk_log_start":"0'0","created":156,"last_epoch_clean":157,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:43:45.087882+0000","last_clean_scrub_stamp":"2023-12-20T05:43:45.087882+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:41.080590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"146'3124","reported_seq":3666,"reported_epoch":157,"state":"active+clean","last_fresh":"2023-12-20T05:43:46.105331+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T05:43:46.105331+0000","last_peered":"2023-12-20T05:43:46.105331+0000","last_clean":"2023-12-20T05:43:46.105331+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T05:43:46.105331+0000","last_undegraded":"2023-12-20T05:43:46.105331+0000","last_fullsized":"2023-12-20T05:43:46.105331+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3124,"log_dups_size":0,"ondisk_log_size":3124,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1213,"num_read_kb":2407,"num_write":381,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":19,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":18,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1213,"num_read_kb":2407,"num_write":381,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3124,"ondisk_log_size":3124,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575749,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":33524,"kb_used_data":2476,"kb_used_omap":0,"kb_used_meta":31040,"kb_avail":94338316,"statfs":{"total":96636764160,"available":96602435584,"internally_reserved":0,"allocated":2535424,"data_stored":1520208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":31784960},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.73799999999999999,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.48699999999999999,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.93500000000000005,"5min":1.133,"15min":1.133},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.75700000000000001,"15min":0.73999999999999999},"min":{"1min":0.52500000000000002,"5min":0.45500000000000002,"15min":0.40300000000000002},"max":{"1min":0.93000000000000005,"5min":0.95599999999999996,"15min":1.077},"last":0.82599999999999996}]},{"osd":1,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.74299999999999999,"15min":0.74099999999999999},"min":{"1min":0.57499999999999996,"5min":0.48099999999999998,"15min":0.45200000000000001},"max":{"1min":1.0269999999999999,"5min":1.0269999999999999,"15min":1.131},"last":0.68700000000000006},{"interface":"front","average":{"1min":0.77600000000000002,"5min":0.76200000000000001,"15min":0.754},"min":{"1min":0.59299999999999997,"5min":0.53700000000000003,"15min":0.47699999999999998},"max":{"1min":0.92100000000000004,"5min":1.0129999999999999,"15min":1.056},"last":0.752}]},{"osd":2,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.73799999999999999,"15min":0.72999999999999998},"min":{"1min":0.56599999999999995,"5min":0.47099999999999997,"15min":0.45500000000000002},"max":{"1min":0.85799999999999998,"5min":1.0589999999999999,"15min":1.0589999999999999},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.79200000000000004,"5min":0.76400000000000001,"15min":0.76800000000000002},"min":{"1min":0.54800000000000004,"5min":0.52200000000000002,"15min":0.501},"max":{"1min":0.97999999999999998,"5min":1.099,"15min":2.2160000000000002},"last":0.84399999999999997}]},{"osd":3,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.75800000000000001,"15min":0.752},"min":{"1min":0.59699999999999998,"5min":0.52100000000000002,"15min":0.46899999999999997},"max":{"1min":0.91000000000000003,"5min":0.98999999999999999,"15min":2.6829999999999998},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.82499999999999996,"5min":0.79900000000000004,"15min":0.79700000000000004},"min":{"1min":0.62,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":2.1259999999999999},"last":0.77300000000000002}]},{"osd":4,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.59199999999999997,"15min":0.60099999999999998},"min":{"1min":0.39900000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.89800000000000002,"5min":0.89800000000000002,"15min":1.8180000000000001},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.57999999999999996,"5min":0.626,"15min":0.63800000000000001},"min":{"1min":0.438,"5min":0.40799999999999997,"15min":0.40200000000000002},"max":{"1min":0.71799999999999997,"5min":1.01,"15min":2.7370000000000001},"last":0.50900000000000001}]},{"osd":5,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.65100000000000002,"5min":0.65600000000000003,"15min":0.66200000000000003},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.96299999999999997,"5min":0.97399999999999998,"15min":2.27},"last":0.48599999999999999},{"interface":"front","average":{"1min":0.68600000000000005,"5min":0.68600000000000005,"15min":0.68400000000000005},"min":{"1min":0.501,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.92800000000000005,"5min":1.0029999999999999,"15min":1.0700000000000001},"last":0.58499999999999996}]},{"osd":6,"last update":"Wed Dec 20 05:42:44 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.747,"15min":0.746},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.42699999999999999},"max":{"1min":1.002,"5min":1.071,"15min":1.1699999999999999},"last":0.53900000000000003},{"interface":"front","average":{"1min":0.71399999999999997,"5min":0.71199999999999997,"15min":0.70899999999999996},"min":{"1min":0.50800000000000001,"5min":0.45400000000000001,"15min":0.45200000000000001},"max":{"1min":0.95899999999999996,"5min":1.1850000000000001,"15min":1.1850000000000001},"last":0.61199999999999999}]}]},{"osd":6,"up_from":13,"seq":55834575749,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28848,"kb_used_data":1896,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342992,"statfs":{"total":96636764160,"available":96607223808,"internally_reserved":0,"allocated":1941504,"data_stored":929840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.74099999999999999,"5min":0.74399999999999999,"15min":0.754},"min":{"1min":0.52100000000000002,"5min":0.45700000000000002,"15min":0.36899999999999999},"max":{"1min":0.99299999999999999,"5min":1,"15min":1.093},"last":0.50800000000000001},{"interface":"front","average":{"1min":0.748,"5min":0.755,"15min":0.755},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":1.0189999999999999,"5min":1.0189999999999999,"15min":1.101},"last":0.80400000000000005}]},{"osd":1,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.75800000000000001,"5min":0.753,"15min":0.752},"min":{"1min":0.54100000000000004,"5min":0.5,"15min":0.42699999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.03},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76100000000000001,"15min":0.77200000000000002},"min":{"1min":0.499,"5min":0.48699999999999999,"15min":0.46200000000000002},"max":{"1min":0.93700000000000006,"5min":1.0149999999999999,"15min":1.038},"last":0.73299999999999998}]},{"osd":2,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.81100000000000005,"5min":0.77600000000000002,"15min":0.78100000000000003},"min":{"1min":0.63900000000000001,"5min":0.48099999999999998,"15min":0.47999999999999998},"max":{"1min":1.048,"5min":1.048,"15min":1.0960000000000001},"last":0.60099999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78600000000000003,"15min":0.78500000000000003},"min":{"1min":0.56699999999999995,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.99099999999999999,"5min":1.0369999999999999,"15min":1.083},"last":0.752}]},{"osd":3,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.83399999999999996,"5min":0.79600000000000004,"15min":0.79400000000000004},"min":{"1min":0.55700000000000005,"5min":0.53100000000000003,"15min":0.503},"max":{"1min":0.97499999999999998,"5min":1.036,"15min":1.0629999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.77500000000000002,"15min":0.78000000000000003},"min":{"1min":0.58299999999999996,"5min":0.504,"15min":0.504},"max":{"1min":0.93500000000000005,"5min":1.0169999999999999,"15min":1.1020000000000001},"last":0.78100000000000003}]},{"osd":4,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.64300000000000002,"15min":0.64100000000000001},"min":{"1min":0.45300000000000001,"5min":0.40799999999999997,"15min":0.36599999999999999},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":0.98599999999999999},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.69899999999999995,"5min":0.65200000000000002,"15min":0.66000000000000003},"min":{"1min":0.48199999999999998,"5min":0.46200000000000002,"15min":0.40999999999999998},"max":{"1min":0.89800000000000002,"5min":1.048,"15min":1.048},"last":0.85199999999999998}]},{"osd":5,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.65900000000000003,"5min":0.64900000000000002,"15min":0.65500000000000003},"min":{"1min":0.47599999999999998,"5min":0.42799999999999999,"15min":0.40200000000000002},"max":{"1min":0.999,"5min":0.999,"15min":1.0429999999999999},"last":0.879},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.71899999999999997},"min":{"1min":0.50700000000000001,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":0.95699999999999996,"5min":1.0109999999999999,"15min":1.0509999999999999},"last":0.81799999999999995}]},{"osd":7,"last update":"Wed Dec 20 05:43:02 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.74199999999999999,"15min":0.73899999999999999},"min":{"1min":0.52100000000000002,"5min":0.51200000000000001,"15min":0.45500000000000002},"max":{"1min":0.98699999999999999,"5min":1.095,"15min":1.095},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.73699999999999999,"15min":0.73799999999999999},"min":{"1min":0.52100000000000002,"5min":0.49299999999999999,"15min":0.49099999999999999},"max":{"1min":0.97599999999999998,"5min":1.0049999999999999,"15min":1.105},"last":0.83499999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":28860,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342980,"statfs":{"total":96636764160,"available":96607211520,"internally_reserved":0,"allocated":1953792,"data_stored":936143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.73599999999999999,"15min":0.73399999999999999},"min":{"1min":0.41899999999999998,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":0.98099999999999998,"5min":1.0229999999999999,"15min":1.0229999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.77300000000000002,"15min":0.76300000000000001},"min":{"1min":0.54300000000000004,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.97399999999999998,"5min":1.145,"15min":1.145},"last":0.66200000000000003}]},{"osd":1,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.71999999999999997,"5min":0.76200000000000001,"15min":0.751},"min":{"1min":0.496,"5min":0.44900000000000001,"15min":0.42799999999999999},"max":{"1min":0.92800000000000005,"5min":1.016,"15min":1.016},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.77800000000000002,"15min":0.76600000000000001},"min":{"1min":0.53900000000000003,"5min":0.47399999999999998,"15min":0.45000000000000001},"max":{"1min":1.0009999999999999,"5min":1.022,"15min":1.0569999999999999},"last":0.80600000000000005}]},{"osd":2,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.79500000000000004,"15min":0.77100000000000002},"min":{"1min":0.55700000000000005,"5min":0.48199999999999998,"15min":0.44700000000000001},"max":{"1min":1.091,"5min":1.204,"15min":1.204},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.76900000000000002,"15min":0.76400000000000001},"min":{"1min":0.58499999999999996,"5min":0.497,"15min":0.49399999999999999},"max":{"1min":0.94599999999999995,"5min":1.052,"15min":1.052},"last":0.82399999999999995}]},{"osd":3,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.79400000000000004,"15min":0.78400000000000003},"min":{"1min":0.56399999999999995,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":1.0369999999999999,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.80900000000000005,"15min":0.78900000000000003},"min":{"1min":0.65300000000000002,"5min":0.56699999999999995,"15min":0.53300000000000003},"max":{"1min":1.0800000000000001,"5min":1.107,"15min":1.107},"last":0.68400000000000005}]},{"osd":4,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.68700000000000006,"5min":0.65300000000000002,"15min":0.63600000000000001},"min":{"1min":0.499,"5min":0.436,"15min":0.41199999999999998},"max":{"1min":0.99099999999999999,"5min":1.167,"15min":1.167},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65700000000000003,"15min":0.63900000000000001},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.39100000000000001},"max":{"1min":0.97899999999999998,"5min":1.008,"15min":1.008},"last":0.70199999999999996}]},{"osd":6,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.76100000000000001,"5min":0.71299999999999997,"15min":0.69299999999999995},"min":{"1min":0.53400000000000003,"5min":0.48899999999999999,"15min":0.435},"max":{"1min":0.95899999999999996,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.73599999999999999},{"interface":"front","average":{"1min":0.67000000000000004,"5min":0.65600000000000003,"15min":0.63500000000000001},"min":{"1min":0.52100000000000002,"5min":0.43099999999999999,"15min":0.40999999999999998},"max":{"1min":0.98999999999999999,"5min":0.98999999999999999,"15min":1.085},"last":0.76800000000000002}]},{"osd":7,"last update":"Wed Dec 20 05:42:50 2023","interfaces":[{"interface":"back","average":{"1min":0.753,"5min":0.71899999999999997,"15min":0.70299999999999996},"min":{"1min":0.51900000000000002,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.98699999999999999,"5min":1.073,"15min":1.1020000000000001},"last":0.83899999999999997},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.78800000000000003,"15min":0.76100000000000001},"min":{"1min":0.70799999999999996,"5min":0.52200000000000002,"15min":0.48199999999999998},"max":{"1min":1.0229999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.85199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575751,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59196,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312644,"statfs":{"total":96636764160,"available":96576147456,"internally_reserved":0,"allocated":1953792,"data_stored":936143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34399999999999997},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56899999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.75,"15min":0.73799999999999999},"min":{"1min":0.57299999999999995,"5min":0.47599999999999998,"15min":0.35699999999999998},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":0.60199999999999998}]},{"osd":1,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.0249999999999999,"5min":1.077,"15min":1.137},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.75,"15min":0.75700000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.72599999999999998}]},{"osd":2,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.76500000000000001,"15min":0.76600000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.624},{"interface":"front","average":{"1min":0.749,"5min":0.78100000000000003,"15min":0.77800000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.51900000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.69599999999999995}]},{"osd":3,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.45500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.79800000000000004,"15min":0.79200000000000004},"min":{"1min":0.58699999999999997,"5min":0.51500000000000001,"15min":0.47999999999999998},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.70999999999999996}]},{"osd":5,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.895,"5min":2.895,"15min":2.895},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.629,"15min":0.621},"min":{"1min":0.41199999999999998,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.53200000000000003}]},{"osd":6,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.68700000000000006,"15min":0.65800000000000003},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.40100000000000002},"max":{"1min":3.8279999999999998,"5min":3.8279999999999998,"15min":3.8279999999999998},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.70499999999999996,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.44700000000000001,"15min":0.436},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.91400000000000003},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.77800000000000002,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.50800000000000001,"15min":0.47099999999999997},"max":{"1min":3.8690000000000002,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":0.76600000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575750,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63312,"kb_used_data":1928,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308528,"statfs":{"total":96636764160,"available":96571932672,"internally_reserved":0,"allocated":1974272,"data_stored":943106,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48099999999999998,"5min":0.47199999999999998,"15min":0.46300000000000002},"min":{"1min":0.311,"5min":0.29199999999999998,"15min":0.28299999999999997},"max":{"1min":0.61599999999999999,"5min":0.75800000000000001,"15min":0.88800000000000001},"last":0.43099999999999999},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.51900000000000002,"15min":0.52000000000000002},"min":{"1min":0.36499999999999999,"5min":0.308,"15min":0.29799999999999999},"max":{"1min":0.75900000000000001,"5min":0.75900000000000001,"15min":0.876},"last":0.317}]},{"osd":1,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47699999999999998,"15min":0.47199999999999998},"min":{"1min":0.34799999999999998,"5min":0.33800000000000002,"15min":0.32800000000000001},"max":{"1min":0.70099999999999996,"5min":0.70299999999999996,"15min":0.81499999999999995},"last":0.35299999999999998},{"interface":"front","average":{"1min":0.57799999999999996,"5min":0.58199999999999996,"15min":0.57399999999999995},"min":{"1min":0.41899999999999998,"5min":0.40000000000000002,"15min":0.34899999999999998},"max":{"1min":0.76600000000000001,"5min":0.76600000000000001,"15min":0.879},"last":0.39700000000000002}]},{"osd":2,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.59399999999999997,"15min":0.59499999999999997},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.40400000000000003},"max":{"1min":0.749,"5min":0.82899999999999996,"15min":0.88600000000000001},"last":0.622},{"interface":"front","average":{"1min":0.60899999999999999,"5min":0.57299999999999995,"15min":0.56499999999999995},"min":{"1min":0.46200000000000002,"5min":0.34999999999999998,"15min":0.34999999999999998},"max":{"1min":0.79600000000000004,"5min":0.79600000000000004,"15min":0.93000000000000005},"last":0.60899999999999999}]},{"osd":4,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.81200000000000006,"5min":0.82199999999999995,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.56000000000000005,"15min":0.55200000000000005},"max":{"1min":1.2150000000000001,"5min":1.2150000000000001,"15min":1.246},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.84499999999999997,"5min":0.83399999999999996,"15min":0.82399999999999995},"min":{"1min":0.52600000000000002,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":1.3959999999999999,"5min":1.3959999999999999,"15min":1.3959999999999999},"last":0.59799999999999998}]},{"osd":5,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.84399999999999997,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.61299999999999999,"5min":0.496,"15min":0.496},"max":{"1min":1.214,"5min":1.214,"15min":1.353},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.85399999999999998,"5min":0.82799999999999996,"15min":0.81499999999999995},"min":{"1min":0.71899999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.232,"5min":1.232,"15min":1.232},"last":0.65300000000000002}]},{"osd":6,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.83999999999999997,"15min":0.82999999999999996},"min":{"1min":0.57599999999999996,"5min":0.57599999999999996,"15min":0.53200000000000003},"max":{"1min":1.278,"5min":1.278,"15min":1.3799999999999999},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.83199999999999996,"5min":0.84799999999999998,"15min":0.84799999999999998},"min":{"1min":0.61499999999999999,"5min":0.56799999999999995,"15min":0.56799999999999995},"max":{"1min":1.335,"5min":1.335,"15min":1.335},"last":0.498}]},{"osd":7,"last update":"Wed Dec 20 05:42:54 2023","interfaces":[{"interface":"back","average":{"1min":0.88500000000000001,"5min":0.84099999999999997,"15min":0.83399999999999996},"min":{"1min":0.624,"5min":0.52300000000000002,"15min":0.52300000000000002},"max":{"1min":1.383,"5min":1.383,"15min":1.383},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.85999999999999999,"15min":0.86099999999999999},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.56699999999999995},"max":{"1min":1.1160000000000001,"5min":1.2470000000000001,"15min":1.323},"last":0.63400000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575749,"num_pgs":2,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":46908,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324932,"statfs":{"total":96636764160,"available":96588730368,"internally_reserved":0,"allocated":1953792,"data_stored":936143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.89700000000000002},"last":0.68300000000000005}]},{"osd":1,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32700000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.50600000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.65700000000000003}]},{"osd":3,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69099999999999995,"15min":0.68400000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.442},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.71199999999999997}]},{"osd":4,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.83799999999999997,"15min":0.81399999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.50900000000000001},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.48199999999999998},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.93100000000000005}]},{"osd":5,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.83599999999999997,"15min":0.82399999999999995},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.84099999999999997},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.85299999999999998,"15min":0.83499999999999996},"min":{"1min":0.623,"5min":0.58699999999999997,"15min":0.57999999999999996},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.78000000000000003}]},{"osd":6,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85999999999999999,"15min":0.83299999999999996},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.86799999999999999,"15min":0.84199999999999997},"min":{"1min":0.58999999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.90000000000000002}]},{"osd":7,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.876,"15min":0.86399999999999999},"min":{"1min":0.60299999999999998,"5min":0.54800000000000004,"15min":0.51800000000000002},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.96899999999999997},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91600000000000004,"15min":0.89400000000000002},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.94399999999999995}]}]},{"osd":1,"up_from":13,"seq":55834575751,"num_pgs":18,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55100,"kb_used_data":1908,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316740,"statfs":{"total":96636764160,"available":96580341760,"internally_reserved":0,"allocated":1953792,"data_stored":936143,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.44,"5min":0.47999999999999998,"15min":0.46500000000000002},"min":{"1min":0.311,"5min":0.311,"15min":0.29499999999999998},"max":{"1min":0.69699999999999995,"5min":0.754,"15min":0.96499999999999997},"last":0.372},{"interface":"front","average":{"1min":0.51300000000000001,"5min":0.52600000000000002,"15min":0.53100000000000003},"min":{"1min":0.34899999999999998,"5min":0.33600000000000002,"15min":0.32500000000000001},"max":{"1min":0.72599999999999998,"5min":0.76200000000000001,"15min":0.98799999999999999},"last":0.51000000000000001}]},{"osd":2,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.50700000000000001,"5min":0.54000000000000004,"15min":0.53300000000000003},"min":{"1min":0.40100000000000002,"5min":0.35599999999999998,"15min":0.33000000000000002},"max":{"1min":0.83299999999999996,"5min":0.83299999999999996,"15min":0.91000000000000003},"last":0.42099999999999999},{"interface":"front","average":{"1min":0.56599999999999995,"5min":0.58199999999999996,"15min":0.58099999999999996},"min":{"1min":0.44500000000000001,"5min":0.38800000000000001,"15min":0.35399999999999998},"max":{"1min":0.92500000000000004,"5min":0.92500000000000004,"15min":0.93799999999999994},"last":0.39100000000000001}]},{"osd":3,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.61899999999999999,"5min":0.65200000000000002,"15min":0.63300000000000001},"min":{"1min":0.45400000000000001,"5min":0.442,"15min":0.42599999999999999},"max":{"1min":0.77400000000000002,"5min":0.876,"15min":0.94899999999999995},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57499999999999996,"15min":0.55400000000000005},"min":{"1min":0.40200000000000002,"5min":0.38900000000000001,"15min":0.35399999999999998},"max":{"1min":0.83199999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"last":0.46300000000000002}]},{"osd":4,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.80000000000000004,"5min":0.82999999999999996,"15min":0.80500000000000005},"min":{"1min":0.624,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.0329999999999999,"5min":1.3169999999999999,"15min":1.3169999999999999},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.83899999999999997,"5min":0.84899999999999998,"15min":0.82499999999999996},"min":{"1min":0.64600000000000002,"5min":0.57599999999999996,"15min":0.56200000000000006},"max":{"1min":1.097,"5min":1.294,"15min":1.294},"last":0.71799999999999997}]},{"osd":5,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.83899999999999997,"5min":0.84899999999999998,"15min":0.81000000000000005},"min":{"1min":0.66200000000000003,"5min":0.56899999999999995,"15min":0.54000000000000004},"max":{"1min":0.96999999999999997,"5min":1.268,"15min":1.2709999999999999},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.81799999999999995,"5min":0.86399999999999999,"15min":0.82199999999999995},"min":{"1min":0.623,"5min":0.623,"15min":0.54500000000000004},"max":{"1min":1.0609999999999999,"5min":1.274,"15min":1.274},"last":0.77100000000000002}]},{"osd":6,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.86699999999999999,"5min":0.87,"15min":0.84099999999999997},"min":{"1min":0.71499999999999997,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1040000000000001,"5min":1.3420000000000001,"15min":1.3420000000000001},"last":0.84499999999999997},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.84799999999999998,"15min":0.82399999999999995},"min":{"1min":0.63400000000000001,"5min":0.627,"15min":0.51900000000000002},"max":{"1min":1.089,"5min":1.1459999999999999,"15min":1.284},"last":0.86199999999999999}]},{"osd":7,"last update":"Wed Dec 20 05:43:06 2023","interfaces":[{"interface":"back","average":{"1min":0.89100000000000001,"5min":0.89800000000000002,"15min":0.86899999999999999},"min":{"1min":0.67000000000000004,"5min":0.61299999999999999,"15min":0.61299999999999999},"max":{"1min":1.1839999999999999,"5min":1.3620000000000001,"15min":1.3620000000000001},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.89400000000000002,"5min":0.91500000000000004,"15min":0.88100000000000001},"min":{"1min":0.68999999999999995,"5min":0.67500000000000004,"15min":0.55900000000000005},"max":{"1min":1.1519999999999999,"5min":1.4159999999999999,"15min":1.4159999999999999},"last":0.74099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575750,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51572,"kb_used_data":2476,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320268,"statfs":{"total":96636764160,"available":96583954432,"internally_reserved":0,"allocated":2535424,"data_stored":1520208,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.88100000000000001},"last":0.372},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.42699999999999999}]},{"osd":2,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.35799999999999998,"5min":0.35399999999999998,"15min":0.35199999999999998},"max":{"1min":0.78000000000000003,"5min":0.874,"15min":1.3859999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77200000000000002,"5min":0.79600000000000004,"15min":1.369},"last":0.56499999999999995}]},{"osd":3,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71599999999999997,"5min":0.84699999999999998,"15min":1.2709999999999999},"last":0.53500000000000003},{"interface":"front","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":2.016},"last":0.45500000000000002}]},{"osd":4,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.56200000000000006,"5min":0.52800000000000002,"15min":0.47799999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.294},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.63200000000000001,"5min":0.58099999999999996,"15min":0.52100000000000002},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.4530000000000001},"last":0.69699999999999995}]},{"osd":5,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80200000000000005,"15min":0.82199999999999995},"min":{"1min":0.59199999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.177,"5min":1.238,"15min":1.6080000000000001},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81699999999999995,"15min":0.81899999999999995},"min":{"1min":0.60999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.129,"5min":1.22,"15min":1.3200000000000001},"last":0.68200000000000005}]},{"osd":6,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.621,"5min":0.499,"15min":0.499},"max":{"1min":1.256,"5min":1.256,"15min":1.665},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55100000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1439999999999999,"5min":1.167,"15min":1.5660000000000001},"last":0.84699999999999998}]},{"osd":7,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.63200000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.212,"5min":1.222,"15min":1.4670000000000001},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.198,"5min":1.2649999999999999,"15min":1.415},"last":0.83399999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":160,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":18,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2202,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":19,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T05:43:51.028 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T05:43:51.028 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T05:43:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.216+0000 7f70dcf98640 1 Processor -- start 2023-12-20T05:43:51.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.216+0000 7f70dcf98640 1 -- start start 2023-12-20T05:43:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.216+0000 7f70dcf98640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f70d815a940 0x7f70d80cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.216+0000 7f70dcf98640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70d815ad20 0x7f70d80ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.216+0000 7f70dcf98640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f70d80cc9d0 con 0x7f70d80cb1c0 2023-12-20T05:43:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.216+0000 7f70dcf98640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f70d80c9b90 con 0x7f70d815ad20 2023-12-20T05:43:51.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.216+0000 7f70dcf98640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d80c9e70 con 0x7f70d815a940 2023-12-20T05:43:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.216+0000 7f70cffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70d815ad20 0x7f70d80ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.216+0000 7f70cffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f70d815ad20 0x7f70d80ccee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33662/0 (socket says 172.21.15.32:33662) 2023-12-20T05:43:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.216+0000 7f70cffff640 1 -- 172.21.15.32:0/2113791734 learned_addr learned my addr 172.21.15.32:0/2113791734 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.220+0000 7f70d659c640 1 --2- 172.21.15.32:0/2113791734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f70d815a940 0x7f70d80cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.220+0000 7f70cffff640 1 -- 172.21.15.32:0/2113791734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f70d815a940 msgr2=0x7f70d80cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.220+0000 7f70cffff640 1 --2- 172.21.15.32:0/2113791734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f70d815a940 0x7f70d80cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.220+0000 7f70cffff640 1 -- 172.21.15.32:0/2113791734 >> v1:172.21.15.39:6789/0 conn(0x7f70d80cb1c0 legacy=0x7f70d8070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.220+0000 7f70cffff640 1 -- 172.21.15.32:0/2113791734 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70d80ca150 con 0x7f70d815ad20 2023-12-20T05:43:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.220+0000 7f70cffff640 1 --2- 172.21.15.32:0/2113791734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d815ad20 0x7f70d80ccee0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009870 tx=0x7f70c001cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=490664d36173f97 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.220+0000 7f70d5d9b640 1 -- 172.21.15.32:0/2113791734 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00a95f0 con 0x7f70d815ad20 2023-12-20T05:43:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.220+0000 7f70d5d9b640 1 -- 172.21.15.32:0/2113791734 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c00a9750 con 0x7f70d815ad20 2023-12-20T05:43:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.220+0000 7f70d5d9b640 1 -- 172.21.15.32:0/2113791734 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00a9ae0 con 0x7f70d815ad20 2023-12-20T05:43:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.220+0000 7f70dcf98640 1 -- 172.21.15.32:0/2113791734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d815ad20 msgr2=0x7f70d80ccee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.220+0000 7f70dcf98640 1 --2- 172.21.15.32:0/2113791734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d815ad20 0x7f70d80ccee0 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7f70c0009870 tx=0x7f70c001cf70 comp rx=0 tx=0).stop 2023-12-20T05:43:51.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 -- 172.21.15.32:0/2113791734 shutdown_connections 2023-12-20T05:43:51.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 --2- 172.21.15.32:0/2113791734 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d815ad20 0x7f70d80ccee0 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 --2- 172.21.15.32:0/2113791734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f70d815a940 0x7f70d80cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 -- 172.21.15.32:0/2113791734 >> 172.21.15.32:0/2113791734 conn(0x7f70d806a4b0 msgr2=0x7f70d80b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:51.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 -- 172.21.15.32:0/2113791734 shutdown_connections 2023-12-20T05:43:51.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 -- 172.21.15.32:0/2113791734 wait complete. 2023-12-20T05:43:51.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 Processor -- start 2023-12-20T05:43:51.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 -- start start 2023-12-20T05:43:51.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f70d80cb1c0 0x7f70d81550e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d815a940 0x7f70d8141f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70d815ad20 0x7f70d80c6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d8154cc0 con 0x7f70d815a940 2023-12-20T05:43:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f70d8154040 con 0x7f70d815ad20 2023-12-20T05:43:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70dcf98640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f70d8154320 con 0x7f70d80cb1c0 2023-12-20T05:43:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d815a940 0x7f70d8141f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d815a940 0x7f70d8141f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33664/0 (socket says 172.21.15.32:33664) 2023-12-20T05:43:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70cffff640 1 -- 172.21.15.32:0/4151151868 learned_addr learned my addr 172.21.15.32:0/4151151868 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70d659c640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f70d80cb1c0 0x7f70d81550e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70d6d9d640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70d815ad20 0x7f70d80c6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70cffff640 1 -- 172.21.15.32:0/4151151868 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f70d80cb1c0 msgr2=0x7f70d81550e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70cffff640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f70d80cb1c0 0x7f70d81550e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70cffff640 1 -- 172.21.15.32:0/4151151868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70d815ad20 msgr2=0x7f70d80c6070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70cffff640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70d815ad20 0x7f70d80c6070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70cffff640 1 -- 172.21.15.32:0/4151151868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f70c00094d0 con 0x7f70d815a940 2023-12-20T05:43:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70d6d9d640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70d815ad20 0x7f70d80c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.224+0000 7f70cffff640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d815a940 0x7f70d8141f90 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f70c001abc0 tx=0x7f70c00192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.228+0000 7f70ceffd640 1 -- 172.21.15.32:0/4151151868 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00a95f0 con 0x7f70d815a940 2023-12-20T05:43:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.228+0000 7f70ceffd640 1 -- 172.21.15.32:0/4151151868 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f70c00b1800 con 0x7f70d815a940 2023-12-20T05:43:51.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.228+0000 7f70ceffd640 1 -- 172.21.15.32:0/4151151868 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f70c00b1b90 con 0x7f70d815a940 2023-12-20T05:43:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.228+0000 7f70dcf98640 1 -- 172.21.15.32:0/4151151868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f70d80cbb60 con 0x7f70d815a940 2023-12-20T05:43:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.228+0000 7f70dcf98640 1 -- 172.21.15.32:0/4151151868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f70d81548e0 con 0x7f70d815a940 2023-12-20T05:43:51.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.228+0000 7f70dcf98640 1 -- 172.21.15.32:0/4151151868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f70d8154e00 con 0x7f70d815a940 2023-12-20T05:43:51.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.228+0000 7f70ceffd640 1 -- 172.21.15.32:0/4151151868 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f70c00b1cf0 con 0x7f70d815a940 2023-12-20T05:43:51.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.228+0000 7f70ceffd640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70b8065b80 0x7f70b8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.228+0000 7f70ceffd640 1 -- 172.21.15.32:0/4151151868 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f70c011d610 con 0x7f70d815a940 2023-12-20T05:43:51.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.228+0000 7f70d659c640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70b8065b80 0x7f70b8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.232+0000 7f70d659c640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70b8065b80 0x7f70b8068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f70c800b9d0 tx=0x7f70c8008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:51.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.232+0000 7f70ceffd640 1 -- 172.21.15.32:0/4151151868 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f70d8154e00 con 0x7f70d815a940 2023-12-20T05:43:51.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.444+0000 7f70dcf98640 1 -- 172.21.15.32:0/4151151868 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f70d8154e00 con 0x7f70d815a940 2023-12-20T05:43:51.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70ceffd640 1 -- 172.21.15.32:0/4151151868 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v42) v1 ==== 190+0+0 (secure 0 0 0) 0x7f70d8154e00 con 0x7f70d815a940 2023-12-20T05:43:51.453 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T05:43:51.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 -- 172.21.15.32:0/4151151868 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70b8065b80 msgr2=0x7f70b8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70b8065b80 0x7f70b8068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f70c800b9d0 tx=0x7f70c8008040 comp rx=0 tx=0).stop 2023-12-20T05:43:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 -- 172.21.15.32:0/4151151868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d815a940 msgr2=0x7f70d8141f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d815a940 0x7f70d8141f90 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7f70c001abc0 tx=0x7f70c00192b0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 -- 172.21.15.32:0/4151151868 shutdown_connections 2023-12-20T05:43:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f70b8065b80 0x7f70b8068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f70d815ad20 0x7f70d80c6070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f70d815a940 0x7f70d8141f90 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 --2- 172.21.15.32:0/4151151868 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f70d80cb1c0 0x7f70d81550e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 -- 172.21.15.32:0/4151151868 >> 172.21.15.32:0/4151151868 conn(0x7f70d806a4b0 msgr2=0x7f70d80c36c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 -- 172.21.15.32:0/4151151868 shutdown_connections 2023-12-20T05:43:51.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.452+0000 7f70dcf98640 1 -- 172.21.15.32:0/4151151868 wait complete. 2023-12-20T05:43:51.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:43:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59c54df640 1 Processor -- start 2023-12-20T05:43:51.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59c54df640 1 -- start start 2023-12-20T05:43:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59c54df640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f59c00d2740 0x7f59c00cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59c54df640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59c00cfab0 0x7f59c00b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59c54df640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f59c00cff00 con 0x7f59c015cdb0 2023-12-20T05:43:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59c54df640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f59c015d190 con 0x7f59c00d2740 2023-12-20T05:43:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59c54df640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c00d16a0 con 0x7f59c00cfab0 2023-12-20T05:43:51.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59beffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f59c015cdb0 0x7f59c00d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43036/0 (socket says 172.21.15.32:43036) 2023-12-20T05:43:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59beffd640 1 -- 172.21.15.32:0/3052000807 learned_addr learned my addr 172.21.15.32:0/3052000807 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59b7fff640 1 --2- 172.21.15.32:0/3052000807 >> v2:172.21.15.39:3300/0 conn(0x7f59c00d2740 0x7f59c00cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59bf7fe640 1 --2- 172.21.15.32:0/3052000807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59c00cfab0 0x7f59c00b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59be7fc640 1 -- 172.21.15.32:0/3052000807 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2035653420 0 0) 0x7f59b0001120 con 0x7f59c015cdb0 2023-12-20T05:43:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59be7fc640 1 -- 172.21.15.32:0/3052000807 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f59ac003530 con 0x7f59c015cdb0 2023-12-20T05:43:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59b7fff640 1 -- 172.21.15.32:0/3052000807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59c00cfab0 msgr2=0x7f59c00b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59b7fff640 1 --2- 172.21.15.32:0/3052000807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59c00cfab0 0x7f59c00b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59b7fff640 1 -- 172.21.15.32:0/3052000807 >> v1:172.21.15.32:6789/0 conn(0x7f59c015cdb0 legacy=0x7f59c00d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59be7fc640 1 -- 172.21.15.32:0/3052000807 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1872244283 0 0) 0x7f59ac003530 con 0x7f59c015cdb0 2023-12-20T05:43:51.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.620+0000 7f59b7fff640 1 -- 172.21.15.32:0/3052000807 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59c00d1980 con 0x7f59c00d2740 2023-12-20T05:43:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.624+0000 7f59b7fff640 1 --2- 172.21.15.32:0/3052000807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c00d2740 0x7f59c00cf570 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f59a80097c0 tx=0x7f59a801eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a52288fde818085a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.624+0000 7f59be7fc640 1 -- 172.21.15.32:0/3052000807 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b0001810 con 0x7f59c00d2740 2023-12-20T05:43:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.624+0000 7f59be7fc640 1 -- 172.21.15.32:0/3052000807 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59a80a85d0 con 0x7f59c00d2740 2023-12-20T05:43:51.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.624+0000 7f59be7fc640 1 -- 172.21.15.32:0/3052000807 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59a80a8960 con 0x7f59c00d2740 2023-12-20T05:43:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.624+0000 7f59c54df640 1 -- 172.21.15.32:0/3052000807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c00d2740 msgr2=0x7f59c00cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.624+0000 7f59c54df640 1 --2- 172.21.15.32:0/3052000807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c00d2740 0x7f59c00cf570 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f59a80097c0 tx=0x7f59a801eb90 comp rx=0 tx=0).stop 2023-12-20T05:43:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.624+0000 7f59c54df640 1 -- 172.21.15.32:0/3052000807 shutdown_connections 2023-12-20T05:43:51.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.624+0000 7f59c54df640 1 --2- 172.21.15.32:0/3052000807 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c00d2740 0x7f59c00cf570 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.624+0000 7f59c54df640 1 --2- 172.21.15.32:0/3052000807 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59c00cfab0 0x7f59c00b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.624+0000 7f59c54df640 1 -- 172.21.15.32:0/3052000807 >> 172.21.15.32:0/3052000807 conn(0x7f59c0065ee0 msgr2=0x7f59c006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59c54df640 1 -- 172.21.15.32:0/3052000807 shutdown_connections 2023-12-20T05:43:51.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59c54df640 1 -- 172.21.15.32:0/3052000807 wait complete. 2023-12-20T05:43:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59c54df640 1 Processor -- start 2023-12-20T05:43:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59c54df640 1 -- start start 2023-12-20T05:43:51.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59c54df640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59c00cfab0 0x7f59c00c79e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59c54df640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c00d2740 0x7f59c014e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59c54df640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c015cdb0 0x7f59c01505f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59c54df640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0151470 con 0x7f59c015cdb0 2023-12-20T05:43:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59c54df640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0151750 con 0x7f59c00d2740 2023-12-20T05:43:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59c54df640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f59c0151a30 con 0x7f59c00cfab0 2023-12-20T05:43:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c015cdb0 0x7f59c01505f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c015cdb0 0x7f59c01505f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33680/0 (socket says 172.21.15.32:33680) 2023-12-20T05:43:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59bf7fe640 1 -- 172.21.15.32:0/193950275 learned_addr learned my addr 172.21.15.32:0/193950275 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:51.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59beffd640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59c00cfab0 0x7f59c00c79e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59b7fff640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c00d2740 0x7f59c014e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59bf7fe640 1 -- 172.21.15.32:0/193950275 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59c00cfab0 msgr2=0x7f59c00c79e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59bf7fe640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59c00cfab0 0x7f59c00c79e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59bf7fe640 1 -- 172.21.15.32:0/193950275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c00d2740 msgr2=0x7f59c014e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59bf7fe640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c00d2740 0x7f59c014e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59bf7fe640 1 -- 172.21.15.32:0/193950275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f59a8009420 con 0x7f59c015cdb0 2023-12-20T05:43:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59b7fff640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c00d2740 0x7f59c014e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59bf7fe640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c015cdb0 0x7f59c01505f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f59b800efc0 tx=0x7f59b800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:51.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.628+0000 7f59b77fe640 1 -- 172.21.15.32:0/193950275 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b8009ba0 con 0x7f59c015cdb0 2023-12-20T05:43:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.632+0000 7f59b77fe640 1 -- 172.21.15.32:0/193950275 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f59b800f040 con 0x7f59c015cdb0 2023-12-20T05:43:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.632+0000 7f59b77fe640 1 -- 172.21.15.32:0/193950275 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f59b80146c0 con 0x7f59c015cdb0 2023-12-20T05:43:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.632+0000 7f59c54df640 1 -- 172.21.15.32:0/193950275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f59c00c7f20 con 0x7f59c015cdb0 2023-12-20T05:43:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.632+0000 7f59c54df640 1 -- 172.21.15.32:0/193950275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f59c014f000 con 0x7f59c015cdb0 2023-12-20T05:43:51.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.632+0000 7f59c54df640 1 -- 172.21.15.32:0/193950275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f59c004e000 con 0x7f59c015cdb0 2023-12-20T05:43:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.632+0000 7f59b77fe640 1 -- 172.21.15.32:0/193950275 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f59b8014820 con 0x7f59c015cdb0 2023-12-20T05:43:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.632+0000 7f59b77fe640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f59a0065b80 0x7f59a0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.632+0000 7f59b77fe640 1 -- 172.21.15.32:0/193950275 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f59b8083f50 con 0x7f59c015cdb0 2023-12-20T05:43:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.632+0000 7f59beffd640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f59a0065b80 0x7f59a0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:51.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.636+0000 7f59beffd640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f59a0065b80 0x7f59a0068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59b0001060 tx=0x7f59b0009040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:51.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.636+0000 7f59b77fe640 1 -- 172.21.15.32:0/193950275 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f59b80516c0 con 0x7f59c015cdb0 2023-12-20T05:43:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.828+0000 7f59c54df640 1 -- 172.21.15.32:0/193950275 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f59c00d1c30 con 0x7f59c015cdb0 2023-12-20T05:43:51.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.828+0000 7f59b77fe640 1 -- 172.21.15.32:0/193950275 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f59b8054ce0 con 0x7f59c015cdb0 2023-12-20T05:43:51.830 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:43:51.830 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2062259714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2062259714},{"type":"v1","addr":"172.21.15.39:6834","nonce":2062259714}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4210073096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":4210073096},{"type":"v1","addr":"172.21.15.39:6835","nonce":4210073096}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8913,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3547769022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3547769022},{"type":"v1","addr":"172.21.15.32:6835","nonce":3547769022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:43:46.505246+0000","modified":"2023-12-20T05:43:49.528294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8928":{"gid":8928,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3239941411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3239941411},{"type":"v1","addr":"172.21.15.32:6837","nonce":3239941411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-20T05:43:51.831 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 71 2023-12-20T05:43:51.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 -- 172.21.15.32:0/193950275 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f59a0065b80 msgr2=0x7f59a0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f59a0065b80 0x7f59a0068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f59b0001060 tx=0x7f59b0009040 comp rx=0 tx=0).stop 2023-12-20T05:43:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 -- 172.21.15.32:0/193950275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c015cdb0 msgr2=0x7f59c01505f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c015cdb0 0x7f59c01505f0 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7f59b800efc0 tx=0x7f59b800c490 comp rx=0 tx=0).stop 2023-12-20T05:43:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 -- 172.21.15.32:0/193950275 shutdown_connections 2023-12-20T05:43:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f59a0065b80 0x7f59a0068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f59c015cdb0 0x7f59c01505f0 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f59c00d2740 0x7f59c014e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 --2- 172.21.15.32:0/193950275 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f59c00cfab0 0x7f59c00c79e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 -- 172.21.15.32:0/193950275 >> 172.21.15.32:0/193950275 conn(0x7f59c0065ee0 msgr2=0x7f59c015b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 -- 172.21.15.32:0/193950275 shutdown_connections 2023-12-20T05:43:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:51.832+0000 7f59c54df640 1 -- 172.21.15.32:0/193950275 wait complete. 2023-12-20T05:43:52.239 INFO:tasks.ceph.mds.a.smithi032.stderr:2023-12-20T05:43:52.240+0000 7f854796d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T05:43:52.254 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 71, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T05:43:46.505246+0000', 'modified': '2023-12-20T05:43:49.528294+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 8928}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_8928': {'gid': 8928, 'name': 'a', 'rank': 0, 'incarnation': 67, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/3239941411', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 3239941411}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 3239941411}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [19], 'metadata_pool': 18, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T05:43:52.255 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T05:43:52.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-20T05:43:52.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.400+0000 7f49d12c4640 1 Processor -- start 2023-12-20T05:43:52.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.400+0000 7f49d12c4640 1 -- start start 2023-12-20T05:43:52.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49d12c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49cc15ce30 0x7f49cc161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49d12c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49cc071920 con 0x7f49cc070950 2023-12-20T05:43:52.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49d12c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49cc071080 con 0x7f49cc06deb0 2023-12-20T05:43:52.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49d12c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc071360 con 0x7f49cc15ce30 2023-12-20T05:43:52.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49c3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f49cc070950 0x7f49cc06d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43050/0 (socket says 172.21.15.32:43050) 2023-12-20T05:43:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49c3fff640 1 -- 172.21.15.32:0/1259438804 learned_addr learned my addr 172.21.15.32:0/1259438804 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49caffd640 1 --2- 172.21.15.32:0/1259438804 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49cc15ce30 0x7f49cc161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49ca7fc640 1 -- 172.21.15.32:0/1259438804 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1412714184 0 0) 0x7f49cc071920 con 0x7f49cc070950 2023-12-20T05:43:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49ca7fc640 1 -- 172.21.15.32:0/1259438804 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49b8003530 con 0x7f49cc070950 2023-12-20T05:43:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49ca7fc640 1 -- 172.21.15.32:0/1259438804 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621267353 0 0) 0x7f49cc071080 con 0x7f49cc06deb0 2023-12-20T05:43:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49ca7fc640 1 -- 172.21.15.32:0/1259438804 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f49cc071920 con 0x7f49cc06deb0 2023-12-20T05:43:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49ca7fc640 1 -- 172.21.15.32:0/1259438804 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1091250858 0 0) 0x7f49b8003530 con 0x7f49cc070950 2023-12-20T05:43:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49caffd640 1 -- 172.21.15.32:0/1259438804 >> v1:172.21.15.39:6789/0 conn(0x7f49cc06deb0 legacy=0x7f49cc0cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49caffd640 1 -- 172.21.15.32:0/1259438804 >> v1:172.21.15.32:6789/0 conn(0x7f49cc070950 legacy=0x7f49cc06d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.404+0000 7f49caffd640 1 -- 172.21.15.32:0/1259438804 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49cc071640 con 0x7f49cc15ce30 2023-12-20T05:43:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49caffd640 1 --2- 172.21.15.32:0/1259438804 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49cc15ce30 0x7f49cc161560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0097f0 tx=0x7f49bc01c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=11020e4cf7e9dc5b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49ca7fc640 1 -- 172.21.15.32:0/1259438804 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4001680 con 0x7f49cc15ce30 2023-12-20T05:43:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49ca7fc640 1 -- 172.21.15.32:0/1259438804 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc0a7050 con 0x7f49cc15ce30 2023-12-20T05:43:52.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49ca7fc640 1 -- 172.21.15.32:0/1259438804 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0ab620 con 0x7f49cc15ce30 2023-12-20T05:43:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49d12c4640 1 -- 172.21.15.32:0/1259438804 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49cc15ce30 msgr2=0x7f49cc161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49d12c4640 1 --2- 172.21.15.32:0/1259438804 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49cc15ce30 0x7f49cc161560 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0097f0 tx=0x7f49bc01c350 comp rx=0 tx=0).stop 2023-12-20T05:43:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49d12c4640 1 -- 172.21.15.32:0/1259438804 shutdown_connections 2023-12-20T05:43:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49d12c4640 1 --2- 172.21.15.32:0/1259438804 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49cc15ce30 0x7f49cc161560 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49d12c4640 1 -- 172.21.15.32:0/1259438804 >> 172.21.15.32:0/1259438804 conn(0x7f49cc0cba50 msgr2=0x7f49cc0cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49d12c4640 1 -- 172.21.15.32:0/1259438804 shutdown_connections 2023-12-20T05:43:52.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49d12c4640 1 -- 172.21.15.32:0/1259438804 wait complete. 2023-12-20T05:43:52.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49d12c4640 1 Processor -- start 2023-12-20T05:43:52.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.408+0000 7f49d12c4640 1 -- start start 2023-12-20T05:43:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49d12c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49cc06deb0 0x7f49cc153f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49d12c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc070950 0x7f49cc157b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49d12c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49cc15ce30 0x7f49cc1580b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49d12c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc09fb10 con 0x7f49cc070950 2023-12-20T05:43:52.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49d12c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc09f3a0 con 0x7f49cc06deb0 2023-12-20T05:43:52.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49d12c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49cc09fd40 con 0x7f49cc15ce30 2023-12-20T05:43:52.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc070950 0x7f49cc157b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc070950 0x7f49cc157b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33694/0 (socket says 172.21.15.32:33694) 2023-12-20T05:43:52.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c3fff640 1 -- 172.21.15.32:0/3276505968 learned_addr learned my addr 172.21.15.32:0/3276505968 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:52.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49caffd640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49cc06deb0 0x7f49cc153f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49cb7fe640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49cc15ce30 0x7f49cc1580b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c3fff640 1 -- 172.21.15.32:0/3276505968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49cc15ce30 msgr2=0x7f49cc1580b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c3fff640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49cc15ce30 0x7f49cc1580b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c3fff640 1 -- 172.21.15.32:0/3276505968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49cc06deb0 msgr2=0x7f49cc153f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c3fff640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49cc06deb0 0x7f49cc153f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c3fff640 1 -- 172.21.15.32:0/3276505968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49bc009420 con 0x7f49cc070950 2023-12-20T05:43:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c3fff640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc070950 0x7f49cc157b70 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f49b400c780 tx=0x7f49b400cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c37fe640 1 -- 172.21.15.32:0/3276505968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4005cf0 con 0x7f49cc070950 2023-12-20T05:43:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c37fe640 1 -- 172.21.15.32:0/3276505968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b4005e70 con 0x7f49cc070950 2023-12-20T05:43:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49d12c4640 1 -- 172.21.15.32:0/3276505968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f49cc09ea70 con 0x7f49cc070950 2023-12-20T05:43:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49c37fe640 1 -- 172.21.15.32:0/3276505968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4007290 con 0x7f49cc070950 2023-12-20T05:43:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.412+0000 7f49d12c4640 1 -- 172.21.15.32:0/3276505968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc09f030 con 0x7f49cc070950 2023-12-20T05:43:52.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49c37fe640 1 -- 172.21.15.32:0/3276505968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f49b40073f0 con 0x7f49cc070950 2023-12-20T05:43:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49c37fe640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49ac065b80 0x7f49ac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49c37fe640 1 -- 172.21.15.32:0/3276505968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f49b4082ad0 con 0x7f49cc070950 2023-12-20T05:43:52.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49caffd640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49ac065b80 0x7f49ac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49d461f000 1 Processor -- start 2023-12-20T05:43:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49d461f000 1 -- start start 2023-12-20T05:43:52.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49caffd640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49ac065b80 0x7f49ac068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0ad000 tx=0x7f49bc0a6000 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:52.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49d461f000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d59f880 0x55b09d59fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49d461f000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d54cc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49d461f000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d54d5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49d461f000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09d370c90 con 0x55b09d54d1c0 2023-12-20T05:43:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49d461f000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09d3a2220 con 0x55b09d59f880 2023-12-20T05:43:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49d461f000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b09d39f7f0 con 0x55b09d5a01a0 2023-12-20T05:43:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d54d5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d54d5a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33702/0 (socket says 172.21.15.32:33702) 2023-12-20T05:43:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49cb7fe640 1 -- 172.21.15.32:0/35210520 learned_addr learned my addr 172.21.15.32:0/35210520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49caffd640 1 --2- 172.21.15.32:0/35210520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d59f880 0x55b09d59fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49c3fff640 1 --2- 172.21.15.32:0/35210520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d54cc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49cb7fe640 1 -- 172.21.15.32:0/35210520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 msgr2=0x55b09d54cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49cb7fe640 1 --2- 172.21.15.32:0/35210520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d54cc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49cb7fe640 1 -- 172.21.15.32:0/35210520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d59f880 msgr2=0x55b09d59fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49cb7fe640 1 --2- 172.21.15.32:0/35210520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d59f880 0x55b09d59fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49cb7fe640 1 -- 172.21.15.32:0/35210520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b09d097890 con 0x55b09d54d1c0 2023-12-20T05:43:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.416+0000 7f49cb7fe640 1 --2- 172.21.15.32:0/35210520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d54d5a0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f49c4001dd0 tx=0x7f49c40016e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae5c0c6400f580f2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c0ff9640 1 -- 172.21.15.32:0/35210520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c400fc40 con 0x55b09d54d1c0 2023-12-20T05:43:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c0ff9640 1 -- 172.21.15.32:0/35210520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49c400fda0 con 0x55b09d54d1c0 2023-12-20T05:43:52.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c0ff9640 1 -- 172.21.15.32:0/35210520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49c4003bc0 con 0x55b09d54d1c0 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 -- 172.21.15.32:0/35210520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 msgr2=0x55b09d54d5a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 --2- 172.21.15.32:0/35210520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d54d5a0 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7f49c4001dd0 tx=0x7f49c40016e0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 -- 172.21.15.32:0/35210520 shutdown_connections 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 --2- 172.21.15.32:0/35210520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d54d5a0 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 --2- 172.21.15.32:0/35210520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d54cc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 --2- 172.21.15.32:0/35210520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d59f880 0x55b09d59fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 -- 172.21.15.32:0/35210520 >> 172.21.15.32:0/35210520 conn(0x55b09d553950 msgr2=0x55b09d54b560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 -- 172.21.15.32:0/35210520 shutdown_connections 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 -- 172.21.15.32:0/35210520 wait complete. 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 -- start start 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 Processor -- start 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d54d1c0 0x55b09d552740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d59f880 0x55b09d561890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d561dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09d39b0e0 con 0x55b09d59f880 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09d33a310 con 0x55b09d54d1c0 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b09d358770 con 0x55b09d5a01a0 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49d461f000 10 client.0 _ll_register_callbacks cb 0x55b09d538b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d59f880 0x55b09d561890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d59f880 0x55b09d561890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33712/0 (socket says 172.21.15.32:33712) 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c3fff640 1 -- 172.21.15.32:0/1665637135 learned_addr learned my addr 172.21.15.32:0/1665637135 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49caffd640 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d54d1c0 0x55b09d552740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49cb7fe640 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d561dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c3fff640 1 -- 172.21.15.32:0/1665637135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 msgr2=0x55b09d561dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c3fff640 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d561dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c3fff640 1 -- 172.21.15.32:0/1665637135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d54d1c0 msgr2=0x55b09d552740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:52.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c3fff640 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d54d1c0 0x55b09d552740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c3fff640 1 -- 172.21.15.32:0/1665637135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc071920 con 0x55b09d59f880 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49c3fff640 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d59f880 0x55b09d561890 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f49b400c750 tx=0x7f49b4054170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8b6ba3aa62174d7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.420+0000 7f49a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1665637135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49b4054490 con 0x55b09d59f880 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1665637135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f49b4052490 con 0x55b09d59f880 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 -- 172.21.15.32:0/1665637135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b09d36ac10 con 0x55b09d59f880 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1665637135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49b4053520 con 0x55b09d59f880 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 10 client.9057 fetch_fsmap learned FSMap version 71 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 -- 172.21.15.32:0/1665637135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55b09d360260 con 0x55b09d59f880 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1665637135 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49b406f070 con 0x55b09d59f880 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1665637135 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f49b405bad0 con 0x55b09d59f880 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 10 client.9057 fetch_fsmap finished waiting for FSMap version 71 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 10 client.9057 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 20 client.9057 populate_metadata read hostname 'smithi032' 2023-12-20T05:43:52.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.32:6836/3239941411,v1:172.21.15.32:6837/3239941411] conn(0x55b09d5a70e0 0x55b09d5a74c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 4 client.9057 mds_command: new command op to 8928 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T05:43:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 -- 172.21.15.32:0/1665637135 --> [v2:172.21.15.32:6836/3239941411,v1:172.21.15.32:6837/3239941411] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55b09d399a30 con 0x55b09d5a70e0 2023-12-20T05:43:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49caffd640 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.32:6836/3239941411,v1:172.21.15.32:6837/3239941411] conn(0x55b09d5a70e0 0x55b09d5a74c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49caffd640 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.32:6836/3239941411,v1:172.21.15.32:6837/3239941411] conn(0x55b09d5a70e0 0x55b09d5a74c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8654c19833bc5cb server_cookie=3271be60c94d6215 in_seq=0 out_seq=0 2023-12-20T05:43:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49a9ffb640 10 client.9057 ms_handle_connect on v2:172.21.15.32:6836/3239941411 2023-12-20T05:43:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1665637135 <== mds.0 v2:172.21.15.32:6836/3239941411 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55b09d399a30 con 0x55b09d5a70e0 2023-12-20T05:43:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49a9ffb640 10 client.9057 handle_command_reply: tid=0 2023-12-20T05:43:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 client.9057 shutdown 2023-12-20T05:43:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 -- 172.21.15.32:0/1665637135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d59f880 msgr2=0x55b09d561890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d59f880 0x55b09d561890 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7f49b400c750 tx=0x7f49b4054170 comp rx=0 tx=0).stop 2023-12-20T05:43:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 -- 172.21.15.32:0/1665637135 shutdown_connections 2023-12-20T05:43:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.32:6836/3239941411,v1:172.21.15.32:6837/3239941411] conn(0x55b09d5a70e0 0x55b09d5a74c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d561dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d59f880 0x55b09d561890 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 --2- 172.21.15.32:0/1665637135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d54d1c0 0x55b09d552740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 -- 172.21.15.32:0/1665637135 >> 172.21.15.32:0/1665637135 conn(0x55b09d553950 msgr2=0x55b09d54bc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49a9ffb640 0 client.9057 ms_handle_reset on v2:172.21.15.32:6836/3239941411 2023-12-20T05:43:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 -- 172.21.15.32:0/1665637135 shutdown_connections 2023-12-20T05:43:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 1 -- 172.21.15.32:0/1665637135 wait complete. 2023-12-20T05:43:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.424+0000 7f49d461f000 20 client.9057 trim_cache size 0 max 16384 2023-12-20T05:43:52.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49d461f000 1 Processor -- start 2023-12-20T05:43:52.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49d461f000 1 -- start start 2023-12-20T05:43:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49d461f000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5f76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49d461f000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d59f880 0x55b09d5f7c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49d461f000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d5f8160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49d461f000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09d058940 con 0x55b09d54d1c0 2023-12-20T05:43:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49d461f000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09d54fd70 con 0x55b09d59f880 2023-12-20T05:43:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49d461f000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b09d3984e0 con 0x55b09d5a01a0 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5f76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5f76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33722/0 (socket says 172.21.15.32:33722) 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49caffd640 1 -- 172.21.15.32:0/3211849208 learned_addr learned my addr 172.21.15.32:0/3211849208 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49c3fff640 1 --2- 172.21.15.32:0/3211849208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d59f880 0x55b09d5f7c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49cb7fe640 1 --2- 172.21.15.32:0/3211849208 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d5f8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49caffd640 1 -- 172.21.15.32:0/3211849208 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 msgr2=0x55b09d5f8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49caffd640 1 --2- 172.21.15.32:0/3211849208 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d5f8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49caffd640 1 -- 172.21.15.32:0/3211849208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d59f880 msgr2=0x55b09d5f7c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49caffd640 1 --2- 172.21.15.32:0/3211849208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d59f880 0x55b09d5f7c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49caffd640 1 -- 172.21.15.32:0/3211849208 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55b09d3463b0 con 0x55b09d54d1c0 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49cb7fe640 1 --2- 172.21.15.32:0/3211849208 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d5f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49caffd640 1 --2- 172.21.15.32:0/3211849208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5f76e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0b9040 tx=0x7f49bc004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f984c33e904d3da5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:52.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49c17fa640 1 -- 172.21.15.32:0/3211849208 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0c7040 con 0x55b09d54d1c0 2023-12-20T05:43:52.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49c17fa640 1 -- 172.21.15.32:0/3211849208 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49bc0c9040 con 0x55b09d54d1c0 2023-12-20T05:43:52.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.440+0000 7f49c17fa640 1 -- 172.21.15.32:0/3211849208 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0cee10 con 0x55b09d54d1c0 2023-12-20T05:43:52.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- 172.21.15.32:0/3211849208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 msgr2=0x55b09d5f76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 --2- 172.21.15.32:0/3211849208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5f76e0 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0b9040 tx=0x7f49bc004740 comp rx=0 tx=0).stop 2023-12-20T05:43:52.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- 172.21.15.32:0/3211849208 shutdown_connections 2023-12-20T05:43:52.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 --2- 172.21.15.32:0/3211849208 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d5a01a0 0x55b09d5f8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 --2- 172.21.15.32:0/3211849208 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d59f880 0x55b09d5f7c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 --2- 172.21.15.32:0/3211849208 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5f76e0 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- 172.21.15.32:0/3211849208 >> 172.21.15.32:0/3211849208 conn(0x55b09d553950 msgr2=0x55b09d561750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:52.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- 172.21.15.32:0/3211849208 shutdown_connections 2023-12-20T05:43:52.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- 172.21.15.32:0/3211849208 wait complete. 2023-12-20T05:43:52.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- start start 2023-12-20T05:43:52.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 Processor -- start 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5e0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d59f880 0x55b09d5e0a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d5a01a0 0x55b09d5e0fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09d396a50 con 0x55b09d54d1c0 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55b09d395660 con 0x55b09d5a01a0 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55b09d397cf0 con 0x55b09d59f880 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 10 client.0 _ll_register_callbacks cb 0x55b09d349640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5e0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5e0540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33734/0 (socket says 172.21.15.32:33734) 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49caffd640 1 -- 172.21.15.32:0/1305368803 learned_addr learned my addr 172.21.15.32:0/1305368803 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49c3fff640 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d59f880 0x55b09d5e0a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49cb7fe640 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d5a01a0 0x55b09d5e0fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49caffd640 1 -- 172.21.15.32:0/1305368803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d59f880 msgr2=0x55b09d5e0a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49caffd640 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d59f880 0x55b09d5e0a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49caffd640 1 -- 172.21.15.32:0/1305368803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d5a01a0 msgr2=0x55b09d5e0fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:52.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49caffd640 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d5a01a0 0x55b09d5e0fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49caffd640 1 -- 172.21.15.32:0/1305368803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49cc071640 con 0x55b09d54d1c0 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49caffd640 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5e0540 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0b9040 tx=0x7f49bc01ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=783adc88dfdf9e2c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1305368803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49bc0c7040 con 0x55b09d54d1c0 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1305368803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f49bc0cee40 con 0x55b09d54d1c0 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- 172.21.15.32:0/1305368803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55b09d360e00 con 0x55b09d54d1c0 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1305368803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=71) v2 ==== 24+0+0 (secure 0 0 0) 0x7f49bc0ab740 con 0x55b09d54d1c0 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 10 client.9063 fetch_fsmap learned FSMap version 71 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- 172.21.15.32:0/1305368803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=71,monmap=2+}) v3 -- 0x55b09d097cc0 con 0x55b09d54d1c0 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1305368803 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55b09d360e00 con 0x55b09d54d1c0 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1305368803 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 71) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f49bc0de020 con 0x55b09d54d1c0 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 10 client.9063 fetch_fsmap finished waiting for FSMap version 71 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 10 client.9063 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 20 client.9063 populate_metadata read hostname 'smithi032' 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.32:6836/3239941411,v1:172.21.15.32:6837/3239941411] conn(0x55b09d5e1500 0x55b09d5e18e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 4 client.9063 mds_command: new command op to 8928 tid=0[{"prefix": "status"}] 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.444+0000 7f49d461f000 1 -- 172.21.15.32:0/1305368803 --> [v2:172.21.15.32:6836/3239941411,v1:172.21.15.32:6837/3239941411] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55b09d394160 con 0x55b09d5e1500 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49c3fff640 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.32:6836/3239941411,v1:172.21.15.32:6837/3239941411] conn(0x55b09d5e1500 0x55b09d5e18e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49c3fff640 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.32:6836/3239941411,v1:172.21.15.32:6837/3239941411] conn(0x55b09d5e1500 0x55b09d5e18e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c5a394115610db21 server_cookie=6c7fea94d205c59c in_seq=0 out_seq=0 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49a9ffb640 10 client.9063 ms_handle_connect on v2:172.21.15.32:6836/3239941411 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49a9ffb640 1 -- 172.21.15.32:0/1305368803 <== mds.0 v2:172.21.15.32:6836/3239941411 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+333 (crc 0 0 0) 0x55b09d394160 con 0x55b09d5e1500 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49a9ffb640 10 client.9063 handle_command_reply: tid=0 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 1 client.9063 shutdown 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 1 -- 172.21.15.32:0/1305368803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 msgr2=0x55b09d5e0540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5e0540 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0b9040 tx=0x7f49bc01ab90 comp rx=0 tx=0).stop 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 1 -- 172.21.15.32:0/1305368803 shutdown_connections 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55b09d5a01a0 0x55b09d5e0fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.32:6836/3239941411,v1:172.21.15.32:6837/3239941411] conn(0x55b09d5e1500 0x55b09d5e18e0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55b09d59f880 0x55b09d5e0a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 1 --2- 172.21.15.32:0/1305368803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55b09d54d1c0 0x55b09d5e0540 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 1 -- 172.21.15.32:0/1305368803 >> 172.21.15.32:0/1305368803 conn(0x55b09d553950 msgr2=0x55b09d59c840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:52.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49a9ffb640 0 client.9063 ms_handle_reset on v2:172.21.15.32:6836/3239941411 2023-12-20T05:43:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 1 -- 172.21.15.32:0/1305368803 shutdown_connections 2023-12-20T05:43:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 1 -- 172.21.15.32:0/1305368803 wait complete. 2023-12-20T05:43:52.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d461f000 20 client.9063 trim_cache size 0 max 16384 2023-12-20T05:43:52.453 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T05:43:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T05:43:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T05:43:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 8928, 2023-12-20T05:43:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T05:43:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T05:43:52.453 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T05:43:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9445463350000001, 2023-12-20T05:43:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 71, 2023-12-20T05:43:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 159, 2023-12-20T05:43:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 159, 2023-12-20T05:43:52.454 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 11.074673328999999 2023-12-20T05:43:52.454 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 -- 172.21.15.32:0/3276505968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49ac065b80 msgr2=0x7f49ac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49ac065b80 0x7f49ac068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f49bc0ad000 tx=0x7f49bc0a6000 comp rx=0 tx=0).stop 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 -- 172.21.15.32:0/3276505968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc070950 msgr2=0x7f49cc157b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc070950 0x7f49cc157b70 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7f49b400c780 tx=0x7f49b400cc40 comp rx=0 tx=0).stop 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 -- 172.21.15.32:0/3276505968 shutdown_connections 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f49ac065b80 0x7f49ac068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f49cc15ce30 0x7f49cc1580b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f49cc070950 0x7f49cc157b70 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 --2- 172.21.15.32:0/3276505968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49cc06deb0 0x7f49cc153f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 -- 172.21.15.32:0/3276505968 >> 172.21.15.32:0/3276505968 conn(0x7f49cc0cba50 msgr2=0x7f49cc15f1b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 -- 172.21.15.32:0/3276505968 shutdown_connections 2023-12-20T05:43:52.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.448+0000 7f49d12c4640 1 -- 172.21.15.32:0/3276505968 wait complete. 2023-12-20T05:43:52.468 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T05:43:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8bbf0640 1 Processor -- start 2023-12-20T05:43:52.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8bbf0640 1 -- start start 2023-12-20T05:43:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8bbf0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e8415ce60 0x7f7e840c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8bbf0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e840c9a40 con 0x7f7e840713e0 2023-12-20T05:43:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8bbf0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e840c9180 con 0x7f7e84161240 2023-12-20T05:43:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8bbf0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e84161690 con 0x7f7e8415ce60 2023-12-20T05:43:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8998c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e8415ce60 0x7f7e840c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8998c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7e8415ce60 0x7f7e840c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33748/0 (socket says 172.21.15.32:33748) 2023-12-20T05:43:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8998c640 1 -- 172.21.15.32:0/3631829953 learned_addr learned my addr 172.21.15.32:0/3631829953 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8918b640 1 --1- 172.21.15.32:0/3631829953 >> v1:172.21.15.32:6789/0 conn(0x7f7e840713e0 0x7f7e84160b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43052/0 (socket says 172.21.15.32:43052) 2023-12-20T05:43:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8998c640 1 -- 172.21.15.32:0/3631829953 >> v1:172.21.15.39:6789/0 conn(0x7f7e84161240 legacy=0x7f7e840b1650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8998c640 1 -- 172.21.15.32:0/3631829953 >> v1:172.21.15.32:6789/0 conn(0x7f7e840713e0 legacy=0x7f7e84160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8998c640 1 -- 172.21.15.32:0/3631829953 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e840c93f0 con 0x7f7e8415ce60 2023-12-20T05:43:52.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.612+0000 7f7e8898a640 1 -- 172.21.15.32:0/3631829953 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2412534746 0 0) 0x7f7e840c9a40 con 0x7f7e840713e0 2023-12-20T05:43:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.616+0000 7f7e8998c640 1 --2- 172.21.15.32:0/3631829953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e8415ce60 0x7f7e840c9ed0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009860 tx=0x7f7e8001c4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b2b21026c5d3656 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.616+0000 7f7e8898a640 1 -- 172.21.15.32:0/3631829953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e800ac410 con 0x7f7e8415ce60 2023-12-20T05:43:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.616+0000 7f7e8898a640 1 -- 172.21.15.32:0/3631829953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e800ac570 con 0x7f7e8415ce60 2023-12-20T05:43:52.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.616+0000 7f7e8898a640 1 -- 172.21.15.32:0/3631829953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e800ac900 con 0x7f7e8415ce60 2023-12-20T05:43:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.616+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/3631829953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e8415ce60 msgr2=0x7f7e840c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.616+0000 7f7e8bbf0640 1 --2- 172.21.15.32:0/3631829953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e8415ce60 0x7f7e840c9ed0 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7f7e80009860 tx=0x7f7e8001c4e0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.616+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/3631829953 shutdown_connections 2023-12-20T05:43:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.616+0000 7f7e8bbf0640 1 --2- 172.21.15.32:0/3631829953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e8415ce60 0x7f7e840c9ed0 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.616+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/3631829953 >> 172.21.15.32:0/3631829953 conn(0x7f7e8406a320 msgr2=0x7f7e840ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.616+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/3631829953 shutdown_connections 2023-12-20T05:43:52.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.616+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/3631829953 wait complete. 2023-12-20T05:43:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8bbf0640 1 Processor -- start 2023-12-20T05:43:52.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8bbf0640 1 -- start start 2023-12-20T05:43:52.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8bbf0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e840713e0 0x7f7e8415ff10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8bbf0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e8415ce60 0x7f7e84150cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8bbf0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e84161240 0x7f7e840c40f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8bbf0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e84153a00 con 0x7f7e84161240 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8bbf0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8415f4c0 con 0x7f7e840713e0 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8bbf0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e8415fa80 con 0x7f7e8415ce60 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8a18d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e84161240 0x7f7e840c40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8a18d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e84161240 0x7f7e840c40f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33754/0 (socket says 172.21.15.32:33754) 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8a18d640 1 -- 172.21.15.32:0/559431206 learned_addr learned my addr 172.21.15.32:0/559431206 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8918b640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e8415ce60 0x7f7e84150cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8998c640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e840713e0 0x7f7e8415ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8a18d640 1 -- 172.21.15.32:0/559431206 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e8415ce60 msgr2=0x7f7e84150cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8a18d640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e8415ce60 0x7f7e84150cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8a18d640 1 -- 172.21.15.32:0/559431206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e840713e0 msgr2=0x7f7e8415ff10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8a18d640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e840713e0 0x7f7e8415ff10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8a18d640 1 -- 172.21.15.32:0/559431206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e80009470 con 0x7f7e84161240 2023-12-20T05:43:52.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8a18d640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e84161240 0x7f7e840c40f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00c7b0 tx=0x7f7e7c00cc70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e7a7fc640 1 -- 172.21.15.32:0/559431206 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c004b70 con 0x7f7e84161240 2023-12-20T05:43:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e7a7fc640 1 -- 172.21.15.32:0/559431206 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e7c004cd0 con 0x7f7e84161240 2023-12-20T05:43:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e7a7fc640 1 -- 172.21.15.32:0/559431206 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e7c00aa90 con 0x7f7e84161240 2023-12-20T05:43:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/559431206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e84153c00 con 0x7f7e84161240 2023-12-20T05:43:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.620+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/559431206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e84151230 con 0x7f7e84161240 2023-12-20T05:43:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.624+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/559431206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7e8415fc30 con 0x7f7e84161240 2023-12-20T05:43:52.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.624+0000 7f7e7a7fc640 1 -- 172.21.15.32:0/559431206 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7e7c00abf0 con 0x7f7e84161240 2023-12-20T05:43:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.624+0000 7f7e7a7fc640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e68065bb0 0x7f7e68068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:52.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.624+0000 7f7e7a7fc640 1 -- 172.21.15.32:0/559431206 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7e7c083350 con 0x7f7e84161240 2023-12-20T05:43:52.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.624+0000 7f7e8998c640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e68065bb0 0x7f7e68068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:52.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.624+0000 7f7e8998c640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e68065bb0 0x7f7e68068070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e800ae000 tx=0x7f7e80008040 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:52.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.628+0000 7f7e7a7fc640 1 -- 172.21.15.32:0/559431206 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7e8415fc30 con 0x7f7e84161240 2023-12-20T05:43:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/559431206 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f7e840caf50 con 0x7f7e84161240 2023-12-20T05:43:52.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e7a7fc640 1 -- 172.21.15.32:0/559431206 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v71) v1 ==== 53+0+73 (secure 0 0 0) 0x7f7e7c050c90 con 0x7f7e84161240 2023-12-20T05:43:52.820 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T05:43:52.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/559431206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e68065bb0 msgr2=0x7f7e68068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e68065bb0 0x7f7e68068070 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f7e800ae000 tx=0x7f7e80008040 comp rx=0 tx=0).stop 2023-12-20T05:43:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/559431206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e84161240 msgr2=0x7f7e840c40f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e84161240 0x7f7e840c40f0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7f7e7c00c7b0 tx=0x7f7e7c00cc70 comp rx=0 tx=0).stop 2023-12-20T05:43:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/559431206 shutdown_connections 2023-12-20T05:43:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7e68065bb0 0x7f7e68068070 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e84161240 0x7f7e840c40f0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e8415ce60 0x7f7e84150cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 --2- 172.21.15.32:0/559431206 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e840713e0 0x7f7e8415ff10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/559431206 >> 172.21.15.32:0/559431206 conn(0x7f7e8406a320 msgr2=0x7f7e84157b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/559431206 shutdown_connections 2023-12-20T05:43:52.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:52.820+0000 7f7e8bbf0640 1 -- 172.21.15.32:0/559431206 wait complete. 2023-12-20T05:43:52.842 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T05:43:52.842 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T05:43:52.842 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T05:43:52.842 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T05:43:52.842 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T05:43:52.842 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T05:43:52.842 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T05:43:52.842 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T05:43:52.842 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T05:43:52.843 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:52.843 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T05:43:52.853 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 37827sec preferred_lft 37827sec 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:43:52.854 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:43:52.855 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:52.856 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T05:43:52.918 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T05:43:52.921 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:52.921 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T05:43:52.973 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T05:43:52.973 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T05:43:52.973 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T05:43:52.973 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T05:43:52.973 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T05:43:52.973 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T05:43:52.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:52.974 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T05:43:52.974 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T05:43:52.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T05:43:52.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T05:43:52.974 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T05:43:52.974 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T05:43:53.068 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:53.068 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T05:43:53.078 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T05:43:53.078 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T05:43:53.078 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T05:43:53.078 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:53.078 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T05:43:53.140 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T05:43:53.140 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T05:43:53.196 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T05:43:53.197 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T05:43:53.197 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T05:43:53.248 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T05:43:53.248 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:53.248 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T05:43:53.343 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T05:43:53.343 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T05:43:53.356 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T05:43:53.356 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T05:43:53.356 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T05:43:53.356 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T05:43:53.357 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T05:43:53.360 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T05:43:53.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:53.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T05:43:53.382 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:53.383 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T05:43:53.440 INFO:teuthology.orchestra.run.smithi032.stdout:client7721 2023-12-20T05:43:53.440 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:43:53.440 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:43:53.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T05:43:53.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a9d6d640 1 Processor -- start 2023-12-20T05:43:53.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a9d6d640 1 -- start start 2023-12-20T05:43:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a9d6d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff8a41593c0 0x7ff8a415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a9d6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a40ae140 con 0x7ff8a406c3e0 2023-12-20T05:43:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a9d6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8a40cf580 con 0x7ff8a40afdb0 2023-12-20T05:43:53.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a9d6d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff8a40ae9e0 con 0x7ff8a41593c0 2023-12-20T05:43:53.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff8a406c3e0 0x7ff8a406c7f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43066/0 (socket says 172.21.15.32:43066) 2023-12-20T05:43:53.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a3fff640 1 -- 172.21.15.32:0/1757052930 learned_addr learned my addr 172.21.15.32:0/1757052930 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:53.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a37fe640 1 --2- 172.21.15.32:0/1757052930 >> v2:172.21.15.39:3300/0 conn(0x7ff8a41593c0 0x7ff8a415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:53.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1402484794 0 0) 0x7ff8a40ae140 con 0x7ff8a406c3e0 2023-12-20T05:43:53.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff88c005740 con 0x7ff8a406c3e0 2023-12-20T05:43:53.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3815620035 0 0) 0x7ff8a40cf580 con 0x7ff8a40afdb0 2023-12-20T05:43:53.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8a40ae140 con 0x7ff8a40afdb0 2023-12-20T05:43:53.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3088080404 0 0) 0x7ff88c005740 con 0x7ff8a406c3e0 2023-12-20T05:43:53.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 >> v2:172.21.15.39:3300/0 conn(0x7ff8a41593c0 msgr2=0x7ff8a415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:53.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 --2- 172.21.15.32:0/1757052930 >> v2:172.21.15.39:3300/0 conn(0x7ff8a41593c0 0x7ff8a415d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:53.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 >> v1:172.21.15.39:6789/0 conn(0x7ff8a40afdb0 legacy=0x7ff8a40cee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:53.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a415dee0 con 0x7ff8a406c3e0 2023-12-20T05:43:53.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7ff89c001a00 con 0x7ff8a406c3e0 2023-12-20T05:43:53.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 >> v1:172.21.15.32:6789/0 conn(0x7ff8a406c3e0 legacy=0x7ff8a406c7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 --2- 172.21.15.32:0/1757052930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff88c005d20 0x7ff88c006120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 --2- 172.21.15.32:0/1757052930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88c0069e0 0x7ff88c006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 --2- 172.21.15.32:0/1757052930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff88c0073a0 0x7ff88c00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c00e2a0 con 0x7ff88c0069e0 2023-12-20T05:43:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c00e410 con 0x7ff88c0073a0 2023-12-20T05:43:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff88c00e5d0 con 0x7ff88c005d20 2023-12-20T05:43:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a2ffd640 1 --2- 172.21.15.32:0/1757052930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88c0069e0 0x7ff88c006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a3fff640 1 --2- 172.21.15.32:0/1757052930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff88c0073a0 0x7ff88c00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:53.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a37fe640 1 --2- 172.21.15.32:0/1757052930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff88c005d20 0x7ff88c006120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a2ffd640 1 -- 172.21.15.32:0/1757052930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff88c005d20 msgr2=0x7ff88c006120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a2ffd640 1 --2- 172.21.15.32:0/1757052930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff88c005d20 0x7ff88c006120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a2ffd640 1 -- 172.21.15.32:0/1757052930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff88c0073a0 msgr2=0x7ff88c00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a2ffd640 1 --2- 172.21.15.32:0/1757052930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff88c0073a0 0x7ff88c00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a2ffd640 1 -- 172.21.15.32:0/1757052930 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff88c005740 con 0x7ff88c0069e0 2023-12-20T05:43:53.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.680+0000 7ff8a3fff640 1 -- 172.21.15.32:0/1757052930 reap_dead start 2023-12-20T05:43:53.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.684+0000 7ff8a2ffd640 1 --2- 172.21.15.32:0/1757052930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88c0069e0 0x7ff88c006e60 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff8980018d0 tx=0x7ff89801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44604de00bb16f7c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.684+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8980052d0 con 0x7ff88c0069e0 2023-12-20T05:43:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.684+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8980a7050 con 0x7ff88c0069e0 2023-12-20T05:43:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.684+0000 7ff8a27fc640 1 -- 172.21.15.32:0/1757052930 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff898003600 con 0x7ff88c0069e0 2023-12-20T05:43:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.684+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/1757052930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88c0069e0 msgr2=0x7ff88c006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.684+0000 7ff8a9d6d640 1 --2- 172.21.15.32:0/1757052930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88c0069e0 0x7ff88c006e60 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7ff8980018d0 tx=0x7ff89801c510 comp rx=0 tx=0).stop 2023-12-20T05:43:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.684+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/1757052930 shutdown_connections 2023-12-20T05:43:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.684+0000 7ff8a9d6d640 1 --2- 172.21.15.32:0/1757052930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff88c0069e0 0x7ff88c006e60 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.684+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/1757052930 >> 172.21.15.32:0/1757052930 conn(0x7ff8a406a320 msgr2=0x7ff8a40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:53.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.684+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/1757052930 shutdown_connections 2023-12-20T05:43:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.684+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/1757052930 wait complete. 2023-12-20T05:43:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a9d6d640 1 Processor -- start 2023-12-20T05:43:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a9d6d640 1 -- start start 2023-12-20T05:43:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a9d6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff88c0069e0 0x7ff8a40acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a9d6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a40b40a0 0x7ff8a4148850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a9d6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a41464a0 0x7ff8a4146920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a9d6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40ad2f0 con 0x7ff8a41464a0 2023-12-20T05:43:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a9d6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40ac920 con 0x7ff8a40b40a0 2023-12-20T05:43:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a9d6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8a40ac2f0 con 0x7ff88c0069e0 2023-12-20T05:43:53.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a41464a0 0x7ff8a4146920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a41464a0 0x7ff8a4146920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33772/0 (socket says 172.21.15.32:33772) 2023-12-20T05:43:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a3fff640 1 -- 172.21.15.32:0/4288825301 learned_addr learned my addr 172.21.15.32:0/4288825301 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a3fff640 1 -- 172.21.15.32:0/4288825301 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff88c0069e0 msgr2=0x7ff8a40acdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a3fff640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff88c0069e0 0x7ff8a40acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a3fff640 1 -- 172.21.15.32:0/4288825301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a40b40a0 msgr2=0x7ff8a4148850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a3fff640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a40b40a0 0x7ff8a4148850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a3fff640 1 -- 172.21.15.32:0/4288825301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8a4148d90 con 0x7ff8a41464a0 2023-12-20T05:43:53.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a3fff640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a41464a0 0x7ff8a4146920 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4071590 tx=0x7ff89c007ae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff88bfff640 1 -- 172.21.15.32:0/4288825301 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c00aa90 con 0x7ff8a41464a0 2023-12-20T05:43:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff88bfff640 1 -- 172.21.15.32:0/4288825301 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff89c00abf0 con 0x7ff8a41464a0 2023-12-20T05:43:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff88bfff640 1 -- 172.21.15.32:0/4288825301 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff89c0133f0 con 0x7ff8a41464a0 2023-12-20T05:43:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/4288825301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8a41e79c0 con 0x7ff8a41464a0 2023-12-20T05:43:53.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.688+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/4288825301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8a40ac5b0 con 0x7ff8a41464a0 2023-12-20T05:43:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.692+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/4288825301 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8a415d310 con 0x7ff8a41464a0 2023-12-20T05:43:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.692+0000 7ff88bfff640 1 -- 172.21.15.32:0/4288825301 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff89c013550 con 0x7ff8a41464a0 2023-12-20T05:43:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.692+0000 7ff88bfff640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff890065b80 0x7ff890068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:53.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.692+0000 7ff88bfff640 1 -- 172.21.15.32:0/4288825301 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff89c083870 con 0x7ff8a41464a0 2023-12-20T05:43:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.692+0000 7ff8a37fe640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff890065b80 0x7ff890068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:53.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.692+0000 7ff8a37fe640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff890065b80 0x7ff890068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff894006cf0 tx=0x7ff894006c60 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:53.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.696+0000 7ff88bfff640 1 -- 172.21.15.32:0/4288825301 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff89c051190 con 0x7ff8a41464a0 2023-12-20T05:43:53.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.872+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/4288825301 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7ff8a41597a0 con 0x7ff890065b80 2023-12-20T05:43:53.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff88bfff640 1 -- 172.21.15.32:0/4288825301 <== mgr.8691 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7ff8940096f0 con 0x7ff890065b80 2023-12-20T05:43:53.877 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T05:43:53.877 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:43:53.877 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T05:43:53.877 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:43:53.877 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T05:43:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/4288825301 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff890065b80 msgr2=0x7ff890068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff890065b80 0x7ff890068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ff894006cf0 tx=0x7ff894006c60 comp rx=0 tx=0).stop 2023-12-20T05:43:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/4288825301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a41464a0 msgr2=0x7ff8a4146920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a41464a0 0x7ff8a4146920 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7ff8a4071590 tx=0x7ff89c007ae0 comp rx=0 tx=0).stop 2023-12-20T05:43:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/4288825301 shutdown_connections 2023-12-20T05:43:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff890065b80 0x7ff890068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8a41464a0 0x7ff8a4146920 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:53.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8a40b40a0 0x7ff8a4148850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:53.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 --2- 172.21.15.32:0/4288825301 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff88c0069e0 0x7ff8a40acdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:53.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/4288825301 >> 172.21.15.32:0/4288825301 conn(0x7ff8a406a320 msgr2=0x7ff8a4070d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:53.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/4288825301 shutdown_connections 2023-12-20T05:43:53.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:53.876+0000 7ff8a9d6d640 1 -- 172.21.15.32:0/4288825301 wait complete. 2023-12-20T05:43:53.898 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T05:43:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 Processor -- start 2023-12-20T05:43:54.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 -- start start 2023-12-20T05:43:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3bc0b0cb0 0x7fa3bc0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:54.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3bc15b6a0 con 0x7fa3bc15abb0 2023-12-20T05:43:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3bc06d8c0 con 0x7fa3bc070230 2023-12-20T05:43:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3bc06e160 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3bbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3bc0b0cb0 0x7fa3bc0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c08b3640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa3bc15abb0 0x7fa3bc15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43068/0 (socket says 172.21.15.32:43068) 2023-12-20T05:43:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c08b3640 1 -- 172.21.15.32:0/1135481951 learned_addr learned my addr 172.21.15.32:0/1135481951 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3bbfff640 1 -- 172.21.15.32:0/1135481951 >> v1:172.21.15.39:6789/0 conn(0x7fa3bc070230 legacy=0x7fa3bc06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3bbfff640 1 -- 172.21.15.32:0/1135481951 >> v1:172.21.15.32:6789/0 conn(0x7fa3bc15abb0 legacy=0x7fa3bc15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3bbfff640 1 -- 172.21.15.32:0/1135481951 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3bc06dba0 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3bb7fe640 1 -- 172.21.15.32:0/1135481951 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 632611330 0 0) 0x7fa3bc15b6a0 con 0x7fa3bc15abb0 2023-12-20T05:43:54.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3bbfff640 1 --2- 172.21.15.32:0/1135481951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3bc0b0cb0 0x7fa3bc0caaf0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80097c0 tx=0x7fa3a801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320d73e55a32a77b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:54.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3bb7fe640 1 -- 172.21.15.32:0/1135481951 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80ab640 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3bb7fe640 1 -- 172.21.15.32:0/1135481951 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a80a7070 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3bb7fe640 1 -- 172.21.15.32:0/1135481951 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a80ab9c0 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 -- 172.21.15.32:0/1135481951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3bc0b0cb0 msgr2=0x7fa3bc0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:54.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 --2- 172.21.15.32:0/1135481951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3bc0b0cb0 0x7fa3bc0caaf0 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80097c0 tx=0x7fa3a801c440 comp rx=0 tx=0).stop 2023-12-20T05:43:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 -- 172.21.15.32:0/1135481951 shutdown_connections 2023-12-20T05:43:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 --2- 172.21.15.32:0/1135481951 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3bc0b0cb0 0x7fa3bc0caaf0 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 -- 172.21.15.32:0/1135481951 >> 172.21.15.32:0/1135481951 conn(0x7fa3bc06a510 msgr2=0x7fa3bc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 -- 172.21.15.32:0/1135481951 shutdown_connections 2023-12-20T05:43:54.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 -- 172.21.15.32:0/1135481951 wait complete. 2023-12-20T05:43:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.048+0000 7fa3c2b17640 1 Processor -- start 2023-12-20T05:43:54.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c2b17640 1 -- start start 2023-12-20T05:43:54.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c2b17640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3bc070230 0x7fa3bc14e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c2b17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3bc0b0cb0 0x7fa3bc150400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c2b17640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3bc15abb0 0x7fa3bc14e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c2b17640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc14d510 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c2b17640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc14f4a0 con 0x7fa3bc070230 2023-12-20T05:43:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c2b17640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3bc14d7d0 con 0x7fa3bc15abb0 2023-12-20T05:43:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3bc0b0cb0 0x7fa3bc150400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3bc0b0cb0 0x7fa3bc150400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33792/0 (socket says 172.21.15.32:33792) 2023-12-20T05:43:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3bbfff640 1 -- 172.21.15.32:0/3551880673 learned_addr learned my addr 172.21.15.32:0/3551880673 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:54.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c08b3640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3bc070230 0x7fa3bc14e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c10b4640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3bc15abb0 0x7fa3bc14e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3bbfff640 1 -- 172.21.15.32:0/3551880673 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3bc15abb0 msgr2=0x7fa3bc14e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3bbfff640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3bc15abb0 0x7fa3bc14e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3bbfff640 1 -- 172.21.15.32:0/3551880673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3bc070230 msgr2=0x7fa3bc14e2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3bbfff640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3bc070230 0x7fa3bc14e2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3bbfff640 1 -- 172.21.15.32:0/3551880673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3a8009420 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c08b3640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3bc070230 0x7fa3bc14e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:43:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3bbfff640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3bc0b0cb0 0x7fa3bc150400 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80097c0 tx=0x7fa3a801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3551880673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8004020 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3551880673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3a80b9070 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3551880673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3a8004380 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c2b17640 1 -- 172.21.15.32:0/3551880673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3bc14dab0 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c2b17640 1 -- 172.21.15.32:0/3551880673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3bc13f590 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.052+0000 7fa3c2b17640 1 -- 172.21.15.32:0/3551880673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3bc04e000 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.056+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3551880673 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 84) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa3a8003620 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.056+0000 7fa3b97fa640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa3a0065b80 0x7fa3a0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.056+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3551880673 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa3a80bb070 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.056+0000 7fa3c08b3640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa3a0065b80 0x7fa3a0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:54.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.056+0000 7fa3c08b3640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa3a0065b80 0x7fa3a0068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0000f70 tx=0x7fa3b0007cd0 comp rx=0 tx=0).ready entity=mgr.8691 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:54.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.064+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3551880673 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa3a80eb4a0 con 0x7fa3bc0b0cb0 2023-12-20T05:43:54.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:54.256+0000 7fa3c2b17640 1 -- 172.21.15.32:0/3551880673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fa3bc14df60 con 0x7fa3bc0b0cb0 2023-12-20T05:43:55.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.100+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3551880673 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 85) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fa3a80eb870 con 0x7fa3bc0b0cb0 2023-12-20T05:43:55.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.100+0000 7fa3b97fa640 1 -- 172.21.15.32:0/3551880673 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v85) v1 ==== 92+0+0 (secure 0 0 0) 0x7fa3a80eef30 con 0x7fa3bc0b0cb0 2023-12-20T05:43:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 -- 172.21.15.32:0/3551880673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa3a0065b80 msgr2=0x7fa3a0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa3a0065b80 0x7fa3a0068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0000f70 tx=0x7fa3b0007cd0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 -- 172.21.15.32:0/3551880673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3bc0b0cb0 msgr2=0x7fa3bc150400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3bc0b0cb0 0x7fa3bc150400 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fa3a80097c0 tx=0x7fa3a801ac80 comp rx=0 tx=0).stop 2023-12-20T05:43:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 -- 172.21.15.32:0/3551880673 shutdown_connections 2023-12-20T05:43:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa3a0065b80 0x7fa3a0068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3bc15abb0 0x7fa3bc14e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3bc0b0cb0 0x7fa3bc150400 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 --2- 172.21.15.32:0/3551880673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3bc070230 0x7fa3bc14e2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 -- 172.21.15.32:0/3551880673 >> 172.21.15.32:0/3551880673 conn(0x7fa3bc06a510 msgr2=0x7fa3bc0c48e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 -- 172.21.15.32:0/3551880673 shutdown_connections 2023-12-20T05:43:55.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.104+0000 7fa3c2b17640 1 -- 172.21.15.32:0/3551880673 wait complete. 2023-12-20T05:43:55.128 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T05:43:55.277 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:55.271+0000 7f09d39ff180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T05:43:55.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e359e640 1 Processor -- start 2023-12-20T05:43:55.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e359e640 1 -- start start 2023-12-20T05:43:55.286 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:55.288+0000 7faf83f97180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T05:43:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e359e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4e415ccf0 0x7fc4e40cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e359e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e40d10f0 con 0x7fc4e40cfaf0 2023-12-20T05:43:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e359e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4e40cdd30 con 0x7fc4e40cd7b0 2023-12-20T05:43:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e359e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e40ce2f0 con 0x7fc4e415ccf0 2023-12-20T05:43:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4e415ccf0 0x7fc4e40cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e2d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc4e40cfaf0 0x7fc4e40d1580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43074/0 (socket says 172.21.15.32:43074) 2023-12-20T05:43:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e2d9d640 1 -- 172.21.15.32:0/2351285335 learned_addr learned my addr 172.21.15.32:0/2351285335 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:43:55.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e259c640 1 -- 172.21.15.32:0/2351285335 >> v1:172.21.15.39:6789/0 conn(0x7fc4e40cd7b0 legacy=0x7fc4e40cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e259c640 1 -- 172.21.15.32:0/2351285335 >> v1:172.21.15.32:6789/0 conn(0x7fc4e40cfaf0 legacy=0x7fc4e40d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.288+0000 7fc4e259c640 1 -- 172.21.15.32:0/2351285335 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e40ce010 con 0x7fc4e415ccf0 2023-12-20T05:43:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.292+0000 7fc4e259c640 1 --2- 172.21.15.32:0/2351285335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e415ccf0 0x7fc4e40cd270 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40097c0 tx=0x7fc4d401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d561b5532e0b9001 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.292+0000 7fc4e1d9b640 1 -- 172.21.15.32:0/2351285335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40ab600 con 0x7fc4e415ccf0 2023-12-20T05:43:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.292+0000 7fc4e1d9b640 1 -- 172.21.15.32:0/2351285335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d40a7050 con 0x7fc4e415ccf0 2023-12-20T05:43:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.292+0000 7fc4e1d9b640 1 -- 172.21.15.32:0/2351285335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d40ab980 con 0x7fc4e415ccf0 2023-12-20T05:43:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.292+0000 7fc4e359e640 1 -- 172.21.15.32:0/2351285335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e415ccf0 msgr2=0x7fc4e40cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.292+0000 7fc4e359e640 1 --2- 172.21.15.32:0/2351285335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e415ccf0 0x7fc4e40cd270 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7fc4d40097c0 tx=0x7fc4d401c3f0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.292+0000 7fc4e359e640 1 -- 172.21.15.32:0/2351285335 shutdown_connections 2023-12-20T05:43:55.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.292+0000 7fc4e359e640 1 --2- 172.21.15.32:0/2351285335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e415ccf0 0x7fc4e40cd270 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.292+0000 7fc4e359e640 1 -- 172.21.15.32:0/2351285335 >> 172.21.15.32:0/2351285335 conn(0x7fc4e40658e0 msgr2=0x7fc4e4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.292+0000 7fc4e359e640 1 -- 172.21.15.32:0/2351285335 shutdown_connections 2023-12-20T05:43:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.292+0000 7fc4e359e640 1 -- 172.21.15.32:0/2351285335 wait complete. 2023-12-20T05:43:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e359e640 1 Processor -- start 2023-12-20T05:43:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e359e640 1 -- start start 2023-12-20T05:43:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e40cd7b0 0x7fc4e40c5890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e359e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e40cfaf0 0x7fc4e414ae20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e415ccf0 0x7fc4e40b4c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e359e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40b74e0 con 0x7fc4e415ccf0 2023-12-20T05:43:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e359e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40b47b0 con 0x7fc4e40cd7b0 2023-12-20T05:43:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e359e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40b6f20 con 0x7fc4e40cfaf0 2023-12-20T05:43:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e415ccf0 0x7fc4e40b4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4dbfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e40cfaf0 0x7fc4e414ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e415ccf0 0x7fc4e40b4c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33844/0 (socket says 172.21.15.32:33844) 2023-12-20T05:43:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e259c640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e40cd7b0 0x7fc4e40c5890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e2d9d640 1 -- 172.21.15.32:0/1216176210 learned_addr learned my addr 172.21.15.32:0/1216176210 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e2d9d640 1 -- 172.21.15.32:0/1216176210 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e40cfaf0 msgr2=0x7fc4e414ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e2d9d640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e40cfaf0 0x7fc4e414ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e2d9d640 1 -- 172.21.15.32:0/1216176210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e40cd7b0 msgr2=0x7fc4e40c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e2d9d640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e40cd7b0 0x7fc4e40c5890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e2d9d640 1 -- 172.21.15.32:0/1216176210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4d4009420 con 0x7fc4e415ccf0 2023-12-20T05:43:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e2d9d640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e415ccf0 0x7fc4e40b4c40 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00caf0 tx=0x7fc4dc00a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4daffd640 1 -- 172.21.15.32:0/1216176210 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc00aca0 con 0x7fc4e415ccf0 2023-12-20T05:43:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4daffd640 1 -- 172.21.15.32:0/1216176210 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc00ae00 con 0x7fc4e415ccf0 2023-12-20T05:43:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4daffd640 1 -- 172.21.15.32:0/1216176210 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc012600 con 0x7fc4e415ccf0 2023-12-20T05:43:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e359e640 1 -- 172.21.15.32:0/1216176210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e40b7200 con 0x7fc4e415ccf0 2023-12-20T05:43:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.296+0000 7fc4e359e640 1 -- 172.21.15.32:0/1216176210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e40b4440 con 0x7fc4e415ccf0 2023-12-20T05:43:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.300+0000 7fc4e359e640 1 -- 172.21.15.32:0/1216176210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e40b3fc0 con 0x7fc4e415ccf0 2023-12-20T05:43:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.300+0000 7fc4daffd640 1 -- 172.21.15.32:0/1216176210 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 85) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fc4dc007070 con 0x7fc4e415ccf0 2023-12-20T05:43:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.300+0000 7fc4daffd640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4c4065c00 0x7fc4c40680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.300+0000 7fc4daffd640 1 -- 172.21.15.32:0/1216176210 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc4dc083250 con 0x7fc4e415ccf0 2023-12-20T05:43:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.300+0000 7fc4e259c640 1 -- 172.21.15.32:0/1216176210 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4c4065c00 msgr2=0x7fc4c40680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.300+0000 7fc4e259c640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4c4065c00 0x7fc4c40680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:55.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.304+0000 7fc4daffd640 1 -- 172.21.15.32:0/1216176210 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc4e40b3fc0 con 0x7fc4e415ccf0 2023-12-20T05:43:55.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.464+0000 7fc4e359e640 1 -- 172.21.15.32:0/1216176210 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fc4e40b3fc0 con 0x7fc4e415ccf0 2023-12-20T05:43:55.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.468+0000 7fc4daffd640 1 -- 172.21.15.32:0/1216176210 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v53)=0 v53) v1 ==== 138+0+0 (secure 0 0 0) 0x7fc4e40b3fc0 con 0x7fc4e415ccf0 2023-12-20T05:43:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 -- 172.21.15.32:0/1216176210 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4c4065c00 msgr2=0x7fc4c40680c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4c4065c00 0x7fc4c40680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 -- 172.21.15.32:0/1216176210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e415ccf0 msgr2=0x7fc4e40b4c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e415ccf0 0x7fc4e40b4c40 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7fc4dc00caf0 tx=0x7fc4dc00a060 comp rx=0 tx=0).stop 2023-12-20T05:43:55.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 -- 172.21.15.32:0/1216176210 shutdown_connections 2023-12-20T05:43:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4c4065c00 0x7fc4c40680c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e415ccf0 0x7fc4e40b4c40 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e40cfaf0 0x7fc4e414ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 --2- 172.21.15.32:0/1216176210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e40cd7b0 0x7fc4e40c5890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 -- 172.21.15.32:0/1216176210 >> 172.21.15.32:0/1216176210 conn(0x7fc4e40658e0 msgr2=0x7fc4e4065cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 -- 172.21.15.32:0/1216176210 shutdown_connections 2023-12-20T05:43:55.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.472+0000 7fc4e359e640 1 -- 172.21.15.32:0/1216176210 wait complete. 2023-12-20T05:43:55.490 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T05:43:55.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.640+0000 7ff92b6c4640 1 Processor -- start 2023-12-20T05:43:55.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff92b6c4640 1 -- start start 2023-12-20T05:43:55.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff92b6c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9240d3300 0x7ff9240d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff92b6c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9240d1300 0x7ff9240b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff92b6c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff9240d2650 con 0x7ff92415ce60 2023-12-20T05:43:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff92b6c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff9240d1a40 con 0x7ff9240d1300 2023-12-20T05:43:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff92b6c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9240d1d20 con 0x7ff9240d3300 2023-12-20T05:43:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff929c61640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9240d1300 0x7ff9240b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff929c61640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff9240d1300 0x7ff9240b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33846/0 (socket says 172.21.15.32:33846) 2023-12-20T05:43:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff929c61640 1 -- 172.21.15.32:0/2447605509 learned_addr learned my addr 172.21.15.32:0/2447605509 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff929460640 1 --1- 172.21.15.32:0/2447605509 >> v1:172.21.15.39:6789/0 conn(0x7ff92415ce60 0x7ff9240d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:37774/0 (socket says 172.21.15.32:37774) 2023-12-20T05:43:55.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff928c5f640 1 --2- 172.21.15.32:0/2447605509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9240d3300 0x7ff9240d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff929c61640 1 -- 172.21.15.32:0/2447605509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9240d3300 msgr2=0x7ff9240d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff929c61640 1 --2- 172.21.15.32:0/2447605509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9240d3300 0x7ff9240d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff929c61640 1 -- 172.21.15.32:0/2447605509 >> v1:172.21.15.39:6789/0 conn(0x7ff92415ce60 legacy=0x7ff9240d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff929c61640 1 -- 172.21.15.32:0/2447605509 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9240d2000 con 0x7ff9240d1300 2023-12-20T05:43:55.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.644+0000 7ff91bfff640 1 -- 172.21.15.32:0/2447605509 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 455559666 0 0) 0x7ff9240d2650 con 0x7ff92415ce60 2023-12-20T05:43:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff929c61640 1 --2- 172.21.15.32:0/2447605509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9240d1300 0x7ff9240b5e90 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009e50 tx=0x7ff91c01f1f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc009527c481b5a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff91bfff640 1 -- 172.21.15.32:0/2447605509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c0af400 con 0x7ff9240d1300 2023-12-20T05:43:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff91bfff640 1 -- 172.21.15.32:0/2447605509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff91c0ab050 con 0x7ff9240d1300 2023-12-20T05:43:55.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff91bfff640 1 -- 172.21.15.32:0/2447605509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff91c0af780 con 0x7ff9240d1300 2023-12-20T05:43:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff92b6c4640 1 -- 172.21.15.32:0/2447605509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9240d1300 msgr2=0x7ff9240b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff92b6c4640 1 --2- 172.21.15.32:0/2447605509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9240d1300 0x7ff9240b5e90 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7ff91c009e50 tx=0x7ff91c01f1f0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff92b6c4640 1 -- 172.21.15.32:0/2447605509 shutdown_connections 2023-12-20T05:43:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff92b6c4640 1 --2- 172.21.15.32:0/2447605509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9240d1300 0x7ff9240b5e90 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff92b6c4640 1 --2- 172.21.15.32:0/2447605509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff9240d3300 0x7ff9240d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff92b6c4640 1 -- 172.21.15.32:0/2447605509 >> 172.21.15.32:0/2447605509 conn(0x7ff92406b5e0 msgr2=0x7ff92406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff92b6c4640 1 -- 172.21.15.32:0/2447605509 shutdown_connections 2023-12-20T05:43:55.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff92b6c4640 1 -- 172.21.15.32:0/2447605509 wait complete. 2023-12-20T05:43:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff92b6c4640 1 Processor -- start 2023-12-20T05:43:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.648+0000 7ff92b6c4640 1 -- start start 2023-12-20T05:43:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff92b6c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9240d1300 0x7ff92414ecf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff92b6c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9240d3300 0x7ff9241e6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:55.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff92b6c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff92415ce60 0x7ff9241e6e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff92b6c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9241e7680 con 0x7ff9240d3300 2023-12-20T05:43:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff92b6c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff9241e7960 con 0x7ff9240d1300 2023-12-20T05:43:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff92b6c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff9241f2410 con 0x7ff92415ce60 2023-12-20T05:43:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff928c5f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9240d3300 0x7ff9241e6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff929460640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9240d1300 0x7ff92414ecf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff929c61640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff92415ce60 0x7ff9241e6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff928c5f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9240d3300 0x7ff9241e6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33856/0 (socket says 172.21.15.32:33856) 2023-12-20T05:43:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff929460640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9240d1300 0x7ff92414ecf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:39954/0 (socket says 172.21.15.32:39954) 2023-12-20T05:43:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff928c5f640 1 -- 172.21.15.32:0/3603971651 learned_addr learned my addr 172.21.15.32:0/3603971651 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff929460640 1 -- 172.21.15.32:0/3603971651 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff92415ce60 msgr2=0x7ff9241e6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff929460640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff92415ce60 0x7ff9241e6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff929460640 1 -- 172.21.15.32:0/3603971651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9240d3300 msgr2=0x7ff9241e6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff929460640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9240d3300 0x7ff9241e6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff929460640 1 -- 172.21.15.32:0/3603971651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff91c009b30 con 0x7ff9240d1300 2023-12-20T05:43:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff928c5f640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9240d3300 0x7ff9241e6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:43:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff929460640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9240d1300 0x7ff92414ecf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff92000c410 tx=0x7ff92000c8d0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff919ffb640 1 -- 172.21.15.32:0/3603971651 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9200074e0 con 0x7ff9240d1300 2023-12-20T05:43:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff919ffb640 1 -- 172.21.15.32:0/3603971651 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff920007640 con 0x7ff9240d1300 2023-12-20T05:43:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff919ffb640 1 -- 172.21.15.32:0/3603971651 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff920004a30 con 0x7ff9240d1300 2023-12-20T05:43:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff92b6c4640 1 -- 172.21.15.32:0/3603971651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff9241f26f0 con 0x7ff9240d1300 2023-12-20T05:43:55.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff92b6c4640 1 -- 172.21.15.32:0/3603971651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff9241f2060 con 0x7ff9240d1300 2023-12-20T05:43:55.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.652+0000 7ff92b6c4640 1 -- 172.21.15.32:0/3603971651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff9240d22c0 con 0x7ff9240d1300 2023-12-20T05:43:55.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.656+0000 7ff919ffb640 1 -- 172.21.15.32:0/3603971651 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 85) v1 ==== 85347+0+0 (secure 0 0 0) 0x7ff920004b90 con 0x7ff9240d1300 2023-12-20T05:43:55.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.656+0000 7ff919ffb640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff908065bd0 0x7ff908068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:55.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.656+0000 7ff919ffb640 1 -- 172.21.15.32:0/3603971651 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7ff920083020 con 0x7ff9240d1300 2023-12-20T05:43:55.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.656+0000 7ff928c5f640 1 -- 172.21.15.32:0/3603971651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff908065bd0 msgr2=0x7ff908068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:55.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.656+0000 7ff928c5f640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff908065bd0 0x7ff908068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:55.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.660+0000 7ff919ffb640 1 -- 172.21.15.32:0/3603971651 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff9240d22c0 con 0x7ff9240d1300 2023-12-20T05:43:55.717 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:55.711+0000 7f09d39ff180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T05:43:55.736 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:55.736+0000 7faf83f97180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T05:43:55.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.820+0000 7ff92b6c4640 1 -- 172.21.15.32:0/3603971651 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7ff9240d22c0 con 0x7ff9240d1300 2023-12-20T05:43:55.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.828+0000 7ff919ffb640 1 -- 172.21.15.32:0/3603971651 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v54)=0 v54) v1 ==== 133+0+0 (secure 0 0 0) 0x7ff9240d22c0 con 0x7ff9240d1300 2023-12-20T05:43:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 -- 172.21.15.32:0/3603971651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff908065bd0 msgr2=0x7ff908068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff908065bd0 0x7ff908068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 -- 172.21.15.32:0/3603971651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9240d1300 msgr2=0x7ff92414ecf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9240d1300 0x7ff92414ecf0 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7ff92000c410 tx=0x7ff92000c8d0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 -- 172.21.15.32:0/3603971651 shutdown_connections 2023-12-20T05:43:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff908065bd0 0x7ff908068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff92415ce60 0x7ff9241e6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff9240d3300 0x7ff9241e6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 --2- 172.21.15.32:0/3603971651 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff9240d1300 0x7ff92414ecf0 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:55.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 -- 172.21.15.32:0/3603971651 >> 172.21.15.32:0/3603971651 conn(0x7ff92406b5e0 msgr2=0x7ff924066030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 -- 172.21.15.32:0/3603971651 shutdown_connections 2023-12-20T05:43:55.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:55.832+0000 7ff92b6c4640 1 -- 172.21.15.32:0/3603971651 wait complete. 2023-12-20T05:43:55.849 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:43:55.849 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-20T05:43:55.866 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1h 2023-12-20T05:43:55.874 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:55.871+0000 7f09d39ff180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T05:43:55.896 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:55.900+0000 7faf83f97180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T05:43:56.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.044+0000 7fa950fa6640 1 Processor -- start 2023-12-20T05:43:56.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa950fa6640 1 -- start start 2023-12-20T05:43:56.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa950fa6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa94c15abb0 0x7fa94c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa950fa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 0x7fa94c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa950fa6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa94c15b6a0 con 0x7fa94c0b0cb0 2023-12-20T05:43:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa950fa6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa94c06e160 con 0x7fa94c15abb0 2023-12-20T05:43:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa950fa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c070c00 con 0x7fa94c070230 2023-12-20T05:43:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa94a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa94c15abb0 0x7fa94c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa94a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa94c15abb0 0x7fa94c0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33870/0 (socket says 172.21.15.32:33870) 2023-12-20T05:43:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa94a59c640 1 -- 172.21.15.32:0/3373890409 learned_addr learned my addr 172.21.15.32:0/3373890409 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa94ad9d640 1 --2- 172.21.15.32:0/3373890409 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 0x7fa94c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa943fff640 1 --1- 172.21.15.32:0/3373890409 >> v1:172.21.15.39:6789/0 conn(0x7fa94c0b0cb0 0x7fa94c15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:37790/0 (socket says 172.21.15.32:37790) 2023-12-20T05:43:56.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa94a59c640 1 -- 172.21.15.32:0/3373890409 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 msgr2=0x7fa94c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:43:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa94a59c640 1 --2- 172.21.15.32:0/3373890409 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 0x7fa94c0706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa94a59c640 1 -- 172.21.15.32:0/3373890409 >> v1:172.21.15.39:6789/0 conn(0x7fa94c0b0cb0 legacy=0x7fa94c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:56.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.048+0000 7fa94a59c640 1 -- 172.21.15.32:0/3373890409 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa94c06d8c0 con 0x7fa94c15abb0 2023-12-20T05:43:56.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa94a59c640 1 --2- 172.21.15.32:0/3373890409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa94c15abb0 0x7fa94c0caaf0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009870 tx=0x7fa93c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d624e602c882451 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:56.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa949d9b640 1 -- 172.21.15.32:0/3373890409 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0a9400 con 0x7fa94c15abb0 2023-12-20T05:43:56.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa949d9b640 1 -- 172.21.15.32:0/3373890409 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c0a9560 con 0x7fa94c15abb0 2023-12-20T05:43:56.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa949d9b640 1 -- 172.21.15.32:0/3373890409 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0a98f0 con 0x7fa94c15abb0 2023-12-20T05:43:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa950fa6640 1 -- 172.21.15.32:0/3373890409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa94c15abb0 msgr2=0x7fa94c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa950fa6640 1 --2- 172.21.15.32:0/3373890409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa94c15abb0 0x7fa94c0caaf0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7fa93c009870 tx=0x7fa93c01c540 comp rx=0 tx=0).stop 2023-12-20T05:43:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa950fa6640 1 -- 172.21.15.32:0/3373890409 shutdown_connections 2023-12-20T05:43:56.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa950fa6640 1 --2- 172.21.15.32:0/3373890409 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 0x7fa94c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:56.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa950fa6640 1 --2- 172.21.15.32:0/3373890409 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa94c15abb0 0x7fa94c0caaf0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:56.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa950fa6640 1 -- 172.21.15.32:0/3373890409 >> 172.21.15.32:0/3373890409 conn(0x7fa94c06a510 msgr2=0x7fa94c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:43:56.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa950fa6640 1 -- 172.21.15.32:0/3373890409 shutdown_connections 2023-12-20T05:43:56.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa950fa6640 1 -- 172.21.15.32:0/3373890409 wait complete. 2023-12-20T05:43:56.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.052+0000 7fa950fa6640 1 Processor -- start 2023-12-20T05:43:56.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa950fa6640 1 -- start start 2023-12-20T05:43:56.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa950fa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 0x7fa94c146c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:56.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa950fa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa94c0b0cb0 0x7fa94c150cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:56.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa950fa6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa94c15abb0 0x7fa94c151200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:56.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa950fa6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c149950 con 0x7fa94c15abb0 2023-12-20T05:43:56.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa950fa6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c148d40 con 0x7fa94c0b0cb0 2023-12-20T05:43:56.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa950fa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa94c149020 con 0x7fa94c070230 2023-12-20T05:43:56.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa94a59c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 0x7fa94c146c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:56.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa94ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa94c15abb0 0x7fa94c151200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:56.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa94a59c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 0x7fa94c146c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49450/0 (socket says 172.21.15.32:49450) 2023-12-20T05:43:56.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa94a59c640 1 -- 172.21.15.32:0/1070165955 learned_addr learned my addr 172.21.15.32:0/1070165955 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:43:56.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa943fff640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa94c0b0cb0 0x7fa94c150cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:43:56.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa94a59c640 1 -- 172.21.15.32:0/1070165955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa94c0b0cb0 msgr2=0x7fa94c150cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:56.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa94a59c640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa94c0b0cb0 0x7fa94c150cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:56.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa94a59c640 1 -- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa94c15abb0 msgr2=0x7fa94c151200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:43:56.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa94a59c640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa94c15abb0 0x7fa94c151200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:56.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa94a59c640 1 -- 172.21.15.32:0/1070165955 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa93c0094d0 con 0x7fa94c070230 2023-12-20T05:43:56.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa94a59c640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 0x7fa94c146c10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa93c01abf0 tx=0x7fa93c0192e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:43:56.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa942ffd640 1 -- 172.21.15.32:0/1070165955 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0a9400 con 0x7fa94c070230 2023-12-20T05:43:56.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa942ffd640 1 -- 172.21.15.32:0/1070165955 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa93c0b5070 con 0x7fa94c070230 2023-12-20T05:43:56.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa942ffd640 1 -- 172.21.15.32:0/1070165955 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa93c0b1a20 con 0x7fa94c070230 2023-12-20T05:43:56.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa950fa6640 1 -- 172.21.15.32:0/1070165955 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa94c149300 con 0x7fa94c070230 2023-12-20T05:43:56.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa950fa6640 1 -- 172.21.15.32:0/1070165955 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa94c14fe70 con 0x7fa94c070230 2023-12-20T05:43:56.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.056+0000 7fa950fa6640 1 -- 172.21.15.32:0/1070165955 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa94c04e000 con 0x7fa94c070230 2023-12-20T05:43:56.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.060+0000 7fa942ffd640 1 -- 172.21.15.32:0/1070165955 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 85) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fa93c0b1b80 con 0x7fa94c070230 2023-12-20T05:43:56.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.060+0000 7fa942ffd640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:43:56.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.060+0000 7fa942ffd640 1 -- 172.21.15.32:0/1070165955 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(159..159 src has 1..159) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fa93c0b7070 con 0x7fa94c070230 2023-12-20T05:43:56.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.060+0000 7fa943fff640 1 -- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 msgr2=0x7fa92c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:56.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.060+0000 7fa943fff640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:43:56.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.064+0000 7fa942ffd640 1 -- 172.21.15.32:0/1070165955 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa93c0eb110 con 0x7fa94c070230 2023-12-20T05:43:56.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.244+0000 7fa950fa6640 1 -- 172.21.15.32:0/1070165955 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fa94c15b330 con 0x7fa92c065bd0 2023-12-20T05:43:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.264+0000 7fa943fff640 1 -- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 msgr2=0x7fa92c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:56.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.264+0000 7fa943fff640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T05:43:56.383 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:56.379+0000 7f09d39ff180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T05:43:56.410 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:56.412+0000 7faf83f97180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T05:43:56.445 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:56.439+0000 7f09d39ff180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T05:43:56.472 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:56.476+0000 7faf83f97180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T05:43:56.503 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:56.499+0000 7f09d39ff180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T05:43:56.530 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:56.532+0000 7faf83f97180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T05:43:56.594 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:56.591+0000 7f09d39ff180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T05:43:56.622 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:56.624+0000 7faf83f97180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T05:43:56.654 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:56.647+0000 7f09d39ff180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T05:43:56.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.668+0000 7fa943fff640 1 -- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 msgr2=0x7fa92c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:56.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:56.668+0000 7fa943fff640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T05:43:56.681 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:56.684+0000 7faf83f97180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T05:43:56.742 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:56.739+0000 7f09d39ff180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T05:43:56.770 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:56.772+0000 7faf83f97180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T05:43:56.801 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:56.795+0000 7f09d39ff180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T05:43:56.829 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:56.832+0000 7faf83f97180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T05:43:56.952 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:56.947+0000 7f09d39ff180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T05:43:56.979 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:56.980+0000 7faf83f97180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T05:43:57.014 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:57.011+0000 7f09d39ff180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T05:43:57.041 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:57.044+0000 7faf83f97180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T05:43:57.266 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T05:43:57.266 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T05:43:57.266 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T05:43:57.274 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:57.271+0000 7f09d39ff180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T05:43:57.291 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T05:43:57.291 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T05:43:57.291 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T05:43:57.300 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:57.304+0000 7faf83f97180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T05:43:57.416 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:57.411+0000 7f09d39ff180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:43:57.442 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:57.444+0000 7faf83f97180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:43:57.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:57.472+0000 7fa943fff640 1 -- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 msgr2=0x7fa92c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:57.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:57.472+0000 7fa943fff640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T05:43:57.547 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:57.543+0000 7f09d39ff180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T05:43:57.574 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:57.576+0000 7faf83f97180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T05:43:57.621 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:57.615+0000 7f09d39ff180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T05:43:57.649 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:57.652+0000 7faf83f97180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T05:43:57.763 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:57.759+0000 7f09d39ff180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:43:57.791 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:57.792+0000 7faf83f97180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:43:58.333 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:58.327+0000 7f09d39ff180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T05:43:58.361 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:58.364+0000 7faf83f97180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T05:43:58.421 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:58.415+0000 7f09d39ff180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T05:43:58.450 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:58.452+0000 7faf83f97180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T05:43:58.578 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:58.575+0000 7f09d39ff180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T05:43:58.607 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:58.608+0000 7faf83f97180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T05:43:58.665 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:58.659+0000 7f09d39ff180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T05:43:58.694 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:58.696+0000 7faf83f97180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T05:43:59.000 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:58.995+0000 7f09d39ff180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T05:43:59.030 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.032+0000 7faf83f97180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T05:43:59.063 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:43:59.059+0000 7f09d39ff180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T05:43:59.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:59.076+0000 7fa943fff640 1 -- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 msgr2=0x7fa92c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:43:59.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:59.076+0000 7fa943fff640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-20T05:43:59.095 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.096+0000 7faf83f97180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T05:43:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:59.112+0000 7fa942ffd640 1 -- 172.21.15.32:0/1070165955 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mgrmap(e 86) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fa93c0eb4e0 con 0x7fa94c070230 2023-12-20T05:43:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:59.112+0000 7fa942ffd640 1 -- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 msgr2=0x7fa92c068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:43:59.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:43:59.112+0000 7fa942ffd640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:43:59.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.228+0000 7faf09e00640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:59.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.228+0000 7faf09e00640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:59.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.228+0000 7faf09e00640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:59.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.228+0000 7faf09e00640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:59.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.228+0000 7faf09e00640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:59.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.232+0000 7faf065f9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:59.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.232+0000 7faf065f9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:59.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.232+0000 7faf065f9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:59.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.232+0000 7faf065f9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:43:59.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:43:59.232+0000 7faf065f9640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:00.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.148+0000 7fa942ffd640 1 -- 172.21.15.32:0/1070165955 <== mon.2 v2:172.21.15.39:3301/0 8 ==== mgrmap(e 87) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fa93c1061a0 con 0x7fa94c070230 2023-12-20T05:44:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.148+0000 7fa942ffd640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.148+0000 7fa942ffd640 1 -- 172.21.15.32:0/1070165955 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1h", "target": ["mon-mgr", ""]}) v1 -- 0x7fa92c033610 con 0x7fa92c065bd0 2023-12-20T05:44:00.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.148+0000 7fa943fff640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.152+0000 7fa943fff640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa94c1502a0 tx=0x7fa934007e50 comp rx=0 tx=0).ready entity=mgr.9099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:00.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.212+0000 7fa942ffd640 1 -- 172.21.15.32:0/1070165955 <== mgr.9099 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fa92c033610 con 0x7fa92c065bd0 2023-12-20T05:44:00.211 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path snap_test_dir1 2023-12-20T05:44:00.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 -- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 msgr2=0x7fa92c068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fa94c1502a0 tx=0x7fa934007e50 comp rx=0 tx=0).stop 2023-12-20T05:44:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 -- 172.21.15.32:0/1070165955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 msgr2=0x7fa94c146c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 0x7fa94c146c10 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa93c01abf0 tx=0x7fa93c0192e0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 -- 172.21.15.32:0/1070165955 shutdown_connections 2023-12-20T05:44:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa92c065bd0 0x7fa92c068090 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa94c15abb0 0x7fa94c151200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa94c0b0cb0 0x7fa94c150cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 --2- 172.21.15.32:0/1070165955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa94c070230 0x7fa94c146c10 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 -- 172.21.15.32:0/1070165955 >> 172.21.15.32:0/1070165955 conn(0x7fa94c06a510 msgr2=0x7fa94c0c3900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 -- 172.21.15.32:0/1070165955 shutdown_connections 2023-12-20T05:44:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.216+0000 7fa950fa6640 1 -- 172.21.15.32:0/1070165955 wait complete. 2023-12-20T05:44:00.234 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-20T05:44:00.234 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-20T05:44:00.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab5317b640 1 Processor -- start 2023-12-20T05:44:00.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab5317b640 1 -- start start 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab5317b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab4c0b0cb0 0x7fab4c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab5317b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab4c070230 0x7fab4c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab5317b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fab4c15b6a0 con 0x7fab4c15abb0 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab5317b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fab4c06e160 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab5317b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c070c00 con 0x7fab4c070230 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab4bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fab4c0b0cb0 0x7fab4c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab50f17640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fab4c15abb0 0x7fab4c15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50426/0 (socket says 172.21.15.32:50426) 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab50f17640 1 -- 172.21.15.32:0/975214134 learned_addr learned my addr 172.21.15.32:0/975214134 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab51718640 1 --2- 172.21.15.32:0/975214134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab4c070230 0x7fab4c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab4bfff640 1 -- 172.21.15.32:0/975214134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab4c070230 msgr2=0x7fab4c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab4bfff640 1 --2- 172.21.15.32:0/975214134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab4c070230 0x7fab4c0706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab4bfff640 1 -- 172.21.15.32:0/975214134 >> v1:172.21.15.32:6789/0 conn(0x7fab4c15abb0 legacy=0x7fab4c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab4bfff640 1 -- 172.21.15.32:0/975214134 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab4c06d8c0 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.384+0000 7fab4b7fe640 1 -- 172.21.15.32:0/975214134 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 823229567 0 0) 0x7fab4c15b6a0 con 0x7fab4c15abb0 2023-12-20T05:44:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab4bfff640 1 --2- 172.21.15.32:0/975214134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0b0cb0 0x7fab4c0caaf0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fab380097c0 tx=0x7fab3801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0172b30e1cfa30b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab4b7fe640 1 -- 172.21.15.32:0/975214134 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380ab680 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab4b7fe640 1 -- 172.21.15.32:0/975214134 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab380ab7e0 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab4b7fe640 1 -- 172.21.15.32:0/975214134 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380abb70 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab5317b640 1 -- 172.21.15.32:0/975214134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0b0cb0 msgr2=0x7fab4c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab5317b640 1 --2- 172.21.15.32:0/975214134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0b0cb0 0x7fab4c0caaf0 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fab380097c0 tx=0x7fab3801c480 comp rx=0 tx=0).stop 2023-12-20T05:44:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab5317b640 1 -- 172.21.15.32:0/975214134 shutdown_connections 2023-12-20T05:44:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab5317b640 1 --2- 172.21.15.32:0/975214134 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c0b0cb0 0x7fab4c0caaf0 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab5317b640 1 --2- 172.21.15.32:0/975214134 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab4c070230 0x7fab4c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab5317b640 1 -- 172.21.15.32:0/975214134 >> 172.21.15.32:0/975214134 conn(0x7fab4c06a510 msgr2=0x7fab4c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab5317b640 1 -- 172.21.15.32:0/975214134 shutdown_connections 2023-12-20T05:44:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab5317b640 1 -- 172.21.15.32:0/975214134 wait complete. 2023-12-20T05:44:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab5317b640 1 Processor -- start 2023-12-20T05:44:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.388+0000 7fab5317b640 1 -- start start 2023-12-20T05:44:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab5317b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab4c070230 0x7fab4c1f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab5317b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab4c0b0cb0 0x7fab4c137fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab5317b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c15abb0 0x7fab4c1384e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab5317b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c14ec70 con 0x7fab4c15abb0 2023-12-20T05:44:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab5317b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c151a90 con 0x7fab4c070230 2023-12-20T05:44:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab5317b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fab4c14e5b0 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab51718640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c15abb0 0x7fab4c1384e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab51718640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c15abb0 0x7fab4c1384e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53716/0 (socket says 172.21.15.32:53716) 2023-12-20T05:44:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab51718640 1 -- 172.21.15.32:0/2378018426 learned_addr learned my addr 172.21.15.32:0/2378018426 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab4bfff640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab4c0b0cb0 0x7fab4c137fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab50f17640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab4c070230 0x7fab4c1f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab4bfff640 1 -- 172.21.15.32:0/2378018426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab4c070230 msgr2=0x7fab4c1f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab4bfff640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab4c070230 0x7fab4c1f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab4bfff640 1 -- 172.21.15.32:0/2378018426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c15abb0 msgr2=0x7fab4c1384e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab4bfff640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c15abb0 0x7fab4c1384e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab4bfff640 1 -- 172.21.15.32:0/2378018426 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fab38009420 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab51718640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c15abb0 0x7fab4c1384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab4bfff640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab4c0b0cb0 0x7fab4c137fa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fab380ac000 tx=0x7fab3801acc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab497fa640 1 -- 172.21.15.32:0/2378018426 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380bf610 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab497fa640 1 -- 172.21.15.32:0/2378018426 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fab380bf790 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab497fa640 1 -- 172.21.15.32:0/2378018426 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fab380bfb40 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab5317b640 1 -- 172.21.15.32:0/2378018426 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fab4c14e890 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab5317b640 1 -- 172.21.15.32:0/2378018426 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fab4c1f5a00 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.392+0000 7fab5317b640 1 -- 172.21.15.32:0/2378018426 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fab4c14e3c0 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.396+0000 7fab497fa640 1 -- 172.21.15.32:0/2378018426 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 87) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fab380b5400 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.396+0000 7fab497fa640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab30065900 0x7fab30067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.396+0000 7fab497fa640 1 -- 172.21.15.32:0/2378018426 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fab3811dcf0 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.396+0000 7fab50f17640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab30065900 0x7fab30067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.396+0000 7fab50f17640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab30065900 0x7fab30067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fab40000f70 tx=0x7fab40007df0 comp rx=0 tx=0).ready entity=mgr.9099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:00.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.400+0000 7fab497fa640 1 -- 172.21.15.32:0/2378018426 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fab380eb920 con 0x7fab4c0b0cb0 2023-12-20T05:44:00.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.576+0000 7fab5317b640 1 -- 172.21.15.32:0/2378018426 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fab4c15b330 con 0x7fab30065900 2023-12-20T05:44:00.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.636+0000 7fab497fa640 1 -- 172.21.15.32:0/2378018426 <== mgr.9099 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fab40004c80 con 0x7fab30065900 2023-12-20T05:44:00.634 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path snap_test_dir1 2023-12-20T05:44:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 -- 172.21.15.32:0/2378018426 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab30065900 msgr2=0x7fab30067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab30065900 0x7fab30067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fab40000f70 tx=0x7fab40007df0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 -- 172.21.15.32:0/2378018426 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab4c0b0cb0 msgr2=0x7fab4c137fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab4c0b0cb0 0x7fab4c137fa0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fab380ac000 tx=0x7fab3801acc0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 -- 172.21.15.32:0/2378018426 shutdown_connections 2023-12-20T05:44:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fab30065900 0x7fab30067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fab4c15abb0 0x7fab4c1384e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fab4c0b0cb0 0x7fab4c137fa0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 --2- 172.21.15.32:0/2378018426 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fab4c070230 0x7fab4c1f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 -- 172.21.15.32:0/2378018426 >> 172.21.15.32:0/2378018426 conn(0x7fab4c06a510 msgr2=0x7fab4c0c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 -- 172.21.15.32:0/2378018426 shutdown_connections 2023-12-20T05:44:00.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.640+0000 7fab5317b640 1 -- 172.21.15.32:0/2378018426 wait complete. 2023-12-20T05:44:00.656 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-20T05:44:00.656 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list snap_test_dir1 2023-12-20T05:44:00.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.804+0000 7fa7dd5b3640 1 Processor -- start 2023-12-20T05:44:00.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.804+0000 7fa7dd5b3640 1 -- start start 2023-12-20T05:44:00.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7dd5b3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa7d8157480 0x7fa7d80c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7dd5b3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d80cde20 con 0x7fa7d80cd4c0 2023-12-20T05:44:00.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7dd5b3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7d80c8e30 con 0x7fa7d80b0760 2023-12-20T05:44:00.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7dd5b3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7d815e480 con 0x7fa7d8157480 2023-12-20T05:44:00.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d77fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa7d80cd4c0 0x7fa7d815dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50432/0 (socket says 172.21.15.32:50432) 2023-12-20T05:44:00.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d77fe640 1 -- 172.21.15.32:0/3729559975 learned_addr learned my addr 172.21.15.32:0/3729559975 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:00.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d6ffd640 1 --2- 172.21.15.32:0/3729559975 >> v2:172.21.15.39:3300/0 conn(0x7fa7d8157480 0x7fa7d80c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 360248687 0 0) 0x7fa7d80cde20 con 0x7fa7d80cd4c0 2023-12-20T05:44:00.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7c4003530 con 0x7fa7d80cd4c0 2023-12-20T05:44:00.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 113303576 0 0) 0x7fa7c4003530 con 0x7fa7d80cd4c0 2023-12-20T05:44:00.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 >> v2:172.21.15.39:3300/0 conn(0x7fa7d8157480 msgr2=0x7fa7d80c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:00.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 --2- 172.21.15.32:0/3729559975 >> v2:172.21.15.39:3300/0 conn(0x7fa7d8157480 0x7fa7d80c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 >> v1:172.21.15.39:6789/0 conn(0x7fa7d80b0760 legacy=0x7fa7d815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d80ce070 con 0x7fa7d80cd4c0 2023-12-20T05:44:00.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fa7d0001a00 con 0x7fa7d80cd4c0 2023-12-20T05:44:00.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 >> v1:172.21.15.32:6789/0 conn(0x7fa7d80cd4c0 legacy=0x7fa7d815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 --2- 172.21.15.32:0/3729559975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c4003980 0x7fa7c4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 --2- 172.21.15.32:0/3729559975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c4004380 0x7fa7c4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 --2- 172.21.15.32:0/3729559975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7c40050f0 0x7fa7c400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d815e480 con 0x7fa7c4004380 2023-12-20T05:44:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c400c030 con 0x7fa7c4003980 2023-12-20T05:44:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7c400c1f0 con 0x7fa7c40050f0 2023-12-20T05:44:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d6ffd640 1 --2- 172.21.15.32:0/3729559975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c4003980 0x7fa7c4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7cffff640 1 --2- 172.21.15.32:0/3729559975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c4004380 0x7fa7c4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d77fe640 1 --2- 172.21.15.32:0/3729559975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7c40050f0 0x7fa7c400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d6ffd640 1 -- 172.21.15.32:0/3729559975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7c40050f0 msgr2=0x7fa7c400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d6ffd640 1 --2- 172.21.15.32:0/3729559975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7c40050f0 0x7fa7c400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d6ffd640 1 -- 172.21.15.32:0/3729559975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c4004380 msgr2=0x7fa7c4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:00.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d6ffd640 1 --2- 172.21.15.32:0/3729559975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7c4004380 0x7fa7c4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d6ffd640 1 -- 172.21.15.32:0/3729559975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7c4003530 con 0x7fa7c4003980 2023-12-20T05:44:00.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.808+0000 7fa7d77fe640 1 -- 172.21.15.32:0/3729559975 reap_dead start 2023-12-20T05:44:00.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7d6ffd640 1 --2- 172.21.15.32:0/3729559975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c4003980 0x7fa7c4003d60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009ae0 tx=0x7fa7c801ca70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9120a75ca91716a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:00.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80a5660 con 0x7fa7c4003980 2023-12-20T05:44:00.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7c80a57c0 con 0x7fa7c4003980 2023-12-20T05:44:00.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7d67fc640 1 -- 172.21.15.32:0/3729559975 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7c80a5b70 con 0x7fa7c4003980 2023-12-20T05:44:00.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7dd5b3640 1 -- 172.21.15.32:0/3729559975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c4003980 msgr2=0x7fa7c4003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7dd5b3640 1 --2- 172.21.15.32:0/3729559975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c4003980 0x7fa7c4003d60 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7fa7c8009ae0 tx=0x7fa7c801ca70 comp rx=0 tx=0).stop 2023-12-20T05:44:00.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7dd5b3640 1 -- 172.21.15.32:0/3729559975 shutdown_connections 2023-12-20T05:44:00.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7dd5b3640 1 --2- 172.21.15.32:0/3729559975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c4003980 0x7fa7c4003d60 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7dd5b3640 1 -- 172.21.15.32:0/3729559975 >> 172.21.15.32:0/3729559975 conn(0x7fa7d806a510 msgr2=0x7fa7d80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:00.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7dd5b3640 1 -- 172.21.15.32:0/3729559975 shutdown_connections 2023-12-20T05:44:00.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7dd5b3640 1 -- 172.21.15.32:0/3729559975 wait complete. 2023-12-20T05:44:00.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7dd5b3640 1 Processor -- start 2023-12-20T05:44:00.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.812+0000 7fa7dd5b3640 1 -- start start 2023-12-20T05:44:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7dd5b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c4003980 0x7fa7d81f1430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7dd5b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d8159050 0x7fa7d8156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7dd5b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7d81594d0 0x7fa7d81565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7dd5b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d81f2410 con 0x7fa7d8159050 2023-12-20T05:44:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7dd5b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d8146bd0 con 0x7fa7c4003980 2023-12-20T05:44:00.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7dd5b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa7d8156b10 con 0x7fa7d81594d0 2023-12-20T05:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d8159050 0x7fa7d8156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7cffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d8159050 0x7fa7d8156090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53724/0 (socket says 172.21.15.32:53724) 2023-12-20T05:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7cffff640 1 -- 172.21.15.32:0/523806752 learned_addr learned my addr 172.21.15.32:0/523806752 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7d6ffd640 1 --2- 172.21.15.32:0/523806752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c4003980 0x7fa7d81f1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7d77fe640 1 --2- 172.21.15.32:0/523806752 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7d81594d0 0x7fa7d81565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7d77fe640 1 -- 172.21.15.32:0/523806752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c4003980 msgr2=0x7fa7d81f1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7d77fe640 1 --2- 172.21.15.32:0/523806752 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7c4003980 0x7fa7d81f1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7d77fe640 1 -- 172.21.15.32:0/523806752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d8159050 msgr2=0x7fa7d8156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7d77fe640 1 --2- 172.21.15.32:0/523806752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d8159050 0x7fa7d8156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:00.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7d77fe640 1 -- 172.21.15.32:0/523806752 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7d814b970 con 0x7fa7d81594d0 2023-12-20T05:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7cffff640 1 --2- 172.21.15.32:0/523806752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa7d8159050 0x7fa7d8156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7d77fe640 1 --2- 172.21.15.32:0/523806752 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7d81594d0 0x7fa7d81565d0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fa7d806dfb0 tx=0x7fa7d0007bd0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7cf7fe640 1 -- 172.21.15.32:0/523806752 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d000ab70 con 0x7fa7d81594d0 2023-12-20T05:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7cf7fe640 1 -- 172.21.15.32:0/523806752 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7d000ad30 con 0x7fa7d81594d0 2023-12-20T05:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7dd5b3640 1 -- 172.21.15.32:0/523806752 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa7d814bc50 con 0x7fa7d81594d0 2023-12-20T05:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7cf7fe640 1 -- 172.21.15.32:0/523806752 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7d000aea0 con 0x7fa7d81594d0 2023-12-20T05:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7dd5b3640 1 -- 172.21.15.32:0/523806752 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa7d814c210 con 0x7fa7d81594d0 2023-12-20T05:44:00.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.816+0000 7fa7dd5b3640 1 -- 172.21.15.32:0/523806752 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa7d8157860 con 0x7fa7d81594d0 2023-12-20T05:44:00.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.820+0000 7fa7cf7fe640 1 -- 172.21.15.32:0/523806752 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 87) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fa7d0013630 con 0x7fa7d81594d0 2023-12-20T05:44:00.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.820+0000 7fa7cf7fe640 1 --2- 172.21.15.32:0/523806752 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa7b8065900 0x7fa7b8067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:00.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.820+0000 7fa7cf7fe640 1 -- 172.21.15.32:0/523806752 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fa7d0083520 con 0x7fa7d81594d0 2023-12-20T05:44:00.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.820+0000 7fa7d6ffd640 1 --2- 172.21.15.32:0/523806752 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa7b8065900 0x7fa7b8067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:00.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.820+0000 7fa7d6ffd640 1 --2- 172.21.15.32:0/523806752 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fa7b8065900 0x7fa7b8067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fa7c80af000 tx=0x7fa7c80046d0 comp rx=0 tx=0).ready entity=mgr.9099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:00.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:00.824+0000 7fa7cf7fe640 1 -- 172.21.15.32:0/523806752 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa7d8157860 con 0x7fa7d81594d0 2023-12-20T05:44:00.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.000+0000 7fa7dd5b3640 1 -- 172.21.15.32:0/523806752 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fa7d80c9080 con 0x7fa7b8065900 2023-12-20T05:44:01.004 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:01.008+0000 7faf10e0e640 -1 mgr.server reply reply (2) No such file or directory SnapSchedule for snap_test_dir1 not found 2023-12-20T05:44:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.008+0000 7fa7cf7fe640 1 -- 172.21.15.32:0/523806752 <== mgr.9099 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: -2 SnapSchedule for snap_test_dir1 not found) v1 ==== 49+0+0 (secure 0 0 0) 0x7fa7c80a5930 con 0x7fa7b8065900 2023-12-20T05:44:01.005 INFO:teuthology.orchestra.run.smithi032.stderr:Error ENOENT: SnapSchedule for snap_test_dir1 not found 2023-12-20T05:44:01.013 DEBUG:teuthology.orchestra.run:got remote process result: 2 2023-12-20T05:44:01.014 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:01.014 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-20T05:44:01.026 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T05:44:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d62441640 1 Processor -- start 2023-12-20T05:44:01.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d62441640 1 -- start start 2023-12-20T05:44:01.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d62441640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9d5c157480 0x7f9d5c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:01.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d62441640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d5c0cd4c0 0x7f9d5c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:01.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d62441640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d5c0b1100 con 0x7f9d5c0b0760 2023-12-20T05:44:01.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d62441640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5c0cde20 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d62441640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d5c0c8e30 con 0x7f9d5c157480 2023-12-20T05:44:01.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d609de640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d5c0cd4c0 0x7f9d5c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:01.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d609de640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9d5c0cd4c0 0x7f9d5c0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53750/0 (socket says 172.21.15.32:53750) 2023-12-20T05:44:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d609de640 1 -- 172.21.15.32:0/2510922221 learned_addr learned my addr 172.21.15.32:0/2510922221 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d5bfff640 1 --2- 172.21.15.32:0/2510922221 >> v2:172.21.15.39:3300/0 conn(0x7f9d5c157480 0x7f9d5c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d609de640 1 -- 172.21.15.32:0/2510922221 >> v2:172.21.15.39:3300/0 conn(0x7f9d5c157480 msgr2=0x7f9d5c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d609de640 1 --2- 172.21.15.32:0/2510922221 >> v2:172.21.15.39:3300/0 conn(0x7f9d5c157480 0x7f9d5c0c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d609de640 1 -- 172.21.15.32:0/2510922221 >> v1:172.21.15.39:6789/0 conn(0x7f9d5c0b0760 legacy=0x7f9d5c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d609de640 1 -- 172.21.15.32:0/2510922221 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d5c0ce070 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.224+0000 7f9d5bfff640 1 --2- 172.21.15.32:0/2510922221 >> v2:172.21.15.39:3300/0 conn(0x7f9d5c157480 0x7f9d5c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:01.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d609de640 1 --2- 172.21.15.32:0/2510922221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5c0cd4c0 0x7f9d5c0b0bc0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0cf410 tx=0x7f9d5401eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7ff89fa937c6f8c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:01.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d5affd640 1 -- 172.21.15.32:0/2510922221 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d5400b2d0 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d5affd640 1 -- 172.21.15.32:0/2510922221 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d5400b430 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d5affd640 1 -- 172.21.15.32:0/2510922221 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d540a7b20 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d62441640 1 -- 172.21.15.32:0/2510922221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5c0cd4c0 msgr2=0x7f9d5c0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:01.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d62441640 1 --2- 172.21.15.32:0/2510922221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5c0cd4c0 0x7f9d5c0b0bc0 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7f9d5c0cf410 tx=0x7f9d5401eb10 comp rx=0 tx=0).stop 2023-12-20T05:44:01.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d62441640 1 -- 172.21.15.32:0/2510922221 shutdown_connections 2023-12-20T05:44:01.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d62441640 1 --2- 172.21.15.32:0/2510922221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5c0cd4c0 0x7f9d5c0b0bc0 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:01.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d62441640 1 --2- 172.21.15.32:0/2510922221 >> v2:172.21.15.39:3300/0 conn(0x7f9d5c157480 0x7f9d5c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:01.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d62441640 1 -- 172.21.15.32:0/2510922221 >> 172.21.15.32:0/2510922221 conn(0x7f9d5c06a510 msgr2=0x7f9d5c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:01.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d62441640 1 -- 172.21.15.32:0/2510922221 shutdown_connections 2023-12-20T05:44:01.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.228+0000 7f9d62441640 1 -- 172.21.15.32:0/2510922221 wait complete. 2023-12-20T05:44:01.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d62441640 1 Processor -- start 2023-12-20T05:44:01.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d62441640 1 -- start start 2023-12-20T05:44:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d62441640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d5c0b0760 0x7f9d5c0b24e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d62441640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5c0cd4c0 0x7f9d5c1500a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:01.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d62441640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d5c157480 0x7f9d5c15c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d62441640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c0b2ba0 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d62441640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c0b2e80 con 0x7f9d5c0b0760 2023-12-20T05:44:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d62441640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d5c0b3160 con 0x7f9d5c157480 2023-12-20T05:44:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5c0cd4c0 0x7f9d5c1500a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d5b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5c0cd4c0 0x7f9d5c1500a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53766/0 (socket says 172.21.15.32:53766) 2023-12-20T05:44:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d5b7fe640 1 -- 172.21.15.32:0/2682894719 learned_addr learned my addr 172.21.15.32:0/2682894719 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d609de640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d5c157480 0x7f9d5c15c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d5bfff640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d5c0b0760 0x7f9d5c0b24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d5b7fe640 1 -- 172.21.15.32:0/2682894719 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d5c157480 msgr2=0x7f9d5c15c1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d5b7fe640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d5c157480 0x7f9d5c15c1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:01.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d5b7fe640 1 -- 172.21.15.32:0/2682894719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d5c0b0760 msgr2=0x7f9d5c0b24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:01.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d5b7fe640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d5c0b0760 0x7f9d5c0b24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:01.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d5b7fe640 1 -- 172.21.15.32:0/2682894719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d4c009420 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d5bfff640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d5c0b0760 0x7f9d5c0b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:01.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d5b7fe640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5c0cd4c0 0x7f9d5c1500a0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c3c0 tx=0x7f9d5000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:01.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d58ff9640 1 -- 172.21.15.32:0/2682894719 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d500052d0 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d58ff9640 1 -- 172.21.15.32:0/2682894719 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d50005430 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d58ff9640 1 -- 172.21.15.32:0/2682894719 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d50003840 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d62441640 1 -- 172.21.15.32:0/2682894719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d5400bd40 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.232+0000 7f9d62441640 1 -- 172.21.15.32:0/2682894719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d5c0b3720 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.236+0000 7f9d62441640 1 -- 172.21.15.32:0/2682894719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d5c04e000 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.236+0000 7f9d58ff9640 1 -- 172.21.15.32:0/2682894719 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 88) v1 ==== 85348+0+0 (secure 0 0 0) 0x7f9d500039a0 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.236+0000 7f9d58ff9640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d48065c00 0x7f9d480680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.236+0000 7f9d58ff9640 1 -- 172.21.15.32:0/2682894719 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f9d500829d0 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.236+0000 7f9d5bfff640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d48065c00 0x7f9d480680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:01.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.236+0000 7f9d5bfff640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d48065c00 0x7f9d480680c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009d40 tx=0x7f9d4c008040 comp rx=0 tx=0).ready entity=mgr.9099 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:01.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.240+0000 7f9d58ff9640 1 -- 172.21.15.32:0/2682894719 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9d50050110 con 0x7f9d5c0cd4c0 2023-12-20T05:44:01.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:01.436+0000 7f9d62441640 1 -- 172.21.15.32:0/2682894719 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f9d5c156ad0 con 0x7f9d5c0cd4c0 2023-12-20T05:44:02.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.168+0000 7f9d58ff9640 1 -- 172.21.15.32:0/2682894719 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 89) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f9d500504e0 con 0x7f9d5c0cd4c0 2023-12-20T05:44:02.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.168+0000 7f9d58ff9640 1 -- 172.21.15.32:0/2682894719 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v89) v1 ==== 93+0+0 (secure 0 0 0) 0x7f9d50050110 con 0x7f9d5c0cd4c0 2023-12-20T05:44:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 -- 172.21.15.32:0/2682894719 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d48065c00 msgr2=0x7f9d480680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:02.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d48065c00 0x7f9d480680c0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f9d4c009d40 tx=0x7f9d4c008040 comp rx=0 tx=0).stop 2023-12-20T05:44:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 -- 172.21.15.32:0/2682894719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5c0cd4c0 msgr2=0x7f9d5c1500a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5c0cd4c0 0x7f9d5c1500a0 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7f9d5000c3c0 tx=0x7f9d5000c880 comp rx=0 tx=0).stop 2023-12-20T05:44:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 -- 172.21.15.32:0/2682894719 shutdown_connections 2023-12-20T05:44:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9d48065c00 0x7f9d480680c0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d5c157480 0x7f9d5c15c1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d5c0cd4c0 0x7f9d5c1500a0 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 --2- 172.21.15.32:0/2682894719 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d5c0b0760 0x7f9d5c0b24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 -- 172.21.15.32:0/2682894719 >> 172.21.15.32:0/2682894719 conn(0x7f9d5c06a510 msgr2=0x7f9d5c0b3fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 -- 172.21.15.32:0/2682894719 shutdown_connections 2023-12-20T05:44:02.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.172+0000 7f9d62441640 1 -- 172.21.15.32:0/2682894719 wait complete. 2023-12-20T05:44:02.193 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T05:44:02.194 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T05:44:02.298 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T05:44:02.343 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:02.339+0000 7fab1391d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T05:44:02.352 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:02.356+0000 7ff784321180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T05:44:02.396 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T05:44:02.396 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:44:02.396 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:44:02.408 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:44:02.408 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:44:02.462 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:44:02.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91758f640 1 Processor -- start 2023-12-20T05:44:02.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91758f640 1 -- start start 2023-12-20T05:44:02.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91758f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9100af5e0 0x7fc9100af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:02.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91758f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc91006ce60 0x7fc91015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:02.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91758f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc91015bbd0 con 0x7fc91015f110 2023-12-20T05:44:02.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91758f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc91015beb0 con 0x7fc91006ce60 2023-12-20T05:44:02.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91758f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc91015e910 con 0x7fc9100af5e0 2023-12-20T05:44:02.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91532b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9100af5e0 0x7fc9100af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:02.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc914b2a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc91006ce60 0x7fc91015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:02.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc915b2c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fc91015f110 0x7fc91015c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:46924/0 (socket says 172.21.15.32:46924) 2023-12-20T05:44:02.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91532b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9100af5e0 0x7fc9100af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:41530/0 (socket says 172.21.15.32:41530) 2023-12-20T05:44:02.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc915b2c640 1 -- 172.21.15.32:0/4051576215 learned_addr learned my addr 172.21.15.32:0/4051576215 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:02.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91532b640 1 -- 172.21.15.32:0/4051576215 >> v2:172.21.15.39:3300/0 conn(0x7fc91006ce60 msgr2=0x7fc91015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:02.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91532b640 1 --2- 172.21.15.32:0/4051576215 >> v2:172.21.15.39:3300/0 conn(0x7fc91006ce60 0x7fc91015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91532b640 1 -- 172.21.15.32:0/4051576215 >> v1:172.21.15.39:6789/0 conn(0x7fc91015f110 legacy=0x7fc91015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:02.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc91532b640 1 -- 172.21.15.32:0/4051576215 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc91015ebf0 con 0x7fc9100af5e0 2023-12-20T05:44:02.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc914b2a640 1 --2- 172.21.15.32:0/4051576215 >> v2:172.21.15.39:3300/0 conn(0x7fc91006ce60 0x7fc91015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:02.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.656+0000 7fc907fff640 1 -- 172.21.15.32:0/4051576215 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3443369629 0 0) 0x7fc91015bbd0 con 0x7fc91015f110 2023-12-20T05:44:02.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc91532b640 1 --2- 172.21.15.32:0/4051576215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9100af5e0 0x7fc9100af9c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0097a0 tx=0x7fc90c01c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9db12e964010e84d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:02.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc907fff640 1 -- 172.21.15.32:0/4051576215 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c0ac620 con 0x7fc9100af5e0 2023-12-20T05:44:02.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc907fff640 1 -- 172.21.15.32:0/4051576215 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90c0ac780 con 0x7fc9100af5e0 2023-12-20T05:44:02.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc907fff640 1 -- 172.21.15.32:0/4051576215 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90c0acb10 con 0x7fc9100af5e0 2023-12-20T05:44:02.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc91758f640 1 -- 172.21.15.32:0/4051576215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9100af5e0 msgr2=0x7fc9100af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:02.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc91758f640 1 --2- 172.21.15.32:0/4051576215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9100af5e0 0x7fc9100af9c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fc90c0097a0 tx=0x7fc90c01c460 comp rx=0 tx=0).stop 2023-12-20T05:44:02.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc91758f640 1 -- 172.21.15.32:0/4051576215 shutdown_connections 2023-12-20T05:44:02.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc91758f640 1 --2- 172.21.15.32:0/4051576215 >> v2:172.21.15.39:3300/0 conn(0x7fc91006ce60 0x7fc91015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc91758f640 1 --2- 172.21.15.32:0/4051576215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9100af5e0 0x7fc9100af9c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc91758f640 1 -- 172.21.15.32:0/4051576215 >> 172.21.15.32:0/4051576215 conn(0x7fc91006a320 msgr2=0x7fc9100ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:02.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc91758f640 1 -- 172.21.15.32:0/4051576215 shutdown_connections 2023-12-20T05:44:02.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.660+0000 7fc91758f640 1 -- 172.21.15.32:0/4051576215 wait complete. 2023-12-20T05:44:02.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91758f640 1 Processor -- start 2023-12-20T05:44:02.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91758f640 1 -- start start 2023-12-20T05:44:02.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91758f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91006ce60 0x7fc91014f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91758f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9100af5e0 0x7fc910071ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91758f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc91015f110 0x7fc9100b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91758f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100b37a0 con 0x7fc9100af5e0 2023-12-20T05:44:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91758f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100b3a80 con 0x7fc91006ce60 2023-12-20T05:44:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91758f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9100b3d60 con 0x7fc91015f110 2023-12-20T05:44:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91532b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91006ce60 0x7fc91014f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc914b2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9100af5e0 0x7fc910071ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91532b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91006ce60 0x7fc91014f5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:45334/0 (socket says 172.21.15.32:45334) 2023-12-20T05:44:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91532b640 1 -- 172.21.15.32:0/2765885145 learned_addr learned my addr 172.21.15.32:0/2765885145 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc914b2a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9100af5e0 0x7fc910071ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53788/0 (socket says 172.21.15.32:53788) 2023-12-20T05:44:02.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc915b2c640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc91015f110 0x7fc9100b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc914b2a640 1 -- 172.21.15.32:0/2765885145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc91015f110 msgr2=0x7fc9100b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc914b2a640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc91015f110 0x7fc9100b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc914b2a640 1 -- 172.21.15.32:0/2765885145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91006ce60 msgr2=0x7fc91014f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc914b2a640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91006ce60 0x7fc91014f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc914b2a640 1 -- 172.21.15.32:0/2765885145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc900009420 con 0x7fc9100af5e0 2023-12-20T05:44:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc914b2a640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9100af5e0 0x7fc910071ab0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e980 tx=0x7fc90000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc905ffb640 1 -- 172.21.15.32:0/2765885145 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc90000ca00 con 0x7fc9100af5e0 2023-12-20T05:44:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc905ffb640 1 -- 172.21.15.32:0/2765885145 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc90000cb60 con 0x7fc9100af5e0 2023-12-20T05:44:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc905ffb640 1 -- 172.21.15.32:0/2765885145 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc900013470 con 0x7fc9100af5e0 2023-12-20T05:44:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91758f640 1 -- 172.21.15.32:0/2765885145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc90c0093b0 con 0x7fc9100af5e0 2023-12-20T05:44:02.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.664+0000 7fc91758f640 1 -- 172.21.15.32:0/2765885145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc91014faf0 con 0x7fc9100af5e0 2023-12-20T05:44:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.668+0000 7fc91758f640 1 -- 172.21.15.32:0/2765885145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc91004e000 con 0x7fc9100af5e0 2023-12-20T05:44:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.668+0000 7fc905ffb640 1 -- 172.21.15.32:0/2765885145 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 89) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fc9000135d0 con 0x7fc9100af5e0 2023-12-20T05:44:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.668+0000 7fc905ffb640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc8f4065b80 0x7fc8f4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.668+0000 7fc905ffb640 1 -- 172.21.15.32:0/2765885145 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fc900083b20 con 0x7fc9100af5e0 2023-12-20T05:44:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.668+0000 7fc91532b640 1 -- 172.21.15.32:0/2765885145 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc8f4065b80 msgr2=0x7fc8f4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:02.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.668+0000 7fc91532b640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc8f4065b80 0x7fc8f4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:02.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.672+0000 7fc905ffb640 1 -- 172.21.15.32:0/2765885145 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc900051260 con 0x7fc9100af5e0 2023-12-20T05:44:02.785 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:02.779+0000 7fab1391d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T05:44:02.808 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:02.812+0000 7ff784321180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T05:44:02.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.832+0000 7fc91758f640 1 -- 172.21.15.32:0/2765885145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc9101579a0 con 0x7fc9100af5e0 2023-12-20T05:44:02.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc905ffb640 1 -- 172.21.15.32:0/2765885145 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16596 (secure 0 0 0) 0x7fc90007be60 con 0x7fc9100af5e0 2023-12-20T05:44:02.832 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:02.832 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":160,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:43:59.101811+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:43:44.235578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-20T05:43:45.508783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3885704352":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/327030673":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/2048889983":"2023-12-21T05:43:59.101776+0000","172.21.15.32:6833/38009":"2023-12-21T05:43:59.101776+0000","172.21.15.32:6832/38009":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/3563338617":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/1355418179":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/616248093":"2023-12-21T05:43:59.101776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:44:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 -- 172.21.15.32:0/2765885145 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc8f4065b80 msgr2=0x7fc8f4068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc8f4065b80 0x7fc8f4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 -- 172.21.15.32:0/2765885145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9100af5e0 msgr2=0x7fc910071ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:02.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9100af5e0 0x7fc910071ab0 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7fc90000e980 tx=0x7fc90000ee40 comp rx=0 tx=0).stop 2023-12-20T05:44:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 -- 172.21.15.32:0/2765885145 shutdown_connections 2023-12-20T05:44:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc8f4065b80 0x7fc8f4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc91015f110 0x7fc9100b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9100af5e0 0x7fc910071ab0 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 --2- 172.21.15.32:0/2765885145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc91006ce60 0x7fc91014f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 -- 172.21.15.32:0/2765885145 >> 172.21.15.32:0/2765885145 conn(0x7fc91006a320 msgr2=0x7fc910156490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 -- 172.21.15.32:0/2765885145 shutdown_connections 2023-12-20T05:44:02.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:02.836+0000 7fc91758f640 1 -- 172.21.15.32:0/2765885145 wait complete. 2023-12-20T05:44:02.855 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T05:43:44.235578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T05:43:45.508783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T05:44:02.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T05:44:02.942 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:02.939+0000 7fab1391d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T05:44:02.971 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:02.972+0000 7ff784321180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T05:44:03.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb2b25640 1 Processor -- start 2023-12-20T05:44:03.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb2b25640 1 -- start start 2023-12-20T05:44:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb2b25640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cac157480 0x7f3cac0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb2b25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cac0b0760 0x7f3cac0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb2b25640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3cac15d660 0x7f3cac15daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb2b25640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac0b1100 con 0x7f3cac157480 2023-12-20T05:44:03.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb2b25640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cac0cde20 con 0x7f3cac15d660 2023-12-20T05:44:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb2b25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac0c8e30 con 0x7f3cac0b0760 2023-12-20T05:44:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb08c1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cac157480 0x7f3cac0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb08c1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cac157480 0x7f3cac0c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53802/0 (socket says 172.21.15.32:53802) 2023-12-20T05:44:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb08c1640 1 -- 172.21.15.32:0/1533981157 learned_addr learned my addr 172.21.15.32:0/1533981157 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cabfff640 1 --2- 172.21.15.32:0/1533981157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cac0b0760 0x7f3cac0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb10c2640 1 --2- 172.21.15.32:0/1533981157 >> v2:172.21.15.39:3300/0 conn(0x7f3cac15d660 0x7f3cac15daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb08c1640 1 -- 172.21.15.32:0/1533981157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cac0b0760 msgr2=0x7f3cac0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb08c1640 1 --2- 172.21.15.32:0/1533981157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cac0b0760 0x7f3cac0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb08c1640 1 -- 172.21.15.32:0/1533981157 >> v2:172.21.15.39:3300/0 conn(0x7f3cac15d660 msgr2=0x7f3cac15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb08c1640 1 --2- 172.21.15.32:0/1533981157 >> v2:172.21.15.39:3300/0 conn(0x7f3cac15d660 0x7f3cac15daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb08c1640 1 -- 172.21.15.32:0/1533981157 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cac0ce070 con 0x7f3cac157480 2023-12-20T05:44:03.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.008+0000 7f3cb10c2640 1 --2- 172.21.15.32:0/1533981157 >> v2:172.21.15.39:3300/0 conn(0x7f3cac15d660 0x7f3cac15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:03.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cb08c1640 1 --2- 172.21.15.32:0/1533981157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cac157480 0x7f3cac0c9840 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0b49b0 tx=0x7f3ca001c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d89d4d351c0820f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:03.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cab7fe640 1 -- 172.21.15.32:0/1533981157 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00ab600 con 0x7f3cac157480 2023-12-20T05:44:03.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cab7fe640 1 -- 172.21.15.32:0/1533981157 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca00ab760 con 0x7f3cac157480 2023-12-20T05:44:03.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cab7fe640 1 -- 172.21.15.32:0/1533981157 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca00abaf0 con 0x7f3cac157480 2023-12-20T05:44:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1533981157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cac157480 msgr2=0x7f3cac0c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cb2b25640 1 --2- 172.21.15.32:0/1533981157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cac157480 0x7f3cac0c9840 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f3cac0b49b0 tx=0x7f3ca001c440 comp rx=0 tx=0).stop 2023-12-20T05:44:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1533981157 shutdown_connections 2023-12-20T05:44:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cb2b25640 1 --2- 172.21.15.32:0/1533981157 >> v2:172.21.15.39:3300/0 conn(0x7f3cac15d660 0x7f3cac15daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cb2b25640 1 --2- 172.21.15.32:0/1533981157 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cac0b0760 0x7f3cac0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cb2b25640 1 --2- 172.21.15.32:0/1533981157 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cac157480 0x7f3cac0c9840 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1533981157 >> 172.21.15.32:0/1533981157 conn(0x7f3cac06a510 msgr2=0x7f3cac0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:03.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1533981157 shutdown_connections 2023-12-20T05:44:03.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.012+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1533981157 wait complete. 2023-12-20T05:44:03.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.016+0000 7f3cb2b25640 1 Processor -- start 2023-12-20T05:44:03.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.016+0000 7f3cb2b25640 1 -- start start 2023-12-20T05:44:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.016+0000 7f3cb2b25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cac0b0760 0x7f3cac1f8820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.016+0000 7f3cb2b25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cac157480 0x7f3cac1f8d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.016+0000 7f3cb2b25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cac15d660 0x7f3cac1521f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.016+0000 7f3cb2b25640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac158550 con 0x7f3cac15d660 2023-12-20T05:44:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.016+0000 7f3cb2b25640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac158830 con 0x7f3cac0b0760 2023-12-20T05:44:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.016+0000 7f3cb2b25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cac158b10 con 0x7f3cac157480 2023-12-20T05:44:03.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb10c2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cac15d660 0x7f3cac1521f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb08c1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cac0b0760 0x7f3cac1f8820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb10c2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cac15d660 0x7f3cac1521f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53812/0 (socket says 172.21.15.32:53812) 2023-12-20T05:44:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb10c2640 1 -- 172.21.15.32:0/1809175337 learned_addr learned my addr 172.21.15.32:0/1809175337 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb08c1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cac0b0760 0x7f3cac1f8820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:45358/0 (socket says 172.21.15.32:45358) 2023-12-20T05:44:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cabfff640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cac157480 0x7f3cac1f8d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb10c2640 1 -- 172.21.15.32:0/1809175337 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cac157480 msgr2=0x7f3cac1f8d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb10c2640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cac157480 0x7f3cac1f8d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb10c2640 1 -- 172.21.15.32:0/1809175337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cac0b0760 msgr2=0x7f3cac1f8820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb10c2640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cac0b0760 0x7f3cac1f8820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb10c2640 1 -- 172.21.15.32:0/1809175337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3ca00094d0 con 0x7f3cac15d660 2023-12-20T05:44:03.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb10c2640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cac15d660 0x7f3cac1521f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4007df0 tx=0x7f3ca400d4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:03.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3ca97fa640 1 -- 172.21.15.32:0/1809175337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca40039a0 con 0x7f3cac15d660 2023-12-20T05:44:03.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3ca97fa640 1 -- 172.21.15.32:0/1809175337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3ca400f040 con 0x7f3cac15d660 2023-12-20T05:44:03.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1809175337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cac158df0 con 0x7f3cac15d660 2023-12-20T05:44:03.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3ca97fa640 1 -- 172.21.15.32:0/1809175337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3ca4003d40 con 0x7f3cac15d660 2023-12-20T05:44:03.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1809175337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cac1f4fc0 con 0x7f3cac15d660 2023-12-20T05:44:03.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1809175337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cac159030 con 0x7f3cac15d660 2023-12-20T05:44:03.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.020+0000 7f3ca97fa640 1 -- 172.21.15.32:0/1809175337 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 89) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f3ca4004030 con 0x7f3cac15d660 2023-12-20T05:44:03.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.024+0000 7f3ca97fa640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3c90065b80 0x7f3c90068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.024+0000 7f3ca97fa640 1 -- 172.21.15.32:0/1809175337 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f3ca40837b0 con 0x7f3cac15d660 2023-12-20T05:44:03.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.024+0000 7f3cb08c1640 1 -- 172.21.15.32:0/1809175337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3c90065b80 msgr2=0x7f3c90068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:03.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.024+0000 7f3cb08c1640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3c90065b80 0x7f3c90068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:03.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.028+0000 7f3ca97fa640 1 -- 172.21.15.32:0/1809175337 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3cac159030 con 0x7f3cac15d660 2023-12-20T05:44:03.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1809175337 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f3cac159030 con 0x7f3cac15d660 2023-12-20T05:44:03.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3ca97fa640 1 -- 172.21.15.32:0/1809175337 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7f3cac159030 con 0x7f3cac15d660 2023-12-20T05:44:03.185 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T05:44:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1809175337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3c90065b80 msgr2=0x7f3c90068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:03.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3c90065b80 0x7f3c90068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1809175337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cac15d660 msgr2=0x7f3cac1521f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cac15d660 0x7f3cac1521f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f3ca4007df0 tx=0x7f3ca400d4a0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1809175337 shutdown_connections 2023-12-20T05:44:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3c90065b80 0x7f3c90068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cac15d660 0x7f3cac1521f0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cac157480 0x7f3cac1f8d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 --2- 172.21.15.32:0/1809175337 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cac0b0760 0x7f3cac1f8820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1809175337 >> 172.21.15.32:0/1809175337 conn(0x7f3cac06a510 msgr2=0x7f3cac162b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1809175337 shutdown_connections 2023-12-20T05:44:03.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.188+0000 7f3cb2b25640 1 -- 172.21.15.32:0/1809175337 wait complete. 2023-12-20T05:44:03.208 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T05:44:03.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.356+0000 7f6cf2e66640 1 Processor -- start 2023-12-20T05:44:03.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.356+0000 7f6cf2e66640 1 -- start start 2023-12-20T05:44:03.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6cf2e66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6cec070940 0x7f6cec0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6cf2e66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cec15b6a0 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6cf2e66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6cec06d8c0 con 0x7f6cec15abb0 2023-12-20T05:44:03.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6cf2e66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec06e160 con 0x7f6cec070940 2023-12-20T05:44:03.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6cebfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6cec0b0cb0 0x7f6cec070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50458/0 (socket says 172.21.15.32:50458) 2023-12-20T05:44:03.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6cebfff640 1 -- 172.21.15.32:0/3952107062 learned_addr learned my addr 172.21.15.32:0/3952107062 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:03.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6cf1403640 1 --2- 172.21.15.32:0/3952107062 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6cec070940 0x7f6cec0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6ceb7fe640 1 -- 172.21.15.32:0/3952107062 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1742114654 0 0) 0x7f6cec15b6a0 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6ceb7fe640 1 -- 172.21.15.32:0/3952107062 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6cdc003530 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6ceb7fe640 1 -- 172.21.15.32:0/3952107062 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1671198022 0 0) 0x7f6cec06d8c0 con 0x7f6cec15abb0 2023-12-20T05:44:03.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6cf1403640 1 -- 172.21.15.32:0/3952107062 >> v1:172.21.15.39:6789/0 conn(0x7f6cec15abb0 legacy=0x7f6cec15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6cf1403640 1 -- 172.21.15.32:0/3952107062 >> v1:172.21.15.32:6789/0 conn(0x7f6cec0b0cb0 legacy=0x7f6cec070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.360+0000 7f6cf1403640 1 -- 172.21.15.32:0/3952107062 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6cec06dba0 con 0x7f6cec070940 2023-12-20T05:44:03.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6cf1403640 1 --2- 172.21.15.32:0/3952107062 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6cec070940 0x7f6cec0caaf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009dc0 tx=0x7f6ce401c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae349413c2fb8804 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:03.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6ceb7fe640 1 -- 172.21.15.32:0/3952107062 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40a5a30 con 0x7f6cec070940 2023-12-20T05:44:03.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6ceb7fe640 1 -- 172.21.15.32:0/3952107062 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6ce40a5b90 con 0x7f6cec070940 2023-12-20T05:44:03.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6ceb7fe640 1 -- 172.21.15.32:0/3952107062 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6ce40ad440 con 0x7f6cec070940 2023-12-20T05:44:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6cf2e66640 1 -- 172.21.15.32:0/3952107062 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6cec070940 msgr2=0x7f6cec0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6cf2e66640 1 --2- 172.21.15.32:0/3952107062 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6cec070940 0x7f6cec0caaf0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f6ce4009dc0 tx=0x7f6ce401c970 comp rx=0 tx=0).stop 2023-12-20T05:44:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6cf2e66640 1 -- 172.21.15.32:0/3952107062 shutdown_connections 2023-12-20T05:44:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6cf2e66640 1 --2- 172.21.15.32:0/3952107062 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6cec070940 0x7f6cec0caaf0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6cf2e66640 1 -- 172.21.15.32:0/3952107062 >> 172.21.15.32:0/3952107062 conn(0x7f6cec06a510 msgr2=0x7f6cec0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:03.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6cf2e66640 1 -- 172.21.15.32:0/3952107062 shutdown_connections 2023-12-20T05:44:03.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6cf2e66640 1 -- 172.21.15.32:0/3952107062 wait complete. 2023-12-20T05:44:03.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.364+0000 7f6cf2e66640 1 Processor -- start 2023-12-20T05:44:03.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf2e66640 1 -- start start 2023-12-20T05:44:03.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf2e66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6cec070940 0x7f6cec159af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf2e66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0b0cb0 0x7f6cec0c83d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf2e66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cec15abb0 0x7f6cec0c7280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf2e66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec158a50 con 0x7f6cec15abb0 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf2e66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec158d30 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf2e66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6cec159010 con 0x7f6cec070940 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf1403640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cec15abb0 0x7f6cec0c7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf0c02640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6cec070940 0x7f6cec159af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf1403640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cec15abb0 0x7f6cec0c7280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53816/0 (socket says 172.21.15.32:53816) 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf1403640 1 -- 172.21.15.32:0/1379841797 learned_addr learned my addr 172.21.15.32:0/1379841797 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cebfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0b0cb0 0x7f6cec0c83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cebfff640 1 -- 172.21.15.32:0/1379841797 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6cec070940 msgr2=0x7f6cec159af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cebfff640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6cec070940 0x7f6cec159af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cebfff640 1 -- 172.21.15.32:0/1379841797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cec15abb0 msgr2=0x7f6cec0c7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cebfff640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cec15abb0 0x7f6cec0c7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cebfff640 1 -- 172.21.15.32:0/1379841797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6ce4009a70 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf1403640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cec15abb0 0x7f6cec0c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cebfff640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0b0cb0 0x7f6cec0c83d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0b1160 tx=0x7f6cd800c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6ce97fa640 1 -- 172.21.15.32:0/1379841797 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd8005cf0 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6ce97fa640 1 -- 172.21.15.32:0/1379841797 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6cd8005e50 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6ce97fa640 1 -- 172.21.15.32:0/1379841797 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6cd80072f0 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf2e66640 1 -- 172.21.15.32:0/1379841797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6cec1592f0 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf2e66640 1 -- 172.21.15.32:0/1379841797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6cec158720 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.368+0000 7f6cf2e66640 1 -- 172.21.15.32:0/1379841797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6cec159810 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.372+0000 7f6ce97fa640 1 -- 172.21.15.32:0/1379841797 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 89) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f6cd8007450 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.372+0000 7f6ce97fa640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6cd0065b80 0x7f6cd0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.372+0000 7f6ce97fa640 1 -- 172.21.15.32:0/1379841797 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f6cd80829c0 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.372+0000 7f6cf0c02640 1 -- 172.21.15.32:0/1379841797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6cd0065b80 msgr2=0x7f6cd0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:03.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.372+0000 7f6cf0c02640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6cd0065b80 0x7f6cd0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:03.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.376+0000 7f6ce97fa640 1 -- 172.21.15.32:0/1379841797 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6cec159810 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.452 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:03.447+0000 7fab1391d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T05:44:03.498 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:03.500+0000 7ff784321180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T05:44:03.513 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:03.507+0000 7fab1391d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T05:44:03.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.536+0000 7f6cf2e66640 1 -- 172.21.15.32:0/1379841797 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f6cec159810 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.536+0000 7f6ce97fa640 1 -- 172.21.15.32:0/1379841797 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v160) v1 ==== 104+0+11 (secure 0 0 0) 0x7f6cec159810 con 0x7f6cec0b0cb0 2023-12-20T05:44:03.533 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T05:44:03.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 -- 172.21.15.32:0/1379841797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6cd0065b80 msgr2=0x7f6cd0068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6cd0065b80 0x7f6cd0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 -- 172.21.15.32:0/1379841797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0b0cb0 msgr2=0x7f6cec0c83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0b0cb0 0x7f6cec0c83d0 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7f6cec0b1160 tx=0x7f6cd800c890 comp rx=0 tx=0).stop 2023-12-20T05:44:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 -- 172.21.15.32:0/1379841797 shutdown_connections 2023-12-20T05:44:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6cd0065b80 0x7f6cd0068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6cec15abb0 0x7f6cec0c7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6cec0b0cb0 0x7f6cec0c83d0 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 --2- 172.21.15.32:0/1379841797 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6cec070940 0x7f6cec159af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 -- 172.21.15.32:0/1379841797 >> 172.21.15.32:0/1379841797 conn(0x7f6cec06a510 msgr2=0x7f6cec0b29a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:03.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 -- 172.21.15.32:0/1379841797 shutdown_connections 2023-12-20T05:44:03.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.540+0000 7f6cf2e66640 1 -- 172.21.15.32:0/1379841797 wait complete. 2023-12-20T05:44:03.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T05:44:03.572 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:03.567+0000 7fab1391d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T05:44:03.572 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:03.576+0000 7ff784321180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T05:44:03.634 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:03.636+0000 7ff784321180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T05:44:03.663 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:03.659+0000 7fab1391d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T05:44:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.716+0000 7f79b74b6640 1 Processor -- start 2023-12-20T05:44:03.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.716+0000 7f79b74b6640 1 -- start start 2023-12-20T05:44:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b74b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79b00d3300 0x7f79b00d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b74b6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f79b00d1300 0x7f79b00b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b74b6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79b00d2650 con 0x7f79b015ce60 2023-12-20T05:44:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b74b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b00d1a40 con 0x7f79b00d3300 2023-12-20T05:44:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b74b6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79b00d1d20 con 0x7f79b00d1300 2023-12-20T05:44:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b4a51640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79b00d3300 0x7f79b00d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b4a51640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f79b00d3300 0x7f79b00d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53830/0 (socket says 172.21.15.32:53830) 2023-12-20T05:44:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b4a51640 1 -- 172.21.15.32:0/1998342667 learned_addr learned my addr 172.21.15.32:0/1998342667 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b5a53640 1 --2- 172.21.15.32:0/1998342667 >> v2:172.21.15.39:3300/0 conn(0x7f79b00d1300 0x7f79b00b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b4a51640 1 -- 172.21.15.32:0/1998342667 >> v2:172.21.15.39:3300/0 conn(0x7f79b00d1300 msgr2=0x7f79b00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:03.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b4a51640 1 --2- 172.21.15.32:0/1998342667 >> v2:172.21.15.39:3300/0 conn(0x7f79b00d1300 0x7f79b00b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b4a51640 1 -- 172.21.15.32:0/1998342667 >> v1:172.21.15.39:6789/0 conn(0x7f79b015ce60 legacy=0x7f79b00d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.720+0000 7f79b4a51640 1 -- 172.21.15.32:0/1998342667 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79b00d2000 con 0x7f79b00d3300 2023-12-20T05:44:03.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79b4a51640 1 --2- 172.21.15.32:0/1998342667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79b00d3300 0x7f79b00d0dc0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f79b00b5de0 tx=0x7f79a001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4911b3642deb5874 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:03.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79a7fff640 1 -- 172.21.15.32:0/1998342667 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00ac640 con 0x7f79b00d3300 2023-12-20T05:44:03.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79a7fff640 1 -- 172.21.15.32:0/1998342667 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a00ac7a0 con 0x7f79b00d3300 2023-12-20T05:44:03.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79a7fff640 1 -- 172.21.15.32:0/1998342667 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a00acb30 con 0x7f79b00d3300 2023-12-20T05:44:03.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79b74b6640 1 -- 172.21.15.32:0/1998342667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79b00d3300 msgr2=0x7f79b00d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79b74b6640 1 --2- 172.21.15.32:0/1998342667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79b00d3300 0x7f79b00d0dc0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f79b00b5de0 tx=0x7f79a001c500 comp rx=0 tx=0).stop 2023-12-20T05:44:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79b74b6640 1 -- 172.21.15.32:0/1998342667 shutdown_connections 2023-12-20T05:44:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79b74b6640 1 --2- 172.21.15.32:0/1998342667 >> v2:172.21.15.39:3300/0 conn(0x7f79b00d1300 0x7f79b00b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79b74b6640 1 --2- 172.21.15.32:0/1998342667 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79b00d3300 0x7f79b00d0dc0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79b74b6640 1 -- 172.21.15.32:0/1998342667 >> 172.21.15.32:0/1998342667 conn(0x7f79b006b5e0 msgr2=0x7f79b006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79b74b6640 1 -- 172.21.15.32:0/1998342667 shutdown_connections 2023-12-20T05:44:03.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79b74b6640 1 -- 172.21.15.32:0/1998342667 wait complete. 2023-12-20T05:44:03.724 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:03.719+0000 7fab1391d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T05:44:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79b74b6640 1 Processor -- start 2023-12-20T05:44:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.724+0000 7f79b74b6640 1 -- start start 2023-12-20T05:44:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b74b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f79b00d1300 0x7f79b0147fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b74b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79b00d3300 0x7f79b00b3f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b74b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79b015ce60 0x7f79b00b4440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b74b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00c6370 con 0x7f79b015ce60 2023-12-20T05:44:03.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b74b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79b0156e50 con 0x7f79b00d3300 2023-12-20T05:44:03.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b74b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79b00c9e40 con 0x7f79b00d1300 2023-12-20T05:44:03.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b5a53640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79b015ce60 0x7f79b00b4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b5a53640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79b015ce60 0x7f79b00b4440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53836/0 (socket says 172.21.15.32:53836) 2023-12-20T05:44:03.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b5a53640 1 -- 172.21.15.32:0/474561805 learned_addr learned my addr 172.21.15.32:0/474561805 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:03.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b5252640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f79b00d1300 0x7f79b0147fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b4a51640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79b00d3300 0x7f79b00b3f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:03.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b5a53640 1 -- 172.21.15.32:0/474561805 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f79b00d1300 msgr2=0x7f79b0147fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b5a53640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f79b00d1300 0x7f79b0147fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b5a53640 1 -- 172.21.15.32:0/474561805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79b00d3300 msgr2=0x7f79b00b3f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b5a53640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79b00d3300 0x7f79b00b3f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b5a53640 1 -- 172.21.15.32:0/474561805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79a00094d0 con 0x7f79b015ce60 2023-12-20T05:44:03.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b5a53640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79b015ce60 0x7f79b00b4440 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f79a800efc0 tx=0x7f79a800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:03.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79a5ffb640 1 -- 172.21.15.32:0/474561805 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a8009b10 con 0x7f79b015ce60 2023-12-20T05:44:03.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79a5ffb640 1 -- 172.21.15.32:0/474561805 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79a800f040 con 0x7f79b015ce60 2023-12-20T05:44:03.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b74b6640 1 -- 172.21.15.32:0/474561805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79b00ca0e0 con 0x7f79b015ce60 2023-12-20T05:44:03.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79a5ffb640 1 -- 172.21.15.32:0/474561805 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79a80146c0 con 0x7f79b015ce60 2023-12-20T05:44:03.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b74b6640 1 -- 172.21.15.32:0/474561805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79b0156ae0 con 0x7f79b015ce60 2023-12-20T05:44:03.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.728+0000 7f79b74b6640 1 -- 172.21.15.32:0/474561805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f79b004e000 con 0x7f79b015ce60 2023-12-20T05:44:03.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.732+0000 7f79a5ffb640 1 -- 172.21.15.32:0/474561805 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 89) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f79a8014820 con 0x7f79b015ce60 2023-12-20T05:44:03.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.732+0000 7f79a5ffb640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7994065b80 0x7f7994068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:03.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.732+0000 7f79b5252640 1 -- 172.21.15.32:0/474561805 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7994065b80 msgr2=0x7f7994068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:03.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.732+0000 7f79a5ffb640 1 -- 172.21.15.32:0/474561805 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f79a8084060 con 0x7f79b015ce60 2023-12-20T05:44:03.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.732+0000 7f79b5252640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7994065b80 0x7f7994068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:03.732 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:03.732+0000 7ff784321180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T05:44:03.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.736+0000 7f79a5ffb640 1 -- 172.21.15.32:0/474561805 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f79a80516b0 con 0x7f79b015ce60 2023-12-20T05:44:03.791 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:03.796+0000 7ff784321180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T05:44:03.811 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:03.807+0000 7fab1391d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T05:44:03.870 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:03.867+0000 7fab1391d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T05:44:03.881 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:03.884+0000 7ff784321180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T05:44:03.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.892+0000 7f79b74b6640 1 -- 172.21.15.32:0/474561805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f79b0156f90 con 0x7f79b015ce60 2023-12-20T05:44:03.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.892+0000 7f79a5ffb640 1 -- 172.21.15.32:0/474561805 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v160) v1 ==== 100+0+11 (secure 0 0 0) 0x7f79a8054cd0 con 0x7f79b015ce60 2023-12-20T05:44:03.892 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T05:44:03.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 -- 172.21.15.32:0/474561805 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7994065b80 msgr2=0x7f7994068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:03.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7994065b80 0x7f7994068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 -- 172.21.15.32:0/474561805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79b015ce60 msgr2=0x7f79b00b4440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:03.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79b015ce60 0x7f79b00b4440 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f79a800efc0 tx=0x7f79a800c490 comp rx=0 tx=0).stop 2023-12-20T05:44:03.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 -- 172.21.15.32:0/474561805 shutdown_connections 2023-12-20T05:44:03.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7994065b80 0x7f7994068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79b015ce60 0x7f79b00b4440 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f79b00d3300 0x7f79b00b3f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 --2- 172.21.15.32:0/474561805 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f79b00d1300 0x7f79b0147fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:03.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 -- 172.21.15.32:0/474561805 >> 172.21.15.32:0/474561805 conn(0x7f79b006b5e0 msgr2=0x7f79b0159270 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:03.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 -- 172.21.15.32:0/474561805 shutdown_connections 2023-12-20T05:44:03.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:03.896+0000 7f79b74b6640 1 -- 172.21.15.32:0/474561805 wait complete. 2023-12-20T05:44:03.914 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T05:44:03.943 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:03.944+0000 7ff784321180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T05:44:04.021 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:04.015+0000 7fab1391d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T05:44:04.082 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:04.079+0000 7fab1391d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T05:44:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d54a7640 1 Processor -- start 2023-12-20T05:44:04.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d54a7640 1 -- start start 2023-12-20T05:44:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d54a7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd4d00b8480 0x7fd4d006af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d54a7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4d006b460 0x7fd4d0160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d54a7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4d0161330 0x7fd4d0153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d54a7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d006b8c0 con 0x7fd4d0161330 2023-12-20T05:44:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d54a7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd4d00cd4d0 con 0x7fd4d00b8480 2023-12-20T05:44:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d54a7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d00ce4e0 con 0x7fd4d006b460 2023-12-20T05:44:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d4ca6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4d0161330 0x7fd4d0153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4cffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd4d00b8480 0x7fd4d006af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4cf7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4d006b460 0x7fd4d0160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d4ca6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd4d0161330 0x7fd4d0153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53842/0 (socket says 172.21.15.32:53842) 2023-12-20T05:44:04.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d4ca6640 1 -- 172.21.15.32:0/3981241317 learned_addr learned my addr 172.21.15.32:0/3981241317 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d4ca6640 1 -- 172.21.15.32:0/3981241317 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4d006b460 msgr2=0x7fd4d0160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d4ca6640 1 --2- 172.21.15.32:0/3981241317 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4d006b460 0x7fd4d0160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d4ca6640 1 -- 172.21.15.32:0/3981241317 >> v2:172.21.15.39:3300/0 conn(0x7fd4d00b8480 msgr2=0x7fd4d006af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d4ca6640 1 --2- 172.21.15.32:0/3981241317 >> v2:172.21.15.39:3300/0 conn(0x7fd4d00b8480 0x7fd4d006af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.092+0000 7fd4d4ca6640 1 -- 172.21.15.32:0/3981241317 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4d00cd720 con 0x7fd4d0161330 2023-12-20T05:44:04.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4d4ca6640 1 --2- 172.21.15.32:0/3981241317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4d0161330 0x7fd4d0153610 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd4d006bb70 tx=0x7fd4c801c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d22a185e4aa6cee2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:04.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4ceffd640 1 -- 172.21.15.32:0/3981241317 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80a5740 con 0x7fd4d0161330 2023-12-20T05:44:04.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4ceffd640 1 -- 172.21.15.32:0/3981241317 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c80a58a0 con 0x7fd4d0161330 2023-12-20T05:44:04.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4ceffd640 1 -- 172.21.15.32:0/3981241317 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80a5c30 con 0x7fd4d0161330 2023-12-20T05:44:04.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3981241317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4d0161330 msgr2=0x7fd4d0153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:04.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4d54a7640 1 --2- 172.21.15.32:0/3981241317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4d0161330 0x7fd4d0153610 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7fd4d006bb70 tx=0x7fd4c801c9d0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3981241317 shutdown_connections 2023-12-20T05:44:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4d54a7640 1 --2- 172.21.15.32:0/3981241317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4d0161330 0x7fd4d0153610 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4d54a7640 1 --2- 172.21.15.32:0/3981241317 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4d006b460 0x7fd4d0160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4d54a7640 1 --2- 172.21.15.32:0/3981241317 >> v2:172.21.15.39:3300/0 conn(0x7fd4d00b8480 0x7fd4d006af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3981241317 >> 172.21.15.32:0/3981241317 conn(0x7fd4d00d2a50 msgr2=0x7fd4d00d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3981241317 shutdown_connections 2023-12-20T05:44:04.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.096+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3981241317 wait complete. 2023-12-20T05:44:04.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d54a7640 1 Processor -- start 2023-12-20T05:44:04.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d54a7640 1 -- start start 2023-12-20T05:44:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d54a7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4d006b460 0x7fd4d014efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d54a7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d00b8480 0x7fd4d0156090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d54a7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4d0161330 0x7fd4d01565d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d54a7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d01501d0 con 0x7fd4d0161330 2023-12-20T05:44:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d54a7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d014f930 con 0x7fd4d00b8480 2023-12-20T05:44:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d54a7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4d014fc10 con 0x7fd4d006b460 2023-12-20T05:44:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d4ca6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4d0161330 0x7fd4d01565d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4cf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d00b8480 0x7fd4d0156090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d4ca6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4d0161330 0x7fd4d01565d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53856/0 (socket says 172.21.15.32:53856) 2023-12-20T05:44:04.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d4ca6640 1 -- 172.21.15.32:0/3294951779 learned_addr learned my addr 172.21.15.32:0/3294951779 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4cffff640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4d006b460 0x7fd4d014efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d4ca6640 1 -- 172.21.15.32:0/3294951779 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4d006b460 msgr2=0x7fd4d014efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d4ca6640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4d006b460 0x7fd4d014efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d4ca6640 1 -- 172.21.15.32:0/3294951779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d00b8480 msgr2=0x7fd4d0156090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d4ca6640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d00b8480 0x7fd4d0156090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d4ca6640 1 -- 172.21.15.32:0/3294951779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd4c8009b30 con 0x7fd4d0161330 2023-12-20T05:44:04.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4cf7fe640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d00b8480 0x7fd4d0156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d4ca6640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4d0161330 0x7fd4d01565d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80af040 tx=0x7fd4c8019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4ccff9640 1 -- 172.21.15.32:0/3294951779 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80a5740 con 0x7fd4d0161330 2023-12-20T05:44:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4ccff9640 1 -- 172.21.15.32:0/3294951779 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd4c80b8690 con 0x7fd4d0161330 2023-12-20T05:44:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4ccff9640 1 -- 172.21.15.32:0/3294951779 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd4c80b8a40 con 0x7fd4d0161330 2023-12-20T05:44:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3294951779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4d014fef0 con 0x7fd4d0161330 2023-12-20T05:44:04.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.100+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3294951779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd4d014f520 con 0x7fd4d0161330 2023-12-20T05:44:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.104+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3294951779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd4d0074100 con 0x7fd4d0161330 2023-12-20T05:44:04.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.104+0000 7fd4ccff9640 1 -- 172.21.15.32:0/3294951779 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 89) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fd4c80b8ba0 con 0x7fd4d0161330 2023-12-20T05:44:04.103 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:04.104+0000 7ff784321180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T05:44:04.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.104+0000 7fd4ccff9640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd4bc065b80 0x7fd4bc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.104+0000 7fd4ccff9640 1 -- 172.21.15.32:0/3294951779 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fd4c811e8d0 con 0x7fd4d0161330 2023-12-20T05:44:04.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.104+0000 7fd4cffff640 1 -- 172.21.15.32:0/3294951779 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd4bc065b80 msgr2=0x7fd4bc068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:04.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.104+0000 7fd4cffff640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd4bc065b80 0x7fd4bc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:04.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.108+0000 7fd4ccff9640 1 -- 172.21.15.32:0/3294951779 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd4c80ec130 con 0x7fd4d0161330 2023-12-20T05:44:04.169 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:04.172+0000 7ff784321180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T05:44:04.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.280+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3294951779 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fd4d00c1460 con 0x7fd4d0161330 2023-12-20T05:44:04.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.280+0000 7fd4ccff9640 1 -- 172.21.15.32:0/3294951779 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v160) v1 ==== 80+0+458 (secure 0 0 0) 0x7fd4c80ef750 con 0x7fd4d0161330 2023-12-20T05:44:04.278 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3885704352 2023-12-21T05:43:59.101776+0000 2023-12-20T05:44:04.278 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/327030673 2023-12-21T05:43:59.101776+0000 2023-12-20T05:44:04.278 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2048889983 2023-12-21T05:43:59.101776+0000 2023-12-20T05:44:04.279 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/38009 2023-12-21T05:43:59.101776+0000 2023-12-20T05:44:04.279 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/38009 2023-12-21T05:43:59.101776+0000 2023-12-20T05:44:04.279 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3563338617 2023-12-21T05:43:59.101776+0000 2023-12-20T05:44:04.279 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1355418179 2023-12-21T05:43:59.101776+0000 2023-12-20T05:44:04.279 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/616248093 2023-12-21T05:43:59.101776+0000 2023-12-20T05:44:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:listed 8 entries 2023-12-20T05:44:04.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3294951779 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd4bc065b80 msgr2=0x7fd4bc068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd4bc065b80 0x7fd4bc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3294951779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4d0161330 msgr2=0x7fd4d01565d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4d0161330 0x7fd4d01565d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7fd4c80af040 tx=0x7fd4c8019770 comp rx=0 tx=0).stop 2023-12-20T05:44:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3294951779 shutdown_connections 2023-12-20T05:44:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd4bc065b80 0x7fd4bc068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd4d0161330 0x7fd4d01565d0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd4d00b8480 0x7fd4d0156090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 --2- 172.21.15.32:0/3294951779 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4d006b460 0x7fd4d014efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3294951779 >> 172.21.15.32:0/3294951779 conn(0x7fd4d00d2a50 msgr2=0x7fd4d00b71c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3294951779 shutdown_connections 2023-12-20T05:44:04.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.284+0000 7fd4d54a7640 1 -- 172.21.15.32:0/3294951779 wait complete. 2023-12-20T05:44:04.300 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T05:44:04.301 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:04.301 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T05:44:04.336 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T05:44:04.336 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T05:44:04.336 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T05:44:04.343 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T05:44:04.343 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:04.343 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T05:44:04.345 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:04.339+0000 7fab1391d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T05:44:04.398 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T05:44:04.399 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:44:04.409 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T05:44:04.409 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:44:04.409 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:44:04.426 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T05:44:04.426 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T05:44:04.426 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T05:44:04.438 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T05:44:04.438 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T05:44:04.438 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:44:04.438 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:04.440+0000 7ff784321180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T05:44:04.458 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T05:44:04.459 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:44:04.488 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:04.483+0000 7fab1391d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:44:04.584 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:04.588+0000 7ff784321180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:44:04.621 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:04.615+0000 7fab1391d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T05:44:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.628+0000 7f5305325640 1 Processor -- start 2023-12-20T05:44:04.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.628+0000 7f5305325640 1 -- start start 2023-12-20T05:44:04.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.628+0000 7f5305325640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f530015cdb0 0x7f53000d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.628+0000 7f5305325640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53000cfc80 0x7f53000b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.628+0000 7f5305325640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f530015d190 con 0x7f53000d2740 2023-12-20T05:44:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.628+0000 7f5305325640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53000d16a0 con 0x7f53000cfc80 2023-12-20T05:44:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.628+0000 7f5305325640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53000d1980 con 0x7f530015cdb0 2023-12-20T05:44:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.628+0000 7f52ff7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53000cfc80 0x7f53000b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.628+0000 7f52feffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f530015cdb0 0x7f53000d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.628+0000 7f52ff7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f53000cfc80 0x7f53000b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53872/0 (socket says 172.21.15.32:53872) 2023-12-20T05:44:04.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.628+0000 7f52ff7fe640 1 -- 172.21.15.32:0/3602135474 learned_addr learned my addr 172.21.15.32:0/3602135474 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.632+0000 7f52feffd640 1 -- 172.21.15.32:0/3602135474 >> v2:172.21.15.32:3300/0 conn(0x7f53000cfc80 msgr2=0x7f53000b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.632+0000 7f52feffd640 1 --2- 172.21.15.32:0/3602135474 >> v2:172.21.15.32:3300/0 conn(0x7f53000cfc80 0x7f53000b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.632+0000 7f52feffd640 1 -- 172.21.15.32:0/3602135474 >> v1:172.21.15.39:6789/0 conn(0x7f53000d2740 legacy=0x7f53000cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:04.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.632+0000 7f52feffd640 1 -- 172.21.15.32:0/3602135474 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53000d1c60 con 0x7f530015cdb0 2023-12-20T05:44:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.632+0000 7f52feffd640 1 --2- 172.21.15.32:0/3602135474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f530015cdb0 0x7f53000d0be0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f53000667b0 tx=0x7f52f001ec50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=44182a1a8c89079b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.632+0000 7f52fdffb640 1 -- 172.21.15.32:0/3602135474 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00a8950 con 0x7f530015cdb0 2023-12-20T05:44:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.632+0000 7f52fdffb640 1 -- 172.21.15.32:0/3602135474 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f00a8ab0 con 0x7f530015cdb0 2023-12-20T05:44:04.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.632+0000 7f52fdffb640 1 -- 172.21.15.32:0/3602135474 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00a8e40 con 0x7f530015cdb0 2023-12-20T05:44:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 -- 172.21.15.32:0/3602135474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f530015cdb0 msgr2=0x7f53000d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 --2- 172.21.15.32:0/3602135474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f530015cdb0 0x7f53000d0be0 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7f53000667b0 tx=0x7f52f001ec50 comp rx=0 tx=0).stop 2023-12-20T05:44:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 -- 172.21.15.32:0/3602135474 shutdown_connections 2023-12-20T05:44:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 --2- 172.21.15.32:0/3602135474 >> v2:172.21.15.32:3300/0 conn(0x7f53000cfc80 0x7f53000b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 --2- 172.21.15.32:0/3602135474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f530015cdb0 0x7f53000d0be0 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 -- 172.21.15.32:0/3602135474 >> 172.21.15.32:0/3602135474 conn(0x7f5300065ee0 msgr2=0x7f530006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:04.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 -- 172.21.15.32:0/3602135474 shutdown_connections 2023-12-20T05:44:04.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 -- 172.21.15.32:0/3602135474 wait complete. 2023-12-20T05:44:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 Processor -- start 2023-12-20T05:44:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 -- start start 2023-12-20T05:44:04.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53000cfc80 0x7f53000c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53000d2740 0x7f5300158460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530015cdb0 0x7f53001589a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53001563f0 con 0x7f530015cdb0 2023-12-20T05:44:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5300155ae0 con 0x7f53000d2740 2023-12-20T05:44:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f5305325640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5300160100 con 0x7f53000cfc80 2023-12-20T05:44:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f52ff7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530015cdb0 0x7f53001589a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:04.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f52feffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53000cfc80 0x7f53000c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f52fe7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53000d2740 0x7f5300158460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f52feffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53000cfc80 0x7f53000c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:41626/0 (socket says 172.21.15.32:41626) 2023-12-20T05:44:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f52ff7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530015cdb0 0x7f53001589a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53882/0 (socket says 172.21.15.32:53882) 2023-12-20T05:44:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f52feffd640 1 -- 172.21.15.32:0/236803435 learned_addr learned my addr 172.21.15.32:0/236803435 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f52feffd640 1 -- 172.21.15.32:0/236803435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53000d2740 msgr2=0x7f5300158460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f52feffd640 1 --2- 172.21.15.32:0/236803435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53000d2740 0x7f5300158460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f52feffd640 1 -- 172.21.15.32:0/236803435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530015cdb0 msgr2=0x7f53001589a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f52feffd640 1 --2- 172.21.15.32:0/236803435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530015cdb0 0x7f53001589a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.636+0000 7f52feffd640 1 -- 172.21.15.32:0/236803435 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52f0009420 con 0x7f53000cfc80 2023-12-20T05:44:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f52feffd640 1 --2- 172.21.15.32:0/236803435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53000cfc80 0x7f53000c85e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f52f00b3040 tx=0x7f52f001b9f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f52e77fe640 1 -- 172.21.15.32:0/236803435 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00b0dd0 con 0x7f53000cfc80 2023-12-20T05:44:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f52e77fe640 1 -- 172.21.15.32:0/236803435 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52f00b9430 con 0x7f53000cfc80 2023-12-20T05:44:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f52e77fe640 1 -- 172.21.15.32:0/236803435 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52f00b97e0 con 0x7f53000cfc80 2023-12-20T05:44:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f5305325640 1 -- 172.21.15.32:0/236803435 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5300155d30 con 0x7f53000cfc80 2023-12-20T05:44:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f5305325640 1 -- 172.21.15.32:0/236803435 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53000c8b20 con 0x7f53000cfc80 2023-12-20T05:44:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f5305325640 1 -- 172.21.15.32:0/236803435 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5300156530 con 0x7f53000cfc80 2023-12-20T05:44:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f52e77fe640 1 -- 172.21.15.32:0/236803435 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 89) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f52f00b7040 con 0x7f53000cfc80 2023-12-20T05:44:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f52e77fe640 1 --2- 172.21.15.32:0/236803435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52ec065b80 0x7f52ec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f52e77fe640 1 -- 172.21.15.32:0/236803435 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f52f0120e20 con 0x7f53000cfc80 2023-12-20T05:44:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f52fe7fc640 1 -- 172.21.15.32:0/236803435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52ec065b80 msgr2=0x7f52ec068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.640+0000 7f52fe7fc640 1 --2- 172.21.15.32:0/236803435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52ec065b80 0x7f52ec068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:04.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.644+0000 7f52e77fe640 1 -- 172.21.15.32:0/236803435 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5300156530 con 0x7f53000cfc80 2023-12-20T05:44:04.697 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:04.691+0000 7fab1391d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T05:44:04.719 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:04.720+0000 7ff784321180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T05:44:04.794 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:04.796+0000 7ff784321180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T05:44:04.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.836+0000 7f5305325640 1 -- 172.21.15.32:0/236803435 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5300156530 con 0x7f53000cfc80 2023-12-20T05:44:04.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.836+0000 7f52e77fe640 1 -- 172.21.15.32:0/236803435 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f5300156530 con 0x7f53000cfc80 2023-12-20T05:44:04.835 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:04.835 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2062259714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2062259714},{"type":"v1","addr":"172.21.15.39:6834","nonce":2062259714}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4210073096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":4210073096},{"type":"v1","addr":"172.21.15.39:6835","nonce":4210073096}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8913,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3547769022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3547769022},{"type":"v1","addr":"172.21.15.32:6835","nonce":3547769022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:43:46.505246+0000","modified":"2023-12-20T05:43:49.528294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8928":{"gid":8928,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3239941411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3239941411},{"type":"v1","addr":"172.21.15.32:6837","nonce":3239941411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-20T05:44:04.835 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 71 2023-12-20T05:44:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 -- 172.21.15.32:0/236803435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52ec065b80 msgr2=0x7f52ec068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 --2- 172.21.15.32:0/236803435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52ec065b80 0x7f52ec068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 -- 172.21.15.32:0/236803435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53000cfc80 msgr2=0x7f53000c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 --2- 172.21.15.32:0/236803435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53000cfc80 0x7f53000c85e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f52f00b3040 tx=0x7f52f001b9f0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 -- 172.21.15.32:0/236803435 shutdown_connections 2023-12-20T05:44:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 --2- 172.21.15.32:0/236803435 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f52ec065b80 0x7f52ec068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 --2- 172.21.15.32:0/236803435 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f530015cdb0 0x7f53001589a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 --2- 172.21.15.32:0/236803435 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53000d2740 0x7f5300158460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 --2- 172.21.15.32:0/236803435 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53000cfc80 0x7f53000c85e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 -- 172.21.15.32:0/236803435 >> 172.21.15.32:0/236803435 conn(0x7f5300065ee0 msgr2=0x7f530015b650 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 -- 172.21.15.32:0/236803435 shutdown_connections 2023-12-20T05:44:04.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:04.840+0000 7f5305325640 1 -- 172.21.15.32:0/236803435 wait complete. 2023-12-20T05:44:04.840 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:04.835+0000 7fab1391d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:44:04.857 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:44:04.943 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:04.944+0000 7ff784321180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:44:05.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.004+0000 7f09c962d640 1 Processor -- start 2023-12-20T05:44:05.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.004+0000 7f09c962d640 1 -- start start 2023-12-20T05:44:05.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c962d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09c41593c0 0x7f09c415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c962d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09c40afdb0 0x7f09c406c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c962d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09c40ae140 con 0x7f09c406c920 2023-12-20T05:44:05.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c962d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09c40ae9e0 con 0x7f09c40afdb0 2023-12-20T05:44:05.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c962d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c406cdb0 con 0x7f09c41593c0 2023-12-20T05:44:05.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09bbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09c40afdb0 0x7f09c406c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09bbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f09c40afdb0 0x7f09c406c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53888/0 (socket says 172.21.15.32:53888) 2023-12-20T05:44:05.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c2ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09c41593c0 0x7f09c415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09bbfff640 1 -- 172.21.15.32:0/1331409237 learned_addr learned my addr 172.21.15.32:0/1331409237 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:05.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c37fe640 1 --1- 172.21.15.32:0/1331409237 >> v1:172.21.15.39:6789/0 conn(0x7f09c406c920 0x7f09c40705a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:46970/0 (socket says 172.21.15.32:46970) 2023-12-20T05:44:05.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c2ffd640 1 -- 172.21.15.32:0/1331409237 >> v2:172.21.15.32:3300/0 conn(0x7f09c40afdb0 msgr2=0x7f09c406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:05.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c2ffd640 1 --2- 172.21.15.32:0/1331409237 >> v2:172.21.15.32:3300/0 conn(0x7f09c40afdb0 0x7f09c406c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c27fc640 1 -- 172.21.15.32:0/1331409237 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2348945652 0 0) 0x7f09c40ae140 con 0x7f09c406c920 2023-12-20T05:44:05.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c2ffd640 1 -- 172.21.15.32:0/1331409237 >> v1:172.21.15.39:6789/0 conn(0x7f09c406c920 legacy=0x7f09c40705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09c2ffd640 1 -- 172.21.15.32:0/1331409237 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09c415dee0 con 0x7f09c41593c0 2023-12-20T05:44:05.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.008+0000 7f09bbfff640 1 --2- 172.21.15.32:0/1331409237 >> v2:172.21.15.32:3300/0 conn(0x7f09c40afdb0 0x7f09c406c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:05.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c2ffd640 1 --2- 172.21.15.32:0/1331409237 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09c41593c0 0x7f09c415d9a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f09b40097a0 tx=0x7f09b401c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ae3d62f8c766c6e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:05.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c27fc640 1 -- 172.21.15.32:0/1331409237 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b40a4ea0 con 0x7f09c41593c0 2023-12-20T05:44:05.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c27fc640 1 -- 172.21.15.32:0/1331409237 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09b40a7050 con 0x7f09c41593c0 2023-12-20T05:44:05.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c27fc640 1 -- 172.21.15.32:0/1331409237 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09b40ab820 con 0x7f09c41593c0 2023-12-20T05:44:05.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c962d640 1 -- 172.21.15.32:0/1331409237 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09c41593c0 msgr2=0x7f09c415d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c962d640 1 --2- 172.21.15.32:0/1331409237 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09c41593c0 0x7f09c415d9a0 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f09b40097a0 tx=0x7f09b401c3a0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c962d640 1 -- 172.21.15.32:0/1331409237 shutdown_connections 2023-12-20T05:44:05.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c962d640 1 --2- 172.21.15.32:0/1331409237 >> v2:172.21.15.32:3300/0 conn(0x7f09c40afdb0 0x7f09c406c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c962d640 1 --2- 172.21.15.32:0/1331409237 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09c41593c0 0x7f09c415d9a0 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c962d640 1 -- 172.21.15.32:0/1331409237 >> 172.21.15.32:0/1331409237 conn(0x7f09c406a320 msgr2=0x7f09c40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:05.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c962d640 1 -- 172.21.15.32:0/1331409237 shutdown_connections 2023-12-20T05:44:05.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c962d640 1 -- 172.21.15.32:0/1331409237 wait complete. 2023-12-20T05:44:05.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c962d640 1 Processor -- start 2023-12-20T05:44:05.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.012+0000 7f09c962d640 1 -- start start 2023-12-20T05:44:05.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09c962d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c406c920 0x7f09c41409d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09c962d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c40afdb0 0x7f09c40c8080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09c962d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09c41593c0 0x7f09c40acb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09c962d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40ce240 con 0x7f09c40afdb0 2023-12-20T05:44:05.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09c962d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09c40ce520 con 0x7f09c406c920 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09c962d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09c415b4c0 con 0x7f09c41593c0 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c40afdb0 0x7f09c40c8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c40afdb0 0x7f09c40c8080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53892/0 (socket says 172.21.15.32:53892) 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bbfff640 1 -- 172.21.15.32:0/2641431490 learned_addr learned my addr 172.21.15.32:0/2641431490 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09c2ffd640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c406c920 0x7f09c41409d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09c37fe640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09c41593c0 0x7f09c40acb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bbfff640 1 -- 172.21.15.32:0/2641431490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09c41593c0 msgr2=0x7f09c40acb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bbfff640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09c41593c0 0x7f09c40acb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bbfff640 1 -- 172.21.15.32:0/2641431490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c406c920 msgr2=0x7f09c41409d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bbfff640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c406c920 0x7f09c41409d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bbfff640 1 -- 172.21.15.32:0/2641431490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ac0094d0 con 0x7f09c40afdb0 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bbfff640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c40afdb0 0x7f09c40c8080 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00e9d0 tx=0x7f09ac00ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:05.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bb7fe640 1 -- 172.21.15.32:0/2641431490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac00ca60 con 0x7f09c40afdb0 2023-12-20T05:44:05.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bb7fe640 1 -- 172.21.15.32:0/2641431490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ac00cbc0 con 0x7f09c40afdb0 2023-12-20T05:44:05.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09bb7fe640 1 -- 172.21.15.32:0/2641431490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ac013470 con 0x7f09c40afdb0 2023-12-20T05:44:05.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09c962d640 1 -- 172.21.15.32:0/2641431490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09b40093b0 con 0x7f09c40afdb0 2023-12-20T05:44:05.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09c962d640 1 -- 172.21.15.32:0/2641431490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09c415b170 con 0x7f09c40afdb0 2023-12-20T05:44:05.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.016+0000 7f09c962d640 1 -- 172.21.15.32:0/2641431490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09c404e000 con 0x7f09c40afdb0 2023-12-20T05:44:05.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.020+0000 7f09bb7fe640 1 -- 172.21.15.32:0/2641431490 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 89) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f09ac0135d0 con 0x7f09c40afdb0 2023-12-20T05:44:05.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.020+0000 7f09bb7fe640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09a4065b80 0x7f09a4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.020+0000 7f09bb7fe640 1 -- 172.21.15.32:0/2641431490 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f09ac0836f0 con 0x7f09c40afdb0 2023-12-20T05:44:05.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.020+0000 7f09c2ffd640 1 -- 172.21.15.32:0/2641431490 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09a4065b80 msgr2=0x7f09a4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:05.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.020+0000 7f09c2ffd640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09a4065b80 0x7f09a4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:05.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.024+0000 7f09bb7fe640 1 -- 172.21.15.32:0/2641431490 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f09ac050ec0 con 0x7f09c40afdb0 2023-12-20T05:44:05.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.184+0000 7f09c962d640 1 -- 172.21.15.32:0/2641431490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f09c40addf0 con 0x7f09c40afdb0 2023-12-20T05:44:05.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.184+0000 7f09bb7fe640 1 -- 172.21.15.32:0/2641431490 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v160) v1 ==== 74+0+16596 (secure 0 0 0) 0x7f09ac0544e0 con 0x7f09c40afdb0 2023-12-20T05:44:05.183 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:05.183 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":160,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:43:59.101811+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:43:44.235578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-20T05:43:45.508783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3885704352":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/327030673":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/2048889983":"2023-12-21T05:43:59.101776+0000","172.21.15.32:6833/38009":"2023-12-21T05:43:59.101776+0000","172.21.15.32:6832/38009":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/3563338617":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/1355418179":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/616248093":"2023-12-21T05:43:59.101776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 -- 172.21.15.32:0/2641431490 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09a4065b80 msgr2=0x7f09a4068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09a4065b80 0x7f09a4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 -- 172.21.15.32:0/2641431490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c40afdb0 msgr2=0x7f09c40c8080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c40afdb0 0x7f09c40c8080 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x7f09ac00e9d0 tx=0x7f09ac00ee90 comp rx=0 tx=0).stop 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 -- 172.21.15.32:0/2641431490 shutdown_connections 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f09a4065b80 0x7f09a4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09c41593c0 0x7f09c40acb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09c40afdb0 0x7f09c40c8080 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 --2- 172.21.15.32:0/2641431490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09c406c920 0x7f09c41409d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 -- 172.21.15.32:0/2641431490 >> 172.21.15.32:0/2641431490 conn(0x7f09c406a320 msgr2=0x7f09c4157980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 -- 172.21.15.32:0/2641431490 shutdown_connections 2023-12-20T05:44:05.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.188+0000 7f09c962d640 1 -- 172.21.15.32:0/2641431490 wait complete. 2023-12-20T05:44:05.206 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 18, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T05:43:44.235578+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '158', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 19, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T05:43:45.508783+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '159', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T05:44:05.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T05:44:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f586c211640 1 Processor -- start 2023-12-20T05:44:05.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f586c211640 1 -- start start 2023-12-20T05:44:05.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f586c211640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5864164670 0x7f58640ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f586c211640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f586406da20 0x7f58640cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f586c211640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58640cb0b0 con 0x7f5864070230 2023-12-20T05:44:05.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f586c211640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58640ca810 con 0x7f5864164670 2023-12-20T05:44:05.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f586c211640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58640caaf0 con 0x7f586406da20 2023-12-20T05:44:05.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f5869fad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5864164670 0x7f58640ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f586a7ae640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5864070230 0x7f586406ca30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50464/0 (socket says 172.21.15.32:50464) 2023-12-20T05:44:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f5869fad640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5864164670 0x7f58640ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53902/0 (socket says 172.21.15.32:53902) 2023-12-20T05:44:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f586a7ae640 1 -- 172.21.15.32:0/2487546639 learned_addr learned my addr 172.21.15.32:0/2487546639 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f58697ac640 1 --2- 172.21.15.32:0/2487546639 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f586406da20 0x7f58640cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f5869fad640 1 -- 172.21.15.32:0/2487546639 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f586406da20 msgr2=0x7f58640cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f5869fad640 1 --2- 172.21.15.32:0/2487546639 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f586406da20 0x7f58640cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f5869fad640 1 -- 172.21.15.32:0/2487546639 >> v1:172.21.15.32:6789/0 conn(0x7f5864070230 legacy=0x7f586406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.360+0000 7f5869fad640 1 -- 172.21.15.32:0/2487546639 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58640cadd0 con 0x7f5864164670 2023-12-20T05:44:05.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f5869fad640 1 --2- 172.21.15.32:0/2487546639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5864164670 0x7f58640ca1a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f58600097c0 tx=0x7f586001c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1dd790d5ef6e852 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:05.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f5868fab640 1 -- 172.21.15.32:0/2487546639 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58600ac640 con 0x7f5864164670 2023-12-20T05:44:05.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f5868fab640 1 -- 172.21.15.32:0/2487546639 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58600ac7a0 con 0x7f5864164670 2023-12-20T05:44:05.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f5868fab640 1 -- 172.21.15.32:0/2487546639 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58600acb30 con 0x7f5864164670 2023-12-20T05:44:05.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f586c211640 1 -- 172.21.15.32:0/2487546639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5864164670 msgr2=0x7f58640ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f586c211640 1 --2- 172.21.15.32:0/2487546639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5864164670 0x7f58640ca1a0 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f58600097c0 tx=0x7f586001c330 comp rx=0 tx=0).stop 2023-12-20T05:44:05.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f586c211640 1 -- 172.21.15.32:0/2487546639 shutdown_connections 2023-12-20T05:44:05.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f586c211640 1 --2- 172.21.15.32:0/2487546639 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f586406da20 0x7f58640cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f586c211640 1 --2- 172.21.15.32:0/2487546639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5864164670 0x7f58640ca1a0 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f586c211640 1 -- 172.21.15.32:0/2487546639 >> 172.21.15.32:0/2487546639 conn(0x7f586406a510 msgr2=0x7f58640ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:05.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f586c211640 1 -- 172.21.15.32:0/2487546639 shutdown_connections 2023-12-20T05:44:05.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.364+0000 7f586c211640 1 -- 172.21.15.32:0/2487546639 wait complete. 2023-12-20T05:44:05.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586c211640 1 Processor -- start 2023-12-20T05:44:05.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586c211640 1 -- start start 2023-12-20T05:44:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586c211640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f586406da20 0x7f58641511d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586c211640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5864070230 0x7f58640c40b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586c211640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5864164670 0x7f58640c45f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586c211640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f586415a040 con 0x7f5864164670 2023-12-20T05:44:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586c211640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5864154ff0 con 0x7f5864070230 2023-12-20T05:44:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586c211640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5864151710 con 0x7f586406da20 2023-12-20T05:44:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586a7ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5864164670 0x7f58640c45f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586a7ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5864164670 0x7f58640c45f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53914/0 (socket says 172.21.15.32:53914) 2023-12-20T05:44:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586a7ae640 1 -- 172.21.15.32:0/746314468 learned_addr learned my addr 172.21.15.32:0/746314468 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:05.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f58697ac640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5864070230 0x7f58640c40b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f5869fad640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f586406da20 0x7f58641511d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586a7ae640 1 -- 172.21.15.32:0/746314468 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f586406da20 msgr2=0x7f58641511d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586a7ae640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f586406da20 0x7f58641511d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586a7ae640 1 -- 172.21.15.32:0/746314468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5864070230 msgr2=0x7f58640c40b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586a7ae640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5864070230 0x7f58640c40b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586a7ae640 1 -- 172.21.15.32:0/746314468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5860009420 con 0x7f5864164670 2023-12-20T05:44:05.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586a7ae640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5864164670 0x7f58640c45f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f585c00cbf0 tx=0x7f585c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:05.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f585a7fc640 1 -- 172.21.15.32:0/746314468 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00ad40 con 0x7f5864164670 2023-12-20T05:44:05.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f585a7fc640 1 -- 172.21.15.32:0/746314468 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f585c012400 con 0x7f5864164670 2023-12-20T05:44:05.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586c211640 1 -- 172.21.15.32:0/746314468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5864162c20 con 0x7f5864164670 2023-12-20T05:44:05.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f585a7fc640 1 -- 172.21.15.32:0/746314468 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f585c00aea0 con 0x7f5864164670 2023-12-20T05:44:05.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.368+0000 7f586c211640 1 -- 172.21.15.32:0/746314468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58641631e0 con 0x7f5864164670 2023-12-20T05:44:05.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.372+0000 7f586c211640 1 -- 172.21.15.32:0/746314468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5864155180 con 0x7f5864164670 2023-12-20T05:44:05.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.372+0000 7f585a7fc640 1 -- 172.21.15.32:0/746314468 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 89) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f585c007070 con 0x7f5864164670 2023-12-20T05:44:05.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.372+0000 7f585a7fc640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5848065bb0 0x7f5848068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.372+0000 7f585a7fc640 1 -- 172.21.15.32:0/746314468 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f585c083390 con 0x7f5864164670 2023-12-20T05:44:05.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.372+0000 7f5869fad640 1 -- 172.21.15.32:0/746314468 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5848065bb0 msgr2=0x7f5848068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:05.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.372+0000 7f5869fad640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5848065bb0 0x7f5848068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:05.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.376+0000 7f585a7fc640 1 -- 172.21.15.32:0/746314468 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f585c050ad0 con 0x7f5864164670 2023-12-20T05:44:05.410 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:05.407+0000 7fab1391d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T05:44:05.499 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:05.495+0000 7fab1391d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T05:44:05.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 -- 172.21.15.32:0/746314468 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f586404e000 con 0x7f5864164670 2023-12-20T05:44:05.532 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:05.536+0000 7ff784321180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T05:44:05.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f585a7fc640 1 -- 172.21.15.32:0/746314468 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v160) v1 ==== 93+0+10 (secure 0 0 0) 0x7f585c017020 con 0x7f5864164670 2023-12-20T05:44:05.533 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T05:44:05.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 -- 172.21.15.32:0/746314468 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5848065bb0 msgr2=0x7f5848068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:05.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5848065bb0 0x7f5848068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 -- 172.21.15.32:0/746314468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5864164670 msgr2=0x7f58640c45f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5864164670 0x7f58640c45f0 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7f585c00cbf0 tx=0x7f585c00a430 comp rx=0 tx=0).stop 2023-12-20T05:44:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 -- 172.21.15.32:0/746314468 shutdown_connections 2023-12-20T05:44:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5848065bb0 0x7f5848068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5864164670 0x7f58640c45f0 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5864070230 0x7f58640c40b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 --2- 172.21.15.32:0/746314468 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f586406da20 0x7f58641511d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 -- 172.21.15.32:0/746314468 >> 172.21.15.32:0/746314468 conn(0x7f586406a510 msgr2=0x7f586415ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 -- 172.21.15.32:0/746314468 shutdown_connections 2023-12-20T05:44:05.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.536+0000 7f586c211640 1 -- 172.21.15.32:0/746314468 wait complete. 2023-12-20T05:44:05.555 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T05:44:05.626 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:05.628+0000 7ff784321180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T05:44:05.656 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:05.651+0000 7fab1391d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T05:44:05.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe349b96640 1 Processor -- start 2023-12-20T05:44:05.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe349b96640 1 -- start start 2023-12-20T05:44:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe349b96640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3440d3300 0x7fe3440d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe349b96640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3440d1300 0x7fe3440b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe349b96640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe3440d2650 con 0x7fe34415ce60 2023-12-20T05:44:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe349b96640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe3440d1a40 con 0x7fe3440d3300 2023-12-20T05:44:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe349b96640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3440d1d20 con 0x7fe3440d1300 2023-12-20T05:44:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe342ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3440d3300 0x7fe3440d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe342ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe3440d3300 0x7fe3440d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53926/0 (socket says 172.21.15.32:53926) 2023-12-20T05:44:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe342ffd640 1 -- 172.21.15.32:0/2302352224 learned_addr learned my addr 172.21.15.32:0/2302352224 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe343fff640 1 --2- 172.21.15.32:0/2302352224 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3440d1300 0x7fe3440b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe342ffd640 1 -- 172.21.15.32:0/2302352224 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3440d1300 msgr2=0x7fe3440b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe342ffd640 1 --2- 172.21.15.32:0/2302352224 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3440d1300 0x7fe3440b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe342ffd640 1 -- 172.21.15.32:0/2302352224 >> v1:172.21.15.39:6789/0 conn(0x7fe34415ce60 legacy=0x7fe3440d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.708+0000 7fe342ffd640 1 -- 172.21.15.32:0/2302352224 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3440d2000 con 0x7fe3440d3300 2023-12-20T05:44:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe342ffd640 1 --2- 172.21.15.32:0/2302352224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3440d3300 0x7fe3440d0dc0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fe338009870 tx=0x7fe33801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ef12207761f66b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe3427fc640 1 -- 172.21.15.32:0/2302352224 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3380a8a30 con 0x7fe3440d3300 2023-12-20T05:44:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe3427fc640 1 -- 172.21.15.32:0/2302352224 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3380a8b90 con 0x7fe3440d3300 2023-12-20T05:44:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe3427fc640 1 -- 172.21.15.32:0/2302352224 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3380b0440 con 0x7fe3440d3300 2023-12-20T05:44:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe349b96640 1 -- 172.21.15.32:0/2302352224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3440d3300 msgr2=0x7fe3440d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe349b96640 1 --2- 172.21.15.32:0/2302352224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3440d3300 0x7fe3440d0dc0 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7fe338009870 tx=0x7fe33801ed30 comp rx=0 tx=0).stop 2023-12-20T05:44:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe349b96640 1 -- 172.21.15.32:0/2302352224 shutdown_connections 2023-12-20T05:44:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe349b96640 1 --2- 172.21.15.32:0/2302352224 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3440d1300 0x7fe3440b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe349b96640 1 --2- 172.21.15.32:0/2302352224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe3440d3300 0x7fe3440d0dc0 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe349b96640 1 -- 172.21.15.32:0/2302352224 >> 172.21.15.32:0/2302352224 conn(0x7fe34406b5e0 msgr2=0x7fe34406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe349b96640 1 -- 172.21.15.32:0/2302352224 shutdown_connections 2023-12-20T05:44:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.712+0000 7fe349b96640 1 -- 172.21.15.32:0/2302352224 wait complete. 2023-12-20T05:44:05.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe349b96640 1 Processor -- start 2023-12-20T05:44:05.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe349b96640 1 -- start start 2023-12-20T05:44:05.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe349b96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3440d1300 0x7fe34414f4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe349b96640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3440d3300 0x7fe34415d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe349b96640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe34415ce60 0x7fe34415dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe349b96640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe3440b2a40 con 0x7fe34415ce60 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe349b96640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe34415f0c0 con 0x7fe3440d1300 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe349b96640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe3440ca870 con 0x7fe3440d3300 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe343fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe34415ce60 0x7fe34415dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe3437fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3440d1300 0x7fe34414f4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe342ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3440d3300 0x7fe34415d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe343fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe34415ce60 0x7fe34415dab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53938/0 (socket says 172.21.15.32:53938) 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe343fff640 1 -- 172.21.15.32:0/952402143 learned_addr learned my addr 172.21.15.32:0/952402143 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe343fff640 1 -- 172.21.15.32:0/952402143 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3440d3300 msgr2=0x7fe34415d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe343fff640 1 --2- 172.21.15.32:0/952402143 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3440d3300 0x7fe34415d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe343fff640 1 -- 172.21.15.32:0/952402143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3440d1300 msgr2=0x7fe34414f4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe343fff640 1 --2- 172.21.15.32:0/952402143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3440d1300 0x7fe34414f4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe343fff640 1 -- 172.21.15.32:0/952402143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe3380094d0 con 0x7fe34415ce60 2023-12-20T05:44:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe343fff640 1 --2- 172.21.15.32:0/952402143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe34415ce60 0x7fe34415dab0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00efc0 tx=0x7fe33c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe32bfff640 1 -- 172.21.15.32:0/952402143 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c009b10 con 0x7fe34415ce60 2023-12-20T05:44:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe32bfff640 1 -- 172.21.15.32:0/952402143 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe33c00f040 con 0x7fe34415ce60 2023-12-20T05:44:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe32bfff640 1 -- 172.21.15.32:0/952402143 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe33c0146c0 con 0x7fe34415ce60 2023-12-20T05:44:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe349b96640 1 -- 172.21.15.32:0/952402143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3440cab50 con 0x7fe34415ce60 2023-12-20T05:44:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.716+0000 7fe349b96640 1 -- 172.21.15.32:0/952402143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe34414fa00 con 0x7fe34415ce60 2023-12-20T05:44:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.720+0000 7fe349b96640 1 -- 172.21.15.32:0/952402143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe34415f2c0 con 0x7fe34415ce60 2023-12-20T05:44:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.720+0000 7fe32bfff640 1 -- 172.21.15.32:0/952402143 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 89) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fe33c014820 con 0x7fe34415ce60 2023-12-20T05:44:05.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.720+0000 7fe32bfff640 1 --2- 172.21.15.32:0/952402143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe330065b80 0x7fe330068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:05.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.720+0000 7fe32bfff640 1 -- 172.21.15.32:0/952402143 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7fe33c0841b0 con 0x7fe34415ce60 2023-12-20T05:44:05.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.720+0000 7fe3437fe640 1 -- 172.21.15.32:0/952402143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe330065b80 msgr2=0x7fe330068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:05.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.720+0000 7fe3437fe640 1 --2- 172.21.15.32:0/952402143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe330065b80 0x7fe330068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:05.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.724+0000 7fe32bfff640 1 -- 172.21.15.32:0/952402143 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe33c051910 con 0x7fe34415ce60 2023-12-20T05:44:05.744 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:05.739+0000 7fab1391d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T05:44:05.788 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:05.792+0000 7ff784321180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T05:44:05.877 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:05.880+0000 7ff784321180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T05:44:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.880+0000 7fe349b96640 1 -- 172.21.15.32:0/952402143 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fe34404e000 con 0x7fe34415ce60 2023-12-20T05:44:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.880+0000 7fe32bfff640 1 -- 172.21.15.32:0/952402143 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v160) v1 ==== 104+0+11 (secure 0 0 0) 0x7fe33c054f30 con 0x7fe34415ce60 2023-12-20T05:44:05.879 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T05:44:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 -- 172.21.15.32:0/952402143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe330065b80 msgr2=0x7fe330068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 --2- 172.21.15.32:0/952402143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe330065b80 0x7fe330068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 -- 172.21.15.32:0/952402143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe34415ce60 msgr2=0x7fe34415dab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 --2- 172.21.15.32:0/952402143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe34415ce60 0x7fe34415dab0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7fe33c00efc0 tx=0x7fe33c00c490 comp rx=0 tx=0).stop 2023-12-20T05:44:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 -- 172.21.15.32:0/952402143 shutdown_connections 2023-12-20T05:44:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 --2- 172.21.15.32:0/952402143 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe330065b80 0x7fe330068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 --2- 172.21.15.32:0/952402143 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe34415ce60 0x7fe34415dab0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 --2- 172.21.15.32:0/952402143 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe3440d3300 0x7fe34415d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 --2- 172.21.15.32:0/952402143 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe3440d1300 0x7fe34414f4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 -- 172.21.15.32:0/952402143 >> 172.21.15.32:0/952402143 conn(0x7fe34406b5e0 msgr2=0x7fe3440b6d20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 -- 172.21.15.32:0/952402143 shutdown_connections 2023-12-20T05:44:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:05.884+0000 7fe349b96640 1 -- 172.21.15.32:0/952402143 wait complete. 2023-12-20T05:44:05.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T05:44:06.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.052+0000 7f61e6b6d640 1 Processor -- start 2023-12-20T05:44:06.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.052+0000 7f61e6b6d640 1 -- start start 2023-12-20T05:44:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.052+0000 7f61e6b6d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f61e00d3300 0x7f61e00d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.052+0000 7f61e6b6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e00d2650 con 0x7f61e00d1300 2023-12-20T05:44:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.052+0000 7f61e6b6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61e00d1a40 con 0x7f61e015ce60 2023-12-20T05:44:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.052+0000 7f61e6b6d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61e00d1d20 con 0x7f61e00d3300 2023-12-20T05:44:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.052+0000 7f61e510a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f61e00d1300 0x7f61e00b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50468/0 (socket says 172.21.15.32:50468) 2023-12-20T05:44:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.052+0000 7f61e510a640 1 -- 172.21.15.32:0/2983518395 learned_addr learned my addr 172.21.15.32:0/2983518395 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:06.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.052+0000 7f61dffff640 1 --2- 172.21.15.32:0/2983518395 >> v2:172.21.15.39:3300/0 conn(0x7f61e00d3300 0x7f61e00d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61df7fe640 1 -- 172.21.15.32:0/2983518395 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2399586658 0 0) 0x7f61e00d2650 con 0x7f61e00d1300 2023-12-20T05:44:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61df7fe640 1 -- 172.21.15.32:0/2983518395 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f61d0003530 con 0x7f61e00d1300 2023-12-20T05:44:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61df7fe640 1 -- 172.21.15.32:0/2983518395 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3180728524 0 0) 0x7f61e00d1a40 con 0x7f61e015ce60 2023-12-20T05:44:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61dffff640 1 -- 172.21.15.32:0/2983518395 >> v1:172.21.15.39:6789/0 conn(0x7f61e015ce60 legacy=0x7f61e00d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61dffff640 1 -- 172.21.15.32:0/2983518395 >> v1:172.21.15.32:6789/0 conn(0x7f61e00d1300 legacy=0x7f61e00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61dffff640 1 -- 172.21.15.32:0/2983518395 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61e00d2000 con 0x7f61e00d3300 2023-12-20T05:44:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61dffff640 1 --2- 172.21.15.32:0/2983518395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e00d3300 0x7f61e00d0dc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0097c0 tx=0x7f61cc01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c81380ef50a0efcd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61df7fe640 1 -- 172.21.15.32:0/2983518395 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0ac640 con 0x7f61e00d3300 2023-12-20T05:44:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61df7fe640 1 -- 172.21.15.32:0/2983518395 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc0ac7a0 con 0x7f61e00d3300 2023-12-20T05:44:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61df7fe640 1 -- 172.21.15.32:0/2983518395 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc0acb30 con 0x7f61e00d3300 2023-12-20T05:44:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61e6b6d640 1 -- 172.21.15.32:0/2983518395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e00d3300 msgr2=0x7f61e00d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.056+0000 7f61e6b6d640 1 --2- 172.21.15.32:0/2983518395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e00d3300 0x7f61e00d0dc0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0097c0 tx=0x7f61cc01c480 comp rx=0 tx=0).stop 2023-12-20T05:44:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 -- 172.21.15.32:0/2983518395 shutdown_connections 2023-12-20T05:44:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 --2- 172.21.15.32:0/2983518395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e00d3300 0x7f61e00d0dc0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 -- 172.21.15.32:0/2983518395 >> 172.21.15.32:0/2983518395 conn(0x7f61e006b5e0 msgr2=0x7f61e006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 -- 172.21.15.32:0/2983518395 shutdown_connections 2023-12-20T05:44:06.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 -- 172.21.15.32:0/2983518395 wait complete. 2023-12-20T05:44:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 Processor -- start 2023-12-20T05:44:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 -- start start 2023-12-20T05:44:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e00d1300 0x7f61e0157060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e00d3300 0x7f61e0159a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61e015ce60 0x7f61e00b2a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e015a3b0 con 0x7f61e00d3300 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61e015a690 con 0x7f61e00d1300 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e6b6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61e015a970 con 0x7f61e015ce60 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61dffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e00d3300 0x7f61e0159a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61dffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e00d3300 0x7f61e0159a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:53954/0 (socket says 172.21.15.32:53954) 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61dffff640 1 -- 172.21.15.32:0/180261576 learned_addr learned my addr 172.21.15.32:0/180261576 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e4909640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e00d1300 0x7f61e0157060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61e510a640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61e015ce60 0x7f61e00b2a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61dffff640 1 -- 172.21.15.32:0/180261576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61e015ce60 msgr2=0x7f61e00b2a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61dffff640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61e015ce60 0x7f61e00b2a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61dffff640 1 -- 172.21.15.32:0/180261576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e00d1300 msgr2=0x7f61e0157060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61dffff640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e00d1300 0x7f61e0157060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.060+0000 7f61dffff640 1 -- 172.21.15.32:0/180261576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61cc009420 con 0x7f61e00d3300 2023-12-20T05:44:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61dffff640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e00d3300 0x7f61e0159a60 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0097c0 tx=0x7f61cc019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61dd7fa640 1 -- 172.21.15.32:0/180261576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc004220 con 0x7f61e00d3300 2023-12-20T05:44:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61dd7fa640 1 -- 172.21.15.32:0/180261576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61cc004380 con 0x7f61e00d3300 2023-12-20T05:44:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61e6b6d640 1 -- 172.21.15.32:0/180261576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61e015ac50 con 0x7f61e00d3300 2023-12-20T05:44:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61dd7fa640 1 -- 172.21.15.32:0/180261576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61cc003840 con 0x7f61e00d3300 2023-12-20T05:44:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61e6b6d640 1 -- 172.21.15.32:0/180261576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f61e0159fa0 con 0x7f61e00d3300 2023-12-20T05:44:06.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61e6b6d640 1 -- 172.21.15.32:0/180261576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f61e004e000 con 0x7f61e00d3300 2023-12-20T05:44:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61dd7fa640 1 -- 172.21.15.32:0/180261576 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 89) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f61cc0b3040 con 0x7f61e00d3300 2023-12-20T05:44:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61dd7fa640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61c4065b80 0x7f61c4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61dd7fa640 1 -- 172.21.15.32:0/180261576 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(160..160 src has 1..160) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f61cc11c9f0 con 0x7f61e00d3300 2023-12-20T05:44:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61e4909640 1 -- 172.21.15.32:0/180261576 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61c4065b80 msgr2=0x7f61c4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.064+0000 7f61e4909640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61c4065b80 0x7f61c4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:06.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.072+0000 7f61dd7fa640 1 -- 172.21.15.32:0/180261576 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f61cc0ea130 con 0x7f61e00d3300 2023-12-20T05:44:06.079 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:06.075+0000 7fab1391d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T05:44:06.142 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:06.139+0000 7fab1391d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T05:44:06.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.184+0000 7f61dd7fa640 1 -- 172.21.15.32:0/180261576 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 90) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f61cc11ed50 con 0x7f61e00d3300 2023-12-20T05:44:06.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.232+0000 7ff784321180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T05:44:06.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 -- 172.21.15.32:0/180261576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f61e015ade0 con 0x7f61e00d3300 2023-12-20T05:44:06.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61dd7fa640 1 -- 172.21.15.32:0/180261576 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v160) v1 ==== 100+0+11 (secure 0 0 0) 0x7f61cc101770 con 0x7f61e00d3300 2023-12-20T05:44:06.232 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T05:44:06.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 -- 172.21.15.32:0/180261576 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61c4065b80 msgr2=0x7f61c4068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:06.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61c4065b80 0x7f61c4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 -- 172.21.15.32:0/180261576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e00d3300 msgr2=0x7f61e0159a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e00d3300 0x7f61e0159a60 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7f61cc0097c0 tx=0x7f61cc019220 comp rx=0 tx=0).stop 2023-12-20T05:44:06.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 -- 172.21.15.32:0/180261576 shutdown_connections 2023-12-20T05:44:06.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f61c4065b80 0x7f61c4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61e015ce60 0x7f61e00b2a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61e00d3300 0x7f61e0159a60 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 --2- 172.21.15.32:0/180261576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61e00d1300 0x7f61e0157060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 -- 172.21.15.32:0/180261576 >> 172.21.15.32:0/180261576 conn(0x7f61e006b5e0 msgr2=0x7f61e006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 -- 172.21.15.32:0/180261576 shutdown_connections 2023-12-20T05:44:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.236+0000 7f61e6b6d640 1 -- 172.21.15.32:0/180261576 wait complete. 2023-12-20T05:44:06.254 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:44:06.296 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.300+0000 7ff784321180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T05:44:06.373 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.376+0000 7ff70c188640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:06.373 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.376+0000 7ff70c188640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:06.373 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.376+0000 7ff70c188640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:06.373 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.376+0000 7ff70c188640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:06.373 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.376+0000 7ff70c188640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:06.373 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.376+0000 7ff70e98d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:06.373 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.376+0000 7ff70e98d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:06.373 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.376+0000 7ff70e98d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:06.374 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.376+0000 7ff70e98d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:06.374 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:06.376+0000 7ff70e98d640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:06.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.420+0000 7f87afac4640 1 Processor -- start 2023-12-20T05:44:06.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.420+0000 7f87afac4640 1 -- start start 2023-12-20T05:44:06.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87afac4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f87a81603e0 0x7f87a80cd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87afac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f87a80cd840 0x7f87a8160af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87afac4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f87a80713f0 con 0x7f87a8161030 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87afac4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f87a80719b0 con 0x7f87a81603e0 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87afac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a81614c0 con 0x7f87a80cd840 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87ae061640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f87a8161030 0x7f87a806c640 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50470/0 (socket says 172.21.15.32:50470) 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87ae061640 1 -- 172.21.15.32:0/1398776520 learned_addr learned my addr 172.21.15.32:0/1398776520 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87ad05f640 1 --2- 172.21.15.32:0/1398776520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f87a80cd840 0x7f87a8160af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87ad860640 1 --2- 172.21.15.32:0/1398776520 >> v2:172.21.15.39:3300/0 conn(0x7f87a81603e0 0x7f87a80cd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87ac85e640 1 -- 172.21.15.32:0/1398776520 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193224694 0 0) 0x7f87a80713f0 con 0x7f87a8161030 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87ac85e640 1 -- 172.21.15.32:0/1398776520 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8794003530 con 0x7f87a8161030 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87ad05f640 1 -- 172.21.15.32:0/1398776520 >> v2:172.21.15.39:3300/0 conn(0x7f87a81603e0 msgr2=0x7f87a80cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87ad05f640 1 --2- 172.21.15.32:0/1398776520 >> v2:172.21.15.39:3300/0 conn(0x7f87a81603e0 0x7f87a80cd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87ad05f640 1 -- 172.21.15.32:0/1398776520 >> v1:172.21.15.32:6789/0 conn(0x7f87a8161030 legacy=0x7f87a806c640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.424+0000 7f87ad05f640 1 -- 172.21.15.32:0/1398776520 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f87a80716d0 con 0x7f87a80cd840 2023-12-20T05:44:06.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87ad05f640 1 --2- 172.21.15.32:0/1398776520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f87a80cd840 0x7f87a8160af0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f87980097c0 tx=0x7f879801c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=a1cd6c7063e15b88 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87ac85e640 1 -- 172.21.15.32:0/1398776520 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87980ab600 con 0x7f87a80cd840 2023-12-20T05:44:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87ac85e640 1 -- 172.21.15.32:0/1398776520 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87980a7050 con 0x7f87a80cd840 2023-12-20T05:44:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87ac85e640 1 -- 172.21.15.32:0/1398776520 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87980ab980 con 0x7f87a80cd840 2023-12-20T05:44:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 -- 172.21.15.32:0/1398776520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f87a80cd840 msgr2=0x7f87a8160af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 --2- 172.21.15.32:0/1398776520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f87a80cd840 0x7f87a8160af0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f87980097c0 tx=0x7f879801c330 comp rx=0 tx=0).stop 2023-12-20T05:44:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 -- 172.21.15.32:0/1398776520 shutdown_connections 2023-12-20T05:44:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 --2- 172.21.15.32:0/1398776520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f87a80cd840 0x7f87a8160af0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 --2- 172.21.15.32:0/1398776520 >> v2:172.21.15.39:3300/0 conn(0x7f87a81603e0 0x7f87a80cd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 -- 172.21.15.32:0/1398776520 >> 172.21.15.32:0/1398776520 conn(0x7f87a806e110 msgr2=0x7f87a806e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 -- 172.21.15.32:0/1398776520 shutdown_connections 2023-12-20T05:44:06.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 -- 172.21.15.32:0/1398776520 wait complete. 2023-12-20T05:44:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 Processor -- start 2023-12-20T05:44:06.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 -- start start 2023-12-20T05:44:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f87a80cd840 0x7f87a80c6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a81603e0 0x7f87a80b3d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87a8161030 0x7f87a80c9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a8156c00 con 0x7f87a81603e0 2023-12-20T05:44:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f87a80c6cc0 con 0x7f87a8161030 2023-12-20T05:44:06.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.428+0000 7f87afac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f87a8157eb0 con 0x7f87a80cd840 2023-12-20T05:44:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87ad05f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a81603e0 0x7f87a80b3d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87ad05f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a81603e0 0x7f87a80b3d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54056/0 (socket says 172.21.15.32:54056) 2023-12-20T05:44:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87ad05f640 1 -- 172.21.15.32:0/3269549728 learned_addr learned my addr 172.21.15.32:0/3269549728 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87ad860640 1 --2- 172.21.15.32:0/3269549728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f87a80cd840 0x7f87a80c6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87ae061640 1 --2- 172.21.15.32:0/3269549728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87a8161030 0x7f87a80c9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87ad05f640 1 -- 172.21.15.32:0/3269549728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f87a80cd840 msgr2=0x7f87a80c6780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87ad05f640 1 --2- 172.21.15.32:0/3269549728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f87a80cd840 0x7f87a80c6780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87ad05f640 1 -- 172.21.15.32:0/3269549728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87a8161030 msgr2=0x7f87a80c9ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87ad05f640 1 --2- 172.21.15.32:0/3269549728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87a8161030 0x7f87a80c9ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87ad05f640 1 -- 172.21.15.32:0/3269549728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8798009420 con 0x7f87a81603e0 2023-12-20T05:44:06.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87ad05f640 1 --2- 172.21.15.32:0/3269549728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a81603e0 0x7f87a80b3d00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f87980ae040 tx=0x7f879801ab70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f879e7fc640 1 -- 172.21.15.32:0/3269549728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87980b7450 con 0x7f87a81603e0 2023-12-20T05:44:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f879e7fc640 1 -- 172.21.15.32:0/3269549728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f87980b75d0 con 0x7f87a81603e0 2023-12-20T05:44:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f879e7fc640 1 -- 172.21.15.32:0/3269549728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f87980b7980 con 0x7f87a81603e0 2023-12-20T05:44:06.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87afac4640 1 -- 172.21.15.32:0/3269549728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f87a8156e10 con 0x7f87a81603e0 2023-12-20T05:44:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87afac4640 1 -- 172.21.15.32:0/3269549728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f87a81573d0 con 0x7f87a81603e0 2023-12-20T05:44:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f87afac4640 1 -- 172.21.15.32:0/3269549728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f87a804e000 con 0x7f87a81603e0 2023-12-20T05:44:06.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.432+0000 7f879e7fc640 1 -- 172.21.15.32:0/3269549728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 91) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f87980b7ae0 con 0x7f87a81603e0 2023-12-20T05:44:06.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.436+0000 7f879e7fc640 1 -- 172.21.15.32:0/3269549728 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f879811d570 con 0x7f87a81603e0 2023-12-20T05:44:06.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.440+0000 7f879e7fc640 1 -- 172.21.15.32:0/3269549728 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f87980eb0c0 con 0x7f87a81603e0 2023-12-20T05:44:06.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.628+0000 7f87afac4640 1 -- 172.21.15.32:0/3269549728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f87a80b4c50 con 0x7f87a81603e0 2023-12-20T05:44:06.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.628+0000 7f879e7fc640 1 -- 172.21.15.32:0/3269549728 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f87980ee6e0 con 0x7f87a81603e0 2023-12-20T05:44:06.627 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:06.627 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2062259714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2062259714},{"type":"v1","addr":"172.21.15.39:6834","nonce":2062259714}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4210073096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":4210073096},{"type":"v1","addr":"172.21.15.39:6835","nonce":4210073096}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8913,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3547769022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3547769022},{"type":"v1","addr":"172.21.15.32:6835","nonce":3547769022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:43:46.505246+0000","modified":"2023-12-20T05:43:49.528294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8928":{"gid":8928,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3239941411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3239941411},{"type":"v1","addr":"172.21.15.32:6837","nonce":3239941411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-20T05:44:06.628 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 71 2023-12-20T05:44:06.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.632+0000 7f87afac4640 1 -- 172.21.15.32:0/3269549728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a81603e0 msgr2=0x7f87a80b3d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.632+0000 7f87afac4640 1 --2- 172.21.15.32:0/3269549728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a81603e0 0x7f87a80b3d00 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7f87980ae040 tx=0x7f879801ab70 comp rx=0 tx=0).stop 2023-12-20T05:44:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.632+0000 7f87afac4640 1 -- 172.21.15.32:0/3269549728 shutdown_connections 2023-12-20T05:44:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.632+0000 7f87afac4640 1 --2- 172.21.15.32:0/3269549728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f87a8161030 0x7f87a80c9ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.632+0000 7f87afac4640 1 --2- 172.21.15.32:0/3269549728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f87a81603e0 0x7f87a80b3d00 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.632+0000 7f87afac4640 1 --2- 172.21.15.32:0/3269549728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f87a80cd840 0x7f87a80c6780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.632+0000 7f87afac4640 1 -- 172.21.15.32:0/3269549728 >> 172.21.15.32:0/3269549728 conn(0x7f87a806e110 msgr2=0x7f87a80c4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.632+0000 7f87afac4640 1 -- 172.21.15.32:0/3269549728 shutdown_connections 2023-12-20T05:44:06.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.632+0000 7f87afac4640 1 -- 172.21.15.32:0/3269549728 wait complete. 2023-12-20T05:44:06.648 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:44:06.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.840+0000 7f478fd6d640 1 Processor -- start 2023-12-20T05:44:06.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.840+0000 7f478fd6d640 1 -- start start 2023-12-20T05:44:06.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478fd6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 0x7f478815d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478fd6d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f478806caf0 0x7f47880705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478fd6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f47880ae140 con 0x7f47880afdb0 2023-12-20T05:44:06.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478fd6d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f47880ae9e0 con 0x7f478806caf0 2023-12-20T05:44:06.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478fd6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f478815dee0 con 0x7f47881593c0 2023-12-20T05:44:06.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478db09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 0x7f478815d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478d308640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f47880afdb0 0x7f478806c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:46998/0 (socket says 172.21.15.32:46998) 2023-12-20T05:44:06.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478db09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 0x7f478815d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:41846/0 (socket says 172.21.15.32:41846) 2023-12-20T05:44:06.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478d308640 1 -- 172.21.15.32:0/1530793377 learned_addr learned my addr 172.21.15.32:0/1530793377 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478e30a640 1 --2- 172.21.15.32:0/1530793377 >> v2:172.21.15.39:3300/0 conn(0x7f478806caf0 0x7f47880705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478db09640 1 -- 172.21.15.32:0/1530793377 >> v2:172.21.15.39:3300/0 conn(0x7f478806caf0 msgr2=0x7f47880705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478db09640 1 --2- 172.21.15.32:0/1530793377 >> v2:172.21.15.39:3300/0 conn(0x7f478806caf0 0x7f47880705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478cb07640 1 -- 172.21.15.32:0/1530793377 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3883110098 0 0) 0x7f47880ae140 con 0x7f47880afdb0 2023-12-20T05:44:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478db09640 1 -- 172.21.15.32:0/1530793377 >> v1:172.21.15.39:6789/0 conn(0x7f47880afdb0 legacy=0x7f478806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478db09640 1 -- 172.21.15.32:0/1530793377 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4788072170 con 0x7f47881593c0 2023-12-20T05:44:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.844+0000 7f478db09640 1 --2- 172.21.15.32:0/1530793377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 0x7f478815d9a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f47840097a0 tx=0x7f478401c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5833ab13ef7d178 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478cb07640 1 -- 172.21.15.32:0/1530793377 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47840ab600 con 0x7f47881593c0 2023-12-20T05:44:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478cb07640 1 -- 172.21.15.32:0/1530793377 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f47840a7050 con 0x7f47881593c0 2023-12-20T05:44:06.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478cb07640 1 -- 172.21.15.32:0/1530793377 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47840ab980 con 0x7f47881593c0 2023-12-20T05:44:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 -- 172.21.15.32:0/1530793377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 msgr2=0x7f478815d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 --2- 172.21.15.32:0/1530793377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 0x7f478815d9a0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f47840097a0 tx=0x7f478401c3a0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 -- 172.21.15.32:0/1530793377 shutdown_connections 2023-12-20T05:44:06.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 --2- 172.21.15.32:0/1530793377 >> v2:172.21.15.39:3300/0 conn(0x7f478806caf0 0x7f47880705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 --2- 172.21.15.32:0/1530793377 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 0x7f478815d9a0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 -- 172.21.15.32:0/1530793377 >> 172.21.15.32:0/1530793377 conn(0x7f478806a320 msgr2=0x7f47880ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 -- 172.21.15.32:0/1530793377 shutdown_connections 2023-12-20T05:44:06.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 -- 172.21.15.32:0/1530793377 wait complete. 2023-12-20T05:44:06.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 Processor -- start 2023-12-20T05:44:06.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 -- start start 2023-12-20T05:44:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478806caf0 0x7f47881540d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47880afdb0 0x7f47880ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 0x7f47880abe20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4788153cd0 con 0x7f47880afdb0 2023-12-20T05:44:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f47880c6b50 con 0x7f478806caf0 2023-12-20T05:44:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478fd6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f47880c6df0 con 0x7f47881593c0 2023-12-20T05:44:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.848+0000 7f478d308640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47880afdb0 0x7f47880ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478db09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478806caf0 0x7f47881540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478d308640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47880afdb0 0x7f47880ab8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54070/0 (socket says 172.21.15.32:54070) 2023-12-20T05:44:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478d308640 1 -- 172.21.15.32:0/398278529 learned_addr learned my addr 172.21.15.32:0/398278529 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478e30a640 1 --2- 172.21.15.32:0/398278529 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 0x7f47880abe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478d308640 1 -- 172.21.15.32:0/398278529 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 msgr2=0x7f47880abe20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478d308640 1 --2- 172.21.15.32:0/398278529 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 0x7f47880abe20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478d308640 1 -- 172.21.15.32:0/398278529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478806caf0 msgr2=0x7f47881540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478d308640 1 --2- 172.21.15.32:0/398278529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478806caf0 0x7f47881540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478d308640 1 -- 172.21.15.32:0/398278529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f47840093b0 con 0x7f47880afdb0 2023-12-20T05:44:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478db09640 1 --2- 172.21.15.32:0/398278529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478806caf0 0x7f47881540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:06.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478d308640 1 --2- 172.21.15.32:0/398278529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47880afdb0 0x7f47880ab8e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f47780011f0 tx=0x7f477800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f477e7fc640 1 -- 172.21.15.32:0/398278529 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f47780074e0 con 0x7f47880afdb0 2023-12-20T05:44:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f477e7fc640 1 -- 172.21.15.32:0/398278529 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4778007640 con 0x7f47880afdb0 2023-12-20T05:44:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478fd6d640 1 -- 172.21.15.32:0/398278529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f47881539f0 con 0x7f47880afdb0 2023-12-20T05:44:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f477e7fc640 1 -- 172.21.15.32:0/398278529 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4778004a10 con 0x7f47880afdb0 2023-12-20T05:44:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478fd6d640 1 -- 172.21.15.32:0/398278529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f47880c7370 con 0x7f47880afdb0 2023-12-20T05:44:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f478fd6d640 1 -- 172.21.15.32:0/398278529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f47880c7590 con 0x7f47880afdb0 2023-12-20T05:44:06.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.852+0000 7f477e7fc640 1 -- 172.21.15.32:0/398278529 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 91) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f4778004b70 con 0x7f47880afdb0 2023-12-20T05:44:06.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.856+0000 7f477e7fc640 1 -- 172.21.15.32:0/398278529 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f4778082120 con 0x7f47880afdb0 2023-12-20T05:44:06.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:06.860+0000 7f477e7fc640 1 -- 172.21.15.32:0/398278529 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f477804fbf0 con 0x7f47880afdb0 2023-12-20T05:44:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.016+0000 7f478fd6d640 1 -- 172.21.15.32:0/398278529 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f478804e000 con 0x7f47880afdb0 2023-12-20T05:44:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.016+0000 7f477e7fc640 1 -- 172.21.15.32:0/398278529 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16965 (secure 0 0 0) 0x7f4778053210 con 0x7f47880afdb0 2023-12-20T05:44:07.015 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:07.015 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":161,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:44:06.302262+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:43:44.235578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-20T05:43:45.508783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2916678609":"2023-12-21T05:44:06.302223+0000","172.21.15.32:0/514691750":"2023-12-21T05:44:06.302223+0000","172.21.15.32:0/1727284782":"2023-12-21T05:44:06.302223+0000","172.21.15.32:0/3885704352":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/327030673":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/2048889983":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/202779734":"2023-12-21T05:44:06.302223+0000","172.21.15.32:0/2323751978":"2023-12-21T05:44:06.302223+0000","172.21.15.32:6833/38009":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/304037821":"2023-12-21T05:44:06.302223+0000","172.21.15.32:6832/38009":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/3563338617":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/1355418179":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/616248093":"2023-12-21T05:43:59.101776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:44:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.020+0000 7f478fd6d640 1 -- 172.21.15.32:0/398278529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47880afdb0 msgr2=0x7f47880ab8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.020+0000 7f478fd6d640 1 --2- 172.21.15.32:0/398278529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47880afdb0 0x7f47880ab8e0 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7f47780011f0 tx=0x7f477800c9b0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.020+0000 7f478fd6d640 1 -- 172.21.15.32:0/398278529 shutdown_connections 2023-12-20T05:44:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.020+0000 7f478fd6d640 1 --2- 172.21.15.32:0/398278529 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f47881593c0 0x7f47880abe20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.020+0000 7f478fd6d640 1 --2- 172.21.15.32:0/398278529 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f47880afdb0 0x7f47880ab8e0 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.020+0000 7f478fd6d640 1 --2- 172.21.15.32:0/398278529 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f478806caf0 0x7f47881540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.020+0000 7f478fd6d640 1 -- 172.21.15.32:0/398278529 >> 172.21.15.32:0/398278529 conn(0x7f478806a320 msgr2=0x7f47880705a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:07.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.020+0000 7f478fd6d640 1 -- 172.21.15.32:0/398278529 shutdown_connections 2023-12-20T05:44:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.020+0000 7f478fd6d640 1 -- 172.21.15.32:0/398278529 wait complete. 2023-12-20T05:44:07.036 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T05:44:07.036 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:44:07.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff592edf640 1 Processor -- start 2023-12-20T05:44:07.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff592edf640 1 -- start start 2023-12-20T05:44:07.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff592edf640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff58c0af5e0 0x7ff58c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff592edf640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff58c06ce60 0x7ff58c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff592edf640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff58c15bbd0 con 0x7ff58c15f110 2023-12-20T05:44:07.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff592edf640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff58c15beb0 con 0x7ff58c0af5e0 2023-12-20T05:44:07.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff592edf640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c15e910 con 0x7ff58c06ce60 2023-12-20T05:44:07.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff590c7b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff58c0af5e0 0x7ff58c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff58bfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff58c06ce60 0x7ff58c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff590c7b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff58c0af5e0 0x7ff58c0af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54084/0 (socket says 172.21.15.32:54084) 2023-12-20T05:44:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff590c7b640 1 -- 172.21.15.32:0/4098037964 learned_addr learned my addr 172.21.15.32:0/4098037964 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff590c7b640 1 -- 172.21.15.32:0/4098037964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff58c06ce60 msgr2=0x7ff58c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff590c7b640 1 --2- 172.21.15.32:0/4098037964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff58c06ce60 0x7ff58c15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff590c7b640 1 -- 172.21.15.32:0/4098037964 >> v1:172.21.15.39:6789/0 conn(0x7ff58c15f110 legacy=0x7ff58c15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.184+0000 7ff590c7b640 1 -- 172.21.15.32:0/4098037964 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff58c15ebf0 con 0x7ff58c0af5e0 2023-12-20T05:44:07.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff590c7b640 1 --2- 172.21.15.32:0/4098037964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff58c0af5e0 0x7ff58c0af9c0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7ff580009860 tx=0x7ff58001c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e923ad07a5de9bf2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff58b7fe640 1 -- 172.21.15.32:0/4098037964 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5800ab680 con 0x7ff58c0af5e0 2023-12-20T05:44:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff58b7fe640 1 -- 172.21.15.32:0/4098037964 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5800ab7e0 con 0x7ff58c0af5e0 2023-12-20T05:44:07.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff58b7fe640 1 -- 172.21.15.32:0/4098037964 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5800abb70 con 0x7ff58c0af5e0 2023-12-20T05:44:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff592edf640 1 -- 172.21.15.32:0/4098037964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff58c0af5e0 msgr2=0x7ff58c0af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff592edf640 1 --2- 172.21.15.32:0/4098037964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff58c0af5e0 0x7ff58c0af9c0 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7ff580009860 tx=0x7ff58001c520 comp rx=0 tx=0).stop 2023-12-20T05:44:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff592edf640 1 -- 172.21.15.32:0/4098037964 shutdown_connections 2023-12-20T05:44:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff592edf640 1 --2- 172.21.15.32:0/4098037964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff58c06ce60 0x7ff58c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff592edf640 1 --2- 172.21.15.32:0/4098037964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff58c0af5e0 0x7ff58c0af9c0 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff592edf640 1 -- 172.21.15.32:0/4098037964 >> 172.21.15.32:0/4098037964 conn(0x7ff58c06a320 msgr2=0x7ff58c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff592edf640 1 -- 172.21.15.32:0/4098037964 shutdown_connections 2023-12-20T05:44:07.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff592edf640 1 -- 172.21.15.32:0/4098037964 wait complete. 2023-12-20T05:44:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.188+0000 7ff592edf640 1 Processor -- start 2023-12-20T05:44:07.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff592edf640 1 -- start start 2023-12-20T05:44:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff592edf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff58c06ce60 0x7ff58c154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff592edf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff58c0af5e0 0x7ff58c151be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff592edf640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff58c15f110 0x7ff58c152120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff592edf640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c152660 con 0x7ff58c06ce60 2023-12-20T05:44:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff592edf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0b22d0 con 0x7ff58c0af5e0 2023-12-20T05:44:07.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff592edf640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff58c0716f0 con 0x7ff58c15f110 2023-12-20T05:44:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff590c7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff58c06ce60 0x7ff58c154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff590c7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff58c06ce60 0x7ff58c154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54094/0 (socket says 172.21.15.32:54094) 2023-12-20T05:44:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff590c7b640 1 -- 172.21.15.32:0/3911895180 learned_addr learned my addr 172.21.15.32:0/3911895180 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff59147c640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff58c15f110 0x7ff58c152120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff58bfff640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff58c0af5e0 0x7ff58c151be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff590c7b640 1 -- 172.21.15.32:0/3911895180 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff58c15f110 msgr2=0x7ff58c152120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff590c7b640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff58c15f110 0x7ff58c152120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff590c7b640 1 -- 172.21.15.32:0/3911895180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff58c0af5e0 msgr2=0x7ff58c151be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff590c7b640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff58c0af5e0 0x7ff58c151be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff590c7b640 1 -- 172.21.15.32:0/3911895180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff580009470 con 0x7ff58c06ce60 2023-12-20T05:44:07.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff590c7b640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff58c06ce60 0x7ff58c154390 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7ff5800ad040 tx=0x7ff58001ad60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff5897fa640 1 -- 172.21.15.32:0/3911895180 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5800ab680 con 0x7ff58c06ce60 2023-12-20T05:44:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff5897fa640 1 -- 172.21.15.32:0/3911895180 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff5800b4820 con 0x7ff58c06ce60 2023-12-20T05:44:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff592edf640 1 -- 172.21.15.32:0/3911895180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff58c0719d0 con 0x7ff58c06ce60 2023-12-20T05:44:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff5897fa640 1 -- 172.21.15.32:0/3911895180 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff5800b4bd0 con 0x7ff58c06ce60 2023-12-20T05:44:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff592edf640 1 -- 172.21.15.32:0/3911895180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff58c071f90 con 0x7ff58c06ce60 2023-12-20T05:44:07.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.192+0000 7ff592edf640 1 -- 172.21.15.32:0/3911895180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff58c0b2490 con 0x7ff58c06ce60 2023-12-20T05:44:07.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.196+0000 7ff5897fa640 1 -- 172.21.15.32:0/3911895180 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 91) v1 ==== 84939+0+0 (secure 0 0 0) 0x7ff5800b4d30 con 0x7ff58c06ce60 2023-12-20T05:44:07.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.196+0000 7ff5897fa640 1 -- 172.21.15.32:0/3911895180 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6339+0+0 (secure 0 0 0) 0x7ff58011dc30 con 0x7ff58c06ce60 2023-12-20T05:44:07.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.200+0000 7ff5897fa640 1 -- 172.21.15.32:0/3911895180 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff5800eb6d0 con 0x7ff58c06ce60 2023-12-20T05:44:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.316+0000 7ff5897fa640 1 -- 172.21.15.32:0/3911895180 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 92) v1 ==== 85066+0+0 (secure 0 0 0) 0x7ff5800eb180 con 0x7ff58c06ce60 2023-12-20T05:44:07.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.316+0000 7ff5897fa640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff57006b140 0x7ff57006d530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.316+0000 7ff58bfff640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff57006b140 0x7ff57006d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.320+0000 7ff58bfff640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff57006b140 0x7ff57006d530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0b4410 tx=0x7ff578008040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.400+0000 7ff592edf640 1 -- 172.21.15.32:0/3911895180 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff58c072180 con 0x7ff58c06ce60 2023-12-20T05:44:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.400+0000 7ff5897fa640 1 -- 172.21.15.32:0/3911895180 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4711 (secure 0 0 0) 0x7ff580104990 con 0x7ff58c06ce60 2023-12-20T05:44:07.398 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:07.398 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2062259714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2062259714},{"type":"v1","addr":"172.21.15.39:6834","nonce":2062259714}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4210073096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":4210073096},{"type":"v1","addr":"172.21.15.39:6835","nonce":4210073096}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8913,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3547769022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3547769022},{"type":"v1","addr":"172.21.15.32:6835","nonce":3547769022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:43:46.505246+0000","modified":"2023-12-20T05:43:49.528294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8928":{"gid":8928,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3239941411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3239941411},{"type":"v1","addr":"172.21.15.32:6837","nonce":3239941411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-20T05:44:07.398 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 71 2023-12-20T05:44:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 -- 172.21.15.32:0/3911895180 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff57006b140 msgr2=0x7ff57006d530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff57006b140 0x7ff57006d530 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff58c0b4410 tx=0x7ff578008040 comp rx=0 tx=0).stop 2023-12-20T05:44:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 -- 172.21.15.32:0/3911895180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff58c06ce60 msgr2=0x7ff58c154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff58c06ce60 0x7ff58c154390 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7ff5800ad040 tx=0x7ff58001ad60 comp rx=0 tx=0).stop 2023-12-20T05:44:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 -- 172.21.15.32:0/3911895180 shutdown_connections 2023-12-20T05:44:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff57006b140 0x7ff57006d530 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff58c15f110 0x7ff58c152120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff58c0af5e0 0x7ff58c151be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 --2- 172.21.15.32:0/3911895180 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff58c06ce60 0x7ff58c154390 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 -- 172.21.15.32:0/3911895180 >> 172.21.15.32:0/3911895180 conn(0x7ff58c06a320 msgr2=0x7ff58c15cf60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 -- 172.21.15.32:0/3911895180 shutdown_connections 2023-12-20T05:44:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.404+0000 7ff592edf640 1 -- 172.21.15.32:0/3911895180 wait complete. 2023-12-20T05:44:07.420 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:44:07.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.564+0000 7f81d0ad7640 1 Processor -- start 2023-12-20T05:44:07.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.564+0000 7f81d0ad7640 1 -- start start 2023-12-20T05:44:07.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81d0ad7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81cc15ace0 0x7f81cc0b1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81d0ad7640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81cc0709d0 0x7f81cc0afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81d0ad7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81cc070570 con 0x7f81cc070e30 2023-12-20T05:44:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81d0ad7640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81cc0cab80 con 0x7f81cc0709d0 2023-12-20T05:44:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81d0ad7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81cc0c9c20 con 0x7f81cc15ace0 2023-12-20T05:44:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81caffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81cc0709d0 0x7f81cc0afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81caffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f81cc0709d0 0x7f81cc0afff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54098/0 (socket says 172.21.15.32:54098) 2023-12-20T05:44:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81caffd640 1 -- 172.21.15.32:0/3203528204 learned_addr learned my addr 172.21.15.32:0/3203528204 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81cb7fe640 1 --2- 172.21.15.32:0/3203528204 >> v2:172.21.15.39:3300/0 conn(0x7f81cc15ace0 0x7f81cc0b1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81caffd640 1 -- 172.21.15.32:0/3203528204 >> v2:172.21.15.39:3300/0 conn(0x7f81cc15ace0 msgr2=0x7f81cc0b1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81caffd640 1 --2- 172.21.15.32:0/3203528204 >> v2:172.21.15.39:3300/0 conn(0x7f81cc15ace0 0x7f81cc0b1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81caffd640 1 -- 172.21.15.32:0/3203528204 >> v1:172.21.15.39:6789/0 conn(0x7f81cc070e30 legacy=0x7f81cc15e870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.568+0000 7f81caffd640 1 -- 172.21.15.32:0/3203528204 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81cc0c9f00 con 0x7f81cc0709d0 2023-12-20T05:44:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81caffd640 1 --2- 172.21.15.32:0/3203528204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc0709d0 0x7f81cc0afff0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0cc060 tx=0x7f81c001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2cbab2c87a5ef4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81ca7fc640 1 -- 172.21.15.32:0/3203528204 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c00ac640 con 0x7f81cc0709d0 2023-12-20T05:44:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81ca7fc640 1 -- 172.21.15.32:0/3203528204 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c00ac7a0 con 0x7f81cc0709d0 2023-12-20T05:44:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81ca7fc640 1 -- 172.21.15.32:0/3203528204 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c00acb30 con 0x7f81cc0709d0 2023-12-20T05:44:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 -- 172.21.15.32:0/3203528204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc0709d0 msgr2=0x7f81cc0afff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 --2- 172.21.15.32:0/3203528204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc0709d0 0x7f81cc0afff0 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f81cc0cc060 tx=0x7f81c001c4d0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 -- 172.21.15.32:0/3203528204 shutdown_connections 2023-12-20T05:44:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 --2- 172.21.15.32:0/3203528204 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc0709d0 0x7f81cc0afff0 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 --2- 172.21.15.32:0/3203528204 >> v2:172.21.15.39:3300/0 conn(0x7f81cc15ace0 0x7f81cc0b1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 -- 172.21.15.32:0/3203528204 >> 172.21.15.32:0/3203528204 conn(0x7f81cc06c060 msgr2=0x7f81cc06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 -- 172.21.15.32:0/3203528204 shutdown_connections 2023-12-20T05:44:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 -- 172.21.15.32:0/3203528204 wait complete. 2023-12-20T05:44:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 Processor -- start 2023-12-20T05:44:07.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 -- start start 2023-12-20T05:44:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81cc0709d0 0x7f81cc0cdd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc070e30 0x7f81cc15d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc15ace0 0x7f81cc153bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc15e430 con 0x7f81cc070e30 2023-12-20T05:44:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc1641a0 con 0x7f81cc0709d0 2023-12-20T05:44:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.572+0000 7f81d0ad7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81cc15dbd0 con 0x7f81cc15ace0 2023-12-20T05:44:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc070e30 0x7f81cc15d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc070e30 0x7f81cc15d690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54112/0 (socket says 172.21.15.32:54112) 2023-12-20T05:44:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81caffd640 1 -- 172.21.15.32:0/404248573 learned_addr learned my addr 172.21.15.32:0/404248573 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81cb7fe640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81cc0709d0 0x7f81cc0cdd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81cbfff640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc15ace0 0x7f81cc153bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81caffd640 1 -- 172.21.15.32:0/404248573 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc15ace0 msgr2=0x7f81cc153bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81caffd640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc15ace0 0x7f81cc153bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81caffd640 1 -- 172.21.15.32:0/404248573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81cc0709d0 msgr2=0x7f81cc0cdd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81caffd640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81cc0709d0 0x7f81cc0cdd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81caffd640 1 -- 172.21.15.32:0/404248573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81c00094d0 con 0x7f81cc070e30 2023-12-20T05:44:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81caffd640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc070e30 0x7f81cc15d690 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f81c00190e0 tx=0x7f81c0019270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81b3fff640 1 -- 172.21.15.32:0/404248573 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c00b5640 con 0x7f81cc070e30 2023-12-20T05:44:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81b3fff640 1 -- 172.21.15.32:0/404248573 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81c00b57a0 con 0x7f81cc070e30 2023-12-20T05:44:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81b3fff640 1 -- 172.21.15.32:0/404248573 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81c00b5b10 con 0x7f81cc070e30 2023-12-20T05:44:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81d0ad7640 1 -- 172.21.15.32:0/404248573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81cc15de70 con 0x7f81cc070e30 2023-12-20T05:44:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81d0ad7640 1 -- 172.21.15.32:0/404248573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f81cc163e30 con 0x7f81cc070e30 2023-12-20T05:44:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81d0ad7640 1 -- 172.21.15.32:0/404248573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f81cc163c90 con 0x7f81cc070e30 2023-12-20T05:44:07.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81b3fff640 1 -- 172.21.15.32:0/404248573 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 92) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f81c00b3070 con 0x7f81cc070e30 2023-12-20T05:44:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.576+0000 7f81b3fff640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81b80658b0 0x7f81b8067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.580+0000 7f81b3fff640 1 -- 172.21.15.32:0/404248573 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6339+0+0 (secure 0 0 0) 0x7f81c011d4d0 con 0x7f81cc070e30 2023-12-20T05:44:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.580+0000 7f81cb7fe640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81b80658b0 0x7f81b8067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.580+0000 7f81cb7fe640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81b80658b0 0x7f81b8067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00b930 tx=0x7f81bc008040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.580+0000 7f81b3fff640 1 -- 172.21.15.32:0/404248573 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f81cc163c90 con 0x7f81cc070e30 2023-12-20T05:44:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.772+0000 7f81d0ad7640 1 -- 172.21.15.32:0/404248573 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f81cc163c90 con 0x7f81cc070e30 2023-12-20T05:44:07.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.772+0000 7f81b3fff640 1 -- 172.21.15.32:0/404248573 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 71 v71) v1 ==== 94+0+4711 (secure 0 0 0) 0x7f81cc163c90 con 0x7f81cc070e30 2023-12-20T05:44:07.770 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:07.770 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":71,"default_fscid":9,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2062259714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2062259714},{"type":"v1","addr":"172.21.15.39:6834","nonce":2062259714}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4210073096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":4210073096},{"type":"v1","addr":"172.21.15.39:6835","nonce":4210073096}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8913,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3547769022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3547769022},{"type":"v1","addr":"172.21.15.32:6835","nonce":3547769022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65}],"filesystems":[{"mdsmap":{"epoch":71,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:43:46.505246+0000","modified":"2023-12-20T05:43:49.528294+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":8928},"failed":[],"damaged":[],"stopped":[],"info":{"gid_8928":{"gid":8928,"name":"a","rank":0,"incarnation":67,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3239941411","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":3239941411},{"type":"v1","addr":"172.21.15.32:6837","nonce":3239941411}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[19],"metadata_pool":18,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":9}]} 2023-12-20T05:44:07.770 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 71 2023-12-20T05:44:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 -- 172.21.15.32:0/404248573 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81b80658b0 msgr2=0x7f81b8067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81b80658b0 0x7f81b8067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f81bc00b930 tx=0x7f81bc008040 comp rx=0 tx=0).stop 2023-12-20T05:44:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 -- 172.21.15.32:0/404248573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc070e30 msgr2=0x7f81cc15d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc070e30 0x7f81cc15d690 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f81c00190e0 tx=0x7f81c0019270 comp rx=0 tx=0).stop 2023-12-20T05:44:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 -- 172.21.15.32:0/404248573 shutdown_connections 2023-12-20T05:44:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f81b80658b0 0x7f81b8067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81cc15ace0 0x7f81cc153bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81cc070e30 0x7f81cc15d690 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 --2- 172.21.15.32:0/404248573 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81cc0709d0 0x7f81cc0cdd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 -- 172.21.15.32:0/404248573 >> 172.21.15.32:0/404248573 conn(0x7f81cc06c060 msgr2=0x7f81cc0b1e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:07.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 -- 172.21.15.32:0/404248573 shutdown_connections 2023-12-20T05:44:07.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.776+0000 7f81d0ad7640 1 -- 172.21.15.32:0/404248573 wait complete. 2023-12-20T05:44:07.791 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:44:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.936+0000 7fbcd9eb6640 1 Processor -- start 2023-12-20T05:44:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.936+0000 7fbcd9eb6640 1 -- start start 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd9eb6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbcd406ce60 0x7fbcd415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd9eb6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbcd415f110 0x7fbcd415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd9eb6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbcd415bbd0 con 0x7fbcd40af5e0 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd9eb6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbcd415beb0 con 0x7fbcd415f110 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd9eb6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd415e910 con 0x7fbcd406ce60 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbcd415f110 0x7fbcd415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbcd415f110 0x7fbcd415c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54124/0 (socket says 172.21.15.32:54124) 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd3fff640 1 -- 172.21.15.32:0/3667862487 learned_addr learned my addr 172.21.15.32:0/3667862487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd2ffd640 1 --2- 172.21.15.32:0/3667862487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbcd406ce60 0x7fbcd415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd3fff640 1 -- 172.21.15.32:0/3667862487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbcd406ce60 msgr2=0x7fbcd415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd3fff640 1 --2- 172.21.15.32:0/3667862487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbcd406ce60 0x7fbcd415b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd3fff640 1 -- 172.21.15.32:0/3667862487 >> v1:172.21.15.39:6789/0 conn(0x7fbcd40af5e0 legacy=0x7fbcd40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.940+0000 7fbcd3fff640 1 -- 172.21.15.32:0/3667862487 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbcd415ebf0 con 0x7fbcd415f110 2023-12-20T05:44:07.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd3fff640 1 --2- 172.21.15.32:0/3667862487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd415f110 0x7fbcd415c440 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009e50 tx=0x7fbccc01c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=169d999e2f2adc2b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:07.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd27fc640 1 -- 172.21.15.32:0/3667862487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc0a57e0 con 0x7fbcd415f110 2023-12-20T05:44:07.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd27fc640 1 -- 172.21.15.32:0/3667862487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbccc0a5940 con 0x7fbcd415f110 2023-12-20T05:44:07.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd27fc640 1 -- 172.21.15.32:0/3667862487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbccc0a5cd0 con 0x7fbcd415f110 2023-12-20T05:44:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/3667862487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd415f110 msgr2=0x7fbcd415c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 --2- 172.21.15.32:0/3667862487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd415f110 0x7fbcd415c440 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7fbccc009e50 tx=0x7fbccc01c980 comp rx=0 tx=0).stop 2023-12-20T05:44:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/3667862487 shutdown_connections 2023-12-20T05:44:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 --2- 172.21.15.32:0/3667862487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd415f110 0x7fbcd415c440 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 --2- 172.21.15.32:0/3667862487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbcd406ce60 0x7fbcd415b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/3667862487 >> 172.21.15.32:0/3667862487 conn(0x7fbcd406a320 msgr2=0x7fbcd40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/3667862487 shutdown_connections 2023-12-20T05:44:07.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/3667862487 wait complete. 2023-12-20T05:44:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 Processor -- start 2023-12-20T05:44:07.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 -- start start 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd406ce60 0x7fbcd415a2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.944+0000 7fbcd9eb6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcd40af5e0 0x7fbcd414a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd9eb6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbcd415f110 0x7fbcd414aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd9eb6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40cbcb0 con 0x7fbcd406ce60 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd9eb6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd40cbf90 con 0x7fbcd40af5e0 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd9eb6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbcd4071cd0 con 0x7fbcd415f110 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd406ce60 0x7fbcd415a2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd406ce60 0x7fbcd415a2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54134/0 (socket says 172.21.15.32:54134) 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd37fe640 1 -- 172.21.15.32:0/358685008 learned_addr learned my addr 172.21.15.32:0/358685008 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd2ffd640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcd40af5e0 0x7fbcd414a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd3fff640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbcd415f110 0x7fbcd414aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd37fe640 1 -- 172.21.15.32:0/358685008 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbcd415f110 msgr2=0x7fbcd414aad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd37fe640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbcd415f110 0x7fbcd414aad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd37fe640 1 -- 172.21.15.32:0/358685008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcd40af5e0 msgr2=0x7fbcd414a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd37fe640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcd40af5e0 0x7fbcd414a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd37fe640 1 -- 172.21.15.32:0/358685008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbccc009b30 con 0x7fbcd406ce60 2023-12-20T05:44:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd37fe640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd406ce60 0x7fbcd415a2f0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400c450 tx=0x7fbcc400c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcbbfff640 1 -- 172.21.15.32:0/358685008 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc40052d0 con 0x7fbcd406ce60 2023-12-20T05:44:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcbbfff640 1 -- 172.21.15.32:0/358685008 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbcc4005430 con 0x7fbcd406ce60 2023-12-20T05:44:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcbbfff640 1 -- 172.21.15.32:0/358685008 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbcc4003890 con 0x7fbcd406ce60 2023-12-20T05:44:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/358685008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbcd4071fb0 con 0x7fbcd406ce60 2023-12-20T05:44:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/358685008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbcd415a830 con 0x7fbcd406ce60 2023-12-20T05:44:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/358685008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbcd4071b00 con 0x7fbcd406ce60 2023-12-20T05:44:07.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.948+0000 7fbcbbfff640 1 -- 172.21.15.32:0/358685008 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 92) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fbcc40039f0 con 0x7fbcd406ce60 2023-12-20T05:44:07.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.952+0000 7fbcbbfff640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcc00658b0 0x7fbcc0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:07.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.952+0000 7fbcbbfff640 1 -- 172.21.15.32:0/358685008 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6339+0+0 (secure 0 0 0) 0x7fbcc4082c60 con 0x7fbcd406ce60 2023-12-20T05:44:07.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.952+0000 7fbcd2ffd640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcc00658b0 0x7fbcc0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:07.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.952+0000 7fbcd2ffd640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcc00658b0 0x7fbcc0067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbcd414ecf0 tx=0x7fbcc8008040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:07.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:07.956+0000 7fbcbbfff640 1 -- 172.21.15.32:0/358685008 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbcc4050610 con 0x7fbcd406ce60 2023-12-20T05:44:08.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.112+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/358685008 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbcd40721a0 con 0x7fbcd406ce60 2023-12-20T05:44:08.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcbbfff640 1 -- 172.21.15.32:0/358685008 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v161) v1 ==== 74+0+16965 (secure 0 0 0) 0x7fbcc4053c30 con 0x7fbcd406ce60 2023-12-20T05:44:08.112 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:08.112 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":161,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:44:06.302262+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":18,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:43:44.235578+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"158","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":19,"pool_name":"cephfs_data","create_time":"2023-12-20T05:43:45.508783+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"159","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2916678609":"2023-12-21T05:44:06.302223+0000","172.21.15.32:0/514691750":"2023-12-21T05:44:06.302223+0000","172.21.15.32:0/1727284782":"2023-12-21T05:44:06.302223+0000","172.21.15.32:0/3885704352":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/327030673":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/2048889983":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/202779734":"2023-12-21T05:44:06.302223+0000","172.21.15.32:0/2323751978":"2023-12-21T05:44:06.302223+0000","172.21.15.32:6833/38009":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/304037821":"2023-12-21T05:44:06.302223+0000","172.21.15.32:6832/38009":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/3563338617":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/1355418179":"2023-12-21T05:43:59.101776+0000","172.21.15.32:0/616248093":"2023-12-21T05:43:59.101776+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/358685008 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcc00658b0 msgr2=0x7fbcc0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcc00658b0 0x7fbcc0067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fbcd414ecf0 tx=0x7fbcc8008040 comp rx=0 tx=0).stop 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/358685008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd406ce60 msgr2=0x7fbcd415a2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd406ce60 0x7fbcd415a2f0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7fbcc400c450 tx=0x7fbcc400c910 comp rx=0 tx=0).stop 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/358685008 shutdown_connections 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbcc00658b0 0x7fbcc0067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbcd415f110 0x7fbcd414aad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbcd40af5e0 0x7fbcd414a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 --2- 172.21.15.32:0/358685008 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbcd406ce60 0x7fbcd415a2f0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/358685008 >> 172.21.15.32:0/358685008 conn(0x7fbcd406a320 msgr2=0x7fbcd4164680 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/358685008 shutdown_connections 2023-12-20T05:44:08.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.116+0000 7fbcd9eb6640 1 -- 172.21.15.32:0/358685008 wait complete. 2023-12-20T05:44:08.133 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T05:44:08.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd98d35a640 1 Processor -- start 2023-12-20T05:44:08.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd98d35a640 1 -- start start 2023-12-20T05:44:08.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd98d35a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd988161500 0x7fd988153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:08.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd98d35a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9880cd4d0 con 0x7fd9880b8480 2023-12-20T05:44:08.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd98d35a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9880ce4e0 con 0x7fd98806b630 2023-12-20T05:44:08.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd98d35a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9880cd720 con 0x7fd988161500 2023-12-20T05:44:08.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd98cb59640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd988161500 0x7fd988153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:08.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd987fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd9880b8480 0x7fd98806af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50472/0 (socket says 172.21.15.32:50472) 2023-12-20T05:44:08.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd987fff640 1 -- 172.21.15.32:0/3818333491 learned_addr learned my addr 172.21.15.32:0/3818333491 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:08.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd98cb59640 1 -- 172.21.15.32:0/3818333491 >> v1:172.21.15.39:6789/0 conn(0x7fd98806b630 legacy=0x7fd988160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd98cb59640 1 -- 172.21.15.32:0/3818333491 >> v1:172.21.15.32:6789/0 conn(0x7fd9880b8480 legacy=0x7fd98806af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.280+0000 7fd98cb59640 1 -- 172.21.15.32:0/3818333491 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9880cda00 con 0x7fd988161500 2023-12-20T05:44:08.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.284+0000 7fd98cb59640 1 --2- 172.21.15.32:0/3818333491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988161500 0x7fd988153610 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd980009d90 tx=0x7fd98001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21be58d8a87d1dfe server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:08.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.284+0000 7fd986ffd640 1 -- 172.21.15.32:0/3818333491 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800a6610 con 0x7fd988161500 2023-12-20T05:44:08.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.284+0000 7fd986ffd640 1 -- 172.21.15.32:0/3818333491 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9800a6770 con 0x7fd988161500 2023-12-20T05:44:08.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.284+0000 7fd986ffd640 1 -- 172.21.15.32:0/3818333491 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9800a6b00 con 0x7fd988161500 2023-12-20T05:44:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.284+0000 7fd98d35a640 1 -- 172.21.15.32:0/3818333491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988161500 msgr2=0x7fd988153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.284+0000 7fd98d35a640 1 --2- 172.21.15.32:0/3818333491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988161500 0x7fd988153610 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x7fd980009d90 tx=0x7fd98001c7e0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.284+0000 7fd98d35a640 1 -- 172.21.15.32:0/3818333491 shutdown_connections 2023-12-20T05:44:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.284+0000 7fd98d35a640 1 --2- 172.21.15.32:0/3818333491 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd988161500 0x7fd988153610 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.284+0000 7fd98d35a640 1 -- 172.21.15.32:0/3818333491 >> 172.21.15.32:0/3818333491 conn(0x7fd9880d2a50 msgr2=0x7fd9880d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.284+0000 7fd98d35a640 1 -- 172.21.15.32:0/3818333491 shutdown_connections 2023-12-20T05:44:08.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.284+0000 7fd98d35a640 1 -- 172.21.15.32:0/3818333491 wait complete. 2023-12-20T05:44:08.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98d35a640 1 Processor -- start 2023-12-20T05:44:08.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98d35a640 1 -- start start 2023-12-20T05:44:08.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98d35a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806b630 0x7fd9880c82a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:08.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98d35a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9880b8480 0x7fd98815f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98d35a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd988161500 0x7fd9880cb350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98d35a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880c9450 con 0x7fd98806b630 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98d35a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880ca5d0 con 0x7fd988161500 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98d35a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9880ca870 con 0x7fd9880b8480 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd987fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806b630 0x7fd9880c82a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd987fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806b630 0x7fd9880c82a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54142/0 (socket says 172.21.15.32:54142) 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd987fff640 1 -- 172.21.15.32:0/2965806285 learned_addr learned my addr 172.21.15.32:0/2965806285 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd9877fe640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9880b8480 0x7fd98815f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98cb59640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd988161500 0x7fd9880cb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd987fff640 1 -- 172.21.15.32:0/2965806285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9880b8480 msgr2=0x7fd98815f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd987fff640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9880b8480 0x7fd98815f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd987fff640 1 -- 172.21.15.32:0/2965806285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd988161500 msgr2=0x7fd9880cb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd987fff640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd988161500 0x7fd9880cb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd987fff640 1 -- 172.21.15.32:0/2965806285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd980009a70 con 0x7fd98806b630 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd9877fe640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9880b8480 0x7fd98815f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:08.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98cb59640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd988161500 0x7fd9880cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd987fff640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806b630 0x7fd9880c82a0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd97800c780 tx=0x7fd97800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd984ff9640 1 -- 172.21.15.32:0/2965806285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9780074e0 con 0x7fd98806b630 2023-12-20T05:44:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd984ff9640 1 -- 172.21.15.32:0/2965806285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd978007640 con 0x7fd98806b630 2023-12-20T05:44:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd984ff9640 1 -- 172.21.15.32:0/2965806285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd978004a30 con 0x7fd98806b630 2023-12-20T05:44:08.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98d35a640 1 -- 172.21.15.32:0/2965806285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9880cab50 con 0x7fd98806b630 2023-12-20T05:44:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.288+0000 7fd98d35a640 1 -- 172.21.15.32:0/2965806285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd98814dd30 con 0x7fd98806b630 2023-12-20T05:44:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.292+0000 7fd98d35a640 1 -- 172.21.15.32:0/2965806285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd988074100 con 0x7fd98806b630 2023-12-20T05:44:08.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.292+0000 7fd984ff9640 1 -- 172.21.15.32:0/2965806285 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 92) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fd978004b90 con 0x7fd98806b630 2023-12-20T05:44:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.292+0000 7fd984ff9640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9740658b0 0x7fd974067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.292+0000 7fd984ff9640 1 -- 172.21.15.32:0/2965806285 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(161..161 src has 1..161) v4 ==== 6339+0+0 (secure 0 0 0) 0x7fd9780825a0 con 0x7fd98806b630 2023-12-20T05:44:08.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.292+0000 7fd9877fe640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9740658b0 0x7fd974067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:08.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.292+0000 7fd9877fe640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9740658b0 0x7fd974067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd98806af90 tx=0x7fd97c009040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:08.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.296+0000 7fd984ff9640 1 -- 172.21.15.32:0/2965806285 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd97804ff50 con 0x7fd98806b630 2023-12-20T05:44:08.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.324+0000 7fd984ff9640 1 -- 172.21.15.32:0/2965806285 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 93) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fd978050320 con 0x7fd98806b630 2023-12-20T05:44:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.500+0000 7fd98d35a640 1 -- 172.21.15.32:0/2965806285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fd9880d0a70 con 0x7fd98806b630 2023-12-20T05:44:08.498 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T05:44:08.500+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T05:44:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd984ff9640 1 -- 172.21.15.32:0/2965806285 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v72) v1 ==== 161+0+0 (secure 0 0 0) 0x7fd9780539e0 con 0x7fd98806b630 2023-12-20T05:44:08.517 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T05:44:08.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 -- 172.21.15.32:0/2965806285 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9740658b0 msgr2=0x7fd974067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9740658b0 0x7fd974067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fd98806af90 tx=0x7fd97c009040 comp rx=0 tx=0).stop 2023-12-20T05:44:08.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 -- 172.21.15.32:0/2965806285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806b630 msgr2=0x7fd9880c82a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806b630 0x7fd9880c82a0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7fd97800c780 tx=0x7fd97800cc40 comp rx=0 tx=0).stop 2023-12-20T05:44:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 -- 172.21.15.32:0/2965806285 shutdown_connections 2023-12-20T05:44:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fd9740658b0 0x7fd974067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd988161500 0x7fd9880cb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9880b8480 0x7fd98815f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 --2- 172.21.15.32:0/2965806285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd98806b630 0x7fd9880c82a0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 -- 172.21.15.32:0/2965806285 >> 172.21.15.32:0/2965806285 conn(0x7fd9880d2a50 msgr2=0x7fd988160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 -- 172.21.15.32:0/2965806285 shutdown_connections 2023-12-20T05:44:08.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.520+0000 7fd98d35a640 1 -- 172.21.15.32:0/2965806285 wait complete. 2023-12-20T05:44:08.525 INFO:tasks.ceph.mds.a.smithi032.stderr: -3862> 2023-12-20T05:43:52.240+0000 7f854796d640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T05:44:08.539 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T05:44:08.574 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T05:44:08.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.688+0000 7f51b7c73640 1 Processor -- start 2023-12-20T05:44:08.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.688+0000 7f51b7c73640 1 -- start start 2023-12-20T05:44:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.688+0000 7f51b7c73640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f51b00b0760 0x7f51b00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:08.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.688+0000 7f51b7c73640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b00b1100 con 0x7f51b0157480 2023-12-20T05:44:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.688+0000 7f51b7c73640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f51b00cde20 con 0x7f51b00cc4b0 2023-12-20T05:44:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.688+0000 7f51b7c73640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f51b00c9f50 con 0x7f51b00b0760 2023-12-20T05:44:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.692+0000 7f51b5a0f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f51b0157480 0x7f51b00c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:50486/0 (socket says 172.21.15.32:50486) 2023-12-20T05:44:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.692+0000 7f51b5a0f640 1 -- 172.21.15.32:0/3409510738 learned_addr learned my addr 172.21.15.32:0/3409510738 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.692+0000 7f51b520e640 1 --2- 172.21.15.32:0/3409510738 >> v2:172.21.15.32:3300/0 conn(0x7f51b00b0760 0x7f51b00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.692+0000 7f51b520e640 1 -- 172.21.15.32:0/3409510738 >> v1:172.21.15.39:6789/0 conn(0x7f51b00cc4b0 legacy=0x7f51b00cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.692+0000 7f51b520e640 1 -- 172.21.15.32:0/3409510738 >> v1:172.21.15.32:6789/0 conn(0x7f51b0157480 legacy=0x7f51b00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.692+0000 7f51b520e640 1 -- 172.21.15.32:0/3409510738 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51b00c8e30 con 0x7f51b00b0760 2023-12-20T05:44:08.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.692+0000 7f51b4a0d640 1 -- 172.21.15.32:0/3409510738 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793926599 0 0) 0x7f51b00b1100 con 0x7f51b0157480 2023-12-20T05:44:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.692+0000 7f51b520e640 1 --2- 172.21.15.32:0/3409510738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00b0760 0x7f51b00b0bc0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f51a00097c0 tx=0x7f51a001c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f29d7fc2ba06f2c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.692+0000 7f51b4a0d640 1 -- 172.21.15.32:0/3409510738 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a00ac620 con 0x7f51b00b0760 2023-12-20T05:44:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.692+0000 7f51b4a0d640 1 -- 172.21.15.32:0/3409510738 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51a00ac780 con 0x7f51b00b0760 2023-12-20T05:44:08.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.692+0000 7f51b4a0d640 1 -- 172.21.15.32:0/3409510738 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51a00acb10 con 0x7f51b00b0760 2023-12-20T05:44:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 -- 172.21.15.32:0/3409510738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00b0760 msgr2=0x7f51b00b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 --2- 172.21.15.32:0/3409510738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00b0760 0x7f51b00b0bc0 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7f51a00097c0 tx=0x7f51a001c3c0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 -- 172.21.15.32:0/3409510738 shutdown_connections 2023-12-20T05:44:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 --2- 172.21.15.32:0/3409510738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00b0760 0x7f51b00b0bc0 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 -- 172.21.15.32:0/3409510738 >> 172.21.15.32:0/3409510738 conn(0x7f51b006a510 msgr2=0x7f51b00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 -- 172.21.15.32:0/3409510738 shutdown_connections 2023-12-20T05:44:08.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 -- 172.21.15.32:0/3409510738 wait complete. 2023-12-20T05:44:08.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 Processor -- start 2023-12-20T05:44:08.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 -- start start 2023-12-20T05:44:08.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00b0760 0x7f51b0151640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:08.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b00cc4b0 0x7f51b0156000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:08.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b0157480 0x7f51b0158270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:08.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0156540 con 0x7f51b00b0760 2023-12-20T05:44:08.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f51b01587b0 con 0x7f51b0157480 2023-12-20T05:44:08.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b7c73640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f51b0158a50 con 0x7f51b00cc4b0 2023-12-20T05:44:08.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b5a0f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00b0760 0x7f51b0151640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b5a0f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00b0760 0x7f51b0151640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54188/0 (socket says 172.21.15.32:54188) 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.696+0000 7f51b5a0f640 1 -- 172.21.15.32:0/176127224 learned_addr learned my addr 172.21.15.32:0/176127224 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51b5a0f640 1 -- 172.21.15.32:0/176127224 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b00cc4b0 msgr2=0x7f51b0156000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51b5a0f640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b00cc4b0 0x7f51b0156000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51b5a0f640 1 -- 172.21.15.32:0/176127224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b0157480 msgr2=0x7f51b0158270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51b5a0f640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b0157480 0x7f51b0158270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51b5a0f640 1 -- 172.21.15.32:0/176127224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f51a0009420 con 0x7f51b00b0760 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51b5a0f640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00b0760 0x7f51b0151640 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00c410 tx=0x7f51ac00c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51a67fc640 1 -- 172.21.15.32:0/176127224 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac0074e0 con 0x7f51b00b0760 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51a67fc640 1 -- 172.21.15.32:0/176127224 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f51ac007640 con 0x7f51b00b0760 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51b7c73640 1 -- 172.21.15.32:0/176127224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f51b0158d30 con 0x7f51b00b0760 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51a67fc640 1 -- 172.21.15.32:0/176127224 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f51ac004a10 con 0x7f51b00b0760 2023-12-20T05:44:08.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51b7c73640 1 -- 172.21.15.32:0/176127224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f51b01592f0 con 0x7f51b00b0760 2023-12-20T05:44:08.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51b7c73640 1 -- 172.21.15.32:0/176127224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f51b004e000 con 0x7f51b00b0760 2023-12-20T05:44:08.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51a67fc640 1 -- 172.21.15.32:0/176127224 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 93) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f51ac004b70 con 0x7f51b00b0760 2023-12-20T05:44:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51a67fc640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5194065af0 0x7f5194067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51a67fc640 1 -- 172.21.15.32:0/176127224 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6425+0+0 (secure 0 0 0) 0x7f51ac083330 con 0x7f51b00b0760 2023-12-20T05:44:08.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.700+0000 7f51b520e640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5194065af0 0x7f5194067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:08.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.704+0000 7f51b520e640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5194065af0 0x7f5194067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51a00ae000 tx=0x7f51a0004740 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:08.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.704+0000 7f51a67fc640 1 -- 172.21.15.32:0/176127224 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f51ac0509c0 con 0x7f51b00b0760 2023-12-20T05:44:08.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:08.896+0000 7f51b7c73640 1 -- 172.21.15.32:0/176127224 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f51b00c9050 con 0x7f51b00b0760 2023-12-20T05:44:09.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.520+0000 7f51a67fc640 1 -- 172.21.15.32:0/176127224 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v73) v1 ==== 104+0+0 (secure 0 0 0) 0x7f51ac053fe0 con 0x7f51b00b0760 2023-12-20T05:44:09.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 -- 172.21.15.32:0/176127224 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5194065af0 msgr2=0x7f5194067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:09.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5194065af0 0x7f5194067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f51a00ae000 tx=0x7f51a0004740 comp rx=0 tx=0).stop 2023-12-20T05:44:09.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 -- 172.21.15.32:0/176127224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00b0760 msgr2=0x7f51b0151640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:09.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00b0760 0x7f51b0151640 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7f51ac00c410 tx=0x7f51ac00c8d0 comp rx=0 tx=0).stop 2023-12-20T05:44:09.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 -- 172.21.15.32:0/176127224 shutdown_connections 2023-12-20T05:44:09.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5194065af0 0x7f5194067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:09.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f51b0157480 0x7f51b0158270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:09.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f51b00cc4b0 0x7f51b0156000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:09.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 --2- 172.21.15.32:0/176127224 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f51b00b0760 0x7f51b0151640 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:09.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 -- 172.21.15.32:0/176127224 >> 172.21.15.32:0/176127224 conn(0x7f51b006a510 msgr2=0x7f51b00c36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:09.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 -- 172.21.15.32:0/176127224 shutdown_connections 2023-12-20T05:44:09.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.524+0000 7f51b7c73640 1 -- 172.21.15.32:0/176127224 wait complete. 2023-12-20T05:44:09.543 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T05:44:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.740+0000 7fc92736d640 1 Processor -- start 2023-12-20T05:44:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.740+0000 7fc92736d640 1 -- start start 2023-12-20T05:44:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.740+0000 7fc92736d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc920070960 0x7fc920070dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.740+0000 7fc92736d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc92006ca60 0x7fc92006cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.740+0000 7fc92736d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9200713e0 con 0x7fc920156ee0 2023-12-20T05:44:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.740+0000 7fc92736d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc92006a670 con 0x7fc920070960 2023-12-20T05:44:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.740+0000 7fc92736d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc92006d450 con 0x7fc92006ca60 2023-12-20T05:44:09.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.740+0000 7fc924908640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc920070960 0x7fc920070dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.740+0000 7fc925109640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc920156ee0 0x7fc92006b050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56120/0 (socket says 172.21.15.32:56120) 2023-12-20T05:44:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.740+0000 7fc925109640 1 -- 172.21.15.32:0/1999474565 learned_addr learned my addr 172.21.15.32:0/1999474565 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.740+0000 7fc92590a640 1 --2- 172.21.15.32:0/1999474565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc92006ca60 0x7fc92006cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.744+0000 7fc917fff640 1 -- 172.21.15.32:0/1999474565 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4158443133 0 0) 0x7fc9200713e0 con 0x7fc920156ee0 2023-12-20T05:44:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.744+0000 7fc924908640 1 -- 172.21.15.32:0/1999474565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc92006ca60 msgr2=0x7fc92006cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.744+0000 7fc924908640 1 --2- 172.21.15.32:0/1999474565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc92006ca60 0x7fc92006cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.744+0000 7fc924908640 1 -- 172.21.15.32:0/1999474565 >> v1:172.21.15.32:6789/0 conn(0x7fc920156ee0 legacy=0x7fc92006b050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:09.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.744+0000 7fc924908640 1 -- 172.21.15.32:0/1999474565 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9200716c0 con 0x7fc920070960 2023-12-20T05:44:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.744+0000 7fc924908640 1 --2- 172.21.15.32:0/1999474565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc920070960 0x7fc920070dd0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fc9100097c0 tx=0x7fc91001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6899705340d6a487 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.744+0000 7fc917fff640 1 -- 172.21.15.32:0/1999474565 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9100ab670 con 0x7fc920070960 2023-12-20T05:44:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.744+0000 7fc917fff640 1 -- 172.21.15.32:0/1999474565 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9100ab7d0 con 0x7fc920070960 2023-12-20T05:44:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.744+0000 7fc917fff640 1 -- 172.21.15.32:0/1999474565 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9100abb60 con 0x7fc920070960 2023-12-20T05:44:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 -- 172.21.15.32:0/1999474565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc920070960 msgr2=0x7fc920070dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 --2- 172.21.15.32:0/1999474565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc920070960 0x7fc920070dd0 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fc9100097c0 tx=0x7fc91001c450 comp rx=0 tx=0).stop 2023-12-20T05:44:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 -- 172.21.15.32:0/1999474565 shutdown_connections 2023-12-20T05:44:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 --2- 172.21.15.32:0/1999474565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc920070960 0x7fc920070dd0 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 --2- 172.21.15.32:0/1999474565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc92006ca60 0x7fc92006cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 -- 172.21.15.32:0/1999474565 >> 172.21.15.32:0/1999474565 conn(0x7fc9200cf8d0 msgr2=0x7fc9200cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 -- 172.21.15.32:0/1999474565 shutdown_connections 2023-12-20T05:44:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 -- 172.21.15.32:0/1999474565 wait complete. 2023-12-20T05:44:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 Processor -- start 2023-12-20T05:44:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 -- start start 2023-12-20T05:44:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc92006ca60 0x7fc9200b4760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc920070960 0x7fc9200c6090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc920156ee0 0x7fc9201452f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc92006ac30 con 0x7fc92006ca60 2023-12-20T05:44:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200c65d0 con 0x7fc920156ee0 2023-12-20T05:44:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92736d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9200accc0 con 0x7fc920070960 2023-12-20T05:44:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc925109640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc92006ca60 0x7fc9200b4760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc925109640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc92006ca60 0x7fc9200b4760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37558/0 (socket says 172.21.15.32:37558) 2023-12-20T05:44:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc925109640 1 -- 172.21.15.32:0/1327921549 learned_addr learned my addr 172.21.15.32:0/1327921549 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc924908640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc920070960 0x7fc9200c6090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc92590a640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc920156ee0 0x7fc9201452f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc925109640 1 -- 172.21.15.32:0/1327921549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc920070960 msgr2=0x7fc9200c6090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc925109640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc920070960 0x7fc9200c6090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc925109640 1 -- 172.21.15.32:0/1327921549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc920156ee0 msgr2=0x7fc9201452f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc925109640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc920156ee0 0x7fc9201452f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc925109640 1 -- 172.21.15.32:0/1327921549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc910009420 con 0x7fc92006ca60 2023-12-20T05:44:09.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.748+0000 7fc925109640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc92006ca60 0x7fc9200b4760 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0011f0 tx=0x7fc91c00c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.752+0000 7fc915ffb640 1 -- 172.21.15.32:0/1327921549 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c0074e0 con 0x7fc92006ca60 2023-12-20T05:44:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.752+0000 7fc915ffb640 1 -- 172.21.15.32:0/1327921549 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc91c007640 con 0x7fc92006ca60 2023-12-20T05:44:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.752+0000 7fc915ffb640 1 -- 172.21.15.32:0/1327921549 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc91c004a10 con 0x7fc92006ca60 2023-12-20T05:44:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.752+0000 7fc92736d640 1 -- 172.21.15.32:0/1327921549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9200c8ee0 con 0x7fc92006ca60 2023-12-20T05:44:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.752+0000 7fc92736d640 1 -- 172.21.15.32:0/1327921549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9200ac670 con 0x7fc92006ca60 2023-12-20T05:44:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.752+0000 7fc92736d640 1 -- 172.21.15.32:0/1327921549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc92004e000 con 0x7fc92006ca60 2023-12-20T05:44:09.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.752+0000 7fc915ffb640 1 -- 172.21.15.32:0/1327921549 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 93) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fc91c004b70 con 0x7fc92006ca60 2023-12-20T05:44:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.752+0000 7fc915ffb640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc904065af0 0x7fc904067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.752+0000 7fc915ffb640 1 -- 172.21.15.32:0/1327921549 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(162..162 src has 1..162) v4 ==== 6425+0+0 (secure 0 0 0) 0x7fc91c082790 con 0x7fc92006ca60 2023-12-20T05:44:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.752+0000 7fc924908640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc904065af0 0x7fc904067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:09.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.756+0000 7fc924908640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc904065af0 0x7fc904067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9100af000 tx=0x7fc9100a7000 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:09.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.756+0000 7fc915ffb640 1 -- 172.21.15.32:0/1327921549 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc91c04fe90 con 0x7fc92006ca60 2023-12-20T05:44:09.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:09.916+0000 7fc92736d640 1 -- 172.21.15.32:0/1327921549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fc920069f40 con 0x7fc92006ca60 2023-12-20T05:44:09.951 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:44:09.955 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:44:10.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.536+0000 7fc915ffb640 1 -- 172.21.15.32:0/1327921549 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc91c050260 con 0x7fc92006ca60 2023-12-20T05:44:10.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.536+0000 7fc915ffb640 1 -- 172.21.15.32:0/1327921549 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v163) v1 ==== 181+0+0 (secure 0 0 0) 0x7fc91c04d3e0 con 0x7fc92006ca60 2023-12-20T05:44:10.538 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T05:44:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.544+0000 7fc92736d640 1 -- 172.21.15.32:0/1327921549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc904065af0 msgr2=0x7fc904067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.544+0000 7fc92736d640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc904065af0 0x7fc904067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fc9100af000 tx=0x7fc9100a7000 comp rx=0 tx=0).stop 2023-12-20T05:44:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.544+0000 7fc92736d640 1 -- 172.21.15.32:0/1327921549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc92006ca60 msgr2=0x7fc9200b4760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:10.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.544+0000 7fc92736d640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc92006ca60 0x7fc9200b4760 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0011f0 tx=0x7fc91c00c9b0 comp rx=0 tx=0).stop 2023-12-20T05:44:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.544+0000 7fc92736d640 1 -- 172.21.15.32:0/1327921549 shutdown_connections 2023-12-20T05:44:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.544+0000 7fc92736d640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc904065af0 0x7fc904067fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.544+0000 7fc92736d640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc920156ee0 0x7fc9201452f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:10.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.544+0000 7fc92736d640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc920070960 0x7fc9200c6090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.544+0000 7fc92736d640 1 --2- 172.21.15.32:0/1327921549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc92006ca60 0x7fc9200b4760 secure :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fc91c0011f0 tx=0x7fc91c00c9b0 comp rx=0 tx=0).stop 2023-12-20T05:44:10.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.544+0000 7fc92736d640 1 -- 172.21.15.32:0/1327921549 >> 172.21.15.32:0/1327921549 conn(0x7fc9200cf8d0 msgr2=0x7fc9200cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.548+0000 7fc92736d640 1 -- 172.21.15.32:0/1327921549 shutdown_connections 2023-12-20T05:44:10.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.548+0000 7fc92736d640 1 -- 172.21.15.32:0/1327921549 wait complete. 2023-12-20T05:44:10.569 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T05:44:10.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8aec8d1640 1 Processor -- start 2023-12-20T05:44:10.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8aec8d1640 1 -- start start 2023-12-20T05:44:10.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8aec8d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8ae8159af0 0x7f8ae80b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:10.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8aec8d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae806d2d0 0x7f8ae806d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:10.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8aec8d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ae806dd50 con 0x7f8ae80b07f0 2023-12-20T05:44:10.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8aec8d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ae80cc940 con 0x7f8ae8159af0 2023-12-20T05:44:10.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8aec8d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80ccc20 con 0x7f8ae806d2d0 2023-12-20T05:44:10.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8adffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae806d2d0 0x7f8ae806d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:10.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8ae659c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8ae8159af0 0x7f8ae80b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:10.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8adffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae806d2d0 0x7f8ae806d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:46834/0 (socket says 172.21.15.32:46834) 2023-12-20T05:44:10.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8adffff640 1 -- 172.21.15.32:0/1581147459 learned_addr learned my addr 172.21.15.32:0/1581147459 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:10.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8adffff640 1 -- 172.21.15.32:0/1581147459 >> v2:172.21.15.39:3300/0 conn(0x7f8ae8159af0 msgr2=0x7f8ae80b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:10.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8adffff640 1 --2- 172.21.15.32:0/1581147459 >> v2:172.21.15.39:3300/0 conn(0x7f8ae8159af0 0x7f8ae80b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:10.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8adffff640 1 -- 172.21.15.32:0/1581147459 >> v1:172.21.15.39:6789/0 conn(0x7f8ae80b07f0 legacy=0x7f8ae80cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:10.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8adffff640 1 -- 172.21.15.32:0/1581147459 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ae80ccf00 con 0x7f8ae806d2d0 2023-12-20T05:44:10.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.720+0000 7f8ae5d9b640 1 -- 172.21.15.32:0/1581147459 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921833262 0 0) 0x7f8ae806dd50 con 0x7f8ae80b07f0 2023-12-20T05:44:10.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8adffff640 1 --2- 172.21.15.32:0/1581147459 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae806d2d0 0x7f8ae806d710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009870 tx=0x7f8ad001c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=171d381d222968be server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:10.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8ae5d9b640 1 -- 172.21.15.32:0/1581147459 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00a95f0 con 0x7f8ae806d2d0 2023-12-20T05:44:10.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8ae5d9b640 1 -- 172.21.15.32:0/1581147459 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad00a9750 con 0x7f8ae806d2d0 2023-12-20T05:44:10.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8ae5d9b640 1 -- 172.21.15.32:0/1581147459 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00a9ae0 con 0x7f8ae806d2d0 2023-12-20T05:44:10.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8aec8d1640 1 -- 172.21.15.32:0/1581147459 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae806d2d0 msgr2=0x7f8ae806d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:10.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8aec8d1640 1 --2- 172.21.15.32:0/1581147459 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae806d2d0 0x7f8ae806d710 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f8ad0009870 tx=0x7f8ad001c510 comp rx=0 tx=0).stop 2023-12-20T05:44:10.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8aec8d1640 1 -- 172.21.15.32:0/1581147459 shutdown_connections 2023-12-20T05:44:10.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8aec8d1640 1 --2- 172.21.15.32:0/1581147459 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae806d2d0 0x7f8ae806d710 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:10.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8aec8d1640 1 --2- 172.21.15.32:0/1581147459 >> v2:172.21.15.39:3300/0 conn(0x7f8ae8159af0 0x7f8ae80b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:10.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8aec8d1640 1 -- 172.21.15.32:0/1581147459 >> 172.21.15.32:0/1581147459 conn(0x7f8ae806a510 msgr2=0x7f8ae80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:10.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8aec8d1640 1 -- 172.21.15.32:0/1581147459 shutdown_connections 2023-12-20T05:44:10.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.724+0000 7f8aec8d1640 1 -- 172.21.15.32:0/1581147459 wait complete. 2023-12-20T05:44:10.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8aec8d1640 1 Processor -- start 2023-12-20T05:44:10.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8aec8d1640 1 -- start start 2023-12-20T05:44:10.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8aec8d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae806d2d0 0x7f8ae80c6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:10.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8aec8d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ae80b07f0 0x7f8ae815a180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8aec8d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae8159af0 0x7f8ae81561f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8aec8d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8154f70 con 0x7f8ae806d2d0 2023-12-20T05:44:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8aec8d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae80c7080 con 0x7f8ae80b07f0 2023-12-20T05:44:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8aec8d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ae8155150 con 0x7f8ae8159af0 2023-12-20T05:44:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8ae659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae806d2d0 0x7f8ae80c6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8ae659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae806d2d0 0x7f8ae80c6b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37570/0 (socket says 172.21.15.32:37570) 2023-12-20T05:44:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8adffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ae80b07f0 0x7f8ae815a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8ae659c640 1 -- 172.21.15.32:0/3451364055 learned_addr learned my addr 172.21.15.32:0/3451364055 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:10.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8ae6d9d640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae8159af0 0x7f8ae81561f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8adffff640 1 -- 172.21.15.32:0/3451364055 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae8159af0 msgr2=0x7f8ae81561f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8adffff640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae8159af0 0x7f8ae81561f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8adffff640 1 -- 172.21.15.32:0/3451364055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae806d2d0 msgr2=0x7f8ae80c6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8adffff640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae806d2d0 0x7f8ae80c6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:10.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8adffff640 1 -- 172.21.15.32:0/3451364055 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ad00094d0 con 0x7f8ae80b07f0 2023-12-20T05:44:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8adffff640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ae80b07f0 0x7f8ae815a180 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00aa000 tx=0x7f8ad00192b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8adeffd640 1 -- 172.21.15.32:0/3451364055 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00b1800 con 0x7f8ae80b07f0 2023-12-20T05:44:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8adeffd640 1 -- 172.21.15.32:0/3451364055 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ad00b1980 con 0x7f8ae80b07f0 2023-12-20T05:44:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8adeffd640 1 -- 172.21.15.32:0/3451364055 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ad00b1d30 con 0x7f8ae80b07f0 2023-12-20T05:44:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8aec8d1640 1 -- 172.21.15.32:0/3451364055 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ae80b1190 con 0x7f8ae80b07f0 2023-12-20T05:44:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.728+0000 7f8aec8d1640 1 -- 172.21.15.32:0/3451364055 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ae8155710 con 0x7f8ae80b07f0 2023-12-20T05:44:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.732+0000 7f8aec8d1640 1 -- 172.21.15.32:0/3451364055 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ae804e000 con 0x7f8ae80b07f0 2023-12-20T05:44:10.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.732+0000 7f8adeffd640 1 -- 172.21.15.32:0/3451364055 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8ad00c1360 con 0x7f8ae80b07f0 2023-12-20T05:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.732+0000 7f8adeffd640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8ac8065bb0 0x7f8ac8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.732+0000 7f8adeffd640 1 -- 172.21.15.32:0/3451364055 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(163..163 src has 1..163) v4 ==== 5987+0+0 (secure 0 0 0) 0x7f8ad011d830 con 0x7f8ae80b07f0 2023-12-20T05:44:10.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.732+0000 7f8ae659c640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8ac8065bb0 0x7f8ac8068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:10.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.732+0000 7f8ae659c640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8ac8065bb0 0x7f8ac8068070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ae815d350 tx=0x7f8ad8009190 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:10.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.736+0000 7f8adeffd640 1 -- 172.21.15.32:0/3451364055 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8ad00eafd0 con 0x7f8ae80b07f0 2023-12-20T05:44:10.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:10.896+0000 7f8aec8d1640 1 -- 172.21.15.32:0/3451364055 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f8ae8158430 con 0x7f8ae80b07f0 2023-12-20T05:44:11.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8adeffd640 1 -- 172.21.15.32:0/3451364055 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v164) v1 ==== 169+0+0 (secure 0 0 0) 0x7f8ad00a95b0 con 0x7f8ae80b07f0 2023-12-20T05:44:11.533 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 -- 172.21.15.32:0/3451364055 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8ac8065bb0 msgr2=0x7f8ac8068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8ac8065bb0 0x7f8ac8068070 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f8ae815d350 tx=0x7f8ad8009190 comp rx=0 tx=0).stop 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 -- 172.21.15.32:0/3451364055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ae80b07f0 msgr2=0x7f8ae815a180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ae80b07f0 0x7f8ae815a180 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x7f8ad00aa000 tx=0x7f8ad00192b0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 -- 172.21.15.32:0/3451364055 shutdown_connections 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8ac8065bb0 0x7f8ac8068070 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ae8159af0 0x7f8ae81561f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ae80b07f0 0x7f8ae815a180 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 --2- 172.21.15.32:0/3451364055 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ae806d2d0 0x7f8ae80c6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 -- 172.21.15.32:0/3451364055 >> 172.21.15.32:0/3451364055 conn(0x7f8ae806a510 msgr2=0x7f8ae80c8c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 -- 172.21.15.32:0/3451364055 shutdown_connections 2023-12-20T05:44:11.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.536+0000 7f8aec8d1640 1 -- 172.21.15.32:0/3451364055 wait complete. 2023-12-20T05:44:11.557 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T05:44:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.704+0000 7fc9db0e5640 1 Processor -- start 2023-12-20T05:44:11.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.704+0000 7fc9db0e5640 1 -- start start 2023-12-20T05:44:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9db0e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc9d415abb0 0x7fc9d40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9db0e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9d40b0cb0 0x7fc9d415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9db0e5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc9d415b6a0 con 0x7fc9d4070230 2023-12-20T05:44:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9db0e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc9d406e160 con 0x7fc9d415abb0 2023-12-20T05:44:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9db0e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d415c070 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9d9682640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc9d4070230 0x7fc9d406c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56132/0 (socket says 172.21.15.32:56132) 2023-12-20T05:44:11.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9d9682640 1 -- 172.21.15.32:0/416655163 learned_addr learned my addr 172.21.15.32:0/416655163 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9d8e81640 1 --2- 172.21.15.32:0/416655163 >> v2:172.21.15.39:3300/0 conn(0x7fc9d415abb0 0x7fc9d40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9d3fff640 1 --2- 172.21.15.32:0/416655163 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9d40b0cb0 0x7fc9d415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9d37fe640 1 -- 172.21.15.32:0/416655163 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3769794992 0 0) 0x7fc9d415b6a0 con 0x7fc9d4070230 2023-12-20T05:44:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9d37fe640 1 -- 172.21.15.32:0/416655163 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc9c4003530 con 0x7fc9d4070230 2023-12-20T05:44:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9d37fe640 1 -- 172.21.15.32:0/416655163 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4192533523 0 0) 0x7fc9c4003530 con 0x7fc9d4070230 2023-12-20T05:44:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9d8e81640 1 -- 172.21.15.32:0/416655163 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9d40b0cb0 msgr2=0x7fc9d415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9d8e81640 1 --2- 172.21.15.32:0/416655163 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9d40b0cb0 0x7fc9d415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9d8e81640 1 -- 172.21.15.32:0/416655163 >> v1:172.21.15.32:6789/0 conn(0x7fc9d4070230 legacy=0x7fc9d406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:11.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.708+0000 7fc9d8e81640 1 -- 172.21.15.32:0/416655163 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9d406d8c0 con 0x7fc9d415abb0 2023-12-20T05:44:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9d8e81640 1 --2- 172.21.15.32:0/416655163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d415abb0 0x7fc9d40caaf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80097c0 tx=0x7fc9c801c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4389efae9b86dfd1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9d37fe640 1 -- 172.21.15.32:0/416655163 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9cc001a00 con 0x7fc9d415abb0 2023-12-20T05:44:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9d37fe640 1 -- 172.21.15.32:0/416655163 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c80a7050 con 0x7fc9d415abb0 2023-12-20T05:44:11.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9d37fe640 1 -- 172.21.15.32:0/416655163 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c80ab620 con 0x7fc9d415abb0 2023-12-20T05:44:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9db0e5640 1 -- 172.21.15.32:0/416655163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d415abb0 msgr2=0x7fc9d40caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9db0e5640 1 --2- 172.21.15.32:0/416655163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d415abb0 0x7fc9d40caaf0 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80097c0 tx=0x7fc9c801c3a0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9db0e5640 1 -- 172.21.15.32:0/416655163 shutdown_connections 2023-12-20T05:44:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9db0e5640 1 --2- 172.21.15.32:0/416655163 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9d40b0cb0 0x7fc9d415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9db0e5640 1 --2- 172.21.15.32:0/416655163 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d415abb0 0x7fc9d40caaf0 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9db0e5640 1 -- 172.21.15.32:0/416655163 >> 172.21.15.32:0/416655163 conn(0x7fc9d406a510 msgr2=0x7fc9d40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9db0e5640 1 -- 172.21.15.32:0/416655163 shutdown_connections 2023-12-20T05:44:11.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.712+0000 7fc9db0e5640 1 -- 172.21.15.32:0/416655163 wait complete. 2023-12-20T05:44:11.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9db0e5640 1 Processor -- start 2023-12-20T05:44:11.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9db0e5640 1 -- start start 2023-12-20T05:44:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9db0e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9d4070230 0x7fc9d40c84e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9db0e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d40b0cb0 0x7fc9d414d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9db0e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d415abb0 0x7fc9d40c5a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9db0e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c96f0 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9db0e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d414df00 con 0x7fc9d415abb0 2023-12-20T05:44:11.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9db0e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc9d40c8ae0 con 0x7fc9d4070230 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d40b0cb0 0x7fc9d414d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d40b0cb0 0x7fc9d414d9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37576/0 (socket says 172.21.15.32:37576) 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d3fff640 1 -- 172.21.15.32:0/272270578 learned_addr learned my addr 172.21.15.32:0/272270578 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d9682640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d415abb0 0x7fc9d40c5a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d8e81640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9d4070230 0x7fc9d40c84e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d3fff640 1 -- 172.21.15.32:0/272270578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9d4070230 msgr2=0x7fc9d40c84e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d3fff640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9d4070230 0x7fc9d40c84e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d3fff640 1 -- 172.21.15.32:0/272270578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d415abb0 msgr2=0x7fc9d40c5a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d3fff640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d415abb0 0x7fc9d40c5a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d3fff640 1 -- 172.21.15.32:0/272270578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc9c8009420 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d3fff640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d40b0cb0 0x7fc9d414d9c0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e960 tx=0x7fc9c000ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d17fa640 1 -- 172.21.15.32:0/272270578 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c0009690 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d17fa640 1 -- 172.21.15.32:0/272270578 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc9c000ca00 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9d17fa640 1 -- 172.21.15.32:0/272270578 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc9c000cd70 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9db0e5640 1 -- 172.21.15.32:0/272270578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc9d40c8dc0 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9db0e5640 1 -- 172.21.15.32:0/272270578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc9d40c9380 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.716+0000 7fc9db0e5640 1 -- 172.21.15.32:0/272270578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc9d404e000 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.720+0000 7fc9d17fa640 1 -- 172.21.15.32:0/272270578 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc9c0013470 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.720+0000 7fc9d17fa640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc9b8065b80 0x7fc9b8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.720+0000 7fc9d17fa640 1 -- 172.21.15.32:0/272270578 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fc9c0083580 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.720+0000 7fc9d8e81640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc9b8065b80 0x7fc9b8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:11.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.720+0000 7fc9d8e81640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc9b8065b80 0x7fc9b8068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80ad000 tx=0x7fc9c8008040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:11.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.724+0000 7fc9d17fa640 1 -- 172.21.15.32:0/272270578 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc9c0050e10 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.880+0000 7fc9db0e5640 1 -- 172.21.15.32:0/272270578 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fc9d40b13e0 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9d17fa640 1 -- 172.21.15.32:0/272270578 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v55) v1 ==== 120+0+0 (secure 0 0 0) 0x7fc9c0054430 con 0x7fc9d40b0cb0 2023-12-20T05:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 -- 172.21.15.32:0/272270578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc9b8065b80 msgr2=0x7fc9b8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc9b8065b80 0x7fc9b8068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fc9c80ad000 tx=0x7fc9c8008040 comp rx=0 tx=0).stop 2023-12-20T05:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 -- 172.21.15.32:0/272270578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d40b0cb0 msgr2=0x7fc9d414d9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d40b0cb0 0x7fc9d414d9c0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7fc9c000e960 tx=0x7fc9c000ee20 comp rx=0 tx=0).stop 2023-12-20T05:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 -- 172.21.15.32:0/272270578 shutdown_connections 2023-12-20T05:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc9b8065b80 0x7fc9b8068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc9d415abb0 0x7fc9d40c5a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc9d40b0cb0 0x7fc9d414d9c0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 --2- 172.21.15.32:0/272270578 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc9d4070230 0x7fc9d40c84e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:11.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 -- 172.21.15.32:0/272270578 >> 172.21.15.32:0/272270578 conn(0x7fc9d406a510 msgr2=0x7fc9d4153aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:11.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 -- 172.21.15.32:0/272270578 shutdown_connections 2023-12-20T05:44:11.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:11.888+0000 7fc9db0e5640 1 -- 172.21.15.32:0/272270578 wait complete. 2023-12-20T05:44:11.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T05:44:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb7932c640 1 Processor -- start 2023-12-20T05:44:12.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb7932c640 1 -- start start 2023-12-20T05:44:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb7932c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb741591e0 0x7feb740caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb7932c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7feb74070940 0x7feb740cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb7932c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb740cc3b0 con 0x7feb7406cdf0 2023-12-20T05:44:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb7932c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb740c8e30 con 0x7feb741591e0 2023-12-20T05:44:12.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb7932c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb7406d280 con 0x7feb74070940 2023-12-20T05:44:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb727fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feb7406cdf0 0x7feb74070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56144/0 (socket says 172.21.15.32:56144) 2023-12-20T05:44:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb72ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb741591e0 0x7feb740caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb727fc640 1 -- 172.21.15.32:0/3583317128 learned_addr learned my addr 172.21.15.32:0/3583317128 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:12.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb737fe640 1 --2- 172.21.15.32:0/3583317128 >> v2:172.21.15.39:3300/0 conn(0x7feb74070940 0x7feb740cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb72ffd640 1 -- 172.21.15.32:0/3583317128 >> v2:172.21.15.39:3300/0 conn(0x7feb74070940 msgr2=0x7feb740cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb72ffd640 1 --2- 172.21.15.32:0/3583317128 >> v2:172.21.15.39:3300/0 conn(0x7feb74070940 0x7feb740cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb72ffd640 1 -- 172.21.15.32:0/3583317128 >> v1:172.21.15.32:6789/0 conn(0x7feb7406cdf0 legacy=0x7feb74070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb72ffd640 1 -- 172.21.15.32:0/3583317128 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb740c90d0 con 0x7feb741591e0 2023-12-20T05:44:12.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.056+0000 7feb71ffb640 1 -- 172.21.15.32:0/3583317128 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3789051076 0 0) 0x7feb740cc3b0 con 0x7feb7406cdf0 2023-12-20T05:44:12.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb72ffd640 1 --2- 172.21.15.32:0/3583317128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb741591e0 0x7feb740caa30 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7feb740c5860 tx=0x7feb6401c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1550a84def70e810 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:12.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb71ffb640 1 -- 172.21.15.32:0/3583317128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640a5790 con 0x7feb741591e0 2023-12-20T05:44:12.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb71ffb640 1 -- 172.21.15.32:0/3583317128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb640a58f0 con 0x7feb741591e0 2023-12-20T05:44:12.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb71ffb640 1 -- 172.21.15.32:0/3583317128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb640a5c80 con 0x7feb741591e0 2023-12-20T05:44:12.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb7932c640 1 -- 172.21.15.32:0/3583317128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb741591e0 msgr2=0x7feb740caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb7932c640 1 --2- 172.21.15.32:0/3583317128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb741591e0 0x7feb740caa30 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7feb740c5860 tx=0x7feb6401c880 comp rx=0 tx=0).stop 2023-12-20T05:44:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb7932c640 1 -- 172.21.15.32:0/3583317128 shutdown_connections 2023-12-20T05:44:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb7932c640 1 --2- 172.21.15.32:0/3583317128 >> v2:172.21.15.39:3300/0 conn(0x7feb74070940 0x7feb740cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb7932c640 1 --2- 172.21.15.32:0/3583317128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb741591e0 0x7feb740caa30 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb7932c640 1 -- 172.21.15.32:0/3583317128 >> 172.21.15.32:0/3583317128 conn(0x7feb7406a510 msgr2=0x7feb740ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb7932c640 1 -- 172.21.15.32:0/3583317128 shutdown_connections 2023-12-20T05:44:12.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.060+0000 7feb7932c640 1 -- 172.21.15.32:0/3583317128 wait complete. 2023-12-20T05:44:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb7932c640 1 Processor -- start 2023-12-20T05:44:12.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb7932c640 1 -- start start 2023-12-20T05:44:12.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb7932c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb74070940 0x7feb74155380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb7932c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb741591e0 0x7feb74156a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb7932c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7414dda0 0x7feb74157c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb7932c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb74155960 con 0x7feb741591e0 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb7932c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb74155c40 con 0x7feb7414dda0 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb7932c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb74155f20 con 0x7feb74070940 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb727fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb741591e0 0x7feb74156a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb727fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb741591e0 0x7feb74156a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37580/0 (socket says 172.21.15.32:37580) 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb727fc640 1 -- 172.21.15.32:0/15032027 learned_addr learned my addr 172.21.15.32:0/15032027 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb72ffd640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb74070940 0x7feb74155380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb737fe640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7414dda0 0x7feb74157c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb727fc640 1 -- 172.21.15.32:0/15032027 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb74070940 msgr2=0x7feb74155380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb727fc640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb74070940 0x7feb74155380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb727fc640 1 -- 172.21.15.32:0/15032027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7414dda0 msgr2=0x7feb74157c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb727fc640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7414dda0 0x7feb74157c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb727fc640 1 -- 172.21.15.32:0/15032027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb64009a70 con 0x7feb741591e0 2023-12-20T05:44:12.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb72ffd640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb74070940 0x7feb74155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb737fe640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7414dda0 0x7feb74157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb727fc640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb741591e0 0x7feb74156a00 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7feb680011f0 tx=0x7feb6800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb5b7fe640 1 -- 172.21.15.32:0/15032027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb680074e0 con 0x7feb741591e0 2023-12-20T05:44:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb5b7fe640 1 -- 172.21.15.32:0/15032027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb68007640 con 0x7feb741591e0 2023-12-20T05:44:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb7932c640 1 -- 172.21.15.32:0/15032027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb74156200 con 0x7feb741591e0 2023-12-20T05:44:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb5b7fe640 1 -- 172.21.15.32:0/15032027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb68004a10 con 0x7feb741591e0 2023-12-20T05:44:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb7932c640 1 -- 172.21.15.32:0/15032027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb74158740 con 0x7feb741591e0 2023-12-20T05:44:12.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.064+0000 7feb7932c640 1 -- 172.21.15.32:0/15032027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb74156720 con 0x7feb741591e0 2023-12-20T05:44:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.068+0000 7feb5b7fe640 1 -- 172.21.15.32:0/15032027 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7feb68004b70 con 0x7feb741591e0 2023-12-20T05:44:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.068+0000 7feb5b7fe640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb60065b80 0x7feb60068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.068+0000 7feb5b7fe640 1 -- 172.21.15.32:0/15032027 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7feb680828e0 con 0x7feb741591e0 2023-12-20T05:44:12.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.068+0000 7feb72ffd640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb60065b80 0x7feb60068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.068+0000 7feb72ffd640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb60065b80 0x7feb60068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb640af000 tx=0x7feb640a7070 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:12.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.072+0000 7feb5b7fe640 1 -- 172.21.15.32:0/15032027 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feb74156720 con 0x7feb741591e0 2023-12-20T05:44:12.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.236+0000 7feb7932c640 1 -- 172.21.15.32:0/15032027 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7feb74156720 con 0x7feb741591e0 2023-12-20T05:44:12.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.240+0000 7feb5b7fe640 1 -- 172.21.15.32:0/15032027 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v56) v1 ==== 115+0+0 (secure 0 0 0) 0x7feb74156720 con 0x7feb741591e0 2023-12-20T05:44:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 -- 172.21.15.32:0/15032027 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb60065b80 msgr2=0x7feb60068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb60065b80 0x7feb60068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7feb640af000 tx=0x7feb640a7070 comp rx=0 tx=0).stop 2023-12-20T05:44:12.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 -- 172.21.15.32:0/15032027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb741591e0 msgr2=0x7feb74156a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb741591e0 0x7feb74156a00 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7feb680011f0 tx=0x7feb6800c9b0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 -- 172.21.15.32:0/15032027 shutdown_connections 2023-12-20T05:44:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feb60065b80 0x7feb60068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb7414dda0 0x7feb74157c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb741591e0 0x7feb74156a00 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 --2- 172.21.15.32:0/15032027 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb74070940 0x7feb74155380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 -- 172.21.15.32:0/15032027 >> 172.21.15.32:0/15032027 conn(0x7feb7406a510 msgr2=0x7feb740cd170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 -- 172.21.15.32:0/15032027 shutdown_connections 2023-12-20T05:44:12.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.244+0000 7feb7932c640 1 -- 172.21.15.32:0/15032027 wait complete. 2023-12-20T05:44:12.262 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T05:44:12.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fe6a66640 1 Processor -- start 2023-12-20T05:44:12.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fe6a66640 1 -- start start 2023-12-20T05:44:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fe6a66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2fe00b0760 0x7f2fe00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fe6a66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe00b1100 con 0x7f2fe00cc4b0 2023-12-20T05:44:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fe6a66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2fe00cde20 con 0x7f2fe0157480 2023-12-20T05:44:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fe6a66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe00c9f50 con 0x7f2fe00b0760 2023-12-20T05:44:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fe4802640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2fe0157480 0x7f2fe00c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:40814/0 (socket says 172.21.15.32:40814) 2023-12-20T05:44:12.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fe4802640 1 -- 172.21.15.32:0/1262933677 learned_addr learned my addr 172.21.15.32:0/1262933677 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:12.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fdffff640 1 --2- 172.21.15.32:0/1262933677 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2fe00b0760 0x7f2fe00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fdf7fe640 1 -- 172.21.15.32:0/1262933677 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3179724506 0 0) 0x7f2fe00cde20 con 0x7f2fe0157480 2023-12-20T05:44:12.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fdf7fe640 1 -- 172.21.15.32:0/1262933677 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fd0003530 con 0x7f2fe0157480 2023-12-20T05:44:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fdf7fe640 1 -- 172.21.15.32:0/1262933677 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098523653 0 0) 0x7f2fe00b1100 con 0x7f2fe00cc4b0 2023-12-20T05:44:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fdf7fe640 1 -- 172.21.15.32:0/1262933677 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2fe00cde20 con 0x7f2fe00cc4b0 2023-12-20T05:44:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fdffff640 1 -- 172.21.15.32:0/1262933677 >> v1:172.21.15.39:6789/0 conn(0x7f2fe0157480 legacy=0x7f2fe00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fdffff640 1 -- 172.21.15.32:0/1262933677 >> v1:172.21.15.32:6789/0 conn(0x7f2fe00cc4b0 legacy=0x7f2fe00cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.412+0000 7f2fdffff640 1 -- 172.21.15.32:0/1262933677 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fe00c8e30 con 0x7f2fe00b0760 2023-12-20T05:44:12.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.416+0000 7f2fdffff640 1 --2- 172.21.15.32:0/1262933677 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2fe00b0760 0x7f2fe00b0bc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0097f0 tx=0x7f2fcc01c400 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c6f0c205b4e442a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.416+0000 7f2fdf7fe640 1 -- 172.21.15.32:0/1262933677 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0ab600 con 0x7f2fe00b0760 2023-12-20T05:44:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.416+0000 7f2fdf7fe640 1 -- 172.21.15.32:0/1262933677 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fcc0ab760 con 0x7f2fe00b0760 2023-12-20T05:44:12.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.416+0000 7f2fdf7fe640 1 -- 172.21.15.32:0/1262933677 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fcc0abaf0 con 0x7f2fe00b0760 2023-12-20T05:44:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.416+0000 7f2fe6a66640 1 -- 172.21.15.32:0/1262933677 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2fe00b0760 msgr2=0x7f2fe00b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.416+0000 7f2fe6a66640 1 --2- 172.21.15.32:0/1262933677 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2fe00b0760 0x7f2fe00b0bc0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0097f0 tx=0x7f2fcc01c400 comp rx=0 tx=0).stop 2023-12-20T05:44:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.416+0000 7f2fe6a66640 1 -- 172.21.15.32:0/1262933677 shutdown_connections 2023-12-20T05:44:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.416+0000 7f2fe6a66640 1 --2- 172.21.15.32:0/1262933677 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2fe00b0760 0x7f2fe00b0bc0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.416+0000 7f2fe6a66640 1 -- 172.21.15.32:0/1262933677 >> 172.21.15.32:0/1262933677 conn(0x7f2fe006a510 msgr2=0x7f2fe00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.416+0000 7f2fe6a66640 1 -- 172.21.15.32:0/1262933677 shutdown_connections 2023-12-20T05:44:12.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.416+0000 7f2fe6a66640 1 -- 172.21.15.32:0/1262933677 wait complete. 2023-12-20T05:44:12.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe6a66640 1 Processor -- start 2023-12-20T05:44:12.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe6a66640 1 -- start start 2023-12-20T05:44:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe6a66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe00b0760 0x7f2fe014ed40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe6a66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fe00cc4b0 0x7f2fe014dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe6a66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2fe0157480 0x7f2fe014c2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe6a66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe014cae0 con 0x7f2fe00b0760 2023-12-20T05:44:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe6a66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe014f280 con 0x7f2fe00cc4b0 2023-12-20T05:44:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe6a66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2fe014ccf0 con 0x7f2fe0157480 2023-12-20T05:44:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe4802640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe00b0760 0x7f2fe014ed40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe4802640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe00b0760 0x7f2fe014ed40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37592/0 (socket says 172.21.15.32:37592) 2023-12-20T05:44:12.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe4802640 1 -- 172.21.15.32:0/3430209135 learned_addr learned my addr 172.21.15.32:0/3430209135 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fdffff640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fe00cc4b0 0x7f2fe014dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe5003640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2fe0157480 0x7f2fe014c2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe4802640 1 -- 172.21.15.32:0/3430209135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2fe0157480 msgr2=0x7f2fe014c2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe4802640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2fe0157480 0x7f2fe014c2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe4802640 1 -- 172.21.15.32:0/3430209135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fe00cc4b0 msgr2=0x7f2fe014dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe4802640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fe00cc4b0 0x7f2fe014dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe4802640 1 -- 172.21.15.32:0/3430209135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fcc009420 con 0x7f2fe00b0760 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe4802640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe00b0760 0x7f2fe014ed40 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c4d0 tx=0x7f2fd400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fdd7fa640 1 -- 172.21.15.32:0/3430209135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd4005cf0 con 0x7f2fe00b0760 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fdd7fa640 1 -- 172.21.15.32:0/3430209135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fd4005e50 con 0x7f2fe00b0760 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fdd7fa640 1 -- 172.21.15.32:0/3430209135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fd40072b0 con 0x7f2fe00b0760 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe6a66640 1 -- 172.21.15.32:0/3430209135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2fe014cfd0 con 0x7f2fe00b0760 2023-12-20T05:44:12.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.420+0000 7f2fe6a66640 1 -- 172.21.15.32:0/3430209135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2fe014d590 con 0x7f2fe00b0760 2023-12-20T05:44:12.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.424+0000 7f2fe6a66640 1 -- 172.21.15.32:0/3430209135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2fe004e000 con 0x7f2fe00b0760 2023-12-20T05:44:12.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.424+0000 7f2fdd7fa640 1 -- 172.21.15.32:0/3430209135 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2fd4007410 con 0x7f2fe00b0760 2023-12-20T05:44:12.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.424+0000 7f2fdd7fa640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2fc4065b80 0x7f2fc4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.424+0000 7f2fdd7fa640 1 -- 172.21.15.32:0/3430209135 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f2fd4082760 con 0x7f2fe00b0760 2023-12-20T05:44:12.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.424+0000 7f2fdffff640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2fc4065b80 0x7f2fc4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.424+0000 7f2fdffff640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2fc4065b80 0x7f2fc4068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0af000 tx=0x7f2fcc0a7000 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:12.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.428+0000 7f2fdd7fa640 1 -- 172.21.15.32:0/3430209135 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2fd40501d0 con 0x7f2fe00b0760 2023-12-20T05:44:12.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.595+0000 7f2fe6a66640 1 -- 172.21.15.32:0/3430209135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f2fe00ca670 con 0x7f2fe00b0760 2023-12-20T05:44:12.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.599+0000 7f2fdd7fa640 1 -- 172.21.15.32:0/3430209135 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v57) v1 ==== 104+0+0 (secure 0 0 0) 0x7f2fd40537f0 con 0x7f2fe00b0760 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 -- 172.21.15.32:0/3430209135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2fc4065b80 msgr2=0x7f2fc4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2fc4065b80 0x7f2fc4068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f2fcc0af000 tx=0x7f2fcc0a7000 comp rx=0 tx=0).stop 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 -- 172.21.15.32:0/3430209135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe00b0760 msgr2=0x7f2fe014ed40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe00b0760 0x7f2fe014ed40 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f2fd400c4d0 tx=0x7f2fd400c990 comp rx=0 tx=0).stop 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 -- 172.21.15.32:0/3430209135 shutdown_connections 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f2fc4065b80 0x7f2fc4068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2fe0157480 0x7f2fe014c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2fe00cc4b0 0x7f2fe014dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 --2- 172.21.15.32:0/3430209135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2fe00b0760 0x7f2fe014ed40 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 -- 172.21.15.32:0/3430209135 >> 172.21.15.32:0/3430209135 conn(0x7f2fe006a510 msgr2=0x7f2fe00c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 -- 172.21.15.32:0/3430209135 shutdown_connections 2023-12-20T05:44:12.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.603+0000 7f2fe6a66640 1 -- 172.21.15.32:0/3430209135 wait complete. 2023-12-20T05:44:12.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove' 2023-12-20T05:44:12.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2c5e66640 1 Processor -- start 2023-12-20T05:44:12.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2c5e66640 1 -- start start 2023-12-20T05:44:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2c5e66640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff2c00af5e0 0x7ff2c00af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2c5e66640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff2c006ce60 0x7ff2c015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2c5e66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2c015bbd0 con 0x7ff2c015f110 2023-12-20T05:44:12.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2c5e66640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c015beb0 con 0x7ff2c00af5e0 2023-12-20T05:44:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2c5e66640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2c015e910 con 0x7ff2c006ce60 2023-12-20T05:44:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2bf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff2c00af5e0 0x7ff2c00af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2bf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff2c00af5e0 0x7ff2c00af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37606/0 (socket says 172.21.15.32:37606) 2023-12-20T05:44:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2bf7fe640 1 -- 172.21.15.32:0/3196036571 learned_addr learned my addr 172.21.15.32:0/3196036571 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2beffd640 1 --2- 172.21.15.32:0/3196036571 >> v2:172.21.15.39:3300/0 conn(0x7ff2c006ce60 0x7ff2c015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.767+0000 7ff2bf7fe640 1 -- 172.21.15.32:0/3196036571 >> v2:172.21.15.39:3300/0 conn(0x7ff2c006ce60 msgr2=0x7ff2c015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2bf7fe640 1 --2- 172.21.15.32:0/3196036571 >> v2:172.21.15.39:3300/0 conn(0x7ff2c006ce60 0x7ff2c015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2bf7fe640 1 -- 172.21.15.32:0/3196036571 >> v1:172.21.15.39:6789/0 conn(0x7ff2c015f110 legacy=0x7ff2c015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:12.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2bf7fe640 1 -- 172.21.15.32:0/3196036571 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2c015ebf0 con 0x7ff2c00af5e0 2023-12-20T05:44:12.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2bf7fe640 1 --2- 172.21.15.32:0/3196036571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c00af5e0 0x7ff2c00af9c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff2c006abe0 tx=0x7ff2b001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7d48a74e64db52b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:12.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2be7fc640 1 -- 172.21.15.32:0/3196036571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00ab620 con 0x7ff2c00af5e0 2023-12-20T05:44:12.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2be7fc640 1 -- 172.21.15.32:0/3196036571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b00ab780 con 0x7ff2c00af5e0 2023-12-20T05:44:12.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2be7fc640 1 -- 172.21.15.32:0/3196036571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00abb10 con 0x7ff2c00af5e0 2023-12-20T05:44:12.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2c5e66640 1 -- 172.21.15.32:0/3196036571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c00af5e0 msgr2=0x7ff2c00af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2c5e66640 1 --2- 172.21.15.32:0/3196036571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c00af5e0 0x7ff2c00af9c0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7ff2c006abe0 tx=0x7ff2b001c4c0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2c5e66640 1 -- 172.21.15.32:0/3196036571 shutdown_connections 2023-12-20T05:44:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2c5e66640 1 --2- 172.21.15.32:0/3196036571 >> v2:172.21.15.39:3300/0 conn(0x7ff2c006ce60 0x7ff2c015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2c5e66640 1 --2- 172.21.15.32:0/3196036571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c00af5e0 0x7ff2c00af9c0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2c5e66640 1 -- 172.21.15.32:0/3196036571 >> 172.21.15.32:0/3196036571 conn(0x7ff2c006a320 msgr2=0x7ff2c00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2c5e66640 1 -- 172.21.15.32:0/3196036571 shutdown_connections 2023-12-20T05:44:12.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.771+0000 7ff2c5e66640 1 -- 172.21.15.32:0/3196036571 wait complete. 2023-12-20T05:44:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2c5e66640 1 Processor -- start 2023-12-20T05:44:12.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2c5e66640 1 -- start start 2023-12-20T05:44:12.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2c5e66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c006ce60 0x7ff2c00c4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2c5e66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2c00af5e0 0x7ff2c014ce30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2c5e66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2c015f110 0x7ff2c014d370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2c5e66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c015cd50 con 0x7ff2c006ce60 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2c5e66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c015e280 con 0x7ff2c015f110 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2c5e66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2c015c420 con 0x7ff2c00af5e0 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c006ce60 0x7ff2c00c4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2bf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c006ce60 0x7ff2c00c4a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37620/0 (socket says 172.21.15.32:37620) 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2bf7fe640 1 -- 172.21.15.32:0/1526320195 learned_addr learned my addr 172.21.15.32:0/1526320195 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2beffd640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2c00af5e0 0x7ff2c014ce30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2bffff640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2c015f110 0x7ff2c014d370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2bf7fe640 1 -- 172.21.15.32:0/1526320195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2c00af5e0 msgr2=0x7ff2c014ce30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2bf7fe640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2c00af5e0 0x7ff2c014ce30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2bf7fe640 1 -- 172.21.15.32:0/1526320195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2c015f110 msgr2=0x7ff2c014d370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2bf7fe640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2c015f110 0x7ff2c014d370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:12.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2bf7fe640 1 -- 172.21.15.32:0/1526320195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2b0009480 con 0x7ff2c006ce60 2023-12-20T05:44:12.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2bffff640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2c015f110 0x7ff2c014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:12.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2bf7fe640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c006ce60 0x7ff2c00c4a60 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff2b001ab70 tx=0x7ff2b0019260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:12.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2a7fff640 1 -- 172.21.15.32:0/1526320195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b00059a0 con 0x7ff2c006ce60 2023-12-20T05:44:12.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2a7fff640 1 -- 172.21.15.32:0/1526320195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2b0005b00 con 0x7ff2c006ce60 2023-12-20T05:44:12.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2a7fff640 1 -- 172.21.15.32:0/1526320195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2b0005e90 con 0x7ff2c006ce60 2023-12-20T05:44:12.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.775+0000 7ff2c5e66640 1 -- 172.21.15.32:0/1526320195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c015c700 con 0x7ff2c006ce60 2023-12-20T05:44:12.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.779+0000 7ff2c5e66640 1 -- 172.21.15.32:0/1526320195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2c00c4fa0 con 0x7ff2c006ce60 2023-12-20T05:44:12.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.779+0000 7ff2c5e66640 1 -- 172.21.15.32:0/1526320195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2c004e000 con 0x7ff2c006ce60 2023-12-20T05:44:12.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.779+0000 7ff2a7fff640 1 -- 172.21.15.32:0/1526320195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff2b00ab8f0 con 0x7ff2c006ce60 2023-12-20T05:44:12.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.779+0000 7ff2a7fff640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2ac065b80 0x7ff2ac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:12.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.779+0000 7ff2a7fff640 1 -- 172.21.15.32:0/1526320195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7ff2b011cfa0 con 0x7ff2c006ce60 2023-12-20T05:44:12.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.779+0000 7ff2beffd640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2ac065b80 0x7ff2ac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:12.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.779+0000 7ff2beffd640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2ac065b80 0x7ff2ac068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00c7120 tx=0x7ff2b4008040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:12.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.783+0000 7ff2a7fff640 1 -- 172.21.15.32:0/1526320195 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff2b00ea730 con 0x7ff2c006ce60 2023-12-20T05:44:12.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:12.999+0000 7ff2c5e66640 1 -- 172.21.15.32:0/1526320195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]} v 0) v1 -- 0x7ff2c00cf450 con 0x7ff2c006ce60 2023-12-20T05:44:13.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.543+0000 7ff2a7fff640 1 -- 172.21.15.32:0/1526320195 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_list_post_schedule_remove"]}]=0 v2451) v1 ==== 174+0+0 (secure 0 0 0) 0x7ff2b00edd50 con 0x7ff2c006ce60 2023-12-20T05:44:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 -- 172.21.15.32:0/1526320195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2ac065b80 msgr2=0x7ff2ac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2ac065b80 0x7ff2ac068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7ff2c00c7120 tx=0x7ff2b4008040 comp rx=0 tx=0).stop 2023-12-20T05:44:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 -- 172.21.15.32:0/1526320195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c006ce60 msgr2=0x7ff2c00c4a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c006ce60 0x7ff2c00c4a60 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7ff2b001ab70 tx=0x7ff2b0019260 comp rx=0 tx=0).stop 2023-12-20T05:44:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 -- 172.21.15.32:0/1526320195 shutdown_connections 2023-12-20T05:44:13.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7ff2ac065b80 0x7ff2ac068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2c015f110 0x7ff2c014d370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2c00af5e0 0x7ff2c014ce30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 --2- 172.21.15.32:0/1526320195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2c006ce60 0x7ff2c00c4a60 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 -- 172.21.15.32:0/1526320195 >> 172.21.15.32:0/1526320195 conn(0x7ff2c006a320 msgr2=0x7ff2c00ccf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 -- 172.21.15.32:0/1526320195 shutdown_connections 2023-12-20T05:44:13.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.547+0000 7ff2c5e66640 1 -- 172.21.15.32:0/1526320195 wait complete. 2023-12-20T05:44:13.570 INFO:tasks.cephfs_test_runner:Test listing snap schedules post removal of a schedule ... ok 2023-12-20T05:44:13.570 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test that snap schedule is resumed after mgr restart 2023-12-20T05:44:13.570 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_mgr_restart (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-20T05:44:13.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-12-20T05:44:13.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.719+0000 7f6c7a925640 1 Processor -- start 2023-12-20T05:44:13.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.719+0000 7f6c7a925640 1 -- start start 2023-12-20T05:44:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.719+0000 7f6c7a925640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c741593c0 0x7f6c7415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.719+0000 7f6c7a925640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c7406caf0 0x7f6c740705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.719+0000 7f6c7a925640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c740ae140 con 0x7f6c740afdb0 2023-12-20T05:44:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.719+0000 7f6c7a925640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c740ae9e0 con 0x7f6c7406caf0 2023-12-20T05:44:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.719+0000 7f6c7a925640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7415dee0 con 0x7f6c741593c0 2023-12-20T05:44:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c78ec2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c7406caf0 0x7f6c740705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c78ec2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c7406caf0 0x7f6c740705a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37636/0 (socket says 172.21.15.32:37636) 2023-12-20T05:44:13.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c78ec2640 1 -- 172.21.15.32:0/1985055737 learned_addr learned my addr 172.21.15.32:0/1985055737 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c73fff640 1 --2- 172.21.15.32:0/1985055737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c741593c0 0x7f6c7415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c78ec2640 1 -- 172.21.15.32:0/1985055737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c741593c0 msgr2=0x7f6c7415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c78ec2640 1 --2- 172.21.15.32:0/1985055737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c741593c0 0x7f6c7415d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c78ec2640 1 -- 172.21.15.32:0/1985055737 >> v1:172.21.15.39:6789/0 conn(0x7f6c740afdb0 legacy=0x7f6c7406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c78ec2640 1 -- 172.21.15.32:0/1985055737 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c74072170 con 0x7f6c7406caf0 2023-12-20T05:44:13.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c73fff640 1 --2- 172.21.15.32:0/1985055737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c741593c0 0x7f6c7415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c78ec2640 1 --2- 172.21.15.32:0/1985055737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7406caf0 0x7f6c740705a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009e50 tx=0x7f6c6c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8b3d22ae8898c14 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c72ffd640 1 -- 172.21.15.32:0/1985055737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0a5790 con 0x7f6c7406caf0 2023-12-20T05:44:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c72ffd640 1 -- 172.21.15.32:0/1985055737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c6c0a58f0 con 0x7f6c7406caf0 2023-12-20T05:44:13.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.723+0000 7f6c72ffd640 1 -- 172.21.15.32:0/1985055737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c6c0a5c80 con 0x7f6c7406caf0 2023-12-20T05:44:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 -- 172.21.15.32:0/1985055737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7406caf0 msgr2=0x7f6c740705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 --2- 172.21.15.32:0/1985055737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7406caf0 0x7f6c740705a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f6c6c009e50 tx=0x7f6c6c01c950 comp rx=0 tx=0).stop 2023-12-20T05:44:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 -- 172.21.15.32:0/1985055737 shutdown_connections 2023-12-20T05:44:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 --2- 172.21.15.32:0/1985055737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7406caf0 0x7f6c740705a0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 --2- 172.21.15.32:0/1985055737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c741593c0 0x7f6c7415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 -- 172.21.15.32:0/1985055737 >> 172.21.15.32:0/1985055737 conn(0x7f6c7406a320 msgr2=0x7f6c740ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 -- 172.21.15.32:0/1985055737 shutdown_connections 2023-12-20T05:44:13.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 -- 172.21.15.32:0/1985055737 wait complete. 2023-12-20T05:44:13.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 Processor -- start 2023-12-20T05:44:13.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 -- start start 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7406caf0 0x7f6c74158cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c740afdb0 0x7f6c7414a340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c741593c0 0x7f6c74149ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c74157890 con 0x7f6c7406caf0 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c74159200 con 0x7f6c740afdb0 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c7a925640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7414f920 con 0x7f6c741593c0 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c73fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7406caf0 0x7f6c74158cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c73fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7406caf0 0x7f6c74158cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37650/0 (socket says 172.21.15.32:37650) 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c73fff640 1 -- 172.21.15.32:0/2523144945 learned_addr learned my addr 172.21.15.32:0/2523144945 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c737fe640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c740afdb0 0x7f6c7414a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.727+0000 7f6c78ec2640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c741593c0 0x7f6c74149ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:13.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c73fff640 1 -- 172.21.15.32:0/2523144945 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c741593c0 msgr2=0x7f6c74149ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:13.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c73fff640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c741593c0 0x7f6c74149ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:13.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c73fff640 1 -- 172.21.15.32:0/2523144945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c740afdb0 msgr2=0x7f6c7414a340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:13.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c73fff640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c740afdb0 0x7f6c7414a340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:13.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c73fff640 1 -- 172.21.15.32:0/2523144945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c6c009b30 con 0x7f6c7406caf0 2023-12-20T05:44:13.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c73fff640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7406caf0 0x7f6c74158cc0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e730 tx=0x7f6c6400ebf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:13.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c70ff9640 1 -- 172.21.15.32:0/2523144945 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64005ba0 con 0x7f6c7406caf0 2023-12-20T05:44:13.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c70ff9640 1 -- 172.21.15.32:0/2523144945 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c64005d00 con 0x7f6c7406caf0 2023-12-20T05:44:13.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c70ff9640 1 -- 172.21.15.32:0/2523144945 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c64004250 con 0x7f6c7406caf0 2023-12-20T05:44:13.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c7a925640 1 -- 172.21.15.32:0/2523144945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7414fc00 con 0x7f6c7406caf0 2023-12-20T05:44:13.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c7a925640 1 -- 172.21.15.32:0/2523144945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c741501c0 con 0x7f6c7406caf0 2023-12-20T05:44:13.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c7a925640 1 -- 172.21.15.32:0/2523144945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7404e000 con 0x7f6c7406caf0 2023-12-20T05:44:13.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c70ff9640 1 -- 172.21.15.32:0/2523144945 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6c640043b0 con 0x7f6c7406caf0 2023-12-20T05:44:13.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c70ff9640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c60065b80 0x7f6c60068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:13.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.731+0000 7f6c70ff9640 1 -- 172.21.15.32:0/2523144945 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f6c64083280 con 0x7f6c7406caf0 2023-12-20T05:44:13.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.735+0000 7f6c737fe640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c60065b80 0x7f6c60068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:13.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.735+0000 7f6c737fe640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c60065b80 0x7f6c60068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6c740c8c80 tx=0x7f6c68009040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:13.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.735+0000 7f6c70ff9640 1 -- 172.21.15.32:0/2523144945 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c64050ab0 con 0x7f6c7406caf0 2023-12-20T05:44:13.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:13.951+0000 7f6c7a925640 1 -- 172.21.15.32:0/2523144945 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f6c740ca5f0 con 0x7f6c7406caf0 2023-12-20T05:44:14.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.547+0000 7f6c70ff9640 1 -- 172.21.15.32:0/2523144945 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2452) v1 ==== 168+0+0 (secure 0 0 0) 0x7f6c640540d0 con 0x7f6c7406caf0 2023-12-20T05:44:14.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 -- 172.21.15.32:0/2523144945 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c60065b80 msgr2=0x7f6c60068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:14.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c60065b80 0x7f6c60068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f6c740c8c80 tx=0x7f6c68009040 comp rx=0 tx=0).stop 2023-12-20T05:44:14.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 -- 172.21.15.32:0/2523144945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7406caf0 msgr2=0x7f6c74158cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:14.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7406caf0 0x7f6c74158cc0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f6c6400e730 tx=0x7f6c6400ebf0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 -- 172.21.15.32:0/2523144945 shutdown_connections 2023-12-20T05:44:14.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6c60065b80 0x7f6c60068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c741593c0 0x7f6c74149ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c740afdb0 0x7f6c7414a340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 --2- 172.21.15.32:0/2523144945 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7406caf0 0x7f6c74158cc0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 -- 172.21.15.32:0/2523144945 >> 172.21.15.32:0/2523144945 conn(0x7f6c7406a320 msgr2=0x7f6c7415bda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:14.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 -- 172.21.15.32:0/2523144945 shutdown_connections 2023-12-20T05:44:14.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.551+0000 7f6c7a925640 1 -- 172.21.15.32:0/2523144945 wait complete. 2023-12-20T05:44:14.570 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T05:44:14.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.723+0000 7f2551266640 1 Processor -- start 2023-12-20T05:44:14.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.723+0000 7f2551266640 1 -- start start 2023-12-20T05:44:14.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.723+0000 7f2551266640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f254c1591e0 0x7f254c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:14.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.723+0000 7f2551266640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254c06cdf0 0x7f254c070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.723+0000 7f2551266640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f254c0cc3b0 con 0x7f254c070770 2023-12-20T05:44:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.723+0000 7f2551266640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f254c070be0 con 0x7f254c1591e0 2023-12-20T05:44:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.723+0000 7f2551266640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c0c8e30 con 0x7f254c06cdf0 2023-12-20T05:44:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.723+0000 7f254affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f254c1591e0 0x7f254c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.723+0000 7f254affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f254c1591e0 0x7f254c0caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37662/0 (socket says 172.21.15.32:37662) 2023-12-20T05:44:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.723+0000 7f254affd640 1 -- 172.21.15.32:0/1136270087 learned_addr learned my addr 172.21.15.32:0/1136270087 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.727+0000 7f2543fff640 1 --2- 172.21.15.32:0/1136270087 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254c06cdf0 0x7f254c070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.727+0000 7f254affd640 1 -- 172.21.15.32:0/1136270087 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254c06cdf0 msgr2=0x7f254c070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.727+0000 7f254affd640 1 --2- 172.21.15.32:0/1136270087 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254c06cdf0 0x7f254c070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.727+0000 7f254affd640 1 -- 172.21.15.32:0/1136270087 >> v1:172.21.15.39:6789/0 conn(0x7f254c070770 legacy=0x7f254c15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:14.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.727+0000 7f254affd640 1 -- 172.21.15.32:0/1136270087 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f254c06d280 con 0x7f254c1591e0 2023-12-20T05:44:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.727+0000 7f254affd640 1 --2- 172.21.15.32:0/1136270087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c1591e0 0x7f254c0caa30 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f253c0098a0 tx=0x7f253c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7bf1782a64842a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.727+0000 7f254a7fc640 1 -- 172.21.15.32:0/1136270087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c0ab600 con 0x7f254c1591e0 2023-12-20T05:44:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.727+0000 7f254a7fc640 1 -- 172.21.15.32:0/1136270087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f253c0a7050 con 0x7f254c1591e0 2023-12-20T05:44:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.727+0000 7f254a7fc640 1 -- 172.21.15.32:0/1136270087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f253c0ab980 con 0x7f254c1591e0 2023-12-20T05:44:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 -- 172.21.15.32:0/1136270087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c1591e0 msgr2=0x7f254c0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 --2- 172.21.15.32:0/1136270087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c1591e0 0x7f254c0caa30 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f253c0098a0 tx=0x7f253c01c490 comp rx=0 tx=0).stop 2023-12-20T05:44:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 -- 172.21.15.32:0/1136270087 shutdown_connections 2023-12-20T05:44:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 --2- 172.21.15.32:0/1136270087 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254c06cdf0 0x7f254c070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 --2- 172.21.15.32:0/1136270087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c1591e0 0x7f254c0caa30 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 -- 172.21.15.32:0/1136270087 >> 172.21.15.32:0/1136270087 conn(0x7f254c06a510 msgr2=0x7f254c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:14.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 -- 172.21.15.32:0/1136270087 shutdown_connections 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 -- 172.21.15.32:0/1136270087 wait complete. 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 Processor -- start 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 -- start start 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f254c070770 0x7f254c167e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254c1591e0 0x7f254c14afe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c14b520 0x7f254c1494f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c158a90 con 0x7f254c14b520 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f254c14ab50 con 0x7f254c070770 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2551266640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f254c149f40 con 0x7f254c1591e0 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f254b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c14b520 0x7f254c1494f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f254b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c14b520 0x7f254c1494f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37676/0 (socket says 172.21.15.32:37676) 2023-12-20T05:44:14.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f254b7fe640 1 -- 172.21.15.32:0/929018863 learned_addr learned my addr 172.21.15.32:0/929018863 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f2543fff640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254c1591e0 0x7f254c14afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f254affd640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f254c070770 0x7f254c167e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f254b7fe640 1 -- 172.21.15.32:0/929018863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254c1591e0 msgr2=0x7f254c14afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f254b7fe640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254c1591e0 0x7f254c14afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f254b7fe640 1 -- 172.21.15.32:0/929018863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f254c070770 msgr2=0x7f254c167e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f254b7fe640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f254c070770 0x7f254c167e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f254b7fe640 1 -- 172.21.15.32:0/929018863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f253c0094d0 con 0x7f254c14b520 2023-12-20T05:44:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f254affd640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f254c070770 0x7f254c167e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:14.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.731+0000 7f254b7fe640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c14b520 0x7f254c1494f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f254400c770 tx=0x7f254400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.735+0000 7f25437fe640 1 -- 172.21.15.32:0/929018863 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2544003b80 con 0x7f254c14b520 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.735+0000 7f25437fe640 1 -- 172.21.15.32:0/929018863 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2544003ce0 con 0x7f254c14b520 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.735+0000 7f25437fe640 1 -- 172.21.15.32:0/929018863 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f254400aa90 con 0x7f254c14b520 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.735+0000 7f2551266640 1 -- 172.21.15.32:0/929018863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f254c14a220 con 0x7f254c14b520 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.735+0000 7f2551266640 1 -- 172.21.15.32:0/929018863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f254c14a7e0 con 0x7f254c14b520 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.735+0000 7f2551266640 1 -- 172.21.15.32:0/929018863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f254c04e000 con 0x7f254c14b520 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.735+0000 7f25437fe640 1 -- 172.21.15.32:0/929018863 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f254400abf0 con 0x7f254c14b520 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.735+0000 7f25437fe640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f252c065b80 0x7f252c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.735+0000 7f25437fe640 1 -- 172.21.15.32:0/929018863 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f25440830d0 con 0x7f254c14b520 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.735+0000 7f254affd640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f252c065b80 0x7f252c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.735+0000 7f254affd640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f252c065b80 0x7f252c068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f253c0ad000 tx=0x7f253c0a4f90 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:14.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.739+0000 7f25437fe640 1 -- 172.21.15.32:0/929018863 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2544050a00 con 0x7f254c14b520 2023-12-20T05:44:14.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.895+0000 7f2551266640 1 -- 172.21.15.32:0/929018863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f254c14ac90 con 0x7f254c14b520 2023-12-20T05:44:14.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f25437fe640 1 -- 172.21.15.32:0/929018863 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v58)=0 v58) v1 ==== 122+0+0 (secure 0 0 0) 0x7f2544054020 con 0x7f254c14b520 2023-12-20T05:44:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 -- 172.21.15.32:0/929018863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f252c065b80 msgr2=0x7f252c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:14.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f252c065b80 0x7f252c068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f253c0ad000 tx=0x7f253c0a4f90 comp rx=0 tx=0).stop 2023-12-20T05:44:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 -- 172.21.15.32:0/929018863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c14b520 msgr2=0x7f254c1494f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c14b520 0x7f254c1494f0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f254400c770 tx=0x7f254400cc30 comp rx=0 tx=0).stop 2023-12-20T05:44:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 -- 172.21.15.32:0/929018863 shutdown_connections 2023-12-20T05:44:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f252c065b80 0x7f252c068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f254c14b520 0x7f254c1494f0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254c1591e0 0x7f254c14afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 --2- 172.21.15.32:0/929018863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f254c070770 0x7f254c167e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 -- 172.21.15.32:0/929018863 >> 172.21.15.32:0/929018863 conn(0x7f254c06a510 msgr2=0x7f254c0c4990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 -- 172.21.15.32:0/929018863 shutdown_connections 2023-12-20T05:44:14.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:14.903+0000 7f2551266640 1 -- 172.21.15.32:0/929018863 wait complete. 2023-12-20T05:44:14.923 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T05:44:14.923 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T05:44:15.002 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T05:44:15.088 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T05:44:15.088 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:44:15.088 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:44:15.099 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T05:44:15.099 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T05:44:15.099 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:44:15.110 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T05:44:15.110 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:44:15.110 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:44:15.152 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T05:44:15.153 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T05:44:15.153 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:44:15.167 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T05:44:15.167 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T05:44:15.167 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:44:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.339+0000 7fe6f6b6d640 1 Processor -- start 2023-12-20T05:44:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.339+0000 7fe6f6b6d640 1 -- start start 2023-12-20T05:44:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6f6b6d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6f01599e0 0x7fe6f00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6f6b6d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe6f00b0d00 0x7fe6f00701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6f6b6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe6f00c98c0 con 0x7fe6f00c90e0 2023-12-20T05:44:15.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6f6b6d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f00c9ba0 con 0x7fe6f01599e0 2023-12-20T05:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6f6b6d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe6f00c9e80 con 0x7fe6f00b0d00 2023-12-20T05:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6f4909640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe6f01599e0 0x7fe6f00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6effff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe6f00c90e0 0x7fe6f00b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56158/0 (socket says 172.21.15.32:56158) 2023-12-20T05:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6effff640 1 -- 172.21.15.32:0/865025629 learned_addr learned my addr 172.21.15.32:0/865025629 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6f510a640 1 --2- 172.21.15.32:0/865025629 >> v2:172.21.15.39:3300/0 conn(0x7fe6f00b0d00 0x7fe6f00701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6f4909640 1 -- 172.21.15.32:0/865025629 >> v2:172.21.15.39:3300/0 conn(0x7fe6f00b0d00 msgr2=0x7fe6f00701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6f4909640 1 --2- 172.21.15.32:0/865025629 >> v2:172.21.15.39:3300/0 conn(0x7fe6f00b0d00 0x7fe6f00701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6f4909640 1 -- 172.21.15.32:0/865025629 >> v1:172.21.15.32:6789/0 conn(0x7fe6f00c90e0 legacy=0x7fe6f00b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6f4909640 1 -- 172.21.15.32:0/865025629 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6f00ca160 con 0x7fe6f01599e0 2023-12-20T05:44:15.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.343+0000 7fe6ef7fe640 1 -- 172.21.15.32:0/865025629 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2128881083 0 0) 0x7fe6f00c98c0 con 0x7fe6f00c90e0 2023-12-20T05:44:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6f4909640 1 --2- 172.21.15.32:0/865025629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f01599e0 0x7fe6f00aece0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00b0060 tx=0x7fe6e401c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a3670c22f71e554 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6ef7fe640 1 -- 172.21.15.32:0/865025629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40ab600 con 0x7fe6f01599e0 2023-12-20T05:44:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6ef7fe640 1 -- 172.21.15.32:0/865025629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e40a7050 con 0x7fe6f01599e0 2023-12-20T05:44:15.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6ef7fe640 1 -- 172.21.15.32:0/865025629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40ab980 con 0x7fe6f01599e0 2023-12-20T05:44:15.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/865025629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f01599e0 msgr2=0x7fe6f00aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6f6b6d640 1 --2- 172.21.15.32:0/865025629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f01599e0 0x7fe6f00aece0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7fe6f00b0060 tx=0x7fe6e401c410 comp rx=0 tx=0).stop 2023-12-20T05:44:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/865025629 shutdown_connections 2023-12-20T05:44:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6f6b6d640 1 --2- 172.21.15.32:0/865025629 >> v2:172.21.15.39:3300/0 conn(0x7fe6f00b0d00 0x7fe6f00701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6f6b6d640 1 --2- 172.21.15.32:0/865025629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f01599e0 0x7fe6f00aece0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/865025629 >> 172.21.15.32:0/865025629 conn(0x7fe6f006a320 msgr2=0x7fe6f00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/865025629 shutdown_connections 2023-12-20T05:44:15.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/865025629 wait complete. 2023-12-20T05:44:15.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6f6b6d640 1 Processor -- start 2023-12-20T05:44:15.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.347+0000 7fe6f6b6d640 1 -- start start 2023-12-20T05:44:15.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f6b6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f00b0d00 0x7fe6f00c6a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f6b6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe6f00c90e0 0x7fe6f00b3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f6b6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6f01599e0 0x7fe6f014e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f6b6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00c8870 con 0x7fe6f00b0d00 2023-12-20T05:44:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f6b6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00c7090 con 0x7fe6f01599e0 2023-12-20T05:44:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f6b6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe6f00c7370 con 0x7fe6f00c90e0 2023-12-20T05:44:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f4909640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f00b0d00 0x7fe6f00c6a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f4909640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f00b0d00 0x7fe6f00c6a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37704/0 (socket says 172.21.15.32:37704) 2023-12-20T05:44:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f4909640 1 -- 172.21.15.32:0/3379869420 learned_addr learned my addr 172.21.15.32:0/3379869420 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6effff640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe6f00c90e0 0x7fe6f00b3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f510a640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6f01599e0 0x7fe6f014e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f4909640 1 -- 172.21.15.32:0/3379869420 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe6f00c90e0 msgr2=0x7fe6f00b3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f4909640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe6f00c90e0 0x7fe6f00b3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f4909640 1 -- 172.21.15.32:0/3379869420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6f01599e0 msgr2=0x7fe6f014e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f4909640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6f01599e0 0x7fe6f014e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f4909640 1 -- 172.21.15.32:0/3379869420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe6e40093b0 con 0x7fe6f00b0d00 2023-12-20T05:44:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f510a640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6f01599e0 0x7fe6f014e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6effff640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe6f00c90e0 0x7fe6f00b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f4909640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f00b0d00 0x7fe6f00c6a20 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fe6e401aac0 tx=0x7fe6e401ac50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6ed7fa640 1 -- 172.21.15.32:0/3379869420 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40b7650 con 0x7fe6f00b0d00 2023-12-20T05:44:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6ed7fa640 1 -- 172.21.15.32:0/3379869420 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe6e40b77d0 con 0x7fe6f00b0d00 2023-12-20T05:44:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6ed7fa640 1 -- 172.21.15.32:0/3379869420 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe6e40b7b80 con 0x7fe6f00b0d00 2023-12-20T05:44:15.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/3379869420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe6f00c7650 con 0x7fe6f00b0d00 2023-12-20T05:44:15.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/3379869420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe6f00c7c10 con 0x7fe6f00b0d00 2023-12-20T05:44:15.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/3379869420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe6f00c7e00 con 0x7fe6f00b0d00 2023-12-20T05:44:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.351+0000 7fe6ed7fa640 1 -- 172.21.15.32:0/3379869420 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe6e40b7ce0 con 0x7fe6f00b0d00 2023-12-20T05:44:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.355+0000 7fe6ed7fa640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6d4065b80 0x7fe6d4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.355+0000 7fe6ed7fa640 1 -- 172.21.15.32:0/3379869420 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(164..164 src has 1..164) v4 ==== 5573+0+0 (secure 0 0 0) 0x7fe6e411dcf0 con 0x7fe6f00b0d00 2023-12-20T05:44:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.355+0000 7fe6effff640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6d4065b80 0x7fe6d4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.355+0000 7fe6effff640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6d4065b80 0x7fe6d4068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc007fd0 tx=0x7fe6dc007c20 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:15.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.359+0000 7fe6ed7fa640 1 -- 172.21.15.32:0/3379869420 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe6e40eb620 con 0x7fe6f00b0d00 2023-12-20T05:44:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.547+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/3379869420 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe6f004e000 con 0x7fe6f00b0d00 2023-12-20T05:44:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6ed7fa640 1 -- 172.21.15.32:0/3379869420 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 73 v73) v1 ==== 94+0+3526 (secure 0 0 0) 0x7fe6e40b0de0 con 0x7fe6f00b0d00 2023-12-20T05:44:15.548 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:15.548 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":73,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/2062259714","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":2062259714},{"type":"v1","addr":"172.21.15.39:6834","nonce":2062259714}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8638,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/4210073096","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":4210073096},{"type":"v1","addr":"172.21.15.39:6835","nonce":4210073096}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":8913,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/3547769022","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3547769022},{"type":"v1","addr":"172.21.15.32:6835","nonce":3547769022}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":65},{"gid":9339,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/788515058","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":788515058},{"type":"v1","addr":"172.21.15.32:6837","nonce":788515058}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":73}],"filesystems":[]} 2023-12-20T05:44:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 73 2023-12-20T05:44:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/3379869420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6d4065b80 msgr2=0x7fe6d4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6d4065b80 0x7fe6d4068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fe6dc007fd0 tx=0x7fe6dc007c20 comp rx=0 tx=0).stop 2023-12-20T05:44:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/3379869420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f00b0d00 msgr2=0x7fe6f00c6a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f00b0d00 0x7fe6f00c6a20 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7fe6e401aac0 tx=0x7fe6e401ac50 comp rx=0 tx=0).stop 2023-12-20T05:44:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/3379869420 shutdown_connections 2023-12-20T05:44:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe6d4065b80 0x7fe6d4068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe6f01599e0 0x7fe6f014e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe6f00c90e0 0x7fe6f00b3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 --2- 172.21.15.32:0/3379869420 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe6f00b0d00 0x7fe6f00c6a20 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/3379869420 >> 172.21.15.32:0/3379869420 conn(0x7fe6f006a320 msgr2=0x7fe6f014fda0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/3379869420 shutdown_connections 2023-12-20T05:44:15.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.551+0000 7fe6f6b6d640 1 -- 172.21.15.32:0/3379869420 wait complete. 2023-12-20T05:44:15.570 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T05:44:15.570 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T05:44:15.570 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T05:44:15.570 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T05:44:15.570 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T05:44:15.570 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T05:44:15.571 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T05:44:15.571 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T05:44:15.571 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T05:44:15.571 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T05:44:15.571 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T05:44:15.571 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T05:44:15.572 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T05:44:15.572 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T05:44:15.572 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T05:44:15.572 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T05:44:15.584 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T05:44:15.584 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T05:44:15.585 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T05:44:15.585 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T05:44:15.587 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T05:44:15.587 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T05:44:15.588 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T05:44:15.588 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T05:44:15.631 INFO:tasks.ceph.mds.a:Started 2023-12-20T05:44:15.631 DEBUG:teuthology.parallel:result is None 2023-12-20T05:44:15.633 INFO:tasks.ceph.mds.c:Started 2023-12-20T05:44:15.633 DEBUG:teuthology.parallel:result is None 2023-12-20T05:44:15.636 INFO:tasks.ceph.mds.b:Started 2023-12-20T05:44:15.637 DEBUG:teuthology.parallel:result is None 2023-12-20T05:44:15.639 INFO:tasks.ceph.mds.d:Started 2023-12-20T05:44:15.639 DEBUG:teuthology.parallel:result is None 2023-12-20T05:44:15.639 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T05:44:15.720 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T05:44:15.721 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T05:44:15.747 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T05:44:15.752 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T05:44:15.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fecd39640 1 Processor -- start 2023-12-20T05:44:15.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fecd39640 1 -- start start 2023-12-20T05:44:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fecd39640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fe815cdb0 0x7f5fe80d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fecd39640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5fe80d2740 0x7f5fe80cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fecd39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fe80d1120 0x7f5fe80cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fecd39640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe815d190 con 0x7f5fe815cdb0 2023-12-20T05:44:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fecd39640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fe80d16a0 con 0x7f5fe80d2740 2023-12-20T05:44:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fecd39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80d1980 con 0x7f5fe80d1120 2023-12-20T05:44:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fe659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fe815cdb0 0x7f5fe80d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fe659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fe815cdb0 0x7f5fe80d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37764/0 (socket says 172.21.15.32:37764) 2023-12-20T05:44:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fe659c640 1 -- 172.21.15.32:0/2038086730 learned_addr learned my addr 172.21.15.32:0/2038086730 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:15.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fdffff640 1 --2- 172.21.15.32:0/2038086730 >> v2:172.21.15.39:3300/0 conn(0x7f5fe80d2740 0x7f5fe80cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.807+0000 7f5fe6d9d640 1 --2- 172.21.15.32:0/2038086730 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fe80d1120 0x7f5fe80cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.811+0000 7f5fe659c640 1 -- 172.21.15.32:0/2038086730 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fe80d1120 msgr2=0x7f5fe80cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:15.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.811+0000 7f5fe659c640 1 --2- 172.21.15.32:0/2038086730 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fe80d1120 0x7f5fe80cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.811+0000 7f5fe659c640 1 -- 172.21.15.32:0/2038086730 >> v2:172.21.15.39:3300/0 conn(0x7f5fe80d2740 msgr2=0x7f5fe80cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:15.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.811+0000 7f5fe659c640 1 --2- 172.21.15.32:0/2038086730 >> v2:172.21.15.39:3300/0 conn(0x7f5fe80d2740 0x7f5fe80cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.811+0000 7f5fe659c640 1 -- 172.21.15.32:0/2038086730 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fe80d1c60 con 0x7f5fe815cdb0 2023-12-20T05:44:15.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.811+0000 7f5fe659c640 1 --2- 172.21.15.32:0/2038086730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe815cdb0 0x7f5fe80d0be0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80cd570 tx=0x7f5fd801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6fac4d3b74df65 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:15.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.811+0000 7f5fe5d9b640 1 -- 172.21.15.32:0/2038086730 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80aa5f0 con 0x7f5fe815cdb0 2023-12-20T05:44:15.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.811+0000 7f5fe5d9b640 1 -- 172.21.15.32:0/2038086730 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd80aa750 con 0x7f5fe815cdb0 2023-12-20T05:44:15.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.811+0000 7f5fe5d9b640 1 -- 172.21.15.32:0/2038086730 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80aaae0 con 0x7f5fe815cdb0 2023-12-20T05:44:15.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 -- 172.21.15.32:0/2038086730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe815cdb0 msgr2=0x7f5fe80d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:15.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 --2- 172.21.15.32:0/2038086730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe815cdb0 0x7f5fe80d0be0 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f5fe80cd570 tx=0x7f5fd801c4d0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 -- 172.21.15.32:0/2038086730 shutdown_connections 2023-12-20T05:44:15.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 --2- 172.21.15.32:0/2038086730 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fe80d1120 0x7f5fe80cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 --2- 172.21.15.32:0/2038086730 >> v2:172.21.15.39:3300/0 conn(0x7f5fe80d2740 0x7f5fe80cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 --2- 172.21.15.32:0/2038086730 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe815cdb0 0x7f5fe80d0be0 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 -- 172.21.15.32:0/2038086730 >> 172.21.15.32:0/2038086730 conn(0x7f5fe8065ee0 msgr2=0x7f5fe806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:15.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 -- 172.21.15.32:0/2038086730 shutdown_connections 2023-12-20T05:44:15.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 -- 172.21.15.32:0/2038086730 wait complete. 2023-12-20T05:44:15.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 Processor -- start 2023-12-20T05:44:15.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 -- start start 2023-12-20T05:44:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe80d1120 0x7f5fe80c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe80d2740 0x7f5fe80c9fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fe815cdb0 0x7f5fe80c6600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80c7740 con 0x7f5fe80d2740 2023-12-20T05:44:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80c7d00 con 0x7f5fe80d1120 2023-12-20T05:44:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fecd39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fe80c6b40 con 0x7f5fe815cdb0 2023-12-20T05:44:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe80d2740 0x7f5fe80c9fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fe659c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe80d1120 0x7f5fe80c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fdffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe80d2740 0x7f5fe80c9fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37768/0 (socket says 172.21.15.32:37768) 2023-12-20T05:44:15.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fdffff640 1 -- 172.21.15.32:0/3708743930 learned_addr learned my addr 172.21.15.32:0/3708743930 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:15.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.815+0000 7f5fe6d9d640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fe815cdb0 0x7f5fe80c6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fe659c640 1 -- 172.21.15.32:0/3708743930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fe815cdb0 msgr2=0x7f5fe80c6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:15.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fe659c640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fe815cdb0 0x7f5fe80c6600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fe659c640 1 -- 172.21.15.32:0/3708743930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe80d2740 msgr2=0x7f5fe80c9fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:15.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fe659c640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe80d2740 0x7f5fe80c9fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:15.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fe659c640 1 -- 172.21.15.32:0/3708743930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fd80094d0 con 0x7f5fe80d1120 2023-12-20T05:44:15.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fe659c640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe80d1120 0x7f5fe80c8190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80ab000 tx=0x7f5fd801ad10 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:15.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fdeffd640 1 -- 172.21.15.32:0/3708743930 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80bd5f0 con 0x7f5fe80d1120 2023-12-20T05:44:15.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fdeffd640 1 -- 172.21.15.32:0/3708743930 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fd80bd790 con 0x7f5fe80d1120 2023-12-20T05:44:15.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fecd39640 1 -- 172.21.15.32:0/3708743930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fe8152460 con 0x7f5fe80d1120 2023-12-20T05:44:15.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fdeffd640 1 -- 172.21.15.32:0/3708743930 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fd80bdb40 con 0x7f5fe80d1120 2023-12-20T05:44:15.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fecd39640 1 -- 172.21.15.32:0/3708743930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fe80c7990 con 0x7f5fe80d1120 2023-12-20T05:44:15.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fecd39640 1 -- 172.21.15.32:0/3708743930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fe8152960 con 0x7f5fe80d1120 2023-12-20T05:44:15.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fdeffd640 1 -- 172.21.15.32:0/3708743930 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5fd80b33f0 con 0x7f5fe80d1120 2023-12-20T05:44:15.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.819+0000 7f5fdeffd640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fc8065b80 0x7f5fc8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:15.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.823+0000 7f5fdeffd640 1 -- 172.21.15.32:0/3708743930 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(168..168 src has 1..168) v4 ==== 5573+0+0 (secure 0 0 0) 0x7f5fd80a7070 con 0x7f5fe80d1120 2023-12-20T05:44:15.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.823+0000 7f5fdffff640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fc8065b80 0x7f5fc8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:15.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.823+0000 7f5fdffff640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fc8065b80 0x7f5fc8068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5fe814cd60 tx=0x7f5fd0009190 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:15.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.827+0000 7f5fdeffd640 1 -- 172.21.15.32:0/3708743930 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5fd80eaf00 con 0x7f5fe80d1120 2023-12-20T05:44:15.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:15.999+0000 7f5fecd39640 1 -- 172.21.15.32:0/3708743930 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f5fe804e000 con 0x7f5fe80d1120 2023-12-20T05:44:16.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fdeffd640 1 -- 172.21.15.32:0/3708743930 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v169) v1 ==== 97+0+0 (secure 0 0 0) 0x7f5fd80ee520 con 0x7f5fe80d1120 2023-12-20T05:44:16.829 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T05:44:16.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 -- 172.21.15.32:0/3708743930 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fc8065b80 msgr2=0x7f5fc8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:16.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fc8065b80 0x7f5fc8068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f5fe814cd60 tx=0x7f5fd0009190 comp rx=0 tx=0).stop 2023-12-20T05:44:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 -- 172.21.15.32:0/3708743930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe80d1120 msgr2=0x7f5fe80c8190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe80d1120 0x7f5fe80c8190 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f5fd80ab000 tx=0x7f5fd801ad10 comp rx=0 tx=0).stop 2023-12-20T05:44:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 -- 172.21.15.32:0/3708743930 shutdown_connections 2023-12-20T05:44:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fc8065b80 0x7f5fc8068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fe815cdb0 0x7f5fe80c6600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fe80d2740 0x7f5fe80c9fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 --2- 172.21.15.32:0/3708743930 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fe80d1120 0x7f5fe80c8190 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 -- 172.21.15.32:0/3708743930 >> 172.21.15.32:0/3708743930 conn(0x7f5fe8065ee0 msgr2=0x7f5fe80cf780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 -- 172.21.15.32:0/3708743930 shutdown_connections 2023-12-20T05:44:16.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:16.831+0000 7f5fecd39640 1 -- 172.21.15.32:0/3708743930 wait complete. 2023-12-20T05:44:16.855 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T05:44:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f990da66640 1 Processor -- start 2023-12-20T05:44:17.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f990da66640 1 -- start start 2023-12-20T05:44:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f990da66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f99080af5e0 0x7f99080af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f990da66640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f990806ce60 0x7f990815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f990da66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f990815bbd0 con 0x7f990815f110 2023-12-20T05:44:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f990da66640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f990815beb0 con 0x7f990806ce60 2023-12-20T05:44:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f990da66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f990815e910 con 0x7f99080af5e0 2023-12-20T05:44:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f9907fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f990815f110 0x7f990815c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56172/0 (socket says 172.21.15.32:56172) 2023-12-20T05:44:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f9907fff640 1 -- 172.21.15.32:0/1102121547 learned_addr learned my addr 172.21.15.32:0/1102121547 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f99077fe640 1 --2- 172.21.15.32:0/1102121547 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f99080af5e0 0x7f99080af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f9906ffd640 1 --2- 172.21.15.32:0/1102121547 >> v2:172.21.15.39:3300/0 conn(0x7f990806ce60 0x7f990815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f99067fc640 1 -- 172.21.15.32:0/1102121547 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1292174265 0 0) 0x7f990815bbd0 con 0x7f990815f110 2023-12-20T05:44:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f99067fc640 1 -- 172.21.15.32:0/1102121547 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f98f0003530 con 0x7f990815f110 2023-12-20T05:44:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f99077fe640 1 -- 172.21.15.32:0/1102121547 >> v2:172.21.15.39:3300/0 conn(0x7f990806ce60 msgr2=0x7f990815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f99077fe640 1 --2- 172.21.15.32:0/1102121547 >> v2:172.21.15.39:3300/0 conn(0x7f990806ce60 0x7f990815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f99077fe640 1 -- 172.21.15.32:0/1102121547 >> v1:172.21.15.32:6789/0 conn(0x7f990815f110 legacy=0x7f990815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f99077fe640 1 -- 172.21.15.32:0/1102121547 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f990815ebf0 con 0x7f99080af5e0 2023-12-20T05:44:17.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.007+0000 7f9906ffd640 1 --2- 172.21.15.32:0/1102121547 >> v2:172.21.15.39:3300/0 conn(0x7f990806ce60 0x7f990815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f99077fe640 1 --2- 172.21.15.32:0/1102121547 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f99080af5e0 0x7f99080af9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f98f80097a0 tx=0x7f98f801c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=50d14c9a76b6594 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f99067fc640 1 -- 172.21.15.32:0/1102121547 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f80ac620 con 0x7f99080af5e0 2023-12-20T05:44:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f99067fc640 1 -- 172.21.15.32:0/1102121547 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f80ac780 con 0x7f99080af5e0 2023-12-20T05:44:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f99067fc640 1 -- 172.21.15.32:0/1102121547 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f80acb10 con 0x7f99080af5e0 2023-12-20T05:44:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f990da66640 1 -- 172.21.15.32:0/1102121547 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f99080af5e0 msgr2=0x7f99080af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f990da66640 1 --2- 172.21.15.32:0/1102121547 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f99080af5e0 0x7f99080af9c0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f98f80097a0 tx=0x7f98f801c460 comp rx=0 tx=0).stop 2023-12-20T05:44:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f990da66640 1 -- 172.21.15.32:0/1102121547 shutdown_connections 2023-12-20T05:44:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f990da66640 1 --2- 172.21.15.32:0/1102121547 >> v2:172.21.15.39:3300/0 conn(0x7f990806ce60 0x7f990815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f990da66640 1 --2- 172.21.15.32:0/1102121547 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f99080af5e0 0x7f99080af9c0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f990da66640 1 -- 172.21.15.32:0/1102121547 >> 172.21.15.32:0/1102121547 conn(0x7f990806a320 msgr2=0x7f99080ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f990da66640 1 -- 172.21.15.32:0/1102121547 shutdown_connections 2023-12-20T05:44:17.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.011+0000 7f990da66640 1 -- 172.21.15.32:0/1102121547 wait complete. 2023-12-20T05:44:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f990da66640 1 Processor -- start 2023-12-20T05:44:17.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f990da66640 1 -- start start 2023-12-20T05:44:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f990da66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990806ce60 0x7f99080c8250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f990da66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f99080af5e0 0x7f9908070ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f990da66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f990815f110 0x7f9908071010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f990da66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f990814fc20 con 0x7f990806ce60 2023-12-20T05:44:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f990da66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9908071640 con 0x7f990815f110 2023-12-20T05:44:17.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f990da66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9908071920 con 0x7f99080af5e0 2023-12-20T05:44:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f99077fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990806ce60 0x7f99080c8250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f99077fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990806ce60 0x7f99080c8250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37782/0 (socket says 172.21.15.32:37782) 2023-12-20T05:44:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f99077fe640 1 -- 172.21.15.32:0/1757877624 learned_addr learned my addr 172.21.15.32:0/1757877624 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f9907fff640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f990815f110 0x7f9908071010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f9906ffd640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f99080af5e0 0x7f9908070ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f99077fe640 1 -- 172.21.15.32:0/1757877624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f99080af5e0 msgr2=0x7f9908070ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f99077fe640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f99080af5e0 0x7f9908070ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f99077fe640 1 -- 172.21.15.32:0/1757877624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f990815f110 msgr2=0x7f9908071010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f99077fe640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f990815f110 0x7f9908071010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f99077fe640 1 -- 172.21.15.32:0/1757877624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc009420 con 0x7f990806ce60 2023-12-20T05:44:17.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f99077fe640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990806ce60 0x7f99080c8250 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f98f801ab10 tx=0x7f98f8019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f98effff640 1 -- 172.21.15.32:0/1757877624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f80ac620 con 0x7f990806ce60 2023-12-20T05:44:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f98effff640 1 -- 172.21.15.32:0/1757877624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f80b5410 con 0x7f990806ce60 2023-12-20T05:44:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f990da66640 1 -- 172.21.15.32:0/1757877624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98f80093b0 con 0x7f990806ce60 2023-12-20T05:44:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f98effff640 1 -- 172.21.15.32:0/1757877624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f80b57a0 con 0x7f990806ce60 2023-12-20T05:44:17.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.015+0000 7f990da66640 1 -- 172.21.15.32:0/1757877624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f990814f8b0 con 0x7f990806ce60 2023-12-20T05:44:17.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.019+0000 7f990da66640 1 -- 172.21.15.32:0/1757877624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f990804e000 con 0x7f990806ce60 2023-12-20T05:44:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.019+0000 7f98effff640 1 -- 172.21.15.32:0/1757877624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f98f80b3040 con 0x7f990806ce60 2023-12-20T05:44:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.019+0000 7f98effff640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98f4065b80 0x7f98f4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.019+0000 7f98effff640 1 -- 172.21.15.32:0/1757877624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f98f811cf20 con 0x7f990806ce60 2023-12-20T05:44:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.019+0000 7f9906ffd640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98f4065b80 0x7f98f4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.019+0000 7f9906ffd640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98f4065b80 0x7f98f4068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f990815e220 tx=0x7f98fc009190 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:17.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.023+0000 7f98effff640 1 -- 172.21.15.32:0/1757877624 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f98f80eaa00 con 0x7f990806ce60 2023-12-20T05:44:17.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.235+0000 7f990da66640 1 -- 172.21.15.32:0/1757877624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f99080cf450 con 0x7f990806ce60 2023-12-20T05:44:17.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.235+0000 7f98effff640 1 -- 172.21.15.32:0/1757877624 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v42) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f98f80b1020 con 0x7f990806ce60 2023-12-20T05:44:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:17.236 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T05:44:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T05:44:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T05:44:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T05:44:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:44:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T05:44:17.236 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T05:44:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T05:44:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T05:44:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:44:17.237 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T05:44:17.238 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T05:44:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T05:44:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:44:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T05:44:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T05:44:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.239 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T05:44:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T05:44:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:44:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:44:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.240 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T05:44:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T05:44:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:44:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:44:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.241 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T05:44:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T05:44:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:44:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:44:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.242 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T05:44:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T05:44:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:44:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:44:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T05:44:17.243 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:44:17.244 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:44:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T05:44:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T05:44:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:44:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:44:17.245 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.246 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T05:44:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T05:44:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:44:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T05:44:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T05:44:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.247 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T05:44:17.248 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:44:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T05:44:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T05:44:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T05:44:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T05:44:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T05:44:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.249 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T05:44:17.250 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T05:44:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T05:44:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T05:44:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T05:44:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T05:44:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.251 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T05:44:17.252 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T05:44:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 -- 172.21.15.32:0/1757877624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98f4065b80 msgr2=0x7f98f4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98f4065b80 0x7f98f4068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f990815e220 tx=0x7f98fc009190 comp rx=0 tx=0).stop 2023-12-20T05:44:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 -- 172.21.15.32:0/1757877624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990806ce60 msgr2=0x7f99080c8250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990806ce60 0x7f99080c8250 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7f98f801ab10 tx=0x7f98f8019200 comp rx=0 tx=0).stop 2023-12-20T05:44:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 -- 172.21.15.32:0/1757877624 shutdown_connections 2023-12-20T05:44:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f98f4065b80 0x7f98f4068040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f990815f110 0x7f9908071010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f99080af5e0 0x7f9908070ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 --2- 172.21.15.32:0/1757877624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f990806ce60 0x7f99080c8250 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 -- 172.21.15.32:0/1757877624 >> 172.21.15.32:0/1757877624 conn(0x7f990806a320 msgr2=0x7f9908159110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:17.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 -- 172.21.15.32:0/1757877624 shutdown_connections 2023-12-20T05:44:17.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.239+0000 7f990da66640 1 -- 172.21.15.32:0/1757877624 wait complete. 2023-12-20T05:44:17.257 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:44:17.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.403+0000 7f6002c1f640 1 Processor -- start 2023-12-20T05:44:17.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.403+0000 7f6002c1f640 1 -- start start 2023-12-20T05:44:17.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f6002c1f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 0x7f5ffc0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f6002c1f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ffc0c98c0 con 0x7f5ffc0c90e0 2023-12-20T05:44:17.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f6002c1f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5ffc0c9ba0 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f6002c1f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc0c9e80 con 0x7f5ffc1599e0 2023-12-20T05:44:17.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f5ffbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5ffc0c90e0 0x7f5ffc0b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56178/0 (socket says 172.21.15.32:56178) 2023-12-20T05:44:17.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f5ffbfff640 1 -- 172.21.15.32:0/1929110916 learned_addr learned my addr 172.21.15.32:0/1929110916 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:17.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f60009bb640 1 --2- 172.21.15.32:0/1929110916 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 0x7f5ffc0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f5ffb7fe640 1 -- 172.21.15.32:0/1929110916 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 921496523 0 0) 0x7f5ffc0c98c0 con 0x7f5ffc0c90e0 2023-12-20T05:44:17.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f5ffb7fe640 1 -- 172.21.15.32:0/1929110916 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fec003530 con 0x7f5ffc0c90e0 2023-12-20T05:44:17.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f5ffb7fe640 1 -- 172.21.15.32:0/1929110916 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2450714332 0 0) 0x7f5ffc0c9ba0 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f5ffb7fe640 1 -- 172.21.15.32:0/1929110916 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5ffc0c98c0 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f60009bb640 1 -- 172.21.15.32:0/1929110916 >> v1:172.21.15.39:6789/0 conn(0x7f5ffc0b0d00 legacy=0x7f5ffc0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f60009bb640 1 -- 172.21.15.32:0/1929110916 >> v1:172.21.15.32:6789/0 conn(0x7f5ffc0c90e0 legacy=0x7f5ffc0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.407+0000 7f60009bb640 1 -- 172.21.15.32:0/1929110916 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ffc0ca160 con 0x7f5ffc1599e0 2023-12-20T05:44:17.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f60009bb640 1 --2- 172.21.15.32:0/1929110916 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 0x7f5ffc0aece0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00097d0 tx=0x7f5ff001c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cee5091a81e095df server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:17.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f5ffb7fe640 1 -- 172.21.15.32:0/1929110916 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00ab600 con 0x7f5ffc1599e0 2023-12-20T05:44:17.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f5ffb7fe640 1 -- 172.21.15.32:0/1929110916 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff00ab760 con 0x7f5ffc1599e0 2023-12-20T05:44:17.409 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f5ffb7fe640 1 -- 172.21.15.32:0/1929110916 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00abaf0 con 0x7f5ffc1599e0 2023-12-20T05:44:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f6002c1f640 1 -- 172.21.15.32:0/1929110916 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 msgr2=0x7f5ffc0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f6002c1f640 1 --2- 172.21.15.32:0/1929110916 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 0x7f5ffc0aece0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00097d0 tx=0x7f5ff001c3e0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f6002c1f640 1 -- 172.21.15.32:0/1929110916 shutdown_connections 2023-12-20T05:44:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f6002c1f640 1 --2- 172.21.15.32:0/1929110916 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 0x7f5ffc0aece0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f6002c1f640 1 -- 172.21.15.32:0/1929110916 >> 172.21.15.32:0/1929110916 conn(0x7f5ffc06a320 msgr2=0x7f5ffc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f6002c1f640 1 -- 172.21.15.32:0/1929110916 shutdown_connections 2023-12-20T05:44:17.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f6002c1f640 1 -- 172.21.15.32:0/1929110916 wait complete. 2023-12-20T05:44:17.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f6002c1f640 1 Processor -- start 2023-12-20T05:44:17.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.411+0000 7f6002c1f640 1 -- start start 2023-12-20T05:44:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f6002c1f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ffc0b0d00 0x7f5ffc1582f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f6002c1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc0c90e0 0x7f5ffc149f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f6002c1f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 0x7f5ffc154ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f6002c1f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc0c5830 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f6002c1f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc14a490 con 0x7f5ffc0c90e0 2023-12-20T05:44:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f6002c1f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5ffc1588f0 con 0x7f5ffc1599e0 2023-12-20T05:44:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f60009bb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ffc0b0d00 0x7f5ffc1582f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f60009bb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ffc0b0d00 0x7f5ffc1582f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37796/0 (socket says 172.21.15.32:37796) 2023-12-20T05:44:17.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f60009bb640 1 -- 172.21.15.32:0/1552469702 learned_addr learned my addr 172.21.15.32:0/1552469702 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:17.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f60011bc640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 0x7f5ffc154ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f5ffbfff640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc0c90e0 0x7f5ffc149f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f60009bb640 1 -- 172.21.15.32:0/1552469702 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 msgr2=0x7f5ffc154ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f60009bb640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 0x7f5ffc154ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f60009bb640 1 -- 172.21.15.32:0/1552469702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc0c90e0 msgr2=0x7f5ffc149f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f60009bb640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc0c90e0 0x7f5ffc149f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f60009bb640 1 -- 172.21.15.32:0/1552469702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5ff00093b0 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f60011bc640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 0x7f5ffc154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:44:17.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f5ffbfff640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc0c90e0 0x7f5ffc149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:44:17.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f60009bb640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ffc0b0d00 0x7f5ffc1582f0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00ad040 tx=0x7f5ff0019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:17.415 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f5ff97fa640 1 -- 172.21.15.32:0/1552469702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00abda0 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f5ff97fa640 1 -- 172.21.15.32:0/1552469702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5ff00bf450 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f6002c1f640 1 -- 172.21.15.32:0/1552469702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5ffc158bd0 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f5ff97fa640 1 -- 172.21.15.32:0/1552469702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5ff00bf800 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f6002c1f640 1 -- 172.21.15.32:0/1552469702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5ffc0c5a80 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f6002c1f640 1 -- 172.21.15.32:0/1552469702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5ffc04e000 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.415+0000 7f5ff97fa640 1 -- 172.21.15.32:0/1552469702 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f5ff00bf960 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.419+0000 7f5ff97fa640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fe0065b80 0x7f5fe0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.419+0000 7f5ff97fa640 1 -- 172.21.15.32:0/1552469702 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f5ff011d9b0 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.419+0000 7f5ffbfff640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fe0065b80 0x7f5fe0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.419+0000 7f5ffbfff640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fe0065b80 0x7f5fe0068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc06cd90 tx=0x7f5fe8009040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:17.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.423+0000 7f5ff97fa640 1 -- 172.21.15.32:0/1552469702 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5ff00eb470 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.579+0000 7f6002c1f640 1 -- 172.21.15.32:0/1552469702 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5ffc0cbcf0 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.579+0000 7f5ff97fa640 1 -- 172.21.15.32:0/1552469702 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v169) v1 ==== 74+0+12395 (secure 0 0 0) 0x7f5ff00c4090 con 0x7f5ffc0b0d00 2023-12-20T05:44:17.579 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:17.579 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":169,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:44:16.799702+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":19,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":157,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:44:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 -- 172.21.15.32:0/1552469702 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fe0065b80 msgr2=0x7f5fe0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fe0065b80 0x7f5fe0068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f5ffc06cd90 tx=0x7f5fe8009040 comp rx=0 tx=0).stop 2023-12-20T05:44:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 -- 172.21.15.32:0/1552469702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ffc0b0d00 msgr2=0x7f5ffc1582f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ffc0b0d00 0x7f5ffc1582f0 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f5ff00ad040 tx=0x7f5ff0019180 comp rx=0 tx=0).stop 2023-12-20T05:44:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 -- 172.21.15.32:0/1552469702 shutdown_connections 2023-12-20T05:44:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f5fe0065b80 0x7f5fe0068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5ffc1599e0 0x7f5ffc154ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5ffc0c90e0 0x7f5ffc149f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 --2- 172.21.15.32:0/1552469702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5ffc0b0d00 0x7f5ffc1582f0 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 -- 172.21.15.32:0/1552469702 >> 172.21.15.32:0/1552469702 conn(0x7f5ffc06a320 msgr2=0x7f5ffc0ce940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 -- 172.21.15.32:0/1552469702 shutdown_connections 2023-12-20T05:44:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.583+0000 7f6002c1f640 1 -- 172.21.15.32:0/1552469702 wait complete. 2023-12-20T05:44:17.601 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T05:44:17.601 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T05:44:17.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe03cba2640 1 Processor -- start 2023-12-20T05:44:17.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe03cba2640 1 -- start start 2023-12-20T05:44:17.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe03cba2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe0381583e0 0x7fe03815ed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe03cba2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380ca560 0x7fe03815aee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe03cba2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe0380caf70 0x7fe0380cb400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe03cba2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0380cb940 con 0x7fe0381583e0 2023-12-20T05:44:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe03cba2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe03815e8e0 con 0x7fe0380caf70 2023-12-20T05:44:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe03cba2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe03815b7a0 con 0x7fe0380ca560 2023-12-20T05:44:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe0377fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe0381583e0 0x7fe03815ed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe0377fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe0381583e0 0x7fe03815ed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37798/0 (socket says 172.21.15.32:37798) 2023-12-20T05:44:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe0377fe640 1 -- 172.21.15.32:0/299243429 learned_addr learned my addr 172.21.15.32:0/299243429 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe037fff640 1 --2- 172.21.15.32:0/299243429 >> v2:172.21.15.39:3300/0 conn(0x7fe0380caf70 0x7fe0380cb400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe036ffd640 1 --2- 172.21.15.32:0/299243429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380ca560 0x7fe03815aee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe0377fe640 1 -- 172.21.15.32:0/299243429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380ca560 msgr2=0x7fe03815aee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:17.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe0377fe640 1 --2- 172.21.15.32:0/299243429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380ca560 0x7fe03815aee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe0377fe640 1 -- 172.21.15.32:0/299243429 >> v2:172.21.15.39:3300/0 conn(0x7fe0380caf70 msgr2=0x7fe0380cb400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:17.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe0377fe640 1 --2- 172.21.15.32:0/299243429 >> v2:172.21.15.39:3300/0 conn(0x7fe0380caf70 0x7fe0380cb400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe0377fe640 1 -- 172.21.15.32:0/299243429 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe03815ba80 con 0x7fe0381583e0 2023-12-20T05:44:17.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe036ffd640 1 --2- 172.21.15.32:0/299243429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380ca560 0x7fe03815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:17.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.751+0000 7fe037fff640 1 --2- 172.21.15.32:0/299243429 >> v2:172.21.15.39:3300/0 conn(0x7fe0380caf70 0x7fe0380cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:17.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe0377fe640 1 --2- 172.21.15.32:0/299243429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0381583e0 0x7fe03815ed70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe0380c2060 tx=0x7fe02801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8a638b2ef1ed4d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:17.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe0367fc640 1 -- 172.21.15.32:0/299243429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0280aa5f0 con 0x7fe0381583e0 2023-12-20T05:44:17.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe0367fc640 1 -- 172.21.15.32:0/299243429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0280aa750 con 0x7fe0381583e0 2023-12-20T05:44:17.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe0367fc640 1 -- 172.21.15.32:0/299243429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0280aaae0 con 0x7fe0381583e0 2023-12-20T05:44:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe03cba2640 1 -- 172.21.15.32:0/299243429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0381583e0 msgr2=0x7fe03815ed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe03cba2640 1 --2- 172.21.15.32:0/299243429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0381583e0 0x7fe03815ed70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7fe0380c2060 tx=0x7fe02801c4d0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe03cba2640 1 -- 172.21.15.32:0/299243429 shutdown_connections 2023-12-20T05:44:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe03cba2640 1 --2- 172.21.15.32:0/299243429 >> v2:172.21.15.39:3300/0 conn(0x7fe0380caf70 0x7fe0380cb400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe03cba2640 1 --2- 172.21.15.32:0/299243429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380ca560 0x7fe03815aee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe03cba2640 1 --2- 172.21.15.32:0/299243429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0381583e0 0x7fe03815ed70 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe03cba2640 1 -- 172.21.15.32:0/299243429 >> 172.21.15.32:0/299243429 conn(0x7fe03806a4e0 msgr2=0x7fe03806bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe03cba2640 1 -- 172.21.15.32:0/299243429 shutdown_connections 2023-12-20T05:44:17.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.755+0000 7fe03cba2640 1 -- 172.21.15.32:0/299243429 wait complete. 2023-12-20T05:44:17.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe03cba2640 1 Processor -- start 2023-12-20T05:44:17.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe03cba2640 1 -- start start 2023-12-20T05:44:17.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe03cba2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0380ca560 0x7fe038150a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe03cba2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380caf70 0x7fe0380c97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe03cba2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0381583e0 0x7fe0380c9ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe03cba2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0380b1180 con 0x7fe0380ca560 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe03cba2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0380b13d0 con 0x7fe0381583e0 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe03cba2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0380b16b0 con 0x7fe0380caf70 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe0377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0380ca560 0x7fe038150a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe0377fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0380ca560 0x7fe038150a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37806/0 (socket says 172.21.15.32:37806) 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe0377fe640 1 -- 172.21.15.32:0/2159975645 learned_addr learned my addr 172.21.15.32:0/2159975645 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe036ffd640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380caf70 0x7fe0380c97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe037fff640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0381583e0 0x7fe0380c9ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe0377fe640 1 -- 172.21.15.32:0/2159975645 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380caf70 msgr2=0x7fe0380c97a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe0377fe640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380caf70 0x7fe0380c97a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe0377fe640 1 -- 172.21.15.32:0/2159975645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0381583e0 msgr2=0x7fe0380c9ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe0377fe640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0381583e0 0x7fe0380c9ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe0377fe640 1 -- 172.21.15.32:0/2159975645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0280094d0 con 0x7fe0380ca560 2023-12-20T05:44:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe036ffd640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380caf70 0x7fe0380c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe0377fe640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0380ca560 0x7fe038150a10 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe0280ac040 tx=0x7fe02801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe01ffff640 1 -- 172.21.15.32:0/2159975645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0280b3800 con 0x7fe0380ca560 2023-12-20T05:44:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe01ffff640 1 -- 172.21.15.32:0/2159975645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0280b3980 con 0x7fe0380ca560 2023-12-20T05:44:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe03cba2640 1 -- 172.21.15.32:0/2159975645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0380b1990 con 0x7fe0380ca560 2023-12-20T05:44:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe01ffff640 1 -- 172.21.15.32:0/2159975645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0280b3d30 con 0x7fe0380ca560 2023-12-20T05:44:17.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe03cba2640 1 -- 172.21.15.32:0/2159975645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0380b1f50 con 0x7fe0380ca560 2023-12-20T05:44:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.759+0000 7fe03cba2640 1 -- 172.21.15.32:0/2159975645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0380cadc0 con 0x7fe0380ca560 2023-12-20T05:44:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.763+0000 7fe01ffff640 1 -- 172.21.15.32:0/2159975645 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe0280b2380 con 0x7fe0380ca560 2023-12-20T05:44:17.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.763+0000 7fe01ffff640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe024065b80 0x7fe024068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:17.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.763+0000 7fe01ffff640 1 -- 172.21.15.32:0/2159975645 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fe0280a7070 con 0x7fe0380ca560 2023-12-20T05:44:17.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.763+0000 7fe036ffd640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe024065b80 0x7fe024068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:17.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.763+0000 7fe036ffd640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe024065b80 0x7fe024068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0380b4280 tx=0x7fe02c008040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:17.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.767+0000 7fe01ffff640 1 -- 172.21.15.32:0/2159975645 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe0280eace0 con 0x7fe0380ca560 2023-12-20T05:44:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.923+0000 7fe03cba2640 1 -- 172.21.15.32:0/2159975645 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe0380b2140 con 0x7fe0380ca560 2023-12-20T05:44:17.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.923+0000 7fe01ffff640 1 -- 172.21.15.32:0/2159975645 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v169) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe0280ee300 con 0x7fe0380ca560 2023-12-20T05:44:17.922 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T05:44:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 -- 172.21.15.32:0/2159975645 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe024065b80 msgr2=0x7fe024068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe024065b80 0x7fe024068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fe0380b4280 tx=0x7fe02c008040 comp rx=0 tx=0).stop 2023-12-20T05:44:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 -- 172.21.15.32:0/2159975645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0380ca560 msgr2=0x7fe038150a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0380ca560 0x7fe038150a10 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7fe0280ac040 tx=0x7fe02801ad10 comp rx=0 tx=0).stop 2023-12-20T05:44:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 -- 172.21.15.32:0/2159975645 shutdown_connections 2023-12-20T05:44:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe024065b80 0x7fe024068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0381583e0 0x7fe0380c9ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0380caf70 0x7fe0380c97a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 --2- 172.21.15.32:0/2159975645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0380ca560 0x7fe038150a10 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 -- 172.21.15.32:0/2159975645 >> 172.21.15.32:0/2159975645 conn(0x7fe03806a4e0 msgr2=0x7fe03806fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 -- 172.21.15.32:0/2159975645 shutdown_connections 2023-12-20T05:44:17.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:17.927+0000 7fe03cba2640 1 -- 172.21.15.32:0/2159975645 wait complete. 2023-12-20T05:44:17.943 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T05:44:18.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07d14d1640 1 Processor -- start 2023-12-20T05:44:18.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07d14d1640 1 -- start start 2023-12-20T05:44:18.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07d14d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f07cc0aece0 0x7f07cc0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07d14d1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07cc0c98c0 con 0x7f07cc1599e0 2023-12-20T05:44:18.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07d14d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07cc0c9ba0 con 0x7f07cc0b0b30 2023-12-20T05:44:18.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07d14d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07cc0c9e80 con 0x7f07cc0aece0 2023-12-20T05:44:18.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07caffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f07cc1599e0 0x7f07cc0c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56186/0 (socket says 172.21.15.32:56186) 2023-12-20T05:44:18.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07caffd640 1 -- 172.21.15.32:0/931530057 learned_addr learned my addr 172.21.15.32:0/931530057 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07c3fff640 1 --2- 172.21.15.32:0/931530057 >> v2:172.21.15.39:3300/0 conn(0x7f07cc0aece0 0x7f07cc0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4083446675 0 0) 0x7f07cc0c98c0 con 0x7f07cc1599e0 2023-12-20T05:44:18.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07b8003530 con 0x7f07cc1599e0 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 932577883 0 0) 0x7f07b8003530 con 0x7f07cc1599e0 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 >> v2:172.21.15.39:3300/0 conn(0x7f07cc0aece0 msgr2=0x7f07cc0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 --2- 172.21.15.32:0/931530057 >> v2:172.21.15.39:3300/0 conn(0x7f07cc0aece0 0x7f07cc0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 >> v1:172.21.15.39:6789/0 conn(0x7f07cc0b0b30 legacy=0x7f07cc0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc0ca160 con 0x7f07cc1599e0 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f07bc001800 con 0x7f07cc1599e0 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 >> v1:172.21.15.32:6789/0 conn(0x7f07cc1599e0 legacy=0x7f07cc0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 --2- 172.21.15.32:0/931530057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07b8003980 0x7f07b8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 --2- 172.21.15.32:0/931530057 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07b80047d0 0x7f07b8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 --2- 172.21.15.32:0/931530057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b8005190 0x7f07b800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0c9e80 con 0x7f07b8005190 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07b800c030 con 0x7f07b8003980 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07b800c1f0 con 0x7f07b80047d0 2023-12-20T05:44:18.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07cb7fe640 1 --2- 172.21.15.32:0/931530057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b8005190 0x7f07b800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07caffd640 1 --2- 172.21.15.32:0/931530057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07b8003980 0x7f07b8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07c3fff640 1 --2- 172.21.15.32:0/931530057 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07b80047d0 0x7f07b8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07cb7fe640 1 -- 172.21.15.32:0/931530057 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07b80047d0 msgr2=0x7f07b8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07cb7fe640 1 --2- 172.21.15.32:0/931530057 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07b80047d0 0x7f07b8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07cb7fe640 1 -- 172.21.15.32:0/931530057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07b8003980 msgr2=0x7f07b8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07cb7fe640 1 --2- 172.21.15.32:0/931530057 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07b8003980 0x7f07b8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.091+0000 7f07cb7fe640 1 -- 172.21.15.32:0/931530057 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07b8003530 con 0x7f07b8005190 2023-12-20T05:44:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07cb7fe640 1 -- 172.21.15.32:0/931530057 reap_dead start 2023-12-20T05:44:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07cb7fe640 1 --2- 172.21.15.32:0/931530057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b8005190 0x7f07b800b990 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f07c4001e70 tx=0x7f07c401c760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=172bf7963267ee6d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4003b80 con 0x7f07b8005190 2023-12-20T05:44:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07c4003d20 con 0x7f07b8005190 2023-12-20T05:44:18.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07ca7fc640 1 -- 172.21.15.32:0/931530057 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07c4005260 con 0x7f07b8005190 2023-12-20T05:44:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07d14d1640 1 -- 172.21.15.32:0/931530057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b8005190 msgr2=0x7f07b800b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07d14d1640 1 --2- 172.21.15.32:0/931530057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b8005190 0x7f07b800b990 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f07c4001e70 tx=0x7f07c401c760 comp rx=0 tx=0).stop 2023-12-20T05:44:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07d14d1640 1 -- 172.21.15.32:0/931530057 shutdown_connections 2023-12-20T05:44:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07d14d1640 1 --2- 172.21.15.32:0/931530057 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07b8005190 0x7f07b800b990 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07d14d1640 1 -- 172.21.15.32:0/931530057 >> 172.21.15.32:0/931530057 conn(0x7f07cc06a320 msgr2=0x7f07cc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07d14d1640 1 -- 172.21.15.32:0/931530057 shutdown_connections 2023-12-20T05:44:18.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.095+0000 7f07d14d1640 1 -- 172.21.15.32:0/931530057 wait complete. 2023-12-20T05:44:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07d14d1640 1 Processor -- start 2023-12-20T05:44:18.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07d14d1640 1 -- start start 2023-12-20T05:44:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07d14d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07b8005190 0x7f07cc14dc50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07d14d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07cc14ea20 0x7f07cc14eea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07d14d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0b1120 0x7f07cc0b15a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07d14d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc14f3e0 con 0x7f07cc14ea20 2023-12-20T05:44:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07d14d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0b1ae0 con 0x7f07cc0b1120 2023-12-20T05:44:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07d14d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07cc0703e0 con 0x7f07b8005190 2023-12-20T05:44:18.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07cc14ea20 0x7f07cc14eea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07cc14ea20 0x7f07cc14eea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37826/0 (socket says 172.21.15.32:37826) 2023-12-20T05:44:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c3fff640 1 -- 172.21.15.32:0/2883280315 learned_addr learned my addr 172.21.15.32:0/2883280315 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07cb7fe640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0b1120 0x7f07cc0b15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07caffd640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07b8005190 0x7f07cc14dc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c3fff640 1 -- 172.21.15.32:0/2883280315 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07b8005190 msgr2=0x7f07cc14dc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c3fff640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07b8005190 0x7f07cc14dc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c3fff640 1 -- 172.21.15.32:0/2883280315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0b1120 msgr2=0x7f07cc0b15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c3fff640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0b1120 0x7f07cc0b15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c3fff640 1 -- 172.21.15.32:0/2883280315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07cc14e190 con 0x7f07cc14ea20 2023-12-20T05:44:18.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c3fff640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07cc14ea20 0x7f07cc14eea0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f07b4011990 tx=0x7f07b4011d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c37fe640 1 -- 172.21.15.32:0/2883280315 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b400fa30 con 0x7f07cc14ea20 2023-12-20T05:44:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c37fe640 1 -- 172.21.15.32:0/2883280315 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07b400fb90 con 0x7f07cc14ea20 2023-12-20T05:44:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07c37fe640 1 -- 172.21.15.32:0/2883280315 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07b4014440 con 0x7f07cc14ea20 2023-12-20T05:44:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07d14d1640 1 -- 172.21.15.32:0/2883280315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07cc0b1d00 con 0x7f07cc14ea20 2023-12-20T05:44:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.099+0000 7f07d14d1640 1 -- 172.21.15.32:0/2883280315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07cc0b22c0 con 0x7f07cc14ea20 2023-12-20T05:44:18.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.103+0000 7f07d14d1640 1 -- 172.21.15.32:0/2883280315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07cc04e000 con 0x7f07cc14ea20 2023-12-20T05:44:18.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.103+0000 7f07c37fe640 1 -- 172.21.15.32:0/2883280315 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f07b40145a0 con 0x7f07cc14ea20 2023-12-20T05:44:18.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.103+0000 7f07c37fe640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f07ac065bb0 0x7f07ac068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.103+0000 7f07c37fe640 1 -- 172.21.15.32:0/2883280315 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4885+0+0 (secure 0 0 0) 0x7f07b40836a0 con 0x7f07cc14ea20 2023-12-20T05:44:18.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.103+0000 7f07caffd640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f07ac065bb0 0x7f07ac068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.103+0000 7f07caffd640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f07ac065bb0 0x7f07ac068070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f07cc157c10 tx=0x7f07bc009040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:18.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.107+0000 7f07c37fe640 1 -- 172.21.15.32:0/2883280315 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f07b40511e0 con 0x7f07cc14ea20 2023-12-20T05:44:18.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.267+0000 7f07d14d1640 1 -- 172.21.15.32:0/2883280315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f07cc0cf550 con 0x7f07cc14ea20 2023-12-20T05:44:18.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07c37fe640 1 -- 172.21.15.32:0/2883280315 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v169) v1 ==== 84+0+65 (secure 0 0 0) 0x7f07b4022090 con 0x7f07cc14ea20 2023-12-20T05:44:18.268 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:18.268 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T05:44:18.268 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:18.268 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T05:44:18.268 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T05:44:18.268 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:18.268 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 -- 172.21.15.32:0/2883280315 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f07ac065bb0 msgr2=0x7f07ac068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f07ac065bb0 0x7f07ac068070 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f07cc157c10 tx=0x7f07bc009040 comp rx=0 tx=0).stop 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 -- 172.21.15.32:0/2883280315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07cc14ea20 msgr2=0x7f07cc14eea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07cc14ea20 0x7f07cc14eea0 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f07b4011990 tx=0x7f07b4011d90 comp rx=0 tx=0).stop 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 -- 172.21.15.32:0/2883280315 shutdown_connections 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f07ac065bb0 0x7f07ac068070 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07cc0b1120 0x7f07cc0b15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07cc14ea20 0x7f07cc14eea0 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 --2- 172.21.15.32:0/2883280315 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07b8005190 0x7f07cc14dc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 -- 172.21.15.32:0/2883280315 >> 172.21.15.32:0/2883280315 conn(0x7f07cc06a320 msgr2=0x7f07cc14fb00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 -- 172.21.15.32:0/2883280315 shutdown_connections 2023-12-20T05:44:18.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.271+0000 7f07d14d1640 1 -- 172.21.15.32:0/2883280315 wait complete. 2023-12-20T05:44:18.289 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T05:44:18.289 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T05:44:18.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5d982d640 1 Processor -- start 2023-12-20T05:44:18.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5d982d640 1 -- start start 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5d982d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe5d4158520 0x7fe5d406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5d982d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5d40ca050 0x7fe5d415ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5d982d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5d40ca7f0 con 0x7fe5d4070230 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5d982d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40caad0 con 0x7fe5d40ca050 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5d982d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5d40cadb0 con 0x7fe5d4158520 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5cbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5d40ca050 0x7fe5d415ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5cbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5d40ca050 0x7fe5d415ef80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37830/0 (socket says 172.21.15.32:37830) 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5cbfff640 1 -- 172.21.15.32:0/2076322977 learned_addr learned my addr 172.21.15.32:0/2076322977 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5d2ffd640 1 --2- 172.21.15.32:0/2076322977 >> v2:172.21.15.39:3300/0 conn(0x7fe5d4158520 0x7fe5d406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5cbfff640 1 -- 172.21.15.32:0/2076322977 >> v2:172.21.15.39:3300/0 conn(0x7fe5d4158520 msgr2=0x7fe5d406dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5cbfff640 1 --2- 172.21.15.32:0/2076322977 >> v2:172.21.15.39:3300/0 conn(0x7fe5d4158520 0x7fe5d406dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5cbfff640 1 -- 172.21.15.32:0/2076322977 >> v1:172.21.15.39:6789/0 conn(0x7fe5d4070230 legacy=0x7fe5d415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:18.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.439+0000 7fe5cbfff640 1 -- 172.21.15.32:0/2076322977 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5d415e780 con 0x7fe5d40ca050 2023-12-20T05:44:18.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5cbfff640 1 --2- 172.21.15.32:0/2076322977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40ca050 0x7fe5d415ef80 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40cf010 tx=0x7fe5bc01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b041ae7e8f5e305 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5d27fc640 1 -- 172.21.15.32:0/2076322977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0aa650 con 0x7fe5d40ca050 2023-12-20T05:44:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5d27fc640 1 -- 172.21.15.32:0/2076322977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0aa7b0 con 0x7fe5d40ca050 2023-12-20T05:44:18.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5d27fc640 1 -- 172.21.15.32:0/2076322977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0aab40 con 0x7fe5d40ca050 2023-12-20T05:44:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5d982d640 1 -- 172.21.15.32:0/2076322977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40ca050 msgr2=0x7fe5d415ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5d982d640 1 --2- 172.21.15.32:0/2076322977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40ca050 0x7fe5d415ef80 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7fe5d40cf010 tx=0x7fe5bc01c4d0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5d982d640 1 -- 172.21.15.32:0/2076322977 shutdown_connections 2023-12-20T05:44:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5d982d640 1 --2- 172.21.15.32:0/2076322977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40ca050 0x7fe5d415ef80 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5d982d640 1 --2- 172.21.15.32:0/2076322977 >> v2:172.21.15.39:3300/0 conn(0x7fe5d4158520 0x7fe5d406dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5d982d640 1 -- 172.21.15.32:0/2076322977 >> 172.21.15.32:0/2076322977 conn(0x7fe5d406a510 msgr2=0x7fe5d40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5d982d640 1 -- 172.21.15.32:0/2076322977 shutdown_connections 2023-12-20T05:44:18.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.443+0000 7fe5d982d640 1 -- 172.21.15.32:0/2076322977 wait complete. 2023-12-20T05:44:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d982d640 1 Processor -- start 2023-12-20T05:44:18.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d982d640 1 -- start start 2023-12-20T05:44:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d982d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d4070230 0x7fe5d414f370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d982d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40ca050 0x7fe5d415d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d982d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d4158520 0x7fe5d4163c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d982d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4150ae0 con 0x7fe5d40ca050 2023-12-20T05:44:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d982d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d4151bc0 con 0x7fe5d4158520 2023-12-20T05:44:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d982d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5d414ff60 con 0x7fe5d4070230 2023-12-20T05:44:18.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40ca050 0x7fe5d415d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40ca050 0x7fe5d415d5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37832/0 (socket says 172.21.15.32:37832) 2023-12-20T05:44:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5cbfff640 1 -- 172.21.15.32:0/1466636272 learned_addr learned my addr 172.21.15.32:0/1466636272 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d2ffd640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d4070230 0x7fe5d414f370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d37fe640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d4158520 0x7fe5d4163c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5cbfff640 1 -- 172.21.15.32:0/1466636272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d4070230 msgr2=0x7fe5d414f370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5cbfff640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d4070230 0x7fe5d414f370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5cbfff640 1 -- 172.21.15.32:0/1466636272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d4158520 msgr2=0x7fe5d4163c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5cbfff640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d4158520 0x7fe5d4163c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5cbfff640 1 -- 172.21.15.32:0/1466636272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5bc0094d0 con 0x7fe5d40ca050 2023-12-20T05:44:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d37fe640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d4158520 0x7fe5d4163c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:18.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5cbfff640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40ca050 0x7fe5d415d5e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0ac040 tx=0x7fe5bc01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:18.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/1466636272 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0b3800 con 0x7fe5d40ca050 2023-12-20T05:44:18.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/1466636272 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5bc0b3980 con 0x7fe5d40ca050 2023-12-20T05:44:18.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d982d640 1 -- 172.21.15.32:0/1466636272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5d4150240 con 0x7fe5d40ca050 2023-12-20T05:44:18.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/1466636272 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5bc0b3d30 con 0x7fe5d40ca050 2023-12-20T05:44:18.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.447+0000 7fe5d982d640 1 -- 172.21.15.32:0/1466636272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5d4150800 con 0x7fe5d40ca050 2023-12-20T05:44:18.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.451+0000 7fe5d982d640 1 -- 172.21.15.32:0/1466636272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5d4151d80 con 0x7fe5d40ca050 2023-12-20T05:44:18.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.451+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/1466636272 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe5bc0bc3e0 con 0x7fe5d40ca050 2023-12-20T05:44:18.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.451+0000 7fe5d0ff9640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5b4065bb0 0x7fe5b4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:18.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.451+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/1466636272 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(169..169 src has 1..169) v4 ==== 4885+0+0 (secure 0 0 0) 0x7fe5bc11d5d0 con 0x7fe5d40ca050 2023-12-20T05:44:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.451+0000 7fe5d2ffd640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5b4065bb0 0x7fe5b4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:18.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.451+0000 7fe5d2ffd640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5b4065bb0 0x7fe5b4068070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40079c0 tx=0x7fe5c4008040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:18.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.455+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/1466636272 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe5bc0eb0b0 con 0x7fe5d40ca050 2023-12-20T05:44:18.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:18.619+0000 7fe5d982d640 1 -- 172.21.15.32:0/1466636272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fe5d404e000 con 0x7fe5d40ca050 2023-12-20T05:44:19.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.595+0000 7fe5d0ff9640 1 -- 172.21.15.32:0/1466636272 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v170) v1 ==== 138+0+0 (secure 0 0 0) 0x7fe5bc0c2020 con 0x7fe5d40ca050 2023-12-20T05:44:19.594 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T05:44:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 -- 172.21.15.32:0/1466636272 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5b4065bb0 msgr2=0x7fe5b4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5b4065bb0 0x7fe5b4068070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe5c40079c0 tx=0x7fe5c4008040 comp rx=0 tx=0).stop 2023-12-20T05:44:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 -- 172.21.15.32:0/1466636272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40ca050 msgr2=0x7fe5d415d5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40ca050 0x7fe5d415d5e0 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0ac040 tx=0x7fe5bc01ad10 comp rx=0 tx=0).stop 2023-12-20T05:44:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 -- 172.21.15.32:0/1466636272 shutdown_connections 2023-12-20T05:44:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fe5b4065bb0 0x7fe5b4068070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5d4158520 0x7fe5d4163c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5d40ca050 0x7fe5d415d5e0 secure :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7fe5bc0ac040 tx=0x7fe5bc01ad10 comp rx=0 tx=0).stop 2023-12-20T05:44:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 --2- 172.21.15.32:0/1466636272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5d4070230 0x7fe5d414f370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:19.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 -- 172.21.15.32:0/1466636272 >> 172.21.15.32:0/1466636272 conn(0x7fe5d406a510 msgr2=0x7fe5d40c4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 -- 172.21.15.32:0/1466636272 shutdown_connections 2023-12-20T05:44:19.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.599+0000 7fe5d982d640 1 -- 172.21.15.32:0/1466636272 wait complete. 2023-12-20T05:44:19.625 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T05:44:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d069ce640 1 Processor -- start 2023-12-20T05:44:19.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d069ce640 1 -- start start 2023-12-20T05:44:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d069ce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0d0015ad20 0x7f0d000cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d069ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d000cb1c0 0x7f0d000cb670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d069ce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d000cc9d0 con 0x7f0d0015a940 2023-12-20T05:44:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d069ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d000cbbb0 con 0x7f0d000cb1c0 2023-12-20T05:44:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d069ce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d000c9b90 con 0x7f0d0015ad20 2023-12-20T05:44:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d04f6b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0d000cb1c0 0x7f0d000cb670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0cfffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0d0015a940 0x7f0d000ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45154/0 (socket says 172.21.15.32:45154) 2023-12-20T05:44:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0cfffff640 1 -- 172.21.15.32:0/653443068 learned_addr learned my addr 172.21.15.32:0/653443068 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0cff7fe640 1 --2- 172.21.15.32:0/653443068 >> v2:172.21.15.39:3300/0 conn(0x7f0d0015ad20 0x7f0d000cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:19.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d04f6b640 1 -- 172.21.15.32:0/653443068 >> v2:172.21.15.39:3300/0 conn(0x7f0d0015ad20 msgr2=0x7f0d000cc380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d04f6b640 1 --2- 172.21.15.32:0/653443068 >> v2:172.21.15.39:3300/0 conn(0x7f0d0015ad20 0x7f0d000cc380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d04f6b640 1 -- 172.21.15.32:0/653443068 >> v1:172.21.15.32:6789/0 conn(0x7f0d0015a940 legacy=0x7f0d000ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0d04f6b640 1 -- 172.21.15.32:0/653443068 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d000c9e70 con 0x7f0d000cb1c0 2023-12-20T05:44:19.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.783+0000 7f0cfeffd640 1 -- 172.21.15.32:0/653443068 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2133803756 0 0) 0x7f0d000cc9d0 con 0x7f0d0015a940 2023-12-20T05:44:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0d04f6b640 1 --2- 172.21.15.32:0/653443068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d000cb1c0 0x7f0d000cb670 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f0d000ce460 tx=0x7f0cf801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97424600a326f2b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0cfeffd640 1 -- 172.21.15.32:0/653443068 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80a59a0 con 0x7f0d000cb1c0 2023-12-20T05:44:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0cfeffd640 1 -- 172.21.15.32:0/653443068 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf80a5b00 con 0x7f0d000cb1c0 2023-12-20T05:44:19.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0cfeffd640 1 -- 172.21.15.32:0/653443068 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf80a5e90 con 0x7f0d000cb1c0 2023-12-20T05:44:19.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0d069ce640 1 -- 172.21.15.32:0/653443068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d000cb1c0 msgr2=0x7f0d000cb670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:19.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0d069ce640 1 --2- 172.21.15.32:0/653443068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d000cb1c0 0x7f0d000cb670 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f0d000ce460 tx=0x7f0cf801c8e0 comp rx=0 tx=0).stop 2023-12-20T05:44:19.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0d069ce640 1 -- 172.21.15.32:0/653443068 shutdown_connections 2023-12-20T05:44:19.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0d069ce640 1 --2- 172.21.15.32:0/653443068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d000cb1c0 0x7f0d000cb670 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0d069ce640 1 --2- 172.21.15.32:0/653443068 >> v2:172.21.15.39:3300/0 conn(0x7f0d0015ad20 0x7f0d000cc380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0d069ce640 1 -- 172.21.15.32:0/653443068 >> 172.21.15.32:0/653443068 conn(0x7f0d0006a4b0 msgr2=0x7f0d000b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0d069ce640 1 -- 172.21.15.32:0/653443068 shutdown_connections 2023-12-20T05:44:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.787+0000 7f0d069ce640 1 -- 172.21.15.32:0/653443068 wait complete. 2023-12-20T05:44:19.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d069ce640 1 Processor -- start 2023-12-20T05:44:19.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d069ce640 1 -- start start 2023-12-20T05:44:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d069ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d000cb1c0 0x7f0d00148010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d069ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d0015a940 0x7f0d001570b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d069ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d0015ad20 0x7f0d001575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d069ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d001492f0 con 0x7f0d000cb1c0 2023-12-20T05:44:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d069ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d00150f90 con 0x7f0d0015ad20 2023-12-20T05:44:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d069ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d00151270 con 0x7f0d0015a940 2023-12-20T05:44:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d000cb1c0 0x7f0d00148010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:19.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d000cb1c0 0x7f0d00148010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58888/0 (socket says 172.21.15.32:58888) 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfffff640 1 -- 172.21.15.32:0/2382379319 learned_addr learned my addr 172.21.15.32:0/2382379319 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d04f6b640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d0015ad20 0x7f0d001575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cff7fe640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d0015a940 0x7f0d001570b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfffff640 1 -- 172.21.15.32:0/2382379319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d0015a940 msgr2=0x7f0d001570b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfffff640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d0015a940 0x7f0d001570b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfffff640 1 -- 172.21.15.32:0/2382379319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d0015ad20 msgr2=0x7f0d001575f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfffff640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d0015ad20 0x7f0d001575f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfffff640 1 -- 172.21.15.32:0/2382379319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0cf8009a70 con 0x7f0d000cb1c0 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfffff640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d000cb1c0 0x7f0d00148010 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00011f0 tx=0x7f0cf000c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfcff9640 1 -- 172.21.15.32:0/2382379319 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf00074e0 con 0x7f0d000cb1c0 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfcff9640 1 -- 172.21.15.32:0/2382379319 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0cf0007640 con 0x7f0d000cb1c0 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d069ce640 1 -- 172.21.15.32:0/2382379319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d00151550 con 0x7f0d000cb1c0 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0cfcff9640 1 -- 172.21.15.32:0/2382379319 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0cf0004a10 con 0x7f0d000cb1c0 2023-12-20T05:44:19.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d069ce640 1 -- 172.21.15.32:0/2382379319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d0014a420 con 0x7f0d000cb1c0 2023-12-20T05:44:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.791+0000 7f0d069ce640 1 -- 172.21.15.32:0/2382379319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d000cc390 con 0x7f0d000cb1c0 2023-12-20T05:44:19.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.795+0000 7f0cfcff9640 1 -- 172.21.15.32:0/2382379319 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0cf001c3b0 con 0x7f0d000cb1c0 2023-12-20T05:44:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.795+0000 7f0cfcff9640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cec065b80 0x7f0cec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.795+0000 7f0cfcff9640 1 -- 172.21.15.32:0/2382379319 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(170..170 src has 1..170) v4 ==== 5255+0+0 (secure 0 0 0) 0x7f0cf0082950 con 0x7f0d000cb1c0 2023-12-20T05:44:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.795+0000 7f0cff7fe640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cec065b80 0x7f0cec068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:19.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.795+0000 7f0cff7fe640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cec065b80 0x7f0cec068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d00157df0 tx=0x7f0cf4008040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:19.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.799+0000 7f0cfcff9640 1 -- 172.21.15.32:0/2382379319 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0cf0050450 con 0x7f0d000cb1c0 2023-12-20T05:44:19.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:19.959+0000 7f0d069ce640 1 -- 172.21.15.32:0/2382379319 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f0d0004e000 con 0x7f0d000cb1c0 2023-12-20T05:44:20.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.607+0000 7f0cfcff9640 1 -- 172.21.15.32:0/2382379319 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v171) v1 ==== 170+0+0 (secure 0 0 0) 0x7f0cf00180d0 con 0x7f0d000cb1c0 2023-12-20T05:44:20.605 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T05:44:20.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.615+0000 7f0d069ce640 1 -- 172.21.15.32:0/2382379319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cec065b80 msgr2=0x7f0cec068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:20.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.615+0000 7f0d069ce640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cec065b80 0x7f0cec068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f0d00157df0 tx=0x7f0cf4008040 comp rx=0 tx=0).stop 2023-12-20T05:44:20.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.615+0000 7f0d069ce640 1 -- 172.21.15.32:0/2382379319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d000cb1c0 msgr2=0x7f0d00148010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:20.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.615+0000 7f0d069ce640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d000cb1c0 0x7f0d00148010 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f0cf00011f0 tx=0x7f0cf000c9b0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.615+0000 7f0d069ce640 1 -- 172.21.15.32:0/2382379319 shutdown_connections 2023-12-20T05:44:20.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.615+0000 7f0d069ce640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f0cec065b80 0x7f0cec068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.615+0000 7f0d069ce640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d0015ad20 0x7f0d001575f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.615+0000 7f0d069ce640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d0015a940 0x7f0d001570b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.615+0000 7f0d069ce640 1 --2- 172.21.15.32:0/2382379319 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d000cb1c0 0x7f0d00148010 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.615+0000 7f0d069ce640 1 -- 172.21.15.32:0/2382379319 >> 172.21.15.32:0/2382379319 conn(0x7f0d0006a4b0 msgr2=0x7f0d0006db80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:20.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.619+0000 7f0d069ce640 1 -- 172.21.15.32:0/2382379319 shutdown_connections 2023-12-20T05:44:20.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.619+0000 7f0d069ce640 1 -- 172.21.15.32:0/2382379319 wait complete. 2023-12-20T05:44:20.645 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T05:44:20.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58fdd8f640 1 Processor -- start 2023-12-20T05:44:20.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58fdd8f640 1 -- start start 2023-12-20T05:44:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58fdd8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f815abb0 0x7f58f80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58fdd8f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58f80b0cb0 0x7f58f815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:20.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58fdd8f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f58f8070230 0x7f58f80706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58fdd8f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f806e160 con 0x7f58f80b0cb0 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58fdd8f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f8070be0 con 0x7f58f8070230 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58fdd8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f815b6a0 con 0x7f58f815abb0 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58f6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58f80b0cb0 0x7f58f815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58f6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58f80b0cb0 0x7f58f815bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58904/0 (socket says 172.21.15.32:58904) 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58f6ffd640 1 -- 172.21.15.32:0/3958877176 learned_addr learned my addr 172.21.15.32:0/3958877176 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58f77fe640 1 --2- 172.21.15.32:0/3958877176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f815abb0 0x7f58f80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58f7fff640 1 --2- 172.21.15.32:0/3958877176 >> v2:172.21.15.39:3300/0 conn(0x7f58f8070230 0x7f58f80706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58f6ffd640 1 -- 172.21.15.32:0/3958877176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f815abb0 msgr2=0x7f58f80caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58f6ffd640 1 --2- 172.21.15.32:0/3958877176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f815abb0 0x7f58f80caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58f6ffd640 1 -- 172.21.15.32:0/3958877176 >> v2:172.21.15.39:3300/0 conn(0x7f58f8070230 msgr2=0x7f58f80706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58f6ffd640 1 --2- 172.21.15.32:0/3958877176 >> v2:172.21.15.39:3300/0 conn(0x7f58f8070230 0x7f58f80706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.795+0000 7f58f6ffd640 1 -- 172.21.15.32:0/3958877176 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f815c070 con 0x7f58f80b0cb0 2023-12-20T05:44:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58f6ffd640 1 --2- 172.21.15.32:0/3958877176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f80b0cb0 0x7f58f815bb30 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f58f80b4470 tx=0x7f58ec01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bb80e1e126ec2f0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58f67fc640 1 -- 172.21.15.32:0/3958877176 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec0aa620 con 0x7f58f80b0cb0 2023-12-20T05:44:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58f67fc640 1 -- 172.21.15.32:0/3958877176 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec0aa780 con 0x7f58f80b0cb0 2023-12-20T05:44:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58f67fc640 1 -- 172.21.15.32:0/3958877176 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec0aab10 con 0x7f58f80b0cb0 2023-12-20T05:44:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58fdd8f640 1 -- 172.21.15.32:0/3958877176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f80b0cb0 msgr2=0x7f58f815bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58fdd8f640 1 --2- 172.21.15.32:0/3958877176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f80b0cb0 0x7f58f815bb30 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f58f80b4470 tx=0x7f58ec01c500 comp rx=0 tx=0).stop 2023-12-20T05:44:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58fdd8f640 1 -- 172.21.15.32:0/3958877176 shutdown_connections 2023-12-20T05:44:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58fdd8f640 1 --2- 172.21.15.32:0/3958877176 >> v2:172.21.15.39:3300/0 conn(0x7f58f8070230 0x7f58f80706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58fdd8f640 1 --2- 172.21.15.32:0/3958877176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f80b0cb0 0x7f58f815bb30 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58fdd8f640 1 --2- 172.21.15.32:0/3958877176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f815abb0 0x7f58f80caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58fdd8f640 1 -- 172.21.15.32:0/3958877176 >> 172.21.15.32:0/3958877176 conn(0x7f58f806a510 msgr2=0x7f58f80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58fdd8f640 1 -- 172.21.15.32:0/3958877176 shutdown_connections 2023-12-20T05:44:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.799+0000 7f58fdd8f640 1 -- 172.21.15.32:0/3958877176 wait complete. 2023-12-20T05:44:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58fdd8f640 1 Processor -- start 2023-12-20T05:44:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58fdd8f640 1 -- start start 2023-12-20T05:44:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58fdd8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f8070230 0x7f58f8139070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58fdd8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58f80b0cb0 0x7f58f8148050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58fdd8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f815abb0 0x7f58f8148590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58fdd8f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8146a00 con 0x7f58f815abb0 2023-12-20T05:44:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58fdd8f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8146ce0 con 0x7f58f80b0cb0 2023-12-20T05:44:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58fdd8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f8146fc0 con 0x7f58f8070230 2023-12-20T05:44:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f815abb0 0x7f58f8148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58f7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f815abb0 0x7f58f8148590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58912/0 (socket says 172.21.15.32:58912) 2023-12-20T05:44:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58f7fff640 1 -- 172.21.15.32:0/122837540 learned_addr learned my addr 172.21.15.32:0/122837540 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58f6ffd640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58f80b0cb0 0x7f58f8148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58f77fe640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f8070230 0x7f58f8139070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58f7fff640 1 -- 172.21.15.32:0/122837540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f8070230 msgr2=0x7f58f8139070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58f7fff640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f8070230 0x7f58f8139070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58f7fff640 1 -- 172.21.15.32:0/122837540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58f80b0cb0 msgr2=0x7f58f8148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58f7fff640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58f80b0cb0 0x7f58f8148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58f7fff640 1 -- 172.21.15.32:0/122837540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58ec0094d0 con 0x7f58f815abb0 2023-12-20T05:44:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.803+0000 7f58f7fff640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f815abb0 0x7f58f8148590 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f58f000eed0 tx=0x7f58f000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.807+0000 7f58dffff640 1 -- 172.21.15.32:0/122837540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0010040 con 0x7f58f815abb0 2023-12-20T05:44:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.807+0000 7f58dffff640 1 -- 172.21.15.32:0/122837540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58f000bd20 con 0x7f58f815abb0 2023-12-20T05:44:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.807+0000 7f58dffff640 1 -- 172.21.15.32:0/122837540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58f0014640 con 0x7f58f815abb0 2023-12-20T05:44:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.807+0000 7f58fdd8f640 1 -- 172.21.15.32:0/122837540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f81472a0 con 0x7f58f815abb0 2023-12-20T05:44:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.807+0000 7f58fdd8f640 1 -- 172.21.15.32:0/122837540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f81395b0 con 0x7f58f815abb0 2023-12-20T05:44:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.807+0000 7f58fdd8f640 1 -- 172.21.15.32:0/122837540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f806acf0 con 0x7f58f815abb0 2023-12-20T05:44:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.807+0000 7f58dffff640 1 -- 172.21.15.32:0/122837540 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f58f00147a0 con 0x7f58f815abb0 2023-12-20T05:44:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.807+0000 7f58dffff640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f58e4065b80 0x7f58e4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.807+0000 7f58dffff640 1 -- 172.21.15.32:0/122837540 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(171..171 src has 1..171) v4 ==== 5637+0+0 (secure 0 0 0) 0x7f58f00841e0 con 0x7f58f815abb0 2023-12-20T05:44:20.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.807+0000 7f58f77fe640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f58e4065b80 0x7f58e4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.811+0000 7f58f77fe640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f58e4065b80 0x7f58e4068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009b60 tx=0x7f58e8009190 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:20.811+0000 7f58dffff640 1 -- 172.21.15.32:0/122837540 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f58f00519c0 con 0x7f58f815abb0 2023-12-20T05:44:21.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.007+0000 7f58fdd8f640 1 -- 172.21.15.32:0/122837540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f58f804e000 con 0x7f58f815abb0 2023-12-20T05:44:21.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.015+0000 7f58dffff640 1 -- 172.21.15.32:0/122837540 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 20 and data pool 21 v75) v1 ==== 174+0+0 (secure 0 0 0) 0x7f58f0018020 con 0x7f58f815abb0 2023-12-20T05:44:21.014 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 20 and data pool 21 2023-12-20T05:44:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.023+0000 7f58fdd8f640 1 -- 172.21.15.32:0/122837540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f58e4065b80 msgr2=0x7f58e4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.027+0000 7f58fdd8f640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f58e4065b80 0x7f58e4068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f58e8009b60 tx=0x7f58e8009190 comp rx=0 tx=0).stop 2023-12-20T05:44:21.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.027+0000 7f58fdd8f640 1 -- 172.21.15.32:0/122837540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f815abb0 msgr2=0x7f58f8148590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.027+0000 7f58fdd8f640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f815abb0 0x7f58f8148590 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f58f000eed0 tx=0x7f58f000b480 comp rx=0 tx=0).stop 2023-12-20T05:44:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.027+0000 7f58fdd8f640 1 -- 172.21.15.32:0/122837540 shutdown_connections 2023-12-20T05:44:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.027+0000 7f58fdd8f640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f58e4065b80 0x7f58e4068040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.027+0000 7f58fdd8f640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f815abb0 0x7f58f8148590 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.027+0000 7f58fdd8f640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58f80b0cb0 0x7f58f8148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.027+0000 7f58fdd8f640 1 --2- 172.21.15.32:0/122837540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f8070230 0x7f58f8139070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.027+0000 7f58fdd8f640 1 -- 172.21.15.32:0/122837540 >> 172.21.15.32:0/122837540 conn(0x7f58f806a510 msgr2=0x7f58f80c4bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.027+0000 7f58fdd8f640 1 -- 172.21.15.32:0/122837540 shutdown_connections 2023-12-20T05:44:21.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.027+0000 7f58fdd8f640 1 -- 172.21.15.32:0/122837540 wait complete. 2023-12-20T05:44:21.056 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:44:21.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.199+0000 7f4560ac4640 1 Processor -- start 2023-12-20T05:44:21.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.199+0000 7f4560ac4640 1 -- start start 2023-12-20T05:44:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4560ac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f455c1599e0 0x7f455c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4560ac4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f455c0b0d00 0x7f455c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4560ac4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f455c0c98c0 con 0x7f455c0c90e0 2023-12-20T05:44:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4560ac4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f455c0c9ba0 con 0x7f455c0b0d00 2023-12-20T05:44:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4560ac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0c9e80 con 0x7f455c1599e0 2023-12-20T05:44:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4553fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f455c0c90e0 0x7f455c0b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45162/0 (socket says 172.21.15.32:45162) 2023-12-20T05:44:21.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4553fff640 1 -- 172.21.15.32:0/4102073563 learned_addr learned my addr 172.21.15.32:0/4102073563 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f455a59c640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f455c1599e0 0x7f455c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f455ad9d640 1 --2- 172.21.15.32:0/4102073563 >> v2:172.21.15.39:3300/0 conn(0x7f455c0b0d00 0x7f455c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2849436476 0 0) 0x7f455c0c98c0 con 0x7f455c0c90e0 2023-12-20T05:44:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4548003530 con 0x7f455c0c90e0 2023-12-20T05:44:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2689465264 0 0) 0x7f4548003530 con 0x7f455c0c90e0 2023-12-20T05:44:21.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f455c1599e0 msgr2=0x7f455c0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f455c1599e0 0x7f455c0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 >> v2:172.21.15.39:3300/0 conn(0x7f455c0b0d00 msgr2=0x7f455c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 --2- 172.21.15.32:0/4102073563 >> v2:172.21.15.39:3300/0 conn(0x7f455c0b0d00 0x7f455c0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c0ca160 con 0x7f455c0c90e0 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f4544001680 con 0x7f455c0c90e0 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 >> v1:172.21.15.32:6789/0 conn(0x7f455c0c90e0 legacy=0x7f455c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4548003980 0x7f4548003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45480047d0 0x7f4548004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f455c1599e0 0x7f4548005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0c9ba0 con 0x7f45480047d0 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0c9e80 con 0x7f4548003980 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f454800b990 con 0x7f455c1599e0 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4553fff640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45480047d0 0x7f4548004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f455a59c640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4548003980 0x7f4548003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f455ad9d640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f455c1599e0 0x7f4548005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4553fff640 1 -- 172.21.15.32:0/4102073563 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f455c1599e0 msgr2=0x7f4548005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:21.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4553fff640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f455c1599e0 0x7f4548005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4553fff640 1 -- 172.21.15.32:0/4102073563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4548003980 msgr2=0x7f4548003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:21.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4553fff640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4548003980 0x7f4548003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.203+0000 7f4553fff640 1 -- 172.21.15.32:0/4102073563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4548003530 con 0x7f45480047d0 2023-12-20T05:44:21.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f4553fff640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45480047d0 0x7f4548004c50 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f4544007fa0 tx=0x7f454401ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdd097e13c076da4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:21.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440a7930 con 0x7f45480047d0 2023-12-20T05:44:21.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45440a7a90 con 0x7f45480047d0 2023-12-20T05:44:21.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f4559d9b640 1 -- 172.21.15.32:0/4102073563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45440a7e60 con 0x7f45480047d0 2023-12-20T05:44:21.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f4560ac4640 1 -- 172.21.15.32:0/4102073563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45480047d0 msgr2=0x7f4548004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f4560ac4640 1 --2- 172.21.15.32:0/4102073563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45480047d0 0x7f4548004c50 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f4544007fa0 tx=0x7f454401ee20 comp rx=0 tx=0).stop 2023-12-20T05:44:21.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f455ad9d640 1 -- 172.21.15.32:0/4102073563 reap_dead start 2023-12-20T05:44:21.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f4560ac4640 1 -- 172.21.15.32:0/4102073563 shutdown_connections 2023-12-20T05:44:21.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f4560ac4640 1 -- 172.21.15.32:0/4102073563 >> 172.21.15.32:0/4102073563 conn(0x7f455c06a320 msgr2=0x7f455c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:21.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f4560ac4640 1 -- 172.21.15.32:0/4102073563 shutdown_connections 2023-12-20T05:44:21.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f4560ac4640 1 -- 172.21.15.32:0/4102073563 wait complete. 2023-12-20T05:44:21.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.207+0000 7f4560ac4640 1 Processor -- start 2023-12-20T05:44:21.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4560ac4640 1 -- start start 2023-12-20T05:44:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4560ac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45480047d0 0x7f455c150590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4560ac4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f455c15bdd0 0x7f455c15c250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4560ac4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c0aced0 0x7f455c0ad360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4560ac4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0ad8a0 con 0x7f455c0aced0 2023-12-20T05:44:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4560ac4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f455c0b2460 con 0x7f455c15bdd0 2023-12-20T05:44:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4560ac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f455c151090 con 0x7f45480047d0 2023-12-20T05:44:21.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455a59c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45480047d0 0x7f455c150590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c0aced0 0x7f455c0ad360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455a59c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45480047d0 0x7f455c150590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:38788/0 (socket says 172.21.15.32:38788) 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c0aced0 0x7f455c0ad360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58936/0 (socket says 172.21.15.32:58936) 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455a59c640 1 -- 172.21.15.32:0/4137523229 learned_addr learned my addr 172.21.15.32:0/4137523229 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4553fff640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f455c15bdd0 0x7f455c15c250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455ad9d640 1 -- 172.21.15.32:0/4137523229 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45480047d0 msgr2=0x7f455c150590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455ad9d640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45480047d0 0x7f455c150590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455ad9d640 1 -- 172.21.15.32:0/4137523229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f455c15bdd0 msgr2=0x7f455c15c250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455ad9d640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f455c15bdd0 0x7f455c15c250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455ad9d640 1 -- 172.21.15.32:0/4137523229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f455c0b13c0 con 0x7f455c0aced0 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455a59c640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45480047d0 0x7f455c150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:21.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f455ad9d640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c0aced0 0x7f455c0ad360 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f454800bb20 tx=0x7f455400f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4552ffd640 1 -- 172.21.15.32:0/4137523229 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f455400e040 con 0x7f455c0aced0 2023-12-20T05:44:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4552ffd640 1 -- 172.21.15.32:0/4137523229 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f455400f040 con 0x7f455c0aced0 2023-12-20T05:44:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4552ffd640 1 -- 172.21.15.32:0/4137523229 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4554015670 con 0x7f455c0aced0 2023-12-20T05:44:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4560ac4640 1 -- 172.21.15.32:0/4137523229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f455c0b16a0 con 0x7f455c0aced0 2023-12-20T05:44:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4560ac4640 1 -- 172.21.15.32:0/4137523229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f455c0b1c60 con 0x7f455c0aced0 2023-12-20T05:44:21.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.211+0000 7f4560ac4640 1 -- 172.21.15.32:0/4137523229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f455c04e000 con 0x7f455c0aced0 2023-12-20T05:44:21.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.215+0000 7f4552ffd640 1 -- 172.21.15.32:0/4137523229 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f45540157d0 con 0x7f455c0aced0 2023-12-20T05:44:21.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.215+0000 7f4552ffd640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f453c065b80 0x7f453c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.215+0000 7f4552ffd640 1 -- 172.21.15.32:0/4137523229 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4554083990 con 0x7f455c0aced0 2023-12-20T05:44:21.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.215+0000 7f455a59c640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f453c065b80 0x7f453c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.215+0000 7f455a59c640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f453c065b80 0x7f453c068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f455c0b2600 tx=0x7f454c006f90 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:21.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.219+0000 7f4552ffd640 1 -- 172.21.15.32:0/4137523229 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4554051180 con 0x7f455c0aced0 2023-12-20T05:44:21.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 -- 172.21.15.32:0/4137523229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f455c0cf550 con 0x7f455c0aced0 2023-12-20T05:44:21.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4552ffd640 1 -- 172.21.15.32:0/4137523229 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16120 (secure 0 0 0) 0x7f45540547a0 con 0x7f455c0aced0 2023-12-20T05:44:21.377 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:21.377 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":172,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:44:21.008988+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:44:18.671073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-20T05:44:20.010875+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:44:21.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 -- 172.21.15.32:0/4137523229 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f453c065b80 msgr2=0x7f453c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f453c065b80 0x7f453c068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f455c0b2600 tx=0x7f454c006f90 comp rx=0 tx=0).stop 2023-12-20T05:44:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 -- 172.21.15.32:0/4137523229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c0aced0 msgr2=0x7f455c0ad360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c0aced0 0x7f455c0ad360 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f454800bb20 tx=0x7f455400f580 comp rx=0 tx=0).stop 2023-12-20T05:44:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 -- 172.21.15.32:0/4137523229 shutdown_connections 2023-12-20T05:44:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f453c065b80 0x7f453c068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f455c0aced0 0x7f455c0ad360 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f455c15bdd0 0x7f455c15c250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 --2- 172.21.15.32:0/4137523229 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f45480047d0 0x7f455c150590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 -- 172.21.15.32:0/4137523229 >> 172.21.15.32:0/4137523229 conn(0x7f455c06a320 msgr2=0x7f455c0caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 -- 172.21.15.32:0/4137523229 shutdown_connections 2023-12-20T05:44:21.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.379+0000 7f4560ac4640 1 -- 172.21.15.32:0/4137523229 wait complete. 2023-12-20T05:44:21.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:44:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.543+0000 7f634a339640 1 Processor -- start 2023-12-20T05:44:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.543+0000 7f634a339640 1 -- start start 2023-12-20T05:44:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.543+0000 7f634a339640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f634406deb0 0x7f6344161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.543+0000 7f634a339640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6344071920 con 0x7f634415ce30 2023-12-20T05:44:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.543+0000 7f634a339640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6344070de0 con 0x7f6344070950 2023-12-20T05:44:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.543+0000 7f634a339640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6344071080 con 0x7f634406deb0 2023-12-20T05:44:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.543+0000 7f6343fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f634415ce30 0x7f634406d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45178/0 (socket says 172.21.15.32:45178) 2023-12-20T05:44:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.543+0000 7f6343fff640 1 -- 172.21.15.32:0/150486214 learned_addr learned my addr 172.21.15.32:0/150486214 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.543+0000 7f63437fe640 1 --2- 172.21.15.32:0/150486214 >> v2:172.21.15.32:3300/0 conn(0x7f634406deb0 0x7f6344161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.547+0000 7f6342ffd640 1 -- 172.21.15.32:0/150486214 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3193880978 0 0) 0x7f6344071920 con 0x7f634415ce30 2023-12-20T05:44:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.547+0000 7f6342ffd640 1 -- 172.21.15.32:0/150486214 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f632c003530 con 0x7f634415ce30 2023-12-20T05:44:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.547+0000 7f63437fe640 1 -- 172.21.15.32:0/150486214 >> v1:172.21.15.39:6789/0 conn(0x7f6344070950 legacy=0x7f63440cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.547+0000 7f63437fe640 1 -- 172.21.15.32:0/150486214 >> v1:172.21.15.32:6789/0 conn(0x7f634415ce30 legacy=0x7f634406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.547+0000 7f63437fe640 1 -- 172.21.15.32:0/150486214 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6344071360 con 0x7f634406deb0 2023-12-20T05:44:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.547+0000 7f63437fe640 1 --2- 172.21.15.32:0/150486214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f634406deb0 0x7f6344161560 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f63380097c0 tx=0x7f633801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f19c3ea0cb5bdf8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.547+0000 7f6342ffd640 1 -- 172.21.15.32:0/150486214 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63380ab600 con 0x7f634406deb0 2023-12-20T05:44:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.547+0000 7f6342ffd640 1 -- 172.21.15.32:0/150486214 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63380a7050 con 0x7f634406deb0 2023-12-20T05:44:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.547+0000 7f6342ffd640 1 -- 172.21.15.32:0/150486214 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63380ab980 con 0x7f634406deb0 2023-12-20T05:44:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.547+0000 7f634a339640 1 -- 172.21.15.32:0/150486214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f634406deb0 msgr2=0x7f6344161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.547+0000 7f634a339640 1 --2- 172.21.15.32:0/150486214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f634406deb0 0x7f6344161560 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f63380097c0 tx=0x7f633801c480 comp rx=0 tx=0).stop 2023-12-20T05:44:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 -- 172.21.15.32:0/150486214 shutdown_connections 2023-12-20T05:44:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 --2- 172.21.15.32:0/150486214 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f634406deb0 0x7f6344161560 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 -- 172.21.15.32:0/150486214 >> 172.21.15.32:0/150486214 conn(0x7f63440cba50 msgr2=0x7f63440cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 -- 172.21.15.32:0/150486214 shutdown_connections 2023-12-20T05:44:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 -- 172.21.15.32:0/150486214 wait complete. 2023-12-20T05:44:21.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 Processor -- start 2023-12-20T05:44:21.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 -- start start 2023-12-20T05:44:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f634406deb0 0x7f6344154760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6344070950 0x7f634409f8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634415ce30 0x7f6344153b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634409d460 con 0x7f634406deb0 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f634409d740 con 0x7f634415ce30 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f634a339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f634409da20 con 0x7f6344070950 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f6343fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f634406deb0 0x7f6344154760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f6343fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f634406deb0 0x7f6344154760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58948/0 (socket says 172.21.15.32:58948) 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f6343fff640 1 -- 172.21.15.32:0/2520553381 learned_addr learned my addr 172.21.15.32:0/2520553381 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f6343fff640 1 -- 172.21.15.32:0/2520553381 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6344070950 msgr2=0x7f634409f8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f6343fff640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6344070950 0x7f634409f8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f6343fff640 1 -- 172.21.15.32:0/2520553381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634415ce30 msgr2=0x7f6344153b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f6343fff640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634415ce30 0x7f6344153b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f6343fff640 1 -- 172.21.15.32:0/2520553381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6338009420 con 0x7f634406deb0 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.551+0000 7f6343fff640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f634406deb0 0x7f6344154760 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f633400c420 tx=0x7f633400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.555+0000 7f6340ff9640 1 -- 172.21.15.32:0/2520553381 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6334005cf0 con 0x7f634406deb0 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.555+0000 7f6340ff9640 1 -- 172.21.15.32:0/2520553381 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6334005e50 con 0x7f634406deb0 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.555+0000 7f6340ff9640 1 -- 172.21.15.32:0/2520553381 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63340072f0 con 0x7f634406deb0 2023-12-20T05:44:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.555+0000 7f634a339640 1 -- 172.21.15.32:0/2520553381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f634409dd00 con 0x7f634406deb0 2023-12-20T05:44:21.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.555+0000 7f634a339640 1 -- 172.21.15.32:0/2520553381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6344154ca0 con 0x7f634406deb0 2023-12-20T05:44:21.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.555+0000 7f634a339640 1 -- 172.21.15.32:0/2520553381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f634404e000 con 0x7f634406deb0 2023-12-20T05:44:21.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.555+0000 7f6340ff9640 1 -- 172.21.15.32:0/2520553381 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6334007450 con 0x7f634406deb0 2023-12-20T05:44:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.555+0000 7f6340ff9640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6330065b80 0x7f6330068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.555+0000 7f6340ff9640 1 -- 172.21.15.32:0/2520553381 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f63340829c0 con 0x7f634406deb0 2023-12-20T05:44:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.555+0000 7f63437fe640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6330065b80 0x7f6330068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.555+0000 7f63437fe640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6330065b80 0x7f6330068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f63380ad000 tx=0x7f6338004740 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:21.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.559+0000 7f6340ff9640 1 -- 172.21.15.32:0/2520553381 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6334053cd0 con 0x7f634406deb0 2023-12-20T05:44:21.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.719+0000 7f634a339640 1 -- 172.21.15.32:0/2520553381 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f634409d5a0 con 0x7f634406deb0 2023-12-20T05:44:21.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.719+0000 7f6340ff9640 1 -- 172.21.15.32:0/2520553381 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v172) v1 ==== 74+0+16120 (secure 0 0 0) 0x7f634409d5a0 con 0x7f634406deb0 2023-12-20T05:44:21.718 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:21.719 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":172,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:44:21.008988+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:44:18.671073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-20T05:44:20.010875+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:44:21.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 -- 172.21.15.32:0/2520553381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6330065b80 msgr2=0x7f6330068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6330065b80 0x7f6330068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f63380ad000 tx=0x7f6338004740 comp rx=0 tx=0).stop 2023-12-20T05:44:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 -- 172.21.15.32:0/2520553381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f634406deb0 msgr2=0x7f6344154760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f634406deb0 0x7f6344154760 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f633400c420 tx=0x7f633400c8e0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 -- 172.21.15.32:0/2520553381 shutdown_connections 2023-12-20T05:44:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6330065b80 0x7f6330068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f634415ce30 0x7f6344153b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6344070950 0x7f634409f8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 --2- 172.21.15.32:0/2520553381 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f634406deb0 0x7f6344154760 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 -- 172.21.15.32:0/2520553381 >> 172.21.15.32:0/2520553381 conn(0x7f63440cba50 msgr2=0x7f63440cbec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 -- 172.21.15.32:0/2520553381 shutdown_connections 2023-12-20T05:44:21.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.723+0000 7f634a339640 1 -- 172.21.15.32:0/2520553381 wait complete. 2023-12-20T05:44:21.740 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T05:44:21.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.887+0000 7f7efcf7b640 1 Processor -- start 2023-12-20T05:44:21.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.887+0000 7f7efcf7b640 1 -- start start 2023-12-20T05:44:21.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7efcf7b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7ef815cef0 0x7f7ef8161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7efcf7b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ef80cde70 0x7f7ef80ce280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7efcf7b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7ef80cdbd0 con 0x7f7ef8071760 2023-12-20T05:44:21.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7efcf7b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef80cd7b0 con 0x7f7ef80cde70 2023-12-20T05:44:21.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7efcf7b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7ef8071be0 con 0x7f7ef815cef0 2023-12-20T05:44:21.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7ef6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7ef80cde70 0x7f7ef80ce280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7eeffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7ef8071760 0x7f7ef80ca270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45186/0 (socket says 172.21.15.32:45186) 2023-12-20T05:44:21.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7eeffff640 1 -- 172.21.15.32:0/856788131 learned_addr learned my addr 172.21.15.32:0/856788131 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:21.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7ef659c640 1 --2- 172.21.15.32:0/856788131 >> v2:172.21.15.39:3300/0 conn(0x7f7ef815cef0 0x7f7ef8161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7ef6d9d640 1 -- 172.21.15.32:0/856788131 >> v2:172.21.15.39:3300/0 conn(0x7f7ef815cef0 msgr2=0x7f7ef8161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:21.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7ef6d9d640 1 --2- 172.21.15.32:0/856788131 >> v2:172.21.15.39:3300/0 conn(0x7f7ef815cef0 0x7f7ef8161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7ef6d9d640 1 -- 172.21.15.32:0/856788131 >> v1:172.21.15.32:6789/0 conn(0x7f7ef8071760 legacy=0x7f7ef80ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7ef6d9d640 1 -- 172.21.15.32:0/856788131 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef80cd460 con 0x7f7ef80cde70 2023-12-20T05:44:21.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.891+0000 7f7ef5d9b640 1 -- 172.21.15.32:0/856788131 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911635548 0 0) 0x7f7ef80cdbd0 con 0x7f7ef8071760 2023-12-20T05:44:21.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7ef6d9d640 1 --2- 172.21.15.32:0/856788131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef80cde70 0x7f7ef80ce280 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80cf8b0 tx=0x7f7ef001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=376e6f4d88e1910f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:21.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7ef5d9b640 1 -- 172.21.15.32:0/856788131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00a5830 con 0x7f7ef80cde70 2023-12-20T05:44:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7ef5d9b640 1 -- 172.21.15.32:0/856788131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ef00a5990 con 0x7f7ef80cde70 2023-12-20T05:44:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7ef5d9b640 1 -- 172.21.15.32:0/856788131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ef00a5d20 con 0x7f7ef80cde70 2023-12-20T05:44:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7efcf7b640 1 -- 172.21.15.32:0/856788131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef80cde70 msgr2=0x7f7ef80ce280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7efcf7b640 1 --2- 172.21.15.32:0/856788131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef80cde70 0x7f7ef80ce280 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f7ef80cf8b0 tx=0x7f7ef001c880 comp rx=0 tx=0).stop 2023-12-20T05:44:21.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7efcf7b640 1 -- 172.21.15.32:0/856788131 shutdown_connections 2023-12-20T05:44:21.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7efcf7b640 1 --2- 172.21.15.32:0/856788131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef80cde70 0x7f7ef80ce280 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7efcf7b640 1 --2- 172.21.15.32:0/856788131 >> v2:172.21.15.39:3300/0 conn(0x7f7ef815cef0 0x7f7ef8161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7efcf7b640 1 -- 172.21.15.32:0/856788131 >> 172.21.15.32:0/856788131 conn(0x7f7ef806b230 msgr2=0x7f7ef806b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:21.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7efcf7b640 1 -- 172.21.15.32:0/856788131 shutdown_connections 2023-12-20T05:44:21.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7efcf7b640 1 -- 172.21.15.32:0/856788131 wait complete. 2023-12-20T05:44:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7efcf7b640 1 Processor -- start 2023-12-20T05:44:21.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.895+0000 7f7efcf7b640 1 -- start start 2023-12-20T05:44:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7efcf7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ef8071760 0x7f7ef815fe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7efcf7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef80cde70 0x7f7ef8149f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7efcf7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ef815cef0 0x7f7ef815dcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7efcf7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef81604f0 con 0x7f7ef80cde70 2023-12-20T05:44:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7efcf7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef815b610 con 0x7f7ef8071760 2023-12-20T05:44:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7efcf7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7ef815a850 con 0x7f7ef815cef0 2023-12-20T05:44:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef80cde70 0x7f7ef8149f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef80cde70 0x7f7ef8149f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58966/0 (socket says 172.21.15.32:58966) 2023-12-20T05:44:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeffff640 1 -- 172.21.15.32:0/474770232 learned_addr learned my addr 172.21.15.32:0/474770232 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:21.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7ef6d9d640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ef815cef0 0x7f7ef815dcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7ef659c640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ef8071760 0x7f7ef815fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeffff640 1 -- 172.21.15.32:0/474770232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ef815cef0 msgr2=0x7f7ef815dcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeffff640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ef815cef0 0x7f7ef815dcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeffff640 1 -- 172.21.15.32:0/474770232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ef8071760 msgr2=0x7f7ef815fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeffff640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ef8071760 0x7f7ef815fe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeffff640 1 -- 172.21.15.32:0/474770232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7ef0009a70 con 0x7f7ef80cde70 2023-12-20T05:44:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7ef659c640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ef8071760 0x7f7ef815fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7ef6d9d640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ef815cef0 0x7f7ef815dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:21.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeffff640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef80cde70 0x7f7ef8149f70 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c520 tx=0x7f7ee000c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeeffd640 1 -- 172.21.15.32:0/474770232 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00074e0 con 0x7f7ef80cde70 2023-12-20T05:44:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeeffd640 1 -- 172.21.15.32:0/474770232 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7ee0007640 con 0x7f7ef80cde70 2023-12-20T05:44:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7eeeffd640 1 -- 172.21.15.32:0/474770232 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ee00049f0 con 0x7f7ef80cde70 2023-12-20T05:44:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7efcf7b640 1 -- 172.21.15.32:0/474770232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7ef815ab30 con 0x7f7ef80cde70 2023-12-20T05:44:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7efcf7b640 1 -- 172.21.15.32:0/474770232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7ef815b0f0 con 0x7f7ef80cde70 2023-12-20T05:44:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.899+0000 7f7efcf7b640 1 -- 172.21.15.32:0/474770232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7ef815b310 con 0x7f7ef80cde70 2023-12-20T05:44:21.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.903+0000 7f7eeeffd640 1 -- 172.21.15.32:0/474770232 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7ee0004b50 con 0x7f7ef80cde70 2023-12-20T05:44:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.903+0000 7f7eeeffd640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ed8065bb0 0x7f7ed8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.903+0000 7f7eeeffd640 1 -- 172.21.15.32:0/474770232 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(172..172 src has 1..172) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f7ee0082490 con 0x7f7ef80cde70 2023-12-20T05:44:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.903+0000 7f7ef659c640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ed8065bb0 0x7f7ed8068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:21.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.903+0000 7f7ef659c640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ed8065bb0 0x7f7ed8068070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009b10 tx=0x7f7ee8008040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:21.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:21.907+0000 7f7eeeffd640 1 -- 172.21.15.32:0/474770232 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7ee004fdc0 con 0x7f7ef80cde70 2023-12-20T05:44:22.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:22.099+0000 7f7efcf7b640 1 -- 172.21.15.32:0/474770232 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f7ef804e000 con 0x7f7ef80cde70 2023-12-20T05:44:23.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7eeeffd640 1 -- 172.21.15.32:0/474770232 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v78) v1 ==== 118+0+0 (secure 0 0 0) 0x7f7ee00180b0 con 0x7f7ef80cde70 2023-12-20T05:44:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 -- 172.21.15.32:0/474770232 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ed8065bb0 msgr2=0x7f7ed8068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ed8065bb0 0x7f7ed8068070 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f7ee8009b10 tx=0x7f7ee8008040 comp rx=0 tx=0).stop 2023-12-20T05:44:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 -- 172.21.15.32:0/474770232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef80cde70 msgr2=0x7f7ef8149f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef80cde70 0x7f7ef8149f70 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f7ee000c520 tx=0x7f7ee000c9e0 comp rx=0 tx=0).stop 2023-12-20T05:44:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 -- 172.21.15.32:0/474770232 shutdown_connections 2023-12-20T05:44:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7ed8065bb0 0x7f7ed8068070 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7ef815cef0 0x7f7ef815dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7ef80cde70 0x7f7ef8149f70 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 --2- 172.21.15.32:0/474770232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7ef8071760 0x7f7ef815fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:23.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 -- 172.21.15.32:0/474770232 >> 172.21.15.32:0/474770232 conn(0x7f7ef806b230 msgr2=0x7f7ef8155f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:23.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 -- 172.21.15.32:0/474770232 shutdown_connections 2023-12-20T05:44:23.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.031+0000 7f7efcf7b640 1 -- 172.21.15.32:0/474770232 wait complete. 2023-12-20T05:44:23.054 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T05:44:23.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T05:44:23.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba1da73640 1 Processor -- start 2023-12-20T05:44:23.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba1da73640 1 -- start start 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba1da73640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fba18070940 0x7fba18164120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba1da73640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba180cc3b0 con 0x7fba181591e0 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba1da73640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fba180c8e30 con 0x7fba1806cdf0 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba1da73640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fba1806d280 con 0x7fba18070940 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba177fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fba181591e0 0x7fba180caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45200/0 (socket says 172.21.15.32:45200) 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba177fe640 1 -- 172.21.15.32:0/3361877315 learned_addr learned my addr 172.21.15.32:0/3361877315 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba17fff640 1 --2- 172.21.15.32:0/3361877315 >> v2:172.21.15.32:3300/0 conn(0x7fba18070940 0x7fba18164120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba167fc640 1 -- 172.21.15.32:0/3361877315 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 129294174 0 0) 0x7fba180cc3b0 con 0x7fba181591e0 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba167fc640 1 -- 172.21.15.32:0/3361877315 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fba00003530 con 0x7fba181591e0 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba17fff640 1 -- 172.21.15.32:0/3361877315 >> v1:172.21.15.39:6789/0 conn(0x7fba1806cdf0 legacy=0x7fba18070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba17fff640 1 -- 172.21.15.32:0/3361877315 >> v1:172.21.15.32:6789/0 conn(0x7fba181591e0 legacy=0x7fba180caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:23.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.207+0000 7fba17fff640 1 -- 172.21.15.32:0/3361877315 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba180c90d0 con 0x7fba18070940 2023-12-20T05:44:23.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.211+0000 7fba17fff640 1 --2- 172.21.15.32:0/3361877315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba18070940 0x7fba18164120 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fba10009d90 tx=0x7fba1001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c845bedf94fc8e3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:23.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.211+0000 7fba167fc640 1 -- 172.21.15.32:0/3361877315 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100a58e0 con 0x7fba18070940 2023-12-20T05:44:23.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.211+0000 7fba167fc640 1 -- 172.21.15.32:0/3361877315 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba100a5a40 con 0x7fba18070940 2023-12-20T05:44:23.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.211+0000 7fba167fc640 1 -- 172.21.15.32:0/3361877315 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100a5dd0 con 0x7fba18070940 2023-12-20T05:44:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.211+0000 7fba1da73640 1 -- 172.21.15.32:0/3361877315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba18070940 msgr2=0x7fba18164120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.211+0000 7fba1da73640 1 --2- 172.21.15.32:0/3361877315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba18070940 0x7fba18164120 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fba10009d90 tx=0x7fba1001c8c0 comp rx=0 tx=0).stop 2023-12-20T05:44:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.211+0000 7fba1da73640 1 -- 172.21.15.32:0/3361877315 shutdown_connections 2023-12-20T05:44:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.211+0000 7fba1da73640 1 --2- 172.21.15.32:0/3361877315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba18070940 0x7fba18164120 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.211+0000 7fba1da73640 1 -- 172.21.15.32:0/3361877315 >> 172.21.15.32:0/3361877315 conn(0x7fba1806a510 msgr2=0x7fba180ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.211+0000 7fba1da73640 1 -- 172.21.15.32:0/3361877315 shutdown_connections 2023-12-20T05:44:23.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.211+0000 7fba1da73640 1 -- 172.21.15.32:0/3361877315 wait complete. 2023-12-20T05:44:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba1da73640 1 Processor -- start 2023-12-20T05:44:23.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba1da73640 1 -- start start 2023-12-20T05:44:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba1da73640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba18070940 0x7fba180b3cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba1da73640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba181591e0 0x7fba18155320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba1da73640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba181470c0 0x7fba181574a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba1da73640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba180b1e00 con 0x7fba181470c0 2023-12-20T05:44:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba1da73640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fba18153420 con 0x7fba181591e0 2023-12-20T05:44:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba1da73640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fba180b1580 con 0x7fba18070940 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba181470c0 0x7fba181574a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba17fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba181470c0 0x7fba181574a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58984/0 (socket says 172.21.15.32:58984) 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba17fff640 1 -- 172.21.15.32:0/959213046 learned_addr learned my addr 172.21.15.32:0/959213046 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba177fe640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba18070940 0x7fba180b3cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba16ffd640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba181591e0 0x7fba18155320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba17fff640 1 -- 172.21.15.32:0/959213046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba18070940 msgr2=0x7fba180b3cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba17fff640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba18070940 0x7fba180b3cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba17fff640 1 -- 172.21.15.32:0/959213046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba181591e0 msgr2=0x7fba18155320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba17fff640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba181591e0 0x7fba18155320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba17fff640 1 -- 172.21.15.32:0/959213046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fba10009a70 con 0x7fba181470c0 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba177fe640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba18070940 0x7fba180b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:23.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba16ffd640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba181591e0 0x7fba18155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba17fff640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba181470c0 0x7fba181574a0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fba1001a4f0 tx=0x7fba10019660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fb9fffff640 1 -- 172.21.15.32:0/959213046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100ad850 con 0x7fba181470c0 2023-12-20T05:44:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fb9fffff640 1 -- 172.21.15.32:0/959213046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fba100ad9b0 con 0x7fba181470c0 2023-12-20T05:44:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba1da73640 1 -- 172.21.15.32:0/959213046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fba180b1840 con 0x7fba181470c0 2023-12-20T05:44:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fb9fffff640 1 -- 172.21.15.32:0/959213046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fba100add20 con 0x7fba181470c0 2023-12-20T05:44:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.215+0000 7fba1da73640 1 -- 172.21.15.32:0/959213046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fba180b42b0 con 0x7fba181470c0 2023-12-20T05:44:23.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.219+0000 7fba1da73640 1 -- 172.21.15.32:0/959213046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fba18153650 con 0x7fba181470c0 2023-12-20T05:44:23.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.219+0000 7fb9fffff640 1 -- 172.21.15.32:0/959213046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fba100b4050 con 0x7fba181470c0 2023-12-20T05:44:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.219+0000 7fb9fffff640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba04065bb0 0x7fba04068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.219+0000 7fb9fffff640 1 -- 172.21.15.32:0/959213046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fba1011df50 con 0x7fba181470c0 2023-12-20T05:44:23.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.219+0000 7fba177fe640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba04065bb0 0x7fba04068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:23.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.219+0000 7fba177fe640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba04065bb0 0x7fba04068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fba080011f0 tx=0x7fba08009040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:23.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.223+0000 7fb9fffff640 1 -- 172.21.15.32:0/959213046 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fba18153650 con 0x7fba181470c0 2023-12-20T05:44:23.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:23.415+0000 7fba1da73640 1 -- 172.21.15.32:0/959213046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fba18153650 con 0x7fba181470c0 2023-12-20T05:44:24.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.031+0000 7fb9fffff640 1 -- 172.21.15.32:0/959213046 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v79) v1 ==== 122+0+0 (secure 0 0 0) 0x7fba18153650 con 0x7fba181470c0 2023-12-20T05:44:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 -- 172.21.15.32:0/959213046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba04065bb0 msgr2=0x7fba04068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba04065bb0 0x7fba04068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fba080011f0 tx=0x7fba08009040 comp rx=0 tx=0).stop 2023-12-20T05:44:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 -- 172.21.15.32:0/959213046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba181470c0 msgr2=0x7fba181574a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba181470c0 0x7fba181574a0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fba1001a4f0 tx=0x7fba10019660 comp rx=0 tx=0).stop 2023-12-20T05:44:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 -- 172.21.15.32:0/959213046 shutdown_connections 2023-12-20T05:44:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fba04065bb0 0x7fba04068070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fba181470c0 0x7fba181574a0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fba181591e0 0x7fba18155320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 --2- 172.21.15.32:0/959213046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fba18070940 0x7fba180b3cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 -- 172.21.15.32:0/959213046 >> 172.21.15.32:0/959213046 conn(0x7fba1806a510 msgr2=0x7fba18156b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 -- 172.21.15.32:0/959213046 shutdown_connections 2023-12-20T05:44:24.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.035+0000 7fba1da73640 1 -- 172.21.15.32:0/959213046 wait complete. 2023-12-20T05:44:24.055 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T05:44:24.055 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:44:24.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.203+0000 7f8edb3f3640 1 Processor -- start 2023-12-20T05:44:24.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.203+0000 7f8edb3f3640 1 -- start start 2023-12-20T05:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.203+0000 7f8edb3f3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8ed406cf10 0x7f8ed40c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.203+0000 7f8edb3f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ed4164ef0 0x7f8ed40c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.203+0000 7f8edb3f3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8ed40ae3f0 con 0x7f8ed40ae010 2023-12-20T05:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.203+0000 7f8edb3f3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8ed4161800 con 0x7f8ed406cf10 2023-12-20T05:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.203+0000 7f8edb3f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4165380 con 0x7f8ed4164ef0 2023-12-20T05:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.203+0000 7f8ed918f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8ed40ae010 0x7f8ed4071470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45210/0 (socket says 172.21.15.32:45210) 2023-12-20T05:44:24.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.203+0000 7f8ed918f640 1 -- 172.21.15.32:0/2216700098 learned_addr learned my addr 172.21.15.32:0/2216700098 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:24.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ed9990640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ed4164ef0 0x7f8ed40c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ed898e640 1 --2- 172.21.15.32:0/2216700098 >> v2:172.21.15.39:3300/0 conn(0x7f8ed406cf10 0x7f8ed40c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1639759095 0 0) 0x7f8ed40ae3f0 con 0x7f8ed40ae010 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8ec0003530 con 0x7f8ed40ae010 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4291261992 0 0) 0x7f8ec0003530 con 0x7f8ed40ae010 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ed4164ef0 msgr2=0x7f8ed40c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ed4164ef0 0x7f8ed40c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 >> v2:172.21.15.39:3300/0 conn(0x7f8ed406cf10 msgr2=0x7f8ed40c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 --2- 172.21.15.32:0/2216700098 >> v2:172.21.15.39:3300/0 conn(0x7f8ed406cf10 0x7f8ed40c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed4165660 con 0x7f8ed40ae010 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f8ed0001800 con 0x7f8ed40ae010 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 >> v1:172.21.15.32:6789/0 conn(0x7f8ed40ae010 legacy=0x7f8ed4071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ec0003980 0x7f8ec0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ec00047d0 0x7f8ec0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 0x7f8ec000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4161800 con 0x7f8ec0005190 2023-12-20T05:44:24.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed4165380 con 0x7f8ec0003980 2023-12-20T05:44:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ec000bf20 con 0x7f8ec00047d0 2023-12-20T05:44:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ed9990640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ed4164ef0 0x7f8ed40c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ed898e640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ec00047d0 0x7f8ec0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ed918f640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ec0003980 0x7f8ec0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ed9990640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 0x7f8ec000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ed9990640 1 -- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ec00047d0 msgr2=0x7f8ec0004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ed9990640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ec00047d0 0x7f8ec0004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ed9990640 1 -- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ec0003980 msgr2=0x7f8ec0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ed9990640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ec0003980 0x7f8ec0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.207+0000 7f8ed9990640 1 -- 172.21.15.32:0/2216700098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ec0003530 con 0x7f8ec0005190 2023-12-20T05:44:24.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8ed9990640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 0x7f8ec000b990 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40cf420 tx=0x7f8ecc01ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2644a54db06f1b74 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc003c60 con 0x7f8ec0005190 2023-12-20T05:44:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ecc00f040 con 0x7f8ec0005190 2023-12-20T05:44:24.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8ecbfff640 1 -- 172.21.15.32:0/2216700098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc0a5940 con 0x7f8ec0005190 2023-12-20T05:44:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2216700098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 msgr2=0x7f8ec000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 --2- 172.21.15.32:0/2216700098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 0x7f8ec000b990 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40cf420 tx=0x7f8ecc01ccc0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8ed9990640 1 -- 172.21.15.32:0/2216700098 reap_dead start 2023-12-20T05:44:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2216700098 shutdown_connections 2023-12-20T05:44:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2216700098 >> 172.21.15.32:0/2216700098 conn(0x7f8ed406a320 msgr2=0x7f8ed40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2216700098 shutdown_connections 2023-12-20T05:44:24.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2216700098 wait complete. 2023-12-20T05:44:24.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 Processor -- start 2023-12-20T05:44:24.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 -- start start 2023-12-20T05:44:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 0x7f8ed40b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ed41f6a90 0x7f8ed40b2d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed40b3270 0x7f8ed40b06f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed40b4530 con 0x7f8ec0005190 2023-12-20T05:44:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed41f74e0 con 0x7f8ed40b3270 2023-12-20T05:44:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.211+0000 7f8edb3f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8ed41f77c0 con 0x7f8ed41f6a90 2023-12-20T05:44:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ed918f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 0x7f8ed40b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ed918f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 0x7f8ed40b3ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58990/0 (socket says 172.21.15.32:58990) 2023-12-20T05:44:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ed918f640 1 -- 172.21.15.32:0/2746491046 learned_addr learned my addr 172.21.15.32:0/2746491046 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ed898e640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ed41f6a90 0x7f8ed40b2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ed9990640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed40b3270 0x7f8ed40b06f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ed918f640 1 -- 172.21.15.32:0/2746491046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ed41f6a90 msgr2=0x7f8ed40b2d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ed918f640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ed41f6a90 0x7f8ed40b2d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ed918f640 1 -- 172.21.15.32:0/2746491046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed40b3270 msgr2=0x7f8ed40b06f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ed918f640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed40b3270 0x7f8ed40b06f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ed918f640 1 -- 172.21.15.32:0/2746491046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8ed40c2870 con 0x7f8ec0005190 2023-12-20T05:44:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ed918f640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 0x7f8ed40b3ff0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0012540 tx=0x7f8ed0012940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ec9ffb640 1 -- 172.21.15.32:0/2746491046 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0014750 con 0x7f8ec0005190 2023-12-20T05:44:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ec9ffb640 1 -- 172.21.15.32:0/2746491046 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8ed00148b0 con 0x7f8ec0005190 2023-12-20T05:44:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2746491046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8ed40c2b50 con 0x7f8ec0005190 2023-12-20T05:44:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ec9ffb640 1 -- 172.21.15.32:0/2746491046 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8ed0014c60 con 0x7f8ec0005190 2023-12-20T05:44:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2746491046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8ed40b1770 con 0x7f8ec0005190 2023-12-20T05:44:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2746491046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8ed41f7a00 con 0x7f8ec0005190 2023-12-20T05:44:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.215+0000 7f8ec9ffb640 1 -- 172.21.15.32:0/2746491046 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8ed00153e0 con 0x7f8ec0005190 2023-12-20T05:44:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.219+0000 7f8ec9ffb640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065bb0 0x7f8eb8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.219+0000 7f8ec9ffb640 1 -- 172.21.15.32:0/2746491046 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f8ed0085e70 con 0x7f8ec0005190 2023-12-20T05:44:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.219+0000 7f8ed898e640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065bb0 0x7f8eb8068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.219+0000 7f8ed898e640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065bb0 0x7f8eb8068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40c1e30 tx=0x7f8ec4009190 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:24.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.223+0000 7f8ec9ffb640 1 -- 172.21.15.32:0/2746491046 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8ed41f7a00 con 0x7f8ec0005190 2023-12-20T05:44:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.411+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2746491046 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f8ed40c26d0 con 0x7f8ec0005190 2023-12-20T05:44:24.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.411+0000 7f8ec9ffb640 1 -- 172.21.15.32:0/2746491046 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4710 (secure 0 0 0) 0x7f8ed40c26d0 con 0x7f8ec0005190 2023-12-20T05:44:24.411 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:24.411 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1922232083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1922232083},{"type":"v1","addr":"172.21.15.39:6833","nonce":1922232083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9103,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428109066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428109066},{"type":"v1","addr":"172.21.15.39:6835","nonce":428109066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1595154235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1595154235},{"type":"v1","addr":"172.21.15.32:6836","nonce":1595154235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:44:21.009338+0000","modified":"2023-12-20T05:44:24.030393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2806059816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2806059816},{"type":"v1","addr":"172.21.15.32:6837","nonce":2806059816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-20T05:44:24.411 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 79 2023-12-20T05:44:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2746491046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065bb0 msgr2=0x7f8eb8068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065bb0 0x7f8eb8068070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f8ed40c1e30 tx=0x7f8ec4009190 comp rx=0 tx=0).stop 2023-12-20T05:44:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2746491046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 msgr2=0x7f8ed40b3ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 0x7f8ed40b3ff0 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7f8ed0012540 tx=0x7f8ed0012940 comp rx=0 tx=0).stop 2023-12-20T05:44:24.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2746491046 shutdown_connections 2023-12-20T05:44:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8eb8065bb0 0x7f8eb8068070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8ed40b3270 0x7f8ed40b06f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8ed41f6a90 0x7f8ed40b2d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 --2- 172.21.15.32:0/2746491046 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8ec0005190 0x7f8ed40b3ff0 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2746491046 >> 172.21.15.32:0/2746491046 conn(0x7f8ed406a320 msgr2=0x7f8ed415ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2746491046 shutdown_connections 2023-12-20T05:44:24.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.415+0000 7f8edb3f3640 1 -- 172.21.15.32:0/2746491046 wait complete. 2023-12-20T05:44:24.433 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T05:44:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4cad7b640 1 Processor -- start 2023-12-20T05:44:24.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4cad7b640 1 -- start start 2023-12-20T05:44:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4cad7b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4c40b2ae0 0x7fc4c40cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4cad7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c41608e0 0x7fc4c4160d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4cad7b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4c40ce6c0 con 0x7fc4c415cd70 2023-12-20T05:44:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4cad7b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4c41618b0 con 0x7fc4c40b2ae0 2023-12-20T05:44:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4cad7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c41615d0 con 0x7fc4c41608e0 2023-12-20T05:44:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4bbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4c40b2ae0 0x7fc4c40cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4bbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4c40b2ae0 0x7fc4c40cf160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59000/0 (socket says 172.21.15.32:59000) 2023-12-20T05:44:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4bbfff640 1 -- 172.21.15.32:0/2532356142 learned_addr learned my addr 172.21.15.32:0/2532356142 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4c9318640 1 --2- 172.21.15.32:0/2532356142 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c41608e0 0x7fc4c4160d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4bbfff640 1 -- 172.21.15.32:0/2532356142 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c41608e0 msgr2=0x7fc4c4160d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4bbfff640 1 --2- 172.21.15.32:0/2532356142 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c41608e0 0x7fc4c4160d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4bbfff640 1 -- 172.21.15.32:0/2532356142 >> v1:172.21.15.39:6789/0 conn(0x7fc4c415cd70 legacy=0x7fc4c40b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.579+0000 7fc4bbfff640 1 -- 172.21.15.32:0/2532356142 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4c40ce960 con 0x7fc4c40b2ae0 2023-12-20T05:44:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4bbfff640 1 --2- 172.21.15.32:0/2532356142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c40b2ae0 0x7fc4c40cf160 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009870 tx=0x7fc4b401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b841622e3826f722 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4bb7fe640 1 -- 172.21.15.32:0/2532356142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40aa430 con 0x7fc4c40b2ae0 2023-12-20T05:44:24.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4bb7fe640 1 -- 172.21.15.32:0/2532356142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4b40aa590 con 0x7fc4c40b2ae0 2023-12-20T05:44:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4bb7fe640 1 -- 172.21.15.32:0/2532356142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4b40aa920 con 0x7fc4c40b2ae0 2023-12-20T05:44:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2532356142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c40b2ae0 msgr2=0x7fc4c40cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4cad7b640 1 --2- 172.21.15.32:0/2532356142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c40b2ae0 0x7fc4c40cf160 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009870 tx=0x7fc4b401c510 comp rx=0 tx=0).stop 2023-12-20T05:44:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2532356142 shutdown_connections 2023-12-20T05:44:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4cad7b640 1 --2- 172.21.15.32:0/2532356142 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c41608e0 0x7fc4c4160d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4cad7b640 1 --2- 172.21.15.32:0/2532356142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c40b2ae0 0x7fc4c40cf160 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2532356142 >> 172.21.15.32:0/2532356142 conn(0x7fc4c40cd3d0 msgr2=0x7fc4c40cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:24.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2532356142 shutdown_connections 2023-12-20T05:44:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.583+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2532356142 wait complete. 2023-12-20T05:44:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4cad7b640 1 Processor -- start 2023-12-20T05:44:24.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4cad7b640 1 -- start start 2023-12-20T05:44:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4cad7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c40b2ae0 0x7fc4c415a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4cad7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c415cd70 0x7fc4c41531e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4cad7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c41608e0 0x7fc4c4153720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4cad7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c4153f50 con 0x7fc4c41608e0 2023-12-20T05:44:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4cad7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c4154230 con 0x7fc4c40b2ae0 2023-12-20T05:44:24.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4cad7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4c4154510 con 0x7fc4c415cd70 2023-12-20T05:44:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4c9318640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c41608e0 0x7fc4c4153720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4c9318640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c41608e0 0x7fc4c4153720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59004/0 (socket says 172.21.15.32:59004) 2023-12-20T05:44:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4c9318640 1 -- 172.21.15.32:0/2790105373 learned_addr learned my addr 172.21.15.32:0/2790105373 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4bbfff640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c415cd70 0x7fc4c41531e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4c8b17640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c40b2ae0 0x7fc4c415a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4c9318640 1 -- 172.21.15.32:0/2790105373 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c415cd70 msgr2=0x7fc4c41531e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4c9318640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c415cd70 0x7fc4c41531e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4c9318640 1 -- 172.21.15.32:0/2790105373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c40b2ae0 msgr2=0x7fc4c415a0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4c9318640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c40b2ae0 0x7fc4c415a0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4c9318640 1 -- 172.21.15.32:0/2790105373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4b40094d0 con 0x7fc4c41608e0 2023-12-20T05:44:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4c9318640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c41608e0 0x7fc4c4153720 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00efc0 tx=0x7fc4bc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4b97fa640 1 -- 172.21.15.32:0/2790105373 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc009ba0 con 0x7fc4c41608e0 2023-12-20T05:44:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4b97fa640 1 -- 172.21.15.32:0/2790105373 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4bc00f040 con 0x7fc4c41608e0 2023-12-20T05:44:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4b97fa640 1 -- 172.21.15.32:0/2790105373 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4bc0146c0 con 0x7fc4c41608e0 2023-12-20T05:44:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2790105373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4c41547f0 con 0x7fc4c41608e0 2023-12-20T05:44:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.587+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2790105373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4c415a620 con 0x7fc4c41608e0 2023-12-20T05:44:24.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.591+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2790105373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4c404e000 con 0x7fc4c41608e0 2023-12-20T05:44:24.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.591+0000 7fc4b97fa640 1 -- 172.21.15.32:0/2790105373 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc4bc014820 con 0x7fc4c41608e0 2023-12-20T05:44:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.591+0000 7fc4b97fa640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4b0065bb0 0x7fc4b0068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.591+0000 7fc4b97fa640 1 -- 172.21.15.32:0/2790105373 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fc4bc083d10 con 0x7fc4c41608e0 2023-12-20T05:44:24.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.591+0000 7fc4c8b17640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4b0065bb0 0x7fc4b0068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.591+0000 7fc4c8b17640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4b0065bb0 0x7fc4b0068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0007fd0 tx=0x7fc4c0009040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:24.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.595+0000 7fc4b97fa640 1 -- 172.21.15.32:0/2790105373 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc4bc051540 con 0x7fc4c41608e0 2023-12-20T05:44:24.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.759+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2790105373 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc4c40b0090 con 0x7fc4c41608e0 2023-12-20T05:44:24.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.759+0000 7fc4b97fa640 1 -- 172.21.15.32:0/2790105373 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v173) v1 ==== 74+0+16107 (secure 0 0 0) 0x7fc4bc054b60 con 0x7fc4c41608e0 2023-12-20T05:44:24.758 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:24.758 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":173,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:44:22.012109+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:44:18.671073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-20T05:44:20.010875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T05:44:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2790105373 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4b0065bb0 msgr2=0x7fc4b0068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4b0065bb0 0x7fc4b0068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc4c0007fd0 tx=0x7fc4c0009040 comp rx=0 tx=0).stop 2023-12-20T05:44:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2790105373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c41608e0 msgr2=0x7fc4c4153720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c41608e0 0x7fc4c4153720 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc00efc0 tx=0x7fc4bc00c490 comp rx=0 tx=0).stop 2023-12-20T05:44:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2790105373 shutdown_connections 2023-12-20T05:44:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fc4b0065bb0 0x7fc4b0068070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4c41608e0 0x7fc4c4153720 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4c415cd70 0x7fc4c41531e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 --2- 172.21.15.32:0/2790105373 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4c40b2ae0 0x7fc4c415a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2790105373 >> 172.21.15.32:0/2790105373 conn(0x7fc4c40cd3d0 msgr2=0x7fc4c4155d10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:24.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2790105373 shutdown_connections 2023-12-20T05:44:24.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.763+0000 7fc4cad7b640 1 -- 172.21.15.32:0/2790105373 wait complete. 2023-12-20T05:44:24.780 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T05:44:24.780 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T05:44:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.923+0000 7f68810fc640 1 Processor -- start 2023-12-20T05:44:24.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.923+0000 7f68810fc640 1 -- start start 2023-12-20T05:44:24.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f68810fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f687c06ce60 0x7f687c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f68810fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f687c15f110 0x7f687c15c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f68810fc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f687c15bbd0 con 0x7f687c0af5e0 2023-12-20T05:44:24.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f68810fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f687c15beb0 con 0x7f687c15f110 2023-12-20T05:44:24.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f68810fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c15e910 con 0x7f687c06ce60 2023-12-20T05:44:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f687b59e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f687c15f110 0x7f687c15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f6873fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f687c06ce60 0x7f687c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f687b59e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f687c15f110 0x7f687c15c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:59898/0 (socket says 172.21.15.32:59898) 2023-12-20T05:44:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f687b59e640 1 -- 172.21.15.32:0/2934923612 learned_addr learned my addr 172.21.15.32:0/2934923612 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f687ad9d640 1 --1- 172.21.15.32:0/2934923612 >> v1:172.21.15.39:6789/0 conn(0x7f687c0af5e0 0x7f687c0af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:50748/0 (socket says 172.21.15.32:50748) 2023-12-20T05:44:24.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f687b59e640 1 -- 172.21.15.32:0/2934923612 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f687c06ce60 msgr2=0x7f687c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:24.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f687b59e640 1 --2- 172.21.15.32:0/2934923612 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f687c06ce60 0x7f687c15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f687b59e640 1 -- 172.21.15.32:0/2934923612 >> v1:172.21.15.39:6789/0 conn(0x7f687c0af5e0 legacy=0x7f687c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.927+0000 7f687b59e640 1 -- 172.21.15.32:0/2934923612 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f687c15ebf0 con 0x7f687c15f110 2023-12-20T05:44:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f687b59e640 1 --2- 172.21.15.32:0/2934923612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f687c15f110 0x7f687c15c440 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6874009e50 tx=0x7f687401c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=71fb93e12f669f33 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f687a59c640 1 -- 172.21.15.32:0/2934923612 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68740a57e0 con 0x7f687c15f110 2023-12-20T05:44:24.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f687a59c640 1 -- 172.21.15.32:0/2934923612 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f68740a5940 con 0x7f687c15f110 2023-12-20T05:44:24.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f687a59c640 1 -- 172.21.15.32:0/2934923612 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f68740a5cd0 con 0x7f687c15f110 2023-12-20T05:44:24.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f68810fc640 1 -- 172.21.15.32:0/2934923612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f687c15f110 msgr2=0x7f687c15c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f68810fc640 1 --2- 172.21.15.32:0/2934923612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f687c15f110 0x7f687c15c440 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f6874009e50 tx=0x7f687401c980 comp rx=0 tx=0).stop 2023-12-20T05:44:24.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f68810fc640 1 -- 172.21.15.32:0/2934923612 shutdown_connections 2023-12-20T05:44:24.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f68810fc640 1 --2- 172.21.15.32:0/2934923612 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f687c15f110 0x7f687c15c440 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f68810fc640 1 --2- 172.21.15.32:0/2934923612 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f687c06ce60 0x7f687c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f68810fc640 1 -- 172.21.15.32:0/2934923612 >> 172.21.15.32:0/2934923612 conn(0x7f687c06a320 msgr2=0x7f687c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:24.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f68810fc640 1 -- 172.21.15.32:0/2934923612 shutdown_connections 2023-12-20T05:44:24.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f68810fc640 1 -- 172.21.15.32:0/2934923612 wait complete. 2023-12-20T05:44:24.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f68810fc640 1 Processor -- start 2023-12-20T05:44:24.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.931+0000 7f68810fc640 1 -- start start 2023-12-20T05:44:24.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68810fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f687c06ce60 0x7f687c1f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68810fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f687c0af5e0 0x7f687c14bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68810fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f687c15f110 0x7f687c14c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68810fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c159aa0 con 0x7f687c0af5e0 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68810fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f687c14d0e0 con 0x7f687c15f110 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68810fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f687c158ed0 con 0x7f687c06ce60 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f6873fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f687c0af5e0 0x7f687c14bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f6873fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f687c0af5e0 0x7f687c14bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59012/0 (socket says 172.21.15.32:59012) 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f6873fff640 1 -- 172.21.15.32:0/2980231376 learned_addr learned my addr 172.21.15.32:0/2980231376 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f687ad9d640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f687c06ce60 0x7f687c1f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f687b59e640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f687c15f110 0x7f687c14c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f6873fff640 1 -- 172.21.15.32:0/2980231376 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f687c06ce60 msgr2=0x7f687c1f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f6873fff640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f687c06ce60 0x7f687c1f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f6873fff640 1 -- 172.21.15.32:0/2980231376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f687c15f110 msgr2=0x7f687c14c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f6873fff640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f687c15f110 0x7f687c14c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f6873fff640 1 -- 172.21.15.32:0/2980231376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6874009b30 con 0x7f687c0af5e0 2023-12-20T05:44:24.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f687b59e640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f687c15f110 0x7f687c14c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:44:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f687ad9d640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f687c06ce60 0x7f687c1f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:44:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f6873fff640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f687c0af5e0 0x7f687c14bc40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f686400e840 tx=0x7f686400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68737fe640 1 -- 172.21.15.32:0/2980231376 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6864009510 con 0x7f687c0af5e0 2023-12-20T05:44:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68737fe640 1 -- 172.21.15.32:0/2980231376 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f686400ee90 con 0x7f687c0af5e0 2023-12-20T05:44:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68737fe640 1 -- 172.21.15.32:0/2980231376 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f686400cb60 con 0x7f687c0af5e0 2023-12-20T05:44:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68810fc640 1 -- 172.21.15.32:0/2980231376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f687c159170 con 0x7f687c0af5e0 2023-12-20T05:44:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68810fc640 1 -- 172.21.15.32:0/2980231376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f687c159730 con 0x7f687c0af5e0 2023-12-20T05:44:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.935+0000 7f68810fc640 1 -- 172.21.15.32:0/2980231376 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f687c04e000 con 0x7f687c0af5e0 2023-12-20T05:44:24.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.939+0000 7f68737fe640 1 -- 172.21.15.32:0/2980231376 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f686400ccc0 con 0x7f687c0af5e0 2023-12-20T05:44:24.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.939+0000 7f68737fe640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f685c065bb0 0x7f685c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:24.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.939+0000 7f68737fe640 1 -- 172.21.15.32:0/2980231376 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f68640834a0 con 0x7f687c0af5e0 2023-12-20T05:44:24.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.939+0000 7f687ad9d640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f685c065bb0 0x7f685c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:24.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.939+0000 7f687ad9d640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f685c065bb0 0x7f685c068070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f686c000fc0 tx=0x7f686c009040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:24.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:24.943+0000 7f68737fe640 1 -- 172.21.15.32:0/2980231376 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6864050c00 con 0x7f687c0af5e0 2023-12-20T05:44:25.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.099+0000 7f68810fc640 1 -- 172.21.15.32:0/2980231376 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f687c158af0 con 0x7f685c065bb0 2023-12-20T05:44:25.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.107+0000 7f68737fe640 1 -- 172.21.15.32:0/2980231376 <== mgr.9201 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289170 (secure 0 0 0) 0x7f686c005870 con 0x7f685c065bb0 2023-12-20T05:44:25.109 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:25.111 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T05:44:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 -- 172.21.15.32:0/2980231376 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f685c065bb0 msgr2=0x7f685c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f685c065bb0 0x7f685c068070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f686c000fc0 tx=0x7f686c009040 comp rx=0 tx=0).stop 2023-12-20T05:44:25.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 -- 172.21.15.32:0/2980231376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f687c0af5e0 msgr2=0x7f687c14bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f687c0af5e0 0x7f687c14bc40 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f686400e840 tx=0x7f686400ec40 comp rx=0 tx=0).stop 2023-12-20T05:44:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 -- 172.21.15.32:0/2980231376 shutdown_connections 2023-12-20T05:44:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f685c065bb0 0x7f685c068070 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f687c15f110 0x7f687c14c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f687c0af5e0 0x7f687c14bc40 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 --2- 172.21.15.32:0/2980231376 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f687c06ce60 0x7f687c1f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 -- 172.21.15.32:0/2980231376 >> 172.21.15.32:0/2980231376 conn(0x7f687c06a320 msgr2=0x7f687c164760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 -- 172.21.15.32:0/2980231376 shutdown_connections 2023-12-20T05:44:25.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.111+0000 7f68810fc640 1 -- 172.21.15.32:0/2980231376 wait complete. 2023-12-20T05:44:25.131 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-20T05:44:24.325450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1405,"num_read_kb":2819,"num_write":465,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3184,"ondisk_log_size":3184,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":378772,"kb_used_data":18004,"kb_used_omap":0,"kb_used_meta":360704,"kb_avail":754595948,"statfs":{"total":773094113280,"available":772706250752,"internally_reserved":0,"allocated":18436096,"data_stored":9431860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":369360896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.729439"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236116+0000","last_change":"2023-12-20T05:44:21.041236+0000","last_active":"2023-12-20T05:44:22.236116+0000","last_peered":"2023-12-20T05:44:22.236116+0000","last_clean":"2023-12-20T05:44:22.236116+0000","last_became_active":"2023-12-20T05:44:21.040830+0000","last_became_peered":"2023-12-20T05:44:21.040830+0000","last_unstale":"2023-12-20T05:44:22.236116+0000","last_undegraded":"2023-12-20T05:44:22.236116+0000","last_fullsized":"2023-12-20T05:44:22.236116+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:43:38.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.037932+0000","last_change":"2023-12-20T05:44:21.032312+0000","last_active":"2023-12-20T05:44:21.037932+0000","last_peered":"2023-12-20T05:44:21.037932+0000","last_clean":"2023-12-20T05:44:21.037932+0000","last_became_active":"2023-12-20T05:44:21.032057+0000","last_became_peered":"2023-12-20T05:44:21.032057+0000","last_unstale":"2023-12-20T05:44:21.037932+0000","last_undegraded":"2023-12-20T05:44:21.037932+0000","last_fullsized":"2023-12-20T05:44:21.037932+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:29:24.149040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045682+0000","last_change":"2023-12-20T05:44:21.040706+0000","last_active":"2023-12-20T05:44:21.045682+0000","last_peered":"2023-12-20T05:44:21.045682+0000","last_clean":"2023-12-20T05:44:21.045682+0000","last_became_active":"2023-12-20T05:44:21.039460+0000","last_became_peered":"2023-12-20T05:44:21.039460+0000","last_unstale":"2023-12-20T05:44:21.045682+0000","last_undegraded":"2023-12-20T05:44:21.045682+0000","last_fullsized":"2023-12-20T05:44:21.045682+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:54:35.407281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031462+0000","last_change":"2023-12-20T05:44:21.043665+0000","last_active":"2023-12-20T05:44:22.031462+0000","last_peered":"2023-12-20T05:44:22.031462+0000","last_clean":"2023-12-20T05:44:22.031462+0000","last_became_active":"2023-12-20T05:44:21.043390+0000","last_became_peered":"2023-12-20T05:44:21.043390+0000","last_unstale":"2023-12-20T05:44:22.031462+0000","last_undegraded":"2023-12-20T05:44:22.031462+0000","last_fullsized":"2023-12-20T05:44:22.031462+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:30:14.521101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031621+0000","last_change":"2023-12-20T05:44:21.044029+0000","last_active":"2023-12-20T05:44:22.031621+0000","last_peered":"2023-12-20T05:44:22.031621+0000","last_clean":"2023-12-20T05:44:22.031621+0000","last_became_active":"2023-12-20T05:44:21.042159+0000","last_became_peered":"2023-12-20T05:44:21.042159+0000","last_unstale":"2023-12-20T05:44:22.031621+0000","last_undegraded":"2023-12-20T05:44:22.031621+0000","last_fullsized":"2023-12-20T05:44:22.031621+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:50:30.482977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235885+0000","last_change":"2023-12-20T05:44:21.043357+0000","last_active":"2023-12-20T05:44:22.235885+0000","last_peered":"2023-12-20T05:44:22.235885+0000","last_clean":"2023-12-20T05:44:22.235885+0000","last_became_active":"2023-12-20T05:44:21.043113+0000","last_became_peered":"2023-12-20T05:44:21.043113+0000","last_unstale":"2023-12-20T05:44:22.235885+0000","last_undegraded":"2023-12-20T05:44:22.235885+0000","last_fullsized":"2023-12-20T05:44:22.235885+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:24:14.449581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025244+0000","last_change":"2023-12-20T05:44:21.031470+0000","last_active":"2023-12-20T05:44:22.025244+0000","last_peered":"2023-12-20T05:44:22.025244+0000","last_clean":"2023-12-20T05:44:22.025244+0000","last_became_active":"2023-12-20T05:44:21.031205+0000","last_became_peered":"2023-12-20T05:44:21.031205+0000","last_unstale":"2023-12-20T05:44:22.025244+0000","last_undegraded":"2023-12-20T05:44:22.025244+0000","last_fullsized":"2023-12-20T05:44:22.025244+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:25:28.845267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235751+0000","last_change":"2023-12-20T05:44:21.040118+0000","last_active":"2023-12-20T05:44:22.235751+0000","last_peered":"2023-12-20T05:44:22.235751+0000","last_clean":"2023-12-20T05:44:22.235751+0000","last_became_active":"2023-12-20T05:44:21.039808+0000","last_became_peered":"2023-12-20T05:44:21.039808+0000","last_unstale":"2023-12-20T05:44:22.235751+0000","last_undegraded":"2023-12-20T05:44:22.235751+0000","last_fullsized":"2023-12-20T05:44:22.235751+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:01:15.439613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.039716+0000","last_change":"2023-12-20T05:44:21.036405+0000","last_active":"2023-12-20T05:44:21.039716+0000","last_peered":"2023-12-20T05:44:21.039716+0000","last_clean":"2023-12-20T05:44:21.039716+0000","last_became_active":"2023-12-20T05:44:21.036123+0000","last_became_peered":"2023-12-20T05:44:21.036123+0000","last_unstale":"2023-12-20T05:44:21.039716+0000","last_undegraded":"2023-12-20T05:44:21.039716+0000","last_fullsized":"2023-12-20T05:44:21.039716+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:04.948693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.049027+0000","last_change":"2023-12-20T05:44:21.046085+0000","last_active":"2023-12-20T05:44:21.049027+0000","last_peered":"2023-12-20T05:44:21.049027+0000","last_clean":"2023-12-20T05:44:21.049027+0000","last_became_active":"2023-12-20T05:44:21.045787+0000","last_became_peered":"2023-12-20T05:44:21.045787+0000","last_unstale":"2023-12-20T05:44:21.049027+0000","last_undegraded":"2023-12-20T05:44:21.049027+0000","last_fullsized":"2023-12-20T05:44:21.049027+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:30:03.168590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235920+0000","last_change":"2023-12-20T05:44:21.041911+0000","last_active":"2023-12-20T05:44:22.235920+0000","last_peered":"2023-12-20T05:44:22.235920+0000","last_clean":"2023-12-20T05:44:22.235920+0000","last_became_active":"2023-12-20T05:44:21.041655+0000","last_became_peered":"2023-12-20T05:44:21.041655+0000","last_unstale":"2023-12-20T05:44:22.235920+0000","last_undegraded":"2023-12-20T05:44:22.235920+0000","last_fullsized":"2023-12-20T05:44:22.235920+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:43:42.093760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.044630+0000","last_change":"2023-12-20T05:44:21.040214+0000","last_active":"2023-12-20T05:44:21.044630+0000","last_peered":"2023-12-20T05:44:21.044630+0000","last_clean":"2023-12-20T05:44:21.044630+0000","last_became_active":"2023-12-20T05:44:21.035744+0000","last_became_peered":"2023-12-20T05:44:21.035744+0000","last_unstale":"2023-12-20T05:44:21.044630+0000","last_undegraded":"2023-12-20T05:44:21.044630+0000","last_fullsized":"2023-12-20T05:44:21.044630+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:24:45.142549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031731+0000","last_change":"2023-12-20T05:44:21.039998+0000","last_active":"2023-12-20T05:44:22.031731+0000","last_peered":"2023-12-20T05:44:22.031731+0000","last_clean":"2023-12-20T05:44:22.031731+0000","last_became_active":"2023-12-20T05:44:21.034989+0000","last_became_peered":"2023-12-20T05:44:21.034989+0000","last_unstale":"2023-12-20T05:44:22.031731+0000","last_undegraded":"2023-12-20T05:44:22.031731+0000","last_fullsized":"2023-12-20T05:44:22.031731+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:54:35.407281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030801+0000","last_change":"2023-12-20T05:44:21.035572+0000","last_active":"2023-12-20T05:44:22.030801+0000","last_peered":"2023-12-20T05:44:22.030801+0000","last_clean":"2023-12-20T05:44:22.030801+0000","last_became_active":"2023-12-20T05:44:21.035286+0000","last_became_peered":"2023-12-20T05:44:21.035286+0000","last_unstale":"2023-12-20T05:44:22.030801+0000","last_undegraded":"2023-12-20T05:44:22.030801+0000","last_fullsized":"2023-12-20T05:44:22.030801+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:00:53.185594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.044019+0000","last_change":"2023-12-20T05:44:21.038954+0000","last_active":"2023-12-20T05:44:21.044019+0000","last_peered":"2023-12-20T05:44:21.044019+0000","last_clean":"2023-12-20T05:44:21.044019+0000","last_became_active":"2023-12-20T05:44:21.038639+0000","last_became_peered":"2023-12-20T05:44:21.038639+0000","last_unstale":"2023-12-20T05:44:21.044019+0000","last_undegraded":"2023-12-20T05:44:21.044019+0000","last_fullsized":"2023-12-20T05:44:21.044019+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:23:19.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.043308+0000","last_change":"2023-12-20T05:44:21.037271+0000","last_active":"2023-12-20T05:44:21.043308+0000","last_peered":"2023-12-20T05:44:21.043308+0000","last_clean":"2023-12-20T05:44:21.043308+0000","last_became_active":"2023-12-20T05:44:21.036877+0000","last_became_peered":"2023-12-20T05:44:21.036877+0000","last_unstale":"2023-12-20T05:44:21.043308+0000","last_undegraded":"2023-12-20T05:44:21.043308+0000","last_fullsized":"2023-12-20T05:44:21.043308+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:58:31.373035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.046362+0000","last_change":"2023-12-20T05:44:21.043437+0000","last_active":"2023-12-20T05:44:21.046362+0000","last_peered":"2023-12-20T05:44:21.046362+0000","last_clean":"2023-12-20T05:44:21.046362+0000","last_became_active":"2023-12-20T05:44:21.043108+0000","last_became_peered":"2023-12-20T05:44:21.043108+0000","last_unstale":"2023-12-20T05:44:21.046362+0000","last_undegraded":"2023-12-20T05:44:21.046362+0000","last_fullsized":"2023-12-20T05:44:21.046362+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:16.798109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045465+0000","last_change":"2023-12-20T05:44:21.041694+0000","last_active":"2023-12-20T05:44:21.045465+0000","last_peered":"2023-12-20T05:44:21.045465+0000","last_clean":"2023-12-20T05:44:21.045465+0000","last_became_active":"2023-12-20T05:44:21.039386+0000","last_became_peered":"2023-12-20T05:44:21.039386+0000","last_unstale":"2023-12-20T05:44:21.045465+0000","last_undegraded":"2023-12-20T05:44:21.045465+0000","last_fullsized":"2023-12-20T05:44:21.045465+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:21.786814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236045+0000","last_change":"2023-12-20T05:44:21.044105+0000","last_active":"2023-12-20T05:44:22.236045+0000","last_peered":"2023-12-20T05:44:22.236045+0000","last_clean":"2023-12-20T05:44:22.236045+0000","last_became_active":"2023-12-20T05:44:21.043842+0000","last_became_peered":"2023-12-20T05:44:21.043842+0000","last_unstale":"2023-12-20T05:44:22.236045+0000","last_undegraded":"2023-12-20T05:44:22.236045+0000","last_fullsized":"2023-12-20T05:44:22.236045+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:01:26.874334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.041237+0000","last_change":"2023-12-20T05:44:21.034426+0000","last_active":"2023-12-20T05:44:21.041237+0000","last_peered":"2023-12-20T05:44:21.041237+0000","last_clean":"2023-12-20T05:44:21.041237+0000","last_became_active":"2023-12-20T05:44:21.034015+0000","last_became_peered":"2023-12-20T05:44:21.034015+0000","last_unstale":"2023-12-20T05:44:21.041237+0000","last_undegraded":"2023-12-20T05:44:21.041237+0000","last_fullsized":"2023-12-20T05:44:21.041237+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:38:02.506656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.040800+0000","last_change":"2023-12-20T05:44:21.037116+0000","last_active":"2023-12-20T05:44:21.040800+0000","last_peered":"2023-12-20T05:44:21.040800+0000","last_clean":"2023-12-20T05:44:21.040800+0000","last_became_active":"2023-12-20T05:44:21.036509+0000","last_became_peered":"2023-12-20T05:44:21.036509+0000","last_unstale":"2023-12-20T05:44:21.040800+0000","last_undegraded":"2023-12-20T05:44:21.040800+0000","last_fullsized":"2023-12-20T05:44:21.040800+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:47.291062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030915+0000","last_change":"2023-12-20T05:44:21.031175+0000","last_active":"2023-12-20T05:44:22.030915+0000","last_peered":"2023-12-20T05:44:22.030915+0000","last_clean":"2023-12-20T05:44:22.030915+0000","last_became_active":"2023-12-20T05:44:21.030872+0000","last_became_peered":"2023-12-20T05:44:21.030872+0000","last_unstale":"2023-12-20T05:44:22.030915+0000","last_undegraded":"2023-12-20T05:44:22.030915+0000","last_fullsized":"2023-12-20T05:44:22.030915+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:29:57.434741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030880+0000","last_change":"2023-12-20T05:44:21.037060+0000","last_active":"2023-12-20T05:44:22.030880+0000","last_peered":"2023-12-20T05:44:22.030880+0000","last_clean":"2023-12-20T05:44:22.030880+0000","last_became_active":"2023-12-20T05:44:21.036796+0000","last_became_peered":"2023-12-20T05:44:21.036796+0000","last_unstale":"2023-12-20T05:44:22.030880+0000","last_undegraded":"2023-12-20T05:44:22.030880+0000","last_fullsized":"2023-12-20T05:44:22.030880+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:13:08.242870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.049029+0000","last_change":"2023-12-20T05:44:21.045480+0000","last_active":"2023-12-20T05:44:21.049029+0000","last_peered":"2023-12-20T05:44:21.049029+0000","last_clean":"2023-12-20T05:44:21.049029+0000","last_became_active":"2023-12-20T05:44:21.045220+0000","last_became_peered":"2023-12-20T05:44:21.045220+0000","last_unstale":"2023-12-20T05:44:21.049029+0000","last_undegraded":"2023-12-20T05:44:21.049029+0000","last_fullsized":"2023-12-20T05:44:21.049029+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:51:35.352950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.046989+0000","last_change":"2023-12-20T05:44:21.044019+0000","last_active":"2023-12-20T05:44:21.046989+0000","last_peered":"2023-12-20T05:44:21.046989+0000","last_clean":"2023-12-20T05:44:21.046989+0000","last_became_active":"2023-12-20T05:44:21.043725+0000","last_became_peered":"2023-12-20T05:44:21.043725+0000","last_unstale":"2023-12-20T05:44:21.046989+0000","last_undegraded":"2023-12-20T05:44:21.046989+0000","last_fullsized":"2023-12-20T05:44:21.046989+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:29:24.999626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236017+0000","last_change":"2023-12-20T05:44:21.043908+0000","last_active":"2023-12-20T05:44:22.236017+0000","last_peered":"2023-12-20T05:44:22.236017+0000","last_clean":"2023-12-20T05:44:22.236017+0000","last_became_active":"2023-12-20T05:44:21.043662+0000","last_became_peered":"2023-12-20T05:44:21.043662+0000","last_unstale":"2023-12-20T05:44:22.236017+0000","last_undegraded":"2023-12-20T05:44:22.236017+0000","last_fullsized":"2023-12-20T05:44:22.236017+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:34:14.606009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.046643+0000","last_change":"2023-12-20T05:44:21.043400+0000","last_active":"2023-12-20T05:44:21.046643+0000","last_peered":"2023-12-20T05:44:21.046643+0000","last_clean":"2023-12-20T05:44:21.046643+0000","last_became_active":"2023-12-20T05:44:21.043144+0000","last_became_peered":"2023-12-20T05:44:21.043144+0000","last_unstale":"2023-12-20T05:44:21.046643+0000","last_undegraded":"2023-12-20T05:44:21.046643+0000","last_fullsized":"2023-12-20T05:44:21.046643+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:25:52.922696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025059+0000","last_change":"2023-12-20T05:44:21.047039+0000","last_active":"2023-12-20T05:44:22.025059+0000","last_peered":"2023-12-20T05:44:22.025059+0000","last_clean":"2023-12-20T05:44:22.025059+0000","last_became_active":"2023-12-20T05:44:21.046633+0000","last_became_peered":"2023-12-20T05:44:21.046633+0000","last_unstale":"2023-12-20T05:44:22.025059+0000","last_undegraded":"2023-12-20T05:44:22.025059+0000","last_fullsized":"2023-12-20T05:44:22.025059+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:32.056451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235794+0000","last_change":"2023-12-20T05:44:21.047446+0000","last_active":"2023-12-20T05:44:22.235794+0000","last_peered":"2023-12-20T05:44:22.235794+0000","last_clean":"2023-12-20T05:44:22.235794+0000","last_became_active":"2023-12-20T05:44:21.046827+0000","last_became_peered":"2023-12-20T05:44:21.046827+0000","last_unstale":"2023-12-20T05:44:22.235794+0000","last_undegraded":"2023-12-20T05:44:22.235794+0000","last_fullsized":"2023-12-20T05:44:22.235794+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:45.593289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031013+0000","last_change":"2023-12-20T05:44:21.034892+0000","last_active":"2023-12-20T05:44:22.031013+0000","last_peered":"2023-12-20T05:44:22.031013+0000","last_clean":"2023-12-20T05:44:22.031013+0000","last_became_active":"2023-12-20T05:44:21.033742+0000","last_became_peered":"2023-12-20T05:44:21.033742+0000","last_unstale":"2023-12-20T05:44:22.031013+0000","last_undegraded":"2023-12-20T05:44:22.031013+0000","last_fullsized":"2023-12-20T05:44:22.031013+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:05:04.359301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.040253+0000","last_change":"2023-12-20T05:44:21.037155+0000","last_active":"2023-12-20T05:44:21.040253+0000","last_peered":"2023-12-20T05:44:21.040253+0000","last_clean":"2023-12-20T05:44:21.040253+0000","last_became_active":"2023-12-20T05:44:21.036900+0000","last_became_peered":"2023-12-20T05:44:21.036900+0000","last_unstale":"2023-12-20T05:44:21.040253+0000","last_undegraded":"2023-12-20T05:44:21.040253+0000","last_fullsized":"2023-12-20T05:44:21.040253+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:00:55.118503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.043284+0000","last_change":"2023-12-20T05:44:21.037120+0000","last_active":"2023-12-20T05:44:21.043284+0000","last_peered":"2023-12-20T05:44:21.043284+0000","last_clean":"2023-12-20T05:44:21.043284+0000","last_became_active":"2023-12-20T05:44:21.036537+0000","last_became_peered":"2023-12-20T05:44:21.036537+0000","last_unstale":"2023-12-20T05:44:21.043284+0000","last_undegraded":"2023-12-20T05:44:21.043284+0000","last_fullsized":"2023-12-20T05:44:21.043284+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:53:22.990765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.042373+0000","last_change":"2023-12-20T05:44:21.037768+0000","last_active":"2023-12-20T05:44:21.042373+0000","last_peered":"2023-12-20T05:44:21.042373+0000","last_clean":"2023-12-20T05:44:21.042373+0000","last_became_active":"2023-12-20T05:44:21.037476+0000","last_became_peered":"2023-12-20T05:44:21.037476+0000","last_unstale":"2023-12-20T05:44:21.042373+0000","last_undegraded":"2023-12-20T05:44:21.042373+0000","last_fullsized":"2023-12-20T05:44:21.042373+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:13:06.355404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"172'4","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045192+0000","last_change":"2023-12-20T05:44:20.630009+0000","last_active":"2023-12-20T05:44:21.045192+0000","last_peered":"2023-12-20T05:44:21.045192+0000","last_clean":"2023-12-20T05:44:21.045192+0000","last_became_active":"2023-12-20T05:44:20.629696+0000","last_became_peered":"2023-12-20T05:44:20.629696+0000","last_unstale":"2023-12-20T05:44:21.045192+0000","last_undegraded":"2023-12-20T05:44:21.045192+0000","last_fullsized":"2023-12-20T05:44:21.045192+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:54:34.396231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.037816+0000","last_change":"2023-12-20T05:44:21.029096+0000","last_active":"2023-12-20T05:44:21.037816+0000","last_peered":"2023-12-20T05:44:21.037816+0000","last_clean":"2023-12-20T05:44:21.037816+0000","last_became_active":"2023-12-20T05:44:21.028832+0000","last_became_peered":"2023-12-20T05:44:21.028832+0000","last_unstale":"2023-12-20T05:44:21.037816+0000","last_undegraded":"2023-12-20T05:44:21.037816+0000","last_fullsized":"2023-12-20T05:44:21.037816+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:35:42.150416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.047152+0000","last_change":"2023-12-20T05:44:20.631309+0000","last_active":"2023-12-20T05:44:21.047152+0000","last_peered":"2023-12-20T05:44:21.047152+0000","last_clean":"2023-12-20T05:44:21.047152+0000","last_became_active":"2023-12-20T05:44:20.630799+0000","last_became_peered":"2023-12-20T05:44:20.630799+0000","last_unstale":"2023-12-20T05:44:21.047152+0000","last_undegraded":"2023-12-20T05:44:21.047152+0000","last_fullsized":"2023-12-20T05:44:21.047152+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:10:07.363510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235827+0000","last_change":"2023-12-20T05:44:21.039734+0000","last_active":"2023-12-20T05:44:22.235827+0000","last_peered":"2023-12-20T05:44:22.235827+0000","last_clean":"2023-12-20T05:44:22.235827+0000","last_became_active":"2023-12-20T05:44:21.039468+0000","last_became_peered":"2023-12-20T05:44:21.039468+0000","last_unstale":"2023-12-20T05:44:22.235827+0000","last_undegraded":"2023-12-20T05:44:22.235827+0000","last_fullsized":"2023-12-20T05:44:22.235827+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:21:51.300866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"172'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031690+0000","last_change":"2023-12-20T05:44:20.622284+0000","last_active":"2023-12-20T05:44:22.031690+0000","last_peered":"2023-12-20T05:44:22.031690+0000","last_clean":"2023-12-20T05:44:22.031690+0000","last_became_active":"2023-12-20T05:44:20.622007+0000","last_became_peered":"2023-12-20T05:44:20.622007+0000","last_unstale":"2023-12-20T05:44:22.031690+0000","last_undegraded":"2023-12-20T05:44:22.031690+0000","last_fullsized":"2023-12-20T05:44:22.031690+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:33:02.537140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.044598+0000","last_change":"2023-12-20T05:44:21.040468+0000","last_active":"2023-12-20T05:44:21.044598+0000","last_peered":"2023-12-20T05:44:21.044598+0000","last_clean":"2023-12-20T05:44:21.044598+0000","last_became_active":"2023-12-20T05:44:21.039122+0000","last_became_peered":"2023-12-20T05:44:21.039122+0000","last_unstale":"2023-12-20T05:44:21.044598+0000","last_undegraded":"2023-12-20T05:44:21.044598+0000","last_fullsized":"2023-12-20T05:44:21.044598+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:18:05.829125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"172'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031571+0000","last_change":"2023-12-20T05:44:20.624440+0000","last_active":"2023-12-20T05:44:22.031571+0000","last_peered":"2023-12-20T05:44:22.031571+0000","last_clean":"2023-12-20T05:44:22.031571+0000","last_became_active":"2023-12-20T05:44:20.624130+0000","last_became_peered":"2023-12-20T05:44:20.624130+0000","last_unstale":"2023-12-20T05:44:22.031571+0000","last_undegraded":"2023-12-20T05:44:22.031571+0000","last_fullsized":"2023-12-20T05:44:22.031571+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:15.787059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.046758+0000","last_change":"2023-12-20T05:44:21.045365+0000","last_active":"2023-12-20T05:44:21.046758+0000","last_peered":"2023-12-20T05:44:21.046758+0000","last_clean":"2023-12-20T05:44:21.046758+0000","last_became_active":"2023-12-20T05:44:21.045118+0000","last_became_peered":"2023-12-20T05:44:21.045118+0000","last_unstale":"2023-12-20T05:44:21.046758+0000","last_undegraded":"2023-12-20T05:44:21.046758+0000","last_fullsized":"2023-12-20T05:44:21.046758+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:56:50.707975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.042642+0000","last_change":"2023-12-20T05:44:20.623265+0000","last_active":"2023-12-20T05:44:21.042642+0000","last_peered":"2023-12-20T05:44:21.042642+0000","last_clean":"2023-12-20T05:44:21.042642+0000","last_became_active":"2023-12-20T05:44:20.623007+0000","last_became_peered":"2023-12-20T05:44:20.623007+0000","last_unstale":"2023-12-20T05:44:21.042642+0000","last_undegraded":"2023-12-20T05:44:21.042642+0000","last_fullsized":"2023-12-20T05:44:21.042642+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:43:03.649161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030955+0000","last_change":"2023-12-20T05:44:21.044438+0000","last_active":"2023-12-20T05:44:22.030955+0000","last_peered":"2023-12-20T05:44:22.030955+0000","last_clean":"2023-12-20T05:44:22.030955+0000","last_became_active":"2023-12-20T05:44:21.044166+0000","last_became_peered":"2023-12-20T05:44:21.044166+0000","last_unstale":"2023-12-20T05:44:22.030955+0000","last_undegraded":"2023-12-20T05:44:22.030955+0000","last_fullsized":"2023-12-20T05:44:22.030955+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:50:47.195501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"172'1","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031641+0000","last_change":"2023-12-20T05:44:20.620159+0000","last_active":"2023-12-20T05:44:22.031641+0000","last_peered":"2023-12-20T05:44:22.031641+0000","last_clean":"2023-12-20T05:44:22.031641+0000","last_became_active":"2023-12-20T05:44:20.619841+0000","last_became_peered":"2023-12-20T05:44:20.619841+0000","last_unstale":"2023-12-20T05:44:22.031641+0000","last_undegraded":"2023-12-20T05:44:22.031641+0000","last_fullsized":"2023-12-20T05:44:22.031641+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:35.619524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.037905+0000","last_change":"2023-12-20T05:44:21.026198+0000","last_active":"2023-12-20T05:44:21.037905+0000","last_peered":"2023-12-20T05:44:21.037905+0000","last_clean":"2023-12-20T05:44:21.037905+0000","last_became_active":"2023-12-20T05:44:21.025916+0000","last_became_peered":"2023-12-20T05:44:21.025916+0000","last_unstale":"2023-12-20T05:44:21.037905+0000","last_undegraded":"2023-12-20T05:44:21.037905+0000","last_fullsized":"2023-12-20T05:44:21.037905+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:22:09.392193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.023031+0000","last_change":"2023-12-20T05:44:20.625433+0000","last_active":"2023-12-20T05:44:21.023031+0000","last_peered":"2023-12-20T05:44:21.023031+0000","last_clean":"2023-12-20T05:44:21.023031+0000","last_became_active":"2023-12-20T05:44:20.625140+0000","last_became_peered":"2023-12-20T05:44:20.625140+0000","last_unstale":"2023-12-20T05:44:21.023031+0000","last_undegraded":"2023-12-20T05:44:21.023031+0000","last_fullsized":"2023-12-20T05:44:21.023031+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:44:34.085866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031479+0000","last_change":"2023-12-20T05:44:21.035024+0000","last_active":"2023-12-20T05:44:22.031479+0000","last_peered":"2023-12-20T05:44:22.031479+0000","last_clean":"2023-12-20T05:44:22.031479+0000","last_became_active":"2023-12-20T05:44:21.033948+0000","last_became_peered":"2023-12-20T05:44:21.033948+0000","last_unstale":"2023-12-20T05:44:22.031479+0000","last_undegraded":"2023-12-20T05:44:22.031479+0000","last_fullsized":"2023-12-20T05:44:22.031479+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:38:22.835946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030684+0000","last_change":"2023-12-20T05:44:20.628128+0000","last_active":"2023-12-20T05:44:22.030684+0000","last_peered":"2023-12-20T05:44:22.030684+0000","last_clean":"2023-12-20T05:44:22.030684+0000","last_became_active":"2023-12-20T05:44:20.627820+0000","last_became_peered":"2023-12-20T05:44:20.627820+0000","last_unstale":"2023-12-20T05:44:22.030684+0000","last_undegraded":"2023-12-20T05:44:22.030684+0000","last_fullsized":"2023-12-20T05:44:22.030684+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:12.314127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.032002+0000","last_change":"2023-12-20T05:44:21.036737+0000","last_active":"2023-12-20T05:44:22.032002+0000","last_peered":"2023-12-20T05:44:22.032002+0000","last_clean":"2023-12-20T05:44:22.032002+0000","last_became_active":"2023-12-20T05:44:21.035560+0000","last_became_peered":"2023-12-20T05:44:21.035560+0000","last_unstale":"2023-12-20T05:44:22.032002+0000","last_undegraded":"2023-12-20T05:44:22.032002+0000","last_fullsized":"2023-12-20T05:44:22.032002+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:21.786814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.014870+0000","last_change":"2023-12-20T05:44:20.624728+0000","last_active":"2023-12-20T05:44:21.014870+0000","last_peered":"2023-12-20T05:44:21.014870+0000","last_clean":"2023-12-20T05:44:21.014870+0000","last_became_active":"2023-12-20T05:44:20.624149+0000","last_became_peered":"2023-12-20T05:44:20.624149+0000","last_unstale":"2023-12-20T05:44:21.014870+0000","last_undegraded":"2023-12-20T05:44:21.014870+0000","last_fullsized":"2023-12-20T05:44:21.014870+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:07.682088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045268+0000","last_change":"2023-12-20T05:44:21.041474+0000","last_active":"2023-12-20T05:44:21.045268+0000","last_peered":"2023-12-20T05:44:21.045268+0000","last_clean":"2023-12-20T05:44:21.045268+0000","last_became_active":"2023-12-20T05:44:21.041208+0000","last_became_peered":"2023-12-20T05:44:21.041208+0000","last_unstale":"2023-12-20T05:44:21.045268+0000","last_undegraded":"2023-12-20T05:44:21.045268+0000","last_fullsized":"2023-12-20T05:44:21.045268+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:33:03.548190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.026556+0000","last_change":"2023-12-20T05:44:20.630001+0000","last_active":"2023-12-20T05:44:21.026556+0000","last_peered":"2023-12-20T05:44:21.026556+0000","last_clean":"2023-12-20T05:44:21.026556+0000","last_became_active":"2023-12-20T05:44:20.629682+0000","last_became_peered":"2023-12-20T05:44:20.629682+0000","last_unstale":"2023-12-20T05:44:21.026556+0000","last_undegraded":"2023-12-20T05:44:21.026556+0000","last_fullsized":"2023-12-20T05:44:21.026556+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:20.775764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.037555+0000","last_change":"2023-12-20T05:44:21.028997+0000","last_active":"2023-12-20T05:44:21.037555+0000","last_peered":"2023-12-20T05:44:21.037555+0000","last_clean":"2023-12-20T05:44:21.037555+0000","last_became_active":"2023-12-20T05:44:21.028644+0000","last_became_peered":"2023-12-20T05:44:21.028644+0000","last_unstale":"2023-12-20T05:44:21.037555+0000","last_undegraded":"2023-12-20T05:44:21.037555+0000","last_fullsized":"2023-12-20T05:44:21.037555+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:30:05.816061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"172'6","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025169+0000","last_change":"2023-12-20T05:44:20.622279+0000","last_active":"2023-12-20T05:44:22.025169+0000","last_peered":"2023-12-20T05:44:22.025169+0000","last_clean":"2023-12-20T05:44:22.025169+0000","last_became_active":"2023-12-20T05:44:20.621957+0000","last_became_peered":"2023-12-20T05:44:20.621957+0000","last_unstale":"2023-12-20T05:44:22.025169+0000","last_undegraded":"2023-12-20T05:44:22.025169+0000","last_fullsized":"2023-12-20T05:44:22.025169+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:15:05.710477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.050931+0000","last_change":"2023-12-20T05:44:21.049368+0000","last_active":"2023-12-20T05:44:21.050931+0000","last_peered":"2023-12-20T05:44:21.050931+0000","last_clean":"2023-12-20T05:44:21.050931+0000","last_became_active":"2023-12-20T05:44:21.048230+0000","last_became_peered":"2023-12-20T05:44:21.048230+0000","last_unstale":"2023-12-20T05:44:21.050931+0000","last_undegraded":"2023-12-20T05:44:21.050931+0000","last_fullsized":"2023-12-20T05:44:21.050931+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:52:24.479671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.047487+0000","last_change":"2023-12-20T05:44:20.619762+0000","last_active":"2023-12-20T05:44:21.047487+0000","last_peered":"2023-12-20T05:44:21.047487+0000","last_clean":"2023-12-20T05:44:21.047487+0000","last_became_active":"2023-12-20T05:44:20.619310+0000","last_became_peered":"2023-12-20T05:44:20.619310+0000","last_unstale":"2023-12-20T05:44:21.047487+0000","last_undegraded":"2023-12-20T05:44:21.047487+0000","last_fullsized":"2023-12-20T05:44:21.047487+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:35.619524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.037874+0000","last_change":"2023-12-20T05:44:21.027971+0000","last_active":"2023-12-20T05:44:21.037874+0000","last_peered":"2023-12-20T05:44:21.037874+0000","last_clean":"2023-12-20T05:44:21.037874+0000","last_became_active":"2023-12-20T05:44:21.027708+0000","last_became_peered":"2023-12-20T05:44:21.027708+0000","last_unstale":"2023-12-20T05:44:21.037874+0000","last_undegraded":"2023-12-20T05:44:21.037874+0000","last_fullsized":"2023-12-20T05:44:21.037874+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:41:44.845747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.035176+0000","last_change":"2023-12-20T05:44:20.625805+0000","last_active":"2023-12-20T05:44:21.035176+0000","last_peered":"2023-12-20T05:44:21.035176+0000","last_clean":"2023-12-20T05:44:21.035176+0000","last_became_active":"2023-12-20T05:44:20.625519+0000","last_became_peered":"2023-12-20T05:44:20.625519+0000","last_unstale":"2023-12-20T05:44:21.035176+0000","last_undegraded":"2023-12-20T05:44:21.035176+0000","last_fullsized":"2023-12-20T05:44:21.035176+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:30:22.493006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031620+0000","last_change":"2023-12-20T05:44:21.040786+0000","last_active":"2023-12-20T05:44:22.031620+0000","last_peered":"2023-12-20T05:44:22.031620+0000","last_clean":"2023-12-20T05:44:22.031620+0000","last_became_active":"2023-12-20T05:44:21.040439+0000","last_became_peered":"2023-12-20T05:44:21.040439+0000","last_unstale":"2023-12-20T05:44:22.031620+0000","last_undegraded":"2023-12-20T05:44:22.031620+0000","last_fullsized":"2023-12-20T05:44:22.031620+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:52:24.479671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235740+0000","last_change":"2023-12-20T05:44:20.624836+0000","last_active":"2023-12-20T05:44:22.235740+0000","last_peered":"2023-12-20T05:44:22.235740+0000","last_clean":"2023-12-20T05:44:22.235740+0000","last_became_active":"2023-12-20T05:44:20.624535+0000","last_became_peered":"2023-12-20T05:44:20.624535+0000","last_unstale":"2023-12-20T05:44:22.235740+0000","last_undegraded":"2023-12-20T05:44:22.235740+0000","last_fullsized":"2023-12-20T05:44:22.235740+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:30:22.493006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.044333+0000","last_change":"2023-12-20T05:44:21.038160+0000","last_active":"2023-12-20T05:44:21.044333+0000","last_peered":"2023-12-20T05:44:21.044333+0000","last_clean":"2023-12-20T05:44:21.044333+0000","last_became_active":"2023-12-20T05:44:21.037862+0000","last_became_peered":"2023-12-20T05:44:21.037862+0000","last_unstale":"2023-12-20T05:44:21.044333+0000","last_undegraded":"2023-12-20T05:44:21.044333+0000","last_fullsized":"2023-12-20T05:44:21.044333+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:38:05.186489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"172'8","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.038672+0000","last_change":"2023-12-20T05:44:20.625606+0000","last_active":"2023-12-20T05:44:21.038672+0000","last_peered":"2023-12-20T05:44:21.038672+0000","last_clean":"2023-12-20T05:44:21.038672+0000","last_became_active":"2023-12-20T05:44:20.625282+0000","last_became_peered":"2023-12-20T05:44:20.625282+0000","last_unstale":"2023-12-20T05:44:21.038672+0000","last_undegraded":"2023-12-20T05:44:21.038672+0000","last_fullsized":"2023-12-20T05:44:21.038672+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:55:57.190271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.042392+0000","last_change":"2023-12-20T05:44:21.037892+0000","last_active":"2023-12-20T05:44:21.042392+0000","last_peered":"2023-12-20T05:44:21.042392+0000","last_clean":"2023-12-20T05:44:21.042392+0000","last_became_active":"2023-12-20T05:44:21.037570+0000","last_became_peered":"2023-12-20T05:44:21.037570+0000","last_unstale":"2023-12-20T05:44:21.042392+0000","last_undegraded":"2023-12-20T05:44:21.042392+0000","last_fullsized":"2023-12-20T05:44:21.042392+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:33.564209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"172'1","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031996+0000","last_change":"2023-12-20T05:44:20.623393+0000","last_active":"2023-12-20T05:44:22.031996+0000","last_peered":"2023-12-20T05:44:22.031996+0000","last_clean":"2023-12-20T05:44:22.031996+0000","last_became_active":"2023-12-20T05:44:20.623037+0000","last_became_peered":"2023-12-20T05:44:20.623037+0000","last_unstale":"2023-12-20T05:44:22.031996+0000","last_undegraded":"2023-12-20T05:44:22.031996+0000","last_fullsized":"2023-12-20T05:44:22.031996+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:04:58.510517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.049417+0000","last_change":"2023-12-20T05:44:21.045716+0000","last_active":"2023-12-20T05:44:21.049417+0000","last_peered":"2023-12-20T05:44:21.049417+0000","last_clean":"2023-12-20T05:44:21.049417+0000","last_became_active":"2023-12-20T05:44:21.043886+0000","last_became_peered":"2023-12-20T05:44:21.043886+0000","last_unstale":"2023-12-20T05:44:21.049417+0000","last_undegraded":"2023-12-20T05:44:21.049417+0000","last_fullsized":"2023-12-20T05:44:21.049417+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:34:53.917983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.049921+0000","last_change":"2023-12-20T05:44:20.629062+0000","last_active":"2023-12-20T05:44:21.049921+0000","last_peered":"2023-12-20T05:44:21.049921+0000","last_clean":"2023-12-20T05:44:21.049921+0000","last_became_active":"2023-12-20T05:44:20.628718+0000","last_became_peered":"2023-12-20T05:44:20.628718+0000","last_unstale":"2023-12-20T05:44:21.049921+0000","last_undegraded":"2023-12-20T05:44:21.049921+0000","last_fullsized":"2023-12-20T05:44:21.049921+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:53:01.154076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.050897+0000","last_change":"2023-12-20T05:44:21.049012+0000","last_active":"2023-12-20T05:44:21.050897+0000","last_peered":"2023-12-20T05:44:21.050897+0000","last_clean":"2023-12-20T05:44:21.050897+0000","last_became_active":"2023-12-20T05:44:21.048292+0000","last_became_peered":"2023-12-20T05:44:21.048292+0000","last_unstale":"2023-12-20T05:44:21.050897+0000","last_undegraded":"2023-12-20T05:44:21.050897+0000","last_fullsized":"2023-12-20T05:44:21.050897+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:38:22.835946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045837+0000","last_change":"2023-12-20T05:44:20.631569+0000","last_active":"2023-12-20T05:44:21.045837+0000","last_peered":"2023-12-20T05:44:21.045837+0000","last_clean":"2023-12-20T05:44:21.045837+0000","last_became_active":"2023-12-20T05:44:20.631233+0000","last_became_peered":"2023-12-20T05:44:20.631233+0000","last_unstale":"2023-12-20T05:44:21.045837+0000","last_undegraded":"2023-12-20T05:44:21.045837+0000","last_fullsized":"2023-12-20T05:44:21.045837+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:42.371279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235961+0000","last_change":"2023-12-20T05:44:21.048122+0000","last_active":"2023-12-20T05:44:22.235961+0000","last_peered":"2023-12-20T05:44:22.235961+0000","last_clean":"2023-12-20T05:44:22.235961+0000","last_became_active":"2023-12-20T05:44:21.047632+0000","last_became_peered":"2023-12-20T05:44:21.047632+0000","last_unstale":"2023-12-20T05:44:22.235961+0000","last_undegraded":"2023-12-20T05:44:22.235961+0000","last_fullsized":"2023-12-20T05:44:22.235961+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:29:27.313698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.014802+0000","last_change":"2023-12-20T05:44:20.619580+0000","last_active":"2023-12-20T05:44:21.014802+0000","last_peered":"2023-12-20T05:44:21.014802+0000","last_clean":"2023-12-20T05:44:21.014802+0000","last_became_active":"2023-12-20T05:44:20.619309+0000","last_became_peered":"2023-12-20T05:44:20.619309+0000","last_unstale":"2023-12-20T05:44:21.014802+0000","last_undegraded":"2023-12-20T05:44:21.014802+0000","last_fullsized":"2023-12-20T05:44:21.014802+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:50:46.184451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031101+0000","last_change":"2023-12-20T05:44:21.038055+0000","last_active":"2023-12-20T05:44:22.031101+0000","last_peered":"2023-12-20T05:44:22.031101+0000","last_clean":"2023-12-20T05:44:22.031101+0000","last_became_active":"2023-12-20T05:44:21.037600+0000","last_became_peered":"2023-12-20T05:44:21.037600+0000","last_unstale":"2023-12-20T05:44:22.031101+0000","last_undegraded":"2023-12-20T05:44:22.031101+0000","last_fullsized":"2023-12-20T05:44:22.031101+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:06:50.945771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"172'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025213+0000","last_change":"2023-12-20T05:44:20.622381+0000","last_active":"2023-12-20T05:44:22.025213+0000","last_peered":"2023-12-20T05:44:22.025213+0000","last_clean":"2023-12-20T05:44:22.025213+0000","last_became_active":"2023-12-20T05:44:20.622082+0000","last_became_peered":"2023-12-20T05:44:20.622082+0000","last_unstale":"2023-12-20T05:44:22.025213+0000","last_undegraded":"2023-12-20T05:44:22.025213+0000","last_fullsized":"2023-12-20T05:44:22.025213+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:19:20.046614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025217+0000","last_change":"2023-12-20T05:44:21.033956+0000","last_active":"2023-12-20T05:44:22.025217+0000","last_peered":"2023-12-20T05:44:22.025217+0000","last_clean":"2023-12-20T05:44:22.025217+0000","last_became_active":"2023-12-20T05:44:21.033669+0000","last_became_peered":"2023-12-20T05:44:21.033669+0000","last_unstale":"2023-12-20T05:44:22.025217+0000","last_undegraded":"2023-12-20T05:44:22.025217+0000","last_fullsized":"2023-12-20T05:44:22.025217+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:22.279077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.020755+0000","last_change":"2023-12-20T05:44:20.630274+0000","last_active":"2023-12-20T05:44:21.020755+0000","last_peered":"2023-12-20T05:44:21.020755+0000","last_clean":"2023-12-20T05:44:21.020755+0000","last_became_active":"2023-12-20T05:44:20.630004+0000","last_became_peered":"2023-12-20T05:44:20.630004+0000","last_unstale":"2023-12-20T05:44:21.020755+0000","last_undegraded":"2023-12-20T05:44:21.020755+0000","last_fullsized":"2023-12-20T05:44:21.020755+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:38:21.824896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030671+0000","last_change":"2023-12-20T05:44:21.033516+0000","last_active":"2023-12-20T05:44:22.030671+0000","last_peered":"2023-12-20T05:44:22.030671+0000","last_clean":"2023-12-20T05:44:22.030671+0000","last_became_active":"2023-12-20T05:44:21.033189+0000","last_became_peered":"2023-12-20T05:44:21.033189+0000","last_unstale":"2023-12-20T05:44:22.030671+0000","last_undegraded":"2023-12-20T05:44:22.030671+0000","last_fullsized":"2023-12-20T05:44:22.030671+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:07.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.020100+0000","last_change":"2023-12-20T05:44:20.630484+0000","last_active":"2023-12-20T05:44:21.020100+0000","last_peered":"2023-12-20T05:44:21.020100+0000","last_clean":"2023-12-20T05:44:21.020100+0000","last_became_active":"2023-12-20T05:44:20.630163+0000","last_became_peered":"2023-12-20T05:44:20.630163+0000","last_unstale":"2023-12-20T05:44:21.020100+0000","last_undegraded":"2023-12-20T05:44:21.020100+0000","last_fullsized":"2023-12-20T05:44:21.020100+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:34:13.594959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025135+0000","last_change":"2023-12-20T05:44:21.031138+0000","last_active":"2023-12-20T05:44:22.025135+0000","last_peered":"2023-12-20T05:44:22.025135+0000","last_clean":"2023-12-20T05:44:22.025135+0000","last_became_active":"2023-12-20T05:44:21.030847+0000","last_became_peered":"2023-12-20T05:44:21.030847+0000","last_unstale":"2023-12-20T05:44:22.025135+0000","last_undegraded":"2023-12-20T05:44:22.025135+0000","last_fullsized":"2023-12-20T05:44:22.025135+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:16:19.834909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.033576+0000","last_change":"2023-12-20T05:44:20.620662+0000","last_active":"2023-12-20T05:44:21.033576+0000","last_peered":"2023-12-20T05:44:21.033576+0000","last_clean":"2023-12-20T05:44:21.033576+0000","last_became_active":"2023-12-20T05:44:20.620375+0000","last_became_peered":"2023-12-20T05:44:20.620375+0000","last_unstale":"2023-12-20T05:44:21.033576+0000","last_undegraded":"2023-12-20T05:44:21.033576+0000","last_fullsized":"2023-12-20T05:44:21.033576+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:36:01.179411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.032074+0000","last_change":"2023-12-20T05:44:21.041404+0000","last_active":"2023-12-20T05:44:22.032074+0000","last_peered":"2023-12-20T05:44:22.032074+0000","last_clean":"2023-12-20T05:44:22.032074+0000","last_became_active":"2023-12-20T05:44:21.040503+0000","last_became_peered":"2023-12-20T05:44:21.040503+0000","last_unstale":"2023-12-20T05:44:22.032074+0000","last_undegraded":"2023-12-20T05:44:22.032074+0000","last_fullsized":"2023-12-20T05:44:22.032074+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:01:26.874334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.047571+0000","last_change":"2023-12-20T05:44:20.630226+0000","last_active":"2023-12-20T05:44:21.047571+0000","last_peered":"2023-12-20T05:44:21.047571+0000","last_clean":"2023-12-20T05:44:21.047571+0000","last_became_active":"2023-12-20T05:44:20.629730+0000","last_became_peered":"2023-12-20T05:44:20.629730+0000","last_unstale":"2023-12-20T05:44:21.047571+0000","last_undegraded":"2023-12-20T05:44:21.047571+0000","last_fullsized":"2023-12-20T05:44:21.047571+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:52:23.468622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.050944+0000","last_change":"2023-12-20T05:44:21.049651+0000","last_active":"2023-12-20T05:44:21.050944+0000","last_peered":"2023-12-20T05:44:21.050944+0000","last_clean":"2023-12-20T05:44:21.050944+0000","last_became_active":"2023-12-20T05:44:21.049339+0000","last_became_peered":"2023-12-20T05:44:21.049339+0000","last_unstale":"2023-12-20T05:44:21.050944+0000","last_undegraded":"2023-12-20T05:44:21.050944+0000","last_fullsized":"2023-12-20T05:44:21.050944+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:50:30.482977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236111+0000","last_change":"2023-12-20T05:44:20.633197+0000","last_active":"2023-12-20T05:44:22.236111+0000","last_peered":"2023-12-20T05:44:22.236111+0000","last_clean":"2023-12-20T05:44:22.236111+0000","last_became_active":"2023-12-20T05:44:20.632907+0000","last_became_peered":"2023-12-20T05:44:20.632907+0000","last_unstale":"2023-12-20T05:44:22.236111+0000","last_undegraded":"2023-12-20T05:44:22.236111+0000","last_fullsized":"2023-12-20T05:44:22.236111+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:36:01.179411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.038060+0000","last_change":"2023-12-20T05:44:21.029664+0000","last_active":"2023-12-20T05:44:21.038060+0000","last_peered":"2023-12-20T05:44:21.038060+0000","last_clean":"2023-12-20T05:44:21.038060+0000","last_became_active":"2023-12-20T05:44:21.028563+0000","last_became_peered":"2023-12-20T05:44:21.028563+0000","last_unstale":"2023-12-20T05:44:21.038060+0000","last_undegraded":"2023-12-20T05:44:21.038060+0000","last_fullsized":"2023-12-20T05:44:21.038060+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:54:19.715671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236110+0000","last_change":"2023-12-20T05:44:20.620429+0000","last_active":"2023-12-20T05:44:22.236110+0000","last_peered":"2023-12-20T05:44:22.236110+0000","last_clean":"2023-12-20T05:44:22.236110+0000","last_became_active":"2023-12-20T05:44:20.620127+0000","last_became_peered":"2023-12-20T05:44:20.620127+0000","last_unstale":"2023-12-20T05:44:22.236110+0000","last_undegraded":"2023-12-20T05:44:22.236110+0000","last_fullsized":"2023-12-20T05:44:22.236110+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:10:53.550958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236022+0000","last_change":"2023-12-20T05:44:21.034830+0000","last_active":"2023-12-20T05:44:22.236022+0000","last_peered":"2023-12-20T05:44:22.236022+0000","last_clean":"2023-12-20T05:44:22.236022+0000","last_became_active":"2023-12-20T05:44:21.034585+0000","last_became_peered":"2023-12-20T05:44:21.034585+0000","last_unstale":"2023-12-20T05:44:22.236022+0000","last_undegraded":"2023-12-20T05:44:22.236022+0000","last_fullsized":"2023-12-20T05:44:22.236022+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:56.360135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235943+0000","last_change":"2023-12-20T05:44:20.621040+0000","last_active":"2023-12-20T05:44:22.235943+0000","last_peered":"2023-12-20T05:44:22.235943+0000","last_clean":"2023-12-20T05:44:22.235943+0000","last_became_active":"2023-12-20T05:44:20.620754+0000","last_became_peered":"2023-12-20T05:44:20.620754+0000","last_unstale":"2023-12-20T05:44:22.235943+0000","last_undegraded":"2023-12-20T05:44:22.235943+0000","last_fullsized":"2023-12-20T05:44:22.235943+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:22.590583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.041108+0000","last_change":"2023-12-20T05:44:21.032210+0000","last_active":"2023-12-20T05:44:21.041108+0000","last_peered":"2023-12-20T05:44:21.041108+0000","last_clean":"2023-12-20T05:44:21.041108+0000","last_became_active":"2023-12-20T05:44:21.031207+0000","last_became_peered":"2023-12-20T05:44:21.031207+0000","last_unstale":"2023-12-20T05:44:21.041108+0000","last_undegraded":"2023-12-20T05:44:21.041108+0000","last_fullsized":"2023-12-20T05:44:21.041108+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:27:26.303679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025305+0000","last_change":"2023-12-20T05:44:20.627350+0000","last_active":"2023-12-20T05:44:22.025305+0000","last_peered":"2023-12-20T05:44:22.025305+0000","last_clean":"2023-12-20T05:44:22.025305+0000","last_became_active":"2023-12-20T05:44:20.627054+0000","last_became_peered":"2023-12-20T05:44:20.627054+0000","last_unstale":"2023-12-20T05:44:22.025305+0000","last_undegraded":"2023-12-20T05:44:22.025305+0000","last_fullsized":"2023-12-20T05:44:22.025305+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.134894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031809+0000","last_change":"2023-12-20T05:44:21.045238+0000","last_active":"2023-12-20T05:44:22.031809+0000","last_peered":"2023-12-20T05:44:22.031809+0000","last_clean":"2023-12-20T05:44:22.031809+0000","last_became_active":"2023-12-20T05:44:21.043982+0000","last_became_peered":"2023-12-20T05:44:21.043982+0000","last_unstale":"2023-12-20T05:44:22.031809+0000","last_undegraded":"2023-12-20T05:44:22.031809+0000","last_fullsized":"2023-12-20T05:44:22.031809+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:34:14.606009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.020755+0000","last_change":"2023-12-20T05:44:20.630028+0000","last_active":"2023-12-20T05:44:21.020755+0000","last_peered":"2023-12-20T05:44:21.020755+0000","last_clean":"2023-12-20T05:44:21.020755+0000","last_became_active":"2023-12-20T05:44:20.629534+0000","last_became_peered":"2023-12-20T05:44:20.629534+0000","last_unstale":"2023-12-20T05:44:21.020755+0000","last_undegraded":"2023-12-20T05:44:21.020755+0000","last_fullsized":"2023-12-20T05:44:21.020755+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:50:29.471927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.044658+0000","last_change":"2023-12-20T05:44:21.039736+0000","last_active":"2023-12-20T05:44:21.044658+0000","last_peered":"2023-12-20T05:44:21.044658+0000","last_clean":"2023-12-20T05:44:21.044658+0000","last_became_active":"2023-12-20T05:44:21.039419+0000","last_became_peered":"2023-12-20T05:44:21.039419+0000","last_unstale":"2023-12-20T05:44:21.044658+0000","last_undegraded":"2023-12-20T05:44:21.044658+0000","last_fullsized":"2023-12-20T05:44:21.044658+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:13.325177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.033592+0000","last_change":"2023-12-20T05:44:20.621499+0000","last_active":"2023-12-20T05:44:21.033592+0000","last_peered":"2023-12-20T05:44:21.033592+0000","last_clean":"2023-12-20T05:44:21.033592+0000","last_became_active":"2023-12-20T05:44:20.621173+0000","last_became_peered":"2023-12-20T05:44:20.621173+0000","last_unstale":"2023-12-20T05:44:21.033592+0000","last_undegraded":"2023-12-20T05:44:21.033592+0000","last_fullsized":"2023-12-20T05:44:21.033592+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:04:58.510517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031587+0000","last_change":"2023-12-20T05:44:21.028956+0000","last_active":"2023-12-20T05:44:22.031587+0000","last_peered":"2023-12-20T05:44:22.031587+0000","last_clean":"2023-12-20T05:44:22.031587+0000","last_became_active":"2023-12-20T05:44:21.028645+0000","last_became_peered":"2023-12-20T05:44:21.028645+0000","last_unstale":"2023-12-20T05:44:22.031587+0000","last_undegraded":"2023-12-20T05:44:22.031587+0000","last_fullsized":"2023-12-20T05:44:22.031587+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:38:05.186489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.014441+0000","last_change":"2023-12-20T05:44:20.631124+0000","last_active":"2023-12-20T05:44:21.014441+0000","last_peered":"2023-12-20T05:44:21.014441+0000","last_clean":"2023-12-20T05:44:21.014441+0000","last_became_active":"2023-12-20T05:44:20.630811+0000","last_became_peered":"2023-12-20T05:44:20.630811+0000","last_unstale":"2023-12-20T05:44:21.014441+0000","last_undegraded":"2023-12-20T05:44:21.014441+0000","last_fullsized":"2023-12-20T05:44:21.014441+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:30:13.510051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025115+0000","last_change":"2023-12-20T05:44:21.033115+0000","last_active":"2023-12-20T05:44:22.025115+0000","last_peered":"2023-12-20T05:44:22.025115+0000","last_clean":"2023-12-20T05:44:22.025115+0000","last_became_active":"2023-12-20T05:44:21.032789+0000","last_became_peered":"2023-12-20T05:44:21.032789+0000","last_unstale":"2023-12-20T05:44:22.025115+0000","last_undegraded":"2023-12-20T05:44:22.025115+0000","last_fullsized":"2023-12-20T05:44:22.025115+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:08.343570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"172'3","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045638+0000","last_change":"2023-12-20T05:44:20.620751+0000","last_active":"2023-12-20T05:44:21.045638+0000","last_peered":"2023-12-20T05:44:21.045638+0000","last_clean":"2023-12-20T05:44:21.045638+0000","last_became_active":"2023-12-20T05:44:20.620461+0000","last_became_peered":"2023-12-20T05:44:20.620461+0000","last_unstale":"2023-12-20T05:44:21.045638+0000","last_undegraded":"2023-12-20T05:44:21.045638+0000","last_fullsized":"2023-12-20T05:44:21.045638+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:22.590583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"161'3146","reported_seq":3756,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-12-20T05:44:20.620997+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T05:44:20.620997+0000","last_peered":"2023-12-20T05:44:20.620997+0000","last_clean":"2023-12-20T05:44:20.620997+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T05:44:20.620997+0000","last_undegraded":"2023-12-20T05:44:20.620997+0000","last_fullsized":"2023-12-20T05:44:20.620997+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3146,"log_dups_size":0,"ondisk_log_size":3146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1405,"num_read_kb":2819,"num_write":421,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1405,"num_read_kb":2819,"num_write":421,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3146,"ondisk_log_size":3146,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575755,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37828,"kb_used_data":2684,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334012,"statfs":{"total":96636764160,"available":96598028288,"internally_reserved":0,"allocated":2748416,"data_stored":1621029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82199999999999995,"5min":0.98099999999999998,"15min":1.133},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.03},"last":0.67700000000000005}]},{"osd":1,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73199999999999998,"15min":0.74099999999999999},"min":{"1min":0.54800000000000004,"5min":0.48099999999999998,"15min":0.45200000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47699999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.59099999999999997}]},{"osd":2,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47399999999999998,"5min":0.47099999999999997,"15min":0.45500000000000002},"max":{"1min":0.93700000000000006,"5min":1.036,"15min":1.0589999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":2.2160000000000002},"last":0.70199999999999996}]},{"osd":3,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.749,"15min":0.753},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.496},"max":{"1min":1.109,"5min":1.109,"15min":2.6829999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.58599999999999997,"5min":0.50600000000000001,"15min":0.49299999999999999},"max":{"1min":1.264,"5min":1.264,"15min":2.1259999999999999},"last":0.63}]},{"osd":4,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59699999999999998,"15min":0.60199999999999998},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":1.8180000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.40200000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":2.7370000000000001},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.48599999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":2.27},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68500000000000005,"15min":0.68999999999999995},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.74099999999999999}]},{"osd":6,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.755,"15min":0.752},"min":{"1min":0.53900000000000003,"5min":0.47299999999999998,"15min":0.42699999999999999},"max":{"1min":1.147,"5min":1.147,"15min":1.1699999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.71099999999999997,"15min":0.71199999999999997},"min":{"1min":0.45800000000000002,"5min":0.45400000000000001,"15min":0.45200000000000001},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.1850000000000001},"last":0.77000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575755,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47108,"kb_used_data":2108,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324732,"statfs":{"total":96636764160,"available":96588525568,"internally_reserved":0,"allocated":2158592,"data_stored":1030155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73499999999999999,"15min":0.748},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.36899999999999999},"max":{"1min":0.98699999999999999,"5min":1,"15min":1.093},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75,"15min":0.75700000000000001},"min":{"1min":0.47499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.006,"5min":1.0189999999999999,"15min":1.101},"last":0.52900000000000003}]},{"osd":1,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":1.002,"15min":1.03},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.749,"15min":0.76800000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":1.038},"last":0.77300000000000002}]},{"osd":2,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.76700000000000002,"15min":0.77800000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0329999999999999,"5min":1.048,"15min":1.0960000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.48399999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":1.0369999999999999,"15min":1.083},"last":0.64600000000000002}]},{"osd":3,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.78000000000000003,"15min":0.79100000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.995,"5min":0.995,"15min":1.0629999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.1020000000000001},"last":0.61599999999999999}]},{"osd":4,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.36599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.67500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.046,"5min":1.048,"15min":1.048},"last":0.72899999999999998}]},{"osd":5,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41799999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.0429999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.47799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.874}]},{"osd":7,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93100000000000005,"5min":1.095,"15min":1.095},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73299999999999998,"15min":0.73599999999999999},"min":{"1min":0.57299999999999995,"5min":0.49299999999999999,"15min":0.49099999999999999},"max":{"1min":0.95899999999999996,"5min":1.0049999999999999,"15min":1.105},"last":0.754}]}]},{"osd":5,"up_from":13,"seq":55834575756,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29052,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342788,"statfs":{"total":96636764160,"available":96607014912,"internally_reserved":0,"allocated":2150400,"data_stored":1030031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.119,"5min":1.145,"15min":1.145},"last":0.52600000000000002}]},{"osd":1,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.749,"15min":0.751},"min":{"1min":0.497,"5min":0.44900000000000001,"15min":0.42799999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.016},"last":0.505},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78200000000000003,"15min":0.76800000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.48199999999999998,"15min":0.44700000000000001},"max":{"1min":1.048,"5min":1.204,"15min":1.204},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.60199999999999998,"5min":0.497,"15min":0.49399999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.66600000000000004}]},{"osd":3,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.78500000000000003},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":1.012,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81100000000000005,"15min":0.79100000000000004},"min":{"1min":0.68300000000000005,"5min":0.56699999999999995,"15min":0.53300000000000003},"max":{"1min":1.097,"5min":1.107,"15min":1.107},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.66800000000000004,"15min":0.64300000000000002},"min":{"1min":0.46700000000000003,"5min":0.436,"15min":0.42999999999999999},"max":{"1min":0.98099999999999998,"5min":1.167,"15min":1.167},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67000000000000004,"15min":0.64600000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.39100000000000001},"max":{"1min":0.94699999999999995,"5min":1.008,"15min":1.008},"last":0.81999999999999995}]},{"osd":6,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.73199999999999998,"15min":0.70199999999999996},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.435},"max":{"1min":1.014,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.66500000000000004,"15min":0.64200000000000002},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.40999999999999998},"max":{"1min":0.92100000000000004,"5min":0.98999999999999999,"15min":1.085},"last":0.54800000000000004}]},{"osd":7,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.72999999999999998,"15min":0.70699999999999996},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0429999999999999,"5min":1.073,"15min":1.1020000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.80200000000000005,"15min":0.76800000000000002},"min":{"1min":0.57399999999999995,"5min":0.52200000000000002,"15min":0.48199999999999998},"max":{"1min":1.0229999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575757,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63492,"kb_used_data":2108,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308348,"statfs":{"total":96636764160,"available":96571748352,"internally_reserved":0,"allocated":2158592,"data_stored":1036337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34399999999999997},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.75,"15min":0.73799999999999999},"min":{"1min":0.57299999999999995,"5min":0.47599999999999998,"15min":0.35699999999999998},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":0.52500000000000002}]},{"osd":1,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.0249999999999999,"5min":1.077,"15min":1.137},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.75,"15min":0.75700000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.68700000000000006}]},{"osd":2,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.76500000000000001,"15min":0.76600000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.78100000000000003,"15min":0.77800000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.51900000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.60099999999999998}]},{"osd":3,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.45500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.624},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.79800000000000004,"15min":0.79200000000000004},"min":{"1min":0.58699999999999997,"5min":0.51500000000000001,"15min":0.47999999999999998},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.895,"5min":2.895,"15min":2.895},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.629,"15min":0.621},"min":{"1min":0.41199999999999998,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.67600000000000005}]},{"osd":6,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.68700000000000006,"15min":0.65800000000000003},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.40100000000000002},"max":{"1min":3.8279999999999998,"5min":3.8279999999999998,"15min":3.8279999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.70499999999999996,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.44700000000000001,"15min":0.436},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.77800000000000002,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.50800000000000001,"15min":0.47099999999999997},"max":{"1min":3.8690000000000002,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":0.82299999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575756,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47100,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324740,"statfs":{"total":96636764160,"available":96588533760,"internally_reserved":0,"allocated":2150400,"data_stored":1030099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46300000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.88800000000000001},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52200000000000002,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.308,"15min":0.29799999999999999},"max":{"1min":0.73699999999999999,"5min":0.75900000000000001,"15min":0.876},"last":0.45600000000000002}]},{"osd":1,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.49099999999999999,"15min":0.47499999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.501},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45100000000000001,"5min":0.44700000000000001,"15min":0.40400000000000003},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.42799999999999999}]},{"osd":4,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55200000000000005},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.86599999999999999,"15min":0.83399999999999996},"min":{"1min":0.54800000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":4.2800000000000002,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":0.78700000000000003}]},{"osd":5,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.85999999999999999,"15min":0.84699999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":4.3259999999999996,"5min":4.3259999999999996,"15min":4.3259999999999996},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.83199999999999996,"15min":0.81699999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.50700000000000001},"max":{"1min":1.1220000000000001,"5min":1.232,"15min":1.232},"last":0.76600000000000001}]},{"osd":6,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83499999999999996,"15min":0.83099999999999996},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.53200000000000003},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.86399999999999999,"15min":0.85399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.89000000000000001}]},{"osd":7,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.198,"5min":1.383,"15min":1.383},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85499999999999998,"15min":0.86199999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.56699999999999995},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.873}]}]},{"osd":2,"up_from":13,"seq":55834575755,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47112,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324728,"statfs":{"total":96636764160,"available":96588521472,"internally_reserved":0,"allocated":2162688,"data_stored":1031643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.89700000000000002},"last":0.373}]},{"osd":1,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32700000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.53500000000000003}]},{"osd":3,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69099999999999995,"15min":0.68400000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.442},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.45600000000000002}]},{"osd":4,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.83799999999999997,"15min":0.81399999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.50900000000000001},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.48199999999999998},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.68300000000000005}]},{"osd":5,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.83599999999999997,"15min":0.82399999999999995},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.85299999999999998,"15min":0.83499999999999996},"min":{"1min":0.623,"5min":0.58699999999999997,"15min":0.57999999999999996},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85999999999999999,"15min":0.83299999999999996},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.86799999999999999,"15min":0.84199999999999997},"min":{"1min":0.58999999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.876,"15min":0.86399999999999999},"min":{"1min":0.60299999999999998,"5min":0.54800000000000004,"15min":0.51800000000000002},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91600000000000004,"15min":0.89400000000000002},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.59999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575757,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55292,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316548,"statfs":{"total":96636764160,"available":96580145152,"internally_reserved":0,"allocated":2150400,"data_stored":1030591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46700000000000003},"min":{"1min":0.318,"5min":0.311,"15min":0.29499999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32500000000000001},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53500000000000003,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33000000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.35399999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.501},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.53600000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.284,"5min":1.284,"15min":1.294},"last":0.82399999999999995}]},{"osd":5,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82799999999999996,"15min":0.81100000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54000000000000004},"max":{"1min":1.131,"5min":1.2010000000000001,"15min":1.2709999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.85099999999999998,"15min":0.82599999999999996},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.54500000000000004},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.89400000000000002}]},{"osd":6,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":0.84299999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.59299999999999997},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.83899999999999997,"15min":0.82699999999999996},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.54700000000000004},"max":{"1min":1.085,"5min":1.1459999999999999,"15min":1.284},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.872,"15min":0.86699999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.036,"5min":1.294,"15min":1.3620000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.90400000000000003,"15min":0.88400000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.55900000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.80100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575756,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51788,"kb_used_data":2692,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320052,"statfs":{"total":96636764160,"available":96583733248,"internally_reserved":0,"allocated":2756608,"data_stored":1621975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.88100000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.35799999999999998,"5min":0.35399999999999998,"15min":0.35199999999999998},"max":{"1min":0.78000000000000003,"5min":0.874,"15min":1.3859999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77200000000000002,"5min":0.79600000000000004,"15min":1.369},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71599999999999997,"5min":0.84699999999999998,"15min":1.2709999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":2.016},"last":0.439}]},{"osd":4,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.56200000000000006,"5min":0.52800000000000002,"15min":0.47799999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.294},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.63200000000000001,"5min":0.58099999999999996,"15min":0.52100000000000002},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.4530000000000001},"last":0.66000000000000003}]},{"osd":5,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80200000000000005,"15min":0.82199999999999995},"min":{"1min":0.59199999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.177,"5min":1.238,"15min":1.6080000000000001},"last":0.753},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81699999999999995,"15min":0.81899999999999995},"min":{"1min":0.60999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.129,"5min":1.22,"15min":1.3200000000000001},"last":0.64000000000000001}]},{"osd":6,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.621,"5min":0.499,"15min":0.499},"max":{"1min":1.256,"5min":1.256,"15min":1.665},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55100000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1439999999999999,"5min":1.167,"15min":1.5660000000000001},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.63200000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.212,"5min":1.222,"15min":1.4670000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.198,"5min":1.2649999999999999,"15min":1.415},"last":0.84399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T05:44:25.136 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T05:44:25.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.319+0000 7fbdcd7ce640 1 Processor -- start 2023-12-20T05:44:25.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.319+0000 7fbdcd7ce640 1 -- start start 2023-12-20T05:44:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdcd7ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbdc81599e0 0x7fbdc80aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdcd7ce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbdc80c90e0 0x7fbdc80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdcd7ce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbdc80711d0 con 0x7fbdc80b0b30 2023-12-20T05:44:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdcd7ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc80c98c0 con 0x7fbdc81599e0 2023-12-20T05:44:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdcd7ce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbdc80c9ba0 con 0x7fbdc80c90e0 2023-12-20T05:44:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdc6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbdc81599e0 0x7fbdc80aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:25.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdc6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbdc81599e0 0x7fbdc80aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59016/0 (socket says 172.21.15.32:59016) 2023-12-20T05:44:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdc6ffd640 1 -- 172.21.15.32:0/2110182147 learned_addr learned my addr 172.21.15.32:0/2110182147 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdc67fc640 1 --2- 172.21.15.32:0/2110182147 >> v2:172.21.15.39:3300/0 conn(0x7fbdc80c90e0 0x7fbdc80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdc6ffd640 1 -- 172.21.15.32:0/2110182147 >> v2:172.21.15.39:3300/0 conn(0x7fbdc80c90e0 msgr2=0x7fbdc80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdc6ffd640 1 --2- 172.21.15.32:0/2110182147 >> v2:172.21.15.39:3300/0 conn(0x7fbdc80c90e0 0x7fbdc80b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdc6ffd640 1 -- 172.21.15.32:0/2110182147 >> v1:172.21.15.39:6789/0 conn(0x7fbdc80b0b30 legacy=0x7fbdc815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdc6ffd640 1 -- 172.21.15.32:0/2110182147 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdc80c9e80 con 0x7fbdc81599e0 2023-12-20T05:44:25.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdc67fc640 1 --2- 172.21.15.32:0/2110182147 >> v2:172.21.15.39:3300/0 conn(0x7fbdc80c90e0 0x7fbdc80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:44:25.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.323+0000 7fbdc6ffd640 1 --2- 172.21.15.32:0/2110182147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc81599e0 0x7fbdc80aece0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fbdc806ab20 tx=0x7fbdb801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19c4099ec397c2c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdc5ffb640 1 -- 172.21.15.32:0/2110182147 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80ac410 con 0x7fbdc81599e0 2023-12-20T05:44:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdc5ffb640 1 -- 172.21.15.32:0/2110182147 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb80ac570 con 0x7fbdc81599e0 2023-12-20T05:44:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdc5ffb640 1 -- 172.21.15.32:0/2110182147 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80ac900 con 0x7fbdc81599e0 2023-12-20T05:44:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/2110182147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc81599e0 msgr2=0x7fbdc80aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 --2- 172.21.15.32:0/2110182147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc81599e0 0x7fbdc80aece0 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7fbdc806ab20 tx=0x7fbdb801c490 comp rx=0 tx=0).stop 2023-12-20T05:44:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/2110182147 shutdown_connections 2023-12-20T05:44:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 --2- 172.21.15.32:0/2110182147 >> v2:172.21.15.39:3300/0 conn(0x7fbdc80c90e0 0x7fbdc80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 --2- 172.21.15.32:0/2110182147 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc81599e0 0x7fbdc80aece0 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/2110182147 >> 172.21.15.32:0/2110182147 conn(0x7fbdc806a320 msgr2=0x7fbdc80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:25.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/2110182147 shutdown_connections 2023-12-20T05:44:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/2110182147 wait complete. 2023-12-20T05:44:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 Processor -- start 2023-12-20T05:44:25.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 -- start start 2023-12-20T05:44:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80b0b30 0x7fbdc815afc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdc80c90e0 0x7fbdc80b11b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbdc81599e0 0x7fbdc80c7b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80c6dd0 con 0x7fbdc80b0b30 2023-12-20T05:44:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc8149b80 con 0x7fbdc80c90e0 2023-12-20T05:44:25.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdcd7ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbdc80c7070 con 0x7fbdc81599e0 2023-12-20T05:44:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.327+0000 7fbdc6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80b0b30 0x7fbdc815afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdc6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80b0b30 0x7fbdc815afc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59032/0 (socket says 172.21.15.32:59032) 2023-12-20T05:44:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdc6ffd640 1 -- 172.21.15.32:0/1208885813 learned_addr learned my addr 172.21.15.32:0/1208885813 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdc67fc640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdc80c90e0 0x7fbdc80b11b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdc77fe640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbdc81599e0 0x7fbdc80c7b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdc6ffd640 1 -- 172.21.15.32:0/1208885813 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbdc81599e0 msgr2=0x7fbdc80c7b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdc6ffd640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbdc81599e0 0x7fbdc80c7b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdc6ffd640 1 -- 172.21.15.32:0/1208885813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdc80c90e0 msgr2=0x7fbdc80b11b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdc6ffd640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdc80c90e0 0x7fbdc80b11b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdc6ffd640 1 -- 172.21.15.32:0/1208885813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbdb8009480 con 0x7fbdc80b0b30 2023-12-20T05:44:25.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdc77fe640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbdc81599e0 0x7fbdc80c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdc6ffd640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80b0b30 0x7fbdc815afc0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fbdb801a0c0 tx=0x7fbdb8019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdaf7fe640 1 -- 172.21.15.32:0/1208885813 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb80059a0 con 0x7fbdc80b0b30 2023-12-20T05:44:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdaf7fe640 1 -- 172.21.15.32:0/1208885813 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbdb8005b00 con 0x7fbdc80b0b30 2023-12-20T05:44:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdaf7fe640 1 -- 172.21.15.32:0/1208885813 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbdb8005e70 con 0x7fbdc80b0b30 2023-12-20T05:44:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/1208885813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbdc80c7350 con 0x7fbdc80b0b30 2023-12-20T05:44:25.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/1208885813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbdc80b16f0 con 0x7fbdc80b0b30 2023-12-20T05:44:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/1208885813 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbdc804e000 con 0x7fbdc80b0b30 2023-12-20T05:44:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdaf7fe640 1 -- 172.21.15.32:0/1208885813 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbdb80b3040 con 0x7fbdc80b0b30 2023-12-20T05:44:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdaf7fe640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdb4065b80 0x7fbdb4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.331+0000 7fbdaf7fe640 1 -- 172.21.15.32:0/1208885813 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7fbdb811c4a0 con 0x7fbdc80b0b30 2023-12-20T05:44:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.335+0000 7fbdc67fc640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdb4065b80 0x7fbdb4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:25.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.335+0000 7fbdc67fc640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdb4065b80 0x7fbdb4068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80c5030 tx=0x7fbdbc008040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:25.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.335+0000 7fbdaf7fe640 1 -- 172.21.15.32:0/1208885813 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbdb80e9c30 con 0x7fbdc80b0b30 2023-12-20T05:44:25.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.491+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/1208885813 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fbdc80b0fa0 con 0x7fbdb4065b80 2023-12-20T05:44:25.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.499+0000 7fbdaf7fe640 1 -- 172.21.15.32:0/1208885813 <== mgr.9201 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289170 (secure 0 0 0) 0x7fbdbc00c040 con 0x7fbdb4065b80 2023-12-20T05:44:25.502 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:25.504 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T05:44:25.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/1208885813 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdb4065b80 msgr2=0x7fbdb4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdb4065b80 0x7fbdb4068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7fbdc80c5030 tx=0x7fbdbc008040 comp rx=0 tx=0).stop 2023-12-20T05:44:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/1208885813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80b0b30 msgr2=0x7fbdc815afc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80b0b30 0x7fbdc815afc0 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7fbdb801a0c0 tx=0x7fbdb8019230 comp rx=0 tx=0).stop 2023-12-20T05:44:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/1208885813 shutdown_connections 2023-12-20T05:44:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7fbdb4065b80 0x7fbdb4068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbdc81599e0 0x7fbdc80c7b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbdc80c90e0 0x7fbdc80b11b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 --2- 172.21.15.32:0/1208885813 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbdc80b0b30 0x7fbdc815afc0 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/1208885813 >> 172.21.15.32:0/1208885813 conn(0x7fbdc806a320 msgr2=0x7fbdc8154fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/1208885813 shutdown_connections 2023-12-20T05:44:25.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.503+0000 7fbdcd7ce640 1 -- 172.21.15.32:0/1208885813 wait complete. 2023-12-20T05:44:25.522 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-20T05:44:24.325450+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1405,"num_read_kb":2819,"num_write":465,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3184,"ondisk_log_size":3184,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":114,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":378772,"kb_used_data":18004,"kb_used_omap":0,"kb_used_meta":360704,"kb_avail":754595948,"statfs":{"total":773094113280,"available":772706250752,"internally_reserved":0,"allocated":18436096,"data_stored":9431860,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":369360896},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":37,"apply_latency_ms":37,"commit_latency_ns":37000000,"apply_latency_ns":37000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.729439"},"pg_stats":[{"pgid":"21.3f","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236116+0000","last_change":"2023-12-20T05:44:21.041236+0000","last_active":"2023-12-20T05:44:22.236116+0000","last_peered":"2023-12-20T05:44:22.236116+0000","last_clean":"2023-12-20T05:44:22.236116+0000","last_became_active":"2023-12-20T05:44:21.040830+0000","last_became_peered":"2023-12-20T05:44:21.040830+0000","last_unstale":"2023-12-20T05:44:22.236116+0000","last_undegraded":"2023-12-20T05:44:22.236116+0000","last_fullsized":"2023-12-20T05:44:22.236116+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:43:38.090630+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.3e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.037932+0000","last_change":"2023-12-20T05:44:21.032312+0000","last_active":"2023-12-20T05:44:21.037932+0000","last_peered":"2023-12-20T05:44:21.037932+0000","last_clean":"2023-12-20T05:44:21.037932+0000","last_became_active":"2023-12-20T05:44:21.032057+0000","last_became_peered":"2023-12-20T05:44:21.032057+0000","last_unstale":"2023-12-20T05:44:21.037932+0000","last_undegraded":"2023-12-20T05:44:21.037932+0000","last_fullsized":"2023-12-20T05:44:21.037932+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:29:24.149040+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.3d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045682+0000","last_change":"2023-12-20T05:44:21.040706+0000","last_active":"2023-12-20T05:44:21.045682+0000","last_peered":"2023-12-20T05:44:21.045682+0000","last_clean":"2023-12-20T05:44:21.045682+0000","last_became_active":"2023-12-20T05:44:21.039460+0000","last_became_peered":"2023-12-20T05:44:21.039460+0000","last_unstale":"2023-12-20T05:44:21.045682+0000","last_undegraded":"2023-12-20T05:44:21.045682+0000","last_fullsized":"2023-12-20T05:44:21.045682+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:54:35.407281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.3c","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031462+0000","last_change":"2023-12-20T05:44:21.043665+0000","last_active":"2023-12-20T05:44:22.031462+0000","last_peered":"2023-12-20T05:44:22.031462+0000","last_clean":"2023-12-20T05:44:22.031462+0000","last_became_active":"2023-12-20T05:44:21.043390+0000","last_became_peered":"2023-12-20T05:44:21.043390+0000","last_unstale":"2023-12-20T05:44:22.031462+0000","last_undegraded":"2023-12-20T05:44:22.031462+0000","last_fullsized":"2023-12-20T05:44:22.031462+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:30:14.521101+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.3b","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031621+0000","last_change":"2023-12-20T05:44:21.044029+0000","last_active":"2023-12-20T05:44:22.031621+0000","last_peered":"2023-12-20T05:44:22.031621+0000","last_clean":"2023-12-20T05:44:22.031621+0000","last_became_active":"2023-12-20T05:44:21.042159+0000","last_became_peered":"2023-12-20T05:44:21.042159+0000","last_unstale":"2023-12-20T05:44:22.031621+0000","last_undegraded":"2023-12-20T05:44:22.031621+0000","last_fullsized":"2023-12-20T05:44:22.031621+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:50:30.482977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.3a","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235885+0000","last_change":"2023-12-20T05:44:21.043357+0000","last_active":"2023-12-20T05:44:22.235885+0000","last_peered":"2023-12-20T05:44:22.235885+0000","last_clean":"2023-12-20T05:44:22.235885+0000","last_became_active":"2023-12-20T05:44:21.043113+0000","last_became_peered":"2023-12-20T05:44:21.043113+0000","last_unstale":"2023-12-20T05:44:22.235885+0000","last_undegraded":"2023-12-20T05:44:22.235885+0000","last_fullsized":"2023-12-20T05:44:22.235885+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:24:14.449581+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.39","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025244+0000","last_change":"2023-12-20T05:44:21.031470+0000","last_active":"2023-12-20T05:44:22.025244+0000","last_peered":"2023-12-20T05:44:22.025244+0000","last_clean":"2023-12-20T05:44:22.025244+0000","last_became_active":"2023-12-20T05:44:21.031205+0000","last_became_peered":"2023-12-20T05:44:21.031205+0000","last_unstale":"2023-12-20T05:44:22.025244+0000","last_undegraded":"2023-12-20T05:44:22.025244+0000","last_fullsized":"2023-12-20T05:44:22.025244+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:25:28.845267+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.38","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235751+0000","last_change":"2023-12-20T05:44:21.040118+0000","last_active":"2023-12-20T05:44:22.235751+0000","last_peered":"2023-12-20T05:44:22.235751+0000","last_clean":"2023-12-20T05:44:22.235751+0000","last_became_active":"2023-12-20T05:44:21.039808+0000","last_became_peered":"2023-12-20T05:44:21.039808+0000","last_unstale":"2023-12-20T05:44:22.235751+0000","last_undegraded":"2023-12-20T05:44:22.235751+0000","last_fullsized":"2023-12-20T05:44:22.235751+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:01:15.439613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.37","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.039716+0000","last_change":"2023-12-20T05:44:21.036405+0000","last_active":"2023-12-20T05:44:21.039716+0000","last_peered":"2023-12-20T05:44:21.039716+0000","last_clean":"2023-12-20T05:44:21.039716+0000","last_became_active":"2023-12-20T05:44:21.036123+0000","last_became_peered":"2023-12-20T05:44:21.036123+0000","last_unstale":"2023-12-20T05:44:21.039716+0000","last_undegraded":"2023-12-20T05:44:21.039716+0000","last_fullsized":"2023-12-20T05:44:21.039716+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:28:04.948693+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.36","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.049027+0000","last_change":"2023-12-20T05:44:21.046085+0000","last_active":"2023-12-20T05:44:21.049027+0000","last_peered":"2023-12-20T05:44:21.049027+0000","last_clean":"2023-12-20T05:44:21.049027+0000","last_became_active":"2023-12-20T05:44:21.045787+0000","last_became_peered":"2023-12-20T05:44:21.045787+0000","last_unstale":"2023-12-20T05:44:21.049027+0000","last_undegraded":"2023-12-20T05:44:21.049027+0000","last_fullsized":"2023-12-20T05:44:21.049027+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:30:03.168590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.35","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235920+0000","last_change":"2023-12-20T05:44:21.041911+0000","last_active":"2023-12-20T05:44:22.235920+0000","last_peered":"2023-12-20T05:44:22.235920+0000","last_clean":"2023-12-20T05:44:22.235920+0000","last_became_active":"2023-12-20T05:44:21.041655+0000","last_became_peered":"2023-12-20T05:44:21.041655+0000","last_unstale":"2023-12-20T05:44:22.235920+0000","last_undegraded":"2023-12-20T05:44:22.235920+0000","last_fullsized":"2023-12-20T05:44:22.235920+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:43:42.093760+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.34","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.044630+0000","last_change":"2023-12-20T05:44:21.040214+0000","last_active":"2023-12-20T05:44:21.044630+0000","last_peered":"2023-12-20T05:44:21.044630+0000","last_clean":"2023-12-20T05:44:21.044630+0000","last_became_active":"2023-12-20T05:44:21.035744+0000","last_became_peered":"2023-12-20T05:44:21.035744+0000","last_unstale":"2023-12-20T05:44:21.044630+0000","last_undegraded":"2023-12-20T05:44:21.044630+0000","last_fullsized":"2023-12-20T05:44:21.044630+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:24:45.142549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.33","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031731+0000","last_change":"2023-12-20T05:44:21.039998+0000","last_active":"2023-12-20T05:44:22.031731+0000","last_peered":"2023-12-20T05:44:22.031731+0000","last_clean":"2023-12-20T05:44:22.031731+0000","last_became_active":"2023-12-20T05:44:21.034989+0000","last_became_peered":"2023-12-20T05:44:21.034989+0000","last_unstale":"2023-12-20T05:44:22.031731+0000","last_undegraded":"2023-12-20T05:44:22.031731+0000","last_fullsized":"2023-12-20T05:44:22.031731+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:54:35.407281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.32","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030801+0000","last_change":"2023-12-20T05:44:21.035572+0000","last_active":"2023-12-20T05:44:22.030801+0000","last_peered":"2023-12-20T05:44:22.030801+0000","last_clean":"2023-12-20T05:44:22.030801+0000","last_became_active":"2023-12-20T05:44:21.035286+0000","last_became_peered":"2023-12-20T05:44:21.035286+0000","last_unstale":"2023-12-20T05:44:22.030801+0000","last_undegraded":"2023-12-20T05:44:22.030801+0000","last_fullsized":"2023-12-20T05:44:22.030801+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:00:53.185594+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.31","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.044019+0000","last_change":"2023-12-20T05:44:21.038954+0000","last_active":"2023-12-20T05:44:21.044019+0000","last_peered":"2023-12-20T05:44:21.044019+0000","last_clean":"2023-12-20T05:44:21.044019+0000","last_became_active":"2023-12-20T05:44:21.038639+0000","last_became_peered":"2023-12-20T05:44:21.038639+0000","last_unstale":"2023-12-20T05:44:21.044019+0000","last_undegraded":"2023-12-20T05:44:21.044019+0000","last_fullsized":"2023-12-20T05:44:21.044019+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:23:19.803660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.30","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.043308+0000","last_change":"2023-12-20T05:44:21.037271+0000","last_active":"2023-12-20T05:44:21.043308+0000","last_peered":"2023-12-20T05:44:21.043308+0000","last_clean":"2023-12-20T05:44:21.043308+0000","last_became_active":"2023-12-20T05:44:21.036877+0000","last_became_peered":"2023-12-20T05:44:21.036877+0000","last_unstale":"2023-12-20T05:44:21.043308+0000","last_undegraded":"2023-12-20T05:44:21.043308+0000","last_fullsized":"2023-12-20T05:44:21.043308+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:58:31.373035+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.046362+0000","last_change":"2023-12-20T05:44:21.043437+0000","last_active":"2023-12-20T05:44:21.046362+0000","last_peered":"2023-12-20T05:44:21.046362+0000","last_clean":"2023-12-20T05:44:21.046362+0000","last_became_active":"2023-12-20T05:44:21.043108+0000","last_became_peered":"2023-12-20T05:44:21.043108+0000","last_unstale":"2023-12-20T05:44:21.046362+0000","last_undegraded":"2023-12-20T05:44:21.046362+0000","last_fullsized":"2023-12-20T05:44:21.046362+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:16.798109+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045465+0000","last_change":"2023-12-20T05:44:21.041694+0000","last_active":"2023-12-20T05:44:21.045465+0000","last_peered":"2023-12-20T05:44:21.045465+0000","last_clean":"2023-12-20T05:44:21.045465+0000","last_became_active":"2023-12-20T05:44:21.039386+0000","last_became_peered":"2023-12-20T05:44:21.039386+0000","last_unstale":"2023-12-20T05:44:21.045465+0000","last_undegraded":"2023-12-20T05:44:21.045465+0000","last_fullsized":"2023-12-20T05:44:21.045465+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:21.786814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.2d","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236045+0000","last_change":"2023-12-20T05:44:21.044105+0000","last_active":"2023-12-20T05:44:22.236045+0000","last_peered":"2023-12-20T05:44:22.236045+0000","last_clean":"2023-12-20T05:44:22.236045+0000","last_became_active":"2023-12-20T05:44:21.043842+0000","last_became_peered":"2023-12-20T05:44:21.043842+0000","last_unstale":"2023-12-20T05:44:22.236045+0000","last_undegraded":"2023-12-20T05:44:22.236045+0000","last_fullsized":"2023-12-20T05:44:22.236045+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:01:26.874334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.041237+0000","last_change":"2023-12-20T05:44:21.034426+0000","last_active":"2023-12-20T05:44:21.041237+0000","last_peered":"2023-12-20T05:44:21.041237+0000","last_clean":"2023-12-20T05:44:21.041237+0000","last_became_active":"2023-12-20T05:44:21.034015+0000","last_became_peered":"2023-12-20T05:44:21.034015+0000","last_unstale":"2023-12-20T05:44:21.041237+0000","last_undegraded":"2023-12-20T05:44:21.041237+0000","last_fullsized":"2023-12-20T05:44:21.041237+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:38:02.506656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.2b","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.040800+0000","last_change":"2023-12-20T05:44:21.037116+0000","last_active":"2023-12-20T05:44:21.040800+0000","last_peered":"2023-12-20T05:44:21.040800+0000","last_clean":"2023-12-20T05:44:21.040800+0000","last_became_active":"2023-12-20T05:44:21.036509+0000","last_became_peered":"2023-12-20T05:44:21.036509+0000","last_unstale":"2023-12-20T05:44:21.040800+0000","last_undegraded":"2023-12-20T05:44:21.040800+0000","last_fullsized":"2023-12-20T05:44:21.040800+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:47.291062+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.2a","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030915+0000","last_change":"2023-12-20T05:44:21.031175+0000","last_active":"2023-12-20T05:44:22.030915+0000","last_peered":"2023-12-20T05:44:22.030915+0000","last_clean":"2023-12-20T05:44:22.030915+0000","last_became_active":"2023-12-20T05:44:21.030872+0000","last_became_peered":"2023-12-20T05:44:21.030872+0000","last_unstale":"2023-12-20T05:44:22.030915+0000","last_undegraded":"2023-12-20T05:44:22.030915+0000","last_fullsized":"2023-12-20T05:44:22.030915+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:29:57.434741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.29","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030880+0000","last_change":"2023-12-20T05:44:21.037060+0000","last_active":"2023-12-20T05:44:22.030880+0000","last_peered":"2023-12-20T05:44:22.030880+0000","last_clean":"2023-12-20T05:44:22.030880+0000","last_became_active":"2023-12-20T05:44:21.036796+0000","last_became_peered":"2023-12-20T05:44:21.036796+0000","last_unstale":"2023-12-20T05:44:22.030880+0000","last_undegraded":"2023-12-20T05:44:22.030880+0000","last_fullsized":"2023-12-20T05:44:22.030880+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:13:08.242870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.28","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.049029+0000","last_change":"2023-12-20T05:44:21.045480+0000","last_active":"2023-12-20T05:44:21.049029+0000","last_peered":"2023-12-20T05:44:21.049029+0000","last_clean":"2023-12-20T05:44:21.049029+0000","last_became_active":"2023-12-20T05:44:21.045220+0000","last_became_peered":"2023-12-20T05:44:21.045220+0000","last_unstale":"2023-12-20T05:44:21.049029+0000","last_undegraded":"2023-12-20T05:44:21.049029+0000","last_fullsized":"2023-12-20T05:44:21.049029+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:51:35.352950+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.27","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.046989+0000","last_change":"2023-12-20T05:44:21.044019+0000","last_active":"2023-12-20T05:44:21.046989+0000","last_peered":"2023-12-20T05:44:21.046989+0000","last_clean":"2023-12-20T05:44:21.046989+0000","last_became_active":"2023-12-20T05:44:21.043725+0000","last_became_peered":"2023-12-20T05:44:21.043725+0000","last_unstale":"2023-12-20T05:44:21.046989+0000","last_undegraded":"2023-12-20T05:44:21.046989+0000","last_fullsized":"2023-12-20T05:44:21.046989+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:29:24.999626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.26","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236017+0000","last_change":"2023-12-20T05:44:21.043908+0000","last_active":"2023-12-20T05:44:22.236017+0000","last_peered":"2023-12-20T05:44:22.236017+0000","last_clean":"2023-12-20T05:44:22.236017+0000","last_became_active":"2023-12-20T05:44:21.043662+0000","last_became_peered":"2023-12-20T05:44:21.043662+0000","last_unstale":"2023-12-20T05:44:22.236017+0000","last_undegraded":"2023-12-20T05:44:22.236017+0000","last_fullsized":"2023-12-20T05:44:22.236017+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:34:14.606009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.25","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.046643+0000","last_change":"2023-12-20T05:44:21.043400+0000","last_active":"2023-12-20T05:44:21.046643+0000","last_peered":"2023-12-20T05:44:21.046643+0000","last_clean":"2023-12-20T05:44:21.046643+0000","last_became_active":"2023-12-20T05:44:21.043144+0000","last_became_peered":"2023-12-20T05:44:21.043144+0000","last_unstale":"2023-12-20T05:44:21.046643+0000","last_undegraded":"2023-12-20T05:44:21.046643+0000","last_fullsized":"2023-12-20T05:44:21.046643+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:25:52.922696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.24","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025059+0000","last_change":"2023-12-20T05:44:21.047039+0000","last_active":"2023-12-20T05:44:22.025059+0000","last_peered":"2023-12-20T05:44:22.025059+0000","last_clean":"2023-12-20T05:44:22.025059+0000","last_became_active":"2023-12-20T05:44:21.046633+0000","last_became_peered":"2023-12-20T05:44:21.046633+0000","last_unstale":"2023-12-20T05:44:22.025059+0000","last_undegraded":"2023-12-20T05:44:22.025059+0000","last_fullsized":"2023-12-20T05:44:22.025059+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:32.056451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.23","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235794+0000","last_change":"2023-12-20T05:44:21.047446+0000","last_active":"2023-12-20T05:44:22.235794+0000","last_peered":"2023-12-20T05:44:22.235794+0000","last_clean":"2023-12-20T05:44:22.235794+0000","last_became_active":"2023-12-20T05:44:21.046827+0000","last_became_peered":"2023-12-20T05:44:21.046827+0000","last_unstale":"2023-12-20T05:44:22.235794+0000","last_undegraded":"2023-12-20T05:44:22.235794+0000","last_fullsized":"2023-12-20T05:44:22.235794+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:45.593289+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.22","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031013+0000","last_change":"2023-12-20T05:44:21.034892+0000","last_active":"2023-12-20T05:44:22.031013+0000","last_peered":"2023-12-20T05:44:22.031013+0000","last_clean":"2023-12-20T05:44:22.031013+0000","last_became_active":"2023-12-20T05:44:21.033742+0000","last_became_peered":"2023-12-20T05:44:21.033742+0000","last_unstale":"2023-12-20T05:44:22.031013+0000","last_undegraded":"2023-12-20T05:44:22.031013+0000","last_fullsized":"2023-12-20T05:44:22.031013+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:05:04.359301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.21","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.040253+0000","last_change":"2023-12-20T05:44:21.037155+0000","last_active":"2023-12-20T05:44:21.040253+0000","last_peered":"2023-12-20T05:44:21.040253+0000","last_clean":"2023-12-20T05:44:21.040253+0000","last_became_active":"2023-12-20T05:44:21.036900+0000","last_became_peered":"2023-12-20T05:44:21.036900+0000","last_unstale":"2023-12-20T05:44:21.040253+0000","last_undegraded":"2023-12-20T05:44:21.040253+0000","last_fullsized":"2023-12-20T05:44:21.040253+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:00:55.118503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.20","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.043284+0000","last_change":"2023-12-20T05:44:21.037120+0000","last_active":"2023-12-20T05:44:21.043284+0000","last_peered":"2023-12-20T05:44:21.043284+0000","last_clean":"2023-12-20T05:44:21.043284+0000","last_became_active":"2023-12-20T05:44:21.036537+0000","last_became_peered":"2023-12-20T05:44:21.036537+0000","last_unstale":"2023-12-20T05:44:21.043284+0000","last_undegraded":"2023-12-20T05:44:21.043284+0000","last_fullsized":"2023-12-20T05:44:21.043284+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:53:22.990765+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.042373+0000","last_change":"2023-12-20T05:44:21.037768+0000","last_active":"2023-12-20T05:44:21.042373+0000","last_peered":"2023-12-20T05:44:21.042373+0000","last_clean":"2023-12-20T05:44:21.042373+0000","last_became_active":"2023-12-20T05:44:21.037476+0000","last_became_peered":"2023-12-20T05:44:21.037476+0000","last_unstale":"2023-12-20T05:44:21.042373+0000","last_undegraded":"2023-12-20T05:44:21.042373+0000","last_fullsized":"2023-12-20T05:44:21.042373+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:13:06.355404+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1f","version":"172'4","reported_seq":14,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045192+0000","last_change":"2023-12-20T05:44:20.630009+0000","last_active":"2023-12-20T05:44:21.045192+0000","last_peered":"2023-12-20T05:44:21.045192+0000","last_clean":"2023-12-20T05:44:21.045192+0000","last_became_active":"2023-12-20T05:44:20.629696+0000","last_became_peered":"2023-12-20T05:44:20.629696+0000","last_unstale":"2023-12-20T05:44:21.045192+0000","last_undegraded":"2023-12-20T05:44:21.045192+0000","last_fullsized":"2023-12-20T05:44:21.045192+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:54:34.396231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.1f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.037816+0000","last_change":"2023-12-20T05:44:21.029096+0000","last_active":"2023-12-20T05:44:21.037816+0000","last_peered":"2023-12-20T05:44:21.037816+0000","last_clean":"2023-12-20T05:44:21.037816+0000","last_became_active":"2023-12-20T05:44:21.028832+0000","last_became_peered":"2023-12-20T05:44:21.028832+0000","last_unstale":"2023-12-20T05:44:21.037816+0000","last_undegraded":"2023-12-20T05:44:21.037816+0000","last_fullsized":"2023-12-20T05:44:21.037816+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:35:42.150416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1e","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.047152+0000","last_change":"2023-12-20T05:44:20.631309+0000","last_active":"2023-12-20T05:44:21.047152+0000","last_peered":"2023-12-20T05:44:21.047152+0000","last_clean":"2023-12-20T05:44:21.047152+0000","last_became_active":"2023-12-20T05:44:20.630799+0000","last_became_peered":"2023-12-20T05:44:20.630799+0000","last_unstale":"2023-12-20T05:44:21.047152+0000","last_undegraded":"2023-12-20T05:44:21.047152+0000","last_fullsized":"2023-12-20T05:44:21.047152+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:10:07.363510+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.1c","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235827+0000","last_change":"2023-12-20T05:44:21.039734+0000","last_active":"2023-12-20T05:44:22.235827+0000","last_peered":"2023-12-20T05:44:22.235827+0000","last_clean":"2023-12-20T05:44:22.235827+0000","last_became_active":"2023-12-20T05:44:21.039468+0000","last_became_peered":"2023-12-20T05:44:21.039468+0000","last_unstale":"2023-12-20T05:44:22.235827+0000","last_undegraded":"2023-12-20T05:44:22.235827+0000","last_fullsized":"2023-12-20T05:44:22.235827+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:21:51.300866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.1d","version":"172'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031690+0000","last_change":"2023-12-20T05:44:20.622284+0000","last_active":"2023-12-20T05:44:22.031690+0000","last_peered":"2023-12-20T05:44:22.031690+0000","last_clean":"2023-12-20T05:44:22.031690+0000","last_became_active":"2023-12-20T05:44:20.622007+0000","last_became_peered":"2023-12-20T05:44:20.622007+0000","last_unstale":"2023-12-20T05:44:22.031690+0000","last_undegraded":"2023-12-20T05:44:22.031690+0000","last_fullsized":"2023-12-20T05:44:22.031690+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:33:02.537140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1d","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.044598+0000","last_change":"2023-12-20T05:44:21.040468+0000","last_active":"2023-12-20T05:44:21.044598+0000","last_peered":"2023-12-20T05:44:21.044598+0000","last_clean":"2023-12-20T05:44:21.044598+0000","last_became_active":"2023-12-20T05:44:21.039122+0000","last_became_peered":"2023-12-20T05:44:21.039122+0000","last_unstale":"2023-12-20T05:44:21.044598+0000","last_undegraded":"2023-12-20T05:44:21.044598+0000","last_fullsized":"2023-12-20T05:44:21.044598+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:18:05.829125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1c","version":"172'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031571+0000","last_change":"2023-12-20T05:44:20.624440+0000","last_active":"2023-12-20T05:44:22.031571+0000","last_peered":"2023-12-20T05:44:22.031571+0000","last_clean":"2023-12-20T05:44:22.031571+0000","last_became_active":"2023-12-20T05:44:20.624130+0000","last_became_peered":"2023-12-20T05:44:20.624130+0000","last_unstale":"2023-12-20T05:44:22.031571+0000","last_undegraded":"2023-12-20T05:44:22.031571+0000","last_fullsized":"2023-12-20T05:44:22.031571+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:57:15.787059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.1a","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.046758+0000","last_change":"2023-12-20T05:44:21.045365+0000","last_active":"2023-12-20T05:44:21.046758+0000","last_peered":"2023-12-20T05:44:21.046758+0000","last_clean":"2023-12-20T05:44:21.046758+0000","last_became_active":"2023-12-20T05:44:21.045118+0000","last_became_peered":"2023-12-20T05:44:21.045118+0000","last_unstale":"2023-12-20T05:44:21.046758+0000","last_undegraded":"2023-12-20T05:44:21.046758+0000","last_fullsized":"2023-12-20T05:44:21.046758+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:56:50.707975+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.1b","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.042642+0000","last_change":"2023-12-20T05:44:20.623265+0000","last_active":"2023-12-20T05:44:21.042642+0000","last_peered":"2023-12-20T05:44:21.042642+0000","last_clean":"2023-12-20T05:44:21.042642+0000","last_became_active":"2023-12-20T05:44:20.623007+0000","last_became_peered":"2023-12-20T05:44:20.623007+0000","last_unstale":"2023-12-20T05:44:21.042642+0000","last_undegraded":"2023-12-20T05:44:21.042642+0000","last_fullsized":"2023-12-20T05:44:21.042642+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:43:03.649161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.1b","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030955+0000","last_change":"2023-12-20T05:44:21.044438+0000","last_active":"2023-12-20T05:44:22.030955+0000","last_peered":"2023-12-20T05:44:22.030955+0000","last_clean":"2023-12-20T05:44:22.030955+0000","last_became_active":"2023-12-20T05:44:21.044166+0000","last_became_peered":"2023-12-20T05:44:21.044166+0000","last_unstale":"2023-12-20T05:44:22.030955+0000","last_undegraded":"2023-12-20T05:44:22.030955+0000","last_fullsized":"2023-12-20T05:44:22.030955+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:50:47.195501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.1a","version":"172'1","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031641+0000","last_change":"2023-12-20T05:44:20.620159+0000","last_active":"2023-12-20T05:44:22.031641+0000","last_peered":"2023-12-20T05:44:22.031641+0000","last_clean":"2023-12-20T05:44:22.031641+0000","last_became_active":"2023-12-20T05:44:20.619841+0000","last_became_peered":"2023-12-20T05:44:20.619841+0000","last_unstale":"2023-12-20T05:44:22.031641+0000","last_undegraded":"2023-12-20T05:44:22.031641+0000","last_fullsized":"2023-12-20T05:44:22.031641+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:35.619524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.18","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.037905+0000","last_change":"2023-12-20T05:44:21.026198+0000","last_active":"2023-12-20T05:44:21.037905+0000","last_peered":"2023-12-20T05:44:21.037905+0000","last_clean":"2023-12-20T05:44:21.037905+0000","last_became_active":"2023-12-20T05:44:21.025916+0000","last_became_peered":"2023-12-20T05:44:21.025916+0000","last_unstale":"2023-12-20T05:44:21.037905+0000","last_undegraded":"2023-12-20T05:44:21.037905+0000","last_fullsized":"2023-12-20T05:44:21.037905+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:22:09.392193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.19","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.023031+0000","last_change":"2023-12-20T05:44:20.625433+0000","last_active":"2023-12-20T05:44:21.023031+0000","last_peered":"2023-12-20T05:44:21.023031+0000","last_clean":"2023-12-20T05:44:21.023031+0000","last_became_active":"2023-12-20T05:44:20.625140+0000","last_became_peered":"2023-12-20T05:44:20.625140+0000","last_unstale":"2023-12-20T05:44:21.023031+0000","last_undegraded":"2023-12-20T05:44:21.023031+0000","last_fullsized":"2023-12-20T05:44:21.023031+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:44:34.085866+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.19","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031479+0000","last_change":"2023-12-20T05:44:21.035024+0000","last_active":"2023-12-20T05:44:22.031479+0000","last_peered":"2023-12-20T05:44:22.031479+0000","last_clean":"2023-12-20T05:44:22.031479+0000","last_became_active":"2023-12-20T05:44:21.033948+0000","last_became_peered":"2023-12-20T05:44:21.033948+0000","last_unstale":"2023-12-20T05:44:22.031479+0000","last_undegraded":"2023-12-20T05:44:22.031479+0000","last_fullsized":"2023-12-20T05:44:22.031479+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:38:22.835946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.18","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030684+0000","last_change":"2023-12-20T05:44:20.628128+0000","last_active":"2023-12-20T05:44:22.030684+0000","last_peered":"2023-12-20T05:44:22.030684+0000","last_clean":"2023-12-20T05:44:22.030684+0000","last_became_active":"2023-12-20T05:44:20.627820+0000","last_became_peered":"2023-12-20T05:44:20.627820+0000","last_unstale":"2023-12-20T05:44:22.030684+0000","last_undegraded":"2023-12-20T05:44:22.030684+0000","last_fullsized":"2023-12-20T05:44:22.030684+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:12.314127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"21.16","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.032002+0000","last_change":"2023-12-20T05:44:21.036737+0000","last_active":"2023-12-20T05:44:22.032002+0000","last_peered":"2023-12-20T05:44:22.032002+0000","last_clean":"2023-12-20T05:44:22.032002+0000","last_became_active":"2023-12-20T05:44:21.035560+0000","last_became_peered":"2023-12-20T05:44:21.035560+0000","last_unstale":"2023-12-20T05:44:22.032002+0000","last_undegraded":"2023-12-20T05:44:22.032002+0000","last_fullsized":"2023-12-20T05:44:22.032002+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:21.786814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.17","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.014870+0000","last_change":"2023-12-20T05:44:20.624728+0000","last_active":"2023-12-20T05:44:21.014870+0000","last_peered":"2023-12-20T05:44:21.014870+0000","last_clean":"2023-12-20T05:44:21.014870+0000","last_became_active":"2023-12-20T05:44:20.624149+0000","last_became_peered":"2023-12-20T05:44:20.624149+0000","last_unstale":"2023-12-20T05:44:21.014870+0000","last_undegraded":"2023-12-20T05:44:21.014870+0000","last_fullsized":"2023-12-20T05:44:21.014870+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:07.682088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.17","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045268+0000","last_change":"2023-12-20T05:44:21.041474+0000","last_active":"2023-12-20T05:44:21.045268+0000","last_peered":"2023-12-20T05:44:21.045268+0000","last_clean":"2023-12-20T05:44:21.045268+0000","last_became_active":"2023-12-20T05:44:21.041208+0000","last_became_peered":"2023-12-20T05:44:21.041208+0000","last_unstale":"2023-12-20T05:44:21.045268+0000","last_undegraded":"2023-12-20T05:44:21.045268+0000","last_fullsized":"2023-12-20T05:44:21.045268+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:33:03.548190+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.16","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.026556+0000","last_change":"2023-12-20T05:44:20.630001+0000","last_active":"2023-12-20T05:44:21.026556+0000","last_peered":"2023-12-20T05:44:21.026556+0000","last_clean":"2023-12-20T05:44:21.026556+0000","last_became_active":"2023-12-20T05:44:20.629682+0000","last_became_peered":"2023-12-20T05:44:20.629682+0000","last_unstale":"2023-12-20T05:44:21.026556+0000","last_undegraded":"2023-12-20T05:44:21.026556+0000","last_fullsized":"2023-12-20T05:44:21.026556+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:20.775764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.15","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.037555+0000","last_change":"2023-12-20T05:44:21.028997+0000","last_active":"2023-12-20T05:44:21.037555+0000","last_peered":"2023-12-20T05:44:21.037555+0000","last_clean":"2023-12-20T05:44:21.037555+0000","last_became_active":"2023-12-20T05:44:21.028644+0000","last_became_peered":"2023-12-20T05:44:21.028644+0000","last_unstale":"2023-12-20T05:44:21.037555+0000","last_undegraded":"2023-12-20T05:44:21.037555+0000","last_fullsized":"2023-12-20T05:44:21.037555+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:30:05.816061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.14","version":"172'6","reported_seq":17,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025169+0000","last_change":"2023-12-20T05:44:20.622279+0000","last_active":"2023-12-20T05:44:22.025169+0000","last_peered":"2023-12-20T05:44:22.025169+0000","last_clean":"2023-12-20T05:44:22.025169+0000","last_became_active":"2023-12-20T05:44:20.621957+0000","last_became_peered":"2023-12-20T05:44:20.621957+0000","last_unstale":"2023-12-20T05:44:22.025169+0000","last_undegraded":"2023-12-20T05:44:22.025169+0000","last_fullsized":"2023-12-20T05:44:22.025169+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:15:05.710477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.12","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.050931+0000","last_change":"2023-12-20T05:44:21.049368+0000","last_active":"2023-12-20T05:44:21.050931+0000","last_peered":"2023-12-20T05:44:21.050931+0000","last_clean":"2023-12-20T05:44:21.050931+0000","last_became_active":"2023-12-20T05:44:21.048230+0000","last_became_peered":"2023-12-20T05:44:21.048230+0000","last_unstale":"2023-12-20T05:44:21.050931+0000","last_undegraded":"2023-12-20T05:44:21.050931+0000","last_fullsized":"2023-12-20T05:44:21.050931+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:52:24.479671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.13","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.047487+0000","last_change":"2023-12-20T05:44:20.619762+0000","last_active":"2023-12-20T05:44:21.047487+0000","last_peered":"2023-12-20T05:44:21.047487+0000","last_clean":"2023-12-20T05:44:21.047487+0000","last_became_active":"2023-12-20T05:44:20.619310+0000","last_became_peered":"2023-12-20T05:44:20.619310+0000","last_unstale":"2023-12-20T05:44:21.047487+0000","last_undegraded":"2023-12-20T05:44:21.047487+0000","last_fullsized":"2023-12-20T05:44:21.047487+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:38:35.619524+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.13","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.037874+0000","last_change":"2023-12-20T05:44:21.027971+0000","last_active":"2023-12-20T05:44:21.037874+0000","last_peered":"2023-12-20T05:44:21.037874+0000","last_clean":"2023-12-20T05:44:21.037874+0000","last_became_active":"2023-12-20T05:44:21.027708+0000","last_became_peered":"2023-12-20T05:44:21.027708+0000","last_unstale":"2023-12-20T05:44:21.037874+0000","last_undegraded":"2023-12-20T05:44:21.037874+0000","last_fullsized":"2023-12-20T05:44:21.037874+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:41:44.845747+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.12","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.035176+0000","last_change":"2023-12-20T05:44:20.625805+0000","last_active":"2023-12-20T05:44:21.035176+0000","last_peered":"2023-12-20T05:44:21.035176+0000","last_clean":"2023-12-20T05:44:21.035176+0000","last_became_active":"2023-12-20T05:44:20.625519+0000","last_became_peered":"2023-12-20T05:44:20.625519+0000","last_unstale":"2023-12-20T05:44:21.035176+0000","last_undegraded":"2023-12-20T05:44:21.035176+0000","last_fullsized":"2023-12-20T05:44:21.035176+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:30:22.493006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.10","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031620+0000","last_change":"2023-12-20T05:44:21.040786+0000","last_active":"2023-12-20T05:44:22.031620+0000","last_peered":"2023-12-20T05:44:22.031620+0000","last_clean":"2023-12-20T05:44:22.031620+0000","last_became_active":"2023-12-20T05:44:21.040439+0000","last_became_peered":"2023-12-20T05:44:21.040439+0000","last_unstale":"2023-12-20T05:44:22.031620+0000","last_undegraded":"2023-12-20T05:44:22.031620+0000","last_fullsized":"2023-12-20T05:44:22.031620+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:52:24.479671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.11","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235740+0000","last_change":"2023-12-20T05:44:20.624836+0000","last_active":"2023-12-20T05:44:22.235740+0000","last_peered":"2023-12-20T05:44:22.235740+0000","last_clean":"2023-12-20T05:44:22.235740+0000","last_became_active":"2023-12-20T05:44:20.624535+0000","last_became_peered":"2023-12-20T05:44:20.624535+0000","last_unstale":"2023-12-20T05:44:22.235740+0000","last_undegraded":"2023-12-20T05:44:22.235740+0000","last_fullsized":"2023-12-20T05:44:22.235740+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:30:22.493006+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.11","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.044333+0000","last_change":"2023-12-20T05:44:21.038160+0000","last_active":"2023-12-20T05:44:21.044333+0000","last_peered":"2023-12-20T05:44:21.044333+0000","last_clean":"2023-12-20T05:44:21.044333+0000","last_became_active":"2023-12-20T05:44:21.037862+0000","last_became_peered":"2023-12-20T05:44:21.037862+0000","last_unstale":"2023-12-20T05:44:21.044333+0000","last_undegraded":"2023-12-20T05:44:21.044333+0000","last_fullsized":"2023-12-20T05:44:21.044333+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:38:05.186489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.10","version":"172'8","reported_seq":18,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.038672+0000","last_change":"2023-12-20T05:44:20.625606+0000","last_active":"2023-12-20T05:44:21.038672+0000","last_peered":"2023-12-20T05:44:21.038672+0000","last_clean":"2023-12-20T05:44:21.038672+0000","last_became_active":"2023-12-20T05:44:20.625282+0000","last_became_peered":"2023-12-20T05:44:20.625282+0000","last_unstale":"2023-12-20T05:44:21.038672+0000","last_undegraded":"2023-12-20T05:44:21.038672+0000","last_fullsized":"2023-12-20T05:44:21.038672+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:55:57.190271+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.e","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.042392+0000","last_change":"2023-12-20T05:44:21.037892+0000","last_active":"2023-12-20T05:44:21.042392+0000","last_peered":"2023-12-20T05:44:21.042392+0000","last_clean":"2023-12-20T05:44:21.042392+0000","last_became_active":"2023-12-20T05:44:21.037570+0000","last_became_peered":"2023-12-20T05:44:21.037570+0000","last_unstale":"2023-12-20T05:44:21.042392+0000","last_undegraded":"2023-12-20T05:44:21.042392+0000","last_fullsized":"2023-12-20T05:44:21.042392+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:33.564209+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.f","version":"172'1","reported_seq":12,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031996+0000","last_change":"2023-12-20T05:44:20.623393+0000","last_active":"2023-12-20T05:44:22.031996+0000","last_peered":"2023-12-20T05:44:22.031996+0000","last_clean":"2023-12-20T05:44:22.031996+0000","last_became_active":"2023-12-20T05:44:20.623037+0000","last_became_peered":"2023-12-20T05:44:20.623037+0000","last_unstale":"2023-12-20T05:44:22.031996+0000","last_undegraded":"2023-12-20T05:44:22.031996+0000","last_fullsized":"2023-12-20T05:44:22.031996+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:04:58.510517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"21.f","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.049417+0000","last_change":"2023-12-20T05:44:21.045716+0000","last_active":"2023-12-20T05:44:21.049417+0000","last_peered":"2023-12-20T05:44:21.049417+0000","last_clean":"2023-12-20T05:44:21.049417+0000","last_became_active":"2023-12-20T05:44:21.043886+0000","last_became_peered":"2023-12-20T05:44:21.043886+0000","last_unstale":"2023-12-20T05:44:21.049417+0000","last_undegraded":"2023-12-20T05:44:21.049417+0000","last_fullsized":"2023-12-20T05:44:21.049417+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:34:53.917983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.e","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.049921+0000","last_change":"2023-12-20T05:44:20.629062+0000","last_active":"2023-12-20T05:44:21.049921+0000","last_peered":"2023-12-20T05:44:21.049921+0000","last_clean":"2023-12-20T05:44:21.049921+0000","last_became_active":"2023-12-20T05:44:20.628718+0000","last_became_peered":"2023-12-20T05:44:20.628718+0000","last_unstale":"2023-12-20T05:44:21.049921+0000","last_undegraded":"2023-12-20T05:44:21.049921+0000","last_fullsized":"2023-12-20T05:44:21.049921+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:53:01.154076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.c","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.050897+0000","last_change":"2023-12-20T05:44:21.049012+0000","last_active":"2023-12-20T05:44:21.050897+0000","last_peered":"2023-12-20T05:44:21.050897+0000","last_clean":"2023-12-20T05:44:21.050897+0000","last_became_active":"2023-12-20T05:44:21.048292+0000","last_became_peered":"2023-12-20T05:44:21.048292+0000","last_unstale":"2023-12-20T05:44:21.050897+0000","last_undegraded":"2023-12-20T05:44:21.050897+0000","last_fullsized":"2023-12-20T05:44:21.050897+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:38:22.835946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.d","version":"172'1","reported_seq":11,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045837+0000","last_change":"2023-12-20T05:44:20.631569+0000","last_active":"2023-12-20T05:44:21.045837+0000","last_peered":"2023-12-20T05:44:21.045837+0000","last_clean":"2023-12-20T05:44:21.045837+0000","last_became_active":"2023-12-20T05:44:20.631233+0000","last_became_peered":"2023-12-20T05:44:20.631233+0000","last_unstale":"2023-12-20T05:44:21.045837+0000","last_undegraded":"2023-12-20T05:44:21.045837+0000","last_fullsized":"2023-12-20T05:44:21.045837+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:42.371279+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"21.d","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235961+0000","last_change":"2023-12-20T05:44:21.048122+0000","last_active":"2023-12-20T05:44:22.235961+0000","last_peered":"2023-12-20T05:44:22.235961+0000","last_clean":"2023-12-20T05:44:22.235961+0000","last_became_active":"2023-12-20T05:44:21.047632+0000","last_became_peered":"2023-12-20T05:44:21.047632+0000","last_unstale":"2023-12-20T05:44:22.235961+0000","last_undegraded":"2023-12-20T05:44:22.235961+0000","last_fullsized":"2023-12-20T05:44:22.235961+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:29:27.313698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.c","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.014802+0000","last_change":"2023-12-20T05:44:20.619580+0000","last_active":"2023-12-20T05:44:21.014802+0000","last_peered":"2023-12-20T05:44:21.014802+0000","last_clean":"2023-12-20T05:44:21.014802+0000","last_became_active":"2023-12-20T05:44:20.619309+0000","last_became_peered":"2023-12-20T05:44:20.619309+0000","last_unstale":"2023-12-20T05:44:21.014802+0000","last_undegraded":"2023-12-20T05:44:21.014802+0000","last_fullsized":"2023-12-20T05:44:21.014802+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:50:46.184451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.a","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031101+0000","last_change":"2023-12-20T05:44:21.038055+0000","last_active":"2023-12-20T05:44:22.031101+0000","last_peered":"2023-12-20T05:44:22.031101+0000","last_clean":"2023-12-20T05:44:22.031101+0000","last_became_active":"2023-12-20T05:44:21.037600+0000","last_became_peered":"2023-12-20T05:44:21.037600+0000","last_unstale":"2023-12-20T05:44:22.031101+0000","last_undegraded":"2023-12-20T05:44:22.031101+0000","last_fullsized":"2023-12-20T05:44:22.031101+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:06:50.945771+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.b","version":"172'2","reported_seq":13,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025213+0000","last_change":"2023-12-20T05:44:20.622381+0000","last_active":"2023-12-20T05:44:22.025213+0000","last_peered":"2023-12-20T05:44:22.025213+0000","last_clean":"2023-12-20T05:44:22.025213+0000","last_became_active":"2023-12-20T05:44:20.622082+0000","last_became_peered":"2023-12-20T05:44:20.622082+0000","last_unstale":"2023-12-20T05:44:22.025213+0000","last_undegraded":"2023-12-20T05:44:22.025213+0000","last_fullsized":"2023-12-20T05:44:22.025213+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:19:20.046614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.b","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025217+0000","last_change":"2023-12-20T05:44:21.033956+0000","last_active":"2023-12-20T05:44:22.025217+0000","last_peered":"2023-12-20T05:44:22.025217+0000","last_clean":"2023-12-20T05:44:22.025217+0000","last_became_active":"2023-12-20T05:44:21.033669+0000","last_became_peered":"2023-12-20T05:44:21.033669+0000","last_unstale":"2023-12-20T05:44:22.025217+0000","last_undegraded":"2023-12-20T05:44:22.025217+0000","last_fullsized":"2023-12-20T05:44:22.025217+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:22.279077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.a","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.020755+0000","last_change":"2023-12-20T05:44:20.630274+0000","last_active":"2023-12-20T05:44:21.020755+0000","last_peered":"2023-12-20T05:44:21.020755+0000","last_clean":"2023-12-20T05:44:21.020755+0000","last_became_active":"2023-12-20T05:44:20.630004+0000","last_became_peered":"2023-12-20T05:44:20.630004+0000","last_unstale":"2023-12-20T05:44:21.020755+0000","last_undegraded":"2023-12-20T05:44:21.020755+0000","last_fullsized":"2023-12-20T05:44:21.020755+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:38:21.824896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.8","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.030671+0000","last_change":"2023-12-20T05:44:21.033516+0000","last_active":"2023-12-20T05:44:22.030671+0000","last_peered":"2023-12-20T05:44:22.030671+0000","last_clean":"2023-12-20T05:44:22.030671+0000","last_became_active":"2023-12-20T05:44:21.033189+0000","last_became_peered":"2023-12-20T05:44:21.033189+0000","last_unstale":"2023-12-20T05:44:22.030671+0000","last_undegraded":"2023-12-20T05:44:22.030671+0000","last_fullsized":"2023-12-20T05:44:22.030671+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:07.321595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"20.9","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.020100+0000","last_change":"2023-12-20T05:44:20.630484+0000","last_active":"2023-12-20T05:44:21.020100+0000","last_peered":"2023-12-20T05:44:21.020100+0000","last_clean":"2023-12-20T05:44:21.020100+0000","last_became_active":"2023-12-20T05:44:20.630163+0000","last_became_peered":"2023-12-20T05:44:20.630163+0000","last_unstale":"2023-12-20T05:44:21.020100+0000","last_undegraded":"2023-12-20T05:44:21.020100+0000","last_fullsized":"2023-12-20T05:44:21.020100+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:34:13.594959+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.9","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025135+0000","last_change":"2023-12-20T05:44:21.031138+0000","last_active":"2023-12-20T05:44:22.025135+0000","last_peered":"2023-12-20T05:44:22.025135+0000","last_clean":"2023-12-20T05:44:22.025135+0000","last_became_active":"2023-12-20T05:44:21.030847+0000","last_became_peered":"2023-12-20T05:44:21.030847+0000","last_unstale":"2023-12-20T05:44:22.025135+0000","last_undegraded":"2023-12-20T05:44:22.025135+0000","last_fullsized":"2023-12-20T05:44:22.025135+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:16:19.834909+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.8","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.033576+0000","last_change":"2023-12-20T05:44:20.620662+0000","last_active":"2023-12-20T05:44:21.033576+0000","last_peered":"2023-12-20T05:44:21.033576+0000","last_clean":"2023-12-20T05:44:21.033576+0000","last_became_active":"2023-12-20T05:44:20.620375+0000","last_became_peered":"2023-12-20T05:44:20.620375+0000","last_unstale":"2023-12-20T05:44:21.033576+0000","last_undegraded":"2023-12-20T05:44:21.033576+0000","last_fullsized":"2023-12-20T05:44:21.033576+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:36:01.179411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.6","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.032074+0000","last_change":"2023-12-20T05:44:21.041404+0000","last_active":"2023-12-20T05:44:22.032074+0000","last_peered":"2023-12-20T05:44:22.032074+0000","last_clean":"2023-12-20T05:44:22.032074+0000","last_became_active":"2023-12-20T05:44:21.040503+0000","last_became_peered":"2023-12-20T05:44:21.040503+0000","last_unstale":"2023-12-20T05:44:22.032074+0000","last_undegraded":"2023-12-20T05:44:22.032074+0000","last_fullsized":"2023-12-20T05:44:22.032074+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:01:26.874334+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.7","version":"172'2","reported_seq":12,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.047571+0000","last_change":"2023-12-20T05:44:20.630226+0000","last_active":"2023-12-20T05:44:21.047571+0000","last_peered":"2023-12-20T05:44:21.047571+0000","last_clean":"2023-12-20T05:44:21.047571+0000","last_became_active":"2023-12-20T05:44:20.629730+0000","last_became_peered":"2023-12-20T05:44:20.629730+0000","last_unstale":"2023-12-20T05:44:21.047571+0000","last_undegraded":"2023-12-20T05:44:21.047571+0000","last_fullsized":"2023-12-20T05:44:21.047571+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:52:23.468622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.7","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.050944+0000","last_change":"2023-12-20T05:44:21.049651+0000","last_active":"2023-12-20T05:44:21.050944+0000","last_peered":"2023-12-20T05:44:21.050944+0000","last_clean":"2023-12-20T05:44:21.050944+0000","last_became_active":"2023-12-20T05:44:21.049339+0000","last_became_peered":"2023-12-20T05:44:21.049339+0000","last_unstale":"2023-12-20T05:44:21.050944+0000","last_undegraded":"2023-12-20T05:44:21.050944+0000","last_fullsized":"2023-12-20T05:44:21.050944+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:50:30.482977+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"20.6","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236111+0000","last_change":"2023-12-20T05:44:20.633197+0000","last_active":"2023-12-20T05:44:22.236111+0000","last_peered":"2023-12-20T05:44:22.236111+0000","last_clean":"2023-12-20T05:44:22.236111+0000","last_became_active":"2023-12-20T05:44:20.632907+0000","last_became_peered":"2023-12-20T05:44:20.632907+0000","last_unstale":"2023-12-20T05:44:22.236111+0000","last_undegraded":"2023-12-20T05:44:22.236111+0000","last_fullsized":"2023-12-20T05:44:22.236111+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:36:01.179411+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.4","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.038060+0000","last_change":"2023-12-20T05:44:21.029664+0000","last_active":"2023-12-20T05:44:21.038060+0000","last_peered":"2023-12-20T05:44:21.038060+0000","last_clean":"2023-12-20T05:44:21.038060+0000","last_became_active":"2023-12-20T05:44:21.028563+0000","last_became_peered":"2023-12-20T05:44:21.028563+0000","last_unstale":"2023-12-20T05:44:21.038060+0000","last_undegraded":"2023-12-20T05:44:21.038060+0000","last_fullsized":"2023-12-20T05:44:21.038060+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:54:19.715671+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"20.5","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236110+0000","last_change":"2023-12-20T05:44:20.620429+0000","last_active":"2023-12-20T05:44:22.236110+0000","last_peered":"2023-12-20T05:44:22.236110+0000","last_clean":"2023-12-20T05:44:22.236110+0000","last_became_active":"2023-12-20T05:44:20.620127+0000","last_became_peered":"2023-12-20T05:44:20.620127+0000","last_unstale":"2023-12-20T05:44:22.236110+0000","last_undegraded":"2023-12-20T05:44:22.236110+0000","last_fullsized":"2023-12-20T05:44:22.236110+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:10:53.550958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.5","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.236022+0000","last_change":"2023-12-20T05:44:21.034830+0000","last_active":"2023-12-20T05:44:22.236022+0000","last_peered":"2023-12-20T05:44:22.236022+0000","last_clean":"2023-12-20T05:44:22.236022+0000","last_became_active":"2023-12-20T05:44:21.034585+0000","last_became_peered":"2023-12-20T05:44:21.034585+0000","last_unstale":"2023-12-20T05:44:22.236022+0000","last_undegraded":"2023-12-20T05:44:22.236022+0000","last_fullsized":"2023-12-20T05:44:22.236022+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:17:56.360135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"20.4","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.235943+0000","last_change":"2023-12-20T05:44:20.621040+0000","last_active":"2023-12-20T05:44:22.235943+0000","last_peered":"2023-12-20T05:44:22.235943+0000","last_clean":"2023-12-20T05:44:22.235943+0000","last_became_active":"2023-12-20T05:44:20.620754+0000","last_became_peered":"2023-12-20T05:44:20.620754+0000","last_unstale":"2023-12-20T05:44:22.235943+0000","last_undegraded":"2023-12-20T05:44:22.235943+0000","last_fullsized":"2023-12-20T05:44:22.235943+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:22.590583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"21.2","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.041108+0000","last_change":"2023-12-20T05:44:21.032210+0000","last_active":"2023-12-20T05:44:21.041108+0000","last_peered":"2023-12-20T05:44:21.041108+0000","last_clean":"2023-12-20T05:44:21.041108+0000","last_became_active":"2023-12-20T05:44:21.031207+0000","last_became_peered":"2023-12-20T05:44:21.031207+0000","last_unstale":"2023-12-20T05:44:21.041108+0000","last_undegraded":"2023-12-20T05:44:21.041108+0000","last_fullsized":"2023-12-20T05:44:21.041108+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:27:26.303679+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"20.3","version":"0'0","reported_seq":11,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025305+0000","last_change":"2023-12-20T05:44:20.627350+0000","last_active":"2023-12-20T05:44:22.025305+0000","last_peered":"2023-12-20T05:44:22.025305+0000","last_clean":"2023-12-20T05:44:22.025305+0000","last_became_active":"2023-12-20T05:44:20.627054+0000","last_became_peered":"2023-12-20T05:44:20.627054+0000","last_unstale":"2023-12-20T05:44:22.025305+0000","last_undegraded":"2023-12-20T05:44:22.025305+0000","last_fullsized":"2023-12-20T05:44:22.025305+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:16.134894+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"21.3","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031809+0000","last_change":"2023-12-20T05:44:21.045238+0000","last_active":"2023-12-20T05:44:22.031809+0000","last_peered":"2023-12-20T05:44:22.031809+0000","last_clean":"2023-12-20T05:44:22.031809+0000","last_became_active":"2023-12-20T05:44:21.043982+0000","last_became_peered":"2023-12-20T05:44:21.043982+0000","last_unstale":"2023-12-20T05:44:22.031809+0000","last_undegraded":"2023-12-20T05:44:22.031809+0000","last_fullsized":"2023-12-20T05:44:22.031809+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:34:14.606009+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.2","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.020755+0000","last_change":"2023-12-20T05:44:20.630028+0000","last_active":"2023-12-20T05:44:21.020755+0000","last_peered":"2023-12-20T05:44:21.020755+0000","last_clean":"2023-12-20T05:44:21.020755+0000","last_became_active":"2023-12-20T05:44:20.629534+0000","last_became_peered":"2023-12-20T05:44:20.629534+0000","last_unstale":"2023-12-20T05:44:21.020755+0000","last_undegraded":"2023-12-20T05:44:21.020755+0000","last_fullsized":"2023-12-20T05:44:21.020755+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:50:29.471927+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"21.0","version":"0'0","reported_seq":9,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.044658+0000","last_change":"2023-12-20T05:44:21.039736+0000","last_active":"2023-12-20T05:44:21.044658+0000","last_peered":"2023-12-20T05:44:21.044658+0000","last_clean":"2023-12-20T05:44:21.044658+0000","last_became_active":"2023-12-20T05:44:21.039419+0000","last_became_peered":"2023-12-20T05:44:21.039419+0000","last_unstale":"2023-12-20T05:44:21.044658+0000","last_undegraded":"2023-12-20T05:44:21.044658+0000","last_fullsized":"2023-12-20T05:44:21.044658+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:13.325177+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"20.1","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.033592+0000","last_change":"2023-12-20T05:44:20.621499+0000","last_active":"2023-12-20T05:44:21.033592+0000","last_peered":"2023-12-20T05:44:21.033592+0000","last_clean":"2023-12-20T05:44:21.033592+0000","last_became_active":"2023-12-20T05:44:20.621173+0000","last_became_peered":"2023-12-20T05:44:20.621173+0000","last_unstale":"2023-12-20T05:44:21.033592+0000","last_undegraded":"2023-12-20T05:44:21.033592+0000","last_fullsized":"2023-12-20T05:44:21.033592+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:04:58.510517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"21.1","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.031587+0000","last_change":"2023-12-20T05:44:21.028956+0000","last_active":"2023-12-20T05:44:22.031587+0000","last_peered":"2023-12-20T05:44:22.031587+0000","last_clean":"2023-12-20T05:44:22.031587+0000","last_became_active":"2023-12-20T05:44:21.028645+0000","last_became_peered":"2023-12-20T05:44:21.028645+0000","last_unstale":"2023-12-20T05:44:22.031587+0000","last_undegraded":"2023-12-20T05:44:22.031587+0000","last_fullsized":"2023-12-20T05:44:22.031587+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:38:05.186489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"20.0","version":"0'0","reported_seq":10,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.014441+0000","last_change":"2023-12-20T05:44:20.631124+0000","last_active":"2023-12-20T05:44:21.014441+0000","last_peered":"2023-12-20T05:44:21.014441+0000","last_clean":"2023-12-20T05:44:21.014441+0000","last_became_active":"2023-12-20T05:44:20.630811+0000","last_became_peered":"2023-12-20T05:44:20.630811+0000","last_unstale":"2023-12-20T05:44:21.014441+0000","last_undegraded":"2023-12-20T05:44:21.014441+0000","last_fullsized":"2023-12-20T05:44:21.014441+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:30:13.510051+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"21.14","version":"0'0","reported_seq":10,"reported_epoch":173,"state":"active+clean","last_fresh":"2023-12-20T05:44:22.025115+0000","last_change":"2023-12-20T05:44:21.033115+0000","last_active":"2023-12-20T05:44:22.025115+0000","last_peered":"2023-12-20T05:44:22.025115+0000","last_clean":"2023-12-20T05:44:22.025115+0000","last_became_active":"2023-12-20T05:44:21.032789+0000","last_became_peered":"2023-12-20T05:44:21.032789+0000","last_unstale":"2023-12-20T05:44:22.025115+0000","last_undegraded":"2023-12-20T05:44:22.025115+0000","last_fullsized":"2023-12-20T05:44:22.025115+0000","mapping_epoch":171,"log_start":"0'0","ondisk_log_start":"0'0","created":171,"last_epoch_clean":172,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:20.596103+0000","last_clean_scrub_stamp":"2023-12-20T05:44:20.596103+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T05:48:08.343570+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"20.15","version":"172'3","reported_seq":13,"reported_epoch":172,"state":"active+clean","last_fresh":"2023-12-20T05:44:21.045638+0000","last_change":"2023-12-20T05:44:20.620751+0000","last_active":"2023-12-20T05:44:21.045638+0000","last_peered":"2023-12-20T05:44:21.045638+0000","last_clean":"2023-12-20T05:44:21.045638+0000","last_became_active":"2023-12-20T05:44:20.620461+0000","last_became_peered":"2023-12-20T05:44:20.620461+0000","last_unstale":"2023-12-20T05:44:21.045638+0000","last_undegraded":"2023-12-20T05:44:21.045638+0000","last_fullsized":"2023-12-20T05:44:21.045638+0000","mapping_epoch":170,"log_start":"0'0","ondisk_log_start":"0'0","created":170,"last_epoch_clean":171,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T05:44:19.585053+0000","last_clean_scrub_stamp":"2023-12-20T05:44:19.585053+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:10:22.590583+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"161'3146","reported_seq":3756,"reported_epoch":171,"state":"active+clean","last_fresh":"2023-12-20T05:44:20.620997+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T05:44:20.620997+0000","last_peered":"2023-12-20T05:44:20.620997+0000","last_clean":"2023-12-20T05:44:20.620997+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T05:44:20.620997+0000","last_undegraded":"2023-12-20T05:44:20.620997+0000","last_fullsized":"2023-12-20T05:44:20.620997+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3146,"log_dups_size":0,"ondisk_log_size":3146,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1405,"num_read_kb":2819,"num_write":421,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":21,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":20,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1405,"num_read_kb":2819,"num_write":421,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3146,"ondisk_log_size":3146,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575755,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":37828,"kb_used_data":2684,"kb_used_omap":0,"kb_used_meta":35136,"kb_avail":94334012,"statfs":{"total":96636764160,"available":96598028288,"internally_reserved":0,"allocated":2748416,"data_stored":1621029,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":35979264},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.67700000000000005,"5min":0.72599999999999998,"15min":0.73299999999999998},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82199999999999995,"5min":0.98099999999999998,"15min":1.133},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.73699999999999999,"15min":0.73899999999999999},"min":{"1min":0.41999999999999998,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":0.95599999999999996,"5min":0.95599999999999996,"15min":1.03},"last":0.67700000000000005}]},{"osd":1,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.73199999999999998,"15min":0.74099999999999999},"min":{"1min":0.54800000000000004,"5min":0.48099999999999998,"15min":0.45200000000000001},"max":{"1min":1.2,"5min":1.2,"15min":1.2},"last":0.75800000000000001},{"interface":"front","average":{"1min":0.75,"5min":0.76000000000000001,"15min":0.753},"min":{"1min":0.49199999999999999,"5min":0.49199999999999999,"15min":0.47699999999999998},"max":{"1min":1.2330000000000001,"5min":1.2330000000000001,"15min":1.2330000000000001},"last":0.59099999999999997}]},{"osd":2,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.47399999999999998,"5min":0.47099999999999997,"15min":0.45500000000000002},"max":{"1min":0.93700000000000006,"5min":1.036,"15min":1.0589999999999999},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.75900000000000001,"15min":0.76800000000000002},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.47199999999999998},"max":{"1min":1.0609999999999999,"5min":1.0609999999999999,"15min":2.2160000000000002},"last":0.70199999999999996}]},{"osd":3,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.749,"15min":0.753},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.496},"max":{"1min":1.109,"5min":1.109,"15min":2.6829999999999998},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.82999999999999996,"5min":0.81299999999999994,"15min":0.79800000000000004},"min":{"1min":0.58599999999999997,"5min":0.50600000000000001,"15min":0.49299999999999999},"max":{"1min":1.264,"5min":1.264,"15min":2.1259999999999999},"last":0.63}]},{"osd":4,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.59699999999999998,"15min":0.60199999999999998},"min":{"1min":0.45700000000000002,"5min":0.36499999999999999,"15min":0.36499999999999999},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":1.8180000000000001},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.67400000000000004,"5min":0.63300000000000001,"15min":0.64100000000000001},"min":{"1min":0.44500000000000001,"5min":0.40799999999999997,"15min":0.40200000000000002},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":2.7370000000000001},"last":0.65700000000000003}]},{"osd":5,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.69899999999999995,"5min":0.65800000000000003,"15min":0.66800000000000004},"min":{"1min":0.48599999999999999,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":2.27},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.68500000000000005,"15min":0.68999999999999995},"min":{"1min":0.52200000000000002,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.095,"5min":1.095,"15min":1.095},"last":0.74099999999999999}]},{"osd":6,"last update":"Wed Dec 20 05:43:46 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.755,"15min":0.752},"min":{"1min":0.53900000000000003,"5min":0.47299999999999998,"15min":0.42699999999999999},"max":{"1min":1.147,"5min":1.147,"15min":1.1699999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.745,"5min":0.71099999999999997,"15min":0.71199999999999997},"min":{"1min":0.45800000000000002,"5min":0.45400000000000001,"15min":0.45200000000000001},"max":{"1min":0.98799999999999999,"5min":1.006,"15min":1.1850000000000001},"last":0.77000000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575755,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47108,"kb_used_data":2108,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324732,"statfs":{"total":96636764160,"available":96588525568,"internally_reserved":0,"allocated":2158592,"data_stored":1030155,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.73499999999999999,"15min":0.748},"min":{"1min":0.38200000000000001,"5min":0.38200000000000001,"15min":0.36899999999999999},"max":{"1min":0.98699999999999999,"5min":1,"15min":1.093},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.76900000000000002,"5min":0.75,"15min":0.75700000000000001},"min":{"1min":0.47499999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.006,"5min":1.0189999999999999,"15min":1.101},"last":0.52900000000000003}]},{"osd":1,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.73399999999999999,"15min":0.748},"min":{"1min":0.41799999999999998,"5min":0.41799999999999998,"15min":0.41799999999999998},"max":{"1min":0.88800000000000001,"5min":1.002,"15min":1.03},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.749,"15min":0.76800000000000002},"min":{"1min":0.45800000000000002,"5min":0.45800000000000002,"15min":0.45800000000000002},"max":{"1min":0.97499999999999998,"5min":0.97499999999999998,"15min":1.038},"last":0.77300000000000002}]},{"osd":2,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72499999999999998,"5min":0.76700000000000002,"15min":0.77800000000000002},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.0329999999999999,"5min":1.048,"15min":1.0960000000000001},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.76800000000000002,"15min":0.78000000000000003},"min":{"1min":0.48399999999999999,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.94199999999999995,"5min":1.0369999999999999,"15min":1.083},"last":0.64600000000000002}]},{"osd":3,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.78000000000000003,"15min":0.79100000000000004},"min":{"1min":0.46899999999999997,"5min":0.46899999999999997,"15min":0.46899999999999997},"max":{"1min":0.995,"5min":0.995,"15min":1.0629999999999999},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.76300000000000001,"15min":0.77700000000000002},"min":{"1min":0.48199999999999998,"5min":0.48199999999999998,"15min":0.48199999999999998},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.1020000000000001},"last":0.61599999999999999}]},{"osd":4,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.65900000000000003,"15min":0.64300000000000002},"min":{"1min":0.48099999999999998,"5min":0.40799999999999997,"15min":0.36599999999999999},"max":{"1min":1.0289999999999999,"5min":1.0289999999999999,"15min":1.0289999999999999},"last":0.84799999999999998},{"interface":"front","average":{"1min":0.73299999999999998,"5min":0.67500000000000004,"15min":0.66500000000000004},"min":{"1min":0.41699999999999998,"5min":0.41699999999999998,"15min":0.40999999999999998},"max":{"1min":1.046,"5min":1.048,"15min":1.048},"last":0.72899999999999998}]},{"osd":5,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.66100000000000003,"15min":0.65800000000000003},"min":{"1min":0.42799999999999999,"5min":0.42799999999999999,"15min":0.41799999999999998},"max":{"1min":1.014,"5min":1.014,"15min":1.0429999999999999},"last":0.56100000000000005},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.72499999999999998,"15min":0.71899999999999997},"min":{"1min":0.47799999999999998,"5min":0.46700000000000003,"15min":0.46700000000000003},"max":{"1min":1.0529999999999999,"5min":1.0529999999999999,"15min":1.0529999999999999},"last":0.874}]},{"osd":7,"last update":"Wed Dec 20 05:44:02 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.72599999999999998,"15min":0.73599999999999999},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.93100000000000005,"5min":1.095,"15min":1.095},"last":0.58299999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73299999999999998,"15min":0.73599999999999999},"min":{"1min":0.57299999999999995,"5min":0.49299999999999999,"15min":0.49099999999999999},"max":{"1min":0.95899999999999996,"5min":1.0049999999999999,"15min":1.105},"last":0.754}]}]},{"osd":5,"up_from":13,"seq":55834575756,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29052,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342788,"statfs":{"total":96636764160,"available":96607014912,"internally_reserved":0,"allocated":2150400,"data_stored":1030031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.70899999999999996,"5min":0.72299999999999998,"15min":0.73299999999999998},"min":{"1min":0.40400000000000003,"5min":0.38100000000000001,"15min":0.38100000000000001},"max":{"1min":1.032,"5min":1.032,"15min":1.032},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.752,"15min":0.75900000000000001},"min":{"1min":0.46200000000000002,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":1.119,"5min":1.145,"15min":1.145},"last":0.52600000000000002}]},{"osd":1,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.749,"15min":0.751},"min":{"1min":0.497,"5min":0.44900000000000001,"15min":0.42799999999999999},"max":{"1min":1.002,"5min":1.002,"15min":1.016},"last":0.505},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.78200000000000003,"15min":0.76800000000000002},"min":{"1min":0.39500000000000002,"5min":0.39500000000000002,"15min":0.39500000000000002},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.80200000000000005,"5min":0.79500000000000004,"15min":0.77400000000000002},"min":{"1min":0.55800000000000005,"5min":0.48199999999999998,"15min":0.44700000000000001},"max":{"1min":1.048,"5min":1.204,"15min":1.204},"last":0.56399999999999995},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.76600000000000001,"15min":0.76600000000000001},"min":{"1min":0.60199999999999998,"5min":0.497,"15min":0.49399999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.66600000000000004}]},{"osd":3,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.79300000000000004,"15min":0.78500000000000003},"min":{"1min":0.54700000000000004,"5min":0.52300000000000002,"15min":0.503},"max":{"1min":1.012,"5min":1.0880000000000001,"15min":1.0880000000000001},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.81899999999999995,"5min":0.81100000000000005,"15min":0.79100000000000004},"min":{"1min":0.68300000000000005,"5min":0.56699999999999995,"15min":0.53300000000000003},"max":{"1min":1.097,"5min":1.107,"15min":1.107},"last":0.64800000000000002}]},{"osd":4,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.66800000000000004,"15min":0.64300000000000002},"min":{"1min":0.46700000000000003,"5min":0.436,"15min":0.42999999999999999},"max":{"1min":0.98099999999999998,"5min":1.167,"15min":1.167},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.73499999999999999,"5min":0.67000000000000004,"15min":0.64600000000000002},"min":{"1min":0.435,"5min":0.435,"15min":0.39100000000000001},"max":{"1min":0.94699999999999995,"5min":1.008,"15min":1.008},"last":0.81999999999999995}]},{"osd":6,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.82399999999999995,"5min":0.73199999999999998,"15min":0.70199999999999996},"min":{"1min":0.53600000000000003,"5min":0.48899999999999999,"15min":0.435},"max":{"1min":1.014,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.75,"5min":0.66500000000000004,"15min":0.64200000000000002},"min":{"1min":0.45100000000000001,"5min":0.43099999999999999,"15min":0.40999999999999998},"max":{"1min":0.92100000000000004,"5min":0.98999999999999999,"15min":1.085},"last":0.54800000000000004}]},{"osd":7,"last update":"Wed Dec 20 05:43:52 2023","interfaces":[{"interface":"back","average":{"1min":0.78000000000000003,"5min":0.72999999999999998,"15min":0.70699999999999996},"min":{"1min":0.504,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.0429999999999999,"5min":1.073,"15min":1.1020000000000001},"last":0.77400000000000002},{"interface":"front","average":{"1min":0.85099999999999998,"5min":0.80200000000000005,"15min":0.76800000000000002},"min":{"1min":0.57399999999999995,"5min":0.52200000000000002,"15min":0.48199999999999998},"max":{"1min":1.0229999999999999,"5min":1.1299999999999999,"15min":1.1299999999999999},"last":0.70999999999999996}]}]},{"osd":4,"up_from":13,"seq":55834575757,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":63492,"kb_used_data":2108,"kb_used_omap":0,"kb_used_meta":61376,"kb_avail":94308348,"statfs":{"total":96636764160,"available":96571748352,"internally_reserved":0,"allocated":2158592,"data_stored":1036337,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":62849024},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70399999999999996,"5min":0.74199999999999999,"15min":0.745},"min":{"1min":0.40799999999999997,"5min":0.40799999999999997,"15min":0.34399999999999997},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.75,"15min":0.73799999999999999},"min":{"1min":0.57299999999999995,"5min":0.47599999999999998,"15min":0.35699999999999998},"max":{"1min":2.617,"5min":2.617,"15min":2.617},"last":0.52500000000000002}]},{"osd":1,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.748,"15min":0.75600000000000001},"min":{"1min":0.48699999999999999,"5min":0.433,"15min":0.433},"max":{"1min":1.0249999999999999,"5min":1.077,"15min":1.137},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.75,"15min":0.75700000000000001},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46100000000000002},"max":{"1min":1.2310000000000001,"5min":1.2310000000000001,"15min":1.2310000000000001},"last":0.68700000000000006}]},{"osd":2,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.80700000000000005,"5min":0.76500000000000001,"15min":0.76600000000000001},"min":{"1min":0.44800000000000001,"5min":0.44800000000000001,"15min":0.44800000000000001},"max":{"1min":2.125,"5min":2.125,"15min":2.125},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.749,"5min":0.78100000000000003,"15min":0.77800000000000002},"min":{"1min":0.56599999999999995,"5min":0.56599999999999995,"15min":0.51900000000000002},"max":{"1min":1.137,"5min":1.137,"15min":1.137},"last":0.60099999999999998}]},{"osd":3,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.75600000000000001,"15min":0.747},"min":{"1min":0.48999999999999999,"5min":0.48799999999999999,"15min":0.45500000000000002},"max":{"1min":1.363,"5min":1.363,"15min":1.363},"last":0.624},{"interface":"front","average":{"1min":0.95199999999999996,"5min":0.79800000000000004,"15min":0.79200000000000004},"min":{"1min":0.58699999999999997,"5min":0.51500000000000001,"15min":0.47999999999999998},"max":{"1min":3.7770000000000001,"5min":3.7770000000000001,"15min":3.7770000000000001},"last":0.71899999999999997}]},{"osd":5,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77300000000000002,"5min":0.64000000000000001,"15min":0.621},"min":{"1min":0.39900000000000002,"5min":0.38400000000000001,"15min":0.38400000000000001},"max":{"1min":2.895,"5min":2.895,"15min":2.895},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.629,"15min":0.621},"min":{"1min":0.41199999999999998,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":2.7029999999999998,"5min":2.7029999999999998,"15min":2.7029999999999998},"last":0.67600000000000005}]},{"osd":6,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.84799999999999998,"5min":0.68700000000000006,"15min":0.65800000000000003},"min":{"1min":0.47199999999999998,"5min":0.45400000000000001,"15min":0.40100000000000002},"max":{"1min":3.8279999999999998,"5min":3.8279999999999998,"15min":3.8279999999999998},"last":0.77700000000000002},{"interface":"front","average":{"1min":0.76300000000000001,"5min":0.72199999999999998,"15min":0.70499999999999996},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.2589999999999999,"5min":1.2589999999999999,"15min":1.2589999999999999},"last":0.77100000000000002}]},{"osd":7,"last update":"Wed Dec 20 05:43:27 2023","interfaces":[{"interface":"back","average":{"1min":0.79000000000000004,"5min":0.70499999999999996,"15min":0.68600000000000005},"min":{"1min":0.505,"5min":0.44700000000000001,"15min":0.436},"max":{"1min":2.6680000000000001,"5min":2.6680000000000001,"15min":2.6680000000000001},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.93700000000000006,"5min":0.77800000000000002,"15min":0.749},"min":{"1min":0.56100000000000005,"5min":0.50800000000000001,"15min":0.47099999999999997},"max":{"1min":3.8690000000000002,"5min":3.8690000000000002,"15min":3.8690000000000002},"last":0.82299999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575756,"num_pgs":5,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47100,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324740,"statfs":{"total":96636764160,"available":96588533760,"internally_reserved":0,"allocated":2150400,"data_stored":1030099,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.48799999999999999,"5min":0.46999999999999997,"15min":0.46300000000000002},"min":{"1min":0.27000000000000002,"5min":0.27000000000000002,"15min":0.27000000000000002},"max":{"1min":0.70699999999999996,"5min":0.70699999999999996,"15min":0.88800000000000001},"last":0.29599999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.52200000000000002,"15min":0.51900000000000002},"min":{"1min":0.317,"5min":0.308,"15min":0.29799999999999999},"max":{"1min":0.73699999999999999,"5min":0.75900000000000001,"15min":0.876},"last":0.45600000000000002}]},{"osd":1,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.49099999999999999,"15min":0.47499999999999998},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.30199999999999999},"max":{"1min":1.4299999999999999,"5min":1.4299999999999999,"15min":1.4299999999999999},"last":0.501},{"interface":"front","average":{"1min":0.54700000000000004,"5min":0.57599999999999996,"15min":0.57299999999999995},"min":{"1min":0.33100000000000002,"5min":0.33100000000000002,"15min":0.33100000000000002},"max":{"1min":0.93600000000000005,"5min":0.93600000000000005,"15min":0.93600000000000005},"last":0.40600000000000003}]},{"osd":2,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.60799999999999998,"15min":0.59599999999999997},"min":{"1min":0.45100000000000001,"5min":0.44700000000000001,"15min":0.40400000000000003},"max":{"1min":1.3700000000000001,"5min":1.3700000000000001,"15min":1.3700000000000001},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.57399999999999995,"5min":0.57699999999999996,"15min":0.56499999999999995},"min":{"1min":0.33500000000000002,"5min":0.33500000000000002,"15min":0.33500000000000002},"max":{"1min":0.96199999999999997,"5min":0.96199999999999997,"15min":0.96199999999999997},"last":0.42799999999999999}]},{"osd":4,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.81399999999999995,"15min":0.81899999999999995},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55200000000000005},"max":{"1min":1.4059999999999999,"5min":1.4059999999999999,"15min":1.4059999999999999},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.96799999999999997,"5min":0.86599999999999999,"15min":0.83399999999999996},"min":{"1min":0.54800000000000004,"5min":0.52600000000000002,"15min":0.52600000000000002},"max":{"1min":4.2800000000000002,"5min":4.2800000000000002,"15min":4.2800000000000002},"last":0.78700000000000003}]},{"osd":5,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.96799999999999997,"5min":0.85999999999999999,"15min":0.84699999999999998},"min":{"1min":0.54400000000000004,"5min":0.54400000000000004,"15min":0.496},"max":{"1min":4.3259999999999996,"5min":4.3259999999999996,"15min":4.3259999999999996},"last":0.83099999999999996},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.83199999999999996,"15min":0.81699999999999995},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.50700000000000001},"max":{"1min":1.1220000000000001,"5min":1.232,"15min":1.232},"last":0.76600000000000001}]},{"osd":6,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.83499999999999996,"15min":0.83099999999999996},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.53200000000000003},"max":{"1min":2.153,"5min":2.153,"15min":2.153},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.89500000000000002,"5min":0.86399999999999999,"15min":0.85399999999999998},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":1.917,"5min":1.917,"15min":1.917},"last":0.89000000000000001}]},{"osd":7,"last update":"Wed Dec 20 05:43:55 2023","interfaces":[{"interface":"back","average":{"1min":0.80600000000000005,"5min":0.83999999999999997,"15min":0.83399999999999996},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48699999999999999},"max":{"1min":1.198,"5min":1.383,"15min":1.383},"last":0.80600000000000005},{"interface":"front","average":{"1min":0.84199999999999997,"5min":0.85499999999999998,"15min":0.86199999999999999},"min":{"1min":0.63400000000000001,"5min":0.63400000000000001,"15min":0.56699999999999995},"max":{"1min":1.5720000000000001,"5min":1.5720000000000001,"15min":1.5720000000000001},"last":0.873}]}]},{"osd":2,"up_from":13,"seq":55834575755,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47112,"kb_used_data":2112,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324728,"statfs":{"total":96636764160,"available":96588521472,"internally_reserved":0,"allocated":2162688,"data_stored":1031643,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46600000000000003,"15min":0.46300000000000002},"min":{"1min":0.26700000000000002,"5min":0.26700000000000002,"15min":0.26700000000000002},"max":{"1min":1.1120000000000001,"5min":1.1120000000000001,"15min":1.1120000000000001},"last":0.39600000000000002},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.52200000000000002,"15min":0.51500000000000001},"min":{"1min":0.29099999999999998,"5min":0.29099999999999998,"15min":0.29099999999999998},"max":{"1min":0.82199999999999995,"5min":0.82199999999999995,"15min":0.89700000000000002},"last":0.373}]},{"osd":1,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.50900000000000001,"5min":0.503,"15min":0.48999999999999999},"min":{"1min":0.33900000000000002,"5min":0.33900000000000002,"15min":0.32700000000000001},"max":{"1min":1.141,"5min":1.141,"15min":1.141},"last":0.32300000000000001},{"interface":"front","average":{"1min":0.57699999999999996,"5min":0.57699999999999996,"15min":0.56100000000000005},"min":{"1min":0.372,"5min":0.372,"15min":0.34699999999999998},"max":{"1min":1.2929999999999999,"5min":1.2929999999999999,"15min":1.2929999999999999},"last":0.53500000000000003}]},{"osd":3,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.66400000000000003,"5min":0.65000000000000002,"15min":0.64100000000000001},"min":{"1min":0.317,"5min":0.317,"15min":0.317},"max":{"1min":2.0619999999999998,"5min":2.0619999999999998,"15min":2.0619999999999998},"last":0.44400000000000001},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.69099999999999995,"15min":0.68400000000000005},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.442},"max":{"1min":2.29,"5min":2.29,"15min":2.29},"last":0.45600000000000002}]},{"osd":4,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.88800000000000001,"5min":0.83799999999999997,"15min":0.81399999999999995},"min":{"1min":0.54500000000000004,"5min":0.54500000000000004,"15min":0.50900000000000001},"max":{"1min":1.8440000000000001,"5min":1.8440000000000001,"15min":1.8440000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.86299999999999999,"5min":0.83099999999999996,"15min":0.80900000000000005},"min":{"1min":0.53100000000000003,"5min":0.52600000000000002,"15min":0.48199999999999998},"max":{"1min":1.9790000000000001,"5min":1.9790000000000001,"15min":1.9790000000000001},"last":0.68300000000000005}]},{"osd":5,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.89200000000000002,"5min":0.83599999999999997,"15min":0.82399999999999995},"min":{"1min":0.54100000000000004,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":2.3130000000000002,"5min":2.3130000000000002,"15min":2.3130000000000002},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.88100000000000001,"5min":0.85299999999999998,"15min":0.83499999999999996},"min":{"1min":0.623,"5min":0.58699999999999997,"15min":0.57999999999999996},"max":{"1min":1.8839999999999999,"5min":1.8839999999999999,"15min":1.8839999999999999},"last":0.59199999999999997}]},{"osd":6,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.876,"5min":0.85999999999999999,"15min":0.83299999999999996},"min":{"1min":0.52100000000000002,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":2.0249999999999999,"5min":2.0249999999999999,"15min":2.0249999999999999},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.91700000000000004,"5min":0.86799999999999999,"15min":0.84199999999999997},"min":{"1min":0.58999999999999997,"5min":0.54200000000000004,"15min":0.54200000000000004},"max":{"1min":2.2730000000000001,"5min":2.2730000000000001,"15min":2.2730000000000001},"last":0.51700000000000002}]},{"osd":7,"last update":"Wed Dec 20 05:43:45 2023","interfaces":[{"interface":"back","average":{"1min":0.92500000000000004,"5min":0.876,"15min":0.86399999999999999},"min":{"1min":0.60299999999999998,"5min":0.54800000000000004,"15min":0.51800000000000002},"max":{"1min":2.0819999999999999,"5min":2.0819999999999999,"15min":2.0819999999999999},"last":0.74199999999999999},{"interface":"front","average":{"1min":0.93999999999999995,"5min":0.91600000000000004,"15min":0.89400000000000002},"min":{"1min":0.60399999999999998,"5min":0.60399999999999998,"15min":0.60399999999999998},"max":{"1min":2.1179999999999999,"5min":2.1179999999999999,"15min":2.1179999999999999},"last":0.59999999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575757,"num_pgs":25,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":55292,"kb_used_data":2100,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94316548,"statfs":{"total":96636764160,"available":96580145152,"internally_reserved":0,"allocated":2150400,"data_stored":1030591,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.48199999999999998,"5min":0.46899999999999997,"15min":0.46700000000000003},"min":{"1min":0.318,"5min":0.311,"15min":0.29499999999999998},"max":{"1min":1.0900000000000001,"5min":1.0900000000000001,"15min":1.0900000000000001},"last":0.47099999999999997},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.53600000000000003,"15min":0.53300000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.32500000000000001},"max":{"1min":1.321,"5min":1.321,"15min":1.321},"last":0.57899999999999996}]},{"osd":2,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.52100000000000002,"5min":0.53500000000000003,"15min":0.53300000000000003},"min":{"1min":0.35299999999999998,"5min":0.35299999999999998,"15min":0.33000000000000002},"max":{"1min":1.0489999999999999,"5min":1.0489999999999999,"15min":1.0489999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.56499999999999995,"5min":0.57099999999999995,"15min":0.58099999999999996},"min":{"1min":0.373,"5min":0.373,"15min":0.35399999999999998},"max":{"1min":1.234,"5min":1.234,"15min":1.234},"last":0.44500000000000001}]},{"osd":3,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.63400000000000001,"15min":0.63200000000000001},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":1.6259999999999999,"5min":1.6259999999999999,"15min":1.6259999999999999},"last":0.501},{"interface":"front","average":{"1min":0.52600000000000002,"5min":0.55600000000000005,"15min":0.55300000000000005},"min":{"1min":0.32100000000000001,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":1.0369999999999999,"5min":1.0369999999999999,"15min":1.0369999999999999},"last":0.56399999999999995}]},{"osd":4,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79100000000000004,"5min":0.81499999999999995,"15min":0.80800000000000005},"min":{"1min":0.53600000000000003,"5min":0.49199999999999999,"15min":0.49199999999999999},"max":{"1min":1.393,"5min":1.393,"15min":1.393},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.78100000000000003,"5min":0.83299999999999996,"15min":0.82599999999999996},"min":{"1min":0.54300000000000004,"5min":0.54300000000000004,"15min":0.54300000000000004},"max":{"1min":1.284,"5min":1.284,"15min":1.294},"last":0.82399999999999995}]},{"osd":5,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82799999999999996,"15min":0.81100000000000005},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.54000000000000004},"max":{"1min":1.131,"5min":1.2010000000000001,"15min":1.2709999999999999},"last":0.71299999999999997},{"interface":"front","average":{"1min":0.80900000000000005,"5min":0.85099999999999998,"15min":0.82599999999999996},"min":{"1min":0.61699999999999999,"5min":0.61699999999999999,"15min":0.54500000000000004},"max":{"1min":1.4450000000000001,"5min":1.4450000000000001,"15min":1.4450000000000001},"last":0.89400000000000002}]},{"osd":6,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.79800000000000004,"5min":0.85999999999999999,"15min":0.84299999999999997},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.59299999999999997},"max":{"1min":1.357,"5min":1.357,"15min":1.357},"last":0.61199999999999999},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.83899999999999997,"15min":0.82699999999999996},"min":{"1min":0.61199999999999999,"5min":0.61199999999999999,"15min":0.54700000000000004},"max":{"1min":1.085,"5min":1.1459999999999999,"15min":1.284},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Dec 20 05:44:07 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.872,"15min":0.86699999999999999},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.58499999999999996},"max":{"1min":1.036,"5min":1.294,"15min":1.3620000000000001},"last":0.80900000000000005},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.90400000000000003,"15min":0.88400000000000001},"min":{"1min":0.64400000000000002,"5min":0.64400000000000002,"15min":0.55900000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.80100000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575756,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51788,"kb_used_data":2692,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320052,"statfs":{"total":96636764160,"available":96583733248,"internally_reserved":0,"allocated":2756608,"data_stored":1621975,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.47399999999999998,"5min":0.45400000000000001,"15min":0.46300000000000002},"min":{"1min":0.27800000000000002,"5min":0.27800000000000002,"15min":0.27800000000000002},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.88100000000000001},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.53000000000000003,"15min":0.53000000000000003},"min":{"1min":0.34899999999999998,"5min":0.28899999999999998,"15min":0.28899999999999998},"max":{"1min":0.94199999999999995,"5min":0.94199999999999995,"15min":0.94199999999999995},"last":0.41599999999999998}]},{"osd":2,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.56299999999999994,"5min":0.56399999999999995,"15min":0.54700000000000004},"min":{"1min":0.35799999999999998,"5min":0.35399999999999998,"15min":0.35199999999999998},"max":{"1min":0.78000000000000003,"5min":0.874,"15min":1.3859999999999999},"last":0.48199999999999998},{"interface":"front","average":{"1min":0.56799999999999995,"5min":0.57599999999999996,"15min":0.58199999999999996},"min":{"1min":0.39300000000000002,"5min":0.39300000000000002,"15min":0.39300000000000002},"max":{"1min":0.77200000000000002,"5min":0.79600000000000004,"15min":1.369},"last":0.54600000000000004}]},{"osd":3,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.52900000000000003,"5min":0.54300000000000004,"15min":0.53200000000000003},"min":{"1min":0.315,"5min":0.315,"15min":0.315},"max":{"1min":0.71599999999999997,"5min":0.84699999999999998,"15min":1.2709999999999999},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.622,"5min":0.61399999999999999,"15min":0.61499999999999999},"min":{"1min":0.40400000000000003,"5min":0.40400000000000003,"15min":0.40400000000000003},"max":{"1min":0.90300000000000002,"5min":0.90300000000000002,"15min":2.016},"last":0.439}]},{"osd":4,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.81699999999999995,"5min":0.79700000000000004,"15min":0.80700000000000005},"min":{"1min":0.56200000000000006,"5min":0.52800000000000002,"15min":0.47799999999999998},"max":{"1min":1.1679999999999999,"5min":1.1679999999999999,"15min":1.294},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.83399999999999996,"5min":0.81200000000000006,"15min":0.81000000000000005},"min":{"1min":0.63200000000000001,"5min":0.58099999999999996,"15min":0.52100000000000002},"max":{"1min":1.3100000000000001,"5min":1.3100000000000001,"15min":1.4530000000000001},"last":0.66000000000000003}]},{"osd":5,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.80300000000000005,"5min":0.80200000000000005,"15min":0.82199999999999995},"min":{"1min":0.59199999999999997,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":1.177,"5min":1.238,"15min":1.6080000000000001},"last":0.753},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.81699999999999995,"15min":0.81899999999999995},"min":{"1min":0.60999999999999999,"5min":0.47799999999999998,"15min":0.47799999999999998},"max":{"1min":1.129,"5min":1.22,"15min":1.3200000000000001},"last":0.64000000000000001}]},{"osd":6,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.82499999999999996,"5min":0.83399999999999996,"15min":0.83399999999999996},"min":{"1min":0.621,"5min":0.499,"15min":0.499},"max":{"1min":1.256,"5min":1.256,"15min":1.665},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.82599999999999996,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55100000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.1439999999999999,"5min":1.167,"15min":1.5660000000000001},"last":0.72699999999999998}]},{"osd":7,"last update":"Wed Dec 20 05:43:36 2023","interfaces":[{"interface":"back","average":{"1min":0.84599999999999997,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.63200000000000001,"5min":0.55600000000000005,"15min":0.55600000000000005},"max":{"1min":1.212,"5min":1.222,"15min":1.4670000000000001},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.83999999999999997,"5min":0.83299999999999996,"15min":0.83699999999999997},"min":{"1min":0.57299999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.198,"5min":1.2649999999999999,"15min":1.415},"last":0.84399999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1598,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":20,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":652,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":21,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T05:44:25.526 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T05:44:25.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T05:44:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f8884d7b640 1 Processor -- start 2023-12-20T05:44:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f8884d7b640 1 -- start start 2023-12-20T05:44:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f8884d7b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f88800cb8d0 0x7f88800cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f8884d7b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88800cc9d0 con 0x7f888015a940 2023-12-20T05:44:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f8884d7b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f88800c9b90 con 0x7f888015ad20 2023-12-20T05:44:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f8884d7b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f88800c9e70 con 0x7f88800cb8d0 2023-12-20T05:44:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f887e59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f888015a940 0x7f88800ccee0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45218/0 (socket says 172.21.15.32:45218) 2023-12-20T05:44:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f887e59c640 1 -- 172.21.15.32:0/100970854 learned_addr learned my addr 172.21.15.32:0/100970854 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:25.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f887ed9d640 1 --2- 172.21.15.32:0/100970854 >> v2:172.21.15.39:3300/0 conn(0x7f88800cb8d0 0x7f88800cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f887dd9b640 1 -- 172.21.15.32:0/100970854 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1473400537 0 0) 0x7f88800cc9d0 con 0x7f888015a940 2023-12-20T05:44:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f887dd9b640 1 -- 172.21.15.32:0/100970854 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f886c003530 con 0x7f888015a940 2023-12-20T05:44:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f887ed9d640 1 -- 172.21.15.32:0/100970854 >> v1:172.21.15.39:6789/0 conn(0x7f888015ad20 legacy=0x7f88800cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f887ed9d640 1 -- 172.21.15.32:0/100970854 >> v1:172.21.15.32:6789/0 conn(0x7f888015a940 legacy=0x7f88800ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.719+0000 7f887ed9d640 1 -- 172.21.15.32:0/100970854 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f88800ca150 con 0x7f88800cb8d0 2023-12-20T05:44:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.723+0000 7f887ed9d640 1 --2- 172.21.15.32:0/100970854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88800cb8d0 0x7f88800cc380 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8878009dc0 tx=0x7f887801c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=896a593ca46a197c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.723+0000 7f887dd9b640 1 -- 172.21.15.32:0/100970854 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88780a5950 con 0x7f88800cb8d0 2023-12-20T05:44:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.723+0000 7f887dd9b640 1 -- 172.21.15.32:0/100970854 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f88780a5ab0 con 0x7f88800cb8d0 2023-12-20T05:44:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.723+0000 7f887dd9b640 1 -- 172.21.15.32:0/100970854 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88780a5e40 con 0x7f88800cb8d0 2023-12-20T05:44:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.723+0000 7f8884d7b640 1 -- 172.21.15.32:0/100970854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88800cb8d0 msgr2=0x7f88800cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.723+0000 7f8884d7b640 1 --2- 172.21.15.32:0/100970854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88800cb8d0 0x7f88800cc380 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f8878009dc0 tx=0x7f887801c870 comp rx=0 tx=0).stop 2023-12-20T05:44:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.723+0000 7f8884d7b640 1 -- 172.21.15.32:0/100970854 shutdown_connections 2023-12-20T05:44:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.723+0000 7f8884d7b640 1 --2- 172.21.15.32:0/100970854 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f88800cb8d0 0x7f88800cc380 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.723+0000 7f8884d7b640 1 -- 172.21.15.32:0/100970854 >> 172.21.15.32:0/100970854 conn(0x7f888006a4b0 msgr2=0x7f88800b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:25.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.723+0000 7f8884d7b640 1 -- 172.21.15.32:0/100970854 shutdown_connections 2023-12-20T05:44:25.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.723+0000 7f8884d7b640 1 -- 172.21.15.32:0/100970854 wait complete. 2023-12-20T05:44:25.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8884d7b640 1 Processor -- start 2023-12-20T05:44:25.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8884d7b640 1 -- start start 2023-12-20T05:44:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8884d7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f88800cb8d0 0x7f8880152ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8884d7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f888015a940 0x7f88800c3630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8884d7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888015ad20 0x7f88800c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8884d7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f88800c49b0 con 0x7f888015ad20 2023-12-20T05:44:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8884d7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f888014d2b0 con 0x7f888015a940 2023-12-20T05:44:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8884d7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f88800c3d30 con 0x7f88800cb8d0 2023-12-20T05:44:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f887ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888015ad20 0x7f88800c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f887ed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888015ad20 0x7f88800c5880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59040/0 (socket says 172.21.15.32:59040) 2023-12-20T05:44:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f887ed9d640 1 -- 172.21.15.32:0/4060786461 learned_addr learned my addr 172.21.15.32:0/4060786461 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f887e59c640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f88800cb8d0 0x7f8880152ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8877fff640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f888015a940 0x7f88800c3630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:25.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f887ed9d640 1 -- 172.21.15.32:0/4060786461 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f88800cb8d0 msgr2=0x7f8880152ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f887ed9d640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f88800cb8d0 0x7f8880152ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f887ed9d640 1 -- 172.21.15.32:0/4060786461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f888015a940 msgr2=0x7f88800c3630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f887ed9d640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f888015a940 0x7f88800c3630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f887ed9d640 1 -- 172.21.15.32:0/4060786461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8878009a70 con 0x7f888015ad20 2023-12-20T05:44:25.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8877fff640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f888015a940 0x7f88800c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:25.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f887e59c640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f88800cb8d0 0x7f8880152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:25.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f887ed9d640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888015ad20 0x7f88800c5880 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f88780b0040 tx=0x7f8878019610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:25.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8876ffd640 1 -- 172.21.15.32:0/4060786461 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88780a5950 con 0x7f888015ad20 2023-12-20T05:44:25.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8876ffd640 1 -- 172.21.15.32:0/4060786461 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8878009450 con 0x7f888015ad20 2023-12-20T05:44:25.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8876ffd640 1 -- 172.21.15.32:0/4060786461 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f88780b9670 con 0x7f888015ad20 2023-12-20T05:44:25.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8884d7b640 1 -- 172.21.15.32:0/4060786461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f88800c4010 con 0x7f888015ad20 2023-12-20T05:44:25.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.727+0000 7f8884d7b640 1 -- 172.21.15.32:0/4060786461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f88800c45d0 con 0x7f888015ad20 2023-12-20T05:44:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.731+0000 7f8884d7b640 1 -- 172.21.15.32:0/4060786461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f888004e000 con 0x7f888015ad20 2023-12-20T05:44:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.731+0000 7f8876ffd640 1 -- 172.21.15.32:0/4060786461 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f88780b97d0 con 0x7f888015ad20 2023-12-20T05:44:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.731+0000 7f8876ffd640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8860065b80 0x7f8860068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.731+0000 7f8876ffd640 1 -- 172.21.15.32:0/4060786461 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f887811ec20 con 0x7f888015ad20 2023-12-20T05:44:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.731+0000 7f887e59c640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8860065b80 0x7f8860068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:25.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.731+0000 7f887e59c640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8860065b80 0x7f8860068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f88700011f0 tx=0x7f8870009040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:25.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.735+0000 7f8876ffd640 1 -- 172.21.15.32:0/4060786461 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f88780ec560 con 0x7f888015ad20 2023-12-20T05:44:25.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.947+0000 7f8884d7b640 1 -- 172.21.15.32:0/4060786461 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f8880150060 con 0x7f888015ad20 2023-12-20T05:44:25.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.951+0000 7f8876ffd640 1 -- 172.21.15.32:0/4060786461 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v44) v1 ==== 190+0+0 (secure 0 0 0) 0x7f88780efb80 con 0x7f888015ad20 2023-12-20T05:44:25.951 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T05:44:25.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 -- 172.21.15.32:0/4060786461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8860065b80 msgr2=0x7f8860068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8860065b80 0x7f8860068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f88700011f0 tx=0x7f8870009040 comp rx=0 tx=0).stop 2023-12-20T05:44:25.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 -- 172.21.15.32:0/4060786461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888015ad20 msgr2=0x7f88800c5880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:25.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888015ad20 0x7f88800c5880 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f88780b0040 tx=0x7f8878019610 comp rx=0 tx=0).stop 2023-12-20T05:44:25.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 -- 172.21.15.32:0/4060786461 shutdown_connections 2023-12-20T05:44:25.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f8860065b80 0x7f8860068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f888015ad20 0x7f88800c5880 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f888015a940 0x7f88800c3630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 --2- 172.21.15.32:0/4060786461 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f88800cb8d0 0x7f8880152ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:25.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 -- 172.21.15.32:0/4060786461 >> 172.21.15.32:0/4060786461 conn(0x7f888006a4b0 msgr2=0x7f8880150580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:25.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 -- 172.21.15.32:0/4060786461 shutdown_connections 2023-12-20T05:44:25.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:25.955+0000 7f8884d7b640 1 -- 172.21.15.32:0/4060786461 wait complete. 2023-12-20T05:44:25.972 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T05:44:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.119+0000 7f3f4f17b640 1 Processor -- start 2023-12-20T05:44:26.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.119+0000 7f3f4f17b640 1 -- start start 2023-12-20T05:44:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.119+0000 7f3f4f17b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f481599e0 0x7f3f480aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.119+0000 7f3f4f17b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3f480b0d00 0x7f3f480701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.119+0000 7f3f4f17b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f480c98c0 con 0x7f3f480c90e0 2023-12-20T05:44:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.119+0000 7f3f4f17b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f480c9ba0 con 0x7f3f481599e0 2023-12-20T05:44:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.119+0000 7f3f4f17b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f480c9e80 con 0x7f3f480b0d00 2023-12-20T05:44:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f4cf17640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f481599e0 0x7f3f480aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f47fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3f480c90e0 0x7f3f480b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45226/0 (socket says 172.21.15.32:45226) 2023-12-20T05:44:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f47fff640 1 -- 172.21.15.32:0/4145346985 learned_addr learned my addr 172.21.15.32:0/4145346985 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f4d718640 1 --2- 172.21.15.32:0/4145346985 >> v2:172.21.15.39:3300/0 conn(0x7f3f480b0d00 0x7f3f480701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f4cf17640 1 -- 172.21.15.32:0/4145346985 >> v2:172.21.15.39:3300/0 conn(0x7f3f480b0d00 msgr2=0x7f3f480701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f4cf17640 1 --2- 172.21.15.32:0/4145346985 >> v2:172.21.15.39:3300/0 conn(0x7f3f480b0d00 0x7f3f480701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f4cf17640 1 -- 172.21.15.32:0/4145346985 >> v1:172.21.15.32:6789/0 conn(0x7f3f480c90e0 legacy=0x7f3f480b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f4cf17640 1 -- 172.21.15.32:0/4145346985 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f480ca160 con 0x7f3f481599e0 2023-12-20T05:44:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f4cf17640 1 --2- 172.21.15.32:0/4145346985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f481599e0 0x7f3f480aece0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f3f480b0060 tx=0x7f3f3c01e620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=547e8ff630421e06 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f477fe640 1 -- 172.21.15.32:0/4145346985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0ab400 con 0x7f3f481599e0 2023-12-20T05:44:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f477fe640 1 -- 172.21.15.32:0/4145346985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f3c0ab560 con 0x7f3f481599e0 2023-12-20T05:44:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.123+0000 7f3f477fe640 1 -- 172.21.15.32:0/4145346985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f3c0ab8f0 con 0x7f3f481599e0 2023-12-20T05:44:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4145346985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f481599e0 msgr2=0x7f3f480aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 --2- 172.21.15.32:0/4145346985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f481599e0 0x7f3f480aece0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f3f480b0060 tx=0x7f3f3c01e620 comp rx=0 tx=0).stop 2023-12-20T05:44:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4145346985 shutdown_connections 2023-12-20T05:44:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 --2- 172.21.15.32:0/4145346985 >> v2:172.21.15.39:3300/0 conn(0x7f3f480b0d00 0x7f3f480701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 --2- 172.21.15.32:0/4145346985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f481599e0 0x7f3f480aece0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4145346985 >> 172.21.15.32:0/4145346985 conn(0x7f3f4806a320 msgr2=0x7f3f480ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4145346985 shutdown_connections 2023-12-20T05:44:26.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4145346985 wait complete. 2023-12-20T05:44:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 Processor -- start 2023-12-20T05:44:26.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 -- start start 2023-12-20T05:44:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f480b0d00 0x7f3f48156730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f480c90e0 0x7f3f4814d790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f481599e0 0x7f3f480c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f481f5350 con 0x7f3f480c90e0 2023-12-20T05:44:26.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f48156d10 con 0x7f3f481599e0 2023-12-20T05:44:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4f17b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f48156ff0 con 0x7f3f480b0d00 2023-12-20T05:44:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f47fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f480c90e0 0x7f3f4814d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f47fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f480c90e0 0x7f3f4814d790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59056/0 (socket says 172.21.15.32:59056) 2023-12-20T05:44:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f47fff640 1 -- 172.21.15.32:0/4175392422 learned_addr learned my addr 172.21.15.32:0/4175392422 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4cf17640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f480b0d00 0x7f3f48156730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4d718640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f481599e0 0x7f3f480c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f47fff640 1 -- 172.21.15.32:0/4175392422 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f480b0d00 msgr2=0x7f3f48156730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f47fff640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f480b0d00 0x7f3f48156730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f47fff640 1 -- 172.21.15.32:0/4175392422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f481599e0 msgr2=0x7f3f480c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f47fff640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f481599e0 0x7f3f480c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f47fff640 1 -- 172.21.15.32:0/4175392422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f3c0093b0 con 0x7f3f480c90e0 2023-12-20T05:44:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.127+0000 7f3f4d718640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f481599e0 0x7f3f480c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.131+0000 7f3f47fff640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f480c90e0 0x7f3f4814d790 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c500 tx=0x7f3f3400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.131+0000 7f3f457fa640 1 -- 172.21.15.32:0/4175392422 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f34005cf0 con 0x7f3f480c90e0 2023-12-20T05:44:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.131+0000 7f3f457fa640 1 -- 172.21.15.32:0/4175392422 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f34005e50 con 0x7f3f480c90e0 2023-12-20T05:44:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.131+0000 7f3f457fa640 1 -- 172.21.15.32:0/4175392422 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f340072f0 con 0x7f3f480c90e0 2023-12-20T05:44:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.131+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4175392422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f481572d0 con 0x7f3f480c90e0 2023-12-20T05:44:26.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.131+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4175392422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f48157890 con 0x7f3f480c90e0 2023-12-20T05:44:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.131+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4175392422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f4804e000 con 0x7f3f480c90e0 2023-12-20T05:44:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.131+0000 7f3f457fa640 1 -- 172.21.15.32:0/4175392422 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3f34007450 con 0x7f3f480c90e0 2023-12-20T05:44:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.131+0000 7f3f457fa640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f2c065b80 0x7f3f2c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.131+0000 7f3f457fa640 1 -- 172.21.15.32:0/4175392422 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f3f340827f0 con 0x7f3f480c90e0 2023-12-20T05:44:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.131+0000 7f3f4cf17640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f2c065b80 0x7f3f2c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.135+0000 7f3f4cf17640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f2c065b80 0x7f3f2c068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0b0000 tx=0x7f3f3c0b5040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:26.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.135+0000 7f3f457fa640 1 -- 172.21.15.32:0/4175392422 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3f34050120 con 0x7f3f480c90e0 2023-12-20T05:44:26.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4175392422 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f3f480cf550 con 0x7f3f480c90e0 2023-12-20T05:44:26.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f457fa640 1 -- 172.21.15.32:0/4175392422 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4710 (secure 0 0 0) 0x7f3f34017090 con 0x7f3f480c90e0 2023-12-20T05:44:26.325 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:44:26.325 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1922232083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1922232083},{"type":"v1","addr":"172.21.15.39:6833","nonce":1922232083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9103,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428109066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428109066},{"type":"v1","addr":"172.21.15.39:6835","nonce":428109066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1595154235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1595154235},{"type":"v1","addr":"172.21.15.32:6836","nonce":1595154235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:44:21.009338+0000","modified":"2023-12-20T05:44:24.030393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2806059816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2806059816},{"type":"v1","addr":"172.21.15.32:6837","nonce":2806059816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-20T05:44:26.325 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 79 2023-12-20T05:44:26.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4175392422 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f2c065b80 msgr2=0x7f3f2c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f2c065b80 0x7f3f2c068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f3f3c0b0000 tx=0x7f3f3c0b5040 comp rx=0 tx=0).stop 2023-12-20T05:44:26.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4175392422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f480c90e0 msgr2=0x7f3f4814d790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f480c90e0 0x7f3f4814d790 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f3f3400c500 tx=0x7f3f3400c9c0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4175392422 shutdown_connections 2023-12-20T05:44:26.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f3f2c065b80 0x7f3f2c068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f481599e0 0x7f3f480c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f480c90e0 0x7f3f4814d790 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 --2- 172.21.15.32:0/4175392422 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f480b0d00 0x7f3f48156730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4175392422 >> 172.21.15.32:0/4175392422 conn(0x7f3f4806a320 msgr2=0x7f3f480aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:26.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4175392422 shutdown_connections 2023-12-20T05:44:26.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.327+0000 7f3f4f17b640 1 -- 172.21.15.32:0/4175392422 wait complete. 2023-12-20T05:44:26.574 INFO:tasks.ceph.mds.c.smithi032.stderr:2023-12-20T05:44:26.575+0000 7f7b4ee19640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T05:44:26.749 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 79, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T05:44:21.009338+0000', 'modified': '2023-12-20T05:44:24.030393+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 9420}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_9420': {'gid': 9420, 'name': 'c', 'rank': 0, 'incarnation': 75, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/2806059816', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 2806059816}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 2806059816}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [21], 'metadata_pool': 20, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T05:44:26.749 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T05:44:26.749 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-20T05:44:26.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f2260cc4640 1 Processor -- start 2023-12-20T05:44:26.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f2260cc4640 1 -- start start 2023-12-20T05:44:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f2260cc4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f225c1599e0 0x7f225c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f2260cc4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f225c0b0d00 0x7f225c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f2260cc4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f225c0c98c0 con 0x7f225c0c90e0 2023-12-20T05:44:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f2260cc4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f225c0c9ba0 con 0x7f225c0b0d00 2023-12-20T05:44:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f2260cc4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c0c9e80 con 0x7f225c1599e0 2023-12-20T05:44:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f225ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f225c0b0d00 0x7f225c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f2253fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f225c0c90e0 0x7f225c0b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45228/0 (socket says 172.21.15.32:45228) 2023-12-20T05:44:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f2253fff640 1 -- 172.21.15.32:0/4208942870 learned_addr learned my addr 172.21.15.32:0/4208942870 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f225a59c640 1 --2- 172.21.15.32:0/4208942870 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f225c1599e0 0x7f225c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f225ad9d640 1 -- 172.21.15.32:0/4208942870 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f225c1599e0 msgr2=0x7f225c0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f225ad9d640 1 --2- 172.21.15.32:0/4208942870 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f225c1599e0 0x7f225c0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f225ad9d640 1 -- 172.21.15.32:0/4208942870 >> v1:172.21.15.32:6789/0 conn(0x7f225c0c90e0 legacy=0x7f225c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f225ad9d640 1 -- 172.21.15.32:0/4208942870 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f225c0ca160 con 0x7f225c0b0d00 2023-12-20T05:44:26.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.899+0000 7f2259d9b640 1 -- 172.21.15.32:0/4208942870 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1502723377 0 0) 0x7f225c0c98c0 con 0x7f225c0c90e0 2023-12-20T05:44:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f225ad9d640 1 --2- 172.21.15.32:0/4208942870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c0b0d00 0x7f225c0701c0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f2254009d90 tx=0x7f225401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9071da2c83903612 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f2259d9b640 1 -- 172.21.15.32:0/4208942870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22540a5860 con 0x7f225c0b0d00 2023-12-20T05:44:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f2259d9b640 1 -- 172.21.15.32:0/4208942870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22540a59c0 con 0x7f225c0b0d00 2023-12-20T05:44:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f2259d9b640 1 -- 172.21.15.32:0/4208942870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22540a5d50 con 0x7f225c0b0d00 2023-12-20T05:44:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f2260cc4640 1 -- 172.21.15.32:0/4208942870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c0b0d00 msgr2=0x7f225c0701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f2260cc4640 1 --2- 172.21.15.32:0/4208942870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c0b0d00 0x7f225c0701c0 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f2254009d90 tx=0x7f225401c890 comp rx=0 tx=0).stop 2023-12-20T05:44:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f2260cc4640 1 -- 172.21.15.32:0/4208942870 shutdown_connections 2023-12-20T05:44:26.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f2260cc4640 1 --2- 172.21.15.32:0/4208942870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c0b0d00 0x7f225c0701c0 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f2260cc4640 1 --2- 172.21.15.32:0/4208942870 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f225c1599e0 0x7f225c0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f2260cc4640 1 -- 172.21.15.32:0/4208942870 >> 172.21.15.32:0/4208942870 conn(0x7f225c06a320 msgr2=0x7f225c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:26.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f2260cc4640 1 -- 172.21.15.32:0/4208942870 shutdown_connections 2023-12-20T05:44:26.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.903+0000 7f2260cc4640 1 -- 172.21.15.32:0/4208942870 wait complete. 2023-12-20T05:44:26.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2260cc4640 1 Processor -- start 2023-12-20T05:44:26.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2260cc4640 1 -- start start 2023-12-20T05:44:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2260cc4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c0b0d00 0x7f225c149f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2260cc4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0c90e0 0x7f225c0adb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2260cc4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f225c1599e0 0x7f225c153ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2260cc4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c15b1b0 con 0x7f225c0b0d00 2023-12-20T05:44:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2260cc4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f225c157180 con 0x7f225c0c90e0 2023-12-20T05:44:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2260cc4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f225c157460 con 0x7f225c1599e0 2023-12-20T05:44:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f225a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c0b0d00 0x7f225c149f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f225a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c0b0d00 0x7f225c149f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59074/0 (socket says 172.21.15.32:59074) 2023-12-20T05:44:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f225a59c640 1 -- 172.21.15.32:0/2405228689 learned_addr learned my addr 172.21.15.32:0/2405228689 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f225ad9d640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f225c1599e0 0x7f225c153ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2253fff640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0c90e0 0x7f225c0adb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f225a59c640 1 -- 172.21.15.32:0/2405228689 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f225c1599e0 msgr2=0x7f225c153ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f225a59c640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f225c1599e0 0x7f225c153ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f225a59c640 1 -- 172.21.15.32:0/2405228689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0c90e0 msgr2=0x7f225c0adb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f225a59c640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0c90e0 0x7f225c0adb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f225a59c640 1 -- 172.21.15.32:0/2405228689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2254009a70 con 0x7f225c0b0d00 2023-12-20T05:44:26.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2253fff640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0c90e0 0x7f225c0adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:26.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f225a59c640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c0b0d00 0x7f225c149f60 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f224c00e870 tx=0x7f224c00ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:26.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2252ffd640 1 -- 172.21.15.32:0/2405228689 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c0095d0 con 0x7f225c0b0d00 2023-12-20T05:44:26.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2252ffd640 1 -- 172.21.15.32:0/2405228689 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c00c940 con 0x7f225c0b0d00 2023-12-20T05:44:26.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2260cc4640 1 -- 172.21.15.32:0/2405228689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f225c157740 con 0x7f225c0b0d00 2023-12-20T05:44:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2252ffd640 1 -- 172.21.15.32:0/2405228689 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c00ccb0 con 0x7f225c0b0d00 2023-12-20T05:44:26.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.907+0000 7f2260cc4640 1 -- 172.21.15.32:0/2405228689 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225c14a4a0 con 0x7f225c0b0d00 2023-12-20T05:44:26.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.911+0000 7f2252ffd640 1 -- 172.21.15.32:0/2405228689 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f224c013470 con 0x7f225c0b0d00 2023-12-20T05:44:26.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.911+0000 7f2252ffd640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f223c065bb0 0x7f223c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.911+0000 7f2252ffd640 1 -- 172.21.15.32:0/2405228689 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f224c083880 con 0x7f225c0b0d00 2023-12-20T05:44:26.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.911+0000 7f2253fff640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f223c065bb0 0x7f223c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.911+0000 7f2264020000 1 Processor -- start 2023-12-20T05:44:26.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.911+0000 7f2264020000 1 -- start start 2023-12-20T05:44:26.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.911+0000 7f2253fff640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f223c065bb0 0x7f223c068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f225c0c8d80 tx=0x7f2244007c40 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:26.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e664c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6651a0 0x558e7e611c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6121c0 0x558e7e6125a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7e435c90 con 0x558e7e664880 2023-12-20T05:44:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7e467220 con 0x558e7e6121c0 2023-12-20T05:44:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e7e4647f0 con 0x558e7e6651a0 2023-12-20T05:44:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f225a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e664c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f225a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e664c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59080/0 (socket says 172.21.15.32:59080) 2023-12-20T05:44:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f225a59c640 1 -- 172.21.15.32:0/2938076691 learned_addr learned my addr 172.21.15.32:0/2938076691 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2253fff640 1 --2- 172.21.15.32:0/2938076691 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6651a0 0x558e7e611c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f225ad9d640 1 --2- 172.21.15.32:0/2938076691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6121c0 0x558e7e6125a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f225a59c640 1 -- 172.21.15.32:0/2938076691 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6651a0 msgr2=0x558e7e611c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:26.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f225a59c640 1 --2- 172.21.15.32:0/2938076691 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6651a0 0x558e7e611c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f225a59c640 1 -- 172.21.15.32:0/2938076691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6121c0 msgr2=0x558e7e6125a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f225a59c640 1 --2- 172.21.15.32:0/2938076691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6121c0 0x558e7e6125a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f225a59c640 1 -- 172.21.15.32:0/2938076691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e7e15c890 con 0x558e7e664880 2023-12-20T05:44:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f225ad9d640 1 --2- 172.21.15.32:0/2938076691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6121c0 0x558e7e6125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f225a59c640 1 --2- 172.21.15.32:0/2938076691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e664c60 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f224c0513a0 tx=0x7f224c058ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f976a338d1d52b80 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f223bfff640 1 -- 172.21.15.32:0/2938076691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c07f070 con 0x558e7e664880 2023-12-20T05:44:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f223bfff640 1 -- 172.21.15.32:0/2938076691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f224c053240 con 0x558e7e664880 2023-12-20T05:44:26.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f223bfff640 1 -- 172.21.15.32:0/2938076691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c081ce0 con 0x558e7e664880 2023-12-20T05:44:26.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 -- 172.21.15.32:0/2938076691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 msgr2=0x558e7e664c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 --2- 172.21.15.32:0/2938076691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e664c60 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f224c0513a0 tx=0x7f224c058ce0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 -- 172.21.15.32:0/2938076691 shutdown_connections 2023-12-20T05:44:26.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 --2- 172.21.15.32:0/2938076691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6121c0 0x558e7e6125a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 --2- 172.21.15.32:0/2938076691 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6651a0 0x558e7e611c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 --2- 172.21.15.32:0/2938076691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e664c60 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 -- 172.21.15.32:0/2938076691 >> 172.21.15.32:0/2938076691 conn(0x558e7e618950 msgr2=0x558e7e610560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 -- 172.21.15.32:0/2938076691 shutdown_connections 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 -- 172.21.15.32:0/2938076691 wait complete. 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 -- start start 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.915+0000 7f2264020000 1 Processor -- start 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e617740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e664880 0x558e7e626890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e6651a0 0x558e7e626dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7e4600e0 con 0x558e7e6651a0 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7e3ff310 con 0x558e7e664880 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e7e41d770 con 0x558e7e6121c0 2023-12-20T05:44:26.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 10 client.0 _ll_register_callbacks cb 0x558e7e5fdb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T05:44:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e6651a0 0x558e7e626dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e6651a0 0x558e7e626dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59082/0 (socket says 172.21.15.32:59082) 2023-12-20T05:44:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225ad9d640 1 -- 172.21.15.32:0/2583723798 learned_addr learned my addr 172.21.15.32:0/2583723798 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2253fff640 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e664880 0x558e7e626890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225a59c640 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e617740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225ad9d640 1 -- 172.21.15.32:0/2583723798 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 msgr2=0x558e7e617740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225ad9d640 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e617740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225ad9d640 1 -- 172.21.15.32:0/2583723798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e664880 msgr2=0x558e7e626890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225ad9d640 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e664880 0x558e7e626890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225ad9d640 1 -- 172.21.15.32:0/2583723798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f225c14a4a0 con 0x558e7e6651a0 2023-12-20T05:44:26.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225ad9d640 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e6651a0 0x558e7e626dd0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f2254009d90 tx=0x7f225401b0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44ab21c8ebc2aa7d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f22397fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f22397fa640 1 -- 172.21.15.32:0/2583723798 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f22540b8440 con 0x558e7e6651a0 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f22397fa640 1 -- 172.21.15.32:0/2583723798 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f22540b85a0 con 0x558e7e6651a0 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 1 -- 172.21.15.32:0/2583723798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e7e42fc10 con 0x558e7e6651a0 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f22397fa640 1 -- 172.21.15.32:0/2583723798 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f22540b8760 con 0x558e7e6651a0 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 10 client.9567 fetch_fsmap learned FSMap version 79 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 1 -- 172.21.15.32:0/2583723798 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x558e7e425260 con 0x558e7e6651a0 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f22397fa640 1 -- 172.21.15.32:0/2583723798 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558e7e42fc10 con 0x558e7e6651a0 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f22397fa640 1 -- 172.21.15.32:0/2583723798 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 79) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f22540b88c0 con 0x558e7e6651a0 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 10 client.9567 fetch_fsmap finished waiting for FSMap version 79 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 10 client.9567 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-20T05:44:26.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 20 client.9567 populate_metadata read hostname 'smithi032' 2023-12-20T05:44:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.32:6835/2806059816,v1:172.21.15.32:6837/2806059816] conn(0x558e7e66c0e0 0x558e7e66c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 4 client.9567 mds_command: new command op to 9420 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T05:44:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f2264020000 1 -- 172.21.15.32:0/2583723798 --> [v2:172.21.15.32:6835/2806059816,v1:172.21.15.32:6837/2806059816] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x558e7e45ea30 con 0x558e7e66c0e0 2023-12-20T05:44:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225a59c640 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.32:6835/2806059816,v1:172.21.15.32:6837/2806059816] conn(0x558e7e66c0e0 0x558e7e66c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f225a59c640 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.32:6835/2806059816,v1:172.21.15.32:6837/2806059816] conn(0x558e7e66c0e0 0x558e7e66c4c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=edf207ee2208db01 server_cookie=febb8167156f35d9 in_seq=0 out_seq=0 2023-12-20T05:44:26.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.919+0000 7f22397fa640 10 client.9567 ms_handle_connect on v2:172.21.15.32:6835/2806059816 2023-12-20T05:44:26.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f22397fa640 1 -- 172.21.15.32:0/2583723798 <== mds.0 v2:172.21.15.32:6835/2806059816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x558e7e45ea30 con 0x558e7e66c0e0 2023-12-20T05:44:26.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f22397fa640 10 client.9567 handle_command_reply: tid=0 2023-12-20T05:44:26.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 1 client.9567 shutdown 2023-12-20T05:44:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 1 -- 172.21.15.32:0/2583723798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e6651a0 msgr2=0x558e7e626dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e6651a0 0x558e7e626dd0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f2254009d90 tx=0x7f225401b0d0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 1 -- 172.21.15.32:0/2583723798 shutdown_connections 2023-12-20T05:44:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.32:6835/2806059816,v1:172.21.15.32:6837/2806059816] conn(0x558e7e66c0e0 0x558e7e66c4c0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e6651a0 0x558e7e626dd0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e664880 0x558e7e626890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f22397fa640 0 client.9567 ms_handle_reset on v2:172.21.15.32:6835/2806059816 2023-12-20T05:44:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 1 --2- 172.21.15.32:0/2583723798 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e617740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 1 -- 172.21.15.32:0/2583723798 >> 172.21.15.32:0/2583723798 conn(0x558e7e618950 msgr2=0x558e7e610c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 1 -- 172.21.15.32:0/2583723798 shutdown_connections 2023-12-20T05:44:26.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 1 -- 172.21.15.32:0/2583723798 wait complete. 2023-12-20T05:44:26.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.923+0000 7f2264020000 20 client.9567 trim_cache size 0 max 16384 2023-12-20T05:44:26.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.935+0000 7f2264020000 1 Processor -- start 2023-12-20T05:44:26.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 -- start start 2023-12-20T05:44:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e6bc6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6bcc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6651a0 0x558e7e6bd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7e11d940 con 0x558e7e664880 2023-12-20T05:44:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7e405770 con 0x558e7e6651a0 2023-12-20T05:44:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e7e45d4e0 con 0x558e7e6121c0 2023-12-20T05:44:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2253fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6bcc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2253fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6bcc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59092/0 (socket says 172.21.15.32:59092) 2023-12-20T05:44:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2253fff640 1 -- 172.21.15.32:0/2828521614 learned_addr learned my addr 172.21.15.32:0/2828521614 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f225a59c640 1 --2- 172.21.15.32:0/2828521614 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e6bc6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f225ad9d640 1 --2- 172.21.15.32:0/2828521614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6651a0 0x558e7e6bd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2253fff640 1 -- 172.21.15.32:0/2828521614 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 msgr2=0x558e7e6bc6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2253fff640 1 --2- 172.21.15.32:0/2828521614 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e6bc6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2253fff640 1 -- 172.21.15.32:0/2828521614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6651a0 msgr2=0x558e7e6bd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2253fff640 1 --2- 172.21.15.32:0/2828521614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6651a0 0x558e7e6bd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2253fff640 1 -- 172.21.15.32:0/2828521614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x558e7e40b3b0 con 0x558e7e664880 2023-12-20T05:44:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f225ad9d640 1 --2- 172.21.15.32:0/2828521614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6651a0 0x558e7e6bd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f225a59c640 1 --2- 172.21.15.32:0/2828521614 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e6bc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2253fff640 1 --2- 172.21.15.32:0/2828521614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6bcc20 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f225c0c52b0 tx=0x7f2244012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90231aba20d54728 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2250ff9640 1 -- 172.21.15.32:0/2828521614 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244023440 con 0x558e7e664880 2023-12-20T05:44:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2250ff9640 1 -- 172.21.15.32:0/2828521614 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22440235a0 con 0x558e7e664880 2023-12-20T05:44:26.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2250ff9640 1 -- 172.21.15.32:0/2828521614 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2244023710 con 0x558e7e664880 2023-12-20T05:44:26.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 -- 172.21.15.32:0/2828521614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 msgr2=0x558e7e6bcc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 --2- 172.21.15.32:0/2828521614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6bcc20 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f225c0c52b0 tx=0x7f2244012000 comp rx=0 tx=0).stop 2023-12-20T05:44:26.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 -- 172.21.15.32:0/2828521614 shutdown_connections 2023-12-20T05:44:26.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 --2- 172.21.15.32:0/2828521614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6651a0 0x558e7e6bd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 --2- 172.21.15.32:0/2828521614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6bcc20 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 --2- 172.21.15.32:0/2828521614 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e6bc6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 -- 172.21.15.32:0/2828521614 >> 172.21.15.32:0/2828521614 conn(0x558e7e618950 msgr2=0x558e7e626750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 -- 172.21.15.32:0/2828521614 shutdown_connections 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 -- 172.21.15.32:0/2828521614 wait complete. 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 -- start start 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.939+0000 7f2264020000 1 Processor -- start 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e6a5330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6a5870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6651a0 0x558e7e6a5db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7e408860 con 0x558e7e664880 2023-12-20T05:44:26.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x558e7e614d70 con 0x558e7e6651a0 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x558e7e4a3ce0 con 0x558e7e6121c0 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 10 client.0 _ll_register_callbacks cb 0x558e7e40e640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2253fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6a5870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2253fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6a5870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59102/0 (socket says 172.21.15.32:59102) 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f225a59c640 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e6a5330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2253fff640 1 -- 172.21.15.32:0/414138256 learned_addr learned my addr 172.21.15.32:0/414138256 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f225ad9d640 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6651a0 0x558e7e6a5db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f225a59c640 1 -- 172.21.15.32:0/414138256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6651a0 msgr2=0x558e7e6a5db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f225a59c640 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6651a0 0x558e7e6a5db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f225a59c640 1 -- 172.21.15.32:0/414138256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 msgr2=0x558e7e6a5870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f225a59c640 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6a5870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f225a59c640 1 -- 172.21.15.32:0/414138256 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f224400f040 con 0x558e7e6121c0 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2253fff640 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f225a59c640 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e6a5330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f224c009280 tx=0x7f224c054b10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5aa4105c27ce38d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f22397fa640 10 client.0 ms_handle_connect on v2:172.21.15.39:3301/0 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f22397fa640 1 -- 172.21.15.32:0/414138256 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f224c072040 con 0x558e7e6121c0 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f22397fa640 1 -- 172.21.15.32:0/414138256 <== mon.2 v2:172.21.15.39:3301/0 2 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f224c009040 con 0x558e7e6121c0 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 1 -- 172.21.15.32:0/414138256 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x558e7e425e00 con 0x558e7e6121c0 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f22397fa640 1 -- 172.21.15.32:0/414138256 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_get_version_reply(handle=1 version=79) v2 ==== 24+0+0 (secure 0 0 0) 0x7f224c07f040 con 0x558e7e6121c0 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 10 client.8072 fetch_fsmap learned FSMap version 79 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 1 -- 172.21.15.32:0/414138256 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,fsmap=79,monmap=2+}) v3 -- 0x558e7e15ccc0 con 0x558e7e6121c0 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f22397fa640 1 -- 172.21.15.32:0/414138256 <== mon.2 v2:172.21.15.39:3301/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x558e7e425e00 con 0x558e7e6121c0 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f22397fa640 1 -- 172.21.15.32:0/414138256 <== mon.2 v2:172.21.15.39:3301/0 5 ==== fsmap(e 79) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f224c07bd20 con 0x558e7e6121c0 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 10 client.8072 fetch_fsmap finished waiting for FSMap version 79 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 10 client.8072 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-20T05:44:26.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 20 client.8072 populate_metadata read hostname 'smithi032' 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.32:6835/2806059816,v1:172.21.15.32:6837/2806059816] conn(0x558e7e6a62f0 0x558e7e6a66d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 4 client.8072 mds_command: new command op to 9420 tid=0[{"prefix": "status"}] 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2264020000 1 -- 172.21.15.32:0/414138256 --> [v2:172.21.15.32:6835/2806059816,v1:172.21.15.32:6837/2806059816] -- command(tid 0: {"prefix": "status"}) v1 -- 0x558e7e459160 con 0x558e7e6a62f0 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2253fff640 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.32:6835/2806059816,v1:172.21.15.32:6837/2806059816] conn(0x558e7e6a62f0 0x558e7e6a66d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f22397fa640 10 client.8072 ms_handle_connect on v2:172.21.15.32:6835/2806059816 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.943+0000 7f2253fff640 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.32:6835/2806059816,v1:172.21.15.32:6837/2806059816] conn(0x558e7e6a62f0 0x558e7e6a66d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a32a8e63f138520 server_cookie=cd4b1a80c648cdb6 in_seq=0 out_seq=0 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f22397fa640 1 -- 172.21.15.32:0/414138256 <== mds.0 v2:172.21.15.32:6835/2806059816 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+327 (crc 0 0 0) 0x558e7e459160 con 0x558e7e6a62f0 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f22397fa640 10 client.8072 handle_command_reply: tid=0 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 1 client.8072 shutdown 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 1 -- 172.21.15.32:0/414138256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 msgr2=0x558e7e6a5330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e6a5330 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f224c009280 tx=0x7f224c054b10 comp rx=0 tx=0).stop 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 1 -- 172.21.15.32:0/414138256 shutdown_connections 2023-12-20T05:44:26.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.32:6835/2806059816,v1:172.21.15.32:6837/2806059816] conn(0x558e7e6a62f0 0x558e7e6a66d0 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x558e7e6651a0 0x558e7e6a5db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f22397fa640 0 client.8072 ms_handle_reset on v2:172.21.15.32:6835/2806059816 2023-12-20T05:44:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x558e7e664880 0x558e7e6a5870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 1 --2- 172.21.15.32:0/414138256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x558e7e6121c0 0x558e7e6a5330 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 1 -- 172.21.15.32:0/414138256 >> 172.21.15.32:0/414138256 conn(0x558e7e618950 msgr2=0x558e7e661870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 1 -- 172.21.15.32:0/414138256 shutdown_connections 2023-12-20T05:44:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 1 -- 172.21.15.32:0/414138256 wait complete. 2023-12-20T05:44:26.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2264020000 20 client.8072 trim_cache size 0 max 16384 2023-12-20T05:44:26.949 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 9420, 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9357296010000002, 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 79, 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 173, 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 173, 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 11.218186056 2023-12-20T05:44:26.950 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T05:44:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 -- 172.21.15.32:0/2405228689 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f223c065bb0 msgr2=0x7f223c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f223c065bb0 0x7f223c068070 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f225c0c8d80 tx=0x7f2244007c40 comp rx=0 tx=0).stop 2023-12-20T05:44:26.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 -- 172.21.15.32:0/2405228689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c0b0d00 msgr2=0x7f225c149f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c0b0d00 0x7f225c149f60 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f224c00e870 tx=0x7f224c00ed30 comp rx=0 tx=0).stop 2023-12-20T05:44:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 -- 172.21.15.32:0/2405228689 shutdown_connections 2023-12-20T05:44:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f223c065bb0 0x7f223c068070 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f225c1599e0 0x7f225c153ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f225c0c90e0 0x7f225c0adb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 --2- 172.21.15.32:0/2405228689 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f225c0b0d00 0x7f225c149f60 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 -- 172.21.15.32:0/2405228689 >> 172.21.15.32:0/2405228689 conn(0x7f225c06a320 msgr2=0x7f225c155780 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 -- 172.21.15.32:0/2405228689 shutdown_connections 2023-12-20T05:44:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:26.947+0000 7f2260cc4640 1 -- 172.21.15.32:0/2405228689 wait complete. 2023-12-20T05:44:26.964 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T05:44:27.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.111+0000 7f997dc66640 1 Processor -- start 2023-12-20T05:44:27.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.111+0000 7f997dc66640 1 -- start start 2023-12-20T05:44:27.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.111+0000 7f997dc66640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f997806ce60 0x7f997815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:27.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.111+0000 7f997dc66640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f997815f110 0x7f997815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:27.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.111+0000 7f997dc66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f997815bbd0 con 0x7f99780af5e0 2023-12-20T05:44:27.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.111+0000 7f997dc66640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997815beb0 con 0x7f997806ce60 2023-12-20T05:44:27.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.111+0000 7f997dc66640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f997815e910 con 0x7f997815f110 2023-12-20T05:44:27.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.111+0000 7f9976ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f997806ce60 0x7f997815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:27.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.111+0000 7f9976ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f997806ce60 0x7f997815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59104/0 (socket says 172.21.15.32:59104) 2023-12-20T05:44:27.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.111+0000 7f9977fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f997815f110 0x7f997815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:27.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.111+0000 7f9976ffd640 1 -- 172.21.15.32:0/1423736738 learned_addr learned my addr 172.21.15.32:0/1423736738 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:27.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.115+0000 7f9976ffd640 1 -- 172.21.15.32:0/1423736738 >> v2:172.21.15.39:3300/0 conn(0x7f997815f110 msgr2=0x7f997815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:27.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.115+0000 7f9976ffd640 1 --2- 172.21.15.32:0/1423736738 >> v2:172.21.15.39:3300/0 conn(0x7f997815f110 0x7f997815c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:27.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.115+0000 7f9976ffd640 1 -- 172.21.15.32:0/1423736738 >> v1:172.21.15.39:6789/0 conn(0x7f99780af5e0 legacy=0x7f99780af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:27.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.115+0000 7f9976ffd640 1 -- 172.21.15.32:0/1423736738 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f997815ebf0 con 0x7f997806ce60 2023-12-20T05:44:27.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.115+0000 7f9977fff640 1 --2- 172.21.15.32:0/1423736738 >> v2:172.21.15.39:3300/0 conn(0x7f997815f110 0x7f997815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:27.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.115+0000 7f9976ffd640 1 --2- 172.21.15.32:0/1423736738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f997806ce60 0x7f997815b280 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f99780c8d40 tx=0x7f996c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=743c1d3f5c59b254 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:27.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.115+0000 7f99767fc640 1 -- 172.21.15.32:0/1423736738 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f997815b9d0 con 0x7f997806ce60 2023-12-20T05:44:27.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.115+0000 7f99767fc640 1 -- 172.21.15.32:0/1423736738 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c0ab610 con 0x7f997806ce60 2023-12-20T05:44:27.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.115+0000 7f99767fc640 1 -- 172.21.15.32:0/1423736738 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0ab9a0 con 0x7f997806ce60 2023-12-20T05:44:27.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.115+0000 7f997dc66640 1 -- 172.21.15.32:0/1423736738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f997806ce60 msgr2=0x7f997815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:27.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.115+0000 7f997dc66640 1 --2- 172.21.15.32:0/1423736738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f997806ce60 0x7f997815b280 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f99780c8d40 tx=0x7f996c01c470 comp rx=0 tx=0).stop 2023-12-20T05:44:27.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 -- 172.21.15.32:0/1423736738 shutdown_connections 2023-12-20T05:44:27.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 --2- 172.21.15.32:0/1423736738 >> v2:172.21.15.39:3300/0 conn(0x7f997815f110 0x7f997815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:27.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 --2- 172.21.15.32:0/1423736738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f997806ce60 0x7f997815b280 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:27.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 -- 172.21.15.32:0/1423736738 >> 172.21.15.32:0/1423736738 conn(0x7f997806a320 msgr2=0x7f99780ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:27.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 -- 172.21.15.32:0/1423736738 shutdown_connections 2023-12-20T05:44:27.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 -- 172.21.15.32:0/1423736738 wait complete. 2023-12-20T05:44:27.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 Processor -- start 2023-12-20T05:44:27.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 -- start start 2023-12-20T05:44:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f997806ce60 0x7f997814ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780af5e0 0x7f99780cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f997815f110 0x7f99780cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9978158040 con 0x7f99780af5e0 2023-12-20T05:44:27.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f99780ce4a0 con 0x7f997815f110 2023-12-20T05:44:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f997dc66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f99780cdee0 con 0x7f997806ce60 2023-12-20T05:44:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f9976ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780af5e0 0x7f99780cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f9976ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780af5e0 0x7f99780cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59110/0 (socket says 172.21.15.32:59110) 2023-12-20T05:44:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f9976ffd640 1 -- 172.21.15.32:0/2322165103 learned_addr learned my addr 172.21.15.32:0/2322165103 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f99777fe640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f997806ce60 0x7f997814ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f9977fff640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f997815f110 0x7f99780cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f9976ffd640 1 -- 172.21.15.32:0/2322165103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f997806ce60 msgr2=0x7f997814ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f9976ffd640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f997806ce60 0x7f997814ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f9976ffd640 1 -- 172.21.15.32:0/2322165103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f997815f110 msgr2=0x7f99780cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:27.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f9976ffd640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f997815f110 0x7f99780cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.119+0000 7f9976ffd640 1 -- 172.21.15.32:0/2322165103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9970009a70 con 0x7f99780af5e0 2023-12-20T05:44:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.123+0000 7f9976ffd640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780af5e0 0x7f99780cb170 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f996c019080 tx=0x7f996c01acb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.123+0000 7f995ffff640 1 -- 172.21.15.32:0/2322165103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0b4600 con 0x7f99780af5e0 2023-12-20T05:44:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.123+0000 7f995ffff640 1 -- 172.21.15.32:0/2322165103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f996c0b4780 con 0x7f99780af5e0 2023-12-20T05:44:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.123+0000 7f995ffff640 1 -- 172.21.15.32:0/2322165103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f996c0b4b30 con 0x7f99780af5e0 2023-12-20T05:44:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.123+0000 7f997dc66640 1 -- 172.21.15.32:0/2322165103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f996c0094d0 con 0x7f99780af5e0 2023-12-20T05:44:27.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.123+0000 7f997dc66640 1 -- 172.21.15.32:0/2322165103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9978157c60 con 0x7f99780af5e0 2023-12-20T05:44:27.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.123+0000 7f997dc66640 1 -- 172.21.15.32:0/2322165103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f997804e000 con 0x7f99780af5e0 2023-12-20T05:44:27.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.123+0000 7f995ffff640 1 -- 172.21.15.32:0/2322165103 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f996c0b4c90 con 0x7f99780af5e0 2023-12-20T05:44:27.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.123+0000 7f995ffff640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9964065b80 0x7f9964068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:27.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.123+0000 7f995ffff640 1 -- 172.21.15.32:0/2322165103 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f996c11d640 con 0x7f99780af5e0 2023-12-20T05:44:27.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.123+0000 7f99777fe640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9964065b80 0x7f9964068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:27.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.127+0000 7f99777fe640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9964065b80 0x7f9964068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9968000fc0 tx=0x7f9968009040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:27.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.127+0000 7f995ffff640 1 -- 172.21.15.32:0/2322165103 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f996c0eadd0 con 0x7f99780af5e0 2023-12-20T05:44:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.323+0000 7f997dc66640 1 -- 172.21.15.32:0/2322165103 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f99780cf450 con 0x7f99780af5e0 2023-12-20T05:44:27.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.323+0000 7f995ffff640 1 -- 172.21.15.32:0/2322165103 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v79) v1 ==== 53+0+73 (secure 0 0 0) 0x7f996c0ee3f0 con 0x7f99780af5e0 2023-12-20T05:44:27.321 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T05:44:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.323+0000 7f997dc66640 1 -- 172.21.15.32:0/2322165103 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9964065b80 msgr2=0x7f9964068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.323+0000 7f997dc66640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9964065b80 0x7f9964068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f9968000fc0 tx=0x7f9968009040 comp rx=0 tx=0).stop 2023-12-20T05:44:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.327+0000 7f997dc66640 1 -- 172.21.15.32:0/2322165103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780af5e0 msgr2=0x7f99780cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.327+0000 7f997dc66640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780af5e0 0x7f99780cb170 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f996c019080 tx=0x7f996c01acb0 comp rx=0 tx=0).stop 2023-12-20T05:44:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.327+0000 7f997dc66640 1 -- 172.21.15.32:0/2322165103 shutdown_connections 2023-12-20T05:44:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.327+0000 7f997dc66640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9964065b80 0x7f9964068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.327+0000 7f997dc66640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f997815f110 0x7f99780cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:27.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.327+0000 7f997dc66640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f99780af5e0 0x7f99780cb170 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:27.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.327+0000 7f997dc66640 1 --2- 172.21.15.32:0/2322165103 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f997806ce60 0x7f997814ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:27.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.327+0000 7f997dc66640 1 -- 172.21.15.32:0/2322165103 >> 172.21.15.32:0/2322165103 conn(0x7f997806a320 msgr2=0x7f9978155bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:27.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.327+0000 7f997dc66640 1 -- 172.21.15.32:0/2322165103 shutdown_connections 2023-12-20T05:44:27.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:27.327+0000 7f997dc66640 1 -- 172.21.15.32:0/2322165103 wait complete. 2023-12-20T05:44:27.343 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T05:44:27.343 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T05:44:27.343 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T05:44:27.343 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T05:44:27.343 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T05:44:27.343 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T05:44:27.343 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T05:44:27.344 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T05:44:27.344 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T05:44:27.344 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:27.344 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T05:44:27.354 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T05:44:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T05:44:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T05:44:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:44:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T05:44:27.354 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 37792sec preferred_lft 37792sec 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T05:44:27.355 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T05:44:27.356 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:44:27.357 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T05:44:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T05:44:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T05:44:27.357 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T05:44:27.357 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:27.357 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T05:44:27.418 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T05:44:27.421 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:27.421 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T05:44:27.473 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T05:44:27.473 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T05:44:27.473 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T05:44:27.474 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T05:44:27.474 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T05:44:27.474 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T05:44:27.474 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:27.474 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T05:44:27.474 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T05:44:27.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T05:44:27.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T05:44:27.474 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T05:44:27.475 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T05:44:27.571 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:27.571 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T05:44:27.581 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T05:44:27.581 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T05:44:27.581 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T05:44:27.581 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:27.582 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T05:44:27.644 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T05:44:27.644 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T05:44:27.696 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T05:44:27.696 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T05:44:27.696 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T05:44:27.747 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T05:44:27.747 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:27.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T05:44:27.842 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T05:44:27.843 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T05:44:27.857 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T05:44:27.858 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T05:44:27.858 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T05:44:27.858 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T05:44:27.858 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T05:44:27.861 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T05:44:27.861 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:27.862 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T05:44:27.882 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:27.883 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T05:44:27.940 INFO:teuthology.orchestra.run.smithi032.stdout:client8078 2023-12-20T05:44:27.941 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T05:44:27.941 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T05:44:27.992 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T05:44:28.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f499759e640 1 Processor -- start 2023-12-20T05:44:28.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f499759e640 1 -- start start 2023-12-20T05:44:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f499759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4998161500 0x7f4998153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f499759e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49980cd4d0 con 0x7f499806b630 2023-12-20T05:44:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f499759e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f49980ce4e0 con 0x7f49980b8480 2023-12-20T05:44:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f499759e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f49980cd720 con 0x7f4998161500 2023-12-20T05:44:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f498ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f499806b630 0x7f4998160df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45238/0 (socket says 172.21.15.32:45238) 2023-12-20T05:44:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f498ffff640 1 -- 172.21.15.32:0/1967388546 learned_addr learned my addr 172.21.15.32:0/1967388546 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:28.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f4996d9d640 1 --2- 172.21.15.32:0/1967388546 >> v2:172.21.15.32:3300/0 conn(0x7f4998161500 0x7f4998153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f4996d9d640 1 -- 172.21.15.32:0/1967388546 >> v1:172.21.15.39:6789/0 conn(0x7f49980b8480 legacy=0x7f499806af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f4996d9d640 1 -- 172.21.15.32:0/1967388546 >> v1:172.21.15.32:6789/0 conn(0x7f499806b630 legacy=0x7f4998160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f4996d9d640 1 -- 172.21.15.32:0/1967388546 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f49980cda00 con 0x7f4998161500 2023-12-20T05:44:28.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.183+0000 7f4995d9b640 1 -- 172.21.15.32:0/1967388546 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 978598236 0 0) 0x7f49980cd4d0 con 0x7f499806b630 2023-12-20T05:44:28.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.187+0000 7f4996d9d640 1 --2- 172.21.15.32:0/1967388546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4998161500 0x7f4998153610 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f4990009d90 tx=0x7f499001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54bc7e54f323e47 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:28.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.187+0000 7f4995d9b640 1 -- 172.21.15.32:0/1967388546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900a57f0 con 0x7f4998161500 2023-12-20T05:44:28.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.187+0000 7f4995d9b640 1 -- 172.21.15.32:0/1967388546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f49900a5950 con 0x7f4998161500 2023-12-20T05:44:28.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.187+0000 7f4995d9b640 1 -- 172.21.15.32:0/1967388546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f49900a5ce0 con 0x7f4998161500 2023-12-20T05:44:28.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.187+0000 7f499759e640 1 -- 172.21.15.32:0/1967388546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4998161500 msgr2=0x7f4998153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.187+0000 7f499759e640 1 --2- 172.21.15.32:0/1967388546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4998161500 0x7f4998153610 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f4990009d90 tx=0x7f499001c950 comp rx=0 tx=0).stop 2023-12-20T05:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.187+0000 7f499759e640 1 -- 172.21.15.32:0/1967388546 shutdown_connections 2023-12-20T05:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.187+0000 7f499759e640 1 --2- 172.21.15.32:0/1967388546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4998161500 0x7f4998153610 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.187+0000 7f499759e640 1 -- 172.21.15.32:0/1967388546 >> 172.21.15.32:0/1967388546 conn(0x7f49980d2a50 msgr2=0x7f49980d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.187+0000 7f499759e640 1 -- 172.21.15.32:0/1967388546 shutdown_connections 2023-12-20T05:44:28.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.187+0000 7f499759e640 1 -- 172.21.15.32:0/1967388546 wait complete. 2023-12-20T05:44:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499759e640 1 Processor -- start 2023-12-20T05:44:28.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499759e640 1 -- start start 2023-12-20T05:44:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499806b630 0x7f499814b110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49980b8480 0x7f49980cb4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:28.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499759e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4998161500 0x7f4998156690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499759e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49980cc5e0 con 0x7f499806b630 2023-12-20T05:44:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499759e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f49980cc020 con 0x7f49980b8480 2023-12-20T05:44:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499759e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f49980cc300 con 0x7f4998161500 2023-12-20T05:44:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499806b630 0x7f499814b110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499806b630 0x7f499814b110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59134/0 (socket says 172.21.15.32:59134) 2023-12-20T05:44:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499659c640 1 -- 172.21.15.32:0/236272782 learned_addr learned my addr 172.21.15.32:0/236272782 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f498ffff640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49980b8480 0x7f49980cb4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:28.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f4996d9d640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4998161500 0x7f4998156690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499659c640 1 -- 172.21.15.32:0/236272782 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4998161500 msgr2=0x7f4998156690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499659c640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4998161500 0x7f4998156690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499659c640 1 -- 172.21.15.32:0/236272782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49980b8480 msgr2=0x7f49980cb4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499659c640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49980b8480 0x7f49980cb4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499659c640 1 -- 172.21.15.32:0/236272782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4990009a70 con 0x7f499806b630 2023-12-20T05:44:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f498ffff640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49980b8480 0x7f49980cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499659c640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499806b630 0x7f499814b110 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f498800c500 tx=0x7f498800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f498effd640 1 -- 172.21.15.32:0/236272782 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988005220 con 0x7f499806b630 2023-12-20T05:44:28.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f498effd640 1 -- 172.21.15.32:0/236272782 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4988005380 con 0x7f499806b630 2023-12-20T05:44:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499759e640 1 -- 172.21.15.32:0/236272782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4998155e90 con 0x7f499806b630 2023-12-20T05:44:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f498effd640 1 -- 172.21.15.32:0/236272782 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4988007270 con 0x7f499806b630 2023-12-20T05:44:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.191+0000 7f499759e640 1 -- 172.21.15.32:0/236272782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f49980cba30 con 0x7f499806b630 2023-12-20T05:44:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.195+0000 7f499759e640 1 -- 172.21.15.32:0/236272782 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4998155cf0 con 0x7f499806b630 2023-12-20T05:44:28.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.195+0000 7f498effd640 1 -- 172.21.15.32:0/236272782 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f49880073d0 con 0x7f499806b630 2023-12-20T05:44:28.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.195+0000 7f498effd640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4978065bd0 0x7f4978068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:28.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.195+0000 7f498effd640 1 -- 172.21.15.32:0/236272782 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4988082c10 con 0x7f499806b630 2023-12-20T05:44:28.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.195+0000 7f498ffff640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4978065bd0 0x7f4978068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:28.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.195+0000 7f498ffff640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4978065bd0 0x7f4978068090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49980ccc10 tx=0x7f4980007c60 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:28.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.199+0000 7f498effd640 1 -- 172.21.15.32:0/236272782 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4998155cf0 con 0x7f499806b630 2023-12-20T05:44:28.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.379+0000 7f499759e640 1 -- 172.21.15.32:0/236272782 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f49980b8860 con 0x7f4978065bd0 2023-12-20T05:44:28.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f498effd640 1 -- 172.21.15.32:0/236272782 <== mgr.9201 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f4980007270 con 0x7f4978065bd0 2023-12-20T05:44:28.380 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T05:44:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T05:44:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T05:44:28.380 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T05:44:28.380 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 -- 172.21.15.32:0/236272782 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4978065bd0 msgr2=0x7f4978068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4978065bd0 0x7f4978068090 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f49980ccc10 tx=0x7f4980007c60 comp rx=0 tx=0).stop 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 -- 172.21.15.32:0/236272782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499806b630 msgr2=0x7f499814b110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499806b630 0x7f499814b110 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f498800c500 tx=0x7f498800c9c0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 -- 172.21.15.32:0/236272782 shutdown_connections 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4978065bd0 0x7f4978068090 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4998161500 0x7f4998156690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f49980b8480 0x7f49980cb4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 --2- 172.21.15.32:0/236272782 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f499806b630 0x7f499814b110 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 -- 172.21.15.32:0/236272782 >> 172.21.15.32:0/236272782 conn(0x7f49980d2a50 msgr2=0x7f4998160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 -- 172.21.15.32:0/236272782 shutdown_connections 2023-12-20T05:44:28.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.383+0000 7f499759e640 1 -- 172.21.15.32:0/236272782 wait complete. 2023-12-20T05:44:28.401 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T05:44:28.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.547+0000 7f42f2f2c640 1 Processor -- start 2023-12-20T05:44:28.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.547+0000 7f42f2f2c640 1 -- start start 2023-12-20T05:44:28.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42f2f2c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f42ec0b0760 0x7f42ec0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:28.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42f2f2c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ec0b1100 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42f2f2c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42ec0cde20 con 0x7f42ec157480 2023-12-20T05:44:28.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42f2f2c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42ec0c9f50 con 0x7f42ec0b0760 2023-12-20T05:44:28.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42f14c9640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f42ec0cc4b0 0x7f42ec0cc8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:45242/0 (socket says 172.21.15.32:45242) 2023-12-20T05:44:28.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42f14c9640 1 -- 172.21.15.32:0/1074723547 learned_addr learned my addr 172.21.15.32:0/1074723547 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:28.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42ebfff640 1 --2- 172.21.15.32:0/1074723547 >> v2:172.21.15.39:3300/0 conn(0x7f42ec0b0760 0x7f42ec0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:28.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42eb7fe640 1 -- 172.21.15.32:0/1074723547 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 454481236 0 0) 0x7f42ec0b1100 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42eb7fe640 1 -- 172.21.15.32:0/1074723547 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42dc003530 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42eb7fe640 1 -- 172.21.15.32:0/1074723547 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1873978774 0 0) 0x7f42dc003530 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42ebfff640 1 -- 172.21.15.32:0/1074723547 >> v1:172.21.15.39:6789/0 conn(0x7f42ec157480 legacy=0x7f42ec0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42ebfff640 1 -- 172.21.15.32:0/1074723547 >> v1:172.21.15.32:6789/0 conn(0x7f42ec0cc4b0 legacy=0x7f42ec0cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.551+0000 7f42ebfff640 1 -- 172.21.15.32:0/1074723547 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42ec0c8e30 con 0x7f42ec0b0760 2023-12-20T05:44:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42ebfff640 1 --2- 172.21.15.32:0/1074723547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42ec0b0760 0x7f42ec0b0bc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f42d80097c0 tx=0x7f42d801c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b7c7adc74da81bdf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42eb7fe640 1 -- 172.21.15.32:0/1074723547 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e4001a00 con 0x7f42ec0b0760 2023-12-20T05:44:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42eb7fe640 1 -- 172.21.15.32:0/1074723547 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d80a7050 con 0x7f42ec0b0760 2023-12-20T05:44:28.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42eb7fe640 1 -- 172.21.15.32:0/1074723547 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d80ab620 con 0x7f42ec0b0760 2023-12-20T05:44:28.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42f2f2c640 1 -- 172.21.15.32:0/1074723547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42ec0b0760 msgr2=0x7f42ec0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42f2f2c640 1 --2- 172.21.15.32:0/1074723547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42ec0b0760 0x7f42ec0b0bc0 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f42d80097c0 tx=0x7f42d801c2e0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42f2f2c640 1 -- 172.21.15.32:0/1074723547 shutdown_connections 2023-12-20T05:44:28.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42f2f2c640 1 --2- 172.21.15.32:0/1074723547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42ec0b0760 0x7f42ec0b0bc0 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42f2f2c640 1 -- 172.21.15.32:0/1074723547 >> 172.21.15.32:0/1074723547 conn(0x7f42ec06a510 msgr2=0x7f42ec0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:28.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42f2f2c640 1 -- 172.21.15.32:0/1074723547 shutdown_connections 2023-12-20T05:44:28.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42f2f2c640 1 -- 172.21.15.32:0/1074723547 wait complete. 2023-12-20T05:44:28.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42f2f2c640 1 Processor -- start 2023-12-20T05:44:28.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.555+0000 7f42f2f2c640 1 -- start start 2023-12-20T05:44:28.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42f2f2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42ec0b0760 0x7f42ec153090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42f2f2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ec0cc4b0 0x7f42ec151420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42f2f2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42ec157480 0x7f42ec1547c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42f2f2c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec151960 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42f2f2c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec153720 con 0x7f42ec0b0760 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42f2f2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42ec153a00 con 0x7f42ec157480 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ec0cc4b0 0x7f42ec151420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ec0cc4b0 0x7f42ec151420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59140/0 (socket says 172.21.15.32:59140) 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42ebfff640 1 -- 172.21.15.32:0/383658150 learned_addr learned my addr 172.21.15.32:0/383658150 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42f0cc8640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42ec0b0760 0x7f42ec153090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42f14c9640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42ec157480 0x7f42ec1547c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42ebfff640 1 -- 172.21.15.32:0/383658150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42ec157480 msgr2=0x7f42ec1547c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42ebfff640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42ec157480 0x7f42ec1547c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42ebfff640 1 -- 172.21.15.32:0/383658150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42ec0b0760 msgr2=0x7f42ec153090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:28.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42ebfff640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42ec0b0760 0x7f42ec153090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42ebfff640 1 -- 172.21.15.32:0/383658150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d8009420 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42ebfff640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ec0cc4b0 0x7f42ec151420 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f42d80ae040 tx=0x7f42d8019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42e97fa640 1 -- 172.21.15.32:0/383658150 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d8004020 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42e97fa640 1 -- 172.21.15.32:0/383658150 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d8004180 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42e97fa640 1 -- 172.21.15.32:0/383658150 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d80044f0 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42f2f2c640 1 -- 172.21.15.32:0/383658150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42ec153ce0 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42f2f2c640 1 -- 172.21.15.32:0/383658150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42ec1542a0 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.559+0000 7f42f2f2c640 1 -- 172.21.15.32:0/383658150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42ec154490 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.563+0000 7f42e97fa640 1 -- 172.21.15.32:0/383658150 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 94) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f42d80b2040 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.563+0000 7f42e97fa640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42d0065b80 0x7f42d0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:28.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.563+0000 7f42e97fa640 1 -- 172.21.15.32:0/383658150 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f42d811d150 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.563+0000 7f42f0cc8640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42d0065b80 0x7f42d0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:28.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.563+0000 7f42f0cc8640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42d0065b80 0x7f42d0068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f42e0007fd0 tx=0x7f42e0009040 comp rx=0 tx=0).ready entity=mgr.9201 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:28.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.567+0000 7f42e97fa640 1 -- 172.21.15.32:0/383658150 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f42d80eaa80 con 0x7f42ec0cc4b0 2023-12-20T05:44:28.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:28.763+0000 7f42f2f2c640 1 -- 172.21.15.32:0/383658150 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f42ec04e000 con 0x7f42ec0cc4b0 2023-12-20T05:44:29.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.131+0000 7f42e97fa640 1 -- 172.21.15.32:0/383658150 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 95) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f42d80eae50 con 0x7f42ec0cc4b0 2023-12-20T05:44:29.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42e97fa640 1 -- 172.21.15.32:0/383658150 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v95) v1 ==== 92+0+0 (secure 0 0 0) 0x7f42d80f21a0 con 0x7f42ec0cc4b0 2023-12-20T05:44:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 -- 172.21.15.32:0/383658150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42d0065b80 msgr2=0x7f42d0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42d0065b80 0x7f42d0068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f42e0007fd0 tx=0x7f42e0009040 comp rx=0 tx=0).stop 2023-12-20T05:44:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 -- 172.21.15.32:0/383658150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ec0cc4b0 msgr2=0x7f42ec151420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ec0cc4b0 0x7f42ec151420 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f42d80ae040 tx=0x7f42d8019080 comp rx=0 tx=0).stop 2023-12-20T05:44:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 -- 172.21.15.32:0/383658150 shutdown_connections 2023-12-20T05:44:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f42d0065b80 0x7f42d0068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42ec157480 0x7f42ec1547c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42ec0cc4b0 0x7f42ec151420 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 --2- 172.21.15.32:0/383658150 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42ec0b0760 0x7f42ec153090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 -- 172.21.15.32:0/383658150 >> 172.21.15.32:0/383658150 conn(0x7f42ec06a510 msgr2=0x7f42ec159030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 -- 172.21.15.32:0/383658150 shutdown_connections 2023-12-20T05:44:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.135+0000 7f42f2f2c640 1 -- 172.21.15.32:0/383658150 wait complete. 2023-12-20T05:44:29.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T05:44:29.306 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:29.306+0000 7fdf0804d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T05:44:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f3b37b640 1 Processor -- start 2023-12-20T05:44:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f3b37b640 1 -- start start 2023-12-20T05:44:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f3b37b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6f34164670 0x7f6f340ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f3b37b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f340cb0b0 con 0x7f6f3406da20 2023-12-20T05:44:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f3b37b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f340ca810 con 0x7f6f34070230 2023-12-20T05:44:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f3b37b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f340ccac0 con 0x7f6f34164670 2023-12-20T05:44:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f38916640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6f3406da20 0x7f6f340cc3b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46610/0 (socket says 172.21.15.32:46610) 2023-12-20T05:44:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f38916640 1 -- 172.21.15.32:0/4024760210 learned_addr learned my addr 172.21.15.32:0/4024760210 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:29.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f39117640 1 --2- 172.21.15.32:0/4024760210 >> v2:172.21.15.39:3300/0 conn(0x7f6f34164670 0x7f6f340ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f2bfff640 1 -- 172.21.15.32:0/4024760210 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3150144494 0 0) 0x7f6f340cb0b0 con 0x7f6f3406da20 2023-12-20T05:44:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f2bfff640 1 -- 172.21.15.32:0/4024760210 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f20003530 con 0x7f6f3406da20 2023-12-20T05:44:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f2bfff640 1 -- 172.21.15.32:0/4024760210 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1021251769 0 0) 0x7f6f340ca810 con 0x7f6f34070230 2023-12-20T05:44:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f2bfff640 1 -- 172.21.15.32:0/4024760210 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f340cb0b0 con 0x7f6f34070230 2023-12-20T05:44:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f39117640 1 -- 172.21.15.32:0/4024760210 >> v1:172.21.15.39:6789/0 conn(0x7f6f34070230 legacy=0x7f6f34070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f39117640 1 -- 172.21.15.32:0/4024760210 >> v1:172.21.15.32:6789/0 conn(0x7f6f3406da20 legacy=0x7f6f340cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.315+0000 7f6f39117640 1 -- 172.21.15.32:0/4024760210 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f340caaf0 con 0x7f6f34164670 2023-12-20T05:44:29.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.319+0000 7f6f39117640 1 --2- 172.21.15.32:0/4024760210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f34164670 0x7f6f340ca1a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6f300097c0 tx=0x7f6f3001c330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2f9b95e9e0b0c7c6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:29.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.319+0000 7f6f2bfff640 1 -- 172.21.15.32:0/4024760210 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f300ab600 con 0x7f6f34164670 2023-12-20T05:44:29.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.319+0000 7f6f2bfff640 1 -- 172.21.15.32:0/4024760210 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f300a7050 con 0x7f6f34164670 2023-12-20T05:44:29.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.319+0000 7f6f2bfff640 1 -- 172.21.15.32:0/4024760210 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f300ab960 con 0x7f6f34164670 2023-12-20T05:44:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.319+0000 7f6f3b37b640 1 -- 172.21.15.32:0/4024760210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f34164670 msgr2=0x7f6f340ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.319+0000 7f6f3b37b640 1 --2- 172.21.15.32:0/4024760210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f34164670 0x7f6f340ca1a0 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f6f300097c0 tx=0x7f6f3001c330 comp rx=0 tx=0).stop 2023-12-20T05:44:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.323+0000 7f6f3b37b640 1 -- 172.21.15.32:0/4024760210 shutdown_connections 2023-12-20T05:44:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.323+0000 7f6f3b37b640 1 --2- 172.21.15.32:0/4024760210 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f34164670 0x7f6f340ca1a0 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.323+0000 7f6f3b37b640 1 -- 172.21.15.32:0/4024760210 >> 172.21.15.32:0/4024760210 conn(0x7f6f3406a510 msgr2=0x7f6f340ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.323+0000 7f6f3b37b640 1 -- 172.21.15.32:0/4024760210 shutdown_connections 2023-12-20T05:44:29.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.323+0000 7f6f3b37b640 1 -- 172.21.15.32:0/4024760210 wait complete. 2023-12-20T05:44:29.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.323+0000 7f6f3b37b640 1 Processor -- start 2023-12-20T05:44:29.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.323+0000 7f6f3b37b640 1 -- start start 2023-12-20T05:44:29.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f3b37b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f3406da20 0x7f6f34151990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:29.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f3b37b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f34070230 0x7f6f340cd3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:29.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f3b37b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f34164670 0x7f6f341509d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:29.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f3b37b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340cdeb0 con 0x7f6f34070230 2023-12-20T05:44:29.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f3b37b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340ce190 con 0x7f6f34164670 2023-12-20T05:44:29.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f3b37b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f340ce470 con 0x7f6f3406da20 2023-12-20T05:44:29.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f38916640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f34070230 0x7f6f340cd3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:29.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f38916640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f34070230 0x7f6f340cd3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58052/0 (socket says 172.21.15.32:58052) 2023-12-20T05:44:29.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f38916640 1 -- 172.21.15.32:0/722871969 learned_addr learned my addr 172.21.15.32:0/722871969 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:29.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f39918640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f34164670 0x7f6f341509d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:29.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f39117640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f3406da20 0x7f6f34151990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f38916640 1 -- 172.21.15.32:0/722871969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f3406da20 msgr2=0x7f6f34151990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f38916640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f3406da20 0x7f6f34151990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f38916640 1 -- 172.21.15.32:0/722871969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f34164670 msgr2=0x7f6f341509d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f38916640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f34164670 0x7f6f341509d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f38916640 1 -- 172.21.15.32:0/722871969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f30009420 con 0x7f6f34070230 2023-12-20T05:44:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f38916640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f34070230 0x7f6f340cd3d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400c420 tx=0x7f6f2400c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f29ffb640 1 -- 172.21.15.32:0/722871969 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f24005cf0 con 0x7f6f34070230 2023-12-20T05:44:29.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f29ffb640 1 -- 172.21.15.32:0/722871969 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f24005e50 con 0x7f6f34070230 2023-12-20T05:44:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f29ffb640 1 -- 172.21.15.32:0/722871969 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f240072d0 con 0x7f6f34070230 2023-12-20T05:44:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f3b37b640 1 -- 172.21.15.32:0/722871969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f340ce750 con 0x7f6f34070230 2023-12-20T05:44:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f3b37b640 1 -- 172.21.15.32:0/722871969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f34151ed0 con 0x7f6f34070230 2023-12-20T05:44:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.327+0000 7f6f3b37b640 1 -- 172.21.15.32:0/722871969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f340cec70 con 0x7f6f34070230 2023-12-20T05:44:29.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.331+0000 7f6f29ffb640 1 -- 172.21.15.32:0/722871969 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 95) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f6f24007430 con 0x7f6f34070230 2023-12-20T05:44:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.331+0000 7f6f29ffb640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6f18065bd0 0x7f6f18068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.331+0000 7f6f29ffb640 1 -- 172.21.15.32:0/722871969 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f6f24082950 con 0x7f6f34070230 2023-12-20T05:44:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.331+0000 7f6f39117640 1 -- 172.21.15.32:0/722871969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6f18065bd0 msgr2=0x7f6f18068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:29.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.331+0000 7f6f39117640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6f18065bd0 0x7f6f18068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:29.329 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:29.331+0000 7f9a51a55180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T05:44:29.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.335+0000 7f6f29ffb640 1 -- 172.21.15.32:0/722871969 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6f340cec70 con 0x7f6f34070230 2023-12-20T05:44:29.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.499+0000 7f6f3b37b640 1 -- 172.21.15.32:0/722871969 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f6f340cec70 con 0x7f6f34070230 2023-12-20T05:44:29.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f29ffb640 1 -- 172.21.15.32:0/722871969 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v59)=0 v59) v1 ==== 138+0+0 (secure 0 0 0) 0x7f6f340cec70 con 0x7f6f34070230 2023-12-20T05:44:29.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 -- 172.21.15.32:0/722871969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6f18065bd0 msgr2=0x7f6f18068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:29.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6f18065bd0 0x7f6f18068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 -- 172.21.15.32:0/722871969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f34070230 msgr2=0x7f6f340cd3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f34070230 0x7f6f340cd3d0 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7f6f2400c420 tx=0x7f6f2400c8e0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 -- 172.21.15.32:0/722871969 shutdown_connections 2023-12-20T05:44:29.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f6f18065bd0 0x7f6f18068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f34164670 0x7f6f341509d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f34070230 0x7f6f340cd3d0 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 --2- 172.21.15.32:0/722871969 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f3406da20 0x7f6f34151990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 -- 172.21.15.32:0/722871969 >> 172.21.15.32:0/722871969 conn(0x7f6f3406a510 msgr2=0x7f6f34162d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:29.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 -- 172.21.15.32:0/722871969 shutdown_connections 2023-12-20T05:44:29.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.507+0000 7f6f3b37b640 1 -- 172.21.15.32:0/722871969 wait complete. 2023-12-20T05:44:29.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T05:44:29.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.675+0000 7fead6d60640 1 Processor -- start 2023-12-20T05:44:29.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.675+0000 7fead6d60640 1 -- start start 2023-12-20T05:44:29.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead6d60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fead015cef0 0x7fead0161510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:29.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead6d60640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fead00cde70 0x7fead00ce2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:29.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead6d60640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fead00cdbd0 con 0x7fead0071760 2023-12-20T05:44:29.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead6d60640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fead00cd7b0 con 0x7fead00cde70 2023-12-20T05:44:29.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead6d60640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead0071be0 con 0x7fead015cef0 2023-12-20T05:44:29.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead52fd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fead00cde70 0x7fead00ce2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:29.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead52fd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fead00cde70 0x7fead00ce2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58066/0 (socket says 172.21.15.32:58066) 2023-12-20T05:44:29.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead52fd640 1 -- 172.21.15.32:0/2175431063 learned_addr learned my addr 172.21.15.32:0/2175431063 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:29.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead4afc640 1 --2- 172.21.15.32:0/2175431063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fead015cef0 0x7fead0161510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:29.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead52fd640 1 -- 172.21.15.32:0/2175431063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fead015cef0 msgr2=0x7fead0161510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:29.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead52fd640 1 --2- 172.21.15.32:0/2175431063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fead015cef0 0x7fead0161510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead52fd640 1 -- 172.21.15.32:0/2175431063 >> v1:172.21.15.32:6789/0 conn(0x7fead0071760 legacy=0x7fead00ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7feacf7fe640 1 -- 172.21.15.32:0/2175431063 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2882347735 0 0) 0x7fead00cdbd0 con 0x7fead0071760 2023-12-20T05:44:29.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead52fd640 1 -- 172.21.15.32:0/2175431063 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fead00cd460 con 0x7fead00cde70 2023-12-20T05:44:29.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.679+0000 7fead52fd640 1 --2- 172.21.15.32:0/2175431063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead00cde70 0x7fead00ce2e0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7feac8009e50 tx=0x7feac801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b83af41aebf72af8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:29.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7feacf7fe640 1 -- 172.21.15.32:0/2175431063 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac80a5810 con 0x7fead00cde70 2023-12-20T05:44:29.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7feacf7fe640 1 -- 172.21.15.32:0/2175431063 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac80a5970 con 0x7fead00cde70 2023-12-20T05:44:29.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7feacf7fe640 1 -- 172.21.15.32:0/2175431063 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac80a5d00 con 0x7fead00cde70 2023-12-20T05:44:29.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7fead6d60640 1 -- 172.21.15.32:0/2175431063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead00cde70 msgr2=0x7fead00ce2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7fead6d60640 1 --2- 172.21.15.32:0/2175431063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead00cde70 0x7fead00ce2e0 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7feac8009e50 tx=0x7feac801c950 comp rx=0 tx=0).stop 2023-12-20T05:44:29.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7fead6d60640 1 -- 172.21.15.32:0/2175431063 shutdown_connections 2023-12-20T05:44:29.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7fead6d60640 1 --2- 172.21.15.32:0/2175431063 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead00cde70 0x7fead00ce2e0 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7fead6d60640 1 --2- 172.21.15.32:0/2175431063 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fead015cef0 0x7fead0161510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7fead6d60640 1 -- 172.21.15.32:0/2175431063 >> 172.21.15.32:0/2175431063 conn(0x7fead006b230 msgr2=0x7fead006b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:29.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7fead6d60640 1 -- 172.21.15.32:0/2175431063 shutdown_connections 2023-12-20T05:44:29.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7fead6d60640 1 -- 172.21.15.32:0/2175431063 wait complete. 2023-12-20T05:44:29.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.683+0000 7fead6d60640 1 Processor -- start 2023-12-20T05:44:29.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead6d60640 1 -- start start 2023-12-20T05:44:29.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead6d60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fead0071760 0x7fead015e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:29.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead6d60640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead00cde70 0x7fead015fda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:29.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead6d60640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead015cef0 0x7fead01f8ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:29.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead6d60640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead0156640 con 0x7fead015cef0 2023-12-20T05:44:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead6d60640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fead015e950 con 0x7fead00cde70 2023-12-20T05:44:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead6d60640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fead0154520 con 0x7fead0071760 2023-12-20T05:44:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead52fd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead015cef0 0x7fead01f8ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead52fd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead015cef0 0x7fead01f8ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58080/0 (socket says 172.21.15.32:58080) 2023-12-20T05:44:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead52fd640 1 -- 172.21.15.32:0/3635097364 learned_addr learned my addr 172.21.15.32:0/3635097364 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead4afc640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fead0071760 0x7fead015e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7feacffff640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead00cde70 0x7fead015fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead52fd640 1 -- 172.21.15.32:0/3635097364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fead0071760 msgr2=0x7fead015e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead52fd640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fead0071760 0x7fead015e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead52fd640 1 -- 172.21.15.32:0/3635097364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead00cde70 msgr2=0x7fead015fda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead52fd640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead00cde70 0x7fead015fda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead52fd640 1 -- 172.21.15.32:0/3635097364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feac8009b30 con 0x7fead015cef0 2023-12-20T05:44:29.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead52fd640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead015cef0 0x7fead01f8ac0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7feac80af040 tx=0x7feac80196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:29.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7feacd7fa640 1 -- 172.21.15.32:0/3635097364 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac80b8670 con 0x7fead015cef0 2023-12-20T05:44:29.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7feacd7fa640 1 -- 172.21.15.32:0/3635097364 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feac80b87d0 con 0x7fead015cef0 2023-12-20T05:44:29.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7feacd7fa640 1 -- 172.21.15.32:0/3635097364 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feac80b8b80 con 0x7fead015cef0 2023-12-20T05:44:29.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead6d60640 1 -- 172.21.15.32:0/3635097364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fead0154800 con 0x7fead015cef0 2023-12-20T05:44:29.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.687+0000 7fead6d60640 1 -- 172.21.15.32:0/3635097364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fead0156b70 con 0x7fead015cef0 2023-12-20T05:44:29.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.691+0000 7fead6d60640 1 -- 172.21.15.32:0/3635097364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fead004e000 con 0x7fead015cef0 2023-12-20T05:44:29.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.691+0000 7feacd7fa640 1 -- 172.21.15.32:0/3635097364 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 95) v1 ==== 85347+0+0 (secure 0 0 0) 0x7feac80b8ce0 con 0x7fead015cef0 2023-12-20T05:44:29.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.691+0000 7feacd7fa640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feab4065bd0 0x7feab4068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:29.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.691+0000 7feacd7fa640 1 -- 172.21.15.32:0/3635097364 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7feac811e800 con 0x7fead015cef0 2023-12-20T05:44:29.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.691+0000 7fead4afc640 1 -- 172.21.15.32:0/3635097364 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feab4065bd0 msgr2=0x7feab4068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:29.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.691+0000 7fead4afc640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feab4065bd0 0x7feab4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:29.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.695+0000 7feacd7fa640 1 -- 172.21.15.32:0/3635097364 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feac80ec130 con 0x7fead015cef0 2023-12-20T05:44:29.744 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:29.742+0000 7fdf0804d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T05:44:29.788 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:29.791+0000 7f9a51a55180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T05:44:29.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.863+0000 7fead6d60640 1 -- 172.21.15.32:0/3635097364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fead015ddf0 con 0x7fead015cef0 2023-12-20T05:44:29.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.871+0000 7feacd7fa640 1 -- 172.21.15.32:0/3635097364 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v60)=0 v60) v1 ==== 133+0+0 (secure 0 0 0) 0x7feac80ef750 con 0x7fead015cef0 2023-12-20T05:44:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 -- 172.21.15.32:0/3635097364 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feab4065bd0 msgr2=0x7feab4068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feab4065bd0 0x7feab4068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 -- 172.21.15.32:0/3635097364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead015cef0 msgr2=0x7fead01f8ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead015cef0 0x7fead01f8ac0 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7feac80af040 tx=0x7feac80196f0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 -- 172.21.15.32:0/3635097364 shutdown_connections 2023-12-20T05:44:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7feab4065bd0 0x7feab4068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fead015cef0 0x7fead01f8ac0 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fead00cde70 0x7fead015fda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 --2- 172.21.15.32:0/3635097364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fead0071760 0x7fead015e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 -- 172.21.15.32:0/3635097364 >> 172.21.15.32:0/3635097364 conn(0x7fead006b230 msgr2=0x7fead015c1c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 -- 172.21.15.32:0/3635097364 shutdown_connections 2023-12-20T05:44:29.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:29.875+0000 7fead6d60640 1 -- 172.21.15.32:0/3635097364 wait complete. 2023-12-20T05:44:29.892 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:29.893 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-20T05:44:29.902 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:29.902+0000 7fdf0804d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T05:44:29.908 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:44:29.908 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1/test_restart) 2023-12-20T05:44:29.948 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:29.951+0000 7f9a51a55180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T05:44:29.959 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add /snap_test_dir1/test_restart 1M 2023-12-20T05:44:30.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.159+0000 7f4ac63b3640 1 Processor -- start 2023-12-20T05:44:30.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.159+0000 7f4ac63b3640 1 -- start start 2023-12-20T05:44:30.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.159+0000 7f4ac63b3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4ac015abb0 0x7f4ac00caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:30.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.159+0000 7f4ac63b3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac015b6a0 con 0x7f4ac00b0cb0 2023-12-20T05:44:30.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.159+0000 7f4ac63b3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4ac006d8c0 con 0x7f4ac0070230 2023-12-20T05:44:30.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.159+0000 7f4ac63b3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4ac006e160 con 0x7f4ac015abb0 2023-12-20T05:44:30.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.159+0000 7f4abf7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4ac00b0cb0 0x7f4ac015bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46630/0 (socket says 172.21.15.32:46630) 2023-12-20T05:44:30.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.159+0000 7f4abf7fe640 1 -- 172.21.15.32:0/3743314359 learned_addr learned my addr 172.21.15.32:0/3743314359 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:44:30.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.159+0000 7f4abffff640 1 --2- 172.21.15.32:0/3743314359 >> v2:172.21.15.39:3300/0 conn(0x7f4ac015abb0 0x7f4ac00caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.159+0000 7f4abeffd640 1 -- 172.21.15.32:0/3743314359 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 820958806 0 0) 0x7f4ac015b6a0 con 0x7f4ac00b0cb0 2023-12-20T05:44:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.159+0000 7f4abeffd640 1 -- 172.21.15.32:0/3743314359 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4aa8003530 con 0x7f4ac00b0cb0 2023-12-20T05:44:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.163+0000 7f4abeffd640 1 -- 172.21.15.32:0/3743314359 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 101094942 0 0) 0x7f4ac006d8c0 con 0x7f4ac0070230 2023-12-20T05:44:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.163+0000 7f4abeffd640 1 -- 172.21.15.32:0/3743314359 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4ac015b6a0 con 0x7f4ac0070230 2023-12-20T05:44:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.163+0000 7f4abffff640 1 -- 172.21.15.32:0/3743314359 >> v1:172.21.15.39:6789/0 conn(0x7f4ac0070230 legacy=0x7f4ac006c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.163+0000 7f4abffff640 1 -- 172.21.15.32:0/3743314359 >> v1:172.21.15.32:6789/0 conn(0x7f4ac00b0cb0 legacy=0x7f4ac015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.163+0000 7f4abffff640 1 -- 172.21.15.32:0/3743314359 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ac006dba0 con 0x7f4ac015abb0 2023-12-20T05:44:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.163+0000 7f4abffff640 1 --2- 172.21.15.32:0/3743314359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ac015abb0 0x7f4ac00caaf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00097c0 tx=0x7f4ab001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ab2ab3e8546c831 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.163+0000 7f4abeffd640 1 -- 172.21.15.32:0/3743314359 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00ab680 con 0x7f4ac015abb0 2023-12-20T05:44:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.163+0000 7f4abeffd640 1 -- 172.21.15.32:0/3743314359 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab00ab7e0 con 0x7f4ac015abb0 2023-12-20T05:44:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.163+0000 7f4abeffd640 1 -- 172.21.15.32:0/3743314359 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00abb70 con 0x7f4ac015abb0 2023-12-20T05:44:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 -- 172.21.15.32:0/3743314359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ac015abb0 msgr2=0x7f4ac00caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 --2- 172.21.15.32:0/3743314359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ac015abb0 0x7f4ac00caaf0 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00097c0 tx=0x7f4ab001c480 comp rx=0 tx=0).stop 2023-12-20T05:44:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 -- 172.21.15.32:0/3743314359 shutdown_connections 2023-12-20T05:44:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 --2- 172.21.15.32:0/3743314359 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ac015abb0 0x7f4ac00caaf0 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 -- 172.21.15.32:0/3743314359 >> 172.21.15.32:0/3743314359 conn(0x7f4ac006a510 msgr2=0x7f4ac00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 -- 172.21.15.32:0/3743314359 shutdown_connections 2023-12-20T05:44:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 -- 172.21.15.32:0/3743314359 wait complete. 2023-12-20T05:44:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 Processor -- start 2023-12-20T05:44:30.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 -- start start 2023-12-20T05:44:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac0070230 0x7f4ac00c93b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ac00b0cb0 0x7f4ac0143120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4ac015abb0 0x7f4ac0152330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac00c98f0 con 0x7f4ac0070230 2023-12-20T05:44:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac014ce50 con 0x7f4ac00b0cb0 2023-12-20T05:44:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac63b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4ac01469b0 con 0x7f4ac015abb0 2023-12-20T05:44:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4abffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac0070230 0x7f4ac00c93b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4abf7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ac00b0cb0 0x7f4ac0143120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4abffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac0070230 0x7f4ac00c93b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58088/0 (socket says 172.21.15.32:58088) 2023-12-20T05:44:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4abffff640 1 -- 172.21.15.32:0/1872260776 learned_addr learned my addr 172.21.15.32:0/1872260776 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.167+0000 7f4ac4950640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4ac015abb0 0x7f4ac0152330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abffff640 1 -- 172.21.15.32:0/1872260776 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4ac015abb0 msgr2=0x7f4ac0152330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abffff640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4ac015abb0 0x7f4ac0152330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abffff640 1 -- 172.21.15.32:0/1872260776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ac00b0cb0 msgr2=0x7f4ac0143120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abffff640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ac00b0cb0 0x7f4ac0143120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abffff640 1 -- 172.21.15.32:0/1872260776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ab0009420 con 0x7f4ac0070230 2023-12-20T05:44:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4ac4950640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4ac015abb0 0x7f4ac0152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T05:44:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abf7fe640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ac00b0cb0 0x7f4ac0143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T05:44:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abffff640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac0070230 0x7f4ac00c93b0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00ad040 tx=0x7f4ab001acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00bf610 con 0x7f4ac0070230 2023-12-20T05:44:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ab00bf770 con 0x7f4ac0070230 2023-12-20T05:44:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ab00bfb20 con 0x7f4ac0070230 2023-12-20T05:44:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4ac63b3640 1 -- 172.21.15.32:0/1872260776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4ac0146c90 con 0x7f4ac0070230 2023-12-20T05:44:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4ac63b3640 1 -- 172.21.15.32:0/1872260776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4ac0147250 con 0x7f4ac0070230 2023-12-20T05:44:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4ac63b3640 1 -- 172.21.15.32:0/1872260776 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4ac0147440 con 0x7f4ac0070230 2023-12-20T05:44:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 95) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f4ab00bfc80 con 0x7f4ac0070230 2023-12-20T05:44:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abcff9640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 0x7f4aac0680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(173..173 src has 1..173) v4 ==== 5737+0+0 (secure 0 0 0) 0x7f4ab011df10 con 0x7f4ac0070230 2023-12-20T05:44:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abf7fe640 1 -- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 msgr2=0x7f4aac0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.171+0000 7f4abf7fe640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 0x7f4aac0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T05:44:30.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.175+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4ab00eb6a0 con 0x7f4ac0070230 2023-12-20T05:44:30.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.355+0000 7f4ac63b3640 1 -- 172.21.15.32:0/1872260776 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4ac006cf00 con 0x7f4aac065c00 2023-12-20T05:44:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.375+0000 7f4abf7fe640 1 -- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 msgr2=0x7f4aac0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:30.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.375+0000 7f4abf7fe640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 0x7f4aac0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T05:44:30.406 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:30.406+0000 7fdf0804d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T05:44:30.461 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:30.463+0000 7f9a51a55180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T05:44:30.467 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:30.466+0000 7fdf0804d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T05:44:30.522 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:30.523+0000 7f9a51a55180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T05:44:30.525 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:30.522+0000 7fdf0804d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T05:44:30.581 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:30.583+0000 7f9a51a55180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T05:44:30.617 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:30.614+0000 7fdf0804d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T05:44:30.672 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:30.675+0000 7f9a51a55180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T05:44:30.676 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:30.674+0000 7fdf0804d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T05:44:30.732 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:30.735+0000 7f9a51a55180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T05:44:30.764 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:30.762+0000 7fdf0804d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T05:44:30.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.779+0000 7f4abf7fe640 1 -- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 msgr2=0x7f4aac0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:30.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:30.779+0000 7f4abf7fe640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 0x7f4aac0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T05:44:30.821 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:30.823+0000 7f9a51a55180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T05:44:30.823 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:30.822+0000 7fdf0804d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T05:44:30.879 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:30.879+0000 7f9a51a55180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T05:44:30.972 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:30.970+0000 7fdf0804d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T05:44:31.030 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:31.031+0000 7f9a51a55180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T05:44:31.033 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:31.030+0000 7fdf0804d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T05:44:31.092 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:31.095+0000 7f9a51a55180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T05:44:31.281 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T05:44:31.281 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T05:44:31.282 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T05:44:31.290 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:31.290+0000 7fdf0804d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T05:44:31.341 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T05:44:31.341 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T05:44:31.341 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T05:44:31.349 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:31.351+0000 7f9a51a55180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T05:44:31.431 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:31.430+0000 7fdf0804d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:44:31.491 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:31.491+0000 7f9a51a55180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:44:31.563 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:31.558+0000 7fdf0804d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T05:44:31.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:31.583+0000 7f4abf7fe640 1 -- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 msgr2=0x7f4aac0680c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/38009 2023-12-20T05:44:31.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:31.583+0000 7f4abf7fe640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 0x7f4aac0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T05:44:31.623 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:31.623+0000 7f9a51a55180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T05:44:31.635 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:31.634+0000 7fdf0804d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T05:44:31.697 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:31.699+0000 7f9a51a55180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T05:44:31.776 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:31.774+0000 7fdf0804d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:44:31.840 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:31.843+0000 7f9a51a55180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:44:32.341 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:32.338+0000 7fdf0804d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T05:44:32.409 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:32.411+0000 7f9a51a55180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T05:44:32.429 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:32.426+0000 7fdf0804d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T05:44:32.497 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:32.499+0000 7f9a51a55180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T05:44:32.584 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:32.582+0000 7fdf0804d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T05:44:32.653 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:32.655+0000 7f9a51a55180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T05:44:32.671 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:32.670+0000 7fdf0804d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T05:44:32.741 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:32.743+0000 7f9a51a55180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T05:44:33.006 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:33.006+0000 7fdf0804d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T05:44:33.069 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T05:44:33.066+0000 7fdf0804d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T05:44:33.076 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.079+0000 7f9a51a55180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T05:44:33.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:33.131+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 96) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f4ab00eba70 con 0x7f4ac0070230 2023-12-20T05:44:33.143 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.143+0000 7f9a51a55180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T05:44:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:33.163+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 97) v1 ==== 84956+0+0 (secure 0 0 0) 0x7f4ab011be20 con 0x7f4ac0070230 2023-12-20T05:44:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:33.163+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 msgr2=0x7f4aac0680c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T05:44:33.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:33.163+0000 7f4abcff9640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 0x7f4aac0680c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:33.288 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.291+0000 7f99d364a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:33.288 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.291+0000 7f99d364a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:33.288 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.291+0000 7f99d364a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:33.288 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.291+0000 7f99d364a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:33.288 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.291+0000 7f99d364a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:33.289 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.291+0000 7f99d6e51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:33.289 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.291+0000 7f99d6e51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:33.289 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.291+0000 7f99d6e51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:33.289 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.291+0000 7f99d6e51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:33.289 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:44:33.291+0000 7f99d6e51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:44:34.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.167+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 <== mon.0 v2:172.21.15.32:3300/0 9 ==== mgrmap(e 98) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f4ab00ec340 con 0x7f4ac0070230 2023-12-20T05:44:34.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.167+0000 7f4abcff9640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 0x7f4aac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:34.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.167+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "/snap_test_dir1/test_restart", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f4aac0273e0 con 0x7f4aac065c00 2023-12-20T05:44:34.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.171+0000 7f4abf7fe640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 0x7f4aac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:34.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.175+0000 7f4abf7fe640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 0x7f4aac068040 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4ac014d3a0 tx=0x7f4ab4007d80 comp rx=0 tx=0).ready entity=mgr.9259 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:34.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4abcff9640 1 -- 172.21.15.32:0/1872260776 <== mgr.9259 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+51 (secure 0 0 0) 0x7f4aac0273e0 con 0x7f4aac065c00 2023-12-20T05:44:34.240 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path /snap_test_dir1/test_restart 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 -- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 msgr2=0x7f4aac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 0x7f4aac068040 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7f4ac014d3a0 tx=0x7f4ab4007d80 comp rx=0 tx=0).stop 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 -- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac0070230 msgr2=0x7f4ac00c93b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac0070230 0x7f4ac00c93b0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7f4ab00ad040 tx=0x7f4ab001acc0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 -- 172.21.15.32:0/1872260776 shutdown_connections 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f4aac065c00 0x7f4aac068040 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4ac015abb0 0x7f4ac0152330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4ac00b0cb0 0x7f4ac0143120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 --2- 172.21.15.32:0/1872260776 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4ac0070230 0x7f4ac00c93b0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 -- 172.21.15.32:0/1872260776 >> 172.21.15.32:0/1872260776 conn(0x7f4ac006a510 msgr2=0x7f4ac00c4a70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 -- 172.21.15.32:0/1872260776 shutdown_connections 2023-12-20T05:44:34.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.243+0000 7f4ac63b3640 1 -- 172.21.15.32:0/1872260776 wait complete. 2023-12-20T05:44:34.264 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path /snap_test_dir1/test_restart 2023-12-20T05:44:34.264 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate /snap_test_dir1/test_restart 1M 2023-12-20T05:44:34.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.411+0000 7f7f3ef51640 1 Processor -- start 2023-12-20T05:44:34.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.411+0000 7f7f3ef51640 1 -- start start 2023-12-20T05:44:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3ef51640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f3815ccf0 0x7f7f380cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3ef51640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f380cd7b0 0x7f7f380cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3ef51640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7f380cf920 0x7f7f380b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3ef51640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f380ce2f0 con 0x7f7f3815ccf0 2023-12-20T05:44:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3ef51640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f380d0b70 con 0x7f7f380cf920 2023-12-20T05:44:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3ef51640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f380cfd80 con 0x7f7f380cd7b0 2023-12-20T05:44:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3df4f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f3815ccf0 0x7f7f380cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:34.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3df4f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f3815ccf0 0x7f7f380cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58210/0 (socket says 172.21.15.32:58210) 2023-12-20T05:44:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3df4f640 1 -- 172.21.15.32:0/2051637364 learned_addr learned my addr 172.21.15.32:0/2051637364 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3d74e640 1 --2- 172.21.15.32:0/2051637364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f380cd7b0 0x7f7f380cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3e750640 1 --2- 172.21.15.32:0/2051637364 >> v2:172.21.15.39:3300/0 conn(0x7f7f380cf920 0x7f7f380b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3df4f640 1 -- 172.21.15.32:0/2051637364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f380cd7b0 msgr2=0x7f7f380cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3df4f640 1 --2- 172.21.15.32:0/2051637364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f380cd7b0 0x7f7f380cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3df4f640 1 -- 172.21.15.32:0/2051637364 >> v2:172.21.15.39:3300/0 conn(0x7f7f380cf920 msgr2=0x7f7f380b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:44:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3df4f640 1 --2- 172.21.15.32:0/2051637364 >> v2:172.21.15.39:3300/0 conn(0x7f7f380cf920 0x7f7f380b4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.415+0000 7f7f3df4f640 1 -- 172.21.15.32:0/2051637364 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f380d10f0 con 0x7f7f3815ccf0 2023-12-20T05:44:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3df4f640 1 --2- 172.21.15.32:0/2051637364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f3815ccf0 0x7f7f380cd270 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f7f380c63d0 tx=0x7f7f3401c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e59a369848874d96 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3cf4d640 1 -- 172.21.15.32:0/2051637364 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f340ab600 con 0x7f7f3815ccf0 2023-12-20T05:44:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3cf4d640 1 -- 172.21.15.32:0/2051637364 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f340a7050 con 0x7f7f3815ccf0 2023-12-20T05:44:34.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3cf4d640 1 -- 172.21.15.32:0/2051637364 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f340ab980 con 0x7f7f3815ccf0 2023-12-20T05:44:34.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2051637364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f3815ccf0 msgr2=0x7f7f380cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:34.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3ef51640 1 --2- 172.21.15.32:0/2051637364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f3815ccf0 0x7f7f380cd270 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f7f380c63d0 tx=0x7f7f3401c180 comp rx=0 tx=0).stop 2023-12-20T05:44:34.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2051637364 shutdown_connections 2023-12-20T05:44:34.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3ef51640 1 --2- 172.21.15.32:0/2051637364 >> v2:172.21.15.39:3300/0 conn(0x7f7f380cf920 0x7f7f380b4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3ef51640 1 --2- 172.21.15.32:0/2051637364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f380cd7b0 0x7f7f380cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3ef51640 1 --2- 172.21.15.32:0/2051637364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f3815ccf0 0x7f7f380cd270 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2051637364 >> 172.21.15.32:0/2051637364 conn(0x7f7f380658e0 msgr2=0x7f7f38065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2051637364 shutdown_connections 2023-12-20T05:44:34.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2051637364 wait complete. 2023-12-20T05:44:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3ef51640 1 Processor -- start 2023-12-20T05:44:34.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.419+0000 7f7f3ef51640 1 -- start start 2023-12-20T05:44:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3ef51640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f380cd7b0 0x7f7f3814f950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3ef51640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380cf920 0x7f7f380cc220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3ef51640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f3815ccf0 0x7f7f381512b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3ef51640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38150230 con 0x7f7f3815ccf0 2023-12-20T05:44:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3ef51640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f38150510 con 0x7f7f380cf920 2023-12-20T05:44:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3ef51640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f381507f0 con 0x7f7f380cd7b0 2023-12-20T05:44:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3e750640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f3815ccf0 0x7f7f381512b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:34.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3e750640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f3815ccf0 0x7f7f381512b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58226/0 (socket says 172.21.15.32:58226) 2023-12-20T05:44:34.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3e750640 1 -- 172.21.15.32:0/2498278133 learned_addr learned my addr 172.21.15.32:0/2498278133 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:44:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3df4f640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f380cd7b0 0x7f7f3814f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3d74e640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380cf920 0x7f7f380cc220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3e750640 1 -- 172.21.15.32:0/2498278133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f380cd7b0 msgr2=0x7f7f3814f950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3e750640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f380cd7b0 0x7f7f3814f950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3e750640 1 -- 172.21.15.32:0/2498278133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380cf920 msgr2=0x7f7f380cc220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3e750640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380cf920 0x7f7f380cc220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3e750640 1 -- 172.21.15.32:0/2498278133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f340094d0 con 0x7f7f3815ccf0 2023-12-20T05:44:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3df4f640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f380cd7b0 0x7f7f3814f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:44:34.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3d74e640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380cf920 0x7f7f380cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T05:44:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3e750640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f3815ccf0 0x7f7f381512b0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000efc0 tx=0x7f7f3000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f2e7fc640 1 -- 172.21.15.32:0/2498278133 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30014480 con 0x7f7f3815ccf0 2023-12-20T05:44:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f2e7fc640 1 -- 172.21.15.32:0/2498278133 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f3000f040 con 0x7f7f3815ccf0 2023-12-20T05:44:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f2e7fc640 1 -- 172.21.15.32:0/2498278133 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f30014820 con 0x7f7f3815ccf0 2023-12-20T05:44:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2498278133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f38150ad0 con 0x7f7f3815ccf0 2023-12-20T05:44:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2498278133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f3814fe90 con 0x7f7f3815ccf0 2023-12-20T05:44:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2498278133 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f38074100 con 0x7f7f3815ccf0 2023-12-20T05:44:34.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.423+0000 7f7f2e7fc640 1 -- 172.21.15.32:0/2498278133 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 98) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f7f30014980 con 0x7f7f3815ccf0 2023-12-20T05:44:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.427+0000 7f7f2e7fc640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7f1c065900 0x7f7f1c067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:44:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.427+0000 7f7f2e7fc640 1 -- 172.21.15.32:0/2498278133 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(174..174 src has 1..174) v4 ==== 6081+0+0 (secure 0 0 0) 0x7f7f30083bd0 con 0x7f7f3815ccf0 2023-12-20T05:44:34.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.427+0000 7f7f3df4f640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7f1c065900 0x7f7f1c067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:44:34.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.427+0000 7f7f3df4f640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7f1c065900 0x7f7f1c067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7f340ad000 tx=0x7f7f34008040 comp rx=0 tx=0).ready entity=mgr.9259 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:44:34.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.431+0000 7f7f2e7fc640 1 -- 172.21.15.32:0/2498278133 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7f30051680 con 0x7f7f3815ccf0 2023-12-20T05:44:34.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.607+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2498278133 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f7f380b5c40 con 0x7f7f1c065900 2023-12-20T05:44:34.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.651+0000 7f7f2e7fc640 1 -- 172.21.15.32:0/2498278133 <== mgr.9259 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7f7f340ab6c0 con 0x7f7f1c065900 2023-12-20T05:44:34.651 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path /snap_test_dir1/test_restart 2023-12-20T05:44:34.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2498278133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7f1c065900 msgr2=0x7f7f1c067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:34.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7f1c065900 0x7f7f1c067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f7f340ad000 tx=0x7f7f34008040 comp rx=0 tx=0).stop 2023-12-20T05:44:34.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2498278133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f3815ccf0 msgr2=0x7f7f381512b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:44:34.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f3815ccf0 0x7f7f381512b0 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f7f3000efc0 tx=0x7f7f3000c490 comp rx=0 tx=0).stop 2023-12-20T05:44:34.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2498278133 shutdown_connections 2023-12-20T05:44:34.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f7f1c065900 0x7f7f1c067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f3815ccf0 0x7f7f381512b0 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f380cf920 0x7f7f380cc220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 --2- 172.21.15.32:0/2498278133 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f380cd7b0 0x7f7f3814f950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:44:34.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2498278133 >> 172.21.15.32:0/2498278133 conn(0x7f7f380658e0 msgr2=0x7f7f380b7df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:44:34.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2498278133 shutdown_connections 2023-12-20T05:44:34.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:44:34.655+0000 7f7f3ef51640 1 -- 172.21.15.32:0/2498278133 wait complete. 2023-12-20T05:44:34.673 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path /snap_test_dir1/test_restart 2023-12-20T05:44:40.015 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:44:40.019 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:45:00.664 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:45:00.666+0000 7f99c5e2f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:45:00.665 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:45:00.666+0000 7f99c5e2f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:45:00.665 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:45:00.666+0000 7f99c5e2f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:45:00.665 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:45:00.666+0000 7f99c5e2f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:45:00.665 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:45:00.666+0000 7f99c5e2f640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:45:10.079 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:45:10.084 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:45:40.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:45:40.169 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:46:10.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:46:10.230 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:46:40.292 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:46:40.297 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:47:10.353 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:47:10.357 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:47:40.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:47:40.416 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:48:10.476 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:48:10.481 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:48:40.574 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:48:40.582 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:49:10.637 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:49:10.645 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:49:40.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:49:40.709 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:50:10.774 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:50:10.778 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:50:40.859 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:50:40.867 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:51:10.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:51:10.937 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:51:40.996 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:51:41.003 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:52:11.068 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:52:11.072 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:52:41.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:52:41.128 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:53:11.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:53:11.184 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:53:41.244 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:53:41.249 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:54:11.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:54:11.337 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:54:41.398 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:54:41.404 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:55:11.463 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:55:11.469 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:55:15.674 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T05:55:15.675 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-20T05:55:15.689 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_45_00_UTC 2023-12-20T05:55:15.689 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_46_00_UTC 2023-12-20T05:55:15.689 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_47_00_UTC 2023-12-20T05:55:15.689 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_48_00_UTC 2023-12-20T05:55:15.689 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_49_00_UTC 2023-12-20T05:55:15.690 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_50_00_UTC 2023-12-20T05:55:15.690 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_51_00_UTC 2023-12-20T05:55:15.690 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_52_00_UTC 2023-12-20T05:55:15.690 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_53_00_UTC 2023-12-20T05:55:15.690 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_54_00_UTC 2023-12-20T05:55:15.690 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_55_00_UTC 2023-12-20T05:55:15.691 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-05_45_00_UTC', 'scheduled-2023-12-20-05_46_00_UTC', 'scheduled-2023-12-20-05_47_00_UTC', 'scheduled-2023-12-20-05_48_00_UTC', 'scheduled-2023-12-20-05_49_00_UTC', 'scheduled-2023-12-20-05_50_00_UTC', 'scheduled-2023-12-20-05_51_00_UTC', 'scheduled-2023-12-20-05_52_00_UTC', 'scheduled-2023-12-20-05_53_00_UTC', 'scheduled-2023-12-20-05_54_00_UTC', 'scheduled-2023-12-20-05_55_00_UTC'] 2023-12-20T05:55:15.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-12-20T05:55:15.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.885+0000 7f9a8b17b640 1 Processor -- start 2023-12-20T05:55:15.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.885+0000 7f9a8b17b640 1 -- start start 2023-12-20T05:55:15.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a8b17b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9a840b0760 0x7f9a840b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:15.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a8b17b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a840cc4b0 0x7f9a840cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:15.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a8b17b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a840b1100 con 0x7f9a84157480 2023-12-20T05:55:15.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a8b17b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a840c9f50 con 0x7f9a840cc4b0 2023-12-20T05:55:15.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a8b17b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a840cde20 con 0x7f9a840b0760 2023-12-20T05:55:15.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a89718640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a840cc4b0 0x7f9a840cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:15.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a89718640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9a840cc4b0 0x7f9a840cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57236/0 (socket says 172.21.15.32:57236) 2023-12-20T05:55:15.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a89718640 1 -- 172.21.15.32:0/290915193 learned_addr learned my addr 172.21.15.32:0/290915193 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:55:15.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a83fff640 1 --2- 172.21.15.32:0/290915193 >> v2:172.21.15.39:3300/0 conn(0x7f9a840b0760 0x7f9a840b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a89718640 1 -- 172.21.15.32:0/290915193 >> v2:172.21.15.39:3300/0 conn(0x7f9a840b0760 msgr2=0x7f9a840b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:55:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a89718640 1 --2- 172.21.15.32:0/290915193 >> v2:172.21.15.39:3300/0 conn(0x7f9a840b0760 0x7f9a840b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a89718640 1 -- 172.21.15.32:0/290915193 >> v1:172.21.15.32:6789/0 conn(0x7f9a84157480 legacy=0x7f9a840c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.889+0000 7f9a89718640 1 -- 172.21.15.32:0/290915193 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a840c8e30 con 0x7f9a840cc4b0 2023-12-20T05:55:15.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a89718640 1 --2- 172.21.15.32:0/290915193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a840cc4b0 0x7f9a840cc8d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f9a8406e0d0 tx=0x7f9a7c01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fee084371092761f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:55:15.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a837fe640 1 -- 172.21.15.32:0/290915193 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c0a57d0 con 0x7f9a840cc4b0 2023-12-20T05:55:15.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a837fe640 1 -- 172.21.15.32:0/290915193 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a7c0a5930 con 0x7f9a840cc4b0 2023-12-20T05:55:15.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a837fe640 1 -- 172.21.15.32:0/290915193 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a7c0a5cc0 con 0x7f9a840cc4b0 2023-12-20T05:55:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a8b17b640 1 -- 172.21.15.32:0/290915193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a840cc4b0 msgr2=0x7f9a840cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a8b17b640 1 --2- 172.21.15.32:0/290915193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a840cc4b0 0x7f9a840cc8d0 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7f9a8406e0d0 tx=0x7f9a7c01c910 comp rx=0 tx=0).stop 2023-12-20T05:55:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a8b17b640 1 -- 172.21.15.32:0/290915193 shutdown_connections 2023-12-20T05:55:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a8b17b640 1 --2- 172.21.15.32:0/290915193 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a840cc4b0 0x7f9a840cc8d0 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a8b17b640 1 --2- 172.21.15.32:0/290915193 >> v2:172.21.15.39:3300/0 conn(0x7f9a840b0760 0x7f9a840b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a8b17b640 1 -- 172.21.15.32:0/290915193 >> 172.21.15.32:0/290915193 conn(0x7f9a8406a510 msgr2=0x7f9a840ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:55:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a8b17b640 1 -- 172.21.15.32:0/290915193 shutdown_connections 2023-12-20T05:55:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a8b17b640 1 -- 172.21.15.32:0/290915193 wait complete. 2023-12-20T05:55:15.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a8b17b640 1 Processor -- start 2023-12-20T05:55:15.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.893+0000 7f9a8b17b640 1 -- start start 2023-12-20T05:55:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a8b17b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a840b0760 0x7f9a8413d130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a8b17b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a840cc4b0 0x7f9a8415adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a8b17b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a84157480 0x7f9a841f61c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a8b17b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a841f7160 con 0x7f9a840b0760 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a8b17b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a8414ec50 con 0x7f9a840cc4b0 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a8b17b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a841f6ba0 con 0x7f9a84157480 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a88f17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a840b0760 0x7f9a8413d130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a88f17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a840b0760 0x7f9a8413d130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57248/0 (socket says 172.21.15.32:57248) 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a88f17640 1 -- 172.21.15.32:0/3493922900 learned_addr learned my addr 172.21.15.32:0/3493922900 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a83fff640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a840cc4b0 0x7f9a8415adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a89718640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a84157480 0x7f9a841f61c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a88f17640 1 -- 172.21.15.32:0/3493922900 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a84157480 msgr2=0x7f9a841f61c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a88f17640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a84157480 0x7f9a841f61c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a88f17640 1 -- 172.21.15.32:0/3493922900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a840cc4b0 msgr2=0x7f9a8415adf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a88f17640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a840cc4b0 0x7f9a8415adf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a88f17640 1 -- 172.21.15.32:0/3493922900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a7c009b30 con 0x7f9a840b0760 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a88f17640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a840b0760 0x7f9a8413d130 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800c500 tx=0x7f9a7800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:55:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a817fa640 1 -- 172.21.15.32:0/3493922900 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78005220 con 0x7f9a840b0760 2023-12-20T05:55:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a817fa640 1 -- 172.21.15.32:0/3493922900 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a78005380 con 0x7f9a840b0760 2023-12-20T05:55:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a8b17b640 1 -- 172.21.15.32:0/3493922900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a841f6e80 con 0x7f9a840b0760 2023-12-20T05:55:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a817fa640 1 -- 172.21.15.32:0/3493922900 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a78007270 con 0x7f9a840b0760 2023-12-20T05:55:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a8b17b640 1 -- 172.21.15.32:0/3493922900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a8413d670 con 0x7f9a840b0760 2023-12-20T05:55:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a8b17b640 1 -- 172.21.15.32:0/3493922900 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a8414e6d0 con 0x7f9a840b0760 2023-12-20T05:55:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.897+0000 7f9a817fa640 1 -- 172.21.15.32:0/3493922900 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 102) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f9a780073d0 con 0x7f9a840b0760 2023-12-20T05:55:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.901+0000 7f9a817fa640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9a68065bd0 0x7f9a68068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.901+0000 7f9a817fa640 1 -- 172.21.15.32:0/3493922900 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6110+0+0 (secure 0 0 0) 0x7f9a78082d30 con 0x7f9a840b0760 2023-12-20T05:55:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.901+0000 7f9a83fff640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9a68065bd0 0x7f9a68068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.901+0000 7f9a83fff640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9a68065bd0 0x7f9a68068090 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9a8415db90 tx=0x7f9a70008040 comp rx=0 tx=0).ready entity=mgr.9259 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:55:15.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:15.905+0000 7f9a817fa640 1 -- 172.21.15.32:0/3493922900 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9a8414e6d0 con 0x7f9a840b0760 2023-12-20T05:55:16.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.081+0000 7f9a8b17b640 1 -- 172.21.15.32:0/3493922900 --> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a840c9360 con 0x7f9a68065bd0 2023-12-20T05:55:16.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.089+0000 7f9a817fa640 1 -- 172.21.15.32:0/3493922900 <== mgr.9259 v2:172.21.15.32:6832/38009 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+360 (secure 0 0 0) 0x7f9a700108a0 con 0x7f9a68065bd0 2023-12-20T05:55:16.091 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:55:16.091 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T05:44:34", "first": "2023-12-20T05:45:00", "last": "2023-12-20T05:55:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-12-20T05:55:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.089+0000 7f9a8b17b640 1 -- 172.21.15.32:0/3493922900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9a68065bd0 msgr2=0x7f9a68068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.089+0000 7f9a8b17b640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9a68065bd0 0x7f9a68068090 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f9a8415db90 tx=0x7f9a70008040 comp rx=0 tx=0).stop 2023-12-20T05:55:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.089+0000 7f9a8b17b640 1 -- 172.21.15.32:0/3493922900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a840b0760 msgr2=0x7f9a8413d130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.089+0000 7f9a8b17b640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a840b0760 0x7f9a8413d130 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7f9a7800c500 tx=0x7f9a7800c9c0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.089+0000 7f9a8b17b640 1 -- 172.21.15.32:0/3493922900 shutdown_connections 2023-12-20T05:55:16.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.089+0000 7f9a8b17b640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f9a68065bd0 0x7f9a68068090 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.089+0000 7f9a8b17b640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a84157480 0x7f9a841f61c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.089+0000 7f9a8b17b640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a840cc4b0 0x7f9a8415adf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.089+0000 7f9a8b17b640 1 --2- 172.21.15.32:0/3493922900 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a840b0760 0x7f9a8413d130 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.089+0000 7f9a8b17b640 1 -- 172.21.15.32:0/3493922900 >> 172.21.15.32:0/3493922900 conn(0x7f9a8406a510 msgr2=0x7f9a840c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:55:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.093+0000 7f9a8b17b640 1 -- 172.21.15.32:0/3493922900 shutdown_connections 2023-12-20T05:55:16.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.093+0000 7f9a8b17b640 1 -- 172.21.15.32:0/3493922900 wait complete. 2023-12-20T05:55:16.111 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T05:44:34", "first": "2023-12-20T05:45:00", "last": "2023-12-20T05:55:00", "last_pruned": null, "created_count": 11, "pruned_count": 0, "active": true}] 2023-12-20T05:55:16.112 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T05:44:34', 'first': '2023-12-20T05:45:00', 'last': '2023-12-20T05:55:00', 'last_pruned': None, 'created_count': 11, 'pruned_count': 0, 'active': True} 2023-12-20T05:55:16.112 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 11 2023-12-20T05:55:16.112 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 11 2023-12-20T05:55:16.112 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2023-12-20T05:55:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.253+0000 7f1628eb6640 1 Processor -- start 2023-12-20T05:55:16.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.253+0000 7f1628eb6640 1 -- start start 2023-12-20T05:55:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f1628eb6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f162406ce60 0x7f162415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f1628eb6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f162415f110 0x7f162415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f1628eb6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162415bbd0 con 0x7f16240af5e0 2023-12-20T05:55:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f1628eb6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162415beb0 con 0x7f162415f110 2023-12-20T05:55:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f1628eb6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162415e910 con 0x7f162406ce60 2023-12-20T05:55:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f162259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f16240af5e0 0x7f16240af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:46402/0 (socket says 172.21.15.32:46402) 2023-12-20T05:55:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f162259c640 1 -- 172.21.15.32:0/454576103 learned_addr learned my addr 172.21.15.32:0/454576103 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T05:55:16.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f1622d9d640 1 --2- 172.21.15.32:0/454576103 >> v2:172.21.15.32:3300/0 conn(0x7f162415f110 0x7f162415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f161bfff640 1 --2- 172.21.15.32:0/454576103 >> v2:172.21.15.39:3300/0 conn(0x7f162406ce60 0x7f162415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f1622d9d640 1 -- 172.21.15.32:0/454576103 >> v2:172.21.15.39:3300/0 conn(0x7f162406ce60 msgr2=0x7f162415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T05:55:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f1622d9d640 1 --2- 172.21.15.32:0/454576103 >> v2:172.21.15.39:3300/0 conn(0x7f162406ce60 0x7f162415b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f1622d9d640 1 -- 172.21.15.32:0/454576103 >> v1:172.21.15.32:6789/0 conn(0x7f16240af5e0 legacy=0x7f16240af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:16.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f1622d9d640 1 -- 172.21.15.32:0/454576103 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162415ebf0 con 0x7f162415f110 2023-12-20T05:55:16.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.257+0000 7f1622d9d640 1 --2- 172.21.15.32:0/454576103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162415f110 0x7f162415c440 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f162406abe0 tx=0x7f161c01c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54344b2f99986733 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:55:16.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1621d9b640 1 -- 172.21.15.32:0/454576103 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f162415b9d0 con 0x7f162415f110 2023-12-20T05:55:16.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1621d9b640 1 -- 172.21.15.32:0/454576103 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c0a58e0 con 0x7f162415f110 2023-12-20T05:55:16.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1621d9b640 1 -- 172.21.15.32:0/454576103 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c0a5c70 con 0x7f162415f110 2023-12-20T05:55:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 -- 172.21.15.32:0/454576103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162415f110 msgr2=0x7f162415c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 --2- 172.21.15.32:0/454576103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162415f110 0x7f162415c440 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7f162406abe0 tx=0x7f161c01c880 comp rx=0 tx=0).stop 2023-12-20T05:55:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 -- 172.21.15.32:0/454576103 shutdown_connections 2023-12-20T05:55:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 --2- 172.21.15.32:0/454576103 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162415f110 0x7f162415c440 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 --2- 172.21.15.32:0/454576103 >> v2:172.21.15.39:3300/0 conn(0x7f162406ce60 0x7f162415b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 -- 172.21.15.32:0/454576103 >> 172.21.15.32:0/454576103 conn(0x7f162406a320 msgr2=0x7f16240ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:55:16.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 -- 172.21.15.32:0/454576103 shutdown_connections 2023-12-20T05:55:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 -- 172.21.15.32:0/454576103 wait complete. 2023-12-20T05:55:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 Processor -- start 2023-12-20T05:55:16.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 -- start start 2023-12-20T05:55:16.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162406ce60 0x7f162414f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:16.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16240af5e0 0x7f162414a310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:16.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162415f110 0x7f16241501d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:16.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1624150d60 con 0x7f162415f110 2023-12-20T05:55:16.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162414d210 con 0x7f16240af5e0 2023-12-20T05:55:16.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1628eb6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1624150fb0 con 0x7f162406ce60 2023-12-20T05:55:16.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1622d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162415f110 0x7f16241501d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1622d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162415f110 0x7f16241501d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:57272/0 (socket says 172.21.15.32:57272) 2023-12-20T05:55:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f1622d9d640 1 -- 172.21.15.32:0/2605809045 learned_addr learned my addr 172.21.15.32:0/2605809045 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T05:55:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.261+0000 7f161bfff640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16240af5e0 0x7f162414a310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f162259c640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162406ce60 0x7f162414f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f1622d9d640 1 -- 172.21.15.32:0/2605809045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162406ce60 msgr2=0x7f162414f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f1622d9d640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162406ce60 0x7f162414f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f1622d9d640 1 -- 172.21.15.32:0/2605809045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16240af5e0 msgr2=0x7f162414a310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f1622d9d640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16240af5e0 0x7f162414a310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f1622d9d640 1 -- 172.21.15.32:0/2605809045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f161c009a70 con 0x7f162415f110 2023-12-20T05:55:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f1622d9d640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162415f110 0x7f16241501d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f161c019490 tx=0x7f161c019620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:55:16.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f161affd640 1 -- 172.21.15.32:0/2605809045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c0b9860 con 0x7f162415f110 2023-12-20T05:55:16.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f161affd640 1 -- 172.21.15.32:0/2605809045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c0b99c0 con 0x7f162415f110 2023-12-20T05:55:16.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f1628eb6640 1 -- 172.21.15.32:0/2605809045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1624151290 con 0x7f162415f110 2023-12-20T05:55:16.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f161affd640 1 -- 172.21.15.32:0/2605809045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c0b9d70 con 0x7f162415f110 2023-12-20T05:55:16.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f1628eb6640 1 -- 172.21.15.32:0/2605809045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1624151850 con 0x7f162415f110 2023-12-20T05:55:16.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f1628eb6640 1 -- 172.21.15.32:0/2605809045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1624151a40 con 0x7f162415f110 2023-12-20T05:55:16.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f161affd640 1 -- 172.21.15.32:0/2605809045 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 102) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f161c0ad440 con 0x7f162415f110 2023-12-20T05:55:16.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f161affd640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065bd0 0x7f1604068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T05:55:16.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.265+0000 7f161affd640 1 -- 172.21.15.32:0/2605809045 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(176..176 src has 1..176) v4 ==== 6110+0+0 (secure 0 0 0) 0x7f161c11e7a0 con 0x7f162415f110 2023-12-20T05:55:16.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.269+0000 7f162259c640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065bd0 0x7f1604068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T05:55:16.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.269+0000 7f162259c640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065bd0 0x7f1604068090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f16140010f0 tx=0x7f1614009040 comp rx=0 tx=0).ready entity=mgr.9259 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T05:55:16.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.269+0000 7f161affd640 1 -- 172.21.15.32:0/2605809045 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f161c0ebf60 con 0x7f162415f110 2023-12-20T05:55:16.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.465+0000 7f1628eb6640 1 -- 172.21.15.32:0/2605809045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) v1 -- 0x7f162404e000 con 0x7f162415f110 2023-12-20T05:55:16.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.469+0000 7f161affd640 1 -- 172.21.15.32:0/2605809045 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v102) v1 ==== 74+0+163570 (secure 0 0 0) 0x7f161c0ef580 con 0x7f162415f110 2023-12-20T05:55:16.474 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T05:55:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 -- 172.21.15.32:0/2605809045 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065bd0 msgr2=0x7f1604068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065bd0 0x7f1604068090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f16140010f0 tx=0x7f1614009040 comp rx=0 tx=0).stop 2023-12-20T05:55:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 -- 172.21.15.32:0/2605809045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162415f110 msgr2=0x7f16241501d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T05:55:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162415f110 0x7f16241501d0 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7f161c019490 tx=0x7f161c019620 comp rx=0 tx=0).stop 2023-12-20T05:55:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 -- 172.21.15.32:0/2605809045 shutdown_connections 2023-12-20T05:55:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.32:6832/38009,v1:172.21.15.32:6833/38009] conn(0x7f1604065bd0 0x7f1604068090 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162415f110 0x7f16241501d0 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16240af5e0 0x7f162414a310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 --2- 172.21.15.32:0/2605809045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162406ce60 0x7f162414f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T05:55:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 -- 172.21.15.32:0/2605809045 >> 172.21.15.32:0/2605809045 conn(0x7f162406a320 msgr2=0x7f16240cd8e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T05:55:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 -- 172.21.15.32:0/2605809045 shutdown_connections 2023-12-20T05:55:16.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T05:55:16.473+0000 7f1628eb6640 1 -- 172.21.15.32:0/2605809045 wait complete. 2023-12-20T05:55:16.493 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":102,"active_gid":9259,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6832","nonce":38009},{"type":"v1","addr":"172.21.15.32:6833","nonce":38009}]},"active_addr":"172.21.15.32:6833/38009","active_change":"2023-12-20T05:44:33.158264+0000","active_mgr_features":4540138322906710015,"available":true,"standbys":[{"gid":8090,"name":"x","mgr_features":4540138322906710015,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","nfs","restful","snap_schedule"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/ceph-grafana:9.4.7","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:0.0.2","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"secs","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":174,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2272871714}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2161329994}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2000830842}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":2721040122}]},{"name":"snap_schedule","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":663256086}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.32:0","nonce":3587707827}]}]} 2023-12-20T05:55:16.497 DEBUG:tasks.cephfs.test_snap_schedules:restarting active mgr: y 2023-12-20T05:55:16.497 INFO:tasks.ceph.mgr.y:Restarting daemon 2023-12-20T05:55:16.497 INFO:tasks.ceph.mgr.y:Stopping old one... 2023-12-20T05:55:16.497 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-20T05:55:16.498 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T05:55:16.559 INFO:tasks.ceph.mgr.y:Stopped 2023-12-20T05:55:16.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i y 2023-12-20T05:55:16.563 INFO:tasks.ceph.mgr.y:Started 2023-12-20T05:55:16.745 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:16.745+0000 7f1a59be8180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T05:55:17.186 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:17.185+0000 7f1a59be8180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T05:55:17.343 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:17.341+0000 7f1a59be8180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T05:55:17.850 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:17.849+0000 7f1a59be8180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T05:55:17.911 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:17.909+0000 7f1a59be8180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T05:55:17.971 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:17.969+0000 7f1a59be8180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T05:55:18.061 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:18.061+0000 7f1a59be8180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T05:55:18.121 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:18.121+0000 7f1a59be8180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T05:55:18.209 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:18.209+0000 7f1a59be8180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T05:55:18.268 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:18.265+0000 7f1a59be8180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T05:55:18.419 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:18.417+0000 7f1a59be8180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T05:55:18.480 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:18.477+0000 7f1a59be8180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T05:55:18.730 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T05:55:18.730 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T05:55:18.730 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T05:55:18.738 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:18.737+0000 7f1a59be8180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T05:55:18.881 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:18.881+0000 7f1a59be8180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:55:19.012 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:19.009+0000 7f1a59be8180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T05:55:19.086 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:19.085+0000 7f1a59be8180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T05:55:19.228 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:19.225+0000 7f1a59be8180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T05:55:19.796 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:19.793+0000 7f1a59be8180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T05:55:19.884 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:19.881+0000 7f1a59be8180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T05:55:20.041 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.041+0000 7f1a59be8180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T05:55:20.129 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.129+0000 7f1a59be8180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T05:55:20.462 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.461+0000 7f1a59be8180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T05:55:20.526 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.525+0000 7f1a59be8180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T05:55:20.672 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.669+0000 7f19ddb51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:55:20.672 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.669+0000 7f19ddb51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:55:20.672 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.669+0000 7f19ddb51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:55:20.672 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.669+0000 7f19ddb51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:55:20.672 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.669+0000 7f19ddb51640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:55:20.673 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.673+0000 7f19dc34e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:55:20.673 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.673+0000 7f19dc34e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:55:20.673 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.673+0000 7f19dc34e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:55:20.673 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.673+0000 7f19dc34e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:55:20.674 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:55:20.673+0000 7f19dc34e640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:55:41.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:55:41.542 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:56:00.669 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:56:00.672+0000 7f19e7364640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:56:00.670 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:56:00.672+0000 7f19e7364640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:56:00.670 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:56:00.672+0000 7f19e7364640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:56:00.670 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:56:00.672+0000 7f19e7364640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:56:00.670 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T05:56:00.672+0000 7f19e7364640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T05:56:11.601 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:56:11.605 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:56:41.658 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:56:41.662 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:57:11.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:57:11.727 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:57:41.812 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:57:41.818 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:58:11.877 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:58:11.882 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:58:41.944 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:58:41.948 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:59:12.014 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:59:12.019 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:59:42.088 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T05:59:42.093 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:00:12.162 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:00:12.167 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:00:16.564 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate /snap_test_dir1/test_restart 1M 2023-12-20T06:00:16.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.723+0000 7fe82be1f640 1 Processor -- start 2023-12-20T06:00:16.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.723+0000 7fe82be1f640 1 -- start start 2023-12-20T06:00:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe82be1f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe824164670 0x7fe8240ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe82be1f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe82406da20 0x7fe8240cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe82be1f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe8240cb0b0 con 0x7fe824070230 2023-12-20T06:00:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe82be1f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe8240ca810 con 0x7fe824164670 2023-12-20T06:00:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe82be1f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8240caaf0 con 0x7fe82406da20 2023-12-20T06:00:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe829bbb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe824164670 0x7fe8240ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe829bbb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe824164670 0x7fe8240ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44660/0 (socket says 172.21.15.32:44660) 2023-12-20T06:00:16.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe829bbb640 1 -- 172.21.15.32:0/4182922305 learned_addr learned my addr 172.21.15.32:0/4182922305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe8293ba640 1 --2- 172.21.15.32:0/4182922305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe82406da20 0x7fe8240cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe829bbb640 1 -- 172.21.15.32:0/4182922305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe82406da20 msgr2=0x7fe8240cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe829bbb640 1 --2- 172.21.15.32:0/4182922305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe82406da20 0x7fe8240cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe829bbb640 1 -- 172.21.15.32:0/4182922305 >> v1:172.21.15.39:6789/0 conn(0x7fe824070230 legacy=0x7fe82406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe829bbb640 1 -- 172.21.15.32:0/4182922305 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8240cadd0 con 0x7fe824164670 2023-12-20T06:00:16.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe8293ba640 1 --2- 172.21.15.32:0/4182922305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe82406da20 0x7fe8240cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:16.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.727+0000 7fe829bbb640 1 --2- 172.21.15.32:0/4182922305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe824164670 0x7fe8240ca1a0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fe820009870 tx=0x7fe82001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b3c94569f6d2cce server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:16.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe828bb9640 1 -- 172.21.15.32:0/4182922305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8200ac630 con 0x7fe824164670 2023-12-20T06:00:16.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe828bb9640 1 -- 172.21.15.32:0/4182922305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8200ac790 con 0x7fe824164670 2023-12-20T06:00:16.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe828bb9640 1 -- 172.21.15.32:0/4182922305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8200acb20 con 0x7fe824164670 2023-12-20T06:00:16.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 -- 172.21.15.32:0/4182922305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe824164670 msgr2=0x7fe8240ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:16.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 --2- 172.21.15.32:0/4182922305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe824164670 0x7fe8240ca1a0 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fe820009870 tx=0x7fe82001c3f0 comp rx=0 tx=0).stop 2023-12-20T06:00:16.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 -- 172.21.15.32:0/4182922305 shutdown_connections 2023-12-20T06:00:16.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 --2- 172.21.15.32:0/4182922305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe82406da20 0x7fe8240cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:16.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 --2- 172.21.15.32:0/4182922305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe824164670 0x7fe8240ca1a0 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:16.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 -- 172.21.15.32:0/4182922305 >> 172.21.15.32:0/4182922305 conn(0x7fe82406a510 msgr2=0x7fe8240ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:16.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 -- 172.21.15.32:0/4182922305 shutdown_connections 2023-12-20T06:00:16.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 -- 172.21.15.32:0/4182922305 wait complete. 2023-12-20T06:00:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 Processor -- start 2023-12-20T06:00:16.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 -- start start 2023-12-20T06:00:16.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82406da20 0x7fe8240b1390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe824070230 0x7fe8240c76e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe824164670 0x7fe824156b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe824162a50 con 0x7fe82406da20 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe8240c8050 con 0x7fe824070230 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.731+0000 7fe82be1f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe8240c8800 con 0x7fe824164670 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe829bbb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82406da20 0x7fe8240b1390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe829bbb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82406da20 0x7fe8240b1390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44670/0 (socket says 172.21.15.32:44670) 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe829bbb640 1 -- 172.21.15.32:0/314158400 learned_addr learned my addr 172.21.15.32:0/314158400 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe8293ba640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe824070230 0x7fe8240c76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe82a3bc640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe824164670 0x7fe824156b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe829bbb640 1 -- 172.21.15.32:0/314158400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe824164670 msgr2=0x7fe824156b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe829bbb640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe824164670 0x7fe824156b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe829bbb640 1 -- 172.21.15.32:0/314158400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe824070230 msgr2=0x7fe8240c76e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe829bbb640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe824070230 0x7fe8240c76e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:16.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe829bbb640 1 -- 172.21.15.32:0/314158400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe8200094d0 con 0x7fe82406da20 2023-12-20T06:00:16.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe8293ba640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe824070230 0x7fe8240c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:16.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe829bbb640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82406da20 0x7fe8240b1390 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fe8200af040 tx=0x7fe82001ac30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:16.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe81a7fc640 1 -- 172.21.15.32:0/314158400 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8200b7620 con 0x7fe82406da20 2023-12-20T06:00:16.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe81a7fc640 1 -- 172.21.15.32:0/314158400 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe8200b7780 con 0x7fe82406da20 2023-12-20T06:00:16.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe81a7fc640 1 -- 172.21.15.32:0/314158400 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe8200b7b30 con 0x7fe82406da20 2023-12-20T06:00:16.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe82be1f640 1 -- 172.21.15.32:0/314158400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe8240c7d70 con 0x7fe82406da20 2023-12-20T06:00:16.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe82be1f640 1 -- 172.21.15.32:0/314158400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe824162670 con 0x7fe82406da20 2023-12-20T06:00:16.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe82be1f640 1 -- 172.21.15.32:0/314158400 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe82406acf0 con 0x7fe82406da20 2023-12-20T06:00:16.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe81a7fc640 1 -- 172.21.15.32:0/314158400 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 108) v1 ==== 85359+0+0 (secure 0 0 0) 0x7fe8200b3040 con 0x7fe82406da20 2023-12-20T06:00:16.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe81a7fc640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe808065c20 0x7fe8080680e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:16.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.735+0000 7fe81a7fc640 1 -- 172.21.15.32:0/314158400 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6368+0+0 (secure 0 0 0) 0x7fe82011dc30 con 0x7fe82406da20 2023-12-20T06:00:16.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.739+0000 7fe8293ba640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe808065c20 0x7fe8080680e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:16.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.739+0000 7fe8293ba640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe808065c20 0x7fe8080680e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe824161e30 tx=0x7fe814008040 comp rx=0 tx=0).ready entity=mgr.9699 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:16.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.743+0000 7fe81a7fc640 1 -- 172.21.15.32:0/314158400 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe8200eb150 con 0x7fe82406da20 2023-12-20T06:00:16.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.919+0000 7fe82be1f640 1 -- 172.21.15.32:0/314158400 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fe8241621f0 con 0x7fe808065c20 2023-12-20T06:00:16.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe81a7fc640 1 -- 172.21.15.32:0/314158400 <== mgr.9699 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fe8241621f0 con 0x7fe808065c20 2023-12-20T06:00:16.977 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path /snap_test_dir1/test_restart 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 -- 172.21.15.32:0/314158400 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe808065c20 msgr2=0x7fe8080680e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe808065c20 0x7fe8080680e0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fe824161e30 tx=0x7fe814008040 comp rx=0 tx=0).stop 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 -- 172.21.15.32:0/314158400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82406da20 msgr2=0x7fe8240b1390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82406da20 0x7fe8240b1390 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fe8200af040 tx=0x7fe82001ac30 comp rx=0 tx=0).stop 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 -- 172.21.15.32:0/314158400 shutdown_connections 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe808065c20 0x7fe8080680e0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe824164670 0x7fe824156b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe824070230 0x7fe8240c76e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 --2- 172.21.15.32:0/314158400 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe82406da20 0x7fe8240b1390 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 -- 172.21.15.32:0/314158400 >> 172.21.15.32:0/314158400 conn(0x7fe82406a510 msgr2=0x7fe8240c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 -- 172.21.15.32:0/314158400 shutdown_connections 2023-12-20T06:00:16.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:16.979+0000 7fe82be1f640 1 -- 172.21.15.32:0/314158400 wait complete. 2023-12-20T06:00:16.998 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path /snap_test_dir1/test_restart 2023-12-20T06:00:16.998 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:16.998 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-20T06:00:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_45_00_UTC 2023-12-20T06:00:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_46_00_UTC 2023-12-20T06:00:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_47_00_UTC 2023-12-20T06:00:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_48_00_UTC 2023-12-20T06:00:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_49_00_UTC 2023-12-20T06:00:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_50_00_UTC 2023-12-20T06:00:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_51_00_UTC 2023-12-20T06:00:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_52_00_UTC 2023-12-20T06:00:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_53_00_UTC 2023-12-20T06:00:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_54_00_UTC 2023-12-20T06:00:17.009 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_55_00_UTC 2023-12-20T06:00:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_56_00_UTC 2023-12-20T06:00:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_57_00_UTC 2023-12-20T06:00:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_58_00_UTC 2023-12-20T06:00:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_59_00_UTC 2023-12-20T06:00:17.010 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_00_00_UTC 2023-12-20T06:00:17.011 DEBUG:tasks.cephfs.test_snap_schedules:snapshots: ['scheduled-2023-12-20-05_45_00_UTC', 'scheduled-2023-12-20-05_46_00_UTC', 'scheduled-2023-12-20-05_47_00_UTC', 'scheduled-2023-12-20-05_48_00_UTC', 'scheduled-2023-12-20-05_49_00_UTC', 'scheduled-2023-12-20-05_50_00_UTC', 'scheduled-2023-12-20-05_51_00_UTC', 'scheduled-2023-12-20-05_52_00_UTC', 'scheduled-2023-12-20-05_53_00_UTC', 'scheduled-2023-12-20-05_54_00_UTC', 'scheduled-2023-12-20-05_55_00_UTC', 'scheduled-2023-12-20-05_56_00_UTC', 'scheduled-2023-12-20-05_57_00_UTC', 'scheduled-2023-12-20-05_58_00_UTC', 'scheduled-2023-12-20-05_59_00_UTC', 'scheduled-2023-12-20-06_00_00_UTC'] 2023-12-20T06:00:17.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status --format json /snap_test_dir1/test_restart 2023-12-20T06:00:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.195+0000 7f1630998640 1 Processor -- start 2023-12-20T06:00:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.195+0000 7f1630998640 1 -- start start 2023-12-20T06:00:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f1630998640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162c0b0cb0 0x7f162c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f1630998640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f162c070230 0x7f162c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f1630998640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f162c15b6a0 con 0x7f162c15abb0 2023-12-20T06:00:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f1630998640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f162c06e160 con 0x7f162c070230 2023-12-20T06:00:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f1630998640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162c070be0 con 0x7f162c0b0cb0 2023-12-20T06:00:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f162ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f162c070230 0x7f162c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f162ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f162c070230 0x7f162c0706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44684/0 (socket says 172.21.15.32:44684) 2023-12-20T06:00:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f162ad9d640 1 -- 172.21.15.32:0/3028699777 learned_addr learned my addr 172.21.15.32:0/3028699777 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f162a59c640 1 --1- 172.21.15.32:0/3028699777 >> v1:172.21.15.32:6789/0 conn(0x7f162c15abb0 0x7f162c15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:58924/0 (socket says 172.21.15.32:58924) 2023-12-20T06:00:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f1623fff640 1 --2- 172.21.15.32:0/3028699777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162c0b0cb0 0x7f162c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f162ad9d640 1 -- 172.21.15.32:0/3028699777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162c0b0cb0 msgr2=0x7f162c0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f162ad9d640 1 --2- 172.21.15.32:0/3028699777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162c0b0cb0 0x7f162c0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f162ad9d640 1 -- 172.21.15.32:0/3028699777 >> v1:172.21.15.32:6789/0 conn(0x7f162c15abb0 legacy=0x7f162c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.199+0000 7f162ad9d640 1 -- 172.21.15.32:0/3028699777 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f162c06d8c0 con 0x7f162c070230 2023-12-20T06:00:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f162ad9d640 1 --2- 172.21.15.32:0/3028699777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c070230 0x7f162c0706a0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f1624009e50 tx=0x7f162401ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dfd707c1bd402f8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1629d9b640 1 -- 172.21.15.32:0/3028699777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16240a58e0 con 0x7f162c070230 2023-12-20T06:00:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1629d9b640 1 -- 172.21.15.32:0/3028699777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16240a5a40 con 0x7f162c070230 2023-12-20T06:00:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1629d9b640 1 -- 172.21.15.32:0/3028699777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16240a5dd0 con 0x7f162c070230 2023-12-20T06:00:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1630998640 1 -- 172.21.15.32:0/3028699777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c070230 msgr2=0x7f162c0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1630998640 1 --2- 172.21.15.32:0/3028699777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c070230 0x7f162c0706a0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7f1624009e50 tx=0x7f162401ca10 comp rx=0 tx=0).stop 2023-12-20T06:00:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1630998640 1 -- 172.21.15.32:0/3028699777 shutdown_connections 2023-12-20T06:00:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1630998640 1 --2- 172.21.15.32:0/3028699777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c070230 0x7f162c0706a0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1630998640 1 --2- 172.21.15.32:0/3028699777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162c0b0cb0 0x7f162c0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1630998640 1 -- 172.21.15.32:0/3028699777 >> 172.21.15.32:0/3028699777 conn(0x7f162c06a510 msgr2=0x7f162c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1630998640 1 -- 172.21.15.32:0/3028699777 shutdown_connections 2023-12-20T06:00:17.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1630998640 1 -- 172.21.15.32:0/3028699777 wait complete. 2023-12-20T06:00:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1630998640 1 Processor -- start 2023-12-20T06:00:17.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.203+0000 7f1630998640 1 -- start start 2023-12-20T06:00:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1630998640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c070230 0x7f162c153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1630998640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c0b0cb0 0x7f162c14c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1630998640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162c15abb0 0x7f162c14c710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1630998640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c14cc50 con 0x7f162c070230 2023-12-20T06:00:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1630998640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f162c1517e0 con 0x7f162c0b0cb0 2023-12-20T06:00:17.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1630998640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f162c14d170 con 0x7f162c15abb0 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f162a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c070230 0x7f162c153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f162a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c070230 0x7f162c153760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44696/0 (socket says 172.21.15.32:44696) 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f162a59c640 1 -- 172.21.15.32:0/3216995906 learned_addr learned my addr 172.21.15.32:0/3216995906 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f162a59c640 1 -- 172.21.15.32:0/3216995906 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162c15abb0 msgr2=0x7f162c14c710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f162a59c640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162c15abb0 0x7f162c14c710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1623fff640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c0b0cb0 0x7f162c14c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f162a59c640 1 -- 172.21.15.32:0/3216995906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c0b0cb0 msgr2=0x7f162c14c1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f162a59c640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c0b0cb0 0x7f162c14c1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f162a59c640 1 -- 172.21.15.32:0/3216995906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1624009b30 con 0x7f162c070230 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1623fff640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c0b0cb0 0x7f162c14c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f162a59c640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c070230 0x7f162c153760 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f161c00c450 tx=0x7f161c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:17.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1622ffd640 1 -- 172.21.15.32:0/3216995906 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c007480 con 0x7f162c070230 2023-12-20T06:00:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1622ffd640 1 -- 172.21.15.32:0/3216995906 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f161c0075e0 con 0x7f162c070230 2023-12-20T06:00:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1622ffd640 1 -- 172.21.15.32:0/3216995906 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f161c004a30 con 0x7f162c070230 2023-12-20T06:00:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1630998640 1 -- 172.21.15.32:0/3216995906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f162c153ca0 con 0x7f162c070230 2023-12-20T06:00:17.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1630998640 1 -- 172.21.15.32:0/3216995906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f162c151190 con 0x7f162c070230 2023-12-20T06:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1630998640 1 -- 172.21.15.32:0/3216995906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f162c04e000 con 0x7f162c070230 2023-12-20T06:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.207+0000 7f1622ffd640 1 -- 172.21.15.32:0/3216995906 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 108) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f161c004b90 con 0x7f162c070230 2023-12-20T06:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.211+0000 7f1622ffd640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f160c065bd0 0x7f160c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.211+0000 7f1622ffd640 1 -- 172.21.15.32:0/3216995906 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6368+0+0 (secure 0 0 0) 0x7f161c082760 con 0x7f162c070230 2023-12-20T06:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.211+0000 7f1623fff640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f160c065bd0 0x7f160c068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:17.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.211+0000 7f1623fff640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f160c065bd0 0x7f160c068090 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f162c0708f0 tx=0x7f1614008040 comp rx=0 tx=0).ready entity=mgr.9699 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:17.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.215+0000 7f1622ffd640 1 -- 172.21.15.32:0/3216995906 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f161c04fd00 con 0x7f162c070230 2023-12-20T06:00:17.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.391+0000 7f1630998640 1 -- 172.21.15.32:0/3216995906 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "/snap_test_dir1/test_restart", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f162c15b330 con 0x7f160c065bd0 2023-12-20T06:00:17.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1622ffd640 1 -- 172.21.15.32:0/3216995906 <== mgr.9699 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+361 (secure 0 0 0) 0x7f1614009df0 con 0x7f160c065bd0 2023-12-20T06:00:17.396 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:17.396 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T05:44:34", "first": "2023-12-20T05:45:00", "last": "2023-12-20T06:00:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 -- 172.21.15.32:0/3216995906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f160c065bd0 msgr2=0x7f160c068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f160c065bd0 0x7f160c068090 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f162c0708f0 tx=0x7f1614008040 comp rx=0 tx=0).stop 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 -- 172.21.15.32:0/3216995906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c070230 msgr2=0x7f162c153760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c070230 0x7f162c153760 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7f161c00c450 tx=0x7f161c00c910 comp rx=0 tx=0).stop 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 -- 172.21.15.32:0/3216995906 shutdown_connections 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f160c065bd0 0x7f160c068090 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f162c15abb0 0x7f162c14c710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f162c0b0cb0 0x7f162c14c1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 --2- 172.21.15.32:0/3216995906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f162c070230 0x7f162c153760 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 -- 172.21.15.32:0/3216995906 >> 172.21.15.32:0/3216995906 conn(0x7f162c06a510 msgr2=0x7f162c156d40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 -- 172.21.15.32:0/3216995906 shutdown_connections 2023-12-20T06:00:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.399+0000 7f1630998640 1 -- 172.21.15.32:0/3216995906 wait complete. 2023-12-20T06:00:17.417 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "cephfs", "subvol": null, "path": "/snap_test_dir1/test_restart", "rel_path": "/snap_test_dir1/test_restart", "schedule": "1M", "retention": {}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T05:44:34", "first": "2023-12-20T05:45:00", "last": "2023-12-20T06:00:00", "last_pruned": null, "created_count": 16, "pruned_count": 0, "active": false}] 2023-12-20T06:00:17.418 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'fs': 'cephfs', 'subvol': None, 'path': '/snap_test_dir1/test_restart', 'rel_path': '/snap_test_dir1/test_restart', 'schedule': '1M', 'retention': {}, 'start': '2023-12-20T00:00:00', 'created': '2023-12-20T05:44:34', 'first': '2023-12-20T05:45:00', 'last': '2023-12-20T06:00:00', 'last_pruned': None, 'created_count': 16, 'pruned_count': 0, 'active': False} 2023-12-20T06:00:17.418 DEBUG:tasks.cephfs.test_snap_schedules:fs_count: 16 2023-12-20T06:00:17.418 DEBUG:tasks.cephfs.test_snap_schedules:db_count: 16 2023-12-20T06:00:17.418 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove /snap_test_dir1/test_restart 1M 2023-12-20T06:00:17.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa121725640 1 Processor -- start 2023-12-20T06:00:17.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa121725640 1 -- start start 2023-12-20T06:00:17.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa121725640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa11c1591e0 0x7fa11c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa121725640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa11c070940 0x7fa11c0cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa121725640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa11c0cc3b0 con 0x7fa11c06cdf0 2023-12-20T06:00:17.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa121725640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c0c8e30 con 0x7fa11c070940 2023-12-20T06:00:17.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa121725640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa11c06d280 con 0x7fa11c1591e0 2023-12-20T06:00:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa11b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa11c070940 0x7fa11c0cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa11b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa11c070940 0x7fa11c0cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44710/0 (socket says 172.21.15.32:44710) 2023-12-20T06:00:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa11b7fe640 1 -- 172.21.15.32:0/1787363553 learned_addr learned my addr 172.21.15.32:0/1787363553 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa11affd640 1 --2- 172.21.15.32:0/1787363553 >> v2:172.21.15.39:3300/0 conn(0x7fa11c1591e0 0x7fa11c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa11b7fe640 1 -- 172.21.15.32:0/1787363553 >> v2:172.21.15.39:3300/0 conn(0x7fa11c1591e0 msgr2=0x7fa11c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa11b7fe640 1 --2- 172.21.15.32:0/1787363553 >> v2:172.21.15.39:3300/0 conn(0x7fa11c1591e0 0x7fa11c0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa11b7fe640 1 -- 172.21.15.32:0/1787363553 >> v1:172.21.15.32:6789/0 conn(0x7fa11c06cdf0 legacy=0x7fa11c070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa11b7fe640 1 -- 172.21.15.32:0/1787363553 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa11c0c90d0 con 0x7fa11c070940 2023-12-20T06:00:17.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.567+0000 7fa11affd640 1 --2- 172.21.15.32:0/1787363553 >> v2:172.21.15.39:3300/0 conn(0x7fa11c1591e0 0x7fa11c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:17.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa11b7fe640 1 --2- 172.21.15.32:0/1787363553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c070940 0x7fa11c0cc840 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0cf890 tx=0x7fa11401c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebf9c5c5066d905 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:17.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa11a7fc640 1 -- 172.21.15.32:0/1787363553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1140a5650 con 0x7fa11c070940 2023-12-20T06:00:17.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa11a7fc640 1 -- 172.21.15.32:0/1787363553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa1140a57b0 con 0x7fa11c070940 2023-12-20T06:00:17.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa11a7fc640 1 -- 172.21.15.32:0/1787363553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa1140a5b40 con 0x7fa11c070940 2023-12-20T06:00:17.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa121725640 1 -- 172.21.15.32:0/1787363553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c070940 msgr2=0x7fa11c0cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa121725640 1 --2- 172.21.15.32:0/1787363553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c070940 0x7fa11c0cc840 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fa11c0cf890 tx=0x7fa11401c940 comp rx=0 tx=0).stop 2023-12-20T06:00:17.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa121725640 1 -- 172.21.15.32:0/1787363553 shutdown_connections 2023-12-20T06:00:17.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa121725640 1 --2- 172.21.15.32:0/1787363553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c070940 0x7fa11c0cc840 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa121725640 1 --2- 172.21.15.32:0/1787363553 >> v2:172.21.15.39:3300/0 conn(0x7fa11c1591e0 0x7fa11c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa121725640 1 -- 172.21.15.32:0/1787363553 >> 172.21.15.32:0/1787363553 conn(0x7fa11c06a510 msgr2=0x7fa11c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:17.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa121725640 1 -- 172.21.15.32:0/1787363553 shutdown_connections 2023-12-20T06:00:17.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.571+0000 7fa121725640 1 -- 172.21.15.32:0/1787363553 wait complete. 2023-12-20T06:00:17.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa121725640 1 Processor -- start 2023-12-20T06:00:17.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa121725640 1 -- start start 2023-12-20T06:00:17.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa121725640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa11c070940 0x7fa11c153920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa121725640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c1591e0 0x7fa11c14aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa121725640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c14b8d0 0x7fa11c14bd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa121725640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c14c2c0 con 0x7fa11c1591e0 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa121725640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c14e9b0 con 0x7fa11c14b8d0 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa121725640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa11c153e60 con 0x7fa11c070940 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa113fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c1591e0 0x7fa11c14aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa113fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c1591e0 0x7fa11c14aeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44722/0 (socket says 172.21.15.32:44722) 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa113fff640 1 -- 172.21.15.32:0/2217418192 learned_addr learned my addr 172.21.15.32:0/2217418192 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa11b7fe640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c14b8d0 0x7fa11c14bd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa11affd640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa11c070940 0x7fa11c153920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa113fff640 1 -- 172.21.15.32:0/2217418192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa11c070940 msgr2=0x7fa11c153920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa113fff640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa11c070940 0x7fa11c153920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa113fff640 1 -- 172.21.15.32:0/2217418192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c14b8d0 msgr2=0x7fa11c14bd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa113fff640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c14b8d0 0x7fa11c14bd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa113fff640 1 -- 172.21.15.32:0/2217418192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa114009b30 con 0x7fa11c1591e0 2023-12-20T06:00:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa113fff640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c1591e0 0x7fa11c14aeb0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa10400c500 tx=0x7fa10400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa1137fe640 1 -- 172.21.15.32:0/2217418192 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104005220 con 0x7fa11c1591e0 2023-12-20T06:00:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa1137fe640 1 -- 172.21.15.32:0/2217418192 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa104005380 con 0x7fa11c1591e0 2023-12-20T06:00:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa121725640 1 -- 172.21.15.32:0/2217418192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa11c149e10 con 0x7fa11c1591e0 2023-12-20T06:00:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa1137fe640 1 -- 172.21.15.32:0/2217418192 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa104007270 con 0x7fa11c1591e0 2023-12-20T06:00:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa121725640 1 -- 172.21.15.32:0/2217418192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa11c14a3d0 con 0x7fa11c1591e0 2023-12-20T06:00:17.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.575+0000 7fa121725640 1 -- 172.21.15.32:0/2217418192 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa11c04e000 con 0x7fa11c1591e0 2023-12-20T06:00:17.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.579+0000 7fa1137fe640 1 -- 172.21.15.32:0/2217418192 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 108) v1 ==== 85359+0+0 (secure 0 0 0) 0x7fa1040073d0 con 0x7fa11c1591e0 2023-12-20T06:00:17.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.579+0000 7fa1137fe640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa0fc065bd0 0x7fa0fc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:17.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.579+0000 7fa1137fe640 1 -- 172.21.15.32:0/2217418192 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6368+0+0 (secure 0 0 0) 0x7fa104082ea0 con 0x7fa11c1591e0 2023-12-20T06:00:17.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.579+0000 7fa11affd640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa0fc065bd0 0x7fa0fc068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:17.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.579+0000 7fa11affd640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa0fc065bd0 0x7fa0fc068090 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa11c14e3e0 tx=0x7fa10c008040 comp rx=0 tx=0).ready entity=mgr.9699 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.583+0000 7fa1137fe640 1 -- 172.21.15.32:0/2217418192 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa104050500 con 0x7fa11c1591e0 2023-12-20T06:00:17.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.763+0000 7fa121725640 1 -- 172.21.15.32:0/2217418192 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "/snap_test_dir1/test_restart", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fa11c0caf70 con 0x7fa0fc065bd0 2023-12-20T06:00:17.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.827+0000 7fa1137fe640 1 -- 172.21.15.32:0/2217418192 <== mgr.9699 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7fa10c004260 con 0x7fa0fc065bd0 2023-12-20T06:00:17.829 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path /snap_test_dir1/test_restart 2023-12-20T06:00:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 -- 172.21.15.32:0/2217418192 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa0fc065bd0 msgr2=0x7fa0fc068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa0fc065bd0 0x7fa0fc068090 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7fa11c14e3e0 tx=0x7fa10c008040 comp rx=0 tx=0).stop 2023-12-20T06:00:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 -- 172.21.15.32:0/2217418192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c1591e0 msgr2=0x7fa11c14aeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c1591e0 0x7fa11c14aeb0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fa10400c500 tx=0x7fa10400c9c0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 -- 172.21.15.32:0/2217418192 shutdown_connections 2023-12-20T06:00:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa0fc065bd0 0x7fa0fc068090 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa11c14b8d0 0x7fa11c14bd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa11c1591e0 0x7fa11c14aeb0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 --2- 172.21.15.32:0/2217418192 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa11c070940 0x7fa11c153920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:17.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 -- 172.21.15.32:0/2217418192 >> 172.21.15.32:0/2217418192 conn(0x7fa11c06a510 msgr2=0x7fa11c0c4070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:17.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 -- 172.21.15.32:0/2217418192 shutdown_connections 2023-12-20T06:00:17.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:17.831+0000 7fa121725640 1 -- 172.21.15.32:0/2217418192 wait complete. 2023-12-20T06:00:17.850 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path /snap_test_dir1/test_restart 2023-12-20T06:00:17.850 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:17.850 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/test_restart/.snap) 2023-12-20T06:00:17.861 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_45_00_UTC 2023-12-20T06:00:17.861 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_46_00_UTC 2023-12-20T06:00:17.861 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_47_00_UTC 2023-12-20T06:00:17.861 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_48_00_UTC 2023-12-20T06:00:17.861 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_49_00_UTC 2023-12-20T06:00:17.861 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_50_00_UTC 2023-12-20T06:00:17.861 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_51_00_UTC 2023-12-20T06:00:17.862 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_52_00_UTC 2023-12-20T06:00:17.862 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_53_00_UTC 2023-12-20T06:00:17.862 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_54_00_UTC 2023-12-20T06:00:17.862 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_55_00_UTC 2023-12-20T06:00:17.862 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_56_00_UTC 2023-12-20T06:00:17.862 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_57_00_UTC 2023-12-20T06:00:17.862 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_58_00_UTC 2023-12-20T06:00:17.862 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-05_59_00_UTC 2023-12-20T06:00:17.862 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_00_00_UTC 2023-12-20T06:00:17.863 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_45_00_UTC 2023-12-20T06:00:17.864 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:17.864 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_45_00_UTC) 2023-12-20T06:00:17.918 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_46_00_UTC 2023-12-20T06:00:17.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:17.918 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_46_00_UTC) 2023-12-20T06:00:17.974 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_47_00_UTC 2023-12-20T06:00:17.974 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:17.975 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_47_00_UTC) 2023-12-20T06:00:18.030 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_48_00_UTC 2023-12-20T06:00:18.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.030 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_48_00_UTC) 2023-12-20T06:00:18.087 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_49_00_UTC 2023-12-20T06:00:18.087 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.087 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_49_00_UTC) 2023-12-20T06:00:18.142 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_50_00_UTC 2023-12-20T06:00:18.142 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.142 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_50_00_UTC) 2023-12-20T06:00:18.198 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_51_00_UTC 2023-12-20T06:00:18.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.198 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_51_00_UTC) 2023-12-20T06:00:18.254 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_52_00_UTC 2023-12-20T06:00:18.254 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.254 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_52_00_UTC) 2023-12-20T06:00:18.310 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_53_00_UTC 2023-12-20T06:00:18.310 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.310 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_53_00_UTC) 2023-12-20T06:00:18.366 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_54_00_UTC 2023-12-20T06:00:18.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.366 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_54_00_UTC) 2023-12-20T06:00:18.426 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_55_00_UTC 2023-12-20T06:00:18.426 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.426 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_55_00_UTC) 2023-12-20T06:00:18.481 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_56_00_UTC 2023-12-20T06:00:18.482 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.482 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_56_00_UTC) 2023-12-20T06:00:18.542 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_57_00_UTC 2023-12-20T06:00:18.542 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.542 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_57_00_UTC) 2023-12-20T06:00:18.597 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_58_00_UTC 2023-12-20T06:00:18.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.598 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_58_00_UTC) 2023-12-20T06:00:18.653 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_59_00_UTC 2023-12-20T06:00:18.654 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.654 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-05_59_00_UTC) 2023-12-20T06:00:18.714 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-06_00_00_UTC 2023-12-20T06:00:18.714 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.714 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart/.snap/scheduled-2023-12-20-06_00_00_UTC) 2023-12-20T06:00:18.770 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:18.770 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/test_restart) 2023-12-20T06:00:18.824 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T06:00:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f390072c640 1 Processor -- start 2023-12-20T06:00:19.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f390072c640 1 -- start start 2023-12-20T06:00:19.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f390072c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 0x7f38f80b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:19.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f390072c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f38f80cc4b0 0x7f38f80cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:19.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f390072c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38f80b1100 con 0x7f38f8157480 2023-12-20T06:00:19.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f390072c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38f80c9f50 con 0x7f38f80cc4b0 2023-12-20T06:00:19.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f390072c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80cde20 con 0x7f38f80b0760 2023-12-20T06:00:19.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f38fecc9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f38f80cc4b0 0x7f38f80cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:19.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f38fdcc7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 0x7f38f80b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:19.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f38fecc9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f38f80cc4b0 0x7f38f80cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:37202/0 (socket says 172.21.15.32:37202) 2023-12-20T06:00:19.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f38fdcc7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 0x7f38f80b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:42976/0 (socket says 172.21.15.32:42976) 2023-12-20T06:00:19.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f38fecc9640 1 -- 172.21.15.32:0/2259024113 learned_addr learned my addr 172.21.15.32:0/2259024113 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:19.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f38fdcc7640 1 -- 172.21.15.32:0/2259024113 >> v2:172.21.15.39:3300/0 conn(0x7f38f80cc4b0 msgr2=0x7f38f80cc940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:19.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f38fdcc7640 1 --2- 172.21.15.32:0/2259024113 >> v2:172.21.15.39:3300/0 conn(0x7f38f80cc4b0 0x7f38f80cc940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:19.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f38fdcc7640 1 -- 172.21.15.32:0/2259024113 >> v1:172.21.15.39:6789/0 conn(0x7f38f8157480 legacy=0x7f38f80c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:19.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.011+0000 7f38fdcc7640 1 -- 172.21.15.32:0/2259024113 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38f80c8e30 con 0x7f38f80b0760 2023-12-20T06:00:19.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f38fdcc7640 1 --2- 172.21.15.32:0/2259024113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 0x7f38f80b0bc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f38e80097c0 tx=0x7f38e801e5d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eb700001de64ca37 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:19.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f38fd4c6640 1 -- 172.21.15.32:0/2259024113 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80b2600 con 0x7f38f80b0760 2023-12-20T06:00:19.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f38fd4c6640 1 -- 172.21.15.32:0/2259024113 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e80b2760 con 0x7f38f80b0760 2023-12-20T06:00:19.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f38fd4c6640 1 -- 172.21.15.32:0/2259024113 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80b2af0 con 0x7f38f80b0760 2023-12-20T06:00:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f390072c640 1 -- 172.21.15.32:0/2259024113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 msgr2=0x7f38f80b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f390072c640 1 --2- 172.21.15.32:0/2259024113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 0x7f38f80b0bc0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7f38e80097c0 tx=0x7f38e801e5d0 comp rx=0 tx=0).stop 2023-12-20T06:00:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f390072c640 1 -- 172.21.15.32:0/2259024113 shutdown_connections 2023-12-20T06:00:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f390072c640 1 --2- 172.21.15.32:0/2259024113 >> v2:172.21.15.39:3300/0 conn(0x7f38f80cc4b0 0x7f38f80cc940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f390072c640 1 --2- 172.21.15.32:0/2259024113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 0x7f38f80b0bc0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f390072c640 1 -- 172.21.15.32:0/2259024113 >> 172.21.15.32:0/2259024113 conn(0x7f38f806a510 msgr2=0x7f38f80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f390072c640 1 -- 172.21.15.32:0/2259024113 shutdown_connections 2023-12-20T06:00:19.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.015+0000 7f390072c640 1 -- 172.21.15.32:0/2259024113 wait complete. 2023-12-20T06:00:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f390072c640 1 Processor -- start 2023-12-20T06:00:19.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f390072c640 1 -- start start 2023-12-20T06:00:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f390072c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 0x7f38f80b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f390072c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80cc4b0 0x7f38f815d600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f390072c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38f8157480 0x7f38f81f0d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f390072c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80b38c0 con 0x7f38f80cc4b0 2023-12-20T06:00:19.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f390072c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80b2580 con 0x7f38f8157480 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f390072c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f38f80b2820 con 0x7f38f80b0760 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fdcc7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80cc4b0 0x7f38f815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fdcc7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80cc4b0 0x7f38f815d600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59974/0 (socket says 172.21.15.32:59974) 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fdcc7640 1 -- 172.21.15.32:0/3136350261 learned_addr learned my addr 172.21.15.32:0/3136350261 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fecc9640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38f8157480 0x7f38f81f0d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fe4c8640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 0x7f38f80b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fdcc7640 1 -- 172.21.15.32:0/3136350261 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 msgr2=0x7f38f80b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fdcc7640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 0x7f38f80b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fdcc7640 1 -- 172.21.15.32:0/3136350261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38f8157480 msgr2=0x7f38f81f0d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fdcc7640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38f8157480 0x7f38f81f0d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fdcc7640 1 -- 172.21.15.32:0/3136350261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38e8009420 con 0x7f38f80cc4b0 2023-12-20T06:00:19.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fecc9640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38f8157480 0x7f38f81f0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38fdcc7640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80cc4b0 0x7f38f815d600 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f38e801cc80 tx=0x7f38e801b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38eeffd640 1 -- 172.21.15.32:0/3136350261 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80be070 con 0x7f38f80cc4b0 2023-12-20T06:00:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38eeffd640 1 -- 172.21.15.32:0/3136350261 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38e80b2c20 con 0x7f38f80cc4b0 2023-12-20T06:00:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f390072c640 1 -- 172.21.15.32:0/3136350261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f38f80b2b00 con 0x7f38f80cc4b0 2023-12-20T06:00:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f38eeffd640 1 -- 172.21.15.32:0/3136350261 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38e80b6040 con 0x7f38f80cc4b0 2023-12-20T06:00:19.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f390072c640 1 -- 172.21.15.32:0/3136350261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38f80b30c0 con 0x7f38f80cc4b0 2023-12-20T06:00:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.019+0000 7f390072c640 1 -- 172.21.15.32:0/3136350261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f38f804e000 con 0x7f38f80cc4b0 2023-12-20T06:00:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.023+0000 7f38eeffd640 1 -- 172.21.15.32:0/3136350261 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 108) v1 ==== 85359+0+0 (secure 0 0 0) 0x7f38f80cd690 con 0x7f38f80cc4b0 2023-12-20T06:00:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.023+0000 7f38eeffd640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f38dc065bd0 0x7f38dc068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.023+0000 7f38eeffd640 1 -- 172.21.15.32:0/3136350261 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6368+0+0 (secure 0 0 0) 0x7f38e811f9f0 con 0x7f38f80cc4b0 2023-12-20T06:00:19.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.023+0000 7f38fe4c8640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f38dc065bd0 0x7f38dc068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:19.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.023+0000 7f38fe4c8640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f38dc065bd0 0x7f38dc068090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38f8153640 tx=0x7f38f4009040 comp rx=0 tx=0).ready entity=mgr.9699 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:19.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.027+0000 7f38eeffd640 1 -- 172.21.15.32:0/3136350261 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f38e80ece80 con 0x7f38f80cc4b0 2023-12-20T06:00:19.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.223+0000 7f390072c640 1 -- 172.21.15.32:0/3136350261 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7f38f80b43d0 con 0x7f38f80cc4b0 2023-12-20T06:00:19.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.711+0000 7f38eeffd640 1 -- 172.21.15.32:0/3136350261 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 109) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f38e8121d50 con 0x7f38f80cc4b0 2023-12-20T06:00:19.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.711+0000 7f38eeffd640 1 -- 172.21.15.32:0/3136350261 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v109) v1 ==== 93+0+0 (secure 0 0 0) 0x7f38e80f20a0 con 0x7f38f80cc4b0 2023-12-20T06:00:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 -- 172.21.15.32:0/3136350261 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f38dc065bd0 msgr2=0x7f38dc068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f38dc065bd0 0x7f38dc068090 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f38f8153640 tx=0x7f38f4009040 comp rx=0 tx=0).stop 2023-12-20T06:00:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 -- 172.21.15.32:0/3136350261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80cc4b0 msgr2=0x7f38f815d600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80cc4b0 0x7f38f815d600 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7f38e801cc80 tx=0x7f38e801b370 comp rx=0 tx=0).stop 2023-12-20T06:00:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 -- 172.21.15.32:0/3136350261 shutdown_connections 2023-12-20T06:00:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f38dc065bd0 0x7f38dc068090 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38f8157480 0x7f38f81f0d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:19.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38f80cc4b0 0x7f38f815d600 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 --2- 172.21.15.32:0/3136350261 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38f80b0760 0x7f38f80b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 -- 172.21.15.32:0/3136350261 >> 172.21.15.32:0/3136350261 conn(0x7f38f806a510 msgr2=0x7f38f80c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 -- 172.21.15.32:0/3136350261 shutdown_connections 2023-12-20T06:00:19.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:19.715+0000 7f390072c640 1 -- 172.21.15.32:0/3136350261 wait complete. 2023-12-20T06:00:19.733 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T06:00:19.733 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:00:19.819 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:00:19.861 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T06:00:19.861 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:00:19.862 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:00:19.872 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:00:19.872 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:00:19.890 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:19.891+0000 7f78bacdc180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:00:19.891 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:19.891+0000 7f6d11541180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:00:19.901 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:00:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabcd220640 1 Processor -- start 2023-12-20T06:00:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabcd220640 1 -- start start 2023-12-20T06:00:20.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabcd220640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabc815ce30 0x7fabc8161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabcd220640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fabc806deb0 0x7fabc80cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabcd220640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabc8071920 con 0x7fabc8070950 2023-12-20T06:00:20.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabcd220640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc8071080 con 0x7fabc815ce30 2023-12-20T06:00:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabcd220640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabc8071360 con 0x7fabc806deb0 2023-12-20T06:00:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabbffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fabc8070950 0x7fabc806d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53498/0 (socket says 172.21.15.32:53498) 2023-12-20T06:00:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabbffff640 1 -- 172.21.15.32:0/3527954788 learned_addr learned my addr 172.21.15.32:0/3527954788 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabc6d9d640 1 --2- 172.21.15.32:0/3527954788 >> v2:172.21.15.32:3300/0 conn(0x7fabc815ce30 0x7fabc8161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabc759e640 1 --2- 172.21.15.32:0/3527954788 >> v2:172.21.15.39:3300/0 conn(0x7fabc806deb0 0x7fabc80cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabc6d9d640 1 -- 172.21.15.32:0/3527954788 >> v2:172.21.15.39:3300/0 conn(0x7fabc806deb0 msgr2=0x7fabc80cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabc6d9d640 1 --2- 172.21.15.32:0/3527954788 >> v2:172.21.15.39:3300/0 conn(0x7fabc806deb0 0x7fabc80cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabc6d9d640 1 -- 172.21.15.32:0/3527954788 >> v1:172.21.15.32:6789/0 conn(0x7fabc8070950 legacy=0x7fabc806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabc6d9d640 1 -- 172.21.15.32:0/3527954788 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabc8071640 con 0x7fabc815ce30 2023-12-20T06:00:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.091+0000 7fabc659c640 1 -- 172.21.15.32:0/3527954788 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1593721676 0 0) 0x7fabc8071920 con 0x7fabc8070950 2023-12-20T06:00:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabc6d9d640 1 --2- 172.21.15.32:0/3527954788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc815ce30 0x7fabc8161560 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fabc80c95b0 tx=0x7fabb801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42be3ba12381ea99 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabc659c640 1 -- 172.21.15.32:0/3527954788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb80ab600 con 0x7fabc815ce30 2023-12-20T06:00:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabc659c640 1 -- 172.21.15.32:0/3527954788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb80a7050 con 0x7fabc815ce30 2023-12-20T06:00:20.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabc659c640 1 -- 172.21.15.32:0/3527954788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb80ab980 con 0x7fabc815ce30 2023-12-20T06:00:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabcd220640 1 -- 172.21.15.32:0/3527954788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc815ce30 msgr2=0x7fabc8161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabcd220640 1 --2- 172.21.15.32:0/3527954788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc815ce30 0x7fabc8161560 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7fabc80c95b0 tx=0x7fabb801c3d0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabcd220640 1 -- 172.21.15.32:0/3527954788 shutdown_connections 2023-12-20T06:00:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabcd220640 1 --2- 172.21.15.32:0/3527954788 >> v2:172.21.15.39:3300/0 conn(0x7fabc806deb0 0x7fabc80cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabcd220640 1 --2- 172.21.15.32:0/3527954788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc815ce30 0x7fabc8161560 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabcd220640 1 -- 172.21.15.32:0/3527954788 >> 172.21.15.32:0/3527954788 conn(0x7fabc80cba50 msgr2=0x7fabc80cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabcd220640 1 -- 172.21.15.32:0/3527954788 shutdown_connections 2023-12-20T06:00:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.095+0000 7fabcd220640 1 -- 172.21.15.32:0/3527954788 wait complete. 2023-12-20T06:00:20.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabcd220640 1 Processor -- start 2023-12-20T06:00:20.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabcd220640 1 -- start start 2023-12-20T06:00:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabcd220640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc806deb0 0x7fabc8157080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabcd220640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabc8070950 0x7fabc8159d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabcd220640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabc815ce30 0x7fabc815a250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabcd220640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b42b0 con 0x7fabc806deb0 2023-12-20T06:00:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabcd220640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b4590 con 0x7fabc815ce30 2023-12-20T06:00:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabcd220640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabc80b4870 con 0x7fabc8070950 2023-12-20T06:00:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabc6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc806deb0 0x7fabc8157080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabc6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc806deb0 0x7fabc8157080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60000/0 (socket says 172.21.15.32:60000) 2023-12-20T06:00:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabc6d9d640 1 -- 172.21.15.32:0/4267077700 learned_addr learned my addr 172.21.15.32:0/4267077700 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabc759e640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabc815ce30 0x7fabc815a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabbffff640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabc8070950 0x7fabc8159d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabc6d9d640 1 -- 172.21.15.32:0/4267077700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabc8070950 msgr2=0x7fabc8159d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabc6d9d640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabc8070950 0x7fabc8159d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabc6d9d640 1 -- 172.21.15.32:0/4267077700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabc815ce30 msgr2=0x7fabc815a250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabc6d9d640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabc815ce30 0x7fabc815a250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabc6d9d640 1 -- 172.21.15.32:0/4267077700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb8009420 con 0x7fabc806deb0 2023-12-20T06:00:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabbffff640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabc8070950 0x7fabc8159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabc6d9d640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc806deb0 0x7fabc8157080 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fabb80ae040 tx=0x7fabb8019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabbf7fe640 1 -- 172.21.15.32:0/4267077700 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb80b4410 con 0x7fabc806deb0 2023-12-20T06:00:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabbf7fe640 1 -- 172.21.15.32:0/4267077700 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb80b4570 con 0x7fabc806deb0 2023-12-20T06:00:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabcd220640 1 -- 172.21.15.32:0/4267077700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabc80b4b50 con 0x7fabc806deb0 2023-12-20T06:00:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabbf7fe640 1 -- 172.21.15.32:0/4267077700 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb80b4920 con 0x7fabc806deb0 2023-12-20T06:00:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.099+0000 7fabcd220640 1 -- 172.21.15.32:0/4267077700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabc80b34e0 con 0x7fabc806deb0 2023-12-20T06:00:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.103+0000 7fabcd220640 1 -- 172.21.15.32:0/4267077700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabc80cb8c0 con 0x7fabc806deb0 2023-12-20T06:00:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.103+0000 7fabbf7fe640 1 -- 172.21.15.32:0/4267077700 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 109) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fabb80b2040 con 0x7fabc806deb0 2023-12-20T06:00:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.103+0000 7fabbf7fe640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 0x7faba8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.103+0000 7fabbf7fe640 1 -- 172.21.15.32:0/4267077700 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6368+0+0 (secure 0 0 0) 0x7fabb811d560 con 0x7fabc806deb0 2023-12-20T06:00:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.103+0000 7fabbffff640 1 -- 172.21.15.32:0/4267077700 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 msgr2=0x7faba8068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.103+0000 7fabbffff640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 0x7faba8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:20.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.107+0000 7fabbf7fe640 1 -- 172.21.15.32:0/4267077700 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fabb80eabf0 con 0x7fabc806deb0 2023-12-20T06:00:20.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.267+0000 7fabcd220640 1 -- 172.21.15.32:0/4267077700 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fabc804e000 con 0x7fabc806deb0 2023-12-20T06:00:20.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.267+0000 7fabbf7fe640 1 -- 172.21.15.32:0/4267077700 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v177) v1 ==== 74+0+17018 (secure 0 0 0) 0x7fabb80ee210 con 0x7fabc806deb0 2023-12-20T06:00:20.268 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:20.268 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":177,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T05:55:20.531168+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:44:18.671073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-20T05:44:20.010875+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"173","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2721040122":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/3587707827":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2000830842":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2272871714":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2161329994":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/663256086":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2039923182":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/3732035268":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/3971579268":"2023-12-21T05:44:33.158109+0000","172.21.15.32:6833/38009":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/1097027017":"2023-12-21T05:44:33.158109+0000","172.21.15.32:6832/38009":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/2617565032":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/2748163648":"2023-12-21T05:44:33.158109+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:00:20.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 -- 172.21.15.32:0/4267077700 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 msgr2=0x7faba8068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:20.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 0x7faba8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 -- 172.21.15.32:0/4267077700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc806deb0 msgr2=0x7fabc8157080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc806deb0 0x7fabc8157080 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7fabb80ae040 tx=0x7fabb8019170 comp rx=0 tx=0).stop 2023-12-20T06:00:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 -- 172.21.15.32:0/4267077700 shutdown_connections 2023-12-20T06:00:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 0x7faba8068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabc815ce30 0x7fabc815a250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabc8070950 0x7fabc8159d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 --2- 172.21.15.32:0/4267077700 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabc806deb0 0x7fabc8157080 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 -- 172.21.15.32:0/4267077700 >> 172.21.15.32:0/4267077700 conn(0x7fabc80cba50 msgr2=0x7fabc809d330 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 -- 172.21.15.32:0/4267077700 shutdown_connections 2023-12-20T06:00:20.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.271+0000 7fabcd220640 1 -- 172.21.15.32:0/4267077700 wait complete. 2023-12-20T06:00:20.291 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T05:44:18.671073+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T05:44:20.010875+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '173', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:00:20.291 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:00:20.326 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:20.327+0000 7f78bacdc180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:00:20.340 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:20.343+0000 7f6d11541180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:00:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f94a9295640 1 Processor -- start 2023-12-20T06:00:20.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f94a9295640 1 -- start start 2023-12-20T06:00:20.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f94a9295640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f94a415ce50 0x7f94a406b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f94a9295640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a40b0550 con 0x7f94a406bb80 2023-12-20T06:00:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f94a9295640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f94a406bfc0 con 0x7f94a40cf7a0 2023-12-20T06:00:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f94a9295640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f94a40aff20 con 0x7f94a415ce50 2023-12-20T06:00:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f949bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f94a406bb80 0x7f94a40cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53502/0 (socket says 172.21.15.32:53502) 2023-12-20T06:00:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f949bfff640 1 -- 172.21.15.32:0/3191863337 learned_addr learned my addr 172.21.15.32:0/3191863337 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f94a2ffd640 1 --2- 172.21.15.32:0/3191863337 >> v2:172.21.15.32:3300/0 conn(0x7f94a415ce50 0x7f94a406b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f94a27fc640 1 -- 172.21.15.32:0/3191863337 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3921137680 0 0) 0x7f94a40b0550 con 0x7f94a406bb80 2023-12-20T06:00:20.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f94a2ffd640 1 -- 172.21.15.32:0/3191863337 >> v1:172.21.15.39:6789/0 conn(0x7f94a40cf7a0 legacy=0x7f94a40b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f94a2ffd640 1 -- 172.21.15.32:0/3191863337 >> v1:172.21.15.32:6789/0 conn(0x7f94a406bb80 legacy=0x7f94a40cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.447+0000 7f94a2ffd640 1 -- 172.21.15.32:0/3191863337 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f94a40cfdc0 con 0x7f94a415ce50 2023-12-20T06:00:20.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.451+0000 7f94a2ffd640 1 --2- 172.21.15.32:0/3191863337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a415ce50 0x7f94a406b640 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f94940097c0 tx=0x7f949401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca5311487dde787b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:20.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.451+0000 7f94a27fc640 1 -- 172.21.15.32:0/3191863337 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94940ad660 con 0x7f94a415ce50 2023-12-20T06:00:20.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.451+0000 7f94a27fc640 1 -- 172.21.15.32:0/3191863337 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f94940ad7c0 con 0x7f94a415ce50 2023-12-20T06:00:20.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.451+0000 7f94a27fc640 1 -- 172.21.15.32:0/3191863337 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f94940adb50 con 0x7f94a415ce50 2023-12-20T06:00:20.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.451+0000 7f94a9295640 1 -- 172.21.15.32:0/3191863337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a415ce50 msgr2=0x7f94a406b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.451+0000 7f94a9295640 1 --2- 172.21.15.32:0/3191863337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a415ce50 0x7f94a406b640 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7f94940097c0 tx=0x7f949401c450 comp rx=0 tx=0).stop 2023-12-20T06:00:20.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.451+0000 7f94a9295640 1 -- 172.21.15.32:0/3191863337 shutdown_connections 2023-12-20T06:00:20.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.451+0000 7f94a9295640 1 --2- 172.21.15.32:0/3191863337 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a415ce50 0x7f94a406b640 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.451+0000 7f94a9295640 1 -- 172.21.15.32:0/3191863337 >> 172.21.15.32:0/3191863337 conn(0x7f94a4099d70 msgr2=0x7f94a40d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:20.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.451+0000 7f94a9295640 1 -- 172.21.15.32:0/3191863337 shutdown_connections 2023-12-20T06:00:20.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.451+0000 7f94a9295640 1 -- 172.21.15.32:0/3191863337 wait complete. 2023-12-20T06:00:20.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.455+0000 7f94a9295640 1 Processor -- start 2023-12-20T06:00:20.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.455+0000 7f94a9295640 1 -- start start 2023-12-20T06:00:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a9295640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f94a406bb80 0x7f94a414a910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a9295640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a40cf7a0 0x7f94a415e480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a9295640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a415ce50 0x7f94a406f010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a9295640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a41529f0 con 0x7f94a415ce50 2023-12-20T06:00:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a9295640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f94a414afa0 con 0x7f94a40cf7a0 2023-12-20T06:00:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a9295640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f94a414b280 con 0x7f94a406bb80 2023-12-20T06:00:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a415ce50 0x7f94a406f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a2ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f94a406bb80 0x7f94a414a910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a415ce50 0x7f94a406f010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60022/0 (socket says 172.21.15.32:60022) 2023-12-20T06:00:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a37fe640 1 -- 172.21.15.32:0/287714785 learned_addr learned my addr 172.21.15.32:0/287714785 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:20.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f949bfff640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a40cf7a0 0x7f94a415e480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a37fe640 1 -- 172.21.15.32:0/287714785 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f94a406bb80 msgr2=0x7f94a414a910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a37fe640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f94a406bb80 0x7f94a414a910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a37fe640 1 -- 172.21.15.32:0/287714785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a40cf7a0 msgr2=0x7f94a415e480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a37fe640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a40cf7a0 0x7f94a415e480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a37fe640 1 -- 172.21.15.32:0/287714785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9494009420 con 0x7f94a415ce50 2023-12-20T06:00:20.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a37fe640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a415ce50 0x7f94a406f010 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c770 tx=0x7f949c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f949b7fe640 1 -- 172.21.15.32:0/287714785 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c003b80 con 0x7f94a415ce50 2023-12-20T06:00:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f949b7fe640 1 -- 172.21.15.32:0/287714785 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f949c003ce0 con 0x7f94a415ce50 2023-12-20T06:00:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f949b7fe640 1 -- 172.21.15.32:0/287714785 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f949c00aa90 con 0x7f94a415ce50 2023-12-20T06:00:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a9295640 1 -- 172.21.15.32:0/287714785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f94a414b560 con 0x7f94a415ce50 2023-12-20T06:00:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a9295640 1 -- 172.21.15.32:0/287714785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f94a414a1b0 con 0x7f94a415ce50 2023-12-20T06:00:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f94a9295640 1 -- 172.21.15.32:0/287714785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f94a404e000 con 0x7f94a415ce50 2023-12-20T06:00:20.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.459+0000 7f949b7fe640 1 -- 172.21.15.32:0/287714785 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 109) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f949c00abf0 con 0x7f94a415ce50 2023-12-20T06:00:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.463+0000 7f949b7fe640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9484065b80 0x7f9484068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.463+0000 7f949b7fe640 1 -- 172.21.15.32:0/287714785 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6368+0+0 (secure 0 0 0) 0x7f949c083270 con 0x7f94a415ce50 2023-12-20T06:00:20.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.463+0000 7f94a2ffd640 1 -- 172.21.15.32:0/287714785 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9484065b80 msgr2=0x7f9484068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:20.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.463+0000 7f94a2ffd640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9484065b80 0x7f9484068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:20.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.467+0000 7f949b7fe640 1 -- 172.21.15.32:0/287714785 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f949c050950 con 0x7f94a415ce50 2023-12-20T06:00:20.484 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:20.483+0000 7f78bacdc180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:00:20.504 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:20.507+0000 7f6d11541180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:00:20.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.623+0000 7f94a9295640 1 -- 172.21.15.32:0/287714785 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f94a40ce120 con 0x7f94a415ce50 2023-12-20T06:00:20.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.623+0000 7f949b7fe640 1 -- 172.21.15.32:0/287714785 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v177) v1 ==== 93+0+10 (secure 0 0 0) 0x7f949c053f70 con 0x7f94a415ce50 2023-12-20T06:00:20.623 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:00:20.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 -- 172.21.15.32:0/287714785 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9484065b80 msgr2=0x7f9484068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9484065b80 0x7f9484068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 -- 172.21.15.32:0/287714785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a415ce50 msgr2=0x7f94a406f010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a415ce50 0x7f94a406f010 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f949c00c770 tx=0x7f949c00cc30 comp rx=0 tx=0).stop 2023-12-20T06:00:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 -- 172.21.15.32:0/287714785 shutdown_connections 2023-12-20T06:00:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9484065b80 0x7f9484068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f94a415ce50 0x7f94a406f010 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f94a40cf7a0 0x7f94a415e480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 --2- 172.21.15.32:0/287714785 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f94a406bb80 0x7f94a414a910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 -- 172.21.15.32:0/287714785 >> 172.21.15.32:0/287714785 conn(0x7f94a4099d70 msgr2=0x7f94a406e500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 -- 172.21.15.32:0/287714785 shutdown_connections 2023-12-20T06:00:20.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.627+0000 7f94a9295640 1 -- 172.21.15.32:0/287714785 wait complete. 2023-12-20T06:00:20.646 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:00:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.799+0000 7fce31396640 1 Processor -- start 2023-12-20T06:00:20.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.799+0000 7fce31396640 1 -- start start 2023-12-20T06:00:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.799+0000 7fce31396640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce2c15abb0 0x7fce2c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.799+0000 7fce31396640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2c15b6a0 con 0x7fce2c070230 2023-12-20T06:00:20.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.799+0000 7fce31396640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce2c06d8c0 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.799+0000 7fce31396640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c06e160 con 0x7fce2c15abb0 2023-12-20T06:00:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.799+0000 7fce2b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fce2c070230 0x7fce2c06c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53514/0 (socket says 172.21.15.32:53514) 2023-12-20T06:00:20.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.799+0000 7fce2b7fe640 1 -- 172.21.15.32:0/3836344508 learned_addr learned my addr 172.21.15.32:0/3836344508 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.799+0000 7fce2affd640 1 --2- 172.21.15.32:0/3836344508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce2c15abb0 0x7fce2c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.803+0000 7fce29ffb640 1 -- 172.21.15.32:0/3836344508 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1076790753 0 0) 0x7fce2c15b6a0 con 0x7fce2c070230 2023-12-20T06:00:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.803+0000 7fce29ffb640 1 -- 172.21.15.32:0/3836344508 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce14003530 con 0x7fce2c070230 2023-12-20T06:00:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.803+0000 7fce29ffb640 1 -- 172.21.15.32:0/3836344508 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845444753 0 0) 0x7fce2c06d8c0 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.803+0000 7fce29ffb640 1 -- 172.21.15.32:0/3836344508 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce2c15b6a0 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.803+0000 7fce2affd640 1 -- 172.21.15.32:0/3836344508 >> v1:172.21.15.39:6789/0 conn(0x7fce2c0b0cb0 legacy=0x7fce2c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.803+0000 7fce2affd640 1 -- 172.21.15.32:0/3836344508 >> v1:172.21.15.32:6789/0 conn(0x7fce2c070230 legacy=0x7fce2c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.803+0000 7fce2affd640 1 -- 172.21.15.32:0/3836344508 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce2c06dba0 con 0x7fce2c15abb0 2023-12-20T06:00:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.803+0000 7fce2affd640 1 --2- 172.21.15.32:0/3836344508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce2c15abb0 0x7fce2c0caaf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0097f0 tx=0x7fce1c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7d8f729d331f8311 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:20.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.803+0000 7fce29ffb640 1 -- 172.21.15.32:0/3836344508 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c0aa620 con 0x7fce2c15abb0 2023-12-20T06:00:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.803+0000 7fce29ffb640 1 -- 172.21.15.32:0/3836344508 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce1c0aa780 con 0x7fce2c15abb0 2023-12-20T06:00:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.803+0000 7fce29ffb640 1 -- 172.21.15.32:0/3836344508 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce1c0aaaf0 con 0x7fce2c15abb0 2023-12-20T06:00:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 -- 172.21.15.32:0/3836344508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce2c15abb0 msgr2=0x7fce2c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 --2- 172.21.15.32:0/3836344508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce2c15abb0 0x7fce2c0caaf0 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7fce1c0097f0 tx=0x7fce1c01c4c0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 -- 172.21.15.32:0/3836344508 shutdown_connections 2023-12-20T06:00:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 --2- 172.21.15.32:0/3836344508 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce2c15abb0 0x7fce2c0caaf0 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 -- 172.21.15.32:0/3836344508 >> 172.21.15.32:0/3836344508 conn(0x7fce2c06a510 msgr2=0x7fce2c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 -- 172.21.15.32:0/3836344508 shutdown_connections 2023-12-20T06:00:20.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 -- 172.21.15.32:0/3836344508 wait complete. 2023-12-20T06:00:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 Processor -- start 2023-12-20T06:00:20.806 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 -- start start 2023-12-20T06:00:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce2c070230 0x7fce2c14ae40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2c0b0cb0 0x7fce2c134f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c15abb0 0x7fce2c1354b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c14c7e0 con 0x7fce2c15abb0 2023-12-20T06:00:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c1359f0 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce31396640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce2c138170 con 0x7fce2c070230 2023-12-20T06:00:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce2b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c15abb0 0x7fce2c1354b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce2a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2c0b0cb0 0x7fce2c134f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce2affd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce2c070230 0x7fce2c14ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce2a7fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2c0b0cb0 0x7fce2c134f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:37252/0 (socket says 172.21.15.32:37252) 2023-12-20T06:00:20.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.807+0000 7fce2a7fc640 1 -- 172.21.15.32:0/3617235199 learned_addr learned my addr 172.21.15.32:0/3617235199 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:20.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce2a7fc640 1 -- 172.21.15.32:0/3617235199 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce2c070230 msgr2=0x7fce2c14ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce2a7fc640 1 --2- 172.21.15.32:0/3617235199 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce2c070230 0x7fce2c14ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce2a7fc640 1 -- 172.21.15.32:0/3617235199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c15abb0 msgr2=0x7fce2c1354b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce2a7fc640 1 --2- 172.21.15.32:0/3617235199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c15abb0 0x7fce2c1354b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce2a7fc640 1 -- 172.21.15.32:0/3617235199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1c009420 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce2a7fc640 1 --2- 172.21.15.32:0/3617235199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2c0b0cb0 0x7fce2c134f70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c440 tx=0x7fce2000c900 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce137fe640 1 -- 172.21.15.32:0/3617235199 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce200052d0 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce137fe640 1 -- 172.21.15.32:0/3617235199 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce20005450 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce137fe640 1 -- 172.21.15.32:0/3617235199 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce20012870 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce31396640 1 -- 172.21.15.32:0/3617235199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce2c138450 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce31396640 1 -- 172.21.15.32:0/3617235199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce2c138a10 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce31396640 1 -- 172.21.15.32:0/3617235199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce2c04e000 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce137fe640 1 -- 172.21.15.32:0/3617235199 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 109) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fce200129d0 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce137fe640 1 --2- 172.21.15.32:0/3617235199 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fce18065b80 0x7fce18068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:20.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce2affd640 1 -- 172.21.15.32:0/3617235199 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fce18065b80 msgr2=0x7fce18068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:20.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce137fe640 1 -- 172.21.15.32:0/3617235199 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6368+0+0 (secure 0 0 0) 0x7fce20082fc0 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.811+0000 7fce2affd640 1 --2- 172.21.15.32:0/3617235199 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fce18065b80 0x7fce18068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:20.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.819+0000 7fce137fe640 1 -- 172.21.15.32:0/3617235199 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fce200506a0 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.975+0000 7fce31396640 1 -- 172.21.15.32:0/3617235199 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fce2c152730 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.975+0000 7fce137fe640 1 -- 172.21.15.32:0/3617235199 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v177) v1 ==== 104+0+11 (secure 0 0 0) 0x7fce20053cc0 con 0x7fce2c0b0cb0 2023-12-20T06:00:20.976 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 -- 172.21.15.32:0/3617235199 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fce18065b80 msgr2=0x7fce18068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 --2- 172.21.15.32:0/3617235199 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fce18065b80 0x7fce18068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 -- 172.21.15.32:0/3617235199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2c0b0cb0 msgr2=0x7fce2c134f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 --2- 172.21.15.32:0/3617235199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2c0b0cb0 0x7fce2c134f70 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7fce2000c440 tx=0x7fce2000c900 comp rx=0 tx=0).stop 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 -- 172.21.15.32:0/3617235199 shutdown_connections 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 --2- 172.21.15.32:0/3617235199 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fce18065b80 0x7fce18068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 --2- 172.21.15.32:0/3617235199 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce2c15abb0 0x7fce2c1354b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 --2- 172.21.15.32:0/3617235199 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce2c0b0cb0 0x7fce2c134f70 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 --2- 172.21.15.32:0/3617235199 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce2c070230 0x7fce2c14ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 -- 172.21.15.32:0/3617235199 >> 172.21.15.32:0/3617235199 conn(0x7fce2c06a510 msgr2=0x7fce2c0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 -- 172.21.15.32:0/3617235199 shutdown_connections 2023-12-20T06:00:20.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:20.979+0000 7fce31396640 1 -- 172.21.15.32:0/3617235199 wait complete. 2023-12-20T06:00:20.996 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:20.995+0000 7f78bacdc180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:00:20.998 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:00:21.022 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:21.023+0000 7f6d11541180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:00:21.058 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:21.059+0000 7f78bacdc180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:00:21.087 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:21.087+0000 7f6d11541180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:00:21.118 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:21.119+0000 7f78bacdc180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:00:21.147 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:21.147+0000 7f6d11541180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:00:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc45af9640 1 Processor -- start 2023-12-20T06:00:21.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc45af9640 1 -- start start 2023-12-20T06:00:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc45af9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc4015abb0 0x7fdc400caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:21.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc45af9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdc40070230 0x7fdc400706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc45af9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc4015b6a0 con 0x7fdc400b0cb0 2023-12-20T06:00:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc45af9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc4006e160 con 0x7fdc40070230 2023-12-20T06:00:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc45af9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40070be0 con 0x7fdc4015abb0 2023-12-20T06:00:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc3effd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdc400b0cb0 0x7fdc4015bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53524/0 (socket says 172.21.15.32:53524) 2023-12-20T06:00:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc3effd640 1 -- 172.21.15.32:0/640327338 learned_addr learned my addr 172.21.15.32:0/640327338 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc3ffff640 1 --2- 172.21.15.32:0/640327338 >> v2:172.21.15.39:3300/0 conn(0x7fdc40070230 0x7fdc400706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc3f7fe640 1 --2- 172.21.15.32:0/640327338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc4015abb0 0x7fdc400caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:21.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.151+0000 7fdc3e7fc640 1 -- 172.21.15.32:0/640327338 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2098624897 0 0) 0x7fdc4015b6a0 con 0x7fdc400b0cb0 2023-12-20T06:00:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.155+0000 7fdc3e7fc640 1 -- 172.21.15.32:0/640327338 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc28003530 con 0x7fdc400b0cb0 2023-12-20T06:00:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.155+0000 7fdc3ffff640 1 -- 172.21.15.32:0/640327338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc4015abb0 msgr2=0x7fdc400caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.155+0000 7fdc3ffff640 1 --2- 172.21.15.32:0/640327338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc4015abb0 0x7fdc400caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.155+0000 7fdc3ffff640 1 -- 172.21.15.32:0/640327338 >> v1:172.21.15.32:6789/0 conn(0x7fdc400b0cb0 legacy=0x7fdc4015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:21.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.155+0000 7fdc3ffff640 1 -- 172.21.15.32:0/640327338 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc4006d8c0 con 0x7fdc40070230 2023-12-20T06:00:21.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.155+0000 7fdc3ffff640 1 --2- 172.21.15.32:0/640327338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc40070230 0x7fdc400706a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009d90 tx=0x7fdc3801c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f7d83e134c59a2c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:21.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.155+0000 7fdc3e7fc640 1 -- 172.21.15.32:0/640327338 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380a5a10 con 0x7fdc40070230 2023-12-20T06:00:21.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.155+0000 7fdc3e7fc640 1 -- 172.21.15.32:0/640327338 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc380a5b70 con 0x7fdc40070230 2023-12-20T06:00:21.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.155+0000 7fdc3e7fc640 1 -- 172.21.15.32:0/640327338 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc380ad440 con 0x7fdc40070230 2023-12-20T06:00:21.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.155+0000 7fdc45af9640 1 -- 172.21.15.32:0/640327338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc40070230 msgr2=0x7fdc400706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:21.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.155+0000 7fdc45af9640 1 --2- 172.21.15.32:0/640327338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc40070230 0x7fdc400706a0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7fdc38009d90 tx=0x7fdc3801c950 comp rx=0 tx=0).stop 2023-12-20T06:00:21.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 -- 172.21.15.32:0/640327338 shutdown_connections 2023-12-20T06:00:21.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 --2- 172.21.15.32:0/640327338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc40070230 0x7fdc400706a0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 --2- 172.21.15.32:0/640327338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc4015abb0 0x7fdc400caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 -- 172.21.15.32:0/640327338 >> 172.21.15.32:0/640327338 conn(0x7fdc4006a510 msgr2=0x7fdc400ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:21.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 -- 172.21.15.32:0/640327338 shutdown_connections 2023-12-20T06:00:21.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 -- 172.21.15.32:0/640327338 wait complete. 2023-12-20T06:00:21.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 Processor -- start 2023-12-20T06:00:21.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 -- start start 2023-12-20T06:00:21.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc40070230 0x7fdc40150020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:21.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc400b0cb0 0x7fdc400cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:21.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc4015abb0 0x7fdc400cc8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc400cdc10 con 0x7fdc400b0cb0 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc40144020 con 0x7fdc4015abb0 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc45af9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc4014df90 con 0x7fdc40070230 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc400b0cb0 0x7fdc400cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc400b0cb0 0x7fdc400cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60038/0 (socket says 172.21.15.32:60038) 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3effd640 1 -- 172.21.15.32:0/489882693 learned_addr learned my addr 172.21.15.32:0/489882693 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3ffff640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc4015abb0 0x7fdc400cc8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3f7fe640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc40070230 0x7fdc40150020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3ffff640 1 -- 172.21.15.32:0/489882693 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc40070230 msgr2=0x7fdc40150020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3ffff640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc40070230 0x7fdc40150020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3ffff640 1 -- 172.21.15.32:0/489882693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc400b0cb0 msgr2=0x7fdc400cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3ffff640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc400b0cb0 0x7fdc400cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3ffff640 1 -- 172.21.15.32:0/489882693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc38009a70 con 0x7fdc4015abb0 2023-12-20T06:00:21.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3f7fe640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc40070230 0x7fdc40150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.159+0000 7fdc3effd640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc400b0cb0 0x7fdc400cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc3ffff640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc4015abb0 0x7fdc400cc8f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdc380ac000 tx=0x7fdc380196f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc27fff640 1 -- 172.21.15.32:0/489882693 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38004240 con 0x7fdc4015abb0 2023-12-20T06:00:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc27fff640 1 -- 172.21.15.32:0/489882693 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc380043a0 con 0x7fdc4015abb0 2023-12-20T06:00:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc27fff640 1 -- 172.21.15.32:0/489882693 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc38004710 con 0x7fdc4015abb0 2023-12-20T06:00:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc45af9640 1 -- 172.21.15.32:0/489882693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc400ccf90 con 0x7fdc4015abb0 2023-12-20T06:00:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc45af9640 1 -- 172.21.15.32:0/489882693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc400cd550 con 0x7fdc4015abb0 2023-12-20T06:00:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc45af9640 1 -- 172.21.15.32:0/489882693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc400cdd50 con 0x7fdc4015abb0 2023-12-20T06:00:21.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc27fff640 1 -- 172.21.15.32:0/489882693 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 109) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fdc380ad440 con 0x7fdc4015abb0 2023-12-20T06:00:21.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc27fff640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc2c065b80 0x7fdc2c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:21.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc27fff640 1 -- 172.21.15.32:0/489882693 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6368+0+0 (secure 0 0 0) 0x7fdc3811dbe0 con 0x7fdc4015abb0 2023-12-20T06:00:21.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc3f7fe640 1 -- 172.21.15.32:0/489882693 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc2c065b80 msgr2=0x7fdc2c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:21.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.163+0000 7fdc3f7fe640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc2c065b80 0x7fdc2c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:21.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.167+0000 7fdc27fff640 1 -- 172.21.15.32:0/489882693 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdc400cdd50 con 0x7fdc4015abb0 2023-12-20T06:00:21.209 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:21.207+0000 7f78bacdc180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:00:21.242 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:21.243+0000 7f6d11541180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:00:21.268 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:21.267+0000 7f78bacdc180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:00:21.302 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:21.303+0000 7f6d11541180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:00:21.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.327+0000 7fdc45af9640 1 -- 172.21.15.32:0/489882693 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fdc400cdf30 con 0x7fdc4015abb0 2023-12-20T06:00:21.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc27fff640 1 -- 172.21.15.32:0/489882693 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v177) v1 ==== 100+0+11 (secure 0 0 0) 0x7fdc400cdf30 con 0x7fdc4015abb0 2023-12-20T06:00:21.328 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 -- 172.21.15.32:0/489882693 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc2c065b80 msgr2=0x7fdc2c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc2c065b80 0x7fdc2c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 -- 172.21.15.32:0/489882693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc4015abb0 msgr2=0x7fdc400cc8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc4015abb0 0x7fdc400cc8f0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7fdc380ac000 tx=0x7fdc380196f0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 -- 172.21.15.32:0/489882693 shutdown_connections 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc2c065b80 0x7fdc2c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc4015abb0 0x7fdc400cc8f0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc400b0cb0 0x7fdc400cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 --2- 172.21.15.32:0/489882693 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc40070230 0x7fdc40150020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 -- 172.21.15.32:0/489882693 >> 172.21.15.32:0/489882693 conn(0x7fdc4006a510 msgr2=0x7fdc400c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 -- 172.21.15.32:0/489882693 shutdown_connections 2023-12-20T06:00:21.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.331+0000 7fdc45af9640 1 -- 172.21.15.32:0/489882693 wait complete. 2023-12-20T06:00:21.351 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T06:00:21.357 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:21.355+0000 7f78bacdc180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:00:21.391 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:21.391+0000 7f6d11541180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:00:21.416 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:21.415+0000 7f78bacdc180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:00:21.451 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:21.451+0000 7f6d11541180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:00:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.535+0000 7fb1633f0640 1 Processor -- start 2023-12-20T06:00:21.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb1633f0640 1 -- start start 2023-12-20T06:00:21.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb1633f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb15c0d2740 0x7fb15c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb1633f0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb15c15d190 con 0x7fb15c15cdb0 2023-12-20T06:00:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb1633f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb15c0d16a0 con 0x7fb15c0cfab0 2023-12-20T06:00:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb1633f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0cff40 con 0x7fb15c0d2740 2023-12-20T06:00:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb16118c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb15c15cdb0 0x7fb15c0d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53526/0 (socket says 172.21.15.32:53526) 2023-12-20T06:00:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb16118c640 1 -- 172.21.15.32:0/3850190607 learned_addr learned my addr 172.21.15.32:0/3850190607 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:21.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb16098b640 1 --2- 172.21.15.32:0/3850190607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb15c0d2740 0x7fb15c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb153fff640 1 -- 172.21.15.32:0/3850190607 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2780061317 0 0) 0x7fb15c15d190 con 0x7fb15c15cdb0 2023-12-20T06:00:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb153fff640 1 -- 172.21.15.32:0/3850190607 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb148003530 con 0x7fb15c15cdb0 2023-12-20T06:00:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb153fff640 1 -- 172.21.15.32:0/3850190607 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 637864757 0 0) 0x7fb15c0d16a0 con 0x7fb15c0cfab0 2023-12-20T06:00:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb153fff640 1 -- 172.21.15.32:0/3850190607 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb15c15d190 con 0x7fb15c0cfab0 2023-12-20T06:00:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb16098b640 1 -- 172.21.15.32:0/3850190607 >> v1:172.21.15.39:6789/0 conn(0x7fb15c0cfab0 legacy=0x7fb15c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb16098b640 1 -- 172.21.15.32:0/3850190607 >> v1:172.21.15.32:6789/0 conn(0x7fb15c15cdb0 legacy=0x7fb15c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:21.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.539+0000 7fb16098b640 1 -- 172.21.15.32:0/3850190607 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb15c0d1980 con 0x7fb15c0d2740 2023-12-20T06:00:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.543+0000 7fb16098b640 1 --2- 172.21.15.32:0/3850190607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb15c0d2740 0x7fb15c0cf570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0097f0 tx=0x7fb14c01ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c0f79f53fe1f60b5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.543+0000 7fb153fff640 1 -- 172.21.15.32:0/3850190607 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0a89b0 con 0x7fb15c0d2740 2023-12-20T06:00:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.543+0000 7fb153fff640 1 -- 172.21.15.32:0/3850190607 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb14c0a8b10 con 0x7fb15c0d2740 2023-12-20T06:00:21.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.543+0000 7fb153fff640 1 -- 172.21.15.32:0/3850190607 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb14c0a8e80 con 0x7fb15c0d2740 2023-12-20T06:00:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.543+0000 7fb1633f0640 1 -- 172.21.15.32:0/3850190607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb15c0d2740 msgr2=0x7fb15c0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.543+0000 7fb1633f0640 1 --2- 172.21.15.32:0/3850190607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb15c0d2740 0x7fb15c0cf570 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fb14c0097f0 tx=0x7fb14c01ecb0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.543+0000 7fb1633f0640 1 -- 172.21.15.32:0/3850190607 shutdown_connections 2023-12-20T06:00:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.543+0000 7fb1633f0640 1 --2- 172.21.15.32:0/3850190607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb15c0d2740 0x7fb15c0cf570 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.543+0000 7fb1633f0640 1 -- 172.21.15.32:0/3850190607 >> 172.21.15.32:0/3850190607 conn(0x7fb15c065ee0 msgr2=0x7fb15c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.543+0000 7fb1633f0640 1 -- 172.21.15.32:0/3850190607 shutdown_connections 2023-12-20T06:00:21.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.543+0000 7fb1633f0640 1 -- 172.21.15.32:0/3850190607 wait complete. 2023-12-20T06:00:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb1633f0640 1 Processor -- start 2023-12-20T06:00:21.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb1633f0640 1 -- start start 2023-12-20T06:00:21.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb1633f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15c0cfab0 0x7fb15c1511f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:21.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb1633f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb15c0d2740 0x7fb15c156110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:21.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb1633f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb15c15cdb0 0x7fb15c0cbc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:21.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb1633f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0cb870 con 0x7fb15c0cfab0 2023-12-20T06:00:21.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb1633f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0cac80 con 0x7fb15c0d2740 2023-12-20T06:00:21.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb1633f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb15c0caed0 con 0x7fb15c15cdb0 2023-12-20T06:00:21.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb16118c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15c0cfab0 0x7fb15c1511f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb16118c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15c0cfab0 0x7fb15c1511f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60048/0 (socket says 172.21.15.32:60048) 2023-12-20T06:00:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb16118c640 1 -- 172.21.15.32:0/1772293131 learned_addr learned my addr 172.21.15.32:0/1772293131 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb16098b640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb15c0d2740 0x7fb15c156110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb16198d640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb15c15cdb0 0x7fb15c0cbc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb16118c640 1 -- 172.21.15.32:0/1772293131 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb15c15cdb0 msgr2=0x7fb15c0cbc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb16118c640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb15c15cdb0 0x7fb15c0cbc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb16118c640 1 -- 172.21.15.32:0/1772293131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb15c0d2740 msgr2=0x7fb15c156110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb16118c640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb15c0d2740 0x7fb15c156110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb16118c640 1 -- 172.21.15.32:0/1772293131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb14c009420 con 0x7fb15c0cfab0 2023-12-20T06:00:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb16118c640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15c0cfab0 0x7fb15c1511f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb15800c450 tx=0x7fb15800c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb151ffb640 1 -- 172.21.15.32:0/1772293131 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158005cf0 con 0x7fb15c0cfab0 2023-12-20T06:00:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb151ffb640 1 -- 172.21.15.32:0/1772293131 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb158005e50 con 0x7fb15c0cfab0 2023-12-20T06:00:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb1633f0640 1 -- 172.21.15.32:0/1772293131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb15c0cb1b0 con 0x7fb15c0cfab0 2023-12-20T06:00:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb151ffb640 1 -- 172.21.15.32:0/1772293131 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb158007270 con 0x7fb15c0cfab0 2023-12-20T06:00:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.547+0000 7fb1633f0640 1 -- 172.21.15.32:0/1772293131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb15c154c10 con 0x7fb15c0cfab0 2023-12-20T06:00:21.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.551+0000 7fb1633f0640 1 -- 172.21.15.32:0/1772293131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb15c04e000 con 0x7fb15c0cfab0 2023-12-20T06:00:21.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.551+0000 7fb151ffb640 1 -- 172.21.15.32:0/1772293131 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 109) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fb1580073d0 con 0x7fb15c0cfab0 2023-12-20T06:00:21.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.551+0000 7fb151ffb640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb140065b80 0x7fb140068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.551+0000 7fb151ffb640 1 -- 172.21.15.32:0/1772293131 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(177..177 src has 1..177) v4 ==== 6368+0+0 (secure 0 0 0) 0x7fb158082950 con 0x7fb15c0cfab0 2023-12-20T06:00:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.551+0000 7fb16098b640 1 -- 172.21.15.32:0/1772293131 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb140065b80 msgr2=0x7fb140068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:21.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.551+0000 7fb16098b640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb140065b80 0x7fb140068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.555+0000 7fb151ffb640 1 -- 172.21.15.32:0/1772293131 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb158050030 con 0x7fb15c0cfab0 2023-12-20T06:00:21.567 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:21.567+0000 7f78bacdc180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:00:21.606 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:21.607+0000 7f6d11541180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:00:21.629 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:21.627+0000 7f78bacdc180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:00:21.668 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:21.667+0000 7f6d11541180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:00:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.743+0000 7fb1633f0640 1 -- 172.21.15.32:0/1772293131 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fb15c06b8d0 con 0x7fb15c0cfab0 2023-12-20T06:00:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.743+0000 7fb151ffb640 1 -- 172.21.15.32:0/1772293131 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 14 entries v178) v1 ==== 81+0+807 (secure 0 0 0) 0x7fb158017080 con 0x7fb15c0cfab0 2023-12-20T06:00:21.742 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2721040122 2023-12-21T05:55:20.531133+0000 2023-12-20T06:00:21.742 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3587707827 2023-12-21T05:55:20.531133+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2000830842 2023-12-21T05:55:20.531133+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2272871714 2023-12-21T05:55:20.531133+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2161329994 2023-12-21T05:55:20.531133+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/663256086 2023-12-21T05:55:20.531133+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2039923182 2023-12-21T05:44:33.158109+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3732035268 2023-12-21T05:44:33.158109+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3971579268 2023-12-21T05:44:33.158109+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/38009 2023-12-21T05:44:33.158109+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1097027017 2023-12-21T05:44:33.158109+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/38009 2023-12-21T05:44:33.158109+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2617565032 2023-12-21T05:44:33.158109+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2748163648 2023-12-21T05:44:33.158109+0000 2023-12-20T06:00:21.743 INFO:teuthology.orchestra.run.smithi032.stderr:listed 14 entries 2023-12-20T06:00:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 -- 172.21.15.32:0/1772293131 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb140065b80 msgr2=0x7fb140068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb140065b80 0x7fb140068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 -- 172.21.15.32:0/1772293131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15c0cfab0 msgr2=0x7fb15c1511f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15c0cfab0 0x7fb15c1511f0 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7fb15800c450 tx=0x7fb15800c910 comp rx=0 tx=0).stop 2023-12-20T06:00:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 -- 172.21.15.32:0/1772293131 shutdown_connections 2023-12-20T06:00:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb140065b80 0x7fb140068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb15c15cdb0 0x7fb15c0cbc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb15c0d2740 0x7fb15c156110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 --2- 172.21.15.32:0/1772293131 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb15c0cfab0 0x7fb15c1511f0 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 -- 172.21.15.32:0/1772293131 >> 172.21.15.32:0/1772293131 conn(0x7fb15c065ee0 msgr2=0x7fb15c0cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 -- 172.21.15.32:0/1772293131 shutdown_connections 2023-12-20T06:00:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:21.747+0000 7fb1633f0640 1 -- 172.21.15.32:0/1772293131 wait complete. 2023-12-20T06:00:21.778 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T06:00:21.778 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:21.778 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T06:00:21.828 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T06:00:21.829 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:21.829 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T06:00:21.891 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:00:21.891 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:00:21.903 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:00:21.903 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:00:21.903 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:00:21.938 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:00:21.938 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:00:21.939 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:00:21.947 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:21.947+0000 7f78bacdc180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:00:21.951 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:00:21.951 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:00:21.951 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:00:21.962 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:00:21.962 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:00:21.991 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:00:21.991 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:00:21.991 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:00:21.999 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:21.999+0000 7f6d11541180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:00:22.092 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:22.091+0000 7f78bacdc180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:00:22.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6f0a95640 1 Processor -- start 2023-12-20T06:00:22.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6f0a95640 1 -- start start 2023-12-20T06:00:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6f0a95640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff6ec0d2740 0x7ff6ec0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6f0a95640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6ec0cfab0 0x7ff6ec0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6f0a95640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6ec0cfec0 con 0x7ff6ec15cdb0 2023-12-20T06:00:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6f0a95640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec15d190 con 0x7ff6ec0cfab0 2023-12-20T06:00:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6f0a95640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6ec0d16a0 con 0x7ff6ec0d2740 2023-12-20T06:00:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6ead9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6ec0cfab0 0x7ff6ec0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6e9d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff6ec0d2740 0x7ff6ec0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6ead9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6ec0cfab0 0x7ff6ec0b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60056/0 (socket says 172.21.15.32:60056) 2023-12-20T06:00:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6ead9d640 1 -- 172.21.15.32:0/2817870 learned_addr learned my addr 172.21.15.32:0/2817870 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6ead9d640 1 -- 172.21.15.32:0/2817870 >> v2:172.21.15.39:3300/0 conn(0x7ff6ec0d2740 msgr2=0x7ff6ec0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6ead9d640 1 --2- 172.21.15.32:0/2817870 >> v2:172.21.15.39:3300/0 conn(0x7ff6ec0d2740 0x7ff6ec0cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6ead9d640 1 -- 172.21.15.32:0/2817870 >> v1:172.21.15.39:6789/0 conn(0x7ff6ec15cdb0 legacy=0x7ff6ec0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.147+0000 7ff6ead9d640 1 -- 172.21.15.32:0/2817870 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6ec0d1980 con 0x7ff6ec0cfab0 2023-12-20T06:00:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.151+0000 7ff6ead9d640 1 --2- 172.21.15.32:0/2817870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6ec0cfab0 0x7ff6ec0b5c90 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0cd570 tx=0x7ff6e401f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c66325be94fcc08d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6e959a640 1 -- 172.21.15.32:0/2817870 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40af400 con 0x7ff6ec0cfab0 2023-12-20T06:00:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6e959a640 1 -- 172.21.15.32:0/2817870 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e40ab050 con 0x7ff6ec0cfab0 2023-12-20T06:00:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6e959a640 1 -- 172.21.15.32:0/2817870 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e40af780 con 0x7ff6ec0cfab0 2023-12-20T06:00:22.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 -- 172.21.15.32:0/2817870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6ec0cfab0 msgr2=0x7ff6ec0b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 --2- 172.21.15.32:0/2817870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6ec0cfab0 0x7ff6ec0b5c90 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7ff6ec0cd570 tx=0x7ff6e401f1c0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 -- 172.21.15.32:0/2817870 shutdown_connections 2023-12-20T06:00:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 --2- 172.21.15.32:0/2817870 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6ec0cfab0 0x7ff6ec0b5c90 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 --2- 172.21.15.32:0/2817870 >> v2:172.21.15.39:3300/0 conn(0x7ff6ec0d2740 0x7ff6ec0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 -- 172.21.15.32:0/2817870 >> 172.21.15.32:0/2817870 conn(0x7ff6ec065ee0 msgr2=0x7ff6ec06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 -- 172.21.15.32:0/2817870 shutdown_connections 2023-12-20T06:00:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 -- 172.21.15.32:0/2817870 wait complete. 2023-12-20T06:00:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 Processor -- start 2023-12-20T06:00:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 -- start start 2023-12-20T06:00:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ec0cfab0 0x7ff6ec14ab20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6ec0d2740 0x7ff6ec157e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.155+0000 7ff6f0a95640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6ec15cdb0 0x7ff6ec159bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6f0a95640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec14b060 con 0x7ff6ec0d2740 2023-12-20T06:00:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6f0a95640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec156a10 con 0x7ff6ec0cfab0 2023-12-20T06:00:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6f0a95640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6ec0c8920 con 0x7ff6ec15cdb0 2023-12-20T06:00:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6e9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6ec0d2740 0x7ff6ec157e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6e9d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6ec0d2740 0x7ff6ec157e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60070/0 (socket says 172.21.15.32:60070) 2023-12-20T06:00:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6e9d9b640 1 -- 172.21.15.32:0/1747296632 learned_addr learned my addr 172.21.15.32:0/1747296632 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6ea59c640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ec0cfab0 0x7ff6ec14ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6ead9d640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6ec15cdb0 0x7ff6ec159bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6e9d9b640 1 -- 172.21.15.32:0/1747296632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6ec15cdb0 msgr2=0x7ff6ec159bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6e9d9b640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6ec15cdb0 0x7ff6ec159bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6e9d9b640 1 -- 172.21.15.32:0/1747296632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ec0cfab0 msgr2=0x7ff6ec14ab20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6e9d9b640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ec0cfab0 0x7ff6ec14ab20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6e9d9b640 1 -- 172.21.15.32:0/1747296632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6e4009b30 con 0x7ff6ec0d2740 2023-12-20T06:00:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6ead9d640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6ec15cdb0 0x7ff6ec159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6e9d9b640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6ec0d2740 0x7ff6ec157e40 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000e930 tx=0x7ff6e000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6daffd640 1 -- 172.21.15.32:0/1747296632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e0009690 con 0x7ff6ec0d2740 2023-12-20T06:00:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6daffd640 1 -- 172.21.15.32:0/1747296632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6e000ca00 con 0x7ff6ec0d2740 2023-12-20T06:00:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6f0a95640 1 -- 172.21.15.32:0/1747296632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6ec156170 con 0x7ff6ec0d2740 2023-12-20T06:00:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6daffd640 1 -- 172.21.15.32:0/1747296632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6e000cd70 con 0x7ff6ec0d2740 2023-12-20T06:00:22.160 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:22.159+0000 7f6d11541180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:00:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6f0a95640 1 -- 172.21.15.32:0/1747296632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6ec156730 con 0x7ff6ec0d2740 2023-12-20T06:00:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.159+0000 7ff6f0a95640 1 -- 172.21.15.32:0/1747296632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6ec04e000 con 0x7ff6ec0d2740 2023-12-20T06:00:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.163+0000 7ff6daffd640 1 -- 172.21.15.32:0/1747296632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 109) v1 ==== 85342+0+0 (secure 0 0 0) 0x7ff6e0013470 con 0x7ff6ec0d2740 2023-12-20T06:00:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.163+0000 7ff6daffd640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6cc065b80 0x7ff6cc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.163+0000 7ff6daffd640 1 -- 172.21.15.32:0/1747296632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6456+0+0 (secure 0 0 0) 0x7ff6e0083f40 con 0x7ff6ec0d2740 2023-12-20T06:00:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.163+0000 7ff6ea59c640 1 -- 172.21.15.32:0/1747296632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6cc065b80 msgr2=0x7ff6cc068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.163+0000 7ff6ea59c640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6cc065b80 0x7ff6cc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.163+0000 7ff6daffd640 1 -- 172.21.15.32:0/1747296632 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff6e00516e0 con 0x7ff6ec0d2740 2023-12-20T06:00:22.245 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:22.243+0000 7f78bacdc180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:00:22.291 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:22.291+0000 7f6d11541180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:00:22.320 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:22.319+0000 7f78bacdc180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:00:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 -- 172.21.15.32:0/1747296632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff6ec06b8d0 con 0x7ff6ec0d2740 2023-12-20T06:00:22.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6daffd640 1 -- 172.21.15.32:0/1747296632 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4710 (secure 0 0 0) 0x7ff6e0059e60 con 0x7ff6ec0d2740 2023-12-20T06:00:22.357 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:22.357 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1922232083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1922232083},{"type":"v1","addr":"172.21.15.39:6833","nonce":1922232083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9103,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428109066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428109066},{"type":"v1","addr":"172.21.15.39:6835","nonce":428109066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1595154235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1595154235},{"type":"v1","addr":"172.21.15.32:6836","nonce":1595154235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:44:21.009338+0000","modified":"2023-12-20T05:44:24.030393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2806059816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2806059816},{"type":"v1","addr":"172.21.15.32:6837","nonce":2806059816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-20T06:00:22.358 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 79 2023-12-20T06:00:22.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 -- 172.21.15.32:0/1747296632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6cc065b80 msgr2=0x7ff6cc068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:22.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6cc065b80 0x7ff6cc068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 -- 172.21.15.32:0/1747296632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6ec0d2740 msgr2=0x7ff6ec157e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6ec0d2740 0x7ff6ec157e40 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7ff6e000e930 tx=0x7ff6e000edf0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 -- 172.21.15.32:0/1747296632 shutdown_connections 2023-12-20T06:00:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6cc065b80 0x7ff6cc068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6ec15cdb0 0x7ff6ec159bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6ec0d2740 0x7ff6ec157e40 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 --2- 172.21.15.32:0/1747296632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6ec0cfab0 0x7ff6ec14ab20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 -- 172.21.15.32:0/1747296632 >> 172.21.15.32:0/1747296632 conn(0x7ff6ec065ee0 msgr2=0x7ff6ec0b6de0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 -- 172.21.15.32:0/1747296632 shutdown_connections 2023-12-20T06:00:22.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.359+0000 7ff6f0a95640 1 -- 172.21.15.32:0/1747296632 wait complete. 2023-12-20T06:00:22.366 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:22.367+0000 7f6d11541180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:00:22.380 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:00:22.465 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:22.463+0000 7f78bacdc180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:00:22.525 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:22.527+0000 7f6d11541180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:00:22.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b3fa6640 1 Processor -- start 2023-12-20T06:00:22.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b3fa6640 1 -- start start 2023-12-20T06:00:22.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b3fa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac15cdb0 0x7f86ac0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b3fa6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86ac15d190 con 0x7f86ac0d2740 2023-12-20T06:00:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b3fa6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86ac0d16a0 con 0x7f86ac0cfc80 2023-12-20T06:00:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b3fa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac0d1980 con 0x7f86ac15cdb0 2023-12-20T06:00:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b1541640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f86ac0d2740 0x7f86ac0cf570 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53536/0 (socket says 172.21.15.32:53536) 2023-12-20T06:00:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b1541640 1 -- 172.21.15.32:0/1457288685 learned_addr learned my addr 172.21.15.32:0/1457288685 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b1d42640 1 --2- 172.21.15.32:0/1457288685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac15cdb0 0x7f86ac0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b0d40640 1 -- 172.21.15.32:0/1457288685 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1237936223 0 0) 0x7f86ac15d190 con 0x7f86ac0d2740 2023-12-20T06:00:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b0d40640 1 -- 172.21.15.32:0/1457288685 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8698003530 con 0x7f86ac0d2740 2023-12-20T06:00:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b0d40640 1 -- 172.21.15.32:0/1457288685 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1494195089 0 0) 0x7f86ac0d16a0 con 0x7f86ac0cfc80 2023-12-20T06:00:22.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b0d40640 1 -- 172.21.15.32:0/1457288685 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f86ac15d190 con 0x7f86ac0cfc80 2023-12-20T06:00:22.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b1d42640 1 -- 172.21.15.32:0/1457288685 >> v1:172.21.15.39:6789/0 conn(0x7f86ac0cfc80 legacy=0x7f86ac0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b1d42640 1 -- 172.21.15.32:0/1457288685 >> v1:172.21.15.32:6789/0 conn(0x7f86ac0d2740 legacy=0x7f86ac0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.579+0000 7f86b1d42640 1 -- 172.21.15.32:0/1457288685 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86ac0d1c60 con 0x7f86ac15cdb0 2023-12-20T06:00:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.583+0000 7f86b1d42640 1 --2- 172.21.15.32:0/1457288685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac15cdb0 0x7f86ac0d0be0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f86a80097f0 tx=0x7f86a801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e06dd66a947c32a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.583+0000 7f86b0d40640 1 -- 172.21.15.32:0/1457288685 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a80a89b0 con 0x7f86ac15cdb0 2023-12-20T06:00:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.583+0000 7f86b0d40640 1 -- 172.21.15.32:0/1457288685 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a80a8b10 con 0x7f86ac15cdb0 2023-12-20T06:00:22.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.583+0000 7f86b0d40640 1 -- 172.21.15.32:0/1457288685 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a80a8ea0 con 0x7f86ac15cdb0 2023-12-20T06:00:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.583+0000 7f86b3fa6640 1 -- 172.21.15.32:0/1457288685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac15cdb0 msgr2=0x7f86ac0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.583+0000 7f86b3fa6640 1 --2- 172.21.15.32:0/1457288685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac15cdb0 0x7f86ac0d0be0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7f86a80097f0 tx=0x7f86a801ecb0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.583+0000 7f86b3fa6640 1 -- 172.21.15.32:0/1457288685 shutdown_connections 2023-12-20T06:00:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.583+0000 7f86b3fa6640 1 --2- 172.21.15.32:0/1457288685 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac15cdb0 0x7f86ac0d0be0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.583+0000 7f86b3fa6640 1 -- 172.21.15.32:0/1457288685 >> 172.21.15.32:0/1457288685 conn(0x7f86ac065ee0 msgr2=0x7f86ac06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.583+0000 7f86b3fa6640 1 -- 172.21.15.32:0/1457288685 shutdown_connections 2023-12-20T06:00:22.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.583+0000 7f86b3fa6640 1 -- 172.21.15.32:0/1457288685 wait complete. 2023-12-20T06:00:22.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b3fa6640 1 Processor -- start 2023-12-20T06:00:22.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b3fa6640 1 -- start start 2023-12-20T06:00:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b3fa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac0cfc80 0x7f86ac0c88c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b3fa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac0d2740 0x7f86ac14c5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b3fa6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ac15cdb0 0x7f86ac13a590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b3fa6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac15eb60 con 0x7f86ac15cdb0 2023-12-20T06:00:22.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b3fa6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac14b400 con 0x7f86ac0d2740 2023-12-20T06:00:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b3fa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f86ac15edb0 con 0x7f86ac0cfc80 2023-12-20T06:00:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b1d42640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac0cfc80 0x7f86ac0c88c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b1541640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac0d2740 0x7f86ac14c5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b2543640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ac15cdb0 0x7f86ac13a590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b1d42640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac0cfc80 0x7f86ac0c88c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:43096/0 (socket says 172.21.15.32:43096) 2023-12-20T06:00:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b1d42640 1 -- 172.21.15.32:0/2204438047 learned_addr learned my addr 172.21.15.32:0/2204438047 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b1d42640 1 -- 172.21.15.32:0/2204438047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac0d2740 msgr2=0x7f86ac14c5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b1d42640 1 --2- 172.21.15.32:0/2204438047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac0d2740 0x7f86ac14c5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b1d42640 1 -- 172.21.15.32:0/2204438047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ac15cdb0 msgr2=0x7f86ac13a590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b1d42640 1 --2- 172.21.15.32:0/2204438047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ac15cdb0 0x7f86ac13a590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b1d42640 1 -- 172.21.15.32:0/2204438047 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86a8009420 con 0x7f86ac0cfc80 2023-12-20T06:00:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b1d42640 1 --2- 172.21.15.32:0/2204438047 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac0cfc80 0x7f86ac0c88c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f86a80097f0 tx=0x7f86a801ba50 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86a27fc640 1 -- 172.21.15.32:0/2204438047 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a80b0d60 con 0x7f86ac0cfc80 2023-12-20T06:00:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86a27fc640 1 -- 172.21.15.32:0/2204438047 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f86a80bb420 con 0x7f86ac0cfc80 2023-12-20T06:00:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86b3fa6640 1 -- 172.21.15.32:0/2204438047 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86ac15f090 con 0x7f86ac0cfc80 2023-12-20T06:00:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.587+0000 7f86a27fc640 1 -- 172.21.15.32:0/2204438047 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f86a80bb7d0 con 0x7f86ac0cfc80 2023-12-20T06:00:22.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.591+0000 7f86b3fa6640 1 -- 172.21.15.32:0/2204438047 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86ac13aad0 con 0x7f86ac0cfc80 2023-12-20T06:00:22.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.591+0000 7f86b3fa6640 1 -- 172.21.15.32:0/2204438047 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86ac04e000 con 0x7f86ac0cfc80 2023-12-20T06:00:22.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.591+0000 7f86a27fc640 1 -- 172.21.15.32:0/2204438047 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 109) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f86a80b7040 con 0x7f86ac0cfc80 2023-12-20T06:00:22.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.591+0000 7f86a27fc640 1 --2- 172.21.15.32:0/2204438047 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8690065b80 0x7f8690068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.591+0000 7f86a27fc640 1 -- 172.21.15.32:0/2204438047 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(178..178 src has 1..178) v4 ==== 6456+0+0 (secure 0 0 0) 0x7f86a8121580 con 0x7f86ac0cfc80 2023-12-20T06:00:22.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.591+0000 7f86b1541640 1 -- 172.21.15.32:0/2204438047 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8690065b80 msgr2=0x7f8690068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:22.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.591+0000 7f86b1541640 1 --2- 172.21.15.32:0/2204438047 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8690065b80 0x7f8690068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:22.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.595+0000 7f86a27fc640 1 -- 172.21.15.32:0/2204438047 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f86a80eed00 con 0x7f86ac0cfc80 2023-12-20T06:00:22.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.759+0000 7f86b3fa6640 1 -- 172.21.15.32:0/2204438047 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f86ac0d13e0 con 0x7f86ac0cfc80 2023-12-20T06:00:22.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86a27fc640 1 -- 172.21.15.32:0/2204438047 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v179) v1 ==== 74+0+17157 (secure 0 0 0) 0x7f86a80f2320 con 0x7f86ac0cfc80 2023-12-20T06:00:22.762 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:22.762 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":179,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:00:22.712195+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:44:18.671073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-20T05:44:20.010875+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":179,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/2721040122":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/3587707827":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2000830842":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2272871714":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2161329994":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/663256086":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2039923182":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/3732035268":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/3971579268":"2023-12-21T05:44:33.158109+0000","172.21.15.32:6833/38009":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/1097027017":"2023-12-21T05:44:33.158109+0000","172.21.15.32:6832/38009":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/2617565032":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/2748163648":"2023-12-21T05:44:33.158109+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[{"pool":21,"snaps":[{"begin":17,"length":1},{"begin":33,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:00:22.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 -- 172.21.15.32:0/2204438047 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8690065b80 msgr2=0x7f8690068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:22.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 --2- 172.21.15.32:0/2204438047 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8690065b80 0x7f8690068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 -- 172.21.15.32:0/2204438047 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac0cfc80 msgr2=0x7f86ac0c88c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 --2- 172.21.15.32:0/2204438047 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac0cfc80 0x7f86ac0c88c0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7f86a80097f0 tx=0x7f86a801ba50 comp rx=0 tx=0).stop 2023-12-20T06:00:22.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 -- 172.21.15.32:0/2204438047 shutdown_connections 2023-12-20T06:00:22.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 --2- 172.21.15.32:0/2204438047 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8690065b80 0x7f8690068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 --2- 172.21.15.32:0/2204438047 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86ac15cdb0 0x7f86ac13a590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 --2- 172.21.15.32:0/2204438047 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86ac0d2740 0x7f86ac14c5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 --2- 172.21.15.32:0/2204438047 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86ac0cfc80 0x7f86ac0c88c0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 -- 172.21.15.32:0/2204438047 >> 172.21.15.32:0/2204438047 conn(0x7f86ac065ee0 msgr2=0x7f86ac15acd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:22.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 -- 172.21.15.32:0/2204438047 shutdown_connections 2023-12-20T06:00:22.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.763+0000 7f86b3fa6640 1 -- 172.21.15.32:0/2204438047 wait complete. 2023-12-20T06:00:22.786 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 20, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T05:44:18.671073+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '172', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 21, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T05:44:20.010875+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '179', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 33, 'snap_epoch': 179, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:00:22.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:00:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.939+0000 7fa71168c640 1 Processor -- start 2023-12-20T06:00:22.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.939+0000 7fa71168c640 1 -- start start 2023-12-20T06:00:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa71168c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa70c0b0cb0 0x7fa70c0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa71168c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa70c070230 0x7fa70c0706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa71168c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa70c15b6a0 con 0x7fa70c15abb0 2023-12-20T06:00:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa71168c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa70c06e160 con 0x7fa70c0b0cb0 2023-12-20T06:00:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa71168c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c070c00 con 0x7fa70c070230 2023-12-20T06:00:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa70affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa70c15abb0 0x7fa70c15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53540/0 (socket says 172.21.15.32:53540) 2023-12-20T06:00:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa70affd640 1 -- 172.21.15.32:0/1710953976 learned_addr learned my addr 172.21.15.32:0/1710953976 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa703fff640 1 --2- 172.21.15.32:0/1710953976 >> v2:172.21.15.39:3300/0 conn(0x7fa70c0b0cb0 0x7fa70c0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa70b7fe640 1 --2- 172.21.15.32:0/1710953976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa70c070230 0x7fa70c0706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa70a7fc640 1 -- 172.21.15.32:0/1710953976 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3006976732 0 0) 0x7fa70c15b6a0 con 0x7fa70c15abb0 2023-12-20T06:00:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa70a7fc640 1 -- 172.21.15.32:0/1710953976 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6f8003530 con 0x7fa70c15abb0 2023-12-20T06:00:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa703fff640 1 -- 172.21.15.32:0/1710953976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa70c070230 msgr2=0x7fa70c0706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:22.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa703fff640 1 --2- 172.21.15.32:0/1710953976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa70c070230 0x7fa70c0706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa703fff640 1 -- 172.21.15.32:0/1710953976 >> v1:172.21.15.32:6789/0 conn(0x7fa70c15abb0 legacy=0x7fa70c15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa703fff640 1 -- 172.21.15.32:0/1710953976 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa70c06d8c0 con 0x7fa70c0b0cb0 2023-12-20T06:00:22.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.943+0000 7fa70b7fe640 1 --2- 172.21.15.32:0/1710953976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa70c070230 0x7fa70c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:22.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa703fff640 1 --2- 172.21.15.32:0/1710953976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa70c0b0cb0 0x7fa70c0caaf0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40097c0 tx=0x7fa6f401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e3c993b37edd482d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa70a7fc640 1 -- 172.21.15.32:0/1710953976 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40ab680 con 0x7fa70c0b0cb0 2023-12-20T06:00:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa70a7fc640 1 -- 172.21.15.32:0/1710953976 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6f40ab7e0 con 0x7fa70c0b0cb0 2023-12-20T06:00:22.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa70a7fc640 1 -- 172.21.15.32:0/1710953976 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6f40abb70 con 0x7fa70c0b0cb0 2023-12-20T06:00:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa71168c640 1 -- 172.21.15.32:0/1710953976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa70c0b0cb0 msgr2=0x7fa70c0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa71168c640 1 --2- 172.21.15.32:0/1710953976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa70c0b0cb0 0x7fa70c0caaf0 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fa6f40097c0 tx=0x7fa6f401c480 comp rx=0 tx=0).stop 2023-12-20T06:00:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa71168c640 1 -- 172.21.15.32:0/1710953976 shutdown_connections 2023-12-20T06:00:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa71168c640 1 --2- 172.21.15.32:0/1710953976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa70c0b0cb0 0x7fa70c0caaf0 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa71168c640 1 --2- 172.21.15.32:0/1710953976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa70c070230 0x7fa70c0706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa71168c640 1 -- 172.21.15.32:0/1710953976 >> 172.21.15.32:0/1710953976 conn(0x7fa70c06a510 msgr2=0x7fa70c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:22.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa71168c640 1 -- 172.21.15.32:0/1710953976 shutdown_connections 2023-12-20T06:00:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.947+0000 7fa71168c640 1 -- 172.21.15.32:0/1710953976 wait complete. 2023-12-20T06:00:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa71168c640 1 Processor -- start 2023-12-20T06:00:22.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa71168c640 1 -- start start 2023-12-20T06:00:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa71168c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa70c070230 0x7fa70c13f520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa71168c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa70c0b0cb0 0x7fa70c13fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa71168c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70c15abb0 0x7fa70c140170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa71168c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0c4070 con 0x7fa70c15abb0 2023-12-20T06:00:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa71168c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c13fa60 con 0x7fa70c070230 2023-12-20T06:00:22.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa71168c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa70c0c42c0 con 0x7fa70c0b0cb0 2023-12-20T06:00:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70c15abb0 0x7fa70c140170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa70b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70c15abb0 0x7fa70c140170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60090/0 (socket says 172.21.15.32:60090) 2023-12-20T06:00:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa703fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa70c0b0cb0 0x7fa70c13fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa70affd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa70c070230 0x7fa70c13f520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa70b7fe640 1 -- 172.21.15.32:0/1541632639 learned_addr learned my addr 172.21.15.32:0/1541632639 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa70b7fe640 1 -- 172.21.15.32:0/1541632639 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa70c0b0cb0 msgr2=0x7fa70c13fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa70b7fe640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa70c0b0cb0 0x7fa70c13fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa70b7fe640 1 -- 172.21.15.32:0/1541632639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa70c070230 msgr2=0x7fa70c13f520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa70b7fe640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa70c070230 0x7fa70c13f520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa70b7fe640 1 -- 172.21.15.32:0/1541632639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa704009a70 con 0x7fa70c15abb0 2023-12-20T06:00:22.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa70affd640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa70c070230 0x7fa70c13f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa70b7fe640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70c15abb0 0x7fa70c140170 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fa70400efc0 tx=0x7fa704009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa7037fe640 1 -- 172.21.15.32:0/1541632639 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa704009d70 con 0x7fa70c15abb0 2023-12-20T06:00:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa7037fe640 1 -- 172.21.15.32:0/1541632639 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa70400f040 con 0x7fa70c15abb0 2023-12-20T06:00:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa71168c640 1 -- 172.21.15.32:0/1541632639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6f4009420 con 0x7fa70c15abb0 2023-12-20T06:00:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa7037fe640 1 -- 172.21.15.32:0/1541632639 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7040039a0 con 0x7fa70c15abb0 2023-12-20T06:00:22.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa71168c640 1 -- 172.21.15.32:0/1541632639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa70c0c4880 con 0x7fa70c15abb0 2023-12-20T06:00:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.951+0000 7fa71168c640 1 -- 172.21.15.32:0/1541632639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa70c04e000 con 0x7fa70c15abb0 2023-12-20T06:00:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.955+0000 7fa7037fe640 1 -- 172.21.15.32:0/1541632639 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 109) v1 ==== 85342+0+0 (secure 0 0 0) 0x7fa704014480 con 0x7fa70c15abb0 2023-12-20T06:00:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.955+0000 7fa7037fe640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6ec065b80 0x7fa6ec068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:22.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.955+0000 7fa7037fe640 1 -- 172.21.15.32:0/1541632639 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6440+0+0 (secure 0 0 0) 0x7fa704084960 con 0x7fa70c15abb0 2023-12-20T06:00:22.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.955+0000 7fa70affd640 1 -- 172.21.15.32:0/1541632639 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6ec065b80 msgr2=0x7fa6ec068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:22.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.955+0000 7fa70affd640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6ec065b80 0x7fa6ec068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:22.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:22.959+0000 7fa7037fe640 1 -- 172.21.15.32:0/1541632639 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa704051f20 con 0x7fa70c15abb0 2023-12-20T06:00:23.078 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:23.078+0000 7f78bacdc180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:00:23.122 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.123+0000 7f6d11541180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:00:23.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.123+0000 7fa71168c640 1 -- 172.21.15.32:0/1541632639 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fa70c06d480 con 0x7fa70c15abb0 2023-12-20T06:00:23.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.123+0000 7fa7037fe640 1 -- 172.21.15.32:0/1541632639 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v179) v1 ==== 93+0+10 (secure 0 0 0) 0x7fa7040180a0 con 0x7fa70c15abb0 2023-12-20T06:00:23.124 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:00:23.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 -- 172.21.15.32:0/1541632639 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6ec065b80 msgr2=0x7fa6ec068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:23.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6ec065b80 0x7fa6ec068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 -- 172.21.15.32:0/1541632639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70c15abb0 msgr2=0x7fa70c140170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70c15abb0 0x7fa70c140170 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fa70400efc0 tx=0x7fa704009f90 comp rx=0 tx=0).stop 2023-12-20T06:00:23.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 -- 172.21.15.32:0/1541632639 shutdown_connections 2023-12-20T06:00:23.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6ec065b80 0x7fa6ec068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa70c15abb0 0x7fa70c140170 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa70c0b0cb0 0x7fa70c13fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 --2- 172.21.15.32:0/1541632639 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa70c070230 0x7fa70c13f520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 -- 172.21.15.32:0/1541632639 >> 172.21.15.32:0/1541632639 conn(0x7fa70c06a510 msgr2=0x7fa70c0cb150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:23.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 -- 172.21.15.32:0/1541632639 shutdown_connections 2023-12-20T06:00:23.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.127+0000 7fa71168c640 1 -- 172.21.15.32:0/1541632639 wait complete. 2023-12-20T06:00:23.146 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:00:23.166 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:23.166+0000 7f78bacdc180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:00:23.212 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.215+0000 7f6d11541180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:00:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f81bc640 1 Processor -- start 2023-12-20T06:00:23.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f81bc640 1 -- start start 2023-12-20T06:00:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f81bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff1f015ce60 0x7ff1f00d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:23.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f81bc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f00d2650 con 0x7ff1f00d14d0 2023-12-20T06:00:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f81bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1f00d1a40 con 0x7ff1f00d3130 2023-12-20T06:00:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f81bc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1f00d35a0 con 0x7ff1f015ce60 2023-12-20T06:00:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f6759640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff1f00d14d0 0x7ff1f00b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53542/0 (socket says 172.21.15.32:53542) 2023-12-20T06:00:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f6759640 1 -- 172.21.15.32:0/3838713261 learned_addr learned my addr 172.21.15.32:0/3838713261 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:23.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f5f58640 1 --2- 172.21.15.32:0/3838713261 >> v2:172.21.15.39:3300/0 conn(0x7ff1f015ce60 0x7ff1f00d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:23.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f4f56640 1 -- 172.21.15.32:0/3838713261 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459448627 0 0) 0x7ff1f00d2650 con 0x7ff1f00d14d0 2023-12-20T06:00:23.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f4f56640 1 -- 172.21.15.32:0/3838713261 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff1dc003530 con 0x7ff1f00d14d0 2023-12-20T06:00:23.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f5f58640 1 -- 172.21.15.32:0/3838713261 >> v1:172.21.15.39:6789/0 conn(0x7ff1f00d3130 legacy=0x7ff1f00d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f4f56640 1 -- 172.21.15.32:0/3838713261 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3092484800 0 0) 0x7ff1f00d1a40 con 0x7ff1f00d3130 2023-12-20T06:00:23.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f5f58640 1 -- 172.21.15.32:0/3838713261 >> v1:172.21.15.32:6789/0 conn(0x7ff1f00d14d0 legacy=0x7ff1f00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.295+0000 7ff1f5f58640 1 -- 172.21.15.32:0/3838713261 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1f00d1d20 con 0x7ff1f015ce60 2023-12-20T06:00:23.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.299+0000 7ff1f5f58640 1 --2- 172.21.15.32:0/3838713261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015ce60 0x7ff1f00d2bf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0097f0 tx=0x7ff1ec01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=703d0e76b00e8d20 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:23.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.299+0000 7ff1f4f56640 1 -- 172.21.15.32:0/3838713261 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec0a8970 con 0x7ff1f015ce60 2023-12-20T06:00:23.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.299+0000 7ff1f4f56640 1 -- 172.21.15.32:0/3838713261 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1ec0a8ad0 con 0x7ff1f015ce60 2023-12-20T06:00:23.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.299+0000 7ff1f4f56640 1 -- 172.21.15.32:0/3838713261 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1ec0a8e40 con 0x7ff1f015ce60 2023-12-20T06:00:23.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.299+0000 7ff1f81bc640 1 -- 172.21.15.32:0/3838713261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015ce60 msgr2=0x7ff1f00d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.299+0000 7ff1f81bc640 1 --2- 172.21.15.32:0/3838713261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015ce60 0x7ff1f00d2bf0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7ff1ec0097f0 tx=0x7ff1ec01ec70 comp rx=0 tx=0).stop 2023-12-20T06:00:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.299+0000 7ff1f81bc640 1 -- 172.21.15.32:0/3838713261 shutdown_connections 2023-12-20T06:00:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.299+0000 7ff1f81bc640 1 --2- 172.21.15.32:0/3838713261 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015ce60 0x7ff1f00d2bf0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.299+0000 7ff1f81bc640 1 -- 172.21.15.32:0/3838713261 >> 172.21.15.32:0/3838713261 conn(0x7ff1f006b5e0 msgr2=0x7ff1f006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f81bc640 1 -- 172.21.15.32:0/3838713261 shutdown_connections 2023-12-20T06:00:23.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f81bc640 1 -- 172.21.15.32:0/3838713261 wait complete. 2023-12-20T06:00:23.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f81bc640 1 Processor -- start 2023-12-20T06:00:23.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f81bc640 1 -- start start 2023-12-20T06:00:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f81bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff1f00d14d0 0x7ff1f015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f81bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f00d3130 0x7ff1f00ca040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f81bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015ce60 0x7ff1f015f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:23.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f81bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f015fd20 con 0x7ff1f00d3130 2023-12-20T06:00:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f81bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f0155480 con 0x7ff1f015ce60 2023-12-20T06:00:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f81bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff1f015de30 con 0x7ff1f00d14d0 2023-12-20T06:00:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f5757640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f00d3130 0x7ff1f00ca040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f5757640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f00d3130 0x7ff1f00ca040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60104/0 (socket says 172.21.15.32:60104) 2023-12-20T06:00:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f5f58640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff1f00d14d0 0x7ff1f015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f5757640 1 -- 172.21.15.32:0/746055371 learned_addr learned my addr 172.21.15.32:0/746055371 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f6759640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015ce60 0x7ff1f015f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f5757640 1 -- 172.21.15.32:0/746055371 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff1f00d14d0 msgr2=0x7ff1f015d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f5757640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff1f00d14d0 0x7ff1f015d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f5757640 1 -- 172.21.15.32:0/746055371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015ce60 msgr2=0x7ff1f015f7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f5757640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015ce60 0x7ff1f015f7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f5757640 1 -- 172.21.15.32:0/746055371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1ec009420 con 0x7ff1f00d3130 2023-12-20T06:00:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1f5757640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f00d3130 0x7ff1f00ca040 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000c4d0 tx=0x7ff1e000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.303+0000 7ff1e67fc640 1 -- 172.21.15.32:0/746055371 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e0005480 con 0x7ff1f00d3130 2023-12-20T06:00:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.307+0000 7ff1e67fc640 1 -- 172.21.15.32:0/746055371 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1e0003600 con 0x7ff1f00d3130 2023-12-20T06:00:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.307+0000 7ff1f81bc640 1 -- 172.21.15.32:0/746055371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1f0160590 con 0x7ff1f00d3130 2023-12-20T06:00:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.307+0000 7ff1e67fc640 1 -- 172.21.15.32:0/746055371 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1e00039b0 con 0x7ff1f00d3130 2023-12-20T06:00:23.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.307+0000 7ff1f81bc640 1 -- 172.21.15.32:0/746055371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff1f00b8390 con 0x7ff1f00d3130 2023-12-20T06:00:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.307+0000 7ff1f81bc640 1 -- 172.21.15.32:0/746055371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff1f004e000 con 0x7ff1f00d3130 2023-12-20T06:00:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.307+0000 7ff1e67fc640 1 -- 172.21.15.32:0/746055371 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 109) v1 ==== 85342+0+0 (secure 0 0 0) 0x7ff1e0003b10 con 0x7ff1f00d3130 2023-12-20T06:00:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.307+0000 7ff1e67fc640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff1d4065b80 0x7ff1d4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:23.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.307+0000 7ff1e67fc640 1 -- 172.21.15.32:0/746055371 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6440+0+0 (secure 0 0 0) 0x7ff1e0082bc0 con 0x7ff1f00d3130 2023-12-20T06:00:23.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.307+0000 7ff1f5f58640 1 -- 172.21.15.32:0/746055371 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff1d4065b80 msgr2=0x7ff1d4068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:23.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.307+0000 7ff1f5f58640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff1d4065b80 0x7ff1d4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:23.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.311+0000 7ff1e67fc640 1 -- 172.21.15.32:0/746055371 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff1e0050260 con 0x7ff1f00d3130 2023-12-20T06:00:23.324 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:23.322+0000 7f78bacdc180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:00:23.373 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.375+0000 7f6d11541180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:00:23.412 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:23.410+0000 7f78bacdc180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:00:23.460 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.463+0000 7f6d11541180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:00:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.471+0000 7ff1f81bc640 1 -- 172.21.15.32:0/746055371 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1f00d22b0 con 0x7ff1f00d3130 2023-12-20T06:00:23.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.471+0000 7ff1e67fc640 1 -- 172.21.15.32:0/746055371 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v179) v1 ==== 104+0+11 (secure 0 0 0) 0x7ff1e0017020 con 0x7ff1f00d3130 2023-12-20T06:00:23.470 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:00:23.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.471+0000 7ff1f81bc640 1 -- 172.21.15.32:0/746055371 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff1d4065b80 msgr2=0x7ff1d4068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.471+0000 7ff1f81bc640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff1d4065b80 0x7ff1d4068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.471+0000 7ff1f81bc640 1 -- 172.21.15.32:0/746055371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f00d3130 msgr2=0x7ff1f00ca040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.471+0000 7ff1f81bc640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f00d3130 0x7ff1f00ca040 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7ff1e000c4d0 tx=0x7ff1e000c990 comp rx=0 tx=0).stop 2023-12-20T06:00:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.475+0000 7ff1f81bc640 1 -- 172.21.15.32:0/746055371 shutdown_connections 2023-12-20T06:00:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.475+0000 7ff1f81bc640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff1d4065b80 0x7ff1d4068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.475+0000 7ff1f81bc640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff1f015ce60 0x7ff1f015f7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.475+0000 7ff1f81bc640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1f00d3130 0x7ff1f00ca040 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.475+0000 7ff1f81bc640 1 --2- 172.21.15.32:0/746055371 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff1f00d14d0 0x7ff1f015d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.475+0000 7ff1f81bc640 1 -- 172.21.15.32:0/746055371 >> 172.21.15.32:0/746055371 conn(0x7ff1f006b5e0 msgr2=0x7ff1f01e6ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:23.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.475+0000 7ff1f81bc640 1 -- 172.21.15.32:0/746055371 shutdown_connections 2023-12-20T06:00:23.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.475+0000 7ff1f81bc640 1 -- 172.21.15.32:0/746055371 wait complete. 2023-12-20T06:00:23.493 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:00:23.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.643+0000 7f8c0c7b3640 1 Processor -- start 2023-12-20T06:00:23.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.643+0000 7f8c0c7b3640 1 -- start start 2023-12-20T06:00:23.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.643+0000 7f8c0c7b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c0415ce60 0x7f8c040d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:23.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.643+0000 7f8c0c7b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8c040d3130 0x7f8c040d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:23.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.643+0000 7f8c0c7b3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8c040d2650 con 0x7f8c040d1300 2023-12-20T06:00:23.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.643+0000 7f8c0c7b3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8c040d3570 con 0x7f8c040d3130 2023-12-20T06:00:23.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.643+0000 7f8c0c7b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040d1a40 con 0x7f8c0415ce60 2023-12-20T06:00:23.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.643+0000 7f8c0ad50640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8c040d1300 0x7f8c040b46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53544/0 (socket says 172.21.15.32:53544) 2023-12-20T06:00:23.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.643+0000 7f8c09d4e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8c040d3130 0x7f8c040d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:23.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.643+0000 7f8c0ad50640 1 -- 172.21.15.32:0/3604889240 learned_addr learned my addr 172.21.15.32:0/3604889240 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:23.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.643+0000 7f8c0a54f640 1 --2- 172.21.15.32:0/3604889240 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c0415ce60 0x7f8c040d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:23.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.647+0000 7f8c09d4e640 1 -- 172.21.15.32:0/3604889240 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c0415ce60 msgr2=0x7f8c040d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:23.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.647+0000 7f8c09d4e640 1 --2- 172.21.15.32:0/3604889240 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c0415ce60 0x7f8c040d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.647+0000 7f8c09d4e640 1 -- 172.21.15.32:0/3604889240 >> v1:172.21.15.32:6789/0 conn(0x7f8c040d1300 legacy=0x7f8c040b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.647+0000 7f8c09d4e640 1 -- 172.21.15.32:0/3604889240 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8c040d1d20 con 0x7f8c040d3130 2023-12-20T06:00:23.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.647+0000 7f8c09d4e640 1 --2- 172.21.15.32:0/3604889240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c040d3130 0x7f8c040d0dc0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40097c0 tx=0x7f8bf401ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d738f0885990168 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:23.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.647+0000 7f8c0954d640 1 -- 172.21.15.32:0/3604889240 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40a8970 con 0x7f8c040d3130 2023-12-20T06:00:23.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.647+0000 7f8c0954d640 1 -- 172.21.15.32:0/3604889240 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8bf40a8ad0 con 0x7f8c040d3130 2023-12-20T06:00:23.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.647+0000 7f8c0954d640 1 -- 172.21.15.32:0/3604889240 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8bf40a8e60 con 0x7f8c040d3130 2023-12-20T06:00:23.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/3604889240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c040d3130 msgr2=0x7f8c040d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 --2- 172.21.15.32:0/3604889240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c040d3130 0x7f8c040d0dc0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f8bf40097c0 tx=0x7f8bf401ec70 comp rx=0 tx=0).stop 2023-12-20T06:00:23.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/3604889240 shutdown_connections 2023-12-20T06:00:23.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 --2- 172.21.15.32:0/3604889240 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c040d3130 0x7f8c040d0dc0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 --2- 172.21.15.32:0/3604889240 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c0415ce60 0x7f8c040d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/3604889240 >> 172.21.15.32:0/3604889240 conn(0x7f8c0406b5e0 msgr2=0x7f8c0406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:23.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/3604889240 shutdown_connections 2023-12-20T06:00:23.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/3604889240 wait complete. 2023-12-20T06:00:23.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 Processor -- start 2023-12-20T06:00:23.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 -- start start 2023-12-20T06:00:23.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c040d1300 0x7f8c040aa920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:23.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c040d3130 0x7f8c04151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:23.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c0415ce60 0x7f8c040ab6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:23.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040aa500 con 0x7f8c040d1300 2023-12-20T06:00:23.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8c041516a0 con 0x7f8c0415ce60 2023-12-20T06:00:23.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0c7b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8c040a95a0 con 0x7f8c040d3130 2023-12-20T06:00:23.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0a54f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c040d1300 0x7f8c040aa920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:23.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0a54f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c040d1300 0x7f8c040aa920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60112/0 (socket says 172.21.15.32:60112) 2023-12-20T06:00:23.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0a54f640 1 -- 172.21.15.32:0/2812165000 learned_addr learned my addr 172.21.15.32:0/2812165000 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:23.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c09d4e640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c040d3130 0x7f8c04151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:23.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0ad50640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c0415ce60 0x7f8c040ab6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:23.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0a54f640 1 -- 172.21.15.32:0/2812165000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c040d3130 msgr2=0x7f8c04151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0a54f640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c040d3130 0x7f8c04151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0a54f640 1 -- 172.21.15.32:0/2812165000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c0415ce60 msgr2=0x7f8c040ab6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0a54f640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c0415ce60 0x7f8c040ab6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0a54f640 1 -- 172.21.15.32:0/2812165000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8bf4009420 con 0x7f8c040d1300 2023-12-20T06:00:23.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.651+0000 7f8c0ad50640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c0415ce60 0x7f8c040ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:23.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8c0a54f640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c040d1300 0x7f8c040aa920 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000e8a0 tx=0x7f8c0000eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:23.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8bfaffd640 1 -- 172.21.15.32:0/2812165000 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c00009510 con 0x7f8c040d1300 2023-12-20T06:00:23.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8bfaffd640 1 -- 172.21.15.32:0/2812165000 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8c0000c940 con 0x7f8c040d1300 2023-12-20T06:00:23.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/2812165000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8c040a9880 con 0x7f8c040d1300 2023-12-20T06:00:23.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8bfaffd640 1 -- 172.21.15.32:0/2812165000 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8c0000ccb0 con 0x7f8c040d1300 2023-12-20T06:00:23.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/2812165000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8c040a9e40 con 0x7f8c040d1300 2023-12-20T06:00:23.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/2812165000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8c040aa770 con 0x7f8c040d1300 2023-12-20T06:00:23.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8bfaffd640 1 -- 172.21.15.32:0/2812165000 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 109) v1 ==== 85342+0+0 (secure 0 0 0) 0x7f8c00013470 con 0x7f8c040d1300 2023-12-20T06:00:23.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8bfaffd640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8be8065bd0 0x7f8be8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:23.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8c09d4e640 1 -- 172.21.15.32:0/2812165000 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8be8065bd0 msgr2=0x7f8be8068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:23.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8bfaffd640 1 -- 172.21.15.32:0/2812165000 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(179..179 src has 1..179) v4 ==== 6440+0+0 (secure 0 0 0) 0x7f8c00083590 con 0x7f8c040d1300 2023-12-20T06:00:23.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.655+0000 7f8c09d4e640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8be8065bd0 0x7f8be8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:23.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.663+0000 7f8bfaffd640 1 -- 172.21.15.32:0/2812165000 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8c00050cf0 con 0x7f8c040d1300 2023-12-20T06:00:23.747 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:23.746+0000 7f78bacdc180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:00:23.806 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.807+0000 7f6d11541180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:00:23.811 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:23.810+0000 7f78bacdc180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:00:23.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/2812165000 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f8c0404e000 con 0x7f8c040d1300 2023-12-20T06:00:23.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8bfaffd640 1 -- 172.21.15.32:0/2812165000 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v179) v1 ==== 100+0+11 (secure 0 0 0) 0x7f8c00054310 con 0x7f8c040d1300 2023-12-20T06:00:23.826 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:00:23.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/2812165000 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8be8065bd0 msgr2=0x7f8be8068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8c0c7b3640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8be8065bd0 0x7f8be8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/2812165000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c040d1300 msgr2=0x7f8c040aa920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8c0c7b3640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c040d1300 0x7f8c040aa920 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f8c0000e8a0 tx=0x7f8c0000eca0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/2812165000 shutdown_connections 2023-12-20T06:00:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8c0c7b3640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8be8065bd0 0x7f8be8068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8c0c7b3640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8c0415ce60 0x7f8c040ab6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8c0c7b3640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8c040d3130 0x7f8c04151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8c0c7b3640 1 --2- 172.21.15.32:0/2812165000 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8c040d1300 0x7f8c040aa920 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:23.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.827+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/2812165000 >> 172.21.15.32:0/2812165000 conn(0x7f8c0406b5e0 msgr2=0x7f8c040b42b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.831+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/2812165000 shutdown_connections 2023-12-20T06:00:23.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:23.831+0000 7f8c0c7b3640 1 -- 172.21.15.32:0/2812165000 wait complete. 2023-12-20T06:00:23.848 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:00:23.874 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.875+0000 7f6d11541180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:00:23.949 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.951+0000 7f6c9cbbb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:23.950 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.951+0000 7f6c9cbbb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:23.950 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.951+0000 7f6c9cbbb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:23.950 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.951+0000 7f6c9cbbb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:23.950 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.951+0000 7f6c9cbbb640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:23.950 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.951+0000 7f6c99bb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:23.950 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.951+0000 7f6c99bb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:23.950 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.951+0000 7f6c99bb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:23.951 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.951+0000 7f6c99bb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:23.951 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:23.951+0000 7f6c99bb5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:24.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0abad2c640 1 Processor -- start 2023-12-20T06:00:24.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0abad2c640 1 -- start start 2023-12-20T06:00:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0abad2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab415cdb0 0x7f0ab40d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0abad2c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ab40d2740 0x7f0ab40cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0abad2c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0ab40d1120 0x7f0ab40cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0abad2c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab415d190 con 0x7f0ab40d2740 2023-12-20T06:00:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0abad2c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0ab40d16a0 con 0x7f0ab40d1120 2023-12-20T06:00:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0abad2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40d1980 con 0x7f0ab415cdb0 2023-12-20T06:00:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0ab3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ab40d2740 0x7f0ab40cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0ab3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0ab40d2740 0x7f0ab40cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60192/0 (socket says 172.21.15.32:60192) 2023-12-20T06:00:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0ab3fff640 1 -- 172.21.15.32:0/1293353195 learned_addr learned my addr 172.21.15.32:0/1293353195 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:24.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0ab8ac8640 1 --2- 172.21.15.32:0/1293353195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab415cdb0 0x7f0ab40d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0ab92c9640 1 --2- 172.21.15.32:0/1293353195 >> v2:172.21.15.39:3300/0 conn(0x7f0ab40d1120 0x7f0ab40cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0ab3fff640 1 -- 172.21.15.32:0/1293353195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab415cdb0 msgr2=0x7f0ab40d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:24.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0ab3fff640 1 --2- 172.21.15.32:0/1293353195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab415cdb0 0x7f0ab40d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0ab3fff640 1 -- 172.21.15.32:0/1293353195 >> v2:172.21.15.39:3300/0 conn(0x7f0ab40d1120 msgr2=0x7f0ab40cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:24.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0ab3fff640 1 --2- 172.21.15.32:0/1293353195 >> v2:172.21.15.39:3300/0 conn(0x7f0ab40d1120 0x7f0ab40cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.007+0000 7f0ab3fff640 1 -- 172.21.15.32:0/1293353195 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0ab40d1c60 con 0x7f0ab40d2740 2023-12-20T06:00:24.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0ab3fff640 1 --2- 172.21.15.32:0/1293353195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab40d2740 0x7f0ab40cf570 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40d3bc0 tx=0x7f0aa001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7600e3ce10d604d5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:24.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0ab37fe640 1 -- 172.21.15.32:0/1293353195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00a89b0 con 0x7f0ab40d2740 2023-12-20T06:00:24.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0ab37fe640 1 -- 172.21.15.32:0/1293353195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa00a8b10 con 0x7f0ab40d2740 2023-12-20T06:00:24.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0ab37fe640 1 -- 172.21.15.32:0/1293353195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00a8ea0 con 0x7f0ab40d2740 2023-12-20T06:00:24.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0abad2c640 1 -- 172.21.15.32:0/1293353195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab40d2740 msgr2=0x7f0ab40cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0abad2c640 1 --2- 172.21.15.32:0/1293353195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab40d2740 0x7f0ab40cf570 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f0ab40d3bc0 tx=0x7f0aa001ecf0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0abad2c640 1 -- 172.21.15.32:0/1293353195 shutdown_connections 2023-12-20T06:00:24.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0abad2c640 1 --2- 172.21.15.32:0/1293353195 >> v2:172.21.15.39:3300/0 conn(0x7f0ab40d1120 0x7f0ab40cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0abad2c640 1 --2- 172.21.15.32:0/1293353195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab40d2740 0x7f0ab40cf570 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0abad2c640 1 --2- 172.21.15.32:0/1293353195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab415cdb0 0x7f0ab40d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0abad2c640 1 -- 172.21.15.32:0/1293353195 >> 172.21.15.32:0/1293353195 conn(0x7f0ab4065ee0 msgr2=0x7f0ab406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:24.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0abad2c640 1 -- 172.21.15.32:0/1293353195 shutdown_connections 2023-12-20T06:00:24.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0abad2c640 1 -- 172.21.15.32:0/1293353195 wait complete. 2023-12-20T06:00:24.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.011+0000 7f0abad2c640 1 Processor -- start 2023-12-20T06:00:24.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0abad2c640 1 -- start start 2023-12-20T06:00:24.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0abad2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab40d1120 0x7f0ab414e710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0abad2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab40d2740 0x7f0ab40cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0abad2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab415cdb0 0x7f0ab40c7ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0abad2c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40cc680 con 0x7f0ab40d1120 2023-12-20T06:00:24.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0abad2c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40cbbd0 con 0x7f0ab40d2740 2023-12-20T06:00:24.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0abad2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0ab40cbde0 con 0x7f0ab415cdb0 2023-12-20T06:00:24.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab8ac8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab40d1120 0x7f0ab414e710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab8ac8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab40d1120 0x7f0ab414e710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60198/0 (socket says 172.21.15.32:60198) 2023-12-20T06:00:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab8ac8640 1 -- 172.21.15.32:0/613902883 learned_addr learned my addr 172.21.15.32:0/613902883 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab3fff640 1 --2- 172.21.15.32:0/613902883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab40d2740 0x7f0ab40cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab92c9640 1 --2- 172.21.15.32:0/613902883 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab415cdb0 0x7f0ab40c7ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab3fff640 1 -- 172.21.15.32:0/613902883 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab415cdb0 msgr2=0x7f0ab40c7ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab3fff640 1 --2- 172.21.15.32:0/613902883 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab415cdb0 0x7f0ab40c7ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab3fff640 1 -- 172.21.15.32:0/613902883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab40d1120 msgr2=0x7f0ab414e710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab3fff640 1 --2- 172.21.15.32:0/613902883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab40d1120 0x7f0ab414e710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab3fff640 1 -- 172.21.15.32:0/613902883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0aa00094d0 con 0x7f0ab40d2740 2023-12-20T06:00:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab8ac8640 1 --2- 172.21.15.32:0/613902883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab40d1120 0x7f0ab414e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:24.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab3fff640 1 --2- 172.21.15.32:0/613902883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab40d2740 0x7f0ab40cb690 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00af000 tx=0x7f0aa001ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab17fa640 1 -- 172.21.15.32:0/613902883 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00b0da0 con 0x7f0ab40d2740 2023-12-20T06:00:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab17fa640 1 -- 172.21.15.32:0/613902883 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0aa00bb460 con 0x7f0ab40d2740 2023-12-20T06:00:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0abad2c640 1 -- 172.21.15.32:0/613902883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0ab40cc0c0 con 0x7f0ab40d2740 2023-12-20T06:00:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0ab17fa640 1 -- 172.21.15.32:0/613902883 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0aa00bb810 con 0x7f0ab40d2740 2023-12-20T06:00:24.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0abad2c640 1 -- 172.21.15.32:0/613902883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0ab415da90 con 0x7f0ab40d2740 2023-12-20T06:00:24.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.015+0000 7f0abad2c640 1 -- 172.21.15.32:0/613902883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0ab415d8f0 con 0x7f0ab40d2740 2023-12-20T06:00:24.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.019+0000 7f0ab17fa640 1 -- 172.21.15.32:0/613902883 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 110) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f0aa00b83f0 con 0x7f0ab40d2740 2023-12-20T06:00:24.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.019+0000 7f0ab17fa640 1 -- 172.21.15.32:0/613902883 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6740+0+0 (secure 0 0 0) 0x7f0aa01209c0 con 0x7f0ab40d2740 2023-12-20T06:00:24.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.023+0000 7f0ab17fa640 1 -- 172.21.15.32:0/613902883 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0ab415d8f0 con 0x7f0ab40d2740 2023-12-20T06:00:24.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.211+0000 7f0abad2c640 1 -- 172.21.15.32:0/613902883 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0ab415d8f0 con 0x7f0ab40d2740 2023-12-20T06:00:24.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.211+0000 7f0ab17fa640 1 -- 172.21.15.32:0/613902883 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4710 (secure 0 0 0) 0x7f0ab415d8f0 con 0x7f0ab40d2740 2023-12-20T06:00:24.212 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:24.212 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1922232083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1922232083},{"type":"v1","addr":"172.21.15.39:6833","nonce":1922232083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9103,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428109066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428109066},{"type":"v1","addr":"172.21.15.39:6835","nonce":428109066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1595154235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1595154235},{"type":"v1","addr":"172.21.15.32:6836","nonce":1595154235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:44:21.009338+0000","modified":"2023-12-20T05:44:24.030393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2806059816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2806059816},{"type":"v1","addr":"172.21.15.32:6837","nonce":2806059816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-20T06:00:24.212 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 79 2023-12-20T06:00:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.215+0000 7f0abad2c640 1 -- 172.21.15.32:0/613902883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab40d2740 msgr2=0x7f0ab40cb690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.215+0000 7f0abad2c640 1 --2- 172.21.15.32:0/613902883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab40d2740 0x7f0ab40cb690 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f0aa00af000 tx=0x7f0aa001ba90 comp rx=0 tx=0).stop 2023-12-20T06:00:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.215+0000 7f0abad2c640 1 -- 172.21.15.32:0/613902883 shutdown_connections 2023-12-20T06:00:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.215+0000 7f0abad2c640 1 --2- 172.21.15.32:0/613902883 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0ab415cdb0 0x7f0ab40c7ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.215+0000 7f0abad2c640 1 --2- 172.21.15.32:0/613902883 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0ab40d2740 0x7f0ab40cb690 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.215+0000 7f0abad2c640 1 --2- 172.21.15.32:0/613902883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0ab40d1120 0x7f0ab414e710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.215+0000 7f0abad2c640 1 -- 172.21.15.32:0/613902883 >> 172.21.15.32:0/613902883 conn(0x7f0ab4065ee0 msgr2=0x7f0ab40cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.215+0000 7f0abad2c640 1 -- 172.21.15.32:0/613902883 shutdown_connections 2023-12-20T06:00:24.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.215+0000 7f0abad2c640 1 -- 172.21.15.32:0/613902883 wait complete. 2023-12-20T06:00:24.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:00:24.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.423+0000 7fb237ec4640 1 Processor -- start 2023-12-20T06:00:24.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.423+0000 7fb237ec4640 1 -- start start 2023-12-20T06:00:24.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb237ec4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb230164ef0 0x7fb2300c9bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb237ec4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2300ae3f0 con 0x7fb2300ae010 2023-12-20T06:00:24.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb237ec4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb230161800 con 0x7fb23006cf10 2023-12-20T06:00:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb237ec4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb230165380 con 0x7fb230164ef0 2023-12-20T06:00:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb235c60640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb2300ae010 0x7fb230071470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53550/0 (socket says 172.21.15.32:53550) 2023-12-20T06:00:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb235c60640 1 -- 172.21.15.32:0/755276045 learned_addr learned my addr 172.21.15.32:0/755276045 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb236461640 1 --2- 172.21.15.32:0/755276045 >> v2:172.21.15.32:3300/0 conn(0x7fb230164ef0 0x7fb2300c9bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb234c5e640 1 -- 172.21.15.32:0/755276045 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3009929114 0 0) 0x7fb2300ae3f0 con 0x7fb2300ae010 2023-12-20T06:00:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb234c5e640 1 -- 172.21.15.32:0/755276045 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb21c003530 con 0x7fb2300ae010 2023-12-20T06:00:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb234c5e640 1 -- 172.21.15.32:0/755276045 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3807340258 0 0) 0x7fb230161800 con 0x7fb23006cf10 2023-12-20T06:00:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb236461640 1 -- 172.21.15.32:0/755276045 >> v1:172.21.15.39:6789/0 conn(0x7fb23006cf10 legacy=0x7fb2300c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb236461640 1 -- 172.21.15.32:0/755276045 >> v1:172.21.15.32:6789/0 conn(0x7fb2300ae010 legacy=0x7fb230071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.427+0000 7fb236461640 1 -- 172.21.15.32:0/755276045 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb230165660 con 0x7fb230164ef0 2023-12-20T06:00:24.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb236461640 1 --2- 172.21.15.32:0/755276045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb230164ef0 0x7fb2300c9bb0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fb228009d90 tx=0x7fb22801c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3b95512ca1f1ca1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:24.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb234c5e640 1 -- 172.21.15.32:0/755276045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2280a5950 con 0x7fb230164ef0 2023-12-20T06:00:24.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb234c5e640 1 -- 172.21.15.32:0/755276045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2280a5ab0 con 0x7fb230164ef0 2023-12-20T06:00:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb234c5e640 1 -- 172.21.15.32:0/755276045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2280a5e40 con 0x7fb230164ef0 2023-12-20T06:00:24.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 -- 172.21.15.32:0/755276045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb230164ef0 msgr2=0x7fb2300c9bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 --2- 172.21.15.32:0/755276045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb230164ef0 0x7fb2300c9bb0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7fb228009d90 tx=0x7fb22801c890 comp rx=0 tx=0).stop 2023-12-20T06:00:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 -- 172.21.15.32:0/755276045 shutdown_connections 2023-12-20T06:00:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 --2- 172.21.15.32:0/755276045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb230164ef0 0x7fb2300c9bb0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 -- 172.21.15.32:0/755276045 >> 172.21.15.32:0/755276045 conn(0x7fb23006a320 msgr2=0x7fb2300ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 -- 172.21.15.32:0/755276045 shutdown_connections 2023-12-20T06:00:24.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 -- 172.21.15.32:0/755276045 wait complete. 2023-12-20T06:00:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 Processor -- start 2023-12-20T06:00:24.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 -- start start 2023-12-20T06:00:24.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2300ae010 0x7fb230157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2301646f0 0x7fb2300cd770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb23014a820 0x7fb23014aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb230164250 con 0x7fb2300ae010 2023-12-20T06:00:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb230163c90 con 0x7fb23014a820 2023-12-20T06:00:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb237ec4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb230163f70 con 0x7fb2301646f0 2023-12-20T06:00:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.431+0000 7fb235c60640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2300ae010 0x7fb230157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb235c60640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2300ae010 0x7fb230157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60220/0 (socket says 172.21.15.32:60220) 2023-12-20T06:00:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb235c60640 1 -- 172.21.15.32:0/2582507552 learned_addr learned my addr 172.21.15.32:0/2582507552 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb236461640 1 --2- 172.21.15.32:0/2582507552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb23014a820 0x7fb23014aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb23545f640 1 --2- 172.21.15.32:0/2582507552 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2301646f0 0x7fb2300cd770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb235c60640 1 -- 172.21.15.32:0/2582507552 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2301646f0 msgr2=0x7fb2300cd770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb235c60640 1 --2- 172.21.15.32:0/2582507552 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2301646f0 0x7fb2300cd770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb235c60640 1 -- 172.21.15.32:0/2582507552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb23014a820 msgr2=0x7fb23014aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb235c60640 1 --2- 172.21.15.32:0/2582507552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb23014a820 0x7fb23014aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb235c60640 1 -- 172.21.15.32:0/2582507552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb228009a70 con 0x7fb2300ae010 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb23545f640 1 --2- 172.21.15.32:0/2582507552 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2301646f0 0x7fb2300cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb235c60640 1 --2- 172.21.15.32:0/2582507552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2300ae010 0x7fb230157e20 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00c700 tx=0x7fb22c00cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb2267fc640 1 -- 172.21.15.32:0/2582507552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c007270 con 0x7fb2300ae010 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb2267fc640 1 -- 172.21.15.32:0/2582507552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb22c0073d0 con 0x7fb2300ae010 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb237ec4640 1 -- 172.21.15.32:0/2582507552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2300c2330 con 0x7fb2300ae010 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb2267fc640 1 -- 172.21.15.32:0/2582507552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb22c001d50 con 0x7fb2300ae010 2023-12-20T06:00:24.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb237ec4640 1 -- 172.21.15.32:0/2582507552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb23014b1e0 con 0x7fb2300ae010 2023-12-20T06:00:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb237ec4640 1 -- 172.21.15.32:0/2582507552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2300c2850 con 0x7fb2300ae010 2023-12-20T06:00:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.435+0000 7fb2267fc640 1 -- 172.21.15.32:0/2582507552 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 110) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fb22c0057b0 con 0x7fb2300ae010 2023-12-20T06:00:24.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.439+0000 7fb2267fc640 1 -- 172.21.15.32:0/2582507552 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6740+0+0 (secure 0 0 0) 0x7fb22c0825e0 con 0x7fb2300ae010 2023-12-20T06:00:24.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.443+0000 7fb2267fc640 1 -- 172.21.15.32:0/2582507552 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb2300c2850 con 0x7fb2300ae010 2023-12-20T06:00:24.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.599+0000 7fb237ec4640 1 -- 172.21.15.32:0/2582507552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb2300c2850 con 0x7fb2300ae010 2023-12-20T06:00:24.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.599+0000 7fb2267fc640 1 -- 172.21.15.32:0/2582507552 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+17578 (secure 0 0 0) 0x7fb2300c2850 con 0x7fb2300ae010 2023-12-20T06:00:24.600 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:24.600 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":180,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:00:23.879697+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:44:18.671073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-20T05:44:20.010875+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":179,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3370551025":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/2193475721":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/333872760":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/3868230478":"2023-12-21T06:00:23.879665+0000","172.21.15.32:6832/64546":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/987167506":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/2721040122":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/3587707827":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2000830842":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/3914770210":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/2272871714":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2161329994":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/663256086":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2039923182":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/3732035268":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/3971579268":"2023-12-21T05:44:33.158109+0000","172.21.15.32:6833/38009":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/1097027017":"2023-12-21T05:44:33.158109+0000","172.21.15.32:6832/38009":"2023-12-21T05:44:33.158109+0000","172.21.15.32:6833/64546":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/2617565032":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/2748163648":"2023-12-21T05:44:33.158109+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:00:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.603+0000 7fb237ec4640 1 -- 172.21.15.32:0/2582507552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2300ae010 msgr2=0x7fb230157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.603+0000 7fb237ec4640 1 --2- 172.21.15.32:0/2582507552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2300ae010 0x7fb230157e20 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x7fb22c00c700 tx=0x7fb22c00cbc0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.603+0000 7fb237ec4640 1 -- 172.21.15.32:0/2582507552 shutdown_connections 2023-12-20T06:00:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.603+0000 7fb237ec4640 1 --2- 172.21.15.32:0/2582507552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb23014a820 0x7fb23014aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.603+0000 7fb237ec4640 1 --2- 172.21.15.32:0/2582507552 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2301646f0 0x7fb2300cd770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.603+0000 7fb237ec4640 1 --2- 172.21.15.32:0/2582507552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2300ae010 0x7fb230157e20 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.603+0000 7fb237ec4640 1 -- 172.21.15.32:0/2582507552 >> 172.21.15.32:0/2582507552 conn(0x7fb23006a320 msgr2=0x7fb230160d30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.603+0000 7fb237ec4640 1 -- 172.21.15.32:0/2582507552 shutdown_connections 2023-12-20T06:00:24.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.603+0000 7fb237ec4640 1 -- 172.21.15.32:0/2582507552 wait complete. 2023-12-20T06:00:24.621 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T06:00:24.621 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:00:24.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.767+0000 7f12955b3640 1 Processor -- start 2023-12-20T06:00:24.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.767+0000 7f12955b3640 1 -- start start 2023-12-20T06:00:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f12955b3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1290069d20 0x7f12900cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f12955b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f129015de00 0x7f129006a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f12955b3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f129015d970 con 0x7f129015d080 2023-12-20T06:00:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f12955b3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12900ce740 con 0x7f1290069d20 2023-12-20T06:00:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f12955b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f129006a640 con 0x7f129015de00 2023-12-20T06:00:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f1287fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f129015d080 0x7f12900cd9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53560/0 (socket says 172.21.15.32:53560) 2023-12-20T06:00:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f1287fff640 1 -- 172.21.15.32:0/2319215053 learned_addr learned my addr 172.21.15.32:0/2319215053 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f128f7fe640 1 --2- 172.21.15.32:0/2319215053 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f129015de00 0x7f129006a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f128effd640 1 --2- 172.21.15.32:0/2319215053 >> v2:172.21.15.39:3300/0 conn(0x7f1290069d20 0x7f12900cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f128e7fc640 1 -- 172.21.15.32:0/2319215053 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 934694155 0 0) 0x7f129015d970 con 0x7f129015d080 2023-12-20T06:00:24.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f128e7fc640 1 -- 172.21.15.32:0/2319215053 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f127c003530 con 0x7f129015d080 2023-12-20T06:00:24.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f128effd640 1 -- 172.21.15.32:0/2319215053 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f129015de00 msgr2=0x7f129006a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:24.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f128effd640 1 --2- 172.21.15.32:0/2319215053 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f129015de00 0x7f129006a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f128effd640 1 -- 172.21.15.32:0/2319215053 >> v1:172.21.15.32:6789/0 conn(0x7f129015d080 legacy=0x7f12900cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f128effd640 1 -- 172.21.15.32:0/2319215053 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12900ce180 con 0x7f1290069d20 2023-12-20T06:00:24.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.771+0000 7f128f7fe640 1 --2- 172.21.15.32:0/2319215053 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f129015de00 0x7f129006a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:24.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f128effd640 1 --2- 172.21.15.32:0/2319215053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1290069d20 0x7f12900cbbb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f12800097c0 tx=0x7f128001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed84249f46f3b8ba server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:24.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f128e7fc640 1 -- 172.21.15.32:0/2319215053 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12800ab650 con 0x7f1290069d20 2023-12-20T06:00:24.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f128e7fc640 1 -- 172.21.15.32:0/2319215053 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12800ab7b0 con 0x7f1290069d20 2023-12-20T06:00:24.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f128e7fc640 1 -- 172.21.15.32:0/2319215053 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12800abb40 con 0x7f1290069d20 2023-12-20T06:00:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f12955b3640 1 -- 172.21.15.32:0/2319215053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1290069d20 msgr2=0x7f12900cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f12955b3640 1 --2- 172.21.15.32:0/2319215053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1290069d20 0x7f12900cbbb0 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f12800097c0 tx=0x7f128001c450 comp rx=0 tx=0).stop 2023-12-20T06:00:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f12955b3640 1 -- 172.21.15.32:0/2319215053 shutdown_connections 2023-12-20T06:00:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f12955b3640 1 --2- 172.21.15.32:0/2319215053 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f129015de00 0x7f129006a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f12955b3640 1 --2- 172.21.15.32:0/2319215053 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1290069d20 0x7f12900cbbb0 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f12955b3640 1 -- 172.21.15.32:0/2319215053 >> 172.21.15.32:0/2319215053 conn(0x7f12900cc310 msgr2=0x7f12900cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f12955b3640 1 -- 172.21.15.32:0/2319215053 shutdown_connections 2023-12-20T06:00:24.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f12955b3640 1 -- 172.21.15.32:0/2319215053 wait complete. 2023-12-20T06:00:24.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f12955b3640 1 Processor -- start 2023-12-20T06:00:24.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.775+0000 7f12955b3640 1 -- start start 2023-12-20T06:00:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12955b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1290069d20 0x7f129014bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12955b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129015d080 0x7f12900b51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12955b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f129015de00 0x7f129014a1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12955b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12900ae820 con 0x7f129015de00 2023-12-20T06:00:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12955b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12900ada20 con 0x7f129015d080 2023-12-20T06:00:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12955b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12900adca0 con 0x7f1290069d20 2023-12-20T06:00:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f128f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f129015de00 0x7f129014a1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f128f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f129015de00 0x7f129014a1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60222/0 (socket says 172.21.15.32:60222) 2023-12-20T06:00:24.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f128f7fe640 1 -- 172.21.15.32:0/59176561 learned_addr learned my addr 172.21.15.32:0/59176561 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f1287fff640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129015d080 0x7f12900b51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f128effd640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1290069d20 0x7f129014bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f128f7fe640 1 -- 172.21.15.32:0/59176561 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1290069d20 msgr2=0x7f129014bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f128f7fe640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1290069d20 0x7f129014bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f128f7fe640 1 -- 172.21.15.32:0/59176561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129015d080 msgr2=0x7f12900b51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f128f7fe640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129015d080 0x7f12900b51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f128f7fe640 1 -- 172.21.15.32:0/59176561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1288009a70 con 0x7f129015de00 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f128effd640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1290069d20 0x7f129014bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f128f7fe640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f129015de00 0x7f129014a1a0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f128800efc0 tx=0x7f128800c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12877fe640 1 -- 172.21.15.32:0/59176561 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1288009cd0 con 0x7f129015de00 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12877fe640 1 -- 172.21.15.32:0/59176561 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f128800f040 con 0x7f129015de00 2023-12-20T06:00:24.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12877fe640 1 -- 172.21.15.32:0/59176561 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12880039a0 con 0x7f129015de00 2023-12-20T06:00:24.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12955b3640 1 -- 172.21.15.32:0/59176561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1280009420 con 0x7f129015de00 2023-12-20T06:00:24.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12955b3640 1 -- 172.21.15.32:0/59176561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f129015e390 con 0x7f129015de00 2023-12-20T06:00:24.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12955b3640 1 -- 172.21.15.32:0/59176561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f129014a860 con 0x7f129015de00 2023-12-20T06:00:24.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.779+0000 7f12877fe640 1 -- 172.21.15.32:0/59176561 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 110) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f1288014480 con 0x7f129015de00 2023-12-20T06:00:24.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.783+0000 7f12877fe640 1 -- 172.21.15.32:0/59176561 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6740+0+0 (secure 0 0 0) 0x7f1288083c90 con 0x7f129015de00 2023-12-20T06:00:24.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.787+0000 7f12877fe640 1 -- 172.21.15.32:0/59176561 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f12880515a0 con 0x7f129015de00 2023-12-20T06:00:24.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.891+0000 7f12877fe640 1 -- 172.21.15.32:0/59176561 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 111) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f1288051050 con 0x7f129015de00 2023-12-20T06:00:24.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.891+0000 7f12877fe640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f127001c9b0 0x7f127006d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:24.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.891+0000 7f128effd640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f127001c9b0 0x7f127006d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:24.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.895+0000 7f128effd640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f127001c9b0 0x7f127006d490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f12800af000 tx=0x7f12800a7000 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:24.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.983+0000 7f12955b3640 1 -- 172.21.15.32:0/59176561 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f129004e000 con 0x7f129015de00 2023-12-20T06:00:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12877fe640 1 -- 172.21.15.32:0/59176561 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4710 (secure 0 0 0) 0x7f128807ed30 con 0x7f129015de00 2023-12-20T06:00:24.985 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:24.985 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1922232083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1922232083},{"type":"v1","addr":"172.21.15.39:6833","nonce":1922232083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9103,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428109066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428109066},{"type":"v1","addr":"172.21.15.39:6835","nonce":428109066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1595154235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1595154235},{"type":"v1","addr":"172.21.15.32:6836","nonce":1595154235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:44:21.009338+0000","modified":"2023-12-20T05:44:24.030393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2806059816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2806059816},{"type":"v1","addr":"172.21.15.32:6837","nonce":2806059816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-20T06:00:24.985 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 79 2023-12-20T06:00:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 -- 172.21.15.32:0/59176561 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f127001c9b0 msgr2=0x7f127006d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f127001c9b0 0x7f127006d490 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f12800af000 tx=0x7f12800a7000 comp rx=0 tx=0).stop 2023-12-20T06:00:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 -- 172.21.15.32:0/59176561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f129015de00 msgr2=0x7f129014a1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f129015de00 0x7f129014a1a0 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f128800efc0 tx=0x7f128800c490 comp rx=0 tx=0).stop 2023-12-20T06:00:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 -- 172.21.15.32:0/59176561 shutdown_connections 2023-12-20T06:00:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f127001c9b0 0x7f127006d490 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f129015de00 0x7f129014a1a0 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f129015d080 0x7f12900b51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 --2- 172.21.15.32:0/59176561 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1290069d20 0x7f129014bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 -- 172.21.15.32:0/59176561 >> 172.21.15.32:0/59176561 conn(0x7f12900cc310 msgr2=0x7f129015f7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 -- 172.21.15.32:0/59176561 shutdown_connections 2023-12-20T06:00:24.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:24.987+0000 7f12955b3640 1 -- 172.21.15.32:0/59176561 wait complete. 2023-12-20T06:00:25.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:00:25.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.151+0000 7f483fd60640 1 Processor -- start 2023-12-20T06:00:25.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.151+0000 7f483fd60640 1 -- start start 2023-12-20T06:00:25.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.151+0000 7f483fd60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f48380ae010 0x7f4838071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.151+0000 7f483fd60640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4838164ef0 0x7f48380719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.151+0000 7f483fd60640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f48380ae3f0 con 0x7f483806cf10 2023-12-20T06:00:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.151+0000 7f483fd60640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f48380ca4b0 con 0x7f4838164ef0 2023-12-20T06:00:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.151+0000 7f483fd60640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4838161800 con 0x7f48380ae010 2023-12-20T06:00:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.151+0000 7f483d2fb640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f483806cf10 0x7f48380701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53562/0 (socket says 172.21.15.32:53562) 2023-12-20T06:00:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.151+0000 7f483d2fb640 1 -- 172.21.15.32:0/2203078933 learned_addr learned my addr 172.21.15.32:0/2203078933 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483dafc640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f48380ae010 0x7f4838071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483e2fd640 1 --2- 172.21.15.32:0/2203078933 >> v2:172.21.15.39:3300/0 conn(0x7f4838164ef0 0x7f48380719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2110357097 0 0) 0x7f48380ae3f0 con 0x7f483806cf10 2023-12-20T06:00:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4824003530 con 0x7f483806cf10 2023-12-20T06:00:25.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 627147763 0 0) 0x7f4824003530 con 0x7f483806cf10 2023-12-20T06:00:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f48380ae010 msgr2=0x7f4838071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f48380ae010 0x7f4838071470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 >> v2:172.21.15.39:3300/0 conn(0x7f4838164ef0 msgr2=0x7f48380719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 --2- 172.21.15.32:0/2203078933 >> v2:172.21.15.39:3300/0 conn(0x7f4838164ef0 0x7f48380719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4838165380 con 0x7f483806cf10 2023-12-20T06:00:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f4828001680 con 0x7f483806cf10 2023-12-20T06:00:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483e2fd640 1 --2- 172.21.15.32:0/2203078933 >> v2:172.21.15.39:3300/0 conn(0x7f4838164ef0 0x7f48380719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 >> v1:172.21.15.32:6789/0 conn(0x7f483806cf10 legacy=0x7f48380701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4824003980 0x7f4824003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4824004370 0x7f4824004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 0x7f482400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48380ca4b0 con 0x7f4824004f60 2023-12-20T06:00:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4838161800 con 0x7f4824004370 2023-12-20T06:00:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f482400bf00 con 0x7f4824003980 2023-12-20T06:00:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483dafc640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f48380ae010 0x7f4838071470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483e2fd640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 0x7f482400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483d2fb640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4824004370 0x7f4824004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483dafc640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4824003980 0x7f4824003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483e2fd640 1 -- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4824003980 msgr2=0x7f4824003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483e2fd640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4824003980 0x7f4824003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483e2fd640 1 -- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4824004370 msgr2=0x7f4824004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483e2fd640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4824004370 0x7f4824004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483e2fd640 1 -- 172.21.15.32:0/2203078933 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4824003530 con 0x7f4824004f60 2023-12-20T06:00:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.155+0000 7f483dafc640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4824003980 0x7f4824003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483e2fd640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 0x7f482400b990 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f483000e350 tx=0x7f483001ce00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf3fd7cb13e4a6c1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300a5e70 con 0x7f4824004f60 2023-12-20T06:00:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483000f040 con 0x7f4824004f60 2023-12-20T06:00:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483cafa640 1 -- 172.21.15.32:0/2203078933 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f48300ad640 con 0x7f4824004f60 2023-12-20T06:00:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483fd60640 1 -- 172.21.15.32:0/2203078933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 msgr2=0x7f482400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483fd60640 1 --2- 172.21.15.32:0/2203078933 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 0x7f482400b990 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f483000e350 tx=0x7f483001ce00 comp rx=0 tx=0).stop 2023-12-20T06:00:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483e2fd640 1 -- 172.21.15.32:0/2203078933 reap_dead start 2023-12-20T06:00:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483fd60640 1 -- 172.21.15.32:0/2203078933 shutdown_connections 2023-12-20T06:00:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483fd60640 1 -- 172.21.15.32:0/2203078933 >> 172.21.15.32:0/2203078933 conn(0x7f483806a320 msgr2=0x7f48380ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483fd60640 1 -- 172.21.15.32:0/2203078933 shutdown_connections 2023-12-20T06:00:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483fd60640 1 -- 172.21.15.32:0/2203078933 wait complete. 2023-12-20T06:00:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483fd60640 1 Processor -- start 2023-12-20T06:00:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.159+0000 7f483fd60640 1 -- start start 2023-12-20T06:00:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483fd60640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 0x7f48380c8260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483fd60640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48380b4080 0x7f48380703a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483fd60640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f48380708e0 0x7f483814e890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483fd60640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f48380c87a0 con 0x7f4824004f60 2023-12-20T06:00:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483fd60640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4838070d60 con 0x7f48380b4080 2023-12-20T06:00:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483fd60640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f483814f600 con 0x7f48380708e0 2023-12-20T06:00:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483dafc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 0x7f48380c8260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483dafc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 0x7f48380c8260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60240/0 (socket says 172.21.15.32:60240) 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483dafc640 1 -- 172.21.15.32:0/1409769735 learned_addr learned my addr 172.21.15.32:0/1409769735 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483d2fb640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48380b4080 0x7f48380703a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483e2fd640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f48380708e0 0x7f483814e890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483dafc640 1 -- 172.21.15.32:0/1409769735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f48380708e0 msgr2=0x7f483814e890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483dafc640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f48380708e0 0x7f483814e890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483dafc640 1 -- 172.21.15.32:0/1409769735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48380b4080 msgr2=0x7f48380703a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483dafc640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48380b4080 0x7f48380703a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483dafc640 1 -- 172.21.15.32:0/1409769735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f48380b32c0 con 0x7f4824004f60 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483dafc640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 0x7f48380c8260 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f4834011660 tx=0x7f4834011b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f482e7fc640 1 -- 172.21.15.32:0/1409769735 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483400f9c0 con 0x7f4824004f60 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f482e7fc640 1 -- 172.21.15.32:0/1409769735 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f483400fb20 con 0x7f4824004f60 2023-12-20T06:00:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f482e7fc640 1 -- 172.21.15.32:0/1409769735 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f483400fe90 con 0x7f4824004f60 2023-12-20T06:00:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483fd60640 1 -- 172.21.15.32:0/1409769735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f48380b35a0 con 0x7f4824004f60 2023-12-20T06:00:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483fd60640 1 -- 172.21.15.32:0/1409769735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f48380b3b60 con 0x7f4824004f60 2023-12-20T06:00:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f483fd60640 1 -- 172.21.15.32:0/1409769735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f483804e000 con 0x7f4824004f60 2023-12-20T06:00:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.163+0000 7f482e7fc640 1 -- 172.21.15.32:0/1409769735 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 111) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f4834015460 con 0x7f4824004f60 2023-12-20T06:00:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.167+0000 7f482e7fc640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f481c0658b0 0x7f481c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.167+0000 7f482e7fc640 1 -- 172.21.15.32:0/1409769735 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6740+0+0 (secure 0 0 0) 0x7f4834083bf0 con 0x7f4824004f60 2023-12-20T06:00:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.167+0000 7f483d2fb640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f481c0658b0 0x7f481c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.167+0000 7f483d2fb640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f481c0658b0 0x7f481c067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4828001060 tx=0x7f4828009040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.171+0000 7f482e7fc640 1 -- 172.21.15.32:0/1409769735 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4834051410 con 0x7f4824004f60 2023-12-20T06:00:25.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.363+0000 7f483fd60640 1 -- 172.21.15.32:0/1409769735 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4838163f20 con 0x7f4824004f60 2023-12-20T06:00:25.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f482e7fc640 1 -- 172.21.15.32:0/1409769735 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 79 v79) v1 ==== 94+0+4710 (secure 0 0 0) 0x7f4834054a30 con 0x7f4824004f60 2023-12-20T06:00:25.365 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:25.365 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":79,"default_fscid":10,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1922232083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1922232083},{"type":"v1","addr":"172.21.15.39:6833","nonce":1922232083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9103,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428109066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428109066},{"type":"v1","addr":"172.21.15.39:6835","nonce":428109066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1595154235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1595154235},{"type":"v1","addr":"172.21.15.32:6836","nonce":1595154235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74}],"filesystems":[{"mdsmap":{"epoch":79,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T05:44:21.009338+0000","modified":"2023-12-20T05:44:24.030393+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":9420},"failed":[],"damaged":[],"stopped":[],"info":{"gid_9420":{"gid":9420,"name":"c","rank":0,"incarnation":75,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2806059816","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":2806059816},{"type":"v1","addr":"172.21.15.32:6837","nonce":2806059816}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[21],"metadata_pool":20,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":10}]} 2023-12-20T06:00:25.365 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 79 2023-12-20T06:00:25.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 -- 172.21.15.32:0/1409769735 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f481c0658b0 msgr2=0x7f481c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f481c0658b0 0x7f481c067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f4828001060 tx=0x7f4828009040 comp rx=0 tx=0).stop 2023-12-20T06:00:25.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 -- 172.21.15.32:0/1409769735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 msgr2=0x7f48380c8260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 0x7f48380c8260 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f4834011660 tx=0x7f4834011b20 comp rx=0 tx=0).stop 2023-12-20T06:00:25.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 -- 172.21.15.32:0/1409769735 shutdown_connections 2023-12-20T06:00:25.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f481c0658b0 0x7f481c067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f48380708e0 0x7f483814e890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f48380b4080 0x7f48380703a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 --2- 172.21.15.32:0/1409769735 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4824004f60 0x7f48380c8260 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 -- 172.21.15.32:0/1409769735 >> 172.21.15.32:0/1409769735 conn(0x7f483806a320 msgr2=0x7f48380cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:25.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 -- 172.21.15.32:0/1409769735 shutdown_connections 2023-12-20T06:00:25.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.367+0000 7f483fd60640 1 -- 172.21.15.32:0/1409769735 wait complete. 2023-12-20T06:00:25.385 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:00:25.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f9366b8f640 1 Processor -- start 2023-12-20T06:00:25.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f9366b8f640 1 -- start start 2023-12-20T06:00:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f9366b8f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f93600aece0 0x7f93600b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f9366b8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93600b0b30 0x7f93600701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f9366b8f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f93600c98c0 con 0x7f93601599e0 2023-12-20T06:00:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f9366b8f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93600c9ba0 con 0x7f93600aece0 2023-12-20T06:00:25.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f9366b8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93600c9e80 con 0x7f93600b0b30 2023-12-20T06:00:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f936512c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93600b0b30 0x7f93600701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f936492b640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f93601599e0 0x7f93600c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:38310/0 (socket says 172.21.15.32:38310) 2023-12-20T06:00:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f936492b640 1 -- 172.21.15.32:0/889074834 learned_addr learned my addr 172.21.15.32:0/889074834 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f935ffff640 1 --2- 172.21.15.32:0/889074834 >> v2:172.21.15.39:3300/0 conn(0x7f93600aece0 0x7f93600b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f935f7fe640 1 -- 172.21.15.32:0/889074834 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3841957579 0 0) 0x7f93600c98c0 con 0x7f93601599e0 2023-12-20T06:00:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f935f7fe640 1 -- 172.21.15.32:0/889074834 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9350003530 con 0x7f93601599e0 2023-12-20T06:00:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f936512c640 1 -- 172.21.15.32:0/889074834 >> v2:172.21.15.39:3300/0 conn(0x7f93600aece0 msgr2=0x7f93600b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f936512c640 1 --2- 172.21.15.32:0/889074834 >> v2:172.21.15.39:3300/0 conn(0x7f93600aece0 0x7f93600b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f936512c640 1 -- 172.21.15.32:0/889074834 >> v1:172.21.15.39:6789/0 conn(0x7f93601599e0 legacy=0x7f93600c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.531+0000 7f936512c640 1 -- 172.21.15.32:0/889074834 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93600ca160 con 0x7f93600b0b30 2023-12-20T06:00:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f936512c640 1 --2- 172.21.15.32:0/889074834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93600b0b30 0x7f93600701c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9358009d90 tx=0x7f935801c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=571165939203370b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:25.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f935f7fe640 1 -- 172.21.15.32:0/889074834 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580a5920 con 0x7f93600b0b30 2023-12-20T06:00:25.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f935f7fe640 1 -- 172.21.15.32:0/889074834 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93580a5a80 con 0x7f93600b0b30 2023-12-20T06:00:25.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f935f7fe640 1 -- 172.21.15.32:0/889074834 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580a5df0 con 0x7f93600b0b30 2023-12-20T06:00:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f9366b8f640 1 -- 172.21.15.32:0/889074834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93600b0b30 msgr2=0x7f93600701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f9366b8f640 1 --2- 172.21.15.32:0/889074834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93600b0b30 0x7f93600701c0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f9358009d90 tx=0x7f935801c950 comp rx=0 tx=0).stop 2023-12-20T06:00:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f9366b8f640 1 -- 172.21.15.32:0/889074834 shutdown_connections 2023-12-20T06:00:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f9366b8f640 1 --2- 172.21.15.32:0/889074834 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93600b0b30 0x7f93600701c0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f9366b8f640 1 --2- 172.21.15.32:0/889074834 >> v2:172.21.15.39:3300/0 conn(0x7f93600aece0 0x7f93600b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f9366b8f640 1 -- 172.21.15.32:0/889074834 >> 172.21.15.32:0/889074834 conn(0x7f936006a320 msgr2=0x7f93600ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f9366b8f640 1 -- 172.21.15.32:0/889074834 shutdown_connections 2023-12-20T06:00:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.535+0000 7f9366b8f640 1 -- 172.21.15.32:0/889074834 wait complete. 2023-12-20T06:00:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f9366b8f640 1 Processor -- start 2023-12-20T06:00:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f9366b8f640 1 -- start start 2023-12-20T06:00:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f9366b8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93600aece0 0x7f93601f5a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f9366b8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93600b0b30 0x7f93600acd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f9366b8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93601599e0 0x7f93600ac070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f9366b8f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9360158120 con 0x7f93601599e0 2023-12-20T06:00:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f9366b8f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93601f5f50 con 0x7f93600aece0 2023-12-20T06:00:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f9366b8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93601f8a90 con 0x7f93600b0b30 2023-12-20T06:00:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f936512c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93601599e0 0x7f93600ac070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f936512c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93601599e0 0x7f93600ac070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60256/0 (socket says 172.21.15.32:60256) 2023-12-20T06:00:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f936512c640 1 -- 172.21.15.32:0/3435153036 learned_addr learned my addr 172.21.15.32:0/3435153036 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f936492b640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93600aece0 0x7f93601f5a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f935ffff640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93600b0b30 0x7f93600acd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f936512c640 1 -- 172.21.15.32:0/3435153036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93600b0b30 msgr2=0x7f93600acd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f936512c640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93600b0b30 0x7f93600acd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f936512c640 1 -- 172.21.15.32:0/3435153036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93600aece0 msgr2=0x7f93601f5a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f936512c640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93600aece0 0x7f93601f5a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f936512c640 1 -- 172.21.15.32:0/3435153036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9358009a70 con 0x7f93601599e0 2023-12-20T06:00:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f936512c640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93601599e0 0x7f93600ac070 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f935801b000 tx=0x7f93580196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f935d7fa640 1 -- 172.21.15.32:0/3435153036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93580a5920 con 0x7f93601599e0 2023-12-20T06:00:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f935d7fa640 1 -- 172.21.15.32:0/3435153036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9358003db0 con 0x7f93601599e0 2023-12-20T06:00:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f935d7fa640 1 -- 172.21.15.32:0/3435153036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9358004240 con 0x7f93601599e0 2023-12-20T06:00:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f9366b8f640 1 -- 172.21.15.32:0/3435153036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93601f8d70 con 0x7f93601599e0 2023-12-20T06:00:25.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.539+0000 7f9366b8f640 1 -- 172.21.15.32:0/3435153036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93601f9330 con 0x7f93601599e0 2023-12-20T06:00:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.543+0000 7f9366b8f640 1 -- 172.21.15.32:0/3435153036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93601f9550 con 0x7f93601599e0 2023-12-20T06:00:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.543+0000 7f935d7fa640 1 -- 172.21.15.32:0/3435153036 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 111) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f93580b4070 con 0x7f93601599e0 2023-12-20T06:00:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.543+0000 7f935d7fa640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93440658b0 0x7f9344067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.543+0000 7f935d7fa640 1 -- 172.21.15.32:0/3435153036 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6740+0+0 (secure 0 0 0) 0x7f935811e420 con 0x7f93601599e0 2023-12-20T06:00:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.543+0000 7f936492b640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93440658b0 0x7f9344067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.543+0000 7f936492b640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93440658b0 0x7f9344067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f93540058e0 tx=0x7f9354009040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.547+0000 7f935d7fa640 1 -- 172.21.15.32:0/3435153036 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f93580ebd60 con 0x7f93601599e0 2023-12-20T06:00:25.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.707+0000 7f9366b8f640 1 -- 172.21.15.32:0/3435153036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f936004e000 con 0x7f93601599e0 2023-12-20T06:00:25.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.707+0000 7f935d7fa640 1 -- 172.21.15.32:0/3435153036 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v180) v1 ==== 74+0+17578 (secure 0 0 0) 0x7f93580ef380 con 0x7f93601599e0 2023-12-20T06:00:25.708 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:25.708 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":180,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:00:23.879697+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":20,"pool_name":"cephfs_metadata","create_time":"2023-12-20T05:44:18.671073+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"172","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":21,"pool_name":"cephfs_data","create_time":"2023-12-20T05:44:20.010875+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"179","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":33,"snap_epoch":179,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"21.c","mappings":[{"from":4,"to":3}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3370551025":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/2193475721":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/333872760":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/3868230478":"2023-12-21T06:00:23.879665+0000","172.21.15.32:6832/64546":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/987167506":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/2721040122":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/3587707827":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2000830842":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/3914770210":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/2272871714":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2161329994":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/663256086":"2023-12-21T05:55:20.531133+0000","172.21.15.32:0/2039923182":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/3732035268":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/3971579268":"2023-12-21T05:44:33.158109+0000","172.21.15.32:6833/38009":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/1097027017":"2023-12-21T05:44:33.158109+0000","172.21.15.32:6832/38009":"2023-12-21T05:44:33.158109+0000","172.21.15.32:6833/64546":"2023-12-21T06:00:23.879665+0000","172.21.15.32:0/2617565032":"2023-12-21T05:44:33.158109+0000","172.21.15.32:0/2748163648":"2023-12-21T05:44:33.158109+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:00:25.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 -- 172.21.15.32:0/3435153036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93440658b0 msgr2=0x7f9344067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93440658b0 0x7f9344067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f93540058e0 tx=0x7f9354009040 comp rx=0 tx=0).stop 2023-12-20T06:00:25.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 -- 172.21.15.32:0/3435153036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93601599e0 msgr2=0x7f93600ac070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93601599e0 0x7f93600ac070 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f935801b000 tx=0x7f93580196f0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 -- 172.21.15.32:0/3435153036 shutdown_connections 2023-12-20T06:00:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93440658b0 0x7f9344067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93601599e0 0x7f93600ac070 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93600b0b30 0x7f93600acd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 --2- 172.21.15.32:0/3435153036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93600aece0 0x7f93601f5a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 -- 172.21.15.32:0/3435153036 >> 172.21.15.32:0/3435153036 conn(0x7f936006a320 msgr2=0x7f9360155220 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 -- 172.21.15.32:0/3435153036 shutdown_connections 2023-12-20T06:00:25.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.711+0000 7f9366b8f640 1 -- 172.21.15.32:0/3435153036 wait complete. 2023-12-20T06:00:25.729 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T06:00:25.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.875+0000 7fe161cf9640 1 Processor -- start 2023-12-20T06:00:25.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.875+0000 7fe161cf9640 1 -- start start 2023-12-20T06:00:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe161cf9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe15c1599e0 0x7fe15c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe161cf9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe15c0c90e0 0x7fe15c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe161cf9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15c0711d0 con 0x7fe15c0b0b30 2023-12-20T06:00:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe161cf9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15c0c98c0 con 0x7fe15c1599e0 2023-12-20T06:00:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe161cf9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe15c0c9ba0 con 0x7fe15c0c90e0 2023-12-20T06:00:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe15b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe15c1599e0 0x7fe15c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe15b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe15c1599e0 0x7fe15c0aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60260/0 (socket says 172.21.15.32:60260) 2023-12-20T06:00:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe15b7fe640 1 -- 172.21.15.32:0/2796606426 learned_addr learned my addr 172.21.15.32:0/2796606426 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe15affd640 1 --2- 172.21.15.32:0/2796606426 >> v2:172.21.15.39:3300/0 conn(0x7fe15c0c90e0 0x7fe15c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe15b7fe640 1 -- 172.21.15.32:0/2796606426 >> v2:172.21.15.39:3300/0 conn(0x7fe15c0c90e0 msgr2=0x7fe15c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:25.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe15b7fe640 1 --2- 172.21.15.32:0/2796606426 >> v2:172.21.15.39:3300/0 conn(0x7fe15c0c90e0 0x7fe15c0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe15b7fe640 1 -- 172.21.15.32:0/2796606426 >> v1:172.21.15.39:6789/0 conn(0x7fe15c0b0b30 legacy=0x7fe15c15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.879+0000 7fe15b7fe640 1 -- 172.21.15.32:0/2796606426 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe15c0c9e80 con 0x7fe15c1599e0 2023-12-20T06:00:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe15b7fe640 1 --2- 172.21.15.32:0/2796606426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c1599e0 0x7fe15c0aece0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fe15c06ab20 tx=0x7fe14c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed1611f1539389a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe15a7fc640 1 -- 172.21.15.32:0/2796606426 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0ab5f0 con 0x7fe15c1599e0 2023-12-20T06:00:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe15a7fc640 1 -- 172.21.15.32:0/2796606426 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c0ab750 con 0x7fe15c1599e0 2023-12-20T06:00:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe15a7fc640 1 -- 172.21.15.32:0/2796606426 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0abae0 con 0x7fe15c1599e0 2023-12-20T06:00:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe161cf9640 1 -- 172.21.15.32:0/2796606426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c1599e0 msgr2=0x7fe15c0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe161cf9640 1 --2- 172.21.15.32:0/2796606426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c1599e0 0x7fe15c0aece0 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fe15c06ab20 tx=0x7fe14c01c4d0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe161cf9640 1 -- 172.21.15.32:0/2796606426 shutdown_connections 2023-12-20T06:00:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe161cf9640 1 --2- 172.21.15.32:0/2796606426 >> v2:172.21.15.39:3300/0 conn(0x7fe15c0c90e0 0x7fe15c0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe161cf9640 1 --2- 172.21.15.32:0/2796606426 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c1599e0 0x7fe15c0aece0 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe161cf9640 1 -- 172.21.15.32:0/2796606426 >> 172.21.15.32:0/2796606426 conn(0x7fe15c06a320 msgr2=0x7fe15c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe161cf9640 1 -- 172.21.15.32:0/2796606426 shutdown_connections 2023-12-20T06:00:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe161cf9640 1 -- 172.21.15.32:0/2796606426 wait complete. 2023-12-20T06:00:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe161cf9640 1 Processor -- start 2023-12-20T06:00:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.883+0000 7fe161cf9640 1 -- start start 2023-12-20T06:00:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe161cf9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe15c0b0b30 0x7fe15c159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe161cf9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15c0c90e0 0x7fe15c14bcc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe161cf9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c1599e0 0x7fe15c1f4d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe161cf9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c14c200 con 0x7fe15c1599e0 2023-12-20T06:00:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe161cf9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c15a4c0 con 0x7fe15c0c90e0 2023-12-20T06:00:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe161cf9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe15c15a7a0 con 0x7fe15c0b0b30 2023-12-20T06:00:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe15bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c1599e0 0x7fe15c1f4d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe15b7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe15c0b0b30 0x7fe15c159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe15bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c1599e0 0x7fe15c1f4d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60266/0 (socket says 172.21.15.32:60266) 2023-12-20T06:00:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe15bfff640 1 -- 172.21.15.32:0/2655042525 learned_addr learned my addr 172.21.15.32:0/2655042525 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe15affd640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15c0c90e0 0x7fe15c14bcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe15bfff640 1 -- 172.21.15.32:0/2655042525 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe15c0b0b30 msgr2=0x7fe15c159dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe15bfff640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe15c0b0b30 0x7fe15c159dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe15bfff640 1 -- 172.21.15.32:0/2655042525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15c0c90e0 msgr2=0x7fe15c14bcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe15bfff640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15c0c90e0 0x7fe15c14bcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe15bfff640 1 -- 172.21.15.32:0/2655042525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c009480 con 0x7fe15c1599e0 2023-12-20T06:00:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe15bfff640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c1599e0 0x7fe15c1f4d20 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fe15400c730 tx=0x7fe15400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe143fff640 1 -- 172.21.15.32:0/2655042525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe154003b80 con 0x7fe15c1599e0 2023-12-20T06:00:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe143fff640 1 -- 172.21.15.32:0/2655042525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe154003ce0 con 0x7fe15c1599e0 2023-12-20T06:00:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe143fff640 1 -- 172.21.15.32:0/2655042525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe15400aa30 con 0x7fe15c1599e0 2023-12-20T06:00:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe161cf9640 1 -- 172.21.15.32:0/2655042525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe15c15aa80 con 0x7fe15c1599e0 2023-12-20T06:00:25.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe161cf9640 1 -- 172.21.15.32:0/2655042525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe15c15b040 con 0x7fe15c1599e0 2023-12-20T06:00:25.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe161cf9640 1 -- 172.21.15.32:0/2655042525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15c04e000 con 0x7fe15c1599e0 2023-12-20T06:00:25.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.887+0000 7fe143fff640 1 -- 172.21.15.32:0/2655042525 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 111) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fe15400ab90 con 0x7fe15c1599e0 2023-12-20T06:00:25.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.891+0000 7fe143fff640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe1480658b0 0x7fe148067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:25.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.891+0000 7fe15b7fe640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe1480658b0 0x7fe148067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.891+0000 7fe143fff640 1 -- 172.21.15.32:0/2655042525 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(180..180 src has 1..180) v4 ==== 6740+0+0 (secure 0 0 0) 0x7fe15400ae40 con 0x7fe15c1599e0 2023-12-20T06:00:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.891+0000 7fe15b7fe640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe1480658b0 0x7fe148067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0af000 tx=0x7fe14c0a7000 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:25.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.895+0000 7fe143fff640 1 -- 172.21.15.32:0/2655042525 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe1540507c0 con 0x7fe15c1599e0 2023-12-20T06:00:25.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:25.899+0000 7fe143fff640 1 -- 172.21.15.32:0/2655042525 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 112) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fe154050b90 con 0x7fe15c1599e0 2023-12-20T06:00:26.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.095+0000 7fe161cf9640 1 -- 172.21.15.32:0/2655042525 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7fe15c155530 con 0x7fe15c1599e0 2023-12-20T06:00:26.095 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T06:00:26.095+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T06:00:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe143fff640 1 -- 172.21.15.32:0/2655042525 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v80) v1 ==== 161+0+0 (secure 0 0 0) 0x7fe15407cc20 con 0x7fe15c1599e0 2023-12-20T06:00:26.108 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T06:00:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 -- 172.21.15.32:0/2655042525 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe1480658b0 msgr2=0x7fe148067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe1480658b0 0x7fe148067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fe14c0af000 tx=0x7fe14c0a7000 comp rx=0 tx=0).stop 2023-12-20T06:00:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 -- 172.21.15.32:0/2655042525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c1599e0 msgr2=0x7fe15c1f4d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c1599e0 0x7fe15c1f4d20 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fe15400c730 tx=0x7fe15400cbf0 comp rx=0 tx=0).stop 2023-12-20T06:00:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 -- 172.21.15.32:0/2655042525 shutdown_connections 2023-12-20T06:00:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe1480658b0 0x7fe148067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe15c1599e0 0x7fe15c1f4d20 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15c0c90e0 0x7fe15c14bcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:26.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 --2- 172.21.15.32:0/2655042525 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe15c0b0b30 0x7fe15c159dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 -- 172.21.15.32:0/2655042525 >> 172.21.15.32:0/2655042525 conn(0x7fe15c06a320 msgr2=0x7fe15c158ab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 -- 172.21.15.32:0/2655042525 shutdown_connections 2023-12-20T06:00:26.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.111+0000 7fe161cf9640 1 -- 172.21.15.32:0/2655042525 wait complete. 2023-12-20T06:00:26.131 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T06:00:26.173 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T06:00:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.279+0000 7fc45b30b640 1 Processor -- start 2023-12-20T06:00:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.279+0000 7fc45b30b640 1 -- start start 2023-12-20T06:00:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.279+0000 7fc45b30b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4540b0010 0x7fc4540b0470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.279+0000 7fc45b30b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4540b09b0 con 0x7fc4540b16c0 2023-12-20T06:00:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.279+0000 7fc45b30b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc4540c9ff0 con 0x7fc45415a920 2023-12-20T06:00:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.279+0000 7fc45b30b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4540b4510 con 0x7fc4540b0010 2023-12-20T06:00:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.279+0000 7fc45ab0a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc4540b16c0 0x7fc4540b1b30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53590/0 (socket says 172.21.15.32:53590) 2023-12-20T06:00:26.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.279+0000 7fc45ab0a640 1 -- 172.21.15.32:0/2405674400 learned_addr learned my addr 172.21.15.32:0/2405674400 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459b08640 1 --2- 172.21.15.32:0/2405674400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4540b0010 0x7fc4540b0470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459307640 1 -- 172.21.15.32:0/2405674400 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 436565024 0 0) 0x7fc4540b09b0 con 0x7fc4540b16c0 2023-12-20T06:00:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459307640 1 -- 172.21.15.32:0/2405674400 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc440003530 con 0x7fc4540b16c0 2023-12-20T06:00:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459307640 1 -- 172.21.15.32:0/2405674400 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2904248376 0 0) 0x7fc4540c9ff0 con 0x7fc45415a920 2023-12-20T06:00:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459307640 1 -- 172.21.15.32:0/2405674400 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc4540b09b0 con 0x7fc45415a920 2023-12-20T06:00:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459307640 1 -- 172.21.15.32:0/2405674400 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3051290129 0 0) 0x7fc440003530 con 0x7fc4540b16c0 2023-12-20T06:00:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459b08640 1 -- 172.21.15.32:0/2405674400 >> v1:172.21.15.39:6789/0 conn(0x7fc45415a920 legacy=0x7fc4540ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459b08640 1 -- 172.21.15.32:0/2405674400 >> v1:172.21.15.32:6789/0 conn(0x7fc4540b16c0 legacy=0x7fc4540b1b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459b08640 1 -- 172.21.15.32:0/2405674400 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4540c93e0 con 0x7fc4540b0010 2023-12-20T06:00:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459b08640 1 --2- 172.21.15.32:0/2405674400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4540b0010 0x7fc4540b0470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc4440097f0 tx=0x7fc44401c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3227209304fb5a3c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459307640 1 -- 172.21.15.32:0/2405674400 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc44c001a00 con 0x7fc4540b0010 2023-12-20T06:00:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459307640 1 -- 172.21.15.32:0/2405674400 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4440a7050 con 0x7fc4540b0010 2023-12-20T06:00:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.283+0000 7fc459307640 1 -- 172.21.15.32:0/2405674400 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4440ab620 con 0x7fc4540b0010 2023-12-20T06:00:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 -- 172.21.15.32:0/2405674400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4540b0010 msgr2=0x7fc4540b0470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 --2- 172.21.15.32:0/2405674400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4540b0010 0x7fc4540b0470 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fc4440097f0 tx=0x7fc44401c3e0 comp rx=0 tx=0).stop 2023-12-20T06:00:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 -- 172.21.15.32:0/2405674400 shutdown_connections 2023-12-20T06:00:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 --2- 172.21.15.32:0/2405674400 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4540b0010 0x7fc4540b0470 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 -- 172.21.15.32:0/2405674400 >> 172.21.15.32:0/2405674400 conn(0x7fc45406c060 msgr2=0x7fc45406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 -- 172.21.15.32:0/2405674400 shutdown_connections 2023-12-20T06:00:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 -- 172.21.15.32:0/2405674400 wait complete. 2023-12-20T06:00:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 Processor -- start 2023-12-20T06:00:26.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 -- start start 2023-12-20T06:00:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4540b0010 0x7fc454158f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4540b16c0 0x7fc4540c4f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:26.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc45415a920 0x7fc45414b350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc454159500 con 0x7fc4540b16c0 2023-12-20T06:00:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc454156560 con 0x7fc4540b0010 2023-12-20T06:00:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc45b30b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc454156840 con 0x7fc45415a920 2023-12-20T06:00:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc459b08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4540b16c0 0x7fc4540c4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc459b08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4540b16c0 0x7fc4540c4f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60296/0 (socket says 172.21.15.32:60296) 2023-12-20T06:00:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.287+0000 7fc459b08640 1 -- 172.21.15.32:0/1282332498 learned_addr learned my addr 172.21.15.32:0/1282332498 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc45ab0a640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc45415a920 0x7fc45414b350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc45a309640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4540b0010 0x7fc454158f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc459b08640 1 -- 172.21.15.32:0/1282332498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc45415a920 msgr2=0x7fc45414b350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:26.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc459b08640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc45415a920 0x7fc45414b350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc459b08640 1 -- 172.21.15.32:0/1282332498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4540b0010 msgr2=0x7fc454158f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc459b08640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4540b0010 0x7fc454158f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc459b08640 1 -- 172.21.15.32:0/1282332498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc444009420 con 0x7fc4540b16c0 2023-12-20T06:00:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc45ab0a640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc45415a920 0x7fc45414b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc45a309640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4540b0010 0x7fc454158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc459b08640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4540b16c0 0x7fc4540c4f90 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fc4440097f0 tx=0x7fc444019180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc44affd640 1 -- 172.21.15.32:0/1282332498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc444004020 con 0x7fc4540b16c0 2023-12-20T06:00:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc44affd640 1 -- 172.21.15.32:0/1282332498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc444004180 con 0x7fc4540b16c0 2023-12-20T06:00:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc44affd640 1 -- 172.21.15.32:0/1282332498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4440044f0 con 0x7fc4540b16c0 2023-12-20T06:00:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc45b30b640 1 -- 172.21.15.32:0/1282332498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc454156b20 con 0x7fc4540b16c0 2023-12-20T06:00:26.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc45b30b640 1 -- 172.21.15.32:0/1282332498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc45414bdf0 con 0x7fc4540b16c0 2023-12-20T06:00:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc45b30b640 1 -- 172.21.15.32:0/1282332498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc454074100 con 0x7fc4540b16c0 2023-12-20T06:00:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc44affd640 1 -- 172.21.15.32:0/1282332498 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 112) v1 ==== 85274+0+0 (secure 0 0 0) 0x7fc4440b2040 con 0x7fc4540b16c0 2023-12-20T06:00:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.291+0000 7fc44affd640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc438065b20 0x7fc438067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.295+0000 7fc44affd640 1 -- 172.21.15.32:0/1282332498 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6826+0+0 (secure 0 0 0) 0x7fc44411d3f0 con 0x7fc4540b16c0 2023-12-20T06:00:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.295+0000 7fc45a309640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc438065b20 0x7fc438067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:26.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.295+0000 7fc45a309640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc438065b20 0x7fc438067fe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc4500011f0 tx=0x7fc450009040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:26.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.295+0000 7fc44affd640 1 -- 172.21.15.32:0/1282332498 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc4440ea8e0 con 0x7fc4540b16c0 2023-12-20T06:00:26.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:26.495+0000 7fc45b30b640 1 -- 172.21.15.32:0/1282332498 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fc4540ae4c0 con 0x7fc4540b16c0 2023-12-20T06:00:27.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.111+0000 7fc44affd640 1 -- 172.21.15.32:0/1282332498 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v81) v1 ==== 104+0+0 (secure 0 0 0) 0x7fc4440edf00 con 0x7fc4540b16c0 2023-12-20T06:00:27.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 -- 172.21.15.32:0/1282332498 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc438065b20 msgr2=0x7fc438067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:27.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc438065b20 0x7fc438067fe0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc4500011f0 tx=0x7fc450009040 comp rx=0 tx=0).stop 2023-12-20T06:00:27.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 -- 172.21.15.32:0/1282332498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4540b16c0 msgr2=0x7fc4540c4f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:27.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4540b16c0 0x7fc4540c4f90 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7fc4440097f0 tx=0x7fc444019180 comp rx=0 tx=0).stop 2023-12-20T06:00:27.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 -- 172.21.15.32:0/1282332498 shutdown_connections 2023-12-20T06:00:27.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc438065b20 0x7fc438067fe0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:27.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc45415a920 0x7fc45414b350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:27.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4540b16c0 0x7fc4540c4f90 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:27.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 --2- 172.21.15.32:0/1282332498 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4540b0010 0x7fc454158f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:27.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 -- 172.21.15.32:0/1282332498 >> 172.21.15.32:0/1282332498 conn(0x7fc45406c060 msgr2=0x7fc4540b2b90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:27.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 -- 172.21.15.32:0/1282332498 shutdown_connections 2023-12-20T06:00:27.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.115+0000 7fc45b30b640 1 -- 172.21.15.32:0/1282332498 wait complete. 2023-12-20T06:00:27.134 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T06:00:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2912666640 1 Processor -- start 2023-12-20T06:00:27.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2912666640 1 -- start start 2023-12-20T06:00:27.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2912666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290c158520 0x7f290c06dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:27.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2912666640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f290c070230 0x7f290c0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:27.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2912666640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f290c0ca7f0 con 0x7f290c0ca050 2023-12-20T06:00:27.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2912666640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f290c070be0 con 0x7f290c070230 2023-12-20T06:00:27.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2912666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0caad0 con 0x7f290c158520 2023-12-20T06:00:27.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f290b7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f290c0ca050 0x7f290c15ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:53594/0 (socket says 172.21.15.32:53594) 2023-12-20T06:00:27.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f290b7fe640 1 -- 172.21.15.32:0/2451241176 learned_addr learned my addr 172.21.15.32:0/2451241176 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:27.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f290bfff640 1 --2- 172.21.15.32:0/2451241176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290c158520 0x7f290c06dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:27.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2910c03640 1 --2- 172.21.15.32:0/2451241176 >> v2:172.21.15.32:3300/0 conn(0x7f290c070230 0x7f290c0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:27.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2910c03640 1 -- 172.21.15.32:0/2451241176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290c158520 msgr2=0x7f290c06dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:27.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2910c03640 1 --2- 172.21.15.32:0/2451241176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290c158520 0x7f290c06dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:27.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2910c03640 1 -- 172.21.15.32:0/2451241176 >> v1:172.21.15.32:6789/0 conn(0x7f290c0ca050 legacy=0x7f290c15ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:27.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f2910c03640 1 -- 172.21.15.32:0/2451241176 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f290c0cadb0 con 0x7f290c070230 2023-12-20T06:00:27.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.327+0000 7f290affd640 1 -- 172.21.15.32:0/2451241176 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4279130719 0 0) 0x7f290c0ca7f0 con 0x7f290c0ca050 2023-12-20T06:00:27.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f2910c03640 1 --2- 172.21.15.32:0/2451241176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290c070230 0x7f290c0706a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f2904009d90 tx=0x7f290401c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3056f8787a7587d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:27.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f290affd640 1 -- 172.21.15.32:0/2451241176 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29040a5950 con 0x7f290c070230 2023-12-20T06:00:27.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f290affd640 1 -- 172.21.15.32:0/2451241176 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29040a5ab0 con 0x7f290c070230 2023-12-20T06:00:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f290affd640 1 -- 172.21.15.32:0/2451241176 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29040a5e40 con 0x7f290c070230 2023-12-20T06:00:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f2912666640 1 -- 172.21.15.32:0/2451241176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290c070230 msgr2=0x7f290c0706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f2912666640 1 --2- 172.21.15.32:0/2451241176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290c070230 0x7f290c0706a0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f2904009d90 tx=0x7f290401c890 comp rx=0 tx=0).stop 2023-12-20T06:00:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f2912666640 1 -- 172.21.15.32:0/2451241176 shutdown_connections 2023-12-20T06:00:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f2912666640 1 --2- 172.21.15.32:0/2451241176 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290c070230 0x7f290c0706a0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f2912666640 1 --2- 172.21.15.32:0/2451241176 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290c158520 0x7f290c06dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f2912666640 1 -- 172.21.15.32:0/2451241176 >> 172.21.15.32:0/2451241176 conn(0x7f290c06a510 msgr2=0x7f290c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f2912666640 1 -- 172.21.15.32:0/2451241176 shutdown_connections 2023-12-20T06:00:27.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f2912666640 1 -- 172.21.15.32:0/2451241176 wait complete. 2023-12-20T06:00:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.331+0000 7f2912666640 1 Processor -- start 2023-12-20T06:00:27.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2912666640 1 -- start start 2023-12-20T06:00:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2912666640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290c070230 0x7f290c15dc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2912666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290c0ca050 0x7f290c163c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2912666640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f290c158520 0x7f290c1546c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2912666640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c15d540 con 0x7f290c070230 2023-12-20T06:00:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2912666640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0cd890 con 0x7f290c158520 2023-12-20T06:00:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2912666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f290c0ce380 con 0x7f290c0ca050 2023-12-20T06:00:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f290bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290c070230 0x7f290c15dc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f290bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290c070230 0x7f290c15dc90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60324/0 (socket says 172.21.15.32:60324) 2023-12-20T06:00:27.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f290bfff640 1 -- 172.21.15.32:0/504942814 learned_addr learned my addr 172.21.15.32:0/504942814 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f290b7fe640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290c0ca050 0x7f290c163c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2910c03640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f290c158520 0x7f290c1546c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f290bfff640 1 -- 172.21.15.32:0/504942814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290c0ca050 msgr2=0x7f290c163c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f290bfff640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290c0ca050 0x7f290c163c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f290bfff640 1 -- 172.21.15.32:0/504942814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f290c158520 msgr2=0x7f290c1546c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f290bfff640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f290c158520 0x7f290c1546c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f290bfff640 1 -- 172.21.15.32:0/504942814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2904009a70 con 0x7f290c070230 2023-12-20T06:00:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f290bfff640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290c070230 0x7f290c15dc90 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00e870 tx=0x7f28fc00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2908ff9640 1 -- 172.21.15.32:0/504942814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc009510 con 0x7f290c070230 2023-12-20T06:00:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2908ff9640 1 -- 172.21.15.32:0/504942814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28fc00eec0 con 0x7f290c070230 2023-12-20T06:00:27.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2908ff9640 1 -- 172.21.15.32:0/504942814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28fc00cb40 con 0x7f290c070230 2023-12-20T06:00:27.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2912666640 1 -- 172.21.15.32:0/504942814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f290c15d800 con 0x7f290c070230 2023-12-20T06:00:27.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2912666640 1 -- 172.21.15.32:0/504942814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f290c0cdb90 con 0x7f290c070230 2023-12-20T06:00:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.335+0000 7f2912666640 1 -- 172.21.15.32:0/504942814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f290c04e000 con 0x7f290c070230 2023-12-20T06:00:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.339+0000 7f2908ff9640 1 -- 172.21.15.32:0/504942814 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 112) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f28fc00cca0 con 0x7f290c070230 2023-12-20T06:00:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.339+0000 7f2908ff9640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f28f8065af0 0x7f28f8067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.339+0000 7f2908ff9640 1 -- 172.21.15.32:0/504942814 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(181..181 src has 1..181) v4 ==== 6826+0+0 (secure 0 0 0) 0x7f28fc083770 con 0x7f290c070230 2023-12-20T06:00:27.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.339+0000 7f290b7fe640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f28f8065af0 0x7f28f8067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:27.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.339+0000 7f290b7fe640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f28f8065af0 0x7f28f8067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f290c154250 tx=0x7f2900007d60 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:27.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.343+0000 7f2908ff9640 1 -- 172.21.15.32:0/504942814 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f28fc050c90 con 0x7f290c070230 2023-12-20T06:00:27.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:27.503+0000 7f2912666640 1 -- 172.21.15.32:0/504942814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f290c0b1510 con 0x7f290c070230 2023-12-20T06:00:28.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.123+0000 7f2908ff9640 1 -- 172.21.15.32:0/504942814 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f28fc051060 con 0x7f290c070230 2023-12-20T06:00:28.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.127+0000 7f2908ff9640 1 -- 172.21.15.32:0/504942814 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v182) v1 ==== 181+0+0 (secure 0 0 0) 0x7f28fc07f3a0 con 0x7f290c070230 2023-12-20T06:00:28.125 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T06:00:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.127+0000 7f2912666640 1 -- 172.21.15.32:0/504942814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f28f8065af0 msgr2=0x7f28f8067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.127+0000 7f2912666640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f28f8065af0 0x7f28f8067fb0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f290c154250 tx=0x7f2900007d60 comp rx=0 tx=0).stop 2023-12-20T06:00:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.127+0000 7f2912666640 1 -- 172.21.15.32:0/504942814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290c070230 msgr2=0x7f290c15dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.127+0000 7f2912666640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290c070230 0x7f290c15dc90 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f28fc00e870 tx=0x7f28fc00ec70 comp rx=0 tx=0).stop 2023-12-20T06:00:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.131+0000 7f2912666640 1 -- 172.21.15.32:0/504942814 shutdown_connections 2023-12-20T06:00:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.131+0000 7f2912666640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f28f8065af0 0x7f28f8067fb0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.131+0000 7f2912666640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f290c158520 0x7f290c1546c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.131+0000 7f2912666640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f290c0ca050 0x7f290c163c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.131+0000 7f2912666640 1 --2- 172.21.15.32:0/504942814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f290c070230 0x7f290c15dc90 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:28.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.131+0000 7f2912666640 1 -- 172.21.15.32:0/504942814 >> 172.21.15.32:0/504942814 conn(0x7f290c06a510 msgr2=0x7f290c0c4390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:28.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.131+0000 7f2912666640 1 -- 172.21.15.32:0/504942814 shutdown_connections 2023-12-20T06:00:28.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.131+0000 7f2912666640 1 -- 172.21.15.32:0/504942814 wait complete. 2023-12-20T06:00:28.150 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T06:00:28.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.303+0000 7f86085f0640 1 Processor -- start 2023-12-20T06:00:28.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.303+0000 7f86085f0640 1 -- start start 2023-12-20T06:00:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f86085f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86001603e0 0x7f86000b8580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f86085f0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f86000b1d90 0x7f86000b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f86085f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f86000d1030 con 0x7f86000b62d0 2023-12-20T06:00:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f86085f0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86000b8ac0 con 0x7f86001603e0 2023-12-20T06:00:28.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f86085f0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f86000d1280 con 0x7f86000b1d90 2023-12-20T06:00:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f860638c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86001603e0 0x7f86000b8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f860638c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f86001603e0 0x7f86000b8580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60334/0 (socket says 172.21.15.32:60334) 2023-12-20T06:00:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f860638c640 1 -- 172.21.15.32:0/2295036588 learned_addr learned my addr 172.21.15.32:0/2295036588 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f8605b8b640 1 --2- 172.21.15.32:0/2295036588 >> v2:172.21.15.39:3300/0 conn(0x7f86000b1d90 0x7f86000b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f860638c640 1 -- 172.21.15.32:0/2295036588 >> v2:172.21.15.39:3300/0 conn(0x7f86000b1d90 msgr2=0x7f86000b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f860638c640 1 --2- 172.21.15.32:0/2295036588 >> v2:172.21.15.39:3300/0 conn(0x7f86000b1d90 0x7f86000b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f860638c640 1 -- 172.21.15.32:0/2295036588 >> v1:172.21.15.39:6789/0 conn(0x7f86000b62d0 legacy=0x7f86000d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:28.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.307+0000 7f860638c640 1 -- 172.21.15.32:0/2295036588 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f86000d1560 con 0x7f86001603e0 2023-12-20T06:00:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f860638c640 1 --2- 172.21.15.32:0/2295036588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86001603e0 0x7f86000b8580 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f86000b8320 tx=0x7f85fc01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e5ee13414abb9f4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f860538a640 1 -- 172.21.15.32:0/2295036588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0ab600 con 0x7f86001603e0 2023-12-20T06:00:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f860538a640 1 -- 172.21.15.32:0/2295036588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85fc0a7050 con 0x7f86001603e0 2023-12-20T06:00:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f860538a640 1 -- 172.21.15.32:0/2295036588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85fc0ab980 con 0x7f86001603e0 2023-12-20T06:00:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f86085f0640 1 -- 172.21.15.32:0/2295036588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86001603e0 msgr2=0x7f86000b8580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f86085f0640 1 --2- 172.21.15.32:0/2295036588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86001603e0 0x7f86000b8580 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f86000b8320 tx=0x7f85fc01c430 comp rx=0 tx=0).stop 2023-12-20T06:00:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f86085f0640 1 -- 172.21.15.32:0/2295036588 shutdown_connections 2023-12-20T06:00:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f86085f0640 1 --2- 172.21.15.32:0/2295036588 >> v2:172.21.15.39:3300/0 conn(0x7f86000b1d90 0x7f86000b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:28.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f86085f0640 1 --2- 172.21.15.32:0/2295036588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86001603e0 0x7f86000b8580 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:28.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f86085f0640 1 -- 172.21.15.32:0/2295036588 >> 172.21.15.32:0/2295036588 conn(0x7f860006b5e0 msgr2=0x7f860006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:28.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f86085f0640 1 -- 172.21.15.32:0/2295036588 shutdown_connections 2023-12-20T06:00:28.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f86085f0640 1 -- 172.21.15.32:0/2295036588 wait complete. 2023-12-20T06:00:28.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f86085f0640 1 Processor -- start 2023-12-20T06:00:28.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.311+0000 7f86085f0640 1 -- start start 2023-12-20T06:00:28.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f86085f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86000b1d90 0x7f86000b2b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:28.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f86085f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86000b62d0 0x7f86001575f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f86085f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86001603e0 0x7f86000caeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f86085f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f86000b30d0 con 0x7f86000b62d0 2023-12-20T06:00:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f86085f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8600159940 con 0x7f86001603e0 2023-12-20T06:00:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f86085f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8600157fe0 con 0x7f86000b1d90 2023-12-20T06:00:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f8605b8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86000b62d0 0x7f86001575f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f8605b8b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86000b62d0 0x7f86001575f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60338/0 (socket says 172.21.15.32:60338) 2023-12-20T06:00:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f8605b8b640 1 -- 172.21.15.32:0/4228559613 learned_addr learned my addr 172.21.15.32:0/4228559613 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f860638c640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86000b1d90 0x7f86000b2b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f8606b8d640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86001603e0 0x7f86000caeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:28.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f8605b8b640 1 -- 172.21.15.32:0/4228559613 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86000b1d90 msgr2=0x7f86000b2b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f8605b8b640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86000b1d90 0x7f86000b2b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f8605b8b640 1 -- 172.21.15.32:0/4228559613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86001603e0 msgr2=0x7f86000caeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f8605b8b640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86001603e0 0x7f86000caeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f8605b8b640 1 -- 172.21.15.32:0/4228559613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f85fc0094d0 con 0x7f86000b62d0 2023-12-20T06:00:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f8605b8b640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86000b62d0 0x7f86001575f0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f85f000d800 tx=0x7f85f000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f85f6ffd640 1 -- 172.21.15.32:0/4228559613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f000b6c0 con 0x7f86000b62d0 2023-12-20T06:00:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f85f6ffd640 1 -- 172.21.15.32:0/4228559613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f85f000b820 con 0x7f86000b62d0 2023-12-20T06:00:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f85f6ffd640 1 -- 172.21.15.32:0/4228559613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f85f000bb90 con 0x7f86000b62d0 2023-12-20T06:00:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f86085f0640 1 -- 172.21.15.32:0/4228559613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f86001582c0 con 0x7f86000b62d0 2023-12-20T06:00:28.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f86085f0640 1 -- 172.21.15.32:0/4228559613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f86000c6720 con 0x7f86000b62d0 2023-12-20T06:00:28.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f86085f0640 1 -- 172.21.15.32:0/4228559613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f86000c68e0 con 0x7f86000b62d0 2023-12-20T06:00:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.315+0000 7f85f6ffd640 1 -- 172.21.15.32:0/4228559613 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f85f000bcf0 con 0x7f86000b62d0 2023-12-20T06:00:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.319+0000 7f85f6ffd640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f85e4065b80 0x7f85e4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.319+0000 7f85f6ffd640 1 -- 172.21.15.32:0/4228559613 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(182..182 src has 1..182) v4 ==== 6388+0+0 (secure 0 0 0) 0x7f85f0083a20 con 0x7f86000b62d0 2023-12-20T06:00:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.319+0000 7f860638c640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f85e4065b80 0x7f85e4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:28.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.319+0000 7f860638c640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f85e4065b80 0x7f85e4068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0ad000 tx=0x7f85fc008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:28.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.323+0000 7f85f6ffd640 1 -- 172.21.15.32:0/4228559613 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f85f00510c0 con 0x7f86000b62d0 2023-12-20T06:00:28.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:28.483+0000 7f86085f0640 1 -- 172.21.15.32:0/4228559613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f86000d2720 con 0x7f86000b62d0 2023-12-20T06:00:29.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.123+0000 7f85f6ffd640 1 -- 172.21.15.32:0/4228559613 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v183) v1 ==== 169+0+0 (secure 0 0 0) 0x7f85f00546e0 con 0x7f86000b62d0 2023-12-20T06:00:29.123 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T06:00:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 -- 172.21.15.32:0/4228559613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f85e4065b80 msgr2=0x7f85e4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f85e4065b80 0x7f85e4068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f85fc0ad000 tx=0x7f85fc008040 comp rx=0 tx=0).stop 2023-12-20T06:00:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 -- 172.21.15.32:0/4228559613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86000b62d0 msgr2=0x7f86001575f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86000b62d0 0x7f86001575f0 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f85f000d800 tx=0x7f85f000dcc0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 -- 172.21.15.32:0/4228559613 shutdown_connections 2023-12-20T06:00:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f85e4065b80 0x7f85e4068040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f86001603e0 0x7f86000caeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f86000b62d0 0x7f86001575f0 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 --2- 172.21.15.32:0/4228559613 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f86000b1d90 0x7f86000b2b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 -- 172.21.15.32:0/4228559613 >> 172.21.15.32:0/4228559613 conn(0x7f860006b5e0 msgr2=0x7f86000ce400 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 -- 172.21.15.32:0/4228559613 shutdown_connections 2023-12-20T06:00:29.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.127+0000 7f86085f0640 1 -- 172.21.15.32:0/4228559613 wait complete. 2023-12-20T06:00:29.152 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T06:00:29.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.299+0000 7f83942e5640 1 Processor -- start 2023-12-20T06:00:29.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.299+0000 7f83942e5640 1 -- start start 2023-12-20T06:00:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f83942e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f838c157480 0x7f838c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f83942e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f838c0b0760 0x7f838c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f83942e5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f838c0b1100 con 0x7f838c15d660 2023-12-20T06:00:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f83942e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f838c0cde20 con 0x7f838c0b0760 2023-12-20T06:00:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f83942e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c0c8e30 con 0x7f838c157480 2023-12-20T06:00:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f8391880640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f838c0b0760 0x7f838c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f8392882640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f838c15d660 0x7f838c15daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47500/0 (socket says 172.21.15.32:47500) 2023-12-20T06:00:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f8392882640 1 -- 172.21.15.32:0/4129447509 learned_addr learned my addr 172.21.15.32:0/4129447509 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f8391880640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f838c0b0760 0x7f838c0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60458/0 (socket says 172.21.15.32:60458) 2023-12-20T06:00:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f8392081640 1 --2- 172.21.15.32:0/4129447509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f838c157480 0x7f838c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:29.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f8391880640 1 -- 172.21.15.32:0/4129447509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f838c157480 msgr2=0x7f838c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f8391880640 1 --2- 172.21.15.32:0/4129447509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f838c157480 0x7f838c0c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f8391880640 1 -- 172.21.15.32:0/4129447509 >> v1:172.21.15.32:6789/0 conn(0x7f838c15d660 legacy=0x7f838c15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.303+0000 7f8391880640 1 -- 172.21.15.32:0/4129447509 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f838c0ce070 con 0x7f838c0b0760 2023-12-20T06:00:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f8391880640 1 --2- 172.21.15.32:0/4129447509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c0b0760 0x7f838c0b0bc0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f837c0097c0 tx=0x7f837c01c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5edd867747d96cf3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f839107f640 1 -- 172.21.15.32:0/4129447509 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c0ac620 con 0x7f838c0b0760 2023-12-20T06:00:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f839107f640 1 -- 172.21.15.32:0/4129447509 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f837c0ac780 con 0x7f838c0b0760 2023-12-20T06:00:29.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f839107f640 1 -- 172.21.15.32:0/4129447509 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f837c0acb10 con 0x7f838c0b0760 2023-12-20T06:00:29.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f83942e5640 1 -- 172.21.15.32:0/4129447509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c0b0760 msgr2=0x7f838c0b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f83942e5640 1 --2- 172.21.15.32:0/4129447509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c0b0760 0x7f838c0b0bc0 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f837c0097c0 tx=0x7f837c01c3c0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f83942e5640 1 -- 172.21.15.32:0/4129447509 shutdown_connections 2023-12-20T06:00:29.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f83942e5640 1 --2- 172.21.15.32:0/4129447509 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c0b0760 0x7f838c0b0bc0 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f83942e5640 1 --2- 172.21.15.32:0/4129447509 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f838c157480 0x7f838c0c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f83942e5640 1 -- 172.21.15.32:0/4129447509 >> 172.21.15.32:0/4129447509 conn(0x7f838c06a510 msgr2=0x7f838c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:29.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f83942e5640 1 -- 172.21.15.32:0/4129447509 shutdown_connections 2023-12-20T06:00:29.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f83942e5640 1 -- 172.21.15.32:0/4129447509 wait complete. 2023-12-20T06:00:29.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f83942e5640 1 Processor -- start 2023-12-20T06:00:29.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.307+0000 7f83942e5640 1 -- start start 2023-12-20T06:00:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f83942e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f838c0b0760 0x7f838c13a120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f83942e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c157480 0x7f838c1560b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f83942e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c15d660 0x7f838c1565f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f83942e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c15adb0 con 0x7f838c15d660 2023-12-20T06:00:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f83942e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f838c156b30 con 0x7f838c157480 2023-12-20T06:00:29.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f83942e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f838c13d030 con 0x7f838c0b0760 2023-12-20T06:00:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8392882640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c15d660 0x7f838c1565f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8392882640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c15d660 0x7f838c1565f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60462/0 (socket says 172.21.15.32:60462) 2023-12-20T06:00:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8392882640 1 -- 172.21.15.32:0/3292310415 learned_addr learned my addr 172.21.15.32:0/3292310415 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8392081640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f838c0b0760 0x7f838c13a120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8391880640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c157480 0x7f838c1560b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8392882640 1 -- 172.21.15.32:0/3292310415 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f838c0b0760 msgr2=0x7f838c13a120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8392882640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f838c0b0760 0x7f838c13a120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8392882640 1 -- 172.21.15.32:0/3292310415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c157480 msgr2=0x7f838c1560b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8392882640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c157480 0x7f838c1560b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8392882640 1 -- 172.21.15.32:0/3292310415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f837c009420 con 0x7f838c15d660 2023-12-20T06:00:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8392882640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c15d660 0x7f838c1565f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f838400cbf0 tx=0x7f838400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8382ffd640 1 -- 172.21.15.32:0/3292310415 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838400ad40 con 0x7f838c15d660 2023-12-20T06:00:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8382ffd640 1 -- 172.21.15.32:0/3292310415 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8384012400 con 0x7f838c15d660 2023-12-20T06:00:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f8382ffd640 1 -- 172.21.15.32:0/3292310415 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f838400aea0 con 0x7f838c15d660 2023-12-20T06:00:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f83942e5640 1 -- 172.21.15.32:0/3292310415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f838c13d310 con 0x7f838c15d660 2023-12-20T06:00:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f83942e5640 1 -- 172.21.15.32:0/3292310415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f838c13d8d0 con 0x7f838c15d660 2023-12-20T06:00:29.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.311+0000 7f83942e5640 1 -- 172.21.15.32:0/3292310415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f838c15af60 con 0x7f838c15d660 2023-12-20T06:00:29.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.315+0000 7f8382ffd640 1 -- 172.21.15.32:0/3292310415 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8384007070 con 0x7f838c15d660 2023-12-20T06:00:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.315+0000 7f8382ffd640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8370065bb0 0x7f8370068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.315+0000 7f8382ffd640 1 -- 172.21.15.32:0/3292310415 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5945+0+0 (secure 0 0 0) 0x7f8384083350 con 0x7f838c15d660 2023-12-20T06:00:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.315+0000 7f8392081640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8370065bb0 0x7f8370068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:29.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.315+0000 7f8392081640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8370065bb0 0x7f8370068070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f838c152570 tx=0x7f8388008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:29.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.319+0000 7f8382ffd640 1 -- 172.21.15.32:0/3292310415 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f838c15af60 con 0x7f838c15d660 2023-12-20T06:00:29.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.479+0000 7f83942e5640 1 -- 172.21.15.32:0/3292310415 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f838c15af60 con 0x7f838c15d660 2023-12-20T06:00:29.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.483+0000 7f8382ffd640 1 -- 172.21.15.32:0/3292310415 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v61) v1 ==== 120+0+0 (secure 0 0 0) 0x7f838c15af60 con 0x7f838c15d660 2023-12-20T06:00:29.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.483+0000 7f83942e5640 1 -- 172.21.15.32:0/3292310415 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8370065bb0 msgr2=0x7f8370068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.483+0000 7f83942e5640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8370065bb0 0x7f8370068070 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f838c152570 tx=0x7f8388008040 comp rx=0 tx=0).stop 2023-12-20T06:00:29.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.483+0000 7f83942e5640 1 -- 172.21.15.32:0/3292310415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c15d660 msgr2=0x7f838c1565f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.483+0000 7f83942e5640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c15d660 0x7f838c1565f0 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f838400cbf0 tx=0x7f838400a430 comp rx=0 tx=0).stop 2023-12-20T06:00:29.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.487+0000 7f83942e5640 1 -- 172.21.15.32:0/3292310415 shutdown_connections 2023-12-20T06:00:29.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.487+0000 7f83942e5640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8370065bb0 0x7f8370068070 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.487+0000 7f83942e5640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f838c15d660 0x7f838c1565f0 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.487+0000 7f83942e5640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f838c157480 0x7f838c1560b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.487+0000 7f83942e5640 1 --2- 172.21.15.32:0/3292310415 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f838c0b0760 0x7f838c13a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.487+0000 7f83942e5640 1 -- 172.21.15.32:0/3292310415 >> 172.21.15.32:0/3292310415 conn(0x7f838c06a510 msgr2=0x7f838c15da70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:29.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.487+0000 7f83942e5640 1 -- 172.21.15.32:0/3292310415 shutdown_connections 2023-12-20T06:00:29.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.487+0000 7f83942e5640 1 -- 172.21.15.32:0/3292310415 wait complete. 2023-12-20T06:00:29.504 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T06:00:29.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff06a6d1640 1 Processor -- start 2023-12-20T06:00:29.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff06a6d1640 1 -- start start 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff06a6d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff0640b0760 0x7ff0640b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff06a6d1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0640b1100 con 0x7ff0640cc4b0 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff06a6d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0640cde20 con 0x7ff064157480 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff06a6d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0640c9f50 con 0x7ff0640b0760 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff068c6e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0640cc4b0 0x7ff0640cc8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47514/0 (socket says 172.21.15.32:47514) 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff068c6e640 1 -- 172.21.15.32:0/3484461726 learned_addr learned my addr 172.21.15.32:0/3484461726 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff0637fe640 1 --2- 172.21.15.32:0/3484461726 >> v2:172.21.15.39:3300/0 conn(0x7ff0640b0760 0x7ff0640b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff062ffd640 1 -- 172.21.15.32:0/3484461726 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1034747915 0 0) 0x7ff0640b1100 con 0x7ff0640cc4b0 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff062ffd640 1 -- 172.21.15.32:0/3484461726 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff04c005740 con 0x7ff0640cc4b0 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff062ffd640 1 -- 172.21.15.32:0/3484461726 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 98106480 0 0) 0x7ff0640cde20 con 0x7ff064157480 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff062ffd640 1 -- 172.21.15.32:0/3484461726 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0640b1100 con 0x7ff064157480 2023-12-20T06:00:29.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff0637fe640 1 -- 172.21.15.32:0/3484461726 >> v1:172.21.15.39:6789/0 conn(0x7ff064157480 legacy=0x7ff0640c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff062ffd640 1 -- 172.21.15.32:0/3484461726 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4241327936 0 0) 0x7ff04c005740 con 0x7ff0640cc4b0 2023-12-20T06:00:29.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff0637fe640 1 -- 172.21.15.32:0/3484461726 >> v1:172.21.15.32:6789/0 conn(0x7ff0640cc4b0 legacy=0x7ff0640cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.651+0000 7ff0637fe640 1 -- 172.21.15.32:0/3484461726 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0640c8e30 con 0x7ff0640b0760 2023-12-20T06:00:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.655+0000 7ff0637fe640 1 --2- 172.21.15.32:0/3484461726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0640b0760 0x7ff0640b0bc0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff0580097c0 tx=0x7ff05801c2e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4aca8b87442eb88 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.655+0000 7ff062ffd640 1 -- 172.21.15.32:0/3484461726 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff05c001a00 con 0x7ff0640b0760 2023-12-20T06:00:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.655+0000 7ff062ffd640 1 -- 172.21.15.32:0/3484461726 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0580a7050 con 0x7ff0640b0760 2023-12-20T06:00:29.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.655+0000 7ff062ffd640 1 -- 172.21.15.32:0/3484461726 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0580ab600 con 0x7ff0640b0760 2023-12-20T06:00:29.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.655+0000 7ff06a6d1640 1 -- 172.21.15.32:0/3484461726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0640b0760 msgr2=0x7ff0640b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.655+0000 7ff06a6d1640 1 --2- 172.21.15.32:0/3484461726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0640b0760 0x7ff0640b0bc0 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7ff0580097c0 tx=0x7ff05801c2e0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.655+0000 7ff06a6d1640 1 -- 172.21.15.32:0/3484461726 shutdown_connections 2023-12-20T06:00:29.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.655+0000 7ff06a6d1640 1 --2- 172.21.15.32:0/3484461726 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0640b0760 0x7ff0640b0bc0 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.655+0000 7ff06a6d1640 1 -- 172.21.15.32:0/3484461726 >> 172.21.15.32:0/3484461726 conn(0x7ff06406a510 msgr2=0x7ff0640ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:29.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.655+0000 7ff06a6d1640 1 -- 172.21.15.32:0/3484461726 shutdown_connections 2023-12-20T06:00:29.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.655+0000 7ff06a6d1640 1 -- 172.21.15.32:0/3484461726 wait complete. 2023-12-20T06:00:29.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff06a6d1640 1 Processor -- start 2023-12-20T06:00:29.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff06a6d1640 1 -- start start 2023-12-20T06:00:29.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff06a6d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0640b0760 0x7ff0641431b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:29.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff06a6d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0640cc4b0 0x7ff064158ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:29.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff06a6d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff064157480 0x7ff064159220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:29.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff06a6d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0640b31a0 con 0x7ff0640cc4b0 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff06a6d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0641517e0 con 0x7ff064157480 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff06a6d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff064159850 con 0x7ff0640b0760 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff0637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0640cc4b0 0x7ff064158ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff0637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0640cc4b0 0x7ff064158ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60468/0 (socket says 172.21.15.32:60468) 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff0637fe640 1 -- 172.21.15.32:0/1801380249 learned_addr learned my addr 172.21.15.32:0/1801380249 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff068c6e640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff064157480 0x7ff064159220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff063fff640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0640b0760 0x7ff0641431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff0637fe640 1 -- 172.21.15.32:0/1801380249 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0640b0760 msgr2=0x7ff0641431b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff0637fe640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0640b0760 0x7ff0641431b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff0637fe640 1 -- 172.21.15.32:0/1801380249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff064157480 msgr2=0x7ff064159220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff0637fe640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff064157480 0x7ff064159220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff0637fe640 1 -- 172.21.15.32:0/1801380249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff058009420 con 0x7ff0640cc4b0 2023-12-20T06:00:29.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff068c6e640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff064157480 0x7ff064159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:29.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff0637fe640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0640cc4b0 0x7ff064158ce0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff058018ef0 tx=0x7ff058019080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff060ff9640 1 -- 172.21.15.32:0/1801380249 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff058006e20 con 0x7ff0640cc4b0 2023-12-20T06:00:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff060ff9640 1 -- 172.21.15.32:0/1801380249 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0580b4400 con 0x7ff0640cc4b0 2023-12-20T06:00:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff06a6d1640 1 -- 172.21.15.32:0/1801380249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0640b2b50 con 0x7ff0640cc4b0 2023-12-20T06:00:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff060ff9640 1 -- 172.21.15.32:0/1801380249 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0580b4770 con 0x7ff0640cc4b0 2023-12-20T06:00:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.659+0000 7ff06a6d1640 1 -- 172.21.15.32:0/1801380249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0641436f0 con 0x7ff0640cc4b0 2023-12-20T06:00:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.663+0000 7ff06a6d1640 1 -- 172.21.15.32:0/1801380249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0640b3310 con 0x7ff0640cc4b0 2023-12-20T06:00:29.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.663+0000 7ff060ff9640 1 -- 172.21.15.32:0/1801380249 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff0580b2040 con 0x7ff0640cc4b0 2023-12-20T06:00:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.663+0000 7ff060ff9640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff050065b80 0x7ff050068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.663+0000 7ff060ff9640 1 -- 172.21.15.32:0/1801380249 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5945+0+0 (secure 0 0 0) 0x7ff05811d300 con 0x7ff0640cc4b0 2023-12-20T06:00:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.663+0000 7ff063fff640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff050065b80 0x7ff050068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:29.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.663+0000 7ff063fff640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff050065b80 0x7ff050068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff0540011f0 tx=0x7ff054009040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:29.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.667+0000 7ff060ff9640 1 -- 172.21.15.32:0/1801380249 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff0580eaa20 con 0x7ff0640cc4b0 2023-12-20T06:00:29.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.839+0000 7ff06a6d1640 1 -- 172.21.15.32:0/1801380249 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7ff06404e000 con 0x7ff0640cc4b0 2023-12-20T06:00:29.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff060ff9640 1 -- 172.21.15.32:0/1801380249 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v62) v1 ==== 115+0+0 (secure 0 0 0) 0x7ff0580ee040 con 0x7ff0640cc4b0 2023-12-20T06:00:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 -- 172.21.15.32:0/1801380249 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff050065b80 msgr2=0x7ff050068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff050065b80 0x7ff050068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7ff0540011f0 tx=0x7ff054009040 comp rx=0 tx=0).stop 2023-12-20T06:00:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 -- 172.21.15.32:0/1801380249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0640cc4b0 msgr2=0x7ff064158ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0640cc4b0 0x7ff064158ce0 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7ff058018ef0 tx=0x7ff058019080 comp rx=0 tx=0).stop 2023-12-20T06:00:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 -- 172.21.15.32:0/1801380249 shutdown_connections 2023-12-20T06:00:29.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff050065b80 0x7ff050068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff064157480 0x7ff064159220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0640cc4b0 0x7ff064158ce0 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 --2- 172.21.15.32:0/1801380249 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0640b0760 0x7ff0641431b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 -- 172.21.15.32:0/1801380249 >> 172.21.15.32:0/1801380249 conn(0x7ff06406a510 msgr2=0x7ff06406d490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 -- 172.21.15.32:0/1801380249 shutdown_connections 2023-12-20T06:00:29.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:29.843+0000 7ff06a6d1640 1 -- 172.21.15.32:0/1801380249 wait complete. 2023-12-20T06:00:29.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T06:00:30.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.011+0000 7fad1fde9640 1 Processor -- start 2023-12-20T06:00:30.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.011+0000 7fad1fde9640 1 -- start start 2023-12-20T06:00:30.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1fde9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad180b0760 0x7fad180b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1fde9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad180cc4b0 0x7fad180cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1fde9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad180b1100 con 0x7fad18157480 2023-12-20T06:00:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1fde9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad180c9f50 con 0x7fad180cc4b0 2023-12-20T06:00:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1fde9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad180cde20 con 0x7fad180b0760 2023-12-20T06:00:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1e386640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad180cc4b0 0x7fad180cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1e386640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad180cc4b0 0x7fad180cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60478/0 (socket says 172.21.15.32:60478) 2023-12-20T06:00:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1e386640 1 -- 172.21.15.32:0/618856388 learned_addr learned my addr 172.21.15.32:0/618856388 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1d384640 1 --2- 172.21.15.32:0/618856388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad180b0760 0x7fad180b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1e386640 1 -- 172.21.15.32:0/618856388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad180b0760 msgr2=0x7fad180b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:30.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1e386640 1 --2- 172.21.15.32:0/618856388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad180b0760 0x7fad180b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1e386640 1 -- 172.21.15.32:0/618856388 >> v1:172.21.15.39:6789/0 conn(0x7fad18157480 legacy=0x7fad180c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:30.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1e386640 1 -- 172.21.15.32:0/618856388 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad180c8e30 con 0x7fad180cc4b0 2023-12-20T06:00:30.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.015+0000 7fad1e386640 1 --2- 172.21.15.32:0/618856388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad180cc4b0 0x7fad180cc940 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fad10009e50 tx=0x7fad1001eb60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe59eafc0940f987 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:30.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1cb83640 1 -- 172.21.15.32:0/618856388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad100a7a30 con 0x7fad180cc4b0 2023-12-20T06:00:30.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1cb83640 1 -- 172.21.15.32:0/618856388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad100a7b90 con 0x7fad180cc4b0 2023-12-20T06:00:30.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1cb83640 1 -- 172.21.15.32:0/618856388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad100af3f0 con 0x7fad180cc4b0 2023-12-20T06:00:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 -- 172.21.15.32:0/618856388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad180cc4b0 msgr2=0x7fad180cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 --2- 172.21.15.32:0/618856388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad180cc4b0 0x7fad180cc940 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fad10009e50 tx=0x7fad1001eb60 comp rx=0 tx=0).stop 2023-12-20T06:00:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 -- 172.21.15.32:0/618856388 shutdown_connections 2023-12-20T06:00:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 --2- 172.21.15.32:0/618856388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad180cc4b0 0x7fad180cc940 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 --2- 172.21.15.32:0/618856388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad180b0760 0x7fad180b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 -- 172.21.15.32:0/618856388 >> 172.21.15.32:0/618856388 conn(0x7fad1806a510 msgr2=0x7fad180ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:30.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 -- 172.21.15.32:0/618856388 shutdown_connections 2023-12-20T06:00:30.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 -- 172.21.15.32:0/618856388 wait complete. 2023-12-20T06:00:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 Processor -- start 2023-12-20T06:00:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 -- start start 2023-12-20T06:00:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad180b0760 0x7fad180c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad180cc4b0 0x7fad18141d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad18157480 0x7fad181422a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad180c5da0 con 0x7fad180b0760 2023-12-20T06:00:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad18142ac0 con 0x7fad18157480 2023-12-20T06:00:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1fde9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad18142da0 con 0x7fad180cc4b0 2023-12-20T06:00:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1db85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad180b0760 0x7fad180c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.019+0000 7fad1db85640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad180b0760 0x7fad180c38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60490/0 (socket says 172.21.15.32:60490) 2023-12-20T06:00:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1db85640 1 -- 172.21.15.32:0/1388877738 learned_addr learned my addr 172.21.15.32:0/1388877738 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1e386640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad18157480 0x7fad181422a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1d384640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad180cc4b0 0x7fad18141d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1db85640 1 -- 172.21.15.32:0/1388877738 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad180cc4b0 msgr2=0x7fad18141d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:30.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1db85640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad180cc4b0 0x7fad18141d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1db85640 1 -- 172.21.15.32:0/1388877738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad18157480 msgr2=0x7fad181422a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1db85640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad18157480 0x7fad181422a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1db85640 1 -- 172.21.15.32:0/1388877738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad10009b30 con 0x7fad180b0760 2023-12-20T06:00:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1d384640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad180cc4b0 0x7fad18141d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1db85640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad180b0760 0x7fad180c38c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fad1400c500 tx=0x7fad1400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad0e7fc640 1 -- 172.21.15.32:0/1388877738 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14005220 con 0x7fad180b0760 2023-12-20T06:00:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad0e7fc640 1 -- 172.21.15.32:0/1388877738 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad14005380 con 0x7fad180b0760 2023-12-20T06:00:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad0e7fc640 1 -- 172.21.15.32:0/1388877738 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad14007270 con 0x7fad180b0760 2023-12-20T06:00:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1fde9640 1 -- 172.21.15.32:0/1388877738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad180c5a30 con 0x7fad180b0760 2023-12-20T06:00:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1fde9640 1 -- 172.21.15.32:0/1388877738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad180c3e00 con 0x7fad180b0760 2023-12-20T06:00:30.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad1fde9640 1 -- 172.21.15.32:0/1388877738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad180c5890 con 0x7fad180b0760 2023-12-20T06:00:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad0e7fc640 1 -- 172.21.15.32:0/1388877738 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fad140073d0 con 0x7fad180b0760 2023-12-20T06:00:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad0e7fc640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7facfc065b80 0x7facfc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.023+0000 7fad0e7fc640 1 -- 172.21.15.32:0/1388877738 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5945+0+0 (secure 0 0 0) 0x7fad14082ce0 con 0x7fad180b0760 2023-12-20T06:00:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.027+0000 7fad1d384640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7facfc065b80 0x7facfc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.027+0000 7fad1d384640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7facfc065b80 0x7facfc068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad180c4a00 tx=0x7fad08008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:30.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.031+0000 7fad0e7fc640 1 -- 172.21.15.32:0/1388877738 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fad180c5890 con 0x7fad180b0760 2023-12-20T06:00:30.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.187+0000 7fad1fde9640 1 -- 172.21.15.32:0/1388877738 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fad180c5890 con 0x7fad180b0760 2023-12-20T06:00:30.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.191+0000 7fad0e7fc640 1 -- 172.21.15.32:0/1388877738 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v63) v1 ==== 104+0+0 (secure 0 0 0) 0x7fad180c5890 con 0x7fad180b0760 2023-12-20T06:00:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 -- 172.21.15.32:0/1388877738 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7facfc065b80 msgr2=0x7facfc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7facfc065b80 0x7facfc068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7fad180c4a00 tx=0x7fad08008040 comp rx=0 tx=0).stop 2023-12-20T06:00:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 -- 172.21.15.32:0/1388877738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad180b0760 msgr2=0x7fad180c38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad180b0760 0x7fad180c38c0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fad1400c500 tx=0x7fad1400c9c0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 -- 172.21.15.32:0/1388877738 shutdown_connections 2023-12-20T06:00:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7facfc065b80 0x7facfc068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad18157480 0x7fad181422a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad180cc4b0 0x7fad18141d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 --2- 172.21.15.32:0/1388877738 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad180b0760 0x7fad180c38c0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 -- 172.21.15.32:0/1388877738 >> 172.21.15.32:0/1388877738 conn(0x7fad1806a510 msgr2=0x7fad180b3c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 -- 172.21.15.32:0/1388877738 shutdown_connections 2023-12-20T06:00:30.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.195+0000 7fad1fde9640 1 -- 172.21.15.32:0/1388877738 wait complete. 2023-12-20T06:00:30.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart' 2023-12-20T06:00:30.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bd5598640 1 Processor -- start 2023-12-20T06:00:30.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bd5598640 1 -- start start 2023-12-20T06:00:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bd5598640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9bd015cac0 0x7f9bd0160800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bd5598640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bd00cde70 0x7f9bd00d1450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bd5598640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9bd00d0c90 con 0x7f9bd015cea0 2023-12-20T06:00:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bd5598640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9bd0161110 con 0x7f9bd00cde70 2023-12-20T06:00:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bd5598640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd01613f0 con 0x7f9bd015cac0 2023-12-20T06:00:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bcf7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9bd00cde70 0x7f9bd00d1450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bce7fc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9bd015cea0 0x7f9bd00cd760 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47516/0 (socket says 172.21.15.32:47516) 2023-12-20T06:00:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bce7fc640 1 -- 172.21.15.32:0/160804366 learned_addr learned my addr 172.21.15.32:0/160804366 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bceffd640 1 --2- 172.21.15.32:0/160804366 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9bd015cac0 0x7f9bd0160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bcf7fe640 1 -- 172.21.15.32:0/160804366 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9bd015cac0 msgr2=0x7f9bd0160800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bcf7fe640 1 --2- 172.21.15.32:0/160804366 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9bd015cac0 0x7f9bd0160800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bcf7fe640 1 -- 172.21.15.32:0/160804366 >> v1:172.21.15.32:6789/0 conn(0x7f9bd015cea0 legacy=0x7f9bd00cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bcdffb640 1 -- 172.21.15.32:0/160804366 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2519211328 0 0) 0x7f9bd00d0c90 con 0x7f9bd015cea0 2023-12-20T06:00:30.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.359+0000 7f9bcf7fe640 1 -- 172.21.15.32:0/160804366 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bd00d0f30 con 0x7f9bd00cde70 2023-12-20T06:00:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bcf7fe640 1 --2- 172.21.15.32:0/160804366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd00cde70 0x7f9bd00d1450 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009d90 tx=0x7f9bc801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579b98d77231d541 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bcdffb640 1 -- 172.21.15.32:0/160804366 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80a5a30 con 0x7f9bd00cde70 2023-12-20T06:00:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bcdffb640 1 -- 172.21.15.32:0/160804366 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc80a5b90 con 0x7f9bd00cde70 2023-12-20T06:00:30.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bcdffb640 1 -- 172.21.15.32:0/160804366 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80ad440 con 0x7f9bd00cde70 2023-12-20T06:00:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bd5598640 1 -- 172.21.15.32:0/160804366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd00cde70 msgr2=0x7f9bd00d1450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bd5598640 1 --2- 172.21.15.32:0/160804366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd00cde70 0x7f9bd00d1450 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7f9bc8009d90 tx=0x7f9bc801c950 comp rx=0 tx=0).stop 2023-12-20T06:00:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bd5598640 1 -- 172.21.15.32:0/160804366 shutdown_connections 2023-12-20T06:00:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bd5598640 1 --2- 172.21.15.32:0/160804366 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd00cde70 0x7f9bd00d1450 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bd5598640 1 --2- 172.21.15.32:0/160804366 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9bd015cac0 0x7f9bd0160800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bd5598640 1 -- 172.21.15.32:0/160804366 >> 172.21.15.32:0/160804366 conn(0x7f9bd006afd0 msgr2=0x7f9bd006b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bd5598640 1 -- 172.21.15.32:0/160804366 shutdown_connections 2023-12-20T06:00:30.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bd5598640 1 -- 172.21.15.32:0/160804366 wait complete. 2023-12-20T06:00:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bd5598640 1 Processor -- start 2023-12-20T06:00:30.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.363+0000 7f9bd5598640 1 -- start start 2023-12-20T06:00:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bd5598640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bd00cde70 0x7f9bd0154810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bd5598640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9bd015cac0 0x7f9bd00c4210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bd5598640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd015cea0 0x7f9bd00c4750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bd5598640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd0146940 con 0x7f9bd015cea0 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bd5598640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd0154d50 con 0x7f9bd00cde70 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bd5598640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9bd01555b0 con 0x7f9bd015cac0 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bcf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd015cea0 0x7f9bd00c4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bcf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd015cea0 0x7f9bd00c4750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60510/0 (socket says 172.21.15.32:60510) 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bcf7fe640 1 -- 172.21.15.32:0/8099097 learned_addr learned my addr 172.21.15.32:0/8099097 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bce7fc640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9bd015cac0 0x7f9bd00c4210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bceffd640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bd00cde70 0x7f9bd0154810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bcf7fe640 1 -- 172.21.15.32:0/8099097 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9bd015cac0 msgr2=0x7f9bd00c4210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bcf7fe640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9bd015cac0 0x7f9bd00c4210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bcf7fe640 1 -- 172.21.15.32:0/8099097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bd00cde70 msgr2=0x7f9bd0154810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bcf7fe640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bd00cde70 0x7f9bd0154810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:30.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bcf7fe640 1 -- 172.21.15.32:0/8099097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9bc8009a70 con 0x7f9bd015cea0 2023-12-20T06:00:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bcf7fe640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd015cea0 0x7f9bd00c4750 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f9bc801b000 tx=0x7f9bd01618e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bb77fe640 1 -- 172.21.15.32:0/8099097 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80be070 con 0x7f9bd015cea0 2023-12-20T06:00:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bb77fe640 1 -- 172.21.15.32:0/8099097 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9bc8009450 con 0x7f9bd015cea0 2023-12-20T06:00:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bb77fe640 1 -- 172.21.15.32:0/8099097 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9bc80addb0 con 0x7f9bd015cea0 2023-12-20T06:00:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bd5598640 1 -- 172.21.15.32:0/8099097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9bd014df60 con 0x7f9bd015cea0 2023-12-20T06:00:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bd5598640 1 -- 172.21.15.32:0/8099097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9bd0154f60 con 0x7f9bd015cea0 2023-12-20T06:00:30.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.367+0000 7f9bd5598640 1 -- 172.21.15.32:0/8099097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9bd0155740 con 0x7f9bd015cea0 2023-12-20T06:00:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.371+0000 7f9bb77fe640 1 -- 172.21.15.32:0/8099097 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9bc80b4040 con 0x7f9bd015cea0 2023-12-20T06:00:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.371+0000 7f9bb77fe640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bbc065b80 0x7f9bbc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.371+0000 7f9bb77fe640 1 -- 172.21.15.32:0/8099097 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5945+0+0 (secure 0 0 0) 0x7f9bc811db80 con 0x7f9bd015cea0 2023-12-20T06:00:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.371+0000 7f9bceffd640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bbc065b80 0x7f9bbc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:30.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.371+0000 7f9bceffd640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bbc065b80 0x7f9bbc068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009b60 tx=0x7f9bc0009190 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:30.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.375+0000 7f9bb77fe640 1 -- 172.21.15.32:0/8099097 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9bc80eb350 con 0x7f9bd015cea0 2023-12-20T06:00:30.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:30.587+0000 7f9bd5598640 1 -- 172.21.15.32:0/8099097 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]} v 0) v1 -- 0x7f9bd0074100 con 0x7f9bd015cea0 2023-12-20T06:00:31.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bb77fe640 1 -- 172.21.15.32:0/8099097 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_mgr_restart"]}]=0 v2961) v1 ==== 165+0+0 (secure 0 0 0) 0x7f9bc80ee970 con 0x7f9bd015cea0 2023-12-20T06:00:31.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 -- 172.21.15.32:0/8099097 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bbc065b80 msgr2=0x7f9bbc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:31.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bbc065b80 0x7f9bbc068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f9bc0009b60 tx=0x7f9bc0009190 comp rx=0 tx=0).stop 2023-12-20T06:00:31.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 -- 172.21.15.32:0/8099097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd015cea0 msgr2=0x7f9bd00c4750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:31.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd015cea0 0x7f9bd00c4750 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7f9bc801b000 tx=0x7f9bd01618e0 comp rx=0 tx=0).stop 2023-12-20T06:00:31.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 -- 172.21.15.32:0/8099097 shutdown_connections 2023-12-20T06:00:31.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bbc065b80 0x7f9bbc068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:31.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9bd015cea0 0x7f9bd00c4750 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:31.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9bd015cac0 0x7f9bd00c4210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:31.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 --2- 172.21.15.32:0/8099097 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9bd00cde70 0x7f9bd0154810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:31.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 -- 172.21.15.32:0/8099097 >> 172.21.15.32:0/8099097 conn(0x7f9bd006afd0 msgr2=0x7f9bd015b2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:31.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 -- 172.21.15.32:0/8099097 shutdown_connections 2023-12-20T06:00:31.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.143+0000 7f9bd5598640 1 -- 172.21.15.32:0/8099097 wait complete. 2023-12-20T06:00:31.163 INFO:tasks.cephfs_test_runner:Test that snap schedule is resumed after mgr restart ... ok 2023-12-20T06:00:31.164 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) Test scheduled snapshots along with rentention policy 2023-12-20T06:00:31.164 INFO:tasks.cephfs_test_runner:test_snap_schedule_with_retention (tasks.cephfs.test_snap_schedules.TestSnapSchedules) 2023-12-20T06:00:31.164 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-12-20T06:00:31.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.307+0000 7fee3ba66640 1 Processor -- start 2023-12-20T06:00:31.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee3ba66640 1 -- start start 2023-12-20T06:00:31.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee3ba66640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fee3406ce60 0x7fee3415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:31.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee3ba66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee3415f110 0x7fee3415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:31.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee3ba66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee3415bbd0 con 0x7fee340af5e0 2023-12-20T06:00:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee3ba66640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee3415beb0 con 0x7fee3406ce60 2023-12-20T06:00:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee3ba66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee3415e910 con 0x7fee3415f110 2023-12-20T06:00:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee39802640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fee340af5e0 0x7fee340af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47518/0 (socket says 172.21.15.32:47518) 2023-12-20T06:00:31.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee39802640 1 -- 172.21.15.32:0/2451404583 learned_addr learned my addr 172.21.15.32:0/2451404583 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee39001640 1 --2- 172.21.15.32:0/2451404583 >> v2:172.21.15.39:3300/0 conn(0x7fee3406ce60 0x7fee3415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee3a003640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee3415f110 0x7fee3415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4162798505 0 0) 0x7fee3415bbd0 con 0x7fee340af5e0 2023-12-20T06:00:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee20003530 con 0x7fee340af5e0 2023-12-20T06:00:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1612878521 0 0) 0x7fee20003530 con 0x7fee340af5e0 2023-12-20T06:00:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee3415f110 msgr2=0x7fee3415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee3415f110 0x7fee3415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 >> v2:172.21.15.39:3300/0 conn(0x7fee3406ce60 msgr2=0x7fee3415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:31.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 --2- 172.21.15.32:0/2451404583 >> v2:172.21.15.39:3300/0 conn(0x7fee3406ce60 0x7fee3415b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee3415ebf0 con 0x7fee340af5e0 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fee300050a0 con 0x7fee340af5e0 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 >> v1:172.21.15.32:6789/0 conn(0x7fee340af5e0 legacy=0x7fee340af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee20003980 0x7fee20003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee200047d0 0x7fee20004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee3415f110 0x7fee20005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3415beb0 con 0x7fee20003980 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee3415e910 con 0x7fee200047d0 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee2000b9c0 con 0x7fee3415f110 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee39802640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee20003980 0x7fee20003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee3a003640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee3415f110 0x7fee20005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee39001640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee200047d0 0x7fee20004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee39802640 1 -- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee3415f110 msgr2=0x7fee20005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:31.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee39802640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee3415f110 0x7fee20005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee39802640 1 -- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee200047d0 msgr2=0x7fee20004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee39802640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee200047d0 0x7fee20004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee39802640 1 -- 172.21.15.32:0/2451404583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee20003530 con 0x7fee20003980 2023-12-20T06:00:31.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.311+0000 7fee3a003640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee3415f110 0x7fee20005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.315+0000 7fee39802640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee20003980 0x7fee20003da0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c240 tx=0x7fee3001cc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ecf8bd05c8a4394c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.315+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee3415b9d0 con 0x7fee20003980 2023-12-20T06:00:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.315+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee300058e0 con 0x7fee20003980 2023-12-20T06:00:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.315+0000 7fee38800640 1 -- 172.21.15.32:0/2451404583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee30005c70 con 0x7fee20003980 2023-12-20T06:00:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.315+0000 7fee3ba66640 1 -- 172.21.15.32:0/2451404583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee20003980 msgr2=0x7fee20003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.315+0000 7fee3ba66640 1 --2- 172.21.15.32:0/2451404583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee20003980 0x7fee20003da0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7fee3000c240 tx=0x7fee3001cc00 comp rx=0 tx=0).stop 2023-12-20T06:00:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.315+0000 7fee3a003640 1 -- 172.21.15.32:0/2451404583 reap_dead start 2023-12-20T06:00:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.315+0000 7fee3ba66640 1 -- 172.21.15.32:0/2451404583 shutdown_connections 2023-12-20T06:00:31.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.315+0000 7fee3ba66640 1 -- 172.21.15.32:0/2451404583 >> 172.21.15.32:0/2451404583 conn(0x7fee3406a320 msgr2=0x7fee340ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.315+0000 7fee3ba66640 1 -- 172.21.15.32:0/2451404583 shutdown_connections 2023-12-20T06:00:31.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.315+0000 7fee3ba66640 1 -- 172.21.15.32:0/2451404583 wait complete. 2023-12-20T06:00:31.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3ba66640 1 Processor -- start 2023-12-20T06:00:31.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3ba66640 1 -- start start 2023-12-20T06:00:31.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3ba66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee20003980 0x7fee340c6250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3ba66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee34153e00 0x7fee341535a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3ba66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee34154330 0x7fee341574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3ba66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee340c5b20 con 0x7fee34154330 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3ba66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee340c7f10 con 0x7fee34153e00 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3ba66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee340c81f0 con 0x7fee20003980 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3a003640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee34154330 0x7fee341574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3a003640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee34154330 0x7fee341574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60532/0 (socket says 172.21.15.32:60532) 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3a003640 1 -- 172.21.15.32:0/1076490840 learned_addr learned my addr 172.21.15.32:0/1076490840 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee39001640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee34153e00 0x7fee341535a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee39802640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee20003980 0x7fee340c6250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3a003640 1 -- 172.21.15.32:0/1076490840 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee20003980 msgr2=0x7fee340c6250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3a003640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee20003980 0x7fee340c6250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:31.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3a003640 1 -- 172.21.15.32:0/1076490840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee34153e00 msgr2=0x7fee341535a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:31.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3a003640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee34153e00 0x7fee341535a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:31.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3a003640 1 -- 172.21.15.32:0/1076490840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee340c84d0 con 0x7fee34154330 2023-12-20T06:00:31.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3a003640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee34154330 0x7fee341574e0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fee3415f4f0 tx=0x7fee2c009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee2a7fc640 1 -- 172.21.15.32:0/1076490840 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c01b070 con 0x7fee34154330 2023-12-20T06:00:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee2a7fc640 1 -- 172.21.15.32:0/1076490840 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee2c00f040 con 0x7fee34154330 2023-12-20T06:00:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3ba66640 1 -- 172.21.15.32:0/1076490840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee340c5d30 con 0x7fee34154330 2023-12-20T06:00:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee2a7fc640 1 -- 172.21.15.32:0/1076490840 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee2c00e050 con 0x7fee34154330 2023-12-20T06:00:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.319+0000 7fee3ba66640 1 -- 172.21.15.32:0/1076490840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee340c7b30 con 0x7fee34154330 2023-12-20T06:00:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.323+0000 7fee3ba66640 1 -- 172.21.15.32:0/1076490840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee340c86c0 con 0x7fee34154330 2023-12-20T06:00:31.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.323+0000 7fee2a7fc640 1 -- 172.21.15.32:0/1076490840 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fee2c015440 con 0x7fee34154330 2023-12-20T06:00:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.323+0000 7fee2a7fc640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee18065bb0 0x7fee18068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.323+0000 7fee2a7fc640 1 -- 172.21.15.32:0/1076490840 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5945+0+0 (secure 0 0 0) 0x7fee2c083920 con 0x7fee34154330 2023-12-20T06:00:31.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.323+0000 7fee39802640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee18065bb0 0x7fee18068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:31.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.323+0000 7fee39802640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee18065bb0 0x7fee18068070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fee300aa000 tx=0x7fee300b0040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:31.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.327+0000 7fee2a7fc640 1 -- 172.21.15.32:0/1076490840 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee340c86c0 con 0x7fee34154330 2023-12-20T06:00:31.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:31.543+0000 7fee3ba66640 1 -- 172.21.15.32:0/1076490840 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7fee340c8660 con 0x7fee34154330 2023-12-20T06:00:32.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee2a7fc640 1 -- 172.21.15.32:0/1076490840 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v2962) v1 ==== 166+0+0 (secure 0 0 0) 0x7fee2c050fe0 con 0x7fee34154330 2023-12-20T06:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee3ba66640 1 -- 172.21.15.32:0/1076490840 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee18065bb0 msgr2=0x7fee18068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee3ba66640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee18065bb0 0x7fee18068070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fee300aa000 tx=0x7fee300b0040 comp rx=0 tx=0).stop 2023-12-20T06:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee3ba66640 1 -- 172.21.15.32:0/1076490840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee34154330 msgr2=0x7fee341574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee3ba66640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee34154330 0x7fee341574e0 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7fee3415f4f0 tx=0x7fee2c009f90 comp rx=0 tx=0).stop 2023-12-20T06:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee3ba66640 1 -- 172.21.15.32:0/1076490840 shutdown_connections 2023-12-20T06:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee3ba66640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee18065bb0 0x7fee18068070 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee3ba66640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee34154330 0x7fee341574e0 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee3ba66640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee34153e00 0x7fee341535a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee3ba66640 1 --2- 172.21.15.32:0/1076490840 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee20003980 0x7fee340c6250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee3ba66640 1 -- 172.21.15.32:0/1076490840 >> 172.21.15.32:0/1076490840 conn(0x7fee3406a320 msgr2=0x7fee3415e080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:32.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.151+0000 7fee3ba66640 1 -- 172.21.15.32:0/1076490840 shutdown_connections 2023-12-20T06:00:32.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.155+0000 7fee3ba66640 1 -- 172.21.15.32:0/1076490840 wait complete. 2023-12-20T06:00:32.173 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T06:00:32.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.319+0000 7f0dbbbe9640 1 Processor -- start 2023-12-20T06:00:32.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.319+0000 7f0dbbbe9640 1 -- start start 2023-12-20T06:00:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0dbbbe9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0db40ca1a0 0x7f0db40cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0dbbbe9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0db4070230 0x7f0db4070610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0dbbbe9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0db40cb0b0 con 0x7f0db4164670 2023-12-20T06:00:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0dbbbe9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db40ca630 con 0x7f0db40ca1a0 2023-12-20T06:00:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0dbbbe9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0db4070b50 con 0x7f0db4070230 2023-12-20T06:00:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0db9184640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0db40ca1a0 0x7f0db40cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0db9184640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0db40ca1a0 0x7f0db40cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60538/0 (socket says 172.21.15.32:60538) 2023-12-20T06:00:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0db9184640 1 -- 172.21.15.32:0/1698701022 learned_addr learned my addr 172.21.15.32:0/1698701022 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:32.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0dba186640 1 --2- 172.21.15.32:0/1698701022 >> v2:172.21.15.39:3300/0 conn(0x7f0db4070230 0x7f0db4070610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0db9184640 1 -- 172.21.15.32:0/1698701022 >> v2:172.21.15.39:3300/0 conn(0x7f0db4070230 msgr2=0x7f0db4070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:32.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0db9184640 1 --2- 172.21.15.32:0/1698701022 >> v2:172.21.15.39:3300/0 conn(0x7f0db4070230 0x7f0db4070610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0db9184640 1 -- 172.21.15.32:0/1698701022 >> v1:172.21.15.39:6789/0 conn(0x7f0db4164670 legacy=0x7f0db406da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:32.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0db9184640 1 -- 172.21.15.32:0/1698701022 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0db40ca810 con 0x7f0db40ca1a0 2023-12-20T06:00:32.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.323+0000 7f0dba186640 1 --2- 172.21.15.32:0/1698701022 >> v2:172.21.15.39:3300/0 conn(0x7f0db4070230 0x7f0db4070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:32.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0db9184640 1 --2- 172.21.15.32:0/1698701022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db40ca1a0 0x7f0db40cc3b0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f0db406e5e0 tx=0x7f0da401c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4ac1560e8aed9ef8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:32.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0db8983640 1 -- 172.21.15.32:0/1698701022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da40ac600 con 0x7f0db40ca1a0 2023-12-20T06:00:32.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0db8983640 1 -- 172.21.15.32:0/1698701022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0da40a8050 con 0x7f0db40ca1a0 2023-12-20T06:00:32.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0db8983640 1 -- 172.21.15.32:0/1698701022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0da40ac980 con 0x7f0db40ca1a0 2023-12-20T06:00:32.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1698701022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db40ca1a0 msgr2=0x7f0db40cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:32.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0dbbbe9640 1 --2- 172.21.15.32:0/1698701022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db40ca1a0 0x7f0db40cc3b0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f0db406e5e0 tx=0x7f0da401c3a0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1698701022 shutdown_connections 2023-12-20T06:00:32.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0dbbbe9640 1 --2- 172.21.15.32:0/1698701022 >> v2:172.21.15.39:3300/0 conn(0x7f0db4070230 0x7f0db4070610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0dbbbe9640 1 --2- 172.21.15.32:0/1698701022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db40ca1a0 0x7f0db40cc3b0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1698701022 >> 172.21.15.32:0/1698701022 conn(0x7f0db406a510 msgr2=0x7f0db40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:32.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1698701022 shutdown_connections 2023-12-20T06:00:32.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1698701022 wait complete. 2023-12-20T06:00:32.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0dbbbe9640 1 Processor -- start 2023-12-20T06:00:32.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.327+0000 7f0dbbbe9640 1 -- start start 2023-12-20T06:00:32.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dbbbe9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0db4070230 0x7f0db40c82c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dbbbe9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0db40ca1a0 0x7f0db415aff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dbbbe9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db4164670 0x7f0db415b530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dbbbe9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db414e350 con 0x7f0db4164670 2023-12-20T06:00:32.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dbbbe9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40c7e30 con 0x7f0db40ca1a0 2023-12-20T06:00:32.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dbbbe9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0db40c8800 con 0x7f0db4070230 2023-12-20T06:00:32.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dba186640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db4164670 0x7f0db415b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dba186640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db4164670 0x7f0db415b530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60546/0 (socket says 172.21.15.32:60546) 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dba186640 1 -- 172.21.15.32:0/1478885950 learned_addr learned my addr 172.21.15.32:0/1478885950 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0db9184640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0db40ca1a0 0x7f0db415aff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0db9985640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0db4070230 0x7f0db40c82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dba186640 1 -- 172.21.15.32:0/1478885950 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0db4070230 msgr2=0x7f0db40c82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dba186640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0db4070230 0x7f0db40c82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dba186640 1 -- 172.21.15.32:0/1478885950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0db40ca1a0 msgr2=0x7f0db415aff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dba186640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0db40ca1a0 0x7f0db415aff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dba186640 1 -- 172.21.15.32:0/1478885950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0da40094d0 con 0x7f0db4164670 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0db9184640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0db40ca1a0 0x7f0db415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dba186640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db4164670 0x7f0db415b530 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00cd80 tx=0x7f0dac00a650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0daa7fc640 1 -- 172.21.15.32:0/1478885950 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac00f680 con 0x7f0db4164670 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0daa7fc640 1 -- 172.21.15.32:0/1478885950 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0dac00f800 con 0x7f0db4164670 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1478885950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0db40c7220 con 0x7f0db4164670 2023-12-20T06:00:32.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0daa7fc640 1 -- 172.21.15.32:0/1478885950 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0dac00fbb0 con 0x7f0db4164670 2023-12-20T06:00:32.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1478885950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0db40c77e0 con 0x7f0db4164670 2023-12-20T06:00:32.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1478885950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0db40b0fd0 con 0x7f0db4164670 2023-12-20T06:00:32.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.331+0000 7f0daa7fc640 1 -- 172.21.15.32:0/1478885950 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0dac00b040 con 0x7f0db4164670 2023-12-20T06:00:32.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.335+0000 7f0daa7fc640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d98065bb0 0x7f0d98068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.335+0000 7f0daa7fc640 1 -- 172.21.15.32:0/1478885950 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5945+0+0 (secure 0 0 0) 0x7f0dac083fa0 con 0x7f0db4164670 2023-12-20T06:00:32.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.335+0000 7f0db9985640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d98065bb0 0x7f0d98068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.335+0000 7f0db9985640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d98065bb0 0x7f0d98068070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0db00016c0 tx=0x7f0db0009040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:32.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.339+0000 7f0daa7fc640 1 -- 172.21.15.32:0/1478885950 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0dac0515b0 con 0x7f0db4164670 2023-12-20T06:00:32.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.495+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1478885950 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f0db404e000 con 0x7f0db4164670 2023-12-20T06:00:32.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0daa7fc640 1 -- 172.21.15.32:0/1478885950 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v64)=0 v64) v1 ==== 122+0+0 (secure 0 0 0) 0x7f0dac054bd0 con 0x7f0db4164670 2023-12-20T06:00:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1478885950 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d98065bb0 msgr2=0x7f0d98068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d98065bb0 0x7f0d98068070 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f0db00016c0 tx=0x7f0db0009040 comp rx=0 tx=0).stop 2023-12-20T06:00:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1478885950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db4164670 msgr2=0x7f0db415b530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db4164670 0x7f0db415b530 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f0dac00cd80 tx=0x7f0dac00a650 comp rx=0 tx=0).stop 2023-12-20T06:00:32.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1478885950 shutdown_connections 2023-12-20T06:00:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d98065bb0 0x7f0d98068070 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0db4164670 0x7f0db415b530 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0db40ca1a0 0x7f0db415aff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 --2- 172.21.15.32:0/1478885950 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0db4070230 0x7f0db40c82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1478885950 >> 172.21.15.32:0/1478885950 conn(0x7f0db406a510 msgr2=0x7f0db40cdd00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1478885950 shutdown_connections 2023-12-20T06:00:32.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.503+0000 7f0dbbbe9640 1 -- 172.21.15.32:0/1478885950 wait complete. 2023-12-20T06:00:32.522 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T06:00:32.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:00:32.575 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:00:32.649 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T06:00:32.650 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:00:32.650 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:00:32.660 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:00:32.660 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:00:32.660 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:00:32.671 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:00:32.671 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:00:32.671 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:00:32.713 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:00:32.713 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:00:32.713 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:00:32.727 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:00:32.727 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T06:00:32.727 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:00:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f617cad1640 1 Processor -- start 2023-12-20T06:00:32.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f617cad1640 1 -- start start 2023-12-20T06:00:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f617cad1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f61780aece0 0x7f61780b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f617cad1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61780b0b30 0x7f61780701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f617cad1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f61780c98c0 con 0x7f61781599e0 2023-12-20T06:00:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f617cad1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61780c9ba0 con 0x7f61780b0b30 2023-12-20T06:00:32.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f617cad1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f61780c9e80 con 0x7f61780aece0 2023-12-20T06:00:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f6176d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61780b0b30 0x7f61780701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f617659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f61781599e0 0x7f61780c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47526/0 (socket says 172.21.15.32:47526) 2023-12-20T06:00:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f6176d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f61780b0b30 0x7f61780701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60562/0 (socket says 172.21.15.32:60562) 2023-12-20T06:00:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f617659c640 1 -- 172.21.15.32:0/2457043073 learned_addr learned my addr 172.21.15.32:0/2457043073 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.899+0000 7f616ffff640 1 --2- 172.21.15.32:0/2457043073 >> v2:172.21.15.39:3300/0 conn(0x7f61780aece0 0x7f61780b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f6176d9d640 1 -- 172.21.15.32:0/2457043073 >> v2:172.21.15.39:3300/0 conn(0x7f61780aece0 msgr2=0x7f61780b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f6176d9d640 1 --2- 172.21.15.32:0/2457043073 >> v2:172.21.15.39:3300/0 conn(0x7f61780aece0 0x7f61780b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f6176d9d640 1 -- 172.21.15.32:0/2457043073 >> v1:172.21.15.32:6789/0 conn(0x7f61781599e0 legacy=0x7f61780c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f6175d9b640 1 -- 172.21.15.32:0/2457043073 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 602157912 0 0) 0x7f61780c98c0 con 0x7f61781599e0 2023-12-20T06:00:32.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f6176d9d640 1 -- 172.21.15.32:0/2457043073 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f61780ca160 con 0x7f61780b0b30 2023-12-20T06:00:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f6176d9d640 1 --2- 172.21.15.32:0/2457043073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61780b0b30 0x7f61780701c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f61780cf520 tx=0x7f617001d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4224722144f12afb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f6175d9b640 1 -- 172.21.15.32:0/2457043073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700a58e0 con 0x7f61780b0b30 2023-12-20T06:00:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f6175d9b640 1 -- 172.21.15.32:0/2457043073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f61700a5a40 con 0x7f61780b0b30 2023-12-20T06:00:32.903 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f6175d9b640 1 -- 172.21.15.32:0/2457043073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61700a5dd0 con 0x7f61780b0b30 2023-12-20T06:00:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f617cad1640 1 -- 172.21.15.32:0/2457043073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61780b0b30 msgr2=0x7f61780701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f617cad1640 1 --2- 172.21.15.32:0/2457043073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61780b0b30 0x7f61780701c0 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7f61780cf520 tx=0x7f617001d370 comp rx=0 tx=0).stop 2023-12-20T06:00:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f617cad1640 1 -- 172.21.15.32:0/2457043073 shutdown_connections 2023-12-20T06:00:32.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f617cad1640 1 --2- 172.21.15.32:0/2457043073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61780b0b30 0x7f61780701c0 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f617cad1640 1 --2- 172.21.15.32:0/2457043073 >> v2:172.21.15.39:3300/0 conn(0x7f61780aece0 0x7f61780b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f617cad1640 1 -- 172.21.15.32:0/2457043073 >> 172.21.15.32:0/2457043073 conn(0x7f617806a320 msgr2=0x7f61780ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f617cad1640 1 -- 172.21.15.32:0/2457043073 shutdown_connections 2023-12-20T06:00:32.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.903+0000 7f617cad1640 1 -- 172.21.15.32:0/2457043073 wait complete. 2023-12-20T06:00:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617cad1640 1 Processor -- start 2023-12-20T06:00:32.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617cad1640 1 -- start start 2023-12-20T06:00:32.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617cad1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61780aece0 0x7f617814cc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617cad1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61780b0b30 0x7f61781579f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617cad1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61781599e0 0x7f6178148480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617cad1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f617814c350 con 0x7f61780aece0 2023-12-20T06:00:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617cad1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f61781489c0 con 0x7f61781599e0 2023-12-20T06:00:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617cad1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f61781580f0 con 0x7f61780b0b30 2023-12-20T06:00:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61780aece0 0x7f617814cc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61780aece0 0x7f617814cc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60576/0 (socket says 172.21.15.32:60576) 2023-12-20T06:00:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617659c640 1 -- 172.21.15.32:0/2180532766 learned_addr learned my addr 172.21.15.32:0/2180532766 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f616ffff640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61780b0b30 0x7f61781579f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617659c640 1 -- 172.21.15.32:0/2180532766 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61780b0b30 msgr2=0x7f61781579f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:32.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f6176d9d640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61781599e0 0x7f6178148480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617659c640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61780b0b30 0x7f61781579f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617659c640 1 -- 172.21.15.32:0/2180532766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61781599e0 msgr2=0x7f6178148480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617659c640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61781599e0 0x7f6178148480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617659c640 1 -- 172.21.15.32:0/2180532766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6170009a70 con 0x7f61780aece0 2023-12-20T06:00:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f6176d9d640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61781599e0 0x7f6178148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f616ffff640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61780b0b30 0x7f61781579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.907+0000 7f617659c640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61780aece0 0x7f617814cc40 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f616800c430 tx=0x7f616800c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.911+0000 7f616effd640 1 -- 172.21.15.32:0/2180532766 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680074e0 con 0x7f61780aece0 2023-12-20T06:00:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.911+0000 7f616effd640 1 -- 172.21.15.32:0/2180532766 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6168007640 con 0x7f61780aece0 2023-12-20T06:00:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.911+0000 7f616effd640 1 -- 172.21.15.32:0/2180532766 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f61680059d0 con 0x7f61780aece0 2023-12-20T06:00:32.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.911+0000 7f617cad1640 1 -- 172.21.15.32:0/2180532766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f61781571f0 con 0x7f61780aece0 2023-12-20T06:00:32.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.911+0000 7f617cad1640 1 -- 172.21.15.32:0/2180532766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f617814c770 con 0x7f61780aece0 2023-12-20T06:00:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.911+0000 7f617cad1640 1 -- 172.21.15.32:0/2180532766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f617814c490 con 0x7f61780aece0 2023-12-20T06:00:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.911+0000 7f616effd640 1 -- 172.21.15.32:0/2180532766 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6168005b30 con 0x7f61780aece0 2023-12-20T06:00:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.911+0000 7f616effd640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6158065b80 0x7f6158068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.911+0000 7f616effd640 1 -- 172.21.15.32:0/2180532766 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(183..183 src has 1..183) v4 ==== 5945+0+0 (secure 0 0 0) 0x7f6168082a30 con 0x7f61780aece0 2023-12-20T06:00:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.911+0000 7f616ffff640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6158065b80 0x7f6158068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:32.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.911+0000 7f616ffff640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6158065b80 0x7f6158068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6178154d20 tx=0x7f6160008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:32.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:32.915+0000 7f616effd640 1 -- 172.21.15.32:0/2180532766 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f617814c490 con 0x7f61780aece0 2023-12-20T06:00:33.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.103+0000 7f617cad1640 1 -- 172.21.15.32:0/2180532766 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f617814c490 con 0x7f61780aece0 2023-12-20T06:00:33.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.103+0000 7f616effd640 1 -- 172.21.15.32:0/2180532766 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 81 v81) v1 ==== 94+0+3520 (secure 0 0 0) 0x7f6168017020 con 0x7f61780aece0 2023-12-20T06:00:33.103 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:33.104 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":81,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9100,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1922232083","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1922232083},{"type":"v1","addr":"172.21.15.39:6833","nonce":1922232083}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9103,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/428109066","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":428109066},{"type":"v1","addr":"172.21.15.39:6835","nonce":428109066}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9417,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/1595154235","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":1595154235},{"type":"v1","addr":"172.21.15.32:6836","nonce":1595154235}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":74},{"gid":9936,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/99340799","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":99340799},{"type":"v1","addr":"172.21.15.32:6837","nonce":99340799}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":81}],"filesystems":[]} 2023-12-20T06:00:33.104 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 81 2023-12-20T06:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 -- 172.21.15.32:0/2180532766 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6158065b80 msgr2=0x7f6158068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6158065b80 0x7f6158068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f6178154d20 tx=0x7f6160008040 comp rx=0 tx=0).stop 2023-12-20T06:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 -- 172.21.15.32:0/2180532766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61780aece0 msgr2=0x7f617814cc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61780aece0 0x7f617814cc40 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7f616800c430 tx=0x7f616800c8f0 comp rx=0 tx=0).stop 2023-12-20T06:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 -- 172.21.15.32:0/2180532766 shutdown_connections 2023-12-20T06:00:33.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6158065b80 0x7f6158068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f61781599e0 0x7f6178148480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f61780b0b30 0x7f61781579f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 --2- 172.21.15.32:0/2180532766 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f61780aece0 0x7f617814cc40 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 -- 172.21.15.32:0/2180532766 >> 172.21.15.32:0/2180532766 conn(0x7f617806a320 msgr2=0x7f6178156690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 -- 172.21.15.32:0/2180532766 shutdown_connections 2023-12-20T06:00:33.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.107+0000 7f617cad1640 1 -- 172.21.15.32:0/2180532766 wait complete. 2023-12-20T06:00:33.127 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T06:00:33.127 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T06:00:33.127 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T06:00:33.127 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:00:33.128 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T06:00:33.128 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T06:00:33.128 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T06:00:33.128 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:00:33.128 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T06:00:33.129 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T06:00:33.129 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T06:00:33.129 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:00:33.129 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T06:00:33.129 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T06:00:33.129 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T06:00:33.130 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:00:33.140 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T06:00:33.141 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T06:00:33.142 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T06:00:33.142 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T06:00:33.147 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T06:00:33.148 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T06:00:33.149 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T06:00:33.149 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T06:00:33.187 INFO:tasks.ceph.mds.c:Started 2023-12-20T06:00:33.188 DEBUG:teuthology.parallel:result is None 2023-12-20T06:00:33.189 INFO:tasks.ceph.mds.a:Started 2023-12-20T06:00:33.189 DEBUG:teuthology.parallel:result is None 2023-12-20T06:00:33.193 INFO:tasks.ceph.mds.d:Started 2023-12-20T06:00:33.194 DEBUG:teuthology.parallel:result is None 2023-12-20T06:00:33.195 INFO:tasks.ceph.mds.b:Started 2023-12-20T06:00:33.196 DEBUG:teuthology.parallel:result is None 2023-12-20T06:00:33.196 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T06:00:33.276 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T06:00:33.278 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T06:00:33.292 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T06:00:33.294 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T06:00:33.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.351+0000 7fe0beac4640 1 Processor -- start 2023-12-20T06:00:33.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.351+0000 7fe0beac4640 1 -- start start 2023-12-20T06:00:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.351+0000 7fe0beac4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe0b815ae80 0x7fe0b80cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.351+0000 7fe0beac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0b80afd50 0x7fe0b80b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.351+0000 7fe0beac4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe0b80ced30 con 0x7fe0b80b2910 2023-12-20T06:00:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.351+0000 7fe0beac4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe0b80ceb50 con 0x7fe0b815ae80 2023-12-20T06:00:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.351+0000 7fe0beac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b80d0410 con 0x7fe0b80afd50 2023-12-20T06:00:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.355+0000 7fe0bc860640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe0b815ae80 0x7fe0b80cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.355+0000 7fe0bd061640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fe0b80b2910 0x7fe0b806fd60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:55836/0 (socket says 172.21.15.32:55836) 2023-12-20T06:00:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.355+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0b80afd50 0x7fe0b80b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:33.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.355+0000 7fe0bc860640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe0b815ae80 0x7fe0b80cfed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:59722/0 (socket says 172.21.15.32:59722) 2023-12-20T06:00:33.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.355+0000 7fe0bd061640 1 -- 172.21.15.32:0/767735443 learned_addr learned my addr 172.21.15.32:0/767735443 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:33.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.355+0000 7fe0bc860640 1 -- 172.21.15.32:0/767735443 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0b80afd50 msgr2=0x7fe0b80b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:33.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.355+0000 7fe0bc860640 1 --2- 172.21.15.32:0/767735443 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0b80afd50 0x7fe0b80b23d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:33.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.355+0000 7fe0bc860640 1 -- 172.21.15.32:0/767735443 >> v1:172.21.15.39:6789/0 conn(0x7fe0b80b2910 legacy=0x7fe0b806fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:33.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.355+0000 7fe0bc860640 1 -- 172.21.15.32:0/767735443 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0b80cf010 con 0x7fe0b815ae80 2023-12-20T06:00:33.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.355+0000 7fe0bc860640 1 --2- 172.21.15.32:0/767735443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0b815ae80 0x7fe0b80cfed0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0097c0 tx=0x7fe0ac01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8ab163085cf653c4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:33.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.355+0000 7fe0b77fe640 1 -- 172.21.15.32:0/767735443 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0a8970 con 0x7fe0b815ae80 2023-12-20T06:00:33.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0b77fe640 1 -- 172.21.15.32:0/767735443 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0ac0a8ad0 con 0x7fe0b815ae80 2023-12-20T06:00:33.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0b77fe640 1 -- 172.21.15.32:0/767735443 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0ac0a8e60 con 0x7fe0b815ae80 2023-12-20T06:00:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 -- 172.21.15.32:0/767735443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0b815ae80 msgr2=0x7fe0b80cfed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 --2- 172.21.15.32:0/767735443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0b815ae80 0x7fe0b80cfed0 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0097c0 tx=0x7fe0ac01ec70 comp rx=0 tx=0).stop 2023-12-20T06:00:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 -- 172.21.15.32:0/767735443 shutdown_connections 2023-12-20T06:00:33.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 --2- 172.21.15.32:0/767735443 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0b80afd50 0x7fe0b80b23d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:33.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 --2- 172.21.15.32:0/767735443 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0b815ae80 0x7fe0b80cfed0 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:33.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 -- 172.21.15.32:0/767735443 >> 172.21.15.32:0/767735443 conn(0x7fe0b80cd3d0 msgr2=0x7fe0b80cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:33.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 -- 172.21.15.32:0/767735443 shutdown_connections 2023-12-20T06:00:33.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 -- 172.21.15.32:0/767735443 wait complete. 2023-12-20T06:00:33.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 Processor -- start 2023-12-20T06:00:33.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 -- start start 2023-12-20T06:00:33.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0b80afd50 0x7fe0b81f69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b80b2910 0x7fe0b8150d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0b815ae80 0x7fe0b80c44c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b80c6d00 con 0x7fe0b80b2910 2023-12-20T06:00:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b80c6fa0 con 0x7fe0b815ae80 2023-12-20T06:00:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0beac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe0b80c7280 con 0x7fe0b80afd50 2023-12-20T06:00:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b80b2910 0x7fe0b8150d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b80b2910 0x7fe0b8150d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60606/0 (socket says 172.21.15.32:60606) 2023-12-20T06:00:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.359+0000 7fe0b7fff640 1 -- 172.21.15.32:0/456358202 learned_addr learned my addr 172.21.15.32:0/456358202 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0bc860640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0b80afd50 0x7fe0b81f69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:33.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0bd061640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0b815ae80 0x7fe0b80c44c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b7fff640 1 -- 172.21.15.32:0/456358202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0b80afd50 msgr2=0x7fe0b81f69a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b7fff640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0b80afd50 0x7fe0b81f69a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b7fff640 1 -- 172.21.15.32:0/456358202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0b815ae80 msgr2=0x7fe0b80c44c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b7fff640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0b815ae80 0x7fe0b80c44c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b7fff640 1 -- 172.21.15.32:0/456358202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe0ac009420 con 0x7fe0b80b2910 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0bd061640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0b815ae80 0x7fe0b80c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b7fff640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b80b2910 0x7fe0b8150d70 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400e930 tx=0x7fe0a400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b57fa640 1 -- 172.21.15.32:0/456358202 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a4009690 con 0x7fe0b80b2910 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b57fa640 1 -- 172.21.15.32:0/456358202 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe0a400ca00 con 0x7fe0b80b2910 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0beac4640 1 -- 172.21.15.32:0/456358202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe0b80c94c0 con 0x7fe0b80b2910 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b57fa640 1 -- 172.21.15.32:0/456358202 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe0a400cd70 con 0x7fe0b80b2910 2023-12-20T06:00:33.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0beac4640 1 -- 172.21.15.32:0/456358202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe0b80c9a80 con 0x7fe0b80b2910 2023-12-20T06:00:33.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0beac4640 1 -- 172.21.15.32:0/456358202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe0b80c4270 con 0x7fe0b80b2910 2023-12-20T06:00:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b57fa640 1 -- 172.21.15.32:0/456358202 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe0a4013470 con 0x7fe0b80b2910 2023-12-20T06:00:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b57fa640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe09c065b80 0x7fe09c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.363+0000 7fe0b57fa640 1 -- 172.21.15.32:0/456358202 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(187..187 src has 1..187) v4 ==== 5945+0+0 (secure 0 0 0) 0x7fe0a4083650 con 0x7fe0b80b2910 2023-12-20T06:00:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.367+0000 7fe0bc860640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe09c065b80 0x7fe09c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:33.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.367+0000 7fe0bc860640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe09c065b80 0x7fe09c068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0b2000 tx=0x7fe0ac008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:33.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.367+0000 7fe0b57fa640 1 -- 172.21.15.32:0/456358202 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe0a4050ee0 con 0x7fe0b80b2910 2023-12-20T06:00:33.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:33.543+0000 7fe0beac4640 1 -- 172.21.15.32:0/456358202 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7fe0b804e000 con 0x7fe0b80b2910 2023-12-20T06:00:34.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0b57fa640 1 -- 172.21.15.32:0/456358202 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v188) v1 ==== 97+0+0 (secure 0 0 0) 0x7fe0a4054500 con 0x7fe0b80b2910 2023-12-20T06:00:34.337 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T06:00:34.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 -- 172.21.15.32:0/456358202 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe09c065b80 msgr2=0x7fe09c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe09c065b80 0x7fe09c068040 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7fe0ac0b2000 tx=0x7fe0ac008040 comp rx=0 tx=0).stop 2023-12-20T06:00:34.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 -- 172.21.15.32:0/456358202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b80b2910 msgr2=0x7fe0b8150d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b80b2910 0x7fe0b8150d70 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7fe0a400e930 tx=0x7fe0a400edf0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 -- 172.21.15.32:0/456358202 shutdown_connections 2023-12-20T06:00:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe09c065b80 0x7fe09c068040 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe0b815ae80 0x7fe0b80c44c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe0b80b2910 0x7fe0b8150d70 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 --2- 172.21.15.32:0/456358202 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe0b80afd50 0x7fe0b81f69a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 -- 172.21.15.32:0/456358202 >> 172.21.15.32:0/456358202 conn(0x7fe0b80cd3d0 msgr2=0x7fe0b80b5300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 -- 172.21.15.32:0/456358202 shutdown_connections 2023-12-20T06:00:34.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.339+0000 7fe0beac4640 1 -- 172.21.15.32:0/456358202 wait complete. 2023-12-20T06:00:34.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T06:00:34.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.511+0000 7f9d36952640 1 Processor -- start 2023-12-20T06:00:34.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.511+0000 7f9d36952640 1 -- start start 2023-12-20T06:00:34.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.511+0000 7f9d36952640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d3006ce60 0x7f9d3015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.511+0000 7f9d36952640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9d3015f110 0x7f9d3015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.511+0000 7f9d36952640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9d3015bbd0 con 0x7f9d300af5e0 2023-12-20T06:00:34.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.511+0000 7f9d36952640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9d3015beb0 con 0x7f9d3015f110 2023-12-20T06:00:34.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.511+0000 7f9d36952640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d3015e910 con 0x7f9d3006ce60 2023-12-20T06:00:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.511+0000 7f9d27fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d3006ce60 0x7f9d3015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.515+0000 7f9d34eef640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9d3015f110 0x7f9d3015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.515+0000 7f9d34eef640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9d3015f110 0x7f9d3015c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:59726/0 (socket says 172.21.15.32:59726) 2023-12-20T06:00:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.515+0000 7f9d34eef640 1 -- 172.21.15.32:0/829640638 learned_addr learned my addr 172.21.15.32:0/829640638 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.515+0000 7f9d34eef640 1 -- 172.21.15.32:0/829640638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d3006ce60 msgr2=0x7f9d3015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.515+0000 7f9d34eef640 1 --2- 172.21.15.32:0/829640638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d3006ce60 0x7f9d3015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.515+0000 7f9d34eef640 1 -- 172.21.15.32:0/829640638 >> v1:172.21.15.39:6789/0 conn(0x7f9d300af5e0 legacy=0x7f9d300af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.515+0000 7f9d34eef640 1 -- 172.21.15.32:0/829640638 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d3015ebf0 con 0x7f9d3015f110 2023-12-20T06:00:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.515+0000 7f9d34eef640 1 --2- 172.21.15.32:0/829640638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d3015f110 0x7f9d3015c440 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009e50 tx=0x7f9d2801c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=dfe36c839848604d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.515+0000 7f9d2f7fe640 1 -- 172.21.15.32:0/829640638 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d280a57e0 con 0x7f9d3015f110 2023-12-20T06:00:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.515+0000 7f9d2f7fe640 1 -- 172.21.15.32:0/829640638 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d280a5940 con 0x7f9d3015f110 2023-12-20T06:00:34.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.515+0000 7f9d2f7fe640 1 -- 172.21.15.32:0/829640638 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d280a5cd0 con 0x7f9d3015f110 2023-12-20T06:00:34.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 -- 172.21.15.32:0/829640638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d3015f110 msgr2=0x7f9d3015c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 --2- 172.21.15.32:0/829640638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d3015f110 0x7f9d3015c440 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f9d28009e50 tx=0x7f9d2801c980 comp rx=0 tx=0).stop 2023-12-20T06:00:34.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 -- 172.21.15.32:0/829640638 shutdown_connections 2023-12-20T06:00:34.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 --2- 172.21.15.32:0/829640638 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d3015f110 0x7f9d3015c440 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 --2- 172.21.15.32:0/829640638 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d3006ce60 0x7f9d3015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 -- 172.21.15.32:0/829640638 >> 172.21.15.32:0/829640638 conn(0x7f9d3006a320 msgr2=0x7f9d300ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 -- 172.21.15.32:0/829640638 shutdown_connections 2023-12-20T06:00:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 -- 172.21.15.32:0/829640638 wait complete. 2023-12-20T06:00:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 Processor -- start 2023-12-20T06:00:34.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 -- start start 2023-12-20T06:00:34.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d3006ce60 0x7f9d3014ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d300af5e0 0x7f9d30163b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d3015f110 0x7f9d300c45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30070940 con 0x7f9d3015f110 2023-12-20T06:00:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30164060 con 0x7f9d3006ce60 2023-12-20T06:00:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d36952640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9d30164300 con 0x7f9d300af5e0 2023-12-20T06:00:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d34eef640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d3015f110 0x7f9d300c45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d34eef640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d3015f110 0x7f9d300c45a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60618/0 (socket says 172.21.15.32:60618) 2023-12-20T06:00:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d34eef640 1 -- 172.21.15.32:0/2776364953 learned_addr learned my addr 172.21.15.32:0/2776364953 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d27fff640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d300af5e0 0x7f9d30163b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d2ffff640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d3006ce60 0x7f9d3014ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d34eef640 1 -- 172.21.15.32:0/2776364953 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d300af5e0 msgr2=0x7f9d30163b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d34eef640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d300af5e0 0x7f9d30163b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d34eef640 1 -- 172.21.15.32:0/2776364953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d3006ce60 msgr2=0x7f9d3014ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d34eef640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d3006ce60 0x7f9d3014ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d34eef640 1 -- 172.21.15.32:0/2776364953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9d28009b30 con 0x7f9d3015f110 2023-12-20T06:00:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.519+0000 7f9d2ffff640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d3006ce60 0x7f9d3014ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.523+0000 7f9d34eef640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d3015f110 0x7f9d300c45a0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9d280af040 tx=0x7f9d28019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.523+0000 7f9d2d7fa640 1 -- 172.21.15.32:0/2776364953 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d280b8640 con 0x7f9d3015f110 2023-12-20T06:00:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.523+0000 7f9d2d7fa640 1 -- 172.21.15.32:0/2776364953 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9d280b87a0 con 0x7f9d3015f110 2023-12-20T06:00:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.523+0000 7f9d2d7fa640 1 -- 172.21.15.32:0/2776364953 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9d280b8b30 con 0x7f9d3015f110 2023-12-20T06:00:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.523+0000 7f9d36952640 1 -- 172.21.15.32:0/2776364953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9d300702f0 con 0x7f9d3015f110 2023-12-20T06:00:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.523+0000 7f9d36952640 1 -- 172.21.15.32:0/2776364953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9d301645e0 con 0x7f9d3015f110 2023-12-20T06:00:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.523+0000 7f9d36952640 1 -- 172.21.15.32:0/2776364953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9d30070ab0 con 0x7f9d3015f110 2023-12-20T06:00:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.523+0000 7f9d2d7fa640 1 -- 172.21.15.32:0/2776364953 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9d280b8c90 con 0x7f9d3015f110 2023-12-20T06:00:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.523+0000 7f9d2d7fa640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9d10065b80 0x7f9d10068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.523+0000 7f9d2d7fa640 1 -- 172.21.15.32:0/2776364953 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f9d2811da10 con 0x7f9d3015f110 2023-12-20T06:00:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.523+0000 7f9d2ffff640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9d10065b80 0x7f9d10068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.527+0000 7f9d2ffff640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9d10065b80 0x7f9d10068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9d3006c7d0 tx=0x7f9d20009040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:34.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.527+0000 7f9d2d7fa640 1 -- 172.21.15.32:0/2776364953 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9d280eb710 con 0x7f9d3015f110 2023-12-20T06:00:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.739+0000 7f9d36952640 1 -- 172.21.15.32:0/2776364953 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f9d3004e000 con 0x7f9d3015f110 2023-12-20T06:00:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.739+0000 7f9d2d7fa640 1 -- 172.21.15.32:0/2776364953 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v50) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f9d280eed30 con 0x7f9d3015f110 2023-12-20T06:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T06:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T06:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T06:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.739 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T06:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T06:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:00:34.740 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T06:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T06:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.741 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T06:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T06:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:00:34.742 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.743 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.744 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T06:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T06:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T06:00:34.745 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:00:34.746 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.747 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T06:00:34.748 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T06:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T06:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:00:34.749 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:00:34.750 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.751 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T06:00:34.752 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T06:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T06:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T06:00:34.753 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:00:34.754 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T06:00:34.755 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:00:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 -- 172.21.15.32:0/2776364953 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9d10065b80 msgr2=0x7f9d10068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9d10065b80 0x7f9d10068040 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f9d3006c7d0 tx=0x7f9d20009040 comp rx=0 tx=0).stop 2023-12-20T06:00:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 -- 172.21.15.32:0/2776364953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d3015f110 msgr2=0x7f9d300c45a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d3015f110 0x7f9d300c45a0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f9d280af040 tx=0x7f9d28019720 comp rx=0 tx=0).stop 2023-12-20T06:00:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 -- 172.21.15.32:0/2776364953 shutdown_connections 2023-12-20T06:00:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9d10065b80 0x7f9d10068040 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9d3015f110 0x7f9d300c45a0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9d300af5e0 0x7f9d30163b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 --2- 172.21.15.32:0/2776364953 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9d3006ce60 0x7f9d3014ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 -- 172.21.15.32:0/2776364953 >> 172.21.15.32:0/2776364953 conn(0x7f9d3006a320 msgr2=0x7f9d30156f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 -- 172.21.15.32:0/2776364953 shutdown_connections 2023-12-20T06:00:34.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.743+0000 7f9d36952640 1 -- 172.21.15.32:0/2776364953 wait complete. 2023-12-20T06:00:34.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:00:34.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f67cb640 1 Processor -- start 2023-12-20T06:00:34.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f67cb640 1 -- start start 2023-12-20T06:00:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f67cb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15f00b8480 0x7f15f006af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f67cb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f15f006b460 0x7f15f0160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f67cb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f15f00cd4d0 con 0x7f15f0161330 2023-12-20T06:00:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f67cb640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f006b8f0 con 0x7f15f00b8480 2023-12-20T06:00:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f67cb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f15f00ce4e0 con 0x7f15f006b460 2023-12-20T06:00:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f57c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15f00b8480 0x7f15f006af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f57c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f15f00b8480 0x7f15f006af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60632/0 (socket says 172.21.15.32:60632) 2023-12-20T06:00:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f57c9640 1 -- 172.21.15.32:0/1733251520 learned_addr learned my addr 172.21.15.32:0/1733251520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:34.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f4fc8640 1 --2- 172.21.15.32:0/1733251520 >> v2:172.21.15.39:3300/0 conn(0x7f15f006b460 0x7f15f0160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f57c9640 1 -- 172.21.15.32:0/1733251520 >> v2:172.21.15.39:3300/0 conn(0x7f15f006b460 msgr2=0x7f15f0160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:34.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f57c9640 1 --2- 172.21.15.32:0/1733251520 >> v2:172.21.15.39:3300/0 conn(0x7f15f006b460 0x7f15f0160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f57c9640 1 -- 172.21.15.32:0/1733251520 >> v1:172.21.15.39:6789/0 conn(0x7f15f0161330 legacy=0x7f15f0153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.907+0000 7f15f57c9640 1 -- 172.21.15.32:0/1733251520 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15f00cd720 con 0x7f15f00b8480 2023-12-20T06:00:34.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15f57c9640 1 --2- 172.21.15.32:0/1733251520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f00b8480 0x7f15f006af20 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f15f008bdf0 tx=0x7f15ec01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd233e38f50ba17 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:34.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15e7fff640 1 -- 172.21.15.32:0/1733251520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0ab5f0 con 0x7f15f00b8480 2023-12-20T06:00:34.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15e7fff640 1 -- 172.21.15.32:0/1733251520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec0ab750 con 0x7f15f00b8480 2023-12-20T06:00:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15e7fff640 1 -- 172.21.15.32:0/1733251520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0abae0 con 0x7f15f00b8480 2023-12-20T06:00:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15f67cb640 1 -- 172.21.15.32:0/1733251520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f00b8480 msgr2=0x7f15f006af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15f67cb640 1 --2- 172.21.15.32:0/1733251520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f00b8480 0x7f15f006af20 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f15f008bdf0 tx=0x7f15ec01c500 comp rx=0 tx=0).stop 2023-12-20T06:00:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15f67cb640 1 -- 172.21.15.32:0/1733251520 shutdown_connections 2023-12-20T06:00:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15f67cb640 1 --2- 172.21.15.32:0/1733251520 >> v2:172.21.15.39:3300/0 conn(0x7f15f006b460 0x7f15f0160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15f67cb640 1 --2- 172.21.15.32:0/1733251520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f00b8480 0x7f15f006af20 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15f67cb640 1 -- 172.21.15.32:0/1733251520 >> 172.21.15.32:0/1733251520 conn(0x7f15f00d2a50 msgr2=0x7f15f00d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15f67cb640 1 -- 172.21.15.32:0/1733251520 shutdown_connections 2023-12-20T06:00:34.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15f67cb640 1 -- 172.21.15.32:0/1733251520 wait complete. 2023-12-20T06:00:34.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15f67cb640 1 Processor -- start 2023-12-20T06:00:34.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.911+0000 7f15f67cb640 1 -- start start 2023-12-20T06:00:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f67cb640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15f006b460 0x7f15f00c4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f67cb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f00b8480 0x7f15f014d7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f67cb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f0161330 0x7f15f0146a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f67cb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f0152620 con 0x7f15f00b8480 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f67cb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f15f014ab60 con 0x7f15f0161330 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f67cb640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f15f014ae40 con 0x7f15f006b460 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f4fc8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f00b8480 0x7f15f014d7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f57c9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15f006b460 0x7f15f00c4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f4fc8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f00b8480 0x7f15f014d7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60636/0 (socket says 172.21.15.32:60636) 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f4fc8640 1 -- 172.21.15.32:0/1112559592 learned_addr learned my addr 172.21.15.32:0/1112559592 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f5fca640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f0161330 0x7f15f0146a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f57c9640 1 -- 172.21.15.32:0/1112559592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f0161330 msgr2=0x7f15f0146a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f57c9640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f0161330 0x7f15f0146a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f57c9640 1 -- 172.21.15.32:0/1112559592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f00b8480 msgr2=0x7f15f014d7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f57c9640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f00b8480 0x7f15f014d7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f57c9640 1 -- 172.21.15.32:0/1112559592 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f15ec0094d0 con 0x7f15f006b460 2023-12-20T06:00:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f4fc8640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f00b8480 0x7f15f014d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f57c9640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15f006b460 0x7f15f00c4fc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0ac000 tx=0x7f15ec01ad40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15e5ffb640 1 -- 172.21.15.32:0/1112559592 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0be630 con 0x7f15f006b460 2023-12-20T06:00:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15e5ffb640 1 -- 172.21.15.32:0/1112559592 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f15ec0be7b0 con 0x7f15f006b460 2023-12-20T06:00:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f67cb640 1 -- 172.21.15.32:0/1112559592 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f15f014b120 con 0x7f15f006b460 2023-12-20T06:00:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15e5ffb640 1 -- 172.21.15.32:0/1112559592 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f15ec0beb60 con 0x7f15f006b460 2023-12-20T06:00:34.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f67cb640 1 -- 172.21.15.32:0/1112559592 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f15f0152220 con 0x7f15f006b460 2023-12-20T06:00:34.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.915+0000 7f15f67cb640 1 -- 172.21.15.32:0/1112559592 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f15f00720d0 con 0x7f15f006b460 2023-12-20T06:00:34.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.919+0000 7f15e5ffb640 1 -- 172.21.15.32:0/1112559592 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f15ec0b4600 con 0x7f15f006b460 2023-12-20T06:00:34.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.919+0000 7f15e5ffb640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f15d4065b80 0x7f15d4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:34.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.919+0000 7f15e5ffb640 1 -- 172.21.15.32:0/1112559592 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f15ec0a8050 con 0x7f15f006b460 2023-12-20T06:00:34.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.919+0000 7f15f4fc8640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f15d4065b80 0x7f15d4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:34.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.919+0000 7f15f4fc8640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f15d4065b80 0x7f15d4068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15f00c5520 tx=0x7f15e0009190 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:34.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:34.923+0000 7f15e5ffb640 1 -- 172.21.15.32:0/1112559592 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f15ec0ebba0 con 0x7f15f006b460 2023-12-20T06:00:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.079+0000 7f15f67cb640 1 -- 172.21.15.32:0/1112559592 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f15f00c5d70 con 0x7f15f006b460 2023-12-20T06:00:35.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15e5ffb640 1 -- 172.21.15.32:0/1112559592 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v188) v1 ==== 74+0+12441 (secure 0 0 0) 0x7f15ec0c3070 con 0x7f15f006b460 2023-12-20T06:00:35.081 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:35.081 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":188,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:00:34.328016+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":21,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":175,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":171,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:00:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 -- 172.21.15.32:0/1112559592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f15d4065b80 msgr2=0x7f15d4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f15d4065b80 0x7f15d4068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f15f00c5520 tx=0x7f15e0009190 comp rx=0 tx=0).stop 2023-12-20T06:00:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 -- 172.21.15.32:0/1112559592 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15f006b460 msgr2=0x7f15f00c4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15f006b460 0x7f15f00c4fc0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f15ec0ac000 tx=0x7f15ec01ad40 comp rx=0 tx=0).stop 2023-12-20T06:00:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 -- 172.21.15.32:0/1112559592 shutdown_connections 2023-12-20T06:00:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f15d4065b80 0x7f15d4068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f15f0161330 0x7f15f0146a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f15f00b8480 0x7f15f014d7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 --2- 172.21.15.32:0/1112559592 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f15f006b460 0x7f15f00c4fc0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 -- 172.21.15.32:0/1112559592 >> 172.21.15.32:0/1112559592 conn(0x7f15f00d2a50 msgr2=0x7f15f00b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:35.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 -- 172.21.15.32:0/1112559592 shutdown_connections 2023-12-20T06:00:35.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.083+0000 7f15f67cb640 1 -- 172.21.15.32:0/1112559592 wait complete. 2023-12-20T06:00:35.102 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:00:35.102 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:00:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.251+0000 7fdc73fa5640 1 Processor -- start 2023-12-20T06:00:35.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.251+0000 7fdc73fa5640 1 -- start start 2023-12-20T06:00:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.251+0000 7fdc73fa5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc6c157480 0x7fdc6c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.251+0000 7fdc73fa5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdc6c0cd4c0 0x7fdc6c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.251+0000 7fdc73fa5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc6c0b1100 con 0x7fdc6c0b0760 2023-12-20T06:00:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.251+0000 7fdc73fa5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c0cde20 con 0x7fdc6c157480 2023-12-20T06:00:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.251+0000 7fdc73fa5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc6c0c8e30 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.251+0000 7fdc71d41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc6c157480 0x7fdc6c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc71d41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc6c157480 0x7fdc6c0c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60652/0 (socket says 172.21.15.32:60652) 2023-12-20T06:00:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc71d41640 1 -- 172.21.15.32:0/1515851452 learned_addr learned my addr 172.21.15.32:0/1515851452 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:35.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc72542640 1 --2- 172.21.15.32:0/1515851452 >> v2:172.21.15.39:3300/0 conn(0x7fdc6c0cd4c0 0x7fdc6c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc71d41640 1 -- 172.21.15.32:0/1515851452 >> v2:172.21.15.39:3300/0 conn(0x7fdc6c0cd4c0 msgr2=0x7fdc6c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc71d41640 1 --2- 172.21.15.32:0/1515851452 >> v2:172.21.15.39:3300/0 conn(0x7fdc6c0cd4c0 0x7fdc6c0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc71d41640 1 -- 172.21.15.32:0/1515851452 >> v1:172.21.15.39:6789/0 conn(0x7fdc6c0b0760 legacy=0x7fdc6c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc71d41640 1 -- 172.21.15.32:0/1515851452 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc6c0ce070 con 0x7fdc6c157480 2023-12-20T06:00:35.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc72542640 1 --2- 172.21.15.32:0/1515851452 >> v2:172.21.15.39:3300/0 conn(0x7fdc6c0cd4c0 0x7fdc6c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:35.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc71d41640 1 --2- 172.21.15.32:0/1515851452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c157480 0x7fdc6c0c9840 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c071420 tx=0x7fdc6801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989ec9465ee7a9f9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:35.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc70d3f640 1 -- 172.21.15.32:0/1515851452 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc680ad660 con 0x7fdc6c157480 2023-12-20T06:00:35.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc70d3f640 1 -- 172.21.15.32:0/1515851452 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc680ad7c0 con 0x7fdc6c157480 2023-12-20T06:00:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.255+0000 7fdc70d3f640 1 -- 172.21.15.32:0/1515851452 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc680adb50 con 0x7fdc6c157480 2023-12-20T06:00:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 -- 172.21.15.32:0/1515851452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c157480 msgr2=0x7fdc6c0c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 --2- 172.21.15.32:0/1515851452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c157480 0x7fdc6c0c9840 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7fdc6c071420 tx=0x7fdc6801c440 comp rx=0 tx=0).stop 2023-12-20T06:00:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 -- 172.21.15.32:0/1515851452 shutdown_connections 2023-12-20T06:00:35.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 --2- 172.21.15.32:0/1515851452 >> v2:172.21.15.39:3300/0 conn(0x7fdc6c0cd4c0 0x7fdc6c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 --2- 172.21.15.32:0/1515851452 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c157480 0x7fdc6c0c9840 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 -- 172.21.15.32:0/1515851452 >> 172.21.15.32:0/1515851452 conn(0x7fdc6c06a510 msgr2=0x7fdc6c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 -- 172.21.15.32:0/1515851452 shutdown_connections 2023-12-20T06:00:35.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 -- 172.21.15.32:0/1515851452 wait complete. 2023-12-20T06:00:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 Processor -- start 2023-12-20T06:00:35.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 -- start start 2023-12-20T06:00:35.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc6c0b0760 0x7fdc6c0c4840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0cd4c0 0x7fdc6c0c4d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc6c157480 0x7fdc6c1f3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c1f3f50 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c155900 con 0x7fdc6c0b0760 2023-12-20T06:00:35.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc73fa5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc6c155be0 con 0x7fdc6c157480 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc71540640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0cd4c0 0x7fdc6c0c4d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc71540640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0cd4c0 0x7fdc6c0c4d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60660/0 (socket says 172.21.15.32:60660) 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc71540640 1 -- 172.21.15.32:0/2121329547 learned_addr learned my addr 172.21.15.32:0/2121329547 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc71d41640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc6c0b0760 0x7fdc6c0c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc72542640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc6c157480 0x7fdc6c1f3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc71540640 1 -- 172.21.15.32:0/2121329547 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc6c157480 msgr2=0x7fdc6c1f3220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc71540640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc6c157480 0x7fdc6c1f3220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc71540640 1 -- 172.21.15.32:0/2121329547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc6c0b0760 msgr2=0x7fdc6c0c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc71540640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc6c0b0760 0x7fdc6c0c4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.259+0000 7fdc71540640 1 -- 172.21.15.32:0/2121329547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc680094d0 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.263+0000 7fdc71540640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0cd4c0 0x7fdc6c0c4d80 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00c3c0 tx=0x7fdc5c00c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.263+0000 7fdc627fc640 1 -- 172.21.15.32:0/2121329547 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c0052d0 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.263+0000 7fdc627fc640 1 -- 172.21.15.32:0/2121329547 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc5c005430 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.263+0000 7fdc627fc640 1 -- 172.21.15.32:0/2121329547 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc5c003840 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.263+0000 7fdc73fa5640 1 -- 172.21.15.32:0/2121329547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc6c155ec0 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.263+0000 7fdc73fa5640 1 -- 172.21.15.32:0/2121329547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc6c144570 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.263+0000 7fdc73fa5640 1 -- 172.21.15.32:0/2121329547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc6c1447f0 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.263+0000 7fdc627fc640 1 -- 172.21.15.32:0/2121329547 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fdc5c0039a0 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.263+0000 7fdc627fc640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc50065b80 0x7fdc50068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.263+0000 7fdc627fc640 1 -- 172.21.15.32:0/2121329547 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fdc5c082530 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.263+0000 7fdc71d41640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc50065b80 0x7fdc50068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.267+0000 7fdc71d41640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc50065b80 0x7fdc50068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdc680af000 tx=0x7fdc68008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:35.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.267+0000 7fdc627fc640 1 -- 172.21.15.32:0/2121329547 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdc5c050100 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 -- 172.21.15.32:0/2121329547 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fdc6c155a40 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc627fc640 1 -- 172.21.15.32:0/2121329547 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v188) v1 ==== 93+0+10 (secure 0 0 0) 0x7fdc5c017020 con 0x7fdc6c0cd4c0 2023-12-20T06:00:35.426 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 -- 172.21.15.32:0/2121329547 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc50065b80 msgr2=0x7fdc50068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc50065b80 0x7fdc50068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fdc680af000 tx=0x7fdc68008040 comp rx=0 tx=0).stop 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 -- 172.21.15.32:0/2121329547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0cd4c0 msgr2=0x7fdc6c0c4d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0cd4c0 0x7fdc6c0c4d80 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7fdc5c00c3c0 tx=0x7fdc5c00c880 comp rx=0 tx=0).stop 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 -- 172.21.15.32:0/2121329547 shutdown_connections 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc50065b80 0x7fdc50068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc6c157480 0x7fdc6c1f3220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc6c0cd4c0 0x7fdc6c0c4d80 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 --2- 172.21.15.32:0/2121329547 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc6c0b0760 0x7fdc6c0c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 -- 172.21.15.32:0/2121329547 >> 172.21.15.32:0/2121329547 conn(0x7fdc6c06a510 msgr2=0x7fdc6c06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 -- 172.21.15.32:0/2121329547 shutdown_connections 2023-12-20T06:00:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.427+0000 7fdc73fa5640 1 -- 172.21.15.32:0/2121329547 wait complete. 2023-12-20T06:00:35.447 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T06:00:35.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41fa36d640 1 Processor -- start 2023-12-20T06:00:35.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41fa36d640 1 -- start start 2023-12-20T06:00:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41fa36d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41f40af5e0 0x7f41f40af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41fa36d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f41f406ce60 0x7f41f415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41fa36d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41f415bbd0 con 0x7f41f415f110 2023-12-20T06:00:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41fa36d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41f415beb0 con 0x7f41f406ce60 2023-12-20T06:00:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41fa36d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f415e910 con 0x7f41f40af5e0 2023-12-20T06:00:35.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41f890a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f41f415f110 0x7f41f415c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47554/0 (socket says 172.21.15.32:47554) 2023-12-20T06:00:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41f890a640 1 -- 172.21.15.32:0/2323570675 learned_addr learned my addr 172.21.15.32:0/2323570675 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41ebfff640 1 --2- 172.21.15.32:0/2323570675 >> v2:172.21.15.39:3300/0 conn(0x7f41f406ce60 0x7f41f415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41f3fff640 1 --2- 172.21.15.32:0/2323570675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41f40af5e0 0x7f41f40af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41f37fe640 1 -- 172.21.15.32:0/2323570675 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 972697914 0 0) 0x7f41f415bbd0 con 0x7f41f415f110 2023-12-20T06:00:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.590+0000 7f41f37fe640 1 -- 172.21.15.32:0/2323570675 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41e0003530 con 0x7f41f415f110 2023-12-20T06:00:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.594+0000 7f41ebfff640 1 -- 172.21.15.32:0/2323570675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41f40af5e0 msgr2=0x7f41f40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.594+0000 7f41ebfff640 1 --2- 172.21.15.32:0/2323570675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41f40af5e0 0x7f41f40af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.594+0000 7f41ebfff640 1 -- 172.21.15.32:0/2323570675 >> v1:172.21.15.32:6789/0 conn(0x7f41f415f110 legacy=0x7f41f415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.594+0000 7f41ebfff640 1 -- 172.21.15.32:0/2323570675 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41f415ebf0 con 0x7f41f406ce60 2023-12-20T06:00:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.594+0000 7f41ebfff640 1 --2- 172.21.15.32:0/2323570675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41f406ce60 0x7f41f415b280 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097c0 tx=0x7f41dc01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=195fc6ad585a9255 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.594+0000 7f41f37fe640 1 -- 172.21.15.32:0/2323570675 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0ac640 con 0x7f41f406ce60 2023-12-20T06:00:35.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.594+0000 7f41f37fe640 1 -- 172.21.15.32:0/2323570675 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41dc0ac7a0 con 0x7f41f406ce60 2023-12-20T06:00:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.594+0000 7f41f37fe640 1 -- 172.21.15.32:0/2323570675 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41dc0acb30 con 0x7f41f406ce60 2023-12-20T06:00:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 -- 172.21.15.32:0/2323570675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41f406ce60 msgr2=0x7f41f415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 --2- 172.21.15.32:0/2323570675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41f406ce60 0x7f41f415b280 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0097c0 tx=0x7f41dc01c480 comp rx=0 tx=0).stop 2023-12-20T06:00:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 -- 172.21.15.32:0/2323570675 shutdown_connections 2023-12-20T06:00:35.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 --2- 172.21.15.32:0/2323570675 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41f406ce60 0x7f41f415b280 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 --2- 172.21.15.32:0/2323570675 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41f40af5e0 0x7f41f40af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 -- 172.21.15.32:0/2323570675 >> 172.21.15.32:0/2323570675 conn(0x7f41f406a320 msgr2=0x7f41f40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 -- 172.21.15.32:0/2323570675 shutdown_connections 2023-12-20T06:00:35.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 -- 172.21.15.32:0/2323570675 wait complete. 2023-12-20T06:00:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 Processor -- start 2023-12-20T06:00:35.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 -- start start 2023-12-20T06:00:35.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f406ce60 0x7f41f4157a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41f40af5e0 0x7f41f41479e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41f415f110 0x7f41f40ce190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40c73e0 con 0x7f41f406ce60 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40c76c0 con 0x7f41f415f110 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41fa36d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41f40c79a0 con 0x7f41f40af5e0 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f406ce60 0x7f41f4157a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f406ce60 0x7f41f4157a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60666/0 (socket says 172.21.15.32:60666) 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41f3fff640 1 -- 172.21.15.32:0/557250256 learned_addr learned my addr 172.21.15.32:0/557250256 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41f3fff640 1 -- 172.21.15.32:0/557250256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41f40af5e0 msgr2=0x7f41f41479e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41f3fff640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41f40af5e0 0x7f41f41479e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41f3fff640 1 -- 172.21.15.32:0/557250256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41f415f110 msgr2=0x7f41f40ce190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41f3fff640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41f415f110 0x7f41f40ce190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.598+0000 7f41f3fff640 1 -- 172.21.15.32:0/557250256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41dc009420 con 0x7f41f406ce60 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.602+0000 7f41f3fff640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f406ce60 0x7f41f4157a80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9b0 tx=0x7f41e400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.602+0000 7f41f17fa640 1 -- 172.21.15.32:0/557250256 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e4009670 con 0x7f41f406ce60 2023-12-20T06:00:35.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.602+0000 7f41f17fa640 1 -- 172.21.15.32:0/557250256 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41e400c9a0 con 0x7f41f406ce60 2023-12-20T06:00:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.602+0000 7f41fa36d640 1 -- 172.21.15.32:0/557250256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41f40c7c80 con 0x7f41f406ce60 2023-12-20T06:00:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.602+0000 7f41f17fa640 1 -- 172.21.15.32:0/557250256 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41e400cd10 con 0x7f41f406ce60 2023-12-20T06:00:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.602+0000 7f41fa36d640 1 -- 172.21.15.32:0/557250256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41f4157fc0 con 0x7f41f406ce60 2023-12-20T06:00:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.602+0000 7f41fa36d640 1 -- 172.21.15.32:0/557250256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41f404e000 con 0x7f41f406ce60 2023-12-20T06:00:35.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.602+0000 7f41f17fa640 1 -- 172.21.15.32:0/557250256 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f41e4013470 con 0x7f41f406ce60 2023-12-20T06:00:35.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.602+0000 7f41f17fa640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f41d4065b80 0x7f41d4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.602+0000 7f41f17fa640 1 -- 172.21.15.32:0/557250256 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f41e4083610 con 0x7f41f406ce60 2023-12-20T06:00:35.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.602+0000 7f41ebfff640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f41d4065b80 0x7f41d4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.606+0000 7f41ebfff640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f41d4065b80 0x7f41d4068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0ae000 tx=0x7f41dc004740 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:35.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.606+0000 7f41f17fa640 1 -- 172.21.15.32:0/557250256 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f41e40511d0 con 0x7f41f406ce60 2023-12-20T06:00:35.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.762+0000 7f41fa36d640 1 -- 172.21.15.32:0/557250256 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f41f40cf450 con 0x7f41f406ce60 2023-12-20T06:00:35.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.762+0000 7f41f17fa640 1 -- 172.21.15.32:0/557250256 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v188) v1 ==== 84+0+65 (secure 0 0 0) 0x7f41e4021060 con 0x7f41f406ce60 2023-12-20T06:00:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:35.764 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T06:00:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T06:00:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T06:00:35.764 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:35.764 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T06:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 -- 172.21.15.32:0/557250256 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f41d4065b80 msgr2=0x7f41d4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f41d4065b80 0x7f41d4068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f41dc0ae000 tx=0x7f41dc004740 comp rx=0 tx=0).stop 2023-12-20T06:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 -- 172.21.15.32:0/557250256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f406ce60 msgr2=0x7f41f4157a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f406ce60 0x7f41f4157a80 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f41e400e9b0 tx=0x7f41e400ee70 comp rx=0 tx=0).stop 2023-12-20T06:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 -- 172.21.15.32:0/557250256 shutdown_connections 2023-12-20T06:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f41d4065b80 0x7f41d4068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41f415f110 0x7f41f40ce190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41f40af5e0 0x7f41f41479e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 --2- 172.21.15.32:0/557250256 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41f406ce60 0x7f41f4157a80 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 -- 172.21.15.32:0/557250256 >> 172.21.15.32:0/557250256 conn(0x7f41f406a320 msgr2=0x7f41f40b3d60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:35.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 -- 172.21.15.32:0/557250256 shutdown_connections 2023-12-20T06:00:35.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.766+0000 7f41fa36d640 1 -- 172.21.15.32:0/557250256 wait complete. 2023-12-20T06:00:35.785 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T06:00:35.785 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T06:00:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d943640 1 Processor -- start 2023-12-20T06:00:35.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d943640 1 -- start start 2023-12-20T06:00:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d943640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3358158fe0 0x7f335815d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d943640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f33580c9520 0x7f33580cac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d943640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33580cb140 0x7f33580affe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d943640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33580cb5d0 con 0x7f33580cb140 2023-12-20T06:00:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d943640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33580c9c70 con 0x7f33580c9520 2023-12-20T06:00:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d943640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33580c9f50 con 0x7f3358158fe0 2023-12-20T06:00:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d142640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33580cb140 0x7f33580affe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d142640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33580cb140 0x7f33580affe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60680/0 (socket says 172.21.15.32:60680) 2023-12-20T06:00:35.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d142640 1 -- 172.21.15.32:0/3930507490 learned_addr learned my addr 172.21.15.32:0/3930507490 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f3357fff640 1 --2- 172.21.15.32:0/3930507490 >> v2:172.21.15.39:3300/0 conn(0x7f33580c9520 0x7f33580cac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335c941640 1 --2- 172.21.15.32:0/3930507490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3358158fe0 0x7f335815d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d142640 1 -- 172.21.15.32:0/3930507490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3358158fe0 msgr2=0x7f335815d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d142640 1 --2- 172.21.15.32:0/3930507490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3358158fe0 0x7f335815d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d142640 1 -- 172.21.15.32:0/3930507490 >> v2:172.21.15.39:3300/0 conn(0x7f33580c9520 msgr2=0x7f33580cac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d142640 1 --2- 172.21.15.32:0/3930507490 >> v2:172.21.15.39:3300/0 conn(0x7f33580c9520 0x7f33580cac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335d142640 1 -- 172.21.15.32:0/3930507490 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33580ca230 con 0x7f33580cb140 2023-12-20T06:00:35.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.934+0000 7f335c941640 1 --2- 172.21.15.32:0/3930507490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3358158fe0 0x7f335815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f335d142640 1 --2- 172.21.15.32:0/3930507490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33580cb140 0x7f33580affe0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f33580b1120 tx=0x7f335001c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6179a9e36f1bf0d8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f33577fe640 1 -- 172.21.15.32:0/3930507490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33500073f0 con 0x7f33580cb140 2023-12-20T06:00:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f33577fe640 1 -- 172.21.15.32:0/3930507490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3350007550 con 0x7f33580cb140 2023-12-20T06:00:35.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f33577fe640 1 -- 172.21.15.32:0/3930507490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33500078e0 con 0x7f33580cb140 2023-12-20T06:00:35.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f335d943640 1 -- 172.21.15.32:0/3930507490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33580cb140 msgr2=0x7f33580affe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f335d943640 1 --2- 172.21.15.32:0/3930507490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33580cb140 0x7f33580affe0 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f33580b1120 tx=0x7f335001c680 comp rx=0 tx=0).stop 2023-12-20T06:00:35.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f335d943640 1 -- 172.21.15.32:0/3930507490 shutdown_connections 2023-12-20T06:00:35.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f335d943640 1 --2- 172.21.15.32:0/3930507490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33580cb140 0x7f33580affe0 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f335d943640 1 --2- 172.21.15.32:0/3930507490 >> v2:172.21.15.39:3300/0 conn(0x7f33580c9520 0x7f33580cac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f335d943640 1 --2- 172.21.15.32:0/3930507490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3358158fe0 0x7f335815d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f335d943640 1 -- 172.21.15.32:0/3930507490 >> 172.21.15.32:0/3930507490 conn(0x7f335806a4e0 msgr2=0x7f335806bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f335d943640 1 -- 172.21.15.32:0/3930507490 shutdown_connections 2023-12-20T06:00:35.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.938+0000 7f335d943640 1 -- 172.21.15.32:0/3930507490 wait complete. 2023-12-20T06:00:35.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335d943640 1 Processor -- start 2023-12-20T06:00:35.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335d943640 1 -- start start 2023-12-20T06:00:35.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335d943640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33580c9520 0x7f33580b2b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335d943640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33580cb140 0x7f335814d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335d943640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3358158fe0 0x7f335814b950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335d943640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f335814d000 con 0x7f33580c9520 2023-12-20T06:00:35.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335d943640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33580b3040 con 0x7f3358158fe0 2023-12-20T06:00:35.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335d943640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f335814c380 con 0x7f33580cb140 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335c941640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33580c9520 0x7f33580b2b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335c941640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33580c9520 0x7f33580b2b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60688/0 (socket says 172.21.15.32:60688) 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335c941640 1 -- 172.21.15.32:0/224681285 learned_addr learned my addr 172.21.15.32:0/224681285 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f3357fff640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33580cb140 0x7f335814d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335d142640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3358158fe0 0x7f335814b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335c941640 1 -- 172.21.15.32:0/224681285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33580cb140 msgr2=0x7f335814d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335c941640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33580cb140 0x7f335814d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335c941640 1 -- 172.21.15.32:0/224681285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3358158fe0 msgr2=0x7f335814b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335c941640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3358158fe0 0x7f335814b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335c941640 1 -- 172.21.15.32:0/224681285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3350009b30 con 0x7f33580c9520 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335c941640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33580c9520 0x7f33580b2b00 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c7f0 tx=0x7f334c00ccb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f33557fa640 1 -- 172.21.15.32:0/224681285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c00cea0 con 0x7f33580c9520 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f33557fa640 1 -- 172.21.15.32:0/224681285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f334c007a30 con 0x7f33580c9520 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f33557fa640 1 -- 172.21.15.32:0/224681285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f334c007de0 con 0x7f33580c9520 2023-12-20T06:00:35.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335d943640 1 -- 172.21.15.32:0/224681285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f335814c660 con 0x7f33580c9520 2023-12-20T06:00:35.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.942+0000 7f335d943640 1 -- 172.21.15.32:0/224681285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f335814cc20 con 0x7f33580c9520 2023-12-20T06:00:35.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.946+0000 7f335d943640 1 -- 172.21.15.32:0/224681285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3358074100 con 0x7f33580c9520 2023-12-20T06:00:35.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.946+0000 7f33557fa640 1 -- 172.21.15.32:0/224681285 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f334c0113f0 con 0x7f33580c9520 2023-12-20T06:00:35.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.946+0000 7f33557fa640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f333c065bb0 0x7f333c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:35.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.946+0000 7f33557fa640 1 -- 172.21.15.32:0/224681285 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(188..188 src has 1..188) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f334c083270 con 0x7f33580c9520 2023-12-20T06:00:35.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.946+0000 7f3357fff640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f333c065bb0 0x7f333c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:35.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.946+0000 7f3357fff640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f333c065bb0 0x7f333c068070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33581f45d0 tx=0x7f3344008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:35.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:35.950+0000 7f33557fa640 1 -- 172.21.15.32:0/224681285 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f334c050bf0 con 0x7f33580c9520 2023-12-20T06:00:36.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.110+0000 7f335d943640 1 -- 172.21.15.32:0/224681285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f33580737e0 con 0x7f33580c9520 2023-12-20T06:00:36.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.186+0000 7f33557fa640 1 -- 172.21.15.32:0/224681285 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v189) v1 ==== 138+0+0 (secure 0 0 0) 0x7f334c054210 con 0x7f33580c9520 2023-12-20T06:00:36.188 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T06:00:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 -- 172.21.15.32:0/224681285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f333c065bb0 msgr2=0x7f333c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f333c065bb0 0x7f333c068070 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f33581f45d0 tx=0x7f3344008040 comp rx=0 tx=0).stop 2023-12-20T06:00:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 -- 172.21.15.32:0/224681285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33580c9520 msgr2=0x7f33580b2b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33580c9520 0x7f33580b2b00 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f334c00c7f0 tx=0x7f334c00ccb0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 -- 172.21.15.32:0/224681285 shutdown_connections 2023-12-20T06:00:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f333c065bb0 0x7f333c068070 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3358158fe0 0x7f335814b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33580cb140 0x7f335814d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 --2- 172.21.15.32:0/224681285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33580c9520 0x7f33580b2b00 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 -- 172.21.15.32:0/224681285 >> 172.21.15.32:0/224681285 conn(0x7f335806a4e0 msgr2=0x7f335814e600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 -- 172.21.15.32:0/224681285 shutdown_connections 2023-12-20T06:00:36.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.190+0000 7f335d943640 1 -- 172.21.15.32:0/224681285 wait complete. 2023-12-20T06:00:36.214 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T06:00:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbfc63640 1 Processor -- start 2023-12-20T06:00:36.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbfc63640 1 -- start start 2023-12-20T06:00:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbfc63640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efcb815a940 0x7efcb80cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbfc63640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efcb815ad20 0x7efcb80ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbfc63640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7efcb80cb1c0 0x7efcb80cb610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbfc63640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb80cbb50 con 0x7efcb815a940 2023-12-20T06:00:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbfc63640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efcb80cc9d0 con 0x7efcb80cb1c0 2023-12-20T06:00:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbfc63640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80c9b90 con 0x7efcb815ad20 2023-12-20T06:00:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbd9ff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efcb815a940 0x7efcb80cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbd9ff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efcb815a940 0x7efcb80cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60692/0 (socket says 172.21.15.32:60692) 2023-12-20T06:00:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbd9ff640 1 -- 172.21.15.32:0/3708153764 learned_addr learned my addr 172.21.15.32:0/3708153764 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:36.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbe200640 1 --2- 172.21.15.32:0/3708153764 >> v2:172.21.15.39:3300/0 conn(0x7efcb80cb1c0 0x7efcb80cb610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:36.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbd1fe640 1 --2- 172.21.15.32:0/3708153764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efcb815ad20 0x7efcb80ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:36.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbd9ff640 1 -- 172.21.15.32:0/3708153764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efcb815ad20 msgr2=0x7efcb80ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:36.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbd9ff640 1 --2- 172.21.15.32:0/3708153764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efcb815ad20 0x7efcb80ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbd9ff640 1 -- 172.21.15.32:0/3708153764 >> v2:172.21.15.39:3300/0 conn(0x7efcb80cb1c0 msgr2=0x7efcb80cb610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:36.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbd9ff640 1 --2- 172.21.15.32:0/3708153764 >> v2:172.21.15.39:3300/0 conn(0x7efcb80cb1c0 0x7efcb80cb610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.366+0000 7efcbd9ff640 1 -- 172.21.15.32:0/3708153764 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb80c9e70 con 0x7efcb815a940 2023-12-20T06:00:36.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbd9ff640 1 --2- 172.21.15.32:0/3708153764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb815a940 0x7efcb80cc380 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7efcb80cded0 tx=0x7efcb401c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9103512c9dadb83 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:36.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbc9fd640 1 -- 172.21.15.32:0/3708153764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb40aa5f0 con 0x7efcb815a940 2023-12-20T06:00:36.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbc9fd640 1 -- 172.21.15.32:0/3708153764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb40aa750 con 0x7efcb815a940 2023-12-20T06:00:36.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbc9fd640 1 -- 172.21.15.32:0/3708153764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb40aaae0 con 0x7efcb815a940 2023-12-20T06:00:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbfc63640 1 -- 172.21.15.32:0/3708153764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb815a940 msgr2=0x7efcb80cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbfc63640 1 --2- 172.21.15.32:0/3708153764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb815a940 0x7efcb80cc380 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7efcb80cded0 tx=0x7efcb401c4d0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbfc63640 1 -- 172.21.15.32:0/3708153764 shutdown_connections 2023-12-20T06:00:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbfc63640 1 --2- 172.21.15.32:0/3708153764 >> v2:172.21.15.39:3300/0 conn(0x7efcb80cb1c0 0x7efcb80cb610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbfc63640 1 --2- 172.21.15.32:0/3708153764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efcb815ad20 0x7efcb80ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbfc63640 1 --2- 172.21.15.32:0/3708153764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb815a940 0x7efcb80cc380 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbfc63640 1 -- 172.21.15.32:0/3708153764 >> 172.21.15.32:0/3708153764 conn(0x7efcb806a4b0 msgr2=0x7efcb80b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbfc63640 1 -- 172.21.15.32:0/3708153764 shutdown_connections 2023-12-20T06:00:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.370+0000 7efcbfc63640 1 -- 172.21.15.32:0/3708153764 wait complete. 2023-12-20T06:00:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbfc63640 1 Processor -- start 2023-12-20T06:00:36.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbfc63640 1 -- start start 2023-12-20T06:00:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbfc63640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb80cb1c0 0x7efcb80ad420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbfc63640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb815a940 0x7efcb81f49f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbfc63640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efcb815ad20 0x7efcb80c7640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbfc63640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80abb50 con 0x7efcb80cb1c0 2023-12-20T06:00:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbfc63640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80c7f00 con 0x7efcb815a940 2023-12-20T06:00:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbfc63640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efcb80c81e0 con 0x7efcb815ad20 2023-12-20T06:00:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbd9ff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb80cb1c0 0x7efcb80ad420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbd9ff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb80cb1c0 0x7efcb80ad420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60706/0 (socket says 172.21.15.32:60706) 2023-12-20T06:00:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbd9ff640 1 -- 172.21.15.32:0/85667380 learned_addr learned my addr 172.21.15.32:0/85667380 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbd1fe640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb815a940 0x7efcb81f49f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbe200640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efcb815ad20 0x7efcb80c7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbd9ff640 1 -- 172.21.15.32:0/85667380 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efcb815ad20 msgr2=0x7efcb80c7640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbd9ff640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efcb815ad20 0x7efcb80c7640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbd9ff640 1 -- 172.21.15.32:0/85667380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb815a940 msgr2=0x7efcb81f49f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbd9ff640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb815a940 0x7efcb81f49f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbd9ff640 1 -- 172.21.15.32:0/85667380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efcb40094d0 con 0x7efcb80cb1c0 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbd9ff640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb80cb1c0 0x7efcb80ad420 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7efcb401ab80 tx=0x7efcb401ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcae7fc640 1 -- 172.21.15.32:0/85667380 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb40bd5f0 con 0x7efcb80cb1c0 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcae7fc640 1 -- 172.21.15.32:0/85667380 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efcb40bd750 con 0x7efcb80cb1c0 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcae7fc640 1 -- 172.21.15.32:0/85667380 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efcb40bdae0 con 0x7efcb80cb1c0 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbfc63640 1 -- 172.21.15.32:0/85667380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efcb80c84c0 con 0x7efcb80cb1c0 2023-12-20T06:00:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.374+0000 7efcbfc63640 1 -- 172.21.15.32:0/85667380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efcb80c7b80 con 0x7efcb80cb1c0 2023-12-20T06:00:36.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.378+0000 7efcbfc63640 1 -- 172.21.15.32:0/85667380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efcb80c8700 con 0x7efcb80cb1c0 2023-12-20T06:00:36.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.378+0000 7efcae7fc640 1 -- 172.21.15.32:0/85667380 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7efcb40b33f0 con 0x7efcb80cb1c0 2023-12-20T06:00:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.378+0000 7efcae7fc640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efc9c065b80 0x7efc9c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.378+0000 7efcae7fc640 1 -- 172.21.15.32:0/85667380 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(189..189 src has 1..189) v4 ==== 5283+0+0 (secure 0 0 0) 0x7efcb40a7070 con 0x7efcb80cb1c0 2023-12-20T06:00:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.378+0000 7efcbd1fe640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efc9c065b80 0x7efc9c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:36.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.378+0000 7efcbd1fe640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efc9c065b80 0x7efc9c068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efcb81522c0 tx=0x7efca8009190 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:36.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.382+0000 7efcae7fc640 1 -- 172.21.15.32:0/85667380 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efcb40eb850 con 0x7efcb80cb1c0 2023-12-20T06:00:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:36.538+0000 7efcbfc63640 1 -- 172.21.15.32:0/85667380 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7efcb80c8320 con 0x7efcb80cb1c0 2023-12-20T06:00:37.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.194+0000 7efcae7fc640 1 -- 172.21.15.32:0/85667380 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v190) v1 ==== 170+0+0 (secure 0 0 0) 0x7efcb80c8320 con 0x7efcb80cb1c0 2023-12-20T06:00:37.195 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T06:00:37.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.202+0000 7efcbfc63640 1 -- 172.21.15.32:0/85667380 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efc9c065b80 msgr2=0x7efc9c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.202+0000 7efcbfc63640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efc9c065b80 0x7efc9c068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7efcb81522c0 tx=0x7efca8009190 comp rx=0 tx=0).stop 2023-12-20T06:00:37.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.202+0000 7efcbfc63640 1 -- 172.21.15.32:0/85667380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb80cb1c0 msgr2=0x7efcb80ad420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.202+0000 7efcbfc63640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb80cb1c0 0x7efcb80ad420 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7efcb401ab80 tx=0x7efcb401ad10 comp rx=0 tx=0).stop 2023-12-20T06:00:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.202+0000 7efcbfc63640 1 -- 172.21.15.32:0/85667380 shutdown_connections 2023-12-20T06:00:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.202+0000 7efcbfc63640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efc9c065b80 0x7efc9c068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.202+0000 7efcbfc63640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efcb815ad20 0x7efcb80c7640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.202+0000 7efcbfc63640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efcb815a940 0x7efcb81f49f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.202+0000 7efcbfc63640 1 --2- 172.21.15.32:0/85667380 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efcb80cb1c0 0x7efcb80ad420 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.202+0000 7efcbfc63640 1 -- 172.21.15.32:0/85667380 >> 172.21.15.32:0/85667380 conn(0x7efcb806a4b0 msgr2=0x7efcb806daa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:37.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.206+0000 7efcbfc63640 1 -- 172.21.15.32:0/85667380 shutdown_connections 2023-12-20T06:00:37.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.206+0000 7efcbfc63640 1 -- 172.21.15.32:0/85667380 wait complete. 2023-12-20T06:00:37.239 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T06:00:37.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.390+0000 7fb958ee5640 1 Processor -- start 2023-12-20T06:00:37.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.390+0000 7fb958ee5640 1 -- start start 2023-12-20T06:00:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.390+0000 7fb958ee5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb95415abb0 0x7fb9540caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.390+0000 7fb958ee5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb954070230 0x7fb954070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.390+0000 7fb958ee5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb954070bc0 con 0x7fb9540b0cb0 2023-12-20T06:00:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.390+0000 7fb958ee5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95406e160 con 0x7fb954070230 2023-12-20T06:00:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.390+0000 7fb958ee5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb95415b6a0 con 0x7fb95415abb0 2023-12-20T06:00:37.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.390+0000 7fb952d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb954070230 0x7fb954070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.390+0000 7fb952d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb954070230 0x7fb954070680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60720/0 (socket says 172.21.15.32:60720) 2023-12-20T06:00:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.390+0000 7fb952d9d640 1 -- 172.21.15.32:0/3393013354 learned_addr learned my addr 172.21.15.32:0/3393013354 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.390+0000 7fb95259c640 1 --2- 172.21.15.32:0/3393013354 >> v2:172.21.15.39:3300/0 conn(0x7fb95415abb0 0x7fb9540caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb952d9d640 1 -- 172.21.15.32:0/3393013354 >> v2:172.21.15.39:3300/0 conn(0x7fb95415abb0 msgr2=0x7fb9540caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb952d9d640 1 --2- 172.21.15.32:0/3393013354 >> v2:172.21.15.39:3300/0 conn(0x7fb95415abb0 0x7fb9540caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb952d9d640 1 -- 172.21.15.32:0/3393013354 >> v1:172.21.15.39:6789/0 conn(0x7fb9540b0cb0 legacy=0x7fb95415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb952d9d640 1 -- 172.21.15.32:0/3393013354 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb95406d8c0 con 0x7fb954070230 2023-12-20T06:00:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb952d9d640 1 --2- 172.21.15.32:0/3393013354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb954070230 0x7fb954070680 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fb9540713f0 tx=0x7fb94c01c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec31a6e7af006e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb951d9b640 1 -- 172.21.15.32:0/3393013354 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0a5890 con 0x7fb954070230 2023-12-20T06:00:37.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb951d9b640 1 -- 172.21.15.32:0/3393013354 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c0a59f0 con 0x7fb954070230 2023-12-20T06:00:37.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb951d9b640 1 -- 172.21.15.32:0/3393013354 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0a5d80 con 0x7fb954070230 2023-12-20T06:00:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb958ee5640 1 -- 172.21.15.32:0/3393013354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb954070230 msgr2=0x7fb954070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb958ee5640 1 --2- 172.21.15.32:0/3393013354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb954070230 0x7fb954070680 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7fb9540713f0 tx=0x7fb94c01c9d0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb958ee5640 1 -- 172.21.15.32:0/3393013354 shutdown_connections 2023-12-20T06:00:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb958ee5640 1 --2- 172.21.15.32:0/3393013354 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb954070230 0x7fb954070680 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb958ee5640 1 --2- 172.21.15.32:0/3393013354 >> v2:172.21.15.39:3300/0 conn(0x7fb95415abb0 0x7fb9540caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb958ee5640 1 -- 172.21.15.32:0/3393013354 >> 172.21.15.32:0/3393013354 conn(0x7fb95406a510 msgr2=0x7fb9540ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.394+0000 7fb958ee5640 1 -- 172.21.15.32:0/3393013354 shutdown_connections 2023-12-20T06:00:37.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb958ee5640 1 -- 172.21.15.32:0/3393013354 wait complete. 2023-12-20T06:00:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb958ee5640 1 Processor -- start 2023-12-20T06:00:37.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb958ee5640 1 -- start start 2023-12-20T06:00:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb958ee5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb954070230 0x7fb95414cd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb958ee5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9540b0cb0 0x7fb954138cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb958ee5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95415abb0 0x7fb954139230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb958ee5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb954139770 con 0x7fb95415abb0 2023-12-20T06:00:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb958ee5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9541482b0 con 0x7fb9540b0cb0 2023-12-20T06:00:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb958ee5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9541476a0 con 0x7fb954070230 2023-12-20T06:00:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb952d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95415abb0 0x7fb954139230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb952d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95415abb0 0x7fb954139230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60724/0 (socket says 172.21.15.32:60724) 2023-12-20T06:00:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb952d9d640 1 -- 172.21.15.32:0/2779712223 learned_addr learned my addr 172.21.15.32:0/2779712223 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb95259c640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb954070230 0x7fb95414cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb94bfff640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9540b0cb0 0x7fb954138cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb952d9d640 1 -- 172.21.15.32:0/2779712223 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb954070230 msgr2=0x7fb95414cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb952d9d640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb954070230 0x7fb95414cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb952d9d640 1 -- 172.21.15.32:0/2779712223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9540b0cb0 msgr2=0x7fb954138cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb952d9d640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9540b0cb0 0x7fb954138cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb952d9d640 1 -- 172.21.15.32:0/2779712223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb94c009b30 con 0x7fb95415abb0 2023-12-20T06:00:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.398+0000 7fb952d9d640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95415abb0 0x7fb954139230 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb94c01a600 tx=0x7fb94c019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.402+0000 7fb94affd640 1 -- 172.21.15.32:0/2779712223 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0b8670 con 0x7fb95415abb0 2023-12-20T06:00:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.402+0000 7fb94affd640 1 -- 172.21.15.32:0/2779712223 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb94c0b87d0 con 0x7fb95415abb0 2023-12-20T06:00:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.402+0000 7fb94affd640 1 -- 172.21.15.32:0/2779712223 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb94c0b8b80 con 0x7fb95415abb0 2023-12-20T06:00:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.402+0000 7fb958ee5640 1 -- 172.21.15.32:0/2779712223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb954147980 con 0x7fb95415abb0 2023-12-20T06:00:37.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.402+0000 7fb958ee5640 1 -- 172.21.15.32:0/2779712223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb954147f40 con 0x7fb95415abb0 2023-12-20T06:00:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.402+0000 7fb958ee5640 1 -- 172.21.15.32:0/2779712223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb95404e000 con 0x7fb95415abb0 2023-12-20T06:00:37.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.402+0000 7fb94affd640 1 -- 172.21.15.32:0/2779712223 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fb94c0b8ce0 con 0x7fb95415abb0 2023-12-20T06:00:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.406+0000 7fb94affd640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb934065b80 0x7fb934068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.406+0000 7fb94affd640 1 -- 172.21.15.32:0/2779712223 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(190..190 src has 1..190) v4 ==== 5665+0+0 (secure 0 0 0) 0x7fb94c11efc0 con 0x7fb95415abb0 2023-12-20T06:00:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.406+0000 7fb95259c640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb934065b80 0x7fb934068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.406+0000 7fb95259c640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb934065b80 0x7fb934068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb944009b60 tx=0x7fb944009190 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:37.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.406+0000 7fb94affd640 1 -- 172.21.15.32:0/2779712223 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb94c0eca20 con 0x7fb95415abb0 2023-12-20T06:00:37.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.598+0000 7fb958ee5640 1 -- 172.21.15.32:0/2779712223 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fb95415bde0 con 0x7fb95415abb0 2023-12-20T06:00:37.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.606+0000 7fb94affd640 1 -- 172.21.15.32:0/2779712223 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 22 and data pool 23 v83) v1 ==== 174+0+0 (secure 0 0 0) 0x7fb94c0f0040 con 0x7fb95415abb0 2023-12-20T06:00:37.610 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 22 and data pool 23 2023-12-20T06:00:37.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.614+0000 7fb958ee5640 1 -- 172.21.15.32:0/2779712223 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb934065b80 msgr2=0x7fb934068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.614+0000 7fb958ee5640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb934065b80 0x7fb934068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fb944009b60 tx=0x7fb944009190 comp rx=0 tx=0).stop 2023-12-20T06:00:37.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.614+0000 7fb958ee5640 1 -- 172.21.15.32:0/2779712223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95415abb0 msgr2=0x7fb954139230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.614+0000 7fb958ee5640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95415abb0 0x7fb954139230 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7fb94c01a600 tx=0x7fb94c019770 comp rx=0 tx=0).stop 2023-12-20T06:00:37.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.618+0000 7fb958ee5640 1 -- 172.21.15.32:0/2779712223 shutdown_connections 2023-12-20T06:00:37.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.618+0000 7fb958ee5640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb934065b80 0x7fb934068040 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.618+0000 7fb958ee5640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb95415abb0 0x7fb954139230 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.618+0000 7fb958ee5640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9540b0cb0 0x7fb954138cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.618+0000 7fb958ee5640 1 --2- 172.21.15.32:0/2779712223 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb954070230 0x7fb95414cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.618+0000 7fb958ee5640 1 -- 172.21.15.32:0/2779712223 >> 172.21.15.32:0/2779712223 conn(0x7fb95406a510 msgr2=0x7fb954150e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:37.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.622+0000 7fb958ee5640 1 -- 172.21.15.32:0/2779712223 shutdown_connections 2023-12-20T06:00:37.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.622+0000 7fb958ee5640 1 -- 172.21.15.32:0/2779712223 wait complete. 2023-12-20T06:00:37.653 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:00:37.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f7308c8c640 1 Processor -- start 2023-12-20T06:00:37.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f7308c8c640 1 -- start start 2023-12-20T06:00:37.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f7308c8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f73040af5e0 0x7f73040af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f7308c8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f730406ce60 0x7f730415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f7308c8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f730415bbd0 con 0x7f730415f110 2023-12-20T06:00:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f7308c8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730415beb0 con 0x7f730406ce60 2023-12-20T06:00:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f7308c8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f730415e910 con 0x7f73040af5e0 2023-12-20T06:00:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f72fbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f730406ce60 0x7f730415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f7302d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f730415f110 0x7f730415c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47562/0 (socket says 172.21.15.32:47562) 2023-12-20T06:00:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f7302d9d640 1 -- 172.21.15.32:0/690824643 learned_addr learned my addr 172.21.15.32:0/690824643 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f730259c640 1 --2- 172.21.15.32:0/690824643 >> v2:172.21.15.39:3300/0 conn(0x7f73040af5e0 0x7f73040af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f72fbfff640 1 -- 172.21.15.32:0/690824643 >> v2:172.21.15.39:3300/0 conn(0x7f73040af5e0 msgr2=0x7f73040af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f72fbfff640 1 --2- 172.21.15.32:0/690824643 >> v2:172.21.15.39:3300/0 conn(0x7f73040af5e0 0x7f73040af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f72fbfff640 1 -- 172.21.15.32:0/690824643 >> v1:172.21.15.32:6789/0 conn(0x7f730415f110 legacy=0x7f730415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f72fbfff640 1 -- 172.21.15.32:0/690824643 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f730415ebf0 con 0x7f730406ce60 2023-12-20T06:00:37.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.798+0000 7f7301d9b640 1 -- 172.21.15.32:0/690824643 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3073606408 0 0) 0x7f730415bbd0 con 0x7f730415f110 2023-12-20T06:00:37.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f72fbfff640 1 --2- 172.21.15.32:0/690824643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f730406ce60 0x7f730415b280 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f730406ac10 tx=0x7f72ec01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3cbafa7f3c7284d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:37.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f7301d9b640 1 -- 172.21.15.32:0/690824643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0ab600 con 0x7f730406ce60 2023-12-20T06:00:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f7301d9b640 1 -- 172.21.15.32:0/690824643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec0ab760 con 0x7f730406ce60 2023-12-20T06:00:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f7301d9b640 1 -- 172.21.15.32:0/690824643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0abaf0 con 0x7f730406ce60 2023-12-20T06:00:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f7308c8c640 1 -- 172.21.15.32:0/690824643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f730406ce60 msgr2=0x7f730415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f7308c8c640 1 --2- 172.21.15.32:0/690824643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f730406ce60 0x7f730415b280 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f730406ac10 tx=0x7f72ec01c460 comp rx=0 tx=0).stop 2023-12-20T06:00:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f7308c8c640 1 -- 172.21.15.32:0/690824643 shutdown_connections 2023-12-20T06:00:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f7308c8c640 1 --2- 172.21.15.32:0/690824643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f730406ce60 0x7f730415b280 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f7308c8c640 1 --2- 172.21.15.32:0/690824643 >> v2:172.21.15.39:3300/0 conn(0x7f73040af5e0 0x7f73040af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f7308c8c640 1 -- 172.21.15.32:0/690824643 >> 172.21.15.32:0/690824643 conn(0x7f730406a320 msgr2=0x7f73040ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f7308c8c640 1 -- 172.21.15.32:0/690824643 shutdown_connections 2023-12-20T06:00:37.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.802+0000 7f7308c8c640 1 -- 172.21.15.32:0/690824643 wait complete. 2023-12-20T06:00:37.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7308c8c640 1 Processor -- start 2023-12-20T06:00:37.805 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7308c8c640 1 -- start start 2023-12-20T06:00:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7308c8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730406ce60 0x7f7304165680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7308c8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73040af5e0 0x7f7304154d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7308c8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f730415f110 0x7f730414c360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7308c8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7304154930 con 0x7f73040af5e0 2023-12-20T06:00:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7308c8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f730415dde0 con 0x7f730406ce60 2023-12-20T06:00:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7308c8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7304153f90 con 0x7f730415f110 2023-12-20T06:00:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72fbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73040af5e0 0x7f7304154d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72fbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73040af5e0 0x7f7304154d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60738/0 (socket says 172.21.15.32:60738) 2023-12-20T06:00:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72fbfff640 1 -- 172.21.15.32:0/2954527976 learned_addr learned my addr 172.21.15.32:0/2954527976 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:37.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f730259c640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730406ce60 0x7f7304165680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7302d9d640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f730415f110 0x7f730414c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72fbfff640 1 -- 172.21.15.32:0/2954527976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f730415f110 msgr2=0x7f730414c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72fbfff640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f730415f110 0x7f730414c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72fbfff640 1 -- 172.21.15.32:0/2954527976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730406ce60 msgr2=0x7f7304165680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72fbfff640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730406ce60 0x7f7304165680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72fbfff640 1 -- 172.21.15.32:0/2954527976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72ec009420 con 0x7f73040af5e0 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7302d9d640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f730415f110 0x7f730414c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72fbfff640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73040af5e0 0x7f7304154d50 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f72ec01a090 tx=0x7f72ec01aca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72faffd640 1 -- 172.21.15.32:0/2954527976 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0bf610 con 0x7f73040af5e0 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72faffd640 1 -- 172.21.15.32:0/2954527976 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72ec0bf790 con 0x7f73040af5e0 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f72faffd640 1 -- 172.21.15.32:0/2954527976 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72ec0bfb40 con 0x7f73040af5e0 2023-12-20T06:00:37.809 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7308c8c640 1 -- 172.21.15.32:0/2954527976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7304154270 con 0x7f73040af5e0 2023-12-20T06:00:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.806+0000 7f7308c8c640 1 -- 172.21.15.32:0/2954527976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f730414c8a0 con 0x7f73040af5e0 2023-12-20T06:00:37.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.810+0000 7f7308c8c640 1 -- 172.21.15.32:0/2954527976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73041540d0 con 0x7f73040af5e0 2023-12-20T06:00:37.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.810+0000 7f72faffd640 1 -- 172.21.15.32:0/2954527976 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f72ec0b5400 con 0x7f73040af5e0 2023-12-20T06:00:37.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.810+0000 7f72faffd640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f72e4065b80 0x7f72e4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:37.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.810+0000 7f72faffd640 1 -- 172.21.15.32:0/2954527976 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f72ec11dfd0 con 0x7f73040af5e0 2023-12-20T06:00:37.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.810+0000 7f730259c640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f72e4065b80 0x7f72e4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:37.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.810+0000 7f730259c640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f72e4065b80 0x7f72e4068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009ad0 tx=0x7f72f4008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:37.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.814+0000 7f72faffd640 1 -- 172.21.15.32:0/2954527976 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f73041540d0 con 0x7f73040af5e0 2023-12-20T06:00:37.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 -- 172.21.15.32:0/2954527976 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f73041540d0 con 0x7f73040af5e0 2023-12-20T06:00:37.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f72faffd640 1 -- 172.21.15.32:0/2954527976 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+16158 (secure 0 0 0) 0x7f73041540d0 con 0x7f73040af5e0 2023-12-20T06:00:37.978 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:37.978 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":191,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:00:37.603764+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:00:36.162516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-20T06:00:36.593153+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:00:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 -- 172.21.15.32:0/2954527976 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f72e4065b80 msgr2=0x7f72e4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f72e4065b80 0x7f72e4068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f72f4009ad0 tx=0x7f72f4008040 comp rx=0 tx=0).stop 2023-12-20T06:00:37.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 -- 172.21.15.32:0/2954527976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73040af5e0 msgr2=0x7f7304154d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:37.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73040af5e0 0x7f7304154d50 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7f72ec01a090 tx=0x7f72ec01aca0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 -- 172.21.15.32:0/2954527976 shutdown_connections 2023-12-20T06:00:37.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f72e4065b80 0x7f72e4068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f730415f110 0x7f730414c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73040af5e0 0x7f7304154d50 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 --2- 172.21.15.32:0/2954527976 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f730406ce60 0x7f7304165680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:37.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 -- 172.21.15.32:0/2954527976 >> 172.21.15.32:0/2954527976 conn(0x7f730406a320 msgr2=0x7f7304156520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:37.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 -- 172.21.15.32:0/2954527976 shutdown_connections 2023-12-20T06:00:37.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:37.978+0000 7f7308c8c640 1 -- 172.21.15.32:0/2954527976 wait complete. 2023-12-20T06:00:38.000 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:00:38.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.142+0000 7f912ee2d640 1 Processor -- start 2023-12-20T06:00:38.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.142+0000 7f912ee2d640 1 -- start start 2023-12-20T06:00:38.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912ee2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912815ce60 0x7f91280c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912ee2d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f91280713e0 0x7f9128071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912ee2d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9128071d80 con 0x7f9128160b30 2023-12-20T06:00:38.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912ee2d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91281616d0 con 0x7f91280713e0 2023-12-20T06:00:38.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912ee2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91280c9a40 con 0x7f912815ce60 2023-12-20T06:00:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912d3ca640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9128160b30 0x7f9128160fc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47578/0 (socket says 172.21.15.32:47578) 2023-12-20T06:00:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912d3ca640 1 -- 172.21.15.32:0/2042695185 learned_addr learned my addr 172.21.15.32:0/2042695185 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912cbc9640 1 --2- 172.21.15.32:0/2042695185 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912815ce60 0x7f91280c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f9127fff640 1 --2- 172.21.15.32:0/2042695185 >> v2:172.21.15.39:3300/0 conn(0x7f91280713e0 0x7f9128071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f91277fe640 1 -- 172.21.15.32:0/2042695185 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3087187474 0 0) 0x7f9128071d80 con 0x7f9128160b30 2023-12-20T06:00:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f91277fe640 1 -- 172.21.15.32:0/2042695185 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9118003530 con 0x7f9128160b30 2023-12-20T06:00:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912cbc9640 1 -- 172.21.15.32:0/2042695185 >> v2:172.21.15.39:3300/0 conn(0x7f91280713e0 msgr2=0x7f9128071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912cbc9640 1 --2- 172.21.15.32:0/2042695185 >> v2:172.21.15.39:3300/0 conn(0x7f91280713e0 0x7f9128071840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912cbc9640 1 -- 172.21.15.32:0/2042695185 >> v1:172.21.15.32:6789/0 conn(0x7f9128160b30 legacy=0x7f9128160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:38.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.146+0000 7f912cbc9640 1 -- 172.21.15.32:0/2042695185 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91280c9180 con 0x7f912815ce60 2023-12-20T06:00:38.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f912cbc9640 1 --2- 172.21.15.32:0/2042695185 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912815ce60 0x7f91280c9ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f911c0097a0 tx=0x7f911c01c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bff762bf150d5bf5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:38.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f91277fe640 1 -- 172.21.15.32:0/2042695185 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c004230 con 0x7f912815ce60 2023-12-20T06:00:38.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f91277fe640 1 -- 172.21.15.32:0/2042695185 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c0a7050 con 0x7f912815ce60 2023-12-20T06:00:38.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f91277fe640 1 -- 172.21.15.32:0/2042695185 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c003620 con 0x7f912815ce60 2023-12-20T06:00:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f912ee2d640 1 -- 172.21.15.32:0/2042695185 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912815ce60 msgr2=0x7f91280c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f912ee2d640 1 --2- 172.21.15.32:0/2042695185 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912815ce60 0x7f91280c9ed0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f911c0097a0 tx=0x7f911c01c460 comp rx=0 tx=0).stop 2023-12-20T06:00:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f912ee2d640 1 -- 172.21.15.32:0/2042695185 shutdown_connections 2023-12-20T06:00:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f912ee2d640 1 --2- 172.21.15.32:0/2042695185 >> v2:172.21.15.39:3300/0 conn(0x7f91280713e0 0x7f9128071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f912ee2d640 1 --2- 172.21.15.32:0/2042695185 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f912815ce60 0x7f91280c9ed0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f912ee2d640 1 -- 172.21.15.32:0/2042695185 >> 172.21.15.32:0/2042695185 conn(0x7f912806a320 msgr2=0x7f91280ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f912ee2d640 1 -- 172.21.15.32:0/2042695185 shutdown_connections 2023-12-20T06:00:38.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f912ee2d640 1 -- 172.21.15.32:0/2042695185 wait complete. 2023-12-20T06:00:38.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f912ee2d640 1 Processor -- start 2023-12-20T06:00:38.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.150+0000 7f912ee2d640 1 -- start start 2023-12-20T06:00:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912ee2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91280713e0 0x7f912815a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912ee2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f912815ce60 0x7f9128153590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912ee2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9128160b30 0x7f912815ff20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912ee2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91281f89a0 con 0x7f91280713e0 2023-12-20T06:00:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912ee2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f912815a9a0 con 0x7f912815ce60 2023-12-20T06:00:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912ee2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f912815abb0 con 0x7f9128160b30 2023-12-20T06:00:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912cbc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91280713e0 0x7f912815a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912cbc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91280713e0 0x7f912815a460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60746/0 (socket says 172.21.15.32:60746) 2023-12-20T06:00:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912cbc9640 1 -- 172.21.15.32:0/303551195 learned_addr learned my addr 172.21.15.32:0/303551195 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:38.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f9127fff640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f912815ce60 0x7f9128153590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912cbc9640 1 -- 172.21.15.32:0/303551195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9128160b30 msgr2=0x7f912815ff20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912d3ca640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9128160b30 0x7f912815ff20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912cbc9640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9128160b30 0x7f912815ff20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912cbc9640 1 -- 172.21.15.32:0/303551195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f912815ce60 msgr2=0x7f9128153590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912cbc9640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f912815ce60 0x7f9128153590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912cbc9640 1 -- 172.21.15.32:0/303551195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f911c0093b0 con 0x7f91280713e0 2023-12-20T06:00:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f9127fff640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f912815ce60 0x7f9128153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912d3ca640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9128160b30 0x7f912815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912cbc9640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91280713e0 0x7f912815a460 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f911c01a090 tx=0x7f911c019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f91257fa640 1 -- 172.21.15.32:0/303551195 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c0bc070 con 0x7f91280713e0 2023-12-20T06:00:38.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f91257fa640 1 -- 172.21.15.32:0/303551195 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f911c0038f0 con 0x7f91280713e0 2023-12-20T06:00:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912ee2d640 1 -- 172.21.15.32:0/303551195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f91280c4880 con 0x7f91280713e0 2023-12-20T06:00:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f91257fa640 1 -- 172.21.15.32:0/303551195 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f911c003cc0 con 0x7f91280713e0 2023-12-20T06:00:38.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912ee2d640 1 -- 172.21.15.32:0/303551195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f91280c4e40 con 0x7f91280713e0 2023-12-20T06:00:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.154+0000 7f912ee2d640 1 -- 172.21.15.32:0/303551195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f912804e000 con 0x7f91280713e0 2023-12-20T06:00:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.158+0000 7f91257fa640 1 -- 172.21.15.32:0/303551195 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f911c0b2050 con 0x7f91280713e0 2023-12-20T06:00:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.158+0000 7f91257fa640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f910c065b80 0x7f910c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.158+0000 7f91257fa640 1 -- 172.21.15.32:0/303551195 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f911c11cc20 con 0x7f91280713e0 2023-12-20T06:00:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.158+0000 7f9127fff640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f910c065b80 0x7f910c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.158+0000 7f9127fff640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f910c065b80 0x7f910c068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f91280ce6f0 tx=0x7f9114008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:38.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.162+0000 7f91257fa640 1 -- 172.21.15.32:0/303551195 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f911c0ea370 con 0x7f91280713e0 2023-12-20T06:00:38.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.322+0000 7f912ee2d640 1 -- 172.21.15.32:0/303551195 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f91280cb9a0 con 0x7f91280713e0 2023-12-20T06:00:38.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.322+0000 7f91257fa640 1 -- 172.21.15.32:0/303551195 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v191) v1 ==== 74+0+16158 (secure 0 0 0) 0x7f911c0b0090 con 0x7f91280713e0 2023-12-20T06:00:38.323 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:38.323 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":191,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:00:37.603764+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:00:36.162516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-20T06:00:36.593153+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:00:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 -- 172.21.15.32:0/303551195 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f910c065b80 msgr2=0x7f910c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f910c065b80 0x7f910c068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f91280ce6f0 tx=0x7f9114008040 comp rx=0 tx=0).stop 2023-12-20T06:00:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 -- 172.21.15.32:0/303551195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91280713e0 msgr2=0x7f912815a460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91280713e0 0x7f912815a460 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f911c01a090 tx=0x7f911c019200 comp rx=0 tx=0).stop 2023-12-20T06:00:38.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 -- 172.21.15.32:0/303551195 shutdown_connections 2023-12-20T06:00:38.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f910c065b80 0x7f910c068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9128160b30 0x7f912815ff20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f912815ce60 0x7f9128153590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 --2- 172.21.15.32:0/303551195 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f91280713e0 0x7f912815a460 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 -- 172.21.15.32:0/303551195 >> 172.21.15.32:0/303551195 conn(0x7f912806a320 msgr2=0x7f9128071840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:38.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 -- 172.21.15.32:0/303551195 shutdown_connections 2023-12-20T06:00:38.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.326+0000 7f912ee2d640 1 -- 172.21.15.32:0/303551195 wait complete. 2023-12-20T06:00:38.344 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T06:00:38.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95fd18f640 1 Processor -- start 2023-12-20T06:00:38.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95fd18f640 1 -- start start 2023-12-20T06:00:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95fd18f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95f81591e0 0x7f95f80caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95fd18f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f95f806cdf0 0x7f95f8070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95fd18f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95f8070770 0x7f95f815cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95fd18f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f80cc3b0 con 0x7f95f81591e0 2023-12-20T06:00:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95fd18f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95f8070ba0 con 0x7f95f806cdf0 2023-12-20T06:00:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95fd18f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80c8e30 con 0x7f95f8070770 2023-12-20T06:00:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95f6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95f81591e0 0x7f95f80caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95f6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f95f81591e0 0x7f95f80caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60760/0 (socket says 172.21.15.32:60760) 2023-12-20T06:00:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95f6d9d640 1 -- 172.21.15.32:0/2121072544 learned_addr learned my addr 172.21.15.32:0/2121072544 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95f6d9d640 1 -- 172.21.15.32:0/2121072544 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95f8070770 msgr2=0x7f95f815cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95f759e640 1 --2- 172.21.15.32:0/2121072544 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95f8070770 0x7f95f815cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95effff640 1 --2- 172.21.15.32:0/2121072544 >> v2:172.21.15.39:3300/0 conn(0x7f95f806cdf0 0x7f95f8070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95f6d9d640 1 --2- 172.21.15.32:0/2121072544 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95f8070770 0x7f95f815cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95f6d9d640 1 -- 172.21.15.32:0/2121072544 >> v2:172.21.15.39:3300/0 conn(0x7f95f806cdf0 msgr2=0x7f95f8070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:38.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95f6d9d640 1 --2- 172.21.15.32:0/2121072544 >> v2:172.21.15.39:3300/0 conn(0x7f95f806cdf0 0x7f95f8070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95f6d9d640 1 -- 172.21.15.32:0/2121072544 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95f806d280 con 0x7f95f81591e0 2023-12-20T06:00:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95f759e640 1 --2- 172.21.15.32:0/2121072544 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95f8070770 0x7f95f815cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:38.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.494+0000 7f95effff640 1 --2- 172.21.15.32:0/2121072544 >> v2:172.21.15.39:3300/0 conn(0x7f95f806cdf0 0x7f95f8070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:38.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95f6d9d640 1 --2- 172.21.15.32:0/2121072544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f81591e0 0x7f95f80caa30 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f95f815de70 tx=0x7f95e801c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4624df29a4dc0d0b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:38.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95f659c640 1 -- 172.21.15.32:0/2121072544 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e80ac410 con 0x7f95f81591e0 2023-12-20T06:00:38.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95f659c640 1 -- 172.21.15.32:0/2121072544 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95e80ac570 con 0x7f95f81591e0 2023-12-20T06:00:38.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95f659c640 1 -- 172.21.15.32:0/2121072544 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95e80ac900 con 0x7f95f81591e0 2023-12-20T06:00:38.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95fd18f640 1 -- 172.21.15.32:0/2121072544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f81591e0 msgr2=0x7f95f80caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:38.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95fd18f640 1 --2- 172.21.15.32:0/2121072544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f81591e0 0x7f95f80caa30 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f95f815de70 tx=0x7f95e801c470 comp rx=0 tx=0).stop 2023-12-20T06:00:38.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95fd18f640 1 -- 172.21.15.32:0/2121072544 shutdown_connections 2023-12-20T06:00:38.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95fd18f640 1 --2- 172.21.15.32:0/2121072544 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95f8070770 0x7f95f815cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95fd18f640 1 --2- 172.21.15.32:0/2121072544 >> v2:172.21.15.39:3300/0 conn(0x7f95f806cdf0 0x7f95f8070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95fd18f640 1 --2- 172.21.15.32:0/2121072544 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f81591e0 0x7f95f80caa30 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95fd18f640 1 -- 172.21.15.32:0/2121072544 >> 172.21.15.32:0/2121072544 conn(0x7f95f806a510 msgr2=0x7f95f80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95fd18f640 1 -- 172.21.15.32:0/2121072544 shutdown_connections 2023-12-20T06:00:38.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.498+0000 7f95fd18f640 1 -- 172.21.15.32:0/2121072544 wait complete. 2023-12-20T06:00:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95fd18f640 1 Processor -- start 2023-12-20T06:00:38.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95fd18f640 1 -- start start 2023-12-20T06:00:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95fd18f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95f8070770 0x7f95f80b3ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95fd18f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81591e0 0x7f95f8157760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95fd18f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f8157ca0 0x7f95f8155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95fd18f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f8158150 con 0x7f95f8157ca0 2023-12-20T06:00:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95fd18f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95f80b4530 con 0x7f95f81591e0 2023-12-20T06:00:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95fd18f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95f813c4a0 con 0x7f95f8070770 2023-12-20T06:00:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95f759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f8157ca0 0x7f95f8155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95f6d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95f8070770 0x7f95f80b3ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95f759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f8157ca0 0x7f95f8155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60766/0 (socket says 172.21.15.32:60766) 2023-12-20T06:00:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95f759e640 1 -- 172.21.15.32:0/3643206552 learned_addr learned my addr 172.21.15.32:0/3643206552 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95effff640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81591e0 0x7f95f8157760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95f759e640 1 -- 172.21.15.32:0/3643206552 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95f8070770 msgr2=0x7f95f80b3ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:38.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95f759e640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95f8070770 0x7f95f80b3ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95f759e640 1 -- 172.21.15.32:0/3643206552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81591e0 msgr2=0x7f95f8157760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95f759e640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81591e0 0x7f95f8157760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95f759e640 1 -- 172.21.15.32:0/3643206552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95e80094d0 con 0x7f95f8157ca0 2023-12-20T06:00:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95f759e640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f8157ca0 0x7f95f8155950 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f95f000cf50 tx=0x7f95f000a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95ef7fe640 1 -- 172.21.15.32:0/3643206552 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f000d840 con 0x7f95f8157ca0 2023-12-20T06:00:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95ef7fe640 1 -- 172.21.15.32:0/3643206552 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95f000d9a0 con 0x7f95f8157ca0 2023-12-20T06:00:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95ef7fe640 1 -- 172.21.15.32:0/3643206552 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95f000dd50 con 0x7f95f8157ca0 2023-12-20T06:00:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95fd18f640 1 -- 172.21.15.32:0/3643206552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95f81571c0 con 0x7f95f8157ca0 2023-12-20T06:00:38.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.502+0000 7f95fd18f640 1 -- 172.21.15.32:0/3643206552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95f8156920 con 0x7f95f8157ca0 2023-12-20T06:00:38.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.506+0000 7f95fd18f640 1 -- 172.21.15.32:0/3643206552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95f80b1570 con 0x7f95f8157ca0 2023-12-20T06:00:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.506+0000 7f95ef7fe640 1 -- 172.21.15.32:0/3643206552 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f95f001e3b0 con 0x7f95f8157ca0 2023-12-20T06:00:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.506+0000 7f95ef7fe640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95d8065b80 0x7f95d8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.506+0000 7f95ef7fe640 1 -- 172.21.15.32:0/3643206552 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(191..191 src has 1..191) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f95f0083c80 con 0x7f95f8157ca0 2023-12-20T06:00:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.506+0000 7f95f6d9d640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95d8065b80 0x7f95d8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:38.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.506+0000 7f95f6d9d640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95d8065b80 0x7f95d8068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f95e80ae000 tx=0x7f95e8009040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:38.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.510+0000 7f95ef7fe640 1 -- 172.21.15.32:0/3643206552 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f95f0051630 con 0x7f95f8157ca0 2023-12-20T06:00:38.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:38.706+0000 7f95fd18f640 1 -- 172.21.15.32:0/3643206552 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f95f804e000 con 0x7f95f8157ca0 2023-12-20T06:00:39.176 INFO:tasks.ceph.mds.a.smithi032.stderr:2023-12-20T06:00:39.174+0000 7fbad3a72640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T06:00:39.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95ef7fe640 1 -- 172.21.15.32:0/3643206552 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v86) v1 ==== 118+0+0 (secure 0 0 0) 0x7f95f001a0b0 con 0x7f95f8157ca0 2023-12-20T06:00:39.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 -- 172.21.15.32:0/3643206552 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95d8065b80 msgr2=0x7f95d8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:39.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95d8065b80 0x7f95d8068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f95e80ae000 tx=0x7f95e8009040 comp rx=0 tx=0).stop 2023-12-20T06:00:39.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 -- 172.21.15.32:0/3643206552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f8157ca0 msgr2=0x7f95f8155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:39.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f8157ca0 0x7f95f8155950 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f95f000cf50 tx=0x7f95f000a7d0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 -- 172.21.15.32:0/3643206552 shutdown_connections 2023-12-20T06:00:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95d8065b80 0x7f95d8068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95f8157ca0 0x7f95f8155950 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95f81591e0 0x7f95f8157760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 --2- 172.21.15.32:0/3643206552 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95f8070770 0x7f95f80b3ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 -- 172.21.15.32:0/3643206552 >> 172.21.15.32:0/3643206552 conn(0x7f95f806a510 msgr2=0x7f95f80c39c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 -- 172.21.15.32:0/3643206552 shutdown_connections 2023-12-20T06:00:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.642+0000 7f95fd18f640 1 -- 172.21.15.32:0/3643206552 wait complete. 2023-12-20T06:00:39.667 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T06:00:39.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T06:00:39.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.818+0000 7fc4eafdf640 1 Processor -- start 2023-12-20T06:00:39.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.818+0000 7fc4eafdf640 1 -- start start 2023-12-20T06:00:39.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4eafdf640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc4e415ccf0 0x7fc4e40cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4eafdf640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e40cd7b0 0x7fc4e40cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4eafdf640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4e40cf920 0x7fc4e40b4490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4eafdf640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e40ce2f0 con 0x7fc4e40cf920 2023-12-20T06:00:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4eafdf640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc4e40d0b70 con 0x7fc4e415ccf0 2023-12-20T06:00:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4eafdf640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40cfd80 con 0x7fc4e40cd7b0 2023-12-20T06:00:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4ea7de640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4e40cf920 0x7fc4e40b4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4ea7de640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc4e40cf920 0x7fc4e40b4490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47298/0 (socket says 172.21.15.32:47298) 2023-12-20T06:00:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4ea7de640 1 -- 172.21.15.32:0/3505768670 learned_addr learned my addr 172.21.15.32:0/3505768670 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4ea7de640 1 -- 172.21.15.32:0/3505768670 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e40cd7b0 msgr2=0x7fc4e40cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4ea7de640 1 --2- 172.21.15.32:0/3505768670 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e40cd7b0 0x7fc4e40cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4ea7de640 1 -- 172.21.15.32:0/3505768670 >> v2:172.21.15.39:3300/0 conn(0x7fc4e415ccf0 msgr2=0x7fc4e40cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:39.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4ea7de640 1 --2- 172.21.15.32:0/3505768670 >> v2:172.21.15.39:3300/0 conn(0x7fc4e415ccf0 0x7fc4e40cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4ea7de640 1 -- 172.21.15.32:0/3505768670 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4e40d10f0 con 0x7fc4e40cf920 2023-12-20T06:00:39.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.822+0000 7fc4ea7de640 1 --2- 172.21.15.32:0/3505768670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e40cf920 0x7fc4e40b4490 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4066600 tx=0x7fc4dc01c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef5dd1e033a0f25 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:39.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4e8fdb640 1 -- 172.21.15.32:0/3505768670 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0a64e0 con 0x7fc4e40cf920 2023-12-20T06:00:39.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4e8fdb640 1 -- 172.21.15.32:0/3505768670 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4dc0a6640 con 0x7fc4e40cf920 2023-12-20T06:00:39.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4e8fdb640 1 -- 172.21.15.32:0/3505768670 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4dc0a69d0 con 0x7fc4e40cf920 2023-12-20T06:00:39.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 -- 172.21.15.32:0/3505768670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e40cf920 msgr2=0x7fc4e40b4490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:39.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 --2- 172.21.15.32:0/3505768670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e40cf920 0x7fc4e40b4490 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7fc4e4066600 tx=0x7fc4dc01c650 comp rx=0 tx=0).stop 2023-12-20T06:00:39.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 -- 172.21.15.32:0/3505768670 shutdown_connections 2023-12-20T06:00:39.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 --2- 172.21.15.32:0/3505768670 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e40cf920 0x7fc4e40b4490 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 --2- 172.21.15.32:0/3505768670 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e40cd7b0 0x7fc4e40cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 --2- 172.21.15.32:0/3505768670 >> v2:172.21.15.39:3300/0 conn(0x7fc4e415ccf0 0x7fc4e40cd270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 -- 172.21.15.32:0/3505768670 >> 172.21.15.32:0/3505768670 conn(0x7fc4e40658e0 msgr2=0x7fc4e4065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:39.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 -- 172.21.15.32:0/3505768670 shutdown_connections 2023-12-20T06:00:39.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 -- 172.21.15.32:0/3505768670 wait complete. 2023-12-20T06:00:39.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 Processor -- start 2023-12-20T06:00:39.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 -- start start 2023-12-20T06:00:39.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e40cd7b0 0x7fc4e4155780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:39.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e40cf920 0x7fc4e41509f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:39.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e415ccf0 0x7fc4e4150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:39.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e4066610 con 0x7fc4e40cf920 2023-12-20T06:00:39.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e40c6ae0 con 0x7fc4e40cd7b0 2023-12-20T06:00:39.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4eafdf640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc4e41515d0 con 0x7fc4e415ccf0 2023-12-20T06:00:39.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.826+0000 7fc4e97dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e40cf920 0x7fc4e41509f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:39.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4e97dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e40cf920 0x7fc4e41509f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47314/0 (socket says 172.21.15.32:47314) 2023-12-20T06:00:39.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4e97dc640 1 -- 172.21.15.32:0/2367650727 learned_addr learned my addr 172.21.15.32:0/2367650727 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:39.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4ea7de640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e415ccf0 0x7fc4e4150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:39.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4e9fdd640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e40cd7b0 0x7fc4e4155780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:39.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4e97dc640 1 -- 172.21.15.32:0/2367650727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e415ccf0 msgr2=0x7fc4e4150f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:39.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4e97dc640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e415ccf0 0x7fc4e4150f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4e97dc640 1 -- 172.21.15.32:0/2367650727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e40cd7b0 msgr2=0x7fc4e4155780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:39.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4e97dc640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e40cd7b0 0x7fc4e4155780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:39.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4e97dc640 1 -- 172.21.15.32:0/2367650727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc4dc009b30 con 0x7fc4e40cf920 2023-12-20T06:00:39.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4e97dc640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e40cf920 0x7fc4e41509f0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400b410 tx=0x7fc4d400b8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:39.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4da7fc640 1 -- 172.21.15.32:0/2367650727 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4011830 con 0x7fc4e40cf920 2023-12-20T06:00:39.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4da7fc640 1 -- 172.21.15.32:0/2367650727 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc4d4011990 con 0x7fc4e40cf920 2023-12-20T06:00:39.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4da7fc640 1 -- 172.21.15.32:0/2367650727 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc4d4011d40 con 0x7fc4e40cf920 2023-12-20T06:00:39.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4eafdf640 1 -- 172.21.15.32:0/2367650727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc4e40c6d60 con 0x7fc4e40cf920 2023-12-20T06:00:39.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4eafdf640 1 -- 172.21.15.32:0/2367650727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc4e40c7320 con 0x7fc4e40cf920 2023-12-20T06:00:39.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4eafdf640 1 -- 172.21.15.32:0/2367650727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc4e40c74e0 con 0x7fc4e40cf920 2023-12-20T06:00:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.830+0000 7fc4da7fc640 1 -- 172.21.15.32:0/2367650727 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc4d401a400 con 0x7fc4e40cf920 2023-12-20T06:00:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.834+0000 7fc4da7fc640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc4c8065bb0 0x7fc4c8068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.834+0000 7fc4da7fc640 1 -- 172.21.15.32:0/2367650727 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc4d4083460 con 0x7fc4e40cf920 2023-12-20T06:00:39.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.834+0000 7fc4e9fdd640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc4c8065bb0 0x7fc4c8068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:39.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.834+0000 7fc4e9fdd640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc4c8065bb0 0x7fc4c8068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009990 tx=0x7fc4e0009110 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:39.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:39.838+0000 7fc4da7fc640 1 -- 172.21.15.32:0/2367650727 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc4d4050c60 con 0x7fc4e40cf920 2023-12-20T06:00:40.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.030+0000 7fc4eafdf640 1 -- 172.21.15.32:0/2367650727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fc4e4074100 con 0x7fc4e40cf920 2023-12-20T06:00:40.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.650+0000 7fc4da7fc640 1 -- 172.21.15.32:0/2367650727 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v87) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc4d4054280 con 0x7fc4e40cf920 2023-12-20T06:00:40.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 -- 172.21.15.32:0/2367650727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc4c8065bb0 msgr2=0x7fc4c8068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:40.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc4c8065bb0 0x7fc4c8068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fc4e0009990 tx=0x7fc4e0009110 comp rx=0 tx=0).stop 2023-12-20T06:00:40.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 -- 172.21.15.32:0/2367650727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e40cf920 msgr2=0x7fc4e41509f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e40cf920 0x7fc4e41509f0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7fc4d400b410 tx=0x7fc4d400b8d0 comp rx=0 tx=0).stop 2023-12-20T06:00:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 -- 172.21.15.32:0/2367650727 shutdown_connections 2023-12-20T06:00:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc4c8065bb0 0x7fc4c8068070 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc4e415ccf0 0x7fc4e4150f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc4e40cf920 0x7fc4e41509f0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 --2- 172.21.15.32:0/2367650727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc4e40cd7b0 0x7fc4e4155780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 -- 172.21.15.32:0/2367650727 >> 172.21.15.32:0/2367650727 conn(0x7fc4e40658e0 msgr2=0x7fc4e40b4150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 -- 172.21.15.32:0/2367650727 shutdown_connections 2023-12-20T06:00:40.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.654+0000 7fc4eafdf640 1 -- 172.21.15.32:0/2367650727 wait complete. 2023-12-20T06:00:40.676 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T06:00:40.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:00:40.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f7282e6f640 1 Processor -- start 2023-12-20T06:00:40.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f7282e6f640 1 -- start start 2023-12-20T06:00:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f7282e6f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f727c0b8480 0x7f727c06af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f7282e6f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f727c161500 0x7f727c153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f7282e6f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f727c0cd4d0 con 0x7f727c06b460 2023-12-20T06:00:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f7282e6f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f727c161930 con 0x7f727c0b8480 2023-12-20T06:00:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f7282e6f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727c06b8f0 con 0x7f727c161500 2023-12-20T06:00:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f728166c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f727c06b460 0x7f727c160df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55568/0 (socket says 172.21.15.32:55568) 2023-12-20T06:00:40.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f728166c640 1 -- 172.21.15.32:0/3985458603 learned_addr learned my addr 172.21.15.32:0/3985458603 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f728266e640 1 --2- 172.21.15.32:0/3985458603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f727c161500 0x7f727c153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f7281e6d640 1 --2- 172.21.15.32:0/3985458603 >> v2:172.21.15.39:3300/0 conn(0x7f727c0b8480 0x7f727c06af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f7280e6b640 1 -- 172.21.15.32:0/3985458603 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1507264707 0 0) 0x7f727c0cd4d0 con 0x7f727c06b460 2023-12-20T06:00:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f7280e6b640 1 -- 172.21.15.32:0/3985458603 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7268003530 con 0x7f727c06b460 2023-12-20T06:00:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f728266e640 1 -- 172.21.15.32:0/3985458603 >> v2:172.21.15.39:3300/0 conn(0x7f727c0b8480 msgr2=0x7f727c06af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f728266e640 1 --2- 172.21.15.32:0/3985458603 >> v2:172.21.15.39:3300/0 conn(0x7f727c0b8480 0x7f727c06af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f728266e640 1 -- 172.21.15.32:0/3985458603 >> v1:172.21.15.32:6789/0 conn(0x7f727c06b460 legacy=0x7f727c160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:40.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.826+0000 7f728266e640 1 -- 172.21.15.32:0/3985458603 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f727c0ce4e0 con 0x7f727c161500 2023-12-20T06:00:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f728266e640 1 --2- 172.21.15.32:0/3985458603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f727c161500 0x7f727c153610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7274009d90 tx=0x7f727401c950 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f19846b4c236d946 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f7280e6b640 1 -- 172.21.15.32:0/3985458603 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72740a5900 con 0x7f727c161500 2023-12-20T06:00:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f7280e6b640 1 -- 172.21.15.32:0/3985458603 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72740a5a60 con 0x7f727c161500 2023-12-20T06:00:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f7280e6b640 1 -- 172.21.15.32:0/3985458603 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72740a5df0 con 0x7f727c161500 2023-12-20T06:00:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f7282e6f640 1 -- 172.21.15.32:0/3985458603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f727c161500 msgr2=0x7f727c153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f7282e6f640 1 --2- 172.21.15.32:0/3985458603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f727c161500 0x7f727c153610 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f7274009d90 tx=0x7f727401c950 comp rx=0 tx=0).stop 2023-12-20T06:00:40.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f7282e6f640 1 -- 172.21.15.32:0/3985458603 shutdown_connections 2023-12-20T06:00:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f7282e6f640 1 --2- 172.21.15.32:0/3985458603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f727c161500 0x7f727c153610 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f7282e6f640 1 --2- 172.21.15.32:0/3985458603 >> v2:172.21.15.39:3300/0 conn(0x7f727c0b8480 0x7f727c06af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f7282e6f640 1 -- 172.21.15.32:0/3985458603 >> 172.21.15.32:0/3985458603 conn(0x7f727c0d2a50 msgr2=0x7f727c0d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f7282e6f640 1 -- 172.21.15.32:0/3985458603 shutdown_connections 2023-12-20T06:00:40.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.830+0000 7f7282e6f640 1 -- 172.21.15.32:0/3985458603 wait complete. 2023-12-20T06:00:40.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7282e6f640 1 Processor -- start 2023-12-20T06:00:40.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7282e6f640 1 -- start start 2023-12-20T06:00:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7282e6f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f727c06b460 0x7f727c149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7282e6f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f727c0b8480 0x7f727c153370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:40.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7282e6f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f727c161500 0x7f727c151810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:40.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7282e6f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c0c12a0 con 0x7f727c0b8480 2023-12-20T06:00:40.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7282e6f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f727c0c1580 con 0x7f727c06b460 2023-12-20T06:00:40.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7282e6f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f727c0c1860 con 0x7f727c161500 2023-12-20T06:00:40.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f728166c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f727c0b8480 0x7f727c153370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:40.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f728166c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f727c0b8480 0x7f727c153370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47316/0 (socket says 172.21.15.32:47316) 2023-12-20T06:00:40.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f728166c640 1 -- 172.21.15.32:0/3276082447 learned_addr learned my addr 172.21.15.32:0/3276082447 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:40.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7281e6d640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f727c06b460 0x7f727c149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:40.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f728266e640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f727c161500 0x7f727c151810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:40.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f728166c640 1 -- 172.21.15.32:0/3276082447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f727c161500 msgr2=0x7f727c151810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:40.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f728166c640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f727c161500 0x7f727c151810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:40.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f728166c640 1 -- 172.21.15.32:0/3276082447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f727c06b460 msgr2=0x7f727c149d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f728166c640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f727c06b460 0x7f727c149d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f728166c640 1 -- 172.21.15.32:0/3276082447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7274009a70 con 0x7f727c0b8480 2023-12-20T06:00:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f728166c640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f727c0b8480 0x7f727c153370 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f726c00c4d0 tx=0x7f726c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f72727fc640 1 -- 172.21.15.32:0/3276082447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c005cf0 con 0x7f727c0b8480 2023-12-20T06:00:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f72727fc640 1 -- 172.21.15.32:0/3276082447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f726c005e50 con 0x7f727c0b8480 2023-12-20T06:00:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f72727fc640 1 -- 172.21.15.32:0/3276082447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f726c007270 con 0x7f727c0b8480 2023-12-20T06:00:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7282e6f640 1 -- 172.21.15.32:0/3276082447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f727c0c1b40 con 0x7f727c0b8480 2023-12-20T06:00:40.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7282e6f640 1 -- 172.21.15.32:0/3276082447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f727c1538b0 con 0x7f727c0b8480 2023-12-20T06:00:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.834+0000 7f7282e6f640 1 -- 172.21.15.32:0/3276082447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f727c074100 con 0x7f727c0b8480 2023-12-20T06:00:40.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.838+0000 7f72727fc640 1 -- 172.21.15.32:0/3276082447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f726c0073d0 con 0x7f727c0b8480 2023-12-20T06:00:40.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.838+0000 7f72727fc640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7260065b80 0x7f7260068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:40.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.838+0000 7f72727fc640 1 -- 172.21.15.32:0/3276082447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f726c082950 con 0x7f727c0b8480 2023-12-20T06:00:40.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.838+0000 7f7281e6d640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7260065b80 0x7f7260068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:40.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.838+0000 7f7281e6d640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7260065b80 0x7f7260068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f727c0ced00 tx=0x7f7278009190 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:40.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:40.842+0000 7f72727fc640 1 -- 172.21.15.32:0/3276082447 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f726c0501d0 con 0x7f727c0b8480 2023-12-20T06:00:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.030+0000 7f7282e6f640 1 -- 172.21.15.32:0/3276082447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f727c073750 con 0x7f727c0b8480 2023-12-20T06:00:41.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.030+0000 7f72727fc640 1 -- 172.21.15.32:0/3276082447 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f726c017020 con 0x7f727c0b8480 2023-12-20T06:00:41.031 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:41.031 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8396,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1626919510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1626919510},{"type":"v1","addr":"172.21.15.39:6833","nonce":1626919510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3001408763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3001408763},{"type":"v1","addr":"172.21.15.39:6835","nonce":3001408763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":10020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/2908145202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2908145202},{"type":"v1","addr":"172.21.15.32:6835","nonce":2908145202}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:00:37.604080+0000","modified":"2023-12-20T06:00:40.642479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10026":{"gid":10026,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2253884246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2253884246},{"type":"v1","addr":"172.21.15.32:6837","nonce":2253884246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-20T06:00:41.031 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 87 2023-12-20T06:00:41.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.030+0000 7f7282e6f640 1 -- 172.21.15.32:0/3276082447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7260065b80 msgr2=0x7f7260068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.030+0000 7f7282e6f640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7260065b80 0x7f7260068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f727c0ced00 tx=0x7f7278009190 comp rx=0 tx=0).stop 2023-12-20T06:00:41.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.030+0000 7f7282e6f640 1 -- 172.21.15.32:0/3276082447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f727c0b8480 msgr2=0x7f727c153370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.030+0000 7f7282e6f640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f727c0b8480 0x7f727c153370 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7f726c00c4d0 tx=0x7f726c00c990 comp rx=0 tx=0).stop 2023-12-20T06:00:41.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.030+0000 7f7282e6f640 1 -- 172.21.15.32:0/3276082447 shutdown_connections 2023-12-20T06:00:41.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.034+0000 7f7282e6f640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7260065b80 0x7f7260068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.034+0000 7f7282e6f640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f727c161500 0x7f727c151810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.034+0000 7f7282e6f640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f727c0b8480 0x7f727c153370 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.034+0000 7f7282e6f640 1 --2- 172.21.15.32:0/3276082447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f727c06b460 0x7f727c149d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.034+0000 7f7282e6f640 1 -- 172.21.15.32:0/3276082447 >> 172.21.15.32:0/3276082447 conn(0x7f727c0d2a50 msgr2=0x7f727c0d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:41.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.034+0000 7f7282e6f640 1 -- 172.21.15.32:0/3276082447 shutdown_connections 2023-12-20T06:00:41.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.034+0000 7f7282e6f640 1 -- 172.21.15.32:0/3276082447 wait complete. 2023-12-20T06:00:41.052 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:00:41.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.194+0000 7fd247152640 1 Processor -- start 2023-12-20T06:00:41.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.194+0000 7fd247152640 1 -- start start 2023-12-20T06:00:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd247152640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd24006ce60 0x7fd24015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd247152640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24015bbd0 con 0x7fd24015f110 2023-12-20T06:00:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd247152640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd24015beb0 con 0x7fd2400af5e0 2023-12-20T06:00:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd247152640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd24015e910 con 0x7fd24006ce60 2023-12-20T06:00:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd23ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd24006ce60 0x7fd24015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd23ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd24006ce60 0x7fd24015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47330/0 (socket says 172.21.15.32:47330) 2023-12-20T06:00:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd2456ef640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd24015f110 0x7fd24015c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55572/0 (socket says 172.21.15.32:55572) 2023-12-20T06:00:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd23ffff640 1 -- 172.21.15.32:0/264738897 learned_addr learned my addr 172.21.15.32:0/264738897 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd23ffff640 1 -- 172.21.15.32:0/264738897 >> v1:172.21.15.39:6789/0 conn(0x7fd2400af5e0 legacy=0x7fd2400af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd23ffff640 1 -- 172.21.15.32:0/264738897 >> v1:172.21.15.32:6789/0 conn(0x7fd24015f110 legacy=0x7fd24015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd23ffff640 1 -- 172.21.15.32:0/264738897 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd24015ebf0 con 0x7fd24006ce60 2023-12-20T06:00:41.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd23ffff640 1 --2- 172.21.15.32:0/264738897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24006ce60 0x7fd24015b280 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009870 tx=0x7fd22c01c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da73851ac284bb5f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:41.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd23f7fe640 1 -- 172.21.15.32:0/264738897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd24015b9d0 con 0x7fd24006ce60 2023-12-20T06:00:41.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd23f7fe640 1 -- 172.21.15.32:0/264738897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd22c0ab650 con 0x7fd24006ce60 2023-12-20T06:00:41.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.198+0000 7fd23f7fe640 1 -- 172.21.15.32:0/264738897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd22c0ab9e0 con 0x7fd24006ce60 2023-12-20T06:00:41.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 -- 172.21.15.32:0/264738897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24006ce60 msgr2=0x7fd24015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 --2- 172.21.15.32:0/264738897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24006ce60 0x7fd24015b280 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fd22c009870 tx=0x7fd22c01c4b0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 -- 172.21.15.32:0/264738897 shutdown_connections 2023-12-20T06:00:41.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 --2- 172.21.15.32:0/264738897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24006ce60 0x7fd24015b280 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 -- 172.21.15.32:0/264738897 >> 172.21.15.32:0/264738897 conn(0x7fd24006a320 msgr2=0x7fd2400ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:41.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 -- 172.21.15.32:0/264738897 shutdown_connections 2023-12-20T06:00:41.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 -- 172.21.15.32:0/264738897 wait complete. 2023-12-20T06:00:41.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 Processor -- start 2023-12-20T06:00:41.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 -- start start 2023-12-20T06:00:41.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24006ce60 0x7fd240150e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2400af5e0 0x7fd2400701c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd24015f110 0x7fd2401493e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2401513d0 con 0x7fd24006ce60 2023-12-20T06:00:41.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd2400717b0 con 0x7fd2400af5e0 2023-12-20T06:00:41.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd247152640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd240070ba0 con 0x7fd24015f110 2023-12-20T06:00:41.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd244eee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24006ce60 0x7fd240150e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd244eee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24006ce60 0x7fd240150e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47338/0 (socket says 172.21.15.32:47338) 2023-12-20T06:00:41.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd244eee640 1 -- 172.21.15.32:0/3140828925 learned_addr learned my addr 172.21.15.32:0/3140828925 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:41.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.202+0000 7fd2456ef640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd24015f110 0x7fd2401493e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd244eee640 1 -- 172.21.15.32:0/3140828925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd24015f110 msgr2=0x7fd2401493e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd23ffff640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2400af5e0 0x7fd2400701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd244eee640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd24015f110 0x7fd2401493e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd244eee640 1 -- 172.21.15.32:0/3140828925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2400af5e0 msgr2=0x7fd2400701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd244eee640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2400af5e0 0x7fd2400701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd244eee640 1 -- 172.21.15.32:0/3140828925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd22c0094d0 con 0x7fd24006ce60 2023-12-20T06:00:41.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd2456ef640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd24015f110 0x7fd2401493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:41.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd23ffff640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2400af5e0 0x7fd2400701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:41.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd244eee640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24006ce60 0x7fd240150e90 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c480 tx=0x7fd23400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:41.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd23d7fa640 1 -- 172.21.15.32:0/3140828925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd2340052d0 con 0x7fd24006ce60 2023-12-20T06:00:41.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd23d7fa640 1 -- 172.21.15.32:0/3140828925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd234005430 con 0x7fd24006ce60 2023-12-20T06:00:41.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd23d7fa640 1 -- 172.21.15.32:0/3140828925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd234003890 con 0x7fd24006ce60 2023-12-20T06:00:41.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd247152640 1 -- 172.21.15.32:0/3140828925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd240070e80 con 0x7fd24006ce60 2023-12-20T06:00:41.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd247152640 1 -- 172.21.15.32:0/3140828925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd240071440 con 0x7fd24006ce60 2023-12-20T06:00:41.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd247152640 1 -- 172.21.15.32:0/3140828925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd24004e000 con 0x7fd24006ce60 2023-12-20T06:00:41.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd23d7fa640 1 -- 172.21.15.32:0/3140828925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd2340039f0 con 0x7fd24006ce60 2023-12-20T06:00:41.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd23d7fa640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd224065bb0 0x7fd224068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd23d7fa640 1 -- 172.21.15.32:0/3140828925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd234082f00 con 0x7fd24006ce60 2023-12-20T06:00:41.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.206+0000 7fd23ffff640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd224065bb0 0x7fd224068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.210+0000 7fd23ffff640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd224065bb0 0x7fd224068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0af000 tx=0x7fd22c008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:41.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.210+0000 7fd23d7fa640 1 -- 172.21.15.32:0/3140828925 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd234050760 con 0x7fd24006ce60 2023-12-20T06:00:41.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.370+0000 7fd247152640 1 -- 172.21.15.32:0/3140828925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd2400cf450 con 0x7fd24006ce60 2023-12-20T06:00:41.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.370+0000 7fd23d7fa640 1 -- 172.21.15.32:0/3140828925 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v192) v1 ==== 74+0+16145 (secure 0 0 0) 0x7fd234053d80 con 0x7fd24006ce60 2023-12-20T06:00:41.371 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:41.372 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":192,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:00:38.607141+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:00:36.162516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-20T06:00:36.593153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:00:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 -- 172.21.15.32:0/3140828925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd224065bb0 msgr2=0x7fd224068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd224065bb0 0x7fd224068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fd22c0af000 tx=0x7fd22c008040 comp rx=0 tx=0).stop 2023-12-20T06:00:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 -- 172.21.15.32:0/3140828925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24006ce60 msgr2=0x7fd240150e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24006ce60 0x7fd240150e90 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fd23400c480 tx=0x7fd23400c940 comp rx=0 tx=0).stop 2023-12-20T06:00:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 -- 172.21.15.32:0/3140828925 shutdown_connections 2023-12-20T06:00:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd224065bb0 0x7fd224068070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd24015f110 0x7fd2401493e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd2400af5e0 0x7fd2400701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 --2- 172.21.15.32:0/3140828925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd24006ce60 0x7fd240150e90 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 -- 172.21.15.32:0/3140828925 >> 172.21.15.32:0/3140828925 conn(0x7fd24006a320 msgr2=0x7fd240156c80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 -- 172.21.15.32:0/3140828925 shutdown_connections 2023-12-20T06:00:41.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.374+0000 7fd247152640 1 -- 172.21.15.32:0/3140828925 wait complete. 2023-12-20T06:00:41.393 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T06:00:41.393 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:00:41.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f408dc2d640 1 Processor -- start 2023-12-20T06:00:41.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f408dc2d640 1 -- start start 2023-12-20T06:00:41.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f408dc2d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40880d2740 0x7f40880cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f408dc2d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40880cfec0 con 0x7f40880cfab0 2023-12-20T06:00:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f408dc2d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f408815d190 con 0x7f408815cdb0 2023-12-20T06:00:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f408dc2d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40880d16a0 con 0x7f40880d2740 2023-12-20T06:00:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f4086ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40880d2740 0x7f40880cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f4087fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f40880cfab0 0x7f40880b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55574/0 (socket says 172.21.15.32:55574) 2023-12-20T06:00:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f4087fff640 1 -- 172.21.15.32:0/4152784967 learned_addr learned my addr 172.21.15.32:0/4152784967 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f4086ffd640 1 -- 172.21.15.32:0/4152784967 >> v1:172.21.15.39:6789/0 conn(0x7f408815cdb0 legacy=0x7f40880d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f4086ffd640 1 -- 172.21.15.32:0/4152784967 >> v1:172.21.15.32:6789/0 conn(0x7f40880cfab0 legacy=0x7f40880b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f40867fc640 1 -- 172.21.15.32:0/4152784967 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3089896452 0 0) 0x7f40880cfec0 con 0x7f40880cfab0 2023-12-20T06:00:41.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.538+0000 7f4086ffd640 1 -- 172.21.15.32:0/4152784967 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40880d1980 con 0x7f40880d2740 2023-12-20T06:00:41.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.542+0000 7f4086ffd640 1 --2- 172.21.15.32:0/4152784967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40880d2740 0x7f40880cf570 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f407c0097c0 tx=0x7f407c01ec30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8db6b510f57a66d2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.542+0000 7f40867fc640 1 -- 172.21.15.32:0/4152784967 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c0a8930 con 0x7f40880d2740 2023-12-20T06:00:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.542+0000 7f40867fc640 1 -- 172.21.15.32:0/4152784967 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f407c0a8a90 con 0x7f40880d2740 2023-12-20T06:00:41.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.542+0000 7f40867fc640 1 -- 172.21.15.32:0/4152784967 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f407c0a8e20 con 0x7f40880d2740 2023-12-20T06:00:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.542+0000 7f408dc2d640 1 -- 172.21.15.32:0/4152784967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40880d2740 msgr2=0x7f40880cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.542+0000 7f408dc2d640 1 --2- 172.21.15.32:0/4152784967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40880d2740 0x7f40880cf570 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7f407c0097c0 tx=0x7f407c01ec30 comp rx=0 tx=0).stop 2023-12-20T06:00:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.542+0000 7f408dc2d640 1 -- 172.21.15.32:0/4152784967 shutdown_connections 2023-12-20T06:00:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.542+0000 7f408dc2d640 1 --2- 172.21.15.32:0/4152784967 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40880d2740 0x7f40880cf570 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.542+0000 7f408dc2d640 1 -- 172.21.15.32:0/4152784967 >> 172.21.15.32:0/4152784967 conn(0x7f4088065ee0 msgr2=0x7f408806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.542+0000 7f408dc2d640 1 -- 172.21.15.32:0/4152784967 shutdown_connections 2023-12-20T06:00:41.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.542+0000 7f408dc2d640 1 -- 172.21.15.32:0/4152784967 wait complete. 2023-12-20T06:00:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f408dc2d640 1 Processor -- start 2023-12-20T06:00:41.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f408dc2d640 1 -- start start 2023-12-20T06:00:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f408dc2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40880cfab0 0x7f40880c8530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f408dc2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40880d2740 0x7f4088156ee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f408dc2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408815cdb0 0x7f408815e250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f408dc2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f408815edc0 con 0x7f408815cdb0 2023-12-20T06:00:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f408dc2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4088156a50 con 0x7f40880d2740 2023-12-20T06:00:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f408dc2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4088156120 con 0x7f40880cfab0 2023-12-20T06:00:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f4087fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408815cdb0 0x7f408815e250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f4087fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408815cdb0 0x7f408815e250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47356/0 (socket says 172.21.15.32:47356) 2023-12-20T06:00:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f4087fff640 1 -- 172.21.15.32:0/3618653589 learned_addr learned my addr 172.21.15.32:0/3618653589 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:41.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f40877fe640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40880cfab0 0x7f40880c8530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f4086ffd640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40880d2740 0x7f4088156ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f4087fff640 1 -- 172.21.15.32:0/3618653589 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40880cfab0 msgr2=0x7f40880c8530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f4087fff640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40880cfab0 0x7f40880c8530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f4087fff640 1 -- 172.21.15.32:0/3618653589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40880d2740 msgr2=0x7f4088156ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f4087fff640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40880d2740 0x7f4088156ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f4087fff640 1 -- 172.21.15.32:0/3618653589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f407c009420 con 0x7f408815cdb0 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f40877fe640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40880cfab0 0x7f40880c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f4087fff640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408815cdb0 0x7f408815e250 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f408000c810 tx=0x7f408000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f406ffff640 1 -- 172.21.15.32:0/3618653589 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408000a640 con 0x7f408815cdb0 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f406ffff640 1 -- 172.21.15.32:0/3618653589 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f408000a7c0 con 0x7f408815cdb0 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f406ffff640 1 -- 172.21.15.32:0/3618653589 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f408000ab70 con 0x7f408815cdb0 2023-12-20T06:00:41.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f408dc2d640 1 -- 172.21.15.32:0/3618653589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4088156400 con 0x7f408815cdb0 2023-12-20T06:00:41.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.546+0000 7f408dc2d640 1 -- 172.21.15.32:0/3618653589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4088142670 con 0x7f408815cdb0 2023-12-20T06:00:41.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.550+0000 7f408dc2d640 1 -- 172.21.15.32:0/3618653589 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4088156be0 con 0x7f408815cdb0 2023-12-20T06:00:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.550+0000 7f406ffff640 1 -- 172.21.15.32:0/3618653589 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f408000acd0 con 0x7f408815cdb0 2023-12-20T06:00:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.550+0000 7f406ffff640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4074065b80 0x7f4074068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.550+0000 7f406ffff640 1 -- 172.21.15.32:0/3618653589 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f4080082820 con 0x7f408815cdb0 2023-12-20T06:00:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.550+0000 7f40877fe640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4074065b80 0x7f4074068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.550+0000 7f40877fe640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4074065b80 0x7f4074068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4078001740 tx=0x7f4078009040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:41.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.554+0000 7f406ffff640 1 -- 172.21.15.32:0/3618653589 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4080050080 con 0x7f408815cdb0 2023-12-20T06:00:41.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.710+0000 7f408dc2d640 1 -- 172.21.15.32:0/3618653589 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f40880d1f40 con 0x7f4074065b80 2023-12-20T06:00:41.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.718+0000 7f406ffff640 1 -- 172.21.15.32:0/3618653589 <== mgr.9786 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+285720 (secure 0 0 0) 0x7f4078001330 con 0x7f4074065b80 2023-12-20T06:00:41.722 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:41.724 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:00:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 -- 172.21.15.32:0/3618653589 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4074065b80 msgr2=0x7f4074068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4074065b80 0x7f4074068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4078001740 tx=0x7f4078009040 comp rx=0 tx=0).stop 2023-12-20T06:00:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 -- 172.21.15.32:0/3618653589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408815cdb0 msgr2=0x7f408815e250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408815cdb0 0x7f408815e250 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f408000c810 tx=0x7f408000ccd0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 -- 172.21.15.32:0/3618653589 shutdown_connections 2023-12-20T06:00:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4074065b80 0x7f4074068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f408815cdb0 0x7f408815e250 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40880d2740 0x7f4088156ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 --2- 172.21.15.32:0/3618653589 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40880cfab0 0x7f40880c8530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 -- 172.21.15.32:0/3618653589 >> 172.21.15.32:0/3618653589 conn(0x7f4088065ee0 msgr2=0x7f40880cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:41.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 -- 172.21.15.32:0/3618653589 shutdown_connections 2023-12-20T06:00:41.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.722+0000 7f408dc2d640 1 -- 172.21.15.32:0/3618653589 wait complete. 2023-12-20T06:00:41.741 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-20T06:00:39.904138+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":591166,"num_objects":15,"num_object_clones":0,"num_object_copies":30,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":15,"num_whiteouts":0,"num_read":1693,"num_read_kb":3437,"num_write":507,"num_write_kb":2427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3202,"ondisk_log_size":3202,"up":148,"acting":148,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":95,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":6,"kb":754974720,"kb_used":398652,"kb_used_data":19836,"kb_used_omap":0,"kb_used_meta":378752,"kb_avail":754576068,"statfs":{"total":773094113280,"available":772685893632,"internally_reserved":0,"allocated":20312064,"data_stored":10406196,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":387842048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":24,"apply_latency_ms":24,"commit_latency_ns":24000000,"apply_latency_ns":24000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":798,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.000577"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.638264+0000","last_change":"2023-12-20T06:00:37.650130+0000","last_active":"2023-12-20T06:00:38.638264+0000","last_peered":"2023-12-20T06:00:38.638264+0000","last_clean":"2023-12-20T06:00:38.638264+0000","last_became_active":"2023-12-20T06:00:37.648977+0000","last_became_peered":"2023-12-20T06:00:37.648977+0000","last_unstale":"2023-12-20T06:00:38.638264+0000","last_undegraded":"2023-12-20T06:00:38.638264+0000","last_fullsized":"2023-12-20T06:00:38.638264+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:05:01.260698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145292+0000","last_change":"2023-12-20T06:00:37.647806+0000","last_active":"2023-12-20T06:00:39.145292+0000","last_peered":"2023-12-20T06:00:39.145292+0000","last_clean":"2023-12-20T06:00:39.145292+0000","last_became_active":"2023-12-20T06:00:37.646357+0000","last_became_peered":"2023-12-20T06:00:37.646357+0000","last_unstale":"2023-12-20T06:00:39.145292+0000","last_undegraded":"2023-12-20T06:00:39.145292+0000","last_fullsized":"2023-12-20T06:00:39.145292+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:23:48.223962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148224+0000","last_change":"2023-12-20T06:00:37.636127+0000","last_active":"2023-12-20T06:00:39.148224+0000","last_peered":"2023-12-20T06:00:39.148224+0000","last_clean":"2023-12-20T06:00:39.148224+0000","last_became_active":"2023-12-20T06:00:37.635841+0000","last_became_peered":"2023-12-20T06:00:37.635841+0000","last_unstale":"2023-12-20T06:00:39.148224+0000","last_undegraded":"2023-12-20T06:00:39.148224+0000","last_fullsized":"2023-12-20T06:00:39.148224+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:38.916609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148047+0000","last_change":"2023-12-20T06:00:37.628715+0000","last_active":"2023-12-20T06:00:39.148047+0000","last_peered":"2023-12-20T06:00:39.148047+0000","last_clean":"2023-12-20T06:00:39.148047+0000","last_became_active":"2023-12-20T06:00:37.627720+0000","last_became_peered":"2023-12-20T06:00:37.627720+0000","last_unstale":"2023-12-20T06:00:39.148047+0000","last_undegraded":"2023-12-20T06:00:39.148047+0000","last_fullsized":"2023-12-20T06:00:39.148047+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:17:17.747988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626810+0000","last_change":"2023-12-20T06:00:37.625525+0000","last_active":"2023-12-20T06:00:38.626810+0000","last_peered":"2023-12-20T06:00:38.626810+0000","last_clean":"2023-12-20T06:00:38.626810+0000","last_became_active":"2023-12-20T06:00:37.625268+0000","last_became_peered":"2023-12-20T06:00:37.625268+0000","last_unstale":"2023-12-20T06:00:38.626810+0000","last_undegraded":"2023-12-20T06:00:38.626810+0000","last_fullsized":"2023-12-20T06:00:38.626810+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:49:26.463498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145385+0000","last_change":"2023-12-20T06:00:37.641497+0000","last_active":"2023-12-20T06:00:39.145385+0000","last_peered":"2023-12-20T06:00:39.145385+0000","last_clean":"2023-12-20T06:00:39.145385+0000","last_became_active":"2023-12-20T06:00:37.641213+0000","last_became_peered":"2023-12-20T06:00:37.641213+0000","last_unstale":"2023-12-20T06:00:39.145385+0000","last_undegraded":"2023-12-20T06:00:39.145385+0000","last_fullsized":"2023-12-20T06:00:39.145385+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:25:46.547193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145133+0000","last_change":"2023-12-20T06:00:37.639889+0000","last_active":"2023-12-20T06:00:39.145133+0000","last_peered":"2023-12-20T06:00:39.145133+0000","last_clean":"2023-12-20T06:00:39.145133+0000","last_became_active":"2023-12-20T06:00:37.639631+0000","last_became_peered":"2023-12-20T06:00:37.639631+0000","last_unstale":"2023-12-20T06:00:39.145133+0000","last_undegraded":"2023-12-20T06:00:39.145133+0000","last_fullsized":"2023-12-20T06:00:39.145133+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:00:08.769947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637519+0000","last_change":"2023-12-20T06:00:37.644413+0000","last_active":"2023-12-20T06:00:38.637519+0000","last_peered":"2023-12-20T06:00:38.637519+0000","last_clean":"2023-12-20T06:00:38.637519+0000","last_became_active":"2023-12-20T06:00:37.644092+0000","last_became_peered":"2023-12-20T06:00:37.644092+0000","last_unstale":"2023-12-20T06:00:38.637519+0000","last_undegraded":"2023-12-20T06:00:38.637519+0000","last_fullsized":"2023-12-20T06:00:38.637519+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:44:59.830860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626598+0000","last_change":"2023-12-20T06:00:37.641854+0000","last_active":"2023-12-20T06:00:38.626598+0000","last_peered":"2023-12-20T06:00:38.626598+0000","last_clean":"2023-12-20T06:00:38.626598+0000","last_became_active":"2023-12-20T06:00:37.641576+0000","last_became_peered":"2023-12-20T06:00:37.641576+0000","last_unstale":"2023-12-20T06:00:38.626598+0000","last_undegraded":"2023-12-20T06:00:38.626598+0000","last_fullsized":"2023-12-20T06:00:38.626598+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:35.973628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148729+0000","last_change":"2023-12-20T06:00:37.643279+0000","last_active":"2023-12-20T06:00:39.148729+0000","last_peered":"2023-12-20T06:00:39.148729+0000","last_clean":"2023-12-20T06:00:39.148729+0000","last_became_active":"2023-12-20T06:00:37.642926+0000","last_became_peered":"2023-12-20T06:00:37.642926+0000","last_unstale":"2023-12-20T06:00:39.148729+0000","last_undegraded":"2023-12-20T06:00:39.148729+0000","last_fullsized":"2023-12-20T06:00:39.148729+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:29:32.185935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694509+0000","last_change":"2023-12-20T06:00:37.634685+0000","last_active":"2023-12-20T06:00:38.694509+0000","last_peered":"2023-12-20T06:00:38.694509+0000","last_clean":"2023-12-20T06:00:38.694509+0000","last_became_active":"2023-12-20T06:00:37.630879+0000","last_became_peered":"2023-12-20T06:00:37.630879+0000","last_unstale":"2023-12-20T06:00:38.694509+0000","last_undegraded":"2023-12-20T06:00:38.694509+0000","last_fullsized":"2023-12-20T06:00:38.694509+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:39:17.523069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148492+0000","last_change":"2023-12-20T06:00:37.631473+0000","last_active":"2023-12-20T06:00:39.148492+0000","last_peered":"2023-12-20T06:00:39.148492+0000","last_clean":"2023-12-20T06:00:39.148492+0000","last_became_active":"2023-12-20T06:00:37.631218+0000","last_became_peered":"2023-12-20T06:00:37.631218+0000","last_unstale":"2023-12-20T06:00:39.148492+0000","last_undegraded":"2023-12-20T06:00:39.148492+0000","last_fullsized":"2023-12-20T06:00:39.148492+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:15:49.306568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145273+0000","last_change":"2023-12-20T06:00:37.642916+0000","last_active":"2023-12-20T06:00:39.145273+0000","last_peered":"2023-12-20T06:00:39.145273+0000","last_clean":"2023-12-20T06:00:39.145273+0000","last_became_active":"2023-12-20T06:00:37.642539+0000","last_became_peered":"2023-12-20T06:00:37.642539+0000","last_unstale":"2023-12-20T06:00:39.145273+0000","last_undegraded":"2023-12-20T06:00:39.145273+0000","last_fullsized":"2023-12-20T06:00:39.145273+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:11:34.374073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148284+0000","last_change":"2023-12-20T06:00:37.646485+0000","last_active":"2023-12-20T06:00:39.148284+0000","last_peered":"2023-12-20T06:00:39.148284+0000","last_clean":"2023-12-20T06:00:39.148284+0000","last_became_active":"2023-12-20T06:00:37.646179+0000","last_became_peered":"2023-12-20T06:00:37.646179+0000","last_unstale":"2023-12-20T06:00:39.148284+0000","last_undegraded":"2023-12-20T06:00:39.148284+0000","last_fullsized":"2023-12-20T06:00:39.148284+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:27:45.042505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147743+0000","last_change":"2023-12-20T06:00:37.623838+0000","last_active":"2023-12-20T06:00:39.147743+0000","last_peered":"2023-12-20T06:00:39.147743+0000","last_clean":"2023-12-20T06:00:39.147743+0000","last_became_active":"2023-12-20T06:00:37.623556+0000","last_became_peered":"2023-12-20T06:00:37.623556+0000","last_unstale":"2023-12-20T06:00:39.147743+0000","last_undegraded":"2023-12-20T06:00:39.147743+0000","last_fullsized":"2023-12-20T06:00:39.147743+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:05:34.079438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145599+0000","last_change":"2023-12-20T06:00:37.638096+0000","last_active":"2023-12-20T06:00:39.145599+0000","last_peered":"2023-12-20T06:00:39.145599+0000","last_clean":"2023-12-20T06:00:39.145599+0000","last_became_active":"2023-12-20T06:00:37.637427+0000","last_became_peered":"2023-12-20T06:00:37.637427+0000","last_unstale":"2023-12-20T06:00:39.145599+0000","last_undegraded":"2023-12-20T06:00:39.145599+0000","last_fullsized":"2023-12-20T06:00:39.145599+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:16.250610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626924+0000","last_change":"2023-12-20T06:00:37.630248+0000","last_active":"2023-12-20T06:00:38.626924+0000","last_peered":"2023-12-20T06:00:38.626924+0000","last_clean":"2023-12-20T06:00:38.626924+0000","last_became_active":"2023-12-20T06:00:37.628304+0000","last_became_peered":"2023-12-20T06:00:37.628304+0000","last_unstale":"2023-12-20T06:00:38.626924+0000","last_undegraded":"2023-12-20T06:00:38.626924+0000","last_fullsized":"2023-12-20T06:00:38.626924+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:59:18.162783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145386+0000","last_change":"2023-12-20T06:00:37.642946+0000","last_active":"2023-12-20T06:00:39.145386+0000","last_peered":"2023-12-20T06:00:39.145386+0000","last_clean":"2023-12-20T06:00:39.145386+0000","last_became_active":"2023-12-20T06:00:37.642617+0000","last_became_peered":"2023-12-20T06:00:37.642617+0000","last_unstale":"2023-12-20T06:00:39.145386+0000","last_undegraded":"2023-12-20T06:00:39.145386+0000","last_fullsized":"2023-12-20T06:00:39.145386+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:53:17.598471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148264+0000","last_change":"2023-12-20T06:00:37.634016+0000","last_active":"2023-12-20T06:00:39.148264+0000","last_peered":"2023-12-20T06:00:39.148264+0000","last_clean":"2023-12-20T06:00:39.148264+0000","last_became_active":"2023-12-20T06:00:37.633656+0000","last_became_peered":"2023-12-20T06:00:37.633656+0000","last_unstale":"2023-12-20T06:00:39.148264+0000","last_undegraded":"2023-12-20T06:00:39.148264+0000","last_fullsized":"2023-12-20T06:00:39.148264+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:29.959480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145485+0000","last_change":"2023-12-20T06:00:37.637830+0000","last_active":"2023-12-20T06:00:39.145485+0000","last_peered":"2023-12-20T06:00:39.145485+0000","last_clean":"2023-12-20T06:00:39.145485+0000","last_became_active":"2023-12-20T06:00:37.637500+0000","last_became_peered":"2023-12-20T06:00:37.637500+0000","last_unstale":"2023-12-20T06:00:39.145485+0000","last_undegraded":"2023-12-20T06:00:39.145485+0000","last_fullsized":"2023-12-20T06:00:39.145485+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:37:11.971643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694133+0000","last_change":"2023-12-20T06:00:37.621414+0000","last_active":"2023-12-20T06:00:38.694133+0000","last_peered":"2023-12-20T06:00:38.694133+0000","last_clean":"2023-12-20T06:00:38.694133+0000","last_became_active":"2023-12-20T06:00:37.621143+0000","last_became_peered":"2023-12-20T06:00:37.621143+0000","last_unstale":"2023-12-20T06:00:38.694133+0000","last_undegraded":"2023-12-20T06:00:38.694133+0000","last_fullsized":"2023-12-20T06:00:38.694133+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:28:20.337729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.627008+0000","last_change":"2023-12-20T06:00:37.626762+0000","last_active":"2023-12-20T06:00:38.627008+0000","last_peered":"2023-12-20T06:00:38.627008+0000","last_clean":"2023-12-20T06:00:38.627008+0000","last_became_active":"2023-12-20T06:00:37.626477+0000","last_became_peered":"2023-12-20T06:00:37.626477+0000","last_unstale":"2023-12-20T06:00:38.627008+0000","last_undegraded":"2023-12-20T06:00:38.627008+0000","last_fullsized":"2023-12-20T06:00:38.627008+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:20:13.190832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"191'4","reported_seq":15,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.638296+0000","last_change":"2023-12-20T06:00:37.214338+0000","last_active":"2023-12-20T06:00:38.638296+0000","last_peered":"2023-12-20T06:00:38.638296+0000","last_clean":"2023-12-20T06:00:38.638296+0000","last_became_active":"2023-12-20T06:00:37.214033+0000","last_became_peered":"2023-12-20T06:00:37.214033+0000","last_unstale":"2023-12-20T06:00:38.638296+0000","last_undegraded":"2023-12-20T06:00:38.638296+0000","last_fullsized":"2023-12-20T06:00:38.638296+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:21:50.035966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.1e","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:36.176550+0000","last_change":"2023-12-20T06:00:36.176550+0000","last_active":"2023-12-20T06:00:36.176550+0000","last_peered":"2023-12-20T06:00:36.176550+0000","last_clean":"2023-12-20T06:00:36.176550+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:36.176550+0000","last_undegraded":"2023-12-20T06:00:36.176550+0000","last_fullsized":"2023-12-20T06:00:36.176550+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148237+0000","last_change":"2023-12-20T06:00:37.645573+0000","last_active":"2023-12-20T06:00:39.148237+0000","last_peered":"2023-12-20T06:00:39.148237+0000","last_clean":"2023-12-20T06:00:39.148237+0000","last_became_active":"2023-12-20T06:00:37.645119+0000","last_became_peered":"2023-12-20T06:00:37.645119+0000","last_unstale":"2023-12-20T06:00:39.148237+0000","last_undegraded":"2023-12-20T06:00:39.148237+0000","last_fullsized":"2023-12-20T06:00:39.148237+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:35:46.496105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148500+0000","last_change":"2023-12-20T06:00:37.226172+0000","last_active":"2023-12-20T06:00:39.148500+0000","last_peered":"2023-12-20T06:00:39.148500+0000","last_clean":"2023-12-20T06:00:39.148500+0000","last_became_active":"2023-12-20T06:00:37.225913+0000","last_became_peered":"2023-12-20T06:00:37.225913+0000","last_unstale":"2023-12-20T06:00:39.148500+0000","last_undegraded":"2023-12-20T06:00:39.148500+0000","last_fullsized":"2023-12-20T06:00:39.148500+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:51.779317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145378+0000","last_change":"2023-12-20T06:00:37.642560+0000","last_active":"2023-12-20T06:00:39.145378+0000","last_peered":"2023-12-20T06:00:39.145378+0000","last_clean":"2023-12-20T06:00:39.145378+0000","last_became_active":"2023-12-20T06:00:37.642221+0000","last_became_peered":"2023-12-20T06:00:37.642221+0000","last_unstale":"2023-12-20T06:00:39.145378+0000","last_undegraded":"2023-12-20T06:00:39.145378+0000","last_fullsized":"2023-12-20T06:00:39.145378+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:39:00.341777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148397+0000","last_change":"2023-12-20T06:00:37.226118+0000","last_active":"2023-12-20T06:00:39.148397+0000","last_peered":"2023-12-20T06:00:39.148397+0000","last_clean":"2023-12-20T06:00:39.148397+0000","last_became_active":"2023-12-20T06:00:37.225814+0000","last_became_peered":"2023-12-20T06:00:37.225814+0000","last_unstale":"2023-12-20T06:00:39.148397+0000","last_undegraded":"2023-12-20T06:00:39.148397+0000","last_fullsized":"2023-12-20T06:00:39.148397+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:58:58.520032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626821+0000","last_change":"2023-12-20T06:00:37.621509+0000","last_active":"2023-12-20T06:00:38.626821+0000","last_peered":"2023-12-20T06:00:38.626821+0000","last_clean":"2023-12-20T06:00:38.626821+0000","last_became_active":"2023-12-20T06:00:37.621229+0000","last_became_peered":"2023-12-20T06:00:37.621229+0000","last_unstale":"2023-12-20T06:00:38.626821+0000","last_undegraded":"2023-12-20T06:00:38.626821+0000","last_fullsized":"2023-12-20T06:00:38.626821+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:01:14.788578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"191'1","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145504+0000","last_change":"2023-12-20T06:00:37.219345+0000","last_active":"2023-12-20T06:00:39.145504+0000","last_peered":"2023-12-20T06:00:39.145504+0000","last_clean":"2023-12-20T06:00:39.145504+0000","last_became_active":"2023-12-20T06:00:37.219038+0000","last_became_peered":"2023-12-20T06:00:37.219038+0000","last_unstale":"2023-12-20T06:00:39.145504+0000","last_undegraded":"2023-12-20T06:00:39.145504+0000","last_fullsized":"2023-12-20T06:00:39.145504+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:40.016860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147960+0000","last_change":"2023-12-20T06:00:37.628271+0000","last_active":"2023-12-20T06:00:39.147960+0000","last_peered":"2023-12-20T06:00:39.147960+0000","last_clean":"2023-12-20T06:00:39.147960+0000","last_became_active":"2023-12-20T06:00:37.627989+0000","last_became_peered":"2023-12-20T06:00:37.627989+0000","last_unstale":"2023-12-20T06:00:39.147960+0000","last_undegraded":"2023-12-20T06:00:39.147960+0000","last_fullsized":"2023-12-20T06:00:39.147960+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:16.250610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"191'1","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147858+0000","last_change":"2023-12-20T06:00:37.211925+0000","last_active":"2023-12-20T06:00:39.147858+0000","last_peered":"2023-12-20T06:00:39.147858+0000","last_clean":"2023-12-20T06:00:39.147858+0000","last_became_active":"2023-12-20T06:00:37.211620+0000","last_became_peered":"2023-12-20T06:00:37.211620+0000","last_unstale":"2023-12-20T06:00:39.147858+0000","last_undegraded":"2023-12-20T06:00:39.147858+0000","last_fullsized":"2023-12-20T06:00:39.147858+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:16.422503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147814+0000","last_change":"2023-12-20T06:00:37.641305+0000","last_active":"2023-12-20T06:00:39.147814+0000","last_peered":"2023-12-20T06:00:39.147814+0000","last_clean":"2023-12-20T06:00:39.147814+0000","last_became_active":"2023-12-20T06:00:37.641031+0000","last_became_peered":"2023-12-20T06:00:37.641031+0000","last_unstale":"2023-12-20T06:00:39.147814+0000","last_undegraded":"2023-12-20T06:00:39.147814+0000","last_fullsized":"2023-12-20T06:00:39.147814+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:33:35.691395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:36.176550+0000","last_change":"2023-12-20T06:00:36.176550+0000","last_active":"2023-12-20T06:00:36.176550+0000","last_peered":"2023-12-20T06:00:36.176550+0000","last_clean":"2023-12-20T06:00:36.176550+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:36.176550+0000","last_undegraded":"2023-12-20T06:00:36.176550+0000","last_fullsized":"2023-12-20T06:00:36.176550+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147839+0000","last_change":"2023-12-20T06:00:37.628030+0000","last_active":"2023-12-20T06:00:39.147839+0000","last_peered":"2023-12-20T06:00:39.147839+0000","last_clean":"2023-12-20T06:00:39.147839+0000","last_became_active":"2023-12-20T06:00:37.627776+0000","last_became_peered":"2023-12-20T06:00:37.627776+0000","last_unstale":"2023-12-20T06:00:39.147839+0000","last_undegraded":"2023-12-20T06:00:39.147839+0000","last_fullsized":"2023-12-20T06:00:39.147839+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:17:06.990952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626689+0000","last_change":"2023-12-20T06:00:37.216185+0000","last_active":"2023-12-20T06:00:38.626689+0000","last_peered":"2023-12-20T06:00:38.626689+0000","last_clean":"2023-12-20T06:00:38.626689+0000","last_became_active":"2023-12-20T06:00:37.215850+0000","last_became_peered":"2023-12-20T06:00:37.215850+0000","last_unstale":"2023-12-20T06:00:38.626689+0000","last_undegraded":"2023-12-20T06:00:38.626689+0000","last_fullsized":"2023-12-20T06:00:38.626689+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:28:19.325527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694533+0000","last_change":"2023-12-20T06:00:37.638082+0000","last_active":"2023-12-20T06:00:38.694533+0000","last_peered":"2023-12-20T06:00:38.694533+0000","last_clean":"2023-12-20T06:00:38.694533+0000","last_became_active":"2023-12-20T06:00:37.637827+0000","last_became_peered":"2023-12-20T06:00:37.637827+0000","last_unstale":"2023-12-20T06:00:38.694533+0000","last_undegraded":"2023-12-20T06:00:38.694533+0000","last_fullsized":"2023-12-20T06:00:38.694533+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:04:39.343446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148724+0000","last_change":"2023-12-20T06:00:37.222429+0000","last_active":"2023-12-20T06:00:39.148724+0000","last_peered":"2023-12-20T06:00:39.148724+0000","last_clean":"2023-12-20T06:00:39.148724+0000","last_became_active":"2023-12-20T06:00:37.222093+0000","last_became_peered":"2023-12-20T06:00:37.222093+0000","last_unstale":"2023-12-20T06:00:39.148724+0000","last_undegraded":"2023-12-20T06:00:39.148724+0000","last_fullsized":"2023-12-20T06:00:39.148724+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:41:43.772223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694368+0000","last_change":"2023-12-20T06:00:37.638142+0000","last_active":"2023-12-20T06:00:38.694368+0000","last_peered":"2023-12-20T06:00:38.694368+0000","last_clean":"2023-12-20T06:00:38.694368+0000","last_became_active":"2023-12-20T06:00:37.637872+0000","last_became_peered":"2023-12-20T06:00:37.637872+0000","last_unstale":"2023-12-20T06:00:38.694368+0000","last_undegraded":"2023-12-20T06:00:38.694368+0000","last_fullsized":"2023-12-20T06:00:38.694368+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:14.126316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"191'3","reported_seq":14,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148390+0000","last_change":"2023-12-20T06:00:37.218644+0000","last_active":"2023-12-20T06:00:39.148390+0000","last_peered":"2023-12-20T06:00:39.148390+0000","last_clean":"2023-12-20T06:00:39.148390+0000","last_became_active":"2023-12-20T06:00:37.218339+0000","last_became_peered":"2023-12-20T06:00:37.218339+0000","last_unstale":"2023-12-20T06:00:39.148390+0000","last_undegraded":"2023-12-20T06:00:39.148390+0000","last_fullsized":"2023-12-20T06:00:39.148390+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:02.720110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.14","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:36.176550+0000","last_change":"2023-12-20T06:00:36.176550+0000","last_active":"2023-12-20T06:00:36.176550+0000","last_peered":"2023-12-20T06:00:36.176550+0000","last_clean":"2023-12-20T06:00:36.176550+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:36.176550+0000","last_undegraded":"2023-12-20T06:00:36.176550+0000","last_fullsized":"2023-12-20T06:00:36.176550+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694240+0000","last_change":"2023-12-20T06:00:37.626217+0000","last_active":"2023-12-20T06:00:38.694240+0000","last_peered":"2023-12-20T06:00:38.694240+0000","last_clean":"2023-12-20T06:00:38.694240+0000","last_became_active":"2023-12-20T06:00:37.625927+0000","last_became_peered":"2023-12-20T06:00:37.625927+0000","last_unstale":"2023-12-20T06:00:38.694240+0000","last_undegraded":"2023-12-20T06:00:38.694240+0000","last_fullsized":"2023-12-20T06:00:38.694240+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:16:18.403896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148052+0000","last_change":"2023-12-20T06:00:37.215547+0000","last_active":"2023-12-20T06:00:39.148052+0000","last_peered":"2023-12-20T06:00:39.148052+0000","last_clean":"2023-12-20T06:00:39.148052+0000","last_became_active":"2023-12-20T06:00:37.214866+0000","last_became_peered":"2023-12-20T06:00:37.214866+0000","last_unstale":"2023-12-20T06:00:39.148052+0000","last_undegraded":"2023-12-20T06:00:39.148052+0000","last_fullsized":"2023-12-20T06:00:39.148052+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:21:44.604741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626964+0000","last_change":"2023-12-20T06:00:37.643805+0000","last_active":"2023-12-20T06:00:38.626964+0000","last_peered":"2023-12-20T06:00:38.626964+0000","last_clean":"2023-12-20T06:00:38.626964+0000","last_became_active":"2023-12-20T06:00:37.643534+0000","last_became_peered":"2023-12-20T06:00:37.643534+0000","last_unstale":"2023-12-20T06:00:38.626964+0000","last_undegraded":"2023-12-20T06:00:38.626964+0000","last_fullsized":"2023-12-20T06:00:38.626964+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:22:08.693939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694340+0000","last_change":"2023-12-20T06:00:37.219599+0000","last_active":"2023-12-20T06:00:38.694340+0000","last_peered":"2023-12-20T06:00:38.694340+0000","last_clean":"2023-12-20T06:00:38.694340+0000","last_became_active":"2023-12-20T06:00:37.219331+0000","last_became_peered":"2023-12-20T06:00:37.219331+0000","last_unstale":"2023-12-20T06:00:38.694340+0000","last_undegraded":"2023-12-20T06:00:38.694340+0000","last_fullsized":"2023-12-20T06:00:38.694340+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:46:38.781709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147829+0000","last_change":"2023-12-20T06:00:37.641860+0000","last_active":"2023-12-20T06:00:39.147829+0000","last_peered":"2023-12-20T06:00:39.147829+0000","last_clean":"2023-12-20T06:00:39.147829+0000","last_became_active":"2023-12-20T06:00:37.641580+0000","last_became_peered":"2023-12-20T06:00:37.641580+0000","last_unstale":"2023-12-20T06:00:39.147829+0000","last_undegraded":"2023-12-20T06:00:39.147829+0000","last_fullsized":"2023-12-20T06:00:39.147829+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:17:33.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147954+0000","last_change":"2023-12-20T06:00:37.224918+0000","last_active":"2023-12-20T06:00:39.147954+0000","last_peered":"2023-12-20T06:00:39.147954+0000","last_clean":"2023-12-20T06:00:39.147954+0000","last_became_active":"2023-12-20T06:00:37.224414+0000","last_became_peered":"2023-12-20T06:00:37.224414+0000","last_unstale":"2023-12-20T06:00:39.147954+0000","last_undegraded":"2023-12-20T06:00:39.147954+0000","last_fullsized":"2023-12-20T06:00:39.147954+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:40:42.440314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145370+0000","last_change":"2023-12-20T06:00:37.635333+0000","last_active":"2023-12-20T06:00:39.145370+0000","last_peered":"2023-12-20T06:00:39.145370+0000","last_clean":"2023-12-20T06:00:39.145370+0000","last_became_active":"2023-12-20T06:00:37.634998+0000","last_became_peered":"2023-12-20T06:00:37.634998+0000","last_unstale":"2023-12-20T06:00:39.145370+0000","last_undegraded":"2023-12-20T06:00:39.145370+0000","last_fullsized":"2023-12-20T06:00:39.145370+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:24:54.128140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:36.176550+0000","last_change":"2023-12-20T06:00:36.176550+0000","last_active":"2023-12-20T06:00:36.176550+0000","last_peered":"2023-12-20T06:00:36.176550+0000","last_clean":"2023-12-20T06:00:36.176550+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:36.176550+0000","last_undegraded":"2023-12-20T06:00:36.176550+0000","last_fullsized":"2023-12-20T06:00:36.176550+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148704+0000","last_change":"2023-12-20T06:00:37.625335+0000","last_active":"2023-12-20T06:00:39.148704+0000","last_peered":"2023-12-20T06:00:39.148704+0000","last_clean":"2023-12-20T06:00:39.148704+0000","last_became_active":"2023-12-20T06:00:37.625017+0000","last_became_peered":"2023-12-20T06:00:37.625017+0000","last_unstale":"2023-12-20T06:00:39.148704+0000","last_undegraded":"2023-12-20T06:00:39.148704+0000","last_fullsized":"2023-12-20T06:00:39.148704+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:48:20.707786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"191'1","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145629+0000","last_change":"2023-12-20T06:00:37.225678+0000","last_active":"2023-12-20T06:00:39.145629+0000","last_peered":"2023-12-20T06:00:39.145629+0000","last_clean":"2023-12-20T06:00:39.145629+0000","last_became_active":"2023-12-20T06:00:37.225392+0000","last_became_peered":"2023-12-20T06:00:37.225392+0000","last_unstale":"2023-12-20T06:00:39.145629+0000","last_undegraded":"2023-12-20T06:00:39.145629+0000","last_fullsized":"2023-12-20T06:00:39.145629+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:14:06.560400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694606+0000","last_change":"2023-12-20T06:00:37.633138+0000","last_active":"2023-12-20T06:00:38.694606+0000","last_peered":"2023-12-20T06:00:38.694606+0000","last_clean":"2023-12-20T06:00:38.694606+0000","last_became_active":"2023-12-20T06:00:37.632842+0000","last_became_peered":"2023-12-20T06:00:37.632842+0000","last_unstale":"2023-12-20T06:00:38.694606+0000","last_undegraded":"2023-12-20T06:00:38.694606+0000","last_fullsized":"2023-12-20T06:00:38.694606+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:03.398362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145373+0000","last_change":"2023-12-20T06:00:37.218432+0000","last_active":"2023-12-20T06:00:39.145373+0000","last_peered":"2023-12-20T06:00:39.145373+0000","last_clean":"2023-12-20T06:00:39.145373+0000","last_became_active":"2023-12-20T06:00:37.218149+0000","last_became_peered":"2023-12-20T06:00:37.218149+0000","last_unstale":"2023-12-20T06:00:39.145373+0000","last_undegraded":"2023-12-20T06:00:39.145373+0000","last_fullsized":"2023-12-20T06:00:39.145373+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:51.779317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148366+0000","last_change":"2023-12-20T06:00:37.630176+0000","last_active":"2023-12-20T06:00:39.148366+0000","last_peered":"2023-12-20T06:00:39.148366+0000","last_clean":"2023-12-20T06:00:39.148366+0000","last_became_active":"2023-12-20T06:00:37.629930+0000","last_became_peered":"2023-12-20T06:00:37.629930+0000","last_unstale":"2023-12-20T06:00:39.148366+0000","last_undegraded":"2023-12-20T06:00:39.148366+0000","last_fullsized":"2023-12-20T06:00:39.148366+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:21:31.062877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"191'1","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.638198+0000","last_change":"2023-12-20T06:00:37.221697+0000","last_active":"2023-12-20T06:00:38.638198+0000","last_peered":"2023-12-20T06:00:38.638198+0000","last_clean":"2023-12-20T06:00:38.638198+0000","last_became_active":"2023-12-20T06:00:37.221423+0000","last_became_peered":"2023-12-20T06:00:37.221423+0000","last_unstale":"2023-12-20T06:00:38.638198+0000","last_undegraded":"2023-12-20T06:00:38.638198+0000","last_fullsized":"2023-12-20T06:00:38.638198+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:20:03.650140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.627189+0000","last_change":"2023-12-20T06:00:37.630088+0000","last_active":"2023-12-20T06:00:38.627189+0000","last_peered":"2023-12-20T06:00:38.627189+0000","last_clean":"2023-12-20T06:00:38.627189+0000","last_became_active":"2023-12-20T06:00:37.629842+0000","last_became_peered":"2023-12-20T06:00:37.629842+0000","last_unstale":"2023-12-20T06:00:38.627189+0000","last_undegraded":"2023-12-20T06:00:38.627189+0000","last_fullsized":"2023-12-20T06:00:38.627189+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:02:30.560429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.638284+0000","last_change":"2023-12-20T06:00:37.217042+0000","last_active":"2023-12-20T06:00:38.638284+0000","last_peered":"2023-12-20T06:00:38.638284+0000","last_clean":"2023-12-20T06:00:38.638284+0000","last_became_active":"2023-12-20T06:00:37.216744+0000","last_became_peered":"2023-12-20T06:00:37.216744+0000","last_unstale":"2023-12-20T06:00:38.638284+0000","last_undegraded":"2023-12-20T06:00:38.638284+0000","last_fullsized":"2023-12-20T06:00:38.638284+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:40.016860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145290+0000","last_change":"2023-12-20T06:00:37.636981+0000","last_active":"2023-12-20T06:00:39.145290+0000","last_peered":"2023-12-20T06:00:39.145290+0000","last_clean":"2023-12-20T06:00:39.145290+0000","last_became_active":"2023-12-20T06:00:37.636712+0000","last_became_peered":"2023-12-20T06:00:37.636712+0000","last_unstale":"2023-12-20T06:00:39.145290+0000","last_undegraded":"2023-12-20T06:00:39.145290+0000","last_fullsized":"2023-12-20T06:00:39.145290+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:19:11.247792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148140+0000","last_change":"2023-12-20T06:00:37.229378+0000","last_active":"2023-12-20T06:00:39.148140+0000","last_peered":"2023-12-20T06:00:39.148140+0000","last_clean":"2023-12-20T06:00:39.148140+0000","last_became_active":"2023-12-20T06:00:37.229043+0000","last_became_peered":"2023-12-20T06:00:37.229043+0000","last_unstale":"2023-12-20T06:00:39.148140+0000","last_undegraded":"2023-12-20T06:00:39.148140+0000","last_fullsized":"2023-12-20T06:00:39.148140+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:40.016860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626859+0000","last_change":"2023-12-20T06:00:37.641759+0000","last_active":"2023-12-20T06:00:38.626859+0000","last_peered":"2023-12-20T06:00:38.626859+0000","last_clean":"2023-12-20T06:00:38.626859+0000","last_became_active":"2023-12-20T06:00:37.641491+0000","last_became_peered":"2023-12-20T06:00:37.641491+0000","last_unstale":"2023-12-20T06:00:38.626859+0000","last_undegraded":"2023-12-20T06:00:38.626859+0000","last_fullsized":"2023-12-20T06:00:38.626859+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:17:57.727737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637888+0000","last_change":"2023-12-20T06:00:37.218888+0000","last_active":"2023-12-20T06:00:38.637888+0000","last_peered":"2023-12-20T06:00:38.637888+0000","last_clean":"2023-12-20T06:00:38.637888+0000","last_became_active":"2023-12-20T06:00:37.218615+0000","last_became_peered":"2023-12-20T06:00:37.218615+0000","last_unstale":"2023-12-20T06:00:38.637888+0000","last_undegraded":"2023-12-20T06:00:38.637888+0000","last_fullsized":"2023-12-20T06:00:38.637888+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:51.779317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637606+0000","last_change":"2023-12-20T06:00:37.649371+0000","last_active":"2023-12-20T06:00:38.637606+0000","last_peered":"2023-12-20T06:00:38.637606+0000","last_clean":"2023-12-20T06:00:38.637606+0000","last_became_active":"2023-12-20T06:00:37.649036+0000","last_became_peered":"2023-12-20T06:00:37.649036+0000","last_unstale":"2023-12-20T06:00:38.637606+0000","last_undegraded":"2023-12-20T06:00:38.637606+0000","last_fullsized":"2023-12-20T06:00:38.637606+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:17:33.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148045+0000","last_change":"2023-12-20T06:00:37.219401+0000","last_active":"2023-12-20T06:00:39.148045+0000","last_peered":"2023-12-20T06:00:39.148045+0000","last_clean":"2023-12-20T06:00:39.148045+0000","last_became_active":"2023-12-20T06:00:37.219103+0000","last_became_peered":"2023-12-20T06:00:37.219103+0000","last_unstale":"2023-12-20T06:00:39.148045+0000","last_undegraded":"2023-12-20T06:00:39.148045+0000","last_fullsized":"2023-12-20T06:00:39.148045+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:42.530486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147921+0000","last_change":"2023-12-20T06:00:37.624653+0000","last_active":"2023-12-20T06:00:39.147921+0000","last_peered":"2023-12-20T06:00:39.147921+0000","last_clean":"2023-12-20T06:00:39.147921+0000","last_became_active":"2023-12-20T06:00:37.624346+0000","last_became_peered":"2023-12-20T06:00:37.624346+0000","last_unstale":"2023-12-20T06:00:39.147921+0000","last_undegraded":"2023-12-20T06:00:39.147921+0000","last_fullsized":"2023-12-20T06:00:39.147921+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:37.912584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637734+0000","last_change":"2023-12-20T06:00:37.221091+0000","last_active":"2023-12-20T06:00:38.637734+0000","last_peered":"2023-12-20T06:00:38.637734+0000","last_clean":"2023-12-20T06:00:38.637734+0000","last_became_active":"2023-12-20T06:00:37.220804+0000","last_became_peered":"2023-12-20T06:00:37.220804+0000","last_unstale":"2023-12-20T06:00:38.637734+0000","last_undegraded":"2023-12-20T06:00:38.637734+0000","last_fullsized":"2023-12-20T06:00:38.637734+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:14:06.560400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694604+0000","last_change":"2023-12-20T06:00:37.635358+0000","last_active":"2023-12-20T06:00:38.694604+0000","last_peered":"2023-12-20T06:00:38.694604+0000","last_clean":"2023-12-20T06:00:38.694604+0000","last_became_active":"2023-12-20T06:00:37.635083+0000","last_became_peered":"2023-12-20T06:00:37.635083+0000","last_unstale":"2023-12-20T06:00:38.694604+0000","last_undegraded":"2023-12-20T06:00:38.694604+0000","last_fullsized":"2023-12-20T06:00:38.694604+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:15:12.634489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694637+0000","last_change":"2023-12-20T06:00:37.228250+0000","last_active":"2023-12-20T06:00:38.694637+0000","last_peered":"2023-12-20T06:00:38.694637+0000","last_clean":"2023-12-20T06:00:38.694637+0000","last_became_active":"2023-12-20T06:00:37.227876+0000","last_became_peered":"2023-12-20T06:00:37.227876+0000","last_unstale":"2023-12-20T06:00:38.694637+0000","last_undegraded":"2023-12-20T06:00:38.694637+0000","last_fullsized":"2023-12-20T06:00:38.694637+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:45:50.803107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626939+0000","last_change":"2023-12-20T06:00:37.624465+0000","last_active":"2023-12-20T06:00:38.626939+0000","last_peered":"2023-12-20T06:00:38.626939+0000","last_clean":"2023-12-20T06:00:38.626939+0000","last_became_active":"2023-12-20T06:00:37.624194+0000","last_became_peered":"2023-12-20T06:00:37.624194+0000","last_unstale":"2023-12-20T06:00:38.626939+0000","last_undegraded":"2023-12-20T06:00:38.626939+0000","last_fullsized":"2023-12-20T06:00:38.626939+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:34:44.122025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145479+0000","last_change":"2023-12-20T06:00:37.214328+0000","last_active":"2023-12-20T06:00:39.145479+0000","last_peered":"2023-12-20T06:00:39.145479+0000","last_clean":"2023-12-20T06:00:39.145479+0000","last_became_active":"2023-12-20T06:00:37.214019+0000","last_became_peered":"2023-12-20T06:00:37.214019+0000","last_unstale":"2023-12-20T06:00:39.145479+0000","last_undegraded":"2023-12-20T06:00:39.145479+0000","last_fullsized":"2023-12-20T06:00:39.145479+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:21:50.035966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148367+0000","last_change":"2023-12-20T06:00:37.632210+0000","last_active":"2023-12-20T06:00:39.148367+0000","last_peered":"2023-12-20T06:00:39.148367+0000","last_clean":"2023-12-20T06:00:39.148367+0000","last_became_active":"2023-12-20T06:00:37.631933+0000","last_became_peered":"2023-12-20T06:00:37.631933+0000","last_unstale":"2023-12-20T06:00:39.148367+0000","last_undegraded":"2023-12-20T06:00:39.148367+0000","last_fullsized":"2023-12-20T06:00:39.148367+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:11:27.937059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:36.176550+0000","last_change":"2023-12-20T06:00:36.176550+0000","last_active":"2023-12-20T06:00:36.176550+0000","last_peered":"2023-12-20T06:00:36.176550+0000","last_clean":"2023-12-20T06:00:36.176550+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:36.176550+0000","last_undegraded":"2023-12-20T06:00:36.176550+0000","last_fullsized":"2023-12-20T06:00:36.176550+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694456+0000","last_change":"2023-12-20T06:00:37.644721+0000","last_active":"2023-12-20T06:00:38.694456+0000","last_peered":"2023-12-20T06:00:38.694456+0000","last_clean":"2023-12-20T06:00:38.694456+0000","last_became_active":"2023-12-20T06:00:37.644415+0000","last_became_peered":"2023-12-20T06:00:37.644415+0000","last_unstale":"2023-12-20T06:00:38.694456+0000","last_undegraded":"2023-12-20T06:00:38.694456+0000","last_fullsized":"2023-12-20T06:00:38.694456+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:46.822701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694390+0000","last_change":"2023-12-20T06:00:37.217692+0000","last_active":"2023-12-20T06:00:38.694390+0000","last_peered":"2023-12-20T06:00:38.694390+0000","last_clean":"2023-12-20T06:00:38.694390+0000","last_became_active":"2023-12-20T06:00:37.217399+0000","last_became_peered":"2023-12-20T06:00:37.217399+0000","last_unstale":"2023-12-20T06:00:38.694390+0000","last_undegraded":"2023-12-20T06:00:38.694390+0000","last_fullsized":"2023-12-20T06:00:38.694390+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:44:48.335840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694236+0000","last_change":"2023-12-20T06:00:37.625627+0000","last_active":"2023-12-20T06:00:38.694236+0000","last_peered":"2023-12-20T06:00:38.694236+0000","last_clean":"2023-12-20T06:00:38.694236+0000","last_became_active":"2023-12-20T06:00:37.623880+0000","last_became_peered":"2023-12-20T06:00:37.623880+0000","last_unstale":"2023-12-20T06:00:38.694236+0000","last_undegraded":"2023-12-20T06:00:38.694236+0000","last_fullsized":"2023-12-20T06:00:38.694236+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:06:23.484756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:36.176550+0000","last_change":"2023-12-20T06:00:36.176550+0000","last_active":"2023-12-20T06:00:36.176550+0000","last_peered":"2023-12-20T06:00:36.176550+0000","last_clean":"2023-12-20T06:00:36.176550+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:36.176550+0000","last_undegraded":"2023-12-20T06:00:36.176550+0000","last_fullsized":"2023-12-20T06:00:36.176550+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:37.188753+0000","last_change":"2023-12-20T06:00:37.188753+0000","last_active":"2023-12-20T06:00:37.188753+0000","last_peered":"2023-12-20T06:00:37.188753+0000","last_clean":"2023-12-20T06:00:37.188753+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:37.188753+0000","last_undegraded":"2023-12-20T06:00:37.188753+0000","last_fullsized":"2023-12-20T06:00:37.188753+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147887+0000","last_change":"2023-12-20T06:00:37.217996+0000","last_active":"2023-12-20T06:00:39.147887+0000","last_peered":"2023-12-20T06:00:39.147887+0000","last_clean":"2023-12-20T06:00:39.147887+0000","last_became_active":"2023-12-20T06:00:37.217046+0000","last_became_peered":"2023-12-20T06:00:37.217046+0000","last_unstale":"2023-12-20T06:00:39.147887+0000","last_undegraded":"2023-12-20T06:00:39.147887+0000","last_fullsized":"2023-12-20T06:00:39.147887+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:09.715643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626657+0000","last_change":"2023-12-20T06:00:37.641033+0000","last_active":"2023-12-20T06:00:38.626657+0000","last_peered":"2023-12-20T06:00:38.626657+0000","last_clean":"2023-12-20T06:00:38.626657+0000","last_became_active":"2023-12-20T06:00:37.640765+0000","last_became_peered":"2023-12-20T06:00:37.640765+0000","last_unstale":"2023-12-20T06:00:38.626657+0000","last_undegraded":"2023-12-20T06:00:38.626657+0000","last_fullsized":"2023-12-20T06:00:38.626657+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:08:10.404165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:36.176550+0000","last_change":"2023-12-20T06:00:36.176550+0000","last_active":"2023-12-20T06:00:36.176550+0000","last_peered":"2023-12-20T06:00:36.176550+0000","last_clean":"2023-12-20T06:00:36.176550+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:36.176550+0000","last_undegraded":"2023-12-20T06:00:36.176550+0000","last_fullsized":"2023-12-20T06:00:36.176550+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145235+0000","last_change":"2023-12-20T06:00:37.634654+0000","last_active":"2023-12-20T06:00:39.145235+0000","last_peered":"2023-12-20T06:00:39.145235+0000","last_clean":"2023-12-20T06:00:39.145235+0000","last_became_active":"2023-12-20T06:00:37.634353+0000","last_became_peered":"2023-12-20T06:00:37.634353+0000","last_unstale":"2023-12-20T06:00:39.145235+0000","last_undegraded":"2023-12-20T06:00:39.145235+0000","last_fullsized":"2023-12-20T06:00:39.145235+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:38.916609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:36.176550+0000","last_change":"2023-12-20T06:00:36.176550+0000","last_active":"2023-12-20T06:00:36.176550+0000","last_peered":"2023-12-20T06:00:36.176550+0000","last_clean":"2023-12-20T06:00:36.176550+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:36.176550+0000","last_undegraded":"2023-12-20T06:00:36.176550+0000","last_fullsized":"2023-12-20T06:00:36.176550+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694540+0000","last_change":"2023-12-20T06:00:37.639634+0000","last_active":"2023-12-20T06:00:38.694540+0000","last_peered":"2023-12-20T06:00:38.694540+0000","last_clean":"2023-12-20T06:00:38.694540+0000","last_became_active":"2023-12-20T06:00:37.639375+0000","last_became_peered":"2023-12-20T06:00:37.639375+0000","last_unstale":"2023-12-20T06:00:38.694540+0000","last_undegraded":"2023-12-20T06:00:38.694540+0000","last_fullsized":"2023-12-20T06:00:38.694540+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:11:22.988664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":0,"reported_epoch":0,"state":"unknown","last_fresh":"2023-12-20T06:00:36.176550+0000","last_change":"2023-12-20T06:00:36.176550+0000","last_active":"2023-12-20T06:00:36.176550+0000","last_peered":"2023-12-20T06:00:36.176550+0000","last_clean":"2023-12-20T06:00:36.176550+0000","last_became_active":"0.000000","last_became_peered":"0.000000","last_unstale":"2023-12-20T06:00:36.176550+0000","last_undegraded":"2023-12-20T06:00:36.176550+0000","last_fullsized":"2023-12-20T06:00:36.176550+0000","mapping_epoch":0,"log_start":"0'0","ondisk_log_start":"0'0","created":0,"last_epoch_clean":0,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[],"acting":[],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":-1,"acting_primary":-1,"purged_snaps":[]},{"pgid":"1.0","version":"180'3179","reported_seq":3899,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-12-20T06:00:37.640057+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:00:37.640057+0000","last_peered":"2023-12-20T06:00:37.640057+0000","last_clean":"2023-12-20T06:00:37.640057+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:00:37.640057+0000","last_undegraded":"2023-12-20T06:00:37.640057+0000","last_fullsized":"2023-12-20T06:00:37.640057+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3179,"log_dups_size":0,"ondisk_log_size":3179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1693,"num_read_kb":3437,"num_write":481,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":100,"acting":100,"num_store_stats":6},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":798,"num_objects":13,"num_object_clones":0,"num_object_copies":26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":13,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":26,"num_write_kb":11,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3068,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":23,"ondisk_log_size":23,"up":46,"acting":46,"num_store_stats":6},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1693,"num_read_kb":3437,"num_write":481,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3179,"ondisk_log_size":3179,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56140,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315700,"statfs":{"total":96636764160,"available":96579276800,"internally_reserved":0,"allocated":3018752,"data_stored":1753309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.73999999999999999},"min":{"1min":0.57199999999999995,"5min":0.439,"15min":0.439},"max":{"1min":0.90600000000000003,"5min":0.97099999999999997,"15min":1.042},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.749},"min":{"1min":0.432,"5min":0.432,"15min":0.34499999999999997},"max":{"1min":0.94799999999999995,"5min":0.99099999999999999,"15min":1.161},"last":0.56699999999999995}]},{"osd":1,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.745,"15min":0.754},"min":{"1min":0.49099999999999999,"5min":0.46000000000000002,"15min":0.432},"max":{"1min":0.94799999999999995,"5min":1.006,"15min":1.1279999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.46999999999999997},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":1.032},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48299999999999998},"max":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":1.0409999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.75,"15min":0.76800000000000002},"min":{"1min":0.53900000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0089999999999999,"5min":1.0189999999999999,"15min":1.0409999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73799999999999999,"15min":0.747},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.47799999999999998},"max":{"1min":0.88900000000000001,"5min":0.98599999999999999,"15min":1.0069999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77500000000000002,"15min":0.79200000000000004},"min":{"1min":0.503,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1.0840000000000001},"last":0.55500000000000005}]},{"osd":4,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":1.04},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67000000000000004,"15min":0.66400000000000003},"min":{"1min":0.53600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":0.99199999999999999,"15min":1.1120000000000001},"last":0.624},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.45600000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.103},"last":0.76700000000000002}]},{"osd":6,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.74299999999999999,"15min":0.749},"min":{"1min":0.54400000000000004,"5min":0.498,"15min":0.45500000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71099999999999997,"15min":0.72399999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.1539999999999999},"last":0.89600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575948,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":51380,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320460,"statfs":{"total":96636764160,"available":96584151040,"internally_reserved":0,"allocated":2338816,"data_stored":1129291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74199999999999999,"15min":0.747},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92300000000000004,"5min":0.99099999999999999,"15min":1.0980000000000001},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.74299999999999999,"15min":0.754},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94999999999999996,"5min":1.016,"15min":1.05},"last":0.63}]},{"osd":1,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72899999999999998,"15min":0.73799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98499999999999999,"5min":0.996,"15min":0.996},"last":0.878},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97299999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.77500000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.47299999999999998},"max":{"1min":0.90900000000000003,"5min":1.069,"15min":1.069},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96199999999999997,"5min":1.073,"15min":1.0760000000000001},"last":0.96199999999999997}]},{"osd":3,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.042,"5min":1.077,"15min":1.077},"last":1.042},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.54300000000000004,"5min":0.51200000000000001,"15min":0.503},"max":{"1min":1.0309999999999999,"5min":1.052,"15min":1.052},"last":0.94599999999999995}]},{"osd":4,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95799999999999996,"5min":0.98199999999999998,"15min":1.0529999999999999},"last":0.92500000000000004},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.47399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":1.1339999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.45500000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.98599999999999999,"15min":1.0009999999999999},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.99299999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.504,"5min":0.48899999999999999,"15min":0.45300000000000001},"max":{"1min":1.103,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":1.103},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0169999999999999,"5min":1.022,"15min":1.0700000000000001},"last":1.0169999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29304,"kb_used_data":2352,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342536,"statfs":{"total":96636764160,"available":96606756864,"internally_reserved":0,"allocated":2408448,"data_stored":1161239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.496,"5min":0.45300000000000001,"15min":0.44400000000000001},"max":{"1min":0.94799999999999995,"5min":1.004,"15min":1.077},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.433},"max":{"1min":0.96599999999999997,"5min":1.028,"15min":1.107},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.754,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":1.0409999999999999,"5min":1.0669999999999999,"15min":1.073},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.754,"15min":0.77000000000000002},"min":{"1min":0.56299999999999994,"5min":0.51300000000000001,"15min":0.46899999999999997},"max":{"1min":0.90600000000000003,"5min":1.016,"15min":1.0640000000000001},"last":0.73699999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.74099999999999999,"15min":0.76800000000000002},"min":{"1min":0.51700000000000002,"5min":0.46999999999999997,"15min":0.46899999999999997},"max":{"1min":0.95699999999999996,"5min":1.01,"15min":1.0249999999999999},"last":0.505},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.58599999999999997,"5min":0.52000000000000002,"15min":0.48299999999999998},"max":{"1min":0.95899999999999996,"5min":1.014,"15min":1.0680000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.52000000000000002},"max":{"1min":1.0269999999999999,"5min":1.0669999999999999,"15min":1.141},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78500000000000003,"15min":0.78600000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.98099999999999998,"5min":1.034,"15min":1.0629999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64900000000000002,"15min":0.63700000000000001},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.0640000000000001,"15min":1.077},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66100000000000003,"15min":0.63700000000000001},"min":{"1min":0.48499999999999999,"5min":0.432,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":1.0029999999999999,"15min":1.054},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.57299999999999995,"5min":0.48699999999999999,"15min":0.439},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65800000000000003,"15min":0.64500000000000002},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.012,"5min":1.012,"15min":1.073},"last":0.59499999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.70799999999999996,"15min":0.70099999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.48099999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0489999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.46700000000000003},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69799999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":47284,"kb_used_data":2284,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324556,"statfs":{"total":96636764160,"available":96588345344,"internally_reserved":0,"allocated":2338816,"data_stored":1129291,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.74399999999999999,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.47399999999999998,"15min":0.38800000000000001},"max":{"1min":0.98999999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.45400000000000001},"max":{"1min":0.96399999999999997,"5min":0.996,"15min":1.014},"last":0.82299999999999995}]},{"osd":1,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.56799999999999995,"5min":0.49199999999999999,"15min":0.45100000000000001},"max":{"1min":0.94799999999999995,"5min":1.042,"15min":1.0900000000000001},"last":0.67800000000000005},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.54700000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0600000000000001},"last":0.79600000000000004}]},{"osd":2,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78300000000000003,"15min":0.78400000000000003},"min":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.71399999999999997},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.505},"max":{"1min":1.006,"5min":1.042,"15min":1.101},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.49199999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0129999999999999},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80300000000000005,"15min":0.81599999999999995},"min":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.081},"last":0.754}]},{"osd":5,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.39900000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.61399999999999999},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66900000000000004,"15min":0.65000000000000002},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.35799999999999998},"max":{"1min":0.97899999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.68999999999999995}]},{"osd":6,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.67100000000000004},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.41499999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.1679999999999999},"last":0.59699999999999998},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.748,"15min":0.72899999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.41199999999999998},"max":{"1min":1.024,"5min":1.0700000000000001,"15min":1.111},"last":0.65700000000000003}]},{"osd":7,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.47899999999999998,"5min":0.39700000000000002,"15min":0.38500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.1160000000000001},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.52600000000000002,"5min":0.505,"15min":0.434},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.63100000000000001}]}]},{"osd":3,"up_from":13,"seq":55834575950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47348,"kb_used_data":2348,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324492,"statfs":{"total":96636764160,"available":96588279808,"internally_reserved":0,"allocated":2404352,"data_stored":1160669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.49299999999999999,"15min":0.47499999999999998},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.28499999999999998},"max":{"1min":0.748,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.36199999999999999,"5min":0.33700000000000002,"15min":0.32700000000000001},"max":{"1min":0.67600000000000005,"5min":1.357,"15min":1.357},"last":0.52900000000000003}]},{"osd":1,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.48999999999999999,"15min":0.48099999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32000000000000001},"max":{"1min":0.622,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.372},"max":{"1min":0.64100000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.40300000000000002},"max":{"1min":0.76400000000000001,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.40200000000000002,"15min":0.34200000000000003},"max":{"1min":0.86099999999999999,"5min":2.944,"15min":2.944},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82599999999999996,"15min":0.82999999999999996},"min":{"1min":0.60099999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.056,"5min":1.669,"15min":1.669},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80600000000000005,"15min":0.82399999999999995},"min":{"1min":0.63800000000000001,"5min":0.54400000000000004,"15min":0.54300000000000004},"max":{"1min":0.97699999999999998,"5min":1.99,"15min":1.99},"last":0.73799999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.84799999999999998,"15min":0.84699999999999998},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.095,"5min":1.901,"15min":1.901},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.82799999999999996,"15min":0.83099999999999996},"min":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.55000000000000004},"max":{"1min":1.0880000000000001,"5min":2.8639999999999999,"15min":2.8639999999999999},"last":0.85999999999999999}]},{"osd":6,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.82099999999999995,"15min":0.82999999999999996},"min":{"1min":0.59099999999999997,"5min":0.58799999999999997,"15min":0.56599999999999995},"max":{"1min":0.99399999999999999,"5min":1.575,"15min":1.575},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.86599999999999999,"15min":0.86199999999999999},"min":{"1min":0.622,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1479999999999999,"5min":1.948,"15min":1.948},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.85399999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.58999999999999997},"max":{"1min":1.1579999999999999,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.871,"15min":0.86399999999999999},"min":{"1min":0.67100000000000004,"5min":0.623,"15min":0.55700000000000005},"max":{"1min":1.0309999999999999,"5min":1.754,"15min":1.754},"last":0.60499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575949,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51444,"kb_used_data":2348,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320396,"statfs":{"total":96636764160,"available":96584085504,"internally_reserved":0,"allocated":2404352,"data_stored":1160725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46000000000000002,"15min":0.45900000000000002},"min":{"1min":0.34100000000000003,"5min":0.28299999999999997,"15min":0.26100000000000001},"max":{"1min":0.70399999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.51300000000000001,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33300000000000002},"max":{"1min":0.89800000000000002,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.497,"15min":0.496},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75800000000000001,"5min":0.77100000000000002,"15min":0.84899999999999998},"last":0.44},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.42399999999999999,"5min":0.40000000000000002,"15min":0.30499999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.004},"last":0.505}]},{"osd":3,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.54300000000000004,"5min":0.42099999999999999,"15min":0.39800000000000002},"max":{"1min":0.86199999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.68899999999999995},"min":{"1min":0.57599999999999996,"5min":0.52800000000000002,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.57299999999999995}]},{"osd":4,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78500000000000003,"15min":0.80800000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.254},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.81000000000000005},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48199999999999998},"max":{"1min":1.085,"5min":1.605,"15min":1.605},"last":0.77500000000000002}]},{"osd":5,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1639999999999999,"5min":1.329,"15min":1.329},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.83499999999999996,"15min":0.83599999999999997},"min":{"1min":0.63700000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.028,"5min":1.0740000000000001,"15min":1.157},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.55100000000000005,"5min":0.52800000000000002,"15min":0.52300000000000002},"max":{"1min":1.0880000000000001,"5min":1.919,"15min":1.919},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.56200000000000006},"max":{"1min":1.117,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.80800000000000005}]},{"osd":7,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85399999999999998,"15min":0.85999999999999999},"min":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.47899999999999998},"max":{"1min":1.196,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.89400000000000002,"15min":0.88900000000000001},"min":{"1min":0.75700000000000001,"5min":0.67300000000000004,"15min":0.60499999999999998},"max":{"1min":1.165,"5min":1.292,"15min":1.292},"last":0.97299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575951,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59636,"kb_used_data":2348,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312204,"statfs":{"total":96636764160,"available":96575696896,"internally_reserved":0,"allocated":2404352,"data_stored":1160669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.47999999999999998,"15min":0.46000000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.23899999999999999},"max":{"1min":0.61899999999999999,"5min":0.746,"15min":0.81000000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69499999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53300000000000003,"15min":0.53200000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36099999999999999},"max":{"1min":0.68400000000000005,"5min":0.76800000000000002,"15min":0.79200000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.38300000000000001},"max":{"1min":0.86099999999999999,"5min":0.88800000000000001,"15min":0.93700000000000006},"last":0.49399999999999999}]},{"osd":3,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88500000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.93500000000000005},"last":0.54700000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77900000000000003,"15min":0.79400000000000004},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0129999999999999,"5min":1.0820000000000001,"15min":1.321},"last":0.874},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81499999999999995,"15min":0.81699999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.58099999999999996},"max":{"1min":1.002,"5min":1.139,"15min":1.25},"last":0.74099999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.80900000000000005},"min":{"1min":0.55100000000000005,"5min":0.51400000000000001,"15min":0.443},"max":{"1min":1.157,"5min":1.224,"15min":1.607},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.80200000000000005,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.024,"5min":1.25,"15min":1.4330000000000001},"last":0.81799999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81699999999999995,"15min":0.83399999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0329999999999999,"5min":1.2430000000000001,"15min":1.341},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81799999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1040000000000001,"5min":1.177,"15min":1.3839999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.57799999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0509999999999999,"5min":1.272,"15min":1.4470000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86199999999999999,"15min":0.873},"min":{"1min":0.62,"5min":0.62,"15min":0.58699999999999997},"max":{"1min":1.1279999999999999,"5min":1.1599999999999999,"15min":1.3839999999999999},"last":0.75900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575950,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56116,"kb_used_data":2924,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315724,"statfs":{"total":96636764160,"available":96579301376,"internally_reserved":0,"allocated":2994176,"data_stored":1751003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.28799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.96799999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.39200000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.752,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60099999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.53600000000000003},"min":{"1min":0.39400000000000002,"5min":0.376,"15min":0.35999999999999999},"max":{"1min":0.76600000000000001,"5min":0.81299999999999994,"15min":0.82499999999999996},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.45900000000000002,"5min":0.40799999999999997,"15min":0.38100000000000001},"max":{"1min":0.77100000000000002,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68500000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.442,"5min":0.34200000000000003,"15min":0.31900000000000001},"max":{"1min":0.78800000000000003,"5min":0.80700000000000005,"15min":1.0429999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.628,"15min":0.627},"min":{"1min":0.47699999999999998,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.78600000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63400000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82799999999999996,"15min":0.82499999999999996},"min":{"1min":0.61899999999999999,"5min":0.56499999999999995,"15min":0.48099999999999998},"max":{"1min":0.996,"5min":1.0669999999999999,"15min":1.147},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82899999999999996,"15min":0.83799999999999997},"min":{"1min":0.68899999999999995,"5min":0.58399999999999996,"15min":0.51400000000000001},"max":{"1min":0.96999999999999997,"5min":1.0760000000000001,"15min":1.2869999999999999},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.84499999999999997},"min":{"1min":0.626,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.2150000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.57299999999999995},"max":{"1min":1.0269999999999999,"5min":1.131,"15min":1.351},"last":0.84099999999999997}]},{"osd":6,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.67400000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0149999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.85799999999999998,"15min":0.86299999999999999},"min":{"1min":0.66500000000000004,"5min":0.62,"15min":0.57099999999999995},"max":{"1min":1.0469999999999999,"5min":1.1379999999999999,"15min":1.3320000000000001},"last":1.008}]},{"osd":7,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.87,"15min":0.873},"min":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.995,"5min":1.129,"15min":1.3560000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86099999999999999,"15min":0.86599999999999999},"min":{"1min":0.68799999999999994,"5min":0.60299999999999998,"15min":0.58899999999999997},"max":{"1min":1.079,"5min":1.139,"15min":1.2549999999999999},"last":0.94599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:00:41.747 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:00:41.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.930+0000 7ff3de5cb640 1 Processor -- start 2023-12-20T06:00:41.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.930+0000 7ff3de5cb640 1 -- start start 2023-12-20T06:00:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3de5cb640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3d80b8480 0x7ff3d806af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3de5cb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff3d8161500 0x7ff3d8153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3de5cb640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff3d806b8c0 con 0x7ff3d806b460 2023-12-20T06:00:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3de5cb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff3d80cd4d0 con 0x7ff3d8161500 2023-12-20T06:00:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3de5cb640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d80ce4e0 con 0x7ff3d80b8480 2023-12-20T06:00:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3dcdc8640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff3d806b460 0x7ff3d8160df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55582/0 (socket says 172.21.15.32:55582) 2023-12-20T06:00:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3dcdc8640 1 -- 172.21.15.32:0/2290201407 learned_addr learned my addr 172.21.15.32:0/2290201407 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:41.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3dd5c9640 1 --2- 172.21.15.32:0/2290201407 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3d80b8480 0x7ff3d806af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3dddca640 1 --2- 172.21.15.32:0/2290201407 >> v2:172.21.15.39:3300/0 conn(0x7ff3d8161500 0x7ff3d8153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3cffff640 1 -- 172.21.15.32:0/2290201407 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1363375147 0 0) 0x7ff3d806b8c0 con 0x7ff3d806b460 2023-12-20T06:00:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3cffff640 1 -- 172.21.15.32:0/2290201407 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff3c4003530 con 0x7ff3d806b460 2023-12-20T06:00:41.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3dddca640 1 -- 172.21.15.32:0/2290201407 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3d80b8480 msgr2=0x7ff3d806af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3dddca640 1 --2- 172.21.15.32:0/2290201407 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3d80b8480 0x7ff3d806af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3dddca640 1 -- 172.21.15.32:0/2290201407 >> v1:172.21.15.32:6789/0 conn(0x7ff3d806b460 legacy=0x7ff3d8160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.934+0000 7ff3dddca640 1 -- 172.21.15.32:0/2290201407 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d80cd720 con 0x7ff3d8161500 2023-12-20T06:00:41.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3dddca640 1 --2- 172.21.15.32:0/2290201407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d8161500 0x7ff3d8153610 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009d90 tx=0x7ff3d001c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8b689aebd385e6c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:41.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3cffff640 1 -- 172.21.15.32:0/2290201407 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00a5810 con 0x7ff3d8161500 2023-12-20T06:00:41.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3cffff640 1 -- 172.21.15.32:0/2290201407 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d00a5970 con 0x7ff3d8161500 2023-12-20T06:00:41.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3cffff640 1 -- 172.21.15.32:0/2290201407 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00a5d00 con 0x7ff3d8161500 2023-12-20T06:00:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3de5cb640 1 -- 172.21.15.32:0/2290201407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d8161500 msgr2=0x7ff3d8153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3de5cb640 1 --2- 172.21.15.32:0/2290201407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d8161500 0x7ff3d8153610 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009d90 tx=0x7ff3d001c950 comp rx=0 tx=0).stop 2023-12-20T06:00:41.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3de5cb640 1 -- 172.21.15.32:0/2290201407 shutdown_connections 2023-12-20T06:00:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3de5cb640 1 --2- 172.21.15.32:0/2290201407 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d8161500 0x7ff3d8153610 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3de5cb640 1 --2- 172.21.15.32:0/2290201407 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3d80b8480 0x7ff3d806af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3de5cb640 1 -- 172.21.15.32:0/2290201407 >> 172.21.15.32:0/2290201407 conn(0x7ff3d80d2a50 msgr2=0x7ff3d80d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3de5cb640 1 -- 172.21.15.32:0/2290201407 shutdown_connections 2023-12-20T06:00:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3de5cb640 1 -- 172.21.15.32:0/2290201407 wait complete. 2023-12-20T06:00:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3de5cb640 1 Processor -- start 2023-12-20T06:00:41.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.938+0000 7ff3de5cb640 1 -- start start 2023-12-20T06:00:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3de5cb640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3d806b460 0x7ff3d81535e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3de5cb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d80b8480 0x7ff3d814d390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3de5cb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3d8161500 0x7ff3d8152b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3de5cb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d814d8d0 con 0x7ff3d8161500 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3de5cb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d8154fa0 con 0x7ff3d80b8480 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3de5cb640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff3d81546b0 con 0x7ff3d806b460 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3dddca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3d8161500 0x7ff3d8152b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3dddca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3d8161500 0x7ff3d8152b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47358/0 (socket says 172.21.15.32:47358) 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3dddca640 1 -- 172.21.15.32:0/1634100763 learned_addr learned my addr 172.21.15.32:0/1634100763 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3dcdc8640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d80b8480 0x7ff3d814d390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3dd5c9640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3d806b460 0x7ff3d81535e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3dddca640 1 -- 172.21.15.32:0/1634100763 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3d806b460 msgr2=0x7ff3d81535e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3dddca640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3d806b460 0x7ff3d81535e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3dddca640 1 -- 172.21.15.32:0/1634100763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d80b8480 msgr2=0x7ff3d814d390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3dddca640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d80b8480 0x7ff3d814d390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:41.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3dddca640 1 -- 172.21.15.32:0/1634100763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3d0009a70 con 0x7ff3d8161500 2023-12-20T06:00:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3dddca640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3d8161500 0x7ff3d8152b80 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009d90 tx=0x7ff3d00196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:41.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3cdffb640 1 -- 172.21.15.32:0/1634100763 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00b9650 con 0x7ff3d8161500 2023-12-20T06:00:41.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3cdffb640 1 -- 172.21.15.32:0/1634100763 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3d00b97b0 con 0x7ff3d8161500 2023-12-20T06:00:41.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3cdffb640 1 -- 172.21.15.32:0/1634100763 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3d00b9b20 con 0x7ff3d8161500 2023-12-20T06:00:41.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3de5cb640 1 -- 172.21.15.32:0/1634100763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff3d8154950 con 0x7ff3d8161500 2023-12-20T06:00:41.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3de5cb640 1 -- 172.21.15.32:0/1634100763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff3d814dae0 con 0x7ff3d8161500 2023-12-20T06:00:41.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3de5cb640 1 -- 172.21.15.32:0/1634100763 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff3d8155150 con 0x7ff3d8161500 2023-12-20T06:00:41.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.942+0000 7ff3cdffb640 1 -- 172.21.15.32:0/1634100763 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff3d00b9c80 con 0x7ff3d8161500 2023-12-20T06:00:41.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.946+0000 7ff3cdffb640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff3bc065b80 0x7ff3bc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:41.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.946+0000 7ff3cdffb640 1 -- 172.21.15.32:0/1634100763 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff3d011e3f0 con 0x7ff3d8161500 2023-12-20T06:00:41.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.946+0000 7ff3dd5c9640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff3bc065b80 0x7ff3bc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:41.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.946+0000 7ff3dd5c9640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff3bc065b80 0x7ff3bc068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009b60 tx=0x7ff3d4009190 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:41.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:41.950+0000 7ff3cdffb640 1 -- 172.21.15.32:0/1634100763 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff3d8155150 con 0x7ff3d8161500 2023-12-20T06:00:42.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.102+0000 7ff3de5cb640 1 -- 172.21.15.32:0/1634100763 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff3d80cdfc0 con 0x7ff3bc065b80 2023-12-20T06:00:42.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.110+0000 7ff3cdffb640 1 -- 172.21.15.32:0/1634100763 <== mgr.9786 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289404 (secure 0 0 0) 0x7ff3d4011ac0 con 0x7ff3bc065b80 2023-12-20T06:00:42.115 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:42.117 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:00:42.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 -- 172.21.15.32:0/1634100763 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff3bc065b80 msgr2=0x7ff3bc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff3bc065b80 0x7ff3bc068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7ff3d4009b60 tx=0x7ff3d4009190 comp rx=0 tx=0).stop 2023-12-20T06:00:42.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 -- 172.21.15.32:0/1634100763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3d8161500 msgr2=0x7ff3d8152b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3d8161500 0x7ff3d8152b80 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7ff3d0009d90 tx=0x7ff3d00196f0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 -- 172.21.15.32:0/1634100763 shutdown_connections 2023-12-20T06:00:42.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff3bc065b80 0x7ff3bc068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff3d8161500 0x7ff3d8152b80 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3d80b8480 0x7ff3d814d390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 --2- 172.21.15.32:0/1634100763 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3d806b460 0x7ff3d81535e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 -- 172.21.15.32:0/1634100763 >> 172.21.15.32:0/1634100763 conn(0x7ff3d80d2a50 msgr2=0x7ff3d80d0fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:42.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 -- 172.21.15.32:0/1634100763 shutdown_connections 2023-12-20T06:00:42.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.114+0000 7ff3de5cb640 1 -- 172.21.15.32:0/1634100763 wait complete. 2023-12-20T06:00:42.134 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":24,"stamp":"2023-12-20T06:00:41.904505+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1693,"num_read_kb":3437,"num_write":525,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3217,"ondisk_log_size":3217,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":144,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":398792,"kb_used_data":19976,"kb_used_omap":0,"kb_used_meta":378752,"kb_avail":754575928,"statfs":{"total":773094113280,"available":772685750272,"internally_reserved":0,"allocated":20455424,"data_stored":10470540,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":387842048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":29,"apply_latency_ms":29,"commit_latency_ns":29000000,"apply_latency_ns":29000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.710647"},"pg_stats":[{"pgid":"23.3f","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.638264+0000","last_change":"2023-12-20T06:00:37.650130+0000","last_active":"2023-12-20T06:00:38.638264+0000","last_peered":"2023-12-20T06:00:38.638264+0000","last_clean":"2023-12-20T06:00:38.638264+0000","last_became_active":"2023-12-20T06:00:37.648977+0000","last_became_peered":"2023-12-20T06:00:37.648977+0000","last_unstale":"2023-12-20T06:00:38.638264+0000","last_undegraded":"2023-12-20T06:00:38.638264+0000","last_fullsized":"2023-12-20T06:00:38.638264+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:05:01.260698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.3e","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.949051+0000","last_change":"2023-12-20T06:00:37.644532+0000","last_active":"2023-12-20T06:00:38.949051+0000","last_peered":"2023-12-20T06:00:38.949051+0000","last_clean":"2023-12-20T06:00:38.949051+0000","last_became_active":"2023-12-20T06:00:37.644227+0000","last_became_peered":"2023-12-20T06:00:37.644227+0000","last_unstale":"2023-12-20T06:00:38.949051+0000","last_undegraded":"2023-12-20T06:00:38.949051+0000","last_fullsized":"2023-12-20T06:00:38.949051+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:19:11.247792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3d","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145292+0000","last_change":"2023-12-20T06:00:37.647806+0000","last_active":"2023-12-20T06:00:39.145292+0000","last_peered":"2023-12-20T06:00:39.145292+0000","last_clean":"2023-12-20T06:00:39.145292+0000","last_became_active":"2023-12-20T06:00:37.646357+0000","last_became_peered":"2023-12-20T06:00:37.646357+0000","last_unstale":"2023-12-20T06:00:39.145292+0000","last_undegraded":"2023-12-20T06:00:39.145292+0000","last_fullsized":"2023-12-20T06:00:39.145292+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:23:48.223962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.3c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148224+0000","last_change":"2023-12-20T06:00:37.636127+0000","last_active":"2023-12-20T06:00:39.148224+0000","last_peered":"2023-12-20T06:00:39.148224+0000","last_clean":"2023-12-20T06:00:39.148224+0000","last_became_active":"2023-12-20T06:00:37.635841+0000","last_became_peered":"2023-12-20T06:00:37.635841+0000","last_unstale":"2023-12-20T06:00:39.148224+0000","last_undegraded":"2023-12-20T06:00:39.148224+0000","last_fullsized":"2023-12-20T06:00:39.148224+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:38.916609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.3b","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.948819+0000","last_change":"2023-12-20T06:00:37.629043+0000","last_active":"2023-12-20T06:00:38.948819+0000","last_peered":"2023-12-20T06:00:38.948819+0000","last_clean":"2023-12-20T06:00:38.948819+0000","last_became_active":"2023-12-20T06:00:37.628743+0000","last_became_peered":"2023-12-20T06:00:37.628743+0000","last_unstale":"2023-12-20T06:00:38.948819+0000","last_undegraded":"2023-12-20T06:00:38.948819+0000","last_fullsized":"2023-12-20T06:00:38.948819+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:39:00.341777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.3a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148047+0000","last_change":"2023-12-20T06:00:37.628715+0000","last_active":"2023-12-20T06:00:39.148047+0000","last_peered":"2023-12-20T06:00:39.148047+0000","last_clean":"2023-12-20T06:00:39.148047+0000","last_became_active":"2023-12-20T06:00:37.627720+0000","last_became_peered":"2023-12-20T06:00:37.627720+0000","last_unstale":"2023-12-20T06:00:39.148047+0000","last_undegraded":"2023-12-20T06:00:39.148047+0000","last_fullsized":"2023-12-20T06:00:39.148047+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:17:17.747988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.39","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626810+0000","last_change":"2023-12-20T06:00:37.625525+0000","last_active":"2023-12-20T06:00:38.626810+0000","last_peered":"2023-12-20T06:00:38.626810+0000","last_clean":"2023-12-20T06:00:38.626810+0000","last_became_active":"2023-12-20T06:00:37.625268+0000","last_became_peered":"2023-12-20T06:00:37.625268+0000","last_unstale":"2023-12-20T06:00:38.626810+0000","last_undegraded":"2023-12-20T06:00:38.626810+0000","last_fullsized":"2023-12-20T06:00:38.626810+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:49:26.463498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.38","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.948662+0000","last_change":"2023-12-20T06:00:37.631224+0000","last_active":"2023-12-20T06:00:38.948662+0000","last_peered":"2023-12-20T06:00:38.948662+0000","last_clean":"2023-12-20T06:00:38.948662+0000","last_became_active":"2023-12-20T06:00:37.630890+0000","last_became_peered":"2023-12-20T06:00:37.630890+0000","last_unstale":"2023-12-20T06:00:38.948662+0000","last_undegraded":"2023-12-20T06:00:38.948662+0000","last_fullsized":"2023-12-20T06:00:38.948662+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:37:11.971643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.37","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.949156+0000","last_change":"2023-12-20T06:00:37.647880+0000","last_active":"2023-12-20T06:00:38.949156+0000","last_peered":"2023-12-20T06:00:38.949156+0000","last_clean":"2023-12-20T06:00:38.949156+0000","last_became_active":"2023-12-20T06:00:37.647595+0000","last_became_peered":"2023-12-20T06:00:37.647595+0000","last_unstale":"2023-12-20T06:00:38.949156+0000","last_undegraded":"2023-12-20T06:00:38.949156+0000","last_fullsized":"2023-12-20T06:00:38.949156+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:53:17.598471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.36","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.949091+0000","last_change":"2023-12-20T06:00:37.629565+0000","last_active":"2023-12-20T06:00:38.949091+0000","last_peered":"2023-12-20T06:00:38.949091+0000","last_clean":"2023-12-20T06:00:38.949091+0000","last_became_active":"2023-12-20T06:00:37.629210+0000","last_became_peered":"2023-12-20T06:00:37.629210+0000","last_unstale":"2023-12-20T06:00:38.949091+0000","last_undegraded":"2023-12-20T06:00:38.949091+0000","last_fullsized":"2023-12-20T06:00:38.949091+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:23:48.223962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.35","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.948977+0000","last_change":"2023-12-20T06:00:37.626744+0000","last_active":"2023-12-20T06:00:38.948977+0000","last_peered":"2023-12-20T06:00:38.948977+0000","last_clean":"2023-12-20T06:00:38.948977+0000","last_became_active":"2023-12-20T06:00:37.626467+0000","last_became_peered":"2023-12-20T06:00:37.626467+0000","last_unstale":"2023-12-20T06:00:38.948977+0000","last_undegraded":"2023-12-20T06:00:38.948977+0000","last_fullsized":"2023-12-20T06:00:38.948977+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:24:54.128140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.34","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.948921+0000","last_change":"2023-12-20T06:00:37.636573+0000","last_active":"2023-12-20T06:00:38.948921+0000","last_peered":"2023-12-20T06:00:38.948921+0000","last_clean":"2023-12-20T06:00:38.948921+0000","last_became_active":"2023-12-20T06:00:37.636285+0000","last_became_peered":"2023-12-20T06:00:37.636285+0000","last_unstale":"2023-12-20T06:00:38.948921+0000","last_undegraded":"2023-12-20T06:00:38.948921+0000","last_fullsized":"2023-12-20T06:00:38.948921+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:25:46.547193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.33","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145385+0000","last_change":"2023-12-20T06:00:37.641497+0000","last_active":"2023-12-20T06:00:39.145385+0000","last_peered":"2023-12-20T06:00:39.145385+0000","last_clean":"2023-12-20T06:00:39.145385+0000","last_became_active":"2023-12-20T06:00:37.641213+0000","last_became_peered":"2023-12-20T06:00:37.641213+0000","last_unstale":"2023-12-20T06:00:39.145385+0000","last_undegraded":"2023-12-20T06:00:39.145385+0000","last_fullsized":"2023-12-20T06:00:39.145385+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:25:46.547193+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.32","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145133+0000","last_change":"2023-12-20T06:00:37.639889+0000","last_active":"2023-12-20T06:00:39.145133+0000","last_peered":"2023-12-20T06:00:39.145133+0000","last_clean":"2023-12-20T06:00:39.145133+0000","last_became_active":"2023-12-20T06:00:37.639631+0000","last_became_peered":"2023-12-20T06:00:37.639631+0000","last_unstale":"2023-12-20T06:00:39.145133+0000","last_undegraded":"2023-12-20T06:00:39.145133+0000","last_fullsized":"2023-12-20T06:00:39.145133+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:00:08.769947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.31","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637519+0000","last_change":"2023-12-20T06:00:37.644413+0000","last_active":"2023-12-20T06:00:38.637519+0000","last_peered":"2023-12-20T06:00:38.637519+0000","last_clean":"2023-12-20T06:00:38.637519+0000","last_became_active":"2023-12-20T06:00:37.644092+0000","last_became_peered":"2023-12-20T06:00:37.644092+0000","last_unstale":"2023-12-20T06:00:38.637519+0000","last_undegraded":"2023-12-20T06:00:38.637519+0000","last_fullsized":"2023-12-20T06:00:38.637519+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:44:59.830860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.30","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626598+0000","last_change":"2023-12-20T06:00:37.641854+0000","last_active":"2023-12-20T06:00:38.626598+0000","last_peered":"2023-12-20T06:00:38.626598+0000","last_clean":"2023-12-20T06:00:38.626598+0000","last_became_active":"2023-12-20T06:00:37.641576+0000","last_became_peered":"2023-12-20T06:00:37.641576+0000","last_unstale":"2023-12-20T06:00:38.626598+0000","last_undegraded":"2023-12-20T06:00:38.626598+0000","last_fullsized":"2023-12-20T06:00:38.626598+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:35.973628+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.2f","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148729+0000","last_change":"2023-12-20T06:00:37.643279+0000","last_active":"2023-12-20T06:00:39.148729+0000","last_peered":"2023-12-20T06:00:39.148729+0000","last_clean":"2023-12-20T06:00:39.148729+0000","last_became_active":"2023-12-20T06:00:37.642926+0000","last_became_peered":"2023-12-20T06:00:37.642926+0000","last_unstale":"2023-12-20T06:00:39.148729+0000","last_undegraded":"2023-12-20T06:00:39.148729+0000","last_fullsized":"2023-12-20T06:00:39.148729+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:29:32.185935+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2e","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694509+0000","last_change":"2023-12-20T06:00:37.634685+0000","last_active":"2023-12-20T06:00:38.694509+0000","last_peered":"2023-12-20T06:00:38.694509+0000","last_clean":"2023-12-20T06:00:38.694509+0000","last_became_active":"2023-12-20T06:00:37.630879+0000","last_became_peered":"2023-12-20T06:00:37.630879+0000","last_unstale":"2023-12-20T06:00:38.694509+0000","last_undegraded":"2023-12-20T06:00:38.694509+0000","last_fullsized":"2023-12-20T06:00:38.694509+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:39:17.523069+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2d","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148492+0000","last_change":"2023-12-20T06:00:37.631473+0000","last_active":"2023-12-20T06:00:39.148492+0000","last_peered":"2023-12-20T06:00:39.148492+0000","last_clean":"2023-12-20T06:00:39.148492+0000","last_became_active":"2023-12-20T06:00:37.631218+0000","last_became_peered":"2023-12-20T06:00:37.631218+0000","last_unstale":"2023-12-20T06:00:39.148492+0000","last_undegraded":"2023-12-20T06:00:39.148492+0000","last_fullsized":"2023-12-20T06:00:39.148492+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:15:49.306568+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145273+0000","last_change":"2023-12-20T06:00:37.642916+0000","last_active":"2023-12-20T06:00:39.145273+0000","last_peered":"2023-12-20T06:00:39.145273+0000","last_clean":"2023-12-20T06:00:39.145273+0000","last_became_active":"2023-12-20T06:00:37.642539+0000","last_became_peered":"2023-12-20T06:00:37.642539+0000","last_unstale":"2023-12-20T06:00:39.145273+0000","last_undegraded":"2023-12-20T06:00:39.145273+0000","last_fullsized":"2023-12-20T06:00:39.145273+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:11:34.374073+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.2b","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148284+0000","last_change":"2023-12-20T06:00:37.646485+0000","last_active":"2023-12-20T06:00:39.148284+0000","last_peered":"2023-12-20T06:00:39.148284+0000","last_clean":"2023-12-20T06:00:39.148284+0000","last_became_active":"2023-12-20T06:00:37.646179+0000","last_became_peered":"2023-12-20T06:00:37.646179+0000","last_unstale":"2023-12-20T06:00:39.148284+0000","last_undegraded":"2023-12-20T06:00:39.148284+0000","last_fullsized":"2023-12-20T06:00:39.148284+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:27:45.042505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.2a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.948772+0000","last_change":"2023-12-20T06:00:37.625313+0000","last_active":"2023-12-20T06:00:38.948772+0000","last_peered":"2023-12-20T06:00:38.948772+0000","last_clean":"2023-12-20T06:00:38.948772+0000","last_became_active":"2023-12-20T06:00:37.624988+0000","last_became_peered":"2023-12-20T06:00:37.624988+0000","last_unstale":"2023-12-20T06:00:38.948772+0000","last_undegraded":"2023-12-20T06:00:38.948772+0000","last_fullsized":"2023-12-20T06:00:38.948772+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:29.959480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.29","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147743+0000","last_change":"2023-12-20T06:00:37.623838+0000","last_active":"2023-12-20T06:00:39.147743+0000","last_peered":"2023-12-20T06:00:39.147743+0000","last_clean":"2023-12-20T06:00:39.147743+0000","last_became_active":"2023-12-20T06:00:37.623556+0000","last_became_peered":"2023-12-20T06:00:37.623556+0000","last_unstale":"2023-12-20T06:00:39.147743+0000","last_undegraded":"2023-12-20T06:00:39.147743+0000","last_fullsized":"2023-12-20T06:00:39.147743+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:05:34.079438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.28","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.948758+0000","last_change":"2023-12-20T06:00:37.639847+0000","last_active":"2023-12-20T06:00:38.948758+0000","last_peered":"2023-12-20T06:00:38.948758+0000","last_clean":"2023-12-20T06:00:38.948758+0000","last_became_active":"2023-12-20T06:00:37.639562+0000","last_became_peered":"2023-12-20T06:00:37.639562+0000","last_unstale":"2023-12-20T06:00:38.948758+0000","last_undegraded":"2023-12-20T06:00:38.948758+0000","last_fullsized":"2023-12-20T06:00:38.948758+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:16.250610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.27","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145599+0000","last_change":"2023-12-20T06:00:37.638096+0000","last_active":"2023-12-20T06:00:39.145599+0000","last_peered":"2023-12-20T06:00:39.145599+0000","last_clean":"2023-12-20T06:00:39.145599+0000","last_became_active":"2023-12-20T06:00:37.637427+0000","last_became_peered":"2023-12-20T06:00:37.637427+0000","last_unstale":"2023-12-20T06:00:39.145599+0000","last_undegraded":"2023-12-20T06:00:39.145599+0000","last_fullsized":"2023-12-20T06:00:39.145599+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:16.250610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.26","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626924+0000","last_change":"2023-12-20T06:00:37.630248+0000","last_active":"2023-12-20T06:00:38.626924+0000","last_peered":"2023-12-20T06:00:38.626924+0000","last_clean":"2023-12-20T06:00:38.626924+0000","last_became_active":"2023-12-20T06:00:37.628304+0000","last_became_peered":"2023-12-20T06:00:37.628304+0000","last_unstale":"2023-12-20T06:00:38.626924+0000","last_undegraded":"2023-12-20T06:00:38.626924+0000","last_fullsized":"2023-12-20T06:00:38.626924+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:59:18.162783+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.25","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145386+0000","last_change":"2023-12-20T06:00:37.642946+0000","last_active":"2023-12-20T06:00:39.145386+0000","last_peered":"2023-12-20T06:00:39.145386+0000","last_clean":"2023-12-20T06:00:39.145386+0000","last_became_active":"2023-12-20T06:00:37.642617+0000","last_became_peered":"2023-12-20T06:00:37.642617+0000","last_unstale":"2023-12-20T06:00:39.145386+0000","last_undegraded":"2023-12-20T06:00:39.145386+0000","last_fullsized":"2023-12-20T06:00:39.145386+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:53:17.598471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.24","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148264+0000","last_change":"2023-12-20T06:00:37.634016+0000","last_active":"2023-12-20T06:00:39.148264+0000","last_peered":"2023-12-20T06:00:39.148264+0000","last_clean":"2023-12-20T06:00:39.148264+0000","last_became_active":"2023-12-20T06:00:37.633656+0000","last_became_peered":"2023-12-20T06:00:37.633656+0000","last_unstale":"2023-12-20T06:00:39.148264+0000","last_undegraded":"2023-12-20T06:00:39.148264+0000","last_fullsized":"2023-12-20T06:00:39.148264+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:29.959480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.23","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145485+0000","last_change":"2023-12-20T06:00:37.637830+0000","last_active":"2023-12-20T06:00:39.145485+0000","last_peered":"2023-12-20T06:00:39.145485+0000","last_clean":"2023-12-20T06:00:39.145485+0000","last_became_active":"2023-12-20T06:00:37.637500+0000","last_became_peered":"2023-12-20T06:00:37.637500+0000","last_unstale":"2023-12-20T06:00:39.145485+0000","last_undegraded":"2023-12-20T06:00:39.145485+0000","last_fullsized":"2023-12-20T06:00:39.145485+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:37:11.971643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.22","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637025+0000","last_change":"2023-12-20T06:00:37.634900+0000","last_active":"2023-12-20T06:00:38.637025+0000","last_peered":"2023-12-20T06:00:38.637025+0000","last_clean":"2023-12-20T06:00:38.637025+0000","last_became_active":"2023-12-20T06:00:37.634619+0000","last_became_peered":"2023-12-20T06:00:37.634619+0000","last_unstale":"2023-12-20T06:00:38.637025+0000","last_undegraded":"2023-12-20T06:00:38.637025+0000","last_fullsized":"2023-12-20T06:00:38.637025+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:46:46.698903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.21","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637186+0000","last_change":"2023-12-20T06:00:37.625193+0000","last_active":"2023-12-20T06:00:38.637186+0000","last_peered":"2023-12-20T06:00:38.637186+0000","last_clean":"2023-12-20T06:00:38.637186+0000","last_became_active":"2023-12-20T06:00:37.624878+0000","last_became_peered":"2023-12-20T06:00:37.624878+0000","last_unstale":"2023-12-20T06:00:38.637186+0000","last_undegraded":"2023-12-20T06:00:38.637186+0000","last_fullsized":"2023-12-20T06:00:38.637186+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:29:50.461399+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.20","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694133+0000","last_change":"2023-12-20T06:00:37.621414+0000","last_active":"2023-12-20T06:00:38.694133+0000","last_peered":"2023-12-20T06:00:38.694133+0000","last_clean":"2023-12-20T06:00:38.694133+0000","last_became_active":"2023-12-20T06:00:37.621143+0000","last_became_peered":"2023-12-20T06:00:37.621143+0000","last_unstale":"2023-12-20T06:00:38.694133+0000","last_undegraded":"2023-12-20T06:00:38.694133+0000","last_fullsized":"2023-12-20T06:00:38.694133+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:28:20.337729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.1e","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.627008+0000","last_change":"2023-12-20T06:00:37.626762+0000","last_active":"2023-12-20T06:00:38.627008+0000","last_peered":"2023-12-20T06:00:38.627008+0000","last_clean":"2023-12-20T06:00:38.627008+0000","last_became_active":"2023-12-20T06:00:37.626477+0000","last_became_peered":"2023-12-20T06:00:37.626477+0000","last_unstale":"2023-12-20T06:00:38.627008+0000","last_undegraded":"2023-12-20T06:00:38.627008+0000","last_fullsized":"2023-12-20T06:00:38.627008+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:20:13.190832+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1f","version":"191'4","reported_seq":15,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.638296+0000","last_change":"2023-12-20T06:00:37.214338+0000","last_active":"2023-12-20T06:00:38.638296+0000","last_peered":"2023-12-20T06:00:38.638296+0000","last_clean":"2023-12-20T06:00:38.638296+0000","last_became_active":"2023-12-20T06:00:37.214033+0000","last_became_peered":"2023-12-20T06:00:37.214033+0000","last_unstale":"2023-12-20T06:00:38.638296+0000","last_undegraded":"2023-12-20T06:00:38.638296+0000","last_fullsized":"2023-12-20T06:00:38.638296+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:21:50.035966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.1f","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.638009+0000","last_change":"2023-12-20T06:00:37.629859+0000","last_active":"2023-12-20T06:00:38.638009+0000","last_peered":"2023-12-20T06:00:38.638009+0000","last_clean":"2023-12-20T06:00:38.638009+0000","last_became_active":"2023-12-20T06:00:37.628728+0000","last_became_peered":"2023-12-20T06:00:37.628728+0000","last_unstale":"2023-12-20T06:00:38.638009+0000","last_undegraded":"2023-12-20T06:00:38.638009+0000","last_fullsized":"2023-12-20T06:00:38.638009+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:23:27.719929+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.1e","version":"191'1","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637810+0000","last_change":"2023-12-20T06:00:37.224852+0000","last_active":"2023-12-20T06:00:38.637810+0000","last_peered":"2023-12-20T06:00:38.637810+0000","last_clean":"2023-12-20T06:00:38.637810+0000","last_became_active":"2023-12-20T06:00:37.224546+0000","last_became_peered":"2023-12-20T06:00:37.224546+0000","last_unstale":"2023-12-20T06:00:38.637810+0000","last_undegraded":"2023-12-20T06:00:38.637810+0000","last_fullsized":"2023-12-20T06:00:38.637810+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:27:11.308119+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.1c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148237+0000","last_change":"2023-12-20T06:00:37.645573+0000","last_active":"2023-12-20T06:00:39.148237+0000","last_peered":"2023-12-20T06:00:39.148237+0000","last_clean":"2023-12-20T06:00:39.148237+0000","last_became_active":"2023-12-20T06:00:37.645119+0000","last_became_peered":"2023-12-20T06:00:37.645119+0000","last_unstale":"2023-12-20T06:00:39.148237+0000","last_undegraded":"2023-12-20T06:00:39.148237+0000","last_fullsized":"2023-12-20T06:00:39.148237+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:35:46.496105+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.1d","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148500+0000","last_change":"2023-12-20T06:00:37.226172+0000","last_active":"2023-12-20T06:00:39.148500+0000","last_peered":"2023-12-20T06:00:39.148500+0000","last_clean":"2023-12-20T06:00:39.148500+0000","last_became_active":"2023-12-20T06:00:37.225913+0000","last_became_peered":"2023-12-20T06:00:37.225913+0000","last_unstale":"2023-12-20T06:00:39.148500+0000","last_undegraded":"2023-12-20T06:00:39.148500+0000","last_fullsized":"2023-12-20T06:00:39.148500+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:51.779317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1d","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145378+0000","last_change":"2023-12-20T06:00:37.642560+0000","last_active":"2023-12-20T06:00:39.145378+0000","last_peered":"2023-12-20T06:00:39.145378+0000","last_clean":"2023-12-20T06:00:39.145378+0000","last_became_active":"2023-12-20T06:00:37.642221+0000","last_became_peered":"2023-12-20T06:00:37.642221+0000","last_unstale":"2023-12-20T06:00:39.145378+0000","last_undegraded":"2023-12-20T06:00:39.145378+0000","last_fullsized":"2023-12-20T06:00:39.145378+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:39:00.341777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.1c","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148397+0000","last_change":"2023-12-20T06:00:37.226118+0000","last_active":"2023-12-20T06:00:39.148397+0000","last_peered":"2023-12-20T06:00:39.148397+0000","last_clean":"2023-12-20T06:00:39.148397+0000","last_became_active":"2023-12-20T06:00:37.225814+0000","last_became_peered":"2023-12-20T06:00:37.225814+0000","last_unstale":"2023-12-20T06:00:39.148397+0000","last_undegraded":"2023-12-20T06:00:39.148397+0000","last_fullsized":"2023-12-20T06:00:39.148397+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:58:58.520032+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.1a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626821+0000","last_change":"2023-12-20T06:00:37.621509+0000","last_active":"2023-12-20T06:00:38.626821+0000","last_peered":"2023-12-20T06:00:38.626821+0000","last_clean":"2023-12-20T06:00:38.626821+0000","last_became_active":"2023-12-20T06:00:37.621229+0000","last_became_peered":"2023-12-20T06:00:37.621229+0000","last_unstale":"2023-12-20T06:00:38.626821+0000","last_undegraded":"2023-12-20T06:00:38.626821+0000","last_fullsized":"2023-12-20T06:00:38.626821+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:01:14.788578+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1b","version":"191'1","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145504+0000","last_change":"2023-12-20T06:00:37.219345+0000","last_active":"2023-12-20T06:00:39.145504+0000","last_peered":"2023-12-20T06:00:39.145504+0000","last_clean":"2023-12-20T06:00:39.145504+0000","last_became_active":"2023-12-20T06:00:37.219038+0000","last_became_peered":"2023-12-20T06:00:37.219038+0000","last_unstale":"2023-12-20T06:00:39.145504+0000","last_undegraded":"2023-12-20T06:00:39.145504+0000","last_fullsized":"2023-12-20T06:00:39.145504+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:40.016860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.1b","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147960+0000","last_change":"2023-12-20T06:00:37.628271+0000","last_active":"2023-12-20T06:00:39.147960+0000","last_peered":"2023-12-20T06:00:39.147960+0000","last_clean":"2023-12-20T06:00:39.147960+0000","last_became_active":"2023-12-20T06:00:37.627989+0000","last_became_peered":"2023-12-20T06:00:37.627989+0000","last_unstale":"2023-12-20T06:00:39.147960+0000","last_undegraded":"2023-12-20T06:00:39.147960+0000","last_fullsized":"2023-12-20T06:00:39.147960+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:16.250610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.1a","version":"191'1","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147858+0000","last_change":"2023-12-20T06:00:37.211925+0000","last_active":"2023-12-20T06:00:39.147858+0000","last_peered":"2023-12-20T06:00:39.147858+0000","last_clean":"2023-12-20T06:00:39.147858+0000","last_became_active":"2023-12-20T06:00:37.211620+0000","last_became_peered":"2023-12-20T06:00:37.211620+0000","last_unstale":"2023-12-20T06:00:39.147858+0000","last_undegraded":"2023-12-20T06:00:39.147858+0000","last_fullsized":"2023-12-20T06:00:39.147858+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:16.422503+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.18","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147814+0000","last_change":"2023-12-20T06:00:37.641305+0000","last_active":"2023-12-20T06:00:39.147814+0000","last_peered":"2023-12-20T06:00:39.147814+0000","last_clean":"2023-12-20T06:00:39.147814+0000","last_became_active":"2023-12-20T06:00:37.641031+0000","last_became_peered":"2023-12-20T06:00:37.641031+0000","last_unstale":"2023-12-20T06:00:39.147814+0000","last_undegraded":"2023-12-20T06:00:39.147814+0000","last_fullsized":"2023-12-20T06:00:39.147814+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:33:35.691395+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.19","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.948649+0000","last_change":"2023-12-20T06:00:37.223365+0000","last_active":"2023-12-20T06:00:38.948649+0000","last_peered":"2023-12-20T06:00:38.948649+0000","last_clean":"2023-12-20T06:00:38.948649+0000","last_became_active":"2023-12-20T06:00:37.222335+0000","last_became_peered":"2023-12-20T06:00:37.222335+0000","last_unstale":"2023-12-20T06:00:38.948649+0000","last_undegraded":"2023-12-20T06:00:38.948649+0000","last_fullsized":"2023-12-20T06:00:38.948649+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:51.779317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.19","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147839+0000","last_change":"2023-12-20T06:00:37.628030+0000","last_active":"2023-12-20T06:00:39.147839+0000","last_peered":"2023-12-20T06:00:39.147839+0000","last_clean":"2023-12-20T06:00:39.147839+0000","last_became_active":"2023-12-20T06:00:37.627776+0000","last_became_peered":"2023-12-20T06:00:37.627776+0000","last_unstale":"2023-12-20T06:00:39.147839+0000","last_undegraded":"2023-12-20T06:00:39.147839+0000","last_fullsized":"2023-12-20T06:00:39.147839+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:17:06.990952+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.18","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626689+0000","last_change":"2023-12-20T06:00:37.216185+0000","last_active":"2023-12-20T06:00:38.626689+0000","last_peered":"2023-12-20T06:00:38.626689+0000","last_clean":"2023-12-20T06:00:38.626689+0000","last_became_active":"2023-12-20T06:00:37.215850+0000","last_became_peered":"2023-12-20T06:00:37.215850+0000","last_unstale":"2023-12-20T06:00:38.626689+0000","last_undegraded":"2023-12-20T06:00:38.626689+0000","last_fullsized":"2023-12-20T06:00:38.626689+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:28:19.325527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"23.17","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694533+0000","last_change":"2023-12-20T06:00:37.638082+0000","last_active":"2023-12-20T06:00:38.694533+0000","last_peered":"2023-12-20T06:00:38.694533+0000","last_clean":"2023-12-20T06:00:38.694533+0000","last_became_active":"2023-12-20T06:00:37.637827+0000","last_became_peered":"2023-12-20T06:00:37.637827+0000","last_unstale":"2023-12-20T06:00:38.694533+0000","last_undegraded":"2023-12-20T06:00:38.694533+0000","last_fullsized":"2023-12-20T06:00:38.694533+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:04:39.343446+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.16","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148724+0000","last_change":"2023-12-20T06:00:37.222429+0000","last_active":"2023-12-20T06:00:39.148724+0000","last_peered":"2023-12-20T06:00:39.148724+0000","last_clean":"2023-12-20T06:00:39.148724+0000","last_became_active":"2023-12-20T06:00:37.222093+0000","last_became_peered":"2023-12-20T06:00:37.222093+0000","last_unstale":"2023-12-20T06:00:39.148724+0000","last_undegraded":"2023-12-20T06:00:39.148724+0000","last_fullsized":"2023-12-20T06:00:39.148724+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:41:43.772223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.14","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694368+0000","last_change":"2023-12-20T06:00:37.638142+0000","last_active":"2023-12-20T06:00:38.694368+0000","last_peered":"2023-12-20T06:00:38.694368+0000","last_clean":"2023-12-20T06:00:38.694368+0000","last_became_active":"2023-12-20T06:00:37.637872+0000","last_became_peered":"2023-12-20T06:00:37.637872+0000","last_unstale":"2023-12-20T06:00:38.694368+0000","last_undegraded":"2023-12-20T06:00:38.694368+0000","last_fullsized":"2023-12-20T06:00:38.694368+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:41:14.126316+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.15","version":"191'3","reported_seq":14,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148390+0000","last_change":"2023-12-20T06:00:37.218644+0000","last_active":"2023-12-20T06:00:39.148390+0000","last_peered":"2023-12-20T06:00:39.148390+0000","last_clean":"2023-12-20T06:00:39.148390+0000","last_became_active":"2023-12-20T06:00:37.218339+0000","last_became_peered":"2023-12-20T06:00:37.218339+0000","last_unstale":"2023-12-20T06:00:39.148390+0000","last_undegraded":"2023-12-20T06:00:39.148390+0000","last_fullsized":"2023-12-20T06:00:39.148390+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:33:02.720110+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"23.15","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637658+0000","last_change":"2023-12-20T06:00:37.645382+0000","last_active":"2023-12-20T06:00:38.637658+0000","last_peered":"2023-12-20T06:00:38.637658+0000","last_clean":"2023-12-20T06:00:38.637658+0000","last_became_active":"2023-12-20T06:00:37.645042+0000","last_became_peered":"2023-12-20T06:00:37.645042+0000","last_unstale":"2023-12-20T06:00:38.637658+0000","last_undegraded":"2023-12-20T06:00:38.637658+0000","last_fullsized":"2023-12-20T06:00:38.637658+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:06:17.032509+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.14","version":"191'6","reported_seq":17,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637369+0000","last_change":"2023-12-20T06:00:37.215497+0000","last_active":"2023-12-20T06:00:38.637369+0000","last_peered":"2023-12-20T06:00:38.637369+0000","last_clean":"2023-12-20T06:00:38.637369+0000","last_became_active":"2023-12-20T06:00:37.215159+0000","last_became_peered":"2023-12-20T06:00:37.215159+0000","last_unstale":"2023-12-20T06:00:38.637369+0000","last_undegraded":"2023-12-20T06:00:38.637369+0000","last_fullsized":"2023-12-20T06:00:38.637369+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:16:25.337846+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.12","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694240+0000","last_change":"2023-12-20T06:00:37.626217+0000","last_active":"2023-12-20T06:00:38.694240+0000","last_peered":"2023-12-20T06:00:38.694240+0000","last_clean":"2023-12-20T06:00:38.694240+0000","last_became_active":"2023-12-20T06:00:37.625927+0000","last_became_peered":"2023-12-20T06:00:37.625927+0000","last_unstale":"2023-12-20T06:00:38.694240+0000","last_undegraded":"2023-12-20T06:00:38.694240+0000","last_fullsized":"2023-12-20T06:00:38.694240+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:16:18.403896+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.13","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148052+0000","last_change":"2023-12-20T06:00:37.215547+0000","last_active":"2023-12-20T06:00:39.148052+0000","last_peered":"2023-12-20T06:00:39.148052+0000","last_clean":"2023-12-20T06:00:39.148052+0000","last_became_active":"2023-12-20T06:00:37.214866+0000","last_became_peered":"2023-12-20T06:00:37.214866+0000","last_unstale":"2023-12-20T06:00:39.148052+0000","last_undegraded":"2023-12-20T06:00:39.148052+0000","last_fullsized":"2023-12-20T06:00:39.148052+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:21:44.604741+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.13","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626964+0000","last_change":"2023-12-20T06:00:37.643805+0000","last_active":"2023-12-20T06:00:38.626964+0000","last_peered":"2023-12-20T06:00:38.626964+0000","last_clean":"2023-12-20T06:00:38.626964+0000","last_became_active":"2023-12-20T06:00:37.643534+0000","last_became_peered":"2023-12-20T06:00:37.643534+0000","last_unstale":"2023-12-20T06:00:38.626964+0000","last_undegraded":"2023-12-20T06:00:38.626964+0000","last_fullsized":"2023-12-20T06:00:38.626964+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:22:08.693939+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.12","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694340+0000","last_change":"2023-12-20T06:00:37.219599+0000","last_active":"2023-12-20T06:00:38.694340+0000","last_peered":"2023-12-20T06:00:38.694340+0000","last_clean":"2023-12-20T06:00:38.694340+0000","last_became_active":"2023-12-20T06:00:37.219331+0000","last_became_peered":"2023-12-20T06:00:37.219331+0000","last_unstale":"2023-12-20T06:00:38.694340+0000","last_undegraded":"2023-12-20T06:00:38.694340+0000","last_fullsized":"2023-12-20T06:00:38.694340+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:46:38.781709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.10","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147829+0000","last_change":"2023-12-20T06:00:37.641860+0000","last_active":"2023-12-20T06:00:39.147829+0000","last_peered":"2023-12-20T06:00:39.147829+0000","last_clean":"2023-12-20T06:00:39.147829+0000","last_became_active":"2023-12-20T06:00:37.641580+0000","last_became_peered":"2023-12-20T06:00:37.641580+0000","last_unstale":"2023-12-20T06:00:39.147829+0000","last_undegraded":"2023-12-20T06:00:39.147829+0000","last_fullsized":"2023-12-20T06:00:39.147829+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:17:33.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.11","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147954+0000","last_change":"2023-12-20T06:00:37.224918+0000","last_active":"2023-12-20T06:00:39.147954+0000","last_peered":"2023-12-20T06:00:39.147954+0000","last_clean":"2023-12-20T06:00:39.147954+0000","last_became_active":"2023-12-20T06:00:37.224414+0000","last_became_peered":"2023-12-20T06:00:37.224414+0000","last_unstale":"2023-12-20T06:00:39.147954+0000","last_undegraded":"2023-12-20T06:00:39.147954+0000","last_fullsized":"2023-12-20T06:00:39.147954+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:40:42.440314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.11","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145370+0000","last_change":"2023-12-20T06:00:37.635333+0000","last_active":"2023-12-20T06:00:39.145370+0000","last_peered":"2023-12-20T06:00:39.145370+0000","last_clean":"2023-12-20T06:00:39.145370+0000","last_became_active":"2023-12-20T06:00:37.634998+0000","last_became_peered":"2023-12-20T06:00:37.634998+0000","last_unstale":"2023-12-20T06:00:39.145370+0000","last_undegraded":"2023-12-20T06:00:39.145370+0000","last_fullsized":"2023-12-20T06:00:39.145370+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:24:54.128140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.10","version":"191'8","reported_seq":19,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637515+0000","last_change":"2023-12-20T06:00:37.217887+0000","last_active":"2023-12-20T06:00:38.637515+0000","last_peered":"2023-12-20T06:00:38.637515+0000","last_clean":"2023-12-20T06:00:38.637515+0000","last_became_active":"2023-12-20T06:00:37.217574+0000","last_became_peered":"2023-12-20T06:00:37.217574+0000","last_unstale":"2023-12-20T06:00:38.637515+0000","last_undegraded":"2023-12-20T06:00:38.637515+0000","last_fullsized":"2023-12-20T06:00:38.637515+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:31:17.951970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.e","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148704+0000","last_change":"2023-12-20T06:00:37.625335+0000","last_active":"2023-12-20T06:00:39.148704+0000","last_peered":"2023-12-20T06:00:39.148704+0000","last_clean":"2023-12-20T06:00:39.148704+0000","last_became_active":"2023-12-20T06:00:37.625017+0000","last_became_peered":"2023-12-20T06:00:37.625017+0000","last_unstale":"2023-12-20T06:00:39.148704+0000","last_undegraded":"2023-12-20T06:00:39.148704+0000","last_fullsized":"2023-12-20T06:00:39.148704+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:48:20.707786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.f","version":"191'1","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145629+0000","last_change":"2023-12-20T06:00:37.225678+0000","last_active":"2023-12-20T06:00:39.145629+0000","last_peered":"2023-12-20T06:00:39.145629+0000","last_clean":"2023-12-20T06:00:39.145629+0000","last_became_active":"2023-12-20T06:00:37.225392+0000","last_became_peered":"2023-12-20T06:00:37.225392+0000","last_unstale":"2023-12-20T06:00:39.145629+0000","last_undegraded":"2023-12-20T06:00:39.145629+0000","last_fullsized":"2023-12-20T06:00:39.145629+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:14:06.560400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.f","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694606+0000","last_change":"2023-12-20T06:00:37.633138+0000","last_active":"2023-12-20T06:00:38.694606+0000","last_peered":"2023-12-20T06:00:38.694606+0000","last_clean":"2023-12-20T06:00:38.694606+0000","last_became_active":"2023-12-20T06:00:37.632842+0000","last_became_peered":"2023-12-20T06:00:37.632842+0000","last_unstale":"2023-12-20T06:00:38.694606+0000","last_undegraded":"2023-12-20T06:00:38.694606+0000","last_fullsized":"2023-12-20T06:00:38.694606+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:04:03.398362+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.e","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145373+0000","last_change":"2023-12-20T06:00:37.218432+0000","last_active":"2023-12-20T06:00:39.145373+0000","last_peered":"2023-12-20T06:00:39.145373+0000","last_clean":"2023-12-20T06:00:39.145373+0000","last_became_active":"2023-12-20T06:00:37.218149+0000","last_became_peered":"2023-12-20T06:00:37.218149+0000","last_unstale":"2023-12-20T06:00:39.145373+0000","last_undegraded":"2023-12-20T06:00:39.145373+0000","last_fullsized":"2023-12-20T06:00:39.145373+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:51.779317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.c","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148366+0000","last_change":"2023-12-20T06:00:37.630176+0000","last_active":"2023-12-20T06:00:39.148366+0000","last_peered":"2023-12-20T06:00:39.148366+0000","last_clean":"2023-12-20T06:00:39.148366+0000","last_became_active":"2023-12-20T06:00:37.629930+0000","last_became_peered":"2023-12-20T06:00:37.629930+0000","last_unstale":"2023-12-20T06:00:39.148366+0000","last_undegraded":"2023-12-20T06:00:39.148366+0000","last_fullsized":"2023-12-20T06:00:39.148366+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:21:31.062877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"22.d","version":"191'1","reported_seq":12,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.638198+0000","last_change":"2023-12-20T06:00:37.221697+0000","last_active":"2023-12-20T06:00:38.638198+0000","last_peered":"2023-12-20T06:00:38.638198+0000","last_clean":"2023-12-20T06:00:38.638198+0000","last_became_active":"2023-12-20T06:00:37.221423+0000","last_became_peered":"2023-12-20T06:00:37.221423+0000","last_unstale":"2023-12-20T06:00:38.638198+0000","last_undegraded":"2023-12-20T06:00:38.638198+0000","last_fullsized":"2023-12-20T06:00:38.638198+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:20:03.650140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.d","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.627189+0000","last_change":"2023-12-20T06:00:37.630088+0000","last_active":"2023-12-20T06:00:38.627189+0000","last_peered":"2023-12-20T06:00:38.627189+0000","last_clean":"2023-12-20T06:00:38.627189+0000","last_became_active":"2023-12-20T06:00:37.629842+0000","last_became_peered":"2023-12-20T06:00:37.629842+0000","last_unstale":"2023-12-20T06:00:38.627189+0000","last_undegraded":"2023-12-20T06:00:38.627189+0000","last_fullsized":"2023-12-20T06:00:38.627189+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:02:30.560429+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.c","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.638284+0000","last_change":"2023-12-20T06:00:37.217042+0000","last_active":"2023-12-20T06:00:38.638284+0000","last_peered":"2023-12-20T06:00:38.638284+0000","last_clean":"2023-12-20T06:00:38.638284+0000","last_became_active":"2023-12-20T06:00:37.216744+0000","last_became_peered":"2023-12-20T06:00:37.216744+0000","last_unstale":"2023-12-20T06:00:38.638284+0000","last_undegraded":"2023-12-20T06:00:38.638284+0000","last_fullsized":"2023-12-20T06:00:38.638284+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:40.016860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.a","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145290+0000","last_change":"2023-12-20T06:00:37.636981+0000","last_active":"2023-12-20T06:00:39.145290+0000","last_peered":"2023-12-20T06:00:39.145290+0000","last_clean":"2023-12-20T06:00:39.145290+0000","last_became_active":"2023-12-20T06:00:37.636712+0000","last_became_peered":"2023-12-20T06:00:37.636712+0000","last_unstale":"2023-12-20T06:00:39.145290+0000","last_undegraded":"2023-12-20T06:00:39.145290+0000","last_fullsized":"2023-12-20T06:00:39.145290+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:19:11.247792+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.b","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148140+0000","last_change":"2023-12-20T06:00:37.229378+0000","last_active":"2023-12-20T06:00:39.148140+0000","last_peered":"2023-12-20T06:00:39.148140+0000","last_clean":"2023-12-20T06:00:39.148140+0000","last_became_active":"2023-12-20T06:00:37.229043+0000","last_became_peered":"2023-12-20T06:00:37.229043+0000","last_unstale":"2023-12-20T06:00:39.148140+0000","last_undegraded":"2023-12-20T06:00:39.148140+0000","last_fullsized":"2023-12-20T06:00:39.148140+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:41:40.016860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.b","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626859+0000","last_change":"2023-12-20T06:00:37.641759+0000","last_active":"2023-12-20T06:00:38.626859+0000","last_peered":"2023-12-20T06:00:38.626859+0000","last_clean":"2023-12-20T06:00:38.626859+0000","last_became_active":"2023-12-20T06:00:37.641491+0000","last_became_peered":"2023-12-20T06:00:37.641491+0000","last_unstale":"2023-12-20T06:00:38.626859+0000","last_undegraded":"2023-12-20T06:00:38.626859+0000","last_fullsized":"2023-12-20T06:00:38.626859+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:17:57.727737+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.a","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637888+0000","last_change":"2023-12-20T06:00:37.218888+0000","last_active":"2023-12-20T06:00:38.637888+0000","last_peered":"2023-12-20T06:00:38.637888+0000","last_clean":"2023-12-20T06:00:38.637888+0000","last_became_active":"2023-12-20T06:00:37.218615+0000","last_became_peered":"2023-12-20T06:00:37.218615+0000","last_unstale":"2023-12-20T06:00:38.637888+0000","last_undegraded":"2023-12-20T06:00:38.637888+0000","last_fullsized":"2023-12-20T06:00:38.637888+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:17:51.779317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.8","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637606+0000","last_change":"2023-12-20T06:00:37.649371+0000","last_active":"2023-12-20T06:00:38.637606+0000","last_peered":"2023-12-20T06:00:38.637606+0000","last_clean":"2023-12-20T06:00:38.637606+0000","last_became_active":"2023-12-20T06:00:37.649036+0000","last_became_peered":"2023-12-20T06:00:37.649036+0000","last_unstale":"2023-12-20T06:00:38.637606+0000","last_undegraded":"2023-12-20T06:00:38.637606+0000","last_fullsized":"2023-12-20T06:00:38.637606+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:17:33.426278+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"22.9","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148045+0000","last_change":"2023-12-20T06:00:37.219401+0000","last_active":"2023-12-20T06:00:39.148045+0000","last_peered":"2023-12-20T06:00:39.148045+0000","last_clean":"2023-12-20T06:00:39.148045+0000","last_became_active":"2023-12-20T06:00:37.219103+0000","last_became_peered":"2023-12-20T06:00:37.219103+0000","last_unstale":"2023-12-20T06:00:39.148045+0000","last_undegraded":"2023-12-20T06:00:39.148045+0000","last_fullsized":"2023-12-20T06:00:39.148045+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:50:42.530486+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.9","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147921+0000","last_change":"2023-12-20T06:00:37.624653+0000","last_active":"2023-12-20T06:00:39.147921+0000","last_peered":"2023-12-20T06:00:39.147921+0000","last_clean":"2023-12-20T06:00:39.147921+0000","last_became_active":"2023-12-20T06:00:37.624346+0000","last_became_peered":"2023-12-20T06:00:37.624346+0000","last_unstale":"2023-12-20T06:00:39.147921+0000","last_undegraded":"2023-12-20T06:00:39.147921+0000","last_fullsized":"2023-12-20T06:00:39.147921+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:37.912584+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.8","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637734+0000","last_change":"2023-12-20T06:00:37.221091+0000","last_active":"2023-12-20T06:00:38.637734+0000","last_peered":"2023-12-20T06:00:38.637734+0000","last_clean":"2023-12-20T06:00:38.637734+0000","last_became_active":"2023-12-20T06:00:37.220804+0000","last_became_peered":"2023-12-20T06:00:37.220804+0000","last_unstale":"2023-12-20T06:00:38.637734+0000","last_undegraded":"2023-12-20T06:00:38.637734+0000","last_fullsized":"2023-12-20T06:00:38.637734+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:14:06.560400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"23.6","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694604+0000","last_change":"2023-12-20T06:00:37.635358+0000","last_active":"2023-12-20T06:00:38.694604+0000","last_peered":"2023-12-20T06:00:38.694604+0000","last_clean":"2023-12-20T06:00:38.694604+0000","last_became_active":"2023-12-20T06:00:37.635083+0000","last_became_peered":"2023-12-20T06:00:37.635083+0000","last_unstale":"2023-12-20T06:00:38.694604+0000","last_undegraded":"2023-12-20T06:00:38.694604+0000","last_fullsized":"2023-12-20T06:00:38.694604+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:15:12.634489+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.7","version":"191'2","reported_seq":13,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694637+0000","last_change":"2023-12-20T06:00:37.228250+0000","last_active":"2023-12-20T06:00:38.694637+0000","last_peered":"2023-12-20T06:00:38.694637+0000","last_clean":"2023-12-20T06:00:38.694637+0000","last_became_active":"2023-12-20T06:00:37.227876+0000","last_became_peered":"2023-12-20T06:00:37.227876+0000","last_unstale":"2023-12-20T06:00:38.694637+0000","last_undegraded":"2023-12-20T06:00:38.694637+0000","last_fullsized":"2023-12-20T06:00:38.694637+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:45:50.803107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.7","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626939+0000","last_change":"2023-12-20T06:00:37.624465+0000","last_active":"2023-12-20T06:00:38.626939+0000","last_peered":"2023-12-20T06:00:38.626939+0000","last_clean":"2023-12-20T06:00:38.626939+0000","last_became_active":"2023-12-20T06:00:37.624194+0000","last_became_peered":"2023-12-20T06:00:37.624194+0000","last_unstale":"2023-12-20T06:00:38.626939+0000","last_undegraded":"2023-12-20T06:00:38.626939+0000","last_fullsized":"2023-12-20T06:00:38.626939+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:34:44.122025+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.6","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145479+0000","last_change":"2023-12-20T06:00:37.214328+0000","last_active":"2023-12-20T06:00:39.145479+0000","last_peered":"2023-12-20T06:00:39.145479+0000","last_clean":"2023-12-20T06:00:39.145479+0000","last_became_active":"2023-12-20T06:00:37.214019+0000","last_became_peered":"2023-12-20T06:00:37.214019+0000","last_unstale":"2023-12-20T06:00:39.145479+0000","last_undegraded":"2023-12-20T06:00:39.145479+0000","last_fullsized":"2023-12-20T06:00:39.145479+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:21:50.035966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"23.4","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.148367+0000","last_change":"2023-12-20T06:00:37.632210+0000","last_active":"2023-12-20T06:00:39.148367+0000","last_peered":"2023-12-20T06:00:39.148367+0000","last_clean":"2023-12-20T06:00:39.148367+0000","last_became_active":"2023-12-20T06:00:37.631933+0000","last_became_peered":"2023-12-20T06:00:37.631933+0000","last_unstale":"2023-12-20T06:00:39.148367+0000","last_undegraded":"2023-12-20T06:00:39.148367+0000","last_fullsized":"2023-12-20T06:00:39.148367+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:11:27.937059+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"22.5","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.948989+0000","last_change":"2023-12-20T06:00:37.222951+0000","last_active":"2023-12-20T06:00:38.948989+0000","last_peered":"2023-12-20T06:00:38.948989+0000","last_clean":"2023-12-20T06:00:38.948989+0000","last_became_active":"2023-12-20T06:00:37.222609+0000","last_became_peered":"2023-12-20T06:00:37.222609+0000","last_unstale":"2023-12-20T06:00:38.948989+0000","last_undegraded":"2023-12-20T06:00:38.948989+0000","last_fullsized":"2023-12-20T06:00:38.948989+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:20:03.650140+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.5","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694456+0000","last_change":"2023-12-20T06:00:37.644721+0000","last_active":"2023-12-20T06:00:38.694456+0000","last_peered":"2023-12-20T06:00:38.694456+0000","last_clean":"2023-12-20T06:00:38.694456+0000","last_became_active":"2023-12-20T06:00:37.644415+0000","last_became_peered":"2023-12-20T06:00:37.644415+0000","last_unstale":"2023-12-20T06:00:38.694456+0000","last_undegraded":"2023-12-20T06:00:38.694456+0000","last_fullsized":"2023-12-20T06:00:38.694456+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:16:46.822701+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.4","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694390+0000","last_change":"2023-12-20T06:00:37.217692+0000","last_active":"2023-12-20T06:00:38.694390+0000","last_peered":"2023-12-20T06:00:38.694390+0000","last_clean":"2023-12-20T06:00:38.694390+0000","last_became_active":"2023-12-20T06:00:37.217399+0000","last_became_peered":"2023-12-20T06:00:37.217399+0000","last_unstale":"2023-12-20T06:00:38.694390+0000","last_undegraded":"2023-12-20T06:00:38.694390+0000","last_fullsized":"2023-12-20T06:00:38.694390+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:44:48.335840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"23.2","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694236+0000","last_change":"2023-12-20T06:00:37.625627+0000","last_active":"2023-12-20T06:00:38.694236+0000","last_peered":"2023-12-20T06:00:38.694236+0000","last_clean":"2023-12-20T06:00:38.694236+0000","last_became_active":"2023-12-20T06:00:37.623880+0000","last_became_peered":"2023-12-20T06:00:37.623880+0000","last_unstale":"2023-12-20T06:00:38.694236+0000","last_undegraded":"2023-12-20T06:00:38.694236+0000","last_fullsized":"2023-12-20T06:00:38.694236+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:06:23.484756+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.3","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637385+0000","last_change":"2023-12-20T06:00:37.218414+0000","last_active":"2023-12-20T06:00:38.637385+0000","last_peered":"2023-12-20T06:00:38.637385+0000","last_clean":"2023-12-20T06:00:38.637385+0000","last_became_active":"2023-12-20T06:00:37.217198+0000","last_became_peered":"2023-12-20T06:00:37.217198+0000","last_unstale":"2023-12-20T06:00:38.637385+0000","last_undegraded":"2023-12-20T06:00:38.637385+0000","last_fullsized":"2023-12-20T06:00:38.637385+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:09:55.705372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.3","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637321+0000","last_change":"2023-12-20T06:00:37.624175+0000","last_active":"2023-12-20T06:00:38.637321+0000","last_peered":"2023-12-20T06:00:38.637321+0000","last_clean":"2023-12-20T06:00:38.637321+0000","last_became_active":"2023-12-20T06:00:37.623837+0000","last_became_peered":"2023-12-20T06:00:37.623837+0000","last_unstale":"2023-12-20T06:00:38.637321+0000","last_undegraded":"2023-12-20T06:00:38.637321+0000","last_fullsized":"2023-12-20T06:00:38.637321+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:56:26.222551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"22.2","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.147887+0000","last_change":"2023-12-20T06:00:37.217996+0000","last_active":"2023-12-20T06:00:39.147887+0000","last_peered":"2023-12-20T06:00:39.147887+0000","last_clean":"2023-12-20T06:00:39.147887+0000","last_became_active":"2023-12-20T06:00:37.217046+0000","last_became_peered":"2023-12-20T06:00:37.217046+0000","last_unstale":"2023-12-20T06:00:39.147887+0000","last_undegraded":"2023-12-20T06:00:39.147887+0000","last_fullsized":"2023-12-20T06:00:39.147887+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:11:09.715643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"23.0","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.626657+0000","last_change":"2023-12-20T06:00:37.641033+0000","last_active":"2023-12-20T06:00:38.626657+0000","last_peered":"2023-12-20T06:00:38.626657+0000","last_clean":"2023-12-20T06:00:38.626657+0000","last_became_active":"2023-12-20T06:00:37.640765+0000","last_became_peered":"2023-12-20T06:00:37.640765+0000","last_unstale":"2023-12-20T06:00:38.626657+0000","last_undegraded":"2023-12-20T06:00:38.626657+0000","last_fullsized":"2023-12-20T06:00:38.626657+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:08:10.404165+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"22.1","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.948811+0000","last_change":"2023-12-20T06:00:37.221873+0000","last_active":"2023-12-20T06:00:38.948811+0000","last_peered":"2023-12-20T06:00:38.948811+0000","last_clean":"2023-12-20T06:00:38.948811+0000","last_became_active":"2023-12-20T06:00:37.221558+0000","last_became_peered":"2023-12-20T06:00:37.221558+0000","last_unstale":"2023-12-20T06:00:38.948811+0000","last_undegraded":"2023-12-20T06:00:38.948811+0000","last_fullsized":"2023-12-20T06:00:38.948811+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:14:06.560400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"23.1","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:39.145235+0000","last_change":"2023-12-20T06:00:37.634654+0000","last_active":"2023-12-20T06:00:39.145235+0000","last_peered":"2023-12-20T06:00:39.145235+0000","last_clean":"2023-12-20T06:00:39.145235+0000","last_became_active":"2023-12-20T06:00:37.634353+0000","last_became_peered":"2023-12-20T06:00:37.634353+0000","last_unstale":"2023-12-20T06:00:39.145235+0000","last_undegraded":"2023-12-20T06:00:39.145235+0000","last_fullsized":"2023-12-20T06:00:39.145235+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:48:38.916609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"22.0","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.637168+0000","last_change":"2023-12-20T06:00:37.214833+0000","last_active":"2023-12-20T06:00:38.637168+0000","last_peered":"2023-12-20T06:00:38.637168+0000","last_clean":"2023-12-20T06:00:38.637168+0000","last_became_active":"2023-12-20T06:00:37.214365+0000","last_became_peered":"2023-12-20T06:00:37.214365+0000","last_unstale":"2023-12-20T06:00:38.637168+0000","last_undegraded":"2023-12-20T06:00:38.637168+0000","last_fullsized":"2023-12-20T06:00:38.637168+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:50:14.111680+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"23.16","version":"0'0","reported_seq":10,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.694540+0000","last_change":"2023-12-20T06:00:37.639634+0000","last_active":"2023-12-20T06:00:38.694540+0000","last_peered":"2023-12-20T06:00:38.694540+0000","last_clean":"2023-12-20T06:00:38.694540+0000","last_became_active":"2023-12-20T06:00:37.639375+0000","last_became_peered":"2023-12-20T06:00:37.639375+0000","last_unstale":"2023-12-20T06:00:38.694540+0000","last_undegraded":"2023-12-20T06:00:38.694540+0000","last_fullsized":"2023-12-20T06:00:38.694540+0000","mapping_epoch":190,"log_start":"0'0","ondisk_log_start":"0'0","created":190,"last_epoch_clean":191,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:37.188753+0000","last_clean_scrub_stamp":"2023-12-20T06:00:37.188753+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:11:22.988664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"22.17","version":"0'0","reported_seq":11,"reported_epoch":192,"state":"active+clean","last_fresh":"2023-12-20T06:00:38.638032+0000","last_change":"2023-12-20T06:00:37.210339+0000","last_active":"2023-12-20T06:00:38.638032+0000","last_peered":"2023-12-20T06:00:38.638032+0000","last_clean":"2023-12-20T06:00:38.638032+0000","last_became_active":"2023-12-20T06:00:37.210043+0000","last_became_peered":"2023-12-20T06:00:37.210043+0000","last_unstale":"2023-12-20T06:00:38.638032+0000","last_undegraded":"2023-12-20T06:00:38.638032+0000","last_fullsized":"2023-12-20T06:00:38.638032+0000","mapping_epoch":189,"log_start":"0'0","ondisk_log_start":"0'0","created":189,"last_epoch_clean":190,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:00:36.176550+0000","last_clean_scrub_stamp":"2023-12-20T06:00:36.176550+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:36:19.074083+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"180'3179","reported_seq":3899,"reported_epoch":191,"state":"active+clean","last_fresh":"2023-12-20T06:00:37.640057+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:00:37.640057+0000","last_peered":"2023-12-20T06:00:37.640057+0000","last_clean":"2023-12-20T06:00:37.640057+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:00:37.640057+0000","last_undegraded":"2023-12-20T06:00:37.640057+0000","last_fullsized":"2023-12-20T06:00:37.640057+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3179,"log_dups_size":0,"ondisk_log_size":3179,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1693,"num_read_kb":3437,"num_write":481,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":23,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":22,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1693,"num_read_kb":3437,"num_write":481,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3179,"ondisk_log_size":3179,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575949,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56140,"kb_used_data":2948,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315700,"statfs":{"total":96636764160,"available":96579276800,"internally_reserved":0,"allocated":3018752,"data_stored":1753309,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.71399999999999997,"15min":0.73999999999999999},"min":{"1min":0.57199999999999995,"5min":0.439,"15min":0.439},"max":{"1min":0.90600000000000003,"5min":0.97099999999999997,"15min":1.042},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.71999999999999997,"5min":0.73699999999999999,"15min":0.749},"min":{"1min":0.432,"5min":0.432,"15min":0.34499999999999997},"max":{"1min":0.94799999999999995,"5min":0.99099999999999999,"15min":1.161},"last":0.56699999999999995}]},{"osd":1,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.748,"5min":0.745,"15min":0.754},"min":{"1min":0.49099999999999999,"5min":0.46000000000000002,"15min":0.432},"max":{"1min":0.94799999999999995,"5min":1.006,"15min":1.1279999999999999},"last":0.51300000000000001},{"interface":"front","average":{"1min":0.754,"5min":0.73799999999999999,"15min":0.75600000000000001},"min":{"1min":0.51100000000000001,"5min":0.48999999999999999,"15min":0.46999999999999997},"max":{"1min":0.879,"5min":0.96699999999999997,"15min":1.032},"last":0.63800000000000001}]},{"osd":2,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73899999999999999,"15min":0.749},"min":{"1min":0.48699999999999999,"5min":0.48699999999999999,"15min":0.48299999999999998},"max":{"1min":0.86099999999999999,"5min":0.95599999999999996,"15min":1.0409999999999999},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.78300000000000003,"5min":0.75,"15min":0.76800000000000002},"min":{"1min":0.53900000000000003,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":1.0089999999999999,"5min":1.0189999999999999,"15min":1.0409999999999999},"last":0.59199999999999997}]},{"osd":3,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.72999999999999998,"5min":0.73799999999999999,"15min":0.747},"min":{"1min":0.53500000000000003,"5min":0.53500000000000003,"15min":0.47799999999999998},"max":{"1min":0.88900000000000001,"5min":0.98599999999999999,"15min":1.0069999999999999},"last":0.60699999999999998},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77500000000000002,"15min":0.79200000000000004},"min":{"1min":0.503,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":0.96699999999999997,"5min":0.96699999999999997,"15min":1.0840000000000001},"last":0.55500000000000005}]},{"osd":4,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.58699999999999997,"5min":0.58399999999999996,"15min":0.59499999999999997},"min":{"1min":0.39800000000000002,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.91800000000000004,"5min":0.91800000000000004,"15min":1.04},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.63900000000000001,"5min":0.64100000000000001,"15min":0.64400000000000002},"min":{"1min":0.41399999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":1.02,"5min":1.02,"15min":1.02},"last":0.68999999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67000000000000004,"15min":0.66400000000000003},"min":{"1min":0.53600000000000003,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.97499999999999998,"5min":0.99199999999999999,"15min":1.1120000000000001},"last":0.624},{"interface":"front","average":{"1min":0.72899999999999998,"5min":0.69299999999999995,"15min":0.69199999999999995},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.45600000000000002},"max":{"1min":1.0229999999999999,"5min":1.0229999999999999,"15min":1.103},"last":0.76700000000000002}]},{"osd":6,"last update":"Wed Dec 20 06:00:20 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.74299999999999999,"15min":0.749},"min":{"1min":0.54400000000000004,"5min":0.498,"15min":0.45500000000000002},"max":{"1min":1.0449999999999999,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.79900000000000004},{"interface":"front","average":{"1min":0.73899999999999999,"5min":0.71099999999999997,"15min":0.72399999999999998},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.0740000000000001,"5min":1.0740000000000001,"15min":1.1539999999999999},"last":0.89600000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575949,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51444,"kb_used_data":2348,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320396,"statfs":{"total":96636764160,"available":96584085504,"internally_reserved":0,"allocated":2404352,"data_stored":1160657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.74199999999999999,"15min":0.747},"min":{"1min":0.35799999999999998,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92300000000000004,"5min":0.99099999999999999,"15min":1.0980000000000001},"last":0.75},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.74299999999999999,"15min":0.754},"min":{"1min":0.376,"5min":0.376,"15min":0.376},"max":{"1min":0.94999999999999996,"5min":1.016,"15min":1.05},"last":0.61799999999999999}]},{"osd":1,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.72899999999999998,"15min":0.73799999999999999},"min":{"1min":0.40100000000000002,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.98499999999999999,"5min":0.996,"15min":0.996},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.76200000000000001,"5min":0.77700000000000002,"15min":0.77700000000000002},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.97299999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.879}]},{"osd":2,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.77900000000000003,"15min":0.78500000000000003},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.47299999999999998},"max":{"1min":0.90900000000000003,"5min":1.069,"15min":1.069},"last":0.92900000000000005},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.76100000000000001,"15min":0.76700000000000002},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.96199999999999997,"5min":1.073,"15min":1.0760000000000001},"last":0.56499999999999995}]},{"osd":3,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.78900000000000003,"15min":0.78700000000000003},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.042,"5min":1.077,"15min":1.077},"last":0.72199999999999998},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.75700000000000001,"15min":0.76400000000000001},"min":{"1min":0.54300000000000004,"5min":0.51200000000000001,"15min":0.503},"max":{"1min":1.0309999999999999,"5min":1.052,"15min":1.052},"last":0.70599999999999996}]},{"osd":4,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.627,"15min":0.63100000000000001},"min":{"1min":0.435,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.95799999999999996,"5min":0.98199999999999998,"15min":1.0529999999999999},"last":1.05},{"interface":"front","average":{"1min":0.66100000000000003,"5min":0.64800000000000002,"15min":0.64700000000000002},"min":{"1min":0.47399999999999998,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.1339999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.67500000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.64900000000000002,"5min":0.63400000000000001,"15min":0.63900000000000001},"min":{"1min":0.45500000000000002,"5min":0.377,"15min":0.377},"max":{"1min":0.86199999999999999,"5min":0.98599999999999999,"15min":1.0009999999999999},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.70799999999999996,"15min":0.70399999999999996},"min":{"1min":0.45700000000000002,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":1.016}]},{"osd":7,"last update":"Wed Dec 20 06:00:31 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73099999999999998,"15min":0.73099999999999998},"min":{"1min":0.504,"5min":0.48899999999999999,"15min":0.45300000000000001},"max":{"1min":1.103,"5min":1.1240000000000001,"15min":1.1240000000000001},"last":0.81599999999999995},{"interface":"front","average":{"1min":0.74199999999999999,"5min":0.71299999999999997,"15min":0.72799999999999998},"min":{"1min":0.53200000000000003,"5min":0.41999999999999998,"15min":0.41999999999999998},"max":{"1min":1.0169999999999999,"5min":1.022,"15min":1.0700000000000001},"last":0.95399999999999996}]}]},{"osd":5,"up_from":13,"seq":55834575950,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29304,"kb_used_data":2352,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342536,"statfs":{"total":96636764160,"available":96606756864,"internally_reserved":0,"allocated":2408448,"data_stored":1161239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.73699999999999999,"15min":0.73699999999999999},"min":{"1min":0.496,"5min":0.45300000000000001,"15min":0.44400000000000001},"max":{"1min":0.94799999999999995,"5min":1.004,"15min":1.077},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.74199999999999999,"15min":0.75700000000000001},"min":{"1min":0.44600000000000001,"5min":0.44600000000000001,"15min":0.433},"max":{"1min":0.96599999999999997,"5min":1.028,"15min":1.107},"last":0.63800000000000001}]},{"osd":1,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.754,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":1.0409999999999999,"5min":1.0669999999999999,"15min":1.073},"last":0.53100000000000003},{"interface":"front","average":{"1min":0.751,"5min":0.754,"15min":0.77000000000000002},"min":{"1min":0.56299999999999994,"5min":0.51300000000000001,"15min":0.46899999999999997},"max":{"1min":0.90600000000000003,"5min":1.016,"15min":1.0640000000000001},"last":0.73699999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.73299999999999998,"5min":0.74099999999999999,"15min":0.76800000000000002},"min":{"1min":0.51700000000000002,"5min":0.46999999999999997,"15min":0.46899999999999997},"max":{"1min":0.95699999999999996,"5min":1.01,"15min":1.0249999999999999},"last":0.505},{"interface":"front","average":{"1min":0.77800000000000002,"5min":0.77800000000000002,"15min":0.77300000000000002},"min":{"1min":0.58599999999999997,"5min":0.52000000000000002,"15min":0.48299999999999998},"max":{"1min":0.95899999999999996,"5min":1.014,"15min":1.0680000000000001},"last":0.56299999999999994}]},{"osd":3,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78800000000000003,"15min":0.78800000000000003},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.52000000000000002},"max":{"1min":1.0269999999999999,"5min":1.0669999999999999,"15min":1.141},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.79800000000000004,"5min":0.78500000000000003,"15min":0.78600000000000003},"min":{"1min":0.52100000000000002,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.98099999999999998,"5min":1.034,"15min":1.0629999999999999},"last":0.66100000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.64900000000000002,"15min":0.63700000000000001},"min":{"1min":0.51300000000000001,"5min":0.42199999999999999,"15min":0.38100000000000001},"max":{"1min":1.004,"5min":1.0640000000000001,"15min":1.077},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66100000000000003,"15min":0.63700000000000001},"min":{"1min":0.48499999999999999,"5min":0.432,"15min":0.41299999999999998},"max":{"1min":0.89100000000000001,"5min":1.0029999999999999,"15min":1.054},"last":0.57899999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.70299999999999996,"15min":0.70099999999999996},"min":{"1min":0.57299999999999995,"5min":0.48699999999999999,"15min":0.439},"max":{"1min":1.01,"5min":1.01,"15min":1.01},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.68899999999999995,"5min":0.65800000000000003,"15min":0.64500000000000002},"min":{"1min":0.47499999999999998,"5min":0.40699999999999997,"15min":0.40699999999999997},"max":{"1min":1.012,"5min":1.012,"15min":1.073},"last":0.59499999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:00:19 2023","interfaces":[{"interface":"back","average":{"1min":0.747,"5min":0.70799999999999996,"15min":0.70099999999999996},"min":{"1min":0.496,"5min":0.496,"15min":0.48099999999999998},"max":{"1min":0.98599999999999999,"5min":0.98599999999999999,"15min":1.0489999999999999},"last":0.77600000000000002},{"interface":"front","average":{"1min":0.80500000000000005,"5min":0.76800000000000002,"15min":0.76400000000000001},"min":{"1min":0.60899999999999999,"5min":0.57799999999999996,"15min":0.46700000000000003},"max":{"1min":1.0409999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.69799999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575951,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47360,"kb_used_data":2360,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324480,"statfs":{"total":96636764160,"available":96588267520,"internally_reserved":0,"allocated":2416640,"data_stored":1162269,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.74399999999999999,"15min":0.749},"min":{"1min":0.51200000000000001,"5min":0.47399999999999998,"15min":0.38800000000000001},"max":{"1min":0.98999999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.56499999999999995},{"interface":"front","average":{"1min":0.72399999999999998,"5min":0.72699999999999998,"15min":0.73699999999999999},"min":{"1min":0.5,"5min":0.5,"15min":0.45400000000000001},"max":{"1min":0.96399999999999997,"5min":0.996,"15min":1.014},"last":0.67400000000000004}]},{"osd":1,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.78400000000000003,"5min":0.75900000000000001,"15min":0.76300000000000001},"min":{"1min":0.56799999999999995,"5min":0.49199999999999999,"15min":0.45100000000000001},"max":{"1min":0.94799999999999995,"5min":1.042,"15min":1.0900000000000001},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.77000000000000002,"15min":0.77500000000000002},"min":{"1min":0.54700000000000004,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.98799999999999999,"5min":0.98799999999999999,"15min":1.0600000000000001},"last":0.72499999999999998}]},{"osd":2,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.78300000000000003,"15min":0.78400000000000003},"min":{"1min":0.56699999999999995,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":1.0940000000000001,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.78800000000000003},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.76900000000000002,"15min":0.78200000000000003},"min":{"1min":0.60599999999999998,"5min":0.55500000000000005,"15min":0.505},"max":{"1min":1.006,"5min":1.042,"15min":1.101},"last":0.76600000000000001}]},{"osd":3,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.75600000000000001,"15min":0.75900000000000001},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.49199999999999999},"max":{"1min":0.97899999999999998,"5min":0.97899999999999998,"15min":1.0129999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.80000000000000004,"5min":0.80300000000000005,"15min":0.81599999999999995},"min":{"1min":0.55000000000000004,"5min":0.54800000000000004,"15min":0.54800000000000004},"max":{"1min":1.04,"5min":1.04,"15min":1.081},"last":0.91200000000000003}]},{"osd":5,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.63600000000000001,"5min":0.65200000000000002,"15min":0.64600000000000002},"min":{"1min":0.442,"5min":0.442,"15min":0.39900000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.05},"last":0.81399999999999995},{"interface":"front","average":{"1min":0.64600000000000002,"5min":0.66900000000000004,"15min":0.65000000000000002},"min":{"1min":0.42499999999999999,"5min":0.41699999999999998,"15min":0.35799999999999998},"max":{"1min":0.97899999999999998,"5min":1.0800000000000001,"15min":1.0800000000000001},"last":0.83599999999999997}]},{"osd":6,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.67900000000000005,"5min":0.68999999999999995,"15min":0.67100000000000004},"min":{"1min":0.44600000000000001,"5min":0.44400000000000001,"15min":0.41499999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.1679999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.748,"15min":0.72899999999999998},"min":{"1min":0.499,"5min":0.499,"15min":0.41199999999999998},"max":{"1min":1.024,"5min":1.0700000000000001,"15min":1.111},"last":0.88500000000000001}]},{"osd":7,"last update":"Wed Dec 20 05:59:57 2023","interfaces":[{"interface":"back","average":{"1min":0.72799999999999998,"5min":0.73299999999999998,"15min":0.72399999999999998},"min":{"1min":0.47899999999999998,"5min":0.39700000000000002,"15min":0.38500000000000001},"max":{"1min":1.0600000000000001,"5min":1.0600000000000001,"15min":1.1160000000000001},"last":0.93300000000000005},{"interface":"front","average":{"1min":0.78700000000000003,"5min":0.79500000000000004,"15min":0.78500000000000003},"min":{"1min":0.52600000000000002,"5min":0.505,"15min":0.434},"max":{"1min":1.1319999999999999,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.95999999999999996}]}]},{"osd":3,"up_from":13,"seq":55834575950,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47348,"kb_used_data":2348,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324492,"statfs":{"total":96636764160,"available":96588279808,"internally_reserved":0,"allocated":2404352,"data_stored":1160669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.46700000000000003,"5min":0.49299999999999999,"15min":0.47499999999999998},"min":{"1min":0.32100000000000001,"5min":0.29599999999999999,"15min":0.28499999999999998},"max":{"1min":0.748,"5min":2.8130000000000002,"15min":2.8130000000000002},"last":0.47199999999999998},{"interface":"front","average":{"1min":0.52400000000000002,"5min":0.53600000000000003,"15min":0.54100000000000004},"min":{"1min":0.36199999999999999,"5min":0.33700000000000002,"15min":0.32700000000000001},"max":{"1min":0.67600000000000005,"5min":1.357,"15min":1.357},"last":0.52900000000000003}]},{"osd":1,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.45200000000000001,"5min":0.48999999999999999,"15min":0.48099999999999998},"min":{"1min":0.32400000000000001,"5min":0.32400000000000001,"15min":0.32000000000000001},"max":{"1min":0.622,"5min":2.0230000000000001,"15min":2.0230000000000001},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.53600000000000003,"5min":0.58599999999999997,"15min":0.58299999999999996},"min":{"1min":0.44,"5min":0.42599999999999999,"15min":0.372},"max":{"1min":0.64100000000000001,"5min":1.4219999999999999,"15min":1.4219999999999999},"last":0.38500000000000001}]},{"osd":2,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.59299999999999997,"5min":0.60799999999999998,"15min":0.60299999999999998},"min":{"1min":0.46400000000000002,"5min":0.40699999999999997,"15min":0.40300000000000002},"max":{"1min":0.76400000000000001,"5min":2.0550000000000002,"15min":2.0550000000000002},"last":0.63900000000000001},{"interface":"front","average":{"1min":0.59099999999999997,"5min":0.60199999999999998,"15min":0.58299999999999996},"min":{"1min":0.44700000000000001,"5min":0.40200000000000002,"15min":0.34200000000000003},"max":{"1min":0.86099999999999999,"5min":2.944,"15min":2.944},"last":0.40799999999999997}]},{"osd":4,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82599999999999996,"15min":0.82999999999999996},"min":{"1min":0.60099999999999998,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.056,"5min":1.669,"15min":1.669},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.76000000000000001,"5min":0.80600000000000005,"15min":0.82399999999999995},"min":{"1min":0.63800000000000001,"5min":0.54400000000000004,"15min":0.54300000000000004},"max":{"1min":0.97699999999999998,"5min":1.99,"15min":1.99},"last":0.73799999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.79500000000000004,"5min":0.84799999999999998,"15min":0.84699999999999998},"min":{"1min":0.56499999999999995,"5min":0.56499999999999995,"15min":0.56499999999999995},"max":{"1min":1.095,"5min":1.901,"15min":1.901},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.82799999999999996,"15min":0.83099999999999996},"min":{"1min":0.59699999999999998,"5min":0.57599999999999996,"15min":0.55000000000000004},"max":{"1min":1.0880000000000001,"5min":2.8639999999999999,"15min":2.8639999999999999},"last":0.85999999999999999}]},{"osd":6,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.76200000000000001,"5min":0.82099999999999995,"15min":0.82999999999999996},"min":{"1min":0.59099999999999997,"5min":0.58799999999999997,"15min":0.56599999999999995},"max":{"1min":0.99399999999999999,"5min":1.575,"15min":1.575},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.81100000000000005,"5min":0.86599999999999999,"15min":0.86199999999999999},"min":{"1min":0.622,"5min":0.59299999999999997,"15min":0.59299999999999997},"max":{"1min":1.1479999999999999,"5min":1.948,"15min":1.948},"last":0.84499999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:00:25 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.85399999999999998},"min":{"1min":0.60299999999999998,"5min":0.60299999999999998,"15min":0.58999999999999997},"max":{"1min":1.1579999999999999,"5min":2.9049999999999998,"15min":2.9049999999999998},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.79400000000000004,"5min":0.871,"15min":0.86399999999999999},"min":{"1min":0.67100000000000004,"5min":0.623,"15min":0.55700000000000005},"max":{"1min":1.0309999999999999,"5min":1.754,"15min":1.754},"last":0.60499999999999998}]}]},{"osd":2,"up_from":13,"seq":55834575949,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51444,"kb_used_data":2348,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320396,"statfs":{"total":96636764160,"available":96584085504,"internally_reserved":0,"allocated":2404352,"data_stored":1160725,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.45500000000000002,"5min":0.46000000000000002,"15min":0.45900000000000002},"min":{"1min":0.34100000000000003,"5min":0.28299999999999997,"15min":0.26100000000000001},"max":{"1min":0.70399999999999996,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.497,"5min":0.51300000000000001,"15min":0.51900000000000002},"min":{"1min":0.36899999999999999,"5min":0.33800000000000002,"15min":0.33300000000000002},"max":{"1min":0.89800000000000002,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.53200000000000003}]},{"osd":1,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.503,"5min":0.497,"15min":0.496},"min":{"1min":0.378,"5min":0.32400000000000001,"15min":0.32400000000000001},"max":{"1min":0.75800000000000001,"5min":0.77100000000000002,"15min":0.84899999999999998},"last":0.44},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.54900000000000004,"15min":0.55100000000000005},"min":{"1min":0.42399999999999999,"5min":0.40000000000000002,"15min":0.30499999999999999},"max":{"1min":0.92600000000000005,"5min":0.92600000000000005,"15min":1.004},"last":0.505}]},{"osd":3,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.64300000000000002,"15min":0.64900000000000002},"min":{"1min":0.54300000000000004,"5min":0.42099999999999999,"15min":0.39800000000000002},"max":{"1min":0.86199999999999999,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.55100000000000005},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.68799999999999994,"15min":0.68899999999999995},"min":{"1min":0.57599999999999996,"5min":0.52800000000000002,"15min":0.42499999999999999},"max":{"1min":0.86899999999999999,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.57299999999999995}]},{"osd":4,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.78500000000000003,"15min":0.80800000000000005},"min":{"1min":0.51700000000000002,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.069,"5min":1.069,"15min":1.254},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80000000000000004,"15min":0.81000000000000005},"min":{"1min":0.505,"5min":0.48599999999999999,"15min":0.48199999999999998},"max":{"1min":1.085,"5min":1.605,"15min":1.605},"last":0.77500000000000002}]},{"osd":5,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.82799999999999996,"5min":0.80300000000000005,"15min":0.80700000000000005},"min":{"1min":0.51400000000000001,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1639999999999999,"5min":1.329,"15min":1.329},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.81999999999999995,"5min":0.83499999999999996,"15min":0.83599999999999997},"min":{"1min":0.63700000000000001,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.028,"5min":1.0740000000000001,"15min":1.157},"last":0.79100000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.81999999999999995,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.55100000000000005,"5min":0.52800000000000002,"15min":0.52300000000000002},"max":{"1min":1.0880000000000001,"5min":1.919,"15min":1.919},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.83699999999999997,"5min":0.82499999999999996,"15min":0.83499999999999996},"min":{"1min":0.58899999999999997,"5min":0.58899999999999997,"15min":0.56200000000000006},"max":{"1min":1.117,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.80800000000000005}]},{"osd":7,"last update":"Wed Dec 20 06:00:21 2023","interfaces":[{"interface":"back","average":{"1min":0.86899999999999999,"5min":0.85399999999999998,"15min":0.85999999999999999},"min":{"1min":0.61599999999999999,"5min":0.59599999999999997,"15min":0.47899999999999998},"max":{"1min":1.196,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.93700000000000006},{"interface":"front","average":{"1min":0.90700000000000003,"5min":0.89400000000000002,"15min":0.88900000000000001},"min":{"1min":0.75700000000000001,"5min":0.67300000000000004,"15min":0.60499999999999998},"max":{"1min":1.165,"5min":1.292,"15min":1.292},"last":0.97299999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575951,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59636,"kb_used_data":2348,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94312204,"statfs":{"total":96636764160,"available":96575696896,"internally_reserved":0,"allocated":2404352,"data_stored":1160669,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.46800000000000003,"5min":0.47999999999999998,"15min":0.46000000000000002},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.23899999999999999},"max":{"1min":0.61899999999999999,"5min":0.746,"15min":0.81000000000000005},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.46700000000000003,"5min":0.51300000000000001,"15min":0.51300000000000001},"min":{"1min":0.28399999999999997,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.69499999999999995,"5min":0.81100000000000005,"15min":0.81100000000000005},"last":0.46200000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53300000000000003,"15min":0.53200000000000003},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36099999999999999},"max":{"1min":0.68400000000000005,"5min":0.76800000000000002,"15min":0.79200000000000004},"last":0.40200000000000002},{"interface":"front","average":{"1min":0.58899999999999997,"5min":0.57299999999999995,"15min":0.57699999999999996},"min":{"1min":0.42499999999999999,"5min":0.39900000000000002,"15min":0.38300000000000001},"max":{"1min":0.86099999999999999,"5min":0.88800000000000001,"15min":0.93700000000000006},"last":0.49399999999999999}]},{"osd":3,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.58299999999999996,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.88500000000000001,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55100000000000005,"15min":0.55500000000000005},"min":{"1min":0.35499999999999998,"5min":0.33900000000000002,"15min":0.33900000000000002},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.93500000000000005},"last":0.54700000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.77900000000000003,"15min":0.79400000000000004},"min":{"1min":0.53500000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.0129999999999999,"5min":1.0820000000000001,"15min":1.321},"last":0.874},{"interface":"front","average":{"1min":0.79700000000000004,"5min":0.81499999999999995,"15min":0.81699999999999995},"min":{"1min":0.60099999999999998,"5min":0.60099999999999998,"15min":0.58099999999999996},"max":{"1min":1.002,"5min":1.139,"15min":1.25},"last":0.74099999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.79600000000000004,"15min":0.80900000000000005},"min":{"1min":0.55100000000000005,"5min":0.51400000000000001,"15min":0.443},"max":{"1min":1.157,"5min":1.224,"15min":1.607},"last":0.80000000000000004},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.80200000000000005,"15min":0.81699999999999995},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.024,"5min":1.25,"15min":1.4330000000000001},"last":0.81799999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.81699999999999995,"15min":0.83399999999999996},"min":{"1min":0.52700000000000002,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0329999999999999,"5min":1.2430000000000001,"15min":1.341},"last":0.78300000000000003},{"interface":"front","average":{"1min":0.81699999999999995,"5min":0.81699999999999995,"15min":0.81799999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1040000000000001,"5min":1.177,"15min":1.3839999999999999},"last":0.88200000000000001}]},{"osd":7,"last update":"Wed Dec 20 06:00:39 2023","interfaces":[{"interface":"back","average":{"1min":0.81599999999999995,"5min":0.85399999999999998,"15min":0.86799999999999999},"min":{"1min":0.57799999999999996,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0509999999999999,"5min":1.272,"15min":1.4470000000000001},"last":0.85499999999999998},{"interface":"front","average":{"1min":0.81599999999999995,"5min":0.86199999999999999,"15min":0.873},"min":{"1min":0.62,"5min":0.62,"15min":0.58699999999999997},"max":{"1min":1.1279999999999999,"5min":1.1599999999999999,"15min":1.3839999999999999},"last":0.75900000000000001}]}]},{"osd":0,"up_from":13,"seq":55834575950,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56116,"kb_used_data":2924,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315724,"statfs":{"total":96636764160,"available":96579301376,"internally_reserved":0,"allocated":2994176,"data_stored":1751003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.47199999999999998,"5min":0.46700000000000003,"15min":0.46600000000000003},"min":{"1min":0.34599999999999997,"5min":0.32600000000000001,"15min":0.28799999999999998},"max":{"1min":0.82299999999999995,"5min":0.82299999999999995,"15min":0.96799999999999997},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.56999999999999995,"5min":0.56799999999999995,"15min":0.55500000000000005},"min":{"1min":0.39200000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.752,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.60099999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.55000000000000004,"5min":0.53100000000000003,"15min":0.53600000000000003},"min":{"1min":0.39400000000000002,"5min":0.376,"15min":0.35999999999999999},"max":{"1min":0.76600000000000001,"5min":0.81299999999999994,"15min":0.82499999999999996},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.61099999999999999,"5min":0.58099999999999996,"15min":0.59099999999999997},"min":{"1min":0.45900000000000002,"5min":0.40799999999999997,"15min":0.38100000000000001},"max":{"1min":0.77100000000000002,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.68500000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.56499999999999995,"15min":0.56599999999999995},"min":{"1min":0.442,"5min":0.34200000000000003,"15min":0.31900000000000001},"max":{"1min":0.78800000000000003,"5min":0.80700000000000005,"15min":1.0429999999999999},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.63100000000000001,"5min":0.628,"15min":0.627},"min":{"1min":0.47699999999999998,"5min":0.438,"15min":0.41999999999999998},"max":{"1min":0.78600000000000003,"5min":0.92100000000000004,"15min":0.92100000000000004},"last":0.63400000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82899999999999996,"5min":0.82799999999999996,"15min":0.82499999999999996},"min":{"1min":0.61899999999999999,"5min":0.56499999999999995,"15min":0.48099999999999998},"max":{"1min":0.996,"5min":1.0669999999999999,"15min":1.147},"last":1.0469999999999999},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.82899999999999996,"15min":0.83799999999999997},"min":{"1min":0.68899999999999995,"5min":0.58399999999999996,"15min":0.51400000000000001},"max":{"1min":0.96999999999999997,"5min":1.0760000000000001,"15min":1.2869999999999999},"last":0.82199999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.82999999999999996,"5min":0.82999999999999996,"15min":0.84499999999999997},"min":{"1min":0.626,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.1259999999999999,"5min":1.1259999999999999,"15min":1.2150000000000001},"last":0.90900000000000003},{"interface":"front","average":{"1min":0.83499999999999996,"5min":0.84499999999999997,"15min":0.84899999999999998},"min":{"1min":0.61499999999999999,"5min":0.61499999999999999,"15min":0.57299999999999995},"max":{"1min":1.0269999999999999,"5min":1.131,"15min":1.351},"last":0.84099999999999997}]},{"osd":6,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85699999999999998,"5min":0.83999999999999997,"15min":0.84799999999999998},"min":{"1min":0.67400000000000004,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":1.0149999999999999,"5min":1.2150000000000001,"15min":1.2150000000000001},"last":0.93000000000000005},{"interface":"front","average":{"1min":0.86599999999999999,"5min":0.85799999999999998,"15min":0.86299999999999999},"min":{"1min":0.66500000000000004,"5min":0.62,"15min":0.57099999999999995},"max":{"1min":1.0469999999999999,"5min":1.1379999999999999,"15min":1.3320000000000001},"last":1.008}]},{"osd":7,"last update":"Wed Dec 20 06:00:07 2023","interfaces":[{"interface":"back","average":{"1min":0.85799999999999998,"5min":0.87,"15min":0.873},"min":{"1min":0.64400000000000002,"5min":0.59399999999999997,"15min":0.59399999999999997},"max":{"1min":0.995,"5min":1.129,"15min":1.3560000000000001},"last":1.0680000000000001},{"interface":"front","average":{"1min":0.86699999999999999,"5min":0.86099999999999999,"15min":0.86599999999999999},"min":{"1min":0.68799999999999994,"5min":0.60299999999999998,"15min":0.58899999999999997},"max":{"1min":1.079,"5min":1.139,"15min":1.2549999999999999},"last":0.94599999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":22,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2306,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":23,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:00:42.138 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T06:00:42.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T06:00:42.249 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:00:42.253 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:00:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.330+0000 7fe414273640 1 Processor -- start 2023-12-20T06:00:42.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.330+0000 7fe414273640 1 -- start start 2023-12-20T06:00:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.330+0000 7fe414273640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe40c0c4320 0x7fe40c06da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.330+0000 7fe414273640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe40c06df80 0x7fe40c06c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.330+0000 7fe414273640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe40c06e3b0 con 0x7fe40c0adf00 2023-12-20T06:00:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.330+0000 7fe414273640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe40c0c3630 con 0x7fe40c0c4320 2023-12-20T06:00:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.330+0000 7fe414273640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c159e60 con 0x7fe40c06df80 2023-12-20T06:00:42.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe41200f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe40c0adf00 0x7fe40c0cd850 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55596/0 (socket says 172.21.15.32:55596) 2023-12-20T06:00:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe41200f640 1 -- 172.21.15.32:0/3004521607 learned_addr learned my addr 172.21.15.32:0/3004521607 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe412810640 1 --2- 172.21.15.32:0/3004521607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe40c06df80 0x7fe40c06c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe41180e640 1 --2- 172.21.15.32:0/3004521607 >> v2:172.21.15.39:3300/0 conn(0x7fe40c0c4320 0x7fe40c06da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe41100d640 1 -- 172.21.15.32:0/3004521607 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2145265598 0 0) 0x7fe40c06e3b0 con 0x7fe40c0adf00 2023-12-20T06:00:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe41100d640 1 -- 172.21.15.32:0/3004521607 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe3f8003530 con 0x7fe40c0adf00 2023-12-20T06:00:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe41180e640 1 -- 172.21.15.32:0/3004521607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe40c06df80 msgr2=0x7fe40c06c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe41180e640 1 --2- 172.21.15.32:0/3004521607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe40c06df80 0x7fe40c06c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe41180e640 1 -- 172.21.15.32:0/3004521607 >> v1:172.21.15.32:6789/0 conn(0x7fe40c0adf00 legacy=0x7fe40c0cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe41180e640 1 -- 172.21.15.32:0/3004521607 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe40c0ce380 con 0x7fe40c0c4320 2023-12-20T06:00:42.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe412810640 1 --2- 172.21.15.32:0/3004521607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe40c06df80 0x7fe40c06c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.334+0000 7fe41180e640 1 --2- 172.21.15.32:0/3004521607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe40c0c4320 0x7fe40c06da40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0097c0 tx=0x7fe3fc01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3ea6c3de72703d61 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe41100d640 1 -- 172.21.15.32:0/3004521607 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0ab650 con 0x7fe40c0c4320 2023-12-20T06:00:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe41100d640 1 -- 172.21.15.32:0/3004521607 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0ab7b0 con 0x7fe40c0c4320 2023-12-20T06:00:42.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe41100d640 1 -- 172.21.15.32:0/3004521607 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0abb40 con 0x7fe40c0c4320 2023-12-20T06:00:42.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 -- 172.21.15.32:0/3004521607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe40c0c4320 msgr2=0x7fe40c06da40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 --2- 172.21.15.32:0/3004521607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe40c0c4320 0x7fe40c06da40 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0097c0 tx=0x7fe3fc01c450 comp rx=0 tx=0).stop 2023-12-20T06:00:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 -- 172.21.15.32:0/3004521607 shutdown_connections 2023-12-20T06:00:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 --2- 172.21.15.32:0/3004521607 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe40c0c4320 0x7fe40c06da40 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 --2- 172.21.15.32:0/3004521607 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe40c06df80 0x7fe40c06c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 -- 172.21.15.32:0/3004521607 >> 172.21.15.32:0/3004521607 conn(0x7fe40c06a4b0 msgr2=0x7fe40c0b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 -- 172.21.15.32:0/3004521607 shutdown_connections 2023-12-20T06:00:42.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 -- 172.21.15.32:0/3004521607 wait complete. 2023-12-20T06:00:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 Processor -- start 2023-12-20T06:00:42.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 -- start start 2023-12-20T06:00:42.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe40c06df80 0x7fe40c0b14b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe40c0adf00 0x7fe40c1539f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe40c0c4320 0x7fe40c1588b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0b1e50 con 0x7fe40c0adf00 2023-12-20T06:00:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0aea10 con 0x7fe40c0c4320 2023-12-20T06:00:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.338+0000 7fe414273640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe40c0b20a0 con 0x7fe40c06df80 2023-12-20T06:00:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe41180e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe40c0adf00 0x7fe40c1539f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe41180e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe40c0adf00 0x7fe40c1539f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47360/0 (socket says 172.21.15.32:47360) 2023-12-20T06:00:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe41180e640 1 -- 172.21.15.32:0/4030295758 learned_addr learned my addr 172.21.15.32:0/4030295758 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe412810640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe40c0c4320 0x7fe40c1588b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe41200f640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe40c06df80 0x7fe40c0b14b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe41180e640 1 -- 172.21.15.32:0/4030295758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe40c06df80 msgr2=0x7fe40c0b14b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe41180e640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe40c06df80 0x7fe40c0b14b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe41180e640 1 -- 172.21.15.32:0/4030295758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe40c0c4320 msgr2=0x7fe40c1588b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe41180e640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe40c0c4320 0x7fe40c1588b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe41180e640 1 -- 172.21.15.32:0/4030295758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe404009a70 con 0x7fe40c0adf00 2023-12-20T06:00:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe41180e640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe40c0adf00 0x7fe40c1539f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0ad040 tx=0x7fe3fc01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe402ffd640 1 -- 172.21.15.32:0/4030295758 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0bf610 con 0x7fe40c0adf00 2023-12-20T06:00:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe402ffd640 1 -- 172.21.15.32:0/4030295758 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe3fc0bf7b0 con 0x7fe40c0adf00 2023-12-20T06:00:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe402ffd640 1 -- 172.21.15.32:0/4030295758 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe3fc0bfb60 con 0x7fe40c0adf00 2023-12-20T06:00:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe414273640 1 -- 172.21.15.32:0/4030295758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe3fc009420 con 0x7fe40c0adf00 2023-12-20T06:00:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe414273640 1 -- 172.21.15.32:0/4030295758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe40c0b2660 con 0x7fe40c0adf00 2023-12-20T06:00:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe414273640 1 -- 172.21.15.32:0/4030295758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe40c0aeb80 con 0x7fe40c0adf00 2023-12-20T06:00:42.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.342+0000 7fe402ffd640 1 -- 172.21.15.32:0/4030295758 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fe3fc0b5400 con 0x7fe40c0adf00 2023-12-20T06:00:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.346+0000 7fe402ffd640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe3f0065b80 0x7fe3f0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.346+0000 7fe402ffd640 1 -- 172.21.15.32:0/4030295758 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe3fc11df60 con 0x7fe40c0adf00 2023-12-20T06:00:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.346+0000 7fe41200f640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe3f0065b80 0x7fe3f0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.346+0000 7fe41200f640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe3f0065b80 0x7fe3f0068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe408005890 tx=0x7fe408009040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:42.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.346+0000 7fe402ffd640 1 -- 172.21.15.32:0/4030295758 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe3fc0eba10 con 0x7fe40c0adf00 2023-12-20T06:00:42.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.558+0000 7fe414273640 1 -- 172.21.15.32:0/4030295758 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe40c04e000 con 0x7fe40c0adf00 2023-12-20T06:00:42.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.562+0000 7fe402ffd640 1 -- 172.21.15.32:0/4030295758 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v51) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe3fc0ef030 con 0x7fe40c0adf00 2023-12-20T06:00:42.564 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T06:00:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 -- 172.21.15.32:0/4030295758 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe3f0065b80 msgr2=0x7fe3f0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe3f0065b80 0x7fe3f0068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7fe408005890 tx=0x7fe408009040 comp rx=0 tx=0).stop 2023-12-20T06:00:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 -- 172.21.15.32:0/4030295758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe40c0adf00 msgr2=0x7fe40c1539f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe40c0adf00 0x7fe40c1539f0 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7fe3fc0ad040 tx=0x7fe3fc01ac90 comp rx=0 tx=0).stop 2023-12-20T06:00:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 -- 172.21.15.32:0/4030295758 shutdown_connections 2023-12-20T06:00:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe3f0065b80 0x7fe3f0068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe40c0c4320 0x7fe40c1588b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe40c0adf00 0x7fe40c1539f0 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 --2- 172.21.15.32:0/4030295758 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe40c06df80 0x7fe40c0b14b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 -- 172.21.15.32:0/4030295758 >> 172.21.15.32:0/4030295758 conn(0x7fe40c06a4b0 msgr2=0x7fe40c0cb230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 -- 172.21.15.32:0/4030295758 shutdown_connections 2023-12-20T06:00:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.566+0000 7fe414273640 1 -- 172.21.15.32:0/4030295758 wait complete. 2023-12-20T06:00:42.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:00:42.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.770+0000 7fccf2a41640 1 Processor -- start 2023-12-20T06:00:42.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccf2a41640 1 -- start start 2023-12-20T06:00:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccf2a41640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fccec0af5e0 0x7fccec0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccf2a41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fccec06ce60 0x7fccec15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccf2a41640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fccec15bbd0 con 0x7fccec15f110 2023-12-20T06:00:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccf2a41640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fccec15beb0 con 0x7fccec0af5e0 2023-12-20T06:00:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccf2a41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec15e910 con 0x7fccec06ce60 2023-12-20T06:00:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fcceb7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fccec06ce60 0x7fccec15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccebfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fccec0af5e0 0x7fccec0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fcceb7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fccec06ce60 0x7fccec15b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:60354/0 (socket says 172.21.15.32:60354) 2023-12-20T06:00:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccebfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fccec0af5e0 0x7fccec0af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:41910/0 (socket says 172.21.15.32:41910) 2023-12-20T06:00:42.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fcceb7fe640 1 -- 172.21.15.32:0/320568715 learned_addr learned my addr 172.21.15.32:0/320568715 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccebfff640 1 -- 172.21.15.32:0/320568715 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fccec06ce60 msgr2=0x7fccec15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccebfff640 1 --2- 172.21.15.32:0/320568715 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fccec06ce60 0x7fccec15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccebfff640 1 -- 172.21.15.32:0/320568715 >> v1:172.21.15.39:6789/0 conn(0x7fccec15f110 legacy=0x7fccec15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.774+0000 7fccebfff640 1 -- 172.21.15.32:0/320568715 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccec15ebf0 con 0x7fccec0af5e0 2023-12-20T06:00:42.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fccebfff640 1 --2- 172.21.15.32:0/320568715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccec0af5e0 0x7fccec0af9c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0097a0 tx=0x7fccdc01c460 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b1ce0e7cd6fceab5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:42.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fcceaffd640 1 -- 172.21.15.32:0/320568715 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0ac620 con 0x7fccec0af5e0 2023-12-20T06:00:42.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fcceaffd640 1 -- 172.21.15.32:0/320568715 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc0ac780 con 0x7fccec0af5e0 2023-12-20T06:00:42.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fcceaffd640 1 -- 172.21.15.32:0/320568715 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0acb10 con 0x7fccec0af5e0 2023-12-20T06:00:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fccf2a41640 1 -- 172.21.15.32:0/320568715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccec0af5e0 msgr2=0x7fccec0af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fccf2a41640 1 --2- 172.21.15.32:0/320568715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccec0af5e0 0x7fccec0af9c0 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0097a0 tx=0x7fccdc01c460 comp rx=0 tx=0).stop 2023-12-20T06:00:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fccf2a41640 1 -- 172.21.15.32:0/320568715 shutdown_connections 2023-12-20T06:00:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fccf2a41640 1 --2- 172.21.15.32:0/320568715 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fccec06ce60 0x7fccec15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fccf2a41640 1 --2- 172.21.15.32:0/320568715 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccec0af5e0 0x7fccec0af9c0 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fccf2a41640 1 -- 172.21.15.32:0/320568715 >> 172.21.15.32:0/320568715 conn(0x7fccec06a320 msgr2=0x7fccec0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fccf2a41640 1 -- 172.21.15.32:0/320568715 shutdown_connections 2023-12-20T06:00:42.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fccf2a41640 1 -- 172.21.15.32:0/320568715 wait complete. 2023-12-20T06:00:42.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.778+0000 7fccf2a41640 1 Processor -- start 2023-12-20T06:00:42.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf2a41640 1 -- start start 2023-12-20T06:00:42.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf2a41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccec06ce60 0x7fccec14caa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf2a41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fccec0af5e0 0x7fccec071a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf2a41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccec15f110 0x7fccec0ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf2a41640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec0ce5d0 con 0x7fccec15f110 2023-12-20T06:00:42.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf2a41640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fccec0701c0 con 0x7fccec06ce60 2023-12-20T06:00:42.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf2a41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fccec1f8f50 con 0x7fccec0af5e0 2023-12-20T06:00:42.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf0fde640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccec15f110 0x7fccec0ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf0fde640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccec15f110 0x7fccec0ce090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47368/0 (socket says 172.21.15.32:47368) 2023-12-20T06:00:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf0fde640 1 -- 172.21.15.32:0/822002800 learned_addr learned my addr 172.21.15.32:0/822002800 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fcceb7fe640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fccec0af5e0 0x7fccec071a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccebfff640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccec06ce60 0x7fccec14caa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccebfff640 1 -- 172.21.15.32:0/822002800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fccec0af5e0 msgr2=0x7fccec071a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccebfff640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fccec0af5e0 0x7fccec071a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccebfff640 1 -- 172.21.15.32:0/822002800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccec15f110 msgr2=0x7fccec0ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccebfff640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccec15f110 0x7fccec0ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccebfff640 1 -- 172.21.15.32:0/822002800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fccdc0093b0 con 0x7fccec06ce60 2023-12-20T06:00:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf0fde640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccec15f110 0x7fccec0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:42.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccebfff640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccec06ce60 0x7fccec14caa0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0ac000 tx=0x7fccdc019200 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:42.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fcce8ff9640 1 -- 172.21.15.32:0/822002800 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc0059a0 con 0x7fccec06ce60 2023-12-20T06:00:42.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fcce8ff9640 1 -- 172.21.15.32:0/822002800 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fccdc005b00 con 0x7fccec06ce60 2023-12-20T06:00:42.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf2a41640 1 -- 172.21.15.32:0/822002800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fccec0720d0 con 0x7fccec06ce60 2023-12-20T06:00:42.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fcce8ff9640 1 -- 172.21.15.32:0/822002800 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fccdc006030 con 0x7fccec06ce60 2023-12-20T06:00:42.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf2a41640 1 -- 172.21.15.32:0/822002800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fccec1f94f0 con 0x7fccec06ce60 2023-12-20T06:00:42.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.782+0000 7fccf2a41640 1 -- 172.21.15.32:0/822002800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fccec04e000 con 0x7fccec06ce60 2023-12-20T06:00:42.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.786+0000 7fcce8ff9640 1 -- 172.21.15.32:0/822002800 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fccdc0b3040 con 0x7fccec06ce60 2023-12-20T06:00:42.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.786+0000 7fcce8ff9640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fccd8065b80 0x7fccd8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:42.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.786+0000 7fcce8ff9640 1 -- 172.21.15.32:0/822002800 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fccdc11d7d0 con 0x7fccec06ce60 2023-12-20T06:00:42.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.786+0000 7fcceb7fe640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fccd8065b80 0x7fccd8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:42.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.786+0000 7fcceb7fe640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fccd8065b80 0x7fccd8068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fccec152550 tx=0x7fcce0009240 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:42.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.790+0000 7fcce8ff9640 1 -- 172.21.15.32:0/822002800 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fccdc0eafc0 con 0x7fccec06ce60 2023-12-20T06:00:42.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 -- 172.21.15.32:0/822002800 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fccec0c4c10 con 0x7fccec06ce60 2023-12-20T06:00:42.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fcce8ff9640 1 -- 172.21.15.32:0/822002800 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4717 (secure 0 0 0) 0x7fccdc0ee5e0 con 0x7fccec06ce60 2023-12-20T06:00:42.977 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:42.978 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8396,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1626919510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1626919510},{"type":"v1","addr":"172.21.15.39:6833","nonce":1626919510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3001408763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3001408763},{"type":"v1","addr":"172.21.15.39:6835","nonce":3001408763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":10020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/2908145202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2908145202},{"type":"v1","addr":"172.21.15.32:6835","nonce":2908145202}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:00:37.604080+0000","modified":"2023-12-20T06:00:40.642479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10026":{"gid":10026,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2253884246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2253884246},{"type":"v1","addr":"172.21.15.32:6837","nonce":2253884246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-20T06:00:42.978 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 87 2023-12-20T06:00:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 -- 172.21.15.32:0/822002800 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fccd8065b80 msgr2=0x7fccd8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fccd8065b80 0x7fccd8068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fccec152550 tx=0x7fcce0009240 comp rx=0 tx=0).stop 2023-12-20T06:00:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 -- 172.21.15.32:0/822002800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccec06ce60 msgr2=0x7fccec14caa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccec06ce60 0x7fccec14caa0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fccdc0ac000 tx=0x7fccdc019200 comp rx=0 tx=0).stop 2023-12-20T06:00:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 -- 172.21.15.32:0/822002800 shutdown_connections 2023-12-20T06:00:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fccd8065b80 0x7fccd8068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fccec15f110 0x7fccec0ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fccec0af5e0 0x7fccec071a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 --2- 172.21.15.32:0/822002800 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fccec06ce60 0x7fccec14caa0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:42.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 -- 172.21.15.32:0/822002800 >> 172.21.15.32:0/822002800 conn(0x7fccec06a320 msgr2=0x7fccec155b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 -- 172.21.15.32:0/822002800 shutdown_connections 2023-12-20T06:00:42.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:42.978+0000 7fccf2a41640 1 -- 172.21.15.32:0/822002800 wait complete. 2023-12-20T06:00:43.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 87, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T06:00:37.604080+0000', 'modified': '2023-12-20T06:00:40.642479+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10026}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10026': {'gid': 10026, 'name': 'a', 'rank': 0, 'incarnation': 83, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/2253884246', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 2253884246}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 2253884246}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [23], 'metadata_pool': 22, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T06:00:43.402 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T06:00:43.402 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-20T06:00:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc12d666640 1 Processor -- start 2023-12-20T06:00:43.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc12d666640 1 -- start start 2023-12-20T06:00:43.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc12d666640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1281599e0 0x7fc1280aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc12d666640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1280c98c0 con 0x7fc1280c90e0 2023-12-20T06:00:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc12d666640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc1280c9ba0 con 0x7fc1280b0d00 2023-12-20T06:00:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc12d666640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc1280c9e80 con 0x7fc1281599e0 2023-12-20T06:00:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc126ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1281599e0 0x7fc1280aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc11ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc1280c90e0 0x7fc1280b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55598/0 (socket says 172.21.15.32:55598) 2023-12-20T06:00:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc126ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc1281599e0 0x7fc1280aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47372/0 (socket says 172.21.15.32:47372) 2023-12-20T06:00:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc11ffff640 1 -- 172.21.15.32:0/2845941752 learned_addr learned my addr 172.21.15.32:0/2845941752 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc1267fc640 1 -- 172.21.15.32:0/2845941752 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 344432285 0 0) 0x7fc1280c98c0 con 0x7fc1280c90e0 2023-12-20T06:00:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc1267fc640 1 -- 172.21.15.32:0/2845941752 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc114003530 con 0x7fc1280c90e0 2023-12-20T06:00:43.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc126ffd640 1 -- 172.21.15.32:0/2845941752 >> v1:172.21.15.39:6789/0 conn(0x7fc1280b0d00 legacy=0x7fc1280701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc126ffd640 1 -- 172.21.15.32:0/2845941752 >> v1:172.21.15.32:6789/0 conn(0x7fc1280c90e0 legacy=0x7fc1280b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.550+0000 7fc126ffd640 1 -- 172.21.15.32:0/2845941752 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc1280ca160 con 0x7fc1281599e0 2023-12-20T06:00:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.554+0000 7fc126ffd640 1 --2- 172.21.15.32:0/2845941752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1281599e0 0x7fc1280aece0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fc11800ba40 tx=0x7fc11801eb00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bace68299f01952 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.554+0000 7fc1267fc640 1 -- 172.21.15.32:0/2845941752 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180a7ae0 con 0x7fc1281599e0 2023-12-20T06:00:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.554+0000 7fc1267fc640 1 -- 172.21.15.32:0/2845941752 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc1180a7c40 con 0x7fc1281599e0 2023-12-20T06:00:43.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.554+0000 7fc1267fc640 1 -- 172.21.15.32:0/2845941752 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180af440 con 0x7fc1281599e0 2023-12-20T06:00:43.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.554+0000 7fc12d666640 1 -- 172.21.15.32:0/2845941752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1281599e0 msgr2=0x7fc1280aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.554+0000 7fc12d666640 1 --2- 172.21.15.32:0/2845941752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1281599e0 0x7fc1280aece0 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7fc11800ba40 tx=0x7fc11801eb00 comp rx=0 tx=0).stop 2023-12-20T06:00:43.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.554+0000 7fc12d666640 1 -- 172.21.15.32:0/2845941752 shutdown_connections 2023-12-20T06:00:43.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.554+0000 7fc12d666640 1 --2- 172.21.15.32:0/2845941752 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1281599e0 0x7fc1280aece0 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.554+0000 7fc12d666640 1 -- 172.21.15.32:0/2845941752 >> 172.21.15.32:0/2845941752 conn(0x7fc12806a320 msgr2=0x7fc1280ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:43.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.554+0000 7fc12d666640 1 -- 172.21.15.32:0/2845941752 shutdown_connections 2023-12-20T06:00:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.554+0000 7fc12d666640 1 -- 172.21.15.32:0/2845941752 wait complete. 2023-12-20T06:00:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc12d666640 1 Processor -- start 2023-12-20T06:00:43.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc12d666640 1 -- start start 2023-12-20T06:00:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc12d666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1280b0d00 0x7fc12815a7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc12d666640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1280c90e0 0x7fc1280c3e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc12d666640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1281599e0 0x7fc1280b1e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc12d666640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc1280c7110 con 0x7fc1281599e0 2023-12-20T06:00:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc12d666640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc12815b040 con 0x7fc1280c90e0 2023-12-20T06:00:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc12d666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc12815b320 con 0x7fc1280b0d00 2023-12-20T06:00:43.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc1277fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1281599e0 0x7fc1280b1e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc1277fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1281599e0 0x7fc1280b1e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47388/0 (socket says 172.21.15.32:47388) 2023-12-20T06:00:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc1277fe640 1 -- 172.21.15.32:0/384442736 learned_addr learned my addr 172.21.15.32:0/384442736 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc11ffff640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1280c90e0 0x7fc1280c3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc126ffd640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1280b0d00 0x7fc12815a7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc1277fe640 1 -- 172.21.15.32:0/384442736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1280b0d00 msgr2=0x7fc12815a7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc1277fe640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1280b0d00 0x7fc12815a7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc1277fe640 1 -- 172.21.15.32:0/384442736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1280c90e0 msgr2=0x7fc1280c3e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc1277fe640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1280c90e0 0x7fc1280c3e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc1277fe640 1 -- 172.21.15.32:0/384442736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc11800b650 con 0x7fc1281599e0 2023-12-20T06:00:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc1277fe640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1281599e0 0x7fc1280b1e30 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fc12000c770 tx=0x7fc12000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:43.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc11f7fe640 1 -- 172.21.15.32:0/384442736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc120003b80 con 0x7fc1281599e0 2023-12-20T06:00:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc11f7fe640 1 -- 172.21.15.32:0/384442736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc120003ce0 con 0x7fc1281599e0 2023-12-20T06:00:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc11f7fe640 1 -- 172.21.15.32:0/384442736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc12000aa90 con 0x7fc1281599e0 2023-12-20T06:00:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc12d666640 1 -- 172.21.15.32:0/384442736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc12815b600 con 0x7fc1281599e0 2023-12-20T06:00:43.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.558+0000 7fc12d666640 1 -- 172.21.15.32:0/384442736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc12815ad30 con 0x7fc1281599e0 2023-12-20T06:00:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc11f7fe640 1 -- 172.21.15.32:0/384442736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc12000abf0 con 0x7fc1281599e0 2023-12-20T06:00:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc11f7fe640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc108065b80 0x7fc108068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc11f7fe640 1 -- 172.21.15.32:0/384442736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc120083230 con 0x7fc1281599e0 2023-12-20T06:00:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc126ffd640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc108065b80 0x7fc108068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc1309bb000 1 Processor -- start 2023-12-20T06:00:43.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc1309bb000 1 -- start start 2023-12-20T06:00:43.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc126ffd640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc108065b80 0x7fc108068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1180b1000 tx=0x7fc118008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:43.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c2d450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d990 0x5614e9be2c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9be31c0 0x5614e9be35a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc1309bb000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e9a06c90 con 0x5614e9c2d070 2023-12-20T06:00:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc1309bb000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e9a38220 con 0x5614e9c2d990 2023-12-20T06:00:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc1309bb000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e9a357f0 con 0x5614e9be31c0 2023-12-20T06:00:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc126ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c2d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc126ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c2d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47400/0 (socket says 172.21.15.32:47400) 2023-12-20T06:00:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc126ffd640 1 -- 172.21.15.32:0/2053917196 learned_addr learned my addr 172.21.15.32:0/2053917196 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:43.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc126ffd640 1 -- 172.21.15.32:0/2053917196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9be31c0 msgr2=0x5614e9be35a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc11ffff640 1 --2- 172.21.15.32:0/2053917196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d990 0x5614e9be2c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc1277fe640 1 --2- 172.21.15.32:0/2053917196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9be31c0 0x5614e9be35a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc126ffd640 1 --2- 172.21.15.32:0/2053917196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9be31c0 0x5614e9be35a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc126ffd640 1 -- 172.21.15.32:0/2053917196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d990 msgr2=0x5614e9be2c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc126ffd640 1 --2- 172.21.15.32:0/2053917196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d990 0x5614e9be2c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc126ffd640 1 -- 172.21.15.32:0/2053917196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e972d890 con 0x5614e9c2d070 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc11ffff640 1 --2- 172.21.15.32:0/2053917196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d990 0x5614e9be2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.562+0000 7fc1277fe640 1 --2- 172.21.15.32:0/2053917196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9be31c0 0x5614e9be35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc126ffd640 1 --2- 172.21.15.32:0/2053917196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c2d450 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fc1180b2040 tx=0x7fc118004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67ef27936c639dfe server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc11cff9640 1 -- 172.21.15.32:0/2053917196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180c1070 con 0x5614e9c2d070 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc11cff9640 1 -- 172.21.15.32:0/2053917196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc118005c20 con 0x5614e9c2d070 2023-12-20T06:00:43.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc11cff9640 1 -- 172.21.15.32:0/2053917196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc118006030 con 0x5614e9c2d070 2023-12-20T06:00:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 -- 172.21.15.32:0/2053917196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 msgr2=0x5614e9c2d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 --2- 172.21.15.32:0/2053917196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c2d450 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7fc1180b2040 tx=0x7fc118004740 comp rx=0 tx=0).stop 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 -- 172.21.15.32:0/2053917196 shutdown_connections 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 --2- 172.21.15.32:0/2053917196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9be31c0 0x5614e9be35a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 --2- 172.21.15.32:0/2053917196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d990 0x5614e9be2c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 --2- 172.21.15.32:0/2053917196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c2d450 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 -- 172.21.15.32:0/2053917196 >> 172.21.15.32:0/2053917196 conn(0x5614e9be9950 msgr2=0x5614e9be1560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 -- 172.21.15.32:0/2053917196 shutdown_connections 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 -- 172.21.15.32:0/2053917196 wait complete. 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 -- start start 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 Processor -- start 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c347f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d070 0x5614e9c34d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d990 0x5614e9c1b030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e9a310e0 con 0x5614e9be31c0 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e99d0310 con 0x5614e9c2d070 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e99ee770 con 0x5614e9c2d990 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1309bb000 10 client.0 _ll_register_callbacks cb 0x5614e9bceb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc126ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c347f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc126ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c347f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47402/0 (socket says 172.21.15.32:47402) 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc126ffd640 1 -- 172.21.15.32:0/2435866152 learned_addr learned my addr 172.21.15.32:0/2435866152 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1277fe640 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d990 0x5614e9c1b030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc11ffff640 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d070 0x5614e9c34d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc126ffd640 1 -- 172.21.15.32:0/2435866152 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d990 msgr2=0x5614e9c1b030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc126ffd640 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d990 0x5614e9c1b030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc126ffd640 1 -- 172.21.15.32:0/2435866152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d070 msgr2=0x5614e9c34d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc126ffd640 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d070 0x5614e9c34d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc126ffd640 1 -- 172.21.15.32:0/2435866152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc1280ca160 con 0x5614e9be31c0 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.566+0000 7fc1277fe640 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d990 0x5614e9c1b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc126ffd640 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c347f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fc11801b710 tx=0x7fc11801d340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d28ab035d67cb6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc105ffb640 1 -- 172.21.15.32:0/2435866152 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180c1070 con 0x5614e9be31c0 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 -- 172.21.15.32:0/2435866152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614e9a00c10 con 0x5614e9be31c0 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc105ffb640 1 -- 172.21.15.32:0/2435866152 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc118006270 con 0x5614e9be31c0 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc105ffb640 1 -- 172.21.15.32:0/2435866152 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc118006430 con 0x5614e9be31c0 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 10 client.10161 fetch_fsmap learned FSMap version 87 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 -- 172.21.15.32:0/2435866152 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x5614e99f6260 con 0x5614e9be31c0 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc105ffb640 1 -- 172.21.15.32:0/2435866152 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614e9a00c10 con 0x5614e9be31c0 2023-12-20T06:00:43.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc105ffb640 1 -- 172.21.15.32:0/2435866152 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc1180cd030 con 0x5614e9be31c0 2023-12-20T06:00:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 10 client.10161 fetch_fsmap finished waiting for FSMap version 87 2023-12-20T06:00:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 10 client.10161 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T06:00:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 20 client.10161 populate_metadata read hostname 'smithi032' 2023-12-20T06:00:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.32:6836/2253884246,v1:172.21.15.32:6837/2253884246] conn(0x5614e9c1bdd0 0x5614e9c1c1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 4 client.10161 mds_command: new command op to 10026 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T06:00:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 -- 172.21.15.32:0/2435866152 --> [v2:172.21.15.32:6836/2253884246,v1:172.21.15.32:6837/2253884246] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5614e9a2fa30 con 0x5614e9c1bdd0 2023-12-20T06:00:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc11ffff640 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.32:6836/2253884246,v1:172.21.15.32:6837/2253884246] conn(0x5614e9c1bdd0 0x5614e9c1c1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc105ffb640 10 client.10161 ms_handle_connect on v2:172.21.15.32:6836/2253884246 2023-12-20T06:00:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc11ffff640 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.32:6836/2253884246,v1:172.21.15.32:6837/2253884246] conn(0x5614e9c1bdd0 0x5614e9c1c1b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eac270d1dbf3d5a5 server_cookie=4d8aa48cdc490de5 in_seq=0 out_seq=0 2023-12-20T06:00:43.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc105ffb640 1 -- 172.21.15.32:0/2435866152 <== mds.0 v2:172.21.15.32:6836/2253884246 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5614e9a2fa30 con 0x5614e9c1bdd0 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc105ffb640 10 client.10161 handle_command_reply: tid=0 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 client.10161 shutdown 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 -- 172.21.15.32:0/2435866152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 msgr2=0x5614e9c347f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c347f0 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7fc11801b710 tx=0x7fc11801d340 comp rx=0 tx=0).stop 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 -- 172.21.15.32:0/2435866152 shutdown_connections 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d990 0x5614e9c1b030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.32:6836/2253884246,v1:172.21.15.32:6837/2253884246] conn(0x5614e9c1bdd0 0x5614e9c1c1b0 crc :-1 s=READY pgs=11 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d070 0x5614e9c34d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 --2- 172.21.15.32:0/2435866152 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c347f0 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 -- 172.21.15.32:0/2435866152 >> 172.21.15.32:0/2435866152 conn(0x5614e9be9950 msgr2=0x5614e9be1c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc105ffb640 0 client.10161 ms_handle_reset on v2:172.21.15.32:6836/2253884246 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 -- 172.21.15.32:0/2435866152 shutdown_connections 2023-12-20T06:00:43.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 1 -- 172.21.15.32:0/2435866152 wait complete. 2023-12-20T06:00:43.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.570+0000 7fc1309bb000 20 client.10161 trim_cache size 0 max 16384 2023-12-20T06:00:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc1309bb000 1 Processor -- start 2023-12-20T06:00:43.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc1309bb000 1 -- start start 2023-12-20T06:00:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9be31c0 0x5614e9c85710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c85c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d990 0x5614e9c86190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc1309bb000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e96ee940 con 0x5614e9c2d070 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc1309bb000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e9be89e0 con 0x5614e9be31c0 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc1309bb000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e9be5d70 con 0x5614e9c2d990 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c85c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc11ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c85c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47412/0 (socket says 172.21.15.32:47412) 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc11ffff640 1 -- 172.21.15.32:0/3400571479 learned_addr learned my addr 172.21.15.32:0/3400571479 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc1277fe640 1 --2- 172.21.15.32:0/3400571479 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d990 0x5614e9c86190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc126ffd640 1 --2- 172.21.15.32:0/3400571479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9be31c0 0x5614e9c85710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc11ffff640 1 -- 172.21.15.32:0/3400571479 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d990 msgr2=0x5614e9c86190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc11ffff640 1 --2- 172.21.15.32:0/3400571479 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d990 0x5614e9c86190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc11ffff640 1 -- 172.21.15.32:0/3400571479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9be31c0 msgr2=0x5614e9c85710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc11ffff640 1 --2- 172.21.15.32:0/3400571479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9be31c0 0x5614e9c85710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.586+0000 7fc11ffff640 1 -- 172.21.15.32:0/3400571479 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5614e99dc3b0 con 0x5614e9c2d070 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc11ffff640 1 --2- 172.21.15.32:0/3400571479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c85c50 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fc1100011f0 tx=0x7fc110009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=383ae4bde41415e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc11d7fa640 1 -- 172.21.15.32:0/3400571479 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc110018e60 con 0x5614e9c2d070 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc11d7fa640 1 -- 172.21.15.32:0/3400571479 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc11000d040 con 0x5614e9c2d070 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc11d7fa640 1 -- 172.21.15.32:0/3400571479 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1100076b0 con 0x5614e9c2d070 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 -- 172.21.15.32:0/3400571479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 msgr2=0x5614e9c85c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 --2- 172.21.15.32:0/3400571479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c85c50 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7fc1100011f0 tx=0x7fc110009040 comp rx=0 tx=0).stop 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 -- 172.21.15.32:0/3400571479 shutdown_connections 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 --2- 172.21.15.32:0/3400571479 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d990 0x5614e9c86190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 --2- 172.21.15.32:0/3400571479 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9c2d070 0x5614e9c85c50 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 --2- 172.21.15.32:0/3400571479 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9be31c0 0x5614e9c85710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 -- 172.21.15.32:0/3400571479 >> 172.21.15.32:0/3400571479 conn(0x5614e9be9950 msgr2=0x5614e9c18180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 -- 172.21.15.32:0/3400571479 shutdown_connections 2023-12-20T06:00:43.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 -- 172.21.15.32:0/3400571479 wait complete. 2023-12-20T06:00:43.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 -- start start 2023-12-20T06:00:43.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 Processor -- start 2023-12-20T06:00:43.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c84ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d070 0x5614e9c85920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d990 0x5614e9c85e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e99d9860 con 0x5614e9be31c0 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5614e99d7da0 con 0x5614e9c2d990 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x5614e9bec7f0 con 0x5614e9c2d070 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 10 client.0 _ll_register_callbacks cb 0x5614e99df640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc126ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c84ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc126ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c84ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47420/0 (socket says 172.21.15.32:47420) 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc126ffd640 1 -- 172.21.15.32:0/4061198045 learned_addr learned my addr 172.21.15.32:0/4061198045 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1277fe640 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d990 0x5614e9c85e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc11ffff640 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d070 0x5614e9c85920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc126ffd640 1 -- 172.21.15.32:0/4061198045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d070 msgr2=0x5614e9c85920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc126ffd640 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d070 0x5614e9c85920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc126ffd640 1 -- 172.21.15.32:0/4061198045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d990 msgr2=0x5614e9c85e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc126ffd640 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d990 0x5614e9c85e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc126ffd640 1 -- 172.21.15.32:0/4061198045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc114003530 con 0x5614e9be31c0 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc126ffd640 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c84ff0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fc1180b9ac0 tx=0x7fc1180be070 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f225c8635490af9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc105ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc105ffb640 1 -- 172.21.15.32:0/4061198045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc1180de070 con 0x5614e9be31c0 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc105ffb640 1 -- 172.21.15.32:0/4061198045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc1180b9d50 con 0x5614e9be31c0 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.590+0000 7fc1309bb000 1 -- 172.21.15.32:0/4061198045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5614e9c2c770 con 0x5614e9be31c0 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc105ffb640 1 -- 172.21.15.32:0/4061198045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=87) v2 ==== 24+0+0 (secure 0 0 0) 0x7fc118006030 con 0x5614e9be31c0 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 10 client.10167 fetch_fsmap learned FSMap version 87 2023-12-20T06:00:43.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 -- 172.21.15.32:0/4061198045 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=87,monmap=2+}) v3 -- 0x5614e972dcc0 con 0x5614e9be31c0 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc105ffb640 1 -- 172.21.15.32:0/4061198045 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5614e9c2c770 con 0x5614e9be31c0 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc105ffb640 1 -- 172.21.15.32:0/4061198045 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 87) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fc1180e1030 con 0x5614e9be31c0 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 10 client.10167 fetch_fsmap finished waiting for FSMap version 87 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 10 client.10167 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 20 client.10167 populate_metadata read hostname 'smithi032' 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.32:6836/2253884246,v1:172.21.15.32:6837/2253884246] conn(0x5614e9c86950 0x5614e9c86d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 4 client.10167 mds_command: new command op to 10026 tid=0[{"prefix": "status"}] 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 -- 172.21.15.32:0/4061198045 --> [v2:172.21.15.32:6836/2253884246,v1:172.21.15.32:6837/2253884246] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5614e99f6e00 con 0x5614e9c86950 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc11ffff640 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.32:6836/2253884246,v1:172.21.15.32:6837/2253884246] conn(0x5614e9c86950 0x5614e9c86d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc11ffff640 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.32:6836/2253884246,v1:172.21.15.32:6837/2253884246] conn(0x5614e9c86950 0x5614e9c86d30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b80dbf143cd3ab5 server_cookie=dde5d188beab20b4 in_seq=0 out_seq=0 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc105ffb640 10 client.10167 ms_handle_connect on v2:172.21.15.32:6836/2253884246 2023-12-20T06:00:43.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc105ffb640 1 -- 172.21.15.32:0/4061198045 <== mds.0 v2:172.21.15.32:6836/2253884246 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+328 (crc 0 0 0) 0x5614e99f6e00 con 0x5614e9c86950 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc105ffb640 10 client.10167 handle_command_reply: tid=0 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 client.10167 shutdown 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 -- 172.21.15.32:0/4061198045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 msgr2=0x5614e9c84ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c84ff0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7fc1180b9ac0 tx=0x7fc1180be070 comp rx=0 tx=0).stop 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 -- 172.21.15.32:0/4061198045 shutdown_connections 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.32:6836/2253884246,v1:172.21.15.32:6837/2253884246] conn(0x5614e9c86950 0x5614e9c86d30 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x5614e9c2d990 0x5614e9c85e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x5614e9c2d070 0x5614e9c85920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 --2- 172.21.15.32:0/4061198045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x5614e9be31c0 0x5614e9c84ff0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc105ffb640 0 client.10167 ms_handle_reset on v2:172.21.15.32:6836/2253884246 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 -- 172.21.15.32:0/4061198045 >> 172.21.15.32:0/4061198045 conn(0x5614e9be9950 msgr2=0x5614e9c18a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 -- 172.21.15.32:0/4061198045 shutdown_connections 2023-12-20T06:00:43.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 1 -- 172.21.15.32:0/4061198045 wait complete. 2023-12-20T06:00:43.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc1309bb000 20 client.10167 trim_cache size 0 max 16384 2023-12-20T06:00:43.601 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:00:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T06:00:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T06:00:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 10026, 2023-12-20T06:00:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T06:00:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T06:00:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T06:00:43.601 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9906644590000004, 2023-12-20T06:00:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 87, 2023-12-20T06:00:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 192, 2023-12-20T06:00:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 192, 2023-12-20T06:00:43.602 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.311161146 2023-12-20T06:00:43.602 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 -- 172.21.15.32:0/384442736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc108065b80 msgr2=0x7fc108068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc108065b80 0x7fc108068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fc1180b1000 tx=0x7fc118008040 comp rx=0 tx=0).stop 2023-12-20T06:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 -- 172.21.15.32:0/384442736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1281599e0 msgr2=0x7fc1280b1e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1281599e0 0x7fc1280b1e30 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7fc12000c770 tx=0x7fc12000cc30 comp rx=0 tx=0).stop 2023-12-20T06:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 -- 172.21.15.32:0/384442736 shutdown_connections 2023-12-20T06:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc108065b80 0x7fc108068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc1281599e0 0x7fc1280b1e30 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc1280c90e0 0x7fc1280c3e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 --2- 172.21.15.32:0/384442736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc1280b0d00 0x7fc12815a7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 -- 172.21.15.32:0/384442736 >> 172.21.15.32:0/384442736 conn(0x7fc12806a320 msgr2=0x7fc1280cb5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:43.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 -- 172.21.15.32:0/384442736 shutdown_connections 2023-12-20T06:00:43.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.594+0000 7fc12d666640 1 -- 172.21.15.32:0/384442736 wait complete. 2023-12-20T06:00:43.614 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T06:00:43.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.758+0000 7f1d9597b640 1 Processor -- start 2023-12-20T06:00:43.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.758+0000 7f1d9597b640 1 -- start start 2023-12-20T06:00:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.758+0000 7f1d9597b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1d901599e0 0x7f1d900aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.758+0000 7f1d9597b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d900c98c0 con 0x7f1d900c90e0 2023-12-20T06:00:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.758+0000 7f1d9597b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d900c9ba0 con 0x7f1d900b0d00 2023-12-20T06:00:43.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.758+0000 7f1d9597b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1d900c9e80 con 0x7f1d901599e0 2023-12-20T06:00:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.762+0000 7f1d87fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1d900c90e0 0x7f1d900b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55600/0 (socket says 172.21.15.32:55600) 2023-12-20T06:00:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.762+0000 7f1d87fff640 1 -- 172.21.15.32:0/2565391701 learned_addr learned my addr 172.21.15.32:0/2565391701 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.762+0000 7f1d8effd640 1 --2- 172.21.15.32:0/2565391701 >> v2:172.21.15.39:3300/0 conn(0x7f1d901599e0 0x7f1d900aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.762+0000 7f1d8e7fc640 1 -- 172.21.15.32:0/2565391701 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1140940758 0 0) 0x7f1d900c98c0 con 0x7f1d900c90e0 2023-12-20T06:00:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.762+0000 7f1d8e7fc640 1 -- 172.21.15.32:0/2565391701 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d7c003530 con 0x7f1d900c90e0 2023-12-20T06:00:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.762+0000 7f1d8effd640 1 -- 172.21.15.32:0/2565391701 >> v1:172.21.15.39:6789/0 conn(0x7f1d900b0d00 legacy=0x7f1d900701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.762+0000 7f1d8effd640 1 -- 172.21.15.32:0/2565391701 >> v1:172.21.15.32:6789/0 conn(0x7f1d900c90e0 legacy=0x7f1d900b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.762+0000 7f1d8effd640 1 -- 172.21.15.32:0/2565391701 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d900ca160 con 0x7f1d901599e0 2023-12-20T06:00:43.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.762+0000 7f1d8effd640 1 --2- 172.21.15.32:0/2565391701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d901599e0 0x7f1d900aece0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1d800097a0 tx=0x7f1d8001c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=abf8c13f80ce1872 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:43.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d8e7fc640 1 -- 172.21.15.32:0/2565391701 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800ab600 con 0x7f1d901599e0 2023-12-20T06:00:43.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d8e7fc640 1 -- 172.21.15.32:0/2565391701 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d800a7050 con 0x7f1d901599e0 2023-12-20T06:00:43.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d8e7fc640 1 -- 172.21.15.32:0/2565391701 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800ab980 con 0x7f1d901599e0 2023-12-20T06:00:43.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 -- 172.21.15.32:0/2565391701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d901599e0 msgr2=0x7f1d900aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 --2- 172.21.15.32:0/2565391701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d901599e0 0x7f1d900aece0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7f1d800097a0 tx=0x7f1d8001c430 comp rx=0 tx=0).stop 2023-12-20T06:00:43.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 -- 172.21.15.32:0/2565391701 shutdown_connections 2023-12-20T06:00:43.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 --2- 172.21.15.32:0/2565391701 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d901599e0 0x7f1d900aece0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 -- 172.21.15.32:0/2565391701 >> 172.21.15.32:0/2565391701 conn(0x7f1d9006a320 msgr2=0x7f1d900ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:43.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 -- 172.21.15.32:0/2565391701 shutdown_connections 2023-12-20T06:00:43.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 -- 172.21.15.32:0/2565391701 wait complete. 2023-12-20T06:00:43.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 Processor -- start 2023-12-20T06:00:43.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 -- start start 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d900b0d00 0x7f1d90158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d900c90e0 0x7f1d900c4490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d901599e0 0x7f1d90149730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d901412b0 con 0x7f1d900b0d00 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d900abaa0 con 0x7f1d900c90e0 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d9597b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d90140a10 con 0x7f1d901599e0 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d8effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d900b0d00 0x7f1d90158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d8effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d900b0d00 0x7f1d90158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47432/0 (socket says 172.21.15.32:47432) 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.766+0000 7f1d8effd640 1 -- 172.21.15.32:0/3272235476 learned_addr learned my addr 172.21.15.32:0/3272235476 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8effd640 1 -- 172.21.15.32:0/3272235476 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d901599e0 msgr2=0x7f1d90149730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d87fff640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d900c90e0 0x7f1d900c4490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8effd640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d901599e0 0x7f1d90149730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8effd640 1 -- 172.21.15.32:0/3272235476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d900c90e0 msgr2=0x7f1d900c4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8effd640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d900c90e0 0x7f1d900c4490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8effd640 1 -- 172.21.15.32:0/3272235476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d800093b0 con 0x7f1d900b0d00 2023-12-20T06:00:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d87fff640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d900c90e0 0x7f1d900c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8effd640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d900b0d00 0x7f1d90158de0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f1d8001a060 tx=0x7f1d800191d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8cff9640 1 -- 172.21.15.32:0/3272235476 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d800abda0 con 0x7f1d900b0d00 2023-12-20T06:00:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8cff9640 1 -- 172.21.15.32:0/3272235476 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d800059a0 con 0x7f1d900b0d00 2023-12-20T06:00:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d9597b640 1 -- 172.21.15.32:0/3272235476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d90140cf0 con 0x7f1d900b0d00 2023-12-20T06:00:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8cff9640 1 -- 172.21.15.32:0/3272235476 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d80005d50 con 0x7f1d900b0d00 2023-12-20T06:00:43.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d9597b640 1 -- 172.21.15.32:0/3272235476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d900c49d0 con 0x7f1d900b0d00 2023-12-20T06:00:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d9597b640 1 -- 172.21.15.32:0/3272235476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d9004e000 con 0x7f1d900b0d00 2023-12-20T06:00:43.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8cff9640 1 -- 172.21.15.32:0/3272235476 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1d800b2040 con 0x7f1d900b0d00 2023-12-20T06:00:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8cff9640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d70065b80 0x7f1d70068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d8cff9640 1 -- 172.21.15.32:0/3272235476 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1d8011cde0 con 0x7f1d900b0d00 2023-12-20T06:00:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.770+0000 7f1d87fff640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d70065b80 0x7f1d70068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:43.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.774+0000 7f1d87fff640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d70065b80 0x7f1d70068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d901f9010 tx=0x7f1d78009040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:43.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.774+0000 7f1d8cff9640 1 -- 172.21.15.32:0/3272235476 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1d800ea530 con 0x7f1d900b0d00 2023-12-20T06:00:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.966+0000 7f1d9597b640 1 -- 172.21.15.32:0/3272235476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f1d900cf550 con 0x7f1d900b0d00 2023-12-20T06:00:43.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.966+0000 7f1d8cff9640 1 -- 172.21.15.32:0/3272235476 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v87) v1 ==== 53+0+73 (secure 0 0 0) 0x7f1d800b0060 con 0x7f1d900b0d00 2023-12-20T06:00:43.968 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T06:00:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 -- 172.21.15.32:0/3272235476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d70065b80 msgr2=0x7f1d70068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d70065b80 0x7f1d70068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1d901f9010 tx=0x7f1d78009040 comp rx=0 tx=0).stop 2023-12-20T06:00:43.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 -- 172.21.15.32:0/3272235476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d900b0d00 msgr2=0x7f1d90158de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d900b0d00 0x7f1d90158de0 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f1d8001a060 tx=0x7f1d800191d0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 -- 172.21.15.32:0/3272235476 shutdown_connections 2023-12-20T06:00:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d70065b80 0x7f1d70068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d901599e0 0x7f1d90149730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d900c90e0 0x7f1d900c4490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 --2- 172.21.15.32:0/3272235476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d900b0d00 0x7f1d90158de0 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 -- 172.21.15.32:0/3272235476 >> 172.21.15.32:0/3272235476 conn(0x7f1d9006a320 msgr2=0x7f1d9015aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 -- 172.21.15.32:0/3272235476 shutdown_connections 2023-12-20T06:00:43.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:43.970+0000 7f1d9597b640 1 -- 172.21.15.32:0/3272235476 wait complete. 2023-12-20T06:00:43.989 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T06:00:43.990 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T06:00:43.990 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T06:00:43.990 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T06:00:43.990 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T06:00:43.990 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T06:00:43.990 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T06:00:43.990 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T06:00:43.990 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:00:43.990 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:43.991 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T06:00:44.001 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T06:00:44.001 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T06:00:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T06:00:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:00:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T06:00:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:00:44.002 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:00:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T06:00:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T06:00:44.002 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T06:00:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:00:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T06:00:44.002 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T06:00:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 36816sec preferred_lft 36816sec 2023-12-20T06:00:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T06:00:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:00:44.003 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:00:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T06:00:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T06:00:44.003 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:00:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:00:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T06:00:44.003 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T06:00:44.003 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:00:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T06:00:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:00:44.004 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T06:00:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T06:00:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T06:00:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:00:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T06:00:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:00:44.004 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T06:00:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T06:00:44.004 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T06:00:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:00:44.005 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T06:00:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T06:00:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T06:00:44.005 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:00:44.005 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:44.005 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T06:00:44.068 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T06:00:44.070 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:44.070 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T06:00:44.126 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T06:00:44.126 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T06:00:44.126 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T06:00:44.126 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T06:00:44.126 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T06:00:44.126 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T06:00:44.127 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:44.127 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T06:00:44.127 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T06:00:44.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T06:00:44.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T06:00:44.127 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T06:00:44.127 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T06:00:44.221 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:44.221 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T06:00:44.234 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T06:00:44.234 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T06:00:44.234 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:00:44.234 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:44.235 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T06:00:44.301 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T06:00:44.302 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T06:00:44.357 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:00:44.357 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T06:00:44.357 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T06:00:44.408 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T06:00:44.408 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:44.408 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:00:44.508 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:00:44.508 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T06:00:44.522 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:00:44.522 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T06:00:44.522 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T06:00:44.522 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T06:00:44.522 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T06:00:44.526 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T06:00:44.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:44.526 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T06:00:44.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:44.548 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T06:00:44.608 INFO:teuthology.orchestra.run.smithi032.stdout:client9766 2023-12-20T06:00:44.609 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:00:44.609 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:00:44.665 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T06:00:44.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5cde8c640 1 Processor -- start 2023-12-20T06:00:44.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5cde8c640 1 -- start start 2023-12-20T06:00:44.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5cde8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c81599e0 0x7fd5c80aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:44.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5cde8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd5c80c90e0 0x7fd5c80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:44.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5cde8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5c80711d0 con 0x7fd5c80b0b30 2023-12-20T06:00:44.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5cde8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5c80c98c0 con 0x7fd5c80c90e0 2023-12-20T06:00:44.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5cde8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c80c9ba0 con 0x7fd5c81599e0 2023-12-20T06:00:44.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5c7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd5c80b0b30 0x7fd5c815d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55604/0 (socket says 172.21.15.32:55604) 2023-12-20T06:00:44.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5c7fff640 1 -- 172.21.15.32:0/232364505 learned_addr learned my addr 172.21.15.32:0/232364505 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:44.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5c77fe640 1 --2- 172.21.15.32:0/232364505 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c81599e0 0x7fd5c80aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:44.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5c6ffd640 1 --2- 172.21.15.32:0/232364505 >> v2:172.21.15.39:3300/0 conn(0x7fd5c80c90e0 0x7fd5c80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:44.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5c67fc640 1 -- 172.21.15.32:0/232364505 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619998579 0 0) 0x7fd5c80711d0 con 0x7fd5c80b0b30 2023-12-20T06:00:44.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5c67fc640 1 -- 172.21.15.32:0/232364505 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd5b0003530 con 0x7fd5c80b0b30 2023-12-20T06:00:44.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5c77fe640 1 -- 172.21.15.32:0/232364505 >> v2:172.21.15.39:3300/0 conn(0x7fd5c80c90e0 msgr2=0x7fd5c80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:44.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5c77fe640 1 --2- 172.21.15.32:0/232364505 >> v2:172.21.15.39:3300/0 conn(0x7fd5c80c90e0 0x7fd5c80b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:44.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5c77fe640 1 -- 172.21.15.32:0/232364505 >> v1:172.21.15.32:6789/0 conn(0x7fd5c80b0b30 legacy=0x7fd5c815d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:44.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5c77fe640 1 -- 172.21.15.32:0/232364505 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5c80c9e80 con 0x7fd5c81599e0 2023-12-20T06:00:44.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.854+0000 7fd5c6ffd640 1 --2- 172.21.15.32:0/232364505 >> v2:172.21.15.39:3300/0 conn(0x7fd5c80c90e0 0x7fd5c80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:44.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5c77fe640 1 --2- 172.21.15.32:0/232364505 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c81599e0 0x7fd5c80aece0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80097a0 tx=0x7fd5b801c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=78515825e7a0b0a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:44.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5c67fc640 1 -- 172.21.15.32:0/232364505 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8004230 con 0x7fd5c81599e0 2023-12-20T06:00:44.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5c67fc640 1 -- 172.21.15.32:0/232364505 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5b80a7050 con 0x7fd5c81599e0 2023-12-20T06:00:44.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5c67fc640 1 -- 172.21.15.32:0/232364505 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5b8003620 con 0x7fd5c81599e0 2023-12-20T06:00:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5cde8c640 1 -- 172.21.15.32:0/232364505 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c81599e0 msgr2=0x7fd5c80aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5cde8c640 1 --2- 172.21.15.32:0/232364505 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c81599e0 0x7fd5c80aece0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80097a0 tx=0x7fd5b801c430 comp rx=0 tx=0).stop 2023-12-20T06:00:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5cde8c640 1 -- 172.21.15.32:0/232364505 shutdown_connections 2023-12-20T06:00:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5cde8c640 1 --2- 172.21.15.32:0/232364505 >> v2:172.21.15.39:3300/0 conn(0x7fd5c80c90e0 0x7fd5c80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5cde8c640 1 --2- 172.21.15.32:0/232364505 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c81599e0 0x7fd5c80aece0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:44.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5cde8c640 1 -- 172.21.15.32:0/232364505 >> 172.21.15.32:0/232364505 conn(0x7fd5c806a320 msgr2=0x7fd5c80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:44.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5cde8c640 1 -- 172.21.15.32:0/232364505 shutdown_connections 2023-12-20T06:00:44.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.858+0000 7fd5cde8c640 1 -- 172.21.15.32:0/232364505 wait complete. 2023-12-20T06:00:44.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5cde8c640 1 Processor -- start 2023-12-20T06:00:44.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5cde8c640 1 -- start start 2023-12-20T06:00:44.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5cde8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c80b0b30 0x7fd5c80c7840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:44.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5cde8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c80c90e0 0x7fd5c8149750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:44.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5cde8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c81599e0 0x7fd5c814ed30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:44.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5cde8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c8158560 con 0x7fd5c80c90e0 2023-12-20T06:00:44.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5cde8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c81587d0 con 0x7fd5c81599e0 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5cde8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5c8158ab0 con 0x7fd5c80b0b30 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c80c90e0 0x7fd5c8149750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c80c90e0 0x7fd5c8149750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47436/0 (socket says 172.21.15.32:47436) 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c6ffd640 1 -- 172.21.15.32:0/2587487505 learned_addr learned my addr 172.21.15.32:0/2587487505 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c7fff640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c81599e0 0x7fd5c814ed30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c77fe640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c80b0b30 0x7fd5c80c7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c6ffd640 1 -- 172.21.15.32:0/2587487505 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c80b0b30 msgr2=0x7fd5c80c7840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c6ffd640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c80b0b30 0x7fd5c80c7840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c6ffd640 1 -- 172.21.15.32:0/2587487505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c81599e0 msgr2=0x7fd5c814ed30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c6ffd640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c81599e0 0x7fd5c814ed30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c6ffd640 1 -- 172.21.15.32:0/2587487505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5bc009420 con 0x7fd5c80c90e0 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c7fff640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c81599e0 0x7fd5c814ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c77fe640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c80b0b30 0x7fd5c80c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:00:44.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5c6ffd640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c80c90e0 0x7fd5c8149750 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00ebe0 tx=0x7fd5bc00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:44.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5affff640 1 -- 172.21.15.32:0/2587487505 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc00eee0 con 0x7fd5c80c90e0 2023-12-20T06:00:44.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5affff640 1 -- 172.21.15.32:0/2587487505 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5bc00cbd0 con 0x7fd5c80c90e0 2023-12-20T06:00:44.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5affff640 1 -- 172.21.15.32:0/2587487505 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5bc013470 con 0x7fd5c80c90e0 2023-12-20T06:00:44.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5cde8c640 1 -- 172.21.15.32:0/2587487505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5b80093b0 con 0x7fd5c80c90e0 2023-12-20T06:00:44.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5cde8c640 1 -- 172.21.15.32:0/2587487505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5c8159070 con 0x7fd5c80c90e0 2023-12-20T06:00:44.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.862+0000 7fd5cde8c640 1 -- 172.21.15.32:0/2587487505 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5c804e000 con 0x7fd5c80c90e0 2023-12-20T06:00:44.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.866+0000 7fd5affff640 1 -- 172.21.15.32:0/2587487505 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd5bc0135d0 con 0x7fd5c80c90e0 2023-12-20T06:00:44.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.866+0000 7fd5affff640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5b4065b80 0x7fd5b4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:44.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.866+0000 7fd5affff640 1 -- 172.21.15.32:0/2587487505 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd5bc083600 con 0x7fd5c80c90e0 2023-12-20T06:00:44.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.866+0000 7fd5c77fe640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5b4065b80 0x7fd5b4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:44.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.866+0000 7fd5c77fe640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5b4065b80 0x7fd5b4068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80ad000 tx=0x7fd5b80a6000 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:44.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:44.870+0000 7fd5affff640 1 -- 172.21.15.32:0/2587487505 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd5bc050e60 con 0x7fd5c80c90e0 2023-12-20T06:00:45.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.050+0000 7fd5cde8c640 1 -- 172.21.15.32:0/2587487505 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5c80b0f40 con 0x7fd5b4065b80 2023-12-20T06:00:45.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5affff640 1 -- 172.21.15.32:0/2587487505 <== mgr.9786 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fd5b8006dc0 con 0x7fd5b4065b80 2023-12-20T06:00:45.054 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T06:00:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:00:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T06:00:45.054 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:00:45.054 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 -- 172.21.15.32:0/2587487505 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5b4065b80 msgr2=0x7fd5b4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5b4065b80 0x7fd5b4068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd5b80ad000 tx=0x7fd5b80a6000 comp rx=0 tx=0).stop 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 -- 172.21.15.32:0/2587487505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c80c90e0 msgr2=0x7fd5c8149750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c80c90e0 0x7fd5c8149750 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7fd5bc00ebe0 tx=0x7fd5bc00c520 comp rx=0 tx=0).stop 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 -- 172.21.15.32:0/2587487505 shutdown_connections 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5b4065b80 0x7fd5b4068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5c81599e0 0x7fd5c814ed30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5c80c90e0 0x7fd5c8149750 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 --2- 172.21.15.32:0/2587487505 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5c80b0b30 0x7fd5c80c7840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 -- 172.21.15.32:0/2587487505 >> 172.21.15.32:0/2587487505 conn(0x7fd5c806a320 msgr2=0x7fd5c815af00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 -- 172.21.15.32:0/2587487505 shutdown_connections 2023-12-20T06:00:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.054+0000 7fd5cde8c640 1 -- 172.21.15.32:0/2587487505 wait complete. 2023-12-20T06:00:45.075 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T06:00:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452882c640 1 Processor -- start 2023-12-20T06:00:45.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452882c640 1 -- start start 2023-12-20T06:00:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452882c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f452415cc30 0x7f45240b8520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452882c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45240d0ca0 0x7f45240d2950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452882c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45240b6b00 con 0x7f45240d2e90 2023-12-20T06:00:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452882c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45240d3320 con 0x7f45240d0ca0 2023-12-20T06:00:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452882c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f452415d010 con 0x7f452415cc30 2023-12-20T06:00:45.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45240d0ca0 0x7f45240d2950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452359e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f45240d2e90 0x7f45240b4440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55612/0 (socket says 172.21.15.32:55612) 2023-12-20T06:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452359e640 1 -- 172.21.15.32:0/1965163166 learned_addr learned my addr 172.21.15.32:0/1965163166 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f4522d9d640 1 --2- 172.21.15.32:0/1965163166 >> v2:172.21.15.39:3300/0 conn(0x7f452415cc30 0x7f45240b8520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f4521d9b640 1 -- 172.21.15.32:0/1965163166 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1647016678 0 0) 0x7f451c0014a0 con 0x7f45240d2e90 2023-12-20T06:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f4521d9b640 1 -- 172.21.15.32:0/1965163166 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f450c003530 con 0x7f45240d2e90 2023-12-20T06:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452259c640 1 -- 172.21.15.32:0/1965163166 >> v2:172.21.15.39:3300/0 conn(0x7f452415cc30 msgr2=0x7f45240b8520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452259c640 1 --2- 172.21.15.32:0/1965163166 >> v2:172.21.15.39:3300/0 conn(0x7f452415cc30 0x7f45240b8520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452259c640 1 -- 172.21.15.32:0/1965163166 >> v1:172.21.15.32:6789/0 conn(0x7f45240d2e90 legacy=0x7f45240b4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f452259c640 1 -- 172.21.15.32:0/1965163166 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45240d2090 con 0x7f45240d0ca0 2023-12-20T06:00:45.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.222+0000 7f4522d9d640 1 --2- 172.21.15.32:0/1965163166 >> v2:172.21.15.39:3300/0 conn(0x7f452415cc30 0x7f45240b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f452259c640 1 --2- 172.21.15.32:0/1965163166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45240d0ca0 0x7f45240d2950 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f452406c030 tx=0x7f451801c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cfe9975ba1fcf8b4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f4521d9b640 1 -- 172.21.15.32:0/1965163166 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45180ab600 con 0x7f45240d0ca0 2023-12-20T06:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f4521d9b640 1 -- 172.21.15.32:0/1965163166 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45180a7050 con 0x7f45240d0ca0 2023-12-20T06:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f4521d9b640 1 -- 172.21.15.32:0/1965163166 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45180ab980 con 0x7f45240d0ca0 2023-12-20T06:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f452882c640 1 -- 172.21.15.32:0/1965163166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45240d0ca0 msgr2=0x7f45240d2950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f452882c640 1 --2- 172.21.15.32:0/1965163166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45240d0ca0 0x7f45240d2950 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f452406c030 tx=0x7f451801c410 comp rx=0 tx=0).stop 2023-12-20T06:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f452882c640 1 -- 172.21.15.32:0/1965163166 shutdown_connections 2023-12-20T06:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f452882c640 1 --2- 172.21.15.32:0/1965163166 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45240d0ca0 0x7f45240d2950 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f452882c640 1 --2- 172.21.15.32:0/1965163166 >> v2:172.21.15.39:3300/0 conn(0x7f452415cc30 0x7f45240b8520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f452882c640 1 -- 172.21.15.32:0/1965163166 >> 172.21.15.32:0/1965163166 conn(0x7f4524065ce0 msgr2=0x7f45240660f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:45.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f452882c640 1 -- 172.21.15.32:0/1965163166 shutdown_connections 2023-12-20T06:00:45.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.226+0000 7f452882c640 1 -- 172.21.15.32:0/1965163166 wait complete. 2023-12-20T06:00:45.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f452882c640 1 Processor -- start 2023-12-20T06:00:45.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f452882c640 1 -- start start 2023-12-20T06:00:45.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f452882c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45240d0ca0 0x7f452415fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f452882c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45240d2e90 0x7f452414b9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f452882c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452415cc30 0x7f452414bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f452882c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f452414c420 con 0x7f45240d0ca0 2023-12-20T06:00:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f452882c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4524159da0 con 0x7f45240d2e90 2023-12-20T06:00:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f452882c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4524160130 con 0x7f452415cc30 2023-12-20T06:00:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f4522d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45240d0ca0 0x7f452415fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f4522d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45240d0ca0 0x7f452415fbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47444/0 (socket says 172.21.15.32:47444) 2023-12-20T06:00:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f4522d9d640 1 -- 172.21.15.32:0/1358856906 learned_addr learned my addr 172.21.15.32:0/1358856906 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f452259c640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45240d2e90 0x7f452414b9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f452359e640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452415cc30 0x7f452414bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f4522d9d640 1 -- 172.21.15.32:0/1358856906 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452415cc30 msgr2=0x7f452414bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f4522d9d640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452415cc30 0x7f452414bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f4522d9d640 1 -- 172.21.15.32:0/1358856906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45240d2e90 msgr2=0x7f452414b9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f4522d9d640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45240d2e90 0x7f452414b9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f4522d9d640 1 -- 172.21.15.32:0/1358856906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4514009420 con 0x7f45240d0ca0 2023-12-20T06:00:45.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f4522d9d640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45240d0ca0 0x7f452415fbf0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f451400ebe0 tx=0x7f451400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:45.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f450b7fe640 1 -- 172.21.15.32:0/1358856906 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f451400eee0 con 0x7f45240d0ca0 2023-12-20T06:00:45.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f450b7fe640 1 -- 172.21.15.32:0/1358856906 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f451400cc30 con 0x7f45240d0ca0 2023-12-20T06:00:45.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f452882c640 1 -- 172.21.15.32:0/1358856906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4518009420 con 0x7f45240d0ca0 2023-12-20T06:00:45.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.230+0000 7f450b7fe640 1 -- 172.21.15.32:0/1358856906 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4514013470 con 0x7f45240d0ca0 2023-12-20T06:00:45.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.234+0000 7f452882c640 1 -- 172.21.15.32:0/1358856906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f452414f680 con 0x7f45240d0ca0 2023-12-20T06:00:45.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.234+0000 7f452882c640 1 -- 172.21.15.32:0/1358856906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4524074100 con 0x7f45240d0ca0 2023-12-20T06:00:45.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.234+0000 7f450b7fe640 1 -- 172.21.15.32:0/1358856906 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 113) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f45140135d0 con 0x7f45240d0ca0 2023-12-20T06:00:45.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.234+0000 7f450b7fe640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4510065b80 0x7f4510068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.234+0000 7f450b7fe640 1 -- 172.21.15.32:0/1358856906 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f4514083610 con 0x7f45240d0ca0 2023-12-20T06:00:45.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.234+0000 7f452259c640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4510065b80 0x7f4510068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:45.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.234+0000 7f452259c640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4510065b80 0x7f4510068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45180ad000 tx=0x7f4518008040 comp rx=0 tx=0).ready entity=mgr.9786 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:45.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.238+0000 7f450b7fe640 1 -- 172.21.15.32:0/1358856906 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4514050e90 con 0x7f45240d0ca0 2023-12-20T06:00:45.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.434+0000 7f452882c640 1 -- 172.21.15.32:0/1358856906 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f4524156030 con 0x7f45240d0ca0 2023-12-20T06:00:45.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.690+0000 7f450b7fe640 1 -- 172.21.15.32:0/1358856906 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 114) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f4514051260 con 0x7f45240d0ca0 2023-12-20T06:00:45.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.690+0000 7f450b7fe640 1 -- 172.21.15.32:0/1358856906 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v114) v1 ==== 92+0+0 (secure 0 0 0) 0x7f451407ff70 con 0x7f45240d0ca0 2023-12-20T06:00:45.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 -- 172.21.15.32:0/1358856906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4510065b80 msgr2=0x7f4510068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4510065b80 0x7f4510068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f45180ad000 tx=0x7f4518008040 comp rx=0 tx=0).stop 2023-12-20T06:00:45.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 -- 172.21.15.32:0/1358856906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45240d0ca0 msgr2=0x7f452415fbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45240d0ca0 0x7f452415fbf0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f451400ebe0 tx=0x7f451400c520 comp rx=0 tx=0).stop 2023-12-20T06:00:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 -- 172.21.15.32:0/1358856906 shutdown_connections 2023-12-20T06:00:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4510065b80 0x7f4510068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f452415cc30 0x7f452414bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45240d2e90 0x7f452414b9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 --2- 172.21.15.32:0/1358856906 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45240d0ca0 0x7f452415fbf0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 -- 172.21.15.32:0/1358856906 >> 172.21.15.32:0/1358856906 conn(0x7f4524065ce0 msgr2=0x7f45240b25b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 -- 172.21.15.32:0/1358856906 shutdown_connections 2023-12-20T06:00:45.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.694+0000 7f452882c640 1 -- 172.21.15.32:0/1358856906 wait complete. 2023-12-20T06:00:45.715 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T06:00:45.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16fa739640 1 Processor -- start 2023-12-20T06:00:45.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16fa739640 1 -- start start 2023-12-20T06:00:45.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16fa739640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16f406d240 0x7f16f40ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16fa739640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f16f41603e0 0x7f16f40ca6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16fa739640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f16f406c480 con 0x7f16f4070490 2023-12-20T06:00:45.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16fa739640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f16f406c760 con 0x7f16f41603e0 2023-12-20T06:00:45.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16fa739640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f406ca40 con 0x7f16f406d240 2023-12-20T06:00:45.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16f8cd6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f16f41603e0 0x7f16f40ca6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:45.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16f37fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16f406d240 0x7f16f40ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16f8cd6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f16f41603e0 0x7f16f40ca6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:42044/0 (socket says 172.21.15.32:42044) 2023-12-20T06:00:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16f8cd6640 1 -- 172.21.15.32:0/2042642463 learned_addr learned my addr 172.21.15.32:0/2042642463 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16f8cd6640 1 -- 172.21.15.32:0/2042642463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16f406d240 msgr2=0x7f16f40ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16f8cd6640 1 --2- 172.21.15.32:0/2042642463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16f406d240 0x7f16f40ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16f8cd6640 1 -- 172.21.15.32:0/2042642463 >> v1:172.21.15.39:6789/0 conn(0x7f16f4070490 legacy=0x7f16f415fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16f8cd6640 1 -- 172.21.15.32:0/2042642463 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16f406cd20 con 0x7f16f41603e0 2023-12-20T06:00:45.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.866+0000 7f16f37fe640 1 --2- 172.21.15.32:0/2042642463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16f406d240 0x7f16f40ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16f8cd6640 1 --2- 172.21.15.32:0/2042642463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f41603e0 0x7f16f40ca6e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009e50 tx=0x7f16ec01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bc16d1fc0c1abd8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16f2ffd640 1 -- 172.21.15.32:0/2042642463 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0a5770 con 0x7f16f41603e0 2023-12-20T06:00:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16f2ffd640 1 -- 172.21.15.32:0/2042642463 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16ec0a58d0 con 0x7f16f41603e0 2023-12-20T06:00:45.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16f2ffd640 1 -- 172.21.15.32:0/2042642463 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16ec0a5c60 con 0x7f16f41603e0 2023-12-20T06:00:45.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16fa739640 1 -- 172.21.15.32:0/2042642463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f41603e0 msgr2=0x7f16f40ca6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16fa739640 1 --2- 172.21.15.32:0/2042642463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f41603e0 0x7f16f40ca6e0 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7f16ec009e50 tx=0x7f16ec01c950 comp rx=0 tx=0).stop 2023-12-20T06:00:45.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16fa739640 1 -- 172.21.15.32:0/2042642463 shutdown_connections 2023-12-20T06:00:45.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16fa739640 1 --2- 172.21.15.32:0/2042642463 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f41603e0 0x7f16f40ca6e0 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16fa739640 1 --2- 172.21.15.32:0/2042642463 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16f406d240 0x7f16f40ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16fa739640 1 -- 172.21.15.32:0/2042642463 >> 172.21.15.32:0/2042642463 conn(0x7f16f406a510 msgr2=0x7f16f40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:45.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16fa739640 1 -- 172.21.15.32:0/2042642463 shutdown_connections 2023-12-20T06:00:45.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.870+0000 7f16fa739640 1 -- 172.21.15.32:0/2042642463 wait complete. 2023-12-20T06:00:45.873 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:45.870+0000 7fb1c164d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:00:45.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.874+0000 7f16fa739640 1 Processor -- start 2023-12-20T06:00:45.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.874+0000 7f16fa739640 1 -- start start 2023-12-20T06:00:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.874+0000 7f16fa739640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16f406d240 0x7f16f414bba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.874+0000 7f16fa739640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f4070490 0x7f16f4153140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.874+0000 7f16fa739640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f41603e0 0x7f16f40b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.874+0000 7f16fa739640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f4152c50 con 0x7f16f4070490 2023-12-20T06:00:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.874+0000 7f16fa739640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f16f414c0e0 con 0x7f16f41603e0 2023-12-20T06:00:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.874+0000 7f16fa739640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f16f4152040 con 0x7f16f406d240 2023-12-20T06:00:45.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.874+0000 7f16f37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f4070490 0x7f16f4153140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f37fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f4070490 0x7f16f4153140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47464/0 (socket says 172.21.15.32:47464) 2023-12-20T06:00:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f37fe640 1 -- 172.21.15.32:0/1730650056 learned_addr learned my addr 172.21.15.32:0/1730650056 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f8cd6640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f41603e0 0x7f16f40b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f3fff640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16f406d240 0x7f16f414bba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f37fe640 1 -- 172.21.15.32:0/1730650056 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16f406d240 msgr2=0x7f16f414bba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f37fe640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16f406d240 0x7f16f414bba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f37fe640 1 -- 172.21.15.32:0/1730650056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f41603e0 msgr2=0x7f16f40b22b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f37fe640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f41603e0 0x7f16f40b22b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f37fe640 1 -- 172.21.15.32:0/1730650056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f16e8009420 con 0x7f16f4070490 2023-12-20T06:00:45.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f37fe640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f4070490 0x7f16f4153140 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e9d0 tx=0x7f16e800ee90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f0ff9640 1 -- 172.21.15.32:0/1730650056 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e800ca60 con 0x7f16f4070490 2023-12-20T06:00:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f0ff9640 1 -- 172.21.15.32:0/1730650056 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f16e800cbc0 con 0x7f16f4070490 2023-12-20T06:00:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16fa739640 1 -- 172.21.15.32:0/1730650056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f16ec009b30 con 0x7f16f4070490 2023-12-20T06:00:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f0ff9640 1 -- 172.21.15.32:0/1730650056 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f16e8013470 con 0x7f16f4070490 2023-12-20T06:00:45.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16fa739640 1 -- 172.21.15.32:0/1730650056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f16f4152600 con 0x7f16f4070490 2023-12-20T06:00:45.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16fa739640 1 -- 172.21.15.32:0/1730650056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f16f4152dc0 con 0x7f16f4070490 2023-12-20T06:00:45.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.878+0000 7f16f0ff9640 1 -- 172.21.15.32:0/1730650056 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 114) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f16e80135d0 con 0x7f16f4070490 2023-12-20T06:00:45.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.882+0000 7f16f0ff9640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f16e0065bd0 0x7f16e0068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:45.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.882+0000 7f16f3fff640 1 -- 172.21.15.32:0/1730650056 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f16e0065bd0 msgr2=0x7f16e0068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:45.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.882+0000 7f16f0ff9640 1 -- 172.21.15.32:0/1730650056 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f16e80832b0 con 0x7f16f4070490 2023-12-20T06:00:45.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.882+0000 7f16f3fff640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f16e0065bd0 0x7f16e0068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:45.882 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:45.882+0000 7fb57fa62180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:00:45.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:45.886+0000 7f16f0ff9640 1 -- 172.21.15.32:0/1730650056 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f16e8050bc0 con 0x7f16f4070490 2023-12-20T06:00:46.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.062+0000 7f16fa739640 1 -- 172.21.15.32:0/1730650056 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f16f404e000 con 0x7f16f4070490 2023-12-20T06:00:46.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16f0ff9640 1 -- 172.21.15.32:0/1730650056 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v65)=0 v65) v1 ==== 138+0+0 (secure 0 0 0) 0x7f16e80541e0 con 0x7f16f4070490 2023-12-20T06:00:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 -- 172.21.15.32:0/1730650056 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f16e0065bd0 msgr2=0x7f16e0068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f16e0065bd0 0x7f16e0068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 -- 172.21.15.32:0/1730650056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f4070490 msgr2=0x7f16f4153140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f4070490 0x7f16f4153140 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7f16e800e9d0 tx=0x7f16e800ee90 comp rx=0 tx=0).stop 2023-12-20T06:00:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 -- 172.21.15.32:0/1730650056 shutdown_connections 2023-12-20T06:00:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f16e0065bd0 0x7f16e0068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f16f41603e0 0x7f16f40b22b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f16f4070490 0x7f16f4153140 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 --2- 172.21.15.32:0/1730650056 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f16f406d240 0x7f16f414bba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 -- 172.21.15.32:0/1730650056 >> 172.21.15.32:0/1730650056 conn(0x7f16f406a510 msgr2=0x7f16f415d580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 -- 172.21.15.32:0/1730650056 shutdown_connections 2023-12-20T06:00:46.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.070+0000 7f16fa739640 1 -- 172.21.15.32:0/1730650056 wait complete. 2023-12-20T06:00:46.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T06:00:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.238+0000 7fac2e895640 1 Processor -- start 2023-12-20T06:00:46.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac2e895640 1 -- start start 2023-12-20T06:00:46.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac2e895640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac2815ce60 0x7fac280d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac2e895640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac280d2650 con 0x7fac280d3130 2023-12-20T06:00:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac2e895640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fac280d3570 con 0x7fac280d14d0 2023-12-20T06:00:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac2e895640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac280d1a40 con 0x7fac2815ce60 2023-12-20T06:00:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac277fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fac280d3130 0x7fac280d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55640/0 (socket says 172.21.15.32:55640) 2023-12-20T06:00:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac277fe640 1 -- 172.21.15.32:0/4085771594 learned_addr learned my addr 172.21.15.32:0/4085771594 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:00:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac27fff640 1 --2- 172.21.15.32:0/4085771594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac2815ce60 0x7fac280d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac26ffd640 1 -- 172.21.15.32:0/4085771594 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3285480714 0 0) 0x7fac280d2650 con 0x7fac280d3130 2023-12-20T06:00:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac26ffd640 1 -- 172.21.15.32:0/4085771594 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac10003530 con 0x7fac280d3130 2023-12-20T06:00:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac26ffd640 1 -- 172.21.15.32:0/4085771594 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 18061377 0 0) 0x7fac280d3570 con 0x7fac280d14d0 2023-12-20T06:00:46.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac26ffd640 1 -- 172.21.15.32:0/4085771594 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fac280d2650 con 0x7fac280d14d0 2023-12-20T06:00:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac27fff640 1 -- 172.21.15.32:0/4085771594 >> v1:172.21.15.39:6789/0 conn(0x7fac280d14d0 legacy=0x7fac280b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac27fff640 1 -- 172.21.15.32:0/4085771594 >> v1:172.21.15.32:6789/0 conn(0x7fac280d3130 legacy=0x7fac280d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:46.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.242+0000 7fac27fff640 1 -- 172.21.15.32:0/4085771594 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac280d1d20 con 0x7fac2815ce60 2023-12-20T06:00:46.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.246+0000 7fac27fff640 1 --2- 172.21.15.32:0/4085771594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac2815ce60 0x7fac280d2bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fac180097f0 tx=0x7fac1801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=13cd8a5467338fe4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:46.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.246+0000 7fac26ffd640 1 -- 172.21.15.32:0/4085771594 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180a89b0 con 0x7fac2815ce60 2023-12-20T06:00:46.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.246+0000 7fac26ffd640 1 -- 172.21.15.32:0/4085771594 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac180a8b10 con 0x7fac2815ce60 2023-12-20T06:00:46.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.246+0000 7fac26ffd640 1 -- 172.21.15.32:0/4085771594 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180a8ea0 con 0x7fac2815ce60 2023-12-20T06:00:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.246+0000 7fac2e895640 1 -- 172.21.15.32:0/4085771594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac2815ce60 msgr2=0x7fac280d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.246+0000 7fac2e895640 1 --2- 172.21.15.32:0/4085771594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac2815ce60 0x7fac280d2bf0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7fac180097f0 tx=0x7fac1801ecb0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.246+0000 7fac2e895640 1 -- 172.21.15.32:0/4085771594 shutdown_connections 2023-12-20T06:00:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.246+0000 7fac2e895640 1 --2- 172.21.15.32:0/4085771594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac2815ce60 0x7fac280d2bf0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.246+0000 7fac2e895640 1 -- 172.21.15.32:0/4085771594 >> 172.21.15.32:0/4085771594 conn(0x7fac2806b5e0 msgr2=0x7fac2806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.246+0000 7fac2e895640 1 -- 172.21.15.32:0/4085771594 shutdown_connections 2023-12-20T06:00:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.246+0000 7fac2e895640 1 -- 172.21.15.32:0/4085771594 wait complete. 2023-12-20T06:00:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac2e895640 1 Processor -- start 2023-12-20T06:00:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac2e895640 1 -- start start 2023-12-20T06:00:46.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac2e895640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac280d14d0 0x7fac28144dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac2e895640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac280d3130 0x7fac281e3820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac2e895640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac2815ce60 0x7fac281e3d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac2e895640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac28142f50 con 0x7fac280d3130 2023-12-20T06:00:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac2e895640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fac28145890 con 0x7fac280d14d0 2023-12-20T06:00:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac2e895640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fac28145310 con 0x7fac2815ce60 2023-12-20T06:00:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac27fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac280d14d0 0x7fac28144dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac2ce32640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac2815ce60 0x7fac281e3d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac27fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac280d14d0 0x7fac28144dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:42062/0 (socket says 172.21.15.32:42062) 2023-12-20T06:00:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac27fff640 1 -- 172.21.15.32:0/1309963593 learned_addr learned my addr 172.21.15.32:0/1309963593 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac277fe640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac280d3130 0x7fac281e3820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac27fff640 1 -- 172.21.15.32:0/1309963593 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac2815ce60 msgr2=0x7fac281e3d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac27fff640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac2815ce60 0x7fac281e3d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac27fff640 1 -- 172.21.15.32:0/1309963593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac280d3130 msgr2=0x7fac281e3820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac27fff640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac280d3130 0x7fac281e3820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac27fff640 1 -- 172.21.15.32:0/1309963593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fac18009420 con 0x7fac280d14d0 2023-12-20T06:00:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac27fff640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac280d14d0 0x7fac28144dd0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fac1801b8c0 tx=0x7fac1801ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac24ff9640 1 -- 172.21.15.32:0/1309963593 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180b0d80 con 0x7fac280d14d0 2023-12-20T06:00:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac24ff9640 1 -- 172.21.15.32:0/1309963593 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fac180b9430 con 0x7fac280d14d0 2023-12-20T06:00:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac24ff9640 1 -- 172.21.15.32:0/1309963593 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fac180b97e0 con 0x7fac280d14d0 2023-12-20T06:00:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac2e895640 1 -- 172.21.15.32:0/1309963593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fac281e7050 con 0x7fac280d14d0 2023-12-20T06:00:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.250+0000 7fac2e895640 1 -- 172.21.15.32:0/1309963593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fac28142be0 con 0x7fac280d14d0 2023-12-20T06:00:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.254+0000 7fac2e895640 1 -- 172.21.15.32:0/1309963593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fac28142a40 con 0x7fac280d14d0 2023-12-20T06:00:46.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.254+0000 7fac24ff9640 1 -- 172.21.15.32:0/1309963593 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 114) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fac180b7040 con 0x7fac280d14d0 2023-12-20T06:00:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.254+0000 7fac24ff9640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fac14065bd0 0x7fac14068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.254+0000 7fac24ff9640 1 -- 172.21.15.32:0/1309963593 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fac18120d80 con 0x7fac280d14d0 2023-12-20T06:00:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.254+0000 7fac277fe640 1 -- 172.21.15.32:0/1309963593 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fac14065bd0 msgr2=0x7fac14068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.254+0000 7fac277fe640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fac14065bd0 0x7fac14068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.258+0000 7fac24ff9640 1 -- 172.21.15.32:0/1309963593 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fac28142a40 con 0x7fac280d14d0 2023-12-20T06:00:46.310 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:46.310+0000 7fb1c164d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:00:46.343 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:46.342+0000 7fb57fa62180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:00:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.418+0000 7fac2e895640 1 -- 172.21.15.32:0/1309963593 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fac280d1fe0 con 0x7fac280d14d0 2023-12-20T06:00:46.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.430+0000 7fac24ff9640 1 -- 172.21.15.32:0/1309963593 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v66)=0 v66) v1 ==== 133+0+0 (secure 0 0 0) 0x7fac280d1fe0 con 0x7fac280d14d0 2023-12-20T06:00:46.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.430+0000 7fac2e895640 1 -- 172.21.15.32:0/1309963593 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fac14065bd0 msgr2=0x7fac14068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.430+0000 7fac2e895640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fac14065bd0 0x7fac14068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.430+0000 7fac2e895640 1 -- 172.21.15.32:0/1309963593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac280d14d0 msgr2=0x7fac28144dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.430+0000 7fac2e895640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac280d14d0 0x7fac28144dd0 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7fac1801b8c0 tx=0x7fac1801ba50 comp rx=0 tx=0).stop 2023-12-20T06:00:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.430+0000 7fac2e895640 1 -- 172.21.15.32:0/1309963593 shutdown_connections 2023-12-20T06:00:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.434+0000 7fac2e895640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fac14065bd0 0x7fac14068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.434+0000 7fac2e895640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fac2815ce60 0x7fac281e3d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.434+0000 7fac2e895640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fac280d3130 0x7fac281e3820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.434+0000 7fac2e895640 1 --2- 172.21.15.32:0/1309963593 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fac280d14d0 0x7fac28144dd0 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.434+0000 7fac2e895640 1 -- 172.21.15.32:0/1309963593 >> 172.21.15.32:0/1309963593 conn(0x7fac2806b5e0 msgr2=0x7fac2815bff0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.434+0000 7fac2e895640 1 -- 172.21.15.32:0/1309963593 shutdown_connections 2023-12-20T06:00:46.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.434+0000 7fac2e895640 1 -- 172.21.15.32:0/1309963593 wait complete. 2023-12-20T06:00:46.453 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:46.453 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-20T06:00:46.469 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:46.466+0000 7fb1c164d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:00:46.470 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-20T06:00:46.503 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:46.502+0000 7fb57fa62180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:00:46.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff67b48c640 1 Processor -- start 2023-12-20T06:00:46.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff67b48c640 1 -- start start 2023-12-20T06:00:46.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff67b48c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff67415abb0 0x7ff6740caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff67b48c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff6740b0cb0 0x7ff67415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff67b48c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 0x7ff6740706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff67b48c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff67406e160 con 0x7ff67415abb0 2023-12-20T06:00:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff67b48c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff674070c00 con 0x7ff6740b0cb0 2023-12-20T06:00:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff67b48c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff67415b6a0 con 0x7ff674070230 2023-12-20T06:00:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff679228640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff67415abb0 0x7ff6740caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff679228640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff67415abb0 0x7ff6740caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47480/0 (socket says 172.21.15.32:47480) 2023-12-20T06:00:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff679228640 1 -- 172.21.15.32:0/745550929 learned_addr learned my addr 172.21.15.32:0/745550929 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff679a29640 1 --2- 172.21.15.32:0/745550929 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 0x7ff6740706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:46.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff678a27640 1 --2- 172.21.15.32:0/745550929 >> v2:172.21.15.39:3300/0 conn(0x7ff6740b0cb0 0x7ff67415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff679228640 1 -- 172.21.15.32:0/745550929 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 msgr2=0x7ff6740706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff679228640 1 --2- 172.21.15.32:0/745550929 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 0x7ff6740706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff679228640 1 -- 172.21.15.32:0/745550929 >> v2:172.21.15.39:3300/0 conn(0x7ff6740b0cb0 msgr2=0x7ff67415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff679228640 1 --2- 172.21.15.32:0/745550929 >> v2:172.21.15.39:3300/0 conn(0x7ff6740b0cb0 0x7ff67415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.650+0000 7ff679228640 1 -- 172.21.15.32:0/745550929 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff67415c070 con 0x7ff67415abb0 2023-12-20T06:00:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.654+0000 7ff679228640 1 --2- 172.21.15.32:0/745550929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff67415abb0 0x7ff6740caaf0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff6740b4470 tx=0x7ff67001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88cf69472ed61b66 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:46.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.654+0000 7ff66bfff640 1 -- 172.21.15.32:0/745550929 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700aa620 con 0x7ff67415abb0 2023-12-20T06:00:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.654+0000 7ff66bfff640 1 -- 172.21.15.32:0/745550929 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6700aa780 con 0x7ff67415abb0 2023-12-20T06:00:46.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.654+0000 7ff66bfff640 1 -- 172.21.15.32:0/745550929 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700aab10 con 0x7ff67415abb0 2023-12-20T06:00:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.654+0000 7ff67b48c640 1 -- 172.21.15.32:0/745550929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff67415abb0 msgr2=0x7ff6740caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.654+0000 7ff67b48c640 1 --2- 172.21.15.32:0/745550929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff67415abb0 0x7ff6740caaf0 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff6740b4470 tx=0x7ff67001c500 comp rx=0 tx=0).stop 2023-12-20T06:00:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.654+0000 7ff67b48c640 1 -- 172.21.15.32:0/745550929 shutdown_connections 2023-12-20T06:00:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.654+0000 7ff67b48c640 1 --2- 172.21.15.32:0/745550929 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 0x7ff6740706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.654+0000 7ff67b48c640 1 --2- 172.21.15.32:0/745550929 >> v2:172.21.15.39:3300/0 conn(0x7ff6740b0cb0 0x7ff67415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.654+0000 7ff67b48c640 1 --2- 172.21.15.32:0/745550929 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff67415abb0 0x7ff6740caaf0 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.654+0000 7ff67b48c640 1 -- 172.21.15.32:0/745550929 >> 172.21.15.32:0/745550929 conn(0x7ff67406a510 msgr2=0x7ff6740ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff67b48c640 1 -- 172.21.15.32:0/745550929 shutdown_connections 2023-12-20T06:00:46.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff67b48c640 1 -- 172.21.15.32:0/745550929 wait complete. 2023-12-20T06:00:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff67b48c640 1 Processor -- start 2023-12-20T06:00:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff67b48c640 1 -- start start 2023-12-20T06:00:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff67b48c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 0x7ff6741524c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff67b48c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6740b0cb0 0x7ff6740a8560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff67b48c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff67415abb0 0x7ff6740cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff67b48c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff674152a00 con 0x7ff6740b0cb0 2023-12-20T06:00:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff67b48c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740a74c0 con 0x7ff67415abb0 2023-12-20T06:00:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff67b48c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6740a77a0 con 0x7ff674070230 2023-12-20T06:00:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff678a27640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6740b0cb0 0x7ff6740a8560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff679228640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 0x7ff6741524c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff679228640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 0x7ff6741524c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:60480/0 (socket says 172.21.15.32:60480) 2023-12-20T06:00:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff679a29640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff67415abb0 0x7ff6740cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff679228640 1 -- 172.21.15.32:0/2355751243 learned_addr learned my addr 172.21.15.32:0/2355751243 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff679228640 1 -- 172.21.15.32:0/2355751243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff67415abb0 msgr2=0x7ff6740cb150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:46.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff679228640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff67415abb0 0x7ff6740cb150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff679228640 1 -- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6740b0cb0 msgr2=0x7ff6740a8560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff679228640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6740b0cb0 0x7ff6740a8560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff679228640 1 -- 172.21.15.32:0/2355751243 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6700094d0 con 0x7ff674070230 2023-12-20T06:00:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.658+0000 7ff679a29640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff67415abb0 0x7ff6740cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:00:46.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff679228640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 0x7ff6741524c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff67001abb0 tx=0x7ff67001ad40 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff669ffb640 1 -- 172.21.15.32:0/2355751243 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700aae00 con 0x7ff674070230 2023-12-20T06:00:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff669ffb640 1 -- 172.21.15.32:0/2355751243 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6700bd3f0 con 0x7ff674070230 2023-12-20T06:00:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff669ffb640 1 -- 172.21.15.32:0/2355751243 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6700bd780 con 0x7ff674070230 2023-12-20T06:00:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff67b48c640 1 -- 172.21.15.32:0/2355751243 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6740a7a80 con 0x7ff674070230 2023-12-20T06:00:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff67b48c640 1 -- 172.21.15.32:0/2355751243 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6740a8040 con 0x7ff674070230 2023-12-20T06:00:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff67b48c640 1 -- 172.21.15.32:0/2355751243 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff67404e000 con 0x7ff674070230 2023-12-20T06:00:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff669ffb640 1 -- 172.21.15.32:0/2355751243 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 114) v1 ==== 85347+0+0 (secure 0 0 0) 0x7ff6700b33f0 con 0x7ff674070230 2023-12-20T06:00:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff669ffb640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:46.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff669ffb640 1 -- 172.21.15.32:0/2355751243 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(192..192 src has 1..192) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff67011d610 con 0x7ff674070230 2023-12-20T06:00:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff678a27640 1 -- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 msgr2=0x7ff658068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:46.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.662+0000 7ff678a27640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:00:46.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.666+0000 7ff669ffb640 1 -- 172.21.15.32:0/2355751243 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff6700eafd0 con 0x7ff674070230 2023-12-20T06:00:46.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.846+0000 7ff67b48c640 1 -- 172.21.15.32:0/2355751243 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff67415b330 con 0x7ff658065bd0 2023-12-20T06:00:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.866+0000 7ff678a27640 1 -- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 msgr2=0x7ff658068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:46.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:46.866+0000 7ff678a27640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T06:00:46.977 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:46.974+0000 7fb1c164d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:00:47.016 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:47.014+0000 7fb57fa62180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:00:47.038 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:47.038+0000 7fb1c164d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:00:47.077 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:47.078+0000 7fb57fa62180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:00:47.096 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:47.094+0000 7fb1c164d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:00:47.135 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:47.134+0000 7fb57fa62180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:00:47.188 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:47.186+0000 7fb1c164d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:00:47.227 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:47.226+0000 7fb57fa62180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:00:47.247 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:47.246+0000 7fb1c164d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:00:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:47.270+0000 7ff678a27640 1 -- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 msgr2=0x7ff658068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:47.270+0000 7ff678a27640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T06:00:47.286 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:47.286+0000 7fb57fa62180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:00:47.335 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:47.334+0000 7fb1c164d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:00:47.374 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:47.374+0000 7fb57fa62180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:00:47.394 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:47.394+0000 7fb1c164d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:00:47.433 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:47.434+0000 7fb57fa62180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:00:47.544 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:47.542+0000 7fb1c164d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:00:47.583 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:47.582+0000 7fb57fa62180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:00:47.605 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:47.602+0000 7fb1c164d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:00:47.645 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:47.646+0000 7fb57fa62180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:00:47.855 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:00:47.855 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:00:47.855 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:00:47.864 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:47.862+0000 7fb1c164d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:00:47.894 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:00:47.940 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:00:47.940 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:00:47.941 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:47.902+0000 7fb57fa62180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:00:48.006 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:48.002+0000 7fb1c164d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:00:48.045 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:48.046+0000 7fb57fa62180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:00:48.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:48.074+0000 7ff678a27640 1 -- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 msgr2=0x7ff658068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:48.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:48.074+0000 7ff678a27640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T06:00:48.136 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:48.134+0000 7fb1c164d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:00:48.176 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:48.174+0000 7fb57fa62180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:00:48.210 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:48.210+0000 7fb1c164d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:00:48.250 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:48.250+0000 7fb57fa62180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:00:48.353 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:48.350+0000 7fb1c164d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:00:48.393 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:48.394+0000 7fb57fa62180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:00:48.922 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:48.918+0000 7fb1c164d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:00:48.959 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:48.958+0000 7fb57fa62180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:00:49.010 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:49.010+0000 7fb1c164d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:00:49.047 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.046+0000 7fb57fa62180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:00:49.166 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:49.166+0000 7fb1c164d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:00:49.203 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.202+0000 7fb57fa62180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:00:49.254 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:49.254+0000 7fb1c164d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:00:49.290 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.290+0000 7fb57fa62180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:00:49.590 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:49.586+0000 7fb1c164d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:00:49.625 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.626+0000 7fb57fa62180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:00:49.653 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:00:49.650+0000 7fb1c164d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:00:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:49.678+0000 7ff678a27640 1 -- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 msgr2=0x7ff658068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:00:49.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:49.678+0000 7ff678a27640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-20T06:00:49.688 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.686+0000 7fb57fa62180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:00:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:49.698+0000 7ff669ffb640 1 -- 172.21.15.32:0/2355751243 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mgrmap(e 115) v1 ==== 84956+0+0 (secure 0 0 0) 0x7ff67011baf0 con 0x7ff674070230 2023-12-20T06:00:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:49.702+0000 7ff669ffb640 1 -- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 msgr2=0x7ff658068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:00:49.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:49.702+0000 7ff669ffb640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:49.819 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.818+0000 7fb5022c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:49.819 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.818+0000 7fb5022c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:49.819 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.818+0000 7fb5022c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:49.819 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.818+0000 7fb5022c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:49.820 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.818+0000 7fb5022c0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:49.820 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.818+0000 7fb5052c6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:49.820 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.818+0000 7fb5052c6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:49.820 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.818+0000 7fb5052c6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:49.820 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.818+0000 7fb5052c6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:49.820 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:00:49.818+0000 7fb5052c6640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:00:50.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.706+0000 7ff669ffb640 1 -- 172.21.15.32:0/2355751243 <== mon.2 v2:172.21.15.39:3301/0 8 ==== mgrmap(e 116) v1 ==== 85083+0+0 (secure 0 0 0) 0x7ff6700eb3a0 con 0x7ff674070230 2023-12-20T06:00:50.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.706+0000 7ff669ffb640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:50.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.706+0000 7ff669ffb640 1 -- 172.21.15.32:0/2355751243 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff658033630 con 0x7ff658065bd0 2023-12-20T06:00:50.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.710+0000 7ff678a27640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:50.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.714+0000 7ff678a27640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff6740c42b0 tx=0x7ff664009190 comp rx=0 tx=0).ready entity=mgr.10206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:50.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff669ffb640 1 -- 172.21.15.32:0/2355751243 <== mgr.10206 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7ff658033630 con 0x7ff658065bd0 2023-12-20T06:00:50.780 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path snap_test_dir1 2023-12-20T06:00:50.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 -- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 msgr2=0x7ff658068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:50.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7ff6740c42b0 tx=0x7ff664009190 comp rx=0 tx=0).stop 2023-12-20T06:00:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 -- 172.21.15.32:0/2355751243 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 msgr2=0x7ff6741524c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 0x7ff6741524c0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7ff67001abb0 tx=0x7ff67001ad40 comp rx=0 tx=0).stop 2023-12-20T06:00:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 -- 172.21.15.32:0/2355751243 shutdown_connections 2023-12-20T06:00:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff658065bd0 0x7ff658068090 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff67415abb0 0x7ff6740cb150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6740b0cb0 0x7ff6740a8560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 --2- 172.21.15.32:0/2355751243 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff674070230 0x7ff6741524c0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 -- 172.21.15.32:0/2355751243 >> 172.21.15.32:0/2355751243 conn(0x7ff67406a510 msgr2=0x7ff6740c4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 -- 172.21.15.32:0/2355751243 shutdown_connections 2023-12-20T06:00:50.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.782+0000 7ff67b48c640 1 -- 172.21.15.32:0/2355751243 wait complete. 2023-12-20T06:00:50.803 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-20T06:00:50.803 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1M 2023-12-20T06:00:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.950+0000 7fd5e6709640 1 Processor -- start 2023-12-20T06:00:50.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.950+0000 7fd5e6709640 1 -- start start 2023-12-20T06:00:50.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5e6709640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 0x7fd5e00b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:50.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5e6709640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5e006d2d0 0x7fd5e006d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:50.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5e6709640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd5e006dd50 con 0x7fd5e00b07f0 2023-12-20T06:00:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5e6709640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd5e00cc940 con 0x7fd5e006d2d0 2023-12-20T06:00:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5e6709640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e00ccc20 con 0x7fd5e0159af0 2023-12-20T06:00:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5df7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5e006d2d0 0x7fd5e006d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5df7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd5e006d2d0 0x7fd5e006d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46782/0 (socket says 172.21.15.32:46782) 2023-12-20T06:00:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5df7fe640 1 -- 172.21.15.32:0/1845565975 learned_addr learned my addr 172.21.15.32:0/1845565975 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5dffff640 1 --2- 172.21.15.32:0/1845565975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 0x7fd5e00b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5df7fe640 1 -- 172.21.15.32:0/1845565975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 msgr2=0x7fd5e00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:00:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5df7fe640 1 --2- 172.21.15.32:0/1845565975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 0x7fd5e00b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5df7fe640 1 -- 172.21.15.32:0/1845565975 >> v1:172.21.15.39:6789/0 conn(0x7fd5e00b07f0 legacy=0x7fd5e00cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5df7fe640 1 -- 172.21.15.32:0/1845565975 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5e00ccf00 con 0x7fd5e006d2d0 2023-12-20T06:00:50.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.954+0000 7fd5dffff640 1 --2- 172.21.15.32:0/1845565975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 0x7fd5e00b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:50.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5df7fe640 1 --2- 172.21.15.32:0/1845565975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e006d2d0 0x7fd5e006d710 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400bb10 tx=0x7fd5d401ebe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40f12d24afd20343 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:50.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5deffd640 1 -- 172.21.15.32:0/1845565975 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40a7cd0 con 0x7fd5e006d2d0 2023-12-20T06:00:50.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5deffd640 1 -- 172.21.15.32:0/1845565975 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d40a7e30 con 0x7fd5e006d2d0 2023-12-20T06:00:50.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5deffd640 1 -- 172.21.15.32:0/1845565975 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40af660 con 0x7fd5e006d2d0 2023-12-20T06:00:50.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5e6709640 1 -- 172.21.15.32:0/1845565975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e006d2d0 msgr2=0x7fd5e006d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5e6709640 1 --2- 172.21.15.32:0/1845565975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e006d2d0 0x7fd5e006d710 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400bb10 tx=0x7fd5d401ebe0 comp rx=0 tx=0).stop 2023-12-20T06:00:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5e6709640 1 -- 172.21.15.32:0/1845565975 shutdown_connections 2023-12-20T06:00:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5e6709640 1 --2- 172.21.15.32:0/1845565975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e006d2d0 0x7fd5e006d710 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5e6709640 1 --2- 172.21.15.32:0/1845565975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 0x7fd5e00b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5e6709640 1 -- 172.21.15.32:0/1845565975 >> 172.21.15.32:0/1845565975 conn(0x7fd5e006a510 msgr2=0x7fd5e00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5e6709640 1 -- 172.21.15.32:0/1845565975 shutdown_connections 2023-12-20T06:00:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5e6709640 1 -- 172.21.15.32:0/1845565975 wait complete. 2023-12-20T06:00:50.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5e6709640 1 Processor -- start 2023-12-20T06:00:50.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.958+0000 7fd5e6709640 1 -- start start 2023-12-20T06:00:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5e6709640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e006d2d0 0x7fd5e00c4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5e6709640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e00b07f0 0x7fd5e01402a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5e6709640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 0x7fd5e00b2f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5e6709640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0144550 con 0x7fd5e00b07f0 2023-12-20T06:00:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5e6709640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e014d740 con 0x7fd5e006d2d0 2023-12-20T06:00:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5e6709640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd5e0143c20 con 0x7fd5e0159af0 2023-12-20T06:00:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5df7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e00b07f0 0x7fd5e01402a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5df7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e00b07f0 0x7fd5e01402a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46794/0 (socket says 172.21.15.32:46794) 2023-12-20T06:00:50.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5df7fe640 1 -- 172.21.15.32:0/1926533549 learned_addr learned my addr 172.21.15.32:0/1926533549 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5dffff640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e006d2d0 0x7fd5e00c4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5e4ca6640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 0x7fd5e00b2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5df7fe640 1 -- 172.21.15.32:0/1926533549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 msgr2=0x7fd5e00b2f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5df7fe640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 0x7fd5e00b2f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5df7fe640 1 -- 172.21.15.32:0/1926533549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e006d2d0 msgr2=0x7fd5e00c4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5df7fe640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e006d2d0 0x7fd5e00c4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5df7fe640 1 -- 172.21.15.32:0/1926533549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd5d400b770 con 0x7fd5e00b07f0 2023-12-20T06:00:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5e4ca6640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 0x7fd5e00b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:50.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5df7fe640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e00b07f0 0x7fd5e01402a0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400b2c0 tx=0x7fd5d401b980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5dcff9640 1 -- 172.21.15.32:0/1926533549 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d4006ee0 con 0x7fd5e00b07f0 2023-12-20T06:00:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5dcff9640 1 -- 172.21.15.32:0/1926533549 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd5d40b8400 con 0x7fd5e00b07f0 2023-12-20T06:00:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5e6709640 1 -- 172.21.15.32:0/1926533549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd5e00b1190 con 0x7fd5e00b07f0 2023-12-20T06:00:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5dcff9640 1 -- 172.21.15.32:0/1926533549 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd5d40afea0 con 0x7fd5e00b07f0 2023-12-20T06:00:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5e6709640 1 -- 172.21.15.32:0/1926533549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd5e01441e0 con 0x7fd5e00b07f0 2023-12-20T06:00:50.963 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5e6709640 1 -- 172.21.15.32:0/1926533549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd5e01446c0 con 0x7fd5e00b07f0 2023-12-20T06:00:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.962+0000 7fd5dcff9640 1 -- 172.21.15.32:0/1926533549 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 116) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fd5d40b6040 con 0x7fd5e00b07f0 2023-12-20T06:00:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.966+0000 7fd5dcff9640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5cc065900 0x7fd5cc067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.966+0000 7fd5dcff9640 1 -- 172.21.15.32:0/1926533549 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fd5d411ef30 con 0x7fd5e00b07f0 2023-12-20T06:00:50.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.966+0000 7fd5dffff640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5cc065900 0x7fd5cc067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:50.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.966+0000 7fd5dffff640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5cc065900 0x7fd5cc067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0006e90 tx=0x7fd5d0008040 comp rx=0 tx=0).ready entity=mgr.10206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:50.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:50.970+0000 7fd5dcff9640 1 -- 172.21.15.32:0/1926533549 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd5d40ec9d0 con 0x7fd5e00b07f0 2023-12-20T06:00:51.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.146+0000 7fd5e6709640 1 -- 172.21.15.32:0/1926533549 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd5e00b1010 con 0x7fd5cc065900 2023-12-20T06:00:51.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.206+0000 7fd5dcff9640 1 -- 172.21.15.32:0/1926533549 <== mgr.10206 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7fd5d00108d0 con 0x7fd5cc065900 2023-12-20T06:00:51.206 INFO:teuthology.orchestra.run.smithi032.stdout:Retention added to path snap_test_dir1 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 -- 172.21.15.32:0/1926533549 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5cc065900 msgr2=0x7fd5cc067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5cc065900 0x7fd5cc067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd5d0006e90 tx=0x7fd5d0008040 comp rx=0 tx=0).stop 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 -- 172.21.15.32:0/1926533549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e00b07f0 msgr2=0x7fd5e01402a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e00b07f0 0x7fd5e01402a0 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fd5d400b2c0 tx=0x7fd5d401b980 comp rx=0 tx=0).stop 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 -- 172.21.15.32:0/1926533549 shutdown_connections 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd5cc065900 0x7fd5cc067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd5e0159af0 0x7fd5e00b2f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd5e00b07f0 0x7fd5e01402a0 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 --2- 172.21.15.32:0/1926533549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd5e006d2d0 0x7fd5e00c4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 -- 172.21.15.32:0/1926533549 >> 172.21.15.32:0/1926533549 conn(0x7fd5e006a510 msgr2=0x7fd5e0158200 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 -- 172.21.15.32:0/1926533549 shutdown_connections 2023-12-20T06:00:51.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.210+0000 7fd5e6709640 1 -- 172.21.15.32:0/1926533549 wait complete. 2023-12-20T06:00:51.228 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-12-20T06:00:51.228 DEBUG:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-20-06_01_00 in ~9s... 2023-12-20T06:00:51.229 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-12-20T06:00:51.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-20T06:00:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.374+0000 7f988968c640 1 Processor -- start 2023-12-20T06:00:51.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.374+0000 7f988968c640 1 -- start start 2023-12-20T06:00:51.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.378+0000 7f988968c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98840cb050 0x7f98840cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.378+0000 7f988968c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988406a1f0 con 0x7f988406d170 2023-12-20T06:00:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.378+0000 7f988968c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f988406a440 con 0x7f9884065d40 2023-12-20T06:00:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.378+0000 7f988968c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f988406a720 con 0x7f98840cb050 2023-12-20T06:00:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.378+0000 7f9882ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98840cb050 0x7f98840cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.378+0000 7f9882ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f98840cb050 0x7f98840cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46806/0 (socket says 172.21.15.32:46806) 2023-12-20T06:00:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.378+0000 7f9882ffd640 1 -- 172.21.15.32:0/934154501 learned_addr learned my addr 172.21.15.32:0/934154501 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.378+0000 7f9882ffd640 1 -- 172.21.15.32:0/934154501 >> v1:172.21.15.39:6789/0 conn(0x7f9884065d40 legacy=0x7f988406ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.378+0000 7f9882ffd640 1 -- 172.21.15.32:0/934154501 >> v1:172.21.15.32:6789/0 conn(0x7f988406d170 legacy=0x7f98840c9720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:51.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.378+0000 7f9882ffd640 1 -- 172.21.15.32:0/934154501 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f988406aa00 con 0x7f98840cb050 2023-12-20T06:00:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.378+0000 7f9882ffd640 1 --2- 172.21.15.32:0/934154501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98840cb050 0x7f98840cb430 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f9874009870 tx=0x7f987401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=442678f1e303d6cf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f98827fc640 1 -- 172.21.15.32:0/934154501 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98740aa670 con 0x7f98840cb050 2023-12-20T06:00:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f98827fc640 1 -- 172.21.15.32:0/934154501 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98740aa7d0 con 0x7f98840cb050 2023-12-20T06:00:51.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f98827fc640 1 -- 172.21.15.32:0/934154501 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98740aab60 con 0x7f98840cb050 2023-12-20T06:00:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 -- 172.21.15.32:0/934154501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98840cb050 msgr2=0x7f98840cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 --2- 172.21.15.32:0/934154501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98840cb050 0x7f98840cb430 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f9874009870 tx=0x7f987401c510 comp rx=0 tx=0).stop 2023-12-20T06:00:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 -- 172.21.15.32:0/934154501 shutdown_connections 2023-12-20T06:00:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 --2- 172.21.15.32:0/934154501 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98840cb050 0x7f98840cb430 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 -- 172.21.15.32:0/934154501 >> 172.21.15.32:0/934154501 conn(0x7f98840cf8d0 msgr2=0x7f98840cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 -- 172.21.15.32:0/934154501 shutdown_connections 2023-12-20T06:00:51.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 -- 172.21.15.32:0/934154501 wait complete. 2023-12-20T06:00:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 Processor -- start 2023-12-20T06:00:51.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 -- start start 2023-12-20T06:00:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9884065d40 0x7f98840c7420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988406d170 0x7f98840c5970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98840cb050 0x7f988414d070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f988414bdf0 con 0x7f988406d170 2023-12-20T06:00:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98840c5eb0 con 0x7f9884065d40 2023-12-20T06:00:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f988968c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98840c7a00 con 0x7f98840cb050 2023-12-20T06:00:51.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f987bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988406d170 0x7f98840c5970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f9882ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9884065d40 0x7f98840c7420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f9882ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9884065d40 0x7f98840c7420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:54208/0 (socket says 172.21.15.32:54208) 2023-12-20T06:00:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f9882ffd640 1 -- 172.21.15.32:0/803085145 learned_addr learned my addr 172.21.15.32:0/803085145 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:00:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.382+0000 7f987bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988406d170 0x7f98840c5970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46820/0 (socket says 172.21.15.32:46820) 2023-12-20T06:00:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f98837fe640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98840cb050 0x7f988414d070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f987bfff640 1 -- 172.21.15.32:0/803085145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98840cb050 msgr2=0x7f988414d070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f987bfff640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98840cb050 0x7f988414d070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f987bfff640 1 -- 172.21.15.32:0/803085145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9884065d40 msgr2=0x7f98840c7420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f987bfff640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9884065d40 0x7f98840c7420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f987bfff640 1 -- 172.21.15.32:0/803085145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98740094d0 con 0x7f988406d170 2023-12-20T06:00:51.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f98837fe640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98840cb050 0x7f988414d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:00:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f987bfff640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988406d170 0x7f98840c5970 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c390 tx=0x7f986c00c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f987b7fe640 1 -- 172.21.15.32:0/803085145 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c005220 con 0x7f988406d170 2023-12-20T06:00:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f987b7fe640 1 -- 172.21.15.32:0/803085145 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f986c005380 con 0x7f988406d170 2023-12-20T06:00:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f987b7fe640 1 -- 172.21.15.32:0/803085145 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f986c007250 con 0x7f988406d170 2023-12-20T06:00:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f988968c640 1 -- 172.21.15.32:0/803085145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98840c7ce0 con 0x7f988406d170 2023-12-20T06:00:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f988968c640 1 -- 172.21.15.32:0/803085145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98840c82a0 con 0x7f988406d170 2023-12-20T06:00:51.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f988968c640 1 -- 172.21.15.32:0/803085145 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f988404e000 con 0x7f988406d170 2023-12-20T06:00:51.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f987b7fe640 1 -- 172.21.15.32:0/803085145 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 116) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f986c0073b0 con 0x7f988406d170 2023-12-20T06:00:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.386+0000 7f987b7fe640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9864065900 0x7f9864067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:00:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.390+0000 7f987b7fe640 1 -- 172.21.15.32:0/803085145 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f986c0823b0 con 0x7f988406d170 2023-12-20T06:00:51.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.390+0000 7f9882ffd640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9864065900 0x7f9864067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:00:51.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.390+0000 7f9882ffd640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9864065900 0x7f9864067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f987401c550 tx=0x7f98740b5040 comp rx=0 tx=0).ready entity=mgr.10206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:00:51.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.394+0000 7f987b7fe640 1 -- 172.21.15.32:0/803085145 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f986c04ff20 con 0x7f988406d170 2023-12-20T06:00:51.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.570+0000 7f988968c640 1 -- 172.21.15.32:0/803085145 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f98841617c0 con 0x7f9864065900 2023-12-20T06:00:51.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f987b7fe640 1 -- 172.21.15.32:0/803085145 <== mgr.10206 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f98841617c0 con 0x7f9864065900 2023-12-20T06:00:51.577 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:00:51.577 INFO:teuthology.orchestra.run.smithi032.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 -- 172.21.15.32:0/803085145 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9864065900 msgr2=0x7f9864067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9864065900 0x7f9864067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f987401c550 tx=0x7f98740b5040 comp rx=0 tx=0).stop 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 -- 172.21.15.32:0/803085145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988406d170 msgr2=0x7f98840c5970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988406d170 0x7f98840c5970 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f986c00c390 tx=0x7f986c00c850 comp rx=0 tx=0).stop 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 -- 172.21.15.32:0/803085145 shutdown_connections 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9864065900 0x7f9864067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98840cb050 0x7f988414d070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f988406d170 0x7f98840c5970 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 --2- 172.21.15.32:0/803085145 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9884065d40 0x7f98840c7420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 -- 172.21.15.32:0/803085145 >> 172.21.15.32:0/803085145 conn(0x7f98840cf8d0 msgr2=0x7f988415d390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 -- 172.21.15.32:0/803085145 shutdown_connections 2023-12-20T06:00:51.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:00:51.578+0000 7f988968c640 1 -- 172.21.15.32:0/803085145 wait complete. 2023-12-20T06:00:51.598 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-20T06:00:51.598 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-12-20T06:00:51.598 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:51.598 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:00:51.610 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T06:00:51.610 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:00:51.610 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:00:52.610 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:52.610 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:00:52.623 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T06:00:52.623 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:00:52.623 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:00:53.623 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:53.624 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:00:53.636 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T06:00:53.636 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:00:53.636 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:00:54.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:54.637 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:00:54.649 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T06:00:54.649 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:00:54.649 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:00:55.651 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:55.651 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:00:55.663 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T06:00:55.663 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:00:55.663 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:00:56.665 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:56.665 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:00:56.681 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T06:00:56.681 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:00:56.681 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:00:57.682 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:57.682 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:00:57.695 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T06:00:57.696 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:00:57.696 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:00:58.697 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:58.697 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:00:58.710 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T06:00:58.710 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:00:58.710 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:00:59.711 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:00:59.712 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:00:59.725 INFO:tasks.cephfs.test_snap_schedules:snapshots: set() 2023-12-20T06:00:59.725 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:00:59.725 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:00.219 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:01:00.218+0000 7fb4f4aa5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:01:00.219 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:01:00.218+0000 7fb4f4aa5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:01:00.219 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:01:00.218+0000 7fb4f4aa5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:01:00.219 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:01:00.218+0000 7fb4f4aa5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:01:00.219 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:01:00.218+0000 7fb4f4aa5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:01:00.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:00.726 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:00.738 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:00.738 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:00.739 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:00.739 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:00.739 DEBUG:tasks.cephfs.test_snap_schedules:snapshots added=['scheduled-2023-12-20-06_01_00_UTC'] 2023-12-20T06:01:00.739 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703052051.2288465, now = 1703052060.7392666, timo = 9 2023-12-20T06:01:01.739 DEBUG:tasks.cephfs.test_snap_schedules:expecting removal of snap snap_test_dir1/.snap/scheduled-2023-12-20-06_01_00 in ~69s... 2023-12-20T06:01:01.740 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:01.740 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:01.754 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:01.754 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:01.754 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:01.754 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:02.754 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:02.755 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:02.766 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:02.767 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:02.767 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:02.767 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:03.768 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:03.768 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:03.779 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:03.780 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:03.780 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:03.780 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:04.781 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:04.781 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:04.792 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:04.793 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:04.794 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:04.794 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:05.795 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:05.795 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:05.806 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:05.807 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:05.807 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:05.808 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:06.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:06.809 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:06.821 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:06.822 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:06.822 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:06.822 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:07.823 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:07.823 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:07.834 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:07.836 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:07.836 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:07.836 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:08.836 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:08.837 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:08.848 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:08.849 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:08.850 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:08.850 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:09.851 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:09.851 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:09.863 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:09.864 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:09.864 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:09.864 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:10.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:10.865 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:10.877 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:10.878 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:10.878 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:10.878 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:11.879 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:11.879 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:11.891 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:11.892 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:11.892 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:11.892 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:12.407 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:01:12.411 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:01:12.893 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:12.893 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:12.905 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:12.906 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:12.906 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:12.906 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:13.907 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:13.908 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:13.919 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:13.920 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:13.920 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:13.921 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:14.922 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:14.922 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:14.933 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:14.934 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:14.934 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:14.934 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:15.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:15.936 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:15.947 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:15.948 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:15.948 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:15.949 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:16.949 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:16.950 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:16.961 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:16.962 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:16.962 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:16.962 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:17.962 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:17.962 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:17.974 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:17.975 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:17.975 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:17.975 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:18.975 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:18.976 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:18.987 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:18.988 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:18.988 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:18.988 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:19.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:19.989 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:20.001 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:20.002 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:20.002 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:20.002 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:21.003 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:21.004 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:21.015 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:21.016 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:21.016 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:21.016 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:22.017 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:22.018 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:22.029 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:22.031 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:22.031 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:22.031 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:23.032 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:23.032 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:23.044 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:23.045 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:23.045 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:23.045 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:24.046 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:24.047 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:24.059 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:24.060 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:24.060 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:24.060 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:25.061 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:25.061 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:25.073 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:25.074 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:25.074 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:25.074 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:26.075 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:26.075 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:26.087 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:26.088 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:26.088 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:26.089 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:27.090 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:27.090 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:27.102 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:27.103 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:27.103 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:27.103 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:28.104 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:28.105 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:28.116 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:28.117 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:28.118 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:28.118 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:29.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:29.119 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:29.130 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:29.131 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:29.131 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:29.131 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:30.132 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:30.133 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:30.143 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:30.144 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:30.144 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:30.144 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:31.145 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:31.145 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:31.157 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:31.158 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:31.158 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:31.158 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:32.159 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:32.160 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:32.171 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:32.173 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:32.173 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:32.173 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:33.174 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:33.174 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:33.186 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:33.187 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:33.187 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:33.187 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:34.188 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:34.189 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:34.200 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:34.202 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:34.202 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:34.202 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:35.202 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:35.203 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:35.214 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:35.216 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:35.216 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:35.216 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:36.217 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:36.218 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:36.229 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:36.230 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:36.230 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:36.230 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:37.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:37.232 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:37.244 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:37.245 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:37.245 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:37.245 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:38.246 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:38.246 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:38.258 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:38.259 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:38.259 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:38.259 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:39.260 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:39.260 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:39.272 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:39.273 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:39.273 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:39.273 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:40.275 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:40.275 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:40.287 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:40.288 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:40.288 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:40.288 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:41.288 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:41.289 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:41.300 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:41.301 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:41.302 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:41.302 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:42.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:42.302 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:42.314 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:42.315 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:42.315 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:42.315 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:42.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:01:42.497 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:01:43.316 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:43.317 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:43.329 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:43.330 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:43.330 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:43.330 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:44.330 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:44.331 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:44.342 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:44.343 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:44.344 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:44.344 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:45.345 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:45.345 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:45.357 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:45.358 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:45.359 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:45.359 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:46.360 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:46.360 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:46.372 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:46.373 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:46.373 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:46.374 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:47.375 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:47.375 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:47.386 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:47.388 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:47.388 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:47.388 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:48.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:48.389 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:48.400 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:48.402 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:48.402 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:48.402 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:49.403 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:49.403 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:49.415 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:49.416 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:49.416 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:49.416 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:50.417 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:50.417 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:50.429 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:50.430 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:50.430 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:50.430 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:51.431 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:51.431 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:51.442 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:51.444 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:51.444 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:51.444 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:52.445 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:52.445 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:52.457 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:52.458 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:52.459 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:52.459 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:53.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:53.460 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:53.471 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:53.472 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:53.472 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:53.472 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:54.473 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:54.474 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:54.486 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:54.487 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:54.487 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:54.487 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:55.487 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:55.488 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:55.499 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:55.501 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:55.501 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:55.501 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:56.501 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:56.502 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:56.513 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:56.514 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:56.515 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:56.515 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:57.515 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:57.515 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:57.527 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:57.528 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:57.528 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:57.528 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:58.529 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:58.530 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:58.541 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:58.542 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:58.543 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:58.543 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:01:59.544 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:01:59.544 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:01:59.556 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_01_00_UTC 2023-12-20T06:01:59.557 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:01:59.557 INFO:tasks.cephfs.test_snap_schedules:added: set() 2023-12-20T06:01:59.557 INFO:tasks.cephfs.test_snap_schedules:removed: set() 2023-12-20T06:02:00.558 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:00.559 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:02:00.570 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_02_00_UTC 2023-12-20T06:02:00.571 INFO:tasks.cephfs.test_snap_schedules:snapshots: {'scheduled-2023-12-20-06_02_00_UTC'} 2023-12-20T06:02:00.571 INFO:tasks.cephfs.test_snap_schedules:added: {'scheduled-2023-12-20-06_02_00_UTC'} 2023-12-20T06:02:00.571 INFO:tasks.cephfs.test_snap_schedules:removed: {'scheduled-2023-12-20-06_01_00_UTC'} 2023-12-20T06:02:00.571 DEBUG:tasks.cephfs.test_snap_schedules:snapshots removed=['scheduled-2023-12-20-06_01_00_UTC'] 2023-12-20T06:02:00.571 DEBUG:tasks.cephfs.test_snap_schedules:exec=1703052051.2288465, now = 1703052120.5717342, timo = 69 2023-12-20T06:02:01.572 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-20T06:02:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.729+0000 7fbcacb52640 1 Processor -- start 2023-12-20T06:02:01.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.729+0000 7fbcacb52640 1 -- start start 2023-12-20T06:02:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.729+0000 7fbcacb52640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbca806bd50 0x7fbca80cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.729+0000 7fbcacb52640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca80b0550 con 0x7fbca80cf5d0 2023-12-20T06:02:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.729+0000 7fbcacb52640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbca80aff20 con 0x7fbca815ce50 2023-12-20T06:02:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.729+0000 7fbcacb52640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbca80cfdc0 con 0x7fbca806bd50 2023-12-20T06:02:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbc9ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbca806bd50 0x7fbca80cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbc9ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbca806bd50 0x7fbca80cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50224/0 (socket says 172.21.15.32:50224) 2023-12-20T06:02:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbc9ffff640 1 -- 172.21.15.32:0/3790430355 learned_addr learned my addr 172.21.15.32:0/3790430355 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:01.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbca6d9d640 1 --1- 172.21.15.32:0/3790430355 >> v1:172.21.15.32:6789/0 conn(0x7fbca80cf5d0 0x7fbca80b09e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39014/0 (socket says 172.21.15.32:39014) 2023-12-20T06:02:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbc9ffff640 1 -- 172.21.15.32:0/3790430355 >> v1:172.21.15.39:6789/0 conn(0x7fbca815ce50 legacy=0x7fbca806b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbc9ffff640 1 -- 172.21.15.32:0/3790430355 >> v1:172.21.15.32:6789/0 conn(0x7fbca80cf5d0 legacy=0x7fbca80b09e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbc9ffff640 1 -- 172.21.15.32:0/3790430355 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbca80d00a0 con 0x7fbca806bd50 2023-12-20T06:02:01.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbca5d9b640 1 -- 172.21.15.32:0/3790430355 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 688790064 0 0) 0x7fbca80b0550 con 0x7fbca80cf5d0 2023-12-20T06:02:01.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbc9ffff640 1 --2- 172.21.15.32:0/3790430355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbca806bd50 0x7fbca80cf090 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009870 tx=0x7fbc9001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37b50b193f03fe50 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:01.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbca5d9b640 1 -- 172.21.15.32:0/3790430355 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900a95f0 con 0x7fbca806bd50 2023-12-20T06:02:01.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbca5d9b640 1 -- 172.21.15.32:0/3790430355 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbc900a9750 con 0x7fbca806bd50 2023-12-20T06:02:01.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.733+0000 7fbca5d9b640 1 -- 172.21.15.32:0/3790430355 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbc900a9ae0 con 0x7fbca806bd50 2023-12-20T06:02:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 -- 172.21.15.32:0/3790430355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbca806bd50 msgr2=0x7fbca80cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 --2- 172.21.15.32:0/3790430355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbca806bd50 0x7fbca80cf090 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7fbc90009870 tx=0x7fbc9001c510 comp rx=0 tx=0).stop 2023-12-20T06:02:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 -- 172.21.15.32:0/3790430355 shutdown_connections 2023-12-20T06:02:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 --2- 172.21.15.32:0/3790430355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbca806bd50 0x7fbca80cf090 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:01.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 -- 172.21.15.32:0/3790430355 >> 172.21.15.32:0/3790430355 conn(0x7fbca8099d70 msgr2=0x7fbca80d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 -- 172.21.15.32:0/3790430355 shutdown_connections 2023-12-20T06:02:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 -- 172.21.15.32:0/3790430355 wait complete. 2023-12-20T06:02:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 Processor -- start 2023-12-20T06:02:01.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 -- start start 2023-12-20T06:02:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbca806bd50 0x7fbca80cbcd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbca80cf5d0 0x7fbca80ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbca815ce50 0x7fbca815bca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80cd6c0 con 0x7fbca815ce50 2023-12-20T06:02:01.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbca814e240 con 0x7fbca80cf5d0 2023-12-20T06:02:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbcacb52640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbca80cc210 con 0x7fbca806bd50 2023-12-20T06:02:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbca6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbca815ce50 0x7fbca815bca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbca6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbca815ce50 0x7fbca815bca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50238/0 (socket says 172.21.15.32:50238) 2023-12-20T06:02:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbca6d9d640 1 -- 172.21.15.32:0/1294431421 learned_addr learned my addr 172.21.15.32:0/1294431421 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbc9ffff640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbca80cf5d0 0x7fbca80ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.737+0000 7fbca659c640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbca806bd50 0x7fbca80cbcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbca6d9d640 1 -- 172.21.15.32:0/1294431421 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbca806bd50 msgr2=0x7fbca80cbcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbca6d9d640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbca806bd50 0x7fbca80cbcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbca6d9d640 1 -- 172.21.15.32:0/1294431421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbca80cf5d0 msgr2=0x7fbca80ae820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbca6d9d640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbca80cf5d0 0x7fbca80ae820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:01.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbca6d9d640 1 -- 172.21.15.32:0/1294431421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbc900094d0 con 0x7fbca815ce50 2023-12-20T06:02:01.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbc9ffff640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbca80cf5d0 0x7fbca80ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:01.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbca6d9d640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbca815ce50 0x7fbca815bca0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fbca0001570 tx=0x7fbca000cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:01.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbc9effd640 1 -- 172.21.15.32:0/1294431421 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000a640 con 0x7fbca815ce50 2023-12-20T06:02:01.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbc9effd640 1 -- 172.21.15.32:0/1294431421 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbca000a7c0 con 0x7fbca815ce50 2023-12-20T06:02:01.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbc9effd640 1 -- 172.21.15.32:0/1294431421 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbca000ab70 con 0x7fbca815ce50 2023-12-20T06:02:01.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbcacb52640 1 -- 172.21.15.32:0/1294431421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbca814dc80 con 0x7fbca815ce50 2023-12-20T06:02:01.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbcacb52640 1 -- 172.21.15.32:0/1294431421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbca80cd8f0 con 0x7fbca815ce50 2023-12-20T06:02:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbcacb52640 1 -- 172.21.15.32:0/1294431421 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbca804e000 con 0x7fbca815ce50 2023-12-20T06:02:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbc9effd640 1 -- 172.21.15.32:0/1294431421 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 119) v1 ==== 85461+0+0 (secure 0 0 0) 0x7fbca000acd0 con 0x7fbca815ce50 2023-12-20T06:02:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbc9effd640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbc88065cf0 0x7fbc880681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbc9effd640 1 -- 172.21.15.32:0/1294431421 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6109+0+0 (secure 0 0 0) 0x7fbca0082e40 con 0x7fbca815ce50 2023-12-20T06:02:01.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.741+0000 7fbca659c640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbc88065cf0 0x7fbc880681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:01.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.745+0000 7fbca659c640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbc88065cf0 0x7fbc880681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc98000f70 tx=0x7fbc98009040 comp rx=0 tx=0).ready entity=mgr.10206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:01.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.745+0000 7fbc9effd640 1 -- 172.21.15.32:0/1294431421 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbca0050270 con 0x7fbca815ce50 2023-12-20T06:02:01.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.925+0000 7fbcacb52640 1 -- 172.21.15.32:0/1294431421 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7fbca80cfa40 con 0x7fbc88065cf0 2023-12-20T06:02:01.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbc9effd640 1 -- 172.21.15.32:0/1294431421 <== mgr.10206 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fbc980050d0 con 0x7fbc88065cf0 2023-12-20T06:02:01.997 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path snap_test_dir1 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 -- 172.21.15.32:0/1294431421 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbc88065cf0 msgr2=0x7fbc880681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbc88065cf0 0x7fbc880681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fbc98000f70 tx=0x7fbc98009040 comp rx=0 tx=0).stop 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 -- 172.21.15.32:0/1294431421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbca815ce50 msgr2=0x7fbca815bca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbca815ce50 0x7fbca815bca0 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7fbca0001570 tx=0x7fbca000cca0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 -- 172.21.15.32:0/1294431421 shutdown_connections 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbc88065cf0 0x7fbc880681b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbca815ce50 0x7fbca815bca0 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbca80cf5d0 0x7fbca80ae820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 --2- 172.21.15.32:0/1294431421 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbca806bd50 0x7fbca80cbcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 -- 172.21.15.32:0/1294431421 >> 172.21.15.32:0/1294431421 conn(0x7fbca8099d70 msgr2=0x7fbca8154a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 -- 172.21.15.32:0/1294431421 shutdown_connections 2023-12-20T06:02:02.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:01.997+0000 7fbcacb52640 1 -- 172.21.15.32:0/1294431421 wait complete. 2023-12-20T06:02:02.019 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-20T06:02:02.019 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:02.019 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:02:02.029 INFO:teuthology.orchestra.run.smithi032.stdout:scheduled-2023-12-20-06_02_00_UTC 2023-12-20T06:02:02.030 DEBUG:tasks.cephfs.test_snap_schedules:removing snapshot: snap_test_dir1/.snap/scheduled-2023-12-20-06_02_00_UTC 2023-12-20T06:02:02.030 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:02.030 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1/.snap/scheduled-2023-12-20-06_02_00_UTC) 2023-12-20T06:02:02.086 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:02.086 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-20T06:02:02.139 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T06:02:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef03760640 1 Processor -- start 2023-12-20T06:02:02.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef03760640 1 -- start start 2023-12-20T06:02:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef03760640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7feefc157480 0x7feefc0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef03760640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feefc0b0760 0x7feefc0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef03760640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feefc0b1100 con 0x7feefc15d660 2023-12-20T06:02:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef03760640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feefc0cde20 con 0x7feefc157480 2023-12-20T06:02:02.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef03760640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0c8e30 con 0x7feefc0b0760 2023-12-20T06:02:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef01cfd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feefc15d660 0x7feefc15daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39018/0 (socket says 172.21.15.32:39018) 2023-12-20T06:02:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef01cfd640 1 -- 172.21.15.32:0/1921814851 learned_addr learned my addr 172.21.15.32:0/1921814851 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef014fc640 1 --2- 172.21.15.32:0/1921814851 >> v2:172.21.15.39:3300/0 conn(0x7feefc157480 0x7feefc0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef00cfb640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feefc0b0760 0x7feefc0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1787511780 0 0) 0x7feefc0b1100 con 0x7feefc15d660 2023-12-20T06:02:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feee8003530 con 0x7feefc15d660 2023-12-20T06:02:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3676468829 0 0) 0x7feee8003530 con 0x7feefc15d660 2023-12-20T06:02:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feefc0b0760 msgr2=0x7feefc0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feefc0b0760 0x7feefc0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 >> v2:172.21.15.39:3300/0 conn(0x7feefc157480 msgr2=0x7feefc0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:02.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 --2- 172.21.15.32:0/1921814851 >> v2:172.21.15.39:3300/0 conn(0x7feefc157480 0x7feefc0c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0ce070 con 0x7feefc15d660 2023-12-20T06:02:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7feef4001a00 con 0x7feefc15d660 2023-12-20T06:02:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 >> v1:172.21.15.32:6789/0 conn(0x7feefc15d660 legacy=0x7feefc15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee8003980 0x7feee8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee80047d0 0x7feee8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feefc0b0760 0x7feee8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0cde20 con 0x7feee80047d0 2023-12-20T06:02:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0c8e30 con 0x7feee8003980 2023-12-20T06:02:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feee800dba0 con 0x7feefc0b0760 2023-12-20T06:02:02.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.325+0000 7fef00cfb640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee80047d0 0x7feee8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:02.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef014fc640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee8003980 0x7feee8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:02.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef01cfd640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feefc0b0760 0x7feee8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:02.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef00cfb640 1 -- 172.21.15.32:0/1921814851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feefc0b0760 msgr2=0x7feee8005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:02.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef00cfb640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feefc0b0760 0x7feee8005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef00cfb640 1 -- 172.21.15.32:0/1921814851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee8003980 msgr2=0x7feee8003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:02.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef00cfb640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feee8003980 0x7feee8003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef00cfb640 1 -- 172.21.15.32:0/1921814851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feee8003530 con 0x7feee80047d0 2023-12-20T06:02:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef00cfb640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee80047d0 0x7feee8004c50 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7feeec009b10 tx=0x7feeec01ca40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1dda15353a12046 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0a6440 con 0x7feee80047d0 2023-12-20T06:02:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feeec0a65a0 con 0x7feee80047d0 2023-12-20T06:02:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7feef3fff640 1 -- 172.21.15.32:0/1921814851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feeec0a6950 con 0x7feee80047d0 2023-12-20T06:02:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef03760640 1 -- 172.21.15.32:0/1921814851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee80047d0 msgr2=0x7feee8004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef03760640 1 --2- 172.21.15.32:0/1921814851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feee80047d0 0x7feee8004c50 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7feeec009b10 tx=0x7feeec01ca40 comp rx=0 tx=0).stop 2023-12-20T06:02:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef01cfd640 1 -- 172.21.15.32:0/1921814851 reap_dead start 2023-12-20T06:02:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef03760640 1 -- 172.21.15.32:0/1921814851 shutdown_connections 2023-12-20T06:02:02.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.329+0000 7fef03760640 1 -- 172.21.15.32:0/1921814851 >> 172.21.15.32:0/1921814851 conn(0x7feefc06a510 msgr2=0x7feefc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef03760640 1 -- 172.21.15.32:0/1921814851 shutdown_connections 2023-12-20T06:02:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef03760640 1 -- 172.21.15.32:0/1921814851 wait complete. 2023-12-20T06:02:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef03760640 1 Processor -- start 2023-12-20T06:02:02.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef03760640 1 -- start start 2023-12-20T06:02:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef03760640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feee80047d0 0x7feefc1565b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef03760640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0b19c0 0x7feefc148ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef03760640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feefc0c8130 0x7feefc0c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef03760640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0b2160 con 0x7feefc0c8130 2023-12-20T06:02:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef03760640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0b1e20 con 0x7feefc0b19c0 2023-12-20T06:02:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef03760640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feefc0cbdb0 con 0x7feee80047d0 2023-12-20T06:02:02.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef01cfd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feefc0c8130 0x7feefc0c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef01cfd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feefc0c8130 0x7feefc0c8590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50264/0 (socket says 172.21.15.32:50264) 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef01cfd640 1 -- 172.21.15.32:0/858601587 learned_addr learned my addr 172.21.15.32:0/858601587 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef00cfb640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0b19c0 0x7feefc148ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef014fc640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feee80047d0 0x7feefc1565b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef01cfd640 1 -- 172.21.15.32:0/858601587 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feee80047d0 msgr2=0x7feefc1565b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef01cfd640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feee80047d0 0x7feefc1565b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef01cfd640 1 -- 172.21.15.32:0/858601587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0b19c0 msgr2=0x7feefc148ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef01cfd640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0b19c0 0x7feefc148ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef01cfd640 1 -- 172.21.15.32:0/858601587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feefc0cb510 con 0x7feefc0c8130 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef00cfb640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0b19c0 0x7feefc148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef014fc640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feee80047d0 0x7feefc1565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:02.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.333+0000 7fef01cfd640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feefc0c8130 0x7feefc0c8590 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7feef4001cc0 tx=0x7feef400e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.337+0000 7feef1ffb640 1 -- 172.21.15.32:0/858601587 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef400ec10 con 0x7feefc0c8130 2023-12-20T06:02:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.337+0000 7feef1ffb640 1 -- 172.21.15.32:0/858601587 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feef400ed90 con 0x7feefc0c8130 2023-12-20T06:02:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.337+0000 7feef1ffb640 1 -- 172.21.15.32:0/858601587 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feef4013660 con 0x7feefc0c8130 2023-12-20T06:02:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.337+0000 7fef03760640 1 -- 172.21.15.32:0/858601587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feefc0cb7f0 con 0x7feefc0c8130 2023-12-20T06:02:02.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.337+0000 7fef03760640 1 -- 172.21.15.32:0/858601587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feefc0c8ad0 con 0x7feefc0c8130 2023-12-20T06:02:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.337+0000 7fef03760640 1 -- 172.21.15.32:0/858601587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feefc04e000 con 0x7feefc0c8130 2023-12-20T06:02:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.337+0000 7feef1ffb640 1 -- 172.21.15.32:0/858601587 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 119) v1 ==== 85461+0+0 (secure 0 0 0) 0x7feef40137c0 con 0x7feefc0c8130 2023-12-20T06:02:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.337+0000 7feef1ffb640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feee0065cf0 0x7feee00681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.337+0000 7feef1ffb640 1 -- 172.21.15.32:0/858601587 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6109+0+0 (secure 0 0 0) 0x7feef4083b50 con 0x7feefc0c8130 2023-12-20T06:02:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.337+0000 7fef014fc640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feee0065cf0 0x7feee00681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:02.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.341+0000 7fef014fc640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feee0065cf0 0x7feee00681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feef8009b50 tx=0x7feef8008040 comp rx=0 tx=0).ready entity=mgr.10206 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:02.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.341+0000 7feef1ffb640 1 -- 172.21.15.32:0/858601587 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feef40510e0 con 0x7feefc0c8130 2023-12-20T06:02:02.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.537+0000 7fef03760640 1 -- 172.21.15.32:0/858601587 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7feefc15d040 con 0x7feefc0c8130 2023-12-20T06:02:02.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.801+0000 7feef1ffb640 1 -- 172.21.15.32:0/858601587 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 120) v1 ==== 85444+0+0 (secure 0 0 0) 0x7feef40514b0 con 0x7feefc0c8130 2023-12-20T06:02:02.804 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.801+0000 7feef1ffb640 1 -- 172.21.15.32:0/858601587 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v120) v1 ==== 93+0+0 (secure 0 0 0) 0x7feef40811c0 con 0x7feefc0c8130 2023-12-20T06:02:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 -- 172.21.15.32:0/858601587 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feee0065cf0 msgr2=0x7feee00681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feee0065cf0 0x7feee00681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7feef8009b50 tx=0x7feef8008040 comp rx=0 tx=0).stop 2023-12-20T06:02:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 -- 172.21.15.32:0/858601587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feefc0c8130 msgr2=0x7feefc0c8590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feefc0c8130 0x7feefc0c8590 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7feef4001cc0 tx=0x7feef400e530 comp rx=0 tx=0).stop 2023-12-20T06:02:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 -- 172.21.15.32:0/858601587 shutdown_connections 2023-12-20T06:02:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feee0065cf0 0x7feee00681b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feefc0c8130 0x7feefc0c8590 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.807 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feefc0b19c0 0x7feefc148ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 --2- 172.21.15.32:0/858601587 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feee80047d0 0x7feefc1565b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:02.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 -- 172.21.15.32:0/858601587 >> 172.21.15.32:0/858601587 conn(0x7feefc06a510 msgr2=0x7feefc0c4f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:02.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 -- 172.21.15.32:0/858601587 shutdown_connections 2023-12-20T06:02:02.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:02.805+0000 7fef03760640 1 -- 172.21.15.32:0/858601587 wait complete. 2023-12-20T06:02:02.827 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T06:02:02.827 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:02:02.943 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:02:02.981 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:02.976+0000 7fe542b5c180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:02:02.986 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:02.985+0000 7f90acc7d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:02:03.012 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T06:02:03.013 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:02:03.013 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:03.023 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:02:03.023 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:03.073 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.257+0000 7febafbbe640 1 Processor -- start 2023-12-20T06:02:03.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febafbbe640 1 -- start start 2023-12-20T06:02:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febafbbe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7feba806cf10 0x7feba80c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febafbbe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feba8164ef0 0x7feba80c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febafbbe640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feba80ae3f0 con 0x7feba80ae010 2023-12-20T06:02:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febafbbe640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba8161800 con 0x7feba8164ef0 2023-12-20T06:02:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febafbbe640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feba8165380 con 0x7feba806cf10 2023-12-20T06:02:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febad95a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feba80ae010 0x7feba8071470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39042/0 (socket says 172.21.15.32:39042) 2023-12-20T06:02:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febad95a640 1 -- 172.21.15.32:0/41090270 learned_addr learned my addr 172.21.15.32:0/41090270 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febae15b640 1 --2- 172.21.15.32:0/41090270 >> v2:172.21.15.32:3300/0 conn(0x7feba8164ef0 0x7feba80c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febad159640 1 --2- 172.21.15.32:0/41090270 >> v2:172.21.15.39:3300/0 conn(0x7feba806cf10 0x7feba80c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febae15b640 1 -- 172.21.15.32:0/41090270 >> v2:172.21.15.39:3300/0 conn(0x7feba806cf10 msgr2=0x7feba80c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febae15b640 1 --2- 172.21.15.32:0/41090270 >> v2:172.21.15.39:3300/0 conn(0x7feba806cf10 0x7feba80c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febae15b640 1 -- 172.21.15.32:0/41090270 >> v1:172.21.15.32:6789/0 conn(0x7feba80ae010 legacy=0x7feba8071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.261+0000 7febae15b640 1 -- 172.21.15.32:0/41090270 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba8165660 con 0x7feba8164ef0 2023-12-20T06:02:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febae15b640 1 --2- 172.21.15.32:0/41090270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba8164ef0 0x7feba80c99e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7feba8164e50 tx=0x7feba001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d3aec48da27c12 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febac958640 1 -- 172.21.15.32:0/41090270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba00a5910 con 0x7feba8164ef0 2023-12-20T06:02:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febac958640 1 -- 172.21.15.32:0/41090270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba00a5a70 con 0x7feba8164ef0 2023-12-20T06:02:03.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febac958640 1 -- 172.21.15.32:0/41090270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba00a5e00 con 0x7feba8164ef0 2023-12-20T06:02:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febafbbe640 1 -- 172.21.15.32:0/41090270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba8164ef0 msgr2=0x7feba80c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febafbbe640 1 --2- 172.21.15.32:0/41090270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba8164ef0 0x7feba80c99e0 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7feba8164e50 tx=0x7feba001c850 comp rx=0 tx=0).stop 2023-12-20T06:02:03.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febafbbe640 1 -- 172.21.15.32:0/41090270 shutdown_connections 2023-12-20T06:02:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febafbbe640 1 --2- 172.21.15.32:0/41090270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba8164ef0 0x7feba80c99e0 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febafbbe640 1 --2- 172.21.15.32:0/41090270 >> v2:172.21.15.39:3300/0 conn(0x7feba806cf10 0x7feba80c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febafbbe640 1 -- 172.21.15.32:0/41090270 >> 172.21.15.32:0/41090270 conn(0x7feba806a320 msgr2=0x7feba80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febafbbe640 1 -- 172.21.15.32:0/41090270 shutdown_connections 2023-12-20T06:02:03.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febafbbe640 1 -- 172.21.15.32:0/41090270 wait complete. 2023-12-20T06:02:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.265+0000 7febafbbe640 1 Processor -- start 2023-12-20T06:02:03.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febafbbe640 1 -- start start 2023-12-20T06:02:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febafbbe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba80ae010 0x7feba80c6680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febafbbe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feba80cdf60 0x7feba8159040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febafbbe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feba80700c0 0x7feba8070540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febafbbe640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba8070a80 con 0x7feba80ae010 2023-12-20T06:02:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febafbbe640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feba80c6bc0 con 0x7feba80700c0 2023-12-20T06:02:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febafbbe640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feba80c8260 con 0x7feba80cdf60 2023-12-20T06:02:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febad95a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba80ae010 0x7feba80c6680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febad95a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba80ae010 0x7feba80c6680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50290/0 (socket says 172.21.15.32:50290) 2023-12-20T06:02:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febad95a640 1 -- 172.21.15.32:0/2801924489 learned_addr learned my addr 172.21.15.32:0/2801924489 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febad159640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feba80cdf60 0x7feba8159040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febae15b640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feba80700c0 0x7feba8070540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febad95a640 1 -- 172.21.15.32:0/2801924489 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feba80cdf60 msgr2=0x7feba8159040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febad95a640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feba80cdf60 0x7feba8159040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febad95a640 1 -- 172.21.15.32:0/2801924489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feba80700c0 msgr2=0x7feba8070540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febad95a640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feba80700c0 0x7feba8070540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febad95a640 1 -- 172.21.15.32:0/2801924489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feba0009a70 con 0x7feba80ae010 2023-12-20T06:02:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febad95a640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba80ae010 0x7feba80c6680 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7feba400c4d0 tx=0x7feba400c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7feb9e7fc640 1 -- 172.21.15.32:0/2801924489 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba4007480 con 0x7feba80ae010 2023-12-20T06:02:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7feb9e7fc640 1 -- 172.21.15.32:0/2801924489 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feba40075e0 con 0x7feba80ae010 2023-12-20T06:02:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7feb9e7fc640 1 -- 172.21.15.32:0/2801924489 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feba40059b0 con 0x7feba80ae010 2023-12-20T06:02:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febafbbe640 1 -- 172.21.15.32:0/2801924489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feba8159580 con 0x7feba80ae010 2023-12-20T06:02:03.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febafbbe640 1 -- 172.21.15.32:0/2801924489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feba80c7930 con 0x7feba80ae010 2023-12-20T06:02:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.269+0000 7febafbbe640 1 -- 172.21.15.32:0/2801924489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feba804e000 con 0x7feba80ae010 2023-12-20T06:02:03.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.273+0000 7feb9e7fc640 1 -- 172.21.15.32:0/2801924489 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 120) v1 ==== 85444+0+0 (secure 0 0 0) 0x7feba4005b10 con 0x7feba80ae010 2023-12-20T06:02:03.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.273+0000 7feb9e7fc640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feb8c065ca0 0x7feb8c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.273+0000 7feb9e7fc640 1 -- 172.21.15.32:0/2801924489 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6109+0+0 (secure 0 0 0) 0x7feba4082b80 con 0x7feba80ae010 2023-12-20T06:02:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.273+0000 7febad159640 1 -- 172.21.15.32:0/2801924489 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feb8c065ca0 msgr2=0x7feb8c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:03.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.273+0000 7febad159640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feb8c065ca0 0x7feb8c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:03.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.277+0000 7feb9e7fc640 1 -- 172.21.15.32:0/2801924489 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feba4050100 con 0x7feba80ae010 2023-12-20T06:02:03.420 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:03.416+0000 7fe542b5c180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:02:03.430 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:03.429+0000 7f90acc7d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:02:03.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.437+0000 7febafbbe640 1 -- 172.21.15.32:0/2801924489 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feba8161290 con 0x7feba80ae010 2023-12-20T06:02:03.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.437+0000 7feb9e7fc640 1 -- 172.21.15.32:0/2801924489 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v193) v1 ==== 74+0+16636 (secure 0 0 0) 0x7feba4017070 con 0x7feba80ae010 2023-12-20T06:02:03.440 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:03.440 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":193,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:00:49.693771+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:00:36.162516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-20T06:00:36.593153+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"192","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3239744916":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/2437361294":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/2674032592":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/4037417795":"2023-12-21T06:00:49.693740+0000","172.21.15.32:6833/64546":"2023-12-21T06:00:49.693740+0000","172.21.15.32:6832/64546":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/4289008383":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/1643902464":"2023-12-21T06:00:49.693740+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 -- 172.21.15.32:0/2801924489 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feb8c065ca0 msgr2=0x7feb8c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feb8c065ca0 0x7feb8c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 -- 172.21.15.32:0/2801924489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba80ae010 msgr2=0x7feba80c6680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba80ae010 0x7feba80c6680 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7feba400c4d0 tx=0x7feba400c990 comp rx=0 tx=0).stop 2023-12-20T06:02:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 -- 172.21.15.32:0/2801924489 shutdown_connections 2023-12-20T06:02:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feb8c065ca0 0x7feb8c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feba80700c0 0x7feba8070540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feba80cdf60 0x7feba8159040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 --2- 172.21.15.32:0/2801924489 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feba80ae010 0x7feba80c6680 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 -- 172.21.15.32:0/2801924489 >> 172.21.15.32:0/2801924489 conn(0x7feba806a320 msgr2=0x7feba815d6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 -- 172.21.15.32:0/2801924489 shutdown_connections 2023-12-20T06:02:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.441+0000 7febafbbe640 1 -- 172.21.15.32:0/2801924489 wait complete. 2023-12-20T06:02:03.465 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T06:00:36.162516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '191', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T06:00:36.593153+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '192', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:02:03.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:02:03.578 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:03.576+0000 7fe542b5c180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:02:03.592 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:03.589+0000 7f90acc7d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:02:03.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.613+0000 7f2498325640 1 Processor -- start 2023-12-20T06:02:03.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.613+0000 7f2498325640 1 -- start start 2023-12-20T06:02:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.613+0000 7f2498325640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24900d3300 0x7f24900d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.613+0000 7f2498325640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24900d2650 con 0x7f249015ce60 2023-12-20T06:02:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.613+0000 7f2498325640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24900d1a40 con 0x7f24900d1300 2023-12-20T06:02:03.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.613+0000 7f2498325640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24900d1d20 con 0x7f24900d3300 2023-12-20T06:02:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.617+0000 7f24960c1640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f249015ce60 0x7f24900d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39048/0 (socket says 172.21.15.32:39048) 2023-12-20T06:02:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.617+0000 7f24960c1640 1 -- 172.21.15.32:0/2612270536 learned_addr learned my addr 172.21.15.32:0/2612270536 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.617+0000 7f24958c0640 1 --2- 172.21.15.32:0/2612270536 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24900d3300 0x7f24900d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.617+0000 7f24950bf640 1 -- 172.21.15.32:0/2612270536 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1900945357 0 0) 0x7f24900d2650 con 0x7f249015ce60 2023-12-20T06:02:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.617+0000 7f24950bf640 1 -- 172.21.15.32:0/2612270536 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f247c003530 con 0x7f249015ce60 2023-12-20T06:02:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.617+0000 7f24950bf640 1 -- 172.21.15.32:0/2612270536 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2864182588 0 0) 0x7f24900d1a40 con 0x7f24900d1300 2023-12-20T06:02:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.617+0000 7f24950bf640 1 -- 172.21.15.32:0/2612270536 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f24900d2650 con 0x7f24900d1300 2023-12-20T06:02:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.617+0000 7f24958c0640 1 -- 172.21.15.32:0/2612270536 >> v1:172.21.15.39:6789/0 conn(0x7f24900d1300 legacy=0x7f24900b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.617+0000 7f24958c0640 1 -- 172.21.15.32:0/2612270536 >> v1:172.21.15.32:6789/0 conn(0x7f249015ce60 legacy=0x7f24900d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.617+0000 7f24958c0640 1 -- 172.21.15.32:0/2612270536 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24900d2000 con 0x7f24900d3300 2023-12-20T06:02:03.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.617+0000 7f24958c0640 1 --2- 172.21.15.32:0/2612270536 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24900d3300 0x7f24900d0dc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f24800097f0 tx=0x7f248001ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dbc1d1eab47560d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:03.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f24950bf640 1 -- 172.21.15.32:0/2612270536 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24800a89b0 con 0x7f24900d3300 2023-12-20T06:02:03.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f24950bf640 1 -- 172.21.15.32:0/2612270536 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24800a8b10 con 0x7f24900d3300 2023-12-20T06:02:03.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f24950bf640 1 -- 172.21.15.32:0/2612270536 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24800a8e80 con 0x7f24900d3300 2023-12-20T06:02:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 -- 172.21.15.32:0/2612270536 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24900d3300 msgr2=0x7f24900d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 --2- 172.21.15.32:0/2612270536 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24900d3300 0x7f24900d0dc0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f24800097f0 tx=0x7f248001ecb0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 -- 172.21.15.32:0/2612270536 shutdown_connections 2023-12-20T06:02:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 --2- 172.21.15.32:0/2612270536 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24900d3300 0x7f24900d0dc0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 -- 172.21.15.32:0/2612270536 >> 172.21.15.32:0/2612270536 conn(0x7f249006b5e0 msgr2=0x7f249006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 -- 172.21.15.32:0/2612270536 shutdown_connections 2023-12-20T06:02:03.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 -- 172.21.15.32:0/2612270536 wait complete. 2023-12-20T06:02:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 Processor -- start 2023-12-20T06:02:03.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 -- start start 2023-12-20T06:02:03.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900d1300 0x7f24901e7300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24900d3300 0x7f24900c8e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.621+0000 7f2498325640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f249015ce60 0x7f24901e2b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f2498325640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24901e6ee0 con 0x7f24900d1300 2023-12-20T06:02:03.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f2498325640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24901e65d0 con 0x7f24900d3300 2023-12-20T06:02:03.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f2498325640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24901e38e0 con 0x7f249015ce60 2023-12-20T06:02:03.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24960c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900d1300 0x7f24901e7300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24960c1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900d1300 0x7f24901e7300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50300/0 (socket says 172.21.15.32:50300) 2023-12-20T06:02:03.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24960c1640 1 -- 172.21.15.32:0/3243246140 learned_addr learned my addr 172.21.15.32:0/3243246140 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:03.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24958c0640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24900d3300 0x7f24900c8e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24968c2640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f249015ce60 0x7f24901e2b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24958c0640 1 -- 172.21.15.32:0/3243246140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f249015ce60 msgr2=0x7f24901e2b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24958c0640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f249015ce60 0x7f24901e2b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24958c0640 1 -- 172.21.15.32:0/3243246140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900d1300 msgr2=0x7f24901e7300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24958c0640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900d1300 0x7f24901e7300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24958c0640 1 -- 172.21.15.32:0/3243246140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2480009420 con 0x7f24900d3300 2023-12-20T06:02:03.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24960c1640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900d1300 0x7f24901e7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:03.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f24958c0640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24900d3300 0x7f24900c8e30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f24900b5850 tx=0x7f248001ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:03.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f2486ffd640 1 -- 172.21.15.32:0/3243246140 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480003820 con 0x7f24900d3300 2023-12-20T06:02:03.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f2486ffd640 1 -- 172.21.15.32:0/3243246140 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2480003980 con 0x7f24900d3300 2023-12-20T06:02:03.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f2498325640 1 -- 172.21.15.32:0/3243246140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24901e6820 con 0x7f24900d3300 2023-12-20T06:02:03.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f2486ffd640 1 -- 172.21.15.32:0/3243246140 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2480003d10 con 0x7f24900d3300 2023-12-20T06:02:03.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f2498325640 1 -- 172.21.15.32:0/3243246140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24900c9370 con 0x7f24900d3300 2023-12-20T06:02:03.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f2498325640 1 -- 172.21.15.32:0/3243246140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24900cefc0 con 0x7f24900d3300 2023-12-20T06:02:03.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.625+0000 7f2486ffd640 1 -- 172.21.15.32:0/3243246140 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 120) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f24800b7040 con 0x7f24900d3300 2023-12-20T06:02:03.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.629+0000 7f2486ffd640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2474065ca0 0x7f2474068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.629+0000 7f24960c1640 1 -- 172.21.15.32:0/3243246140 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2474065ca0 msgr2=0x7f2474068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:03.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.629+0000 7f2486ffd640 1 -- 172.21.15.32:0/3243246140 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(193..193 src has 1..193) v4 ==== 6109+0+0 (secure 0 0 0) 0x7f24801210d0 con 0x7f24900d3300 2023-12-20T06:02:03.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.629+0000 7f24960c1640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2474065ca0 0x7f2474068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:03.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.633+0000 7f2486ffd640 1 -- 172.21.15.32:0/3243246140 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f24800ee640 con 0x7f24900d3300 2023-12-20T06:02:03.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.789+0000 7f2498325640 1 -- 172.21.15.32:0/3243246140 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f249004e000 con 0x7f24900d3300 2023-12-20T06:02:03.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2486ffd640 1 -- 172.21.15.32:0/3243246140 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v193) v1 ==== 93+0+10 (secure 0 0 0) 0x7f24800f1c60 con 0x7f24900d3300 2023-12-20T06:02:03.792 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:02:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 -- 172.21.15.32:0/3243246140 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2474065ca0 msgr2=0x7f2474068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2474065ca0 0x7f2474068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 -- 172.21.15.32:0/3243246140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24900d3300 msgr2=0x7f24900c8e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24900d3300 0x7f24900c8e30 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f24900b5850 tx=0x7f248001ba50 comp rx=0 tx=0).stop 2023-12-20T06:02:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 -- 172.21.15.32:0/3243246140 shutdown_connections 2023-12-20T06:02:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2474065ca0 0x7f2474068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f249015ce60 0x7f24901e2b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24900d3300 0x7f24900c8e30 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 --2- 172.21.15.32:0/3243246140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24900d1300 0x7f24901e7300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 -- 172.21.15.32:0/3243246140 >> 172.21.15.32:0/3243246140 conn(0x7f249006b5e0 msgr2=0x7f249006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:03.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 -- 172.21.15.32:0/3243246140 shutdown_connections 2023-12-20T06:02:03.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.793+0000 7f2498325640 1 -- 172.21.15.32:0/3243246140 wait complete. 2023-12-20T06:02:03.818 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:02:03.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.965+0000 7fd6f91bc640 1 Processor -- start 2023-12-20T06:02:03.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.965+0000 7fd6f91bc640 1 -- start start 2023-12-20T06:02:03.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f91bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 0x7fd6f40b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f91bc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f40d2650 con 0x7fd6f415ce60 2023-12-20T06:02:03.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f91bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd6f40d1a40 con 0x7fd6f40d3300 2023-12-20T06:02:03.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f91bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f40d1d20 con 0x7fd6f40d14d0 2023-12-20T06:02:03.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f2d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd6f415ce60 0x7fd6f40d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39062/0 (socket says 172.21.15.32:39062) 2023-12-20T06:02:03.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f2d9d640 1 -- 172.21.15.32:0/3781574597 learned_addr learned my addr 172.21.15.32:0/3781574597 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:03.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f359e640 1 --2- 172.21.15.32:0/3781574597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 0x7fd6f40b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f1d9b640 1 -- 172.21.15.32:0/3781574597 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 895657849 0 0) 0x7fd6f40d2650 con 0x7fd6f415ce60 2023-12-20T06:02:03.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f1d9b640 1 -- 172.21.15.32:0/3781574597 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd6dc003530 con 0x7fd6f415ce60 2023-12-20T06:02:03.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f359e640 1 -- 172.21.15.32:0/3781574597 >> v1:172.21.15.39:6789/0 conn(0x7fd6f40d3300 legacy=0x7fd6f40d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f359e640 1 -- 172.21.15.32:0/3781574597 >> v1:172.21.15.32:6789/0 conn(0x7fd6f415ce60 legacy=0x7fd6f40d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.969+0000 7fd6f359e640 1 -- 172.21.15.32:0/3781574597 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6f40d2000 con 0x7fd6f40d14d0 2023-12-20T06:02:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f359e640 1 --2- 172.21.15.32:0/3781574597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 0x7fd6f40b6ea0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009dc0 tx=0x7fd6ec01f150 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d112b73b832fd0a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f1d9b640 1 -- 172.21.15.32:0/3781574597 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0af600 con 0x7fd6f40d14d0 2023-12-20T06:02:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f1d9b640 1 -- 172.21.15.32:0/3781574597 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6ec0ab050 con 0x7fd6f40d14d0 2023-12-20T06:02:03.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f1d9b640 1 -- 172.21.15.32:0/3781574597 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6ec0af980 con 0x7fd6f40d14d0 2023-12-20T06:02:03.974 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f91bc640 1 -- 172.21.15.32:0/3781574597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 msgr2=0x7fd6f40b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f91bc640 1 --2- 172.21.15.32:0/3781574597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 0x7fd6f40b6ea0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd6ec009dc0 tx=0x7fd6ec01f150 comp rx=0 tx=0).stop 2023-12-20T06:02:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f91bc640 1 -- 172.21.15.32:0/3781574597 shutdown_connections 2023-12-20T06:02:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f91bc640 1 --2- 172.21.15.32:0/3781574597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 0x7fd6f40b6ea0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f91bc640 1 -- 172.21.15.32:0/3781574597 >> 172.21.15.32:0/3781574597 conn(0x7fd6f406b5e0 msgr2=0x7fd6f406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f91bc640 1 -- 172.21.15.32:0/3781574597 shutdown_connections 2023-12-20T06:02:03.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f91bc640 1 -- 172.21.15.32:0/3781574597 wait complete. 2023-12-20T06:02:03.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.973+0000 7fd6f91bc640 1 Processor -- start 2023-12-20T06:02:03.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f91bc640 1 -- start start 2023-12-20T06:02:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f91bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 0x7fd6f414b000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f91bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6f40d3300 0x7fd6f415f050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f91bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6f415ce60 0x7fd6f40b2440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f91bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f414ba20 con 0x7fd6f415ce60 2023-12-20T06:02:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f91bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f415fb40 con 0x7fd6f40d3300 2023-12-20T06:02:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f91bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd6f415fe20 con 0x7fd6f40d14d0 2023-12-20T06:02:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f2d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 0x7fd6f414b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f359e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6f415ce60 0x7fd6f40b2440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f2d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 0x7fd6f414b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:46174/0 (socket says 172.21.15.32:46174) 2023-12-20T06:02:03.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f2d9d640 1 -- 172.21.15.32:0/1510699061 learned_addr learned my addr 172.21.15.32:0/1510699061 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:03.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f259c640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6f40d3300 0x7fd6f415f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:03.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f2d9d640 1 -- 172.21.15.32:0/1510699061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6f40d3300 msgr2=0x7fd6f415f050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f2d9d640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6f40d3300 0x7fd6f415f050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f2d9d640 1 -- 172.21.15.32:0/1510699061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6f415ce60 msgr2=0x7fd6f40b2440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:03.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f2d9d640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6f415ce60 0x7fd6f40b2440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:03.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f2d9d640 1 -- 172.21.15.32:0/1510699061 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd6ec009a70 con 0x7fd6f40d14d0 2023-12-20T06:02:03.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f359e640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6f415ce60 0x7fd6f40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:03.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f2d9d640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 0x7fd6f414b000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400c420 tx=0x7fd6e400c8e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:03.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6db7fe640 1 -- 172.21.15.32:0/1510699061 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e4005cf0 con 0x7fd6f40d14d0 2023-12-20T06:02:03.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6db7fe640 1 -- 172.21.15.32:0/1510699061 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd6e4005e50 con 0x7fd6f40d14d0 2023-12-20T06:02:03.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6db7fe640 1 -- 172.21.15.32:0/1510699061 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd6e40072d0 con 0x7fd6f40d14d0 2023-12-20T06:02:03.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f91bc640 1 -- 172.21.15.32:0/1510699061 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd6f4160100 con 0x7fd6f40d14d0 2023-12-20T06:02:03.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f91bc640 1 -- 172.21.15.32:0/1510699061 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd6f415a6a0 con 0x7fd6f40d14d0 2023-12-20T06:02:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.977+0000 7fd6f91bc640 1 -- 172.21.15.32:0/1510699061 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd6f4160320 con 0x7fd6f40d14d0 2023-12-20T06:02:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.981+0000 7fd6db7fe640 1 -- 172.21.15.32:0/1510699061 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 120) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fd6e4007430 con 0x7fd6f40d14d0 2023-12-20T06:02:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.981+0000 7fd6db7fe640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd6e0065cd0 0x7fd6e0068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.981+0000 7fd6db7fe640 1 -- 172.21.15.32:0/1510699061 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fd6e4082c90 con 0x7fd6f40d14d0 2023-12-20T06:02:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.981+0000 7fd6f259c640 1 -- 172.21.15.32:0/1510699061 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd6e0065cd0 msgr2=0x7fd6e0068190 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:03.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.981+0000 7fd6f259c640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd6e0065cd0 0x7fd6e0068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:03.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:03.985+0000 7fd6db7fe640 1 -- 172.21.15.32:0/1510699061 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd6e40501a0 con 0x7fd6f40d14d0 2023-12-20T06:02:04.096 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:04.092+0000 7fe542b5c180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:04.120 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:04.117+0000 7f90acc7d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:04.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.145+0000 7fd6f91bc640 1 -- 172.21.15.32:0/1510699061 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd6f415ff60 con 0x7fd6f40d14d0 2023-12-20T06:02:04.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.145+0000 7fd6db7fe640 1 -- 172.21.15.32:0/1510699061 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v194) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd6f415ff60 con 0x7fd6f40d14d0 2023-12-20T06:02:04.147 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:02:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 -- 172.21.15.32:0/1510699061 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd6e0065cd0 msgr2=0x7fd6e0068190 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd6e0065cd0 0x7fd6e0068190 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 -- 172.21.15.32:0/1510699061 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 msgr2=0x7fd6f414b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 0x7fd6f414b000 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd6e400c420 tx=0x7fd6e400c8e0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 -- 172.21.15.32:0/1510699061 shutdown_connections 2023-12-20T06:02:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd6e0065cd0 0x7fd6e0068190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd6f415ce60 0x7fd6f40b2440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd6f40d3300 0x7fd6f415f050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 --2- 172.21.15.32:0/1510699061 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd6f40d14d0 0x7fd6f414b000 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 -- 172.21.15.32:0/1510699061 >> 172.21.15.32:0/1510699061 conn(0x7fd6f406b5e0 msgr2=0x7fd6f40d0dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 -- 172.21.15.32:0/1510699061 shutdown_connections 2023-12-20T06:02:04.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.149+0000 7fd6f91bc640 1 -- 172.21.15.32:0/1510699061 wait complete. 2023-12-20T06:02:04.158 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:04.156+0000 7fe542b5c180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:02:04.179 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:02:04.195 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:04.193+0000 7f90acc7d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:02:04.217 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:04.212+0000 7fe542b5c180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:02:04.254 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:04.253+0000 7f90acc7d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:02:04.313 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:04.308+0000 7fe542b5c180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:02:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.329+0000 7f64cc5f0640 1 Processor -- start 2023-12-20T06:02:04.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.329+0000 7f64cc5f0640 1 -- start start 2023-12-20T06:02:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.329+0000 7f64cc5f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64c4161510 0x7f64c4071760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:04.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64cc5f0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c40cdbd0 con 0x7f64c415cef0 2023-12-20T06:02:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64cc5f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f64c40cd460 con 0x7f64c40cde70 2023-12-20T06:02:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64cc5f0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f64c40cd7b0 con 0x7f64c4161510 2023-12-20T06:02:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64c9b8b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f64c4161510 0x7f64c4071760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64ca38c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f64c415cef0 0x7f64c40ca270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39070/0 (socket says 172.21.15.32:39070) 2023-12-20T06:02:04.333 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64ca38c640 1 -- 172.21.15.32:0/2429012451 learned_addr learned my addr 172.21.15.32:0/2429012451 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:04.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64c9b8b640 1 -- 172.21.15.32:0/2429012451 >> v1:172.21.15.39:6789/0 conn(0x7f64c40cde70 legacy=0x7f64c40cc7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64c9b8b640 1 -- 172.21.15.32:0/2429012451 >> v1:172.21.15.32:6789/0 conn(0x7f64c415cef0 legacy=0x7f64c40ca270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64c9b8b640 1 -- 172.21.15.32:0/2429012451 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64c4071ef0 con 0x7f64c4161510 2023-12-20T06:02:04.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64c938a640 1 -- 172.21.15.32:0/2429012451 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1659548262 0 0) 0x7f64c40cd460 con 0x7f64c40cde70 2023-12-20T06:02:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64c9b8b640 1 --2- 172.21.15.32:0/2429012451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c4161510 0x7f64c4071760 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f64b40097c0 tx=0x7f64b401bf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6dd43e275b53f53 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64c938a640 1 -- 172.21.15.32:0/2429012451 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b40a5700 con 0x7f64c4161510 2023-12-20T06:02:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64c938a640 1 -- 172.21.15.32:0/2429012451 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64b40a5860 con 0x7f64c4161510 2023-12-20T06:02:04.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.333+0000 7f64c938a640 1 -- 172.21.15.32:0/2429012451 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64b40a5bf0 con 0x7f64c4161510 2023-12-20T06:02:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 -- 172.21.15.32:0/2429012451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c4161510 msgr2=0x7f64c4071760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 --2- 172.21.15.32:0/2429012451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c4161510 0x7f64c4071760 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f64b40097c0 tx=0x7f64b401bf30 comp rx=0 tx=0).stop 2023-12-20T06:02:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 -- 172.21.15.32:0/2429012451 shutdown_connections 2023-12-20T06:02:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 --2- 172.21.15.32:0/2429012451 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c4161510 0x7f64c4071760 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 -- 172.21.15.32:0/2429012451 >> 172.21.15.32:0/2429012451 conn(0x7f64c406b230 msgr2=0x7f64c406b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 -- 172.21.15.32:0/2429012451 shutdown_connections 2023-12-20T06:02:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 -- 172.21.15.32:0/2429012451 wait complete. 2023-12-20T06:02:04.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 Processor -- start 2023-12-20T06:02:04.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 -- start start 2023-12-20T06:02:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c40cde70 0x7f64c40c9340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c415cef0 0x7f64c4153f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64c4161510 0x7f64c40b24b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c40ad470 con 0x7f64c40cde70 2023-12-20T06:02:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64c40c8ff0 con 0x7f64c4161510 2023-12-20T06:02:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cc5f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64c4159c40 con 0x7f64c415cef0 2023-12-20T06:02:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64ca38c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c40cde70 0x7f64c40c9340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64ca38c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c40cde70 0x7f64c40c9340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50330/0 (socket says 172.21.15.32:50330) 2023-12-20T06:02:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64ca38c640 1 -- 172.21.15.32:0/4214283402 learned_addr learned my addr 172.21.15.32:0/4214283402 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64c9b8b640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c415cef0 0x7f64c4153f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:04.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.337+0000 7f64cab8d640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64c4161510 0x7f64c40b24b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64cab8d640 1 -- 172.21.15.32:0/4214283402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c415cef0 msgr2=0x7f64c4153f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64cab8d640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c415cef0 0x7f64c4153f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64cab8d640 1 -- 172.21.15.32:0/4214283402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c40cde70 msgr2=0x7f64c40c9340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64cab8d640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c40cde70 0x7f64c40c9340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64cab8d640 1 -- 172.21.15.32:0/4214283402 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64b4009420 con 0x7f64c4161510 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64ca38c640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c40cde70 0x7f64c40c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64c9b8b640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c415cef0 0x7f64c4153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64cab8d640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64c4161510 0x7f64c40b24b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00c780 tx=0x7f64bc00cc40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64baffd640 1 -- 172.21.15.32:0/4214283402 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc003b80 con 0x7f64c4161510 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64baffd640 1 -- 172.21.15.32:0/4214283402 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64bc003ce0 con 0x7f64c4161510 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64baffd640 1 -- 172.21.15.32:0/4214283402 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64bc0052b0 con 0x7f64c4161510 2023-12-20T06:02:04.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64cc5f0640 1 -- 172.21.15.32:0/4214283402 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f64c40c8d10 con 0x7f64c4161510 2023-12-20T06:02:04.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64cc5f0640 1 -- 172.21.15.32:0/4214283402 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f64c40c75b0 con 0x7f64c4161510 2023-12-20T06:02:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64cc5f0640 1 -- 172.21.15.32:0/4214283402 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f64c40ad5e0 con 0x7f64c4161510 2023-12-20T06:02:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64baffd640 1 -- 172.21.15.32:0/4214283402 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 120) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f64bc005410 con 0x7f64c4161510 2023-12-20T06:02:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.341+0000 7f64baffd640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64a8065ca0 0x7f64a8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.345+0000 7f64baffd640 1 -- 172.21.15.32:0/4214283402 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6197+0+0 (secure 0 0 0) 0x7f64bc082e80 con 0x7f64c4161510 2023-12-20T06:02:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.345+0000 7f64ca38c640 1 -- 172.21.15.32:0/4214283402 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64a8065ca0 msgr2=0x7f64a8068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:04.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.345+0000 7f64ca38c640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64a8065ca0 0x7f64a8068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:04.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.345+0000 7f64baffd640 1 -- 172.21.15.32:0/4214283402 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f64bc050380 con 0x7f64c4161510 2023-12-20T06:02:04.353 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:04.353+0000 7f90acc7d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:02:04.373 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:04.368+0000 7fe542b5c180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:02:04.414 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:04.413+0000 7f90acc7d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:02:04.462 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:04.460+0000 7fe542b5c180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:02:04.505 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:04.505+0000 7f90acc7d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:02:04.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.505+0000 7f64cc5f0640 1 -- 172.21.15.32:0/4214283402 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f64c404e000 con 0x7f64c4161510 2023-12-20T06:02:04.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.505+0000 7f64baffd640 1 -- 172.21.15.32:0/4214283402 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v194) v1 ==== 100+0+11 (secure 0 0 0) 0x7f64bc017070 con 0x7f64c4161510 2023-12-20T06:02:04.508 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:02:04.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 -- 172.21.15.32:0/4214283402 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64a8065ca0 msgr2=0x7f64a8068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:04.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64a8065ca0 0x7f64a8068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 -- 172.21.15.32:0/4214283402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64c4161510 msgr2=0x7f64c40b24b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64c4161510 0x7f64c40b24b0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f64bc00c780 tx=0x7f64bc00cc40 comp rx=0 tx=0).stop 2023-12-20T06:02:04.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 -- 172.21.15.32:0/4214283402 shutdown_connections 2023-12-20T06:02:04.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64a8065ca0 0x7f64a8068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64c4161510 0x7f64c40b24b0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64c415cef0 0x7f64c4153f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 --2- 172.21.15.32:0/4214283402 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64c40cde70 0x7f64c40c9340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 -- 172.21.15.32:0/4214283402 >> 172.21.15.32:0/4214283402 conn(0x7f64c406b230 msgr2=0x7f64c40cba40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 -- 172.21.15.32:0/4214283402 shutdown_connections 2023-12-20T06:02:04.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.509+0000 7f64cc5f0640 1 -- 172.21.15.32:0/4214283402 wait complete. 2023-12-20T06:02:04.522 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:04.516+0000 7fe542b5c180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:02:04.530 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T06:02:04.565 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:04.565+0000 7f90acc7d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:02:04.673 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:04.668+0000 7fe542b5c180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:02:04.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.709+0000 7fc82b812640 1 Processor -- start 2023-12-20T06:02:04.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.709+0000 7fc82b812640 1 -- start start 2023-12-20T06:02:04.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.709+0000 7fc82b812640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8241599e0 0x7fc8240aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:04.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.709+0000 7fc82b812640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8240b0d00 0x7fc8240701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:04.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.709+0000 7fc82b812640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8240c98c0 con 0x7fc8240c90e0 2023-12-20T06:02:04.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.709+0000 7fc82b812640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8240c9ba0 con 0x7fc8240b0d00 2023-12-20T06:02:04.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.709+0000 7fc82b812640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240c9e80 con 0x7fc8241599e0 2023-12-20T06:02:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.709+0000 7fc8295ae640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8241599e0 0x7fc8240aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.709+0000 7fc829daf640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8240b0d00 0x7fc8240701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.709+0000 7fc828dad640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc8240c90e0 0x7fc8240b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39078/0 (socket says 172.21.15.32:39078) 2023-12-20T06:02:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.709+0000 7fc828dad640 1 -- 172.21.15.32:0/2640191379 learned_addr learned my addr 172.21.15.32:0/2640191379 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.713+0000 7fc8295ae640 1 -- 172.21.15.32:0/2640191379 >> v2:172.21.15.32:3300/0 conn(0x7fc8240b0d00 msgr2=0x7fc8240701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.713+0000 7fc8295ae640 1 --2- 172.21.15.32:0/2640191379 >> v2:172.21.15.32:3300/0 conn(0x7fc8240b0d00 0x7fc8240701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.713+0000 7fc8295ae640 1 -- 172.21.15.32:0/2640191379 >> v1:172.21.15.32:6789/0 conn(0x7fc8240c90e0 legacy=0x7fc8240b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.713+0000 7fc8295ae640 1 -- 172.21.15.32:0/2640191379 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8240ca160 con 0x7fc8241599e0 2023-12-20T06:02:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.713+0000 7fc8295ae640 1 --2- 172.21.15.32:0/2640191379 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8241599e0 0x7fc8240aece0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc8200097a0 tx=0x7fc82001e5b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=18a3174e9412431d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:04.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.713+0000 7fc81bfff640 1 -- 172.21.15.32:0/2640191379 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200b2600 con 0x7fc8241599e0 2023-12-20T06:02:04.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.713+0000 7fc81bfff640 1 -- 172.21.15.32:0/2640191379 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8200b2760 con 0x7fc8241599e0 2023-12-20T06:02:04.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.713+0000 7fc81bfff640 1 -- 172.21.15.32:0/2640191379 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200b2af0 con 0x7fc8241599e0 2023-12-20T06:02:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 -- 172.21.15.32:0/2640191379 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8241599e0 msgr2=0x7fc8240aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 --2- 172.21.15.32:0/2640191379 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8241599e0 0x7fc8240aece0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7fc8200097a0 tx=0x7fc82001e5b0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 -- 172.21.15.32:0/2640191379 shutdown_connections 2023-12-20T06:02:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 --2- 172.21.15.32:0/2640191379 >> v2:172.21.15.32:3300/0 conn(0x7fc8240b0d00 0x7fc8240701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 --2- 172.21.15.32:0/2640191379 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8241599e0 0x7fc8240aece0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 -- 172.21.15.32:0/2640191379 >> 172.21.15.32:0/2640191379 conn(0x7fc82406a320 msgr2=0x7fc8240ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 -- 172.21.15.32:0/2640191379 shutdown_connections 2023-12-20T06:02:04.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 -- 172.21.15.32:0/2640191379 wait complete. 2023-12-20T06:02:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 Processor -- start 2023-12-20T06:02:04.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 -- start start 2023-12-20T06:02:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8240b0d00 0x7fc8240aec60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8240c90e0 0x7fc82414b990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8241599e0 0x7fc824158020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc82414bed0 con 0x7fc8240b0d00 2023-12-20T06:02:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8241588e0 con 0x7fc8240c90e0 2023-12-20T06:02:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc82b812640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8240c58d0 con 0x7fc8241599e0 2023-12-20T06:02:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc8295ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8240b0d00 0x7fc8240aec60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc8295ae640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8240b0d00 0x7fc8240aec60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50348/0 (socket says 172.21.15.32:50348) 2023-12-20T06:02:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc8295ae640 1 -- 172.21.15.32:0/588808966 learned_addr learned my addr 172.21.15.32:0/588808966 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:04.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc829daf640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8241599e0 0x7fc824158020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc828dad640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8240c90e0 0x7fc82414b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc8295ae640 1 -- 172.21.15.32:0/588808966 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8241599e0 msgr2=0x7fc824158020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc8295ae640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8241599e0 0x7fc824158020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc8295ae640 1 -- 172.21.15.32:0/588808966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8240c90e0 msgr2=0x7fc82414b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc8295ae640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8240c90e0 0x7fc82414b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.717+0000 7fc8295ae640 1 -- 172.21.15.32:0/588808966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8200093b0 con 0x7fc8240b0d00 2023-12-20T06:02:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc8295ae640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8240b0d00 0x7fc8240aec60 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fc82001cc60 tx=0x7fc82001b350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc819ffb640 1 -- 172.21.15.32:0/588808966 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200b2ae0 con 0x7fc8240b0d00 2023-12-20T06:02:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc819ffb640 1 -- 172.21.15.32:0/588808966 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8200b2c60 con 0x7fc8240b0d00 2023-12-20T06:02:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc819ffb640 1 -- 172.21.15.32:0/588808966 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8200059a0 con 0x7fc8240b0d00 2023-12-20T06:02:04.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc82b812640 1 -- 172.21.15.32:0/588808966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8240c5bb0 con 0x7fc8240b0d00 2023-12-20T06:02:04.723 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:04.721+0000 7f90acc7d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:02:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc82b812640 1 -- 172.21.15.32:0/588808966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc824158bc0 con 0x7fc8240b0d00 2023-12-20T06:02:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc82b812640 1 -- 172.21.15.32:0/588808966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc82404e000 con 0x7fc8240b0d00 2023-12-20T06:02:04.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc819ffb640 1 -- 172.21.15.32:0/588808966 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 120) v1 ==== 85444+0+0 (secure 0 0 0) 0x7fc820005b20 con 0x7fc8240b0d00 2023-12-20T06:02:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc819ffb640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc808065ca0 0x7fc808068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc819ffb640 1 -- 172.21.15.32:0/588808966 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(194..194 src has 1..194) v4 ==== 6197+0+0 (secure 0 0 0) 0x7fc82011d910 con 0x7fc8240b0d00 2023-12-20T06:02:04.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc828dad640 1 -- 172.21.15.32:0/588808966 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc808065ca0 msgr2=0x7fc808068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:04.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.721+0000 7fc828dad640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc808065ca0 0x7fc808068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:04.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.729+0000 7fc819ffb640 1 -- 172.21.15.32:0/588808966 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc8200ead20 con 0x7fc8240b0d00 2023-12-20T06:02:04.735 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:04.732+0000 7fe542b5c180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:02:04.784 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:04.781+0000 7f90acc7d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:02:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 -- 172.21.15.32:0/588808966 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7fc8240cf550 con 0x7fc8240b0d00 2023-12-20T06:02:04.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc819ffb640 1 -- 172.21.15.32:0/588808966 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v195) v1 ==== 80+0+460 (secure 0 0 0) 0x7fc8200ee340 con 0x7fc8240b0d00 2023-12-20T06:02:04.913 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3239744916 2023-12-21T06:00:49.693740+0000 2023-12-20T06:02:04.913 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2437361294 2023-12-21T06:00:49.693740+0000 2023-12-20T06:02:04.913 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2674032592 2023-12-21T06:00:49.693740+0000 2023-12-20T06:02:04.913 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/4037417795 2023-12-21T06:00:49.693740+0000 2023-12-20T06:02:04.913 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/64546 2023-12-21T06:00:49.693740+0000 2023-12-20T06:02:04.913 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/64546 2023-12-21T06:00:49.693740+0000 2023-12-20T06:02:04.913 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/4289008383 2023-12-21T06:00:49.693740+0000 2023-12-20T06:02:04.913 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1643902464 2023-12-21T06:00:49.693740+0000 2023-12-20T06:02:04.914 INFO:teuthology.orchestra.run.smithi032.stderr:listed 8 entries 2023-12-20T06:02:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 -- 172.21.15.32:0/588808966 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc808065ca0 msgr2=0x7fc808068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc808065ca0 0x7fc808068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 -- 172.21.15.32:0/588808966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8240b0d00 msgr2=0x7fc8240aec60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8240b0d00 0x7fc8240aec60 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7fc82001cc60 tx=0x7fc82001b350 comp rx=0 tx=0).stop 2023-12-20T06:02:04.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 -- 172.21.15.32:0/588808966 shutdown_connections 2023-12-20T06:02:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc808065ca0 0x7fc808068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8241599e0 0x7fc824158020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8240c90e0 0x7fc82414b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 --2- 172.21.15.32:0/588808966 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8240b0d00 0x7fc8240aec60 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 -- 172.21.15.32:0/588808966 >> 172.21.15.32:0/588808966 conn(0x7fc82406a320 msgr2=0x7fc8240cd9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 -- 172.21.15.32:0/588808966 shutdown_connections 2023-12-20T06:02:04.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:04.913+0000 7fc82b812640 1 -- 172.21.15.32:0/588808966 wait complete. 2023-12-20T06:02:04.935 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T06:02:04.935 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:04.935 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T06:02:04.989 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T06:02:04.989 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:04.989 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T06:02:04.994 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:02:04.994 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:02:04.994 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:02:05.003 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:05.000+0000 7fe542b5c180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:02:05.008 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:02:05.008 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:05.018 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:02:05.018 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:02:05.018 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:05.056 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:02:05.056 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:02:05.056 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:02:05.061 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:02:05.061 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:02:05.061 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:05.066 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:05.065+0000 7f90acc7d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:02:05.071 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:02:05.071 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:02:05.145 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:05.140+0000 7fe542b5c180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:05.212 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:05.209+0000 7f90acc7d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f35280fc640 1 Processor -- start 2023-12-20T06:02:05.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f35280fc640 1 -- start start 2023-12-20T06:02:05.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f35280fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f35200af5e0 0x7f35200af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f35280fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f352015f110 0x7f352015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f35280fc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f352015b990 con 0x7f352006ce60 2023-12-20T06:02:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f35280fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f352015bbd0 con 0x7f35200af5e0 2023-12-20T06:02:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f35280fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352015beb0 con 0x7f352015f110 2023-12-20T06:02:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f3525e98640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f35200af5e0 0x7f35200af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f3525e98640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f35200af5e0 0x7f35200af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:39926/0 (socket says 172.21.15.32:39926) 2023-12-20T06:02:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f3525e98640 1 -- 172.21.15.32:0/3649532394 learned_addr learned my addr 172.21.15.32:0/3649532394 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f3525697640 1 --1- 172.21.15.32:0/3649532394 >> v1:172.21.15.39:6789/0 conn(0x7f352006ce60 0x7f352015b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:45616/0 (socket says 172.21.15.32:45616) 2023-12-20T06:02:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.217+0000 7f3526699640 1 --2- 172.21.15.32:0/3649532394 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f352015f110 0x7f352015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.221+0000 7f3525e98640 1 -- 172.21.15.32:0/3649532394 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f352015f110 msgr2=0x7f352015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.221+0000 7f3525e98640 1 --2- 172.21.15.32:0/3649532394 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f352015f110 0x7f352015c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.221+0000 7f3525e98640 1 -- 172.21.15.32:0/3649532394 >> v1:172.21.15.39:6789/0 conn(0x7f352006ce60 legacy=0x7f352015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.221+0000 7f3525e98640 1 -- 172.21.15.32:0/3649532394 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f352015e910 con 0x7f35200af5e0 2023-12-20T06:02:05.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.221+0000 7f3525e98640 1 --2- 172.21.15.32:0/3649532394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35200af5e0 0x7f35200af9c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f351c006fd0 tx=0x7f351c01c3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=504673ddc1a9711d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.221+0000 7f3524e96640 1 -- 172.21.15.32:0/3649532394 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c0ac640 con 0x7f35200af5e0 2023-12-20T06:02:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.221+0000 7f3524e96640 1 -- 172.21.15.32:0/3649532394 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c0ac7a0 con 0x7f35200af5e0 2023-12-20T06:02:05.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.221+0000 7f3524e96640 1 -- 172.21.15.32:0/3649532394 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c0acb30 con 0x7f35200af5e0 2023-12-20T06:02:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 -- 172.21.15.32:0/3649532394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35200af5e0 msgr2=0x7f35200af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 --2- 172.21.15.32:0/3649532394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35200af5e0 0x7f35200af9c0 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f351c006fd0 tx=0x7f351c01c3b0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 -- 172.21.15.32:0/3649532394 shutdown_connections 2023-12-20T06:02:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 --2- 172.21.15.32:0/3649532394 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f352015f110 0x7f352015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 --2- 172.21.15.32:0/3649532394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f35200af5e0 0x7f35200af9c0 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 -- 172.21.15.32:0/3649532394 >> 172.21.15.32:0/3649532394 conn(0x7f352006a320 msgr2=0x7f35200ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:05.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 -- 172.21.15.32:0/3649532394 shutdown_connections 2023-12-20T06:02:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 -- 172.21.15.32:0/3649532394 wait complete. 2023-12-20T06:02:05.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 Processor -- start 2023-12-20T06:02:05.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 -- start start 2023-12-20T06:02:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f352006ce60 0x7f3520153210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35200af5e0 0x7f3520159ed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352015f110 0x7f3520150b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3520152d80 con 0x7f352006ce60 2023-12-20T06:02:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f352015d6d0 con 0x7f352015f110 2023-12-20T06:02:05.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f35280fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f352015d9b0 con 0x7f35200af5e0 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f3525e98640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f352006ce60 0x7f3520153210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f3525e98640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f352006ce60 0x7f3520153210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50362/0 (socket says 172.21.15.32:50362) 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f3525e98640 1 -- 172.21.15.32:0/253236444 learned_addr learned my addr 172.21.15.32:0/253236444 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f3526699640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352015f110 0x7f3520150b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.225+0000 7f3525697640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35200af5e0 0x7f3520159ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f3525e98640 1 -- 172.21.15.32:0/253236444 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35200af5e0 msgr2=0x7f3520159ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f3525e98640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35200af5e0 0x7f3520159ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f3525e98640 1 -- 172.21.15.32:0/253236444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352015f110 msgr2=0x7f3520150b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f3525e98640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352015f110 0x7f3520150b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f3525e98640 1 -- 172.21.15.32:0/253236444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f351c009480 con 0x7f352006ce60 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f3525e98640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f352006ce60 0x7f3520153210 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f351c01aa60 tx=0x7f351c019150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:05.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f35167fc640 1 -- 172.21.15.32:0/253236444 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c003820 con 0x7f352006ce60 2023-12-20T06:02:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f35167fc640 1 -- 172.21.15.32:0/253236444 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f351c003980 con 0x7f352006ce60 2023-12-20T06:02:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f35280fc640 1 -- 172.21.15.32:0/253236444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3520152730 con 0x7f352006ce60 2023-12-20T06:02:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f35167fc640 1 -- 172.21.15.32:0/253236444 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f351c003d30 con 0x7f352006ce60 2023-12-20T06:02:05.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f35280fc640 1 -- 172.21.15.32:0/253236444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3520153750 con 0x7f352006ce60 2023-12-20T06:02:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f35280fc640 1 -- 172.21.15.32:0/253236444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f352004e000 con 0x7f352006ce60 2023-12-20T06:02:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f35167fc640 1 -- 172.21.15.32:0/253236444 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 120) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f351c0b3070 con 0x7f352006ce60 2023-12-20T06:02:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f35167fc640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3504065ca0 0x7f3504068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f35167fc640 1 -- 172.21.15.32:0/253236444 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6181+0+0 (secure 0 0 0) 0x7f351c11dc40 con 0x7f352006ce60 2023-12-20T06:02:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f3525697640 1 -- 172.21.15.32:0/253236444 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3504065ca0 msgr2=0x7f3504068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:05.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.229+0000 7f3525697640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3504065ca0 0x7f3504068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:05.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.233+0000 7f35167fc640 1 -- 172.21.15.32:0/253236444 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f351c0eb0d0 con 0x7f352006ce60 2023-12-20T06:02:05.279 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:05.276+0000 7fe542b5c180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:05.348 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:05.349+0000 7f90acc7d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:05.353 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:05.348+0000 7fe542b5c180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:02:05.424 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:05.421+0000 7f90acc7d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:02:05.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.425+0000 7f35280fc640 1 -- 172.21.15.32:0/253236444 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f352004e000 con 0x7f352006ce60 2023-12-20T06:02:05.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.425+0000 7f35167fc640 1 -- 172.21.15.32:0/253236444 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f351c0b1070 con 0x7f352006ce60 2023-12-20T06:02:05.427 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:05.427 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8396,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1626919510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1626919510},{"type":"v1","addr":"172.21.15.39:6833","nonce":1626919510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3001408763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3001408763},{"type":"v1","addr":"172.21.15.39:6835","nonce":3001408763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":10020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/2908145202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2908145202},{"type":"v1","addr":"172.21.15.32:6835","nonce":2908145202}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:00:37.604080+0000","modified":"2023-12-20T06:00:40.642479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10026":{"gid":10026,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2253884246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2253884246},{"type":"v1","addr":"172.21.15.32:6837","nonce":2253884246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-20T06:02:05.427 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 87 2023-12-20T06:02:05.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 -- 172.21.15.32:0/253236444 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3504065ca0 msgr2=0x7f3504068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:05.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3504065ca0 0x7f3504068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 -- 172.21.15.32:0/253236444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f352006ce60 msgr2=0x7f3520153210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f352006ce60 0x7f3520153210 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f351c01aa60 tx=0x7f351c019150 comp rx=0 tx=0).stop 2023-12-20T06:02:05.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 -- 172.21.15.32:0/253236444 shutdown_connections 2023-12-20T06:02:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3504065ca0 0x7f3504068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f352015f110 0x7f3520150b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35200af5e0 0x7f3520159ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 --2- 172.21.15.32:0/253236444 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f352006ce60 0x7f3520153210 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 -- 172.21.15.32:0/253236444 >> 172.21.15.32:0/253236444 conn(0x7f352006a320 msgr2=0x7f35200ccf50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 -- 172.21.15.32:0/253236444 shutdown_connections 2023-12-20T06:02:05.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.429+0000 7f35280fc640 1 -- 172.21.15.32:0/253236444 wait complete. 2023-12-20T06:02:05.450 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:05.496 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:05.492+0000 7fe542b5c180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:05.572 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:05.569+0000 7f90acc7d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.593+0000 7f4a472f9640 1 Processor -- start 2023-12-20T06:02:05.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.593+0000 7f4a472f9640 1 -- start start 2023-12-20T06:02:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.593+0000 7f4a472f9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a400ae010 0x7f4a40071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.593+0000 7f4a472f9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4a4006cf10 0x7f4a400719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.593+0000 7f4a472f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4a400ae3f0 con 0x7f4a40164ef0 2023-12-20T06:02:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.593+0000 7f4a472f9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a400ca4b0 con 0x7f4a400ae010 2023-12-20T06:02:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.593+0000 7f4a472f9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4a40161800 con 0x7f4a4006cf10 2023-12-20T06:02:05.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a45896640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4a40164ef0 0x7f4a400701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39092/0 (socket says 172.21.15.32:39092) 2023-12-20T06:02:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a45095640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4a400ae010 0x7f4a40071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a45896640 1 -- 172.21.15.32:0/810699440 learned_addr learned my addr 172.21.15.32:0/810699440 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a44894640 1 --2- 172.21.15.32:0/810699440 >> v2:172.21.15.39:3300/0 conn(0x7f4a4006cf10 0x7f4a400719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a45095640 1 -- 172.21.15.32:0/810699440 >> v2:172.21.15.39:3300/0 conn(0x7f4a4006cf10 msgr2=0x7f4a400719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a45095640 1 --2- 172.21.15.32:0/810699440 >> v2:172.21.15.39:3300/0 conn(0x7f4a4006cf10 0x7f4a400719b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a45095640 1 -- 172.21.15.32:0/810699440 >> v1:172.21.15.32:6789/0 conn(0x7f4a40164ef0 legacy=0x7f4a400701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a45095640 1 -- 172.21.15.32:0/810699440 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a40165380 con 0x7f4a400ae010 2023-12-20T06:02:05.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a37fff640 1 -- 172.21.15.32:0/810699440 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1636613278 0 0) 0x7f4a400ae3f0 con 0x7f4a40164ef0 2023-12-20T06:02:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a45095640 1 --2- 172.21.15.32:0/810699440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a400ae010 0x7f4a40071470 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f4a40164e50 tx=0x7f4a3c01c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3afc25b79430cefe server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a37fff640 1 -- 172.21.15.32:0/810699440 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0ad680 con 0x7f4a400ae010 2023-12-20T06:02:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a37fff640 1 -- 172.21.15.32:0/810699440 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c0ad7e0 con 0x7f4a400ae010 2023-12-20T06:02:05.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.597+0000 7f4a37fff640 1 -- 172.21.15.32:0/810699440 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0adb70 con 0x7f4a400ae010 2023-12-20T06:02:05.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 -- 172.21.15.32:0/810699440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a400ae010 msgr2=0x7f4a40071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 --2- 172.21.15.32:0/810699440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a400ae010 0x7f4a40071470 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7f4a40164e50 tx=0x7f4a3c01c380 comp rx=0 tx=0).stop 2023-12-20T06:02:05.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 -- 172.21.15.32:0/810699440 shutdown_connections 2023-12-20T06:02:05.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 --2- 172.21.15.32:0/810699440 >> v2:172.21.15.39:3300/0 conn(0x7f4a4006cf10 0x7f4a400719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 --2- 172.21.15.32:0/810699440 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a400ae010 0x7f4a40071470 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 -- 172.21.15.32:0/810699440 >> 172.21.15.32:0/810699440 conn(0x7f4a4006a320 msgr2=0x7f4a400ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:05.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 -- 172.21.15.32:0/810699440 shutdown_connections 2023-12-20T06:02:05.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 -- 172.21.15.32:0/810699440 wait complete. 2023-12-20T06:02:05.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 Processor -- start 2023-12-20T06:02:05.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 -- start start 2023-12-20T06:02:05.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a400ae010 0x7f4a401633e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a400c45d0 0x7f4a400c4a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a400c2340 0x7f4a400c27c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a400c4f90 con 0x7f4a400c2340 2023-12-20T06:02:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4a400c30b0 con 0x7f4a400ae010 2023-12-20T06:02:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a472f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4a400c3390 con 0x7f4a400c45d0 2023-12-20T06:02:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.601+0000 7f4a45896640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a400c2340 0x7f4a400c27c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a45095640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a400ae010 0x7f4a401633e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a45095640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a400ae010 0x7f4a401633e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:39952/0 (socket says 172.21.15.32:39952) 2023-12-20T06:02:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a45095640 1 -- 172.21.15.32:0/3916557519 learned_addr learned my addr 172.21.15.32:0/3916557519 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:05.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a44894640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a400c45d0 0x7f4a400c4a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a45095640 1 -- 172.21.15.32:0/3916557519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a400c45d0 msgr2=0x7f4a400c4a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a45095640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a400c45d0 0x7f4a400c4a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a45095640 1 -- 172.21.15.32:0/3916557519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a400c2340 msgr2=0x7f4a400c27c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a45095640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a400c2340 0x7f4a400c27c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a45095640 1 -- 172.21.15.32:0/3916557519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4a3c0093b0 con 0x7f4a400ae010 2023-12-20T06:02:05.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a45896640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a400c2340 0x7f4a400c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:05.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a45095640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a400ae010 0x7f4a401633e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c000c10 tx=0x7f4a3c019120 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:05.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a35ffb640 1 -- 172.21.15.32:0/3916557519 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c005db0 con 0x7f4a400ae010 2023-12-20T06:02:05.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a35ffb640 1 -- 172.21.15.32:0/3916557519 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4a3c0ad950 con 0x7f4a400ae010 2023-12-20T06:02:05.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a35ffb640 1 -- 172.21.15.32:0/3916557519 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4a3c0adcc0 con 0x7f4a400ae010 2023-12-20T06:02:05.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a472f9640 1 -- 172.21.15.32:0/3916557519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4a400c3670 con 0x7f4a400ae010 2023-12-20T06:02:05.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a472f9640 1 -- 172.21.15.32:0/3916557519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4a400c3c30 con 0x7f4a400ae010 2023-12-20T06:02:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a472f9640 1 -- 172.21.15.32:0/3916557519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4a400c3e50 con 0x7f4a400ae010 2023-12-20T06:02:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.605+0000 7f4a35ffb640 1 -- 172.21.15.32:0/3916557519 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 120) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f4a3c0b4040 con 0x7f4a400ae010 2023-12-20T06:02:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.609+0000 7f4a35ffb640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4a24065ca0 0x7f4a24068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.609+0000 7f4a35ffb640 1 -- 172.21.15.32:0/3916557519 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6181+0+0 (secure 0 0 0) 0x7f4a3c11d990 con 0x7f4a400ae010 2023-12-20T06:02:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.609+0000 7f4a44894640 1 -- 172.21.15.32:0/3916557519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4a24065ca0 msgr2=0x7f4a24068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:05.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.609+0000 7f4a44894640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4a24065ca0 0x7f4a24068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:05.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.613+0000 7f4a35ffb640 1 -- 172.21.15.32:0/3916557519 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4a3c0ead10 con 0x7f4a400ae010 2023-12-20T06:02:05.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.777+0000 7f4a472f9640 1 -- 172.21.15.32:0/3916557519 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f4a4004e000 con 0x7f4a400ae010 2023-12-20T06:02:05.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.777+0000 7f4a35ffb640 1 -- 172.21.15.32:0/3916557519 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v195) v1 ==== 74+0+16771 (secure 0 0 0) 0x7f4a3c0ba090 con 0x7f4a400ae010 2023-12-20T06:02:05.779 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:05.779 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":195,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:04.803186+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:00:36.162516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-20T06:00:36.593153+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":195,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3239744916":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/2437361294":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/2674032592":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/4037417795":"2023-12-21T06:00:49.693740+0000","172.21.15.32:6833/64546":"2023-12-21T06:00:49.693740+0000","172.21.15.32:6832/64546":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/4289008383":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/1643902464":"2023-12-21T06:00:49.693740+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[{"pool":23,"snaps":[{"begin":3,"length":1},{"begin":5,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:05.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 -- 172.21.15.32:0/3916557519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4a24065ca0 msgr2=0x7f4a24068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4a24065ca0 0x7f4a24068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 -- 172.21.15.32:0/3916557519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a400ae010 msgr2=0x7f4a401633e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a400ae010 0x7f4a401633e0 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f4a3c000c10 tx=0x7f4a3c019120 comp rx=0 tx=0).stop 2023-12-20T06:02:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 -- 172.21.15.32:0/3916557519 shutdown_connections 2023-12-20T06:02:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4a24065ca0 0x7f4a24068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4a400c2340 0x7f4a400c27c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4a400c45d0 0x7f4a400c4a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 --2- 172.21.15.32:0/3916557519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4a400ae010 0x7f4a401633e0 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 -- 172.21.15.32:0/3916557519 >> 172.21.15.32:0/3916557519 conn(0x7f4a4006a320 msgr2=0x7f4a4015d7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 -- 172.21.15.32:0/3916557519 shutdown_connections 2023-12-20T06:02:05.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.781+0000 7f4a472f9640 1 -- 172.21.15.32:0/3916557519 wait complete. 2023-12-20T06:02:05.801 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 22, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T06:00:36.162516+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '191', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 2, 'score_stable': 2, 'optimal_score': 1, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 23, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T06:00:36.593153+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '195', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 5, 'snap_epoch': 195, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:02:05.801 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:02:05.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.953+0000 7f6b7d3a6640 1 Processor -- start 2023-12-20T06:02:05.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.953+0000 7f6b7d3a6640 1 -- start start 2023-12-20T06:02:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.953+0000 7f6b7d3a6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b7815ce60 0x7f6b780d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.953+0000 7f6b7d3a6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b780d2650 con 0x7f6b780d14d0 2023-12-20T06:02:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.953+0000 7f6b7d3a6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b780d3570 con 0x7f6b780d3130 2023-12-20T06:02:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.953+0000 7f6b7d3a6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b780d1a40 con 0x7f6b7815ce60 2023-12-20T06:02:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.953+0000 7f6b777fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6b780d14d0 0x7f6b780b5e90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39096/0 (socket says 172.21.15.32:39096) 2023-12-20T06:02:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.953+0000 7f6b777fe640 1 -- 172.21.15.32:0/3705341843 learned_addr learned my addr 172.21.15.32:0/3705341843 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:05.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.953+0000 7f6b76ffd640 1 --2- 172.21.15.32:0/3705341843 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b7815ce60 0x7f6b780d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.957+0000 7f6b767fc640 1 -- 172.21.15.32:0/3705341843 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 224317232 0 0) 0x7f6b780d2650 con 0x7f6b780d14d0 2023-12-20T06:02:05.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.957+0000 7f6b767fc640 1 -- 172.21.15.32:0/3705341843 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b64003530 con 0x7f6b780d14d0 2023-12-20T06:02:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.957+0000 7f6b767fc640 1 -- 172.21.15.32:0/3705341843 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4181346500 0 0) 0x7f6b780d3570 con 0x7f6b780d3130 2023-12-20T06:02:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.957+0000 7f6b767fc640 1 -- 172.21.15.32:0/3705341843 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b780d2650 con 0x7f6b780d3130 2023-12-20T06:02:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.957+0000 7f6b76ffd640 1 -- 172.21.15.32:0/3705341843 >> v1:172.21.15.39:6789/0 conn(0x7f6b780d3130 legacy=0x7f6b780d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.957+0000 7f6b76ffd640 1 -- 172.21.15.32:0/3705341843 >> v1:172.21.15.32:6789/0 conn(0x7f6b780d14d0 legacy=0x7f6b780b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.957+0000 7f6b76ffd640 1 -- 172.21.15.32:0/3705341843 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b780d1d20 con 0x7f6b7815ce60 2023-12-20T06:02:05.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.957+0000 7f6b76ffd640 1 --2- 172.21.15.32:0/3705341843 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b7815ce60 0x7f6b780d2bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6b680097f0 tx=0x7f6b6801ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8a6f057bd3e821bd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:05.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.957+0000 7f6b767fc640 1 -- 172.21.15.32:0/3705341843 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680a89b0 con 0x7f6b7815ce60 2023-12-20T06:02:05.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.957+0000 7f6b767fc640 1 -- 172.21.15.32:0/3705341843 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b680a8b10 con 0x7f6b7815ce60 2023-12-20T06:02:05.960 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.957+0000 7f6b767fc640 1 -- 172.21.15.32:0/3705341843 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680a8ea0 con 0x7f6b7815ce60 2023-12-20T06:02:05.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/3705341843 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b7815ce60 msgr2=0x7f6b780d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 --2- 172.21.15.32:0/3705341843 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b7815ce60 0x7f6b780d2bf0 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f6b680097f0 tx=0x7f6b6801ecb0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/3705341843 shutdown_connections 2023-12-20T06:02:05.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 --2- 172.21.15.32:0/3705341843 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b7815ce60 0x7f6b780d2bf0 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/3705341843 >> 172.21.15.32:0/3705341843 conn(0x7f6b7806b5e0 msgr2=0x7f6b7806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:05.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/3705341843 shutdown_connections 2023-12-20T06:02:05.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/3705341843 wait complete. 2023-12-20T06:02:05.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 Processor -- start 2023-12-20T06:02:05.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 -- start start 2023-12-20T06:02:05.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b780d14d0 0x7f6b78152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d3130 0x7f6b78155690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b7815ce60 0x7f6b78154510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b781e6be0 con 0x7f6b780d3130 2023-12-20T06:02:05.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b78152ef0 con 0x7f6b780d14d0 2023-12-20T06:02:05.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b7d3a6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b781e6520 con 0x7f6b7815ce60 2023-12-20T06:02:05.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d3130 0x7f6b78155690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b76ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b780d14d0 0x7f6b78152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b6ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d3130 0x7f6b78155690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50388/0 (socket says 172.21.15.32:50388) 2023-12-20T06:02:05.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b76ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b780d14d0 0x7f6b78152910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:39960/0 (socket says 172.21.15.32:39960) 2023-12-20T06:02:05.965 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b76ffd640 1 -- 172.21.15.32:0/1641802665 learned_addr learned my addr 172.21.15.32:0/1641802665 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.961+0000 7f6b777fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b7815ce60 0x7f6b78154510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b76ffd640 1 -- 172.21.15.32:0/1641802665 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b7815ce60 msgr2=0x7f6b78154510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b76ffd640 1 --2- 172.21.15.32:0/1641802665 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b7815ce60 0x7f6b78154510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b76ffd640 1 -- 172.21.15.32:0/1641802665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d3130 msgr2=0x7f6b78155690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b76ffd640 1 --2- 172.21.15.32:0/1641802665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d3130 0x7f6b78155690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b76ffd640 1 -- 172.21.15.32:0/1641802665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b68009420 con 0x7f6b780d14d0 2023-12-20T06:02:05.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b76ffd640 1 --2- 172.21.15.32:0/1641802665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b780d14d0 0x7f6b78152910 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6b6801b8c0 tx=0x7f6b6801ba50 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:05.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b6f7fe640 1 -- 172.21.15.32:0/1641802665 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680b0d80 con 0x7f6b780d14d0 2023-12-20T06:02:05.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b6f7fe640 1 -- 172.21.15.32:0/1641802665 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b680b9430 con 0x7f6b780d14d0 2023-12-20T06:02:05.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b6f7fe640 1 -- 172.21.15.32:0/1641802665 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b680b97e0 con 0x7f6b780d14d0 2023-12-20T06:02:05.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/1641802665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b781e6800 con 0x7f6b780d14d0 2023-12-20T06:02:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/1641802665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b78155bd0 con 0x7f6b780d14d0 2023-12-20T06:02:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/1641802665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b781e6d20 con 0x7f6b780d14d0 2023-12-20T06:02:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b6f7fe640 1 -- 172.21.15.32:0/1641802665 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 120) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f6b680b7040 con 0x7f6b780d14d0 2023-12-20T06:02:05.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b6f7fe640 1 --2- 172.21.15.32:0/1641802665 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b58065ca0 0x7f6b58068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:05.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b6f7fe640 1 -- 172.21.15.32:0/1641802665 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6181+0+0 (secure 0 0 0) 0x7f6b68121120 con 0x7f6b780d14d0 2023-12-20T06:02:05.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b6ffff640 1 -- 172.21.15.32:0/1641802665 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b58065ca0 msgr2=0x7f6b58068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:05.969 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.965+0000 7f6b6ffff640 1 --2- 172.21.15.32:0/1641802665 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b58065ca0 0x7f6b58068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:05.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:05.969+0000 7f6b6f7fe640 1 -- 172.21.15.32:0/1641802665 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6b680ee640 con 0x7f6b780d14d0 2023-12-20T06:02:06.075 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:06.072+0000 7fe542b5c180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:02:06.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.129+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/1641802665 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6b7804e000 con 0x7f6b780d14d0 2023-12-20T06:02:06.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b6f7fe640 1 -- 172.21.15.32:0/1641802665 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v195) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6b680f1c60 con 0x7f6b780d14d0 2023-12-20T06:02:06.133 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:02:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/1641802665 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b58065ca0 msgr2=0x7f6b58068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 --2- 172.21.15.32:0/1641802665 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b58065ca0 0x7f6b58068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/1641802665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b780d14d0 msgr2=0x7f6b78152910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 --2- 172.21.15.32:0/1641802665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b780d14d0 0x7f6b78152910 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f6b6801b8c0 tx=0x7f6b6801ba50 comp rx=0 tx=0).stop 2023-12-20T06:02:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/1641802665 shutdown_connections 2023-12-20T06:02:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 --2- 172.21.15.32:0/1641802665 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b58065ca0 0x7f6b58068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 --2- 172.21.15.32:0/1641802665 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b7815ce60 0x7f6b78154510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 --2- 172.21.15.32:0/1641802665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b780d3130 0x7f6b78155690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 --2- 172.21.15.32:0/1641802665 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b780d14d0 0x7f6b78152910 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/1641802665 >> 172.21.15.32:0/1641802665 conn(0x7f6b7806b5e0 msgr2=0x7f6b78151fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/1641802665 shutdown_connections 2023-12-20T06:02:06.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.133+0000 7f6b7d3a6640 1 -- 172.21.15.32:0/1641802665 wait complete. 2023-12-20T06:02:06.155 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:02:06.163 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:06.160+0000 7fe542b5c180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:02:06.164 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.161+0000 7f90acc7d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:02:06.254 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.253+0000 7f90acc7d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:02:06.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.301+0000 7faabcd5a640 1 Processor -- start 2023-12-20T06:02:06.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.301+0000 7faabcd5a640 1 -- start start 2023-12-20T06:02:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faabcd5a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faab815ccf0 0x7faab80cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faabcd5a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faab80cd7b0 0x7faab80cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faabcd5a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faab80cfd80 con 0x7faab80cf920 2023-12-20T06:02:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faabcd5a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faab80ce2f0 con 0x7faab815ccf0 2023-12-20T06:02:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faabcd5a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab80d0b70 con 0x7faab80cd7b0 2023-12-20T06:02:06.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faab77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faab815ccf0 0x7faab80cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faab77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7faab815ccf0 0x7faab80cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50400/0 (socket says 172.21.15.32:50400) 2023-12-20T06:02:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faab77fe640 1 -- 172.21.15.32:0/4013665005 learned_addr learned my addr 172.21.15.32:0/4013665005 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faab6ffd640 1 --2- 172.21.15.32:0/4013665005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faab80cd7b0 0x7faab80cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faab77fe640 1 -- 172.21.15.32:0/4013665005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faab80cd7b0 msgr2=0x7faab80cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faab77fe640 1 --2- 172.21.15.32:0/4013665005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faab80cd7b0 0x7faab80cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faab77fe640 1 -- 172.21.15.32:0/4013665005 >> v1:172.21.15.39:6789/0 conn(0x7faab80cf920 legacy=0x7faab80b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:06.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.305+0000 7faab77fe640 1 -- 172.21.15.32:0/4013665005 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faab80d10f0 con 0x7faab815ccf0 2023-12-20T06:02:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faab77fe640 1 --2- 172.21.15.32:0/4013665005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab815ccf0 0x7faab80cd270 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009870 tx=0x7faaa801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5608138d030d13 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faab67fc640 1 -- 172.21.15.32:0/4013665005 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa80ac600 con 0x7faab815ccf0 2023-12-20T06:02:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faab67fc640 1 -- 172.21.15.32:0/4013665005 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa80a8050 con 0x7faab815ccf0 2023-12-20T06:02:06.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faab67fc640 1 -- 172.21.15.32:0/4013665005 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa80ac980 con 0x7faab815ccf0 2023-12-20T06:02:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faabcd5a640 1 -- 172.21.15.32:0/4013665005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab815ccf0 msgr2=0x7faab80cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:06.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faabcd5a640 1 --2- 172.21.15.32:0/4013665005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab815ccf0 0x7faab80cd270 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7faaa8009870 tx=0x7faaa801c480 comp rx=0 tx=0).stop 2023-12-20T06:02:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faabcd5a640 1 -- 172.21.15.32:0/4013665005 shutdown_connections 2023-12-20T06:02:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faabcd5a640 1 --2- 172.21.15.32:0/4013665005 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faab80cd7b0 0x7faab80cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faabcd5a640 1 --2- 172.21.15.32:0/4013665005 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab815ccf0 0x7faab80cd270 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faabcd5a640 1 -- 172.21.15.32:0/4013665005 >> 172.21.15.32:0/4013665005 conn(0x7faab80658e0 msgr2=0x7faab8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faabcd5a640 1 -- 172.21.15.32:0/4013665005 shutdown_connections 2023-12-20T06:02:06.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faabcd5a640 1 -- 172.21.15.32:0/4013665005 wait complete. 2023-12-20T06:02:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faabcd5a640 1 Processor -- start 2023-12-20T06:02:06.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.309+0000 7faabcd5a640 1 -- start start 2023-12-20T06:02:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faabcd5a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab80cd7b0 0x7faab80b75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faabcd5a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faab80cf920 0x7faab80c8580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faabcd5a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faab815ccf0 0x7faab8153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faabcd5a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab80c8f80 con 0x7faab80cd7b0 2023-12-20T06:02:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faabcd5a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faab80c9260 con 0x7faab80cf920 2023-12-20T06:02:06.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faabcd5a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faab80c6540 con 0x7faab815ccf0 2023-12-20T06:02:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faab77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab80cd7b0 0x7faab80b75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faab77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab80cd7b0 0x7faab80b75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50414/0 (socket says 172.21.15.32:50414) 2023-12-20T06:02:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faab77fe640 1 -- 172.21.15.32:0/2665267887 learned_addr learned my addr 172.21.15.32:0/2665267887 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faab6ffd640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faab80cf920 0x7faab80c8580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faab7fff640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faab815ccf0 0x7faab8153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faab77fe640 1 -- 172.21.15.32:0/2665267887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faab815ccf0 msgr2=0x7faab8153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faab77fe640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faab815ccf0 0x7faab8153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faab77fe640 1 -- 172.21.15.32:0/2665267887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faab80cf920 msgr2=0x7faab80c8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faab77fe640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faab80cf920 0x7faab80c8580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faab77fe640 1 -- 172.21.15.32:0/2665267887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faaa80094d0 con 0x7faab80cd7b0 2023-12-20T06:02:06.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faab77fe640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab80cd7b0 0x7faab80b75d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7faaa801ab30 tx=0x7faaa801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faa9ffff640 1 -- 172.21.15.32:0/2665267887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa80b7620 con 0x7faab80cd7b0 2023-12-20T06:02:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faa9ffff640 1 -- 172.21.15.32:0/2665267887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faaa80ace90 con 0x7faab80cd7b0 2023-12-20T06:02:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faabcd5a640 1 -- 172.21.15.32:0/2665267887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faab80c6820 con 0x7faab80cd7b0 2023-12-20T06:02:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faa9ffff640 1 -- 172.21.15.32:0/2665267887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faaa80b79e0 con 0x7faab80cd7b0 2023-12-20T06:02:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faabcd5a640 1 -- 172.21.15.32:0/2665267887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faab8148600 con 0x7faab80cd7b0 2023-12-20T06:02:06.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faabcd5a640 1 -- 172.21.15.32:0/2665267887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faab80c6d40 con 0x7faab80cd7b0 2023-12-20T06:02:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.313+0000 7faa9ffff640 1 -- 172.21.15.32:0/2665267887 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 120) v1 ==== 85444+0+0 (secure 0 0 0) 0x7faaa80b3040 con 0x7faab80cd7b0 2023-12-20T06:02:06.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.317+0000 7faa9ffff640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faaa4065ca0 0x7faaa4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.317+0000 7faab6ffd640 1 -- 172.21.15.32:0/2665267887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faaa4065ca0 msgr2=0x7faaa4068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.317+0000 7faa9ffff640 1 -- 172.21.15.32:0/2665267887 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6181+0+0 (secure 0 0 0) 0x7faaa811dbd0 con 0x7faab80cd7b0 2023-12-20T06:02:06.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.317+0000 7faab6ffd640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faaa4065ca0 0x7faaa4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:06.320 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:06.316+0000 7fe542b5c180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:02:06.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.321+0000 7faa9ffff640 1 -- 172.21.15.32:0/2665267887 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faab80c6d40 con 0x7faab80cd7b0 2023-12-20T06:02:06.408 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:06.404+0000 7fe542b5c180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:02:06.415 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.413+0000 7f90acc7d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:02:06.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.477+0000 7faabcd5a640 1 -- 172.21.15.32:0/2665267887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7faab80c6d40 con 0x7faab80cd7b0 2023-12-20T06:02:06.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faa9ffff640 1 -- 172.21.15.32:0/2665267887 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v195) v1 ==== 104+0+11 (secure 0 0 0) 0x7faab80c6d40 con 0x7faab80cd7b0 2023-12-20T06:02:06.481 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:02:06.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 -- 172.21.15.32:0/2665267887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faaa4065ca0 msgr2=0x7faaa4068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:06.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faaa4065ca0 0x7faaa4068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 -- 172.21.15.32:0/2665267887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab80cd7b0 msgr2=0x7faab80b75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:06.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab80cd7b0 0x7faab80b75d0 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7faaa801ab30 tx=0x7faaa801acc0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 -- 172.21.15.32:0/2665267887 shutdown_connections 2023-12-20T06:02:06.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faaa4065ca0 0x7faaa4068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faab815ccf0 0x7faab8153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faab80cf920 0x7faab80c8580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 --2- 172.21.15.32:0/2665267887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faab80cd7b0 0x7faab80b75d0 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 -- 172.21.15.32:0/2665267887 >> 172.21.15.32:0/2665267887 conn(0x7faab80658e0 msgr2=0x7faab80b40a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 -- 172.21.15.32:0/2665267887 shutdown_connections 2023-12-20T06:02:06.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.481+0000 7faabcd5a640 1 -- 172.21.15.32:0/2665267887 wait complete. 2023-12-20T06:02:06.502 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:02:06.505 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.505+0000 7f90acc7d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:02:06.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.649+0000 7f71ada41640 1 Processor -- start 2023-12-20T06:02:06.652 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71ada41640 1 -- start start 2023-12-20T06:02:06.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71ada41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71a815abb0 0x7f71a80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71ada41640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f71a80b0cb0 0x7f71a815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71ada41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71a8070230 0x7f71a80706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71ada41640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a806e160 con 0x7f71a8070230 2023-12-20T06:02:06.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71ada41640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f71a8070be0 con 0x7f71a80b0cb0 2023-12-20T06:02:06.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71ada41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a815b6a0 con 0x7f71a815abb0 2023-12-20T06:02:06.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71a77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71a8070230 0x7f71a80706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:06.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71a77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f71a8070230 0x7f71a80706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50418/0 (socket says 172.21.15.32:50418) 2023-12-20T06:02:06.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71a77fe640 1 -- 172.21.15.32:0/2938487263 learned_addr learned my addr 172.21.15.32:0/2938487263 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:06.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71a6ffd640 1 --2- 172.21.15.32:0/2938487263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71a815abb0 0x7f71a80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71a67fc640 1 --2- 172.21.15.32:0/2938487263 >> v2:172.21.15.39:3300/0 conn(0x7f71a80b0cb0 0x7f71a815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71a77fe640 1 -- 172.21.15.32:0/2938487263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71a815abb0 msgr2=0x7f71a80caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71a77fe640 1 --2- 172.21.15.32:0/2938487263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71a815abb0 0x7f71a80caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71a77fe640 1 -- 172.21.15.32:0/2938487263 >> v2:172.21.15.39:3300/0 conn(0x7f71a80b0cb0 msgr2=0x7f71a815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71a77fe640 1 --2- 172.21.15.32:0/2938487263 >> v2:172.21.15.39:3300/0 conn(0x7f71a80b0cb0 0x7f71a815bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.653+0000 7f71a77fe640 1 -- 172.21.15.32:0/2938487263 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a815c070 con 0x7f71a8070230 2023-12-20T06:02:06.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71a77fe640 1 --2- 172.21.15.32:0/2938487263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a8070230 0x7f71a80706a0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f71a80716d0 tx=0x7f71a0020de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c9bf37292a99dd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71a5ffb640 1 -- 172.21.15.32:0/2938487263 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00a9b80 con 0x7f71a8070230 2023-12-20T06:02:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71a5ffb640 1 -- 172.21.15.32:0/2938487263 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71a00a9ce0 con 0x7f71a8070230 2023-12-20T06:02:06.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71a5ffb640 1 -- 172.21.15.32:0/2938487263 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71a00b1610 con 0x7f71a8070230 2023-12-20T06:02:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71ada41640 1 -- 172.21.15.32:0/2938487263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a8070230 msgr2=0x7f71a80706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71ada41640 1 --2- 172.21.15.32:0/2938487263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a8070230 0x7f71a80706a0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7f71a80716d0 tx=0x7f71a0020de0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71ada41640 1 -- 172.21.15.32:0/2938487263 shutdown_connections 2023-12-20T06:02:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71ada41640 1 --2- 172.21.15.32:0/2938487263 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a8070230 0x7f71a80706a0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71ada41640 1 --2- 172.21.15.32:0/2938487263 >> v2:172.21.15.39:3300/0 conn(0x7f71a80b0cb0 0x7f71a815bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71ada41640 1 --2- 172.21.15.32:0/2938487263 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71a815abb0 0x7f71a80caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71ada41640 1 -- 172.21.15.32:0/2938487263 >> 172.21.15.32:0/2938487263 conn(0x7f71a806a510 msgr2=0x7f71a80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71ada41640 1 -- 172.21.15.32:0/2938487263 shutdown_connections 2023-12-20T06:02:06.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71ada41640 1 -- 172.21.15.32:0/2938487263 wait complete. 2023-12-20T06:02:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.657+0000 7f71ada41640 1 Processor -- start 2023-12-20T06:02:06.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71ada41640 1 -- start start 2023-12-20T06:02:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71ada41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71a8070230 0x7f71a814c680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71ada41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a80b0cb0 0x7f71a81e3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71ada41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71a815abb0 0x7f71a81e3c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71ada41640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a8158000 con 0x7f71a80b0cb0 2023-12-20T06:02:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71ada41640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f71a8151660 con 0x7f71a8070230 2023-12-20T06:02:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71ada41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f71a81e5960 con 0x7f71a815abb0 2023-12-20T06:02:06.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71a67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a80b0cb0 0x7f71a81e3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71a67fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a80b0cb0 0x7f71a81e3720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50424/0 (socket says 172.21.15.32:50424) 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71a67fc640 1 -- 172.21.15.32:0/2325194173 learned_addr learned my addr 172.21.15.32:0/2325194173 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71a6ffd640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71a8070230 0x7f71a814c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71a77fe640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71a815abb0 0x7f71a81e3c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71a67fc640 1 -- 172.21.15.32:0/2325194173 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71a815abb0 msgr2=0x7f71a81e3c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71a67fc640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71a815abb0 0x7f71a81e3c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71a67fc640 1 -- 172.21.15.32:0/2325194173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71a8070230 msgr2=0x7f71a814c680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71a67fc640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71a8070230 0x7f71a814c680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71a67fc640 1 -- 172.21.15.32:0/2325194173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f71a0009b30 con 0x7f71a80b0cb0 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71a67fc640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a80b0cb0 0x7f71a81e3720 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f719c00e8a0 tx=0x7f719c00ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f718ffff640 1 -- 172.21.15.32:0/2325194173 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00ee90 con 0x7f71a80b0cb0 2023-12-20T06:02:06.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f718ffff640 1 -- 172.21.15.32:0/2325194173 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719c00c940 con 0x7f71a80b0cb0 2023-12-20T06:02:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f718ffff640 1 -- 172.21.15.32:0/2325194173 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719c00ccf0 con 0x7f71a80b0cb0 2023-12-20T06:02:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71ada41640 1 -- 172.21.15.32:0/2325194173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f71a81e5c40 con 0x7f71a80b0cb0 2023-12-20T06:02:06.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71ada41640 1 -- 172.21.15.32:0/2325194173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f71a8158250 con 0x7f71a80b0cb0 2023-12-20T06:02:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.661+0000 7f71ada41640 1 -- 172.21.15.32:0/2325194173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f71a804e000 con 0x7f71a80b0cb0 2023-12-20T06:02:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.665+0000 7f718ffff640 1 -- 172.21.15.32:0/2325194173 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 120) v1 ==== 85444+0+0 (secure 0 0 0) 0x7f719c013470 con 0x7f71a80b0cb0 2023-12-20T06:02:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.665+0000 7f718ffff640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7194065ca0 0x7f7194068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.665+0000 7f71a6ffd640 1 -- 172.21.15.32:0/2325194173 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7194065ca0 msgr2=0x7f7194068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.665+0000 7f718ffff640 1 -- 172.21.15.32:0/2325194173 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(195..195 src has 1..195) v4 ==== 6181+0+0 (secure 0 0 0) 0x7f719c083790 con 0x7f71a80b0cb0 2023-12-20T06:02:06.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.665+0000 7f71a6ffd640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7194065ca0 0x7f7194068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:06.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.669+0000 7f718ffff640 1 -- 172.21.15.32:0/2325194173 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f719c050cb0 con 0x7f71a80b0cb0 2023-12-20T06:02:06.743 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:06.740+0000 7fe542b5c180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:02:06.806 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:06.804+0000 7fe542b5c180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:02:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.825+0000 7f71ada41640 1 -- 172.21.15.32:0/2325194173 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f71a8071150 con 0x7f71a80b0cb0 2023-12-20T06:02:06.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.825+0000 7f718ffff640 1 -- 172.21.15.32:0/2325194173 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v195) v1 ==== 100+0+11 (secure 0 0 0) 0x7f719c081e60 con 0x7f71a80b0cb0 2023-12-20T06:02:06.828 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:02:06.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 -- 172.21.15.32:0/2325194173 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7194065ca0 msgr2=0x7f7194068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7194065ca0 0x7f7194068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 -- 172.21.15.32:0/2325194173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a80b0cb0 msgr2=0x7f71a81e3720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a80b0cb0 0x7f71a81e3720 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f719c00e8a0 tx=0x7f719c00ed60 comp rx=0 tx=0).stop 2023-12-20T06:02:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 -- 172.21.15.32:0/2325194173 shutdown_connections 2023-12-20T06:02:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7194065ca0 0x7f7194068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f71a815abb0 0x7f71a81e3c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f71a80b0cb0 0x7f71a81e3720 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 --2- 172.21.15.32:0/2325194173 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f71a8070230 0x7f71a814c680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 -- 172.21.15.32:0/2325194173 >> 172.21.15.32:0/2325194173 conn(0x7f71a806a510 msgr2=0x7f71a8157390 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 -- 172.21.15.32:0/2325194173 shutdown_connections 2023-12-20T06:02:06.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:06.829+0000 7f71ada41640 1 -- 172.21.15.32:0/2325194173 wait complete. 2023-12-20T06:02:06.853 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:02:06.854 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.853+0000 7f90acc7d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:02:06.921 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.920+0000 7f90acc7d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:02:06.996 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.992+0000 7f90353ff640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:06.996 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.992+0000 7f90353ff640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:06.996 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.992+0000 7f90353ff640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:06.996 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.992+0000 7f90353ff640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:06.996 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.992+0000 7f90353ff640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:06.997 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.996+0000 7f9037c04640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:06.997 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.996+0000 7f9037c04640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:06.997 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.996+0000 7f9037c04640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:06.997 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.996+0000 7f9037c04640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:06.997 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:06.996+0000 7f9037c04640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3b262640 1 Processor -- start 2023-12-20T06:02:07.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3b262640 1 -- start start 2023-12-20T06:02:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3b262640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b34161500 0x7f6b34153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3b262640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b340cd4d0 con 0x7f6b340b8480 2023-12-20T06:02:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3b262640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b340ce4e0 con 0x7f6b3406b630 2023-12-20T06:02:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3b262640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340cd720 con 0x7f6b34161500 2023-12-20T06:02:07.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3a260640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6b340b8480 0x7f6b3406af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39098/0 (socket says 172.21.15.32:39098) 2023-12-20T06:02:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3a260640 1 -- 172.21.15.32:0/4029696939 learned_addr learned my addr 172.21.15.32:0/4029696939 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3aa61640 1 --2- 172.21.15.32:0/4029696939 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b34161500 0x7f6b34153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3925e640 1 -- 172.21.15.32:0/4029696939 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 84686607 0 0) 0x7f6b340cd4d0 con 0x7f6b340b8480 2023-12-20T06:02:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3925e640 1 -- 172.21.15.32:0/4029696939 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b20003530 con 0x7f6b340b8480 2023-12-20T06:02:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3925e640 1 -- 172.21.15.32:0/4029696939 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2475850917 0 0) 0x7f6b340ce4e0 con 0x7f6b3406b630 2023-12-20T06:02:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3925e640 1 -- 172.21.15.32:0/4029696939 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6b340cd4d0 con 0x7f6b3406b630 2023-12-20T06:02:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3aa61640 1 -- 172.21.15.32:0/4029696939 >> v1:172.21.15.39:6789/0 conn(0x7f6b3406b630 legacy=0x7f6b34160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3aa61640 1 -- 172.21.15.32:0/4029696939 >> v1:172.21.15.32:6789/0 conn(0x7f6b340b8480 legacy=0x7f6b3406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.012+0000 7f6b3aa61640 1 -- 172.21.15.32:0/4029696939 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b340cda00 con 0x7f6b34161500 2023-12-20T06:02:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.016+0000 7f6b3aa61640 1 --2- 172.21.15.32:0/4029696939 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b34161500 0x7f6b34153610 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009dc0 tx=0x7f6b2c01c970 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5fb78b461754bea9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.016+0000 7f6b3925e640 1 -- 172.21.15.32:0/4029696939 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0a5810 con 0x7f6b34161500 2023-12-20T06:02:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.016+0000 7f6b3925e640 1 -- 172.21.15.32:0/4029696939 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c0a5970 con 0x7f6b34161500 2023-12-20T06:02:07.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.016+0000 7f6b3925e640 1 -- 172.21.15.32:0/4029696939 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0a5d00 con 0x7f6b34161500 2023-12-20T06:02:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.016+0000 7f6b3b262640 1 -- 172.21.15.32:0/4029696939 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b34161500 msgr2=0x7f6b34153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.016+0000 7f6b3b262640 1 --2- 172.21.15.32:0/4029696939 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b34161500 0x7f6b34153610 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c009dc0 tx=0x7f6b2c01c970 comp rx=0 tx=0).stop 2023-12-20T06:02:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.016+0000 7f6b3b262640 1 -- 172.21.15.32:0/4029696939 shutdown_connections 2023-12-20T06:02:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.016+0000 7f6b3b262640 1 --2- 172.21.15.32:0/4029696939 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b34161500 0x7f6b34153610 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.016+0000 7f6b3b262640 1 -- 172.21.15.32:0/4029696939 >> 172.21.15.32:0/4029696939 conn(0x7f6b340d2a50 msgr2=0x7f6b340d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.016+0000 7f6b3b262640 1 -- 172.21.15.32:0/4029696939 shutdown_connections 2023-12-20T06:02:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.016+0000 7f6b3b262640 1 -- 172.21.15.32:0/4029696939 wait complete. 2023-12-20T06:02:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3b262640 1 Processor -- start 2023-12-20T06:02:07.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3b262640 1 -- start start 2023-12-20T06:02:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3b262640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b3406b630 0x7f6b340c3b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3b262640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b340b8480 0x7f6b340c64f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3b262640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b34161500 0x7f6b340c6a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3b262640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340c5da0 con 0x7f6b34161500 2023-12-20T06:02:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3b262640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340c5630 con 0x7f6b340b8480 2023-12-20T06:02:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3b262640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b340c40a0 con 0x7f6b3406b630 2023-12-20T06:02:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3aa61640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b34161500 0x7f6b340c6a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3aa61640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b34161500 0x7f6b340c6a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50544/0 (socket says 172.21.15.32:50544) 2023-12-20T06:02:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3aa61640 1 -- 172.21.15.32:0/3596215190 learned_addr learned my addr 172.21.15.32:0/3596215190 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3a260640 1 --2- 172.21.15.32:0/3596215190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b3406b630 0x7f6b340c3b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b39a5f640 1 --2- 172.21.15.32:0/3596215190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b340b8480 0x7f6b340c64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3aa61640 1 -- 172.21.15.32:0/3596215190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b3406b630 msgr2=0x7f6b340c3b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3aa61640 1 --2- 172.21.15.32:0/3596215190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b3406b630 0x7f6b340c3b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3aa61640 1 -- 172.21.15.32:0/3596215190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b340b8480 msgr2=0x7f6b340c64f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3aa61640 1 --2- 172.21.15.32:0/3596215190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b340b8480 0x7f6b340c64f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3aa61640 1 -- 172.21.15.32:0/3596215190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b2c009a70 con 0x7f6b34161500 2023-12-20T06:02:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b39a5f640 1 --2- 172.21.15.32:0/3596215190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b340b8480 0x7f6b340c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3aa61640 1 --2- 172.21.15.32:0/3596215190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b34161500 0x7f6b340c6a30 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c01a5a0 tx=0x7f6b2c019710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b2affd640 1 -- 172.21.15.32:0/3596215190 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0ad640 con 0x7f6b34161500 2023-12-20T06:02:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b2affd640 1 -- 172.21.15.32:0/3596215190 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b2c0ad7a0 con 0x7f6b34161500 2023-12-20T06:02:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b2affd640 1 -- 172.21.15.32:0/3596215190 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b2c0adb50 con 0x7f6b34161500 2023-12-20T06:02:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3b262640 1 -- 172.21.15.32:0/3596215190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b340c4ab0 con 0x7f6b34161500 2023-12-20T06:02:07.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3b262640 1 -- 172.21.15.32:0/3596215190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b340c5070 con 0x7f6b34161500 2023-12-20T06:02:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.020+0000 7f6b3b262640 1 -- 172.21.15.32:0/3596215190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b34074100 con 0x7f6b34161500 2023-12-20T06:02:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.024+0000 7f6b2affd640 1 -- 172.21.15.32:0/3596215190 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 122) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f6b2c0b4050 con 0x7f6b34161500 2023-12-20T06:02:07.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.024+0000 7f6b2affd640 1 -- 172.21.15.32:0/3596215190 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6481+0+0 (secure 0 0 0) 0x7f6b2c11dca0 con 0x7f6b34161500 2023-12-20T06:02:07.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.028+0000 7f6b2affd640 1 -- 172.21.15.32:0/3596215190 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6b2c0eb6b0 con 0x7f6b34161500 2023-12-20T06:02:07.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.220+0000 7f6b3b262640 1 -- 172.21.15.32:0/3596215190 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6b34155660 con 0x7f6b34161500 2023-12-20T06:02:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.220+0000 7f6b2affd640 1 -- 172.21.15.32:0/3596215190 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f6b2c0b2090 con 0x7f6b34161500 2023-12-20T06:02:07.224 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:07.224 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8396,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1626919510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1626919510},{"type":"v1","addr":"172.21.15.39:6833","nonce":1626919510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3001408763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3001408763},{"type":"v1","addr":"172.21.15.39:6835","nonce":3001408763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":10020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/2908145202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2908145202},{"type":"v1","addr":"172.21.15.32:6835","nonce":2908145202}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:00:37.604080+0000","modified":"2023-12-20T06:00:40.642479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10026":{"gid":10026,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2253884246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2253884246},{"type":"v1","addr":"172.21.15.32:6837","nonce":2253884246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-20T06:02:07.224 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 87 2023-12-20T06:02:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.224+0000 7f6b3b262640 1 -- 172.21.15.32:0/3596215190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b34161500 msgr2=0x7f6b340c6a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.224+0000 7f6b3b262640 1 --2- 172.21.15.32:0/3596215190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b34161500 0x7f6b340c6a30 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f6b2c01a5a0 tx=0x7f6b2c019710 comp rx=0 tx=0).stop 2023-12-20T06:02:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.224+0000 7f6b3b262640 1 -- 172.21.15.32:0/3596215190 shutdown_connections 2023-12-20T06:02:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.224+0000 7f6b3b262640 1 --2- 172.21.15.32:0/3596215190 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b34161500 0x7f6b340c6a30 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.224+0000 7f6b3b262640 1 --2- 172.21.15.32:0/3596215190 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b340b8480 0x7f6b340c64f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.224+0000 7f6b3b262640 1 --2- 172.21.15.32:0/3596215190 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b3406b630 0x7f6b340c3b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.224+0000 7f6b3b262640 1 -- 172.21.15.32:0/3596215190 >> 172.21.15.32:0/3596215190 conn(0x7f6b340d2a50 msgr2=0x7f6b340d2e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.224+0000 7f6b3b262640 1 -- 172.21.15.32:0/3596215190 shutdown_connections 2023-12-20T06:02:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.224+0000 7f6b3b262640 1 -- 172.21.15.32:0/3596215190 wait complete. 2023-12-20T06:02:07.245 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:07.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea092b6640 1 Processor -- start 2023-12-20T06:02:07.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea092b6640 1 -- start start 2023-12-20T06:02:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea092b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea0415cdb0 0x7fea040d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea092b6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fea040cfc80 0x7fea040b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea092b6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea0415d190 con 0x7fea040d2740 2023-12-20T06:02:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea092b6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea040d16a0 con 0x7fea040cfc80 2023-12-20T06:02:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea092b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea040d1980 con 0x7fea0415cdb0 2023-12-20T06:02:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea037fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fea040cfc80 0x7fea040b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea037fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fea040cfc80 0x7fea040b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:40088/0 (socket says 172.21.15.32:40088) 2023-12-20T06:02:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea037fe640 1 -- 172.21.15.32:0/1292574806 learned_addr learned my addr 172.21.15.32:0/1292574806 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea02ffd640 1 --2- 172.21.15.32:0/1292574806 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea0415cdb0 0x7fea040d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea037fe640 1 -- 172.21.15.32:0/1292574806 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea0415cdb0 msgr2=0x7fea040d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea037fe640 1 --2- 172.21.15.32:0/1292574806 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea0415cdb0 0x7fea040d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea037fe640 1 -- 172.21.15.32:0/1292574806 >> v1:172.21.15.39:6789/0 conn(0x7fea040d2740 legacy=0x7fea040cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.388+0000 7fea037fe640 1 -- 172.21.15.32:0/1292574806 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea040d1c60 con 0x7fea040cfc80 2023-12-20T06:02:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea037fe640 1 --2- 172.21.15.32:0/1292574806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea040cfc80 0x7fea040b5c90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009e50 tx=0x7fe9fc01f1f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a0dc5c2a42ded455 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea027fc640 1 -- 172.21.15.32:0/1292574806 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc0af400 con 0x7fea040cfc80 2023-12-20T06:02:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea027fc640 1 -- 172.21.15.32:0/1292574806 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9fc0ab050 con 0x7fea040cfc80 2023-12-20T06:02:07.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea027fc640 1 -- 172.21.15.32:0/1292574806 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9fc0af780 con 0x7fea040cfc80 2023-12-20T06:02:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea092b6640 1 -- 172.21.15.32:0/1292574806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea040cfc80 msgr2=0x7fea040b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea092b6640 1 --2- 172.21.15.32:0/1292574806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea040cfc80 0x7fea040b5c90 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fe9fc009e50 tx=0x7fe9fc01f1f0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea092b6640 1 -- 172.21.15.32:0/1292574806 shutdown_connections 2023-12-20T06:02:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea092b6640 1 --2- 172.21.15.32:0/1292574806 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea040cfc80 0x7fea040b5c90 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea092b6640 1 --2- 172.21.15.32:0/1292574806 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea0415cdb0 0x7fea040d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea092b6640 1 -- 172.21.15.32:0/1292574806 >> 172.21.15.32:0/1292574806 conn(0x7fea04065ee0 msgr2=0x7fea0406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea092b6640 1 -- 172.21.15.32:0/1292574806 shutdown_connections 2023-12-20T06:02:07.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.392+0000 7fea092b6640 1 -- 172.21.15.32:0/1292574806 wait complete. 2023-12-20T06:02:07.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea092b6640 1 Processor -- start 2023-12-20T06:02:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea092b6640 1 -- start start 2023-12-20T06:02:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea092b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea040cfc80 0x7fea0415aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea092b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea040d2740 0x7fea041f5590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea092b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea0415cdb0 0x7fea0415fe50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea092b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea041f5ad0 con 0x7fea040d2740 2023-12-20T06:02:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea092b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea0415edb0 con 0x7fea040cfc80 2023-12-20T06:02:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea092b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea0415f090 con 0x7fea0415cdb0 2023-12-20T06:02:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea02ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea040cfc80 0x7fea0415aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fe9fbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea040d2740 0x7fea041f5590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea02ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea040cfc80 0x7fea0415aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:40096/0 (socket says 172.21.15.32:40096) 2023-12-20T06:02:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea02ffd640 1 -- 172.21.15.32:0/2609580556 learned_addr learned my addr 172.21.15.32:0/2609580556 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea037fe640 1 --2- 172.21.15.32:0/2609580556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea0415cdb0 0x7fea0415fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea02ffd640 1 -- 172.21.15.32:0/2609580556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea0415cdb0 msgr2=0x7fea0415fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea02ffd640 1 --2- 172.21.15.32:0/2609580556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea0415cdb0 0x7fea0415fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea02ffd640 1 -- 172.21.15.32:0/2609580556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea040d2740 msgr2=0x7fea041f5590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea02ffd640 1 --2- 172.21.15.32:0/2609580556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea040d2740 0x7fea041f5590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea02ffd640 1 -- 172.21.15.32:0/2609580556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9fc009b30 con 0x7fea040cfc80 2023-12-20T06:02:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fe9fbfff640 1 --2- 172.21.15.32:0/2609580556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea040d2740 0x7fea041f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:07.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea037fe640 1 --2- 172.21.15.32:0/2609580556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea0415cdb0 0x7fea0415fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea02ffd640 1 --2- 172.21.15.32:0/2609580556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea040cfc80 0x7fea0415aca0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400e8a0 tx=0x7fe9f400eca0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fe9fb7fe640 1 -- 172.21.15.32:0/2609580556 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f4009510 con 0x7fea040cfc80 2023-12-20T06:02:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fe9fb7fe640 1 -- 172.21.15.32:0/2609580556 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9f400c940 con 0x7fea040cfc80 2023-12-20T06:02:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fe9fb7fe640 1 -- 172.21.15.32:0/2609580556 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9f400ccb0 con 0x7fea040cfc80 2023-12-20T06:02:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea092b6640 1 -- 172.21.15.32:0/2609580556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea0415f370 con 0x7fea040cfc80 2023-12-20T06:02:07.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.396+0000 7fea092b6640 1 -- 172.21.15.32:0/2609580556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea0415f930 con 0x7fea040cfc80 2023-12-20T06:02:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.400+0000 7fea092b6640 1 -- 172.21.15.32:0/2609580556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea0404e000 con 0x7fea040cfc80 2023-12-20T06:02:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.400+0000 7fe9fb7fe640 1 -- 172.21.15.32:0/2609580556 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 122) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fe9f4013470 con 0x7fea040cfc80 2023-12-20T06:02:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.400+0000 7fe9fb7fe640 1 -- 172.21.15.32:0/2609580556 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6481+0+0 (secure 0 0 0) 0x7fe9f4083080 con 0x7fea040cfc80 2023-12-20T06:02:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.404+0000 7fe9fb7fe640 1 -- 172.21.15.32:0/2609580556 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe9f40b40a0 con 0x7fea040cfc80 2023-12-20T06:02:07.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.564+0000 7fea092b6640 1 -- 172.21.15.32:0/2609580556 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fea0406b8d0 con 0x7fea040cfc80 2023-12-20T06:02:07.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.568+0000 7fe9fb7fe640 1 -- 172.21.15.32:0/2609580556 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+17198 (secure 0 0 0) 0x7fe9f4017090 con 0x7fea040cfc80 2023-12-20T06:02:07.569 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:07.569 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":196,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:06.926932+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:00:36.162516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-20T06:00:36.593153+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":195,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1840226039":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/3016054758":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/3042178755":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/1302755445":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/3527982629":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/3239744916":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/2437361294":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/1388061423":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/968197028":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/2674032592":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/4037417795":"2023-12-21T06:00:49.693740+0000","172.21.15.32:6833/64546":"2023-12-21T06:00:49.693740+0000","172.21.15.32:6832/64546":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/555339143":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/4289008383":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/1643902464":"2023-12-21T06:00:49.693740+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.568+0000 7fea092b6640 1 -- 172.21.15.32:0/2609580556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea040cfc80 msgr2=0x7fea0415aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.568+0000 7fea092b6640 1 --2- 172.21.15.32:0/2609580556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea040cfc80 0x7fea0415aca0 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fe9f400e8a0 tx=0x7fe9f400eca0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.568+0000 7fea092b6640 1 -- 172.21.15.32:0/2609580556 shutdown_connections 2023-12-20T06:02:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.568+0000 7fea092b6640 1 --2- 172.21.15.32:0/2609580556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea0415cdb0 0x7fea0415fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.568+0000 7fea092b6640 1 --2- 172.21.15.32:0/2609580556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea040d2740 0x7fea041f5590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.568+0000 7fea092b6640 1 --2- 172.21.15.32:0/2609580556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea040cfc80 0x7fea0415aca0 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.568+0000 7fea092b6640 1 -- 172.21.15.32:0/2609580556 >> 172.21.15.32:0/2609580556 conn(0x7fea04065ee0 msgr2=0x7fea040d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.568+0000 7fea092b6640 1 -- 172.21.15.32:0/2609580556 shutdown_connections 2023-12-20T06:02:07.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.568+0000 7fea092b6640 1 -- 172.21.15.32:0/2609580556 wait complete. 2023-12-20T06:02:07.591 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T06:02:07.591 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:02:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f286d3b3640 1 Processor -- start 2023-12-20T06:02:07.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f286d3b3640 1 -- start start 2023-12-20T06:02:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f286d3b3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f28681597e0 0x7f2868159bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f286d3b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28680c8580 0x7f28680b0800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f286d3b3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f286806c180 con 0x7f28680cb090 2023-12-20T06:02:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f286d3b3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286806c460 con 0x7f28680c8580 2023-12-20T06:02:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f286d3b3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f286806c740 con 0x7f28681597e0 2023-12-20T06:02:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f28667fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28680c8580 0x7f28680b0800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f28667fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f28680c8580 0x7f28680b0800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50556/0 (socket says 172.21.15.32:50556) 2023-12-20T06:02:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f28667fc640 1 -- 172.21.15.32:0/2915814158 learned_addr learned my addr 172.21.15.32:0/2915814158 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f2866ffd640 1 --2- 172.21.15.32:0/2915814158 >> v2:172.21.15.39:3300/0 conn(0x7f28681597e0 0x7f2868159bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f28667fc640 1 -- 172.21.15.32:0/2915814158 >> v2:172.21.15.39:3300/0 conn(0x7f28681597e0 msgr2=0x7f2868159bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f28667fc640 1 --2- 172.21.15.32:0/2915814158 >> v2:172.21.15.39:3300/0 conn(0x7f28681597e0 0x7f2868159bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f28667fc640 1 -- 172.21.15.32:0/2915814158 >> v1:172.21.15.39:6789/0 conn(0x7f28680cb090 legacy=0x7f28680b1550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.736+0000 7f28667fc640 1 -- 172.21.15.32:0/2915814158 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f286806ca20 con 0x7f28680c8580 2023-12-20T06:02:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f28667fc640 1 --2- 172.21.15.32:0/2915814158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680c8580 0x7f28680b0800 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f2868071350 tx=0x7f285c01eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a3de9b0847b9d8f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f2865ffb640 1 -- 172.21.15.32:0/2915814158 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c0a7bd0 con 0x7f28680c8580 2023-12-20T06:02:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f2865ffb640 1 -- 172.21.15.32:0/2915814158 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c0a7d30 con 0x7f28680c8580 2023-12-20T06:02:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f2865ffb640 1 -- 172.21.15.32:0/2915814158 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c0a7ea0 con 0x7f28680c8580 2023-12-20T06:02:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f286d3b3640 1 -- 172.21.15.32:0/2915814158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680c8580 msgr2=0x7f28680b0800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f286d3b3640 1 --2- 172.21.15.32:0/2915814158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680c8580 0x7f28680b0800 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f2868071350 tx=0x7f285c01eba0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f286d3b3640 1 -- 172.21.15.32:0/2915814158 shutdown_connections 2023-12-20T06:02:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f286d3b3640 1 --2- 172.21.15.32:0/2915814158 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680c8580 0x7f28680b0800 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f286d3b3640 1 --2- 172.21.15.32:0/2915814158 >> v2:172.21.15.39:3300/0 conn(0x7f28681597e0 0x7f2868159bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f286d3b3640 1 -- 172.21.15.32:0/2915814158 >> 172.21.15.32:0/2915814158 conn(0x7f2868069d20 msgr2=0x7f286806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f286d3b3640 1 -- 172.21.15.32:0/2915814158 shutdown_connections 2023-12-20T06:02:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.740+0000 7f286d3b3640 1 -- 172.21.15.32:0/2915814158 wait complete. 2023-12-20T06:02:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f286d3b3640 1 Processor -- start 2023-12-20T06:02:07.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f286d3b3640 1 -- start start 2023-12-20T06:02:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f286d3b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28680c8580 0x7f286815c5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f286d3b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cb090 0x7f28680c4b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f286d3b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28681597e0 0x7f286815d5c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f286d3b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28680c50b0 con 0x7f28680cb090 2023-12-20T06:02:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f286d3b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f28680c2c80 con 0x7f28681597e0 2023-12-20T06:02:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f286d3b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f286815db00 con 0x7f28680c8580 2023-12-20T06:02:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f28667fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cb090 0x7f28680c4b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f28667fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cb090 0x7f28680c4b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50562/0 (socket says 172.21.15.32:50562) 2023-12-20T06:02:07.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f28667fc640 1 -- 172.21.15.32:0/1171299540 learned_addr learned my addr 172.21.15.32:0/1171299540 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f28667fc640 1 -- 172.21.15.32:0/1171299540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28680c8580 msgr2=0x7f286815c5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f28667fc640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28680c8580 0x7f286815c5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f28667fc640 1 -- 172.21.15.32:0/1171299540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28681597e0 msgr2=0x7f286815d5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f28667fc640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28681597e0 0x7f286815d5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f28667fc640 1 -- 172.21.15.32:0/1171299540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f285c00b770 con 0x7f28680cb090 2023-12-20T06:02:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f28667fc640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cb090 0x7f28680c4b70 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f285c00b2c0 tx=0x7f285c01b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f284f7fe640 1 -- 172.21.15.32:0/1171299540 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c0a7ea0 con 0x7f28680cb090 2023-12-20T06:02:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f284f7fe640 1 -- 172.21.15.32:0/1171299540 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285c0ba440 con 0x7f28680cb090 2023-12-20T06:02:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f284f7fe640 1 -- 172.21.15.32:0/1171299540 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285c0a7bd0 con 0x7f28680cb090 2023-12-20T06:02:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f286d3b3640 1 -- 172.21.15.32:0/1171299540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f28680cba30 con 0x7f28680cb090 2023-12-20T06:02:07.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f286d3b3640 1 -- 172.21.15.32:0/1171299540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28680c2350 con 0x7f28680cb090 2023-12-20T06:02:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.744+0000 7f286d3b3640 1 -- 172.21.15.32:0/1171299540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f28680caf00 con 0x7f28680cb090 2023-12-20T06:02:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.748+0000 7f284f7fe640 1 -- 172.21.15.32:0/1171299540 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 122) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f285c0b93e0 con 0x7f28680cb090 2023-12-20T06:02:07.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.748+0000 7f284f7fe640 1 -- 172.21.15.32:0/1171299540 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6481+0+0 (secure 0 0 0) 0x7f285c11d720 con 0x7f28680cb090 2023-12-20T06:02:07.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.752+0000 7f284f7fe640 1 -- 172.21.15.32:0/1171299540 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f285c0eb140 con 0x7f28680cb090 2023-12-20T06:02:07.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.936+0000 7f284f7fe640 1 -- 172.21.15.32:0/1171299540 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 123) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f285c0eabf0 con 0x7f28680cb090 2023-12-20T06:02:07.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.936+0000 7f284f7fe640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f285406b2a0 0x7f285406d690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:07.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.936+0000 7f2866ffd640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f285406b2a0 0x7f285406d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:07.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.940+0000 7f2866ffd640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f285406b2a0 0x7f285406d690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f285800b850 tx=0x7f2858008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:07.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 -- 172.21.15.32:0/1171299540 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2868074100 con 0x7f28680cb090 2023-12-20T06:02:07.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f284f7fe640 1 -- 172.21.15.32:0/1171299540 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f285c0f0360 con 0x7f28680cb090 2023-12-20T06:02:07.954 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:07.954 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8396,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1626919510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1626919510},{"type":"v1","addr":"172.21.15.39:6833","nonce":1626919510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3001408763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3001408763},{"type":"v1","addr":"172.21.15.39:6835","nonce":3001408763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":10020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/2908145202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2908145202},{"type":"v1","addr":"172.21.15.32:6835","nonce":2908145202}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:00:37.604080+0000","modified":"2023-12-20T06:00:40.642479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10026":{"gid":10026,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2253884246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2253884246},{"type":"v1","addr":"172.21.15.32:6837","nonce":2253884246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-20T06:02:07.955 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 87 2023-12-20T06:02:07.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 -- 172.21.15.32:0/1171299540 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f285406b2a0 msgr2=0x7f285406d690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f285406b2a0 0x7f285406d690 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f285800b850 tx=0x7f2858008040 comp rx=0 tx=0).stop 2023-12-20T06:02:07.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 -- 172.21.15.32:0/1171299540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cb090 msgr2=0x7f28680c4b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:07.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cb090 0x7f28680c4b70 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7f285c00b2c0 tx=0x7f285c01b940 comp rx=0 tx=0).stop 2023-12-20T06:02:07.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 -- 172.21.15.32:0/1171299540 shutdown_connections 2023-12-20T06:02:07.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f285406b2a0 0x7f285406d690 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28681597e0 0x7f286815d5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28680cb090 0x7f28680c4b70 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 --2- 172.21.15.32:0/1171299540 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28680c8580 0x7f286815c5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:07.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 -- 172.21.15.32:0/1171299540 >> 172.21.15.32:0/1171299540 conn(0x7f2868069d20 msgr2=0x7f286806a100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:07.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 -- 172.21.15.32:0/1171299540 shutdown_connections 2023-12-20T06:02:07.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:07.952+0000 7f286d3b3640 1 -- 172.21.15.32:0/1171299540 wait complete. 2023-12-20T06:02:07.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:02:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d69b2c640 1 Processor -- start 2023-12-20T06:02:08.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d69b2c640 1 -- start start 2023-12-20T06:02:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d69b2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d6415ae80 0x7f7d640cfed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d69b2c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d640afd50 0x7f7d640b23d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d69b2c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7d640ced30 con 0x7f7d640b2910 2023-12-20T06:02:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d69b2c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7d640ceb50 con 0x7f7d640afd50 2023-12-20T06:02:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d69b2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d640d0410 con 0x7f7d6415ae80 2023-12-20T06:02:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d62ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d640afd50 0x7f7d640b23d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d62ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7d640afd50 0x7f7d640b23d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50564/0 (socket says 172.21.15.32:50564) 2023-12-20T06:02:08.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d62ffd640 1 -- 172.21.15.32:0/2360939110 learned_addr learned my addr 172.21.15.32:0/2360939110 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:08.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d637fe640 1 --2- 172.21.15.32:0/2360939110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d6415ae80 0x7f7d640cfed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d62ffd640 1 -- 172.21.15.32:0/2360939110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d6415ae80 msgr2=0x7f7d640cfed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:08.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d62ffd640 1 --2- 172.21.15.32:0/2360939110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d6415ae80 0x7f7d640cfed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d62ffd640 1 -- 172.21.15.32:0/2360939110 >> v1:172.21.15.39:6789/0 conn(0x7f7d640b2910 legacy=0x7f7d6406fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.116+0000 7f7d62ffd640 1 -- 172.21.15.32:0/2360939110 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d640cf010 con 0x7f7d640afd50 2023-12-20T06:02:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d62ffd640 1 --2- 172.21.15.32:0/2360939110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d640afd50 0x7f7d640b23d0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009870 tx=0x7f7d5801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afeb3ff53bbd9d75 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d627fc640 1 -- 172.21.15.32:0/2360939110 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580a8a30 con 0x7f7d640afd50 2023-12-20T06:02:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d627fc640 1 -- 172.21.15.32:0/2360939110 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d580a8b90 con 0x7f7d640afd50 2023-12-20T06:02:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d627fc640 1 -- 172.21.15.32:0/2360939110 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d580b0440 con 0x7f7d640afd50 2023-12-20T06:02:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2360939110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d640afd50 msgr2=0x7f7d640b23d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d69b2c640 1 --2- 172.21.15.32:0/2360939110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d640afd50 0x7f7d640b23d0 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7f7d58009870 tx=0x7f7d5801ed30 comp rx=0 tx=0).stop 2023-12-20T06:02:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2360939110 shutdown_connections 2023-12-20T06:02:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d69b2c640 1 --2- 172.21.15.32:0/2360939110 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d640afd50 0x7f7d640b23d0 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d69b2c640 1 --2- 172.21.15.32:0/2360939110 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d6415ae80 0x7f7d640cfed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2360939110 >> 172.21.15.32:0/2360939110 conn(0x7f7d640cd3d0 msgr2=0x7f7d640cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2360939110 shutdown_connections 2023-12-20T06:02:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.120+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2360939110 wait complete. 2023-12-20T06:02:08.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d69b2c640 1 Processor -- start 2023-12-20T06:02:08.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d69b2c640 1 -- start start 2023-12-20T06:02:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d69b2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d640afd50 0x7f7d6406ae30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d69b2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d640b2910 0x7f7d641f2ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d69b2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d6415ae80 0x7f7d641f1d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d69b2c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641f33e0 con 0x7f7d640afd50 2023-12-20T06:02:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d69b2c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7d641f22b0 con 0x7f7d640b2910 2023-12-20T06:02:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d69b2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7d64159950 con 0x7f7d6415ae80 2023-12-20T06:02:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d640afd50 0x7f7d6406ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d637fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d640afd50 0x7f7d6406ae30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50568/0 (socket says 172.21.15.32:50568) 2023-12-20T06:02:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d637fe640 1 -- 172.21.15.32:0/2825939207 learned_addr learned my addr 172.21.15.32:0/2825939207 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d63fff640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d6415ae80 0x7f7d641f1d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d62ffd640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d640b2910 0x7f7d641f2ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d637fe640 1 -- 172.21.15.32:0/2825939207 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d6415ae80 msgr2=0x7f7d641f1d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d637fe640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d6415ae80 0x7f7d641f1d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d637fe640 1 -- 172.21.15.32:0/2825939207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d640b2910 msgr2=0x7f7d641f2ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d637fe640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d640b2910 0x7f7d641f2ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d637fe640 1 -- 172.21.15.32:0/2825939207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7d580094d0 con 0x7f7d640afd50 2023-12-20T06:02:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d637fe640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d640afd50 0x7f7d6406ae30 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400e870 tx=0x7f7d5400ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d4bfff640 1 -- 172.21.15.32:0/2825939207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d54009510 con 0x7f7d640afd50 2023-12-20T06:02:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d4bfff640 1 -- 172.21.15.32:0/2825939207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7d5400eec0 con 0x7f7d640afd50 2023-12-20T06:02:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2825939207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7d64159c30 con 0x7f7d640afd50 2023-12-20T06:02:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d4bfff640 1 -- 172.21.15.32:0/2825939207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7d5400cb40 con 0x7f7d640afd50 2023-12-20T06:02:08.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.124+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2825939207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7d640c4d30 con 0x7f7d640afd50 2023-12-20T06:02:08.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.128+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2825939207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7d6404e000 con 0x7f7d640afd50 2023-12-20T06:02:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.128+0000 7f7d4bfff640 1 -- 172.21.15.32:0/2825939207 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 123) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f7d5400cca0 con 0x7f7d640afd50 2023-12-20T06:02:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.128+0000 7f7d4bfff640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7d500658b0 0x7f7d50067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.128+0000 7f7d4bfff640 1 -- 172.21.15.32:0/2825939207 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6481+0+0 (secure 0 0 0) 0x7f7d54083310 con 0x7f7d640afd50 2023-12-20T06:02:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.128+0000 7f7d62ffd640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7d500658b0 0x7f7d50067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.128+0000 7f7d62ffd640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7d500658b0 0x7f7d50067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7d580b2000 tx=0x7f7d58008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:08.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.132+0000 7f7d4bfff640 1 -- 172.21.15.32:0/2825939207 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7d54050c30 con 0x7f7d640afd50 2023-12-20T06:02:08.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2825939207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7d640b15f0 con 0x7f7d640afd50 2023-12-20T06:02:08.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d4bfff640 1 -- 172.21.15.32:0/2825939207 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 87 v87) v1 ==== 94+0+4717 (secure 0 0 0) 0x7f7d54054250 con 0x7f7d640afd50 2023-12-20T06:02:08.327 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:08.327 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":87,"default_fscid":11,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8396,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1626919510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1626919510},{"type":"v1","addr":"172.21.15.39:6833","nonce":1626919510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":9634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3001408763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3001408763},{"type":"v1","addr":"172.21.15.39:6835","nonce":3001408763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":10020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/2908145202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2908145202},{"type":"v1","addr":"172.21.15.32:6835","nonce":2908145202}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[{"mdsmap":{"epoch":87,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:00:37.604080+0000","modified":"2023-12-20T06:00:40.642479+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10026},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10026":{"gid":10026,"name":"a","rank":0,"incarnation":83,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/2253884246","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":2253884246},{"type":"v1","addr":"172.21.15.32:6837","nonce":2253884246}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[23],"metadata_pool":22,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":11}]} 2023-12-20T06:02:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 87 2023-12-20T06:02:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2825939207 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7d500658b0 msgr2=0x7f7d50067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7d500658b0 0x7f7d50067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f7d580b2000 tx=0x7f7d58008040 comp rx=0 tx=0).stop 2023-12-20T06:02:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2825939207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d640afd50 msgr2=0x7f7d6406ae30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d640afd50 0x7f7d6406ae30 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7f7d5400e870 tx=0x7f7d5400ec70 comp rx=0 tx=0).stop 2023-12-20T06:02:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2825939207 shutdown_connections 2023-12-20T06:02:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7d500658b0 0x7f7d50067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7d6415ae80 0x7f7d641f1d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7d640b2910 0x7f7d641f2ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 --2- 172.21.15.32:0/2825939207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7d640afd50 0x7f7d6406ae30 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2825939207 >> 172.21.15.32:0/2825939207 conn(0x7f7d640cd3d0 msgr2=0x7f7d6415d300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2825939207 shutdown_connections 2023-12-20T06:02:08.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.324+0000 7f7d69b2c640 1 -- 172.21.15.32:0/2825939207 wait complete. 2023-12-20T06:02:08.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:08.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.488+0000 7f09912c4640 1 Processor -- start 2023-12-20T06:02:08.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.488+0000 7f09912c4640 1 -- start start 2023-12-20T06:02:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f09912c4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f098c15aa50 0x7f098c0716f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f09912c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098c06b160 con 0x7f098c15ae30 2023-12-20T06:02:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f09912c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f098c06a550 con 0x7f098c06d840 2023-12-20T06:02:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f09912c4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f098c06a830 con 0x7f098c15aa50 2023-12-20T06:02:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f0983fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f098c15ae30 0x7f098c069d20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39106/0 (socket says 172.21.15.32:39106) 2023-12-20T06:02:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f0983fff640 1 -- 172.21.15.32:0/2024996787 learned_addr learned my addr 172.21.15.32:0/2024996787 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098affd640 1 --2- 172.21.15.32:0/2024996787 >> v2:172.21.15.39:3300/0 conn(0x7f098c15aa50 0x7f098c0716f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2240320736 0 0) 0x7f098c06b160 con 0x7f098c15ae30 2023-12-20T06:02:08.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0978003530 con 0x7f098c15ae30 2023-12-20T06:02:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2764465452 0 0) 0x7f0978003530 con 0x7f098c15ae30 2023-12-20T06:02:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 >> v2:172.21.15.39:3300/0 conn(0x7f098c15aa50 msgr2=0x7f098c0716f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 --2- 172.21.15.32:0/2024996787 >> v2:172.21.15.39:3300/0 conn(0x7f098c15aa50 0x7f098c0716f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 >> v1:172.21.15.39:6789/0 conn(0x7f098c06d840 legacy=0x7f098c06dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c06ab10 con 0x7f098c15ae30 2023-12-20T06:02:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f0974001680 con 0x7f098c15ae30 2023-12-20T06:02:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 >> v1:172.21.15.32:6789/0 conn(0x7f098c15ae30 legacy=0x7f098c069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 --2- 172.21.15.32:0/2024996787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0978003980 0x7f0978003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 --2- 172.21.15.32:0/2024996787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09780047d0 0x7f0978004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 --2- 172.21.15.32:0/2024996787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0978005190 0x7f097800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c06a830 con 0x7f09780047d0 2023-12-20T06:02:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f097800c030 con 0x7f0978005190 2023-12-20T06:02:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f097800c1f0 con 0x7f0978003980 2023-12-20T06:02:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f0983fff640 1 --2- 172.21.15.32:0/2024996787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09780047d0 0x7f0978004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098affd640 1 --2- 172.21.15.32:0/2024996787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0978003980 0x7f0978003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098b7fe640 1 --2- 172.21.15.32:0/2024996787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0978005190 0x7f097800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f0983fff640 1 -- 172.21.15.32:0/2024996787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0978003980 msgr2=0x7f0978003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f0983fff640 1 --2- 172.21.15.32:0/2024996787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0978003980 0x7f0978003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f0983fff640 1 -- 172.21.15.32:0/2024996787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0978005190 msgr2=0x7f097800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f0983fff640 1 --2- 172.21.15.32:0/2024996787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0978005190 0x7f097800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f0983fff640 1 -- 172.21.15.32:0/2024996787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0978003530 con 0x7f09780047d0 2023-12-20T06:02:08.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.492+0000 7f098b7fe640 1 -- 172.21.15.32:0/2024996787 reap_dead start 2023-12-20T06:02:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f0983fff640 1 --2- 172.21.15.32:0/2024996787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09780047d0 0x7f0978004c50 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f097400c240 tx=0x7f097401f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f477d4967c7043e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974005ae0 con 0x7f09780047d0 2023-12-20T06:02:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09740ab070 con 0x7f09780047d0 2023-12-20T06:02:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f098a7fc640 1 -- 172.21.15.32:0/2024996787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974007070 con 0x7f09780047d0 2023-12-20T06:02:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f09912c4640 1 -- 172.21.15.32:0/2024996787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09780047d0 msgr2=0x7f0978004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f09912c4640 1 --2- 172.21.15.32:0/2024996787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09780047d0 0x7f0978004c50 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7f097400c240 tx=0x7f097401f3b0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f09912c4640 1 -- 172.21.15.32:0/2024996787 shutdown_connections 2023-12-20T06:02:08.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f09912c4640 1 --2- 172.21.15.32:0/2024996787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09780047d0 0x7f0978004c50 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f09912c4640 1 -- 172.21.15.32:0/2024996787 >> 172.21.15.32:0/2024996787 conn(0x7f098c0cba50 msgr2=0x7f098c0cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f09912c4640 1 -- 172.21.15.32:0/2024996787 shutdown_connections 2023-12-20T06:02:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f09912c4640 1 -- 172.21.15.32:0/2024996787 wait complete. 2023-12-20T06:02:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f09912c4640 1 Processor -- start 2023-12-20T06:02:08.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.496+0000 7f09912c4640 1 -- start start 2023-12-20T06:02:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09912c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09780047d0 0x7f098c09fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09912c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f098c0c2640 0x7f098c0b09f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09912c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f098c0c2b00 0x7f098c0b0f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09912c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c147350 con 0x7f098c0c2640 2023-12-20T06:02:08.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09912c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0c3330 con 0x7f098c0c2b00 2023-12-20T06:02:08.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09912c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f098c0a02b0 con 0x7f09780047d0 2023-12-20T06:02:08.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f0983fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f098c0c2640 0x7f098c0b09f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f0983fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f098c0c2640 0x7f098c0b09f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50596/0 (socket says 172.21.15.32:50596) 2023-12-20T06:02:08.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f0983fff640 1 -- 172.21.15.32:0/898133729 learned_addr learned my addr 172.21.15.32:0/898133729 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:08.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f098affd640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09780047d0 0x7f098c09fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f098b7fe640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f098c0c2b00 0x7f098c0b0f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f0983fff640 1 -- 172.21.15.32:0/898133729 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09780047d0 msgr2=0x7f098c09fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f0983fff640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09780047d0 0x7f098c09fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f0983fff640 1 -- 172.21.15.32:0/898133729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f098c0c2b00 msgr2=0x7f098c0b0f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f0983fff640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f098c0c2b00 0x7f098c0b0f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f0983fff640 1 -- 172.21.15.32:0/898133729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f098c0c3050 con 0x7f098c0c2640 2023-12-20T06:02:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f098b7fe640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f098c0c2b00 0x7f098c0b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f0983fff640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f098c0c2640 0x7f098c0b09f0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f097401f1e0 tx=0x7f0978005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09837fe640 1 -- 172.21.15.32:0/898133729 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974001640 con 0x7f098c0c2640 2023-12-20T06:02:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09837fe640 1 -- 172.21.15.32:0/898133729 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0974007070 con 0x7f098c0c2640 2023-12-20T06:02:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09837fe640 1 -- 172.21.15.32:0/898133729 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0974007400 con 0x7f098c0c2640 2023-12-20T06:02:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09912c4640 1 -- 172.21.15.32:0/898133729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f098c0c5490 con 0x7f098c0c2640 2023-12-20T06:02:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09912c4640 1 -- 172.21.15.32:0/898133729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f098c0c5a50 con 0x7f098c0c2640 2023-12-20T06:02:08.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.500+0000 7f09912c4640 1 -- 172.21.15.32:0/898133729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f098c0660f0 con 0x7f098c0c2640 2023-12-20T06:02:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.504+0000 7f09837fe640 1 -- 172.21.15.32:0/898133729 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 123) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f09740b9030 con 0x7f098c0c2640 2023-12-20T06:02:08.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.504+0000 7f09837fe640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f096c0658b0 0x7f096c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.504+0000 7f09837fe640 1 -- 172.21.15.32:0/898133729 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6481+0+0 (secure 0 0 0) 0x7f09740b7070 con 0x7f098c0c2640 2023-12-20T06:02:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.504+0000 7f098affd640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f096c0658b0 0x7f096c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.504+0000 7f098affd640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f096c0658b0 0x7f096c067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f098c069560 tx=0x7f097c009190 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:08.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.508+0000 7f09837fe640 1 -- 172.21.15.32:0/898133729 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f09740ed7a0 con 0x7f098c0c2640 2023-12-20T06:02:08.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.668+0000 7f09912c4640 1 -- 172.21.15.32:0/898133729 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f098c0c52c0 con 0x7f098c0c2640 2023-12-20T06:02:08.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.668+0000 7f09837fe640 1 -- 172.21.15.32:0/898133729 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v196) v1 ==== 74+0+17198 (secure 0 0 0) 0x7f09740c1030 con 0x7f098c0c2640 2023-12-20T06:02:08.671 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:08.671 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":196,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:06.926932+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":22,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:00:36.162516+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"191","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":23,"pool_name":"cephfs_data","create_time":"2023-12-20T06:00:36.593153+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"195","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":5,"snap_epoch":195,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1840226039":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/3016054758":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/3042178755":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/1302755445":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/3527982629":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/3239744916":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/2437361294":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/1388061423":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/968197028":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/2674032592":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/4037417795":"2023-12-21T06:00:49.693740+0000","172.21.15.32:6833/64546":"2023-12-21T06:00:49.693740+0000","172.21.15.32:6832/64546":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/555339143":"2023-12-21T06:02:06.926874+0000","172.21.15.32:0/4289008383":"2023-12-21T06:00:49.693740+0000","172.21.15.32:0/1643902464":"2023-12-21T06:00:49.693740+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]},{"pool":23,"snaps":[{"begin":2,"length":4}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:08.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 -- 172.21.15.32:0/898133729 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f096c0658b0 msgr2=0x7f096c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f096c0658b0 0x7f096c067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f098c069560 tx=0x7f097c009190 comp rx=0 tx=0).stop 2023-12-20T06:02:08.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 -- 172.21.15.32:0/898133729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f098c0c2640 msgr2=0x7f098c0b09f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f098c0c2640 0x7f098c0b09f0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x7f097401f1e0 tx=0x7f0978005800 comp rx=0 tx=0).stop 2023-12-20T06:02:08.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 -- 172.21.15.32:0/898133729 shutdown_connections 2023-12-20T06:02:08.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f096c0658b0 0x7f096c067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f098c0c2b00 0x7f098c0b0f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f098c0c2640 0x7f098c0b09f0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 --2- 172.21.15.32:0/898133729 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09780047d0 0x7f098c09fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 -- 172.21.15.32:0/898133729 >> 172.21.15.32:0/898133729 conn(0x7f098c0cba50 msgr2=0x7f098c0cced0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:08.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 -- 172.21.15.32:0/898133729 shutdown_connections 2023-12-20T06:02:08.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.672+0000 7f09912c4640 1 -- 172.21.15.32:0/898133729 wait complete. 2023-12-20T06:02:08.693 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T06:02:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27f1b6d640 1 Processor -- start 2023-12-20T06:02:08.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27f1b6d640 1 -- start start 2023-12-20T06:02:08.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27f1b6d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27ec0aece0 0x7f27ec0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27f1b6d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f27ec0b0b30 0x7f27ec0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27f1b6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f27ec0c98c0 con 0x7f27ec1599e0 2023-12-20T06:02:08.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27f1b6d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ec0c9ba0 con 0x7f27ec0aece0 2023-12-20T06:02:08.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27f1b6d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f27ec0c9e80 con 0x7f27ec0b0b30 2023-12-20T06:02:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27eaffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27ec0aece0 0x7f27ec0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27eaffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f27ec0aece0 0x7f27ec0b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45856/0 (socket says 172.21.15.32:45856) 2023-12-20T06:02:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27eaffd640 1 -- 172.21.15.32:0/2081951858 learned_addr learned my addr 172.21.15.32:0/2081951858 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27ebfff640 1 --2- 172.21.15.32:0/2081951858 >> v2:172.21.15.39:3300/0 conn(0x7f27ec0b0b30 0x7f27ec0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27eaffd640 1 -- 172.21.15.32:0/2081951858 >> v2:172.21.15.39:3300/0 conn(0x7f27ec0b0b30 msgr2=0x7f27ec0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27eaffd640 1 --2- 172.21.15.32:0/2081951858 >> v2:172.21.15.39:3300/0 conn(0x7f27ec0b0b30 0x7f27ec0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27eaffd640 1 -- 172.21.15.32:0/2081951858 >> v1:172.21.15.39:6789/0 conn(0x7f27ec1599e0 legacy=0x7f27ec0c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:08.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.836+0000 7f27eaffd640 1 -- 172.21.15.32:0/2081951858 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27ec0ca160 con 0x7f27ec0aece0 2023-12-20T06:02:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27eaffd640 1 --2- 172.21.15.32:0/2081951858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27ec0aece0 0x7f27ec0b05f0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0b0060 tx=0x7f27e001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=322707c0f840b788 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27ea7fc640 1 -- 172.21.15.32:0/2081951858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00ab610 con 0x7f27ec0aece0 2023-12-20T06:02:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27ea7fc640 1 -- 172.21.15.32:0/2081951858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e00ab770 con 0x7f27ec0aece0 2023-12-20T06:02:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27ea7fc640 1 -- 172.21.15.32:0/2081951858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e00abb00 con 0x7f27ec0aece0 2023-12-20T06:02:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27f1b6d640 1 -- 172.21.15.32:0/2081951858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27ec0aece0 msgr2=0x7f27ec0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27f1b6d640 1 --2- 172.21.15.32:0/2081951858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27ec0aece0 0x7f27ec0b05f0 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7f27ec0b0060 tx=0x7f27e001c500 comp rx=0 tx=0).stop 2023-12-20T06:02:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27f1b6d640 1 -- 172.21.15.32:0/2081951858 shutdown_connections 2023-12-20T06:02:08.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27f1b6d640 1 --2- 172.21.15.32:0/2081951858 >> v2:172.21.15.39:3300/0 conn(0x7f27ec0b0b30 0x7f27ec0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27f1b6d640 1 --2- 172.21.15.32:0/2081951858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27ec0aece0 0x7f27ec0b05f0 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27f1b6d640 1 -- 172.21.15.32:0/2081951858 >> 172.21.15.32:0/2081951858 conn(0x7f27ec06a320 msgr2=0x7f27ec0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27f1b6d640 1 -- 172.21.15.32:0/2081951858 shutdown_connections 2023-12-20T06:02:08.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.840+0000 7f27f1b6d640 1 -- 172.21.15.32:0/2081951858 wait complete. 2023-12-20T06:02:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27f1b6d640 1 Processor -- start 2023-12-20T06:02:08.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27f1b6d640 1 -- start start 2023-12-20T06:02:08.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27f1b6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0aece0 0x7f27ec145f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27f1b6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27ec0b0b30 0x7f27ec14ba40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27f1b6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27ec1599e0 0x7f27ec0cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27f1b6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec14a720 con 0x7f27ec1599e0 2023-12-20T06:02:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27f1b6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec1464c0 con 0x7f27ec0aece0 2023-12-20T06:02:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27f1b6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f27ec14c340 con 0x7f27ec0b0b30 2023-12-20T06:02:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27ec1599e0 0x7f27ec0cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27ebfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27ec1599e0 0x7f27ec0cdf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45864/0 (socket says 172.21.15.32:45864) 2023-12-20T06:02:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27ebfff640 1 -- 172.21.15.32:0/3750061244 learned_addr learned my addr 172.21.15.32:0/3750061244 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27eb7fe640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0aece0 0x7f27ec145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27eaffd640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27ec0b0b30 0x7f27ec14ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27ebfff640 1 -- 172.21.15.32:0/3750061244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27ec0b0b30 msgr2=0x7f27ec14ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27ebfff640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27ec0b0b30 0x7f27ec14ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27ebfff640 1 -- 172.21.15.32:0/3750061244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0aece0 msgr2=0x7f27ec145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27ebfff640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0aece0 0x7f27ec145f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27ebfff640 1 -- 172.21.15.32:0/3750061244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f27e00094d0 con 0x7f27ec1599e0 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27eb7fe640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0aece0 0x7f27ec145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27ebfff640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27ec1599e0 0x7f27ec0cdf80 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f27e4007e40 tx=0x7f27e400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27d3fff640 1 -- 172.21.15.32:0/3750061244 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e4014620 con 0x7f27ec1599e0 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27d3fff640 1 -- 172.21.15.32:0/3750061244 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27e400ee90 con 0x7f27ec1599e0 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27f1b6d640 1 -- 172.21.15.32:0/3750061244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f27ec14a930 con 0x7f27ec1599e0 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27d3fff640 1 -- 172.21.15.32:0/3750061244 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27e40149e0 con 0x7f27ec1599e0 2023-12-20T06:02:08.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.844+0000 7f27f1b6d640 1 -- 172.21.15.32:0/3750061244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27ec14aef0 con 0x7f27ec1599e0 2023-12-20T06:02:08.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.848+0000 7f27f1b6d640 1 -- 172.21.15.32:0/3750061244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f27ec14b0e0 con 0x7f27ec1599e0 2023-12-20T06:02:08.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.848+0000 7f27d3fff640 1 -- 172.21.15.32:0/3750061244 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 123) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f27e4014b40 con 0x7f27ec1599e0 2023-12-20T06:02:08.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.848+0000 7f27d3fff640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f27d80658b0 0x7f27d8067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:08.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.848+0000 7f27d3fff640 1 -- 172.21.15.32:0/3750061244 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(196..196 src has 1..196) v4 ==== 6481+0+0 (secure 0 0 0) 0x7f27e4083ab0 con 0x7f27ec1599e0 2023-12-20T06:02:08.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.848+0000 7f27eb7fe640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f27d80658b0 0x7f27d8067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:08.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.848+0000 7f27eb7fe640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f27d80658b0 0x7f27d8067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007fd0 tx=0x7f27dc009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:08.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.852+0000 7f27d3fff640 1 -- 172.21.15.32:0/3750061244 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f27e40513d0 con 0x7f27ec1599e0 2023-12-20T06:02:08.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:08.944+0000 7f27d3fff640 1 -- 172.21.15.32:0/3750061244 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 124) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f27e40517a0 con 0x7f27ec1599e0 2023-12-20T06:02:09.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.052+0000 7f27f1b6d640 1 -- 172.21.15.32:0/3750061244 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f27ec04e000 con 0x7f27ec1599e0 2023-12-20T06:02:09.056 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T06:02:09.056+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T06:02:09.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.064+0000 7f27d3fff640 1 -- 172.21.15.32:0/3750061244 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v88) v1 ==== 161+0+0 (secure 0 0 0) 0x7f27e407fb60 con 0x7f27ec1599e0 2023-12-20T06:02:09.068 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T06:02:09.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 -- 172.21.15.32:0/3750061244 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f27d80658b0 msgr2=0x7f27d8067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:09.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f27d80658b0 0x7f27d8067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f27dc007fd0 tx=0x7f27dc009040 comp rx=0 tx=0).stop 2023-12-20T06:02:09.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 -- 172.21.15.32:0/3750061244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27ec1599e0 msgr2=0x7f27ec0cdf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:09.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27ec1599e0 0x7f27ec0cdf80 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7f27e4007e40 tx=0x7f27e400e4b0 comp rx=0 tx=0).stop 2023-12-20T06:02:09.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 -- 172.21.15.32:0/3750061244 shutdown_connections 2023-12-20T06:02:09.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f27d80658b0 0x7f27d8067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:09.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f27ec1599e0 0x7f27ec0cdf80 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:09.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f27ec0b0b30 0x7f27ec14ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:09.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 --2- 172.21.15.32:0/3750061244 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27ec0aece0 0x7f27ec145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:09.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 -- 172.21.15.32:0/3750061244 >> 172.21.15.32:0/3750061244 conn(0x7f27ec06a320 msgr2=0x7f27ec158a30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:09.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 -- 172.21.15.32:0/3750061244 shutdown_connections 2023-12-20T06:02:09.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.068+0000 7f27f1b6d640 1 -- 172.21.15.32:0/3750061244 wait complete. 2023-12-20T06:02:09.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T06:02:09.131 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T06:02:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f026e57b640 1 Processor -- start 2023-12-20T06:02:09.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f026e57b640 1 -- start start 2023-12-20T06:02:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f026e57b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02680cb050 0x7f02680cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f026e57b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0268065d40 0x7f0268161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f026e57b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f026806a1f0 con 0x7f026806ca60 2023-12-20T06:02:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f026e57b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0268161860 con 0x7f02680cb050 2023-12-20T06:02:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f026e57b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f026806a440 con 0x7f0268065d40 2023-12-20T06:02:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f0267fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02680cb050 0x7f02680cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f0267fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f02680cb050 0x7f02680cb430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45892/0 (socket says 172.21.15.32:45892) 2023-12-20T06:02:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f0267fff640 1 -- 172.21.15.32:0/2790222579 learned_addr learned my addr 172.21.15.32:0/2790222579 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:09.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f02677fe640 1 --2- 172.21.15.32:0/2790222579 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0268065d40 0x7f0268161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.236+0000 7f0267fff640 1 -- 172.21.15.32:0/2790222579 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0268065d40 msgr2=0x7f0268161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f0267fff640 1 --2- 172.21.15.32:0/2790222579 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0268065d40 0x7f0268161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f0267fff640 1 -- 172.21.15.32:0/2790222579 >> v1:172.21.15.39:6789/0 conn(0x7f026806ca60 legacy=0x7f0268070960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:09.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f0267fff640 1 -- 172.21.15.32:0/2790222579 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f026806a720 con 0x7f02680cb050 2023-12-20T06:02:09.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f0267fff640 1 --2- 172.21.15.32:0/2790222579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02680cb050 0x7f02680cb430 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f0258009870 tx=0x7f025801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2916e150961556d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:09.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f0266ffd640 1 -- 172.21.15.32:0/2790222579 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02580aa670 con 0x7f02680cb050 2023-12-20T06:02:09.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f0266ffd640 1 -- 172.21.15.32:0/2790222579 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02580aa7d0 con 0x7f02680cb050 2023-12-20T06:02:09.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f0266ffd640 1 -- 172.21.15.32:0/2790222579 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02580aab60 con 0x7f02680cb050 2023-12-20T06:02:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f026e57b640 1 -- 172.21.15.32:0/2790222579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02680cb050 msgr2=0x7f02680cb430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f026e57b640 1 --2- 172.21.15.32:0/2790222579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02680cb050 0x7f02680cb430 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7f0258009870 tx=0x7f025801c510 comp rx=0 tx=0).stop 2023-12-20T06:02:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f026e57b640 1 -- 172.21.15.32:0/2790222579 shutdown_connections 2023-12-20T06:02:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f026e57b640 1 --2- 172.21.15.32:0/2790222579 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0268065d40 0x7f0268161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f026e57b640 1 --2- 172.21.15.32:0/2790222579 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f02680cb050 0x7f02680cb430 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f026e57b640 1 -- 172.21.15.32:0/2790222579 >> 172.21.15.32:0/2790222579 conn(0x7f02680cf8d0 msgr2=0x7f02680cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.240+0000 7f026e57b640 1 -- 172.21.15.32:0/2790222579 shutdown_connections 2023-12-20T06:02:09.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f026e57b640 1 -- 172.21.15.32:0/2790222579 wait complete. 2023-12-20T06:02:09.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f026e57b640 1 Processor -- start 2023-12-20T06:02:09.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f026e57b640 1 -- start start 2023-12-20T06:02:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f026e57b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268065d40 0x7f02680c52a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f026e57b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f026806ca60 0x7f026815ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f026e57b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02680cb050 0x7f02680b0b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f026e57b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02680c57e0 con 0x7f0268065d40 2023-12-20T06:02:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f026e57b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f02681446d0 con 0x7f026806ca60 2023-12-20T06:02:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f026e57b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f02681449b0 con 0x7f02680cb050 2023-12-20T06:02:09.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f0267fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268065d40 0x7f02680c52a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f0267fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268065d40 0x7f02680c52a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45908/0 (socket says 172.21.15.32:45908) 2023-12-20T06:02:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f0267fff640 1 -- 172.21.15.32:0/4116189691 learned_addr learned my addr 172.21.15.32:0/4116189691 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f02677fe640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f026806ca60 0x7f026815ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f026cb18640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02680cb050 0x7f02680b0b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f0267fff640 1 -- 172.21.15.32:0/4116189691 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02680cb050 msgr2=0x7f02680b0b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f0267fff640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02680cb050 0x7f02680b0b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f0267fff640 1 -- 172.21.15.32:0/4116189691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f026806ca60 msgr2=0x7f026815ba60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f0267fff640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f026806ca60 0x7f026815ba60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f0267fff640 1 -- 172.21.15.32:0/4116189691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f02580094d0 con 0x7f0268065d40 2023-12-20T06:02:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f026cb18640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02680cb050 0x7f02680b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:09.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.244+0000 7f0267fff640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268065d40 0x7f02680c52a0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f02580ac040 tx=0x7f025801ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.248+0000 7f0264ff9640 1 -- 172.21.15.32:0/4116189691 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02580b3800 con 0x7f0268065d40 2023-12-20T06:02:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.248+0000 7f0264ff9640 1 -- 172.21.15.32:0/4116189691 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f02580b3960 con 0x7f0268065d40 2023-12-20T06:02:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.248+0000 7f0264ff9640 1 -- 172.21.15.32:0/4116189691 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f02580b3d10 con 0x7f0268065d40 2023-12-20T06:02:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.248+0000 7f026e57b640 1 -- 172.21.15.32:0/4116189691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0268144c90 con 0x7f0268065d40 2023-12-20T06:02:09.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.248+0000 7f026e57b640 1 -- 172.21.15.32:0/4116189691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0268145250 con 0x7f0268065d40 2023-12-20T06:02:09.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.248+0000 7f026e57b640 1 -- 172.21.15.32:0/4116189691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0268145440 con 0x7f0268065d40 2023-12-20T06:02:09.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.248+0000 7f0264ff9640 1 -- 172.21.15.32:0/4116189691 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 124) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f02580b2380 con 0x7f0268065d40 2023-12-20T06:02:09.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.248+0000 7f0264ff9640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0254065af0 0x7f0254067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:09.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.248+0000 7f0264ff9640 1 -- 172.21.15.32:0/4116189691 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(197..197 src has 1..197) v4 ==== 6567+0+0 (secure 0 0 0) 0x7f025811de80 con 0x7f0268065d40 2023-12-20T06:02:09.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.248+0000 7f02677fe640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0254065af0 0x7f0254067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:09.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.252+0000 7f02677fe640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0254065af0 0x7f0254067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f02681473b0 tx=0x7f025c009190 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:09.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.252+0000 7f0264ff9640 1 -- 172.21.15.32:0/4116189691 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f02580eb4a0 con 0x7f0268065d40 2023-12-20T06:02:09.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:09.444+0000 7f026e57b640 1 -- 172.21.15.32:0/4116189691 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f026804e000 con 0x7f0268065d40 2023-12-20T06:02:10.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.068+0000 7f0264ff9640 1 -- 172.21.15.32:0/4116189691 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v89) v1 ==== 104+0+0 (secure 0 0 0) 0x7f02580eeac0 con 0x7f0268065d40 2023-12-20T06:02:10.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.068+0000 7f026e57b640 1 -- 172.21.15.32:0/4116189691 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0254065af0 msgr2=0x7f0254067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:10.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.068+0000 7f026e57b640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0254065af0 0x7f0254067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f02681473b0 tx=0x7f025c009190 comp rx=0 tx=0).stop 2023-12-20T06:02:10.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.068+0000 7f026e57b640 1 -- 172.21.15.32:0/4116189691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268065d40 msgr2=0x7f02680c52a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:10.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.068+0000 7f026e57b640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268065d40 0x7f02680c52a0 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7f02580ac040 tx=0x7f025801ad50 comp rx=0 tx=0).stop 2023-12-20T06:02:10.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.072+0000 7f026e57b640 1 -- 172.21.15.32:0/4116189691 shutdown_connections 2023-12-20T06:02:10.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.072+0000 7f026e57b640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0254065af0 0x7f0254067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:10.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.072+0000 7f026e57b640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f02680cb050 0x7f02680b0b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:10.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.072+0000 7f026e57b640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f026806ca60 0x7f026815ba60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:10.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.072+0000 7f026e57b640 1 --2- 172.21.15.32:0/4116189691 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0268065d40 0x7f02680c52a0 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:10.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.072+0000 7f026e57b640 1 -- 172.21.15.32:0/4116189691 >> 172.21.15.32:0/4116189691 conn(0x7f02680cf8d0 msgr2=0x7f0268070960 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:10.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.072+0000 7f026e57b640 1 -- 172.21.15.32:0/4116189691 shutdown_connections 2023-12-20T06:02:10.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.072+0000 7f026e57b640 1 -- 172.21.15.32:0/4116189691 wait complete. 2023-12-20T06:02:10.093 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T06:02:10.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.280+0000 7f6f3ef39640 1 Processor -- start 2023-12-20T06:02:10.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.280+0000 7f6f3ef39640 1 -- start start 2023-12-20T06:02:10.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.280+0000 7f6f3ef39640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6f380ce290 0x7f6f38070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.280+0000 7f6f3ef39640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f38159630 con 0x7f6f38070770 2023-12-20T06:02:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.280+0000 7f6f3ef39640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6f380c9480 con 0x7f6f38159250 2023-12-20T06:02:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.280+0000 7f6f3ef39640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6f38070ba0 con 0x7f6f380ce290 2023-12-20T06:02:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.280+0000 7f6f3d4d6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6f38070770 0x7f6f380cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41234/0 (socket says 172.21.15.32:41234) 2023-12-20T06:02:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.280+0000 7f6f3d4d6640 1 -- 172.21.15.32:0/2051857742 learned_addr learned my addr 172.21.15.32:0/2051857742 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.280+0000 7f6f37fff640 1 --2- 172.21.15.32:0/2051857742 >> v2:172.21.15.39:3300/0 conn(0x7f6f380ce290 0x7f6f38070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.280+0000 7f6f377fe640 1 -- 172.21.15.32:0/2051857742 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2280064733 0 0) 0x7f6f38159630 con 0x7f6f38070770 2023-12-20T06:02:10.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f377fe640 1 -- 172.21.15.32:0/2051857742 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f28003530 con 0x7f6f38070770 2023-12-20T06:02:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f377fe640 1 -- 172.21.15.32:0/2051857742 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4185851173 0 0) 0x7f6f380c9480 con 0x7f6f38159250 2023-12-20T06:02:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f377fe640 1 -- 172.21.15.32:0/2051857742 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6f38159630 con 0x7f6f38159250 2023-12-20T06:02:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f37fff640 1 -- 172.21.15.32:0/2051857742 >> v1:172.21.15.39:6789/0 conn(0x7f6f38159250 legacy=0x7f6f380caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f37fff640 1 -- 172.21.15.32:0/2051857742 >> v1:172.21.15.32:6789/0 conn(0x7f6f38070770 legacy=0x7f6f380cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f377fe640 1 -- 172.21.15.32:0/2051857742 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 4237909636 0 0) 0x7f6f28003530 con 0x7f6f38070770 2023-12-20T06:02:10.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f37fff640 1 -- 172.21.15.32:0/2051857742 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f3806cf90 con 0x7f6f380ce290 2023-12-20T06:02:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f37fff640 1 --2- 172.21.15.32:0/2051857742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f380ce290 0x7f6f38070230 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6f240097c0 tx=0x7f6f2401c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b63ef3d964fd9180 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:10.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f377fe640 1 -- 172.21.15.32:0/2051857742 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f30001a00 con 0x7f6f380ce290 2023-12-20T06:02:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f377fe640 1 -- 172.21.15.32:0/2051857742 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f240a7050 con 0x7f6f380ce290 2023-12-20T06:02:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f377fe640 1 -- 172.21.15.32:0/2051857742 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f240ab620 con 0x7f6f380ce290 2023-12-20T06:02:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f3ef39640 1 -- 172.21.15.32:0/2051857742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f380ce290 msgr2=0x7f6f38070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.284+0000 7f6f3ef39640 1 --2- 172.21.15.32:0/2051857742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f380ce290 0x7f6f38070230 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f6f240097c0 tx=0x7f6f2401c3a0 comp rx=0 tx=0).stop 2023-12-20T06:02:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 -- 172.21.15.32:0/2051857742 shutdown_connections 2023-12-20T06:02:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 --2- 172.21.15.32:0/2051857742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f380ce290 0x7f6f38070230 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 -- 172.21.15.32:0/2051857742 >> 172.21.15.32:0/2051857742 conn(0x7f6f3806a510 msgr2=0x7f6f380ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 -- 172.21.15.32:0/2051857742 shutdown_connections 2023-12-20T06:02:10.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 -- 172.21.15.32:0/2051857742 wait complete. 2023-12-20T06:02:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 Processor -- start 2023-12-20T06:02:10.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 -- start start 2023-12-20T06:02:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f38070770 0x7f6f3814b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f380ce290 0x7f6f3814c060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f38159250 0x7f6f380b2f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3814c5a0 con 0x7f6f38070770 2023-12-20T06:02:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3814c7d0 con 0x7f6f38159250 2023-12-20T06:02:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ef39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6f3814cab0 con 0x7f6f380ce290 2023-12-20T06:02:10.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ccd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f38070770 0x7f6f3814b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ccd5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f38070770 0x7f6f3814b730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45912/0 (socket says 172.21.15.32:45912) 2023-12-20T06:02:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ccd5640 1 -- 172.21.15.32:0/3093319388 learned_addr learned my addr 172.21.15.32:0/3093319388 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f37fff640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f380ce290 0x7f6f3814c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3d4d6640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f38159250 0x7f6f380b2f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ccd5640 1 -- 172.21.15.32:0/3093319388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f380ce290 msgr2=0x7f6f3814c060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ccd5640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f380ce290 0x7f6f3814c060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ccd5640 1 -- 172.21.15.32:0/3093319388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f38159250 msgr2=0x7f6f380b2f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ccd5640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f38159250 0x7f6f380b2f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.288+0000 7f6f3ccd5640 1 -- 172.21.15.32:0/3093319388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6f24009420 con 0x7f6f38070770 2023-12-20T06:02:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.292+0000 7f6f3ccd5640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f38070770 0x7f6f3814b730 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c430 tx=0x7f6f2c00c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:10.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.292+0000 7f6f357fa640 1 -- 172.21.15.32:0/3093319388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c0052d0 con 0x7f6f38070770 2023-12-20T06:02:10.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.292+0000 7f6f357fa640 1 -- 172.21.15.32:0/3093319388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6f2c005450 con 0x7f6f38070770 2023-12-20T06:02:10.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.292+0000 7f6f3ef39640 1 -- 172.21.15.32:0/3093319388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6f3814cd90 con 0x7f6f38070770 2023-12-20T06:02:10.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.292+0000 7f6f357fa640 1 -- 172.21.15.32:0/3093319388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6f2c003840 con 0x7f6f38070770 2023-12-20T06:02:10.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.292+0000 7f6f3ef39640 1 -- 172.21.15.32:0/3093319388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6f3814d350 con 0x7f6f38070770 2023-12-20T06:02:10.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.292+0000 7f6f3ef39640 1 -- 172.21.15.32:0/3093319388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6f3814d540 con 0x7f6f38070770 2023-12-20T06:02:10.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.292+0000 7f6f357fa640 1 -- 172.21.15.32:0/3093319388 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 124) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f6f2c0039a0 con 0x7f6f38070770 2023-12-20T06:02:10.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.292+0000 7f6f357fa640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6f1c065ac0 0x7f6f1c067f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:10.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.292+0000 7f6f357fa640 1 -- 172.21.15.32:0/3093319388 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(198..198 src has 1..198) v4 ==== 6567+0+0 (secure 0 0 0) 0x7f6f2c082b50 con 0x7f6f38070770 2023-12-20T06:02:10.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.292+0000 7f6f37fff640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6f1c065ac0 0x7f6f1c067f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:10.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.296+0000 7f6f37fff640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6f1c065ac0 0x7f6f1c067f80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6f240ad000 tx=0x7f6f240a6000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:10.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.296+0000 7f6f357fa640 1 -- 172.21.15.32:0/3093319388 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6f2c050200 con 0x7f6f38070770 2023-12-20T06:02:10.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:10.456+0000 7f6f3ef39640 1 -- 172.21.15.32:0/3093319388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6f3804e000 con 0x7f6f38070770 2023-12-20T06:02:11.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.084+0000 7f6f357fa640 1 -- 172.21.15.32:0/3093319388 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6f2c0505d0 con 0x7f6f38070770 2023-12-20T06:02:11.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.084+0000 7f6f357fa640 1 -- 172.21.15.32:0/3093319388 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v199) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6f2c06d7d0 con 0x7f6f38070770 2023-12-20T06:02:11.088 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T06:02:11.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 -- 172.21.15.32:0/3093319388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6f1c065ac0 msgr2=0x7f6f1c067f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6f1c065ac0 0x7f6f1c067f80 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6f240ad000 tx=0x7f6f240a6000 comp rx=0 tx=0).stop 2023-12-20T06:02:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 -- 172.21.15.32:0/3093319388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f38070770 msgr2=0x7f6f3814b730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f38070770 0x7f6f3814b730 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7f6f2c00c430 tx=0x7f6f2c00c8f0 comp rx=0 tx=0).stop 2023-12-20T06:02:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 -- 172.21.15.32:0/3093319388 shutdown_connections 2023-12-20T06:02:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6f1c065ac0 0x7f6f1c067f80 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6f38159250 0x7f6f380b2f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6f380ce290 0x7f6f3814c060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 --2- 172.21.15.32:0/3093319388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6f38070770 0x7f6f3814b730 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 -- 172.21.15.32:0/3093319388 >> 172.21.15.32:0/3093319388 conn(0x7f6f3806a510 msgr2=0x7f6f381512c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 -- 172.21.15.32:0/3093319388 shutdown_connections 2023-12-20T06:02:11.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.088+0000 7f6f3ef39640 1 -- 172.21.15.32:0/3093319388 wait complete. 2023-12-20T06:02:11.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T06:02:11.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f8400f09640 1 Processor -- start 2023-12-20T06:02:11.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f8400f09640 1 -- start start 2023-12-20T06:02:11.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f8400f09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc159af0 0x7f83fc0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:11.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f8400f09640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc06dd50 con 0x7f83fc0b0f00 2023-12-20T06:02:11.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f8400f09640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83fc0cc940 con 0x7f83fc06d2d0 2023-12-20T06:02:11.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f8400f09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0ccc20 con 0x7f83fc159af0 2023-12-20T06:02:11.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83fad9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f83fc0b0f00 0x7f83fc0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41242/0 (socket says 172.21.15.32:41242) 2023-12-20T06:02:11.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83fad9d640 1 -- 172.21.15.32:0/3846559826 learned_addr learned my addr 172.21.15.32:0/3846559826 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:11.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83fa59c640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc159af0 0x7f83fc0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:11.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 470014438 0 0) 0x7f83fc06dd50 con 0x7f83fc0b0f00 2023-12-20T06:02:11.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83e8003530 con 0x7f83fc0b0f00 2023-12-20T06:02:11.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1881926944 0 0) 0x7f83fc0cc940 con 0x7f83fc06d2d0 2023-12-20T06:02:11.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83fc06dd50 con 0x7f83fc06d2d0 2023-12-20T06:02:11.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1153366322 0 0) 0x7f83e8003530 con 0x7f83fc0b0f00 2023-12-20T06:02:11.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc159af0 msgr2=0x7f83fc0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:11.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83f9d9b640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc159af0 0x7f83fc0b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:11.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 >> v1:172.21.15.39:6789/0 conn(0x7f83fc06d2d0 legacy=0x7f83fc0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:11.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc0ccf00 con 0x7f83fc0b0f00 2023-12-20T06:02:11.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f83f4001a00 con 0x7f83fc0b0f00 2023-12-20T06:02:11.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.264+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 >> v1:172.21.15.32:6789/0 conn(0x7f83fc0b0f00 legacy=0x7f83fc0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:11.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83f9d9b640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8003b40 0x7f83e8003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:11.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83f9d9b640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc159af0 0x7f83e80047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:11.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83f9d9b640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83e8004d10 0x7f83e8005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:11.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0ccc20 con 0x7f83e8004d10 2023-12-20T06:02:11.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83e800ba40 con 0x7f83e8003b40 2023-12-20T06:02:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83e800bc00 con 0x7f83fc159af0 2023-12-20T06:02:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83fad9d640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83e8004d10 0x7f83e8005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83f3fff640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc159af0 0x7f83e80047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83fa59c640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8003b40 0x7f83e8003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83fad9d640 1 -- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc159af0 msgr2=0x7f83e80047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83fad9d640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc159af0 0x7f83e80047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83fad9d640 1 -- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8003b40 msgr2=0x7f83e8003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83fad9d640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8003b40 0x7f83e8003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83fad9d640 1 -- 172.21.15.32:0/3846559826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e8003530 con 0x7f83e8004d10 2023-12-20T06:02:11.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83fa59c640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83e8003b40 0x7f83e8003f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83fad9d640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83e8004d10 0x7f83e8005190 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f83f400a6c0 tx=0x7f83f401f900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=486cf3412ff9e4ad server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4007330 con 0x7f83e8004d10 2023-12-20T06:02:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83f4007490 con 0x7f83e8004d10 2023-12-20T06:02:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.268+0000 7f83f9d9b640 1 -- 172.21.15.32:0/3846559826 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83f4007880 con 0x7f83e8004d10 2023-12-20T06:02:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 -- 172.21.15.32:0/3846559826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83e8004d10 msgr2=0x7f83e8005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 --2- 172.21.15.32:0/3846559826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83e8004d10 0x7f83e8005190 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7f83f400a6c0 tx=0x7f83f401f900 comp rx=0 tx=0).stop 2023-12-20T06:02:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f83fad9d640 1 -- 172.21.15.32:0/3846559826 reap_dead start 2023-12-20T06:02:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 -- 172.21.15.32:0/3846559826 shutdown_connections 2023-12-20T06:02:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 -- 172.21.15.32:0/3846559826 >> 172.21.15.32:0/3846559826 conn(0x7f83fc06a510 msgr2=0x7f83fc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:11.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 -- 172.21.15.32:0/3846559826 shutdown_connections 2023-12-20T06:02:11.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 -- 172.21.15.32:0/3846559826 wait complete. 2023-12-20T06:02:11.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 Processor -- start 2023-12-20T06:02:11.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 -- start start 2023-12-20T06:02:11.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83fc14f160 0x7f83fc15c110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83fc0c5b30 0x7f83fc0c7fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc157e90 0x7f83fc158310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc158850 con 0x7f83fc0c5b30 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc15bc80 con 0x7f83fc14f160 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f8400f09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83fc0c6b20 con 0x7f83fc157e90 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f83f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83fc0c5b30 0x7f83fc0c7fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f83f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83fc0c5b30 0x7f83fc0c7fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45920/0 (socket says 172.21.15.32:45920) 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f83f3fff640 1 -- 172.21.15.32:0/2567735764 learned_addr learned my addr 172.21.15.32:0/2567735764 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.272+0000 7f83fad9d640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc157e90 0x7f83fc158310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83fa59c640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83fc14f160 0x7f83fc15c110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f3fff640 1 -- 172.21.15.32:0/2567735764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc157e90 msgr2=0x7f83fc158310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:11.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f3fff640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc157e90 0x7f83fc158310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:11.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f3fff640 1 -- 172.21.15.32:0/2567735764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83fc14f160 msgr2=0x7f83fc15c110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:11.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f3fff640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83fc14f160 0x7f83fc15c110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:11.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f3fff640 1 -- 172.21.15.32:0/2567735764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83fc0c6560 con 0x7f83fc0c5b30 2023-12-20T06:02:11.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83fa59c640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83fc14f160 0x7f83fc15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:11.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f3fff640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83fc0c5b30 0x7f83fc0c7fa0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f83e400f7a0 tx=0x7f83e400fc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:11.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f2ffd640 1 -- 172.21.15.32:0/2567735764 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400fe50 con 0x7f83fc0c5b30 2023-12-20T06:02:11.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f2ffd640 1 -- 172.21.15.32:0/2567735764 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e40052e0 con 0x7f83fc0c5b30 2023-12-20T06:02:11.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f2ffd640 1 -- 172.21.15.32:0/2567735764 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e400d970 con 0x7f83fc0c5b30 2023-12-20T06:02:11.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f8400f09640 1 -- 172.21.15.32:0/2567735764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83fc0c6840 con 0x7f83fc0c5b30 2023-12-20T06:02:11.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f8400f09640 1 -- 172.21.15.32:0/2567735764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83fc15c650 con 0x7f83fc0c5b30 2023-12-20T06:02:11.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f8400f09640 1 -- 172.21.15.32:0/2567735764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83fc15be10 con 0x7f83fc0c5b30 2023-12-20T06:02:11.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f2ffd640 1 -- 172.21.15.32:0/2567735764 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f83e400dad0 con 0x7f83fc0c5b30 2023-12-20T06:02:11.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f2ffd640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83dc065b10 0x7f83dc067fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:11.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83f2ffd640 1 -- 172.21.15.32:0/2567735764 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(199..199 src has 1..199) v4 ==== 6101+0+0 (secure 0 0 0) 0x7f83e4083500 con 0x7f83fc0c5b30 2023-12-20T06:02:11.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.276+0000 7f83fa59c640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83dc065b10 0x7f83dc067fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:11.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.280+0000 7f83fa59c640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83dc065b10 0x7f83dc067fd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f83fc156df0 tx=0x7f83ec008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:11.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.280+0000 7f83f2ffd640 1 -- 172.21.15.32:0/2567735764 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f83e4054610 con 0x7f83fc0c5b30 2023-12-20T06:02:11.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:11.440+0000 7f8400f09640 1 -- 172.21.15.32:0/2567735764 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f83fc04e000 con 0x7f83fc0c5b30 2023-12-20T06:02:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.080+0000 7f83f2ffd640 1 -- 172.21.15.32:0/2567735764 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v200) v1 ==== 169+0+0 (secure 0 0 0) 0x7f83e40541a0 con 0x7f83fc0c5b30 2023-12-20T06:02:12.084 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T06:02:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 -- 172.21.15.32:0/2567735764 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83dc065b10 msgr2=0x7f83dc067fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83dc065b10 0x7f83dc067fd0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f83fc156df0 tx=0x7f83ec008040 comp rx=0 tx=0).stop 2023-12-20T06:02:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 -- 172.21.15.32:0/2567735764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83fc0c5b30 msgr2=0x7f83fc0c7fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83fc0c5b30 0x7f83fc0c7fa0 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7f83e400f7a0 tx=0x7f83e400fc60 comp rx=0 tx=0).stop 2023-12-20T06:02:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 -- 172.21.15.32:0/2567735764 shutdown_connections 2023-12-20T06:02:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83dc065b10 0x7f83dc067fd0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83fc157e90 0x7f83fc158310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83fc0c5b30 0x7f83fc0c7fa0 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 --2- 172.21.15.32:0/2567735764 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83fc14f160 0x7f83fc15c110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 -- 172.21.15.32:0/2567735764 >> 172.21.15.32:0/2567735764 conn(0x7f83fc06a510 msgr2=0x7f83fc14fb30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 -- 172.21.15.32:0/2567735764 shutdown_connections 2023-12-20T06:02:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.084+0000 7f8400f09640 1 -- 172.21.15.32:0/2567735764 wait complete. 2023-12-20T06:02:12.113 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T06:02:12.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.264+0000 7f5091dce640 1 Processor -- start 2023-12-20T06:02:12.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.264+0000 7f5091dce640 1 -- start start 2023-12-20T06:02:12.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.268+0000 7f5091dce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f508c0b0bc0 0x7f508c0ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:12.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.268+0000 7f5091dce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c0cde20 con 0x7f508c157480 2023-12-20T06:02:12.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.268+0000 7f5091dce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f508c0c8e30 con 0x7f508c0b0760 2023-12-20T06:02:12.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.268+0000 7f5091dce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f508c0c9f50 con 0x7f508c0b0bc0 2023-12-20T06:02:12.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.268+0000 7f508bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f508c0b0bc0 0x7f508c0ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.268+0000 7f508bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f508c0b0bc0 0x7f508c0ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45922/0 (socket says 172.21.15.32:45922) 2023-12-20T06:02:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.268+0000 7f508bfff640 1 -- 172.21.15.32:0/3952542449 learned_addr learned my addr 172.21.15.32:0/3952542449 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.268+0000 7f508bfff640 1 -- 172.21.15.32:0/3952542449 >> v1:172.21.15.39:6789/0 conn(0x7f508c0b0760 legacy=0x7f508c0cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.268+0000 7f508bfff640 1 -- 172.21.15.32:0/3952542449 >> v1:172.21.15.32:6789/0 conn(0x7f508c157480 legacy=0x7f508c0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.268+0000 7f508bfff640 1 -- 172.21.15.32:0/3952542449 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508c0ce070 con 0x7f508c0b0bc0 2023-12-20T06:02:12.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f508bfff640 1 --2- 172.21.15.32:0/3952542449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f508c0b0bc0 0x7f508c0ccbc0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f508400d150 tx=0x7f508401eb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd1145177d5bce6b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:12.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f508a7fc640 1 -- 172.21.15.32:0/3952542449 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f508400b2d0 con 0x7f508c0b0bc0 2023-12-20T06:02:12.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f508a7fc640 1 -- 172.21.15.32:0/3952542449 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f508400b430 con 0x7f508c0b0bc0 2023-12-20T06:02:12.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f508a7fc640 1 -- 172.21.15.32:0/3952542449 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50840a7bc0 con 0x7f508c0b0bc0 2023-12-20T06:02:12.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f5091dce640 1 -- 172.21.15.32:0/3952542449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f508c0b0bc0 msgr2=0x7f508c0ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f5091dce640 1 --2- 172.21.15.32:0/3952542449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f508c0b0bc0 0x7f508c0ccbc0 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7f508400d150 tx=0x7f508401eb20 comp rx=0 tx=0).stop 2023-12-20T06:02:12.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f5091dce640 1 -- 172.21.15.32:0/3952542449 shutdown_connections 2023-12-20T06:02:12.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f5091dce640 1 --2- 172.21.15.32:0/3952542449 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f508c0b0bc0 0x7f508c0ccbc0 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f5091dce640 1 -- 172.21.15.32:0/3952542449 >> 172.21.15.32:0/3952542449 conn(0x7f508c06a510 msgr2=0x7f508c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:12.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f5091dce640 1 -- 172.21.15.32:0/3952542449 shutdown_connections 2023-12-20T06:02:12.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f5091dce640 1 -- 172.21.15.32:0/3952542449 wait complete. 2023-12-20T06:02:12.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f5091dce640 1 Processor -- start 2023-12-20T06:02:12.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.272+0000 7f5091dce640 1 -- start start 2023-12-20T06:02:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5091dce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f508c0b0760 0x7f508c0c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5091dce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f508c0b0bc0 0x7f508c14fa00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5091dce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508c157480 0x7f508c14ff40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5091dce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c0c7f60 con 0x7f508c0b0760 2023-12-20T06:02:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5091dce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f508c150b60 con 0x7f508c157480 2023-12-20T06:02:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5091dce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f508c1505a0 con 0x7f508c0b0bc0 2023-12-20T06:02:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f508b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f508c0b0760 0x7f508c0c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f508b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f508c0b0760 0x7f508c0c8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45938/0 (socket says 172.21.15.32:45938) 2023-12-20T06:02:12.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f508b7fe640 1 -- 172.21.15.32:0/713202643 learned_addr learned my addr 172.21.15.32:0/713202643 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f508bfff640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508c157480 0x7f508c14ff40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f508affd640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f508c0b0bc0 0x7f508c14fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f508b7fe640 1 -- 172.21.15.32:0/713202643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f508c0b0bc0 msgr2=0x7f508c14fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f508b7fe640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f508c0b0bc0 0x7f508c14fa00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f508b7fe640 1 -- 172.21.15.32:0/713202643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508c157480 msgr2=0x7f508c14ff40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f508b7fe640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508c157480 0x7f508c14ff40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f508b7fe640 1 -- 172.21.15.32:0/713202643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f508400bd40 con 0x7f508c0b0760 2023-12-20T06:02:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f508b7fe640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f508c0b0760 0x7f508c0c8610 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c500 tx=0x7f507c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5073fff640 1 -- 172.21.15.32:0/713202643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c005220 con 0x7f508c0b0760 2023-12-20T06:02:12.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5073fff640 1 -- 172.21.15.32:0/713202643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f507c005380 con 0x7f508c0b0760 2023-12-20T06:02:12.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5073fff640 1 -- 172.21.15.32:0/713202643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f507c007270 con 0x7f508c0b0760 2023-12-20T06:02:12.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5091dce640 1 -- 172.21.15.32:0/713202643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f508c150880 con 0x7f508c0b0760 2023-12-20T06:02:12.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5091dce640 1 -- 172.21.15.32:0/713202643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f508c0b1d00 con 0x7f508c0b0760 2023-12-20T06:02:12.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5091dce640 1 -- 172.21.15.32:0/713202643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f508c04e000 con 0x7f508c0b0760 2023-12-20T06:02:12.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.276+0000 7f5073fff640 1 -- 172.21.15.32:0/713202643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f507c0073d0 con 0x7f508c0b0760 2023-12-20T06:02:12.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.280+0000 7f5073fff640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5078065b80 0x7f5078068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:12.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.280+0000 7f5073fff640 1 -- 172.21.15.32:0/713202643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f507c082570 con 0x7f508c0b0760 2023-12-20T06:02:12.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.280+0000 7f508affd640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5078065b80 0x7f5078068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:12.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.280+0000 7f508affd640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5078065b80 0x7f5078068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f508c1f5120 tx=0x7f5080009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:12.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.284+0000 7f5073fff640 1 -- 172.21.15.32:0/713202643 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f507c04fef0 con 0x7f508c0b0760 2023-12-20T06:02:12.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.440+0000 7f5091dce640 1 -- 172.21.15.32:0/713202643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f508c070e70 con 0x7f508c0b0760 2023-12-20T06:02:12.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.444+0000 7f5073fff640 1 -- 172.21.15.32:0/713202643 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v67) v1 ==== 120+0+0 (secure 0 0 0) 0x7f507c053510 con 0x7f508c0b0760 2023-12-20T06:02:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 -- 172.21.15.32:0/713202643 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5078065b80 msgr2=0x7f5078068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5078065b80 0x7f5078068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f508c1f5120 tx=0x7f5080009040 comp rx=0 tx=0).stop 2023-12-20T06:02:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 -- 172.21.15.32:0/713202643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f508c0b0760 msgr2=0x7f508c0c8610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f508c0b0760 0x7f508c0c8610 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7f507c00c500 tx=0x7f507c00c9c0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 -- 172.21.15.32:0/713202643 shutdown_connections 2023-12-20T06:02:12.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5078065b80 0x7f5078068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f508c157480 0x7f508c14ff40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f508c0b0bc0 0x7f508c14fa00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 --2- 172.21.15.32:0/713202643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f508c0b0760 0x7f508c0c8610 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 -- 172.21.15.32:0/713202643 >> 172.21.15.32:0/713202643 conn(0x7f508c06a510 msgr2=0x7f508c0c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 -- 172.21.15.32:0/713202643 shutdown_connections 2023-12-20T06:02:12.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.448+0000 7f5091dce640 1 -- 172.21.15.32:0/713202643 wait complete. 2023-12-20T06:02:12.471 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T06:02:12.560 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:02:12.564 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:02:12.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24d28f9640 1 Processor -- start 2023-12-20T06:02:12.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24d28f9640 1 -- start start 2023-12-20T06:02:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24d28f9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f24cc157480 0x7f24cc0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24d28f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc0b0760 0x7f24cc0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24d28f9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24cc0b1100 con 0x7f24cc15d660 2023-12-20T06:02:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24d28f9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24cc0cde20 con 0x7f24cc157480 2023-12-20T06:02:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24d28f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc0c8e30 con 0x7f24cc0b0760 2023-12-20T06:02:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24cb7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc0b0760 0x7f24cc0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24cbfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f24cc157480 0x7f24cc0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24cbfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f24cc157480 0x7f24cc0c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:60506/0 (socket says 172.21.15.32:60506) 2023-12-20T06:02:12.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24cb7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc0b0760 0x7f24cc0b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49100/0 (socket says 172.21.15.32:49100) 2023-12-20T06:02:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24cbfff640 1 -- 172.21.15.32:0/1450343676 learned_addr learned my addr 172.21.15.32:0/1450343676 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24d0e96640 1 --1- 172.21.15.32:0/1450343676 >> v1:172.21.15.39:6789/0 conn(0x7f24cc15d660 0x7f24cc15daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:46836/0 (socket says 172.21.15.32:46836) 2023-12-20T06:02:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24cbfff640 1 -- 172.21.15.32:0/1450343676 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc0b0760 msgr2=0x7f24cc0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24cbfff640 1 --2- 172.21.15.32:0/1450343676 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc0b0760 0x7f24cc0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24cbfff640 1 -- 172.21.15.32:0/1450343676 >> v1:172.21.15.39:6789/0 conn(0x7f24cc15d660 legacy=0x7f24cc15daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24cbfff640 1 -- 172.21.15.32:0/1450343676 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24cc0ce070 con 0x7f24cc157480 2023-12-20T06:02:12.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.616+0000 7f24cb7fe640 1 --2- 172.21.15.32:0/1450343676 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc0b0760 0x7f24cc0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:12.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24cbfff640 1 --2- 172.21.15.32:0/1450343676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc157480 0x7f24cc0c9840 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009870 tx=0x7f24bc01c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6ca505db59b4bba3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:12.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24caffd640 1 -- 172.21.15.32:0/1450343676 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0ab680 con 0x7f24cc157480 2023-12-20T06:02:12.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24caffd640 1 -- 172.21.15.32:0/1450343676 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24bc0ab7e0 con 0x7f24cc157480 2023-12-20T06:02:12.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24caffd640 1 -- 172.21.15.32:0/1450343676 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0abb70 con 0x7f24cc157480 2023-12-20T06:02:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24d28f9640 1 -- 172.21.15.32:0/1450343676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc157480 msgr2=0x7f24cc0c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24d28f9640 1 --2- 172.21.15.32:0/1450343676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc157480 0x7f24cc0c9840 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f24bc009870 tx=0x7f24bc01c480 comp rx=0 tx=0).stop 2023-12-20T06:02:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24d28f9640 1 -- 172.21.15.32:0/1450343676 shutdown_connections 2023-12-20T06:02:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24d28f9640 1 --2- 172.21.15.32:0/1450343676 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc0b0760 0x7f24cc0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24d28f9640 1 --2- 172.21.15.32:0/1450343676 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc157480 0x7f24cc0c9840 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24d28f9640 1 -- 172.21.15.32:0/1450343676 >> 172.21.15.32:0/1450343676 conn(0x7f24cc06a510 msgr2=0x7f24cc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24d28f9640 1 -- 172.21.15.32:0/1450343676 shutdown_connections 2023-12-20T06:02:12.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.620+0000 7f24d28f9640 1 -- 172.21.15.32:0/1450343676 wait complete. 2023-12-20T06:02:12.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24d28f9640 1 Processor -- start 2023-12-20T06:02:12.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24d28f9640 1 -- start start 2023-12-20T06:02:12.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24d28f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24cc0b0760 0x7f24cc1453f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:12.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24d28f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc157480 0x7f24cc0ce770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:12.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24d28f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc15d660 0x7f24cc0cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:12.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24d28f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc152820 con 0x7f24cc0b0760 2023-12-20T06:02:12.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24d28f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc15d280 con 0x7f24cc157480 2023-12-20T06:02:12.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24d28f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24cc145930 con 0x7f24cc15d660 2023-12-20T06:02:12.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24cc0b0760 0x7f24cc1453f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:12.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24cbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24cc0b0760 0x7f24cc1453f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45946/0 (socket says 172.21.15.32:45946) 2023-12-20T06:02:12.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24cbfff640 1 -- 172.21.15.32:0/613858922 learned_addr learned my addr 172.21.15.32:0/613858922 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:12.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24cb7fe640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc157480 0x7f24cc0ce770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:12.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24d0e96640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc15d660 0x7f24cc0cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:12.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24cbfff640 1 -- 172.21.15.32:0/613858922 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc15d660 msgr2=0x7f24cc0cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24cbfff640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc15d660 0x7f24cc0cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24cbfff640 1 -- 172.21.15.32:0/613858922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc157480 msgr2=0x7f24cc0ce770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24cbfff640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc157480 0x7f24cc0ce770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24cbfff640 1 -- 172.21.15.32:0/613858922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24c0009420 con 0x7f24cc0b0760 2023-12-20T06:02:12.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24cb7fe640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc157480 0x7f24cc0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:12.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24cbfff640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24cc0b0760 0x7f24cc1453f0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0ad040 tx=0x7f24bc01acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:12.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24c8ff9640 1 -- 172.21.15.32:0/613858922 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0be5f0 con 0x7f24cc0b0760 2023-12-20T06:02:12.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24c8ff9640 1 -- 172.21.15.32:0/613858922 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24bc0be770 con 0x7f24cc0b0760 2023-12-20T06:02:12.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24d28f9640 1 -- 172.21.15.32:0/613858922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24bc0094d0 con 0x7f24cc0b0760 2023-12-20T06:02:12.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24c8ff9640 1 -- 172.21.15.32:0/613858922 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24bc0beb20 con 0x7f24cc0b0760 2023-12-20T06:02:12.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.624+0000 7f24d28f9640 1 -- 172.21.15.32:0/613858922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24cc15ccc0 con 0x7f24cc0b0760 2023-12-20T06:02:12.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.628+0000 7f24d28f9640 1 -- 172.21.15.32:0/613858922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24cc04e000 con 0x7f24cc0b0760 2023-12-20T06:02:12.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.628+0000 7f24c8ff9640 1 -- 172.21.15.32:0/613858922 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f24bc0b43f0 con 0x7f24cc0b0760 2023-12-20T06:02:12.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.628+0000 7f24c8ff9640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f24b8065b80 0x7f24b8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:12.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.628+0000 7f24c8ff9640 1 -- 172.21.15.32:0/613858922 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f24bc11e650 con 0x7f24cc0b0760 2023-12-20T06:02:12.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.628+0000 7f24cb7fe640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f24b8065b80 0x7f24b8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:12.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.628+0000 7f24cb7fe640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f24b8065b80 0x7f24b8068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0b1f20 tx=0x7f24c0008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:12.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.632+0000 7f24c8ff9640 1 -- 172.21.15.32:0/613858922 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f24bc0ebe90 con 0x7f24cc0b0760 2023-12-20T06:02:12.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.788+0000 7f24d28f9640 1 -- 172.21.15.32:0/613858922 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f24cc0ca0b0 con 0x7f24cc0b0760 2023-12-20T06:02:12.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24c8ff9640 1 -- 172.21.15.32:0/613858922 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v68) v1 ==== 115+0+0 (secure 0 0 0) 0x7f24bc0ef4b0 con 0x7f24cc0b0760 2023-12-20T06:02:12.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 -- 172.21.15.32:0/613858922 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f24b8065b80 msgr2=0x7f24b8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f24b8065b80 0x7f24b8068040 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f24cc0b1f20 tx=0x7f24c0008040 comp rx=0 tx=0).stop 2023-12-20T06:02:12.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 -- 172.21.15.32:0/613858922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24cc0b0760 msgr2=0x7f24cc1453f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:12.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24cc0b0760 0x7f24cc1453f0 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7f24bc0ad040 tx=0x7f24bc01acc0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 -- 172.21.15.32:0/613858922 shutdown_connections 2023-12-20T06:02:12.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f24b8065b80 0x7f24b8068040 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24cc15d660 0x7f24cc0cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24cc157480 0x7f24cc0ce770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 --2- 172.21.15.32:0/613858922 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24cc0b0760 0x7f24cc1453f0 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:12.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 -- 172.21.15.32:0/613858922 >> 172.21.15.32:0/613858922 conn(0x7f24cc06a510 msgr2=0x7f24cc163f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:12.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 -- 172.21.15.32:0/613858922 shutdown_connections 2023-12-20T06:02:12.801 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:12.796+0000 7f24d28f9640 1 -- 172.21.15.32:0/613858922 wait complete. 2023-12-20T06:02:12.819 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T06:02:13.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c7f97b640 1 Processor -- start 2023-12-20T06:02:13.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c7f97b640 1 -- start start 2023-12-20T06:02:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c7f97b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c78157480 0x7f1c780c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c7f97b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c780cde20 con 0x7f1c780cd4c0 2023-12-20T06:02:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c7f97b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1c780c8e30 con 0x7f1c780b0760 2023-12-20T06:02:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c7f97b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7815e480 con 0x7f1c78157480 2023-12-20T06:02:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c7df18640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1c780cd4c0 0x7f1c7815dd70 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41256/0 (socket says 172.21.15.32:41256) 2023-12-20T06:02:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c7df18640 1 -- 172.21.15.32:0/2112721565 learned_addr learned my addr 172.21.15.32:0/2112721565 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c7d717640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c78157480 0x7f1c780c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3894067871 0 0) 0x7f1c780cde20 con 0x7f1c780cd4c0 2023-12-20T06:02:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c64003530 con 0x7f1c780cd4c0 2023-12-20T06:02:13.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2591702358 0 0) 0x7f1c780c8e30 con 0x7f1c780b0760 2023-12-20T06:02:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1c780cde20 con 0x7f1c780b0760 2023-12-20T06:02:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 611420057 0 0) 0x7f1c64003530 con 0x7f1c780cd4c0 2023-12-20T06:02:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c78157480 msgr2=0x7f1c780c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c78157480 0x7f1c780c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 >> v1:172.21.15.39:6789/0 conn(0x7f1c780b0760 legacy=0x7f1c7815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c780ce070 con 0x7f1c780cd4c0 2023-12-20T06:02:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f1c70001a00 con 0x7f1c780cd4c0 2023-12-20T06:02:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 >> v1:172.21.15.32:6789/0 conn(0x7f1c780cd4c0 legacy=0x7f1c7815dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:13.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 0x7f1c64003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c78157480 0x7f1c640047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c64004d10 0x7f1c64005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7815e480 con 0x7f1c64003b40 2023-12-20T06:02:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6400ba90 con 0x7f1c64004d10 2023-12-20T06:02:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.004+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c6400bc50 con 0x7f1c78157480 2023-12-20T06:02:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c7d717640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 0x7f1c64003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c7cf16640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c78157480 0x7f1c640047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c7df18640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c64004d10 0x7f1c64005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c7d717640 1 -- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c78157480 msgr2=0x7f1c640047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c7d717640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c78157480 0x7f1c640047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c7d717640 1 -- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c64004d10 msgr2=0x7f1c64005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:13.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c7d717640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c64004d10 0x7f1c64005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c7d717640 1 -- 172.21.15.32:0/2112721565 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c64003530 con 0x7f1c64003b40 2023-12-20T06:02:13.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c7df18640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c64004d10 0x7f1c64005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:13.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c7cf16640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c78157480 0x7f1c640047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c7d717640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 0x7f1c64003f40 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009ae0 tx=0x7f1c7401ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f602e308974455 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740a5590 con 0x7f1c64003b40 2023-12-20T06:02:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c740a56f0 con 0x7f1c64003b40 2023-12-20T06:02:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.008+0000 7f1c6ffff640 1 -- 172.21.15.32:0/2112721565 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740a5aa0 con 0x7f1c64003b40 2023-12-20T06:02:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 -- 172.21.15.32:0/2112721565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 msgr2=0x7f1c64003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 --2- 172.21.15.32:0/2112721565 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 0x7f1c64003f40 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7f1c74009ae0 tx=0x7f1c7401ca20 comp rx=0 tx=0).stop 2023-12-20T06:02:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7df18640 1 -- 172.21.15.32:0/2112721565 reap_dead start 2023-12-20T06:02:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 -- 172.21.15.32:0/2112721565 shutdown_connections 2023-12-20T06:02:13.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 -- 172.21.15.32:0/2112721565 >> 172.21.15.32:0/2112721565 conn(0x7f1c7806a510 msgr2=0x7f1c780ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 -- 172.21.15.32:0/2112721565 shutdown_connections 2023-12-20T06:02:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 -- 172.21.15.32:0/2112721565 wait complete. 2023-12-20T06:02:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 Processor -- start 2023-12-20T06:02:13.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 -- start start 2023-12-20T06:02:13.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 0x7f1c781451d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7815dfc0 0x7f1c780c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c7815ac80 0x7f1c7815b100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c7815b640 con 0x7f1c64003b40 2023-12-20T06:02:13.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780c61b0 con 0x7f1c7815dfc0 2023-12-20T06:02:13.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7f97b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1c780c6490 con 0x7f1c7815ac80 2023-12-20T06:02:13.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7d717640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 0x7f1c781451d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7d717640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 0x7f1c781451d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45960/0 (socket says 172.21.15.32:45960) 2023-12-20T06:02:13.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7d717640 1 -- 172.21.15.32:0/3309234894 learned_addr learned my addr 172.21.15.32:0/3309234894 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:13.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7cf16640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7815dfc0 0x7f1c780c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7d717640 1 -- 172.21.15.32:0/3309234894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c7815ac80 msgr2=0x7f1c7815b100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:13.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7d717640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c7815ac80 0x7f1c7815b100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7d717640 1 -- 172.21.15.32:0/3309234894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7815dfc0 msgr2=0x7f1c780c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:13.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7d717640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7815dfc0 0x7f1c780c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.012+0000 7f1c7d717640 1 -- 172.21.15.32:0/3309234894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1c780c6770 con 0x7f1c64003b40 2023-12-20T06:02:13.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.016+0000 7f1c7d717640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 0x7f1c781451d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f1c7401c850 tx=0x7f1c64005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:13.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.016+0000 7f1c6dffb640 1 -- 172.21.15.32:0/3309234894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740a5590 con 0x7f1c64003b40 2023-12-20T06:02:13.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.016+0000 7f1c6dffb640 1 -- 172.21.15.32:0/3309234894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1c740b8680 con 0x7f1c64003b40 2023-12-20T06:02:13.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.016+0000 7f1c6dffb640 1 -- 172.21.15.32:0/3309234894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1c740a5ea0 con 0x7f1c64003b40 2023-12-20T06:02:13.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.016+0000 7f1c7f97b640 1 -- 172.21.15.32:0/3309234894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1c780c6a50 con 0x7f1c64003b40 2023-12-20T06:02:13.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.016+0000 7f1c7f97b640 1 -- 172.21.15.32:0/3309234894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1c78145710 con 0x7f1c64003b40 2023-12-20T06:02:13.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.016+0000 7f1c7f97b640 1 -- 172.21.15.32:0/3309234894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1c7804e000 con 0x7f1c64003b40 2023-12-20T06:02:13.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.016+0000 7f1c6dffb640 1 -- 172.21.15.32:0/3309234894 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1c740b4040 con 0x7f1c64003b40 2023-12-20T06:02:13.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.016+0000 7f1c6dffb640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1c5c065ba0 0x7f1c5c068060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.020+0000 7f1c6dffb640 1 -- 172.21.15.32:0/3309234894 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f1c7411cc80 con 0x7f1c64003b40 2023-12-20T06:02:13.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.020+0000 7f1c7cf16640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1c5c065ba0 0x7f1c5c068060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.020+0000 7f1c7cf16640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1c5c065ba0 0x7f1c5c068060 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1c781591b0 tx=0x7f1c68009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:13.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.020+0000 7f1c6dffb640 1 -- 172.21.15.32:0/3309234894 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1c740edf30 con 0x7f1c64003b40 2023-12-20T06:02:13.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.180+0000 7f1c7f97b640 1 -- 172.21.15.32:0/3309234894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f1c780c6f00 con 0x7f1c64003b40 2023-12-20T06:02:13.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.184+0000 7f1c6dffb640 1 -- 172.21.15.32:0/3309234894 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v69) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1c740edac0 con 0x7f1c64003b40 2023-12-20T06:02:13.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 -- 172.21.15.32:0/3309234894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1c5c065ba0 msgr2=0x7f1c5c068060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1c5c065ba0 0x7f1c5c068060 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f1c781591b0 tx=0x7f1c68009040 comp rx=0 tx=0).stop 2023-12-20T06:02:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 -- 172.21.15.32:0/3309234894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 msgr2=0x7f1c781451d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 0x7f1c781451d0 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7f1c7401c850 tx=0x7f1c64005800 comp rx=0 tx=0).stop 2023-12-20T06:02:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 -- 172.21.15.32:0/3309234894 shutdown_connections 2023-12-20T06:02:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1c5c065ba0 0x7f1c5c068060 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1c7815ac80 0x7f1c7815b100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1c7815dfc0 0x7f1c780c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 --2- 172.21.15.32:0/3309234894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1c64003b40 0x7f1c781451d0 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 -- 172.21.15.32:0/3309234894 >> 172.21.15.32:0/3309234894 conn(0x7f1c7806a510 msgr2=0x7f1c780c45f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:13.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 -- 172.21.15.32:0/3309234894 shutdown_connections 2023-12-20T06:02:13.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.188+0000 7f1c7f97b640 1 -- 172.21.15.32:0/3309234894 wait complete. 2023-12-20T06:02:13.218 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention' 2023-12-20T06:02:13.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66d1b6d640 1 Processor -- start 2023-12-20T06:02:13.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66d1b6d640 1 -- start start 2023-12-20T06:02:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66d1b6d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66cc15cdb0 0x7f66cc0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66d1b6d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f66cc0cfc80 0x7f66cc0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66d1b6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f66cc15d190 con 0x7f66cc0d2740 2023-12-20T06:02:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66d1b6d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66cc0d16a0 con 0x7f66cc15cdb0 2023-12-20T06:02:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66d1b6d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f66cc0d1980 con 0x7f66cc0cfc80 2023-12-20T06:02:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66cb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66cc15cdb0 0x7f66cc0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66cb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f66cc15cdb0 0x7f66cc0d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45974/0 (socket says 172.21.15.32:45974) 2023-12-20T06:02:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66cb7fe640 1 -- 172.21.15.32:0/1706990562 learned_addr learned my addr 172.21.15.32:0/1706990562 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66cbfff640 1 --2- 172.21.15.32:0/1706990562 >> v2:172.21.15.39:3300/0 conn(0x7f66cc0cfc80 0x7f66cc0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66cb7fe640 1 -- 172.21.15.32:0/1706990562 >> v2:172.21.15.39:3300/0 conn(0x7f66cc0cfc80 msgr2=0x7f66cc0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66cb7fe640 1 --2- 172.21.15.32:0/1706990562 >> v2:172.21.15.39:3300/0 conn(0x7f66cc0cfc80 0x7f66cc0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66cb7fe640 1 -- 172.21.15.32:0/1706990562 >> v1:172.21.15.39:6789/0 conn(0x7f66cc0d2740 legacy=0x7f66cc0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:13.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.360+0000 7f66cb7fe640 1 -- 172.21.15.32:0/1706990562 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66cc0d1c60 con 0x7f66cc15cdb0 2023-12-20T06:02:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66cb7fe640 1 --2- 172.21.15.32:0/1706990562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66cc15cdb0 0x7f66cc0d0be0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f66cc06c100 tx=0x7f66bc01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=178c36440e4cd576 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66caffd640 1 -- 172.21.15.32:0/1706990562 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0a89f0 con 0x7f66cc15cdb0 2023-12-20T06:02:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66caffd640 1 -- 172.21.15.32:0/1706990562 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc0a8b50 con 0x7f66cc15cdb0 2023-12-20T06:02:13.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66caffd640 1 -- 172.21.15.32:0/1706990562 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0b0440 con 0x7f66cc15cdb0 2023-12-20T06:02:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1706990562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66cc15cdb0 msgr2=0x7f66cc0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66d1b6d640 1 --2- 172.21.15.32:0/1706990562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66cc15cdb0 0x7f66cc0d0be0 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7f66cc06c100 tx=0x7f66bc01ecf0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1706990562 shutdown_connections 2023-12-20T06:02:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66d1b6d640 1 --2- 172.21.15.32:0/1706990562 >> v2:172.21.15.39:3300/0 conn(0x7f66cc0cfc80 0x7f66cc0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66d1b6d640 1 --2- 172.21.15.32:0/1706990562 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66cc15cdb0 0x7f66cc0d0be0 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1706990562 >> 172.21.15.32:0/1706990562 conn(0x7f66cc065ee0 msgr2=0x7f66cc06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1706990562 shutdown_connections 2023-12-20T06:02:13.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.364+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1706990562 wait complete. 2023-12-20T06:02:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66d1b6d640 1 Processor -- start 2023-12-20T06:02:13.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66d1b6d640 1 -- start start 2023-12-20T06:02:13.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66d1b6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66cc0cfc80 0x7f66cc148b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66d1b6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d2740 0x7f66cc15d910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66d1b6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66cc15cdb0 0x7f66cc15de50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66d1b6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc14d1c0 con 0x7f66cc0cfc80 2023-12-20T06:02:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66d1b6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc15e390 con 0x7f66cc0d2740 2023-12-20T06:02:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66d1b6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f66cc15e640 con 0x7f66cc15cdb0 2023-12-20T06:02:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66cc0cfc80 0x7f66cc148b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66cc0cfc80 0x7f66cc148b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45976/0 (socket says 172.21.15.32:45976) 2023-12-20T06:02:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66cb7fe640 1 -- 172.21.15.32:0/1778729615 learned_addr learned my addr 172.21.15.32:0/1778729615 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66c3fff640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d2740 0x7f66cc15d910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66cbfff640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66cc15cdb0 0x7f66cc15de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66cb7fe640 1 -- 172.21.15.32:0/1778729615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66cc15cdb0 msgr2=0x7f66cc15de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66cb7fe640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66cc15cdb0 0x7f66cc15de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66cb7fe640 1 -- 172.21.15.32:0/1778729615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d2740 msgr2=0x7f66cc15d910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66cb7fe640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d2740 0x7f66cc15d910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66cb7fe640 1 -- 172.21.15.32:0/1778729615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f66bc0094d0 con 0x7f66cc0cfc80 2023-12-20T06:02:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66cb7fe640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66cc0cfc80 0x7f66cc148b00 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f66bc01b900 tx=0x7f66bc01ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66c8ff9640 1 -- 172.21.15.32:0/1778729615 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0ba600 con 0x7f66cc0cfc80 2023-12-20T06:02:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66c8ff9640 1 -- 172.21.15.32:0/1778729615 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66bc0ba760 con 0x7f66cc0cfc80 2023-12-20T06:02:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66c8ff9640 1 -- 172.21.15.32:0/1778729615 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66bc0bab10 con 0x7f66cc0cfc80 2023-12-20T06:02:13.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1778729615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f66cc15e920 con 0x7f66cc0cfc80 2023-12-20T06:02:13.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.368+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1778729615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f66cc15eee0 con 0x7f66cc0cfc80 2023-12-20T06:02:13.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.372+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1778729615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f66cc14cf90 con 0x7f66cc0cfc80 2023-12-20T06:02:13.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.372+0000 7f66c8ff9640 1 -- 172.21.15.32:0/1778729615 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f66bc0bac70 con 0x7f66cc0cfc80 2023-12-20T06:02:13.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.372+0000 7f66c8ff9640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f66ac065b80 0x7f66ac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:13.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.372+0000 7f66c8ff9640 1 -- 172.21.15.32:0/1778729615 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f66bc1218e0 con 0x7f66cc0cfc80 2023-12-20T06:02:13.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.372+0000 7f66c3fff640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f66ac065b80 0x7f66ac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:13.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.372+0000 7f66c3fff640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f66ac065b80 0x7f66ac068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f66cc1f6250 tx=0x7f66b4009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:13.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.376+0000 7f66c8ff9640 1 -- 172.21.15.32:0/1778729615 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f66cc14cf90 con 0x7f66cc0cfc80 2023-12-20T06:02:13.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:13.588+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1778729615 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]} v 0) v1 -- 0x7f66cc14cf90 con 0x7f66cc0cfc80 2023-12-20T06:02:14.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66c8ff9640 1 -- 172.21.15.32:0/1778729615 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedules.test_snap_schedule_with_retention"]}]=0 v3025) v1 ==== 163+0+0 (secure 0 0 0) 0x7f66cc14cf90 con 0x7f66cc0cfc80 2023-12-20T06:02:14.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1778729615 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f66ac065b80 msgr2=0x7f66ac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:14.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f66ac065b80 0x7f66ac068040 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f66cc1f6250 tx=0x7f66b4009040 comp rx=0 tx=0).stop 2023-12-20T06:02:14.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1778729615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66cc0cfc80 msgr2=0x7f66cc148b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:14.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66cc0cfc80 0x7f66cc148b00 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7f66bc01b900 tx=0x7f66bc01ba90 comp rx=0 tx=0).stop 2023-12-20T06:02:14.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1778729615 shutdown_connections 2023-12-20T06:02:14.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f66ac065b80 0x7f66ac068040 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:14.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66cc15cdb0 0x7f66cc15de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:14.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66cc0d2740 0x7f66cc15d910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:14.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 --2- 172.21.15.32:0/1778729615 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66cc0cfc80 0x7f66cc148b00 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:14.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1778729615 >> 172.21.15.32:0/1778729615 conn(0x7f66cc065ee0 msgr2=0x7f66cc15b490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:14.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1778729615 shutdown_connections 2023-12-20T06:02:14.100 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.096+0000 7f66d1b6d640 1 -- 172.21.15.32:0/1778729615 wait complete. 2023-12-20T06:02:14.120 INFO:tasks.cephfs_test_runner:Test scheduled snapshots along with rentention policy ... ok 2023-12-20T06:02:14.121 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command succes with --fs argument for non-default fs 2023-12-20T06:02:14.121 INFO:tasks.cephfs_test_runner:test_snap_schedule_for_non_default_fs (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-12-20T06:02:14.121 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-12-20T06:02:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.264+0000 7f1d41255640 1 Processor -- start 2023-12-20T06:02:14.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.264+0000 7f1d41255640 1 -- start start 2023-12-20T06:02:14.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.264+0000 7f1d41255640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d3c0ae010 0x7f1d3c071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.264+0000 7f1d41255640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3c0ae3f0 con 0x7f1d3c06cf10 2023-12-20T06:02:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.264+0000 7f1d41255640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1d3c0ca4b0 con 0x7f1d3c164ef0 2023-12-20T06:02:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.264+0000 7f1d41255640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c161800 con 0x7f1d3c0ae010 2023-12-20T06:02:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.264+0000 7f1d33fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1d3c06cf10 0x7f1d3c0701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41262/0 (socket says 172.21.15.32:41262) 2023-12-20T06:02:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.264+0000 7f1d33fff640 1 -- 172.21.15.32:0/578544335 learned_addr learned my addr 172.21.15.32:0/578544335 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.264+0000 7f1d3ad9d640 1 --2- 172.21.15.32:0/578544335 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d3c0ae010 0x7f1d3c071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.268+0000 7f1d3a59c640 1 -- 172.21.15.32:0/578544335 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1485702156 0 0) 0x7f1d3c0ae3f0 con 0x7f1d3c06cf10 2023-12-20T06:02:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.268+0000 7f1d3a59c640 1 -- 172.21.15.32:0/578544335 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f1d28003530 con 0x7f1d3c06cf10 2023-12-20T06:02:14.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.268+0000 7f1d3ad9d640 1 -- 172.21.15.32:0/578544335 >> v1:172.21.15.39:6789/0 conn(0x7f1d3c164ef0 legacy=0x7f1d3c0708d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.268+0000 7f1d3ad9d640 1 -- 172.21.15.32:0/578544335 >> v1:172.21.15.32:6789/0 conn(0x7f1d3c06cf10 legacy=0x7f1d3c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.268+0000 7f1d3a59c640 1 -- 172.21.15.32:0/578544335 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3340095678 0 0) 0x7f1d28003530 con 0x7f1d3c06cf10 2023-12-20T06:02:14.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.268+0000 7f1d3ad9d640 1 -- 172.21.15.32:0/578544335 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d3c070fe0 con 0x7f1d3c0ae010 2023-12-20T06:02:14.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.268+0000 7f1d3ad9d640 1 --2- 172.21.15.32:0/578544335 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d3c0ae010 0x7f1d3c071470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0097d0 tx=0x7f1d2c01c300 comp rx=0 tx=0).ready entity=mon.2 client_cookie=55071c0db76aa3dd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:14.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.268+0000 7f1d3a59c640 1 -- 172.21.15.32:0/578544335 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d24001680 con 0x7f1d3c0ae010 2023-12-20T06:02:14.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.268+0000 7f1d3a59c640 1 -- 172.21.15.32:0/578544335 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c0a7050 con 0x7f1d3c0ae010 2023-12-20T06:02:14.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.268+0000 7f1d3a59c640 1 -- 172.21.15.32:0/578544335 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0ab620 con 0x7f1d3c0ae010 2023-12-20T06:02:14.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 -- 172.21.15.32:0/578544335 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d3c0ae010 msgr2=0x7f1d3c071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:14.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 --2- 172.21.15.32:0/578544335 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d3c0ae010 0x7f1d3c071470 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0097d0 tx=0x7f1d2c01c300 comp rx=0 tx=0).stop 2023-12-20T06:02:14.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 -- 172.21.15.32:0/578544335 shutdown_connections 2023-12-20T06:02:14.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 --2- 172.21.15.32:0/578544335 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d3c0ae010 0x7f1d3c071470 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:14.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 -- 172.21.15.32:0/578544335 >> 172.21.15.32:0/578544335 conn(0x7f1d3c06a320 msgr2=0x7f1d3c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:14.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 -- 172.21.15.32:0/578544335 shutdown_connections 2023-12-20T06:02:14.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 -- 172.21.15.32:0/578544335 wait complete. 2023-12-20T06:02:14.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 Processor -- start 2023-12-20T06:02:14.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 -- start start 2023-12-20T06:02:14.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0ae010 0x7f1d3c1645d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:14.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d3c1623c0 0x7f1d3c161940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c157fb0 0x7f1d3c158430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c1fe060 con 0x7f1d3c0ae010 2023-12-20T06:02:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c153720 con 0x7f1d3c157fb0 2023-12-20T06:02:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d41255640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1d3c158970 con 0x7f1d3c1623c0 2023-12-20T06:02:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d3ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0ae010 0x7f1d3c1645d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d3ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0ae010 0x7f1d3c1645d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45990/0 (socket says 172.21.15.32:45990) 2023-12-20T06:02:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d3ad9d640 1 -- 172.21.15.32:0/1924533878 learned_addr learned my addr 172.21.15.32:0/1924533878 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d3b59e640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c157fb0 0x7f1d3c158430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d33fff640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d3c1623c0 0x7f1d3c161940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:14.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d3ad9d640 1 -- 172.21.15.32:0/1924533878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d3c1623c0 msgr2=0x7f1d3c161940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d3ad9d640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d3c1623c0 0x7f1d3c161940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d3ad9d640 1 -- 172.21.15.32:0/1924533878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c157fb0 msgr2=0x7f1d3c158430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d3ad9d640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c157fb0 0x7f1d3c158430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.272+0000 7f1d3ad9d640 1 -- 172.21.15.32:0/1924533878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1d2c0093b0 con 0x7f1d3c0ae010 2023-12-20T06:02:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.276+0000 7f1d3ad9d640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0ae010 0x7f1d3c1645d0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0ae040 tx=0x7f1d2c01ab40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.276+0000 7f1d337fe640 1 -- 172.21.15.32:0/1924533878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c0059a0 con 0x7f1d3c0ae010 2023-12-20T06:02:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.276+0000 7f1d337fe640 1 -- 172.21.15.32:0/1924533878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1d2c005b00 con 0x7f1d3c0ae010 2023-12-20T06:02:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.276+0000 7f1d41255640 1 -- 172.21.15.32:0/1924533878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1d3c162840 con 0x7f1d3c0ae010 2023-12-20T06:02:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.276+0000 7f1d337fe640 1 -- 172.21.15.32:0/1924533878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1d2c006030 con 0x7f1d3c0ae010 2023-12-20T06:02:14.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.276+0000 7f1d41255640 1 -- 172.21.15.32:0/1924533878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1d3c161e80 con 0x7f1d3c0ae010 2023-12-20T06:02:14.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.276+0000 7f1d41255640 1 -- 172.21.15.32:0/1924533878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1d3c04e000 con 0x7f1d3c0ae010 2023-12-20T06:02:14.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.276+0000 7f1d337fe640 1 -- 172.21.15.32:0/1924533878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1d2c006190 con 0x7f1d3c0ae010 2023-12-20T06:02:14.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.276+0000 7f1d337fe640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d1c065b80 0x7f1d1c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:14.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.276+0000 7f1d337fe640 1 -- 172.21.15.32:0/1924533878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f1d2c11deb0 con 0x7f1d3c0ae010 2023-12-20T06:02:14.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.276+0000 7f1d33fff640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d1c065b80 0x7f1d1c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:14.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.280+0000 7f1d33fff640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d1c065b80 0x7f1d1c068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1d24000f70 tx=0x7f1d24009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:14.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.280+0000 7f1d337fe640 1 -- 172.21.15.32:0/1924533878 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1d2c0eb770 con 0x7f1d3c0ae010 2023-12-20T06:02:14.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:14.496+0000 7f1d41255640 1 -- 172.21.15.32:0/1924533878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7f1d3c0cf450 con 0x7f1d3c0ae010 2023-12-20T06:02:15.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.096+0000 7f1d337fe640 1 -- 172.21.15.32:0/1924533878 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3026) v1 ==== 189+0+0 (secure 0 0 0) 0x7f1d2c0eed90 con 0x7f1d3c0ae010 2023-12-20T06:02:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 -- 172.21.15.32:0/1924533878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d1c065b80 msgr2=0x7f1d1c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d1c065b80 0x7f1d1c068040 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f1d24000f70 tx=0x7f1d24009040 comp rx=0 tx=0).stop 2023-12-20T06:02:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 -- 172.21.15.32:0/1924533878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0ae010 msgr2=0x7f1d3c1645d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0ae010 0x7f1d3c1645d0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7f1d2c0ae040 tx=0x7f1d2c01ab40 comp rx=0 tx=0).stop 2023-12-20T06:02:15.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 -- 172.21.15.32:0/1924533878 shutdown_connections 2023-12-20T06:02:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1d1c065b80 0x7f1d1c068040 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1d3c157fb0 0x7f1d3c158430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1d3c1623c0 0x7f1d3c161940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 --2- 172.21.15.32:0/1924533878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1d3c0ae010 0x7f1d3c1645d0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 -- 172.21.15.32:0/1924533878 >> 172.21.15.32:0/1924533878 conn(0x7f1d3c06a320 msgr2=0x7f1d3c0af9a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 -- 172.21.15.32:0/1924533878 shutdown_connections 2023-12-20T06:02:15.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.100+0000 7f1d41255640 1 -- 172.21.15.32:0/1924533878 wait complete. 2023-12-20T06:02:15.124 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T06:02:15.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.268+0000 7f274750a640 1 Processor -- start 2023-12-20T06:02:15.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.268+0000 7f274750a640 1 -- start start 2023-12-20T06:02:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f274750a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2740070940 0x7f27400caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f274750a640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274015b6a0 con 0x7f27400b0cb0 2023-12-20T06:02:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f274750a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f274006d8c0 con 0x7f274015abb0 2023-12-20T06:02:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f274750a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f274006e160 con 0x7f2740070940 2023-12-20T06:02:15.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f2744aa5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f27400b0cb0 0x7f2740070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41270/0 (socket says 172.21.15.32:41270) 2023-12-20T06:02:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f2744aa5640 1 -- 172.21.15.32:0/2058349277 learned_addr learned my addr 172.21.15.32:0/2058349277 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f2745aa7640 1 --2- 172.21.15.32:0/2058349277 >> v2:172.21.15.39:3300/0 conn(0x7f2740070940 0x7f27400caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f2737fff640 1 -- 172.21.15.32:0/2058349277 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2487223937 0 0) 0x7f274015b6a0 con 0x7f27400b0cb0 2023-12-20T06:02:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f2737fff640 1 -- 172.21.15.32:0/2058349277 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f272c003530 con 0x7f27400b0cb0 2023-12-20T06:02:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f2745aa7640 1 -- 172.21.15.32:0/2058349277 >> v1:172.21.15.39:6789/0 conn(0x7f274015abb0 legacy=0x7f274015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f2745aa7640 1 -- 172.21.15.32:0/2058349277 >> v1:172.21.15.32:6789/0 conn(0x7f27400b0cb0 legacy=0x7f2740070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.272+0000 7f2745aa7640 1 -- 172.21.15.32:0/2058349277 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f274006dba0 con 0x7f2740070940 2023-12-20T06:02:15.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f2745aa7640 1 --2- 172.21.15.32:0/2058349277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2740070940 0x7f27400caaf0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2738009d90 tx=0x7f273801c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d1de9cff14b34679 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f2737fff640 1 -- 172.21.15.32:0/2058349277 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27380a5a10 con 0x7f2740070940 2023-12-20T06:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f2737fff640 1 -- 172.21.15.32:0/2058349277 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27380a5b70 con 0x7f2740070940 2023-12-20T06:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f2737fff640 1 -- 172.21.15.32:0/2058349277 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27380ad440 con 0x7f2740070940 2023-12-20T06:02:15.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f274750a640 1 -- 172.21.15.32:0/2058349277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2740070940 msgr2=0x7f27400caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f274750a640 1 --2- 172.21.15.32:0/2058349277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2740070940 0x7f27400caaf0 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f2738009d90 tx=0x7f273801c950 comp rx=0 tx=0).stop 2023-12-20T06:02:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f274750a640 1 -- 172.21.15.32:0/2058349277 shutdown_connections 2023-12-20T06:02:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f274750a640 1 --2- 172.21.15.32:0/2058349277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2740070940 0x7f27400caaf0 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f274750a640 1 -- 172.21.15.32:0/2058349277 >> 172.21.15.32:0/2058349277 conn(0x7f274006a510 msgr2=0x7f27400ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f274750a640 1 -- 172.21.15.32:0/2058349277 shutdown_connections 2023-12-20T06:02:15.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f274750a640 1 -- 172.21.15.32:0/2058349277 wait complete. 2023-12-20T06:02:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f274750a640 1 Processor -- start 2023-12-20T06:02:15.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.276+0000 7f274750a640 1 -- start start 2023-12-20T06:02:15.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f274750a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2740070940 0x7f27400c8590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:15.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f274750a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27400b0cb0 0x7f2740154480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:15.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f274750a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274015abb0 0x7f27400c9300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:15.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f274750a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27400c9840 con 0x7f274015abb0 2023-12-20T06:02:15.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f274750a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f27400c8c70 con 0x7f27400b0cb0 2023-12-20T06:02:15.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f274750a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2740157da0 con 0x7f2740070940 2023-12-20T06:02:15.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2745aa7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274015abb0 0x7f27400c9300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2745aa7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274015abb0 0x7f27400c9300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:45992/0 (socket says 172.21.15.32:45992) 2023-12-20T06:02:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2745aa7640 1 -- 172.21.15.32:0/271201532 learned_addr learned my addr 172.21.15.32:0/271201532 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2744aa5640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27400b0cb0 0x7f2740154480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f27452a6640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2740070940 0x7f27400c8590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2745aa7640 1 -- 172.21.15.32:0/271201532 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2740070940 msgr2=0x7f27400c8590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2745aa7640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2740070940 0x7f27400c8590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2745aa7640 1 -- 172.21.15.32:0/271201532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27400b0cb0 msgr2=0x7f2740154480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2745aa7640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27400b0cb0 0x7f2740154480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2745aa7640 1 -- 172.21.15.32:0/271201532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2738009a70 con 0x7f274015abb0 2023-12-20T06:02:15.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f27452a6640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2740070940 0x7f27400c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:15.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2744aa5640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27400b0cb0 0x7f2740154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:15.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2745aa7640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274015abb0 0x7f27400c9300 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f2738009d90 tx=0x7f27380196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:15.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2735ffb640 1 -- 172.21.15.32:0/271201532 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27380aece0 con 0x7f274015abb0 2023-12-20T06:02:15.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2735ffb640 1 -- 172.21.15.32:0/271201532 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f27380aee40 con 0x7f274015abb0 2023-12-20T06:02:15.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f274750a640 1 -- 172.21.15.32:0/271201532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2740158080 con 0x7f274015abb0 2023-12-20T06:02:15.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f2735ffb640 1 -- 172.21.15.32:0/271201532 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f27380b7650 con 0x7f274015abb0 2023-12-20T06:02:15.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f274750a640 1 -- 172.21.15.32:0/271201532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f27401579c0 con 0x7f274015abb0 2023-12-20T06:02:15.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.280+0000 7f274750a640 1 -- 172.21.15.32:0/271201532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f274006acf0 con 0x7f274015abb0 2023-12-20T06:02:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.284+0000 7f2735ffb640 1 -- 172.21.15.32:0/271201532 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f27380b4050 con 0x7f274015abb0 2023-12-20T06:02:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.284+0000 7f2735ffb640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2724065b80 0x7f2724068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.284+0000 7f2735ffb640 1 -- 172.21.15.32:0/271201532 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f273811e4d0 con 0x7f274015abb0 2023-12-20T06:02:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.284+0000 7f27452a6640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2724065b80 0x7f2724068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:15.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.284+0000 7f27452a6640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2724065b80 0x7f2724068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f273c001650 tx=0x7f273c009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:15.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.288+0000 7f2735ffb640 1 -- 172.21.15.32:0/271201532 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f27380ebd80 con 0x7f274015abb0 2023-12-20T06:02:15.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.444+0000 7f274750a640 1 -- 172.21.15.32:0/271201532 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7f274004e000 con 0x7f274015abb0 2023-12-20T06:02:15.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f2735ffb640 1 -- 172.21.15.32:0/271201532 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v70)=0 v70) v1 ==== 122+0+0 (secure 0 0 0) 0x7f27380ef3a0 con 0x7f274015abb0 2023-12-20T06:02:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 -- 172.21.15.32:0/271201532 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2724065b80 msgr2=0x7f2724068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2724065b80 0x7f2724068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f273c001650 tx=0x7f273c009040 comp rx=0 tx=0).stop 2023-12-20T06:02:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 -- 172.21.15.32:0/271201532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274015abb0 msgr2=0x7f27400c9300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274015abb0 0x7f27400c9300 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7f2738009d90 tx=0x7f27380196f0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 -- 172.21.15.32:0/271201532 shutdown_connections 2023-12-20T06:02:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2724065b80 0x7f2724068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f274015abb0 0x7f27400c9300 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f27400b0cb0 0x7f2740154480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 --2- 172.21.15.32:0/271201532 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2740070940 0x7f27400c8590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 -- 172.21.15.32:0/271201532 >> 172.21.15.32:0/271201532 conn(0x7f274006a510 msgr2=0x7f27400c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 -- 172.21.15.32:0/271201532 shutdown_connections 2023-12-20T06:02:15.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.452+0000 7f274750a640 1 -- 172.21.15.32:0/271201532 wait complete. 2023-12-20T06:02:15.475 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T06:02:15.475 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:02:15.540 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:02:15.585 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T06:02:15.585 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:02:15.585 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:15.595 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:02:15.595 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:02:15.595 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:15.635 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:02:15.635 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:02:15.635 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:15.648 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:02:15.648 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:02:15.649 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:15.687 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:02:15.687 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T06:02:15.687 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:02:15.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.832+0000 7f09fa3e9640 1 Processor -- start 2023-12-20T06:02:15.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.832+0000 7f09fa3e9640 1 -- start start 2023-12-20T06:02:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.832+0000 7f09fa3e9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f09f415f110 0x7f09f415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.832+0000 7f09fa3e9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f415b990 con 0x7f09f40af5e0 2023-12-20T06:02:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.832+0000 7f09fa3e9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f09f415bbd0 con 0x7f09f406ce60 2023-12-20T06:02:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.832+0000 7f09fa3e9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f09f415beb0 con 0x7f09f415f110 2023-12-20T06:02:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.832+0000 7f09f3fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f09f40af5e0 0x7f09f40af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41278/0 (socket says 172.21.15.32:41278) 2023-12-20T06:02:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.832+0000 7f09f3fff640 1 -- 172.21.15.32:0/2485806870 learned_addr learned my addr 172.21.15.32:0/2485806870 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:15.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.836+0000 7f09f8986640 1 --2- 172.21.15.32:0/2485806870 >> v2:172.21.15.39:3300/0 conn(0x7f09f415f110 0x7f09f415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.836+0000 7f09f2ffd640 1 -- 172.21.15.32:0/2485806870 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2965559156 0 0) 0x7f09e40012a0 con 0x7f09f40af5e0 2023-12-20T06:02:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.836+0000 7f09f2ffd640 1 -- 172.21.15.32:0/2485806870 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f09dc003530 con 0x7f09f40af5e0 2023-12-20T06:02:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.836+0000 7f09f2ffd640 1 -- 172.21.15.32:0/2485806870 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1729215212 0 0) 0x7f09dc003530 con 0x7f09f40af5e0 2023-12-20T06:02:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.836+0000 7f09f8986640 1 -- 172.21.15.32:0/2485806870 >> v1:172.21.15.39:6789/0 conn(0x7f09f406ce60 legacy=0x7f09f415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.836+0000 7f09f8986640 1 -- 172.21.15.32:0/2485806870 >> v1:172.21.15.32:6789/0 conn(0x7f09f40af5e0 legacy=0x7f09f40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.836+0000 7f09f8986640 1 -- 172.21.15.32:0/2485806870 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09f415e910 con 0x7f09f415f110 2023-12-20T06:02:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.836+0000 7f09f8986640 1 --2- 172.21.15.32:0/2485806870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09f415f110 0x7f09f415c440 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009d90 tx=0x7f09ec01c660 comp rx=0 tx=0).ready entity=mon.1 client_cookie=d5757807d06435c2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.836+0000 7f09f2ffd640 1 -- 172.21.15.32:0/2485806870 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4001990 con 0x7f09f415f110 2023-12-20T06:02:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.836+0000 7f09f2ffd640 1 -- 172.21.15.32:0/2485806870 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09ec0a6490 con 0x7f09f415f110 2023-12-20T06:02:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.836+0000 7f09f2ffd640 1 -- 172.21.15.32:0/2485806870 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09ec0a6800 con 0x7f09f415f110 2023-12-20T06:02:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 -- 172.21.15.32:0/2485806870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09f415f110 msgr2=0x7f09f415c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 --2- 172.21.15.32:0/2485806870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09f415f110 0x7f09f415c440 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x7f09ec009d90 tx=0x7f09ec01c660 comp rx=0 tx=0).stop 2023-12-20T06:02:15.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 -- 172.21.15.32:0/2485806870 shutdown_connections 2023-12-20T06:02:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 --2- 172.21.15.32:0/2485806870 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09f415f110 0x7f09f415c440 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 -- 172.21.15.32:0/2485806870 >> 172.21.15.32:0/2485806870 conn(0x7f09f406a320 msgr2=0x7f09f40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 -- 172.21.15.32:0/2485806870 shutdown_connections 2023-12-20T06:02:15.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 -- 172.21.15.32:0/2485806870 wait complete. 2023-12-20T06:02:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 Processor -- start 2023-12-20T06:02:15.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 -- start start 2023-12-20T06:02:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f406ce60 0x7f09f40b3cc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09f40af5e0 0x7f09f4157590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09f415f110 0x7f09f4148700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:15.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f40b33f0 con 0x7f09f406ce60 2023-12-20T06:02:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4157ad0 con 0x7f09f415f110 2023-12-20T06:02:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09fa3e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f09f4157d70 con 0x7f09f40af5e0 2023-12-20T06:02:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f406ce60 0x7f09f40b3cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f406ce60 0x7f09f40b3cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46008/0 (socket says 172.21.15.32:46008) 2023-12-20T06:02:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09f3fff640 1 -- 172.21.15.32:0/1260352247 learned_addr learned my addr 172.21.15.32:0/1260352247 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09f8986640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09f415f110 0x7f09f4148700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09f37fe640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09f40af5e0 0x7f09f4157590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.840+0000 7f09f3fff640 1 -- 172.21.15.32:0/1260352247 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09f40af5e0 msgr2=0x7f09f4157590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09f3fff640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09f40af5e0 0x7f09f4157590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09f3fff640 1 -- 172.21.15.32:0/1260352247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09f415f110 msgr2=0x7f09f4148700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09f3fff640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09f415f110 0x7f09f4148700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09f3fff640 1 -- 172.21.15.32:0/1260352247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f09ec009a70 con 0x7f09f406ce60 2023-12-20T06:02:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09f3fff640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f406ce60 0x7f09f40b3cc0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f09e400c6d0 tx=0x7f09e400cb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09f0ff9640 1 -- 172.21.15.32:0/1260352247 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e4007270 con 0x7f09f406ce60 2023-12-20T06:02:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09f0ff9640 1 -- 172.21.15.32:0/1260352247 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f09e40073d0 con 0x7f09f406ce60 2023-12-20T06:02:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09f0ff9640 1 -- 172.21.15.32:0/1260352247 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f09e40057b0 con 0x7f09f406ce60 2023-12-20T06:02:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09fa3e9640 1 -- 172.21.15.32:0/1260352247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f09f40b2b50 con 0x7f09f406ce60 2023-12-20T06:02:15.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09fa3e9640 1 -- 172.21.15.32:0/1260352247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f09f40b3110 con 0x7f09f406ce60 2023-12-20T06:02:15.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09fa3e9640 1 -- 172.21.15.32:0/1260352247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f09f40c8ab0 con 0x7f09f406ce60 2023-12-20T06:02:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09f0ff9640 1 -- 172.21.15.32:0/1260352247 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f09e4005910 con 0x7f09f406ce60 2023-12-20T06:02:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09f0ff9640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f09e0065b80 0x7f09e0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.844+0000 7f09f0ff9640 1 -- 172.21.15.32:0/1260352247 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(200..200 src has 1..200) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f09e4082870 con 0x7f09f406ce60 2023-12-20T06:02:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.848+0000 7f09f37fe640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f09e0065b80 0x7f09e0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:15.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.848+0000 7f09f37fe640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f09e0065b80 0x7f09e0068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09f4151f40 tx=0x7f09e8009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:15.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:15.848+0000 7f09f0ff9640 1 -- 172.21.15.32:0/1260352247 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f09e40501c0 con 0x7f09f406ce60 2023-12-20T06:02:16.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.044+0000 7f09fa3e9640 1 -- 172.21.15.32:0/1260352247 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f09f415b990 con 0x7f09f406ce60 2023-12-20T06:02:16.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.044+0000 7f09f0ff9640 1 -- 172.21.15.32:0/1260352247 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 89 v89) v1 ==== 94+0+3530 (secure 0 0 0) 0x7f09f415b990 con 0x7f09f406ce60 2023-12-20T06:02:16.047 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:16.048 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":89,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":8396,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/1626919510","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1626919510},{"type":"v1","addr":"172.21.15.39:6833","nonce":1626919510}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":8699,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1466578073","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1466578073},{"type":"v1","addr":"172.21.15.32:6837","nonce":1466578073}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":89},{"gid":9634,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3001408763","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":3001408763},{"type":"v1","addr":"172.21.15.39:6835","nonce":3001408763}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82},{"gid":10020,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/2908145202","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":2908145202},{"type":"v1","addr":"172.21.15.32:6835","nonce":2908145202}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":82}],"filesystems":[]} 2023-12-20T06:02:16.048 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 89 2023-12-20T06:02:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 -- 172.21.15.32:0/1260352247 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f09e0065b80 msgr2=0x7f09e0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f09e0065b80 0x7f09e0068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f09f4151f40 tx=0x7f09e8009040 comp rx=0 tx=0).stop 2023-12-20T06:02:16.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 -- 172.21.15.32:0/1260352247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f406ce60 msgr2=0x7f09f40b3cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f406ce60 0x7f09f40b3cc0 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7f09e400c6d0 tx=0x7f09e400cb90 comp rx=0 tx=0).stop 2023-12-20T06:02:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 -- 172.21.15.32:0/1260352247 shutdown_connections 2023-12-20T06:02:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f09e0065b80 0x7f09e0068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f09f415f110 0x7f09f4148700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f09f40af5e0 0x7f09f4157590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 --2- 172.21.15.32:0/1260352247 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f09f406ce60 0x7f09f40b3cc0 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 -- 172.21.15.32:0/1260352247 >> 172.21.15.32:0/1260352247 conn(0x7f09f406a320 msgr2=0x7f09f40cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 -- 172.21.15.32:0/1260352247 shutdown_connections 2023-12-20T06:02:16.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:16.048+0000 7f09fa3e9640 1 -- 172.21.15.32:0/1260352247 wait complete. 2023-12-20T06:02:16.069 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T06:02:16.069 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T06:02:16.070 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T06:02:16.070 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:02:16.070 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T06:02:16.070 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T06:02:16.071 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T06:02:16.071 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:02:16.071 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T06:02:16.071 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T06:02:16.071 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T06:02:16.072 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:02:16.072 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T06:02:16.072 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T06:02:16.072 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T06:02:16.072 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:02:16.084 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T06:02:16.084 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T06:02:16.087 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T06:02:16.087 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T06:02:16.088 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T06:02:16.088 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T06:02:16.131 INFO:tasks.ceph.mds.a:Started 2023-12-20T06:02:16.132 DEBUG:teuthology.parallel:result is None 2023-12-20T06:02:16.137 INFO:tasks.ceph.mds.d:Started 2023-12-20T06:02:16.137 DEBUG:teuthology.parallel:result is None 2023-12-20T06:02:16.139 INFO:tasks.ceph.mds.b:Started 2023-12-20T06:02:16.140 DEBUG:teuthology.parallel:result is None 2023-12-20T06:02:16.218 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T06:02:16.235 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T06:02:16.238 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T06:02:16.288 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d is failed for ~0s 2023-12-20T06:02:22.084 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T06:02:22.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T06:02:22.090 INFO:tasks.ceph.mds.c:Started 2023-12-20T06:02:22.090 DEBUG:teuthology.parallel:result is None 2023-12-20T06:02:22.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T06:02:22.188 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T06:02:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.244+0000 7f560741f640 1 Processor -- start 2023-12-20T06:02:22.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.244+0000 7f560741f640 1 -- start start 2023-12-20T06:02:22.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f560741f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f560006ce60 0x7f560015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f560741f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560015bbd0 con 0x7f56000af5e0 2023-12-20T06:02:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f560741f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f560015beb0 con 0x7f560015f110 2023-12-20T06:02:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f560741f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f560015e910 con 0x7f560006ce60 2023-12-20T06:02:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f56049ba640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f560006ce60 0x7f560015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f56051bb640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f56000af5e0 0x7f56000af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51102/0 (socket says 172.21.15.32:51102) 2023-12-20T06:02:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f56051bb640 1 -- 172.21.15.32:0/1510162170 learned_addr learned my addr 172.21.15.32:0/1510162170 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f55f7fff640 1 -- 172.21.15.32:0/1510162170 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 742371225 0 0) 0x7f560015bbd0 con 0x7f56000af5e0 2023-12-20T06:02:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f55f7fff640 1 -- 172.21.15.32:0/1510162170 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55ec003530 con 0x7f56000af5e0 2023-12-20T06:02:22.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f56049ba640 1 -- 172.21.15.32:0/1510162170 >> v1:172.21.15.39:6789/0 conn(0x7f560015f110 legacy=0x7f560015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f56049ba640 1 -- 172.21.15.32:0/1510162170 >> v1:172.21.15.32:6789/0 conn(0x7f56000af5e0 legacy=0x7f56000af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:22.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f56049ba640 1 -- 172.21.15.32:0/1510162170 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f560015ebf0 con 0x7f560006ce60 2023-12-20T06:02:22.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.248+0000 7f56049ba640 1 --2- 172.21.15.32:0/1510162170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560006ce60 0x7f560015b280 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f55f00097c0 tx=0x7f55f001c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12deeab786c2721 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:22.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f55f7fff640 1 -- 172.21.15.32:0/1510162170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55f00a8050 con 0x7f560006ce60 2023-12-20T06:02:22.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f55f7fff640 1 -- 172.21.15.32:0/1510162170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55f00ac5f0 con 0x7f560006ce60 2023-12-20T06:02:22.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f55f7fff640 1 -- 172.21.15.32:0/1510162170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f560015b9d0 con 0x7f560006ce60 2023-12-20T06:02:22.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 -- 172.21.15.32:0/1510162170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560006ce60 msgr2=0x7f560015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:22.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 --2- 172.21.15.32:0/1510162170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560006ce60 0x7f560015b280 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7f55f00097c0 tx=0x7f55f001c3f0 comp rx=0 tx=0).stop 2023-12-20T06:02:22.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 -- 172.21.15.32:0/1510162170 shutdown_connections 2023-12-20T06:02:22.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 --2- 172.21.15.32:0/1510162170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560006ce60 0x7f560015b280 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:22.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 -- 172.21.15.32:0/1510162170 >> 172.21.15.32:0/1510162170 conn(0x7f560006a320 msgr2=0x7f56000ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:22.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 -- 172.21.15.32:0/1510162170 shutdown_connections 2023-12-20T06:02:22.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 -- 172.21.15.32:0/1510162170 wait complete. 2023-12-20T06:02:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 Processor -- start 2023-12-20T06:02:22.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 -- start start 2023-12-20T06:02:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560006ce60 0x7f5600150e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56000af5e0 0x7f560015a900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f560015f110 0x7f56000cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f560015ae40 con 0x7f560006ce60 2023-12-20T06:02:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5600151da0 con 0x7f56000af5e0 2023-12-20T06:02:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f560741f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5600151ff0 con 0x7f560015f110 2023-12-20T06:02:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.252+0000 7f56051bb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560006ce60 0x7f5600150e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:22.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f56051bb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560006ce60 0x7f5600150e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46848/0 (socket says 172.21.15.32:46848) 2023-12-20T06:02:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f56051bb640 1 -- 172.21.15.32:0/3064497521 learned_addr learned my addr 172.21.15.32:0/3064497521 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f56049ba640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56000af5e0 0x7f560015a900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f56059bc640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f560015f110 0x7f56000cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f56051bb640 1 -- 172.21.15.32:0/3064497521 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f560015f110 msgr2=0x7f56000cd6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f56051bb640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f560015f110 0x7f56000cd6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f56051bb640 1 -- 172.21.15.32:0/3064497521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56000af5e0 msgr2=0x7f560015a900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f56051bb640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56000af5e0 0x7f560015a900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f56051bb640 1 -- 172.21.15.32:0/3064497521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55f0009420 con 0x7f560006ce60 2023-12-20T06:02:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f56051bb640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560006ce60 0x7f5600150e30 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f55fc001270 tx=0x7f55fc00cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:22.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f55f5ffb640 1 -- 172.21.15.32:0/3064497521 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc007270 con 0x7f560006ce60 2023-12-20T06:02:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f55f5ffb640 1 -- 172.21.15.32:0/3064497521 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55fc0073d0 con 0x7f560006ce60 2023-12-20T06:02:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f55f5ffb640 1 -- 172.21.15.32:0/3064497521 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55fc00ce50 con 0x7f560006ce60 2023-12-20T06:02:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f560741f640 1 -- 172.21.15.32:0/3064497521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56001522d0 con 0x7f560006ce60 2023-12-20T06:02:22.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f560741f640 1 -- 172.21.15.32:0/3064497521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5600070eb0 con 0x7f560006ce60 2023-12-20T06:02:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f560741f640 1 -- 172.21.15.32:0/3064497521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f560004e000 con 0x7f560006ce60 2023-12-20T06:02:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.256+0000 7f55f5ffb640 1 -- 172.21.15.32:0/3064497521 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f55fc0057b0 con 0x7f560006ce60 2023-12-20T06:02:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.260+0000 7f55f5ffb640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55e4065b80 0x7f55e4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:22.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.260+0000 7f55f5ffb640 1 -- 172.21.15.32:0/3064497521 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(204..204 src has 1..204) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f55fc0829c0 con 0x7f560006ce60 2023-12-20T06:02:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.260+0000 7f56049ba640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55e4065b80 0x7f55e4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:22.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.260+0000 7f56049ba640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55e4065b80 0x7f55e4068040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55f00ae000 tx=0x7f55f00a7000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:22.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.264+0000 7f55f5ffb640 1 -- 172.21.15.32:0/3064497521 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f55fc053c70 con 0x7f560006ce60 2023-12-20T06:02:22.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:22.432+0000 7f560741f640 1 -- 172.21.15.32:0/3064497521 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f56000cf450 con 0x7f560006ce60 2023-12-20T06:02:23.001 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mds.d has been restored 2023-12-20T06:02:23.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.208+0000 7f55f5ffb640 1 -- 172.21.15.32:0/3064497521 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v205) v1 ==== 97+0+0 (secure 0 0 0) 0x7f55fc017070 con 0x7f560006ce60 2023-12-20T06:02:23.210 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T06:02:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 -- 172.21.15.32:0/3064497521 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55e4065b80 msgr2=0x7f55e4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55e4065b80 0x7f55e4068040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f55f00ae000 tx=0x7f55f00a7000 comp rx=0 tx=0).stop 2023-12-20T06:02:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 -- 172.21.15.32:0/3064497521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560006ce60 msgr2=0x7f5600150e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560006ce60 0x7f5600150e30 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7f55fc001270 tx=0x7f55fc00cb10 comp rx=0 tx=0).stop 2023-12-20T06:02:23.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 -- 172.21.15.32:0/3064497521 shutdown_connections 2023-12-20T06:02:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55e4065b80 0x7f55e4068040 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f560015f110 0x7f56000cd6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56000af5e0 0x7f560015a900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 --2- 172.21.15.32:0/3064497521 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f560006ce60 0x7f5600150e30 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 -- 172.21.15.32:0/3064497521 >> 172.21.15.32:0/3064497521 conn(0x7f560006a320 msgr2=0x7f56000cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 -- 172.21.15.32:0/3064497521 shutdown_connections 2023-12-20T06:02:23.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.212+0000 7f560741f640 1 -- 172.21.15.32:0/3064497521 wait complete. 2023-12-20T06:02:23.236 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T06:02:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.380+0000 7f3ce3255640 1 Processor -- start 2023-12-20T06:02:23.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.380+0000 7f3ce3255640 1 -- start start 2023-12-20T06:02:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3ce3255640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3cdc1599e0 0x7f3cdc0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3ce3255640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cdc0c90e0 0x7f3cdc0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3ce3255640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3cdc0711d0 con 0x7f3cdc0b0b30 2023-12-20T06:02:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3ce3255640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cdc0c98c0 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3ce3255640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3cdc0c9ba0 con 0x7f3cdc1599e0 2023-12-20T06:02:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3cdbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cdc0c90e0 0x7f3cdc0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3cdbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3cdc0c90e0 0x7f3cdc0b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46860/0 (socket says 172.21.15.32:46860) 2023-12-20T06:02:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3cdbfff640 1 -- 172.21.15.32:0/146115127 learned_addr learned my addr 172.21.15.32:0/146115127 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3ce0ff1640 1 --2- 172.21.15.32:0/146115127 >> v2:172.21.15.39:3300/0 conn(0x7f3cdc1599e0 0x7f3cdc0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3cdbfff640 1 -- 172.21.15.32:0/146115127 >> v2:172.21.15.39:3300/0 conn(0x7f3cdc1599e0 msgr2=0x7f3cdc0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:23.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3cdbfff640 1 --2- 172.21.15.32:0/146115127 >> v2:172.21.15.39:3300/0 conn(0x7f3cdc1599e0 0x7f3cdc0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3cdbfff640 1 -- 172.21.15.32:0/146115127 >> v1:172.21.15.39:6789/0 conn(0x7f3cdc0b0b30 legacy=0x7f3cdc15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3cdbfff640 1 -- 172.21.15.32:0/146115127 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cdc0c9e80 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.384+0000 7f3cdbfff640 1 --2- 172.21.15.32:0/146115127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc0c90e0 0x7f3cdc0b05f0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc06e3c0 tx=0x7f3cc801eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d45e2ad19324ff62 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:23.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3cdb7fe640 1 -- 172.21.15.32:0/146115127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80a7b90 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3cdb7fe640 1 -- 172.21.15.32:0/146115127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc80a7cf0 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3cdb7fe640 1 -- 172.21.15.32:0/146115127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80af660 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 -- 172.21.15.32:0/146115127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc0c90e0 msgr2=0x7f3cdc0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 --2- 172.21.15.32:0/146115127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc0c90e0 0x7f3cdc0b05f0 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc06e3c0 tx=0x7f3cc801eba0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 -- 172.21.15.32:0/146115127 shutdown_connections 2023-12-20T06:02:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 --2- 172.21.15.32:0/146115127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc0c90e0 0x7f3cdc0b05f0 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 --2- 172.21.15.32:0/146115127 >> v2:172.21.15.39:3300/0 conn(0x7f3cdc1599e0 0x7f3cdc0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 -- 172.21.15.32:0/146115127 >> 172.21.15.32:0/146115127 conn(0x7f3cdc06a320 msgr2=0x7f3cdc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 -- 172.21.15.32:0/146115127 shutdown_connections 2023-12-20T06:02:23.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 -- 172.21.15.32:0/146115127 wait complete. 2023-12-20T06:02:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 Processor -- start 2023-12-20T06:02:23.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 -- start start 2023-12-20T06:02:23.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cdc0b0b30 0x7f3cdc0b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc0c90e0 0x7f3cdc1f9160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cdc1599e0 0x7f3cdc1f96a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc1f6170 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc1f5550 con 0x7f3cdc1599e0 2023-12-20T06:02:23.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3ce3255640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3cdc1f57d0 con 0x7f3cdc0b0b30 2023-12-20T06:02:23.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.388+0000 7f3cdbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc0c90e0 0x7f3cdc1f9160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cdbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc0c90e0 0x7f3cdc1f9160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46868/0 (socket says 172.21.15.32:46868) 2023-12-20T06:02:23.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cdbfff640 1 -- 172.21.15.32:0/771622737 learned_addr learned my addr 172.21.15.32:0/771622737 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:23.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3ce0ff1640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cdc0b0b30 0x7f3cdc0b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3ce17f2640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cdc1599e0 0x7f3cdc1f96a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cdbfff640 1 -- 172.21.15.32:0/771622737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cdc0b0b30 msgr2=0x7f3cdc0b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cdbfff640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cdc0b0b30 0x7f3cdc0b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cdbfff640 1 -- 172.21.15.32:0/771622737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cdc1599e0 msgr2=0x7f3cdc1f96a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cdbfff640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cdc1599e0 0x7f3cdc1f96a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cdbfff640 1 -- 172.21.15.32:0/771622737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3cc800b770 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3ce17f2640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cdc1599e0 0x7f3cdc1f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cdbfff640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc0c90e0 0x7f3cdc1f9160 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800b2c0 tx=0x7f3cc801b940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cd97fa640 1 -- 172.21.15.32:0/771622737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80afdf0 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cd97fa640 1 -- 172.21.15.32:0/771622737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3cc80ba440 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cd97fa640 1 -- 172.21.15.32:0/771622737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3cc80ba7d0 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3ce3255640 1 -- 172.21.15.32:0/771622737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3cdc1f5ab0 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3ce3255640 1 -- 172.21.15.32:0/771622737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3cdc0b3c10 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3ce3255640 1 -- 172.21.15.32:0/771622737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3cdc1f62b0 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cd97fa640 1 -- 172.21.15.32:0/771622737 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3cc80b93e0 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cd97fa640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3cc0065bb0 0x7f3cc0068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3cd97fa640 1 -- 172.21.15.32:0/771622737 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f3cc811d550 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.392+0000 7f3ce0ff1640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3cc0065bb0 0x7f3cc0068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.396+0000 7f3ce0ff1640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3cc0065bb0 0x7f3cc0068070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0afe50 tx=0x7f3cd0006f90 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:23.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.396+0000 7f3cd97fa640 1 -- 172.21.15.32:0/771622737 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3cdc1f62b0 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.608+0000 7f3ce3255640 1 -- 172.21.15.32:0/771622737 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f3cdc1f62b0 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.608+0000 7f3cd97fa640 1 -- 172.21.15.32:0/771622737 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v51) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f3cdc1f62b0 con 0x7f3cdc0c90e0 2023-12-20T06:02:23.612 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:23.612 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:02:23.612 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T06:02:23.612 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.612 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T06:02:23.612 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T06:02:23.612 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.612 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:02:23.612 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:02:23.613 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:02:23.613 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.613 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.613 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.613 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.613 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T06:02:23.613 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T06:02:23.613 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.613 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:02:23.613 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:02:23.614 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:02:23.614 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.614 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.614 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.614 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.614 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T06:02:23.614 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T06:02:23.614 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.614 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:02:23.614 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:02:23.615 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:02:23.615 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.615 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.615 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.615 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.615 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T06:02:23.615 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T06:02:23.615 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.615 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:02:23.615 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:02:23.615 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:02:23.616 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.616 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.616 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.616 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T06:02:23.616 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T06:02:23.616 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.616 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:02:23.616 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:02:23.616 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.616 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.617 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.617 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T06:02:23.617 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T06:02:23.617 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.617 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:02:23.617 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:02:23.617 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.617 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.617 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.618 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T06:02:23.618 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T06:02:23.618 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.618 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:02:23.618 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:02:23.618 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.618 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.618 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.618 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.619 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T06:02:23.619 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T06:02:23.619 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.619 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:02:23.619 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:02:23.619 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.619 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.619 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.619 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.619 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T06:02:23.619 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T06:02:23.620 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.620 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:02:23.620 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:02:23.620 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.620 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.620 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.620 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.620 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T06:02:23.620 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T06:02:23.620 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.620 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:02:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:02:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T06:02:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T06:02:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:02:23.621 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:02:23.622 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.622 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.622 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.622 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.622 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T06:02:23.622 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T06:02:23.622 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.622 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:02:23.622 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:02:23.622 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.623 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.623 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T06:02:23.623 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T06:02:23.623 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.623 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:02:23.623 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T06:02:23.623 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T06:02:23.623 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.623 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.624 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T06:02:23.624 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T06:02:23.624 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.624 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:02:23.624 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:02:23.624 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:02:23.624 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:02:23.624 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.624 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.624 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.625 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T06:02:23.625 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T06:02:23.625 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.625 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:02:23.625 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:02:23.625 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:02:23.625 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:02:23.625 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.625 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.625 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.625 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T06:02:23.626 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T06:02:23.626 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.626 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:02:23.626 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:02:23.626 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:02:23.626 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:02:23.626 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.626 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.626 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.626 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T06:02:23.626 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.627 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:02:23.628 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T06:02:23.628 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.628 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.628 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:23.628 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:23.628 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T06:02:23.628 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T06:02:23.628 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:02:23.628 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:02:23.628 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T06:02:23.628 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:02:23.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.629 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:23.629 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T06:02:23.629 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:02:23.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 -- 172.21.15.32:0/771622737 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3cc0065bb0 msgr2=0x7f3cc0068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3cc0065bb0 0x7f3cc0068070 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc0afe50 tx=0x7f3cd0006f90 comp rx=0 tx=0).stop 2023-12-20T06:02:23.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 -- 172.21.15.32:0/771622737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc0c90e0 msgr2=0x7f3cdc1f9160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc0c90e0 0x7f3cdc1f9160 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7f3cc800b2c0 tx=0x7f3cc801b940 comp rx=0 tx=0).stop 2023-12-20T06:02:23.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 -- 172.21.15.32:0/771622737 shutdown_connections 2023-12-20T06:02:23.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3cc0065bb0 0x7f3cc0068070 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3cdc1599e0 0x7f3cdc1f96a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3cdc0c90e0 0x7f3cdc1f9160 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 --2- 172.21.15.32:0/771622737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3cdc0b0b30 0x7f3cdc0b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 -- 172.21.15.32:0/771622737 >> 172.21.15.32:0/771622737 conn(0x7f3cdc06a320 msgr2=0x7f3cdc0caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:23.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 -- 172.21.15.32:0/771622737 shutdown_connections 2023-12-20T06:02:23.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.612+0000 7f3ce3255640 1 -- 172.21.15.32:0/771622737 wait complete. 2023-12-20T06:02:23.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc256fbc640 1 Processor -- start 2023-12-20T06:02:23.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc256fbc640 1 -- start start 2023-12-20T06:02:23.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc256fbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2500aece0 0x7fc2500b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc256fbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2500b0b30 0x7fc2500701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc256fbc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc2500c98c0 con 0x7fc2501599e0 2023-12-20T06:02:23.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc256fbc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2500c9ba0 con 0x7fc2500aece0 2023-12-20T06:02:23.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc256fbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500c9e80 con 0x7fc2500b0b30 2023-12-20T06:02:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc24ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2500aece0 0x7fc2500b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc24ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc2500aece0 0x7fc2500b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46880/0 (socket says 172.21.15.32:46880) 2023-12-20T06:02:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc24ffff640 1 -- 172.21.15.32:0/1434389257 learned_addr learned my addr 172.21.15.32:0/1434389257 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc255559640 1 --2- 172.21.15.32:0/1434389257 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2500b0b30 0x7fc2500701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc24ffff640 1 -- 172.21.15.32:0/1434389257 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2500b0b30 msgr2=0x7fc2500701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc24ffff640 1 --2- 172.21.15.32:0/1434389257 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2500b0b30 0x7fc2500701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc24ffff640 1 -- 172.21.15.32:0/1434389257 >> v1:172.21.15.39:6789/0 conn(0x7fc2501599e0 legacy=0x7fc2500c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc24f7fe640 1 -- 172.21.15.32:0/1434389257 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1482825054 0 0) 0x7fc2500c98c0 con 0x7fc2501599e0 2023-12-20T06:02:23.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.776+0000 7fc24ffff640 1 -- 172.21.15.32:0/1434389257 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2500ca160 con 0x7fc2500aece0 2023-12-20T06:02:23.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc24ffff640 1 --2- 172.21.15.32:0/1434389257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2500aece0 0x7fc2500b05f0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009870 tx=0x7fc23c01c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5efb51c98f816a6b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:23.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc24f7fe640 1 -- 172.21.15.32:0/1434389257 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0ab650 con 0x7fc2500aece0 2023-12-20T06:02:23.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc24f7fe640 1 -- 172.21.15.32:0/1434389257 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c0ab7b0 con 0x7fc2500aece0 2023-12-20T06:02:23.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc24f7fe640 1 -- 172.21.15.32:0/1434389257 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0abb40 con 0x7fc2500aece0 2023-12-20T06:02:23.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc256fbc640 1 -- 172.21.15.32:0/1434389257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2500aece0 msgr2=0x7fc2500b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc256fbc640 1 --2- 172.21.15.32:0/1434389257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2500aece0 0x7fc2500b05f0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7fc23c009870 tx=0x7fc23c01c540 comp rx=0 tx=0).stop 2023-12-20T06:02:23.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc256fbc640 1 -- 172.21.15.32:0/1434389257 shutdown_connections 2023-12-20T06:02:23.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc256fbc640 1 --2- 172.21.15.32:0/1434389257 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2500b0b30 0x7fc2500701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc256fbc640 1 --2- 172.21.15.32:0/1434389257 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2500aece0 0x7fc2500b05f0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc256fbc640 1 -- 172.21.15.32:0/1434389257 >> 172.21.15.32:0/1434389257 conn(0x7fc25006a320 msgr2=0x7fc2500ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:23.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc256fbc640 1 -- 172.21.15.32:0/1434389257 shutdown_connections 2023-12-20T06:02:23.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.780+0000 7fc256fbc640 1 -- 172.21.15.32:0/1434389257 wait complete. 2023-12-20T06:02:23.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc256fbc640 1 Processor -- start 2023-12-20T06:02:23.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc256fbc640 1 -- start start 2023-12-20T06:02:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc256fbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2500aece0 0x7fc25014a940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc256fbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2500b0b30 0x7fc2500cd9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc256fbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2501599e0 0x7fc2500cdf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc256fbc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc250157080 con 0x7fc2500aece0 2023-12-20T06:02:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc256fbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2500ce450 con 0x7fc2501599e0 2023-12-20T06:02:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc256fbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc250157320 con 0x7fc2500b0b30 2023-12-20T06:02:23.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc254d58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2500aece0 0x7fc25014a940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc254d58640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2500aece0 0x7fc25014a940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46896/0 (socket says 172.21.15.32:46896) 2023-12-20T06:02:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc254d58640 1 -- 172.21.15.32:0/3053213563 learned_addr learned my addr 172.21.15.32:0/3053213563 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc24ffff640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2500b0b30 0x7fc2500cd9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc255559640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2501599e0 0x7fc2500cdf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc254d58640 1 -- 172.21.15.32:0/3053213563 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2500b0b30 msgr2=0x7fc2500cd9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc254d58640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2500b0b30 0x7fc2500cd9d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc254d58640 1 -- 172.21.15.32:0/3053213563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2501599e0 msgr2=0x7fc2500cdf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc254d58640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2501599e0 0x7fc2500cdf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc254d58640 1 -- 172.21.15.32:0/3053213563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c0094d0 con 0x7fc2500aece0 2023-12-20T06:02:23.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc254d58640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2500aece0 0x7fc25014a940 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fc24400c410 tx=0x7fc24400c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc24d7fa640 1 -- 172.21.15.32:0/3053213563 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2440054e0 con 0x7fc2500aece0 2023-12-20T06:02:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc24d7fa640 1 -- 172.21.15.32:0/3053213563 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc244003650 con 0x7fc2500aece0 2023-12-20T06:02:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc24d7fa640 1 -- 172.21.15.32:0/3053213563 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc244003a00 con 0x7fc2500aece0 2023-12-20T06:02:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc256fbc640 1 -- 172.21.15.32:0/3053213563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc250157600 con 0x7fc2500aece0 2023-12-20T06:02:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc256fbc640 1 -- 172.21.15.32:0/3053213563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc2500ce700 con 0x7fc2500aece0 2023-12-20T06:02:23.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.784+0000 7fc256fbc640 1 -- 172.21.15.32:0/3053213563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc25004e000 con 0x7fc2500aece0 2023-12-20T06:02:23.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.788+0000 7fc24d7fa640 1 -- 172.21.15.32:0/3053213563 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fc244003b60 con 0x7fc2500aece0 2023-12-20T06:02:23.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.788+0000 7fc24d7fa640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc234065b80 0x7fc234068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:23.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.788+0000 7fc24d7fa640 1 -- 172.21.15.32:0/3053213563 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fc2440825e0 con 0x7fc2500aece0 2023-12-20T06:02:23.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.788+0000 7fc24ffff640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc234065b80 0x7fc234068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:23.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.788+0000 7fc24ffff640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc234065b80 0x7fc234068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0af000 tx=0x7fc23c0a7000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:23.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.792+0000 7fc24d7fa640 1 -- 172.21.15.32:0/3053213563 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc244050230 con 0x7fc2500aece0 2023-12-20T06:02:23.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 -- 172.21.15.32:0/3053213563 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc250157ab0 con 0x7fc2500aece0 2023-12-20T06:02:23.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc24d7fa640 1 -- 172.21.15.32:0/3053213563 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v205) v1 ==== 74+0+12441 (secure 0 0 0) 0x7fc244017090 con 0x7fc2500aece0 2023-12-20T06:02:23.954 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:23.955 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":205,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:23.208131+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":23,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":190,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 -- 172.21.15.32:0/3053213563 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc234065b80 msgr2=0x7fc234068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc234065b80 0x7fc234068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7fc23c0af000 tx=0x7fc23c0a7000 comp rx=0 tx=0).stop 2023-12-20T06:02:23.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 -- 172.21.15.32:0/3053213563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2500aece0 msgr2=0x7fc25014a940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2500aece0 0x7fc25014a940 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7fc24400c410 tx=0x7fc24400c8d0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 -- 172.21.15.32:0/3053213563 shutdown_connections 2023-12-20T06:02:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc234065b80 0x7fc234068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2501599e0 0x7fc2500cdf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2500b0b30 0x7fc2500cd9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 --2- 172.21.15.32:0/3053213563 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc2500aece0 0x7fc25014a940 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 -- 172.21.15.32:0/3053213563 >> 172.21.15.32:0/3053213563 conn(0x7fc25006a320 msgr2=0x7fc2500cb500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 -- 172.21.15.32:0/3053213563 shutdown_connections 2023-12-20T06:02:23.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:23.952+0000 7fc256fbc640 1 -- 172.21.15.32:0/3053213563 wait complete. 2023-12-20T06:02:23.976 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:02:23.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:02:24.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.120+0000 7ff0ccc73640 1 Processor -- start 2023-12-20T06:02:24.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0ccc73640 1 -- start start 2023-12-20T06:02:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0ccc73640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff0c80b0cb0 0x7ff0c80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0ccc73640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c815b6a0 con 0x7ff0c815abb0 2023-12-20T06:02:24.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0ccc73640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff0c806d8c0 con 0x7ff0c8070230 2023-12-20T06:02:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0ccc73640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff0c806e160 con 0x7ff0c80b0cb0 2023-12-20T06:02:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff0c815abb0 0x7ff0c815bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51118/0 (socket says 172.21.15.32:51118) 2023-12-20T06:02:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c659c640 1 -- 172.21.15.32:0/4283956330 learned_addr learned my addr 172.21.15.32:0/4283956330 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c5d9b640 1 --2- 172.21.15.32:0/4283956330 >> v2:172.21.15.39:3300/0 conn(0x7ff0c80b0cb0 0x7ff0c80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2511479710 0 0) 0x7ff0c815b6a0 con 0x7ff0c815abb0 2023-12-20T06:02:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff0b0003530 con 0x7ff0c815abb0 2023-12-20T06:02:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2624619832 0 0) 0x7ff0b0003530 con 0x7ff0c815abb0 2023-12-20T06:02:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 >> v2:172.21.15.39:3300/0 conn(0x7ff0c80b0cb0 msgr2=0x7ff0c80caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:24.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 --2- 172.21.15.32:0/4283956330 >> v2:172.21.15.39:3300/0 conn(0x7ff0c80b0cb0 0x7ff0c80caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 >> v1:172.21.15.39:6789/0 conn(0x7ff0c8070230 legacy=0x7ff0c806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c806dba0 con 0x7ff0c815abb0 2023-12-20T06:02:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7ff0b8001680 con 0x7ff0c815abb0 2023-12-20T06:02:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 >> v1:172.21.15.32:6789/0 conn(0x7ff0c815abb0 legacy=0x7ff0c815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 --2- 172.21.15.32:0/4283956330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0b0003980 0x7ff0b0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 --2- 172.21.15.32:0/4283956330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b00047d0 0x7ff0b0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 --2- 172.21.15.32:0/4283956330 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0b0005190 0x7ff0b000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c806e160 con 0x7ff0b00047d0 2023-12-20T06:02:24.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b000c030 con 0x7ff0b0003980 2023-12-20T06:02:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0b000c1f0 con 0x7ff0b0005190 2023-12-20T06:02:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c5d9b640 1 --2- 172.21.15.32:0/4283956330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b00047d0 0x7ff0b0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c659c640 1 --2- 172.21.15.32:0/4283956330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0b0003980 0x7ff0b0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c6d9d640 1 --2- 172.21.15.32:0/4283956330 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0b0005190 0x7ff0b000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c5d9b640 1 -- 172.21.15.32:0/4283956330 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0b0005190 msgr2=0x7ff0b000b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c5d9b640 1 --2- 172.21.15.32:0/4283956330 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0b0005190 0x7ff0b000b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c5d9b640 1 -- 172.21.15.32:0/4283956330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0b0003980 msgr2=0x7ff0b0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c5d9b640 1 --2- 172.21.15.32:0/4283956330 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0b0003980 0x7ff0b0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c5d9b640 1 -- 172.21.15.32:0/4283956330 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0b0003530 con 0x7ff0b00047d0 2023-12-20T06:02:24.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.124+0000 7ff0c6d9d640 1 -- 172.21.15.32:0/4283956330 reap_dead start 2023-12-20T06:02:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.128+0000 7ff0c5d9b640 1 --2- 172.21.15.32:0/4283956330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b00047d0 0x7ff0b0004c50 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009ab0 tx=0x7ff0bc01ca30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf736457e3b6db02 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.128+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0a6420 con 0x7ff0b00047d0 2023-12-20T06:02:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.128+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0bc0a6580 con 0x7ff0b00047d0 2023-12-20T06:02:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.128+0000 7ff0c559a640 1 -- 172.21.15.32:0/4283956330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0bc0a6930 con 0x7ff0b00047d0 2023-12-20T06:02:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.128+0000 7ff0ccc73640 1 -- 172.21.15.32:0/4283956330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b00047d0 msgr2=0x7ff0b0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.128+0000 7ff0ccc73640 1 --2- 172.21.15.32:0/4283956330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b00047d0 0x7ff0b0004c50 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7ff0bc009ab0 tx=0x7ff0bc01ca30 comp rx=0 tx=0).stop 2023-12-20T06:02:24.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.128+0000 7ff0ccc73640 1 -- 172.21.15.32:0/4283956330 shutdown_connections 2023-12-20T06:02:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.128+0000 7ff0ccc73640 1 --2- 172.21.15.32:0/4283956330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0b00047d0 0x7ff0b0004c50 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.128+0000 7ff0ccc73640 1 -- 172.21.15.32:0/4283956330 >> 172.21.15.32:0/4283956330 conn(0x7ff0c806a510 msgr2=0x7ff0c80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.128+0000 7ff0ccc73640 1 -- 172.21.15.32:0/4283956330 shutdown_connections 2023-12-20T06:02:24.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.128+0000 7ff0ccc73640 1 -- 172.21.15.32:0/4283956330 wait complete. 2023-12-20T06:02:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0ccc73640 1 Processor -- start 2023-12-20T06:02:24.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0ccc73640 1 -- start start 2023-12-20T06:02:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0ccc73640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0b00047d0 0x7ff0c8156400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0ccc73640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0c80a7be0 0x7ff0c80a8060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0ccc73640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80aa380 0x7ff0c80aa800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0ccc73640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80aad40 con 0x7ff0c80aa380 2023-12-20T06:02:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0ccc73640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c80a9360 con 0x7ff0b00047d0 2023-12-20T06:02:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0ccc73640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff0c8156940 con 0x7ff0c80a7be0 2023-12-20T06:02:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0c6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80aa380 0x7ff0c80aa800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0c6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80aa380 0x7ff0c80aa800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46926/0 (socket says 172.21.15.32:46926) 2023-12-20T06:02:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0c6d9d640 1 -- 172.21.15.32:0/1428124428 learned_addr learned my addr 172.21.15.32:0/1428124428 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:24.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0c5d9b640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0c80a7be0 0x7ff0c80a8060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0c659c640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0b00047d0 0x7ff0c8156400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0c6d9d640 1 -- 172.21.15.32:0/1428124428 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0c80a7be0 msgr2=0x7ff0c80a8060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0c6d9d640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0c80a7be0 0x7ff0c80a8060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0c6d9d640 1 -- 172.21.15.32:0/1428124428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0b00047d0 msgr2=0x7ff0c8156400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0c6d9d640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0b00047d0 0x7ff0c8156400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0c6d9d640 1 -- 172.21.15.32:0/1428124428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff0c81f2820 con 0x7ff0c80aa380 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0c6d9d640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80aa380 0x7ff0c80aa800 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80b13f0 tx=0x7ff0c000eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0b6ffd640 1 -- 172.21.15.32:0/1428124428 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c000eed0 con 0x7ff0c80aa380 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0b6ffd640 1 -- 172.21.15.32:0/1428124428 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff0c0003980 con 0x7ff0c80aa380 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0b6ffd640 1 -- 172.21.15.32:0/1428124428 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff0c0003d70 con 0x7ff0c80aa380 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0ccc73640 1 -- 172.21.15.32:0/1428124428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff0c80a8750 con 0x7ff0c80aa380 2023-12-20T06:02:24.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.132+0000 7ff0ccc73640 1 -- 172.21.15.32:0/1428124428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff0c80a8d10 con 0x7ff0c80aa380 2023-12-20T06:02:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.136+0000 7ff0ccc73640 1 -- 172.21.15.32:0/1428124428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff0c80a94a0 con 0x7ff0c80aa380 2023-12-20T06:02:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.136+0000 7ff0b6ffd640 1 -- 172.21.15.32:0/1428124428 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff0c00050d0 con 0x7ff0c80aa380 2023-12-20T06:02:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.136+0000 7ff0b6ffd640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff0a8065b80 0x7ff0a8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.136+0000 7ff0b6ffd640 1 -- 172.21.15.32:0/1428124428 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 4913+0+0 (secure 0 0 0) 0x7ff0c0082b80 con 0x7ff0c80aa380 2023-12-20T06:02:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.136+0000 7ff0c659c640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff0a8065b80 0x7ff0a8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.136+0000 7ff0c659c640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff0a8065b80 0x7ff0a8068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8001060 tx=0x7ff0b8001c60 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:24.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.140+0000 7ff0b6ffd640 1 -- 172.21.15.32:0/1428124428 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff0c80a94a0 con 0x7ff0c80aa380 2023-12-20T06:02:24.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.296+0000 7ff0ccc73640 1 -- 172.21.15.32:0/1428124428 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff0c80a94a0 con 0x7ff0c80aa380 2023-12-20T06:02:24.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.296+0000 7ff0b6ffd640 1 -- 172.21.15.32:0/1428124428 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v205) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff0c80a94a0 con 0x7ff0c80aa380 2023-12-20T06:02:24.300 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:02:24.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 -- 172.21.15.32:0/1428124428 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff0a8065b80 msgr2=0x7ff0a8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff0a8065b80 0x7ff0a8068040 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7ff0b8001060 tx=0x7ff0b8001c60 comp rx=0 tx=0).stop 2023-12-20T06:02:24.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 -- 172.21.15.32:0/1428124428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80aa380 msgr2=0x7ff0c80aa800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80aa380 0x7ff0c80aa800 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80b13f0 tx=0x7ff0c000eb90 comp rx=0 tx=0).stop 2023-12-20T06:02:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 -- 172.21.15.32:0/1428124428 shutdown_connections 2023-12-20T06:02:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff0a8065b80 0x7ff0a8068040 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff0c80aa380 0x7ff0c80aa800 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff0c80a7be0 0x7ff0c80a8060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 --2- 172.21.15.32:0/1428124428 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff0b00047d0 0x7ff0c8156400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 -- 172.21.15.32:0/1428124428 >> 172.21.15.32:0/1428124428 conn(0x7ff0c806a510 msgr2=0x7ff0c815da20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 -- 172.21.15.32:0/1428124428 shutdown_connections 2023-12-20T06:02:24.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.300+0000 7ff0ccc73640 1 -- 172.21.15.32:0/1428124428 wait complete. 2023-12-20T06:02:24.322 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T06:02:24.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7f666640 1 Processor -- start 2023-12-20T06:02:24.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7f666640 1 -- start start 2023-12-20T06:02:24.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7f666640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc7806ce60 0x7fdc7815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7f666640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdc7815f110 0x7fdc7815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7f666640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdc7815bbd0 con 0x7fdc780af5e0 2023-12-20T06:02:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7f666640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7815beb0 con 0x7fdc7806ce60 2023-12-20T06:02:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7f666640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdc7815e910 con 0x7fdc7815f110 2023-12-20T06:02:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7dc03640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdc7815f110 0x7fdc7815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7d402640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fdc780af5e0 0x7fdc780af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:41208/0 (socket says 172.21.15.32:41208) 2023-12-20T06:02:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7d402640 1 -- 172.21.15.32:0/3852793638 learned_addr learned my addr 172.21.15.32:0/3852793638 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:24.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7cc01640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdc7806ce60 0x7fdc7815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc6ffff640 1 -- 172.21.15.32:0/3852793638 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080775428 0 0) 0x7fdc7815bbd0 con 0x7fdc780af5e0 2023-12-20T06:02:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc6ffff640 1 -- 172.21.15.32:0/3852793638 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdc64003530 con 0x7fdc780af5e0 2023-12-20T06:02:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7cc01640 1 -- 172.21.15.32:0/3852793638 >> v2:172.21.15.39:3300/0 conn(0x7fdc7815f110 msgr2=0x7fdc7815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7cc01640 1 --2- 172.21.15.32:0/3852793638 >> v2:172.21.15.39:3300/0 conn(0x7fdc7815f110 0x7fdc7815c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7cc01640 1 -- 172.21.15.32:0/3852793638 >> v1:172.21.15.39:6789/0 conn(0x7fdc780af5e0 legacy=0x7fdc780af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.464+0000 7fdc7cc01640 1 -- 172.21.15.32:0/3852793638 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc7815ebf0 con 0x7fdc7806ce60 2023-12-20T06:02:24.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc7cc01640 1 --2- 172.21.15.32:0/3852793638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc7806ce60 0x7fdc7815b280 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fdc780c8d40 tx=0x7fdc6801c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0bc1b837877530d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:24.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc6ffff640 1 -- 172.21.15.32:0/3852793638 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc7815b9d0 con 0x7fdc7806ce60 2023-12-20T06:02:24.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc6ffff640 1 -- 172.21.15.32:0/3852793638 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc680a7050 con 0x7fdc7806ce60 2023-12-20T06:02:24.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc6ffff640 1 -- 172.21.15.32:0/3852793638 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc680ab800 con 0x7fdc7806ce60 2023-12-20T06:02:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc7f666640 1 -- 172.21.15.32:0/3852793638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc7806ce60 msgr2=0x7fdc7815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc7f666640 1 --2- 172.21.15.32:0/3852793638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc7806ce60 0x7fdc7815b280 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7fdc780c8d40 tx=0x7fdc6801c3d0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc7f666640 1 -- 172.21.15.32:0/3852793638 shutdown_connections 2023-12-20T06:02:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc7f666640 1 --2- 172.21.15.32:0/3852793638 >> v2:172.21.15.39:3300/0 conn(0x7fdc7815f110 0x7fdc7815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc7f666640 1 --2- 172.21.15.32:0/3852793638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc7806ce60 0x7fdc7815b280 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc7f666640 1 -- 172.21.15.32:0/3852793638 >> 172.21.15.32:0/3852793638 conn(0x7fdc7806a320 msgr2=0x7fdc780ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc7f666640 1 -- 172.21.15.32:0/3852793638 shutdown_connections 2023-12-20T06:02:24.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.468+0000 7fdc7f666640 1 -- 172.21.15.32:0/3852793638 wait complete. 2023-12-20T06:02:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7f666640 1 Processor -- start 2023-12-20T06:02:24.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7f666640 1 -- start start 2023-12-20T06:02:24.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7f666640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7806ce60 0x7fdc780c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7f666640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc780af5e0 0x7fdc78154d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7f666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc7815f110 0x7fdc78141240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7f666640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc780c6060 con 0x7fdc780af5e0 2023-12-20T06:02:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7f666640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdc7814fee0 con 0x7fdc7806ce60 2023-12-20T06:02:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7f666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdc781501c0 con 0x7fdc7815f110 2023-12-20T06:02:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7cc01640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc780af5e0 0x7fdc78154d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7cc01640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc780af5e0 0x7fdc78154d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46946/0 (socket says 172.21.15.32:46946) 2023-12-20T06:02:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7cc01640 1 -- 172.21.15.32:0/2367126254 learned_addr learned my addr 172.21.15.32:0/2367126254 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7d402640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7806ce60 0x7fdc780c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7dc03640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc7815f110 0x7fdc78141240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7cc01640 1 -- 172.21.15.32:0/2367126254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc7815f110 msgr2=0x7fdc78141240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7cc01640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc7815f110 0x7fdc78141240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7cc01640 1 -- 172.21.15.32:0/2367126254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7806ce60 msgr2=0x7fdc780c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7cc01640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7806ce60 0x7fdc780c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7cc01640 1 -- 172.21.15.32:0/2367126254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdc68009420 con 0x7fdc780af5e0 2023-12-20T06:02:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7dc03640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc7815f110 0x7fdc78141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7cc01640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc780af5e0 0x7fdc78154d30 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fdc68000c10 tx=0x7fdc68019170 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc6dffb640 1 -- 172.21.15.32:0/2367126254 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc68006e20 con 0x7fdc780af5e0 2023-12-20T06:02:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc6dffb640 1 -- 172.21.15.32:0/2367126254 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdc680b4400 con 0x7fdc780af5e0 2023-12-20T06:02:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc7f666640 1 -- 172.21.15.32:0/2367126254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdc781504a0 con 0x7fdc780af5e0 2023-12-20T06:02:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.472+0000 7fdc6dffb640 1 -- 172.21.15.32:0/2367126254 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdc680b4770 con 0x7fdc780af5e0 2023-12-20T06:02:24.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.476+0000 7fdc7f666640 1 -- 172.21.15.32:0/2367126254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdc780c5d70 con 0x7fdc780af5e0 2023-12-20T06:02:24.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.476+0000 7fdc7f666640 1 -- 172.21.15.32:0/2367126254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdc7804e000 con 0x7fdc780af5e0 2023-12-20T06:02:24.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.476+0000 7fdc6dffb640 1 -- 172.21.15.32:0/2367126254 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fdc680b2040 con 0x7fdc780af5e0 2023-12-20T06:02:24.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.476+0000 7fdc6dffb640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc5c065b80 0x7fdc5c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.476+0000 7fdc6dffb640 1 -- 172.21.15.32:0/2367126254 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fdc6811cf50 con 0x7fdc780af5e0 2023-12-20T06:02:24.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.476+0000 7fdc7d402640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc5c065b80 0x7fdc5c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.476+0000 7fdc7d402640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc5c065b80 0x7fdc5c068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc740058e0 tx=0x7fdc74009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:24.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.480+0000 7fdc6dffb640 1 -- 172.21.15.32:0/2367126254 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdc680eaa00 con 0x7fdc780af5e0 2023-12-20T06:02:24.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.640+0000 7fdc7f666640 1 -- 172.21.15.32:0/2367126254 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fdc780cf450 con 0x7fdc780af5e0 2023-12-20T06:02:24.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.640+0000 7fdc6dffb640 1 -- 172.21.15.32:0/2367126254 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v205) v1 ==== 84+0+65 (secure 0 0 0) 0x7fdc680b0090 con 0x7fdc780af5e0 2023-12-20T06:02:24.644 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:24.644 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T06:02:24.644 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:24.644 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T06:02:24.644 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T06:02:24.644 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:24.644 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T06:02:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 -- 172.21.15.32:0/2367126254 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc5c065b80 msgr2=0x7fdc5c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc5c065b80 0x7fdc5c068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fdc740058e0 tx=0x7fdc74009040 comp rx=0 tx=0).stop 2023-12-20T06:02:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 -- 172.21.15.32:0/2367126254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc780af5e0 msgr2=0x7fdc78154d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc780af5e0 0x7fdc78154d30 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7fdc68000c10 tx=0x7fdc68019170 comp rx=0 tx=0).stop 2023-12-20T06:02:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 -- 172.21.15.32:0/2367126254 shutdown_connections 2023-12-20T06:02:24.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdc5c065b80 0x7fdc5c068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdc7815f110 0x7fdc78141240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdc780af5e0 0x7fdc78154d30 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 --2- 172.21.15.32:0/2367126254 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdc7806ce60 0x7fdc780c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 -- 172.21.15.32:0/2367126254 >> 172.21.15.32:0/2367126254 conn(0x7fdc7806a320 msgr2=0x7fdc78155c70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 -- 172.21.15.32:0/2367126254 shutdown_connections 2023-12-20T06:02:24.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.644+0000 7fdc7f666640 1 -- 172.21.15.32:0/2367126254 wait complete. 2023-12-20T06:02:24.665 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T06:02:24.666 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T06:02:24.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf92ff3640 1 Processor -- start 2023-12-20T06:02:24.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf92ff3640 1 -- start start 2023-12-20T06:02:24.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf92ff3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcf8c158520 0x7fcf8c06dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf92ff3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8c0ca7f0 con 0x7fcf8c070230 2023-12-20T06:02:24.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf92ff3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf8c0caad0 con 0x7fcf8c0ca050 2023-12-20T06:02:24.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf92ff3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf8c0cadb0 con 0x7fcf8c158520 2023-12-20T06:02:24.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf91590640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcf8c070230 0x7fcf8c15b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51124/0 (socket says 172.21.15.32:51124) 2023-12-20T06:02:24.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf91590640 1 -- 172.21.15.32:0/3368754390 learned_addr learned my addr 172.21.15.32:0/3368754390 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:24.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf90d8f640 1 --2- 172.21.15.32:0/3368754390 >> v2:172.21.15.39:3300/0 conn(0x7fcf8c158520 0x7fcf8c06dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf90d8f640 1 -- 172.21.15.32:0/3368754390 >> v1:172.21.15.39:6789/0 conn(0x7fcf8c0ca050 legacy=0x7fcf8c15ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf8b7fe640 1 -- 172.21.15.32:0/3368754390 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1489766845 0 0) 0x7fcf8c0caad0 con 0x7fcf8c0ca050 2023-12-20T06:02:24.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf90d8f640 1 -- 172.21.15.32:0/3368754390 >> v1:172.21.15.32:6789/0 conn(0x7fcf8c070230 legacy=0x7fcf8c15b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.812+0000 7fcf90d8f640 1 -- 172.21.15.32:0/3368754390 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf8c15e780 con 0x7fcf8c158520 2023-12-20T06:02:24.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.816+0000 7fcf90d8f640 1 --2- 172.21.15.32:0/3368754390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf8c158520 0x7fcf8c06dcb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fcf800097c0 tx=0x7fcf8001c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b578f24658f2e337 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:24.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.816+0000 7fcf8b7fe640 1 -- 172.21.15.32:0/3368754390 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf800ac640 con 0x7fcf8c158520 2023-12-20T06:02:24.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.816+0000 7fcf8b7fe640 1 -- 172.21.15.32:0/3368754390 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf800ac7a0 con 0x7fcf8c158520 2023-12-20T06:02:24.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.816+0000 7fcf8b7fe640 1 -- 172.21.15.32:0/3368754390 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf800acb30 con 0x7fcf8c158520 2023-12-20T06:02:24.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.816+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3368754390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf8c158520 msgr2=0x7fcf8c06dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.816+0000 7fcf92ff3640 1 --2- 172.21.15.32:0/3368754390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf8c158520 0x7fcf8c06dcb0 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7fcf800097c0 tx=0x7fcf8001c3c0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.816+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3368754390 shutdown_connections 2023-12-20T06:02:24.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.816+0000 7fcf92ff3640 1 --2- 172.21.15.32:0/3368754390 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf8c158520 0x7fcf8c06dcb0 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.816+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3368754390 >> 172.21.15.32:0/3368754390 conn(0x7fcf8c06a510 msgr2=0x7fcf8c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:24.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.816+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3368754390 shutdown_connections 2023-12-20T06:02:24.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.816+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3368754390 wait complete. 2023-12-20T06:02:24.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf92ff3640 1 Processor -- start 2023-12-20T06:02:24.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf92ff3640 1 -- start start 2023-12-20T06:02:24.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf92ff3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf8c070230 0x7fcf8c158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf92ff3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf8c0ca050 0x7fcf8c0c49b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf92ff3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf8c158520 0x7fcf8c14da80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf92ff3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c1514a0 con 0x7fcf8c158520 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf92ff3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c159d10 con 0x7fcf8c070230 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf92ff3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf8c151cb0 con 0x7fcf8c0ca050 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf91590640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf8c158520 0x7fcf8c14da80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf91590640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf8c158520 0x7fcf8c14da80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46962/0 (socket says 172.21.15.32:46962) 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf91590640 1 -- 172.21.15.32:0/3938227159 learned_addr learned my addr 172.21.15.32:0/3938227159 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf90d8f640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf8c070230 0x7fcf8c158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf8bfff640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf8c0ca050 0x7fcf8c0c49b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf91590640 1 -- 172.21.15.32:0/3938227159 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf8c0ca050 msgr2=0x7fcf8c0c49b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf91590640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf8c0ca050 0x7fcf8c0c49b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf91590640 1 -- 172.21.15.32:0/3938227159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf8c070230 msgr2=0x7fcf8c158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf91590640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf8c070230 0x7fcf8c158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:24.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf91590640 1 -- 172.21.15.32:0/3938227159 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf80009420 con 0x7fcf8c158520 2023-12-20T06:02:24.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf8bfff640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf8c0ca050 0x7fcf8c0c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:24.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf91590640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf8c158520 0x7fcf8c14da80 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400cc20 tx=0x7fcf8400a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:24.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf897fa640 1 -- 172.21.15.32:0/3938227159 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8400ad40 con 0x7fcf8c158520 2023-12-20T06:02:24.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf897fa640 1 -- 172.21.15.32:0/3938227159 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf84012400 con 0x7fcf8c158520 2023-12-20T06:02:24.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf897fa640 1 -- 172.21.15.32:0/3938227159 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf8400aea0 con 0x7fcf8c158520 2023-12-20T06:02:24.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3938227159 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf8c070bd0 con 0x7fcf8c158520 2023-12-20T06:02:24.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.820+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3938227159 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf8c1519d0 con 0x7fcf8c158520 2023-12-20T06:02:24.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.824+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3938227159 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf8c04e000 con 0x7fcf8c158520 2023-12-20T06:02:24.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.824+0000 7fcf897fa640 1 -- 172.21.15.32:0/3938227159 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcf84007070 con 0x7fcf8c158520 2023-12-20T06:02:24.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.824+0000 7fcf897fa640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf70065b80 0x7fcf70068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:24.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.824+0000 7fcf897fa640 1 -- 172.21.15.32:0/3938227159 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(205..205 src has 1..205) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fcf84082f00 con 0x7fcf8c158520 2023-12-20T06:02:24.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.824+0000 7fcf90d8f640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf70065b80 0x7fcf70068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:24.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.824+0000 7fcf90d8f640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf70065b80 0x7fcf70068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf800ae000 tx=0x7fcf800a7000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:24.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.828+0000 7fcf897fa640 1 -- 172.21.15.32:0/3938227159 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcf84050ad0 con 0x7fcf8c158520 2023-12-20T06:02:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:24.988+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3938227159 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fcf8c159e50 con 0x7fcf8c158520 2023-12-20T06:02:25.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.528+0000 7fcf897fa640 1 -- 172.21.15.32:0/3938227159 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v206) v1 ==== 138+0+0 (secure 0 0 0) 0x7fcf84017020 con 0x7fcf8c158520 2023-12-20T06:02:25.529 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T06:02:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.528+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3938227159 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf70065b80 msgr2=0x7fcf70068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.528+0000 7fcf92ff3640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf70065b80 0x7fcf70068040 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fcf800ae000 tx=0x7fcf800a7000 comp rx=0 tx=0).stop 2023-12-20T06:02:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.528+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3938227159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf8c158520 msgr2=0x7fcf8c14da80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.528+0000 7fcf92ff3640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf8c158520 0x7fcf8c14da80 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7fcf8400cc20 tx=0x7fcf8400a430 comp rx=0 tx=0).stop 2023-12-20T06:02:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.532+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3938227159 shutdown_connections 2023-12-20T06:02:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.532+0000 7fcf92ff3640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf70065b80 0x7fcf70068040 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.532+0000 7fcf92ff3640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf8c158520 0x7fcf8c14da80 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.532+0000 7fcf92ff3640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf8c0ca050 0x7fcf8c0c49b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.532+0000 7fcf92ff3640 1 --2- 172.21.15.32:0/3938227159 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf8c070230 0x7fcf8c158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.532+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3938227159 >> 172.21.15.32:0/3938227159 conn(0x7fcf8c06a510 msgr2=0x7fcf8c0c36d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.532+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3938227159 shutdown_connections 2023-12-20T06:02:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.532+0000 7fcf92ff3640 1 -- 172.21.15.32:0/3938227159 wait complete. 2023-12-20T06:02:25.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T06:02:25.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7dba1f0640 1 Processor -- start 2023-12-20T06:02:25.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7dba1f0640 1 -- start start 2023-12-20T06:02:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7dba1f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7db406dbf0 0x7f7db40cb700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7dba1f0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7db414fa00 0x7f7db414fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7dba1f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7db406cbc0 con 0x7f7db40cb320 2023-12-20T06:02:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7dba1f0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db406ce30 con 0x7f7db406dbf0 2023-12-20T06:02:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7dba1f0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7db406d110 con 0x7f7db414fa00 2023-12-20T06:02:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7db2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7db406dbf0 0x7f7db40cb700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7db2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7db406dbf0 0x7f7db40cb700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46978/0 (socket says 172.21.15.32:46978) 2023-12-20T06:02:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7db2ffd640 1 -- 172.21.15.32:0/2756273638 learned_addr learned my addr 172.21.15.32:0/2756273638 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7db3fff640 1 --2- 172.21.15.32:0/2756273638 >> v2:172.21.15.39:3300/0 conn(0x7f7db414fa00 0x7f7db414fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7db2ffd640 1 -- 172.21.15.32:0/2756273638 >> v2:172.21.15.39:3300/0 conn(0x7f7db414fa00 msgr2=0x7f7db414fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.708+0000 7f7db2ffd640 1 --2- 172.21.15.32:0/2756273638 >> v2:172.21.15.39:3300/0 conn(0x7f7db414fa00 0x7f7db414fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7db2ffd640 1 -- 172.21.15.32:0/2756273638 >> v1:172.21.15.39:6789/0 conn(0x7f7db40cb320 legacy=0x7f7db4159f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:25.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7db2ffd640 1 -- 172.21.15.32:0/2756273638 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7db406d3f0 con 0x7f7db406dbf0 2023-12-20T06:02:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7db2ffd640 1 --2- 172.21.15.32:0/2756273638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db406dbf0 0x7f7db40cb700 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f7db40c3c90 tx=0x7f7da801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe0a5d05bc9ccc74 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7db27fc640 1 -- 172.21.15.32:0/2756273638 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da80ac600 con 0x7f7db406dbf0 2023-12-20T06:02:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7db27fc640 1 -- 172.21.15.32:0/2756273638 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da80a8050 con 0x7f7db406dbf0 2023-12-20T06:02:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7db27fc640 1 -- 172.21.15.32:0/2756273638 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da80ac980 con 0x7f7db406dbf0 2023-12-20T06:02:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2756273638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db406dbf0 msgr2=0x7f7db40cb700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7dba1f0640 1 --2- 172.21.15.32:0/2756273638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db406dbf0 0x7f7db40cb700 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f7db40c3c90 tx=0x7f7da801c490 comp rx=0 tx=0).stop 2023-12-20T06:02:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2756273638 shutdown_connections 2023-12-20T06:02:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7dba1f0640 1 --2- 172.21.15.32:0/2756273638 >> v2:172.21.15.39:3300/0 conn(0x7f7db414fa00 0x7f7db414fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7dba1f0640 1 --2- 172.21.15.32:0/2756273638 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db406dbf0 0x7f7db40cb700 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2756273638 >> 172.21.15.32:0/2756273638 conn(0x7f7db406a4b0 msgr2=0x7f7db40b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2756273638 shutdown_connections 2023-12-20T06:02:25.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.712+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2756273638 wait complete. 2023-12-20T06:02:25.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7dba1f0640 1 Processor -- start 2023-12-20T06:02:25.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7dba1f0640 1 -- start start 2023-12-20T06:02:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7dba1f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7db406dbf0 0x7f7db414c760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7dba1f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40cb320 0x7f7db40c9e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7dba1f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7db414fa00 0x7f7db40ca340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:25.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7dba1f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40c7110 con 0x7f7db40cb320 2023-12-20T06:02:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7dba1f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40af2b0 con 0x7f7db406dbf0 2023-12-20T06:02:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7dba1f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7db40af590 con 0x7f7db414fa00 2023-12-20T06:02:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7db2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40cb320 0x7f7db40c9e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7db2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40cb320 0x7f7db40c9e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46990/0 (socket says 172.21.15.32:46990) 2023-12-20T06:02:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7db2ffd640 1 -- 172.21.15.32:0/2071433847 learned_addr learned my addr 172.21.15.32:0/2071433847 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7db37fe640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7db406dbf0 0x7f7db414c760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7db3fff640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7db414fa00 0x7f7db40ca340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7db2ffd640 1 -- 172.21.15.32:0/2071433847 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7db414fa00 msgr2=0x7f7db40ca340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:25.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7db2ffd640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7db414fa00 0x7f7db40ca340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7db2ffd640 1 -- 172.21.15.32:0/2071433847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7db406dbf0 msgr2=0x7f7db414c760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7db2ffd640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7db406dbf0 0x7f7db414c760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7db2ffd640 1 -- 172.21.15.32:0/2071433847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7da80094d0 con 0x7f7db40cb320 2023-12-20T06:02:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7db2ffd640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40cb320 0x7f7db40c9e00 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f7da801a0c0 tx=0x7f7da801acd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7d9bfff640 1 -- 172.21.15.32:0/2071433847 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da80bc070 con 0x7f7db40cb320 2023-12-20T06:02:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7d9bfff640 1 -- 172.21.15.32:0/2071433847 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7da80ace70 con 0x7f7db40cb320 2023-12-20T06:02:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7d9bfff640 1 -- 172.21.15.32:0/2071433847 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7da80b7650 con 0x7f7db40cb320 2023-12-20T06:02:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.716+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2071433847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7db40af870 con 0x7f7db40cb320 2023-12-20T06:02:25.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.720+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2071433847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7db414cca0 con 0x7f7db40cb320 2023-12-20T06:02:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.720+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2071433847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7db404e000 con 0x7f7db40cb320 2023-12-20T06:02:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.720+0000 7f7d9bfff640 1 -- 172.21.15.32:0/2071433847 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7da80b3070 con 0x7f7db40cb320 2023-12-20T06:02:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.720+0000 7f7d9bfff640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7da0065b80 0x7f7da0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:25.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.720+0000 7f7d9bfff640 1 -- 172.21.15.32:0/2071433847 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(206..206 src has 1..206) v4 ==== 5283+0+0 (secure 0 0 0) 0x7f7da811d870 con 0x7f7db40cb320 2023-12-20T06:02:25.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.720+0000 7f7db37fe640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7da0065b80 0x7f7da0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:25.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.720+0000 7f7db37fe640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7da0065b80 0x7f7da0068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7da4007fd0 tx=0x7f7da4007e40 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:25.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.724+0000 7f7d9bfff640 1 -- 172.21.15.32:0/2071433847 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7da80eb3d0 con 0x7f7db40cb320 2023-12-20T06:02:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:25.884+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2071433847 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f7db40c3fa0 con 0x7f7db40cb320 2023-12-20T06:02:26.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.532+0000 7f7d9bfff640 1 -- 172.21.15.32:0/2071433847 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v207) v1 ==== 170+0+0 (secure 0 0 0) 0x7f7da80ee9f0 con 0x7f7db40cb320 2023-12-20T06:02:26.535 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T06:02:26.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.544+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2071433847 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7da0065b80 msgr2=0x7f7da0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:26.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.544+0000 7f7dba1f0640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7da0065b80 0x7f7da0068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f7da4007fd0 tx=0x7f7da4007e40 comp rx=0 tx=0).stop 2023-12-20T06:02:26.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.544+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2071433847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40cb320 msgr2=0x7f7db40c9e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:26.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.544+0000 7f7dba1f0640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40cb320 0x7f7db40c9e00 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f7da801a0c0 tx=0x7f7da801acd0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.544+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2071433847 shutdown_connections 2023-12-20T06:02:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.544+0000 7f7dba1f0640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7da0065b80 0x7f7da0068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.544+0000 7f7dba1f0640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7db414fa00 0x7f7db40ca340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.544+0000 7f7dba1f0640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7db40cb320 0x7f7db40c9e00 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.544+0000 7f7dba1f0640 1 --2- 172.21.15.32:0/2071433847 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7db406dbf0 0x7f7db414c760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.544+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2071433847 >> 172.21.15.32:0/2071433847 conn(0x7f7db406a4b0 msgr2=0x7f7db4159f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:26.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.548+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2071433847 shutdown_connections 2023-12-20T06:02:26.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.548+0000 7f7dba1f0640 1 -- 172.21.15.32:0/2071433847 wait complete. 2023-12-20T06:02:26.581 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T06:02:26.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.728+0000 7fa790066640 1 Processor -- start 2023-12-20T06:02:26.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.728+0000 7fa790066640 1 -- start start 2023-12-20T06:02:26.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.728+0000 7fa790066640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa7880b0760 0x7fa7880b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:26.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.728+0000 7fa790066640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7880b1100 con 0x7fa7880cc4b0 2023-12-20T06:02:26.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.728+0000 7fa790066640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa7880cde20 con 0x7fa788157480 2023-12-20T06:02:26.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.728+0000 7fa790066640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa7880c9f50 con 0x7fa7880b0760 2023-12-20T06:02:26.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.732+0000 7fa78e603640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa7880cc4b0 0x7fa7880cc8d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51134/0 (socket says 172.21.15.32:51134) 2023-12-20T06:02:26.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.732+0000 7fa78e603640 1 -- 172.21.15.32:0/3611201771 learned_addr learned my addr 172.21.15.32:0/3611201771 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:26.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.732+0000 7fa78d601640 1 --2- 172.21.15.32:0/3611201771 >> v2:172.21.15.39:3300/0 conn(0x7fa7880b0760 0x7fa7880b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:26.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.732+0000 7fa78ce00640 1 -- 172.21.15.32:0/3611201771 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4251492856 0 0) 0x7fa7880b1100 con 0x7fa7880cc4b0 2023-12-20T06:02:26.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.732+0000 7fa78ce00640 1 -- 172.21.15.32:0/3611201771 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa774005740 con 0x7fa7880cc4b0 2023-12-20T06:02:26.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.732+0000 7fa78ce00640 1 -- 172.21.15.32:0/3611201771 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2892252280 0 0) 0x7fa7880cde20 con 0x7fa788157480 2023-12-20T06:02:26.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.732+0000 7fa78ce00640 1 -- 172.21.15.32:0/3611201771 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa7880b1100 con 0x7fa788157480 2023-12-20T06:02:26.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.732+0000 7fa78d601640 1 -- 172.21.15.32:0/3611201771 >> v1:172.21.15.39:6789/0 conn(0x7fa788157480 legacy=0x7fa7880c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:26.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.732+0000 7fa78d601640 1 -- 172.21.15.32:0/3611201771 >> v1:172.21.15.32:6789/0 conn(0x7fa7880cc4b0 legacy=0x7fa7880cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:26.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.732+0000 7fa78d601640 1 -- 172.21.15.32:0/3611201771 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa7880c8e30 con 0x7fa7880b0760 2023-12-20T06:02:26.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.732+0000 7fa78d601640 1 --2- 172.21.15.32:0/3611201771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7880b0760 0x7fa7880b0bc0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa7780097c0 tx=0x7fa77801c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea371c90678ff5a7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:26.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa78ce00640 1 -- 172.21.15.32:0/3611201771 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7780ac620 con 0x7fa7880b0760 2023-12-20T06:02:26.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa78ce00640 1 -- 172.21.15.32:0/3611201771 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa7780ac780 con 0x7fa7880b0760 2023-12-20T06:02:26.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa78ce00640 1 -- 172.21.15.32:0/3611201771 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa7780acaf0 con 0x7fa7880b0760 2023-12-20T06:02:26.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 -- 172.21.15.32:0/3611201771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7880b0760 msgr2=0x7fa7880b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:26.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 --2- 172.21.15.32:0/3611201771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7880b0760 0x7fa7880b0bc0 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7fa7780097c0 tx=0x7fa77801c3c0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 -- 172.21.15.32:0/3611201771 shutdown_connections 2023-12-20T06:02:26.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 --2- 172.21.15.32:0/3611201771 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7880b0760 0x7fa7880b0bc0 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 -- 172.21.15.32:0/3611201771 >> 172.21.15.32:0/3611201771 conn(0x7fa78806a510 msgr2=0x7fa7880ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:26.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 -- 172.21.15.32:0/3611201771 shutdown_connections 2023-12-20T06:02:26.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 -- 172.21.15.32:0/3611201771 wait complete. 2023-12-20T06:02:26.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 Processor -- start 2023-12-20T06:02:26.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 -- start start 2023-12-20T06:02:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7880b0760 0x7fa788152a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7880cc4b0 0x7fa78815c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788157480 0x7fa7881f5520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa7881f5a60 con 0x7fa788157480 2023-12-20T06:02:26.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa78815af70 con 0x7fa7880cc4b0 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.736+0000 7fa790066640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa78815b250 con 0x7fa7880b0760 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa78e603640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788157480 0x7fa7881f5520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa78e603640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788157480 0x7fa7881f5520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46994/0 (socket says 172.21.15.32:46994) 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa78e603640 1 -- 172.21.15.32:0/2382107170 learned_addr learned my addr 172.21.15.32:0/2382107170 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa78d601640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7880cc4b0 0x7fa78815c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa78de02640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7880b0760 0x7fa788152a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa78e603640 1 -- 172.21.15.32:0/2382107170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7880b0760 msgr2=0x7fa788152a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa78e603640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7880b0760 0x7fa788152a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa78e603640 1 -- 172.21.15.32:0/2382107170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7880cc4b0 msgr2=0x7fa78815c010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa78e603640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7880cc4b0 0x7fa78815c010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa78e603640 1 -- 172.21.15.32:0/2382107170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa778009420 con 0x7fa788157480 2023-12-20T06:02:26.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa78e603640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788157480 0x7fa7881f5520 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fa78000cba0 tx=0x7fa78000a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:26.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa77e7fc640 1 -- 172.21.15.32:0/2382107170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa78000ace0 con 0x7fa788157480 2023-12-20T06:02:26.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa77e7fc640 1 -- 172.21.15.32:0/2382107170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa78000ae40 con 0x7fa788157480 2023-12-20T06:02:26.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa77e7fc640 1 -- 172.21.15.32:0/2382107170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa780012620 con 0x7fa788157480 2023-12-20T06:02:26.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa790066640 1 -- 172.21.15.32:0/2382107170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa78815b530 con 0x7fa788157480 2023-12-20T06:02:26.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa790066640 1 -- 172.21.15.32:0/2382107170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa78815baf0 con 0x7fa788157480 2023-12-20T06:02:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa790066640 1 -- 172.21.15.32:0/2382107170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa78815bce0 con 0x7fa788157480 2023-12-20T06:02:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.740+0000 7fa77e7fc640 1 -- 172.21.15.32:0/2382107170 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa780007070 con 0x7fa788157480 2023-12-20T06:02:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.744+0000 7fa77e7fc640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa76c065bd0 0x7fa76c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:26.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.744+0000 7fa77e7fc640 1 -- 172.21.15.32:0/2382107170 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(207..207 src has 1..207) v4 ==== 5665+0+0 (secure 0 0 0) 0x7fa7800836f0 con 0x7fa788157480 2023-12-20T06:02:26.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.744+0000 7fa78de02640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa76c065bd0 0x7fa76c068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:26.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.744+0000 7fa78de02640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa76c065bd0 0x7fa76c068090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa7840011f0 tx=0x7fa784009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:26.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.748+0000 7fa77e7fc640 1 -- 172.21.15.32:0/2382107170 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa780051050 con 0x7fa788157480 2023-12-20T06:02:26.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.936+0000 7fa790066640 1 -- 172.21.15.32:0/2382107170 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7fa78804e000 con 0x7fa788157480 2023-12-20T06:02:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.944+0000 7fa77e7fc640 1 -- 172.21.15.32:0/2382107170 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 24 and data pool 25 v92) v1 ==== 174+0+0 (secure 0 0 0) 0x7fa780054670 con 0x7fa788157480 2023-12-20T06:02:26.946 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 24 and data pool 25 2023-12-20T06:02:26.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 -- 172.21.15.32:0/2382107170 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa76c065bd0 msgr2=0x7fa76c068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:26.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa76c065bd0 0x7fa76c068090 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fa7840011f0 tx=0x7fa784009040 comp rx=0 tx=0).stop 2023-12-20T06:02:26.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 -- 172.21.15.32:0/2382107170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788157480 msgr2=0x7fa7881f5520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:26.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788157480 0x7fa7881f5520 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7fa78000cba0 tx=0x7fa78000a430 comp rx=0 tx=0).stop 2023-12-20T06:02:26.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 -- 172.21.15.32:0/2382107170 shutdown_connections 2023-12-20T06:02:26.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa76c065bd0 0x7fa76c068090 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa788157480 0x7fa7881f5520 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa7880cc4b0 0x7fa78815c010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 --2- 172.21.15.32:0/2382107170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa7880b0760 0x7fa788152a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:26.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 -- 172.21.15.32:0/2382107170 >> 172.21.15.32:0/2382107170 conn(0x7fa78806a510 msgr2=0x7fa7880c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:26.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 -- 172.21.15.32:0/2382107170 shutdown_connections 2023-12-20T06:02:26.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:26.952+0000 7fa790066640 1 -- 172.21.15.32:0/2382107170 wait complete. 2023-12-20T06:02:26.995 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.136+0000 7f04304b6640 1 Processor -- start 2023-12-20T06:02:27.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.136+0000 7f04304b6640 1 -- start start 2023-12-20T06:02:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.136+0000 7f04304b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0428160af0 0x7f0428160f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.136+0000 7f04304b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04280ac260 0x7f04280ac6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.136+0000 7f04304b6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0428161490 con 0x7f04281603e0 2023-12-20T06:02:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.136+0000 7f04304b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f04280719b0 con 0x7f04280ac260 2023-12-20T06:02:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.136+0000 7f04304b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04280713f0 con 0x7f0428160af0 2023-12-20T06:02:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.136+0000 7f042ea53640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f04280ac260 0x7f04280ac6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.136+0000 7f042e252640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f04281603e0 0x7f04280cd300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51148/0 (socket says 172.21.15.32:51148) 2023-12-20T06:02:27.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.136+0000 7f042e252640 1 -- 172.21.15.32:0/205864312 learned_addr learned my addr 172.21.15.32:0/205864312 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.136+0000 7f042da51640 1 --2- 172.21.15.32:0/205864312 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0428160af0 0x7f0428160f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.140+0000 7f042ea53640 1 -- 172.21.15.32:0/205864312 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0428160af0 msgr2=0x7f0428160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.140+0000 7f042ea53640 1 --2- 172.21.15.32:0/205864312 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0428160af0 0x7f0428160f50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.140+0000 7f042ea53640 1 -- 172.21.15.32:0/205864312 >> v1:172.21.15.32:6789/0 conn(0x7f04281603e0 legacy=0x7f04280cd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.140+0000 7f042ea53640 1 -- 172.21.15.32:0/205864312 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f04280716d0 con 0x7f04280ac260 2023-12-20T06:02:27.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.140+0000 7f042ea53640 1 --2- 172.21.15.32:0/205864312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04280ac260 0x7f04280ac6f0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f0420009d90 tx=0x7f042001c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2610eae5a94a02b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:27.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.140+0000 7f042d250640 1 -- 172.21.15.32:0/205864312 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04200a58c0 con 0x7f04280ac260 2023-12-20T06:02:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.140+0000 7f042d250640 1 -- 172.21.15.32:0/205864312 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f04200a5a20 con 0x7f04280ac260 2023-12-20T06:02:27.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.140+0000 7f042d250640 1 -- 172.21.15.32:0/205864312 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04200a5db0 con 0x7f04280ac260 2023-12-20T06:02:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.140+0000 7f04304b6640 1 -- 172.21.15.32:0/205864312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04280ac260 msgr2=0x7f04280ac6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.140+0000 7f04304b6640 1 --2- 172.21.15.32:0/205864312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04280ac260 0x7f04280ac6f0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f0420009d90 tx=0x7f042001c800 comp rx=0 tx=0).stop 2023-12-20T06:02:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 -- 172.21.15.32:0/205864312 shutdown_connections 2023-12-20T06:02:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 --2- 172.21.15.32:0/205864312 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04280ac260 0x7f04280ac6f0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 --2- 172.21.15.32:0/205864312 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0428160af0 0x7f0428160f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 -- 172.21.15.32:0/205864312 >> 172.21.15.32:0/205864312 conn(0x7f042806e110 msgr2=0x7f042806e520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 -- 172.21.15.32:0/205864312 shutdown_connections 2023-12-20T06:02:27.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 -- 172.21.15.32:0/205864312 wait complete. 2023-12-20T06:02:27.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 Processor -- start 2023-12-20T06:02:27.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 -- start start 2023-12-20T06:02:27.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04280ac260 0x7f04280c4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f04281603e0 0x7f04280c4aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0428160af0 0x7f04281f8ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042815dbb0 con 0x7f04280ac260 2023-12-20T06:02:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f042814fac0 con 0x7f0428160af0 2023-12-20T06:02:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f04281f5410 con 0x7f04281603e0 2023-12-20T06:02:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042e252640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04280ac260 0x7f04280c4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042e252640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04280ac260 0x7f04280c4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47010/0 (socket says 172.21.15.32:47010) 2023-12-20T06:02:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042e252640 1 -- 172.21.15.32:0/543806520 learned_addr learned my addr 172.21.15.32:0/543806520 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:27.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042da51640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f04281603e0 0x7f04280c4aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042ea53640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0428160af0 0x7f04281f8ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042e252640 1 -- 172.21.15.32:0/543806520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f04281603e0 msgr2=0x7f04280c4aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042e252640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f04281603e0 0x7f04280c4aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042e252640 1 -- 172.21.15.32:0/543806520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0428160af0 msgr2=0x7f04281f8ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042e252640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0428160af0 0x7f04281f8ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042e252640 1 -- 172.21.15.32:0/543806520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0420009a70 con 0x7f04280ac260 2023-12-20T06:02:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042ea53640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0428160af0 0x7f04281f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f042e252640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04280ac260 0x7f04280c4560 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f042400c500 tx=0x7f042400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f041effd640 1 -- 172.21.15.32:0/543806520 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0424005cf0 con 0x7f04280ac260 2023-12-20T06:02:27.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f041effd640 1 -- 172.21.15.32:0/543806520 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0424005e50 con 0x7f04280ac260 2023-12-20T06:02:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f04304b6640 1 -- 172.21.15.32:0/543806520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f042815cf30 con 0x7f04280ac260 2023-12-20T06:02:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.144+0000 7f041effd640 1 -- 172.21.15.32:0/543806520 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f04240072f0 con 0x7f04280ac260 2023-12-20T06:02:27.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.148+0000 7f04304b6640 1 -- 172.21.15.32:0/543806520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f042815d4f0 con 0x7f04280ac260 2023-12-20T06:02:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.148+0000 7f04304b6640 1 -- 172.21.15.32:0/543806520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f04280ae380 con 0x7f04280ac260 2023-12-20T06:02:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.148+0000 7f041effd640 1 -- 172.21.15.32:0/543806520 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0424007450 con 0x7f04280ac260 2023-12-20T06:02:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.148+0000 7f041effd640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f040c065b80 0x7f040c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.148+0000 7f041effd640 1 -- 172.21.15.32:0/543806520 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0424082840 con 0x7f04280ac260 2023-12-20T06:02:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.148+0000 7f042da51640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f040c065b80 0x7f040c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.148+0000 7f042da51640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f040c065b80 0x7f040c068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f04280c7180 tx=0x7f0418008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:27.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.152+0000 7f041effd640 1 -- 172.21.15.32:0/543806520 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0424050150 con 0x7f04280ac260 2023-12-20T06:02:27.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 -- 172.21.15.32:0/543806520 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f042804e050 con 0x7f04280ac260 2023-12-20T06:02:27.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f041effd640 1 -- 172.21.15.32:0/543806520 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+16166 (secure 0 0 0) 0x7f0424017090 con 0x7f04280ac260 2023-12-20T06:02:27.314 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:27.314 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":208,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:26.942198+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:02:25.045126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-20T06:02:25.938205+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 -- 172.21.15.32:0/543806520 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f040c065b80 msgr2=0x7f040c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f040c065b80 0x7f040c068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f04280c7180 tx=0x7f0418008040 comp rx=0 tx=0).stop 2023-12-20T06:02:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 -- 172.21.15.32:0/543806520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04280ac260 msgr2=0x7f04280c4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04280ac260 0x7f04280c4560 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f042400c500 tx=0x7f042400c9c0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 -- 172.21.15.32:0/543806520 shutdown_connections 2023-12-20T06:02:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f040c065b80 0x7f040c068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0428160af0 0x7f04281f8ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f04281603e0 0x7f04280c4aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 --2- 172.21.15.32:0/543806520 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f04280ac260 0x7f04280c4560 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 -- 172.21.15.32:0/543806520 >> 172.21.15.32:0/543806520 conn(0x7f042806e110 msgr2=0x7f042806c640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 -- 172.21.15.32:0/543806520 shutdown_connections 2023-12-20T06:02:27.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.312+0000 7f04304b6640 1 -- 172.21.15.32:0/543806520 wait complete. 2023-12-20T06:02:27.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:27.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.476+0000 7f56de6df640 1 Processor -- start 2023-12-20T06:02:27.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.476+0000 7f56de6df640 1 -- start start 2023-12-20T06:02:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56de6df640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56d80af5e0 0x7f56d80af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56de6df640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f56d815f110 0x7f56d815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56de6df640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f56d815b990 con 0x7f56d806ce60 2023-12-20T06:02:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56de6df640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d815bbd0 con 0x7f56d80af5e0 2023-12-20T06:02:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56de6df640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f56d815beb0 con 0x7f56d815f110 2023-12-20T06:02:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56d7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56d80af5e0 0x7f56d80af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56d7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f56d80af5e0 0x7f56d80af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47016/0 (socket says 172.21.15.32:47016) 2023-12-20T06:02:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56d7fff640 1 -- 172.21.15.32:0/1426297201 learned_addr learned my addr 172.21.15.32:0/1426297201 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:27.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56dcc7c640 1 --2- 172.21.15.32:0/1426297201 >> v2:172.21.15.39:3300/0 conn(0x7f56d815f110 0x7f56d815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56d7fff640 1 -- 172.21.15.32:0/1426297201 >> v2:172.21.15.39:3300/0 conn(0x7f56d815f110 msgr2=0x7f56d815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56d7fff640 1 --2- 172.21.15.32:0/1426297201 >> v2:172.21.15.39:3300/0 conn(0x7f56d815f110 0x7f56d815c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56d7fff640 1 -- 172.21.15.32:0/1426297201 >> v1:172.21.15.39:6789/0 conn(0x7f56d806ce60 legacy=0x7f56d815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56d7fff640 1 -- 172.21.15.32:0/1426297201 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56d815e910 con 0x7f56d80af5e0 2023-12-20T06:02:27.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56dcc7c640 1 --2- 172.21.15.32:0/1426297201 >> v2:172.21.15.39:3300/0 conn(0x7f56d815f110 0x7f56d815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:27.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.480+0000 7f56d7fff640 1 --2- 172.21.15.32:0/1426297201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56d80af5e0 0x7f56d80af9c0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f56d80c8d40 tx=0x7f56c801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97ff3452878dbecd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:27.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56d6ffd640 1 -- 172.21.15.32:0/1426297201 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c80aa610 con 0x7f56d80af5e0 2023-12-20T06:02:27.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56d6ffd640 1 -- 172.21.15.32:0/1426297201 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c80a6040 con 0x7f56d80af5e0 2023-12-20T06:02:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56d6ffd640 1 -- 172.21.15.32:0/1426297201 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c80aa990 con 0x7f56d80af5e0 2023-12-20T06:02:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 -- 172.21.15.32:0/1426297201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56d80af5e0 msgr2=0x7f56d80af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 --2- 172.21.15.32:0/1426297201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56d80af5e0 0x7f56d80af9c0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f56d80c8d40 tx=0x7f56c801c490 comp rx=0 tx=0).stop 2023-12-20T06:02:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 -- 172.21.15.32:0/1426297201 shutdown_connections 2023-12-20T06:02:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 --2- 172.21.15.32:0/1426297201 >> v2:172.21.15.39:3300/0 conn(0x7f56d815f110 0x7f56d815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 --2- 172.21.15.32:0/1426297201 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56d80af5e0 0x7f56d80af9c0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 -- 172.21.15.32:0/1426297201 >> 172.21.15.32:0/1426297201 conn(0x7f56d806a320 msgr2=0x7f56d80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 -- 172.21.15.32:0/1426297201 shutdown_connections 2023-12-20T06:02:27.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 -- 172.21.15.32:0/1426297201 wait complete. 2023-12-20T06:02:27.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 Processor -- start 2023-12-20T06:02:27.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 -- start start 2023-12-20T06:02:27.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56d806ce60 0x7f56d814fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d80af5e0 0x7f56d80c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56d815f110 0x7f56d80b2040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d80c8710 con 0x7f56d806ce60 2023-12-20T06:02:27.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f56d8154010 con 0x7f56d80af5e0 2023-12-20T06:02:27.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56de6df640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f56d80c8430 con 0x7f56d815f110 2023-12-20T06:02:27.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.484+0000 7f56d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56d806ce60 0x7f56d814fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56d806ce60 0x7f56d814fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47030/0 (socket says 172.21.15.32:47030) 2023-12-20T06:02:27.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d7fff640 1 -- 172.21.15.32:0/1280476692 learned_addr learned my addr 172.21.15.32:0/1280476692 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:27.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56dcc7c640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56d815f110 0x7f56d80b2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d77fe640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d80af5e0 0x7f56d80c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d7fff640 1 -- 172.21.15.32:0/1280476692 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56d815f110 msgr2=0x7f56d80b2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d7fff640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56d815f110 0x7f56d80b2040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d7fff640 1 -- 172.21.15.32:0/1280476692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d80af5e0 msgr2=0x7f56d80c5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d7fff640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d80af5e0 0x7f56d80c5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d7fff640 1 -- 172.21.15.32:0/1280476692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f56c8009480 con 0x7f56d806ce60 2023-12-20T06:02:27.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d77fe640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d80af5e0 0x7f56d80c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:27.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d7fff640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56d806ce60 0x7f56d814fbc0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f56c8000c10 tx=0x7f56c8019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d4ff9640 1 -- 172.21.15.32:0/1280476692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c80bb070 con 0x7f56d806ce60 2023-12-20T06:02:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d4ff9640 1 -- 172.21.15.32:0/1280476692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f56c8003b20 con 0x7f56d806ce60 2023-12-20T06:02:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56de6df640 1 -- 172.21.15.32:0/1280476692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f56d81539c0 con 0x7f56d806ce60 2023-12-20T06:02:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d4ff9640 1 -- 172.21.15.32:0/1280476692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f56c80aa410 con 0x7f56d806ce60 2023-12-20T06:02:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56de6df640 1 -- 172.21.15.32:0/1280476692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f56d80c5d70 con 0x7f56d806ce60 2023-12-20T06:02:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56de6df640 1 -- 172.21.15.32:0/1280476692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f56d8153820 con 0x7f56d806ce60 2023-12-20T06:02:27.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.488+0000 7f56d4ff9640 1 -- 172.21.15.32:0/1280476692 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f56c80aa570 con 0x7f56d806ce60 2023-12-20T06:02:27.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.492+0000 7f56d4ff9640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f56c4065b80 0x7f56c4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.492+0000 7f56d4ff9640 1 -- 172.21.15.32:0/1280476692 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f56c811d300 con 0x7f56d806ce60 2023-12-20T06:02:27.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.492+0000 7f56d77fe640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f56c4065b80 0x7f56c4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.492+0000 7f56d77fe640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f56c4065b80 0x7f56c4068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f56d8153670 tx=0x7f56cc009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:27.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.496+0000 7f56d4ff9640 1 -- 172.21.15.32:0/1280476692 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f56d8153820 con 0x7f56d806ce60 2023-12-20T06:02:27.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.652+0000 7f56de6df640 1 -- 172.21.15.32:0/1280476692 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f56d815b990 con 0x7f56d806ce60 2023-12-20T06:02:27.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56d4ff9640 1 -- 172.21.15.32:0/1280476692 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v208) v1 ==== 74+0+16166 (secure 0 0 0) 0x7f56d815b990 con 0x7f56d806ce60 2023-12-20T06:02:27.657 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:27.657 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":208,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:26.942198+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:02:25.045126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-20T06:02:25.938205+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:27.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 -- 172.21.15.32:0/1280476692 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f56c4065b80 msgr2=0x7f56c4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f56c4065b80 0x7f56c4068040 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f56d8153670 tx=0x7f56cc009040 comp rx=0 tx=0).stop 2023-12-20T06:02:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 -- 172.21.15.32:0/1280476692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56d806ce60 msgr2=0x7f56d814fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56d806ce60 0x7f56d814fbc0 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f56c8000c10 tx=0x7f56c8019230 comp rx=0 tx=0).stop 2023-12-20T06:02:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 -- 172.21.15.32:0/1280476692 shutdown_connections 2023-12-20T06:02:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f56c4065b80 0x7f56c4068040 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f56d815f110 0x7f56d80b2040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f56d80af5e0 0x7f56d80c5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 --2- 172.21.15.32:0/1280476692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f56d806ce60 0x7f56d814fbc0 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 -- 172.21.15.32:0/1280476692 >> 172.21.15.32:0/1280476692 conn(0x7f56d806a320 msgr2=0x7f56d80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 -- 172.21.15.32:0/1280476692 shutdown_connections 2023-12-20T06:02:27.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.656+0000 7f56de6df640 1 -- 172.21.15.32:0/1280476692 wait complete. 2023-12-20T06:02:27.679 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T06:02:27.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e6866640 1 Processor -- start 2023-12-20T06:02:27.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e6866640 1 -- start start 2023-12-20T06:02:27.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e6866640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f23e00b0760 0x7f23e00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e6866640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23e00cc4b0 0x7f23e00cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e6866640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f23e00b1100 con 0x7f23e0157480 2023-12-20T06:02:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e6866640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e00c9f50 con 0x7f23e00cc4b0 2023-12-20T06:02:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e6866640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f23e00cde20 con 0x7f23e00b0760 2023-12-20T06:02:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e4e03640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23e00cc4b0 0x7f23e00cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e4e03640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f23e00cc4b0 0x7f23e00cc8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47036/0 (socket says 172.21.15.32:47036) 2023-12-20T06:02:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e4e03640 1 -- 172.21.15.32:0/272191285 learned_addr learned my addr 172.21.15.32:0/272191285 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23df7fe640 1 --2- 172.21.15.32:0/272191285 >> v2:172.21.15.39:3300/0 conn(0x7f23e00b0760 0x7f23e00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e4e03640 1 -- 172.21.15.32:0/272191285 >> v2:172.21.15.39:3300/0 conn(0x7f23e00b0760 msgr2=0x7f23e00b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e4e03640 1 --2- 172.21.15.32:0/272191285 >> v2:172.21.15.39:3300/0 conn(0x7f23e00b0760 0x7f23e00b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e4e03640 1 -- 172.21.15.32:0/272191285 >> v1:172.21.15.39:6789/0 conn(0x7f23e0157480 legacy=0x7f23e00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.824+0000 7f23e4e03640 1 -- 172.21.15.32:0/272191285 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23e00c8e30 con 0x7f23e00cc4b0 2023-12-20T06:02:27.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23e4e03640 1 --2- 172.21.15.32:0/272191285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e00cc4b0 0x7f23e00cc8d0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f23e006e0d0 tx=0x7f23d801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=609234fdbb62b514 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23deffd640 1 -- 172.21.15.32:0/272191285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d80a57d0 con 0x7f23e00cc4b0 2023-12-20T06:02:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23deffd640 1 -- 172.21.15.32:0/272191285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d80a5930 con 0x7f23e00cc4b0 2023-12-20T06:02:27.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23deffd640 1 -- 172.21.15.32:0/272191285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d80a5cc0 con 0x7f23e00cc4b0 2023-12-20T06:02:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23e6866640 1 -- 172.21.15.32:0/272191285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e00cc4b0 msgr2=0x7f23e00cc8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23e6866640 1 --2- 172.21.15.32:0/272191285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e00cc4b0 0x7f23e00cc8d0 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7f23e006e0d0 tx=0x7f23d801c910 comp rx=0 tx=0).stop 2023-12-20T06:02:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23e6866640 1 -- 172.21.15.32:0/272191285 shutdown_connections 2023-12-20T06:02:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23e6866640 1 --2- 172.21.15.32:0/272191285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e00cc4b0 0x7f23e00cc8d0 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23e6866640 1 --2- 172.21.15.32:0/272191285 >> v2:172.21.15.39:3300/0 conn(0x7f23e00b0760 0x7f23e00b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23e6866640 1 -- 172.21.15.32:0/272191285 >> 172.21.15.32:0/272191285 conn(0x7f23e006a510 msgr2=0x7f23e00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:27.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23e6866640 1 -- 172.21.15.32:0/272191285 shutdown_connections 2023-12-20T06:02:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.828+0000 7f23e6866640 1 -- 172.21.15.32:0/272191285 wait complete. 2023-12-20T06:02:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e6866640 1 Processor -- start 2023-12-20T06:02:27.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e6866640 1 -- start start 2023-12-20T06:02:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e6866640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e00b0760 0x7f23e015da30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e6866640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e00cc4b0 0x7f23e00b18f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e6866640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e0157480 0x7f23e0162b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e6866640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e015adf0 con 0x7f23e00b0760 2023-12-20T06:02:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e6866640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f23e015b040 con 0x7f23e00cc4b0 2023-12-20T06:02:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e6866640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f23e015b320 con 0x7f23e0157480 2023-12-20T06:02:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e00b0760 0x7f23e015da30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e00b0760 0x7f23e015da30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47050/0 (socket says 172.21.15.32:47050) 2023-12-20T06:02:27.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dffff640 1 -- 172.21.15.32:0/2865364136 learned_addr learned my addr 172.21.15.32:0/2865364136 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23df7fe640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e00cc4b0 0x7f23e00b18f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e4e03640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e0157480 0x7f23e0162b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dffff640 1 -- 172.21.15.32:0/2865364136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e0157480 msgr2=0x7f23e0162b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dffff640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e0157480 0x7f23e0162b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dffff640 1 -- 172.21.15.32:0/2865364136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e00cc4b0 msgr2=0x7f23e00b18f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dffff640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e00cc4b0 0x7f23e00b18f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dffff640 1 -- 172.21.15.32:0/2865364136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f23d8009b30 con 0x7f23e00b0760 2023-12-20T06:02:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e4e03640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e0157480 0x7f23e0162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dffff640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e00b0760 0x7f23e015da30 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c500 tx=0x7f23d000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:27.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dcff9640 1 -- 172.21.15.32:0/2865364136 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0005220 con 0x7f23e00b0760 2023-12-20T06:02:27.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dcff9640 1 -- 172.21.15.32:0/2865364136 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f23d0005380 con 0x7f23e00b0760 2023-12-20T06:02:27.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23dcff9640 1 -- 172.21.15.32:0/2865364136 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f23d0007270 con 0x7f23e00b0760 2023-12-20T06:02:27.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e6866640 1 -- 172.21.15.32:0/2865364136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f23e015b600 con 0x7f23e00b0760 2023-12-20T06:02:27.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.832+0000 7f23e6866640 1 -- 172.21.15.32:0/2865364136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f23e00b1e30 con 0x7f23e00b0760 2023-12-20T06:02:27.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.836+0000 7f23e6866640 1 -- 172.21.15.32:0/2865364136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f23e004e000 con 0x7f23e00b0760 2023-12-20T06:02:27.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.836+0000 7f23dcff9640 1 -- 172.21.15.32:0/2865364136 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f23d00073d0 con 0x7f23e00b0760 2023-12-20T06:02:27.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.836+0000 7f23dcff9640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f23cc065b80 0x7f23cc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:27.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.836+0000 7f23dcff9640 1 -- 172.21.15.32:0/2865364136 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(208..208 src has 1..208) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f23d00825c0 con 0x7f23e00b0760 2023-12-20T06:02:27.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.836+0000 7f23df7fe640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f23cc065b80 0x7f23cc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:27.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.836+0000 7f23df7fe640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f23cc065b80 0x7f23cc068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23e00c7f40 tx=0x7f23d4008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:27.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:27.840+0000 7f23dcff9640 1 -- 172.21.15.32:0/2865364136 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f23d004fef0 con 0x7f23e00b0760 2023-12-20T06:02:28.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.036+0000 7f23e6866640 1 -- 172.21.15.32:0/2865364136 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f23e01f8410 con 0x7f23e00b0760 2023-12-20T06:02:28.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.952+0000 7f23dcff9640 1 -- 172.21.15.32:0/2865364136 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v95) v1 ==== 118+0+0 (secure 0 0 0) 0x7f23d0053510 con 0x7f23e00b0760 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 -- 172.21.15.32:0/2865364136 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f23cc065b80 msgr2=0x7f23cc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f23cc065b80 0x7f23cc068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f23e00c7f40 tx=0x7f23d4008040 comp rx=0 tx=0).stop 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 -- 172.21.15.32:0/2865364136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e00b0760 msgr2=0x7f23e015da30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e00b0760 0x7f23e015da30 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7f23d000c500 tx=0x7f23d000c9c0 comp rx=0 tx=0).stop 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 -- 172.21.15.32:0/2865364136 shutdown_connections 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f23cc065b80 0x7f23cc068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f23e0157480 0x7f23e0162b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f23e00cc4b0 0x7f23e00b18f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 --2- 172.21.15.32:0/2865364136 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f23e00b0760 0x7f23e015da30 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 -- 172.21.15.32:0/2865364136 >> 172.21.15.32:0/2865364136 conn(0x7f23e006a510 msgr2=0x7f23e00c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 -- 172.21.15.32:0/2865364136 shutdown_connections 2023-12-20T06:02:28.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:28.956+0000 7f23e6866640 1 -- 172.21.15.32:0/2865364136 wait complete. 2023-12-20T06:02:28.981 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T06:02:28.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T06:02:29.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3c495640 1 Processor -- start 2023-12-20T06:02:29.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3c495640 1 -- start start 2023-12-20T06:02:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3c495640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbf3415ce50 0x7fbf3406b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3c495640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3406bb80 0x7fbf340cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:29.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3c495640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf340b0550 con 0x7fbf340cf5d0 2023-12-20T06:02:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3c495640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf3406bfc0 con 0x7fbf3415ce50 2023-12-20T06:02:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3c495640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf340aff20 con 0x7fbf3406bb80 2023-12-20T06:02:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3a231640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbf3415ce50 0x7fbf3406b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf39a30640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3406bb80 0x7fbf340cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3a231640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbf3415ce50 0x7fbf3406b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:45310/0 (socket says 172.21.15.32:45310) 2023-12-20T06:02:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3a231640 1 -- 172.21.15.32:0/469827406 learned_addr learned my addr 172.21.15.32:0/469827406 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3a231640 1 -- 172.21.15.32:0/469827406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3406bb80 msgr2=0x7fbf340cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:29.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3a231640 1 --2- 172.21.15.32:0/469827406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3406bb80 0x7fbf340cf090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3a231640 1 -- 172.21.15.32:0/469827406 >> v1:172.21.15.39:6789/0 conn(0x7fbf340cf5d0 legacy=0x7fbf34160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3a231640 1 -- 172.21.15.32:0/469827406 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340cfdc0 con 0x7fbf3415ce50 2023-12-20T06:02:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf39a30640 1 --2- 172.21.15.32:0/469827406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3406bb80 0x7fbf340cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:29.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.132+0000 7fbf3922f640 1 -- 172.21.15.32:0/469827406 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3551765020 0 0) 0x7fbf340b0550 con 0x7fbf340cf5d0 2023-12-20T06:02:29.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3a231640 1 --2- 172.21.15.32:0/469827406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3415ce50 0x7fbf3406b640 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009870 tx=0x7fbf3001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=93ff68e77a852cc5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:29.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3922f640 1 -- 172.21.15.32:0/469827406 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300aa670 con 0x7fbf3415ce50 2023-12-20T06:02:29.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3922f640 1 -- 172.21.15.32:0/469827406 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf300aa7d0 con 0x7fbf3415ce50 2023-12-20T06:02:29.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3922f640 1 -- 172.21.15.32:0/469827406 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf300aab60 con 0x7fbf3415ce50 2023-12-20T06:02:29.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3c495640 1 -- 172.21.15.32:0/469827406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3415ce50 msgr2=0x7fbf3406b640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:29.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3c495640 1 --2- 172.21.15.32:0/469827406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3415ce50 0x7fbf3406b640 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7fbf30009870 tx=0x7fbf3001c510 comp rx=0 tx=0).stop 2023-12-20T06:02:29.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3c495640 1 -- 172.21.15.32:0/469827406 shutdown_connections 2023-12-20T06:02:29.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3c495640 1 --2- 172.21.15.32:0/469827406 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3406bb80 0x7fbf340cf090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:29.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3c495640 1 --2- 172.21.15.32:0/469827406 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3415ce50 0x7fbf3406b640 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:29.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3c495640 1 -- 172.21.15.32:0/469827406 >> 172.21.15.32:0/469827406 conn(0x7fbf34099d70 msgr2=0x7fbf340d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:29.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3c495640 1 -- 172.21.15.32:0/469827406 shutdown_connections 2023-12-20T06:02:29.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.136+0000 7fbf3c495640 1 -- 172.21.15.32:0/469827406 wait complete. 2023-12-20T06:02:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3c495640 1 Processor -- start 2023-12-20T06:02:29.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3c495640 1 -- start start 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3c495640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3406bb80 0x7fbf34157a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3c495640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf340cf5d0 0x7fbf341531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3c495640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3415ce50 0x7fbf341536e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3c495640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf34160240 con 0x7fbf340cf5d0 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3c495640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf34156720 con 0x7fbf3415ce50 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3c495640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf34157580 con 0x7fbf3406bb80 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf39a30640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf340cf5d0 0x7fbf341531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf39a30640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf340cf5d0 0x7fbf341531a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42798/0 (socket says 172.21.15.32:42798) 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf39a30640 1 -- 172.21.15.32:0/773610555 learned_addr learned my addr 172.21.15.32:0/773610555 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3a231640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3406bb80 0x7fbf34157a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3aa32640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3415ce50 0x7fbf341536e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf39a30640 1 -- 172.21.15.32:0/773610555 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3406bb80 msgr2=0x7fbf34157a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf39a30640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3406bb80 0x7fbf34157a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:29.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf39a30640 1 -- 172.21.15.32:0/773610555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3415ce50 msgr2=0x7fbf341536e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:29.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf39a30640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3415ce50 0x7fbf341536e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:29.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf39a30640 1 -- 172.21.15.32:0/773610555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf24009420 con 0x7fbf340cf5d0 2023-12-20T06:02:29.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf39a30640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf340cf5d0 0x7fbf341531a0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400ebe0 tx=0x7fbf2400c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:29.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf2affd640 1 -- 172.21.15.32:0/773610555 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2400eee0 con 0x7fbf340cf5d0 2023-12-20T06:02:29.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf2affd640 1 -- 172.21.15.32:0/773610555 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2400cc30 con 0x7fbf340cf5d0 2023-12-20T06:02:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf2affd640 1 -- 172.21.15.32:0/773610555 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf24013470 con 0x7fbf340cf5d0 2023-12-20T06:02:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3c495640 1 -- 172.21.15.32:0/773610555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf300094d0 con 0x7fbf340cf5d0 2023-12-20T06:02:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.140+0000 7fbf3c495640 1 -- 172.21.15.32:0/773610555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf34156c50 con 0x7fbf340cf5d0 2023-12-20T06:02:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.144+0000 7fbf3c495640 1 -- 172.21.15.32:0/773610555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf341576f0 con 0x7fbf340cf5d0 2023-12-20T06:02:29.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.144+0000 7fbf2affd640 1 -- 172.21.15.32:0/773610555 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fbf240135d0 con 0x7fbf340cf5d0 2023-12-20T06:02:29.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.144+0000 7fbf2affd640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf18065b80 0x7fbf18068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:29.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.144+0000 7fbf2affd640 1 -- 172.21.15.32:0/773610555 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fbf240835a0 con 0x7fbf340cf5d0 2023-12-20T06:02:29.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.144+0000 7fbf3a231640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf18065b80 0x7fbf18068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:29.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.144+0000 7fbf3a231640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf18065b80 0x7fbf18068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf300ae000 tx=0x7fbf300b5040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:29.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.148+0000 7fbf2affd640 1 -- 172.21.15.32:0/773610555 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbf24050eb0 con 0x7fbf340cf5d0 2023-12-20T06:02:29.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.340+0000 7fbf3c495640 1 -- 172.21.15.32:0/773610555 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7fbf340ce120 con 0x7fbf340cf5d0 2023-12-20T06:02:29.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.952+0000 7fbf2affd640 1 -- 172.21.15.32:0/773610555 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v96) v1 ==== 122+0+0 (secure 0 0 0) 0x7fbf240544d0 con 0x7fbf340cf5d0 2023-12-20T06:02:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 -- 172.21.15.32:0/773610555 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf18065b80 msgr2=0x7fbf18068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:29.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf18065b80 0x7fbf18068040 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fbf300ae000 tx=0x7fbf300b5040 comp rx=0 tx=0).stop 2023-12-20T06:02:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 -- 172.21.15.32:0/773610555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf340cf5d0 msgr2=0x7fbf341531a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf340cf5d0 0x7fbf341531a0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fbf2400ebe0 tx=0x7fbf2400c520 comp rx=0 tx=0).stop 2023-12-20T06:02:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 -- 172.21.15.32:0/773610555 shutdown_connections 2023-12-20T06:02:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf18065b80 0x7fbf18068040 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf3415ce50 0x7fbf341536e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf340cf5d0 0x7fbf341531a0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 --2- 172.21.15.32:0/773610555 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3406bb80 0x7fbf34157a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 -- 172.21.15.32:0/773610555 >> 172.21.15.32:0/773610555 conn(0x7fbf34099d70 msgr2=0x7fbf340b5f60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 -- 172.21.15.32:0/773610555 shutdown_connections 2023-12-20T06:02:29.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:29.956+0000 7fbf3c495640 1 -- 172.21.15.32:0/773610555 wait complete. 2023-12-20T06:02:29.982 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T06:02:29.982 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:02:30.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3de2e5640 1 Processor -- start 2023-12-20T06:02:30.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3de2e5640 1 -- start start 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3de2e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa3d80d2740 0x7fa3d80cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3de2e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3d80cfab0 0x7fa3d80b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3de2e5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3d80cfec0 con 0x7fa3d815cdb0 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3de2e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d815d190 con 0x7fa3d80cfab0 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3de2e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3d80d16a0 con 0x7fa3d80d2740 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3dc882640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3d80cfab0 0x7fa3d80b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3dc882640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa3d80cfab0 0x7fa3d80b5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42806/0 (socket says 172.21.15.32:42806) 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3dc882640 1 -- 172.21.15.32:0/4245512525 learned_addr learned my addr 172.21.15.32:0/4245512525 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3d77fe640 1 --2- 172.21.15.32:0/4245512525 >> v2:172.21.15.39:3300/0 conn(0x7fa3d80d2740 0x7fa3d80cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3dc882640 1 -- 172.21.15.32:0/4245512525 >> v2:172.21.15.39:3300/0 conn(0x7fa3d80d2740 msgr2=0x7fa3d80cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3dc882640 1 --2- 172.21.15.32:0/4245512525 >> v2:172.21.15.39:3300/0 conn(0x7fa3d80d2740 0x7fa3d80cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3dc882640 1 -- 172.21.15.32:0/4245512525 >> v1:172.21.15.39:6789/0 conn(0x7fa3d815cdb0 legacy=0x7fa3d80d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3dc882640 1 -- 172.21.15.32:0/4245512525 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d80d1980 con 0x7fa3d80cfab0 2023-12-20T06:02:30.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.132+0000 7fa3d77fe640 1 --2- 172.21.15.32:0/4245512525 >> v2:172.21.15.39:3300/0 conn(0x7fa3d80d2740 0x7fa3d80cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:30.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3dc882640 1 --2- 172.21.15.32:0/4245512525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d80cfab0 0x7fa3d80b5c90 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80cd570 tx=0x7fa3d001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=942201ba38b952e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:30.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3d6ffd640 1 -- 172.21.15.32:0/4245512525 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00af400 con 0x7fa3d80cfab0 2023-12-20T06:02:30.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3d6ffd640 1 -- 172.21.15.32:0/4245512525 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d00ab050 con 0x7fa3d80cfab0 2023-12-20T06:02:30.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3d6ffd640 1 -- 172.21.15.32:0/4245512525 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00af780 con 0x7fa3d80cfab0 2023-12-20T06:02:30.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3de2e5640 1 -- 172.21.15.32:0/4245512525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d80cfab0 msgr2=0x7fa3d80b5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3de2e5640 1 --2- 172.21.15.32:0/4245512525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d80cfab0 0x7fa3d80b5c90 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fa3d80cd570 tx=0x7fa3d001f1c0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3de2e5640 1 -- 172.21.15.32:0/4245512525 shutdown_connections 2023-12-20T06:02:30.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3de2e5640 1 --2- 172.21.15.32:0/4245512525 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d80cfab0 0x7fa3d80b5c90 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3de2e5640 1 --2- 172.21.15.32:0/4245512525 >> v2:172.21.15.39:3300/0 conn(0x7fa3d80d2740 0x7fa3d80cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3de2e5640 1 -- 172.21.15.32:0/4245512525 >> 172.21.15.32:0/4245512525 conn(0x7fa3d8065ee0 msgr2=0x7fa3d806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:30.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3de2e5640 1 -- 172.21.15.32:0/4245512525 shutdown_connections 2023-12-20T06:02:30.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.136+0000 7fa3de2e5640 1 -- 172.21.15.32:0/4245512525 wait complete. 2023-12-20T06:02:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3de2e5640 1 Processor -- start 2023-12-20T06:02:30.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3de2e5640 1 -- start start 2023-12-20T06:02:30.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3de2e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3d80cfab0 0x7fa3d80c8be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3de2e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d80d2740 0x7fa3d81e7960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3de2e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d815cdb0 0x7fa3d815da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3de2e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81e8fd0 con 0x7fa3d815cdb0 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3de2e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81e8450 con 0x7fa3d80d2740 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3de2e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3d81e8730 con 0x7fa3d80cfab0 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3dc882640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d815cdb0 0x7fa3d815da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3dc882640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d815cdb0 0x7fa3d815da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42818/0 (socket says 172.21.15.32:42818) 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3dc882640 1 -- 172.21.15.32:0/147871632 learned_addr learned my addr 172.21.15.32:0/147871632 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3d77fe640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d80d2740 0x7fa3d81e7960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3d7fff640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3d80cfab0 0x7fa3d80c8be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3dc882640 1 -- 172.21.15.32:0/147871632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3d80cfab0 msgr2=0x7fa3d80c8be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3dc882640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3d80cfab0 0x7fa3d80c8be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3dc882640 1 -- 172.21.15.32:0/147871632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d80d2740 msgr2=0x7fa3d81e7960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3dc882640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d80d2740 0x7fa3d81e7960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3dc882640 1 -- 172.21.15.32:0/147871632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3d0009b30 con 0x7fa3d815cdb0 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3d77fe640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d80d2740 0x7fa3d81e7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3dc882640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d815cdb0 0x7fa3d815da60 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fa3d001bdd0 tx=0x7fa3d001bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:30.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3d4ff9640 1 -- 172.21.15.32:0/147871632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00afdc0 con 0x7fa3d815cdb0 2023-12-20T06:02:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3d4ff9640 1 -- 172.21.15.32:0/147871632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3d0009510 con 0x7fa3d815cdb0 2023-12-20T06:02:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3d4ff9640 1 -- 172.21.15.32:0/147871632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3d00ba660 con 0x7fa3d815cdb0 2023-12-20T06:02:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3de2e5640 1 -- 172.21.15.32:0/147871632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3d81e8a10 con 0x7fa3d815cdb0 2023-12-20T06:02:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.140+0000 7fa3de2e5640 1 -- 172.21.15.32:0/147871632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3d80c9120 con 0x7fa3d815cdb0 2023-12-20T06:02:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.144+0000 7fa3de2e5640 1 -- 172.21.15.32:0/147871632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3d804e000 con 0x7fa3d815cdb0 2023-12-20T06:02:30.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.144+0000 7fa3d4ff9640 1 -- 172.21.15.32:0/147871632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa3d00b6040 con 0x7fa3d815cdb0 2023-12-20T06:02:30.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.144+0000 7fa3d4ff9640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3c4065b80 0x7fa3c4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.144+0000 7fa3d4ff9640 1 -- 172.21.15.32:0/147871632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa3d0121ba0 con 0x7fa3d815cdb0 2023-12-20T06:02:30.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.144+0000 7fa3d7fff640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3c4065b80 0x7fa3c4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.144+0000 7fa3d7fff640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3c4065b80 0x7fa3c4068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8000f70 tx=0x7fa3c8009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:30.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.148+0000 7fa3d4ff9640 1 -- 172.21.15.32:0/147871632 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa3d00ef410 con 0x7fa3d815cdb0 2023-12-20T06:02:30.338 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.340+0000 7fa3de2e5640 1 -- 172.21.15.32:0/147871632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fa3d815d7c0 con 0x7fa3d815cdb0 2023-12-20T06:02:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.340+0000 7fa3d4ff9640 1 -- 172.21.15.32:0/147871632 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4710 (secure 0 0 0) 0x7fa3d00f2a30 con 0x7fa3d815cdb0 2023-12-20T06:02:30.339 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:30.339 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/3001088147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3001088147},{"type":"v1","addr":"172.21.15.39:6833","nonce":3001088147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:26.942526+0000","modified":"2023-12-20T06:02:29.955675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10539":{"gid":10539,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/888710554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":888710554},{"type":"v1","addr":"172.21.15.32:6837","nonce":888710554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-12-20T06:02:30.339 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 96 2023-12-20T06:02:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 -- 172.21.15.32:0/147871632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3c4065b80 msgr2=0x7fa3c4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3c4065b80 0x7fa3c4068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fa3c8000f70 tx=0x7fa3c8009040 comp rx=0 tx=0).stop 2023-12-20T06:02:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 -- 172.21.15.32:0/147871632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d815cdb0 msgr2=0x7fa3d815da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d815cdb0 0x7fa3d815da60 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fa3d001bdd0 tx=0x7fa3d001bf60 comp rx=0 tx=0).stop 2023-12-20T06:02:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 -- 172.21.15.32:0/147871632 shutdown_connections 2023-12-20T06:02:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3c4065b80 0x7fa3c4068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3d815cdb0 0x7fa3d815da60 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3d80d2740 0x7fa3d81e7960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 --2- 172.21.15.32:0/147871632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3d80cfab0 0x7fa3d80c8be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 -- 172.21.15.32:0/147871632 >> 172.21.15.32:0/147871632 conn(0x7fa3d8065ee0 msgr2=0x7fa3d80cd380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 -- 172.21.15.32:0/147871632 shutdown_connections 2023-12-20T06:02:30.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.344+0000 7fa3de2e5640 1 -- 172.21.15.32:0/147871632 wait complete. 2023-12-20T06:02:30.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.504+0000 7fa49b02d640 1 Processor -- start 2023-12-20T06:02:30.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.504+0000 7fa49b02d640 1 -- start start 2023-12-20T06:02:30.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa49b02d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4940af5e0 0x7fa4940af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa49b02d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49415b990 con 0x7fa49406ce60 2023-12-20T06:02:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa49b02d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa49415bbd0 con 0x7fa49415f110 2023-12-20T06:02:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa49b02d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49415beb0 con 0x7fa4940af5e0 2023-12-20T06:02:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa493fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa49406ce60 0x7fa49415b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54772/0 (socket says 172.21.15.32:54772) 2023-12-20T06:02:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa493fff640 1 -- 172.21.15.32:0/1412500124 learned_addr learned my addr 172.21.15.32:0/1412500124 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa498dc9640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4940af5e0 0x7fa4940af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2860883464 0 0) 0x7fa480001120 con 0x7fa49406ce60 2023-12-20T06:02:30.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa484003530 con 0x7fa49406ce60 2023-12-20T06:02:30.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 149631694 0 0) 0x7fa49415bbd0 con 0x7fa49415f110 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa480001120 con 0x7fa49415f110 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1835401124 0 0) 0x7fa484003530 con 0x7fa49406ce60 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4940af5e0 msgr2=0x7fa4940af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4940af5e0 0x7fa4940af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 >> v1:172.21.15.39:6789/0 conn(0x7fa49415f110 legacy=0x7fa49415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49415e910 con 0x7fa49406ce60 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fa480001810 con 0x7fa49406ce60 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 >> v1:172.21.15.32:6789/0 conn(0x7fa49406ce60 legacy=0x7fa49415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa484003b40 0x7fa484003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4840047d0 0x7fa484004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 0x7fa48400fda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49415beb0 con 0x7fa484005190 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4840103c0 con 0x7fa484003b40 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa484010580 con 0x7fa4840047d0 2023-12-20T06:02:30.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4995ca640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 0x7fa48400fda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa493fff640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4840047d0 0x7fa484004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa498dc9640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa484003b40 0x7fa484003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4995ca640 1 -- 172.21.15.32:0/1412500124 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4840047d0 msgr2=0x7fa484004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4995ca640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4840047d0 0x7fa484004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4995ca640 1 -- 172.21.15.32:0/1412500124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa484003b40 msgr2=0x7fa484003f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4995ca640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa484003b40 0x7fa484003f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.508+0000 7fa4995ca640 1 -- 172.21.15.32:0/1412500124 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa484003530 con 0x7fa484005190 2023-12-20T06:02:30.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.512+0000 7fa4995ca640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 0x7fa48400fda0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fa48c001fd0 tx=0x7fa48c01c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f2cfe3e491c7f6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:30.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.512+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c003b80 con 0x7fa484005190 2023-12-20T06:02:30.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.512+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa48c003d00 con 0x7fa484005190 2023-12-20T06:02:30.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.512+0000 7fa4937fe640 1 -- 172.21.15.32:0/1412500124 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48c005310 con 0x7fa484005190 2023-12-20T06:02:30.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.512+0000 7fa49b02d640 1 -- 172.21.15.32:0/1412500124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 msgr2=0x7fa48400fda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.512+0000 7fa49b02d640 1 --2- 172.21.15.32:0/1412500124 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 0x7fa48400fda0 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7fa48c001fd0 tx=0x7fa48c01c8e0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.512+0000 7fa4995ca640 1 -- 172.21.15.32:0/1412500124 reap_dead start 2023-12-20T06:02:30.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.512+0000 7fa49b02d640 1 -- 172.21.15.32:0/1412500124 shutdown_connections 2023-12-20T06:02:30.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.512+0000 7fa49b02d640 1 -- 172.21.15.32:0/1412500124 >> 172.21.15.32:0/1412500124 conn(0x7fa49406a320 msgr2=0x7fa4940ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:30.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.512+0000 7fa49b02d640 1 -- 172.21.15.32:0/1412500124 shutdown_connections 2023-12-20T06:02:30.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.512+0000 7fa49b02d640 1 -- 172.21.15.32:0/1412500124 wait complete. 2023-12-20T06:02:30.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa49b02d640 1 Processor -- start 2023-12-20T06:02:30.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa49b02d640 1 -- start start 2023-12-20T06:02:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa49b02d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 0x7fa49414fbb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa49b02d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa49414f6b0 0x7fa49415c310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa49b02d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49414edd0 0x7fa4941527f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa49b02d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4941f1b90 con 0x7fa484005190 2023-12-20T06:02:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa49b02d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa49415a2f0 con 0x7fa49414edd0 2023-12-20T06:02:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa49b02d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa49415a5d0 con 0x7fa49414f6b0 2023-12-20T06:02:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa498dc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 0x7fa49414fbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa498dc9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 0x7fa49414fbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42828/0 (socket says 172.21.15.32:42828) 2023-12-20T06:02:30.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa498dc9640 1 -- 172.21.15.32:0/1111360963 learned_addr learned my addr 172.21.15.32:0/1111360963 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa493fff640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa49414f6b0 0x7fa49415c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa4995ca640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49414edd0 0x7fa4941527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa498dc9640 1 -- 172.21.15.32:0/1111360963 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa49414f6b0 msgr2=0x7fa49415c310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa498dc9640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa49414f6b0 0x7fa49415c310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa498dc9640 1 -- 172.21.15.32:0/1111360963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49414edd0 msgr2=0x7fa4941527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa498dc9640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49414edd0 0x7fa4941527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa498dc9640 1 -- 172.21.15.32:0/1111360963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa49415a8b0 con 0x7fa484005190 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa498dc9640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 0x7fa49414fbb0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fa488011890 tx=0x7fa488011d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa4917fa640 1 -- 172.21.15.32:0/1111360963 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa488004240 con 0x7fa484005190 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa4917fa640 1 -- 172.21.15.32:0/1111360963 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4880043a0 con 0x7fa484005190 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa4917fa640 1 -- 172.21.15.32:0/1111360963 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa48800fa40 con 0x7fa484005190 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa49b02d640 1 -- 172.21.15.32:0/1111360963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa49415ab90 con 0x7fa484005190 2023-12-20T06:02:30.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa49b02d640 1 -- 172.21.15.32:0/1111360963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa49415c850 con 0x7fa484005190 2023-12-20T06:02:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.516+0000 7fa49b02d640 1 -- 172.21.15.32:0/1111360963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa49415ad80 con 0x7fa484005190 2023-12-20T06:02:30.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.520+0000 7fa4917fa640 1 -- 172.21.15.32:0/1111360963 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa48800fba0 con 0x7fa484005190 2023-12-20T06:02:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.520+0000 7fa4917fa640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa478065bb0 0x7fa478068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.520+0000 7fa4917fa640 1 -- 172.21.15.32:0/1111360963 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa4880837f0 con 0x7fa484005190 2023-12-20T06:02:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.520+0000 7fa493fff640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa478065bb0 0x7fa478068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.520+0000 7fa493fff640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa478065bb0 0x7fa478068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa480001940 tx=0x7fa480001650 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:30.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.524+0000 7fa4917fa640 1 -- 172.21.15.32:0/1111360963 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa488051100 con 0x7fa484005190 2023-12-20T06:02:30.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 -- 172.21.15.32:0/1111360963 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa49404e000 con 0x7fa484005190 2023-12-20T06:02:30.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa4917fa640 1 -- 172.21.15.32:0/1111360963 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) v1 ==== 74+0+16153 (secure 0 0 0) 0x7fa488023090 con 0x7fa484005190 2023-12-20T06:02:30.682 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:30.682 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":209,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:27.945804+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:02:25.045126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-20T06:02:25.938205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 -- 172.21.15.32:0/1111360963 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa478065bb0 msgr2=0x7fa478068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa478065bb0 0x7fa478068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fa480001940 tx=0x7fa480001650 comp rx=0 tx=0).stop 2023-12-20T06:02:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 -- 172.21.15.32:0/1111360963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 msgr2=0x7fa49414fbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 0x7fa49414fbb0 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7fa488011890 tx=0x7fa488011d50 comp rx=0 tx=0).stop 2023-12-20T06:02:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 -- 172.21.15.32:0/1111360963 shutdown_connections 2023-12-20T06:02:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa478065bb0 0x7fa478068070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa49414edd0 0x7fa4941527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa49414f6b0 0x7fa49415c310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 --2- 172.21.15.32:0/1111360963 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa484005190 0x7fa49414fbb0 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 -- 172.21.15.32:0/1111360963 >> 172.21.15.32:0/1111360963 conn(0x7fa49406a320 msgr2=0x7fa4940af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:30.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 -- 172.21.15.32:0/1111360963 shutdown_connections 2023-12-20T06:02:30.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.684+0000 7fa49b02d640 1 -- 172.21.15.32:0/1111360963 wait complete. 2023-12-20T06:02:30.704 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T06:02:30.704 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:02:30.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.848+0000 7f2172c8c640 1 Processor -- start 2023-12-20T06:02:30.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.848+0000 7f2172c8c640 1 -- start start 2023-12-20T06:02:30.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2172c8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c1593c0 0x7f216c15d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2172c8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f216c06caf0 0x7f216c0705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2172c8c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f216c0ae140 con 0x7f216c0afdb0 2023-12-20T06:02:30.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2172c8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f216c0ae9e0 con 0x7f216c06caf0 2023-12-20T06:02:30.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2172c8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c15dee0 con 0x7f216c1593c0 2023-12-20T06:02:30.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2170a28640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c1593c0 0x7f216c15d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2171229640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f216c06caf0 0x7f216c0705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2170a28640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c1593c0 0x7f216c15d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:46872/0 (socket says 172.21.15.32:46872) 2023-12-20T06:02:30.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2170a28640 1 -- 172.21.15.32:0/1465148197 learned_addr learned my addr 172.21.15.32:0/1465148197 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:30.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2170a28640 1 -- 172.21.15.32:0/1465148197 >> v2:172.21.15.39:3300/0 conn(0x7f216c06caf0 msgr2=0x7f216c0705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:30.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2170a28640 1 --2- 172.21.15.32:0/1465148197 >> v2:172.21.15.39:3300/0 conn(0x7f216c06caf0 0x7f216c0705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2170a28640 1 -- 172.21.15.32:0/1465148197 >> v1:172.21.15.39:6789/0 conn(0x7f216c0afdb0 legacy=0x7f216c06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.852+0000 7f2170a28640 1 -- 172.21.15.32:0/1465148197 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f216c072170 con 0x7f216c1593c0 2023-12-20T06:02:30.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f2170a28640 1 --2- 172.21.15.32:0/1465148197 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c1593c0 0x7f216c15d9a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2160006fd0 tx=0x7f216001c460 comp rx=0 tx=0).ready entity=mon.2 client_cookie=eaf6c61532b5c438 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:30.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f216b7fe640 1 -- 172.21.15.32:0/1465148197 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21600ab600 con 0x7f216c1593c0 2023-12-20T06:02:30.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f216b7fe640 1 -- 172.21.15.32:0/1465148197 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21600a7050 con 0x7f216c1593c0 2023-12-20T06:02:30.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f216b7fe640 1 -- 172.21.15.32:0/1465148197 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21600ab980 con 0x7f216c1593c0 2023-12-20T06:02:30.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f2172c8c640 1 -- 172.21.15.32:0/1465148197 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c1593c0 msgr2=0x7f216c15d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f2172c8c640 1 --2- 172.21.15.32:0/1465148197 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c1593c0 0x7f216c15d9a0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f2160006fd0 tx=0x7f216001c460 comp rx=0 tx=0).stop 2023-12-20T06:02:30.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f2172c8c640 1 -- 172.21.15.32:0/1465148197 shutdown_connections 2023-12-20T06:02:30.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f2172c8c640 1 --2- 172.21.15.32:0/1465148197 >> v2:172.21.15.39:3300/0 conn(0x7f216c06caf0 0x7f216c0705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f2172c8c640 1 --2- 172.21.15.32:0/1465148197 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c1593c0 0x7f216c15d9a0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f2172c8c640 1 -- 172.21.15.32:0/1465148197 >> 172.21.15.32:0/1465148197 conn(0x7f216c06a320 msgr2=0x7f216c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f2172c8c640 1 -- 172.21.15.32:0/1465148197 shutdown_connections 2023-12-20T06:02:30.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f2172c8c640 1 -- 172.21.15.32:0/1465148197 wait complete. 2023-12-20T06:02:30.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f2172c8c640 1 Processor -- start 2023-12-20T06:02:30.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.856+0000 7f2172c8c640 1 -- start start 2023-12-20T06:02:30.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2172c8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f216c06caf0 0x7f216c15c4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2172c8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c0afdb0 0x7f216c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2172c8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c1593c0 0x7f216c0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2172c8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c167f20 con 0x7f216c1593c0 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2172c8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f216c148700 con 0x7f216c06caf0 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2172c8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f216c0cd9d0 con 0x7f216c0afdb0 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2171229640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c1593c0 0x7f216c0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2171229640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c1593c0 0x7f216c0ccba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42832/0 (socket says 172.21.15.32:42832) 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2171229640 1 -- 172.21.15.32:0/409828032 learned_addr learned my addr 172.21.15.32:0/409828032 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2170a28640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f216c06caf0 0x7f216c15c4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f216bfff640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c0afdb0 0x7f216c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2171229640 1 -- 172.21.15.32:0/409828032 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c0afdb0 msgr2=0x7f216c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2171229640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c0afdb0 0x7f216c0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2171229640 1 -- 172.21.15.32:0/409828032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f216c06caf0 msgr2=0x7f216c15c4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2171229640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f216c06caf0 0x7f216c15c4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2171229640 1 -- 172.21.15.32:0/409828032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2160009480 con 0x7f216c1593c0 2023-12-20T06:02:30.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f216bfff640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c0afdb0 0x7f216c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2171229640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c1593c0 0x7f216c0ccba0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f216400ef90 tx=0x7f216400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f21697fa640 1 -- 172.21.15.32:0/409828032 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2164009b20 con 0x7f216c1593c0 2023-12-20T06:02:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f21697fa640 1 -- 172.21.15.32:0/409828032 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f216400f040 con 0x7f216c1593c0 2023-12-20T06:02:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f21697fa640 1 -- 172.21.15.32:0/409828032 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21640146c0 con 0x7f216c1593c0 2023-12-20T06:02:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2172c8c640 1 -- 172.21.15.32:0/409828032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f216c0cdcb0 con 0x7f216c1593c0 2023-12-20T06:02:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2172c8c640 1 -- 172.21.15.32:0/409828032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f216c168340 con 0x7f216c1593c0 2023-12-20T06:02:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.860+0000 7f2172c8c640 1 -- 172.21.15.32:0/409828032 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f216c168060 con 0x7f216c1593c0 2023-12-20T06:02:30.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.864+0000 7f21697fa640 1 -- 172.21.15.32:0/409828032 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2164014820 con 0x7f216c1593c0 2023-12-20T06:02:30.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.864+0000 7f21697fa640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2150065b80 0x7f2150068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:30.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.864+0000 7f21697fa640 1 -- 172.21.15.32:0/409828032 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f2164084290 con 0x7f216c1593c0 2023-12-20T06:02:30.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.864+0000 7f2170a28640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2150065b80 0x7f2150068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:30.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.864+0000 7f2170a28640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2150065b80 0x7f2150068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f21600ad000 tx=0x7f21600a6000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:30.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:30.868+0000 7f21697fa640 1 -- 172.21.15.32:0/409828032 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f216c168060 con 0x7f216c1593c0 2023-12-20T06:02:31.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.024+0000 7f2172c8c640 1 -- 172.21.15.32:0/409828032 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f216c1597a0 con 0x7f2150065b80 2023-12-20T06:02:31.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.036+0000 7f21697fa640 1 -- 172.21.15.32:0/409828032 <== mgr.10314 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289598 (secure 0 0 0) 0x7f2160004340 con 0x7f2150065b80 2023-12-20T06:02:31.035 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:31.037 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:02:31.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.036+0000 7f2172c8c640 1 -- 172.21.15.32:0/409828032 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2150065b80 msgr2=0x7f2150068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.036+0000 7f2172c8c640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2150065b80 0x7f2150068040 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f21600ad000 tx=0x7f21600a6000 comp rx=0 tx=0).stop 2023-12-20T06:02:31.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.036+0000 7f2172c8c640 1 -- 172.21.15.32:0/409828032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c1593c0 msgr2=0x7f216c0ccba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.036+0000 7f2172c8c640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c1593c0 0x7f216c0ccba0 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f216400ef90 tx=0x7f216400c490 comp rx=0 tx=0).stop 2023-12-20T06:02:31.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.040+0000 7f2172c8c640 1 -- 172.21.15.32:0/409828032 shutdown_connections 2023-12-20T06:02:31.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.040+0000 7f2172c8c640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2150065b80 0x7f2150068040 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.040+0000 7f2172c8c640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f216c1593c0 0x7f216c0ccba0 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.040+0000 7f2172c8c640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f216c0afdb0 0x7f216c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.040+0000 7f2172c8c640 1 --2- 172.21.15.32:0/409828032 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f216c06caf0 0x7f216c15c4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.040+0000 7f2172c8c640 1 -- 172.21.15.32:0/409828032 >> 172.21.15.32:0/409828032 conn(0x7f216c06a320 msgr2=0x7f216c155690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:31.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.040+0000 7f2172c8c640 1 -- 172.21.15.32:0/409828032 shutdown_connections 2023-12-20T06:02:31.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.040+0000 7f2172c8c640 1 -- 172.21.15.32:0/409828032 wait complete. 2023-12-20T06:02:31.055 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-12-20T06:02:30.954649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":565,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3239,"ondisk_log_size":3239,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":400392,"kb_used_data":21576,"kb_used_omap":0,"kb_used_meta":378752,"kb_avail":754574328,"statfs":{"total":773094113280,"available":772684111872,"internally_reserved":0,"allocated":22093824,"data_stored":11290076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":387842048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.424669"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983748+0000","last_change":"2023-12-20T06:02:26.982741+0000","last_active":"2023-12-20T06:02:26.983748+0000","last_peered":"2023-12-20T06:02:26.983748+0000","last_clean":"2023-12-20T06:02:26.983748+0000","last_became_active":"2023-12-20T06:02:26.982454+0000","last_became_peered":"2023-12-20T06:02:26.982454+0000","last_unstale":"2023-12-20T06:02:26.983748+0000","last_undegraded":"2023-12-20T06:02:26.983748+0000","last_fullsized":"2023-12-20T06:02:26.983748+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:59:08.944850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.963944+0000","last_change":"2023-12-20T06:02:26.956043+0000","last_active":"2023-12-20T06:02:26.963944+0000","last_peered":"2023-12-20T06:02:26.963944+0000","last_clean":"2023-12-20T06:02:26.963944+0000","last_became_active":"2023-12-20T06:02:26.955652+0000","last_became_peered":"2023-12-20T06:02:26.955652+0000","last_unstale":"2023-12-20T06:02:26.963944+0000","last_undegraded":"2023-12-20T06:02:26.963944+0000","last_fullsized":"2023-12-20T06:02:26.963944+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:02:53.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.970951+0000","last_change":"2023-12-20T06:02:26.968444+0000","last_active":"2023-12-20T06:02:26.970951+0000","last_peered":"2023-12-20T06:02:26.970951+0000","last_clean":"2023-12-20T06:02:26.970951+0000","last_became_active":"2023-12-20T06:02:26.967931+0000","last_became_peered":"2023-12-20T06:02:26.967931+0000","last_unstale":"2023-12-20T06:02:26.970951+0000","last_undegraded":"2023-12-20T06:02:26.970951+0000","last_fullsized":"2023-12-20T06:02:26.970951+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:33:14.150434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.961739+0000","last_change":"2023-12-20T06:02:26.955353+0000","last_active":"2023-12-20T06:02:26.961739+0000","last_peered":"2023-12-20T06:02:26.961739+0000","last_clean":"2023-12-20T06:02:26.961739+0000","last_became_active":"2023-12-20T06:02:26.954910+0000","last_became_peered":"2023-12-20T06:02:26.954910+0000","last_unstale":"2023-12-20T06:02:26.961739+0000","last_undegraded":"2023-12-20T06:02:26.961739+0000","last_fullsized":"2023-12-20T06:02:26.961739+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:44:12.085696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.968041+0000","last_change":"2023-12-20T06:02:26.962506+0000","last_active":"2023-12-20T06:02:26.968041+0000","last_peered":"2023-12-20T06:02:26.968041+0000","last_clean":"2023-12-20T06:02:26.968041+0000","last_became_active":"2023-12-20T06:02:26.962262+0000","last_became_peered":"2023-12-20T06:02:26.962262+0000","last_unstale":"2023-12-20T06:02:26.968041+0000","last_undegraded":"2023-12-20T06:02:26.968041+0000","last_fullsized":"2023-12-20T06:02:26.968041+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:09:09.108347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981133+0000","last_change":"2023-12-20T06:02:26.979634+0000","last_active":"2023-12-20T06:02:26.981133+0000","last_peered":"2023-12-20T06:02:26.981133+0000","last_clean":"2023-12-20T06:02:26.981133+0000","last_became_active":"2023-12-20T06:02:26.979366+0000","last_became_peered":"2023-12-20T06:02:26.979366+0000","last_unstale":"2023-12-20T06:02:26.981133+0000","last_undegraded":"2023-12-20T06:02:26.981133+0000","last_fullsized":"2023-12-20T06:02:26.981133+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:34:19.170698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981603+0000","last_change":"2023-12-20T06:02:26.978795+0000","last_active":"2023-12-20T06:02:26.981603+0000","last_peered":"2023-12-20T06:02:26.981603+0000","last_clean":"2023-12-20T06:02:26.981603+0000","last_became_active":"2023-12-20T06:02:26.978497+0000","last_became_peered":"2023-12-20T06:02:26.978497+0000","last_unstale":"2023-12-20T06:02:26.981603+0000","last_undegraded":"2023-12-20T06:02:26.981603+0000","last_fullsized":"2023-12-20T06:02:26.981603+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:06:01.713370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983266+0000","last_change":"2023-12-20T06:02:26.971111+0000","last_active":"2023-12-20T06:02:26.983266+0000","last_peered":"2023-12-20T06:02:26.983266+0000","last_clean":"2023-12-20T06:02:26.983266+0000","last_became_active":"2023-12-20T06:02:26.970783+0000","last_became_peered":"2023-12-20T06:02:26.970783+0000","last_unstale":"2023-12-20T06:02:26.983266+0000","last_undegraded":"2023-12-20T06:02:26.983266+0000","last_fullsized":"2023-12-20T06:02:26.983266+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:45.692179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.963937+0000","last_change":"2023-12-20T06:02:26.960947+0000","last_active":"2023-12-20T06:02:26.963937+0000","last_peered":"2023-12-20T06:02:26.963937+0000","last_clean":"2023-12-20T06:02:26.963937+0000","last_became_active":"2023-12-20T06:02:26.958931+0000","last_became_peered":"2023-12-20T06:02:26.958931+0000","last_unstale":"2023-12-20T06:02:26.963937+0000","last_undegraded":"2023-12-20T06:02:26.963937+0000","last_fullsized":"2023-12-20T06:02:26.963937+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979774+0000","last_change":"2023-12-20T06:02:26.973655+0000","last_active":"2023-12-20T06:02:26.979774+0000","last_peered":"2023-12-20T06:02:26.979774+0000","last_clean":"2023-12-20T06:02:26.979774+0000","last_became_active":"2023-12-20T06:02:26.973281+0000","last_became_peered":"2023-12-20T06:02:26.973281+0000","last_unstale":"2023-12-20T06:02:26.979774+0000","last_undegraded":"2023-12-20T06:02:26.979774+0000","last_fullsized":"2023-12-20T06:02:26.979774+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:36.523098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979425+0000","last_change":"2023-12-20T06:02:26.976628+0000","last_active":"2023-12-20T06:02:26.979425+0000","last_peered":"2023-12-20T06:02:26.979425+0000","last_clean":"2023-12-20T06:02:26.979425+0000","last_became_active":"2023-12-20T06:02:26.976337+0000","last_became_peered":"2023-12-20T06:02:26.976337+0000","last_unstale":"2023-12-20T06:02:26.979425+0000","last_undegraded":"2023-12-20T06:02:26.979425+0000","last_fullsized":"2023-12-20T06:02:26.979425+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:15.147652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979019+0000","last_change":"2023-12-20T06:02:26.976708+0000","last_active":"2023-12-20T06:02:26.979019+0000","last_peered":"2023-12-20T06:02:26.979019+0000","last_clean":"2023-12-20T06:02:26.979019+0000","last_became_active":"2023-12-20T06:02:26.976349+0000","last_became_peered":"2023-12-20T06:02:26.976349+0000","last_unstale":"2023-12-20T06:02:26.979019+0000","last_undegraded":"2023-12-20T06:02:26.979019+0000","last_fullsized":"2023-12-20T06:02:26.979019+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.969536+0000","last_change":"2023-12-20T06:02:26.967177+0000","last_active":"2023-12-20T06:02:26.969536+0000","last_peered":"2023-12-20T06:02:26.969536+0000","last_clean":"2023-12-20T06:02:26.969536+0000","last_became_active":"2023-12-20T06:02:26.966560+0000","last_became_peered":"2023-12-20T06:02:26.966560+0000","last_unstale":"2023-12-20T06:02:26.969536+0000","last_undegraded":"2023-12-20T06:02:26.969536+0000","last_fullsized":"2023-12-20T06:02:26.969536+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:12:37.519537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982556+0000","last_change":"2023-12-20T06:02:26.978685+0000","last_active":"2023-12-20T06:02:26.982556+0000","last_peered":"2023-12-20T06:02:26.982556+0000","last_clean":"2023-12-20T06:02:26.982556+0000","last_became_active":"2023-12-20T06:02:26.978364+0000","last_became_peered":"2023-12-20T06:02:26.978364+0000","last_unstale":"2023-12-20T06:02:26.982556+0000","last_undegraded":"2023-12-20T06:02:26.982556+0000","last_fullsized":"2023-12-20T06:02:26.982556+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:19:37.257026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.975739+0000","last_change":"2023-12-20T06:02:26.974037+0000","last_active":"2023-12-20T06:02:26.975739+0000","last_peered":"2023-12-20T06:02:26.975739+0000","last_clean":"2023-12-20T06:02:26.975739+0000","last_became_active":"2023-12-20T06:02:26.973742+0000","last_became_peered":"2023-12-20T06:02:26.973742+0000","last_unstale":"2023-12-20T06:02:26.975739+0000","last_undegraded":"2023-12-20T06:02:26.975739+0000","last_fullsized":"2023-12-20T06:02:26.975739+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.975395+0000","last_change":"2023-12-20T06:02:26.966987+0000","last_active":"2023-12-20T06:02:26.975395+0000","last_peered":"2023-12-20T06:02:26.975395+0000","last_clean":"2023-12-20T06:02:26.975395+0000","last_became_active":"2023-12-20T06:02:26.966704+0000","last_became_peered":"2023-12-20T06:02:26.966704+0000","last_unstale":"2023-12-20T06:02:26.975395+0000","last_undegraded":"2023-12-20T06:02:26.975395+0000","last_fullsized":"2023-12-20T06:02:26.975395+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:41:37.729575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.961962+0000","last_change":"2023-12-20T06:02:26.959816+0000","last_active":"2023-12-20T06:02:26.961962+0000","last_peered":"2023-12-20T06:02:26.961962+0000","last_clean":"2023-12-20T06:02:26.961962+0000","last_became_active":"2023-12-20T06:02:26.959542+0000","last_became_peered":"2023-12-20T06:02:26.959542+0000","last_unstale":"2023-12-20T06:02:26.961962+0000","last_undegraded":"2023-12-20T06:02:26.961962+0000","last_fullsized":"2023-12-20T06:02:26.961962+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:49:15.081247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983632+0000","last_change":"2023-12-20T06:02:26.981004+0000","last_active":"2023-12-20T06:02:26.983632+0000","last_peered":"2023-12-20T06:02:26.983632+0000","last_clean":"2023-12-20T06:02:26.983632+0000","last_became_active":"2023-12-20T06:02:26.980597+0000","last_became_peered":"2023-12-20T06:02:26.980597+0000","last_unstale":"2023-12-20T06:02:26.983632+0000","last_undegraded":"2023-12-20T06:02:26.983632+0000","last_fullsized":"2023-12-20T06:02:26.983632+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:13.705554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.977962+0000","last_change":"2023-12-20T06:02:26.972124+0000","last_active":"2023-12-20T06:02:26.977962+0000","last_peered":"2023-12-20T06:02:26.977962+0000","last_clean":"2023-12-20T06:02:26.977962+0000","last_became_active":"2023-12-20T06:02:26.971870+0000","last_became_peered":"2023-12-20T06:02:26.971870+0000","last_unstale":"2023-12-20T06:02:26.977962+0000","last_undegraded":"2023-12-20T06:02:26.977962+0000","last_fullsized":"2023-12-20T06:02:26.977962+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:06:44.203571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.958870+0000","last_change":"2023-12-20T06:02:26.957746+0000","last_active":"2023-12-20T06:02:26.958870+0000","last_peered":"2023-12-20T06:02:26.958870+0000","last_clean":"2023-12-20T06:02:26.958870+0000","last_became_active":"2023-12-20T06:02:26.957407+0000","last_became_peered":"2023-12-20T06:02:26.957407+0000","last_unstale":"2023-12-20T06:02:26.958870+0000","last_undegraded":"2023-12-20T06:02:26.958870+0000","last_fullsized":"2023-12-20T06:02:26.958870+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:51.962994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.975680+0000","last_change":"2023-12-20T06:02:26.969431+0000","last_active":"2023-12-20T06:02:26.975680+0000","last_peered":"2023-12-20T06:02:26.975680+0000","last_clean":"2023-12-20T06:02:26.975680+0000","last_became_active":"2023-12-20T06:02:26.969161+0000","last_became_peered":"2023-12-20T06:02:26.969161+0000","last_unstale":"2023-12-20T06:02:26.975680+0000","last_undegraded":"2023-12-20T06:02:26.975680+0000","last_fullsized":"2023-12-20T06:02:26.975680+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:49:57.166700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979236+0000","last_change":"2023-12-20T06:02:26.975847+0000","last_active":"2023-12-20T06:02:26.979236+0000","last_peered":"2023-12-20T06:02:26.979236+0000","last_clean":"2023-12-20T06:02:26.979236+0000","last_became_active":"2023-12-20T06:02:26.975128+0000","last_became_peered":"2023-12-20T06:02:26.975128+0000","last_unstale":"2023-12-20T06:02:26.979236+0000","last_undegraded":"2023-12-20T06:02:26.979236+0000","last_fullsized":"2023-12-20T06:02:26.979236+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:39:21.684738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.980912+0000","last_change":"2023-12-20T06:02:26.979485+0000","last_active":"2023-12-20T06:02:26.980912+0000","last_peered":"2023-12-20T06:02:26.980912+0000","last_clean":"2023-12-20T06:02:26.980912+0000","last_became_active":"2023-12-20T06:02:26.979208+0000","last_became_peered":"2023-12-20T06:02:26.979208+0000","last_unstale":"2023-12-20T06:02:26.980912+0000","last_undegraded":"2023-12-20T06:02:26.980912+0000","last_fullsized":"2023-12-20T06:02:26.980912+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:04:09.949376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.975379+0000","last_change":"2023-12-20T06:02:26.965794+0000","last_active":"2023-12-20T06:02:26.975379+0000","last_peered":"2023-12-20T06:02:26.975379+0000","last_clean":"2023-12-20T06:02:26.975379+0000","last_became_active":"2023-12-20T06:02:26.965505+0000","last_became_peered":"2023-12-20T06:02:26.965505+0000","last_unstale":"2023-12-20T06:02:26.975379+0000","last_undegraded":"2023-12-20T06:02:26.975379+0000","last_fullsized":"2023-12-20T06:02:26.975379+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:09.034710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978921+0000","last_change":"2023-12-20T06:02:26.976267+0000","last_active":"2023-12-20T06:02:26.978921+0000","last_peered":"2023-12-20T06:02:26.978921+0000","last_clean":"2023-12-20T06:02:26.978921+0000","last_became_active":"2023-12-20T06:02:26.975927+0000","last_became_peered":"2023-12-20T06:02:26.975927+0000","last_unstale":"2023-12-20T06:02:26.978921+0000","last_undegraded":"2023-12-20T06:02:26.978921+0000","last_fullsized":"2023-12-20T06:02:26.978921+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:19:37.257026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981794+0000","last_change":"2023-12-20T06:02:26.979767+0000","last_active":"2023-12-20T06:02:26.981794+0000","last_peered":"2023-12-20T06:02:26.981794+0000","last_clean":"2023-12-20T06:02:26.981794+0000","last_became_active":"2023-12-20T06:02:26.979450+0000","last_became_peered":"2023-12-20T06:02:26.979450+0000","last_unstale":"2023-12-20T06:02:26.981794+0000","last_undegraded":"2023-12-20T06:02:26.981794+0000","last_fullsized":"2023-12-20T06:02:26.981794+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:47:57.463784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983120+0000","last_change":"2023-12-20T06:02:26.981676+0000","last_active":"2023-12-20T06:02:26.983120+0000","last_peered":"2023-12-20T06:02:26.983120+0000","last_clean":"2023-12-20T06:02:26.983120+0000","last_became_active":"2023-12-20T06:02:26.980905+0000","last_became_peered":"2023-12-20T06:02:26.980905+0000","last_unstale":"2023-12-20T06:02:26.983120+0000","last_undegraded":"2023-12-20T06:02:26.983120+0000","last_fullsized":"2023-12-20T06:02:26.983120+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:43:42.660130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978607+0000","last_change":"2023-12-20T06:02:26.977513+0000","last_active":"2023-12-20T06:02:26.978607+0000","last_peered":"2023-12-20T06:02:26.978607+0000","last_clean":"2023-12-20T06:02:26.978607+0000","last_became_active":"2023-12-20T06:02:26.977220+0000","last_became_peered":"2023-12-20T06:02:26.977220+0000","last_unstale":"2023-12-20T06:02:26.978607+0000","last_undegraded":"2023-12-20T06:02:26.978607+0000","last_fullsized":"2023-12-20T06:02:26.978607+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:52.600966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.975837+0000","last_change":"2023-12-20T06:02:26.973161+0000","last_active":"2023-12-20T06:02:26.975837+0000","last_peered":"2023-12-20T06:02:26.975837+0000","last_clean":"2023-12-20T06:02:26.975837+0000","last_became_active":"2023-12-20T06:02:26.972883+0000","last_became_peered":"2023-12-20T06:02:26.972883+0000","last_unstale":"2023-12-20T06:02:26.975837+0000","last_undegraded":"2023-12-20T06:02:26.975837+0000","last_fullsized":"2023-12-20T06:02:26.975837+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:11.127223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.980434+0000","last_change":"2023-12-20T06:02:26.977346+0000","last_active":"2023-12-20T06:02:26.980434+0000","last_peered":"2023-12-20T06:02:26.980434+0000","last_clean":"2023-12-20T06:02:26.980434+0000","last_became_active":"2023-12-20T06:02:26.976527+0000","last_became_peered":"2023-12-20T06:02:26.976527+0000","last_unstale":"2023-12-20T06:02:26.980434+0000","last_undegraded":"2023-12-20T06:02:26.980434+0000","last_fullsized":"2023-12-20T06:02:26.980434+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:11.127223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.970248+0000","last_change":"2023-12-20T06:02:26.965576+0000","last_active":"2023-12-20T06:02:26.970248+0000","last_peered":"2023-12-20T06:02:26.970248+0000","last_clean":"2023-12-20T06:02:26.970248+0000","last_became_active":"2023-12-20T06:02:26.965189+0000","last_became_peered":"2023-12-20T06:02:26.965189+0000","last_unstale":"2023-12-20T06:02:26.970248+0000","last_undegraded":"2023-12-20T06:02:26.970248+0000","last_fullsized":"2023-12-20T06:02:26.970248+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:38.051872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.971230+0000","last_change":"2023-12-20T06:02:26.963846+0000","last_active":"2023-12-20T06:02:26.971230+0000","last_peered":"2023-12-20T06:02:26.971230+0000","last_clean":"2023-12-20T06:02:26.971230+0000","last_became_active":"2023-12-20T06:02:26.963552+0000","last_became_peered":"2023-12-20T06:02:26.963552+0000","last_unstale":"2023-12-20T06:02:26.971230+0000","last_undegraded":"2023-12-20T06:02:26.971230+0000","last_fullsized":"2023-12-20T06:02:26.971230+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:17:08.792842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983412+0000","last_change":"2023-12-20T06:02:26.966538+0000","last_active":"2023-12-20T06:02:26.983412+0000","last_peered":"2023-12-20T06:02:26.983412+0000","last_clean":"2023-12-20T06:02:26.983412+0000","last_became_active":"2023-12-20T06:02:26.966278+0000","last_became_peered":"2023-12-20T06:02:26.966278+0000","last_unstale":"2023-12-20T06:02:26.983412+0000","last_undegraded":"2023-12-20T06:02:26.983412+0000","last_fullsized":"2023-12-20T06:02:26.983412+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:12.114643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"208'4","reported_seq":14,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.980496+0000","last_change":"2023-12-20T06:02:26.559340+0000","last_active":"2023-12-20T06:02:26.980496+0000","last_peered":"2023-12-20T06:02:26.980496+0000","last_clean":"2023-12-20T06:02:26.980496+0000","last_became_active":"2023-12-20T06:02:26.559045+0000","last_became_peered":"2023-12-20T06:02:26.559045+0000","last_unstale":"2023-12-20T06:02:26.980496+0000","last_undegraded":"2023-12-20T06:02:26.980496+0000","last_fullsized":"2023-12-20T06:02:26.980496+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:33.930072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982402+0000","last_change":"2023-12-20T06:02:26.978246+0000","last_active":"2023-12-20T06:02:26.982402+0000","last_peered":"2023-12-20T06:02:26.982402+0000","last_clean":"2023-12-20T06:02:26.982402+0000","last_became_active":"2023-12-20T06:02:26.977975+0000","last_became_peered":"2023-12-20T06:02:26.977975+0000","last_unstale":"2023-12-20T06:02:26.982402+0000","last_undegraded":"2023-12-20T06:02:26.982402+0000","last_fullsized":"2023-12-20T06:02:26.982402+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:05:43.763955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"208'1","reported_seq":11,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.962669+0000","last_change":"2023-12-20T06:02:26.547864+0000","last_active":"2023-12-20T06:02:26.962669+0000","last_peered":"2023-12-20T06:02:26.962669+0000","last_clean":"2023-12-20T06:02:26.962669+0000","last_became_active":"2023-12-20T06:02:26.547556+0000","last_became_peered":"2023-12-20T06:02:26.547556+0000","last_unstale":"2023-12-20T06:02:26.962669+0000","last_undegraded":"2023-12-20T06:02:26.962669+0000","last_fullsized":"2023-12-20T06:02:26.962669+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:04.689001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.970351+0000","last_change":"2023-12-20T06:02:26.967888+0000","last_active":"2023-12-20T06:02:26.970351+0000","last_peered":"2023-12-20T06:02:26.970351+0000","last_clean":"2023-12-20T06:02:26.970351+0000","last_became_active":"2023-12-20T06:02:26.967582+0000","last_became_peered":"2023-12-20T06:02:26.967582+0000","last_unstale":"2023-12-20T06:02:26.970351+0000","last_undegraded":"2023-12-20T06:02:26.970351+0000","last_fullsized":"2023-12-20T06:02:26.970351+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:26:03.948722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979018+0000","last_change":"2023-12-20T06:02:26.560653+0000","last_active":"2023-12-20T06:02:26.979018+0000","last_peered":"2023-12-20T06:02:26.979018+0000","last_clean":"2023-12-20T06:02:26.979018+0000","last_became_active":"2023-12-20T06:02:26.560323+0000","last_became_peered":"2023-12-20T06:02:26.560323+0000","last_unstale":"2023-12-20T06:02:26.979018+0000","last_undegraded":"2023-12-20T06:02:26.979018+0000","last_fullsized":"2023-12-20T06:02:26.979018+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:55.646229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.970993+0000","last_change":"2023-12-20T06:02:26.966379+0000","last_active":"2023-12-20T06:02:26.970993+0000","last_peered":"2023-12-20T06:02:26.970993+0000","last_clean":"2023-12-20T06:02:26.970993+0000","last_became_active":"2023-12-20T06:02:26.965974+0000","last_became_peered":"2023-12-20T06:02:26.965974+0000","last_unstale":"2023-12-20T06:02:26.970993+0000","last_undegraded":"2023-12-20T06:02:26.970993+0000","last_fullsized":"2023-12-20T06:02:26.970993+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:28.156831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.968846+0000","last_change":"2023-12-20T06:02:26.545330+0000","last_active":"2023-12-20T06:02:26.968846+0000","last_peered":"2023-12-20T06:02:26.968846+0000","last_clean":"2023-12-20T06:02:26.968846+0000","last_became_active":"2023-12-20T06:02:26.545018+0000","last_became_peered":"2023-12-20T06:02:26.545018+0000","last_unstale":"2023-12-20T06:02:26.968846+0000","last_undegraded":"2023-12-20T06:02:26.968846+0000","last_fullsized":"2023-12-20T06:02:26.968846+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:21:20.189262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979216+0000","last_change":"2023-12-20T06:02:26.974647+0000","last_active":"2023-12-20T06:02:26.979216+0000","last_peered":"2023-12-20T06:02:26.979216+0000","last_clean":"2023-12-20T06:02:26.979216+0000","last_became_active":"2023-12-20T06:02:26.974369+0000","last_became_peered":"2023-12-20T06:02:26.974369+0000","last_unstale":"2023-12-20T06:02:26.979216+0000","last_undegraded":"2023-12-20T06:02:26.979216+0000","last_fullsized":"2023-12-20T06:02:26.979216+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:36.494152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"208'1","reported_seq":11,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978836+0000","last_change":"2023-12-20T06:02:26.560921+0000","last_active":"2023-12-20T06:02:26.978836+0000","last_peered":"2023-12-20T06:02:26.978836+0000","last_clean":"2023-12-20T06:02:26.978836+0000","last_became_active":"2023-12-20T06:02:26.560567+0000","last_became_peered":"2023-12-20T06:02:26.560567+0000","last_unstale":"2023-12-20T06:02:26.978836+0000","last_undegraded":"2023-12-20T06:02:26.978836+0000","last_fullsized":"2023-12-20T06:02:26.978836+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:01:42.572698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.973792+0000","last_change":"2023-12-20T06:02:26.966409+0000","last_active":"2023-12-20T06:02:26.973792+0000","last_peered":"2023-12-20T06:02:26.973792+0000","last_clean":"2023-12-20T06:02:26.973792+0000","last_became_active":"2023-12-20T06:02:26.966153+0000","last_became_peered":"2023-12-20T06:02:26.966153+0000","last_unstale":"2023-12-20T06:02:26.973792+0000","last_undegraded":"2023-12-20T06:02:26.973792+0000","last_fullsized":"2023-12-20T06:02:26.973792+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:27.909325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"208'1","reported_seq":11,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982710+0000","last_change":"2023-12-20T06:02:26.556300+0000","last_active":"2023-12-20T06:02:26.982710+0000","last_peered":"2023-12-20T06:02:26.982710+0000","last_clean":"2023-12-20T06:02:26.982710+0000","last_became_active":"2023-12-20T06:02:26.556007+0000","last_became_peered":"2023-12-20T06:02:26.556007+0000","last_unstale":"2023-12-20T06:02:26.982710+0000","last_undegraded":"2023-12-20T06:02:26.982710+0000","last_fullsized":"2023-12-20T06:02:26.982710+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:58.838168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981583+0000","last_change":"2023-12-20T06:02:26.977844+0000","last_active":"2023-12-20T06:02:26.981583+0000","last_peered":"2023-12-20T06:02:26.981583+0000","last_clean":"2023-12-20T06:02:26.981583+0000","last_became_active":"2023-12-20T06:02:26.977331+0000","last_became_peered":"2023-12-20T06:02:26.977331+0000","last_unstale":"2023-12-20T06:02:26.981583+0000","last_undegraded":"2023-12-20T06:02:26.981583+0000","last_fullsized":"2023-12-20T06:02:26.981583+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:59:51.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.967171+0000","last_change":"2023-12-20T06:02:26.553243+0000","last_active":"2023-12-20T06:02:26.967171+0000","last_peered":"2023-12-20T06:02:26.967171+0000","last_clean":"2023-12-20T06:02:26.967171+0000","last_became_active":"2023-12-20T06:02:26.551953+0000","last_became_peered":"2023-12-20T06:02:26.551953+0000","last_unstale":"2023-12-20T06:02:26.967171+0000","last_undegraded":"2023-12-20T06:02:26.967171+0000","last_fullsized":"2023-12-20T06:02:26.967171+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:09:54.113512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.977120+0000","last_change":"2023-12-20T06:02:26.974738+0000","last_active":"2023-12-20T06:02:26.977120+0000","last_peered":"2023-12-20T06:02:26.977120+0000","last_clean":"2023-12-20T06:02:26.977120+0000","last_became_active":"2023-12-20T06:02:26.974494+0000","last_became_peered":"2023-12-20T06:02:26.974494+0000","last_unstale":"2023-12-20T06:02:26.977120+0000","last_undegraded":"2023-12-20T06:02:26.977120+0000","last_fullsized":"2023-12-20T06:02:26.977120+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.960421+0000","last_change":"2023-12-20T06:02:26.557303+0000","last_active":"2023-12-20T06:02:26.960421+0000","last_peered":"2023-12-20T06:02:26.960421+0000","last_clean":"2023-12-20T06:02:26.960421+0000","last_became_active":"2023-12-20T06:02:26.556838+0000","last_became_peered":"2023-12-20T06:02:26.556838+0000","last_unstale":"2023-12-20T06:02:26.960421+0000","last_undegraded":"2023-12-20T06:02:26.960421+0000","last_fullsized":"2023-12-20T06:02:26.960421+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:12:36.511646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.985674+0000","last_change":"2023-12-20T06:02:26.981970+0000","last_active":"2023-12-20T06:02:26.985674+0000","last_peered":"2023-12-20T06:02:26.985674+0000","last_clean":"2023-12-20T06:02:26.985674+0000","last_became_active":"2023-12-20T06:02:26.981688+0000","last_became_peered":"2023-12-20T06:02:26.981688+0000","last_unstale":"2023-12-20T06:02:26.985674+0000","last_undegraded":"2023-12-20T06:02:26.985674+0000","last_fullsized":"2023-12-20T06:02:26.985674+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:29:15.785654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.954445+0000","last_change":"2023-12-20T06:02:26.551507+0000","last_active":"2023-12-20T06:02:26.954445+0000","last_peered":"2023-12-20T06:02:26.954445+0000","last_clean":"2023-12-20T06:02:26.954445+0000","last_became_active":"2023-12-20T06:02:26.551215+0000","last_became_peered":"2023-12-20T06:02:26.551215+0000","last_unstale":"2023-12-20T06:02:26.954445+0000","last_undegraded":"2023-12-20T06:02:26.954445+0000","last_fullsized":"2023-12-20T06:02:26.954445+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:58:17.512600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.969999+0000","last_change":"2023-12-20T06:02:26.966176+0000","last_active":"2023-12-20T06:02:26.969999+0000","last_peered":"2023-12-20T06:02:26.969999+0000","last_clean":"2023-12-20T06:02:26.969999+0000","last_became_active":"2023-12-20T06:02:26.965898+0000","last_became_peered":"2023-12-20T06:02:26.965898+0000","last_unstale":"2023-12-20T06:02:26.969999+0000","last_undegraded":"2023-12-20T06:02:26.969999+0000","last_fullsized":"2023-12-20T06:02:26.969999+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:54:07.886847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"208'3","reported_seq":13,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.986365+0000","last_change":"2023-12-20T06:02:26.555063+0000","last_active":"2023-12-20T06:02:26.986365+0000","last_peered":"2023-12-20T06:02:26.986365+0000","last_clean":"2023-12-20T06:02:26.986365+0000","last_became_active":"2023-12-20T06:02:26.554776+0000","last_became_peered":"2023-12-20T06:02:26.554776+0000","last_unstale":"2023-12-20T06:02:26.986365+0000","last_undegraded":"2023-12-20T06:02:26.986365+0000","last_fullsized":"2023-12-20T06:02:26.986365+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:59:43.701253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981209+0000","last_change":"2023-12-20T06:02:26.978068+0000","last_active":"2023-12-20T06:02:26.981209+0000","last_peered":"2023-12-20T06:02:26.981209+0000","last_clean":"2023-12-20T06:02:26.981209+0000","last_became_active":"2023-12-20T06:02:26.977750+0000","last_became_peered":"2023-12-20T06:02:26.977750+0000","last_unstale":"2023-12-20T06:02:26.981209+0000","last_undegraded":"2023-12-20T06:02:26.981209+0000","last_fullsized":"2023-12-20T06:02:26.981209+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:05:59.363948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"208'6","reported_seq":16,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.986505+0000","last_change":"2023-12-20T06:02:26.558119+0000","last_active":"2023-12-20T06:02:26.986505+0000","last_peered":"2023-12-20T06:02:26.986505+0000","last_clean":"2023-12-20T06:02:26.986505+0000","last_became_active":"2023-12-20T06:02:26.557699+0000","last_became_peered":"2023-12-20T06:02:26.557699+0000","last_unstale":"2023-12-20T06:02:26.986505+0000","last_undegraded":"2023-12-20T06:02:26.986505+0000","last_fullsized":"2023-12-20T06:02:26.986505+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:21:20.189262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978860+0000","last_change":"2023-12-20T06:02:26.976279+0000","last_active":"2023-12-20T06:02:26.978860+0000","last_peered":"2023-12-20T06:02:26.978860+0000","last_clean":"2023-12-20T06:02:26.978860+0000","last_became_active":"2023-12-20T06:02:26.976019+0000","last_became_peered":"2023-12-20T06:02:26.976019+0000","last_unstale":"2023-12-20T06:02:26.978860+0000","last_undegraded":"2023-12-20T06:02:26.978860+0000","last_fullsized":"2023-12-20T06:02:26.978860+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983496+0000","last_change":"2023-12-20T06:02:26.552924+0000","last_active":"2023-12-20T06:02:26.983496+0000","last_peered":"2023-12-20T06:02:26.983496+0000","last_clean":"2023-12-20T06:02:26.983496+0000","last_became_active":"2023-12-20T06:02:26.552647+0000","last_became_peered":"2023-12-20T06:02:26.552647+0000","last_unstale":"2023-12-20T06:02:26.983496+0000","last_undegraded":"2023-12-20T06:02:26.983496+0000","last_fullsized":"2023-12-20T06:02:26.983496+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:44.700263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981642+0000","last_change":"2023-12-20T06:02:26.979070+0000","last_active":"2023-12-20T06:02:26.981642+0000","last_peered":"2023-12-20T06:02:26.981642+0000","last_clean":"2023-12-20T06:02:26.981642+0000","last_became_active":"2023-12-20T06:02:26.978777+0000","last_became_peered":"2023-12-20T06:02:26.978777+0000","last_unstale":"2023-12-20T06:02:26.981642+0000","last_undegraded":"2023-12-20T06:02:26.981642+0000","last_fullsized":"2023-12-20T06:02:26.981642+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:54.315390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.966048+0000","last_change":"2023-12-20T06:02:26.553685+0000","last_active":"2023-12-20T06:02:26.966048+0000","last_peered":"2023-12-20T06:02:26.966048+0000","last_clean":"2023-12-20T06:02:26.966048+0000","last_became_active":"2023-12-20T06:02:26.553411+0000","last_became_peered":"2023-12-20T06:02:26.553411+0000","last_unstale":"2023-12-20T06:02:26.966048+0000","last_undegraded":"2023-12-20T06:02:26.966048+0000","last_fullsized":"2023-12-20T06:02:26.966048+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:52.266518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982431+0000","last_change":"2023-12-20T06:02:26.978618+0000","last_active":"2023-12-20T06:02:26.982431+0000","last_peered":"2023-12-20T06:02:26.982431+0000","last_clean":"2023-12-20T06:02:26.982431+0000","last_became_active":"2023-12-20T06:02:26.978305+0000","last_became_peered":"2023-12-20T06:02:26.978305+0000","last_unstale":"2023-12-20T06:02:26.982431+0000","last_undegraded":"2023-12-20T06:02:26.982431+0000","last_fullsized":"2023-12-20T06:02:26.982431+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:02:53.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.947748+0000","last_change":"2023-12-20T06:02:26.558390+0000","last_active":"2023-12-20T06:02:26.947748+0000","last_peered":"2023-12-20T06:02:26.947748+0000","last_clean":"2023-12-20T06:02:26.947748+0000","last_became_active":"2023-12-20T06:02:26.558078+0000","last_became_peered":"2023-12-20T06:02:26.558078+0000","last_unstale":"2023-12-20T06:02:26.947748+0000","last_undegraded":"2023-12-20T06:02:26.947748+0000","last_fullsized":"2023-12-20T06:02:26.947748+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:47:56.455893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978437+0000","last_change":"2023-12-20T06:02:26.970325+0000","last_active":"2023-12-20T06:02:26.978437+0000","last_peered":"2023-12-20T06:02:26.978437+0000","last_clean":"2023-12-20T06:02:26.978437+0000","last_became_active":"2023-12-20T06:02:26.970054+0000","last_became_peered":"2023-12-20T06:02:26.970054+0000","last_unstale":"2023-12-20T06:02:26.978437+0000","last_undegraded":"2023-12-20T06:02:26.978437+0000","last_fullsized":"2023-12-20T06:02:26.978437+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:23:08.197185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"208'8","reported_seq":18,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.984485+0000","last_change":"2023-12-20T06:02:26.552002+0000","last_active":"2023-12-20T06:02:26.984485+0000","last_peered":"2023-12-20T06:02:26.984485+0000","last_clean":"2023-12-20T06:02:26.984485+0000","last_became_active":"2023-12-20T06:02:26.551664+0000","last_became_peered":"2023-12-20T06:02:26.551664+0000","last_unstale":"2023-12-20T06:02:26.984485+0000","last_undegraded":"2023-12-20T06:02:26.984485+0000","last_fullsized":"2023-12-20T06:02:26.984485+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:49:14.073356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979755+0000","last_change":"2023-12-20T06:02:26.973527+0000","last_active":"2023-12-20T06:02:26.979755+0000","last_peered":"2023-12-20T06:02:26.979755+0000","last_clean":"2023-12-20T06:02:26.979755+0000","last_became_active":"2023-12-20T06:02:26.972995+0000","last_became_peered":"2023-12-20T06:02:26.972995+0000","last_unstale":"2023-12-20T06:02:26.979755+0000","last_undegraded":"2023-12-20T06:02:26.979755+0000","last_fullsized":"2023-12-20T06:02:26.979755+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:15:22.108587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"208'1","reported_seq":11,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979050+0000","last_change":"2023-12-20T06:02:26.559146+0000","last_active":"2023-12-20T06:02:26.979050+0000","last_peered":"2023-12-20T06:02:26.979050+0000","last_clean":"2023-12-20T06:02:26.979050+0000","last_became_active":"2023-12-20T06:02:26.558865+0000","last_became_peered":"2023-12-20T06:02:26.558865+0000","last_unstale":"2023-12-20T06:02:26.979050+0000","last_undegraded":"2023-12-20T06:02:26.979050+0000","last_fullsized":"2023-12-20T06:02:26.979050+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:23:28.617951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982604+0000","last_change":"2023-12-20T06:02:26.979723+0000","last_active":"2023-12-20T06:02:26.982604+0000","last_peered":"2023-12-20T06:02:26.982604+0000","last_clean":"2023-12-20T06:02:26.982604+0000","last_became_active":"2023-12-20T06:02:26.979447+0000","last_became_peered":"2023-12-20T06:02:26.979447+0000","last_unstale":"2023-12-20T06:02:26.982604+0000","last_undegraded":"2023-12-20T06:02:26.982604+0000","last_fullsized":"2023-12-20T06:02:26.982604+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:44:12.085696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983482+0000","last_change":"2023-12-20T06:02:26.561295+0000","last_active":"2023-12-20T06:02:26.983482+0000","last_peered":"2023-12-20T06:02:26.983482+0000","last_clean":"2023-12-20T06:02:26.983482+0000","last_became_active":"2023-12-20T06:02:26.561016+0000","last_became_peered":"2023-12-20T06:02:26.561016+0000","last_unstale":"2023-12-20T06:02:26.983482+0000","last_undegraded":"2023-12-20T06:02:26.983482+0000","last_fullsized":"2023-12-20T06:02:26.983482+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:53.307500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.980468+0000","last_change":"2023-12-20T06:02:26.977809+0000","last_active":"2023-12-20T06:02:26.980468+0000","last_peered":"2023-12-20T06:02:26.980468+0000","last_clean":"2023-12-20T06:02:26.980468+0000","last_became_active":"2023-12-20T06:02:26.976835+0000","last_became_peered":"2023-12-20T06:02:26.976835+0000","last_unstale":"2023-12-20T06:02:26.980468+0000","last_undegraded":"2023-12-20T06:02:26.980468+0000","last_fullsized":"2023-12-20T06:02:26.980468+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"208'1","reported_seq":11,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983800+0000","last_change":"2023-12-20T06:02:26.553441+0000","last_active":"2023-12-20T06:02:26.983800+0000","last_peered":"2023-12-20T06:02:26.983800+0000","last_clean":"2023-12-20T06:02:26.983800+0000","last_became_active":"2023-12-20T06:02:26.551705+0000","last_became_peered":"2023-12-20T06:02:26.551705+0000","last_unstale":"2023-12-20T06:02:26.983800+0000","last_undegraded":"2023-12-20T06:02:26.983800+0000","last_fullsized":"2023-12-20T06:02:26.983800+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:51.593076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.965339+0000","last_change":"2023-12-20T06:02:26.958574+0000","last_active":"2023-12-20T06:02:26.965339+0000","last_peered":"2023-12-20T06:02:26.965339+0000","last_clean":"2023-12-20T06:02:26.965339+0000","last_became_active":"2023-12-20T06:02:26.958284+0000","last_became_peered":"2023-12-20T06:02:26.958284+0000","last_unstale":"2023-12-20T06:02:26.965339+0000","last_undegraded":"2023-12-20T06:02:26.965339+0000","last_fullsized":"2023-12-20T06:02:26.965339+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:28.156831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.960627+0000","last_change":"2023-12-20T06:02:26.559666+0000","last_active":"2023-12-20T06:02:26.960627+0000","last_peered":"2023-12-20T06:02:26.960627+0000","last_clean":"2023-12-20T06:02:26.960627+0000","last_became_active":"2023-12-20T06:02:26.559324+0000","last_became_peered":"2023-12-20T06:02:26.559324+0000","last_unstale":"2023-12-20T06:02:26.960627+0000","last_undegraded":"2023-12-20T06:02:26.960627+0000","last_fullsized":"2023-12-20T06:02:26.960627+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:27.148940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978454+0000","last_change":"2023-12-20T06:02:26.972273+0000","last_active":"2023-12-20T06:02:26.978454+0000","last_peered":"2023-12-20T06:02:26.978454+0000","last_clean":"2023-12-20T06:02:26.978454+0000","last_became_active":"2023-12-20T06:02:26.971983+0000","last_became_peered":"2023-12-20T06:02:26.971983+0000","last_unstale":"2023-12-20T06:02:26.978454+0000","last_undegraded":"2023-12-20T06:02:26.978454+0000","last_fullsized":"2023-12-20T06:02:26.978454+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:35:59.779548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.971821+0000","last_change":"2023-12-20T06:02:26.565662+0000","last_active":"2023-12-20T06:02:26.971821+0000","last_peered":"2023-12-20T06:02:26.971821+0000","last_clean":"2023-12-20T06:02:26.971821+0000","last_became_active":"2023-12-20T06:02:26.564737+0000","last_became_peered":"2023-12-20T06:02:26.564737+0000","last_unstale":"2023-12-20T06:02:26.971821+0000","last_undegraded":"2023-12-20T06:02:26.971821+0000","last_fullsized":"2023-12-20T06:02:26.971821+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:59:26.669013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983895+0000","last_change":"2023-12-20T06:02:26.981736+0000","last_active":"2023-12-20T06:02:26.983895+0000","last_peered":"2023-12-20T06:02:26.983895+0000","last_clean":"2023-12-20T06:02:26.983895+0000","last_became_active":"2023-12-20T06:02:26.981185+0000","last_became_peered":"2023-12-20T06:02:26.981185+0000","last_unstale":"2023-12-20T06:02:26.983895+0000","last_undegraded":"2023-12-20T06:02:26.983895+0000","last_fullsized":"2023-12-20T06:02:26.983895+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:59.846058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.953076+0000","last_change":"2023-12-20T06:02:26.554403+0000","last_active":"2023-12-20T06:02:26.953076+0000","last_peered":"2023-12-20T06:02:26.953076+0000","last_clean":"2023-12-20T06:02:26.953076+0000","last_became_active":"2023-12-20T06:02:26.554112+0000","last_became_peered":"2023-12-20T06:02:26.554112+0000","last_unstale":"2023-12-20T06:02:26.953076+0000","last_undegraded":"2023-12-20T06:02:26.953076+0000","last_fullsized":"2023-12-20T06:02:26.953076+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:53:48.616814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.970727+0000","last_change":"2023-12-20T06:02:26.968137+0000","last_active":"2023-12-20T06:02:26.970727+0000","last_peered":"2023-12-20T06:02:26.970727+0000","last_clean":"2023-12-20T06:02:26.970727+0000","last_became_active":"2023-12-20T06:02:26.967861+0000","last_became_peered":"2023-12-20T06:02:26.967861+0000","last_unstale":"2023-12-20T06:02:26.970727+0000","last_undegraded":"2023-12-20T06:02:26.970727+0000","last_fullsized":"2023-12-20T06:02:26.970727+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:34:19.170698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.956777+0000","last_change":"2023-12-20T06:02:26.560641+0000","last_active":"2023-12-20T06:02:26.956777+0000","last_peered":"2023-12-20T06:02:26.956777+0000","last_clean":"2023-12-20T06:02:26.956777+0000","last_became_active":"2023-12-20T06:02:26.560352+0000","last_became_peered":"2023-12-20T06:02:26.560352+0000","last_unstale":"2023-12-20T06:02:26.956777+0000","last_undegraded":"2023-12-20T06:02:26.956777+0000","last_fullsized":"2023-12-20T06:02:26.956777+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:49.756970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.964489+0000","last_change":"2023-12-20T06:02:26.961186+0000","last_active":"2023-12-20T06:02:26.964489+0000","last_peered":"2023-12-20T06:02:26.964489+0000","last_clean":"2023-12-20T06:02:26.964489+0000","last_became_active":"2023-12-20T06:02:26.960904+0000","last_became_peered":"2023-12-20T06:02:26.960904+0000","last_unstale":"2023-12-20T06:02:26.964489+0000","last_undegraded":"2023-12-20T06:02:26.964489+0000","last_fullsized":"2023-12-20T06:02:26.964489+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:45.708153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.954337+0000","last_change":"2023-12-20T06:02:26.547934+0000","last_active":"2023-12-20T06:02:26.954337+0000","last_peered":"2023-12-20T06:02:26.954337+0000","last_clean":"2023-12-20T06:02:26.954337+0000","last_became_active":"2023-12-20T06:02:26.547651+0000","last_became_peered":"2023-12-20T06:02:26.547651+0000","last_unstale":"2023-12-20T06:02:26.954337+0000","last_undegraded":"2023-12-20T06:02:26.954337+0000","last_fullsized":"2023-12-20T06:02:26.954337+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:42:23.285931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.971578+0000","last_change":"2023-12-20T06:02:26.966239+0000","last_active":"2023-12-20T06:02:26.971578+0000","last_peered":"2023-12-20T06:02:26.971578+0000","last_clean":"2023-12-20T06:02:26.971578+0000","last_became_active":"2023-12-20T06:02:26.964925+0000","last_became_peered":"2023-12-20T06:02:26.964925+0000","last_unstale":"2023-12-20T06:02:26.971578+0000","last_undegraded":"2023-12-20T06:02:26.971578+0000","last_fullsized":"2023-12-20T06:02:26.971578+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:09.937547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.980446+0000","last_change":"2023-12-20T06:02:26.549840+0000","last_active":"2023-12-20T06:02:26.980446+0000","last_peered":"2023-12-20T06:02:26.980446+0000","last_clean":"2023-12-20T06:02:26.980446+0000","last_became_active":"2023-12-20T06:02:26.549537+0000","last_became_peered":"2023-12-20T06:02:26.549537+0000","last_unstale":"2023-12-20T06:02:26.980446+0000","last_undegraded":"2023-12-20T06:02:26.980446+0000","last_fullsized":"2023-12-20T06:02:26.980446+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:50.955103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982980+0000","last_change":"2023-12-20T06:02:26.981756+0000","last_active":"2023-12-20T06:02:26.982980+0000","last_peered":"2023-12-20T06:02:26.982980+0000","last_clean":"2023-12-20T06:02:26.982980+0000","last_became_active":"2023-12-20T06:02:26.980456+0000","last_became_peered":"2023-12-20T06:02:26.980456+0000","last_unstale":"2023-12-20T06:02:26.982980+0000","last_undegraded":"2023-12-20T06:02:26.982980+0000","last_fullsized":"2023-12-20T06:02:26.982980+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:33:51.384201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.968770+0000","last_change":"2023-12-20T06:02:26.555848+0000","last_active":"2023-12-20T06:02:26.968770+0000","last_peered":"2023-12-20T06:02:26.968770+0000","last_clean":"2023-12-20T06:02:26.968770+0000","last_became_active":"2023-12-20T06:02:26.555560+0000","last_became_peered":"2023-12-20T06:02:26.555560+0000","last_unstale":"2023-12-20T06:02:26.968770+0000","last_undegraded":"2023-12-20T06:02:26.968770+0000","last_fullsized":"2023-12-20T06:02:26.968770+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:44.684288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983950+0000","last_change":"2023-12-20T06:02:26.969245+0000","last_active":"2023-12-20T06:02:26.983950+0000","last_peered":"2023-12-20T06:02:26.983950+0000","last_clean":"2023-12-20T06:02:26.983950+0000","last_became_active":"2023-12-20T06:02:26.969011+0000","last_became_peered":"2023-12-20T06:02:26.969011+0000","last_unstale":"2023-12-20T06:02:26.983950+0000","last_undegraded":"2023-12-20T06:02:26.983950+0000","last_fullsized":"2023-12-20T06:02:26.983950+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:53.274408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.966257+0000","last_change":"2023-12-20T06:02:26.556970+0000","last_active":"2023-12-20T06:02:26.966257+0000","last_peered":"2023-12-20T06:02:26.966257+0000","last_clean":"2023-12-20T06:02:26.966257+0000","last_became_active":"2023-12-20T06:02:26.556577+0000","last_became_peered":"2023-12-20T06:02:26.556577+0000","last_unstale":"2023-12-20T06:02:26.966257+0000","last_undegraded":"2023-12-20T06:02:26.966257+0000","last_fullsized":"2023-12-20T06:02:26.966257+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:45.673434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.977301+0000","last_change":"2023-12-20T06:02:26.967116+0000","last_active":"2023-12-20T06:02:26.977301+0000","last_peered":"2023-12-20T06:02:26.977301+0000","last_clean":"2023-12-20T06:02:26.977301+0000","last_became_active":"2023-12-20T06:02:26.966832+0000","last_became_peered":"2023-12-20T06:02:26.966832+0000","last_unstale":"2023-12-20T06:02:26.977301+0000","last_undegraded":"2023-12-20T06:02:26.977301+0000","last_fullsized":"2023-12-20T06:02:26.977301+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:52:23.105026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.948070+0000","last_change":"2023-12-20T06:02:26.560397+0000","last_active":"2023-12-20T06:02:26.948070+0000","last_peered":"2023-12-20T06:02:26.948070+0000","last_clean":"2023-12-20T06:02:26.948070+0000","last_became_active":"2023-12-20T06:02:26.560128+0000","last_became_peered":"2023-12-20T06:02:26.560128+0000","last_unstale":"2023-12-20T06:02:26.948070+0000","last_undegraded":"2023-12-20T06:02:26.948070+0000","last_fullsized":"2023-12-20T06:02:26.948070+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:54:06.878956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.984298+0000","last_change":"2023-12-20T06:02:26.983027+0000","last_active":"2023-12-20T06:02:26.984298+0000","last_peered":"2023-12-20T06:02:26.984298+0000","last_clean":"2023-12-20T06:02:26.984298+0000","last_became_active":"2023-12-20T06:02:26.982556+0000","last_became_peered":"2023-12-20T06:02:26.982556+0000","last_unstale":"2023-12-20T06:02:26.984298+0000","last_undegraded":"2023-12-20T06:02:26.984298+0000","last_fullsized":"2023-12-20T06:02:26.984298+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:42:24.293821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.955310+0000","last_change":"2023-12-20T06:02:26.556385+0000","last_active":"2023-12-20T06:02:26.955310+0000","last_peered":"2023-12-20T06:02:26.955310+0000","last_clean":"2023-12-20T06:02:26.955310+0000","last_became_active":"2023-12-20T06:02:26.555308+0000","last_became_peered":"2023-12-20T06:02:26.555308+0000","last_unstale":"2023-12-20T06:02:26.955310+0000","last_undegraded":"2023-12-20T06:02:26.955310+0000","last_fullsized":"2023-12-20T06:02:26.955310+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:31.032511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983559+0000","last_change":"2023-12-20T06:02:26.980532+0000","last_active":"2023-12-20T06:02:26.983559+0000","last_peered":"2023-12-20T06:02:26.983559+0000","last_clean":"2023-12-20T06:02:26.983559+0000","last_became_active":"2023-12-20T06:02:26.980234+0000","last_became_peered":"2023-12-20T06:02:26.980234+0000","last_unstale":"2023-12-20T06:02:26.983559+0000","last_undegraded":"2023-12-20T06:02:26.983559+0000","last_fullsized":"2023-12-20T06:02:26.983559+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:32.040401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.954575+0000","last_change":"2023-12-20T06:02:26.556064+0000","last_active":"2023-12-20T06:02:26.954575+0000","last_peered":"2023-12-20T06:02:26.954575+0000","last_clean":"2023-12-20T06:02:26.954575+0000","last_became_active":"2023-12-20T06:02:26.554935+0000","last_became_peered":"2023-12-20T06:02:26.554935+0000","last_unstale":"2023-12-20T06:02:26.954575+0000","last_undegraded":"2023-12-20T06:02:26.954575+0000","last_fullsized":"2023-12-20T06:02:26.954575+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:12.697664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.965443+0000","last_change":"2023-12-20T06:02:26.962551+0000","last_active":"2023-12-20T06:02:26.965443+0000","last_peered":"2023-12-20T06:02:26.965443+0000","last_clean":"2023-12-20T06:02:26.965443+0000","last_became_active":"2023-12-20T06:02:26.962269+0000","last_became_peered":"2023-12-20T06:02:26.962269+0000","last_unstale":"2023-12-20T06:02:26.965443+0000","last_undegraded":"2023-12-20T06:02:26.965443+0000","last_fullsized":"2023-12-20T06:02:26.965443+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:43:42.660130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.949016+0000","last_change":"2023-12-20T06:02:26.548501+0000","last_active":"2023-12-20T06:02:26.949016+0000","last_peered":"2023-12-20T06:02:26.949016+0000","last_clean":"2023-12-20T06:02:26.949016+0000","last_became_active":"2023-12-20T06:02:26.548203+0000","last_became_peered":"2023-12-20T06:02:26.548203+0000","last_unstale":"2023-12-20T06:02:26.949016+0000","last_undegraded":"2023-12-20T06:02:26.949016+0000","last_fullsized":"2023-12-20T06:02:26.949016+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:55.646229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.964966+0000","last_change":"2023-12-20T06:02:26.961645+0000","last_active":"2023-12-20T06:02:26.964966+0000","last_peered":"2023-12-20T06:02:26.964966+0000","last_clean":"2023-12-20T06:02:26.964966+0000","last_became_active":"2023-12-20T06:02:26.961386+0000","last_became_peered":"2023-12-20T06:02:26.961386+0000","last_unstale":"2023-12-20T06:02:26.964966+0000","last_undegraded":"2023-12-20T06:02:26.964966+0000","last_fullsized":"2023-12-20T06:02:26.964966+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:52:17.428938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.956509+0000","last_change":"2023-12-20T06:02:26.560054+0000","last_active":"2023-12-20T06:02:26.956509+0000","last_peered":"2023-12-20T06:02:26.956509+0000","last_clean":"2023-12-20T06:02:26.956509+0000","last_became_active":"2023-12-20T06:02:26.559777+0000","last_became_peered":"2023-12-20T06:02:26.559777+0000","last_unstale":"2023-12-20T06:02:26.956509+0000","last_undegraded":"2023-12-20T06:02:26.956509+0000","last_fullsized":"2023-12-20T06:02:26.956509+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:40:31.615384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.977670+0000","last_change":"2023-12-20T06:02:26.971114+0000","last_active":"2023-12-20T06:02:26.977670+0000","last_peered":"2023-12-20T06:02:26.977670+0000","last_clean":"2023-12-20T06:02:26.977670+0000","last_became_active":"2023-12-20T06:02:26.970836+0000","last_became_peered":"2023-12-20T06:02:26.970836+0000","last_unstale":"2023-12-20T06:02:26.977670+0000","last_undegraded":"2023-12-20T06:02:26.977670+0000","last_fullsized":"2023-12-20T06:02:26.977670+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:50:58.624420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.969717+0000","last_change":"2023-12-20T06:02:26.550026+0000","last_active":"2023-12-20T06:02:26.969717+0000","last_peered":"2023-12-20T06:02:26.969717+0000","last_clean":"2023-12-20T06:02:26.969717+0000","last_became_active":"2023-12-20T06:02:26.549712+0000","last_became_peered":"2023-12-20T06:02:26.549712+0000","last_unstale":"2023-12-20T06:02:26.969717+0000","last_undegraded":"2023-12-20T06:02:26.969717+0000","last_fullsized":"2023-12-20T06:02:26.969717+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:08.929656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"196'3201","reported_seq":3991,"reported_epoch":207,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.536771+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:02:26.536771+0000","last_peered":"2023-12-20T06:02:26.536771+0000","last_clean":"2023-12-20T06:02:26.536771+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:02:26.536771+0000","last_undegraded":"2023-12-20T06:02:26.536771+0000","last_fullsized":"2023-12-20T06:02:26.536771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3201,"log_dups_size":0,"ondisk_log_size":3201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":521,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":521,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3201,"ondisk_log_size":3201,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56324,"kb_used_data":3132,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315516,"statfs":{"total":96636764160,"available":96579088384,"internally_reserved":0,"allocated":3207168,"data_stored":1853549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.73499999999999999},"min":{"1min":0.497,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.042},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72599999999999998,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.34499999999999997},"max":{"1min":0.874,"5min":0.99099999999999999,"15min":1.161},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73199999999999998,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.432},"max":{"1min":0.88700000000000001,"5min":0.996,"15min":1.1279999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.53100000000000003,"5min":0.51100000000000001,"15min":0.46999999999999997},"max":{"1min":0.90900000000000003,"5min":0.96699999999999997,"15min":1.032},"last":0.49399999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72299999999999998,"15min":0.748},"min":{"1min":0.51900000000000002,"5min":0.48699999999999999,"15min":0.48299999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0409999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96999999999999997,"5min":1.0189999999999999,"15min":1.038},"last":0.56999999999999995}]},{"osd":3,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72899999999999998,"15min":0.746},"min":{"1min":0.61899999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.5},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77900000000000003,"15min":0.79000000000000004},"min":{"1min":0.53800000000000003,"5min":0.503,"15min":0.40699999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0840000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77600000000000002,"5min":0.91800000000000004,"15min":1.04},"last":0.379},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.91800000000000004,"5min":1.02,"15min":1.02},"last":0.41799999999999998}]},{"osd":5,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86699999999999999,"5min":0.97499999999999998,"15min":1.1120000000000001},"last":0.62},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.54300000000000004,"5min":0.47199999999999998,"15min":0.45600000000000002},"max":{"1min":0.84199999999999997,"5min":1.0229999999999999,"15min":1.103},"last":0.45300000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76500000000000001,"15min":0.751},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.45500000000000002},"max":{"1min":0.94399999999999995,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92300000000000004,"5min":1.0740000000000001,"15min":1.1539999999999999},"last":0.52100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575970,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51640,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320200,"statfs":{"total":96636764160,"available":96583884800,"internally_reserved":0,"allocated":2605056,"data_stored":1263077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.74399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.0980000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73499999999999999,"15min":0.754},"min":{"1min":0.51300000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.89800000000000002,"5min":1.016,"15min":1.05},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72699999999999998,"15min":0.73799999999999999},"min":{"1min":0.45100000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94999999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.88200000000000001}]},{"osd":2,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96199999999999997,"5min":1.069,"15min":1.069},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.745,"15min":0.76700000000000002},"min":{"1min":0.54900000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.96799999999999997,"5min":0.97799999999999998,"15min":1.0760000000000001},"last":0.84799999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78600000000000003,"15min":0.78900000000000003},"min":{"1min":0.627,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":0.95799999999999996,"5min":1.077,"15min":1.077},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.57299999999999995,"5min":0.51200000000000001,"15min":0.503},"max":{"1min":0.93500000000000005,"5min":1.052,"15min":1.052},"last":0.92500000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.63100000000000001},"min":{"1min":0.49199999999999999,"5min":0.40200000000000002,"15min":0.36099999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.0529999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.46000000000000002,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.80200000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.91700000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.53400000000000003,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":1.0009999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71299999999999997,"15min":0.70399999999999996},"min":{"1min":0.54600000000000004,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.004}]},{"osd":7,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73499999999999999,"15min":0.73199999999999998},"min":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.45300000000000001},"max":{"1min":0.997,"5min":1.103,"15min":1.1240000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71999999999999997,"15min":0.72799999999999998},"min":{"1min":0.52300000000000002,"5min":0.49399999999999999,"15min":0.41999999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.0700000000000001},"last":1.0409999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29496,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342344,"statfs":{"total":96636764160,"available":96606560256,"internally_reserved":0,"allocated":2605056,"data_stored":1263077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.44,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94499999999999995,"5min":0.97499999999999998,"15min":1.077},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73199999999999998,"15min":0.755},"min":{"1min":0.57699999999999996,"5min":0.432,"15min":0.432},"max":{"1min":0.91400000000000003,"5min":0.96699999999999997,"15min":1.028},"last":0.45600000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.74299999999999999,"15min":0.754},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.442},"max":{"1min":0.89000000000000001,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.46899999999999997},"max":{"1min":0.95699999999999996,"5min":1.016,"15min":1.0640000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73199999999999998,"15min":0.75900000000000001},"min":{"1min":0.60299999999999998,"5min":0.46999999999999997,"15min":0.46899999999999997},"max":{"1min":0.93400000000000005,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76500000000000001,"15min":0.77200000000000002},"min":{"1min":0.63100000000000001,"5min":0.55800000000000005,"15min":0.48299999999999998},"max":{"1min":0.93500000000000005,"5min":1.0349999999999999,"15min":1.0680000000000001},"last":0.72799999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.0669999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.65700000000000003,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.91700000000000004,"5min":1.004,"15min":1.0629999999999999},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47199999999999998,"5min":0.42199999999999999,"15min":0.38100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0640000000000001,"15min":1.077},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67000000000000004,"15min":0.64200000000000002},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41299999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69399999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.70299999999999996},"min":{"1min":0.56299999999999994,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.95199999999999996,"5min":1.012,"15min":1.073},"last":0.66800000000000004}]},{"osd":7,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.56799999999999995,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.63300000000000001,"5min":0.56299999999999994,"15min":0.46700000000000003},"max":{"1min":0.97599999999999998,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575972,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47560,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324280,"statfs":{"total":96636764160,"available":96588062720,"internally_reserved":0,"allocated":2621440,"data_stored":1265223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71899999999999997,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.38800000000000001},"max":{"1min":0.84399999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.56899999999999995,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":0.95699999999999996,"5min":0.96499999999999997,"15min":1.0900000000000001},"last":0.751},{"interface":"front","average":{"1min":0.752,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.52200000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91400000000000003,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.63}]},{"osd":2,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76200000000000001,"15min":0.77600000000000002},"min":{"1min":0.58999999999999997,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.78100000000000003},"min":{"1min":0.56999999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98599999999999999,"5min":1.0089999999999999,"15min":1.101},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.59899999999999998,"5min":0.51800000000000002,"15min":0.49199999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.64500000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.95999999999999996,"5min":1.04,"15min":1.081},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.39900000000000002},"max":{"1min":0.93899999999999995,"5min":1.05,"15min":1.05},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.86099999999999999,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94699999999999995,"5min":1.016,"15min":1.1679999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.41199999999999998},"max":{"1min":0.97999999999999998,"5min":1.107,"15min":1.111},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":1.01,"5min":1.0600000000000001,"15min":1.1160000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.52900000000000003,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.034,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.78100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575971,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47556,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324284,"statfs":{"total":96636764160,"available":96588066816,"internally_reserved":0,"allocated":2617344,"data_stored":1263271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.59299999999999997,"5min":0.748,"15min":2.8130000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.51100000000000001,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64600000000000002,"5min":0.80000000000000004,"15min":1.357},"last":0.39100000000000001}]},{"osd":1,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":1.4219999999999999},"last":0.45000000000000001}]},{"osd":2,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.34200000000000003},"max":{"1min":0.72599999999999998,"5min":0.86099999999999999,"15min":2.944},"last":0.5}]},{"osd":4,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":1.0209999999999999,"5min":1.2090000000000001,"15min":1.669},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.99},"last":0.91600000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.59599999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97199999999999998,"5min":1.355,"15min":1.901},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.82399999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97699999999999998,"5min":1.196,"15min":2.8639999999999999},"last":0.93300000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.81999999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.575},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.81799999999999995,"15min":0.85099999999999998},"min":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0149999999999999,"5min":1.2350000000000001,"15min":1.948},"last":0.86499999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80400000000000005,"15min":0.84099999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.004,"5min":1.3149999999999999,"15min":2.9049999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82399999999999995,"15min":0.85299999999999998},"min":{"1min":0.61899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.089,"5min":1.28,"15min":1.754},"last":0.97099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51652,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320188,"statfs":{"total":96636764160,"available":96583872512,"internally_reserved":0,"allocated":2617344,"data_stored":1263771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.26100000000000001},"max":{"1min":0.75800000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.376,"5min":0.34000000000000002,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.30499999999999999},"max":{"1min":0.71299999999999997,"5min":0.92600000000000005,"15min":1.004},"last":0.36799999999999999}]},{"osd":3,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.65800000000000003,"15min":0.64900000000000002},"min":{"1min":0.47499999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.089,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.46500000000000002,"15min":0.42499999999999999},"max":{"1min":1.069,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79200000000000004,"15min":0.80300000000000005},"min":{"1min":0.58699999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.081,"5min":1.119,"15min":1.254},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.59199999999999997,"5min":0.505,"15min":0.48199999999999998},"max":{"1min":1.02,"5min":1.605,"15min":1.605},"last":0.40000000000000002}]},{"osd":5,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81200000000000006,"15min":0.80700000000000005},"min":{"1min":0.62,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1020000000000001,"5min":1.329,"15min":1.329},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.83899999999999997,"15min":0.83399999999999996},"min":{"1min":0.68500000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0309999999999999,"5min":1.0740000000000001,"15min":1.157},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.60299999999999998,"5min":0.55100000000000005,"15min":0.52300000000000002},"max":{"1min":1.157,"5min":1.919,"15min":1.919},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63300000000000001,"5min":0.56999999999999995,"15min":0.56200000000000006},"max":{"1min":1.0449999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.621}]},{"osd":7,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.875,"15min":0.85999999999999999},"min":{"1min":0.67400000000000004,"5min":0.60699999999999998,"15min":0.47899999999999998},"max":{"1min":1.3360000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.89000000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.262,"5min":1.292,"15min":1.292},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575972,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2617344,"data_stored":1264641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48799999999999999,"15min":0.46200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.69599999999999995,"5min":0.746,"15min":0.81000000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.313,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.77300000000000002,"5min":0.78200000000000003,"15min":0.81100000000000005},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72399999999999998,"5min":0.88800000000000001,"15min":0.93700000000000006},"last":0.38100000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77800000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.73899999999999999,"5min":0.88700000000000001,"15min":0.93500000000000005},"last":0.48899999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.53200000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.321},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.95999999999999996,"5min":1.139,"15min":1.25},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80300000000000005,"15min":0.80900000000000005},"min":{"1min":0.59499999999999997,"5min":0.51400000000000001,"15min":0.443},"max":{"1min":1.131,"5min":1.224,"15min":1.607},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80400000000000005,"15min":0.81200000000000006},"min":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0569999999999999,"5min":1.25,"15min":1.4330000000000001},"last":0.63400000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.81699999999999995,"15min":0.82999999999999996},"min":{"1min":0.57899999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0029999999999999,"5min":1.2430000000000001,"15min":1.341},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.54300000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.03,"5min":1.177,"15min":1.3839999999999999},"last":0.70199999999999996}]},{"osd":7,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86299999999999999,"15min":0.86699999999999999},"min":{"1min":0.63300000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0960000000000001,"5min":1.272,"15min":1.4470000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85599999999999998,"15min":0.86699999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.121,"5min":1.1599999999999999,"15min":1.3839999999999999},"last":0.80400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56320,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315520,"statfs":{"total":96636764160,"available":96579092480,"internally_reserved":0,"allocated":3203072,"data_stored":1853467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.28799999999999998},"max":{"1min":0.71999999999999997,"5min":0.82299999999999995,"15min":0.96799999999999997},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.34899999999999998,"5min":0.33200000000000002,"15min":0.33000000000000002},"max":{"1min":0.76200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.502}]},{"osd":2,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.435,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.65000000000000002,"5min":0.77700000000000002,"15min":0.81299999999999994},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.38,"15min":0.38},"max":{"1min":0.81499999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65700000000000003,"5min":0.80500000000000005,"15min":1.0429999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.626},"min":{"1min":0.496,"5min":0.46100000000000002,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97899999999999998,"5min":1.0469999999999999,"15min":1.147},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.51400000000000001},"max":{"1min":0.94299999999999995,"5min":1.0760000000000001,"15min":1.2869999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.008,"5min":1.1259999999999999,"15min":1.2150000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83299999999999996,"15min":0.84299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.351},"last":0.70399999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.98999999999999999,"5min":1.1040000000000001,"15min":1.2150000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.85599999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57099999999999995},"max":{"1min":0.97899999999999998,"5min":1.1379999999999999,"15min":1.3320000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85099999999999998,"15min":0.86299999999999999},"min":{"1min":0.627,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97399999999999998,"5min":1.129,"15min":1.3560000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.85399999999999998,"15min":0.85899999999999999},"min":{"1min":0.61399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0009999999999999,"5min":1.139,"15min":1.2549999999999999},"last":0.65900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:02:31.060 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:02:31.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.240+0000 7f8f3f12c640 1 Processor -- start 2023-12-20T06:02:31.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.240+0000 7f8f3f12c640 1 -- start start 2023-12-20T06:02:31.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.240+0000 7f8f3f12c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f3806ce60 0x7f8f3815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:31.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.240+0000 7f8f3f12c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f3815f110 0x7f8f3815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:31.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.240+0000 7f8f3f12c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8f3815bbd0 con 0x7f8f380af5e0 2023-12-20T06:02:31.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.240+0000 7f8f3f12c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f3815beb0 con 0x7f8f3806ce60 2023-12-20T06:02:31.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.240+0000 7f8f3f12c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f3815e910 con 0x7f8f3815f110 2023-12-20T06:02:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f37fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f3806ce60 0x7f8f3815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f37fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f3806ce60 0x7f8f3815b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42838/0 (socket says 172.21.15.32:42838) 2023-12-20T06:02:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f37fff640 1 -- 172.21.15.32:0/2387319854 learned_addr learned my addr 172.21.15.32:0/2387319854 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f3d6c9640 1 --2- 172.21.15.32:0/2387319854 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f3815f110 0x7f8f3815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f37fff640 1 -- 172.21.15.32:0/2387319854 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f3815f110 msgr2=0x7f8f3815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f37fff640 1 --2- 172.21.15.32:0/2387319854 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f3815f110 0x7f8f3815c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f37fff640 1 -- 172.21.15.32:0/2387319854 >> v1:172.21.15.39:6789/0 conn(0x7f8f380af5e0 legacy=0x7f8f380af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f37fff640 1 -- 172.21.15.32:0/2387319854 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f3815ebf0 con 0x7f8f3806ce60 2023-12-20T06:02:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f37fff640 1 --2- 172.21.15.32:0/2387319854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f3806ce60 0x7f8f3815b280 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009870 tx=0x7f8f2401c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=737aadb1285986dc server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f377fe640 1 -- 172.21.15.32:0/2387319854 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f3815b9d0 con 0x7f8f3806ce60 2023-12-20T06:02:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f377fe640 1 -- 172.21.15.32:0/2387319854 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f240ab650 con 0x7f8f3806ce60 2023-12-20T06:02:31.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.244+0000 7f8f377fe640 1 -- 172.21.15.32:0/2387319854 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f240ab9e0 con 0x7f8f3806ce60 2023-12-20T06:02:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2387319854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f3806ce60 msgr2=0x7f8f3815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 --2- 172.21.15.32:0/2387319854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f3806ce60 0x7f8f3815b280 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f8f24009870 tx=0x7f8f2401c4b0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2387319854 shutdown_connections 2023-12-20T06:02:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 --2- 172.21.15.32:0/2387319854 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f3815f110 0x7f8f3815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 --2- 172.21.15.32:0/2387319854 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f3806ce60 0x7f8f3815b280 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2387319854 >> 172.21.15.32:0/2387319854 conn(0x7f8f3806a320 msgr2=0x7f8f380ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2387319854 shutdown_connections 2023-12-20T06:02:31.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2387319854 wait complete. 2023-12-20T06:02:31.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 Processor -- start 2023-12-20T06:02:31.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 -- start start 2023-12-20T06:02:31.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f3806ce60 0x7f8f381f6480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:31.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f380af5e0 0x7f8f381518f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f3815f110 0x7f8f38154150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f381f69c0 con 0x7f8f3806ce60 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f380b1f60 con 0x7f8f380af5e0 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3f12c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f38070610 con 0x7f8f3815f110 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3cec8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f3806ce60 0x7f8f381f6480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3cec8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f3806ce60 0x7f8f381f6480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42848/0 (socket says 172.21.15.32:42848) 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3cec8640 1 -- 172.21.15.32:0/2475213584 learned_addr learned my addr 172.21.15.32:0/2475213584 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3d6c9640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f3815f110 0x7f8f38154150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f37fff640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f380af5e0 0x7f8f381518f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3cec8640 1 -- 172.21.15.32:0/2475213584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f3815f110 msgr2=0x7f8f38154150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3cec8640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f3815f110 0x7f8f38154150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3cec8640 1 -- 172.21.15.32:0/2475213584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f380af5e0 msgr2=0x7f8f381518f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3cec8640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f380af5e0 0x7f8f381518f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3cec8640 1 -- 172.21.15.32:0/2475213584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f240094d0 con 0x7f8f3806ce60 2023-12-20T06:02:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f3d6c9640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f3815f110 0x7f8f38154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.248+0000 7f8f37fff640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f380af5e0 0x7f8f381518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.252+0000 7f8f3cec8640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f3806ce60 0x7f8f381f6480 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c450 tx=0x7f8f2c00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.252+0000 7f8f357fa640 1 -- 172.21.15.32:0/2475213584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c0052d0 con 0x7f8f3806ce60 2023-12-20T06:02:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.252+0000 7f8f357fa640 1 -- 172.21.15.32:0/2475213584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f2c005430 con 0x7f8f3806ce60 2023-12-20T06:02:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.252+0000 7f8f357fa640 1 -- 172.21.15.32:0/2475213584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f2c003890 con 0x7f8f3806ce60 2023-12-20T06:02:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.252+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2475213584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f380b3dc0 con 0x7f8f3806ce60 2023-12-20T06:02:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.252+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2475213584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f381523e0 con 0x7f8f3806ce60 2023-12-20T06:02:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.252+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2475213584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f380b3c20 con 0x7f8f3806ce60 2023-12-20T06:02:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.252+0000 7f8f357fa640 1 -- 172.21.15.32:0/2475213584 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8f2c0039f0 con 0x7f8f3806ce60 2023-12-20T06:02:31.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.252+0000 7f8f357fa640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f1c065b80 0x7f8f1c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:31.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.252+0000 7f8f357fa640 1 -- 172.21.15.32:0/2475213584 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f8f2c082e20 con 0x7f8f3806ce60 2023-12-20T06:02:31.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.252+0000 7f8f37fff640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f1c065b80 0x7f8f1c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:31.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.256+0000 7f8f37fff640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f1c065b80 0x7f8f1c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f240af000 tx=0x7f8f240a7000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.256+0000 7f8f357fa640 1 -- 172.21.15.32:0/2475213584 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8f380b3c20 con 0x7f8f3806ce60 2023-12-20T06:02:31.410 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.412+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2475213584 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f8f3815b170 con 0x7f8f1c065b80 2023-12-20T06:02:31.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.420+0000 7f8f357fa640 1 -- 172.21.15.32:0/2475213584 <== mgr.10314 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289598 (secure 0 0 0) 0x7f8f3815b170 con 0x7f8f1c065b80 2023-12-20T06:02:31.421 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:31.423 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2475213584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f1c065b80 msgr2=0x7f8f1c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f1c065b80 0x7f8f1c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f8f240af000 tx=0x7f8f240a7000 comp rx=0 tx=0).stop 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2475213584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f3806ce60 msgr2=0x7f8f381f6480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f3806ce60 0x7f8f381f6480 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x7f8f2c00c450 tx=0x7f8f2c00c910 comp rx=0 tx=0).stop 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2475213584 shutdown_connections 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f1c065b80 0x7f8f1c068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f3815f110 0x7f8f38154150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f380af5e0 0x7f8f381518f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 --2- 172.21.15.32:0/2475213584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f3806ce60 0x7f8f381f6480 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2475213584 >> 172.21.15.32:0/2475213584 conn(0x7f8f3806a320 msgr2=0x7f8f380ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2475213584 shutdown_connections 2023-12-20T06:02:31.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.424+0000 7f8f3f12c640 1 -- 172.21.15.32:0/2475213584 wait complete. 2023-12-20T06:02:31.442 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2023-12-20T06:02:30.954649+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":565,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3239,"ondisk_log_size":3239,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":107,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":400392,"kb_used_data":21576,"kb_used_omap":0,"kb_used_meta":378752,"kb_avail":754574328,"statfs":{"total":773094113280,"available":772684111872,"internally_reserved":0,"allocated":22093824,"data_stored":11290076,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":387842048},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.424669"},"pg_stats":[{"pgid":"25.3f","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983748+0000","last_change":"2023-12-20T06:02:26.982741+0000","last_active":"2023-12-20T06:02:26.983748+0000","last_peered":"2023-12-20T06:02:26.983748+0000","last_clean":"2023-12-20T06:02:26.983748+0000","last_became_active":"2023-12-20T06:02:26.982454+0000","last_became_peered":"2023-12-20T06:02:26.982454+0000","last_unstale":"2023-12-20T06:02:26.983748+0000","last_undegraded":"2023-12-20T06:02:26.983748+0000","last_fullsized":"2023-12-20T06:02:26.983748+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:59:08.944850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.963944+0000","last_change":"2023-12-20T06:02:26.956043+0000","last_active":"2023-12-20T06:02:26.963944+0000","last_peered":"2023-12-20T06:02:26.963944+0000","last_clean":"2023-12-20T06:02:26.963944+0000","last_became_active":"2023-12-20T06:02:26.955652+0000","last_became_peered":"2023-12-20T06:02:26.955652+0000","last_unstale":"2023-12-20T06:02:26.963944+0000","last_undegraded":"2023-12-20T06:02:26.963944+0000","last_fullsized":"2023-12-20T06:02:26.963944+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:02:53.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.970951+0000","last_change":"2023-12-20T06:02:26.968444+0000","last_active":"2023-12-20T06:02:26.970951+0000","last_peered":"2023-12-20T06:02:26.970951+0000","last_clean":"2023-12-20T06:02:26.970951+0000","last_became_active":"2023-12-20T06:02:26.967931+0000","last_became_peered":"2023-12-20T06:02:26.967931+0000","last_unstale":"2023-12-20T06:02:26.970951+0000","last_undegraded":"2023-12-20T06:02:26.970951+0000","last_fullsized":"2023-12-20T06:02:26.970951+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:33:14.150434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.961739+0000","last_change":"2023-12-20T06:02:26.955353+0000","last_active":"2023-12-20T06:02:26.961739+0000","last_peered":"2023-12-20T06:02:26.961739+0000","last_clean":"2023-12-20T06:02:26.961739+0000","last_became_active":"2023-12-20T06:02:26.954910+0000","last_became_peered":"2023-12-20T06:02:26.954910+0000","last_unstale":"2023-12-20T06:02:26.961739+0000","last_undegraded":"2023-12-20T06:02:26.961739+0000","last_fullsized":"2023-12-20T06:02:26.961739+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:44:12.085696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.968041+0000","last_change":"2023-12-20T06:02:26.962506+0000","last_active":"2023-12-20T06:02:26.968041+0000","last_peered":"2023-12-20T06:02:26.968041+0000","last_clean":"2023-12-20T06:02:26.968041+0000","last_became_active":"2023-12-20T06:02:26.962262+0000","last_became_peered":"2023-12-20T06:02:26.962262+0000","last_unstale":"2023-12-20T06:02:26.968041+0000","last_undegraded":"2023-12-20T06:02:26.968041+0000","last_fullsized":"2023-12-20T06:02:26.968041+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:09:09.108347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981133+0000","last_change":"2023-12-20T06:02:26.979634+0000","last_active":"2023-12-20T06:02:26.981133+0000","last_peered":"2023-12-20T06:02:26.981133+0000","last_clean":"2023-12-20T06:02:26.981133+0000","last_became_active":"2023-12-20T06:02:26.979366+0000","last_became_peered":"2023-12-20T06:02:26.979366+0000","last_unstale":"2023-12-20T06:02:26.981133+0000","last_undegraded":"2023-12-20T06:02:26.981133+0000","last_fullsized":"2023-12-20T06:02:26.981133+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:34:19.170698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981603+0000","last_change":"2023-12-20T06:02:26.978795+0000","last_active":"2023-12-20T06:02:26.981603+0000","last_peered":"2023-12-20T06:02:26.981603+0000","last_clean":"2023-12-20T06:02:26.981603+0000","last_became_active":"2023-12-20T06:02:26.978497+0000","last_became_peered":"2023-12-20T06:02:26.978497+0000","last_unstale":"2023-12-20T06:02:26.981603+0000","last_undegraded":"2023-12-20T06:02:26.981603+0000","last_fullsized":"2023-12-20T06:02:26.981603+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:06:01.713370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983266+0000","last_change":"2023-12-20T06:02:26.971111+0000","last_active":"2023-12-20T06:02:26.983266+0000","last_peered":"2023-12-20T06:02:26.983266+0000","last_clean":"2023-12-20T06:02:26.983266+0000","last_became_active":"2023-12-20T06:02:26.970783+0000","last_became_peered":"2023-12-20T06:02:26.970783+0000","last_unstale":"2023-12-20T06:02:26.983266+0000","last_undegraded":"2023-12-20T06:02:26.983266+0000","last_fullsized":"2023-12-20T06:02:26.983266+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:45.692179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.963937+0000","last_change":"2023-12-20T06:02:26.960947+0000","last_active":"2023-12-20T06:02:26.963937+0000","last_peered":"2023-12-20T06:02:26.963937+0000","last_clean":"2023-12-20T06:02:26.963937+0000","last_became_active":"2023-12-20T06:02:26.958931+0000","last_became_peered":"2023-12-20T06:02:26.958931+0000","last_unstale":"2023-12-20T06:02:26.963937+0000","last_undegraded":"2023-12-20T06:02:26.963937+0000","last_fullsized":"2023-12-20T06:02:26.963937+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979774+0000","last_change":"2023-12-20T06:02:26.973655+0000","last_active":"2023-12-20T06:02:26.979774+0000","last_peered":"2023-12-20T06:02:26.979774+0000","last_clean":"2023-12-20T06:02:26.979774+0000","last_became_active":"2023-12-20T06:02:26.973281+0000","last_became_peered":"2023-12-20T06:02:26.973281+0000","last_unstale":"2023-12-20T06:02:26.979774+0000","last_undegraded":"2023-12-20T06:02:26.979774+0000","last_fullsized":"2023-12-20T06:02:26.979774+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:36.523098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979425+0000","last_change":"2023-12-20T06:02:26.976628+0000","last_active":"2023-12-20T06:02:26.979425+0000","last_peered":"2023-12-20T06:02:26.979425+0000","last_clean":"2023-12-20T06:02:26.979425+0000","last_became_active":"2023-12-20T06:02:26.976337+0000","last_became_peered":"2023-12-20T06:02:26.976337+0000","last_unstale":"2023-12-20T06:02:26.979425+0000","last_undegraded":"2023-12-20T06:02:26.979425+0000","last_fullsized":"2023-12-20T06:02:26.979425+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:15.147652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979019+0000","last_change":"2023-12-20T06:02:26.976708+0000","last_active":"2023-12-20T06:02:26.979019+0000","last_peered":"2023-12-20T06:02:26.979019+0000","last_clean":"2023-12-20T06:02:26.979019+0000","last_became_active":"2023-12-20T06:02:26.976349+0000","last_became_peered":"2023-12-20T06:02:26.976349+0000","last_unstale":"2023-12-20T06:02:26.979019+0000","last_undegraded":"2023-12-20T06:02:26.979019+0000","last_fullsized":"2023-12-20T06:02:26.979019+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.969536+0000","last_change":"2023-12-20T06:02:26.967177+0000","last_active":"2023-12-20T06:02:26.969536+0000","last_peered":"2023-12-20T06:02:26.969536+0000","last_clean":"2023-12-20T06:02:26.969536+0000","last_became_active":"2023-12-20T06:02:26.966560+0000","last_became_peered":"2023-12-20T06:02:26.966560+0000","last_unstale":"2023-12-20T06:02:26.969536+0000","last_undegraded":"2023-12-20T06:02:26.969536+0000","last_fullsized":"2023-12-20T06:02:26.969536+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:12:37.519537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982556+0000","last_change":"2023-12-20T06:02:26.978685+0000","last_active":"2023-12-20T06:02:26.982556+0000","last_peered":"2023-12-20T06:02:26.982556+0000","last_clean":"2023-12-20T06:02:26.982556+0000","last_became_active":"2023-12-20T06:02:26.978364+0000","last_became_peered":"2023-12-20T06:02:26.978364+0000","last_unstale":"2023-12-20T06:02:26.982556+0000","last_undegraded":"2023-12-20T06:02:26.982556+0000","last_fullsized":"2023-12-20T06:02:26.982556+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:19:37.257026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.975739+0000","last_change":"2023-12-20T06:02:26.974037+0000","last_active":"2023-12-20T06:02:26.975739+0000","last_peered":"2023-12-20T06:02:26.975739+0000","last_clean":"2023-12-20T06:02:26.975739+0000","last_became_active":"2023-12-20T06:02:26.973742+0000","last_became_peered":"2023-12-20T06:02:26.973742+0000","last_unstale":"2023-12-20T06:02:26.975739+0000","last_undegraded":"2023-12-20T06:02:26.975739+0000","last_fullsized":"2023-12-20T06:02:26.975739+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.975395+0000","last_change":"2023-12-20T06:02:26.966987+0000","last_active":"2023-12-20T06:02:26.975395+0000","last_peered":"2023-12-20T06:02:26.975395+0000","last_clean":"2023-12-20T06:02:26.975395+0000","last_became_active":"2023-12-20T06:02:26.966704+0000","last_became_peered":"2023-12-20T06:02:26.966704+0000","last_unstale":"2023-12-20T06:02:26.975395+0000","last_undegraded":"2023-12-20T06:02:26.975395+0000","last_fullsized":"2023-12-20T06:02:26.975395+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:41:37.729575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.961962+0000","last_change":"2023-12-20T06:02:26.959816+0000","last_active":"2023-12-20T06:02:26.961962+0000","last_peered":"2023-12-20T06:02:26.961962+0000","last_clean":"2023-12-20T06:02:26.961962+0000","last_became_active":"2023-12-20T06:02:26.959542+0000","last_became_peered":"2023-12-20T06:02:26.959542+0000","last_unstale":"2023-12-20T06:02:26.961962+0000","last_undegraded":"2023-12-20T06:02:26.961962+0000","last_fullsized":"2023-12-20T06:02:26.961962+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:49:15.081247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983632+0000","last_change":"2023-12-20T06:02:26.981004+0000","last_active":"2023-12-20T06:02:26.983632+0000","last_peered":"2023-12-20T06:02:26.983632+0000","last_clean":"2023-12-20T06:02:26.983632+0000","last_became_active":"2023-12-20T06:02:26.980597+0000","last_became_peered":"2023-12-20T06:02:26.980597+0000","last_unstale":"2023-12-20T06:02:26.983632+0000","last_undegraded":"2023-12-20T06:02:26.983632+0000","last_fullsized":"2023-12-20T06:02:26.983632+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:13.705554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.977962+0000","last_change":"2023-12-20T06:02:26.972124+0000","last_active":"2023-12-20T06:02:26.977962+0000","last_peered":"2023-12-20T06:02:26.977962+0000","last_clean":"2023-12-20T06:02:26.977962+0000","last_became_active":"2023-12-20T06:02:26.971870+0000","last_became_peered":"2023-12-20T06:02:26.971870+0000","last_unstale":"2023-12-20T06:02:26.977962+0000","last_undegraded":"2023-12-20T06:02:26.977962+0000","last_fullsized":"2023-12-20T06:02:26.977962+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:06:44.203571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.958870+0000","last_change":"2023-12-20T06:02:26.957746+0000","last_active":"2023-12-20T06:02:26.958870+0000","last_peered":"2023-12-20T06:02:26.958870+0000","last_clean":"2023-12-20T06:02:26.958870+0000","last_became_active":"2023-12-20T06:02:26.957407+0000","last_became_peered":"2023-12-20T06:02:26.957407+0000","last_unstale":"2023-12-20T06:02:26.958870+0000","last_undegraded":"2023-12-20T06:02:26.958870+0000","last_fullsized":"2023-12-20T06:02:26.958870+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:51.962994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.975680+0000","last_change":"2023-12-20T06:02:26.969431+0000","last_active":"2023-12-20T06:02:26.975680+0000","last_peered":"2023-12-20T06:02:26.975680+0000","last_clean":"2023-12-20T06:02:26.975680+0000","last_became_active":"2023-12-20T06:02:26.969161+0000","last_became_peered":"2023-12-20T06:02:26.969161+0000","last_unstale":"2023-12-20T06:02:26.975680+0000","last_undegraded":"2023-12-20T06:02:26.975680+0000","last_fullsized":"2023-12-20T06:02:26.975680+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:49:57.166700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979236+0000","last_change":"2023-12-20T06:02:26.975847+0000","last_active":"2023-12-20T06:02:26.979236+0000","last_peered":"2023-12-20T06:02:26.979236+0000","last_clean":"2023-12-20T06:02:26.979236+0000","last_became_active":"2023-12-20T06:02:26.975128+0000","last_became_peered":"2023-12-20T06:02:26.975128+0000","last_unstale":"2023-12-20T06:02:26.979236+0000","last_undegraded":"2023-12-20T06:02:26.979236+0000","last_fullsized":"2023-12-20T06:02:26.979236+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:39:21.684738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.980912+0000","last_change":"2023-12-20T06:02:26.979485+0000","last_active":"2023-12-20T06:02:26.980912+0000","last_peered":"2023-12-20T06:02:26.980912+0000","last_clean":"2023-12-20T06:02:26.980912+0000","last_became_active":"2023-12-20T06:02:26.979208+0000","last_became_peered":"2023-12-20T06:02:26.979208+0000","last_unstale":"2023-12-20T06:02:26.980912+0000","last_undegraded":"2023-12-20T06:02:26.980912+0000","last_fullsized":"2023-12-20T06:02:26.980912+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:04:09.949376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.975379+0000","last_change":"2023-12-20T06:02:26.965794+0000","last_active":"2023-12-20T06:02:26.975379+0000","last_peered":"2023-12-20T06:02:26.975379+0000","last_clean":"2023-12-20T06:02:26.975379+0000","last_became_active":"2023-12-20T06:02:26.965505+0000","last_became_peered":"2023-12-20T06:02:26.965505+0000","last_unstale":"2023-12-20T06:02:26.975379+0000","last_undegraded":"2023-12-20T06:02:26.975379+0000","last_fullsized":"2023-12-20T06:02:26.975379+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:09.034710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978921+0000","last_change":"2023-12-20T06:02:26.976267+0000","last_active":"2023-12-20T06:02:26.978921+0000","last_peered":"2023-12-20T06:02:26.978921+0000","last_clean":"2023-12-20T06:02:26.978921+0000","last_became_active":"2023-12-20T06:02:26.975927+0000","last_became_peered":"2023-12-20T06:02:26.975927+0000","last_unstale":"2023-12-20T06:02:26.978921+0000","last_undegraded":"2023-12-20T06:02:26.978921+0000","last_fullsized":"2023-12-20T06:02:26.978921+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:19:37.257026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981794+0000","last_change":"2023-12-20T06:02:26.979767+0000","last_active":"2023-12-20T06:02:26.981794+0000","last_peered":"2023-12-20T06:02:26.981794+0000","last_clean":"2023-12-20T06:02:26.981794+0000","last_became_active":"2023-12-20T06:02:26.979450+0000","last_became_peered":"2023-12-20T06:02:26.979450+0000","last_unstale":"2023-12-20T06:02:26.981794+0000","last_undegraded":"2023-12-20T06:02:26.981794+0000","last_fullsized":"2023-12-20T06:02:26.981794+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:47:57.463784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983120+0000","last_change":"2023-12-20T06:02:26.981676+0000","last_active":"2023-12-20T06:02:26.983120+0000","last_peered":"2023-12-20T06:02:26.983120+0000","last_clean":"2023-12-20T06:02:26.983120+0000","last_became_active":"2023-12-20T06:02:26.980905+0000","last_became_peered":"2023-12-20T06:02:26.980905+0000","last_unstale":"2023-12-20T06:02:26.983120+0000","last_undegraded":"2023-12-20T06:02:26.983120+0000","last_fullsized":"2023-12-20T06:02:26.983120+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:43:42.660130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978607+0000","last_change":"2023-12-20T06:02:26.977513+0000","last_active":"2023-12-20T06:02:26.978607+0000","last_peered":"2023-12-20T06:02:26.978607+0000","last_clean":"2023-12-20T06:02:26.978607+0000","last_became_active":"2023-12-20T06:02:26.977220+0000","last_became_peered":"2023-12-20T06:02:26.977220+0000","last_unstale":"2023-12-20T06:02:26.978607+0000","last_undegraded":"2023-12-20T06:02:26.978607+0000","last_fullsized":"2023-12-20T06:02:26.978607+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:52.600966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.975837+0000","last_change":"2023-12-20T06:02:26.973161+0000","last_active":"2023-12-20T06:02:26.975837+0000","last_peered":"2023-12-20T06:02:26.975837+0000","last_clean":"2023-12-20T06:02:26.975837+0000","last_became_active":"2023-12-20T06:02:26.972883+0000","last_became_peered":"2023-12-20T06:02:26.972883+0000","last_unstale":"2023-12-20T06:02:26.975837+0000","last_undegraded":"2023-12-20T06:02:26.975837+0000","last_fullsized":"2023-12-20T06:02:26.975837+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:11.127223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.980434+0000","last_change":"2023-12-20T06:02:26.977346+0000","last_active":"2023-12-20T06:02:26.980434+0000","last_peered":"2023-12-20T06:02:26.980434+0000","last_clean":"2023-12-20T06:02:26.980434+0000","last_became_active":"2023-12-20T06:02:26.976527+0000","last_became_peered":"2023-12-20T06:02:26.976527+0000","last_unstale":"2023-12-20T06:02:26.980434+0000","last_undegraded":"2023-12-20T06:02:26.980434+0000","last_fullsized":"2023-12-20T06:02:26.980434+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:11.127223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.970248+0000","last_change":"2023-12-20T06:02:26.965576+0000","last_active":"2023-12-20T06:02:26.970248+0000","last_peered":"2023-12-20T06:02:26.970248+0000","last_clean":"2023-12-20T06:02:26.970248+0000","last_became_active":"2023-12-20T06:02:26.965189+0000","last_became_peered":"2023-12-20T06:02:26.965189+0000","last_unstale":"2023-12-20T06:02:26.970248+0000","last_undegraded":"2023-12-20T06:02:26.970248+0000","last_fullsized":"2023-12-20T06:02:26.970248+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:38.051872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.971230+0000","last_change":"2023-12-20T06:02:26.963846+0000","last_active":"2023-12-20T06:02:26.971230+0000","last_peered":"2023-12-20T06:02:26.971230+0000","last_clean":"2023-12-20T06:02:26.971230+0000","last_became_active":"2023-12-20T06:02:26.963552+0000","last_became_peered":"2023-12-20T06:02:26.963552+0000","last_unstale":"2023-12-20T06:02:26.971230+0000","last_undegraded":"2023-12-20T06:02:26.971230+0000","last_fullsized":"2023-12-20T06:02:26.971230+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:17:08.792842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983412+0000","last_change":"2023-12-20T06:02:26.966538+0000","last_active":"2023-12-20T06:02:26.983412+0000","last_peered":"2023-12-20T06:02:26.983412+0000","last_clean":"2023-12-20T06:02:26.983412+0000","last_became_active":"2023-12-20T06:02:26.966278+0000","last_became_peered":"2023-12-20T06:02:26.966278+0000","last_unstale":"2023-12-20T06:02:26.983412+0000","last_undegraded":"2023-12-20T06:02:26.983412+0000","last_fullsized":"2023-12-20T06:02:26.983412+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:12.114643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1f","version":"208'4","reported_seq":14,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.980496+0000","last_change":"2023-12-20T06:02:26.559340+0000","last_active":"2023-12-20T06:02:26.980496+0000","last_peered":"2023-12-20T06:02:26.980496+0000","last_clean":"2023-12-20T06:02:26.980496+0000","last_became_active":"2023-12-20T06:02:26.559045+0000","last_became_peered":"2023-12-20T06:02:26.559045+0000","last_unstale":"2023-12-20T06:02:26.980496+0000","last_undegraded":"2023-12-20T06:02:26.980496+0000","last_fullsized":"2023-12-20T06:02:26.980496+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:33.930072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982402+0000","last_change":"2023-12-20T06:02:26.978246+0000","last_active":"2023-12-20T06:02:26.982402+0000","last_peered":"2023-12-20T06:02:26.982402+0000","last_clean":"2023-12-20T06:02:26.982402+0000","last_became_active":"2023-12-20T06:02:26.977975+0000","last_became_peered":"2023-12-20T06:02:26.977975+0000","last_unstale":"2023-12-20T06:02:26.982402+0000","last_undegraded":"2023-12-20T06:02:26.982402+0000","last_fullsized":"2023-12-20T06:02:26.982402+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:05:43.763955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1e","version":"208'1","reported_seq":11,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.962669+0000","last_change":"2023-12-20T06:02:26.547864+0000","last_active":"2023-12-20T06:02:26.962669+0000","last_peered":"2023-12-20T06:02:26.962669+0000","last_clean":"2023-12-20T06:02:26.962669+0000","last_became_active":"2023-12-20T06:02:26.547556+0000","last_became_peered":"2023-12-20T06:02:26.547556+0000","last_unstale":"2023-12-20T06:02:26.962669+0000","last_undegraded":"2023-12-20T06:02:26.962669+0000","last_fullsized":"2023-12-20T06:02:26.962669+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:04.689001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.970351+0000","last_change":"2023-12-20T06:02:26.967888+0000","last_active":"2023-12-20T06:02:26.970351+0000","last_peered":"2023-12-20T06:02:26.970351+0000","last_clean":"2023-12-20T06:02:26.970351+0000","last_became_active":"2023-12-20T06:02:26.967582+0000","last_became_peered":"2023-12-20T06:02:26.967582+0000","last_unstale":"2023-12-20T06:02:26.970351+0000","last_undegraded":"2023-12-20T06:02:26.970351+0000","last_fullsized":"2023-12-20T06:02:26.970351+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:26:03.948722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.1d","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979018+0000","last_change":"2023-12-20T06:02:26.560653+0000","last_active":"2023-12-20T06:02:26.979018+0000","last_peered":"2023-12-20T06:02:26.979018+0000","last_clean":"2023-12-20T06:02:26.979018+0000","last_became_active":"2023-12-20T06:02:26.560323+0000","last_became_peered":"2023-12-20T06:02:26.560323+0000","last_unstale":"2023-12-20T06:02:26.979018+0000","last_undegraded":"2023-12-20T06:02:26.979018+0000","last_fullsized":"2023-12-20T06:02:26.979018+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:55.646229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.970993+0000","last_change":"2023-12-20T06:02:26.966379+0000","last_active":"2023-12-20T06:02:26.970993+0000","last_peered":"2023-12-20T06:02:26.970993+0000","last_clean":"2023-12-20T06:02:26.970993+0000","last_became_active":"2023-12-20T06:02:26.965974+0000","last_became_peered":"2023-12-20T06:02:26.965974+0000","last_unstale":"2023-12-20T06:02:26.970993+0000","last_undegraded":"2023-12-20T06:02:26.970993+0000","last_fullsized":"2023-12-20T06:02:26.970993+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:28.156831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.968846+0000","last_change":"2023-12-20T06:02:26.545330+0000","last_active":"2023-12-20T06:02:26.968846+0000","last_peered":"2023-12-20T06:02:26.968846+0000","last_clean":"2023-12-20T06:02:26.968846+0000","last_became_active":"2023-12-20T06:02:26.545018+0000","last_became_peered":"2023-12-20T06:02:26.545018+0000","last_unstale":"2023-12-20T06:02:26.968846+0000","last_undegraded":"2023-12-20T06:02:26.968846+0000","last_fullsized":"2023-12-20T06:02:26.968846+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:21:20.189262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979216+0000","last_change":"2023-12-20T06:02:26.974647+0000","last_active":"2023-12-20T06:02:26.979216+0000","last_peered":"2023-12-20T06:02:26.979216+0000","last_clean":"2023-12-20T06:02:26.979216+0000","last_became_active":"2023-12-20T06:02:26.974369+0000","last_became_peered":"2023-12-20T06:02:26.974369+0000","last_unstale":"2023-12-20T06:02:26.979216+0000","last_undegraded":"2023-12-20T06:02:26.979216+0000","last_fullsized":"2023-12-20T06:02:26.979216+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:36.494152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1b","version":"208'1","reported_seq":11,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978836+0000","last_change":"2023-12-20T06:02:26.560921+0000","last_active":"2023-12-20T06:02:26.978836+0000","last_peered":"2023-12-20T06:02:26.978836+0000","last_clean":"2023-12-20T06:02:26.978836+0000","last_became_active":"2023-12-20T06:02:26.560567+0000","last_became_peered":"2023-12-20T06:02:26.560567+0000","last_unstale":"2023-12-20T06:02:26.978836+0000","last_undegraded":"2023-12-20T06:02:26.978836+0000","last_fullsized":"2023-12-20T06:02:26.978836+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:01:42.572698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.973792+0000","last_change":"2023-12-20T06:02:26.966409+0000","last_active":"2023-12-20T06:02:26.973792+0000","last_peered":"2023-12-20T06:02:26.973792+0000","last_clean":"2023-12-20T06:02:26.973792+0000","last_became_active":"2023-12-20T06:02:26.966153+0000","last_became_peered":"2023-12-20T06:02:26.966153+0000","last_unstale":"2023-12-20T06:02:26.973792+0000","last_undegraded":"2023-12-20T06:02:26.973792+0000","last_fullsized":"2023-12-20T06:02:26.973792+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:27.909325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1a","version":"208'1","reported_seq":11,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982710+0000","last_change":"2023-12-20T06:02:26.556300+0000","last_active":"2023-12-20T06:02:26.982710+0000","last_peered":"2023-12-20T06:02:26.982710+0000","last_clean":"2023-12-20T06:02:26.982710+0000","last_became_active":"2023-12-20T06:02:26.556007+0000","last_became_peered":"2023-12-20T06:02:26.556007+0000","last_unstale":"2023-12-20T06:02:26.982710+0000","last_undegraded":"2023-12-20T06:02:26.982710+0000","last_fullsized":"2023-12-20T06:02:26.982710+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:58.838168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981583+0000","last_change":"2023-12-20T06:02:26.977844+0000","last_active":"2023-12-20T06:02:26.981583+0000","last_peered":"2023-12-20T06:02:26.981583+0000","last_clean":"2023-12-20T06:02:26.981583+0000","last_became_active":"2023-12-20T06:02:26.977331+0000","last_became_peered":"2023-12-20T06:02:26.977331+0000","last_unstale":"2023-12-20T06:02:26.981583+0000","last_undegraded":"2023-12-20T06:02:26.981583+0000","last_fullsized":"2023-12-20T06:02:26.981583+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:59:51.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.967171+0000","last_change":"2023-12-20T06:02:26.553243+0000","last_active":"2023-12-20T06:02:26.967171+0000","last_peered":"2023-12-20T06:02:26.967171+0000","last_clean":"2023-12-20T06:02:26.967171+0000","last_became_active":"2023-12-20T06:02:26.551953+0000","last_became_peered":"2023-12-20T06:02:26.551953+0000","last_unstale":"2023-12-20T06:02:26.967171+0000","last_undegraded":"2023-12-20T06:02:26.967171+0000","last_fullsized":"2023-12-20T06:02:26.967171+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:09:54.113512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.977120+0000","last_change":"2023-12-20T06:02:26.974738+0000","last_active":"2023-12-20T06:02:26.977120+0000","last_peered":"2023-12-20T06:02:26.977120+0000","last_clean":"2023-12-20T06:02:26.977120+0000","last_became_active":"2023-12-20T06:02:26.974494+0000","last_became_peered":"2023-12-20T06:02:26.974494+0000","last_unstale":"2023-12-20T06:02:26.977120+0000","last_undegraded":"2023-12-20T06:02:26.977120+0000","last_fullsized":"2023-12-20T06:02:26.977120+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.960421+0000","last_change":"2023-12-20T06:02:26.557303+0000","last_active":"2023-12-20T06:02:26.960421+0000","last_peered":"2023-12-20T06:02:26.960421+0000","last_clean":"2023-12-20T06:02:26.960421+0000","last_became_active":"2023-12-20T06:02:26.556838+0000","last_became_peered":"2023-12-20T06:02:26.556838+0000","last_unstale":"2023-12-20T06:02:26.960421+0000","last_undegraded":"2023-12-20T06:02:26.960421+0000","last_fullsized":"2023-12-20T06:02:26.960421+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:12:36.511646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.985674+0000","last_change":"2023-12-20T06:02:26.981970+0000","last_active":"2023-12-20T06:02:26.985674+0000","last_peered":"2023-12-20T06:02:26.985674+0000","last_clean":"2023-12-20T06:02:26.985674+0000","last_became_active":"2023-12-20T06:02:26.981688+0000","last_became_peered":"2023-12-20T06:02:26.981688+0000","last_unstale":"2023-12-20T06:02:26.985674+0000","last_undegraded":"2023-12-20T06:02:26.985674+0000","last_fullsized":"2023-12-20T06:02:26.985674+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:29:15.785654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.954445+0000","last_change":"2023-12-20T06:02:26.551507+0000","last_active":"2023-12-20T06:02:26.954445+0000","last_peered":"2023-12-20T06:02:26.954445+0000","last_clean":"2023-12-20T06:02:26.954445+0000","last_became_active":"2023-12-20T06:02:26.551215+0000","last_became_peered":"2023-12-20T06:02:26.551215+0000","last_unstale":"2023-12-20T06:02:26.954445+0000","last_undegraded":"2023-12-20T06:02:26.954445+0000","last_fullsized":"2023-12-20T06:02:26.954445+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:58:17.512600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.969999+0000","last_change":"2023-12-20T06:02:26.966176+0000","last_active":"2023-12-20T06:02:26.969999+0000","last_peered":"2023-12-20T06:02:26.969999+0000","last_clean":"2023-12-20T06:02:26.969999+0000","last_became_active":"2023-12-20T06:02:26.965898+0000","last_became_peered":"2023-12-20T06:02:26.965898+0000","last_unstale":"2023-12-20T06:02:26.969999+0000","last_undegraded":"2023-12-20T06:02:26.969999+0000","last_fullsized":"2023-12-20T06:02:26.969999+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:54:07.886847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.15","version":"208'3","reported_seq":13,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.986365+0000","last_change":"2023-12-20T06:02:26.555063+0000","last_active":"2023-12-20T06:02:26.986365+0000","last_peered":"2023-12-20T06:02:26.986365+0000","last_clean":"2023-12-20T06:02:26.986365+0000","last_became_active":"2023-12-20T06:02:26.554776+0000","last_became_peered":"2023-12-20T06:02:26.554776+0000","last_unstale":"2023-12-20T06:02:26.986365+0000","last_undegraded":"2023-12-20T06:02:26.986365+0000","last_fullsized":"2023-12-20T06:02:26.986365+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:59:43.701253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981209+0000","last_change":"2023-12-20T06:02:26.978068+0000","last_active":"2023-12-20T06:02:26.981209+0000","last_peered":"2023-12-20T06:02:26.981209+0000","last_clean":"2023-12-20T06:02:26.981209+0000","last_became_active":"2023-12-20T06:02:26.977750+0000","last_became_peered":"2023-12-20T06:02:26.977750+0000","last_unstale":"2023-12-20T06:02:26.981209+0000","last_undegraded":"2023-12-20T06:02:26.981209+0000","last_fullsized":"2023-12-20T06:02:26.981209+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:05:59.363948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.14","version":"208'6","reported_seq":16,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.986505+0000","last_change":"2023-12-20T06:02:26.558119+0000","last_active":"2023-12-20T06:02:26.986505+0000","last_peered":"2023-12-20T06:02:26.986505+0000","last_clean":"2023-12-20T06:02:26.986505+0000","last_became_active":"2023-12-20T06:02:26.557699+0000","last_became_peered":"2023-12-20T06:02:26.557699+0000","last_unstale":"2023-12-20T06:02:26.986505+0000","last_undegraded":"2023-12-20T06:02:26.986505+0000","last_fullsized":"2023-12-20T06:02:26.986505+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:21:20.189262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978860+0000","last_change":"2023-12-20T06:02:26.976279+0000","last_active":"2023-12-20T06:02:26.978860+0000","last_peered":"2023-12-20T06:02:26.978860+0000","last_clean":"2023-12-20T06:02:26.978860+0000","last_became_active":"2023-12-20T06:02:26.976019+0000","last_became_peered":"2023-12-20T06:02:26.976019+0000","last_unstale":"2023-12-20T06:02:26.978860+0000","last_undegraded":"2023-12-20T06:02:26.978860+0000","last_fullsized":"2023-12-20T06:02:26.978860+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.13","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983496+0000","last_change":"2023-12-20T06:02:26.552924+0000","last_active":"2023-12-20T06:02:26.983496+0000","last_peered":"2023-12-20T06:02:26.983496+0000","last_clean":"2023-12-20T06:02:26.983496+0000","last_became_active":"2023-12-20T06:02:26.552647+0000","last_became_peered":"2023-12-20T06:02:26.552647+0000","last_unstale":"2023-12-20T06:02:26.983496+0000","last_undegraded":"2023-12-20T06:02:26.983496+0000","last_fullsized":"2023-12-20T06:02:26.983496+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:44.700263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.981642+0000","last_change":"2023-12-20T06:02:26.979070+0000","last_active":"2023-12-20T06:02:26.981642+0000","last_peered":"2023-12-20T06:02:26.981642+0000","last_clean":"2023-12-20T06:02:26.981642+0000","last_became_active":"2023-12-20T06:02:26.978777+0000","last_became_peered":"2023-12-20T06:02:26.978777+0000","last_unstale":"2023-12-20T06:02:26.981642+0000","last_undegraded":"2023-12-20T06:02:26.981642+0000","last_fullsized":"2023-12-20T06:02:26.981642+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:54.315390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.966048+0000","last_change":"2023-12-20T06:02:26.553685+0000","last_active":"2023-12-20T06:02:26.966048+0000","last_peered":"2023-12-20T06:02:26.966048+0000","last_clean":"2023-12-20T06:02:26.966048+0000","last_became_active":"2023-12-20T06:02:26.553411+0000","last_became_peered":"2023-12-20T06:02:26.553411+0000","last_unstale":"2023-12-20T06:02:26.966048+0000","last_undegraded":"2023-12-20T06:02:26.966048+0000","last_fullsized":"2023-12-20T06:02:26.966048+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:52.266518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982431+0000","last_change":"2023-12-20T06:02:26.978618+0000","last_active":"2023-12-20T06:02:26.982431+0000","last_peered":"2023-12-20T06:02:26.982431+0000","last_clean":"2023-12-20T06:02:26.982431+0000","last_became_active":"2023-12-20T06:02:26.978305+0000","last_became_peered":"2023-12-20T06:02:26.978305+0000","last_unstale":"2023-12-20T06:02:26.982431+0000","last_undegraded":"2023-12-20T06:02:26.982431+0000","last_fullsized":"2023-12-20T06:02:26.982431+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:02:53.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.947748+0000","last_change":"2023-12-20T06:02:26.558390+0000","last_active":"2023-12-20T06:02:26.947748+0000","last_peered":"2023-12-20T06:02:26.947748+0000","last_clean":"2023-12-20T06:02:26.947748+0000","last_became_active":"2023-12-20T06:02:26.558078+0000","last_became_peered":"2023-12-20T06:02:26.558078+0000","last_unstale":"2023-12-20T06:02:26.947748+0000","last_undegraded":"2023-12-20T06:02:26.947748+0000","last_fullsized":"2023-12-20T06:02:26.947748+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:47:56.455893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978437+0000","last_change":"2023-12-20T06:02:26.970325+0000","last_active":"2023-12-20T06:02:26.978437+0000","last_peered":"2023-12-20T06:02:26.978437+0000","last_clean":"2023-12-20T06:02:26.978437+0000","last_became_active":"2023-12-20T06:02:26.970054+0000","last_became_peered":"2023-12-20T06:02:26.970054+0000","last_unstale":"2023-12-20T06:02:26.978437+0000","last_undegraded":"2023-12-20T06:02:26.978437+0000","last_fullsized":"2023-12-20T06:02:26.978437+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:23:08.197185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.10","version":"208'8","reported_seq":18,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.984485+0000","last_change":"2023-12-20T06:02:26.552002+0000","last_active":"2023-12-20T06:02:26.984485+0000","last_peered":"2023-12-20T06:02:26.984485+0000","last_clean":"2023-12-20T06:02:26.984485+0000","last_became_active":"2023-12-20T06:02:26.551664+0000","last_became_peered":"2023-12-20T06:02:26.551664+0000","last_unstale":"2023-12-20T06:02:26.984485+0000","last_undegraded":"2023-12-20T06:02:26.984485+0000","last_fullsized":"2023-12-20T06:02:26.984485+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:49:14.073356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979755+0000","last_change":"2023-12-20T06:02:26.973527+0000","last_active":"2023-12-20T06:02:26.979755+0000","last_peered":"2023-12-20T06:02:26.979755+0000","last_clean":"2023-12-20T06:02:26.979755+0000","last_became_active":"2023-12-20T06:02:26.972995+0000","last_became_peered":"2023-12-20T06:02:26.972995+0000","last_unstale":"2023-12-20T06:02:26.979755+0000","last_undegraded":"2023-12-20T06:02:26.979755+0000","last_fullsized":"2023-12-20T06:02:26.979755+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:15:22.108587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.f","version":"208'1","reported_seq":11,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.979050+0000","last_change":"2023-12-20T06:02:26.559146+0000","last_active":"2023-12-20T06:02:26.979050+0000","last_peered":"2023-12-20T06:02:26.979050+0000","last_clean":"2023-12-20T06:02:26.979050+0000","last_became_active":"2023-12-20T06:02:26.558865+0000","last_became_peered":"2023-12-20T06:02:26.558865+0000","last_unstale":"2023-12-20T06:02:26.979050+0000","last_undegraded":"2023-12-20T06:02:26.979050+0000","last_fullsized":"2023-12-20T06:02:26.979050+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:23:28.617951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982604+0000","last_change":"2023-12-20T06:02:26.979723+0000","last_active":"2023-12-20T06:02:26.982604+0000","last_peered":"2023-12-20T06:02:26.982604+0000","last_clean":"2023-12-20T06:02:26.982604+0000","last_became_active":"2023-12-20T06:02:26.979447+0000","last_became_peered":"2023-12-20T06:02:26.979447+0000","last_unstale":"2023-12-20T06:02:26.982604+0000","last_undegraded":"2023-12-20T06:02:26.982604+0000","last_fullsized":"2023-12-20T06:02:26.982604+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:44:12.085696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.e","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983482+0000","last_change":"2023-12-20T06:02:26.561295+0000","last_active":"2023-12-20T06:02:26.983482+0000","last_peered":"2023-12-20T06:02:26.983482+0000","last_clean":"2023-12-20T06:02:26.983482+0000","last_became_active":"2023-12-20T06:02:26.561016+0000","last_became_peered":"2023-12-20T06:02:26.561016+0000","last_unstale":"2023-12-20T06:02:26.983482+0000","last_undegraded":"2023-12-20T06:02:26.983482+0000","last_fullsized":"2023-12-20T06:02:26.983482+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:53.307500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.980468+0000","last_change":"2023-12-20T06:02:26.977809+0000","last_active":"2023-12-20T06:02:26.980468+0000","last_peered":"2023-12-20T06:02:26.980468+0000","last_clean":"2023-12-20T06:02:26.980468+0000","last_became_active":"2023-12-20T06:02:26.976835+0000","last_became_peered":"2023-12-20T06:02:26.976835+0000","last_unstale":"2023-12-20T06:02:26.980468+0000","last_undegraded":"2023-12-20T06:02:26.980468+0000","last_fullsized":"2023-12-20T06:02:26.980468+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"208'1","reported_seq":11,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983800+0000","last_change":"2023-12-20T06:02:26.553441+0000","last_active":"2023-12-20T06:02:26.983800+0000","last_peered":"2023-12-20T06:02:26.983800+0000","last_clean":"2023-12-20T06:02:26.983800+0000","last_became_active":"2023-12-20T06:02:26.551705+0000","last_became_peered":"2023-12-20T06:02:26.551705+0000","last_unstale":"2023-12-20T06:02:26.983800+0000","last_undegraded":"2023-12-20T06:02:26.983800+0000","last_fullsized":"2023-12-20T06:02:26.983800+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:51.593076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.965339+0000","last_change":"2023-12-20T06:02:26.958574+0000","last_active":"2023-12-20T06:02:26.965339+0000","last_peered":"2023-12-20T06:02:26.965339+0000","last_clean":"2023-12-20T06:02:26.965339+0000","last_became_active":"2023-12-20T06:02:26.958284+0000","last_became_peered":"2023-12-20T06:02:26.958284+0000","last_unstale":"2023-12-20T06:02:26.965339+0000","last_undegraded":"2023-12-20T06:02:26.965339+0000","last_fullsized":"2023-12-20T06:02:26.965339+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:28.156831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.960627+0000","last_change":"2023-12-20T06:02:26.559666+0000","last_active":"2023-12-20T06:02:26.960627+0000","last_peered":"2023-12-20T06:02:26.960627+0000","last_clean":"2023-12-20T06:02:26.960627+0000","last_became_active":"2023-12-20T06:02:26.559324+0000","last_became_peered":"2023-12-20T06:02:26.559324+0000","last_unstale":"2023-12-20T06:02:26.960627+0000","last_undegraded":"2023-12-20T06:02:26.960627+0000","last_fullsized":"2023-12-20T06:02:26.960627+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:27.148940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.978454+0000","last_change":"2023-12-20T06:02:26.972273+0000","last_active":"2023-12-20T06:02:26.978454+0000","last_peered":"2023-12-20T06:02:26.978454+0000","last_clean":"2023-12-20T06:02:26.978454+0000","last_became_active":"2023-12-20T06:02:26.971983+0000","last_became_peered":"2023-12-20T06:02:26.971983+0000","last_unstale":"2023-12-20T06:02:26.978454+0000","last_undegraded":"2023-12-20T06:02:26.978454+0000","last_fullsized":"2023-12-20T06:02:26.978454+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:35:59.779548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.b","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.971821+0000","last_change":"2023-12-20T06:02:26.565662+0000","last_active":"2023-12-20T06:02:26.971821+0000","last_peered":"2023-12-20T06:02:26.971821+0000","last_clean":"2023-12-20T06:02:26.971821+0000","last_became_active":"2023-12-20T06:02:26.564737+0000","last_became_peered":"2023-12-20T06:02:26.564737+0000","last_unstale":"2023-12-20T06:02:26.971821+0000","last_undegraded":"2023-12-20T06:02:26.971821+0000","last_fullsized":"2023-12-20T06:02:26.971821+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:59:26.669013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983895+0000","last_change":"2023-12-20T06:02:26.981736+0000","last_active":"2023-12-20T06:02:26.983895+0000","last_peered":"2023-12-20T06:02:26.983895+0000","last_clean":"2023-12-20T06:02:26.983895+0000","last_became_active":"2023-12-20T06:02:26.981185+0000","last_became_peered":"2023-12-20T06:02:26.981185+0000","last_unstale":"2023-12-20T06:02:26.983895+0000","last_undegraded":"2023-12-20T06:02:26.983895+0000","last_fullsized":"2023-12-20T06:02:26.983895+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:59.846058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.953076+0000","last_change":"2023-12-20T06:02:26.554403+0000","last_active":"2023-12-20T06:02:26.953076+0000","last_peered":"2023-12-20T06:02:26.953076+0000","last_clean":"2023-12-20T06:02:26.953076+0000","last_became_active":"2023-12-20T06:02:26.554112+0000","last_became_peered":"2023-12-20T06:02:26.554112+0000","last_unstale":"2023-12-20T06:02:26.953076+0000","last_undegraded":"2023-12-20T06:02:26.953076+0000","last_fullsized":"2023-12-20T06:02:26.953076+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:53:48.616814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.970727+0000","last_change":"2023-12-20T06:02:26.968137+0000","last_active":"2023-12-20T06:02:26.970727+0000","last_peered":"2023-12-20T06:02:26.970727+0000","last_clean":"2023-12-20T06:02:26.970727+0000","last_became_active":"2023-12-20T06:02:26.967861+0000","last_became_peered":"2023-12-20T06:02:26.967861+0000","last_unstale":"2023-12-20T06:02:26.970727+0000","last_undegraded":"2023-12-20T06:02:26.970727+0000","last_fullsized":"2023-12-20T06:02:26.970727+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:34:19.170698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.956777+0000","last_change":"2023-12-20T06:02:26.560641+0000","last_active":"2023-12-20T06:02:26.956777+0000","last_peered":"2023-12-20T06:02:26.956777+0000","last_clean":"2023-12-20T06:02:26.956777+0000","last_became_active":"2023-12-20T06:02:26.560352+0000","last_became_peered":"2023-12-20T06:02:26.560352+0000","last_unstale":"2023-12-20T06:02:26.956777+0000","last_undegraded":"2023-12-20T06:02:26.956777+0000","last_fullsized":"2023-12-20T06:02:26.956777+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:49.756970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.964489+0000","last_change":"2023-12-20T06:02:26.961186+0000","last_active":"2023-12-20T06:02:26.964489+0000","last_peered":"2023-12-20T06:02:26.964489+0000","last_clean":"2023-12-20T06:02:26.964489+0000","last_became_active":"2023-12-20T06:02:26.960904+0000","last_became_peered":"2023-12-20T06:02:26.960904+0000","last_unstale":"2023-12-20T06:02:26.964489+0000","last_undegraded":"2023-12-20T06:02:26.964489+0000","last_fullsized":"2023-12-20T06:02:26.964489+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:45.708153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.954337+0000","last_change":"2023-12-20T06:02:26.547934+0000","last_active":"2023-12-20T06:02:26.954337+0000","last_peered":"2023-12-20T06:02:26.954337+0000","last_clean":"2023-12-20T06:02:26.954337+0000","last_became_active":"2023-12-20T06:02:26.547651+0000","last_became_peered":"2023-12-20T06:02:26.547651+0000","last_unstale":"2023-12-20T06:02:26.954337+0000","last_undegraded":"2023-12-20T06:02:26.954337+0000","last_fullsized":"2023-12-20T06:02:26.954337+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:42:23.285931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.971578+0000","last_change":"2023-12-20T06:02:26.966239+0000","last_active":"2023-12-20T06:02:26.971578+0000","last_peered":"2023-12-20T06:02:26.971578+0000","last_clean":"2023-12-20T06:02:26.971578+0000","last_became_active":"2023-12-20T06:02:26.964925+0000","last_became_peered":"2023-12-20T06:02:26.964925+0000","last_unstale":"2023-12-20T06:02:26.971578+0000","last_undegraded":"2023-12-20T06:02:26.971578+0000","last_fullsized":"2023-12-20T06:02:26.971578+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:09.937547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.7","version":"208'2","reported_seq":12,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.980446+0000","last_change":"2023-12-20T06:02:26.549840+0000","last_active":"2023-12-20T06:02:26.980446+0000","last_peered":"2023-12-20T06:02:26.980446+0000","last_clean":"2023-12-20T06:02:26.980446+0000","last_became_active":"2023-12-20T06:02:26.549537+0000","last_became_peered":"2023-12-20T06:02:26.549537+0000","last_unstale":"2023-12-20T06:02:26.980446+0000","last_undegraded":"2023-12-20T06:02:26.980446+0000","last_fullsized":"2023-12-20T06:02:26.980446+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:50.955103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.982980+0000","last_change":"2023-12-20T06:02:26.981756+0000","last_active":"2023-12-20T06:02:26.982980+0000","last_peered":"2023-12-20T06:02:26.982980+0000","last_clean":"2023-12-20T06:02:26.982980+0000","last_became_active":"2023-12-20T06:02:26.980456+0000","last_became_peered":"2023-12-20T06:02:26.980456+0000","last_unstale":"2023-12-20T06:02:26.982980+0000","last_undegraded":"2023-12-20T06:02:26.982980+0000","last_fullsized":"2023-12-20T06:02:26.982980+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:33:51.384201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.6","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.968770+0000","last_change":"2023-12-20T06:02:26.555848+0000","last_active":"2023-12-20T06:02:26.968770+0000","last_peered":"2023-12-20T06:02:26.968770+0000","last_clean":"2023-12-20T06:02:26.968770+0000","last_became_active":"2023-12-20T06:02:26.555560+0000","last_became_peered":"2023-12-20T06:02:26.555560+0000","last_unstale":"2023-12-20T06:02:26.968770+0000","last_undegraded":"2023-12-20T06:02:26.968770+0000","last_fullsized":"2023-12-20T06:02:26.968770+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:44.684288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983950+0000","last_change":"2023-12-20T06:02:26.969245+0000","last_active":"2023-12-20T06:02:26.983950+0000","last_peered":"2023-12-20T06:02:26.983950+0000","last_clean":"2023-12-20T06:02:26.983950+0000","last_became_active":"2023-12-20T06:02:26.969011+0000","last_became_peered":"2023-12-20T06:02:26.969011+0000","last_unstale":"2023-12-20T06:02:26.983950+0000","last_undegraded":"2023-12-20T06:02:26.983950+0000","last_fullsized":"2023-12-20T06:02:26.983950+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:53.274408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.966257+0000","last_change":"2023-12-20T06:02:26.556970+0000","last_active":"2023-12-20T06:02:26.966257+0000","last_peered":"2023-12-20T06:02:26.966257+0000","last_clean":"2023-12-20T06:02:26.966257+0000","last_became_active":"2023-12-20T06:02:26.556577+0000","last_became_peered":"2023-12-20T06:02:26.556577+0000","last_unstale":"2023-12-20T06:02:26.966257+0000","last_undegraded":"2023-12-20T06:02:26.966257+0000","last_fullsized":"2023-12-20T06:02:26.966257+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:45.673434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.977301+0000","last_change":"2023-12-20T06:02:26.967116+0000","last_active":"2023-12-20T06:02:26.977301+0000","last_peered":"2023-12-20T06:02:26.977301+0000","last_clean":"2023-12-20T06:02:26.977301+0000","last_became_active":"2023-12-20T06:02:26.966832+0000","last_became_peered":"2023-12-20T06:02:26.966832+0000","last_unstale":"2023-12-20T06:02:26.977301+0000","last_undegraded":"2023-12-20T06:02:26.977301+0000","last_fullsized":"2023-12-20T06:02:26.977301+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:52:23.105026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.948070+0000","last_change":"2023-12-20T06:02:26.560397+0000","last_active":"2023-12-20T06:02:26.948070+0000","last_peered":"2023-12-20T06:02:26.948070+0000","last_clean":"2023-12-20T06:02:26.948070+0000","last_became_active":"2023-12-20T06:02:26.560128+0000","last_became_peered":"2023-12-20T06:02:26.560128+0000","last_unstale":"2023-12-20T06:02:26.948070+0000","last_undegraded":"2023-12-20T06:02:26.948070+0000","last_fullsized":"2023-12-20T06:02:26.948070+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:54:06.878956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.984298+0000","last_change":"2023-12-20T06:02:26.983027+0000","last_active":"2023-12-20T06:02:26.984298+0000","last_peered":"2023-12-20T06:02:26.984298+0000","last_clean":"2023-12-20T06:02:26.984298+0000","last_became_active":"2023-12-20T06:02:26.982556+0000","last_became_peered":"2023-12-20T06:02:26.982556+0000","last_unstale":"2023-12-20T06:02:26.984298+0000","last_undegraded":"2023-12-20T06:02:26.984298+0000","last_fullsized":"2023-12-20T06:02:26.984298+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:42:24.293821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.955310+0000","last_change":"2023-12-20T06:02:26.556385+0000","last_active":"2023-12-20T06:02:26.955310+0000","last_peered":"2023-12-20T06:02:26.955310+0000","last_clean":"2023-12-20T06:02:26.955310+0000","last_became_active":"2023-12-20T06:02:26.555308+0000","last_became_peered":"2023-12-20T06:02:26.555308+0000","last_unstale":"2023-12-20T06:02:26.955310+0000","last_undegraded":"2023-12-20T06:02:26.955310+0000","last_fullsized":"2023-12-20T06:02:26.955310+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:31.032511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.983559+0000","last_change":"2023-12-20T06:02:26.980532+0000","last_active":"2023-12-20T06:02:26.983559+0000","last_peered":"2023-12-20T06:02:26.983559+0000","last_clean":"2023-12-20T06:02:26.983559+0000","last_became_active":"2023-12-20T06:02:26.980234+0000","last_became_peered":"2023-12-20T06:02:26.980234+0000","last_unstale":"2023-12-20T06:02:26.983559+0000","last_undegraded":"2023-12-20T06:02:26.983559+0000","last_fullsized":"2023-12-20T06:02:26.983559+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:32.040401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.954575+0000","last_change":"2023-12-20T06:02:26.556064+0000","last_active":"2023-12-20T06:02:26.954575+0000","last_peered":"2023-12-20T06:02:26.954575+0000","last_clean":"2023-12-20T06:02:26.954575+0000","last_became_active":"2023-12-20T06:02:26.554935+0000","last_became_peered":"2023-12-20T06:02:26.554935+0000","last_unstale":"2023-12-20T06:02:26.954575+0000","last_undegraded":"2023-12-20T06:02:26.954575+0000","last_fullsized":"2023-12-20T06:02:26.954575+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:12.697664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.965443+0000","last_change":"2023-12-20T06:02:26.962551+0000","last_active":"2023-12-20T06:02:26.965443+0000","last_peered":"2023-12-20T06:02:26.965443+0000","last_clean":"2023-12-20T06:02:26.965443+0000","last_became_active":"2023-12-20T06:02:26.962269+0000","last_became_peered":"2023-12-20T06:02:26.962269+0000","last_unstale":"2023-12-20T06:02:26.965443+0000","last_undegraded":"2023-12-20T06:02:26.965443+0000","last_fullsized":"2023-12-20T06:02:26.965443+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:43:42.660130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.949016+0000","last_change":"2023-12-20T06:02:26.548501+0000","last_active":"2023-12-20T06:02:26.949016+0000","last_peered":"2023-12-20T06:02:26.949016+0000","last_clean":"2023-12-20T06:02:26.949016+0000","last_became_active":"2023-12-20T06:02:26.548203+0000","last_became_peered":"2023-12-20T06:02:26.548203+0000","last_unstale":"2023-12-20T06:02:26.949016+0000","last_undegraded":"2023-12-20T06:02:26.949016+0000","last_fullsized":"2023-12-20T06:02:26.949016+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:55.646229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.964966+0000","last_change":"2023-12-20T06:02:26.961645+0000","last_active":"2023-12-20T06:02:26.964966+0000","last_peered":"2023-12-20T06:02:26.964966+0000","last_clean":"2023-12-20T06:02:26.964966+0000","last_became_active":"2023-12-20T06:02:26.961386+0000","last_became_peered":"2023-12-20T06:02:26.961386+0000","last_unstale":"2023-12-20T06:02:26.964966+0000","last_undegraded":"2023-12-20T06:02:26.964966+0000","last_fullsized":"2023-12-20T06:02:26.964966+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:52:17.428938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.956509+0000","last_change":"2023-12-20T06:02:26.560054+0000","last_active":"2023-12-20T06:02:26.956509+0000","last_peered":"2023-12-20T06:02:26.956509+0000","last_clean":"2023-12-20T06:02:26.956509+0000","last_became_active":"2023-12-20T06:02:26.559777+0000","last_became_peered":"2023-12-20T06:02:26.559777+0000","last_unstale":"2023-12-20T06:02:26.956509+0000","last_undegraded":"2023-12-20T06:02:26.956509+0000","last_fullsized":"2023-12-20T06:02:26.956509+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:40:31.615384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":9,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.977670+0000","last_change":"2023-12-20T06:02:26.971114+0000","last_active":"2023-12-20T06:02:26.977670+0000","last_peered":"2023-12-20T06:02:26.977670+0000","last_clean":"2023-12-20T06:02:26.977670+0000","last_became_active":"2023-12-20T06:02:26.970836+0000","last_became_peered":"2023-12-20T06:02:26.970836+0000","last_unstale":"2023-12-20T06:02:26.977670+0000","last_undegraded":"2023-12-20T06:02:26.977670+0000","last_fullsized":"2023-12-20T06:02:26.977670+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:50:58.624420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":10,"reported_epoch":208,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.969717+0000","last_change":"2023-12-20T06:02:26.550026+0000","last_active":"2023-12-20T06:02:26.969717+0000","last_peered":"2023-12-20T06:02:26.969717+0000","last_clean":"2023-12-20T06:02:26.969717+0000","last_became_active":"2023-12-20T06:02:26.549712+0000","last_became_peered":"2023-12-20T06:02:26.549712+0000","last_unstale":"2023-12-20T06:02:26.969717+0000","last_undegraded":"2023-12-20T06:02:26.969717+0000","last_fullsized":"2023-12-20T06:02:26.969717+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:08.929656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"196'3201","reported_seq":3991,"reported_epoch":207,"state":"active+clean","last_fresh":"2023-12-20T06:02:26.536771+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:02:26.536771+0000","last_peered":"2023-12-20T06:02:26.536771+0000","last_clean":"2023-12-20T06:02:26.536771+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:02:26.536771+0000","last_undegraded":"2023-12-20T06:02:26.536771+0000","last_fullsized":"2023-12-20T06:02:26.536771+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3201,"log_dups_size":0,"ondisk_log_size":3201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":521,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":521,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3201,"ondisk_log_size":3201,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575970,"num_pgs":24,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56324,"kb_used_data":3132,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315516,"statfs":{"total":96636764160,"available":96579088384,"internally_reserved":0,"allocated":3207168,"data_stored":1853549,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.73499999999999999},"min":{"1min":0.497,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.042},"last":0.53600000000000003},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72599999999999998,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.34499999999999997},"max":{"1min":0.874,"5min":0.99099999999999999,"15min":1.161},"last":0.55100000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73199999999999998,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.432},"max":{"1min":0.88700000000000001,"5min":0.996,"15min":1.1279999999999999},"last":0.59599999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.53100000000000003,"5min":0.51100000000000001,"15min":0.46999999999999997},"max":{"1min":0.90900000000000003,"5min":0.96699999999999997,"15min":1.032},"last":0.49399999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72299999999999998,"15min":0.748},"min":{"1min":0.51900000000000002,"5min":0.48699999999999999,"15min":0.48299999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0409999999999999},"last":0.46999999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96999999999999997,"5min":1.0189999999999999,"15min":1.038},"last":0.56999999999999995}]},{"osd":3,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72899999999999998,"15min":0.746},"min":{"1min":0.61899999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.5},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77900000000000003,"15min":0.79000000000000004},"min":{"1min":0.53800000000000003,"5min":0.503,"15min":0.40699999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0840000000000001},"last":0.64100000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77600000000000002,"5min":0.91800000000000004,"15min":1.04},"last":0.379},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.91800000000000004,"5min":1.02,"15min":1.02},"last":0.41799999999999998}]},{"osd":5,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86699999999999999,"5min":0.97499999999999998,"15min":1.1120000000000001},"last":0.62},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.54300000000000004,"5min":0.47199999999999998,"15min":0.45600000000000002},"max":{"1min":0.84199999999999997,"5min":1.0229999999999999,"15min":1.103},"last":0.45300000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76500000000000001,"15min":0.751},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.45500000000000002},"max":{"1min":0.94399999999999995,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.51600000000000001},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92300000000000004,"5min":1.0740000000000001,"15min":1.1539999999999999},"last":0.52100000000000002}]}]},{"osd":6,"up_from":13,"seq":55834575970,"num_pgs":17,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51640,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320200,"statfs":{"total":96636764160,"available":96583884800,"internally_reserved":0,"allocated":2605056,"data_stored":1263077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.71299999999999997,"5min":0.72999999999999998,"15min":0.74399999999999999},"min":{"1min":0.39100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":1.016,"5min":1.016,"15min":1.0980000000000001},"last":0.60899999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.73499999999999999,"15min":0.754},"min":{"1min":0.51300000000000001,"5min":0.376,"15min":0.376},"max":{"1min":0.89800000000000002,"5min":1.016,"15min":1.05},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.72699999999999998,"15min":0.73799999999999999},"min":{"1min":0.45100000000000001,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.78600000000000003},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.77000000000000002,"15min":0.77300000000000002},"min":{"1min":0.42099999999999999,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.94999999999999996,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.88200000000000001}]},{"osd":2,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.78600000000000003,"5min":0.78000000000000003,"15min":0.78300000000000003},"min":{"1min":0.45200000000000001,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96199999999999997,"5min":1.069,"15min":1.069},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.745,"15min":0.76700000000000002},"min":{"1min":0.54900000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.96799999999999997,"5min":0.97799999999999998,"15min":1.0760000000000001},"last":0.84799999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.78600000000000003,"15min":0.78900000000000003},"min":{"1min":0.627,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":0.95799999999999996,"5min":1.077,"15min":1.077},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.77300000000000002,"5min":0.76200000000000001,"15min":0.76600000000000001},"min":{"1min":0.57299999999999995,"5min":0.51200000000000001,"15min":0.503},"max":{"1min":0.93500000000000005,"5min":1.052,"15min":1.052},"last":0.92500000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.63200000000000001,"15min":0.63100000000000001},"min":{"1min":0.49199999999999999,"5min":0.40200000000000002,"15min":0.36099999999999999},"max":{"1min":1.05,"5min":1.05,"15min":1.0529999999999999},"last":0.95999999999999996},{"interface":"front","average":{"1min":0.65000000000000002,"5min":0.64900000000000002,"15min":0.64800000000000002},"min":{"1min":0.46000000000000002,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.80200000000000005,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.91700000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.66600000000000004,"5min":0.64100000000000001,"15min":0.64000000000000001},"min":{"1min":0.53400000000000003,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":0.97999999999999998,"5min":0.97999999999999998,"15min":1.0009999999999999},"last":0.98699999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.71299999999999997,"15min":0.70399999999999996},"min":{"1min":0.54600000000000004,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":1.016,"5min":1.016,"15min":1.016},"last":1.004}]},{"osd":7,"last update":"Wed Dec 20 06:01:32 2023","interfaces":[{"interface":"back","average":{"1min":0.72599999999999998,"5min":0.73499999999999999,"15min":0.73199999999999998},"min":{"1min":0.50600000000000001,"5min":0.48899999999999999,"15min":0.45300000000000001},"max":{"1min":0.997,"5min":1.103,"15min":1.1240000000000001},"last":0.94799999999999995},{"interface":"front","average":{"1min":0.73699999999999999,"5min":0.71999999999999997,"15min":0.72799999999999998},"min":{"1min":0.52300000000000002,"5min":0.49399999999999999,"15min":0.41999999999999998},"max":{"1min":1.022,"5min":1.022,"15min":1.0700000000000001},"last":1.0409999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575971,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":29496,"kb_used_data":2544,"kb_used_omap":0,"kb_used_meta":26944,"kb_avail":94342344,"statfs":{"total":96636764160,"available":96606560256,"internally_reserved":0,"allocated":2605056,"data_stored":1263077,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":27590656},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.44,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94499999999999995,"5min":0.97499999999999998,"15min":1.077},"last":0.76400000000000001},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73199999999999998,"15min":0.755},"min":{"1min":0.57699999999999996,"5min":0.432,"15min":0.432},"max":{"1min":0.91400000000000003,"5min":0.96699999999999997,"15min":1.028},"last":0.45600000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.74299999999999999,"15min":0.754},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.442},"max":{"1min":0.89000000000000001,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.46899999999999997},"max":{"1min":0.95699999999999996,"5min":1.016,"15min":1.0640000000000001},"last":0.53700000000000003}]},{"osd":2,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73199999999999998,"15min":0.75900000000000001},"min":{"1min":0.60299999999999998,"5min":0.46999999999999997,"15min":0.46899999999999997},"max":{"1min":0.93400000000000005,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76500000000000001,"15min":0.77200000000000002},"min":{"1min":0.63100000000000001,"5min":0.55800000000000005,"15min":0.48299999999999998},"max":{"1min":0.93500000000000005,"5min":1.0349999999999999,"15min":1.0680000000000001},"last":0.72799999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.0669999999999999},"last":0.59399999999999997},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.65700000000000003,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.91700000000000004,"5min":1.004,"15min":1.0629999999999999},"last":0.60699999999999998}]},{"osd":4,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47199999999999998,"5min":0.42199999999999999,"15min":0.38100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0640000000000001,"15min":1.077},"last":0.56000000000000005},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67000000000000004,"15min":0.64200000000000002},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41299999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.69399999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.70299999999999996},"min":{"1min":0.56299999999999994,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.95199999999999996,"5min":1.012,"15min":1.073},"last":0.66800000000000004}]},{"osd":7,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.56799999999999995,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.63300000000000001,"5min":0.56299999999999994,"15min":0.46700000000000003},"max":{"1min":0.97599999999999998,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.71399999999999997}]}]},{"osd":4,"up_from":13,"seq":55834575972,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47560,"kb_used_data":2560,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324280,"statfs":{"total":96636764160,"available":96588062720,"internally_reserved":0,"allocated":2621440,"data_stored":1265223,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71899999999999997,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.38800000000000001},"max":{"1min":0.84399999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.56899999999999995,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":0.95699999999999996,"5min":0.96499999999999997,"15min":1.0900000000000001},"last":0.751},{"interface":"front","average":{"1min":0.752,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.52200000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91400000000000003,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.63}]},{"osd":2,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76200000000000001,"15min":0.77600000000000002},"min":{"1min":0.58999999999999997,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.66400000000000003},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.78100000000000003},"min":{"1min":0.56999999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98599999999999999,"5min":1.0089999999999999,"15min":1.101},"last":0.51000000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.59899999999999998,"5min":0.51800000000000002,"15min":0.49199999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.64500000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.95999999999999996,"5min":1.04,"15min":1.081},"last":0.57299999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.39900000000000002},"max":{"1min":0.93899999999999995,"5min":1.05,"15min":1.05},"last":0.45200000000000001},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.86099999999999999,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.47599999999999998}]},{"osd":6,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94699999999999995,"5min":1.016,"15min":1.1679999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.41199999999999998},"max":{"1min":0.97999999999999998,"5min":1.107,"15min":1.111},"last":0.68200000000000005}]},{"osd":7,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":1.01,"5min":1.0600000000000001,"15min":1.1160000000000001},"last":0.54100000000000004},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.52900000000000003,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.034,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.78100000000000003}]}]},{"osd":3,"up_from":13,"seq":55834575971,"num_pgs":4,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47556,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324284,"statfs":{"total":96636764160,"available":96588066816,"internally_reserved":0,"allocated":2617344,"data_stored":1263271,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.59299999999999997,"5min":0.748,"15min":2.8130000000000002},"last":0.55000000000000004},{"interface":"front","average":{"1min":0.504,"5min":0.51100000000000001,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64600000000000002,"5min":0.80000000000000004,"15min":1.357},"last":0.39100000000000001}]},{"osd":1,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":1.4219999999999999},"last":0.45000000000000001}]},{"osd":2,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.34200000000000003},"max":{"1min":0.72599999999999998,"5min":0.86099999999999999,"15min":2.944},"last":0.5}]},{"osd":4,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":1.0209999999999999,"5min":1.2090000000000001,"15min":1.669},"last":1.0009999999999999},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.99},"last":0.91600000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.59599999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97199999999999998,"5min":1.355,"15min":1.901},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.82399999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97699999999999998,"5min":1.196,"15min":2.8639999999999999},"last":0.93300000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.81999999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.575},"last":0.79300000000000004},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.81799999999999995,"15min":0.85099999999999998},"min":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0149999999999999,"5min":1.2350000000000001,"15min":1.948},"last":0.86499999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80400000000000005,"15min":0.84099999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.004,"5min":1.3149999999999999,"15min":2.9049999999999998},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82399999999999995,"15min":0.85299999999999998},"min":{"1min":0.61899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.089,"5min":1.28,"15min":1.754},"last":0.97099999999999997}]}]},{"osd":2,"up_from":13,"seq":55834575970,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51652,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320188,"statfs":{"total":96636764160,"available":96583872512,"internally_reserved":0,"allocated":2617344,"data_stored":1263771,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.26100000000000001},"max":{"1min":0.75800000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.32400000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.376,"5min":0.34000000000000002,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.48199999999999998}]},{"osd":1,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.52800000000000002},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.30499999999999999},"max":{"1min":0.71299999999999997,"5min":0.92600000000000005,"15min":1.004},"last":0.36799999999999999}]},{"osd":3,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.65800000000000003,"15min":0.64900000000000002},"min":{"1min":0.47499999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.089,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.46500000000000002,"15min":0.42499999999999999},"max":{"1min":1.069,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.38900000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79200000000000004,"15min":0.80300000000000005},"min":{"1min":0.58699999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.081,"5min":1.119,"15min":1.254},"last":0.63200000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.59199999999999997,"5min":0.505,"15min":0.48199999999999998},"max":{"1min":1.02,"5min":1.605,"15min":1.605},"last":0.40000000000000002}]},{"osd":5,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81200000000000006,"15min":0.80700000000000005},"min":{"1min":0.62,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1020000000000001,"5min":1.329,"15min":1.329},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.83899999999999997,"15min":0.83399999999999996},"min":{"1min":0.68500000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0309999999999999,"5min":1.0740000000000001,"15min":1.157},"last":0.56399999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.60299999999999998,"5min":0.55100000000000005,"15min":0.52300000000000002},"max":{"1min":1.157,"5min":1.919,"15min":1.919},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63300000000000001,"5min":0.56999999999999995,"15min":0.56200000000000006},"max":{"1min":1.0449999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.621}]},{"osd":7,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.875,"15min":0.85999999999999999},"min":{"1min":0.67400000000000004,"5min":0.60699999999999998,"15min":0.47899999999999998},"max":{"1min":1.3360000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.89000000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.262,"5min":1.292,"15min":1.292},"last":0.67800000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575972,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59844,"kb_used_data":2556,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311996,"statfs":{"total":96636764160,"available":96575483904,"internally_reserved":0,"allocated":2617344,"data_stored":1264641,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.48799999999999999,"15min":0.46200000000000002},"min":{"1min":0.22700000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.69599999999999995,"5min":0.746,"15min":0.81000000000000005},"last":0.40999999999999998},{"interface":"front","average":{"1min":0.54300000000000004,"5min":0.51900000000000002,"15min":0.51500000000000001},"min":{"1min":0.313,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.77300000000000002,"5min":0.78200000000000003,"15min":0.81100000000000005},"last":0.33300000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.53600000000000003,"5min":0.53900000000000003,"15min":0.53300000000000003},"min":{"1min":0.33800000000000002,"5min":0.33800000000000002,"15min":0.33800000000000002},"max":{"1min":0.82599999999999996,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45600000000000002},{"interface":"front","average":{"1min":0.55800000000000005,"5min":0.57399999999999995,"15min":0.57599999999999996},"min":{"1min":0.36599999999999999,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.72399999999999998,"5min":0.88800000000000001,"15min":0.93700000000000006},"last":0.38100000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.61699999999999999,"15min":0.61599999999999999},"min":{"1min":0.434,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.77800000000000002,"5min":0.90100000000000002,"15min":0.90100000000000002},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.54200000000000004,"5min":0.55800000000000005,"15min":0.55500000000000005},"min":{"1min":0.36199999999999999,"5min":0.35499999999999998,"15min":0.33900000000000002},"max":{"1min":0.73899999999999999,"5min":0.88700000000000001,"15min":0.93500000000000005},"last":0.48899999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.78300000000000003,"15min":0.79000000000000004},"min":{"1min":0.53200000000000003,"5min":0.49299999999999999,"15min":0.49299999999999999},"max":{"1min":1.107,"5min":1.107,"15min":1.321},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.57399999999999995,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.95999999999999996,"5min":1.139,"15min":1.25},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.80300000000000005,"15min":0.80900000000000005},"min":{"1min":0.59499999999999997,"5min":0.51400000000000001,"15min":0.443},"max":{"1min":1.131,"5min":1.224,"15min":1.607},"last":0.86199999999999999},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.80400000000000005,"15min":0.81200000000000006},"min":{"1min":0.56000000000000005,"5min":0.55200000000000005,"15min":0.55200000000000005},"max":{"1min":1.0569999999999999,"5min":1.25,"15min":1.4330000000000001},"last":0.63400000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.79200000000000004,"5min":0.81699999999999995,"15min":0.82999999999999996},"min":{"1min":0.57899999999999996,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0029999999999999,"5min":1.2430000000000001,"15min":1.341},"last":0.68300000000000005},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81899999999999995,"15min":0.81599999999999995},"min":{"1min":0.54300000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.03,"5min":1.177,"15min":1.3839999999999999},"last":0.70199999999999996}]},{"osd":7,"last update":"Wed Dec 20 06:01:41 2023","interfaces":[{"interface":"back","average":{"1min":0.84999999999999998,"5min":0.86299999999999999,"15min":0.86699999999999999},"min":{"1min":0.63300000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0960000000000001,"5min":1.272,"15min":1.4470000000000001},"last":0.83399999999999996},{"interface":"front","average":{"1min":0.81399999999999995,"5min":0.85599999999999998,"15min":0.86699999999999999},"min":{"1min":0.58399999999999996,"5min":0.58399999999999996,"15min":0.58399999999999996},"max":{"1min":1.121,"5min":1.1599999999999999,"15min":1.3839999999999999},"last":0.80400000000000005}]}]},{"osd":0,"up_from":13,"seq":55834575971,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56320,"kb_used_data":3128,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315520,"statfs":{"total":96636764160,"available":96579092480,"internally_reserved":0,"allocated":3203072,"data_stored":1853467,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.28799999999999998},"max":{"1min":0.71999999999999997,"5min":0.82299999999999995,"15min":0.96799999999999997},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.34899999999999998,"5min":0.33200000000000002,"15min":0.33000000000000002},"max":{"1min":0.76200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.502}]},{"osd":2,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.435,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.65000000000000002,"5min":0.77700000000000002,"15min":0.81299999999999994},"last":0.48699999999999999},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.38,"15min":0.38},"max":{"1min":0.81499999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.40100000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65700000000000003,"5min":0.80500000000000005,"15min":1.0429999999999999},"last":0.45400000000000001},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.626},"min":{"1min":0.496,"5min":0.46100000000000002,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.41399999999999998}]},{"osd":4,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97899999999999998,"5min":1.0469999999999999,"15min":1.147},"last":0.57199999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.51400000000000001},"max":{"1min":0.94299999999999995,"5min":1.0760000000000001,"15min":1.2869999999999999},"last":0.67600000000000005}]},{"osd":5,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.008,"5min":1.1259999999999999,"15min":1.2150000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83299999999999996,"15min":0.84299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.351},"last":0.70399999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.98999999999999999,"5min":1.1040000000000001,"15min":1.2150000000000001},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.85599999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57099999999999995},"max":{"1min":0.97899999999999998,"5min":1.1379999999999999,"15min":1.3320000000000001},"last":0.69299999999999995}]},{"osd":7,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85099999999999998,"15min":0.86299999999999999},"min":{"1min":0.627,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97399999999999998,"5min":1.129,"15min":1.3560000000000001},"last":0.71699999999999997},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.85399999999999998,"15min":0.85899999999999999},"min":{"1min":0.61399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0009999999999999,"5min":1.139,"15min":1.2549999999999999},"last":0.65900000000000003}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:02:31.446 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T06:02:31.446 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T06:02:31.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f667c152640 1 Processor -- start 2023-12-20T06:02:31.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f667c152640 1 -- start start 2023-12-20T06:02:31.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f667c152640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f66740d13a0 0x7f66740cb590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:31.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f667c152640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667415ef70 con 0x7f66740cbad0 2023-12-20T06:02:31.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f667c152640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f667415f200 con 0x7f667415acf0 2023-12-20T06:02:31.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f667c152640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f667415f7c0 con 0x7f66740d13a0 2023-12-20T06:02:31.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f667a6ef640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f66740cbad0 0x7f66740ceff0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54776/0 (socket says 172.21.15.32:54776) 2023-12-20T06:02:31.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f667a6ef640 1 -- 172.21.15.32:0/1048926111 learned_addr learned my addr 172.21.15.32:0/1048926111 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:31.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f66796ed640 1 --2- 172.21.15.32:0/1048926111 >> v2:172.21.15.39:3300/0 conn(0x7f66740d13a0 0x7f66740cb590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:31.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f6678eec640 1 -- 172.21.15.32:0/1048926111 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 150415185 0 0) 0x7f667415ef70 con 0x7f66740cbad0 2023-12-20T06:02:31.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f66796ed640 1 -- 172.21.15.32:0/1048926111 >> v1:172.21.15.39:6789/0 conn(0x7f667415acf0 legacy=0x7f667415e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f66796ed640 1 -- 172.21.15.32:0/1048926111 >> v1:172.21.15.32:6789/0 conn(0x7f66740cbad0 legacy=0x7f66740ceff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.636+0000 7f66796ed640 1 -- 172.21.15.32:0/1048926111 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f667415f4e0 con 0x7f66740d13a0 2023-12-20T06:02:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.640+0000 7f66796ed640 1 --2- 172.21.15.32:0/1048926111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66740d13a0 0x7f66740cb590 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6664009740 tx=0x7f666401c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e6c4c768e6f01e33 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.640+0000 7f6678eec640 1 -- 172.21.15.32:0/1048926111 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66640ab600 con 0x7f66740d13a0 2023-12-20T06:02:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.640+0000 7f6678eec640 1 -- 172.21.15.32:0/1048926111 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66640a7050 con 0x7f66740d13a0 2023-12-20T06:02:31.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.640+0000 7f6678eec640 1 -- 172.21.15.32:0/1048926111 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66640ab980 con 0x7f66740d13a0 2023-12-20T06:02:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.640+0000 7f667c152640 1 -- 172.21.15.32:0/1048926111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66740d13a0 msgr2=0x7f66740cb590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.640+0000 7f667c152640 1 --2- 172.21.15.32:0/1048926111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66740d13a0 0x7f66740cb590 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7f6664009740 tx=0x7f666401c280 comp rx=0 tx=0).stop 2023-12-20T06:02:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.640+0000 7f667c152640 1 -- 172.21.15.32:0/1048926111 shutdown_connections 2023-12-20T06:02:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.640+0000 7f667c152640 1 --2- 172.21.15.32:0/1048926111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f66740d13a0 0x7f66740cb590 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.640+0000 7f667c152640 1 -- 172.21.15.32:0/1048926111 >> 172.21.15.32:0/1048926111 conn(0x7f667408a760 msgr2=0x7f6674099d50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.640+0000 7f667c152640 1 -- 172.21.15.32:0/1048926111 shutdown_connections 2023-12-20T06:02:31.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.640+0000 7f667c152640 1 -- 172.21.15.32:0/1048926111 wait complete. 2023-12-20T06:02:31.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f667c152640 1 Processor -- start 2023-12-20T06:02:31.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f667c152640 1 -- start start 2023-12-20T06:02:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f667c152640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66740cbad0 0x7f6674151a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f667c152640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66740d13a0 0x7f66741573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f667c152640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667415acf0 0x7f66741f8640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f667c152640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6674157cb0 con 0x7f66740d13a0 2023-12-20T06:02:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f667c152640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6674157f90 con 0x7f667415acf0 2023-12-20T06:02:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f667c152640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6674158270 con 0x7f66740cbad0 2023-12-20T06:02:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f66796ed640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66740d13a0 0x7f66741573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:31.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f66796ed640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66740d13a0 0x7f66741573d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42860/0 (socket says 172.21.15.32:42860) 2023-12-20T06:02:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f66796ed640 1 -- 172.21.15.32:0/1180530917 learned_addr learned my addr 172.21.15.32:0/1180530917 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f6679eee640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66740cbad0 0x7f6674151a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f667a6ef640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667415acf0 0x7f66741f8640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f66796ed640 1 -- 172.21.15.32:0/1180530917 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66740cbad0 msgr2=0x7f6674151a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f66796ed640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66740cbad0 0x7f6674151a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f66796ed640 1 -- 172.21.15.32:0/1180530917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667415acf0 msgr2=0x7f66741f8640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f66796ed640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667415acf0 0x7f66741f8640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f66796ed640 1 -- 172.21.15.32:0/1180530917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6664009420 con 0x7f66740d13a0 2023-12-20T06:02:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f66796ed640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66740d13a0 0x7f66741573d0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f6664018e90 tx=0x7f666401aac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f666a7fc640 1 -- 172.21.15.32:0/1180530917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66640b7650 con 0x7f66740d13a0 2023-12-20T06:02:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f666a7fc640 1 -- 172.21.15.32:0/1180530917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f66640abe90 con 0x7f66740d13a0 2023-12-20T06:02:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f666a7fc640 1 -- 172.21.15.32:0/1180530917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f66640b7a10 con 0x7f66740d13a0 2023-12-20T06:02:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f667c152640 1 -- 172.21.15.32:0/1180530917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6674158550 con 0x7f66740d13a0 2023-12-20T06:02:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.644+0000 7f667c152640 1 -- 172.21.15.32:0/1180530917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6674158830 con 0x7f66740d13a0 2023-12-20T06:02:31.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.648+0000 7f667c152640 1 -- 172.21.15.32:0/1180530917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6674099df0 con 0x7f66740d13a0 2023-12-20T06:02:31.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.648+0000 7f666a7fc640 1 -- 172.21.15.32:0/1180530917 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f66640b7b70 con 0x7f66740d13a0 2023-12-20T06:02:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.648+0000 7f666a7fc640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6658065b80 0x7f6658068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.648+0000 7f666a7fc640 1 -- 172.21.15.32:0/1180530917 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f666411deb0 con 0x7f66740d13a0 2023-12-20T06:02:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.648+0000 7f6679eee640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6658065b80 0x7f6658068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:31.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.648+0000 7f6679eee640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6658065b80 0x7f6658068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6670000f70 tx=0x7f6670009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:31.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.652+0000 7f666a7fc640 1 -- 172.21.15.32:0/1180530917 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f66640eb570 con 0x7f66740d13a0 2023-12-20T06:02:31.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.868+0000 7f667c152640 1 -- 172.21.15.32:0/1180530917 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f667404e000 con 0x7f66740d13a0 2023-12-20T06:02:31.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f666a7fc640 1 -- 172.21.15.32:0/1180530917 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v52) v1 ==== 190+0+0 (secure 0 0 0) 0x7f66640b00e0 con 0x7f66740d13a0 2023-12-20T06:02:31.873 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T06:02:31.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 -- 172.21.15.32:0/1180530917 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6658065b80 msgr2=0x7f6658068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6658065b80 0x7f6658068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f6670000f70 tx=0x7f6670009040 comp rx=0 tx=0).stop 2023-12-20T06:02:31.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 -- 172.21.15.32:0/1180530917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66740d13a0 msgr2=0x7f66741573d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:31.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66740d13a0 0x7f66741573d0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f6664018e90 tx=0x7f666401aac0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 -- 172.21.15.32:0/1180530917 shutdown_connections 2023-12-20T06:02:31.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6658065b80 0x7f6658068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f667415acf0 0x7f66741f8640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f66740d13a0 0x7f66741573d0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 --2- 172.21.15.32:0/1180530917 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f66740cbad0 0x7f6674151a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:31.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 -- 172.21.15.32:0/1180530917 >> 172.21.15.32:0/1180530917 conn(0x7f667408a760 msgr2=0x7f6674166430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:31.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 -- 172.21.15.32:0/1180530917 shutdown_connections 2023-12-20T06:02:31.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:31.876+0000 7f667c152640 1 -- 172.21.15.32:0/1180530917 wait complete. 2023-12-20T06:02:31.894 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:02:32.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.040+0000 7fee03b18640 1 Processor -- start 2023-12-20T06:02:32.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.040+0000 7fee03b18640 1 -- start start 2023-12-20T06:02:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee03b18640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedfc06cf10 0x7fedfc0c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee03b18640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedfc164ef0 0x7fedfc0c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee03b18640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedfc0ae3f0 con 0x7fedfc0ae010 2023-12-20T06:02:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee03b18640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedfc161800 con 0x7fedfc164ef0 2023-12-20T06:02:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee03b18640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc165380 con 0x7fedfc06cf10 2023-12-20T06:02:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee020b5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedfc164ef0 0x7fedfc0c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee020b5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedfc164ef0 0x7fedfc0c99e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42864/0 (socket says 172.21.15.32:42864) 2023-12-20T06:02:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee020b5640 1 -- 172.21.15.32:0/4333597 learned_addr learned my addr 172.21.15.32:0/4333597 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee010b3640 1 --2- 172.21.15.32:0/4333597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedfc06cf10 0x7fedfc0c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee020b5640 1 -- 172.21.15.32:0/4333597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedfc06cf10 msgr2=0x7fedfc0c94a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:32.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee020b5640 1 --2- 172.21.15.32:0/4333597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedfc06cf10 0x7fedfc0c94a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee020b5640 1 -- 172.21.15.32:0/4333597 >> v1:172.21.15.39:6789/0 conn(0x7fedfc0ae010 legacy=0x7fedfc071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.044+0000 7fee020b5640 1 -- 172.21.15.32:0/4333597 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedfc165660 con 0x7fedfc164ef0 2023-12-20T06:02:32.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee020b5640 1 --2- 172.21.15.32:0/4333597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc164ef0 0x7fedfc0c99e0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009e50 tx=0x7fedf401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=875cfddc36cb4bfc server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:32.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee008b2640 1 -- 172.21.15.32:0/4333597 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf40a5810 con 0x7fedfc164ef0 2023-12-20T06:02:32.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee008b2640 1 -- 172.21.15.32:0/4333597 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf40a5970 con 0x7fedfc164ef0 2023-12-20T06:02:32.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee008b2640 1 -- 172.21.15.32:0/4333597 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf40a5d00 con 0x7fedfc164ef0 2023-12-20T06:02:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 -- 172.21.15.32:0/4333597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc164ef0 msgr2=0x7fedfc0c99e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 --2- 172.21.15.32:0/4333597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc164ef0 0x7fedfc0c99e0 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7fedf4009e50 tx=0x7fedf401c950 comp rx=0 tx=0).stop 2023-12-20T06:02:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 -- 172.21.15.32:0/4333597 shutdown_connections 2023-12-20T06:02:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 --2- 172.21.15.32:0/4333597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc164ef0 0x7fedfc0c99e0 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 --2- 172.21.15.32:0/4333597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedfc06cf10 0x7fedfc0c94a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 -- 172.21.15.32:0/4333597 >> 172.21.15.32:0/4333597 conn(0x7fedfc06a320 msgr2=0x7fedfc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 -- 172.21.15.32:0/4333597 shutdown_connections 2023-12-20T06:02:32.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 -- 172.21.15.32:0/4333597 wait complete. 2023-12-20T06:02:32.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 Processor -- start 2023-12-20T06:02:32.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 -- start start 2023-12-20T06:02:32.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0ae010 0x7fedfc0c26a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedfc0c40c0 0x7fedfc0c4540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedfc158a00 0x7fedfc158e80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc1593c0 con 0x7fedfc0ae010 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc15e790 con 0x7fedfc0c40c0 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee03b18640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedfc0c4c40 con 0x7fedfc158a00 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.048+0000 7fee018b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0ae010 0x7fedfc0c26a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee018b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0ae010 0x7fedfc0c26a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42872/0 (socket says 172.21.15.32:42872) 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee018b4640 1 -- 172.21.15.32:0/3057622644 learned_addr learned my addr 172.21.15.32:0/3057622644 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee010b3640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedfc0c40c0 0x7fedfc0c4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee020b5640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedfc158a00 0x7fedfc158e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee018b4640 1 -- 172.21.15.32:0/3057622644 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedfc158a00 msgr2=0x7fedfc158e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee018b4640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedfc158a00 0x7fedfc158e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee018b4640 1 -- 172.21.15.32:0/3057622644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedfc0c40c0 msgr2=0x7fedfc0c4540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee018b4640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedfc0c40c0 0x7fedfc0c4540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee018b4640 1 -- 172.21.15.32:0/3057622644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedf4009b30 con 0x7fedfc0ae010 2023-12-20T06:02:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee018b4640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0ae010 0x7fedfc0c26a0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c480 tx=0x7fedf800c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fedf27fc640 1 -- 172.21.15.32:0/3057622644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf80052d0 con 0x7fedfc0ae010 2023-12-20T06:02:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fedf27fc640 1 -- 172.21.15.32:0/3057622644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf8005430 con 0x7fedfc0ae010 2023-12-20T06:02:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fedf27fc640 1 -- 172.21.15.32:0/3057622644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf8003890 con 0x7fedfc0ae010 2023-12-20T06:02:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee03b18640 1 -- 172.21.15.32:0/3057622644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedfc0c4f20 con 0x7fedfc0ae010 2023-12-20T06:02:32.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee03b18640 1 -- 172.21.15.32:0/3057622644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedfc15e420 con 0x7fedfc0ae010 2023-12-20T06:02:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fee03b18640 1 -- 172.21.15.32:0/3057622644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedfc04e000 con 0x7fedfc0ae010 2023-12-20T06:02:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fedf27fc640 1 -- 172.21.15.32:0/3057622644 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fedf80039f0 con 0x7fedfc0ae010 2023-12-20T06:02:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fedf27fc640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fede0065b80 0x7fede0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.052+0000 7fedf27fc640 1 -- 172.21.15.32:0/3057622644 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fedf8082e10 con 0x7fedfc0ae010 2023-12-20T06:02:32.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.056+0000 7fee010b3640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fede0065b80 0x7fede0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.056+0000 7fee010b3640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fede0065b80 0x7fede0068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedfc164b40 tx=0x7fedec009190 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:32.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.056+0000 7fedf27fc640 1 -- 172.21.15.32:0/3057622644 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fedf8086030 con 0x7fedfc0ae010 2023-12-20T06:02:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.248+0000 7fee03b18640 1 -- 172.21.15.32:0/3057622644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fedfc160fb0 con 0x7fedfc0ae010 2023-12-20T06:02:32.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.248+0000 7fedf27fc640 1 -- 172.21.15.32:0/3057622644 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 96 v96) v1 ==== 94+0+4710 (secure 0 0 0) 0x7fedf8050720 con 0x7fedfc0ae010 2023-12-20T06:02:32.247 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:32.248 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":96,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10530,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/3001088147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3001088147},{"type":"v1","addr":"172.21.15.39:6833","nonce":3001088147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:26.942526+0000","modified":"2023-12-20T06:02:29.955675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10539":{"gid":10539,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/888710554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":888710554},{"type":"v1","addr":"172.21.15.32:6837","nonce":888710554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12}]} 2023-12-20T06:02:32.248 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 96 2023-12-20T06:02:32.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 -- 172.21.15.32:0/3057622644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fede0065b80 msgr2=0x7fede0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fede0065b80 0x7fede0068040 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7fedfc164b40 tx=0x7fedec009190 comp rx=0 tx=0).stop 2023-12-20T06:02:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 -- 172.21.15.32:0/3057622644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0ae010 msgr2=0x7fedfc0c26a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0ae010 0x7fedfc0c26a0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7fedf800c480 tx=0x7fedf800c940 comp rx=0 tx=0).stop 2023-12-20T06:02:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 -- 172.21.15.32:0/3057622644 shutdown_connections 2023-12-20T06:02:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fede0065b80 0x7fede0068040 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedfc158a00 0x7fedfc158e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedfc0c40c0 0x7fedfc0c4540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 --2- 172.21.15.32:0/3057622644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedfc0ae010 0x7fedfc0c26a0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 -- 172.21.15.32:0/3057622644 >> 172.21.15.32:0/3057622644 conn(0x7fedfc06a320 msgr2=0x7fedfc073230 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 -- 172.21.15.32:0/3057622644 shutdown_connections 2023-12-20T06:02:32.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.252+0000 7fee03b18640 1 -- 172.21.15.32:0/3057622644 wait complete. 2023-12-20T06:02:32.511 INFO:tasks.ceph.mds.c.smithi032.stderr:2023-12-20T06:02:32.512+0000 7f426b674640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T06:02:32.671 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 96, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T06:02:26.942526+0000', 'modified': '2023-12-20T06:02:29.955675+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10539}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10539': {'gid': 10539, 'name': 'c', 'rank': 0, 'incarnation': 92, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/888710554', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6836', 'nonce': 888710554}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 888710554}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [25], 'metadata_pool': 24, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T06:02:32.672 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T06:02:32.672 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-20T06:02:32.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0370fa6640 1 Processor -- start 2023-12-20T06:02:32.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0370fa6640 1 -- start start 2023-12-20T06:02:32.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0370fa6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f036c0707b0 0x7f036c0c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0370fa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f036c06cc10 0x7f036c06d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0370fa6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f036c0b17f0 con 0x7f036c15adc0 2023-12-20T06:02:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0370fa6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f036c0b1ad0 con 0x7f036c0707b0 2023-12-20T06:02:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0370fa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0b1db0 con 0x7f036c06cc10 2023-12-20T06:02:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0363fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f036c0707b0 0x7f036c0c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0363fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f036c0707b0 0x7f036c0c5be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42884/0 (socket says 172.21.15.32:42884) 2023-12-20T06:02:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0363fff640 1 -- 172.21.15.32:0/1843238986 learned_addr learned my addr 172.21.15.32:0/1843238986 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f036ad9d640 1 --2- 172.21.15.32:0/1843238986 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f036c06cc10 0x7f036c06d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0363fff640 1 -- 172.21.15.32:0/1843238986 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f036c06cc10 msgr2=0x7f036c06d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0363fff640 1 --2- 172.21.15.32:0/1843238986 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f036c06cc10 0x7f036c06d0a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0363fff640 1 -- 172.21.15.32:0/1843238986 >> v1:172.21.15.39:6789/0 conn(0x7f036c15adc0 legacy=0x7f036c0700a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.824+0000 7f0363fff640 1 -- 172.21.15.32:0/1843238986 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f036c0b2090 con 0x7f036c0707b0 2023-12-20T06:02:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0363fff640 1 --2- 172.21.15.32:0/1843238986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c0707b0 0x7f036c0c5be0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f0354009870 tx=0x7f035401cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=704399d40b43aeae server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:32.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0369d9b640 1 -- 172.21.15.32:0/1843238986 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03540a95f0 con 0x7f036c0707b0 2023-12-20T06:02:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0369d9b640 1 -- 172.21.15.32:0/1843238986 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f03540a9750 con 0x7f036c0707b0 2023-12-20T06:02:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0369d9b640 1 -- 172.21.15.32:0/1843238986 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03540a9ae0 con 0x7f036c0707b0 2023-12-20T06:02:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0370fa6640 1 -- 172.21.15.32:0/1843238986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c0707b0 msgr2=0x7f036c0c5be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0370fa6640 1 --2- 172.21.15.32:0/1843238986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c0707b0 0x7f036c0c5be0 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f0354009870 tx=0x7f035401cf70 comp rx=0 tx=0).stop 2023-12-20T06:02:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0370fa6640 1 -- 172.21.15.32:0/1843238986 shutdown_connections 2023-12-20T06:02:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0370fa6640 1 --2- 172.21.15.32:0/1843238986 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f036c06cc10 0x7f036c06d0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0370fa6640 1 --2- 172.21.15.32:0/1843238986 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c0707b0 0x7f036c0c5be0 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0370fa6640 1 -- 172.21.15.32:0/1843238986 >> 172.21.15.32:0/1843238986 conn(0x7f036c0ce650 msgr2=0x7f036c0cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0370fa6640 1 -- 172.21.15.32:0/1843238986 shutdown_connections 2023-12-20T06:02:32.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0370fa6640 1 -- 172.21.15.32:0/1843238986 wait complete. 2023-12-20T06:02:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.828+0000 7f0370fa6640 1 Processor -- start 2023-12-20T06:02:32.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0370fa6640 1 -- start start 2023-12-20T06:02:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0370fa6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c06cc10 0x7f036c155950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0370fa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f036c0707b0 0x7f036c0c6bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0370fa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c15adc0 0x7f036c0c7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0370fa6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c14b5b0 con 0x7f036c06cc10 2023-12-20T06:02:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0370fa6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f036c0cc550 con 0x7f036c15adc0 2023-12-20T06:02:32.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0370fa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f036c15d590 con 0x7f036c0707b0 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f036a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c06cc10 0x7f036c155950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f036a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c06cc10 0x7f036c155950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42898/0 (socket says 172.21.15.32:42898) 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f036a59c640 1 -- 172.21.15.32:0/3360589666 learned_addr learned my addr 172.21.15.32:0/3360589666 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f036ad9d640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c15adc0 0x7f036c0c7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0363fff640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f036c0707b0 0x7f036c0c6bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f036a59c640 1 -- 172.21.15.32:0/3360589666 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f036c0707b0 msgr2=0x7f036c0c6bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f036a59c640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f036c0707b0 0x7f036c0c6bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f036a59c640 1 -- 172.21.15.32:0/3360589666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c15adc0 msgr2=0x7f036c0c7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f036a59c640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c15adc0 0x7f036c0c7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f036a59c640 1 -- 172.21.15.32:0/3360589666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f03540094d0 con 0x7f036c06cc10 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f036a59c640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c06cc10 0x7f036c155950 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f035c00c370 tx=0x7f035c00c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:32.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0362ffd640 1 -- 172.21.15.32:0/3360589666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c0052d0 con 0x7f036c06cc10 2023-12-20T06:02:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0362ffd640 1 -- 172.21.15.32:0/3360589666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c005430 con 0x7f036c06cc10 2023-12-20T06:02:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0362ffd640 1 -- 172.21.15.32:0/3360589666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c007260 con 0x7f036c06cc10 2023-12-20T06:02:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0370fa6640 1 -- 172.21.15.32:0/3360589666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f036c15d870 con 0x7f036c06cc10 2023-12-20T06:02:32.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.832+0000 7f0370fa6640 1 -- 172.21.15.32:0/3360589666 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c0cca80 con 0x7f036c06cc10 2023-12-20T06:02:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0362ffd640 1 -- 172.21.15.32:0/3360589666 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f035c0073c0 con 0x7f036c06cc10 2023-12-20T06:02:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0362ffd640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f034c065b80 0x7f034c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0362ffd640 1 -- 172.21.15.32:0/3360589666 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f035c082650 con 0x7f036c06cc10 2023-12-20T06:02:32.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0363fff640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f034c065b80 0x7f034c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0374309000 1 Processor -- start 2023-12-20T06:02:32.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0374309000 1 -- start start 2023-12-20T06:02:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0363fff640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f034c065b80 0x7f034c068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f03540ae000 tx=0x7f03540b3040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0374309000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c2dd450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0374309000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd990 0x55c64c292c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0374309000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 0x55c64c2935a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0374309000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64c0b6c90 con 0x55c64c2dd070 2023-12-20T06:02:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0374309000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64c0e8220 con 0x55c64c2931c0 2023-12-20T06:02:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0374309000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c64c0e57f0 con 0x55c64c2dd990 2023-12-20T06:02:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f036a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c2dd450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f036a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c2dd450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42914/0 (socket says 172.21.15.32:42914) 2023-12-20T06:02:32.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f036a59c640 1 -- 172.21.15.32:0/1097478332 learned_addr learned my addr 172.21.15.32:0/1097478332 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f036ad9d640 1 --2- 172.21.15.32:0/1097478332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 0x55c64c2935a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f0363fff640 1 --2- 172.21.15.32:0/1097478332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd990 0x55c64c292c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f036ad9d640 1 -- 172.21.15.32:0/1097478332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd990 msgr2=0x55c64c292c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f036ad9d640 1 --2- 172.21.15.32:0/1097478332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd990 0x55c64c292c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f036ad9d640 1 -- 172.21.15.32:0/1097478332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 msgr2=0x55c64c2dd450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f036ad9d640 1 --2- 172.21.15.32:0/1097478332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c2dd450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f036ad9d640 1 -- 172.21.15.32:0/1097478332 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c64bddd890 con 0x55c64c2931c0 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.836+0000 7f036a59c640 1 --2- 172.21.15.32:0/1097478332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c2dd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f036ad9d640 1 --2- 172.21.15.32:0/1097478332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 0x55c64c2935a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f03640113f0 tx=0x7f0364009750 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9c9a587d3fe838f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f034bfff640 1 -- 172.21.15.32:0/1097478332 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036400e040 con 0x55c64c2931c0 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f034bfff640 1 -- 172.21.15.32:0/1097478332 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f036400f040 con 0x55c64c2931c0 2023-12-20T06:02:32.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f034bfff640 1 -- 172.21.15.32:0/1097478332 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0364003da0 con 0x55c64c2931c0 2023-12-20T06:02:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 -- 172.21.15.32:0/1097478332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 msgr2=0x55c64c2935a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 --2- 172.21.15.32:0/1097478332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 0x55c64c2935a0 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7f03640113f0 tx=0x7f0364009750 comp rx=0 tx=0).stop 2023-12-20T06:02:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 -- 172.21.15.32:0/1097478332 shutdown_connections 2023-12-20T06:02:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 --2- 172.21.15.32:0/1097478332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 0x55c64c2935a0 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 --2- 172.21.15.32:0/1097478332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd990 0x55c64c292c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 --2- 172.21.15.32:0/1097478332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c2dd450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 -- 172.21.15.32:0/1097478332 >> 172.21.15.32:0/1097478332 conn(0x55c64c299950 msgr2=0x55c64c291560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 -- 172.21.15.32:0/1097478332 shutdown_connections 2023-12-20T06:02:32.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 -- 172.21.15.32:0/1097478332 wait complete. 2023-12-20T06:02:32.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 -- start start 2023-12-20T06:02:32.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 Processor -- start 2023-12-20T06:02:32.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 0x55c64c298780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd070 0x55c64c29f080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd990 0x55c64c29f5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64c0e10e0 con 0x55c64c2dd990 2023-12-20T06:02:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64c080310 con 0x55c64c2931c0 2023-12-20T06:02:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c64c09e770 con 0x55c64c2dd070 2023-12-20T06:02:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0374309000 10 client.0 _ll_register_callbacks cb 0x55c64c27eb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:02:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f036ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd990 0x55c64c29f5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f036ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd990 0x55c64c29f5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42924/0 (socket says 172.21.15.32:42924) 2023-12-20T06:02:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f036ad9d640 1 -- 172.21.15.32:0/256185603 learned_addr learned my addr 172.21.15.32:0/256185603 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0363fff640 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd070 0x55c64c29f080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f036a59c640 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 0x55c64c298780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f036ad9d640 1 -- 172.21.15.32:0/256185603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd070 msgr2=0x55c64c29f080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:32.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f036ad9d640 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd070 0x55c64c29f080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f036ad9d640 1 -- 172.21.15.32:0/256185603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 msgr2=0x55c64c298780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f036ad9d640 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 0x55c64c298780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f036ad9d640 1 -- 172.21.15.32:0/256185603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c0cca80 con 0x55c64c2dd990 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f036a59c640 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 0x55c64c298780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.840+0000 7f0363fff640 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd070 0x55c64c29f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f036ad9d640 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd990 0x55c64c29f5c0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f0364000f90 tx=0x7f0364004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99432507295df36c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f03497fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f03497fa640 1 -- 172.21.15.32:0/256185603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f036400e040 con 0x55c64c2dd990 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 -- 172.21.15.32:0/256185603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c64c0b0c10 con 0x55c64c2dd990 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f03497fa640 1 -- 172.21.15.32:0/256185603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0364015b10 con 0x55c64c2dd990 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f03497fa640 1 -- 172.21.15.32:0/256185603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f0364015cd0 con 0x55c64c2dd990 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 10 client.10659 fetch_fsmap learned FSMap version 96 2023-12-20T06:02:32.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 -- 172.21.15.32:0/256185603 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x7f0364009e10 con 0x55c64c2dd990 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f03497fa640 1 -- 172.21.15.32:0/256185603 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c64c0b0c10 con 0x55c64c2dd990 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f03497fa640 1 -- 172.21.15.32:0/256185603 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 96) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f036402f020 con 0x55c64c2dd990 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 10 client.10659 fetch_fsmap finished waiting for FSMap version 96 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 10 client.10659 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 20 client.10659 populate_metadata read hostname 'smithi032' 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.32:6836/888710554,v1:172.21.15.32:6837/888710554] conn(0x55c64c2e48d0 0x55c64c2e4cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 4 client.10659 mds_command: new command op to 10539 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 -- 172.21.15.32:0/256185603 --> [v2:172.21.15.32:6836/888710554,v1:172.21.15.32:6837/888710554] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55c64c0dfa30 con 0x55c64c2e48d0 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f036a59c640 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.32:6836/888710554,v1:172.21.15.32:6837/888710554] conn(0x55c64c2e48d0 0x55c64c2e4cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f03497fa640 10 client.10659 ms_handle_connect on v2:172.21.15.32:6836/888710554 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f036a59c640 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.32:6836/888710554,v1:172.21.15.32:6837/888710554] conn(0x55c64c2e48d0 0x55c64c2e4cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f66a43319b339d8 server_cookie=affeb04f33da258d in_seq=0 out_seq=0 2023-12-20T06:02:32.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f03497fa640 1 -- 172.21.15.32:0/256185603 <== mds.0 v2:172.21.15.32:6836/888710554 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55c64c0dfa30 con 0x55c64c2e48d0 2023-12-20T06:02:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f03497fa640 10 client.10659 handle_command_reply: tid=0 2023-12-20T06:02:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 client.10659 shutdown 2023-12-20T06:02:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 -- 172.21.15.32:0/256185603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd990 msgr2=0x55c64c29f5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd990 0x55c64c29f5c0 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f0364000f90 tx=0x7f0364004ac0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 -- 172.21.15.32:0/256185603 shutdown_connections 2023-12-20T06:02:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.32:6836/888710554,v1:172.21.15.32:6837/888710554] conn(0x55c64c2e48d0 0x55c64c2e4cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd990 0x55c64c29f5c0 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd070 0x55c64c29f080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 --2- 172.21.15.32:0/256185603 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2931c0 0x55c64c298780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f03497fa640 0 client.10659 ms_handle_reset on v2:172.21.15.32:6836/888710554 2023-12-20T06:02:32.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 -- 172.21.15.32:0/256185603 >> 172.21.15.32:0/256185603 conn(0x55c64c299950 msgr2=0x55c64c291c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 -- 172.21.15.32:0/256185603 shutdown_connections 2023-12-20T06:02:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 1 -- 172.21.15.32:0/256185603 wait complete. 2023-12-20T06:02:32.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.844+0000 7f0374309000 20 client.10659 trim_cache size 0 max 16384 2023-12-20T06:02:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f0374309000 1 Processor -- start 2023-12-20T06:02:32.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f0374309000 1 -- start start 2023-12-20T06:02:32.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f0374309000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2931c0 0x55c64c3352f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f0374309000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd070 0x55c64c335830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f0374309000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2dd990 0x55c64c335d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f0374309000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64bd9e940 con 0x55c64c2931c0 2023-12-20T06:02:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f0374309000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64c1b1b30 con 0x55c64c2dd990 2023-12-20T06:02:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f0374309000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c64c0e50e0 con 0x55c64c2dd070 2023-12-20T06:02:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f036a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2931c0 0x55c64c3352f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f036a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2931c0 0x55c64c3352f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42936/0 (socket says 172.21.15.32:42936) 2023-12-20T06:02:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f036a59c640 1 -- 172.21.15.32:0/587711911 learned_addr learned my addr 172.21.15.32:0/587711911 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f036a59c640 1 -- 172.21.15.32:0/587711911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd070 msgr2=0x55c64c335830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f036a59c640 1 --2- 172.21.15.32:0/587711911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd070 0x55c64c335830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f036a59c640 1 -- 172.21.15.32:0/587711911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2dd990 msgr2=0x55c64c335d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f036a59c640 1 --2- 172.21.15.32:0/587711911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2dd990 0x55c64c335d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.860+0000 7f036a59c640 1 -- 172.21.15.32:0/587711911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55c64c0a6260 con 0x55c64c2931c0 2023-12-20T06:02:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f036a59c640 1 --2- 172.21.15.32:0/587711911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2931c0 0x55c64c3352f0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f035c05b620 tx=0x7f035c056040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13bf9f6cc431dec server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0360ff9640 1 -- 172.21.15.32:0/587711911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c0579a0 con 0x55c64c2931c0 2023-12-20T06:02:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0360ff9640 1 -- 172.21.15.32:0/587711911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f035c05bb50 con 0x55c64c2931c0 2023-12-20T06:02:32.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0360ff9640 1 -- 172.21.15.32:0/587711911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f035c050330 con 0x55c64c2931c0 2023-12-20T06:02:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 -- 172.21.15.32:0/587711911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2931c0 msgr2=0x55c64c3352f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 --2- 172.21.15.32:0/587711911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2931c0 0x55c64c3352f0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7f035c05b620 tx=0x7f035c056040 comp rx=0 tx=0).stop 2023-12-20T06:02:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 -- 172.21.15.32:0/587711911 shutdown_connections 2023-12-20T06:02:32.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 --2- 172.21.15.32:0/587711911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2dd990 0x55c64c335d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 --2- 172.21.15.32:0/587711911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2dd070 0x55c64c335830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 --2- 172.21.15.32:0/587711911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2931c0 0x55c64c3352f0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 -- 172.21.15.32:0/587711911 >> 172.21.15.32:0/587711911 conn(0x55c64c299950 msgr2=0x55c64c29ef40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 -- 172.21.15.32:0/587711911 shutdown_connections 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 -- 172.21.15.32:0/587711911 wait complete. 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 -- start start 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 Processor -- start 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2931c0 0x55c64c31dd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c31e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2dd990 0x55c64c31e7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64c28a9d0 con 0x55c64c2dd070 2023-12-20T06:02:32.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55c64c0de4e0 con 0x55c64c2dd990 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55c64c089860 con 0x55c64c2931c0 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0374309000 10 client.0 _ll_register_callbacks cb 0x55c64c08f640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0363fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c31e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0363fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c31e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42946/0 (socket says 172.21.15.32:42946) 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0363fff640 1 -- 172.21.15.32:0/720226858 learned_addr learned my addr 172.21.15.32:0/720226858 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f036a59c640 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2931c0 0x55c64c31dd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f036ad9d640 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2dd990 0x55c64c31e7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0363fff640 1 -- 172.21.15.32:0/720226858 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2931c0 msgr2=0x55c64c31dd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0363fff640 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2931c0 0x55c64c31dd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0363fff640 1 -- 172.21.15.32:0/720226858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2dd990 msgr2=0x55c64c31e7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0363fff640 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2dd990 0x55c64c31e7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.864+0000 7f0363fff640 1 -- 172.21.15.32:0/720226858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f036c15d870 con 0x55c64c2dd070 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0363fff640 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c31e2a0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f035401a140 tx=0x7f03540192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e8febe3db513317 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f03497fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f03497fa640 1 -- 172.21.15.32:0/720226858 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f03540cb070 con 0x55c64c2dd070 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f03497fa640 1 -- 172.21.15.32:0/720226858 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f03540b6070 con 0x55c64c2dd070 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 -- 172.21.15.32:0/720226858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55c64c0a6e00 con 0x55c64c2dd070 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f03497fa640 1 -- 172.21.15.32:0/720226858 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=96) v2 ==== 24+0+0 (secure 0 0 0) 0x7f03540c1050 con 0x55c64c2dd070 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 10 client.10665 fetch_fsmap learned FSMap version 96 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 -- 172.21.15.32:0/720226858 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=96,monmap=2+}) v3 -- 0x55c64c08c3b0 con 0x55c64c2dd070 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f03497fa640 1 -- 172.21.15.32:0/720226858 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55c64c0a6e00 con 0x55c64c2dd070 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f03497fa640 1 -- 172.21.15.32:0/720226858 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 96) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f03540bbca0 con 0x55c64c2dd070 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 10 client.10665 fetch_fsmap finished waiting for FSMap version 96 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 10 client.10665 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-20T06:02:32.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 20 client.10665 populate_metadata read hostname 'smithi032' 2023-12-20T06:02:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.32:6836/888710554,v1:172.21.15.32:6837/888710554] conn(0x55c64c31ed20 0x55c64c31f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 4 client.10665 mds_command: new command op to 10539 tid=0[{"prefix": "status"}] 2023-12-20T06:02:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 -- 172.21.15.32:0/720226858 --> [v2:172.21.15.32:6836/888710554,v1:172.21.15.32:6837/888710554] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55c64c0da160 con 0x55c64c31ed20 2023-12-20T06:02:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f036a59c640 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.32:6836/888710554,v1:172.21.15.32:6837/888710554] conn(0x55c64c31ed20 0x55c64c31f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f036a59c640 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.32:6836/888710554,v1:172.21.15.32:6837/888710554] conn(0x55c64c31ed20 0x55c64c31f100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b49f36db2f1af46a server_cookie=7dabca5f3c392742 in_seq=0 out_seq=0 2023-12-20T06:02:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f03497fa640 10 client.10665 ms_handle_connect on v2:172.21.15.32:6836/888710554 2023-12-20T06:02:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f03497fa640 1 -- 172.21.15.32:0/720226858 <== mds.0 v2:172.21.15.32:6836/888710554 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+334 (crc 0 0 0) 0x55c64c0da160 con 0x55c64c31ed20 2023-12-20T06:02:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f03497fa640 10 client.10665 handle_command_reply: tid=0 2023-12-20T06:02:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 client.10665 shutdown 2023-12-20T06:02:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 -- 172.21.15.32:0/720226858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 msgr2=0x55c64c31e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c31e2a0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7f035401a140 tx=0x7f03540192b0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 -- 172.21.15.32:0/720226858 shutdown_connections 2023-12-20T06:02:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.32:6836/888710554,v1:172.21.15.32:6837/888710554] conn(0x55c64c31ed20 0x55c64c31f100 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55c64c2dd990 0x55c64c31e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55c64c2dd070 0x55c64c31e2a0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f03497fa640 0 client.10665 ms_handle_reset on v2:172.21.15.32:6836/888710554 2023-12-20T06:02:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 --2- 172.21.15.32:0/720226858 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55c64c2931c0 0x55c64c31dd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 -- 172.21.15.32:0/720226858 >> 172.21.15.32:0/720226858 conn(0x55c64c299950 msgr2=0x55c64c2da060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 -- 172.21.15.32:0/720226858 shutdown_connections 2023-12-20T06:02:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 1 -- 172.21.15.32:0/720226858 wait complete. 2023-12-20T06:02:32.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0374309000 20 client.10665 trim_cache size 0 max 16384 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 10539, 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9242460799999996, 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 96, 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 209, 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 209, 2023-12-20T06:02:32.873 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.675071261999999 2023-12-20T06:02:32.874 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:02:32.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 -- 172.21.15.32:0/3360589666 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f034c065b80 msgr2=0x7f034c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f034c065b80 0x7f034c068040 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f03540ae000 tx=0x7f03540b3040 comp rx=0 tx=0).stop 2023-12-20T06:02:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 -- 172.21.15.32:0/3360589666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c06cc10 msgr2=0x7f036c155950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c06cc10 0x7f036c155950 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f035c00c370 tx=0x7f035c00c830 comp rx=0 tx=0).stop 2023-12-20T06:02:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 -- 172.21.15.32:0/3360589666 shutdown_connections 2023-12-20T06:02:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f034c065b80 0x7f034c068040 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f036c15adc0 0x7f036c0c7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f036c0707b0 0x7f036c0c6bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 --2- 172.21.15.32:0/3360589666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f036c06cc10 0x7f036c155950 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 -- 172.21.15.32:0/3360589666 >> 172.21.15.32:0/3360589666 conn(0x7f036c0ce650 msgr2=0x7f036c070bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 -- 172.21.15.32:0/3360589666 shutdown_connections 2023-12-20T06:02:32.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:32.868+0000 7f0370fa6640 1 -- 172.21.15.32:0/3360589666 wait complete. 2023-12-20T06:02:32.886 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T06:02:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.036+0000 7f42ccf98640 1 Processor -- start 2023-12-20T06:02:33.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.036+0000 7f42ccf98640 1 -- start start 2023-12-20T06:02:33.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42ccf98640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42c80af5e0 0x7f42c80af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:33.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42ccf98640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f42c806ce60 0x7f42c815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42ccf98640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42c815f110 0x7f42c815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42ccf98640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c815bbd0 con 0x7f42c80af5e0 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42ccf98640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42c815beb0 con 0x7f42c806ce60 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42ccf98640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c815e910 con 0x7f42c815f110 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42c659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42c80af5e0 0x7f42c80af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42c659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f42c80af5e0 0x7f42c80af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42954/0 (socket says 172.21.15.32:42954) 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42c659c640 1 -- 172.21.15.32:0/2724515429 learned_addr learned my addr 172.21.15.32:0/2724515429 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42c659c640 1 -- 172.21.15.32:0/2724515429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42c815f110 msgr2=0x7f42c815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42c659c640 1 --2- 172.21.15.32:0/2724515429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42c815f110 0x7f42c815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42c659c640 1 -- 172.21.15.32:0/2724515429 >> v2:172.21.15.39:3300/0 conn(0x7f42c806ce60 msgr2=0x7f42c815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42bffff640 1 --2- 172.21.15.32:0/2724515429 >> v2:172.21.15.39:3300/0 conn(0x7f42c806ce60 0x7f42c815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:33.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42c659c640 1 --2- 172.21.15.32:0/2724515429 >> v2:172.21.15.39:3300/0 conn(0x7f42c806ce60 0x7f42c815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42c659c640 1 -- 172.21.15.32:0/2724515429 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42c815ebf0 con 0x7f42c80af5e0 2023-12-20T06:02:33.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42bffff640 1 --2- 172.21.15.32:0/2724515429 >> v2:172.21.15.39:3300/0 conn(0x7f42c806ce60 0x7f42c815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.040+0000 7f42c659c640 1 --2- 172.21.15.32:0/2724515429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42c80af5e0 0x7f42c80af9c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f42c806abe0 tx=0x7f42b801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c820ef0cb3e038f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:33.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42c5d9b640 1 -- 172.21.15.32:0/2724515429 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b80ab620 con 0x7f42c80af5e0 2023-12-20T06:02:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42c5d9b640 1 -- 172.21.15.32:0/2724515429 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b80ab780 con 0x7f42c80af5e0 2023-12-20T06:02:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42c5d9b640 1 -- 172.21.15.32:0/2724515429 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b80abb10 con 0x7f42c80af5e0 2023-12-20T06:02:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 -- 172.21.15.32:0/2724515429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42c80af5e0 msgr2=0x7f42c80af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 --2- 172.21.15.32:0/2724515429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42c80af5e0 0x7f42c80af9c0 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7f42c806abe0 tx=0x7f42b801c500 comp rx=0 tx=0).stop 2023-12-20T06:02:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 -- 172.21.15.32:0/2724515429 shutdown_connections 2023-12-20T06:02:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 --2- 172.21.15.32:0/2724515429 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42c815f110 0x7f42c815c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 --2- 172.21.15.32:0/2724515429 >> v2:172.21.15.39:3300/0 conn(0x7f42c806ce60 0x7f42c815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 --2- 172.21.15.32:0/2724515429 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42c80af5e0 0x7f42c80af9c0 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 -- 172.21.15.32:0/2724515429 >> 172.21.15.32:0/2724515429 conn(0x7f42c806a320 msgr2=0x7f42c80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 -- 172.21.15.32:0/2724515429 shutdown_connections 2023-12-20T06:02:33.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 -- 172.21.15.32:0/2724515429 wait complete. 2023-12-20T06:02:33.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 Processor -- start 2023-12-20T06:02:33.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 -- start start 2023-12-20T06:02:33.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42c806ce60 0x7f42c8145b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42c80af5e0 0x7f42c815a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42c815f110 0x7f42c8152d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c80cdc50 con 0x7f42c806ce60 2023-12-20T06:02:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42c81595a0 con 0x7f42c815f110 2023-12-20T06:02:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42ccf98640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42c8159880 con 0x7f42c80af5e0 2023-12-20T06:02:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42c659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42c806ce60 0x7f42c8145b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42c659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42c806ce60 0x7f42c8145b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42962/0 (socket says 172.21.15.32:42962) 2023-12-20T06:02:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.044+0000 7f42c659c640 1 -- 172.21.15.32:0/999007289 learned_addr learned my addr 172.21.15.32:0/999007289 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42bffff640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42c80af5e0 0x7f42c815a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:33.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42c6d9d640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42c815f110 0x7f42c8152d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42c659c640 1 -- 172.21.15.32:0/999007289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42c80af5e0 msgr2=0x7f42c815a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42c659c640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42c80af5e0 0x7f42c815a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42c659c640 1 -- 172.21.15.32:0/999007289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42c815f110 msgr2=0x7f42c8152d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42c659c640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42c815f110 0x7f42c8152d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42c659c640 1 -- 172.21.15.32:0/999007289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42b8009480 con 0x7f42c806ce60 2023-12-20T06:02:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42c6d9d640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42c815f110 0x7f42c8152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42c659c640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42c806ce60 0x7f42c8145b60 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f42b801c330 tx=0x7f42b80192a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42beffd640 1 -- 172.21.15.32:0/999007289 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b80059a0 con 0x7f42c806ce60 2023-12-20T06:02:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42beffd640 1 -- 172.21.15.32:0/999007289 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42b8005b20 con 0x7f42c806ce60 2023-12-20T06:02:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42ccf98640 1 -- 172.21.15.32:0/999007289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42c8159b60 con 0x7f42c806ce60 2023-12-20T06:02:33.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42beffd640 1 -- 172.21.15.32:0/999007289 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42b80ab8f0 con 0x7f42c806ce60 2023-12-20T06:02:33.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42ccf98640 1 -- 172.21.15.32:0/999007289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42c80cd870 con 0x7f42c806ce60 2023-12-20T06:02:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42ccf98640 1 -- 172.21.15.32:0/999007289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42c804e000 con 0x7f42c806ce60 2023-12-20T06:02:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42beffd640 1 -- 172.21.15.32:0/999007289 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f42b80aba50 con 0x7f42c806ce60 2023-12-20T06:02:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42beffd640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42a8065b80 0x7f42a8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.048+0000 7f42beffd640 1 -- 172.21.15.32:0/999007289 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f42b811cfa0 con 0x7f42c806ce60 2023-12-20T06:02:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.052+0000 7f42bffff640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42a8065b80 0x7f42a8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:33.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.052+0000 7f42bffff640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42a8065b80 0x7f42a8068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f42c8151f70 tx=0x7f42b0008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:33.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.052+0000 7f42beffd640 1 -- 172.21.15.32:0/999007289 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f42b80ea670 con 0x7f42c806ce60 2023-12-20T06:02:33.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 -- 172.21.15.32:0/999007289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f42c80cf450 con 0x7f42c806ce60 2023-12-20T06:02:33.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42beffd640 1 -- 172.21.15.32:0/999007289 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v96) v1 ==== 53+0+73 (secure 0 0 0) 0x7f42b80edc90 con 0x7f42c806ce60 2023-12-20T06:02:33.242 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T06:02:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 -- 172.21.15.32:0/999007289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42a8065b80 msgr2=0x7f42a8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42a8065b80 0x7f42a8068040 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f42c8151f70 tx=0x7f42b0008040 comp rx=0 tx=0).stop 2023-12-20T06:02:33.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 -- 172.21.15.32:0/999007289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42c806ce60 msgr2=0x7f42c8145b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:33.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42c806ce60 0x7f42c8145b60 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7f42b801c330 tx=0x7f42b80192a0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 -- 172.21.15.32:0/999007289 shutdown_connections 2023-12-20T06:02:33.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42a8065b80 0x7f42a8068040 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42c815f110 0x7f42c8152d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42c80af5e0 0x7f42c815a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 --2- 172.21.15.32:0/999007289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42c806ce60 0x7f42c8145b60 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:33.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 -- 172.21.15.32:0/999007289 >> 172.21.15.32:0/999007289 conn(0x7f42c806a320 msgr2=0x7f42c8156490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:33.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 -- 172.21.15.32:0/999007289 shutdown_connections 2023-12-20T06:02:33.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:33.244+0000 7f42ccf98640 1 -- 172.21.15.32:0/999007289 wait complete. 2023-12-20T06:02:33.264 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T06:02:33.264 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T06:02:33.264 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T06:02:33.264 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T06:02:33.264 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T06:02:33.264 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T06:02:33.264 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T06:02:33.264 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T06:02:33.264 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:02:33.265 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:33.265 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T06:02:33.275 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T06:02:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T06:02:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T06:02:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:02:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T06:02:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:02:33.276 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:02:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T06:02:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T06:02:33.276 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T06:02:33.276 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:02:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T06:02:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T06:02:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 36707sec preferred_lft 36707sec 2023-12-20T06:02:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T06:02:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:02:33.277 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:02:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T06:02:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T06:02:33.277 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:02:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:02:33.277 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T06:02:33.278 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T06:02:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T06:02:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:02:33.279 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T06:02:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T06:02:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T06:02:33.279 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:02:33.279 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:33.279 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T06:02:33.343 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T06:02:33.346 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:33.346 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T06:02:33.398 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T06:02:33.398 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T06:02:33.398 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T06:02:33.399 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T06:02:33.399 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T06:02:33.399 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T06:02:33.399 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:33.399 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T06:02:33.399 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T06:02:33.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T06:02:33.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T06:02:33.399 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T06:02:33.400 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T06:02:33.498 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:33.498 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T06:02:33.508 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T06:02:33.508 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T06:02:33.509 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:02:33.509 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:33.509 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T06:02:33.573 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T06:02:33.573 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T06:02:33.629 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:02:33.629 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T06:02:33.629 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T06:02:33.681 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T06:02:33.681 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:33.681 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:02:33.776 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:02:33.776 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T06:02:33.790 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:02:33.790 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T06:02:33.790 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T06:02:33.790 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T06:02:33.790 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T06:02:33.794 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T06:02:33.794 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:33.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T06:02:33.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:33.816 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T06:02:33.878 INFO:teuthology.orchestra.run.smithi032.stdout:client10276 2023-12-20T06:02:33.878 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:02:33.878 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:33.929 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-12-20T06:02:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.120+0000 7ffa24339640 1 Processor -- start 2023-12-20T06:02:34.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.120+0000 7ffa24339640 1 -- start start 2023-12-20T06:02:34.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa24339640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ffa1c159250 0x7ffa1c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa24339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c070940 0x7ffa1c0cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa24339640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa1c159630 con 0x7ffa1c0ce290 2023-12-20T06:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa24339640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa1c0c9480 con 0x7ffa1c159250 2023-12-20T06:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa24339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c06cf90 con 0x7ffa1c070940 2023-12-20T06:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa228d6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c070940 0x7ffa1c0cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa220d5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ffa1c159250 0x7ffa1c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa228d6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c070940 0x7ffa1c0cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:47018/0 (socket says 172.21.15.32:47018) 2023-12-20T06:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa228d6640 1 -- 172.21.15.32:0/881332359 learned_addr learned my addr 172.21.15.32:0/881332359 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:34.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa218d4640 1 --1- 172.21.15.32:0/881332359 >> v1:172.21.15.39:6789/0 conn(0x7ffa1c0ce290 0x7ffa1c070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:57862/0 (socket says 172.21.15.32:57862) 2023-12-20T06:02:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa228d6640 1 -- 172.21.15.32:0/881332359 >> v2:172.21.15.39:3300/0 conn(0x7ffa1c159250 msgr2=0x7ffa1c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa228d6640 1 --2- 172.21.15.32:0/881332359 >> v2:172.21.15.39:3300/0 conn(0x7ffa1c159250 0x7ffa1c0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa228d6640 1 -- 172.21.15.32:0/881332359 >> v1:172.21.15.39:6789/0 conn(0x7ffa1c0ce290 legacy=0x7ffa1c070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa228d6640 1 -- 172.21.15.32:0/881332359 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa1c06d270 con 0x7ffa1c070940 2023-12-20T06:02:34.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.124+0000 7ffa220d5640 1 --2- 172.21.15.32:0/881332359 >> v2:172.21.15.39:3300/0 conn(0x7ffa1c159250 0x7ffa1c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa228d6640 1 --2- 172.21.15.32:0/881332359 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c070940 0x7ffa1c0cc5b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400d150 tx=0x7ffa1401ebe0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5527c3c1f6ac3a06 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa210d3640 1 -- 172.21.15.32:0/881332359 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa140a7c30 con 0x7ffa1c070940 2023-12-20T06:02:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa210d3640 1 -- 172.21.15.32:0/881332359 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa140ab070 con 0x7ffa1c070940 2023-12-20T06:02:34.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa210d3640 1 -- 172.21.15.32:0/881332359 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa140b1400 con 0x7ffa1c070940 2023-12-20T06:02:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa24339640 1 -- 172.21.15.32:0/881332359 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c070940 msgr2=0x7ffa1c0cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa24339640 1 --2- 172.21.15.32:0/881332359 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c070940 0x7ffa1c0cc5b0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ffa1400d150 tx=0x7ffa1401ebe0 comp rx=0 tx=0).stop 2023-12-20T06:02:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa24339640 1 -- 172.21.15.32:0/881332359 shutdown_connections 2023-12-20T06:02:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa24339640 1 --2- 172.21.15.32:0/881332359 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c070940 0x7ffa1c0cc5b0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:34.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa24339640 1 --2- 172.21.15.32:0/881332359 >> v2:172.21.15.39:3300/0 conn(0x7ffa1c159250 0x7ffa1c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:34.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa24339640 1 -- 172.21.15.32:0/881332359 >> 172.21.15.32:0/881332359 conn(0x7ffa1c06a510 msgr2=0x7ffa1c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:34.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa24339640 1 -- 172.21.15.32:0/881332359 shutdown_connections 2023-12-20T06:02:34.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa24339640 1 -- 172.21.15.32:0/881332359 wait complete. 2023-12-20T06:02:34.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa24339640 1 Processor -- start 2023-12-20T06:02:34.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.128+0000 7ffa24339640 1 -- start start 2023-12-20T06:02:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa24339640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa1c070940 0x7ffa1c0b1400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:34.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa24339640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa1c0ce290 0x7ffa1c14ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa24339640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c159250 0x7ffa1c151000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa24339640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c0b2660 con 0x7ffa1c0ce290 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa24339640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c14d3d0 con 0x7ffa1c070940 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa24339640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa1c0b19e0 con 0x7ffa1c159250 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa218d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa1c0ce290 0x7ffa1c14ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa218d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa1c0ce290 0x7ffa1c14ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42974/0 (socket says 172.21.15.32:42974) 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa218d4640 1 -- 172.21.15.32:0/1131832277 learned_addr learned my addr 172.21.15.32:0/1131832277 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa228d6640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c159250 0x7ffa1c151000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa220d5640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa1c070940 0x7ffa1c0b1400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa218d4640 1 -- 172.21.15.32:0/1131832277 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c159250 msgr2=0x7ffa1c151000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa218d4640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c159250 0x7ffa1c151000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa218d4640 1 -- 172.21.15.32:0/1131832277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa1c070940 msgr2=0x7ffa1c0b1400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa218d4640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa1c070940 0x7ffa1c0b1400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa218d4640 1 -- 172.21.15.32:0/1131832277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa18009420 con 0x7ffa1c0ce290 2023-12-20T06:02:34.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa218d4640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa1c0ce290 0x7ffa1c14ce90 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c500 tx=0x7ffa0c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa12ffd640 1 -- 172.21.15.32:0/1131832277 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c007500 con 0x7ffa1c0ce290 2023-12-20T06:02:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa12ffd640 1 -- 172.21.15.32:0/1131832277 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffa0c007660 con 0x7ffa1c0ce290 2023-12-20T06:02:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa12ffd640 1 -- 172.21.15.32:0/1131832277 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffa0c004a10 con 0x7ffa1c0ce290 2023-12-20T06:02:34.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa24339640 1 -- 172.21.15.32:0/1131832277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa1400bd40 con 0x7ffa1c0ce290 2023-12-20T06:02:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa24339640 1 -- 172.21.15.32:0/1131832277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa1c0b1fa0 con 0x7ffa1c0ce290 2023-12-20T06:02:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa24339640 1 -- 172.21.15.32:0/1131832277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa1c0caac0 con 0x7ffa1c0ce290 2023-12-20T06:02:34.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.132+0000 7ffa12ffd640 1 -- 172.21.15.32:0/1131832277 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ffa0c004b70 con 0x7ffa1c0ce290 2023-12-20T06:02:34.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.136+0000 7ffa12ffd640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffa00065b80 0x7ffa00068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:34.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.136+0000 7ffa12ffd640 1 -- 172.21.15.32:0/1131832277 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ffa0c082530 con 0x7ffa1c0ce290 2023-12-20T06:02:34.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.136+0000 7ffa220d5640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffa00065b80 0x7ffa00068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:34.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.136+0000 7ffa220d5640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffa00065b80 0x7ffa00068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009be0 tx=0x7ffa18008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:34.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.140+0000 7ffa12ffd640 1 -- 172.21.15.32:0/1131832277 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffa0c04fe40 con 0x7ffa1c0ce290 2023-12-20T06:02:34.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.328+0000 7ffa24339640 1 -- 172.21.15.32:0/1131832277 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7ffa1c06d820 con 0x7ffa1c0ce290 2023-12-20T06:02:34.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:34.996+0000 7ffa12ffd640 1 -- 172.21.15.32:0/1131832277 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v97) v1 ==== 136+0+0 (secure 0 0 0) 0x7ffa0c053460 con 0x7ffa1c0ce290 2023-12-20T06:02:34.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 -- 172.21.15.32:0/1131832277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffa00065b80 msgr2=0x7ffa00068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:34.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffa00065b80 0x7ffa00068040 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7ffa18009be0 tx=0x7ffa18008040 comp rx=0 tx=0).stop 2023-12-20T06:02:34.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 -- 172.21.15.32:0/1131832277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa1c0ce290 msgr2=0x7ffa1c14ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:34.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa1c0ce290 0x7ffa1c14ce90 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7ffa0c00c500 tx=0x7ffa0c00c9c0 comp rx=0 tx=0).stop 2023-12-20T06:02:34.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 -- 172.21.15.32:0/1131832277 shutdown_connections 2023-12-20T06:02:34.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffa00065b80 0x7ffa00068040 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa1c159250 0x7ffa1c151000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa1c0ce290 0x7ffa1c14ce90 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 --2- 172.21.15.32:0/1131832277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa1c070940 0x7ffa1c0b1400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 -- 172.21.15.32:0/1131832277 >> 172.21.15.32:0/1131832277 conn(0x7ffa1c06a510 msgr2=0x7ffa1c06c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 -- 172.21.15.32:0/1131832277 shutdown_connections 2023-12-20T06:02:34.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.000+0000 7ffa24339640 1 -- 172.21.15.32:0/1131832277 wait complete. 2023-12-20T06:02:35.020 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd7454e5640 1 Processor -- start 2023-12-20T06:02:35.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd7454e5640 1 -- start start 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd7454e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd74015cd70 0x7fd7400cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd7454e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd7400b23d0 0x7fd7400b2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd7454e5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7400b2d70 con 0x7fd7401608e0 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd7454e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7401618b0 con 0x7fd7400b23d0 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd7454e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7400ce6c0 con 0x7fd74015cd70 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd737fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd7400b23d0 0x7fd7400b2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd737fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd7400b23d0 0x7fd7400b2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:42988/0 (socket says 172.21.15.32:42988) 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd737fff640 1 -- 172.21.15.32:0/4062591703 learned_addr learned my addr 172.21.15.32:0/4062591703 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd73effd640 1 --2- 172.21.15.32:0/4062591703 >> v2:172.21.15.39:3300/0 conn(0x7fd74015cd70 0x7fd7400cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd737fff640 1 -- 172.21.15.32:0/4062591703 >> v2:172.21.15.39:3300/0 conn(0x7fd74015cd70 msgr2=0x7fd7400cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd737fff640 1 --2- 172.21.15.32:0/4062591703 >> v2:172.21.15.39:3300/0 conn(0x7fd74015cd70 0x7fd7400cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd737fff640 1 -- 172.21.15.32:0/4062591703 >> v1:172.21.15.32:6789/0 conn(0x7fd7401608e0 legacy=0x7fd740160d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd73e7fc640 1 -- 172.21.15.32:0/4062591703 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1079078191 0 0) 0x7fd7400b2d70 con 0x7fd7401608e0 2023-12-20T06:02:35.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd737fff640 1 -- 172.21.15.32:0/4062591703 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7401615d0 con 0x7fd7400b23d0 2023-12-20T06:02:35.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.168+0000 7fd73effd640 1 --2- 172.21.15.32:0/4062591703 >> v2:172.21.15.39:3300/0 conn(0x7fd74015cd70 0x7fd7400cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd737fff640 1 --2- 172.21.15.32:0/4062591703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7400b23d0 0x7fd7400b2830 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fd74006f450 tx=0x7fd72801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88dc50824ce42381 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:35.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd73e7fc640 1 -- 172.21.15.32:0/4062591703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7280aa430 con 0x7fd7400b23d0 2023-12-20T06:02:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd73e7fc640 1 -- 172.21.15.32:0/4062591703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7280aa590 con 0x7fd7400b23d0 2023-12-20T06:02:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd73e7fc640 1 -- 172.21.15.32:0/4062591703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7280aa920 con 0x7fd7400b23d0 2023-12-20T06:02:35.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd7454e5640 1 -- 172.21.15.32:0/4062591703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7400b23d0 msgr2=0x7fd7400b2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd7454e5640 1 --2- 172.21.15.32:0/4062591703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7400b23d0 0x7fd7400b2830 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7fd74006f450 tx=0x7fd72801c4d0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd7454e5640 1 -- 172.21.15.32:0/4062591703 shutdown_connections 2023-12-20T06:02:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd7454e5640 1 --2- 172.21.15.32:0/4062591703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7400b23d0 0x7fd7400b2830 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd7454e5640 1 --2- 172.21.15.32:0/4062591703 >> v2:172.21.15.39:3300/0 conn(0x7fd74015cd70 0x7fd7400cf160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd7454e5640 1 -- 172.21.15.32:0/4062591703 >> 172.21.15.32:0/4062591703 conn(0x7fd7400cd3d0 msgr2=0x7fd7400cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd7454e5640 1 -- 172.21.15.32:0/4062591703 shutdown_connections 2023-12-20T06:02:35.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.172+0000 7fd7454e5640 1 -- 172.21.15.32:0/4062591703 wait complete. 2023-12-20T06:02:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7454e5640 1 Processor -- start 2023-12-20T06:02:35.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7454e5640 1 -- start start 2023-12-20T06:02:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7454e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7400b23d0 0x7fd740157370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7454e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd74015cd70 0x7fd7400c6e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7454e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7401608e0 0x7fd74015f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7454e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400b4ab0 con 0x7fd7400b23d0 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7454e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400c5a40 con 0x7fd74015cd70 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7454e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7400c5230 con 0x7fd7401608e0 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd73effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7400b23d0 0x7fd740157370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd73effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7400b23d0 0x7fd740157370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43000/0 (socket says 172.21.15.32:43000) 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd73effd640 1 -- 172.21.15.32:0/2964655285 learned_addr learned my addr 172.21.15.32:0/2964655285 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd737fff640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd74015cd70 0x7fd7400c6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd73f7fe640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7401608e0 0x7fd74015f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd73effd640 1 -- 172.21.15.32:0/2964655285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7401608e0 msgr2=0x7fd74015f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd73effd640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7401608e0 0x7fd74015f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd73effd640 1 -- 172.21.15.32:0/2964655285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd74015cd70 msgr2=0x7fd7400c6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd73effd640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd74015cd70 0x7fd7400c6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd73effd640 1 -- 172.21.15.32:0/2964655285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7280094d0 con 0x7fd7400b23d0 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd73f7fe640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7401608e0 0x7fd74015f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:35.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd73effd640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7400b23d0 0x7fd740157370 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c7c0 tx=0x7fd73000cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7377fe640 1 -- 172.21.15.32:0/2964655285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7300079d0 con 0x7fd7400b23d0 2023-12-20T06:02:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7377fe640 1 -- 172.21.15.32:0/2964655285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd730007b30 con 0x7fd7400b23d0 2023-12-20T06:02:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7377fe640 1 -- 172.21.15.32:0/2964655285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd73000ce70 con 0x7fd7400b23d0 2023-12-20T06:02:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7454e5640 1 -- 172.21.15.32:0/2964655285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7400c5480 con 0x7fd7400b23d0 2023-12-20T06:02:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.176+0000 7fd7454e5640 1 -- 172.21.15.32:0/2964655285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7400b46d0 con 0x7fd7400b23d0 2023-12-20T06:02:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.180+0000 7fd7454e5640 1 -- 172.21.15.32:0/2964655285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd74004e000 con 0x7fd7400b23d0 2023-12-20T06:02:35.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.180+0000 7fd7377fe640 1 -- 172.21.15.32:0/2964655285 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd7300113f0 con 0x7fd7400b23d0 2023-12-20T06:02:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.180+0000 7fd7377fe640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd720065b80 0x7fd720068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.180+0000 7fd7377fe640 1 -- 172.21.15.32:0/2964655285 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fd7300838f0 con 0x7fd7400b23d0 2023-12-20T06:02:35.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.180+0000 7fd737fff640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd720065b80 0x7fd720068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.180+0000 7fd737fff640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd720065b80 0x7fd720068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd7280ae000 tx=0x7fd7280a7000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:35.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.184+0000 7fd7377fe640 1 -- 172.21.15.32:0/2964655285 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd730051060 con 0x7fd7400b23d0 2023-12-20T06:02:35.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.344+0000 7fd7454e5640 1 -- 172.21.15.32:0/2964655285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd7400cf1f0 con 0x7fd7400b23d0 2023-12-20T06:02:35.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.344+0000 7fd7377fe640 1 -- 172.21.15.32:0/2964655285 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v209) v1 ==== 74+0+16153 (secure 0 0 0) 0x7fd730054680 con 0x7fd7400b23d0 2023-12-20T06:02:35.344 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:35.344 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":209,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:27.945804+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":25,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:02:25.045126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-20T06:02:25.938205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":207,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:35.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 -- 172.21.15.32:0/2964655285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd720065b80 msgr2=0x7fd720068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd720065b80 0x7fd720068040 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd7280ae000 tx=0x7fd7280a7000 comp rx=0 tx=0).stop 2023-12-20T06:02:35.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 -- 172.21.15.32:0/2964655285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7400b23d0 msgr2=0x7fd740157370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7400b23d0 0x7fd740157370 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7fd73000c7c0 tx=0x7fd73000cc80 comp rx=0 tx=0).stop 2023-12-20T06:02:35.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 -- 172.21.15.32:0/2964655285 shutdown_connections 2023-12-20T06:02:35.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd720065b80 0x7fd720068040 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7401608e0 0x7fd74015f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd74015cd70 0x7fd7400c6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 --2- 172.21.15.32:0/2964655285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7400b23d0 0x7fd740157370 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 -- 172.21.15.32:0/2964655285 >> 172.21.15.32:0/2964655285 conn(0x7fd7400cd3d0 msgr2=0x7fd74015c050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:35.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 -- 172.21.15.32:0/2964655285 shutdown_connections 2023-12-20T06:02:35.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.348+0000 7fd7454e5640 1 -- 172.21.15.32:0/2964655285 wait complete. 2023-12-20T06:02:35.366 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T06:02:25.045126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '208', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T06:02:25.938205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '209', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:02:35.367 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:02:35.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e1a9bc640 1 Processor -- start 2023-12-20T06:02:35.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e1a9bc640 1 -- start start 2023-12-20T06:02:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e1a9bc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e14157480 0x7f4e140c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e1a9bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4e140cd4c0 0x7f4e140b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e1a9bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e140b1100 con 0x7f4e140b0760 2023-12-20T06:02:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e1a9bc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e140cde20 con 0x7f4e14157480 2023-12-20T06:02:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e1a9bc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e140c8e30 con 0x7f4e140cd4c0 2023-12-20T06:02:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e13fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e14157480 0x7f4e140c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e13fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e14157480 0x7f4e140c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43016/0 (socket says 172.21.15.32:43016) 2023-12-20T06:02:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e13fff640 1 -- 172.21.15.32:0/2113560645 learned_addr learned my addr 172.21.15.32:0/2113560645 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:35.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e18f59640 1 --2- 172.21.15.32:0/2113560645 >> v2:172.21.15.39:3300/0 conn(0x7f4e140cd4c0 0x7f4e140b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e13fff640 1 -- 172.21.15.32:0/2113560645 >> v2:172.21.15.39:3300/0 conn(0x7f4e140cd4c0 msgr2=0x7f4e140b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e13fff640 1 --2- 172.21.15.32:0/2113560645 >> v2:172.21.15.39:3300/0 conn(0x7f4e140cd4c0 0x7f4e140b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e13fff640 1 -- 172.21.15.32:0/2113560645 >> v1:172.21.15.39:6789/0 conn(0x7f4e140b0760 legacy=0x7f4e1415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:35.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.516+0000 7f4e13fff640 1 -- 172.21.15.32:0/2113560645 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e140ce070 con 0x7f4e14157480 2023-12-20T06:02:35.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e13fff640 1 --2- 172.21.15.32:0/2113560645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e14157480 0x7f4e140c9840 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f4e14071420 tx=0x7f4e0401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46a8321afe439be1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e12ffd640 1 -- 172.21.15.32:0/2113560645 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e040ad660 con 0x7f4e14157480 2023-12-20T06:02:35.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e12ffd640 1 -- 172.21.15.32:0/2113560645 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e040ad7c0 con 0x7f4e14157480 2023-12-20T06:02:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e12ffd640 1 -- 172.21.15.32:0/2113560645 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e040adb50 con 0x7f4e14157480 2023-12-20T06:02:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/2113560645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e14157480 msgr2=0x7f4e140c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e1a9bc640 1 --2- 172.21.15.32:0/2113560645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e14157480 0x7f4e140c9840 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f4e14071420 tx=0x7f4e0401c440 comp rx=0 tx=0).stop 2023-12-20T06:02:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/2113560645 shutdown_connections 2023-12-20T06:02:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e1a9bc640 1 --2- 172.21.15.32:0/2113560645 >> v2:172.21.15.39:3300/0 conn(0x7f4e140cd4c0 0x7f4e140b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e1a9bc640 1 --2- 172.21.15.32:0/2113560645 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e14157480 0x7f4e140c9840 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/2113560645 >> 172.21.15.32:0/2113560645 conn(0x7f4e1406a510 msgr2=0x7f4e140ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/2113560645 shutdown_connections 2023-12-20T06:02:35.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.520+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/2113560645 wait complete. 2023-12-20T06:02:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e1a9bc640 1 Processor -- start 2023-12-20T06:02:35.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e1a9bc640 1 -- start start 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e1a9bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e140b0760 0x7f4e14151f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e1a9bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e140cd4c0 0x7f4e1414fd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e1a9bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e14157480 0x7f4e14150de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e1a9bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14153370 con 0x7f4e140cd4c0 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e1a9bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e14152450 con 0x7f4e14157480 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e1a9bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e141526f0 con 0x7f4e140b0760 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e137fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e140cd4c0 0x7f4e1414fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e137fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e140cd4c0 0x7f4e1414fd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43026/0 (socket says 172.21.15.32:43026) 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e137fe640 1 -- 172.21.15.32:0/1358834459 learned_addr learned my addr 172.21.15.32:0/1358834459 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e18f59640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e14157480 0x7f4e14150de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e13fff640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e140b0760 0x7f4e14151f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e137fe640 1 -- 172.21.15.32:0/1358834459 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e140b0760 msgr2=0x7f4e14151f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e137fe640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e140b0760 0x7f4e14151f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e137fe640 1 -- 172.21.15.32:0/1358834459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e14157480 msgr2=0x7f4e14150de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e137fe640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e14157480 0x7f4e14150de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e137fe640 1 -- 172.21.15.32:0/1358834459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e040094d0 con 0x7f4e140cd4c0 2023-12-20T06:02:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e137fe640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e140cd4c0 0x7f4e1414fd10 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800c3c0 tx=0x7f4e0800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e10ff9640 1 -- 172.21.15.32:0/1358834459 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e080052d0 con 0x7f4e140cd4c0 2023-12-20T06:02:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e10ff9640 1 -- 172.21.15.32:0/1358834459 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e08005430 con 0x7f4e140cd4c0 2023-12-20T06:02:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/1358834459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e141529d0 con 0x7f4e140cd4c0 2023-12-20T06:02:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e10ff9640 1 -- 172.21.15.32:0/1358834459 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e08003840 con 0x7f4e140cd4c0 2023-12-20T06:02:35.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.524+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/1358834459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e14152f90 con 0x7f4e140cd4c0 2023-12-20T06:02:35.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.528+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/1358834459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e141534b0 con 0x7f4e140cd4c0 2023-12-20T06:02:35.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.528+0000 7f4e10ff9640 1 -- 172.21.15.32:0/1358834459 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4e080039a0 con 0x7f4e140cd4c0 2023-12-20T06:02:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.528+0000 7f4e10ff9640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e00065b80 0x7f4e00068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.528+0000 7f4e10ff9640 1 -- 172.21.15.32:0/1358834459 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f4e08082800 con 0x7f4e140cd4c0 2023-12-20T06:02:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.528+0000 7f4e13fff640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e00065b80 0x7f4e00068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.528+0000 7f4e13fff640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e00065b80 0x7f4e00068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4e040af000 tx=0x7f4e04008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:35.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.532+0000 7f4e10ff9640 1 -- 172.21.15.32:0/1358834459 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4e08050100 con 0x7f4e140cd4c0 2023-12-20T06:02:35.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/1358834459 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f4e140c54b0 con 0x7f4e140cd4c0 2023-12-20T06:02:35.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e10ff9640 1 -- 172.21.15.32:0/1358834459 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v209) v1 ==== 93+0+10 (secure 0 0 0) 0x7f4e080170a0 con 0x7f4e140cd4c0 2023-12-20T06:02:35.690 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:02:35.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/1358834459 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e00065b80 msgr2=0x7f4e00068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e00065b80 0x7f4e00068040 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4e040af000 tx=0x7f4e04008040 comp rx=0 tx=0).stop 2023-12-20T06:02:35.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/1358834459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e140cd4c0 msgr2=0x7f4e1414fd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e140cd4c0 0x7f4e1414fd10 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f4e0800c3c0 tx=0x7f4e0800c880 comp rx=0 tx=0).stop 2023-12-20T06:02:35.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/1358834459 shutdown_connections 2023-12-20T06:02:35.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e00065b80 0x7f4e00068040 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e14157480 0x7f4e14150de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e140cd4c0 0x7f4e1414fd10 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 --2- 172.21.15.32:0/1358834459 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e140b0760 0x7f4e14151f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/1358834459 >> 172.21.15.32:0/1358834459 conn(0x7f4e1406a510 msgr2=0x7f4e1415d170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:35.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/1358834459 shutdown_connections 2023-12-20T06:02:35.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.692+0000 7f4e1a9bc640 1 -- 172.21.15.32:0/1358834459 wait complete. 2023-12-20T06:02:35.711 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:02:35.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f341aff0640 1 Processor -- start 2023-12-20T06:02:35.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f341aff0640 1 -- start start 2023-12-20T06:02:35.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f341aff0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34141603e0 0x7f34140ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f341aff0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341406c480 con 0x7f3414070490 2023-12-20T06:02:35.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f341aff0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f341406c760 con 0x7f341406d240 2023-12-20T06:02:35.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f341aff0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341406ca40 con 0x7f34141603e0 2023-12-20T06:02:35.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f3418d8c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3414070490 0x7f341415fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54800/0 (socket says 172.21.15.32:54800) 2023-12-20T06:02:35.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f3418d8c640 1 -- 172.21.15.32:0/4154903296 learned_addr learned my addr 172.21.15.32:0/4154903296 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:35.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f341958d640 1 --2- 172.21.15.32:0/4154903296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34141603e0 0x7f34140ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f34137fe640 1 -- 172.21.15.32:0/4154903296 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3033472691 0 0) 0x7f341406c480 con 0x7f3414070490 2023-12-20T06:02:35.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f34137fe640 1 -- 172.21.15.32:0/4154903296 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3404003530 con 0x7f3414070490 2023-12-20T06:02:35.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f34137fe640 1 -- 172.21.15.32:0/4154903296 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2385489037 0 0) 0x7f341406c760 con 0x7f341406d240 2023-12-20T06:02:35.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f34137fe640 1 -- 172.21.15.32:0/4154903296 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f341406c480 con 0x7f341406d240 2023-12-20T06:02:35.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f341958d640 1 -- 172.21.15.32:0/4154903296 >> v1:172.21.15.39:6789/0 conn(0x7f341406d240 legacy=0x7f34140ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f341958d640 1 -- 172.21.15.32:0/4154903296 >> v1:172.21.15.32:6789/0 conn(0x7f3414070490 legacy=0x7f341415fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f34137fe640 1 -- 172.21.15.32:0/4154903296 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 93234620 0 0) 0x7f3404003530 con 0x7f3414070490 2023-12-20T06:02:35.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.860+0000 7f341958d640 1 -- 172.21.15.32:0/4154903296 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f341406cd20 con 0x7f34141603e0 2023-12-20T06:02:35.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.864+0000 7f341958d640 1 --2- 172.21.15.32:0/4154903296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34141603e0 0x7f34140ca8b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f340c009dc0 tx=0x7f340c01c8b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4e43f2a064cd9ac6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:35.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.864+0000 7f34137fe640 1 -- 172.21.15.32:0/4154903296 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c0a5990 con 0x7f34141603e0 2023-12-20T06:02:35.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.864+0000 7f34137fe640 1 -- 172.21.15.32:0/4154903296 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f340c0a5af0 con 0x7f34141603e0 2023-12-20T06:02:35.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.864+0000 7f34137fe640 1 -- 172.21.15.32:0/4154903296 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f340c0a5e80 con 0x7f34141603e0 2023-12-20T06:02:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.864+0000 7f341aff0640 1 -- 172.21.15.32:0/4154903296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34141603e0 msgr2=0x7f34140ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.864+0000 7f341aff0640 1 --2- 172.21.15.32:0/4154903296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34141603e0 0x7f34140ca8b0 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7f340c009dc0 tx=0x7f340c01c8b0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.864+0000 7f341aff0640 1 -- 172.21.15.32:0/4154903296 shutdown_connections 2023-12-20T06:02:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.864+0000 7f341aff0640 1 --2- 172.21.15.32:0/4154903296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34141603e0 0x7f34140ca8b0 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.864+0000 7f341aff0640 1 -- 172.21.15.32:0/4154903296 >> 172.21.15.32:0/4154903296 conn(0x7f341406a510 msgr2=0x7f34140ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.864+0000 7f341aff0640 1 -- 172.21.15.32:0/4154903296 shutdown_connections 2023-12-20T06:02:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.864+0000 7f341aff0640 1 -- 172.21.15.32:0/4154903296 wait complete. 2023-12-20T06:02:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f341aff0640 1 Processor -- start 2023-12-20T06:02:35.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f341aff0640 1 -- start start 2023-12-20T06:02:35.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f341aff0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341406d240 0x7f34140c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f341aff0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3414070490 0x7f3414158d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f341aff0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34141603e0 0x7f341414f230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f341aff0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34140c5df0 con 0x7f3414070490 2023-12-20T06:02:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f341aff0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f341415a350 con 0x7f341406d240 2023-12-20T06:02:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f341aff0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f341415a630 con 0x7f34141603e0 2023-12-20T06:02:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f3413fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3414070490 0x7f3414158d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f3413fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3414070490 0x7f3414158d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43032/0 (socket says 172.21.15.32:43032) 2023-12-20T06:02:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f3413fff640 1 -- 172.21.15.32:0/706963814 learned_addr learned my addr 172.21.15.32:0/706963814 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f341958d640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34141603e0 0x7f341414f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f3418d8c640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341406d240 0x7f34140c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f3413fff640 1 -- 172.21.15.32:0/706963814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34141603e0 msgr2=0x7f341414f230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f3413fff640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34141603e0 0x7f341414f230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f3413fff640 1 -- 172.21.15.32:0/706963814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341406d240 msgr2=0x7f34140c58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f3413fff640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341406d240 0x7f34140c58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f3413fff640 1 -- 172.21.15.32:0/706963814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f340c009a70 con 0x7f3414070490 2023-12-20T06:02:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f3413fff640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3414070490 0x7f3414158d40 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f340000c430 tx=0x7f340000c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f34117fa640 1 -- 172.21.15.32:0/706963814 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34000052d0 con 0x7f3414070490 2023-12-20T06:02:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f34117fa640 1 -- 172.21.15.32:0/706963814 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3400005430 con 0x7f3414070490 2023-12-20T06:02:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f34117fa640 1 -- 172.21.15.32:0/706963814 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3400003840 con 0x7f3414070490 2023-12-20T06:02:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f341aff0640 1 -- 172.21.15.32:0/706963814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34141543f0 con 0x7f3414070490 2023-12-20T06:02:35.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.868+0000 7f341aff0640 1 -- 172.21.15.32:0/706963814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34141549b0 con 0x7f3414070490 2023-12-20T06:02:35.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.872+0000 7f341aff0640 1 -- 172.21.15.32:0/706963814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f341404e000 con 0x7f3414070490 2023-12-20T06:02:35.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.872+0000 7f34117fa640 1 -- 172.21.15.32:0/706963814 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f34000039a0 con 0x7f3414070490 2023-12-20T06:02:35.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.872+0000 7f34117fa640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33f8065b80 0x7f33f8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:35.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.872+0000 7f34117fa640 1 -- 172.21.15.32:0/706963814 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3400082aa0 con 0x7f3414070490 2023-12-20T06:02:35.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.872+0000 7f3418d8c640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33f8065b80 0x7f33f8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:35.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.872+0000 7f3418d8c640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33f8065b80 0x7f33f8068040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f34080011f0 tx=0x7f3408009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:35.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:35.876+0000 7f34117fa640 1 -- 172.21.15.32:0/706963814 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3400050270 con 0x7f3414070490 2023-12-20T06:02:36.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.032+0000 7f341aff0640 1 -- 172.21.15.32:0/706963814 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f34140cf4d0 con 0x7f3414070490 2023-12-20T06:02:36.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.032+0000 7f34117fa640 1 -- 172.21.15.32:0/706963814 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v209) v1 ==== 104+0+11 (secure 0 0 0) 0x7f3400017020 con 0x7f3414070490 2023-12-20T06:02:36.032 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:02:36.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 -- 172.21.15.32:0/706963814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33f8065b80 msgr2=0x7f33f8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33f8065b80 0x7f33f8068040 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f34080011f0 tx=0x7f3408009040 comp rx=0 tx=0).stop 2023-12-20T06:02:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 -- 172.21.15.32:0/706963814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3414070490 msgr2=0x7f3414158d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3414070490 0x7f3414158d40 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f340000c430 tx=0x7f340000c8f0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 -- 172.21.15.32:0/706963814 shutdown_connections 2023-12-20T06:02:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33f8065b80 0x7f33f8068040 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34141603e0 0x7f341414f230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3414070490 0x7f3414158d40 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 --2- 172.21.15.32:0/706963814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f341406d240 0x7f34140c58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 -- 172.21.15.32:0/706963814 >> 172.21.15.32:0/706963814 conn(0x7f341406a510 msgr2=0x7f34140c4e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:36.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 -- 172.21.15.32:0/706963814 shutdown_connections 2023-12-20T06:02:36.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.036+0000 7f341aff0640 1 -- 172.21.15.32:0/706963814 wait complete. 2023-12-20T06:02:36.053 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:02:36.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.200+0000 7f62e5ce5640 1 Processor -- start 2023-12-20T06:02:36.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.200+0000 7f62e5ce5640 1 -- start start 2023-12-20T06:02:36.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.200+0000 7f62e5ce5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f62e006cdf0 0x7f62e0070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.200+0000 7f62e5ce5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62e0070770 0x7f62e00cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.200+0000 7f62e5ce5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62e00cc3b0 con 0x7f62e01591e0 2023-12-20T06:02:36.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.200+0000 7f62e5ce5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e0070ba0 con 0x7f62e0070770 2023-12-20T06:02:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.200+0000 7f62e5ce5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62e00c8e30 con 0x7f62e006cdf0 2023-12-20T06:02:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62dffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62e0070770 0x7f62e00cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62dffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f62e0070770 0x7f62e00cc840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43036/0 (socket says 172.21.15.32:43036) 2023-12-20T06:02:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62dffff640 1 -- 172.21.15.32:0/2440903012 learned_addr learned my addr 172.21.15.32:0/2440903012 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62deffd640 1 --2- 172.21.15.32:0/2440903012 >> v2:172.21.15.39:3300/0 conn(0x7f62e006cdf0 0x7f62e0070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62dffff640 1 -- 172.21.15.32:0/2440903012 >> v2:172.21.15.39:3300/0 conn(0x7f62e006cdf0 msgr2=0x7f62e0070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62dffff640 1 --2- 172.21.15.32:0/2440903012 >> v2:172.21.15.39:3300/0 conn(0x7f62e006cdf0 0x7f62e0070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62dffff640 1 -- 172.21.15.32:0/2440903012 >> v1:172.21.15.39:6789/0 conn(0x7f62e01591e0 legacy=0x7f62e00caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62dffff640 1 -- 172.21.15.32:0/2440903012 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62e006d280 con 0x7f62e0070770 2023-12-20T06:02:36.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62dffff640 1 --2- 172.21.15.32:0/2440903012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62e0070770 0x7f62e00cc840 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f62e015de70 tx=0x7f62d801c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be20998223fcf770 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:36.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62de7fc640 1 -- 172.21.15.32:0/2440903012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d80a5670 con 0x7f62e0070770 2023-12-20T06:02:36.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62de7fc640 1 -- 172.21.15.32:0/2440903012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d80a57d0 con 0x7f62e0070770 2023-12-20T06:02:36.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.204+0000 7f62de7fc640 1 -- 172.21.15.32:0/2440903012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d80a5b60 con 0x7f62e0070770 2023-12-20T06:02:36.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 -- 172.21.15.32:0/2440903012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62e0070770 msgr2=0x7f62e00cc840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 --2- 172.21.15.32:0/2440903012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62e0070770 0x7f62e00cc840 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f62e015de70 tx=0x7f62d801c940 comp rx=0 tx=0).stop 2023-12-20T06:02:36.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 -- 172.21.15.32:0/2440903012 shutdown_connections 2023-12-20T06:02:36.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 --2- 172.21.15.32:0/2440903012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62e0070770 0x7f62e00cc840 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 --2- 172.21.15.32:0/2440903012 >> v2:172.21.15.39:3300/0 conn(0x7f62e006cdf0 0x7f62e0070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 -- 172.21.15.32:0/2440903012 >> 172.21.15.32:0/2440903012 conn(0x7f62e006a510 msgr2=0x7f62e00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:36.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 -- 172.21.15.32:0/2440903012 shutdown_connections 2023-12-20T06:02:36.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 -- 172.21.15.32:0/2440903012 wait complete. 2023-12-20T06:02:36.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 Processor -- start 2023-12-20T06:02:36.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 -- start start 2023-12-20T06:02:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62e0070770 0x7f62e00c58b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62e01591e0 0x7f62e0151160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e00c7160 0x7f62e00b1930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00c6d40 con 0x7f62e0070770 2023-12-20T06:02:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00c5df0 con 0x7f62e00c7160 2023-12-20T06:02:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62e5ce5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62e00c60c0 con 0x7f62e01591e0 2023-12-20T06:02:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62df7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62e0070770 0x7f62e00c58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62df7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62e0070770 0x7f62e00c58b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43044/0 (socket says 172.21.15.32:43044) 2023-12-20T06:02:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62df7fe640 1 -- 172.21.15.32:0/3210503571 learned_addr learned my addr 172.21.15.32:0/3210503571 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62dffff640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e00c7160 0x7f62e00b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62deffd640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62e01591e0 0x7f62e0151160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62df7fe640 1 -- 172.21.15.32:0/3210503571 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62e01591e0 msgr2=0x7f62e0151160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62df7fe640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62e01591e0 0x7f62e0151160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62df7fe640 1 -- 172.21.15.32:0/3210503571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e00c7160 msgr2=0x7f62e00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62df7fe640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e00c7160 0x7f62e00b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62df7fe640 1 -- 172.21.15.32:0/3210503571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f62d8009b30 con 0x7f62e0070770 2023-12-20T06:02:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.208+0000 7f62deffd640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62e01591e0 0x7f62e0151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.212+0000 7f62df7fe640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62e0070770 0x7f62e00c58b0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c3c0 tx=0x7f62d000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.212+0000 7f62c7fff640 1 -- 172.21.15.32:0/3210503571 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d00052d0 con 0x7f62e0070770 2023-12-20T06:02:36.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.212+0000 7f62c7fff640 1 -- 172.21.15.32:0/3210503571 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62d0005430 con 0x7f62e0070770 2023-12-20T06:02:36.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.212+0000 7f62e5ce5640 1 -- 172.21.15.32:0/3210503571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62e00c63a0 con 0x7f62e0070770 2023-12-20T06:02:36.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.212+0000 7f62c7fff640 1 -- 172.21.15.32:0/3210503571 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62d0003840 con 0x7f62e0070770 2023-12-20T06:02:36.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.212+0000 7f62e5ce5640 1 -- 172.21.15.32:0/3210503571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62e00c6960 con 0x7f62e0070770 2023-12-20T06:02:36.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.212+0000 7f62e5ce5640 1 -- 172.21.15.32:0/3210503571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f62e004e000 con 0x7f62e0070770 2023-12-20T06:02:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.212+0000 7f62c7fff640 1 -- 172.21.15.32:0/3210503571 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f62d00039a0 con 0x7f62e0070770 2023-12-20T06:02:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.212+0000 7f62c7fff640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f62cc065b80 0x7f62cc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.212+0000 7f62c7fff640 1 -- 172.21.15.32:0/3210503571 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f62d0082860 con 0x7f62e0070770 2023-12-20T06:02:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.212+0000 7f62deffd640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f62cc065b80 0x7f62cc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.216+0000 7f62deffd640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f62cc065b80 0x7f62cc068040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62d4007a70 tx=0x7f62d4008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.216+0000 7f62c7fff640 1 -- 172.21.15.32:0/3210503571 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f62d00500e0 con 0x7f62e0070770 2023-12-20T06:02:36.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.372+0000 7f62e5ce5640 1 -- 172.21.15.32:0/3210503571 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f62e00cc0a0 con 0x7f62e0070770 2023-12-20T06:02:36.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.372+0000 7f62c7fff640 1 -- 172.21.15.32:0/3210503571 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v209) v1 ==== 100+0+11 (secure 0 0 0) 0x7f62d0017020 con 0x7f62e0070770 2023-12-20T06:02:36.373 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:02:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 -- 172.21.15.32:0/3210503571 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f62cc065b80 msgr2=0x7f62cc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f62cc065b80 0x7f62cc068040 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f62d4007a70 tx=0x7f62d4008040 comp rx=0 tx=0).stop 2023-12-20T06:02:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 -- 172.21.15.32:0/3210503571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62e0070770 msgr2=0x7f62e00c58b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62e0070770 0x7f62e00c58b0 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f62d000c3c0 tx=0x7f62d000c880 comp rx=0 tx=0).stop 2023-12-20T06:02:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 -- 172.21.15.32:0/3210503571 shutdown_connections 2023-12-20T06:02:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f62cc065b80 0x7f62cc068040 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f62e00c7160 0x7f62e00b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62e01591e0 0x7f62e0151160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 --2- 172.21.15.32:0/3210503571 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62e0070770 0x7f62e00c58b0 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 -- 172.21.15.32:0/3210503571 >> 172.21.15.32:0/3210503571 conn(0x7f62e006a510 msgr2=0x7f62e01569d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 -- 172.21.15.32:0/3210503571 shutdown_connections 2023-12-20T06:02:36.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.376+0000 7f62e5ce5640 1 -- 172.21.15.32:0/3210503571 wait complete. 2023-12-20T06:02:36.394 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T06:02:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e2be2d640 1 Processor -- start 2023-12-20T06:02:36.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e2be2d640 1 -- start start 2023-12-20T06:02:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e2be2d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8e240b0d00 0x7f8e24164a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e2be2d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e240711d0 con 0x7f8e241599e0 2023-12-20T06:02:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e2be2d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8e240c98c0 con 0x7f8e240aece0 2023-12-20T06:02:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e2be2d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8e240c9ba0 con 0x7f8e240b0d00 2023-12-20T06:02:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e29bc9640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f8e241599e0 0x7f8e240c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54816/0 (socket says 172.21.15.32:54816) 2023-12-20T06:02:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e29bc9640 1 -- 172.21.15.32:0/4115830656 learned_addr learned my addr 172.21.15.32:0/4115830656 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e2a3ca640 1 --2- 172.21.15.32:0/4115830656 >> v2:172.21.15.32:3300/0 conn(0x7f8e240b0d00 0x7f8e24164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e28bc7640 1 -- 172.21.15.32:0/4115830656 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3431284632 0 0) 0x7f8e240711d0 con 0x7f8e241599e0 2023-12-20T06:02:36.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e28bc7640 1 -- 172.21.15.32:0/4115830656 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8e10003530 con 0x7f8e241599e0 2023-12-20T06:02:36.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e2a3ca640 1 -- 172.21.15.32:0/4115830656 >> v1:172.21.15.39:6789/0 conn(0x7f8e240aece0 legacy=0x7f8e240b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e28bc7640 1 -- 172.21.15.32:0/4115830656 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2107179155 0 0) 0x7f8e240c98c0 con 0x7f8e240aece0 2023-12-20T06:02:36.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e2a3ca640 1 -- 172.21.15.32:0/4115830656 >> v1:172.21.15.32:6789/0 conn(0x7f8e241599e0 legacy=0x7f8e240c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.540+0000 7f8e2a3ca640 1 -- 172.21.15.32:0/4115830656 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e240c9e80 con 0x7f8e240b0d00 2023-12-20T06:02:36.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.544+0000 7f8e2a3ca640 1 --2- 172.21.15.32:0/4115830656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e240b0d00 0x7f8e24164a70 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009d90 tx=0x7f8e1c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b22652551b47f5c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:36.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.544+0000 7f8e28bc7640 1 -- 172.21.15.32:0/4115830656 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0a5900 con 0x7f8e240b0d00 2023-12-20T06:02:36.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.544+0000 7f8e28bc7640 1 -- 172.21.15.32:0/4115830656 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e1c0a5a60 con 0x7f8e240b0d00 2023-12-20T06:02:36.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.544+0000 7f8e28bc7640 1 -- 172.21.15.32:0/4115830656 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e1c0a5dd0 con 0x7f8e240b0d00 2023-12-20T06:02:36.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.544+0000 7f8e2be2d640 1 -- 172.21.15.32:0/4115830656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e240b0d00 msgr2=0x7f8e24164a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.544+0000 7f8e2be2d640 1 --2- 172.21.15.32:0/4115830656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e240b0d00 0x7f8e24164a70 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f8e1c009d90 tx=0x7f8e1c01c950 comp rx=0 tx=0).stop 2023-12-20T06:02:36.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.544+0000 7f8e2be2d640 1 -- 172.21.15.32:0/4115830656 shutdown_connections 2023-12-20T06:02:36.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.544+0000 7f8e2be2d640 1 --2- 172.21.15.32:0/4115830656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e240b0d00 0x7f8e24164a70 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.544+0000 7f8e2be2d640 1 -- 172.21.15.32:0/4115830656 >> 172.21.15.32:0/4115830656 conn(0x7f8e2406a320 msgr2=0x7f8e240ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:36.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.544+0000 7f8e2be2d640 1 -- 172.21.15.32:0/4115830656 shutdown_connections 2023-12-20T06:02:36.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.544+0000 7f8e2be2d640 1 -- 172.21.15.32:0/4115830656 wait complete. 2023-12-20T06:02:36.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e2be2d640 1 Processor -- start 2023-12-20T06:02:36.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e2be2d640 1 -- start start 2023-12-20T06:02:36.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e2be2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e240aece0 0x7f8e241f9030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e2be2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e240b0d00 0x7f8e2414f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e2be2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e241599e0 0x7f8e241577a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e2be2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e241f9570 con 0x7f8e240b0d00 2023-12-20T06:02:36.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e2be2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8e240c5830 con 0x7f8e240aece0 2023-12-20T06:02:36.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e2be2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8e240b1770 con 0x7f8e241599e0 2023-12-20T06:02:36.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e293c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e240b0d00 0x7f8e2414f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e293c8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e240b0d00 0x7f8e2414f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43054/0 (socket says 172.21.15.32:43054) 2023-12-20T06:02:36.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e293c8640 1 -- 172.21.15.32:0/1125422151 learned_addr learned my addr 172.21.15.32:0/1125422151 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:36.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e29bc9640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e240aece0 0x7f8e241f9030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e2a3ca640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e241599e0 0x7f8e241577a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e293c8640 1 -- 172.21.15.32:0/1125422151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e241599e0 msgr2=0x7f8e241577a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e293c8640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e241599e0 0x7f8e241577a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e293c8640 1 -- 172.21.15.32:0/1125422151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e240aece0 msgr2=0x7f8e241f9030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e293c8640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e240aece0 0x7f8e241f9030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e293c8640 1 -- 172.21.15.32:0/1125422151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8e1c009a70 con 0x7f8e240b0d00 2023-12-20T06:02:36.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e293c8640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e240b0d00 0x7f8e2414f840 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400c480 tx=0x7f8e1400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:36.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e1a7fc640 1 -- 172.21.15.32:0/1125422151 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e14005480 con 0x7f8e240b0d00 2023-12-20T06:02:36.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e1a7fc640 1 -- 172.21.15.32:0/1125422151 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8e14003600 con 0x7f8e240b0d00 2023-12-20T06:02:36.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e2be2d640 1 -- 172.21.15.32:0/1125422151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8e240b1a50 con 0x7f8e240b0d00 2023-12-20T06:02:36.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e1a7fc640 1 -- 172.21.15.32:0/1125422151 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8e140039b0 con 0x7f8e240b0d00 2023-12-20T06:02:36.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.548+0000 7f8e2be2d640 1 -- 172.21.15.32:0/1125422151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8e240b2010 con 0x7f8e240b0d00 2023-12-20T06:02:36.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.552+0000 7f8e2be2d640 1 -- 172.21.15.32:0/1125422151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8e2404e000 con 0x7f8e240b0d00 2023-12-20T06:02:36.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.552+0000 7f8e1a7fc640 1 -- 172.21.15.32:0/1125422151 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f8e14003b10 con 0x7f8e240b0d00 2023-12-20T06:02:36.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.552+0000 7f8e1a7fc640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8e08065b80 0x7f8e08068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.552+0000 7f8e1a7fc640 1 -- 172.21.15.32:0/1125422151 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f8e14082fa0 con 0x7f8e240b0d00 2023-12-20T06:02:36.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.552+0000 7f8e29bc9640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8e08065b80 0x7f8e08068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.552+0000 7f8e29bc9640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8e08065b80 0x7f8e08068040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e20001370 tx=0x7f8e20009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:36.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.556+0000 7f8e1a7fc640 1 -- 172.21.15.32:0/1125422151 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8e14050800 con 0x7f8e240b0d00 2023-12-20T06:02:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.712+0000 7f8e2be2d640 1 -- 172.21.15.32:0/1125422151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f8e240b0680 con 0x7f8e240b0d00 2023-12-20T06:02:36.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.712+0000 7f8e1a7fc640 1 -- 172.21.15.32:0/1125422151 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v209) v1 ==== 84+0+209 (secure 0 0 0) 0x7f8e14053e20 con 0x7f8e240b0d00 2023-12-20T06:02:36.712 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:36.712 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T06:02:36.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:36.713 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T06:02:36.713 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T06:02:36.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:36.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:36.713 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 24, 2023-12-20T06:02:36.713 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "cephfs_metadata" 2023-12-20T06:02:36.713 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:36.713 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:36.713 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 25, 2023-12-20T06:02:36.713 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "cephfs_data" 2023-12-20T06:02:36.714 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:36.714 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T06:02:36.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 -- 172.21.15.32:0/1125422151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8e08065b80 msgr2=0x7f8e08068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8e08065b80 0x7f8e08068040 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f8e20001370 tx=0x7f8e20009040 comp rx=0 tx=0).stop 2023-12-20T06:02:36.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 -- 172.21.15.32:0/1125422151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e240b0d00 msgr2=0x7f8e2414f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e240b0d00 0x7f8e2414f840 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x7f8e1400c480 tx=0x7f8e1400c940 comp rx=0 tx=0).stop 2023-12-20T06:02:36.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 -- 172.21.15.32:0/1125422151 shutdown_connections 2023-12-20T06:02:36.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8e08065b80 0x7f8e08068040 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8e241599e0 0x7f8e241577a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8e240b0d00 0x7f8e2414f840 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 --2- 172.21.15.32:0/1125422151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8e240aece0 0x7f8e241f9030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 -- 172.21.15.32:0/1125422151 >> 172.21.15.32:0/1125422151 conn(0x7f8e2406a320 msgr2=0x7f8e24155450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 -- 172.21.15.32:0/1125422151 shutdown_connections 2023-12-20T06:02:36.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.716+0000 7f8e2be2d640 1 -- 172.21.15.32:0/1125422151 wait complete. 2023-12-20T06:02:36.733 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-12-20T06:02:36.734 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-12-20T06:02:36.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa54d61f640 1 Processor -- start 2023-12-20T06:02:36.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa54d61f640 1 -- start start 2023-12-20T06:02:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa54d61f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa548158520 0x7fa54806dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa54d61f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5480ca050 0x7fa54815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa54d61f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5480ca7f0 con 0x7fa548070230 2023-12-20T06:02:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa54d61f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5480caad0 con 0x7fa5480ca050 2023-12-20T06:02:36.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa54d61f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5480cadb0 con 0x7fa548158520 2023-12-20T06:02:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa53ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5480ca050 0x7fa54815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa5477fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa548070230 0x7fa54815b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54818/0 (socket says 172.21.15.32:54818) 2023-12-20T06:02:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa5477fe640 1 -- 172.21.15.32:0/53344241 learned_addr learned my addr 172.21.15.32:0/53344241 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa546ffd640 1 --2- 172.21.15.32:0/53344241 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa548158520 0x7fa54806dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa5467fc640 1 -- 172.21.15.32:0/53344241 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 544746509 0 0) 0x7fa5480ca7f0 con 0x7fa548070230 2023-12-20T06:02:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa53ffff640 1 -- 172.21.15.32:0/53344241 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa548158520 msgr2=0x7fa54806dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa53ffff640 1 --2- 172.21.15.32:0/53344241 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa548158520 0x7fa54806dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa53ffff640 1 -- 172.21.15.32:0/53344241 >> v1:172.21.15.32:6789/0 conn(0x7fa548070230 legacy=0x7fa54815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.884+0000 7fa53ffff640 1 -- 172.21.15.32:0/53344241 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa54815e780 con 0x7fa5480ca050 2023-12-20T06:02:36.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa53ffff640 1 --2- 172.21.15.32:0/53344241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5480ca050 0x7fa54815ef80 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fa5300097c0 tx=0x7fa53001c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=401b1c50e754a91f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:36.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa5467fc640 1 -- 172.21.15.32:0/53344241 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300ab670 con 0x7fa5480ca050 2023-12-20T06:02:36.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa5467fc640 1 -- 172.21.15.32:0/53344241 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5300ab7d0 con 0x7fa5480ca050 2023-12-20T06:02:36.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa5467fc640 1 -- 172.21.15.32:0/53344241 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300abb60 con 0x7fa5480ca050 2023-12-20T06:02:36.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa54d61f640 1 -- 172.21.15.32:0/53344241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5480ca050 msgr2=0x7fa54815ef80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa54d61f640 1 --2- 172.21.15.32:0/53344241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5480ca050 0x7fa54815ef80 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7fa5300097c0 tx=0x7fa53001c450 comp rx=0 tx=0).stop 2023-12-20T06:02:36.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa54d61f640 1 -- 172.21.15.32:0/53344241 shutdown_connections 2023-12-20T06:02:36.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa54d61f640 1 --2- 172.21.15.32:0/53344241 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5480ca050 0x7fa54815ef80 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa54d61f640 1 --2- 172.21.15.32:0/53344241 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa548158520 0x7fa54806dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa54d61f640 1 -- 172.21.15.32:0/53344241 >> 172.21.15.32:0/53344241 conn(0x7fa54806a510 msgr2=0x7fa5480ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa54d61f640 1 -- 172.21.15.32:0/53344241 shutdown_connections 2023-12-20T06:02:36.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.888+0000 7fa54d61f640 1 -- 172.21.15.32:0/53344241 wait complete. 2023-12-20T06:02:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa54d61f640 1 Processor -- start 2023-12-20T06:02:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa54d61f640 1 -- start start 2023-12-20T06:02:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa54d61f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa548070230 0x7fa548159180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa54d61f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5480ca050 0x7fa54813f0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa54d61f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa548158520 0x7fa54815cc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa54d61f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54813f610 con 0x7fa5480ca050 2023-12-20T06:02:36.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa54d61f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa54814fd70 con 0x7fa548158520 2023-12-20T06:02:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa54d61f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa548150050 con 0x7fa548070230 2023-12-20T06:02:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5480ca050 0x7fa54813f0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5480ca050 0x7fa54813f0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43066/0 (socket says 172.21.15.32:43066) 2023-12-20T06:02:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53ffff640 1 -- 172.21.15.32:0/3928993115 learned_addr learned my addr 172.21.15.32:0/3928993115 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa546ffd640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa548070230 0x7fa548159180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa5477fe640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa548158520 0x7fa54815cc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53ffff640 1 -- 172.21.15.32:0/3928993115 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa548070230 msgr2=0x7fa548159180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53ffff640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa548070230 0x7fa548159180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53ffff640 1 -- 172.21.15.32:0/3928993115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa548158520 msgr2=0x7fa54815cc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:36.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53ffff640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa548158520 0x7fa54815cc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53ffff640 1 -- 172.21.15.32:0/3928993115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa530009420 con 0x7fa5480ca050 2023-12-20T06:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53ffff640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5480ca050 0x7fa54813f0d0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fa5300ad040 tx=0x7fa53001ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53f7fe640 1 -- 172.21.15.32:0/3928993115 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300b5810 con 0x7fa5480ca050 2023-12-20T06:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53f7fe640 1 -- 172.21.15.32:0/3928993115 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5300b5990 con 0x7fa5480ca050 2023-12-20T06:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa53f7fe640 1 -- 172.21.15.32:0/3928993115 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5300b5d40 con 0x7fa5480ca050 2023-12-20T06:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa54d61f640 1 -- 172.21.15.32:0/3928993115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa548150330 con 0x7fa5480ca050 2023-12-20T06:02:36.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.892+0000 7fa54d61f640 1 -- 172.21.15.32:0/3928993115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa548154370 con 0x7fa5480ca050 2023-12-20T06:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.896+0000 7fa54d61f640 1 -- 172.21.15.32:0/3928993115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa548150980 con 0x7fa5480ca050 2023-12-20T06:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.896+0000 7fa53f7fe640 1 -- 172.21.15.32:0/3928993115 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fa5300b4390 con 0x7fa5480ca050 2023-12-20T06:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.896+0000 7fa53f7fe640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa528065b80 0x7fa528068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.896+0000 7fa53f7fe640 1 -- 172.21.15.32:0/3928993115 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(209..209 src has 1..209) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa53011df40 con 0x7fa5480ca050 2023-12-20T06:02:36.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.896+0000 7fa546ffd640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa528065b80 0x7fa528068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:36.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.896+0000 7fa546ffd640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa528065b80 0x7fa528068040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa538009ac0 tx=0x7fa538009190 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:36.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:36.900+0000 7fa53f7fe640 1 -- 172.21.15.32:0/3928993115 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa5300eb740 con 0x7fa5480ca050 2023-12-20T06:02:37.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:37.060+0000 7fa54d61f640 1 -- 172.21.15.32:0/3928993115 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa548150190 con 0x7fa5480ca050 2023-12-20T06:02:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.024+0000 7fa53f7fe640 1 -- 172.21.15.32:0/3928993115 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v210) v1 ==== 144+0+0 (secure 0 0 0) 0x7fa548150190 con 0x7fa5480ca050 2023-12-20T06:02:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'backup_fs_metadata' created 2023-12-20T06:02:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 -- 172.21.15.32:0/3928993115 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa528065b80 msgr2=0x7fa528068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa528065b80 0x7fa528068040 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fa538009ac0 tx=0x7fa538009190 comp rx=0 tx=0).stop 2023-12-20T06:02:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 -- 172.21.15.32:0/3928993115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5480ca050 msgr2=0x7fa54813f0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5480ca050 0x7fa54813f0d0 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7fa5300ad040 tx=0x7fa53001ac90 comp rx=0 tx=0).stop 2023-12-20T06:02:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 -- 172.21.15.32:0/3928993115 shutdown_connections 2023-12-20T06:02:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa528065b80 0x7fa528068040 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa548158520 0x7fa54815cc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5480ca050 0x7fa54813f0d0 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:38.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 --2- 172.21.15.32:0/3928993115 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa548070230 0x7fa548159180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:38.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 -- 172.21.15.32:0/3928993115 >> 172.21.15.32:0/3928993115 conn(0x7fa54806a510 msgr2=0x7fa5480ce750 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:38.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 -- 172.21.15.32:0/3928993115 shutdown_connections 2023-12-20T06:02:38.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.028+0000 7fa54d61f640 1 -- 172.21.15.32:0/3928993115 wait complete. 2023-12-20T06:02:38.057 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T06:02:38.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.212+0000 7f8343b2c640 1 Processor -- start 2023-12-20T06:02:38.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.212+0000 7f8343b2c640 1 -- start start 2023-12-20T06:02:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f8343b2c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f833c15ad20 0x7f833c0cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f8343b2c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c0cc9d0 con 0x7f833c0cb1c0 2023-12-20T06:02:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f8343b2c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f833c0c9b90 con 0x7f833c15a940 2023-12-20T06:02:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f8343b2c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f833c0c9e70 con 0x7f833c15ad20 2023-12-20T06:02:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f83420c9640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f833c0cb1c0 0x7f833c06cc80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:54828/0 (socket says 172.21.15.32:54828) 2023-12-20T06:02:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f83420c9640 1 -- 172.21.15.32:0/3485281728 learned_addr learned my addr 172.21.15.32:0/3485281728 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f83410c7640 1 --2- 172.21.15.32:0/3485281728 >> v2:172.21.15.39:3300/0 conn(0x7f833c15ad20 0x7f833c0cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:38.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f83408c6640 1 -- 172.21.15.32:0/3485281728 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3793201863 0 0) 0x7f833c0cc9d0 con 0x7f833c0cb1c0 2023-12-20T06:02:38.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f83408c6640 1 -- 172.21.15.32:0/3485281728 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8328003530 con 0x7f833c0cb1c0 2023-12-20T06:02:38.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f83410c7640 1 -- 172.21.15.32:0/3485281728 >> v1:172.21.15.39:6789/0 conn(0x7f833c15a940 legacy=0x7f833c0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:38.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f83410c7640 1 -- 172.21.15.32:0/3485281728 >> v1:172.21.15.32:6789/0 conn(0x7f833c0cb1c0 legacy=0x7f833c06cc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:38.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.216+0000 7f83410c7640 1 -- 172.21.15.32:0/3485281728 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f833c0ca150 con 0x7f833c15ad20 2023-12-20T06:02:38.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f83410c7640 1 --2- 172.21.15.32:0/3485281728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f833c15ad20 0x7f833c0cc380 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f832c0097c0 tx=0x7f832c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=104473f43b1fb2de server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:38.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f83408c6640 1 -- 172.21.15.32:0/3485281728 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c0ab670 con 0x7f833c15ad20 2023-12-20T06:02:38.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f83408c6640 1 -- 172.21.15.32:0/3485281728 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c0ab7d0 con 0x7f833c15ad20 2023-12-20T06:02:38.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f83408c6640 1 -- 172.21.15.32:0/3485281728 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c0abb60 con 0x7f833c15ad20 2023-12-20T06:02:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f8343b2c640 1 -- 172.21.15.32:0/3485281728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f833c15ad20 msgr2=0x7f833c0cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:38.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f8343b2c640 1 --2- 172.21.15.32:0/3485281728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f833c15ad20 0x7f833c0cc380 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f832c0097c0 tx=0x7f832c01c450 comp rx=0 tx=0).stop 2023-12-20T06:02:38.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f8343b2c640 1 -- 172.21.15.32:0/3485281728 shutdown_connections 2023-12-20T06:02:38.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f8343b2c640 1 --2- 172.21.15.32:0/3485281728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f833c15ad20 0x7f833c0cc380 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:38.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f8343b2c640 1 -- 172.21.15.32:0/3485281728 >> 172.21.15.32:0/3485281728 conn(0x7f833c06a4b0 msgr2=0x7f833c0b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:38.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f8343b2c640 1 -- 172.21.15.32:0/3485281728 shutdown_connections 2023-12-20T06:02:38.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f8343b2c640 1 -- 172.21.15.32:0/3485281728 wait complete. 2023-12-20T06:02:38.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f8343b2c640 1 Processor -- start 2023-12-20T06:02:38.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.220+0000 7f8343b2c640 1 -- start start 2023-12-20T06:02:38.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f8343b2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f833c0cb1c0 0x7f833c0b37c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:38.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f8343b2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c15a940 0x7f833c1508d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:38.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f8343b2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f833c15ad20 0x7f833c14f270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:38.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f8343b2c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c1554d0 con 0x7f833c15a940 2023-12-20T06:02:38.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f8343b2c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f833c151390 con 0x7f833c15ad20 2023-12-20T06:02:38.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f8343b2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f833c151670 con 0x7f833c0cb1c0 2023-12-20T06:02:38.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83410c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c15a940 0x7f833c1508d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:38.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83410c7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c15a940 0x7f833c1508d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:43082/0 (socket says 172.21.15.32:43082) 2023-12-20T06:02:38.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83410c7640 1 -- 172.21.15.32:0/3647720973 learned_addr learned my addr 172.21.15.32:0/3647720973 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:38.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83420c9640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f833c15ad20 0x7f833c14f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:38.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83418c8640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f833c0cb1c0 0x7f833c0b37c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:38.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83410c7640 1 -- 172.21.15.32:0/3647720973 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f833c0cb1c0 msgr2=0x7f833c0b37c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:38.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83410c7640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f833c0cb1c0 0x7f833c0b37c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:38.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83410c7640 1 -- 172.21.15.32:0/3647720973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f833c15ad20 msgr2=0x7f833c14f270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:38.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83410c7640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f833c15ad20 0x7f833c14f270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:38.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83410c7640 1 -- 172.21.15.32:0/3647720973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f832c009420 con 0x7f833c15a940 2023-12-20T06:02:38.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83410c7640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c15a940 0x7f833c1508d0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f832c0ad040 tx=0x7f832c01ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:38.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83327fc640 1 -- 172.21.15.32:0/3647720973 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c0bf610 con 0x7f833c15a940 2023-12-20T06:02:38.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83327fc640 1 -- 172.21.15.32:0/3647720973 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f832c0bf7b0 con 0x7f833c15a940 2023-12-20T06:02:38.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f83327fc640 1 -- 172.21.15.32:0/3647720973 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f832c0bfb60 con 0x7f833c15a940 2023-12-20T06:02:38.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f8343b2c640 1 -- 172.21.15.32:0/3647720973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f833c151950 con 0x7f833c15a940 2023-12-20T06:02:38.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f8343b2c640 1 -- 172.21.15.32:0/3647720973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f833c1550f0 con 0x7f833c15a940 2023-12-20T06:02:38.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.224+0000 7f8343b2c640 1 -- 172.21.15.32:0/3647720973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f833c155640 con 0x7f833c15a940 2023-12-20T06:02:38.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.228+0000 7f83327fc640 1 -- 172.21.15.32:0/3647720973 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f832c0b5400 con 0x7f833c15a940 2023-12-20T06:02:38.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.228+0000 7f83327fc640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8320065b80 0x7f8320068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:38.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.228+0000 7f83327fc640 1 -- 172.21.15.32:0/3647720973 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(210..210 src has 1..210) v4 ==== 6138+0+0 (secure 0 0 0) 0x7f832c11e9a0 con 0x7f833c15a940 2023-12-20T06:02:38.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.228+0000 7f83418c8640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8320065b80 0x7f8320068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:38.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.228+0000 7f83418c8640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8320065b80 0x7f8320068040 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8338007fd0 tx=0x7f8338009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:38.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.232+0000 7f83327fc640 1 -- 172.21.15.32:0/3647720973 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f832c0ec240 con 0x7f833c15a940 2023-12-20T06:02:38.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:38.388+0000 7f8343b2c640 1 -- 172.21.15.32:0/3647720973 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f833c04e000 con 0x7f833c15a940 2023-12-20T06:02:39.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.040+0000 7f83327fc640 1 -- 172.21.15.32:0/3647720973 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v211) v1 ==== 176+0+0 (secure 0 0 0) 0x7f832c0ef860 con 0x7f833c15a940 2023-12-20T06:02:39.040 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'backup_fs_data' created 2023-12-20T06:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.048+0000 7f8343b2c640 1 -- 172.21.15.32:0/3647720973 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8320065b80 msgr2=0x7f8320068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.048+0000 7f8343b2c640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8320065b80 0x7f8320068040 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f8338007fd0 tx=0x7f8338009040 comp rx=0 tx=0).stop 2023-12-20T06:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.048+0000 7f8343b2c640 1 -- 172.21.15.32:0/3647720973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c15a940 msgr2=0x7f833c1508d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.048+0000 7f8343b2c640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c15a940 0x7f833c1508d0 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7f832c0ad040 tx=0x7f832c01ac90 comp rx=0 tx=0).stop 2023-12-20T06:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.052+0000 7f8343b2c640 1 -- 172.21.15.32:0/3647720973 shutdown_connections 2023-12-20T06:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.052+0000 7f8343b2c640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8320065b80 0x7f8320068040 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.052+0000 7f8343b2c640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f833c15ad20 0x7f833c14f270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.052+0000 7f8343b2c640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f833c15a940 0x7f833c1508d0 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.052+0000 7f8343b2c640 1 --2- 172.21.15.32:0/3647720973 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f833c0cb1c0 0x7f833c0b37c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.052+0000 7f8343b2c640 1 -- 172.21.15.32:0/3647720973 >> 172.21.15.32:0/3647720973 conn(0x7f833c06a4b0 msgr2=0x7f833c070060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.052+0000 7f8343b2c640 1 -- 172.21.15.32:0/3647720973 shutdown_connections 2023-12-20T06:02:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.052+0000 7f8343b2c640 1 -- 172.21.15.32:0/3647720973 wait complete. 2023-12-20T06:02:39.085 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-12-20T06:02:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.232+0000 7f2b99e20640 1 Processor -- start 2023-12-20T06:02:39.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.232+0000 7f2b99e20640 1 -- start start 2023-12-20T06:02:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b99e20640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2b9415abb0 0x7f2b940caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b99e20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b94070230 0x7f2b940706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b99e20640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b9415b6a0 con 0x7f2b940b0cb0 2023-12-20T06:02:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b99e20640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b9406e160 con 0x7f2b9415abb0 2023-12-20T06:02:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b99e20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b94070c00 con 0x7f2b94070230 2023-12-20T06:02:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b92ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2b940b0cb0 0x7f2b9415bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60414/0 (socket says 172.21.15.32:60414) 2023-12-20T06:02:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b92ffd640 1 -- 172.21.15.32:0/2757531523 learned_addr learned my addr 172.21.15.32:0/2757531523 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b937fe640 1 --2- 172.21.15.32:0/2757531523 >> v2:172.21.15.39:3300/0 conn(0x7f2b9415abb0 0x7f2b940caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b93fff640 1 --2- 172.21.15.32:0/2757531523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b94070230 0x7f2b940706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b927fc640 1 -- 172.21.15.32:0/2757531523 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4113492155 0 0) 0x7f2b9415b6a0 con 0x7f2b940b0cb0 2023-12-20T06:02:39.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b927fc640 1 -- 172.21.15.32:0/2757531523 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b7c003530 con 0x7f2b940b0cb0 2023-12-20T06:02:39.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b937fe640 1 -- 172.21.15.32:0/2757531523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b94070230 msgr2=0x7f2b940706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:39.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b937fe640 1 --2- 172.21.15.32:0/2757531523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b94070230 0x7f2b940706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b937fe640 1 -- 172.21.15.32:0/2757531523 >> v1:172.21.15.32:6789/0 conn(0x7f2b940b0cb0 legacy=0x7f2b9415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b937fe640 1 -- 172.21.15.32:0/2757531523 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b9406d8c0 con 0x7f2b9415abb0 2023-12-20T06:02:39.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.236+0000 7f2b93fff640 1 --2- 172.21.15.32:0/2757531523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b94070230 0x7f2b940706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:39.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b937fe640 1 --2- 172.21.15.32:0/2757531523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b9415abb0 0x7f2b940caaf0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2b840097c0 tx=0x7f2b8401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=19b54dc1b5331c1a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:39.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b927fc640 1 -- 172.21.15.32:0/2757531523 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840ab680 con 0x7f2b9415abb0 2023-12-20T06:02:39.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b927fc640 1 -- 172.21.15.32:0/2757531523 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b840ab7e0 con 0x7f2b9415abb0 2023-12-20T06:02:39.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b927fc640 1 -- 172.21.15.32:0/2757531523 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b840abb70 con 0x7f2b9415abb0 2023-12-20T06:02:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b99e20640 1 -- 172.21.15.32:0/2757531523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b9415abb0 msgr2=0x7f2b940caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b99e20640 1 --2- 172.21.15.32:0/2757531523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b9415abb0 0x7f2b940caaf0 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f2b840097c0 tx=0x7f2b8401c480 comp rx=0 tx=0).stop 2023-12-20T06:02:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b99e20640 1 -- 172.21.15.32:0/2757531523 shutdown_connections 2023-12-20T06:02:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b99e20640 1 --2- 172.21.15.32:0/2757531523 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b94070230 0x7f2b940706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b99e20640 1 --2- 172.21.15.32:0/2757531523 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b9415abb0 0x7f2b940caaf0 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b99e20640 1 -- 172.21.15.32:0/2757531523 >> 172.21.15.32:0/2757531523 conn(0x7f2b9406a510 msgr2=0x7f2b940ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:39.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b99e20640 1 -- 172.21.15.32:0/2757531523 shutdown_connections 2023-12-20T06:02:39.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b99e20640 1 -- 172.21.15.32:0/2757531523 wait complete. 2023-12-20T06:02:39.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.240+0000 7f2b99e20640 1 Processor -- start 2023-12-20T06:02:39.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b99e20640 1 -- start start 2023-12-20T06:02:39.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b99e20640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b94070230 0x7f2b941504f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b99e20640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b940b0cb0 0x7f2b94146440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b99e20640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b9415abb0 0x7f2b94150d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b99e20640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b941474c0 con 0x7f2b940b0cb0 2023-12-20T06:02:39.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b99e20640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b94144bf0 con 0x7f2b94070230 2023-12-20T06:02:39.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b99e20640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b94144e90 con 0x7f2b9415abb0 2023-12-20T06:02:39.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b92ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b940b0cb0 0x7f2b94146440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b937fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b94070230 0x7f2b941504f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b92ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b940b0cb0 0x7f2b94146440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46738/0 (socket says 172.21.15.32:46738) 2023-12-20T06:02:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b92ffd640 1 -- 172.21.15.32:0/1213963590 learned_addr learned my addr 172.21.15.32:0/1213963590 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b93fff640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b9415abb0 0x7f2b94150d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b92ffd640 1 -- 172.21.15.32:0/1213963590 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b9415abb0 msgr2=0x7f2b94150d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b92ffd640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b9415abb0 0x7f2b94150d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b92ffd640 1 -- 172.21.15.32:0/1213963590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b94070230 msgr2=0x7f2b941504f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b92ffd640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b94070230 0x7f2b941504f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b92ffd640 1 -- 172.21.15.32:0/1213963590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c009a70 con 0x7f2b940b0cb0 2023-12-20T06:02:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b92ffd640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b940b0cb0 0x7f2b94146440 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800c420 tx=0x7f2b8800c820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b7bfff640 1 -- 172.21.15.32:0/1213963590 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b88005cf0 con 0x7f2b940b0cb0 2023-12-20T06:02:39.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b7bfff640 1 -- 172.21.15.32:0/1213963590 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b88005e70 con 0x7f2b940b0cb0 2023-12-20T06:02:39.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b99e20640 1 -- 172.21.15.32:0/1213963590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b84009420 con 0x7f2b940b0cb0 2023-12-20T06:02:39.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b7bfff640 1 -- 172.21.15.32:0/1213963590 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b880072f0 con 0x7f2b940b0cb0 2023-12-20T06:02:39.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b99e20640 1 -- 172.21.15.32:0/1213963590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b941469d0 con 0x7f2b940b0cb0 2023-12-20T06:02:39.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.244+0000 7f2b99e20640 1 -- 172.21.15.32:0/1213963590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b94145040 con 0x7f2b940b0cb0 2023-12-20T06:02:39.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.248+0000 7f2b7bfff640 1 -- 172.21.15.32:0/1213963590 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2b88007450 con 0x7f2b940b0cb0 2023-12-20T06:02:39.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.248+0000 7f2b7bfff640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b80065b80 0x7f2b80068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.248+0000 7f2b7bfff640 1 -- 172.21.15.32:0/1213963590 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(211..211 src has 1..211) v4 ==== 6523+0+0 (secure 0 0 0) 0x7f2b88082b90 con 0x7f2b940b0cb0 2023-12-20T06:02:39.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.248+0000 7f2b937fe640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b80065b80 0x7f2b80068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.248+0000 7f2b937fe640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b80065b80 0x7f2b80068040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b840af000 tx=0x7f2b840a7000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:39.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.252+0000 7f2b7bfff640 1 -- 172.21.15.32:0/1213963590 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2b94145040 con 0x7f2b940b0cb0 2023-12-20T06:02:39.442 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.444+0000 7f2b99e20640 1 -- 172.21.15.32:0/1213963590 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7f2b94145000 con 0x7f2b940b0cb0 2023-12-20T06:02:39.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.452+0000 7f2b7bfff640 1 -- 172.21.15.32:0/1213963590 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 26 and data pool 27 v98) v1 ==== 183+0+0 (secure 0 0 0) 0x7f2b88017020 con 0x7f2b940b0cb0 2023-12-20T06:02:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 26 and data pool 27 2023-12-20T06:02:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.456+0000 7f2b99e20640 1 -- 172.21.15.32:0/1213963590 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b80065b80 msgr2=0x7f2b80068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.456+0000 7f2b99e20640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b80065b80 0x7f2b80068040 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f2b840af000 tx=0x7f2b840a7000 comp rx=0 tx=0).stop 2023-12-20T06:02:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.456+0000 7f2b99e20640 1 -- 172.21.15.32:0/1213963590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b940b0cb0 msgr2=0x7f2b94146440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.456+0000 7f2b99e20640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b940b0cb0 0x7f2b94146440 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7f2b8800c420 tx=0x7f2b8800c820 comp rx=0 tx=0).stop 2023-12-20T06:02:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.460+0000 7f2b99e20640 1 -- 172.21.15.32:0/1213963590 shutdown_connections 2023-12-20T06:02:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.460+0000 7f2b99e20640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b80065b80 0x7f2b80068040 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.460+0000 7f2b99e20640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b9415abb0 0x7f2b94150d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.460+0000 7f2b99e20640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b940b0cb0 0x7f2b94146440 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.460+0000 7f2b99e20640 1 --2- 172.21.15.32:0/1213963590 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b94070230 0x7f2b941504f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.460+0000 7f2b99e20640 1 -- 172.21.15.32:0/1213963590 >> 172.21.15.32:0/1213963590 conn(0x7f2b9406a510 msgr2=0x7f2b94154820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:39.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.464+0000 7f2b99e20640 1 -- 172.21.15.32:0/1213963590 shutdown_connections 2023-12-20T06:02:39.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.464+0000 7f2b99e20640 1 -- 172.21.15.32:0/1213963590 wait complete. 2023-12-20T06:02:39.498 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2c68c640 1 Processor -- start 2023-12-20T06:02:39.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2c68c640 1 -- start start 2023-12-20T06:02:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2c68c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7eff241599e0 0x7eff240aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2c68c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff240b0d00 0x7eff240701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2c68c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7eff240c98c0 con 0x7eff240c90e0 2023-12-20T06:02:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2c68c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7eff240c9ba0 con 0x7eff241599e0 2023-12-20T06:02:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2c68c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff240c9e80 con 0x7eff240b0d00 2023-12-20T06:02:39.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff29c27640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7eff240c90e0 0x7eff240b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60424/0 (socket says 172.21.15.32:60424) 2023-12-20T06:02:39.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff29c27640 1 -- 172.21.15.32:0/3666145649 learned_addr learned my addr 172.21.15.32:0/3666145649 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:39.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2ac29640 1 --2- 172.21.15.32:0/3666145649 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff240b0d00 0x7eff240701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2a428640 1 --2- 172.21.15.32:0/3666145649 >> v2:172.21.15.39:3300/0 conn(0x7eff241599e0 0x7eff240aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff29426640 1 -- 172.21.15.32:0/3666145649 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2949414647 0 0) 0x7eff240c98c0 con 0x7eff240c90e0 2023-12-20T06:02:39.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff29426640 1 -- 172.21.15.32:0/3666145649 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7eff10003530 con 0x7eff240c90e0 2023-12-20T06:02:39.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2ac29640 1 -- 172.21.15.32:0/3666145649 >> v2:172.21.15.39:3300/0 conn(0x7eff241599e0 msgr2=0x7eff240aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:39.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2ac29640 1 --2- 172.21.15.32:0/3666145649 >> v2:172.21.15.39:3300/0 conn(0x7eff241599e0 0x7eff240aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2ac29640 1 -- 172.21.15.32:0/3666145649 >> v1:172.21.15.32:6789/0 conn(0x7eff240c90e0 legacy=0x7eff240b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.644+0000 7eff2ac29640 1 -- 172.21.15.32:0/3666145649 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff240ca160 con 0x7eff240b0d00 2023-12-20T06:02:39.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff2ac29640 1 --2- 172.21.15.32:0/3666145649 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff240b0d00 0x7eff240701c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009d90 tx=0x7eff1c01c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9139b8e804508b92 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:39.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff29426640 1 -- 172.21.15.32:0/3666145649 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0a5970 con 0x7eff240b0d00 2023-12-20T06:02:39.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff29426640 1 -- 172.21.15.32:0/3666145649 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c0a5ad0 con 0x7eff240b0d00 2023-12-20T06:02:39.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff29426640 1 -- 172.21.15.32:0/3666145649 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0a5e60 con 0x7eff240b0d00 2023-12-20T06:02:39.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff2c68c640 1 -- 172.21.15.32:0/3666145649 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff240b0d00 msgr2=0x7eff240701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff2c68c640 1 --2- 172.21.15.32:0/3666145649 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff240b0d00 0x7eff240701c0 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7eff1c009d90 tx=0x7eff1c01c890 comp rx=0 tx=0).stop 2023-12-20T06:02:39.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff2c68c640 1 -- 172.21.15.32:0/3666145649 shutdown_connections 2023-12-20T06:02:39.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff2c68c640 1 --2- 172.21.15.32:0/3666145649 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff240b0d00 0x7eff240701c0 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff2c68c640 1 --2- 172.21.15.32:0/3666145649 >> v2:172.21.15.39:3300/0 conn(0x7eff241599e0 0x7eff240aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff2c68c640 1 -- 172.21.15.32:0/3666145649 >> 172.21.15.32:0/3666145649 conn(0x7eff2406a320 msgr2=0x7eff240ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:39.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff2c68c640 1 -- 172.21.15.32:0/3666145649 shutdown_connections 2023-12-20T06:02:39.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.648+0000 7eff2c68c640 1 -- 172.21.15.32:0/3666145649 wait complete. 2023-12-20T06:02:39.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2c68c640 1 Processor -- start 2023-12-20T06:02:39.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2c68c640 1 -- start start 2023-12-20T06:02:39.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2c68c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff240b0d00 0x7eff241472a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2c68c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff240c90e0 0x7eff240ab8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2c68c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff241599e0 0x7eff24071220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2c68c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff240ad570 con 0x7eff241599e0 2023-12-20T06:02:39.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2c68c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7eff240ad850 con 0x7eff240c90e0 2023-12-20T06:02:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2c68c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7eff240adb30 con 0x7eff240b0d00 2023-12-20T06:02:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2ac29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff241599e0 0x7eff24071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2ac29640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff241599e0 0x7eff24071220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46746/0 (socket says 172.21.15.32:46746) 2023-12-20T06:02:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2ac29640 1 -- 172.21.15.32:0/32009648 learned_addr learned my addr 172.21.15.32:0/32009648 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2a428640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff240b0d00 0x7eff241472a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff29c27640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff240c90e0 0x7eff240ab8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2ac29640 1 -- 172.21.15.32:0/32009648 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff240b0d00 msgr2=0x7eff241472a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2ac29640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff240b0d00 0x7eff241472a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2ac29640 1 -- 172.21.15.32:0/32009648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff240c90e0 msgr2=0x7eff240ab8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2ac29640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff240c90e0 0x7eff240ab8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2ac29640 1 -- 172.21.15.32:0/32009648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7eff1c009a70 con 0x7eff241599e0 2023-12-20T06:02:39.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff29c27640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff240c90e0 0x7eff240ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:39.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2ac29640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff241599e0 0x7eff24071220 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7eff1c01af40 tx=0x7eff1c019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:39.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff1affd640 1 -- 172.21.15.32:0/32009648 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0adc60 con 0x7eff241599e0 2023-12-20T06:02:39.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff1affd640 1 -- 172.21.15.32:0/32009648 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7eff1c0addc0 con 0x7eff241599e0 2023-12-20T06:02:39.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff1affd640 1 -- 172.21.15.32:0/32009648 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7eff1c0b9650 con 0x7eff241599e0 2023-12-20T06:02:39.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.652+0000 7eff2c68c640 1 -- 172.21.15.32:0/32009648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7eff240ade10 con 0x7eff241599e0 2023-12-20T06:02:39.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.656+0000 7eff2c68c640 1 -- 172.21.15.32:0/32009648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7eff241477e0 con 0x7eff241599e0 2023-12-20T06:02:39.655 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.656+0000 7eff2c68c640 1 -- 172.21.15.32:0/32009648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7eff2404e000 con 0x7eff241599e0 2023-12-20T06:02:39.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.656+0000 7eff1affd640 1 -- 172.21.15.32:0/32009648 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7eff1c0b97b0 con 0x7eff241599e0 2023-12-20T06:02:39.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.656+0000 7eff1affd640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7eff08065bb0 0x7eff08068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.656+0000 7eff1affd640 1 -- 172.21.15.32:0/32009648 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6629+0+0 (secure 0 0 0) 0x7eff1c11eed0 con 0x7eff241599e0 2023-12-20T06:02:39.656 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.656+0000 7eff2a428640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7eff08065bb0 0x7eff08068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.656+0000 7eff2a428640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7eff08065bb0 0x7eff08068070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff20009ad0 tx=0x7eff20008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:39.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.660+0000 7eff1affd640 1 -- 172.21.15.32:0/32009648 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7eff1c0ec550 con 0x7eff241599e0 2023-12-20T06:02:39.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.820+0000 7eff2c68c640 1 -- 172.21.15.32:0/32009648 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7eff240ae2c0 con 0x7eff241599e0 2023-12-20T06:02:39.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.820+0000 7eff1affd640 1 -- 172.21.15.32:0/32009648 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+19894 (secure 0 0 0) 0x7eff1c009450 con 0x7eff241599e0 2023-12-20T06:02:39.820 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:39.820 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":212,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:39.446667+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:02:25.045126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-20T06:02:25.938205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:02:37.109832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:02:38.440830+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 -- 172.21.15.32:0/32009648 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7eff08065bb0 msgr2=0x7eff08068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7eff08065bb0 0x7eff08068070 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7eff20009ad0 tx=0x7eff20008040 comp rx=0 tx=0).stop 2023-12-20T06:02:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 -- 172.21.15.32:0/32009648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff241599e0 msgr2=0x7eff24071220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff241599e0 0x7eff24071220 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7eff1c01af40 tx=0x7eff1c019630 comp rx=0 tx=0).stop 2023-12-20T06:02:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 -- 172.21.15.32:0/32009648 shutdown_connections 2023-12-20T06:02:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7eff08065bb0 0x7eff08068070 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7eff241599e0 0x7eff24071220 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7eff240c90e0 0x7eff240ab8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 --2- 172.21.15.32:0/32009648 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7eff240b0d00 0x7eff241472a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 -- 172.21.15.32:0/32009648 >> 172.21.15.32:0/32009648 conn(0x7eff2406a320 msgr2=0x7eff2406c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:39.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 -- 172.21.15.32:0/32009648 shutdown_connections 2023-12-20T06:02:39.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.824+0000 7eff2c68c640 1 -- 172.21.15.32:0/32009648 wait complete. 2023-12-20T06:02:39.842 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:39.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf7b96d640 1 Processor -- start 2023-12-20T06:02:39.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf7b96d640 1 -- start start 2023-12-20T06:02:39.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf7b96d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf741599e0 0x7fcf740aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf7b96d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcf740c90e0 0x7fcf740b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf7b96d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf740b0b30 0x7fcf7415d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf7b96d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740c98c0 con 0x7fcf740b0b30 2023-12-20T06:02:39.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf7b96d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf740c9ba0 con 0x7fcf740c90e0 2023-12-20T06:02:39.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf7b96d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740c9e80 con 0x7fcf741599e0 2023-12-20T06:02:39.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf79f0a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf740b0b30 0x7fcf7415d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf79f0a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf740b0b30 0x7fcf7415d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46750/0 (socket says 172.21.15.32:46750) 2023-12-20T06:02:39.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf79f0a640 1 -- 172.21.15.32:0/2217605908 learned_addr learned my addr 172.21.15.32:0/2217605908 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:39.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf79709640 1 --2- 172.21.15.32:0/2217605908 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf741599e0 0x7fcf740aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf78f08640 1 --2- 172.21.15.32:0/2217605908 >> v2:172.21.15.39:3300/0 conn(0x7fcf740c90e0 0x7fcf740b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf79f0a640 1 -- 172.21.15.32:0/2217605908 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf741599e0 msgr2=0x7fcf740aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:39.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf79f0a640 1 --2- 172.21.15.32:0/2217605908 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf741599e0 0x7fcf740aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.988+0000 7fcf79f0a640 1 -- 172.21.15.32:0/2217605908 >> v2:172.21.15.39:3300/0 conn(0x7fcf740c90e0 msgr2=0x7fcf740b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:39.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf79f0a640 1 --2- 172.21.15.32:0/2217605908 >> v2:172.21.15.39:3300/0 conn(0x7fcf740c90e0 0x7fcf740b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf79f0a640 1 -- 172.21.15.32:0/2217605908 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf740ca160 con 0x7fcf740b0b30 2023-12-20T06:02:39.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf78f08640 1 --2- 172.21.15.32:0/2217605908 >> v2:172.21.15.39:3300/0 conn(0x7fcf740c90e0 0x7fcf740b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:39.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf79f0a640 1 --2- 172.21.15.32:0/2217605908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf740b0b30 0x7fcf7415d6e0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fcf7406d6b0 tx=0x7fcf6c01c6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=873ece943bdfc90d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:39.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf6bfff640 1 -- 172.21.15.32:0/2217605908 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0a5580 con 0x7fcf740b0b30 2023-12-20T06:02:39.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf6bfff640 1 -- 172.21.15.32:0/2217605908 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6c0a56e0 con 0x7fcf740b0b30 2023-12-20T06:02:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf6bfff640 1 -- 172.21.15.32:0/2217605908 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c0a5a70 con 0x7fcf740b0b30 2023-12-20T06:02:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf7b96d640 1 -- 172.21.15.32:0/2217605908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf740b0b30 msgr2=0x7fcf7415d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf7b96d640 1 --2- 172.21.15.32:0/2217605908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf740b0b30 0x7fcf7415d6e0 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7fcf7406d6b0 tx=0x7fcf6c01c6e0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf7b96d640 1 -- 172.21.15.32:0/2217605908 shutdown_connections 2023-12-20T06:02:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf7b96d640 1 --2- 172.21.15.32:0/2217605908 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf740b0b30 0x7fcf7415d6e0 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf7b96d640 1 --2- 172.21.15.32:0/2217605908 >> v2:172.21.15.39:3300/0 conn(0x7fcf740c90e0 0x7fcf740b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf7b96d640 1 --2- 172.21.15.32:0/2217605908 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf741599e0 0x7fcf740aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf7b96d640 1 -- 172.21.15.32:0/2217605908 >> 172.21.15.32:0/2217605908 conn(0x7fcf7406a320 msgr2=0x7fcf740ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf7b96d640 1 -- 172.21.15.32:0/2217605908 shutdown_connections 2023-12-20T06:02:39.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.992+0000 7fcf7b96d640 1 -- 172.21.15.32:0/2217605908 wait complete. 2023-12-20T06:02:39.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf7b96d640 1 Processor -- start 2023-12-20T06:02:39.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf7b96d640 1 -- start start 2023-12-20T06:02:39.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf7b96d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf740b0b30 0x7fcf74163690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf7b96d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf740c90e0 0x7fcf740b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf7b96d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf741599e0 0x7fcf7414eff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:39.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf7b96d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf740b2e20 con 0x7fcf740c90e0 2023-12-20T06:02:39.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf7b96d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74150580 con 0x7fcf741599e0 2023-12-20T06:02:39.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf7b96d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf74150820 con 0x7fcf740b0b30 2023-12-20T06:02:39.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf78f08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf740c90e0 0x7fcf740b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf78f08640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf740c90e0 0x7fcf740b28e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46758/0 (socket says 172.21.15.32:46758) 2023-12-20T06:02:39.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf78f08640 1 -- 172.21.15.32:0/888253968 learned_addr learned my addr 172.21.15.32:0/888253968 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:39.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf79709640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf740b0b30 0x7fcf74163690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf79f0a640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf741599e0 0x7fcf7414eff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:39.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf78f08640 1 -- 172.21.15.32:0/888253968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf740b0b30 msgr2=0x7fcf74163690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf78f08640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf740b0b30 0x7fcf74163690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf78f08640 1 -- 172.21.15.32:0/888253968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf741599e0 msgr2=0x7fcf7414eff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:39.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf78f08640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf741599e0 0x7fcf7414eff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:39.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf78f08640 1 -- 172.21.15.32:0/888253968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf6c009b30 con 0x7fcf740c90e0 2023-12-20T06:02:39.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf78f08640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf740c90e0 0x7fcf740b28e0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400e810 tx=0x7fcf6400ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:39.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf69ffb640 1 -- 172.21.15.32:0/888253968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400c8d0 con 0x7fcf740c90e0 2023-12-20T06:02:39.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf69ffb640 1 -- 172.21.15.32:0/888253968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf6400ca30 con 0x7fcf740c90e0 2023-12-20T06:02:39.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf69ffb640 1 -- 172.21.15.32:0/888253968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf6400cdc0 con 0x7fcf740c90e0 2023-12-20T06:02:39.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf7b96d640 1 -- 172.21.15.32:0/888253968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf74150b00 con 0x7fcf740c90e0 2023-12-20T06:02:39.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:39.996+0000 7fcf7b96d640 1 -- 172.21.15.32:0/888253968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf740b3050 con 0x7fcf740c90e0 2023-12-20T06:02:39.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.000+0000 7fcf7b96d640 1 -- 172.21.15.32:0/888253968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf7404e000 con 0x7fcf740c90e0 2023-12-20T06:02:40.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.000+0000 7fcf69ffb640 1 -- 172.21.15.32:0/888253968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcf64013410 con 0x7fcf740c90e0 2023-12-20T06:02:40.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.000+0000 7fcf69ffb640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf58065b80 0x7fcf58068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:40.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.000+0000 7fcf69ffb640 1 -- 172.21.15.32:0/888253968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6629+0+0 (secure 0 0 0) 0x7fcf640835f0 con 0x7fcf740c90e0 2023-12-20T06:02:40.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.000+0000 7fcf79709640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf58065b80 0x7fcf58068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:40.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.004+0000 7fcf79709640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf58065b80 0x7fcf58068040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf70006fd0 tx=0x7fcf70008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:40.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.004+0000 7fcf69ffb640 1 -- 172.21.15.32:0/888253968 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcf640509e0 con 0x7fcf740c90e0 2023-12-20T06:02:40.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 -- 172.21.15.32:0/888253968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcf7415da00 con 0x7fcf740c90e0 2023-12-20T06:02:40.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf69ffb640 1 -- 172.21.15.32:0/888253968 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v212) v1 ==== 74+0+19894 (secure 0 0 0) 0x7fcf64054000 con 0x7fcf740c90e0 2023-12-20T06:02:40.166 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:40.167 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":212,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:39.446667+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:02:25.045126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-20T06:02:25.938205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:02:37.109832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:02:38.440830+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 -- 172.21.15.32:0/888253968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf58065b80 msgr2=0x7fcf58068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf58065b80 0x7fcf58068040 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcf70006fd0 tx=0x7fcf70008040 comp rx=0 tx=0).stop 2023-12-20T06:02:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 -- 172.21.15.32:0/888253968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf740c90e0 msgr2=0x7fcf740b28e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf740c90e0 0x7fcf740b28e0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7fcf6400e810 tx=0x7fcf6400ecd0 comp rx=0 tx=0).stop 2023-12-20T06:02:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 -- 172.21.15.32:0/888253968 shutdown_connections 2023-12-20T06:02:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf58065b80 0x7fcf58068040 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf741599e0 0x7fcf7414eff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf740c90e0 0x7fcf740b28e0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:40.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 --2- 172.21.15.32:0/888253968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf740b0b30 0x7fcf74163690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 -- 172.21.15.32:0/888253968 >> 172.21.15.32:0/888253968 conn(0x7fcf7406a320 msgr2=0x7fcf740caf20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.168+0000 7fcf7b96d640 1 -- 172.21.15.32:0/888253968 shutdown_connections 2023-12-20T06:02:40.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.172+0000 7fcf7b96d640 1 -- 172.21.15.32:0/888253968 wait complete. 2023-12-20T06:02:40.188 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-12-20T06:02:40.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.336+0000 7fee7fe66640 1 Processor -- start 2023-12-20T06:02:40.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.336+0000 7fee7fe66640 1 -- start start 2023-12-20T06:02:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.336+0000 7fee7fe66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee78157480 0x7fee780c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.336+0000 7fee7fe66640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fee780b0760 0x7fee780b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.336+0000 7fee7fe66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee780b1100 con 0x7fee7815d660 2023-12-20T06:02:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.336+0000 7fee7fe66640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee780cde20 con 0x7fee780b0760 2023-12-20T06:02:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.336+0000 7fee7fe66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee780c8e30 con 0x7fee78157480 2023-12-20T06:02:40.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.336+0000 7fee7e403640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fee7815d660 0x7fee7815daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60440/0 (socket says 172.21.15.32:60440) 2023-12-20T06:02:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.336+0000 7fee7e403640 1 -- 172.21.15.32:0/3783570985 learned_addr learned my addr 172.21.15.32:0/3783570985 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.336+0000 7fee7dc02640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee78157480 0x7fee780c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:40.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.336+0000 7fee7d401640 1 --2- 172.21.15.32:0/3783570985 >> v2:172.21.15.39:3300/0 conn(0x7fee780b0760 0x7fee780b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:40.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4237732054 0 0) 0x7fee780b1100 con 0x7fee7815d660 2023-12-20T06:02:40.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fee64003530 con 0x7fee7815d660 2023-12-20T06:02:40.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3978997660 0 0) 0x7fee64003530 con 0x7fee7815d660 2023-12-20T06:02:40.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee78157480 msgr2=0x7fee780c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:40.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee78157480 0x7fee780c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:40.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 >> v2:172.21.15.39:3300/0 conn(0x7fee780b0760 msgr2=0x7fee780b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 --2- 172.21.15.32:0/3783570985 >> v2:172.21.15.39:3300/0 conn(0x7fee780b0760 0x7fee780b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee780ce070 con 0x7fee7815d660 2023-12-20T06:02:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fee70001a00 con 0x7fee7815d660 2023-12-20T06:02:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 >> v1:172.21.15.32:6789/0 conn(0x7fee7815d660 legacy=0x7fee7815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee64003980 0x7fee64003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640047d0 0x7fee64004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee78157480 0x7fee64005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780cde20 con 0x7fee640047d0 2023-12-20T06:02:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780c8e30 con 0x7fee64003980 2023-12-20T06:02:40.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee6400b990 con 0x7fee78157480 2023-12-20T06:02:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7d401640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640047d0 0x7fee64004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7dc02640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee64003980 0x7fee64003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7e403640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee78157480 0x7fee64005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7d401640 1 -- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee78157480 msgr2=0x7fee64005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7d401640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee78157480 0x7fee64005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7d401640 1 -- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee64003980 msgr2=0x7fee64003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7d401640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee64003980 0x7fee64003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7d401640 1 -- 172.21.15.32:0/3783570985 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee64003530 con 0x7fee640047d0 2023-12-20T06:02:40.341 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.340+0000 7fee7dc02640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee64003980 0x7fee64003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7d401640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640047d0 0x7fee64004c50 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fee68009b10 tx=0x7fee6801ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=184fcb5b5c20f58a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680a5550 con 0x7fee640047d0 2023-12-20T06:02:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee680a56b0 con 0x7fee640047d0 2023-12-20T06:02:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7cc00640 1 -- 172.21.15.32:0/3783570985 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee680a5a60 con 0x7fee640047d0 2023-12-20T06:02:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7fe66640 1 -- 172.21.15.32:0/3783570985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640047d0 msgr2=0x7fee64004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7fe66640 1 --2- 172.21.15.32:0/3783570985 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee640047d0 0x7fee64004c50 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7fee68009b10 tx=0x7fee6801ca60 comp rx=0 tx=0).stop 2023-12-20T06:02:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7e403640 1 -- 172.21.15.32:0/3783570985 reap_dead start 2023-12-20T06:02:40.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7fe66640 1 -- 172.21.15.32:0/3783570985 shutdown_connections 2023-12-20T06:02:40.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7fe66640 1 -- 172.21.15.32:0/3783570985 >> 172.21.15.32:0/3783570985 conn(0x7fee7806a510 msgr2=0x7fee780ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:40.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7fe66640 1 -- 172.21.15.32:0/3783570985 shutdown_connections 2023-12-20T06:02:40.344 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7fe66640 1 -- 172.21.15.32:0/3783570985 wait complete. 2023-12-20T06:02:40.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7fe66640 1 Processor -- start 2023-12-20T06:02:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.344+0000 7fee7fe66640 1 -- start start 2023-12-20T06:02:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7fe66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640047d0 0x7fee781f4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7fe66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee780c7510 0x7fee780c7990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7fe66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee780c62c0 0x7fee780c6740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7fe66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee78150100 con 0x7fee780c62c0 2023-12-20T06:02:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7fe66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee780c7ed0 con 0x7fee780c7510 2023-12-20T06:02:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7fe66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee780c6c80 con 0x7fee640047d0 2023-12-20T06:02:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7e403640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee780c62c0 0x7fee780c6740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7e403640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee780c62c0 0x7fee780c6740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46776/0 (socket says 172.21.15.32:46776) 2023-12-20T06:02:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7e403640 1 -- 172.21.15.32:0/4036124604 learned_addr learned my addr 172.21.15.32:0/4036124604 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:40.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7dc02640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640047d0 0x7fee781f4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7d401640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee780c7510 0x7fee780c7990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7e403640 1 -- 172.21.15.32:0/4036124604 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640047d0 msgr2=0x7fee781f4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7e403640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640047d0 0x7fee781f4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7e403640 1 -- 172.21.15.32:0/4036124604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee780c7510 msgr2=0x7fee780c7990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7e403640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee780c7510 0x7fee780c7990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7e403640 1 -- 172.21.15.32:0/4036124604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee7814cb90 con 0x7fee780c62c0 2023-12-20T06:02:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7dc02640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640047d0 0x7fee781f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7d401640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee780c7510 0x7fee780c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7e403640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee780c62c0 0x7fee780c6740 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fee70000f90 tx=0x7fee7000e530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee6e7fc640 1 -- 172.21.15.32:0/4036124604 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee7000eb30 con 0x7fee780c62c0 2023-12-20T06:02:40.348 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee6e7fc640 1 -- 172.21.15.32:0/4036124604 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee7000ec90 con 0x7fee780c62c0 2023-12-20T06:02:40.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee6e7fc640 1 -- 172.21.15.32:0/4036124604 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee70014430 con 0x7fee780c62c0 2023-12-20T06:02:40.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7fe66640 1 -- 172.21.15.32:0/4036124604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fee780c71b0 con 0x7fee780c62c0 2023-12-20T06:02:40.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7fe66640 1 -- 172.21.15.32:0/4036124604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee780c6ed0 con 0x7fee780c62c0 2023-12-20T06:02:40.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee7fe66640 1 -- 172.21.15.32:0/4036124604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee7804e000 con 0x7fee780c62c0 2023-12-20T06:02:40.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.348+0000 7fee6e7fc640 1 -- 172.21.15.32:0/4036124604 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fee70014590 con 0x7fee780c62c0 2023-12-20T06:02:40.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.352+0000 7fee6e7fc640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee5c065b80 0x7fee5c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:40.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.352+0000 7fee6e7fc640 1 -- 172.21.15.32:0/4036124604 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(212..212 src has 1..212) v4 ==== 6629+0+0 (secure 0 0 0) 0x7fee70083420 con 0x7fee780c62c0 2023-12-20T06:02:40.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.352+0000 7fee7dc02640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee5c065b80 0x7fee5c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:40.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.352+0000 7fee7dc02640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee5c065b80 0x7fee5c068040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee74009a90 tx=0x7fee74008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:40.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.356+0000 7fee6e7fc640 1 -- 172.21.15.32:0/4036124604 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee70050920 con 0x7fee780c62c0 2023-12-20T06:02:40.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:40.548+0000 7fee7fe66640 1 -- 172.21.15.32:0/4036124604 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fee780b0360 con 0x7fee780c62c0 2023-12-20T06:02:41.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.508+0000 7fee6e7fc640 1 -- 172.21.15.32:0/4036124604 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v101) v1 ==== 121+0+0 (secure 0 0 0) 0x7fee70053f40 con 0x7fee780c62c0 2023-12-20T06:02:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 -- 172.21.15.32:0/4036124604 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee5c065b80 msgr2=0x7fee5c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee5c065b80 0x7fee5c068040 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fee74009a90 tx=0x7fee74008040 comp rx=0 tx=0).stop 2023-12-20T06:02:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 -- 172.21.15.32:0/4036124604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee780c62c0 msgr2=0x7fee780c6740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee780c62c0 0x7fee780c6740 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7fee70000f90 tx=0x7fee7000e530 comp rx=0 tx=0).stop 2023-12-20T06:02:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 -- 172.21.15.32:0/4036124604 shutdown_connections 2023-12-20T06:02:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fee5c065b80 0x7fee5c068040 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee780c62c0 0x7fee780c6740 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee780c7510 0x7fee780c7990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 --2- 172.21.15.32:0/4036124604 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee640047d0 0x7fee781f4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:41.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 -- 172.21.15.32:0/4036124604 >> 172.21.15.32:0/4036124604 conn(0x7fee7806a510 msgr2=0x7fee780cba80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 -- 172.21.15.32:0/4036124604 shutdown_connections 2023-12-20T06:02:41.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.512+0000 7fee7fe66640 1 -- 172.21.15.32:0/4036124604 wait complete. 2023-12-20T06:02:41.534 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T06:02:41.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-12-20T06:02:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f18623db640 1 Processor -- start 2023-12-20T06:02:41.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f18623db640 1 -- start start 2023-12-20T06:02:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f18623db640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f185c1591e0 0x7f185c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f18623db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c06cdf0 0x7f185c070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f18623db640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f185c0cc3b0 con 0x7f185c070770 2023-12-20T06:02:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f18623db640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f185c070be0 con 0x7f185c1591e0 2023-12-20T06:02:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f18623db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185c0c8e30 con 0x7f185c06cdf0 2023-12-20T06:02:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f185b7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c06cdf0 0x7f185c070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f185bfff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f185c1591e0 0x7f185c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f185b7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c06cdf0 0x7f185c070230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:59746/0 (socket says 172.21.15.32:59746) 2023-12-20T06:02:41.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f185b7fe640 1 -- 172.21.15.32:0/3429884519 learned_addr learned my addr 172.21.15.32:0/3429884519 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:41.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f185b7fe640 1 -- 172.21.15.32:0/3429884519 >> v2:172.21.15.39:3300/0 conn(0x7f185c1591e0 msgr2=0x7f185c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:41.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f185b7fe640 1 --2- 172.21.15.32:0/3429884519 >> v2:172.21.15.39:3300/0 conn(0x7f185c1591e0 0x7f185c0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:41.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f185b7fe640 1 -- 172.21.15.32:0/3429884519 >> v1:172.21.15.39:6789/0 conn(0x7f185c070770 legacy=0x7f185c15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:41.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.688+0000 7f185b7fe640 1 -- 172.21.15.32:0/3429884519 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f185c06d280 con 0x7f185c06cdf0 2023-12-20T06:02:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f185b7fe640 1 --2- 172.21.15.32:0/3429884519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c06cdf0 0x7f185c070230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1850009870 tx=0x7f185001c4b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=1a10abfc90d448e0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f185affd640 1 -- 172.21.15.32:0/3429884519 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18500ab600 con 0x7f185c06cdf0 2023-12-20T06:02:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f185affd640 1 -- 172.21.15.32:0/3429884519 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18500a7050 con 0x7f185c06cdf0 2023-12-20T06:02:41.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f185affd640 1 -- 172.21.15.32:0/3429884519 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18500ab980 con 0x7f185c06cdf0 2023-12-20T06:02:41.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f18623db640 1 -- 172.21.15.32:0/3429884519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c06cdf0 msgr2=0x7f185c070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:41.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f18623db640 1 --2- 172.21.15.32:0/3429884519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c06cdf0 0x7f185c070230 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f1850009870 tx=0x7f185001c4b0 comp rx=0 tx=0).stop 2023-12-20T06:02:41.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f18623db640 1 -- 172.21.15.32:0/3429884519 shutdown_connections 2023-12-20T06:02:41.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f18623db640 1 --2- 172.21.15.32:0/3429884519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c06cdf0 0x7f185c070230 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:41.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f18623db640 1 --2- 172.21.15.32:0/3429884519 >> v2:172.21.15.39:3300/0 conn(0x7f185c1591e0 0x7f185c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:41.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f18623db640 1 -- 172.21.15.32:0/3429884519 >> 172.21.15.32:0/3429884519 conn(0x7f185c06a510 msgr2=0x7f185c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:41.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f18623db640 1 -- 172.21.15.32:0/3429884519 shutdown_connections 2023-12-20T06:02:41.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.692+0000 7f18623db640 1 -- 172.21.15.32:0/3429884519 wait complete. 2023-12-20T06:02:41.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f18623db640 1 Processor -- start 2023-12-20T06:02:41.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f18623db640 1 -- start start 2023-12-20T06:02:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f18623db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c070770 0x7f185c14efa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f18623db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c1591e0 0x7f185c14ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f18623db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f185c14b220 0x7f185c140000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:41.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f18623db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c14b6a0 con 0x7f185c1591e0 2023-12-20T06:02:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f18623db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f185c14b980 con 0x7f185c14b220 2023-12-20T06:02:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f18623db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f185c14bc60 con 0x7f185c070770 2023-12-20T06:02:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f185b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c1591e0 0x7f185c14ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f185b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c1591e0 0x7f185c14ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46790/0 (socket says 172.21.15.32:46790) 2023-12-20T06:02:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f185b7fe640 1 -- 172.21.15.32:0/1201604978 learned_addr learned my addr 172.21.15.32:0/1201604978 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f185bfff640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c070770 0x7f185c14efa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f1860978640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f185c14b220 0x7f185c140000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f185b7fe640 1 -- 172.21.15.32:0/1201604978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c070770 msgr2=0x7f185c14efa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f185b7fe640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c070770 0x7f185c14efa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f185b7fe640 1 -- 172.21.15.32:0/1201604978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f185c14b220 msgr2=0x7f185c140000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:41.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f185b7fe640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f185c14b220 0x7f185c140000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f185b7fe640 1 -- 172.21.15.32:0/1201604978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18500094d0 con 0x7f185c1591e0 2023-12-20T06:02:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f185bfff640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c070770 0x7f185c14efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f185b7fe640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c1591e0 0x7f185c14ace0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f185001a0e0 tx=0x7f1850019250 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f1858ff9640 1 -- 172.21.15.32:0/1201604978 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18500b4440 con 0x7f185c1591e0 2023-12-20T06:02:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f1858ff9640 1 -- 172.21.15.32:0/1201604978 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18500b45a0 con 0x7f185c1591e0 2023-12-20T06:02:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f18623db640 1 -- 172.21.15.32:0/1201604978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f185c14bf40 con 0x7f185c1591e0 2023-12-20T06:02:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.696+0000 7f1858ff9640 1 -- 172.21.15.32:0/1201604978 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18500b4930 con 0x7f185c1591e0 2023-12-20T06:02:41.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.700+0000 7f18623db640 1 -- 172.21.15.32:0/1201604978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f185c14f4e0 con 0x7f185c1591e0 2023-12-20T06:02:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.700+0000 7f18623db640 1 -- 172.21.15.32:0/1201604978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f185c04e000 con 0x7f185c1591e0 2023-12-20T06:02:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.700+0000 7f1858ff9640 1 -- 172.21.15.32:0/1201604978 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f18500b2070 con 0x7f185c1591e0 2023-12-20T06:02:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.700+0000 7f1858ff9640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1848065b80 0x7f1848068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:41.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.700+0000 7f1858ff9640 1 -- 172.21.15.32:0/1201604978 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f185011d480 con 0x7f185c1591e0 2023-12-20T06:02:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.700+0000 7f185bfff640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1848065b80 0x7f1848068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:41.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.700+0000 7f185bfff640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1848065b80 0x7f1848068040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f184c009b10 tx=0x7f184c008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:41.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.704+0000 7f1858ff9640 1 -- 172.21.15.32:0/1201604978 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f18500eaa30 con 0x7f185c1591e0 2023-12-20T06:02:41.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:41.896+0000 7f18623db640 1 -- 172.21.15.32:0/1201604978 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f185c14b7e0 con 0x7f185c1591e0 2023-12-20T06:02:42.121 INFO:tasks.ceph.mds.d.smithi039.stderr:2023-12-20T06:02:42.120+0000 7ffbd7ed2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T06:02:42.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.512+0000 7f1858ff9640 1 -- 172.21.15.32:0/1201604978 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v102) v1 ==== 125+0+0 (secure 0 0 0) 0x7f185c14b7e0 con 0x7f185c1591e0 2023-12-20T06:02:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 -- 172.21.15.32:0/1201604978 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1848065b80 msgr2=0x7f1848068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1848065b80 0x7f1848068040 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f184c009b10 tx=0x7f184c008040 comp rx=0 tx=0).stop 2023-12-20T06:02:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 -- 172.21.15.32:0/1201604978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c1591e0 msgr2=0x7f185c14ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c1591e0 0x7f185c14ace0 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f185001a0e0 tx=0x7f1850019250 comp rx=0 tx=0).stop 2023-12-20T06:02:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 -- 172.21.15.32:0/1201604978 shutdown_connections 2023-12-20T06:02:42.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1848065b80 0x7f1848068040 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f185c14b220 0x7f185c140000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f185c1591e0 0x7f185c14ace0 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 --2- 172.21.15.32:0/1201604978 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f185c070770 0x7f185c14efa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 -- 172.21.15.32:0/1201604978 >> 172.21.15.32:0/1201604978 conn(0x7f185c06a510 msgr2=0x7f185c166570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 -- 172.21.15.32:0/1201604978 shutdown_connections 2023-12-20T06:02:42.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.516+0000 7f18623db640 1 -- 172.21.15.32:0/1201604978 wait complete. 2023-12-20T06:02:42.537 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-12-20T06:02:42.537 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:02:42.622 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:02:42.626 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:02:42.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.688+0000 7ff24c5e9640 1 Processor -- start 2023-12-20T06:02:42.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.688+0000 7ff24c5e9640 1 -- start start 2023-12-20T06:02:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.688+0000 7ff24c5e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff24415f110 0x7ff24415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.688+0000 7ff24c5e9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24415b990 con 0x7ff2440af5e0 2023-12-20T06:02:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.688+0000 7ff24c5e9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff24415bbd0 con 0x7ff24406ce60 2023-12-20T06:02:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.688+0000 7ff24c5e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff24415beb0 con 0x7ff24415f110 2023-12-20T06:02:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff24a385640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff2440af5e0 0x7ff2440af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60454/0 (socket says 172.21.15.32:60454) 2023-12-20T06:02:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff24a385640 1 -- 172.21.15.32:0/1277985742 learned_addr learned my addr 172.21.15.32:0/1277985742 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:42.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff24ab86640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff24415f110 0x7ff24415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2156072687 0 0) 0x7ff2400012a0 con 0x7ff2440af5e0 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff230003530 con 0x7ff2440af5e0 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1695499191 0 0) 0x7ff230003530 con 0x7ff2440af5e0 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff24415f110 msgr2=0x7ff24415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff24415f110 0x7ff24415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 >> v1:172.21.15.39:6789/0 conn(0x7ff24406ce60 legacy=0x7ff24415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff24415e910 con 0x7ff2440af5e0 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7ff240001990 con 0x7ff2440af5e0 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 >> v1:172.21.15.32:6789/0 conn(0x7ff2440af5e0 legacy=0x7ff2440af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2300039b0 0x7ff230003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff230004360 0x7ff230004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:42.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff230005080 0x7ff23000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:42.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24415beb0 con 0x7ff230005080 2023-12-20T06:02:42.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff23000bfb0 con 0x7ff2300039b0 2023-12-20T06:02:42.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff23000c170 con 0x7ff230004360 2023-12-20T06:02:42.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff24ab86640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff24415f110 0x7ff24415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:42.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff24ab86640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff230005080 0x7ff23000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:42.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff24a385640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2300039b0 0x7ff230003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:42.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff249b84640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff230004360 0x7ff230004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:42.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff24ab86640 1 -- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff230004360 msgr2=0x7ff230004830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:42.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff24ab86640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff230004360 0x7ff230004830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff24ab86640 1 -- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2300039b0 msgr2=0x7ff230003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:42.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff24ab86640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2300039b0 0x7ff230003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.692+0000 7ff24ab86640 1 -- 172.21.15.32:0/1277985742 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff230003530 con 0x7ff230005080 2023-12-20T06:02:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff24ab86640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff230005080 0x7ff23000b990 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00e1f0 tx=0x7ff23c01cd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f32fff75dbb6ec3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c0a5d50 con 0x7ff230005080 2023-12-20T06:02:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23c00f040 con 0x7ff230005080 2023-12-20T06:02:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff249383640 1 -- 172.21.15.32:0/1277985742 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23c003bc0 con 0x7ff230005080 2023-12-20T06:02:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1277985742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff230005080 msgr2=0x7ff23000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff24c5e9640 1 --2- 172.21.15.32:0/1277985742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff230005080 0x7ff23000b990 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7ff23c00e1f0 tx=0x7ff23c01cd00 comp rx=0 tx=0).stop 2023-12-20T06:02:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff24ab86640 1 -- 172.21.15.32:0/1277985742 reap_dead start 2023-12-20T06:02:42.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1277985742 shutdown_connections 2023-12-20T06:02:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1277985742 >> 172.21.15.32:0/1277985742 conn(0x7ff24406a320 msgr2=0x7ff2440ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1277985742 shutdown_connections 2023-12-20T06:02:42.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1277985742 wait complete. 2023-12-20T06:02:42.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff24c5e9640 1 Processor -- start 2023-12-20T06:02:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.696+0000 7ff24c5e9640 1 -- start start 2023-12-20T06:02:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24c5e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff230005080 0x7ff24414ec90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24c5e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2440701c0 0x7ff244070f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24c5e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2441e54e0 0x7ff24414dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24c5e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff24414eab0 con 0x7ff2440701c0 2023-12-20T06:02:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24c5e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff244070ae0 con 0x7ff230005080 2023-12-20T06:02:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24c5e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2440707b0 con 0x7ff2441e54e0 2023-12-20T06:02:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff249b84640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2440701c0 0x7ff244070f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24a385640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff230005080 0x7ff24414ec90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff249b84640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2440701c0 0x7ff244070f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46812/0 (socket says 172.21.15.32:46812) 2023-12-20T06:02:42.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24a385640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff230005080 0x7ff24414ec90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:37974/0 (socket says 172.21.15.32:37974) 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24a385640 1 -- 172.21.15.32:0/1693323911 learned_addr learned my addr 172.21.15.32:0/1693323911 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24ab86640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2441e54e0 0x7ff24414dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff249b84640 1 -- 172.21.15.32:0/1693323911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2441e54e0 msgr2=0x7ff24414dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff249b84640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2441e54e0 0x7ff24414dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff249b84640 1 -- 172.21.15.32:0/1693323911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff230005080 msgr2=0x7ff24414ec90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff249b84640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff230005080 0x7ff24414ec90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff249b84640 1 -- 172.21.15.32:0/1693323911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff244071e30 con 0x7ff2440701c0 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24a385640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff230005080 0x7ff24414ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24ab86640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2441e54e0 0x7ff24414dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff249b84640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2440701c0 0x7ff244070f70 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7ff234011c30 tx=0x7ff23400f5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff23affd640 1 -- 172.21.15.32:0/1693323911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23400fc50 con 0x7ff2440701c0 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff23affd640 1 -- 172.21.15.32:0/1693323911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff23400fdd0 con 0x7ff2440701c0 2023-12-20T06:02:42.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff23affd640 1 -- 172.21.15.32:0/1693323911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2340050d0 con 0x7ff2440701c0 2023-12-20T06:02:42.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1693323911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff244072110 con 0x7ff2440701c0 2023-12-20T06:02:42.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1693323911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2440714b0 con 0x7ff2440701c0 2023-12-20T06:02:42.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.700+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1693323911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff24415ebd0 con 0x7ff2440701c0 2023-12-20T06:02:42.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.704+0000 7ff23affd640 1 -- 172.21.15.32:0/1693323911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff234005230 con 0x7ff2440701c0 2023-12-20T06:02:42.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.704+0000 7ff23affd640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff228065bb0 0x7ff228068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:42.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.704+0000 7ff23affd640 1 -- 172.21.15.32:0/1693323911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7ff2340865f0 con 0x7ff2440701c0 2023-12-20T06:02:42.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.704+0000 7ff24a385640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff228065bb0 0x7ff228068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:42.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.704+0000 7ff24a385640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff228065bb0 0x7ff228068070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff2441e60d0 tx=0x7ff240007e70 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:42.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.708+0000 7ff23affd640 1 -- 172.21.15.32:0/1693323911 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff24415ebd0 con 0x7ff2440701c0 2023-12-20T06:02:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.896+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1693323911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff24415ebd0 con 0x7ff2440701c0 2023-12-20T06:02:42.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.896+0000 7ff23affd640 1 -- 172.21.15.32:0/1693323911 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5902 (secure 0 0 0) 0x7ff24415ebd0 con 0x7ff2440701c0 2023-12-20T06:02:42.896 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:42.896 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:26.942526+0000","modified":"2023-12-20T06:02:29.955675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10539":{"gid":10539,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/888710554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":888710554},{"type":"v1","addr":"172.21.15.32:6837","nonce":888710554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:39.446992+0000","modified":"2023-12-20T06:02:42.509418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10530":{"gid":10530,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/3001088147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3001088147},{"type":"v1","addr":"172.21.15.39:6833","nonce":3001088147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-20T06:02:42.896 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 102 2023-12-20T06:02:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1693323911 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff228065bb0 msgr2=0x7ff228068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff228065bb0 0x7ff228068070 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7ff2441e60d0 tx=0x7ff240007e70 comp rx=0 tx=0).stop 2023-12-20T06:02:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1693323911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2440701c0 msgr2=0x7ff244070f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2440701c0 0x7ff244070f70 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7ff234011c30 tx=0x7ff23400f5a0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1693323911 shutdown_connections 2023-12-20T06:02:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff228065bb0 0x7ff228068070 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2441e54e0 0x7ff24414dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2440701c0 0x7ff244070f70 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 --2- 172.21.15.32:0/1693323911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff230005080 0x7ff24414ec90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1693323911 >> 172.21.15.32:0/1693323911 conn(0x7ff24406a320 msgr2=0x7ff2440ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1693323911 shutdown_connections 2023-12-20T06:02:42.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:42.900+0000 7ff24c5e9640 1 -- 172.21.15.32:0/1693323911 wait complete. 2023-12-20T06:02:42.917 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d8a873640 1 Processor -- start 2023-12-20T06:02:43.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d8a873640 1 -- start start 2023-12-20T06:02:43.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d8a873640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d840af5e0 0x7f0d840af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d8a873640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0d8415f110 0x7f0d8415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d8a873640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0d8415b990 con 0x7f0d8406ce60 2023-12-20T06:02:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d8a873640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0d8415bbd0 con 0x7f0d8415f110 2023-12-20T06:02:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d8a873640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d8415beb0 con 0x7f0d840af5e0 2023-12-20T06:02:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d88e10640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0d8415f110 0x7f0d8415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d83fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d840af5e0 0x7f0d840af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d88e10640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0d8415f110 0x7f0d8415c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:37986/0 (socket says 172.21.15.32:37986) 2023-12-20T06:02:43.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d88e10640 1 -- 172.21.15.32:0/4193956009 learned_addr learned my addr 172.21.15.32:0/4193956009 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:43.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d83fff640 1 -- 172.21.15.32:0/4193956009 >> v2:172.21.15.39:3300/0 conn(0x7f0d8415f110 msgr2=0x7f0d8415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:43.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d83fff640 1 --2- 172.21.15.32:0/4193956009 >> v2:172.21.15.39:3300/0 conn(0x7f0d8415f110 0x7f0d8415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d83fff640 1 -- 172.21.15.32:0/4193956009 >> v1:172.21.15.39:6789/0 conn(0x7f0d8406ce60 legacy=0x7f0d8415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d82ffd640 1 -- 172.21.15.32:0/4193956009 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2845796569 0 0) 0x7f0d78001120 con 0x7f0d8406ce60 2023-12-20T06:02:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d83fff640 1 -- 172.21.15.32:0/4193956009 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d8415e910 con 0x7f0d840af5e0 2023-12-20T06:02:43.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.112+0000 7f0d88e10640 1 --2- 172.21.15.32:0/4193956009 >> v2:172.21.15.39:3300/0 conn(0x7f0d8415f110 0x7f0d8415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:43.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d83fff640 1 --2- 172.21.15.32:0/4193956009 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d840af5e0 0x7f0d840af9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0d740097a0 tx=0x7f0d7401c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3a8c661844d4cbf0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d82ffd640 1 -- 172.21.15.32:0/4193956009 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d740ab650 con 0x7f0d840af5e0 2023-12-20T06:02:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d82ffd640 1 -- 172.21.15.32:0/4193956009 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d740ab7b0 con 0x7f0d840af5e0 2023-12-20T06:02:43.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d82ffd640 1 -- 172.21.15.32:0/4193956009 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d740abb40 con 0x7f0d840af5e0 2023-12-20T06:02:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d8a873640 1 -- 172.21.15.32:0/4193956009 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d840af5e0 msgr2=0x7f0d840af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d8a873640 1 --2- 172.21.15.32:0/4193956009 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d840af5e0 0x7f0d840af9c0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f0d740097a0 tx=0x7f0d7401c430 comp rx=0 tx=0).stop 2023-12-20T06:02:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d8a873640 1 -- 172.21.15.32:0/4193956009 shutdown_connections 2023-12-20T06:02:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d8a873640 1 --2- 172.21.15.32:0/4193956009 >> v2:172.21.15.39:3300/0 conn(0x7f0d8415f110 0x7f0d8415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d8a873640 1 --2- 172.21.15.32:0/4193956009 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d840af5e0 0x7f0d840af9c0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d8a873640 1 -- 172.21.15.32:0/4193956009 >> 172.21.15.32:0/4193956009 conn(0x7f0d8406a320 msgr2=0x7f0d840ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:43.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d8a873640 1 -- 172.21.15.32:0/4193956009 shutdown_connections 2023-12-20T06:02:43.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.116+0000 7f0d8a873640 1 -- 172.21.15.32:0/4193956009 wait complete. 2023-12-20T06:02:43.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d8a873640 1 Processor -- start 2023-12-20T06:02:43.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d8a873640 1 -- start start 2023-12-20T06:02:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d8a873640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d8406ce60 0x7f0d841f7130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d8a873640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840af5e0 0x7f0d84159610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d8a873640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d8415f110 0x7f0d84159b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d8a873640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840cca10 con 0x7f0d8415f110 2023-12-20T06:02:43.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d8a873640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0d840705b0 con 0x7f0d840af5e0 2023-12-20T06:02:43.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d8a873640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0d84070890 con 0x7f0d8406ce60 2023-12-20T06:02:43.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d88e10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d8415f110 0x7f0d84159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d88e10640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d8415f110 0x7f0d84159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46822/0 (socket says 172.21.15.32:46822) 2023-12-20T06:02:43.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d88e10640 1 -- 172.21.15.32:0/2449381476 learned_addr learned my addr 172.21.15.32:0/2449381476 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:43.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d837fe640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840af5e0 0x7f0d84159610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d83fff640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d8406ce60 0x7f0d841f7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d88e10640 1 -- 172.21.15.32:0/2449381476 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d8406ce60 msgr2=0x7f0d841f7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d88e10640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d8406ce60 0x7f0d841f7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d88e10640 1 -- 172.21.15.32:0/2449381476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840af5e0 msgr2=0x7f0d84159610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d88e10640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840af5e0 0x7f0d84159610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d88e10640 1 -- 172.21.15.32:0/2449381476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0d7c009b30 con 0x7f0d8415f110 2023-12-20T06:02:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d837fe640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840af5e0 0x7f0d84159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d88e10640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d8415f110 0x7f0d84159b50 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00efc0 tx=0x7f0d7c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d80ff9640 1 -- 172.21.15.32:0/2449381476 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c018070 con 0x7f0d8415f110 2023-12-20T06:02:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d80ff9640 1 -- 172.21.15.32:0/2449381476 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0d7c00f040 con 0x7f0d8415f110 2023-12-20T06:02:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d80ff9640 1 -- 172.21.15.32:0/2449381476 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0d7c00ce00 con 0x7f0d8415f110 2023-12-20T06:02:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d8a873640 1 -- 172.21.15.32:0/2449381476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0d740093b0 con 0x7f0d8415f110 2023-12-20T06:02:43.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.120+0000 7f0d8a873640 1 -- 172.21.15.32:0/2449381476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0d84070e50 con 0x7f0d8415f110 2023-12-20T06:02:43.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.124+0000 7f0d8a873640 1 -- 172.21.15.32:0/2449381476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0d8404e000 con 0x7f0d8415f110 2023-12-20T06:02:43.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.124+0000 7f0d80ff9640 1 -- 172.21.15.32:0/2449381476 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f0d7c014480 con 0x7f0d8415f110 2023-12-20T06:02:43.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.124+0000 7f0d80ff9640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d70065b80 0x7f0d70068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.124+0000 7f0d80ff9640 1 -- 172.21.15.32:0/2449381476 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f0d7c083a70 con 0x7f0d8415f110 2023-12-20T06:02:43.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.124+0000 7f0d83fff640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d70065b80 0x7f0d70068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.124+0000 7f0d83fff640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d70065b80 0x7f0d70068040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d740af000 tx=0x7f0d740a7000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:43.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.128+0000 7f0d80ff9640 1 -- 172.21.15.32:0/2449381476 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0d7c050e60 con 0x7f0d8415f110 2023-12-20T06:02:43.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.284+0000 7f0d8a873640 1 -- 172.21.15.32:0/2449381476 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f0d840cf450 con 0x7f0d8415f110 2023-12-20T06:02:43.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d80ff9640 1 -- 172.21.15.32:0/2449381476 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v213) v1 ==== 74+0+19881 (secure 0 0 0) 0x7f0d7c054480 con 0x7f0d8415f110 2023-12-20T06:02:43.285 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:43.286 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":213,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:40.449354+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:02:25.045126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-20T06:02:25.938205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:02:37.109832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:02:38.440830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 -- 172.21.15.32:0/2449381476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d70065b80 msgr2=0x7f0d70068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d70065b80 0x7f0d70068040 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0d740af000 tx=0x7f0d740a7000 comp rx=0 tx=0).stop 2023-12-20T06:02:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 -- 172.21.15.32:0/2449381476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d8415f110 msgr2=0x7f0d84159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d8415f110 0x7f0d84159b50 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f0d7c00efc0 tx=0x7f0d7c00c490 comp rx=0 tx=0).stop 2023-12-20T06:02:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 -- 172.21.15.32:0/2449381476 shutdown_connections 2023-12-20T06:02:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0d70065b80 0x7f0d70068040 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0d8415f110 0x7f0d84159b50 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0d840af5e0 0x7f0d84159610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 --2- 172.21.15.32:0/2449381476 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0d8406ce60 0x7f0d841f7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 -- 172.21.15.32:0/2449381476 >> 172.21.15.32:0/2449381476 conn(0x7f0d8406a320 msgr2=0x7f0d840cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 -- 172.21.15.32:0/2449381476 shutdown_connections 2023-12-20T06:02:43.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.288+0000 7f0d8a873640 1 -- 172.21.15.32:0/2449381476 wait complete. 2023-12-20T06:02:43.307 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T06:02:43.307 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:02:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.452+0000 7f9a650d1640 1 Processor -- start 2023-12-20T06:02:43.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.452+0000 7f9a650d1640 1 -- start start 2023-12-20T06:02:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.452+0000 7f9a650d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9a6015ce30 0x7f9a60161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.452+0000 7f9a650d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 0x7f9a600cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.452+0000 7f9a650d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9a60071920 con 0x7f9a60070950 2023-12-20T06:02:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.452+0000 7f9a650d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9a60071080 con 0x7f9a6015ce30 2023-12-20T06:02:43.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.452+0000 7f9a650d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a60071360 con 0x7f9a6006deb0 2023-12-20T06:02:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5f59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 0x7f9a600cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5ed9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9a6015ce30 0x7f9a60161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5f59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 0x7f9a600cd520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:59812/0 (socket says 172.21.15.32:59812) 2023-12-20T06:02:43.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5f59e640 1 -- 172.21.15.32:0/3576322573 learned_addr learned my addr 172.21.15.32:0/3576322573 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5f59e640 1 -- 172.21.15.32:0/3576322573 >> v2:172.21.15.39:3300/0 conn(0x7f9a6015ce30 msgr2=0x7f9a60161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5f59e640 1 --2- 172.21.15.32:0/3576322573 >> v2:172.21.15.39:3300/0 conn(0x7f9a6015ce30 0x7f9a60161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5f59e640 1 -- 172.21.15.32:0/3576322573 >> v1:172.21.15.39:6789/0 conn(0x7f9a60070950 legacy=0x7f9a6006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5e59c640 1 -- 172.21.15.32:0/3576322573 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2693566077 0 0) 0x7f9a60071920 con 0x7f9a60070950 2023-12-20T06:02:43.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5f59e640 1 -- 172.21.15.32:0/3576322573 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a60071640 con 0x7f9a6006deb0 2023-12-20T06:02:43.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5f59e640 1 --2- 172.21.15.32:0/3576322573 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 0x7f9a600cd520 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009e50 tx=0x7f9a5801c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=467eb04ab4d663f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:43.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5e59c640 1 -- 172.21.15.32:0/3576322573 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a580a56d0 con 0x7f9a6006deb0 2023-12-20T06:02:43.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5e59c640 1 -- 172.21.15.32:0/3576322573 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a580a5830 con 0x7f9a6006deb0 2023-12-20T06:02:43.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.456+0000 7f9a5e59c640 1 -- 172.21.15.32:0/3576322573 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a580a5bc0 con 0x7f9a6006deb0 2023-12-20T06:02:43.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 -- 172.21.15.32:0/3576322573 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 msgr2=0x7f9a600cd520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 --2- 172.21.15.32:0/3576322573 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 0x7f9a600cd520 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009e50 tx=0x7f9a5801c980 comp rx=0 tx=0).stop 2023-12-20T06:02:43.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 -- 172.21.15.32:0/3576322573 shutdown_connections 2023-12-20T06:02:43.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 --2- 172.21.15.32:0/3576322573 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 0x7f9a600cd520 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 --2- 172.21.15.32:0/3576322573 >> v2:172.21.15.39:3300/0 conn(0x7f9a6015ce30 0x7f9a60161560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 -- 172.21.15.32:0/3576322573 >> 172.21.15.32:0/3576322573 conn(0x7f9a600cba50 msgr2=0x7f9a600cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:43.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 -- 172.21.15.32:0/3576322573 shutdown_connections 2023-12-20T06:02:43.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 -- 172.21.15.32:0/3576322573 wait complete. 2023-12-20T06:02:43.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 Processor -- start 2023-12-20T06:02:43.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 -- start start 2023-12-20T06:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 0x7f9a60157d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a60070950 0x7f9a6015df20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6015ce30 0x7f9a6015e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6009f480 con 0x7f9a6015ce30 2023-12-20T06:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6009e870 con 0x7f9a60070950 2023-12-20T06:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a650d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9a6009eb50 con 0x7f9a6006deb0 2023-12-20T06:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a5f59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6015ce30 0x7f9a6015e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a5f59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6015ce30 0x7f9a6015e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46836/0 (socket says 172.21.15.32:46836) 2023-12-20T06:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.460+0000 7f9a5f59e640 1 -- 172.21.15.32:0/1526765122 learned_addr learned my addr 172.21.15.32:0/1526765122 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a5ed9d640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 0x7f9a60157d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a57fff640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a60070950 0x7f9a6015df20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a5f59e640 1 -- 172.21.15.32:0/1526765122 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 msgr2=0x7f9a60157d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a5f59e640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 0x7f9a60157d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a5f59e640 1 -- 172.21.15.32:0/1526765122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a60070950 msgr2=0x7f9a6015df20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a5f59e640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a60070950 0x7f9a6015df20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a5f59e640 1 -- 172.21.15.32:0/1526765122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9a58009b30 con 0x7f9a6015ce30 2023-12-20T06:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a5f59e640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6015ce30 0x7f9a6015e460 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f9a5801a5b0 tx=0x7f9a58019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a577fe640 1 -- 172.21.15.32:0/1526765122 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a580b8670 con 0x7f9a6015ce30 2023-12-20T06:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a577fe640 1 -- 172.21.15.32:0/1526765122 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9a580b87d0 con 0x7f9a6015ce30 2023-12-20T06:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a577fe640 1 -- 172.21.15.32:0/1526765122 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9a580b8b80 con 0x7f9a6015ce30 2023-12-20T06:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a650d1640 1 -- 172.21.15.32:0/1526765122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9a6009ee30 con 0x7f9a6015ce30 2023-12-20T06:02:43.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a650d1640 1 -- 172.21.15.32:0/1526765122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9a6009e520 con 0x7f9a6015ce30 2023-12-20T06:02:43.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a650d1640 1 -- 172.21.15.32:0/1526765122 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9a6009f5f0 con 0x7f9a6015ce30 2023-12-20T06:02:43.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a577fe640 1 -- 172.21.15.32:0/1526765122 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9a580b8ce0 con 0x7f9a6015ce30 2023-12-20T06:02:43.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.464+0000 7f9a577fe640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9a40065b80 0x7f9a40068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.468+0000 7f9a577fe640 1 -- 172.21.15.32:0/1526765122 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f9a5811ea50 con 0x7f9a6015ce30 2023-12-20T06:02:43.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.468+0000 7f9a5ed9d640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9a40065b80 0x7f9a40068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.468+0000 7f9a5ed9d640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9a40065b80 0x7f9a40068040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009ac0 tx=0x7f9a50009190 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:43.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.472+0000 7f9a577fe640 1 -- 172.21.15.32:0/1526765122 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9a580ec090 con 0x7f9a6015ce30 2023-12-20T06:02:43.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.628+0000 7f9a650d1640 1 -- 172.21.15.32:0/1526765122 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f9a600ce270 con 0x7f9a40065b80 2023-12-20T06:02:43.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.644+0000 7f9a577fe640 1 -- 172.21.15.32:0/1526765122 <== mgr.10314 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528568 (secure 0 0 0) 0x7f9a500119e0 con 0x7f9a40065b80 2023-12-20T06:02:43.649 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:43.656 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:02:43.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 -- 172.21.15.32:0/1526765122 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9a40065b80 msgr2=0x7f9a40068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9a40065b80 0x7f9a40068040 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f9a50009ac0 tx=0x7f9a50009190 comp rx=0 tx=0).stop 2023-12-20T06:02:43.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 -- 172.21.15.32:0/1526765122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6015ce30 msgr2=0x7f9a6015e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6015ce30 0x7f9a6015e460 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f9a5801a5b0 tx=0x7f9a58019720 comp rx=0 tx=0).stop 2023-12-20T06:02:43.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 -- 172.21.15.32:0/1526765122 shutdown_connections 2023-12-20T06:02:43.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9a40065b80 0x7f9a40068040 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9a6015ce30 0x7f9a6015e460 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9a60070950 0x7f9a6015df20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 --2- 172.21.15.32:0/1526765122 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9a6006deb0 0x7f9a60157d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 -- 172.21.15.32:0/1526765122 >> 172.21.15.32:0/1526765122 conn(0x7f9a600cba50 msgr2=0x7f9a600af3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 -- 172.21.15.32:0/1526765122 shutdown_connections 2023-12-20T06:02:43.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.648+0000 7f9a650d1640 1 -- 172.21.15.32:0/1526765122 wait complete. 2023-12-20T06:02:43.668 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-12-20T06:02:42.957498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":613,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3281,"ondisk_log_size":3281,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":418984,"kb_used_data":22120,"kb_used_omap":0,"kb_used_meta":396800,"kb_avail":754555736,"statfs":{"total":773094113280,"available":772665073664,"internally_reserved":0,"allocated":22650880,"data_stored":11578906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":406323200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.932362"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454462+0000","last_change":"2023-12-20T06:02:39.471108+0000","last_active":"2023-12-20T06:02:40.454462+0000","last_peered":"2023-12-20T06:02:40.454462+0000","last_clean":"2023-12-20T06:02:40.454462+0000","last_became_active":"2023-12-20T06:02:39.470838+0000","last_became_peered":"2023-12-20T06:02:39.470838+0000","last_unstale":"2023-12-20T06:02:40.454462+0000","last_undegraded":"2023-12-20T06:02:40.454462+0000","last_fullsized":"2023-12-20T06:02:40.454462+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:42:23.074715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"196'3201","reported_seq":3996,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465654+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:02:39.465654+0000","last_peered":"2023-12-20T06:02:39.465654+0000","last_clean":"2023-12-20T06:02:39.465654+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:02:39.465654+0000","last_undegraded":"2023-12-20T06:02:39.465654+0000","last_fullsized":"2023-12-20T06:02:39.465654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3201,"log_dups_size":0,"ondisk_log_size":3201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":521,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465147+0000","last_change":"2023-12-20T06:02:26.550026+0000","last_active":"2023-12-20T06:02:39.465147+0000","last_peered":"2023-12-20T06:02:39.465147+0000","last_clean":"2023-12-20T06:02:39.465147+0000","last_became_active":"2023-12-20T06:02:26.549712+0000","last_became_peered":"2023-12-20T06:02:26.549712+0000","last_unstale":"2023-12-20T06:02:39.465147+0000","last_undegraded":"2023-12-20T06:02:39.465147+0000","last_fullsized":"2023-12-20T06:02:39.465147+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:08.929656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452323+0000","last_change":"2023-12-20T06:02:26.971114+0000","last_active":"2023-12-20T06:02:39.452323+0000","last_peered":"2023-12-20T06:02:39.452323+0000","last_clean":"2023-12-20T06:02:39.452323+0000","last_became_active":"2023-12-20T06:02:26.970836+0000","last_became_peered":"2023-12-20T06:02:26.970836+0000","last_unstale":"2023-12-20T06:02:39.452323+0000","last_undegraded":"2023-12-20T06:02:39.452323+0000","last_fullsized":"2023-12-20T06:02:39.452323+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:50:58.624420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"212'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457896+0000","last_change":"2023-12-20T06:02:39.062821+0000","last_active":"2023-12-20T06:02:40.457896+0000","last_peered":"2023-12-20T06:02:40.457896+0000","last_clean":"2023-12-20T06:02:40.457896+0000","last_became_active":"2023-12-20T06:02:39.062495+0000","last_became_peered":"2023-12-20T06:02:39.062495+0000","last_unstale":"2023-12-20T06:02:40.457896+0000","last_undegraded":"2023-12-20T06:02:40.457896+0000","last_fullsized":"2023-12-20T06:02:40.457896+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:35:15.487675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457654+0000","last_change":"2023-12-20T06:02:39.462238+0000","last_active":"2023-12-20T06:02:40.457654+0000","last_peered":"2023-12-20T06:02:40.457654+0000","last_clean":"2023-12-20T06:02:40.457654+0000","last_became_active":"2023-12-20T06:02:39.461928+0000","last_became_peered":"2023-12-20T06:02:39.461928+0000","last_unstale":"2023-12-20T06:02:40.457654+0000","last_undegraded":"2023-12-20T06:02:40.457654+0000","last_fullsized":"2023-12-20T06:02:40.457654+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:58.188244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.462583+0000","last_change":"2023-12-20T06:02:26.560054+0000","last_active":"2023-12-20T06:02:39.462583+0000","last_peered":"2023-12-20T06:02:39.462583+0000","last_clean":"2023-12-20T06:02:39.462583+0000","last_became_active":"2023-12-20T06:02:26.559777+0000","last_became_peered":"2023-12-20T06:02:26.559777+0000","last_unstale":"2023-12-20T06:02:39.462583+0000","last_undegraded":"2023-12-20T06:02:39.462583+0000","last_fullsized":"2023-12-20T06:02:39.462583+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:40:31.615384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463934+0000","last_change":"2023-12-20T06:02:26.961645+0000","last_active":"2023-12-20T06:02:39.463934+0000","last_peered":"2023-12-20T06:02:39.463934+0000","last_clean":"2023-12-20T06:02:39.463934+0000","last_became_active":"2023-12-20T06:02:26.961386+0000","last_became_peered":"2023-12-20T06:02:26.961386+0000","last_unstale":"2023-12-20T06:02:39.463934+0000","last_undegraded":"2023-12-20T06:02:39.463934+0000","last_fullsized":"2023-12-20T06:02:39.463934+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:52:17.428938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343408+0000","last_change":"2023-12-20T06:02:39.066345+0000","last_active":"2023-12-20T06:02:41.343408+0000","last_peered":"2023-12-20T06:02:41.343408+0000","last_clean":"2023-12-20T06:02:41.343408+0000","last_became_active":"2023-12-20T06:02:39.065968+0000","last_became_peered":"2023-12-20T06:02:39.065968+0000","last_unstale":"2023-12-20T06:02:41.343408+0000","last_undegraded":"2023-12-20T06:02:41.343408+0000","last_fullsized":"2023-12-20T06:02:41.343408+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:03:22.846131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454664+0000","last_change":"2023-12-20T06:02:39.482489+0000","last_active":"2023-12-20T06:02:40.454664+0000","last_peered":"2023-12-20T06:02:40.454664+0000","last_clean":"2023-12-20T06:02:40.454664+0000","last_became_active":"2023-12-20T06:02:39.482228+0000","last_became_peered":"2023-12-20T06:02:39.482228+0000","last_unstale":"2023-12-20T06:02:40.454664+0000","last_undegraded":"2023-12-20T06:02:40.454664+0000","last_fullsized":"2023-12-20T06:02:40.454664+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:55:08.298301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452293+0000","last_change":"2023-12-20T06:02:26.548501+0000","last_active":"2023-12-20T06:02:39.452293+0000","last_peered":"2023-12-20T06:02:39.452293+0000","last_clean":"2023-12-20T06:02:39.452293+0000","last_became_active":"2023-12-20T06:02:26.548203+0000","last_became_peered":"2023-12-20T06:02:26.548203+0000","last_unstale":"2023-12-20T06:02:39.452293+0000","last_undegraded":"2023-12-20T06:02:39.452293+0000","last_fullsized":"2023-12-20T06:02:39.452293+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:55.646229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452572+0000","last_change":"2023-12-20T06:02:26.962551+0000","last_active":"2023-12-20T06:02:39.452572+0000","last_peered":"2023-12-20T06:02:39.452572+0000","last_clean":"2023-12-20T06:02:39.452572+0000","last_became_active":"2023-12-20T06:02:26.962269+0000","last_became_peered":"2023-12-20T06:02:26.962269+0000","last_unstale":"2023-12-20T06:02:39.452572+0000","last_undegraded":"2023-12-20T06:02:39.452572+0000","last_fullsized":"2023-12-20T06:02:39.452572+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:43:42.660130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246731+0000","last_change":"2023-12-20T06:02:39.068124+0000","last_active":"2023-12-20T06:02:41.246731+0000","last_peered":"2023-12-20T06:02:41.246731+0000","last_clean":"2023-12-20T06:02:41.246731+0000","last_became_active":"2023-12-20T06:02:39.067842+0000","last_became_peered":"2023-12-20T06:02:39.067842+0000","last_unstale":"2023-12-20T06:02:41.246731+0000","last_undegraded":"2023-12-20T06:02:41.246731+0000","last_fullsized":"2023-12-20T06:02:41.246731+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:03:21.014214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457298+0000","last_change":"2023-12-20T06:02:39.476607+0000","last_active":"2023-12-20T06:02:40.457298+0000","last_peered":"2023-12-20T06:02:40.457298+0000","last_clean":"2023-12-20T06:02:40.457298+0000","last_became_active":"2023-12-20T06:02:39.476262+0000","last_became_peered":"2023-12-20T06:02:39.476262+0000","last_unstale":"2023-12-20T06:02:40.457298+0000","last_undegraded":"2023-12-20T06:02:40.457298+0000","last_fullsized":"2023-12-20T06:02:40.457298+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:38:05.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464853+0000","last_change":"2023-12-20T06:02:26.556064+0000","last_active":"2023-12-20T06:02:39.464853+0000","last_peered":"2023-12-20T06:02:39.464853+0000","last_clean":"2023-12-20T06:02:39.464853+0000","last_became_active":"2023-12-20T06:02:26.554935+0000","last_became_peered":"2023-12-20T06:02:26.554935+0000","last_unstale":"2023-12-20T06:02:39.464853+0000","last_undegraded":"2023-12-20T06:02:39.464853+0000","last_fullsized":"2023-12-20T06:02:39.464853+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:12.697664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467764+0000","last_change":"2023-12-20T06:02:26.980532+0000","last_active":"2023-12-20T06:02:39.467764+0000","last_peered":"2023-12-20T06:02:39.467764+0000","last_clean":"2023-12-20T06:02:39.467764+0000","last_became_active":"2023-12-20T06:02:26.980234+0000","last_became_peered":"2023-12-20T06:02:26.980234+0000","last_unstale":"2023-12-20T06:02:39.467764+0000","last_undegraded":"2023-12-20T06:02:39.467764+0000","last_fullsized":"2023-12-20T06:02:39.467764+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:32.040401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.456725+0000","last_change":"2023-12-20T06:02:39.063288+0000","last_active":"2023-12-20T06:02:40.456725+0000","last_peered":"2023-12-20T06:02:40.456725+0000","last_clean":"2023-12-20T06:02:40.456725+0000","last_became_active":"2023-12-20T06:02:39.063009+0000","last_became_peered":"2023-12-20T06:02:39.063009+0000","last_unstale":"2023-12-20T06:02:40.456725+0000","last_undegraded":"2023-12-20T06:02:40.456725+0000","last_fullsized":"2023-12-20T06:02:40.456725+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:23.116478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342060+0000","last_change":"2023-12-20T06:02:39.481671+0000","last_active":"2023-12-20T06:02:41.342060+0000","last_peered":"2023-12-20T06:02:41.342060+0000","last_clean":"2023-12-20T06:02:41.342060+0000","last_became_active":"2023-12-20T06:02:39.480777+0000","last_became_peered":"2023-12-20T06:02:39.480777+0000","last_unstale":"2023-12-20T06:02:41.342060+0000","last_undegraded":"2023-12-20T06:02:41.342060+0000","last_fullsized":"2023-12-20T06:02:41.342060+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:05:06.811455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465878+0000","last_change":"2023-12-20T06:02:26.556385+0000","last_active":"2023-12-20T06:02:39.465878+0000","last_peered":"2023-12-20T06:02:39.465878+0000","last_clean":"2023-12-20T06:02:39.465878+0000","last_became_active":"2023-12-20T06:02:26.555308+0000","last_became_peered":"2023-12-20T06:02:26.555308+0000","last_unstale":"2023-12-20T06:02:39.465878+0000","last_undegraded":"2023-12-20T06:02:39.465878+0000","last_fullsized":"2023-12-20T06:02:39.465878+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:31.032511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465489+0000","last_change":"2023-12-20T06:02:26.983027+0000","last_active":"2023-12-20T06:02:39.465489+0000","last_peered":"2023-12-20T06:02:39.465489+0000","last_clean":"2023-12-20T06:02:39.465489+0000","last_became_active":"2023-12-20T06:02:26.982556+0000","last_became_peered":"2023-12-20T06:02:26.982556+0000","last_unstale":"2023-12-20T06:02:39.465489+0000","last_undegraded":"2023-12-20T06:02:39.465489+0000","last_fullsized":"2023-12-20T06:02:39.465489+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:42:24.293821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338410+0000","last_change":"2023-12-20T06:02:39.064927+0000","last_active":"2023-12-20T06:02:41.338410+0000","last_peered":"2023-12-20T06:02:41.338410+0000","last_clean":"2023-12-20T06:02:41.338410+0000","last_became_active":"2023-12-20T06:02:39.064648+0000","last_became_peered":"2023-12-20T06:02:39.064648+0000","last_unstale":"2023-12-20T06:02:41.338410+0000","last_undegraded":"2023-12-20T06:02:41.338410+0000","last_fullsized":"2023-12-20T06:02:41.338410+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:37:41.166384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.455349+0000","last_change":"2023-12-20T06:02:39.463206+0000","last_active":"2023-12-20T06:02:40.455349+0000","last_peered":"2023-12-20T06:02:40.455349+0000","last_clean":"2023-12-20T06:02:40.455349+0000","last_became_active":"2023-12-20T06:02:39.462905+0000","last_became_peered":"2023-12-20T06:02:39.462905+0000","last_unstale":"2023-12-20T06:02:40.455349+0000","last_undegraded":"2023-12-20T06:02:40.455349+0000","last_fullsized":"2023-12-20T06:02:40.455349+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:29:42.276728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465991+0000","last_change":"2023-12-20T06:02:26.560397+0000","last_active":"2023-12-20T06:02:39.465991+0000","last_peered":"2023-12-20T06:02:39.465991+0000","last_clean":"2023-12-20T06:02:39.465991+0000","last_became_active":"2023-12-20T06:02:26.560128+0000","last_became_peered":"2023-12-20T06:02:26.560128+0000","last_unstale":"2023-12-20T06:02:39.465991+0000","last_undegraded":"2023-12-20T06:02:39.465991+0000","last_fullsized":"2023-12-20T06:02:39.465991+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:54:06.878956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467204+0000","last_change":"2023-12-20T06:02:26.967116+0000","last_active":"2023-12-20T06:02:39.467204+0000","last_peered":"2023-12-20T06:02:39.467204+0000","last_clean":"2023-12-20T06:02:39.467204+0000","last_became_active":"2023-12-20T06:02:26.966832+0000","last_became_peered":"2023-12-20T06:02:26.966832+0000","last_unstale":"2023-12-20T06:02:39.467204+0000","last_undegraded":"2023-12-20T06:02:39.467204+0000","last_fullsized":"2023-12-20T06:02:39.467204+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:52:23.105026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.6","version":"213'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.249069+0000","last_change":"2023-12-20T06:02:39.065762+0000","last_active":"2023-12-20T06:02:41.249069+0000","last_peered":"2023-12-20T06:02:41.249069+0000","last_clean":"2023-12-20T06:02:41.249069+0000","last_became_active":"2023-12-20T06:02:39.065464+0000","last_became_peered":"2023-12-20T06:02:39.065464+0000","last_unstale":"2023-12-20T06:02:41.249069+0000","last_undegraded":"2023-12-20T06:02:41.249069+0000","last_fullsized":"2023-12-20T06:02:41.249069+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:09.323660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458475+0000","last_change":"2023-12-20T06:02:39.471117+0000","last_active":"2023-12-20T06:02:40.458475+0000","last_peered":"2023-12-20T06:02:40.458475+0000","last_clean":"2023-12-20T06:02:40.458475+0000","last_became_active":"2023-12-20T06:02:39.468897+0000","last_became_peered":"2023-12-20T06:02:39.468897+0000","last_unstale":"2023-12-20T06:02:40.458475+0000","last_undegraded":"2023-12-20T06:02:40.458475+0000","last_fullsized":"2023-12-20T06:02:40.458475+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:40.405390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.455635+0000","last_change":"2023-12-20T06:02:26.556970+0000","last_active":"2023-12-20T06:02:39.455635+0000","last_peered":"2023-12-20T06:02:39.455635+0000","last_clean":"2023-12-20T06:02:39.455635+0000","last_became_active":"2023-12-20T06:02:26.556577+0000","last_became_peered":"2023-12-20T06:02:26.556577+0000","last_unstale":"2023-12-20T06:02:39.455635+0000","last_undegraded":"2023-12-20T06:02:39.455635+0000","last_fullsized":"2023-12-20T06:02:39.455635+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:45.673434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465910+0000","last_change":"2023-12-20T06:02:26.969245+0000","last_active":"2023-12-20T06:02:39.465910+0000","last_peered":"2023-12-20T06:02:39.465910+0000","last_clean":"2023-12-20T06:02:39.465910+0000","last_became_active":"2023-12-20T06:02:26.969011+0000","last_became_peered":"2023-12-20T06:02:26.969011+0000","last_unstale":"2023-12-20T06:02:39.465910+0000","last_undegraded":"2023-12-20T06:02:39.465910+0000","last_fullsized":"2023-12-20T06:02:39.465910+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:53.274408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342301+0000","last_change":"2023-12-20T06:02:39.069539+0000","last_active":"2023-12-20T06:02:41.342301+0000","last_peered":"2023-12-20T06:02:41.342301+0000","last_clean":"2023-12-20T06:02:41.342301+0000","last_became_active":"2023-12-20T06:02:39.069249+0000","last_became_peered":"2023-12-20T06:02:39.069249+0000","last_unstale":"2023-12-20T06:02:41.342301+0000","last_undegraded":"2023-12-20T06:02:41.342301+0000","last_fullsized":"2023-12-20T06:02:41.342301+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:44:23.563050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338767+0000","last_change":"2023-12-20T06:02:39.472568+0000","last_active":"2023-12-20T06:02:41.338767+0000","last_peered":"2023-12-20T06:02:41.338767+0000","last_clean":"2023-12-20T06:02:41.338767+0000","last_became_active":"2023-12-20T06:02:39.472269+0000","last_became_peered":"2023-12-20T06:02:39.472269+0000","last_unstale":"2023-12-20T06:02:41.338767+0000","last_undegraded":"2023-12-20T06:02:41.338767+0000","last_fullsized":"2023-12-20T06:02:41.338767+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:48.990217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.6","version":"209'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.454995+0000","last_change":"2023-12-20T06:02:26.555848+0000","last_active":"2023-12-20T06:02:39.454995+0000","last_peered":"2023-12-20T06:02:39.454995+0000","last_clean":"2023-12-20T06:02:39.454995+0000","last_became_active":"2023-12-20T06:02:26.555560+0000","last_became_peered":"2023-12-20T06:02:26.555560+0000","last_unstale":"2023-12-20T06:02:39.454995+0000","last_undegraded":"2023-12-20T06:02:39.454995+0000","last_fullsized":"2023-12-20T06:02:39.454995+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:44.684288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.456510+0000","last_change":"2023-12-20T06:02:26.981756+0000","last_active":"2023-12-20T06:02:39.456510+0000","last_peered":"2023-12-20T06:02:39.456510+0000","last_clean":"2023-12-20T06:02:39.456510+0000","last_became_active":"2023-12-20T06:02:26.980456+0000","last_became_peered":"2023-12-20T06:02:26.980456+0000","last_unstale":"2023-12-20T06:02:39.456510+0000","last_undegraded":"2023-12-20T06:02:39.456510+0000","last_fullsized":"2023-12-20T06:02:39.456510+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:33:51.384201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342755+0000","last_change":"2023-12-20T06:02:39.065294+0000","last_active":"2023-12-20T06:02:41.342755+0000","last_peered":"2023-12-20T06:02:41.342755+0000","last_clean":"2023-12-20T06:02:41.342755+0000","last_became_active":"2023-12-20T06:02:39.065013+0000","last_became_peered":"2023-12-20T06:02:39.065013+0000","last_unstale":"2023-12-20T06:02:41.342755+0000","last_undegraded":"2023-12-20T06:02:41.342755+0000","last_fullsized":"2023-12-20T06:02:41.342755+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:22.604577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458529+0000","last_change":"2023-12-20T06:02:39.476789+0000","last_active":"2023-12-20T06:02:40.458529+0000","last_peered":"2023-12-20T06:02:40.458529+0000","last_clean":"2023-12-20T06:02:40.458529+0000","last_became_active":"2023-12-20T06:02:39.476418+0000","last_became_peered":"2023-12-20T06:02:39.476418+0000","last_unstale":"2023-12-20T06:02:40.458529+0000","last_undegraded":"2023-12-20T06:02:40.458529+0000","last_fullsized":"2023-12-20T06:02:40.458529+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:41:50.225640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.468112+0000","last_change":"2023-12-20T06:02:26.549840+0000","last_active":"2023-12-20T06:02:39.468112+0000","last_peered":"2023-12-20T06:02:39.468112+0000","last_clean":"2023-12-20T06:02:39.468112+0000","last_became_active":"2023-12-20T06:02:26.549537+0000","last_became_peered":"2023-12-20T06:02:26.549537+0000","last_unstale":"2023-12-20T06:02:39.468112+0000","last_undegraded":"2023-12-20T06:02:39.468112+0000","last_fullsized":"2023-12-20T06:02:39.468112+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:50.955103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464485+0000","last_change":"2023-12-20T06:02:26.966239+0000","last_active":"2023-12-20T06:02:39.464485+0000","last_peered":"2023-12-20T06:02:39.464485+0000","last_clean":"2023-12-20T06:02:39.464485+0000","last_became_active":"2023-12-20T06:02:26.964925+0000","last_became_peered":"2023-12-20T06:02:26.964925+0000","last_unstale":"2023-12-20T06:02:39.464485+0000","last_undegraded":"2023-12-20T06:02:39.464485+0000","last_fullsized":"2023-12-20T06:02:39.464485+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:09.937547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.344274+0000","last_change":"2023-12-20T06:02:39.057584+0000","last_active":"2023-12-20T06:02:41.344274+0000","last_peered":"2023-12-20T06:02:41.344274+0000","last_clean":"2023-12-20T06:02:41.344274+0000","last_became_active":"2023-12-20T06:02:39.057300+0000","last_became_peered":"2023-12-20T06:02:39.057300+0000","last_unstale":"2023-12-20T06:02:41.344274+0000","last_undegraded":"2023-12-20T06:02:41.344274+0000","last_fullsized":"2023-12-20T06:02:41.344274+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:35:15.487675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342858+0000","last_change":"2023-12-20T06:02:39.475953+0000","last_active":"2023-12-20T06:02:41.342858+0000","last_peered":"2023-12-20T06:02:41.342858+0000","last_clean":"2023-12-20T06:02:41.342858+0000","last_became_active":"2023-12-20T06:02:39.475668+0000","last_became_peered":"2023-12-20T06:02:39.475668+0000","last_unstale":"2023-12-20T06:02:41.342858+0000","last_undegraded":"2023-12-20T06:02:41.342858+0000","last_fullsized":"2023-12-20T06:02:41.342858+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:06:14.209435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463909+0000","last_change":"2023-12-20T06:02:26.547934+0000","last_active":"2023-12-20T06:02:39.463909+0000","last_peered":"2023-12-20T06:02:39.463909+0000","last_clean":"2023-12-20T06:02:39.463909+0000","last_became_active":"2023-12-20T06:02:26.547651+0000","last_became_peered":"2023-12-20T06:02:26.547651+0000","last_unstale":"2023-12-20T06:02:39.463909+0000","last_undegraded":"2023-12-20T06:02:39.463909+0000","last_fullsized":"2023-12-20T06:02:39.463909+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:42:23.285931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463859+0000","last_change":"2023-12-20T06:02:26.961186+0000","last_active":"2023-12-20T06:02:39.463859+0000","last_peered":"2023-12-20T06:02:39.463859+0000","last_clean":"2023-12-20T06:02:39.463859+0000","last_became_active":"2023-12-20T06:02:26.960904+0000","last_became_peered":"2023-12-20T06:02:26.960904+0000","last_unstale":"2023-12-20T06:02:39.463859+0000","last_undegraded":"2023-12-20T06:02:39.463859+0000","last_fullsized":"2023-12-20T06:02:39.463859+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:45.708153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454752+0000","last_change":"2023-12-20T06:02:39.069300+0000","last_active":"2023-12-20T06:02:40.454752+0000","last_peered":"2023-12-20T06:02:40.454752+0000","last_clean":"2023-12-20T06:02:40.454752+0000","last_became_active":"2023-12-20T06:02:39.069037+0000","last_became_peered":"2023-12-20T06:02:39.069037+0000","last_unstale":"2023-12-20T06:02:40.454752+0000","last_undegraded":"2023-12-20T06:02:40.454752+0000","last_fullsized":"2023-12-20T06:02:40.454752+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:25:58.506206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341821+0000","last_change":"2023-12-20T06:02:39.472628+0000","last_active":"2023-12-20T06:02:41.341821+0000","last_peered":"2023-12-20T06:02:41.341821+0000","last_clean":"2023-12-20T06:02:41.341821+0000","last_became_active":"2023-12-20T06:02:39.471675+0000","last_became_peered":"2023-12-20T06:02:39.471675+0000","last_unstale":"2023-12-20T06:02:41.341821+0000","last_undegraded":"2023-12-20T06:02:41.341821+0000","last_fullsized":"2023-12-20T06:02:41.341821+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:58:55.988186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.461020+0000","last_change":"2023-12-20T06:02:26.560641+0000","last_active":"2023-12-20T06:02:39.461020+0000","last_peered":"2023-12-20T06:02:39.461020+0000","last_clean":"2023-12-20T06:02:39.461020+0000","last_became_active":"2023-12-20T06:02:26.560352+0000","last_became_peered":"2023-12-20T06:02:26.560352+0000","last_unstale":"2023-12-20T06:02:39.461020+0000","last_undegraded":"2023-12-20T06:02:39.461020+0000","last_fullsized":"2023-12-20T06:02:39.461020+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:49.756970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452516+0000","last_change":"2023-12-20T06:02:26.968137+0000","last_active":"2023-12-20T06:02:39.452516+0000","last_peered":"2023-12-20T06:02:39.452516+0000","last_clean":"2023-12-20T06:02:39.452516+0000","last_became_active":"2023-12-20T06:02:26.967861+0000","last_became_peered":"2023-12-20T06:02:26.967861+0000","last_unstale":"2023-12-20T06:02:39.452516+0000","last_undegraded":"2023-12-20T06:02:39.452516+0000","last_fullsized":"2023-12-20T06:02:39.452516+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:34:19.170698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343218+0000","last_change":"2023-12-20T06:02:39.062332+0000","last_active":"2023-12-20T06:02:41.343218+0000","last_peered":"2023-12-20T06:02:41.343218+0000","last_clean":"2023-12-20T06:02:41.343218+0000","last_became_active":"2023-12-20T06:02:39.061352+0000","last_became_peered":"2023-12-20T06:02:39.061352+0000","last_unstale":"2023-12-20T06:02:41.343218+0000","last_undegraded":"2023-12-20T06:02:41.343218+0000","last_fullsized":"2023-12-20T06:02:41.343218+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:23:19.674539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338275+0000","last_change":"2023-12-20T06:02:39.475911+0000","last_active":"2023-12-20T06:02:41.338275+0000","last_peered":"2023-12-20T06:02:41.338275+0000","last_clean":"2023-12-20T06:02:41.338275+0000","last_became_active":"2023-12-20T06:02:39.473988+0000","last_became_peered":"2023-12-20T06:02:39.473988+0000","last_unstale":"2023-12-20T06:02:41.338275+0000","last_undegraded":"2023-12-20T06:02:41.338275+0000","last_fullsized":"2023-12-20T06:02:41.338275+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:06:56.699636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464158+0000","last_change":"2023-12-20T06:02:26.554403+0000","last_active":"2023-12-20T06:02:39.464158+0000","last_peered":"2023-12-20T06:02:39.464158+0000","last_clean":"2023-12-20T06:02:39.464158+0000","last_became_active":"2023-12-20T06:02:26.554112+0000","last_became_peered":"2023-12-20T06:02:26.554112+0000","last_unstale":"2023-12-20T06:02:39.464158+0000","last_undegraded":"2023-12-20T06:02:39.464158+0000","last_fullsized":"2023-12-20T06:02:39.464158+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:53:48.616814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464798+0000","last_change":"2023-12-20T06:02:26.981736+0000","last_active":"2023-12-20T06:02:39.464798+0000","last_peered":"2023-12-20T06:02:39.464798+0000","last_clean":"2023-12-20T06:02:39.464798+0000","last_became_active":"2023-12-20T06:02:26.981185+0000","last_became_peered":"2023-12-20T06:02:26.981185+0000","last_unstale":"2023-12-20T06:02:39.464798+0000","last_undegraded":"2023-12-20T06:02:39.464798+0000","last_fullsized":"2023-12-20T06:02:39.464798+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:59.846058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341738+0000","last_change":"2023-12-20T06:02:39.065594+0000","last_active":"2023-12-20T06:02:41.341738+0000","last_peered":"2023-12-20T06:02:41.341738+0000","last_clean":"2023-12-20T06:02:41.341738+0000","last_became_active":"2023-12-20T06:02:39.065283+0000","last_became_peered":"2023-12-20T06:02:39.065283+0000","last_unstale":"2023-12-20T06:02:41.341738+0000","last_undegraded":"2023-12-20T06:02:41.341738+0000","last_fullsized":"2023-12-20T06:02:41.341738+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:08.387196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457720+0000","last_change":"2023-12-20T06:02:39.486070+0000","last_active":"2023-12-20T06:02:40.457720+0000","last_peered":"2023-12-20T06:02:40.457720+0000","last_clean":"2023-12-20T06:02:40.457720+0000","last_became_active":"2023-12-20T06:02:39.485809+0000","last_became_peered":"2023-12-20T06:02:39.485809+0000","last_unstale":"2023-12-20T06:02:40.457720+0000","last_undegraded":"2023-12-20T06:02:40.457720+0000","last_fullsized":"2023-12-20T06:02:40.457720+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:59.177389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.b","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467169+0000","last_change":"2023-12-20T06:02:26.565662+0000","last_active":"2023-12-20T06:02:39.467169+0000","last_peered":"2023-12-20T06:02:39.467169+0000","last_clean":"2023-12-20T06:02:39.467169+0000","last_became_active":"2023-12-20T06:02:26.564737+0000","last_became_peered":"2023-12-20T06:02:26.564737+0000","last_unstale":"2023-12-20T06:02:39.467169+0000","last_undegraded":"2023-12-20T06:02:39.467169+0000","last_fullsized":"2023-12-20T06:02:39.467169+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:59:26.669013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453539+0000","last_change":"2023-12-20T06:02:26.972273+0000","last_active":"2023-12-20T06:02:39.453539+0000","last_peered":"2023-12-20T06:02:39.453539+0000","last_clean":"2023-12-20T06:02:39.453539+0000","last_became_active":"2023-12-20T06:02:26.971983+0000","last_became_peered":"2023-12-20T06:02:26.971983+0000","last_unstale":"2023-12-20T06:02:39.453539+0000","last_undegraded":"2023-12-20T06:02:39.453539+0000","last_fullsized":"2023-12-20T06:02:39.453539+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:35:59.779548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454298+0000","last_change":"2023-12-20T06:02:39.073826+0000","last_active":"2023-12-20T06:02:40.454298+0000","last_peered":"2023-12-20T06:02:40.454298+0000","last_clean":"2023-12-20T06:02:40.454298+0000","last_became_active":"2023-12-20T06:02:39.073502+0000","last_became_peered":"2023-12-20T06:02:39.073502+0000","last_unstale":"2023-12-20T06:02:40.454298+0000","last_undegraded":"2023-12-20T06:02:40.454298+0000","last_fullsized":"2023-12-20T06:02:40.454298+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:23.668307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.459506+0000","last_change":"2023-12-20T06:02:39.476958+0000","last_active":"2023-12-20T06:02:40.459506+0000","last_peered":"2023-12-20T06:02:40.459506+0000","last_clean":"2023-12-20T06:02:40.459506+0000","last_became_active":"2023-12-20T06:02:39.476215+0000","last_became_peered":"2023-12-20T06:02:39.476215+0000","last_unstale":"2023-12-20T06:02:40.459506+0000","last_undegraded":"2023-12-20T06:02:40.459506+0000","last_fullsized":"2023-12-20T06:02:40.459506+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:40.405390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.460043+0000","last_change":"2023-12-20T06:02:26.559666+0000","last_active":"2023-12-20T06:02:39.460043+0000","last_peered":"2023-12-20T06:02:39.460043+0000","last_clean":"2023-12-20T06:02:39.460043+0000","last_became_active":"2023-12-20T06:02:26.559324+0000","last_became_peered":"2023-12-20T06:02:26.559324+0000","last_unstale":"2023-12-20T06:02:39.460043+0000","last_undegraded":"2023-12-20T06:02:39.460043+0000","last_fullsized":"2023-12-20T06:02:39.460043+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:27.148940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453188+0000","last_change":"2023-12-20T06:02:26.958574+0000","last_active":"2023-12-20T06:02:39.453188+0000","last_peered":"2023-12-20T06:02:39.453188+0000","last_clean":"2023-12-20T06:02:39.453188+0000","last_became_active":"2023-12-20T06:02:26.958284+0000","last_became_peered":"2023-12-20T06:02:26.958284+0000","last_unstale":"2023-12-20T06:02:39.453188+0000","last_undegraded":"2023-12-20T06:02:39.453188+0000","last_fullsized":"2023-12-20T06:02:39.453188+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:28.156831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.459304+0000","last_change":"2023-12-20T06:02:39.065892+0000","last_active":"2023-12-20T06:02:40.459304+0000","last_peered":"2023-12-20T06:02:40.459304+0000","last_clean":"2023-12-20T06:02:40.459304+0000","last_became_active":"2023-12-20T06:02:39.065606+0000","last_became_peered":"2023-12-20T06:02:39.065606+0000","last_unstale":"2023-12-20T06:02:40.459304+0000","last_undegraded":"2023-12-20T06:02:40.459304+0000","last_fullsized":"2023-12-20T06:02:40.459304+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:48:11.323412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.459066+0000","last_change":"2023-12-20T06:02:39.487282+0000","last_active":"2023-12-20T06:02:40.459066+0000","last_peered":"2023-12-20T06:02:40.459066+0000","last_clean":"2023-12-20T06:02:40.459066+0000","last_became_active":"2023-12-20T06:02:39.483132+0000","last_became_peered":"2023-12-20T06:02:39.483132+0000","last_unstale":"2023-12-20T06:02:40.459066+0000","last_undegraded":"2023-12-20T06:02:40.459066+0000","last_fullsized":"2023-12-20T06:02:40.459066+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:37:42.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"208'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467300+0000","last_change":"2023-12-20T06:02:26.553441+0000","last_active":"2023-12-20T06:02:39.467300+0000","last_peered":"2023-12-20T06:02:39.467300+0000","last_clean":"2023-12-20T06:02:39.467300+0000","last_became_active":"2023-12-20T06:02:26.551705+0000","last_became_peered":"2023-12-20T06:02:26.551705+0000","last_unstale":"2023-12-20T06:02:39.467300+0000","last_undegraded":"2023-12-20T06:02:39.467300+0000","last_fullsized":"2023-12-20T06:02:39.467300+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:51.593076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464368+0000","last_change":"2023-12-20T06:02:26.977809+0000","last_active":"2023-12-20T06:02:39.464368+0000","last_peered":"2023-12-20T06:02:39.464368+0000","last_clean":"2023-12-20T06:02:39.464368+0000","last_became_active":"2023-12-20T06:02:26.976835+0000","last_became_peered":"2023-12-20T06:02:26.976835+0000","last_unstale":"2023-12-20T06:02:39.464368+0000","last_undegraded":"2023-12-20T06:02:39.464368+0000","last_fullsized":"2023-12-20T06:02:39.464368+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"212'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.344720+0000","last_change":"2023-12-20T06:02:39.068363+0000","last_active":"2023-12-20T06:02:41.344720+0000","last_peered":"2023-12-20T06:02:41.344720+0000","last_clean":"2023-12-20T06:02:41.344720+0000","last_became_active":"2023-12-20T06:02:39.065458+0000","last_became_peered":"2023-12-20T06:02:39.065458+0000","last_unstale":"2023-12-20T06:02:41.344720+0000","last_undegraded":"2023-12-20T06:02:41.344720+0000","last_fullsized":"2023-12-20T06:02:41.344720+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:25:06.985482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343802+0000","last_change":"2023-12-20T06:02:39.476547+0000","last_active":"2023-12-20T06:02:41.343802+0000","last_peered":"2023-12-20T06:02:41.343802+0000","last_clean":"2023-12-20T06:02:41.343802+0000","last_became_active":"2023-12-20T06:02:39.476284+0000","last_became_peered":"2023-12-20T06:02:39.476284+0000","last_unstale":"2023-12-20T06:02:41.343802+0000","last_undegraded":"2023-12-20T06:02:41.343802+0000","last_fullsized":"2023-12-20T06:02:41.343802+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:24.687018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.468631+0000","last_change":"2023-12-20T06:02:26.561295+0000","last_active":"2023-12-20T06:02:39.468631+0000","last_peered":"2023-12-20T06:02:39.468631+0000","last_clean":"2023-12-20T06:02:39.468631+0000","last_became_active":"2023-12-20T06:02:26.561016+0000","last_became_peered":"2023-12-20T06:02:26.561016+0000","last_unstale":"2023-12-20T06:02:39.468631+0000","last_undegraded":"2023-12-20T06:02:39.468631+0000","last_fullsized":"2023-12-20T06:02:39.468631+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:53.307500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464153+0000","last_change":"2023-12-20T06:02:26.979723+0000","last_active":"2023-12-20T06:02:39.464153+0000","last_peered":"2023-12-20T06:02:39.464153+0000","last_clean":"2023-12-20T06:02:39.464153+0000","last_became_active":"2023-12-20T06:02:26.979447+0000","last_became_peered":"2023-12-20T06:02:26.979447+0000","last_unstale":"2023-12-20T06:02:39.464153+0000","last_undegraded":"2023-12-20T06:02:39.464153+0000","last_fullsized":"2023-12-20T06:02:39.464153+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:44:12.085696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343043+0000","last_change":"2023-12-20T06:02:39.059882+0000","last_active":"2023-12-20T06:02:41.343043+0000","last_peered":"2023-12-20T06:02:41.343043+0000","last_clean":"2023-12-20T06:02:41.343043+0000","last_became_active":"2023-12-20T06:02:39.059354+0000","last_became_peered":"2023-12-20T06:02:39.059354+0000","last_unstale":"2023-12-20T06:02:41.343043+0000","last_undegraded":"2023-12-20T06:02:41.343043+0000","last_fullsized":"2023-12-20T06:02:41.343043+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:52.925504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342658+0000","last_change":"2023-12-20T06:02:39.482797+0000","last_active":"2023-12-20T06:02:41.342658+0000","last_peered":"2023-12-20T06:02:41.342658+0000","last_clean":"2023-12-20T06:02:41.342658+0000","last_became_active":"2023-12-20T06:02:39.482116+0000","last_became_peered":"2023-12-20T06:02:39.482116+0000","last_unstale":"2023-12-20T06:02:41.342658+0000","last_undegraded":"2023-12-20T06:02:41.342658+0000","last_fullsized":"2023-12-20T06:02:41.342658+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:27.643717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"208'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.469653+0000","last_change":"2023-12-20T06:02:26.559146+0000","last_active":"2023-12-20T06:02:39.469653+0000","last_peered":"2023-12-20T06:02:39.469653+0000","last_clean":"2023-12-20T06:02:39.469653+0000","last_became_active":"2023-12-20T06:02:26.558865+0000","last_became_peered":"2023-12-20T06:02:26.558865+0000","last_unstale":"2023-12-20T06:02:39.469653+0000","last_undegraded":"2023-12-20T06:02:39.469653+0000","last_fullsized":"2023-12-20T06:02:39.469653+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:23:28.617951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.454912+0000","last_change":"2023-12-20T06:02:26.973527+0000","last_active":"2023-12-20T06:02:39.454912+0000","last_peered":"2023-12-20T06:02:39.454912+0000","last_clean":"2023-12-20T06:02:39.454912+0000","last_became_active":"2023-12-20T06:02:26.972995+0000","last_became_peered":"2023-12-20T06:02:26.972995+0000","last_unstale":"2023-12-20T06:02:39.454912+0000","last_undegraded":"2023-12-20T06:02:39.454912+0000","last_fullsized":"2023-12-20T06:02:39.454912+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:15:22.108587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"212'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343885+0000","last_change":"2023-12-20T06:02:39.071884+0000","last_active":"2023-12-20T06:02:41.343885+0000","last_peered":"2023-12-20T06:02:41.343885+0000","last_clean":"2023-12-20T06:02:41.343885+0000","last_became_active":"2023-12-20T06:02:39.071083+0000","last_became_peered":"2023-12-20T06:02:39.071083+0000","last_unstale":"2023-12-20T06:02:41.343885+0000","last_undegraded":"2023-12-20T06:02:41.343885+0000","last_fullsized":"2023-12-20T06:02:41.343885+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:51:10.101774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342270+0000","last_change":"2023-12-20T06:02:39.480913+0000","last_active":"2023-12-20T06:02:41.342270+0000","last_peered":"2023-12-20T06:02:41.342270+0000","last_clean":"2023-12-20T06:02:41.342270+0000","last_became_active":"2023-12-20T06:02:39.480356+0000","last_became_peered":"2023-12-20T06:02:39.480356+0000","last_unstale":"2023-12-20T06:02:41.342270+0000","last_undegraded":"2023-12-20T06:02:41.342270+0000","last_fullsized":"2023-12-20T06:02:41.342270+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:25.169128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.10","version":"208'8","reported_seq":23,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465560+0000","last_change":"2023-12-20T06:02:26.552002+0000","last_active":"2023-12-20T06:02:39.465560+0000","last_peered":"2023-12-20T06:02:39.465560+0000","last_clean":"2023-12-20T06:02:39.465560+0000","last_became_active":"2023-12-20T06:02:26.551664+0000","last_became_peered":"2023-12-20T06:02:26.551664+0000","last_unstale":"2023-12-20T06:02:39.465560+0000","last_undegraded":"2023-12-20T06:02:39.465560+0000","last_fullsized":"2023-12-20T06:02:39.465560+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:49:14.073356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452751+0000","last_change":"2023-12-20T06:02:26.970325+0000","last_active":"2023-12-20T06:02:39.452751+0000","last_peered":"2023-12-20T06:02:39.452751+0000","last_clean":"2023-12-20T06:02:39.452751+0000","last_became_active":"2023-12-20T06:02:26.970054+0000","last_became_peered":"2023-12-20T06:02:26.970054+0000","last_unstale":"2023-12-20T06:02:39.452751+0000","last_undegraded":"2023-12-20T06:02:39.452751+0000","last_fullsized":"2023-12-20T06:02:39.452751+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:23:08.197185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338321+0000","last_change":"2023-12-20T06:02:39.070297+0000","last_active":"2023-12-20T06:02:41.338321+0000","last_peered":"2023-12-20T06:02:41.338321+0000","last_clean":"2023-12-20T06:02:41.338321+0000","last_became_active":"2023-12-20T06:02:39.066645+0000","last_became_peered":"2023-12-20T06:02:39.066645+0000","last_unstale":"2023-12-20T06:02:41.338321+0000","last_undegraded":"2023-12-20T06:02:41.338321+0000","last_fullsized":"2023-12-20T06:02:41.338321+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:20:36.068915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343236+0000","last_change":"2023-12-20T06:02:39.477713+0000","last_active":"2023-12-20T06:02:41.343236+0000","last_peered":"2023-12-20T06:02:41.343236+0000","last_clean":"2023-12-20T06:02:41.343236+0000","last_became_active":"2023-12-20T06:02:39.477419+0000","last_became_peered":"2023-12-20T06:02:39.477419+0000","last_unstale":"2023-12-20T06:02:41.343236+0000","last_undegraded":"2023-12-20T06:02:41.343236+0000","last_fullsized":"2023-12-20T06:02:41.343236+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:38:05.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465107+0000","last_change":"2023-12-20T06:02:26.558390+0000","last_active":"2023-12-20T06:02:39.465107+0000","last_peered":"2023-12-20T06:02:39.465107+0000","last_clean":"2023-12-20T06:02:39.465107+0000","last_became_active":"2023-12-20T06:02:26.558078+0000","last_became_peered":"2023-12-20T06:02:26.558078+0000","last_unstale":"2023-12-20T06:02:39.465107+0000","last_undegraded":"2023-12-20T06:02:39.465107+0000","last_fullsized":"2023-12-20T06:02:39.465107+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:47:56.455893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463814+0000","last_change":"2023-12-20T06:02:26.978618+0000","last_active":"2023-12-20T06:02:39.463814+0000","last_peered":"2023-12-20T06:02:39.463814+0000","last_clean":"2023-12-20T06:02:39.463814+0000","last_became_active":"2023-12-20T06:02:26.978305+0000","last_became_peered":"2023-12-20T06:02:26.978305+0000","last_unstale":"2023-12-20T06:02:39.463814+0000","last_undegraded":"2023-12-20T06:02:39.463814+0000","last_fullsized":"2023-12-20T06:02:39.463814+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:02:53.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.13","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338510+0000","last_change":"2023-12-20T06:02:39.068327+0000","last_active":"2023-12-20T06:02:41.338510+0000","last_peered":"2023-12-20T06:02:41.338510+0000","last_clean":"2023-12-20T06:02:41.338510+0000","last_became_active":"2023-12-20T06:02:39.067332+0000","last_became_peered":"2023-12-20T06:02:39.067332+0000","last_unstale":"2023-12-20T06:02:41.338510+0000","last_undegraded":"2023-12-20T06:02:41.338510+0000","last_fullsized":"2023-12-20T06:02:41.338510+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:11:37.511013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454276+0000","last_change":"2023-12-20T06:02:39.479905+0000","last_active":"2023-12-20T06:02:40.454276+0000","last_peered":"2023-12-20T06:02:40.454276+0000","last_clean":"2023-12-20T06:02:40.454276+0000","last_became_active":"2023-12-20T06:02:39.479376+0000","last_became_peered":"2023-12-20T06:02:39.479376+0000","last_unstale":"2023-12-20T06:02:40.454276+0000","last_undegraded":"2023-12-20T06:02:40.454276+0000","last_fullsized":"2023-12-20T06:02:40.454276+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:41.309037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453629+0000","last_change":"2023-12-20T06:02:26.553685+0000","last_active":"2023-12-20T06:02:39.453629+0000","last_peered":"2023-12-20T06:02:39.453629+0000","last_clean":"2023-12-20T06:02:39.453629+0000","last_became_active":"2023-12-20T06:02:26.553411+0000","last_became_peered":"2023-12-20T06:02:26.553411+0000","last_unstale":"2023-12-20T06:02:39.453629+0000","last_undegraded":"2023-12-20T06:02:39.453629+0000","last_fullsized":"2023-12-20T06:02:39.453629+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:52.266518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464689+0000","last_change":"2023-12-20T06:02:26.979070+0000","last_active":"2023-12-20T06:02:39.464689+0000","last_peered":"2023-12-20T06:02:39.464689+0000","last_clean":"2023-12-20T06:02:39.464689+0000","last_became_active":"2023-12-20T06:02:26.978777+0000","last_became_peered":"2023-12-20T06:02:26.978777+0000","last_unstale":"2023-12-20T06:02:39.464689+0000","last_undegraded":"2023-12-20T06:02:39.464689+0000","last_fullsized":"2023-12-20T06:02:39.464689+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:54.315390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"212'8","reported_seq":19,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342160+0000","last_change":"2023-12-20T06:02:39.064515+0000","last_active":"2023-12-20T06:02:41.342160+0000","last_peered":"2023-12-20T06:02:41.342160+0000","last_clean":"2023-12-20T06:02:41.342160+0000","last_became_active":"2023-12-20T06:02:39.063462+0000","last_became_peered":"2023-12-20T06:02:39.063462+0000","last_unstale":"2023-12-20T06:02:41.342160+0000","last_undegraded":"2023-12-20T06:02:41.342160+0000","last_fullsized":"2023-12-20T06:02:41.342160+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:19:48.734380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342263+0000","last_change":"2023-12-20T06:02:39.475798+0000","last_active":"2023-12-20T06:02:41.342263+0000","last_peered":"2023-12-20T06:02:41.342263+0000","last_clean":"2023-12-20T06:02:41.342263+0000","last_became_active":"2023-12-20T06:02:39.475515+0000","last_became_peered":"2023-12-20T06:02:39.475515+0000","last_unstale":"2023-12-20T06:02:41.342263+0000","last_undegraded":"2023-12-20T06:02:41.342263+0000","last_fullsized":"2023-12-20T06:02:41.342263+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:53:40.467317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.13","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467873+0000","last_change":"2023-12-20T06:02:26.552924+0000","last_active":"2023-12-20T06:02:39.467873+0000","last_peered":"2023-12-20T06:02:39.467873+0000","last_clean":"2023-12-20T06:02:39.467873+0000","last_became_active":"2023-12-20T06:02:26.552647+0000","last_became_peered":"2023-12-20T06:02:26.552647+0000","last_unstale":"2023-12-20T06:02:39.467873+0000","last_undegraded":"2023-12-20T06:02:39.467873+0000","last_fullsized":"2023-12-20T06:02:39.467873+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:44.700263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464153+0000","last_change":"2023-12-20T06:02:26.976279+0000","last_active":"2023-12-20T06:02:39.464153+0000","last_peered":"2023-12-20T06:02:39.464153+0000","last_clean":"2023-12-20T06:02:39.464153+0000","last_became_active":"2023-12-20T06:02:26.976019+0000","last_became_peered":"2023-12-20T06:02:26.976019+0000","last_unstale":"2023-12-20T06:02:39.464153+0000","last_undegraded":"2023-12-20T06:02:39.464153+0000","last_fullsized":"2023-12-20T06:02:39.464153+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341732+0000","last_change":"2023-12-20T06:02:39.064453+0000","last_active":"2023-12-20T06:02:41.341732+0000","last_peered":"2023-12-20T06:02:41.341732+0000","last_clean":"2023-12-20T06:02:41.341732+0000","last_became_active":"2023-12-20T06:02:39.064128+0000","last_became_peered":"2023-12-20T06:02:39.064128+0000","last_unstale":"2023-12-20T06:02:41.341732+0000","last_undegraded":"2023-12-20T06:02:41.341732+0000","last_fullsized":"2023-12-20T06:02:41.341732+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:48.000452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342190+0000","last_change":"2023-12-20T06:02:39.478799+0000","last_active":"2023-12-20T06:02:41.342190+0000","last_peered":"2023-12-20T06:02:41.342190+0000","last_clean":"2023-12-20T06:02:41.342190+0000","last_became_active":"2023-12-20T06:02:39.478537+0000","last_became_peered":"2023-12-20T06:02:39.478537+0000","last_unstale":"2023-12-20T06:02:41.342190+0000","last_undegraded":"2023-12-20T06:02:41.342190+0000","last_fullsized":"2023-12-20T06:02:41.342190+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:26:16.444787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"209'8","reported_seq":23,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467468+0000","last_change":"2023-12-20T06:02:26.558119+0000","last_active":"2023-12-20T06:02:39.467468+0000","last_peered":"2023-12-20T06:02:39.467468+0000","last_clean":"2023-12-20T06:02:39.467468+0000","last_became_active":"2023-12-20T06:02:26.557699+0000","last_became_peered":"2023-12-20T06:02:26.557699+0000","last_unstale":"2023-12-20T06:02:39.467468+0000","last_undegraded":"2023-12-20T06:02:39.467468+0000","last_fullsized":"2023-12-20T06:02:39.467468+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:21:20.189262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.455517+0000","last_change":"2023-12-20T06:02:26.978068+0000","last_active":"2023-12-20T06:02:39.455517+0000","last_peered":"2023-12-20T06:02:39.455517+0000","last_clean":"2023-12-20T06:02:39.455517+0000","last_became_active":"2023-12-20T06:02:26.977750+0000","last_became_peered":"2023-12-20T06:02:26.977750+0000","last_unstale":"2023-12-20T06:02:39.455517+0000","last_undegraded":"2023-12-20T06:02:39.455517+0000","last_fullsized":"2023-12-20T06:02:39.455517+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:05:59.363948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338658+0000","last_change":"2023-12-20T06:02:39.069292+0000","last_active":"2023-12-20T06:02:41.338658+0000","last_peered":"2023-12-20T06:02:41.338658+0000","last_clean":"2023-12-20T06:02:41.338658+0000","last_became_active":"2023-12-20T06:02:39.068739+0000","last_became_peered":"2023-12-20T06:02:39.068739+0000","last_unstale":"2023-12-20T06:02:41.338658+0000","last_undegraded":"2023-12-20T06:02:41.338658+0000","last_fullsized":"2023-12-20T06:02:41.338658+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:16:17.340651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.247332+0000","last_change":"2023-12-20T06:02:39.489204+0000","last_active":"2023-12-20T06:02:41.247332+0000","last_peered":"2023-12-20T06:02:41.247332+0000","last_clean":"2023-12-20T06:02:41.247332+0000","last_became_active":"2023-12-20T06:02:39.488895+0000","last_became_peered":"2023-12-20T06:02:39.488895+0000","last_unstale":"2023-12-20T06:02:41.247332+0000","last_undegraded":"2023-12-20T06:02:41.247332+0000","last_fullsized":"2023-12-20T06:02:41.247332+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:17:37.870090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"208'3","reported_seq":18,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464279+0000","last_change":"2023-12-20T06:02:26.555063+0000","last_active":"2023-12-20T06:02:39.464279+0000","last_peered":"2023-12-20T06:02:39.464279+0000","last_clean":"2023-12-20T06:02:39.464279+0000","last_became_active":"2023-12-20T06:02:26.554776+0000","last_became_peered":"2023-12-20T06:02:26.554776+0000","last_unstale":"2023-12-20T06:02:39.464279+0000","last_undegraded":"2023-12-20T06:02:39.464279+0000","last_fullsized":"2023-12-20T06:02:39.464279+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:59:43.701253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.459957+0000","last_change":"2023-12-20T06:02:26.966176+0000","last_active":"2023-12-20T06:02:39.459957+0000","last_peered":"2023-12-20T06:02:39.459957+0000","last_clean":"2023-12-20T06:02:39.459957+0000","last_became_active":"2023-12-20T06:02:26.965898+0000","last_became_peered":"2023-12-20T06:02:26.965898+0000","last_unstale":"2023-12-20T06:02:39.459957+0000","last_undegraded":"2023-12-20T06:02:39.459957+0000","last_fullsized":"2023-12-20T06:02:39.459957+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:54:07.886847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.344475+0000","last_change":"2023-12-20T06:02:39.069670+0000","last_active":"2023-12-20T06:02:41.344475+0000","last_peered":"2023-12-20T06:02:41.344475+0000","last_clean":"2023-12-20T06:02:41.344475+0000","last_became_active":"2023-12-20T06:02:39.069404+0000","last_became_peered":"2023-12-20T06:02:39.069404+0000","last_unstale":"2023-12-20T06:02:41.344475+0000","last_undegraded":"2023-12-20T06:02:41.344475+0000","last_fullsized":"2023-12-20T06:02:41.344475+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:13:24.966993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454993+0000","last_change":"2023-12-20T06:02:39.475003+0000","last_active":"2023-12-20T06:02:40.454993+0000","last_peered":"2023-12-20T06:02:40.454993+0000","last_clean":"2023-12-20T06:02:40.454993+0000","last_became_active":"2023-12-20T06:02:39.473932+0000","last_became_peered":"2023-12-20T06:02:39.473932+0000","last_unstale":"2023-12-20T06:02:40.454993+0000","last_undegraded":"2023-12-20T06:02:40.454993+0000","last_fullsized":"2023-12-20T06:02:40.454993+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:02:36.525311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464547+0000","last_change":"2023-12-20T06:02:26.551507+0000","last_active":"2023-12-20T06:02:39.464547+0000","last_peered":"2023-12-20T06:02:39.464547+0000","last_clean":"2023-12-20T06:02:39.464547+0000","last_became_active":"2023-12-20T06:02:26.551215+0000","last_became_peered":"2023-12-20T06:02:26.551215+0000","last_unstale":"2023-12-20T06:02:39.464547+0000","last_undegraded":"2023-12-20T06:02:39.464547+0000","last_fullsized":"2023-12-20T06:02:39.464547+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:58:17.512600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.470106+0000","last_change":"2023-12-20T06:02:26.981970+0000","last_active":"2023-12-20T06:02:39.470106+0000","last_peered":"2023-12-20T06:02:39.470106+0000","last_clean":"2023-12-20T06:02:39.470106+0000","last_became_active":"2023-12-20T06:02:26.981688+0000","last_became_peered":"2023-12-20T06:02:26.981688+0000","last_unstale":"2023-12-20T06:02:39.470106+0000","last_undegraded":"2023-12-20T06:02:39.470106+0000","last_fullsized":"2023-12-20T06:02:39.470106+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:29:15.785654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"212'6","reported_seq":17,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246990+0000","last_change":"2023-12-20T06:02:39.070888+0000","last_active":"2023-12-20T06:02:41.246990+0000","last_peered":"2023-12-20T06:02:41.246990+0000","last_clean":"2023-12-20T06:02:41.246990+0000","last_became_active":"2023-12-20T06:02:39.070561+0000","last_became_peered":"2023-12-20T06:02:39.070561+0000","last_unstale":"2023-12-20T06:02:41.246990+0000","last_undegraded":"2023-12-20T06:02:41.246990+0000","last_fullsized":"2023-12-20T06:02:41.246990+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:07:20.758077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.247045+0000","last_change":"2023-12-20T06:02:39.482522+0000","last_active":"2023-12-20T06:02:41.247045+0000","last_peered":"2023-12-20T06:02:41.247045+0000","last_clean":"2023-12-20T06:02:41.247045+0000","last_became_active":"2023-12-20T06:02:39.481770+0000","last_became_peered":"2023-12-20T06:02:39.481770+0000","last_unstale":"2023-12-20T06:02:41.247045+0000","last_undegraded":"2023-12-20T06:02:41.247045+0000","last_fullsized":"2023-12-20T06:02:41.247045+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:03:23.864841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465049+0000","last_change":"2023-12-20T06:02:26.557303+0000","last_active":"2023-12-20T06:02:39.465049+0000","last_peered":"2023-12-20T06:02:39.465049+0000","last_clean":"2023-12-20T06:02:39.465049+0000","last_became_active":"2023-12-20T06:02:26.556838+0000","last_became_peered":"2023-12-20T06:02:26.556838+0000","last_unstale":"2023-12-20T06:02:39.465049+0000","last_undegraded":"2023-12-20T06:02:39.465049+0000","last_fullsized":"2023-12-20T06:02:39.465049+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:12:36.511646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467349+0000","last_change":"2023-12-20T06:02:26.974738+0000","last_active":"2023-12-20T06:02:39.467349+0000","last_peered":"2023-12-20T06:02:39.467349+0000","last_clean":"2023-12-20T06:02:39.467349+0000","last_became_active":"2023-12-20T06:02:26.974494+0000","last_became_peered":"2023-12-20T06:02:26.974494+0000","last_unstale":"2023-12-20T06:02:39.467349+0000","last_undegraded":"2023-12-20T06:02:39.467349+0000","last_fullsized":"2023-12-20T06:02:39.467349+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.15","version":"212'3","reported_seq":14,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.344116+0000","last_change":"2023-12-20T06:02:39.067994+0000","last_active":"2023-12-20T06:02:41.344116+0000","last_peered":"2023-12-20T06:02:41.344116+0000","last_clean":"2023-12-20T06:02:41.344116+0000","last_became_active":"2023-12-20T06:02:39.067516+0000","last_became_peered":"2023-12-20T06:02:39.067516+0000","last_unstale":"2023-12-20T06:02:41.344116+0000","last_undegraded":"2023-12-20T06:02:41.344116+0000","last_fullsized":"2023-12-20T06:02:41.344116+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:05:55.241309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246913+0000","last_change":"2023-12-20T06:02:39.484273+0000","last_active":"2023-12-20T06:02:41.246913+0000","last_peered":"2023-12-20T06:02:41.246913+0000","last_clean":"2023-12-20T06:02:41.246913+0000","last_became_active":"2023-12-20T06:02:39.483945+0000","last_became_peered":"2023-12-20T06:02:39.483945+0000","last_unstale":"2023-12-20T06:02:41.246913+0000","last_undegraded":"2023-12-20T06:02:41.246913+0000","last_fullsized":"2023-12-20T06:02:41.246913+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:24.135189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465458+0000","last_change":"2023-12-20T06:02:26.553243+0000","last_active":"2023-12-20T06:02:39.465458+0000","last_peered":"2023-12-20T06:02:39.465458+0000","last_clean":"2023-12-20T06:02:39.465458+0000","last_became_active":"2023-12-20T06:02:26.551953+0000","last_became_peered":"2023-12-20T06:02:26.551953+0000","last_unstale":"2023-12-20T06:02:39.465458+0000","last_undegraded":"2023-12-20T06:02:39.465458+0000","last_fullsized":"2023-12-20T06:02:39.465458+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:09:54.113512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.458492+0000","last_change":"2023-12-20T06:02:26.977844+0000","last_active":"2023-12-20T06:02:39.458492+0000","last_peered":"2023-12-20T06:02:39.458492+0000","last_clean":"2023-12-20T06:02:39.458492+0000","last_became_active":"2023-12-20T06:02:26.977331+0000","last_became_peered":"2023-12-20T06:02:26.977331+0000","last_unstale":"2023-12-20T06:02:39.458492+0000","last_undegraded":"2023-12-20T06:02:39.458492+0000","last_fullsized":"2023-12-20T06:02:39.458492+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:59:51.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1a","version":"212'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342946+0000","last_change":"2023-12-20T06:02:39.070793+0000","last_active":"2023-12-20T06:02:41.342946+0000","last_peered":"2023-12-20T06:02:41.342946+0000","last_clean":"2023-12-20T06:02:41.342946+0000","last_became_active":"2023-12-20T06:02:39.070475+0000","last_became_peered":"2023-12-20T06:02:39.070475+0000","last_unstale":"2023-12-20T06:02:41.342946+0000","last_undegraded":"2023-12-20T06:02:41.342946+0000","last_fullsized":"2023-12-20T06:02:41.342946+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:48:11.323412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246623+0000","last_change":"2023-12-20T06:02:39.480392+0000","last_active":"2023-12-20T06:02:41.246623+0000","last_peered":"2023-12-20T06:02:41.246623+0000","last_clean":"2023-12-20T06:02:41.246623+0000","last_became_active":"2023-12-20T06:02:39.480114+0000","last_became_peered":"2023-12-20T06:02:39.480114+0000","last_unstale":"2023-12-20T06:02:41.246623+0000","last_undegraded":"2023-12-20T06:02:41.246623+0000","last_fullsized":"2023-12-20T06:02:41.246623+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:09.405907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"208'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465572+0000","last_change":"2023-12-20T06:02:26.556300+0000","last_active":"2023-12-20T06:02:39.465572+0000","last_peered":"2023-12-20T06:02:39.465572+0000","last_clean":"2023-12-20T06:02:39.465572+0000","last_became_active":"2023-12-20T06:02:26.556007+0000","last_became_peered":"2023-12-20T06:02:26.556007+0000","last_unstale":"2023-12-20T06:02:39.465572+0000","last_undegraded":"2023-12-20T06:02:39.465572+0000","last_fullsized":"2023-12-20T06:02:39.465572+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:58.838168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465458+0000","last_change":"2023-12-20T06:02:26.966409+0000","last_active":"2023-12-20T06:02:39.465458+0000","last_peered":"2023-12-20T06:02:39.465458+0000","last_clean":"2023-12-20T06:02:39.465458+0000","last_became_active":"2023-12-20T06:02:26.966153+0000","last_became_peered":"2023-12-20T06:02:26.966153+0000","last_unstale":"2023-12-20T06:02:39.465458+0000","last_undegraded":"2023-12-20T06:02:39.465458+0000","last_fullsized":"2023-12-20T06:02:39.465458+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:27.909325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338066+0000","last_change":"2023-12-20T06:02:39.067435+0000","last_active":"2023-12-20T06:02:41.338066+0000","last_peered":"2023-12-20T06:02:41.338066+0000","last_clean":"2023-12-20T06:02:41.338066+0000","last_became_active":"2023-12-20T06:02:39.064642+0000","last_became_peered":"2023-12-20T06:02:39.064642+0000","last_unstale":"2023-12-20T06:02:41.338066+0000","last_undegraded":"2023-12-20T06:02:41.338066+0000","last_fullsized":"2023-12-20T06:02:41.338066+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:35:06.589787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341644+0000","last_change":"2023-12-20T06:02:39.479299+0000","last_active":"2023-12-20T06:02:41.341644+0000","last_peered":"2023-12-20T06:02:41.341644+0000","last_clean":"2023-12-20T06:02:41.341644+0000","last_became_active":"2023-12-20T06:02:39.477232+0000","last_became_peered":"2023-12-20T06:02:39.477232+0000","last_unstale":"2023-12-20T06:02:41.341644+0000","last_undegraded":"2023-12-20T06:02:41.341644+0000","last_fullsized":"2023-12-20T06:02:41.341644+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:20:37.087626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1b","version":"208'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464771+0000","last_change":"2023-12-20T06:02:26.560921+0000","last_active":"2023-12-20T06:02:39.464771+0000","last_peered":"2023-12-20T06:02:39.464771+0000","last_clean":"2023-12-20T06:02:39.464771+0000","last_became_active":"2023-12-20T06:02:26.560567+0000","last_became_peered":"2023-12-20T06:02:26.560567+0000","last_unstale":"2023-12-20T06:02:39.464771+0000","last_undegraded":"2023-12-20T06:02:39.464771+0000","last_fullsized":"2023-12-20T06:02:39.464771+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:01:42.572698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453444+0000","last_change":"2023-12-20T06:02:26.974647+0000","last_active":"2023-12-20T06:02:39.453444+0000","last_peered":"2023-12-20T06:02:39.453444+0000","last_clean":"2023-12-20T06:02:39.453444+0000","last_became_active":"2023-12-20T06:02:26.974369+0000","last_became_peered":"2023-12-20T06:02:26.974369+0000","last_unstale":"2023-12-20T06:02:39.453444+0000","last_undegraded":"2023-12-20T06:02:39.453444+0000","last_fullsized":"2023-12-20T06:02:39.453444+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:36.494152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341647+0000","last_change":"2023-12-20T06:02:39.058821+0000","last_active":"2023-12-20T06:02:41.341647+0000","last_peered":"2023-12-20T06:02:41.341647+0000","last_clean":"2023-12-20T06:02:41.341647+0000","last_became_active":"2023-12-20T06:02:39.057269+0000","last_became_peered":"2023-12-20T06:02:39.057269+0000","last_unstale":"2023-12-20T06:02:41.341647+0000","last_undegraded":"2023-12-20T06:02:41.341647+0000","last_fullsized":"2023-12-20T06:02:41.341647+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:23.116478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338763+0000","last_change":"2023-12-20T06:02:39.478830+0000","last_active":"2023-12-20T06:02:41.338763+0000","last_peered":"2023-12-20T06:02:41.338763+0000","last_clean":"2023-12-20T06:02:41.338763+0000","last_became_active":"2023-12-20T06:02:39.473968+0000","last_became_peered":"2023-12-20T06:02:39.473968+0000","last_unstale":"2023-12-20T06:02:41.338763+0000","last_undegraded":"2023-12-20T06:02:41.338763+0000","last_fullsized":"2023-12-20T06:02:41.338763+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:34:03.880266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"212'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.459150+0000","last_change":"2023-12-20T06:02:39.066507+0000","last_active":"2023-12-20T06:02:40.459150+0000","last_peered":"2023-12-20T06:02:40.459150+0000","last_clean":"2023-12-20T06:02:40.459150+0000","last_became_active":"2023-12-20T06:02:39.066215+0000","last_became_peered":"2023-12-20T06:02:39.066215+0000","last_unstale":"2023-12-20T06:02:40.459150+0000","last_undegraded":"2023-12-20T06:02:40.459150+0000","last_fullsized":"2023-12-20T06:02:40.459150+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:13:24.966993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453539+0000","last_change":"2023-12-20T06:02:26.545330+0000","last_active":"2023-12-20T06:02:39.453539+0000","last_peered":"2023-12-20T06:02:39.453539+0000","last_clean":"2023-12-20T06:02:39.453539+0000","last_became_active":"2023-12-20T06:02:26.545018+0000","last_became_peered":"2023-12-20T06:02:26.545018+0000","last_unstale":"2023-12-20T06:02:39.453539+0000","last_undegraded":"2023-12-20T06:02:39.453539+0000","last_fullsized":"2023-12-20T06:02:39.453539+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:21:20.189262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463698+0000","last_change":"2023-12-20T06:02:26.966379+0000","last_active":"2023-12-20T06:02:39.463698+0000","last_peered":"2023-12-20T06:02:39.463698+0000","last_clean":"2023-12-20T06:02:39.463698+0000","last_became_active":"2023-12-20T06:02:26.965974+0000","last_became_peered":"2023-12-20T06:02:26.965974+0000","last_unstale":"2023-12-20T06:02:39.463698+0000","last_undegraded":"2023-12-20T06:02:39.463698+0000","last_fullsized":"2023-12-20T06:02:39.463698+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:28.156831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458963+0000","last_change":"2023-12-20T06:02:39.483649+0000","last_active":"2023-12-20T06:02:40.458963+0000","last_peered":"2023-12-20T06:02:40.458963+0000","last_clean":"2023-12-20T06:02:40.458963+0000","last_became_active":"2023-12-20T06:02:39.481512+0000","last_became_peered":"2023-12-20T06:02:39.481512+0000","last_unstale":"2023-12-20T06:02:40.458963+0000","last_undegraded":"2023-12-20T06:02:40.458963+0000","last_fullsized":"2023-12-20T06:02:40.458963+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:52:35.601091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"212'4","reported_seq":15,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458734+0000","last_change":"2023-12-20T06:02:39.066833+0000","last_active":"2023-12-20T06:02:40.458734+0000","last_peered":"2023-12-20T06:02:40.458734+0000","last_clean":"2023-12-20T06:02:40.458734+0000","last_became_active":"2023-12-20T06:02:39.066523+0000","last_became_peered":"2023-12-20T06:02:39.066523+0000","last_unstale":"2023-12-20T06:02:40.458734+0000","last_undegraded":"2023-12-20T06:02:40.458734+0000","last_fullsized":"2023-12-20T06:02:40.458734+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:08.387196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1d","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467472+0000","last_change":"2023-12-20T06:02:26.560653+0000","last_active":"2023-12-20T06:02:39.467472+0000","last_peered":"2023-12-20T06:02:39.467472+0000","last_clean":"2023-12-20T06:02:39.467472+0000","last_became_active":"2023-12-20T06:02:26.560323+0000","last_became_peered":"2023-12-20T06:02:26.560323+0000","last_unstale":"2023-12-20T06:02:39.467472+0000","last_undegraded":"2023-12-20T06:02:39.467472+0000","last_fullsized":"2023-12-20T06:02:39.467472+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:55.646229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.459873+0000","last_change":"2023-12-20T06:02:26.967888+0000","last_active":"2023-12-20T06:02:39.459873+0000","last_peered":"2023-12-20T06:02:39.459873+0000","last_clean":"2023-12-20T06:02:39.459873+0000","last_became_active":"2023-12-20T06:02:26.967582+0000","last_became_peered":"2023-12-20T06:02:26.967582+0000","last_unstale":"2023-12-20T06:02:39.459873+0000","last_undegraded":"2023-12-20T06:02:39.459873+0000","last_fullsized":"2023-12-20T06:02:39.459873+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:26:03.948722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343688+0000","last_change":"2023-12-20T06:02:39.475404+0000","last_active":"2023-12-20T06:02:41.343688+0000","last_peered":"2023-12-20T06:02:41.343688+0000","last_clean":"2023-12-20T06:02:41.343688+0000","last_became_active":"2023-12-20T06:02:39.474043+0000","last_became_peered":"2023-12-20T06:02:39.474043+0000","last_unstale":"2023-12-20T06:02:41.343688+0000","last_undegraded":"2023-12-20T06:02:41.343688+0000","last_fullsized":"2023-12-20T06:02:41.343688+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:40:28.433800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"208'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464049+0000","last_change":"2023-12-20T06:02:26.547864+0000","last_active":"2023-12-20T06:02:39.464049+0000","last_peered":"2023-12-20T06:02:39.464049+0000","last_clean":"2023-12-20T06:02:39.464049+0000","last_became_active":"2023-12-20T06:02:26.547556+0000","last_became_peered":"2023-12-20T06:02:26.547556+0000","last_unstale":"2023-12-20T06:02:39.464049+0000","last_undegraded":"2023-12-20T06:02:39.464049+0000","last_fullsized":"2023-12-20T06:02:39.464049+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:04.689001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.456356+0000","last_change":"2023-12-20T06:02:26.978246+0000","last_active":"2023-12-20T06:02:39.456356+0000","last_peered":"2023-12-20T06:02:39.456356+0000","last_clean":"2023-12-20T06:02:39.456356+0000","last_became_active":"2023-12-20T06:02:26.977975+0000","last_became_peered":"2023-12-20T06:02:26.977975+0000","last_unstale":"2023-12-20T06:02:39.456356+0000","last_undegraded":"2023-12-20T06:02:39.456356+0000","last_fullsized":"2023-12-20T06:02:39.456356+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:05:43.763955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1c","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.247315+0000","last_change":"2023-12-20T06:02:39.068719+0000","last_active":"2023-12-20T06:02:41.247315+0000","last_peered":"2023-12-20T06:02:41.247315+0000","last_clean":"2023-12-20T06:02:41.247315+0000","last_became_active":"2023-12-20T06:02:39.068439+0000","last_became_peered":"2023-12-20T06:02:39.068439+0000","last_unstale":"2023-12-20T06:02:41.247315+0000","last_undegraded":"2023-12-20T06:02:41.247315+0000","last_fullsized":"2023-12-20T06:02:41.247315+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:03:56.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454643+0000","last_change":"2023-12-20T06:02:39.471635+0000","last_active":"2023-12-20T06:02:40.454643+0000","last_peered":"2023-12-20T06:02:40.454643+0000","last_clean":"2023-12-20T06:02:40.454643+0000","last_became_active":"2023-12-20T06:02:39.471384+0000","last_became_peered":"2023-12-20T06:02:39.471384+0000","last_unstale":"2023-12-20T06:02:40.454643+0000","last_undegraded":"2023-12-20T06:02:40.454643+0000","last_fullsized":"2023-12-20T06:02:40.454643+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:03.707844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1f","version":"208'4","reported_seq":19,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.468962+0000","last_change":"2023-12-20T06:02:26.559340+0000","last_active":"2023-12-20T06:02:39.468962+0000","last_peered":"2023-12-20T06:02:39.468962+0000","last_clean":"2023-12-20T06:02:39.468962+0000","last_became_active":"2023-12-20T06:02:26.559045+0000","last_became_peered":"2023-12-20T06:02:26.559045+0000","last_unstale":"2023-12-20T06:02:39.468962+0000","last_undegraded":"2023-12-20T06:02:39.468962+0000","last_fullsized":"2023-12-20T06:02:39.468962+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:33.930072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.468546+0000","last_change":"2023-12-20T06:02:26.966538+0000","last_active":"2023-12-20T06:02:39.468546+0000","last_peered":"2023-12-20T06:02:39.468546+0000","last_clean":"2023-12-20T06:02:39.468546+0000","last_became_active":"2023-12-20T06:02:26.966278+0000","last_became_peered":"2023-12-20T06:02:26.966278+0000","last_unstale":"2023-12-20T06:02:39.468546+0000","last_undegraded":"2023-12-20T06:02:39.468546+0000","last_fullsized":"2023-12-20T06:02:39.468546+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:12.114643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1d","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342867+0000","last_change":"2023-12-20T06:02:39.066219+0000","last_active":"2023-12-20T06:02:41.342867+0000","last_peered":"2023-12-20T06:02:41.342867+0000","last_clean":"2023-12-20T06:02:41.342867+0000","last_became_active":"2023-12-20T06:02:39.065906+0000","last_became_peered":"2023-12-20T06:02:39.065906+0000","last_unstale":"2023-12-20T06:02:41.342867+0000","last_undegraded":"2023-12-20T06:02:41.342867+0000","last_fullsized":"2023-12-20T06:02:41.342867+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:05:06.470711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457234+0000","last_change":"2023-12-20T06:02:39.479009+0000","last_active":"2023-12-20T06:02:40.457234+0000","last_peered":"2023-12-20T06:02:40.457234+0000","last_clean":"2023-12-20T06:02:40.457234+0000","last_became_active":"2023-12-20T06:02:39.478661+0000","last_became_peered":"2023-12-20T06:02:39.478661+0000","last_unstale":"2023-12-20T06:02:40.457234+0000","last_undegraded":"2023-12-20T06:02:40.457234+0000","last_fullsized":"2023-12-20T06:02:40.457234+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:24.610708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464761+0000","last_change":"2023-12-20T06:02:26.963846+0000","last_active":"2023-12-20T06:02:39.464761+0000","last_peered":"2023-12-20T06:02:39.464761+0000","last_clean":"2023-12-20T06:02:39.464761+0000","last_became_active":"2023-12-20T06:02:26.963552+0000","last_became_peered":"2023-12-20T06:02:26.963552+0000","last_unstale":"2023-12-20T06:02:39.464761+0000","last_undegraded":"2023-12-20T06:02:39.464761+0000","last_fullsized":"2023-12-20T06:02:39.464761+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:17:08.792842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457537+0000","last_change":"2023-12-20T06:02:39.472176+0000","last_active":"2023-12-20T06:02:40.457537+0000","last_peered":"2023-12-20T06:02:40.457537+0000","last_clean":"2023-12-20T06:02:40.457537+0000","last_became_active":"2023-12-20T06:02:39.471883+0000","last_became_peered":"2023-12-20T06:02:39.471883+0000","last_unstale":"2023-12-20T06:02:40.457537+0000","last_undegraded":"2023-12-20T06:02:40.457537+0000","last_fullsized":"2023-12-20T06:02:40.457537+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:59.177389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.460941+0000","last_change":"2023-12-20T06:02:26.965576+0000","last_active":"2023-12-20T06:02:39.460941+0000","last_peered":"2023-12-20T06:02:39.460941+0000","last_clean":"2023-12-20T06:02:39.460941+0000","last_became_active":"2023-12-20T06:02:26.965189+0000","last_became_peered":"2023-12-20T06:02:26.965189+0000","last_unstale":"2023-12-20T06:02:39.460941+0000","last_undegraded":"2023-12-20T06:02:39.460941+0000","last_fullsized":"2023-12-20T06:02:39.460941+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:38.051872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454119+0000","last_change":"2023-12-20T06:02:39.466101+0000","last_active":"2023-12-20T06:02:40.454119+0000","last_peered":"2023-12-20T06:02:40.454119+0000","last_clean":"2023-12-20T06:02:40.454119+0000","last_became_active":"2023-12-20T06:02:39.465827+0000","last_became_peered":"2023-12-20T06:02:39.465827+0000","last_unstale":"2023-12-20T06:02:40.454119+0000","last_undegraded":"2023-12-20T06:02:40.454119+0000","last_fullsized":"2023-12-20T06:02:40.454119+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:20:35.645018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464768+0000","last_change":"2023-12-20T06:02:26.977346+0000","last_active":"2023-12-20T06:02:39.464768+0000","last_peered":"2023-12-20T06:02:39.464768+0000","last_clean":"2023-12-20T06:02:39.464768+0000","last_became_active":"2023-12-20T06:02:26.976527+0000","last_became_peered":"2023-12-20T06:02:26.976527+0000","last_unstale":"2023-12-20T06:02:39.464768+0000","last_undegraded":"2023-12-20T06:02:39.464768+0000","last_fullsized":"2023-12-20T06:02:39.464768+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:11.127223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341625+0000","last_change":"2023-12-20T06:02:39.473998+0000","last_active":"2023-12-20T06:02:41.341625+0000","last_peered":"2023-12-20T06:02:41.341625+0000","last_clean":"2023-12-20T06:02:41.341625+0000","last_became_active":"2023-12-20T06:02:39.473011+0000","last_became_peered":"2023-12-20T06:02:39.473011+0000","last_unstale":"2023-12-20T06:02:41.341625+0000","last_undegraded":"2023-12-20T06:02:41.341625+0000","last_fullsized":"2023-12-20T06:02:41.341625+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:57:42.952159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452445+0000","last_change":"2023-12-20T06:02:26.973161+0000","last_active":"2023-12-20T06:02:39.452445+0000","last_peered":"2023-12-20T06:02:39.452445+0000","last_clean":"2023-12-20T06:02:39.452445+0000","last_became_active":"2023-12-20T06:02:26.972883+0000","last_became_peered":"2023-12-20T06:02:26.972883+0000","last_unstale":"2023-12-20T06:02:39.452445+0000","last_undegraded":"2023-12-20T06:02:39.452445+0000","last_fullsized":"2023-12-20T06:02:39.452445+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:11.127223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.455126+0000","last_change":"2023-12-20T06:02:39.482573+0000","last_active":"2023-12-20T06:02:40.455126+0000","last_peered":"2023-12-20T06:02:40.455126+0000","last_clean":"2023-12-20T06:02:40.455126+0000","last_became_active":"2023-12-20T06:02:39.482213+0000","last_became_peered":"2023-12-20T06:02:39.482213+0000","last_unstale":"2023-12-20T06:02:40.455126+0000","last_undegraded":"2023-12-20T06:02:40.455126+0000","last_fullsized":"2023-12-20T06:02:40.455126+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:30.717618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465106+0000","last_change":"2023-12-20T06:02:26.977513+0000","last_active":"2023-12-20T06:02:39.465106+0000","last_peered":"2023-12-20T06:02:39.465106+0000","last_clean":"2023-12-20T06:02:39.465106+0000","last_became_active":"2023-12-20T06:02:26.977220+0000","last_became_peered":"2023-12-20T06:02:26.977220+0000","last_unstale":"2023-12-20T06:02:39.465106+0000","last_undegraded":"2023-12-20T06:02:39.465106+0000","last_fullsized":"2023-12-20T06:02:39.465106+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:52.600966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458787+0000","last_change":"2023-12-20T06:02:39.477148+0000","last_active":"2023-12-20T06:02:40.458787+0000","last_peered":"2023-12-20T06:02:40.458787+0000","last_clean":"2023-12-20T06:02:40.458787+0000","last_became_active":"2023-12-20T06:02:39.476654+0000","last_became_peered":"2023-12-20T06:02:39.476654+0000","last_unstale":"2023-12-20T06:02:40.458787+0000","last_undegraded":"2023-12-20T06:02:40.458787+0000","last_fullsized":"2023-12-20T06:02:40.458787+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:20:37.087626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467349+0000","last_change":"2023-12-20T06:02:26.981676+0000","last_active":"2023-12-20T06:02:39.467349+0000","last_peered":"2023-12-20T06:02:39.467349+0000","last_clean":"2023-12-20T06:02:39.467349+0000","last_became_active":"2023-12-20T06:02:26.980905+0000","last_became_peered":"2023-12-20T06:02:26.980905+0000","last_unstale":"2023-12-20T06:02:39.467349+0000","last_undegraded":"2023-12-20T06:02:39.467349+0000","last_fullsized":"2023-12-20T06:02:39.467349+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:43:42.660130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342532+0000","last_change":"2023-12-20T06:02:39.482998+0000","last_active":"2023-12-20T06:02:41.342532+0000","last_peered":"2023-12-20T06:02:41.342532+0000","last_clean":"2023-12-20T06:02:41.342532+0000","last_became_active":"2023-12-20T06:02:39.482499+0000","last_became_peered":"2023-12-20T06:02:39.482499+0000","last_unstale":"2023-12-20T06:02:41.342532+0000","last_undegraded":"2023-12-20T06:02:41.342532+0000","last_fullsized":"2023-12-20T06:02:41.342532+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:48:09.959849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.466629+0000","last_change":"2023-12-20T06:02:26.979767+0000","last_active":"2023-12-20T06:02:39.466629+0000","last_peered":"2023-12-20T06:02:39.466629+0000","last_clean":"2023-12-20T06:02:39.466629+0000","last_became_active":"2023-12-20T06:02:26.979450+0000","last_became_peered":"2023-12-20T06:02:26.979450+0000","last_unstale":"2023-12-20T06:02:39.466629+0000","last_undegraded":"2023-12-20T06:02:39.466629+0000","last_fullsized":"2023-12-20T06:02:39.466629+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:47:57.463784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458283+0000","last_change":"2023-12-20T06:02:39.472365+0000","last_active":"2023-12-20T06:02:40.458283+0000","last_peered":"2023-12-20T06:02:40.458283+0000","last_clean":"2023-12-20T06:02:40.458283+0000","last_became_active":"2023-12-20T06:02:39.468354+0000","last_became_peered":"2023-12-20T06:02:39.468354+0000","last_unstale":"2023-12-20T06:02:40.458283+0000","last_undegraded":"2023-12-20T06:02:40.458283+0000","last_fullsized":"2023-12-20T06:02:40.458283+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:21.530775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453535+0000","last_change":"2023-12-20T06:02:26.976267+0000","last_active":"2023-12-20T06:02:39.453535+0000","last_peered":"2023-12-20T06:02:39.453535+0000","last_clean":"2023-12-20T06:02:39.453535+0000","last_became_active":"2023-12-20T06:02:26.975927+0000","last_became_peered":"2023-12-20T06:02:26.975927+0000","last_unstale":"2023-12-20T06:02:39.453535+0000","last_undegraded":"2023-12-20T06:02:39.453535+0000","last_fullsized":"2023-12-20T06:02:39.453535+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:19:37.257026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338301+0000","last_change":"2023-12-20T06:02:39.476377+0000","last_active":"2023-12-20T06:02:41.338301+0000","last_peered":"2023-12-20T06:02:41.338301+0000","last_clean":"2023-12-20T06:02:41.338301+0000","last_became_active":"2023-12-20T06:02:39.476094+0000","last_became_peered":"2023-12-20T06:02:39.476094+0000","last_unstale":"2023-12-20T06:02:41.338301+0000","last_undegraded":"2023-12-20T06:02:41.338301+0000","last_fullsized":"2023-12-20T06:02:41.338301+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:23:41.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464888+0000","last_change":"2023-12-20T06:02:26.965794+0000","last_active":"2023-12-20T06:02:39.464888+0000","last_peered":"2023-12-20T06:02:39.464888+0000","last_clean":"2023-12-20T06:02:39.464888+0000","last_became_active":"2023-12-20T06:02:26.965505+0000","last_became_peered":"2023-12-20T06:02:26.965505+0000","last_unstale":"2023-12-20T06:02:39.464888+0000","last_undegraded":"2023-12-20T06:02:39.464888+0000","last_fullsized":"2023-12-20T06:02:39.464888+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:09.034710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246668+0000","last_change":"2023-12-20T06:02:39.486650+0000","last_active":"2023-12-20T06:02:41.246668+0000","last_peered":"2023-12-20T06:02:41.246668+0000","last_clean":"2023-12-20T06:02:41.246668+0000","last_became_active":"2023-12-20T06:02:39.486371+0000","last_became_peered":"2023-12-20T06:02:39.486371+0000","last_unstale":"2023-12-20T06:02:41.246668+0000","last_undegraded":"2023-12-20T06:02:41.246668+0000","last_fullsized":"2023-12-20T06:02:41.246668+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:48:12.342123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.460864+0000","last_change":"2023-12-20T06:02:26.979485+0000","last_active":"2023-12-20T06:02:39.460864+0000","last_peered":"2023-12-20T06:02:39.460864+0000","last_clean":"2023-12-20T06:02:39.460864+0000","last_became_active":"2023-12-20T06:02:26.979208+0000","last_became_peered":"2023-12-20T06:02:26.979208+0000","last_unstale":"2023-12-20T06:02:39.460864+0000","last_undegraded":"2023-12-20T06:02:39.460864+0000","last_fullsized":"2023-12-20T06:02:39.460864+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:04:09.949376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246538+0000","last_change":"2023-12-20T06:02:39.486665+0000","last_active":"2023-12-20T06:02:41.246538+0000","last_peered":"2023-12-20T06:02:41.246538+0000","last_clean":"2023-12-20T06:02:41.246538+0000","last_became_active":"2023-12-20T06:02:39.486378+0000","last_became_peered":"2023-12-20T06:02:39.486378+0000","last_unstale":"2023-12-20T06:02:41.246538+0000","last_undegraded":"2023-12-20T06:02:41.246538+0000","last_fullsized":"2023-12-20T06:02:41.246538+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:13:25.985704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453188+0000","last_change":"2023-12-20T06:02:26.975847+0000","last_active":"2023-12-20T06:02:39.453188+0000","last_peered":"2023-12-20T06:02:39.453188+0000","last_clean":"2023-12-20T06:02:39.453188+0000","last_became_active":"2023-12-20T06:02:26.975128+0000","last_became_peered":"2023-12-20T06:02:26.975128+0000","last_unstale":"2023-12-20T06:02:39.453188+0000","last_undegraded":"2023-12-20T06:02:39.453188+0000","last_fullsized":"2023-12-20T06:02:39.453188+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:39:21.684738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338200+0000","last_change":"2023-12-20T06:02:39.472707+0000","last_active":"2023-12-20T06:02:41.338200+0000","last_peered":"2023-12-20T06:02:41.338200+0000","last_clean":"2023-12-20T06:02:41.338200+0000","last_became_active":"2023-12-20T06:02:39.472409+0000","last_became_peered":"2023-12-20T06:02:39.472409+0000","last_unstale":"2023-12-20T06:02:41.338200+0000","last_undegraded":"2023-12-20T06:02:41.338200+0000","last_fullsized":"2023-12-20T06:02:41.338200+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:36:12.275613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464662+0000","last_change":"2023-12-20T06:02:26.969431+0000","last_active":"2023-12-20T06:02:39.464662+0000","last_peered":"2023-12-20T06:02:39.464662+0000","last_clean":"2023-12-20T06:02:39.464662+0000","last_became_active":"2023-12-20T06:02:26.969161+0000","last_became_peered":"2023-12-20T06:02:26.969161+0000","last_unstale":"2023-12-20T06:02:39.464662+0000","last_undegraded":"2023-12-20T06:02:39.464662+0000","last_fullsized":"2023-12-20T06:02:39.464662+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:49:57.166700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343953+0000","last_change":"2023-12-20T06:02:39.472245+0000","last_active":"2023-12-20T06:02:41.343953+0000","last_peered":"2023-12-20T06:02:41.343953+0000","last_clean":"2023-12-20T06:02:41.343953+0000","last_became_active":"2023-12-20T06:02:39.471954+0000","last_became_peered":"2023-12-20T06:02:39.471954+0000","last_unstale":"2023-12-20T06:02:41.343953+0000","last_undegraded":"2023-12-20T06:02:41.343953+0000","last_fullsized":"2023-12-20T06:02:41.343953+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:07:55.492672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464936+0000","last_change":"2023-12-20T06:02:26.957746+0000","last_active":"2023-12-20T06:02:39.464936+0000","last_peered":"2023-12-20T06:02:39.464936+0000","last_clean":"2023-12-20T06:02:39.464936+0000","last_became_active":"2023-12-20T06:02:26.957407+0000","last_became_peered":"2023-12-20T06:02:26.957407+0000","last_unstale":"2023-12-20T06:02:39.464936+0000","last_undegraded":"2023-12-20T06:02:39.464936+0000","last_fullsized":"2023-12-20T06:02:39.464936+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:51.962994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.455202+0000","last_change":"2023-12-20T06:02:39.476908+0000","last_active":"2023-12-20T06:02:40.455202+0000","last_peered":"2023-12-20T06:02:40.455202+0000","last_clean":"2023-12-20T06:02:40.455202+0000","last_became_active":"2023-12-20T06:02:39.476633+0000","last_became_peered":"2023-12-20T06:02:39.476633+0000","last_unstale":"2023-12-20T06:02:40.455202+0000","last_undegraded":"2023-12-20T06:02:40.455202+0000","last_fullsized":"2023-12-20T06:02:40.455202+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:36:47.240370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.455442+0000","last_change":"2023-12-20T06:02:26.972124+0000","last_active":"2023-12-20T06:02:39.455442+0000","last_peered":"2023-12-20T06:02:39.455442+0000","last_clean":"2023-12-20T06:02:39.455442+0000","last_became_active":"2023-12-20T06:02:26.971870+0000","last_became_peered":"2023-12-20T06:02:26.971870+0000","last_unstale":"2023-12-20T06:02:39.455442+0000","last_undegraded":"2023-12-20T06:02:39.455442+0000","last_fullsized":"2023-12-20T06:02:39.455442+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:06:44.203571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343934+0000","last_change":"2023-12-20T06:02:39.475322+0000","last_active":"2023-12-20T06:02:41.343934+0000","last_peered":"2023-12-20T06:02:41.343934+0000","last_clean":"2023-12-20T06:02:41.343934+0000","last_became_active":"2023-12-20T06:02:39.474906+0000","last_became_peered":"2023-12-20T06:02:39.474906+0000","last_unstale":"2023-12-20T06:02:41.343934+0000","last_undegraded":"2023-12-20T06:02:41.343934+0000","last_fullsized":"2023-12-20T06:02:41.343934+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:21.530775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467816+0000","last_change":"2023-12-20T06:02:26.981004+0000","last_active":"2023-12-20T06:02:39.467816+0000","last_peered":"2023-12-20T06:02:39.467816+0000","last_clean":"2023-12-20T06:02:39.467816+0000","last_became_active":"2023-12-20T06:02:26.980597+0000","last_became_peered":"2023-12-20T06:02:26.980597+0000","last_unstale":"2023-12-20T06:02:39.467816+0000","last_undegraded":"2023-12-20T06:02:39.467816+0000","last_fullsized":"2023-12-20T06:02:39.467816+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:13.705554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.344045+0000","last_change":"2023-12-20T06:02:39.475270+0000","last_active":"2023-12-20T06:02:41.344045+0000","last_peered":"2023-12-20T06:02:41.344045+0000","last_clean":"2023-12-20T06:02:41.344045+0000","last_became_active":"2023-12-20T06:02:39.472631+0000","last_became_peered":"2023-12-20T06:02:39.472631+0000","last_unstale":"2023-12-20T06:02:41.344045+0000","last_undegraded":"2023-12-20T06:02:41.344045+0000","last_fullsized":"2023-12-20T06:02:41.344045+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:15:43.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465373+0000","last_change":"2023-12-20T06:02:26.959816+0000","last_active":"2023-12-20T06:02:39.465373+0000","last_peered":"2023-12-20T06:02:39.465373+0000","last_clean":"2023-12-20T06:02:39.465373+0000","last_became_active":"2023-12-20T06:02:26.959542+0000","last_became_peered":"2023-12-20T06:02:26.959542+0000","last_unstale":"2023-12-20T06:02:39.465373+0000","last_undegraded":"2023-12-20T06:02:39.465373+0000","last_fullsized":"2023-12-20T06:02:39.465373+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:49:15.081247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454373+0000","last_change":"2023-12-20T06:02:39.459375+0000","last_active":"2023-12-20T06:02:40.454373+0000","last_peered":"2023-12-20T06:02:40.454373+0000","last_clean":"2023-12-20T06:02:40.454373+0000","last_became_active":"2023-12-20T06:02:39.459059+0000","last_became_peered":"2023-12-20T06:02:39.459059+0000","last_unstale":"2023-12-20T06:02:40.454373+0000","last_undegraded":"2023-12-20T06:02:40.454373+0000","last_fullsized":"2023-12-20T06:02:40.454373+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:40:30.277423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464885+0000","last_change":"2023-12-20T06:02:26.966987+0000","last_active":"2023-12-20T06:02:39.464885+0000","last_peered":"2023-12-20T06:02:39.464885+0000","last_clean":"2023-12-20T06:02:39.464885+0000","last_became_active":"2023-12-20T06:02:26.966704+0000","last_became_peered":"2023-12-20T06:02:26.966704+0000","last_unstale":"2023-12-20T06:02:39.464885+0000","last_undegraded":"2023-12-20T06:02:39.464885+0000","last_fullsized":"2023-12-20T06:02:39.464885+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:41:37.729575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457770+0000","last_change":"2023-12-20T06:02:39.461702+0000","last_active":"2023-12-20T06:02:40.457770+0000","last_peered":"2023-12-20T06:02:40.457770+0000","last_clean":"2023-12-20T06:02:40.457770+0000","last_became_active":"2023-12-20T06:02:39.461185+0000","last_became_peered":"2023-12-20T06:02:39.461185+0000","last_unstale":"2023-12-20T06:02:40.457770+0000","last_undegraded":"2023-12-20T06:02:40.457770+0000","last_fullsized":"2023-12-20T06:02:40.457770+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:38:05.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452377+0000","last_change":"2023-12-20T06:02:26.974037+0000","last_active":"2023-12-20T06:02:39.452377+0000","last_peered":"2023-12-20T06:02:39.452377+0000","last_clean":"2023-12-20T06:02:39.452377+0000","last_became_active":"2023-12-20T06:02:26.973742+0000","last_became_peered":"2023-12-20T06:02:26.973742+0000","last_unstale":"2023-12-20T06:02:39.452377+0000","last_undegraded":"2023-12-20T06:02:39.452377+0000","last_fullsized":"2023-12-20T06:02:39.452377+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341826+0000","last_change":"2023-12-20T06:02:39.478307+0000","last_active":"2023-12-20T06:02:41.341826+0000","last_peered":"2023-12-20T06:02:41.341826+0000","last_clean":"2023-12-20T06:02:41.341826+0000","last_became_active":"2023-12-20T06:02:39.478043+0000","last_became_peered":"2023-12-20T06:02:39.478043+0000","last_unstale":"2023-12-20T06:02:41.341826+0000","last_undegraded":"2023-12-20T06:02:41.341826+0000","last_fullsized":"2023-12-20T06:02:41.341826+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:44:56.585869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464687+0000","last_change":"2023-12-20T06:02:26.978685+0000","last_active":"2023-12-20T06:02:39.464687+0000","last_peered":"2023-12-20T06:02:39.464687+0000","last_clean":"2023-12-20T06:02:39.464687+0000","last_became_active":"2023-12-20T06:02:26.978364+0000","last_became_peered":"2023-12-20T06:02:26.978364+0000","last_unstale":"2023-12-20T06:02:39.464687+0000","last_undegraded":"2023-12-20T06:02:39.464687+0000","last_fullsized":"2023-12-20T06:02:39.464687+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:19:37.257026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454205+0000","last_change":"2023-12-20T06:02:39.471033+0000","last_active":"2023-12-20T06:02:40.454205+0000","last_peered":"2023-12-20T06:02:40.454205+0000","last_clean":"2023-12-20T06:02:40.454205+0000","last_became_active":"2023-12-20T06:02:39.470713+0000","last_became_peered":"2023-12-20T06:02:39.470713+0000","last_unstale":"2023-12-20T06:02:40.454205+0000","last_undegraded":"2023-12-20T06:02:40.454205+0000","last_fullsized":"2023-12-20T06:02:40.454205+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:16:31.234579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452352+0000","last_change":"2023-12-20T06:02:26.967177+0000","last_active":"2023-12-20T06:02:39.452352+0000","last_peered":"2023-12-20T06:02:39.452352+0000","last_clean":"2023-12-20T06:02:39.452352+0000","last_became_active":"2023-12-20T06:02:26.966560+0000","last_became_peered":"2023-12-20T06:02:26.966560+0000","last_unstale":"2023-12-20T06:02:39.452352+0000","last_undegraded":"2023-12-20T06:02:39.452352+0000","last_fullsized":"2023-12-20T06:02:39.452352+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:12:37.519537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343695+0000","last_change":"2023-12-20T06:02:39.472321+0000","last_active":"2023-12-20T06:02:41.343695+0000","last_peered":"2023-12-20T06:02:41.343695+0000","last_clean":"2023-12-20T06:02:41.343695+0000","last_became_active":"2023-12-20T06:02:39.471969+0000","last_became_peered":"2023-12-20T06:02:39.471969+0000","last_unstale":"2023-12-20T06:02:41.343695+0000","last_undegraded":"2023-12-20T06:02:41.343695+0000","last_fullsized":"2023-12-20T06:02:41.343695+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:25:59.524917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452876+0000","last_change":"2023-12-20T06:02:26.976708+0000","last_active":"2023-12-20T06:02:39.452876+0000","last_peered":"2023-12-20T06:02:39.452876+0000","last_clean":"2023-12-20T06:02:39.452876+0000","last_became_active":"2023-12-20T06:02:26.976349+0000","last_became_peered":"2023-12-20T06:02:26.976349+0000","last_unstale":"2023-12-20T06:02:39.452876+0000","last_undegraded":"2023-12-20T06:02:39.452876+0000","last_fullsized":"2023-12-20T06:02:39.452876+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.459397+0000","last_change":"2023-12-20T06:02:39.482121+0000","last_active":"2023-12-20T06:02:40.459397+0000","last_peered":"2023-12-20T06:02:40.459397+0000","last_clean":"2023-12-20T06:02:40.459397+0000","last_became_active":"2023-12-20T06:02:39.481795+0000","last_became_peered":"2023-12-20T06:02:39.481795+0000","last_unstale":"2023-12-20T06:02:40.459397+0000","last_undegraded":"2023-12-20T06:02:40.459397+0000","last_fullsized":"2023-12-20T06:02:40.459397+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:58.188244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467588+0000","last_change":"2023-12-20T06:02:26.976628+0000","last_active":"2023-12-20T06:02:39.467588+0000","last_peered":"2023-12-20T06:02:39.467588+0000","last_clean":"2023-12-20T06:02:39.467588+0000","last_became_active":"2023-12-20T06:02:26.976337+0000","last_became_peered":"2023-12-20T06:02:26.976337+0000","last_unstale":"2023-12-20T06:02:39.467588+0000","last_undegraded":"2023-12-20T06:02:39.467588+0000","last_fullsized":"2023-12-20T06:02:39.467588+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:15.147652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458033+0000","last_change":"2023-12-20T06:02:39.466735+0000","last_active":"2023-12-20T06:02:40.458033+0000","last_peered":"2023-12-20T06:02:40.458033+0000","last_clean":"2023-12-20T06:02:40.458033+0000","last_became_active":"2023-12-20T06:02:39.466439+0000","last_became_peered":"2023-12-20T06:02:39.466439+0000","last_unstale":"2023-12-20T06:02:40.458033+0000","last_undegraded":"2023-12-20T06:02:40.458033+0000","last_fullsized":"2023-12-20T06:02:40.458033+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:41:50.225640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.454838+0000","last_change":"2023-12-20T06:02:26.973655+0000","last_active":"2023-12-20T06:02:39.454838+0000","last_peered":"2023-12-20T06:02:39.454838+0000","last_clean":"2023-12-20T06:02:39.454838+0000","last_became_active":"2023-12-20T06:02:26.973281+0000","last_became_peered":"2023-12-20T06:02:26.973281+0000","last_unstale":"2023-12-20T06:02:39.454838+0000","last_undegraded":"2023-12-20T06:02:39.454838+0000","last_fullsized":"2023-12-20T06:02:39.454838+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:36.523098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342824+0000","last_change":"2023-12-20T06:02:39.481487+0000","last_active":"2023-12-20T06:02:41.342824+0000","last_peered":"2023-12-20T06:02:41.342824+0000","last_clean":"2023-12-20T06:02:41.342824+0000","last_became_active":"2023-12-20T06:02:39.481013+0000","last_became_peered":"2023-12-20T06:02:39.481013+0000","last_unstale":"2023-12-20T06:02:41.342824+0000","last_undegraded":"2023-12-20T06:02:41.342824+0000","last_fullsized":"2023-12-20T06:02:41.342824+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:03:22.032925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464010+0000","last_change":"2023-12-20T06:02:26.960947+0000","last_active":"2023-12-20T06:02:39.464010+0000","last_peered":"2023-12-20T06:02:39.464010+0000","last_clean":"2023-12-20T06:02:39.464010+0000","last_became_active":"2023-12-20T06:02:26.958931+0000","last_became_peered":"2023-12-20T06:02:26.958931+0000","last_unstale":"2023-12-20T06:02:39.464010+0000","last_undegraded":"2023-12-20T06:02:39.464010+0000","last_fullsized":"2023-12-20T06:02:39.464010+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342785+0000","last_change":"2023-12-20T06:02:39.484029+0000","last_active":"2023-12-20T06:02:41.342785+0000","last_peered":"2023-12-20T06:02:41.342785+0000","last_clean":"2023-12-20T06:02:41.342785+0000","last_became_active":"2023-12-20T06:02:39.483768+0000","last_became_peered":"2023-12-20T06:02:39.483768+0000","last_unstale":"2023-12-20T06:02:41.342785+0000","last_undegraded":"2023-12-20T06:02:41.342785+0000","last_fullsized":"2023-12-20T06:02:41.342785+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:59.177389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464784+0000","last_change":"2023-12-20T06:02:26.971111+0000","last_active":"2023-12-20T06:02:39.464784+0000","last_peered":"2023-12-20T06:02:39.464784+0000","last_clean":"2023-12-20T06:02:39.464784+0000","last_became_active":"2023-12-20T06:02:26.970783+0000","last_became_peered":"2023-12-20T06:02:26.970783+0000","last_unstale":"2023-12-20T06:02:39.464784+0000","last_undegraded":"2023-12-20T06:02:39.464784+0000","last_fullsized":"2023-12-20T06:02:39.464784+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:45.692179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343808+0000","last_change":"2023-12-20T06:02:39.479054+0000","last_active":"2023-12-20T06:02:41.343808+0000","last_peered":"2023-12-20T06:02:41.343808+0000","last_clean":"2023-12-20T06:02:41.343808+0000","last_became_active":"2023-12-20T06:02:39.477268+0000","last_became_peered":"2023-12-20T06:02:39.477268+0000","last_unstale":"2023-12-20T06:02:41.343808+0000","last_undegraded":"2023-12-20T06:02:41.343808+0000","last_fullsized":"2023-12-20T06:02:41.343808+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:40.405390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.458405+0000","last_change":"2023-12-20T06:02:26.978795+0000","last_active":"2023-12-20T06:02:39.458405+0000","last_peered":"2023-12-20T06:02:39.458405+0000","last_clean":"2023-12-20T06:02:39.458405+0000","last_became_active":"2023-12-20T06:02:26.978497+0000","last_became_peered":"2023-12-20T06:02:26.978497+0000","last_unstale":"2023-12-20T06:02:39.458405+0000","last_undegraded":"2023-12-20T06:02:39.458405+0000","last_fullsized":"2023-12-20T06:02:39.458405+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:06:01.713370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342662+0000","last_change":"2023-12-20T06:02:39.468047+0000","last_active":"2023-12-20T06:02:41.342662+0000","last_peered":"2023-12-20T06:02:41.342662+0000","last_clean":"2023-12-20T06:02:41.342662+0000","last_became_active":"2023-12-20T06:02:39.467724+0000","last_became_peered":"2023-12-20T06:02:39.467724+0000","last_unstale":"2023-12-20T06:02:41.342662+0000","last_undegraded":"2023-12-20T06:02:41.342662+0000","last_fullsized":"2023-12-20T06:02:41.342662+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:58.188244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463986+0000","last_change":"2023-12-20T06:02:26.979634+0000","last_active":"2023-12-20T06:02:39.463986+0000","last_peered":"2023-12-20T06:02:39.463986+0000","last_clean":"2023-12-20T06:02:39.463986+0000","last_became_active":"2023-12-20T06:02:26.979366+0000","last_became_peered":"2023-12-20T06:02:26.979366+0000","last_unstale":"2023-12-20T06:02:39.463986+0000","last_undegraded":"2023-12-20T06:02:39.463986+0000","last_fullsized":"2023-12-20T06:02:39.463986+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:34:19.170698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454484+0000","last_change":"2023-12-20T06:02:39.477742+0000","last_active":"2023-12-20T06:02:40.454484+0000","last_peered":"2023-12-20T06:02:40.454484+0000","last_clean":"2023-12-20T06:02:40.454484+0000","last_became_active":"2023-12-20T06:02:39.477334+0000","last_became_peered":"2023-12-20T06:02:39.477334+0000","last_unstale":"2023-12-20T06:02:40.454484+0000","last_undegraded":"2023-12-20T06:02:40.454484+0000","last_fullsized":"2023-12-20T06:02:40.454484+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:06:54.630645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464660+0000","last_change":"2023-12-20T06:02:26.962506+0000","last_active":"2023-12-20T06:02:39.464660+0000","last_peered":"2023-12-20T06:02:39.464660+0000","last_clean":"2023-12-20T06:02:39.464660+0000","last_became_active":"2023-12-20T06:02:26.962262+0000","last_became_peered":"2023-12-20T06:02:26.962262+0000","last_unstale":"2023-12-20T06:02:39.464660+0000","last_undegraded":"2023-12-20T06:02:39.464660+0000","last_fullsized":"2023-12-20T06:02:39.464660+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:09:09.108347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458853+0000","last_change":"2023-12-20T06:02:39.480339+0000","last_active":"2023-12-20T06:02:40.458853+0000","last_peered":"2023-12-20T06:02:40.458853+0000","last_clean":"2023-12-20T06:02:40.458853+0000","last_became_active":"2023-12-20T06:02:39.480039+0000","last_became_peered":"2023-12-20T06:02:39.480039+0000","last_unstale":"2023-12-20T06:02:40.458853+0000","last_undegraded":"2023-12-20T06:02:40.458853+0000","last_fullsized":"2023-12-20T06:02:40.458853+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:52:40.939793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452796+0000","last_change":"2023-12-20T06:02:26.955353+0000","last_active":"2023-12-20T06:02:39.452796+0000","last_peered":"2023-12-20T06:02:39.452796+0000","last_clean":"2023-12-20T06:02:39.452796+0000","last_became_active":"2023-12-20T06:02:26.954910+0000","last_became_peered":"2023-12-20T06:02:26.954910+0000","last_unstale":"2023-12-20T06:02:39.452796+0000","last_undegraded":"2023-12-20T06:02:39.452796+0000","last_fullsized":"2023-12-20T06:02:39.452796+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:44:12.085696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.455467+0000","last_change":"2023-12-20T06:02:39.474033+0000","last_active":"2023-12-20T06:02:40.455467+0000","last_peered":"2023-12-20T06:02:40.455467+0000","last_clean":"2023-12-20T06:02:40.455467+0000","last_became_active":"2023-12-20T06:02:39.473755+0000","last_became_peered":"2023-12-20T06:02:39.473755+0000","last_unstale":"2023-12-20T06:02:40.455467+0000","last_undegraded":"2023-12-20T06:02:40.455467+0000","last_fullsized":"2023-12-20T06:02:40.455467+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:50.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467315+0000","last_change":"2023-12-20T06:02:26.968444+0000","last_active":"2023-12-20T06:02:39.467315+0000","last_peered":"2023-12-20T06:02:39.467315+0000","last_clean":"2023-12-20T06:02:39.467315+0000","last_became_active":"2023-12-20T06:02:26.967931+0000","last_became_peered":"2023-12-20T06:02:26.967931+0000","last_unstale":"2023-12-20T06:02:39.467315+0000","last_undegraded":"2023-12-20T06:02:39.467315+0000","last_fullsized":"2023-12-20T06:02:39.467315+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:33:14.150434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246851+0000","last_change":"2023-12-20T06:02:39.480502+0000","last_active":"2023-12-20T06:02:41.246851+0000","last_peered":"2023-12-20T06:02:41.246851+0000","last_clean":"2023-12-20T06:02:41.246851+0000","last_became_active":"2023-12-20T06:02:39.480217+0000","last_became_peered":"2023-12-20T06:02:39.480217+0000","last_unstale":"2023-12-20T06:02:41.246851+0000","last_undegraded":"2023-12-20T06:02:41.246851+0000","last_fullsized":"2023-12-20T06:02:41.246851+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:35:16.506386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453354+0000","last_change":"2023-12-20T06:02:26.956043+0000","last_active":"2023-12-20T06:02:39.453354+0000","last_peered":"2023-12-20T06:02:39.453354+0000","last_clean":"2023-12-20T06:02:39.453354+0000","last_became_active":"2023-12-20T06:02:26.955652+0000","last_became_peered":"2023-12-20T06:02:26.955652+0000","last_unstale":"2023-12-20T06:02:39.453354+0000","last_undegraded":"2023-12-20T06:02:39.453354+0000","last_fullsized":"2023-12-20T06:02:39.453354+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:02:53.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342688+0000","last_change":"2023-12-20T06:02:39.484997+0000","last_active":"2023-12-20T06:02:41.342688+0000","last_peered":"2023-12-20T06:02:41.342688+0000","last_clean":"2023-12-20T06:02:41.342688+0000","last_became_active":"2023-12-20T06:02:39.484052+0000","last_became_peered":"2023-12-20T06:02:39.484052+0000","last_unstale":"2023-12-20T06:02:41.342688+0000","last_undegraded":"2023-12-20T06:02:41.342688+0000","last_fullsized":"2023-12-20T06:02:41.342688+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:00:04.448601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.468006+0000","last_change":"2023-12-20T06:02:26.982741+0000","last_active":"2023-12-20T06:02:39.468006+0000","last_peered":"2023-12-20T06:02:39.468006+0000","last_clean":"2023-12-20T06:02:39.468006+0000","last_became_active":"2023-12-20T06:02:26.982454+0000","last_became_peered":"2023-12-20T06:02:26.982454+0000","last_unstale":"2023-12-20T06:02:39.468006+0000","last_undegraded":"2023-12-20T06:02:39.468006+0000","last_fullsized":"2023-12-20T06:02:39.468006+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:59:08.944850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":521,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3201,"ondisk_log_size":3201,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575973,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56396,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315444,"statfs":{"total":96636764160,"available":96579014656,"internally_reserved":0,"allocated":3280896,"data_stored":1890316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.73499999999999999},"min":{"1min":0.497,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.042},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72599999999999998,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.34499999999999997},"max":{"1min":0.874,"5min":0.99099999999999999,"15min":1.161},"last":0.504}]},{"osd":1,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73199999999999998,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.432},"max":{"1min":0.88700000000000001,"5min":0.996,"15min":1.1279999999999999},"last":0.433},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.53100000000000003,"5min":0.51100000000000001,"15min":0.46999999999999997},"max":{"1min":0.90900000000000003,"5min":0.96699999999999997,"15min":1.032},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72299999999999998,"15min":0.748},"min":{"1min":0.51900000000000002,"5min":0.48699999999999999,"15min":0.48299999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0409999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96999999999999997,"5min":1.0189999999999999,"15min":1.038},"last":0.55600000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72899999999999998,"15min":0.746},"min":{"1min":0.61899999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77900000000000003,"15min":0.79000000000000004},"min":{"1min":0.53800000000000003,"5min":0.503,"15min":0.40699999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0840000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77600000000000002,"5min":0.91800000000000004,"15min":1.04},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.91800000000000004,"5min":1.02,"15min":1.02},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86699999999999999,"5min":0.97499999999999998,"15min":1.1120000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.54300000000000004,"5min":0.47199999999999998,"15min":0.45600000000000002},"max":{"1min":0.84199999999999997,"5min":1.0229999999999999,"15min":1.103},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76500000000000001,"15min":0.751},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.45500000000000002},"max":{"1min":0.94399999999999995,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.625},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92300000000000004,"5min":1.0740000000000001,"15min":1.1539999999999999},"last":0.76000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575973,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51704,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320136,"statfs":{"total":96636764160,"available":96583819264,"internally_reserved":0,"allocated":2670592,"data_stored":1298862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71199999999999997,"15min":0.73999999999999999},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92400000000000004,"5min":1.016,"15min":1.0980000000000001},"last":0.312},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72999999999999998,"15min":0.75},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.006,"5min":1.016,"15min":1.05},"last":0.44}]},{"osd":1,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.47399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90100000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.624},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91200000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.371}]},{"osd":2,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76900000000000002,"15min":0.78100000000000003},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97699999999999998,"5min":1.069,"15min":1.069},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.54400000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.93100000000000005,"5min":0.97799999999999998,"15min":1.0760000000000001},"last":0.59699999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78900000000000003},"min":{"1min":0.61299999999999999,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.0109999999999999,"5min":1.077,"15min":1.077},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.997,"5min":1.052,"15min":1.052},"last":0.499}]},{"osd":4,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.36099999999999999},"max":{"1min":0.96299999999999997,"5min":1.05,"15min":1.0529999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66400000000000003,"15min":0.65100000000000002},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64900000000000002,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.46800000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.55700000000000005,"5min":0.48899999999999999,"15min":0.45300000000000001},"max":{"1min":1.0429999999999999,"5min":1.103,"15min":1.1240000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.67800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575974,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47608,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324232,"statfs":{"total":96636764160,"available":96588013568,"internally_reserved":0,"allocated":2670592,"data_stored":1298862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.44,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94499999999999995,"5min":0.97499999999999998,"15min":1.077},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73199999999999998,"15min":0.755},"min":{"1min":0.57699999999999996,"5min":0.432,"15min":0.432},"max":{"1min":0.91400000000000003,"5min":0.96699999999999997,"15min":1.028},"last":0.60699999999999998}]},{"osd":1,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.74299999999999999,"15min":0.754},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.442},"max":{"1min":0.89000000000000001,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.46899999999999997},"max":{"1min":0.95699999999999996,"5min":1.016,"15min":1.0640000000000001},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73199999999999998,"15min":0.75900000000000001},"min":{"1min":0.60299999999999998,"5min":0.46999999999999997,"15min":0.46899999999999997},"max":{"1min":0.93400000000000005,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76500000000000001,"15min":0.77200000000000002},"min":{"1min":0.63100000000000001,"5min":0.55800000000000005,"15min":0.48299999999999998},"max":{"1min":0.93500000000000005,"5min":1.0349999999999999,"15min":1.0680000000000001},"last":1.0669999999999999}]},{"osd":3,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.0669999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.65700000000000003,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.91700000000000004,"5min":1.004,"15min":1.0629999999999999},"last":0.95499999999999996}]},{"osd":4,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47199999999999998,"5min":0.42199999999999999,"15min":0.38100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0640000000000001,"15min":1.077},"last":0.871},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67000000000000004,"15min":0.64200000000000002},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41299999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.70299999999999996},"min":{"1min":0.56299999999999994,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.95199999999999996,"5min":1.012,"15min":1.073},"last":1.097}]},{"osd":7,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.56799999999999995,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":1},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.63300000000000001,"5min":0.56299999999999994,"15min":0.46700000000000003},"max":{"1min":0.97599999999999998,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.98799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575975,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47640,"kb_used_data":2640,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324200,"statfs":{"total":96636764160,"available":96587980800,"internally_reserved":0,"allocated":2703360,"data_stored":1303335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71899999999999997,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.38800000000000001},"max":{"1min":0.84399999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.56899999999999995,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":0.95699999999999996,"5min":0.96499999999999997,"15min":1.0900000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.52200000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91400000000000003,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76200000000000001,"15min":0.77600000000000002},"min":{"1min":0.58999999999999997,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.78100000000000003},"min":{"1min":0.56999999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98599999999999999,"5min":1.0089999999999999,"15min":1.101},"last":0.69599999999999995}]},{"osd":3,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.59899999999999998,"5min":0.51800000000000002,"15min":0.49199999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.64500000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.95999999999999996,"5min":1.04,"15min":1.081},"last":0.51500000000000001}]},{"osd":5,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.39900000000000002},"max":{"1min":0.93899999999999995,"5min":1.05,"15min":1.05},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.86099999999999999,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94699999999999995,"5min":1.016,"15min":1.1679999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.41199999999999998},"max":{"1min":0.97999999999999998,"5min":1.107,"15min":1.111},"last":0.749}]},{"osd":7,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":1.01,"5min":1.0600000000000001,"15min":1.1160000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.52900000000000003,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.034,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.73199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575974,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47628,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324212,"statfs":{"total":96636764160,"available":96587993088,"internally_reserved":0,"allocated":2691072,"data_stored":1298656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.59299999999999997,"5min":0.748,"15min":2.8130000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.51100000000000001,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64600000000000002,"5min":0.80000000000000004,"15min":1.357},"last":0.35699999999999998}]},{"osd":1,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":1.4219999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.34200000000000003},"max":{"1min":0.72599999999999998,"5min":0.86099999999999999,"15min":2.944},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":1.0209999999999999,"5min":1.2090000000000001,"15min":1.669},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.99},"last":0.70299999999999996}]},{"osd":5,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.59599999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97199999999999998,"5min":1.355,"15min":1.901},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.82399999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97699999999999998,"5min":1.196,"15min":2.8639999999999999},"last":0.875}]},{"osd":6,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.81999999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.575},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.81799999999999995,"15min":0.85099999999999998},"min":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0149999999999999,"5min":1.2350000000000001,"15min":1.948},"last":0.626}]},{"osd":7,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80400000000000005,"15min":0.84099999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.004,"5min":1.3149999999999999,"15min":2.9049999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82399999999999995,"15min":0.85299999999999998},"min":{"1min":0.61899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.089,"5min":1.28,"15min":1.754},"last":0.76200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575973,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51712,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320128,"statfs":{"total":96636764160,"available":96583811072,"internally_reserved":0,"allocated":2678784,"data_stored":1298974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.26100000000000001},"max":{"1min":0.75800000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.376,"5min":0.34000000000000002,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.505},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.30499999999999999},"max":{"1min":0.71299999999999997,"5min":0.92600000000000005,"15min":1.004},"last":0.48699999999999999}]},{"osd":3,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.65800000000000003,"15min":0.64900000000000002},"min":{"1min":0.47499999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.089,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.46500000000000002,"15min":0.42499999999999999},"max":{"1min":1.069,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.54300000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79200000000000004,"15min":0.80300000000000005},"min":{"1min":0.58699999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.081,"5min":1.119,"15min":1.254},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.59199999999999997,"5min":0.505,"15min":0.48199999999999998},"max":{"1min":1.02,"5min":1.605,"15min":1.605},"last":0.90300000000000002}]},{"osd":5,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81200000000000006,"15min":0.80700000000000005},"min":{"1min":0.62,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1020000000000001,"5min":1.329,"15min":1.329},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.83899999999999997,"15min":0.83399999999999996},"min":{"1min":0.68500000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0309999999999999,"5min":1.0740000000000001,"15min":1.157},"last":0.91700000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.60299999999999998,"5min":0.55100000000000005,"15min":0.52300000000000002},"max":{"1min":1.157,"5min":1.919,"15min":1.919},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63300000000000001,"5min":0.56999999999999995,"15min":0.56200000000000006},"max":{"1min":1.0449999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.89400000000000002}]},{"osd":7,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.875,"15min":0.85999999999999999},"min":{"1min":0.67400000000000004,"5min":0.60699999999999998,"15min":0.47899999999999998},"max":{"1min":1.3360000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.89000000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.262,"5min":1.292,"15min":1.292},"last":0.93600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575975,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59912,"kb_used_data":2624,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311928,"statfs":{"total":96636764160,"available":96575414272,"internally_reserved":0,"allocated":2686976,"data_stored":1301141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47899999999999998,"15min":0.46200000000000002},"min":{"1min":0.26500000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59299999999999997,"5min":0.746,"15min":0.81000000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.29399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.71999999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.497},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":0.88800000000000001,"15min":0.93700000000000006},"last":0.52500000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.42999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.90100000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.93500000000000005},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.78500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92000000000000004,"5min":1.107,"15min":1.321},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.995,"5min":1.139,"15min":1.25},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78700000000000003,"15min":0.80200000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.97899999999999998,"5min":1.224,"15min":1.2849999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0740000000000001,"5min":1.1990000000000001,"15min":1.4330000000000001},"last":0.65800000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.82399999999999995},"min":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.2430000000000001,"15min":1.341},"last":0.621},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.56399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1379999999999999,"5min":1.177,"15min":1.3839999999999999},"last":0.59199999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0900000000000001,"5min":1.272,"15min":1.4470000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0509999999999999,"5min":1.1599999999999999,"15min":1.3839999999999999},"last":0.74099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575974,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56384,"kb_used_data":3192,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315456,"statfs":{"total":96636764160,"available":96579026944,"internally_reserved":0,"allocated":3268608,"data_stored":1888760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.28799999999999998},"max":{"1min":0.71999999999999997,"5min":0.82299999999999995,"15min":0.96799999999999997},"last":0.435},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.34899999999999998,"5min":0.33200000000000002,"15min":0.33000000000000002},"max":{"1min":0.76200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.45900000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.435,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.65000000000000002,"5min":0.77700000000000002,"15min":0.81299999999999994},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.38,"15min":0.38},"max":{"1min":0.81499999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65700000000000003,"5min":0.80500000000000005,"15min":1.0429999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.626},"min":{"1min":0.496,"5min":0.46100000000000002,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.499}]},{"osd":4,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97899999999999998,"5min":1.0469999999999999,"15min":1.147},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.51400000000000001},"max":{"1min":0.94299999999999995,"5min":1.0760000000000001,"15min":1.2869999999999999},"last":0.872}]},{"osd":5,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.008,"5min":1.1259999999999999,"15min":1.2150000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83299999999999996,"15min":0.84299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.351},"last":0.69399999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.98999999999999999,"5min":1.1040000000000001,"15min":1.2150000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.85599999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57099999999999995},"max":{"1min":0.97899999999999998,"5min":1.1379999999999999,"15min":1.3320000000000001},"last":0.76200000000000001}]},{"osd":7,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85099999999999998,"15min":0.86299999999999999},"min":{"1min":0.627,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97399999999999998,"5min":1.129,"15min":1.3560000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.85399999999999998,"15min":0.85899999999999999},"min":{"1min":0.61399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0009999999999999,"5min":1.139,"15min":1.2549999999999999},"last":0.92100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:02:43.677 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:02:43.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.860+0000 7f2ff76e5640 1 Processor -- start 2023-12-20T06:02:43.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.860+0000 7f2ff76e5640 1 -- start start 2023-12-20T06:02:43.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.860+0000 7f2ff76e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ff00ae010 0x7f2ff0071470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.860+0000 7f2ff76e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ff0164ef0 0x7f2ff00719b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.860+0000 7f2ff76e5640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ff00ae3f0 con 0x7f2ff006cf10 2023-12-20T06:02:43.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.860+0000 7f2ff76e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ff00ca4b0 con 0x7f2ff00ae010 2023-12-20T06:02:43.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.860+0000 7f2ff76e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff0161800 con 0x7f2ff0164ef0 2023-12-20T06:02:43.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2ff5481640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ff00ae010 0x7f2ff0071470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2ff5481640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ff00ae010 0x7f2ff0071470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46838/0 (socket says 172.21.15.32:46838) 2023-12-20T06:02:43.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2ff5481640 1 -- 172.21.15.32:0/3987660370 learned_addr learned my addr 172.21.15.32:0/3987660370 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:43.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2ff4c80640 1 --1- 172.21.15.32:0/3987660370 >> v1:172.21.15.32:6789/0 conn(0x7f2ff006cf10 0x7f2ff00701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60466/0 (socket says 172.21.15.32:60466) 2023-12-20T06:02:43.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2ff5481640 1 -- 172.21.15.32:0/3987660370 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ff0164ef0 msgr2=0x7f2ff00719b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:43.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2ff5481640 1 --2- 172.21.15.32:0/3987660370 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ff0164ef0 0x7f2ff00719b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2ff5481640 1 -- 172.21.15.32:0/3987660370 >> v1:172.21.15.32:6789/0 conn(0x7f2ff006cf10 legacy=0x7f2ff00701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2ff5481640 1 -- 172.21.15.32:0/3987660370 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ff0165380 con 0x7f2ff00ae010 2023-12-20T06:02:43.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2fe7fff640 1 -- 172.21.15.32:0/3987660370 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204100353 0 0) 0x7f2ff00ae3f0 con 0x7f2ff006cf10 2023-12-20T06:02:43.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2ff5481640 1 --2- 172.21.15.32:0/3987660370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff00ae010 0x7f2ff0071470 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f2fec006fd0 tx=0x7f2fec01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=777ae0116a2f7c5c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:43.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2fe7fff640 1 -- 172.21.15.32:0/3987660370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0ab660 con 0x7f2ff00ae010 2023-12-20T06:02:43.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2fe7fff640 1 -- 172.21.15.32:0/3987660370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fec0ab7c0 con 0x7f2ff00ae010 2023-12-20T06:02:43.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.864+0000 7f2fe7fff640 1 -- 172.21.15.32:0/3987660370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fec0abb50 con 0x7f2ff00ae010 2023-12-20T06:02:43.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 -- 172.21.15.32:0/3987660370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff00ae010 msgr2=0x7f2ff0071470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 --2- 172.21.15.32:0/3987660370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff00ae010 0x7f2ff0071470 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f2fec006fd0 tx=0x7f2fec01c460 comp rx=0 tx=0).stop 2023-12-20T06:02:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 -- 172.21.15.32:0/3987660370 shutdown_connections 2023-12-20T06:02:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 --2- 172.21.15.32:0/3987660370 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ff0164ef0 0x7f2ff00719b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 --2- 172.21.15.32:0/3987660370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff00ae010 0x7f2ff0071470 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 -- 172.21.15.32:0/3987660370 >> 172.21.15.32:0/3987660370 conn(0x7f2ff006a320 msgr2=0x7f2ff00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 -- 172.21.15.32:0/3987660370 shutdown_connections 2023-12-20T06:02:43.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 -- 172.21.15.32:0/3987660370 wait complete. 2023-12-20T06:02:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 Processor -- start 2023-12-20T06:02:43.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 -- start start 2023-12-20T06:02:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ff00ae010 0x7f2ff015fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff015cfc0 0x7f2ff015b570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff015d420 0x7f2ff015bab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00c21a0 con 0x7f2ff015d420 2023-12-20T06:02:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00c3b80 con 0x7f2ff015cfc0 2023-12-20T06:02:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff76e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ff00c3e60 con 0x7f2ff00ae010 2023-12-20T06:02:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff5c82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff015d420 0x7f2ff015bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff5c82640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff015d420 0x7f2ff015bab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46844/0 (socket says 172.21.15.32:46844) 2023-12-20T06:02:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff5c82640 1 -- 172.21.15.32:0/835324229 learned_addr learned my addr 172.21.15.32:0/835324229 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff5481640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ff00ae010 0x7f2ff015fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff4c80640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff015cfc0 0x7f2ff015b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff5c82640 1 -- 172.21.15.32:0/835324229 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ff00ae010 msgr2=0x7f2ff015fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff5c82640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ff00ae010 0x7f2ff015fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff5c82640 1 -- 172.21.15.32:0/835324229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff015cfc0 msgr2=0x7f2ff015b570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff5c82640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff015cfc0 0x7f2ff015b570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.868+0000 7f2ff5c82640 1 -- 172.21.15.32:0/835324229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2fec009480 con 0x7f2ff015d420 2023-12-20T06:02:43.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.872+0000 7f2ff5c82640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff015d420 0x7f2ff015bab0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800b9e0 tx=0x7f2fe800bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.872+0000 7f2fe5ffb640 1 -- 172.21.15.32:0/835324229 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe800cb20 con 0x7f2ff015d420 2023-12-20T06:02:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.872+0000 7f2fe5ffb640 1 -- 172.21.15.32:0/835324229 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2fe800cca0 con 0x7f2ff015d420 2023-12-20T06:02:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.872+0000 7f2ff76e5640 1 -- 172.21.15.32:0/835324229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ff00c4140 con 0x7f2ff015d420 2023-12-20T06:02:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.872+0000 7f2fe5ffb640 1 -- 172.21.15.32:0/835324229 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2fe8014680 con 0x7f2ff015d420 2023-12-20T06:02:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.872+0000 7f2ff76e5640 1 -- 172.21.15.32:0/835324229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ff00c37c0 con 0x7f2ff015d420 2023-12-20T06:02:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.872+0000 7f2ff76e5640 1 -- 172.21.15.32:0/835324229 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ff00c4940 con 0x7f2ff015d420 2023-12-20T06:02:43.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.872+0000 7f2fe5ffb640 1 -- 172.21.15.32:0/835324229 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f2fe801d3b0 con 0x7f2ff015d420 2023-12-20T06:02:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.872+0000 7f2fe5ffb640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2fd4065bb0 0x7f2fd4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.872+0000 7f2fe5ffb640 1 -- 172.21.15.32:0/835324229 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f2fe8083570 con 0x7f2ff015d420 2023-12-20T06:02:43.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.872+0000 7f2ff5481640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2fd4065bb0 0x7f2fd4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:43.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.876+0000 7f2ff5481640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2fd4065bb0 0x7f2fd4068070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0af000 tx=0x7f2fec0a7000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:43.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:43.876+0000 7f2fe5ffb640 1 -- 172.21.15.32:0/835324229 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2ff00c4940 con 0x7f2ff015d420 2023-12-20T06:02:44.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.040+0000 7f2ff76e5640 1 -- 172.21.15.32:0/835324229 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ff0165c20 con 0x7f2fd4065bb0 2023-12-20T06:02:44.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.052+0000 7f2fe5ffb640 1 -- 172.21.15.32:0/835324229 <== mgr.10314 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528568 (secure 0 0 0) 0x7f2fec004040 con 0x7f2fd4065bb0 2023-12-20T06:02:44.056 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:44.060 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.056+0000 7f2ff76e5640 1 -- 172.21.15.32:0/835324229 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2fd4065bb0 msgr2=0x7f2fd4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.056+0000 7f2ff76e5640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2fd4065bb0 0x7f2fd4068070 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f2fec0af000 tx=0x7f2fec0a7000 comp rx=0 tx=0).stop 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.056+0000 7f2ff76e5640 1 -- 172.21.15.32:0/835324229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff015d420 msgr2=0x7f2ff015bab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.056+0000 7f2ff76e5640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff015d420 0x7f2ff015bab0 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f2fe800b9e0 tx=0x7f2fe800bea0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.056+0000 7f2ff76e5640 1 -- 172.21.15.32:0/835324229 shutdown_connections 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.056+0000 7f2ff76e5640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2fd4065bb0 0x7f2fd4068070 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.056+0000 7f2ff76e5640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ff015d420 0x7f2ff015bab0 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.056+0000 7f2ff76e5640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ff015cfc0 0x7f2ff015b570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.056+0000 7f2ff76e5640 1 --2- 172.21.15.32:0/835324229 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ff00ae010 0x7f2ff015fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.056+0000 7f2ff76e5640 1 -- 172.21.15.32:0/835324229 >> 172.21.15.32:0/835324229 conn(0x7f2ff006a320 msgr2=0x7f2ff00cce70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.060+0000 7f2ff76e5640 1 -- 172.21.15.32:0/835324229 shutdown_connections 2023-12-20T06:02:44.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.060+0000 7f2ff76e5640 1 -- 172.21.15.32:0/835324229 wait complete. 2023-12-20T06:02:44.075 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":38,"stamp":"2023-12-20T06:02:42.957498+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":46,"num_object_clones":0,"num_object_copies":92,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":46,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":613,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":28,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3281,"ondisk_log_size":3281,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":386,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":418984,"kb_used_data":22120,"kb_used_omap":0,"kb_used_meta":396800,"kb_avail":754555736,"statfs":{"total":773094113280,"available":772665073664,"internally_reserved":0,"allocated":22650880,"data_stored":11578906,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":406323200},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":42,"apply_latency_ms":42,"commit_latency_ns":42000000,"apply_latency_ns":42000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.932362"},"pg_stats":[{"pgid":"27.1a","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454462+0000","last_change":"2023-12-20T06:02:39.471108+0000","last_active":"2023-12-20T06:02:40.454462+0000","last_peered":"2023-12-20T06:02:40.454462+0000","last_clean":"2023-12-20T06:02:40.454462+0000","last_became_active":"2023-12-20T06:02:39.470838+0000","last_became_peered":"2023-12-20T06:02:39.470838+0000","last_unstale":"2023-12-20T06:02:40.454462+0000","last_undegraded":"2023-12-20T06:02:40.454462+0000","last_fullsized":"2023-12-20T06:02:40.454462+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:42:23.074715+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"196'3201","reported_seq":3996,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465654+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:02:39.465654+0000","last_peered":"2023-12-20T06:02:39.465654+0000","last_clean":"2023-12-20T06:02:39.465654+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:02:39.465654+0000","last_undegraded":"2023-12-20T06:02:39.465654+0000","last_fullsized":"2023-12-20T06:02:39.465654+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3201,"log_dups_size":0,"ondisk_log_size":3201,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":521,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.19","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465147+0000","last_change":"2023-12-20T06:02:26.550026+0000","last_active":"2023-12-20T06:02:39.465147+0000","last_peered":"2023-12-20T06:02:39.465147+0000","last_clean":"2023-12-20T06:02:39.465147+0000","last_became_active":"2023-12-20T06:02:26.549712+0000","last_became_peered":"2023-12-20T06:02:26.549712+0000","last_unstale":"2023-12-20T06:02:39.465147+0000","last_undegraded":"2023-12-20T06:02:39.465147+0000","last_fullsized":"2023-12-20T06:02:39.465147+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:08.929656+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.18","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452323+0000","last_change":"2023-12-20T06:02:26.971114+0000","last_active":"2023-12-20T06:02:39.452323+0000","last_peered":"2023-12-20T06:02:39.452323+0000","last_clean":"2023-12-20T06:02:39.452323+0000","last_became_active":"2023-12-20T06:02:26.970836+0000","last_became_peered":"2023-12-20T06:02:26.970836+0000","last_unstale":"2023-12-20T06:02:39.452323+0000","last_undegraded":"2023-12-20T06:02:39.452323+0000","last_fullsized":"2023-12-20T06:02:39.452323+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:50:58.624420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1b","version":"212'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457896+0000","last_change":"2023-12-20T06:02:39.062821+0000","last_active":"2023-12-20T06:02:40.457896+0000","last_peered":"2023-12-20T06:02:40.457896+0000","last_clean":"2023-12-20T06:02:40.457896+0000","last_became_active":"2023-12-20T06:02:39.062495+0000","last_became_peered":"2023-12-20T06:02:39.062495+0000","last_unstale":"2023-12-20T06:02:40.457896+0000","last_undegraded":"2023-12-20T06:02:40.457896+0000","last_fullsized":"2023-12-20T06:02:40.457896+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:35:15.487675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457654+0000","last_change":"2023-12-20T06:02:39.462238+0000","last_active":"2023-12-20T06:02:40.457654+0000","last_peered":"2023-12-20T06:02:40.457654+0000","last_clean":"2023-12-20T06:02:40.457654+0000","last_became_active":"2023-12-20T06:02:39.461928+0000","last_became_peered":"2023-12-20T06:02:39.461928+0000","last_unstale":"2023-12-20T06:02:40.457654+0000","last_undegraded":"2023-12-20T06:02:40.457654+0000","last_fullsized":"2023-12-20T06:02:40.457654+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:58.188244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.0","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.462583+0000","last_change":"2023-12-20T06:02:26.560054+0000","last_active":"2023-12-20T06:02:39.462583+0000","last_peered":"2023-12-20T06:02:39.462583+0000","last_clean":"2023-12-20T06:02:39.462583+0000","last_became_active":"2023-12-20T06:02:26.559777+0000","last_became_peered":"2023-12-20T06:02:26.559777+0000","last_unstale":"2023-12-20T06:02:39.462583+0000","last_undegraded":"2023-12-20T06:02:39.462583+0000","last_fullsized":"2023-12-20T06:02:39.462583+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:40:31.615384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463934+0000","last_change":"2023-12-20T06:02:26.961645+0000","last_active":"2023-12-20T06:02:39.463934+0000","last_peered":"2023-12-20T06:02:39.463934+0000","last_clean":"2023-12-20T06:02:39.463934+0000","last_became_active":"2023-12-20T06:02:26.961386+0000","last_became_peered":"2023-12-20T06:02:26.961386+0000","last_unstale":"2023-12-20T06:02:39.463934+0000","last_undegraded":"2023-12-20T06:02:39.463934+0000","last_fullsized":"2023-12-20T06:02:39.463934+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:52:17.428938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.2","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343408+0000","last_change":"2023-12-20T06:02:39.066345+0000","last_active":"2023-12-20T06:02:41.343408+0000","last_peered":"2023-12-20T06:02:41.343408+0000","last_clean":"2023-12-20T06:02:41.343408+0000","last_became_active":"2023-12-20T06:02:39.065968+0000","last_became_peered":"2023-12-20T06:02:39.065968+0000","last_unstale":"2023-12-20T06:02:41.343408+0000","last_undegraded":"2023-12-20T06:02:41.343408+0000","last_fullsized":"2023-12-20T06:02:41.343408+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:03:22.846131+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.2","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454664+0000","last_change":"2023-12-20T06:02:39.482489+0000","last_active":"2023-12-20T06:02:40.454664+0000","last_peered":"2023-12-20T06:02:40.454664+0000","last_clean":"2023-12-20T06:02:40.454664+0000","last_became_active":"2023-12-20T06:02:39.482228+0000","last_became_peered":"2023-12-20T06:02:39.482228+0000","last_unstale":"2023-12-20T06:02:40.454664+0000","last_undegraded":"2023-12-20T06:02:40.454664+0000","last_fullsized":"2023-12-20T06:02:40.454664+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:55:08.298301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452293+0000","last_change":"2023-12-20T06:02:26.548501+0000","last_active":"2023-12-20T06:02:39.452293+0000","last_peered":"2023-12-20T06:02:39.452293+0000","last_clean":"2023-12-20T06:02:39.452293+0000","last_became_active":"2023-12-20T06:02:26.548203+0000","last_became_peered":"2023-12-20T06:02:26.548203+0000","last_unstale":"2023-12-20T06:02:39.452293+0000","last_undegraded":"2023-12-20T06:02:39.452293+0000","last_fullsized":"2023-12-20T06:02:39.452293+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:55.646229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.0","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452572+0000","last_change":"2023-12-20T06:02:26.962551+0000","last_active":"2023-12-20T06:02:39.452572+0000","last_peered":"2023-12-20T06:02:39.452572+0000","last_clean":"2023-12-20T06:02:39.452572+0000","last_became_active":"2023-12-20T06:02:26.962269+0000","last_became_peered":"2023-12-20T06:02:26.962269+0000","last_unstale":"2023-12-20T06:02:39.452572+0000","last_undegraded":"2023-12-20T06:02:39.452572+0000","last_fullsized":"2023-12-20T06:02:39.452572+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:43:42.660130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.3","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246731+0000","last_change":"2023-12-20T06:02:39.068124+0000","last_active":"2023-12-20T06:02:41.246731+0000","last_peered":"2023-12-20T06:02:41.246731+0000","last_clean":"2023-12-20T06:02:41.246731+0000","last_became_active":"2023-12-20T06:02:39.067842+0000","last_became_peered":"2023-12-20T06:02:39.067842+0000","last_unstale":"2023-12-20T06:02:41.246731+0000","last_undegraded":"2023-12-20T06:02:41.246731+0000","last_fullsized":"2023-12-20T06:02:41.246731+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:03:21.014214+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457298+0000","last_change":"2023-12-20T06:02:39.476607+0000","last_active":"2023-12-20T06:02:40.457298+0000","last_peered":"2023-12-20T06:02:40.457298+0000","last_clean":"2023-12-20T06:02:40.457298+0000","last_became_active":"2023-12-20T06:02:39.476262+0000","last_became_peered":"2023-12-20T06:02:39.476262+0000","last_unstale":"2023-12-20T06:02:40.457298+0000","last_undegraded":"2023-12-20T06:02:40.457298+0000","last_fullsized":"2023-12-20T06:02:40.457298+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:38:05.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.2","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464853+0000","last_change":"2023-12-20T06:02:26.556064+0000","last_active":"2023-12-20T06:02:39.464853+0000","last_peered":"2023-12-20T06:02:39.464853+0000","last_clean":"2023-12-20T06:02:39.464853+0000","last_became_active":"2023-12-20T06:02:26.554935+0000","last_became_peered":"2023-12-20T06:02:26.554935+0000","last_unstale":"2023-12-20T06:02:39.464853+0000","last_undegraded":"2023-12-20T06:02:39.464853+0000","last_fullsized":"2023-12-20T06:02:39.464853+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:12.697664+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467764+0000","last_change":"2023-12-20T06:02:26.980532+0000","last_active":"2023-12-20T06:02:39.467764+0000","last_peered":"2023-12-20T06:02:39.467764+0000","last_clean":"2023-12-20T06:02:39.467764+0000","last_became_active":"2023-12-20T06:02:26.980234+0000","last_became_peered":"2023-12-20T06:02:26.980234+0000","last_unstale":"2023-12-20T06:02:39.467764+0000","last_undegraded":"2023-12-20T06:02:39.467764+0000","last_fullsized":"2023-12-20T06:02:39.467764+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:32.040401+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.0","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.456725+0000","last_change":"2023-12-20T06:02:39.063288+0000","last_active":"2023-12-20T06:02:40.456725+0000","last_peered":"2023-12-20T06:02:40.456725+0000","last_clean":"2023-12-20T06:02:40.456725+0000","last_became_active":"2023-12-20T06:02:39.063009+0000","last_became_peered":"2023-12-20T06:02:39.063009+0000","last_unstale":"2023-12-20T06:02:40.456725+0000","last_undegraded":"2023-12-20T06:02:40.456725+0000","last_fullsized":"2023-12-20T06:02:40.456725+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:23.116478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.0","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342060+0000","last_change":"2023-12-20T06:02:39.481671+0000","last_active":"2023-12-20T06:02:41.342060+0000","last_peered":"2023-12-20T06:02:41.342060+0000","last_clean":"2023-12-20T06:02:41.342060+0000","last_became_active":"2023-12-20T06:02:39.480777+0000","last_became_peered":"2023-12-20T06:02:39.480777+0000","last_unstale":"2023-12-20T06:02:41.342060+0000","last_undegraded":"2023-12-20T06:02:41.342060+0000","last_fullsized":"2023-12-20T06:02:41.342060+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:05:06.811455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.3","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465878+0000","last_change":"2023-12-20T06:02:26.556385+0000","last_active":"2023-12-20T06:02:39.465878+0000","last_peered":"2023-12-20T06:02:39.465878+0000","last_clean":"2023-12-20T06:02:39.465878+0000","last_became_active":"2023-12-20T06:02:26.555308+0000","last_became_peered":"2023-12-20T06:02:26.555308+0000","last_unstale":"2023-12-20T06:02:39.465878+0000","last_undegraded":"2023-12-20T06:02:39.465878+0000","last_fullsized":"2023-12-20T06:02:39.465878+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:50:31.032511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.2","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465489+0000","last_change":"2023-12-20T06:02:26.983027+0000","last_active":"2023-12-20T06:02:39.465489+0000","last_peered":"2023-12-20T06:02:39.465489+0000","last_clean":"2023-12-20T06:02:39.465489+0000","last_became_active":"2023-12-20T06:02:26.982556+0000","last_became_peered":"2023-12-20T06:02:26.982556+0000","last_unstale":"2023-12-20T06:02:39.465489+0000","last_undegraded":"2023-12-20T06:02:39.465489+0000","last_fullsized":"2023-12-20T06:02:39.465489+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:42:24.293821+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"26.1","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338410+0000","last_change":"2023-12-20T06:02:39.064927+0000","last_active":"2023-12-20T06:02:41.338410+0000","last_peered":"2023-12-20T06:02:41.338410+0000","last_clean":"2023-12-20T06:02:41.338410+0000","last_became_active":"2023-12-20T06:02:39.064648+0000","last_became_peered":"2023-12-20T06:02:39.064648+0000","last_unstale":"2023-12-20T06:02:41.338410+0000","last_undegraded":"2023-12-20T06:02:41.338410+0000","last_fullsized":"2023-12-20T06:02:41.338410+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:37:41.166384+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.7","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.455349+0000","last_change":"2023-12-20T06:02:39.463206+0000","last_active":"2023-12-20T06:02:40.455349+0000","last_peered":"2023-12-20T06:02:40.455349+0000","last_clean":"2023-12-20T06:02:40.455349+0000","last_became_active":"2023-12-20T06:02:39.462905+0000","last_became_peered":"2023-12-20T06:02:39.462905+0000","last_unstale":"2023-12-20T06:02:40.455349+0000","last_undegraded":"2023-12-20T06:02:40.455349+0000","last_fullsized":"2023-12-20T06:02:40.455349+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:29:42.276728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.4","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465991+0000","last_change":"2023-12-20T06:02:26.560397+0000","last_active":"2023-12-20T06:02:39.465991+0000","last_peered":"2023-12-20T06:02:39.465991+0000","last_clean":"2023-12-20T06:02:39.465991+0000","last_became_active":"2023-12-20T06:02:26.560128+0000","last_became_peered":"2023-12-20T06:02:26.560128+0000","last_unstale":"2023-12-20T06:02:39.465991+0000","last_undegraded":"2023-12-20T06:02:39.465991+0000","last_fullsized":"2023-12-20T06:02:39.465991+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:54:06.878956+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.5","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467204+0000","last_change":"2023-12-20T06:02:26.967116+0000","last_active":"2023-12-20T06:02:39.467204+0000","last_peered":"2023-12-20T06:02:39.467204+0000","last_clean":"2023-12-20T06:02:39.467204+0000","last_became_active":"2023-12-20T06:02:26.966832+0000","last_became_peered":"2023-12-20T06:02:26.966832+0000","last_unstale":"2023-12-20T06:02:39.467204+0000","last_undegraded":"2023-12-20T06:02:39.467204+0000","last_fullsized":"2023-12-20T06:02:39.467204+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:52:23.105026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.6","version":"213'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.249069+0000","last_change":"2023-12-20T06:02:39.065762+0000","last_active":"2023-12-20T06:02:41.249069+0000","last_peered":"2023-12-20T06:02:41.249069+0000","last_clean":"2023-12-20T06:02:41.249069+0000","last_became_active":"2023-12-20T06:02:39.065464+0000","last_became_peered":"2023-12-20T06:02:39.065464+0000","last_unstale":"2023-12-20T06:02:41.249069+0000","last_undegraded":"2023-12-20T06:02:41.249069+0000","last_fullsized":"2023-12-20T06:02:41.249069+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:45:09.323660+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.6","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458475+0000","last_change":"2023-12-20T06:02:39.471117+0000","last_active":"2023-12-20T06:02:40.458475+0000","last_peered":"2023-12-20T06:02:40.458475+0000","last_clean":"2023-12-20T06:02:40.458475+0000","last_became_active":"2023-12-20T06:02:39.468897+0000","last_became_peered":"2023-12-20T06:02:39.468897+0000","last_unstale":"2023-12-20T06:02:40.458475+0000","last_undegraded":"2023-12-20T06:02:40.458475+0000","last_fullsized":"2023-12-20T06:02:40.458475+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:40.405390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.5","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.455635+0000","last_change":"2023-12-20T06:02:26.556970+0000","last_active":"2023-12-20T06:02:39.455635+0000","last_peered":"2023-12-20T06:02:39.455635+0000","last_clean":"2023-12-20T06:02:39.455635+0000","last_became_active":"2023-12-20T06:02:26.556577+0000","last_became_peered":"2023-12-20T06:02:26.556577+0000","last_unstale":"2023-12-20T06:02:39.455635+0000","last_undegraded":"2023-12-20T06:02:39.455635+0000","last_fullsized":"2023-12-20T06:02:39.455635+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:45.673434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.4","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465910+0000","last_change":"2023-12-20T06:02:26.969245+0000","last_active":"2023-12-20T06:02:39.465910+0000","last_peered":"2023-12-20T06:02:39.465910+0000","last_clean":"2023-12-20T06:02:39.465910+0000","last_became_active":"2023-12-20T06:02:26.969011+0000","last_became_peered":"2023-12-20T06:02:26.969011+0000","last_unstale":"2023-12-20T06:02:39.465910+0000","last_undegraded":"2023-12-20T06:02:39.465910+0000","last_fullsized":"2023-12-20T06:02:39.465910+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:53.274408+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.7","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342301+0000","last_change":"2023-12-20T06:02:39.069539+0000","last_active":"2023-12-20T06:02:41.342301+0000","last_peered":"2023-12-20T06:02:41.342301+0000","last_clean":"2023-12-20T06:02:41.342301+0000","last_became_active":"2023-12-20T06:02:39.069249+0000","last_became_peered":"2023-12-20T06:02:39.069249+0000","last_unstale":"2023-12-20T06:02:41.342301+0000","last_undegraded":"2023-12-20T06:02:41.342301+0000","last_fullsized":"2023-12-20T06:02:41.342301+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:44:23.563050+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.5","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338767+0000","last_change":"2023-12-20T06:02:39.472568+0000","last_active":"2023-12-20T06:02:41.338767+0000","last_peered":"2023-12-20T06:02:41.338767+0000","last_clean":"2023-12-20T06:02:41.338767+0000","last_became_active":"2023-12-20T06:02:39.472269+0000","last_became_peered":"2023-12-20T06:02:39.472269+0000","last_unstale":"2023-12-20T06:02:41.338767+0000","last_undegraded":"2023-12-20T06:02:41.338767+0000","last_fullsized":"2023-12-20T06:02:41.338767+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:48.990217+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.6","version":"209'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.454995+0000","last_change":"2023-12-20T06:02:26.555848+0000","last_active":"2023-12-20T06:02:39.454995+0000","last_peered":"2023-12-20T06:02:39.454995+0000","last_clean":"2023-12-20T06:02:39.454995+0000","last_became_active":"2023-12-20T06:02:26.555560+0000","last_became_peered":"2023-12-20T06:02:26.555560+0000","last_unstale":"2023-12-20T06:02:39.454995+0000","last_undegraded":"2023-12-20T06:02:39.454995+0000","last_fullsized":"2023-12-20T06:02:39.454995+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:44.684288+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.7","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.456510+0000","last_change":"2023-12-20T06:02:26.981756+0000","last_active":"2023-12-20T06:02:39.456510+0000","last_peered":"2023-12-20T06:02:39.456510+0000","last_clean":"2023-12-20T06:02:39.456510+0000","last_became_active":"2023-12-20T06:02:26.980456+0000","last_became_peered":"2023-12-20T06:02:26.980456+0000","last_unstale":"2023-12-20T06:02:39.456510+0000","last_undegraded":"2023-12-20T06:02:39.456510+0000","last_fullsized":"2023-12-20T06:02:39.456510+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:33:51.384201+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.4","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342755+0000","last_change":"2023-12-20T06:02:39.065294+0000","last_active":"2023-12-20T06:02:41.342755+0000","last_peered":"2023-12-20T06:02:41.342755+0000","last_clean":"2023-12-20T06:02:41.342755+0000","last_became_active":"2023-12-20T06:02:39.065013+0000","last_became_peered":"2023-12-20T06:02:39.065013+0000","last_unstale":"2023-12-20T06:02:41.342755+0000","last_undegraded":"2023-12-20T06:02:41.342755+0000","last_fullsized":"2023-12-20T06:02:41.342755+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:22.604577+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.4","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458529+0000","last_change":"2023-12-20T06:02:39.476789+0000","last_active":"2023-12-20T06:02:40.458529+0000","last_peered":"2023-12-20T06:02:40.458529+0000","last_clean":"2023-12-20T06:02:40.458529+0000","last_became_active":"2023-12-20T06:02:39.476418+0000","last_became_peered":"2023-12-20T06:02:39.476418+0000","last_unstale":"2023-12-20T06:02:40.458529+0000","last_undegraded":"2023-12-20T06:02:40.458529+0000","last_fullsized":"2023-12-20T06:02:40.458529+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:41:50.225640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.7","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.468112+0000","last_change":"2023-12-20T06:02:26.549840+0000","last_active":"2023-12-20T06:02:39.468112+0000","last_peered":"2023-12-20T06:02:39.468112+0000","last_clean":"2023-12-20T06:02:39.468112+0000","last_became_active":"2023-12-20T06:02:26.549537+0000","last_became_peered":"2023-12-20T06:02:26.549537+0000","last_unstale":"2023-12-20T06:02:39.468112+0000","last_undegraded":"2023-12-20T06:02:39.468112+0000","last_fullsized":"2023-12-20T06:02:39.468112+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:50.955103+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.6","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464485+0000","last_change":"2023-12-20T06:02:26.966239+0000","last_active":"2023-12-20T06:02:39.464485+0000","last_peered":"2023-12-20T06:02:39.464485+0000","last_clean":"2023-12-20T06:02:39.464485+0000","last_became_active":"2023-12-20T06:02:26.964925+0000","last_became_peered":"2023-12-20T06:02:26.964925+0000","last_unstale":"2023-12-20T06:02:39.464485+0000","last_undegraded":"2023-12-20T06:02:39.464485+0000","last_fullsized":"2023-12-20T06:02:39.464485+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:09.937547+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.5","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.344274+0000","last_change":"2023-12-20T06:02:39.057584+0000","last_active":"2023-12-20T06:02:41.344274+0000","last_peered":"2023-12-20T06:02:41.344274+0000","last_clean":"2023-12-20T06:02:41.344274+0000","last_became_active":"2023-12-20T06:02:39.057300+0000","last_became_peered":"2023-12-20T06:02:39.057300+0000","last_unstale":"2023-12-20T06:02:41.344274+0000","last_undegraded":"2023-12-20T06:02:41.344274+0000","last_fullsized":"2023-12-20T06:02:41.344274+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:35:15.487675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.b","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342858+0000","last_change":"2023-12-20T06:02:39.475953+0000","last_active":"2023-12-20T06:02:41.342858+0000","last_peered":"2023-12-20T06:02:41.342858+0000","last_clean":"2023-12-20T06:02:41.342858+0000","last_became_active":"2023-12-20T06:02:39.475668+0000","last_became_peered":"2023-12-20T06:02:39.475668+0000","last_unstale":"2023-12-20T06:02:41.342858+0000","last_undegraded":"2023-12-20T06:02:41.342858+0000","last_fullsized":"2023-12-20T06:02:41.342858+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:06:14.209435+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.8","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463909+0000","last_change":"2023-12-20T06:02:26.547934+0000","last_active":"2023-12-20T06:02:39.463909+0000","last_peered":"2023-12-20T06:02:39.463909+0000","last_clean":"2023-12-20T06:02:39.463909+0000","last_became_active":"2023-12-20T06:02:26.547651+0000","last_became_peered":"2023-12-20T06:02:26.547651+0000","last_unstale":"2023-12-20T06:02:39.463909+0000","last_undegraded":"2023-12-20T06:02:39.463909+0000","last_fullsized":"2023-12-20T06:02:39.463909+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:42:23.285931+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.9","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463859+0000","last_change":"2023-12-20T06:02:26.961186+0000","last_active":"2023-12-20T06:02:39.463859+0000","last_peered":"2023-12-20T06:02:39.463859+0000","last_clean":"2023-12-20T06:02:39.463859+0000","last_became_active":"2023-12-20T06:02:26.960904+0000","last_became_peered":"2023-12-20T06:02:26.960904+0000","last_unstale":"2023-12-20T06:02:39.463859+0000","last_undegraded":"2023-12-20T06:02:39.463859+0000","last_fullsized":"2023-12-20T06:02:39.463859+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:45.708153+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"26.a","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454752+0000","last_change":"2023-12-20T06:02:39.069300+0000","last_active":"2023-12-20T06:02:40.454752+0000","last_peered":"2023-12-20T06:02:40.454752+0000","last_clean":"2023-12-20T06:02:40.454752+0000","last_became_active":"2023-12-20T06:02:39.069037+0000","last_became_peered":"2023-12-20T06:02:39.069037+0000","last_unstale":"2023-12-20T06:02:40.454752+0000","last_undegraded":"2023-12-20T06:02:40.454752+0000","last_fullsized":"2023-12-20T06:02:40.454752+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:25:58.506206+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.a","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341821+0000","last_change":"2023-12-20T06:02:39.472628+0000","last_active":"2023-12-20T06:02:41.341821+0000","last_peered":"2023-12-20T06:02:41.341821+0000","last_clean":"2023-12-20T06:02:41.341821+0000","last_became_active":"2023-12-20T06:02:39.471675+0000","last_became_peered":"2023-12-20T06:02:39.471675+0000","last_unstale":"2023-12-20T06:02:41.341821+0000","last_undegraded":"2023-12-20T06:02:41.341821+0000","last_fullsized":"2023-12-20T06:02:41.341821+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:58:55.988186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.9","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.461020+0000","last_change":"2023-12-20T06:02:26.560641+0000","last_active":"2023-12-20T06:02:39.461020+0000","last_peered":"2023-12-20T06:02:39.461020+0000","last_clean":"2023-12-20T06:02:39.461020+0000","last_became_active":"2023-12-20T06:02:26.560352+0000","last_became_peered":"2023-12-20T06:02:26.560352+0000","last_unstale":"2023-12-20T06:02:39.461020+0000","last_undegraded":"2023-12-20T06:02:39.461020+0000","last_fullsized":"2023-12-20T06:02:39.461020+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:49.756970+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.8","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452516+0000","last_change":"2023-12-20T06:02:26.968137+0000","last_active":"2023-12-20T06:02:39.452516+0000","last_peered":"2023-12-20T06:02:39.452516+0000","last_clean":"2023-12-20T06:02:39.452516+0000","last_became_active":"2023-12-20T06:02:26.967861+0000","last_became_peered":"2023-12-20T06:02:26.967861+0000","last_unstale":"2023-12-20T06:02:39.452516+0000","last_undegraded":"2023-12-20T06:02:39.452516+0000","last_fullsized":"2023-12-20T06:02:39.452516+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:34:19.170698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.b","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343218+0000","last_change":"2023-12-20T06:02:39.062332+0000","last_active":"2023-12-20T06:02:41.343218+0000","last_peered":"2023-12-20T06:02:41.343218+0000","last_clean":"2023-12-20T06:02:41.343218+0000","last_became_active":"2023-12-20T06:02:39.061352+0000","last_became_peered":"2023-12-20T06:02:39.061352+0000","last_unstale":"2023-12-20T06:02:41.343218+0000","last_undegraded":"2023-12-20T06:02:41.343218+0000","last_fullsized":"2023-12-20T06:02:41.343218+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:23:19.674539+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.9","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338275+0000","last_change":"2023-12-20T06:02:39.475911+0000","last_active":"2023-12-20T06:02:41.338275+0000","last_peered":"2023-12-20T06:02:41.338275+0000","last_clean":"2023-12-20T06:02:41.338275+0000","last_became_active":"2023-12-20T06:02:39.473988+0000","last_became_peered":"2023-12-20T06:02:39.473988+0000","last_unstale":"2023-12-20T06:02:41.338275+0000","last_undegraded":"2023-12-20T06:02:41.338275+0000","last_fullsized":"2023-12-20T06:02:41.338275+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:06:56.699636+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"24.a","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464158+0000","last_change":"2023-12-20T06:02:26.554403+0000","last_active":"2023-12-20T06:02:39.464158+0000","last_peered":"2023-12-20T06:02:39.464158+0000","last_clean":"2023-12-20T06:02:39.464158+0000","last_became_active":"2023-12-20T06:02:26.554112+0000","last_became_peered":"2023-12-20T06:02:26.554112+0000","last_unstale":"2023-12-20T06:02:39.464158+0000","last_undegraded":"2023-12-20T06:02:39.464158+0000","last_fullsized":"2023-12-20T06:02:39.464158+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:53:48.616814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.b","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464798+0000","last_change":"2023-12-20T06:02:26.981736+0000","last_active":"2023-12-20T06:02:39.464798+0000","last_peered":"2023-12-20T06:02:39.464798+0000","last_clean":"2023-12-20T06:02:39.464798+0000","last_became_active":"2023-12-20T06:02:26.981185+0000","last_became_peered":"2023-12-20T06:02:26.981185+0000","last_unstale":"2023-12-20T06:02:39.464798+0000","last_undegraded":"2023-12-20T06:02:39.464798+0000","last_fullsized":"2023-12-20T06:02:39.464798+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:59.846058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.8","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341738+0000","last_change":"2023-12-20T06:02:39.065594+0000","last_active":"2023-12-20T06:02:41.341738+0000","last_peered":"2023-12-20T06:02:41.341738+0000","last_clean":"2023-12-20T06:02:41.341738+0000","last_became_active":"2023-12-20T06:02:39.065283+0000","last_became_peered":"2023-12-20T06:02:39.065283+0000","last_unstale":"2023-12-20T06:02:41.341738+0000","last_undegraded":"2023-12-20T06:02:41.341738+0000","last_fullsized":"2023-12-20T06:02:41.341738+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:08.387196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.8","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457720+0000","last_change":"2023-12-20T06:02:39.486070+0000","last_active":"2023-12-20T06:02:40.457720+0000","last_peered":"2023-12-20T06:02:40.457720+0000","last_clean":"2023-12-20T06:02:40.457720+0000","last_became_active":"2023-12-20T06:02:39.485809+0000","last_became_peered":"2023-12-20T06:02:39.485809+0000","last_unstale":"2023-12-20T06:02:40.457720+0000","last_undegraded":"2023-12-20T06:02:40.457720+0000","last_fullsized":"2023-12-20T06:02:40.457720+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:59.177389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.b","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467169+0000","last_change":"2023-12-20T06:02:26.565662+0000","last_active":"2023-12-20T06:02:39.467169+0000","last_peered":"2023-12-20T06:02:39.467169+0000","last_clean":"2023-12-20T06:02:39.467169+0000","last_became_active":"2023-12-20T06:02:26.564737+0000","last_became_peered":"2023-12-20T06:02:26.564737+0000","last_unstale":"2023-12-20T06:02:39.467169+0000","last_undegraded":"2023-12-20T06:02:39.467169+0000","last_fullsized":"2023-12-20T06:02:39.467169+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:59:26.669013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.a","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453539+0000","last_change":"2023-12-20T06:02:26.972273+0000","last_active":"2023-12-20T06:02:39.453539+0000","last_peered":"2023-12-20T06:02:39.453539+0000","last_clean":"2023-12-20T06:02:39.453539+0000","last_became_active":"2023-12-20T06:02:26.971983+0000","last_became_peered":"2023-12-20T06:02:26.971983+0000","last_unstale":"2023-12-20T06:02:39.453539+0000","last_undegraded":"2023-12-20T06:02:39.453539+0000","last_fullsized":"2023-12-20T06:02:39.453539+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:35:59.779548+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.9","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454298+0000","last_change":"2023-12-20T06:02:39.073826+0000","last_active":"2023-12-20T06:02:40.454298+0000","last_peered":"2023-12-20T06:02:40.454298+0000","last_clean":"2023-12-20T06:02:40.454298+0000","last_became_active":"2023-12-20T06:02:39.073502+0000","last_became_peered":"2023-12-20T06:02:39.073502+0000","last_unstale":"2023-12-20T06:02:40.454298+0000","last_undegraded":"2023-12-20T06:02:40.454298+0000","last_fullsized":"2023-12-20T06:02:40.454298+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:23.668307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.f","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.459506+0000","last_change":"2023-12-20T06:02:39.476958+0000","last_active":"2023-12-20T06:02:40.459506+0000","last_peered":"2023-12-20T06:02:40.459506+0000","last_clean":"2023-12-20T06:02:40.459506+0000","last_became_active":"2023-12-20T06:02:39.476215+0000","last_became_peered":"2023-12-20T06:02:39.476215+0000","last_unstale":"2023-12-20T06:02:40.459506+0000","last_undegraded":"2023-12-20T06:02:40.459506+0000","last_fullsized":"2023-12-20T06:02:40.459506+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:40.405390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.c","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.460043+0000","last_change":"2023-12-20T06:02:26.559666+0000","last_active":"2023-12-20T06:02:39.460043+0000","last_peered":"2023-12-20T06:02:39.460043+0000","last_clean":"2023-12-20T06:02:39.460043+0000","last_became_active":"2023-12-20T06:02:26.559324+0000","last_became_peered":"2023-12-20T06:02:26.559324+0000","last_unstale":"2023-12-20T06:02:39.460043+0000","last_undegraded":"2023-12-20T06:02:39.460043+0000","last_fullsized":"2023-12-20T06:02:39.460043+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:27.148940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.d","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453188+0000","last_change":"2023-12-20T06:02:26.958574+0000","last_active":"2023-12-20T06:02:39.453188+0000","last_peered":"2023-12-20T06:02:39.453188+0000","last_clean":"2023-12-20T06:02:39.453188+0000","last_became_active":"2023-12-20T06:02:26.958284+0000","last_became_peered":"2023-12-20T06:02:26.958284+0000","last_unstale":"2023-12-20T06:02:39.453188+0000","last_undegraded":"2023-12-20T06:02:39.453188+0000","last_fullsized":"2023-12-20T06:02:39.453188+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:28.156831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"26.e","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.459304+0000","last_change":"2023-12-20T06:02:39.065892+0000","last_active":"2023-12-20T06:02:40.459304+0000","last_peered":"2023-12-20T06:02:40.459304+0000","last_clean":"2023-12-20T06:02:40.459304+0000","last_became_active":"2023-12-20T06:02:39.065606+0000","last_became_peered":"2023-12-20T06:02:39.065606+0000","last_unstale":"2023-12-20T06:02:40.459304+0000","last_undegraded":"2023-12-20T06:02:40.459304+0000","last_fullsized":"2023-12-20T06:02:40.459304+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:48:11.323412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.e","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.459066+0000","last_change":"2023-12-20T06:02:39.487282+0000","last_active":"2023-12-20T06:02:40.459066+0000","last_peered":"2023-12-20T06:02:40.459066+0000","last_clean":"2023-12-20T06:02:40.459066+0000","last_became_active":"2023-12-20T06:02:39.483132+0000","last_became_peered":"2023-12-20T06:02:39.483132+0000","last_unstale":"2023-12-20T06:02:40.459066+0000","last_undegraded":"2023-12-20T06:02:40.459066+0000","last_fullsized":"2023-12-20T06:02:40.459066+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:37:42.185095+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.d","version":"208'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467300+0000","last_change":"2023-12-20T06:02:26.553441+0000","last_active":"2023-12-20T06:02:39.467300+0000","last_peered":"2023-12-20T06:02:39.467300+0000","last_clean":"2023-12-20T06:02:39.467300+0000","last_became_active":"2023-12-20T06:02:26.551705+0000","last_became_peered":"2023-12-20T06:02:26.551705+0000","last_unstale":"2023-12-20T06:02:39.467300+0000","last_undegraded":"2023-12-20T06:02:39.467300+0000","last_fullsized":"2023-12-20T06:02:39.467300+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:51.593076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.c","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464368+0000","last_change":"2023-12-20T06:02:26.977809+0000","last_active":"2023-12-20T06:02:39.464368+0000","last_peered":"2023-12-20T06:02:39.464368+0000","last_clean":"2023-12-20T06:02:39.464368+0000","last_became_active":"2023-12-20T06:02:26.976835+0000","last_became_peered":"2023-12-20T06:02:26.976835+0000","last_unstale":"2023-12-20T06:02:39.464368+0000","last_undegraded":"2023-12-20T06:02:39.464368+0000","last_fullsized":"2023-12-20T06:02:39.464368+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.f","version":"212'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.344720+0000","last_change":"2023-12-20T06:02:39.068363+0000","last_active":"2023-12-20T06:02:41.344720+0000","last_peered":"2023-12-20T06:02:41.344720+0000","last_clean":"2023-12-20T06:02:41.344720+0000","last_became_active":"2023-12-20T06:02:39.065458+0000","last_became_peered":"2023-12-20T06:02:39.065458+0000","last_unstale":"2023-12-20T06:02:41.344720+0000","last_undegraded":"2023-12-20T06:02:41.344720+0000","last_fullsized":"2023-12-20T06:02:41.344720+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:25:06.985482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.d","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343802+0000","last_change":"2023-12-20T06:02:39.476547+0000","last_active":"2023-12-20T06:02:41.343802+0000","last_peered":"2023-12-20T06:02:41.343802+0000","last_clean":"2023-12-20T06:02:41.343802+0000","last_became_active":"2023-12-20T06:02:39.476284+0000","last_became_peered":"2023-12-20T06:02:39.476284+0000","last_unstale":"2023-12-20T06:02:41.343802+0000","last_undegraded":"2023-12-20T06:02:41.343802+0000","last_fullsized":"2023-12-20T06:02:41.343802+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:35:24.687018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.e","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.468631+0000","last_change":"2023-12-20T06:02:26.561295+0000","last_active":"2023-12-20T06:02:39.468631+0000","last_peered":"2023-12-20T06:02:39.468631+0000","last_clean":"2023-12-20T06:02:39.468631+0000","last_became_active":"2023-12-20T06:02:26.561016+0000","last_became_peered":"2023-12-20T06:02:26.561016+0000","last_unstale":"2023-12-20T06:02:39.468631+0000","last_undegraded":"2023-12-20T06:02:39.468631+0000","last_fullsized":"2023-12-20T06:02:39.468631+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:53.307500+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.f","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464153+0000","last_change":"2023-12-20T06:02:26.979723+0000","last_active":"2023-12-20T06:02:39.464153+0000","last_peered":"2023-12-20T06:02:39.464153+0000","last_clean":"2023-12-20T06:02:39.464153+0000","last_became_active":"2023-12-20T06:02:26.979447+0000","last_became_peered":"2023-12-20T06:02:26.979447+0000","last_unstale":"2023-12-20T06:02:39.464153+0000","last_undegraded":"2023-12-20T06:02:39.464153+0000","last_fullsized":"2023-12-20T06:02:39.464153+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:44:12.085696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.c","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343043+0000","last_change":"2023-12-20T06:02:39.059882+0000","last_active":"2023-12-20T06:02:41.343043+0000","last_peered":"2023-12-20T06:02:41.343043+0000","last_clean":"2023-12-20T06:02:41.343043+0000","last_became_active":"2023-12-20T06:02:39.059354+0000","last_became_peered":"2023-12-20T06:02:39.059354+0000","last_unstale":"2023-12-20T06:02:41.343043+0000","last_undegraded":"2023-12-20T06:02:41.343043+0000","last_fullsized":"2023-12-20T06:02:41.343043+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:52.925504+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.c","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342658+0000","last_change":"2023-12-20T06:02:39.482797+0000","last_active":"2023-12-20T06:02:41.342658+0000","last_peered":"2023-12-20T06:02:41.342658+0000","last_clean":"2023-12-20T06:02:41.342658+0000","last_became_active":"2023-12-20T06:02:39.482116+0000","last_became_peered":"2023-12-20T06:02:39.482116+0000","last_unstale":"2023-12-20T06:02:41.342658+0000","last_undegraded":"2023-12-20T06:02:41.342658+0000","last_fullsized":"2023-12-20T06:02:41.342658+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:27.643717+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.f","version":"208'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.469653+0000","last_change":"2023-12-20T06:02:26.559146+0000","last_active":"2023-12-20T06:02:39.469653+0000","last_peered":"2023-12-20T06:02:39.469653+0000","last_clean":"2023-12-20T06:02:39.469653+0000","last_became_active":"2023-12-20T06:02:26.558865+0000","last_became_peered":"2023-12-20T06:02:26.558865+0000","last_unstale":"2023-12-20T06:02:39.469653+0000","last_undegraded":"2023-12-20T06:02:39.469653+0000","last_fullsized":"2023-12-20T06:02:39.469653+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:23:28.617951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.e","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.454912+0000","last_change":"2023-12-20T06:02:26.973527+0000","last_active":"2023-12-20T06:02:39.454912+0000","last_peered":"2023-12-20T06:02:39.454912+0000","last_clean":"2023-12-20T06:02:39.454912+0000","last_became_active":"2023-12-20T06:02:26.972995+0000","last_became_peered":"2023-12-20T06:02:26.972995+0000","last_unstale":"2023-12-20T06:02:39.454912+0000","last_undegraded":"2023-12-20T06:02:39.454912+0000","last_fullsized":"2023-12-20T06:02:39.454912+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:15:22.108587+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.d","version":"212'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343885+0000","last_change":"2023-12-20T06:02:39.071884+0000","last_active":"2023-12-20T06:02:41.343885+0000","last_peered":"2023-12-20T06:02:41.343885+0000","last_clean":"2023-12-20T06:02:41.343885+0000","last_became_active":"2023-12-20T06:02:39.071083+0000","last_became_peered":"2023-12-20T06:02:39.071083+0000","last_unstale":"2023-12-20T06:02:41.343885+0000","last_undegraded":"2023-12-20T06:02:41.343885+0000","last_fullsized":"2023-12-20T06:02:41.343885+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:51:10.101774+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.13","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342270+0000","last_change":"2023-12-20T06:02:39.480913+0000","last_active":"2023-12-20T06:02:41.342270+0000","last_peered":"2023-12-20T06:02:41.342270+0000","last_clean":"2023-12-20T06:02:41.342270+0000","last_became_active":"2023-12-20T06:02:39.480356+0000","last_became_peered":"2023-12-20T06:02:39.480356+0000","last_unstale":"2023-12-20T06:02:41.342270+0000","last_undegraded":"2023-12-20T06:02:41.342270+0000","last_fullsized":"2023-12-20T06:02:41.342270+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:25.169128+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.10","version":"208'8","reported_seq":23,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465560+0000","last_change":"2023-12-20T06:02:26.552002+0000","last_active":"2023-12-20T06:02:39.465560+0000","last_peered":"2023-12-20T06:02:39.465560+0000","last_clean":"2023-12-20T06:02:39.465560+0000","last_became_active":"2023-12-20T06:02:26.551664+0000","last_became_peered":"2023-12-20T06:02:26.551664+0000","last_unstale":"2023-12-20T06:02:39.465560+0000","last_undegraded":"2023-12-20T06:02:39.465560+0000","last_fullsized":"2023-12-20T06:02:39.465560+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:49:14.073356+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.11","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452751+0000","last_change":"2023-12-20T06:02:26.970325+0000","last_active":"2023-12-20T06:02:39.452751+0000","last_peered":"2023-12-20T06:02:39.452751+0000","last_clean":"2023-12-20T06:02:39.452751+0000","last_became_active":"2023-12-20T06:02:26.970054+0000","last_became_peered":"2023-12-20T06:02:26.970054+0000","last_unstale":"2023-12-20T06:02:39.452751+0000","last_undegraded":"2023-12-20T06:02:39.452751+0000","last_fullsized":"2023-12-20T06:02:39.452751+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:23:08.197185+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.12","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338321+0000","last_change":"2023-12-20T06:02:39.070297+0000","last_active":"2023-12-20T06:02:41.338321+0000","last_peered":"2023-12-20T06:02:41.338321+0000","last_clean":"2023-12-20T06:02:41.338321+0000","last_became_active":"2023-12-20T06:02:39.066645+0000","last_became_peered":"2023-12-20T06:02:39.066645+0000","last_unstale":"2023-12-20T06:02:41.338321+0000","last_undegraded":"2023-12-20T06:02:41.338321+0000","last_fullsized":"2023-12-20T06:02:41.338321+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:20:36.068915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.12","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343236+0000","last_change":"2023-12-20T06:02:39.477713+0000","last_active":"2023-12-20T06:02:41.343236+0000","last_peered":"2023-12-20T06:02:41.343236+0000","last_clean":"2023-12-20T06:02:41.343236+0000","last_became_active":"2023-12-20T06:02:39.477419+0000","last_became_peered":"2023-12-20T06:02:39.477419+0000","last_unstale":"2023-12-20T06:02:41.343236+0000","last_undegraded":"2023-12-20T06:02:41.343236+0000","last_fullsized":"2023-12-20T06:02:41.343236+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:38:05.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.11","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465107+0000","last_change":"2023-12-20T06:02:26.558390+0000","last_active":"2023-12-20T06:02:39.465107+0000","last_peered":"2023-12-20T06:02:39.465107+0000","last_clean":"2023-12-20T06:02:39.465107+0000","last_became_active":"2023-12-20T06:02:26.558078+0000","last_became_peered":"2023-12-20T06:02:26.558078+0000","last_unstale":"2023-12-20T06:02:39.465107+0000","last_undegraded":"2023-12-20T06:02:39.465107+0000","last_fullsized":"2023-12-20T06:02:39.465107+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:47:56.455893+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.10","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463814+0000","last_change":"2023-12-20T06:02:26.978618+0000","last_active":"2023-12-20T06:02:39.463814+0000","last_peered":"2023-12-20T06:02:39.463814+0000","last_clean":"2023-12-20T06:02:39.463814+0000","last_became_active":"2023-12-20T06:02:26.978305+0000","last_became_peered":"2023-12-20T06:02:26.978305+0000","last_unstale":"2023-12-20T06:02:39.463814+0000","last_undegraded":"2023-12-20T06:02:39.463814+0000","last_fullsized":"2023-12-20T06:02:39.463814+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:02:53.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.13","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338510+0000","last_change":"2023-12-20T06:02:39.068327+0000","last_active":"2023-12-20T06:02:41.338510+0000","last_peered":"2023-12-20T06:02:41.338510+0000","last_clean":"2023-12-20T06:02:41.338510+0000","last_became_active":"2023-12-20T06:02:39.067332+0000","last_became_peered":"2023-12-20T06:02:39.067332+0000","last_unstale":"2023-12-20T06:02:41.338510+0000","last_undegraded":"2023-12-20T06:02:41.338510+0000","last_fullsized":"2023-12-20T06:02:41.338510+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:11:37.511013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.11","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454276+0000","last_change":"2023-12-20T06:02:39.479905+0000","last_active":"2023-12-20T06:02:40.454276+0000","last_peered":"2023-12-20T06:02:40.454276+0000","last_clean":"2023-12-20T06:02:40.454276+0000","last_became_active":"2023-12-20T06:02:39.479376+0000","last_became_peered":"2023-12-20T06:02:39.479376+0000","last_unstale":"2023-12-20T06:02:40.454276+0000","last_undegraded":"2023-12-20T06:02:40.454276+0000","last_fullsized":"2023-12-20T06:02:40.454276+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:41.309037+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.12","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453629+0000","last_change":"2023-12-20T06:02:26.553685+0000","last_active":"2023-12-20T06:02:39.453629+0000","last_peered":"2023-12-20T06:02:39.453629+0000","last_clean":"2023-12-20T06:02:39.453629+0000","last_became_active":"2023-12-20T06:02:26.553411+0000","last_became_peered":"2023-12-20T06:02:26.553411+0000","last_unstale":"2023-12-20T06:02:39.453629+0000","last_undegraded":"2023-12-20T06:02:39.453629+0000","last_fullsized":"2023-12-20T06:02:39.453629+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:52.266518+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.13","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464689+0000","last_change":"2023-12-20T06:02:26.979070+0000","last_active":"2023-12-20T06:02:39.464689+0000","last_peered":"2023-12-20T06:02:39.464689+0000","last_clean":"2023-12-20T06:02:39.464689+0000","last_became_active":"2023-12-20T06:02:26.978777+0000","last_became_peered":"2023-12-20T06:02:26.978777+0000","last_unstale":"2023-12-20T06:02:39.464689+0000","last_undegraded":"2023-12-20T06:02:39.464689+0000","last_fullsized":"2023-12-20T06:02:39.464689+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:04:54.315390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.10","version":"212'8","reported_seq":19,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342160+0000","last_change":"2023-12-20T06:02:39.064515+0000","last_active":"2023-12-20T06:02:41.342160+0000","last_peered":"2023-12-20T06:02:41.342160+0000","last_clean":"2023-12-20T06:02:41.342160+0000","last_became_active":"2023-12-20T06:02:39.063462+0000","last_became_peered":"2023-12-20T06:02:39.063462+0000","last_unstale":"2023-12-20T06:02:41.342160+0000","last_undegraded":"2023-12-20T06:02:41.342160+0000","last_fullsized":"2023-12-20T06:02:41.342160+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:19:48.734380+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.10","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342263+0000","last_change":"2023-12-20T06:02:39.475798+0000","last_active":"2023-12-20T06:02:41.342263+0000","last_peered":"2023-12-20T06:02:41.342263+0000","last_clean":"2023-12-20T06:02:41.342263+0000","last_became_active":"2023-12-20T06:02:39.475515+0000","last_became_peered":"2023-12-20T06:02:39.475515+0000","last_unstale":"2023-12-20T06:02:41.342263+0000","last_undegraded":"2023-12-20T06:02:41.342263+0000","last_fullsized":"2023-12-20T06:02:41.342263+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:53:40.467317+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.13","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467873+0000","last_change":"2023-12-20T06:02:26.552924+0000","last_active":"2023-12-20T06:02:39.467873+0000","last_peered":"2023-12-20T06:02:39.467873+0000","last_clean":"2023-12-20T06:02:39.467873+0000","last_became_active":"2023-12-20T06:02:26.552647+0000","last_became_peered":"2023-12-20T06:02:26.552647+0000","last_unstale":"2023-12-20T06:02:39.467873+0000","last_undegraded":"2023-12-20T06:02:39.467873+0000","last_fullsized":"2023-12-20T06:02:39.467873+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:44.700263+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.12","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464153+0000","last_change":"2023-12-20T06:02:26.976279+0000","last_active":"2023-12-20T06:02:39.464153+0000","last_peered":"2023-12-20T06:02:39.464153+0000","last_clean":"2023-12-20T06:02:39.464153+0000","last_became_active":"2023-12-20T06:02:26.976019+0000","last_became_peered":"2023-12-20T06:02:26.976019+0000","last_unstale":"2023-12-20T06:02:39.464153+0000","last_undegraded":"2023-12-20T06:02:39.464153+0000","last_fullsized":"2023-12-20T06:02:39.464153+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.11","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341732+0000","last_change":"2023-12-20T06:02:39.064453+0000","last_active":"2023-12-20T06:02:41.341732+0000","last_peered":"2023-12-20T06:02:41.341732+0000","last_clean":"2023-12-20T06:02:41.341732+0000","last_became_active":"2023-12-20T06:02:39.064128+0000","last_became_peered":"2023-12-20T06:02:39.064128+0000","last_unstale":"2023-12-20T06:02:41.341732+0000","last_undegraded":"2023-12-20T06:02:41.341732+0000","last_fullsized":"2023-12-20T06:02:41.341732+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:48.000452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.17","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342190+0000","last_change":"2023-12-20T06:02:39.478799+0000","last_active":"2023-12-20T06:02:41.342190+0000","last_peered":"2023-12-20T06:02:41.342190+0000","last_clean":"2023-12-20T06:02:41.342190+0000","last_became_active":"2023-12-20T06:02:39.478537+0000","last_became_peered":"2023-12-20T06:02:39.478537+0000","last_unstale":"2023-12-20T06:02:41.342190+0000","last_undegraded":"2023-12-20T06:02:41.342190+0000","last_fullsized":"2023-12-20T06:02:41.342190+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:26:16.444787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"24.14","version":"209'8","reported_seq":23,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467468+0000","last_change":"2023-12-20T06:02:26.558119+0000","last_active":"2023-12-20T06:02:39.467468+0000","last_peered":"2023-12-20T06:02:39.467468+0000","last_clean":"2023-12-20T06:02:39.467468+0000","last_became_active":"2023-12-20T06:02:26.557699+0000","last_became_peered":"2023-12-20T06:02:26.557699+0000","last_unstale":"2023-12-20T06:02:39.467468+0000","last_undegraded":"2023-12-20T06:02:39.467468+0000","last_fullsized":"2023-12-20T06:02:39.467468+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:21:20.189262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.15","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.455517+0000","last_change":"2023-12-20T06:02:26.978068+0000","last_active":"2023-12-20T06:02:39.455517+0000","last_peered":"2023-12-20T06:02:39.455517+0000","last_clean":"2023-12-20T06:02:39.455517+0000","last_became_active":"2023-12-20T06:02:26.977750+0000","last_became_peered":"2023-12-20T06:02:26.977750+0000","last_unstale":"2023-12-20T06:02:39.455517+0000","last_undegraded":"2023-12-20T06:02:39.455517+0000","last_fullsized":"2023-12-20T06:02:39.455517+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:05:59.363948+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.16","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338658+0000","last_change":"2023-12-20T06:02:39.069292+0000","last_active":"2023-12-20T06:02:41.338658+0000","last_peered":"2023-12-20T06:02:41.338658+0000","last_clean":"2023-12-20T06:02:41.338658+0000","last_became_active":"2023-12-20T06:02:39.068739+0000","last_became_peered":"2023-12-20T06:02:39.068739+0000","last_unstale":"2023-12-20T06:02:41.338658+0000","last_undegraded":"2023-12-20T06:02:41.338658+0000","last_fullsized":"2023-12-20T06:02:41.338658+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:16:17.340651+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.16","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.247332+0000","last_change":"2023-12-20T06:02:39.489204+0000","last_active":"2023-12-20T06:02:41.247332+0000","last_peered":"2023-12-20T06:02:41.247332+0000","last_clean":"2023-12-20T06:02:41.247332+0000","last_became_active":"2023-12-20T06:02:39.488895+0000","last_became_peered":"2023-12-20T06:02:39.488895+0000","last_unstale":"2023-12-20T06:02:41.247332+0000","last_undegraded":"2023-12-20T06:02:41.247332+0000","last_fullsized":"2023-12-20T06:02:41.247332+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:17:37.870090+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.15","version":"208'3","reported_seq":18,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464279+0000","last_change":"2023-12-20T06:02:26.555063+0000","last_active":"2023-12-20T06:02:39.464279+0000","last_peered":"2023-12-20T06:02:39.464279+0000","last_clean":"2023-12-20T06:02:39.464279+0000","last_became_active":"2023-12-20T06:02:26.554776+0000","last_became_peered":"2023-12-20T06:02:26.554776+0000","last_unstale":"2023-12-20T06:02:39.464279+0000","last_undegraded":"2023-12-20T06:02:39.464279+0000","last_fullsized":"2023-12-20T06:02:39.464279+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:59:43.701253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.14","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.459957+0000","last_change":"2023-12-20T06:02:26.966176+0000","last_active":"2023-12-20T06:02:39.459957+0000","last_peered":"2023-12-20T06:02:39.459957+0000","last_clean":"2023-12-20T06:02:39.459957+0000","last_became_active":"2023-12-20T06:02:26.965898+0000","last_became_peered":"2023-12-20T06:02:26.965898+0000","last_unstale":"2023-12-20T06:02:39.459957+0000","last_undegraded":"2023-12-20T06:02:39.459957+0000","last_fullsized":"2023-12-20T06:02:39.459957+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:54:07.886847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.17","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.344475+0000","last_change":"2023-12-20T06:02:39.069670+0000","last_active":"2023-12-20T06:02:41.344475+0000","last_peered":"2023-12-20T06:02:41.344475+0000","last_clean":"2023-12-20T06:02:41.344475+0000","last_became_active":"2023-12-20T06:02:39.069404+0000","last_became_peered":"2023-12-20T06:02:39.069404+0000","last_unstale":"2023-12-20T06:02:41.344475+0000","last_undegraded":"2023-12-20T06:02:41.344475+0000","last_fullsized":"2023-12-20T06:02:41.344475+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:13:24.966993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.15","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454993+0000","last_change":"2023-12-20T06:02:39.475003+0000","last_active":"2023-12-20T06:02:40.454993+0000","last_peered":"2023-12-20T06:02:40.454993+0000","last_clean":"2023-12-20T06:02:40.454993+0000","last_became_active":"2023-12-20T06:02:39.473932+0000","last_became_peered":"2023-12-20T06:02:39.473932+0000","last_unstale":"2023-12-20T06:02:40.454993+0000","last_undegraded":"2023-12-20T06:02:40.454993+0000","last_fullsized":"2023-12-20T06:02:40.454993+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:02:36.525311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.16","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464547+0000","last_change":"2023-12-20T06:02:26.551507+0000","last_active":"2023-12-20T06:02:39.464547+0000","last_peered":"2023-12-20T06:02:39.464547+0000","last_clean":"2023-12-20T06:02:39.464547+0000","last_became_active":"2023-12-20T06:02:26.551215+0000","last_became_peered":"2023-12-20T06:02:26.551215+0000","last_unstale":"2023-12-20T06:02:39.464547+0000","last_undegraded":"2023-12-20T06:02:39.464547+0000","last_fullsized":"2023-12-20T06:02:39.464547+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:58:17.512600+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.17","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.470106+0000","last_change":"2023-12-20T06:02:26.981970+0000","last_active":"2023-12-20T06:02:39.470106+0000","last_peered":"2023-12-20T06:02:39.470106+0000","last_clean":"2023-12-20T06:02:39.470106+0000","last_became_active":"2023-12-20T06:02:26.981688+0000","last_became_peered":"2023-12-20T06:02:26.981688+0000","last_unstale":"2023-12-20T06:02:39.470106+0000","last_undegraded":"2023-12-20T06:02:39.470106+0000","last_fullsized":"2023-12-20T06:02:39.470106+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:29:15.785654+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.14","version":"212'6","reported_seq":17,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246990+0000","last_change":"2023-12-20T06:02:39.070888+0000","last_active":"2023-12-20T06:02:41.246990+0000","last_peered":"2023-12-20T06:02:41.246990+0000","last_clean":"2023-12-20T06:02:41.246990+0000","last_became_active":"2023-12-20T06:02:39.070561+0000","last_became_peered":"2023-12-20T06:02:39.070561+0000","last_unstale":"2023-12-20T06:02:41.246990+0000","last_undegraded":"2023-12-20T06:02:41.246990+0000","last_fullsized":"2023-12-20T06:02:41.246990+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:07:20.758077+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.14","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.247045+0000","last_change":"2023-12-20T06:02:39.482522+0000","last_active":"2023-12-20T06:02:41.247045+0000","last_peered":"2023-12-20T06:02:41.247045+0000","last_clean":"2023-12-20T06:02:41.247045+0000","last_became_active":"2023-12-20T06:02:39.481770+0000","last_became_peered":"2023-12-20T06:02:39.481770+0000","last_unstale":"2023-12-20T06:02:41.247045+0000","last_undegraded":"2023-12-20T06:02:41.247045+0000","last_fullsized":"2023-12-20T06:02:41.247045+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:03:23.864841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.17","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465049+0000","last_change":"2023-12-20T06:02:26.557303+0000","last_active":"2023-12-20T06:02:39.465049+0000","last_peered":"2023-12-20T06:02:39.465049+0000","last_clean":"2023-12-20T06:02:39.465049+0000","last_became_active":"2023-12-20T06:02:26.556838+0000","last_became_peered":"2023-12-20T06:02:26.556838+0000","last_unstale":"2023-12-20T06:02:39.465049+0000","last_undegraded":"2023-12-20T06:02:39.465049+0000","last_fullsized":"2023-12-20T06:02:39.465049+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:12:36.511646+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.16","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467349+0000","last_change":"2023-12-20T06:02:26.974738+0000","last_active":"2023-12-20T06:02:39.467349+0000","last_peered":"2023-12-20T06:02:39.467349+0000","last_clean":"2023-12-20T06:02:39.467349+0000","last_became_active":"2023-12-20T06:02:26.974494+0000","last_became_peered":"2023-12-20T06:02:26.974494+0000","last_unstale":"2023-12-20T06:02:39.467349+0000","last_undegraded":"2023-12-20T06:02:39.467349+0000","last_fullsized":"2023-12-20T06:02:39.467349+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"26.15","version":"212'3","reported_seq":14,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.344116+0000","last_change":"2023-12-20T06:02:39.067994+0000","last_active":"2023-12-20T06:02:41.344116+0000","last_peered":"2023-12-20T06:02:41.344116+0000","last_clean":"2023-12-20T06:02:41.344116+0000","last_became_active":"2023-12-20T06:02:39.067516+0000","last_became_peered":"2023-12-20T06:02:39.067516+0000","last_unstale":"2023-12-20T06:02:41.344116+0000","last_undegraded":"2023-12-20T06:02:41.344116+0000","last_fullsized":"2023-12-20T06:02:41.344116+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:05:55.241309+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.1b","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246913+0000","last_change":"2023-12-20T06:02:39.484273+0000","last_active":"2023-12-20T06:02:41.246913+0000","last_peered":"2023-12-20T06:02:41.246913+0000","last_clean":"2023-12-20T06:02:41.246913+0000","last_became_active":"2023-12-20T06:02:39.483945+0000","last_became_peered":"2023-12-20T06:02:39.483945+0000","last_unstale":"2023-12-20T06:02:41.246913+0000","last_undegraded":"2023-12-20T06:02:41.246913+0000","last_fullsized":"2023-12-20T06:02:41.246913+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:24.135189+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.18","version":"0'0","reported_seq":15,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465458+0000","last_change":"2023-12-20T06:02:26.553243+0000","last_active":"2023-12-20T06:02:39.465458+0000","last_peered":"2023-12-20T06:02:39.465458+0000","last_clean":"2023-12-20T06:02:39.465458+0000","last_became_active":"2023-12-20T06:02:26.551953+0000","last_became_peered":"2023-12-20T06:02:26.551953+0000","last_unstale":"2023-12-20T06:02:39.465458+0000","last_undegraded":"2023-12-20T06:02:39.465458+0000","last_fullsized":"2023-12-20T06:02:39.465458+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:09:54.113512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.19","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.458492+0000","last_change":"2023-12-20T06:02:26.977844+0000","last_active":"2023-12-20T06:02:39.458492+0000","last_peered":"2023-12-20T06:02:39.458492+0000","last_clean":"2023-12-20T06:02:39.458492+0000","last_became_active":"2023-12-20T06:02:26.977331+0000","last_became_peered":"2023-12-20T06:02:26.977331+0000","last_unstale":"2023-12-20T06:02:39.458492+0000","last_undegraded":"2023-12-20T06:02:39.458492+0000","last_fullsized":"2023-12-20T06:02:39.458492+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:59:51.952536+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1a","version":"212'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342946+0000","last_change":"2023-12-20T06:02:39.070793+0000","last_active":"2023-12-20T06:02:41.342946+0000","last_peered":"2023-12-20T06:02:41.342946+0000","last_clean":"2023-12-20T06:02:41.342946+0000","last_became_active":"2023-12-20T06:02:39.070475+0000","last_became_peered":"2023-12-20T06:02:39.070475+0000","last_unstale":"2023-12-20T06:02:41.342946+0000","last_undegraded":"2023-12-20T06:02:41.342946+0000","last_fullsized":"2023-12-20T06:02:41.342946+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:48:11.323412+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.19","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246623+0000","last_change":"2023-12-20T06:02:39.480392+0000","last_active":"2023-12-20T06:02:41.246623+0000","last_peered":"2023-12-20T06:02:41.246623+0000","last_clean":"2023-12-20T06:02:41.246623+0000","last_became_active":"2023-12-20T06:02:39.480114+0000","last_became_peered":"2023-12-20T06:02:39.480114+0000","last_unstale":"2023-12-20T06:02:41.246623+0000","last_undegraded":"2023-12-20T06:02:41.246623+0000","last_fullsized":"2023-12-20T06:02:41.246623+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:09.405907+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"24.1a","version":"208'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465572+0000","last_change":"2023-12-20T06:02:26.556300+0000","last_active":"2023-12-20T06:02:39.465572+0000","last_peered":"2023-12-20T06:02:39.465572+0000","last_clean":"2023-12-20T06:02:39.465572+0000","last_became_active":"2023-12-20T06:02:26.556007+0000","last_became_peered":"2023-12-20T06:02:26.556007+0000","last_unstale":"2023-12-20T06:02:39.465572+0000","last_undegraded":"2023-12-20T06:02:39.465572+0000","last_fullsized":"2023-12-20T06:02:39.465572+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:47:58.838168+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.1b","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465458+0000","last_change":"2023-12-20T06:02:26.966409+0000","last_active":"2023-12-20T06:02:39.465458+0000","last_peered":"2023-12-20T06:02:39.465458+0000","last_clean":"2023-12-20T06:02:39.465458+0000","last_became_active":"2023-12-20T06:02:26.966153+0000","last_became_peered":"2023-12-20T06:02:26.966153+0000","last_unstale":"2023-12-20T06:02:39.465458+0000","last_undegraded":"2023-12-20T06:02:39.465458+0000","last_fullsized":"2023-12-20T06:02:39.465458+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:27.909325+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.18","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338066+0000","last_change":"2023-12-20T06:02:39.067435+0000","last_active":"2023-12-20T06:02:41.338066+0000","last_peered":"2023-12-20T06:02:41.338066+0000","last_clean":"2023-12-20T06:02:41.338066+0000","last_became_active":"2023-12-20T06:02:39.064642+0000","last_became_peered":"2023-12-20T06:02:39.064642+0000","last_unstale":"2023-12-20T06:02:41.338066+0000","last_undegraded":"2023-12-20T06:02:41.338066+0000","last_fullsized":"2023-12-20T06:02:41.338066+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:35:06.589787+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.18","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341644+0000","last_change":"2023-12-20T06:02:39.479299+0000","last_active":"2023-12-20T06:02:41.341644+0000","last_peered":"2023-12-20T06:02:41.341644+0000","last_clean":"2023-12-20T06:02:41.341644+0000","last_became_active":"2023-12-20T06:02:39.477232+0000","last_became_peered":"2023-12-20T06:02:39.477232+0000","last_unstale":"2023-12-20T06:02:41.341644+0000","last_undegraded":"2023-12-20T06:02:41.341644+0000","last_fullsized":"2023-12-20T06:02:41.341644+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:20:37.087626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"24.1b","version":"208'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464771+0000","last_change":"2023-12-20T06:02:26.560921+0000","last_active":"2023-12-20T06:02:39.464771+0000","last_peered":"2023-12-20T06:02:39.464771+0000","last_clean":"2023-12-20T06:02:39.464771+0000","last_became_active":"2023-12-20T06:02:26.560567+0000","last_became_peered":"2023-12-20T06:02:26.560567+0000","last_unstale":"2023-12-20T06:02:39.464771+0000","last_undegraded":"2023-12-20T06:02:39.464771+0000","last_fullsized":"2023-12-20T06:02:39.464771+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:01:42.572698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1a","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453444+0000","last_change":"2023-12-20T06:02:26.974647+0000","last_active":"2023-12-20T06:02:39.453444+0000","last_peered":"2023-12-20T06:02:39.453444+0000","last_clean":"2023-12-20T06:02:39.453444+0000","last_became_active":"2023-12-20T06:02:26.974369+0000","last_became_peered":"2023-12-20T06:02:26.974369+0000","last_unstale":"2023-12-20T06:02:39.453444+0000","last_undegraded":"2023-12-20T06:02:39.453444+0000","last_fullsized":"2023-12-20T06:02:39.453444+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:37:36.494152+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.19","version":"0'0","reported_seq":11,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341647+0000","last_change":"2023-12-20T06:02:39.058821+0000","last_active":"2023-12-20T06:02:41.341647+0000","last_peered":"2023-12-20T06:02:41.341647+0000","last_clean":"2023-12-20T06:02:41.341647+0000","last_became_active":"2023-12-20T06:02:39.057269+0000","last_became_peered":"2023-12-20T06:02:39.057269+0000","last_unstale":"2023-12-20T06:02:41.341647+0000","last_undegraded":"2023-12-20T06:02:41.341647+0000","last_fullsized":"2023-12-20T06:02:41.341647+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:23.116478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.1f","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338763+0000","last_change":"2023-12-20T06:02:39.478830+0000","last_active":"2023-12-20T06:02:41.338763+0000","last_peered":"2023-12-20T06:02:41.338763+0000","last_clean":"2023-12-20T06:02:41.338763+0000","last_became_active":"2023-12-20T06:02:39.473968+0000","last_became_peered":"2023-12-20T06:02:39.473968+0000","last_unstale":"2023-12-20T06:02:41.338763+0000","last_undegraded":"2023-12-20T06:02:41.338763+0000","last_fullsized":"2023-12-20T06:02:41.338763+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:34:03.880266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"26.1e","version":"212'1","reported_seq":12,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.459150+0000","last_change":"2023-12-20T06:02:39.066507+0000","last_active":"2023-12-20T06:02:40.459150+0000","last_peered":"2023-12-20T06:02:40.459150+0000","last_clean":"2023-12-20T06:02:40.459150+0000","last_became_active":"2023-12-20T06:02:39.066215+0000","last_became_peered":"2023-12-20T06:02:39.066215+0000","last_unstale":"2023-12-20T06:02:40.459150+0000","last_undegraded":"2023-12-20T06:02:40.459150+0000","last_fullsized":"2023-12-20T06:02:40.459150+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:13:24.966993+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"24.1c","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453539+0000","last_change":"2023-12-20T06:02:26.545330+0000","last_active":"2023-12-20T06:02:39.453539+0000","last_peered":"2023-12-20T06:02:39.453539+0000","last_clean":"2023-12-20T06:02:39.453539+0000","last_became_active":"2023-12-20T06:02:26.545018+0000","last_became_peered":"2023-12-20T06:02:26.545018+0000","last_unstale":"2023-12-20T06:02:39.453539+0000","last_undegraded":"2023-12-20T06:02:39.453539+0000","last_fullsized":"2023-12-20T06:02:39.453539+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:21:20.189262+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.1d","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463698+0000","last_change":"2023-12-20T06:02:26.966379+0000","last_active":"2023-12-20T06:02:39.463698+0000","last_peered":"2023-12-20T06:02:39.463698+0000","last_clean":"2023-12-20T06:02:39.463698+0000","last_became_active":"2023-12-20T06:02:26.965974+0000","last_became_peered":"2023-12-20T06:02:26.965974+0000","last_unstale":"2023-12-20T06:02:39.463698+0000","last_undegraded":"2023-12-20T06:02:39.463698+0000","last_fullsized":"2023-12-20T06:02:39.463698+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:28.156831+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.1e","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458963+0000","last_change":"2023-12-20T06:02:39.483649+0000","last_active":"2023-12-20T06:02:40.458963+0000","last_peered":"2023-12-20T06:02:40.458963+0000","last_clean":"2023-12-20T06:02:40.458963+0000","last_became_active":"2023-12-20T06:02:39.481512+0000","last_became_peered":"2023-12-20T06:02:39.481512+0000","last_unstale":"2023-12-20T06:02:40.458963+0000","last_undegraded":"2023-12-20T06:02:40.458963+0000","last_fullsized":"2023-12-20T06:02:40.458963+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:52:35.601091+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"26.1f","version":"212'4","reported_seq":15,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458734+0000","last_change":"2023-12-20T06:02:39.066833+0000","last_active":"2023-12-20T06:02:40.458734+0000","last_peered":"2023-12-20T06:02:40.458734+0000","last_clean":"2023-12-20T06:02:40.458734+0000","last_became_active":"2023-12-20T06:02:39.066523+0000","last_became_peered":"2023-12-20T06:02:39.066523+0000","last_unstale":"2023-12-20T06:02:40.458734+0000","last_undegraded":"2023-12-20T06:02:40.458734+0000","last_fullsized":"2023-12-20T06:02:40.458734+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:15:08.387196+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"24.1d","version":"208'2","reported_seq":17,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467472+0000","last_change":"2023-12-20T06:02:26.560653+0000","last_active":"2023-12-20T06:02:39.467472+0000","last_peered":"2023-12-20T06:02:39.467472+0000","last_clean":"2023-12-20T06:02:39.467472+0000","last_became_active":"2023-12-20T06:02:26.560323+0000","last_became_peered":"2023-12-20T06:02:26.560323+0000","last_unstale":"2023-12-20T06:02:39.467472+0000","last_undegraded":"2023-12-20T06:02:39.467472+0000","last_fullsized":"2023-12-20T06:02:39.467472+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:55.646229+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.1c","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.459873+0000","last_change":"2023-12-20T06:02:26.967888+0000","last_active":"2023-12-20T06:02:39.459873+0000","last_peered":"2023-12-20T06:02:39.459873+0000","last_clean":"2023-12-20T06:02:39.459873+0000","last_became_active":"2023-12-20T06:02:26.967582+0000","last_became_peered":"2023-12-20T06:02:26.967582+0000","last_unstale":"2023-12-20T06:02:39.459873+0000","last_undegraded":"2023-12-20T06:02:39.459873+0000","last_fullsized":"2023-12-20T06:02:39.459873+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:26:03.948722+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.1d","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343688+0000","last_change":"2023-12-20T06:02:39.475404+0000","last_active":"2023-12-20T06:02:41.343688+0000","last_peered":"2023-12-20T06:02:41.343688+0000","last_clean":"2023-12-20T06:02:41.343688+0000","last_became_active":"2023-12-20T06:02:39.474043+0000","last_became_peered":"2023-12-20T06:02:39.474043+0000","last_unstale":"2023-12-20T06:02:41.343688+0000","last_undegraded":"2023-12-20T06:02:41.343688+0000","last_fullsized":"2023-12-20T06:02:41.343688+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:40:28.433800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"24.1e","version":"208'1","reported_seq":16,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464049+0000","last_change":"2023-12-20T06:02:26.547864+0000","last_active":"2023-12-20T06:02:39.464049+0000","last_peered":"2023-12-20T06:02:39.464049+0000","last_clean":"2023-12-20T06:02:39.464049+0000","last_became_active":"2023-12-20T06:02:26.547556+0000","last_became_peered":"2023-12-20T06:02:26.547556+0000","last_unstale":"2023-12-20T06:02:39.464049+0000","last_undegraded":"2023-12-20T06:02:39.464049+0000","last_fullsized":"2023-12-20T06:02:39.464049+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:04.689001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.1f","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.456356+0000","last_change":"2023-12-20T06:02:26.978246+0000","last_active":"2023-12-20T06:02:39.456356+0000","last_peered":"2023-12-20T06:02:39.456356+0000","last_clean":"2023-12-20T06:02:39.456356+0000","last_became_active":"2023-12-20T06:02:26.977975+0000","last_became_peered":"2023-12-20T06:02:26.977975+0000","last_unstale":"2023-12-20T06:02:39.456356+0000","last_undegraded":"2023-12-20T06:02:39.456356+0000","last_fullsized":"2023-12-20T06:02:39.456356+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:05:43.763955+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"26.1c","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.247315+0000","last_change":"2023-12-20T06:02:39.068719+0000","last_active":"2023-12-20T06:02:41.247315+0000","last_peered":"2023-12-20T06:02:41.247315+0000","last_clean":"2023-12-20T06:02:41.247315+0000","last_became_active":"2023-12-20T06:02:39.068439+0000","last_became_peered":"2023-12-20T06:02:39.068439+0000","last_unstale":"2023-12-20T06:02:41.247315+0000","last_undegraded":"2023-12-20T06:02:41.247315+0000","last_fullsized":"2023-12-20T06:02:41.247315+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:03:56.778527+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.1c","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454643+0000","last_change":"2023-12-20T06:02:39.471635+0000","last_active":"2023-12-20T06:02:40.454643+0000","last_peered":"2023-12-20T06:02:40.454643+0000","last_clean":"2023-12-20T06:02:40.454643+0000","last_became_active":"2023-12-20T06:02:39.471384+0000","last_became_peered":"2023-12-20T06:02:39.471384+0000","last_unstale":"2023-12-20T06:02:40.454643+0000","last_undegraded":"2023-12-20T06:02:40.454643+0000","last_fullsized":"2023-12-20T06:02:40.454643+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:03.707844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"24.1f","version":"208'4","reported_seq":19,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.468962+0000","last_change":"2023-12-20T06:02:26.559340+0000","last_active":"2023-12-20T06:02:39.468962+0000","last_peered":"2023-12-20T06:02:39.468962+0000","last_clean":"2023-12-20T06:02:39.468962+0000","last_became_active":"2023-12-20T06:02:26.559045+0000","last_became_peered":"2023-12-20T06:02:26.559045+0000","last_unstale":"2023-12-20T06:02:39.468962+0000","last_undegraded":"2023-12-20T06:02:39.468962+0000","last_fullsized":"2023-12-20T06:02:39.468962+0000","mapping_epoch":206,"log_start":"0'0","ondisk_log_start":"0'0","created":206,"last_epoch_clean":207,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:25.521263+0000","last_clean_scrub_stamp":"2023-12-20T06:02:25.521263+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:33.930072+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.1e","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.468546+0000","last_change":"2023-12-20T06:02:26.966538+0000","last_active":"2023-12-20T06:02:39.468546+0000","last_peered":"2023-12-20T06:02:39.468546+0000","last_clean":"2023-12-20T06:02:39.468546+0000","last_became_active":"2023-12-20T06:02:26.966278+0000","last_became_peered":"2023-12-20T06:02:26.966278+0000","last_unstale":"2023-12-20T06:02:39.468546+0000","last_undegraded":"2023-12-20T06:02:39.468546+0000","last_fullsized":"2023-12-20T06:02:39.468546+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:12.114643+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"26.1d","version":"212'2","reported_seq":13,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342867+0000","last_change":"2023-12-20T06:02:39.066219+0000","last_active":"2023-12-20T06:02:41.342867+0000","last_peered":"2023-12-20T06:02:41.342867+0000","last_clean":"2023-12-20T06:02:41.342867+0000","last_became_active":"2023-12-20T06:02:39.065906+0000","last_became_peered":"2023-12-20T06:02:39.065906+0000","last_unstale":"2023-12-20T06:02:41.342867+0000","last_undegraded":"2023-12-20T06:02:41.342867+0000","last_fullsized":"2023-12-20T06:02:41.342867+0000","mapping_epoch":210,"log_start":"0'0","ondisk_log_start":"0'0","created":210,"last_epoch_clean":211,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:38.006508+0000","last_clean_scrub_stamp":"2023-12-20T06:02:38.006508+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:05:06.470711+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.22","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457234+0000","last_change":"2023-12-20T06:02:39.479009+0000","last_active":"2023-12-20T06:02:40.457234+0000","last_peered":"2023-12-20T06:02:40.457234+0000","last_clean":"2023-12-20T06:02:40.457234+0000","last_became_active":"2023-12-20T06:02:39.478661+0000","last_became_peered":"2023-12-20T06:02:39.478661+0000","last_unstale":"2023-12-20T06:02:40.457234+0000","last_undegraded":"2023-12-20T06:02:40.457234+0000","last_fullsized":"2023-12-20T06:02:40.457234+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:24.610708+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.20","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464761+0000","last_change":"2023-12-20T06:02:26.963846+0000","last_active":"2023-12-20T06:02:39.464761+0000","last_peered":"2023-12-20T06:02:39.464761+0000","last_clean":"2023-12-20T06:02:39.464761+0000","last_became_active":"2023-12-20T06:02:26.963552+0000","last_became_peered":"2023-12-20T06:02:26.963552+0000","last_unstale":"2023-12-20T06:02:39.464761+0000","last_undegraded":"2023-12-20T06:02:39.464761+0000","last_fullsized":"2023-12-20T06:02:39.464761+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:17:08.792842+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.23","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457537+0000","last_change":"2023-12-20T06:02:39.472176+0000","last_active":"2023-12-20T06:02:40.457537+0000","last_peered":"2023-12-20T06:02:40.457537+0000","last_clean":"2023-12-20T06:02:40.457537+0000","last_became_active":"2023-12-20T06:02:39.471883+0000","last_became_peered":"2023-12-20T06:02:39.471883+0000","last_unstale":"2023-12-20T06:02:40.457537+0000","last_undegraded":"2023-12-20T06:02:40.457537+0000","last_fullsized":"2023-12-20T06:02:40.457537+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:59.177389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.21","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.460941+0000","last_change":"2023-12-20T06:02:26.965576+0000","last_active":"2023-12-20T06:02:39.460941+0000","last_peered":"2023-12-20T06:02:39.460941+0000","last_clean":"2023-12-20T06:02:39.460941+0000","last_became_active":"2023-12-20T06:02:26.965189+0000","last_became_peered":"2023-12-20T06:02:26.965189+0000","last_unstale":"2023-12-20T06:02:39.460941+0000","last_undegraded":"2023-12-20T06:02:39.460941+0000","last_fullsized":"2023-12-20T06:02:39.460941+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:17:38.051872+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.20","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454119+0000","last_change":"2023-12-20T06:02:39.466101+0000","last_active":"2023-12-20T06:02:40.454119+0000","last_peered":"2023-12-20T06:02:40.454119+0000","last_clean":"2023-12-20T06:02:40.454119+0000","last_became_active":"2023-12-20T06:02:39.465827+0000","last_became_peered":"2023-12-20T06:02:39.465827+0000","last_unstale":"2023-12-20T06:02:40.454119+0000","last_undegraded":"2023-12-20T06:02:40.454119+0000","last_fullsized":"2023-12-20T06:02:40.454119+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:20:35.645018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.22","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464768+0000","last_change":"2023-12-20T06:02:26.977346+0000","last_active":"2023-12-20T06:02:39.464768+0000","last_peered":"2023-12-20T06:02:39.464768+0000","last_clean":"2023-12-20T06:02:39.464768+0000","last_became_active":"2023-12-20T06:02:26.976527+0000","last_became_peered":"2023-12-20T06:02:26.976527+0000","last_unstale":"2023-12-20T06:02:39.464768+0000","last_undegraded":"2023-12-20T06:02:39.464768+0000","last_fullsized":"2023-12-20T06:02:39.464768+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:11.127223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.21","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341625+0000","last_change":"2023-12-20T06:02:39.473998+0000","last_active":"2023-12-20T06:02:41.341625+0000","last_peered":"2023-12-20T06:02:41.341625+0000","last_clean":"2023-12-20T06:02:41.341625+0000","last_became_active":"2023-12-20T06:02:39.473011+0000","last_became_peered":"2023-12-20T06:02:39.473011+0000","last_unstale":"2023-12-20T06:02:41.341625+0000","last_undegraded":"2023-12-20T06:02:41.341625+0000","last_fullsized":"2023-12-20T06:02:41.341625+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:57:42.952159+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.23","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452445+0000","last_change":"2023-12-20T06:02:26.973161+0000","last_active":"2023-12-20T06:02:39.452445+0000","last_peered":"2023-12-20T06:02:39.452445+0000","last_clean":"2023-12-20T06:02:39.452445+0000","last_became_active":"2023-12-20T06:02:26.972883+0000","last_became_peered":"2023-12-20T06:02:26.972883+0000","last_unstale":"2023-12-20T06:02:39.452445+0000","last_undegraded":"2023-12-20T06:02:39.452445+0000","last_fullsized":"2023-12-20T06:02:39.452445+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:56:11.127223+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.26","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.455126+0000","last_change":"2023-12-20T06:02:39.482573+0000","last_active":"2023-12-20T06:02:40.455126+0000","last_peered":"2023-12-20T06:02:40.455126+0000","last_clean":"2023-12-20T06:02:40.455126+0000","last_became_active":"2023-12-20T06:02:39.482213+0000","last_became_peered":"2023-12-20T06:02:39.482213+0000","last_unstale":"2023-12-20T06:02:40.455126+0000","last_undegraded":"2023-12-20T06:02:40.455126+0000","last_fullsized":"2023-12-20T06:02:40.455126+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:30.717618+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.24","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465106+0000","last_change":"2023-12-20T06:02:26.977513+0000","last_active":"2023-12-20T06:02:39.465106+0000","last_peered":"2023-12-20T06:02:39.465106+0000","last_clean":"2023-12-20T06:02:39.465106+0000","last_became_active":"2023-12-20T06:02:26.977220+0000","last_became_peered":"2023-12-20T06:02:26.977220+0000","last_unstale":"2023-12-20T06:02:39.465106+0000","last_undegraded":"2023-12-20T06:02:39.465106+0000","last_fullsized":"2023-12-20T06:02:39.465106+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:53:52.600966+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.27","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458787+0000","last_change":"2023-12-20T06:02:39.477148+0000","last_active":"2023-12-20T06:02:40.458787+0000","last_peered":"2023-12-20T06:02:40.458787+0000","last_clean":"2023-12-20T06:02:40.458787+0000","last_became_active":"2023-12-20T06:02:39.476654+0000","last_became_peered":"2023-12-20T06:02:39.476654+0000","last_unstale":"2023-12-20T06:02:40.458787+0000","last_undegraded":"2023-12-20T06:02:40.458787+0000","last_fullsized":"2023-12-20T06:02:40.458787+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:20:37.087626+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.25","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467349+0000","last_change":"2023-12-20T06:02:26.981676+0000","last_active":"2023-12-20T06:02:39.467349+0000","last_peered":"2023-12-20T06:02:39.467349+0000","last_clean":"2023-12-20T06:02:39.467349+0000","last_became_active":"2023-12-20T06:02:26.980905+0000","last_became_peered":"2023-12-20T06:02:26.980905+0000","last_unstale":"2023-12-20T06:02:39.467349+0000","last_undegraded":"2023-12-20T06:02:39.467349+0000","last_fullsized":"2023-12-20T06:02:39.467349+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:43:42.660130+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.24","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342532+0000","last_change":"2023-12-20T06:02:39.482998+0000","last_active":"2023-12-20T06:02:41.342532+0000","last_peered":"2023-12-20T06:02:41.342532+0000","last_clean":"2023-12-20T06:02:41.342532+0000","last_became_active":"2023-12-20T06:02:39.482499+0000","last_became_peered":"2023-12-20T06:02:39.482499+0000","last_unstale":"2023-12-20T06:02:41.342532+0000","last_undegraded":"2023-12-20T06:02:41.342532+0000","last_fullsized":"2023-12-20T06:02:41.342532+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:48:09.959849+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.26","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.466629+0000","last_change":"2023-12-20T06:02:26.979767+0000","last_active":"2023-12-20T06:02:39.466629+0000","last_peered":"2023-12-20T06:02:39.466629+0000","last_clean":"2023-12-20T06:02:39.466629+0000","last_became_active":"2023-12-20T06:02:26.979450+0000","last_became_peered":"2023-12-20T06:02:26.979450+0000","last_unstale":"2023-12-20T06:02:39.466629+0000","last_undegraded":"2023-12-20T06:02:39.466629+0000","last_fullsized":"2023-12-20T06:02:39.466629+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:47:57.463784+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.25","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458283+0000","last_change":"2023-12-20T06:02:39.472365+0000","last_active":"2023-12-20T06:02:40.458283+0000","last_peered":"2023-12-20T06:02:40.458283+0000","last_clean":"2023-12-20T06:02:40.458283+0000","last_became_active":"2023-12-20T06:02:39.468354+0000","last_became_peered":"2023-12-20T06:02:39.468354+0000","last_unstale":"2023-12-20T06:02:40.458283+0000","last_undegraded":"2023-12-20T06:02:40.458283+0000","last_fullsized":"2023-12-20T06:02:40.458283+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:21.530775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.27","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453535+0000","last_change":"2023-12-20T06:02:26.976267+0000","last_active":"2023-12-20T06:02:39.453535+0000","last_peered":"2023-12-20T06:02:39.453535+0000","last_clean":"2023-12-20T06:02:39.453535+0000","last_became_active":"2023-12-20T06:02:26.975927+0000","last_became_peered":"2023-12-20T06:02:26.975927+0000","last_unstale":"2023-12-20T06:02:39.453535+0000","last_undegraded":"2023-12-20T06:02:39.453535+0000","last_fullsized":"2023-12-20T06:02:39.453535+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:19:37.257026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.2a","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338301+0000","last_change":"2023-12-20T06:02:39.476377+0000","last_active":"2023-12-20T06:02:41.338301+0000","last_peered":"2023-12-20T06:02:41.338301+0000","last_clean":"2023-12-20T06:02:41.338301+0000","last_became_active":"2023-12-20T06:02:39.476094+0000","last_became_peered":"2023-12-20T06:02:39.476094+0000","last_unstale":"2023-12-20T06:02:41.338301+0000","last_undegraded":"2023-12-20T06:02:41.338301+0000","last_fullsized":"2023-12-20T06:02:41.338301+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:23:41.894945+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.28","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464888+0000","last_change":"2023-12-20T06:02:26.965794+0000","last_active":"2023-12-20T06:02:39.464888+0000","last_peered":"2023-12-20T06:02:39.464888+0000","last_clean":"2023-12-20T06:02:39.464888+0000","last_became_active":"2023-12-20T06:02:26.965505+0000","last_became_peered":"2023-12-20T06:02:26.965505+0000","last_unstale":"2023-12-20T06:02:39.464888+0000","last_undegraded":"2023-12-20T06:02:39.464888+0000","last_fullsized":"2023-12-20T06:02:39.464888+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:09.034710+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.2b","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246668+0000","last_change":"2023-12-20T06:02:39.486650+0000","last_active":"2023-12-20T06:02:41.246668+0000","last_peered":"2023-12-20T06:02:41.246668+0000","last_clean":"2023-12-20T06:02:41.246668+0000","last_became_active":"2023-12-20T06:02:39.486371+0000","last_became_peered":"2023-12-20T06:02:39.486371+0000","last_unstale":"2023-12-20T06:02:41.246668+0000","last_undegraded":"2023-12-20T06:02:41.246668+0000","last_fullsized":"2023-12-20T06:02:41.246668+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:48:12.342123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.29","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.460864+0000","last_change":"2023-12-20T06:02:26.979485+0000","last_active":"2023-12-20T06:02:39.460864+0000","last_peered":"2023-12-20T06:02:39.460864+0000","last_clean":"2023-12-20T06:02:39.460864+0000","last_became_active":"2023-12-20T06:02:26.979208+0000","last_became_peered":"2023-12-20T06:02:26.979208+0000","last_unstale":"2023-12-20T06:02:39.460864+0000","last_undegraded":"2023-12-20T06:02:39.460864+0000","last_fullsized":"2023-12-20T06:02:39.460864+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:04:09.949376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.28","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246538+0000","last_change":"2023-12-20T06:02:39.486665+0000","last_active":"2023-12-20T06:02:41.246538+0000","last_peered":"2023-12-20T06:02:41.246538+0000","last_clean":"2023-12-20T06:02:41.246538+0000","last_became_active":"2023-12-20T06:02:39.486378+0000","last_became_peered":"2023-12-20T06:02:39.486378+0000","last_unstale":"2023-12-20T06:02:41.246538+0000","last_undegraded":"2023-12-20T06:02:41.246538+0000","last_fullsized":"2023-12-20T06:02:41.246538+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:13:25.985704+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.2a","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453188+0000","last_change":"2023-12-20T06:02:26.975847+0000","last_active":"2023-12-20T06:02:39.453188+0000","last_peered":"2023-12-20T06:02:39.453188+0000","last_clean":"2023-12-20T06:02:39.453188+0000","last_became_active":"2023-12-20T06:02:26.975128+0000","last_became_peered":"2023-12-20T06:02:26.975128+0000","last_unstale":"2023-12-20T06:02:39.453188+0000","last_undegraded":"2023-12-20T06:02:39.453188+0000","last_fullsized":"2023-12-20T06:02:39.453188+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:39:21.684738+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.29","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.338200+0000","last_change":"2023-12-20T06:02:39.472707+0000","last_active":"2023-12-20T06:02:41.338200+0000","last_peered":"2023-12-20T06:02:41.338200+0000","last_clean":"2023-12-20T06:02:41.338200+0000","last_became_active":"2023-12-20T06:02:39.472409+0000","last_became_peered":"2023-12-20T06:02:39.472409+0000","last_unstale":"2023-12-20T06:02:41.338200+0000","last_undegraded":"2023-12-20T06:02:41.338200+0000","last_fullsized":"2023-12-20T06:02:41.338200+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:36:12.275613+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"25.2b","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464662+0000","last_change":"2023-12-20T06:02:26.969431+0000","last_active":"2023-12-20T06:02:39.464662+0000","last_peered":"2023-12-20T06:02:39.464662+0000","last_clean":"2023-12-20T06:02:39.464662+0000","last_became_active":"2023-12-20T06:02:26.969161+0000","last_became_peered":"2023-12-20T06:02:26.969161+0000","last_unstale":"2023-12-20T06:02:39.464662+0000","last_undegraded":"2023-12-20T06:02:39.464662+0000","last_fullsized":"2023-12-20T06:02:39.464662+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:49:57.166700+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2e","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343953+0000","last_change":"2023-12-20T06:02:39.472245+0000","last_active":"2023-12-20T06:02:41.343953+0000","last_peered":"2023-12-20T06:02:41.343953+0000","last_clean":"2023-12-20T06:02:41.343953+0000","last_became_active":"2023-12-20T06:02:39.471954+0000","last_became_peered":"2023-12-20T06:02:39.471954+0000","last_unstale":"2023-12-20T06:02:41.343953+0000","last_undegraded":"2023-12-20T06:02:41.343953+0000","last_fullsized":"2023-12-20T06:02:41.343953+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:07:55.492672+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2c","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464936+0000","last_change":"2023-12-20T06:02:26.957746+0000","last_active":"2023-12-20T06:02:39.464936+0000","last_peered":"2023-12-20T06:02:39.464936+0000","last_clean":"2023-12-20T06:02:39.464936+0000","last_became_active":"2023-12-20T06:02:26.957407+0000","last_became_peered":"2023-12-20T06:02:26.957407+0000","last_unstale":"2023-12-20T06:02:39.464936+0000","last_undegraded":"2023-12-20T06:02:39.464936+0000","last_fullsized":"2023-12-20T06:02:39.464936+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:51.962994+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.2f","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.455202+0000","last_change":"2023-12-20T06:02:39.476908+0000","last_active":"2023-12-20T06:02:40.455202+0000","last_peered":"2023-12-20T06:02:40.455202+0000","last_clean":"2023-12-20T06:02:40.455202+0000","last_became_active":"2023-12-20T06:02:39.476633+0000","last_became_peered":"2023-12-20T06:02:39.476633+0000","last_unstale":"2023-12-20T06:02:40.455202+0000","last_undegraded":"2023-12-20T06:02:40.455202+0000","last_fullsized":"2023-12-20T06:02:40.455202+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:36:47.240370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.2d","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.455442+0000","last_change":"2023-12-20T06:02:26.972124+0000","last_active":"2023-12-20T06:02:39.455442+0000","last_peered":"2023-12-20T06:02:39.455442+0000","last_clean":"2023-12-20T06:02:39.455442+0000","last_became_active":"2023-12-20T06:02:26.971870+0000","last_became_peered":"2023-12-20T06:02:26.971870+0000","last_unstale":"2023-12-20T06:02:39.455442+0000","last_undegraded":"2023-12-20T06:02:39.455442+0000","last_fullsized":"2023-12-20T06:02:39.455442+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:06:44.203571+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.2c","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343934+0000","last_change":"2023-12-20T06:02:39.475322+0000","last_active":"2023-12-20T06:02:41.343934+0000","last_peered":"2023-12-20T06:02:41.343934+0000","last_clean":"2023-12-20T06:02:41.343934+0000","last_became_active":"2023-12-20T06:02:39.474906+0000","last_became_peered":"2023-12-20T06:02:39.474906+0000","last_unstale":"2023-12-20T06:02:41.343934+0000","last_undegraded":"2023-12-20T06:02:41.343934+0000","last_fullsized":"2023-12-20T06:02:41.343934+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:21.530775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.2e","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467816+0000","last_change":"2023-12-20T06:02:26.981004+0000","last_active":"2023-12-20T06:02:39.467816+0000","last_peered":"2023-12-20T06:02:39.467816+0000","last_clean":"2023-12-20T06:02:39.467816+0000","last_became_active":"2023-12-20T06:02:26.980597+0000","last_became_peered":"2023-12-20T06:02:26.980597+0000","last_unstale":"2023-12-20T06:02:39.467816+0000","last_undegraded":"2023-12-20T06:02:39.467816+0000","last_fullsized":"2023-12-20T06:02:39.467816+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:10:13.705554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"27.2d","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.344045+0000","last_change":"2023-12-20T06:02:39.475270+0000","last_active":"2023-12-20T06:02:41.344045+0000","last_peered":"2023-12-20T06:02:41.344045+0000","last_clean":"2023-12-20T06:02:41.344045+0000","last_became_active":"2023-12-20T06:02:39.472631+0000","last_became_peered":"2023-12-20T06:02:39.472631+0000","last_unstale":"2023-12-20T06:02:41.344045+0000","last_undegraded":"2023-12-20T06:02:41.344045+0000","last_fullsized":"2023-12-20T06:02:41.344045+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:15:43.615696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.2f","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.465373+0000","last_change":"2023-12-20T06:02:26.959816+0000","last_active":"2023-12-20T06:02:39.465373+0000","last_peered":"2023-12-20T06:02:39.465373+0000","last_clean":"2023-12-20T06:02:39.465373+0000","last_became_active":"2023-12-20T06:02:26.959542+0000","last_became_peered":"2023-12-20T06:02:26.959542+0000","last_unstale":"2023-12-20T06:02:39.465373+0000","last_undegraded":"2023-12-20T06:02:39.465373+0000","last_fullsized":"2023-12-20T06:02:39.465373+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:49:15.081247+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.32","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454373+0000","last_change":"2023-12-20T06:02:39.459375+0000","last_active":"2023-12-20T06:02:40.454373+0000","last_peered":"2023-12-20T06:02:40.454373+0000","last_clean":"2023-12-20T06:02:40.454373+0000","last_became_active":"2023-12-20T06:02:39.459059+0000","last_became_peered":"2023-12-20T06:02:39.459059+0000","last_unstale":"2023-12-20T06:02:40.454373+0000","last_undegraded":"2023-12-20T06:02:40.454373+0000","last_fullsized":"2023-12-20T06:02:40.454373+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:40:30.277423+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.30","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464885+0000","last_change":"2023-12-20T06:02:26.966987+0000","last_active":"2023-12-20T06:02:39.464885+0000","last_peered":"2023-12-20T06:02:39.464885+0000","last_clean":"2023-12-20T06:02:39.464885+0000","last_became_active":"2023-12-20T06:02:26.966704+0000","last_became_peered":"2023-12-20T06:02:26.966704+0000","last_unstale":"2023-12-20T06:02:39.464885+0000","last_undegraded":"2023-12-20T06:02:39.464885+0000","last_fullsized":"2023-12-20T06:02:39.464885+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:41:37.729575+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.33","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.457770+0000","last_change":"2023-12-20T06:02:39.461702+0000","last_active":"2023-12-20T06:02:40.457770+0000","last_peered":"2023-12-20T06:02:40.457770+0000","last_clean":"2023-12-20T06:02:40.457770+0000","last_became_active":"2023-12-20T06:02:39.461185+0000","last_became_peered":"2023-12-20T06:02:39.461185+0000","last_unstale":"2023-12-20T06:02:40.457770+0000","last_undegraded":"2023-12-20T06:02:40.457770+0000","last_fullsized":"2023-12-20T06:02:40.457770+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:38:05.770473+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.31","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452377+0000","last_change":"2023-12-20T06:02:26.974037+0000","last_active":"2023-12-20T06:02:39.452377+0000","last_peered":"2023-12-20T06:02:39.452377+0000","last_clean":"2023-12-20T06:02:39.452377+0000","last_became_active":"2023-12-20T06:02:26.973742+0000","last_became_peered":"2023-12-20T06:02:26.973742+0000","last_unstale":"2023-12-20T06:02:39.452377+0000","last_undegraded":"2023-12-20T06:02:39.452377+0000","last_fullsized":"2023-12-20T06:02:39.452377+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:56:41.448149+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.30","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.341826+0000","last_change":"2023-12-20T06:02:39.478307+0000","last_active":"2023-12-20T06:02:41.341826+0000","last_peered":"2023-12-20T06:02:41.341826+0000","last_clean":"2023-12-20T06:02:41.341826+0000","last_became_active":"2023-12-20T06:02:39.478043+0000","last_became_peered":"2023-12-20T06:02:39.478043+0000","last_unstale":"2023-12-20T06:02:41.341826+0000","last_undegraded":"2023-12-20T06:02:41.341826+0000","last_fullsized":"2023-12-20T06:02:41.341826+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:44:56.585869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.32","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464687+0000","last_change":"2023-12-20T06:02:26.978685+0000","last_active":"2023-12-20T06:02:39.464687+0000","last_peered":"2023-12-20T06:02:39.464687+0000","last_clean":"2023-12-20T06:02:39.464687+0000","last_became_active":"2023-12-20T06:02:26.978364+0000","last_became_peered":"2023-12-20T06:02:26.978364+0000","last_unstale":"2023-12-20T06:02:39.464687+0000","last_undegraded":"2023-12-20T06:02:39.464687+0000","last_fullsized":"2023-12-20T06:02:39.464687+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:19:37.257026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.31","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454205+0000","last_change":"2023-12-20T06:02:39.471033+0000","last_active":"2023-12-20T06:02:40.454205+0000","last_peered":"2023-12-20T06:02:40.454205+0000","last_clean":"2023-12-20T06:02:40.454205+0000","last_became_active":"2023-12-20T06:02:39.470713+0000","last_became_peered":"2023-12-20T06:02:39.470713+0000","last_unstale":"2023-12-20T06:02:40.454205+0000","last_undegraded":"2023-12-20T06:02:40.454205+0000","last_fullsized":"2023-12-20T06:02:40.454205+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:16:31.234579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.33","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452352+0000","last_change":"2023-12-20T06:02:26.967177+0000","last_active":"2023-12-20T06:02:39.452352+0000","last_peered":"2023-12-20T06:02:39.452352+0000","last_clean":"2023-12-20T06:02:39.452352+0000","last_became_active":"2023-12-20T06:02:26.966560+0000","last_became_peered":"2023-12-20T06:02:26.966560+0000","last_unstale":"2023-12-20T06:02:39.452352+0000","last_undegraded":"2023-12-20T06:02:39.452352+0000","last_fullsized":"2023-12-20T06:02:39.452352+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:12:37.519537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.36","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343695+0000","last_change":"2023-12-20T06:02:39.472321+0000","last_active":"2023-12-20T06:02:41.343695+0000","last_peered":"2023-12-20T06:02:41.343695+0000","last_clean":"2023-12-20T06:02:41.343695+0000","last_became_active":"2023-12-20T06:02:39.471969+0000","last_became_peered":"2023-12-20T06:02:39.471969+0000","last_unstale":"2023-12-20T06:02:41.343695+0000","last_undegraded":"2023-12-20T06:02:41.343695+0000","last_fullsized":"2023-12-20T06:02:41.343695+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:25:59.524917+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"25.34","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452876+0000","last_change":"2023-12-20T06:02:26.976708+0000","last_active":"2023-12-20T06:02:39.452876+0000","last_peered":"2023-12-20T06:02:39.452876+0000","last_clean":"2023-12-20T06:02:39.452876+0000","last_became_active":"2023-12-20T06:02:26.976349+0000","last_became_peered":"2023-12-20T06:02:26.976349+0000","last_unstale":"2023-12-20T06:02:39.452876+0000","last_undegraded":"2023-12-20T06:02:39.452876+0000","last_fullsized":"2023-12-20T06:02:39.452876+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.37","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.459397+0000","last_change":"2023-12-20T06:02:39.482121+0000","last_active":"2023-12-20T06:02:40.459397+0000","last_peered":"2023-12-20T06:02:40.459397+0000","last_clean":"2023-12-20T06:02:40.459397+0000","last_became_active":"2023-12-20T06:02:39.481795+0000","last_became_peered":"2023-12-20T06:02:39.481795+0000","last_unstale":"2023-12-20T06:02:40.459397+0000","last_undegraded":"2023-12-20T06:02:40.459397+0000","last_fullsized":"2023-12-20T06:02:40.459397+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:58.188244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.35","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467588+0000","last_change":"2023-12-20T06:02:26.976628+0000","last_active":"2023-12-20T06:02:39.467588+0000","last_peered":"2023-12-20T06:02:39.467588+0000","last_clean":"2023-12-20T06:02:39.467588+0000","last_became_active":"2023-12-20T06:02:26.976337+0000","last_became_peered":"2023-12-20T06:02:26.976337+0000","last_unstale":"2023-12-20T06:02:39.467588+0000","last_undegraded":"2023-12-20T06:02:39.467588+0000","last_fullsized":"2023-12-20T06:02:39.467588+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:01:15.147652+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.34","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458033+0000","last_change":"2023-12-20T06:02:39.466735+0000","last_active":"2023-12-20T06:02:40.458033+0000","last_peered":"2023-12-20T06:02:40.458033+0000","last_clean":"2023-12-20T06:02:40.458033+0000","last_became_active":"2023-12-20T06:02:39.466439+0000","last_became_peered":"2023-12-20T06:02:39.466439+0000","last_unstale":"2023-12-20T06:02:40.458033+0000","last_undegraded":"2023-12-20T06:02:40.458033+0000","last_fullsized":"2023-12-20T06:02:40.458033+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:41:50.225640+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"25.36","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.454838+0000","last_change":"2023-12-20T06:02:26.973655+0000","last_active":"2023-12-20T06:02:39.454838+0000","last_peered":"2023-12-20T06:02:39.454838+0000","last_clean":"2023-12-20T06:02:39.454838+0000","last_became_active":"2023-12-20T06:02:26.973281+0000","last_became_peered":"2023-12-20T06:02:26.973281+0000","last_unstale":"2023-12-20T06:02:39.454838+0000","last_undegraded":"2023-12-20T06:02:39.454838+0000","last_fullsized":"2023-12-20T06:02:39.454838+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:36.523098+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"27.35","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342824+0000","last_change":"2023-12-20T06:02:39.481487+0000","last_active":"2023-12-20T06:02:41.342824+0000","last_peered":"2023-12-20T06:02:41.342824+0000","last_clean":"2023-12-20T06:02:41.342824+0000","last_became_active":"2023-12-20T06:02:39.481013+0000","last_became_peered":"2023-12-20T06:02:39.481013+0000","last_unstale":"2023-12-20T06:02:41.342824+0000","last_undegraded":"2023-12-20T06:02:41.342824+0000","last_fullsized":"2023-12-20T06:02:41.342824+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:03:22.032925+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.37","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464010+0000","last_change":"2023-12-20T06:02:26.960947+0000","last_active":"2023-12-20T06:02:39.464010+0000","last_peered":"2023-12-20T06:02:39.464010+0000","last_clean":"2023-12-20T06:02:39.464010+0000","last_became_active":"2023-12-20T06:02:26.958931+0000","last_became_peered":"2023-12-20T06:02:26.958931+0000","last_unstale":"2023-12-20T06:02:39.464010+0000","last_undegraded":"2023-12-20T06:02:39.464010+0000","last_fullsized":"2023-12-20T06:02:39.464010+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:54.993357+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"27.3a","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342785+0000","last_change":"2023-12-20T06:02:39.484029+0000","last_active":"2023-12-20T06:02:41.342785+0000","last_peered":"2023-12-20T06:02:41.342785+0000","last_clean":"2023-12-20T06:02:41.342785+0000","last_became_active":"2023-12-20T06:02:39.483768+0000","last_became_peered":"2023-12-20T06:02:39.483768+0000","last_unstale":"2023-12-20T06:02:41.342785+0000","last_undegraded":"2023-12-20T06:02:41.342785+0000","last_fullsized":"2023-12-20T06:02:41.342785+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:36:59.177389+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.38","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464784+0000","last_change":"2023-12-20T06:02:26.971111+0000","last_active":"2023-12-20T06:02:39.464784+0000","last_peered":"2023-12-20T06:02:39.464784+0000","last_clean":"2023-12-20T06:02:39.464784+0000","last_became_active":"2023-12-20T06:02:26.970783+0000","last_became_peered":"2023-12-20T06:02:26.970783+0000","last_unstale":"2023-12-20T06:02:39.464784+0000","last_undegraded":"2023-12-20T06:02:39.464784+0000","last_fullsized":"2023-12-20T06:02:39.464784+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:45.692179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"27.3b","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.343808+0000","last_change":"2023-12-20T06:02:39.479054+0000","last_active":"2023-12-20T06:02:41.343808+0000","last_peered":"2023-12-20T06:02:41.343808+0000","last_clean":"2023-12-20T06:02:41.343808+0000","last_became_active":"2023-12-20T06:02:39.477268+0000","last_became_peered":"2023-12-20T06:02:39.477268+0000","last_unstale":"2023-12-20T06:02:41.343808+0000","last_undegraded":"2023-12-20T06:02:41.343808+0000","last_fullsized":"2023-12-20T06:02:41.343808+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:35:40.405390+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.39","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.458405+0000","last_change":"2023-12-20T06:02:26.978795+0000","last_active":"2023-12-20T06:02:39.458405+0000","last_peered":"2023-12-20T06:02:39.458405+0000","last_clean":"2023-12-20T06:02:39.458405+0000","last_became_active":"2023-12-20T06:02:26.978497+0000","last_became_peered":"2023-12-20T06:02:26.978497+0000","last_unstale":"2023-12-20T06:02:39.458405+0000","last_undegraded":"2023-12-20T06:02:39.458405+0000","last_fullsized":"2023-12-20T06:02:39.458405+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:06:01.713370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"27.38","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342662+0000","last_change":"2023-12-20T06:02:39.468047+0000","last_active":"2023-12-20T06:02:41.342662+0000","last_peered":"2023-12-20T06:02:41.342662+0000","last_clean":"2023-12-20T06:02:41.342662+0000","last_became_active":"2023-12-20T06:02:39.467724+0000","last_became_peered":"2023-12-20T06:02:39.467724+0000","last_unstale":"2023-12-20T06:02:41.342662+0000","last_undegraded":"2023-12-20T06:02:41.342662+0000","last_fullsized":"2023-12-20T06:02:41.342662+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:47:58.188244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"25.3a","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.463986+0000","last_change":"2023-12-20T06:02:26.979634+0000","last_active":"2023-12-20T06:02:39.463986+0000","last_peered":"2023-12-20T06:02:39.463986+0000","last_clean":"2023-12-20T06:02:39.463986+0000","last_became_active":"2023-12-20T06:02:26.979366+0000","last_became_peered":"2023-12-20T06:02:26.979366+0000","last_unstale":"2023-12-20T06:02:39.463986+0000","last_undegraded":"2023-12-20T06:02:39.463986+0000","last_fullsized":"2023-12-20T06:02:39.463986+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:34:19.170698+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.39","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.454484+0000","last_change":"2023-12-20T06:02:39.477742+0000","last_active":"2023-12-20T06:02:40.454484+0000","last_peered":"2023-12-20T06:02:40.454484+0000","last_clean":"2023-12-20T06:02:40.454484+0000","last_became_active":"2023-12-20T06:02:39.477334+0000","last_became_peered":"2023-12-20T06:02:39.477334+0000","last_unstale":"2023-12-20T06:02:40.454484+0000","last_undegraded":"2023-12-20T06:02:40.454484+0000","last_fullsized":"2023-12-20T06:02:40.454484+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:06:54.630645+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3b","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.464660+0000","last_change":"2023-12-20T06:02:26.962506+0000","last_active":"2023-12-20T06:02:39.464660+0000","last_peered":"2023-12-20T06:02:39.464660+0000","last_clean":"2023-12-20T06:02:39.464660+0000","last_became_active":"2023-12-20T06:02:26.962262+0000","last_became_peered":"2023-12-20T06:02:26.962262+0000","last_unstale":"2023-12-20T06:02:39.464660+0000","last_undegraded":"2023-12-20T06:02:39.464660+0000","last_fullsized":"2023-12-20T06:02:39.464660+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:09:09.108347+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"27.3e","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.458853+0000","last_change":"2023-12-20T06:02:39.480339+0000","last_active":"2023-12-20T06:02:40.458853+0000","last_peered":"2023-12-20T06:02:40.458853+0000","last_clean":"2023-12-20T06:02:40.458853+0000","last_became_active":"2023-12-20T06:02:39.480039+0000","last_became_peered":"2023-12-20T06:02:39.480039+0000","last_unstale":"2023-12-20T06:02:40.458853+0000","last_undegraded":"2023-12-20T06:02:40.458853+0000","last_fullsized":"2023-12-20T06:02:40.458853+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:52:40.939793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"25.3c","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.452796+0000","last_change":"2023-12-20T06:02:26.955353+0000","last_active":"2023-12-20T06:02:39.452796+0000","last_peered":"2023-12-20T06:02:39.452796+0000","last_clean":"2023-12-20T06:02:39.452796+0000","last_became_active":"2023-12-20T06:02:26.954910+0000","last_became_peered":"2023-12-20T06:02:26.954910+0000","last_unstale":"2023-12-20T06:02:39.452796+0000","last_undegraded":"2023-12-20T06:02:39.452796+0000","last_fullsized":"2023-12-20T06:02:39.452796+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:44:12.085696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3f","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:40.455467+0000","last_change":"2023-12-20T06:02:39.474033+0000","last_active":"2023-12-20T06:02:40.455467+0000","last_peered":"2023-12-20T06:02:40.455467+0000","last_clean":"2023-12-20T06:02:40.455467+0000","last_became_active":"2023-12-20T06:02:39.473755+0000","last_became_peered":"2023-12-20T06:02:39.473755+0000","last_unstale":"2023-12-20T06:02:40.455467+0000","last_undegraded":"2023-12-20T06:02:40.455467+0000","last_fullsized":"2023-12-20T06:02:40.455467+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:09:50.926085+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"25.3d","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.467315+0000","last_change":"2023-12-20T06:02:26.968444+0000","last_active":"2023-12-20T06:02:39.467315+0000","last_peered":"2023-12-20T06:02:39.467315+0000","last_clean":"2023-12-20T06:02:39.467315+0000","last_became_active":"2023-12-20T06:02:26.967931+0000","last_became_peered":"2023-12-20T06:02:26.967931+0000","last_unstale":"2023-12-20T06:02:39.467315+0000","last_undegraded":"2023-12-20T06:02:39.467315+0000","last_fullsized":"2023-12-20T06:02:39.467315+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:33:14.150434+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"27.3c","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.246851+0000","last_change":"2023-12-20T06:02:39.480502+0000","last_active":"2023-12-20T06:02:41.246851+0000","last_peered":"2023-12-20T06:02:41.246851+0000","last_clean":"2023-12-20T06:02:41.246851+0000","last_became_active":"2023-12-20T06:02:39.480217+0000","last_became_peered":"2023-12-20T06:02:39.480217+0000","last_unstale":"2023-12-20T06:02:41.246851+0000","last_undegraded":"2023-12-20T06:02:41.246851+0000","last_fullsized":"2023-12-20T06:02:41.246851+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:35:16.506386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"25.3e","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.453354+0000","last_change":"2023-12-20T06:02:26.956043+0000","last_active":"2023-12-20T06:02:39.453354+0000","last_peered":"2023-12-20T06:02:39.453354+0000","last_clean":"2023-12-20T06:02:39.453354+0000","last_became_active":"2023-12-20T06:02:26.955652+0000","last_became_peered":"2023-12-20T06:02:26.955652+0000","last_unstale":"2023-12-20T06:02:39.453354+0000","last_undegraded":"2023-12-20T06:02:39.453354+0000","last_fullsized":"2023-12-20T06:02:39.453354+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:02:53.706417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"27.3d","version":"0'0","reported_seq":10,"reported_epoch":213,"state":"active+clean","last_fresh":"2023-12-20T06:02:41.342688+0000","last_change":"2023-12-20T06:02:39.484997+0000","last_active":"2023-12-20T06:02:41.342688+0000","last_peered":"2023-12-20T06:02:41.342688+0000","last_clean":"2023-12-20T06:02:41.342688+0000","last_became_active":"2023-12-20T06:02:39.484052+0000","last_became_peered":"2023-12-20T06:02:39.484052+0000","last_unstale":"2023-12-20T06:02:41.342688+0000","last_undegraded":"2023-12-20T06:02:41.342688+0000","last_fullsized":"2023-12-20T06:02:41.342688+0000","mapping_epoch":211,"log_start":"0'0","ondisk_log_start":"0'0","created":211,"last_epoch_clean":212,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:39.025218+0000","last_clean_scrub_stamp":"2023-12-20T06:02:39.025218+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:00:04.448601+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"25.3f","version":"0'0","reported_seq":14,"reported_epoch":212,"state":"active+clean","last_fresh":"2023-12-20T06:02:39.468006+0000","last_change":"2023-12-20T06:02:26.982741+0000","last_active":"2023-12-20T06:02:39.468006+0000","last_peered":"2023-12-20T06:02:39.468006+0000","last_clean":"2023-12-20T06:02:39.468006+0000","last_became_active":"2023-12-20T06:02:26.982454+0000","last_became_peered":"2023-12-20T06:02:26.982454+0000","last_unstale":"2023-12-20T06:02:39.468006+0000","last_undegraded":"2023-12-20T06:02:39.468006+0000","last_fullsized":"2023-12-20T06:02:39.468006+0000","mapping_epoch":207,"log_start":"0'0","ondisk_log_start":"0'0","created":207,"last_epoch_clean":208,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:02:26.529154+0000","last_clean_scrub_stamp":"2023-12-20T06:02:26.529154+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:59:08.944850+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":26,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":39,"ondisk_log_size":39,"up":64,"acting":64,"num_store_stats":8},{"poolid":25,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":24,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":27,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1885,"num_read_kb":3849,"num_write":521,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3201,"ondisk_log_size":3201,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575973,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56396,"kb_used_data":3204,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315444,"statfs":{"total":96636764160,"available":96579014656,"internally_reserved":0,"allocated":3280896,"data_stored":1890316,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.72199999999999998,"15min":0.73499999999999999},"min":{"1min":0.497,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":1.028,"5min":1.028,"15min":1.042},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.71899999999999997,"5min":0.72599999999999998,"15min":0.74199999999999999},"min":{"1min":0.53100000000000003,"5min":0.432,"15min":0.34499999999999997},"max":{"1min":0.874,"5min":0.99099999999999999,"15min":1.161},"last":0.504}]},{"osd":1,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.71899999999999997,"5min":0.73199999999999998,"15min":0.751},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.432},"max":{"1min":0.88700000000000001,"5min":0.996,"15min":1.1279999999999999},"last":0.433},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.73399999999999999,"15min":0.753},"min":{"1min":0.53100000000000003,"5min":0.51100000000000001,"15min":0.46999999999999997},"max":{"1min":0.90900000000000003,"5min":0.96699999999999997,"15min":1.032},"last":0.57499999999999996}]},{"osd":2,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.72299999999999998,"15min":0.748},"min":{"1min":0.51900000000000002,"5min":0.48699999999999999,"15min":0.48299999999999998},"max":{"1min":0.95399999999999996,"5min":0.95399999999999996,"15min":1.0409999999999999},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.753,"15min":0.76600000000000001},"min":{"1min":0.56999999999999995,"5min":0.49099999999999999,"15min":0.49099999999999999},"max":{"1min":0.96999999999999997,"5min":1.0189999999999999,"15min":1.038},"last":0.55600000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.77500000000000002,"5min":0.72899999999999998,"15min":0.746},"min":{"1min":0.61899999999999999,"5min":0.442,"15min":0.442},"max":{"1min":1.0089999999999999,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70799999999999996},{"interface":"front","average":{"1min":0.78400000000000003,"5min":0.77900000000000003,"15min":0.79000000000000004},"min":{"1min":0.53800000000000003,"5min":0.503,"15min":0.40699999999999997},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0840000000000001},"last":0.66200000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.59499999999999997,"5min":0.60599999999999998,"15min":0.59599999999999997},"min":{"1min":0.439,"5min":0.39800000000000002,"15min":0.39800000000000002},"max":{"1min":0.77600000000000002,"5min":0.91800000000000004,"15min":1.04},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.65900000000000003,"15min":0.65000000000000002},"min":{"1min":0.41999999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.91800000000000004,"5min":1.02,"15min":1.02},"last":0.63800000000000001}]},{"osd":5,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.62,"5min":0.67200000000000004,"15min":0.66500000000000004},"min":{"1min":0.39900000000000002,"5min":0.39900000000000002,"15min":0.39900000000000002},"max":{"1min":0.86699999999999999,"5min":0.97499999999999998,"15min":1.1120000000000001},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.67800000000000005,"5min":0.69799999999999995,"15min":0.69299999999999995},"min":{"1min":0.54300000000000004,"5min":0.47199999999999998,"15min":0.45600000000000002},"max":{"1min":0.84199999999999997,"5min":1.0229999999999999,"15min":1.103},"last":0.60199999999999998}]},{"osd":6,"last update":"Wed Dec 20 06:02:20 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.76500000000000001,"15min":0.751},"min":{"1min":0.57399999999999995,"5min":0.54400000000000004,"15min":0.45500000000000002},"max":{"1min":0.94399999999999995,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.625},{"interface":"front","average":{"1min":0.73799999999999999,"5min":0.72299999999999998,"15min":0.72499999999999998},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.92300000000000004,"5min":1.0740000000000001,"15min":1.1539999999999999},"last":0.76000000000000001}]}]},{"osd":6,"up_from":13,"seq":55834575973,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51704,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320136,"statfs":{"total":96636764160,"available":96583819264,"internally_reserved":0,"allocated":2670592,"data_stored":1298862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71199999999999997,"15min":0.73999999999999999},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92400000000000004,"5min":1.016,"15min":1.0980000000000001},"last":0.312},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72999999999999998,"15min":0.75},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.006,"5min":1.016,"15min":1.05},"last":0.44}]},{"osd":1,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.47399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90100000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.624},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91200000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.371}]},{"osd":2,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76900000000000002,"15min":0.78100000000000003},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97699999999999998,"5min":1.069,"15min":1.069},"last":0.57999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.54400000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.93100000000000005,"5min":0.97799999999999998,"15min":1.0760000000000001},"last":0.59699999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78900000000000003},"min":{"1min":0.61299999999999999,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.0109999999999999,"5min":1.077,"15min":1.077},"last":0.64200000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.997,"5min":1.052,"15min":1.052},"last":0.499}]},{"osd":4,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.36099999999999999},"max":{"1min":0.96299999999999997,"5min":1.05,"15min":1.0529999999999999},"last":0.50900000000000001},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66400000000000003,"15min":0.65100000000000002},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.66800000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64900000000000002,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.46800000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.55700000000000005,"5min":0.48899999999999999,"15min":0.45300000000000001},"max":{"1min":1.0429999999999999,"5min":1.103,"15min":1.1240000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.67800000000000005}]}]},{"osd":5,"up_from":13,"seq":55834575974,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47608,"kb_used_data":2608,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324232,"statfs":{"total":96636764160,"available":96588013568,"internally_reserved":0,"allocated":2670592,"data_stored":1298862,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.44,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.94499999999999995,"5min":0.97499999999999998,"15min":1.077},"last":0.91600000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.73199999999999998,"15min":0.755},"min":{"1min":0.57699999999999996,"5min":0.432,"15min":0.432},"max":{"1min":0.91400000000000003,"5min":0.96699999999999997,"15min":1.028},"last":0.60699999999999998}]},{"osd":1,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.74299999999999999,"15min":0.754},"min":{"1min":0.55400000000000005,"5min":0.48299999999999998,"15min":0.442},"max":{"1min":0.89000000000000001,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.77100000000000002,"5min":0.753,"15min":0.76500000000000001},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.46899999999999997},"max":{"1min":0.95699999999999996,"5min":1.016,"15min":1.0640000000000001},"last":0.66300000000000003}]},{"osd":2,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76600000000000001,"5min":0.73199999999999998,"15min":0.75900000000000001},"min":{"1min":0.60299999999999998,"5min":0.46999999999999997,"15min":0.46899999999999997},"max":{"1min":0.93400000000000005,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.74299999999999999},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.76500000000000001,"15min":0.77200000000000002},"min":{"1min":0.63100000000000001,"5min":0.55800000000000005,"15min":0.48299999999999998},"max":{"1min":0.93500000000000005,"5min":1.0349999999999999,"15min":1.0680000000000001},"last":1.0669999999999999}]},{"osd":3,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.82199999999999995,"5min":0.77800000000000002,"15min":0.78500000000000003},"min":{"1min":0.54800000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"max":{"1min":1.05,"5min":1.05,"15min":1.0669999999999999},"last":0.82999999999999996},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.77100000000000002,"15min":0.78200000000000003},"min":{"1min":0.65700000000000003,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.91700000000000004,"5min":1.004,"15min":1.0629999999999999},"last":0.95499999999999996}]},{"osd":4,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.63600000000000001,"15min":0.63600000000000001},"min":{"1min":0.47199999999999998,"5min":0.42199999999999999,"15min":0.38100000000000001},"max":{"1min":1.0109999999999999,"5min":1.0640000000000001,"15min":1.077},"last":0.871},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.67000000000000004,"15min":0.64200000000000002},"min":{"1min":0.46400000000000002,"5min":0.432,"15min":0.41299999999999998},"max":{"1min":1.1100000000000001,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.89300000000000002}]},{"osd":6,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.71599999999999997,"15min":0.70299999999999996},"min":{"1min":0.56299999999999994,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.0820000000000001,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.81299999999999994},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.66700000000000004,"15min":0.64800000000000002},"min":{"1min":0.45600000000000002,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.95199999999999996,"5min":1.012,"15min":1.073},"last":1.097}]},{"osd":7,"last update":"Wed Dec 20 06:02:23 2023","interfaces":[{"interface":"back","average":{"1min":0.76700000000000002,"5min":0.72699999999999998,"15min":0.70599999999999996},"min":{"1min":0.56799999999999995,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.153,"5min":1.153,"15min":1.153},"last":1},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.77200000000000002,"15min":0.76500000000000001},"min":{"1min":0.63300000000000001,"5min":0.56299999999999994,"15min":0.46700000000000003},"max":{"1min":0.97599999999999998,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.98799999999999999}]}]},{"osd":4,"up_from":13,"seq":55834575975,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47640,"kb_used_data":2640,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324200,"statfs":{"total":96636764160,"available":96587980800,"internally_reserved":0,"allocated":2703360,"data_stored":1303335,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.71899999999999997,"15min":0.73899999999999999},"min":{"1min":0.46000000000000002,"5min":0.443,"15min":0.38800000000000001},"max":{"1min":0.84399999999999997,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.49099999999999999},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.71799999999999997,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.58299999999999996}]},{"osd":1,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.746,"15min":0.75600000000000001},"min":{"1min":0.56899999999999995,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":0.95699999999999996,"5min":0.96499999999999997,"15min":1.0900000000000001},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.752,"5min":0.746,"15min":0.76800000000000002},"min":{"1min":0.52200000000000002,"5min":0.44500000000000001,"15min":0.44500000000000001},"max":{"1min":0.91400000000000003,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.61599999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.74299999999999999,"5min":0.76200000000000001,"15min":0.77600000000000002},"min":{"1min":0.58999999999999997,"5min":0.52600000000000002,"15min":0.46600000000000003},"max":{"1min":0.98699999999999999,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.46500000000000002},{"interface":"front","average":{"1min":0.78900000000000003,"5min":0.76800000000000002,"15min":0.78100000000000003},"min":{"1min":0.56999999999999995,"5min":0.51200000000000001,"15min":0.51200000000000001},"max":{"1min":0.98599999999999999,"5min":1.0089999999999999,"15min":1.101},"last":0.69599999999999995}]},{"osd":3,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.73899999999999999,"15min":0.75600000000000001},"min":{"1min":0.59899999999999998,"5min":0.51800000000000002,"15min":0.49199999999999999},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.79300000000000004,"5min":0.78900000000000003,"15min":0.80900000000000005},"min":{"1min":0.64500000000000002,"5min":0.48999999999999999,"15min":0.48999999999999999},"max":{"1min":0.95999999999999996,"5min":1.04,"15min":1.081},"last":0.51500000000000001}]},{"osd":5,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.63300000000000001,"5min":0.64100000000000001,"15min":0.64600000000000002},"min":{"1min":0.42699999999999999,"5min":0.42199999999999999,"15min":0.39900000000000002},"max":{"1min":0.93899999999999995,"5min":1.05,"15min":1.05},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.66300000000000003,"15min":0.65900000000000003},"min":{"1min":0.45400000000000001,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.86099999999999999,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.64800000000000002}]},{"osd":6,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.67300000000000004,"15min":0.67300000000000004},"min":{"1min":0.46700000000000003,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.94699999999999995,"5min":1.016,"15min":1.1679999999999999},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.71699999999999997,"5min":0.72899999999999998,"15min":0.72899999999999998},"min":{"1min":0.48599999999999999,"5min":0.48599999999999999,"15min":0.41199999999999998},"max":{"1min":0.97999999999999998,"5min":1.107,"15min":1.111},"last":0.749}]},{"osd":7,"last update":"Wed Dec 20 06:02:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.72499999999999998,"15min":0.73199999999999998},"min":{"1min":0.51900000000000002,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":1.01,"5min":1.0600000000000001,"15min":1.1160000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.748,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.52900000000000003,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.034,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.73199999999999998}]}]},{"osd":3,"up_from":13,"seq":55834575974,"num_pgs":52,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47628,"kb_used_data":2628,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324212,"statfs":{"total":96636764160,"available":96587993088,"internally_reserved":0,"allocated":2691072,"data_stored":1298656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.59299999999999997,"5min":0.748,"15min":2.8130000000000002},"last":0.51100000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.51100000000000001,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64600000000000002,"5min":0.80000000000000004,"15min":1.357},"last":0.35699999999999998}]},{"osd":1,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.48999999999999999},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":1.4219999999999999},"last":0.47099999999999997}]},{"osd":2,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.34200000000000003},"max":{"1min":0.72599999999999998,"5min":0.86099999999999999,"15min":2.944},"last":0.54600000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":1.0209999999999999,"5min":1.2090000000000001,"15min":1.669},"last":0.63600000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.99},"last":0.70299999999999996}]},{"osd":5,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.59599999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97199999999999998,"5min":1.355,"15min":1.901},"last":0.65100000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.82399999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97699999999999998,"5min":1.196,"15min":2.8639999999999999},"last":0.875}]},{"osd":6,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.81999999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.575},"last":0.71999999999999997},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.81799999999999995,"15min":0.85099999999999998},"min":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0149999999999999,"5min":1.2350000000000001,"15min":1.948},"last":0.626}]},{"osd":7,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80400000000000005,"15min":0.84099999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.004,"5min":1.3149999999999999,"15min":2.9049999999999998},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82399999999999995,"15min":0.85299999999999998},"min":{"1min":0.61899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.089,"5min":1.28,"15min":1.754},"last":0.76200000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575973,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51712,"kb_used_data":2616,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94320128,"statfs":{"total":96636764160,"available":96583811072,"internally_reserved":0,"allocated":2678784,"data_stored":1298974,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.47099999999999997,"5min":0.46899999999999997,"15min":0.46200000000000002},"min":{"1min":0.32600000000000001,"5min":0.29799999999999999,"15min":0.26100000000000001},"max":{"1min":0.75800000000000001,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.38700000000000001},{"interface":"front","average":{"1min":0.57599999999999996,"5min":0.53700000000000003,"15min":0.52300000000000002},"min":{"1min":0.376,"5min":0.34000000000000002,"15min":0.33800000000000002},"max":{"1min":0.78800000000000003,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.498,"5min":0.503,"15min":0.496},"min":{"1min":0.35099999999999998,"5min":0.32100000000000001,"15min":0.32100000000000001},"max":{"1min":0.79800000000000004,"5min":0.79800000000000004,"15min":0.84899999999999998},"last":0.505},{"interface":"front","average":{"1min":0.55900000000000005,"5min":0.55100000000000005,"15min":0.55300000000000005},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.30499999999999999},"max":{"1min":0.71299999999999997,"5min":0.92600000000000005,"15min":1.004},"last":0.48699999999999999}]},{"osd":3,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.69299999999999995,"5min":0.65800000000000003,"15min":0.64900000000000002},"min":{"1min":0.47499999999999998,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.089,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.69699999999999995,"5min":0.68700000000000006,"15min":0.68500000000000005},"min":{"1min":0.496,"5min":0.46500000000000002,"15min":0.42499999999999999},"max":{"1min":1.069,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.54300000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.79700000000000004,"5min":0.79200000000000004,"15min":0.80300000000000005},"min":{"1min":0.58699999999999997,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":1.081,"5min":1.119,"15min":1.254},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.80800000000000005,"5min":0.80200000000000005,"15min":0.80400000000000005},"min":{"1min":0.59199999999999997,"5min":0.505,"15min":0.48199999999999998},"max":{"1min":1.02,"5min":1.605,"15min":1.605},"last":0.90300000000000002}]},{"osd":5,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.82099999999999995,"5min":0.81200000000000006,"15min":0.80700000000000005},"min":{"1min":0.62,"5min":0.51400000000000001,"15min":0.51400000000000001},"max":{"1min":1.1020000000000001,"5min":1.329,"15min":1.329},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.84799999999999998,"5min":0.83899999999999997,"15min":0.83399999999999996},"min":{"1min":0.68500000000000005,"5min":0.53900000000000003,"15min":0.53900000000000003},"max":{"1min":1.0309999999999999,"5min":1.0740000000000001,"15min":1.157},"last":0.91700000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.82599999999999996,"15min":0.82199999999999995},"min":{"1min":0.60299999999999998,"5min":0.55100000000000005,"15min":0.52300000000000002},"max":{"1min":1.157,"5min":1.919,"15min":1.919},"last":0.79800000000000004},{"interface":"front","average":{"1min":0.82799999999999996,"5min":0.83199999999999996,"15min":0.83199999999999996},"min":{"1min":0.63300000000000001,"5min":0.56999999999999995,"15min":0.56200000000000006},"max":{"1min":1.0449999999999999,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.89400000000000002}]},{"osd":7,"last update":"Wed Dec 20 06:02:22 2023","interfaces":[{"interface":"back","average":{"1min":0.90800000000000003,"5min":0.875,"15min":0.85999999999999999},"min":{"1min":0.67400000000000004,"5min":0.60699999999999998,"15min":0.47899999999999998},"max":{"1min":1.3360000000000001,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.89800000000000002,"5min":0.90000000000000002,"15min":0.89000000000000001},"min":{"1min":0.57099999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":1.262,"5min":1.292,"15min":1.292},"last":0.93600000000000005}]}]},{"osd":1,"up_from":13,"seq":55834575975,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":59912,"kb_used_data":2624,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311928,"statfs":{"total":96636764160,"available":96575414272,"internally_reserved":0,"allocated":2686976,"data_stored":1301141,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47899999999999998,"15min":0.46200000000000002},"min":{"1min":0.26500000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59299999999999997,"5min":0.746,"15min":0.81000000000000005},"last":0.40400000000000003},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.29399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.47799999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.71999999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.497},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":0.88800000000000001,"15min":0.93700000000000006},"last":0.52500000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.42999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.90100000000000002},"last":0.42999999999999999},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.93500000000000005},"last":0.33000000000000002}]},{"osd":4,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.78500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92000000000000004,"5min":1.107,"15min":1.321},"last":0.55900000000000005},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.995,"5min":1.139,"15min":1.25},"last":0.68700000000000006}]},{"osd":5,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78700000000000003,"15min":0.80200000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.97899999999999998,"5min":1.224,"15min":1.2849999999999999},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.746,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0740000000000001,"5min":1.1990000000000001,"15min":1.4330000000000001},"last":0.65800000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.82399999999999995},"min":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.2430000000000001,"15min":1.341},"last":0.621},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.56399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1379999999999999,"5min":1.177,"15min":1.3839999999999999},"last":0.59199999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0900000000000001,"5min":1.272,"15min":1.4470000000000001},"last":0.71899999999999997},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0509999999999999,"5min":1.1599999999999999,"15min":1.3839999999999999},"last":0.74099999999999999}]}]},{"osd":0,"up_from":13,"seq":55834575974,"num_pgs":58,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56384,"kb_used_data":3192,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315456,"statfs":{"total":96636764160,"available":96579026944,"internally_reserved":0,"allocated":3268608,"data_stored":1888760,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.47599999999999998,"15min":0.46700000000000003},"min":{"1min":0.33200000000000002,"5min":0.33200000000000002,"15min":0.28799999999999998},"max":{"1min":0.71999999999999997,"5min":0.82299999999999995,"15min":0.96799999999999997},"last":0.435},{"interface":"front","average":{"1min":0.52200000000000002,"5min":0.55800000000000005,"15min":0.55000000000000004},"min":{"1min":0.34899999999999998,"5min":0.33200000000000002,"15min":0.33000000000000002},"max":{"1min":0.76200000000000001,"5min":0.93799999999999994,"15min":0.93799999999999994},"last":0.45900000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51100000000000001,"5min":0.53900000000000003,"15min":0.53500000000000003},"min":{"1min":0.435,"5min":0.38400000000000001,"15min":0.35999999999999999},"max":{"1min":0.65000000000000002,"5min":0.77700000000000002,"15min":0.81299999999999994},"last":0.56299999999999994},{"interface":"front","average":{"1min":0.54900000000000004,"5min":0.58199999999999996,"15min":0.58799999999999997},"min":{"1min":0.432,"5min":0.38,"15min":0.38},"max":{"1min":0.81499999999999995,"5min":0.88100000000000001,"15min":0.88100000000000001},"last":0.41099999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.51800000000000002,"5min":0.55900000000000005,"15min":0.55900000000000005},"min":{"1min":0.39300000000000002,"5min":0.38900000000000001,"15min":0.31900000000000001},"max":{"1min":0.65700000000000003,"5min":0.80500000000000005,"15min":1.0429999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.61499999999999999,"5min":0.627,"15min":0.626},"min":{"1min":0.496,"5min":0.46100000000000002,"15min":0.41999999999999998},"max":{"1min":0.78700000000000003,"5min":0.97499999999999998,"15min":0.97499999999999998},"last":0.499}]},{"osd":4,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.81399999999999995,"15min":0.81799999999999995},"min":{"1min":0.47899999999999998,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.97899999999999998,"5min":1.0469999999999999,"15min":1.147},"last":0.96399999999999997},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.83199999999999996,"15min":0.83299999999999996},"min":{"1min":0.64200000000000002,"5min":0.55300000000000005,"15min":0.51400000000000001},"max":{"1min":0.94299999999999995,"5min":1.0760000000000001,"15min":1.2869999999999999},"last":0.872}]},{"osd":5,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79900000000000004,"5min":0.82799999999999996,"15min":0.83799999999999997},"min":{"1min":0.60999999999999999,"5min":0.54900000000000004,"15min":0.54900000000000004},"max":{"1min":1.008,"5min":1.1259999999999999,"15min":1.2150000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.79900000000000004,"5min":0.83299999999999996,"15min":0.84299999999999997},"min":{"1min":0.495,"5min":0.495,"15min":0.495},"max":{"1min":1.1419999999999999,"5min":1.1419999999999999,"15min":1.351},"last":0.69399999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.79400000000000004,"5min":0.83599999999999997,"15min":0.84099999999999997},"min":{"1min":0.60099999999999998,"5min":0.56200000000000006,"15min":0.56200000000000006},"max":{"1min":0.98999999999999999,"5min":1.1040000000000001,"15min":1.2150000000000001},"last":0.93999999999999995},{"interface":"front","average":{"1min":0.82299999999999995,"5min":0.84599999999999997,"15min":0.85599999999999998},"min":{"1min":0.58599999999999997,"5min":0.58599999999999997,"15min":0.57099999999999995},"max":{"1min":0.97899999999999998,"5min":1.1379999999999999,"15min":1.3320000000000001},"last":0.76200000000000001}]},{"osd":7,"last update":"Wed Dec 20 06:02:09 2023","interfaces":[{"interface":"back","average":{"1min":0.78800000000000003,"5min":0.85099999999999998,"15min":0.86299999999999999},"min":{"1min":0.627,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97399999999999998,"5min":1.129,"15min":1.3560000000000001},"last":0.88100000000000001},{"interface":"front","average":{"1min":0.83599999999999997,"5min":0.85399999999999998,"15min":0.85899999999999999},"min":{"1min":0.61399999999999999,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":1.0009999999999999,"5min":1.139,"15min":1.2549999999999999},"last":0.92100000000000004}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":2805,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":3387,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":24,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":25,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1668,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":26,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":1564,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":27,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:02:44.116 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T06:02:44.116 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:02:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.260+0000 7f36ee117640 1 Processor -- start 2023-12-20T06:02:44.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.260+0000 7f36ee117640 1 -- start start 2023-12-20T06:02:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.264+0000 7f36ee117640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36e81599e0 0x7f36e80aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.264+0000 7f36ee117640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e80c98c0 con 0x7f36e80c90e0 2023-12-20T06:02:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.264+0000 7f36ee117640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36e80c9ba0 con 0x7f36e80b0d00 2023-12-20T06:02:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.264+0000 7f36ee117640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36e80c9e80 con 0x7f36e81599e0 2023-12-20T06:02:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.264+0000 7f36e77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36e81599e0 0x7f36e80aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.264+0000 7f36e77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36e81599e0 0x7f36e80aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46846/0 (socket says 172.21.15.32:46846) 2023-12-20T06:02:44.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.264+0000 7f36e77fe640 1 -- 172.21.15.32:0/357448917 learned_addr learned my addr 172.21.15.32:0/357448917 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.264+0000 7f36e77fe640 1 -- 172.21.15.32:0/357448917 >> v1:172.21.15.39:6789/0 conn(0x7f36e80b0d00 legacy=0x7f36e80701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.264+0000 7f36e77fe640 1 -- 172.21.15.32:0/357448917 >> v1:172.21.15.32:6789/0 conn(0x7f36e80c90e0 legacy=0x7f36e80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:44.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.264+0000 7f36e77fe640 1 -- 172.21.15.32:0/357448917 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36e80ca160 con 0x7f36e81599e0 2023-12-20T06:02:44.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36e77fe640 1 --2- 172.21.15.32:0/357448917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e81599e0 0x7f36e80aece0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f36d8006fd0 tx=0x7f36d801e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16b5ac2387d4bd9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36e67fc640 1 -- 172.21.15.32:0/357448917 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d80a86b0 con 0x7f36e81599e0 2023-12-20T06:02:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36e67fc640 1 -- 172.21.15.32:0/357448917 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36d80a8810 con 0x7f36e81599e0 2023-12-20T06:02:44.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36e67fc640 1 -- 172.21.15.32:0/357448917 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36d80a8ba0 con 0x7f36e81599e0 2023-12-20T06:02:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36ee117640 1 -- 172.21.15.32:0/357448917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e81599e0 msgr2=0x7f36e80aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36ee117640 1 --2- 172.21.15.32:0/357448917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e81599e0 0x7f36e80aece0 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f36d8006fd0 tx=0x7f36d801e6c0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36ee117640 1 -- 172.21.15.32:0/357448917 shutdown_connections 2023-12-20T06:02:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36ee117640 1 --2- 172.21.15.32:0/357448917 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e81599e0 0x7f36e80aece0 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36ee117640 1 -- 172.21.15.32:0/357448917 >> 172.21.15.32:0/357448917 conn(0x7f36e806a320 msgr2=0x7f36e80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36ee117640 1 -- 172.21.15.32:0/357448917 shutdown_connections 2023-12-20T06:02:44.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36ee117640 1 -- 172.21.15.32:0/357448917 wait complete. 2023-12-20T06:02:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36ee117640 1 Processor -- start 2023-12-20T06:02:44.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.268+0000 7f36ee117640 1 -- start start 2023-12-20T06:02:44.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36ee117640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e80b0d00 0x7f36e814c0d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36ee117640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e80c90e0 0x7f36e80c6d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36ee117640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36e81599e0 0x7f36e814d7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36ee117640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e814c700 con 0x7f36e80c90e0 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36ee117640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36e814c9e0 con 0x7f36e80b0d00 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36ee117640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36e814ccc0 con 0x7f36e81599e0 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36e6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e80c90e0 0x7f36e80c6d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36e6ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e80c90e0 0x7f36e80c6d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46856/0 (socket says 172.21.15.32:46856) 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36e6ffd640 1 -- 172.21.15.32:0/1212080065 learned_addr learned my addr 172.21.15.32:0/1212080065 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36e77fe640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e80b0d00 0x7f36e814c0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36e7fff640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36e81599e0 0x7f36e814d7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36e6ffd640 1 -- 172.21.15.32:0/1212080065 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36e81599e0 msgr2=0x7f36e814d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:44.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36e6ffd640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36e81599e0 0x7f36e814d7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36e6ffd640 1 -- 172.21.15.32:0/1212080065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e80b0d00 msgr2=0x7f36e814c0d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36e6ffd640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e80b0d00 0x7f36e814c0d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36e6ffd640 1 -- 172.21.15.32:0/1212080065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d8009480 con 0x7f36e80c90e0 2023-12-20T06:02:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36e6ffd640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e80c90e0 0x7f36e80c6d50 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00c500 tx=0x7f36dc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36cffff640 1 -- 172.21.15.32:0/1212080065 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc005220 con 0x7f36e80c90e0 2023-12-20T06:02:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36cffff640 1 -- 172.21.15.32:0/1212080065 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36dc005380 con 0x7f36e80c90e0 2023-12-20T06:02:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36cffff640 1 -- 172.21.15.32:0/1212080065 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36dc007270 con 0x7f36e80c90e0 2023-12-20T06:02:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36ee117640 1 -- 172.21.15.32:0/1212080065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36e814cfa0 con 0x7f36e80c90e0 2023-12-20T06:02:44.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36ee117640 1 -- 172.21.15.32:0/1212080065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36e80c7290 con 0x7f36e80c90e0 2023-12-20T06:02:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36ee117640 1 -- 172.21.15.32:0/1212080065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36e814d4c0 con 0x7f36e80c90e0 2023-12-20T06:02:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.272+0000 7f36cffff640 1 -- 172.21.15.32:0/1212080065 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f36dc0073d0 con 0x7f36e80c90e0 2023-12-20T06:02:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.276+0000 7f36cffff640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36d4065bb0 0x7f36d4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.276+0000 7f36cffff640 1 -- 172.21.15.32:0/1212080065 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f36dc082f70 con 0x7f36e80c90e0 2023-12-20T06:02:44.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.276+0000 7f36e77fe640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36d4065bb0 0x7f36d4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:44.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.276+0000 7f36e77fe640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36d4065bb0 0x7f36d4068070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36d80b1000 tx=0x7f36d80a9000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:44.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.276+0000 7f36cffff640 1 -- 172.21.15.32:0/1212080065 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f36e814d4c0 con 0x7f36e80c90e0 2023-12-20T06:02:44.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 -- 172.21.15.32:0/1212080065 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f36e814d4c0 con 0x7f36e80c90e0 2023-12-20T06:02:44.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36cffff640 1 -- 172.21.15.32:0/1212080065 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5902 (secure 0 0 0) 0x7f36e814d4c0 con 0x7f36e80c90e0 2023-12-20T06:02:44.466 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:44.466 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:26.942526+0000","modified":"2023-12-20T06:02:29.955675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10539":{"gid":10539,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/888710554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":888710554},{"type":"v1","addr":"172.21.15.32:6837","nonce":888710554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:39.446992+0000","modified":"2023-12-20T06:02:42.509418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10530":{"gid":10530,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/3001088147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3001088147},{"type":"v1","addr":"172.21.15.39:6833","nonce":3001088147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-20T06:02:44.466 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 102 2023-12-20T06:02:44.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 -- 172.21.15.32:0/1212080065 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36d4065bb0 msgr2=0x7f36d4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36d4065bb0 0x7f36d4068070 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f36d80b1000 tx=0x7f36d80a9000 comp rx=0 tx=0).stop 2023-12-20T06:02:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 -- 172.21.15.32:0/1212080065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e80c90e0 msgr2=0x7f36e80c6d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e80c90e0 0x7f36e80c6d50 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7f36dc00c500 tx=0x7f36dc00c9c0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 -- 172.21.15.32:0/1212080065 shutdown_connections 2023-12-20T06:02:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36d4065bb0 0x7f36d4068070 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36e81599e0 0x7f36e814d7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36e80c90e0 0x7f36e80c6d50 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 --2- 172.21.15.32:0/1212080065 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36e80b0d00 0x7f36e814c0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 -- 172.21.15.32:0/1212080065 >> 172.21.15.32:0/1212080065 conn(0x7f36e806a320 msgr2=0x7f36e80ce3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 -- 172.21.15.32:0/1212080065 shutdown_connections 2023-12-20T06:02:44.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:44.468+0000 7f36ee117640 1 -- 172.21.15.32:0/1212080065 wait complete. 2023-12-20T06:02:44.890 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 102, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T06:02:39.446992+0000', 'modified': '2023-12-20T06:02:42.509418+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 10530}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_10530': {'gid': 10530, 'name': 'd', 'rank': 0, 'incarnation': 98, 'state': 'up:active', 'state_seq': 7, 'addr': '172.21.15.39:6833/3001088147', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.39:6832', 'nonce': 3001088147}, {'type': 'v1', 'addr': '172.21.15.39:6833', 'nonce': 3001088147}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [27], 'metadata_pool': 26, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T06:02:44.890 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T06:02:44.890 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.d status 2023-12-20T06:02:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f18393db640 1 Processor -- start 2023-12-20T06:02:45.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f18393db640 1 -- start start 2023-12-20T06:02:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f18393db640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f183415cdb0 0x7f18340d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f18393db640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f18340d2740 0x7f18340cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f18393db640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f18340b6ca0 con 0x7f18340d1120 2023-12-20T06:02:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f18393db640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f183415d190 con 0x7f183415cdb0 2023-12-20T06:02:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f18393db640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18340d16a0 con 0x7f18340d2740 2023-12-20T06:02:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f1832ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f183415cdb0 0x7f18340d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f18337fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f18340d1120 0x7f18340b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60488/0 (socket says 172.21.15.32:60488) 2023-12-20T06:02:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f18337fe640 1 -- 172.21.15.32:0/2225394607 learned_addr learned my addr 172.21.15.32:0/2225394607 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f182bfff640 1 --2- 172.21.15.32:0/2225394607 >> v2:172.21.15.39:3300/0 conn(0x7f18340d2740 0x7f18340cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f1832ffd640 1 -- 172.21.15.32:0/2225394607 >> v2:172.21.15.39:3300/0 conn(0x7f18340d2740 msgr2=0x7f18340cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f1832ffd640 1 --2- 172.21.15.32:0/2225394607 >> v2:172.21.15.39:3300/0 conn(0x7f18340d2740 0x7f18340cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f1832ffd640 1 -- 172.21.15.32:0/2225394607 >> v1:172.21.15.32:6789/0 conn(0x7f18340d1120 legacy=0x7f18340b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.040+0000 7f1832ffd640 1 -- 172.21.15.32:0/2225394607 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18340d1980 con 0x7f183415cdb0 2023-12-20T06:02:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f1832ffd640 1 --2- 172.21.15.32:0/2225394607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183415cdb0 0x7f18340d0be0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f18340cd570 tx=0x7f182401ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1189e9bba478d657 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f18327fc640 1 -- 172.21.15.32:0/2225394607 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240a8950 con 0x7f183415cdb0 2023-12-20T06:02:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f18327fc640 1 -- 172.21.15.32:0/2225394607 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18240a8ab0 con 0x7f183415cdb0 2023-12-20T06:02:45.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f18327fc640 1 -- 172.21.15.32:0/2225394607 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240a8e40 con 0x7f183415cdb0 2023-12-20T06:02:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f18393db640 1 -- 172.21.15.32:0/2225394607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183415cdb0 msgr2=0x7f18340d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f18393db640 1 --2- 172.21.15.32:0/2225394607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183415cdb0 0x7f18340d0be0 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7f18340cd570 tx=0x7f182401ec50 comp rx=0 tx=0).stop 2023-12-20T06:02:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f18393db640 1 -- 172.21.15.32:0/2225394607 shutdown_connections 2023-12-20T06:02:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f18393db640 1 --2- 172.21.15.32:0/2225394607 >> v2:172.21.15.39:3300/0 conn(0x7f18340d2740 0x7f18340cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f18393db640 1 --2- 172.21.15.32:0/2225394607 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f183415cdb0 0x7f18340d0be0 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f18393db640 1 -- 172.21.15.32:0/2225394607 >> 172.21.15.32:0/2225394607 conn(0x7f1834065ee0 msgr2=0x7f183406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f18393db640 1 -- 172.21.15.32:0/2225394607 shutdown_connections 2023-12-20T06:02:45.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.044+0000 7f18393db640 1 -- 172.21.15.32:0/2225394607 wait complete. 2023-12-20T06:02:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f18393db640 1 Processor -- start 2023-12-20T06:02:45.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f18393db640 1 -- start start 2023-12-20T06:02:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f18393db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18340d1120 0x7f1834151370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f18393db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18340d2740 0x7f183415fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f18393db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f183415cdb0 0x7f1834150830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f18393db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1834151190 con 0x7f18340d1120 2023-12-20T06:02:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f18393db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18341e5210 con 0x7f183415cdb0 2023-12-20T06:02:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f18393db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18341e54d0 con 0x7f18340d2740 2023-12-20T06:02:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f1832ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18340d1120 0x7f1834151370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f1832ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18340d1120 0x7f1834151370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46870/0 (socket says 172.21.15.32:46870) 2023-12-20T06:02:45.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f1832ffd640 1 -- 172.21.15.32:0/3639150288 learned_addr learned my addr 172.21.15.32:0/3639150288 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f182bfff640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18340d2740 0x7f183415fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f18337fe640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f183415cdb0 0x7f1834150830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f1832ffd640 1 -- 172.21.15.32:0/3639150288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18340d2740 msgr2=0x7f183415fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f1832ffd640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18340d2740 0x7f183415fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f1832ffd640 1 -- 172.21.15.32:0/3639150288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f183415cdb0 msgr2=0x7f1834150830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f1832ffd640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f183415cdb0 0x7f1834150830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f1832ffd640 1 -- 172.21.15.32:0/3639150288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1824009420 con 0x7f18340d1120 2023-12-20T06:02:45.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f1832ffd640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18340d1120 0x7f1834151370 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f18240b3040 tx=0x7f182401b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f182b7fe640 1 -- 172.21.15.32:0/3639150288 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240b9630 con 0x7f18340d1120 2023-12-20T06:02:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f182b7fe640 1 -- 172.21.15.32:0/3639150288 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18240b97b0 con 0x7f18340d1120 2023-12-20T06:02:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f182b7fe640 1 -- 172.21.15.32:0/3639150288 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240b9b60 con 0x7f18340d1120 2023-12-20T06:02:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f18393db640 1 -- 172.21.15.32:0/3639150288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18341e57b0 con 0x7f18340d1120 2023-12-20T06:02:45.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.048+0000 7f18393db640 1 -- 172.21.15.32:0/3639150288 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18341e5d70 con 0x7f18340d1120 2023-12-20T06:02:45.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.052+0000 7f182b7fe640 1 -- 172.21.15.32:0/3639150288 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f18240b7040 con 0x7f18340d1120 2023-12-20T06:02:45.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.052+0000 7f182b7fe640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1814065b80 0x7f1814068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.052+0000 7f182b7fe640 1 -- 172.21.15.32:0/3639150288 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f18241215b0 con 0x7f18340d1120 2023-12-20T06:02:45.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.052+0000 7f182bfff640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1814065b80 0x7f1814068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.052+0000 7f183c74a000 1 Processor -- start 2023-12-20T06:02:45.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.052+0000 7f183c74a000 1 -- start start 2023-12-20T06:02:45.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f182bfff640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1814065b80 0x7f1814068040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f183415b020 tx=0x7f181c008040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de3b450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756de3b990 0x55756ddf0c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756ddf11c0 0x55756ddf15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756dc14c90 con 0x55756de3b990 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756dc46220 con 0x55756de3b070 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55756dc437f0 con 0x55756ddf11c0 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f182bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756de3b990 0x55756ddf0c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f182bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756de3b990 0x55756ddf0c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46876/0 (socket says 172.21.15.32:46876) 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f182bfff640 1 -- 172.21.15.32:0/3861849594 learned_addr learned my addr 172.21.15.32:0/3861849594 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f18337fe640 1 --2- 172.21.15.32:0/3861849594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756ddf11c0 0x55756ddf15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f1832ffd640 1 --2- 172.21.15.32:0/3861849594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de3b450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f182bfff640 1 -- 172.21.15.32:0/3861849594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756ddf11c0 msgr2=0x55756ddf15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f182bfff640 1 --2- 172.21.15.32:0/3861849594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756ddf11c0 0x55756ddf15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f182bfff640 1 -- 172.21.15.32:0/3861849594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 msgr2=0x55756de3b450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f182bfff640 1 --2- 172.21.15.32:0/3861849594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de3b450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f182bfff640 1 -- 172.21.15.32:0/3861849594 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55756d93b890 con 0x55756de3b990 2023-12-20T06:02:45.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f182bfff640 1 --2- 172.21.15.32:0/3861849594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756de3b990 0x55756ddf0c80 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f181c010b50 tx=0x7f181c013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6791597f510726c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f1828ff9640 1 -- 172.21.15.32:0/3861849594 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c016ca0 con 0x55756de3b990 2023-12-20T06:02:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f1828ff9640 1 -- 172.21.15.32:0/3861849594 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f181c01a070 con 0x55756de3b990 2023-12-20T06:02:45.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f1828ff9640 1 -- 172.21.15.32:0/3861849594 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f181c021440 con 0x55756de3b990 2023-12-20T06:02:45.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 -- 172.21.15.32:0/3861849594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756de3b990 msgr2=0x55756ddf0c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 --2- 172.21.15.32:0/3861849594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756de3b990 0x55756ddf0c80 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7f181c010b50 tx=0x7f181c013000 comp rx=0 tx=0).stop 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 -- 172.21.15.32:0/3861849594 shutdown_connections 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 --2- 172.21.15.32:0/3861849594 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756ddf11c0 0x55756ddf15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 --2- 172.21.15.32:0/3861849594 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756de3b990 0x55756ddf0c80 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 --2- 172.21.15.32:0/3861849594 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de3b450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 -- 172.21.15.32:0/3861849594 >> 172.21.15.32:0/3861849594 conn(0x55756ddf7950 msgr2=0x55756ddef560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 -- 172.21.15.32:0/3861849594 shutdown_connections 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 -- 172.21.15.32:0/3861849594 wait complete. 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 -- start start 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.056+0000 7f183c74a000 1 Processor -- start 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de427f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de42d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756de3b990 0x55756de29030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756dc3f0e0 con 0x55756ddf11c0 2023-12-20T06:02:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756dbde310 con 0x55756de3b070 2023-12-20T06:02:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55756dbfc770 con 0x55756de3b990 2023-12-20T06:02:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 10 client.0 _ll_register_callbacks cb 0x55756dddcb60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:02:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1832ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de427f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1832ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de427f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46888/0 (socket says 172.21.15.32:46888) 2023-12-20T06:02:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1832ffd640 1 -- 172.21.15.32:0/1022872860 learned_addr learned my addr 172.21.15.32:0/1022872860 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f182bfff640 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de42d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f18337fe640 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756de3b990 0x55756de29030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1832ffd640 1 -- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756de3b990 msgr2=0x55756de29030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1832ffd640 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756de3b990 0x55756de29030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1832ffd640 1 -- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 msgr2=0x55756de42d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1832ffd640 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de42d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1832ffd640 1 -- 172.21.15.32:0/1022872860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181c00e040 con 0x55756ddf11c0 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1832ffd640 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de427f0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f1824020260 tx=0x7f18240f2f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a58c4bb1a54c6ccf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1811ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1811ffb640 1 -- 172.21.15.32:0/1022872860 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240fa750 con 0x55756ddf11c0 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 1 -- 172.21.15.32:0/1022872860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55756dc0ec10 con 0x55756ddf11c0 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1811ffb640 1 -- 172.21.15.32:0/1022872860 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f18240f1c60 con 0x55756ddf11c0 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1811ffb640 1 -- 172.21.15.32:0/1022872860 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f18240f1e20 con 0x55756ddf11c0 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 10 client.10797 fetch_fsmap learned FSMap version 102 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 1 -- 172.21.15.32:0/1022872860 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55756dc04260 con 0x55756ddf11c0 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1811ffb640 1 -- 172.21.15.32:0/1022872860 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55756dc0ec10 con 0x55756ddf11c0 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1811ffb640 1 -- 172.21.15.32:0/1022872860 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 102) v1 ==== 3681+0+0 (secure 0 0 0) 0x7f182411f120 con 0x55756ddf11c0 2023-12-20T06:02:45.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 10 client.10797 fetch_fsmap finished waiting for FSMap version 102 2023-12-20T06:02:45.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 10 client.10797 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-20T06:02:45.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 20 client.10797 populate_metadata read hostname 'smithi032' 2023-12-20T06:02:45.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:6832/3001088147,v1:172.21.15.39:6833/3001088147] conn(0x55756de29da0 0x55756de2a180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 4 client.10797 mds_command: new command op to 10530 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T06:02:45.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f183c74a000 1 -- 172.21.15.32:0/1022872860 --> [v2:172.21.15.39:6832/3001088147,v1:172.21.15.39:6833/3001088147] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55756dc3da30 con 0x55756de29da0 2023-12-20T06:02:45.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f182bfff640 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:6832/3001088147,v1:172.21.15.39:6833/3001088147] conn(0x55756de29da0 0x55756de2a180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f1811ffb640 10 client.10797 ms_handle_connect on v2:172.21.15.39:6832/3001088147 2023-12-20T06:02:45.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.060+0000 7f182bfff640 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:6832/3001088147,v1:172.21.15.39:6833/3001088147] conn(0x55756de29da0 0x55756de2a180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7a48da46a0c688b server_cookie=5f093bc032402f3a in_seq=0 out_seq=0 2023-12-20T06:02:45.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f1811ffb640 1 -- 172.21.15.32:0/1022872860 <== mds.0 v2:172.21.15.39:6832/3001088147 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55756dc3da30 con 0x55756de29da0 2023-12-20T06:02:45.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f1811ffb640 10 client.10797 handle_command_reply: tid=0 2023-12-20T06:02:45.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 1 client.10797 shutdown 2023-12-20T06:02:45.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 1 -- 172.21.15.32:0/1022872860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 msgr2=0x55756de427f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de427f0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x7f1824020260 tx=0x7f18240f2f30 comp rx=0 tx=0).stop 2023-12-20T06:02:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 1 -- 172.21.15.32:0/1022872860 shutdown_connections 2023-12-20T06:02:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:6832/3001088147,v1:172.21.15.39:6833/3001088147] conn(0x55756de29da0 0x55756de2a180 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756de3b990 0x55756de29030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de42d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 1 --2- 172.21.15.32:0/1022872860 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de427f0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f1811ffb640 0 client.10797 ms_handle_reset on v2:172.21.15.39:6832/3001088147 2023-12-20T06:02:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 1 -- 172.21.15.32:0/1022872860 >> 172.21.15.32:0/1022872860 conn(0x55756ddf7950 msgr2=0x55756ddefc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 1 -- 172.21.15.32:0/1022872860 shutdown_connections 2023-12-20T06:02:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 1 -- 172.21.15.32:0/1022872860 wait complete. 2023-12-20T06:02:45.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.064+0000 7f183c74a000 20 client.10797 trim_cache size 0 max 16384 2023-12-20T06:02:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 Processor -- start 2023-12-20T06:02:45.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 -- start start 2023-12-20T06:02:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf7950 0x55756de93500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756ddf11c0 0x55756de93a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756de3b070 0x55756de93f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756d8fc940 con 0x55756ddf7950 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756ddf69f0 con 0x55756ddf11c0 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55756dc3df70 con 0x55756de3b070 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f1832ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf7950 0x55756de93500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f1832ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf7950 0x55756de93500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46892/0 (socket says 172.21.15.32:46892) 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f1832ffd640 1 -- 172.21.15.32:0/358261490 learned_addr learned my addr 172.21.15.32:0/358261490 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f182bfff640 1 --2- 172.21.15.32:0/358261490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756ddf11c0 0x55756de93a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f18337fe640 1 --2- 172.21.15.32:0/358261490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756de3b070 0x55756de93f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f1832ffd640 1 -- 172.21.15.32:0/358261490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756de3b070 msgr2=0x55756de93f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f1832ffd640 1 --2- 172.21.15.32:0/358261490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756de3b070 0x55756de93f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f1832ffd640 1 -- 172.21.15.32:0/358261490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756ddf11c0 msgr2=0x55756de93a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:45.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f1832ffd640 1 --2- 172.21.15.32:0/358261490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756ddf11c0 0x55756de93a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f1832ffd640 1 -- 172.21.15.32:0/358261490 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55756dbea3b0 con 0x55756ddf7950 2023-12-20T06:02:45.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f182bfff640 1 --2- 172.21.15.32:0/358261490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756ddf11c0 0x55756de93a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:45.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f18337fe640 1 --2- 172.21.15.32:0/358261490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756de3b070 0x55756de93f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:45.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f1832ffd640 1 --2- 172.21.15.32:0/358261490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf7950 0x55756de93500 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f182401b860 tx=0x7f18240bb000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d29442de3d16984f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f18297fa640 1 -- 172.21.15.32:0/358261490 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240f22b0 con 0x55756ddf7950 2023-12-20T06:02:45.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f18297fa640 1 -- 172.21.15.32:0/358261490 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18240fa750 con 0x55756ddf7950 2023-12-20T06:02:45.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f18297fa640 1 -- 172.21.15.32:0/358261490 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1824110310 con 0x55756ddf7950 2023-12-20T06:02:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 -- 172.21.15.32:0/358261490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf7950 msgr2=0x55756de93500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 --2- 172.21.15.32:0/358261490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf7950 0x55756de93500 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f182401b860 tx=0x7f18240bb000 comp rx=0 tx=0).stop 2023-12-20T06:02:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 -- 172.21.15.32:0/358261490 shutdown_connections 2023-12-20T06:02:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 --2- 172.21.15.32:0/358261490 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756de3b070 0x55756de93f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 --2- 172.21.15.32:0/358261490 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756ddf11c0 0x55756de93a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 --2- 172.21.15.32:0/358261490 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf7950 0x55756de93500 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 -- 172.21.15.32:0/358261490 >> 172.21.15.32:0/358261490 conn(0x55756de3a7c0 msgr2=0x55756de26180 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 -- 172.21.15.32:0/358261490 shutdown_connections 2023-12-20T06:02:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 -- 172.21.15.32:0/358261490 wait complete. 2023-12-20T06:02:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 -- start start 2023-12-20T06:02:45.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.080+0000 7f183c74a000 1 Processor -- start 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de4de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756ddf7950 0x55756de4e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de4e8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756dde89d0 con 0x55756ddf11c0 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55756dbe7860 con 0x55756de3b070 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55756dd0fb30 con 0x55756ddf7950 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 10 client.0 _ll_register_callbacks cb 0x55756dbed640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1832ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de4de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1832ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de4de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46902/0 (socket says 172.21.15.32:46902) 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1832ffd640 1 -- 172.21.15.32:0/218751418 learned_addr learned my addr 172.21.15.32:0/218751418 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f18337fe640 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de4e8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f182bfff640 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756ddf7950 0x55756de4e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1832ffd640 1 -- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756ddf7950 msgr2=0x55756de4e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1832ffd640 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756ddf7950 0x55756de4e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1832ffd640 1 -- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 msgr2=0x55756de4e8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1832ffd640 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de4e8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1832ffd640 1 -- 172.21.15.32:0/218751418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f181c00e040 con 0x55756ddf11c0 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1832ffd640 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de4de50 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f18240a8450 tx=0x7f1824000bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=663ab23b962da8ce server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1811ffb640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1811ffb640 1 -- 172.21.15.32:0/218751418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18240f22b0 con 0x55756ddf11c0 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1811ffb640 1 -- 172.21.15.32:0/218751418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f1824110310 con 0x55756ddf11c0 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 1 -- 172.21.15.32:0/218751418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55756dc04e00 con 0x55756ddf11c0 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1811ffb640 1 -- 172.21.15.32:0/218751418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=102) v2 ==== 24+0+0 (secure 0 0 0) 0x7f182411f450 con 0x55756ddf11c0 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 10 client.10803 fetch_fsmap learned FSMap version 102 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 1 -- 172.21.15.32:0/218751418 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=102,monmap=2+}) v3 -- 0x55756d93bcc0 con 0x55756ddf11c0 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1811ffb640 1 -- 172.21.15.32:0/218751418 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18240ff020 con 0x55756ddf11c0 2023-12-20T06:02:45.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1811ffb640 1 -- 172.21.15.32:0/218751418 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 102) v1 ==== 3681+0+0 (secure 0 0 0) 0x7f1824117060 con 0x55756ddf11c0 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 10 client.10803 fetch_fsmap finished waiting for FSMap version 102 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 10 client.10803 resolve_mds: resolved name 'd' to daemon mds.d 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 20 client.10803 populate_metadata read hostname 'smithi032' 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:6832/3001088147,v1:172.21.15.39:6833/3001088147] conn(0x55756de3b990 0x55756de4ee10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 4 client.10803 mds_command: new command op to 10530 tid=0[{"prefix": "status"}] 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 1 -- 172.21.15.32:0/218751418 --> [v2:172.21.15.39:6832/3001088147,v1:172.21.15.39:6833/3001088147] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55756dc38160 con 0x55756de3b990 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f182bfff640 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:6832/3001088147,v1:172.21.15.39:6833/3001088147] conn(0x55756de3b990 0x55756de4ee10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1811ffb640 10 client.10803 ms_handle_connect on v2:172.21.15.39:6832/3001088147 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f182bfff640 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:6832/3001088147,v1:172.21.15.39:6833/3001088147] conn(0x55756de3b990 0x55756de4ee10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=860892daa1a297b0 server_cookie=2944b3cc15de0a99 in_seq=0 out_seq=0 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1811ffb640 1 -- 172.21.15.32:0/218751418 <== mds.0 v2:172.21.15.39:6832/3001088147 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+338 (crc 0 0 0) 0x55756dc38160 con 0x55756de3b990 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f1811ffb640 10 client.10803 handle_command_reply: tid=0 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.084+0000 7f183c74a000 1 client.10803 shutdown 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f183c74a000 1 -- 172.21.15.32:0/218751418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 msgr2=0x55756de4de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f183c74a000 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de4de50 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f18240a8450 tx=0x7f1824000bd0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f183c74a000 1 -- 172.21.15.32:0/218751418 shutdown_connections 2023-12-20T06:02:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f183c74a000 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:6832/3001088147,v1:172.21.15.39:6833/3001088147] conn(0x55756de3b990 0x55756de4ee10 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f183c74a000 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55756de3b070 0x55756de4e8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f183c74a000 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55756ddf7950 0x55756de4e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f1811ffb640 0 client.10803 ms_handle_reset on v2:172.21.15.39:6832/3001088147 2023-12-20T06:02:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f183c74a000 1 --2- 172.21.15.32:0/218751418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55756ddf11c0 0x55756de4de50 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f183c74a000 1 -- 172.21.15.32:0/218751418 >> 172.21.15.32:0/218751418 conn(0x55756de3a7c0 msgr2=0x55756de26a10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f183c74a000 1 -- 172.21.15.32:0/218751418 shutdown_connections 2023-12-20T06:02:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f183c74a000 1 -- 172.21.15.32:0/218751418 wait complete. 2023-12-20T06:02:45.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f183c74a000 20 client.10803 trim_cache size 0 max 16384 2023-12-20T06:02:45.089 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 10530, 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "backup_fs", 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.6377632330000003, 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 102, 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 213, 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 213, 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 28.844816342000001 2023-12-20T06:02:45.090 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:02:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 -- 172.21.15.32:0/3639150288 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1814065b80 msgr2=0x7f1814068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1814065b80 0x7f1814068040 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f183415b020 tx=0x7f181c008040 comp rx=0 tx=0).stop 2023-12-20T06:02:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 -- 172.21.15.32:0/3639150288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18340d1120 msgr2=0x7f1834151370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18340d1120 0x7f1834151370 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7f18240b3040 tx=0x7f182401b9f0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 -- 172.21.15.32:0/3639150288 shutdown_connections 2023-12-20T06:02:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1814065b80 0x7f1814068040 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f183415cdb0 0x7f1834150830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18340d2740 0x7f183415fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 --2- 172.21.15.32:0/3639150288 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f18340d1120 0x7f1834151370 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 -- 172.21.15.32:0/3639150288 >> 172.21.15.32:0/3639150288 conn(0x7f1834065ee0 msgr2=0x7f18340d3450 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 -- 172.21.15.32:0/3639150288 shutdown_connections 2023-12-20T06:02:45.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.088+0000 7f18393db640 1 -- 172.21.15.32:0/3639150288 wait complete. 2023-12-20T06:02:45.104 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T06:02:45.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.248+0000 7f3f20939640 1 Processor -- start 2023-12-20T06:02:45.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.248+0000 7f3f20939640 1 -- start start 2023-12-20T06:02:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.252+0000 7f3f20939640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f1c1599e0 0x7f3f1c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.252+0000 7f3f20939640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c0c98c0 con 0x7f3f1c0c90e0 2023-12-20T06:02:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.252+0000 7f3f20939640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f3f1c0c9ba0 con 0x7f3f1c0b0d00 2023-12-20T06:02:45.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.252+0000 7f3f20939640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3f1c0c9e80 con 0x7f3f1c1599e0 2023-12-20T06:02:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.252+0000 7f3f1a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3f1c1599e0 0x7f3f1c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.252+0000 7f3f13fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f3f1c0c90e0 0x7f3f1c0b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60504/0 (socket says 172.21.15.32:60504) 2023-12-20T06:02:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.252+0000 7f3f13fff640 1 -- 172.21.15.32:0/2309860676 learned_addr learned my addr 172.21.15.32:0/2309860676 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.252+0000 7f3f1a59c640 1 -- 172.21.15.32:0/2309860676 >> v1:172.21.15.39:6789/0 conn(0x7f3f1c0b0d00 legacy=0x7f3f1c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.252+0000 7f3f1a59c640 1 -- 172.21.15.32:0/2309860676 >> v1:172.21.15.32:6789/0 conn(0x7f3f1c0c90e0 legacy=0x7f3f1c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.252+0000 7f3f1a59c640 1 -- 172.21.15.32:0/2309860676 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f1c0ca160 con 0x7f3f1c1599e0 2023-12-20T06:02:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f1a59c640 1 --2- 172.21.15.32:0/2309860676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f1c1599e0 0x7f3f1c0aece0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0097a0 tx=0x7f3f0c01c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4212814691a876 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f19d9b640 1 -- 172.21.15.32:0/2309860676 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0ab600 con 0x7f3f1c1599e0 2023-12-20T06:02:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f19d9b640 1 -- 172.21.15.32:0/2309860676 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f0c0a7050 con 0x7f3f1c1599e0 2023-12-20T06:02:45.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f19d9b640 1 -- 172.21.15.32:0/2309860676 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f0c0ab980 con 0x7f3f1c1599e0 2023-12-20T06:02:45.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f20939640 1 -- 172.21.15.32:0/2309860676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f1c1599e0 msgr2=0x7f3f1c0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f20939640 1 --2- 172.21.15.32:0/2309860676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f1c1599e0 0x7f3f1c0aece0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f3f0c0097a0 tx=0x7f3f0c01c430 comp rx=0 tx=0).stop 2023-12-20T06:02:45.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f20939640 1 -- 172.21.15.32:0/2309860676 shutdown_connections 2023-12-20T06:02:45.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f20939640 1 --2- 172.21.15.32:0/2309860676 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f1c1599e0 0x7f3f1c0aece0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f20939640 1 -- 172.21.15.32:0/2309860676 >> 172.21.15.32:0/2309860676 conn(0x7f3f1c06a320 msgr2=0x7f3f1c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:45.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f20939640 1 -- 172.21.15.32:0/2309860676 shutdown_connections 2023-12-20T06:02:45.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f20939640 1 -- 172.21.15.32:0/2309860676 wait complete. 2023-12-20T06:02:45.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f20939640 1 Processor -- start 2023-12-20T06:02:45.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.256+0000 7f3f20939640 1 -- start start 2023-12-20T06:02:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f20939640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f1c0b0d00 0x7f3f1c158b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f20939640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f1c0c90e0 0x7f3f1c14a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f20939640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f1c1599e0 0x7f3f1c0c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f20939640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c14aca0 con 0x7f3f1c1599e0 2023-12-20T06:02:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f20939640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0c7380 con 0x7f3f1c0b0d00 2023-12-20T06:02:45.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f20939640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3f1c0c62e0 con 0x7f3f1c0c90e0 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f1ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f1c1599e0 0x7f3f1c0c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f1ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f1c1599e0 0x7f3f1c0c75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46930/0 (socket says 172.21.15.32:46930) 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f1ad9d640 1 -- 172.21.15.32:0/3872786215 learned_addr learned my addr 172.21.15.32:0/3872786215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f1a59c640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f1c0b0d00 0x7f3f1c158b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f13fff640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f1c0c90e0 0x7f3f1c14a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f1ad9d640 1 -- 172.21.15.32:0/3872786215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f1c0c90e0 msgr2=0x7f3f1c14a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f1ad9d640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f1c0c90e0 0x7f3f1c14a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f1ad9d640 1 -- 172.21.15.32:0/3872786215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f1c0b0d00 msgr2=0x7f3f1c158b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f1ad9d640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f1c0b0d00 0x7f3f1c158b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f1ad9d640 1 -- 172.21.15.32:0/3872786215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3f0c0093b0 con 0x7f3f1c1599e0 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f1a59c640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f1c0b0d00 0x7f3f1c158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:45.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f1ad9d640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f1c1599e0 0x7f3f1c0c75d0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400c770 tx=0x7f3f1400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f12ffd640 1 -- 172.21.15.32:0/3872786215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f14003b80 con 0x7f3f1c1599e0 2023-12-20T06:02:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f12ffd640 1 -- 172.21.15.32:0/3872786215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3f14003ce0 con 0x7f3f1c1599e0 2023-12-20T06:02:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f12ffd640 1 -- 172.21.15.32:0/3872786215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3f1400aa90 con 0x7f3f1c1599e0 2023-12-20T06:02:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f20939640 1 -- 172.21.15.32:0/3872786215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3f1c0c65c0 con 0x7f3f1c1599e0 2023-12-20T06:02:45.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f20939640 1 -- 172.21.15.32:0/3872786215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3f1c0c6b80 con 0x7f3f1c1599e0 2023-12-20T06:02:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f20939640 1 -- 172.21.15.32:0/3872786215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3f1c04e000 con 0x7f3f1c1599e0 2023-12-20T06:02:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.260+0000 7f3f12ffd640 1 -- 172.21.15.32:0/3872786215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3f1400abf0 con 0x7f3f1c1599e0 2023-12-20T06:02:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.264+0000 7f3f12ffd640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3efc065b80 0x7f3efc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.264+0000 7f3f12ffd640 1 -- 172.21.15.32:0/3872786215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f3f14083530 con 0x7f3f1c1599e0 2023-12-20T06:02:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.264+0000 7f3f1a59c640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3efc065b80 0x7f3efc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.264+0000 7f3f1a59c640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3efc065b80 0x7f3efc068040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c06ab20 tx=0x7f3f0c0a6000 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.268+0000 7f3f12ffd640 1 -- 172.21.15.32:0/3872786215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3f14050a40 con 0x7f3f1c1599e0 2023-12-20T06:02:45.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.444+0000 7f3f20939640 1 -- 172.21.15.32:0/3872786215 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7f3f1c159dc0 con 0x7f3efc065b80 2023-12-20T06:02:45.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f12ffd640 1 -- 172.21.15.32:0/3872786215 <== mgr.10314 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7f3f0c0ab8e0 con 0x7f3efc065b80 2023-12-20T06:02:45.445 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T06:02:45.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:45.445 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T06:02:45.445 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:02:45.445 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:02:45.445 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "backup_fs" 2023-12-20T06:02:45.446 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:02:45.446 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T06:02:45.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 -- 172.21.15.32:0/3872786215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3efc065b80 msgr2=0x7f3efc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3efc065b80 0x7f3efc068040 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3f1c06ab20 tx=0x7f3f0c0a6000 comp rx=0 tx=0).stop 2023-12-20T06:02:45.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 -- 172.21.15.32:0/3872786215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f1c1599e0 msgr2=0x7f3f1c0c75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f1c1599e0 0x7f3f1c0c75d0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f3f1400c770 tx=0x7f3f1400cc30 comp rx=0 tx=0).stop 2023-12-20T06:02:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 -- 172.21.15.32:0/3872786215 shutdown_connections 2023-12-20T06:02:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3efc065b80 0x7f3efc068040 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3f1c1599e0 0x7f3f1c0c75d0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3f1c0c90e0 0x7f3f1c14a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 --2- 172.21.15.32:0/3872786215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3f1c0b0d00 0x7f3f1c158b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 -- 172.21.15.32:0/3872786215 >> 172.21.15.32:0/3872786215 conn(0x7f3f1c06a320 msgr2=0x7f3f1c0ce3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 -- 172.21.15.32:0/3872786215 shutdown_connections 2023-12-20T06:02:45.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.448+0000 7f3f20939640 1 -- 172.21.15.32:0/3872786215 wait complete. 2023-12-20T06:02:45.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T06:02:45.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b3a2d640 1 Processor -- start 2023-12-20T06:02:45.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b3a2d640 1 -- start start 2023-12-20T06:02:45.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b3a2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08ac0b0760 0x7f08ac0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b3a2d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08ac0cc4b0 0x7f08ac0cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b3a2d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08ac0b1100 con 0x7f08ac157480 2023-12-20T06:02:45.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b3a2d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08ac0c9f50 con 0x7f08ac0cc4b0 2023-12-20T06:02:45.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b3a2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac0cde20 con 0x7f08ac0b0760 2023-12-20T06:02:45.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b1fca640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08ac0cc4b0 0x7f08ac0cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b1fca640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08ac0cc4b0 0x7f08ac0cc940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46934/0 (socket says 172.21.15.32:46934) 2023-12-20T06:02:45.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b1fca640 1 -- 172.21.15.32:0/2591534799 learned_addr learned my addr 172.21.15.32:0/2591534799 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:45.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b0fc8640 1 --2- 172.21.15.32:0/2591534799 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08ac0b0760 0x7f08ac0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b1fca640 1 -- 172.21.15.32:0/2591534799 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08ac0b0760 msgr2=0x7f08ac0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:45.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b1fca640 1 --2- 172.21.15.32:0/2591534799 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08ac0b0760 0x7f08ac0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b1fca640 1 -- 172.21.15.32:0/2591534799 >> v1:172.21.15.39:6789/0 conn(0x7f08ac157480 legacy=0x7f08ac0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:45.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.616+0000 7f08b1fca640 1 -- 172.21.15.32:0/2591534799 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08ac0c8e30 con 0x7f08ac0cc4b0 2023-12-20T06:02:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08b1fca640 1 --2- 172.21.15.32:0/2591534799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac0cc4b0 0x7f08ac0cc940 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009e50 tx=0x7f08a401c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c4d984abb5c7767b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08a3fff640 1 -- 172.21.15.32:0/2591534799 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40a5810 con 0x7f08ac0cc4b0 2023-12-20T06:02:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08a3fff640 1 -- 172.21.15.32:0/2591534799 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a40a5970 con 0x7f08ac0cc4b0 2023-12-20T06:02:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08a3fff640 1 -- 172.21.15.32:0/2591534799 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40a5d00 con 0x7f08ac0cc4b0 2023-12-20T06:02:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08b3a2d640 1 -- 172.21.15.32:0/2591534799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac0cc4b0 msgr2=0x7f08ac0cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08b3a2d640 1 --2- 172.21.15.32:0/2591534799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac0cc4b0 0x7f08ac0cc940 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f08a4009e50 tx=0x7f08a401c950 comp rx=0 tx=0).stop 2023-12-20T06:02:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08b3a2d640 1 -- 172.21.15.32:0/2591534799 shutdown_connections 2023-12-20T06:02:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08b3a2d640 1 --2- 172.21.15.32:0/2591534799 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac0cc4b0 0x7f08ac0cc940 unknown :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08b3a2d640 1 --2- 172.21.15.32:0/2591534799 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08ac0b0760 0x7f08ac0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08b3a2d640 1 -- 172.21.15.32:0/2591534799 >> 172.21.15.32:0/2591534799 conn(0x7f08ac06a510 msgr2=0x7f08ac0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08b3a2d640 1 -- 172.21.15.32:0/2591534799 shutdown_connections 2023-12-20T06:02:45.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.620+0000 7f08b3a2d640 1 -- 172.21.15.32:0/2591534799 wait complete. 2023-12-20T06:02:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b3a2d640 1 Processor -- start 2023-12-20T06:02:45.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b3a2d640 1 -- start start 2023-12-20T06:02:45.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b3a2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08ac0b0760 0x7f08ac152e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b3a2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08ac0cc4b0 0x7f08ac159510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b3a2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac157480 0x7f08ac151120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b3a2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac153380 con 0x7f08ac157480 2023-12-20T06:02:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b3a2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac159a50 con 0x7f08ac0b0760 2023-12-20T06:02:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b3a2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08ac154a80 con 0x7f08ac0cc4b0 2023-12-20T06:02:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b1fca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac157480 0x7f08ac151120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b1fca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac157480 0x7f08ac151120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46944/0 (socket says 172.21.15.32:46944) 2023-12-20T06:02:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b1fca640 1 -- 172.21.15.32:0/953387921 learned_addr learned my addr 172.21.15.32:0/953387921 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b17c9640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08ac0b0760 0x7f08ac152e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b0fc8640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08ac0cc4b0 0x7f08ac159510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b1fca640 1 -- 172.21.15.32:0/953387921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08ac0cc4b0 msgr2=0x7f08ac159510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b1fca640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08ac0cc4b0 0x7f08ac159510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b1fca640 1 -- 172.21.15.32:0/953387921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08ac0b0760 msgr2=0x7f08ac152e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b1fca640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08ac0b0760 0x7f08ac152e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b1fca640 1 -- 172.21.15.32:0/953387921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08a4009b30 con 0x7f08ac157480 2023-12-20T06:02:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b1fca640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac157480 0x7f08ac151120 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f08a40af040 tx=0x7f08a40196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08a1ffb640 1 -- 172.21.15.32:0/953387921 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40b8670 con 0x7f08ac157480 2023-12-20T06:02:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08a1ffb640 1 -- 172.21.15.32:0/953387921 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08a40b87d0 con 0x7f08ac157480 2023-12-20T06:02:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08b3a2d640 1 -- 172.21.15.32:0/953387921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08ac154d60 con 0x7f08ac157480 2023-12-20T06:02:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.624+0000 7f08a1ffb640 1 -- 172.21.15.32:0/953387921 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08a40b8b60 con 0x7f08ac157480 2023-12-20T06:02:45.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.628+0000 7f08b3a2d640 1 -- 172.21.15.32:0/953387921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08ac155320 con 0x7f08ac157480 2023-12-20T06:02:45.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.628+0000 7f08b3a2d640 1 -- 172.21.15.32:0/953387921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08ac04e000 con 0x7f08ac157480 2023-12-20T06:02:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.628+0000 7f08a1ffb640 1 -- 172.21.15.32:0/953387921 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 125) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f08ac0cd690 con 0x7f08ac157480 2023-12-20T06:02:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.628+0000 7f08a1ffb640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0890065b80 0x7f0890068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.628+0000 7f08a1ffb640 1 -- 172.21.15.32:0/953387921 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f08a411e920 con 0x7f08ac157480 2023-12-20T06:02:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.628+0000 7f08b17c9640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0890065b80 0x7f0890068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:45.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.628+0000 7f08b17c9640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0890065b80 0x7f0890068040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08a8007fd0 tx=0x7f08a8009040 comp rx=0 tx=0).ready entity=mgr.10314 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:45.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.632+0000 7f08a1ffb640 1 -- 172.21.15.32:0/953387921 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f08a40ebf60 con 0x7f08ac157480 2023-12-20T06:02:45.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:45.832+0000 7f08b3a2d640 1 -- 172.21.15.32:0/953387921 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f08ac151660 con 0x7f08ac157480 2023-12-20T06:02:46.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.540+0000 7f08a1ffb640 1 -- 172.21.15.32:0/953387921 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 126) v1 ==== 85347+0+0 (secure 0 0 0) 0x7f08a40ec330 con 0x7f08ac157480 2023-12-20T06:02:46.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.540+0000 7f08a1ffb640 1 -- 172.21.15.32:0/953387921 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v126) v1 ==== 92+0+0 (secure 0 0 0) 0x7f08a40ebf60 con 0x7f08ac157480 2023-12-20T06:02:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 -- 172.21.15.32:0/953387921 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0890065b80 msgr2=0x7f0890068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0890065b80 0x7f0890068040 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f08a8007fd0 tx=0x7f08a8009040 comp rx=0 tx=0).stop 2023-12-20T06:02:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 -- 172.21.15.32:0/953387921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac157480 msgr2=0x7f08ac151120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:46.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac157480 0x7f08ac151120 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7f08a40af040 tx=0x7f08a40196f0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 -- 172.21.15.32:0/953387921 shutdown_connections 2023-12-20T06:02:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0890065b80 0x7f0890068040 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08ac157480 0x7f08ac151120 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08ac0cc4b0 0x7f08ac159510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 --2- 172.21.15.32:0/953387921 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08ac0b0760 0x7f08ac152e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 -- 172.21.15.32:0/953387921 >> 172.21.15.32:0/953387921 conn(0x7f08ac06a510 msgr2=0x7f08ac163f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 -- 172.21.15.32:0/953387921 shutdown_connections 2023-12-20T06:02:46.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.544+0000 7f08b3a2d640 1 -- 172.21.15.32:0/953387921 wait complete. 2023-12-20T06:02:46.566 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T06:02:46.715 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:46.711+0000 7fbce4c9b180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:02:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75ffe9640 1 Processor -- start 2023-12-20T06:02:46.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75ffe9640 1 -- start start 2023-12-20T06:02:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75ffe9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff7580d3300 0x7ff7580d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:46.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75ffe9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7580d1300 0x7ff7580b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75ffe9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff7580d2650 con 0x7ff75815ce60 2023-12-20T06:02:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75ffe9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7580d1a40 con 0x7ff7580d1300 2023-12-20T06:02:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75ffe9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff7580d1d20 con 0x7ff7580d3300 2023-12-20T06:02:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75e586640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7580d1300 0x7ff7580b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75d584640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff7580d3300 0x7ff7580d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75e586640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff7580d1300 0x7ff7580b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46960/0 (socket says 172.21.15.32:46960) 2023-12-20T06:02:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75e586640 1 -- 172.21.15.32:0/2045179702 learned_addr learned my addr 172.21.15.32:0/2045179702 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75dd85640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ff75815ce60 0x7ff7580d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:59522/0 (socket says 172.21.15.32:59522) 2023-12-20T06:02:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75e586640 1 -- 172.21.15.32:0/2045179702 >> v2:172.21.15.39:3300/0 conn(0x7ff7580d3300 msgr2=0x7ff7580d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75e586640 1 --2- 172.21.15.32:0/2045179702 >> v2:172.21.15.39:3300/0 conn(0x7ff7580d3300 0x7ff7580d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75e586640 1 -- 172.21.15.32:0/2045179702 >> v1:172.21.15.39:6789/0 conn(0x7ff75815ce60 legacy=0x7ff7580d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75e586640 1 -- 172.21.15.32:0/2045179702 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff7580d2000 con 0x7ff7580d1300 2023-12-20T06:02:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75cd83640 1 -- 172.21.15.32:0/2045179702 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142324647 0 0) 0x7ff7580d2650 con 0x7ff75815ce60 2023-12-20T06:02:46.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.724+0000 7ff75d584640 1 --2- 172.21.15.32:0/2045179702 >> v2:172.21.15.39:3300/0 conn(0x7ff7580d3300 0x7ff7580d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.728+0000 7ff75e586640 1 --2- 172.21.15.32:0/2045179702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7580d1300 0x7ff7580b5e90 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7ff7580cd240 tx=0x7ff75001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b5f508d0fc7056 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.728+0000 7ff75cd83640 1 -- 172.21.15.32:0/2045179702 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7500a56c0 con 0x7ff7580d1300 2023-12-20T06:02:46.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75cd83640 1 -- 172.21.15.32:0/2045179702 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff7500a5820 con 0x7ff7580d1300 2023-12-20T06:02:46.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75cd83640 1 -- 172.21.15.32:0/2045179702 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff7500a5bb0 con 0x7ff7580d1300 2023-12-20T06:02:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 -- 172.21.15.32:0/2045179702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7580d1300 msgr2=0x7ff7580b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 --2- 172.21.15.32:0/2045179702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7580d1300 0x7ff7580b5e90 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7ff7580cd240 tx=0x7ff75001c9d0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 -- 172.21.15.32:0/2045179702 shutdown_connections 2023-12-20T06:02:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 --2- 172.21.15.32:0/2045179702 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7580d1300 0x7ff7580b5e90 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 --2- 172.21.15.32:0/2045179702 >> v2:172.21.15.39:3300/0 conn(0x7ff7580d3300 0x7ff7580d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 -- 172.21.15.32:0/2045179702 >> 172.21.15.32:0/2045179702 conn(0x7ff75806b5e0 msgr2=0x7ff75806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 -- 172.21.15.32:0/2045179702 shutdown_connections 2023-12-20T06:02:46.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 -- 172.21.15.32:0/2045179702 wait complete. 2023-12-20T06:02:46.731 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:46.732+0000 7fb8ab647180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:02:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 Processor -- start 2023-12-20T06:02:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 -- start start 2023-12-20T06:02:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7580d1300 0x7ff758154550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7580d3300 0x7ff75815b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff75815ce60 0x7ff75815b720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff75815ac70 con 0x7ff7580d3300 2023-12-20T06:02:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff758154a90 con 0x7ff75815ce60 2023-12-20T06:02:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75ffe9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff75815a060 con 0x7ff7580d1300 2023-12-20T06:02:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75d584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7580d3300 0x7ff75815b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75d584640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7580d3300 0x7ff75815b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:46970/0 (socket says 172.21.15.32:46970) 2023-12-20T06:02:46.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75d584640 1 -- 172.21.15.32:0/258554507 learned_addr learned my addr 172.21.15.32:0/258554507 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.732+0000 7ff75dd85640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7580d1300 0x7ff758154550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff75e586640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff75815ce60 0x7ff75815b720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff75d584640 1 -- 172.21.15.32:0/258554507 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7580d1300 msgr2=0x7ff758154550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff75d584640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7580d1300 0x7ff758154550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff75d584640 1 -- 172.21.15.32:0/258554507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff75815ce60 msgr2=0x7ff75815b720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff75d584640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff75815ce60 0x7ff75815b720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff75d584640 1 -- 172.21.15.32:0/258554507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff748009420 con 0x7ff7580d3300 2023-12-20T06:02:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff75d584640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7580d3300 0x7ff75815b1e0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7ff74800ebb0 tx=0x7ff74800c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff74e7fc640 1 -- 172.21.15.32:0/258554507 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff74800eeb0 con 0x7ff7580d3300 2023-12-20T06:02:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff74e7fc640 1 -- 172.21.15.32:0/258554507 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff74800cc30 con 0x7ff7580d3300 2023-12-20T06:02:46.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff75ffe9640 1 -- 172.21.15.32:0/258554507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff750009b30 con 0x7ff7580d3300 2023-12-20T06:02:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff74e7fc640 1 -- 172.21.15.32:0/258554507 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff748013470 con 0x7ff7580d3300 2023-12-20T06:02:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff75ffe9640 1 -- 172.21.15.32:0/258554507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff75815a620 con 0x7ff7580d3300 2023-12-20T06:02:46.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff75ffe9640 1 -- 172.21.15.32:0/258554507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff75804e000 con 0x7ff7580d3300 2023-12-20T06:02:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.736+0000 7ff74e7fc640 1 -- 172.21.15.32:0/258554507 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 126) v1 ==== 85347+0+0 (secure 0 0 0) 0x7ff7480135d0 con 0x7ff7580d3300 2023-12-20T06:02:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.740+0000 7ff74e7fc640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff73c065bd0 0x7ff73c068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.740+0000 7ff74e7fc640 1 -- 172.21.15.32:0/258554507 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7ff7480839a0 con 0x7ff7580d3300 2023-12-20T06:02:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.740+0000 7ff75dd85640 1 -- 172.21.15.32:0/258554507 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff73c065bd0 msgr2=0x7ff73c068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:46.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.740+0000 7ff75dd85640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff73c065bd0 0x7ff73c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:46.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.748+0000 7ff74e7fc640 1 -- 172.21.15.32:0/258554507 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff748050f50 con 0x7ff7580d3300 2023-12-20T06:02:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.908+0000 7ff75ffe9640 1 -- 172.21.15.32:0/258554507 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7ff7580b6f10 con 0x7ff7580d3300 2023-12-20T06:02:46.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.912+0000 7ff74e7fc640 1 -- 172.21.15.32:0/258554507 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v71)=0 v71) v1 ==== 138+0+0 (secure 0 0 0) 0x7ff748054570 con 0x7ff7580d3300 2023-12-20T06:02:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 -- 172.21.15.32:0/258554507 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff73c065bd0 msgr2=0x7ff73c068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff73c065bd0 0x7ff73c068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 -- 172.21.15.32:0/258554507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7580d3300 msgr2=0x7ff75815b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7580d3300 0x7ff75815b1e0 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7ff74800ebb0 tx=0x7ff74800c520 comp rx=0 tx=0).stop 2023-12-20T06:02:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 -- 172.21.15.32:0/258554507 shutdown_connections 2023-12-20T06:02:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff73c065bd0 0x7ff73c068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff75815ce60 0x7ff75815b720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff7580d3300 0x7ff75815b1e0 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 --2- 172.21.15.32:0/258554507 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff7580d1300 0x7ff758154550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 -- 172.21.15.32:0/258554507 >> 172.21.15.32:0/258554507 conn(0x7ff75806b5e0 msgr2=0x7ff75806b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 -- 172.21.15.32:0/258554507 shutdown_connections 2023-12-20T06:02:46.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:46.916+0000 7ff75ffe9640 1 -- 172.21.15.32:0/258554507 wait complete. 2023-12-20T06:02:46.934 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T06:02:47.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.084+0000 7fbbb78d1640 1 Processor -- start 2023-12-20T06:02:47.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.084+0000 7fbbb78d1640 1 -- start start 2023-12-20T06:02:47.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb78d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbbb015ce60 0x7fbbb00d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:47.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb78d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 0x7fbbb00b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:47.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb78d1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb00d2650 con 0x7fbbb00d3130 2023-12-20T06:02:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb78d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb00d35a0 con 0x7fbbb015ce60 2023-12-20T06:02:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb78d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00d1a40 con 0x7fbbb00d14d0 2023-12-20T06:02:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb4e6c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbbb00d3130 0x7fbbb00d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60506/0 (socket says 172.21.15.32:60506) 2023-12-20T06:02:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb4e6c640 1 -- 172.21.15.32:0/2411668457 learned_addr learned my addr 172.21.15.32:0/2411668457 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb566d640 1 --2- 172.21.15.32:0/2411668457 >> v2:172.21.15.39:3300/0 conn(0x7fbbb015ce60 0x7fbbb00d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb5e6e640 1 --2- 172.21.15.32:0/2411668457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 0x7fbbb00b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbba7fff640 1 -- 172.21.15.32:0/2411668457 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2057384025 0 0) 0x7fbbb00d2650 con 0x7fbbb00d3130 2023-12-20T06:02:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbba7fff640 1 -- 172.21.15.32:0/2411668457 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb9c003530 con 0x7fbbb00d3130 2023-12-20T06:02:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb566d640 1 -- 172.21.15.32:0/2411668457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 msgr2=0x7fbbb00b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb566d640 1 --2- 172.21.15.32:0/2411668457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 0x7fbbb00b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb566d640 1 -- 172.21.15.32:0/2411668457 >> v1:172.21.15.32:6789/0 conn(0x7fbbb00d3130 legacy=0x7fbbb00d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:47.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb566d640 1 -- 172.21.15.32:0/2411668457 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb00d1d20 con 0x7fbbb015ce60 2023-12-20T06:02:47.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.088+0000 7fbbb5e6e640 1 --2- 172.21.15.32:0/2411668457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 0x7fbbb00b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:47.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbbb566d640 1 --2- 172.21.15.32:0/2411668457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb015ce60 0x7fbbb00d2bf0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0097c0 tx=0x7fbbac01ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8a7fc1ea692b66e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:47.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbba7fff640 1 -- 172.21.15.32:0/2411668457 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0a8970 con 0x7fbbb015ce60 2023-12-20T06:02:47.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbba7fff640 1 -- 172.21.15.32:0/2411668457 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac0a8ad0 con 0x7fbbb015ce60 2023-12-20T06:02:47.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbba7fff640 1 -- 172.21.15.32:0/2411668457 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0a8e40 con 0x7fbbb015ce60 2023-12-20T06:02:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbbb78d1640 1 -- 172.21.15.32:0/2411668457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb015ce60 msgr2=0x7fbbb00d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbbb78d1640 1 --2- 172.21.15.32:0/2411668457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb015ce60 0x7fbbb00d2bf0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7fbbac0097c0 tx=0x7fbbac01ec70 comp rx=0 tx=0).stop 2023-12-20T06:02:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbbb78d1640 1 -- 172.21.15.32:0/2411668457 shutdown_connections 2023-12-20T06:02:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbbb78d1640 1 --2- 172.21.15.32:0/2411668457 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 0x7fbbb00b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbbb78d1640 1 --2- 172.21.15.32:0/2411668457 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb015ce60 0x7fbbb00d2bf0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbbb78d1640 1 -- 172.21.15.32:0/2411668457 >> 172.21.15.32:0/2411668457 conn(0x7fbbb006b5e0 msgr2=0x7fbbb006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:47.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbbb78d1640 1 -- 172.21.15.32:0/2411668457 shutdown_connections 2023-12-20T06:02:47.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbbb78d1640 1 -- 172.21.15.32:0/2411668457 wait complete. 2023-12-20T06:02:47.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbbb78d1640 1 Processor -- start 2023-12-20T06:02:47.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.092+0000 7fbbb78d1640 1 -- start start 2023-12-20T06:02:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb78d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 0x7fbbb0155e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb78d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb00d3130 0x7fbbb0151c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb78d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb015ce60 0x7fbbb01600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb78d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb00ccc20 con 0x7fbbb015ce60 2023-12-20T06:02:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb78d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb015eb10 con 0x7fbbb00d3130 2023-12-20T06:02:47.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb78d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb01521d0 con 0x7fbbb00d14d0 2023-12-20T06:02:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb566d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 0x7fbbb0155e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb4e6c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb00d3130 0x7fbbb0151c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb566d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 0x7fbbb0155e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:59974/0 (socket says 172.21.15.32:59974) 2023-12-20T06:02:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb566d640 1 -- 172.21.15.32:0/3847554338 learned_addr learned my addr 172.21.15.32:0/3847554338 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb5e6e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb015ce60 0x7fbbb01600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb566d640 1 -- 172.21.15.32:0/3847554338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb00d3130 msgr2=0x7fbbb0151c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb566d640 1 --2- 172.21.15.32:0/3847554338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb00d3130 0x7fbbb0151c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb566d640 1 -- 172.21.15.32:0/3847554338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb015ce60 msgr2=0x7fbbb01600f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb566d640 1 --2- 172.21.15.32:0/3847554338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb015ce60 0x7fbbb01600f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb566d640 1 -- 172.21.15.32:0/3847554338 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba8009a70 con 0x7fbbb00d14d0 2023-12-20T06:02:47.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb566d640 1 --2- 172.21.15.32:0/3847554338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 0x7fbbb0155e20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbbac01c8a0 tx=0x7fbbac01ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbba5ffb640 1 -- 172.21.15.32:0/3847554338 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0b1d50 con 0x7fbbb00d14d0 2023-12-20T06:02:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbba5ffb640 1 -- 172.21.15.32:0/3847554338 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbac0b9410 con 0x7fbbb00d14d0 2023-12-20T06:02:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb78d1640 1 -- 172.21.15.32:0/3847554338 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbac009420 con 0x7fbbb00d14d0 2023-12-20T06:02:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbba5ffb640 1 -- 172.21.15.32:0/3847554338 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbac0b9780 con 0x7fbbb00d14d0 2023-12-20T06:02:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb78d1640 1 -- 172.21.15.32:0/3847554338 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb00cc5d0 con 0x7fbbb00d14d0 2023-12-20T06:02:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.096+0000 7fbbb78d1640 1 -- 172.21.15.32:0/3847554338 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb004e000 con 0x7fbbb00d14d0 2023-12-20T06:02:47.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.100+0000 7fbba5ffb640 1 -- 172.21.15.32:0/3847554338 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 126) v1 ==== 85347+0+0 (secure 0 0 0) 0x7fbbac0b7040 con 0x7fbbb00d14d0 2023-12-20T06:02:47.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.100+0000 7fbba5ffb640 1 --2- 172.21.15.32:0/3847554338 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb94065bd0 0x7fbb94068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:47.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.100+0000 7fbba5ffb640 1 -- 172.21.15.32:0/3847554338 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7fbbac121190 con 0x7fbbb00d14d0 2023-12-20T06:02:47.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.100+0000 7fbbb4e6c640 1 -- 172.21.15.32:0/3847554338 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb94065bd0 msgr2=0x7fbb94068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:47.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.100+0000 7fbbb4e6c640 1 --2- 172.21.15.32:0/3847554338 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb94065bd0 0x7fbb94068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:47.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.104+0000 7fbba5ffb640 1 -- 172.21.15.32:0/3847554338 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbbac0ee6b0 con 0x7fbbb00d14d0 2023-12-20T06:02:47.155 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:47.151+0000 7fbce4c9b180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:02:47.183 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:47.184+0000 7fb8ab647180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:02:47.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.264+0000 7fbbb78d1640 1 -- 172.21.15.32:0/3847554338 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7fbbb00b52d0 con 0x7fbbb00d14d0 2023-12-20T06:02:47.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.268+0000 7fbba5ffb640 1 -- 172.21.15.32:0/3847554338 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v72)=0 v72) v1 ==== 133+0+0 (secure 0 0 0) 0x7fbbac0f1cd0 con 0x7fbbb00d14d0 2023-12-20T06:02:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 -- 172.21.15.32:0/3847554338 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb94065bd0 msgr2=0x7fbb94068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 --2- 172.21.15.32:0/3847554338 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb94065bd0 0x7fbb94068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 -- 172.21.15.32:0/3847554338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 msgr2=0x7fbbb0155e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 --2- 172.21.15.32:0/3847554338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 0x7fbbb0155e20 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fbbac01c8a0 tx=0x7fbbac01ba10 comp rx=0 tx=0).stop 2023-12-20T06:02:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 -- 172.21.15.32:0/3847554338 shutdown_connections 2023-12-20T06:02:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 --2- 172.21.15.32:0/3847554338 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb94065bd0 0x7fbb94068090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 --2- 172.21.15.32:0/3847554338 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb015ce60 0x7fbbb01600f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 --2- 172.21.15.32:0/3847554338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb00d3130 0x7fbbb0151c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 --2- 172.21.15.32:0/3847554338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb00d14d0 0x7fbbb0155e20 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 -- 172.21.15.32:0/3847554338 >> 172.21.15.32:0/3847554338 conn(0x7fbbb006b5e0 msgr2=0x7fbbb00cd7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 -- 172.21.15.32:0/3847554338 shutdown_connections 2023-12-20T06:02:47.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.272+0000 7fbbb78d1640 1 -- 172.21.15.32:0/3847554338 wait complete. 2023-12-20T06:02:47.290 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:47.290 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-12-20T06:02:47.305 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-20T06:02:47.314 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:47.311+0000 7fbce4c9b180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:02:47.345 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:47.348+0000 7fb8ab647180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:02:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa0cbce640 1 Processor -- start 2023-12-20T06:02:47.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa0cbce640 1 -- start start 2023-12-20T06:02:47.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa0cbce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ffa080b1d90 0x7ffa080b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa0cbce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa080d1030 con 0x7ffa080b62d0 2023-12-20T06:02:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa0cbce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffa080d1280 con 0x7ffa081603e0 2023-12-20T06:02:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa0cbce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffa080d1560 con 0x7ffa080b1d90 2023-12-20T06:02:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa05d9b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ffa080b1d90 0x7ffa080b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa0659c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7ffa081603e0 0x7ffa080b8580 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:59532/0 (socket says 172.21.15.32:59532) 2023-12-20T06:02:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa0659c640 1 -- 172.21.15.32:0/2251767673 learned_addr learned my addr 172.21.15.32:0/2251767673 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:47.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa06d9d640 1 --1- 172.21.15.32:0/2251767673 >> v1:172.21.15.32:6789/0 conn(0x7ffa080b62d0 0x7ffa080d2040 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:60512/0 (socket says 172.21.15.32:60512) 2023-12-20T06:02:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa05d9b640 1 -- 172.21.15.32:0/2251767673 >> v1:172.21.15.39:6789/0 conn(0x7ffa081603e0 legacy=0x7ffa080b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa05d9b640 1 -- 172.21.15.32:0/2251767673 >> v1:172.21.15.32:6789/0 conn(0x7ffa080b62d0 legacy=0x7ffa080d2040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa0559a640 1 -- 172.21.15.32:0/2251767673 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 204983861 0 0) 0x7ffa080d1280 con 0x7ffa081603e0 2023-12-20T06:02:47.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.500+0000 7ffa05d9b640 1 -- 172.21.15.32:0/2251767673 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffa080d1840 con 0x7ffa080b1d90 2023-12-20T06:02:47.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.504+0000 7ffa05d9b640 1 --2- 172.21.15.32:0/2251767673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa080b1d90 0x7ffa080b5d90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0097c0 tx=0x7ff9fc01c280 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ddee6e04a93bd095 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:47.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.504+0000 7ffa0559a640 1 -- 172.21.15.32:0/2251767673 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0ab600 con 0x7ffa080b1d90 2023-12-20T06:02:47.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.504+0000 7ffa0559a640 1 -- 172.21.15.32:0/2251767673 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9fc0a7050 con 0x7ffa080b1d90 2023-12-20T06:02:47.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.504+0000 7ffa0559a640 1 -- 172.21.15.32:0/2251767673 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0ab980 con 0x7ffa080b1d90 2023-12-20T06:02:47.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.504+0000 7ffa0cbce640 1 -- 172.21.15.32:0/2251767673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa080b1d90 msgr2=0x7ffa080b5d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:47.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.504+0000 7ffa0cbce640 1 --2- 172.21.15.32:0/2251767673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa080b1d90 0x7ffa080b5d90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0097c0 tx=0x7ff9fc01c280 comp rx=0 tx=0).stop 2023-12-20T06:02:47.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.504+0000 7ffa0cbce640 1 -- 172.21.15.32:0/2251767673 shutdown_connections 2023-12-20T06:02:47.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.504+0000 7ffa0cbce640 1 --2- 172.21.15.32:0/2251767673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa080b1d90 0x7ffa080b5d90 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.504+0000 7ffa0cbce640 1 -- 172.21.15.32:0/2251767673 >> 172.21.15.32:0/2251767673 conn(0x7ffa0806b5e0 msgr2=0x7ffa0806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:47.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.504+0000 7ffa0cbce640 1 -- 172.21.15.32:0/2251767673 shutdown_connections 2023-12-20T06:02:47.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.504+0000 7ffa0cbce640 1 -- 172.21.15.32:0/2251767673 wait complete. 2023-12-20T06:02:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0cbce640 1 Processor -- start 2023-12-20T06:02:47.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0cbce640 1 -- start start 2023-12-20T06:02:47.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0cbce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa080b1d90 0x7ffa080b2240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:47.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0cbce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa080b62d0 0x7ffa081f6550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:47.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0cbce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa081603e0 0x7ffa0814aeb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:47.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0cbce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa081f6a90 con 0x7ffa080b1d90 2023-12-20T06:02:47.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0cbce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080b2860 con 0x7ffa080b62d0 2023-12-20T06:02:47.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0cbce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffa080b2b40 con 0x7ffa081603e0 2023-12-20T06:02:47.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa080b1d90 0x7ffa080b2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:47.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa080b1d90 0x7ffa080b2240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47000/0 (socket says 172.21.15.32:47000) 2023-12-20T06:02:47.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0659c640 1 -- 172.21.15.32:0/1570767022 learned_addr learned my addr 172.21.15.32:0/1570767022 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:47.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa05d9b640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa080b62d0 0x7ffa081f6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:47.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa06d9d640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa081603e0 0x7ffa0814aeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:47.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0659c640 1 -- 172.21.15.32:0/1570767022 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa081603e0 msgr2=0x7ffa0814aeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:47.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0659c640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa081603e0 0x7ffa0814aeb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0659c640 1 -- 172.21.15.32:0/1570767022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa080b62d0 msgr2=0x7ffa081f6550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0659c640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa080b62d0 0x7ffa081f6550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0659c640 1 -- 172.21.15.32:0/1570767022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff9fc009420 con 0x7ffa080b1d90 2023-12-20T06:02:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa06d9d640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa081603e0 0x7ffa0814aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0659c640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa080b1d90 0x7ffa080b2240 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800c4a0 tx=0x7ff9f800c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ff9f6ffd640 1 -- 172.21.15.32:0/1570767022 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f80074e0 con 0x7ffa080b1d90 2023-12-20T06:02:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0cbce640 1 -- 172.21.15.32:0/1570767022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffa080b2e20 con 0x7ffa080b1d90 2023-12-20T06:02:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ff9f6ffd640 1 -- 172.21.15.32:0/1570767022 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9f8007640 con 0x7ffa080b1d90 2023-12-20T06:02:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ff9f6ffd640 1 -- 172.21.15.32:0/1570767022 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9f8004a30 con 0x7ffa080b1d90 2023-12-20T06:02:47.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.508+0000 7ffa0cbce640 1 -- 172.21.15.32:0/1570767022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffa080b33e0 con 0x7ffa080b1d90 2023-12-20T06:02:47.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.512+0000 7ffa0cbce640 1 -- 172.21.15.32:0/1570767022 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffa0804e000 con 0x7ffa080b1d90 2023-12-20T06:02:47.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.512+0000 7ff9f6ffd640 1 -- 172.21.15.32:0/1570767022 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 126) v1 ==== 85347+0+0 (secure 0 0 0) 0x7ff9f8004b90 con 0x7ffa080b1d90 2023-12-20T06:02:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.512+0000 7ff9f6ffd640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.512+0000 7ff9f6ffd640 1 -- 172.21.15.32:0/1570767022 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(213..213 src has 1..213) v4 ==== 6629+0+0 (secure 0 0 0) 0x7ff9f80828b0 con 0x7ffa080b1d90 2023-12-20T06:02:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.512+0000 7ffa05d9b640 1 -- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 msgr2=0x7ff9e8068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:47.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.512+0000 7ffa05d9b640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:47.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.516+0000 7ff9f6ffd640 1 -- 172.21.15.32:0/1570767022 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff9f804fe80 con 0x7ffa080b1d90 2023-12-20T06:02:47.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.696+0000 7ffa0cbce640 1 -- 172.21.15.32:0/1570767022 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ffa080d1b20 con 0x7ff9e8065bd0 2023-12-20T06:02:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.716+0000 7ffa05d9b640 1 -- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 msgr2=0x7ff9e8068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:47.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:47.716+0000 7ffa05d9b640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T06:02:47.826 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:47.823+0000 7fbce4c9b180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:47.862 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:47.864+0000 7fb8ab647180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:47.888 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:47.883+0000 7fbce4c9b180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:02:47.924 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:47.924+0000 7fb8ab647180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:02:47.947 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:47.943+0000 7fbce4c9b180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:02:47.982 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:47.984+0000 7fb8ab647180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:02:48.039 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:48.035+0000 7fbce4c9b180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:02:48.072 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:48.075+0000 7fb8ab647180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:02:48.098 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:48.095+0000 7fbce4c9b180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:02:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:48.119+0000 7ffa05d9b640 1 -- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 msgr2=0x7ff9e8068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:48.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:48.119+0000 7ffa05d9b640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T06:02:48.132 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:48.131+0000 7fb8ab647180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:02:48.187 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:48.183+0000 7fbce4c9b180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:02:48.220 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:48.219+0000 7fb8ab647180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:02:48.246 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:48.243+0000 7fbce4c9b180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:02:48.279 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:48.279+0000 7fb8ab647180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:02:48.398 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:48.395+0000 7fbce4c9b180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:02:48.429 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:48.431+0000 7fb8ab647180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:02:48.460 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:48.455+0000 7fbce4c9b180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:02:48.496 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:48.495+0000 7fb8ab647180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:02:48.711 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:02:48.711 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:02:48.711 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:02:48.719 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:48.715+0000 7fbce4c9b180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:02:48.754 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:02:48.754 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:02:48.754 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:02:48.763 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:48.763+0000 7fb8ab647180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:02:48.862 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:48.859+0000 7fbce4c9b180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:48.906 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:48.907+0000 7fb8ab647180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:48.923+0000 7ffa05d9b640 1 -- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 msgr2=0x7ff9e8068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:48.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:48.923+0000 7ffa05d9b640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T06:02:48.994 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:48.991+0000 7fbce4c9b180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:49.037 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:49.039+0000 7fb8ab647180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:49.068 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:49.063+0000 7fbce4c9b180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:02:49.111 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:49.111+0000 7fb8ab647180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:02:49.211 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:49.207+0000 7fbce4c9b180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:49.254 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:49.255+0000 7fb8ab647180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:49.783 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:49.779+0000 7fbce4c9b180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:02:49.820 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:49.823+0000 7fb8ab647180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:02:49.872 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:49.867+0000 7fbce4c9b180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:02:49.909 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:49.911+0000 7fb8ab647180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:02:50.029 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:50.023+0000 7fbce4c9b180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:02:50.066 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.067+0000 7fb8ab647180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:02:50.117 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:50.111+0000 7fbce4c9b180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:02:50.154 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.155+0000 7fb8ab647180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:02:50.453 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:50.447+0000 7fbce4c9b180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:02:50.490 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.491+0000 7fb8ab647180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:02:50.517 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:50.511+0000 7fbce4c9b180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:02:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:50.527+0000 7ffa05d9b640 1 -- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 msgr2=0x7ff9e8068090 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:50.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:50.527+0000 7ffa05d9b640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-20T06:02:50.555 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.555+0000 7fb8ab647180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:02:50.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:50.567+0000 7ff9f6ffd640 1 -- 172.21.15.32:0/1570767022 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 127) v1 ==== 84956+0+0 (secure 0 0 0) 0x7ff9f8050250 con 0x7ffa080b1d90 2023-12-20T06:02:50.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:50.567+0000 7ff9f6ffd640 1 -- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 msgr2=0x7ff9e8068090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:50.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:50.567+0000 7ff9f6ffd640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:50.729 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.731+0000 7fb830130640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.729 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.731+0000 7fb830130640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.729 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.731+0000 7fb830130640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.729 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.731+0000 7fb830130640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.729 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.731+0000 7fb830130640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.730 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.731+0000 7fb82d12a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.730 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.731+0000 7fb82d12a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.730 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.731+0000 7fb82d12a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.730 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.731+0000 7fb82d12a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.730 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.731+0000 7fb82d12a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.741 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.743+0000 7fb830130640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.741 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.743+0000 7fb830130640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.741 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.743+0000 7fb830130640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.742 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.743+0000 7fb830130640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.742 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.743+0000 7fb830130640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.742 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.743+0000 7fb82d12a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.742 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.743+0000 7fb82d12a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.742 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.743+0000 7fb82d12a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.742 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.743+0000 7fb82d12a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:50.742 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:50.743+0000 7fb82d12a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:51.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.591+0000 7ff9f6ffd640 1 -- 172.21.15.32:0/1570767022 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 128) v1 ==== 85083+0+0 (secure 0 0 0) 0x7ff9f8080a00 con 0x7ffa080b1d90 2023-12-20T06:02:51.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.591+0000 7ff9f6ffd640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:51.592 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.591+0000 7ff9f6ffd640 1 -- 172.21.15.32:0/1570767022 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7ff9e8033630 con 0x7ff9e8065bd0 2023-12-20T06:02:51.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.595+0000 7ffa05d9b640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:51.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.599+0000 7ffa05d9b640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0ad000 tx=0x7ff9fc0a6000 comp rx=0 tx=0).ready entity=mgr.10824 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:51.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.659+0000 7ff9f6ffd640 1 -- 172.21.15.32:0/1570767022 <== mgr.10824 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+53 (secure 0 0 0) 0x7ff9e8033630 con 0x7ff9e8065bd0 2023-12-20T06:02:51.660 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path mandatory_fs_argument_test_dir 2023-12-20T06:02:51.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 -- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 msgr2=0x7ff9e8068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:51.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0ad000 tx=0x7ff9fc0a6000 comp rx=0 tx=0).stop 2023-12-20T06:02:51.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 -- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa080b1d90 msgr2=0x7ffa080b2240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:51.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa080b1d90 0x7ffa080b2240 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7ff9f800c4a0 tx=0x7ff9f800c960 comp rx=0 tx=0).stop 2023-12-20T06:02:51.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 -- 172.21.15.32:0/1570767022 shutdown_connections 2023-12-20T06:02:51.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff9e8065bd0 0x7ff9e8068090 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:51.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffa081603e0 0x7ffa0814aeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:51.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffa080b62d0 0x7ffa081f6550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:51.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 --2- 172.21.15.32:0/1570767022 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffa080b1d90 0x7ffa080b2240 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:51.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 -- 172.21.15.32:0/1570767022 >> 172.21.15.32:0/1570767022 conn(0x7ffa0806b5e0 msgr2=0x7ffa080cd070 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:51.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 -- 172.21.15.32:0/1570767022 shutdown_connections 2023-12-20T06:02:51.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.663+0000 7ffa0cbce640 1 -- 172.21.15.32:0/1570767022 wait complete. 2023-12-20T06:02:51.683 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path mandatory_fs_argument_test_dir 2023-12-20T06:02:51.683 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule activate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-20T06:02:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b82f2640 1 Processor -- start 2023-12-20T06:02:51.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b82f2640 1 -- start start 2023-12-20T06:02:51.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b82f2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb3b0159af0 0x7fb3b00b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:51.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b82f2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3b00b0f00 0x7fb3b006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b82f2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb3b006dd50 con 0x7fb3b006d2d0 2023-12-20T06:02:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b82f2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b00cc940 con 0x7fb3b00b0f00 2023-12-20T06:02:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b82f2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3b00ccc20 con 0x7fb3b0159af0 2023-12-20T06:02:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b688f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3b00b0f00 0x7fb3b006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b688f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3b00b0f00 0x7fb3b006d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:59996/0 (socket says 172.21.15.32:59996) 2023-12-20T06:02:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b688f640 1 -- 172.21.15.32:0/2023549940 learned_addr learned my addr 172.21.15.32:0/2023549940 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b608e640 1 --2- 172.21.15.32:0/2023549940 >> v2:172.21.15.39:3300/0 conn(0x7fb3b0159af0 0x7fb3b00b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.835+0000 7fb3b688f640 1 -- 172.21.15.32:0/2023549940 >> v2:172.21.15.39:3300/0 conn(0x7fb3b0159af0 msgr2=0x7fb3b00b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.839+0000 7fb3b688f640 1 --2- 172.21.15.32:0/2023549940 >> v2:172.21.15.39:3300/0 conn(0x7fb3b0159af0 0x7fb3b00b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.839+0000 7fb3b688f640 1 -- 172.21.15.32:0/2023549940 >> v1:172.21.15.39:6789/0 conn(0x7fb3b006d2d0 legacy=0x7fb3b00b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:51.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.839+0000 7fb3b688f640 1 -- 172.21.15.32:0/2023549940 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3b00ccf00 con 0x7fb3b00b0f00 2023-12-20T06:02:51.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.839+0000 7fb3b688f640 1 --2- 172.21.15.32:0/2023549940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b00b0f00 0x7fb3b006d740 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00c8de0 tx=0x7fb3a801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b19838fc3b3c99c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:51.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.839+0000 7fb3b508c640 1 -- 172.21.15.32:0/2023549940 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80a5880 con 0x7fb3b00b0f00 2023-12-20T06:02:51.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.839+0000 7fb3b508c640 1 -- 172.21.15.32:0/2023549940 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3a80a59e0 con 0x7fb3b00b0f00 2023-12-20T06:02:51.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.839+0000 7fb3b508c640 1 -- 172.21.15.32:0/2023549940 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3a80a5d70 con 0x7fb3b00b0f00 2023-12-20T06:02:51.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.839+0000 7fb3b82f2640 1 -- 172.21.15.32:0/2023549940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b00b0f00 msgr2=0x7fb3b006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:51.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.839+0000 7fb3b82f2640 1 --2- 172.21.15.32:0/2023549940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b00b0f00 0x7fb3b006d740 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7fb3b00c8de0 tx=0x7fb3a801c9a0 comp rx=0 tx=0).stop 2023-12-20T06:02:51.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 -- 172.21.15.32:0/2023549940 shutdown_connections 2023-12-20T06:02:51.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 --2- 172.21.15.32:0/2023549940 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b00b0f00 0x7fb3b006d740 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:51.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 --2- 172.21.15.32:0/2023549940 >> v2:172.21.15.39:3300/0 conn(0x7fb3b0159af0 0x7fb3b00b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:51.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 -- 172.21.15.32:0/2023549940 >> 172.21.15.32:0/2023549940 conn(0x7fb3b006a510 msgr2=0x7fb3b00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:51.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 -- 172.21.15.32:0/2023549940 shutdown_connections 2023-12-20T06:02:51.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 -- 172.21.15.32:0/2023549940 wait complete. 2023-12-20T06:02:51.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 Processor -- start 2023-12-20T06:02:51.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 -- start start 2023-12-20T06:02:51.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006d2d0 0x7fb3b0146c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:51.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3b00b0f00 0x7fb3b014b590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3b0159af0 0x7fb3b014bad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b01471c0 con 0x7fb3b006d2d0 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b0147690 con 0x7fb3b0159af0 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b82f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb3b014c0e0 con 0x7fb3b00b0f00 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b608e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006d2d0 0x7fb3b0146c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b608e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006d2d0 0x7fb3b0146c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60010/0 (socket says 172.21.15.32:60010) 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b608e640 1 -- 172.21.15.32:0/3941463427 learned_addr learned my addr 172.21.15.32:0/3941463427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b688f640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3b0159af0 0x7fb3b014bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b588d640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3b00b0f00 0x7fb3b014b590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b608e640 1 -- 172.21.15.32:0/3941463427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3b00b0f00 msgr2=0x7fb3b014b590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b608e640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3b00b0f00 0x7fb3b014b590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b608e640 1 -- 172.21.15.32:0/3941463427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3b0159af0 msgr2=0x7fb3b014bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b608e640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3b0159af0 0x7fb3b014bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:51.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b608e640 1 -- 172.21.15.32:0/3941463427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3a8009b30 con 0x7fb3b006d2d0 2023-12-20T06:02:51.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.843+0000 7fb3b608e640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006d2d0 0x7fb3b0146c80 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00e870 tx=0x7fb3ac00ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:51.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.847+0000 7fb3a6ffd640 1 -- 172.21.15.32:0/3941463427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac009510 con 0x7fb3b006d2d0 2023-12-20T06:02:51.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.847+0000 7fb3a6ffd640 1 -- 172.21.15.32:0/3941463427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3ac00eec0 con 0x7fb3b006d2d0 2023-12-20T06:02:51.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.847+0000 7fb3b82f2640 1 -- 172.21.15.32:0/3941463427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3b014c3c0 con 0x7fb3b006d2d0 2023-12-20T06:02:51.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.847+0000 7fb3a6ffd640 1 -- 172.21.15.32:0/3941463427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3ac00cb40 con 0x7fb3b006d2d0 2023-12-20T06:02:51.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.847+0000 7fb3b82f2640 1 -- 172.21.15.32:0/3941463427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3b014c980 con 0x7fb3b006d2d0 2023-12-20T06:02:51.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.847+0000 7fb3b82f2640 1 -- 172.21.15.32:0/3941463427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb3b00b0ce0 con 0x7fb3b006d2d0 2023-12-20T06:02:51.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.847+0000 7fb3a6ffd640 1 -- 172.21.15.32:0/3941463427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 128) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fb3ac00cca0 con 0x7fb3b006d2d0 2023-12-20T06:02:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.847+0000 7fb3a6ffd640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb394065900 0x7fb394067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.847+0000 7fb3a6ffd640 1 -- 172.21.15.32:0/3941463427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7fb3ac083410 con 0x7fb3b006d2d0 2023-12-20T06:02:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.847+0000 7fb3b588d640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb394065900 0x7fb394067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:51.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.851+0000 7fb3b588d640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb394065900 0x7fb394067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb3b01579e0 tx=0x7fb3a0009040 comp rx=0 tx=0).ready entity=mgr.10824 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:51.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:51.851+0000 7fb3a6ffd640 1 -- 172.21.15.32:0/3941463427 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb3ac050c20 con 0x7fb3b006d2d0 2023-12-20T06:02:52.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.035+0000 7fb3b82f2640 1 -- 172.21.15.32:0/3941463427 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule activate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb3b00cdf10 con 0x7fb394065900 2023-12-20T06:02:52.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3a6ffd640 1 -- 172.21.15.32:0/3941463427 <== mgr.10824 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7fb3a0001370 con 0x7fb394065900 2023-12-20T06:02:52.077 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule activated for path mandatory_fs_argument_test_dir 2023-12-20T06:02:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 -- 172.21.15.32:0/3941463427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb394065900 msgr2=0x7fb394067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb394065900 0x7fb394067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb3b01579e0 tx=0x7fb3a0009040 comp rx=0 tx=0).stop 2023-12-20T06:02:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 -- 172.21.15.32:0/3941463427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006d2d0 msgr2=0x7fb3b0146c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006d2d0 0x7fb3b0146c80 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7fb3ac00e870 tx=0x7fb3ac00ec70 comp rx=0 tx=0).stop 2023-12-20T06:02:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 -- 172.21.15.32:0/3941463427 shutdown_connections 2023-12-20T06:02:52.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb394065900 0x7fb394067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb3b0159af0 0x7fb3b014bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3b00b0f00 0x7fb3b014b590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 --2- 172.21.15.32:0/3941463427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3b006d2d0 0x7fb3b0146c80 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 -- 172.21.15.32:0/3941463427 >> 172.21.15.32:0/3941463427 conn(0x7fb3b006a510 msgr2=0x7fb3b0158100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 -- 172.21.15.32:0/3941463427 shutdown_connections 2023-12-20T06:02:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.079+0000 7fb3b82f2640 1 -- 172.21.15.32:0/3941463427 wait complete. 2023-12-20T06:02:52.099 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule activated for path mandatory_fs_argument_test_dir 2023-12-20T06:02:52.099 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-20T06:02:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf38db3640 1 Processor -- start 2023-12-20T06:02:52.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf38db3640 1 -- start start 2023-12-20T06:02:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf38db3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbf340cb8d0 0x7fbf340cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf38db3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf340cc9d0 con 0x7fbf3415ad20 2023-12-20T06:02:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf38db3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf340c9b90 con 0x7fbf3415a940 2023-12-20T06:02:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf38db3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf340c9e70 con 0x7fbf340cb8d0 2023-12-20T06:02:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf31d9b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbf3415ad20 0x7fbf340cb1c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42496/0 (socket says 172.21.15.32:42496) 2023-12-20T06:02:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf31d9b640 1 -- 172.21.15.32:0/959562922 learned_addr learned my addr 172.21.15.32:0/959562922 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf32d9d640 1 --2- 172.21.15.32:0/959562922 >> v2:172.21.15.39:3300/0 conn(0x7fbf340cb8d0 0x7fbf340cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf3159a640 1 -- 172.21.15.32:0/959562922 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1529619890 0 0) 0x7fbf340cc9d0 con 0x7fbf3415ad20 2023-12-20T06:02:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf3159a640 1 -- 172.21.15.32:0/959562922 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf1c003530 con 0x7fbf3415ad20 2023-12-20T06:02:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf3159a640 1 -- 172.21.15.32:0/959562922 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2704497257 0 0) 0x7fbf340c9b90 con 0x7fbf3415a940 2023-12-20T06:02:52.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf3159a640 1 -- 172.21.15.32:0/959562922 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf340cc9d0 con 0x7fbf3415a940 2023-12-20T06:02:52.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf32d9d640 1 -- 172.21.15.32:0/959562922 >> v1:172.21.15.39:6789/0 conn(0x7fbf3415a940 legacy=0x7fbf340ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf32d9d640 1 -- 172.21.15.32:0/959562922 >> v1:172.21.15.32:6789/0 conn(0x7fbf3415ad20 legacy=0x7fbf340cb1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.251+0000 7fbf32d9d640 1 -- 172.21.15.32:0/959562922 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf340ca150 con 0x7fbf340cb8d0 2023-12-20T06:02:52.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.255+0000 7fbf32d9d640 1 --2- 172.21.15.32:0/959562922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf340cb8d0 0x7fbf340cc380 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009dc0 tx=0x7fbf2c01c870 comp rx=0 tx=0).ready entity=mon.1 client_cookie=13c380ee417da65e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:52.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.255+0000 7fbf3159a640 1 -- 172.21.15.32:0/959562922 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c0a5950 con 0x7fbf340cb8d0 2023-12-20T06:02:52.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.255+0000 7fbf3159a640 1 -- 172.21.15.32:0/959562922 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf2c0a5ab0 con 0x7fbf340cb8d0 2023-12-20T06:02:52.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.255+0000 7fbf3159a640 1 -- 172.21.15.32:0/959562922 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf2c0a5e40 con 0x7fbf340cb8d0 2023-12-20T06:02:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.255+0000 7fbf38db3640 1 -- 172.21.15.32:0/959562922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf340cb8d0 msgr2=0x7fbf340cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.255+0000 7fbf38db3640 1 --2- 172.21.15.32:0/959562922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf340cb8d0 0x7fbf340cc380 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7fbf2c009dc0 tx=0x7fbf2c01c870 comp rx=0 tx=0).stop 2023-12-20T06:02:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.255+0000 7fbf38db3640 1 -- 172.21.15.32:0/959562922 shutdown_connections 2023-12-20T06:02:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.255+0000 7fbf38db3640 1 --2- 172.21.15.32:0/959562922 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf340cb8d0 0x7fbf340cc380 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.255+0000 7fbf38db3640 1 -- 172.21.15.32:0/959562922 >> 172.21.15.32:0/959562922 conn(0x7fbf3406a4b0 msgr2=0x7fbf340b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.255+0000 7fbf38db3640 1 -- 172.21.15.32:0/959562922 shutdown_connections 2023-12-20T06:02:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.255+0000 7fbf38db3640 1 -- 172.21.15.32:0/959562922 wait complete. 2023-12-20T06:02:52.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf38db3640 1 Processor -- start 2023-12-20T06:02:52.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf38db3640 1 -- start start 2023-12-20T06:02:52.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf38db3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf340cb8d0 0x7fbf340c51f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:52.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf38db3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3415a940 0x7fbf34158040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:52.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf38db3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3415ad20 0x7fbf3413a510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:52.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf38db3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf34158580 con 0x7fbf3415a940 2023-12-20T06:02:52.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf38db3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf34158d30 con 0x7fbf340cb8d0 2023-12-20T06:02:52.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf38db3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf34158770 con 0x7fbf3415ad20 2023-12-20T06:02:52.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf31d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3415a940 0x7fbf34158040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:52.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf31d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3415a940 0x7fbf34158040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60022/0 (socket says 172.21.15.32:60022) 2023-12-20T06:02:52.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf31d9b640 1 -- 172.21.15.32:0/2862610519 learned_addr learned my addr 172.21.15.32:0/2862610519 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:52.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf3259c640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf340cb8d0 0x7fbf340c51f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:52.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf32d9d640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3415ad20 0x7fbf3413a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:52.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf31d9b640 1 -- 172.21.15.32:0/2862610519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3415ad20 msgr2=0x7fbf3413a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf31d9b640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3415ad20 0x7fbf3413a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf31d9b640 1 -- 172.21.15.32:0/2862610519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf340cb8d0 msgr2=0x7fbf340c51f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf31d9b640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf340cb8d0 0x7fbf340c51f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf31d9b640 1 -- 172.21.15.32:0/2862610519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf2c009a70 con 0x7fbf3415a940 2023-12-20T06:02:52.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf31d9b640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3415a940 0x7fbf34158040 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c420 tx=0x7fbf2800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:52.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf22ffd640 1 -- 172.21.15.32:0/2862610519 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf28005cf0 con 0x7fbf3415a940 2023-12-20T06:02:52.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf22ffd640 1 -- 172.21.15.32:0/2862610519 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf28005e50 con 0x7fbf3415a940 2023-12-20T06:02:52.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf22ffd640 1 -- 172.21.15.32:0/2862610519 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf280072d0 con 0x7fbf3415a940 2023-12-20T06:02:52.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf38db3640 1 -- 172.21.15.32:0/2862610519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf34158a50 con 0x7fbf3415a940 2023-12-20T06:02:52.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.259+0000 7fbf38db3640 1 -- 172.21.15.32:0/2862610519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf34150130 con 0x7fbf3415a940 2023-12-20T06:02:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.263+0000 7fbf38db3640 1 -- 172.21.15.32:0/2862610519 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf3404e000 con 0x7fbf3415a940 2023-12-20T06:02:52.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.263+0000 7fbf22ffd640 1 -- 172.21.15.32:0/2862610519 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 128) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fbf28007430 con 0x7fbf3415a940 2023-12-20T06:02:52.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.263+0000 7fbf22ffd640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf14065930 0x7fbf14067df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:52.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.263+0000 7fbf22ffd640 1 -- 172.21.15.32:0/2862610519 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7fbf28082880 con 0x7fbf3415a940 2023-12-20T06:02:52.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.263+0000 7fbf3259c640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf14065930 0x7fbf14067df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:52.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.263+0000 7fbf3259c640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf14065930 0x7fbf14067df0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbf240017c0 tx=0x7fbf24009040 comp rx=0 tx=0).ready entity=mgr.10824 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:52.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.267+0000 7fbf22ffd640 1 -- 172.21.15.32:0/2862610519 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbf2804fff0 con 0x7fbf3415a940 2023-12-20T06:02:52.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.443+0000 7fbf38db3640 1 -- 172.21.15.32:0/2862610519 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf3406ad00 con 0x7fbf14065930 2023-12-20T06:02:52.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.503+0000 7fbf22ffd640 1 -- 172.21.15.32:0/2862610519 <== mgr.10824 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+55 (secure 0 0 0) 0x7fbf24001460 con 0x7fbf14065930 2023-12-20T06:02:52.503 INFO:teuthology.orchestra.run.smithi032.stdout:Retention added to path mandatory_fs_argument_test_dir 2023-12-20T06:02:52.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.503+0000 7fbf38db3640 1 -- 172.21.15.32:0/2862610519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf14065930 msgr2=0x7fbf14067df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.503+0000 7fbf38db3640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf14065930 0x7fbf14067df0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fbf240017c0 tx=0x7fbf24009040 comp rx=0 tx=0).stop 2023-12-20T06:02:52.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.507+0000 7fbf38db3640 1 -- 172.21.15.32:0/2862610519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3415a940 msgr2=0x7fbf34158040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.507+0000 7fbf38db3640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3415a940 0x7fbf34158040 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7fbf2800c420 tx=0x7fbf2800c8e0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.507+0000 7fbf38db3640 1 -- 172.21.15.32:0/2862610519 shutdown_connections 2023-12-20T06:02:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.507+0000 7fbf38db3640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf14065930 0x7fbf14067df0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.507+0000 7fbf38db3640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf3415ad20 0x7fbf3413a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.507+0000 7fbf38db3640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf3415a940 0x7fbf34158040 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.507+0000 7fbf38db3640 1 --2- 172.21.15.32:0/2862610519 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf340cb8d0 0x7fbf340c51f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.507+0000 7fbf38db3640 1 -- 172.21.15.32:0/2862610519 >> 172.21.15.32:0/2862610519 conn(0x7fbf3406a4b0 msgr2=0x7fbf34158f50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.507+0000 7fbf38db3640 1 -- 172.21.15.32:0/2862610519 shutdown_connections 2023-12-20T06:02:52.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.507+0000 7fbf38db3640 1 -- 172.21.15.32:0/2862610519 wait complete. 2023-12-20T06:02:52.525 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path mandatory_fs_argument_test_dir 2023-12-20T06:02:52.525 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-12-20T06:02:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e759e640 1 Processor -- start 2023-12-20T06:02:52.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e759e640 1 -- start start 2023-12-20T06:02:52.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e759e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e8159770 0x7f33e80ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:52.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e759e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f33e80cc130 0x7f33e8159b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e759e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33e80ce070 con 0x7f33e80affe0 2023-12-20T06:02:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e759e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33e80cacd0 con 0x7f33e80cc130 2023-12-20T06:02:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e759e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e815a480 con 0x7f33e8159770 2023-12-20T06:02:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e6d9d640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f33e80affe0 0x7f33e80717b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:43678/0 (socket says 172.21.15.32:43678) 2023-12-20T06:02:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e6d9d640 1 -- 172.21.15.32:0/4024223127 learned_addr learned my addr 172.21.15.32:0/4024223127 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e5d9b640 1 --2- 172.21.15.32:0/4024223127 >> v2:172.21.15.39:3300/0 conn(0x7f33e80cc130 0x7f33e8159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e659c640 1 --2- 172.21.15.32:0/4024223127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e8159770 0x7f33e80ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e559a640 1 -- 172.21.15.32:0/4024223127 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1132783156 0 0) 0x7f33e80ce070 con 0x7f33e80affe0 2023-12-20T06:02:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e559a640 1 -- 172.21.15.32:0/4024223127 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f33d0003530 con 0x7f33e80affe0 2023-12-20T06:02:52.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e5d9b640 1 -- 172.21.15.32:0/4024223127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e8159770 msgr2=0x7f33e80ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e5d9b640 1 --2- 172.21.15.32:0/4024223127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e8159770 0x7f33e80ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e5d9b640 1 -- 172.21.15.32:0/4024223127 >> v1:172.21.15.39:6789/0 conn(0x7f33e80affe0 legacy=0x7f33e80717b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e5d9b640 1 -- 172.21.15.32:0/4024223127 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33e815a760 con 0x7f33e80cc130 2023-12-20T06:02:52.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.683+0000 7f33e659c640 1 --2- 172.21.15.32:0/4024223127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e8159770 0x7f33e80ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e5d9b640 1 --2- 172.21.15.32:0/4024223127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e80cc130 0x7f33e8159b50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0097f0 tx=0x7f33dc01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=cadfcd0e2aa2d0a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e559a640 1 -- 172.21.15.32:0/4024223127 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0ab650 con 0x7f33e80cc130 2023-12-20T06:02:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e559a640 1 -- 172.21.15.32:0/4024223127 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33dc0ab7b0 con 0x7f33e80cc130 2023-12-20T06:02:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e559a640 1 -- 172.21.15.32:0/4024223127 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33dc0abb40 con 0x7f33e80cc130 2023-12-20T06:02:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e759e640 1 -- 172.21.15.32:0/4024223127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e80cc130 msgr2=0x7f33e8159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e759e640 1 --2- 172.21.15.32:0/4024223127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e80cc130 0x7f33e8159b50 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0097f0 tx=0x7f33dc01c450 comp rx=0 tx=0).stop 2023-12-20T06:02:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e759e640 1 -- 172.21.15.32:0/4024223127 shutdown_connections 2023-12-20T06:02:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e759e640 1 --2- 172.21.15.32:0/4024223127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e80cc130 0x7f33e8159b50 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e759e640 1 --2- 172.21.15.32:0/4024223127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e8159770 0x7f33e80ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e759e640 1 -- 172.21.15.32:0/4024223127 >> 172.21.15.32:0/4024223127 conn(0x7f33e806a4e0 msgr2=0x7f33e806bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e759e640 1 -- 172.21.15.32:0/4024223127 shutdown_connections 2023-12-20T06:02:52.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.687+0000 7f33e759e640 1 -- 172.21.15.32:0/4024223127 wait complete. 2023-12-20T06:02:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e759e640 1 Processor -- start 2023-12-20T06:02:52.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e759e640 1 -- start start 2023-12-20T06:02:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e80affe0 0x7f33e814f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e759e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e80cc130 0x7f33e81ef8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e759e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e8159770 0x7f33e81efe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:52.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e759e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e81f1060 con 0x7f33e80affe0 2023-12-20T06:02:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e759e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33e81f0450 con 0x7f33e80cc130 2023-12-20T06:02:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e759e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33e81f0730 con 0x7f33e8159770 2023-12-20T06:02:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e80affe0 0x7f33e814f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e659c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e80affe0 0x7f33e814f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60056/0 (socket says 172.21.15.32:60056) 2023-12-20T06:02:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e659c640 1 -- 172.21.15.32:0/518641790 learned_addr learned my addr 172.21.15.32:0/518641790 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e5d9b640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e80cc130 0x7f33e81ef8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e6d9d640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e8159770 0x7f33e81efe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e659c640 1 -- 172.21.15.32:0/518641790 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e8159770 msgr2=0x7f33e81efe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e659c640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e8159770 0x7f33e81efe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e659c640 1 -- 172.21.15.32:0/518641790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e80cc130 msgr2=0x7f33e81ef8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e659c640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e80cc130 0x7f33e81ef8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e659c640 1 -- 172.21.15.32:0/518641790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33d8009420 con 0x7f33e80affe0 2023-12-20T06:02:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e659c640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e80affe0 0x7f33e814f700 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f33d800e980 tx=0x7f33d800ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33d6ffd640 1 -- 172.21.15.32:0/518641790 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d800ca00 con 0x7f33e80affe0 2023-12-20T06:02:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33d6ffd640 1 -- 172.21.15.32:0/518641790 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33d800cb60 con 0x7f33e80affe0 2023-12-20T06:02:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33d6ffd640 1 -- 172.21.15.32:0/518641790 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33d8013470 con 0x7f33e80affe0 2023-12-20T06:02:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e759e640 1 -- 172.21.15.32:0/518641790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33dc009420 con 0x7f33e80affe0 2023-12-20T06:02:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e759e640 1 -- 172.21.15.32:0/518641790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33e81f0cf0 con 0x7f33e80affe0 2023-12-20T06:02:52.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.691+0000 7f33e759e640 1 -- 172.21.15.32:0/518641790 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33e8074100 con 0x7f33e80affe0 2023-12-20T06:02:52.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.695+0000 7f33d6ffd640 1 -- 172.21.15.32:0/518641790 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 129) v1 ==== 85450+0+0 (secure 0 0 0) 0x7f33d80135d0 con 0x7f33e80affe0 2023-12-20T06:02:52.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.695+0000 7f33d6ffd640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33c8065cf0 0x7f33c80681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:52.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.695+0000 7f33d6ffd640 1 -- 172.21.15.32:0/518641790 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f33d80840d0 con 0x7f33e80affe0 2023-12-20T06:02:52.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.695+0000 7f33e5d9b640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33c8065cf0 0x7f33c80681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:52.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.695+0000 7f33e5d9b640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33c8065cf0 0x7f33c80681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0af000 tx=0x7f33dc0a7000 comp rx=0 tx=0).ready entity=mgr.10824 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:52.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.699+0000 7f33d6ffd640 1 -- 172.21.15.32:0/518641790 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f33d80512e0 con 0x7f33e80affe0 2023-12-20T06:02:52.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.875+0000 7f33e759e640 1 -- 172.21.15.32:0/518641790 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33e8071220 con 0x7f33c8065cf0 2023-12-20T06:02:52.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33d6ffd640 1 -- 172.21.15.32:0/518641790 <== mgr.10824 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+88 (secure 0 0 0) 0x7f33dc0ab920 con 0x7f33c8065cf0 2023-12-20T06:02:52.882 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:52.882 INFO:teuthology.orchestra.run.smithi032.stdout:{"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-20T06:02:52.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 -- 172.21.15.32:0/518641790 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33c8065cf0 msgr2=0x7f33c80681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33c8065cf0 0x7f33c80681b0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f33dc0af000 tx=0x7f33dc0a7000 comp rx=0 tx=0).stop 2023-12-20T06:02:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 -- 172.21.15.32:0/518641790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e80affe0 msgr2=0x7f33e814f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e80affe0 0x7f33e814f700 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7f33d800e980 tx=0x7f33d800ee40 comp rx=0 tx=0).stop 2023-12-20T06:02:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 -- 172.21.15.32:0/518641790 shutdown_connections 2023-12-20T06:02:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f33c8065cf0 0x7f33c80681b0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33e8159770 0x7f33e81efe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33e80cc130 0x7f33e81ef8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 --2- 172.21.15.32:0/518641790 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33e80affe0 0x7f33e814f700 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 -- 172.21.15.32:0/518641790 >> 172.21.15.32:0/518641790 conn(0x7f33e806a4e0 msgr2=0x7f33e8150810 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:52.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 -- 172.21.15.32:0/518641790 shutdown_connections 2023-12-20T06:02:52.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:52.883+0000 7f33e759e640 1 -- 172.21.15.32:0/518641790 wait complete. 2023-12-20T06:02:52.903 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "mandatory_fs_argument_test_dir", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-20T06:02:52.903 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule status mandatory_fs_argument_test_dir --fs backup_fs --format json 2023-12-20T06:02:53.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.051+0000 7fa6e2f7b640 1 Processor -- start 2023-12-20T06:02:53.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.051+0000 7fa6e2f7b640 1 -- start start 2023-12-20T06:02:53.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.051+0000 7fa6e2f7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6dc159250 0x7fa6dc0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.051+0000 7fa6e2f7b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6dc159630 con 0x7fa6dc070940 2023-12-20T06:02:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6e2f7b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa6dc0c9480 con 0x7fa6dc0ce290 2023-12-20T06:02:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6e2f7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc06cf90 con 0x7fa6dc159250 2023-12-20T06:02:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6e1518640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa6dc070940 0x7fa6dc0cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42502/0 (socket says 172.21.15.32:42502) 2023-12-20T06:02:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6e1518640 1 -- 172.21.15.32:0/3629056874 learned_addr learned my addr 172.21.15.32:0/3629056874 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6dbfff640 1 --1- 172.21.15.32:0/3629056874 >> v1:172.21.15.39:6789/0 conn(0x7fa6dc0ce290 0x7fa6dc070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:43686/0 (socket says 172.21.15.32:43686) 2023-12-20T06:02:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6e0d17640 1 --2- 172.21.15.32:0/3629056874 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6dc159250 0x7fa6dc0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6db7fe640 1 -- 172.21.15.32:0/3629056874 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2398526446 0 0) 0x7fa6dc0c9480 con 0x7fa6dc0ce290 2023-12-20T06:02:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6db7fe640 1 -- 172.21.15.32:0/3629056874 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa6cc003530 con 0x7fa6dc0ce290 2023-12-20T06:02:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6db7fe640 1 -- 172.21.15.32:0/3629056874 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3120810743 0 0) 0x7fa6dc159630 con 0x7fa6dc070940 2023-12-20T06:02:53.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6e0d17640 1 -- 172.21.15.32:0/3629056874 >> v1:172.21.15.39:6789/0 conn(0x7fa6dc0ce290 legacy=0x7fa6dc070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6e0d17640 1 -- 172.21.15.32:0/3629056874 >> v1:172.21.15.32:6789/0 conn(0x7fa6dc070940 legacy=0x7fa6dc0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6e0d17640 1 -- 172.21.15.32:0/3629056874 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6dc06d270 con 0x7fa6dc159250 2023-12-20T06:02:53.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6e0d17640 1 --2- 172.21.15.32:0/3629056874 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6dc159250 0x7fa6dc0caa30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000a990 tx=0x7fa6d001d860 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fa97a45888f29e43 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:53.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6db7fe640 1 -- 172.21.15.32:0/3629056874 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d00a6890 con 0x7fa6dc159250 2023-12-20T06:02:53.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6db7fe640 1 -- 172.21.15.32:0/3629056874 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d00a69f0 con 0x7fa6dc159250 2023-12-20T06:02:53.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.055+0000 7fa6db7fe640 1 -- 172.21.15.32:0/3629056874 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d00a6d60 con 0x7fa6dc159250 2023-12-20T06:02:53.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3629056874 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6dc159250 msgr2=0x7fa6dc0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 --2- 172.21.15.32:0/3629056874 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6dc159250 0x7fa6dc0caa30 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000a990 tx=0x7fa6d001d860 comp rx=0 tx=0).stop 2023-12-20T06:02:53.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3629056874 shutdown_connections 2023-12-20T06:02:53.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 --2- 172.21.15.32:0/3629056874 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6dc159250 0x7fa6dc0caa30 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3629056874 >> 172.21.15.32:0/3629056874 conn(0x7fa6dc06a510 msgr2=0x7fa6dc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:53.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3629056874 shutdown_connections 2023-12-20T06:02:53.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3629056874 wait complete. 2023-12-20T06:02:53.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 Processor -- start 2023-12-20T06:02:53.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 -- start start 2023-12-20T06:02:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6dc070940 0x7fa6dc14fbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6dc0ce290 0x7fa6dc15be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6dc159250 0x7fa6dc14f3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc0b29c0 con 0x7fa6dc070940 2023-12-20T06:02:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc150450 con 0x7fa6dc0ce290 2023-12-20T06:02:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e2f7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa6dc15c380 con 0x7fa6dc159250 2023-12-20T06:02:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e0d17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6dc070940 0x7fa6dc14fbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e0d17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6dc070940 0x7fa6dc14fbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60070/0 (socket says 172.21.15.32:60070) 2023-12-20T06:02:53.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e0d17640 1 -- 172.21.15.32:0/3120690924 learned_addr learned my addr 172.21.15.32:0/3120690924 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6dbfff640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6dc0ce290 0x7fa6dc15be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.059+0000 7fa6e1518640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6dc159250 0x7fa6dc14f3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6e0d17640 1 -- 172.21.15.32:0/3120690924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6dc159250 msgr2=0x7fa6dc14f3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6e0d17640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6dc159250 0x7fa6dc14f3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6e0d17640 1 -- 172.21.15.32:0/3120690924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6dc0ce290 msgr2=0x7fa6dc15be40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6e0d17640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6dc0ce290 0x7fa6dc15be40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6e0d17640 1 -- 172.21.15.32:0/3120690924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa6d000a5c0 con 0x7fa6dc070940 2023-12-20T06:02:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6e0d17640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6dc070940 0x7fa6dc14fbc0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000a990 tx=0x7fa6d001a600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6d97fa640 1 -- 172.21.15.32:0/3120690924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0004450 con 0x7fa6dc070940 2023-12-20T06:02:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6d97fa640 1 -- 172.21.15.32:0/3120690924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa6d0003620 con 0x7fa6dc070940 2023-12-20T06:02:53.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6d97fa640 1 -- 172.21.15.32:0/3120690924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa6d0003990 con 0x7fa6dc070940 2023-12-20T06:02:53.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3120690924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa6dc0b2090 con 0x7fa6dc070940 2023-12-20T06:02:53.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3120690924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa6dc0b2650 con 0x7fa6dc070940 2023-12-20T06:02:53.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3120690924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa6dc04e000 con 0x7fa6dc070940 2023-12-20T06:02:53.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6d97fa640 1 -- 172.21.15.32:0/3120690924 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 129) v1 ==== 85450+0+0 (secure 0 0 0) 0x7fa6d00b5040 con 0x7fa6dc070940 2023-12-20T06:02:53.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6d97fa640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6c0065cf0 0x7fa6c00681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6d97fa640 1 -- 172.21.15.32:0/3120690924 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7fa6d011dfa0 con 0x7fa6dc070940 2023-12-20T06:02:53.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.063+0000 7fa6dbfff640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6c0065cf0 0x7fa6c00681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.067+0000 7fa6dbfff640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6c0065cf0 0x7fa6c00681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0b4af0 tx=0x7fa6c8007d40 comp rx=0 tx=0).ready entity=mgr.10824 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:53.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.067+0000 7fa6d97fa640 1 -- 172.21.15.32:0/3120690924 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa6d00eb120 con 0x7fa6dc070940 2023-12-20T06:02:53.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.243+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3120690924 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa6dc0caf70 con 0x7fa6c0065cf0 2023-12-20T06:02:53.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6d97fa640 1 -- 172.21.15.32:0/3120690924 <== mgr.10824 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+338 (secure 0 0 0) 0x7fa6c8007300 con 0x7fa6c0065cf0 2023-12-20T06:02:53.250 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:53.250 INFO:teuthology.orchestra.run.smithi032.stdout:[{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T06:02:51", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-20T06:02:53.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3120690924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6c0065cf0 msgr2=0x7fa6c00681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6c0065cf0 0x7fa6c00681b0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fa6dc0b4af0 tx=0x7fa6c8007d40 comp rx=0 tx=0).stop 2023-12-20T06:02:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3120690924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6dc070940 msgr2=0x7fa6dc14fbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6dc070940 0x7fa6dc14fbc0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7fa6d000a990 tx=0x7fa6d001a600 comp rx=0 tx=0).stop 2023-12-20T06:02:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3120690924 shutdown_connections 2023-12-20T06:02:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa6c0065cf0 0x7fa6c00681b0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa6dc159250 0x7fa6dc14f3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa6dc0ce290 0x7fa6dc15be40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 --2- 172.21.15.32:0/3120690924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa6dc070940 0x7fa6dc14fbc0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3120690924 >> 172.21.15.32:0/3120690924 conn(0x7fa6dc06a510 msgr2=0x7fa6dc0c4c20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3120690924 shutdown_connections 2023-12-20T06:02:53.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.251+0000 7fa6e2f7b640 1 -- 172.21.15.32:0/3120690924 wait complete. 2023-12-20T06:02:53.272 DEBUG:tasks.cephfs.test_snap_schedules:res= [{"fs": "backup_fs", "subvol": null, "path": "mandatory_fs_argument_test_dir", "rel_path": "mandatory_fs_argument_test_dir", "schedule": "1M", "retention": {"M": 1}, "start": "2023-12-20T00:00:00", "created": "2023-12-20T06:02:51", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true}] 2023-12-20T06:02:53.272 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-20T06:02:53.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f650996d640 1 Processor -- start 2023-12-20T06:02:53.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f650996d640 1 -- start start 2023-12-20T06:02:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f650996d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f65040adf00 0x7f65040cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f650996d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6504159e60 con 0x7f65040c4320 2023-12-20T06:02:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f650996d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65040ce380 con 0x7f650406e150 2023-12-20T06:02:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f650996d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65040c3630 con 0x7f65040adf00 2023-12-20T06:02:53.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f64fbfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f65040c4320 0x7f650406da40 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42510/0 (socket says 172.21.15.32:42510) 2023-12-20T06:02:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f64fbfff640 1 -- 172.21.15.32:0/1388255549 learned_addr learned my addr 172.21.15.32:0/1388255549 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f6502ffd640 1 --2- 172.21.15.32:0/1388255549 >> v2:172.21.15.39:3300/0 conn(0x7f65040adf00 0x7f65040cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1675078188 0 0) 0x7f6504159e60 con 0x7f65040c4320 2023-12-20T06:02:53.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f64f0003530 con 0x7f65040c4320 2023-12-20T06:02:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2937428034 0 0) 0x7f64f0003530 con 0x7f65040c4320 2023-12-20T06:02:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 >> v2:172.21.15.39:3300/0 conn(0x7f65040adf00 msgr2=0x7f65040cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 --2- 172.21.15.32:0/1388255549 >> v2:172.21.15.39:3300/0 conn(0x7f65040adf00 0x7f65040cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 >> v1:172.21.15.39:6789/0 conn(0x7f650406e150 legacy=0x7f650406c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040c3840 con 0x7f65040c4320 2023-12-20T06:02:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f64ec001680 con 0x7f65040c4320 2023-12-20T06:02:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 >> v1:172.21.15.32:6789/0 conn(0x7f65040c4320 legacy=0x7f650406da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f0003980 0x7f64f0003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64f00047d0 0x7f64f0004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64f0005190 0x7f64f000dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040c3630 con 0x7f64f00047d0 2023-12-20T06:02:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f64f000e240 con 0x7f64f0003980 2023-12-20T06:02:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f64f000e400 con 0x7f64f0005190 2023-12-20T06:02:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f6502ffd640 1 --2- 172.21.15.32:0/1388255549 >> v2:172.21.15.39:3300/0 conn(0x7f65040adf00 0x7f65040cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f64fbfff640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64f00047d0 0x7f64f0004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f6502ffd640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f0003980 0x7f64f0003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.423+0000 7f65037fe640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64f0005190 0x7f64f000dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f64fbfff640 1 -- 172.21.15.32:0/1388255549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64f0005190 msgr2=0x7f64f000dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f64fbfff640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64f0005190 0x7f64f000dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f64fbfff640 1 -- 172.21.15.32:0/1388255549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f0003980 msgr2=0x7f64f0003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f64fbfff640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f0003980 0x7f64f0003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f64fbfff640 1 -- 172.21.15.32:0/1388255549 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f64f0003530 con 0x7f64f00047d0 2023-12-20T06:02:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f6502ffd640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f64f0003980 0x7f64f0003da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:53.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f65037fe640 1 -- 172.21.15.32:0/1388255549 reap_dead start 2023-12-20T06:02:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f64fbfff640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64f00047d0 0x7f64f0004c50 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00c240 tx=0x7f64ec01f3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff42c0db71ffe9eb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec005ae0 con 0x7f64f00047d0 2023-12-20T06:02:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64ec0ab070 con 0x7f64f00047d0 2023-12-20T06:02:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f65027fc640 1 -- 172.21.15.32:0/1388255549 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64ec007070 con 0x7f64f00047d0 2023-12-20T06:02:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f650996d640 1 -- 172.21.15.32:0/1388255549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64f00047d0 msgr2=0x7f64f0004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f650996d640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64f00047d0 0x7f64f0004c50 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7f64ec00c240 tx=0x7f64ec01f3b0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f650996d640 1 -- 172.21.15.32:0/1388255549 shutdown_connections 2023-12-20T06:02:53.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f650996d640 1 --2- 172.21.15.32:0/1388255549 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f64f00047d0 0x7f64f0004c50 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f650996d640 1 -- 172.21.15.32:0/1388255549 >> 172.21.15.32:0/1388255549 conn(0x7f650406a4b0 msgr2=0x7f65040b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:53.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.427+0000 7f650996d640 1 -- 172.21.15.32:0/1388255549 shutdown_connections 2023-12-20T06:02:53.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f650996d640 1 -- 172.21.15.32:0/1388255549 wait complete. 2023-12-20T06:02:53.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f650996d640 1 Processor -- start 2023-12-20T06:02:53.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f650996d640 1 -- start start 2023-12-20T06:02:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f650996d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64f00047d0 0x7f6504141380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f650996d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65040afa10 0x7f65040afe90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f650996d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65040b2c20 0x7f65040b30a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f650996d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6504141ce0 con 0x7f65040b2c20 2023-12-20T06:02:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f650996d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65040b03d0 con 0x7f65040afa10 2023-12-20T06:02:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f650996d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65040abc20 con 0x7f64f00047d0 2023-12-20T06:02:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f65037fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65040b2c20 0x7f65040b30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f65037fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65040b2c20 0x7f65040b30a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60090/0 (socket says 172.21.15.32:60090) 2023-12-20T06:02:53.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f65037fe640 1 -- 172.21.15.32:0/170787644 learned_addr learned my addr 172.21.15.32:0/170787644 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f64fbfff640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65040afa10 0x7f65040afe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f6502ffd640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64f00047d0 0x7f6504141380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f65037fe640 1 -- 172.21.15.32:0/170787644 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64f00047d0 msgr2=0x7f6504141380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f65037fe640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64f00047d0 0x7f6504141380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f65037fe640 1 -- 172.21.15.32:0/170787644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65040afa10 msgr2=0x7f65040afe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f65037fe640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65040afa10 0x7f65040afe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f65037fe640 1 -- 172.21.15.32:0/170787644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65040b3900 con 0x7f65040b2c20 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f65037fe640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65040b2c20 0x7f65040b30a0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f65040c4c90 tx=0x7f64fc00eb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f6500ff9640 1 -- 172.21.15.32:0/170787644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc00ee90 con 0x7f65040b2c20 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f6500ff9640 1 -- 172.21.15.32:0/170787644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f64fc003980 con 0x7f65040b2c20 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f6500ff9640 1 -- 172.21.15.32:0/170787644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f64fc003d70 con 0x7f65040b2c20 2023-12-20T06:02:53.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f650996d640 1 -- 172.21.15.32:0/170787644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65040b3be0 con 0x7f65040b2c20 2023-12-20T06:02:53.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.431+0000 7f650996d640 1 -- 172.21.15.32:0/170787644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65040b41a0 con 0x7f65040b2c20 2023-12-20T06:02:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.435+0000 7f650996d640 1 -- 172.21.15.32:0/170787644 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65040ce110 con 0x7f65040b2c20 2023-12-20T06:02:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.435+0000 7f6500ff9640 1 -- 172.21.15.32:0/170787644 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 129) v1 ==== 85450+0+0 (secure 0 0 0) 0x7f64fc0050d0 con 0x7f65040b2c20 2023-12-20T06:02:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.435+0000 7f6500ff9640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64e4065d20 0x7f64e40681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.435+0000 7f6500ff9640 1 -- 172.21.15.32:0/170787644 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f64fc0836a0 con 0x7f65040b2c20 2023-12-20T06:02:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.435+0000 7f6502ffd640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64e4065d20 0x7f64e40681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.435+0000 7f6502ffd640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64e4065d20 0x7f64e40681e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6504069560 tx=0x7f64f4008040 comp rx=0 tx=0).ready entity=mgr.10824 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:53.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.439+0000 7f6500ff9640 1 -- 172.21.15.32:0/170787644 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f64fc050810 con 0x7f65040b2c20 2023-12-20T06:02:53.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.619+0000 7f650996d640 1 -- 172.21.15.32:0/170787644 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f65040c3e00 con 0x7f64e4065d20 2023-12-20T06:02:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.671+0000 7f6500ff9640 1 -- 172.21.15.32:0/170787644 <== mgr.10824 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+59 (secure 0 0 0) 0x7f64f4009950 con 0x7f64e4065d20 2023-12-20T06:02:53.672 INFO:teuthology.orchestra.run.smithi032.stdout:Retention removed from path mandatory_fs_argument_test_dir 2023-12-20T06:02:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 -- 172.21.15.32:0/170787644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64e4065d20 msgr2=0x7f64e40681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64e4065d20 0x7f64e40681e0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6504069560 tx=0x7f64f4008040 comp rx=0 tx=0).stop 2023-12-20T06:02:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 -- 172.21.15.32:0/170787644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65040b2c20 msgr2=0x7f65040b30a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65040b2c20 0x7f65040b30a0 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7f65040c4c90 tx=0x7f64fc00eb50 comp rx=0 tx=0).stop 2023-12-20T06:02:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 -- 172.21.15.32:0/170787644 shutdown_connections 2023-12-20T06:02:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f64e4065d20 0x7f64e40681e0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f65040b2c20 0x7f65040b30a0 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65040afa10 0x7f65040afe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 --2- 172.21.15.32:0/170787644 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f64f00047d0 0x7f6504141380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 -- 172.21.15.32:0/170787644 >> 172.21.15.32:0/170787644 conn(0x7f650406a4b0 msgr2=0x7f650406c3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 -- 172.21.15.32:0/170787644 shutdown_connections 2023-12-20T06:02:53.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.675+0000 7f650996d640 1 -- 172.21.15.32:0/170787644 wait complete. 2023-12-20T06:02:53.694 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention removed from path mandatory_fs_argument_test_dir 2023-12-20T06:02:53.694 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule deactivate mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-20T06:02:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.843+0000 7fb301d8f640 1 Processor -- start 2023-12-20T06:02:53.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.843+0000 7fb301d8f640 1 -- start start 2023-12-20T06:02:53.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb301d8f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb2fc159af0 0x7fb2fc0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb301d8f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb2fc0b0f00 0x7fb2fc06d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb301d8f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb2fc06dd50 con 0x7fb2fc06d2d0 2023-12-20T06:02:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb301d8f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2fc0cc940 con 0x7fb2fc159af0 2023-12-20T06:02:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb301d8f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb2fc0ccc20 con 0x7fb2fc0b0f00 2023-12-20T06:02:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb2fb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb2fc159af0 0x7fb2fc0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb2fb7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb2fc159af0 0x7fb2fc0b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60092/0 (socket says 172.21.15.32:60092) 2023-12-20T06:02:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb2fb7fe640 1 -- 172.21.15.32:0/2446237554 learned_addr learned my addr 172.21.15.32:0/2446237554 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb2fbfff640 1 --2- 172.21.15.32:0/2446237554 >> v2:172.21.15.39:3300/0 conn(0x7fb2fc0b0f00 0x7fb2fc06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb2fb7fe640 1 -- 172.21.15.32:0/2446237554 >> v2:172.21.15.39:3300/0 conn(0x7fb2fc0b0f00 msgr2=0x7fb2fc06d740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb2fb7fe640 1 --2- 172.21.15.32:0/2446237554 >> v2:172.21.15.39:3300/0 conn(0x7fb2fc0b0f00 0x7fb2fc06d740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb2fb7fe640 1 -- 172.21.15.32:0/2446237554 >> v1:172.21.15.39:6789/0 conn(0x7fb2fc06d2d0 legacy=0x7fb2fc0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb2fb7fe640 1 -- 172.21.15.32:0/2446237554 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc0ccf00 con 0x7fb2fc159af0 2023-12-20T06:02:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb2fbfff640 1 --2- 172.21.15.32:0/2446237554 >> v2:172.21.15.39:3300/0 conn(0x7fb2fc0b0f00 0x7fb2fc06d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.847+0000 7fb2fb7fe640 1 --2- 172.21.15.32:0/2446237554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc159af0 0x7fb2fc0b1930 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc15d5c0 tx=0x7fb2ec01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c89e95c3247d71b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb2fa7fc640 1 -- 172.21.15.32:0/2446237554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0aa610 con 0x7fb2fc159af0 2023-12-20T06:02:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb2fa7fc640 1 -- 172.21.15.32:0/2446237554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2ec0a6040 con 0x7fb2fc159af0 2023-12-20T06:02:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb2fa7fc640 1 -- 172.21.15.32:0/2446237554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2ec0aa990 con 0x7fb2fc159af0 2023-12-20T06:02:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 -- 172.21.15.32:0/2446237554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc159af0 msgr2=0x7fb2fc0b1930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 --2- 172.21.15.32:0/2446237554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc159af0 0x7fb2fc0b1930 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7fb2fc15d5c0 tx=0x7fb2ec01c490 comp rx=0 tx=0).stop 2023-12-20T06:02:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 -- 172.21.15.32:0/2446237554 shutdown_connections 2023-12-20T06:02:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 --2- 172.21.15.32:0/2446237554 >> v2:172.21.15.39:3300/0 conn(0x7fb2fc0b0f00 0x7fb2fc06d740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 --2- 172.21.15.32:0/2446237554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc159af0 0x7fb2fc0b1930 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 -- 172.21.15.32:0/2446237554 >> 172.21.15.32:0/2446237554 conn(0x7fb2fc06a510 msgr2=0x7fb2fc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 -- 172.21.15.32:0/2446237554 shutdown_connections 2023-12-20T06:02:53.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 -- 172.21.15.32:0/2446237554 wait complete. 2023-12-20T06:02:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 Processor -- start 2023-12-20T06:02:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 -- start start 2023-12-20T06:02:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc06d2d0 0x7fb2fc1361e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc0b0f00 0x7fb2fc0c7020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc159af0 0x7fb2fc14a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc136720 con 0x7fb2fc0b0f00 2023-12-20T06:02:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc148da0 con 0x7fb2fc159af0 2023-12-20T06:02:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb301d8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc149360 con 0x7fb2fc06d2d0 2023-12-20T06:02:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.851+0000 7fb2faffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc0b0f00 0x7fb2fc0c7020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2faffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc0b0f00 0x7fb2fc0c7020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60094/0 (socket says 172.21.15.32:60094) 2023-12-20T06:02:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2faffd640 1 -- 172.21.15.32:0/1764544708 learned_addr learned my addr 172.21.15.32:0/1764544708 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2fbfff640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc159af0 0x7fb2fc14a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2fb7fe640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc06d2d0 0x7fb2fc1361e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2faffd640 1 -- 172.21.15.32:0/1764544708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc06d2d0 msgr2=0x7fb2fc1361e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2faffd640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc06d2d0 0x7fb2fc1361e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2faffd640 1 -- 172.21.15.32:0/1764544708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc159af0 msgr2=0x7fb2fc14a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2faffd640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc159af0 0x7fb2fc14a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2faffd640 1 -- 172.21.15.32:0/1764544708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2ec0094d0 con 0x7fb2fc0b0f00 2023-12-20T06:02:53.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2fb7fe640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc06d2d0 0x7fb2fc1361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:53.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2fbfff640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc159af0 0x7fb2fc14a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:53.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2faffd640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc0b0f00 0x7fb2fc0c7020 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000c450 tx=0x7fb2f000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:53.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2e3fff640 1 -- 172.21.15.32:0/1764544708 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0012830 con 0x7fb2fc0b0f00 2023-12-20T06:02:53.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2e3fff640 1 -- 172.21.15.32:0/1764544708 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb2f00129b0 con 0x7fb2fc0b0f00 2023-12-20T06:02:53.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb301d8f640 1 -- 172.21.15.32:0/1764544708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb2fc148570 con 0x7fb2fc0b0f00 2023-12-20T06:02:53.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2e3fff640 1 -- 172.21.15.32:0/1764544708 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb2f0012d60 con 0x7fb2fc0b0f00 2023-12-20T06:02:53.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb301d8f640 1 -- 172.21.15.32:0/1764544708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb2fc148ff0 con 0x7fb2fc0b0f00 2023-12-20T06:02:53.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb301d8f640 1 -- 172.21.15.32:0/1764544708 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb2fc149510 con 0x7fb2fc0b0f00 2023-12-20T06:02:53.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2e3fff640 1 -- 172.21.15.32:0/1764544708 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 129) v1 ==== 85450+0+0 (secure 0 0 0) 0x7fb2f001c460 con 0x7fb2fc0b0f00 2023-12-20T06:02:53.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2e3fff640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2e8065cf0 0x7fb2e80681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:53.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.855+0000 7fb2e3fff640 1 -- 172.21.15.32:0/1764544708 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7fb2f0083890 con 0x7fb2fc0b0f00 2023-12-20T06:02:53.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.859+0000 7fb2fb7fe640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2e8065cf0 0x7fb2e80681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:53.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.859+0000 7fb2fb7fe640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2e8065cf0 0x7fb2e80681b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0ae000 tx=0x7fb2ec0a7000 comp rx=0 tx=0).ready entity=mgr.10824 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:53.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:53.863+0000 7fb2e3fff640 1 -- 172.21.15.32:0/1764544708 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb2f0050a90 con 0x7fb2fc0b0f00 2023-12-20T06:02:54.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.039+0000 7fb301d8f640 1 -- 172.21.15.32:0/1764544708 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule deactivate", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fb2fc0cdf10 con 0x7fb2e8065cf0 2023-12-20T06:02:54.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.091+0000 7fb2e3fff640 1 -- 172.21.15.32:0/1764544708 <== mgr.10824 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+61 (secure 0 0 0) 0x7fb2ec0aa8f0 con 0x7fb2e8065cf0 2023-12-20T06:02:54.092 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule deactivated for path mandatory_fs_argument_test_dir 2023-12-20T06:02:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 -- 172.21.15.32:0/1764544708 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2e8065cf0 msgr2=0x7fb2e80681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2e8065cf0 0x7fb2e80681b0 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fb2ec0ae000 tx=0x7fb2ec0a7000 comp rx=0 tx=0).stop 2023-12-20T06:02:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 -- 172.21.15.32:0/1764544708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc0b0f00 msgr2=0x7fb2fc0c7020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc0b0f00 0x7fb2fc0c7020 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7fb2f000c450 tx=0x7fb2f000c910 comp rx=0 tx=0).stop 2023-12-20T06:02:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 -- 172.21.15.32:0/1764544708 shutdown_connections 2023-12-20T06:02:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2e8065cf0 0x7fb2e80681b0 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc159af0 0x7fb2fc14a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc0b0f00 0x7fb2fc0c7020 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 --2- 172.21.15.32:0/1764544708 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc06d2d0 0x7fb2fc1361e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 -- 172.21.15.32:0/1764544708 >> 172.21.15.32:0/1764544708 conn(0x7fb2fc06a510 msgr2=0x7fb2fc14ef70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:54.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 -- 172.21.15.32:0/1764544708 shutdown_connections 2023-12-20T06:02:54.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.095+0000 7fb301d8f640 1 -- 172.21.15.32:0/1764544708 wait complete. 2023-12-20T06:02:54.114 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule deactivated for path mandatory_fs_argument_test_dir 2023-12-20T06:02:54.114 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove mandatory_fs_argument_test_dir --fs backup_fs 1M 2023-12-20T06:02:54.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.263+0000 7fbf22666640 1 Processor -- start 2023-12-20T06:02:54.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.263+0000 7fbf22666640 1 -- start start 2023-12-20T06:02:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.263+0000 7fbf22666640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbf1c0b1930 0x7fbf1c0b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.263+0000 7fbf22666640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c06dd50 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.263+0000 7fbf22666640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf1c0b1140 con 0x7fbf1c159af0 2023-12-20T06:02:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf22666640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf1c0b1da0 con 0x7fbf1c0b1930 2023-12-20T06:02:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf20c03640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbf1c0b0d30 0x7fbf1c0cd700 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42520/0 (socket says 172.21.15.32:42520) 2023-12-20T06:02:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf20c03640 1 -- 172.21.15.32:0/1113521504 learned_addr learned my addr 172.21.15.32:0/1113521504 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:54.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf1b7fe640 1 --2- 172.21.15.32:0/1113521504 >> v2:172.21.15.39:3300/0 conn(0x7fbf1c0b1930 0x7fbf1c0b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf1affd640 1 -- 172.21.15.32:0/1113521504 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911200758 0 0) 0x7fbf1c06dd50 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf1affd640 1 -- 172.21.15.32:0/1113521504 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf04003530 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf1affd640 1 -- 172.21.15.32:0/1113521504 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2298796093 0 0) 0x7fbf1c0b1140 con 0x7fbf1c159af0 2023-12-20T06:02:54.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf1affd640 1 -- 172.21.15.32:0/1113521504 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbf1c06dd50 con 0x7fbf1c159af0 2023-12-20T06:02:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf1b7fe640 1 -- 172.21.15.32:0/1113521504 >> v1:172.21.15.39:6789/0 conn(0x7fbf1c159af0 legacy=0x7fbf1c06d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf1b7fe640 1 -- 172.21.15.32:0/1113521504 >> v1:172.21.15.32:6789/0 conn(0x7fbf1c0b0d30 legacy=0x7fbf1c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf1b7fe640 1 -- 172.21.15.32:0/1113521504 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf1c0cc940 con 0x7fbf1c0b1930 2023-12-20T06:02:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.267+0000 7fbf1b7fe640 1 --2- 172.21.15.32:0/1113521504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf1c0b1930 0x7fbf1c0b07f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbf100097c0 tx=0x7fbf1001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=db18933b687572ed server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf1affd640 1 -- 172.21.15.32:0/1113521504 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf100ab650 con 0x7fbf1c0b1930 2023-12-20T06:02:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf1affd640 1 -- 172.21.15.32:0/1113521504 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf100ab7b0 con 0x7fbf1c0b1930 2023-12-20T06:02:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf1affd640 1 -- 172.21.15.32:0/1113521504 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf100abb40 con 0x7fbf1c0b1930 2023-12-20T06:02:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 -- 172.21.15.32:0/1113521504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf1c0b1930 msgr2=0x7fbf1c0b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 --2- 172.21.15.32:0/1113521504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf1c0b1930 0x7fbf1c0b07f0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7fbf100097c0 tx=0x7fbf1001c450 comp rx=0 tx=0).stop 2023-12-20T06:02:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 -- 172.21.15.32:0/1113521504 shutdown_connections 2023-12-20T06:02:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 --2- 172.21.15.32:0/1113521504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf1c0b1930 0x7fbf1c0b07f0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 -- 172.21.15.32:0/1113521504 >> 172.21.15.32:0/1113521504 conn(0x7fbf1c06a510 msgr2=0x7fbf1c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 -- 172.21.15.32:0/1113521504 shutdown_connections 2023-12-20T06:02:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 -- 172.21.15.32:0/1113521504 wait complete. 2023-12-20T06:02:54.270 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 Processor -- start 2023-12-20T06:02:54.271 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 -- start start 2023-12-20T06:02:54.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1c0b0d30 0x7fbf1c1f8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf1c0b1930 0x7fbf1c1f48c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf1c159af0 0x7fbf1c070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1585f0 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1f9500 con 0x7fbf1c0b1930 2023-12-20T06:02:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf22666640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf1c1579e0 con 0x7fbf1c159af0 2023-12-20T06:02:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf1bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1c0b0d30 0x7fbf1c1f8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf1b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf1c0b1930 0x7fbf1c1f48c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf1b7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf1c0b1930 0x7fbf1c1f48c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:42866/0 (socket says 172.21.15.32:42866) 2023-12-20T06:02:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf1b7fe640 1 -- 172.21.15.32:0/3294673751 learned_addr learned my addr 172.21.15.32:0/3294673751 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:54.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.271+0000 7fbf1bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1c0b0d30 0x7fbf1c1f8fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60108/0 (socket says 172.21.15.32:60108) 2023-12-20T06:02:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf20c03640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf1c159af0 0x7fbf1c070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf1bfff640 1 -- 172.21.15.32:0/3294673751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf1c159af0 msgr2=0x7fbf1c070130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf1bfff640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf1c159af0 0x7fbf1c070130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf1bfff640 1 -- 172.21.15.32:0/3294673751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf1c0b1930 msgr2=0x7fbf1c1f48c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf1bfff640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf1c0b1930 0x7fbf1c1f48c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf1bfff640 1 -- 172.21.15.32:0/3294673751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf10009420 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf1b7fe640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf1c0b1930 0x7fbf1c1f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf1bfff640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1c0b0d30 0x7fbf1c1f8fc0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00c3f0 tx=0x7fbf0c00c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf18ff9640 1 -- 172.21.15.32:0/3294673751 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c0052d0 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf18ff9640 1 -- 172.21.15.32:0/3294673751 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf0c005430 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf18ff9640 1 -- 172.21.15.32:0/3294673751 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf0c003840 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf22666640 1 -- 172.21.15.32:0/3294673751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf1c157cc0 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf22666640 1 -- 172.21.15.32:0/3294673751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf1c158280 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf22666640 1 -- 172.21.15.32:0/3294673751 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf1c04e000 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf18ff9640 1 -- 172.21.15.32:0/3294673751 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 129) v1 ==== 85450+0+0 (secure 0 0 0) 0x7fbf0c0039a0 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf18ff9640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf08065cf0 0x7fbf080681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:54.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.275+0000 7fbf18ff9640 1 -- 172.21.15.32:0/3294673751 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7fbf0c082ee0 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.279+0000 7fbf1b7fe640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf08065cf0 0x7fbf080681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:54.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.279+0000 7fbf1b7fe640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf08065cf0 0x7fbf080681b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009710 tx=0x7fbf100a7000 comp rx=0 tx=0).ready entity=mgr.10824 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:54.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.279+0000 7fbf18ff9640 1 -- 172.21.15.32:0/3294673751 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbf0c0501d0 con 0x7fbf1c0b0d30 2023-12-20T06:02:54.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.459+0000 7fbf22666640 1 -- 172.21.15.32:0/3294673751 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "mandatory_fs_argument_test_dir", "fs": "backup_fs", "repeat": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fbf1c159ed0 con 0x7fbf08065cf0 2023-12-20T06:02:54.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf18ff9640 1 -- 172.21.15.32:0/3294673751 <== mgr.10824 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+57 (secure 0 0 0) 0x7fbf100ab450 con 0x7fbf08065cf0 2023-12-20T06:02:54.522 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path mandatory_fs_argument_test_dir 2023-12-20T06:02:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 -- 172.21.15.32:0/3294673751 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf08065cf0 msgr2=0x7fbf080681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf08065cf0 0x7fbf080681b0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbf10009710 tx=0x7fbf100a7000 comp rx=0 tx=0).stop 2023-12-20T06:02:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 -- 172.21.15.32:0/3294673751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1c0b0d30 msgr2=0x7fbf1c1f8fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1c0b0d30 0x7fbf1c1f8fc0 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7fbf0c00c3f0 tx=0x7fbf0c00c8b0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 -- 172.21.15.32:0/3294673751 shutdown_connections 2023-12-20T06:02:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbf08065cf0 0x7fbf080681b0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf1c159af0 0x7fbf1c070130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf1c0b1930 0x7fbf1c1f48c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 --2- 172.21.15.32:0/3294673751 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf1c0b0d30 0x7fbf1c1f8fc0 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 -- 172.21.15.32:0/3294673751 >> 172.21.15.32:0/3294673751 conn(0x7fbf1c06a510 msgr2=0x7fbf1c14ca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 -- 172.21.15.32:0/3294673751 shutdown_connections 2023-12-20T06:02:54.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.523+0000 7fbf22666640 1 -- 172.21.15.32:0/3294673751 wait complete. 2023-12-20T06:02:54.543 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path mandatory_fs_argument_test_dir 2023-12-20T06:02:54.543 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:54.544 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-12-20T06:02:54.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T06:02:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb9416d640 1 Processor -- start 2023-12-20T06:02:54.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb9416d640 1 -- start start 2023-12-20T06:02:54.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb9416d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb8c0d3300 0x7fbb8c0d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:54.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb9416d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb8c0d1300 0x7fbb8c0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:54.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb9416d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb8c0d2650 con 0x7fbb8c15ce60 2023-12-20T06:02:54.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb9416d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb8c0d1a40 con 0x7fbb8c0d3300 2023-12-20T06:02:54.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb9416d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0d1d20 con 0x7fbb8c0d1300 2023-12-20T06:02:54.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb91708640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb8c0d3300 0x7fbb8c0d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:54.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb91f09640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbb8c15ce60 0x7fbb8c0d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42522/0 (socket says 172.21.15.32:42522) 2023-12-20T06:02:54.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb91f09640 1 -- 172.21.15.32:0/3984961737 learned_addr learned my addr 172.21.15.32:0/3984961737 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:54.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb9270a640 1 --2- 172.21.15.32:0/3984961737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb8c0d1300 0x7fbb8c0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:54.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb91708640 1 -- 172.21.15.32:0/3984961737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb8c0d1300 msgr2=0x7fbb8c0b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:54.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb91708640 1 --2- 172.21.15.32:0/3984961737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb8c0d1300 0x7fbb8c0b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb91708640 1 -- 172.21.15.32:0/3984961737 >> v1:172.21.15.32:6789/0 conn(0x7fbb8c15ce60 legacy=0x7fbb8c0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.751+0000 7fbb91708640 1 -- 172.21.15.32:0/3984961737 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb8c0d2000 con 0x7fbb8c0d3300 2023-12-20T06:02:54.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb91708640 1 --2- 172.21.15.32:0/3984961737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0d3300 0x7fbb8c0d0dc0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0097c0 tx=0x7fbb7c01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fced50898cd3bb78 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:54.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb90f07640 1 -- 172.21.15.32:0/3984961737 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0a8970 con 0x7fbb8c0d3300 2023-12-20T06:02:54.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb90f07640 1 -- 172.21.15.32:0/3984961737 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c0a8ad0 con 0x7fbb8c0d3300 2023-12-20T06:02:54.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb90f07640 1 -- 172.21.15.32:0/3984961737 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0a8e60 con 0x7fbb8c0d3300 2023-12-20T06:02:54.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb9416d640 1 -- 172.21.15.32:0/3984961737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0d3300 msgr2=0x7fbb8c0d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb9416d640 1 --2- 172.21.15.32:0/3984961737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0d3300 0x7fbb8c0d0dc0 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0097c0 tx=0x7fbb7c01ec70 comp rx=0 tx=0).stop 2023-12-20T06:02:54.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb9416d640 1 -- 172.21.15.32:0/3984961737 shutdown_connections 2023-12-20T06:02:54.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb9416d640 1 --2- 172.21.15.32:0/3984961737 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0d3300 0x7fbb8c0d0dc0 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb9416d640 1 --2- 172.21.15.32:0/3984961737 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb8c0d1300 0x7fbb8c0b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb9416d640 1 -- 172.21.15.32:0/3984961737 >> 172.21.15.32:0/3984961737 conn(0x7fbb8c06b5e0 msgr2=0x7fbb8c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:54.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb9416d640 1 -- 172.21.15.32:0/3984961737 shutdown_connections 2023-12-20T06:02:54.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.755+0000 7fbb9416d640 1 -- 172.21.15.32:0/3984961737 wait complete. 2023-12-20T06:02:54.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb9416d640 1 Processor -- start 2023-12-20T06:02:54.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb9416d640 1 -- start start 2023-12-20T06:02:54.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb9416d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb8c0d1300 0x7fbb8c1f2350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb9416d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0d3300 0x7fbb8c15fff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb9416d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb8c15ce60 0x7fbb8c0b2e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb9416d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0cc190 con 0x7fbb8c0d3300 2023-12-20T06:02:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb9416d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0cc470 con 0x7fbb8c15ce60 2023-12-20T06:02:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb9416d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb8c0cc750 con 0x7fbb8c0d1300 2023-12-20T06:02:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb91708640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0d3300 0x7fbb8c15fff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb91708640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0d3300 0x7fbb8c15fff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60130/0 (socket says 172.21.15.32:60130) 2023-12-20T06:02:54.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb91f09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb8c0d1300 0x7fbb8c1f2350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb91708640 1 -- 172.21.15.32:0/1186594974 learned_addr learned my addr 172.21.15.32:0/1186594974 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb9270a640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb8c15ce60 0x7fbb8c0b2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb91708640 1 -- 172.21.15.32:0/1186594974 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb8c0d1300 msgr2=0x7fbb8c1f2350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb91708640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb8c0d1300 0x7fbb8c1f2350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb91708640 1 -- 172.21.15.32:0/1186594974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb8c15ce60 msgr2=0x7fbb8c0b2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb91708640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb8c15ce60 0x7fbb8c0b2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb91708640 1 -- 172.21.15.32:0/1186594974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb7c009420 con 0x7fbb8c0d3300 2023-12-20T06:02:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb91f09640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb8c0d1300 0x7fbb8c1f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb91708640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0d3300 0x7fbb8c15fff0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0b3040 tx=0x7fbb7c01ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb827fc640 1 -- 172.21.15.32:0/1186594974 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0b9630 con 0x7fbb8c0d3300 2023-12-20T06:02:54.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb827fc640 1 -- 172.21.15.32:0/1186594974 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb7c0b97b0 con 0x7fbb8c0d3300 2023-12-20T06:02:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb9416d640 1 -- 172.21.15.32:0/1186594974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb8c0cca30 con 0x7fbb8c0d3300 2023-12-20T06:02:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.759+0000 7fbb827fc640 1 -- 172.21.15.32:0/1186594974 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb7c0b9b60 con 0x7fbb8c0d3300 2023-12-20T06:02:54.763 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.763+0000 7fbb9416d640 1 -- 172.21.15.32:0/1186594974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb8c160530 con 0x7fbb8c0d3300 2023-12-20T06:02:54.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.763+0000 7fbb9416d640 1 -- 172.21.15.32:0/1186594974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb8c04e000 con 0x7fbb8c0d3300 2023-12-20T06:02:54.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.763+0000 7fbb827fc640 1 -- 172.21.15.32:0/1186594974 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 129) v1 ==== 85450+0+0 (secure 0 0 0) 0x7fbb7c0b7040 con 0x7fbb8c0d3300 2023-12-20T06:02:54.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.763+0000 7fbb827fc640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb70065d20 0x7fbb700681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:54.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.763+0000 7fbb827fc640 1 -- 172.21.15.32:0/1186594974 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7fbb7c1219c0 con 0x7fbb8c0d3300 2023-12-20T06:02:54.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.763+0000 7fbb91f09640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb70065d20 0x7fbb700681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:54.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.763+0000 7fbb91f09640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb70065d20 0x7fbb700681e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb88005890 tx=0x7fbb88009040 comp rx=0 tx=0).ready entity=mgr.10824 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:54.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.767+0000 7fbb827fc640 1 -- 172.21.15.32:0/1186594974 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbb7c0eec60 con 0x7fbb8c0d3300 2023-12-20T06:02:54.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:54.963+0000 7fbb9416d640 1 -- 172.21.15.32:0/1186594974 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fbb8c0c5f60 con 0x7fbb8c0d3300 2023-12-20T06:02:55.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.519+0000 7fbb827fc640 1 -- 172.21.15.32:0/1186594974 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 130) v1 ==== 85489+0+0 (secure 0 0 0) 0x7fbb7c0ef030 con 0x7fbb8c0d3300 2023-12-20T06:02:55.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.519+0000 7fbb827fc640 1 -- 172.21.15.32:0/1186594974 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v130) v1 ==== 93+0+0 (secure 0 0 0) 0x7fbb7c0f5a20 con 0x7fbb8c0d3300 2023-12-20T06:02:55.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 -- 172.21.15.32:0/1186594974 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb70065d20 msgr2=0x7fbb700681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:55.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb70065d20 0x7fbb700681e0 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7fbb88005890 tx=0x7fbb88009040 comp rx=0 tx=0).stop 2023-12-20T06:02:55.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 -- 172.21.15.32:0/1186594974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0d3300 msgr2=0x7fbb8c15fff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:55.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0d3300 0x7fbb8c15fff0 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7fbb7c0b3040 tx=0x7fbb7c01ba10 comp rx=0 tx=0).stop 2023-12-20T06:02:55.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 -- 172.21.15.32:0/1186594974 shutdown_connections 2023-12-20T06:02:55.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb70065d20 0x7fbb700681e0 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:55.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb8c15ce60 0x7fbb8c0b2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:55.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb8c0d3300 0x7fbb8c15fff0 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:55.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 --2- 172.21.15.32:0/1186594974 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb8c0d1300 0x7fbb8c1f2350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:55.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 -- 172.21.15.32:0/1186594974 >> 172.21.15.32:0/1186594974 conn(0x7fbb8c06b5e0 msgr2=0x7fbb8c06b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:55.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 -- 172.21.15.32:0/1186594974 shutdown_connections 2023-12-20T06:02:55.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.523+0000 7fbb9416d640 1 -- 172.21.15.32:0/1186594974 wait complete. 2023-12-20T06:02:55.553 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T06:02:55.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:02:55.632 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:02:55.704 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:55.699+0000 7f97d082d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:02:55.710 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:55.711+0000 7f4eaf423180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:02:55.733 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T06:02:55.733 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:02:55.733 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:55.744 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:02:55.744 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:55.794 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:55.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f105610b640 1 Processor -- start 2023-12-20T06:02:55.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f105610b640 1 -- start start 2023-12-20T06:02:55.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f105610b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f10500b1600 0x7f10500afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f105610b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10500b0980 con 0x7f105015ace0 2023-12-20T06:02:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f105610b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1050070570 con 0x7f10500b0530 2023-12-20T06:02:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f105610b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10500cab80 con 0x7f10500b1600 2023-12-20T06:02:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1055109640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f105015ace0 0x7f10500709d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42534/0 (socket says 172.21.15.32:42534) 2023-12-20T06:02:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1055109640 1 -- 172.21.15.32:0/750925395 learned_addr learned my addr 172.21.15.32:0/750925395 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1054908640 1 --2- 172.21.15.32:0/750925395 >> v2:172.21.15.39:3300/0 conn(0x7f10500b1600 0x7f10500afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1625613161 0 0) 0x7f104c001120 con 0x7f105015ace0 2023-12-20T06:02:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f103c003530 con 0x7f105015ace0 2023-12-20T06:02:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1354127978 0 0) 0x7f1050070570 con 0x7f10500b0530 2023-12-20T06:02:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f104c001120 con 0x7f10500b0530 2023-12-20T06:02:55.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 817282558 0 0) 0x7f103c003530 con 0x7f105015ace0 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 >> v2:172.21.15.39:3300/0 conn(0x7f10500b1600 msgr2=0x7f10500afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 --2- 172.21.15.32:0/750925395 >> v2:172.21.15.39:3300/0 conn(0x7f10500b1600 0x7f10500afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 >> v1:172.21.15.39:6789/0 conn(0x7f10500b0530 legacy=0x7f10500cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10500710e0 con 0x7f105015ace0 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f104c001810 con 0x7f105015ace0 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1054908640 1 --2- 172.21.15.32:0/750925395 >> v2:172.21.15.39:3300/0 conn(0x7f10500b1600 0x7f10500afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 >> v1:172.21.15.32:6789/0 conn(0x7f105015ace0 legacy=0x7f10500709d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f103c003b10 0x7f103c003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c0047d0 0x7f103c004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f103c005190 0x7f103c00b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10500cab80 con 0x7f103c005190 2023-12-20T06:02:55.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f103c00c010 con 0x7f103c003b10 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f103c00c1d0 con 0x7f103c0047d0 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1054908640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c0047d0 0x7f103c004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1055109640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f103c003b10 0x7f103c003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f105590a640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f103c005190 0x7f103c00b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1055109640 1 -- 172.21.15.32:0/750925395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c0047d0 msgr2=0x7f103c004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1055109640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c0047d0 0x7f103c004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1055109640 1 -- 172.21.15.32:0/750925395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f103c005190 msgr2=0x7f103c00b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1055109640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f103c005190 0x7f103c00b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1055109640 1 -- 172.21.15.32:0/750925395 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c003530 con 0x7f103c003b10 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f105590a640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f103c005190 0x7f103c00b990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f1054908640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c0047d0 0x7f103c004c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:55.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.983+0000 7f105590a640 1 -- 172.21.15.32:0/750925395 reap_dead start 2023-12-20T06:02:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.987+0000 7f1055109640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f103c003b10 0x7f103c003f10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f104c00c160 tx=0x7f104c01f330 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ea6c9d23d7d29b01 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.987+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c005ae0 con 0x7f103c003b10 2023-12-20T06:02:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.987+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c0ab070 con 0x7f103c003b10 2023-12-20T06:02:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.987+0000 7f1047fff640 1 -- 172.21.15.32:0/750925395 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c007070 con 0x7f103c003b10 2023-12-20T06:02:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.987+0000 7f105610b640 1 -- 172.21.15.32:0/750925395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f103c003b10 msgr2=0x7f103c003f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.987+0000 7f105610b640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f103c003b10 0x7f103c003f10 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f104c00c160 tx=0x7f104c01f330 comp rx=0 tx=0).stop 2023-12-20T06:02:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.987+0000 7f105610b640 1 -- 172.21.15.32:0/750925395 shutdown_connections 2023-12-20T06:02:55.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.987+0000 7f105610b640 1 --2- 172.21.15.32:0/750925395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f103c003b10 0x7f103c003f10 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:55.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.987+0000 7f105610b640 1 -- 172.21.15.32:0/750925395 >> 172.21.15.32:0/750925395 conn(0x7f105006c060 msgr2=0x7f105006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:55.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.987+0000 7f105610b640 1 -- 172.21.15.32:0/750925395 shutdown_connections 2023-12-20T06:02:55.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.987+0000 7f105610b640 1 -- 172.21.15.32:0/750925395 wait complete. 2023-12-20T06:02:55.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f105610b640 1 Processor -- start 2023-12-20T06:02:55.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f105610b640 1 -- start start 2023-12-20T06:02:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f105610b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c003b10 0x7f10500b1eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f105610b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10500b23f0 0x7f10500c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f105610b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500c5b80 0x7f10500c7e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f105610b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10500c7430 con 0x7f10500c5b80 2023-12-20T06:02:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f105610b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10500c67e0 con 0x7f10500b23f0 2023-12-20T06:02:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f105610b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10500c6030 con 0x7f103c003b10 2023-12-20T06:02:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1055109640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c003b10 0x7f10500b1eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1055109640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c003b10 0x7f10500b1eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49438/0 (socket says 172.21.15.32:49438) 2023-12-20T06:02:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1055109640 1 -- 172.21.15.32:0/3649278592 learned_addr learned my addr 172.21.15.32:0/3649278592 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f105590a640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500c5b80 0x7f10500c7e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:55.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1054908640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10500b23f0 0x7f10500c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1055109640 1 -- 172.21.15.32:0/3649278592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10500b23f0 msgr2=0x7f10500c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1055109640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10500b23f0 0x7f10500c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1055109640 1 -- 172.21.15.32:0/3649278592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500c5b80 msgr2=0x7f10500c7e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1055109640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500c5b80 0x7f10500c7e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1055109640 1 -- 172.21.15.32:0/3649278592 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10500b2850 con 0x7f103c003b10 2023-12-20T06:02:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1055109640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c003b10 0x7f10500b1eb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f104c001a50 tx=0x7f103c005800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1045ffb640 1 -- 172.21.15.32:0/3649278592 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c0b6070 con 0x7f103c003b10 2023-12-20T06:02:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1045ffb640 1 -- 172.21.15.32:0/3649278592 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f104c005980 con 0x7f103c003b10 2023-12-20T06:02:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f105610b640 1 -- 172.21.15.32:0/3649278592 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10500c6e30 con 0x7f103c003b10 2023-12-20T06:02:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.991+0000 7f1045ffb640 1 -- 172.21.15.32:0/3649278592 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f104c005cf0 con 0x7f103c003b10 2023-12-20T06:02:55.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.995+0000 7f105610b640 1 -- 172.21.15.32:0/3649278592 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10500c6500 con 0x7f103c003b10 2023-12-20T06:02:55.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.995+0000 7f105610b640 1 -- 172.21.15.32:0/3649278592 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1050074100 con 0x7f103c003b10 2023-12-20T06:02:55.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.995+0000 7f1045ffb640 1 -- 172.21.15.32:0/3649278592 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 130) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f104c0ad020 con 0x7f103c003b10 2023-12-20T06:02:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.995+0000 7f1045ffb640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065d30 0x7f10340681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.995+0000 7f1054908640 1 -- 172.21.15.32:0/3649278592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065d30 msgr2=0x7f10340681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.995+0000 7f1045ffb640 1 -- 172.21.15.32:0/3649278592 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f104c120a90 con 0x7f103c003b10 2023-12-20T06:02:55.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:55.995+0000 7f1054908640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065d30 0x7f10340681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:56.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.003+0000 7f1045ffb640 1 -- 172.21.15.32:0/3649278592 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f104c0edb40 con 0x7f103c003b10 2023-12-20T06:02:56.143 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:56.139+0000 7f97d082d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:02:56.154 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:56.155+0000 7f4eaf423180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:02:56.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.167+0000 7f105610b640 1 -- 172.21.15.32:0/3649278592 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f10500b0980 con 0x7f103c003b10 2023-12-20T06:02:56.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.167+0000 7f1045ffb640 1 -- 172.21.15.32:0/3649278592 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v214) v1 ==== 74+0+20371 (secure 0 0 0) 0x7f10500b0980 con 0x7f103c003b10 2023-12-20T06:02:56.168 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:56.168 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":214,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:50.560918+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:02:25.045126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-20T06:02:25.938205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:02:37.109832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:02:38.440830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/587127774":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3681500114":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/1096471740":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/1437478066":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3373411185":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6833/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6832/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/4110141711":"2023-12-21T06:02:50.560868+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:56.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 -- 172.21.15.32:0/3649278592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065d30 msgr2=0x7f10340681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065d30 0x7f10340681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 -- 172.21.15.32:0/3649278592 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c003b10 msgr2=0x7f10500b1eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c003b10 0x7f10500b1eb0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f104c001a50 tx=0x7f103c005800 comp rx=0 tx=0).stop 2023-12-20T06:02:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 -- 172.21.15.32:0/3649278592 shutdown_connections 2023-12-20T06:02:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065d30 0x7f10340681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10500c5b80 0x7f10500c7e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10500b23f0 0x7f10500c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 --2- 172.21.15.32:0/3649278592 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f103c003b10 0x7f10500b1eb0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 -- 172.21.15.32:0/3649278592 >> 172.21.15.32:0/3649278592 conn(0x7f105006c060 msgr2=0x7f10500b3b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 -- 172.21.15.32:0/3649278592 shutdown_connections 2023-12-20T06:02:56.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.171+0000 7f105610b640 1 -- 172.21.15.32:0/3649278592 wait complete. 2023-12-20T06:02:56.192 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T06:02:25.045126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '208', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T06:02:25.938205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '209', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-20T06:02:37.109832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-20T06:02:38.440830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '213', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:02:56.193 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:02:56.301 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:56.295+0000 7f97d082d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:02:56.318 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:56.319+0000 7f4eaf423180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:02:56.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.343+0000 7f33184b6640 1 Processor -- start 2023-12-20T06:02:56.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.343+0000 7f33184b6640 1 -- start start 2023-12-20T06:02:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.343+0000 7f33184b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33100d3300 0x7f33100d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.343+0000 7f33184b6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33100d2650 con 0x7f331015ce60 2023-12-20T06:02:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.343+0000 7f33184b6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33100d1a40 con 0x7f33100d1300 2023-12-20T06:02:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.343+0000 7f33184b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33100d1d20 con 0x7f33100d3300 2023-12-20T06:02:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.343+0000 7f3316252640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f331015ce60 0x7f33100d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42536/0 (socket says 172.21.15.32:42536) 2023-12-20T06:02:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.343+0000 7f3316252640 1 -- 172.21.15.32:0/1718854211 learned_addr learned my addr 172.21.15.32:0/1718854211 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.343+0000 7f3315a51640 1 --2- 172.21.15.32:0/1718854211 >> v2:172.21.15.32:3300/0 conn(0x7f33100d3300 0x7f33100d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:56.345 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.347+0000 7f3315250640 1 -- 172.21.15.32:0/1718854211 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 457923492 0 0) 0x7f33100d2650 con 0x7f331015ce60 2023-12-20T06:02:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.347+0000 7f3315250640 1 -- 172.21.15.32:0/1718854211 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f32fc003530 con 0x7f331015ce60 2023-12-20T06:02:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.347+0000 7f3315a51640 1 -- 172.21.15.32:0/1718854211 >> v1:172.21.15.39:6789/0 conn(0x7f33100d1300 legacy=0x7f33100b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.347+0000 7f3315250640 1 -- 172.21.15.32:0/1718854211 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2600294662 0 0) 0x7f33100d1a40 con 0x7f33100d1300 2023-12-20T06:02:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.347+0000 7f3315a51640 1 -- 172.21.15.32:0/1718854211 >> v1:172.21.15.32:6789/0 conn(0x7f331015ce60 legacy=0x7f33100d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.347+0000 7f3315a51640 1 -- 172.21.15.32:0/1718854211 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33100d2000 con 0x7f33100d3300 2023-12-20T06:02:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.351+0000 7f3315a51640 1 --2- 172.21.15.32:0/1718854211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33100d3300 0x7f33100d0dc0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f33000097c0 tx=0x7f330001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c61bfacc95708050 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.351+0000 7f3315250640 1 -- 172.21.15.32:0/1718854211 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33000ac640 con 0x7f33100d3300 2023-12-20T06:02:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.351+0000 7f3315250640 1 -- 172.21.15.32:0/1718854211 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33000ac7a0 con 0x7f33100d3300 2023-12-20T06:02:56.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.351+0000 7f3315250640 1 -- 172.21.15.32:0/1718854211 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33000acb10 con 0x7f33100d3300 2023-12-20T06:02:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.351+0000 7f33184b6640 1 -- 172.21.15.32:0/1718854211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33100d3300 msgr2=0x7f33100d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.351+0000 7f33184b6640 1 --2- 172.21.15.32:0/1718854211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33100d3300 0x7f33100d0dc0 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f33000097c0 tx=0x7f330001c480 comp rx=0 tx=0).stop 2023-12-20T06:02:56.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.351+0000 7f33184b6640 1 -- 172.21.15.32:0/1718854211 shutdown_connections 2023-12-20T06:02:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.351+0000 7f33184b6640 1 --2- 172.21.15.32:0/1718854211 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33100d3300 0x7f33100d0dc0 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.351+0000 7f33184b6640 1 -- 172.21.15.32:0/1718854211 >> 172.21.15.32:0/1718854211 conn(0x7f331006b5e0 msgr2=0x7f331006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.351+0000 7f33184b6640 1 -- 172.21.15.32:0/1718854211 shutdown_connections 2023-12-20T06:02:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.351+0000 7f33184b6640 1 -- 172.21.15.32:0/1718854211 wait complete. 2023-12-20T06:02:56.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f33184b6640 1 Processor -- start 2023-12-20T06:02:56.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f33184b6640 1 -- start start 2023-12-20T06:02:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f33184b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33100d1300 0x7f3310157280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f33184b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33100d3300 0x7f331015a640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f33184b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f331015ce60 0x7f3310158b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f33184b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33101577c0 con 0x7f33100d3300 2023-12-20T06:02:56.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f33184b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33101595c0 con 0x7f33100d1300 2023-12-20T06:02:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f33184b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33101598a0 con 0x7f331015ce60 2023-12-20T06:02:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3315a51640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33100d3300 0x7f331015a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3315a51640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33100d3300 0x7f331015a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60186/0 (socket says 172.21.15.32:60186) 2023-12-20T06:02:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3315a51640 1 -- 172.21.15.32:0/2817800017 learned_addr learned my addr 172.21.15.32:0/2817800017 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3316252640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33100d1300 0x7f3310157280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3316a53640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f331015ce60 0x7f3310158b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3315a51640 1 -- 172.21.15.32:0/2817800017 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f331015ce60 msgr2=0x7f3310158b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3315a51640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f331015ce60 0x7f3310158b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3315a51640 1 -- 172.21.15.32:0/2817800017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33100d1300 msgr2=0x7f3310157280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3315a51640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33100d1300 0x7f3310157280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3315a51640 1 -- 172.21.15.32:0/2817800017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3300009420 con 0x7f33100d3300 2023-12-20T06:02:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3316252640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33100d1300 0x7f3310157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:02:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3315a51640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33100d3300 0x7f331015a640 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f3300000c10 tx=0x7f3300019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3306ffd640 1 -- 172.21.15.32:0/2817800017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33000b5600 con 0x7f33100d3300 2023-12-20T06:02:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3306ffd640 1 -- 172.21.15.32:0/2817800017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33000b5760 con 0x7f33100d3300 2023-12-20T06:02:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f33184b6640 1 -- 172.21.15.32:0/2817800017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3310159b80 con 0x7f33100d3300 2023-12-20T06:02:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f3306ffd640 1 -- 172.21.15.32:0/2817800017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33000b5ad0 con 0x7f33100d3300 2023-12-20T06:02:56.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f33184b6640 1 -- 172.21.15.32:0/2817800017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f331015a140 con 0x7f33100d3300 2023-12-20T06:02:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.355+0000 7f33184b6640 1 -- 172.21.15.32:0/2817800017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f331004e000 con 0x7f33100d3300 2023-12-20T06:02:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.359+0000 7f3306ffd640 1 -- 172.21.15.32:0/2817800017 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 130) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f33000b3040 con 0x7f33100d3300 2023-12-20T06:02:56.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.359+0000 7f3306ffd640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f32f4065d60 0x7f32f4068220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.359+0000 7f3306ffd640 1 -- 172.21.15.32:0/2817800017 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f330011cd10 con 0x7f33100d3300 2023-12-20T06:02:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.359+0000 7f3316252640 1 -- 172.21.15.32:0/2817800017 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f32f4065d60 msgr2=0x7f32f4068220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:56.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.359+0000 7f3316252640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f32f4065d60 0x7f32f4068220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:56.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.363+0000 7f3306ffd640 1 -- 172.21.15.32:0/2817800017 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f33000e9f00 con 0x7f33100d3300 2023-12-20T06:02:56.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 -- 172.21.15.32:0/2817800017 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f33100c8cf0 con 0x7f33100d3300 2023-12-20T06:02:56.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f3306ffd640 1 -- 172.21.15.32:0/2817800017 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v214) v1 ==== 93+0+10 (secure 0 0 0) 0x7f33000ed520 con 0x7f33100d3300 2023-12-20T06:02:56.522 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:02:56.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 -- 172.21.15.32:0/2817800017 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f32f4065d60 msgr2=0x7f32f4068220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f32f4065d60 0x7f32f4068220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 -- 172.21.15.32:0/2817800017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33100d3300 msgr2=0x7f331015a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33100d3300 0x7f331015a640 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f3300000c10 tx=0x7f3300019220 comp rx=0 tx=0).stop 2023-12-20T06:02:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 -- 172.21.15.32:0/2817800017 shutdown_connections 2023-12-20T06:02:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f32f4065d60 0x7f32f4068220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f331015ce60 0x7f3310158b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33100d3300 0x7f331015a640 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 --2- 172.21.15.32:0/2817800017 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33100d1300 0x7f3310157280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 -- 172.21.15.32:0/2817800017 >> 172.21.15.32:0/2817800017 conn(0x7f331006b5e0 msgr2=0x7f331006b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 -- 172.21.15.32:0/2817800017 shutdown_connections 2023-12-20T06:02:56.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.523+0000 7f33184b6640 1 -- 172.21.15.32:0/2817800017 wait complete. 2023-12-20T06:02:56.544 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:02:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.695+0000 7fa57612c640 1 Processor -- start 2023-12-20T06:02:56.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.695+0000 7fa57612c640 1 -- start start 2023-12-20T06:02:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.695+0000 7fa57612c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa57015ce60 0x7fa5700d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.695+0000 7fa57612c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5700d14d0 0x7fa5700b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.695+0000 7fa57612c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5700d2650 con 0x7fa5700d3130 2023-12-20T06:02:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.695+0000 7fa57612c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5700d35a0 con 0x7fa57015ce60 2023-12-20T06:02:56.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.695+0000 7fa57612c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700d1a40 con 0x7fa5700d14d0 2023-12-20T06:02:56.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.695+0000 7fa56ffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5700d14d0 0x7fa5700b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:56.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa57015ce60 0x7fa5700d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:56.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56ffff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5700d14d0 0x7fa5700b5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49464/0 (socket says 172.21.15.32:49464) 2023-12-20T06:02:56.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56ffff640 1 -- 172.21.15.32:0/2586064415 learned_addr learned my addr 172.21.15.32:0/2586064415 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:56.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56effd640 1 --1- 172.21.15.32:0/2586064415 >> v1:172.21.15.39:6789/0 conn(0x7fa5700d3130 0x7fa5700d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:43742/0 (socket says 172.21.15.32:43742) 2023-12-20T06:02:56.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56ffff640 1 -- 172.21.15.32:0/2586064415 >> v2:172.21.15.39:3300/0 conn(0x7fa57015ce60 msgr2=0x7fa5700d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:56.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56ffff640 1 --2- 172.21.15.32:0/2586064415 >> v2:172.21.15.39:3300/0 conn(0x7fa57015ce60 0x7fa5700d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56ffff640 1 -- 172.21.15.32:0/2586064415 >> v1:172.21.15.39:6789/0 conn(0x7fa5700d3130 legacy=0x7fa5700d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56ffff640 1 -- 172.21.15.32:0/2586064415 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5700d1d20 con 0x7fa5700d14d0 2023-12-20T06:02:56.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56f7fe640 1 --2- 172.21.15.32:0/2586064415 >> v2:172.21.15.39:3300/0 conn(0x7fa57015ce60 0x7fa5700d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:56.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56ffff640 1 --2- 172.21.15.32:0/2586064415 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5700d14d0 0x7fa5700b5e90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa568009e50 tx=0x7fa56801f1f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7e67cd0ba7f0f86 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:56.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56e7fc640 1 -- 172.21.15.32:0/2586064415 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680af400 con 0x7fa5700d14d0 2023-12-20T06:02:56.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56e7fc640 1 -- 172.21.15.32:0/2586064415 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5680ab050 con 0x7fa5700d14d0 2023-12-20T06:02:56.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.699+0000 7fa56e7fc640 1 -- 172.21.15.32:0/2586064415 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5680af780 con 0x7fa5700d14d0 2023-12-20T06:02:56.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 -- 172.21.15.32:0/2586064415 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5700d14d0 msgr2=0x7fa5700b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 --2- 172.21.15.32:0/2586064415 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5700d14d0 0x7fa5700b5e90 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa568009e50 tx=0x7fa56801f1f0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 -- 172.21.15.32:0/2586064415 shutdown_connections 2023-12-20T06:02:56.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 --2- 172.21.15.32:0/2586064415 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5700d14d0 0x7fa5700b5e90 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 --2- 172.21.15.32:0/2586064415 >> v2:172.21.15.39:3300/0 conn(0x7fa57015ce60 0x7fa5700d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 -- 172.21.15.32:0/2586064415 >> 172.21.15.32:0/2586064415 conn(0x7fa57006b5e0 msgr2=0x7fa57006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:56.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 -- 172.21.15.32:0/2586064415 shutdown_connections 2023-12-20T06:02:56.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 -- 172.21.15.32:0/2586064415 wait complete. 2023-12-20T06:02:56.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 Processor -- start 2023-12-20T06:02:56.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 -- start start 2023-12-20T06:02:56.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5700d14d0 0x7fa5700c89c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:56.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5700d3130 0x7fa57015f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:56.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa57015ce60 0x7fa57015f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:56.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700c85a0 con 0x7fa5700d14d0 2023-12-20T06:02:56.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa57015ff20 con 0x7fa5700d3130 2023-12-20T06:02:56.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa57612c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5700c7c40 con 0x7fa57015ce60 2023-12-20T06:02:56.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5700d14d0 0x7fa5700c89c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:56.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5700d14d0 0x7fa5700c89c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60190/0 (socket says 172.21.15.32:60190) 2023-12-20T06:02:56.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa56f7fe640 1 -- 172.21.15.32:0/2103574450 learned_addr learned my addr 172.21.15.32:0/2103574450 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:56.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa56effd640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5700d3130 0x7fa57015f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:56.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.703+0000 7fa56ffff640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa57015ce60 0x7fa57015f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:56.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa56f7fe640 1 -- 172.21.15.32:0/2103574450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa57015ce60 msgr2=0x7fa57015f8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa56f7fe640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa57015ce60 0x7fa57015f8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa56f7fe640 1 -- 172.21.15.32:0/2103574450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5700d3130 msgr2=0x7fa57015f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa56f7fe640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5700d3130 0x7fa57015f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa56f7fe640 1 -- 172.21.15.32:0/2103574450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa560009420 con 0x7fa5700d14d0 2023-12-20T06:02:56.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa56f7fe640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5700d14d0 0x7fa5700c89c0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fa56000e9b0 tx=0x7fa56000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:56.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa557fff640 1 -- 172.21.15.32:0/2103574450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa56000ca00 con 0x7fa5700d14d0 2023-12-20T06:02:56.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa557fff640 1 -- 172.21.15.32:0/2103574450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa56000cb60 con 0x7fa5700d14d0 2023-12-20T06:02:56.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa557fff640 1 -- 172.21.15.32:0/2103574450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa560013470 con 0x7fa5700d14d0 2023-12-20T06:02:56.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa57612c640 1 -- 172.21.15.32:0/2103574450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa568009b30 con 0x7fa5700d14d0 2023-12-20T06:02:56.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa57612c640 1 -- 172.21.15.32:0/2103574450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5700c7ee0 con 0x7fa5700d14d0 2023-12-20T06:02:56.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa57612c640 1 -- 172.21.15.32:0/2103574450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa57004e000 con 0x7fa5700d14d0 2023-12-20T06:02:56.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa557fff640 1 -- 172.21.15.32:0/2103574450 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 130) v1 ==== 85489+0+0 (secure 0 0 0) 0x7fa5600135d0 con 0x7fa5700d14d0 2023-12-20T06:02:56.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.707+0000 7fa557fff640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa55c065d30 0x7fa55c0681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:56.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.711+0000 7fa557fff640 1 -- 172.21.15.32:0/2103574450 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7fa560083cd0 con 0x7fa5700d14d0 2023-12-20T06:02:56.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.711+0000 7fa56effd640 1 -- 172.21.15.32:0/2103574450 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa55c065d30 msgr2=0x7fa55c0681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:56.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.711+0000 7fa56effd640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa55c065d30 0x7fa55c0681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:56.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.711+0000 7fa557fff640 1 -- 172.21.15.32:0/2103574450 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa560050f00 con 0x7fa5700d14d0 2023-12-20T06:02:56.810 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:56.807+0000 7f97d082d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:56.840 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:56.839+0000 7f4eaf423180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:56.871 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:56.867+0000 7f97d082d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:02:56.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.871+0000 7fa57612c640 1 -- 172.21.15.32:0/2103574450 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fa5700d22b0 con 0x7fa5700d14d0 2023-12-20T06:02:56.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa557fff640 1 -- 172.21.15.32:0/2103574450 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v214) v1 ==== 104+0+11 (secure 0 0 0) 0x7fa560017020 con 0x7fa5700d14d0 2023-12-20T06:02:56.873 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:02:56.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 -- 172.21.15.32:0/2103574450 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa55c065d30 msgr2=0x7fa55c0681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:56.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa55c065d30 0x7fa55c0681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 -- 172.21.15.32:0/2103574450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5700d14d0 msgr2=0x7fa5700c89c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:56.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5700d14d0 0x7fa5700c89c0 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7fa56000e9b0 tx=0x7fa56000ee70 comp rx=0 tx=0).stop 2023-12-20T06:02:56.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 -- 172.21.15.32:0/2103574450 shutdown_connections 2023-12-20T06:02:56.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa55c065d30 0x7fa55c0681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa57015ce60 0x7fa57015f8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5700d3130 0x7fa57015f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 --2- 172.21.15.32:0/2103574450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5700d14d0 0x7fa5700c89c0 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:56.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 -- 172.21.15.32:0/2103574450 >> 172.21.15.32:0/2103574450 conn(0x7fa57006b5e0 msgr2=0x7fa5701e7170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:56.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 -- 172.21.15.32:0/2103574450 shutdown_connections 2023-12-20T06:02:56.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:56.875+0000 7fa57612c640 1 -- 172.21.15.32:0/2103574450 wait complete. 2023-12-20T06:02:56.895 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:02:56.903 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:56.903+0000 7f4eaf423180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:02:56.930 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:56.923+0000 7f97d082d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:02:56.963 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:56.963+0000 7f4eaf423180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:02:57.020 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:57.015+0000 7f97d082d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:02:57.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f39947e9640 1 Processor -- start 2023-12-20T06:02:57.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f39947e9640 1 -- start start 2023-12-20T06:02:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f39947e9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f398c0d3300 0x7f398c0d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f39947e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f398c0d1300 0x7f398c0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f39947e9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f398c0d2650 con 0x7f398c15ce60 2023-12-20T06:02:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f39947e9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f398c0d1a40 con 0x7f398c0d3300 2023-12-20T06:02:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f39947e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0d1d20 con 0x7f398c0d1300 2023-12-20T06:02:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f3992585640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f398c15ce60 0x7f398c0d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42544/0 (socket says 172.21.15.32:42544) 2023-12-20T06:02:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f3992585640 1 -- 172.21.15.32:0/2006482153 learned_addr learned my addr 172.21.15.32:0/2006482153 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f3992d86640 1 --2- 172.21.15.32:0/2006482153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f398c0d1300 0x7f398c0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f3991d84640 1 --2- 172.21.15.32:0/2006482153 >> v2:172.21.15.39:3300/0 conn(0x7f398c0d3300 0x7f398c0d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f3991583640 1 -- 172.21.15.32:0/2006482153 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3554148336 0 0) 0x7f398c0d2650 con 0x7f398c15ce60 2023-12-20T06:02:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f3991583640 1 -- 172.21.15.32:0/2006482153 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3978003530 con 0x7f398c15ce60 2023-12-20T06:02:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f3992d86640 1 -- 172.21.15.32:0/2006482153 >> v2:172.21.15.39:3300/0 conn(0x7f398c0d3300 msgr2=0x7f398c0d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f3992d86640 1 --2- 172.21.15.32:0/2006482153 >> v2:172.21.15.39:3300/0 conn(0x7f398c0d3300 0x7f398c0d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f3992d86640 1 -- 172.21.15.32:0/2006482153 >> v1:172.21.15.32:6789/0 conn(0x7f398c15ce60 legacy=0x7f398c0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f3992d86640 1 -- 172.21.15.32:0/2006482153 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f398c0d2000 con 0x7f398c0d1300 2023-12-20T06:02:57.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.047+0000 7f3991d84640 1 --2- 172.21.15.32:0/2006482153 >> v2:172.21.15.39:3300/0 conn(0x7f398c0d3300 0x7f398c0d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:57.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f3992d86640 1 --2- 172.21.15.32:0/2006482153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f398c0d1300 0x7f398c0b5e90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3984009d90 tx=0x7f398401f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ec9070a24b67399e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:57.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f3991583640 1 -- 172.21.15.32:0/2006482153 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39840af600 con 0x7f398c0d1300 2023-12-20T06:02:57.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f3991583640 1 -- 172.21.15.32:0/2006482153 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f39840ab050 con 0x7f398c0d1300 2023-12-20T06:02:57.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f3991583640 1 -- 172.21.15.32:0/2006482153 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f39840af960 con 0x7f398c0d1300 2023-12-20T06:02:57.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f39947e9640 1 -- 172.21.15.32:0/2006482153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f398c0d1300 msgr2=0x7f398c0b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f39947e9640 1 --2- 172.21.15.32:0/2006482153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f398c0d1300 0x7f398c0b5e90 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7f3984009d90 tx=0x7f398401f130 comp rx=0 tx=0).stop 2023-12-20T06:02:57.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f39947e9640 1 -- 172.21.15.32:0/2006482153 shutdown_connections 2023-12-20T06:02:57.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f39947e9640 1 --2- 172.21.15.32:0/2006482153 >> v2:172.21.15.39:3300/0 conn(0x7f398c0d3300 0x7f398c0d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f39947e9640 1 --2- 172.21.15.32:0/2006482153 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f398c0d1300 0x7f398c0b5e90 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f39947e9640 1 -- 172.21.15.32:0/2006482153 >> 172.21.15.32:0/2006482153 conn(0x7f398c06b5e0 msgr2=0x7f398c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:57.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f39947e9640 1 -- 172.21.15.32:0/2006482153 shutdown_connections 2023-12-20T06:02:57.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.051+0000 7f39947e9640 1 -- 172.21.15.32:0/2006482153 wait complete. 2023-12-20T06:02:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f39947e9640 1 Processor -- start 2023-12-20T06:02:57.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f39947e9640 1 -- start start 2023-12-20T06:02:57.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f39947e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f398c0d1300 0x7f398c1519f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f39947e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f398c0d3300 0x7f398c0cbe30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f39947e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f398c15ce60 0x7f398c0cc370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f39947e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0c9d30 con 0x7f398c0d3300 2023-12-20T06:02:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f39947e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f398c15ede0 con 0x7f398c0d1300 2023-12-20T06:02:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f39947e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f398c0c9ff0 con 0x7f398c15ce60 2023-12-20T06:02:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3991d84640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f398c0d3300 0x7f398c0cbe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3992585640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f398c0d1300 0x7f398c1519f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3991d84640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f398c0d3300 0x7f398c0cbe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60194/0 (socket says 172.21.15.32:60194) 2023-12-20T06:02:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3991d84640 1 -- 172.21.15.32:0/2400187599 learned_addr learned my addr 172.21.15.32:0/2400187599 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:57.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3992d86640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f398c15ce60 0x7f398c0cc370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3991d84640 1 -- 172.21.15.32:0/2400187599 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f398c15ce60 msgr2=0x7f398c0cc370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3991d84640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f398c15ce60 0x7f398c0cc370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3991d84640 1 -- 172.21.15.32:0/2400187599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f398c0d1300 msgr2=0x7f398c1519f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3991d84640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f398c0d1300 0x7f398c1519f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3991d84640 1 -- 172.21.15.32:0/2400187599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f397c009420 con 0x7f398c0d3300 2023-12-20T06:02:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3992585640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f398c0d1300 0x7f398c1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3991d84640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f398c0d3300 0x7f398c0cbe30 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f397c00ebb0 tx=0x7f397c00c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3982ffd640 1 -- 172.21.15.32:0/2400187599 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c00eeb0 con 0x7f398c0d3300 2023-12-20T06:02:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3982ffd640 1 -- 172.21.15.32:0/2400187599 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f397c00cc30 con 0x7f398c0d3300 2023-12-20T06:02:57.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f3982ffd640 1 -- 172.21.15.32:0/2400187599 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f397c013470 con 0x7f398c0d3300 2023-12-20T06:02:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f39947e9640 1 -- 172.21.15.32:0/2400187599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3984009a70 con 0x7f398c0d3300 2023-12-20T06:02:57.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.055+0000 7f39947e9640 1 -- 172.21.15.32:0/2400187599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f398c0ca5b0 con 0x7f398c0d3300 2023-12-20T06:02:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.059+0000 7f39947e9640 1 -- 172.21.15.32:0/2400187599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f398c0cd430 con 0x7f398c0d3300 2023-12-20T06:02:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.059+0000 7f3982ffd640 1 -- 172.21.15.32:0/2400187599 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 130) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f397c0135d0 con 0x7f398c0d3300 2023-12-20T06:02:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.059+0000 7f3982ffd640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3970065d60 0x7f3970068220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.059+0000 7f3992585640 1 -- 172.21.15.32:0/2400187599 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3970065d60 msgr2=0x7f3970068220 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.059+0000 7f3982ffd640 1 -- 172.21.15.32:0/2400187599 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f397c083c80 con 0x7f398c0d3300 2023-12-20T06:02:57.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.059+0000 7f3992585640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3970065d60 0x7f3970068220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:57.060 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:57.059+0000 7f4eaf423180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:02:57.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.063+0000 7f3982ffd640 1 -- 172.21.15.32:0/2400187599 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f397c050f00 con 0x7f398c0d3300 2023-12-20T06:02:57.080 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:57.075+0000 7f97d082d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:02:57.127 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:57.127+0000 7f4eaf423180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:02:57.167 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:57.163+0000 7f97d082d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:02:57.217 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:57.219+0000 7f4eaf423180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:02:57.227 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:57.223+0000 7f97d082d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:02:57.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 -- 172.21.15.32:0/2400187599 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f398c04e000 con 0x7f398c0d3300 2023-12-20T06:02:57.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f3982ffd640 1 -- 172.21.15.32:0/2400187599 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v214) v1 ==== 100+0+11 (secure 0 0 0) 0x7f397c0170a0 con 0x7f398c0d3300 2023-12-20T06:02:57.230 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:02:57.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 -- 172.21.15.32:0/2400187599 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3970065d60 msgr2=0x7f3970068220 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3970065d60 0x7f3970068220 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 -- 172.21.15.32:0/2400187599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f398c0d3300 msgr2=0x7f398c0cbe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f398c0d3300 0x7f398c0cbe30 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f397c00ebb0 tx=0x7f397c00c520 comp rx=0 tx=0).stop 2023-12-20T06:02:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 -- 172.21.15.32:0/2400187599 shutdown_connections 2023-12-20T06:02:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3970065d60 0x7f3970068220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f398c15ce60 0x7f398c0cc370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f398c0d3300 0x7f398c0cbe30 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 --2- 172.21.15.32:0/2400187599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f398c0d1300 0x7f398c1519f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 -- 172.21.15.32:0/2400187599 >> 172.21.15.32:0/2400187599 conn(0x7f398c06b5e0 msgr2=0x7f398c15c110 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 -- 172.21.15.32:0/2400187599 shutdown_connections 2023-12-20T06:02:57.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.231+0000 7f39947e9640 1 -- 172.21.15.32:0/2400187599 wait complete. 2023-12-20T06:02:57.252 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-20T06:02:57.278 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:57.279+0000 7f4eaf423180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:02:57.377 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:57.371+0000 7f97d082d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:02:57.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13fa66640 1 Processor -- start 2023-12-20T06:02:57.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13fa66640 1 -- start start 2023-12-20T06:02:57.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13fa66640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1380ca1a0 0x7ff1380cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13fa66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1380cb0b0 con 0x7ff138164670 2023-12-20T06:02:57.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13fa66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff1380ca810 con 0x7ff138070230 2023-12-20T06:02:57.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13fa66640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff1380ca630 con 0x7ff1380ca1a0 2023-12-20T06:02:57.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13d001640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff1380ca1a0 0x7ff1380cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13d802640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff138164670 0x7ff13806da20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42554/0 (socket says 172.21.15.32:42554) 2023-12-20T06:02:57.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13d802640 1 -- 172.21.15.32:0/2415832871 learned_addr learned my addr 172.21.15.32:0/2415832871 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:57.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13d001640 1 -- 172.21.15.32:0/2415832871 >> v1:172.21.15.39:6789/0 conn(0x7ff138070230 legacy=0x7ff138070610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13d001640 1 -- 172.21.15.32:0/2415832871 >> v1:172.21.15.32:6789/0 conn(0x7ff138164670 legacy=0x7ff13806da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.427+0000 7ff13d001640 1 -- 172.21.15.32:0/2415832871 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff1380caaf0 con 0x7ff1380ca1a0 2023-12-20T06:02:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.431+0000 7ff13d001640 1 --2- 172.21.15.32:0/2415832871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1380ca1a0 0x7ff1380cc3b0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7ff1280097c0 tx=0x7ff12801c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1b735251f2b15c5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.431+0000 7ff13c800640 1 -- 172.21.15.32:0/2415832871 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1280ad660 con 0x7ff1380ca1a0 2023-12-20T06:02:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.431+0000 7ff13c800640 1 -- 172.21.15.32:0/2415832871 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1280ad7c0 con 0x7ff1380ca1a0 2023-12-20T06:02:57.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.431+0000 7ff13c800640 1 -- 172.21.15.32:0/2415832871 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1280adb50 con 0x7ff1380ca1a0 2023-12-20T06:02:57.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.431+0000 7ff13fa66640 1 -- 172.21.15.32:0/2415832871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1380ca1a0 msgr2=0x7ff1380cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.431+0000 7ff13fa66640 1 --2- 172.21.15.32:0/2415832871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1380ca1a0 0x7ff1380cc3b0 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7ff1280097c0 tx=0x7ff12801c360 comp rx=0 tx=0).stop 2023-12-20T06:02:57.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.431+0000 7ff13fa66640 1 -- 172.21.15.32:0/2415832871 shutdown_connections 2023-12-20T06:02:57.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.431+0000 7ff13fa66640 1 --2- 172.21.15.32:0/2415832871 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1380ca1a0 0x7ff1380cc3b0 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.431+0000 7ff13fa66640 1 -- 172.21.15.32:0/2415832871 >> 172.21.15.32:0/2415832871 conn(0x7ff13806a510 msgr2=0x7ff1380ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:57.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.431+0000 7ff13fa66640 1 -- 172.21.15.32:0/2415832871 shutdown_connections 2023-12-20T06:02:57.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.431+0000 7ff13fa66640 1 -- 172.21.15.32:0/2415832871 wait complete. 2023-12-20T06:02:57.433 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:57.435+0000 7f4eaf423180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:02:57.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13fa66640 1 Processor -- start 2023-12-20T06:02:57.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13fa66640 1 -- start start 2023-12-20T06:02:57.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13fa66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff138070230 0x7ff138163b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13fa66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1380ca1a0 0x7ff1380b1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13fa66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff138164670 0x7ff138153360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13fa66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff138158700 con 0x7ff1380ca1a0 2023-12-20T06:02:57.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13fa66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff138157af0 con 0x7ff138070230 2023-12-20T06:02:57.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13fa66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff138157dd0 con 0x7ff138164670 2023-12-20T06:02:57.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13d001640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1380ca1a0 0x7ff1380b1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13d802640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff138070230 0x7ff138163b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13d001640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1380ca1a0 0x7ff1380b1ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60204/0 (socket says 172.21.15.32:60204) 2023-12-20T06:02:57.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13d001640 1 -- 172.21.15.32:0/3730274427 learned_addr learned my addr 172.21.15.32:0/3730274427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:57.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13e003640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff138164670 0x7ff138153360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13d001640 1 -- 172.21.15.32:0/3730274427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff138164670 msgr2=0x7ff138153360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13d001640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff138164670 0x7ff138153360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13d001640 1 -- 172.21.15.32:0/3730274427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff138070230 msgr2=0x7ff138163b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13d001640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff138070230 0x7ff138163b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13d001640 1 -- 172.21.15.32:0/3730274427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff128009420 con 0x7ff1380ca1a0 2023-12-20T06:02:57.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13d802640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff138070230 0x7ff138163b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:57.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13d001640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1380ca1a0 0x7ff1380b1ab0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7ff1280b0040 tx=0x7ff128019100 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:57.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff12e7fc640 1 -- 172.21.15.32:0/3730274427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1280b6820 con 0x7ff1380ca1a0 2023-12-20T06:02:57.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff12e7fc640 1 -- 172.21.15.32:0/3730274427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff1280b69c0 con 0x7ff1380ca1a0 2023-12-20T06:02:57.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff12e7fc640 1 -- 172.21.15.32:0/3730274427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff1280b6d70 con 0x7ff1380ca1a0 2023-12-20T06:02:57.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13fa66640 1 -- 172.21.15.32:0/3730274427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff1381580b0 con 0x7ff1380ca1a0 2023-12-20T06:02:57.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.435+0000 7ff13fa66640 1 -- 172.21.15.32:0/3730274427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff138157710 con 0x7ff1380ca1a0 2023-12-20T06:02:57.439 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:57.435+0000 7f97d082d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:02:57.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.439+0000 7ff13fa66640 1 -- 172.21.15.32:0/3730274427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff13804e000 con 0x7ff1380ca1a0 2023-12-20T06:02:57.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.439+0000 7ff12e7fc640 1 -- 172.21.15.32:0/3730274427 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 130) v1 ==== 85489+0+0 (secure 0 0 0) 0x7ff1280b4040 con 0x7ff1380ca1a0 2023-12-20T06:02:57.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.439+0000 7ff12e7fc640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff11c065d30 0x7ff11c0681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.439+0000 7ff12e7fc640 1 -- 172.21.15.32:0/3730274427 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7ff12811dfc0 con 0x7ff1380ca1a0 2023-12-20T06:02:57.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.439+0000 7ff13d802640 1 -- 172.21.15.32:0/3730274427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff11c065d30 msgr2=0x7ff11c0681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:57.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.439+0000 7ff13d802640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff11c065d30 0x7ff11c0681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:57.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.443+0000 7ff12e7fc640 1 -- 172.21.15.32:0/3730274427 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff1280eb0a0 con 0x7ff1380ca1a0 2023-12-20T06:02:57.497 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:57.499+0000 7f4eaf423180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:02:57.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.603+0000 7ff13fa66640 1 -- 172.21.15.32:0/3730274427 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7ff1380cf4d0 con 0x7ff1380ca1a0 2023-12-20T06:02:57.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff12e7fc640 1 -- 172.21.15.32:0/3730274427 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v214) v1 ==== 107+0+11 (secure 0 0 0) 0x7ff1280ad620 con 0x7ff1380ca1a0 2023-12-20T06:02:57.605 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:02:57.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 -- 172.21.15.32:0/3730274427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff11c065d30 msgr2=0x7ff11c0681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:57.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff11c065d30 0x7ff11c0681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 -- 172.21.15.32:0/3730274427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1380ca1a0 msgr2=0x7ff1380b1ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1380ca1a0 0x7ff1380b1ab0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7ff1280b0040 tx=0x7ff128019100 comp rx=0 tx=0).stop 2023-12-20T06:02:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 -- 172.21.15.32:0/3730274427 shutdown_connections 2023-12-20T06:02:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff11c065d30 0x7ff11c0681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff138164670 0x7ff138153360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff1380ca1a0 0x7ff1380b1ab0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 --2- 172.21.15.32:0/3730274427 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff138070230 0x7ff138163b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 -- 172.21.15.32:0/3730274427 >> 172.21.15.32:0/3730274427 conn(0x7ff13806a510 msgr2=0x7ff1380c4b30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 -- 172.21.15.32:0/3730274427 shutdown_connections 2023-12-20T06:02:57.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.607+0000 7ff13fa66640 1 -- 172.21.15.32:0/3730274427 wait complete. 2023-12-20T06:02:57.628 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-20T06:02:57.687 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:02:57.687 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:02:57.687 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:02:57.696 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:57.691+0000 7f97d082d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:02:57.750 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:02:57.751 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:02:57.751 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:02:57.760 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:57.759+0000 7f4eaf423180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:02:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f196c4640 1 Processor -- start 2023-12-20T06:02:57.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f196c4640 1 -- start start 2023-12-20T06:02:57.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f196c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f1415ce50 0x7f7f1406b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f196c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f140cf7a0 0x7f7f140b09e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f196c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f140b0550 con 0x7f7f1406bb80 2023-12-20T06:02:57.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f196c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7f1406bfc0 con 0x7f7f140cf7a0 2023-12-20T06:02:57.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f196c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f140aff20 con 0x7f7f1415ce50 2023-12-20T06:02:57.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f137fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7f140cf7a0 0x7f7f140b09e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f127fc640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f7f1406bb80 0x7f7f140cf090 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:43768/0 (socket says 172.21.15.32:43768) 2023-12-20T06:02:57.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f12ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f1415ce50 0x7f7f1406b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f127fc640 1 -- 172.21.15.32:0/1135156108 learned_addr learned my addr 172.21.15.32:0/1135156108 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:57.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f137fe640 1 -- 172.21.15.32:0/1135156108 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f1415ce50 msgr2=0x7f7f1406b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:57.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f137fe640 1 --2- 172.21.15.32:0/1135156108 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f1415ce50 0x7f7f1406b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f137fe640 1 -- 172.21.15.32:0/1135156108 >> v1:172.21.15.39:6789/0 conn(0x7f7f1406bb80 legacy=0x7f7f140cf090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f11ffb640 1 -- 172.21.15.32:0/1135156108 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2343437270 0 0) 0x7f7f140b0550 con 0x7f7f1406bb80 2023-12-20T06:02:57.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.815+0000 7f7f137fe640 1 -- 172.21.15.32:0/1135156108 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f140cfdc0 con 0x7f7f140cf7a0 2023-12-20T06:02:57.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.819+0000 7f7f137fe640 1 --2- 172.21.15.32:0/1135156108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f140cf7a0 0x7f7f140b09e0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009d90 tx=0x7f7f0c01c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3b9450f01e80d00 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:57.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.819+0000 7f7f11ffb640 1 -- 172.21.15.32:0/1135156108 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0a59e0 con 0x7f7f140cf7a0 2023-12-20T06:02:57.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.819+0000 7f7f11ffb640 1 -- 172.21.15.32:0/1135156108 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f0c0a5b40 con 0x7f7f140cf7a0 2023-12-20T06:02:57.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.819+0000 7f7f11ffb640 1 -- 172.21.15.32:0/1135156108 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c0ad440 con 0x7f7f140cf7a0 2023-12-20T06:02:57.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 -- 172.21.15.32:0/1135156108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f140cf7a0 msgr2=0x7f7f140b09e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 --2- 172.21.15.32:0/1135156108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f140cf7a0 0x7f7f140b09e0 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009d90 tx=0x7f7f0c01c920 comp rx=0 tx=0).stop 2023-12-20T06:02:57.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 -- 172.21.15.32:0/1135156108 shutdown_connections 2023-12-20T06:02:57.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 --2- 172.21.15.32:0/1135156108 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f140cf7a0 0x7f7f140b09e0 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 --2- 172.21.15.32:0/1135156108 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f1415ce50 0x7f7f1406b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 -- 172.21.15.32:0/1135156108 >> 172.21.15.32:0/1135156108 conn(0x7f7f14099d70 msgr2=0x7f7f140d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:57.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 -- 172.21.15.32:0/1135156108 shutdown_connections 2023-12-20T06:02:57.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 -- 172.21.15.32:0/1135156108 wait complete. 2023-12-20T06:02:57.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 Processor -- start 2023-12-20T06:02:57.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 -- start start 2023-12-20T06:02:57.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f1406bb80 0x7f7f14155bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f140cf7a0 0x7f7f1406f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f1415ce50 0x7f7f1415e140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1415df50 con 0x7f7f140cf7a0 2023-12-20T06:02:57.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1406e6a0 con 0x7f7f1406bb80 2023-12-20T06:02:57.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f196c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f1406e940 con 0x7f7f1415ce50 2023-12-20T06:02:57.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f127fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f140cf7a0 0x7f7f1406f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f127fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f140cf7a0 0x7f7f1406f700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:60212/0 (socket says 172.21.15.32:60212) 2023-12-20T06:02:57.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f127fc640 1 -- 172.21.15.32:0/415254627 learned_addr learned my addr 172.21.15.32:0/415254627 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:57.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f137fe640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f1415ce50 0x7f7f1415e140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.823+0000 7f7f12ffd640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f1406bb80 0x7f7f14155bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:57.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7f127fc640 1 -- 172.21.15.32:0/415254627 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f1415ce50 msgr2=0x7f7f1415e140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7f127fc640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f1415ce50 0x7f7f1415e140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7f127fc640 1 -- 172.21.15.32:0/415254627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f1406bb80 msgr2=0x7f7f14155bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7f127fc640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f1406bb80 0x7f7f14155bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7f127fc640 1 -- 172.21.15.32:0/415254627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f0c009a70 con 0x7f7f140cf7a0 2023-12-20T06:02:57.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7f127fc640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f140cf7a0 0x7f7f1406f700 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f7f080011f0 tx=0x7f7f0800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:57.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7efb7fe640 1 -- 172.21.15.32:0/415254627 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f080074e0 con 0x7f7f140cf7a0 2023-12-20T06:02:57.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7efb7fe640 1 -- 172.21.15.32:0/415254627 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f08007640 con 0x7f7f140cf7a0 2023-12-20T06:02:57.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7efb7fe640 1 -- 172.21.15.32:0/415254627 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f08004a10 con 0x7f7f140cf7a0 2023-12-20T06:02:57.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7f196c4640 1 -- 172.21.15.32:0/415254627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f1406ec20 con 0x7f7f140cf7a0 2023-12-20T06:02:57.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7f196c4640 1 -- 172.21.15.32:0/415254627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f1406f1e0 con 0x7f7f140cf7a0 2023-12-20T06:02:57.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7f196c4640 1 -- 172.21.15.32:0/415254627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f1404e000 con 0x7f7f140cf7a0 2023-12-20T06:02:57.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7efb7fe640 1 -- 172.21.15.32:0/415254627 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 130) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f7f08004b70 con 0x7f7f140cf7a0 2023-12-20T06:02:57.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7efb7fe640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f00065d30 0x7f7f000681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:57.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7efb7fe640 1 -- 172.21.15.32:0/415254627 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f7f08082d40 con 0x7f7f140cf7a0 2023-12-20T06:02:57.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7f12ffd640 1 -- 172.21.15.32:0/415254627 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f00065d30 msgr2=0x7f7f000681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:57.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.827+0000 7f7f12ffd640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f00065d30 0x7f7f000681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:57.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.831+0000 7f7efb7fe640 1 -- 172.21.15.32:0/415254627 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7f0804ff70 con 0x7f7f140cf7a0 2023-12-20T06:02:57.839 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:57.835+0000 7f97d082d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:57.909 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:57.911+0000 7f4eaf423180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:57.970 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:57.963+0000 7f97d082d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:57.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.991+0000 7f7f196c4640 1 -- 172.21.15.32:0/415254627 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f7f140ce120 con 0x7f7f140cf7a0 2023-12-20T06:02:57.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.991+0000 7f7efb7fe640 1 -- 172.21.15.32:0/415254627 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v214) v1 ==== 103+0+11 (secure 0 0 0) 0x7f7f08018070 con 0x7f7f140cf7a0 2023-12-20T06:02:57.992 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:02:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 -- 172.21.15.32:0/415254627 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f00065d30 msgr2=0x7f7f000681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f00065d30 0x7f7f000681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 -- 172.21.15.32:0/415254627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f140cf7a0 msgr2=0x7f7f1406f700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:57.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f140cf7a0 0x7f7f1406f700 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7f7f080011f0 tx=0x7f7f0800c9b0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 -- 172.21.15.32:0/415254627 shutdown_connections 2023-12-20T06:02:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f00065d30 0x7f7f000681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f1415ce50 0x7f7f1415e140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f140cf7a0 0x7f7f1406f700 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 --2- 172.21.15.32:0/415254627 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f1406bb80 0x7f7f14155bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 -- 172.21.15.32:0/415254627 >> 172.21.15.32:0/415254627 conn(0x7f7f14099d70 msgr2=0x7f7f140ae160 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 -- 172.21.15.32:0/415254627 shutdown_connections 2023-12-20T06:02:57.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:57.995+0000 7f7f196c4640 1 -- 172.21.15.32:0/415254627 wait complete. 2023-12-20T06:02:58.021 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T06:02:58.043 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:58.039+0000 7f97d082d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:02:58.051 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:58.051+0000 7f4eaf423180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:02:58.127 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:58.127+0000 7f4eaf423180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:02:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.167+0000 7ff8d1c66640 1 Processor -- start 2023-12-20T06:02:58.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.167+0000 7ff8d1c66640 1 -- start start 2023-12-20T06:02:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8d1c66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc15cd70 0x7ff8cc0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8d1c66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8cc0ce6c0 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8d1c66640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff8cc1615d0 con 0x7ff8cc160ff0 2023-12-20T06:02:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8d1c66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc1618b0 con 0x7ff8cc15cd70 2023-12-20T06:02:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8caffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff8cc0b23d0 0x7ff8cc1608e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42562/0 (socket says 172.21.15.32:42562) 2023-12-20T06:02:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8caffd640 1 -- 172.21.15.32:0/3850534092 learned_addr learned my addr 172.21.15.32:0/3850534092 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:58.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8cb7fe640 1 --2- 172.21.15.32:0/3850534092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc15cd70 0x7ff8cc0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8ca7fc640 1 -- 172.21.15.32:0/3850534092 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1510979428 0 0) 0x7ff8cc0ce6c0 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8ca7fc640 1 -- 172.21.15.32:0/3850534092 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8b4003530 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8ca7fc640 1 -- 172.21.15.32:0/3850534092 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2369492117 0 0) 0x7ff8cc1615d0 con 0x7ff8cc160ff0 2023-12-20T06:02:58.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8ca7fc640 1 -- 172.21.15.32:0/3850534092 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff8cc0ce6c0 con 0x7ff8cc160ff0 2023-12-20T06:02:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8cb7fe640 1 -- 172.21.15.32:0/3850534092 >> v1:172.21.15.39:6789/0 conn(0x7ff8cc160ff0 legacy=0x7ff8cc06b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8cb7fe640 1 -- 172.21.15.32:0/3850534092 >> v1:172.21.15.32:6789/0 conn(0x7ff8cc0b23d0 legacy=0x7ff8cc1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:58.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.171+0000 7ff8cb7fe640 1 -- 172.21.15.32:0/3850534092 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8cc0ce960 con 0x7ff8cc15cd70 2023-12-20T06:02:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8cb7fe640 1 --2- 172.21.15.32:0/3850534092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc15cd70 0x7ff8cc0cf160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009820 tx=0x7ff8bc01c470 comp rx=0 tx=0).ready entity=mon.2 client_cookie=12ebf5bbb6f49168 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8ca7fc640 1 -- 172.21.15.32:0/3850534092 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0ad680 con 0x7ff8cc15cd70 2023-12-20T06:02:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8ca7fc640 1 -- 172.21.15.32:0/3850534092 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc0ad7e0 con 0x7ff8cc15cd70 2023-12-20T06:02:58.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8ca7fc640 1 -- 172.21.15.32:0/3850534092 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0adb70 con 0x7ff8cc15cd70 2023-12-20T06:02:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3850534092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc15cd70 msgr2=0x7ff8cc0cf160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8d1c66640 1 --2- 172.21.15.32:0/3850534092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc15cd70 0x7ff8cc0cf160 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009820 tx=0x7ff8bc01c470 comp rx=0 tx=0).stop 2023-12-20T06:02:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3850534092 shutdown_connections 2023-12-20T06:02:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8d1c66640 1 --2- 172.21.15.32:0/3850534092 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc15cd70 0x7ff8cc0cf160 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3850534092 >> 172.21.15.32:0/3850534092 conn(0x7ff8cc0cd3d0 msgr2=0x7ff8cc0cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3850534092 shutdown_connections 2023-12-20T06:02:58.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3850534092 wait complete. 2023-12-20T06:02:58.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8d1c66640 1 Processor -- start 2023-12-20T06:02:58.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.175+0000 7ff8d1c66640 1 -- start start 2023-12-20T06:02:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8d1c66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc0b23d0 0x7ff8cc0c72c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8d1c66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc15cd70 0x7ff8cc0b3500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8d1c66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc160ff0 0x7ff8cc0b17c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8d1c66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc156fa0 con 0x7ff8cc15cd70 2023-12-20T06:02:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8d1c66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0c8660 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8d1c66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff8cc0c7ac0 con 0x7ff8cc160ff0 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8caffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc15cd70 0x7ff8cc0b3500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8cb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc0b23d0 0x7ff8cc0c72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8cbfff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc160ff0 0x7ff8cc0b17c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8cb7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc0b23d0 0x7ff8cc0c72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:42988/0 (socket says 172.21.15.32:42988) 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8cb7fe640 1 -- 172.21.15.32:0/3789679174 learned_addr learned my addr 172.21.15.32:0/3789679174 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8cb7fe640 1 -- 172.21.15.32:0/3789679174 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc160ff0 msgr2=0x7ff8cc0b17c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8cb7fe640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc160ff0 0x7ff8cc0b17c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8cb7fe640 1 -- 172.21.15.32:0/3789679174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc15cd70 msgr2=0x7ff8cc0b3500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8cb7fe640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc15cd70 0x7ff8cc0b3500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8cb7fe640 1 -- 172.21.15.32:0/3789679174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff8bc009420 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8caffd640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc15cd70 0x7ff8cc0b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8cbfff640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc160ff0 0x7ff8cc0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8cb7fe640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc0b23d0 0x7ff8cc0c72c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009740 tx=0x7ff8bc01acb0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8b3fff640 1 -- 172.21.15.32:0/3789679174 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0be070 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8b3fff640 1 -- 172.21.15.32:0/3789679174 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff8bc0adb60 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3789679174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff8cc0c7800 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8b3fff640 1 -- 172.21.15.32:0/3789679174 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff8bc0b8420 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3789679174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff8cc0c8010 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.179+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3789679174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff8cc04e000 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.183+0000 7ff8b3fff640 1 -- 172.21.15.32:0/3789679174 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 130) v1 ==== 85489+0+0 (secure 0 0 0) 0x7ff8bc0b4040 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.183+0000 7ff8b3fff640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff8b8065d30 0x7ff8b80681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:58.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.183+0000 7ff8b3fff640 1 -- 172.21.15.32:0/3789679174 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7ff8bc11d570 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.183+0000 7ff8caffd640 1 -- 172.21.15.32:0/3789679174 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff8b8065d30 msgr2=0x7ff8b80681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:58.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.183+0000 7ff8caffd640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff8b8065d30 0x7ff8b80681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:58.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.187+0000 7ff8b3fff640 1 -- 172.21.15.32:0/3789679174 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff8bc0ea720 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.186 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:58.183+0000 7f97d082d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:58.273 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:58.275+0000 7f4eaf423180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:02:58.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.355+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3789679174 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7ff8cc0b68d0 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8b3fff640 1 -- 172.21.15.32:0/3789679174 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 8 entries v214) v1 ==== 80+0+459 (secure 0 0 0) 0x7ff8bc0edd40 con 0x7ff8cc0b23d0 2023-12-20T06:02:58.357 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/587127774 2023-12-21T06:02:50.560868+0000 2023-12-20T06:02:58.357 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3681500114 2023-12-21T06:02:50.560868+0000 2023-12-20T06:02:58.357 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1096471740 2023-12-21T06:02:50.560868+0000 2023-12-20T06:02:58.357 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1437478066 2023-12-21T06:02:50.560868+0000 2023-12-20T06:02:58.357 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3373411185 2023-12-21T06:02:50.560868+0000 2023-12-20T06:02:58.357 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/64546 2023-12-21T06:02:50.560868+0000 2023-12-20T06:02:58.357 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/64546 2023-12-21T06:02:50.560868+0000 2023-12-20T06:02:58.357 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/4110141711 2023-12-21T06:02:50.560868+0000 2023-12-20T06:02:58.358 INFO:teuthology.orchestra.run.smithi032.stderr:listed 8 entries 2023-12-20T06:02:58.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3789679174 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff8b8065d30 msgr2=0x7ff8b80681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:58.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff8b8065d30 0x7ff8b80681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3789679174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc0b23d0 msgr2=0x7ff8cc0c72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:58.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc0b23d0 0x7ff8cc0c72c0 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7ff8bc009740 tx=0x7ff8bc01acb0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3789679174 shutdown_connections 2023-12-20T06:02:58.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff8b8065d30 0x7ff8b80681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff8cc160ff0 0x7ff8cc0b17c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff8cc15cd70 0x7ff8cc0b3500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 --2- 172.21.15.32:0/3789679174 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff8cc0b23d0 0x7ff8cc0c72c0 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3789679174 >> 172.21.15.32:0/3789679174 conn(0x7ff8cc0cd3d0 msgr2=0x7ff8cc160870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:58.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3789679174 shutdown_connections 2023-12-20T06:02:58.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.359+0000 7ff8d1c66640 1 -- 172.21.15.32:0/3789679174 wait complete. 2023-12-20T06:02:58.379 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T06:02:58.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:58.379 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T06:02:58.447 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T06:02:58.447 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:02:58.447 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T06:02:58.497 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:02:58.497 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:58.506 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:02:58.507 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:02:58.507 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:58.549 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:02:58.549 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:02:58.549 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:02:58.559 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:02:58.559 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:02:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.739+0000 7f42eb6fc640 1 Processor -- start 2023-12-20T06:02:58.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.739+0000 7f42eb6fc640 1 -- start start 2023-12-20T06:02:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42eb6fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f42e406ce60 0x7f42e415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42eb6fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e415f110 0x7f42e415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42eb6fc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42e415bbd0 con 0x7f42e40af5e0 2023-12-20T06:02:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42eb6fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42e415beb0 con 0x7f42e406ce60 2023-12-20T06:02:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42eb6fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e415e910 con 0x7f42e415f110 2023-12-20T06:02:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42e9498640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f42e40af5e0 0x7f42e40af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:42576/0 (socket says 172.21.15.32:42576) 2023-12-20T06:02:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42e9498640 1 -- 172.21.15.32:0/1820319394 learned_addr learned my addr 172.21.15.32:0/1820319394 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42e8c97640 1 --2- 172.21.15.32:0/1820319394 >> v2:172.21.15.39:3300/0 conn(0x7f42e406ce60 0x7f42e415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42e9c99640 1 --2- 172.21.15.32:0/1820319394 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e415f110 0x7f42e415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:58.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42dbfff640 1 -- 172.21.15.32:0/1820319394 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 137819079 0 0) 0x7f42e415bbd0 con 0x7f42e40af5e0 2023-12-20T06:02:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42dbfff640 1 -- 172.21.15.32:0/1820319394 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42d0003530 con 0x7f42e40af5e0 2023-12-20T06:02:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42e8c97640 1 -- 172.21.15.32:0/1820319394 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e415f110 msgr2=0x7f42e415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42e8c97640 1 --2- 172.21.15.32:0/1820319394 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e415f110 0x7f42e415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42e8c97640 1 -- 172.21.15.32:0/1820319394 >> v1:172.21.15.32:6789/0 conn(0x7f42e40af5e0 legacy=0x7f42e40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:58.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42e8c97640 1 -- 172.21.15.32:0/1820319394 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42e415ebf0 con 0x7f42e406ce60 2023-12-20T06:02:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.743+0000 7f42e8c97640 1 --2- 172.21.15.32:0/1820319394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e406ce60 0x7f42e415b280 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f42d40097c0 tx=0x7f42d401c3f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=98e6685a9a02e20a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42dbfff640 1 -- 172.21.15.32:0/1820319394 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e415b9d0 con 0x7f42e406ce60 2023-12-20T06:02:58.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42dbfff640 1 -- 172.21.15.32:0/1820319394 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42d40ad640 con 0x7f42e406ce60 2023-12-20T06:02:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42dbfff640 1 -- 172.21.15.32:0/1820319394 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42d40ad9d0 con 0x7f42e406ce60 2023-12-20T06:02:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 -- 172.21.15.32:0/1820319394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e406ce60 msgr2=0x7f42e415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 --2- 172.21.15.32:0/1820319394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e406ce60 0x7f42e415b280 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7f42d40097c0 tx=0x7f42d401c3f0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 -- 172.21.15.32:0/1820319394 shutdown_connections 2023-12-20T06:02:58.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 --2- 172.21.15.32:0/1820319394 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e406ce60 0x7f42e415b280 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 --2- 172.21.15.32:0/1820319394 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e415f110 0x7f42e415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 -- 172.21.15.32:0/1820319394 >> 172.21.15.32:0/1820319394 conn(0x7f42e406a320 msgr2=0x7f42e40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:58.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 -- 172.21.15.32:0/1820319394 shutdown_connections 2023-12-20T06:02:58.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 -- 172.21.15.32:0/1820319394 wait complete. 2023-12-20T06:02:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 Processor -- start 2023-12-20T06:02:58.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 -- start start 2023-12-20T06:02:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e406ce60 0x7f42e40c1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42e40af5e0 0x7f42e4158e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e415f110 0x7f42e4159380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e40704f0 con 0x7f42e40af5e0 2023-12-20T06:02:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42e4159a20 con 0x7f42e415f110 2023-12-20T06:02:58.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.747+0000 7f42eb6fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42e4159d00 con 0x7f42e406ce60 2023-12-20T06:02:58.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42e9498640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e406ce60 0x7f42e40c1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:58.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42e8c97640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42e40af5e0 0x7f42e4158e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:58.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42e9c99640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e415f110 0x7f42e4159380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:58.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42e9498640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e406ce60 0x7f42e40c1800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49550/0 (socket says 172.21.15.32:49550) 2023-12-20T06:02:58.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42e9498640 1 -- 172.21.15.32:0/3484857277 learned_addr learned my addr 172.21.15.32:0/3484857277 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:58.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42e9498640 1 -- 172.21.15.32:0/3484857277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e415f110 msgr2=0x7f42e4159380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42e9498640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e415f110 0x7f42e4159380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42e9498640 1 -- 172.21.15.32:0/3484857277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42e40af5e0 msgr2=0x7f42e4158e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42e9498640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42e40af5e0 0x7f42e4158e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42e9498640 1 -- 172.21.15.32:0/3484857277 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42d4009420 con 0x7f42e406ce60 2023-12-20T06:02:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42e9498640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e406ce60 0x7f42e40c1800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f42e0001270 tx=0x7f42e000cb10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42d9ffb640 1 -- 172.21.15.32:0/3484857277 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e0007270 con 0x7f42e406ce60 2023-12-20T06:02:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42d9ffb640 1 -- 172.21.15.32:0/3484857277 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42e00073d0 con 0x7f42e406ce60 2023-12-20T06:02:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42d9ffb640 1 -- 172.21.15.32:0/3484857277 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42e000ce50 con 0x7f42e406ce60 2023-12-20T06:02:58.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42eb6fc640 1 -- 172.21.15.32:0/3484857277 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42e4159fe0 con 0x7f42e406ce60 2023-12-20T06:02:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42eb6fc640 1 -- 172.21.15.32:0/3484857277 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42e415a5a0 con 0x7f42e406ce60 2023-12-20T06:02:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42eb6fc640 1 -- 172.21.15.32:0/3484857277 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42e40cbeb0 con 0x7f42e406ce60 2023-12-20T06:02:58.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.751+0000 7f42d9ffb640 1 -- 172.21.15.32:0/3484857277 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 130) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f42e00057b0 con 0x7f42e406ce60 2023-12-20T06:02:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.755+0000 7f42d9ffb640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42c8065d30 0x7f42c80681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.755+0000 7f42d9ffb640 1 -- 172.21.15.32:0/3484857277 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f42e0083130 con 0x7f42e406ce60 2023-12-20T06:02:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.755+0000 7f42e8c97640 1 -- 172.21.15.32:0/3484857277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42c8065d30 msgr2=0x7f42c80681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:58.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.755+0000 7f42e8c97640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42c8065d30 0x7f42c80681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:58.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.759+0000 7f42d9ffb640 1 -- 172.21.15.32:0/3484857277 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f42e0050320 con 0x7f42e406ce60 2023-12-20T06:02:58.757 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:58.751+0000 7f97d082d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:02:58.846 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:58.843+0000 7f97d082d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:02:58.859 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:58.859+0000 7f4eaf423180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:02:58.947 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:58.947+0000 7f4eaf423180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:02:58.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.955+0000 7f42eb6fc640 1 -- 172.21.15.32:0/3484857277 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f42e4159e40 con 0x7f42e406ce60 2023-12-20T06:02:58.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.955+0000 7f42d9ffb640 1 -- 172.21.15.32:0/3484857277 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5902 (secure 0 0 0) 0x7f42e4159e40 con 0x7f42e406ce60 2023-12-20T06:02:58.954 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:58.955 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:26.942526+0000","modified":"2023-12-20T06:02:29.955675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10539":{"gid":10539,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/888710554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":888710554},{"type":"v1","addr":"172.21.15.32:6837","nonce":888710554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:39.446992+0000","modified":"2023-12-20T06:02:42.509418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10530":{"gid":10530,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/3001088147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3001088147},{"type":"v1","addr":"172.21.15.39:6833","nonce":3001088147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-20T06:02:58.955 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 102 2023-12-20T06:02:58.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42e8c97640 1 -- 172.21.15.32:0/3484857277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42c8065d30 msgr2=0x7f42c80681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:58.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42e8c97640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42c8065d30 0x7f42c80681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 -- 172.21.15.32:0/3484857277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42c8065d30 msgr2=0x7f42c80681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42c8065d30 0x7f42c80681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 -- 172.21.15.32:0/3484857277 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e406ce60 msgr2=0x7f42e40c1800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e406ce60 0x7f42e40c1800 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f42e0001270 tx=0x7f42e000cb10 comp rx=0 tx=0).stop 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 -- 172.21.15.32:0/3484857277 shutdown_connections 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42c8065d30 0x7f42c80681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42e415f110 0x7f42e4159380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42e40af5e0 0x7f42e4158e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 --2- 172.21.15.32:0/3484857277 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42e406ce60 0x7f42e40c1800 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 -- 172.21.15.32:0/3484857277 >> 172.21.15.32:0/3484857277 conn(0x7f42e406a320 msgr2=0x7f42e40cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 -- 172.21.15.32:0/3484857277 shutdown_connections 2023-12-20T06:02:58.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:58.959+0000 7f42eb6fc640 1 -- 172.21.15.32:0/3484857277 wait complete. 2023-12-20T06:02:58.981 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:02:59.007 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:59.003+0000 7f97d082d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:02:59.094 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:59.091+0000 7f97d082d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:02:59.114 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.115+0000 7f4eaf423180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:02:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.127+0000 7f9529804640 1 Processor -- start 2023-12-20T06:02:59.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.127+0000 7f9529804640 1 -- start start 2023-12-20T06:02:59.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9529804640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f952415cac0 0x7f9524160800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9529804640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f952415cea0 0x7f95240cd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9529804640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95240d0c90 con 0x7f95240b84d0 2023-12-20T06:02:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9529804640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9524161110 con 0x7f952415cac0 2023-12-20T06:02:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9529804640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95241613f0 con 0x7f952415cea0 2023-12-20T06:02:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9522ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f952415cac0 0x7f9524160800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9522ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f952415cac0 0x7f9524160800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33468/0 (socket says 172.21.15.32:33468) 2023-12-20T06:02:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9522ffd640 1 -- 172.21.15.32:0/905415580 learned_addr learned my addr 172.21.15.32:0/905415580 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f951bfff640 1 --2- 172.21.15.32:0/905415580 >> v2:172.21.15.39:3300/0 conn(0x7f952415cea0 0x7f95240cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9522ffd640 1 -- 172.21.15.32:0/905415580 >> v2:172.21.15.39:3300/0 conn(0x7f952415cea0 msgr2=0x7f95240cd760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:59.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9522ffd640 1 --2- 172.21.15.32:0/905415580 >> v2:172.21.15.39:3300/0 conn(0x7f952415cea0 0x7f95240cd760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9522ffd640 1 -- 172.21.15.32:0/905415580 >> v1:172.21.15.39:6789/0 conn(0x7f95240b84d0 legacy=0x7f95240b4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.131+0000 7f9522ffd640 1 -- 172.21.15.32:0/905415580 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95240d0f30 con 0x7f952415cac0 2023-12-20T06:02:59.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f9522ffd640 1 --2- 172.21.15.32:0/905415580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952415cac0 0x7f9524160800 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f95240cd070 tx=0x7f951401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f36674bf495f89a9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f95227fc640 1 -- 172.21.15.32:0/905415580 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140aa5f0 con 0x7f952415cac0 2023-12-20T06:02:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f95227fc640 1 -- 172.21.15.32:0/905415580 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95140aa750 con 0x7f952415cac0 2023-12-20T06:02:59.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f95227fc640 1 -- 172.21.15.32:0/905415580 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95140aaae0 con 0x7f952415cac0 2023-12-20T06:02:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f9529804640 1 -- 172.21.15.32:0/905415580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952415cac0 msgr2=0x7f9524160800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f9529804640 1 --2- 172.21.15.32:0/905415580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952415cac0 0x7f9524160800 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7f95240cd070 tx=0x7f951401c470 comp rx=0 tx=0).stop 2023-12-20T06:02:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f9529804640 1 -- 172.21.15.32:0/905415580 shutdown_connections 2023-12-20T06:02:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f9529804640 1 --2- 172.21.15.32:0/905415580 >> v2:172.21.15.39:3300/0 conn(0x7f952415cea0 0x7f95240cd760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f9529804640 1 --2- 172.21.15.32:0/905415580 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f952415cac0 0x7f9524160800 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f9529804640 1 -- 172.21.15.32:0/905415580 >> 172.21.15.32:0/905415580 conn(0x7f952406afd0 msgr2=0x7f952406b3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f9529804640 1 -- 172.21.15.32:0/905415580 shutdown_connections 2023-12-20T06:02:59.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.135+0000 7f9529804640 1 -- 172.21.15.32:0/905415580 wait complete. 2023-12-20T06:02:59.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9529804640 1 Processor -- start 2023-12-20T06:02:59.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9529804640 1 -- start start 2023-12-20T06:02:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9529804640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95240b84d0 0x7f952414b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9529804640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952415cac0 0x7f9524150700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9529804640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f952415cea0 0x7f95240c6b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9529804640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952415d440 con 0x7f95240b84d0 2023-12-20T06:02:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9529804640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f952415d720 con 0x7f952415cac0 2023-12-20T06:02:59.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9529804640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f952415da00 con 0x7f952415cea0 2023-12-20T06:02:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9522ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95240b84d0 0x7f952414b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f951bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952415cac0 0x7f9524150700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f951bfff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952415cac0 0x7f9524150700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:60710/0 (socket says 172.21.15.32:60710) 2023-12-20T06:02:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f951bfff640 1 -- 172.21.15.32:0/2158801325 learned_addr learned my addr 172.21.15.32:0/2158801325 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f95237fe640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f952415cea0 0x7f95240c6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f951bfff640 1 -- 172.21.15.32:0/2158801325 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f952415cea0 msgr2=0x7f95240c6b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f951bfff640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f952415cea0 0x7f95240c6b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f951bfff640 1 -- 172.21.15.32:0/2158801325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95240b84d0 msgr2=0x7f952414b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f951bfff640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95240b84d0 0x7f952414b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f951bfff640 1 -- 172.21.15.32:0/2158801325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95140094d0 con 0x7f952415cac0 2023-12-20T06:02:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f951bfff640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952415cac0 0x7f9524150700 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f950c00e840 tx=0x7f950c00ec40 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9520ff9640 1 -- 172.21.15.32:0/2158801325 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c009510 con 0x7f952415cac0 2023-12-20T06:02:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9520ff9640 1 -- 172.21.15.32:0/2158801325 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f950c00ee90 con 0x7f952415cac0 2023-12-20T06:02:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9520ff9640 1 -- 172.21.15.32:0/2158801325 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f950c00cb80 con 0x7f952415cac0 2023-12-20T06:02:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9529804640 1 -- 172.21.15.32:0/2158801325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f952415dce0 con 0x7f952415cac0 2023-12-20T06:02:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.139+0000 7f9529804640 1 -- 172.21.15.32:0/2158801325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f952415e4e0 con 0x7f952415cac0 2023-12-20T06:02:59.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.143+0000 7f9529804640 1 -- 172.21.15.32:0/2158801325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9524074100 con 0x7f952415cac0 2023-12-20T06:02:59.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.143+0000 7f9520ff9640 1 -- 172.21.15.32:0/2158801325 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 130) v1 ==== 85489+0+0 (secure 0 0 0) 0x7f950c00cce0 con 0x7f952415cac0 2023-12-20T06:02:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.143+0000 7f9520ff9640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9504065d30 0x7f95040681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.143+0000 7f9520ff9640 1 -- 172.21.15.32:0/2158801325 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7f950c083b00 con 0x7f952415cac0 2023-12-20T06:02:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.143+0000 7f9522ffd640 1 -- 172.21.15.32:0/2158801325 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9504065d30 msgr2=0x7f95040681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:59.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.143+0000 7f9522ffd640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9504065d30 0x7f95040681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:59.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.147+0000 7f9520ff9640 1 -- 172.21.15.32:0/2158801325 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f950c085e60 con 0x7f952415cac0 2023-12-20T06:02:59.207 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.207+0000 7f4eaf423180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:02:59.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.307+0000 7f9529804640 1 -- 172.21.15.32:0/2158801325 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f952415d860 con 0x7f952415cac0 2023-12-20T06:02:59.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9520ff9640 1 -- 172.21.15.32:0/2158801325 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v214) v1 ==== 74+0+20371 (secure 0 0 0) 0x7f952415d860 con 0x7f952415cac0 2023-12-20T06:02:59.310 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:02:59.310 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":214,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:50.560918+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:02:25.045126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-20T06:02:25.938205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:02:37.109832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:02:38.440830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/587127774":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3681500114":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/1096471740":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/1437478066":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3373411185":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6833/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6832/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/4110141711":"2023-12-21T06:02:50.560868+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:02:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 -- 172.21.15.32:0/2158801325 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9504065d30 msgr2=0x7f95040681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9504065d30 0x7f95040681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 -- 172.21.15.32:0/2158801325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952415cac0 msgr2=0x7f9524150700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952415cac0 0x7f9524150700 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f950c00e840 tx=0x7f950c00ec40 comp rx=0 tx=0).stop 2023-12-20T06:02:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 -- 172.21.15.32:0/2158801325 shutdown_connections 2023-12-20T06:02:59.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9504065d30 0x7f95040681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f952415cea0 0x7f95240c6b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f952415cac0 0x7f9524150700 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 --2- 172.21.15.32:0/2158801325 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95240b84d0 0x7f952414b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 -- 172.21.15.32:0/2158801325 >> 172.21.15.32:0/2158801325 conn(0x7f952406afd0 msgr2=0x7f952406b5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 -- 172.21.15.32:0/2158801325 shutdown_connections 2023-12-20T06:02:59.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.311+0000 7f9529804640 1 -- 172.21.15.32:0/2158801325 wait complete. 2023-12-20T06:02:59.332 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 24, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T06:02:25.045126+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '208', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 25, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T06:02:25.938205+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '209', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.3799999952316284, 'score_stable': 1.3799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.3799999952316284, 'raw_score_stable': 1.3799999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-20T06:02:37.109832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-20T06:02:38.440830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '213', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:02:59.333 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:02:59.431 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:59.427+0000 7f97d082d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:02:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff358d25640 1 Processor -- start 2023-12-20T06:02:59.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff358d25640 1 -- start start 2023-12-20T06:02:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff358d25640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff35415abb0 0x7ff3540caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff358d25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3540b0cb0 0x7ff35415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff358d25640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff35415b6a0 con 0x7ff354070230 2023-12-20T06:02:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff358d25640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff35406e160 con 0x7ff35415abb0 2023-12-20T06:02:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff358d25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35415c070 con 0x7ff3540b0cb0 2023-12-20T06:02:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff35259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff35415abb0 0x7ff3540caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff35259c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff35415abb0 0x7ff3540caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33492/0 (socket says 172.21.15.32:33492) 2023-12-20T06:02:59.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff35259c640 1 -- 172.21.15.32:0/2210372865 learned_addr learned my addr 172.21.15.32:0/2210372865 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:59.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff351d9b640 1 --2- 172.21.15.32:0/2210372865 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3540b0cb0 0x7ff35415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff35259c640 1 -- 172.21.15.32:0/2210372865 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3540b0cb0 msgr2=0x7ff35415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:59.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff35259c640 1 --2- 172.21.15.32:0/2210372865 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3540b0cb0 0x7ff35415bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff35259c640 1 -- 172.21.15.32:0/2210372865 >> v1:172.21.15.39:6789/0 conn(0x7ff354070230 legacy=0x7ff35406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.483+0000 7ff35259c640 1 -- 172.21.15.32:0/2210372865 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff35406d8c0 con 0x7ff35415abb0 2023-12-20T06:02:59.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff35259c640 1 --2- 172.21.15.32:0/2210372865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35415abb0 0x7ff3540caaf0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7ff344009870 tx=0x7ff34401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4884801b43dc29c9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:59.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff35159a640 1 -- 172.21.15.32:0/2210372865 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3440a9400 con 0x7ff35415abb0 2023-12-20T06:02:59.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff35159a640 1 -- 172.21.15.32:0/2210372865 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3440a9560 con 0x7ff35415abb0 2023-12-20T06:02:59.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff35159a640 1 -- 172.21.15.32:0/2210372865 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3440a98f0 con 0x7ff35415abb0 2023-12-20T06:02:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff358d25640 1 -- 172.21.15.32:0/2210372865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35415abb0 msgr2=0x7ff3540caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff358d25640 1 --2- 172.21.15.32:0/2210372865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35415abb0 0x7ff3540caaf0 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7ff344009870 tx=0x7ff34401c540 comp rx=0 tx=0).stop 2023-12-20T06:02:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff358d25640 1 -- 172.21.15.32:0/2210372865 shutdown_connections 2023-12-20T06:02:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff358d25640 1 --2- 172.21.15.32:0/2210372865 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff3540b0cb0 0x7ff35415bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff358d25640 1 --2- 172.21.15.32:0/2210372865 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff35415abb0 0x7ff3540caaf0 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff358d25640 1 -- 172.21.15.32:0/2210372865 >> 172.21.15.32:0/2210372865 conn(0x7ff35406a510 msgr2=0x7ff3540ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff358d25640 1 -- 172.21.15.32:0/2210372865 shutdown_connections 2023-12-20T06:02:59.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.487+0000 7ff358d25640 1 -- 172.21.15.32:0/2210372865 wait complete. 2023-12-20T06:02:59.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff358d25640 1 Processor -- start 2023-12-20T06:02:59.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff358d25640 1 -- start start 2023-12-20T06:02:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff358d25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff354070230 0x7ff3541494c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff358d25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3540b0cb0 0x7ff3541505a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff358d25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff35415abb0 0x7ff3541f34a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff358d25640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff35414eb50 con 0x7ff354070230 2023-12-20T06:02:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff358d25640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff354149f50 con 0x7ff3540b0cb0 2023-12-20T06:02:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff358d25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff35414a230 con 0x7ff35415abb0 2023-12-20T06:02:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff35259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff354070230 0x7ff3541494c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff35259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff354070230 0x7ff3541494c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33496/0 (socket says 172.21.15.32:33496) 2023-12-20T06:02:59.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff35259c640 1 -- 172.21.15.32:0/1208319727 learned_addr learned my addr 172.21.15.32:0/1208319727 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff351d9b640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3540b0cb0 0x7ff3541505a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff352d9d640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff35415abb0 0x7ff3541f34a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff35259c640 1 -- 172.21.15.32:0/1208319727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff35415abb0 msgr2=0x7ff3541f34a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff35259c640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff35415abb0 0x7ff3541f34a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff35259c640 1 -- 172.21.15.32:0/1208319727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3540b0cb0 msgr2=0x7ff3541505a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff35259c640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3540b0cb0 0x7ff3541505a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff35259c640 1 -- 172.21.15.32:0/1208319727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff3440094d0 con 0x7ff354070230 2023-12-20T06:02:59.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.491+0000 7ff35259c640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff354070230 0x7ff3541494c0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7ff34401abf0 tx=0x7ff34401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.495+0000 7ff342ffd640 1 -- 172.21.15.32:0/1208319727 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3440a9400 con 0x7ff354070230 2023-12-20T06:02:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.495+0000 7ff358d25640 1 -- 172.21.15.32:0/1208319727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff35414edf0 con 0x7ff354070230 2023-12-20T06:02:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.495+0000 7ff342ffd640 1 -- 172.21.15.32:0/1208319727 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff3440b1820 con 0x7ff354070230 2023-12-20T06:02:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.495+0000 7ff342ffd640 1 -- 172.21.15.32:0/1208319727 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff3440b1bd0 con 0x7ff354070230 2023-12-20T06:02:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.495+0000 7ff358d25640 1 -- 172.21.15.32:0/1208319727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff35414e4a0 con 0x7ff354070230 2023-12-20T06:02:59.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.495+0000 7ff358d25640 1 -- 172.21.15.32:0/1208319727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff35414f030 con 0x7ff354070230 2023-12-20T06:02:59.495 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:02:59.491+0000 7f97d082d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:02:59.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.495+0000 7ff342ffd640 1 -- 172.21.15.32:0/1208319727 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 130) v1 ==== 85489+0+0 (secure 0 0 0) 0x7ff3440b1d30 con 0x7ff354070230 2023-12-20T06:02:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.499+0000 7ff342ffd640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff334065d30 0x7ff3340681f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.499+0000 7ff351d9b640 1 -- 172.21.15.32:0/1208319727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff334065d30 msgr2=0x7ff3340681f0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:02:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.499+0000 7ff351d9b640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff334065d30 0x7ff3340681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:02:59.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.499+0000 7ff342ffd640 1 -- 172.21.15.32:0/1208319727 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(214..214 src has 1..214) v4 ==== 6973+0+0 (secure 0 0 0) 0x7ff3440ef420 con 0x7ff354070230 2023-12-20T06:02:59.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.507+0000 7ff342ffd640 1 -- 172.21.15.32:0/1208319727 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff35414f030 con 0x7ff354070230 2023-12-20T06:02:59.560 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.563+0000 7f4eaf423180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:02:59.626 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.627+0000 7f4eaf423180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:02:59.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.639+0000 7ff342ffd640 1 -- 172.21.15.32:0/1208319727 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 131) v1 ==== 84939+0+0 (secure 0 0 0) 0x7ff3440ebb30 con 0x7ff354070230 2023-12-20T06:02:59.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.639+0000 7ff342ffd640 1 -- 172.21.15.32:0/1208319727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff334065d30 msgr2=0x7ff3340681f0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:02:59.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.639+0000 7ff342ffd640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff334065d30 0x7ff3340681f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.691+0000 7ff358d25640 1 -- 172.21.15.32:0/1208319727 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7ff35414f030 con 0x7ff354070230 2023-12-20T06:02:59.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.691+0000 7ff342ffd640 1 -- 172.21.15.32:0/1208319727 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v215) v1 ==== 93+0+10 (secure 0 0 0) 0x7ff35414f030 con 0x7ff354070230 2023-12-20T06:02:59.691 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:02:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.695+0000 7ff358d25640 1 -- 172.21.15.32:0/1208319727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff354070230 msgr2=0x7ff3541494c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.695+0000 7ff358d25640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff354070230 0x7ff3541494c0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7ff34401abf0 tx=0x7ff34401ad80 comp rx=0 tx=0).stop 2023-12-20T06:02:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.695+0000 7ff358d25640 1 -- 172.21.15.32:0/1208319727 shutdown_connections 2023-12-20T06:02:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.695+0000 7ff358d25640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff334065d30 0x7ff3340681f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.695+0000 7ff358d25640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff35415abb0 0x7ff3541f34a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.695+0000 7ff358d25640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff3540b0cb0 0x7ff3541505a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.695+0000 7ff358d25640 1 --2- 172.21.15.32:0/1208319727 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff354070230 0x7ff3541494c0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.695+0000 7ff358d25640 1 -- 172.21.15.32:0/1208319727 >> 172.21.15.32:0/1208319727 conn(0x7ff35406a510 msgr2=0x7ff3540c47d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.695+0000 7ff358d25640 1 -- 172.21.15.32:0/1208319727 shutdown_connections 2023-12-20T06:02:59.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.695+0000 7ff358d25640 1 -- 172.21.15.32:0/1208319727 wait complete. 2023-12-20T06:02:59.706 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.707+0000 7f4e3ac9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.706 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.707+0000 7f4e3ac9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.706 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.707+0000 7f4e3ac9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.706 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.707+0000 7f4e3ac9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.706 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.707+0000 7f4e3ac9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.707 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.707+0000 7f4e37c94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.707 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.707+0000 7f4e37c94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.707 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.707+0000 7f4e37c94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.707 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.707+0000 7f4e37c94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.707 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.707+0000 7f4e37c94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.718 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:02:59.719 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.719+0000 7f4e37c94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.719 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.719+0000 7f4e37c94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.719 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.719+0000 7f4e37c94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.719 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.719+0000 7f4e37c94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.719 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.719+0000 7f4e37c94640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.720 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.719+0000 7f4e3ac9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.720 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.719+0000 7f4e3ac9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.720 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.719+0000 7f4e3ac9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.720 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.719+0000 7f4e3ac9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.720 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:02:59.719+0000 7f4e3ac9a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:02:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f007c398640 1 Processor -- start 2023-12-20T06:02:59.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f007c398640 1 -- start start 2023-12-20T06:02:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f007c398640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074157480 0x7f00740c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f007c398640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00740cde20 con 0x7f00740b0760 2023-12-20T06:02:59.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f007c398640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f00740c8e30 con 0x7f00740cd4c0 2023-12-20T06:02:59.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f007c398640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007415e480 con 0x7f0074157480 2023-12-20T06:02:59.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079933640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f00740b0760 0x7f007415d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43842/0 (socket says 172.21.15.32:43842) 2023-12-20T06:02:59.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079933640 1 -- 172.21.15.32:0/3083606272 learned_addr learned my addr 172.21.15.32:0/3083606272 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:02:59.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2727040447 0 0) 0x7f00740cde20 con 0x7f00740b0760 2023-12-20T06:02:59.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f007a134640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074157480 0x7f00740c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0060003530 con 0x7f00740b0760 2023-12-20T06:02:59.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 127741435 0 0) 0x7f0060003530 con 0x7f00740b0760 2023-12-20T06:02:59.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074157480 msgr2=0x7f00740c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:59.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074157480 0x7f00740c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 >> v1:172.21.15.39:6789/0 conn(0x7f00740cd4c0 legacy=0x7f007415dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f00740ce070 con 0x7f00740b0760 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f0064001680 con 0x7f00740b0760 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 >> v1:172.21.15.32:6789/0 conn(0x7f00740b0760 legacy=0x7f007415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074157480 0x7f00600039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060003ef0 0x7f00600047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0060004d10 0x7f0060005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f007415e480 con 0x7f0060004d10 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f006000baa0 con 0x7f0060003ef0 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f006000bc60 con 0x7f0074157480 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f007a134640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074157480 0x7f00600039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f0079933640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060003ef0 0x7f00600047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.867+0000 7f007a935640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0060004d10 0x7f0060005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007a134640 1 -- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060003ef0 msgr2=0x7f00600047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:59.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007a134640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060003ef0 0x7f00600047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007a134640 1 -- 172.21.15.32:0/3083606272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0060004d10 msgr2=0x7f0060005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:02:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007a134640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0060004d10 0x7f0060005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007a134640 1 -- 172.21.15.32:0/3083606272 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0060003530 con 0x7f0074157480 2023-12-20T06:02:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f0079933640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0060003ef0 0x7f00600047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:59.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007a935640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0060004d10 0x7f0060005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:02:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007a134640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074157480 0x7f00600039b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0070009ae0 tx=0x7f007001ca60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bd4b74d3549aceff server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00700a55d0 con 0x7f0074157480 2023-12-20T06:02:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f00700a5730 con 0x7f0074157480 2023-12-20T06:02:59.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f0079132640 1 -- 172.21.15.32:0/3083606272 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00700a5ae0 con 0x7f0074157480 2023-12-20T06:02:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007c398640 1 -- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074157480 msgr2=0x7f00600039b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007c398640 1 --2- 172.21.15.32:0/3083606272 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074157480 0x7f00600039b0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f0070009ae0 tx=0x7f007001ca60 comp rx=0 tx=0).stop 2023-12-20T06:02:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007a935640 1 -- 172.21.15.32:0/3083606272 reap_dead start 2023-12-20T06:02:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007c398640 1 -- 172.21.15.32:0/3083606272 shutdown_connections 2023-12-20T06:02:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.871+0000 7f007c398640 1 -- 172.21.15.32:0/3083606272 >> 172.21.15.32:0/3083606272 conn(0x7f007406a510 msgr2=0x7f00740ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:02:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007c398640 1 -- 172.21.15.32:0/3083606272 shutdown_connections 2023-12-20T06:02:59.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007c398640 1 -- 172.21.15.32:0/3083606272 wait complete. 2023-12-20T06:02:59.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007c398640 1 Processor -- start 2023-12-20T06:02:59.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007c398640 1 -- start start 2023-12-20T06:02:59.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007c398640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0074157480 0x7f00740b2be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007c398640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00740b3120 0x7f007413fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007c398640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074140230 0x7f007415b310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007c398640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f00741406e0 con 0x7f0074157480 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007c398640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0074167870 con 0x7f00740b3120 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007c398640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f007415c240 con 0x7f0074140230 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007a134640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0074157480 0x7f00740b2be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007a134640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0074157480 0x7f00740b2be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33684/0 (socket says 172.21.15.32:33684) 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007a134640 1 -- 172.21.15.32:0/3553578597 learned_addr learned my addr 172.21.15.32:0/3553578597 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007a935640 1 --2- 172.21.15.32:0/3553578597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074140230 0x7f007415b310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f0079933640 1 --2- 172.21.15.32:0/3553578597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00740b3120 0x7f007413fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007a134640 1 -- 172.21.15.32:0/3553578597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074140230 msgr2=0x7f007415b310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007a134640 1 --2- 172.21.15.32:0/3553578597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074140230 0x7f007415b310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007a134640 1 -- 172.21.15.32:0/3553578597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00740b3120 msgr2=0x7f007413fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007a134640 1 --2- 172.21.15.32:0/3553578597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00740b3120 0x7f007413fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007a134640 1 -- 172.21.15.32:0/3553578597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f007415c520 con 0x7f0074157480 2023-12-20T06:02:59.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f0079933640 1 --2- 172.21.15.32:0/3553578597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00740b3120 0x7f007413fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:02:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007a134640 1 --2- 172.21.15.32:0/3553578597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0074157480 0x7f00740b2be0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f0070009ae0 tx=0x7f0060005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:02:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f006affd640 1 -- 172.21.15.32:0/3553578597 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00700a5ea0 con 0x7f0074157480 2023-12-20T06:02:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f006affd640 1 -- 172.21.15.32:0/3553578597 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0070004030 con 0x7f0074157480 2023-12-20T06:02:59.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f006affd640 1 -- 172.21.15.32:0/3553578597 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f00700043a0 con 0x7f0074157480 2023-12-20T06:02:59.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.875+0000 7f007c398640 1 -- 172.21.15.32:0/3553578597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0074167b10 con 0x7f0074157480 2023-12-20T06:02:59.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.879+0000 7f007c398640 1 -- 172.21.15.32:0/3553578597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f007415b850 con 0x7f0074157480 2023-12-20T06:02:59.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.879+0000 7f007c398640 1 -- 172.21.15.32:0/3553578597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0074167ca0 con 0x7f0074157480 2023-12-20T06:02:59.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.879+0000 7f006affd640 1 -- 172.21.15.32:0/3553578597 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 131) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f00700b4040 con 0x7f0074157480 2023-12-20T06:02:59.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.879+0000 7f006affd640 1 -- 172.21.15.32:0/3553578597 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f007011d5b0 con 0x7f0074157480 2023-12-20T06:02:59.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:02:59.883+0000 7f006affd640 1 -- 172.21.15.32:0/3553578597 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f00700eaf20 con 0x7f0074157480 2023-12-20T06:03:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.039+0000 7f007c398640 1 -- 172.21.15.32:0/3553578597 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f0074140640 con 0x7f0074157480 2023-12-20T06:03:00.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.043+0000 7f006affd640 1 -- 172.21.15.32:0/3553578597 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v215) v1 ==== 104+0+11 (secure 0 0 0) 0x7f00700ee540 con 0x7f0074157480 2023-12-20T06:03:00.041 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:03:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.043+0000 7f007c398640 1 -- 172.21.15.32:0/3553578597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0074157480 msgr2=0x7f00740b2be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.043+0000 7f007c398640 1 --2- 172.21.15.32:0/3553578597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0074157480 0x7f00740b2be0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f0070009ae0 tx=0x7f0060005800 comp rx=0 tx=0).stop 2023-12-20T06:03:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.043+0000 7f007c398640 1 -- 172.21.15.32:0/3553578597 shutdown_connections 2023-12-20T06:03:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.043+0000 7f007c398640 1 --2- 172.21.15.32:0/3553578597 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0074140230 0x7f007415b310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.043+0000 7f007c398640 1 --2- 172.21.15.32:0/3553578597 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f00740b3120 0x7f007413fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.043+0000 7f007c398640 1 --2- 172.21.15.32:0/3553578597 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0074157480 0x7f00740b2be0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.043+0000 7f007c398640 1 -- 172.21.15.32:0/3553578597 >> 172.21.15.32:0/3553578597 conn(0x7f007406a510 msgr2=0x7f00740ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.043+0000 7f007c398640 1 -- 172.21.15.32:0/3553578597 shutdown_connections 2023-12-20T06:03:00.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.043+0000 7f007c398640 1 -- 172.21.15.32:0/3553578597 wait complete. 2023-12-20T06:03:00.061 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:03:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.207+0000 7fabdfb25640 1 Processor -- start 2023-12-20T06:03:00.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.207+0000 7fabdfb25640 1 -- start start 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabdfb25640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabd80b0bc0 0x7fabd80ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabdfb25640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd80cde20 con 0x7fabd8157480 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabdfb25640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabd80c8e30 con 0x7fabd80b0760 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabdfb25640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fabd80c9f50 con 0x7fabd80b0bc0 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabde0c2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabd80b0bc0 0x7fabd80ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabde0c2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fabd80b0bc0 0x7fabd80ccbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33692/0 (socket says 172.21.15.32:33692) 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabde0c2640 1 -- 172.21.15.32:0/1190122071 learned_addr learned my addr 172.21.15.32:0/1190122071 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabdd8c1640 1 --1- 172.21.15.32:0/1190122071 >> v1:172.21.15.32:6789/0 conn(0x7fabd8157480 0x7fabd80c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43858/0 (socket says 172.21.15.32:43858) 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabde0c2640 1 -- 172.21.15.32:0/1190122071 >> v1:172.21.15.39:6789/0 conn(0x7fabd80b0760 legacy=0x7fabd80cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabde0c2640 1 -- 172.21.15.32:0/1190122071 >> v1:172.21.15.32:6789/0 conn(0x7fabd8157480 legacy=0x7fabd80c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabde0c2640 1 -- 172.21.15.32:0/1190122071 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd80ce070 con 0x7fabd80b0bc0 2023-12-20T06:03:00.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.211+0000 7fabdc8bf640 1 -- 172.21.15.32:0/1190122071 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1420817526 0 0) 0x7fabd80cde20 con 0x7fabd8157480 2023-12-20T06:03:00.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabde0c2640 1 --2- 172.21.15.32:0/1190122071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabd80b0bc0 0x7fabd80ccbc0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009e50 tx=0x7fabd001c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df23f796681b0a61 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:00.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdc8bf640 1 -- 172.21.15.32:0/1190122071 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd00a5750 con 0x7fabd80b0bc0 2023-12-20T06:03:00.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdc8bf640 1 -- 172.21.15.32:0/1190122071 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd00a58b0 con 0x7fabd80b0bc0 2023-12-20T06:03:00.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdc8bf640 1 -- 172.21.15.32:0/1190122071 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd00a5c40 con 0x7fabd80b0bc0 2023-12-20T06:03:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdfb25640 1 -- 172.21.15.32:0/1190122071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabd80b0bc0 msgr2=0x7fabd80ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdfb25640 1 --2- 172.21.15.32:0/1190122071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabd80b0bc0 0x7fabd80ccbc0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7fabd0009e50 tx=0x7fabd001c8e0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdfb25640 1 -- 172.21.15.32:0/1190122071 shutdown_connections 2023-12-20T06:03:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdfb25640 1 --2- 172.21.15.32:0/1190122071 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabd80b0bc0 0x7fabd80ccbc0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdfb25640 1 -- 172.21.15.32:0/1190122071 >> 172.21.15.32:0/1190122071 conn(0x7fabd806a510 msgr2=0x7fabd80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdfb25640 1 -- 172.21.15.32:0/1190122071 shutdown_connections 2023-12-20T06:03:00.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdfb25640 1 -- 172.21.15.32:0/1190122071 wait complete. 2023-12-20T06:03:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdfb25640 1 Processor -- start 2023-12-20T06:03:00.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.215+0000 7fabdfb25640 1 -- start start 2023-12-20T06:03:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdfb25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabd80b0760 0x7fabd815d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdfb25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabd80b0bc0 0x7fabd80c6af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdfb25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabd8157480 0x7fabd8164af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdfb25640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80c7440 con 0x7fabd80b0bc0 2023-12-20T06:03:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdfb25640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80c7720 con 0x7fabd8157480 2023-12-20T06:03:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdfb25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabd80c7a00 con 0x7fabd80b0760 2023-12-20T06:03:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdd0c0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabd80b0bc0 0x7fabd80c6af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdd0c0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabd80b0bc0 0x7fabd80c6af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33702/0 (socket says 172.21.15.32:33702) 2023-12-20T06:03:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdd8c1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabd80b0760 0x7fabd815d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdd0c0640 1 -- 172.21.15.32:0/282068975 learned_addr learned my addr 172.21.15.32:0/282068975 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:00.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabde0c2640 1 --2- 172.21.15.32:0/282068975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabd8157480 0x7fabd8164af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabde0c2640 1 -- 172.21.15.32:0/282068975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabd80b0760 msgr2=0x7fabd815d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabde0c2640 1 --2- 172.21.15.32:0/282068975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabd80b0760 0x7fabd815d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabde0c2640 1 -- 172.21.15.32:0/282068975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabd80b0bc0 msgr2=0x7fabd80c6af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabde0c2640 1 --2- 172.21.15.32:0/282068975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabd80b0bc0 0x7fabd80c6af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabde0c2640 1 -- 172.21.15.32:0/282068975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabd0009b30 con 0x7fabd8157480 2023-12-20T06:03:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdd0c0640 1 --2- 172.21.15.32:0/282068975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabd80b0bc0 0x7fabd80c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabde0c2640 1 --2- 172.21.15.32:0/282068975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabd8157480 0x7fabd8164af0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fabd00ab000 tx=0x7fabd0019680 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabce7fc640 1 -- 172.21.15.32:0/282068975 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd00b8470 con 0x7fabd8157480 2023-12-20T06:03:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabce7fc640 1 -- 172.21.15.32:0/282068975 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabd00b85d0 con 0x7fabd8157480 2023-12-20T06:03:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabce7fc640 1 -- 172.21.15.32:0/282068975 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabd00b8980 con 0x7fabd8157480 2023-12-20T06:03:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdfb25640 1 -- 172.21.15.32:0/282068975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabd80c7ce0 con 0x7fabd8157480 2023-12-20T06:03:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdfb25640 1 -- 172.21.15.32:0/282068975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabd815deb0 con 0x7fabd8157480 2023-12-20T06:03:00.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabdfb25640 1 -- 172.21.15.32:0/282068975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabd815adf0 con 0x7fabd8157480 2023-12-20T06:03:00.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.219+0000 7fabce7fc640 1 -- 172.21.15.32:0/282068975 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 131) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fabd00b8ae0 con 0x7fabd8157480 2023-12-20T06:03:00.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.223+0000 7fabce7fc640 1 -- 172.21.15.32:0/282068975 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fabd011e720 con 0x7fabd8157480 2023-12-20T06:03:00.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.227+0000 7fabce7fc640 1 -- 172.21.15.32:0/282068975 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fabd815adf0 con 0x7fabd8157480 2023-12-20T06:03:00.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.387+0000 7fabdfb25640 1 -- 172.21.15.32:0/282068975 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7fabd815adf0 con 0x7fabd8157480 2023-12-20T06:03:00.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.387+0000 7fabce7fc640 1 -- 172.21.15.32:0/282068975 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v215) v1 ==== 100+0+11 (secure 0 0 0) 0x7fabd815adf0 con 0x7fabd8157480 2023-12-20T06:03:00.387 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:03:00.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.391+0000 7fabdfb25640 1 -- 172.21.15.32:0/282068975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabd8157480 msgr2=0x7fabd8164af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.391+0000 7fabdfb25640 1 --2- 172.21.15.32:0/282068975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabd8157480 0x7fabd8164af0 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7fabd00ab000 tx=0x7fabd0019680 comp rx=0 tx=0).stop 2023-12-20T06:03:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.391+0000 7fabdfb25640 1 -- 172.21.15.32:0/282068975 shutdown_connections 2023-12-20T06:03:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.391+0000 7fabdfb25640 1 --2- 172.21.15.32:0/282068975 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabd8157480 0x7fabd8164af0 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.391+0000 7fabdfb25640 1 --2- 172.21.15.32:0/282068975 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabd80b0bc0 0x7fabd80c6af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.391+0000 7fabdfb25640 1 --2- 172.21.15.32:0/282068975 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabd80b0760 0x7fabd815d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.391+0000 7fabdfb25640 1 -- 172.21.15.32:0/282068975 >> 172.21.15.32:0/282068975 conn(0x7fabd806a510 msgr2=0x7fabd8164050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.391+0000 7fabdfb25640 1 -- 172.21.15.32:0/282068975 shutdown_connections 2023-12-20T06:03:00.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.391+0000 7fabdfb25640 1 -- 172.21.15.32:0/282068975 wait complete. 2023-12-20T06:03:00.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-20T06:03:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.555+0000 7fdd56212640 1 Processor -- start 2023-12-20T06:03:00.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.555+0000 7fdd56212640 1 -- start start 2023-12-20T06:03:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd56212640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdd500b0cb0 0x7fdd500caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd56212640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdd50070230 0x7fdd50070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd56212640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdd50070bc0 con 0x7fdd5015abb0 2023-12-20T06:03:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd56212640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5006e160 con 0x7fdd50070230 2023-12-20T06:03:00.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd56212640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdd5015b6a0 con 0x7fdd500b0cb0 2023-12-20T06:03:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd4f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdd5015abb0 0x7fdd5015bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43860/0 (socket says 172.21.15.32:43860) 2023-12-20T06:03:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd4f7fe640 1 -- 172.21.15.32:0/209879434 learned_addr learned my addr 172.21.15.32:0/209879434 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd4ffff640 1 --2- 172.21.15.32:0/209879434 >> v2:172.21.15.32:3300/0 conn(0x7fdd50070230 0x7fdd50070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd4effd640 1 --2- 172.21.15.32:0/209879434 >> v2:172.21.15.39:3300/0 conn(0x7fdd500b0cb0 0x7fdd500caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd4e7fc640 1 -- 172.21.15.32:0/209879434 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1141363673 0 0) 0x7fdd40001120 con 0x7fdd5015abb0 2023-12-20T06:03:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd4ffff640 1 -- 172.21.15.32:0/209879434 >> v2:172.21.15.39:3300/0 conn(0x7fdd500b0cb0 msgr2=0x7fdd500caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd4ffff640 1 --2- 172.21.15.32:0/209879434 >> v2:172.21.15.39:3300/0 conn(0x7fdd500b0cb0 0x7fdd500caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd4ffff640 1 -- 172.21.15.32:0/209879434 >> v1:172.21.15.32:6789/0 conn(0x7fdd5015abb0 legacy=0x7fdd5015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd4ffff640 1 -- 172.21.15.32:0/209879434 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd5006d8c0 con 0x7fdd50070230 2023-12-20T06:03:00.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.559+0000 7fdd4ffff640 1 --2- 172.21.15.32:0/209879434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd50070230 0x7fdd50070680 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fdd500b4470 tx=0x7fdd4801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76f5f4b93f556c64 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:00.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd4e7fc640 1 -- 172.21.15.32:0/209879434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480a59d0 con 0x7fdd50070230 2023-12-20T06:03:00.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd4e7fc640 1 -- 172.21.15.32:0/209879434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd480a5b30 con 0x7fdd50070230 2023-12-20T06:03:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd4e7fc640 1 -- 172.21.15.32:0/209879434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd480ad440 con 0x7fdd50070230 2023-12-20T06:03:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 -- 172.21.15.32:0/209879434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd50070230 msgr2=0x7fdd50070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 --2- 172.21.15.32:0/209879434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd50070230 0x7fdd50070680 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fdd500b4470 tx=0x7fdd4801c910 comp rx=0 tx=0).stop 2023-12-20T06:03:00.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 -- 172.21.15.32:0/209879434 shutdown_connections 2023-12-20T06:03:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 --2- 172.21.15.32:0/209879434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd50070230 0x7fdd50070680 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 --2- 172.21.15.32:0/209879434 >> v2:172.21.15.39:3300/0 conn(0x7fdd500b0cb0 0x7fdd500caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 -- 172.21.15.32:0/209879434 >> 172.21.15.32:0/209879434 conn(0x7fdd5006a510 msgr2=0x7fdd500ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 -- 172.21.15.32:0/209879434 shutdown_connections 2023-12-20T06:03:00.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 -- 172.21.15.32:0/209879434 wait complete. 2023-12-20T06:03:00.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 Processor -- start 2023-12-20T06:03:00.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 -- start start 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd50070230 0x7fdd5013f040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd500b0cb0 0x7fdd50146410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdd5015abb0 0x7fdd50141be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5013f580 con 0x7fdd500b0cb0 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdd50147620 con 0x7fdd50070230 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.563+0000 7fdd56212640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdd5014eff0 con 0x7fdd5015abb0 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd4effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd500b0cb0 0x7fdd50146410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd4effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd500b0cb0 0x7fdd50146410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33720/0 (socket says 172.21.15.32:33720) 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd4effd640 1 -- 172.21.15.32:0/1656153793 learned_addr learned my addr 172.21.15.32:0/1656153793 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd4f7fe640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd50070230 0x7fdd5013f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd4ffff640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdd5015abb0 0x7fdd50141be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd4effd640 1 -- 172.21.15.32:0/1656153793 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdd5015abb0 msgr2=0x7fdd50141be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd4effd640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdd5015abb0 0x7fdd50141be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd4effd640 1 -- 172.21.15.32:0/1656153793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd50070230 msgr2=0x7fdd5013f040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd4effd640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd50070230 0x7fdd5013f040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd4effd640 1 -- 172.21.15.32:0/1656153793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdd48009a70 con 0x7fdd500b0cb0 2023-12-20T06:03:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd4effd640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd500b0cb0 0x7fdd50146410 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400e930 tx=0x7fdd4400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:00.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd37fff640 1 -- 172.21.15.32:0/1656153793 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd44009690 con 0x7fdd500b0cb0 2023-12-20T06:03:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd37fff640 1 -- 172.21.15.32:0/1656153793 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdd4400ca00 con 0x7fdd500b0cb0 2023-12-20T06:03:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd37fff640 1 -- 172.21.15.32:0/1656153793 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdd4400cd90 con 0x7fdd500b0cb0 2023-12-20T06:03:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd56212640 1 -- 172.21.15.32:0/1656153793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdd50146d80 con 0x7fdd500b0cb0 2023-12-20T06:03:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd56212640 1 -- 172.21.15.32:0/1656153793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdd50147340 con 0x7fdd500b0cb0 2023-12-20T06:03:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd56212640 1 -- 172.21.15.32:0/1656153793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdd5014ee50 con 0x7fdd500b0cb0 2023-12-20T06:03:00.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.567+0000 7fdd37fff640 1 -- 172.21.15.32:0/1656153793 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 131) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fdd44013470 con 0x7fdd500b0cb0 2023-12-20T06:03:00.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.571+0000 7fdd37fff640 1 -- 172.21.15.32:0/1656153793 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fdd440835a0 con 0x7fdd500b0cb0 2023-12-20T06:03:00.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.575+0000 7fdd37fff640 1 -- 172.21.15.32:0/1656153793 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdd5014ee50 con 0x7fdd500b0cb0 2023-12-20T06:03:00.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.643+0000 7fdd37fff640 1 -- 172.21.15.32:0/1656153793 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 132) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fdd44050760 con 0x7fdd500b0cb0 2023-12-20T06:03:00.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.643+0000 7fdd37fff640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd3c01c9b0 0x7fdd3c06f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.643+0000 7fdd4f7fe640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd3c01c9b0 0x7fdd3c06f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.647+0000 7fdd4f7fe640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd3c01c9b0 0x7fdd3c06f8f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdd40000f70 tx=0x7fdd40007c10 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.739+0000 7fdd56212640 1 -- 172.21.15.32:0/1656153793 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fdd5014ee50 con 0x7fdd500b0cb0 2023-12-20T06:03:00.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.739+0000 7fdd37fff640 1 -- 172.21.15.32:0/1656153793 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v215) v1 ==== 107+0+11 (secure 0 0 0) 0x7fdd5014ee50 con 0x7fdd500b0cb0 2023-12-20T06:03:00.738 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 -- 172.21.15.32:0/1656153793 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd3c01c9b0 msgr2=0x7fdd3c06f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd3c01c9b0 0x7fdd3c06f8f0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7fdd40000f70 tx=0x7fdd40007c10 comp rx=0 tx=0).stop 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 -- 172.21.15.32:0/1656153793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd500b0cb0 msgr2=0x7fdd50146410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd500b0cb0 0x7fdd50146410 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7fdd4400e930 tx=0x7fdd4400edf0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 -- 172.21.15.32:0/1656153793 shutdown_connections 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd3c01c9b0 0x7fdd3c06f8f0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdd5015abb0 0x7fdd50141be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdd500b0cb0 0x7fdd50146410 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 --2- 172.21.15.32:0/1656153793 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdd50070230 0x7fdd5013f040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 -- 172.21.15.32:0/1656153793 >> 172.21.15.32:0/1656153793 conn(0x7fdd5006a510 msgr2=0x7fdd500c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 -- 172.21.15.32:0/1656153793 shutdown_connections 2023-12-20T06:03:00.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.743+0000 7fdd56212640 1 -- 172.21.15.32:0/1656153793 wait complete. 2023-12-20T06:03:00.760 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-20T06:03:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.907+0000 7f95b38c4640 1 Processor -- start 2023-12-20T06:03:00.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.907+0000 7f95b38c4640 1 -- start start 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b38c4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f95ac0b0760 0x7f95ac0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b38c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95ac0cc4b0 0x7f95ac0cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b38c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f95ac0b1100 con 0x7f95ac157480 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b38c4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f95ac0c9f50 con 0x7f95ac0b0760 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b38c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0cde20 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b1660640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f95ac157480 0x7f95ac0c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43864/0 (socket says 172.21.15.32:43864) 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b1660640 1 -- 172.21.15.32:0/3672219016 learned_addr learned my addr 172.21.15.32:0/3672219016 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b1e61640 1 --2- 172.21.15.32:0/3672219016 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95ac0cc4b0 0x7f95ac0cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b0e5f640 1 --2- 172.21.15.32:0/3672219016 >> v2:172.21.15.39:3300/0 conn(0x7f95ac0b0760 0x7f95ac0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95a3fff640 1 -- 172.21.15.32:0/3672219016 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3026566015 0 0) 0x7f95ac0b1100 con 0x7f95ac157480 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95a3fff640 1 -- 172.21.15.32:0/3672219016 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9598003530 con 0x7f95ac157480 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b1e61640 1 -- 172.21.15.32:0/3672219016 >> v2:172.21.15.39:3300/0 conn(0x7f95ac0b0760 msgr2=0x7f95ac0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b1e61640 1 --2- 172.21.15.32:0/3672219016 >> v2:172.21.15.39:3300/0 conn(0x7f95ac0b0760 0x7f95ac0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b1e61640 1 -- 172.21.15.32:0/3672219016 >> v1:172.21.15.32:6789/0 conn(0x7f95ac157480 legacy=0x7f95ac0c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b1e61640 1 -- 172.21.15.32:0/3672219016 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f95ac0c8e30 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.911+0000 7f95b0e5f640 1 --2- 172.21.15.32:0/3672219016 >> v2:172.21.15.39:3300/0 conn(0x7f95ac0b0760 0x7f95ac0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:00.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b1e61640 1 --2- 172.21.15.32:0/3672219016 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95ac0cc4b0 0x7f95ac0cc920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009d90 tx=0x7f95a401c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=cf48dd69b460621b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95a3fff640 1 -- 172.21.15.32:0/3672219016 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40a5950 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95a3fff640 1 -- 172.21.15.32:0/3672219016 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f95a40a5ab0 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95a3fff640 1 -- 172.21.15.32:0/3672219016 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f95a40a5e40 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 -- 172.21.15.32:0/3672219016 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95ac0cc4b0 msgr2=0x7f95ac0cc920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 --2- 172.21.15.32:0/3672219016 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95ac0cc4b0 0x7f95ac0cc920 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f95a4009d90 tx=0x7f95a401c890 comp rx=0 tx=0).stop 2023-12-20T06:03:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 -- 172.21.15.32:0/3672219016 shutdown_connections 2023-12-20T06:03:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 --2- 172.21.15.32:0/3672219016 >> v2:172.21.15.39:3300/0 conn(0x7f95ac0b0760 0x7f95ac0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 --2- 172.21.15.32:0/3672219016 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95ac0cc4b0 0x7f95ac0cc920 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 -- 172.21.15.32:0/3672219016 >> 172.21.15.32:0/3672219016 conn(0x7f95ac06a510 msgr2=0x7f95ac0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 -- 172.21.15.32:0/3672219016 shutdown_connections 2023-12-20T06:03:00.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 -- 172.21.15.32:0/3672219016 wait complete. 2023-12-20T06:03:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 Processor -- start 2023-12-20T06:03:00.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 -- start start 2023-12-20T06:03:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95ac0b0760 0x7f95ac145680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.915+0000 7f95b38c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0cc4b0 0x7f95ac0c7c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b38c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95ac157480 0x7f95ac0c8190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b38c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac14a9a0 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b38c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac145bc0 con 0x7f95ac0b0760 2023-12-20T06:03:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b38c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f95ac0b1930 con 0x7f95ac157480 2023-12-20T06:03:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b0e5f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0cc4b0 0x7f95ac0c7c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.917 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b0e5f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0cc4b0 0x7f95ac0c7c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33734/0 (socket says 172.21.15.32:33734) 2023-12-20T06:03:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b0e5f640 1 -- 172.21.15.32:0/3100428098 learned_addr learned my addr 172.21.15.32:0/3100428098 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b1660640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95ac0b0760 0x7f95ac145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b1e61640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95ac157480 0x7f95ac0c8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b0e5f640 1 -- 172.21.15.32:0/3100428098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95ac157480 msgr2=0x7f95ac0c8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b0e5f640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95ac157480 0x7f95ac0c8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b0e5f640 1 -- 172.21.15.32:0/3100428098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95ac0b0760 msgr2=0x7f95ac145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b0e5f640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95ac0b0760 0x7f95ac145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b0e5f640 1 -- 172.21.15.32:0/3100428098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f959c009420 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b0e5f640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0cc4b0 0x7f95ac0c7c50 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f959c00e9b0 tx=0x7f959c00ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95a1ffb640 1 -- 172.21.15.32:0/3100428098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c00ca00 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95a1ffb640 1 -- 172.21.15.32:0/3100428098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f959c00cb60 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b38c4640 1 -- 172.21.15.32:0/3100428098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f95a4009a70 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95a1ffb640 1 -- 172.21.15.32:0/3100428098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f959c013470 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b38c4640 1 -- 172.21.15.32:0/3100428098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f95ac0c7460 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95b38c4640 1 -- 172.21.15.32:0/3100428098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f95ac04e000 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.919+0000 7f95a1ffb640 1 -- 172.21.15.32:0/3100428098 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 132) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f959c0135d0 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.923+0000 7f95a1ffb640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95900658b0 0x7f9590067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.923+0000 7f95a1ffb640 1 -- 172.21.15.32:0/3100428098 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f959c0836c0 con 0x7f95ac0cc4b0 2023-12-20T06:03:00.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.923+0000 7f95b1660640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95900658b0 0x7f9590067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:00.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.923+0000 7f95b1660640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95900658b0 0x7f9590067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005890 tx=0x7f95a8009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:00.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:00.927+0000 7f95a1ffb640 1 -- 172.21.15.32:0/3100428098 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f959c050d50 con 0x7f95ac0cc4b0 2023-12-20T06:03:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95b38c4640 1 -- 172.21.15.32:0/3100428098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f95ac0c9050 con 0x7f95ac0cc4b0 2023-12-20T06:03:01.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95a1ffb640 1 -- 172.21.15.32:0/3100428098 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v215) v1 ==== 103+0+11 (secure 0 0 0) 0x7f959c054370 con 0x7f95ac0cc4b0 2023-12-20T06:03:01.082 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:03:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95b38c4640 1 -- 172.21.15.32:0/3100428098 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95900658b0 msgr2=0x7f9590067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95b38c4640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95900658b0 0x7f9590067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f95a8005890 tx=0x7f95a8009040 comp rx=0 tx=0).stop 2023-12-20T06:03:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95b38c4640 1 -- 172.21.15.32:0/3100428098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0cc4b0 msgr2=0x7f95ac0c7c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95b38c4640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0cc4b0 0x7f95ac0c7c50 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7f959c00e9b0 tx=0x7f959c00ee70 comp rx=0 tx=0).stop 2023-12-20T06:03:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95b38c4640 1 -- 172.21.15.32:0/3100428098 shutdown_connections 2023-12-20T06:03:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95b38c4640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f95900658b0 0x7f9590067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95b38c4640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f95ac157480 0x7f95ac0c8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95b38c4640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f95ac0cc4b0 0x7f95ac0c7c50 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95b38c4640 1 --2- 172.21.15.32:0/3100428098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f95ac0b0760 0x7f95ac145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.083+0000 7f95b38c4640 1 -- 172.21.15.32:0/3100428098 >> 172.21.15.32:0/3100428098 conn(0x7f95ac06a510 msgr2=0x7f95ac06d4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:01.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.087+0000 7f95b38c4640 1 -- 172.21.15.32:0/3100428098 shutdown_connections 2023-12-20T06:03:01.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.087+0000 7f95b38c4640 1 -- 172.21.15.32:0/3100428098 wait complete. 2023-12-20T06:03:01.104 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:01.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.247+0000 7fb667d09640 1 Processor -- start 2023-12-20T06:03:01.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.247+0000 7fb667d09640 1 -- start start 2023-12-20T06:03:01.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb667d09640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb6601599e0 0x7fb6600aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb667d09640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6600c98c0 con 0x7fb6600c90e0 2023-12-20T06:03:01.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb667d09640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6600c9ba0 con 0x7fb6600b0d00 2023-12-20T06:03:01.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb667d09640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6600c9e80 con 0x7fb6601599e0 2023-12-20T06:03:01.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb6652a4640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb6600c90e0 0x7fb6600b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43868/0 (socket says 172.21.15.32:43868) 2023-12-20T06:03:01.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb6652a4640 1 -- 172.21.15.32:0/817480386 learned_addr learned my addr 172.21.15.32:0/817480386 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:01.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb665aa5640 1 --2- 172.21.15.32:0/817480386 >> v2:172.21.15.39:3300/0 conn(0x7fb6601599e0 0x7fb6600aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb664aa3640 1 -- 172.21.15.32:0/817480386 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4098687936 0 0) 0x7fb6600c98c0 con 0x7fb6600c90e0 2023-12-20T06:03:01.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb664aa3640 1 -- 172.21.15.32:0/817480386 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb64c003530 con 0x7fb6600c90e0 2023-12-20T06:03:01.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb665aa5640 1 -- 172.21.15.32:0/817480386 >> v1:172.21.15.39:6789/0 conn(0x7fb6600b0d00 legacy=0x7fb6600701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb664aa3640 1 -- 172.21.15.32:0/817480386 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1969843521 0 0) 0x7fb6600c9ba0 con 0x7fb6600b0d00 2023-12-20T06:03:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb665aa5640 1 -- 172.21.15.32:0/817480386 >> v1:172.21.15.32:6789/0 conn(0x7fb6600c90e0 legacy=0x7fb6600b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.251+0000 7fb665aa5640 1 -- 172.21.15.32:0/817480386 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6600ca160 con 0x7fb6601599e0 2023-12-20T06:03:01.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb665aa5640 1 --2- 172.21.15.32:0/817480386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6601599e0 0x7fb6600aece0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0097a0 tx=0x7fb65c01c430 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4f11fee810a5bec4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb664aa3640 1 -- 172.21.15.32:0/817480386 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c0ab650 con 0x7fb6601599e0 2023-12-20T06:03:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb664aa3640 1 -- 172.21.15.32:0/817480386 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65c0ab7b0 con 0x7fb6601599e0 2023-12-20T06:03:01.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb664aa3640 1 -- 172.21.15.32:0/817480386 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb65c0abb40 con 0x7fb6601599e0 2023-12-20T06:03:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb667d09640 1 -- 172.21.15.32:0/817480386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6601599e0 msgr2=0x7fb6600aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb667d09640 1 --2- 172.21.15.32:0/817480386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6601599e0 0x7fb6600aece0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0097a0 tx=0x7fb65c01c430 comp rx=0 tx=0).stop 2023-12-20T06:03:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb667d09640 1 -- 172.21.15.32:0/817480386 shutdown_connections 2023-12-20T06:03:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb667d09640 1 --2- 172.21.15.32:0/817480386 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6601599e0 0x7fb6600aece0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb667d09640 1 -- 172.21.15.32:0/817480386 >> 172.21.15.32:0/817480386 conn(0x7fb66006a320 msgr2=0x7fb6600ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb667d09640 1 -- 172.21.15.32:0/817480386 shutdown_connections 2023-12-20T06:03:01.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb667d09640 1 -- 172.21.15.32:0/817480386 wait complete. 2023-12-20T06:03:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb667d09640 1 Processor -- start 2023-12-20T06:03:01.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.255+0000 7fb667d09640 1 -- start start 2023-12-20T06:03:01.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb667d09640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6600b0d00 0x7fb6600b1f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb667d09640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6600c90e0 0x7fb6600c41d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb667d09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6601599e0 0x7fb66014a1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb667d09640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600b1ab0 con 0x7fb6601599e0 2023-12-20T06:03:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb667d09640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600b11c0 con 0x7fb6600b0d00 2023-12-20T06:03:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb667d09640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6600b37a0 con 0x7fb6600c90e0 2023-12-20T06:03:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6662a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6601599e0 0x7fb66014a1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6662a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6601599e0 0x7fb66014a1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33738/0 (socket says 172.21.15.32:33738) 2023-12-20T06:03:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6662a6640 1 -- 172.21.15.32:0/4022614018 learned_addr learned my addr 172.21.15.32:0/4022614018 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:01.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb665aa5640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6600b0d00 0x7fb6600b1f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6652a4640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6600c90e0 0x7fb6600c41d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6662a6640 1 -- 172.21.15.32:0/4022614018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6600c90e0 msgr2=0x7fb6600c41d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6662a6640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6600c90e0 0x7fb6600c41d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6662a6640 1 -- 172.21.15.32:0/4022614018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6600b0d00 msgr2=0x7fb6600b1f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6662a6640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6600b0d00 0x7fb6600b1f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6662a6640 1 -- 172.21.15.32:0/4022614018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb65c0093b0 con 0x7fb6601599e0 2023-12-20T06:03:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6662a6640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6601599e0 0x7fb66014a1f0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fb658002240 tx=0x7fb65800cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6567fc640 1 -- 172.21.15.32:0/4022614018 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658003db0 con 0x7fb6601599e0 2023-12-20T06:03:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6567fc640 1 -- 172.21.15.32:0/4022614018 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb65800ce30 con 0x7fb6601599e0 2023-12-20T06:03:01.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb6567fc640 1 -- 172.21.15.32:0/4022614018 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb658005270 con 0x7fb6601599e0 2023-12-20T06:03:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb667d09640 1 -- 172.21.15.32:0/4022614018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6600b1460 con 0x7fb6601599e0 2023-12-20T06:03:01.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb667d09640 1 -- 172.21.15.32:0/4022614018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6600c4710 con 0x7fb6601599e0 2023-12-20T06:03:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.259+0000 7fb667d09640 1 -- 172.21.15.32:0/4022614018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb66004e000 con 0x7fb6601599e0 2023-12-20T06:03:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.263+0000 7fb6567fc640 1 -- 172.21.15.32:0/4022614018 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 132) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fb6580053d0 con 0x7fb6601599e0 2023-12-20T06:03:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.263+0000 7fb6567fc640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6440658b0 0x7fb644067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.263+0000 7fb6567fc640 1 -- 172.21.15.32:0/4022614018 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fb658082c10 con 0x7fb6601599e0 2023-12-20T06:03:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.263+0000 7fb665aa5640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6440658b0 0x7fb644067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.263+0000 7fb665aa5640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6440658b0 0x7fb644067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0af000 tx=0x7fb65c0a7000 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:01.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.267+0000 7fb6567fc640 1 -- 172.21.15.32:0/4022614018 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb6580502a0 con 0x7fb6601599e0 2023-12-20T06:03:01.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 -- 172.21.15.32:0/4022614018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb6600b1bf0 con 0x7fb6601599e0 2023-12-20T06:03:01.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb6567fc640 1 -- 172.21.15.32:0/4022614018 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5902 (secure 0 0 0) 0x7fb658017020 con 0x7fb6601599e0 2023-12-20T06:03:01.458 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:01.459 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:26.942526+0000","modified":"2023-12-20T06:02:29.955675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10539":{"gid":10539,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/888710554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":888710554},{"type":"v1","addr":"172.21.15.32:6837","nonce":888710554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:39.446992+0000","modified":"2023-12-20T06:02:42.509418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10530":{"gid":10530,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/3001088147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3001088147},{"type":"v1","addr":"172.21.15.39:6833","nonce":3001088147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-20T06:03:01.459 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 102 2023-12-20T06:03:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 -- 172.21.15.32:0/4022614018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6440658b0 msgr2=0x7fb644067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6440658b0 0x7fb644067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb65c0af000 tx=0x7fb65c0a7000 comp rx=0 tx=0).stop 2023-12-20T06:03:01.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 -- 172.21.15.32:0/4022614018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6601599e0 msgr2=0x7fb66014a1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6601599e0 0x7fb66014a1f0 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fb658002240 tx=0x7fb65800cbe0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 -- 172.21.15.32:0/4022614018 shutdown_connections 2023-12-20T06:03:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6440658b0 0x7fb644067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6601599e0 0x7fb66014a1f0 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6600c90e0 0x7fb6600c41d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 --2- 172.21.15.32:0/4022614018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6600b0d00 0x7fb6600b1f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 -- 172.21.15.32:0/4022614018 >> 172.21.15.32:0/4022614018 conn(0x7fb66006a320 msgr2=0x7fb660071360 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 -- 172.21.15.32:0/4022614018 shutdown_connections 2023-12-20T06:03:01.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.459+0000 7fb667d09640 1 -- 172.21.15.32:0/4022614018 wait complete. 2023-12-20T06:03:01.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb142e5640 1 Processor -- start 2023-12-20T06:03:01.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb142e5640 1 -- start start 2023-12-20T06:03:01.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb142e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb0c15cdb0 0x7feb0c0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb142e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7feb0c0d2740 0x7feb0c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb142e5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb0c0b6ca0 con 0x7feb0c0d1120 2023-12-20T06:03:01.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb142e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c15d190 con 0x7feb0c15cdb0 2023-12-20T06:03:01.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb142e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb0c0d16a0 con 0x7feb0c0d2740 2023-12-20T06:03:01.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb12081640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb0c15cdb0 0x7feb0c0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb12081640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7feb0c15cdb0 0x7feb0c0d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33750/0 (socket says 172.21.15.32:33750) 2023-12-20T06:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb12081640 1 -- 172.21.15.32:0/3098889225 learned_addr learned my addr 172.21.15.32:0/3098889225 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb11880640 1 --2- 172.21.15.32:0/3098889225 >> v2:172.21.15.39:3300/0 conn(0x7feb0c0d2740 0x7feb0c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb12081640 1 -- 172.21.15.32:0/3098889225 >> v2:172.21.15.39:3300/0 conn(0x7feb0c0d2740 msgr2=0x7feb0c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb12081640 1 --2- 172.21.15.32:0/3098889225 >> v2:172.21.15.39:3300/0 conn(0x7feb0c0d2740 0x7feb0c0cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb12081640 1 -- 172.21.15.32:0/3098889225 >> v1:172.21.15.39:6789/0 conn(0x7feb0c0d1120 legacy=0x7feb0c0b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb12081640 1 -- 172.21.15.32:0/3098889225 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb0c0d1980 con 0x7feb0c15cdb0 2023-12-20T06:03:01.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.627+0000 7feb11880640 1 --2- 172.21.15.32:0/3098889225 >> v2:172.21.15.39:3300/0 conn(0x7feb0c0d2740 0x7feb0c0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:01.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb12081640 1 --2- 172.21.15.32:0/3098889225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c15cdb0 0x7feb0c0d0be0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0cd570 tx=0x7feb0801ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3bf8fbebbf1b3e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:01.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb1107f640 1 -- 172.21.15.32:0/3098889225 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb080a89f0 con 0x7feb0c15cdb0 2023-12-20T06:03:01.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb1107f640 1 -- 172.21.15.32:0/3098889225 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb080a8b50 con 0x7feb0c15cdb0 2023-12-20T06:03:01.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb1107f640 1 -- 172.21.15.32:0/3098889225 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb080b0440 con 0x7feb0c15cdb0 2023-12-20T06:03:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb142e5640 1 -- 172.21.15.32:0/3098889225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c15cdb0 msgr2=0x7feb0c0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb142e5640 1 --2- 172.21.15.32:0/3098889225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c15cdb0 0x7feb0c0d0be0 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7feb0c0cd570 tx=0x7feb0801ecf0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb142e5640 1 -- 172.21.15.32:0/3098889225 shutdown_connections 2023-12-20T06:03:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb142e5640 1 --2- 172.21.15.32:0/3098889225 >> v2:172.21.15.39:3300/0 conn(0x7feb0c0d2740 0x7feb0c0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb142e5640 1 --2- 172.21.15.32:0/3098889225 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c15cdb0 0x7feb0c0d0be0 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb142e5640 1 -- 172.21.15.32:0/3098889225 >> 172.21.15.32:0/3098889225 conn(0x7feb0c065ee0 msgr2=0x7feb0c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb142e5640 1 -- 172.21.15.32:0/3098889225 shutdown_connections 2023-12-20T06:03:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.631+0000 7feb142e5640 1 -- 172.21.15.32:0/3098889225 wait complete. 2023-12-20T06:03:01.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb142e5640 1 Processor -- start 2023-12-20T06:03:01.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb142e5640 1 -- start start 2023-12-20T06:03:01.634 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb142e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb0c0d1120 0x7feb0c15fcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb142e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0d2740 0x7feb0c1e5c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb142e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c15cdb0 0x7feb0c1e7200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb142e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c15d900 con 0x7feb0c0d2740 2023-12-20T06:03:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb142e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c1e6160 con 0x7feb0c15cdb0 2023-12-20T06:03:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb142e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb0c15d2d0 con 0x7feb0c0d1120 2023-12-20T06:03:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb11880640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0d2740 0x7feb0c1e5c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb11880640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0d2740 0x7feb0c1e5c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33764/0 (socket says 172.21.15.32:33764) 2023-12-20T06:03:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb11880640 1 -- 172.21.15.32:0/2128549875 learned_addr learned my addr 172.21.15.32:0/2128549875 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb12081640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb0c0d1120 0x7feb0c15fcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb12882640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c15cdb0 0x7feb0c1e7200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb11880640 1 -- 172.21.15.32:0/2128549875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb0c0d1120 msgr2=0x7feb0c15fcf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb11880640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb0c0d1120 0x7feb0c15fcf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb11880640 1 -- 172.21.15.32:0/2128549875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c15cdb0 msgr2=0x7feb0c1e7200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb11880640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c15cdb0 0x7feb0c1e7200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb11880640 1 -- 172.21.15.32:0/2128549875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb080094d0 con 0x7feb0c0d2740 2023-12-20T06:03:01.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb11880640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0d2740 0x7feb0c1e5c20 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e810 tx=0x7feafc00ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb02ffd640 1 -- 172.21.15.32:0/2128549875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00c870 con 0x7feb0c0d2740 2023-12-20T06:03:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb02ffd640 1 -- 172.21.15.32:0/2128549875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feafc00c9d0 con 0x7feb0c0d2740 2023-12-20T06:03:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb142e5640 1 -- 172.21.15.32:0/2128549875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb0c1f2bd0 con 0x7feb0c0d2740 2023-12-20T06:03:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb02ffd640 1 -- 172.21.15.32:0/2128549875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feafc00cd80 con 0x7feb0c0d2740 2023-12-20T06:03:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb142e5640 1 -- 172.21.15.32:0/2128549875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb0c1e6670 con 0x7feb0c0d2740 2023-12-20T06:03:01.637 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.635+0000 7feb142e5640 1 -- 172.21.15.32:0/2128549875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb0c04e000 con 0x7feb0c0d2740 2023-12-20T06:03:01.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.639+0000 7feb02ffd640 1 -- 172.21.15.32:0/2128549875 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 132) v1 ==== 85066+0+0 (secure 0 0 0) 0x7feafc013410 con 0x7feb0c0d2740 2023-12-20T06:03:01.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.639+0000 7feb02ffd640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feaf00658b0 0x7feaf0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.639+0000 7feb12081640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feaf00658b0 0x7feaf0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.639+0000 7feb02ffd640 1 -- 172.21.15.32:0/2128549875 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7360+0+0 (secure 0 0 0) 0x7feafc054520 con 0x7feb0c0d2740 2023-12-20T06:03:01.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.639+0000 7feb12081640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feaf00658b0 0x7feaf0067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feb080b2000 tx=0x7feb08009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:01.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.643+0000 7feb02ffd640 1 -- 172.21.15.32:0/2128549875 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feafc050860 con 0x7feb0c0d2740 2023-12-20T06:03:01.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.651+0000 7feb02ffd640 1 -- 172.21.15.32:0/2128549875 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 133) v1 ==== 85376+0+0 (secure 0 0 0) 0x7feafc050c30 con 0x7feb0c0d2740 2023-12-20T06:03:01.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.815+0000 7feb142e5640 1 -- 172.21.15.32:0/2128549875 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7feb0c06b8d0 con 0x7feb0c0d2740 2023-12-20T06:03:01.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb02ffd640 1 -- 172.21.15.32:0/2128549875 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+20928 (secure 0 0 0) 0x7feafc07ef80 con 0x7feb0c0d2740 2023-12-20T06:03:01.817 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:01.817 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":215,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:59.631293+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:02:25.045126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-20T06:02:25.938205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:02:37.109832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:02:38.440830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3767986898":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/2893447374":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/4287660207":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1968262556":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1215843411":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1623434834":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/3650825501":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/2603552779":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/587127774":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3681500114":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/1096471740":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/922716312":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1437478066":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3373411185":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6833/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6832/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/4110141711":"2023-12-21T06:02:50.560868+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 -- 172.21.15.32:0/2128549875 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feaf00658b0 msgr2=0x7feaf0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feaf00658b0 0x7feaf0067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7feb080b2000 tx=0x7feb08009040 comp rx=0 tx=0).stop 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 -- 172.21.15.32:0/2128549875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0d2740 msgr2=0x7feb0c1e5c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0d2740 0x7feb0c1e5c20 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7feafc00e810 tx=0x7feafc00ecd0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 -- 172.21.15.32:0/2128549875 shutdown_connections 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7feaf00658b0 0x7feaf0067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb0c15cdb0 0x7feb0c1e7200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb0c0d2740 0x7feb0c1e5c20 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 --2- 172.21.15.32:0/2128549875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb0c0d1120 0x7feb0c15fcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 -- 172.21.15.32:0/2128549875 >> 172.21.15.32:0/2128549875 conn(0x7feb0c065ee0 msgr2=0x7feb0c0cd870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 -- 172.21.15.32:0/2128549875 shutdown_connections 2023-12-20T06:03:01.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.819+0000 7feb142e5640 1 -- 172.21.15.32:0/2128549875 wait complete. 2023-12-20T06:03:01.838 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T06:03:01.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34ba6fc640 1 Processor -- start 2023-12-20T06:03:01.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34ba6fc640 1 -- start start 2023-12-20T06:03:01.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34ba6fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f34b41593c0 0x7f34b415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34ba6fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34b406caf0 0x7f34b40705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34ba6fc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f34b40ae140 con 0x7f34b40afdb0 2023-12-20T06:03:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34ba6fc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b40ae9e0 con 0x7f34b406caf0 2023-12-20T06:03:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34ba6fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f34b415dee0 con 0x7f34b41593c0 2023-12-20T06:03:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34b8c99640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34b406caf0 0x7f34b40705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34b8c99640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f34b406caf0 0x7f34b40705a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33790/0 (socket says 172.21.15.32:33790) 2023-12-20T06:03:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34b8c99640 1 -- 172.21.15.32:0/1726099952 learned_addr learned my addr 172.21.15.32:0/1726099952 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34b3fff640 1 --2- 172.21.15.32:0/1726099952 >> v2:172.21.15.39:3300/0 conn(0x7f34b41593c0 0x7f34b415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34b8c99640 1 -- 172.21.15.32:0/1726099952 >> v2:172.21.15.39:3300/0 conn(0x7f34b41593c0 msgr2=0x7f34b415d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34b8c99640 1 --2- 172.21.15.32:0/1726099952 >> v2:172.21.15.39:3300/0 conn(0x7f34b41593c0 0x7f34b415d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34b8c99640 1 -- 172.21.15.32:0/1726099952 >> v1:172.21.15.32:6789/0 conn(0x7f34b40afdb0 legacy=0x7f34b406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.983+0000 7f34b8c99640 1 -- 172.21.15.32:0/1726099952 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34b4072170 con 0x7f34b406caf0 2023-12-20T06:03:01.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34b8c99640 1 --2- 172.21.15.32:0/1726099952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b406caf0 0x7f34b40705a0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f34b406ab00 tx=0x7f34ac01eb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20385cf727dde8bf server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:01.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34b2ffd640 1 -- 172.21.15.32:0/1726099952 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac00b2d0 con 0x7f34b406caf0 2023-12-20T06:03:01.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34b2ffd640 1 -- 172.21.15.32:0/1726099952 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34ac00b430 con 0x7f34b406caf0 2023-12-20T06:03:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34b2ffd640 1 -- 172.21.15.32:0/1726099952 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34ac0a7b80 con 0x7f34b406caf0 2023-12-20T06:03:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34ba6fc640 1 -- 172.21.15.32:0/1726099952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b406caf0 msgr2=0x7f34b40705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34ba6fc640 1 --2- 172.21.15.32:0/1726099952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b406caf0 0x7f34b40705a0 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f34b406ab00 tx=0x7f34ac01eb10 comp rx=0 tx=0).stop 2023-12-20T06:03:01.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34ba6fc640 1 -- 172.21.15.32:0/1726099952 shutdown_connections 2023-12-20T06:03:01.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34ba6fc640 1 --2- 172.21.15.32:0/1726099952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b406caf0 0x7f34b40705a0 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34ba6fc640 1 --2- 172.21.15.32:0/1726099952 >> v2:172.21.15.39:3300/0 conn(0x7f34b41593c0 0x7f34b415d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34ba6fc640 1 -- 172.21.15.32:0/1726099952 >> 172.21.15.32:0/1726099952 conn(0x7f34b406a320 msgr2=0x7f34b40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:01.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34ba6fc640 1 -- 172.21.15.32:0/1726099952 shutdown_connections 2023-12-20T06:03:01.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.987+0000 7f34ba6fc640 1 -- 172.21.15.32:0/1726099952 wait complete. 2023-12-20T06:03:01.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34ba6fc640 1 Processor -- start 2023-12-20T06:03:01.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34ba6fc640 1 -- start start 2023-12-20T06:03:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34ba6fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b406caf0 0x7f34b40b3220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34ba6fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b40afdb0 0x7f34b40abe10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34ba6fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34b41593c0 0x7f34b40c44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34ba6fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40c5e80 con 0x7f34b406caf0 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34ba6fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40b3b90 con 0x7f34b40afdb0 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34ba6fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f34b40b3e70 con 0x7f34b41593c0 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b406caf0 0x7f34b40b3220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b406caf0 0x7f34b40b3220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33804/0 (socket says 172.21.15.32:33804) 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b3fff640 1 -- 172.21.15.32:0/3991707012 learned_addr learned my addr 172.21.15.32:0/3991707012 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b37fe640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b40afdb0 0x7f34b40abe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b8c99640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34b41593c0 0x7f34b40c44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b3fff640 1 -- 172.21.15.32:0/3991707012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34b41593c0 msgr2=0x7f34b40c44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b3fff640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34b41593c0 0x7f34b40c44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b3fff640 1 -- 172.21.15.32:0/3991707012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b40afdb0 msgr2=0x7f34b40abe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b3fff640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b40afdb0 0x7f34b40abe10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:01.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b3fff640 1 -- 172.21.15.32:0/3991707012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f34ac00bd40 con 0x7f34b406caf0 2023-12-20T06:03:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b3fff640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b406caf0 0x7f34b40b3220 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f34a400e9b0 tx=0x7f34a400ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b0ff9640 1 -- 172.21.15.32:0/3991707012 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a4009670 con 0x7f34b406caf0 2023-12-20T06:03:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b0ff9640 1 -- 172.21.15.32:0/3991707012 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34a400c9a0 con 0x7f34b406caf0 2023-12-20T06:03:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34ba6fc640 1 -- 172.21.15.32:0/3991707012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f34b40c5830 con 0x7f34b406caf0 2023-12-20T06:03:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34b0ff9640 1 -- 172.21.15.32:0/3991707012 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34a400cd10 con 0x7f34b406caf0 2023-12-20T06:03:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34ba6fc640 1 -- 172.21.15.32:0/3991707012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f34b40b3760 con 0x7f34b406caf0 2023-12-20T06:03:01.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.991+0000 7f34ba6fc640 1 -- 172.21.15.32:0/3991707012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f34b404e000 con 0x7f34b406caf0 2023-12-20T06:03:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.995+0000 7f34b0ff9640 1 -- 172.21.15.32:0/3991707012 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 133) v1 ==== 85376+0+0 (secure 0 0 0) 0x7f34a4013470 con 0x7f34b406caf0 2023-12-20T06:03:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.995+0000 7f34b0ff9640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34a0065c40 0x7f34a0068100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.995+0000 7f34b0ff9640 1 -- 172.21.15.32:0/3991707012 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f34a4084080 con 0x7f34b406caf0 2023-12-20T06:03:01.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.995+0000 7f34b37fe640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34a0065c40 0x7f34a0068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:01.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.995+0000 7f34b37fe640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34a0065c40 0x7f34a0068100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f34b40b47c0 tx=0x7f34a8009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:01.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:01.999+0000 7f34b0ff9640 1 -- 172.21.15.32:0/3991707012 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f34a40b50a0 con 0x7f34b406caf0 2023-12-20T06:03:02.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 -- 172.21.15.32:0/3991707012 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f34b40ca5f0 con 0x7f34b406caf0 2023-12-20T06:03:02.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34b0ff9640 1 -- 172.21.15.32:0/3991707012 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5902 (secure 0 0 0) 0x7f34a4021090 con 0x7f34b406caf0 2023-12-20T06:03:02.190 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:02.191 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:26.942526+0000","modified":"2023-12-20T06:02:29.955675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10539":{"gid":10539,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/888710554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":888710554},{"type":"v1","addr":"172.21.15.32:6837","nonce":888710554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:39.446992+0000","modified":"2023-12-20T06:02:42.509418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10530":{"gid":10530,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/3001088147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3001088147},{"type":"v1","addr":"172.21.15.39:6833","nonce":3001088147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-20T06:03:02.191 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 102 2023-12-20T06:03:02.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 -- 172.21.15.32:0/3991707012 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34a0065c40 msgr2=0x7f34a0068100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34a0065c40 0x7f34a0068100 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f34b40b47c0 tx=0x7f34a8009040 comp rx=0 tx=0).stop 2023-12-20T06:03:02.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 -- 172.21.15.32:0/3991707012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b406caf0 msgr2=0x7f34b40b3220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b406caf0 0x7f34b40b3220 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f34a400e9b0 tx=0x7f34a400ee70 comp rx=0 tx=0).stop 2023-12-20T06:03:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 -- 172.21.15.32:0/3991707012 shutdown_connections 2023-12-20T06:03:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34a0065c40 0x7f34a0068100 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f34b41593c0 0x7f34b40c44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f34b40afdb0 0x7f34b40abe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 --2- 172.21.15.32:0/3991707012 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f34b406caf0 0x7f34b40b3220 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 -- 172.21.15.32:0/3991707012 >> 172.21.15.32:0/3991707012 conn(0x7f34b406a320 msgr2=0x7f34b40c90e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 -- 172.21.15.32:0/3991707012 shutdown_connections 2023-12-20T06:03:02.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.191+0000 7f34ba6fc640 1 -- 172.21.15.32:0/3991707012 wait complete. 2023-12-20T06:03:02.212 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:02.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.355+0000 7fe2d321f640 1 Processor -- start 2023-12-20T06:03:02.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.359+0000 7fe2d321f640 1 -- start start 2023-12-20T06:03:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.359+0000 7fe2d321f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2cc0af5e0 0x7fe2cc0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.359+0000 7fe2d321f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2cc15b990 con 0x7fe2cc15f110 2023-12-20T06:03:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.359+0000 7fe2d321f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe2cc15bbd0 con 0x7fe2cc06ce60 2023-12-20T06:03:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.359+0000 7fe2d321f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe2cc15beb0 con 0x7fe2cc0af5e0 2023-12-20T06:03:02.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.359+0000 7fe2d0fbb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2cc0af5e0 0x7fe2cc0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:02.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.359+0000 7fe2d0fbb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe2cc0af5e0 0x7fe2cc0af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33816/0 (socket says 172.21.15.32:33816) 2023-12-20T06:03:02.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.359+0000 7fe2d0fbb640 1 -- 172.21.15.32:0/3444002427 learned_addr learned my addr 172.21.15.32:0/3444002427 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:02.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.359+0000 7fe2d0fbb640 1 -- 172.21.15.32:0/3444002427 >> v1:172.21.15.39:6789/0 conn(0x7fe2cc06ce60 legacy=0x7fe2cc15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.359+0000 7fe2d0fbb640 1 -- 172.21.15.32:0/3444002427 >> v1:172.21.15.32:6789/0 conn(0x7fe2cc15f110 legacy=0x7fe2cc15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.359+0000 7fe2d0fbb640 1 -- 172.21.15.32:0/3444002427 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2cc15e910 con 0x7fe2cc0af5e0 2023-12-20T06:03:02.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2d0fbb640 1 --2- 172.21.15.32:0/3444002427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2cc0af5e0 0x7fe2cc0af9c0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0006fd0 tx=0x7fe2c001c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31f29c0495f81d55 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:02.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2cb7fe640 1 -- 172.21.15.32:0/3444002427 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00aa670 con 0x7fe2cc0af5e0 2023-12-20T06:03:02.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2cb7fe640 1 -- 172.21.15.32:0/3444002427 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c00aa7d0 con 0x7fe2cc0af5e0 2023-12-20T06:03:02.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2cb7fe640 1 -- 172.21.15.32:0/3444002427 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c00aab60 con 0x7fe2cc0af5e0 2023-12-20T06:03:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2d321f640 1 -- 172.21.15.32:0/3444002427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2cc0af5e0 msgr2=0x7fe2cc0af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2d321f640 1 --2- 172.21.15.32:0/3444002427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2cc0af5e0 0x7fe2cc0af9c0 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fe2c0006fd0 tx=0x7fe2c001c4f0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2d321f640 1 -- 172.21.15.32:0/3444002427 shutdown_connections 2023-12-20T06:03:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2d321f640 1 --2- 172.21.15.32:0/3444002427 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2cc0af5e0 0x7fe2cc0af9c0 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2d321f640 1 -- 172.21.15.32:0/3444002427 >> 172.21.15.32:0/3444002427 conn(0x7fe2cc06a320 msgr2=0x7fe2cc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2d321f640 1 -- 172.21.15.32:0/3444002427 shutdown_connections 2023-12-20T06:03:02.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2d321f640 1 -- 172.21.15.32:0/3444002427 wait complete. 2023-12-20T06:03:02.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2d321f640 1 Processor -- start 2023-12-20T06:03:02.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.363+0000 7fe2d321f640 1 -- start start 2023-12-20T06:03:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d321f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2cc06ce60 0x7fe2cc0b2840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d321f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2cc0af5e0 0x7fe2cc156a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d321f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2cc15f110 0x7fe2cc163dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d321f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc0cb7c0 con 0x7fe2cc15f110 2023-12-20T06:03:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d321f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc0cbc80 con 0x7fe2cc0af5e0 2023-12-20T06:03:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d321f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe2cc0b1ae0 con 0x7fe2cc06ce60 2023-12-20T06:03:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d17bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2cc15f110 0x7fe2cc163dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:02.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d17bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2cc15f110 0x7fe2cc163dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33824/0 (socket says 172.21.15.32:33824) 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d17bc640 1 -- 172.21.15.32:0/964153221 learned_addr learned my addr 172.21.15.32:0/964153221 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d0fbb640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2cc06ce60 0x7fe2cc0b2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2cbfff640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2cc0af5e0 0x7fe2cc156a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d17bc640 1 -- 172.21.15.32:0/964153221 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2cc06ce60 msgr2=0x7fe2cc0b2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d17bc640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2cc06ce60 0x7fe2cc0b2840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d17bc640 1 -- 172.21.15.32:0/964153221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2cc0af5e0 msgr2=0x7fe2cc156a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d17bc640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2cc0af5e0 0x7fe2cc156a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d17bc640 1 -- 172.21.15.32:0/964153221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe2c0009480 con 0x7fe2cc15f110 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d17bc640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2cc15f110 0x7fe2cc163dd0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c730 tx=0x7fe2c400cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2c97fa640 1 -- 172.21.15.32:0/964153221 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c4003b80 con 0x7fe2cc15f110 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2c97fa640 1 -- 172.21.15.32:0/964153221 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe2c4003ce0 con 0x7fe2cc15f110 2023-12-20T06:03:02.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2c97fa640 1 -- 172.21.15.32:0/964153221 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe2c400aa30 con 0x7fe2cc15f110 2023-12-20T06:03:02.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d321f640 1 -- 172.21.15.32:0/964153221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe2cc0cb9a0 con 0x7fe2cc15f110 2023-12-20T06:03:02.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d321f640 1 -- 172.21.15.32:0/964153221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe2cc0b2040 con 0x7fe2cc15f110 2023-12-20T06:03:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.367+0000 7fe2d321f640 1 -- 172.21.15.32:0/964153221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe2cc04e000 con 0x7fe2cc15f110 2023-12-20T06:03:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.371+0000 7fe2c97fa640 1 -- 172.21.15.32:0/964153221 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 133) v1 ==== 85376+0+0 (secure 0 0 0) 0x7fe2c400ab90 con 0x7fe2cc15f110 2023-12-20T06:03:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.371+0000 7fe2c97fa640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe2b0065c10 0x7fe2b00680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.371+0000 7fe2c97fa640 1 -- 172.21.15.32:0/964153221 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fe2c40835e0 con 0x7fe2cc15f110 2023-12-20T06:03:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.371+0000 7fe2d0fbb640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe2b0065c10 0x7fe2b00680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:02.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.371+0000 7fe2d0fbb640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe2b0065c10 0x7fe2b00680d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00ae000 tx=0x7fe2c00a7000 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:02.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.375+0000 7fe2c97fa640 1 -- 172.21.15.32:0/964153221 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe2c40506e0 con 0x7fe2cc15f110 2023-12-20T06:03:02.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.563+0000 7fe2d321f640 1 -- 172.21.15.32:0/964153221 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fe2cc15b990 con 0x7fe2cc15f110 2023-12-20T06:03:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2c97fa640 1 -- 172.21.15.32:0/964153221 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 102 v102) v1 ==== 95+0+5902 (secure 0 0 0) 0x7fe2cc15b990 con 0x7fe2cc15f110 2023-12-20T06:03:02.565 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:02.565 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":102,"default_fscid":12,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90}],"filesystems":[{"mdsmap":{"epoch":96,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:26.942526+0000","modified":"2023-12-20T06:02:29.955675+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10539},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10539":{"gid":10539,"name":"c","rank":0,"incarnation":92,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/888710554","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":888710554},{"type":"v1","addr":"172.21.15.32:6837","nonce":888710554}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[25],"metadata_pool":24,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":12},{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:39.446992+0000","modified":"2023-12-20T06:02:42.509418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10530":{"gid":10530,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/3001088147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3001088147},{"type":"v1","addr":"172.21.15.39:6833","nonce":3001088147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-20T06:03:02.565 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 102 2023-12-20T06:03:02.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 -- 172.21.15.32:0/964153221 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe2b0065c10 msgr2=0x7fe2b00680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe2b0065c10 0x7fe2b00680d0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fe2c00ae000 tx=0x7fe2c00a7000 comp rx=0 tx=0).stop 2023-12-20T06:03:02.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 -- 172.21.15.32:0/964153221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2cc15f110 msgr2=0x7fe2cc163dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2cc15f110 0x7fe2cc163dd0 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fe2c400c730 tx=0x7fe2c400cbf0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 -- 172.21.15.32:0/964153221 shutdown_connections 2023-12-20T06:03:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe2b0065c10 0x7fe2b00680d0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe2cc15f110 0x7fe2cc163dd0 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe2cc0af5e0 0x7fe2cc156a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 --2- 172.21.15.32:0/964153221 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe2cc06ce60 0x7fe2cc0b2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 -- 172.21.15.32:0/964153221 >> 172.21.15.32:0/964153221 conn(0x7fe2cc06a320 msgr2=0x7fe2cc0ce740 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 -- 172.21.15.32:0/964153221 shutdown_connections 2023-12-20T06:03:02.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.567+0000 7fe2d321f640 1 -- 172.21.15.32:0/964153221 wait complete. 2023-12-20T06:03:02.586 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:02.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.731+0000 7fe11eba2640 1 Processor -- start 2023-12-20T06:03:02.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.731+0000 7fe11eba2640 1 -- start start 2023-12-20T06:03:02.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.735+0000 7fe11eba2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe118161000 0x7fe1180caca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.735+0000 7fe11eba2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1180cb770 con 0x7fe1181613e0 2023-12-20T06:03:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.735+0000 7fe11eba2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1180cba50 con 0x7fe1180b0700 2023-12-20T06:03:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.735+0000 7fe11eba2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1180cbd30 con 0x7fe118161000 2023-12-20T06:03:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.735+0000 7fe11dba0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe118161000 0x7fe1180caca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.735+0000 7fe11dba0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe118161000 0x7fe1180caca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33832/0 (socket says 172.21.15.32:33832) 2023-12-20T06:03:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.735+0000 7fe11dba0640 1 -- 172.21.15.32:0/2397615581 learned_addr learned my addr 172.21.15.32:0/2397615581 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.735+0000 7fe11dba0640 1 -- 172.21.15.32:0/2397615581 >> v1:172.21.15.39:6789/0 conn(0x7fe1180b0700 legacy=0x7fe1180ccf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.735+0000 7fe11dba0640 1 -- 172.21.15.32:0/2397615581 >> v1:172.21.15.32:6789/0 conn(0x7fe1181613e0 legacy=0x7fe1180afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.735+0000 7fe11dba0640 1 -- 172.21.15.32:0/2397615581 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1180cc710 con 0x7fe118161000 2023-12-20T06:03:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.735+0000 7fe11dba0640 1 --2- 172.21.15.32:0/2397615581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118161000 0x7fe1180caca0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fe114009870 tx=0x7fe11401c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a1c0a55a565bdc4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:02.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11cb9e640 1 -- 172.21.15.32:0/2397615581 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140ab660 con 0x7fe118161000 2023-12-20T06:03:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11cb9e640 1 -- 172.21.15.32:0/2397615581 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1140ab7c0 con 0x7fe118161000 2023-12-20T06:03:02.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11cb9e640 1 -- 172.21.15.32:0/2397615581 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1140abb50 con 0x7fe118161000 2023-12-20T06:03:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 -- 172.21.15.32:0/2397615581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118161000 msgr2=0x7fe1180caca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 --2- 172.21.15.32:0/2397615581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118161000 0x7fe1180caca0 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7fe114009870 tx=0x7fe11401c440 comp rx=0 tx=0).stop 2023-12-20T06:03:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 -- 172.21.15.32:0/2397615581 shutdown_connections 2023-12-20T06:03:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 --2- 172.21.15.32:0/2397615581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118161000 0x7fe1180caca0 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 -- 172.21.15.32:0/2397615581 >> 172.21.15.32:0/2397615581 conn(0x7fe11806c060 msgr2=0x7fe11806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 -- 172.21.15.32:0/2397615581 shutdown_connections 2023-12-20T06:03:02.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 -- 172.21.15.32:0/2397615581 wait complete. 2023-12-20T06:03:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 Processor -- start 2023-12-20T06:03:02.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 -- start start 2023-12-20T06:03:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1180b0700 0x7fe1181528b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118161000 0x7fe118152160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1181613e0 0x7fe118158c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1181551a0 con 0x7fe118161000 2023-12-20T06:03:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1180c3cf0 con 0x7fe1181613e0 2023-12-20T06:03:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11eba2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1181526a0 con 0x7fe1180b0700 2023-12-20T06:03:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11d39f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118161000 0x7fe118152160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11d39f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118161000 0x7fe118152160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33834/0 (socket says 172.21.15.32:33834) 2023-12-20T06:03:02.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.739+0000 7fe11d39f640 1 -- 172.21.15.32:0/2201779690 learned_addr learned my addr 172.21.15.32:0/2201779690 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11e3a1640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1181613e0 0x7fe118158c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11dba0640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1180b0700 0x7fe1181528b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11d39f640 1 -- 172.21.15.32:0/2201779690 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1180b0700 msgr2=0x7fe1181528b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11d39f640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1180b0700 0x7fe1181528b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11d39f640 1 -- 172.21.15.32:0/2201779690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1181613e0 msgr2=0x7fe118158c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11d39f640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1181613e0 0x7fe118158c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11d39f640 1 -- 172.21.15.32:0/2201779690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1140094d0 con 0x7fe118161000 2023-12-20T06:03:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11dba0640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1180b0700 0x7fe1181528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11d39f640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118161000 0x7fe118152160 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c500 tx=0x7fe10800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe10e7fc640 1 -- 172.21.15.32:0/2201779690 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108005220 con 0x7fe118161000 2023-12-20T06:03:02.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe10e7fc640 1 -- 172.21.15.32:0/2201779690 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe108005380 con 0x7fe118161000 2023-12-20T06:03:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe10e7fc640 1 -- 172.21.15.32:0/2201779690 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe108007270 con 0x7fe118161000 2023-12-20T06:03:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11eba2640 1 -- 172.21.15.32:0/2201779690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe118156d00 con 0x7fe118161000 2023-12-20T06:03:02.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11eba2640 1 -- 172.21.15.32:0/2201779690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1180c36a0 con 0x7fe118161000 2023-12-20T06:03:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11eba2640 1 -- 172.21.15.32:0/2201779690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe1180c3500 con 0x7fe118161000 2023-12-20T06:03:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe10e7fc640 1 -- 172.21.15.32:0/2201779690 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 133) v1 ==== 85376+0+0 (secure 0 0 0) 0x7fe1080073d0 con 0x7fe118161000 2023-12-20T06:03:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe10e7fc640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe0fc065c10 0x7fe0fc0680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe10e7fc640 1 -- 172.21.15.32:0/2201779690 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7360+0+0 (secure 0 0 0) 0x7fe1080833a0 con 0x7fe118161000 2023-12-20T06:03:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.743+0000 7fe11dba0640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe0fc065c10 0x7fe0fc0680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:02.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.747+0000 7fe11dba0640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe0fc065c10 0x7fe0fc0680d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe1140af000 tx=0x7fe1140a7000 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:02.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.747+0000 7fe10e7fc640 1 -- 172.21.15.32:0/2201779690 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe1180c3500 con 0x7fe118161000 2023-12-20T06:03:02.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.907+0000 7fe11eba2640 1 -- 172.21.15.32:0/2201779690 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe1180c3500 con 0x7fe118161000 2023-12-20T06:03:02.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe10e7fc640 1 -- 172.21.15.32:0/2201779690 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v215) v1 ==== 74+0+20928 (secure 0 0 0) 0x7fe1180c3500 con 0x7fe118161000 2023-12-20T06:03:02.909 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:02.909 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":215,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:02:59.631293+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":24,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:02:25.045126+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"208","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":25,"pool_name":"cephfs_data","create_time":"2023-12-20T06:02:25.938205+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"209","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.3799999952316284,"score_stable":1.3799999952316284,"optimal_score":1,"raw_score_acting":1.3799999952316284,"raw_score_stable":1.3799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:02:37.109832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:02:38.440830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3767986898":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/2893447374":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/4287660207":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1968262556":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1215843411":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1623434834":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/3650825501":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/2603552779":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/587127774":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3681500114":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/1096471740":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/922716312":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1437478066":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3373411185":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6833/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6832/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/4110141711":"2023-12-21T06:02:50.560868+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:02.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 -- 172.21.15.32:0/2201779690 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe0fc065c10 msgr2=0x7fe0fc0680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe0fc065c10 0x7fe0fc0680d0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fe1140af000 tx=0x7fe1140a7000 comp rx=0 tx=0).stop 2023-12-20T06:03:02.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 -- 172.21.15.32:0/2201779690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118161000 msgr2=0x7fe118152160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118161000 0x7fe118152160 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7fe10800c500 tx=0x7fe10800c9c0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 -- 172.21.15.32:0/2201779690 shutdown_connections 2023-12-20T06:03:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe0fc065c10 0x7fe0fc0680d0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1181613e0 0x7fe118158c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe118161000 0x7fe118152160 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 --2- 172.21.15.32:0/2201779690 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1180b0700 0x7fe1181528b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 -- 172.21.15.32:0/2201779690 >> 172.21.15.32:0/2201779690 conn(0x7fe11806c060 msgr2=0x7fe11806c440 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 -- 172.21.15.32:0/2201779690 shutdown_connections 2023-12-20T06:03:02.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:02.911+0000 7fe11eba2640 1 -- 172.21.15.32:0/2201779690 wait complete. 2023-12-20T06:03:02.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T06:03:03.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.075+0000 7f7a5efa6640 1 Processor -- start 2023-12-20T06:03:03.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.075+0000 7f7a5efa6640 1 -- start start 2023-12-20T06:03:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5efa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a5806ce60 0x7f7a5815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5efa6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a5815f110 0x7f7a5815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5efa6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a5815bbd0 con 0x7f7a580af5e0 2023-12-20T06:03:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5efa6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a5815beb0 con 0x7f7a5815f110 2023-12-20T06:03:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5efa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5815e910 con 0x7f7a5806ce60 2023-12-20T06:03:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5d543640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a5815f110 0x7f7a5815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:03.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5d543640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a5815f110 0x7f7a5815c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33840/0 (socket says 172.21.15.32:33840) 2023-12-20T06:03:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5d543640 1 -- 172.21.15.32:0/3443706732 learned_addr learned my addr 172.21.15.32:0/3443706732 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a57fff640 1 --2- 172.21.15.32:0/3443706732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a5806ce60 0x7f7a5815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5d543640 1 -- 172.21.15.32:0/3443706732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a5806ce60 msgr2=0x7f7a5815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5d543640 1 --2- 172.21.15.32:0/3443706732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a5806ce60 0x7f7a5815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5d543640 1 -- 172.21.15.32:0/3443706732 >> v1:172.21.15.39:6789/0 conn(0x7f7a580af5e0 legacy=0x7f7a580af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:03.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.079+0000 7f7a5d543640 1 -- 172.21.15.32:0/3443706732 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a5815ebf0 con 0x7f7a5815f110 2023-12-20T06:03:03.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a5d543640 1 --2- 172.21.15.32:0/3443706732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5815f110 0x7f7a5815c440 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009e50 tx=0x7f7a5001c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d07f7715fecc14b7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:03.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a577fe640 1 -- 172.21.15.32:0/3443706732 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500a57e0 con 0x7f7a5815f110 2023-12-20T06:03:03.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a577fe640 1 -- 172.21.15.32:0/3443706732 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a500a5940 con 0x7f7a5815f110 2023-12-20T06:03:03.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a577fe640 1 -- 172.21.15.32:0/3443706732 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500a5cd0 con 0x7f7a5815f110 2023-12-20T06:03:03.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a5efa6640 1 -- 172.21.15.32:0/3443706732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5815f110 msgr2=0x7f7a5815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:03.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a5efa6640 1 --2- 172.21.15.32:0/3443706732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5815f110 0x7f7a5815c440 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f7a50009e50 tx=0x7f7a5001c980 comp rx=0 tx=0).stop 2023-12-20T06:03:03.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a5efa6640 1 -- 172.21.15.32:0/3443706732 shutdown_connections 2023-12-20T06:03:03.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a5efa6640 1 --2- 172.21.15.32:0/3443706732 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5815f110 0x7f7a5815c440 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a5efa6640 1 --2- 172.21.15.32:0/3443706732 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a5806ce60 0x7f7a5815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a5efa6640 1 -- 172.21.15.32:0/3443706732 >> 172.21.15.32:0/3443706732 conn(0x7f7a5806a320 msgr2=0x7f7a580ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:03.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a5efa6640 1 -- 172.21.15.32:0/3443706732 shutdown_connections 2023-12-20T06:03:03.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a5efa6640 1 -- 172.21.15.32:0/3443706732 wait complete. 2023-12-20T06:03:03.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a5efa6640 1 Processor -- start 2023-12-20T06:03:03.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.083+0000 7f7a5efa6640 1 -- start start 2023-12-20T06:03:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5efa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a5806ce60 0x7f7a581f5380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5efa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a580af5e0 0x7f7a5814bc40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5efa6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5815f110 0x7f7a5814c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5efa6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58159aa0 con 0x7f7a5815f110 2023-12-20T06:03:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5efa6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a5814d0e0 con 0x7f7a580af5e0 2023-12-20T06:03:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5efa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a58158ed0 con 0x7f7a5806ce60 2023-12-20T06:03:03.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5d543640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5815f110 0x7f7a5814c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5d543640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5815f110 0x7f7a5814c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33846/0 (socket says 172.21.15.32:33846) 2023-12-20T06:03:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5d543640 1 -- 172.21.15.32:0/1097898827 learned_addr learned my addr 172.21.15.32:0/1097898827 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5cd42640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a5806ce60 0x7f7a581f5380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a57fff640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a580af5e0 0x7f7a5814bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5d543640 1 -- 172.21.15.32:0/1097898827 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a5806ce60 msgr2=0x7f7a581f5380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5d543640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a5806ce60 0x7f7a581f5380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5d543640 1 -- 172.21.15.32:0/1097898827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a580af5e0 msgr2=0x7f7a5814bc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5d543640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a580af5e0 0x7f7a5814bc40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5d543640 1 -- 172.21.15.32:0/1097898827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a50009b30 con 0x7f7a5815f110 2023-12-20T06:03:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5d543640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5815f110 0x7f7a5814c180 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f7a500af040 tx=0x7f7a50019720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:03.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a557fa640 1 -- 172.21.15.32:0/1097898827 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500be070 con 0x7f7a5815f110 2023-12-20T06:03:03.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5efa6640 1 -- 172.21.15.32:0/1097898827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a58159170 con 0x7f7a5815f110 2023-12-20T06:03:03.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a557fa640 1 -- 172.21.15.32:0/1097898827 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a500b8440 con 0x7f7a5815f110 2023-12-20T06:03:03.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a557fa640 1 -- 172.21.15.32:0/1097898827 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a500b87f0 con 0x7f7a5815f110 2023-12-20T06:03:03.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5efa6640 1 -- 172.21.15.32:0/1097898827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a58159730 con 0x7f7a5815f110 2023-12-20T06:03:03.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a5efa6640 1 -- 172.21.15.32:0/1097898827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a58159c10 con 0x7f7a5815f110 2023-12-20T06:03:03.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.087+0000 7f7a557fa640 1 -- 172.21.15.32:0/1097898827 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 133) v1 ==== 85376+0+0 (secure 0 0 0) 0x7f7a500b8950 con 0x7f7a5815f110 2023-12-20T06:03:03.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.091+0000 7f7a557fa640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a3c065c40 0x7f7a3c068100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:03.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.091+0000 7f7a557fa640 1 -- 172.21.15.32:0/1097898827 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(215..215 src has 1..215) v4 ==== 7360+0+0 (secure 0 0 0) 0x7f7a5011e2c0 con 0x7f7a5815f110 2023-12-20T06:03:03.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.091+0000 7f7a5cd42640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a3c065c40 0x7f7a3c068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:03.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.091+0000 7f7a5cd42640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a3c065c40 0x7f7a3c068100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c001740 tx=0x7f7a4c009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:03.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.095+0000 7f7a557fa640 1 -- 172.21.15.32:0/1097898827 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7a500eb330 con 0x7f7a5815f110 2023-12-20T06:03:03.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.287+0000 7f7a5efa6640 1 -- 172.21.15.32:0/1097898827 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f7a5804e000 con 0x7f7a5815f110 2023-12-20T06:03:03.289 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T06:03:03.291+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T06:03:03.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.303+0000 7f7a557fa640 1 -- 172.21.15.32:0/1097898827 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v103) v1 ==== 161+0+0 (secure 0 0 0) 0x7f7a500ee950 con 0x7f7a5815f110 2023-12-20T06:03:03.302 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T06:03:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.303+0000 7f7a5efa6640 1 -- 172.21.15.32:0/1097898827 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a3c065c40 msgr2=0x7f7a3c068100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.303+0000 7f7a5efa6640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a3c065c40 0x7f7a3c068100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f7a4c001740 tx=0x7f7a4c009040 comp rx=0 tx=0).stop 2023-12-20T06:03:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.303+0000 7f7a5efa6640 1 -- 172.21.15.32:0/1097898827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5815f110 msgr2=0x7f7a5814c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.303+0000 7f7a5efa6640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5815f110 0x7f7a5814c180 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f7a500af040 tx=0x7f7a50019720 comp rx=0 tx=0).stop 2023-12-20T06:03:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.307+0000 7f7a5efa6640 1 -- 172.21.15.32:0/1097898827 shutdown_connections 2023-12-20T06:03:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.307+0000 7f7a5efa6640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a3c065c40 0x7f7a3c068100 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.307+0000 7f7a5efa6640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a5815f110 0x7f7a5814c180 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.307+0000 7f7a5efa6640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a580af5e0 0x7f7a5814bc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.307+0000 7f7a5efa6640 1 --2- 172.21.15.32:0/1097898827 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a5806ce60 0x7f7a581f5380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.307+0000 7f7a5efa6640 1 -- 172.21.15.32:0/1097898827 >> 172.21.15.32:0/1097898827 conn(0x7f7a5806a320 msgr2=0x7f7a58164760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.307+0000 7f7a5efa6640 1 -- 172.21.15.32:0/1097898827 shutdown_connections 2023-12-20T06:03:03.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.307+0000 7f7a5efa6640 1 -- 172.21.15.32:0/1097898827 wait complete. 2023-12-20T06:03:03.310 INFO:tasks.ceph.mds.c.smithi032.stderr: -4259> 2023-12-20T06:02:32.512+0000 7f426b674640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T06:03:03.325 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T06:03:03.360 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T06:03:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb42b4d7640 1 Processor -- start 2023-12-20T06:03:03.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb42b4d7640 1 -- start start 2023-12-20T06:03:03.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb42b4d7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240d3060 0x7fb4241676d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:03.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb42b4d7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4240b6b00 con 0x7fb4240b8520 2023-12-20T06:03:03.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb42b4d7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb4240d2090 con 0x7fb42415cc30 2023-12-20T06:03:03.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb42b4d7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb42415d010 con 0x7fb4240d3060 2023-12-20T06:03:03.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb429cd4640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb4240b8520 0x7fb4240d2950 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43892/0 (socket says 172.21.15.32:43892) 2023-12-20T06:03:03.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb429cd4640 1 -- 172.21.15.32:0/3641344949 learned_addr learned my addr 172.21.15.32:0/3641344949 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:03.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb42acd6640 1 --2- 172.21.15.32:0/3641344949 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240d3060 0x7fb4241676d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:03.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb4294d3640 1 -- 172.21.15.32:0/3641344949 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 139371381 0 0) 0x7fb414001120 con 0x7fb4240b8520 2023-12-20T06:03:03.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb4294d3640 1 -- 172.21.15.32:0/3641344949 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb410003530 con 0x7fb4240b8520 2023-12-20T06:03:03.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb4294d3640 1 -- 172.21.15.32:0/3641344949 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 163859124 0 0) 0x7fb4240d2090 con 0x7fb42415cc30 2023-12-20T06:03:03.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb4294d3640 1 -- 172.21.15.32:0/3641344949 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb414001120 con 0x7fb42415cc30 2023-12-20T06:03:03.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb42acd6640 1 -- 172.21.15.32:0/3641344949 >> v1:172.21.15.39:6789/0 conn(0x7fb42415cc30 legacy=0x7fb4240d0ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:03.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb42acd6640 1 -- 172.21.15.32:0/3641344949 >> v1:172.21.15.32:6789/0 conn(0x7fb4240b8520 legacy=0x7fb4240d2950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:03.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.475+0000 7fb42acd6640 1 -- 172.21.15.32:0/3641344949 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb4240b8980 con 0x7fb4240d3060 2023-12-20T06:03:03.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.479+0000 7fb42acd6640 1 --2- 172.21.15.32:0/3641344949 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240d3060 0x7fb4241676d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009dc0 tx=0x7fb41c01c940 comp rx=0 tx=0).ready entity=mon.2 client_cookie=75b37966dd028dc0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:03.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.479+0000 7fb4294d3640 1 -- 172.21.15.32:0/3641344949 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c0a5a00 con 0x7fb4240d3060 2023-12-20T06:03:03.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.479+0000 7fb4294d3640 1 -- 172.21.15.32:0/3641344949 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb41c0a5b60 con 0x7fb4240d3060 2023-12-20T06:03:03.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.479+0000 7fb4294d3640 1 -- 172.21.15.32:0/3641344949 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb41c0ad440 con 0x7fb4240d3060 2023-12-20T06:03:03.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.479+0000 7fb42b4d7640 1 -- 172.21.15.32:0/3641344949 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240d3060 msgr2=0x7fb4241676d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:03.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.479+0000 7fb42b4d7640 1 --2- 172.21.15.32:0/3641344949 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240d3060 0x7fb4241676d0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fb41c009dc0 tx=0x7fb41c01c940 comp rx=0 tx=0).stop 2023-12-20T06:03:03.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.479+0000 7fb42b4d7640 1 -- 172.21.15.32:0/3641344949 shutdown_connections 2023-12-20T06:03:03.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.479+0000 7fb42b4d7640 1 --2- 172.21.15.32:0/3641344949 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240d3060 0x7fb4241676d0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.479+0000 7fb42b4d7640 1 -- 172.21.15.32:0/3641344949 >> 172.21.15.32:0/3641344949 conn(0x7fb424065ce0 msgr2=0x7fb4240660f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:03.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42b4d7640 1 -- 172.21.15.32:0/3641344949 shutdown_connections 2023-12-20T06:03:03.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42b4d7640 1 -- 172.21.15.32:0/3641344949 wait complete. 2023-12-20T06:03:03.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42b4d7640 1 Processor -- start 2023-12-20T06:03:03.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42b4d7640 1 -- start start 2023-12-20T06:03:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42b4d7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240b8520 0x7fb4240cb740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42b4d7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4240d3060 0x7fb42415e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42b4d7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb42415cc30 0x7fb424158770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:03.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42b4d7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb424167030 con 0x7fb4240d3060 2023-12-20T06:03:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42b4d7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb4241645e0 con 0x7fb42415cc30 2023-12-20T06:03:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42b4d7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb424166700 con 0x7fb4240b8520 2023-12-20T06:03:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb429cd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4240d3060 0x7fb42415e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb429cd4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4240d3060 0x7fb42415e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33884/0 (socket says 172.21.15.32:33884) 2023-12-20T06:03:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb429cd4640 1 -- 172.21.15.32:0/2888661802 learned_addr learned my addr 172.21.15.32:0/2888661802 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42a4d5640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240b8520 0x7fb4240cb740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42acd6640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb42415cc30 0x7fb424158770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb429cd4640 1 -- 172.21.15.32:0/2888661802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240b8520 msgr2=0x7fb4240cb740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb429cd4640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240b8520 0x7fb4240cb740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb429cd4640 1 -- 172.21.15.32:0/2888661802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb42415cc30 msgr2=0x7fb424158770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb429cd4640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb42415cc30 0x7fb424158770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb429cd4640 1 -- 172.21.15.32:0/2888661802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb41c009a70 con 0x7fb4240d3060 2023-12-20T06:03:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42a4d5640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240b8520 0x7fb4240cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb42acd6640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb42415cc30 0x7fb424158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.483+0000 7fb429cd4640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4240d3060 0x7fb42415e6d0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c650 tx=0x7fb41400cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.487+0000 7fb41affd640 1 -- 172.21.15.32:0/2888661802 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb414005ac0 con 0x7fb4240d3060 2023-12-20T06:03:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.487+0000 7fb41affd640 1 -- 172.21.15.32:0/2888661802 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb414005c40 con 0x7fb4240d3060 2023-12-20T06:03:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.487+0000 7fb41affd640 1 -- 172.21.15.32:0/2888661802 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb414007070 con 0x7fb4240d3060 2023-12-20T06:03:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.487+0000 7fb42b4d7640 1 -- 172.21.15.32:0/2888661802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4241669e0 con 0x7fb4240d3060 2023-12-20T06:03:03.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.487+0000 7fb42b4d7640 1 -- 172.21.15.32:0/2888661802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4241674c0 con 0x7fb4240d3060 2023-12-20T06:03:03.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.487+0000 7fb42b4d7640 1 -- 172.21.15.32:0/2888661802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb4241671a0 con 0x7fb4240d3060 2023-12-20T06:03:03.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.487+0000 7fb41affd640 1 -- 172.21.15.32:0/2888661802 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 133) v1 ==== 85376+0+0 (secure 0 0 0) 0x7fb4140071f0 con 0x7fb4240d3060 2023-12-20T06:03:03.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.487+0000 7fb41affd640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb408065c10 0x7fb4080680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:03.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.487+0000 7fb41affd640 1 -- 172.21.15.32:0/2888661802 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7446+0+0 (secure 0 0 0) 0x7fb414083160 con 0x7fb4240d3060 2023-12-20T06:03:03.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.487+0000 7fb42a4d5640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb408065c10 0x7fb4080680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:03.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.487+0000 7fb42a4d5640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb408065c10 0x7fb4080680d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb4200011f0 tx=0x7fb420009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:03.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.491+0000 7fb41affd640 1 -- 172.21.15.32:0/2888661802 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb4140501c0 con 0x7fb4240d3060 2023-12-20T06:03:03.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:03.687+0000 7fb42b4d7640 1 -- 172.21.15.32:0/2888661802 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fb4240b6b00 con 0x7fb4240d3060 2023-12-20T06:03:04.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.319+0000 7fb41affd640 1 -- 172.21.15.32:0/2888661802 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb414050590 con 0x7fb4240d3060 2023-12-20T06:03:04.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb41affd640 1 -- 172.21.15.32:0/2888661802 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v104) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb4140501c0 con 0x7fb4240d3060 2023-12-20T06:03:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 -- 172.21.15.32:0/2888661802 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb408065c10 msgr2=0x7fb4080680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb408065c10 0x7fb4080680d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fb4200011f0 tx=0x7fb420009040 comp rx=0 tx=0).stop 2023-12-20T06:03:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 -- 172.21.15.32:0/2888661802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4240d3060 msgr2=0x7fb42415e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4240d3060 0x7fb42415e6d0 secure :-1 s=READY pgs=2163 cs=0 l=1 rev1=1 crypto rx=0x7fb41400c650 tx=0x7fb41400cb10 comp rx=0 tx=0).stop 2023-12-20T06:03:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 -- 172.21.15.32:0/2888661802 shutdown_connections 2023-12-20T06:03:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb408065c10 0x7fb4080680d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb42415cc30 0x7fb424158770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:04.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb4240d3060 0x7fb42415e6d0 unknown :-1 s=CLOSED pgs=2163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 --2- 172.21.15.32:0/2888661802 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb4240b8520 0x7fb4240cb740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 -- 172.21.15.32:0/2888661802 >> 172.21.15.32:0/2888661802 conn(0x7fb424065ce0 msgr2=0x7fb4240cda90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 -- 172.21.15.32:0/2888661802 shutdown_connections 2023-12-20T06:03:04.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.323+0000 7fb42b4d7640 1 -- 172.21.15.32:0/2888661802 wait complete. 2023-12-20T06:03:04.347 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T06:03:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.543+0000 7f43b2d7b640 1 Processor -- start 2023-12-20T06:03:04.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.543+0000 7f43b2d7b640 1 -- start start 2023-12-20T06:03:04.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.543+0000 7f43b2d7b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43ac06dcb0 0x7f43ac0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:04.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.543+0000 7f43b2d7b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f43ac070230 0x7f43ac0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:04.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.543+0000 7f43b2d7b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43ac06e100 con 0x7f43ac158520 2023-12-20T06:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.543+0000 7f43b2d7b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac070be0 con 0x7f43ac06dcb0 2023-12-20T06:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.543+0000 7f43b2d7b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43ac0ca590 con 0x7f43ac070230 2023-12-20T06:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43abfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43ac06dcb0 0x7f43ac0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43abfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43ac06dcb0 0x7f43ac0ca050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33890/0 (socket says 172.21.15.32:33890) 2023-12-20T06:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43abfff640 1 -- 172.21.15.32:0/3083264330 learned_addr learned my addr 172.21.15.32:0/3083264330 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43b1318640 1 --2- 172.21.15.32:0/3083264330 >> v2:172.21.15.39:3300/0 conn(0x7f43ac070230 0x7f43ac0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43abfff640 1 -- 172.21.15.32:0/3083264330 >> v2:172.21.15.39:3300/0 conn(0x7f43ac070230 msgr2=0x7f43ac0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43abfff640 1 --2- 172.21.15.32:0/3083264330 >> v2:172.21.15.39:3300/0 conn(0x7f43ac070230 0x7f43ac0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43abfff640 1 -- 172.21.15.32:0/3083264330 >> v1:172.21.15.39:6789/0 conn(0x7f43ac158520 legacy=0x7f43ac15ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:04.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43abfff640 1 -- 172.21.15.32:0/3083264330 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43ac0ca7f0 con 0x7f43ac06dcb0 2023-12-20T06:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43abfff640 1 --2- 172.21.15.32:0/3083264330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ac06dcb0 0x7f43ac0ca050 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0b24e0 tx=0x7f439801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e42a38ab0f379e9b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43ab7fe640 1 -- 172.21.15.32:0/3083264330 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43980ab640 con 0x7f43ac06dcb0 2023-12-20T06:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43ab7fe640 1 -- 172.21.15.32:0/3083264330 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43980a7070 con 0x7f43ac06dcb0 2023-12-20T06:03:04.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.547+0000 7f43ab7fe640 1 -- 172.21.15.32:0/3083264330 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43980ab9c0 con 0x7f43ac06dcb0 2023-12-20T06:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 -- 172.21.15.32:0/3083264330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ac06dcb0 msgr2=0x7f43ac0ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 --2- 172.21.15.32:0/3083264330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ac06dcb0 0x7f43ac0ca050 secure :-1 s=READY pgs=2164 cs=0 l=1 rev1=1 crypto rx=0x7f43ac0b24e0 tx=0x7f439801c440 comp rx=0 tx=0).stop 2023-12-20T06:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 -- 172.21.15.32:0/3083264330 shutdown_connections 2023-12-20T06:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 --2- 172.21.15.32:0/3083264330 >> v2:172.21.15.39:3300/0 conn(0x7f43ac070230 0x7f43ac0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 --2- 172.21.15.32:0/3083264330 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ac06dcb0 0x7f43ac0ca050 unknown :-1 s=CLOSED pgs=2164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 -- 172.21.15.32:0/3083264330 >> 172.21.15.32:0/3083264330 conn(0x7f43ac06a510 msgr2=0x7f43ac0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 -- 172.21.15.32:0/3083264330 shutdown_connections 2023-12-20T06:03:04.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 -- 172.21.15.32:0/3083264330 wait complete. 2023-12-20T06:03:04.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 Processor -- start 2023-12-20T06:03:04.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 -- start start 2023-12-20T06:03:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac06dcb0 0x7f43ac157410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43ac070230 0x7f43ac155b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ac158520 0x7f43ac1400e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:04.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac156f80 con 0x7f43ac158520 2023-12-20T06:03:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac156370 con 0x7f43ac06dcb0 2023-12-20T06:03:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b2d7b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43ac156650 con 0x7f43ac070230 2023-12-20T06:03:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b1318640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ac158520 0x7f43ac1400e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b1318640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ac158520 0x7f43ac1400e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33904/0 (socket says 172.21.15.32:33904) 2023-12-20T06:03:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b1318640 1 -- 172.21.15.32:0/1940527948 learned_addr learned my addr 172.21.15.32:0/1940527948 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43b0b17640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac06dcb0 0x7f43ac157410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.551+0000 7f43abfff640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43ac070230 0x7f43ac155b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43b1318640 1 -- 172.21.15.32:0/1940527948 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43ac070230 msgr2=0x7f43ac155b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43b1318640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43ac070230 0x7f43ac155b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43b1318640 1 -- 172.21.15.32:0/1940527948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac06dcb0 msgr2=0x7f43ac157410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:04.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43b1318640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac06dcb0 0x7f43ac157410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:04.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43b1318640 1 -- 172.21.15.32:0/1940527948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43980094d0 con 0x7f43ac158520 2023-12-20T06:03:04.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43b1318640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ac158520 0x7f43ac1400e0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f43a400efc0 tx=0x7f43a400c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:04.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43a97fa640 1 -- 172.21.15.32:0/1940527948 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4014480 con 0x7f43ac158520 2023-12-20T06:03:04.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43a97fa640 1 -- 172.21.15.32:0/1940527948 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43a400f040 con 0x7f43ac158520 2023-12-20T06:03:04.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43a97fa640 1 -- 172.21.15.32:0/1940527948 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43a4014820 con 0x7f43ac158520 2023-12-20T06:03:04.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43b2d7b640 1 -- 172.21.15.32:0/1940527948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43ac156930 con 0x7f43ac158520 2023-12-20T06:03:04.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43b2d7b640 1 -- 172.21.15.32:0/1940527948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43ac156060 con 0x7f43ac158520 2023-12-20T06:03:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43b2d7b640 1 -- 172.21.15.32:0/1940527948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43ac1570f0 con 0x7f43ac158520 2023-12-20T06:03:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43a97fa640 1 -- 172.21.15.32:0/1940527948 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f43a4014980 con 0x7f43ac158520 2023-12-20T06:03:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43a97fa640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4390065cd0 0x7f4390068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43a97fa640 1 -- 172.21.15.32:0/1940527948 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(216..216 src has 1..216) v4 ==== 7446+0+0 (secure 0 0 0) 0x7f43a4084910 con 0x7f43ac158520 2023-12-20T06:03:04.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.555+0000 7f43b0b17640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4390065cd0 0x7f4390068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:04.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.559+0000 7f43b0b17640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4390065cd0 0x7f4390068190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f43a0007fd0 tx=0x7f43a0009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:04.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.559+0000 7f43a97fa640 1 -- 172.21.15.32:0/1940527948 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f43a4051710 con 0x7f43ac158520 2023-12-20T06:03:04.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:04.719+0000 7f43b2d7b640 1 -- 172.21.15.32:0/1940527948 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f43ac04e000 con 0x7f43ac158520 2023-12-20T06:03:05.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.687+0000 7f43a97fa640 1 -- 172.21.15.32:0/1940527948 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v217) v1 ==== 181+0+0 (secure 0 0 0) 0x7f43a4018020 con 0x7f43ac158520 2023-12-20T06:03:05.688 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T06:03:05.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 -- 172.21.15.32:0/1940527948 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4390065cd0 msgr2=0x7f4390068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4390065cd0 0x7f4390068190 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f43a0007fd0 tx=0x7f43a0009040 comp rx=0 tx=0).stop 2023-12-20T06:03:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 -- 172.21.15.32:0/1940527948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ac158520 msgr2=0x7f43ac1400e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ac158520 0x7f43ac1400e0 secure :-1 s=READY pgs=2165 cs=0 l=1 rev1=1 crypto rx=0x7f43a400efc0 tx=0x7f43a400c490 comp rx=0 tx=0).stop 2023-12-20T06:03:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 -- 172.21.15.32:0/1940527948 shutdown_connections 2023-12-20T06:03:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4390065cd0 0x7f4390068190 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43ac158520 0x7f43ac1400e0 unknown :-1 s=CLOSED pgs=2165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43ac070230 0x7f43ac155b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 --2- 172.21.15.32:0/1940527948 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43ac06dcb0 0x7f43ac157410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 -- 172.21.15.32:0/1940527948 >> 172.21.15.32:0/1940527948 conn(0x7f43ac06a510 msgr2=0x7f43ac0c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:05.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 -- 172.21.15.32:0/1940527948 shutdown_connections 2023-12-20T06:03:05.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.691+0000 7f43b2d7b640 1 -- 172.21.15.32:0/1940527948 wait complete. 2023-12-20T06:03:05.718 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T06:03:05.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.871+0000 7fca7c1f3640 1 Processor -- start 2023-12-20T06:03:05.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.871+0000 7fca7c1f3640 1 -- start start 2023-12-20T06:03:05.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.871+0000 7fca7c1f3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fca74159250 0x7fca740caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.871+0000 7fca7c1f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca740ce290 0x7fca74070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.871+0000 7fca7c1f3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca74159630 con 0x7fca74070770 2023-12-20T06:03:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.871+0000 7fca7c1f3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca74070c00 con 0x7fca74159250 2023-12-20T06:03:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.871+0000 7fca7c1f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca740c9480 con 0x7fca740ce290 2023-12-20T06:03:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.871+0000 7fca79f8f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fca74159250 0x7fca740caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.871+0000 7fca79f8f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fca74159250 0x7fca740caa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:32878/0 (socket says 172.21.15.32:32878) 2023-12-20T06:03:05.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.871+0000 7fca79f8f640 1 -- 172.21.15.32:0/579235347 learned_addr learned my addr 172.21.15.32:0/579235347 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.871+0000 7fca7978e640 1 --2- 172.21.15.32:0/579235347 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca740ce290 0x7fca74070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca79f8f640 1 -- 172.21.15.32:0/579235347 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca740ce290 msgr2=0x7fca74070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca79f8f640 1 --2- 172.21.15.32:0/579235347 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca740ce290 0x7fca74070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca79f8f640 1 -- 172.21.15.32:0/579235347 >> v1:172.21.15.39:6789/0 conn(0x7fca74070770 legacy=0x7fca7415cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca78f8d640 1 -- 172.21.15.32:0/579235347 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1198672477 0 0) 0x7fca74159630 con 0x7fca74070770 2023-12-20T06:03:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca79f8f640 1 -- 172.21.15.32:0/579235347 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca7406cf90 con 0x7fca74159250 2023-12-20T06:03:05.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca79f8f640 1 --2- 172.21.15.32:0/579235347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca74159250 0x7fca740caa30 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fca70009870 tx=0x7fca7001c510 comp rx=0 tx=0).ready entity=mon.1 client_cookie=2a32739403bad35d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca78f8d640 1 -- 172.21.15.32:0/579235347 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca700ad680 con 0x7fca74159250 2023-12-20T06:03:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca78f8d640 1 -- 172.21.15.32:0/579235347 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca700ad7e0 con 0x7fca74159250 2023-12-20T06:03:05.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca78f8d640 1 -- 172.21.15.32:0/579235347 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca700adb70 con 0x7fca74159250 2023-12-20T06:03:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca7c1f3640 1 -- 172.21.15.32:0/579235347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca74159250 msgr2=0x7fca740caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca7c1f3640 1 --2- 172.21.15.32:0/579235347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca74159250 0x7fca740caa30 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fca70009870 tx=0x7fca7001c510 comp rx=0 tx=0).stop 2023-12-20T06:03:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca7c1f3640 1 -- 172.21.15.32:0/579235347 shutdown_connections 2023-12-20T06:03:05.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca7c1f3640 1 --2- 172.21.15.32:0/579235347 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca740ce290 0x7fca74070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca7c1f3640 1 --2- 172.21.15.32:0/579235347 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca74159250 0x7fca740caa30 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.875+0000 7fca7c1f3640 1 -- 172.21.15.32:0/579235347 >> 172.21.15.32:0/579235347 conn(0x7fca7406a510 msgr2=0x7fca740ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7c1f3640 1 -- 172.21.15.32:0/579235347 shutdown_connections 2023-12-20T06:03:05.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7c1f3640 1 -- 172.21.15.32:0/579235347 wait complete. 2023-12-20T06:03:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7c1f3640 1 Processor -- start 2023-12-20T06:03:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7c1f3640 1 -- start start 2023-12-20T06:03:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7c1f3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca74070770 0x7fca740b3a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7c1f3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca740ce290 0x7fca740c7520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7c1f3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca74159250 0x7fca74157860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7c1f3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca74150540 con 0x7fca74159250 2023-12-20T06:03:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7c1f3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca740b3f60 con 0x7fca74070770 2023-12-20T06:03:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7c1f3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca741507e0 con 0x7fca740ce290 2023-12-20T06:03:05.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7a790640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca74159250 0x7fca74157860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7a790640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca74159250 0x7fca74157860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33910/0 (socket says 172.21.15.32:33910) 2023-12-20T06:03:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7a790640 1 -- 172.21.15.32:0/2520754296 learned_addr learned my addr 172.21.15.32:0/2520754296 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7978e640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca740ce290 0x7fca740c7520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca79f8f640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca74070770 0x7fca740b3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7a790640 1 -- 172.21.15.32:0/2520754296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca740ce290 msgr2=0x7fca740c7520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7a790640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca740ce290 0x7fca740c7520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7a790640 1 -- 172.21.15.32:0/2520754296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca74070770 msgr2=0x7fca740b3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7a790640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca74070770 0x7fca740b3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7a790640 1 -- 172.21.15.32:0/2520754296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca700094d0 con 0x7fca74159250 2023-12-20T06:03:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca79f8f640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca74070770 0x7fca740b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7978e640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca740ce290 0x7fca740c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7a790640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca74159250 0x7fca74157860 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00c7e0 tx=0x7fca6c00cca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca6a7fc640 1 -- 172.21.15.32:0/2520754296 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c003db0 con 0x7fca74159250 2023-12-20T06:03:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca6a7fc640 1 -- 172.21.15.32:0/2520754296 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca6c005070 con 0x7fca74159250 2023-12-20T06:03:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca6a7fc640 1 -- 172.21.15.32:0/2520754296 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca6c005420 con 0x7fca74159250 2023-12-20T06:03:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.879+0000 7fca7c1f3640 1 -- 172.21.15.32:0/2520754296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca740c8000 con 0x7fca74159250 2023-12-20T06:03:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.883+0000 7fca7c1f3640 1 -- 172.21.15.32:0/2520754296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca740c85c0 con 0x7fca74159250 2023-12-20T06:03:05.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.883+0000 7fca7c1f3640 1 -- 172.21.15.32:0/2520754296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca740c87e0 con 0x7fca74159250 2023-12-20T06:03:05.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.883+0000 7fca6a7fc640 1 -- 172.21.15.32:0/2520754296 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fca6c005580 con 0x7fca74159250 2023-12-20T06:03:05.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.883+0000 7fca6a7fc640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca58065ca0 0x7fca58068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:05.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.883+0000 7fca6a7fc640 1 -- 172.21.15.32:0/2520754296 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(217..217 src has 1..217) v4 ==== 7008+0+0 (secure 0 0 0) 0x7fca6c083940 con 0x7fca74159250 2023-12-20T06:03:05.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.883+0000 7fca79f8f640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca58065ca0 0x7fca58068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:05.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.887+0000 7fca79f8f640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca58065ca0 0x7fca58068160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fca700af000 tx=0x7fca700a7000 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:05.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:05.887+0000 7fca6a7fc640 1 -- 172.21.15.32:0/2520754296 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fca6c050a10 con 0x7fca74159250 2023-12-20T06:03:06.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.047+0000 7fca7c1f3640 1 -- 172.21.15.32:0/2520754296 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fca7404e000 con 0x7fca74159250 2023-12-20T06:03:06.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.691+0000 7fca6a7fc640 1 -- 172.21.15.32:0/2520754296 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v218) v1 ==== 169+0+0 (secure 0 0 0) 0x7fca6c054030 con 0x7fca74159250 2023-12-20T06:03:06.691 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T06:03:06.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 -- 172.21.15.32:0/2520754296 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca58065ca0 msgr2=0x7fca58068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:06.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca58065ca0 0x7fca58068160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7fca700af000 tx=0x7fca700a7000 comp rx=0 tx=0).stop 2023-12-20T06:03:06.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 -- 172.21.15.32:0/2520754296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca74159250 msgr2=0x7fca74157860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:06.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca74159250 0x7fca74157860 secure :-1 s=READY pgs=2166 cs=0 l=1 rev1=1 crypto rx=0x7fca6c00c7e0 tx=0x7fca6c00cca0 comp rx=0 tx=0).stop 2023-12-20T06:03:06.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 -- 172.21.15.32:0/2520754296 shutdown_connections 2023-12-20T06:03:06.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca58065ca0 0x7fca58068160 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:06.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca74159250 0x7fca74157860 unknown :-1 s=CLOSED pgs=2166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:06.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca740ce290 0x7fca740c7520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:06.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 --2- 172.21.15.32:0/2520754296 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca74070770 0x7fca740b3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:06.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 -- 172.21.15.32:0/2520754296 >> 172.21.15.32:0/2520754296 conn(0x7fca7406a510 msgr2=0x7fca7415ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:06.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 -- 172.21.15.32:0/2520754296 shutdown_connections 2023-12-20T06:03:06.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.695+0000 7fca7c1f3640 1 -- 172.21.15.32:0/2520754296 wait complete. 2023-12-20T06:03:06.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.867+0000 7fc8d94f9640 1 Processor -- start 2023-12-20T06:03:06.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.867+0000 7fc8d94f9640 1 -- start start 2023-12-20T06:03:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d94f9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8d41593c0 0x7fc8d415d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d94f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8d406caf0 0x7fc8d40705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d94f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8d40ae140 con 0x7fc8d40afdb0 2023-12-20T06:03:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d94f9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8d40ae9e0 con 0x7fc8d41593c0 2023-12-20T06:03:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d94f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d415dee0 con 0x7fc8d406caf0 2023-12-20T06:03:06.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8d41593c0 0x7fc8d415d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d2ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc8d41593c0 0x7fc8d415d9a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33922/0 (socket says 172.21.15.32:33922) 2023-12-20T06:03:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d2ffd640 1 -- 172.21.15.32:0/1816926170 learned_addr learned my addr 172.21.15.32:0/1816926170 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d2ffd640 1 -- 172.21.15.32:0/1816926170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8d406caf0 msgr2=0x7fc8d40705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d37fe640 1 --2- 172.21.15.32:0/1816926170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8d406caf0 0x7fc8d40705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d2ffd640 1 --2- 172.21.15.32:0/1816926170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8d406caf0 0x7fc8d40705a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d2ffd640 1 -- 172.21.15.32:0/1816926170 >> v1:172.21.15.32:6789/0 conn(0x7fc8d40afdb0 legacy=0x7fc8d406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d2ffd640 1 -- 172.21.15.32:0/1816926170 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8d4072170 con 0x7fc8d41593c0 2023-12-20T06:03:06.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d37fe640 1 --2- 172.21.15.32:0/1816926170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8d406caf0 0x7fc8d40705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d2ffd640 1 --2- 172.21.15.32:0/1816926170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d41593c0 0x7fc8d415d9a0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4006fd0 tx=0x7fc8c401c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53248fc23e23719b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:06.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d27fc640 1 -- 172.21.15.32:0/1816926170 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40ab600 con 0x7fc8d41593c0 2023-12-20T06:03:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d27fc640 1 -- 172.21.15.32:0/1816926170 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c40a7050 con 0x7fc8d41593c0 2023-12-20T06:03:06.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.871+0000 7fc8d27fc640 1 -- 172.21.15.32:0/1816926170 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c40ab980 con 0x7fc8d41593c0 2023-12-20T06:03:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1816926170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d41593c0 msgr2=0x7fc8d415d9a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 --2- 172.21.15.32:0/1816926170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d41593c0 0x7fc8d415d9a0 secure :-1 s=READY pgs=2168 cs=0 l=1 rev1=1 crypto rx=0x7fc8c4006fd0 tx=0x7fc8c401c460 comp rx=0 tx=0).stop 2023-12-20T06:03:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1816926170 shutdown_connections 2023-12-20T06:03:06.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 --2- 172.21.15.32:0/1816926170 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8d406caf0 0x7fc8d40705a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 --2- 172.21.15.32:0/1816926170 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d41593c0 0x7fc8d415d9a0 unknown :-1 s=CLOSED pgs=2168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1816926170 >> 172.21.15.32:0/1816926170 conn(0x7fc8d406a320 msgr2=0x7fc8d40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1816926170 shutdown_connections 2023-12-20T06:03:06.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1816926170 wait complete. 2023-12-20T06:03:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 Processor -- start 2023-12-20T06:03:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 -- start start 2023-12-20T06:03:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d406caf0 0x7fc8d40b3e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8d40afdb0 0x7fc8d40b1800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8d41593c0 0x7fc8d40b1d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d40b26f0 con 0x7fc8d406caf0 2023-12-20T06:03:06.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d4148180 con 0x7fc8d41593c0 2023-12-20T06:03:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d94f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8d415a900 con 0x7fc8d40afdb0 2023-12-20T06:03:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d406caf0 0x7fc8d40b3e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d406caf0 0x7fc8d40b3e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33936/0 (socket says 172.21.15.32:33936) 2023-12-20T06:03:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d2ffd640 1 -- 172.21.15.32:0/1092934213 learned_addr learned my addr 172.21.15.32:0/1092934213 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d2ffd640 1 -- 172.21.15.32:0/1092934213 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8d40afdb0 msgr2=0x7fc8d40b1800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d2ffd640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8d40afdb0 0x7fc8d40b1800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d2ffd640 1 -- 172.21.15.32:0/1092934213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8d41593c0 msgr2=0x7fc8d40b1d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:06.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d2ffd640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8d41593c0 0x7fc8d40b1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.875+0000 7fc8d2ffd640 1 -- 172.21.15.32:0/1092934213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8c4009480 con 0x7fc8d406caf0 2023-12-20T06:03:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.879+0000 7fc8d2ffd640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d406caf0 0x7fc8d40b3e20 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fc8c401a090 tx=0x7fc8c4019200 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.879+0000 7fc8cb7fe640 1 -- 172.21.15.32:0/1092934213 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4005e70 con 0x7fc8d406caf0 2023-12-20T06:03:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.879+0000 7fc8cb7fe640 1 -- 172.21.15.32:0/1092934213 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8c40037c0 con 0x7fc8d406caf0 2023-12-20T06:03:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.879+0000 7fc8cb7fe640 1 -- 172.21.15.32:0/1092934213 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8c4003b70 con 0x7fc8d406caf0 2023-12-20T06:03:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.879+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1092934213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8d4148890 con 0x7fc8d406caf0 2023-12-20T06:03:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.879+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1092934213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8d415ab10 con 0x7fc8d406caf0 2023-12-20T06:03:06.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.879+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1092934213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8d404e000 con 0x7fc8d406caf0 2023-12-20T06:03:06.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.879+0000 7fc8cb7fe640 1 -- 172.21.15.32:0/1092934213 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fc8c40b2070 con 0x7fc8d406caf0 2023-12-20T06:03:06.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.879+0000 7fc8cb7fe640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc8b4065ca0 0x7fc8b4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:06.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.879+0000 7fc8cb7fe640 1 -- 172.21.15.32:0/1092934213 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fc8c411cd30 con 0x7fc8d406caf0 2023-12-20T06:03:06.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.879+0000 7fc8cbfff640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc8b4065ca0 0x7fc8b4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:06.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.883+0000 7fc8cbfff640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc8b4065ca0 0x7fc8b4068160 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc000f70 tx=0x7fc8bc0079c0 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:06.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:06.883+0000 7fc8cb7fe640 1 -- 172.21.15.32:0/1092934213 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc8c40e9f30 con 0x7fc8d406caf0 2023-12-20T06:03:07.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.051+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1092934213 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fc8d41550e0 con 0x7fc8d406caf0 2023-12-20T06:03:07.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.055+0000 7fc8cb7fe640 1 -- 172.21.15.32:0/1092934213 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v218) v1 ==== 74+0+17344 (secure 0 0 0) 0x7fc8c40ed550 con 0x7fc8d406caf0 2023-12-20T06:03:07.056 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:07.056 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":218,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:06.687492+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:02:37.109832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:02:38.440830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/888710554":"2023-12-21T06:03:03.292086+0000","172.21.15.32:0/3767986898":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/2893447374":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/4287660207":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1968262556":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1215843411":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1623434834":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/3650825501":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/2603552779":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/587127774":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3681500114":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/1096471740":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6836/888710554":"2023-12-21T06:03:03.292086+0000","172.21.15.32:0/922716312":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1437478066":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3373411185":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6833/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6832/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/4110141711":"2023-12-21T06:02:50.560868+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1092934213 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc8b4065ca0 msgr2=0x7fc8b4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc8b4065ca0 0x7fc8b4068160 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7fc8bc000f70 tx=0x7fc8bc0079c0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1092934213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d406caf0 msgr2=0x7fc8d40b3e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d406caf0 0x7fc8d40b3e20 secure :-1 s=READY pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fc8c401a090 tx=0x7fc8c4019200 comp rx=0 tx=0).stop 2023-12-20T06:03:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1092934213 shutdown_connections 2023-12-20T06:03:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc8b4065ca0 0x7fc8b4068160 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8d41593c0 0x7fc8d40b1d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8d40afdb0 0x7fc8d40b1800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 --2- 172.21.15.32:0/1092934213 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8d406caf0 0x7fc8d40b3e20 secure :-1 s=CLOSED pgs=2169 cs=0 l=1 rev1=1 crypto rx=0x7fc8c401a090 tx=0x7fc8c4019200 comp rx=0 tx=0).stop 2023-12-20T06:03:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1092934213 >> 172.21.15.32:0/1092934213 conn(0x7fc8d406a320 msgr2=0x7fc8d40cde40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1092934213 shutdown_connections 2023-12-20T06:03:07.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.059+0000 7fc8d94f9640 1 -- 172.21.15.32:0/1092934213 wait complete. 2023-12-20T06:03:07.080 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 26, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-20T06:02:37.109832+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '212', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 27, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-20T06:02:38.440830+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '213', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:03:07.080 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:03:07.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.227+0000 7f675c525640 1 Processor -- start 2023-12-20T06:03:07.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.227+0000 7f675c525640 1 -- start start 2023-12-20T06:03:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.227+0000 7f675c525640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67540b0bc0 0x7f67540ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.227+0000 7f675c525640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67540cde20 con 0x7f67540b0760 2023-12-20T06:03:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.227+0000 7f675c525640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67540c8e30 con 0x7f6754157480 2023-12-20T06:03:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.227+0000 7f675c525640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67540c9f50 con 0x7f67540b0bc0 2023-12-20T06:03:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.227+0000 7f6759ac0640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f67540b0760 0x7f67540cc4b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43912/0 (socket says 172.21.15.32:43912) 2023-12-20T06:03:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.227+0000 7f6759ac0640 1 -- 172.21.15.32:0/2413181096 learned_addr learned my addr 172.21.15.32:0/2413181096 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:07.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f675aac2640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67540b0bc0 0x7f67540ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3839526328 0 0) 0x7f67540cde20 con 0x7f67540b0760 2023-12-20T06:03:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6740003530 con 0x7f67540b0760 2023-12-20T06:03:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 750821013 0 0) 0x7f67540c8e30 con 0x7f6754157480 2023-12-20T06:03:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f67540cde20 con 0x7f6754157480 2023-12-20T06:03:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2829896738 0 0) 0x7f6740003530 con 0x7f67540b0760 2023-12-20T06:03:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67540b0bc0 msgr2=0x7f67540ccbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67540b0bc0 0x7f67540ccbc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 >> v1:172.21.15.39:6789/0 conn(0x7f6754157480 legacy=0x7f67540c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67540ce070 con 0x7f67540b0760 2023-12-20T06:03:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f6744001680 con 0x7f67540b0760 2023-12-20T06:03:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f675aac2640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67540b0bc0 0x7f67540ccbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 >> v1:172.21.15.32:6789/0 conn(0x7f67540b0760 legacy=0x7f67540cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6740003b40 0x7f6740003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67540b0bc0 0x7f67400047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6740004d10 0x7f6740005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540c9f50 con 0x7f6740003b40 2023-12-20T06:03:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f674000baa0 con 0x7f6740004d10 2023-12-20T06:03:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f674000bc60 con 0x7f67540b0bc0 2023-12-20T06:03:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f675a2c1640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6740003b40 0x7f6740003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f6759ac0640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67540b0bc0 0x7f67400047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f675aac2640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6740004d10 0x7f6740005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f675a2c1640 1 -- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67540b0bc0 msgr2=0x7f67400047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f675a2c1640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67540b0bc0 0x7f67400047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f675a2c1640 1 -- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6740004d10 msgr2=0x7f6740005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:07.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f675a2c1640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6740004d10 0x7f6740005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.231+0000 7f675a2c1640 1 -- 172.21.15.32:0/2413181096 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f674c009a70 con 0x7f6740003b40 2023-12-20T06:03:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f675a2c1640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6740003b40 0x7f6740003f40 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f67500018d0 tx=0x7f675001c530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=598da11edc64d26b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:07.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67500052d0 con 0x7f6740003b40 2023-12-20T06:03:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6750005450 con 0x7f6740003b40 2023-12-20T06:03:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f67592bf640 1 -- 172.21.15.32:0/2413181096 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6750003840 con 0x7f6740003b40 2023-12-20T06:03:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f675c525640 1 -- 172.21.15.32:0/2413181096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6740003b40 msgr2=0x7f6740003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f675c525640 1 --2- 172.21.15.32:0/2413181096 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6740003b40 0x7f6740003f40 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f67500018d0 tx=0x7f675001c530 comp rx=0 tx=0).stop 2023-12-20T06:03:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f675aac2640 1 -- 172.21.15.32:0/2413181096 reap_dead start 2023-12-20T06:03:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f675c525640 1 -- 172.21.15.32:0/2413181096 shutdown_connections 2023-12-20T06:03:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f675c525640 1 -- 172.21.15.32:0/2413181096 >> 172.21.15.32:0/2413181096 conn(0x7f675406a510 msgr2=0x7f67540ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f675c525640 1 -- 172.21.15.32:0/2413181096 shutdown_connections 2023-12-20T06:03:07.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f675c525640 1 -- 172.21.15.32:0/2413181096 wait complete. 2023-12-20T06:03:07.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f675c525640 1 Processor -- start 2023-12-20T06:03:07.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.235+0000 7f675c525640 1 -- start start 2023-12-20T06:03:07.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675c525640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6740003b40 0x7f6754152fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675c525640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67540b1b50 0x7f6754149650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675c525640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67540b1fd0 0x7f6754149b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675c525640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67540c6090 con 0x7f67540b1fd0 2023-12-20T06:03:07.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675c525640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f675414a0d0 con 0x7f67540b1b50 2023-12-20T06:03:07.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675c525640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6754152d80 con 0x7f6740003b40 2023-12-20T06:03:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675aac2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67540b1fd0 0x7f6754149b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675aac2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67540b1fd0 0x7f6754149b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33952/0 (socket says 172.21.15.32:33952) 2023-12-20T06:03:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675aac2640 1 -- 172.21.15.32:0/628974367 learned_addr learned my addr 172.21.15.32:0/628974367 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675a2c1640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6740003b40 0x7f6754152fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f6759ac0640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67540b1b50 0x7f6754149650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675aac2640 1 -- 172.21.15.32:0/628974367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6740003b40 msgr2=0x7f6754152fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675aac2640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6740003b40 0x7f6754152fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675aac2640 1 -- 172.21.15.32:0/628974367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67540b1b50 msgr2=0x7f6754149650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675aac2640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67540b1b50 0x7f6754149650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675aac2640 1 -- 172.21.15.32:0/628974367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6754152630 con 0x7f67540b1fd0 2023-12-20T06:03:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675aac2640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67540b1fd0 0x7f6754149b90 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f67540c8dc0 tx=0x7f674c00f580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f674affd640 1 -- 172.21.15.32:0/628974367 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c00e040 con 0x7f67540b1fd0 2023-12-20T06:03:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f674affd640 1 -- 172.21.15.32:0/628974367 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f674c00f040 con 0x7f67540b1fd0 2023-12-20T06:03:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f674affd640 1 -- 172.21.15.32:0/628974367 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f674c003bc0 con 0x7f67540b1fd0 2023-12-20T06:03:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675c525640 1 -- 172.21.15.32:0/628974367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67540c6270 con 0x7f67540b1fd0 2023-12-20T06:03:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675c525640 1 -- 172.21.15.32:0/628974367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67540c6830 con 0x7f67540b1fd0 2023-12-20T06:03:07.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f675c525640 1 -- 172.21.15.32:0/628974367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67540c6a20 con 0x7f67540b1fd0 2023-12-20T06:03:07.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.239+0000 7f674affd640 1 -- 172.21.15.32:0/628974367 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f674c015430 con 0x7f67540b1fd0 2023-12-20T06:03:07.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.243+0000 7f674affd640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6738065cd0 0x7f6738068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.243+0000 7f674affd640 1 -- 172.21.15.32:0/628974367 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f674c084310 con 0x7f67540b1fd0 2023-12-20T06:03:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.243+0000 7f675a2c1640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6738065cd0 0x7f6738068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.243+0000 7f675a2c1640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6738065cd0 0x7f6738068190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f67500ad000 tx=0x7f67500a7000 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:07.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.247+0000 7f674affd640 1 -- 172.21.15.32:0/628974367 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f674c051580 con 0x7f67540b1fd0 2023-12-20T06:03:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.403+0000 7f675c525640 1 -- 172.21.15.32:0/628974367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f675404e000 con 0x7f67540b1fd0 2023-12-20T06:03:07.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f674affd640 1 -- 172.21.15.32:0/628974367 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v218) v1 ==== 93+0+10 (secure 0 0 0) 0x7f674c054ba0 con 0x7f67540b1fd0 2023-12-20T06:03:07.405 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:03:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 -- 172.21.15.32:0/628974367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6738065cd0 msgr2=0x7f6738068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6738065cd0 0x7f6738068190 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f67500ad000 tx=0x7f67500a7000 comp rx=0 tx=0).stop 2023-12-20T06:03:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 -- 172.21.15.32:0/628974367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67540b1fd0 msgr2=0x7f6754149b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67540b1fd0 0x7f6754149b90 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f67540c8dc0 tx=0x7f674c00f580 comp rx=0 tx=0).stop 2023-12-20T06:03:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 -- 172.21.15.32:0/628974367 shutdown_connections 2023-12-20T06:03:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6738065cd0 0x7f6738068190 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67540b1fd0 0x7f6754149b90 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67540b1b50 0x7f6754149650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 --2- 172.21.15.32:0/628974367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6740003b40 0x7f6754152fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 -- 172.21.15.32:0/628974367 >> 172.21.15.32:0/628974367 conn(0x7f675406a510 msgr2=0x7f6754070990 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 -- 172.21.15.32:0/628974367 shutdown_connections 2023-12-20T06:03:07.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.407+0000 7f675c525640 1 -- 172.21.15.32:0/628974367 wait complete. 2023-12-20T06:03:07.426 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-20T06:03:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05bcd96640 1 Processor -- start 2023-12-20T06:03:07.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05bcd96640 1 -- start start 2023-12-20T06:03:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05bcd96640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f05b815abb0 0x7f05b80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05bcd96640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b815b6a0 con 0x7f05b8070230 2023-12-20T06:03:07.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05bcd96640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f05b806d8c0 con 0x7f05b80b0cb0 2023-12-20T06:03:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05bcd96640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f05b806e160 con 0x7f05b815abb0 2023-12-20T06:03:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05b6d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f05b8070230 0x7f05b806c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43920/0 (socket says 172.21.15.32:43920) 2023-12-20T06:03:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05b6d9d640 1 -- 172.21.15.32:0/648855899 learned_addr learned my addr 172.21.15.32:0/648855899 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05b659c640 1 --2- 172.21.15.32:0/648855899 >> v2:172.21.15.39:3300/0 conn(0x7f05b815abb0 0x7f05b80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05b5d9b640 1 -- 172.21.15.32:0/648855899 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3641014547 0 0) 0x7f05b815b6a0 con 0x7f05b8070230 2023-12-20T06:03:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05b5d9b640 1 -- 172.21.15.32:0/648855899 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f05a4003530 con 0x7f05b8070230 2023-12-20T06:03:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05b659c640 1 -- 172.21.15.32:0/648855899 >> v1:172.21.15.39:6789/0 conn(0x7f05b80b0cb0 legacy=0x7f05b815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05b5d9b640 1 -- 172.21.15.32:0/648855899 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2619342985 0 0) 0x7f05b806d8c0 con 0x7f05b80b0cb0 2023-12-20T06:03:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05b659c640 1 -- 172.21.15.32:0/648855899 >> v1:172.21.15.32:6789/0 conn(0x7f05b8070230 legacy=0x7f05b806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.575+0000 7f05b659c640 1 -- 172.21.15.32:0/648855899 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05b806dba0 con 0x7f05b815abb0 2023-12-20T06:03:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.579+0000 7f05b659c640 1 --2- 172.21.15.32:0/648855899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b815abb0 0x7f05b80caaf0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f05a80097c0 tx=0x7f05a801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e4dbf273b4a8ecc6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.579+0000 7f05b5d9b640 1 -- 172.21.15.32:0/648855899 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80ab680 con 0x7f05b815abb0 2023-12-20T06:03:07.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.579+0000 7f05b5d9b640 1 -- 172.21.15.32:0/648855899 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05a80ab7e0 con 0x7f05b815abb0 2023-12-20T06:03:07.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.579+0000 7f05b5d9b640 1 -- 172.21.15.32:0/648855899 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05a80abb70 con 0x7f05b815abb0 2023-12-20T06:03:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.579+0000 7f05bcd96640 1 -- 172.21.15.32:0/648855899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b815abb0 msgr2=0x7f05b80caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.579+0000 7f05bcd96640 1 --2- 172.21.15.32:0/648855899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b815abb0 0x7f05b80caaf0 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7f05a80097c0 tx=0x7f05a801c480 comp rx=0 tx=0).stop 2023-12-20T06:03:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.579+0000 7f05bcd96640 1 -- 172.21.15.32:0/648855899 shutdown_connections 2023-12-20T06:03:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.579+0000 7f05bcd96640 1 --2- 172.21.15.32:0/648855899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b815abb0 0x7f05b80caaf0 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.579+0000 7f05bcd96640 1 -- 172.21.15.32:0/648855899 >> 172.21.15.32:0/648855899 conn(0x7f05b806a510 msgr2=0x7f05b80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.579+0000 7f05bcd96640 1 -- 172.21.15.32:0/648855899 shutdown_connections 2023-12-20T06:03:07.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.579+0000 7f05bcd96640 1 -- 172.21.15.32:0/648855899 wait complete. 2023-12-20T06:03:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05bcd96640 1 Processor -- start 2023-12-20T06:03:07.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05bcd96640 1 -- start start 2023-12-20T06:03:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05bcd96640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05b8070230 0x7f05b8158100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05bcd96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b80b0cb0 0x7f05b8158da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05bcd96640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b815abb0 0x7f05b8157890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05bcd96640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81506f0 con 0x7f05b815abb0 2023-12-20T06:03:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05bcd96640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81f90f0 con 0x7f05b80b0cb0 2023-12-20T06:03:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05bcd96640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f05b81f93d0 con 0x7f05b8070230 2023-12-20T06:03:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05b6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b815abb0 0x7f05b8157890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05b6d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b815abb0 0x7f05b8157890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33954/0 (socket says 172.21.15.32:33954) 2023-12-20T06:03:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05b6d9d640 1 -- 172.21.15.32:0/4008144270 learned_addr learned my addr 172.21.15.32:0/4008144270 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05b659c640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05b8070230 0x7f05b8158100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05affff640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b80b0cb0 0x7f05b8158da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05b6d9d640 1 -- 172.21.15.32:0/4008144270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05b8070230 msgr2=0x7f05b8158100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05b6d9d640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05b8070230 0x7f05b8158100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05b6d9d640 1 -- 172.21.15.32:0/4008144270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b80b0cb0 msgr2=0x7f05b8158da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05b6d9d640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b80b0cb0 0x7f05b8158da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05b6d9d640 1 -- 172.21.15.32:0/4008144270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f05a8009420 con 0x7f05b815abb0 2023-12-20T06:03:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05b6d9d640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b815abb0 0x7f05b8157890 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f05b000cb40 tx=0x7f05b000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05aeffd640 1 -- 172.21.15.32:0/4008144270 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b000aca0 con 0x7f05b815abb0 2023-12-20T06:03:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05aeffd640 1 -- 172.21.15.32:0/4008144270 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f05b000ae00 con 0x7f05b815abb0 2023-12-20T06:03:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05bcd96640 1 -- 172.21.15.32:0/4008144270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05b81f96b0 con 0x7f05b815abb0 2023-12-20T06:03:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05aeffd640 1 -- 172.21.15.32:0/4008144270 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05b0012620 con 0x7f05b815abb0 2023-12-20T06:03:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05bcd96640 1 -- 172.21.15.32:0/4008144270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f05b81592e0 con 0x7f05b815abb0 2023-12-20T06:03:07.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.583+0000 7f05bcd96640 1 -- 172.21.15.32:0/4008144270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f05b8150860 con 0x7f05b815abb0 2023-12-20T06:03:07.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.587+0000 7f05aeffd640 1 -- 172.21.15.32:0/4008144270 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f05b0007070 con 0x7f05b815abb0 2023-12-20T06:03:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.587+0000 7f05aeffd640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0598065ca0 0x7f0598068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.587+0000 7f05aeffd640 1 -- 172.21.15.32:0/4008144270 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f05b00838f0 con 0x7f05b815abb0 2023-12-20T06:03:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.587+0000 7f05b659c640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0598065ca0 0x7f0598068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.587+0000 7f05b659c640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0598065ca0 0x7f0598068160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f05a80af000 tx=0x7f05a80a7000 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:07.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.591+0000 7f05aeffd640 1 -- 172.21.15.32:0/4008144270 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f05b0050c70 con 0x7f05b815abb0 2023-12-20T06:03:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.747+0000 7f05bcd96640 1 -- 172.21.15.32:0/4008144270 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f05b804e000 con 0x7f05b815abb0 2023-12-20T06:03:07.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05aeffd640 1 -- 172.21.15.32:0/4008144270 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v218) v1 ==== 107+0+11 (secure 0 0 0) 0x7f05b00170b0 con 0x7f05b815abb0 2023-12-20T06:03:07.749 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:03:07.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 -- 172.21.15.32:0/4008144270 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0598065ca0 msgr2=0x7f0598068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0598065ca0 0x7f0598068160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f05a80af000 tx=0x7f05a80a7000 comp rx=0 tx=0).stop 2023-12-20T06:03:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 -- 172.21.15.32:0/4008144270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b815abb0 msgr2=0x7f05b8157890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b815abb0 0x7f05b8157890 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f05b000cb40 tx=0x7f05b000a060 comp rx=0 tx=0).stop 2023-12-20T06:03:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 -- 172.21.15.32:0/4008144270 shutdown_connections 2023-12-20T06:03:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0598065ca0 0x7f0598068160 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f05b815abb0 0x7f05b8157890 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f05b80b0cb0 0x7f05b8158da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 --2- 172.21.15.32:0/4008144270 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05b8070230 0x7f05b8158100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 -- 172.21.15.32:0/4008144270 >> 172.21.15.32:0/4008144270 conn(0x7f05b806a510 msgr2=0x7f05b8152820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 -- 172.21.15.32:0/4008144270 shutdown_connections 2023-12-20T06:03:07.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.751+0000 7f05bcd96640 1 -- 172.21.15.32:0/4008144270 wait complete. 2023-12-20T06:03:07.771 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-20T06:03:07.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.919+0000 7f4afd466640 1 Processor -- start 2023-12-20T06:03:07.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.919+0000 7f4afd466640 1 -- start start 2023-12-20T06:03:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.919+0000 7f4afd466640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4af815abb0 0x7f4af80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.919+0000 7f4afd466640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af815b6a0 con 0x7f4af80b0cb0 2023-12-20T06:03:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.919+0000 7f4afd466640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4af806d8c0 con 0x7f4af8070230 2023-12-20T06:03:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.919+0000 7f4afd466640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4af806e160 con 0x7f4af815abb0 2023-12-20T06:03:07.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.919+0000 7f4af6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4af815abb0 0x7f4af80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.919+0000 7f4af6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4af815abb0 0x7f4af80caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33970/0 (socket says 172.21.15.32:33970) 2023-12-20T06:03:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.919+0000 7f4af6ffd640 1 -- 172.21.15.32:0/1811121105 learned_addr learned my addr 172.21.15.32:0/1811121105 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.919+0000 7f4aeffff640 1 --1- 172.21.15.32:0/1811121105 >> v1:172.21.15.32:6789/0 conn(0x7f4af80b0cb0 0x7f4af815bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:43934/0 (socket says 172.21.15.32:43934) 2023-12-20T06:03:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.923+0000 7f4af6ffd640 1 -- 172.21.15.32:0/1811121105 >> v1:172.21.15.39:6789/0 conn(0x7f4af8070230 legacy=0x7f4af806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.923+0000 7f4af6ffd640 1 -- 172.21.15.32:0/1811121105 >> v1:172.21.15.32:6789/0 conn(0x7f4af80b0cb0 legacy=0x7f4af815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.923+0000 7f4af6ffd640 1 -- 172.21.15.32:0/1811121105 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4af806dba0 con 0x7f4af815abb0 2023-12-20T06:03:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.923+0000 7f4af6ffd640 1 --2- 172.21.15.32:0/1811121105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af815abb0 0x7f4af80caaf0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009870 tx=0x7f4ae801c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0ffffffaa02811a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.923+0000 7f4af67fc640 1 -- 172.21.15.32:0/1811121105 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80a9400 con 0x7f4af815abb0 2023-12-20T06:03:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.923+0000 7f4af67fc640 1 -- 172.21.15.32:0/1811121105 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4ae80a9560 con 0x7f4af815abb0 2023-12-20T06:03:07.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.923+0000 7f4af67fc640 1 -- 172.21.15.32:0/1811121105 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4ae80a98f0 con 0x7f4af815abb0 2023-12-20T06:03:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 -- 172.21.15.32:0/1811121105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af815abb0 msgr2=0x7f4af80caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 --2- 172.21.15.32:0/1811121105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af815abb0 0x7f4af80caaf0 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f4ae8009870 tx=0x7f4ae801c540 comp rx=0 tx=0).stop 2023-12-20T06:03:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 -- 172.21.15.32:0/1811121105 shutdown_connections 2023-12-20T06:03:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 --2- 172.21.15.32:0/1811121105 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af815abb0 0x7f4af80caaf0 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 -- 172.21.15.32:0/1811121105 >> 172.21.15.32:0/1811121105 conn(0x7f4af806a510 msgr2=0x7f4af80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 -- 172.21.15.32:0/1811121105 shutdown_connections 2023-12-20T06:03:07.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 -- 172.21.15.32:0/1811121105 wait complete. 2023-12-20T06:03:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 Processor -- start 2023-12-20T06:03:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 -- start start 2023-12-20T06:03:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4af8070230 0x7f4af814d090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af80b0cb0 0x7f4af814d7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af815abb0 0x7f4af814dd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af815a210 con 0x7f4af815abb0 2023-12-20T06:03:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4af814d5d0 con 0x7f4af80b0cb0 2023-12-20T06:03:07.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4afd466640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4af8151bb0 con 0x7f4af8070230 2023-12-20T06:03:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4af77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af815abb0 0x7f4af814dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4af77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af815abb0 0x7f4af814dd00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33980/0 (socket says 172.21.15.32:33980) 2023-12-20T06:03:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4af77fe640 1 -- 172.21.15.32:0/1741466728 learned_addr learned my addr 172.21.15.32:0/1741466728 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4aeffff640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af80b0cb0 0x7f4af814d7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4af6ffd640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4af8070230 0x7f4af814d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4af77fe640 1 -- 172.21.15.32:0/1741466728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4af8070230 msgr2=0x7f4af814d090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4af77fe640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4af8070230 0x7f4af814d090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4af77fe640 1 -- 172.21.15.32:0/1741466728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af80b0cb0 msgr2=0x7f4af814d7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:07.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4af77fe640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af80b0cb0 0x7f4af814d7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:07.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4af77fe640 1 -- 172.21.15.32:0/1741466728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4ae80094d0 con 0x7f4af815abb0 2023-12-20T06:03:07.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.927+0000 7f4af77fe640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af815abb0 0x7f4af814dd00 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f4af000c7a0 tx=0x7f4af000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:07.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.931+0000 7f4aef7fe640 1 -- 172.21.15.32:0/1741466728 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af0003b80 con 0x7f4af815abb0 2023-12-20T06:03:07.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.931+0000 7f4aef7fe640 1 -- 172.21.15.32:0/1741466728 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4af0003ce0 con 0x7f4af815abb0 2023-12-20T06:03:07.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.931+0000 7f4afd466640 1 -- 172.21.15.32:0/1741466728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4af81518d0 con 0x7f4af815abb0 2023-12-20T06:03:07.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.931+0000 7f4aef7fe640 1 -- 172.21.15.32:0/1741466728 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4af000aa90 con 0x7f4af815abb0 2023-12-20T06:03:07.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.931+0000 7f4afd466640 1 -- 172.21.15.32:0/1741466728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4af80cdb20 con 0x7f4af815abb0 2023-12-20T06:03:07.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.931+0000 7f4afd466640 1 -- 172.21.15.32:0/1741466728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4af804e000 con 0x7f4af815abb0 2023-12-20T06:03:07.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.931+0000 7f4aef7fe640 1 -- 172.21.15.32:0/1741466728 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f4af000abf0 con 0x7f4af815abb0 2023-12-20T06:03:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.931+0000 7f4aef7fe640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4ad8065ca0 0x7f4ad8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.931+0000 7f4aef7fe640 1 -- 172.21.15.32:0/1741466728 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f4af0083650 con 0x7f4af815abb0 2023-12-20T06:03:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.931+0000 7f4af6ffd640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4ad8065ca0 0x7f4ad8068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:07.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.931+0000 7f4af6ffd640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4ad8065ca0 0x7f4ad8068160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80ae000 tx=0x7f4ae8008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:07.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:07.935+0000 7f4aef7fe640 1 -- 172.21.15.32:0/1741466728 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4af00509d0 con 0x7f4af815abb0 2023-12-20T06:03:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.095+0000 7f4afd466640 1 -- 172.21.15.32:0/1741466728 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f4af815a350 con 0x7f4af815abb0 2023-12-20T06:03:08.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.095+0000 7f4aef7fe640 1 -- 172.21.15.32:0/1741466728 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v218) v1 ==== 103+0+11 (secure 0 0 0) 0x7f4af0053ff0 con 0x7f4af815abb0 2023-12-20T06:03:08.095 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:03:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.095+0000 7f4afd466640 1 -- 172.21.15.32:0/1741466728 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4ad8065ca0 msgr2=0x7f4ad8068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.095+0000 7f4afd466640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4ad8065ca0 0x7f4ad8068160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f4ae80ae000 tx=0x7f4ae8008040 comp rx=0 tx=0).stop 2023-12-20T06:03:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.095+0000 7f4afd466640 1 -- 172.21.15.32:0/1741466728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af815abb0 msgr2=0x7f4af814dd00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.095+0000 7f4afd466640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af815abb0 0x7f4af814dd00 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7f4af000c7a0 tx=0x7f4af000cc60 comp rx=0 tx=0).stop 2023-12-20T06:03:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.099+0000 7f4afd466640 1 -- 172.21.15.32:0/1741466728 shutdown_connections 2023-12-20T06:03:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.099+0000 7f4afd466640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4ad8065ca0 0x7f4ad8068160 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.099+0000 7f4afd466640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4af815abb0 0x7f4af814dd00 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.099+0000 7f4afd466640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4af80b0cb0 0x7f4af814d7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.099+0000 7f4afd466640 1 --2- 172.21.15.32:0/1741466728 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4af8070230 0x7f4af814d090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.099+0000 7f4afd466640 1 -- 172.21.15.32:0/1741466728 >> 172.21.15.32:0/1741466728 conn(0x7f4af806a510 msgr2=0x7f4af80c44c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:08.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.099+0000 7f4afd466640 1 -- 172.21.15.32:0/1741466728 shutdown_connections 2023-12-20T06:03:08.099 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.099+0000 7f4afd466640 1 -- 172.21.15.32:0/1741466728 wait complete. 2023-12-20T06:03:08.117 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:08.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.259+0000 7f52ed0a7640 1 Processor -- start 2023-12-20T06:03:08.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ed0a7640 1 -- start start 2023-12-20T06:03:08.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ed0a7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52e80cd2e0 0x7f52e80ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ed0a7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f52e80b3e90 0x7f52e80ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ed0a7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f52e80ca0b0 con 0x7f52e8159ab0 2023-12-20T06:03:08.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ed0a7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f52e80ad7e0 con 0x7f52e80b3e90 2023-12-20T06:03:08.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ed0a7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e80ada80 con 0x7f52e80cd2e0 2023-12-20T06:03:08.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ec8a6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f52e80b3e90 0x7f52e80ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52e77fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52e80cd2e0 0x7f52e80ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ec8a6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f52e80b3e90 0x7f52e80ca820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:32942/0 (socket says 172.21.15.32:32942) 2023-12-20T06:03:08.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ec8a6640 1 -- 172.21.15.32:0/1434815716 learned_addr learned my addr 172.21.15.32:0/1434815716 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:08.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ec8a6640 1 -- 172.21.15.32:0/1434815716 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52e80cd2e0 msgr2=0x7f52e80ca2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:08.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ec8a6640 1 --2- 172.21.15.32:0/1434815716 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52e80cd2e0 0x7f52e80ca2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ec8a6640 1 -- 172.21.15.32:0/1434815716 >> v1:172.21.15.39:6789/0 conn(0x7f52e8159ab0 legacy=0x7f52e80714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.263+0000 7f52ec8a6640 1 -- 172.21.15.32:0/1434815716 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e80c98b0 con 0x7f52e80b3e90 2023-12-20T06:03:08.266 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52ec8a6640 1 --2- 172.21.15.32:0/1434815716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e80b3e90 0x7f52e80ca820 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009e50 tx=0x7f52e001c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=fbd42f86dfaba679 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:08.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52e6ffd640 1 -- 172.21.15.32:0/1434815716 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e00a5740 con 0x7f52e80b3e90 2023-12-20T06:03:08.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52e6ffd640 1 -- 172.21.15.32:0/1434815716 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e00a58a0 con 0x7f52e80b3e90 2023-12-20T06:03:08.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52e6ffd640 1 -- 172.21.15.32:0/1434815716 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e00a5c30 con 0x7f52e80b3e90 2023-12-20T06:03:08.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52ed0a7640 1 -- 172.21.15.32:0/1434815716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e80b3e90 msgr2=0x7f52e80ca820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52ed0a7640 1 --2- 172.21.15.32:0/1434815716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e80b3e90 0x7f52e80ca820 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f52e0009e50 tx=0x7f52e001c9e0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52ed0a7640 1 -- 172.21.15.32:0/1434815716 shutdown_connections 2023-12-20T06:03:08.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52ed0a7640 1 --2- 172.21.15.32:0/1434815716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e80b3e90 0x7f52e80ca820 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52ed0a7640 1 --2- 172.21.15.32:0/1434815716 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52e80cd2e0 0x7f52e80ca2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.267 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52ed0a7640 1 -- 172.21.15.32:0/1434815716 >> 172.21.15.32:0/1434815716 conn(0x7f52e806c060 msgr2=0x7f52e806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:08.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52ed0a7640 1 -- 172.21.15.32:0/1434815716 shutdown_connections 2023-12-20T06:03:08.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52ed0a7640 1 -- 172.21.15.32:0/1434815716 wait complete. 2023-12-20T06:03:08.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52ed0a7640 1 Processor -- start 2023-12-20T06:03:08.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.267+0000 7f52ed0a7640 1 -- start start 2023-12-20T06:03:08.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ed0a7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52e80b3e90 0x7f52e80c1a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ed0a7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e80cd2e0 0x7f52e81fd5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ed0a7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8159ab0 0x7f52e815a0a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ed0a7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e8149490 con 0x7f52e8159ab0 2023-12-20T06:03:08.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ed0a7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f52e814a680 con 0x7f52e80cd2e0 2023-12-20T06:03:08.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ed0a7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f52e81fdb30 con 0x7f52e80b3e90 2023-12-20T06:03:08.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ec8a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8159ab0 0x7f52e815a0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52e77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e80cd2e0 0x7f52e81fd5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52e7fff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52e80b3e90 0x7f52e80c1a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52e77fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e80cd2e0 0x7f52e81fd5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:32952/0 (socket says 172.21.15.32:32952) 2023-12-20T06:03:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52e77fe640 1 -- 172.21.15.32:0/192071025 learned_addr learned my addr 172.21.15.32:0/192071025 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ec8a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8159ab0 0x7f52e815a0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33982/0 (socket says 172.21.15.32:33982) 2023-12-20T06:03:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ec8a6640 1 -- 172.21.15.32:0/192071025 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52e80b3e90 msgr2=0x7f52e80c1a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ec8a6640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52e80b3e90 0x7f52e80c1a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ec8a6640 1 -- 172.21.15.32:0/192071025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e80cd2e0 msgr2=0x7f52e81fd5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ec8a6640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e80cd2e0 0x7f52e81fd5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ec8a6640 1 -- 172.21.15.32:0/192071025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f52e0009b30 con 0x7f52e8159ab0 2023-12-20T06:03:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52e7fff640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52e80b3e90 0x7f52e80c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ec8a6640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8159ab0 0x7f52e815a0a0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f52e806cb80 tx=0x7f52e0019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52e4ff9640 1 -- 172.21.15.32:0/192071025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e00b8670 con 0x7f52e8159ab0 2023-12-20T06:03:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ed0a7640 1 -- 172.21.15.32:0/192071025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f52e8149a70 con 0x7f52e8159ab0 2023-12-20T06:03:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52e4ff9640 1 -- 172.21.15.32:0/192071025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f52e00b87d0 con 0x7f52e8159ab0 2023-12-20T06:03:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52e4ff9640 1 -- 172.21.15.32:0/192071025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f52e00b8b80 con 0x7f52e8159ab0 2023-12-20T06:03:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ed0a7640 1 -- 172.21.15.32:0/192071025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f52e814a030 con 0x7f52e8159ab0 2023-12-20T06:03:08.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.271+0000 7f52ed0a7640 1 -- 172.21.15.32:0/192071025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f52e8074100 con 0x7f52e8159ab0 2023-12-20T06:03:08.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.275+0000 7f52e4ff9640 1 -- 172.21.15.32:0/192071025 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f52e00b8ce0 con 0x7f52e8159ab0 2023-12-20T06:03:08.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.275+0000 7f52e4ff9640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f52d4065cd0 0x7f52d4068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.275+0000 7f52e4ff9640 1 -- 172.21.15.32:0/192071025 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f52e011ec10 con 0x7f52e8159ab0 2023-12-20T06:03:08.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.275+0000 7f52e7fff640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f52d4065cd0 0x7f52d4068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.275+0000 7f52e7fff640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f52d4065cd0 0x7f52d4068190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f52d8007fd0 tx=0x7f52d8009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:08.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.279+0000 7f52e4ff9640 1 -- 172.21.15.32:0/192071025 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f52e00ec030 con 0x7f52e8159ab0 2023-12-20T06:03:08.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 -- 172.21.15.32:0/192071025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f52e814a7c0 con 0x7f52e8159ab0 2023-12-20T06:03:08.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52e4ff9640 1 -- 172.21.15.32:0/192071025 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4719 (secure 0 0 0) 0x7f52e0009510 con 0x7f52e8159ab0 2023-12-20T06:03:08.470 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:08.470 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":104,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":11145,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1517557733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1517557733},{"type":"v1","addr":"172.21.15.32:6837","nonce":1517557733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:39.446992+0000","modified":"2023-12-20T06:02:42.509418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10530":{"gid":10530,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/3001088147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3001088147},{"type":"v1","addr":"172.21.15.39:6833","nonce":3001088147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-20T06:03:08.471 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 104 2023-12-20T06:03:08.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 -- 172.21.15.32:0/192071025 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f52d4065cd0 msgr2=0x7f52d4068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f52d4065cd0 0x7f52d4068190 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f52d8007fd0 tx=0x7f52d8009040 comp rx=0 tx=0).stop 2023-12-20T06:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 -- 172.21.15.32:0/192071025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8159ab0 msgr2=0x7f52e815a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8159ab0 0x7f52e815a0a0 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7f52e806cb80 tx=0x7f52e0019780 comp rx=0 tx=0).stop 2023-12-20T06:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 -- 172.21.15.32:0/192071025 shutdown_connections 2023-12-20T06:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f52d4065cd0 0x7f52d4068190 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f52e8159ab0 0x7f52e815a0a0 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f52e80cd2e0 0x7f52e81fd5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 --2- 172.21.15.32:0/192071025 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f52e80b3e90 0x7f52e80c1a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 -- 172.21.15.32:0/192071025 >> 172.21.15.32:0/192071025 conn(0x7f52e806c060 msgr2=0x7f52e80cc120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:08.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 -- 172.21.15.32:0/192071025 shutdown_connections 2023-12-20T06:03:08.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.471+0000 7f52ed0a7640 1 -- 172.21.15.32:0/192071025 wait complete. 2023-12-20T06:03:08.492 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.635+0000 7fdac4dbe640 1 Processor -- start 2023-12-20T06:03:08.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.635+0000 7fdac4dbe640 1 -- start start 2023-12-20T06:03:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdac4dbe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdac015ce60 0x7fdac00c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdac4dbe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdac00713e0 0x7fdac0071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdac4dbe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac0160b30 0x7fdac0160fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdac4dbe640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac0071d80 con 0x7fdac00713e0 2023-12-20T06:03:08.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdac4dbe640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdac00c9a40 con 0x7fdac015ce60 2023-12-20T06:03:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdac4dbe640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac00c9180 con 0x7fdac0160b30 2023-12-20T06:03:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdab7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdac00713e0 0x7fdac0071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdab7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdac00713e0 0x7fdac0071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33994/0 (socket says 172.21.15.32:33994) 2023-12-20T06:03:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdab7fff640 1 -- 172.21.15.32:0/158117498 learned_addr learned my addr 172.21.15.32:0/158117498 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdabed9d640 1 --2- 172.21.15.32:0/158117498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac0160b30 0x7fdac0160fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdabe59c640 1 --2- 172.21.15.32:0/158117498 >> v2:172.21.15.39:3300/0 conn(0x7fdac015ce60 0x7fdac00c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdab7fff640 1 -- 172.21.15.32:0/158117498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac0160b30 msgr2=0x7fdac0160fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:08.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdab7fff640 1 --2- 172.21.15.32:0/158117498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac0160b30 0x7fdac0160fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdab7fff640 1 -- 172.21.15.32:0/158117498 >> v2:172.21.15.39:3300/0 conn(0x7fdac015ce60 msgr2=0x7fdac00c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:08.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdab7fff640 1 --2- 172.21.15.32:0/158117498 >> v2:172.21.15.39:3300/0 conn(0x7fdac015ce60 0x7fdac00c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdab7fff640 1 -- 172.21.15.32:0/158117498 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdac00c93f0 con 0x7fdac00713e0 2023-12-20T06:03:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.639+0000 7fdab7fff640 1 --2- 172.21.15.32:0/158117498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac00713e0 0x7fdac0071840 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fdac00cc230 tx=0x7fdaa801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5404c3fc39ac3945 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdabdd9b640 1 -- 172.21.15.32:0/158117498 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80ad640 con 0x7fdac00713e0 2023-12-20T06:03:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdabdd9b640 1 -- 172.21.15.32:0/158117498 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdaa80ad7a0 con 0x7fdac00713e0 2023-12-20T06:03:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdabdd9b640 1 -- 172.21.15.32:0/158117498 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdaa80adb30 con 0x7fdac00713e0 2023-12-20T06:03:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 -- 172.21.15.32:0/158117498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac00713e0 msgr2=0x7fdac0071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 --2- 172.21.15.32:0/158117498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac00713e0 0x7fdac0071840 secure :-1 s=READY pgs=2179 cs=0 l=1 rev1=1 crypto rx=0x7fdac00cc230 tx=0x7fdaa801c500 comp rx=0 tx=0).stop 2023-12-20T06:03:08.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 -- 172.21.15.32:0/158117498 shutdown_connections 2023-12-20T06:03:08.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 --2- 172.21.15.32:0/158117498 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac0160b30 0x7fdac0160fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 --2- 172.21.15.32:0/158117498 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac00713e0 0x7fdac0071840 unknown :-1 s=CLOSED pgs=2179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 --2- 172.21.15.32:0/158117498 >> v2:172.21.15.39:3300/0 conn(0x7fdac015ce60 0x7fdac00c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 -- 172.21.15.32:0/158117498 >> 172.21.15.32:0/158117498 conn(0x7fdac006a320 msgr2=0x7fdac00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:08.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 -- 172.21.15.32:0/158117498 shutdown_connections 2023-12-20T06:03:08.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 -- 172.21.15.32:0/158117498 wait complete. 2023-12-20T06:03:08.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 Processor -- start 2023-12-20T06:03:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 -- start start 2023-12-20T06:03:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac00713e0 0x7fdac015eef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac015ce60 0x7fdac0152900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac0160b30 0x7fdac006c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac015f430 con 0x7fdac0160b30 2023-12-20T06:03:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdac00c2a50 con 0x7fdac00713e0 2023-12-20T06:03:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdac4dbe640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdac00c2d30 con 0x7fdac015ce60 2023-12-20T06:03:08.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac0160b30 0x7fdac006c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdabed9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac0160b30 0x7fdac006c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33998/0 (socket says 172.21.15.32:33998) 2023-12-20T06:03:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.643+0000 7fdabed9d640 1 -- 172.21.15.32:0/1967343167 learned_addr learned my addr 172.21.15.32:0/1967343167 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdab7fff640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac015ce60 0x7fdac0152900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdabe59c640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac00713e0 0x7fdac015eef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdabed9d640 1 -- 172.21.15.32:0/1967343167 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac015ce60 msgr2=0x7fdac0152900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdabed9d640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac015ce60 0x7fdac0152900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdabed9d640 1 -- 172.21.15.32:0/1967343167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac00713e0 msgr2=0x7fdac015eef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdabed9d640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac00713e0 0x7fdac015eef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdabed9d640 1 -- 172.21.15.32:0/1967343167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa80094d0 con 0x7fdac0160b30 2023-12-20T06:03:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdabe59c640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac00713e0 0x7fdac015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdabed9d640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac0160b30 0x7fdac006c3e0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fdab8007e40 tx=0x7fdab800e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdab6ffd640 1 -- 172.21.15.32:0/1967343167 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab8014620 con 0x7fdac0160b30 2023-12-20T06:03:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdab6ffd640 1 -- 172.21.15.32:0/1967343167 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdab800ee90 con 0x7fdac0160b30 2023-12-20T06:03:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdac4dbe640 1 -- 172.21.15.32:0/1967343167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdac0161680 con 0x7fdac0160b30 2023-12-20T06:03:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdab6ffd640 1 -- 172.21.15.32:0/1967343167 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdab80149e0 con 0x7fdac0160b30 2023-12-20T06:03:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdac4dbe640 1 -- 172.21.15.32:0/1967343167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdac00c32f0 con 0x7fdac0160b30 2023-12-20T06:03:08.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdac4dbe640 1 -- 172.21.15.32:0/1967343167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdac004e000 con 0x7fdac0160b30 2023-12-20T06:03:08.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.647+0000 7fdab6ffd640 1 -- 172.21.15.32:0/1967343167 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fdab8014b40 con 0x7fdac0160b30 2023-12-20T06:03:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.651+0000 7fdab6ffd640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdaa0065ca0 0x7fdaa0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.651+0000 7fdab6ffd640 1 -- 172.21.15.32:0/1967343167 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fdab8084100 con 0x7fdac0160b30 2023-12-20T06:03:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.651+0000 7fdabe59c640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdaa0065ca0 0x7fdaa0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.651+0000 7fdabe59c640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdaa0065ca0 0x7fdaa0068160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdab0006fd0 tx=0x7fdab0008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:08.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.655+0000 7fdab6ffd640 1 -- 172.21.15.32:0/1967343167 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdab80514b0 con 0x7fdac0160b30 2023-12-20T06:03:08.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.815+0000 7fdac4dbe640 1 -- 172.21.15.32:0/1967343167 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fdac00c34e0 con 0x7fdac0160b30 2023-12-20T06:03:08.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.815+0000 7fdab6ffd640 1 -- 172.21.15.32:0/1967343167 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v218) v1 ==== 74+0+17344 (secure 0 0 0) 0x7fdab8054ad0 con 0x7fdac0160b30 2023-12-20T06:03:08.815 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:08.816 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":218,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:06.687492+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:02:37.109832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:02:38.440830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/888710554":"2023-12-21T06:03:03.292086+0000","172.21.15.32:0/3767986898":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/2893447374":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/4287660207":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1968262556":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1215843411":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1623434834":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/3650825501":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/2603552779":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/587127774":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3681500114":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/1096471740":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6836/888710554":"2023-12-21T06:03:03.292086+0000","172.21.15.32:0/922716312":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1437478066":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3373411185":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6833/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6832/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/4110141711":"2023-12-21T06:02:50.560868+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 -- 172.21.15.32:0/1967343167 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdaa0065ca0 msgr2=0x7fdaa0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdaa0065ca0 0x7fdaa0068160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fdab0006fd0 tx=0x7fdab0008040 comp rx=0 tx=0).stop 2023-12-20T06:03:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 -- 172.21.15.32:0/1967343167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac0160b30 msgr2=0x7fdac006c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac0160b30 0x7fdac006c3e0 secure :-1 s=READY pgs=2180 cs=0 l=1 rev1=1 crypto rx=0x7fdab8007e40 tx=0x7fdab800e4b0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 -- 172.21.15.32:0/1967343167 shutdown_connections 2023-12-20T06:03:08.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdaa0065ca0 0x7fdaa0068160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdac0160b30 0x7fdac006c3e0 unknown :-1 s=CLOSED pgs=2180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdac015ce60 0x7fdac0152900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 --2- 172.21.15.32:0/1967343167 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdac00713e0 0x7fdac015eef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 -- 172.21.15.32:0/1967343167 >> 172.21.15.32:0/1967343167 conn(0x7fdac006a320 msgr2=0x7fdac00b0f10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 -- 172.21.15.32:0/1967343167 shutdown_connections 2023-12-20T06:03:08.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.819+0000 7fdac4dbe640 1 -- 172.21.15.32:0/1967343167 wait complete. 2023-12-20T06:03:08.837 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-12-20T06:03:08.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2f9a6640 1 Processor -- start 2023-12-20T06:03:08.981 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2f9a6640 1 -- start start 2023-12-20T06:03:08.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2f9a6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4e280cfc80 0x7f4e28167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2f9a6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e280b6ca0 con 0x7f4e2815cdb0 2023-12-20T06:03:08.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2f9a6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4e2815d190 con 0x7f4e280d2740 2023-12-20T06:03:08.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2f9a6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4e280d16a0 con 0x7f4e280cfc80 2023-12-20T06:03:08.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2d742640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4e2815cdb0 0x7f4e280d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56768/0 (socket says 172.21.15.32:56768) 2023-12-20T06:03:08.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2d742640 1 -- 172.21.15.32:0/233833733 learned_addr learned my addr 172.21.15.32:0/233833733 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:08.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2df43640 1 --2- 172.21.15.32:0/233833733 >> v2:172.21.15.32:3300/0 conn(0x7f4e280cfc80 0x7f4e28167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2df43640 1 -- 172.21.15.32:0/233833733 >> v1:172.21.15.39:6789/0 conn(0x7f4e280d2740 legacy=0x7f4e280cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2df43640 1 -- 172.21.15.32:0/233833733 >> v1:172.21.15.32:6789/0 conn(0x7f4e2815cdb0 legacy=0x7f4e280d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.983+0000 7f4e2df43640 1 -- 172.21.15.32:0/233833733 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e280d1980 con 0x7f4e280cfc80 2023-12-20T06:03:08.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.987+0000 7f4e2df43640 1 --2- 172.21.15.32:0/233833733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e280cfc80 0x7f4e28167910 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009d90 tx=0x7f4e2001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed2cdc15839a6422 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:08.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.987+0000 7f4e1ffff640 1 -- 172.21.15.32:0/233833733 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200af600 con 0x7f4e280cfc80 2023-12-20T06:03:08.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.987+0000 7f4e1ffff640 1 -- 172.21.15.32:0/233833733 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e200ab050 con 0x7f4e280cfc80 2023-12-20T06:03:08.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.987+0000 7f4e1ffff640 1 -- 172.21.15.32:0/233833733 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200af980 con 0x7f4e280cfc80 2023-12-20T06:03:08.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.987+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/233833733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e280cfc80 msgr2=0x7f4e28167910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.987+0000 7f4e2f9a6640 1 --2- 172.21.15.32:0/233833733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e280cfc80 0x7f4e28167910 secure :-1 s=READY pgs=2182 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009d90 tx=0x7f4e2001f130 comp rx=0 tx=0).stop 2023-12-20T06:03:08.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.987+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/233833733 shutdown_connections 2023-12-20T06:03:08.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.987+0000 7f4e2f9a6640 1 --2- 172.21.15.32:0/233833733 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e280cfc80 0x7f4e28167910 unknown :-1 s=CLOSED pgs=2182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.987+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/233833733 >> 172.21.15.32:0/233833733 conn(0x7f4e28065ee0 msgr2=0x7f4e2806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:08.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.987+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/233833733 shutdown_connections 2023-12-20T06:03:08.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.987+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/233833733 wait complete. 2023-12-20T06:03:08.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2f9a6640 1 Processor -- start 2023-12-20T06:03:08.989 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2f9a6640 1 -- start start 2023-12-20T06:03:08.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2f9a6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e280cfc80 0x7f4e2814e230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2f9a6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e280d2740 0x7f4e280cc360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2f9a6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e2815cdb0 0x7f4e281548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2f9a6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e28157280 con 0x7f4e2815cdb0 2023-12-20T06:03:08.992 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2f9a6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4e2814e770 con 0x7f4e280d2740 2023-12-20T06:03:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2f9a6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4e28156600 con 0x7f4e280cfc80 2023-12-20T06:03:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2df43640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e2815cdb0 0x7f4e281548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2df43640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e2815cdb0 0x7f4e281548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52026/0 (socket says 172.21.15.32:52026) 2023-12-20T06:03:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2df43640 1 -- 172.21.15.32:0/3500184553 learned_addr learned my addr 172.21.15.32:0/3500184553 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2d742640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e280cfc80 0x7f4e2814e230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2cf41640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e280d2740 0x7f4e280cc360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.993 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2df43640 1 -- 172.21.15.32:0/3500184553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e280cfc80 msgr2=0x7f4e2814e230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2df43640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e280cfc80 0x7f4e2814e230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2df43640 1 -- 172.21.15.32:0/3500184553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e280d2740 msgr2=0x7f4e280cc360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:08.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2df43640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e280d2740 0x7f4e280cc360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:08.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2df43640 1 -- 172.21.15.32:0/3500184553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4e20009a70 con 0x7f4e2815cdb0 2023-12-20T06:03:08.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2df43640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e2815cdb0 0x7f4e281548c0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009d90 tx=0x7f4e2001bed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:08.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e1dffb640 1 -- 172.21.15.32:0/3500184553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200b9610 con 0x7f4e2815cdb0 2023-12-20T06:03:08.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e1dffb640 1 -- 172.21.15.32:0/3500184553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4e200b9770 con 0x7f4e2815cdb0 2023-12-20T06:03:08.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e1dffb640 1 -- 172.21.15.32:0/3500184553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4e200b9b20 con 0x7f4e2815cdb0 2023-12-20T06:03:08.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/3500184553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4e281568e0 con 0x7f4e2815cdb0 2023-12-20T06:03:08.995 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.991+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/3500184553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4e28156ea0 con 0x7f4e2815cdb0 2023-12-20T06:03:08.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.995+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/3500184553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4e281573c0 con 0x7f4e2815cdb0 2023-12-20T06:03:08.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.995+0000 7f4e1dffb640 1 -- 172.21.15.32:0/3500184553 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f4e200b6050 con 0x7f4e2815cdb0 2023-12-20T06:03:08.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.995+0000 7f4e1dffb640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e0c065ca0 0x7f4e0c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:08.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.995+0000 7f4e1dffb640 1 -- 172.21.15.32:0/3500184553 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f4e20121f30 con 0x7f4e2815cdb0 2023-12-20T06:03:08.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.995+0000 7f4e2d742640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e0c065ca0 0x7f4e0c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:08.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.995+0000 7f4e2d742640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e0c065ca0 0x7f4e0c068160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005890 tx=0x7f4e24009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:08.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:08.999+0000 7f4e1dffb640 1 -- 172.21.15.32:0/3500184553 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4e200ef2a0 con 0x7f4e2815cdb0 2023-12-20T06:03:09.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.187+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/3500184553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f4e2804e000 con 0x7f4e2815cdb0 2023-12-20T06:03:09.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.187+0000 7f4e1dffb640 1 -- 172.21.15.32:0/3500184553 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4719 (secure 0 0 0) 0x7f4e200b4020 con 0x7f4e2815cdb0 2023-12-20T06:03:09.188 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:09.189 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":104,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":11145,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1517557733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1517557733},{"type":"v1","addr":"172.21.15.32:6837","nonce":1517557733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:39.446992+0000","modified":"2023-12-20T06:02:42.509418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10530":{"gid":10530,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/3001088147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3001088147},{"type":"v1","addr":"172.21.15.39:6833","nonce":3001088147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-20T06:03:09.189 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 104 2023-12-20T06:03:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/3500184553 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e0c065ca0 msgr2=0x7f4e0c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e0c065ca0 0x7f4e0c068160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f4e24005890 tx=0x7f4e24009040 comp rx=0 tx=0).stop 2023-12-20T06:03:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/3500184553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e2815cdb0 msgr2=0x7f4e281548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e2815cdb0 0x7f4e281548c0 secure :-1 s=READY pgs=2183 cs=0 l=1 rev1=1 crypto rx=0x7f4e20009d90 tx=0x7f4e2001bed0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/3500184553 shutdown_connections 2023-12-20T06:03:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4e0c065ca0 0x7f4e0c068160 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4e2815cdb0 0x7f4e281548c0 unknown :-1 s=CLOSED pgs=2183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4e280d2740 0x7f4e280cc360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 --2- 172.21.15.32:0/3500184553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4e280cfc80 0x7f4e2814e230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/3500184553 >> 172.21.15.32:0/3500184553 conn(0x7f4e28065ee0 msgr2=0x7f4e2806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:09.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/3500184553 shutdown_connections 2023-12-20T06:03:09.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.191+0000 7f4e2f9a6640 1 -- 172.21.15.32:0/3500184553 wait complete. 2023-12-20T06:03:09.210 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c41073640 1 Processor -- start 2023-12-20T06:03:09.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c41073640 1 -- start start 2023-12-20T06:03:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c41073640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c3c0af5e0 0x7f5c3c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c41073640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c3c15f110 0x7f5c3c15c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c41073640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5c3c15b990 con 0x7f5c3c06ce60 2023-12-20T06:03:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c41073640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5c3c15bbd0 con 0x7f5c3c15f110 2023-12-20T06:03:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c41073640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c15beb0 con 0x7f5c3c0af5e0 2023-12-20T06:03:09.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c3b59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c3c15f110 0x7f5c3c15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c3b59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5c3c15f110 0x7f5c3c15c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52042/0 (socket says 172.21.15.32:52042) 2023-12-20T06:03:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c3b59e640 1 -- 172.21.15.32:0/4035233014 learned_addr learned my addr 172.21.15.32:0/4035233014 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c3ad9d640 1 --2- 172.21.15.32:0/4035233014 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c3c0af5e0 0x7f5c3c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c3b59e640 1 -- 172.21.15.32:0/4035233014 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c3c0af5e0 msgr2=0x7f5c3c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c3b59e640 1 --2- 172.21.15.32:0/4035233014 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c3c0af5e0 0x7f5c3c0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c3b59e640 1 -- 172.21.15.32:0/4035233014 >> v1:172.21.15.39:6789/0 conn(0x7f5c3c06ce60 legacy=0x7f5c3c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c39d9b640 1 -- 172.21.15.32:0/4035233014 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1594981506 0 0) 0x7f5c30001120 con 0x7f5c3c06ce60 2023-12-20T06:03:09.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.355+0000 7f5c3b59e640 1 -- 172.21.15.32:0/4035233014 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c3c15e910 con 0x7f5c3c15f110 2023-12-20T06:03:09.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c3b59e640 1 --2- 172.21.15.32:0/4035233014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3c15f110 0x7f5c3c15c440 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009e50 tx=0x7f5c3401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=420116a3a2e78010 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c39d9b640 1 -- 172.21.15.32:0/4035233014 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c340a58d0 con 0x7f5c3c15f110 2023-12-20T06:03:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c39d9b640 1 -- 172.21.15.32:0/4035233014 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c340a5a30 con 0x7f5c3c15f110 2023-12-20T06:03:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c39d9b640 1 -- 172.21.15.32:0/4035233014 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c340a5dc0 con 0x7f5c3c15f110 2023-12-20T06:03:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c41073640 1 -- 172.21.15.32:0/4035233014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3c15f110 msgr2=0x7f5c3c15c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c41073640 1 --2- 172.21.15.32:0/4035233014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3c15f110 0x7f5c3c15c440 secure :-1 s=READY pgs=2184 cs=0 l=1 rev1=1 crypto rx=0x7f5c34009e50 tx=0x7f5c3401c9e0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c41073640 1 -- 172.21.15.32:0/4035233014 shutdown_connections 2023-12-20T06:03:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c41073640 1 --2- 172.21.15.32:0/4035233014 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3c15f110 0x7f5c3c15c440 unknown :-1 s=CLOSED pgs=2184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c41073640 1 --2- 172.21.15.32:0/4035233014 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c3c0af5e0 0x7f5c3c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c41073640 1 -- 172.21.15.32:0/4035233014 >> 172.21.15.32:0/4035233014 conn(0x7f5c3c06a320 msgr2=0x7f5c3c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c41073640 1 -- 172.21.15.32:0/4035233014 shutdown_connections 2023-12-20T06:03:09.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.359+0000 7f5c41073640 1 -- 172.21.15.32:0/4035233014 wait complete. 2023-12-20T06:03:09.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c41073640 1 Processor -- start 2023-12-20T06:03:09.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c41073640 1 -- start start 2023-12-20T06:03:09.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c41073640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c3c06ce60 0x7f5c3c155260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:09.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c41073640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c3c0af5e0 0x7f5c3c0c4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:09.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c41073640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3c15f110 0x7f5c3c0cd6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:09.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c41073640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c154dd0 con 0x7f5c3c15f110 2023-12-20T06:03:09.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c41073640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c0c81f0 con 0x7f5c3c06ce60 2023-12-20T06:03:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c41073640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5c3c150460 con 0x7f5c3c0af5e0 2023-12-20T06:03:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3b59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3c15f110 0x7f5c3c0cd6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3b59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3c15f110 0x7f5c3c0cd6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52048/0 (socket says 172.21.15.32:52048) 2023-12-20T06:03:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3b59e640 1 -- 172.21.15.32:0/1258038605 learned_addr learned my addr 172.21.15.32:0/1258038605 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3ad9d640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c3c06ce60 0x7f5c3c155260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3b59e640 1 -- 172.21.15.32:0/1258038605 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c3c0af5e0 msgr2=0x7f5c3c0c4510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3a59c640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c3c0af5e0 0x7f5c3c0c4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3b59e640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c3c0af5e0 0x7f5c3c0c4510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3b59e640 1 -- 172.21.15.32:0/1258038605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c3c06ce60 msgr2=0x7f5c3c155260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3b59e640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c3c06ce60 0x7f5c3c155260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3b59e640 1 -- 172.21.15.32:0/1258038605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5c34009b30 con 0x7f5c3c15f110 2023-12-20T06:03:09.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3ad9d640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c3c06ce60 0x7f5c3c155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:09.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3a59c640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c3c0af5e0 0x7f5c3c0c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:09.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c3b59e640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3c15f110 0x7f5c3c0cd6b0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f5c3401a610 tx=0x7f5c34019780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:09.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c237fe640 1 -- 172.21.15.32:0/1258038605 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c340be070 con 0x7f5c3c15f110 2023-12-20T06:03:09.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c237fe640 1 -- 172.21.15.32:0/1258038605 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5c340b8440 con 0x7f5c3c15f110 2023-12-20T06:03:09.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c41073640 1 -- 172.21.15.32:0/1258038605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5c3c0c4a50 con 0x7f5c3c15f110 2023-12-20T06:03:09.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c237fe640 1 -- 172.21.15.32:0/1258038605 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5c340b87f0 con 0x7f5c3c15f110 2023-12-20T06:03:09.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.363+0000 7f5c41073640 1 -- 172.21.15.32:0/1258038605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5c3c154780 con 0x7f5c3c15f110 2023-12-20T06:03:09.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.367+0000 7f5c41073640 1 -- 172.21.15.32:0/1258038605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5c3c154f80 con 0x7f5c3c15f110 2023-12-20T06:03:09.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.367+0000 7f5c237fe640 1 -- 172.21.15.32:0/1258038605 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f5c340b8950 con 0x7f5c3c15f110 2023-12-20T06:03:09.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.367+0000 7f5c237fe640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5c28065ca0 0x7f5c28068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:09.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.367+0000 7f5c237fe640 1 -- 172.21.15.32:0/1258038605 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f5c3411e570 con 0x7f5c3c15f110 2023-12-20T06:03:09.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.367+0000 7f5c3ad9d640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5c28065ca0 0x7f5c28068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:09.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.367+0000 7f5c3ad9d640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5c28065ca0 0x7f5c28068160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009ac0 tx=0x7f5c2c008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:09.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.371+0000 7f5c237fe640 1 -- 172.21.15.32:0/1258038605 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5c3c154f80 con 0x7f5c3c15f110 2023-12-20T06:03:09.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.563+0000 7f5c41073640 1 -- 172.21.15.32:0/1258038605 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f5c3c154f80 con 0x7f5c3c15f110 2023-12-20T06:03:09.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.563+0000 7f5c237fe640 1 -- 172.21.15.32:0/1258038605 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 104 v104) v1 ==== 95+0+4719 (secure 0 0 0) 0x7f5c3c154f80 con 0x7f5c3c15f110 2023-12-20T06:03:09.563 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:09.563 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":104,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":11145,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1517557733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1517557733},{"type":"v1","addr":"172.21.15.32:6837","nonce":1517557733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[{"mdsmap":{"epoch":102,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:02:39.446992+0000","modified":"2023-12-20T06:02:42.509418+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":10530},"failed":[],"damaged":[],"stopped":[],"info":{"gid_10530":{"gid":10530,"name":"d","rank":0,"incarnation":98,"state":"up:active","state_seq":7,"addr":"172.21.15.39:6833/3001088147","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3001088147},{"type":"v1","addr":"172.21.15.39:6833","nonce":3001088147}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[27],"metadata_pool":26,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":13}]} 2023-12-20T06:03:09.564 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 104 2023-12-20T06:03:09.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 -- 172.21.15.32:0/1258038605 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5c28065ca0 msgr2=0x7f5c28068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5c28065ca0 0x7f5c28068160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f5c2c009ac0 tx=0x7f5c2c008040 comp rx=0 tx=0).stop 2023-12-20T06:03:09.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 -- 172.21.15.32:0/1258038605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3c15f110 msgr2=0x7f5c3c0cd6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3c15f110 0x7f5c3c0cd6b0 secure :-1 s=READY pgs=2185 cs=0 l=1 rev1=1 crypto rx=0x7f5c3401a610 tx=0x7f5c34019780 comp rx=0 tx=0).stop 2023-12-20T06:03:09.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 -- 172.21.15.32:0/1258038605 shutdown_connections 2023-12-20T06:03:09.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5c28065ca0 0x7f5c28068160 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5c3c15f110 0x7f5c3c0cd6b0 unknown :-1 s=CLOSED pgs=2185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5c3c0af5e0 0x7f5c3c0c4510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 --2- 172.21.15.32:0/1258038605 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5c3c06ce60 0x7f5c3c155260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 -- 172.21.15.32:0/1258038605 >> 172.21.15.32:0/1258038605 conn(0x7f5c3c06a320 msgr2=0x7f5c3c150640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 -- 172.21.15.32:0/1258038605 shutdown_connections 2023-12-20T06:03:09.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.567+0000 7f5c41073640 1 -- 172.21.15.32:0/1258038605 wait complete. 2023-12-20T06:03:09.585 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe7cfc640 1 Processor -- start 2023-12-20T06:03:09.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe7cfc640 1 -- start start 2023-12-20T06:03:09.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe7cfc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbe006cf10 0x7fbbe00c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe7cfc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe00ae3f0 con 0x7fbbe00ae010 2023-12-20T06:03:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe7cfc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbe0161800 con 0x7fbbe0164ef0 2023-12-20T06:03:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe7cfc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbe0165380 con 0x7fbbe006cf10 2023-12-20T06:03:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe5297640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbe006cf10 0x7fbbe00c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe5297640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbe006cf10 0x7fbbe00c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52064/0 (socket says 172.21.15.32:52064) 2023-12-20T06:03:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe5297640 1 -- 172.21.15.32:0/1595593788 learned_addr learned my addr 172.21.15.32:0/1595593788 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe5297640 1 -- 172.21.15.32:0/1595593788 >> v1:172.21.15.39:6789/0 conn(0x7fbbe0164ef0 legacy=0x7fbbe00c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe5297640 1 -- 172.21.15.32:0/1595593788 >> v1:172.21.15.32:6789/0 conn(0x7fbbe00ae010 legacy=0x7fbbe0071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.731+0000 7fbbe5297640 1 -- 172.21.15.32:0/1595593788 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbe0165660 con 0x7fbbe006cf10 2023-12-20T06:03:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.735+0000 7fbbe5297640 1 --2- 172.21.15.32:0/1595593788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe006cf10 0x7fbbe00c94a0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009870 tx=0x7fbbd001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a892a2b3c535dd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.735+0000 7fbbe4a96640 1 -- 172.21.15.32:0/1595593788 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00ab680 con 0x7fbbe006cf10 2023-12-20T06:03:09.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.735+0000 7fbbe4a96640 1 -- 172.21.15.32:0/1595593788 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbd00ab7e0 con 0x7fbbe006cf10 2023-12-20T06:03:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.735+0000 7fbbe4a96640 1 -- 172.21.15.32:0/1595593788 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbd00abb70 con 0x7fbbe006cf10 2023-12-20T06:03:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.735+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/1595593788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe006cf10 msgr2=0x7fbbe00c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.735+0000 7fbbe7cfc640 1 --2- 172.21.15.32:0/1595593788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe006cf10 0x7fbbe00c94a0 secure :-1 s=READY pgs=2187 cs=0 l=1 rev1=1 crypto rx=0x7fbbd0009870 tx=0x7fbbd001c480 comp rx=0 tx=0).stop 2023-12-20T06:03:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.735+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/1595593788 shutdown_connections 2023-12-20T06:03:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.735+0000 7fbbe7cfc640 1 --2- 172.21.15.32:0/1595593788 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe006cf10 0x7fbbe00c94a0 unknown :-1 s=CLOSED pgs=2187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.735+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/1595593788 >> 172.21.15.32:0/1595593788 conn(0x7fbbe006a320 msgr2=0x7fbbe00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.735+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/1595593788 shutdown_connections 2023-12-20T06:03:09.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.735+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/1595593788 wait complete. 2023-12-20T06:03:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe7cfc640 1 Processor -- start 2023-12-20T06:03:09.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe7cfc640 1 -- start start 2023-12-20T06:03:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe7cfc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe00ae010 0x7fbbe01515a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe7cfc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe0151fc0 0x7fbbe0156a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe7cfc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbe0152a60 0x7fbbe0152f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe7cfc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe0153450 con 0x7fbbe00ae010 2023-12-20T06:03:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe7cfc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe01573d0 con 0x7fbbe0151fc0 2023-12-20T06:03:09.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe7cfc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbe01576b0 con 0x7fbbe0152a60 2023-12-20T06:03:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe5a98640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe00ae010 0x7fbbe01515a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe5a98640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe00ae010 0x7fbbe01515a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52072/0 (socket says 172.21.15.32:52072) 2023-12-20T06:03:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe5a98640 1 -- 172.21.15.32:0/575901385 learned_addr learned my addr 172.21.15.32:0/575901385 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe5297640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe0151fc0 0x7fbbe0156a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe6299640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbe0152a60 0x7fbbe0152f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe5a98640 1 -- 172.21.15.32:0/575901385 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbe0152a60 msgr2=0x7fbbe0152f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe5a98640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbe0152a60 0x7fbbe0152f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe5a98640 1 -- 172.21.15.32:0/575901385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe0151fc0 msgr2=0x7fbbe0156a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe5a98640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe0151fc0 0x7fbbe0156a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe5a98640 1 -- 172.21.15.32:0/575901385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbd00094d0 con 0x7fbbe00ae010 2023-12-20T06:03:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe5297640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe0151fc0 0x7fbbe0156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe5a98640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe00ae010 0x7fbbe01515a0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c450 tx=0x7fbbdc00c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbd67fc640 1 -- 172.21.15.32:0/575901385 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc005480 con 0x7fbbe00ae010 2023-12-20T06:03:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbd67fc640 1 -- 172.21.15.32:0/575901385 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbdc003600 con 0x7fbbe00ae010 2023-12-20T06:03:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbd67fc640 1 -- 172.21.15.32:0/575901385 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbdc0039b0 con 0x7fbbe00ae010 2023-12-20T06:03:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.739+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/575901385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbe0157990 con 0x7fbbe00ae010 2023-12-20T06:03:09.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.743+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/575901385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbe0157f50 con 0x7fbbe00ae010 2023-12-20T06:03:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.743+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/575901385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbe0158140 con 0x7fbbe00ae010 2023-12-20T06:03:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.743+0000 7fbbd67fc640 1 -- 172.21.15.32:0/575901385 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fbbdc003b10 con 0x7fbbe00ae010 2023-12-20T06:03:09.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.743+0000 7fbbd67fc640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbbc4065ca0 0x7fbbc4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.743+0000 7fbbd67fc640 1 -- 172.21.15.32:0/575901385 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6594+0+0 (secure 0 0 0) 0x7fbbdc082ea0 con 0x7fbbe00ae010 2023-12-20T06:03:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.743+0000 7fbbe5297640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbbc4065ca0 0x7fbbc4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:09.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.743+0000 7fbbe5297640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbbc4065ca0 0x7fbbc4068160 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00af000 tx=0x7fbbd00b6040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:09.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.747+0000 7fbbd67fc640 1 -- 172.21.15.32:0/575901385 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbbdc050190 con 0x7fbbe00ae010 2023-12-20T06:03:09.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.907+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/575901385 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fbbe004e000 con 0x7fbbe00ae010 2023-12-20T06:03:09.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbd67fc640 1 -- 172.21.15.32:0/575901385 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v218) v1 ==== 74+0+17344 (secure 0 0 0) 0x7fbbdc017070 con 0x7fbbe00ae010 2023-12-20T06:03:09.910 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:09.910 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":218,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:06.687492+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":26,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:02:37.109832+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"212","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":27,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:02:38.440830+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"213","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/888710554":"2023-12-21T06:03:03.292086+0000","172.21.15.32:0/3767986898":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/2893447374":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/4287660207":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1968262556":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1215843411":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1623434834":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/3650825501":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/2603552779":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/587127774":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3681500114":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/1096471740":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6836/888710554":"2023-12-21T06:03:03.292086+0000","172.21.15.32:0/922716312":"2023-12-21T06:02:59.631257+0000","172.21.15.32:0/1437478066":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/3373411185":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6833/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:6832/64546":"2023-12-21T06:02:50.560868+0000","172.21.15.32:0/4110141711":"2023-12-21T06:02:50.560868+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:09.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/575901385 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbbc4065ca0 msgr2=0x7fbbc4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbbc4065ca0 0x7fbbc4068160 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7fbbd00af000 tx=0x7fbbd00b6040 comp rx=0 tx=0).stop 2023-12-20T06:03:09.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/575901385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe00ae010 msgr2=0x7fbbe01515a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:09.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe00ae010 0x7fbbe01515a0 secure :-1 s=READY pgs=2188 cs=0 l=1 rev1=1 crypto rx=0x7fbbdc00c450 tx=0x7fbbdc00c910 comp rx=0 tx=0).stop 2023-12-20T06:03:09.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/575901385 shutdown_connections 2023-12-20T06:03:09.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbbc4065ca0 0x7fbbc4068160 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbe0152a60 0x7fbbe0152f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbe0151fc0 0x7fbbe0156a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 --2- 172.21.15.32:0/575901385 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbe00ae010 0x7fbbe01515a0 unknown :-1 s=CLOSED pgs=2188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:09.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/575901385 >> 172.21.15.32:0/575901385 conn(0x7fbbe006a320 msgr2=0x7fbbe00cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:09.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/575901385 shutdown_connections 2023-12-20T06:03:09.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:09.911+0000 7fbbe7cfc640 1 -- 172.21.15.32:0/575901385 wait complete. 2023-12-20T06:03:09.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-12-20T06:03:10.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.075+0000 7f4c6e525640 1 Processor -- start 2023-12-20T06:03:10.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.075+0000 7f4c6e525640 1 -- start start 2023-12-20T06:03:10.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c6e525640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4c6815f110 0x7f4c6815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:10.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c6e525640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6815b990 con 0x7f4c6806ce60 2023-12-20T06:03:10.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c6e525640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f4c6815bbd0 con 0x7f4c680af5e0 2023-12-20T06:03:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c6e525640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c6815beb0 con 0x7f4c6815f110 2023-12-20T06:03:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c677fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f4c6806ce60 0x7f4c6815b280 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56780/0 (socket says 172.21.15.32:56780) 2023-12-20T06:03:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c677fe640 1 -- 172.21.15.32:0/3557360558 learned_addr learned my addr 172.21.15.32:0/3557360558 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c6cac2640 1 --2- 172.21.15.32:0/3557360558 >> v2:172.21.15.39:3300/0 conn(0x7f4c6815f110 0x7f4c6815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c66ffd640 1 -- 172.21.15.32:0/3557360558 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1941665061 0 0) 0x7f4c5c001120 con 0x7f4c6806ce60 2023-12-20T06:03:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c66ffd640 1 -- 172.21.15.32:0/3557360558 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c50003530 con 0x7f4c6806ce60 2023-12-20T06:03:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c66ffd640 1 -- 172.21.15.32:0/3557360558 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3471582397 0 0) 0x7f4c6815bbd0 con 0x7f4c680af5e0 2023-12-20T06:03:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c66ffd640 1 -- 172.21.15.32:0/3557360558 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4c5c001120 con 0x7f4c680af5e0 2023-12-20T06:03:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c6cac2640 1 -- 172.21.15.32:0/3557360558 >> v1:172.21.15.39:6789/0 conn(0x7f4c680af5e0 legacy=0x7f4c680af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:10.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c6cac2640 1 -- 172.21.15.32:0/3557360558 >> v1:172.21.15.32:6789/0 conn(0x7f4c6806ce60 legacy=0x7f4c6815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:10.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.079+0000 7f4c6cac2640 1 -- 172.21.15.32:0/3557360558 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c6815e910 con 0x7f4c6815f110 2023-12-20T06:03:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c6cac2640 1 --2- 172.21.15.32:0/3557360558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c6815f110 0x7f4c6815c440 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009d90 tx=0x7f4c6001c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f928d447ea734ed server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c66ffd640 1 -- 172.21.15.32:0/3557360558 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c600a58f0 con 0x7f4c6815f110 2023-12-20T06:03:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c66ffd640 1 -- 172.21.15.32:0/3557360558 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c600a5a50 con 0x7f4c6815f110 2023-12-20T06:03:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c66ffd640 1 -- 172.21.15.32:0/3557360558 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c600a5dc0 con 0x7f4c6815f110 2023-12-20T06:03:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c6e525640 1 -- 172.21.15.32:0/3557360558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c6815f110 msgr2=0x7f4c6815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:10.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c6e525640 1 --2- 172.21.15.32:0/3557360558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c6815f110 0x7f4c6815c440 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f4c60009d90 tx=0x7f4c6001c890 comp rx=0 tx=0).stop 2023-12-20T06:03:10.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c6e525640 1 -- 172.21.15.32:0/3557360558 shutdown_connections 2023-12-20T06:03:10.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c6e525640 1 --2- 172.21.15.32:0/3557360558 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c6815f110 0x7f4c6815c440 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c6e525640 1 -- 172.21.15.32:0/3557360558 >> 172.21.15.32:0/3557360558 conn(0x7f4c6806a320 msgr2=0x7f4c680ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:10.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c6e525640 1 -- 172.21.15.32:0/3557360558 shutdown_connections 2023-12-20T06:03:10.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c6e525640 1 -- 172.21.15.32:0/3557360558 wait complete. 2023-12-20T06:03:10.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c6e525640 1 Processor -- start 2023-12-20T06:03:10.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.083+0000 7f4c6e525640 1 -- start start 2023-12-20T06:03:10.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c6e525640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c6806ce60 0x7f4c6814dee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:10.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c6e525640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c680af5e0 0x7f4c68159c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:10.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c6e525640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c6815f110 0x7f4c6814e8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:10.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c6e525640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6815a180 con 0x7f4c680af5e0 2023-12-20T06:03:10.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c6e525640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c68150f00 con 0x7f4c6815f110 2023-12-20T06:03:10.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c6e525640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c6814f330 con 0x7f4c6806ce60 2023-12-20T06:03:10.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c680af5e0 0x7f4c68159c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:10.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c677fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c680af5e0 0x7f4c68159c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52080/0 (socket says 172.21.15.32:52080) 2023-12-20T06:03:10.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c677fe640 1 -- 172.21.15.32:0/552790968 learned_addr learned my addr 172.21.15.32:0/552790968 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:10.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c67fff640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c6806ce60 0x7f4c6814dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:10.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c6cac2640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c6815f110 0x7f4c6814e8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:10.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c677fe640 1 -- 172.21.15.32:0/552790968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c6806ce60 msgr2=0x7f4c6814dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:10.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c677fe640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c6806ce60 0x7f4c6814dee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c677fe640 1 -- 172.21.15.32:0/552790968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c6815f110 msgr2=0x7f4c6814e8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:10.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c677fe640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c6815f110 0x7f4c6814e8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c677fe640 1 -- 172.21.15.32:0/552790968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c60009a70 con 0x7f4c680af5e0 2023-12-20T06:03:10.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c677fe640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c680af5e0 0x7f4c68159c40 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00c680 tx=0x7f4c5c00cb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:10.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c64ff9640 1 -- 172.21.15.32:0/552790968 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c005ac0 con 0x7f4c680af5e0 2023-12-20T06:03:10.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c64ff9640 1 -- 172.21.15.32:0/552790968 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c5c005c20 con 0x7f4c680af5e0 2023-12-20T06:03:10.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c6e525640 1 -- 172.21.15.32:0/552790968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c6814f610 con 0x7f4c680af5e0 2023-12-20T06:03:10.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c64ff9640 1 -- 172.21.15.32:0/552790968 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c5c007070 con 0x7f4c680af5e0 2023-12-20T06:03:10.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c6e525640 1 -- 172.21.15.32:0/552790968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c68150b90 con 0x7f4c680af5e0 2023-12-20T06:03:10.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c6e525640 1 -- 172.21.15.32:0/552790968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f4c6804e000 con 0x7f4c680af5e0 2023-12-20T06:03:10.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.087+0000 7f4c64ff9640 1 -- 172.21.15.32:0/552790968 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f4c5c0071d0 con 0x7f4c680af5e0 2023-12-20T06:03:10.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.091+0000 7f4c64ff9640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4c54065ca0 0x7f4c54068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:10.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.091+0000 7f4c64ff9640 1 -- 172.21.15.32:0/552790968 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(218..218 src has 1..218) v4 ==== 6594+0+0 (secure 0 0 0) 0x7f4c5c082e60 con 0x7f4c680af5e0 2023-12-20T06:03:10.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.091+0000 7f4c67fff640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4c54065ca0 0x7f4c54068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:10.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.091+0000 7f4c67fff640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4c54065ca0 0x7f4c54068160 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4c58000fc0 tx=0x7f4c58009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:10.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.095+0000 7f4c64ff9640 1 -- 172.21.15.32:0/552790968 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4c5c050150 con 0x7f4c680af5e0 2023-12-20T06:03:10.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.283+0000 7f4c6e525640 1 -- 172.21.15.32:0/552790968 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7f4c6815b990 con 0x7f4c680af5e0 2023-12-20T06:03:10.285 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T06:03:10.287+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T06:03:10.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.299+0000 7f4c64ff9640 1 -- 172.21.15.32:0/552790968 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v105) v1 ==== 167+0+0 (secure 0 0 0) 0x7f4c6815b990 con 0x7f4c680af5e0 2023-12-20T06:03:10.299 INFO:teuthology.orchestra.run.smithi032.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T06:03:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 -- 172.21.15.32:0/552790968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4c54065ca0 msgr2=0x7f4c54068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4c54065ca0 0x7f4c54068160 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f4c58000fc0 tx=0x7f4c58009040 comp rx=0 tx=0).stop 2023-12-20T06:03:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 -- 172.21.15.32:0/552790968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c680af5e0 msgr2=0x7f4c68159c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c680af5e0 0x7f4c68159c40 secure :-1 s=READY pgs=2190 cs=0 l=1 rev1=1 crypto rx=0x7f4c5c00c680 tx=0x7f4c5c00cb40 comp rx=0 tx=0).stop 2023-12-20T06:03:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 -- 172.21.15.32:0/552790968 shutdown_connections 2023-12-20T06:03:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4c54065ca0 0x7f4c54068160 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c6815f110 0x7f4c6814e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c680af5e0 0x7f4c68159c40 unknown :-1 s=CLOSED pgs=2190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 --2- 172.21.15.32:0/552790968 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c6806ce60 0x7f4c6814dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 -- 172.21.15.32:0/552790968 >> 172.21.15.32:0/552790968 conn(0x7f4c6806a320 msgr2=0x7f4c680cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 -- 172.21.15.32:0/552790968 shutdown_connections 2023-12-20T06:03:10.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.303+0000 7f4c6e525640 1 -- 172.21.15.32:0/552790968 wait complete. 2023-12-20T06:03:10.305 INFO:tasks.ceph.mds.d.smithi039.stderr: -1740> 2023-12-20T06:02:42.120+0000 7ffbd7ed2640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T06:03:10.321 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-12-20T06:03:10.351 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T06:03:10.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa40148c640 1 Processor -- start 2023-12-20T06:03:10.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa40148c640 1 -- start start 2023-12-20T06:03:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa40148c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3fc065d40 0x7fa3fc0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa40148c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa3fc06d170 0x7fa3fc161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa40148c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3fc06a1f0 con 0x7fa3fc0cb050 2023-12-20T06:03:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa40148c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3fc161860 con 0x7fa3fc06d170 2023-12-20T06:03:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa40148c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc06a440 con 0x7fa3fc065d40 2023-12-20T06:03:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa3faffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa3fc0cb050 0x7fa3fc06ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56792/0 (socket says 172.21.15.32:56792) 2023-12-20T06:03:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa3faffd640 1 -- 172.21.15.32:0/2159006305 learned_addr learned my addr 172.21.15.32:0/2159006305 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:10.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa3fb7fe640 1 --2- 172.21.15.32:0/2159006305 >> v2:172.21.15.39:3300/0 conn(0x7fa3fc06d170 0x7fa3fc161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa3f3fff640 1 --2- 172.21.15.32:0/2159006305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3fc065d40 0x7fa3fc0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa3fa7fc640 1 -- 172.21.15.32:0/2159006305 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1235334013 0 0) 0x7fa3fc06a1f0 con 0x7fa3fc0cb050 2023-12-20T06:03:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa3fa7fc640 1 -- 172.21.15.32:0/2159006305 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa3e8003530 con 0x7fa3fc0cb050 2023-12-20T06:03:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa3fb7fe640 1 -- 172.21.15.32:0/2159006305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3fc065d40 msgr2=0x7fa3fc0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa3fb7fe640 1 --2- 172.21.15.32:0/2159006305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3fc065d40 0x7fa3fc0cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa3fb7fe640 1 -- 172.21.15.32:0/2159006305 >> v1:172.21.15.32:6789/0 conn(0x7fa3fc0cb050 legacy=0x7fa3fc06ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:10.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.471+0000 7fa3fb7fe640 1 -- 172.21.15.32:0/2159006305 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3fc06a720 con 0x7fa3fc06d170 2023-12-20T06:03:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa3fb7fe640 1 --2- 172.21.15.32:0/2159006305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3fc06d170 0x7fa3fc161320 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009d90 tx=0x7fa3f401c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=476fe51d6897a2e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa3fa7fc640 1 -- 172.21.15.32:0/2159006305 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40a59e0 con 0x7fa3fc06d170 2023-12-20T06:03:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa3fa7fc640 1 -- 172.21.15.32:0/2159006305 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3f40a5b40 con 0x7fa3fc06d170 2023-12-20T06:03:10.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa3fa7fc640 1 -- 172.21.15.32:0/2159006305 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3f40ad440 con 0x7fa3fc06d170 2023-12-20T06:03:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa40148c640 1 -- 172.21.15.32:0/2159006305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3fc06d170 msgr2=0x7fa3fc161320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa40148c640 1 --2- 172.21.15.32:0/2159006305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3fc06d170 0x7fa3fc161320 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7fa3f4009d90 tx=0x7fa3f401c920 comp rx=0 tx=0).stop 2023-12-20T06:03:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa40148c640 1 -- 172.21.15.32:0/2159006305 shutdown_connections 2023-12-20T06:03:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa40148c640 1 --2- 172.21.15.32:0/2159006305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3fc06d170 0x7fa3fc161320 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa40148c640 1 --2- 172.21.15.32:0/2159006305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3fc065d40 0x7fa3fc0cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa40148c640 1 -- 172.21.15.32:0/2159006305 >> 172.21.15.32:0/2159006305 conn(0x7fa3fc0cf8d0 msgr2=0x7fa3fc0cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa40148c640 1 -- 172.21.15.32:0/2159006305 shutdown_connections 2023-12-20T06:03:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.475+0000 7fa40148c640 1 -- 172.21.15.32:0/2159006305 wait complete. 2023-12-20T06:03:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa40148c640 1 Processor -- start 2023-12-20T06:03:10.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa40148c640 1 -- start start 2023-12-20T06:03:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa40148c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3fc065d40 0x7fa3fc152d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa40148c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3fc06d170 0x7fa3fc151400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa40148c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3fc0cb050 0x7fa3fc15b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa40148c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc158bc0 con 0x7fa3fc06d170 2023-12-20T06:03:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa40148c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc151c70 con 0x7fa3fc0cb050 2023-12-20T06:03:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa40148c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3fc151f50 con 0x7fa3fc065d40 2023-12-20T06:03:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3fc06d170 0x7fa3fc151400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3fc06d170 0x7fa3fc151400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52084/0 (socket says 172.21.15.32:52084) 2023-12-20T06:03:10.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f3fff640 1 -- 172.21.15.32:0/283808576 learned_addr learned my addr 172.21.15.32:0/283808576 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3faffd640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3fc065d40 0x7fa3fc152d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3fb7fe640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3fc0cb050 0x7fa3fc15b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f3fff640 1 -- 172.21.15.32:0/283808576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3fc065d40 msgr2=0x7fa3fc152d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f3fff640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3fc065d40 0x7fa3fc152d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f3fff640 1 -- 172.21.15.32:0/283808576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3fc0cb050 msgr2=0x7fa3fc15b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f3fff640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3fc0cb050 0x7fa3fc15b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f3fff640 1 -- 172.21.15.32:0/283808576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3f4009a70 con 0x7fa3fc06d170 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f3fff640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3fc06d170 0x7fa3fc151400 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400e840 tx=0x7fa3e400ec40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f37fe640 1 -- 172.21.15.32:0/283808576 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e4009510 con 0x7fa3fc06d170 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f37fe640 1 -- 172.21.15.32:0/283808576 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3e400ee90 con 0x7fa3fc06d170 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa3f37fe640 1 -- 172.21.15.32:0/283808576 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3e400cb80 con 0x7fa3fc06d170 2023-12-20T06:03:10.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa40148c640 1 -- 172.21.15.32:0/283808576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3fc152230 con 0x7fa3fc06d170 2023-12-20T06:03:10.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.479+0000 7fa40148c640 1 -- 172.21.15.32:0/283808576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3fc1527f0 con 0x7fa3fc06d170 2023-12-20T06:03:10.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.483+0000 7fa40148c640 1 -- 172.21.15.32:0/283808576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3fc071840 con 0x7fa3fc06d170 2023-12-20T06:03:10.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.483+0000 7fa3f37fe640 1 -- 172.21.15.32:0/283808576 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fa3e400cce0 con 0x7fa3fc06d170 2023-12-20T06:03:10.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.483+0000 7fa3f37fe640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3dc065ca0 0x7fa3dc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:10.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.483+0000 7fa3f37fe640 1 -- 172.21.15.32:0/283808576 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6680+0+0 (secure 0 0 0) 0x7fa3e4083dd0 con 0x7fa3fc06d170 2023-12-20T06:03:10.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.483+0000 7fa3faffd640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3dc065ca0 0x7fa3dc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:10.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.483+0000 7fa3faffd640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3dc065ca0 0x7fa3dc068160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0011f0 tx=0x7fa3ec007d40 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:10.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.487+0000 7fa3f37fe640 1 -- 172.21.15.32:0/283808576 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa3e4051070 con 0x7fa3fc06d170 2023-12-20T06:03:10.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:10.687+0000 7fa40148c640 1 -- 172.21.15.32:0/283808576 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7fa3fc06afc0 con 0x7fa3fc06d170 2023-12-20T06:03:11.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.303+0000 7fa3f37fe640 1 -- 172.21.15.32:0/283808576 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v106) v1 ==== 107+0+0 (secure 0 0 0) 0x7fa3e40210b0 con 0x7fa3fc06d170 2023-12-20T06:03:11.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 -- 172.21.15.32:0/283808576 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3dc065ca0 msgr2=0x7fa3dc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:11.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3dc065ca0 0x7fa3dc068160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7fa3ec0011f0 tx=0x7fa3ec007d40 comp rx=0 tx=0).stop 2023-12-20T06:03:11.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 -- 172.21.15.32:0/283808576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3fc06d170 msgr2=0x7fa3fc151400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:11.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3fc06d170 0x7fa3fc151400 secure :-1 s=READY pgs=2193 cs=0 l=1 rev1=1 crypto rx=0x7fa3e400e840 tx=0x7fa3e400ec40 comp rx=0 tx=0).stop 2023-12-20T06:03:11.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 -- 172.21.15.32:0/283808576 shutdown_connections 2023-12-20T06:03:11.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3dc065ca0 0x7fa3dc068160 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:11.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3fc0cb050 0x7fa3fc15b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:11.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3fc06d170 0x7fa3fc151400 unknown :-1 s=CLOSED pgs=2193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:11.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 --2- 172.21.15.32:0/283808576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3fc065d40 0x7fa3fc152d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:11.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 -- 172.21.15.32:0/283808576 >> 172.21.15.32:0/283808576 conn(0x7fa3fc0cf8d0 msgr2=0x7fa3fc0cb430 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:11.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 -- 172.21.15.32:0/283808576 shutdown_connections 2023-12-20T06:03:11.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.307+0000 7fa40148c640 1 -- 172.21.15.32:0/283808576 wait complete. 2023-12-20T06:03:11.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-12-20T06:03:11.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddba9a2640 1 Processor -- start 2023-12-20T06:03:11.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddba9a2640 1 -- start start 2023-12-20T06:03:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddba9a2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fddb415a810 0x7fddb40b13b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddba9a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb40701b0 0x7fddb40705f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddba9a2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddb40caf10 0x7fddb40cb3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddba9a2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb4070b30 con 0x7fddb40caf10 2023-12-20T06:03:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddba9a2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb40ca220 con 0x7fddb415a810 2023-12-20T06:03:11.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddba9a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb40ca750 con 0x7fddb40701b0 2023-12-20T06:03:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddba1a1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddb40caf10 0x7fddb40cb3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddb99a0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fddb415a810 0x7fddb40b13b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddb99a0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fddb415a810 0x7fddb40b13b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48594/0 (socket says 172.21.15.32:48594) 2023-12-20T06:03:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddba1a1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddb40caf10 0x7fddb40cb3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52094/0 (socket says 172.21.15.32:52094) 2023-12-20T06:03:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddb99a0640 1 -- 172.21.15.32:0/1194951857 learned_addr learned my addr 172.21.15.32:0/1194951857 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddb919f640 1 --2- 172.21.15.32:0/1194951857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb40701b0 0x7fddb40705f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddb99a0640 1 -- 172.21.15.32:0/1194951857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb40701b0 msgr2=0x7fddb40705f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddb99a0640 1 --2- 172.21.15.32:0/1194951857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb40701b0 0x7fddb40705f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddb99a0640 1 -- 172.21.15.32:0/1194951857 >> v2:172.21.15.32:3300/0 conn(0x7fddb40caf10 msgr2=0x7fddb40cb3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:11.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddb99a0640 1 --2- 172.21.15.32:0/1194951857 >> v2:172.21.15.32:3300/0 conn(0x7fddb40caf10 0x7fddb40cb3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddb99a0640 1 -- 172.21.15.32:0/1194951857 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb40cb8e0 con 0x7fddb415a810 2023-12-20T06:03:11.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.487+0000 7fddba1a1640 1 --2- 172.21.15.32:0/1194951857 >> v2:172.21.15.32:3300/0 conn(0x7fddb40caf10 0x7fddb40cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddb99a0640 1 --2- 172.21.15.32:0/1194951857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb415a810 0x7fddb40b13b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fddb406e410 tx=0x7fddb001c470 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f5256e15042a5eaa server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddb899e640 1 -- 172.21.15.32:0/1194951857 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00ac440 con 0x7fddb415a810 2023-12-20T06:03:11.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddb899e640 1 -- 172.21.15.32:0/1194951857 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb00ac5a0 con 0x7fddb415a810 2023-12-20T06:03:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddb899e640 1 -- 172.21.15.32:0/1194951857 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00ac930 con 0x7fddb415a810 2023-12-20T06:03:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddba9a2640 1 -- 172.21.15.32:0/1194951857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb415a810 msgr2=0x7fddb40b13b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddba9a2640 1 --2- 172.21.15.32:0/1194951857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb415a810 0x7fddb40b13b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7fddb406e410 tx=0x7fddb001c470 comp rx=0 tx=0).stop 2023-12-20T06:03:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddba9a2640 1 -- 172.21.15.32:0/1194951857 shutdown_connections 2023-12-20T06:03:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddba9a2640 1 --2- 172.21.15.32:0/1194951857 >> v2:172.21.15.32:3300/0 conn(0x7fddb40caf10 0x7fddb40cb3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddba9a2640 1 --2- 172.21.15.32:0/1194951857 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb40701b0 0x7fddb40705f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddba9a2640 1 --2- 172.21.15.32:0/1194951857 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb415a810 0x7fddb40b13b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddba9a2640 1 -- 172.21.15.32:0/1194951857 >> 172.21.15.32:0/1194951857 conn(0x7fddb406a4e0 msgr2=0x7fddb406bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddba9a2640 1 -- 172.21.15.32:0/1194951857 shutdown_connections 2023-12-20T06:03:11.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.491+0000 7fddba9a2640 1 -- 172.21.15.32:0/1194951857 wait complete. 2023-12-20T06:03:11.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddba9a2640 1 Processor -- start 2023-12-20T06:03:11.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddba9a2640 1 -- start start 2023-12-20T06:03:11.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddba9a2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40701b0 0x7fddb40ae9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddba9a2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb40caf10 0x7fddb4163b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddba9a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415a810 0x7fddb41573d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddba9a2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb41f4e90 con 0x7fddb40701b0 2023-12-20T06:03:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddba9a2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb40aef10 con 0x7fddb40caf10 2023-12-20T06:03:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddba9a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb40c5c00 con 0x7fddb415a810 2023-12-20T06:03:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddb99a0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40701b0 0x7fddb40ae9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddb99a0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40701b0 0x7fddb40ae9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52110/0 (socket says 172.21.15.32:52110) 2023-12-20T06:03:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddb99a0640 1 -- 172.21.15.32:0/2920545655 learned_addr learned my addr 172.21.15.32:0/2920545655 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddb919f640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb40caf10 0x7fddb4163b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:11.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddba1a1640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415a810 0x7fddb41573d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddb99a0640 1 -- 172.21.15.32:0/2920545655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415a810 msgr2=0x7fddb41573d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddb99a0640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415a810 0x7fddb41573d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddb99a0640 1 -- 172.21.15.32:0/2920545655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb40caf10 msgr2=0x7fddb4163b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddb99a0640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb40caf10 0x7fddb4163b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddb99a0640 1 -- 172.21.15.32:0/2920545655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac009a70 con 0x7fddb40701b0 2023-12-20T06:03:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddb919f640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb40caf10 0x7fddb4163b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddb99a0640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40701b0 0x7fddb40ae9d0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7fddb00af040 tx=0x7fddb0019210 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddaa7fc640 1 -- 172.21.15.32:0/2920545655 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00ac440 con 0x7fddb40701b0 2023-12-20T06:03:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddaa7fc640 1 -- 172.21.15.32:0/2920545655 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb00b7680 con 0x7fddb40701b0 2023-12-20T06:03:11.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddaa7fc640 1 -- 172.21.15.32:0/2920545655 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00b7a30 con 0x7fddb40701b0 2023-12-20T06:03:11.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.495+0000 7fddba9a2640 1 -- 172.21.15.32:0/2920545655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb00094d0 con 0x7fddb40701b0 2023-12-20T06:03:11.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.499+0000 7fddba9a2640 1 -- 172.21.15.32:0/2920545655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb40c61c0 con 0x7fddb40701b0 2023-12-20T06:03:11.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.499+0000 7fddba9a2640 1 -- 172.21.15.32:0/2920545655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb4074100 con 0x7fddb40701b0 2023-12-20T06:03:11.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.499+0000 7fddaa7fc640 1 -- 172.21.15.32:0/2920545655 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fddb00b7b90 con 0x7fddb40701b0 2023-12-20T06:03:11.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.499+0000 7fddaa7fc640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065cd0 0x7fdd98068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:11.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.499+0000 7fddaa7fc640 1 -- 172.21.15.32:0/2920545655 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(219..219 src has 1..219) v4 ==== 6680+0+0 (secure 0 0 0) 0x7fddb011e430 con 0x7fddb40701b0 2023-12-20T06:03:11.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.499+0000 7fddb919f640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065cd0 0x7fdd98068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:11.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.499+0000 7fddb919f640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065cd0 0x7fdd98068190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fddb40c2020 tx=0x7fdda4008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:11.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.503+0000 7fddaa7fc640 1 -- 172.21.15.32:0/2920545655 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fddb00eb620 con 0x7fddb40701b0 2023-12-20T06:03:11.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:11.663+0000 7fddba9a2640 1 -- 172.21.15.32:0/2920545655 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fddb4070770 con 0x7fddb40701b0 2023-12-20T06:03:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.311+0000 7fddaa7fc640 1 -- 172.21.15.32:0/2920545655 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v220) v1 ==== 190+0+0 (secure 0 0 0) 0x7fddb00eec40 con 0x7fddb40701b0 2023-12-20T06:03:12.311 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'backup_fs_metadata' removed 2023-12-20T06:03:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 -- 172.21.15.32:0/2920545655 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065cd0 msgr2=0x7fdd98068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065cd0 0x7fdd98068190 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fddb40c2020 tx=0x7fdda4008040 comp rx=0 tx=0).stop 2023-12-20T06:03:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 -- 172.21.15.32:0/2920545655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40701b0 msgr2=0x7fddb40ae9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40701b0 0x7fddb40ae9d0 secure :-1 s=READY pgs=2194 cs=0 l=1 rev1=1 crypto rx=0x7fddb00af040 tx=0x7fddb0019210 comp rx=0 tx=0).stop 2023-12-20T06:03:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 -- 172.21.15.32:0/2920545655 shutdown_connections 2023-12-20T06:03:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065cd0 0x7fdd98068190 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:12.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415a810 0x7fddb41573d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb40caf10 0x7fddb4163b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 --2- 172.21.15.32:0/2920545655 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40701b0 0x7fddb40ae9d0 unknown :-1 s=CLOSED pgs=2194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 -- 172.21.15.32:0/2920545655 >> 172.21.15.32:0/2920545655 conn(0x7fddb406a4e0 msgr2=0x7fddb40b0830 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 -- 172.21.15.32:0/2920545655 shutdown_connections 2023-12-20T06:03:12.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.315+0000 7fddba9a2640 1 -- 172.21.15.32:0/2920545655 wait complete. 2023-12-20T06:03:12.336 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-12-20T06:03:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb8ab6640 1 Processor -- start 2023-12-20T06:03:12.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb8ab6640 1 -- start start 2023-12-20T06:03:12.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb8ab6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbeb4158520 0x7fbeb406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:12.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb8ab6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb4070230 0x7fbeb40706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:12.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb8ab6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbeb40ca7f0 con 0x7fbeb40ca050 2023-12-20T06:03:12.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb8ab6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbeb4070be0 con 0x7fbeb4158520 2023-12-20T06:03:12.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb8ab6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40caad0 con 0x7fbeb4070230 2023-12-20T06:03:12.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb259c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbeb4158520 0x7fbeb406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:12.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb2d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb4070230 0x7fbeb40706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:12.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeabfff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fbeb40ca050 0x7fbeb415ef80 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:44528/0 (socket says 172.21.15.32:44528) 2023-12-20T06:03:12.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeabfff640 1 -- 172.21.15.32:0/2855612494 learned_addr learned my addr 172.21.15.32:0/2855612494 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:12.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb259c640 1 -- 172.21.15.32:0/2855612494 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb4070230 msgr2=0x7fbeb40706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:12.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb259c640 1 --2- 172.21.15.32:0/2855612494 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb4070230 0x7fbeb40706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:12.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb259c640 1 -- 172.21.15.32:0/2855612494 >> v1:172.21.15.39:6789/0 conn(0x7fbeb40ca050 legacy=0x7fbeb415ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:12.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb259c640 1 -- 172.21.15.32:0/2855612494 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeb40cadb0 con 0x7fbeb4158520 2023-12-20T06:03:12.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.487+0000 7fbeb1d9b640 1 -- 172.21.15.32:0/2855612494 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3065013450 0 0) 0x7fbeb40ca7f0 con 0x7fbeb40ca050 2023-12-20T06:03:12.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb2d9d640 1 --2- 172.21.15.32:0/2855612494 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb4070230 0x7fbeb40706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:12.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb259c640 1 --2- 172.21.15.32:0/2855612494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4158520 0x7fbeb406dcb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbea40097c0 tx=0x7fbea401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a52e39c666efbbdb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:12.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb1d9b640 1 -- 172.21.15.32:0/2855612494 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40ac620 con 0x7fbeb4158520 2023-12-20T06:03:12.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb1d9b640 1 -- 172.21.15.32:0/2855612494 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea40ac780 con 0x7fbeb4158520 2023-12-20T06:03:12.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb1d9b640 1 -- 172.21.15.32:0/2855612494 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40acb10 con 0x7fbeb4158520 2023-12-20T06:03:12.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/2855612494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4158520 msgr2=0x7fbeb406dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:12.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb8ab6640 1 --2- 172.21.15.32:0/2855612494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4158520 0x7fbeb406dcb0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7fbea40097c0 tx=0x7fbea401c3c0 comp rx=0 tx=0).stop 2023-12-20T06:03:12.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/2855612494 shutdown_connections 2023-12-20T06:03:12.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb8ab6640 1 --2- 172.21.15.32:0/2855612494 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb4070230 0x7fbeb40706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:12.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb8ab6640 1 --2- 172.21.15.32:0/2855612494 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb4158520 0x7fbeb406dcb0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:12.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/2855612494 >> 172.21.15.32:0/2855612494 conn(0x7fbeb406a510 msgr2=0x7fbeb40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:12.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/2855612494 shutdown_connections 2023-12-20T06:03:12.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.491+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/2855612494 wait complete. 2023-12-20T06:03:12.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb8ab6640 1 Processor -- start 2023-12-20T06:03:12.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb8ab6640 1 -- start start 2023-12-20T06:03:12.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb8ab6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb4070230 0x7fbeb414a9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:12.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb8ab6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb40ca050 0x7fbeb4153220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:12.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb8ab6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb4158520 0x7fbeb4153760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:12.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb8ab6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40ce2e0 con 0x7fbeb4070230 2023-12-20T06:03:12.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb8ab6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb415e390 con 0x7fbeb40ca050 2023-12-20T06:03:12.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb8ab6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbeb40cd6a0 con 0x7fbeb4158520 2023-12-20T06:03:12.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb4070230 0x7fbeb414a9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:12.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb4070230 0x7fbeb414a9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52120/0 (socket says 172.21.15.32:52120) 2023-12-20T06:03:12.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb259c640 1 -- 172.21.15.32:0/3276562962 learned_addr learned my addr 172.21.15.32:0/3276562962 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:12.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb259c640 1 -- 172.21.15.32:0/3276562962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb4158520 msgr2=0x7fbeb4153760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:12.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb259c640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb4158520 0x7fbeb4153760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:12.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb259c640 1 -- 172.21.15.32:0/3276562962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb40ca050 msgr2=0x7fbeb4153220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:12.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb259c640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb40ca050 0x7fbeb4153220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:12.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb259c640 1 -- 172.21.15.32:0/3276562962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbeac009a70 con 0x7fbeb4070230 2023-12-20T06:03:12.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.495+0000 7fbeb259c640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb4070230 0x7fbeb414a9e0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fbea40af040 tx=0x7fbea401ac00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:12.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.499+0000 7fbeaaffd640 1 -- 172.21.15.32:0/3276562962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40ac620 con 0x7fbeb4070230 2023-12-20T06:03:12.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.499+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/3276562962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbea4009420 con 0x7fbeb4070230 2023-12-20T06:03:12.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.499+0000 7fbeaaffd640 1 -- 172.21.15.32:0/3276562962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbea40b8450 con 0x7fbeb4070230 2023-12-20T06:03:12.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.499+0000 7fbeaaffd640 1 -- 172.21.15.32:0/3276562962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbea40b8820 con 0x7fbeb4070230 2023-12-20T06:03:12.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.499+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/3276562962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbeb40cdc20 con 0x7fbeb4070230 2023-12-20T06:03:12.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.499+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/3276562962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbeb404e000 con 0x7fbeb4070230 2023-12-20T06:03:12.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.499+0000 7fbeaaffd640 1 -- 172.21.15.32:0/3276562962 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fbea40aa070 con 0x7fbeb4070230 2023-12-20T06:03:12.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.499+0000 7fbeaaffd640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbe94065ca0 0x7fbe94068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:12.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.499+0000 7fbeaaffd640 1 -- 172.21.15.32:0/3276562962 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(220..220 src has 1..220) v4 ==== 6236+0+0 (secure 0 0 0) 0x7fbea411ded0 con 0x7fbeb4070230 2023-12-20T06:03:12.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.499+0000 7fbeabfff640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbe94065ca0 0x7fbe94068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:12.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.499+0000 7fbeabfff640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbe94065ca0 0x7fbe94068160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0011f0 tx=0x7fbe9c007930 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:12.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.503+0000 7fbeaaffd640 1 -- 172.21.15.32:0/3276562962 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbea40eb290 con 0x7fbeb4070230 2023-12-20T06:03:12.661 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:12.663+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/3276562962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fbeb40b1f90 con 0x7fbeb4070230 2023-12-20T06:03:12.680 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:03:12.684 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:03:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.315+0000 7fbeaaffd640 1 -- 172.21.15.32:0/3276562962 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v221) v1 ==== 178+0+0 (secure 0 0 0) 0x7fbea40b10d0 con 0x7fbeb4070230 2023-12-20T06:03:13.313 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'backup_fs_data' removed 2023-12-20T06:03:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.315+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/3276562962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbe94065ca0 msgr2=0x7fbe94068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:13.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.315+0000 7fbeb8ab6640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbe94065ca0 0x7fbe94068160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fbe9c0011f0 tx=0x7fbe9c007930 comp rx=0 tx=0).stop 2023-12-20T06:03:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.315+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/3276562962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb4070230 msgr2=0x7fbeb414a9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.315+0000 7fbeb8ab6640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb4070230 0x7fbeb414a9e0 secure :-1 s=READY pgs=2195 cs=0 l=1 rev1=1 crypto rx=0x7fbea40af040 tx=0x7fbea401ac00 comp rx=0 tx=0).stop 2023-12-20T06:03:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.315+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/3276562962 shutdown_connections 2023-12-20T06:03:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.315+0000 7fbeb8ab6640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbe94065ca0 0x7fbe94068160 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.315+0000 7fbeb8ab6640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbeb4158520 0x7fbeb4153760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.315+0000 7fbeb8ab6640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbeb40ca050 0x7fbeb4153220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.315+0000 7fbeb8ab6640 1 --2- 172.21.15.32:0/3276562962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbeb4070230 0x7fbeb414a9e0 unknown :-1 s=CLOSED pgs=2195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.315+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/3276562962 >> 172.21.15.32:0/3276562962 conn(0x7fbeb406a510 msgr2=0x7fbeb4152580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:13.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.319+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/3276562962 shutdown_connections 2023-12-20T06:03:13.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.319+0000 7fbeb8ab6640 1 -- 172.21.15.32:0/3276562962 wait complete. 2023-12-20T06:03:13.342 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T06:03:13.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.487+0000 7f36d8eb6640 1 Processor -- start 2023-12-20T06:03:13.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.487+0000 7f36d8eb6640 1 -- start start 2023-12-20T06:03:13.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36d8eb6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36d40b0cb0 0x7f36d40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36d8eb6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f36d4070230 0x7f36d4070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36d8eb6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f36d4070bc0 con 0x7f36d415abb0 2023-12-20T06:03:13.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36d8eb6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d406e160 con 0x7f36d40b0cb0 2023-12-20T06:03:13.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36d8eb6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f36d415b6a0 con 0x7f36d4070230 2023-12-20T06:03:13.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36cbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36d40b0cb0 0x7f36d40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36cbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f36d40b0cb0 0x7f36d40caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52132/0 (socket says 172.21.15.32:52132) 2023-12-20T06:03:13.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36cbfff640 1 -- 172.21.15.32:0/747919750 learned_addr learned my addr 172.21.15.32:0/747919750 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:13.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36d2d9d640 1 --2- 172.21.15.32:0/747919750 >> v2:172.21.15.39:3300/0 conn(0x7f36d4070230 0x7f36d4070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36cbfff640 1 -- 172.21.15.32:0/747919750 >> v2:172.21.15.39:3300/0 conn(0x7f36d4070230 msgr2=0x7f36d4070680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:13.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36cbfff640 1 --2- 172.21.15.32:0/747919750 >> v2:172.21.15.39:3300/0 conn(0x7f36d4070230 0x7f36d4070680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36cbfff640 1 -- 172.21.15.32:0/747919750 >> v1:172.21.15.39:6789/0 conn(0x7f36d415abb0 legacy=0x7f36d415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:13.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.491+0000 7f36cbfff640 1 -- 172.21.15.32:0/747919750 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36d406d8c0 con 0x7f36d40b0cb0 2023-12-20T06:03:13.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36cbfff640 1 --2- 172.21.15.32:0/747919750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d40b0cb0 0x7f36d40caaf0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f36d40b0c40 tx=0x7f36bc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e1ec0f2b1a0e243 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:13.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d1d9b640 1 -- 172.21.15.32:0/747919750 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0aa660 con 0x7f36d40b0cb0 2023-12-20T06:03:13.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d1d9b640 1 -- 172.21.15.32:0/747919750 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36bc0aa7c0 con 0x7f36d40b0cb0 2023-12-20T06:03:13.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d1d9b640 1 -- 172.21.15.32:0/747919750 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36bc0aab50 con 0x7f36d40b0cb0 2023-12-20T06:03:13.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d8eb6640 1 -- 172.21.15.32:0/747919750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d40b0cb0 msgr2=0x7f36d40caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:13.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d8eb6640 1 --2- 172.21.15.32:0/747919750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d40b0cb0 0x7f36d40caaf0 secure :-1 s=READY pgs=2196 cs=0 l=1 rev1=1 crypto rx=0x7f36d40b0c40 tx=0x7f36bc01c500 comp rx=0 tx=0).stop 2023-12-20T06:03:13.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d8eb6640 1 -- 172.21.15.32:0/747919750 shutdown_connections 2023-12-20T06:03:13.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d8eb6640 1 --2- 172.21.15.32:0/747919750 >> v2:172.21.15.39:3300/0 conn(0x7f36d4070230 0x7f36d4070680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d8eb6640 1 --2- 172.21.15.32:0/747919750 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d40b0cb0 0x7f36d40caaf0 unknown :-1 s=CLOSED pgs=2196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d8eb6640 1 -- 172.21.15.32:0/747919750 >> 172.21.15.32:0/747919750 conn(0x7f36d406a510 msgr2=0x7f36d40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:13.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d8eb6640 1 -- 172.21.15.32:0/747919750 shutdown_connections 2023-12-20T06:03:13.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d8eb6640 1 -- 172.21.15.32:0/747919750 wait complete. 2023-12-20T06:03:13.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d8eb6640 1 Processor -- start 2023-12-20T06:03:13.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.495+0000 7f36d8eb6640 1 -- start start 2023-12-20T06:03:13.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d8eb6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36d4070230 0x7f36d4135c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d8eb6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36d40b0cb0 0x7f36d414a760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d8eb6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d415abb0 0x7f36d41476b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d8eb6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d414aca0 con 0x7f36d415abb0 2023-12-20T06:03:13.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d8eb6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f36d4147bf0 con 0x7f36d40b0cb0 2023-12-20T06:03:13.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d8eb6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f36d414ee70 con 0x7f36d4070230 2023-12-20T06:03:13.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d415abb0 0x7f36d41476b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d2d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d415abb0 0x7f36d41476b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52142/0 (socket says 172.21.15.32:52142) 2023-12-20T06:03:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d2d9d640 1 -- 172.21.15.32:0/2450594040 learned_addr learned my addr 172.21.15.32:0/2450594040 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36cbfff640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36d40b0cb0 0x7f36d414a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d259c640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36d4070230 0x7f36d4135c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d2d9d640 1 -- 172.21.15.32:0/2450594040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36d4070230 msgr2=0x7f36d4135c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d2d9d640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36d4070230 0x7f36d4135c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d2d9d640 1 -- 172.21.15.32:0/2450594040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36d40b0cb0 msgr2=0x7f36d414a760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d2d9d640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36d40b0cb0 0x7f36d414a760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d2d9d640 1 -- 172.21.15.32:0/2450594040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f36bc0094d0 con 0x7f36d415abb0 2023-12-20T06:03:13.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d2d9d640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d415abb0 0x7f36d41476b0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007e40 tx=0x7f36cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:13.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36caffd640 1 -- 172.21.15.32:0/2450594040 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc014620 con 0x7f36d415abb0 2023-12-20T06:03:13.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36caffd640 1 -- 172.21.15.32:0/2450594040 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f36cc00ee90 con 0x7f36d415abb0 2023-12-20T06:03:13.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36caffd640 1 -- 172.21.15.32:0/2450594040 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f36cc0149e0 con 0x7f36d415abb0 2023-12-20T06:03:13.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d8eb6640 1 -- 172.21.15.32:0/2450594040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f36d414f150 con 0x7f36d415abb0 2023-12-20T06:03:13.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d8eb6640 1 -- 172.21.15.32:0/2450594040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f36d414f710 con 0x7f36d415abb0 2023-12-20T06:03:13.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36d8eb6640 1 -- 172.21.15.32:0/2450594040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f36d404e000 con 0x7f36d415abb0 2023-12-20T06:03:13.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.499+0000 7f36caffd640 1 -- 172.21.15.32:0/2450594040 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f36cc014b40 con 0x7f36d415abb0 2023-12-20T06:03:13.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.503+0000 7f36caffd640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36b4065ca0 0x7f36b4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.503+0000 7f36caffd640 1 -- 172.21.15.32:0/2450594040 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 5816+0+0 (secure 0 0 0) 0x7f36cc083df0 con 0x7f36d415abb0 2023-12-20T06:03:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.503+0000 7f36d259c640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36b4065ca0 0x7f36b4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.503+0000 7f36d259c640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36b4065ca0 0x7f36b4068160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f36c4007fd0 tx=0x7f36c4009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:13.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.507+0000 7f36caffd640 1 -- 172.21.15.32:0/2450594040 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f36cc051540 con 0x7f36d415abb0 2023-12-20T06:03:13.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.663+0000 7f36d8eb6640 1 -- 172.21.15.32:0/2450594040 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f36d40c4150 con 0x7f36d415abb0 2023-12-20T06:03:13.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.667+0000 7f36caffd640 1 -- 172.21.15.32:0/2450594040 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v73) v1 ==== 120+0+0 (secure 0 0 0) 0x7f36cc054b60 con 0x7f36d415abb0 2023-12-20T06:03:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 -- 172.21.15.32:0/2450594040 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36b4065ca0 msgr2=0x7f36b4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:13.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36b4065ca0 0x7f36b4068160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f36c4007fd0 tx=0x7f36c4009040 comp rx=0 tx=0).stop 2023-12-20T06:03:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 -- 172.21.15.32:0/2450594040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d415abb0 msgr2=0x7f36d41476b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d415abb0 0x7f36d41476b0 secure :-1 s=READY pgs=2197 cs=0 l=1 rev1=1 crypto rx=0x7f36cc007e40 tx=0x7f36cc00e4b0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 -- 172.21.15.32:0/2450594040 shutdown_connections 2023-12-20T06:03:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f36b4065ca0 0x7f36b4068160 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f36d415abb0 0x7f36d41476b0 unknown :-1 s=CLOSED pgs=2197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f36d40b0cb0 0x7f36d414a760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 --2- 172.21.15.32:0/2450594040 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f36d4070230 0x7f36d4135c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 -- 172.21.15.32:0/2450594040 >> 172.21.15.32:0/2450594040 conn(0x7f36d406a510 msgr2=0x7f36d406cc80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 -- 172.21.15.32:0/2450594040 shutdown_connections 2023-12-20T06:03:13.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.671+0000 7f36d8eb6640 1 -- 172.21.15.32:0/2450594040 wait complete. 2023-12-20T06:03:13.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T06:03:13.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.835+0000 7f93ed72c640 1 Processor -- start 2023-12-20T06:03:13.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.835+0000 7f93ed72c640 1 -- start start 2023-12-20T06:03:13.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93ed72c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f93e815ce60 0x7f93e80d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93ed72c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93e80d3130 0x7f93e80d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93ed72c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93e80d1300 0x7f93e80b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93ed72c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e80d35a0 con 0x7f93e80d3130 2023-12-20T06:03:13.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93ed72c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f93e80d2650 con 0x7f93e815ce60 2023-12-20T06:03:13.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93ed72c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e80d1a40 con 0x7f93e80d1300 2023-12-20T06:03:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93dffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93e80d3130 0x7f93e80d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93dffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f93e80d3130 0x7f93e80d0dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52152/0 (socket says 172.21.15.32:52152) 2023-12-20T06:03:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93dffff640 1 -- 172.21.15.32:0/2548412977 learned_addr learned my addr 172.21.15.32:0/2548412977 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93e6ffd640 1 --2- 172.21.15.32:0/2548412977 >> v2:172.21.15.39:3300/0 conn(0x7f93e815ce60 0x7f93e80d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93e77fe640 1 --2- 172.21.15.32:0/2548412977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93e80d1300 0x7f93e80b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93dffff640 1 -- 172.21.15.32:0/2548412977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93e80d1300 msgr2=0x7f93e80b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93dffff640 1 --2- 172.21.15.32:0/2548412977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93e80d1300 0x7f93e80b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93dffff640 1 -- 172.21.15.32:0/2548412977 >> v2:172.21.15.39:3300/0 conn(0x7f93e815ce60 msgr2=0x7f93e80d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93dffff640 1 --2- 172.21.15.32:0/2548412977 >> v2:172.21.15.39:3300/0 conn(0x7f93e815ce60 0x7f93e80d2bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93dffff640 1 -- 172.21.15.32:0/2548412977 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93e80d1d20 con 0x7f93e80d3130 2023-12-20T06:03:13.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.839+0000 7f93dffff640 1 --2- 172.21.15.32:0/2548412977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93e80d3130 0x7f93e80d0dc0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f93e80b5820 tx=0x7f93d001ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b517f0dd192f7dd4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:13.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93e67fc640 1 -- 172.21.15.32:0/2548412977 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d00a89b0 con 0x7f93e80d3130 2023-12-20T06:03:13.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93e67fc640 1 -- 172.21.15.32:0/2548412977 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93d00a8b10 con 0x7f93e80d3130 2023-12-20T06:03:13.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93e67fc640 1 -- 172.21.15.32:0/2548412977 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93d00a8ea0 con 0x7f93e80d3130 2023-12-20T06:03:13.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 -- 172.21.15.32:0/2548412977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93e80d3130 msgr2=0x7f93e80d0dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:13.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 --2- 172.21.15.32:0/2548412977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93e80d3130 0x7f93e80d0dc0 secure :-1 s=READY pgs=2198 cs=0 l=1 rev1=1 crypto rx=0x7f93e80b5820 tx=0x7f93d001ecf0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 -- 172.21.15.32:0/2548412977 shutdown_connections 2023-12-20T06:03:13.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 --2- 172.21.15.32:0/2548412977 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93e80d1300 0x7f93e80b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 --2- 172.21.15.32:0/2548412977 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93e80d3130 0x7f93e80d0dc0 unknown :-1 s=CLOSED pgs=2198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 --2- 172.21.15.32:0/2548412977 >> v2:172.21.15.39:3300/0 conn(0x7f93e815ce60 0x7f93e80d2bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 -- 172.21.15.32:0/2548412977 >> 172.21.15.32:0/2548412977 conn(0x7f93e806b5e0 msgr2=0x7f93e806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 -- 172.21.15.32:0/2548412977 shutdown_connections 2023-12-20T06:03:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 -- 172.21.15.32:0/2548412977 wait complete. 2023-12-20T06:03:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 Processor -- start 2023-12-20T06:03:13.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 -- start start 2023-12-20T06:03:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e80d1300 0x7f93e81520e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93e80d3130 0x7f93e8158810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93e815ce60 0x7f93e815a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e8156c90 con 0x7f93e815ce60 2023-12-20T06:03:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f93e81560c0 con 0x7f93e80d1300 2023-12-20T06:03:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.843+0000 7f93ed72c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f93e81562f0 con 0x7f93e80d3130 2023-12-20T06:03:13.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93e77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93e815ce60 0x7f93e815a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93e77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93e815ce60 0x7f93e815a480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52162/0 (socket says 172.21.15.32:52162) 2023-12-20T06:03:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93e77fe640 1 -- 172.21.15.32:0/4281254099 learned_addr learned my addr 172.21.15.32:0/4281254099 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93e6ffd640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e80d1300 0x7f93e81520e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93dffff640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93e80d3130 0x7f93e8158810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93e77fe640 1 -- 172.21.15.32:0/4281254099 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93e80d3130 msgr2=0x7f93e8158810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93e77fe640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93e80d3130 0x7f93e8158810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93e77fe640 1 -- 172.21.15.32:0/4281254099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e80d1300 msgr2=0x7f93e81520e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93e77fe640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e80d1300 0x7f93e81520e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93e77fe640 1 -- 172.21.15.32:0/4281254099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f93d00094d0 con 0x7f93e815ce60 2023-12-20T06:03:13.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93e77fe640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93e815ce60 0x7f93e815a480 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f93e000eed0 tx=0x7f93e000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93df7fe640 1 -- 172.21.15.32:0/4281254099 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0010040 con 0x7f93e815ce60 2023-12-20T06:03:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93df7fe640 1 -- 172.21.15.32:0/4281254099 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f93e000bd10 con 0x7f93e815ce60 2023-12-20T06:03:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93df7fe640 1 -- 172.21.15.32:0/4281254099 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f93e0014640 con 0x7f93e815ce60 2023-12-20T06:03:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93ed72c640 1 -- 172.21.15.32:0/4281254099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f93e81565d0 con 0x7f93e815ce60 2023-12-20T06:03:13.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93ed72c640 1 -- 172.21.15.32:0/4281254099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f93e8152620 con 0x7f93e815ce60 2023-12-20T06:03:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93ed72c640 1 -- 172.21.15.32:0/4281254099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f93e8156dd0 con 0x7f93e815ce60 2023-12-20T06:03:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.847+0000 7f93df7fe640 1 -- 172.21.15.32:0/4281254099 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f93e00147a0 con 0x7f93e815ce60 2023-12-20T06:03:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.851+0000 7f93df7fe640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93c8065ca0 0x7f93c8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:13.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.851+0000 7f93df7fe640 1 -- 172.21.15.32:0/4281254099 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 5816+0+0 (secure 0 0 0) 0x7f93e0083c70 con 0x7f93e815ce60 2023-12-20T06:03:13.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.851+0000 7f93e6ffd640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93c8065ca0 0x7f93c8068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:13.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.851+0000 7f93e6ffd640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93c8065ca0 0x7f93c8068160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f93e815b7e0 tx=0x7f93d8009190 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:13.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:13.851+0000 7f93df7fe640 1 -- 172.21.15.32:0/4281254099 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f93e8156dd0 con 0x7f93e815ce60 2023-12-20T06:03:14.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.011+0000 7f93ed72c640 1 -- 172.21.15.32:0/4281254099 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f93e8156dd0 con 0x7f93e815ce60 2023-12-20T06:03:14.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.015+0000 7f93df7fe640 1 -- 172.21.15.32:0/4281254099 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v74) v1 ==== 115+0+0 (secure 0 0 0) 0x7f93e8156dd0 con 0x7f93e815ce60 2023-12-20T06:03:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 -- 172.21.15.32:0/4281254099 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93c8065ca0 msgr2=0x7f93c8068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93c8065ca0 0x7f93c8068160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f93e815b7e0 tx=0x7f93d8009190 comp rx=0 tx=0).stop 2023-12-20T06:03:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 -- 172.21.15.32:0/4281254099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93e815ce60 msgr2=0x7f93e815a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93e815ce60 0x7f93e815a480 secure :-1 s=READY pgs=2199 cs=0 l=1 rev1=1 crypto rx=0x7f93e000eed0 tx=0x7f93e000b480 comp rx=0 tx=0).stop 2023-12-20T06:03:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 -- 172.21.15.32:0/4281254099 shutdown_connections 2023-12-20T06:03:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f93c8065ca0 0x7f93c8068160 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f93e815ce60 0x7f93e815a480 unknown :-1 s=CLOSED pgs=2199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f93e80d3130 0x7f93e8158810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 --2- 172.21.15.32:0/4281254099 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f93e80d1300 0x7f93e81520e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 -- 172.21.15.32:0/4281254099 >> 172.21.15.32:0/4281254099 conn(0x7f93e806b5e0 msgr2=0x7f93e80b36a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 -- 172.21.15.32:0/4281254099 shutdown_connections 2023-12-20T06:03:14.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.019+0000 7f93ed72c640 1 -- 172.21.15.32:0/4281254099 wait complete. 2023-12-20T06:03:14.039 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T06:03:14.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.187+0000 7fb486273640 1 Processor -- start 2023-12-20T06:03:14.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.187+0000 7fb486273640 1 -- start start 2023-12-20T06:03:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.191+0000 7fb486273640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4800b0cb0 0x7fb4800caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.191+0000 7fb486273640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb480070230 0x7fb4800706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.191+0000 7fb486273640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb48015b6a0 con 0x7fb48015abb0 2023-12-20T06:03:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.191+0000 7fb486273640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb48006e160 con 0x7fb4800b0cb0 2023-12-20T06:03:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.191+0000 7fb486273640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb480070c00 con 0x7fb480070230 2023-12-20T06:03:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.195+0000 7fb47f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4800b0cb0 0x7fb4800caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.195+0000 7fb47f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb4800b0cb0 0x7fb4800caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48668/0 (socket says 172.21.15.32:48668) 2023-12-20T06:03:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.195+0000 7fb47f7fe640 1 -- 172.21.15.32:0/25052814 learned_addr learned my addr 172.21.15.32:0/25052814 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:14.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.195+0000 7fb484810640 1 --2- 172.21.15.32:0/25052814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb480070230 0x7fb4800706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.195+0000 7fb47f7fe640 1 -- 172.21.15.32:0/25052814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb480070230 msgr2=0x7fb4800706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.195+0000 7fb47f7fe640 1 --2- 172.21.15.32:0/25052814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb480070230 0x7fb4800706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.195+0000 7fb47f7fe640 1 -- 172.21.15.32:0/25052814 >> v1:172.21.15.32:6789/0 conn(0x7fb48015abb0 legacy=0x7fb48015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.195+0000 7fb47f7fe640 1 -- 172.21.15.32:0/25052814 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb48006d8c0 con 0x7fb4800b0cb0 2023-12-20T06:03:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.195+0000 7fb47effd640 1 -- 172.21.15.32:0/25052814 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3379998630 0 0) 0x7fb48015b6a0 con 0x7fb48015abb0 2023-12-20T06:03:14.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.195+0000 7fb484810640 1 --2- 172.21.15.32:0/25052814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb480070230 0x7fb4800706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:14.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.199+0000 7fb47f7fe640 1 --2- 172.21.15.32:0/25052814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4800b0cb0 0x7fb4800caaf0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb474009870 tx=0x7fb47401c540 comp rx=0 tx=0).ready entity=mon.1 client_cookie=514c340e7eabd1ae server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.199+0000 7fb47effd640 1 -- 172.21.15.32:0/25052814 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740a9400 con 0x7fb4800b0cb0 2023-12-20T06:03:14.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.199+0000 7fb47effd640 1 -- 172.21.15.32:0/25052814 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4740a9560 con 0x7fb4800b0cb0 2023-12-20T06:03:14.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.199+0000 7fb47effd640 1 -- 172.21.15.32:0/25052814 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740a96f0 con 0x7fb4800b0cb0 2023-12-20T06:03:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.203+0000 7fb486273640 1 -- 172.21.15.32:0/25052814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4800b0cb0 msgr2=0x7fb4800caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.203+0000 7fb486273640 1 --2- 172.21.15.32:0/25052814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4800b0cb0 0x7fb4800caaf0 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7fb474009870 tx=0x7fb47401c540 comp rx=0 tx=0).stop 2023-12-20T06:03:14.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.203+0000 7fb486273640 1 -- 172.21.15.32:0/25052814 shutdown_connections 2023-12-20T06:03:14.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.203+0000 7fb486273640 1 --2- 172.21.15.32:0/25052814 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4800b0cb0 0x7fb4800caaf0 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.203+0000 7fb486273640 1 --2- 172.21.15.32:0/25052814 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb480070230 0x7fb4800706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.203+0000 7fb486273640 1 -- 172.21.15.32:0/25052814 >> 172.21.15.32:0/25052814 conn(0x7fb48006a510 msgr2=0x7fb4800ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:14.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.203+0000 7fb486273640 1 -- 172.21.15.32:0/25052814 shutdown_connections 2023-12-20T06:03:14.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.203+0000 7fb486273640 1 -- 172.21.15.32:0/25052814 wait complete. 2023-12-20T06:03:14.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.207+0000 7fb486273640 1 Processor -- start 2023-12-20T06:03:14.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.207+0000 7fb486273640 1 -- start start 2023-12-20T06:03:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.207+0000 7fb486273640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480070230 0x7fb4801499b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.207+0000 7fb486273640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4800b0cb0 0x7fb480147f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.207+0000 7fb486273640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb48015abb0 0x7fb48014b730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.207+0000 7fb486273640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb480148ba0 con 0x7fb480070230 2023-12-20T06:03:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.207+0000 7fb486273640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb480144c00 con 0x7fb4800b0cb0 2023-12-20T06:03:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.207+0000 7fb486273640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb48014a090 con 0x7fb48015abb0 2023-12-20T06:03:14.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480070230 0x7fb4801499b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480070230 0x7fb4801499b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52172/0 (socket says 172.21.15.32:52172) 2023-12-20T06:03:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47ffff640 1 -- 172.21.15.32:0/34118646 learned_addr learned my addr 172.21.15.32:0/34118646 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47f7fe640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4800b0cb0 0x7fb480147f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb484810640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb48015abb0 0x7fb48014b730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47f7fe640 1 -- 172.21.15.32:0/34118646 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb48015abb0 msgr2=0x7fb48014b730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47f7fe640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb48015abb0 0x7fb48014b730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47f7fe640 1 -- 172.21.15.32:0/34118646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480070230 msgr2=0x7fb4801499b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47f7fe640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480070230 0x7fb4801499b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47f7fe640 1 -- 172.21.15.32:0/34118646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb478009a70 con 0x7fb4800b0cb0 2023-12-20T06:03:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb484810640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb48015abb0 0x7fb48014b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47f7fe640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4800b0cb0 0x7fb480147f30 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb4740aa000 tx=0x7fb47401ad80 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47cff9640 1 -- 172.21.15.32:0/34118646 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740a9400 con 0x7fb4800b0cb0 2023-12-20T06:03:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47cff9640 1 -- 172.21.15.32:0/34118646 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb4740b1840 con 0x7fb4800b0cb0 2023-12-20T06:03:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb47cff9640 1 -- 172.21.15.32:0/34118646 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb4740b1bf0 con 0x7fb4800b0cb0 2023-12-20T06:03:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb486273640 1 -- 172.21.15.32:0/34118646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb4740094d0 con 0x7fb4800b0cb0 2023-12-20T06:03:14.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb486273640 1 -- 172.21.15.32:0/34118646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb4801451a0 con 0x7fb4800b0cb0 2023-12-20T06:03:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.211+0000 7fb486273640 1 -- 172.21.15.32:0/34118646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb48004e000 con 0x7fb4800b0cb0 2023-12-20T06:03:14.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.215+0000 7fb47cff9640 1 -- 172.21.15.32:0/34118646 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb4740b1d50 con 0x7fb4800b0cb0 2023-12-20T06:03:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.215+0000 7fb47cff9640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb46c065ca0 0x7fb46c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.215+0000 7fb47cff9640 1 -- 172.21.15.32:0/34118646 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 5816+0+0 (secure 0 0 0) 0x7fb47411de70 con 0x7fb4800b0cb0 2023-12-20T06:03:14.214 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.215+0000 7fb47ffff640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb46c065ca0 0x7fb46c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.215+0000 7fb47ffff640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb46c065ca0 0x7fb46c068160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb470007fd0 tx=0x7fb470009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:14.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.219+0000 7fb47cff9640 1 -- 172.21.15.32:0/34118646 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb4740eb4c0 con 0x7fb4800b0cb0 2023-12-20T06:03:14.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.375+0000 7fb486273640 1 -- 172.21.15.32:0/34118646 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fb48015b300 con 0x7fb4800b0cb0 2023-12-20T06:03:14.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb47cff9640 1 -- 172.21.15.32:0/34118646 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v75) v1 ==== 104+0+0 (secure 0 0 0) 0x7fb4740a6070 con 0x7fb4800b0cb0 2023-12-20T06:03:14.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb486273640 1 -- 172.21.15.32:0/34118646 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb46c065ca0 msgr2=0x7fb46c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb486273640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb46c065ca0 0x7fb46c068160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb470007fd0 tx=0x7fb470009040 comp rx=0 tx=0).stop 2023-12-20T06:03:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb486273640 1 -- 172.21.15.32:0/34118646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4800b0cb0 msgr2=0x7fb480147f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb486273640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4800b0cb0 0x7fb480147f30 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7fb4740aa000 tx=0x7fb47401ad80 comp rx=0 tx=0).stop 2023-12-20T06:03:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb486273640 1 -- 172.21.15.32:0/34118646 shutdown_connections 2023-12-20T06:03:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb486273640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb46c065ca0 0x7fb46c068160 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb486273640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb48015abb0 0x7fb48014b730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb486273640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb4800b0cb0 0x7fb480147f30 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb486273640 1 --2- 172.21.15.32:0/34118646 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb480070230 0x7fb4801499b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb486273640 1 -- 172.21.15.32:0/34118646 >> 172.21.15.32:0/34118646 conn(0x7fb48006a510 msgr2=0x7fb48006cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:14.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.383+0000 7fb486273640 1 -- 172.21.15.32:0/34118646 shutdown_connections 2023-12-20T06:03:14.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.387+0000 7fb486273640 1 -- 172.21.15.32:0/34118646 wait complete. 2023-12-20T06:03:14.404 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs' 2023-12-20T06:03:14.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8e0952640 1 Processor -- start 2023-12-20T06:03:14.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8e0952640 1 -- start start 2023-12-20T06:03:14.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8e0952640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc1599e0 0x7fb8dc0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8e0952640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8dc0c90e0 0x7fb8dc0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8e0952640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8dc0711d0 con 0x7fb8dc0b0b30 2023-12-20T06:03:14.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8e0952640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb8dc0c98c0 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8e0952640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc0c9ba0 con 0x7fb8dc1599e0 2023-12-20T06:03:14.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8d3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8dc0c90e0 0x7fb8dc0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8d3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb8dc0c90e0 0x7fb8dc0b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52188/0 (socket says 172.21.15.32:52188) 2023-12-20T06:03:14.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8d3fff640 1 -- 172.21.15.32:0/3538149603 learned_addr learned my addr 172.21.15.32:0/3538149603 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:14.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8da59c640 1 --2- 172.21.15.32:0/3538149603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc1599e0 0x7fb8dc0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8d3fff640 1 -- 172.21.15.32:0/3538149603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc1599e0 msgr2=0x7fb8dc0aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:14.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8d3fff640 1 --2- 172.21.15.32:0/3538149603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc1599e0 0x7fb8dc0aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8d3fff640 1 -- 172.21.15.32:0/3538149603 >> v1:172.21.15.39:6789/0 conn(0x7fb8dc0b0b30 legacy=0x7fb8dc15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.551+0000 7fb8d3fff640 1 -- 172.21.15.32:0/3538149603 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8dc0c9e80 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8d3fff640 1 --2- 172.21.15.32:0/3538149603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8dc0c90e0 0x7fb8dc0b05f0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009870 tx=0x7fb8c401c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27508651920605e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8d9d9b640 1 -- 172.21.15.32:0/3538149603 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40a7070 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8d9d9b640 1 -- 172.21.15.32:0/3538149603 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c40ab430 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8d9d9b640 1 -- 172.21.15.32:0/3538149603 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40ab7c0 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8e0952640 1 -- 172.21.15.32:0/3538149603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8dc0c90e0 msgr2=0x7fb8dc0b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8e0952640 1 --2- 172.21.15.32:0/3538149603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8dc0c90e0 0x7fb8dc0b05f0 secure :-1 s=READY pgs=2201 cs=0 l=1 rev1=1 crypto rx=0x7fb8c4009870 tx=0x7fb8c401c510 comp rx=0 tx=0).stop 2023-12-20T06:03:14.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8e0952640 1 -- 172.21.15.32:0/3538149603 shutdown_connections 2023-12-20T06:03:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8e0952640 1 --2- 172.21.15.32:0/3538149603 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8dc0c90e0 0x7fb8dc0b05f0 unknown :-1 s=CLOSED pgs=2201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8e0952640 1 --2- 172.21.15.32:0/3538149603 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc1599e0 0x7fb8dc0aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8e0952640 1 -- 172.21.15.32:0/3538149603 >> 172.21.15.32:0/3538149603 conn(0x7fb8dc06a320 msgr2=0x7fb8dc0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8e0952640 1 -- 172.21.15.32:0/3538149603 shutdown_connections 2023-12-20T06:03:14.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.555+0000 7fb8e0952640 1 -- 172.21.15.32:0/3538149603 wait complete. 2023-12-20T06:03:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8e0952640 1 Processor -- start 2023-12-20T06:03:14.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8e0952640 1 -- start start 2023-12-20T06:03:14.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8e0952640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc0b0b30 0x7fb8dc1477c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8e0952640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8dc0c90e0 0x7fb8dc14cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8e0952640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8dc1599e0 0x7fb8dc14e5f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8e0952640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc15a640 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8e0952640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc14df30 con 0x7fb8dc1599e0 2023-12-20T06:03:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8e0952640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8dc15abe0 con 0x7fb8dc0b0b30 2023-12-20T06:03:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8d3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8dc0c90e0 0x7fb8dc14cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8da59c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc0b0b30 0x7fb8dc1477c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8da59c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc0b0b30 0x7fb8dc1477c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:57502/0 (socket says 172.21.15.32:57502) 2023-12-20T06:03:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8da59c640 1 -- 172.21.15.32:0/1415962128 learned_addr learned my addr 172.21.15.32:0/1415962128 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:14.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8dad9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8dc1599e0 0x7fb8dc14e5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8d3fff640 1 -- 172.21.15.32:0/1415962128 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc0b0b30 msgr2=0x7fb8dc1477c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8d3fff640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc0b0b30 0x7fb8dc1477c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8d3fff640 1 -- 172.21.15.32:0/1415962128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8dc1599e0 msgr2=0x7fb8dc14e5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8d3fff640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8dc1599e0 0x7fb8dc14e5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8d3fff640 1 -- 172.21.15.32:0/1415962128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8c40094d0 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8da59c640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc0b0b30 0x7fb8dc1477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:14.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8d3fff640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8dc0c90e0 0x7fb8dc14cf90 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40ac000 tx=0x7fb8c40192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8d2ffd640 1 -- 172.21.15.32:0/1415962128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40b4630 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8d2ffd640 1 -- 172.21.15.32:0/1415962128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8c40b4790 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8d2ffd640 1 -- 172.21.15.32:0/1415962128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8c40b4b40 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8e0952640 1 -- 172.21.15.32:0/1415962128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8dc14d970 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8e0952640 1 -- 172.21.15.32:0/1415962128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8dc15a870 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.559+0000 7fb8e0952640 1 -- 172.21.15.32:0/1415962128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8dc04e000 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.563+0000 7fb8d2ffd640 1 -- 172.21.15.32:0/1415962128 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb8c40bf3f0 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.563+0000 7fb8d2ffd640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8bc065ca0 0x7fb8bc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.563+0000 7fb8d2ffd640 1 -- 172.21.15.32:0/1415962128 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 5816+0+0 (secure 0 0 0) 0x7fb8c411cd40 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.563+0000 7fb8da59c640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8bc065ca0 0x7fb8bc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:14.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.563+0000 7fb8da59c640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8bc065ca0 0x7fb8bc068160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009f50 tx=0x7fb8cc006f90 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:14.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.567+0000 7fb8d2ffd640 1 -- 172.21.15.32:0/1415962128 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb8c40ea230 con 0x7fb8dc0c90e0 2023-12-20T06:03:14.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:14.779+0000 7fb8e0952640 1 -- 172.21.15.32:0/1415962128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]} v 0) v1 -- 0x7fb8dc15d070 con 0x7fb8dc0c90e0 2023-12-20T06:03:15.336 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.335+0000 7fb8d2ffd640 1 -- 172.21.15.32:0/1415962128 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_for_non_default_fs"]}]=0 v3080) v1 ==== 186+0+0 (secure 0 0 0) 0x7fb8c40ed850 con 0x7fb8dc0c90e0 2023-12-20T06:03:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 -- 172.21.15.32:0/1415962128 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8bc065ca0 msgr2=0x7fb8bc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8bc065ca0 0x7fb8bc068160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb8cc009f50 tx=0x7fb8cc006f90 comp rx=0 tx=0).stop 2023-12-20T06:03:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 -- 172.21.15.32:0/1415962128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8dc0c90e0 msgr2=0x7fb8dc14cf90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8dc0c90e0 0x7fb8dc14cf90 secure :-1 s=READY pgs=2202 cs=0 l=1 rev1=1 crypto rx=0x7fb8c40ac000 tx=0x7fb8c40192b0 comp rx=0 tx=0).stop 2023-12-20T06:03:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 -- 172.21.15.32:0/1415962128 shutdown_connections 2023-12-20T06:03:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8bc065ca0 0x7fb8bc068160 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8dc1599e0 0x7fb8dc14e5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:15.339 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8dc0c90e0 0x7fb8dc14cf90 unknown :-1 s=CLOSED pgs=2202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 --2- 172.21.15.32:0/1415962128 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8dc0b0b30 0x7fb8dc1477c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 -- 172.21.15.32:0/1415962128 >> 172.21.15.32:0/1415962128 conn(0x7fb8dc06a320 msgr2=0x7fb8dc14fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 -- 172.21.15.32:0/1415962128 shutdown_connections 2023-12-20T06:03:15.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.339+0000 7fb8e0952640 1 -- 172.21.15.32:0/1415962128 wait complete. 2023-12-20T06:03:15.361 INFO:tasks.cephfs_test_runner:Test command succes with --fs argument for non-default fs ... ok 2023-12-20T06:03:15.361 INFO:tasks.cephfs_test_runner:Starting test: test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) Test command fails without --fs argument in presence of multiple fs 2023-12-20T06:03:15.362 INFO:tasks.cephfs_test_runner:test_snap_schedule_without_fs_argument (tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument) 2023-12-20T06:03:15.362 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-12-20T06:03:15.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42ad4f0640 1 Processor -- start 2023-12-20T06:03:15.510 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42ad4f0640 1 -- start start 2023-12-20T06:03:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42ad4f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42a80b8480 0x7f42a806af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42ad4f0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a806b8c0 con 0x7f42a8161500 2023-12-20T06:03:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42ad4f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42a80cd4d0 con 0x7f42a806b460 2023-12-20T06:03:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42ad4f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a80ce4e0 con 0x7f42a80b8480 2023-12-20T06:03:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42accef640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f42a8161500 0x7f42a8153610 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56824/0 (socket says 172.21.15.32:56824) 2023-12-20T06:03:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42accef640 1 -- 172.21.15.32:0/939179510 learned_addr learned my addr 172.21.15.32:0/939179510 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42a7fff640 1 --2- 172.21.15.32:0/939179510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42a80b8480 0x7f42a806af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42a6ffd640 1 -- 172.21.15.32:0/939179510 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3678912098 0 0) 0x7f42a806b8c0 con 0x7f42a8161500 2023-12-20T06:03:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42a6ffd640 1 -- 172.21.15.32:0/939179510 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f4290003530 con 0x7f42a8161500 2023-12-20T06:03:15.512 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42a6ffd640 1 -- 172.21.15.32:0/939179510 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1514920253 0 0) 0x7f42a80cd4d0 con 0x7f42a806b460 2023-12-20T06:03:15.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42a6ffd640 1 -- 172.21.15.32:0/939179510 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42a806b8c0 con 0x7f42a806b460 2023-12-20T06:03:15.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42a7fff640 1 -- 172.21.15.32:0/939179510 >> v1:172.21.15.39:6789/0 conn(0x7f42a806b460 legacy=0x7f42a8160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:15.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42a7fff640 1 -- 172.21.15.32:0/939179510 >> v1:172.21.15.32:6789/0 conn(0x7f42a8161500 legacy=0x7f42a8153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:15.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42a6ffd640 1 -- 172.21.15.32:0/939179510 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2273464081 0 0) 0x7f4290003530 con 0x7f42a8161500 2023-12-20T06:03:15.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.511+0000 7f42a7fff640 1 -- 172.21.15.32:0/939179510 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42a80cd720 con 0x7f42a80b8480 2023-12-20T06:03:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.515+0000 7f42a7fff640 1 --2- 172.21.15.32:0/939179510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42a80b8480 0x7f42a806af20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f42980097f0 tx=0x7f429801c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e7c00d444466271 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.515+0000 7f42a6ffd640 1 -- 172.21.15.32:0/939179510 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42a0001a00 con 0x7f42a80b8480 2023-12-20T06:03:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.515+0000 7f42a6ffd640 1 -- 172.21.15.32:0/939179510 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42980a7050 con 0x7f42a80b8480 2023-12-20T06:03:15.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.515+0000 7f42a6ffd640 1 -- 172.21.15.32:0/939179510 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42980ab620 con 0x7f42a80b8480 2023-12-20T06:03:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.515+0000 7f42ad4f0640 1 -- 172.21.15.32:0/939179510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42a80b8480 msgr2=0x7f42a806af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.515+0000 7f42ad4f0640 1 --2- 172.21.15.32:0/939179510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42a80b8480 0x7f42a806af20 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f42980097f0 tx=0x7f429801c3e0 comp rx=0 tx=0).stop 2023-12-20T06:03:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.515+0000 7f42ad4f0640 1 -- 172.21.15.32:0/939179510 shutdown_connections 2023-12-20T06:03:15.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.515+0000 7f42ad4f0640 1 --2- 172.21.15.32:0/939179510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42a80b8480 0x7f42a806af20 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.515+0000 7f42ad4f0640 1 -- 172.21.15.32:0/939179510 >> 172.21.15.32:0/939179510 conn(0x7f42a80d2a50 msgr2=0x7f42a80d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.515+0000 7f42ad4f0640 1 -- 172.21.15.32:0/939179510 shutdown_connections 2023-12-20T06:03:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.515+0000 7f42ad4f0640 1 -- 172.21.15.32:0/939179510 wait complete. 2023-12-20T06:03:15.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42ad4f0640 1 Processor -- start 2023-12-20T06:03:15.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42ad4f0640 1 -- start start 2023-12-20T06:03:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42ad4f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42a806b460 0x7f42a814dfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:15.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42ad4f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80b8480 0x7f42a80c9a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42ad4f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a8161500 0x7f42a8149e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42ad4f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a814e630 con 0x7f42a80b8480 2023-12-20T06:03:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42ad4f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42a814e910 con 0x7f42a8161500 2023-12-20T06:03:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42ad4f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42a814ebf0 con 0x7f42a806b460 2023-12-20T06:03:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80b8480 0x7f42a80c9a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80b8480 0x7f42a80c9a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52202/0 (socket says 172.21.15.32:52202) 2023-12-20T06:03:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a77fe640 1 -- 172.21.15.32:0/4264274322 learned_addr learned my addr 172.21.15.32:0/4264274322 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a7fff640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42a806b460 0x7f42a814dfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:15.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42accef640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a8161500 0x7f42a8149e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a77fe640 1 -- 172.21.15.32:0/4264274322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42a806b460 msgr2=0x7f42a814dfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a77fe640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42a806b460 0x7f42a814dfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a77fe640 1 -- 172.21.15.32:0/4264274322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a8161500 msgr2=0x7f42a8149e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a77fe640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a8161500 0x7f42a8149e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a77fe640 1 -- 172.21.15.32:0/4264274322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4298009420 con 0x7f42a80b8480 2023-12-20T06:03:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a77fe640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80b8480 0x7f42a80c9a90 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c490 tx=0x7f429c00c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a4ff9640 1 -- 172.21.15.32:0/4264274322 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c0052d0 con 0x7f42a80b8480 2023-12-20T06:03:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a4ff9640 1 -- 172.21.15.32:0/4264274322 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f429c005430 con 0x7f42a80b8480 2023-12-20T06:03:15.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42a4ff9640 1 -- 172.21.15.32:0/4264274322 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f429c012870 con 0x7f42a80b8480 2023-12-20T06:03:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42ad4f0640 1 -- 172.21.15.32:0/4264274322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42a814eed0 con 0x7f42a80b8480 2023-12-20T06:03:15.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.519+0000 7f42ad4f0640 1 -- 172.21.15.32:0/4264274322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42a814bc00 con 0x7f42a80b8480 2023-12-20T06:03:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.523+0000 7f42ad4f0640 1 -- 172.21.15.32:0/4264274322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42a8074100 con 0x7f42a80b8480 2023-12-20T06:03:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.523+0000 7f42a4ff9640 1 -- 172.21.15.32:0/4264274322 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f429c0129d0 con 0x7f42a80b8480 2023-12-20T06:03:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.523+0000 7f42a4ff9640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4294065ca0 0x7f4294068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.523+0000 7f42a4ff9640 1 -- 172.21.15.32:0/4264274322 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 5816+0+0 (secure 0 0 0) 0x7f429c083150 con 0x7f42a80b8480 2023-12-20T06:03:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.523+0000 7f42a7fff640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4294065ca0 0x7f4294068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:15.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.523+0000 7f42a7fff640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4294065ca0 0x7f4294068160 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42980ad000 tx=0x7f42980a6000 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:15.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.527+0000 7f42a4ff9640 1 -- 172.21.15.32:0/4264274322 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f429c050750 con 0x7f42a80b8480 2023-12-20T06:03:15.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:15.739+0000 7f42ad4f0640 1 -- 172.21.15.32:0/4264274322 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7f42a80d2810 con 0x7f42a80b8480 2023-12-20T06:03:16.347 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.347+0000 7f42a4ff9640 1 -- 172.21.15.32:0/4264274322 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3081) v1 ==== 190+0+0 (secure 0 0 0) 0x7f429c053d70 con 0x7f42a80b8480 2023-12-20T06:03:16.350 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 -- 172.21.15.32:0/4264274322 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4294065ca0 msgr2=0x7f4294068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:16.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4294065ca0 0x7f4294068160 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f42980ad000 tx=0x7f42980a6000 comp rx=0 tx=0).stop 2023-12-20T06:03:16.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 -- 172.21.15.32:0/4264274322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80b8480 msgr2=0x7f42a80c9a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:16.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80b8480 0x7f42a80c9a90 secure :-1 s=READY pgs=2204 cs=0 l=1 rev1=1 crypto rx=0x7f429c00c490 tx=0x7f429c00c950 comp rx=0 tx=0).stop 2023-12-20T06:03:16.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 -- 172.21.15.32:0/4264274322 shutdown_connections 2023-12-20T06:03:16.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4294065ca0 0x7f4294068160 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f42a8161500 0x7f42a8149e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f42a80b8480 0x7f42a80c9a90 unknown :-1 s=CLOSED pgs=2204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 --2- 172.21.15.32:0/4264274322 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f42a806b460 0x7f42a814dfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 -- 172.21.15.32:0/4264274322 >> 172.21.15.32:0/4264274322 conn(0x7f42a80d2a50 msgr2=0x7f42a8160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:16.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 -- 172.21.15.32:0/4264274322 shutdown_connections 2023-12-20T06:03:16.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.351+0000 7f42ad4f0640 1 -- 172.21.15.32:0/4264274322 wait complete. 2023-12-20T06:03:16.372 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T06:03:16.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7d6063640 1 Processor -- start 2023-12-20T06:03:16.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7d6063640 1 -- start start 2023-12-20T06:03:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7d6063640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d0164670 0x7fc7d00ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:16.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7d6063640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7d006da20 0x7fc7d00cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7d6063640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc7d00cb0b0 con 0x7fc7d0070230 2023-12-20T06:03:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7d6063640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc7d00ca810 con 0x7fc7d006da20 2023-12-20T06:03:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7d6063640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d00caaf0 con 0x7fc7d0164670 2023-12-20T06:03:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7ceffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7d006da20 0x7fc7d00cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7cf7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d0164670 0x7fc7d00ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7ceffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc7d006da20 0x7fc7d00cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48706/0 (socket says 172.21.15.32:48706) 2023-12-20T06:03:16.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7ceffd640 1 -- 172.21.15.32:0/3818589139 learned_addr learned my addr 172.21.15.32:0/3818589139 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:16.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7cf7fe640 1 -- 172.21.15.32:0/3818589139 >> v2:172.21.15.39:3300/0 conn(0x7fc7d006da20 msgr2=0x7fc7d00cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:16.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7cf7fe640 1 --2- 172.21.15.32:0/3818589139 >> v2:172.21.15.39:3300/0 conn(0x7fc7d006da20 0x7fc7d00cc3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7cf7fe640 1 -- 172.21.15.32:0/3818589139 >> v1:172.21.15.39:6789/0 conn(0x7fc7d0070230 legacy=0x7fc7d006ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:16.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.523+0000 7fc7cf7fe640 1 -- 172.21.15.32:0/3818589139 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7d00cadd0 con 0x7fc7d0164670 2023-12-20T06:03:16.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7cf7fe640 1 --2- 172.21.15.32:0/3818589139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d0164670 0x7fc7d00ca1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00097c0 tx=0x7fc7c001c330 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3237363644b44610 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:16.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7ce7fc640 1 -- 172.21.15.32:0/3818589139 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00ac640 con 0x7fc7d0164670 2023-12-20T06:03:16.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7ce7fc640 1 -- 172.21.15.32:0/3818589139 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c00ac7a0 con 0x7fc7d0164670 2023-12-20T06:03:16.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7ce7fc640 1 -- 172.21.15.32:0/3818589139 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c00acb30 con 0x7fc7d0164670 2023-12-20T06:03:16.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7d6063640 1 -- 172.21.15.32:0/3818589139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d0164670 msgr2=0x7fc7d00ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:16.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7d6063640 1 --2- 172.21.15.32:0/3818589139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d0164670 0x7fc7d00ca1a0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00097c0 tx=0x7fc7c001c330 comp rx=0 tx=0).stop 2023-12-20T06:03:16.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7d6063640 1 -- 172.21.15.32:0/3818589139 shutdown_connections 2023-12-20T06:03:16.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7d6063640 1 --2- 172.21.15.32:0/3818589139 >> v2:172.21.15.39:3300/0 conn(0x7fc7d006da20 0x7fc7d00cc3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7d6063640 1 --2- 172.21.15.32:0/3818589139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d0164670 0x7fc7d00ca1a0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7d6063640 1 -- 172.21.15.32:0/3818589139 >> 172.21.15.32:0/3818589139 conn(0x7fc7d006a510 msgr2=0x7fc7d00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:16.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7d6063640 1 -- 172.21.15.32:0/3818589139 shutdown_connections 2023-12-20T06:03:16.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.527+0000 7fc7d6063640 1 -- 172.21.15.32:0/3818589139 wait complete. 2023-12-20T06:03:16.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7d6063640 1 Processor -- start 2023-12-20T06:03:16.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7d6063640 1 -- start start 2023-12-20T06:03:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7d6063640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d006da20 0x7fc7d00b0760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7d6063640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d0070230 0x7fc7d0156b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7d6063640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7d0164670 0x7fc7d015bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7d6063640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d015aee0 con 0x7fc7d0070230 2023-12-20T06:03:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7d6063640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d0157040 con 0x7fc7d0164670 2023-12-20T06:03:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7d6063640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc7d015a8b0 con 0x7fc7d006da20 2023-12-20T06:03:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7ceffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d0070230 0x7fc7d0156b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:16.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7cf7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d006da20 0x7fc7d00b0760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:16.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7ceffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d0070230 0x7fc7d0156b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52206/0 (socket says 172.21.15.32:52206) 2023-12-20T06:03:16.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7ceffd640 1 -- 172.21.15.32:0/891876308 learned_addr learned my addr 172.21.15.32:0/891876308 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:16.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7cffff640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7d0164670 0x7fc7d015bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:16.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7ceffd640 1 -- 172.21.15.32:0/891876308 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d006da20 msgr2=0x7fc7d00b0760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:16.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7ceffd640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d006da20 0x7fc7d00b0760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7ceffd640 1 -- 172.21.15.32:0/891876308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7d0164670 msgr2=0x7fc7d015bd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:16.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7ceffd640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7d0164670 0x7fc7d015bd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7ceffd640 1 -- 172.21.15.32:0/891876308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc7c0009420 con 0x7fc7d0070230 2023-12-20T06:03:16.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7cf7fe640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d006da20 0x7fc7d00b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:16.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7ceffd640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d0070230 0x7fc7d0156b00 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400e8c0 tx=0x7fc7c400ed80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7b7fff640 1 -- 172.21.15.32:0/891876308 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c4009620 con 0x7fc7d0070230 2023-12-20T06:03:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7b7fff640 1 -- 172.21.15.32:0/891876308 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc7c400c940 con 0x7fc7d0070230 2023-12-20T06:03:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7d6063640 1 -- 172.21.15.32:0/891876308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc7d00b0d40 con 0x7fc7d0070230 2023-12-20T06:03:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7b7fff640 1 -- 172.21.15.32:0/891876308 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc7c400ccf0 con 0x7fc7d0070230 2023-12-20T06:03:16.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.531+0000 7fc7d6063640 1 -- 172.21.15.32:0/891876308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc7d015ab00 con 0x7fc7d0070230 2023-12-20T06:03:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.535+0000 7fc7d6063640 1 -- 172.21.15.32:0/891876308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc7d004e000 con 0x7fc7d0070230 2023-12-20T06:03:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.535+0000 7fc7b7fff640 1 -- 172.21.15.32:0/891876308 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fc7c4013470 con 0x7fc7d0070230 2023-12-20T06:03:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.535+0000 7fc7b7fff640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc7bc065ca0 0x7fc7bc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.535+0000 7fc7b7fff640 1 -- 172.21.15.32:0/891876308 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 5816+0+0 (secure 0 0 0) 0x7fc7c4083750 con 0x7fc7d0070230 2023-12-20T06:03:16.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.535+0000 7fc7cf7fe640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc7bc065ca0 0x7fc7bc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:16.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.539+0000 7fc7cf7fe640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc7bc065ca0 0x7fc7bc068160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00ae000 tx=0x7fc7c00a6070 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:16.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.539+0000 7fc7b7fff640 1 -- 172.21.15.32:0/891876308 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc7c4050de0 con 0x7fc7d0070230 2023-12-20T06:03:16.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.699+0000 7fc7d6063640 1 -- 172.21.15.32:0/891876308 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7fc7d00cf4d0 con 0x7fc7d0070230 2023-12-20T06:03:16.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.703+0000 7fc7b7fff640 1 -- 172.21.15.32:0/891876308 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v76)=0 v76) v1 ==== 122+0+0 (secure 0 0 0) 0x7fc7c4054400 con 0x7fc7d0070230 2023-12-20T06:03:16.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 -- 172.21.15.32:0/891876308 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc7bc065ca0 msgr2=0x7fc7bc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:16.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc7bc065ca0 0x7fc7bc068160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fc7c00ae000 tx=0x7fc7c00a6070 comp rx=0 tx=0).stop 2023-12-20T06:03:16.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 -- 172.21.15.32:0/891876308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d0070230 msgr2=0x7fc7d0156b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:16.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d0070230 0x7fc7d0156b00 secure :-1 s=READY pgs=2205 cs=0 l=1 rev1=1 crypto rx=0x7fc7c400e8c0 tx=0x7fc7c400ed80 comp rx=0 tx=0).stop 2023-12-20T06:03:16.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 -- 172.21.15.32:0/891876308 shutdown_connections 2023-12-20T06:03:16.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc7bc065ca0 0x7fc7bc068160 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc7d0164670 0x7fc7d015bd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc7d0070230 0x7fc7d0156b00 unknown :-1 s=CLOSED pgs=2205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 --2- 172.21.15.32:0/891876308 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc7d006da20 0x7fc7d00b0760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:16.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 -- 172.21.15.32:0/891876308 >> 172.21.15.32:0/891876308 conn(0x7fc7d006a510 msgr2=0x7fc7d00c4f30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:16.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 -- 172.21.15.32:0/891876308 shutdown_connections 2023-12-20T06:03:16.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:16.707+0000 7fc7d6063640 1 -- 172.21.15.32:0/891876308 wait complete. 2023-12-20T06:03:16.725 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T06:03:16.725 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:03:16.788 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:03:16.869 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T06:03:16.869 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:03:16.869 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:03:16.879 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:03:16.879 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:03:16.879 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:03:16.890 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:03:16.890 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:03:16.890 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:03:16.929 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:03:16.929 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:03:16.929 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:03:16.943 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:03:16.943 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T06:03:16.944 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc247344640 1 Processor -- start 2023-12-20T06:03:17.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc247344640 1 -- start start 2023-12-20T06:03:17.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc247344640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc240159ab0 0x7fc2400714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc247344640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc2400b3e90 0x7fc240071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc247344640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc240071f40 con 0x7fc2400cd2e0 2023-12-20T06:03:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc247344640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2400ca0b0 con 0x7fc240159ab0 2023-12-20T06:03:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc247344640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc2400ad7e0 con 0x7fc2400b3e90 2023-12-20T06:03:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc246342640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc240159ab0 0x7fc2400714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc246342640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc240159ab0 0x7fc2400714c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52222/0 (socket says 172.21.15.32:52222) 2023-12-20T06:03:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc246342640 1 -- 172.21.15.32:0/4257502518 learned_addr learned my addr 172.21.15.32:0/4257502518 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc246b43640 1 --2- 172.21.15.32:0/4257502518 >> v2:172.21.15.39:3300/0 conn(0x7fc2400b3e90 0x7fc240071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc246342640 1 -- 172.21.15.32:0/4257502518 >> v2:172.21.15.39:3300/0 conn(0x7fc2400b3e90 msgr2=0x7fc240071a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc246342640 1 --2- 172.21.15.32:0/4257502518 >> v2:172.21.15.39:3300/0 conn(0x7fc2400b3e90 0x7fc240071a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc246342640 1 -- 172.21.15.32:0/4257502518 >> v1:172.21.15.39:6789/0 conn(0x7fc2400cd2e0 legacy=0x7fc2400cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc246342640 1 -- 172.21.15.32:0/4257502518 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc2400ada80 con 0x7fc240159ab0 2023-12-20T06:03:17.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.115+0000 7fc246b43640 1 --2- 172.21.15.32:0/4257502518 >> v2:172.21.15.39:3300/0 conn(0x7fc2400b3e90 0x7fc240071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:17.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc246342640 1 --2- 172.21.15.32:0/4257502518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc240159ab0 0x7fc2400714c0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fc2400b3e40 tx=0x7fc23c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de6a3209a2510986 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:17.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc245340640 1 -- 172.21.15.32:0/4257502518 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0ac410 con 0x7fc240159ab0 2023-12-20T06:03:17.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc245340640 1 -- 172.21.15.32:0/4257502518 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c0ac570 con 0x7fc240159ab0 2023-12-20T06:03:17.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc245340640 1 -- 172.21.15.32:0/4257502518 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c0ac900 con 0x7fc240159ab0 2023-12-20T06:03:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc247344640 1 -- 172.21.15.32:0/4257502518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc240159ab0 msgr2=0x7fc2400714c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc247344640 1 --2- 172.21.15.32:0/4257502518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc240159ab0 0x7fc2400714c0 secure :-1 s=READY pgs=2206 cs=0 l=1 rev1=1 crypto rx=0x7fc2400b3e40 tx=0x7fc23c01c490 comp rx=0 tx=0).stop 2023-12-20T06:03:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc247344640 1 -- 172.21.15.32:0/4257502518 shutdown_connections 2023-12-20T06:03:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc247344640 1 --2- 172.21.15.32:0/4257502518 >> v2:172.21.15.39:3300/0 conn(0x7fc2400b3e90 0x7fc240071a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc247344640 1 --2- 172.21.15.32:0/4257502518 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc240159ab0 0x7fc2400714c0 unknown :-1 s=CLOSED pgs=2206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc247344640 1 -- 172.21.15.32:0/4257502518 >> 172.21.15.32:0/4257502518 conn(0x7fc24006c060 msgr2=0x7fc24006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc247344640 1 -- 172.21.15.32:0/4257502518 shutdown_connections 2023-12-20T06:03:17.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.119+0000 7fc247344640 1 -- 172.21.15.32:0/4257502518 wait complete. 2023-12-20T06:03:17.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc247344640 1 Processor -- start 2023-12-20T06:03:17.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc247344640 1 -- start start 2023-12-20T06:03:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc247344640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2400b3e90 0x7fc24015a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc247344640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2400cd2e0 0x7fc240151de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc247344640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc240159ab0 0x7fc2400b25e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:17.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc247344640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400ac0d0 con 0x7fc240159ab0 2023-12-20T06:03:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc247344640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc240153db0 con 0x7fc2400cd2e0 2023-12-20T06:03:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc247344640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc2400b31c0 con 0x7fc2400b3e90 2023-12-20T06:03:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc246b43640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc240159ab0 0x7fc2400b25e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc246342640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2400b3e90 0x7fc24015a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc246342640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2400b3e90 0x7fc24015a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:57554/0 (socket says 172.21.15.32:57554) 2023-12-20T06:03:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc246342640 1 -- 172.21.15.32:0/1701432805 learned_addr learned my addr 172.21.15.32:0/1701432805 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc245b41640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2400cd2e0 0x7fc240151de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc246b43640 1 -- 172.21.15.32:0/1701432805 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2400b3e90 msgr2=0x7fc24015a010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc246b43640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2400b3e90 0x7fc24015a010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc246b43640 1 -- 172.21.15.32:0/1701432805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2400cd2e0 msgr2=0x7fc240151de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:17.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc246b43640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2400cd2e0 0x7fc240151de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc246b43640 1 -- 172.21.15.32:0/1701432805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc23c0094d0 con 0x7fc240159ab0 2023-12-20T06:03:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc246342640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2400b3e90 0x7fc24015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc246b43640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc240159ab0 0x7fc2400b25e0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fc23800cdf0 tx=0x7fc23800a710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc236ffd640 1 -- 172.21.15.32:0/1701432805 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23800d840 con 0x7fc240159ab0 2023-12-20T06:03:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc236ffd640 1 -- 172.21.15.32:0/1701432805 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23800d9a0 con 0x7fc240159ab0 2023-12-20T06:03:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc236ffd640 1 -- 172.21.15.32:0/1701432805 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23800dd90 con 0x7fc240159ab0 2023-12-20T06:03:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc247344640 1 -- 172.21.15.32:0/1701432805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc2400aba80 con 0x7fc240159ab0 2023-12-20T06:03:17.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.123+0000 7fc247344640 1 -- 172.21.15.32:0/1701432805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc240152320 con 0x7fc240159ab0 2023-12-20T06:03:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.127+0000 7fc247344640 1 -- 172.21.15.32:0/1701432805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc2400ab8e0 con 0x7fc240159ab0 2023-12-20T06:03:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.127+0000 7fc236ffd640 1 -- 172.21.15.32:0/1701432805 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fc23801e3b0 con 0x7fc240159ab0 2023-12-20T06:03:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.127+0000 7fc236ffd640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc224065ca0 0x7fc224068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:17.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.127+0000 7fc236ffd640 1 -- 172.21.15.32:0/1701432805 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(221..221 src has 1..221) v4 ==== 5816+0+0 (secure 0 0 0) 0x7fc238083f80 con 0x7fc240159ab0 2023-12-20T06:03:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.127+0000 7fc246342640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc224065ca0 0x7fc224068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:17.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.127+0000 7fc246342640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc224065ca0 0x7fc224068160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01c4d0 tx=0x7fc23c009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:17.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.131+0000 7fc236ffd640 1 -- 172.21.15.32:0/1701432805 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc2400ab8e0 con 0x7fc240159ab0 2023-12-20T06:03:17.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 -- 172.21.15.32:0/1701432805 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc2400ab8e0 con 0x7fc240159ab0 2023-12-20T06:03:17.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc236ffd640 1 -- 172.21.15.32:0/1701432805 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 106 v106) v1 ==== 95+0+3527 (secure 0 0 0) 0x7fc2400ab8e0 con 0x7fc240159ab0 2023-12-20T06:03:17.322 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:17.322 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":106,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":10123,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/1870721119","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6834","nonce":1870721119},{"type":"v1","addr":"172.21.15.39:6835","nonce":1870721119}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10521,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6835/79010426","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":79010426},{"type":"v1","addr":"172.21.15.32:6835","nonce":79010426}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":90},{"gid":10843,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6833/592913087","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":592913087},{"type":"v1","addr":"172.21.15.39:6833","nonce":592913087}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":106},{"gid":11145,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/1517557733","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6836","nonce":1517557733},{"type":"v1","addr":"172.21.15.32:6837","nonce":1517557733}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":104}],"filesystems":[]} 2023-12-20T06:03:17.322 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 106 2023-12-20T06:03:17.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 -- 172.21.15.32:0/1701432805 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc224065ca0 msgr2=0x7fc224068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:17.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc224065ca0 0x7fc224068160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7fc23c01c4d0 tx=0x7fc23c009040 comp rx=0 tx=0).stop 2023-12-20T06:03:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 -- 172.21.15.32:0/1701432805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc240159ab0 msgr2=0x7fc2400b25e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc240159ab0 0x7fc2400b25e0 secure :-1 s=READY pgs=2207 cs=0 l=1 rev1=1 crypto rx=0x7fc23800cdf0 tx=0x7fc23800a710 comp rx=0 tx=0).stop 2023-12-20T06:03:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 -- 172.21.15.32:0/1701432805 shutdown_connections 2023-12-20T06:03:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc224065ca0 0x7fc224068160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc240159ab0 0x7fc2400b25e0 unknown :-1 s=CLOSED pgs=2207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc2400cd2e0 0x7fc240151de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 --2- 172.21.15.32:0/1701432805 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc2400b3e90 0x7fc24015a010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 -- 172.21.15.32:0/1701432805 >> 172.21.15.32:0/1701432805 conn(0x7fc24006c060 msgr2=0x7fc2400cecc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 -- 172.21.15.32:0/1701432805 shutdown_connections 2023-12-20T06:03:17.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.323+0000 7fc247344640 1 -- 172.21.15.32:0/1701432805 wait complete. 2023-12-20T06:03:17.344 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T06:03:17.344 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T06:03:17.344 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T06:03:17.344 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:03:17.345 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T06:03:17.345 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T06:03:17.345 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T06:03:17.345 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:03:17.345 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T06:03:17.346 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T06:03:17.346 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T06:03:17.346 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:03:17.346 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T06:03:17.346 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T06:03:17.346 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T06:03:17.347 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:03:17.358 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T06:03:17.358 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T06:03:17.359 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T06:03:17.360 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T06:03:17.367 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T06:03:17.367 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T06:03:17.368 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T06:03:17.368 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T06:03:17.407 INFO:tasks.ceph.mds.c:Started 2023-12-20T06:03:17.408 DEBUG:teuthology.parallel:result is None 2023-12-20T06:03:17.410 INFO:tasks.ceph.mds.a:Started 2023-12-20T06:03:17.410 DEBUG:teuthology.parallel:result is None 2023-12-20T06:03:17.413 INFO:tasks.ceph.mds.d:Started 2023-12-20T06:03:17.414 DEBUG:teuthology.parallel:result is None 2023-12-20T06:03:17.415 INFO:tasks.ceph.mds.b:Started 2023-12-20T06:03:17.416 DEBUG:teuthology.parallel:result is None 2023-12-20T06:03:17.416 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T06:03:17.495 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T06:03:17.497 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T06:03:17.518 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T06:03:17.518 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T06:03:17.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.571+0000 7f427d441640 1 Processor -- start 2023-12-20T06:03:17.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.571+0000 7f427d441640 1 -- start start 2023-12-20T06:03:17.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.571+0000 7f427d441640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f427815d080 0x7f42780cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:17.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.571+0000 7f427d441640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f427815d970 con 0x7f427815de00 2023-12-20T06:03:17.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.571+0000 7f427d441640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f42780ce180 con 0x7f4278069d20 2023-12-20T06:03:17.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.571+0000 7f427d441640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f42780ce740 con 0x7f427815d080 2023-12-20T06:03:17.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.571+0000 7f426ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f427815d080 0x7f42780cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:17.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.571+0000 7f426ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f427815d080 0x7f42780cbbb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52248/0 (socket says 172.21.15.32:52248) 2023-12-20T06:03:17.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.571+0000 7f426ffff640 1 -- 172.21.15.32:0/2819613285 learned_addr learned my addr 172.21.15.32:0/2819613285 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:17.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.571+0000 7f42777fe640 1 --1- 172.21.15.32:0/2819613285 >> v1:172.21.15.32:6789/0 conn(0x7f427815de00 0x7f427806a100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56840/0 (socket says 172.21.15.32:56840) 2023-12-20T06:03:17.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.575+0000 7f426ffff640 1 -- 172.21.15.32:0/2819613285 >> v1:172.21.15.39:6789/0 conn(0x7f4278069d20 legacy=0x7f42780cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:17.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.575+0000 7f426ffff640 1 -- 172.21.15.32:0/2819613285 >> v1:172.21.15.32:6789/0 conn(0x7f427815de00 legacy=0x7f427806a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:17.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.575+0000 7f426ffff640 1 -- 172.21.15.32:0/2819613285 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42780ce460 con 0x7f427815d080 2023-12-20T06:03:17.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.575+0000 7f426ffff640 1 --2- 172.21.15.32:0/2819613285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427815d080 0x7f42780cbbb0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f4260009870 tx=0x7f426001c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=997f8a10a28704e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.575+0000 7f42767fc640 1 -- 172.21.15.32:0/2819613285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600a95f0 con 0x7f427815d080 2023-12-20T06:03:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.575+0000 7f42767fc640 1 -- 172.21.15.32:0/2819613285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f42600a9750 con 0x7f427815d080 2023-12-20T06:03:17.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.575+0000 7f42767fc640 1 -- 172.21.15.32:0/2819613285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f42600a9ae0 con 0x7f427815d080 2023-12-20T06:03:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.575+0000 7f427d441640 1 -- 172.21.15.32:0/2819613285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427815d080 msgr2=0x7f42780cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.575+0000 7f427d441640 1 --2- 172.21.15.32:0/2819613285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427815d080 0x7f42780cbbb0 secure :-1 s=READY pgs=2214 cs=0 l=1 rev1=1 crypto rx=0x7f4260009870 tx=0x7f426001c510 comp rx=0 tx=0).stop 2023-12-20T06:03:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 -- 172.21.15.32:0/2819613285 shutdown_connections 2023-12-20T06:03:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 --2- 172.21.15.32:0/2819613285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427815d080 0x7f42780cbbb0 unknown :-1 s=CLOSED pgs=2214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 -- 172.21.15.32:0/2819613285 >> 172.21.15.32:0/2819613285 conn(0x7f42780cc310 msgr2=0x7f42780cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 -- 172.21.15.32:0/2819613285 shutdown_connections 2023-12-20T06:03:17.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 -- 172.21.15.32:0/2819613285 wait complete. 2023-12-20T06:03:17.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 Processor -- start 2023-12-20T06:03:17.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 -- start start 2023-12-20T06:03:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278069d20 0x7f42780aba70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f427815d080 0x7f427814dd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427815de00 0x7f42780adf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42780ac3d0 con 0x7f427815de00 2023-12-20T06:03:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f42780ac620 con 0x7f4278069d20 2023-12-20T06:03:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f427d441640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f42780ac900 con 0x7f427815d080 2023-12-20T06:03:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f42777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427815de00 0x7f42780adf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:17.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f42777fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427815de00 0x7f42780adf00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52254/0 (socket says 172.21.15.32:52254) 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f42777fe640 1 -- 172.21.15.32:0/339223952 learned_addr learned my addr 172.21.15.32:0/339223952 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f426ffff640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f427815d080 0x7f427814dd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f4276ffd640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278069d20 0x7f42780aba70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f42777fe640 1 -- 172.21.15.32:0/339223952 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f427815d080 msgr2=0x7f427814dd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f42777fe640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f427815d080 0x7f427814dd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f42777fe640 1 -- 172.21.15.32:0/339223952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278069d20 msgr2=0x7f42780aba70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f42777fe640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278069d20 0x7f42780aba70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f42777fe640 1 -- 172.21.15.32:0/339223952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f42600094d0 con 0x7f427815de00 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.579+0000 7f4276ffd640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278069d20 0x7f42780aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.583+0000 7f42777fe640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427815de00 0x7f42780adf00 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f427000c7a0 tx=0x7f427000cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.583+0000 7f426f7fe640 1 -- 172.21.15.32:0/339223952 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000a640 con 0x7f427815de00 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.583+0000 7f426f7fe640 1 -- 172.21.15.32:0/339223952 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f427000a7a0 con 0x7f427815de00 2023-12-20T06:03:17.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.583+0000 7f427d441640 1 -- 172.21.15.32:0/339223952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f42780acbe0 con 0x7f427815de00 2023-12-20T06:03:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.583+0000 7f426f7fe640 1 -- 172.21.15.32:0/339223952 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f427000ab10 con 0x7f427815de00 2023-12-20T06:03:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.583+0000 7f427d441640 1 -- 172.21.15.32:0/339223952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f42780ad1a0 con 0x7f427815de00 2023-12-20T06:03:17.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.583+0000 7f427d441640 1 -- 172.21.15.32:0/339223952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f42780ad390 con 0x7f427815de00 2023-12-20T06:03:17.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.583+0000 7f426f7fe640 1 -- 172.21.15.32:0/339223952 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f427000ac70 con 0x7f427815de00 2023-12-20T06:03:17.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.583+0000 7f426f7fe640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4258065ca0 0x7f4258068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:17.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.583+0000 7f426f7fe640 1 -- 172.21.15.32:0/339223952 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(225..225 src has 1..225) v4 ==== 5816+0+0 (secure 0 0 0) 0x7f4270082dd0 con 0x7f427815de00 2023-12-20T06:03:17.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.583+0000 7f4276ffd640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4258065ca0 0x7f4258068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:17.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.587+0000 7f4276ffd640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4258065ca0 0x7f4258068160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42781510a0 tx=0x7f4268009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:17.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.587+0000 7f426f7fe640 1 -- 172.21.15.32:0/339223952 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f4270050470 con 0x7f427815de00 2023-12-20T06:03:17.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:17.759+0000 7f427d441640 1 -- 172.21.15.32:0/339223952 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f427804e000 con 0x7f427815de00 2023-12-20T06:03:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.563+0000 7f426f7fe640 1 -- 172.21.15.32:0/339223952 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v226) v1 ==== 97+0+0 (secure 0 0 0) 0x7f4270017080 con 0x7f427815de00 2023-12-20T06:03:18.562 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T06:03:18.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.563+0000 7f427d441640 1 -- 172.21.15.32:0/339223952 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4258065ca0 msgr2=0x7f4258068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:18.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.563+0000 7f427d441640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4258065ca0 0x7f4258068160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f42781510a0 tx=0x7f4268009040 comp rx=0 tx=0).stop 2023-12-20T06:03:18.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.563+0000 7f427d441640 1 -- 172.21.15.32:0/339223952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427815de00 msgr2=0x7f42780adf00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:18.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.563+0000 7f427d441640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427815de00 0x7f42780adf00 secure :-1 s=READY pgs=2215 cs=0 l=1 rev1=1 crypto rx=0x7f427000c7a0 tx=0x7f427000cc60 comp rx=0 tx=0).stop 2023-12-20T06:03:18.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.567+0000 7f427d441640 1 -- 172.21.15.32:0/339223952 shutdown_connections 2023-12-20T06:03:18.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.567+0000 7f427d441640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4258065ca0 0x7f4258068160 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.567+0000 7f427d441640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f427815de00 0x7f42780adf00 unknown :-1 s=CLOSED pgs=2215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.567+0000 7f427d441640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f427815d080 0x7f427814dd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.567+0000 7f427d441640 1 --2- 172.21.15.32:0/339223952 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4278069d20 0x7f42780aba70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.567+0000 7f427d441640 1 -- 172.21.15.32:0/339223952 >> 172.21.15.32:0/339223952 conn(0x7f42780cc310 msgr2=0x7f427806f2a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:18.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.567+0000 7f427d441640 1 -- 172.21.15.32:0/339223952 shutdown_connections 2023-12-20T06:03:18.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.567+0000 7f427d441640 1 -- 172.21.15.32:0/339223952 wait complete. 2023-12-20T06:03:18.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T06:03:18.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff24628c640 1 Processor -- start 2023-12-20T06:03:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff24628c640 1 -- start start 2023-12-20T06:03:18.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff24628c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff240161560 0x7ff240070950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff24628c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff240071920 con 0x7ff24015ce30 2023-12-20T06:03:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff24628c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2401619f0 con 0x7ff24006deb0 2023-12-20T06:03:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff24628c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff240070e90 con 0x7ff240161560 2023-12-20T06:03:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff24015ce30 0x7ff24006d7a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:56852/0 (socket says 172.21.15.32:56852) 2023-12-20T06:03:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23ffff640 1 -- 172.21.15.32:0/2473202135 learned_addr learned my addr 172.21.15.32:0/2473202135 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff244829640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff240161560 0x7ff240070950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1406558226 0 0) 0x7ff240071920 con 0x7ff24015ce30 2023-12-20T06:03:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7ff228003530 con 0x7ff24015ce30 2023-12-20T06:03:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1827965805 0 0) 0x7ff228003530 con 0x7ff24015ce30 2023-12-20T06:03:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff240161560 msgr2=0x7ff240070950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff240161560 0x7ff240070950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 >> v1:172.21.15.39:6789/0 conn(0x7ff24006deb0 legacy=0x7ff2400cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff240071080 con 0x7ff24015ce30 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7ff230001680 con 0x7ff24015ce30 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 >> v1:172.21.15.32:6789/0 conn(0x7ff24015ce30 legacy=0x7ff24006d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2280039b0 0x7ff228003dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff240161560 0x7ff2280047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff228004d10 0x7ff228005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22800ba40 con 0x7ff2280039b0 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff22800bbb0 con 0x7ff228004d10 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff22800bd70 con 0x7ff240161560 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23ffff640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2280039b0 0x7ff228003dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff23f7fe640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff240161560 0x7ff2280047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.735+0000 7ff244829640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff228004d10 0x7ff228005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff23ffff640 1 -- 172.21.15.32:0/2473202135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff240161560 msgr2=0x7ff2280047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff23ffff640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff240161560 0x7ff2280047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff23ffff640 1 -- 172.21.15.32:0/2473202135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff228004d10 msgr2=0x7ff228005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff23ffff640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff228004d10 0x7ff228005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff23ffff640 1 -- 172.21.15.32:0/2473202135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff228003530 con 0x7ff2280039b0 2023-12-20T06:03:18.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff23ffff640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2280039b0 0x7ff228003dd0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c210 tx=0x7ff23001cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d30ff7562e554d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:18.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff230005ae0 con 0x7ff2280039b0 2023-12-20T06:03:18.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff230005c40 con 0x7ff2280039b0 2023-12-20T06:03:18.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff23effd640 1 -- 172.21.15.32:0/2473202135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2300a57e0 con 0x7ff2280039b0 2023-12-20T06:03:18.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff24628c640 1 -- 172.21.15.32:0/2473202135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2280039b0 msgr2=0x7ff228003dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:18.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff24628c640 1 --2- 172.21.15.32:0/2473202135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2280039b0 0x7ff228003dd0 secure :-1 s=READY pgs=2217 cs=0 l=1 rev1=1 crypto rx=0x7ff23000c210 tx=0x7ff23001cbf0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff244829640 1 -- 172.21.15.32:0/2473202135 reap_dead start 2023-12-20T06:03:18.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff24628c640 1 -- 172.21.15.32:0/2473202135 shutdown_connections 2023-12-20T06:03:18.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff24628c640 1 -- 172.21.15.32:0/2473202135 >> 172.21.15.32:0/2473202135 conn(0x7ff2400cba50 msgr2=0x7ff2400cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:18.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.739+0000 7ff24628c640 1 -- 172.21.15.32:0/2473202135 shutdown_connections 2023-12-20T06:03:18.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff24628c640 1 -- 172.21.15.32:0/2473202135 wait complete. 2023-12-20T06:03:18.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff24628c640 1 Processor -- start 2023-12-20T06:03:18.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff24628c640 1 -- start start 2023-12-20T06:03:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff24628c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2280039b0 0x7ff2400c60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff24628c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff240154d40 0x7ff2401546a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff24628c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2401559c0 0x7ff240155e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff24628c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff240154be0 con 0x7ff240154d40 2023-12-20T06:03:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff24628c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff240156b40 con 0x7ff2280039b0 2023-12-20T06:03:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff24628c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2400c8b20 con 0x7ff2401559c0 2023-12-20T06:03:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2280039b0 0x7ff2400c60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff240154d40 0x7ff2401546a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff240154d40 0x7ff2401546a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:52264/0 (socket says 172.21.15.32:52264) 2023-12-20T06:03:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23ffff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2280039b0 0x7ff2400c60a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:48788/0 (socket says 172.21.15.32:48788) 2023-12-20T06:03:18.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23f7fe640 1 -- 172.21.15.32:0/918107417 learned_addr learned my addr 172.21.15.32:0/918107417 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff244829640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2401559c0 0x7ff240155e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23f7fe640 1 -- 172.21.15.32:0/918107417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2401559c0 msgr2=0x7ff240155e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23f7fe640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2401559c0 0x7ff240155e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23f7fe640 1 -- 172.21.15.32:0/918107417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2280039b0 msgr2=0x7ff2400c60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23f7fe640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2280039b0 0x7ff2400c60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23f7fe640 1 -- 172.21.15.32:0/918107417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2400c8d90 con 0x7ff240154d40 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23ffff640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2280039b0 0x7ff2400c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.743+0000 7ff23f7fe640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff240154d40 0x7ff2401546a0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7ff23400f850 tx=0x7ff23400fd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.747+0000 7ff23cff9640 1 -- 172.21.15.32:0/918107417 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23400fe40 con 0x7ff240154d40 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.747+0000 7ff23cff9640 1 -- 172.21.15.32:0/918107417 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2340052e0 con 0x7ff240154d40 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.747+0000 7ff23cff9640 1 -- 172.21.15.32:0/918107417 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff23400d8f0 con 0x7ff240154d40 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.747+0000 7ff24628c640 1 -- 172.21.15.32:0/918107417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2401567d0 con 0x7ff240154d40 2023-12-20T06:03:18.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.747+0000 7ff24628c640 1 -- 172.21.15.32:0/918107417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff240156380 con 0x7ff240154d40 2023-12-20T06:03:18.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.747+0000 7ff24628c640 1 -- 172.21.15.32:0/918107417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff240156990 con 0x7ff240154d40 2023-12-20T06:03:18.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.747+0000 7ff23cff9640 1 -- 172.21.15.32:0/918107417 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff23400da50 con 0x7ff240154d40 2023-12-20T06:03:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.747+0000 7ff23cff9640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff22c065cd0 0x7ff22c068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.747+0000 7ff23cff9640 1 -- 172.21.15.32:0/918107417 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 4913+0+0 (secure 0 0 0) 0x7ff234083090 con 0x7ff240154d40 2023-12-20T06:03:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.747+0000 7ff23ffff640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff22c065cd0 0x7ff22c068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:18.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.751+0000 7ff23ffff640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff22c065cd0 0x7ff22c068190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff2300ab000 tx=0x7ff2300a7070 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:18.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.751+0000 7ff23cff9640 1 -- 172.21.15.32:0/918107417 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff234050a10 con 0x7ff240154d40 2023-12-20T06:03:18.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.963+0000 7ff24628c640 1 -- 172.21.15.32:0/918107417 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7ff2400c2680 con 0x7ff240154d40 2023-12-20T06:03:18.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff23cff9640 1 -- 172.21.15.32:0/918107417 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v52) v1 ==== 80+0+5304 (secure 0 0 0) 0x7ff234054030 con 0x7ff240154d40 2023-12-20T06:03:18.965 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:18.965 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:03:18.965 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T06:03:18.965 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.965 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T06:03:18.965 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T06:03:18.965 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.965 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:03:18.965 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:03:18.965 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.966 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:03:18.967 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.968 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:03:18.969 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T06:03:18.970 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.971 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:03:18.972 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T06:03:18.973 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:03:18.974 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T06:03:18.975 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.976 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:18.977 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:18.977 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T06:03:18.977 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T06:03:18.977 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:03:18.977 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:03:18.977 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T06:03:18.977 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:03:18.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.977 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:18.977 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T06:03:18.977 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:03:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 -- 172.21.15.32:0/918107417 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff22c065cd0 msgr2=0x7ff22c068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff22c065cd0 0x7ff22c068190 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7ff2300ab000 tx=0x7ff2300a7070 comp rx=0 tx=0).stop 2023-12-20T06:03:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 -- 172.21.15.32:0/918107417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff240154d40 msgr2=0x7ff2401546a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff240154d40 0x7ff2401546a0 secure :-1 s=READY pgs=2218 cs=0 l=1 rev1=1 crypto rx=0x7ff23400f850 tx=0x7ff23400fd10 comp rx=0 tx=0).stop 2023-12-20T06:03:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 -- 172.21.15.32:0/918107417 shutdown_connections 2023-12-20T06:03:18.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff22c065cd0 0x7ff22c068190 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2401559c0 0x7ff240155e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff240154d40 0x7ff2401546a0 unknown :-1 s=CLOSED pgs=2218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 --2- 172.21.15.32:0/918107417 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2280039b0 0x7ff2400c60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 -- 172.21.15.32:0/918107417 >> 172.21.15.32:0/918107417 conn(0x7ff2400cba50 msgr2=0x7ff24009fc50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 -- 172.21.15.32:0/918107417 shutdown_connections 2023-12-20T06:03:18.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:18.967+0000 7ff24628c640 1 -- 172.21.15.32:0/918107417 wait complete. 2023-12-20T06:03:18.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:19.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f507092c640 1 Processor -- start 2023-12-20T06:03:19.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f507092c640 1 -- start start 2023-12-20T06:03:19.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f507092c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f506c06ce60 0x7f506c15b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f507092c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f506c15f110 0x7f506c15c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f507092c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f506c15bbd0 con 0x7f506c0af5e0 2023-12-20T06:03:19.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f507092c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f506c15beb0 con 0x7f506c15f110 2023-12-20T06:03:19.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f507092c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c15e910 con 0x7f506c06ce60 2023-12-20T06:03:19.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f506ad9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f506c15f110 0x7f506c15c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f506ad9d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f506c15f110 0x7f506c15c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:36436/0 (socket says 172.21.15.32:36436) 2023-12-20T06:03:19.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f506a59c640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f506c0af5e0 0x7f506c0af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:57596/0 (socket says 172.21.15.32:57596) 2023-12-20T06:03:19.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f506ad9d640 1 -- 172.21.15.32:0/1640766327 learned_addr learned my addr 172.21.15.32:0/1640766327 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:19.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.131+0000 7f5063fff640 1 --2- 172.21.15.32:0/1640766327 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f506c06ce60 0x7f506c15b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.135+0000 7f506ad9d640 1 -- 172.21.15.32:0/1640766327 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f506c06ce60 msgr2=0x7f506c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:19.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.135+0000 7f506ad9d640 1 --2- 172.21.15.32:0/1640766327 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f506c06ce60 0x7f506c15b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.135+0000 7f506ad9d640 1 -- 172.21.15.32:0/1640766327 >> v1:172.21.15.39:6789/0 conn(0x7f506c0af5e0 legacy=0x7f506c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.135+0000 7f506ad9d640 1 -- 172.21.15.32:0/1640766327 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f506c15ebf0 con 0x7f506c15f110 2023-12-20T06:03:19.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.135+0000 7f5063fff640 1 --2- 172.21.15.32:0/1640766327 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f506c06ce60 0x7f506c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:19.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.135+0000 7f506ad9d640 1 --2- 172.21.15.32:0/1640766327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c15f110 0x7f506c15c440 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f5064009e50 tx=0x7f506401c980 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f93f0ddd50562c54 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:19.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.135+0000 7f5069d9b640 1 -- 172.21.15.32:0/1640766327 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50640a57e0 con 0x7f506c15f110 2023-12-20T06:03:19.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.135+0000 7f5069d9b640 1 -- 172.21.15.32:0/1640766327 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50640a5940 con 0x7f506c15f110 2023-12-20T06:03:19.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.135+0000 7f5069d9b640 1 -- 172.21.15.32:0/1640766327 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50640a5cd0 con 0x7f506c15f110 2023-12-20T06:03:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 -- 172.21.15.32:0/1640766327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c15f110 msgr2=0x7f506c15c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 --2- 172.21.15.32:0/1640766327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c15f110 0x7f506c15c440 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f5064009e50 tx=0x7f506401c980 comp rx=0 tx=0).stop 2023-12-20T06:03:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 -- 172.21.15.32:0/1640766327 shutdown_connections 2023-12-20T06:03:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 --2- 172.21.15.32:0/1640766327 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c15f110 0x7f506c15c440 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 --2- 172.21.15.32:0/1640766327 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f506c06ce60 0x7f506c15b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 -- 172.21.15.32:0/1640766327 >> 172.21.15.32:0/1640766327 conn(0x7f506c06a320 msgr2=0x7f506c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 -- 172.21.15.32:0/1640766327 shutdown_connections 2023-12-20T06:03:19.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 -- 172.21.15.32:0/1640766327 wait complete. 2023-12-20T06:03:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 Processor -- start 2023-12-20T06:03:19.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 -- start start 2023-12-20T06:03:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f506c06ce60 0x7f506c071d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f506c0af5e0 0x7f506c15d320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c15f110 0x7f506c149890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c072260 con 0x7f506c06ce60 2023-12-20T06:03:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f506c070710 con 0x7f506c15f110 2023-12-20T06:03:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f507092c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f506c153760 con 0x7f506c0af5e0 2023-12-20T06:03:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f506a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f506c06ce60 0x7f506c071d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f506a59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f506c06ce60 0x7f506c071d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33192/0 (socket says 172.21.15.32:33192) 2023-12-20T06:03:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f506a59c640 1 -- 172.21.15.32:0/1869081714 learned_addr learned my addr 172.21.15.32:0/1869081714 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f5063fff640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f506c0af5e0 0x7f506c15d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f506ad9d640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c15f110 0x7f506c149890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f506a59c640 1 -- 172.21.15.32:0/1869081714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f506c0af5e0 msgr2=0x7f506c15d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f506a59c640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f506c0af5e0 0x7f506c15d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f506a59c640 1 -- 172.21.15.32:0/1869081714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c15f110 msgr2=0x7f506c149890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f506a59c640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c15f110 0x7f506c149890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.139+0000 7f506a59c640 1 -- 172.21.15.32:0/1869081714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5054009420 con 0x7f506c06ce60 2023-12-20T06:03:19.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.143+0000 7f506a59c640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f506c06ce60 0x7f506c071d20 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f505c00c4d0 tx=0x7f505c00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.143+0000 7f5062ffd640 1 -- 172.21.15.32:0/1869081714 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c005c80 con 0x7f506c06ce60 2023-12-20T06:03:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.143+0000 7f5062ffd640 1 -- 172.21.15.32:0/1869081714 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f505c005de0 con 0x7f506c06ce60 2023-12-20T06:03:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.143+0000 7f5062ffd640 1 -- 172.21.15.32:0/1869081714 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f505c007270 con 0x7f506c06ce60 2023-12-20T06:03:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.143+0000 7f507092c640 1 -- 172.21.15.32:0/1869081714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5064009b30 con 0x7f506c06ce60 2023-12-20T06:03:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.143+0000 7f507092c640 1 -- 172.21.15.32:0/1869081714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f506c153d20 con 0x7f506c06ce60 2023-12-20T06:03:19.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.143+0000 7f507092c640 1 -- 172.21.15.32:0/1869081714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f506c0c8b70 con 0x7f506c06ce60 2023-12-20T06:03:19.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.143+0000 7f5062ffd640 1 -- 172.21.15.32:0/1869081714 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f505c0073d0 con 0x7f506c06ce60 2023-12-20T06:03:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.143+0000 7f5062ffd640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f504c065ca0 0x7f504c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.143+0000 7f5062ffd640 1 -- 172.21.15.32:0/1869081714 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f505c082790 con 0x7f506c06ce60 2023-12-20T06:03:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.143+0000 7f5063fff640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f504c065ca0 0x7f504c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.147+0000 7f5063fff640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f504c065ca0 0x7f504c068160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f506c14aeb0 tx=0x7f5054009190 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:19.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.147+0000 7f5062ffd640 1 -- 172.21.15.32:0/1869081714 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f505c050110 con 0x7f506c06ce60 2023-12-20T06:03:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.307+0000 7f507092c640 1 -- 172.21.15.32:0/1869081714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f506c04e000 con 0x7f506c06ce60 2023-12-20T06:03:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.307+0000 7f5062ffd640 1 -- 172.21.15.32:0/1869081714 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v226) v1 ==== 74+0+12441 (secure 0 0 0) 0x7f505c017020 con 0x7f506c06ce60 2023-12-20T06:03:19.308 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:19.308 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":226,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:18.550602+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":27,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":211,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 -- 172.21.15.32:0/1869081714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f504c065ca0 msgr2=0x7f504c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f504c065ca0 0x7f504c068160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f506c14aeb0 tx=0x7f5054009190 comp rx=0 tx=0).stop 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 -- 172.21.15.32:0/1869081714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f506c06ce60 msgr2=0x7f506c071d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f506c06ce60 0x7f506c071d20 secure :-1 s=READY pgs=2219 cs=0 l=1 rev1=1 crypto rx=0x7f505c00c4d0 tx=0x7f505c00c990 comp rx=0 tx=0).stop 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 -- 172.21.15.32:0/1869081714 shutdown_connections 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f504c065ca0 0x7f504c068160 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f506c15f110 0x7f506c149890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f506c0af5e0 0x7f506c15d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 --2- 172.21.15.32:0/1869081714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f506c06ce60 0x7f506c071d20 unknown :-1 s=CLOSED pgs=2219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 -- 172.21.15.32:0/1869081714 >> 172.21.15.32:0/1869081714 conn(0x7f506c06a320 msgr2=0x7f506c157a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 -- 172.21.15.32:0/1869081714 shutdown_connections 2023-12-20T06:03:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.311+0000 7f507092c640 1 -- 172.21.15.32:0/1869081714 wait complete. 2023-12-20T06:03:19.329 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:03:19.330 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:03:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.479+0000 7f6c5312d640 1 Processor -- start 2023-12-20T06:03:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.479+0000 7f6c5312d640 1 -- start start 2023-12-20T06:03:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5312d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c0cf840 0x7f6c4c0d1c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5312d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4c0d10f0 con 0x7f6c4c15ccf0 2023-12-20T06:03:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5312d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c4c0cdd30 con 0x7f6c4c0cf3e0 2023-12-20T06:03:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5312d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c0ce2f0 con 0x7f6c4c0cf840 2023-12-20T06:03:19.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5212b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6c4c15ccf0 0x7f6c4c0cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37220/0 (socket says 172.21.15.32:37220) 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5212b640 1 -- 172.21.15.32:0/2226220424 learned_addr learned my addr 172.21.15.32:0/2226220424 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5292c640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c0cf840 0x7f6c4c0d1c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1025347549 0 0) 0x7f6c4c0d10f0 con 0x7f6c4c15ccf0 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c38003530 con 0x7f6c4c15ccf0 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 864109905 0 0) 0x7f6c4c0cdd30 con 0x7f6c4c0cf3e0 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c4c0d10f0 con 0x7f6c4c0cf3e0 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1800230670 0 0) 0x7f6c38003530 con 0x7f6c4c15ccf0 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c0cf840 msgr2=0x7f6c4c0d1c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c0cf840 0x7f6c4c0d1c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 >> v1:172.21.15.39:6789/0 conn(0x7f6c4c0cf3e0 legacy=0x7f6c4c0d1580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c0ce010 con 0x7f6c4c15ccf0 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f6c48001680 con 0x7f6c4c15ccf0 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 >> v1:172.21.15.32:6789/0 conn(0x7f6c4c15ccf0 legacy=0x7f6c4c0cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 0x7f6c38003f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c0cf840 0x7f6c380047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38004d10 0x7f6c38005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c0ce2f0 con 0x7f6c38003b40 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3800bac0 con 0x7f6c38004d10 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c3800bc80 con 0x7f6c4c0cf840 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5212b640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 0x7f6c38003f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5192a640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c0cf840 0x7f6c380047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5292c640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38004d10 0x7f6c38005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5212b640 1 -- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c0cf840 msgr2=0x7f6c380047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5212b640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c0cf840 0x7f6c380047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5212b640 1 -- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38004d10 msgr2=0x7f6c38005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5212b640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38004d10 0x7f6c38005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5212b640 1 -- 172.21.15.32:0/2226220424 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c38003530 con 0x7f6c38003b40 2023-12-20T06:03:19.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.483+0000 7f6c5292c640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c38004d10 0x7f6c38005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.487+0000 7f6c5212b640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 0x7f6c38003f40 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c160 tx=0x7f6c4801cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a09aeda1b46c76f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.487+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c480058e0 con 0x7f6c38003b40 2023-12-20T06:03:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.487+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c48005a60 con 0x7f6c38003b40 2023-12-20T06:03:19.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.487+0000 7f6c51129640 1 -- 172.21.15.32:0/2226220424 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48005e10 con 0x7f6c38003b40 2023-12-20T06:03:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.487+0000 7f6c5312d640 1 -- 172.21.15.32:0/2226220424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 msgr2=0x7f6c38003f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.487+0000 7f6c5312d640 1 --2- 172.21.15.32:0/2226220424 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 0x7f6c38003f40 secure :-1 s=READY pgs=2221 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c160 tx=0x7f6c4801cbc0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.487+0000 7f6c5292c640 1 -- 172.21.15.32:0/2226220424 reap_dead start 2023-12-20T06:03:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.487+0000 7f6c5312d640 1 -- 172.21.15.32:0/2226220424 shutdown_connections 2023-12-20T06:03:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.487+0000 7f6c5312d640 1 -- 172.21.15.32:0/2226220424 >> 172.21.15.32:0/2226220424 conn(0x7f6c4c0658e0 msgr2=0x7f6c4c065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.487+0000 7f6c5312d640 1 -- 172.21.15.32:0/2226220424 shutdown_connections 2023-12-20T06:03:19.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.487+0000 7f6c5312d640 1 -- 172.21.15.32:0/2226220424 wait complete. 2023-12-20T06:03:19.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5312d640 1 Processor -- start 2023-12-20T06:03:19.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5312d640 1 -- start start 2023-12-20T06:03:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5312d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 0x7f6c4c0cbfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5312d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c1489e0 0x7f6c4c14f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5312d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c0c80f0 0x7f6c4c0c8570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5312d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c1475a0 con 0x7f6c38003b40 2023-12-20T06:03:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5312d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c0cc620 con 0x7f6c4c0c80f0 2023-12-20T06:03:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5312d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c4c146bd0 con 0x7f6c4c1489e0 2023-12-20T06:03:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5212b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 0x7f6c4c0cbfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5212b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 0x7f6c4c0cbfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33218/0 (socket says 172.21.15.32:33218) 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5212b640 1 -- 172.21.15.32:0/1045540041 learned_addr learned my addr 172.21.15.32:0/1045540041 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5192a640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c1489e0 0x7f6c4c14f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5292c640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c0c80f0 0x7f6c4c0c8570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5212b640 1 -- 172.21.15.32:0/1045540041 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c1489e0 msgr2=0x7f6c4c14f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5212b640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c1489e0 0x7f6c4c14f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5212b640 1 -- 172.21.15.32:0/1045540041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c0c80f0 msgr2=0x7f6c4c0c8570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5212b640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c0c80f0 0x7f6c4c0c8570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5212b640 1 -- 172.21.15.32:0/1045540041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c4c1462e0 con 0x7f6c38003b40 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5212b640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 0x7f6c4c0cbfb0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c160 tx=0x7f6c38005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c42ffd640 1 -- 172.21.15.32:0/1045540041 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48001be0 con 0x7f6c38003b40 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c42ffd640 1 -- 172.21.15.32:0/1045540041 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c48005df0 con 0x7f6c38003b40 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c42ffd640 1 -- 172.21.15.32:0/1045540041 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c48001480 con 0x7f6c38003b40 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5312d640 1 -- 172.21.15.32:0/1045540041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c4c147060 con 0x7f6c38003b40 2023-12-20T06:03:19.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5312d640 1 -- 172.21.15.32:0/1045540041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4c146860 con 0x7f6c38003b40 2023-12-20T06:03:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.491+0000 7f6c5312d640 1 -- 172.21.15.32:0/1045540041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4c074100 con 0x7f6c38003b40 2023-12-20T06:03:19.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.495+0000 7f6c42ffd640 1 -- 172.21.15.32:0/1045540041 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f6c48007070 con 0x7f6c38003b40 2023-12-20T06:03:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.495+0000 7f6c42ffd640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c30065cd0 0x7f6c30068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.495+0000 7f6c42ffd640 1 -- 172.21.15.32:0/1045540041 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f6c4811c890 con 0x7f6c38003b40 2023-12-20T06:03:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.495+0000 7f6c5192a640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c30065cd0 0x7f6c30068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.495+0000 7f6c5192a640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c30065cd0 0x7f6c30068190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c007fd0 tx=0x7f6c3c009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:19.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.499+0000 7f6c42ffd640 1 -- 172.21.15.32:0/1045540041 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c480ea370 con 0x7f6c38003b40 2023-12-20T06:03:19.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.659+0000 7f6c5312d640 1 -- 172.21.15.32:0/1045540041 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f6c4c0d0720 con 0x7f6c38003b40 2023-12-20T06:03:19.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.659+0000 7f6c42ffd640 1 -- 172.21.15.32:0/1045540041 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v226) v1 ==== 93+0+10 (secure 0 0 0) 0x7f6c480ed990 con 0x7f6c38003b40 2023-12-20T06:03:19.660 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:03:19.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 -- 172.21.15.32:0/1045540041 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c30065cd0 msgr2=0x7f6c30068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c30065cd0 0x7f6c30068190 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c007fd0 tx=0x7f6c3c009040 comp rx=0 tx=0).stop 2023-12-20T06:03:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 -- 172.21.15.32:0/1045540041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 msgr2=0x7f6c4c0cbfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 0x7f6c4c0cbfb0 secure :-1 s=READY pgs=2222 cs=0 l=1 rev1=1 crypto rx=0x7f6c4800c160 tx=0x7f6c38005800 comp rx=0 tx=0).stop 2023-12-20T06:03:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 -- 172.21.15.32:0/1045540041 shutdown_connections 2023-12-20T06:03:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c30065cd0 0x7f6c30068190 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4c0c80f0 0x7f6c4c0c8570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c4c1489e0 0x7f6c4c14f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 --2- 172.21.15.32:0/1045540041 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c38003b40 0x7f6c4c0cbfb0 unknown :-1 s=CLOSED pgs=2222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 -- 172.21.15.32:0/1045540041 >> 172.21.15.32:0/1045540041 conn(0x7f6c4c0658e0 msgr2=0x7f6c4c150150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 -- 172.21.15.32:0/1045540041 shutdown_connections 2023-12-20T06:03:19.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.663+0000 7f6c5312d640 1 -- 172.21.15.32:0/1045540041 wait complete. 2023-12-20T06:03:19.682 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T06:03:19.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.827+0000 7fd43957b640 1 Processor -- start 2023-12-20T06:03:19.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.827+0000 7fd43957b640 1 -- start start 2023-12-20T06:03:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.827+0000 7fd43957b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd43406ce60 0x7fd43415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.827+0000 7fd43957b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd43415f110 0x7fd43415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.827+0000 7fd43957b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd43415bbd0 con 0x7fd4340af5e0 2023-12-20T06:03:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.827+0000 7fd43957b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43415beb0 con 0x7fd43406ce60 2023-12-20T06:03:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.827+0000 7fd43957b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd43415e910 con 0x7fd43415f110 2023-12-20T06:03:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.827+0000 7fd42bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd43406ce60 0x7fd43415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd432ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd4340af5e0 0x7fd4340af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37226/0 (socket says 172.21.15.32:37226) 2023-12-20T06:03:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd432ffd640 1 -- 172.21.15.32:0/2320000018 learned_addr learned my addr 172.21.15.32:0/2320000018 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd4337fe640 1 --2- 172.21.15.32:0/2320000018 >> v2:172.21.15.39:3300/0 conn(0x7fd43415f110 0x7fd43415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd42bfff640 1 -- 172.21.15.32:0/2320000018 >> v2:172.21.15.39:3300/0 conn(0x7fd43415f110 msgr2=0x7fd43415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd42bfff640 1 --2- 172.21.15.32:0/2320000018 >> v2:172.21.15.39:3300/0 conn(0x7fd43415f110 0x7fd43415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd42bfff640 1 -- 172.21.15.32:0/2320000018 >> v1:172.21.15.32:6789/0 conn(0x7fd4340af5e0 legacy=0x7fd4340af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd4327fc640 1 -- 172.21.15.32:0/2320000018 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443299919 0 0) 0x7fd43415bbd0 con 0x7fd4340af5e0 2023-12-20T06:03:19.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd42bfff640 1 -- 172.21.15.32:0/2320000018 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd43415ebf0 con 0x7fd43406ce60 2023-12-20T06:03:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd42bfff640 1 --2- 172.21.15.32:0/2320000018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd43406ce60 0x7fd43415b280 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fd4340c8d40 tx=0x7fd41c01c3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad5a55a155f12431 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd4327fc640 1 -- 172.21.15.32:0/2320000018 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd43415b9d0 con 0x7fd43406ce60 2023-12-20T06:03:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd4327fc640 1 -- 172.21.15.32:0/2320000018 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd41c0a7050 con 0x7fd43406ce60 2023-12-20T06:03:19.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.831+0000 7fd4327fc640 1 -- 172.21.15.32:0/2320000018 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd41c0ab820 con 0x7fd43406ce60 2023-12-20T06:03:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 -- 172.21.15.32:0/2320000018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd43406ce60 msgr2=0x7fd43415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 --2- 172.21.15.32:0/2320000018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd43406ce60 0x7fd43415b280 secure :-1 s=READY pgs=2224 cs=0 l=1 rev1=1 crypto rx=0x7fd4340c8d40 tx=0x7fd41c01c3d0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 -- 172.21.15.32:0/2320000018 shutdown_connections 2023-12-20T06:03:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 --2- 172.21.15.32:0/2320000018 >> v2:172.21.15.39:3300/0 conn(0x7fd43415f110 0x7fd43415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 --2- 172.21.15.32:0/2320000018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd43406ce60 0x7fd43415b280 unknown :-1 s=CLOSED pgs=2224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 -- 172.21.15.32:0/2320000018 >> 172.21.15.32:0/2320000018 conn(0x7fd43406a320 msgr2=0x7fd4340ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 -- 172.21.15.32:0/2320000018 shutdown_connections 2023-12-20T06:03:19.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 -- 172.21.15.32:0/2320000018 wait complete. 2023-12-20T06:03:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 Processor -- start 2023-12-20T06:03:19.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 -- start start 2023-12-20T06:03:19.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd43406ce60 0x7fd4340cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4340af5e0 0x7fd4341f54c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd43415f110 0x7fd4341574e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4340cd730 con 0x7fd43415f110 2023-12-20T06:03:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd4341488d0 con 0x7fd43406ce60 2023-12-20T06:03:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd43957b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd4340cce90 con 0x7fd4340af5e0 2023-12-20T06:03:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd4337fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd43415f110 0x7fd4341574e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd4337fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd43415f110 0x7fd4341574e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33238/0 (socket says 172.21.15.32:33238) 2023-12-20T06:03:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd4337fe640 1 -- 172.21.15.32:0/3839144901 learned_addr learned my addr 172.21.15.32:0/3839144901 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd432ffd640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd43406ce60 0x7fd4340cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.835+0000 7fd42bfff640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4340af5e0 0x7fd4341f54c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd4337fe640 1 -- 172.21.15.32:0/3839144901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4340af5e0 msgr2=0x7fd4341f54c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd4337fe640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4340af5e0 0x7fd4341f54c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd4337fe640 1 -- 172.21.15.32:0/3839144901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd43406ce60 msgr2=0x7fd4340cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:19.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd4337fe640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd43406ce60 0x7fd4340cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd4337fe640 1 -- 172.21.15.32:0/3839144901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd41c009420 con 0x7fd43415f110 2023-12-20T06:03:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd4337fe640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd43415f110 0x7fd4341574e0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fd42c007e40 tx=0x7fd42c00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd42b7fe640 1 -- 172.21.15.32:0/3839144901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c014620 con 0x7fd43415f110 2023-12-20T06:03:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd42b7fe640 1 -- 172.21.15.32:0/3839144901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd42c00ee90 con 0x7fd43415f110 2023-12-20T06:03:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd42b7fe640 1 -- 172.21.15.32:0/3839144901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd42c0149e0 con 0x7fd43415f110 2023-12-20T06:03:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd43957b640 1 -- 172.21.15.32:0/3839144901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd4340cd170 con 0x7fd43415f110 2023-12-20T06:03:19.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd43957b640 1 -- 172.21.15.32:0/3839144901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd43414f9a0 con 0x7fd43415f110 2023-12-20T06:03:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd43957b640 1 -- 172.21.15.32:0/3839144901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd43414f800 con 0x7fd43415f110 2023-12-20T06:03:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd42b7fe640 1 -- 172.21.15.32:0/3839144901 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fd42c014b40 con 0x7fd43415f110 2023-12-20T06:03:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd42b7fe640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd414065ca0 0x7fd414068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd42b7fe640 1 -- 172.21.15.32:0/3839144901 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fd42c083c10 con 0x7fd43415f110 2023-12-20T06:03:19.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.839+0000 7fd432ffd640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd414065ca0 0x7fd414068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:19.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.843+0000 7fd432ffd640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd414065ca0 0x7fd414068160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd424007fd0 tx=0x7fd424009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:19.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:19.843+0000 7fd42b7fe640 1 -- 172.21.15.32:0/3839144901 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd43414f800 con 0x7fd43415f110 2023-12-20T06:03:20.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 -- 172.21.15.32:0/3839144901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fd43414f800 con 0x7fd43415f110 2023-12-20T06:03:20.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd42b7fe640 1 -- 172.21.15.32:0/3839144901 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v226) v1 ==== 84+0+65 (secure 0 0 0) 0x7fd43414f800 con 0x7fd43415f110 2023-12-20T06:03:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:20.002 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T06:03:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:20.002 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T06:03:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T06:03:20.003 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:20.003 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T06:03:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 -- 172.21.15.32:0/3839144901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd414065ca0 msgr2=0x7fd414068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd414065ca0 0x7fd414068160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fd424007fd0 tx=0x7fd424009040 comp rx=0 tx=0).stop 2023-12-20T06:03:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 -- 172.21.15.32:0/3839144901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd43415f110 msgr2=0x7fd4341574e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:20.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd43415f110 0x7fd4341574e0 secure :-1 s=READY pgs=2225 cs=0 l=1 rev1=1 crypto rx=0x7fd42c007e40 tx=0x7fd42c00e4b0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 -- 172.21.15.32:0/3839144901 shutdown_connections 2023-12-20T06:03:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd414065ca0 0x7fd414068160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd43415f110 0x7fd4341574e0 unknown :-1 s=CLOSED pgs=2225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd4340af5e0 0x7fd4341f54c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 --2- 172.21.15.32:0/3839144901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd43406ce60 0x7fd4340cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 -- 172.21.15.32:0/3839144901 >> 172.21.15.32:0/3839144901 conn(0x7fd43406a320 msgr2=0x7fd4340cbb10 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 -- 172.21.15.32:0/3839144901 shutdown_connections 2023-12-20T06:03:20.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.003+0000 7fd43957b640 1 -- 172.21.15.32:0/3839144901 wait complete. 2023-12-20T06:03:20.024 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T06:03:20.024 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T06:03:20.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.171+0000 7fad36996640 1 Processor -- start 2023-12-20T06:03:20.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.171+0000 7fad36996640 1 -- start start 2023-12-20T06:03:20.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad36996640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fad30159af0 0x7fad300b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad36996640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad300b0f00 0x7fad3006d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad36996640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fad3006dd50 con 0x7fad3006d2d0 2023-12-20T06:03:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad36996640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad300cc940 con 0x7fad300b0f00 2023-12-20T06:03:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad36996640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fad300ccc20 con 0x7fad30159af0 2023-12-20T06:03:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad34f33640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad300b0f00 0x7fad3006d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad34f33640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fad300b0f00 0x7fad3006d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33250/0 (socket says 172.21.15.32:33250) 2023-12-20T06:03:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad34f33640 1 -- 172.21.15.32:0/3843135588 learned_addr learned my addr 172.21.15.32:0/3843135588 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad2ffff640 1 --2- 172.21.15.32:0/3843135588 >> v2:172.21.15.39:3300/0 conn(0x7fad30159af0 0x7fad300b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad34f33640 1 -- 172.21.15.32:0/3843135588 >> v2:172.21.15.39:3300/0 conn(0x7fad30159af0 msgr2=0x7fad300b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad34f33640 1 --2- 172.21.15.32:0/3843135588 >> v2:172.21.15.39:3300/0 conn(0x7fad30159af0 0x7fad300b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad34f33640 1 -- 172.21.15.32:0/3843135588 >> v1:172.21.15.39:6789/0 conn(0x7fad3006d2d0 legacy=0x7fad300b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:20.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.175+0000 7fad34f33640 1 -- 172.21.15.32:0/3843135588 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad300ccf00 con 0x7fad300b0f00 2023-12-20T06:03:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad34f33640 1 --2- 172.21.15.32:0/3843135588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad300b0f00 0x7fad3006d740 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fad300c8de0 tx=0x7fad2801c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=258efed307be591 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad2effd640 1 -- 172.21.15.32:0/3843135588 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad280a5840 con 0x7fad300b0f00 2023-12-20T06:03:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad2effd640 1 -- 172.21.15.32:0/3843135588 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad280a59a0 con 0x7fad300b0f00 2023-12-20T06:03:20.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad2effd640 1 -- 172.21.15.32:0/3843135588 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad280a5d30 con 0x7fad300b0f00 2023-12-20T06:03:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad36996640 1 -- 172.21.15.32:0/3843135588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad300b0f00 msgr2=0x7fad3006d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad36996640 1 --2- 172.21.15.32:0/3843135588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad300b0f00 0x7fad3006d740 secure :-1 s=READY pgs=2226 cs=0 l=1 rev1=1 crypto rx=0x7fad300c8de0 tx=0x7fad2801c960 comp rx=0 tx=0).stop 2023-12-20T06:03:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad36996640 1 -- 172.21.15.32:0/3843135588 shutdown_connections 2023-12-20T06:03:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad36996640 1 --2- 172.21.15.32:0/3843135588 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad300b0f00 0x7fad3006d740 unknown :-1 s=CLOSED pgs=2226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad36996640 1 --2- 172.21.15.32:0/3843135588 >> v2:172.21.15.39:3300/0 conn(0x7fad30159af0 0x7fad300b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad36996640 1 -- 172.21.15.32:0/3843135588 >> 172.21.15.32:0/3843135588 conn(0x7fad3006a510 msgr2=0x7fad300ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad36996640 1 -- 172.21.15.32:0/3843135588 shutdown_connections 2023-12-20T06:03:20.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad36996640 1 -- 172.21.15.32:0/3843135588 wait complete. 2023-12-20T06:03:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad36996640 1 Processor -- start 2023-12-20T06:03:20.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.179+0000 7fad36996640 1 -- start start 2023-12-20T06:03:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad36996640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3006d2d0 0x7fad30144b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad36996640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad300b0f00 0x7fad3013fd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad36996640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad30159af0 0x7fad300b28e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad36996640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad30143b10 con 0x7fad3006d2d0 2023-12-20T06:03:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad36996640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fad30144670 con 0x7fad30159af0 2023-12-20T06:03:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad36996640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fad30145040 con 0x7fad300b0f00 2023-12-20T06:03:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3006d2d0 0x7fad30144b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3006d2d0 0x7fad30144b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33266/0 (socket says 172.21.15.32:33266) 2023-12-20T06:03:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2ffff640 1 -- 172.21.15.32:0/2502024384 learned_addr learned my addr 172.21.15.32:0/2502024384 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2f7fe640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad300b0f00 0x7fad3013fd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad34f33640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad30159af0 0x7fad300b28e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2ffff640 1 -- 172.21.15.32:0/2502024384 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad300b0f00 msgr2=0x7fad3013fd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2ffff640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad300b0f00 0x7fad3013fd30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2ffff640 1 -- 172.21.15.32:0/2502024384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad30159af0 msgr2=0x7fad300b28e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2ffff640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad30159af0 0x7fad300b28e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2ffff640 1 -- 172.21.15.32:0/2502024384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fad28009b30 con 0x7fad3006d2d0 2023-12-20T06:03:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad34f33640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad30159af0 0x7fad300b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2ffff640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3006d2d0 0x7fad30144b00 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fad2000d5f0 tx=0x7fad2000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:20.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2cff9640 1 -- 172.21.15.32:0/2502024384 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad200125f0 con 0x7fad3006d2d0 2023-12-20T06:03:20.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2cff9640 1 -- 172.21.15.32:0/2502024384 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fad20012750 con 0x7fad3006d2d0 2023-12-20T06:03:20.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2cff9640 1 -- 172.21.15.32:0/2502024384 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fad20012ae0 con 0x7fad3006d2d0 2023-12-20T06:03:20.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad36996640 1 -- 172.21.15.32:0/2502024384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fad300b31e0 con 0x7fad3006d2d0 2023-12-20T06:03:20.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad36996640 1 -- 172.21.15.32:0/2502024384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fad30144020 con 0x7fad3006d2d0 2023-12-20T06:03:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad36996640 1 -- 172.21.15.32:0/2502024384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fad3004e000 con 0x7fad3006d2d0 2023-12-20T06:03:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.183+0000 7fad2cff9640 1 -- 172.21.15.32:0/2502024384 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fad20012c40 con 0x7fad3006d2d0 2023-12-20T06:03:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.187+0000 7fad2cff9640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fad1c065ca0 0x7fad1c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.187+0000 7fad2cff9640 1 -- 172.21.15.32:0/2502024384 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(226..226 src has 1..226) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fad20083750 con 0x7fad3006d2d0 2023-12-20T06:03:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.187+0000 7fad2f7fe640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fad1c065ca0 0x7fad1c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.187+0000 7fad2f7fe640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fad1c065ca0 0x7fad1c068160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad300c44b0 tx=0x7fad24009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:20.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.191+0000 7fad2cff9640 1 -- 172.21.15.32:0/2502024384 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fad200510c0 con 0x7fad3006d2d0 2023-12-20T06:03:20.346 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.347+0000 7fad36996640 1 -- 172.21.15.32:0/2502024384 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fad3014ad10 con 0x7fad3006d2d0 2023-12-20T06:03:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.519+0000 7fad2cff9640 1 -- 172.21.15.32:0/2502024384 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v227) v1 ==== 138+0+0 (secure 0 0 0) 0x7fad20007380 con 0x7fad3006d2d0 2023-12-20T06:03:20.520 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T06:03:20.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 -- 172.21.15.32:0/2502024384 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fad1c065ca0 msgr2=0x7fad1c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:20.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fad1c065ca0 0x7fad1c068160 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fad300c44b0 tx=0x7fad24009040 comp rx=0 tx=0).stop 2023-12-20T06:03:20.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 -- 172.21.15.32:0/2502024384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3006d2d0 msgr2=0x7fad30144b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3006d2d0 0x7fad30144b00 secure :-1 s=READY pgs=2227 cs=0 l=1 rev1=1 crypto rx=0x7fad2000d5f0 tx=0x7fad2000dab0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 -- 172.21.15.32:0/2502024384 shutdown_connections 2023-12-20T06:03:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fad1c065ca0 0x7fad1c068160 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fad30159af0 0x7fad300b28e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fad300b0f00 0x7fad3013fd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 --2- 172.21.15.32:0/2502024384 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fad3006d2d0 0x7fad30144b00 unknown :-1 s=CLOSED pgs=2227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 -- 172.21.15.32:0/2502024384 >> 172.21.15.32:0/2502024384 conn(0x7fad3006a510 msgr2=0x7fad300ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 -- 172.21.15.32:0/2502024384 shutdown_connections 2023-12-20T06:03:20.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.523+0000 7fad36996640 1 -- 172.21.15.32:0/2502024384 wait complete. 2023-12-20T06:03:20.546 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T06:03:20.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96ee0c4640 1 Processor -- start 2023-12-20T06:03:20.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96ee0c4640 1 -- start start 2023-12-20T06:03:20.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96ee0c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96e80cb320 0x7f96e806dbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96ee0c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96e80cb700 0x7f96e8159f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96ee0c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f96e806cbc0 con 0x7f96e815a450 2023-12-20T06:03:20.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96ee0c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f96e806ce30 con 0x7f96e80cb700 2023-12-20T06:03:20.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96ee0c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e806d110 con 0x7f96e80cb320 2023-12-20T06:03:20.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96dffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96e80cb700 0x7f96e8159f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96dffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f96e80cb700 0x7f96e8159f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33282/0 (socket says 172.21.15.32:33282) 2023-12-20T06:03:20.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96dffff640 1 -- 172.21.15.32:0/1929199877 learned_addr learned my addr 172.21.15.32:0/1929199877 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:20.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96e7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f96e815a450 0x7f96e814fa00 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37236/0 (socket says 172.21.15.32:37236) 2023-12-20T06:03:20.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96e77fe640 1 --2- 172.21.15.32:0/1929199877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96e80cb320 0x7f96e806dbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96dffff640 1 -- 172.21.15.32:0/1929199877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96e80cb320 msgr2=0x7f96e806dbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:20.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96dffff640 1 --2- 172.21.15.32:0/1929199877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96e80cb320 0x7f96e806dbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96dffff640 1 -- 172.21.15.32:0/1929199877 >> v1:172.21.15.32:6789/0 conn(0x7f96e815a450 legacy=0x7f96e814fa00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:20.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96dffff640 1 -- 172.21.15.32:0/1929199877 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96e806d3f0 con 0x7f96e80cb700 2023-12-20T06:03:20.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96e77fe640 1 --2- 172.21.15.32:0/1929199877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96e80cb320 0x7f96e806dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:20.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.699+0000 7f96e6ffd640 1 -- 172.21.15.32:0/1929199877 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 278762550 0 0) 0x7f96e806cbc0 con 0x7f96e815a450 2023-12-20T06:03:20.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96dffff640 1 --2- 172.21.15.32:0/1929199877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e80cb700 0x7f96e8159f10 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f96d000aa10 tx=0x7f96d001d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e153c7ac8168b6a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96e6ffd640 1 -- 172.21.15.32:0/1929199877 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00a6880 con 0x7f96e80cb700 2023-12-20T06:03:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96e6ffd640 1 -- 172.21.15.32:0/1929199877 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d00a69e0 con 0x7f96e80cb700 2023-12-20T06:03:20.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96e6ffd640 1 -- 172.21.15.32:0/1929199877 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d00a6d70 con 0x7f96e80cb700 2023-12-20T06:03:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96ee0c4640 1 -- 172.21.15.32:0/1929199877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e80cb700 msgr2=0x7f96e8159f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96ee0c4640 1 --2- 172.21.15.32:0/1929199877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e80cb700 0x7f96e8159f10 secure :-1 s=READY pgs=2229 cs=0 l=1 rev1=1 crypto rx=0x7f96d000aa10 tx=0x7f96d001d910 comp rx=0 tx=0).stop 2023-12-20T06:03:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96ee0c4640 1 -- 172.21.15.32:0/1929199877 shutdown_connections 2023-12-20T06:03:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96ee0c4640 1 --2- 172.21.15.32:0/1929199877 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e80cb700 0x7f96e8159f10 unknown :-1 s=CLOSED pgs=2229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96ee0c4640 1 --2- 172.21.15.32:0/1929199877 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96e80cb320 0x7f96e806dbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96ee0c4640 1 -- 172.21.15.32:0/1929199877 >> 172.21.15.32:0/1929199877 conn(0x7f96e806a4b0 msgr2=0x7f96e80b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96ee0c4640 1 -- 172.21.15.32:0/1929199877 shutdown_connections 2023-12-20T06:03:20.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.703+0000 7f96ee0c4640 1 -- 172.21.15.32:0/1929199877 wait complete. 2023-12-20T06:03:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96ee0c4640 1 Processor -- start 2023-12-20T06:03:20.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96ee0c4640 1 -- start start 2023-12-20T06:03:20.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96ee0c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e80cb320 0x7f96e814c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96ee0c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96e80cb700 0x7f96e81568d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96ee0c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96e815a450 0x7f96e80b3660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96ee0c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e814c540 con 0x7f96e80cb320 2023-12-20T06:03:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96ee0c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80b1360 con 0x7f96e80cb700 2023-12-20T06:03:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96ee0c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f96e80b15b0 con 0x7f96e815a450 2023-12-20T06:03:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e80cb320 0x7f96e814c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e80cb320 0x7f96e814c000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33292/0 (socket says 172.21.15.32:33292) 2023-12-20T06:03:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e77fe640 1 -- 172.21.15.32:0/751546740 learned_addr learned my addr 172.21.15.32:0/751546740 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e7fff640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96e815a450 0x7f96e80b3660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96dffff640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96e80cb700 0x7f96e81568d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e77fe640 1 -- 172.21.15.32:0/751546740 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96e815a450 msgr2=0x7f96e80b3660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e77fe640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96e815a450 0x7f96e80b3660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e77fe640 1 -- 172.21.15.32:0/751546740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96e80cb700 msgr2=0x7f96e81568d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e77fe640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96e80cb700 0x7f96e81568d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e77fe640 1 -- 172.21.15.32:0/751546740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f96d000a670 con 0x7f96e80cb320 2023-12-20T06:03:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e77fe640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e80cb320 0x7f96e814c000 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f96d800e810 tx=0x7f96d800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e4ff9640 1 -- 172.21.15.32:0/751546740 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d800c8d0 con 0x7f96e80cb320 2023-12-20T06:03:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e4ff9640 1 -- 172.21.15.32:0/751546740 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f96d800ca30 con 0x7f96e80cb320 2023-12-20T06:03:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96e4ff9640 1 -- 172.21.15.32:0/751546740 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f96d800cde0 con 0x7f96e80cb320 2023-12-20T06:03:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96ee0c4640 1 -- 172.21.15.32:0/751546740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f96e80b1890 con 0x7f96e80cb320 2023-12-20T06:03:20.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96ee0c4640 1 -- 172.21.15.32:0/751546740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f96e80b1e50 con 0x7f96e80cb320 2023-12-20T06:03:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.707+0000 7f96ee0c4640 1 -- 172.21.15.32:0/751546740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f96e804e000 con 0x7f96e80cb320 2023-12-20T06:03:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.711+0000 7f96e4ff9640 1 -- 172.21.15.32:0/751546740 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f96d8026430 con 0x7f96e80cb320 2023-12-20T06:03:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.711+0000 7f96e4ff9640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f96c8065ca0 0x7f96c8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.711+0000 7f96e4ff9640 1 -- 172.21.15.32:0/751546740 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(227..227 src has 1..227) v4 ==== 5283+0+0 (secure 0 0 0) 0x7f96d80836b0 con 0x7f96e80cb320 2023-12-20T06:03:20.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.711+0000 7f96dffff640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f96c8065ca0 0x7f96c8068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:20.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.711+0000 7f96dffff640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f96c8065ca0 0x7f96c8068160 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f96d00b0000 tx=0x7f96d0008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:20.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.715+0000 7f96e4ff9640 1 -- 172.21.15.32:0/751546740 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f96d8050ec0 con 0x7f96e80cb320 2023-12-20T06:03:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:20.875+0000 7f96ee0c4640 1 -- 172.21.15.32:0/751546740 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7f96e80c3fa0 con 0x7f96e80cb320 2023-12-20T06:03:21.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.547+0000 7f96e4ff9640 1 -- 172.21.15.32:0/751546740 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v228) v1 ==== 170+0+0 (secure 0 0 0) 0x7f96d80544e0 con 0x7f96e80cb320 2023-12-20T06:03:21.547 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T06:03:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.551+0000 7f96ee0c4640 1 -- 172.21.15.32:0/751546740 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f96c8065ca0 msgr2=0x7f96c8068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.551+0000 7f96ee0c4640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f96c8065ca0 0x7f96c8068160 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f96d00b0000 tx=0x7f96d0008040 comp rx=0 tx=0).stop 2023-12-20T06:03:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.551+0000 7f96ee0c4640 1 -- 172.21.15.32:0/751546740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e80cb320 msgr2=0x7f96e814c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.551+0000 7f96ee0c4640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e80cb320 0x7f96e814c000 secure :-1 s=READY pgs=2230 cs=0 l=1 rev1=1 crypto rx=0x7f96d800e810 tx=0x7f96d800ecd0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.551+0000 7f96ee0c4640 1 -- 172.21.15.32:0/751546740 shutdown_connections 2023-12-20T06:03:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.551+0000 7f96ee0c4640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f96c8065ca0 0x7f96c8068160 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.551+0000 7f96ee0c4640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f96e815a450 0x7f96e80b3660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.551+0000 7f96ee0c4640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f96e80cb700 0x7f96e81568d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.551+0000 7f96ee0c4640 1 --2- 172.21.15.32:0/751546740 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f96e80cb320 0x7f96e814c000 unknown :-1 s=CLOSED pgs=2230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.551+0000 7f96ee0c4640 1 -- 172.21.15.32:0/751546740 >> 172.21.15.32:0/751546740 conn(0x7f96e806a4b0 msgr2=0x7f96e80c5050 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:21.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.555+0000 7f96ee0c4640 1 -- 172.21.15.32:0/751546740 shutdown_connections 2023-12-20T06:03:21.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.555+0000 7f96ee0c4640 1 -- 172.21.15.32:0/751546740 wait complete. 2023-12-20T06:03:21.590 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T06:03:21.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.739+0000 7f0549739640 1 Processor -- start 2023-12-20T06:03:21.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.739+0000 7f0549739640 1 -- start start 2023-12-20T06:03:21.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.739+0000 7f0549739640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 0x7f05440caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.739+0000 7f0549739640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054415b6a0 con 0x7f0544070230 2023-12-20T06:03:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.739+0000 7f0549739640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f054406d8c0 con 0x7f054415abb0 2023-12-20T06:03:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.739+0000 7f0549739640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f054406e160 con 0x7f05440b0cb0 2023-12-20T06:03:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.743+0000 7f05437fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0544070230 0x7f054406c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37252/0 (socket says 172.21.15.32:37252) 2023-12-20T06:03:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.743+0000 7f05437fe640 1 -- 172.21.15.32:0/86309286 learned_addr learned my addr 172.21.15.32:0/86309286 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:21.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.743+0000 7f053bfff640 1 --2- 172.21.15.32:0/86309286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 0x7f05440caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.743+0000 7f05427fc640 1 -- 172.21.15.32:0/86309286 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 437920197 0 0) 0x7f054415b6a0 con 0x7f0544070230 2023-12-20T06:03:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.743+0000 7f05427fc640 1 -- 172.21.15.32:0/86309286 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0530003530 con 0x7f0544070230 2023-12-20T06:03:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.743+0000 7f05427fc640 1 -- 172.21.15.32:0/86309286 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 230413398 0 0) 0x7f054406d8c0 con 0x7f054415abb0 2023-12-20T06:03:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.743+0000 7f05427fc640 1 -- 172.21.15.32:0/86309286 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f054415b6a0 con 0x7f054415abb0 2023-12-20T06:03:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.743+0000 7f053bfff640 1 -- 172.21.15.32:0/86309286 >> v1:172.21.15.39:6789/0 conn(0x7f054415abb0 legacy=0x7f054415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.743+0000 7f053bfff640 1 -- 172.21.15.32:0/86309286 >> v1:172.21.15.32:6789/0 conn(0x7f0544070230 legacy=0x7f054406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:21.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.743+0000 7f053bfff640 1 -- 172.21.15.32:0/86309286 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f054406dba0 con 0x7f05440b0cb0 2023-12-20T06:03:21.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.743+0000 7f053bfff640 1 --2- 172.21.15.32:0/86309286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 0x7f05440caaf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f052c0097f0 tx=0x7f052c01c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=efddb25fc86077ee server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:21.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f05427fc640 1 -- 172.21.15.32:0/86309286 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0aa620 con 0x7f05440b0cb0 2023-12-20T06:03:21.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f05427fc640 1 -- 172.21.15.32:0/86309286 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f052c0aa780 con 0x7f05440b0cb0 2023-12-20T06:03:21.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f05427fc640 1 -- 172.21.15.32:0/86309286 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f052c0aaaf0 con 0x7f05440b0cb0 2023-12-20T06:03:21.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 -- 172.21.15.32:0/86309286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 msgr2=0x7f05440caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 --2- 172.21.15.32:0/86309286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 0x7f05440caaf0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f052c0097f0 tx=0x7f052c01c4c0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 -- 172.21.15.32:0/86309286 shutdown_connections 2023-12-20T06:03:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 --2- 172.21.15.32:0/86309286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 0x7f05440caaf0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 -- 172.21.15.32:0/86309286 >> 172.21.15.32:0/86309286 conn(0x7f054406a510 msgr2=0x7f05440ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 -- 172.21.15.32:0/86309286 shutdown_connections 2023-12-20T06:03:21.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 -- 172.21.15.32:0/86309286 wait complete. 2023-12-20T06:03:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 Processor -- start 2023-12-20T06:03:21.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 -- start start 2023-12-20T06:03:21.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0544070230 0x7f05441427c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 0x7f05441402b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054415abb0 0x7f05441493d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0544148eb0 con 0x7f0544070230 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0544151360 con 0x7f054415abb0 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0549739640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0544151ee0 con 0x7f05440b0cb0 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.747+0000 7f0542ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0544070230 0x7f05441427c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f0542ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0544070230 0x7f05441427c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33298/0 (socket says 172.21.15.32:33298) 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f0542ffd640 1 -- 172.21.15.32:0/168419970 learned_addr learned my addr 172.21.15.32:0/168419970 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f053bfff640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 0x7f05441402b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f05437fe640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054415abb0 0x7f05441493d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f0542ffd640 1 -- 172.21.15.32:0/168419970 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 msgr2=0x7f05441402b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f0542ffd640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 0x7f05441402b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f0542ffd640 1 -- 172.21.15.32:0/168419970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054415abb0 msgr2=0x7f05441493d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:21.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f0542ffd640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054415abb0 0x7f05441493d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f0542ffd640 1 -- 172.21.15.32:0/168419970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f052c009420 con 0x7f0544070230 2023-12-20T06:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f053bfff640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 0x7f05441402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f0542ffd640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0544070230 0x7f05441427c0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f053400c460 tx=0x7f053400c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f053b7fe640 1 -- 172.21.15.32:0/168419970 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f05340052d0 con 0x7f0544070230 2023-12-20T06:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f053b7fe640 1 -- 172.21.15.32:0/168419970 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0534005450 con 0x7f0544070230 2023-12-20T06:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f053b7fe640 1 -- 172.21.15.32:0/168419970 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0534012870 con 0x7f0544070230 2023-12-20T06:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f0549739640 1 -- 172.21.15.32:0/168419970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f05441515b0 con 0x7f0544070230 2023-12-20T06:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f0549739640 1 -- 172.21.15.32:0/168419970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0544151b70 con 0x7f0544070230 2023-12-20T06:03:21.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f0549739640 1 -- 172.21.15.32:0/168419970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f054404e000 con 0x7f0544070230 2023-12-20T06:03:21.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f053b7fe640 1 -- 172.21.15.32:0/168419970 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f05340129d0 con 0x7f0544070230 2023-12-20T06:03:21.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.751+0000 7f053b7fe640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0524065ca0 0x7f0524068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:21.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.755+0000 7f053b7fe640 1 -- 172.21.15.32:0/168419970 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(228..228 src has 1..228) v4 ==== 5665+0+0 (secure 0 0 0) 0x7f0534083140 con 0x7f0544070230 2023-12-20T06:03:21.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.755+0000 7f053bfff640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0524065ca0 0x7f0524068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:21.753 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.755+0000 7f053bfff640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0524065ca0 0x7f0524068160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f052c0ae000 tx=0x7f052c0a7000 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:21.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.759+0000 7f053b7fe640 1 -- 172.21.15.32:0/168419970 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0534050720 con 0x7f0544070230 2023-12-20T06:03:21.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.947+0000 7f0549739640 1 -- 172.21.15.32:0/168419970 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f05440b4950 con 0x7f0544070230 2023-12-20T06:03:21.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.955+0000 7f053b7fe640 1 -- 172.21.15.32:0/168419970 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 28 and data pool 29 v108) v1 ==== 174+0+0 (secure 0 0 0) 0x7f0534053d40 con 0x7f0544070230 2023-12-20T06:03:21.955 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 28 and data pool 29 2023-12-20T06:03:21.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.963+0000 7f0549739640 1 -- 172.21.15.32:0/168419970 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0524065ca0 msgr2=0x7f0524068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:21.961 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.963+0000 7f0549739640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0524065ca0 0x7f0524068160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f052c0ae000 tx=0x7f052c0a7000 comp rx=0 tx=0).stop 2023-12-20T06:03:21.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.963+0000 7f0549739640 1 -- 172.21.15.32:0/168419970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0544070230 msgr2=0x7f05441427c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:21.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.963+0000 7f0549739640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0544070230 0x7f05441427c0 secure :-1 s=READY pgs=2232 cs=0 l=1 rev1=1 crypto rx=0x7f053400c460 tx=0x7f053400c920 comp rx=0 tx=0).stop 2023-12-20T06:03:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.967+0000 7f0549739640 1 -- 172.21.15.32:0/168419970 shutdown_connections 2023-12-20T06:03:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.967+0000 7f0549739640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0524065ca0 0x7f0524068160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.967+0000 7f0549739640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f054415abb0 0x7f05441493d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.967+0000 7f0549739640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f05440b0cb0 0x7f05441402b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.967+0000 7f0549739640 1 --2- 172.21.15.32:0/168419970 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0544070230 0x7f05441427c0 unknown :-1 s=CLOSED pgs=2232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:21.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.967+0000 7f0549739640 1 -- 172.21.15.32:0/168419970 >> 172.21.15.32:0/168419970 conn(0x7f054406a510 msgr2=0x7f05440cb950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.967+0000 7f0549739640 1 -- 172.21.15.32:0/168419970 shutdown_connections 2023-12-20T06:03:21.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:21.967+0000 7f0549739640 1 -- 172.21.15.32:0/168419970 wait complete. 2023-12-20T06:03:22.002 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:22.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6f4f9640 1 Processor -- start 2023-12-20T06:03:22.145 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6f4f9640 1 -- start start 2023-12-20T06:03:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6f4f9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5b6815cdb0 0x7f5b680d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6f4f9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b680d2740 0x7f5b680cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6f4f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5b680b6ca0 con 0x7f5b680d1120 2023-12-20T06:03:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6f4f9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b6815d190 con 0x7f5b680d2740 2023-12-20T06:03:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6f4f9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5b680d16a0 con 0x7f5b6815cdb0 2023-12-20T06:03:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6ca94640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b680d2740 0x7f5b680cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6d295640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5b6815cdb0 0x7f5b680d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6ca94640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5b680d2740 0x7f5b680cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33310/0 (socket says 172.21.15.32:33310) 2023-12-20T06:03:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6ca94640 1 -- 172.21.15.32:0/1271418365 learned_addr learned my addr 172.21.15.32:0/1271418365 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:22.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6da96640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5b680d1120 0x7f5b680b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37262/0 (socket says 172.21.15.32:37262) 2023-12-20T06:03:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6d295640 1 --2- 172.21.15.32:0/1271418365 >> v2:172.21.15.39:3300/0 conn(0x7f5b6815cdb0 0x7f5b680d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:36518/0 (socket says 172.21.15.32:36518) 2023-12-20T06:03:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6ca94640 1 -- 172.21.15.32:0/1271418365 >> v2:172.21.15.39:3300/0 conn(0x7f5b6815cdb0 msgr2=0x7f5b680d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6ca94640 1 --2- 172.21.15.32:0/1271418365 >> v2:172.21.15.39:3300/0 conn(0x7f5b6815cdb0 0x7f5b680d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6ca94640 1 -- 172.21.15.32:0/1271418365 >> v1:172.21.15.32:6789/0 conn(0x7f5b680d1120 legacy=0x7f5b680b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.147+0000 7f5b6ca94640 1 -- 172.21.15.32:0/1271418365 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b680d1980 con 0x7f5b680d2740 2023-12-20T06:03:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b6ca94640 1 --2- 172.21.15.32:0/1271418365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b680d2740 0x7f5b680cf570 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f5b680667b0 tx=0x7f5b5801c420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3aac34b014be0a5b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b5ffff640 1 -- 172.21.15.32:0/1271418365 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b580ac620 con 0x7f5b680d2740 2023-12-20T06:03:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b5ffff640 1 -- 172.21.15.32:0/1271418365 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b580ac780 con 0x7f5b680d2740 2023-12-20T06:03:22.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b5ffff640 1 -- 172.21.15.32:0/1271418365 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b580acb10 con 0x7f5b680d2740 2023-12-20T06:03:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1271418365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b680d2740 msgr2=0x7f5b680cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b6f4f9640 1 --2- 172.21.15.32:0/1271418365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b680d2740 0x7f5b680cf570 secure :-1 s=READY pgs=2234 cs=0 l=1 rev1=1 crypto rx=0x7f5b680667b0 tx=0x7f5b5801c420 comp rx=0 tx=0).stop 2023-12-20T06:03:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1271418365 shutdown_connections 2023-12-20T06:03:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b6f4f9640 1 --2- 172.21.15.32:0/1271418365 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b680d2740 0x7f5b680cf570 unknown :-1 s=CLOSED pgs=2234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b6f4f9640 1 --2- 172.21.15.32:0/1271418365 >> v2:172.21.15.39:3300/0 conn(0x7f5b6815cdb0 0x7f5b680d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1271418365 >> 172.21.15.32:0/1271418365 conn(0x7f5b68065ee0 msgr2=0x7f5b6806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1271418365 shutdown_connections 2023-12-20T06:03:22.151 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1271418365 wait complete. 2023-12-20T06:03:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.151+0000 7f5b6f4f9640 1 Processor -- start 2023-12-20T06:03:22.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6f4f9640 1 -- start start 2023-12-20T06:03:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6f4f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5b680d1120 0x7f5b68155ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6f4f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b680d2740 0x7f5b680c8200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6f4f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b6815cdb0 0x7f5b6815fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6f4f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6815dce0 con 0x7f5b6815cdb0 2023-12-20T06:03:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6f4f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5b6815e580 con 0x7f5b680d2740 2023-12-20T06:03:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6f4f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5b680c8740 con 0x7f5b680d1120 2023-12-20T06:03:22.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6da96640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b6815cdb0 0x7f5b6815fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6da96640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b6815cdb0 0x7f5b6815fd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33312/0 (socket says 172.21.15.32:33312) 2023-12-20T06:03:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6da96640 1 -- 172.21.15.32:0/1138294021 learned_addr learned my addr 172.21.15.32:0/1138294021 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6ca94640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b680d2740 0x7f5b680c8200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6d295640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5b680d1120 0x7f5b68155ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6da96640 1 -- 172.21.15.32:0/1138294021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5b680d1120 msgr2=0x7f5b68155ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6da96640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5b680d1120 0x7f5b68155ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6da96640 1 -- 172.21.15.32:0/1138294021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b680d2740 msgr2=0x7f5b680c8200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6da96640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b680d2740 0x7f5b680c8200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6da96640 1 -- 172.21.15.32:0/1138294021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5b580094d0 con 0x7f5b6815cdb0 2023-12-20T06:03:22.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6da96640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b6815cdb0 0x7f5b6815fd80 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005a20 tx=0x7f5b6000d430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b5dffb640 1 -- 172.21.15.32:0/1138294021 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000dba0 con 0x7f5b6815cdb0 2023-12-20T06:03:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b5dffb640 1 -- 172.21.15.32:0/1138294021 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5b6000dd00 con 0x7f5b6815cdb0 2023-12-20T06:03:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b5dffb640 1 -- 172.21.15.32:0/1138294021 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5b6000a310 con 0x7f5b6815cdb0 2023-12-20T06:03:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1138294021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5b681565d0 con 0x7f5b6815cdb0 2023-12-20T06:03:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1138294021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5b6815e210 con 0x7f5b6815cdb0 2023-12-20T06:03:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.155+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1138294021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5b680737e0 con 0x7f5b6815cdb0 2023-12-20T06:03:22.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.159+0000 7f5b5dffb640 1 -- 172.21.15.32:0/1138294021 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f5b6000a470 con 0x7f5b6815cdb0 2023-12-20T06:03:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.159+0000 7f5b5dffb640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5b4c065ca0 0x7f5b4c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.159+0000 7f5b5dffb640 1 -- 172.21.15.32:0/1138294021 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5b60085fe0 con 0x7f5b6815cdb0 2023-12-20T06:03:22.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.159+0000 7f5b6d295640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5b4c065ca0 0x7f5b4c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.159+0000 7f5b6d295640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5b4c065ca0 0x7f5b4c068160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b68158990 tx=0x7f5b64009190 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.163+0000 7f5b5dffb640 1 -- 172.21.15.32:0/1138294021 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5b60053670 con 0x7f5b6815cdb0 2023-12-20T06:03:22.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.319+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1138294021 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f5b680b6ca0 con 0x7f5b6815cdb0 2023-12-20T06:03:22.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b5dffb640 1 -- 172.21.15.32:0/1138294021 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v229) v1 ==== 74+0+16166 (secure 0 0 0) 0x7f5b680b6ca0 con 0x7f5b6815cdb0 2023-12-20T06:03:22.321 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:22.321 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":229,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:21.950779+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:03:20.399088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-20T06:03:20.926367+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:22.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1138294021 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5b4c065ca0 msgr2=0x7f5b4c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5b4c065ca0 0x7f5b4c068160 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f5b68158990 tx=0x7f5b64009190 comp rx=0 tx=0).stop 2023-12-20T06:03:22.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1138294021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b6815cdb0 msgr2=0x7f5b6815fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b6815cdb0 0x7f5b6815fd80 secure :-1 s=READY pgs=2235 cs=0 l=1 rev1=1 crypto rx=0x7f5b60005a20 tx=0x7f5b6000d430 comp rx=0 tx=0).stop 2023-12-20T06:03:22.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1138294021 shutdown_connections 2023-12-20T06:03:22.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5b4c065ca0 0x7f5b4c068160 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5b6815cdb0 0x7f5b6815fd80 unknown :-1 s=CLOSED pgs=2235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5b680d2740 0x7f5b680c8200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 --2- 172.21.15.32:0/1138294021 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5b680d1120 0x7f5b68155ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1138294021 >> 172.21.15.32:0/1138294021 conn(0x7f5b68065ee0 msgr2=0x7f5b680d2b20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:22.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1138294021 shutdown_connections 2023-12-20T06:03:22.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.323+0000 7f5b6f4f9640 1 -- 172.21.15.32:0/1138294021 wait complete. 2023-12-20T06:03:22.343 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:22.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.487+0000 7fcc0c97b640 1 Processor -- start 2023-12-20T06:03:22.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.487+0000 7fcc0c97b640 1 -- start start 2023-12-20T06:03:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.487+0000 7fcc0c97b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcc0815ce60 0x7fcc080c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.487+0000 7fcc0c97b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc080713e0 0x7fcc08071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.487+0000 7fcc0c97b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc08071d80 con 0x7fcc08160b30 2023-12-20T06:03:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.487+0000 7fcc0c97b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc08161690 con 0x7fcc080713e0 2023-12-20T06:03:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.487+0000 7fcc0c97b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc080c9a40 con 0x7fcc0815ce60 2023-12-20T06:03:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.487+0000 7fcbfffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc080713e0 0x7fcc08071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.487+0000 7fcbfffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc080713e0 0x7fcc08071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33316/0 (socket says 172.21.15.32:33316) 2023-12-20T06:03:22.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.487+0000 7fcbfffff640 1 -- 172.21.15.32:0/4175810883 learned_addr learned my addr 172.21.15.32:0/4175810883 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc0659c640 1 --2- 172.21.15.32:0/4175810883 >> v2:172.21.15.39:3300/0 conn(0x7fcc0815ce60 0x7fcc080c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcbfffff640 1 -- 172.21.15.32:0/4175810883 >> v2:172.21.15.39:3300/0 conn(0x7fcc0815ce60 msgr2=0x7fcc080c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcbfffff640 1 --2- 172.21.15.32:0/4175810883 >> v2:172.21.15.39:3300/0 conn(0x7fcc0815ce60 0x7fcc080c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcbfffff640 1 -- 172.21.15.32:0/4175810883 >> v1:172.21.15.39:6789/0 conn(0x7fcc08160b30 legacy=0x7fcc08160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcbfffff640 1 -- 172.21.15.32:0/4175810883 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc080c9180 con 0x7fcc080713e0 2023-12-20T06:03:22.490 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc0659c640 1 --2- 172.21.15.32:0/4175810883 >> v2:172.21.15.39:3300/0 conn(0x7fcc0815ce60 0x7fcc080c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcbfffff640 1 --2- 172.21.15.32:0/4175810883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc080713e0 0x7fcc08071840 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fcc080cc230 tx=0x7fcbf001c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=137e9988eb29d240 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc05d9b640 1 -- 172.21.15.32:0/4175810883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00ac410 con 0x7fcc080713e0 2023-12-20T06:03:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc05d9b640 1 -- 172.21.15.32:0/4175810883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcbf00ac570 con 0x7fcc080713e0 2023-12-20T06:03:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc05d9b640 1 -- 172.21.15.32:0/4175810883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcbf00ac900 con 0x7fcc080713e0 2023-12-20T06:03:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc0c97b640 1 -- 172.21.15.32:0/4175810883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc080713e0 msgr2=0x7fcc08071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc0c97b640 1 --2- 172.21.15.32:0/4175810883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc080713e0 0x7fcc08071840 secure :-1 s=READY pgs=2236 cs=0 l=1 rev1=1 crypto rx=0x7fcc080cc230 tx=0x7fcbf001c4c0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc0c97b640 1 -- 172.21.15.32:0/4175810883 shutdown_connections 2023-12-20T06:03:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc0c97b640 1 --2- 172.21.15.32:0/4175810883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc080713e0 0x7fcc08071840 unknown :-1 s=CLOSED pgs=2236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc0c97b640 1 --2- 172.21.15.32:0/4175810883 >> v2:172.21.15.39:3300/0 conn(0x7fcc0815ce60 0x7fcc080c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc0c97b640 1 -- 172.21.15.32:0/4175810883 >> 172.21.15.32:0/4175810883 conn(0x7fcc0806a320 msgr2=0x7fcc080ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:22.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc0c97b640 1 -- 172.21.15.32:0/4175810883 shutdown_connections 2023-12-20T06:03:22.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.491+0000 7fcc0c97b640 1 -- 172.21.15.32:0/4175810883 wait complete. 2023-12-20T06:03:22.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc0c97b640 1 Processor -- start 2023-12-20T06:03:22.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc0c97b640 1 -- start start 2023-12-20T06:03:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc0c97b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080713e0 0x7fcc08157c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc0c97b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc0815ce60 0x7fcc08144a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc0c97b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08160b30 0x7fcc081426a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc0c97b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc08158180 con 0x7fcc08160b30 2023-12-20T06:03:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc0c97b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc0815fed0 con 0x7fcc080713e0 2023-12-20T06:03:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc0c97b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc081601b0 con 0x7fcc0815ce60 2023-12-20T06:03:22.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc06d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08160b30 0x7fcc081426a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc06d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08160b30 0x7fcc081426a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33328/0 (socket says 172.21.15.32:33328) 2023-12-20T06:03:22.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc06d9d640 1 -- 172.21.15.32:0/779714073 learned_addr learned my addr 172.21.15.32:0/779714073 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:22.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc0659c640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080713e0 0x7fcc08157c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcbfffff640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc0815ce60 0x7fcc08144a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc06d9d640 1 -- 172.21.15.32:0/779714073 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc0815ce60 msgr2=0x7fcc08144a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc06d9d640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc0815ce60 0x7fcc08144a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc06d9d640 1 -- 172.21.15.32:0/779714073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080713e0 msgr2=0x7fcc08157c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc06d9d640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080713e0 0x7fcc08157c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc06d9d640 1 -- 172.21.15.32:0/779714073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcbf00094d0 con 0x7fcc08160b30 2023-12-20T06:03:22.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.495+0000 7fcc06d9d640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08160b30 0x7fcc081426a0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000b970 tx=0x7fcc0000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:22.498 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.499+0000 7fcbfeffd640 1 -- 172.21.15.32:0/779714073 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc0000cb40 con 0x7fcc08160b30 2023-12-20T06:03:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.499+0000 7fcbfeffd640 1 -- 172.21.15.32:0/779714073 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc0000cca0 con 0x7fcc08160b30 2023-12-20T06:03:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.499+0000 7fcc0c97b640 1 -- 172.21.15.32:0/779714073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc08157460 con 0x7fcc08160b30 2023-12-20T06:03:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.499+0000 7fcbfeffd640 1 -- 172.21.15.32:0/779714073 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc00014680 con 0x7fcc08160b30 2023-12-20T06:03:22.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.499+0000 7fcc0c97b640 1 -- 172.21.15.32:0/779714073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc08157160 con 0x7fcc08160b30 2023-12-20T06:03:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.499+0000 7fcc0c97b640 1 -- 172.21.15.32:0/779714073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc0804e000 con 0x7fcc08160b30 2023-12-20T06:03:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.499+0000 7fcbfeffd640 1 -- 172.21.15.32:0/779714073 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fcc0001d3b0 con 0x7fcc08160b30 2023-12-20T06:03:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.499+0000 7fcbfeffd640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcbe8065ca0 0x7fcbe8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.499+0000 7fcbfeffd640 1 -- 172.21.15.32:0/779714073 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcc00083430 con 0x7fcc08160b30 2023-12-20T06:03:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.499+0000 7fcc0659c640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcbe8065ca0 0x7fcbe8068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.499+0000 7fcc0659c640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcbe8065ca0 0x7fcbe8068160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800a990 tx=0x7fcbf8008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:22.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.503+0000 7fcbfeffd640 1 -- 172.21.15.32:0/779714073 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcc00050a50 con 0x7fcc08160b30 2023-12-20T06:03:22.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.663+0000 7fcc0c97b640 1 -- 172.21.15.32:0/779714073 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fcc080cf450 con 0x7fcc08160b30 2023-12-20T06:03:22.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.663+0000 7fcbfeffd640 1 -- 172.21.15.32:0/779714073 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v229) v1 ==== 74+0+16166 (secure 0 0 0) 0x7fcc00054070 con 0x7fcc08160b30 2023-12-20T06:03:22.664 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:22.664 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":229,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:21.950779+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:03:20.399088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-20T06:03:20.926367+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:22.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 -- 172.21.15.32:0/779714073 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcbe8065ca0 msgr2=0x7fcbe8068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcbe8065ca0 0x7fcbe8068160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fcbf800a990 tx=0x7fcbf8008040 comp rx=0 tx=0).stop 2023-12-20T06:03:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 -- 172.21.15.32:0/779714073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08160b30 msgr2=0x7fcc081426a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08160b30 0x7fcc081426a0 secure :-1 s=READY pgs=2237 cs=0 l=1 rev1=1 crypto rx=0x7fcc0000b970 tx=0x7fcc0000be30 comp rx=0 tx=0).stop 2023-12-20T06:03:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 -- 172.21.15.32:0/779714073 shutdown_connections 2023-12-20T06:03:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcbe8065ca0 0x7fcbe8068160 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc08160b30 0x7fcc081426a0 unknown :-1 s=CLOSED pgs=2237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc0815ce60 0x7fcc08144a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 --2- 172.21.15.32:0/779714073 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc080713e0 0x7fcc08157c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 -- 172.21.15.32:0/779714073 >> 172.21.15.32:0/779714073 conn(0x7fcc0806a320 msgr2=0x7fcc08159280 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:22.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 -- 172.21.15.32:0/779714073 shutdown_connections 2023-12-20T06:03:22.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.667+0000 7fcc0c97b640 1 -- 172.21.15.32:0/779714073 wait complete. 2023-12-20T06:03:22.686 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T06:03:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0bea8c640 1 Processor -- start 2023-12-20T06:03:22.833 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0bea8c640 1 -- start start 2023-12-20T06:03:22.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0bea8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc0b8070490 0x7fc0b815fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0bea8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0b806d240 0x7fc0b8160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0bea8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc0b806c480 con 0x7fc0b8070870 2023-12-20T06:03:22.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0bea8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc0b8160750 con 0x7fc0b806d240 2023-12-20T06:03:22.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0bea8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b806c760 con 0x7fc0b8070490 2023-12-20T06:03:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0b7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc0b806d240 0x7fc0b8160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0bd029640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc0b8070870 0x7fc0b80ca1a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37276/0 (socket says 172.21.15.32:37276) 2023-12-20T06:03:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0bd029640 1 -- 172.21.15.32:0/198159876 learned_addr learned my addr 172.21.15.32:0/198159876 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0bc828640 1 --2- 172.21.15.32:0/198159876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc0b8070490 0x7fc0b815fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0b7fff640 1 -- 172.21.15.32:0/198159876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc0b8070490 msgr2=0x7fc0b815fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0b7fff640 1 --2- 172.21.15.32:0/198159876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc0b8070490 0x7fc0b815fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0b7fff640 1 -- 172.21.15.32:0/198159876 >> v1:172.21.15.32:6789/0 conn(0x7fc0b8070870 legacy=0x7fc0b80ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.835+0000 7fc0b7fff640 1 -- 172.21.15.32:0/198159876 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0b806ca40 con 0x7fc0b806d240 2023-12-20T06:03:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0b7fff640 1 --2- 172.21.15.32:0/198159876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806d240 0x7fc0b8160210 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40097c0 tx=0x7fc0a401c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=104c7e7e5ad79e70 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0b77fe640 1 -- 172.21.15.32:0/198159876 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40ab650 con 0x7fc0b806d240 2023-12-20T06:03:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0b77fe640 1 -- 172.21.15.32:0/198159876 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0a40ab7b0 con 0x7fc0b806d240 2023-12-20T06:03:22.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0b77fe640 1 -- 172.21.15.32:0/198159876 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0a40abb40 con 0x7fc0b806d240 2023-12-20T06:03:22.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0bea8c640 1 -- 172.21.15.32:0/198159876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806d240 msgr2=0x7fc0b8160210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0bea8c640 1 --2- 172.21.15.32:0/198159876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806d240 0x7fc0b8160210 secure :-1 s=READY pgs=2239 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40097c0 tx=0x7fc0a401c450 comp rx=0 tx=0).stop 2023-12-20T06:03:22.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0bea8c640 1 -- 172.21.15.32:0/198159876 shutdown_connections 2023-12-20T06:03:22.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0bea8c640 1 --2- 172.21.15.32:0/198159876 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806d240 0x7fc0b8160210 unknown :-1 s=CLOSED pgs=2239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0bea8c640 1 --2- 172.21.15.32:0/198159876 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc0b8070490 0x7fc0b815fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0bea8c640 1 -- 172.21.15.32:0/198159876 >> 172.21.15.32:0/198159876 conn(0x7fc0b806a510 msgr2=0x7fc0b80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:22.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0bea8c640 1 -- 172.21.15.32:0/198159876 shutdown_connections 2023-12-20T06:03:22.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.839+0000 7fc0bea8c640 1 -- 172.21.15.32:0/198159876 wait complete. 2023-12-20T06:03:22.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bea8c640 1 Processor -- start 2023-12-20T06:03:22.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bea8c640 1 -- start start 2023-12-20T06:03:22.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bea8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806d240 0x7fc0b814e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bea8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0b8070490 0x7fc0b8159320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bea8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc0b8070870 0x7fc0b8146d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bea8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b8159860 con 0x7fc0b806d240 2023-12-20T06:03:22.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bea8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b8147260 con 0x7fc0b8070490 2023-12-20T06:03:22.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bea8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc0b814f650 con 0x7fc0b8070870 2023-12-20T06:03:22.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bc828640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806d240 0x7fc0b814e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bc828640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806d240 0x7fc0b814e740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33350/0 (socket says 172.21.15.32:33350) 2023-12-20T06:03:22.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bc828640 1 -- 172.21.15.32:0/848373320 learned_addr learned my addr 172.21.15.32:0/848373320 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:22.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bd029640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc0b8070870 0x7fc0b8146d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0b7fff640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0b8070490 0x7fc0b8159320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bc828640 1 -- 172.21.15.32:0/848373320 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc0b8070870 msgr2=0x7fc0b8146d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bc828640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc0b8070870 0x7fc0b8146d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bc828640 1 -- 172.21.15.32:0/848373320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0b8070490 msgr2=0x7fc0b8159320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bc828640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0b8070490 0x7fc0b8159320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bc828640 1 -- 172.21.15.32:0/848373320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc0a4009420 con 0x7fc0b806d240 2023-12-20T06:03:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bd029640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc0b8070870 0x7fc0b8146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0b7fff640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0b8070490 0x7fc0b8159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bc828640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806d240 0x7fc0b814e740 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00e990 tx=0x7fc0ac00ee50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0b57fa640 1 -- 172.21.15.32:0/848373320 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac009690 con 0x7fc0b806d240 2023-12-20T06:03:22.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0b57fa640 1 -- 172.21.15.32:0/848373320 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc0ac00c9e0 con 0x7fc0b806d240 2023-12-20T06:03:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0b57fa640 1 -- 172.21.15.32:0/848373320 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc0ac00cd50 con 0x7fc0b806d240 2023-12-20T06:03:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bea8c640 1 -- 172.21.15.32:0/848373320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc0b814edb0 con 0x7fc0b806d240 2023-12-20T06:03:22.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bea8c640 1 -- 172.21.15.32:0/848373320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc0b814f370 con 0x7fc0b806d240 2023-12-20T06:03:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.843+0000 7fc0bea8c640 1 -- 172.21.15.32:0/848373320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc0b804e000 con 0x7fc0b806d240 2023-12-20T06:03:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.847+0000 7fc0b57fa640 1 -- 172.21.15.32:0/848373320 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fc0ac013470 con 0x7fc0b806d240 2023-12-20T06:03:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.847+0000 7fc0b57fa640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc09c065ca0 0x7fc09c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.847+0000 7fc0b57fa640 1 -- 172.21.15.32:0/848373320 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(229..229 src has 1..229) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc0ac083880 con 0x7fc0b806d240 2023-12-20T06:03:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.847+0000 7fc0b7fff640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc09c065ca0 0x7fc09c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:22.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.847+0000 7fc0b7fff640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc09c065ca0 0x7fc09c068160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40af000 tx=0x7fc0a40a7000 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:22.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:22.851+0000 7fc0b57fa640 1 -- 172.21.15.32:0/848373320 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc0ac050ed0 con 0x7fc0b806d240 2023-12-20T06:03:23.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.043+0000 7fc0bea8c640 1 -- 172.21.15.32:0/848373320 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7fc0b8159d70 con 0x7fc0b806d240 2023-12-20T06:03:23.376 INFO:tasks.ceph.mds.a.smithi032.stderr:2023-12-20T06:03:23.375+0000 7f0fc3cef640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T06:03:23.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.979+0000 7fc0b57fa640 1 -- 172.21.15.32:0/848373320 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v111) v1 ==== 118+0+0 (secure 0 0 0) 0x7fc0ac0170b0 con 0x7fc0b806d240 2023-12-20T06:03:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 -- 172.21.15.32:0/848373320 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc09c065ca0 msgr2=0x7fc09c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc09c065ca0 0x7fc09c068160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fc0a40af000 tx=0x7fc0a40a7000 comp rx=0 tx=0).stop 2023-12-20T06:03:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 -- 172.21.15.32:0/848373320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806d240 msgr2=0x7fc0b814e740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806d240 0x7fc0b814e740 secure :-1 s=READY pgs=2240 cs=0 l=1 rev1=1 crypto rx=0x7fc0ac00e990 tx=0x7fc0ac00ee50 comp rx=0 tx=0).stop 2023-12-20T06:03:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 -- 172.21.15.32:0/848373320 shutdown_connections 2023-12-20T06:03:23.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc09c065ca0 0x7fc09c068160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc0b8070870 0x7fc0b8146d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc0b8070490 0x7fc0b8159320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 --2- 172.21.15.32:0/848373320 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc0b806d240 0x7fc0b814e740 unknown :-1 s=CLOSED pgs=2240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 -- 172.21.15.32:0/848373320 >> 172.21.15.32:0/848373320 conn(0x7fc0b806a510 msgr2=0x7fc0b80c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 -- 172.21.15.32:0/848373320 shutdown_connections 2023-12-20T06:03:23.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:23.983+0000 7fc0bea8c640 1 -- 172.21.15.32:0/848373320 wait complete. 2023-12-20T06:03:24.005 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T06:03:24.006 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T06:03:24.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.155+0000 7f57815a5640 1 Processor -- start 2023-12-20T06:03:24.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f57815a5640 1 -- start start 2023-12-20T06:03:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f57815a5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f577c0cb050 0x7f577c0cb430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f57815a5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f577c065d40 0x7f577c161320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f57815a5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f577c06ca60 0x7f577c06cf10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f57815a5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c06d450 con 0x7f577c06ca60 2023-12-20T06:03:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f57815a5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f577c06a1f0 con 0x7f577c0cb050 2023-12-20T06:03:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f57815a5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c161860 con 0x7f577c065d40 2023-12-20T06:03:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f577b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f577c06ca60 0x7f577c06cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:24.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f577b7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f577c06ca60 0x7f577c06cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33354/0 (socket says 172.21.15.32:33354) 2023-12-20T06:03:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f577b7fe640 1 -- 172.21.15.32:0/710527786 learned_addr learned my addr 172.21.15.32:0/710527786 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f577affd640 1 --2- 172.21.15.32:0/710527786 >> v2:172.21.15.39:3300/0 conn(0x7f577c0cb050 0x7f577c0cb430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f5773fff640 1 --2- 172.21.15.32:0/710527786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f577c065d40 0x7f577c161320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f577b7fe640 1 -- 172.21.15.32:0/710527786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f577c065d40 msgr2=0x7f577c161320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f577b7fe640 1 --2- 172.21.15.32:0/710527786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f577c065d40 0x7f577c161320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f577b7fe640 1 -- 172.21.15.32:0/710527786 >> v2:172.21.15.39:3300/0 conn(0x7f577c0cb050 msgr2=0x7f577c0cb430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f577b7fe640 1 --2- 172.21.15.32:0/710527786 >> v2:172.21.15.39:3300/0 conn(0x7f577c0cb050 0x7f577c0cb430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f577b7fe640 1 -- 172.21.15.32:0/710527786 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f577c06a440 con 0x7f577c06ca60 2023-12-20T06:03:24.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.159+0000 7f5773fff640 1 --2- 172.21.15.32:0/710527786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f577c065d40 0x7f577c161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:24.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f577b7fe640 1 --2- 172.21.15.32:0/710527786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c06ca60 0x7f577c06cf10 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f577c071ad0 tx=0x7f577401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=228a3ce8b9213b6f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f577a7fc640 1 -- 172.21.15.32:0/710527786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57740a5820 con 0x7f577c06ca60 2023-12-20T06:03:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f577a7fc640 1 -- 172.21.15.32:0/710527786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f57740a5980 con 0x7f577c06ca60 2023-12-20T06:03:24.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f577a7fc640 1 -- 172.21.15.32:0/710527786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f57740a5d10 con 0x7f577c06ca60 2023-12-20T06:03:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f57815a5640 1 -- 172.21.15.32:0/710527786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c06ca60 msgr2=0x7f577c06cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f57815a5640 1 --2- 172.21.15.32:0/710527786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c06ca60 0x7f577c06cf10 secure :-1 s=READY pgs=2241 cs=0 l=1 rev1=1 crypto rx=0x7f577c071ad0 tx=0x7f577401c9a0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f57815a5640 1 -- 172.21.15.32:0/710527786 shutdown_connections 2023-12-20T06:03:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f57815a5640 1 --2- 172.21.15.32:0/710527786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c06ca60 0x7f577c06cf10 unknown :-1 s=CLOSED pgs=2241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f57815a5640 1 --2- 172.21.15.32:0/710527786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f577c065d40 0x7f577c161320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f57815a5640 1 --2- 172.21.15.32:0/710527786 >> v2:172.21.15.39:3300/0 conn(0x7f577c0cb050 0x7f577c0cb430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f57815a5640 1 -- 172.21.15.32:0/710527786 >> 172.21.15.32:0/710527786 conn(0x7f577c0cf8d0 msgr2=0x7f577c0cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:24.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f57815a5640 1 -- 172.21.15.32:0/710527786 shutdown_connections 2023-12-20T06:03:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f57815a5640 1 -- 172.21.15.32:0/710527786 wait complete. 2023-12-20T06:03:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f57815a5640 1 Processor -- start 2023-12-20T06:03:24.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.163+0000 7f57815a5640 1 -- start start 2023-12-20T06:03:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57815a5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c065d40 0x7f577c15b430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57815a5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f577c06ca60 0x7f577c153290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57815a5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f577c0cb050 0x7f577c1525e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57815a5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c149f70 con 0x7f577c065d40 2023-12-20T06:03:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57815a5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f577c154960 con 0x7f577c0cb050 2023-12-20T06:03:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57815a5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f577c153d50 con 0x7f577c06ca60 2023-12-20T06:03:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f577affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c065d40 0x7f577c15b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f577affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c065d40 0x7f577c15b430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33364/0 (socket says 172.21.15.32:33364) 2023-12-20T06:03:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f577affd640 1 -- 172.21.15.32:0/242666514 learned_addr learned my addr 172.21.15.32:0/242666514 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:24.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f5773fff640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f577c06ca60 0x7f577c153290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f577b7fe640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f577c0cb050 0x7f577c1525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f577affd640 1 -- 172.21.15.32:0/242666514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f577c06ca60 msgr2=0x7f577c153290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f577affd640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f577c06ca60 0x7f577c153290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f577affd640 1 -- 172.21.15.32:0/242666514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f577c0cb050 msgr2=0x7f577c1525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f577affd640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f577c0cb050 0x7f577c1525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f577affd640 1 -- 172.21.15.32:0/242666514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5774009b30 con 0x7f577c065d40 2023-12-20T06:03:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f577affd640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c065d40 0x7f577c15b430 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f576c00d800 tx=0x7f576c00dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57737fe640 1 -- 172.21.15.32:0/242666514 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c00b6c0 con 0x7f577c065d40 2023-12-20T06:03:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57737fe640 1 -- 172.21.15.32:0/242666514 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f576c00b820 con 0x7f577c065d40 2023-12-20T06:03:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57815a5640 1 -- 172.21.15.32:0/242666514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f577c154030 con 0x7f577c065d40 2023-12-20T06:03:24.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57737fe640 1 -- 172.21.15.32:0/242666514 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f576c00bb90 con 0x7f577c065d40 2023-12-20T06:03:24.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57815a5640 1 -- 172.21.15.32:0/242666514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f577c1545f0 con 0x7f577c065d40 2023-12-20T06:03:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57815a5640 1 -- 172.21.15.32:0/242666514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f577c04e000 con 0x7f577c065d40 2023-12-20T06:03:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.167+0000 7f57737fe640 1 -- 172.21.15.32:0/242666514 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f576c00bcf0 con 0x7f577c065d40 2023-12-20T06:03:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.171+0000 7f57737fe640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f575c065cd0 0x7f575c068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.171+0000 7f57737fe640 1 -- 172.21.15.32:0/242666514 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f576c083b00 con 0x7f577c065d40 2023-12-20T06:03:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.171+0000 7f5773fff640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f575c065cd0 0x7f575c068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:24.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.171+0000 7f5773fff640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f575c065cd0 0x7f575c068190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5764006fd0 tx=0x7f5764008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:24.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.175+0000 7f57737fe640 1 -- 172.21.15.32:0/242666514 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f576c051100 con 0x7f577c065d40 2023-12-20T06:03:24.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.363+0000 7f57815a5640 1 -- 172.21.15.32:0/242666514 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f577c06ace0 con 0x7f577c065d40 2023-12-20T06:03:24.987 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.987+0000 7f57737fe640 1 -- 172.21.15.32:0/242666514 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v112) v1 ==== 122+0+0 (secure 0 0 0) 0x7f576c054720 con 0x7f577c065d40 2023-12-20T06:03:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 -- 172.21.15.32:0/242666514 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f575c065cd0 msgr2=0x7f575c068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f575c065cd0 0x7f575c068190 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5764006fd0 tx=0x7f5764008040 comp rx=0 tx=0).stop 2023-12-20T06:03:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 -- 172.21.15.32:0/242666514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c065d40 msgr2=0x7f577c15b430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c065d40 0x7f577c15b430 secure :-1 s=READY pgs=2242 cs=0 l=1 rev1=1 crypto rx=0x7f576c00d800 tx=0x7f576c00dcc0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 -- 172.21.15.32:0/242666514 shutdown_connections 2023-12-20T06:03:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f575c065cd0 0x7f575c068190 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f577c0cb050 0x7f577c1525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f577c06ca60 0x7f577c153290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 --2- 172.21.15.32:0/242666514 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f577c065d40 0x7f577c15b430 unknown :-1 s=CLOSED pgs=2242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:24.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 -- 172.21.15.32:0/242666514 >> 172.21.15.32:0/242666514 conn(0x7f577c0cf8d0 msgr2=0x7f577c0c5500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:24.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 -- 172.21.15.32:0/242666514 shutdown_connections 2023-12-20T06:03:24.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:24.991+0000 7f57815a5640 1 -- 172.21.15.32:0/242666514 wait complete. 2023-12-20T06:03:25.011 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T06:03:25.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:25.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387d012640 1 Processor -- start 2023-12-20T06:03:25.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387d012640 1 -- start start 2023-12-20T06:03:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387d012640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38780cfc80 0x7f3878167910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387d012640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f38780b6ca0 con 0x7f387815cdb0 2023-12-20T06:03:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387d012640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f387815d190 con 0x7f38780d2740 2023-12-20T06:03:25.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387d012640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f38780d16a0 con 0x7f38780cfc80 2023-12-20T06:03:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38780cfc80 0x7f3878167910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387759e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f38780cfc80 0x7f3878167910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33372/0 (socket says 172.21.15.32:33372) 2023-12-20T06:03:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387759e640 1 -- 172.21.15.32:0/2812898747 learned_addr learned my addr 172.21.15.32:0/2812898747 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387759e640 1 -- 172.21.15.32:0/2812898747 >> v1:172.21.15.39:6789/0 conn(0x7f38780d2740 legacy=0x7f38780cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387759e640 1 -- 172.21.15.32:0/2812898747 >> v1:172.21.15.32:6789/0 conn(0x7f387815cdb0 legacy=0x7f38780d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.159+0000 7f387759e640 1 -- 172.21.15.32:0/2812898747 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f38780d1980 con 0x7f38780cfc80 2023-12-20T06:03:25.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.163+0000 7f387759e640 1 --2- 172.21.15.32:0/2812898747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38780cfc80 0x7f3878167910 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f3870009e50 tx=0x7f387001f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb84cdd9c9f4511 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.163+0000 7f387659c640 1 -- 172.21.15.32:0/2812898747 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38700af400 con 0x7f38780cfc80 2023-12-20T06:03:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.163+0000 7f387659c640 1 -- 172.21.15.32:0/2812898747 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f38700ab050 con 0x7f38780cfc80 2023-12-20T06:03:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.163+0000 7f387659c640 1 -- 172.21.15.32:0/2812898747 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38700af780 con 0x7f38780cfc80 2023-12-20T06:03:25.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.163+0000 7f387d012640 1 -- 172.21.15.32:0/2812898747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38780cfc80 msgr2=0x7f3878167910 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.163+0000 7f387d012640 1 --2- 172.21.15.32:0/2812898747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38780cfc80 0x7f3878167910 secure :-1 s=READY pgs=2244 cs=0 l=1 rev1=1 crypto rx=0x7f3870009e50 tx=0x7f387001f1c0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.163+0000 7f387d012640 1 -- 172.21.15.32:0/2812898747 shutdown_connections 2023-12-20T06:03:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.163+0000 7f387d012640 1 --2- 172.21.15.32:0/2812898747 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f38780cfc80 0x7f3878167910 unknown :-1 s=CLOSED pgs=2244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.163+0000 7f387d012640 1 -- 172.21.15.32:0/2812898747 >> 172.21.15.32:0/2812898747 conn(0x7f3878065ee0 msgr2=0x7f387806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.163+0000 7f387d012640 1 -- 172.21.15.32:0/2812898747 shutdown_connections 2023-12-20T06:03:25.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.163+0000 7f387d012640 1 -- 172.21.15.32:0/2812898747 wait complete. 2023-12-20T06:03:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387d012640 1 Processor -- start 2023-12-20T06:03:25.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387d012640 1 -- start start 2023-12-20T06:03:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387d012640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38780cfc80 0x7f38781f2770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387d012640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38780d2740 0x7f387814e860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387d012640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387815cdb0 0x7f387814eda0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387d012640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387814f3d0 con 0x7f387815cdb0 2023-12-20T06:03:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387d012640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f387814f6b0 con 0x7f38780cfc80 2023-12-20T06:03:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387d012640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f387814f990 con 0x7f38780d2740 2023-12-20T06:03:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387815cdb0 0x7f387814eda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387759e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387815cdb0 0x7f387814eda0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33388/0 (socket says 172.21.15.32:33388) 2023-12-20T06:03:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387759e640 1 -- 172.21.15.32:0/3735983745 learned_addr learned my addr 172.21.15.32:0/3735983745 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f3876d9d640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38780cfc80 0x7f38781f2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387759e640 1 -- 172.21.15.32:0/3735983745 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38780d2740 msgr2=0x7f387814e860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f386ffff640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38780d2740 0x7f387814e860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387759e640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38780d2740 0x7f387814e860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387759e640 1 -- 172.21.15.32:0/3735983745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38780cfc80 msgr2=0x7f38781f2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387759e640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38780cfc80 0x7f38781f2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387759e640 1 -- 172.21.15.32:0/3735983745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3870009b30 con 0x7f387815cdb0 2023-12-20T06:03:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f3876d9d640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38780cfc80 0x7f38781f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:25.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f386ffff640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38780d2740 0x7f387814e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387759e640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387815cdb0 0x7f387814eda0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f38700b2040 tx=0x7f387001bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f386f7fe640 1 -- 172.21.15.32:0/3735983745 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38700afdc0 con 0x7f387815cdb0 2023-12-20T06:03:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f386f7fe640 1 -- 172.21.15.32:0/3735983745 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3870009510 con 0x7f387815cdb0 2023-12-20T06:03:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f386f7fe640 1 -- 172.21.15.32:0/3735983745 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f38700ba660 con 0x7f387815cdb0 2023-12-20T06:03:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387d012640 1 -- 172.21.15.32:0/3735983745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f387814fc70 con 0x7f387815cdb0 2023-12-20T06:03:25.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.167+0000 7f387d012640 1 -- 172.21.15.32:0/3735983745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f38781f2cb0 con 0x7f387815cdb0 2023-12-20T06:03:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.171+0000 7f387d012640 1 -- 172.21.15.32:0/3735983745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f387804e000 con 0x7f387815cdb0 2023-12-20T06:03:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.171+0000 7f386f7fe640 1 -- 172.21.15.32:0/3735983745 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f38700b6040 con 0x7f387815cdb0 2023-12-20T06:03:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.171+0000 7f386f7fe640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3858065ca0 0x7f3858068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.171+0000 7f386f7fe640 1 -- 172.21.15.32:0/3735983745 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f3870121a10 con 0x7f387815cdb0 2023-12-20T06:03:25.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.171+0000 7f3876d9d640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3858065ca0 0x7f3858068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.171+0000 7f3876d9d640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3858065ca0 0x7f3858068160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3868000f70 tx=0x7f3868009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:25.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.175+0000 7f386f7fe640 1 -- 172.21.15.32:0/3735983745 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f38700eef50 con 0x7f387815cdb0 2023-12-20T06:03:25.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 -- 172.21.15.32:0/3735983745 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f387806b8d0 con 0x7f387815cdb0 2023-12-20T06:03:25.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f386f7fe640 1 -- 172.21.15.32:0/3735983745 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4718 (secure 0 0 0) 0x7f38700b4090 con 0x7f387815cdb0 2023-12-20T06:03:25.366 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:25.366 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":11298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/60134876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":60134876},{"type":"v1","addr":"172.21.15.32:6836","nonce":60134876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:21.951240+0000","modified":"2023-12-20T06:03:24.980742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11301":{"gid":11301,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1623322556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1623322556},{"type":"v1","addr":"172.21.15.32:6837","nonce":1623322556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-12-20T06:03:25.366 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 112 2023-12-20T06:03:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 -- 172.21.15.32:0/3735983745 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3858065ca0 msgr2=0x7f3858068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3858065ca0 0x7f3858068160 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f3868000f70 tx=0x7f3868009040 comp rx=0 tx=0).stop 2023-12-20T06:03:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 -- 172.21.15.32:0/3735983745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387815cdb0 msgr2=0x7f387814eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387815cdb0 0x7f387814eda0 secure :-1 s=READY pgs=2245 cs=0 l=1 rev1=1 crypto rx=0x7f38700b2040 tx=0x7f387001bf60 comp rx=0 tx=0).stop 2023-12-20T06:03:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 -- 172.21.15.32:0/3735983745 shutdown_connections 2023-12-20T06:03:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3858065ca0 0x7f3858068160 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f387815cdb0 0x7f387814eda0 unknown :-1 s=CLOSED pgs=2245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f38780d2740 0x7f387814e860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 --2- 172.21.15.32:0/3735983745 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f38780cfc80 0x7f38781f2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 -- 172.21.15.32:0/3735983745 >> 172.21.15.32:0/3735983745 conn(0x7f3878065ee0 msgr2=0x7f387815b600 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 -- 172.21.15.32:0/3735983745 shutdown_connections 2023-12-20T06:03:25.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.367+0000 7f387d012640 1 -- 172.21.15.32:0/3735983745 wait complete. 2023-12-20T06:03:25.387 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.531+0000 7f299c9b3640 1 Processor -- start 2023-12-20T06:03:25.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.531+0000 7f299c9b3640 1 -- start start 2023-12-20T06:03:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f299c9b3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f29981599e0 0x7f29980aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f299c9b3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29980c98c0 con 0x7f29980b0d00 2023-12-20T06:03:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f299c9b3640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f29980c9ba0 con 0x7f29980c90e0 2023-12-20T06:03:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f299c9b3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f29980c9e80 con 0x7f29981599e0 2023-12-20T06:03:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2996d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f29980b0d00 0x7f29980701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37304/0 (socket says 172.21.15.32:37304) 2023-12-20T06:03:25.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2996d9d640 1 -- 172.21.15.32:0/2012535803 learned_addr learned my addr 172.21.15.32:0/2012535803 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f299659c640 1 --2- 172.21.15.32:0/2012535803 >> v2:172.21.15.39:3300/0 conn(0x7f29981599e0 0x7f29980aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3803865632 0 0) 0x7f29980c98c0 con 0x7f29980b0d00 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2984003430 con 0x7f29980b0d00 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3096110142 0 0) 0x7f2984003430 con 0x7f29980b0d00 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 >> v2:172.21.15.39:3300/0 conn(0x7f29981599e0 msgr2=0x7f29980aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 --2- 172.21.15.32:0/2012535803 >> v2:172.21.15.39:3300/0 conn(0x7f29981599e0 0x7f29980aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 >> v1:172.21.15.39:6789/0 conn(0x7f29980c90e0 legacy=0x7f29980b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29980ca160 con 0x7f29980b0d00 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f2990001a00 con 0x7f29980b0d00 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 >> v1:172.21.15.32:6789/0 conn(0x7f29980b0d00 legacy=0x7f29980701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 --2- 172.21.15.32:0/2012535803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2984004a60 0x7f2984004e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 --2- 172.21.15.32:0/2012535803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2984003670 0x7f2984003af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 --2- 172.21.15.32:0/2012535803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2984004230 0x7f298400fb80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29980c9e80 con 0x7f2984003670 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2984010440 con 0x7f2984004a60 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2984010600 con 0x7f2984004230 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f298ffff640 1 --2- 172.21.15.32:0/2012535803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2984003670 0x7f2984003af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f299659c640 1 --2- 172.21.15.32:0/2012535803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2984004a60 0x7f2984004e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2996d9d640 1 --2- 172.21.15.32:0/2012535803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2984004230 0x7f298400fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f298ffff640 1 -- 172.21.15.32:0/2012535803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2984004230 msgr2=0x7f298400fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f298ffff640 1 --2- 172.21.15.32:0/2012535803 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2984004230 0x7f298400fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f298ffff640 1 -- 172.21.15.32:0/2012535803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2984004a60 msgr2=0x7f2984004e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f298ffff640 1 --2- 172.21.15.32:0/2012535803 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2984004a60 0x7f2984004e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f298ffff640 1 -- 172.21.15.32:0/2012535803 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2984003430 con 0x7f2984003670 2023-12-20T06:03:25.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.535+0000 7f2996d9d640 1 -- 172.21.15.32:0/2012535803 reap_dead start 2023-12-20T06:03:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.539+0000 7f298ffff640 1 --2- 172.21.15.32:0/2012535803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2984003670 0x7f2984003af0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f2980001780 tx=0x7f298001c3e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=876b2f2afe21bee server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.539+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f29800a4ea0 con 0x7f2984003670 2023-12-20T06:03:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.539+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f29800a7050 con 0x7f2984003670 2023-12-20T06:03:25.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.539+0000 7f2995d9b640 1 -- 172.21.15.32:0/2012535803 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2980003600 con 0x7f2984003670 2023-12-20T06:03:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.539+0000 7f299c9b3640 1 -- 172.21.15.32:0/2012535803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2984003670 msgr2=0x7f2984003af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.539+0000 7f299c9b3640 1 --2- 172.21.15.32:0/2012535803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2984003670 0x7f2984003af0 secure :-1 s=READY pgs=2247 cs=0 l=1 rev1=1 crypto rx=0x7f2980001780 tx=0x7f298001c3e0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.539+0000 7f299c9b3640 1 -- 172.21.15.32:0/2012535803 shutdown_connections 2023-12-20T06:03:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.539+0000 7f299c9b3640 1 --2- 172.21.15.32:0/2012535803 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2984003670 0x7f2984003af0 unknown :-1 s=CLOSED pgs=2247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.539+0000 7f299c9b3640 1 -- 172.21.15.32:0/2012535803 >> 172.21.15.32:0/2012535803 conn(0x7f299806a320 msgr2=0x7f29980ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.539+0000 7f299c9b3640 1 -- 172.21.15.32:0/2012535803 shutdown_connections 2023-12-20T06:03:25.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.539+0000 7f299c9b3640 1 -- 172.21.15.32:0/2012535803 wait complete. 2023-12-20T06:03:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299c9b3640 1 Processor -- start 2023-12-20T06:03:25.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299c9b3640 1 -- start start 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299c9b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2984003670 0x7f29980b28d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299c9b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29980c7650 0x7f29980b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299c9b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29980b2e10 0x7f29981f1860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299c9b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29980b1830 con 0x7f29980b2e10 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299c9b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f29980b1b10 con 0x7f2984003670 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299c9b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f29980b1df0 con 0x7f29980c7650 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f2996d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29980b2e10 0x7f29981f1860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f2996d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29980b2e10 0x7f29981f1860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33404/0 (socket says 172.21.15.32:33404) 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f2996d9d640 1 -- 172.21.15.32:0/1166505336 learned_addr learned my addr 172.21.15.32:0/1166505336 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299659c640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2984003670 0x7f29980b28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f298ffff640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29980c7650 0x7f29980b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f2996d9d640 1 -- 172.21.15.32:0/1166505336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29980c7650 msgr2=0x7f29980b3be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f2996d9d640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29980c7650 0x7f29980b3be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f2996d9d640 1 -- 172.21.15.32:0/1166505336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2984003670 msgr2=0x7f29980b28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f2996d9d640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2984003670 0x7f29980b28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f2996d9d640 1 -- 172.21.15.32:0/1166505336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f29980b20d0 con 0x7f29980b2e10 2023-12-20T06:03:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299659c640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2984003670 0x7f29980b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f298ffff640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29980c7650 0x7f29980b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:25.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f2996d9d640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29980b2e10 0x7f29981f1860 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f2990002240 tx=0x7f2990011c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f298effd640 1 -- 172.21.15.32:0/1166505336 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299000f8a0 con 0x7f29980b2e10 2023-12-20T06:03:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f298effd640 1 -- 172.21.15.32:0/1166505336 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f299000fa40 con 0x7f29980b2e10 2023-12-20T06:03:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299c9b3640 1 -- 172.21.15.32:0/1166505336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f29980b23b0 con 0x7f29980b2e10 2023-12-20T06:03:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f298effd640 1 -- 172.21.15.32:0/1166505336 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f299000fdf0 con 0x7f29980b2e10 2023-12-20T06:03:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.543+0000 7f299c9b3640 1 -- 172.21.15.32:0/1166505336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f29980b4120 con 0x7f29980b2e10 2023-12-20T06:03:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.547+0000 7f299c9b3640 1 -- 172.21.15.32:0/1166505336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f29980b25a0 con 0x7f29980b2e10 2023-12-20T06:03:25.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.547+0000 7f298effd640 1 -- 172.21.15.32:0/1166505336 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f299001f460 con 0x7f29980b2e10 2023-12-20T06:03:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.547+0000 7f298effd640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2978065ca0 0x7f2978068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.547+0000 7f298effd640 1 -- 172.21.15.32:0/1166505336 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f29900865c0 con 0x7f29980b2e10 2023-12-20T06:03:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.547+0000 7f299659c640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2978065ca0 0x7f2978068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.547+0000 7f299659c640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2978065ca0 0x7f2978068160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2988009c40 tx=0x7f2988008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:25.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.551+0000 7f298effd640 1 -- 172.21.15.32:0/1166505336 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2990053bd0 con 0x7f29980b2e10 2023-12-20T06:03:25.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.711+0000 7f299c9b3640 1 -- 172.21.15.32:0/1166505336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f29980b4310 con 0x7f29980b2e10 2023-12-20T06:03:25.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f298effd640 1 -- 172.21.15.32:0/1166505336 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v230) v1 ==== 74+0+16153 (secure 0 0 0) 0x7f29900571f0 con 0x7f29980b2e10 2023-12-20T06:03:25.713 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:25.713 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":230,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:22.953603+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:03:20.399088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-20T06:03:20.926367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:25.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 -- 172.21.15.32:0/1166505336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2978065ca0 msgr2=0x7f2978068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2978065ca0 0x7f2978068160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f2988009c40 tx=0x7f2988008040 comp rx=0 tx=0).stop 2023-12-20T06:03:25.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 -- 172.21.15.32:0/1166505336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29980b2e10 msgr2=0x7f29981f1860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29980b2e10 0x7f29981f1860 secure :-1 s=READY pgs=2248 cs=0 l=1 rev1=1 crypto rx=0x7f2990002240 tx=0x7f2990011c20 comp rx=0 tx=0).stop 2023-12-20T06:03:25.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 -- 172.21.15.32:0/1166505336 shutdown_connections 2023-12-20T06:03:25.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2978065ca0 0x7f2978068160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f29980b2e10 0x7f29981f1860 unknown :-1 s=CLOSED pgs=2248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f29980c7650 0x7f29980b3be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 --2- 172.21.15.32:0/1166505336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2984003670 0x7f29980b28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 -- 172.21.15.32:0/1166505336 >> 172.21.15.32:0/1166505336 conn(0x7f299806a320 msgr2=0x7f29980cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 -- 172.21.15.32:0/1166505336 shutdown_connections 2023-12-20T06:03:25.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.715+0000 7f299c9b3640 1 -- 172.21.15.32:0/1166505336 wait complete. 2023-12-20T06:03:25.735 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T06:03:25.735 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:03:25.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.879+0000 7ff6fe52c640 1 Processor -- start 2023-12-20T06:03:25.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.879+0000 7ff6fe52c640 1 -- start start 2023-12-20T06:03:25.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fe52c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff6f81599e0 0x7ff6f80aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fe52c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6f80b0d00 0x7ff6f80701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fe52c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6f80c98c0 con 0x7ff6f80c90e0 2023-12-20T06:03:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fe52c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f80c9ba0 con 0x7ff6f80b0d00 2023-12-20T06:03:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fe52c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6f80c9e80 con 0x7ff6f81599e0 2023-12-20T06:03:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fcac9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6f80b0d00 0x7ff6f80701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fcac9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff6f80b0d00 0x7ff6f80701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33414/0 (socket says 172.21.15.32:33414) 2023-12-20T06:03:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fcac9640 1 -- 172.21.15.32:0/2039284656 learned_addr learned my addr 172.21.15.32:0/2039284656 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6f7fff640 1 --2- 172.21.15.32:0/2039284656 >> v2:172.21.15.39:3300/0 conn(0x7ff6f81599e0 0x7ff6f80aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fcac9640 1 -- 172.21.15.32:0/2039284656 >> v2:172.21.15.39:3300/0 conn(0x7ff6f81599e0 msgr2=0x7ff6f80aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fcac9640 1 --2- 172.21.15.32:0/2039284656 >> v2:172.21.15.39:3300/0 conn(0x7ff6f81599e0 0x7ff6f80aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fcac9640 1 -- 172.21.15.32:0/2039284656 >> v1:172.21.15.39:6789/0 conn(0x7ff6f80c90e0 legacy=0x7ff6f80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fcac9640 1 -- 172.21.15.32:0/2039284656 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f80ca160 con 0x7ff6f80b0d00 2023-12-20T06:03:25.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.883+0000 7ff6fcac9640 1 --2- 172.21.15.32:0/2039284656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f80b0d00 0x7ff6f80701c0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7ff6f806e3c0 tx=0x7ff6f001c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=233008f7c079e56f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6f6ffd640 1 -- 172.21.15.32:0/2039284656 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00a5880 con 0x7ff6f80b0d00 2023-12-20T06:03:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6f6ffd640 1 -- 172.21.15.32:0/2039284656 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f00a59e0 con 0x7ff6f80b0d00 2023-12-20T06:03:25.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6f6ffd640 1 -- 172.21.15.32:0/2039284656 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00a5d70 con 0x7ff6f80b0d00 2023-12-20T06:03:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 -- 172.21.15.32:0/2039284656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f80b0d00 msgr2=0x7ff6f80701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 --2- 172.21.15.32:0/2039284656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f80b0d00 0x7ff6f80701c0 secure :-1 s=READY pgs=2249 cs=0 l=1 rev1=1 crypto rx=0x7ff6f806e3c0 tx=0x7ff6f001c9a0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 -- 172.21.15.32:0/2039284656 shutdown_connections 2023-12-20T06:03:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 --2- 172.21.15.32:0/2039284656 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f80b0d00 0x7ff6f80701c0 unknown :-1 s=CLOSED pgs=2249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 --2- 172.21.15.32:0/2039284656 >> v2:172.21.15.39:3300/0 conn(0x7ff6f81599e0 0x7ff6f80aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 -- 172.21.15.32:0/2039284656 >> 172.21.15.32:0/2039284656 conn(0x7ff6f806a320 msgr2=0x7ff6f80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 -- 172.21.15.32:0/2039284656 shutdown_connections 2023-12-20T06:03:25.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 -- 172.21.15.32:0/2039284656 wait complete. 2023-12-20T06:03:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 Processor -- start 2023-12-20T06:03:25.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 -- start start 2023-12-20T06:03:25.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f80b0d00 0x7ff6f81f8cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6f80c90e0 0x7ff6f80c7320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f81599e0 0x7ff6f8154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f814c6d0 con 0x7ff6f81599e0 2023-12-20T06:03:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f81f9230 con 0x7ff6f80b0d00 2023-12-20T06:03:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fe52c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6f80c5df0 con 0x7ff6f80c90e0 2023-12-20T06:03:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fcac9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f81599e0 0x7ff6f8154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fcac9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f81599e0 0x7ff6f8154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33430/0 (socket says 172.21.15.32:33430) 2023-12-20T06:03:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.887+0000 7ff6fcac9640 1 -- 172.21.15.32:0/877486233 learned_addr learned my addr 172.21.15.32:0/877486233 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6f7fff640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f80b0d00 0x7ff6f81f8cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6f77fe640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6f80c90e0 0x7ff6f80c7320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6fcac9640 1 -- 172.21.15.32:0/877486233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6f80c90e0 msgr2=0x7ff6f80c7320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6fcac9640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6f80c90e0 0x7ff6f80c7320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6fcac9640 1 -- 172.21.15.32:0/877486233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f80b0d00 msgr2=0x7ff6f81f8cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6fcac9640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f80b0d00 0x7ff6f81f8cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6fcac9640 1 -- 172.21.15.32:0/877486233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6f0009b30 con 0x7ff6f81599e0 2023-12-20T06:03:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6f7fff640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f80b0d00 0x7ff6f81f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6f77fe640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6f80c90e0 0x7ff6f80c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6fcac9640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f81599e0 0x7ff6f8154f10 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00195b0 tx=0x7ff6f0019740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6f4ff9640 1 -- 172.21.15.32:0/877486233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00b8670 con 0x7ff6f81599e0 2023-12-20T06:03:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6f4ff9640 1 -- 172.21.15.32:0/877486233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6f00b87f0 con 0x7ff6f81599e0 2023-12-20T06:03:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6f4ff9640 1 -- 172.21.15.32:0/877486233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6f00b8ba0 con 0x7ff6f81599e0 2023-12-20T06:03:25.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6fe52c640 1 -- 172.21.15.32:0/877486233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6f80c60d0 con 0x7ff6f81599e0 2023-12-20T06:03:25.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6fe52c640 1 -- 172.21.15.32:0/877486233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6f80c6690 con 0x7ff6f81599e0 2023-12-20T06:03:25.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6fe52c640 1 -- 172.21.15.32:0/877486233 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6f814c840 con 0x7ff6f81599e0 2023-12-20T06:03:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6f4ff9640 1 -- 172.21.15.32:0/877486233 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff6f00b8d00 con 0x7ff6f81599e0 2023-12-20T06:03:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6f4ff9640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6e4065ca0 0x7ff6e4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.891+0000 7ff6f4ff9640 1 -- 172.21.15.32:0/877486233 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff6f011eaf0 con 0x7ff6f81599e0 2023-12-20T06:03:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.895+0000 7ff6f7fff640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6e4065ca0 0x7ff6e4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:25.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.895+0000 7ff6f7fff640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6e4065ca0 0x7ff6e4068160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009b90 tx=0x7ff6e8008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:25.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:25.895+0000 7ff6f4ff9640 1 -- 172.21.15.32:0/877486233 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff6f00ec1b0 con 0x7ff6f81599e0 2023-12-20T06:03:26.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.051+0000 7ff6fe52c640 1 -- 172.21.15.32:0/877486233 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7ff6f8159dc0 con 0x7ff6e4065ca0 2023-12-20T06:03:26.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.063+0000 7ff6f4ff9640 1 -- 172.21.15.32:0/877486233 <== mgr.10956 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289552 (secure 0 0 0) 0x7ff6e8011a60 con 0x7ff6e4065ca0 2023-12-20T06:03:26.065 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:26.067 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:03:26.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 -- 172.21.15.32:0/877486233 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6e4065ca0 msgr2=0x7ff6e4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6e4065ca0 0x7ff6e4068160 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7ff6e8009b90 tx=0x7ff6e8008040 comp rx=0 tx=0).stop 2023-12-20T06:03:26.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 -- 172.21.15.32:0/877486233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f81599e0 msgr2=0x7ff6f8154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f81599e0 0x7ff6f8154f10 secure :-1 s=READY pgs=2250 cs=0 l=1 rev1=1 crypto rx=0x7ff6f00195b0 tx=0x7ff6f0019740 comp rx=0 tx=0).stop 2023-12-20T06:03:26.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 -- 172.21.15.32:0/877486233 shutdown_connections 2023-12-20T06:03:26.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6e4065ca0 0x7ff6e4068160 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6f81599e0 0x7ff6f8154f10 unknown :-1 s=CLOSED pgs=2250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6f80c90e0 0x7ff6f80c7320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 --2- 172.21.15.32:0/877486233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6f80b0d00 0x7ff6f81f8cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 -- 172.21.15.32:0/877486233 >> 172.21.15.32:0/877486233 conn(0x7ff6f806a320 msgr2=0x7ff6f80714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:26.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 -- 172.21.15.32:0/877486233 shutdown_connections 2023-12-20T06:03:26.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.067+0000 7ff6fe52c640 1 -- 172.21.15.32:0/877486233 wait complete. 2023-12-20T06:03:26.085 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-12-20T06:03:25.658460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":605,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3261,"ondisk_log_size":3261,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":428860,"kb_used_data":23804,"kb_used_omap":0,"kb_used_meta":404992,"kb_avail":754545860,"statfs":{"total":773094113280,"available":772654960640,"internally_reserved":0,"allocated":24375296,"data_stored":12475368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":414711808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.136924"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963240+0000","last_change":"2023-12-20T06:03:21.976746+0000","last_active":"2023-12-20T06:03:22.963240+0000","last_peered":"2023-12-20T06:03:22.963240+0000","last_clean":"2023-12-20T06:03:22.963240+0000","last_became_active":"2023-12-20T06:03:21.976467+0000","last_became_peered":"2023-12-20T06:03:21.976467+0000","last_unstale":"2023-12-20T06:03:22.963240+0000","last_undegraded":"2023-12-20T06:03:22.963240+0000","last_fullsized":"2023-12-20T06:03:22.963240+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:13.213910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089596+0000","last_change":"2023-12-20T06:03:21.988965+0000","last_active":"2023-12-20T06:03:24.089596+0000","last_peered":"2023-12-20T06:03:24.089596+0000","last_clean":"2023-12-20T06:03:24.089596+0000","last_became_active":"2023-12-20T06:03:21.988334+0000","last_became_peered":"2023-12-20T06:03:21.988334+0000","last_unstale":"2023-12-20T06:03:24.089596+0000","last_undegraded":"2023-12-20T06:03:24.089596+0000","last_fullsized":"2023-12-20T06:03:24.089596+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:04.642017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963412+0000","last_change":"2023-12-20T06:03:21.987434+0000","last_active":"2023-12-20T06:03:22.963412+0000","last_peered":"2023-12-20T06:03:22.963412+0000","last_clean":"2023-12-20T06:03:22.963412+0000","last_became_active":"2023-12-20T06:03:21.987118+0000","last_became_peered":"2023-12-20T06:03:21.987118+0000","last_unstale":"2023-12-20T06:03:22.963412+0000","last_undegraded":"2023-12-20T06:03:22.963412+0000","last_fullsized":"2023-12-20T06:03:22.963412+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817040+0000","last_change":"2023-12-20T06:03:21.981328+0000","last_active":"2023-12-20T06:03:24.817040+0000","last_peered":"2023-12-20T06:03:24.817040+0000","last_clean":"2023-12-20T06:03:24.817040+0000","last_became_active":"2023-12-20T06:03:21.981001+0000","last_became_peered":"2023-12-20T06:03:21.981001+0000","last_unstale":"2023-12-20T06:03:24.817040+0000","last_undegraded":"2023-12-20T06:03:24.817040+0000","last_fullsized":"2023-12-20T06:03:24.817040+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:48.782916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816928+0000","last_change":"2023-12-20T06:03:21.978878+0000","last_active":"2023-12-20T06:03:24.816928+0000","last_peered":"2023-12-20T06:03:24.816928+0000","last_clean":"2023-12-20T06:03:24.816928+0000","last_became_active":"2023-12-20T06:03:21.977017+0000","last_became_peered":"2023-12-20T06:03:21.977017+0000","last_unstale":"2023-12-20T06:03:24.816928+0000","last_undegraded":"2023-12-20T06:03:24.816928+0000","last_fullsized":"2023-12-20T06:03:24.816928+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:55:10.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115451+0000","last_change":"2023-12-20T06:03:21.976982+0000","last_active":"2023-12-20T06:03:23.115451+0000","last_peered":"2023-12-20T06:03:23.115451+0000","last_clean":"2023-12-20T06:03:23.115451+0000","last_became_active":"2023-12-20T06:03:21.976721+0000","last_became_peered":"2023-12-20T06:03:21.976721+0000","last_unstale":"2023-12-20T06:03:23.115451+0000","last_undegraded":"2023-12-20T06:03:23.115451+0000","last_fullsized":"2023-12-20T06:03:23.115451+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:57.966779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962469+0000","last_change":"2023-12-20T06:03:21.979840+0000","last_active":"2023-12-20T06:03:22.962469+0000","last_peered":"2023-12-20T06:03:22.962469+0000","last_clean":"2023-12-20T06:03:22.962469+0000","last_became_active":"2023-12-20T06:03:21.979570+0000","last_became_peered":"2023-12-20T06:03:21.979570+0000","last_unstale":"2023-12-20T06:03:22.962469+0000","last_undegraded":"2023-12-20T06:03:22.962469+0000","last_fullsized":"2023-12-20T06:03:22.962469+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.958428+0000","last_change":"2023-12-20T06:03:21.976244+0000","last_active":"2023-12-20T06:03:22.958428+0000","last_peered":"2023-12-20T06:03:22.958428+0000","last_clean":"2023-12-20T06:03:22.958428+0000","last_became_active":"2023-12-20T06:03:21.975968+0000","last_became_peered":"2023-12-20T06:03:21.975968+0000","last_unstale":"2023-12-20T06:03:22.958428+0000","last_undegraded":"2023-12-20T06:03:22.958428+0000","last_fullsized":"2023-12-20T06:03:22.958428+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:07:21.324505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963729+0000","last_change":"2023-12-20T06:03:21.987598+0000","last_active":"2023-12-20T06:03:22.963729+0000","last_peered":"2023-12-20T06:03:22.963729+0000","last_clean":"2023-12-20T06:03:22.963729+0000","last_became_active":"2023-12-20T06:03:21.987291+0000","last_became_peered":"2023-12-20T06:03:21.987291+0000","last_unstale":"2023-12-20T06:03:22.963729+0000","last_undegraded":"2023-12-20T06:03:22.963729+0000","last_fullsized":"2023-12-20T06:03:22.963729+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:04.642017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115682+0000","last_change":"2023-12-20T06:03:21.969809+0000","last_active":"2023-12-20T06:03:23.115682+0000","last_peered":"2023-12-20T06:03:23.115682+0000","last_clean":"2023-12-20T06:03:23.115682+0000","last_became_active":"2023-12-20T06:03:21.969515+0000","last_became_peered":"2023-12-20T06:03:21.969515+0000","last_unstale":"2023-12-20T06:03:23.115682+0000","last_undegraded":"2023-12-20T06:03:23.115682+0000","last_fullsized":"2023-12-20T06:03:23.115682+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:15:47.777614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962623+0000","last_change":"2023-12-20T06:03:21.979757+0000","last_active":"2023-12-20T06:03:22.962623+0000","last_peered":"2023-12-20T06:03:22.962623+0000","last_clean":"2023-12-20T06:03:22.962623+0000","last_became_active":"2023-12-20T06:03:21.979456+0000","last_became_peered":"2023-12-20T06:03:21.979456+0000","last_unstale":"2023-12-20T06:03:22.962623+0000","last_undegraded":"2023-12-20T06:03:22.962623+0000","last_fullsized":"2023-12-20T06:03:22.962623+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:23.805328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089484+0000","last_change":"2023-12-20T06:03:21.982252+0000","last_active":"2023-12-20T06:03:24.089484+0000","last_peered":"2023-12-20T06:03:24.089484+0000","last_clean":"2023-12-20T06:03:24.089484+0000","last_became_active":"2023-12-20T06:03:21.981980+0000","last_became_peered":"2023-12-20T06:03:21.981980+0000","last_unstale":"2023-12-20T06:03:24.089484+0000","last_undegraded":"2023-12-20T06:03:24.089484+0000","last_fullsized":"2023-12-20T06:03:24.089484+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:13.633088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089323+0000","last_change":"2023-12-20T06:03:21.975662+0000","last_active":"2023-12-20T06:03:24.089323+0000","last_peered":"2023-12-20T06:03:24.089323+0000","last_clean":"2023-12-20T06:03:24.089323+0000","last_became_active":"2023-12-20T06:03:21.975017+0000","last_became_peered":"2023-12-20T06:03:21.975017+0000","last_unstale":"2023-12-20T06:03:24.089323+0000","last_undegraded":"2023-12-20T06:03:24.089323+0000","last_fullsized":"2023-12-20T06:03:24.089323+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:06:16.446428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.442758+0000","last_change":"2023-12-20T06:03:21.973459+0000","last_active":"2023-12-20T06:03:23.442758+0000","last_peered":"2023-12-20T06:03:23.442758+0000","last_clean":"2023-12-20T06:03:23.442758+0000","last_became_active":"2023-12-20T06:03:21.973101+0000","last_became_peered":"2023-12-20T06:03:21.973101+0000","last_unstale":"2023-12-20T06:03:23.442758+0000","last_undegraded":"2023-12-20T06:03:23.442758+0000","last_fullsized":"2023-12-20T06:03:23.442758+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:03:34.663696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816300+0000","last_change":"2023-12-20T06:03:21.982673+0000","last_active":"2023-12-20T06:03:24.816300+0000","last_peered":"2023-12-20T06:03:24.816300+0000","last_clean":"2023-12-20T06:03:24.816300+0000","last_became_active":"2023-12-20T06:03:21.982010+0000","last_became_peered":"2023-12-20T06:03:21.982010+0000","last_unstale":"2023-12-20T06:03:24.816300+0000","last_undegraded":"2023-12-20T06:03:24.816300+0000","last_fullsized":"2023-12-20T06:03:24.816300+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:32:19.664417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089253+0000","last_change":"2023-12-20T06:03:21.977347+0000","last_active":"2023-12-20T06:03:24.089253+0000","last_peered":"2023-12-20T06:03:24.089253+0000","last_clean":"2023-12-20T06:03:24.089253+0000","last_became_active":"2023-12-20T06:03:21.977040+0000","last_became_peered":"2023-12-20T06:03:21.977040+0000","last_unstale":"2023-12-20T06:03:24.089253+0000","last_undegraded":"2023-12-20T06:03:24.089253+0000","last_fullsized":"2023-12-20T06:03:24.089253+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.443179+0000","last_change":"2023-12-20T06:03:21.973052+0000","last_active":"2023-12-20T06:03:23.443179+0000","last_peered":"2023-12-20T06:03:23.443179+0000","last_clean":"2023-12-20T06:03:23.443179+0000","last_became_active":"2023-12-20T06:03:21.972760+0000","last_became_peered":"2023-12-20T06:03:21.972760+0000","last_unstale":"2023-12-20T06:03:23.443179+0000","last_undegraded":"2023-12-20T06:03:23.443179+0000","last_fullsized":"2023-12-20T06:03:23.443179+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:16:41.285330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115729+0000","last_change":"2023-12-20T06:03:21.991211+0000","last_active":"2023-12-20T06:03:23.115729+0000","last_peered":"2023-12-20T06:03:23.115729+0000","last_clean":"2023-12-20T06:03:23.115729+0000","last_became_active":"2023-12-20T06:03:21.990873+0000","last_became_peered":"2023-12-20T06:03:21.990873+0000","last_unstale":"2023-12-20T06:03:23.115729+0000","last_undegraded":"2023-12-20T06:03:23.115729+0000","last_fullsized":"2023-12-20T06:03:23.115729+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:31:08.457333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817251+0000","last_change":"2023-12-20T06:03:21.982366+0000","last_active":"2023-12-20T06:03:24.817251+0000","last_peered":"2023-12-20T06:03:24.817251+0000","last_clean":"2023-12-20T06:03:24.817251+0000","last_became_active":"2023-12-20T06:03:21.978098+0000","last_became_peered":"2023-12-20T06:03:21.978098+0000","last_unstale":"2023-12-20T06:03:24.817251+0000","last_undegraded":"2023-12-20T06:03:24.817251+0000","last_fullsized":"2023-12-20T06:03:24.817251+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115560+0000","last_change":"2023-12-20T06:03:21.967196+0000","last_active":"2023-12-20T06:03:23.115560+0000","last_peered":"2023-12-20T06:03:23.115560+0000","last_clean":"2023-12-20T06:03:23.115560+0000","last_became_active":"2023-12-20T06:03:21.966332+0000","last_became_peered":"2023-12-20T06:03:21.966332+0000","last_unstale":"2023-12-20T06:03:23.115560+0000","last_undegraded":"2023-12-20T06:03:23.115560+0000","last_fullsized":"2023-12-20T06:03:23.115560+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:51:08.151013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817031+0000","last_change":"2023-12-20T06:03:21.977328+0000","last_active":"2023-12-20T06:03:24.817031+0000","last_peered":"2023-12-20T06:03:24.817031+0000","last_clean":"2023-12-20T06:03:24.817031+0000","last_became_active":"2023-12-20T06:03:21.976722+0000","last_became_peered":"2023-12-20T06:03:21.976722+0000","last_unstale":"2023-12-20T06:03:24.817031+0000","last_undegraded":"2023-12-20T06:03:24.817031+0000","last_fullsized":"2023-12-20T06:03:24.817031+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:35:37.344273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816401+0000","last_change":"2023-12-20T06:03:21.974137+0000","last_active":"2023-12-20T06:03:24.816401+0000","last_peered":"2023-12-20T06:03:24.816401+0000","last_clean":"2023-12-20T06:03:24.816401+0000","last_became_active":"2023-12-20T06:03:21.973889+0000","last_became_peered":"2023-12-20T06:03:21.973889+0000","last_unstale":"2023-12-20T06:03:24.816401+0000","last_undegraded":"2023-12-20T06:03:24.816401+0000","last_fullsized":"2023-12-20T06:03:24.816401+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:08:46.618627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.961970+0000","last_change":"2023-12-20T06:03:21.967911+0000","last_active":"2023-12-20T06:03:22.961970+0000","last_peered":"2023-12-20T06:03:22.961970+0000","last_clean":"2023-12-20T06:03:22.961970+0000","last_became_active":"2023-12-20T06:03:21.967093+0000","last_became_peered":"2023-12-20T06:03:21.967093+0000","last_unstale":"2023-12-20T06:03:22.961970+0000","last_undegraded":"2023-12-20T06:03:22.961970+0000","last_fullsized":"2023-12-20T06:03:22.961970+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:43:05.571007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816564+0000","last_change":"2023-12-20T06:03:21.974077+0000","last_active":"2023-12-20T06:03:24.816564+0000","last_peered":"2023-12-20T06:03:24.816564+0000","last_clean":"2023-12-20T06:03:24.816564+0000","last_became_active":"2023-12-20T06:03:21.973810+0000","last_became_peered":"2023-12-20T06:03:21.973810+0000","last_unstale":"2023-12-20T06:03:24.816564+0000","last_undegraded":"2023-12-20T06:03:24.816564+0000","last_fullsized":"2023-12-20T06:03:24.816564+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:00.841811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963340+0000","last_change":"2023-12-20T06:03:21.976310+0000","last_active":"2023-12-20T06:03:22.963340+0000","last_peered":"2023-12-20T06:03:22.963340+0000","last_clean":"2023-12-20T06:03:22.963340+0000","last_became_active":"2023-12-20T06:03:21.976042+0000","last_became_peered":"2023-12-20T06:03:21.976042+0000","last_unstale":"2023-12-20T06:03:22.963340+0000","last_undegraded":"2023-12-20T06:03:22.963340+0000","last_fullsized":"2023-12-20T06:03:22.963340+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:46.204136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.959198+0000","last_change":"2023-12-20T06:03:21.981857+0000","last_active":"2023-12-20T06:03:22.959198+0000","last_peered":"2023-12-20T06:03:22.959198+0000","last_clean":"2023-12-20T06:03:22.959198+0000","last_became_active":"2023-12-20T06:03:21.981595+0000","last_became_peered":"2023-12-20T06:03:21.981595+0000","last_unstale":"2023-12-20T06:03:22.959198+0000","last_undegraded":"2023-12-20T06:03:22.959198+0000","last_fullsized":"2023-12-20T06:03:22.959198+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:39:40.018104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962845+0000","last_change":"2023-12-20T06:03:21.979813+0000","last_active":"2023-12-20T06:03:22.962845+0000","last_peered":"2023-12-20T06:03:22.962845+0000","last_clean":"2023-12-20T06:03:22.962845+0000","last_became_active":"2023-12-20T06:03:21.979541+0000","last_became_peered":"2023-12-20T06:03:21.979541+0000","last_unstale":"2023-12-20T06:03:22.962845+0000","last_undegraded":"2023-12-20T06:03:22.962845+0000","last_fullsized":"2023-12-20T06:03:22.962845+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:29.736662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115621+0000","last_change":"2023-12-20T06:03:21.985481+0000","last_active":"2023-12-20T06:03:23.115621+0000","last_peered":"2023-12-20T06:03:23.115621+0000","last_clean":"2023-12-20T06:03:23.115621+0000","last_became_active":"2023-12-20T06:03:21.985202+0000","last_became_peered":"2023-12-20T06:03:21.985202+0000","last_unstale":"2023-12-20T06:03:23.115621+0000","last_undegraded":"2023-12-20T06:03:23.115621+0000","last_fullsized":"2023-12-20T06:03:23.115621+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:57:07.294252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.442846+0000","last_change":"2023-12-20T06:03:21.989526+0000","last_active":"2023-12-20T06:03:23.442846+0000","last_peered":"2023-12-20T06:03:23.442846+0000","last_clean":"2023-12-20T06:03:23.442846+0000","last_became_active":"2023-12-20T06:03:21.989062+0000","last_became_peered":"2023-12-20T06:03:21.989062+0000","last_unstale":"2023-12-20T06:03:23.442846+0000","last_undegraded":"2023-12-20T06:03:23.442846+0000","last_fullsized":"2023-12-20T06:03:23.442846+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:17.456340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962001+0000","last_change":"2023-12-20T06:03:21.978099+0000","last_active":"2023-12-20T06:03:22.962001+0000","last_peered":"2023-12-20T06:03:22.962001+0000","last_clean":"2023-12-20T06:03:22.962001+0000","last_became_active":"2023-12-20T06:03:21.977814+0000","last_became_peered":"2023-12-20T06:03:21.977814+0000","last_unstale":"2023-12-20T06:03:22.962001+0000","last_undegraded":"2023-12-20T06:03:22.962001+0000","last_fullsized":"2023-12-20T06:03:22.962001+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:17.140851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816645+0000","last_change":"2023-12-20T06:03:21.984020+0000","last_active":"2023-12-20T06:03:24.816645+0000","last_peered":"2023-12-20T06:03:24.816645+0000","last_clean":"2023-12-20T06:03:24.816645+0000","last_became_active":"2023-12-20T06:03:21.983429+0000","last_became_peered":"2023-12-20T06:03:21.983429+0000","last_unstale":"2023-12-20T06:03:24.816645+0000","last_undegraded":"2023-12-20T06:03:24.816645+0000","last_fullsized":"2023-12-20T06:03:24.816645+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816686+0000","last_change":"2023-12-20T06:03:21.985095+0000","last_active":"2023-12-20T06:03:24.816686+0000","last_peered":"2023-12-20T06:03:24.816686+0000","last_clean":"2023-12-20T06:03:24.816686+0000","last_became_active":"2023-12-20T06:03:21.984659+0000","last_became_peered":"2023-12-20T06:03:21.984659+0000","last_unstale":"2023-12-20T06:03:24.816686+0000","last_undegraded":"2023-12-20T06:03:24.816686+0000","last_fullsized":"2023-12-20T06:03:24.816686+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:24:17.425367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963294+0000","last_change":"2023-12-20T06:03:21.985587+0000","last_active":"2023-12-20T06:03:22.963294+0000","last_peered":"2023-12-20T06:03:22.963294+0000","last_clean":"2023-12-20T06:03:22.963294+0000","last_became_active":"2023-12-20T06:03:21.985231+0000","last_became_peered":"2023-12-20T06:03:21.985231+0000","last_unstale":"2023-12-20T06:03:22.963294+0000","last_undegraded":"2023-12-20T06:03:22.963294+0000","last_fullsized":"2023-12-20T06:03:22.963294+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:44:13.389812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"229'4","reported_seq":15,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.992951+0000","last_change":"2023-12-20T06:03:21.558246+0000","last_active":"2023-12-20T06:03:21.992951+0000","last_peered":"2023-12-20T06:03:21.992951+0000","last_clean":"2023-12-20T06:03:21.992951+0000","last_became_active":"2023-12-20T06:03:21.557732+0000","last_became_peered":"2023-12-20T06:03:21.557732+0000","last_unstale":"2023-12-20T06:03:21.992951+0000","last_undegraded":"2023-12-20T06:03:21.992951+0000","last_fullsized":"2023-12-20T06:03:21.992951+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:01:07.326834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.959268+0000","last_change":"2023-12-20T06:03:21.975027+0000","last_active":"2023-12-20T06:03:22.959268+0000","last_peered":"2023-12-20T06:03:22.959268+0000","last_clean":"2023-12-20T06:03:22.959268+0000","last_became_active":"2023-12-20T06:03:21.974227+0000","last_became_peered":"2023-12-20T06:03:21.974227+0000","last_unstale":"2023-12-20T06:03:22.959268+0000","last_undegraded":"2023-12-20T06:03:22.959268+0000","last_fullsized":"2023-12-20T06:03:22.959268+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:38:46.179532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"229'1","reported_seq":12,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.992901+0000","last_change":"2023-12-20T06:03:21.559869+0000","last_active":"2023-12-20T06:03:21.992901+0000","last_peered":"2023-12-20T06:03:21.992901+0000","last_clean":"2023-12-20T06:03:21.992901+0000","last_became_active":"2023-12-20T06:03:21.559555+0000","last_became_peered":"2023-12-20T06:03:21.559555+0000","last_unstale":"2023-12-20T06:03:21.992901+0000","last_undegraded":"2023-12-20T06:03:21.992901+0000","last_fullsized":"2023-12-20T06:03:21.992901+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:15.926622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.443063+0000","last_change":"2023-12-20T06:03:21.988797+0000","last_active":"2023-12-20T06:03:23.443063+0000","last_peered":"2023-12-20T06:03:23.443063+0000","last_clean":"2023-12-20T06:03:23.443063+0000","last_became_active":"2023-12-20T06:03:21.988418+0000","last_became_peered":"2023-12-20T06:03:21.988418+0000","last_unstale":"2023-12-20T06:03:23.443063+0000","last_undegraded":"2023-12-20T06:03:23.443063+0000","last_fullsized":"2023-12-20T06:03:23.443063+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:21:31.283475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.992998+0000","last_change":"2023-12-20T06:03:21.566883+0000","last_active":"2023-12-20T06:03:21.992998+0000","last_peered":"2023-12-20T06:03:21.992998+0000","last_clean":"2023-12-20T06:03:21.992998+0000","last_became_active":"2023-12-20T06:03:21.566612+0000","last_became_peered":"2023-12-20T06:03:21.566612+0000","last_unstale":"2023-12-20T06:03:21.992998+0000","last_undegraded":"2023-12-20T06:03:21.992998+0000","last_fullsized":"2023-12-20T06:03:21.992998+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:03:33.646493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962169+0000","last_change":"2023-12-20T06:03:21.976016+0000","last_active":"2023-12-20T06:03:22.962169+0000","last_peered":"2023-12-20T06:03:22.962169+0000","last_clean":"2023-12-20T06:03:22.962169+0000","last_became_active":"2023-12-20T06:03:21.975686+0000","last_became_peered":"2023-12-20T06:03:21.975686+0000","last_unstale":"2023-12-20T06:03:22.962169+0000","last_undegraded":"2023-12-20T06:03:22.962169+0000","last_fullsized":"2023-12-20T06:03:22.962169+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:40:22.788736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"229'1","reported_seq":12,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.980712+0000","last_change":"2023-12-20T06:03:21.567258+0000","last_active":"2023-12-20T06:03:21.980712+0000","last_peered":"2023-12-20T06:03:21.980712+0000","last_clean":"2023-12-20T06:03:21.980712+0000","last_became_active":"2023-12-20T06:03:21.565535+0000","last_became_peered":"2023-12-20T06:03:21.565535+0000","last_unstale":"2023-12-20T06:03:21.980712+0000","last_undegraded":"2023-12-20T06:03:21.980712+0000","last_fullsized":"2023-12-20T06:03:21.980712+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:27:30.578745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817065+0000","last_change":"2023-12-20T06:03:21.988028+0000","last_active":"2023-12-20T06:03:24.817065+0000","last_peered":"2023-12-20T06:03:24.817065+0000","last_clean":"2023-12-20T06:03:24.817065+0000","last_became_active":"2023-12-20T06:03:21.986936+0000","last_became_peered":"2023-12-20T06:03:21.986936+0000","last_unstale":"2023-12-20T06:03:24.817065+0000","last_undegraded":"2023-12-20T06:03:24.817065+0000","last_fullsized":"2023-12-20T06:03:24.817065+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"229'1","reported_seq":12,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.987858+0000","last_change":"2023-12-20T06:03:21.564120+0000","last_active":"2023-12-20T06:03:21.987858+0000","last_peered":"2023-12-20T06:03:21.987858+0000","last_clean":"2023-12-20T06:03:21.987858+0000","last_became_active":"2023-12-20T06:03:21.563826+0000","last_became_peered":"2023-12-20T06:03:21.563826+0000","last_unstale":"2023-12-20T06:03:21.987858+0000","last_undegraded":"2023-12-20T06:03:21.987858+0000","last_fullsized":"2023-12-20T06:03:21.987858+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115549+0000","last_change":"2023-12-20T06:03:21.975377+0000","last_active":"2023-12-20T06:03:23.115549+0000","last_peered":"2023-12-20T06:03:23.115549+0000","last_clean":"2023-12-20T06:03:23.115549+0000","last_became_active":"2023-12-20T06:03:21.974995+0000","last_became_peered":"2023-12-20T06:03:21.974995+0000","last_unstale":"2023-12-20T06:03:23.115549+0000","last_undegraded":"2023-12-20T06:03:23.115549+0000","last_fullsized":"2023-12-20T06:03:23.115549+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:46.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.966769+0000","last_change":"2023-12-20T06:03:21.565400+0000","last_active":"2023-12-20T06:03:21.966769+0000","last_peered":"2023-12-20T06:03:21.966769+0000","last_clean":"2023-12-20T06:03:21.966769+0000","last_became_active":"2023-12-20T06:03:21.565079+0000","last_became_peered":"2023-12-20T06:03:21.565079+0000","last_unstale":"2023-12-20T06:03:21.966769+0000","last_undegraded":"2023-12-20T06:03:21.966769+0000","last_fullsized":"2023-12-20T06:03:21.966769+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:22.788125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089335+0000","last_change":"2023-12-20T06:03:21.977819+0000","last_active":"2023-12-20T06:03:24.089335+0000","last_peered":"2023-12-20T06:03:24.089335+0000","last_clean":"2023-12-20T06:03:24.089335+0000","last_became_active":"2023-12-20T06:03:21.977540+0000","last_became_peered":"2023-12-20T06:03:21.977540+0000","last_unstale":"2023-12-20T06:03:24.089335+0000","last_undegraded":"2023-12-20T06:03:24.089335+0000","last_fullsized":"2023-12-20T06:03:24.089335+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:44:13.389812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.956297+0000","last_change":"2023-12-20T06:03:21.564947+0000","last_active":"2023-12-20T06:03:21.956297+0000","last_peered":"2023-12-20T06:03:21.956297+0000","last_clean":"2023-12-20T06:03:21.956297+0000","last_became_active":"2023-12-20T06:03:21.564634+0000","last_became_peered":"2023-12-20T06:03:21.564634+0000","last_unstale":"2023-12-20T06:03:21.956297+0000","last_undegraded":"2023-12-20T06:03:21.956297+0000","last_fullsized":"2023-12-20T06:03:21.956297+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:05.795673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089641+0000","last_change":"2023-12-20T06:03:21.989153+0000","last_active":"2023-12-20T06:03:24.089641+0000","last_peered":"2023-12-20T06:03:24.089641+0000","last_clean":"2023-12-20T06:03:24.089641+0000","last_became_active":"2023-12-20T06:03:21.988699+0000","last_became_peered":"2023-12-20T06:03:21.988699+0000","last_unstale":"2023-12-20T06:03:24.089641+0000","last_undegraded":"2023-12-20T06:03:24.089641+0000","last_fullsized":"2023-12-20T06:03:24.089641+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:11.133160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.958154+0000","last_change":"2023-12-20T06:03:21.574001+0000","last_active":"2023-12-20T06:03:21.958154+0000","last_peered":"2023-12-20T06:03:21.958154+0000","last_clean":"2023-12-20T06:03:21.958154+0000","last_became_active":"2023-12-20T06:03:21.573735+0000","last_became_peered":"2023-12-20T06:03:21.573735+0000","last_unstale":"2023-12-20T06:03:21.958154+0000","last_undegraded":"2023-12-20T06:03:21.958154+0000","last_fullsized":"2023-12-20T06:03:21.958154+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:13:55.127307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963434+0000","last_change":"2023-12-20T06:03:21.967818+0000","last_active":"2023-12-20T06:03:22.963434+0000","last_peered":"2023-12-20T06:03:22.963434+0000","last_clean":"2023-12-20T06:03:22.963434+0000","last_became_active":"2023-12-20T06:03:21.967516+0000","last_became_peered":"2023-12-20T06:03:21.967516+0000","last_unstale":"2023-12-20T06:03:22.963434+0000","last_undegraded":"2023-12-20T06:03:22.963434+0000","last_fullsized":"2023-12-20T06:03:22.963434+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:07:37.126937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.963740+0000","last_change":"2023-12-20T06:03:21.563619+0000","last_active":"2023-12-20T06:03:21.963740+0000","last_peered":"2023-12-20T06:03:21.963740+0000","last_clean":"2023-12-20T06:03:21.963740+0000","last_became_active":"2023-12-20T06:03:21.563273+0000","last_became_peered":"2023-12-20T06:03:21.563273+0000","last_unstale":"2023-12-20T06:03:21.963740+0000","last_undegraded":"2023-12-20T06:03:21.963740+0000","last_fullsized":"2023-12-20T06:03:21.963740+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:39:30.597039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.443172+0000","last_change":"2023-12-20T06:03:21.990214+0000","last_active":"2023-12-20T06:03:23.443172+0000","last_peered":"2023-12-20T06:03:23.443172+0000","last_clean":"2023-12-20T06:03:23.443172+0000","last_became_active":"2023-12-20T06:03:21.989788+0000","last_became_peered":"2023-12-20T06:03:21.989788+0000","last_unstale":"2023-12-20T06:03:23.443172+0000","last_undegraded":"2023-12-20T06:03:23.443172+0000","last_fullsized":"2023-12-20T06:03:23.443172+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"229'3","reported_seq":14,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.983578+0000","last_change":"2023-12-20T06:03:21.564537+0000","last_active":"2023-12-20T06:03:21.983578+0000","last_peered":"2023-12-20T06:03:21.983578+0000","last_clean":"2023-12-20T06:03:21.983578+0000","last_became_active":"2023-12-20T06:03:21.564167+0000","last_became_peered":"2023-12-20T06:03:21.564167+0000","last_unstale":"2023-12-20T06:03:21.983578+0000","last_undegraded":"2023-12-20T06:03:21.983578+0000","last_fullsized":"2023-12-20T06:03:21.983578+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:41:11.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962735+0000","last_change":"2023-12-20T06:03:21.963443+0000","last_active":"2023-12-20T06:03:22.962735+0000","last_peered":"2023-12-20T06:03:22.962735+0000","last_clean":"2023-12-20T06:03:22.962735+0000","last_became_active":"2023-12-20T06:03:21.962645+0000","last_became_peered":"2023-12-20T06:03:21.962645+0000","last_unstale":"2023-12-20T06:03:22.962735+0000","last_undegraded":"2023-12-20T06:03:22.962735+0000","last_fullsized":"2023-12-20T06:03:22.962735+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:52:50.983047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"229'6","reported_seq":17,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.987803+0000","last_change":"2023-12-20T06:03:21.565407+0000","last_active":"2023-12-20T06:03:21.987803+0000","last_peered":"2023-12-20T06:03:21.987803+0000","last_clean":"2023-12-20T06:03:21.987803+0000","last_became_active":"2023-12-20T06:03:21.564376+0000","last_became_peered":"2023-12-20T06:03:21.564376+0000","last_unstale":"2023-12-20T06:03:21.987803+0000","last_undegraded":"2023-12-20T06:03:21.987803+0000","last_fullsized":"2023-12-20T06:03:21.987803+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:23:07.083767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816533+0000","last_change":"2023-12-20T06:03:21.971942+0000","last_active":"2023-12-20T06:03:24.816533+0000","last_peered":"2023-12-20T06:03:24.816533+0000","last_clean":"2023-12-20T06:03:24.816533+0000","last_became_active":"2023-12-20T06:03:21.971674+0000","last_became_peered":"2023-12-20T06:03:21.971674+0000","last_unstale":"2023-12-20T06:03:24.816533+0000","last_undegraded":"2023-12-20T06:03:24.816533+0000","last_fullsized":"2023-12-20T06:03:24.816533+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:56.167044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.995022+0000","last_change":"2023-12-20T06:03:21.567100+0000","last_active":"2023-12-20T06:03:21.995022+0000","last_peered":"2023-12-20T06:03:21.995022+0000","last_clean":"2023-12-20T06:03:21.995022+0000","last_became_active":"2023-12-20T06:03:21.566805+0000","last_became_peered":"2023-12-20T06:03:21.566805+0000","last_unstale":"2023-12-20T06:03:21.995022+0000","last_undegraded":"2023-12-20T06:03:21.995022+0000","last_fullsized":"2023-12-20T06:03:21.995022+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:45:38.064958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963529+0000","last_change":"2023-12-20T06:03:21.984619+0000","last_active":"2023-12-20T06:03:22.963529+0000","last_peered":"2023-12-20T06:03:22.963529+0000","last_clean":"2023-12-20T06:03:22.963529+0000","last_became_active":"2023-12-20T06:03:21.983940+0000","last_became_peered":"2023-12-20T06:03:21.983940+0000","last_unstale":"2023-12-20T06:03:22.963529+0000","last_undegraded":"2023-12-20T06:03:22.963529+0000","last_fullsized":"2023-12-20T06:03:22.963529+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:13.633088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.963950+0000","last_change":"2023-12-20T06:03:21.561473+0000","last_active":"2023-12-20T06:03:21.963950+0000","last_peered":"2023-12-20T06:03:21.963950+0000","last_clean":"2023-12-20T06:03:21.963950+0000","last_became_active":"2023-12-20T06:03:21.561038+0000","last_became_peered":"2023-12-20T06:03:21.561038+0000","last_unstale":"2023-12-20T06:03:21.963950+0000","last_undegraded":"2023-12-20T06:03:21.963950+0000","last_fullsized":"2023-12-20T06:03:21.963950+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:57:46.598880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962055+0000","last_change":"2023-12-20T06:03:21.987462+0000","last_active":"2023-12-20T06:03:22.962055+0000","last_peered":"2023-12-20T06:03:22.962055+0000","last_clean":"2023-12-20T06:03:22.962055+0000","last_became_active":"2023-12-20T06:03:21.986488+0000","last_became_peered":"2023-12-20T06:03:21.986488+0000","last_unstale":"2023-12-20T06:03:22.962055+0000","last_undegraded":"2023-12-20T06:03:22.962055+0000","last_fullsized":"2023-12-20T06:03:22.962055+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:00.841811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.962000+0000","last_change":"2023-12-20T06:03:21.564490+0000","last_active":"2023-12-20T06:03:21.962000+0000","last_peered":"2023-12-20T06:03:21.962000+0000","last_clean":"2023-12-20T06:03:21.962000+0000","last_became_active":"2023-12-20T06:03:21.562571+0000","last_became_peered":"2023-12-20T06:03:21.562571+0000","last_unstale":"2023-12-20T06:03:21.962000+0000","last_undegraded":"2023-12-20T06:03:21.962000+0000","last_fullsized":"2023-12-20T06:03:21.962000+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.442700+0000","last_change":"2023-12-20T06:03:21.978828+0000","last_active":"2023-12-20T06:03:23.442700+0000","last_peered":"2023-12-20T06:03:23.442700+0000","last_clean":"2023-12-20T06:03:23.442700+0000","last_became_active":"2023-12-20T06:03:21.977595+0000","last_became_peered":"2023-12-20T06:03:21.977595+0000","last_unstale":"2023-12-20T06:03:23.442700+0000","last_undegraded":"2023-12-20T06:03:23.442700+0000","last_fullsized":"2023-12-20T06:03:23.442700+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:53:41.601867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"229'8","reported_seq":19,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.993308+0000","last_change":"2023-12-20T06:03:21.556640+0000","last_active":"2023-12-20T06:03:21.993308+0000","last_peered":"2023-12-20T06:03:21.993308+0000","last_clean":"2023-12-20T06:03:21.993308+0000","last_became_active":"2023-12-20T06:03:21.556362+0000","last_became_peered":"2023-12-20T06:03:21.556362+0000","last_unstale":"2023-12-20T06:03:21.993308+0000","last_undegraded":"2023-12-20T06:03:21.993308+0000","last_fullsized":"2023-12-20T06:03:21.993308+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:16:40.268127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.959238+0000","last_change":"2023-12-20T06:03:21.972553+0000","last_active":"2023-12-20T06:03:22.959238+0000","last_peered":"2023-12-20T06:03:22.959238+0000","last_clean":"2023-12-20T06:03:22.959238+0000","last_became_active":"2023-12-20T06:03:21.972295+0000","last_became_peered":"2023-12-20T06:03:21.972295+0000","last_unstale":"2023-12-20T06:03:22.959238+0000","last_undegraded":"2023-12-20T06:03:22.959238+0000","last_fullsized":"2023-12-20T06:03:22.959238+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:44.890915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"229'1","reported_seq":12,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.987728+0000","last_change":"2023-12-20T06:03:21.563512+0000","last_active":"2023-12-20T06:03:21.987728+0000","last_peered":"2023-12-20T06:03:21.987728+0000","last_clean":"2023-12-20T06:03:21.987728+0000","last_became_active":"2023-12-20T06:03:21.563130+0000","last_became_peered":"2023-12-20T06:03:21.563130+0000","last_unstale":"2023-12-20T06:03:21.987728+0000","last_undegraded":"2023-12-20T06:03:21.987728+0000","last_fullsized":"2023-12-20T06:03:21.987728+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:28.719459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115713+0000","last_change":"2023-12-20T06:03:21.991246+0000","last_active":"2023-12-20T06:03:23.115713+0000","last_peered":"2023-12-20T06:03:23.115713+0000","last_clean":"2023-12-20T06:03:23.115713+0000","last_became_active":"2023-12-20T06:03:21.990892+0000","last_became_peered":"2023-12-20T06:03:21.990892+0000","last_unstale":"2023-12-20T06:03:23.115713+0000","last_undegraded":"2023-12-20T06:03:23.115713+0000","last_fullsized":"2023-12-20T06:03:23.115713+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:52.328250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.980713+0000","last_change":"2023-12-20T06:03:21.576334+0000","last_active":"2023-12-20T06:03:21.980713+0000","last_peered":"2023-12-20T06:03:21.980713+0000","last_clean":"2023-12-20T06:03:21.980713+0000","last_became_active":"2023-12-20T06:03:21.575847+0000","last_became_peered":"2023-12-20T06:03:21.575847+0000","last_unstale":"2023-12-20T06:03:21.980713+0000","last_undegraded":"2023-12-20T06:03:21.980713+0000","last_fullsized":"2023-12-20T06:03:21.980713+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:46:33.911145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963084+0000","last_change":"2023-12-20T06:03:21.988502+0000","last_active":"2023-12-20T06:03:22.963084+0000","last_peered":"2023-12-20T06:03:22.963084+0000","last_clean":"2023-12-20T06:03:22.963084+0000","last_became_active":"2023-12-20T06:03:21.988209+0000","last_became_peered":"2023-12-20T06:03:21.988209+0000","last_unstale":"2023-12-20T06:03:22.963084+0000","last_undegraded":"2023-12-20T06:03:22.963084+0000","last_fullsized":"2023-12-20T06:03:22.963084+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:35:37.344273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"229'1","reported_seq":12,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.979298+0000","last_change":"2023-12-20T06:03:21.574438+0000","last_active":"2023-12-20T06:03:21.979298+0000","last_peered":"2023-12-20T06:03:21.979298+0000","last_clean":"2023-12-20T06:03:21.979298+0000","last_became_active":"2023-12-20T06:03:21.574161+0000","last_became_peered":"2023-12-20T06:03:21.574161+0000","last_unstale":"2023-12-20T06:03:21.979298+0000","last_undegraded":"2023-12-20T06:03:21.979298+0000","last_fullsized":"2023-12-20T06:03:21.979298+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:21:17.124107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963497+0000","last_change":"2023-12-20T06:03:21.985314+0000","last_active":"2023-12-20T06:03:22.963497+0000","last_peered":"2023-12-20T06:03:22.963497+0000","last_clean":"2023-12-20T06:03:22.963497+0000","last_became_active":"2023-12-20T06:03:21.984916+0000","last_became_peered":"2023-12-20T06:03:21.984916+0000","last_unstale":"2023-12-20T06:03:22.963497+0000","last_undegraded":"2023-12-20T06:03:22.963497+0000","last_fullsized":"2023-12-20T06:03:22.963497+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:55:10.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.956791+0000","last_change":"2023-12-20T06:03:21.576277+0000","last_active":"2023-12-20T06:03:21.956791+0000","last_peered":"2023-12-20T06:03:21.956791+0000","last_clean":"2023-12-20T06:03:21.956791+0000","last_became_active":"2023-12-20T06:03:21.575791+0000","last_became_peered":"2023-12-20T06:03:21.575791+0000","last_unstale":"2023-12-20T06:03:21.956791+0000","last_undegraded":"2023-12-20T06:03:21.956791+0000","last_fullsized":"2023-12-20T06:03:21.956791+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:24:57.234123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816899+0000","last_change":"2023-12-20T06:03:21.977046+0000","last_active":"2023-12-20T06:03:24.816899+0000","last_peered":"2023-12-20T06:03:24.816899+0000","last_clean":"2023-12-20T06:03:24.816899+0000","last_became_active":"2023-12-20T06:03:21.976756+0000","last_became_peered":"2023-12-20T06:03:21.976756+0000","last_unstale":"2023-12-20T06:03:24.816899+0000","last_undegraded":"2023-12-20T06:03:24.816899+0000","last_fullsized":"2023-12-20T06:03:24.816899+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:17.456340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.989855+0000","last_change":"2023-12-20T06:03:21.573650+0000","last_active":"2023-12-20T06:03:21.989855+0000","last_peered":"2023-12-20T06:03:21.989855+0000","last_clean":"2023-12-20T06:03:21.989855+0000","last_became_active":"2023-12-20T06:03:21.573349+0000","last_became_peered":"2023-12-20T06:03:21.573349+0000","last_unstale":"2023-12-20T06:03:21.989855+0000","last_undegraded":"2023-12-20T06:03:21.989855+0000","last_fullsized":"2023-12-20T06:03:21.989855+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:22.788125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963018+0000","last_change":"2023-12-20T06:03:21.983774+0000","last_active":"2023-12-20T06:03:22.963018+0000","last_peered":"2023-12-20T06:03:22.963018+0000","last_clean":"2023-12-20T06:03:22.963018+0000","last_became_active":"2023-12-20T06:03:21.983388+0000","last_became_peered":"2023-12-20T06:03:21.983388+0000","last_unstale":"2023-12-20T06:03:22.963018+0000","last_undegraded":"2023-12-20T06:03:22.963018+0000","last_fullsized":"2023-12-20T06:03:22.963018+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:11.133160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.971795+0000","last_change":"2023-12-20T06:03:21.567023+0000","last_active":"2023-12-20T06:03:21.971795+0000","last_peered":"2023-12-20T06:03:21.971795+0000","last_clean":"2023-12-20T06:03:21.971795+0000","last_became_active":"2023-12-20T06:03:21.566708+0000","last_became_peered":"2023-12-20T06:03:21.566708+0000","last_unstale":"2023-12-20T06:03:21.971795+0000","last_undegraded":"2023-12-20T06:03:21.971795+0000","last_fullsized":"2023-12-20T06:03:21.971795+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.958428+0000","last_change":"2023-12-20T06:03:21.972573+0000","last_active":"2023-12-20T06:03:22.958428+0000","last_peered":"2023-12-20T06:03:22.958428+0000","last_clean":"2023-12-20T06:03:22.958428+0000","last_became_active":"2023-12-20T06:03:21.972302+0000","last_became_peered":"2023-12-20T06:03:21.972302+0000","last_unstale":"2023-12-20T06:03:22.958428+0000","last_undegraded":"2023-12-20T06:03:22.958428+0000","last_fullsized":"2023-12-20T06:03:22.958428+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:58:25.448451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.966874+0000","last_change":"2023-12-20T06:03:21.570826+0000","last_active":"2023-12-20T06:03:21.966874+0000","last_peered":"2023-12-20T06:03:21.966874+0000","last_clean":"2023-12-20T06:03:21.966874+0000","last_became_active":"2023-12-20T06:03:21.570512+0000","last_became_peered":"2023-12-20T06:03:21.570512+0000","last_unstale":"2023-12-20T06:03:21.966874+0000","last_undegraded":"2023-12-20T06:03:21.966874+0000","last_fullsized":"2023-12-20T06:03:21.966874+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:52:49.965844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816676+0000","last_change":"2023-12-20T06:03:21.976420+0000","last_active":"2023-12-20T06:03:24.816676+0000","last_peered":"2023-12-20T06:03:24.816676+0000","last_clean":"2023-12-20T06:03:24.816676+0000","last_became_active":"2023-12-20T06:03:21.976141+0000","last_became_peered":"2023-12-20T06:03:21.976141+0000","last_unstale":"2023-12-20T06:03:24.816676+0000","last_undegraded":"2023-12-20T06:03:24.816676+0000","last_fullsized":"2023-12-20T06:03:24.816676+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:27:59.998522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.971971+0000","last_change":"2023-12-20T06:03:21.567122+0000","last_active":"2023-12-20T06:03:21.971971+0000","last_peered":"2023-12-20T06:03:21.971971+0000","last_clean":"2023-12-20T06:03:21.971971+0000","last_became_active":"2023-12-20T06:03:21.566816+0000","last_became_peered":"2023-12-20T06:03:21.566816+0000","last_unstale":"2023-12-20T06:03:21.971971+0000","last_undegraded":"2023-12-20T06:03:21.971971+0000","last_fullsized":"2023-12-20T06:03:21.971971+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:41:11.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817051+0000","last_change":"2023-12-20T06:03:21.975113+0000","last_active":"2023-12-20T06:03:24.817051+0000","last_peered":"2023-12-20T06:03:24.817051+0000","last_clean":"2023-12-20T06:03:24.817051+0000","last_became_active":"2023-12-20T06:03:21.974862+0000","last_became_peered":"2023-12-20T06:03:21.974862+0000","last_unstale":"2023-12-20T06:03:24.817051+0000","last_undegraded":"2023-12-20T06:03:24.817051+0000","last_fullsized":"2023-12-20T06:03:24.817051+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:07:37.126937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.990432+0000","last_change":"2023-12-20T06:03:21.569876+0000","last_active":"2023-12-20T06:03:21.990432+0000","last_peered":"2023-12-20T06:03:21.990432+0000","last_clean":"2023-12-20T06:03:21.990432+0000","last_became_active":"2023-12-20T06:03:21.568286+0000","last_became_peered":"2023-12-20T06:03:21.568286+0000","last_unstale":"2023-12-20T06:03:21.990432+0000","last_undegraded":"2023-12-20T06:03:21.990432+0000","last_fullsized":"2023-12-20T06:03:21.990432+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963768+0000","last_change":"2023-12-20T06:03:21.991437+0000","last_active":"2023-12-20T06:03:22.963768+0000","last_peered":"2023-12-20T06:03:22.963768+0000","last_clean":"2023-12-20T06:03:22.963768+0000","last_became_active":"2023-12-20T06:03:21.991098+0000","last_became_peered":"2023-12-20T06:03:21.991098+0000","last_unstale":"2023-12-20T06:03:22.963768+0000","last_undegraded":"2023-12-20T06:03:22.963768+0000","last_fullsized":"2023-12-20T06:03:22.963768+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:48.782916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.956825+0000","last_change":"2023-12-20T06:03:21.574738+0000","last_active":"2023-12-20T06:03:21.956825+0000","last_peered":"2023-12-20T06:03:21.956825+0000","last_clean":"2023-12-20T06:03:21.956825+0000","last_became_active":"2023-12-20T06:03:21.574282+0000","last_became_peered":"2023-12-20T06:03:21.574282+0000","last_unstale":"2023-12-20T06:03:21.956825+0000","last_undegraded":"2023-12-20T06:03:21.956825+0000","last_fullsized":"2023-12-20T06:03:21.956825+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:43:56.675318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963216+0000","last_change":"2023-12-20T06:03:21.985967+0000","last_active":"2023-12-20T06:03:22.963216+0000","last_peered":"2023-12-20T06:03:22.963216+0000","last_clean":"2023-12-20T06:03:22.963216+0000","last_became_active":"2023-12-20T06:03:21.985460+0000","last_became_peered":"2023-12-20T06:03:21.985460+0000","last_unstale":"2023-12-20T06:03:22.963216+0000","last_undegraded":"2023-12-20T06:03:22.963216+0000","last_fullsized":"2023-12-20T06:03:22.963216+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:06:16.446428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.958446+0000","last_change":"2023-12-20T06:03:21.573978+0000","last_active":"2023-12-20T06:03:21.958446+0000","last_peered":"2023-12-20T06:03:21.958446+0000","last_clean":"2023-12-20T06:03:21.958446+0000","last_became_active":"2023-12-20T06:03:21.573682+0000","last_became_peered":"2023-12-20T06:03:21.573682+0000","last_unstale":"2023-12-20T06:03:21.958446+0000","last_undegraded":"2023-12-20T06:03:21.958446+0000","last_fullsized":"2023-12-20T06:03:21.958446+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:43.873712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817240+0000","last_change":"2023-12-20T06:03:21.980139+0000","last_active":"2023-12-20T06:03:24.817240+0000","last_peered":"2023-12-20T06:03:24.817240+0000","last_clean":"2023-12-20T06:03:24.817240+0000","last_became_active":"2023-12-20T06:03:21.979858+0000","last_became_peered":"2023-12-20T06:03:21.979858+0000","last_unstale":"2023-12-20T06:03:24.817240+0000","last_undegraded":"2023-12-20T06:03:24.817240+0000","last_fullsized":"2023-12-20T06:03:24.817240+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:21:31.283475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.973764+0000","last_change":"2023-12-20T06:03:21.570642+0000","last_active":"2023-12-20T06:03:21.973764+0000","last_peered":"2023-12-20T06:03:21.973764+0000","last_clean":"2023-12-20T06:03:21.973764+0000","last_became_active":"2023-12-20T06:03:21.569631+0000","last_became_peered":"2023-12-20T06:03:21.569631+0000","last_unstale":"2023-12-20T06:03:21.973764+0000","last_undegraded":"2023-12-20T06:03:21.973764+0000","last_fullsized":"2023-12-20T06:03:21.973764+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089448+0000","last_change":"2023-12-20T06:03:21.963940+0000","last_active":"2023-12-20T06:03:24.089448+0000","last_peered":"2023-12-20T06:03:24.089448+0000","last_clean":"2023-12-20T06:03:24.089448+0000","last_became_active":"2023-12-20T06:03:21.963679+0000","last_became_peered":"2023-12-20T06:03:21.963679+0000","last_unstale":"2023-12-20T06:03:24.089448+0000","last_undegraded":"2023-12-20T06:03:24.089448+0000","last_fullsized":"2023-12-20T06:03:24.089448+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.971959+0000","last_change":"2023-12-20T06:03:21.573888+0000","last_active":"2023-12-20T06:03:21.971959+0000","last_peered":"2023-12-20T06:03:21.971959+0000","last_clean":"2023-12-20T06:03:21.971959+0000","last_became_active":"2023-12-20T06:03:21.572823+0000","last_became_peered":"2023-12-20T06:03:21.572823+0000","last_unstale":"2023-12-20T06:03:21.971959+0000","last_undegraded":"2023-12-20T06:03:21.971959+0000","last_fullsized":"2023-12-20T06:03:21.971959+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963147+0000","last_change":"2023-12-20T06:03:21.983107+0000","last_active":"2023-12-20T06:03:22.963147+0000","last_peered":"2023-12-20T06:03:22.963147+0000","last_clean":"2023-12-20T06:03:22.963147+0000","last_became_active":"2023-12-20T06:03:21.981255+0000","last_became_peered":"2023-12-20T06:03:21.981255+0000","last_unstale":"2023-12-20T06:03:22.963147+0000","last_undegraded":"2023-12-20T06:03:22.963147+0000","last_fullsized":"2023-12-20T06:03:22.963147+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.963949+0000","last_change":"2023-12-20T06:03:21.558503+0000","last_active":"2023-12-20T06:03:21.963949+0000","last_peered":"2023-12-20T06:03:21.963949+0000","last_clean":"2023-12-20T06:03:21.963949+0000","last_became_active":"2023-12-20T06:03:21.557644+0000","last_became_peered":"2023-12-20T06:03:21.557644+0000","last_unstale":"2023-12-20T06:03:21.963949+0000","last_undegraded":"2023-12-20T06:03:21.963949+0000","last_fullsized":"2023-12-20T06:03:21.963949+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:48.619779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962106+0000","last_change":"2023-12-20T06:03:21.986446+0000","last_active":"2023-12-20T06:03:22.962106+0000","last_peered":"2023-12-20T06:03:22.962106+0000","last_clean":"2023-12-20T06:03:22.962106+0000","last_became_active":"2023-12-20T06:03:21.986120+0000","last_became_peered":"2023-12-20T06:03:21.986120+0000","last_unstale":"2023-12-20T06:03:22.962106+0000","last_undegraded":"2023-12-20T06:03:22.962106+0000","last_fullsized":"2023-12-20T06:03:22.962106+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:27:59.998522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.962095+0000","last_change":"2023-12-20T06:03:21.570337+0000","last_active":"2023-12-20T06:03:21.962095+0000","last_peered":"2023-12-20T06:03:21.962095+0000","last_clean":"2023-12-20T06:03:21.962095+0000","last_became_active":"2023-12-20T06:03:21.569986+0000","last_became_peered":"2023-12-20T06:03:21.569986+0000","last_unstale":"2023-12-20T06:03:21.962095+0000","last_undegraded":"2023-12-20T06:03:21.962095+0000","last_fullsized":"2023-12-20T06:03:21.962095+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:28.719459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962082+0000","last_change":"2023-12-20T06:03:21.968100+0000","last_active":"2023-12-20T06:03:22.962082+0000","last_peered":"2023-12-20T06:03:22.962082+0000","last_clean":"2023-12-20T06:03:22.962082+0000","last_became_active":"2023-12-20T06:03:21.967593+0000","last_became_peered":"2023-12-20T06:03:21.967593+0000","last_unstale":"2023-12-20T06:03:22.962082+0000","last_undegraded":"2023-12-20T06:03:22.962082+0000","last_fullsized":"2023-12-20T06:03:22.962082+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:00:01.717868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.971646+0000","last_change":"2023-12-20T06:03:21.568281+0000","last_active":"2023-12-20T06:03:21.971646+0000","last_peered":"2023-12-20T06:03:21.971646+0000","last_clean":"2023-12-20T06:03:21.971646+0000","last_became_active":"2023-12-20T06:03:21.567976+0000","last_became_peered":"2023-12-20T06:03:21.567976+0000","last_unstale":"2023-12-20T06:03:21.971646+0000","last_undegraded":"2023-12-20T06:03:21.971646+0000","last_fullsized":"2023-12-20T06:03:21.971646+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:03:18.004400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962433+0000","last_change":"2023-12-20T06:03:21.973527+0000","last_active":"2023-12-20T06:03:22.962433+0000","last_peered":"2023-12-20T06:03:22.962433+0000","last_clean":"2023-12-20T06:03:22.962433+0000","last_became_active":"2023-12-20T06:03:21.973178+0000","last_became_peered":"2023-12-20T06:03:21.973178+0000","last_unstale":"2023-12-20T06:03:22.962433+0000","last_undegraded":"2023-12-20T06:03:22.962433+0000","last_fullsized":"2023-12-20T06:03:22.962433+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:55:50.794593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.991846+0000","last_change":"2023-12-20T06:03:21.567717+0000","last_active":"2023-12-20T06:03:21.991846+0000","last_peered":"2023-12-20T06:03:21.991846+0000","last_clean":"2023-12-20T06:03:21.991846+0000","last_became_active":"2023-12-20T06:03:21.567425+0000","last_became_peered":"2023-12-20T06:03:21.567425+0000","last_unstale":"2023-12-20T06:03:21.991846+0000","last_undegraded":"2023-12-20T06:03:21.991846+0000","last_fullsized":"2023-12-20T06:03:21.991846+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:45.186933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"215'3223","reported_seq":4089,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.956202+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:03:21.956202+0000","last_peered":"2023-12-20T06:03:21.956202+0000","last_clean":"2023-12-20T06:03:21.956202+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:03:21.956202+0000","last_undegraded":"2023-12-20T06:03:21.956202+0000","last_fullsized":"2023-12-20T06:03:21.956202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3223,"log_dups_size":0,"ondisk_log_size":3223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":561,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":561,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3223,"ondisk_log_size":3223,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575980,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60696,"kb_used_data":3408,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311144,"statfs":{"total":96636764160,"available":96574611456,"internally_reserved":0,"allocated":3489792,"data_stored":2002471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.69599999999999995,"15min":0.72799999999999998},"min":{"1min":0.45400000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":1.028,"15min":1.042},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.99099999999999999,"15min":1.161},"last":0.871}]},{"osd":1,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71499999999999997,"15min":0.74199999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":0.996,"15min":1.1279999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.71599999999999997,"15min":0.745},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.85499999999999998,"5min":0.96699999999999997,"15min":1.032},"last":0.68600000000000005}]},{"osd":2,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.70599999999999996,"15min":0.73899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84499999999999997,"5min":0.95399999999999996,"15min":1.014},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.49099999999999999},"max":{"1min":0.88700000000000001,"5min":1.0089999999999999,"15min":1.0189999999999999},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.47699999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.85299999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.89900000000000002,"5min":1.0660000000000001,"15min":1.0840000000000001},"last":0.66500000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.04},"last":0.749},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.02,"15min":1.02},"last":0.70399999999999996}]},{"osd":5,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68300000000000005,"15min":0.66800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.052},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.875,"5min":1.0229999999999999,"15min":1.103},"last":0.82599999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76300000000000001,"15min":0.746},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72399999999999998,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93500000000000005,"5min":1.0740000000000001,"15min":1.081},"last":0.57999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575980,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51928,"kb_used_data":2832,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319912,"statfs":{"total":96636764160,"available":96583589888,"internally_reserved":0,"allocated":2899968,"data_stored":1412089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71199999999999997,"15min":0.73999999999999999},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92400000000000004,"5min":1.016,"15min":1.0980000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72999999999999998,"15min":0.75},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.006,"5min":1.016,"15min":1.05},"last":0.80200000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.47399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90100000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91200000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.78600000000000003}]},{"osd":2,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76900000000000002,"15min":0.78100000000000003},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97699999999999998,"5min":1.069,"15min":1.069},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.54400000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.93100000000000005,"5min":0.97799999999999998,"15min":1.0760000000000001},"last":0.69099999999999995}]},{"osd":3,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78900000000000003},"min":{"1min":0.61299999999999999,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.0109999999999999,"5min":1.077,"15min":1.077},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.997,"5min":1.052,"15min":1.052},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.36099999999999999},"max":{"1min":0.96299999999999997,"5min":1.05,"15min":1.0529999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66400000000000003,"15min":0.65100000000000002},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64900000000000002,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.72499999999999998}]},{"osd":7,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.55700000000000005,"5min":0.48899999999999999,"15min":0.45300000000000001},"max":{"1min":1.0429999999999999,"5min":1.103,"15min":1.1240000000000001},"last":0.749},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.76700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575981,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51920,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319920,"statfs":{"total":96636764160,"available":96583598080,"internally_reserved":0,"allocated":2891776,"data_stored":1412033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71399999999999997,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91600000000000004,"5min":0.94799999999999995,"15min":1.077},"last":0.62},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71499999999999997,"15min":0.746},"min":{"1min":0.45600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.80300000000000005,"5min":0.96699999999999997,"15min":1.028},"last":0.80300000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":0.875,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88500000000000001,"5min":0.98799999999999999,"15min":1.0209999999999999},"last":0.75}]},{"osd":2,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.751},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89000000000000001,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.75700000000000001,"15min":0.77000000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.48299999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0680000000000001},"last":0.92700000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"max":{"1min":0.91000000000000003,"5min":1.05,"15min":1.0669999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.0629999999999999},"last":0.78600000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.56000000000000005,"5min":0.42199999999999999,"15min":0.38200000000000001},"max":{"1min":0.94399999999999995,"5min":1.0640000000000001,"15min":1.077},"last":0.873},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.64800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"max":{"1min":1.0649999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.0449999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.72199999999999998}]},{"osd":7,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73799999999999999,"15min":0.70999999999999996},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.153,"15min":1.153},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.48299999999999998},"max":{"1min":0.98799999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.94299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575982,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47828,"kb_used_data":2828,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324012,"statfs":{"total":96636764160,"available":96587788288,"internally_reserved":0,"allocated":2895872,"data_stored":1412615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.69299999999999995,"15min":0.72899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96499999999999997,"5min":0.98999999999999999,"15min":1.0209999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.69299999999999995,"15min":0.72299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.80900000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73099999999999998,"15min":0.752},"min":{"1min":0.498,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":0.89600000000000002,"5min":0.95699999999999996,"15min":1.0900000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73899999999999999,"15min":0.76400000000000001},"min":{"1min":0.47899999999999998,"5min":0.47399999999999998,"15min":0.44500000000000001},"max":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79000000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.746,"15min":0.76900000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98199999999999998,"5min":1.0089999999999999,"15min":1.101},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.75},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49199999999999999},"max":{"1min":0.88,"5min":1.014,"15min":1.014},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.80100000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1,"5min":1.04,"15min":1.081},"last":0.90600000000000003}]},{"osd":5,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64700000000000002,"15min":0.65000000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98099999999999998,"5min":1.05,"15min":1.05},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.47599999999999998,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.96999999999999997,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.1379999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.41199999999999998},"max":{"1min":1.056,"5min":1.107,"15min":1.111},"last":0.94399999999999995}]},{"osd":7,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72899999999999998,"15min":0.72999999999999998},"min":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":1.0329999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.626,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.1100000000000001,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.69199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575981,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47840,"kb_used_data":2840,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324000,"statfs":{"total":96636764160,"available":96587776000,"internally_reserved":0,"allocated":2908160,"data_stored":1413667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.59299999999999997,"5min":0.748,"15min":2.8130000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.51100000000000001,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64600000000000002,"5min":0.80000000000000004,"15min":1.357},"last":0.47999999999999998}]},{"osd":1,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":1.4219999999999999},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.34200000000000003},"max":{"1min":0.72599999999999998,"5min":0.86099999999999999,"15min":2.944},"last":0.56200000000000006}]},{"osd":4,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":1.0209999999999999,"5min":1.2090000000000001,"15min":1.669},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.99},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.59599999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97199999999999998,"5min":1.355,"15min":1.901},"last":0.871},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.82399999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97699999999999998,"5min":1.196,"15min":2.8639999999999999},"last":0.82299999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.81999999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.575},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.81799999999999995,"15min":0.85099999999999998},"min":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0149999999999999,"5min":1.2350000000000001,"15min":1.948},"last":0.92400000000000004}]},{"osd":7,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80400000000000005,"15min":0.84099999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.004,"5min":1.3149999999999999,"15min":2.9049999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82399999999999995,"15min":0.85299999999999998},"min":{"1min":0.61899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.089,"5min":1.28,"15min":1.754},"last":0.94199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51940,"kb_used_data":2844,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319900,"statfs":{"total":96636764160,"available":96583577600,"internally_reserved":0,"allocated":2912256,"data_stored":1413653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.45000000000000001,"15min":0.45500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52900000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.60699999999999998,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.79800000000000004,"15min":0.82199999999999995},"last":0.374},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30499999999999999},"max":{"1min":0.61199999999999999,"5min":0.92600000000000005,"15min":1.004},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.65500000000000003,"15min":0.67400000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.59199999999999997,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.55800000000000005}]},{"osd":4,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":1.254},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77900000000000003,"15min":0.79600000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":1.605,"15min":1.605},"last":0.753}]},{"osd":5,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97099999999999997,"5min":1.329,"15min":1.329},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98199999999999998,"5min":1.0740000000000001,"15min":1.095},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92200000000000004,"5min":1.919,"15min":1.919},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.95099999999999996,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0049999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.88200000000000001,"15min":0.88500000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.996,"5min":1.292,"15min":1.292},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60104,"kb_used_data":2816,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311736,"statfs":{"total":96636764160,"available":96575217664,"internally_reserved":0,"allocated":2883584,"data_stored":1405767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47899999999999998,"15min":0.46200000000000002},"min":{"1min":0.26500000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59299999999999997,"5min":0.746,"15min":0.81000000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.29399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54700000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.71999999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":0.88800000000000001,"15min":0.93700000000000006},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.42999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.90100000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.93500000000000005},"last":0.48399999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.78500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92000000000000004,"5min":1.107,"15min":1.321},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.995,"5min":1.139,"15min":1.25},"last":0.77500000000000002}]},{"osd":5,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78700000000000003,"15min":0.80200000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.97899999999999998,"5min":1.224,"15min":1.2849999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0740000000000001,"5min":1.1990000000000001,"15min":1.4330000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.82399999999999995},"min":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.2430000000000001,"15min":1.341},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.56399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1379999999999999,"5min":1.177,"15min":1.3839999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0900000000000001,"5min":1.272,"15min":1.4470000000000001},"last":0.621},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0509999999999999,"5min":1.1599999999999999,"15min":1.3839999999999999},"last":0.71299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575981,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56604,"kb_used_data":3412,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315236,"statfs":{"total":96636764160,"available":96578801664,"internally_reserved":0,"allocated":3493888,"data_stored":2003073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33000000000000002},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.35699999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.40999999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82999999999999996,"15min":0.82599999999999996},"min":{"1min":0.56699999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.86099999999999999,"15min":0.85099999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.83399999999999996,"15min":0.84799999999999998},"min":{"1min":0.54600000000000004,"5min":0.495,"15min":0.495},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.63200000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.89100000000000001,"15min":0.871},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57099999999999995},"max":{"1min":5.6310000000000002,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.61799999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":5.7160000000000002,"5min":5.7160000000000002,"15min":5.7160000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.012,"5min":0.88200000000000001,"15min":0.871},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":4.2119999999999997,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:03:26.090 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:03:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.271+0000 7fb530fbd640 1 Processor -- start 2023-12-20T06:03:26.272 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.271+0000 7fb530fbd640 1 -- start start 2023-12-20T06:03:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb530fbd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb52c160cb0 0x7fb52c0cb4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb530fbd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb52c0cba10 0x7fb52c0cd220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb530fbd640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb52c161290 con 0x7fb52c06c8f0 2023-12-20T06:03:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb530fbd640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb52c161570 con 0x7fb52c160cb0 2023-12-20T06:03:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb530fbd640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c161850 con 0x7fb52c0cba10 2023-12-20T06:03:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb52affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb52c160cb0 0x7fb52c0cb4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb52affd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb52c160cb0 0x7fb52c0cb4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33432/0 (socket says 172.21.15.32:33432) 2023-12-20T06:03:26.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb52affd640 1 -- 172.21.15.32:0/688809911 learned_addr learned my addr 172.21.15.32:0/688809911 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:26.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb52bfff640 1 --2- 172.21.15.32:0/688809911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb52c0cba10 0x7fb52c0cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb52affd640 1 -- 172.21.15.32:0/688809911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb52c0cba10 msgr2=0x7fb52c0cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:26.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb52affd640 1 --2- 172.21.15.32:0/688809911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb52c0cba10 0x7fb52c0cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb52affd640 1 -- 172.21.15.32:0/688809911 >> v1:172.21.15.39:6789/0 conn(0x7fb52c06c8f0 legacy=0x7fb52c15e130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:26.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.275+0000 7fb52affd640 1 -- 172.21.15.32:0/688809911 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb52c0ce9d0 con 0x7fb52c160cb0 2023-12-20T06:03:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb52affd640 1 --2- 172.21.15.32:0/688809911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb52c160cb0 0x7fb52c0cb4d0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fb520009870 tx=0x7fb52001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=22e19e0d0918fdd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb52a7fc640 1 -- 172.21.15.32:0/688809911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5200ab600 con 0x7fb52c160cb0 2023-12-20T06:03:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb52a7fc640 1 -- 172.21.15.32:0/688809911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb5200a7050 con 0x7fb52c160cb0 2023-12-20T06:03:26.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb52a7fc640 1 -- 172.21.15.32:0/688809911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5200ab980 con 0x7fb52c160cb0 2023-12-20T06:03:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 -- 172.21.15.32:0/688809911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb52c160cb0 msgr2=0x7fb52c0cb4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 --2- 172.21.15.32:0/688809911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb52c160cb0 0x7fb52c0cb4d0 secure :-1 s=READY pgs=2251 cs=0 l=1 rev1=1 crypto rx=0x7fb520009870 tx=0x7fb52001c500 comp rx=0 tx=0).stop 2023-12-20T06:03:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 -- 172.21.15.32:0/688809911 shutdown_connections 2023-12-20T06:03:26.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 --2- 172.21.15.32:0/688809911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb52c0cba10 0x7fb52c0cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 --2- 172.21.15.32:0/688809911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb52c160cb0 0x7fb52c0cb4d0 unknown :-1 s=CLOSED pgs=2251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 -- 172.21.15.32:0/688809911 >> 172.21.15.32:0/688809911 conn(0x7fb52c06e280 msgr2=0x7fb52c06e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:26.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 -- 172.21.15.32:0/688809911 shutdown_connections 2023-12-20T06:03:26.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 -- 172.21.15.32:0/688809911 wait complete. 2023-12-20T06:03:26.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 Processor -- start 2023-12-20T06:03:26.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 -- start start 2023-12-20T06:03:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb52c06c8f0 0x7fb52c1531a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb52c0cba10 0x7fb52c0b5d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb52c160cb0 0x7fb52c0b3d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0c6b70 con 0x7fb52c160cb0 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c0c6e10 con 0x7fb52c06c8f0 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.279+0000 7fb530fbd640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb52c15ad70 con 0x7fb52c0cba10 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb52c160cb0 0x7fb52c0b3d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb52affd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb52c0cba10 0x7fb52c0b5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb52bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb52c160cb0 0x7fb52c0b3d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33442/0 (socket says 172.21.15.32:33442) 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb52bfff640 1 -- 172.21.15.32:0/426608864 learned_addr learned my addr 172.21.15.32:0/426608864 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb52b7fe640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb52c06c8f0 0x7fb52c1531a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb52bfff640 1 -- 172.21.15.32:0/426608864 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb52c0cba10 msgr2=0x7fb52c0b5d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb52bfff640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb52c0cba10 0x7fb52c0b5d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb52bfff640 1 -- 172.21.15.32:0/426608864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb52c06c8f0 msgr2=0x7fb52c1531a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb52bfff640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb52c06c8f0 0x7fb52c1531a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb52bfff640 1 -- 172.21.15.32:0/426608864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb5200094d0 con 0x7fb52c160cb0 2023-12-20T06:03:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb52bfff640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb52c160cb0 0x7fb52c0b3d80 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fb524007e30 tx=0x7fb52400e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb513fff640 1 -- 172.21.15.32:0/426608864 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb524014620 con 0x7fb52c160cb0 2023-12-20T06:03:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb513fff640 1 -- 172.21.15.32:0/426608864 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb52400ee90 con 0x7fb52c160cb0 2023-12-20T06:03:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb530fbd640 1 -- 172.21.15.32:0/426608864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb52c15b050 con 0x7fb52c160cb0 2023-12-20T06:03:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb513fff640 1 -- 172.21.15.32:0/426608864 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb5240149c0 con 0x7fb52c160cb0 2023-12-20T06:03:26.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb530fbd640 1 -- 172.21.15.32:0/426608864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb52c0c70f0 con 0x7fb52c160cb0 2023-12-20T06:03:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb530fbd640 1 -- 172.21.15.32:0/426608864 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb52c15b290 con 0x7fb52c160cb0 2023-12-20T06:03:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.283+0000 7fb513fff640 1 -- 172.21.15.32:0/426608864 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb524014b20 con 0x7fb52c160cb0 2023-12-20T06:03:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.287+0000 7fb513fff640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb518065ca0 0x7fb518068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.287+0000 7fb513fff640 1 -- 172.21.15.32:0/426608864 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb524083dd0 con 0x7fb52c160cb0 2023-12-20T06:03:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.287+0000 7fb52b7fe640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb518065ca0 0x7fb518068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.287+0000 7fb52b7fe640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb518065ca0 0x7fb518068160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb51c007fd0 tx=0x7fb51c007e40 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:26.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.291+0000 7fb513fff640 1 -- 172.21.15.32:0/426608864 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb5240514d0 con 0x7fb52c160cb0 2023-12-20T06:03:26.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.443+0000 7fb530fbd640 1 -- 172.21.15.32:0/426608864 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fb52c0c6f80 con 0x7fb518065ca0 2023-12-20T06:03:26.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.451+0000 7fb513fff640 1 -- 172.21.15.32:0/426608864 <== mgr.10956 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289552 (secure 0 0 0) 0x7fb52c0c6f80 con 0x7fb518065ca0 2023-12-20T06:03:26.453 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:26.455 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:03:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 -- 172.21.15.32:0/426608864 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb518065ca0 msgr2=0x7fb518068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb518065ca0 0x7fb518068160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fb51c007fd0 tx=0x7fb51c007e40 comp rx=0 tx=0).stop 2023-12-20T06:03:26.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 -- 172.21.15.32:0/426608864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb52c160cb0 msgr2=0x7fb52c0b3d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb52c160cb0 0x7fb52c0b3d80 secure :-1 s=READY pgs=2252 cs=0 l=1 rev1=1 crypto rx=0x7fb524007e30 tx=0x7fb52400e4b0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 -- 172.21.15.32:0/426608864 shutdown_connections 2023-12-20T06:03:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb518065ca0 0x7fb518068160 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb52c160cb0 0x7fb52c0b3d80 unknown :-1 s=CLOSED pgs=2252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb52c0cba10 0x7fb52c0b5d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 --2- 172.21.15.32:0/426608864 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb52c06c8f0 0x7fb52c1531a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 -- 172.21.15.32:0/426608864 >> 172.21.15.32:0/426608864 conn(0x7fb52c06e280 msgr2=0x7fb52c0cc7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 -- 172.21.15.32:0/426608864 shutdown_connections 2023-12-20T06:03:26.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.455+0000 7fb530fbd640 1 -- 172.21.15.32:0/426608864 wait complete. 2023-12-20T06:03:26.474 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-12-20T06:03:25.658460+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":605,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3261,"ondisk_log_size":3261,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":194,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":428860,"kb_used_data":23804,"kb_used_omap":0,"kb_used_meta":404992,"kb_avail":754545860,"statfs":{"total":773094113280,"available":772654960640,"internally_reserved":0,"allocated":24375296,"data_stored":12475368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":414711808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":35,"apply_latency_ms":35,"commit_latency_ns":35000000,"apply_latency_ns":35000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.136924"},"pg_stats":[{"pgid":"29.3f","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963240+0000","last_change":"2023-12-20T06:03:21.976746+0000","last_active":"2023-12-20T06:03:22.963240+0000","last_peered":"2023-12-20T06:03:22.963240+0000","last_clean":"2023-12-20T06:03:22.963240+0000","last_became_active":"2023-12-20T06:03:21.976467+0000","last_became_peered":"2023-12-20T06:03:21.976467+0000","last_unstale":"2023-12-20T06:03:22.963240+0000","last_undegraded":"2023-12-20T06:03:22.963240+0000","last_fullsized":"2023-12-20T06:03:22.963240+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:13.213910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089596+0000","last_change":"2023-12-20T06:03:21.988965+0000","last_active":"2023-12-20T06:03:24.089596+0000","last_peered":"2023-12-20T06:03:24.089596+0000","last_clean":"2023-12-20T06:03:24.089596+0000","last_became_active":"2023-12-20T06:03:21.988334+0000","last_became_peered":"2023-12-20T06:03:21.988334+0000","last_unstale":"2023-12-20T06:03:24.089596+0000","last_undegraded":"2023-12-20T06:03:24.089596+0000","last_fullsized":"2023-12-20T06:03:24.089596+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:04.642017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963412+0000","last_change":"2023-12-20T06:03:21.987434+0000","last_active":"2023-12-20T06:03:22.963412+0000","last_peered":"2023-12-20T06:03:22.963412+0000","last_clean":"2023-12-20T06:03:22.963412+0000","last_became_active":"2023-12-20T06:03:21.987118+0000","last_became_peered":"2023-12-20T06:03:21.987118+0000","last_unstale":"2023-12-20T06:03:22.963412+0000","last_undegraded":"2023-12-20T06:03:22.963412+0000","last_fullsized":"2023-12-20T06:03:22.963412+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817040+0000","last_change":"2023-12-20T06:03:21.981328+0000","last_active":"2023-12-20T06:03:24.817040+0000","last_peered":"2023-12-20T06:03:24.817040+0000","last_clean":"2023-12-20T06:03:24.817040+0000","last_became_active":"2023-12-20T06:03:21.981001+0000","last_became_peered":"2023-12-20T06:03:21.981001+0000","last_unstale":"2023-12-20T06:03:24.817040+0000","last_undegraded":"2023-12-20T06:03:24.817040+0000","last_fullsized":"2023-12-20T06:03:24.817040+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:48.782916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816928+0000","last_change":"2023-12-20T06:03:21.978878+0000","last_active":"2023-12-20T06:03:24.816928+0000","last_peered":"2023-12-20T06:03:24.816928+0000","last_clean":"2023-12-20T06:03:24.816928+0000","last_became_active":"2023-12-20T06:03:21.977017+0000","last_became_peered":"2023-12-20T06:03:21.977017+0000","last_unstale":"2023-12-20T06:03:24.816928+0000","last_undegraded":"2023-12-20T06:03:24.816928+0000","last_fullsized":"2023-12-20T06:03:24.816928+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:55:10.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115451+0000","last_change":"2023-12-20T06:03:21.976982+0000","last_active":"2023-12-20T06:03:23.115451+0000","last_peered":"2023-12-20T06:03:23.115451+0000","last_clean":"2023-12-20T06:03:23.115451+0000","last_became_active":"2023-12-20T06:03:21.976721+0000","last_became_peered":"2023-12-20T06:03:21.976721+0000","last_unstale":"2023-12-20T06:03:23.115451+0000","last_undegraded":"2023-12-20T06:03:23.115451+0000","last_fullsized":"2023-12-20T06:03:23.115451+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:57.966779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962469+0000","last_change":"2023-12-20T06:03:21.979840+0000","last_active":"2023-12-20T06:03:22.962469+0000","last_peered":"2023-12-20T06:03:22.962469+0000","last_clean":"2023-12-20T06:03:22.962469+0000","last_became_active":"2023-12-20T06:03:21.979570+0000","last_became_peered":"2023-12-20T06:03:21.979570+0000","last_unstale":"2023-12-20T06:03:22.962469+0000","last_undegraded":"2023-12-20T06:03:22.962469+0000","last_fullsized":"2023-12-20T06:03:22.962469+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.958428+0000","last_change":"2023-12-20T06:03:21.976244+0000","last_active":"2023-12-20T06:03:22.958428+0000","last_peered":"2023-12-20T06:03:22.958428+0000","last_clean":"2023-12-20T06:03:22.958428+0000","last_became_active":"2023-12-20T06:03:21.975968+0000","last_became_peered":"2023-12-20T06:03:21.975968+0000","last_unstale":"2023-12-20T06:03:22.958428+0000","last_undegraded":"2023-12-20T06:03:22.958428+0000","last_fullsized":"2023-12-20T06:03:22.958428+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:07:21.324505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963729+0000","last_change":"2023-12-20T06:03:21.987598+0000","last_active":"2023-12-20T06:03:22.963729+0000","last_peered":"2023-12-20T06:03:22.963729+0000","last_clean":"2023-12-20T06:03:22.963729+0000","last_became_active":"2023-12-20T06:03:21.987291+0000","last_became_peered":"2023-12-20T06:03:21.987291+0000","last_unstale":"2023-12-20T06:03:22.963729+0000","last_undegraded":"2023-12-20T06:03:22.963729+0000","last_fullsized":"2023-12-20T06:03:22.963729+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:04.642017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115682+0000","last_change":"2023-12-20T06:03:21.969809+0000","last_active":"2023-12-20T06:03:23.115682+0000","last_peered":"2023-12-20T06:03:23.115682+0000","last_clean":"2023-12-20T06:03:23.115682+0000","last_became_active":"2023-12-20T06:03:21.969515+0000","last_became_peered":"2023-12-20T06:03:21.969515+0000","last_unstale":"2023-12-20T06:03:23.115682+0000","last_undegraded":"2023-12-20T06:03:23.115682+0000","last_fullsized":"2023-12-20T06:03:23.115682+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:15:47.777614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962623+0000","last_change":"2023-12-20T06:03:21.979757+0000","last_active":"2023-12-20T06:03:22.962623+0000","last_peered":"2023-12-20T06:03:22.962623+0000","last_clean":"2023-12-20T06:03:22.962623+0000","last_became_active":"2023-12-20T06:03:21.979456+0000","last_became_peered":"2023-12-20T06:03:21.979456+0000","last_unstale":"2023-12-20T06:03:22.962623+0000","last_undegraded":"2023-12-20T06:03:22.962623+0000","last_fullsized":"2023-12-20T06:03:22.962623+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:23.805328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089484+0000","last_change":"2023-12-20T06:03:21.982252+0000","last_active":"2023-12-20T06:03:24.089484+0000","last_peered":"2023-12-20T06:03:24.089484+0000","last_clean":"2023-12-20T06:03:24.089484+0000","last_became_active":"2023-12-20T06:03:21.981980+0000","last_became_peered":"2023-12-20T06:03:21.981980+0000","last_unstale":"2023-12-20T06:03:24.089484+0000","last_undegraded":"2023-12-20T06:03:24.089484+0000","last_fullsized":"2023-12-20T06:03:24.089484+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:13.633088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089323+0000","last_change":"2023-12-20T06:03:21.975662+0000","last_active":"2023-12-20T06:03:24.089323+0000","last_peered":"2023-12-20T06:03:24.089323+0000","last_clean":"2023-12-20T06:03:24.089323+0000","last_became_active":"2023-12-20T06:03:21.975017+0000","last_became_peered":"2023-12-20T06:03:21.975017+0000","last_unstale":"2023-12-20T06:03:24.089323+0000","last_undegraded":"2023-12-20T06:03:24.089323+0000","last_fullsized":"2023-12-20T06:03:24.089323+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:06:16.446428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.442758+0000","last_change":"2023-12-20T06:03:21.973459+0000","last_active":"2023-12-20T06:03:23.442758+0000","last_peered":"2023-12-20T06:03:23.442758+0000","last_clean":"2023-12-20T06:03:23.442758+0000","last_became_active":"2023-12-20T06:03:21.973101+0000","last_became_peered":"2023-12-20T06:03:21.973101+0000","last_unstale":"2023-12-20T06:03:23.442758+0000","last_undegraded":"2023-12-20T06:03:23.442758+0000","last_fullsized":"2023-12-20T06:03:23.442758+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:03:34.663696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816300+0000","last_change":"2023-12-20T06:03:21.982673+0000","last_active":"2023-12-20T06:03:24.816300+0000","last_peered":"2023-12-20T06:03:24.816300+0000","last_clean":"2023-12-20T06:03:24.816300+0000","last_became_active":"2023-12-20T06:03:21.982010+0000","last_became_peered":"2023-12-20T06:03:21.982010+0000","last_unstale":"2023-12-20T06:03:24.816300+0000","last_undegraded":"2023-12-20T06:03:24.816300+0000","last_fullsized":"2023-12-20T06:03:24.816300+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:32:19.664417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089253+0000","last_change":"2023-12-20T06:03:21.977347+0000","last_active":"2023-12-20T06:03:24.089253+0000","last_peered":"2023-12-20T06:03:24.089253+0000","last_clean":"2023-12-20T06:03:24.089253+0000","last_became_active":"2023-12-20T06:03:21.977040+0000","last_became_peered":"2023-12-20T06:03:21.977040+0000","last_unstale":"2023-12-20T06:03:24.089253+0000","last_undegraded":"2023-12-20T06:03:24.089253+0000","last_fullsized":"2023-12-20T06:03:24.089253+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.443179+0000","last_change":"2023-12-20T06:03:21.973052+0000","last_active":"2023-12-20T06:03:23.443179+0000","last_peered":"2023-12-20T06:03:23.443179+0000","last_clean":"2023-12-20T06:03:23.443179+0000","last_became_active":"2023-12-20T06:03:21.972760+0000","last_became_peered":"2023-12-20T06:03:21.972760+0000","last_unstale":"2023-12-20T06:03:23.443179+0000","last_undegraded":"2023-12-20T06:03:23.443179+0000","last_fullsized":"2023-12-20T06:03:23.443179+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:16:41.285330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115729+0000","last_change":"2023-12-20T06:03:21.991211+0000","last_active":"2023-12-20T06:03:23.115729+0000","last_peered":"2023-12-20T06:03:23.115729+0000","last_clean":"2023-12-20T06:03:23.115729+0000","last_became_active":"2023-12-20T06:03:21.990873+0000","last_became_peered":"2023-12-20T06:03:21.990873+0000","last_unstale":"2023-12-20T06:03:23.115729+0000","last_undegraded":"2023-12-20T06:03:23.115729+0000","last_fullsized":"2023-12-20T06:03:23.115729+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:31:08.457333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817251+0000","last_change":"2023-12-20T06:03:21.982366+0000","last_active":"2023-12-20T06:03:24.817251+0000","last_peered":"2023-12-20T06:03:24.817251+0000","last_clean":"2023-12-20T06:03:24.817251+0000","last_became_active":"2023-12-20T06:03:21.978098+0000","last_became_peered":"2023-12-20T06:03:21.978098+0000","last_unstale":"2023-12-20T06:03:24.817251+0000","last_undegraded":"2023-12-20T06:03:24.817251+0000","last_fullsized":"2023-12-20T06:03:24.817251+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115560+0000","last_change":"2023-12-20T06:03:21.967196+0000","last_active":"2023-12-20T06:03:23.115560+0000","last_peered":"2023-12-20T06:03:23.115560+0000","last_clean":"2023-12-20T06:03:23.115560+0000","last_became_active":"2023-12-20T06:03:21.966332+0000","last_became_peered":"2023-12-20T06:03:21.966332+0000","last_unstale":"2023-12-20T06:03:23.115560+0000","last_undegraded":"2023-12-20T06:03:23.115560+0000","last_fullsized":"2023-12-20T06:03:23.115560+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:51:08.151013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817031+0000","last_change":"2023-12-20T06:03:21.977328+0000","last_active":"2023-12-20T06:03:24.817031+0000","last_peered":"2023-12-20T06:03:24.817031+0000","last_clean":"2023-12-20T06:03:24.817031+0000","last_became_active":"2023-12-20T06:03:21.976722+0000","last_became_peered":"2023-12-20T06:03:21.976722+0000","last_unstale":"2023-12-20T06:03:24.817031+0000","last_undegraded":"2023-12-20T06:03:24.817031+0000","last_fullsized":"2023-12-20T06:03:24.817031+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:35:37.344273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816401+0000","last_change":"2023-12-20T06:03:21.974137+0000","last_active":"2023-12-20T06:03:24.816401+0000","last_peered":"2023-12-20T06:03:24.816401+0000","last_clean":"2023-12-20T06:03:24.816401+0000","last_became_active":"2023-12-20T06:03:21.973889+0000","last_became_peered":"2023-12-20T06:03:21.973889+0000","last_unstale":"2023-12-20T06:03:24.816401+0000","last_undegraded":"2023-12-20T06:03:24.816401+0000","last_fullsized":"2023-12-20T06:03:24.816401+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:08:46.618627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.961970+0000","last_change":"2023-12-20T06:03:21.967911+0000","last_active":"2023-12-20T06:03:22.961970+0000","last_peered":"2023-12-20T06:03:22.961970+0000","last_clean":"2023-12-20T06:03:22.961970+0000","last_became_active":"2023-12-20T06:03:21.967093+0000","last_became_peered":"2023-12-20T06:03:21.967093+0000","last_unstale":"2023-12-20T06:03:22.961970+0000","last_undegraded":"2023-12-20T06:03:22.961970+0000","last_fullsized":"2023-12-20T06:03:22.961970+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:43:05.571007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816564+0000","last_change":"2023-12-20T06:03:21.974077+0000","last_active":"2023-12-20T06:03:24.816564+0000","last_peered":"2023-12-20T06:03:24.816564+0000","last_clean":"2023-12-20T06:03:24.816564+0000","last_became_active":"2023-12-20T06:03:21.973810+0000","last_became_peered":"2023-12-20T06:03:21.973810+0000","last_unstale":"2023-12-20T06:03:24.816564+0000","last_undegraded":"2023-12-20T06:03:24.816564+0000","last_fullsized":"2023-12-20T06:03:24.816564+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:00.841811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963340+0000","last_change":"2023-12-20T06:03:21.976310+0000","last_active":"2023-12-20T06:03:22.963340+0000","last_peered":"2023-12-20T06:03:22.963340+0000","last_clean":"2023-12-20T06:03:22.963340+0000","last_became_active":"2023-12-20T06:03:21.976042+0000","last_became_peered":"2023-12-20T06:03:21.976042+0000","last_unstale":"2023-12-20T06:03:22.963340+0000","last_undegraded":"2023-12-20T06:03:22.963340+0000","last_fullsized":"2023-12-20T06:03:22.963340+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:46.204136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.959198+0000","last_change":"2023-12-20T06:03:21.981857+0000","last_active":"2023-12-20T06:03:22.959198+0000","last_peered":"2023-12-20T06:03:22.959198+0000","last_clean":"2023-12-20T06:03:22.959198+0000","last_became_active":"2023-12-20T06:03:21.981595+0000","last_became_peered":"2023-12-20T06:03:21.981595+0000","last_unstale":"2023-12-20T06:03:22.959198+0000","last_undegraded":"2023-12-20T06:03:22.959198+0000","last_fullsized":"2023-12-20T06:03:22.959198+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:39:40.018104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962845+0000","last_change":"2023-12-20T06:03:21.979813+0000","last_active":"2023-12-20T06:03:22.962845+0000","last_peered":"2023-12-20T06:03:22.962845+0000","last_clean":"2023-12-20T06:03:22.962845+0000","last_became_active":"2023-12-20T06:03:21.979541+0000","last_became_peered":"2023-12-20T06:03:21.979541+0000","last_unstale":"2023-12-20T06:03:22.962845+0000","last_undegraded":"2023-12-20T06:03:22.962845+0000","last_fullsized":"2023-12-20T06:03:22.962845+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:29.736662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115621+0000","last_change":"2023-12-20T06:03:21.985481+0000","last_active":"2023-12-20T06:03:23.115621+0000","last_peered":"2023-12-20T06:03:23.115621+0000","last_clean":"2023-12-20T06:03:23.115621+0000","last_became_active":"2023-12-20T06:03:21.985202+0000","last_became_peered":"2023-12-20T06:03:21.985202+0000","last_unstale":"2023-12-20T06:03:23.115621+0000","last_undegraded":"2023-12-20T06:03:23.115621+0000","last_fullsized":"2023-12-20T06:03:23.115621+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:57:07.294252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.442846+0000","last_change":"2023-12-20T06:03:21.989526+0000","last_active":"2023-12-20T06:03:23.442846+0000","last_peered":"2023-12-20T06:03:23.442846+0000","last_clean":"2023-12-20T06:03:23.442846+0000","last_became_active":"2023-12-20T06:03:21.989062+0000","last_became_peered":"2023-12-20T06:03:21.989062+0000","last_unstale":"2023-12-20T06:03:23.442846+0000","last_undegraded":"2023-12-20T06:03:23.442846+0000","last_fullsized":"2023-12-20T06:03:23.442846+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:17.456340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962001+0000","last_change":"2023-12-20T06:03:21.978099+0000","last_active":"2023-12-20T06:03:22.962001+0000","last_peered":"2023-12-20T06:03:22.962001+0000","last_clean":"2023-12-20T06:03:22.962001+0000","last_became_active":"2023-12-20T06:03:21.977814+0000","last_became_peered":"2023-12-20T06:03:21.977814+0000","last_unstale":"2023-12-20T06:03:22.962001+0000","last_undegraded":"2023-12-20T06:03:22.962001+0000","last_fullsized":"2023-12-20T06:03:22.962001+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:17.140851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816645+0000","last_change":"2023-12-20T06:03:21.984020+0000","last_active":"2023-12-20T06:03:24.816645+0000","last_peered":"2023-12-20T06:03:24.816645+0000","last_clean":"2023-12-20T06:03:24.816645+0000","last_became_active":"2023-12-20T06:03:21.983429+0000","last_became_peered":"2023-12-20T06:03:21.983429+0000","last_unstale":"2023-12-20T06:03:24.816645+0000","last_undegraded":"2023-12-20T06:03:24.816645+0000","last_fullsized":"2023-12-20T06:03:24.816645+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816686+0000","last_change":"2023-12-20T06:03:21.985095+0000","last_active":"2023-12-20T06:03:24.816686+0000","last_peered":"2023-12-20T06:03:24.816686+0000","last_clean":"2023-12-20T06:03:24.816686+0000","last_became_active":"2023-12-20T06:03:21.984659+0000","last_became_peered":"2023-12-20T06:03:21.984659+0000","last_unstale":"2023-12-20T06:03:24.816686+0000","last_undegraded":"2023-12-20T06:03:24.816686+0000","last_fullsized":"2023-12-20T06:03:24.816686+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:24:17.425367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963294+0000","last_change":"2023-12-20T06:03:21.985587+0000","last_active":"2023-12-20T06:03:22.963294+0000","last_peered":"2023-12-20T06:03:22.963294+0000","last_clean":"2023-12-20T06:03:22.963294+0000","last_became_active":"2023-12-20T06:03:21.985231+0000","last_became_peered":"2023-12-20T06:03:21.985231+0000","last_unstale":"2023-12-20T06:03:22.963294+0000","last_undegraded":"2023-12-20T06:03:22.963294+0000","last_fullsized":"2023-12-20T06:03:22.963294+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:44:13.389812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"229'4","reported_seq":15,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.992951+0000","last_change":"2023-12-20T06:03:21.558246+0000","last_active":"2023-12-20T06:03:21.992951+0000","last_peered":"2023-12-20T06:03:21.992951+0000","last_clean":"2023-12-20T06:03:21.992951+0000","last_became_active":"2023-12-20T06:03:21.557732+0000","last_became_peered":"2023-12-20T06:03:21.557732+0000","last_unstale":"2023-12-20T06:03:21.992951+0000","last_undegraded":"2023-12-20T06:03:21.992951+0000","last_fullsized":"2023-12-20T06:03:21.992951+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:01:07.326834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.959268+0000","last_change":"2023-12-20T06:03:21.975027+0000","last_active":"2023-12-20T06:03:22.959268+0000","last_peered":"2023-12-20T06:03:22.959268+0000","last_clean":"2023-12-20T06:03:22.959268+0000","last_became_active":"2023-12-20T06:03:21.974227+0000","last_became_peered":"2023-12-20T06:03:21.974227+0000","last_unstale":"2023-12-20T06:03:22.959268+0000","last_undegraded":"2023-12-20T06:03:22.959268+0000","last_fullsized":"2023-12-20T06:03:22.959268+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:38:46.179532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"229'1","reported_seq":12,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.992901+0000","last_change":"2023-12-20T06:03:21.559869+0000","last_active":"2023-12-20T06:03:21.992901+0000","last_peered":"2023-12-20T06:03:21.992901+0000","last_clean":"2023-12-20T06:03:21.992901+0000","last_became_active":"2023-12-20T06:03:21.559555+0000","last_became_peered":"2023-12-20T06:03:21.559555+0000","last_unstale":"2023-12-20T06:03:21.992901+0000","last_undegraded":"2023-12-20T06:03:21.992901+0000","last_fullsized":"2023-12-20T06:03:21.992901+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:15.926622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.443063+0000","last_change":"2023-12-20T06:03:21.988797+0000","last_active":"2023-12-20T06:03:23.443063+0000","last_peered":"2023-12-20T06:03:23.443063+0000","last_clean":"2023-12-20T06:03:23.443063+0000","last_became_active":"2023-12-20T06:03:21.988418+0000","last_became_peered":"2023-12-20T06:03:21.988418+0000","last_unstale":"2023-12-20T06:03:23.443063+0000","last_undegraded":"2023-12-20T06:03:23.443063+0000","last_fullsized":"2023-12-20T06:03:23.443063+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:21:31.283475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.992998+0000","last_change":"2023-12-20T06:03:21.566883+0000","last_active":"2023-12-20T06:03:21.992998+0000","last_peered":"2023-12-20T06:03:21.992998+0000","last_clean":"2023-12-20T06:03:21.992998+0000","last_became_active":"2023-12-20T06:03:21.566612+0000","last_became_peered":"2023-12-20T06:03:21.566612+0000","last_unstale":"2023-12-20T06:03:21.992998+0000","last_undegraded":"2023-12-20T06:03:21.992998+0000","last_fullsized":"2023-12-20T06:03:21.992998+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:03:33.646493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962169+0000","last_change":"2023-12-20T06:03:21.976016+0000","last_active":"2023-12-20T06:03:22.962169+0000","last_peered":"2023-12-20T06:03:22.962169+0000","last_clean":"2023-12-20T06:03:22.962169+0000","last_became_active":"2023-12-20T06:03:21.975686+0000","last_became_peered":"2023-12-20T06:03:21.975686+0000","last_unstale":"2023-12-20T06:03:22.962169+0000","last_undegraded":"2023-12-20T06:03:22.962169+0000","last_fullsized":"2023-12-20T06:03:22.962169+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:40:22.788736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"229'1","reported_seq":12,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.980712+0000","last_change":"2023-12-20T06:03:21.567258+0000","last_active":"2023-12-20T06:03:21.980712+0000","last_peered":"2023-12-20T06:03:21.980712+0000","last_clean":"2023-12-20T06:03:21.980712+0000","last_became_active":"2023-12-20T06:03:21.565535+0000","last_became_peered":"2023-12-20T06:03:21.565535+0000","last_unstale":"2023-12-20T06:03:21.980712+0000","last_undegraded":"2023-12-20T06:03:21.980712+0000","last_fullsized":"2023-12-20T06:03:21.980712+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:27:30.578745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817065+0000","last_change":"2023-12-20T06:03:21.988028+0000","last_active":"2023-12-20T06:03:24.817065+0000","last_peered":"2023-12-20T06:03:24.817065+0000","last_clean":"2023-12-20T06:03:24.817065+0000","last_became_active":"2023-12-20T06:03:21.986936+0000","last_became_peered":"2023-12-20T06:03:21.986936+0000","last_unstale":"2023-12-20T06:03:24.817065+0000","last_undegraded":"2023-12-20T06:03:24.817065+0000","last_fullsized":"2023-12-20T06:03:24.817065+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"229'1","reported_seq":12,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.987858+0000","last_change":"2023-12-20T06:03:21.564120+0000","last_active":"2023-12-20T06:03:21.987858+0000","last_peered":"2023-12-20T06:03:21.987858+0000","last_clean":"2023-12-20T06:03:21.987858+0000","last_became_active":"2023-12-20T06:03:21.563826+0000","last_became_peered":"2023-12-20T06:03:21.563826+0000","last_unstale":"2023-12-20T06:03:21.987858+0000","last_undegraded":"2023-12-20T06:03:21.987858+0000","last_fullsized":"2023-12-20T06:03:21.987858+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115549+0000","last_change":"2023-12-20T06:03:21.975377+0000","last_active":"2023-12-20T06:03:23.115549+0000","last_peered":"2023-12-20T06:03:23.115549+0000","last_clean":"2023-12-20T06:03:23.115549+0000","last_became_active":"2023-12-20T06:03:21.974995+0000","last_became_peered":"2023-12-20T06:03:21.974995+0000","last_unstale":"2023-12-20T06:03:23.115549+0000","last_undegraded":"2023-12-20T06:03:23.115549+0000","last_fullsized":"2023-12-20T06:03:23.115549+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:46.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.966769+0000","last_change":"2023-12-20T06:03:21.565400+0000","last_active":"2023-12-20T06:03:21.966769+0000","last_peered":"2023-12-20T06:03:21.966769+0000","last_clean":"2023-12-20T06:03:21.966769+0000","last_became_active":"2023-12-20T06:03:21.565079+0000","last_became_peered":"2023-12-20T06:03:21.565079+0000","last_unstale":"2023-12-20T06:03:21.966769+0000","last_undegraded":"2023-12-20T06:03:21.966769+0000","last_fullsized":"2023-12-20T06:03:21.966769+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:22.788125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089335+0000","last_change":"2023-12-20T06:03:21.977819+0000","last_active":"2023-12-20T06:03:24.089335+0000","last_peered":"2023-12-20T06:03:24.089335+0000","last_clean":"2023-12-20T06:03:24.089335+0000","last_became_active":"2023-12-20T06:03:21.977540+0000","last_became_peered":"2023-12-20T06:03:21.977540+0000","last_unstale":"2023-12-20T06:03:24.089335+0000","last_undegraded":"2023-12-20T06:03:24.089335+0000","last_fullsized":"2023-12-20T06:03:24.089335+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:44:13.389812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.956297+0000","last_change":"2023-12-20T06:03:21.564947+0000","last_active":"2023-12-20T06:03:21.956297+0000","last_peered":"2023-12-20T06:03:21.956297+0000","last_clean":"2023-12-20T06:03:21.956297+0000","last_became_active":"2023-12-20T06:03:21.564634+0000","last_became_peered":"2023-12-20T06:03:21.564634+0000","last_unstale":"2023-12-20T06:03:21.956297+0000","last_undegraded":"2023-12-20T06:03:21.956297+0000","last_fullsized":"2023-12-20T06:03:21.956297+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:05.795673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089641+0000","last_change":"2023-12-20T06:03:21.989153+0000","last_active":"2023-12-20T06:03:24.089641+0000","last_peered":"2023-12-20T06:03:24.089641+0000","last_clean":"2023-12-20T06:03:24.089641+0000","last_became_active":"2023-12-20T06:03:21.988699+0000","last_became_peered":"2023-12-20T06:03:21.988699+0000","last_unstale":"2023-12-20T06:03:24.089641+0000","last_undegraded":"2023-12-20T06:03:24.089641+0000","last_fullsized":"2023-12-20T06:03:24.089641+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:11.133160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.958154+0000","last_change":"2023-12-20T06:03:21.574001+0000","last_active":"2023-12-20T06:03:21.958154+0000","last_peered":"2023-12-20T06:03:21.958154+0000","last_clean":"2023-12-20T06:03:21.958154+0000","last_became_active":"2023-12-20T06:03:21.573735+0000","last_became_peered":"2023-12-20T06:03:21.573735+0000","last_unstale":"2023-12-20T06:03:21.958154+0000","last_undegraded":"2023-12-20T06:03:21.958154+0000","last_fullsized":"2023-12-20T06:03:21.958154+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:13:55.127307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963434+0000","last_change":"2023-12-20T06:03:21.967818+0000","last_active":"2023-12-20T06:03:22.963434+0000","last_peered":"2023-12-20T06:03:22.963434+0000","last_clean":"2023-12-20T06:03:22.963434+0000","last_became_active":"2023-12-20T06:03:21.967516+0000","last_became_peered":"2023-12-20T06:03:21.967516+0000","last_unstale":"2023-12-20T06:03:22.963434+0000","last_undegraded":"2023-12-20T06:03:22.963434+0000","last_fullsized":"2023-12-20T06:03:22.963434+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:07:37.126937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.963740+0000","last_change":"2023-12-20T06:03:21.563619+0000","last_active":"2023-12-20T06:03:21.963740+0000","last_peered":"2023-12-20T06:03:21.963740+0000","last_clean":"2023-12-20T06:03:21.963740+0000","last_became_active":"2023-12-20T06:03:21.563273+0000","last_became_peered":"2023-12-20T06:03:21.563273+0000","last_unstale":"2023-12-20T06:03:21.963740+0000","last_undegraded":"2023-12-20T06:03:21.963740+0000","last_fullsized":"2023-12-20T06:03:21.963740+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:39:30.597039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.443172+0000","last_change":"2023-12-20T06:03:21.990214+0000","last_active":"2023-12-20T06:03:23.443172+0000","last_peered":"2023-12-20T06:03:23.443172+0000","last_clean":"2023-12-20T06:03:23.443172+0000","last_became_active":"2023-12-20T06:03:21.989788+0000","last_became_peered":"2023-12-20T06:03:21.989788+0000","last_unstale":"2023-12-20T06:03:23.443172+0000","last_undegraded":"2023-12-20T06:03:23.443172+0000","last_fullsized":"2023-12-20T06:03:23.443172+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"229'3","reported_seq":14,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.983578+0000","last_change":"2023-12-20T06:03:21.564537+0000","last_active":"2023-12-20T06:03:21.983578+0000","last_peered":"2023-12-20T06:03:21.983578+0000","last_clean":"2023-12-20T06:03:21.983578+0000","last_became_active":"2023-12-20T06:03:21.564167+0000","last_became_peered":"2023-12-20T06:03:21.564167+0000","last_unstale":"2023-12-20T06:03:21.983578+0000","last_undegraded":"2023-12-20T06:03:21.983578+0000","last_fullsized":"2023-12-20T06:03:21.983578+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:41:11.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962735+0000","last_change":"2023-12-20T06:03:21.963443+0000","last_active":"2023-12-20T06:03:22.962735+0000","last_peered":"2023-12-20T06:03:22.962735+0000","last_clean":"2023-12-20T06:03:22.962735+0000","last_became_active":"2023-12-20T06:03:21.962645+0000","last_became_peered":"2023-12-20T06:03:21.962645+0000","last_unstale":"2023-12-20T06:03:22.962735+0000","last_undegraded":"2023-12-20T06:03:22.962735+0000","last_fullsized":"2023-12-20T06:03:22.962735+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:52:50.983047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"229'6","reported_seq":17,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.987803+0000","last_change":"2023-12-20T06:03:21.565407+0000","last_active":"2023-12-20T06:03:21.987803+0000","last_peered":"2023-12-20T06:03:21.987803+0000","last_clean":"2023-12-20T06:03:21.987803+0000","last_became_active":"2023-12-20T06:03:21.564376+0000","last_became_peered":"2023-12-20T06:03:21.564376+0000","last_unstale":"2023-12-20T06:03:21.987803+0000","last_undegraded":"2023-12-20T06:03:21.987803+0000","last_fullsized":"2023-12-20T06:03:21.987803+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:23:07.083767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816533+0000","last_change":"2023-12-20T06:03:21.971942+0000","last_active":"2023-12-20T06:03:24.816533+0000","last_peered":"2023-12-20T06:03:24.816533+0000","last_clean":"2023-12-20T06:03:24.816533+0000","last_became_active":"2023-12-20T06:03:21.971674+0000","last_became_peered":"2023-12-20T06:03:21.971674+0000","last_unstale":"2023-12-20T06:03:24.816533+0000","last_undegraded":"2023-12-20T06:03:24.816533+0000","last_fullsized":"2023-12-20T06:03:24.816533+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:56.167044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.995022+0000","last_change":"2023-12-20T06:03:21.567100+0000","last_active":"2023-12-20T06:03:21.995022+0000","last_peered":"2023-12-20T06:03:21.995022+0000","last_clean":"2023-12-20T06:03:21.995022+0000","last_became_active":"2023-12-20T06:03:21.566805+0000","last_became_peered":"2023-12-20T06:03:21.566805+0000","last_unstale":"2023-12-20T06:03:21.995022+0000","last_undegraded":"2023-12-20T06:03:21.995022+0000","last_fullsized":"2023-12-20T06:03:21.995022+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:45:38.064958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963529+0000","last_change":"2023-12-20T06:03:21.984619+0000","last_active":"2023-12-20T06:03:22.963529+0000","last_peered":"2023-12-20T06:03:22.963529+0000","last_clean":"2023-12-20T06:03:22.963529+0000","last_became_active":"2023-12-20T06:03:21.983940+0000","last_became_peered":"2023-12-20T06:03:21.983940+0000","last_unstale":"2023-12-20T06:03:22.963529+0000","last_undegraded":"2023-12-20T06:03:22.963529+0000","last_fullsized":"2023-12-20T06:03:22.963529+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:13.633088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.963950+0000","last_change":"2023-12-20T06:03:21.561473+0000","last_active":"2023-12-20T06:03:21.963950+0000","last_peered":"2023-12-20T06:03:21.963950+0000","last_clean":"2023-12-20T06:03:21.963950+0000","last_became_active":"2023-12-20T06:03:21.561038+0000","last_became_peered":"2023-12-20T06:03:21.561038+0000","last_unstale":"2023-12-20T06:03:21.963950+0000","last_undegraded":"2023-12-20T06:03:21.963950+0000","last_fullsized":"2023-12-20T06:03:21.963950+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:57:46.598880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962055+0000","last_change":"2023-12-20T06:03:21.987462+0000","last_active":"2023-12-20T06:03:22.962055+0000","last_peered":"2023-12-20T06:03:22.962055+0000","last_clean":"2023-12-20T06:03:22.962055+0000","last_became_active":"2023-12-20T06:03:21.986488+0000","last_became_peered":"2023-12-20T06:03:21.986488+0000","last_unstale":"2023-12-20T06:03:22.962055+0000","last_undegraded":"2023-12-20T06:03:22.962055+0000","last_fullsized":"2023-12-20T06:03:22.962055+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:00.841811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.962000+0000","last_change":"2023-12-20T06:03:21.564490+0000","last_active":"2023-12-20T06:03:21.962000+0000","last_peered":"2023-12-20T06:03:21.962000+0000","last_clean":"2023-12-20T06:03:21.962000+0000","last_became_active":"2023-12-20T06:03:21.562571+0000","last_became_peered":"2023-12-20T06:03:21.562571+0000","last_unstale":"2023-12-20T06:03:21.962000+0000","last_undegraded":"2023-12-20T06:03:21.962000+0000","last_fullsized":"2023-12-20T06:03:21.962000+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.442700+0000","last_change":"2023-12-20T06:03:21.978828+0000","last_active":"2023-12-20T06:03:23.442700+0000","last_peered":"2023-12-20T06:03:23.442700+0000","last_clean":"2023-12-20T06:03:23.442700+0000","last_became_active":"2023-12-20T06:03:21.977595+0000","last_became_peered":"2023-12-20T06:03:21.977595+0000","last_unstale":"2023-12-20T06:03:23.442700+0000","last_undegraded":"2023-12-20T06:03:23.442700+0000","last_fullsized":"2023-12-20T06:03:23.442700+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:53:41.601867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"229'8","reported_seq":19,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.993308+0000","last_change":"2023-12-20T06:03:21.556640+0000","last_active":"2023-12-20T06:03:21.993308+0000","last_peered":"2023-12-20T06:03:21.993308+0000","last_clean":"2023-12-20T06:03:21.993308+0000","last_became_active":"2023-12-20T06:03:21.556362+0000","last_became_peered":"2023-12-20T06:03:21.556362+0000","last_unstale":"2023-12-20T06:03:21.993308+0000","last_undegraded":"2023-12-20T06:03:21.993308+0000","last_fullsized":"2023-12-20T06:03:21.993308+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:16:40.268127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.959238+0000","last_change":"2023-12-20T06:03:21.972553+0000","last_active":"2023-12-20T06:03:22.959238+0000","last_peered":"2023-12-20T06:03:22.959238+0000","last_clean":"2023-12-20T06:03:22.959238+0000","last_became_active":"2023-12-20T06:03:21.972295+0000","last_became_peered":"2023-12-20T06:03:21.972295+0000","last_unstale":"2023-12-20T06:03:22.959238+0000","last_undegraded":"2023-12-20T06:03:22.959238+0000","last_fullsized":"2023-12-20T06:03:22.959238+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:44.890915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"229'1","reported_seq":12,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.987728+0000","last_change":"2023-12-20T06:03:21.563512+0000","last_active":"2023-12-20T06:03:21.987728+0000","last_peered":"2023-12-20T06:03:21.987728+0000","last_clean":"2023-12-20T06:03:21.987728+0000","last_became_active":"2023-12-20T06:03:21.563130+0000","last_became_peered":"2023-12-20T06:03:21.563130+0000","last_unstale":"2023-12-20T06:03:21.987728+0000","last_undegraded":"2023-12-20T06:03:21.987728+0000","last_fullsized":"2023-12-20T06:03:21.987728+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:28.719459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:23.115713+0000","last_change":"2023-12-20T06:03:21.991246+0000","last_active":"2023-12-20T06:03:23.115713+0000","last_peered":"2023-12-20T06:03:23.115713+0000","last_clean":"2023-12-20T06:03:23.115713+0000","last_became_active":"2023-12-20T06:03:21.990892+0000","last_became_peered":"2023-12-20T06:03:21.990892+0000","last_unstale":"2023-12-20T06:03:23.115713+0000","last_undegraded":"2023-12-20T06:03:23.115713+0000","last_fullsized":"2023-12-20T06:03:23.115713+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:52.328250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.980713+0000","last_change":"2023-12-20T06:03:21.576334+0000","last_active":"2023-12-20T06:03:21.980713+0000","last_peered":"2023-12-20T06:03:21.980713+0000","last_clean":"2023-12-20T06:03:21.980713+0000","last_became_active":"2023-12-20T06:03:21.575847+0000","last_became_peered":"2023-12-20T06:03:21.575847+0000","last_unstale":"2023-12-20T06:03:21.980713+0000","last_undegraded":"2023-12-20T06:03:21.980713+0000","last_fullsized":"2023-12-20T06:03:21.980713+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:46:33.911145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963084+0000","last_change":"2023-12-20T06:03:21.988502+0000","last_active":"2023-12-20T06:03:22.963084+0000","last_peered":"2023-12-20T06:03:22.963084+0000","last_clean":"2023-12-20T06:03:22.963084+0000","last_became_active":"2023-12-20T06:03:21.988209+0000","last_became_peered":"2023-12-20T06:03:21.988209+0000","last_unstale":"2023-12-20T06:03:22.963084+0000","last_undegraded":"2023-12-20T06:03:22.963084+0000","last_fullsized":"2023-12-20T06:03:22.963084+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:35:37.344273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"229'1","reported_seq":12,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.979298+0000","last_change":"2023-12-20T06:03:21.574438+0000","last_active":"2023-12-20T06:03:21.979298+0000","last_peered":"2023-12-20T06:03:21.979298+0000","last_clean":"2023-12-20T06:03:21.979298+0000","last_became_active":"2023-12-20T06:03:21.574161+0000","last_became_peered":"2023-12-20T06:03:21.574161+0000","last_unstale":"2023-12-20T06:03:21.979298+0000","last_undegraded":"2023-12-20T06:03:21.979298+0000","last_fullsized":"2023-12-20T06:03:21.979298+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:21:17.124107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963497+0000","last_change":"2023-12-20T06:03:21.985314+0000","last_active":"2023-12-20T06:03:22.963497+0000","last_peered":"2023-12-20T06:03:22.963497+0000","last_clean":"2023-12-20T06:03:22.963497+0000","last_became_active":"2023-12-20T06:03:21.984916+0000","last_became_peered":"2023-12-20T06:03:21.984916+0000","last_unstale":"2023-12-20T06:03:22.963497+0000","last_undegraded":"2023-12-20T06:03:22.963497+0000","last_fullsized":"2023-12-20T06:03:22.963497+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:55:10.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.956791+0000","last_change":"2023-12-20T06:03:21.576277+0000","last_active":"2023-12-20T06:03:21.956791+0000","last_peered":"2023-12-20T06:03:21.956791+0000","last_clean":"2023-12-20T06:03:21.956791+0000","last_became_active":"2023-12-20T06:03:21.575791+0000","last_became_peered":"2023-12-20T06:03:21.575791+0000","last_unstale":"2023-12-20T06:03:21.956791+0000","last_undegraded":"2023-12-20T06:03:21.956791+0000","last_fullsized":"2023-12-20T06:03:21.956791+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:24:57.234123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816899+0000","last_change":"2023-12-20T06:03:21.977046+0000","last_active":"2023-12-20T06:03:24.816899+0000","last_peered":"2023-12-20T06:03:24.816899+0000","last_clean":"2023-12-20T06:03:24.816899+0000","last_became_active":"2023-12-20T06:03:21.976756+0000","last_became_peered":"2023-12-20T06:03:21.976756+0000","last_unstale":"2023-12-20T06:03:24.816899+0000","last_undegraded":"2023-12-20T06:03:24.816899+0000","last_fullsized":"2023-12-20T06:03:24.816899+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:17.456340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.989855+0000","last_change":"2023-12-20T06:03:21.573650+0000","last_active":"2023-12-20T06:03:21.989855+0000","last_peered":"2023-12-20T06:03:21.989855+0000","last_clean":"2023-12-20T06:03:21.989855+0000","last_became_active":"2023-12-20T06:03:21.573349+0000","last_became_peered":"2023-12-20T06:03:21.573349+0000","last_unstale":"2023-12-20T06:03:21.989855+0000","last_undegraded":"2023-12-20T06:03:21.989855+0000","last_fullsized":"2023-12-20T06:03:21.989855+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:22.788125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963018+0000","last_change":"2023-12-20T06:03:21.983774+0000","last_active":"2023-12-20T06:03:22.963018+0000","last_peered":"2023-12-20T06:03:22.963018+0000","last_clean":"2023-12-20T06:03:22.963018+0000","last_became_active":"2023-12-20T06:03:21.983388+0000","last_became_peered":"2023-12-20T06:03:21.983388+0000","last_unstale":"2023-12-20T06:03:22.963018+0000","last_undegraded":"2023-12-20T06:03:22.963018+0000","last_fullsized":"2023-12-20T06:03:22.963018+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:11.133160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.971795+0000","last_change":"2023-12-20T06:03:21.567023+0000","last_active":"2023-12-20T06:03:21.971795+0000","last_peered":"2023-12-20T06:03:21.971795+0000","last_clean":"2023-12-20T06:03:21.971795+0000","last_became_active":"2023-12-20T06:03:21.566708+0000","last_became_peered":"2023-12-20T06:03:21.566708+0000","last_unstale":"2023-12-20T06:03:21.971795+0000","last_undegraded":"2023-12-20T06:03:21.971795+0000","last_fullsized":"2023-12-20T06:03:21.971795+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.958428+0000","last_change":"2023-12-20T06:03:21.972573+0000","last_active":"2023-12-20T06:03:22.958428+0000","last_peered":"2023-12-20T06:03:22.958428+0000","last_clean":"2023-12-20T06:03:22.958428+0000","last_became_active":"2023-12-20T06:03:21.972302+0000","last_became_peered":"2023-12-20T06:03:21.972302+0000","last_unstale":"2023-12-20T06:03:22.958428+0000","last_undegraded":"2023-12-20T06:03:22.958428+0000","last_fullsized":"2023-12-20T06:03:22.958428+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:58:25.448451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.966874+0000","last_change":"2023-12-20T06:03:21.570826+0000","last_active":"2023-12-20T06:03:21.966874+0000","last_peered":"2023-12-20T06:03:21.966874+0000","last_clean":"2023-12-20T06:03:21.966874+0000","last_became_active":"2023-12-20T06:03:21.570512+0000","last_became_peered":"2023-12-20T06:03:21.570512+0000","last_unstale":"2023-12-20T06:03:21.966874+0000","last_undegraded":"2023-12-20T06:03:21.966874+0000","last_fullsized":"2023-12-20T06:03:21.966874+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:52:49.965844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.816676+0000","last_change":"2023-12-20T06:03:21.976420+0000","last_active":"2023-12-20T06:03:24.816676+0000","last_peered":"2023-12-20T06:03:24.816676+0000","last_clean":"2023-12-20T06:03:24.816676+0000","last_became_active":"2023-12-20T06:03:21.976141+0000","last_became_peered":"2023-12-20T06:03:21.976141+0000","last_unstale":"2023-12-20T06:03:24.816676+0000","last_undegraded":"2023-12-20T06:03:24.816676+0000","last_fullsized":"2023-12-20T06:03:24.816676+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:27:59.998522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.971971+0000","last_change":"2023-12-20T06:03:21.567122+0000","last_active":"2023-12-20T06:03:21.971971+0000","last_peered":"2023-12-20T06:03:21.971971+0000","last_clean":"2023-12-20T06:03:21.971971+0000","last_became_active":"2023-12-20T06:03:21.566816+0000","last_became_peered":"2023-12-20T06:03:21.566816+0000","last_unstale":"2023-12-20T06:03:21.971971+0000","last_undegraded":"2023-12-20T06:03:21.971971+0000","last_fullsized":"2023-12-20T06:03:21.971971+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:41:11.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817051+0000","last_change":"2023-12-20T06:03:21.975113+0000","last_active":"2023-12-20T06:03:24.817051+0000","last_peered":"2023-12-20T06:03:24.817051+0000","last_clean":"2023-12-20T06:03:24.817051+0000","last_became_active":"2023-12-20T06:03:21.974862+0000","last_became_peered":"2023-12-20T06:03:21.974862+0000","last_unstale":"2023-12-20T06:03:24.817051+0000","last_undegraded":"2023-12-20T06:03:24.817051+0000","last_fullsized":"2023-12-20T06:03:24.817051+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:07:37.126937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.990432+0000","last_change":"2023-12-20T06:03:21.569876+0000","last_active":"2023-12-20T06:03:21.990432+0000","last_peered":"2023-12-20T06:03:21.990432+0000","last_clean":"2023-12-20T06:03:21.990432+0000","last_became_active":"2023-12-20T06:03:21.568286+0000","last_became_peered":"2023-12-20T06:03:21.568286+0000","last_unstale":"2023-12-20T06:03:21.990432+0000","last_undegraded":"2023-12-20T06:03:21.990432+0000","last_fullsized":"2023-12-20T06:03:21.990432+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963768+0000","last_change":"2023-12-20T06:03:21.991437+0000","last_active":"2023-12-20T06:03:22.963768+0000","last_peered":"2023-12-20T06:03:22.963768+0000","last_clean":"2023-12-20T06:03:22.963768+0000","last_became_active":"2023-12-20T06:03:21.991098+0000","last_became_peered":"2023-12-20T06:03:21.991098+0000","last_unstale":"2023-12-20T06:03:22.963768+0000","last_undegraded":"2023-12-20T06:03:22.963768+0000","last_fullsized":"2023-12-20T06:03:22.963768+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:48.782916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.956825+0000","last_change":"2023-12-20T06:03:21.574738+0000","last_active":"2023-12-20T06:03:21.956825+0000","last_peered":"2023-12-20T06:03:21.956825+0000","last_clean":"2023-12-20T06:03:21.956825+0000","last_became_active":"2023-12-20T06:03:21.574282+0000","last_became_peered":"2023-12-20T06:03:21.574282+0000","last_unstale":"2023-12-20T06:03:21.956825+0000","last_undegraded":"2023-12-20T06:03:21.956825+0000","last_fullsized":"2023-12-20T06:03:21.956825+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:43:56.675318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963216+0000","last_change":"2023-12-20T06:03:21.985967+0000","last_active":"2023-12-20T06:03:22.963216+0000","last_peered":"2023-12-20T06:03:22.963216+0000","last_clean":"2023-12-20T06:03:22.963216+0000","last_became_active":"2023-12-20T06:03:21.985460+0000","last_became_peered":"2023-12-20T06:03:21.985460+0000","last_unstale":"2023-12-20T06:03:22.963216+0000","last_undegraded":"2023-12-20T06:03:22.963216+0000","last_fullsized":"2023-12-20T06:03:22.963216+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:06:16.446428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.958446+0000","last_change":"2023-12-20T06:03:21.573978+0000","last_active":"2023-12-20T06:03:21.958446+0000","last_peered":"2023-12-20T06:03:21.958446+0000","last_clean":"2023-12-20T06:03:21.958446+0000","last_became_active":"2023-12-20T06:03:21.573682+0000","last_became_peered":"2023-12-20T06:03:21.573682+0000","last_unstale":"2023-12-20T06:03:21.958446+0000","last_undegraded":"2023-12-20T06:03:21.958446+0000","last_fullsized":"2023-12-20T06:03:21.958446+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:43.873712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.817240+0000","last_change":"2023-12-20T06:03:21.980139+0000","last_active":"2023-12-20T06:03:24.817240+0000","last_peered":"2023-12-20T06:03:24.817240+0000","last_clean":"2023-12-20T06:03:24.817240+0000","last_became_active":"2023-12-20T06:03:21.979858+0000","last_became_peered":"2023-12-20T06:03:21.979858+0000","last_unstale":"2023-12-20T06:03:24.817240+0000","last_undegraded":"2023-12-20T06:03:24.817240+0000","last_fullsized":"2023-12-20T06:03:24.817240+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:21:31.283475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.973764+0000","last_change":"2023-12-20T06:03:21.570642+0000","last_active":"2023-12-20T06:03:21.973764+0000","last_peered":"2023-12-20T06:03:21.973764+0000","last_clean":"2023-12-20T06:03:21.973764+0000","last_became_active":"2023-12-20T06:03:21.569631+0000","last_became_peered":"2023-12-20T06:03:21.569631+0000","last_unstale":"2023-12-20T06:03:21.973764+0000","last_undegraded":"2023-12-20T06:03:21.973764+0000","last_fullsized":"2023-12-20T06:03:21.973764+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:24.089448+0000","last_change":"2023-12-20T06:03:21.963940+0000","last_active":"2023-12-20T06:03:24.089448+0000","last_peered":"2023-12-20T06:03:24.089448+0000","last_clean":"2023-12-20T06:03:24.089448+0000","last_became_active":"2023-12-20T06:03:21.963679+0000","last_became_peered":"2023-12-20T06:03:21.963679+0000","last_unstale":"2023-12-20T06:03:24.089448+0000","last_undegraded":"2023-12-20T06:03:24.089448+0000","last_fullsized":"2023-12-20T06:03:24.089448+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.971959+0000","last_change":"2023-12-20T06:03:21.573888+0000","last_active":"2023-12-20T06:03:21.971959+0000","last_peered":"2023-12-20T06:03:21.971959+0000","last_clean":"2023-12-20T06:03:21.971959+0000","last_became_active":"2023-12-20T06:03:21.572823+0000","last_became_peered":"2023-12-20T06:03:21.572823+0000","last_unstale":"2023-12-20T06:03:21.971959+0000","last_undegraded":"2023-12-20T06:03:21.971959+0000","last_fullsized":"2023-12-20T06:03:21.971959+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.963147+0000","last_change":"2023-12-20T06:03:21.983107+0000","last_active":"2023-12-20T06:03:22.963147+0000","last_peered":"2023-12-20T06:03:22.963147+0000","last_clean":"2023-12-20T06:03:22.963147+0000","last_became_active":"2023-12-20T06:03:21.981255+0000","last_became_peered":"2023-12-20T06:03:21.981255+0000","last_unstale":"2023-12-20T06:03:22.963147+0000","last_undegraded":"2023-12-20T06:03:22.963147+0000","last_fullsized":"2023-12-20T06:03:22.963147+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.963949+0000","last_change":"2023-12-20T06:03:21.558503+0000","last_active":"2023-12-20T06:03:21.963949+0000","last_peered":"2023-12-20T06:03:21.963949+0000","last_clean":"2023-12-20T06:03:21.963949+0000","last_became_active":"2023-12-20T06:03:21.557644+0000","last_became_peered":"2023-12-20T06:03:21.557644+0000","last_unstale":"2023-12-20T06:03:21.963949+0000","last_undegraded":"2023-12-20T06:03:21.963949+0000","last_fullsized":"2023-12-20T06:03:21.963949+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:48.619779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962106+0000","last_change":"2023-12-20T06:03:21.986446+0000","last_active":"2023-12-20T06:03:22.962106+0000","last_peered":"2023-12-20T06:03:22.962106+0000","last_clean":"2023-12-20T06:03:22.962106+0000","last_became_active":"2023-12-20T06:03:21.986120+0000","last_became_peered":"2023-12-20T06:03:21.986120+0000","last_unstale":"2023-12-20T06:03:22.962106+0000","last_undegraded":"2023-12-20T06:03:22.962106+0000","last_fullsized":"2023-12-20T06:03:22.962106+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:27:59.998522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.962095+0000","last_change":"2023-12-20T06:03:21.570337+0000","last_active":"2023-12-20T06:03:21.962095+0000","last_peered":"2023-12-20T06:03:21.962095+0000","last_clean":"2023-12-20T06:03:21.962095+0000","last_became_active":"2023-12-20T06:03:21.569986+0000","last_became_peered":"2023-12-20T06:03:21.569986+0000","last_unstale":"2023-12-20T06:03:21.962095+0000","last_undegraded":"2023-12-20T06:03:21.962095+0000","last_fullsized":"2023-12-20T06:03:21.962095+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:28.719459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962082+0000","last_change":"2023-12-20T06:03:21.968100+0000","last_active":"2023-12-20T06:03:22.962082+0000","last_peered":"2023-12-20T06:03:22.962082+0000","last_clean":"2023-12-20T06:03:22.962082+0000","last_became_active":"2023-12-20T06:03:21.967593+0000","last_became_peered":"2023-12-20T06:03:21.967593+0000","last_unstale":"2023-12-20T06:03:22.962082+0000","last_undegraded":"2023-12-20T06:03:22.962082+0000","last_fullsized":"2023-12-20T06:03:22.962082+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:00:01.717868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":11,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.971646+0000","last_change":"2023-12-20T06:03:21.568281+0000","last_active":"2023-12-20T06:03:21.971646+0000","last_peered":"2023-12-20T06:03:21.971646+0000","last_clean":"2023-12-20T06:03:21.971646+0000","last_became_active":"2023-12-20T06:03:21.567976+0000","last_became_peered":"2023-12-20T06:03:21.567976+0000","last_unstale":"2023-12-20T06:03:21.971646+0000","last_undegraded":"2023-12-20T06:03:21.971646+0000","last_fullsized":"2023-12-20T06:03:21.971646+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:03:18.004400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":10,"reported_epoch":230,"state":"active+clean","last_fresh":"2023-12-20T06:03:22.962433+0000","last_change":"2023-12-20T06:03:21.973527+0000","last_active":"2023-12-20T06:03:22.962433+0000","last_peered":"2023-12-20T06:03:22.962433+0000","last_clean":"2023-12-20T06:03:22.962433+0000","last_became_active":"2023-12-20T06:03:21.973178+0000","last_became_peered":"2023-12-20T06:03:21.973178+0000","last_unstale":"2023-12-20T06:03:22.962433+0000","last_undegraded":"2023-12-20T06:03:22.962433+0000","last_fullsized":"2023-12-20T06:03:22.962433+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:55:50.794593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"229'2","reported_seq":13,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.991846+0000","last_change":"2023-12-20T06:03:21.567717+0000","last_active":"2023-12-20T06:03:21.991846+0000","last_peered":"2023-12-20T06:03:21.991846+0000","last_clean":"2023-12-20T06:03:21.991846+0000","last_became_active":"2023-12-20T06:03:21.567425+0000","last_became_peered":"2023-12-20T06:03:21.567425+0000","last_unstale":"2023-12-20T06:03:21.991846+0000","last_undegraded":"2023-12-20T06:03:21.991846+0000","last_fullsized":"2023-12-20T06:03:21.991846+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:45.186933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"215'3223","reported_seq":4089,"reported_epoch":229,"state":"active+clean","last_fresh":"2023-12-20T06:03:21.956202+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:03:21.956202+0000","last_peered":"2023-12-20T06:03:21.956202+0000","last_clean":"2023-12-20T06:03:21.956202+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:03:21.956202+0000","last_undegraded":"2023-12-20T06:03:21.956202+0000","last_fullsized":"2023-12-20T06:03:21.956202+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3223,"log_dups_size":0,"ondisk_log_size":3223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":561,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":561,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3223,"ondisk_log_size":3223,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575980,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60696,"kb_used_data":3408,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311144,"statfs":{"total":96636764160,"available":96574611456,"internally_reserved":0,"allocated":3489792,"data_stored":2002471,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.69599999999999995,"15min":0.72799999999999998},"min":{"1min":0.45400000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":1.028,"15min":1.042},"last":0.49299999999999999},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.99099999999999999,"15min":1.161},"last":0.871}]},{"osd":1,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71499999999999997,"15min":0.74199999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":0.996,"15min":1.1279999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.71599999999999997,"15min":0.745},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.85499999999999998,"5min":0.96699999999999997,"15min":1.032},"last":0.68600000000000005}]},{"osd":2,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.70599999999999996,"15min":0.73899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84499999999999997,"5min":0.95399999999999996,"15min":1.014},"last":0.77800000000000002},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.49099999999999999},"max":{"1min":0.88700000000000001,"5min":1.0089999999999999,"15min":1.0189999999999999},"last":0.55100000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.47699999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.85299999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.89900000000000002,"5min":1.0660000000000001,"15min":1.0840000000000001},"last":0.66500000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.04},"last":0.749},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.02,"15min":1.02},"last":0.70399999999999996}]},{"osd":5,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68300000000000005,"15min":0.66800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.052},"last":0.80200000000000005},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.875,"5min":1.0229999999999999,"15min":1.103},"last":0.82599999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76300000000000001,"15min":0.746},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.90100000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72399999999999998,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93500000000000005,"5min":1.0740000000000001,"15min":1.081},"last":0.57999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575980,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51928,"kb_used_data":2832,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319912,"statfs":{"total":96636764160,"available":96583589888,"internally_reserved":0,"allocated":2899968,"data_stored":1412089,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71199999999999997,"15min":0.73999999999999999},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92400000000000004,"5min":1.016,"15min":1.0980000000000001},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72999999999999998,"15min":0.75},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.006,"5min":1.016,"15min":1.05},"last":0.80200000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.47399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90100000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.70899999999999996},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91200000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.78600000000000003}]},{"osd":2,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76900000000000002,"15min":0.78100000000000003},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97699999999999998,"5min":1.069,"15min":1.069},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.54400000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.93100000000000005,"5min":0.97799999999999998,"15min":1.0760000000000001},"last":0.69099999999999995}]},{"osd":3,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78900000000000003},"min":{"1min":0.61299999999999999,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.0109999999999999,"5min":1.077,"15min":1.077},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.997,"5min":1.052,"15min":1.052},"last":0.66300000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.36099999999999999},"max":{"1min":0.96299999999999997,"5min":1.05,"15min":1.0529999999999999},"last":0.52600000000000002},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66400000000000003,"15min":0.65100000000000002},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.59299999999999997}]},{"osd":5,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64900000000000002,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.72499999999999998}]},{"osd":7,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.55700000000000005,"5min":0.48899999999999999,"15min":0.45300000000000001},"max":{"1min":1.0429999999999999,"5min":1.103,"15min":1.1240000000000001},"last":0.749},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.76700000000000002}]}]},{"osd":5,"up_from":13,"seq":55834575981,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51920,"kb_used_data":2824,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319920,"statfs":{"total":96636764160,"available":96583598080,"internally_reserved":0,"allocated":2891776,"data_stored":1412033,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71399999999999997,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91600000000000004,"5min":0.94799999999999995,"15min":1.077},"last":0.62},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71499999999999997,"15min":0.746},"min":{"1min":0.45600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.80300000000000005,"5min":0.96699999999999997,"15min":1.028},"last":0.80300000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":0.875,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88500000000000001,"5min":0.98799999999999999,"15min":1.0209999999999999},"last":0.75}]},{"osd":2,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.751},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89000000000000001,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.68400000000000005},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.75700000000000001,"15min":0.77000000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.48299999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0680000000000001},"last":0.92700000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"max":{"1min":0.91000000000000003,"5min":1.05,"15min":1.0669999999999999},"last":0.90200000000000002},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.0629999999999999},"last":0.78600000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.56000000000000005,"5min":0.42199999999999999,"15min":0.38200000000000001},"max":{"1min":0.94399999999999995,"5min":1.0640000000000001,"15min":1.077},"last":0.873},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.64800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"max":{"1min":1.0649999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.65000000000000002}]},{"osd":6,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.0449999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.72199999999999998}]},{"osd":7,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73799999999999999,"15min":0.70999999999999996},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.153,"15min":1.153},"last":0.85699999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.48299999999999998},"max":{"1min":0.98799999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.94299999999999995}]}]},{"osd":4,"up_from":13,"seq":55834575982,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47828,"kb_used_data":2828,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324012,"statfs":{"total":96636764160,"available":96587788288,"internally_reserved":0,"allocated":2895872,"data_stored":1412615,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":8,"apply_latency_ms":8,"commit_latency_ns":8000000,"apply_latency_ns":8000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.69299999999999995,"15min":0.72899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96499999999999997,"5min":0.98999999999999999,"15min":1.0209999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.69299999999999995,"15min":0.72299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.80900000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.60299999999999998}]},{"osd":1,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73099999999999998,"15min":0.752},"min":{"1min":0.498,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":0.89600000000000002,"5min":0.95699999999999996,"15min":1.0900000000000001},"last":0.54600000000000004},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73899999999999999,"15min":0.76400000000000001},"min":{"1min":0.47899999999999998,"5min":0.47399999999999998,"15min":0.44500000000000001},"max":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.79000000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.746,"15min":0.76900000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98199999999999998,"5min":1.0089999999999999,"15min":1.101},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.75},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49199999999999999},"max":{"1min":0.88,"5min":1.014,"15min":1.014},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.80100000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1,"5min":1.04,"15min":1.081},"last":0.90600000000000003}]},{"osd":5,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64700000000000002,"15min":0.65000000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98099999999999998,"5min":1.05,"15min":1.05},"last":0.83299999999999996},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.47599999999999998,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.96999999999999997,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.66700000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.1379999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.41199999999999998},"max":{"1min":1.056,"5min":1.107,"15min":1.111},"last":0.94399999999999995}]},{"osd":7,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72899999999999998,"15min":0.72999999999999998},"min":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":1.0329999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.626,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.1100000000000001,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.69199999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575981,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47840,"kb_used_data":2840,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94324000,"statfs":{"total":96636764160,"available":96587776000,"internally_reserved":0,"allocated":2908160,"data_stored":1413667,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.46200000000000002,"15min":0.47099999999999997},"min":{"1min":0.30199999999999999,"5min":0.30199999999999999,"15min":0.28499999999999998},"max":{"1min":0.59299999999999997,"5min":0.748,"15min":2.8130000000000002},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.504,"5min":0.51100000000000001,"15min":0.53200000000000003},"min":{"1min":0.32800000000000001,"5min":0.32500000000000001,"15min":0.32500000000000001},"max":{"1min":0.64600000000000002,"5min":0.80000000000000004,"15min":1.357},"last":0.47999999999999998}]},{"osd":1,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.45600000000000002,"5min":0.46800000000000003,"15min":0.47999999999999998},"min":{"1min":0.34000000000000002,"5min":0.318,"15min":0.318},"max":{"1min":0.81299999999999994,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.39300000000000002},{"interface":"front","average":{"1min":0.52500000000000002,"5min":0.55000000000000004,"15min":0.57299999999999995},"min":{"1min":0.30599999999999999,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.73699999999999999,"5min":0.81799999999999995,"15min":1.4219999999999999},"last":0.54400000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.58199999999999996,"5min":0.58899999999999997,"15min":0.59999999999999998},"min":{"1min":0.432,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.52900000000000003},{"interface":"front","average":{"1min":0.54800000000000004,"5min":0.56000000000000005,"15min":0.57699999999999996},"min":{"1min":0.40500000000000003,"5min":0.375,"15min":0.34200000000000003},"max":{"1min":0.72599999999999998,"5min":0.86099999999999999,"15min":2.944},"last":0.56200000000000006}]},{"osd":4,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77100000000000002,"5min":0.78300000000000003,"15min":0.81999999999999995},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":1.0209999999999999,"5min":1.2090000000000001,"15min":1.669},"last":0.83999999999999997},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.77300000000000002,"15min":0.81399999999999995},"min":{"1min":0.52500000000000002,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1299999999999999,"5min":1.1299999999999999,"15min":1.99},"last":0.75900000000000001}]},{"osd":5,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.80900000000000005,"15min":0.83499999999999996},"min":{"1min":0.59599999999999997,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.97199999999999998,"5min":1.355,"15min":1.901},"last":0.871},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.77500000000000002,"15min":0.82399999999999995},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.97699999999999998,"5min":1.196,"15min":2.8639999999999999},"last":0.82299999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.77900000000000003,"5min":0.79400000000000004,"15min":0.81999999999999995},"min":{"1min":0.53300000000000003,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":1.0600000000000001,"5min":1.194,"15min":1.575},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.78600000000000003,"5min":0.81799999999999995,"15min":0.85099999999999998},"min":{"1min":0.55000000000000004,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0149999999999999,"5min":1.2350000000000001,"15min":1.948},"last":0.92400000000000004}]},{"osd":7,"last update":"Wed Dec 20 06:02:26 2023","interfaces":[{"interface":"back","average":{"1min":0.752,"5min":0.80400000000000005,"15min":0.84099999999999997},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":1.004,"5min":1.3149999999999999,"15min":2.9049999999999998},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.80600000000000005,"5min":0.82399999999999995,"15min":0.85299999999999998},"min":{"1min":0.61899999999999999,"5min":0.502,"15min":0.502},"max":{"1min":1.089,"5min":1.28,"15min":1.754},"last":0.94199999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575980,"num_pgs":23,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51940,"kb_used_data":2844,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319900,"statfs":{"total":96636764160,"available":96583577600,"internally_reserved":0,"allocated":2912256,"data_stored":1413653,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.45000000000000001,"15min":0.45500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52900000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.47699999999999998},{"interface":"front","average":{"1min":0.438,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.60699999999999998,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.45800000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.79800000000000004,"15min":0.82199999999999995},"last":0.374},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30499999999999999},"max":{"1min":0.61199999999999999,"5min":0.92600000000000005,"15min":1.004},"last":0.50700000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.53000000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.65500000000000003,"15min":0.67400000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.59199999999999997,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.55800000000000005}]},{"osd":4,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":1.254},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77900000000000003,"15min":0.79600000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":1.605,"15min":1.605},"last":0.753}]},{"osd":5,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97099999999999997,"5min":1.329,"15min":1.329},"last":0.76500000000000001},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98199999999999998,"5min":1.0740000000000001,"15min":1.095},"last":0.84999999999999998}]},{"osd":6,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92200000000000004,"5min":1.919,"15min":1.919},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.95099999999999996,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0049999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.88200000000000001,"15min":0.88500000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.996,"5min":1.292,"15min":1.292},"last":0.73799999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575982,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60104,"kb_used_data":2816,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311736,"statfs":{"total":96636764160,"available":96575217664,"internally_reserved":0,"allocated":2883584,"data_stored":1405767,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47899999999999998,"15min":0.46200000000000002},"min":{"1min":0.26500000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59299999999999997,"5min":0.746,"15min":0.81000000000000005},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.29399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.54700000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.71999999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45700000000000002},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":0.88800000000000001,"15min":0.93700000000000006},"last":0.63200000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.42999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.90100000000000002},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.93500000000000005},"last":0.48399999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.78500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92000000000000004,"5min":1.107,"15min":1.321},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.995,"5min":1.139,"15min":1.25},"last":0.77500000000000002}]},{"osd":5,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78700000000000003,"15min":0.80200000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.97899999999999998,"5min":1.224,"15min":1.2849999999999999},"last":0.52500000000000002},{"interface":"front","average":{"1min":0.746,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0740000000000001,"5min":1.1990000000000001,"15min":1.4330000000000001},"last":0.67700000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.82399999999999995},"min":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.2430000000000001,"15min":1.341},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.56399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1379999999999999,"5min":1.177,"15min":1.3839999999999999},"last":0.66300000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0900000000000001,"5min":1.272,"15min":1.4470000000000001},"last":0.621},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0509999999999999,"5min":1.1599999999999999,"15min":1.3839999999999999},"last":0.71299999999999997}]}]},{"osd":0,"up_from":13,"seq":55834575981,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56604,"kb_used_data":3412,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315236,"statfs":{"total":96636764160,"available":96578801664,"internally_reserved":0,"allocated":3493888,"data_stored":2003073,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.29299999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33000000000000002},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.35699999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.38800000000000001},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.40999999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.59899999999999998},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.44500000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82999999999999996,"15min":0.82599999999999996},"min":{"1min":0.56699999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.66700000000000004},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.53400000000000003}]},{"osd":5,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.86099999999999999,"15min":0.85099999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.64700000000000002},{"interface":"front","average":{"1min":0.873,"5min":0.83399999999999996,"15min":0.84799999999999998},"min":{"1min":0.54600000000000004,"5min":0.495,"15min":0.495},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":0.63200000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":0.55800000000000005},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.89100000000000001,"15min":0.871},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57099999999999995},"max":{"1min":5.6310000000000002,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.61799999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":5.7160000000000002,"5min":5.7160000000000002,"15min":5.7160000000000002},"last":0.69299999999999995},{"interface":"front","average":{"1min":1.012,"5min":0.88200000000000001,"15min":0.871},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":4.2119999999999997,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.56899999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":1634,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:03:26.478 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T06:03:26.478 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T06:03:26.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.671+0000 7fe5d32f2640 1 Processor -- start 2023-12-20T06:03:26.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.671+0000 7fe5d32f2640 1 -- start start 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.671+0000 7fe5d32f2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5cc15a940 0x7fe5cc0cc380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.671+0000 7fe5d32f2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe5cc15ad20 0x7fe5cc0ccee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.671+0000 7fe5d32f2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe5cc0cc9d0 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.671+0000 7fe5d32f2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5cc0c9b90 con 0x7fe5cc15a940 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.671+0000 7fe5d32f2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe5cc0c9e70 con 0x7fe5cc15ad20 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.671+0000 7fe5d108e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5cc15a940 0x7fe5cc0cc380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5d108e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe5cc15a940 0x7fe5cc0cc380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33450/0 (socket says 172.21.15.32:33450) 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5d108e640 1 -- 172.21.15.32:0/1876146081 learned_addr learned my addr 172.21.15.32:0/1876146081 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5d088d640 1 --2- 172.21.15.32:0/1876146081 >> v2:172.21.15.39:3300/0 conn(0x7fe5cc15ad20 0x7fe5cc0ccee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5d108e640 1 -- 172.21.15.32:0/1876146081 >> v2:172.21.15.39:3300/0 conn(0x7fe5cc15ad20 msgr2=0x7fe5cc0ccee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5d108e640 1 --2- 172.21.15.32:0/1876146081 >> v2:172.21.15.39:3300/0 conn(0x7fe5cc15ad20 0x7fe5cc0ccee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5d108e640 1 -- 172.21.15.32:0/1876146081 >> v1:172.21.15.32:6789/0 conn(0x7fe5cc0cb1c0 legacy=0x7fe5cc070210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5d108e640 1 -- 172.21.15.32:0/1876146081 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5cc0ca150 con 0x7fe5cc15a940 2023-12-20T06:03:26.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5c3fff640 1 -- 172.21.15.32:0/1876146081 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2862409758 0 0) 0x7fe5cc0cc9d0 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5d108e640 1 --2- 172.21.15.32:0/1876146081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc15a940 0x7fe5cc0cc380 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0ce460 tx=0x7fe5c801cf30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb19c7c5c6b0de58 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:26.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5c3fff640 1 -- 172.21.15.32:0/1876146081 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80aa650 con 0x7fe5cc15a940 2023-12-20T06:03:26.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5c3fff640 1 -- 172.21.15.32:0/1876146081 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c80aa7b0 con 0x7fe5cc15a940 2023-12-20T06:03:26.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.675+0000 7fe5c3fff640 1 -- 172.21.15.32:0/1876146081 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80aab40 con 0x7fe5cc15a940 2023-12-20T06:03:26.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 -- 172.21.15.32:0/1876146081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc15a940 msgr2=0x7fe5cc0cc380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 --2- 172.21.15.32:0/1876146081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc15a940 0x7fe5cc0cc380 secure :-1 s=READY pgs=2254 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0ce460 tx=0x7fe5c801cf30 comp rx=0 tx=0).stop 2023-12-20T06:03:26.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 -- 172.21.15.32:0/1876146081 shutdown_connections 2023-12-20T06:03:26.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 --2- 172.21.15.32:0/1876146081 >> v2:172.21.15.39:3300/0 conn(0x7fe5cc15ad20 0x7fe5cc0ccee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 --2- 172.21.15.32:0/1876146081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc15a940 0x7fe5cc0cc380 unknown :-1 s=CLOSED pgs=2254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 -- 172.21.15.32:0/1876146081 >> 172.21.15.32:0/1876146081 conn(0x7fe5cc06a4b0 msgr2=0x7fe5cc0b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:26.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 -- 172.21.15.32:0/1876146081 shutdown_connections 2023-12-20T06:03:26.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 -- 172.21.15.32:0/1876146081 wait complete. 2023-12-20T06:03:26.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 Processor -- start 2023-12-20T06:03:26.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 -- start start 2023-12-20T06:03:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0cb1c0 0x7fe5cc14ddc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc15a940 0x7fe5cc0c5880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5cc15ad20 0x7fe5cc14fbf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc0aba00 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc0abce0 con 0x7fe5cc15a940 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d32f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe5cc0abfc0 con 0x7fe5cc15ad20 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d108e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0cb1c0 0x7fe5cc14ddc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d108e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0cb1c0 0x7fe5cc14ddc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33452/0 (socket says 172.21.15.32:33452) 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d108e640 1 -- 172.21.15.32:0/945922311 learned_addr learned my addr 172.21.15.32:0/945922311 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d088d640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc15a940 0x7fe5cc0c5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d188f640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5cc15ad20 0x7fe5cc14fbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d108e640 1 -- 172.21.15.32:0/945922311 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5cc15ad20 msgr2=0x7fe5cc14fbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d108e640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5cc15ad20 0x7fe5cc14fbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d108e640 1 -- 172.21.15.32:0/945922311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc15a940 msgr2=0x7fe5cc0c5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d108e640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc15a940 0x7fe5cc0c5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d108e640 1 -- 172.21.15.32:0/945922311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe5c80094d0 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.679+0000 7fe5d188f640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5cc15ad20 0x7fe5cc14fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:26.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.683+0000 7fe5d108e640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0cb1c0 0x7fe5cc14ddc0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fe5c801ab80 tx=0x7fe5c801ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.683+0000 7fe5c1ffb640 1 -- 172.21.15.32:0/945922311 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c80aad30 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.683+0000 7fe5c1ffb640 1 -- 172.21.15.32:0/945922311 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe5c80b7070 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.683+0000 7fe5c1ffb640 1 -- 172.21.15.32:0/945922311 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe5c8004220 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.683+0000 7fe5d32f2640 1 -- 172.21.15.32:0/945922311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe5cc0ac2a0 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.683+0000 7fe5d32f2640 1 -- 172.21.15.32:0/945922311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe5cc14e300 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.683+0000 7fe5d32f2640 1 -- 172.21.15.32:0/945922311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe5cc04e000 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.683+0000 7fe5c1ffb640 1 -- 172.21.15.32:0/945922311 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fe5c8004380 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.687+0000 7fe5c1ffb640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe5b0065ca0 0x7fe5b0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.687+0000 7fe5c1ffb640 1 -- 172.21.15.32:0/945922311 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe5c811da00 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.687+0000 7fe5d088d640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe5b0065ca0 0x7fe5b0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:26.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.687+0000 7fe5d088d640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe5b0065ca0 0x7fe5b0068160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0c5810 tx=0x7fe5bc008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:26.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.691+0000 7fe5c1ffb640 1 -- 172.21.15.32:0/945922311 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe5c80eb010 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.899+0000 7fe5d32f2640 1 -- 172.21.15.32:0/945922311 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7fe5cc0ccfa0 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5c1ffb640 1 -- 172.21.15.32:0/945922311 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v53) v1 ==== 190+0+0 (secure 0 0 0) 0x7fe5c80b9020 con 0x7fe5cc0cb1c0 2023-12-20T06:03:26.905 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T06:03:26.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 -- 172.21.15.32:0/945922311 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe5b0065ca0 msgr2=0x7fe5b0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe5b0065ca0 0x7fe5b0068160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fe5cc0c5810 tx=0x7fe5bc008040 comp rx=0 tx=0).stop 2023-12-20T06:03:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 -- 172.21.15.32:0/945922311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0cb1c0 msgr2=0x7fe5cc14ddc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0cb1c0 0x7fe5cc14ddc0 secure :-1 s=READY pgs=2255 cs=0 l=1 rev1=1 crypto rx=0x7fe5c801ab80 tx=0x7fe5c801ad10 comp rx=0 tx=0).stop 2023-12-20T06:03:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 -- 172.21.15.32:0/945922311 shutdown_connections 2023-12-20T06:03:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe5b0065ca0 0x7fe5b0068160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe5cc15ad20 0x7fe5cc14fbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe5cc15a940 0x7fe5cc0c5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 --2- 172.21.15.32:0/945922311 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe5cc0cb1c0 0x7fe5cc14ddc0 unknown :-1 s=CLOSED pgs=2255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 -- 172.21.15.32:0/945922311 >> 172.21.15.32:0/945922311 conn(0x7fe5cc06a4b0 msgr2=0x7fe5cc154570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 -- 172.21.15.32:0/945922311 shutdown_connections 2023-12-20T06:03:26.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:26.907+0000 7fe5d32f2640 1 -- 172.21.15.32:0/945922311 wait complete. 2023-12-20T06:03:26.926 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:27.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.071+0000 7ff2e5598640 1 Processor -- start 2023-12-20T06:03:27.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.071+0000 7ff2e5598640 1 -- start start 2023-12-20T06:03:27.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2e5598640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff2e006ce60 0x7ff2e015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2e5598640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2e015f110 0x7ff2e015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2e5598640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff2e015bbd0 con 0x7ff2e00af5e0 2023-12-20T06:03:27.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2e5598640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff2e015beb0 con 0x7ff2e006ce60 2023-12-20T06:03:27.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2e5598640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e015e910 con 0x7ff2e015f110 2023-12-20T06:03:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2d7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff2e006ce60 0x7ff2e015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2df7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2e015f110 0x7ff2e015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2d7fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff2e006ce60 0x7ff2e015b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:36662/0 (socket says 172.21.15.32:36662) 2023-12-20T06:03:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2d7fff640 1 -- 172.21.15.32:0/696768664 learned_addr learned my addr 172.21.15.32:0/696768664 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2d7fff640 1 -- 172.21.15.32:0/696768664 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2e015f110 msgr2=0x7ff2e015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:27.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2d7fff640 1 --2- 172.21.15.32:0/696768664 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2e015f110 0x7ff2e015c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2d7fff640 1 -- 172.21.15.32:0/696768664 >> v1:172.21.15.39:6789/0 conn(0x7ff2e00af5e0 legacy=0x7ff2e00af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2d7fff640 1 -- 172.21.15.32:0/696768664 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2e015ebf0 con 0x7ff2e006ce60 2023-12-20T06:03:27.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.075+0000 7ff2df7fe640 1 --2- 172.21.15.32:0/696768664 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2e015f110 0x7ff2e015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:27.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2d7fff640 1 --2- 172.21.15.32:0/696768664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e006ce60 0x7ff2e015b280 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009870 tx=0x7ff2c801c4b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=6eba3bf14e83a222 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2de7fc640 1 -- 172.21.15.32:0/696768664 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2e015b9d0 con 0x7ff2e006ce60 2023-12-20T06:03:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2de7fc640 1 -- 172.21.15.32:0/696768664 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c80ab650 con 0x7ff2e006ce60 2023-12-20T06:03:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2de7fc640 1 -- 172.21.15.32:0/696768664 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80ab9e0 con 0x7ff2e006ce60 2023-12-20T06:03:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2e5598640 1 -- 172.21.15.32:0/696768664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e006ce60 msgr2=0x7ff2e015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2e5598640 1 --2- 172.21.15.32:0/696768664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e006ce60 0x7ff2e015b280 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7ff2c8009870 tx=0x7ff2c801c4b0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2e5598640 1 -- 172.21.15.32:0/696768664 shutdown_connections 2023-12-20T06:03:27.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2e5598640 1 --2- 172.21.15.32:0/696768664 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2e015f110 0x7ff2e015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2e5598640 1 --2- 172.21.15.32:0/696768664 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e006ce60 0x7ff2e015b280 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2e5598640 1 -- 172.21.15.32:0/696768664 >> 172.21.15.32:0/696768664 conn(0x7ff2e006a320 msgr2=0x7ff2e00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2e5598640 1 -- 172.21.15.32:0/696768664 shutdown_connections 2023-12-20T06:03:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2e5598640 1 -- 172.21.15.32:0/696768664 wait complete. 2023-12-20T06:03:27.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2e5598640 1 Processor -- start 2023-12-20T06:03:27.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.079+0000 7ff2e5598640 1 -- start start 2023-12-20T06:03:27.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2e5598640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2e006ce60 0x7ff2e015a370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2e5598640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2e00af5e0 0x7ff2e00c5830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2e5598640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e015f110 0x7ff2e00c74f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2e5598640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e00c5d70 con 0x7ff2e00af5e0 2023-12-20T06:03:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2e5598640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e0159ee0 con 0x7ff2e015f110 2023-12-20T06:03:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2e5598640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff2e0157240 con 0x7ff2e006ce60 2023-12-20T06:03:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2e00af5e0 0x7ff2e00c5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2e00af5e0 0x7ff2e00c5830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33456/0 (socket says 172.21.15.32:33456) 2023-12-20T06:03:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d7fff640 1 -- 172.21.15.32:0/646542107 learned_addr learned my addr 172.21.15.32:0/646542107 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2df7fe640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e015f110 0x7ff2e00c74f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2deffd640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2e006ce60 0x7ff2e015a370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d7fff640 1 -- 172.21.15.32:0/646542107 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2e006ce60 msgr2=0x7ff2e015a370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d7fff640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2e006ce60 0x7ff2e015a370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d7fff640 1 -- 172.21.15.32:0/646542107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e015f110 msgr2=0x7ff2e00c74f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d7fff640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e015f110 0x7ff2e00c74f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d7fff640 1 -- 172.21.15.32:0/646542107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff2d8009a70 con 0x7ff2e00af5e0 2023-12-20T06:03:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d7fff640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2e00af5e0 0x7ff2e00c5830 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7ff2c801a0e0 tx=0x7ff2c801acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d77fe640 1 -- 172.21.15.32:0/646542107 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80b4600 con 0x7ff2e00af5e0 2023-12-20T06:03:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d77fe640 1 -- 172.21.15.32:0/646542107 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff2c80b4780 con 0x7ff2e00af5e0 2023-12-20T06:03:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d77fe640 1 -- 172.21.15.32:0/646542107 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff2c80b4b30 con 0x7ff2e00af5e0 2023-12-20T06:03:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2e5598640 1 -- 172.21.15.32:0/646542107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff2c80094d0 con 0x7ff2e00af5e0 2023-12-20T06:03:27.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2e5598640 1 -- 172.21.15.32:0/646542107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff2e00b3e40 con 0x7ff2e00af5e0 2023-12-20T06:03:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2e5598640 1 -- 172.21.15.32:0/646542107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff2e004e000 con 0x7ff2e00af5e0 2023-12-20T06:03:27.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.083+0000 7ff2d77fe640 1 -- 172.21.15.32:0/646542107 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff2c80b4c90 con 0x7ff2e00af5e0 2023-12-20T06:03:27.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.087+0000 7ff2d77fe640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff2c0065ca0 0x7ff2c0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.087+0000 7ff2d77fe640 1 -- 172.21.15.32:0/646542107 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff2c811dca0 con 0x7ff2e00af5e0 2023-12-20T06:03:27.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.087+0000 7ff2deffd640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff2c0065ca0 0x7ff2c0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.087+0000 7ff2deffd640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff2c0065ca0 0x7ff2c0068160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00017e0 tx=0x7ff2d0009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:27.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.091+0000 7ff2d77fe640 1 -- 172.21.15.32:0/646542107 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff2c80eb470 con 0x7ff2e00af5e0 2023-12-20T06:03:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.279+0000 7ff2e5598640 1 -- 172.21.15.32:0/646542107 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff2e015a020 con 0x7ff2e00af5e0 2023-12-20T06:03:27.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.279+0000 7ff2d77fe640 1 -- 172.21.15.32:0/646542107 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 112 v112) v1 ==== 95+0+4718 (secure 0 0 0) 0x7ff2c80eea90 con 0x7ff2e00af5e0 2023-12-20T06:03:27.279 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:27.279 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":112,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":11298,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/60134876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":60134876},{"type":"v1","addr":"172.21.15.32:6836","nonce":60134876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:21.951240+0000","modified":"2023-12-20T06:03:24.980742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11301":{"gid":11301,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1623322556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1623322556},{"type":"v1","addr":"172.21.15.32:6837","nonce":1623322556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14}]} 2023-12-20T06:03:27.279 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 112 2023-12-20T06:03:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.279+0000 7ff2e5598640 1 -- 172.21.15.32:0/646542107 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff2c0065ca0 msgr2=0x7ff2c0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.279+0000 7ff2e5598640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff2c0065ca0 0x7ff2c0068160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7ff2d00017e0 tx=0x7ff2d0009040 comp rx=0 tx=0).stop 2023-12-20T06:03:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.279+0000 7ff2e5598640 1 -- 172.21.15.32:0/646542107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2e00af5e0 msgr2=0x7ff2e00c5830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.279+0000 7ff2e5598640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2e00af5e0 0x7ff2e00c5830 secure :-1 s=READY pgs=2256 cs=0 l=1 rev1=1 crypto rx=0x7ff2c801a0e0 tx=0x7ff2c801acf0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.279+0000 7ff2e5598640 1 -- 172.21.15.32:0/646542107 shutdown_connections 2023-12-20T06:03:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.279+0000 7ff2e5598640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff2c0065ca0 0x7ff2c0068160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.283+0000 7ff2e5598640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff2e015f110 0x7ff2e00c74f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.283+0000 7ff2e5598640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff2e00af5e0 0x7ff2e00c5830 unknown :-1 s=CLOSED pgs=2256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.283+0000 7ff2e5598640 1 --2- 172.21.15.32:0/646542107 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff2e006ce60 0x7ff2e015a370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.283+0000 7ff2e5598640 1 -- 172.21.15.32:0/646542107 >> 172.21.15.32:0/646542107 conn(0x7ff2e006a320 msgr2=0x7ff2e00cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.283+0000 7ff2e5598640 1 -- 172.21.15.32:0/646542107 shutdown_connections 2023-12-20T06:03:27.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.283+0000 7ff2e5598640 1 -- 172.21.15.32:0/646542107 wait complete. 2023-12-20T06:03:27.703 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 112, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T06:03:21.951240+0000', 'modified': '2023-12-20T06:03:24.980742+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11301}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11301': {'gid': 11301, 'name': 'a', 'rank': 0, 'incarnation': 108, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/1623322556', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 1623322556}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 1623322556}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [29], 'metadata_pool': 28, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T06:03:27.703 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T06:03:27.703 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-20T06:03:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe159fbe640 1 Processor -- start 2023-12-20T06:03:27.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe159fbe640 1 -- start start 2023-12-20T06:03:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe159fbe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe154161500 0x7fe154153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe159fbe640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1540cd4d0 con 0x7fe15406b630 2023-12-20T06:03:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe159fbe640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1540ce4e0 con 0x7fe1540b8480 2023-12-20T06:03:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe159fbe640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1540cd720 con 0x7fe154161500 2023-12-20T06:03:27.851 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe1597bd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe154161500 0x7fe154153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe153fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe15406b630 0x7fe154160df0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:37322/0 (socket says 172.21.15.32:37322) 2023-12-20T06:03:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe153fff640 1 -- 172.21.15.32:0/2319818881 learned_addr learned my addr 172.21.15.32:0/2319818881 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe1597bd640 1 -- 172.21.15.32:0/2319818881 >> v1:172.21.15.39:6789/0 conn(0x7fe1540b8480 legacy=0x7fe15406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe1597bd640 1 -- 172.21.15.32:0/2319818881 >> v1:172.21.15.32:6789/0 conn(0x7fe15406b630 legacy=0x7fe154160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe1597bd640 1 -- 172.21.15.32:0/2319818881 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1540cda00 con 0x7fe154161500 2023-12-20T06:03:27.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.851+0000 7fe1537fe640 1 -- 172.21.15.32:0/2319818881 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2887487519 0 0) 0x7fe1540cd4d0 con 0x7fe15406b630 2023-12-20T06:03:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.855+0000 7fe1597bd640 1 --2- 172.21.15.32:0/2319818881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe154161500 0x7fe154153610 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009d90 tx=0x7fe14c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b97c73b24d30948d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.855+0000 7fe1537fe640 1 -- 172.21.15.32:0/2319818881 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0a57f0 con 0x7fe154161500 2023-12-20T06:03:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.855+0000 7fe1537fe640 1 -- 172.21.15.32:0/2319818881 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c0a5950 con 0x7fe154161500 2023-12-20T06:03:27.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.855+0000 7fe1537fe640 1 -- 172.21.15.32:0/2319818881 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0a5ce0 con 0x7fe154161500 2023-12-20T06:03:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.855+0000 7fe159fbe640 1 -- 172.21.15.32:0/2319818881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe154161500 msgr2=0x7fe154153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.855+0000 7fe159fbe640 1 --2- 172.21.15.32:0/2319818881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe154161500 0x7fe154153610 secure :-1 s=READY pgs=2258 cs=0 l=1 rev1=1 crypto rx=0x7fe14c009d90 tx=0x7fe14c01c950 comp rx=0 tx=0).stop 2023-12-20T06:03:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.855+0000 7fe159fbe640 1 -- 172.21.15.32:0/2319818881 shutdown_connections 2023-12-20T06:03:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.855+0000 7fe159fbe640 1 --2- 172.21.15.32:0/2319818881 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe154161500 0x7fe154153610 unknown :-1 s=CLOSED pgs=2258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.855+0000 7fe159fbe640 1 -- 172.21.15.32:0/2319818881 >> 172.21.15.32:0/2319818881 conn(0x7fe1540d2a50 msgr2=0x7fe1540d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.855+0000 7fe159fbe640 1 -- 172.21.15.32:0/2319818881 shutdown_connections 2023-12-20T06:03:27.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.855+0000 7fe159fbe640 1 -- 172.21.15.32:0/2319818881 wait complete. 2023-12-20T06:03:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe159fbe640 1 Processor -- start 2023-12-20T06:03:27.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe159fbe640 1 -- start start 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe159fbe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15406b630 0x7fe1541f9260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe159fbe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1540b8480 0x7fe1541f97a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe159fbe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe154161500 0x7fe154150cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe159fbe640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540b74f0 con 0x7fe1540b8480 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe159fbe640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540b7a20 con 0x7fe15406b630 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe159fbe640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1540b7740 con 0x7fe154161500 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe153fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1540b8480 0x7fe1541f97a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe153fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1540b8480 0x7fe1541f97a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33470/0 (socket says 172.21.15.32:33470) 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe153fff640 1 -- 172.21.15.32:0/931260447 learned_addr learned my addr 172.21.15.32:0/931260447 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe1597bd640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe154161500 0x7fe154150cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe158fbc640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15406b630 0x7fe1541f9260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe153fff640 1 -- 172.21.15.32:0/931260447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe154161500 msgr2=0x7fe154150cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe153fff640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe154161500 0x7fe154150cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe153fff640 1 -- 172.21.15.32:0/931260447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15406b630 msgr2=0x7fe1541f9260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe153fff640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15406b630 0x7fe1541f9260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe153fff640 1 -- 172.21.15.32:0/931260447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe14c009a70 con 0x7fe1540b8480 2023-12-20T06:03:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe1597bd640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe154161500 0x7fe154150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe158fbc640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15406b630 0x7fe1541f9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe153fff640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1540b8480 0x7fe1541f97a0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fe14000c4d0 tx=0x7fe14000c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe1517fa640 1 -- 172.21.15.32:0/931260447 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1400074e0 con 0x7fe1540b8480 2023-12-20T06:03:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe1517fa640 1 -- 172.21.15.32:0/931260447 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe140007640 con 0x7fe1540b8480 2023-12-20T06:03:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe1517fa640 1 -- 172.21.15.32:0/931260447 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe140004a30 con 0x7fe1540b8480 2023-12-20T06:03:27.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe159fbe640 1 -- 172.21.15.32:0/931260447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1541504d0 con 0x7fe1540b8480 2023-12-20T06:03:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.859+0000 7fe159fbe640 1 -- 172.21.15.32:0/931260447 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1540c9310 con 0x7fe1540b8480 2023-12-20T06:03:27.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe1517fa640 1 -- 172.21.15.32:0/931260447 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fe140004b90 con 0x7fe1540b8480 2023-12-20T06:03:27.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe1517fa640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe138065ca0 0x7fe138068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe1517fa640 1 -- 172.21.15.32:0/931260447 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe140082870 con 0x7fe1540b8480 2023-12-20T06:03:27.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe158fbc640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe138065ca0 0x7fe138068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe15e582000 1 Processor -- start 2023-12-20T06:03:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe15e582000 1 -- start start 2023-12-20T06:03:27.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe158fbc640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe138065ca0 0x7fe138068160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe148000f70 tx=0x7fe148009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:27.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe15e582000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601070 0x55d622601450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe15e582000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d622601990 0x55d6225b6c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe15e582000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6225b71c0 0x55d6225b75a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe15e582000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6223dac90 con 0x55d6225b71c0 2023-12-20T06:03:27.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.863+0000 7fe15e582000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d62240c220 con 0x55d622601990 2023-12-20T06:03:27.865 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6224097f0 con 0x55d622601070 2023-12-20T06:03:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe1597bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6225b71c0 0x55d6225b75a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe1597bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6225b71c0 0x55d6225b75a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33482/0 (socket says 172.21.15.32:33482) 2023-12-20T06:03:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe1597bd640 1 -- 172.21.15.32:0/838850332 learned_addr learned my addr 172.21.15.32:0/838850332 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe153fff640 1 --2- 172.21.15.32:0/838850332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d622601990 0x55d6225b6c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe158fbc640 1 --2- 172.21.15.32:0/838850332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601070 0x55d622601450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe1597bd640 1 -- 172.21.15.32:0/838850332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601070 msgr2=0x55d622601450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe1597bd640 1 --2- 172.21.15.32:0/838850332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601070 0x55d622601450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe1597bd640 1 -- 172.21.15.32:0/838850332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d622601990 msgr2=0x55d6225b6c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe1597bd640 1 --2- 172.21.15.32:0/838850332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d622601990 0x55d6225b6c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe1597bd640 1 -- 172.21.15.32:0/838850332 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d622101890 con 0x55d6225b71c0 2023-12-20T06:03:27.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe1597bd640 1 --2- 172.21.15.32:0/838850332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6225b71c0 0x55d6225b75a0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fe14c019560 tx=0x7fe14c004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8a7f7bbd456a18a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe13e7fc640 1 -- 172.21.15.32:0/838850332 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0a5d40 con 0x55d6225b71c0 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe13e7fc640 1 -- 172.21.15.32:0/838850332 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe14c0039a0 con 0x55d6225b71c0 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe13e7fc640 1 -- 172.21.15.32:0/838850332 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c0a5ea0 con 0x55d6225b71c0 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 -- 172.21.15.32:0/838850332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6225b71c0 msgr2=0x55d6225b75a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 --2- 172.21.15.32:0/838850332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6225b71c0 0x55d6225b75a0 secure :-1 s=READY pgs=2260 cs=0 l=1 rev1=1 crypto rx=0x7fe14c019560 tx=0x7fe14c004ac0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 -- 172.21.15.32:0/838850332 shutdown_connections 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 --2- 172.21.15.32:0/838850332 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d6225b71c0 0x55d6225b75a0 unknown :-1 s=CLOSED pgs=2260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 --2- 172.21.15.32:0/838850332 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d622601990 0x55d6225b6c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 --2- 172.21.15.32:0/838850332 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601070 0x55d622601450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 -- 172.21.15.32:0/838850332 >> 172.21.15.32:0/838850332 conn(0x55d6225bd950 msgr2=0x55d6225b5560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 -- 172.21.15.32:0/838850332 shutdown_connections 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 -- 172.21.15.32:0/838850332 wait complete. 2023-12-20T06:03:27.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 -- start start 2023-12-20T06:03:27.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.867+0000 7fe15e582000 1 Processor -- start 2023-12-20T06:03:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6225b71c0 0x55d6225bc720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d6225c3080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601990 0x55d6225c35c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6224050e0 con 0x55d622601070 2023-12-20T06:03:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6223a4310 con 0x55d6225b71c0 2023-12-20T06:03:27.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6223c2770 con 0x55d622601990 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 10 client.0 _ll_register_callbacks cb 0x55d6225a2b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe153fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d6225c3080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe153fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d6225c3080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33494/0 (socket says 172.21.15.32:33494) 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe153fff640 1 -- 172.21.15.32:0/3877917556 learned_addr learned my addr 172.21.15.32:0/3877917556 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe1597bd640 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601990 0x55d6225c35c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe158fbc640 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6225b71c0 0x55d6225bc720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe153fff640 1 -- 172.21.15.32:0/3877917556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601990 msgr2=0x55d6225c35c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe153fff640 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601990 0x55d6225c35c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe153fff640 1 -- 172.21.15.32:0/3877917556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6225b71c0 msgr2=0x55d6225bc720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe153fff640 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6225b71c0 0x55d6225bc720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe153fff640 1 -- 172.21.15.32:0/3877917556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1540cda00 con 0x55d622601070 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe153fff640 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d6225c3080 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fe140053840 tx=0x7fe140004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=827938563ee09c3c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:27.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe1337fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe1337fe640 1 -- 172.21.15.32:0/3877917556 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14005b890 con 0x55d622601070 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe1337fe640 1 -- 172.21.15.32:0/3877917556 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe14005bb20 con 0x55d622601070 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 -- 172.21.15.32:0/3877917556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6223d4c10 con 0x55d622601070 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe1337fe640 1 -- 172.21.15.32:0/3877917556 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe140080b90 con 0x55d622601070 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 10 client.11451 fetch_fsmap learned FSMap version 112 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 -- 172.21.15.32:0/3877917556 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55d6223ca260 con 0x55d622601070 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe1337fe640 1 -- 172.21.15.32:0/3877917556 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d6223d4c10 con 0x55d622601070 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe1337fe640 1 -- 172.21.15.32:0/3877917556 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe140080cf0 con 0x55d622601070 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 10 client.11451 fetch_fsmap finished waiting for FSMap version 112 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 10 client.11451 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 20 client.11451 populate_metadata read hostname 'smithi032' 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.32:6835/1623322556,v1:172.21.15.32:6837/1623322556] conn(0x55d6226088d0 0x55d622608cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 4 client.11451 mds_command: new command op to 11301 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 -- 172.21.15.32:0/3877917556 --> [v2:172.21.15.32:6835/1623322556,v1:172.21.15.32:6837/1623322556] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x55d622403a30 con 0x55d6226088d0 2023-12-20T06:03:27.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe158fbc640 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.32:6835/1623322556,v1:172.21.15.32:6837/1623322556] conn(0x55d6226088d0 0x55d622608cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe158fbc640 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.32:6835/1623322556,v1:172.21.15.32:6837/1623322556] conn(0x55d6226088d0 0x55d622608cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fb8a311fed196767 server_cookie=99df09c01c78bc86 in_seq=0 out_seq=0 2023-12-20T06:03:27.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe1337fe640 10 client.11451 ms_handle_connect on v2:172.21.15.32:6835/1623322556 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe1337fe640 1 -- 172.21.15.32:0/3877917556 <== mds.0 v2:172.21.15.32:6835/1623322556 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x55d622403a30 con 0x55d6226088d0 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe1337fe640 10 client.11451 handle_command_reply: tid=0 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 client.11451 shutdown 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 -- 172.21.15.32:0/3877917556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 msgr2=0x55d6225c3080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d6225c3080 secure :-1 s=READY pgs=2261 cs=0 l=1 rev1=1 crypto rx=0x7fe140053840 tx=0x7fe140004f80 comp rx=0 tx=0).stop 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 -- 172.21.15.32:0/3877917556 shutdown_connections 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601990 0x55d6225c35c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.32:6835/1623322556,v1:172.21.15.32:6837/1623322556] conn(0x55d6226088d0 0x55d622608cb0 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d6225c3080 unknown :-1 s=CLOSED pgs=2261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 --2- 172.21.15.32:0/3877917556 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6225b71c0 0x55d6225bc720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe1337fe640 0 client.11451 ms_handle_reset on v2:172.21.15.32:6835/1623322556 2023-12-20T06:03:27.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 -- 172.21.15.32:0/3877917556 >> 172.21.15.32:0/3877917556 conn(0x55d6225bd950 msgr2=0x55d6225b5c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.871+0000 7fe15e582000 1 -- 172.21.15.32:0/3877917556 shutdown_connections 2023-12-20T06:03:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.875+0000 7fe15e582000 1 -- 172.21.15.32:0/3877917556 wait complete. 2023-12-20T06:03:27.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.875+0000 7fe15e582000 20 client.11451 trim_cache size 0 max 16384 2023-12-20T06:03:27.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.887+0000 7fe15e582000 1 Processor -- start 2023-12-20T06:03:27.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.887+0000 7fe15e582000 1 -- start start 2023-12-20T06:03:27.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.887+0000 7fe15e582000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6225b71c0 0x55d6226592f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.887+0000 7fe15e582000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d622659830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.887+0000 7fe15e582000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601990 0x55d622659d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.887+0000 7fe15e582000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6220c2940 con 0x55d622601070 2023-12-20T06:03:27.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.887+0000 7fe15e582000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d622407c40 con 0x55d6225b71c0 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.887+0000 7fe15e582000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6224090e0 con 0x55d622601990 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe153fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d622659830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe153fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d622659830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33508/0 (socket says 172.21.15.32:33508) 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe153fff640 1 -- 172.21.15.32:0/3487447553 learned_addr learned my addr 172.21.15.32:0/3487447553 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe1597bd640 1 --2- 172.21.15.32:0/3487447553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601990 0x55d622659d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe158fbc640 1 --2- 172.21.15.32:0/3487447553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6225b71c0 0x55d6226592f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe153fff640 1 -- 172.21.15.32:0/3487447553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601990 msgr2=0x55d622659d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe153fff640 1 --2- 172.21.15.32:0/3487447553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601990 0x55d622659d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe153fff640 1 -- 172.21.15.32:0/3487447553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6225b71c0 msgr2=0x55d6226592f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe153fff640 1 --2- 172.21.15.32:0/3487447553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6225b71c0 0x55d6226592f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe153fff640 1 -- 172.21.15.32:0/3487447553 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x55d6225bd3a0 con 0x55d622601070 2023-12-20T06:03:27.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe153fff640 1 --2- 172.21.15.32:0/3487447553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d622659830 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fe14007ab50 tx=0x7fe14006a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e3773777c22750 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:27.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe13effd640 1 -- 172.21.15.32:0/3487447553 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14005b890 con 0x55d622601070 2023-12-20T06:03:27.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe13effd640 1 -- 172.21.15.32:0/3487447553 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe140053bc0 con 0x55d622601070 2023-12-20T06:03:27.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe13effd640 1 -- 172.21.15.32:0/3487447553 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe140080d90 con 0x55d622601070 2023-12-20T06:03:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 -- 172.21.15.32:0/3487447553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 msgr2=0x55d622659830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 --2- 172.21.15.32:0/3487447553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d622659830 secure :-1 s=READY pgs=2262 cs=0 l=1 rev1=1 crypto rx=0x7fe14007ab50 tx=0x7fe14006a000 comp rx=0 tx=0).stop 2023-12-20T06:03:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 -- 172.21.15.32:0/3487447553 shutdown_connections 2023-12-20T06:03:27.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 --2- 172.21.15.32:0/3487447553 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d622601990 0x55d622659d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 --2- 172.21.15.32:0/3487447553 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601070 0x55d622659830 unknown :-1 s=CLOSED pgs=2262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 --2- 172.21.15.32:0/3487447553 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d6225b71c0 0x55d6226592f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 -- 172.21.15.32:0/3487447553 >> 172.21.15.32:0/3487447553 conn(0x55d6225bd950 msgr2=0x55d6225c2f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 -- 172.21.15.32:0/3487447553 shutdown_connections 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 -- 172.21.15.32:0/3487447553 wait complete. 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 -- start start 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 Processor -- start 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d6225b71c0 0x55d62265e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d622601070 0x55d62265ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601990 0x55d622641f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d6224d5b30 con 0x55d622601990 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x55d622406670 con 0x55d622601070 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x55d6223ad860 con 0x55d6225b71c0 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe15e582000 10 client.0 _ll_register_callbacks cb 0x55d6223b3640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:03:27.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe1597bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601990 0x55d622641f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe1597bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601990 0x55d622641f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33518/0 (socket says 172.21.15.32:33518) 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe1597bd640 1 -- 172.21.15.32:0/1408812098 learned_addr learned my addr 172.21.15.32:0/1408812098 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe153fff640 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d622601070 0x55d62265ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.891+0000 7fe158fbc640 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d6225b71c0 0x55d62265e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1597bd640 1 -- 172.21.15.32:0/1408812098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d6225b71c0 msgr2=0x55d62265e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1597bd640 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d6225b71c0 0x55d62265e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1597bd640 1 -- 172.21.15.32:0/1408812098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d622601070 msgr2=0x55d62265ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1597bd640 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d622601070 0x55d62265ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1597bd640 1 -- 172.21.15.32:0/1408812098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x55d6223ca260 con 0x55d622601990 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe158fbc640 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d6225b71c0 0x55d62265e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1597bd640 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601990 0x55d622641f70 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fe14c019560 tx=0x7fe14c004a50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a32800daa21f3a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1337fe640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1337fe640 1 -- 172.21.15.32:0/1408812098 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe14c003ba0 con 0x55d622601990 2023-12-20T06:03:27.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1337fe640 1 -- 172.21.15.32:0/1408812098 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe14c0c8070 con 0x55d622601990 2023-12-20T06:03:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 -- 172.21.15.32:0/1408812098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x55d6223cae00 con 0x55d622601990 2023-12-20T06:03:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1337fe640 1 -- 172.21.15.32:0/1408812098 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=112) v2 ==== 24+0+0 (secure 0 0 0) 0x7fe14c003d00 con 0x55d622601990 2023-12-20T06:03:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 10 client.11457 fetch_fsmap learned FSMap version 112 2023-12-20T06:03:27.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 -- 172.21.15.32:0/1408812098 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=112,monmap=2+}) v3 -- 0x55d6223b03b0 con 0x55d622601990 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1337fe640 1 -- 172.21.15.32:0/1408812098 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x55d6223cae00 con 0x55d622601990 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1337fe640 1 -- 172.21.15.32:0/1408812098 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 112) v1 ==== 3114+0+0 (secure 0 0 0) 0x7fe14c0d6070 con 0x55d622601990 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 10 client.11457 fetch_fsmap finished waiting for FSMap version 112 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 10 client.11457 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 20 client.11457 populate_metadata read hostname 'smithi032' 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.32:6835/1623322556,v1:172.21.15.32:6837/1623322556] conn(0x55d6226424b0 0x55d622642890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 4 client.11457 mds_command: new command op to 11301 tid=0[{"prefix": "status"}] 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 -- 172.21.15.32:0/1408812098 --> [v2:172.21.15.32:6835/1623322556,v1:172.21.15.32:6837/1623322556] -- command(tid 0: {"prefix": "status"}) v1 -- 0x55d6223fe160 con 0x55d6226424b0 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe158fbc640 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.32:6835/1623322556,v1:172.21.15.32:6837/1623322556] conn(0x55d6226424b0 0x55d622642890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe158fbc640 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.32:6835/1623322556,v1:172.21.15.32:6837/1623322556] conn(0x55d6226424b0 0x55d622642890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c107f2df5d97a7f3 server_cookie=b7ff0b1ab26ad0bb in_seq=0 out_seq=0 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1337fe640 10 client.11457 ms_handle_connect on v2:172.21.15.32:6835/1623322556 2023-12-20T06:03:27.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1337fe640 1 -- 172.21.15.32:0/1408812098 <== mds.0 v2:172.21.15.32:6835/1623322556 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x55d6223fe160 con 0x55d6226424b0 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1337fe640 10 client.11457 handle_command_reply: tid=0 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 client.11457 shutdown 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 -- 172.21.15.32:0/1408812098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601990 msgr2=0x55d622641f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601990 0x55d622641f70 secure :-1 s=READY pgs=2263 cs=0 l=1 rev1=1 crypto rx=0x7fe14c019560 tx=0x7fe14c004a50 comp rx=0 tx=0).stop 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 -- 172.21.15.32:0/1408812098 shutdown_connections 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.32:6835/1623322556,v1:172.21.15.32:6837/1623322556] conn(0x55d6226424b0 0x55d622642890 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x55d622601990 0x55d622641f70 unknown :-1 s=CLOSED pgs=2263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x55d622601070 0x55d62265ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe1337fe640 0 client.11457 ms_handle_reset on v2:172.21.15.32:6835/1623322556 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 --2- 172.21.15.32:0/1408812098 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x55d6225b71c0 0x55d62265e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 -- 172.21.15.32:0/1408812098 >> 172.21.15.32:0/1408812098 conn(0x55d6225bd950 msgr2=0x55d6225fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 -- 172.21.15.32:0/1408812098 shutdown_connections 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 1 -- 172.21.15.32:0/1408812098 wait complete. 2023-12-20T06:03:27.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe15e582000 20 client.11457 trim_cache size 0 max 16384 2023-12-20T06:03:27.900 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:03:27.900 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T06:03:27.900 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T06:03:27.900 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 11301, 2023-12-20T06:03:27.900 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T06:03:27.900 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T06:03:27.900 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T06:03:27.900 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9444349770000002, 2023-12-20T06:03:27.900 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 112, 2023-12-20T06:03:27.900 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 230, 2023-12-20T06:03:27.900 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 230, 2023-12-20T06:03:27.901 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 10.393686483 2023-12-20T06:03:27.901 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:03:27.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe159fbe640 1 -- 172.21.15.32:0/931260447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe138065ca0 msgr2=0x7fe138068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe159fbe640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe138065ca0 0x7fe138068160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fe148000f70 tx=0x7fe148009040 comp rx=0 tx=0).stop 2023-12-20T06:03:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe159fbe640 1 -- 172.21.15.32:0/931260447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1540b8480 msgr2=0x7fe1541f97a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.895+0000 7fe159fbe640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1540b8480 0x7fe1541f97a0 secure :-1 s=READY pgs=2259 cs=0 l=1 rev1=1 crypto rx=0x7fe14000c4d0 tx=0x7fe14000c990 comp rx=0 tx=0).stop 2023-12-20T06:03:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.899+0000 7fe159fbe640 1 -- 172.21.15.32:0/931260447 shutdown_connections 2023-12-20T06:03:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.899+0000 7fe159fbe640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe138065ca0 0x7fe138068160 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.899+0000 7fe159fbe640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe154161500 0x7fe154150cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.899+0000 7fe159fbe640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1540b8480 0x7fe1541f97a0 unknown :-1 s=CLOSED pgs=2259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.899+0000 7fe159fbe640 1 --2- 172.21.15.32:0/931260447 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15406b630 0x7fe1541f9260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.899+0000 7fe159fbe640 1 -- 172.21.15.32:0/931260447 >> 172.21.15.32:0/931260447 conn(0x7fe1540d2a50 msgr2=0x7fe15406af20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.899+0000 7fe159fbe640 1 -- 172.21.15.32:0/931260447 shutdown_connections 2023-12-20T06:03:27.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:27.899+0000 7fe159fbe640 1 -- 172.21.15.32:0/931260447 wait complete. 2023-12-20T06:03:27.915 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T06:03:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.059+0000 7f83f8f39640 1 Processor -- start 2023-12-20T06:03:28.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.059+0000 7f83f8f39640 1 -- start start 2023-12-20T06:03:28.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.059+0000 7f83f8f39640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f83f40af5e0 0x7f83f40af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:28.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.059+0000 7f83f8f39640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83f406ce60 0x7f83f415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:28.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.059+0000 7f83f8f39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f415f110 0x7f83f415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:28.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.059+0000 7f83f8f39640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f415bbd0 con 0x7f83f406ce60 2023-12-20T06:03:28.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.059+0000 7f83f8f39640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f415beb0 con 0x7f83f40af5e0 2023-12-20T06:03:28.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.059+0000 7f83f8f39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f415e910 con 0x7f83f415f110 2023-12-20T06:03:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.059+0000 7f83ebfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83f406ce60 0x7f83f415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.059+0000 7f83ebfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f83f406ce60 0x7f83f415b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33526/0 (socket says 172.21.15.32:33526) 2023-12-20T06:03:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.059+0000 7f83ebfff640 1 -- 172.21.15.32:0/1821245278 learned_addr learned my addr 172.21.15.32:0/1821245278 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83f259c640 1 --2- 172.21.15.32:0/1821245278 >> v2:172.21.15.39:3300/0 conn(0x7f83f40af5e0 0x7f83f40af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83f2d9d640 1 --2- 172.21.15.32:0/1821245278 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f415f110 0x7f83f415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83ebfff640 1 -- 172.21.15.32:0/1821245278 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f415f110 msgr2=0x7f83f415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83ebfff640 1 --2- 172.21.15.32:0/1821245278 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f415f110 0x7f83f415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83ebfff640 1 -- 172.21.15.32:0/1821245278 >> v2:172.21.15.39:3300/0 conn(0x7f83f40af5e0 msgr2=0x7f83f40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83ebfff640 1 --2- 172.21.15.32:0/1821245278 >> v2:172.21.15.39:3300/0 conn(0x7f83f40af5e0 0x7f83f40af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83ebfff640 1 -- 172.21.15.32:0/1821245278 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f415ebf0 con 0x7f83f406ce60 2023-12-20T06:03:28.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83f2d9d640 1 --2- 172.21.15.32:0/1821245278 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f415f110 0x7f83f415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:28.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83f259c640 1 --2- 172.21.15.32:0/1821245278 >> v2:172.21.15.39:3300/0 conn(0x7f83f40af5e0 0x7f83f40af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83ebfff640 1 --2- 172.21.15.32:0/1821245278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f406ce60 0x7f83f415b280 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f83f406ac10 tx=0x7f83dc01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7db88ac8c65ec24e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83f1d9b640 1 -- 172.21.15.32:0/1821245278 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0ab620 con 0x7f83f406ce60 2023-12-20T06:03:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83f1d9b640 1 -- 172.21.15.32:0/1821245278 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83dc0ab780 con 0x7f83f406ce60 2023-12-20T06:03:28.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.063+0000 7f83f1d9b640 1 -- 172.21.15.32:0/1821245278 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83dc0abb10 con 0x7f83f406ce60 2023-12-20T06:03:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 -- 172.21.15.32:0/1821245278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f406ce60 msgr2=0x7f83f415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 --2- 172.21.15.32:0/1821245278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f406ce60 0x7f83f415b280 secure :-1 s=READY pgs=2264 cs=0 l=1 rev1=1 crypto rx=0x7f83f406ac10 tx=0x7f83dc01c500 comp rx=0 tx=0).stop 2023-12-20T06:03:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 -- 172.21.15.32:0/1821245278 shutdown_connections 2023-12-20T06:03:28.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 --2- 172.21.15.32:0/1821245278 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f415f110 0x7f83f415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 --2- 172.21.15.32:0/1821245278 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f406ce60 0x7f83f415b280 unknown :-1 s=CLOSED pgs=2264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 --2- 172.21.15.32:0/1821245278 >> v2:172.21.15.39:3300/0 conn(0x7f83f40af5e0 0x7f83f40af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 -- 172.21.15.32:0/1821245278 >> 172.21.15.32:0/1821245278 conn(0x7f83f406a320 msgr2=0x7f83f40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:28.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 -- 172.21.15.32:0/1821245278 shutdown_connections 2023-12-20T06:03:28.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 -- 172.21.15.32:0/1821245278 wait complete. 2023-12-20T06:03:28.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 Processor -- start 2023-12-20T06:03:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 -- start start 2023-12-20T06:03:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f406ce60 0x7f83f41544b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40af5e0 0x7f83f40b33a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f415f110 0x7f83f4157790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40b4100 con 0x7f83f406ce60 2023-12-20T06:03:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f40b3e20 con 0x7f83f40af5e0 2023-12-20T06:03:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f8f39640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f4159c70 con 0x7f83f415f110 2023-12-20T06:03:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f406ce60 0x7f83f41544b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f406ce60 0x7f83f41544b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:33534/0 (socket says 172.21.15.32:33534) 2023-12-20T06:03:28.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f259c640 1 -- 172.21.15.32:0/176836175 learned_addr learned my addr 172.21.15.32:0/176836175 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f259c640 1 -- 172.21.15.32:0/176836175 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f415f110 msgr2=0x7f83f4157790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f259c640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f415f110 0x7f83f4157790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f259c640 1 -- 172.21.15.32:0/176836175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40af5e0 msgr2=0x7f83f40b33a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83ebfff640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40af5e0 0x7f83f40b33a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f259c640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40af5e0 0x7f83f40b33a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83f259c640 1 -- 172.21.15.32:0/176836175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83dc0094d0 con 0x7f83f406ce60 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.067+0000 7f83ebfff640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40af5e0 0x7f83f40b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83f259c640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f406ce60 0x7f83f41544b0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f83e400e7e0 tx=0x7f83e400eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83eaffd640 1 -- 172.21.15.32:0/176836175 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e40059a0 con 0x7f83f406ce60 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83eaffd640 1 -- 172.21.15.32:0/176836175 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e4005b00 con 0x7f83f406ce60 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83eaffd640 1 -- 172.21.15.32:0/176836175 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e4005e90 con 0x7f83f406ce60 2023-12-20T06:03:28.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83f8f39640 1 -- 172.21.15.32:0/176836175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83f4159f50 con 0x7f83f406ce60 2023-12-20T06:03:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83f8f39640 1 -- 172.21.15.32:0/176836175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f4154f70 con 0x7f83f406ce60 2023-12-20T06:03:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83f8f39640 1 -- 172.21.15.32:0/176836175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f404e000 con 0x7f83f406ce60 2023-12-20T06:03:28.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83eaffd640 1 -- 172.21.15.32:0/176836175 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f83e4004030 con 0x7f83f406ce60 2023-12-20T06:03:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83eaffd640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d4065d20 0x7f83d40681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83eaffd640 1 -- 172.21.15.32:0/176836175 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f83e4083460 con 0x7f83f406ce60 2023-12-20T06:03:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83ebfff640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d4065d20 0x7f83d40681e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:28.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.071+0000 7f83ebfff640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d4065d20 0x7f83d40681e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0af000 tx=0x7f83dc008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:28.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.075+0000 7f83eaffd640 1 -- 172.21.15.32:0/176836175 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f83e4050ae0 con 0x7f83f406ce60 2023-12-20T06:03:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 -- 172.21.15.32:0/176836175 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f83f4164b50 con 0x7f83f406ce60 2023-12-20T06:03:28.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83eaffd640 1 -- 172.21.15.32:0/176836175 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v112) v1 ==== 53+0+73 (secure 0 0 0) 0x7f83e4054100 con 0x7f83f406ce60 2023-12-20T06:03:28.266 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T06:03:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 -- 172.21.15.32:0/176836175 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d4065d20 msgr2=0x7f83d40681e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d4065d20 0x7f83d40681e0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f83dc0af000 tx=0x7f83dc008040 comp rx=0 tx=0).stop 2023-12-20T06:03:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 -- 172.21.15.32:0/176836175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f406ce60 msgr2=0x7f83f41544b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f406ce60 0x7f83f41544b0 secure :-1 s=READY pgs=2265 cs=0 l=1 rev1=1 crypto rx=0x7f83e400e7e0 tx=0x7f83e400eca0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 -- 172.21.15.32:0/176836175 shutdown_connections 2023-12-20T06:03:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d4065d20 0x7f83d40681e0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f415f110 0x7f83f4157790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f40af5e0 0x7f83f40b33a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 --2- 172.21.15.32:0/176836175 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f406ce60 0x7f83f41544b0 unknown :-1 s=CLOSED pgs=2265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:28.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 -- 172.21.15.32:0/176836175 >> 172.21.15.32:0/176836175 conn(0x7f83f406a320 msgr2=0x7f83f4164e30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 -- 172.21.15.32:0/176836175 shutdown_connections 2023-12-20T06:03:28.269 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:28.267+0000 7f83f8f39640 1 -- 172.21.15.32:0/176836175 wait complete. 2023-12-20T06:03:28.286 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T06:03:28.287 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T06:03:28.287 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T06:03:28.287 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T06:03:28.287 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T06:03:28.287 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T06:03:28.287 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T06:03:28.287 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T06:03:28.287 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:03:28.287 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:28.287 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T06:03:28.298 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T06:03:28.298 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T06:03:28.298 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T06:03:28.299 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:03:28.299 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T06:03:28.299 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:03:28.299 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:03:28.299 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T06:03:28.299 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T06:03:28.299 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T06:03:28.299 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:03:28.299 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T06:03:28.299 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T06:03:28.300 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 36652sec preferred_lft 36652sec 2023-12-20T06:03:28.300 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T06:03:28.300 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:03:28.300 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:03:28.300 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T06:03:28.300 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T06:03:28.300 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:03:28.300 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:03:28.300 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T06:03:28.300 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T06:03:28.300 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:03:28.301 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T06:03:28.301 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:03:28.301 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T06:03:28.301 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T06:03:28.301 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T06:03:28.301 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:03:28.301 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T06:03:28.301 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:03:28.301 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T06:03:28.301 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T06:03:28.302 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T06:03:28.302 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:03:28.302 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T06:03:28.302 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T06:03:28.302 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T06:03:28.302 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:03:28.302 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:28.302 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T06:03:28.363 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T06:03:28.366 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:28.366 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T06:03:28.422 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T06:03:28.422 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T06:03:28.422 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T06:03:28.422 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T06:03:28.423 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T06:03:28.423 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T06:03:28.423 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:28.423 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T06:03:28.423 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T06:03:28.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T06:03:28.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T06:03:28.423 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T06:03:28.423 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T06:03:28.521 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:28.521 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T06:03:28.532 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T06:03:28.532 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T06:03:28.532 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:03:28.532 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:28.532 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T06:03:28.597 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T06:03:28.597 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T06:03:28.649 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:03:28.649 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T06:03:28.650 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T06:03:28.701 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T06:03:28.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:28.701 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:03:28.796 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:03:28.796 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T06:03:28.809 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:03:28.809 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T06:03:28.809 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T06:03:28.809 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T06:03:28.809 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T06:03:28.813 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T06:03:28.813 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:28.813 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T06:03:28.878 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:28.878 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T06:03:28.938 INFO:teuthology.orchestra.run.smithi032.stdout:client11053 2023-12-20T06:03:28.938 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:03:28.938 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:03:28.990 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs flag set enable_multiple true --yes-i-really-mean-it 2023-12-20T06:03:29.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.179+0000 7f9902e12640 1 Processor -- start 2023-12-20T06:03:29.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.179+0000 7f9902e12640 1 -- start start 2023-12-20T06:03:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f9902e12640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98fc159af0 0x7f98fc0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f9902e12640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98fc0b0f00 0x7f98fc06d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f9902e12640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98fc06dd50 con 0x7f98fc06d2d0 2023-12-20T06:03:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f9902e12640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98fc0cc940 con 0x7f98fc0b0f00 2023-12-20T06:03:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f9902e12640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc0ccc20 con 0x7f98fc159af0 2023-12-20T06:03:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f99013af640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98fc0b0f00 0x7f98fc06d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f9900bae640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98fc159af0 0x7f98fc0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f99013af640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98fc0b0f00 0x7f98fc06d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:54910/0 (socket says 172.21.15.32:54910) 2023-12-20T06:03:29.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f9900bae640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98fc159af0 0x7f98fc0b1930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:33038/0 (socket says 172.21.15.32:33038) 2023-12-20T06:03:29.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f99013af640 1 -- 172.21.15.32:0/3500914722 learned_addr learned my addr 172.21.15.32:0/3500914722 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:29.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f99013af640 1 -- 172.21.15.32:0/3500914722 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98fc159af0 msgr2=0x7f98fc0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:29.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f99013af640 1 --2- 172.21.15.32:0/3500914722 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98fc159af0 0x7f98fc0b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:29.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f99013af640 1 -- 172.21.15.32:0/3500914722 >> v1:172.21.15.39:6789/0 conn(0x7f98fc06d2d0 legacy=0x7f98fc0b07f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:29.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.183+0000 7f99013af640 1 -- 172.21.15.32:0/3500914722 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98fc0ccf00 con 0x7f98fc0b0f00 2023-12-20T06:03:29.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f99013af640 1 --2- 172.21.15.32:0/3500914722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98fc0b0f00 0x7f98fc06d710 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009e50 tx=0x7f98f401c9e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7714be540965c2ac server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:29.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f98fb7fe640 1 -- 172.21.15.32:0/3500914722 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f40a58d0 con 0x7f98fc0b0f00 2023-12-20T06:03:29.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f98fb7fe640 1 -- 172.21.15.32:0/3500914722 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98f40a5a30 con 0x7f98fc0b0f00 2023-12-20T06:03:29.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f98fb7fe640 1 -- 172.21.15.32:0/3500914722 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98f40a5dc0 con 0x7f98fc0b0f00 2023-12-20T06:03:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 -- 172.21.15.32:0/3500914722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98fc0b0f00 msgr2=0x7f98fc06d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 --2- 172.21.15.32:0/3500914722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98fc0b0f00 0x7f98fc06d710 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f98f4009e50 tx=0x7f98f401c9e0 comp rx=0 tx=0).stop 2023-12-20T06:03:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 -- 172.21.15.32:0/3500914722 shutdown_connections 2023-12-20T06:03:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 --2- 172.21.15.32:0/3500914722 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98fc0b0f00 0x7f98fc06d710 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:29.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 --2- 172.21.15.32:0/3500914722 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98fc159af0 0x7f98fc0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:29.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 -- 172.21.15.32:0/3500914722 >> 172.21.15.32:0/3500914722 conn(0x7f98fc06a510 msgr2=0x7f98fc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:29.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 -- 172.21.15.32:0/3500914722 shutdown_connections 2023-12-20T06:03:29.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 -- 172.21.15.32:0/3500914722 wait complete. 2023-12-20T06:03:29.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 Processor -- start 2023-12-20T06:03:29.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 -- start start 2023-12-20T06:03:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98fc06d2d0 0x7f98fc156100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98fc0b0f00 0x7f98fc14b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98fc159af0 0x7f98fc1632b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:29.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc14b7e0 con 0x7f98fc0b0f00 2023-12-20T06:03:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc155060 con 0x7f98fc06d2d0 2023-12-20T06:03:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.187+0000 7f9902e12640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98fc155340 con 0x7f98fc159af0 2023-12-20T06:03:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98fbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98fc0b0f00 0x7f98fc14b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98fbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98fc0b0f00 0x7f98fc14b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44086/0 (socket says 172.21.15.32:44086) 2023-12-20T06:03:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98fbfff640 1 -- 172.21.15.32:0/4113871439 learned_addr learned my addr 172.21.15.32:0/4113871439 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f9900bae640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98fc06d2d0 0x7f98fc156100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f99013af640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98fc159af0 0x7f98fc1632b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98fbfff640 1 -- 172.21.15.32:0/4113871439 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98fc159af0 msgr2=0x7f98fc1632b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98fbfff640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98fc159af0 0x7f98fc1632b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:29.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98fbfff640 1 -- 172.21.15.32:0/4113871439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98fc06d2d0 msgr2=0x7f98fc156100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98fbfff640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98fc06d2d0 0x7f98fc156100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98fbfff640 1 -- 172.21.15.32:0/4113871439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98f4009b30 con 0x7f98fc0b0f00 2023-12-20T06:03:29.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98fbfff640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98fc0b0f00 0x7f98fc14b2a0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f98e800c500 tx=0x7f98e800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98f97fa640 1 -- 172.21.15.32:0/4113871439 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e8005220 con 0x7f98fc0b0f00 2023-12-20T06:03:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98f97fa640 1 -- 172.21.15.32:0/4113871439 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98e8005380 con 0x7f98fc0b0f00 2023-12-20T06:03:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98f97fa640 1 -- 172.21.15.32:0/4113871439 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98e8007270 con 0x7f98fc0b0f00 2023-12-20T06:03:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f9902e12640 1 -- 172.21.15.32:0/4113871439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f98fc155620 con 0x7f98fc0b0f00 2023-12-20T06:03:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f9902e12640 1 -- 172.21.15.32:0/4113871439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f98fc155be0 con 0x7f98fc0b0f00 2023-12-20T06:03:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f9902e12640 1 -- 172.21.15.32:0/4113871439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f98fc1588c0 con 0x7f98fc0b0f00 2023-12-20T06:03:29.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.191+0000 7f98f97fa640 1 -- 172.21.15.32:0/4113871439 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f98e80073d0 con 0x7f98fc0b0f00 2023-12-20T06:03:29.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.195+0000 7f98f97fa640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f98e0065ca0 0x7f98e0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:29.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.195+0000 7f98f97fa640 1 -- 172.21.15.32:0/4113871439 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f98e8082e30 con 0x7f98fc0b0f00 2023-12-20T06:03:29.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.195+0000 7f9900bae640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f98e0065ca0 0x7f98e0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:29.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.195+0000 7f9900bae640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f98e0065ca0 0x7f98e0068160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009ac0 tx=0x7f98f0009190 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:29.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.195+0000 7f98f97fa640 1 -- 172.21.15.32:0/4113871439 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f98e8050470 con 0x7f98fc0b0f00 2023-12-20T06:03:29.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:29.391+0000 7f9902e12640 1 -- 172.21.15.32:0/4113871439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f98fc04e000 con 0x7f98fc0b0f00 2023-12-20T06:03:30.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.046+0000 7f98f97fa640 1 -- 172.21.15.32:0/4113871439 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs flag set", "flag_name": "enable_multiple", "val": "true", "yes_i_really_mean_it": true}]=0 v113) v1 ==== 136+0+0 (secure 0 0 0) 0x7f98e8053a90 con 0x7f98fc0b0f00 2023-12-20T06:03:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 -- 172.21.15.32:0/4113871439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f98e0065ca0 msgr2=0x7f98e0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f98e0065ca0 0x7f98e0068160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f98f0009ac0 tx=0x7f98f0009190 comp rx=0 tx=0).stop 2023-12-20T06:03:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 -- 172.21.15.32:0/4113871439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98fc0b0f00 msgr2=0x7f98fc14b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98fc0b0f00 0x7f98fc14b2a0 secure :-1 s=READY pgs=2266 cs=0 l=1 rev1=1 crypto rx=0x7f98e800c500 tx=0x7f98e800c9c0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 -- 172.21.15.32:0/4113871439 shutdown_connections 2023-12-20T06:03:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f98e0065ca0 0x7f98e0068160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98fc159af0 0x7f98fc1632b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98fc0b0f00 0x7f98fc14b2a0 unknown :-1 s=CLOSED pgs=2266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 --2- 172.21.15.32:0/4113871439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98fc06d2d0 0x7f98fc156100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 -- 172.21.15.32:0/4113871439 >> 172.21.15.32:0/4113871439 conn(0x7f98fc06a510 msgr2=0x7f98fc0ce7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 -- 172.21.15.32:0/4113871439 shutdown_connections 2023-12-20T06:03:30.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.050+0000 7f9902e12640 1 -- 172.21.15.32:0/4113871439 wait complete. 2023-12-20T06:03:30.070 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.214+0000 7fb8afac4640 1 Processor -- start 2023-12-20T06:03:30.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.214+0000 7fb8afac4640 1 -- start start 2023-12-20T06:03:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8afac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815adc0 0x7fb8a80c5be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8afac4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a80b17f0 con 0x7fb8a80700a0 2023-12-20T06:03:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8afac4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb8a80b1ad0 con 0x7fb8a806cc10 2023-12-20T06:03:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8afac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a806d7b0 con 0x7fb8a815adc0 2023-12-20T06:03:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ad05f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb8a80700a0 0x7fb8a8070500 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51166/0 (socket says 172.21.15.32:51166) 2023-12-20T06:03:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ad05f640 1 -- 172.21.15.32:0/3565909683 learned_addr learned my addr 172.21.15.32:0/3565909683 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ad860640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815adc0 0x7fb8a80c5be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1375652724 0 0) 0x7fb8a80b17f0 con 0x7fb8a80700a0 2023-12-20T06:03:30.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb894003530 con 0x7fb8a80700a0 2023-12-20T06:03:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2628235005 0 0) 0x7fb894003530 con 0x7fb8a80700a0 2023-12-20T06:03:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815adc0 msgr2=0x7fb8a80c5be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815adc0 0x7fb8a80c5be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 >> v1:172.21.15.39:6789/0 conn(0x7fb8a806cc10 legacy=0x7fb8a806d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a80b1db0 con 0x7fb8a80700a0 2023-12-20T06:03:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fb898001680 con 0x7fb8a80700a0 2023-12-20T06:03:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 >> v1:172.21.15.32:6789/0 conn(0x7fb8a80700a0 legacy=0x7fb8a8070500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 0x7fb894003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815adc0 0x7fb8940047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb894004d10 0x7fb894005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a806d7b0 con 0x7fb8940039b0 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb89400b990 con 0x7fb894004d10 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb89400bb50 con 0x7fb8a815adc0 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ad860640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 0x7fb894003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ad05f640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815adc0 0x7fb8940047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ae061640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb894004d10 0x7fb894005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ad860640 1 -- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815adc0 msgr2=0x7fb8940047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ad860640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815adc0 0x7fb8940047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ad860640 1 -- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb894004d10 msgr2=0x7fb894005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ad860640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb894004d10 0x7fb894005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ad860640 1 -- 172.21.15.32:0/3565909683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb894003530 con 0x7fb8940039b0 2023-12-20T06:03:30.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.218+0000 7fb8ae061640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb894004d10 0x7fb894005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8ad860640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 0x7fb894003d90 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009ae0 tx=0x7fb8a401c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=97dba0f6c8dd53b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a40a5580 con 0x7fb8940039b0 2023-12-20T06:03:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a40a56e0 con 0x7fb8940039b0 2023-12-20T06:03:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8ac85e640 1 -- 172.21.15.32:0/3565909683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a40a5a90 con 0x7fb8940039b0 2023-12-20T06:03:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8afac4640 1 -- 172.21.15.32:0/3565909683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 msgr2=0x7fb894003d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8afac4640 1 --2- 172.21.15.32:0/3565909683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 0x7fb894003d90 secure :-1 s=READY pgs=2268 cs=0 l=1 rev1=1 crypto rx=0x7fb8a4009ae0 tx=0x7fb8a401c990 comp rx=0 tx=0).stop 2023-12-20T06:03:30.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8ae061640 1 -- 172.21.15.32:0/3565909683 reap_dead start 2023-12-20T06:03:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8afac4640 1 -- 172.21.15.32:0/3565909683 shutdown_connections 2023-12-20T06:03:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8afac4640 1 -- 172.21.15.32:0/3565909683 >> 172.21.15.32:0/3565909683 conn(0x7fb8a80ce650 msgr2=0x7fb8a80cea60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8afac4640 1 -- 172.21.15.32:0/3565909683 shutdown_connections 2023-12-20T06:03:30.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8afac4640 1 -- 172.21.15.32:0/3565909683 wait complete. 2023-12-20T06:03:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.222+0000 7fb8afac4640 1 Processor -- start 2023-12-20T06:03:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8afac4640 1 -- start start 2023-12-20T06:03:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8afac4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 0x7fb8a8156f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8afac4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815e970 0x7fb8a80c3710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8afac4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8a80cc550 0x7fb8a80cc9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8afac4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80ccf10 con 0x7fb8940039b0 2023-12-20T06:03:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8afac4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a80c4d80 con 0x7fb8a80cc550 2023-12-20T06:03:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8afac4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb8a81574c0 con 0x7fb8a815e970 2023-12-20T06:03:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8ad860640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 0x7fb8a8156f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8ad860640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 0x7fb8a8156f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44100/0 (socket says 172.21.15.32:44100) 2023-12-20T06:03:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8ad860640 1 -- 172.21.15.32:0/1400946495 learned_addr learned my addr 172.21.15.32:0/1400946495 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8ad860640 1 -- 172.21.15.32:0/1400946495 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815e970 msgr2=0x7fb8a80c3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8ad860640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815e970 0x7fb8a80c3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8ad860640 1 -- 172.21.15.32:0/1400946495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8a80cc550 msgr2=0x7fb8a80cc9d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8ad860640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8a80cc550 0x7fb8a80cc9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8ad860640 1 -- 172.21.15.32:0/1400946495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8a815ef00 con 0x7fb8940039b0 2023-12-20T06:03:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8ad860640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 0x7fb8a8156f80 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fb8a401c7c0 tx=0x7fb894005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb89e7fc640 1 -- 172.21.15.32:0/1400946495 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a40bc070 con 0x7fb8940039b0 2023-12-20T06:03:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb89e7fc640 1 -- 172.21.15.32:0/1400946495 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8a4004020 con 0x7fb8940039b0 2023-12-20T06:03:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb89e7fc640 1 -- 172.21.15.32:0/1400946495 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8a4004390 con 0x7fb8940039b0 2023-12-20T06:03:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8afac4640 1 -- 172.21.15.32:0/1400946495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb8a815b450 con 0x7fb8940039b0 2023-12-20T06:03:30.229 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8afac4640 1 -- 172.21.15.32:0/1400946495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb8a815ba10 con 0x7fb8940039b0 2023-12-20T06:03:30.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.226+0000 7fb8afac4640 1 -- 172.21.15.32:0/1400946495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb8a8074100 con 0x7fb8940039b0 2023-12-20T06:03:30.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.230+0000 7fb89e7fc640 1 -- 172.21.15.32:0/1400946495 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb8a40b4040 con 0x7fb8940039b0 2023-12-20T06:03:30.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.230+0000 7fb89e7fc640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb88c065ca0 0x7fb88c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.230+0000 7fb89e7fc640 1 -- 172.21.15.32:0/1400946495 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb8a411cbc0 con 0x7fb8940039b0 2023-12-20T06:03:30.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.230+0000 7fb8ad05f640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb88c065ca0 0x7fb88c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.231 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.230+0000 7fb8ad05f640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb88c065ca0 0x7fb88c068160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80b4490 tx=0x7fb8980014a0 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:30.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.234+0000 7fb89e7fc640 1 -- 172.21.15.32:0/1400946495 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb8a40ea1d0 con 0x7fb8940039b0 2023-12-20T06:03:30.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.390+0000 7fb8afac4640 1 -- 172.21.15.32:0/1400946495 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb8a8158ff0 con 0x7fb8940039b0 2023-12-20T06:03:30.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb89e7fc640 1 -- 172.21.15.32:0/1400946495 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v230) v1 ==== 74+0+16153 (secure 0 0 0) 0x7fb8a40ed7f0 con 0x7fb8940039b0 2023-12-20T06:03:30.394 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:30.394 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":230,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:22.953603+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":29,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:03:20.399088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-20T06:03:20.926367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":228,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 -- 172.21.15.32:0/1400946495 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb88c065ca0 msgr2=0x7fb88c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb88c065ca0 0x7fb88c068160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fb8a80b4490 tx=0x7fb8980014a0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 -- 172.21.15.32:0/1400946495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 msgr2=0x7fb8a8156f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 0x7fb8a8156f80 secure :-1 s=READY pgs=2269 cs=0 l=1 rev1=1 crypto rx=0x7fb8a401c7c0 tx=0x7fb894005800 comp rx=0 tx=0).stop 2023-12-20T06:03:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 -- 172.21.15.32:0/1400946495 shutdown_connections 2023-12-20T06:03:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb88c065ca0 0x7fb88c068160 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb8a80cc550 0x7fb8a80cc9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb8a815e970 0x7fb8a80c3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 --2- 172.21.15.32:0/1400946495 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb8940039b0 0x7fb8a8156f80 unknown :-1 s=CLOSED pgs=2269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 -- 172.21.15.32:0/1400946495 >> 172.21.15.32:0/1400946495 conn(0x7fb8a80ce650 msgr2=0x7fb8a80cd490 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 -- 172.21.15.32:0/1400946495 shutdown_connections 2023-12-20T06:03:30.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.394+0000 7fb8afac4640 1 -- 172.21.15.32:0/1400946495 wait complete. 2023-12-20T06:03:30.415 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T06:03:20.399088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '229', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T06:03:20.926367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '230', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:03:30.415 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:03:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.562+0000 7f1f9eba6640 1 Processor -- start 2023-12-20T06:03:30.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.562+0000 7f1f9eba6640 1 -- start start 2023-12-20T06:03:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.562+0000 7f1f9eba6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1f98157480 0x7f1f980c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.562+0000 7f1f9eba6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f980cd4c0 0x7f1f980b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.562+0000 7f1f9eba6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1f980b1100 con 0x7f1f980b0760 2023-12-20T06:03:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.562+0000 7f1f9eba6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f980cde20 con 0x7f1f980cd4c0 2023-12-20T06:03:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.562+0000 7f1f9eba6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1f980c8e30 con 0x7f1f98157480 2023-12-20T06:03:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f9d143640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f980cd4c0 0x7f1f980b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f9d143640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1f980cd4c0 0x7f1f980b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44112/0 (socket says 172.21.15.32:44112) 2023-12-20T06:03:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f9d143640 1 -- 172.21.15.32:0/3603052663 learned_addr learned my addr 172.21.15.32:0/3603052663 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:30.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f9c942640 1 --2- 172.21.15.32:0/3603052663 >> v2:172.21.15.39:3300/0 conn(0x7f1f98157480 0x7f1f980c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f9d143640 1 -- 172.21.15.32:0/3603052663 >> v2:172.21.15.39:3300/0 conn(0x7f1f98157480 msgr2=0x7f1f980c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f9d143640 1 --2- 172.21.15.32:0/3603052663 >> v2:172.21.15.39:3300/0 conn(0x7f1f98157480 0x7f1f980c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f9d143640 1 -- 172.21.15.32:0/3603052663 >> v1:172.21.15.39:6789/0 conn(0x7f1f980b0760 legacy=0x7f1f9815d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:30.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f9d143640 1 -- 172.21.15.32:0/3603052663 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f980ce070 con 0x7f1f980cd4c0 2023-12-20T06:03:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f9d143640 1 --2- 172.21.15.32:0/3603052663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f980cd4c0 0x7f1f980b0bc0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f1f980cf410 tx=0x7f1f9001c8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=476fd1fe5ec8fd4b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f977fe640 1 -- 172.21.15.32:0/3603052663 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f900a56b0 con 0x7f1f980cd4c0 2023-12-20T06:03:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f977fe640 1 -- 172.21.15.32:0/3603052663 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f900a5810 con 0x7f1f980cd4c0 2023-12-20T06:03:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.566+0000 7f1f977fe640 1 -- 172.21.15.32:0/3603052663 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f900a5ba0 con 0x7f1f980cd4c0 2023-12-20T06:03:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 -- 172.21.15.32:0/3603052663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f980cd4c0 msgr2=0x7f1f980b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 --2- 172.21.15.32:0/3603052663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f980cd4c0 0x7f1f980b0bc0 secure :-1 s=READY pgs=2270 cs=0 l=1 rev1=1 crypto rx=0x7f1f980cf410 tx=0x7f1f9001c8d0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 -- 172.21.15.32:0/3603052663 shutdown_connections 2023-12-20T06:03:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 --2- 172.21.15.32:0/3603052663 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f980cd4c0 0x7f1f980b0bc0 unknown :-1 s=CLOSED pgs=2270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 --2- 172.21.15.32:0/3603052663 >> v2:172.21.15.39:3300/0 conn(0x7f1f98157480 0x7f1f980c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 -- 172.21.15.32:0/3603052663 >> 172.21.15.32:0/3603052663 conn(0x7f1f9806a510 msgr2=0x7f1f980ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 -- 172.21.15.32:0/3603052663 shutdown_connections 2023-12-20T06:03:30.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 -- 172.21.15.32:0/3603052663 wait complete. 2023-12-20T06:03:30.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 Processor -- start 2023-12-20T06:03:30.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 -- start start 2023-12-20T06:03:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f980b0760 0x7f1f9814ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f980cd4c0 0x7f1f98150c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f98157480 0x7f1f98151180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9814ce70 con 0x7f1f98157480 2023-12-20T06:03:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9815acf0 con 0x7f1f980cd4c0 2023-12-20T06:03:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9eba6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1f9815db30 con 0x7f1f980b0760 2023-12-20T06:03:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9d143640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f98157480 0x7f1f98151180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9d143640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f98157480 0x7f1f98151180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44114/0 (socket says 172.21.15.32:44114) 2023-12-20T06:03:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9d143640 1 -- 172.21.15.32:0/476961104 learned_addr learned my addr 172.21.15.32:0/476961104 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9c942640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f980b0760 0x7f1f9814ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f97fff640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f980cd4c0 0x7f1f98150c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9d143640 1 -- 172.21.15.32:0/476961104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f980b0760 msgr2=0x7f1f9814ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9d143640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f980b0760 0x7f1f9814ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9d143640 1 -- 172.21.15.32:0/476961104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f980cd4c0 msgr2=0x7f1f98150c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9d143640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f980cd4c0 0x7f1f98150c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.570+0000 7f1f9d143640 1 -- 172.21.15.32:0/476961104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1f90009b30 con 0x7f1f98157480 2023-12-20T06:03:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.574+0000 7f1f9d143640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f98157480 0x7f1f98151180 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f1f900af040 tx=0x7f1f90019670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.574+0000 7f1f957fa640 1 -- 172.21.15.32:0/476961104 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f900b8670 con 0x7f1f98157480 2023-12-20T06:03:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.574+0000 7f1f957fa640 1 -- 172.21.15.32:0/476961104 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1f900b87f0 con 0x7f1f98157480 2023-12-20T06:03:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.574+0000 7f1f9eba6640 1 -- 172.21.15.32:0/476961104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1f9815d850 con 0x7f1f98157480 2023-12-20T06:03:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.574+0000 7f1f957fa640 1 -- 172.21.15.32:0/476961104 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1f900b8ba0 con 0x7f1f98157480 2023-12-20T06:03:30.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.574+0000 7f1f9eba6640 1 -- 172.21.15.32:0/476961104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1f9814d360 con 0x7f1f98157480 2023-12-20T06:03:30.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.574+0000 7f1f9eba6640 1 -- 172.21.15.32:0/476961104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1f9814d550 con 0x7f1f98157480 2023-12-20T06:03:30.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.574+0000 7f1f957fa640 1 -- 172.21.15.32:0/476961104 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f1f900b8d00 con 0x7f1f98157480 2023-12-20T06:03:30.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.574+0000 7f1f957fa640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f7c065ca0 0x7f1f7c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.578+0000 7f1f957fa640 1 -- 172.21.15.32:0/476961104 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1f9011ea70 con 0x7f1f98157480 2023-12-20T06:03:30.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.578+0000 7f1f9c942640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f7c065ca0 0x7f1f7c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.577 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.578+0000 7f1f9c942640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f7c065ca0 0x7f1f7c068160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c007a60 tx=0x7f1f8c008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:30.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.582+0000 7f1f957fa640 1 -- 172.21.15.32:0/476961104 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1f900ec1c0 con 0x7f1f98157480 2023-12-20T06:03:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.738+0000 7f1f9eba6640 1 -- 172.21.15.32:0/476961104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f1f9804e000 con 0x7f1f98157480 2023-12-20T06:03:30.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.738+0000 7f1f957fa640 1 -- 172.21.15.32:0/476961104 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v230) v1 ==== 93+0+10 (secure 0 0 0) 0x7f1f900bc020 con 0x7f1f98157480 2023-12-20T06:03:30.739 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:03:30.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 -- 172.21.15.32:0/476961104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f7c065ca0 msgr2=0x7f1f7c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f7c065ca0 0x7f1f7c068160 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f1f8c007a60 tx=0x7f1f8c008040 comp rx=0 tx=0).stop 2023-12-20T06:03:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 -- 172.21.15.32:0/476961104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f98157480 msgr2=0x7f1f98151180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f98157480 0x7f1f98151180 secure :-1 s=READY pgs=2271 cs=0 l=1 rev1=1 crypto rx=0x7f1f900af040 tx=0x7f1f90019670 comp rx=0 tx=0).stop 2023-12-20T06:03:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 -- 172.21.15.32:0/476961104 shutdown_connections 2023-12-20T06:03:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f7c065ca0 0x7f1f7c068160 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1f98157480 0x7f1f98151180 unknown :-1 s=CLOSED pgs=2271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1f980cd4c0 0x7f1f98150c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 --2- 172.21.15.32:0/476961104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1f980b0760 0x7f1f9814ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 -- 172.21.15.32:0/476961104 >> 172.21.15.32:0/476961104 conn(0x7f1f9806a510 msgr2=0x7f1f980cb1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:30.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 -- 172.21.15.32:0/476961104 shutdown_connections 2023-12-20T06:03:30.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.742+0000 7f1f9eba6640 1 -- 172.21.15.32:0/476961104 wait complete. 2023-12-20T06:03:30.761 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:03:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.906+0000 7f81751b0640 1 Processor -- start 2023-12-20T06:03:30.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.906+0000 7f81751b0640 1 -- start start 2023-12-20T06:03:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f81751b0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81700cf3e0 0x7f81700cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f81751b0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81700d1580 0x7f81700d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f81751b0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81700cfd80 con 0x7f817015ccf0 2023-12-20T06:03:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f81751b0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81700ce2f0 con 0x7f81700cf3e0 2023-12-20T06:03:30.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f81751b0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81700d0b70 con 0x7f81700d1580 2023-12-20T06:03:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f816f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81700cf3e0 0x7f81700cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f81749af640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81700d1580 0x7f81700d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f816f7fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f81700cf3e0 0x7f81700cf840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:54952/0 (socket says 172.21.15.32:54952) 2023-12-20T06:03:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f816f7fe640 1 -- 172.21.15.32:0/2365658232 learned_addr learned my addr 172.21.15.32:0/2365658232 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f81749af640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81700d1580 0x7f81700d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:33096/0 (socket says 172.21.15.32:33096) 2023-12-20T06:03:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f816ffff640 1 --1- 172.21.15.32:0/2365658232 >> v1:172.21.15.39:6789/0 conn(0x7f817015ccf0 0x7f81700cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:35634/0 (socket says 172.21.15.32:35634) 2023-12-20T06:03:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f816f7fe640 1 -- 172.21.15.32:0/2365658232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81700d1580 msgr2=0x7f81700d1a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f816f7fe640 1 --2- 172.21.15.32:0/2365658232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81700d1580 0x7f81700d1a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f816f7fe640 1 -- 172.21.15.32:0/2365658232 >> v1:172.21.15.39:6789/0 conn(0x7f817015ccf0 legacy=0x7f81700cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.910+0000 7f816f7fe640 1 -- 172.21.15.32:0/2365658232 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81700d10f0 con 0x7f81700cf3e0 2023-12-20T06:03:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f816f7fe640 1 --2- 172.21.15.32:0/2365658232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81700cf3e0 0x7f81700cf840 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8164009870 tx=0x7f816401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b760818a64321ad3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f816effd640 1 -- 172.21.15.32:0/2365658232 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640ab680 con 0x7f81700cf3e0 2023-12-20T06:03:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f816effd640 1 -- 172.21.15.32:0/2365658232 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81640ab7e0 con 0x7f81700cf3e0 2023-12-20T06:03:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f816effd640 1 -- 172.21.15.32:0/2365658232 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640abb70 con 0x7f81700cf3e0 2023-12-20T06:03:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f81751b0640 1 -- 172.21.15.32:0/2365658232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81700cf3e0 msgr2=0x7f81700cf840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f81751b0640 1 --2- 172.21.15.32:0/2365658232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81700cf3e0 0x7f81700cf840 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7f8164009870 tx=0x7f816401c480 comp rx=0 tx=0).stop 2023-12-20T06:03:30.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f81751b0640 1 -- 172.21.15.32:0/2365658232 shutdown_connections 2023-12-20T06:03:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f81751b0640 1 --2- 172.21.15.32:0/2365658232 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81700d1580 0x7f81700d1a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f81751b0640 1 --2- 172.21.15.32:0/2365658232 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f81700cf3e0 0x7f81700cf840 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f81751b0640 1 -- 172.21.15.32:0/2365658232 >> 172.21.15.32:0/2365658232 conn(0x7f81700658e0 msgr2=0x7f8170065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f81751b0640 1 -- 172.21.15.32:0/2365658232 shutdown_connections 2023-12-20T06:03:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f81751b0640 1 -- 172.21.15.32:0/2365658232 wait complete. 2023-12-20T06:03:30.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f81751b0640 1 Processor -- start 2023-12-20T06:03:30.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.914+0000 7f81751b0640 1 -- start start 2023-12-20T06:03:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f81751b0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81700cf3e0 0x7f81701482b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f81751b0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81700d1580 0x7f8170146740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f81751b0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817015ccf0 0x7f8170153f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f81751b0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170154460 con 0x7f81700d1580 2023-12-20T06:03:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f81751b0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8170146d70 con 0x7f817015ccf0 2023-12-20T06:03:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f81751b0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8170148a50 con 0x7f81700cf3e0 2023-12-20T06:03:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81700d1580 0x7f8170146740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81700d1580 0x7f8170146740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44124/0 (socket says 172.21.15.32:44124) 2023-12-20T06:03:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816f7fe640 1 -- 172.21.15.32:0/653813020 learned_addr learned my addr 172.21.15.32:0/653813020 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816ffff640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81700cf3e0 0x7f81701482b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f81749af640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817015ccf0 0x7f8170153f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816f7fe640 1 -- 172.21.15.32:0/653813020 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81700cf3e0 msgr2=0x7f81701482b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816f7fe640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81700cf3e0 0x7f81701482b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816f7fe640 1 -- 172.21.15.32:0/653813020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817015ccf0 msgr2=0x7f8170153f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:30.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816f7fe640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817015ccf0 0x7f8170153f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:30.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816f7fe640 1 -- 172.21.15.32:0/653813020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81640094d0 con 0x7f81700d1580 2023-12-20T06:03:30.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816f7fe640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81700d1580 0x7f8170146740 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f81640ad040 tx=0x7f816401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:30.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816cff9640 1 -- 172.21.15.32:0/653813020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640b4800 con 0x7f81700d1580 2023-12-20T06:03:30.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816cff9640 1 -- 172.21.15.32:0/653813020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f81640b4980 con 0x7f81700d1580 2023-12-20T06:03:30.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816cff9640 1 -- 172.21.15.32:0/653813020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f81640b4d30 con 0x7f81700d1580 2023-12-20T06:03:30.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f81751b0640 1 -- 172.21.15.32:0/653813020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8170154690 con 0x7f81700d1580 2023-12-20T06:03:30.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f81751b0640 1 -- 172.21.15.32:0/653813020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8170154c50 con 0x7f81700d1580 2023-12-20T06:03:30.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f81751b0640 1 -- 172.21.15.32:0/653813020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8170074100 con 0x7f81700d1580 2023-12-20T06:03:30.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.918+0000 7f816cff9640 1 -- 172.21.15.32:0/653813020 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f81640c7440 con 0x7f81700d1580 2023-12-20T06:03:30.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.922+0000 7f816cff9640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f815c065ca0 0x7f815c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:30.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.922+0000 7f816cff9640 1 -- 172.21.15.32:0/653813020 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f81700d2760 con 0x7f81700d1580 2023-12-20T06:03:30.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.922+0000 7f816ffff640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f815c065ca0 0x7f815c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:30.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.922+0000 7f816ffff640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f815c065ca0 0x7f815c068160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8160000f70 tx=0x7f8160007e50 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:30.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:30.926+0000 7f816cff9640 1 -- 172.21.15.32:0/653813020 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f81640ebac0 con 0x7f81700d1580 2023-12-20T06:03:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 -- 172.21.15.32:0/653813020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f81700c5bd0 con 0x7f81700d1580 2023-12-20T06:03:31.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f816cff9640 1 -- 172.21.15.32:0/653813020 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v230) v1 ==== 104+0+11 (secure 0 0 0) 0x7f81640ef0e0 con 0x7f81700d1580 2023-12-20T06:03:31.082 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:03:31.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 -- 172.21.15.32:0/653813020 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f815c065ca0 msgr2=0x7f815c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f815c065ca0 0x7f815c068160 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f8160000f70 tx=0x7f8160007e50 comp rx=0 tx=0).stop 2023-12-20T06:03:31.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 -- 172.21.15.32:0/653813020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81700d1580 msgr2=0x7f8170146740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81700d1580 0x7f8170146740 secure :-1 s=READY pgs=2272 cs=0 l=1 rev1=1 crypto rx=0x7f81640ad040 tx=0x7f816401acc0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 -- 172.21.15.32:0/653813020 shutdown_connections 2023-12-20T06:03:31.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f815c065ca0 0x7f815c068160 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f817015ccf0 0x7f8170153f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f81700d1580 0x7f8170146740 unknown :-1 s=CLOSED pgs=2272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 --2- 172.21.15.32:0/653813020 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81700cf3e0 0x7f81701482b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 -- 172.21.15.32:0/653813020 >> 172.21.15.32:0/653813020 conn(0x7f81700658e0 msgr2=0x7f81700b7bf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:31.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 -- 172.21.15.32:0/653813020 shutdown_connections 2023-12-20T06:03:31.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.082+0000 7f81751b0640 1 -- 172.21.15.32:0/653813020 wait complete. 2023-12-20T06:03:31.104 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:03:31.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ec4ad1640 1 Processor -- start 2023-12-20T06:03:31.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ec4ad1640 1 -- start start 2023-12-20T06:03:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ec4ad1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ec00b0cb0 0x7f2ec00caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ec4ad1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2ec0070230 0x7f2ec00706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ec4ad1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ec015b6a0 con 0x7f2ec015abb0 2023-12-20T06:03:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ec4ad1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ec006e160 con 0x7f2ec0070230 2023-12-20T06:03:31.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ec4ad1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec0070be0 con 0x7f2ec00b0cb0 2023-12-20T06:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebe59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2ec015abb0 0x7f2ec015bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51176/0 (socket says 172.21.15.32:51176) 2023-12-20T06:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebe59c640 1 -- 172.21.15.32:0/804458138 learned_addr learned my addr 172.21.15.32:0/804458138 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebed9d640 1 --2- 172.21.15.32:0/804458138 >> v2:172.21.15.39:3300/0 conn(0x7f2ec0070230 0x7f2ec00706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2eb7fff640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ec00b0cb0 0x7f2ec00caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1500482346 0 0) 0x7f2ec015b6a0 con 0x7f2ec015abb0 2023-12-20T06:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2eac003530 con 0x7f2ec015abb0 2023-12-20T06:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3524638108 0 0) 0x7f2eac003530 con 0x7f2ec015abb0 2023-12-20T06:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ec00b0cb0 msgr2=0x7f2ec00caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ec00b0cb0 0x7f2ec00caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 >> v2:172.21.15.39:3300/0 conn(0x7f2ec0070230 msgr2=0x7f2ec00706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:31.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 --2- 172.21.15.32:0/804458138 >> v2:172.21.15.39:3300/0 conn(0x7f2ec0070230 0x7f2ec00706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec006d8c0 con 0x7f2ec015abb0 2023-12-20T06:03:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f2eb0001680 con 0x7f2ec015abb0 2023-12-20T06:03:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 >> v1:172.21.15.32:6789/0 conn(0x7f2ec015abb0 legacy=0x7f2ec015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2eac003980 0x7f2eac003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ec00b0cb0 0x7f2eac0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eac004d10 0x7f2eac005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec006e160 con 0x7f2eac004d10 2023-12-20T06:03:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec0070be0 con 0x7f2eac003980 2023-12-20T06:03:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2eac00b990 con 0x7f2ec00b0cb0 2023-12-20T06:03:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebed9d640 1 --2- 172.21.15.32:0/804458138 >> v2:172.21.15.39:3300/0 conn(0x7f2ec0070230 0x7f2ec00706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:31.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebed9d640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eac004d10 0x7f2eac005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2ebe59c640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2eac003980 0x7f2eac003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.250+0000 7f2eb7fff640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ec00b0cb0 0x7f2eac0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ebed9d640 1 -- 172.21.15.32:0/804458138 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ec00b0cb0 msgr2=0x7f2eac0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:31.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ebed9d640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ec00b0cb0 0x7f2eac0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ebed9d640 1 -- 172.21.15.32:0/804458138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2eac003980 msgr2=0x7f2eac003da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:31.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ebed9d640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2eac003980 0x7f2eac003da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ebed9d640 1 -- 172.21.15.32:0/804458138 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2eac003530 con 0x7f2eac004d10 2023-12-20T06:03:31.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ebed9d640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eac004d10 0x7f2eac005190 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800e380 tx=0x7f2eb801ce20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a396afd365a6b65 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb80a5e30 con 0x7f2eac004d10 2023-12-20T06:03:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2eb800f040 con 0x7f2eac004d10 2023-12-20T06:03:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ebdd9b640 1 -- 172.21.15.32:0/804458138 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2eb8003ba0 con 0x7f2eac004d10 2023-12-20T06:03:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/804458138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eac004d10 msgr2=0x7f2eac005190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ec4ad1640 1 --2- 172.21.15.32:0/804458138 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2eac004d10 0x7f2eac005190 secure :-1 s=READY pgs=2274 cs=0 l=1 rev1=1 crypto rx=0x7f2eb800e380 tx=0x7f2eb801ce20 comp rx=0 tx=0).stop 2023-12-20T06:03:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ebed9d640 1 -- 172.21.15.32:0/804458138 reap_dead start 2023-12-20T06:03:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/804458138 shutdown_connections 2023-12-20T06:03:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/804458138 >> 172.21.15.32:0/804458138 conn(0x7f2ec006a510 msgr2=0x7f2ec00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/804458138 shutdown_connections 2023-12-20T06:03:31.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.254+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/804458138 wait complete. 2023-12-20T06:03:31.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2ec4ad1640 1 Processor -- start 2023-12-20T06:03:31.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2ec4ad1640 1 -- start start 2023-12-20T06:03:31.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2ec4ad1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2eac004d10 0x7f2ec00c9370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2ec4ad1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ec01452d0 0x7f2ec0158290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2ec4ad1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ec00cb600 0x7f2ec00cba80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2ec4ad1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00cbfc0 con 0x7f2ec01452d0 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2ec4ad1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00c6250 con 0x7f2ec00cb600 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2ec4ad1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ec00c8ee0 con 0x7f2eac004d10 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2eb7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ec01452d0 0x7f2ec0158290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2eb7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ec01452d0 0x7f2ec0158290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44150/0 (socket says 172.21.15.32:44150) 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2eb7fff640 1 -- 172.21.15.32:0/2195317924 learned_addr learned my addr 172.21.15.32:0/2195317924 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2ebed9d640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ec00cb600 0x7f2ec00cba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2ebe59c640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2eac004d10 0x7f2ec00c9370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2eb7fff640 1 -- 172.21.15.32:0/2195317924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2eac004d10 msgr2=0x7f2ec00c9370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2eb7fff640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2eac004d10 0x7f2ec00c9370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2eb7fff640 1 -- 172.21.15.32:0/2195317924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ec00cb600 msgr2=0x7f2ec00cba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2eb7fff640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ec00cb600 0x7f2ec00cba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2eb7fff640 1 -- 172.21.15.32:0/2195317924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ec00c64a0 con 0x7f2ec01452d0 2023-12-20T06:03:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2ebed9d640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ec00cb600 0x7f2ec00cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2eb7fff640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ec01452d0 0x7f2ec0158290 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8011a00 tx=0x7f2ea8011ec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2eb6ffd640 1 -- 172.21.15.32:0/2195317924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea800fb50 con 0x7f2ec01452d0 2023-12-20T06:03:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.258+0000 7f2eb6ffd640 1 -- 172.21.15.32:0/2195317924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2ea800fcb0 con 0x7f2ec01452d0 2023-12-20T06:03:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.262+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/2195317924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ec00c6780 con 0x7f2ec01452d0 2023-12-20T06:03:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.262+0000 7f2eb6ffd640 1 -- 172.21.15.32:0/2195317924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2ea8014440 con 0x7f2ec01452d0 2023-12-20T06:03:31.263 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.262+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/2195317924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ec00c6a60 con 0x7f2ec01452d0 2023-12-20T06:03:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.262+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/2195317924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ec00c9090 con 0x7f2ec01452d0 2023-12-20T06:03:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.262+0000 7f2eb6ffd640 1 -- 172.21.15.32:0/2195317924 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f2ea80145a0 con 0x7f2ec01452d0 2023-12-20T06:03:31.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.262+0000 7f2eb6ffd640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2ea0065ca0 0x7f2ea0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.262+0000 7f2eb6ffd640 1 -- 172.21.15.32:0/2195317924 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f2ea8083fb0 con 0x7f2ec01452d0 2023-12-20T06:03:31.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.262+0000 7f2ebe59c640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2ea0065ca0 0x7f2ea0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.265 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.266+0000 7f2ebe59c640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2ea0065ca0 0x7f2ea0068160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0001060 tx=0x7f2eb0009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:31.268 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.266+0000 7f2eb6ffd640 1 -- 172.21.15.32:0/2195317924 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2ec00c9090 con 0x7f2ec01452d0 2023-12-20T06:03:31.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/2195317924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f2ec004e000 con 0x7f2ec01452d0 2023-12-20T06:03:31.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2eb6ffd640 1 -- 172.21.15.32:0/2195317924 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v230) v1 ==== 100+0+11 (secure 0 0 0) 0x7f2ea80515f0 con 0x7f2ec01452d0 2023-12-20T06:03:31.427 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:03:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/2195317924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2ea0065ca0 msgr2=0x7f2ea0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2ea0065ca0 0x7f2ea0068160 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f2eb0001060 tx=0x7f2eb0009040 comp rx=0 tx=0).stop 2023-12-20T06:03:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/2195317924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ec01452d0 msgr2=0x7f2ec0158290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ec01452d0 0x7f2ec0158290 secure :-1 s=READY pgs=2275 cs=0 l=1 rev1=1 crypto rx=0x7f2ea8011a00 tx=0x7f2ea8011ec0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/2195317924 shutdown_connections 2023-12-20T06:03:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2ea0065ca0 0x7f2ea0068160 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ec00cb600 0x7f2ec00cba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ec01452d0 0x7f2ec0158290 unknown :-1 s=CLOSED pgs=2275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 --2- 172.21.15.32:0/2195317924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2eac004d10 0x7f2ec00c9370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/2195317924 >> 172.21.15.32:0/2195317924 conn(0x7f2ec006a510 msgr2=0x7f2ec01565c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:31.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/2195317924 shutdown_connections 2023-12-20T06:03:31.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.426+0000 7f2ec4ad1640 1 -- 172.21.15.32:0/2195317924 wait complete. 2023-12-20T06:03:31.448 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T06:03:31.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fd0d8e640 1 Processor -- start 2023-12-20T06:03:31.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fd0d8e640 1 -- start start 2023-12-20T06:03:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fd0d8e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5fcc161500 0x7f5fcc153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fd0d8e640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fcc0cd4d0 con 0x7f5fcc0b8480 2023-12-20T06:03:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fd0d8e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fcc0ce4e0 con 0x7f5fcc06b630 2023-12-20T06:03:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fd0d8e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fcc0cd720 con 0x7f5fcc161500 2023-12-20T06:03:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fcb7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5fcc0b8480 0x7f5fcc06af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51190/0 (socket says 172.21.15.32:51190) 2023-12-20T06:03:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fcb7fe640 1 -- 172.21.15.32:0/298307008 learned_addr learned my addr 172.21.15.32:0/298307008 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fcbfff640 1 --2- 172.21.15.32:0/298307008 >> v2:172.21.15.39:3300/0 conn(0x7f5fcc161500 0x7f5fcc153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fca7fc640 1 -- 172.21.15.32:0/298307008 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 817515892 0 0) 0x7f5fcc0cd4d0 con 0x7f5fcc0b8480 2023-12-20T06:03:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fca7fc640 1 -- 172.21.15.32:0/298307008 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5fb4003530 con 0x7f5fcc0b8480 2023-12-20T06:03:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fcbfff640 1 -- 172.21.15.32:0/298307008 >> v1:172.21.15.39:6789/0 conn(0x7f5fcc06b630 legacy=0x7f5fcc160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fcbfff640 1 -- 172.21.15.32:0/298307008 >> v1:172.21.15.32:6789/0 conn(0x7f5fcc0b8480 legacy=0x7f5fcc06af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.594+0000 7f5fcbfff640 1 -- 172.21.15.32:0/298307008 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc0cda00 con 0x7f5fcc161500 2023-12-20T06:03:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.598+0000 7f5fcbfff640 1 --2- 172.21.15.32:0/298307008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc161500 0x7f5fcc153610 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009d90 tx=0x7f5fc401c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=198476a42900d4e1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.598+0000 7f5fca7fc640 1 -- 172.21.15.32:0/298307008 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40a57f0 con 0x7f5fcc161500 2023-12-20T06:03:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.598+0000 7f5fca7fc640 1 -- 172.21.15.32:0/298307008 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fc40a5950 con 0x7f5fcc161500 2023-12-20T06:03:31.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.598+0000 7f5fca7fc640 1 -- 172.21.15.32:0/298307008 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fc40a5ce0 con 0x7f5fcc161500 2023-12-20T06:03:31.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.598+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/298307008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc161500 msgr2=0x7f5fcc153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.598+0000 7f5fd0d8e640 1 --2- 172.21.15.32:0/298307008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc161500 0x7f5fcc153610 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7f5fc4009d90 tx=0x7f5fc401c950 comp rx=0 tx=0).stop 2023-12-20T06:03:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.598+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/298307008 shutdown_connections 2023-12-20T06:03:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.598+0000 7f5fd0d8e640 1 --2- 172.21.15.32:0/298307008 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc161500 0x7f5fcc153610 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.598+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/298307008 >> 172.21.15.32:0/298307008 conn(0x7f5fcc0d2a50 msgr2=0x7f5fcc0d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.598+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/298307008 shutdown_connections 2023-12-20T06:03:31.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.598+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/298307008 wait complete. 2023-12-20T06:03:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fd0d8e640 1 Processor -- start 2023-12-20T06:03:31.601 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fd0d8e640 1 -- start start 2023-12-20T06:03:31.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fd0d8e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc06b630 0x7f5fcc0c9090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fd0d8e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc0b8480 0x7f5fcc0b78f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fd0d8e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc161500 0x7f5fcc15e880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fd0d8e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc14dd30 con 0x7f5fcc06b630 2023-12-20T06:03:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fd0d8e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc0c9ba0 con 0x7f5fcc161500 2023-12-20T06:03:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fd0d8e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc0ccfd0 con 0x7f5fcc0b8480 2023-12-20T06:03:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fcb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc06b630 0x7f5fcc0c9090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fcb7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc06b630 0x7f5fcc0c9090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44156/0 (socket says 172.21.15.32:44156) 2023-12-20T06:03:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fcb7fe640 1 -- 172.21.15.32:0/2631048624 learned_addr learned my addr 172.21.15.32:0/2631048624 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fcbfff640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc161500 0x7f5fcc15e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fcaffd640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc0b8480 0x7f5fcc0b78f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fcb7fe640 1 -- 172.21.15.32:0/2631048624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc0b8480 msgr2=0x7f5fcc0b78f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fcb7fe640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc0b8480 0x7f5fcc0b78f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fcb7fe640 1 -- 172.21.15.32:0/2631048624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc161500 msgr2=0x7f5fcc15e880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fcb7fe640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc161500 0x7f5fcc15e880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fcb7fe640 1 -- 172.21.15.32:0/2631048624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fc4009a70 con 0x7f5fcc06b630 2023-12-20T06:03:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fcb7fe640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc06b630 0x7f5fcc0c9090 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00c500 tx=0x7f5fbc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fb3fff640 1 -- 172.21.15.32:0/2631048624 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc005cf0 con 0x7f5fcc06b630 2023-12-20T06:03:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fb3fff640 1 -- 172.21.15.32:0/2631048624 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fbc005e50 con 0x7f5fcc06b630 2023-12-20T06:03:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fb3fff640 1 -- 172.21.15.32:0/2631048624 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fbc0072f0 con 0x7f5fcc06b630 2023-12-20T06:03:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/2631048624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc0cca10 con 0x7f5fcc06b630 2023-12-20T06:03:31.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/2631048624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fcc154990 con 0x7f5fcc06b630 2023-12-20T06:03:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.602+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/2631048624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fcc154b80 con 0x7f5fcc06b630 2023-12-20T06:03:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.606+0000 7f5fb3fff640 1 -- 172.21.15.32:0/2631048624 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f5fbc007450 con 0x7f5fcc06b630 2023-12-20T06:03:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.606+0000 7f5fb3fff640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fb8065ca0 0x7f5fb8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.606+0000 7f5fb3fff640 1 -- 172.21.15.32:0/2631048624 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5fbc082bf0 con 0x7f5fcc06b630 2023-12-20T06:03:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.606+0000 7f5fcaffd640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fb8065ca0 0x7f5fb8068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.606+0000 7f5fcaffd640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fb8065ca0 0x7f5fb8068160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc152e10 tx=0x7f5fc0009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:31.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.610+0000 7f5fb3fff640 1 -- 172.21.15.32:0/2631048624 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5fbc050230 con 0x7f5fcc06b630 2023-12-20T06:03:31.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/2631048624 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7f5fcc074100 con 0x7f5fcc06b630 2023-12-20T06:03:31.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fb3fff640 1 -- 172.21.15.32:0/2631048624 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v230) v1 ==== 84+0+209 (secure 0 0 0) 0x7f5fbc017090 con 0x7f5fcc06b630 2023-12-20T06:03:31.770 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:31.770 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T06:03:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T06:03:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T06:03:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 28, 2023-12-20T06:03:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "cephfs_metadata" 2023-12-20T06:03:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 29, 2023-12-20T06:03:31.771 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": "cephfs_data" 2023-12-20T06:03:31.772 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:31.772 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T06:03:31.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/2631048624 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fb8065ca0 msgr2=0x7f5fb8068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fb8065ca0 0x7f5fb8068160 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f5fcc152e10 tx=0x7f5fc0009040 comp rx=0 tx=0).stop 2023-12-20T06:03:31.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/2631048624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc06b630 msgr2=0x7f5fcc0c9090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc06b630 0x7f5fcc0c9090 secure :-1 s=READY pgs=2277 cs=0 l=1 rev1=1 crypto rx=0x7f5fbc00c500 tx=0x7f5fbc00c9c0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/2631048624 shutdown_connections 2023-12-20T06:03:31.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fb8065ca0 0x7f5fb8068160 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc161500 0x7f5fcc15e880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc0b8480 0x7f5fcc0b78f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 --2- 172.21.15.32:0/2631048624 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc06b630 0x7f5fcc0c9090 unknown :-1 s=CLOSED pgs=2277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/2631048624 >> 172.21.15.32:0/2631048624 conn(0x7f5fcc0d2a50 msgr2=0x7f5fcc160df0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:31.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/2631048624 shutdown_connections 2023-12-20T06:03:31.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.770+0000 7f5fd0d8e640 1 -- 172.21.15.32:0/2631048624 wait complete. 2023-12-20T06:03:31.792 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'backup_fs' 2023-12-20T06:03:31.793 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_metadata --pg_num_min 64 2023-12-20T06:03:31.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c9798640 1 Processor -- start 2023-12-20T06:03:31.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c9798640 1 -- start start 2023-12-20T06:03:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c9798640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa3c415ce50 0x7fa3c406b640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c9798640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c406bb80 0x7fa3c40cf090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c9798640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa3c40b0550 con 0x7fa3c40cf5d0 2023-12-20T06:03:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c9798640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa3c406bfc0 con 0x7fa3c415ce50 2023-12-20T06:03:31.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c9798640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40aff20 con 0x7fa3c406bb80 2023-12-20T06:03:31.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c2ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa3c415ce50 0x7fa3c406b640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c27fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c406bb80 0x7fa3c40cf090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c27fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c406bb80 0x7fa3c40cf090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:33164/0 (socket says 172.21.15.32:33164) 2023-12-20T06:03:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c27fc640 1 -- 172.21.15.32:0/518963150 learned_addr learned my addr 172.21.15.32:0/518963150 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c2ffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa3c415ce50 0x7fa3c406b640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:55018/0 (socket says 172.21.15.32:55018) 2023-12-20T06:03:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c27fc640 1 -- 172.21.15.32:0/518963150 >> v2:172.21.15.39:3300/0 conn(0x7fa3c415ce50 msgr2=0x7fa3c406b640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c27fc640 1 --2- 172.21.15.32:0/518963150 >> v2:172.21.15.39:3300/0 conn(0x7fa3c415ce50 0x7fa3c406b640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c27fc640 1 -- 172.21.15.32:0/518963150 >> v1:172.21.15.39:6789/0 conn(0x7fa3c40cf5d0 legacy=0x7fa3c4160a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c27fc640 1 -- 172.21.15.32:0/518963150 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3c40cfdc0 con 0x7fa3c406bb80 2023-12-20T06:03:31.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.942+0000 7fa3c2ffd640 1 --2- 172.21.15.32:0/518963150 >> v2:172.21.15.39:3300/0 conn(0x7fa3c415ce50 0x7fa3c406b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c27fc640 1 --2- 172.21.15.32:0/518963150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c406bb80 0x7fa3c40cf090 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009870 tx=0x7fa3b801c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=96c545d087c245f7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c1ffb640 1 -- 172.21.15.32:0/518963150 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80aa670 con 0x7fa3c406bb80 2023-12-20T06:03:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c1ffb640 1 -- 172.21.15.32:0/518963150 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b80aa7d0 con 0x7fa3c406bb80 2023-12-20T06:03:31.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c1ffb640 1 -- 172.21.15.32:0/518963150 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b80aab60 con 0x7fa3c406bb80 2023-12-20T06:03:31.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c9798640 1 -- 172.21.15.32:0/518963150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c406bb80 msgr2=0x7fa3c40cf090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c9798640 1 --2- 172.21.15.32:0/518963150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c406bb80 0x7fa3c40cf090 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8009870 tx=0x7fa3b801c510 comp rx=0 tx=0).stop 2023-12-20T06:03:31.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c9798640 1 -- 172.21.15.32:0/518963150 shutdown_connections 2023-12-20T06:03:31.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c9798640 1 --2- 172.21.15.32:0/518963150 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c406bb80 0x7fa3c40cf090 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c9798640 1 --2- 172.21.15.32:0/518963150 >> v2:172.21.15.39:3300/0 conn(0x7fa3c415ce50 0x7fa3c406b640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c9798640 1 -- 172.21.15.32:0/518963150 >> 172.21.15.32:0/518963150 conn(0x7fa3c4099d70 msgr2=0x7fa3c40d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:31.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c9798640 1 -- 172.21.15.32:0/518963150 shutdown_connections 2023-12-20T06:03:31.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.946+0000 7fa3c9798640 1 -- 172.21.15.32:0/518963150 wait complete. 2023-12-20T06:03:31.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c9798640 1 Processor -- start 2023-12-20T06:03:31.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c9798640 1 -- start start 2023-12-20T06:03:31.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c9798640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c406bb80 0x7fa3c40ae820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c9798640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3c40cf5d0 0x7fa3c40aed60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c9798640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c415ce50 0x7fa3c414b260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c9798640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40cdfe0 con 0x7fa3c406bb80 2023-12-20T06:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c9798640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40cdd00 con 0x7fa3c40cf5d0 2023-12-20T06:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c9798640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa3c40c4770 con 0x7fa3c415ce50 2023-12-20T06:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c406bb80 0x7fa3c40ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c406bb80 0x7fa3c40ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44170/0 (socket says 172.21.15.32:44170) 2023-12-20T06:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c2ffd640 1 -- 172.21.15.32:0/826937434 learned_addr learned my addr 172.21.15.32:0/826937434 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c37fe640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c415ce50 0x7fa3c414b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c27fc640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3c40cf5d0 0x7fa3c40aed60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c2ffd640 1 -- 172.21.15.32:0/826937434 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c415ce50 msgr2=0x7fa3c414b260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c2ffd640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c415ce50 0x7fa3c414b260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c2ffd640 1 -- 172.21.15.32:0/826937434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3c40cf5d0 msgr2=0x7fa3c40aed60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c2ffd640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3c40cf5d0 0x7fa3c40aed60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c2ffd640 1 -- 172.21.15.32:0/826937434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa3b4009420 con 0x7fa3c406bb80 2023-12-20T06:03:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c2ffd640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c406bb80 0x7fa3c40ae820 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400e980 tx=0x7fa3b400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3ab7fe640 1 -- 172.21.15.32:0/826937434 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b400ca00 con 0x7fa3c406bb80 2023-12-20T06:03:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3ab7fe640 1 -- 172.21.15.32:0/826937434 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3b400cb60 con 0x7fa3c406bb80 2023-12-20T06:03:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c9798640 1 -- 172.21.15.32:0/826937434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa3b80094d0 con 0x7fa3c406bb80 2023-12-20T06:03:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3ab7fe640 1 -- 172.21.15.32:0/826937434 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4013470 con 0x7fa3c406bb80 2023-12-20T06:03:31.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.950+0000 7fa3c9798640 1 -- 172.21.15.32:0/826937434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3c40c4d30 con 0x7fa3c406bb80 2023-12-20T06:03:31.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.954+0000 7fa3c9798640 1 -- 172.21.15.32:0/826937434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa3c40c45b0 con 0x7fa3c406bb80 2023-12-20T06:03:31.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.954+0000 7fa3ab7fe640 1 -- 172.21.15.32:0/826937434 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fa3b40135d0 con 0x7fa3c406bb80 2023-12-20T06:03:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.954+0000 7fa3ab7fe640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3b0065ca0 0x7fa3b0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.954+0000 7fa3ab7fe640 1 -- 172.21.15.32:0/826937434 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(230..230 src has 1..230) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa3b40837e0 con 0x7fa3c406bb80 2023-12-20T06:03:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.954+0000 7fa3c27fc640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3b0065ca0 0x7fa3b0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:31.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.954+0000 7fa3c27fc640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3b0065ca0 0x7fa3b0068160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80ae000 tx=0x7fa3b80b5040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:31.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:31.958+0000 7fa3ab7fe640 1 -- 172.21.15.32:0/826937434 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa3b4050ea0 con 0x7fa3c406bb80 2023-12-20T06:03:32.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:32.122+0000 7fa3c9798640 1 -- 172.21.15.32:0/826937434 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7fa3c404e000 con 0x7fa3c406bb80 2023-12-20T06:03:33.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.090+0000 7fa3ab7fe640 1 -- 172.21.15.32:0/826937434 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_metadata", "pg_num_min": 64}]=0 pool 'backup_fs_metadata' created v231) v1 ==== 144+0+0 (secure 0 0 0) 0x7fa3b40544c0 con 0x7fa3c406bb80 2023-12-20T06:03:33.091 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'backup_fs_metadata' created 2023-12-20T06:03:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.094+0000 7fa3c9798640 1 -- 172.21.15.32:0/826937434 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3b0065ca0 msgr2=0x7fa3b0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.094+0000 7fa3c9798640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3b0065ca0 0x7fa3b0068160 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80ae000 tx=0x7fa3b80b5040 comp rx=0 tx=0).stop 2023-12-20T06:03:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.094+0000 7fa3c9798640 1 -- 172.21.15.32:0/826937434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c406bb80 msgr2=0x7fa3c40ae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:33.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.094+0000 7fa3c9798640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c406bb80 0x7fa3c40ae820 secure :-1 s=READY pgs=2278 cs=0 l=1 rev1=1 crypto rx=0x7fa3b400e980 tx=0x7fa3b400ee40 comp rx=0 tx=0).stop 2023-12-20T06:03:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.094+0000 7fa3c9798640 1 -- 172.21.15.32:0/826937434 shutdown_connections 2023-12-20T06:03:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.094+0000 7fa3c9798640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa3b0065ca0 0x7fa3b0068160 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.094+0000 7fa3c9798640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa3c415ce50 0x7fa3c414b260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.094+0000 7fa3c9798640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3c40cf5d0 0x7fa3c40aed60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.094+0000 7fa3c9798640 1 --2- 172.21.15.32:0/826937434 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3c406bb80 0x7fa3c40ae820 unknown :-1 s=CLOSED pgs=2278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:33.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.094+0000 7fa3c9798640 1 -- 172.21.15.32:0/826937434 >> 172.21.15.32:0/826937434 conn(0x7fa3c4099d70 msgr2=0x7fa3c4155880 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:33.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.094+0000 7fa3c9798640 1 -- 172.21.15.32:0/826937434 shutdown_connections 2023-12-20T06:03:33.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.098+0000 7fa3c9798640 1 -- 172.21.15.32:0/826937434 wait complete. 2023-12-20T06:03:33.125 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create backup_fs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T06:03:33.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.278+0000 7fb38b3ce640 1 Processor -- start 2023-12-20T06:03:33.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.278+0000 7fb38b3ce640 1 -- start start 2023-12-20T06:03:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38b3ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3840c1b80 0x7fb3840b15c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38b3ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb38406dac0 0x7fb384069d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38b3ce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb38406d170 con 0x7fb38406a260 2023-12-20T06:03:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38b3ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb38406a6f0 con 0x7fb3840c1b80 2023-12-20T06:03:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38b3ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb38406c5f0 con 0x7fb38406dac0 2023-12-20T06:03:33.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38916a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3840c1b80 0x7fb3840b15c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38916a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb3840c1b80 0x7fb3840b15c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44178/0 (socket says 172.21.15.32:44178) 2023-12-20T06:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38916a640 1 -- 172.21.15.32:0/3854435875 learned_addr learned my addr 172.21.15.32:0/3854435875 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38996b640 1 --1- 172.21.15.32:0/3854435875 >> v1:172.21.15.32:6789/0 conn(0x7fb38406a260 0x7fb384070470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51192/0 (socket says 172.21.15.32:51192) 2023-12-20T06:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb388969640 1 --2- 172.21.15.32:0/3854435875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb38406dac0 0x7fb384069d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38916a640 1 -- 172.21.15.32:0/3854435875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb38406dac0 msgr2=0x7fb384069d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38916a640 1 --2- 172.21.15.32:0/3854435875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb38406dac0 0x7fb384069d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38916a640 1 -- 172.21.15.32:0/3854435875 >> v1:172.21.15.32:6789/0 conn(0x7fb38406a260 legacy=0x7fb384070470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb38916a640 1 -- 172.21.15.32:0/3854435875 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb38406c8d0 con 0x7fb3840c1b80 2023-12-20T06:03:33.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.282+0000 7fb37bfff640 1 -- 172.21.15.32:0/3854435875 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2277740992 0 0) 0x7fb38406d170 con 0x7fb38406a260 2023-12-20T06:03:33.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38916a640 1 --2- 172.21.15.32:0/3854435875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3840c1b80 0x7fb3840b15c0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fb380009870 tx=0x7fb38001c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86aed58b65d27285 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:33.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb37bfff640 1 -- 172.21.15.32:0/3854435875 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3800aa610 con 0x7fb3840c1b80 2023-12-20T06:03:33.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb37bfff640 1 -- 172.21.15.32:0/3854435875 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3800a6040 con 0x7fb3840c1b80 2023-12-20T06:03:33.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb37bfff640 1 -- 172.21.15.32:0/3854435875 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3800aa990 con 0x7fb3840c1b80 2023-12-20T06:03:33.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 -- 172.21.15.32:0/3854435875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3840c1b80 msgr2=0x7fb3840b15c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:33.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 --2- 172.21.15.32:0/3854435875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3840c1b80 0x7fb3840b15c0 secure :-1 s=READY pgs=2280 cs=0 l=1 rev1=1 crypto rx=0x7fb380009870 tx=0x7fb38001c4b0 comp rx=0 tx=0).stop 2023-12-20T06:03:33.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 -- 172.21.15.32:0/3854435875 shutdown_connections 2023-12-20T06:03:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 --2- 172.21.15.32:0/3854435875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb38406dac0 0x7fb384069d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 --2- 172.21.15.32:0/3854435875 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb3840c1b80 0x7fb3840b15c0 unknown :-1 s=CLOSED pgs=2280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 -- 172.21.15.32:0/3854435875 >> 172.21.15.32:0/3854435875 conn(0x7fb3840b0940 msgr2=0x7fb3840ce730 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 -- 172.21.15.32:0/3854435875 shutdown_connections 2023-12-20T06:03:33.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 -- 172.21.15.32:0/3854435875 wait complete. 2023-12-20T06:03:33.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 Processor -- start 2023-12-20T06:03:33.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 -- start start 2023-12-20T06:03:33.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38406a260 0x7fb3840c6800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb38406dac0 0x7fb384158e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3840c1b80 0x7fb3840c8010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb384163f30 con 0x7fb38406a260 2023-12-20T06:03:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb384163320 con 0x7fb38406dac0 2023-12-20T06:03:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38b3ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb384163600 con 0x7fb3840c1b80 2023-12-20T06:03:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.286+0000 7fb38916a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38406a260 0x7fb3840c6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38916a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38406a260 0x7fb3840c6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44192/0 (socket says 172.21.15.32:44192) 2023-12-20T06:03:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38916a640 1 -- 172.21.15.32:0/4027369336 learned_addr learned my addr 172.21.15.32:0/4027369336 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:33.291 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb388969640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb38406dac0 0x7fb384158e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38996b640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3840c1b80 0x7fb3840c8010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38916a640 1 -- 172.21.15.32:0/4027369336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3840c1b80 msgr2=0x7fb3840c8010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38916a640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3840c1b80 0x7fb3840c8010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38916a640 1 -- 172.21.15.32:0/4027369336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb38406dac0 msgr2=0x7fb384158e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38916a640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb38406dac0 0x7fb384158e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38916a640 1 -- 172.21.15.32:0/4027369336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3800094d0 con 0x7fb38406a260 2023-12-20T06:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38996b640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3840c1b80 0x7fb3840c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38916a640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38406a260 0x7fb3840c6800 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fb3800ac040 tx=0x7fb38001acf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb379ffb640 1 -- 172.21.15.32:0/4027369336 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3800bd5f0 con 0x7fb38406a260 2023-12-20T06:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb379ffb640 1 -- 172.21.15.32:0/4027369336 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb3800bd770 con 0x7fb38406a260 2023-12-20T06:03:33.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38b3ce640 1 -- 172.21.15.32:0/4027369336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb3841638e0 con 0x7fb38406a260 2023-12-20T06:03:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb379ffb640 1 -- 172.21.15.32:0/4027369336 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb3800bdb20 con 0x7fb38406a260 2023-12-20T06:03:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38b3ce640 1 -- 172.21.15.32:0/4027369336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb3841593b0 con 0x7fb38406a260 2023-12-20T06:03:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.290+0000 7fb38b3ce640 1 -- 172.21.15.32:0/4027369336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb384074100 con 0x7fb38406a260 2023-12-20T06:03:33.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.294+0000 7fb379ffb640 1 -- 172.21.15.32:0/4027369336 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb3800b33f0 con 0x7fb38406a260 2023-12-20T06:03:33.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.294+0000 7fb379ffb640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb368065cd0 0x7fb368068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:33.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.294+0000 7fb379ffb640 1 -- 172.21.15.32:0/4027369336 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(231..231 src has 1..231) v4 ==== 6138+0+0 (secure 0 0 0) 0x7fb38011e380 con 0x7fb38406a260 2023-12-20T06:03:33.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.294+0000 7fb388969640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb368065cd0 0x7fb368068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:33.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.294+0000 7fb388969640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb368065cd0 0x7fb368068190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb3840c1680 tx=0x7fb374009190 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:33.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.298+0000 7fb379ffb640 1 -- 172.21.15.32:0/4027369336 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb3800eb8d0 con 0x7fb38406a260 2023-12-20T06:03:33.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:33.454+0000 7fb38b3ce640 1 -- 172.21.15.32:0/4027369336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fb384163740 con 0x7fb38406a260 2023-12-20T06:03:34.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.102+0000 7fb379ffb640 1 -- 172.21.15.32:0/4027369336 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "backup_fs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'backup_fs_data' created v232) v1 ==== 176+0+0 (secure 0 0 0) 0x7fb384163740 con 0x7fb38406a260 2023-12-20T06:03:34.104 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'backup_fs_data' created 2023-12-20T06:03:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.114+0000 7fb38b3ce640 1 -- 172.21.15.32:0/4027369336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb368065cd0 msgr2=0x7fb368068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.114+0000 7fb38b3ce640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb368065cd0 0x7fb368068190 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb3840c1680 tx=0x7fb374009190 comp rx=0 tx=0).stop 2023-12-20T06:03:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.114+0000 7fb38b3ce640 1 -- 172.21.15.32:0/4027369336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38406a260 msgr2=0x7fb3840c6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.114+0000 7fb38b3ce640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38406a260 0x7fb3840c6800 secure :-1 s=READY pgs=2281 cs=0 l=1 rev1=1 crypto rx=0x7fb3800ac040 tx=0x7fb38001acf0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.114+0000 7fb38b3ce640 1 -- 172.21.15.32:0/4027369336 shutdown_connections 2023-12-20T06:03:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.114+0000 7fb38b3ce640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb368065cd0 0x7fb368068190 secure :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7fb3840c1680 tx=0x7fb374009190 comp rx=0 tx=0).stop 2023-12-20T06:03:34.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.114+0000 7fb38b3ce640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb3840c1b80 0x7fb3840c8010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.114+0000 7fb38b3ce640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb38406dac0 0x7fb384158e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.114+0000 7fb38b3ce640 1 --2- 172.21.15.32:0/4027369336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb38406a260 0x7fb3840c6800 unknown :-1 s=CLOSED pgs=2281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.114+0000 7fb38b3ce640 1 -- 172.21.15.32:0/4027369336 >> 172.21.15.32:0/4027369336 conn(0x7fb3840b0940 msgr2=0x7fb384070320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:34.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.114+0000 7fb38b3ce640 1 -- 172.21.15.32:0/4027369336 shutdown_connections 2023-12-20T06:03:34.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.118+0000 7fb38b3ce640 1 -- 172.21.15.32:0/4027369336 wait complete. 2023-12-20T06:03:34.149 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new backup_fs backup_fs_metadata backup_fs_data 2023-12-20T06:03:34.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.294+0000 7fbbbea66640 1 Processor -- start 2023-12-20T06:03:34.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.294+0000 7fbbbea66640 1 -- start start 2023-12-20T06:03:34.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbbea66640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbbb815abb0 0x7fbbb80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbbea66640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbb80b0cb0 0x7fbbb815bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbbea66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbbb815b6a0 con 0x7fbbb8070230 2023-12-20T06:03:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbbea66640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb806e160 con 0x7fbbb80b0cb0 2023-12-20T06:03:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbbea66640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbbb815c070 con 0x7fbbb815abb0 2023-12-20T06:03:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbb7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbb80b0cb0 0x7fbbb815bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbb7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbbb80b0cb0 0x7fbbb815bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44208/0 (socket says 172.21.15.32:44208) 2023-12-20T06:03:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbb7fff640 1 -- 172.21.15.32:0/1718718856 learned_addr learned my addr 172.21.15.32:0/1718718856 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbbc802640 1 --2- 172.21.15.32:0/1718718856 >> v2:172.21.15.39:3300/0 conn(0x7fbbb815abb0 0x7fbbb80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbb7fff640 1 -- 172.21.15.32:0/1718718856 >> v2:172.21.15.39:3300/0 conn(0x7fbbb815abb0 msgr2=0x7fbbb80caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbb7fff640 1 --2- 172.21.15.32:0/1718718856 >> v2:172.21.15.39:3300/0 conn(0x7fbbb815abb0 0x7fbbb80caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbb7fff640 1 -- 172.21.15.32:0/1718718856 >> v1:172.21.15.32:6789/0 conn(0x7fbbb8070230 legacy=0x7fbbb806c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.298+0000 7fbbb7fff640 1 -- 172.21.15.32:0/1718718856 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbbb806d8c0 con 0x7fbbb80b0cb0 2023-12-20T06:03:34.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbb7fff640 1 --2- 172.21.15.32:0/1718718856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb80b0cb0 0x7fbbb815bb30 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80713f0 tx=0x7fbba401c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bd5acefc342dd9e9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:34.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbb77fe640 1 -- 172.21.15.32:0/1718718856 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba40aa660 con 0x7fbbb80b0cb0 2023-12-20T06:03:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbb77fe640 1 -- 172.21.15.32:0/1718718856 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbba40aa7c0 con 0x7fbbb80b0cb0 2023-12-20T06:03:34.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbb77fe640 1 -- 172.21.15.32:0/1718718856 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbba40aab50 con 0x7fbbb80b0cb0 2023-12-20T06:03:34.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbbea66640 1 -- 172.21.15.32:0/1718718856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb80b0cb0 msgr2=0x7fbbb815bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbbea66640 1 --2- 172.21.15.32:0/1718718856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb80b0cb0 0x7fbbb815bb30 secure :-1 s=READY pgs=2283 cs=0 l=1 rev1=1 crypto rx=0x7fbbb80713f0 tx=0x7fbba401c500 comp rx=0 tx=0).stop 2023-12-20T06:03:34.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbbea66640 1 -- 172.21.15.32:0/1718718856 shutdown_connections 2023-12-20T06:03:34.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbbea66640 1 --2- 172.21.15.32:0/1718718856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb80b0cb0 0x7fbbb815bb30 unknown :-1 s=CLOSED pgs=2283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbbea66640 1 --2- 172.21.15.32:0/1718718856 >> v2:172.21.15.39:3300/0 conn(0x7fbbb815abb0 0x7fbbb80caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbbea66640 1 -- 172.21.15.32:0/1718718856 >> 172.21.15.32:0/1718718856 conn(0x7fbbb806a510 msgr2=0x7fbbb80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:34.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbbea66640 1 -- 172.21.15.32:0/1718718856 shutdown_connections 2023-12-20T06:03:34.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbbea66640 1 -- 172.21.15.32:0/1718718856 wait complete. 2023-12-20T06:03:34.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbbea66640 1 Processor -- start 2023-12-20T06:03:34.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.302+0000 7fbbbea66640 1 -- start start 2023-12-20T06:03:34.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbea66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb8070230 0x7fbbb81f8ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbea66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb80b0cb0 0x7fbbb81f61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbea66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb815abb0 0x7fbbb80c43c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbea66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81f97f0 con 0x7fbbb815abb0 2023-12-20T06:03:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbea66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81f9ad0 con 0x7fbbb80b0cb0 2023-12-20T06:03:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbea66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbbb81f9db0 con 0x7fbbb8070230 2023-12-20T06:03:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbd003640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb815abb0 0x7fbbb80c43c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbd003640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb815abb0 0x7fbbb80c43c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44218/0 (socket says 172.21.15.32:44218) 2023-12-20T06:03:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbd003640 1 -- 172.21.15.32:0/1323878762 learned_addr learned my addr 172.21.15.32:0/1323878762 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbc802640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb8070230 0x7fbbb81f8ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbb7fff640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb80b0cb0 0x7fbbb81f61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:34.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbd003640 1 -- 172.21.15.32:0/1323878762 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb8070230 msgr2=0x7fbbb81f8ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbd003640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb8070230 0x7fbbb81f8ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbd003640 1 -- 172.21.15.32:0/1323878762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb80b0cb0 msgr2=0x7fbbb81f61e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbd003640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb80b0cb0 0x7fbbb81f61e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbd003640 1 -- 172.21.15.32:0/1323878762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbba40094d0 con 0x7fbbb815abb0 2023-12-20T06:03:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbb7fff640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb80b0cb0 0x7fbbb81f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbd003640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb815abb0 0x7fbbb80c43c0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000c770 tx=0x7fbbb000cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbb57fa640 1 -- 172.21.15.32:0/1323878762 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb0003b80 con 0x7fbbb815abb0 2023-12-20T06:03:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbb57fa640 1 -- 172.21.15.32:0/1323878762 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbbb0003ce0 con 0x7fbbb815abb0 2023-12-20T06:03:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbb57fa640 1 -- 172.21.15.32:0/1323878762 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbbb000aa90 con 0x7fbbb815abb0 2023-12-20T06:03:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbea66640 1 -- 172.21.15.32:0/1323878762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbbb81f59e0 con 0x7fbbb815abb0 2023-12-20T06:03:34.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbea66640 1 -- 172.21.15.32:0/1323878762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbbb81f9400 con 0x7fbbb815abb0 2023-12-20T06:03:34.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.306+0000 7fbbbea66640 1 -- 172.21.15.32:0/1323878762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbbb81f95c0 con 0x7fbbb815abb0 2023-12-20T06:03:34.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.310+0000 7fbbb57fa640 1 -- 172.21.15.32:0/1323878762 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fbbb000abf0 con 0x7fbbb815abb0 2023-12-20T06:03:34.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.310+0000 7fbbb57fa640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb9c065ca0 0x7fbb9c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.310+0000 7fbbb57fa640 1 -- 172.21.15.32:0/1323878762 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(232..232 src has 1..232) v4 ==== 6523+0+0 (secure 0 0 0) 0x7fbbb0083720 con 0x7fbbb815abb0 2023-12-20T06:03:34.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.310+0000 7fbbbc802640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb9c065ca0 0x7fbb9c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:34.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.310+0000 7fbbbc802640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb9c065ca0 0x7fbb9c068160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009ac0 tx=0x7fbbac009190 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:34.314 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.314+0000 7fbbb57fa640 1 -- 172.21.15.32:0/1323878762 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbbb00509b0 con 0x7fbbb815abb0 2023-12-20T06:03:34.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.510+0000 7fbbbea66640 1 -- 172.21.15.32:0/1323878762 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"} v 0) v1 -- 0x7fbbb804e000 con 0x7fbbb815abb0 2023-12-20T06:03:34.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.522+0000 7fbbb57fa640 1 -- 172.21.15.32:0/1323878762 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "backup_fs", "metadata": "backup_fs_metadata", "data": "backup_fs_data"}]=0 new fs with metadata pool 30 and data pool 31 v114) v1 ==== 183+0+0 (secure 0 0 0) 0x7fbbb0053fd0 con 0x7fbbb815abb0 2023-12-20T06:03:34.524 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 30 and data pool 31 2023-12-20T06:03:34.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.526+0000 7fbbbea66640 1 -- 172.21.15.32:0/1323878762 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb9c065ca0 msgr2=0x7fbb9c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.526+0000 7fbbbea66640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb9c065ca0 0x7fbb9c068160 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009ac0 tx=0x7fbbac009190 comp rx=0 tx=0).stop 2023-12-20T06:03:34.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.526+0000 7fbbbea66640 1 -- 172.21.15.32:0/1323878762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb815abb0 msgr2=0x7fbbb80c43c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.526+0000 7fbbbea66640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb815abb0 0x7fbbb80c43c0 secure :-1 s=READY pgs=2284 cs=0 l=1 rev1=1 crypto rx=0x7fbbb000c770 tx=0x7fbbb000cc30 comp rx=0 tx=0).stop 2023-12-20T06:03:34.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.530+0000 7fbbbea66640 1 -- 172.21.15.32:0/1323878762 shutdown_connections 2023-12-20T06:03:34.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.530+0000 7fbbbea66640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb9c065ca0 0x7fbb9c068160 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.530+0000 7fbbbea66640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbbb815abb0 0x7fbbb80c43c0 unknown :-1 s=CLOSED pgs=2284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.530+0000 7fbbbea66640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbbb80b0cb0 0x7fbbb81f61e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.530+0000 7fbbbea66640 1 --2- 172.21.15.32:0/1323878762 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbbb8070230 0x7fbbb81f8ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.530+0000 7fbbbea66640 1 -- 172.21.15.32:0/1323878762 >> 172.21.15.32:0/1323878762 conn(0x7fbbb806a510 msgr2=0x7fbbb80cd240 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:34.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.534+0000 7fbbbea66640 1 -- 172.21.15.32:0/1323878762 shutdown_connections 2023-12-20T06:03:34.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.534+0000 7fbbbea66640 1 -- 172.21.15.32:0/1323878762 wait complete. 2023-12-20T06:03:34.574 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:34.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f60df3640 1 Processor -- start 2023-12-20T06:03:34.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f60df3640 1 -- start start 2023-12-20T06:03:34.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f60df3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f8f5c15cdb0 0x7f8f5c0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f60df3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f5c0d2740 0x7f8f5c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f60df3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f5c0d1120 0x7f8f5c0cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f60df3640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5c15d190 con 0x7f8f5c0d1120 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f60df3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f8f5c0d16a0 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f60df3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0d1980 con 0x7f8f5c0d2740 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f5ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f5c0d1120 0x7f8f5c0cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f5ad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f8f5c0d1120 0x7f8f5c0cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44228/0 (socket says 172.21.15.32:44228) 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f5ad9d640 1 -- 172.21.15.32:0/2061812081 learned_addr learned my addr 172.21.15.32:0/2061812081 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f5ad9d640 1 -- 172.21.15.32:0/2061812081 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f5c0d2740 msgr2=0x7f8f5c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f5ad9d640 1 --2- 172.21.15.32:0/2061812081 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f5c0d2740 0x7f8f5c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f5ad9d640 1 -- 172.21.15.32:0/2061812081 >> v2:172.21.15.39:3300/0 conn(0x7f8f5c15cdb0 msgr2=0x7f8f5c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f5ad9d640 1 --2- 172.21.15.32:0/2061812081 >> v2:172.21.15.39:3300/0 conn(0x7f8f5c15cdb0 0x7f8f5c0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.718+0000 7f8f5ad9d640 1 -- 172.21.15.32:0/2061812081 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f5c0d1c60 con 0x7f8f5c0d1120 2023-12-20T06:03:34.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f5ad9d640 1 --2- 172.21.15.32:0/2061812081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5c0d1120 0x7f8f5c0cfab0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c0667b0 tx=0x7f8f5401f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bcfe070f9f0e09ad server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f5959a640 1 -- 172.21.15.32:0/2061812081 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540af400 con 0x7f8f5c0d1120 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f5959a640 1 -- 172.21.15.32:0/2061812081 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f540ab050 con 0x7f8f5c0d1120 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f5959a640 1 -- 172.21.15.32:0/2061812081 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540af780 con 0x7f8f5c0d1120 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f60df3640 1 -- 172.21.15.32:0/2061812081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5c0d1120 msgr2=0x7f8f5c0cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f60df3640 1 --2- 172.21.15.32:0/2061812081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5c0d1120 0x7f8f5c0cfab0 secure :-1 s=READY pgs=2285 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c0667b0 tx=0x7f8f5401f1c0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f60df3640 1 -- 172.21.15.32:0/2061812081 shutdown_connections 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f60df3640 1 --2- 172.21.15.32:0/2061812081 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5c0d1120 0x7f8f5c0cfab0 unknown :-1 s=CLOSED pgs=2285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f60df3640 1 --2- 172.21.15.32:0/2061812081 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f5c0d2740 0x7f8f5c0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f60df3640 1 --2- 172.21.15.32:0/2061812081 >> v2:172.21.15.39:3300/0 conn(0x7f8f5c15cdb0 0x7f8f5c0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f60df3640 1 -- 172.21.15.32:0/2061812081 >> 172.21.15.32:0/2061812081 conn(0x7f8f5c065ee0 msgr2=0x7f8f5c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f60df3640 1 -- 172.21.15.32:0/2061812081 shutdown_connections 2023-12-20T06:03:34.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.722+0000 7f8f60df3640 1 -- 172.21.15.32:0/2061812081 wait complete. 2023-12-20T06:03:34.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f60df3640 1 Processor -- start 2023-12-20T06:03:34.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f60df3640 1 -- start start 2023-12-20T06:03:34.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f60df3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f5c0d1120 0x7f8f5c13dba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f60df3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f5c0d2740 0x7f8f5c15db10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f60df3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5c15cdb0 0x7f8f5c1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f60df3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0cb370 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f60df3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c1578b0 con 0x7f8f5c0d2740 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f60df3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f8f5c0cad20 con 0x7f8f5c0d1120 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f5ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5c15cdb0 0x7f8f5c1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f5ad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5c15cdb0 0x7f8f5c1548c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44244/0 (socket says 172.21.15.32:44244) 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f5ad9d640 1 -- 172.21.15.32:0/3811285879 learned_addr learned my addr 172.21.15.32:0/3811285879 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f59d9b640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f5c0d2740 0x7f8f5c15db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f5a59c640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f5c0d1120 0x7f8f5c13dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f5ad9d640 1 -- 172.21.15.32:0/3811285879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f5c0d1120 msgr2=0x7f8f5c13dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f5ad9d640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f5c0d1120 0x7f8f5c13dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f5ad9d640 1 -- 172.21.15.32:0/3811285879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f5c0d2740 msgr2=0x7f8f5c15db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f5ad9d640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f5c0d2740 0x7f8f5c15db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f5ad9d640 1 -- 172.21.15.32:0/3811285879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8f54009b30 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f5ad9d640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5c15cdb0 0x7f8f5c1548c0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f8f540b2040 tx=0x7f8f5401bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:34.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f4affd640 1 -- 172.21.15.32:0/3811285879 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540afdc0 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f4affd640 1 -- 172.21.15.32:0/3811285879 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8f54009510 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f60df3640 1 -- 172.21.15.32:0/3811285879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f8f5c0cb000 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f4affd640 1 -- 172.21.15.32:0/3811285879 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8f540b8640 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.726+0000 7f8f60df3640 1 -- 172.21.15.32:0/3811285879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8f5c15e050 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.730+0000 7f8f60df3640 1 -- 172.21.15.32:0/3811285879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8f5c0cb4e0 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.730+0000 7f8f4affd640 1 -- 172.21.15.32:0/3811285879 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f8f540b6040 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.730+0000 7f8f4affd640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f3c065cd0 0x7f8f3c068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.730+0000 7f8f4affd640 1 -- 172.21.15.32:0/3811285879 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f8f541218f0 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.730+0000 7f8f5a59c640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f3c065cd0 0x7f8f3c068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:34.732 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.734+0000 7f8f5a59c640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f3c065cd0 0x7f8f3c068190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c13da70 tx=0x7f8f4c006f90 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:34.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.734+0000 7f8f4affd640 1 -- 172.21.15.32:0/3811285879 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8f540eec20 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f60df3640 1 -- 172.21.15.32:0/3811285879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f8f5c04e000 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f4affd640 1 -- 172.21.15.32:0/3811285879 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v233) v1 ==== 74+0+19830 (secure 0 0 0) 0x7f8f540b4090 con 0x7f8f5c15cdb0 2023-12-20T06:03:34.899 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:34.899 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":233,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:34.513565+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:03:20.399088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-20T06:03:20.926367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:03:32.174574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:03:33.509224+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f60df3640 1 -- 172.21.15.32:0/3811285879 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f3c065cd0 msgr2=0x7f8f3c068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f60df3640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f3c065cd0 0x7f8f3c068190 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f8f5c13da70 tx=0x7f8f4c006f90 comp rx=0 tx=0).stop 2023-12-20T06:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f60df3640 1 -- 172.21.15.32:0/3811285879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5c15cdb0 msgr2=0x7f8f5c1548c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f60df3640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5c15cdb0 0x7f8f5c1548c0 secure :-1 s=READY pgs=2286 cs=0 l=1 rev1=1 crypto rx=0x7f8f540b2040 tx=0x7f8f5401bf60 comp rx=0 tx=0).stop 2023-12-20T06:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f60df3640 1 -- 172.21.15.32:0/3811285879 shutdown_connections 2023-12-20T06:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f60df3640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8f3c065cd0 0x7f8f3c068190 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f60df3640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f8f5c15cdb0 0x7f8f5c1548c0 unknown :-1 s=CLOSED pgs=2286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f60df3640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f8f5c0d2740 0x7f8f5c15db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f60df3640 1 --2- 172.21.15.32:0/3811285879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f8f5c0d1120 0x7f8f5c13dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.898+0000 7f8f60df3640 1 -- 172.21.15.32:0/3811285879 >> 172.21.15.32:0/3811285879 conn(0x7f8f5c065ee0 msgr2=0x7f8f5c0cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.902+0000 7f8f60df3640 1 -- 172.21.15.32:0/3811285879 shutdown_connections 2023-12-20T06:03:34.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:34.902+0000 7f8f60df3640 1 -- 172.21.15.32:0/3811285879 wait complete. 2023-12-20T06:03:34.921 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.062+0000 7f9866587640 1 Processor -- start 2023-12-20T06:03:35.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9866587640 1 -- start start 2023-12-20T06:03:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9866587640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f98600cd2e0 0x7f98600ca2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9866587640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98600b3e90 0x7f98600ca820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9866587640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f98600ca0b0 con 0x7f9860159ab0 2023-12-20T06:03:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9866587640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f98600ad7e0 con 0x7f98600cd2e0 2023-12-20T06:03:35.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9866587640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f98600ada80 con 0x7f98600b3e90 2023-12-20T06:03:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9865585640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9860159ab0 0x7f98600714c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51210/0 (socket says 172.21.15.32:51210) 2023-12-20T06:03:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9865585640 1 -- 172.21.15.32:0/308225338 learned_addr learned my addr 172.21.15.32:0/308225338 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9864d84640 1 --2- 172.21.15.32:0/308225338 >> v2:172.21.15.39:3300/0 conn(0x7f98600cd2e0 0x7f98600ca2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9865d86640 1 --2- 172.21.15.32:0/308225338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98600b3e90 0x7f98600ca820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9857fff640 1 -- 172.21.15.32:0/308225338 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2747083707 0 0) 0x7f98600ca0b0 con 0x7f9860159ab0 2023-12-20T06:03:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9857fff640 1 -- 172.21.15.32:0/308225338 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f984c003530 con 0x7f9860159ab0 2023-12-20T06:03:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9864d84640 1 -- 172.21.15.32:0/308225338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98600b3e90 msgr2=0x7f98600ca820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9864d84640 1 --2- 172.21.15.32:0/308225338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98600b3e90 0x7f98600ca820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9864d84640 1 -- 172.21.15.32:0/308225338 >> v1:172.21.15.32:6789/0 conn(0x7f9860159ab0 legacy=0x7f98600714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9864d84640 1 -- 172.21.15.32:0/308225338 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f98600c98b0 con 0x7f98600cd2e0 2023-12-20T06:03:35.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.066+0000 7f9865d86640 1 --2- 172.21.15.32:0/308225338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98600b3e90 0x7f98600ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:35.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9864d84640 1 --2- 172.21.15.32:0/308225338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98600cd2e0 0x7f98600ca2e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f98500097c0 tx=0x7f985001c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=e57125e7391de742 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:35.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9857fff640 1 -- 172.21.15.32:0/308225338 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98500ab600 con 0x7f98600cd2e0 2023-12-20T06:03:35.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9857fff640 1 -- 172.21.15.32:0/308225338 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98500a7050 con 0x7f98600cd2e0 2023-12-20T06:03:35.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9857fff640 1 -- 172.21.15.32:0/308225338 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98500ab980 con 0x7f98600cd2e0 2023-12-20T06:03:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9866587640 1 -- 172.21.15.32:0/308225338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98600cd2e0 msgr2=0x7f98600ca2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9866587640 1 --2- 172.21.15.32:0/308225338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98600cd2e0 0x7f98600ca2e0 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7f98500097c0 tx=0x7f985001c450 comp rx=0 tx=0).stop 2023-12-20T06:03:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9866587640 1 -- 172.21.15.32:0/308225338 shutdown_connections 2023-12-20T06:03:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9866587640 1 --2- 172.21.15.32:0/308225338 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f98600cd2e0 0x7f98600ca2e0 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9866587640 1 --2- 172.21.15.32:0/308225338 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98600b3e90 0x7f98600ca820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9866587640 1 -- 172.21.15.32:0/308225338 >> 172.21.15.32:0/308225338 conn(0x7f986006c060 msgr2=0x7f986006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:35.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9866587640 1 -- 172.21.15.32:0/308225338 shutdown_connections 2023-12-20T06:03:35.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.070+0000 7f9866587640 1 -- 172.21.15.32:0/308225338 wait complete. 2023-12-20T06:03:35.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9866587640 1 Processor -- start 2023-12-20T06:03:35.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9866587640 1 -- start start 2023-12-20T06:03:35.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9866587640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98600b3e90 0x7f986015a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:35.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9866587640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98600cd2e0 0x7f9860158050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:35.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9866587640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9860159ab0 0x7f98600c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:35.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9866587640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986015a630 con 0x7f98600cd2e0 2023-12-20T06:03:35.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9866587640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f986015a900 con 0x7f9860159ab0 2023-12-20T06:03:35.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9866587640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f986015abe0 con 0x7f98600b3e90 2023-12-20T06:03:35.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9864d84640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98600cd2e0 0x7f9860158050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:35.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9864d84640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98600cd2e0 0x7f9860158050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44258/0 (socket says 172.21.15.32:44258) 2023-12-20T06:03:35.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9864d84640 1 -- 172.21.15.32:0/349730430 learned_addr learned my addr 172.21.15.32:0/349730430 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9865585640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98600b3e90 0x7f986015a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9865d86640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9860159ab0 0x7f98600c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9864d84640 1 -- 172.21.15.32:0/349730430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98600b3e90 msgr2=0x7f986015a0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9864d84640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98600b3e90 0x7f986015a0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9864d84640 1 -- 172.21.15.32:0/349730430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9860159ab0 msgr2=0x7f98600c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9864d84640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9860159ab0 0x7f98600c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9864d84640 1 -- 172.21.15.32:0/349730430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9858009a70 con 0x7f98600cd2e0 2023-12-20T06:03:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9864d84640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98600cd2e0 0x7f9860158050 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f98500ae040 tx=0x7f98500191f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9855ffb640 1 -- 172.21.15.32:0/349730430 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98500abea0 con 0x7f98600cd2e0 2023-12-20T06:03:35.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9855ffb640 1 -- 172.21.15.32:0/349730430 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f98500b4430 con 0x7f98600cd2e0 2023-12-20T06:03:35.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9855ffb640 1 -- 172.21.15.32:0/349730430 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f98500b4800 con 0x7f98600cd2e0 2023-12-20T06:03:35.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9866587640 1 -- 172.21.15.32:0/349730430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9850009420 con 0x7f98600cd2e0 2023-12-20T06:03:35.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9866587640 1 -- 172.21.15.32:0/349730430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f986015b1a0 con 0x7f98600cd2e0 2023-12-20T06:03:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.074+0000 7f9866587640 1 -- 172.21.15.32:0/349730430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9860074100 con 0x7f98600cd2e0 2023-12-20T06:03:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.078+0000 7f9855ffb640 1 -- 172.21.15.32:0/349730430 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f98500b2040 con 0x7f98600cd2e0 2023-12-20T06:03:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.078+0000 7f9855ffb640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9844065cd0 0x7f9844068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.078+0000 7f9855ffb640 1 -- 172.21.15.32:0/349730430 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f985011d7c0 con 0x7f98600cd2e0 2023-12-20T06:03:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.078+0000 7f9865585640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9844065cd0 0x7f9844068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:35.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.078+0000 7f9865585640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9844065cd0 0x7f9844068190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f985c005890 tx=0x7f985c009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:35.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.082+0000 7f9855ffb640 1 -- 172.21.15.32:0/349730430 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f98500eaa30 con 0x7f98600cd2e0 2023-12-20T06:03:35.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.242+0000 7f9866587640 1 -- 172.21.15.32:0/349730430 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f98600c1de0 con 0x7f98600cd2e0 2023-12-20T06:03:35.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9855ffb640 1 -- 172.21.15.32:0/349730430 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v233) v1 ==== 74+0+19830 (secure 0 0 0) 0x7f98500ee050 con 0x7f98600cd2e0 2023-12-20T06:03:35.246 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:35.246 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":233,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:34.513565+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:03:20.399088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-20T06:03:20.926367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:03:32.174574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:03:33.509224+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 -- 172.21.15.32:0/349730430 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9844065cd0 msgr2=0x7f9844068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9844065cd0 0x7f9844068190 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f985c005890 tx=0x7f985c009040 comp rx=0 tx=0).stop 2023-12-20T06:03:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 -- 172.21.15.32:0/349730430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98600cd2e0 msgr2=0x7f9860158050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98600cd2e0 0x7f9860158050 secure :-1 s=READY pgs=2288 cs=0 l=1 rev1=1 crypto rx=0x7f98500ae040 tx=0x7f98500191f0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 -- 172.21.15.32:0/349730430 shutdown_connections 2023-12-20T06:03:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9844065cd0 0x7f9844068190 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9860159ab0 0x7f98600c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f98600cd2e0 0x7f9860158050 unknown :-1 s=CLOSED pgs=2288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 --2- 172.21.15.32:0/349730430 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f98600b3e90 0x7f986015a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 -- 172.21.15.32:0/349730430 >> 172.21.15.32:0/349730430 conn(0x7f986006c060 msgr2=0x7f98600714c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:35.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 -- 172.21.15.32:0/349730430 shutdown_connections 2023-12-20T06:03:35.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.246+0000 7f9866587640 1 -- 172.21.15.32:0/349730430 wait complete. 2023-12-20T06:03:35.267 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs standby_count_wanted 0 2023-12-20T06:03:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c9e58f640 1 Processor -- start 2023-12-20T06:03:35.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c9e58f640 1 -- start start 2023-12-20T06:03:35.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c9e58f640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6c980b0bc0 0x7f6c980ccbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:35.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c9e58f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c980cde20 con 0x7f6c98157480 2023-12-20T06:03:35.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c9e58f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c980c8e30 con 0x7f6c980b0760 2023-12-20T06:03:35.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c9e58f640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c980c9f50 con 0x7f6c980b0bc0 2023-12-20T06:03:35.416 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c97fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6c98157480 0x7f6c980c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51224/0 (socket says 172.21.15.32:51224) 2023-12-20T06:03:35.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c97fff640 1 -- 172.21.15.32:0/1740832136 learned_addr learned my addr 172.21.15.32:0/1740832136 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:35.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c9cb2c640 1 --2- 172.21.15.32:0/1740832136 >> v2:172.21.15.39:3300/0 conn(0x7f6c980b0bc0 0x7f6c980ccbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:35.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c96ffd640 1 -- 172.21.15.32:0/1740832136 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 559124842 0 0) 0x7f6c980cde20 con 0x7f6c98157480 2023-12-20T06:03:35.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c96ffd640 1 -- 172.21.15.32:0/1740832136 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c80003530 con 0x7f6c98157480 2023-12-20T06:03:35.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c9cb2c640 1 -- 172.21.15.32:0/1740832136 >> v1:172.21.15.39:6789/0 conn(0x7f6c980b0760 legacy=0x7f6c980cc4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:35.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c9cb2c640 1 -- 172.21.15.32:0/1740832136 >> v1:172.21.15.32:6789/0 conn(0x7f6c98157480 legacy=0x7f6c980c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:35.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.414+0000 7f6c9cb2c640 1 -- 172.21.15.32:0/1740832136 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c980ce070 con 0x7f6c980b0bc0 2023-12-20T06:03:35.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.418+0000 7f6c9cb2c640 1 --2- 172.21.15.32:0/1740832136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c980b0bc0 0x7f6c980ccbc0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009d90 tx=0x7f6c9001c860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3bba7eb3b2bf7c99 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:35.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.418+0000 7f6c96ffd640 1 -- 172.21.15.32:0/1740832136 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c900a58d0 con 0x7f6c980b0bc0 2023-12-20T06:03:35.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.418+0000 7f6c96ffd640 1 -- 172.21.15.32:0/1740832136 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c900a5a30 con 0x7f6c980b0bc0 2023-12-20T06:03:35.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.418+0000 7f6c96ffd640 1 -- 172.21.15.32:0/1740832136 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c900a5dc0 con 0x7f6c980b0bc0 2023-12-20T06:03:35.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.418+0000 7f6c9e58f640 1 -- 172.21.15.32:0/1740832136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c980b0bc0 msgr2=0x7f6c980ccbc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:35.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.418+0000 7f6c9e58f640 1 --2- 172.21.15.32:0/1740832136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c980b0bc0 0x7f6c980ccbc0 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7f6c90009d90 tx=0x7f6c9001c860 comp rx=0 tx=0).stop 2023-12-20T06:03:35.420 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.418+0000 7f6c9e58f640 1 -- 172.21.15.32:0/1740832136 shutdown_connections 2023-12-20T06:03:35.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.418+0000 7f6c9e58f640 1 --2- 172.21.15.32:0/1740832136 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c980b0bc0 0x7f6c980ccbc0 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.418+0000 7f6c9e58f640 1 -- 172.21.15.32:0/1740832136 >> 172.21.15.32:0/1740832136 conn(0x7f6c9806a510 msgr2=0x7f6c980ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:35.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.418+0000 7f6c9e58f640 1 -- 172.21.15.32:0/1740832136 shutdown_connections 2023-12-20T06:03:35.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.418+0000 7f6c9e58f640 1 -- 172.21.15.32:0/1740832136 wait complete. 2023-12-20T06:03:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9e58f640 1 Processor -- start 2023-12-20T06:03:35.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9e58f640 1 -- start start 2023-12-20T06:03:35.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9e58f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c980b0760 0x7f6c980c81d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9e58f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c980b0bc0 0x7f6c98143240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9e58f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c98157480 0x7f6c980b3e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9e58f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c980c8710 con 0x7f6c980b0760 2023-12-20T06:03:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9e58f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c980b2d90 con 0x7f6c98157480 2023-12-20T06:03:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9e58f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c980b3070 con 0x7f6c980b0bc0 2023-12-20T06:03:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c97fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c980b0760 0x7f6c980c81d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c97fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c980b0760 0x7f6c980c81d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44260/0 (socket says 172.21.15.32:44260) 2023-12-20T06:03:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c97fff640 1 -- 172.21.15.32:0/3073636215 learned_addr learned my addr 172.21.15.32:0/3073636215 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:35.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c977fe640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c980b0bc0 0x7f6c98143240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9cb2c640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c98157480 0x7f6c980b3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c97fff640 1 -- 172.21.15.32:0/3073636215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c980b0bc0 msgr2=0x7f6c98143240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c97fff640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c980b0bc0 0x7f6c98143240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c97fff640 1 -- 172.21.15.32:0/3073636215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c98157480 msgr2=0x7f6c980b3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c97fff640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c98157480 0x7f6c980b3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c97fff640 1 -- 172.21.15.32:0/3073636215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c90009a70 con 0x7f6c980b0760 2023-12-20T06:03:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9cb2c640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c98157480 0x7f6c980b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c977fe640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c980b0bc0 0x7f6c98143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c97fff640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c980b0760 0x7f6c980c81d0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800c420 tx=0x7f6c8800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:35.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c94ff9640 1 -- 172.21.15.32:0/3073636215 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c88005cf0 con 0x7f6c980b0760 2023-12-20T06:03:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c94ff9640 1 -- 172.21.15.32:0/3073636215 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c88005e50 con 0x7f6c980b0760 2023-12-20T06:03:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c94ff9640 1 -- 172.21.15.32:0/3073636215 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c880072d0 con 0x7f6c980b0760 2023-12-20T06:03:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9e58f640 1 -- 172.21.15.32:0/3073636215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c980b3350 con 0x7f6c980b0760 2023-12-20T06:03:35.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.422+0000 7f6c9e58f640 1 -- 172.21.15.32:0/3073636215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c980b3910 con 0x7f6c980b0760 2023-12-20T06:03:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.426+0000 7f6c9e58f640 1 -- 172.21.15.32:0/3073636215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c9804e000 con 0x7f6c980b0760 2023-12-20T06:03:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.426+0000 7f6c94ff9640 1 -- 172.21.15.32:0/3073636215 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f6c88007430 con 0x7f6c980b0760 2023-12-20T06:03:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.426+0000 7f6c94ff9640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c84065ca0 0x7f6c84068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.426+0000 7f6c94ff9640 1 -- 172.21.15.32:0/3073636215 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(233..233 src has 1..233) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f6c88082db0 con 0x7f6c980b0760 2023-12-20T06:03:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.426+0000 7f6c977fe640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c84065ca0 0x7f6c84068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:35.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.426+0000 7f6c977fe640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c84065ca0 0x7f6c84068160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c981f8170 tx=0x7f6c8c009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:35.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.430+0000 7f6c94ff9640 1 -- 172.21.15.32:0/3073636215 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c88050190 con 0x7f6c980b0760 2023-12-20T06:03:35.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:35.626+0000 7f6c9e58f640 1 -- 172.21.15.32:0/3073636215 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7f6c980c5000 con 0x7f6c980b0760 2023-12-20T06:03:36.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c94ff9640 1 -- 172.21.15.32:0/3073636215 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "standby_count_wanted", "val": "0"}]=0 v117) v1 ==== 121+0+0 (secure 0 0 0) 0x7f6c88017020 con 0x7f6c980b0760 2023-12-20T06:03:36.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 -- 172.21.15.32:0/3073636215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c84065ca0 msgr2=0x7f6c84068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:36.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c84065ca0 0x7f6c84068160 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7f6c981f8170 tx=0x7f6c8c009040 comp rx=0 tx=0).stop 2023-12-20T06:03:36.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 -- 172.21.15.32:0/3073636215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c980b0760 msgr2=0x7f6c980c81d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:36.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c980b0760 0x7f6c980c81d0 secure :-1 s=READY pgs=2290 cs=0 l=1 rev1=1 crypto rx=0x7f6c8800c420 tx=0x7f6c8800c8e0 comp rx=0 tx=0).stop 2023-12-20T06:03:36.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 -- 172.21.15.32:0/3073636215 shutdown_connections 2023-12-20T06:03:36.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c84065ca0 0x7f6c84068160 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:36.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c98157480 0x7f6c980b3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:36.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c980b0bc0 0x7f6c98143240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:36.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 --2- 172.21.15.32:0/3073636215 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c980b0760 0x7f6c980c81d0 unknown :-1 s=CLOSED pgs=2290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:36.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 -- 172.21.15.32:0/3073636215 >> 172.21.15.32:0/3073636215 conn(0x7f6c9806a510 msgr2=0x7f6c98159090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:36.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 -- 172.21.15.32:0/3073636215 shutdown_connections 2023-12-20T06:03:36.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.550+0000 7f6c9e58f640 1 -- 172.21.15.32:0/3073636215 wait complete. 2023-12-20T06:03:36.576 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T06:03:36.576 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set backup_fs allow_standby_replay false 2023-12-20T06:03:36.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.726+0000 7f10adfce640 1 Processor -- start 2023-12-20T06:03:36.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.726+0000 7f10adfce640 1 -- start start 2023-12-20T06:03:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10adfce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10a806cdf0 0x7f10a8070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10adfce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f10a8070770 0x7f10a80cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10adfce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10a80cc3b0 con 0x7f10a81591e0 2023-12-20T06:03:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10adfce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a8070ba0 con 0x7f10a806cdf0 2023-12-20T06:03:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10adfce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10a80c8e30 con 0x7f10a8070770 2023-12-20T06:03:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10a6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10a806cdf0 0x7f10a8070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10a6ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f10a806cdf0 0x7f10a8070230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44262/0 (socket says 172.21.15.32:44262) 2023-12-20T06:03:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10a77fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f10a81591e0 0x7f10a80caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51226/0 (socket says 172.21.15.32:51226) 2023-12-20T06:03:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10a6ffd640 1 -- 172.21.15.32:0/2829191045 learned_addr learned my addr 172.21.15.32:0/2829191045 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10a7fff640 1 --2- 172.21.15.32:0/2829191045 >> v2:172.21.15.39:3300/0 conn(0x7f10a8070770 0x7f10a80cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:36.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10a6ffd640 1 -- 172.21.15.32:0/2829191045 >> v2:172.21.15.39:3300/0 conn(0x7f10a8070770 msgr2=0x7f10a80cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:36.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10a6ffd640 1 --2- 172.21.15.32:0/2829191045 >> v2:172.21.15.39:3300/0 conn(0x7f10a8070770 0x7f10a80cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:36.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10a6ffd640 1 -- 172.21.15.32:0/2829191045 >> v1:172.21.15.32:6789/0 conn(0x7f10a81591e0 legacy=0x7f10a80caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:36.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10a6ffd640 1 -- 172.21.15.32:0/2829191045 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10a806d280 con 0x7f10a806cdf0 2023-12-20T06:03:36.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.730+0000 7f10a6ffd640 1 --2- 172.21.15.32:0/2829191045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a806cdf0 0x7f10a8070230 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f10a80c5860 tx=0x7f109c01eb40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74cf956b7d78f3a3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:36.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10a67fc640 1 -- 172.21.15.32:0/2829191045 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c0a7b90 con 0x7f10a806cdf0 2023-12-20T06:03:36.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10a67fc640 1 -- 172.21.15.32:0/2829191045 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f109c0a7cf0 con 0x7f10a806cdf0 2023-12-20T06:03:36.733 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10a67fc640 1 -- 172.21.15.32:0/2829191045 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f109c0af660 con 0x7f10a806cdf0 2023-12-20T06:03:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 -- 172.21.15.32:0/2829191045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a806cdf0 msgr2=0x7f10a8070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 --2- 172.21.15.32:0/2829191045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a806cdf0 0x7f10a8070230 secure :-1 s=READY pgs=2292 cs=0 l=1 rev1=1 crypto rx=0x7f10a80c5860 tx=0x7f109c01eb40 comp rx=0 tx=0).stop 2023-12-20T06:03:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 -- 172.21.15.32:0/2829191045 shutdown_connections 2023-12-20T06:03:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 --2- 172.21.15.32:0/2829191045 >> v2:172.21.15.39:3300/0 conn(0x7f10a8070770 0x7f10a80cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 --2- 172.21.15.32:0/2829191045 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a806cdf0 0x7f10a8070230 unknown :-1 s=CLOSED pgs=2292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 -- 172.21.15.32:0/2829191045 >> 172.21.15.32:0/2829191045 conn(0x7f10a806a510 msgr2=0x7f10a80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 -- 172.21.15.32:0/2829191045 shutdown_connections 2023-12-20T06:03:36.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 -- 172.21.15.32:0/2829191045 wait complete. 2023-12-20T06:03:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 Processor -- start 2023-12-20T06:03:36.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 -- start start 2023-12-20T06:03:36.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a8070770 0x7f10a80b3ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:36.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10a81591e0 0x7f10a81555b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10a8143f50 0x7f10a80c4a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a80c7ec0 con 0x7f10a8070770 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10a8155f90 con 0x7f10a81591e0 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10adfce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10a81564c0 con 0x7f10a8143f50 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10a77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a8070770 0x7f10a80b3ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10a77fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a8070770 0x7f10a80b3ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44276/0 (socket says 172.21.15.32:44276) 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.734+0000 7f10a77fe640 1 -- 172.21.15.32:0/1138280673 learned_addr learned my addr 172.21.15.32:0/1138280673 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f10a77fe640 1 -- 172.21.15.32:0/1138280673 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10a8143f50 msgr2=0x7f10a80c4a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f10a77fe640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10a8143f50 0x7f10a80c4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f10a77fe640 1 -- 172.21.15.32:0/1138280673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10a81591e0 msgr2=0x7f10a81555b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f10a6ffd640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10a81591e0 0x7f10a81555b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f10a77fe640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10a81591e0 0x7f10a81555b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:36.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f10a77fe640 1 -- 172.21.15.32:0/1138280673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f109c00b770 con 0x7f10a8070770 2023-12-20T06:03:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f10a6ffd640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10a81591e0 0x7f10a81555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f10a77fe640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a8070770 0x7f10a80b3ec0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f109800c460 tx=0x7f109800c920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f108ffff640 1 -- 172.21.15.32:0/1138280673 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098007480 con 0x7f10a8070770 2023-12-20T06:03:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f108ffff640 1 -- 172.21.15.32:0/1138280673 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10980075e0 con 0x7f10a8070770 2023-12-20T06:03:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f108ffff640 1 -- 172.21.15.32:0/1138280673 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1098004a10 con 0x7f10a8070770 2023-12-20T06:03:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f10adfce640 1 -- 172.21.15.32:0/1138280673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10a81561e0 con 0x7f10a8070770 2023-12-20T06:03:36.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f10adfce640 1 -- 172.21.15.32:0/1138280673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10a80c83d0 con 0x7f10a8070770 2023-12-20T06:03:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f10adfce640 1 -- 172.21.15.32:0/1138280673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10a80ce6f0 con 0x7f10a8070770 2023-12-20T06:03:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f108ffff640 1 -- 172.21.15.32:0/1138280673 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f1098004b70 con 0x7f10a8070770 2023-12-20T06:03:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f108ffff640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1094065ca0 0x7f1094068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.738+0000 7f108ffff640 1 -- 172.21.15.32:0/1138280673 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f1098082ab0 con 0x7f10a8070770 2023-12-20T06:03:36.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.742+0000 7f10a6ffd640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1094065ca0 0x7f1094068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:36.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.742+0000 7f10a6ffd640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1094065ca0 0x7f1094068160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f109c0b1000 tx=0x7f109c008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:36.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.742+0000 7f108ffff640 1 -- 172.21.15.32:0/1138280673 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f109804fd60 con 0x7f10a8070770 2023-12-20T06:03:36.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:36.934+0000 7f10adfce640 1 -- 172.21.15.32:0/1138280673 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f10a804e000 con 0x7f10a8070770 2023-12-20T06:03:37.551 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.550+0000 7f108ffff640 1 -- 172.21.15.32:0/1138280673 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "backup_fs", "var": "allow_standby_replay", "val": "false"}]=0 v118) v1 ==== 125+0+0 (secure 0 0 0) 0x7f10980180c0 con 0x7f10a8070770 2023-12-20T06:03:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 -- 172.21.15.32:0/1138280673 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1094065ca0 msgr2=0x7f1094068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1094065ca0 0x7f1094068160 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f109c0b1000 tx=0x7f109c008040 comp rx=0 tx=0).stop 2023-12-20T06:03:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 -- 172.21.15.32:0/1138280673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a8070770 msgr2=0x7f10a80b3ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a8070770 0x7f10a80b3ec0 secure :-1 s=READY pgs=2293 cs=0 l=1 rev1=1 crypto rx=0x7f109800c460 tx=0x7f109800c920 comp rx=0 tx=0).stop 2023-12-20T06:03:37.554 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 -- 172.21.15.32:0/1138280673 shutdown_connections 2023-12-20T06:03:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1094065ca0 0x7f1094068160 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10a8143f50 0x7f10a80c4a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10a81591e0 0x7f10a81555b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 --2- 172.21.15.32:0/1138280673 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10a8070770 0x7f10a80b3ec0 unknown :-1 s=CLOSED pgs=2293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 -- 172.21.15.32:0/1138280673 >> 172.21.15.32:0/1138280673 conn(0x7f10a806a510 msgr2=0x7f10a80c3ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 -- 172.21.15.32:0/1138280673 shutdown_connections 2023-12-20T06:03:37.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.554+0000 7f10adfce640 1 -- 172.21.15.32:0/1138280673 wait complete. 2023-12-20T06:03:37.577 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'backup_fs' 2023-12-20T06:03:37.577 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:37.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.722+0000 7f0a8b8fc640 1 Processor -- start 2023-12-20T06:03:37.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.722+0000 7f0a8b8fc640 1 -- start start 2023-12-20T06:03:37.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.722+0000 7f0a8b8fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0a841599e0 0x7f0a840aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:37.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.722+0000 7f0a8b8fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a840b0d00 0x7f0a840701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:37.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.722+0000 7f0a8b8fc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a840c98c0 con 0x7f0a840c90e0 2023-12-20T06:03:37.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.722+0000 7f0a8b8fc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a840c9ba0 con 0x7f0a841599e0 2023-12-20T06:03:37.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.722+0000 7f0a8b8fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840c9e80 con 0x7f0a840b0d00 2023-12-20T06:03:37.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.722+0000 7f0a88e97640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a840c90e0 0x7f0a840b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:51238/0 (socket says 172.21.15.32:51238) 2023-12-20T06:03:37.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.722+0000 7f0a88e97640 1 -- 172.21.15.32:0/2736988472 learned_addr learned my addr 172.21.15.32:0/2736988472 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.722+0000 7f0a89698640 1 --2- 172.21.15.32:0/2736988472 >> v2:172.21.15.39:3300/0 conn(0x7f0a841599e0 0x7f0a840aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.722+0000 7f0a89e99640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a840b0d00 0x7f0a840701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 330781491 0 0) 0x7f0a840c98c0 con 0x7f0a840c90e0 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a70003530 con 0x7f0a840c90e0 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 265243561 0 0) 0x7f0a70003530 con 0x7f0a840c90e0 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a840b0d00 msgr2=0x7f0a840701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a840b0d00 0x7f0a840701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 >> v2:172.21.15.39:3300/0 conn(0x7f0a841599e0 msgr2=0x7f0a840aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 --2- 172.21.15.32:0/2736988472 >> v2:172.21.15.39:3300/0 conn(0x7f0a841599e0 0x7f0a840aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a840ca160 con 0x7f0a840c90e0 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f0a74001680 con 0x7f0a840c90e0 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 >> v1:172.21.15.32:6789/0 conn(0x7f0a840c90e0 legacy=0x7f0a840b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:37.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 0x7f0a70003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a700047d0 0x7f0a70004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a840b0d00 0x7f0a70005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840c9ba0 con 0x7f0a70003980 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a840c9e80 con 0x7f0a700047d0 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a7000dba0 con 0x7f0a840b0d00 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a89698640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 0x7f0a70003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a88e97640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a700047d0 0x7f0a70004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a89e99640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a840b0d00 0x7f0a70005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a89698640 1 -- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a840b0d00 msgr2=0x7f0a70005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a89698640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a840b0d00 0x7f0a70005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a89698640 1 -- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a700047d0 msgr2=0x7f0a70004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:37.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a89698640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a700047d0 0x7f0a70004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a89698640 1 -- 172.21.15.32:0/2736988472 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a70003530 con 0x7f0a70003980 2023-12-20T06:03:37.729 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.726+0000 7f0a89e99640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a840b0d00 0x7f0a70005190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a89698640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 0x7f0a70003da0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009af0 tx=0x7f0a8001ca20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65313c96f23e78b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a800059a0 con 0x7f0a70003980 2023-12-20T06:03:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a80005b00 con 0x7f0a70003980 2023-12-20T06:03:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a7bfff640 1 -- 172.21.15.32:0/2736988472 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80006030 con 0x7f0a70003980 2023-12-20T06:03:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/2736988472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 msgr2=0x7f0a70003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 --2- 172.21.15.32:0/2736988472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 0x7f0a70003da0 secure :-1 s=READY pgs=2295 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009af0 tx=0x7f0a8001ca20 comp rx=0 tx=0).stop 2023-12-20T06:03:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a89e99640 1 -- 172.21.15.32:0/2736988472 reap_dead start 2023-12-20T06:03:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/2736988472 shutdown_connections 2023-12-20T06:03:37.730 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/2736988472 >> 172.21.15.32:0/2736988472 conn(0x7f0a8406a320 msgr2=0x7f0a840ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/2736988472 shutdown_connections 2023-12-20T06:03:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/2736988472 wait complete. 2023-12-20T06:03:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 Processor -- start 2023-12-20T06:03:37.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 -- start start 2023-12-20T06:03:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 0x7f0a841f99c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a841f2e50 0x7f0a841f5a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a841f5f90 0x7f0a840caf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841f9f00 con 0x7f0a70003980 2023-12-20T06:03:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841f64a0 con 0x7f0a841f5f90 2023-12-20T06:03:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a8b8fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a841f6780 con 0x7f0a841f2e50 2023-12-20T06:03:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.730+0000 7f0a89698640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 0x7f0a841f99c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a89698640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 0x7f0a841f99c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44286/0 (socket says 172.21.15.32:44286) 2023-12-20T06:03:37.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a89698640 1 -- 172.21.15.32:0/4118858897 learned_addr learned my addr 172.21.15.32:0/4118858897 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a88e97640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a841f2e50 0x7f0a841f5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a89e99640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a841f5f90 0x7f0a840caf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a89698640 1 -- 172.21.15.32:0/4118858897 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a841f2e50 msgr2=0x7f0a841f5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a89698640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a841f2e50 0x7f0a841f5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a89698640 1 -- 172.21.15.32:0/4118858897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a841f5f90 msgr2=0x7f0a840caf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a89698640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a841f5f90 0x7f0a840caf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a89698640 1 -- 172.21.15.32:0/4118858897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a841f91c0 con 0x7f0a70003980 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a89e99640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a841f5f90 0x7f0a840caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a89698640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 0x7f0a841f99c0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009af0 tx=0x7f0a800197c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a79ffb640 1 -- 172.21.15.32:0/4118858897 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80004020 con 0x7f0a70003980 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a79ffb640 1 -- 172.21.15.32:0/4118858897 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a800041a0 con 0x7f0a70003980 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a79ffb640 1 -- 172.21.15.32:0/4118858897 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a80003620 con 0x7f0a70003980 2023-12-20T06:03:37.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/4118858897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a841f94a0 con 0x7f0a70003980 2023-12-20T06:03:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/4118858897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a840c7f20 con 0x7f0a70003980 2023-12-20T06:03:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/4118858897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a840c8110 con 0x7f0a70003980 2023-12-20T06:03:37.736 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.734+0000 7f0a79ffb640 1 -- 172.21.15.32:0/4118858897 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f0a80003780 con 0x7f0a70003980 2023-12-20T06:03:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.738+0000 7f0a79ffb640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a68065cd0 0x7f0a68068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.738+0000 7f0a79ffb640 1 -- 172.21.15.32:0/4118858897 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f0a8011cd00 con 0x7f0a70003980 2023-12-20T06:03:37.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.738+0000 7f0a88e97640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a68065cd0 0x7f0a68068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:37.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.738+0000 7f0a88e97640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a68065cd0 0x7f0a68068190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a840c7ae0 tx=0x7f0a74007900 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:37.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.742+0000 7f0a79ffb640 1 -- 172.21.15.32:0/4118858897 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0a840c8110 con 0x7f0a70003980 2023-12-20T06:03:37.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.930+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/4118858897 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a840c8110 con 0x7f0a70003980 2023-12-20T06:03:37.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a79ffb640 1 -- 172.21.15.32:0/4118858897 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5909 (secure 0 0 0) 0x7f0a840c8110 con 0x7f0a70003980 2023-12-20T06:03:37.933 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:37.934 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:21.951240+0000","modified":"2023-12-20T06:03:24.980742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11301":{"gid":11301,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1623322556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1623322556},{"type":"v1","addr":"172.21.15.32:6837","nonce":1623322556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:34.513908+0000","modified":"2023-12-20T06:03:37.549965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11298":{"gid":11298,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6836/60134876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":60134876},{"type":"v1","addr":"172.21.15.32:6836","nonce":60134876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-20T06:03:37.934 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 118 2023-12-20T06:03:37.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/4118858897 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a68065cd0 msgr2=0x7f0a68068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:37.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a68065cd0 0x7f0a68068190 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f0a840c7ae0 tx=0x7f0a74007900 comp rx=0 tx=0).stop 2023-12-20T06:03:37.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/4118858897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 msgr2=0x7f0a841f99c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:37.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 0x7f0a841f99c0 secure :-1 s=READY pgs=2296 cs=0 l=1 rev1=1 crypto rx=0x7f0a80009af0 tx=0x7f0a800197c0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/4118858897 shutdown_connections 2023-12-20T06:03:37.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a68065cd0 0x7f0a68068190 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a841f5f90 0x7f0a840caf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a841f2e50 0x7f0a841f5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 --2- 172.21.15.32:0/4118858897 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a70003980 0x7f0a841f99c0 unknown :-1 s=CLOSED pgs=2296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:37.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/4118858897 >> 172.21.15.32:0/4118858897 conn(0x7f0a8406a320 msgr2=0x7f0a84158b60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:37.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/4118858897 shutdown_connections 2023-12-20T06:03:37.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:37.934+0000 7f0a8b8fc640 1 -- 172.21.15.32:0/4118858897 wait complete. 2023-12-20T06:03:37.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb70d1640 1 Processor -- start 2023-12-20T06:03:38.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb70d1640 1 -- start start 2023-12-20T06:03:38.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb70d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedb006ce60 0x7fedb015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb70d1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedb015f110 0x7fedb015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb70d1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fedb015bbd0 con 0x7fedb00af5e0 2023-12-20T06:03:38.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb70d1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fedb015beb0 con 0x7fedb015f110 2023-12-20T06:03:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb70d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb015e910 con 0x7fedb006ce60 2023-12-20T06:03:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb566e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedb015f110 0x7fedb015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb566e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fedb015f110 0x7fedb015c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44292/0 (socket says 172.21.15.32:44292) 2023-12-20T06:03:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb566e640 1 -- 172.21.15.32:0/3804747146 learned_addr learned my addr 172.21.15.32:0/3804747146 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7feda7fff640 1 --2- 172.21.15.32:0/3804747146 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedb006ce60 0x7fedb015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb566e640 1 -- 172.21.15.32:0/3804747146 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedb006ce60 msgr2=0x7fedb015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb566e640 1 --2- 172.21.15.32:0/3804747146 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedb006ce60 0x7fedb015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb566e640 1 -- 172.21.15.32:0/3804747146 >> v1:172.21.15.32:6789/0 conn(0x7fedb00af5e0 legacy=0x7fedb00af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.105 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7fedb566e640 1 -- 172.21.15.32:0/3804747146 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fedb015ebf0 con 0x7fedb015f110 2023-12-20T06:03:38.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.102+0000 7feda77fe640 1 -- 172.21.15.32:0/3804747146 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3655237975 0 0) 0x7fedb015bbd0 con 0x7fedb00af5e0 2023-12-20T06:03:38.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7fedb566e640 1 --2- 172.21.15.32:0/3804747146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb015f110 0x7fedb015c440 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7feda8009e50 tx=0x7feda801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cde40d7b36ebcb5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:38.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7feda77fe640 1 -- 172.21.15.32:0/3804747146 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda80a57e0 con 0x7fedb015f110 2023-12-20T06:03:38.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7feda77fe640 1 -- 172.21.15.32:0/3804747146 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda80a5940 con 0x7fedb015f110 2023-12-20T06:03:38.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7feda77fe640 1 -- 172.21.15.32:0/3804747146 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda80a5cd0 con 0x7fedb015f110 2023-12-20T06:03:38.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7fedb70d1640 1 -- 172.21.15.32:0/3804747146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb015f110 msgr2=0x7fedb015c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7fedb70d1640 1 --2- 172.21.15.32:0/3804747146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb015f110 0x7fedb015c440 secure :-1 s=READY pgs=2298 cs=0 l=1 rev1=1 crypto rx=0x7feda8009e50 tx=0x7feda801c980 comp rx=0 tx=0).stop 2023-12-20T06:03:38.108 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7fedb70d1640 1 -- 172.21.15.32:0/3804747146 shutdown_connections 2023-12-20T06:03:38.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7fedb70d1640 1 --2- 172.21.15.32:0/3804747146 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb015f110 0x7fedb015c440 unknown :-1 s=CLOSED pgs=2298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7fedb70d1640 1 --2- 172.21.15.32:0/3804747146 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedb006ce60 0x7fedb015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7fedb70d1640 1 -- 172.21.15.32:0/3804747146 >> 172.21.15.32:0/3804747146 conn(0x7fedb006a320 msgr2=0x7fedb00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:38.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7fedb70d1640 1 -- 172.21.15.32:0/3804747146 shutdown_connections 2023-12-20T06:03:38.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.106+0000 7fedb70d1640 1 -- 172.21.15.32:0/3804747146 wait complete. 2023-12-20T06:03:38.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb70d1640 1 Processor -- start 2023-12-20T06:03:38.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb70d1640 1 -- start start 2023-12-20T06:03:38.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb70d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedb006ce60 0x7fedb01f8bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb70d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00af5e0 0x7fedb015d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb70d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedb015f110 0x7fedb0150f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb70d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb01f9440 con 0x7fedb00af5e0 2023-12-20T06:03:38.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb70d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fedb01f9720 con 0x7fedb006ce60 2023-12-20T06:03:38.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb70d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fedb01f9a00 con 0x7fedb015f110 2023-12-20T06:03:38.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00af5e0 0x7fedb015d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00af5e0 0x7fedb015d3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44302/0 (socket says 172.21.15.32:44302) 2023-12-20T06:03:38.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda7fff640 1 -- 172.21.15.32:0/1792560160 learned_addr learned my addr 172.21.15.32:0/1792560160 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:38.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb566e640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedb015f110 0x7fedb0150f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb4e6d640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedb006ce60 0x7fedb01f8bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda7fff640 1 -- 172.21.15.32:0/1792560160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedb015f110 msgr2=0x7fedb0150f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda7fff640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedb015f110 0x7fedb0150f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda7fff640 1 -- 172.21.15.32:0/1792560160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedb006ce60 msgr2=0x7fedb01f8bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda7fff640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedb006ce60 0x7fedb01f8bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda7fff640 1 -- 172.21.15.32:0/1792560160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feda8009b30 con 0x7fedb00af5e0 2023-12-20T06:03:38.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb4e6d640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedb006ce60 0x7fedb01f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:38.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda7fff640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00af5e0 0x7fedb015d3f0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7feda000d5d0 tx=0x7feda000da90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:38.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda57fa640 1 -- 172.21.15.32:0/1792560160 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0004240 con 0x7fedb00af5e0 2023-12-20T06:03:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda57fa640 1 -- 172.21.15.32:0/1792560160 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feda00043a0 con 0x7fedb00af5e0 2023-12-20T06:03:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb70d1640 1 -- 172.21.15.32:0/1792560160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedb00b1d50 con 0x7fedb00af5e0 2023-12-20T06:03:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7feda57fa640 1 -- 172.21.15.32:0/1792560160 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feda0003840 con 0x7fedb00af5e0 2023-12-20T06:03:38.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.110+0000 7fedb70d1640 1 -- 172.21.15.32:0/1792560160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fedb01f9130 con 0x7fedb00af5e0 2023-12-20T06:03:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.114+0000 7fedb70d1640 1 -- 172.21.15.32:0/1792560160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fedb004e000 con 0x7fedb00af5e0 2023-12-20T06:03:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.114+0000 7feda57fa640 1 -- 172.21.15.32:0/1792560160 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7feda00039a0 con 0x7fedb00af5e0 2023-12-20T06:03:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.114+0000 7feda57fa640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fed9c065ca0 0x7fed9c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.114+0000 7feda57fa640 1 -- 172.21.15.32:0/1792560160 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7feda0083b60 con 0x7fedb00af5e0 2023-12-20T06:03:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.114+0000 7fedb4e6d640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fed9c065ca0 0x7fed9c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.114+0000 7fedb4e6d640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fed9c065ca0 0x7fed9c068160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fedac0017e0 tx=0x7fedac009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:38.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.118+0000 7feda57fa640 1 -- 172.21.15.32:0/1792560160 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feda0050d80 con 0x7fedb00af5e0 2023-12-20T06:03:38.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.278+0000 7fedb70d1640 1 -- 172.21.15.32:0/1792560160 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fedb015e070 con 0x7fedb00af5e0 2023-12-20T06:03:38.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.278+0000 7feda57fa640 1 -- 172.21.15.32:0/1792560160 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v234) v1 ==== 74+0+19817 (secure 0 0 0) 0x7feda00543a0 con 0x7fedb00af5e0 2023-12-20T06:03:38.280 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:38.280 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":234,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:35.521299+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:03:20.399088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-20T06:03:20.926367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:03:32.174574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:03:33.509224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:38.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 -- 172.21.15.32:0/1792560160 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fed9c065ca0 msgr2=0x7fed9c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fed9c065ca0 0x7fed9c068160 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fedac0017e0 tx=0x7fedac009040 comp rx=0 tx=0).stop 2023-12-20T06:03:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 -- 172.21.15.32:0/1792560160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00af5e0 msgr2=0x7fedb015d3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00af5e0 0x7fedb015d3f0 secure :-1 s=READY pgs=2299 cs=0 l=1 rev1=1 crypto rx=0x7feda000d5d0 tx=0x7feda000da90 comp rx=0 tx=0).stop 2023-12-20T06:03:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 -- 172.21.15.32:0/1792560160 shutdown_connections 2023-12-20T06:03:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fed9c065ca0 0x7fed9c068160 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fedb015f110 0x7fedb0150f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fedb00af5e0 0x7fedb015d3f0 unknown :-1 s=CLOSED pgs=2299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.282 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 --2- 172.21.15.32:0/1792560160 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fedb006ce60 0x7fedb01f8bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 -- 172.21.15.32:0/1792560160 >> 172.21.15.32:0/1792560160 conn(0x7fedb006a320 msgr2=0x7fedb00af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:38.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 -- 172.21.15.32:0/1792560160 shutdown_connections 2023-12-20T06:03:38.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.282+0000 7fedb70d1640 1 -- 172.21.15.32:0/1792560160 wait complete. 2023-12-20T06:03:38.301 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T06:03:38.301 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:03:38.376 INFO:tasks.ceph.mds.c.smithi032.stderr:2023-12-20T06:03:38.374+0000 7fd0fc8a5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T06:03:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa5697a2640 1 Processor -- start 2023-12-20T06:03:38.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa5697a2640 1 -- start start 2023-12-20T06:03:38.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa5697a2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5640b8480 0x7fa56406af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa5697a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa564161500 0x7fa564153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa5697a2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5640cd4d0 con 0x7fa56406b460 2023-12-20T06:03:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa5697a2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa564161930 con 0x7fa5640b8480 2023-12-20T06:03:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa5697a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa56406b8f0 con 0x7fa564161500 2023-12-20T06:03:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa563fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5640b8480 0x7fa56406af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa563fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa5640b8480 0x7fa56406af20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44312/0 (socket says 172.21.15.32:44312) 2023-12-20T06:03:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa563fff640 1 -- 172.21.15.32:0/3405831574 learned_addr learned my addr 172.21.15.32:0/3405831574 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa568fa1640 1 --2- 172.21.15.32:0/3405831574 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa564161500 0x7fa564153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa563fff640 1 -- 172.21.15.32:0/3405831574 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa564161500 msgr2=0x7fa564153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:38.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa563fff640 1 --2- 172.21.15.32:0/3405831574 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa564161500 0x7fa564153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa563fff640 1 -- 172.21.15.32:0/3405831574 >> v1:172.21.15.39:6789/0 conn(0x7fa56406b460 legacy=0x7fa564160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.449 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.446+0000 7fa563fff640 1 -- 172.21.15.32:0/3405831574 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5640ce4e0 con 0x7fa5640b8480 2023-12-20T06:03:38.450 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa563fff640 1 --2- 172.21.15.32:0/3405831574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5640b8480 0x7fa56406af20 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fa554009870 tx=0x7fa55401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bba711c4bc75a5e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:38.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa562ffd640 1 -- 172.21.15.32:0/3405831574 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5540a7070 con 0x7fa5640b8480 2023-12-20T06:03:38.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa562ffd640 1 -- 172.21.15.32:0/3405831574 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5540ab430 con 0x7fa5640b8480 2023-12-20T06:03:38.451 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa562ffd640 1 -- 172.21.15.32:0/3405831574 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5540ab7c0 con 0x7fa5640b8480 2023-12-20T06:03:38.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa5697a2640 1 -- 172.21.15.32:0/3405831574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5640b8480 msgr2=0x7fa56406af20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa5697a2640 1 --2- 172.21.15.32:0/3405831574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5640b8480 0x7fa56406af20 secure :-1 s=READY pgs=2300 cs=0 l=1 rev1=1 crypto rx=0x7fa554009870 tx=0x7fa55401c540 comp rx=0 tx=0).stop 2023-12-20T06:03:38.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa5697a2640 1 -- 172.21.15.32:0/3405831574 shutdown_connections 2023-12-20T06:03:38.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa5697a2640 1 --2- 172.21.15.32:0/3405831574 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa564161500 0x7fa564153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa5697a2640 1 --2- 172.21.15.32:0/3405831574 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5640b8480 0x7fa56406af20 unknown :-1 s=CLOSED pgs=2300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa5697a2640 1 -- 172.21.15.32:0/3405831574 >> 172.21.15.32:0/3405831574 conn(0x7fa5640d2a50 msgr2=0x7fa5640d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:38.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa5697a2640 1 -- 172.21.15.32:0/3405831574 shutdown_connections 2023-12-20T06:03:38.452 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.450+0000 7fa5697a2640 1 -- 172.21.15.32:0/3405831574 wait complete. 2023-12-20T06:03:38.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5697a2640 1 Processor -- start 2023-12-20T06:03:38.453 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5697a2640 1 -- start start 2023-12-20T06:03:38.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5697a2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56406b460 0x7fa56415f850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5697a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5640b8480 0x7fa564152910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5697a2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa564161500 0x7fa564148f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5697a2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56415fee0 con 0x7fa564161500 2023-12-20T06:03:38.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5697a2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa56414a570 con 0x7fa56406b460 2023-12-20T06:03:38.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5697a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa564149c40 con 0x7fa5640b8480 2023-12-20T06:03:38.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa568fa1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa564161500 0x7fa564148f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa568fa1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa564161500 0x7fa564148f00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:44316/0 (socket says 172.21.15.32:44316) 2023-12-20T06:03:38.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa563fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56406b460 0x7fa56415f850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa568fa1640 1 -- 172.21.15.32:0/3848002649 learned_addr learned my addr 172.21.15.32:0/3848002649 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:38.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5637fe640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5640b8480 0x7fa564152910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa568fa1640 1 -- 172.21.15.32:0/3848002649 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5640b8480 msgr2=0x7fa564152910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa568fa1640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5640b8480 0x7fa564152910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa568fa1640 1 -- 172.21.15.32:0/3848002649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56406b460 msgr2=0x7fa56415f850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa568fa1640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56406b460 0x7fa56415f850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa568fa1640 1 -- 172.21.15.32:0/3848002649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5540094d0 con 0x7fa564161500 2023-12-20T06:03:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5637fe640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5640b8480 0x7fa564152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa568fa1640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa564161500 0x7fa564148f00 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009d30 tx=0x7fa55c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa560ff9640 1 -- 172.21.15.32:0/3848002649 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c009ba0 con 0x7fa564161500 2023-12-20T06:03:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa560ff9640 1 -- 172.21.15.32:0/3848002649 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa55c00f040 con 0x7fa564161500 2023-12-20T06:03:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa560ff9640 1 -- 172.21.15.32:0/3848002649 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa55c0146c0 con 0x7fa564161500 2023-12-20T06:03:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5697a2640 1 -- 172.21.15.32:0/3848002649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa564149f20 con 0x7fa564161500 2023-12-20T06:03:38.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.454+0000 7fa5697a2640 1 -- 172.21.15.32:0/3848002649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5640cccf0 con 0x7fa564161500 2023-12-20T06:03:38.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.458+0000 7fa5697a2640 1 -- 172.21.15.32:0/3848002649 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa564074100 con 0x7fa564161500 2023-12-20T06:03:38.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.458+0000 7fa560ff9640 1 -- 172.21.15.32:0/3848002649 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fa55c014820 con 0x7fa564161500 2023-12-20T06:03:38.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.458+0000 7fa560ff9640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa550065cd0 0x7fa550068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.458+0000 7fa560ff9640 1 -- 172.21.15.32:0/3848002649 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7fa55c084500 con 0x7fa564161500 2023-12-20T06:03:38.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.458+0000 7fa563fff640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa550065cd0 0x7fa550068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.458+0000 7fa563fff640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa550065cd0 0x7fa550068190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa5540af000 tx=0x7fa554008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:38.463 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.462+0000 7fa560ff9640 1 -- 172.21.15.32:0/3848002649 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa55c0516a0 con 0x7fa564161500 2023-12-20T06:03:38.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.622+0000 7fa5697a2640 1 -- 172.21.15.32:0/3848002649 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fa5640cdfc0 con 0x7fa550065cd0 2023-12-20T06:03:38.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.638+0000 7fa560ff9640 1 -- 172.21.15.32:0/3848002649 <== mgr.10956 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528501 (secure 0 0 0) 0x7fa5540bc090 con 0x7fa550065cd0 2023-12-20T06:03:38.645 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:38.651 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:03:38.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.642+0000 7fa5697a2640 1 -- 172.21.15.32:0/3848002649 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa550065cd0 msgr2=0x7fa550068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.642+0000 7fa5697a2640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa550065cd0 0x7fa550068190 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7fa5540af000 tx=0x7fa554008040 comp rx=0 tx=0).stop 2023-12-20T06:03:38.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.642+0000 7fa5697a2640 1 -- 172.21.15.32:0/3848002649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa564161500 msgr2=0x7fa564148f00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.642+0000 7fa5697a2640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa564161500 0x7fa564148f00 secure :-1 s=READY pgs=2301 cs=0 l=1 rev1=1 crypto rx=0x7fa55c009d30 tx=0x7fa55c00c490 comp rx=0 tx=0).stop 2023-12-20T06:03:38.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.642+0000 7fa5697a2640 1 -- 172.21.15.32:0/3848002649 shutdown_connections 2023-12-20T06:03:38.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.642+0000 7fa5697a2640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa550065cd0 0x7fa550068190 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.642+0000 7fa5697a2640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa564161500 0x7fa564148f00 unknown :-1 s=CLOSED pgs=2301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.642+0000 7fa5697a2640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5640b8480 0x7fa564152910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.642+0000 7fa5697a2640 1 --2- 172.21.15.32:0/3848002649 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa56406b460 0x7fa56415f850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.642+0000 7fa5697a2640 1 -- 172.21.15.32:0/3848002649 >> 172.21.15.32:0/3848002649 conn(0x7fa5640d2a50 msgr2=0x7fa5640b7130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:38.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.646+0000 7fa5697a2640 1 -- 172.21.15.32:0/3848002649 shutdown_connections 2023-12-20T06:03:38.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.646+0000 7fa5697a2640 1 -- 172.21.15.32:0/3848002649 wait complete. 2023-12-20T06:03:38.665 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-12-20T06:03:37.661112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":652,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3302,"ondisk_log_size":3302,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":429224,"kb_used_data":24168,"kb_used_omap":0,"kb_used_meta":404992,"kb_avail":754545496,"statfs":{"total":773094113280,"available":772654587904,"internally_reserved":0,"allocated":24748032,"data_stored":12661922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":414711808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3349,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.570006"},"pg_stats":[{"pgid":"31.3d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574240+0000","last_change":"2023-12-20T06:03:34.545959+0000","last_active":"2023-12-20T06:03:35.574240+0000","last_peered":"2023-12-20T06:03:35.574240+0000","last_clean":"2023-12-20T06:03:35.574240+0000","last_became_active":"2023-12-20T06:03:34.545677+0000","last_became_peered":"2023-12-20T06:03:34.545677+0000","last_unstale":"2023-12-20T06:03:35.574240+0000","last_undegraded":"2023-12-20T06:03:35.574240+0000","last_fullsized":"2023-12-20T06:03:35.574240+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:54:36.284984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.541147+0000","last_change":"2023-12-20T06:03:21.976746+0000","last_active":"2023-12-20T06:03:34.541147+0000","last_peered":"2023-12-20T06:03:34.541147+0000","last_clean":"2023-12-20T06:03:34.541147+0000","last_became_active":"2023-12-20T06:03:21.976467+0000","last_became_peered":"2023-12-20T06:03:21.976467+0000","last_unstale":"2023-12-20T06:03:34.541147+0000","last_undegraded":"2023-12-20T06:03:34.541147+0000","last_fullsized":"2023-12-20T06:03:34.541147+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:13.213910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573792+0000","last_change":"2023-12-20T06:03:34.547546+0000","last_active":"2023-12-20T06:03:35.573792+0000","last_peered":"2023-12-20T06:03:35.573792+0000","last_clean":"2023-12-20T06:03:35.573792+0000","last_became_active":"2023-12-20T06:03:34.547300+0000","last_became_peered":"2023-12-20T06:03:34.547300+0000","last_unstale":"2023-12-20T06:03:35.573792+0000","last_undegraded":"2023-12-20T06:03:35.573792+0000","last_fullsized":"2023-12-20T06:03:35.573792+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:28.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.542778+0000","last_change":"2023-12-20T06:03:21.988965+0000","last_active":"2023-12-20T06:03:34.542778+0000","last_peered":"2023-12-20T06:03:34.542778+0000","last_clean":"2023-12-20T06:03:34.542778+0000","last_became_active":"2023-12-20T06:03:21.988334+0000","last_became_peered":"2023-12-20T06:03:21.988334+0000","last_unstale":"2023-12-20T06:03:34.542778+0000","last_undegraded":"2023-12-20T06:03:34.542778+0000","last_fullsized":"2023-12-20T06:03:34.542778+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:04.642017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575378+0000","last_change":"2023-12-20T06:03:34.556254+0000","last_active":"2023-12-20T06:03:35.575378+0000","last_peered":"2023-12-20T06:03:35.575378+0000","last_clean":"2023-12-20T06:03:35.575378+0000","last_became_active":"2023-12-20T06:03:34.555988+0000","last_became_peered":"2023-12-20T06:03:34.555988+0000","last_unstale":"2023-12-20T06:03:35.575378+0000","last_undegraded":"2023-12-20T06:03:35.575378+0000","last_fullsized":"2023-12-20T06:03:35.575378+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:42.632244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530357+0000","last_change":"2023-12-20T06:03:21.987434+0000","last_active":"2023-12-20T06:03:34.530357+0000","last_peered":"2023-12-20T06:03:34.530357+0000","last_clean":"2023-12-20T06:03:34.530357+0000","last_became_active":"2023-12-20T06:03:21.987118+0000","last_became_peered":"2023-12-20T06:03:21.987118+0000","last_unstale":"2023-12-20T06:03:34.530357+0000","last_undegraded":"2023-12-20T06:03:34.530357+0000","last_fullsized":"2023-12-20T06:03:34.530357+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575165+0000","last_change":"2023-12-20T06:03:34.549559+0000","last_active":"2023-12-20T06:03:35.575165+0000","last_peered":"2023-12-20T06:03:35.575165+0000","last_clean":"2023-12-20T06:03:35.575165+0000","last_became_active":"2023-12-20T06:03:34.549296+0000","last_became_peered":"2023-12-20T06:03:34.549296+0000","last_unstale":"2023-12-20T06:03:35.575165+0000","last_undegraded":"2023-12-20T06:03:35.575165+0000","last_fullsized":"2023-12-20T06:03:35.575165+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:39:44.183895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.535103+0000","last_change":"2023-12-20T06:03:21.981328+0000","last_active":"2023-12-20T06:03:34.535103+0000","last_peered":"2023-12-20T06:03:34.535103+0000","last_clean":"2023-12-20T06:03:34.535103+0000","last_became_active":"2023-12-20T06:03:21.981001+0000","last_became_peered":"2023-12-20T06:03:21.981001+0000","last_unstale":"2023-12-20T06:03:34.535103+0000","last_undegraded":"2023-12-20T06:03:34.535103+0000","last_fullsized":"2023-12-20T06:03:34.535103+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:48.782916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570635+0000","last_change":"2023-12-20T06:03:34.560291+0000","last_active":"2023-12-20T06:03:35.570635+0000","last_peered":"2023-12-20T06:03:35.570635+0000","last_clean":"2023-12-20T06:03:35.570635+0000","last_became_active":"2023-12-20T06:03:34.559999+0000","last_became_peered":"2023-12-20T06:03:34.559999+0000","last_unstale":"2023-12-20T06:03:35.570635+0000","last_undegraded":"2023-12-20T06:03:35.570635+0000","last_fullsized":"2023-12-20T06:03:35.570635+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:08:10.768420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537739+0000","last_change":"2023-12-20T06:03:21.978878+0000","last_active":"2023-12-20T06:03:34.537739+0000","last_peered":"2023-12-20T06:03:34.537739+0000","last_clean":"2023-12-20T06:03:34.537739+0000","last_became_active":"2023-12-20T06:03:21.977017+0000","last_became_peered":"2023-12-20T06:03:21.977017+0000","last_unstale":"2023-12-20T06:03:34.537739+0000","last_undegraded":"2023-12-20T06:03:34.537739+0000","last_fullsized":"2023-12-20T06:03:34.537739+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:55:10.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536476+0000","last_change":"2023-12-20T06:03:34.543028+0000","last_active":"2023-12-20T06:03:35.536476+0000","last_peered":"2023-12-20T06:03:35.536476+0000","last_clean":"2023-12-20T06:03:35.536476+0000","last_became_active":"2023-12-20T06:03:34.542240+0000","last_became_peered":"2023-12-20T06:03:34.542240+0000","last_unstale":"2023-12-20T06:03:35.536476+0000","last_undegraded":"2023-12-20T06:03:35.536476+0000","last_fullsized":"2023-12-20T06:03:35.536476+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:46:47.498001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.524859+0000","last_change":"2023-12-20T06:03:21.976982+0000","last_active":"2023-12-20T06:03:34.524859+0000","last_peered":"2023-12-20T06:03:34.524859+0000","last_clean":"2023-12-20T06:03:34.524859+0000","last_became_active":"2023-12-20T06:03:21.976721+0000","last_became_peered":"2023-12-20T06:03:21.976721+0000","last_unstale":"2023-12-20T06:03:34.524859+0000","last_undegraded":"2023-12-20T06:03:34.524859+0000","last_fullsized":"2023-12-20T06:03:34.524859+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:57.966779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.534878+0000","last_change":"2023-12-20T06:03:34.539532+0000","last_active":"2023-12-20T06:03:35.534878+0000","last_peered":"2023-12-20T06:03:35.534878+0000","last_clean":"2023-12-20T06:03:35.534878+0000","last_became_active":"2023-12-20T06:03:34.539195+0000","last_became_peered":"2023-12-20T06:03:34.539195+0000","last_unstale":"2023-12-20T06:03:35.534878+0000","last_undegraded":"2023-12-20T06:03:35.534878+0000","last_fullsized":"2023-12-20T06:03:35.534878+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:25:10.820979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529760+0000","last_change":"2023-12-20T06:03:21.979840+0000","last_active":"2023-12-20T06:03:34.529760+0000","last_peered":"2023-12-20T06:03:34.529760+0000","last_clean":"2023-12-20T06:03:34.529760+0000","last_became_active":"2023-12-20T06:03:21.979570+0000","last_became_peered":"2023-12-20T06:03:21.979570+0000","last_unstale":"2023-12-20T06:03:34.529760+0000","last_undegraded":"2023-12-20T06:03:34.529760+0000","last_fullsized":"2023-12-20T06:03:34.529760+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572992+0000","last_change":"2023-12-20T06:03:34.555425+0000","last_active":"2023-12-20T06:03:35.572992+0000","last_peered":"2023-12-20T06:03:35.572992+0000","last_clean":"2023-12-20T06:03:35.572992+0000","last_became_active":"2023-12-20T06:03:34.555195+0000","last_became_peered":"2023-12-20T06:03:34.555195+0000","last_unstale":"2023-12-20T06:03:35.572992+0000","last_undegraded":"2023-12-20T06:03:35.572992+0000","last_fullsized":"2023-12-20T06:03:35.572992+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:53:54.171520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530147+0000","last_change":"2023-12-20T06:03:21.976244+0000","last_active":"2023-12-20T06:03:34.530147+0000","last_peered":"2023-12-20T06:03:34.530147+0000","last_clean":"2023-12-20T06:03:34.530147+0000","last_became_active":"2023-12-20T06:03:21.975968+0000","last_became_peered":"2023-12-20T06:03:21.975968+0000","last_unstale":"2023-12-20T06:03:34.530147+0000","last_undegraded":"2023-12-20T06:03:34.530147+0000","last_fullsized":"2023-12-20T06:03:34.530147+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:07:21.324505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535762+0000","last_change":"2023-12-20T06:03:34.553662+0000","last_active":"2023-12-20T06:03:35.535762+0000","last_peered":"2023-12-20T06:03:35.535762+0000","last_clean":"2023-12-20T06:03:35.535762+0000","last_became_active":"2023-12-20T06:03:34.553406+0000","last_became_peered":"2023-12-20T06:03:34.553406+0000","last_unstale":"2023-12-20T06:03:35.535762+0000","last_undegraded":"2023-12-20T06:03:35.535762+0000","last_fullsized":"2023-12-20T06:03:35.535762+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:22:40.617264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530821+0000","last_change":"2023-12-20T06:03:21.987598+0000","last_active":"2023-12-20T06:03:34.530821+0000","last_peered":"2023-12-20T06:03:34.530821+0000","last_clean":"2023-12-20T06:03:34.530821+0000","last_became_active":"2023-12-20T06:03:21.987291+0000","last_became_peered":"2023-12-20T06:03:21.987291+0000","last_unstale":"2023-12-20T06:03:34.530821+0000","last_undegraded":"2023-12-20T06:03:34.530821+0000","last_fullsized":"2023-12-20T06:03:34.530821+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:04.642017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536872+0000","last_change":"2023-12-20T06:03:34.554758+0000","last_active":"2023-12-20T06:03:35.536872+0000","last_peered":"2023-12-20T06:03:35.536872+0000","last_clean":"2023-12-20T06:03:35.536872+0000","last_became_active":"2023-12-20T06:03:34.553805+0000","last_became_peered":"2023-12-20T06:03:34.553805+0000","last_unstale":"2023-12-20T06:03:35.536872+0000","last_undegraded":"2023-12-20T06:03:35.536872+0000","last_fullsized":"2023-12-20T06:03:35.536872+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:38:13.606277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.526390+0000","last_change":"2023-12-20T06:03:21.969809+0000","last_active":"2023-12-20T06:03:34.526390+0000","last_peered":"2023-12-20T06:03:34.526390+0000","last_clean":"2023-12-20T06:03:34.526390+0000","last_became_active":"2023-12-20T06:03:21.969515+0000","last_became_peered":"2023-12-20T06:03:21.969515+0000","last_unstale":"2023-12-20T06:03:34.526390+0000","last_undegraded":"2023-12-20T06:03:34.526390+0000","last_fullsized":"2023-12-20T06:03:34.526390+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:15:47.777614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572576+0000","last_change":"2023-12-20T06:03:34.559372+0000","last_active":"2023-12-20T06:03:35.572576+0000","last_peered":"2023-12-20T06:03:35.572576+0000","last_clean":"2023-12-20T06:03:35.572576+0000","last_became_active":"2023-12-20T06:03:34.559122+0000","last_became_peered":"2023-12-20T06:03:34.559122+0000","last_unstale":"2023-12-20T06:03:35.572576+0000","last_undegraded":"2023-12-20T06:03:35.572576+0000","last_fullsized":"2023-12-20T06:03:35.572576+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:23:20.670623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540624+0000","last_change":"2023-12-20T06:03:21.979757+0000","last_active":"2023-12-20T06:03:34.540624+0000","last_peered":"2023-12-20T06:03:34.540624+0000","last_clean":"2023-12-20T06:03:34.540624+0000","last_became_active":"2023-12-20T06:03:21.979456+0000","last_became_peered":"2023-12-20T06:03:21.979456+0000","last_unstale":"2023-12-20T06:03:34.540624+0000","last_undegraded":"2023-12-20T06:03:34.540624+0000","last_fullsized":"2023-12-20T06:03:34.540624+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:23.805328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571721+0000","last_change":"2023-12-20T06:03:34.537769+0000","last_active":"2023-12-20T06:03:35.571721+0000","last_peered":"2023-12-20T06:03:35.571721+0000","last_clean":"2023-12-20T06:03:35.571721+0000","last_became_active":"2023-12-20T06:03:34.537481+0000","last_became_peered":"2023-12-20T06:03:34.537481+0000","last_unstale":"2023-12-20T06:03:35.571721+0000","last_undegraded":"2023-12-20T06:03:35.571721+0000","last_fullsized":"2023-12-20T06:03:35.571721+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:04:55.642946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540635+0000","last_change":"2023-12-20T06:03:21.982252+0000","last_active":"2023-12-20T06:03:34.540635+0000","last_peered":"2023-12-20T06:03:34.540635+0000","last_clean":"2023-12-20T06:03:34.540635+0000","last_became_active":"2023-12-20T06:03:21.981980+0000","last_became_peered":"2023-12-20T06:03:21.981980+0000","last_unstale":"2023-12-20T06:03:34.540635+0000","last_undegraded":"2023-12-20T06:03:34.540635+0000","last_fullsized":"2023-12-20T06:03:34.540635+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:13.633088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570380+0000","last_change":"2023-12-20T06:03:34.554260+0000","last_active":"2023-12-20T06:03:35.570380+0000","last_peered":"2023-12-20T06:03:35.570380+0000","last_clean":"2023-12-20T06:03:35.570380+0000","last_became_active":"2023-12-20T06:03:34.553373+0000","last_became_peered":"2023-12-20T06:03:34.553373+0000","last_unstale":"2023-12-20T06:03:35.570380+0000","last_undegraded":"2023-12-20T06:03:35.570380+0000","last_fullsized":"2023-12-20T06:03:35.570380+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:14:15.201659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.539684+0000","last_change":"2023-12-20T06:03:21.975662+0000","last_active":"2023-12-20T06:03:34.539684+0000","last_peered":"2023-12-20T06:03:34.539684+0000","last_clean":"2023-12-20T06:03:34.539684+0000","last_became_active":"2023-12-20T06:03:21.975017+0000","last_became_peered":"2023-12-20T06:03:21.975017+0000","last_unstale":"2023-12-20T06:03:34.539684+0000","last_undegraded":"2023-12-20T06:03:34.539684+0000","last_fullsized":"2023-12-20T06:03:34.539684+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:06:16.446428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572375+0000","last_change":"2023-12-20T06:03:34.532302+0000","last_active":"2023-12-20T06:03:35.572375+0000","last_peered":"2023-12-20T06:03:35.572375+0000","last_clean":"2023-12-20T06:03:35.572375+0000","last_became_active":"2023-12-20T06:03:34.532026+0000","last_became_peered":"2023-12-20T06:03:34.532026+0000","last_unstale":"2023-12-20T06:03:35.572375+0000","last_undegraded":"2023-12-20T06:03:35.572375+0000","last_fullsized":"2023-12-20T06:03:35.572375+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:12.020921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528317+0000","last_change":"2023-12-20T06:03:21.973459+0000","last_active":"2023-12-20T06:03:34.528317+0000","last_peered":"2023-12-20T06:03:34.528317+0000","last_clean":"2023-12-20T06:03:34.528317+0000","last_became_active":"2023-12-20T06:03:21.973101+0000","last_became_peered":"2023-12-20T06:03:21.973101+0000","last_unstale":"2023-12-20T06:03:34.528317+0000","last_undegraded":"2023-12-20T06:03:34.528317+0000","last_fullsized":"2023-12-20T06:03:34.528317+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:03:34.663696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571294+0000","last_change":"2023-12-20T06:03:34.553541+0000","last_active":"2023-12-20T06:03:35.571294+0000","last_peered":"2023-12-20T06:03:35.571294+0000","last_clean":"2023-12-20T06:03:35.571294+0000","last_became_active":"2023-12-20T06:03:34.553268+0000","last_became_peered":"2023-12-20T06:03:34.553268+0000","last_unstale":"2023-12-20T06:03:35.571294+0000","last_undegraded":"2023-12-20T06:03:35.571294+0000","last_fullsized":"2023-12-20T06:03:35.571294+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:27:22.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537295+0000","last_change":"2023-12-20T06:03:21.982673+0000","last_active":"2023-12-20T06:03:34.537295+0000","last_peered":"2023-12-20T06:03:34.537295+0000","last_clean":"2023-12-20T06:03:34.537295+0000","last_became_active":"2023-12-20T06:03:21.982010+0000","last_became_peered":"2023-12-20T06:03:21.982010+0000","last_unstale":"2023-12-20T06:03:34.537295+0000","last_undegraded":"2023-12-20T06:03:34.537295+0000","last_fullsized":"2023-12-20T06:03:34.537295+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:32:19.664417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572826+0000","last_change":"2023-12-20T06:03:34.544343+0000","last_active":"2023-12-20T06:03:35.572826+0000","last_peered":"2023-12-20T06:03:35.572826+0000","last_clean":"2023-12-20T06:03:35.572826+0000","last_became_active":"2023-12-20T06:03:34.544066+0000","last_became_peered":"2023-12-20T06:03:34.544066+0000","last_unstale":"2023-12-20T06:03:35.572826+0000","last_undegraded":"2023-12-20T06:03:35.572826+0000","last_fullsized":"2023-12-20T06:03:35.572826+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:49:46.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537888+0000","last_change":"2023-12-20T06:03:21.977347+0000","last_active":"2023-12-20T06:03:34.537888+0000","last_peered":"2023-12-20T06:03:34.537888+0000","last_clean":"2023-12-20T06:03:34.537888+0000","last_became_active":"2023-12-20T06:03:21.977040+0000","last_became_peered":"2023-12-20T06:03:21.977040+0000","last_unstale":"2023-12-20T06:03:34.537888+0000","last_undegraded":"2023-12-20T06:03:34.537888+0000","last_fullsized":"2023-12-20T06:03:34.537888+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.537333+0000","last_change":"2023-12-20T06:03:34.554193+0000","last_active":"2023-12-20T06:03:35.537333+0000","last_peered":"2023-12-20T06:03:35.537333+0000","last_clean":"2023-12-20T06:03:35.537333+0000","last_became_active":"2023-12-20T06:03:34.553937+0000","last_became_peered":"2023-12-20T06:03:34.553937+0000","last_unstale":"2023-12-20T06:03:35.537333+0000","last_undegraded":"2023-12-20T06:03:35.537333+0000","last_fullsized":"2023-12-20T06:03:35.537333+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:39:44.183895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529300+0000","last_change":"2023-12-20T06:03:21.973052+0000","last_active":"2023-12-20T06:03:34.529300+0000","last_peered":"2023-12-20T06:03:34.529300+0000","last_clean":"2023-12-20T06:03:34.529300+0000","last_became_active":"2023-12-20T06:03:21.972760+0000","last_became_peered":"2023-12-20T06:03:21.972760+0000","last_unstale":"2023-12-20T06:03:34.529300+0000","last_undegraded":"2023-12-20T06:03:34.529300+0000","last_fullsized":"2023-12-20T06:03:34.529300+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:16:41.285330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574145+0000","last_change":"2023-12-20T06:03:34.549623+0000","last_active":"2023-12-20T06:03:35.574145+0000","last_peered":"2023-12-20T06:03:35.574145+0000","last_clean":"2023-12-20T06:03:35.574145+0000","last_became_active":"2023-12-20T06:03:34.549164+0000","last_became_peered":"2023-12-20T06:03:34.549164+0000","last_unstale":"2023-12-20T06:03:35.574145+0000","last_undegraded":"2023-12-20T06:03:35.574145+0000","last_fullsized":"2023-12-20T06:03:35.574145+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:41:30.995579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.526390+0000","last_change":"2023-12-20T06:03:21.991211+0000","last_active":"2023-12-20T06:03:34.526390+0000","last_peered":"2023-12-20T06:03:34.526390+0000","last_clean":"2023-12-20T06:03:34.526390+0000","last_became_active":"2023-12-20T06:03:21.990873+0000","last_became_peered":"2023-12-20T06:03:21.990873+0000","last_unstale":"2023-12-20T06:03:34.526390+0000","last_undegraded":"2023-12-20T06:03:34.526390+0000","last_fullsized":"2023-12-20T06:03:34.526390+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:31:08.457333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.537415+0000","last_change":"2023-12-20T06:03:34.546022+0000","last_active":"2023-12-20T06:03:35.537415+0000","last_peered":"2023-12-20T06:03:35.537415+0000","last_clean":"2023-12-20T06:03:35.537415+0000","last_became_active":"2023-12-20T06:03:34.545718+0000","last_became_peered":"2023-12-20T06:03:34.545718+0000","last_unstale":"2023-12-20T06:03:35.537415+0000","last_undegraded":"2023-12-20T06:03:35.537415+0000","last_fullsized":"2023-12-20T06:03:35.537415+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:56:54.852713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534615+0000","last_change":"2023-12-20T06:03:21.982366+0000","last_active":"2023-12-20T06:03:34.534615+0000","last_peered":"2023-12-20T06:03:34.534615+0000","last_clean":"2023-12-20T06:03:34.534615+0000","last_became_active":"2023-12-20T06:03:21.978098+0000","last_became_peered":"2023-12-20T06:03:21.978098+0000","last_unstale":"2023-12-20T06:03:34.534615+0000","last_undegraded":"2023-12-20T06:03:34.534615+0000","last_fullsized":"2023-12-20T06:03:34.534615+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575201+0000","last_change":"2023-12-20T06:03:34.553869+0000","last_active":"2023-12-20T06:03:35.575201+0000","last_peered":"2023-12-20T06:03:35.575201+0000","last_clean":"2023-12-20T06:03:35.575201+0000","last_became_active":"2023-12-20T06:03:34.553639+0000","last_became_peered":"2023-12-20T06:03:34.553639+0000","last_unstale":"2023-12-20T06:03:35.575201+0000","last_undegraded":"2023-12-20T06:03:35.575201+0000","last_fullsized":"2023-12-20T06:03:35.575201+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:34:13.655323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.525145+0000","last_change":"2023-12-20T06:03:21.967196+0000","last_active":"2023-12-20T06:03:34.525145+0000","last_peered":"2023-12-20T06:03:34.525145+0000","last_clean":"2023-12-20T06:03:34.525145+0000","last_became_active":"2023-12-20T06:03:21.966332+0000","last_became_peered":"2023-12-20T06:03:21.966332+0000","last_unstale":"2023-12-20T06:03:34.525145+0000","last_undegraded":"2023-12-20T06:03:34.525145+0000","last_fullsized":"2023-12-20T06:03:34.525145+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:51:08.151013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536617+0000","last_change":"2023-12-20T06:03:34.543964+0000","last_active":"2023-12-20T06:03:35.536617+0000","last_peered":"2023-12-20T06:03:35.536617+0000","last_clean":"2023-12-20T06:03:35.536617+0000","last_became_active":"2023-12-20T06:03:34.543592+0000","last_became_peered":"2023-12-20T06:03:34.543592+0000","last_unstale":"2023-12-20T06:03:35.536617+0000","last_undegraded":"2023-12-20T06:03:35.536617+0000","last_fullsized":"2023-12-20T06:03:35.536617+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:02.206635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537812+0000","last_change":"2023-12-20T06:03:21.977328+0000","last_active":"2023-12-20T06:03:34.537812+0000","last_peered":"2023-12-20T06:03:34.537812+0000","last_clean":"2023-12-20T06:03:34.537812+0000","last_became_active":"2023-12-20T06:03:21.976722+0000","last_became_peered":"2023-12-20T06:03:21.976722+0000","last_unstale":"2023-12-20T06:03:34.537812+0000","last_undegraded":"2023-12-20T06:03:34.537812+0000","last_fullsized":"2023-12-20T06:03:34.537812+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:35:37.344273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572450+0000","last_change":"2023-12-20T06:03:34.559075+0000","last_active":"2023-12-20T06:03:35.572450+0000","last_peered":"2023-12-20T06:03:35.572450+0000","last_clean":"2023-12-20T06:03:35.572450+0000","last_became_active":"2023-12-20T06:03:34.556828+0000","last_became_peered":"2023-12-20T06:03:34.556828+0000","last_unstale":"2023-12-20T06:03:35.572450+0000","last_undegraded":"2023-12-20T06:03:35.572450+0000","last_fullsized":"2023-12-20T06:03:35.572450+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:46:47.498001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537560+0000","last_change":"2023-12-20T06:03:21.974137+0000","last_active":"2023-12-20T06:03:34.537560+0000","last_peered":"2023-12-20T06:03:34.537560+0000","last_clean":"2023-12-20T06:03:34.537560+0000","last_became_active":"2023-12-20T06:03:21.973889+0000","last_became_peered":"2023-12-20T06:03:21.973889+0000","last_unstale":"2023-12-20T06:03:34.537560+0000","last_undegraded":"2023-12-20T06:03:34.537560+0000","last_fullsized":"2023-12-20T06:03:34.537560+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:08:46.618627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536842+0000","last_change":"2023-12-20T06:03:34.541415+0000","last_active":"2023-12-20T06:03:35.536842+0000","last_peered":"2023-12-20T06:03:35.536842+0000","last_clean":"2023-12-20T06:03:35.536842+0000","last_became_active":"2023-12-20T06:03:34.541154+0000","last_became_peered":"2023-12-20T06:03:34.541154+0000","last_unstale":"2023-12-20T06:03:35.536842+0000","last_undegraded":"2023-12-20T06:03:35.536842+0000","last_fullsized":"2023-12-20T06:03:35.536842+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:54:48.032421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.538161+0000","last_change":"2023-12-20T06:03:21.967911+0000","last_active":"2023-12-20T06:03:34.538161+0000","last_peered":"2023-12-20T06:03:34.538161+0000","last_clean":"2023-12-20T06:03:34.538161+0000","last_became_active":"2023-12-20T06:03:21.967093+0000","last_became_peered":"2023-12-20T06:03:21.967093+0000","last_unstale":"2023-12-20T06:03:34.538161+0000","last_undegraded":"2023-12-20T06:03:34.538161+0000","last_fullsized":"2023-12-20T06:03:34.538161+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:43:05.571007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570788+0000","last_change":"2023-12-20T06:03:34.539342+0000","last_active":"2023-12-20T06:03:35.570788+0000","last_peered":"2023-12-20T06:03:35.570788+0000","last_clean":"2023-12-20T06:03:35.570788+0000","last_became_active":"2023-12-20T06:03:34.539047+0000","last_became_peered":"2023-12-20T06:03:34.539047+0000","last_unstale":"2023-12-20T06:03:35.570788+0000","last_undegraded":"2023-12-20T06:03:35.570788+0000","last_fullsized":"2023-12-20T06:03:35.570788+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:04:45.298268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534526+0000","last_change":"2023-12-20T06:03:21.974077+0000","last_active":"2023-12-20T06:03:34.534526+0000","last_peered":"2023-12-20T06:03:34.534526+0000","last_clean":"2023-12-20T06:03:34.534526+0000","last_became_active":"2023-12-20T06:03:21.973810+0000","last_became_peered":"2023-12-20T06:03:21.973810+0000","last_unstale":"2023-12-20T06:03:34.534526+0000","last_undegraded":"2023-12-20T06:03:34.534526+0000","last_fullsized":"2023-12-20T06:03:34.534526+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:00.841811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535821+0000","last_change":"2023-12-20T06:03:34.553105+0000","last_active":"2023-12-20T06:03:35.535821+0000","last_peered":"2023-12-20T06:03:35.535821+0000","last_clean":"2023-12-20T06:03:35.535821+0000","last_became_active":"2023-12-20T06:03:34.552811+0000","last_became_peered":"2023-12-20T06:03:34.552811+0000","last_unstale":"2023-12-20T06:03:35.535821+0000","last_undegraded":"2023-12-20T06:03:35.535821+0000","last_fullsized":"2023-12-20T06:03:35.535821+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:53:17.378650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.541237+0000","last_change":"2023-12-20T06:03:21.976310+0000","last_active":"2023-12-20T06:03:34.541237+0000","last_peered":"2023-12-20T06:03:34.541237+0000","last_clean":"2023-12-20T06:03:34.541237+0000","last_became_active":"2023-12-20T06:03:21.976042+0000","last_became_peered":"2023-12-20T06:03:21.976042+0000","last_unstale":"2023-12-20T06:03:34.541237+0000","last_undegraded":"2023-12-20T06:03:34.541237+0000","last_fullsized":"2023-12-20T06:03:34.541237+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:46.204136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573869+0000","last_change":"2023-12-20T06:03:34.559246+0000","last_active":"2023-12-20T06:03:35.573869+0000","last_peered":"2023-12-20T06:03:35.573869+0000","last_clean":"2023-12-20T06:03:35.573869+0000","last_became_active":"2023-12-20T06:03:34.558427+0000","last_became_peered":"2023-12-20T06:03:34.558427+0000","last_unstale":"2023-12-20T06:03:35.573869+0000","last_undegraded":"2023-12-20T06:03:35.573869+0000","last_fullsized":"2023-12-20T06:03:35.573869+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:29:03.073420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531003+0000","last_change":"2023-12-20T06:03:21.981857+0000","last_active":"2023-12-20T06:03:34.531003+0000","last_peered":"2023-12-20T06:03:34.531003+0000","last_clean":"2023-12-20T06:03:34.531003+0000","last_became_active":"2023-12-20T06:03:21.981595+0000","last_became_peered":"2023-12-20T06:03:21.981595+0000","last_unstale":"2023-12-20T06:03:34.531003+0000","last_undegraded":"2023-12-20T06:03:34.531003+0000","last_fullsized":"2023-12-20T06:03:34.531003+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:39:40.018104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575357+0000","last_change":"2023-12-20T06:03:34.546245+0000","last_active":"2023-12-20T06:03:35.575357+0000","last_peered":"2023-12-20T06:03:35.575357+0000","last_clean":"2023-12-20T06:03:35.575357+0000","last_became_active":"2023-12-20T06:03:34.545741+0000","last_became_peered":"2023-12-20T06:03:34.545741+0000","last_unstale":"2023-12-20T06:03:35.575357+0000","last_undegraded":"2023-12-20T06:03:35.575357+0000","last_fullsized":"2023-12-20T06:03:35.575357+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:15:01.352569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540773+0000","last_change":"2023-12-20T06:03:21.979813+0000","last_active":"2023-12-20T06:03:34.540773+0000","last_peered":"2023-12-20T06:03:34.540773+0000","last_clean":"2023-12-20T06:03:34.540773+0000","last_became_active":"2023-12-20T06:03:21.979541+0000","last_became_peered":"2023-12-20T06:03:21.979541+0000","last_unstale":"2023-12-20T06:03:34.540773+0000","last_undegraded":"2023-12-20T06:03:34.540773+0000","last_fullsized":"2023-12-20T06:03:34.540773+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:29.736662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571852+0000","last_change":"2023-12-20T06:03:34.553048+0000","last_active":"2023-12-20T06:03:35.571852+0000","last_peered":"2023-12-20T06:03:35.571852+0000","last_clean":"2023-12-20T06:03:35.571852+0000","last_became_active":"2023-12-20T06:03:34.552750+0000","last_became_peered":"2023-12-20T06:03:34.552750+0000","last_unstale":"2023-12-20T06:03:35.571852+0000","last_undegraded":"2023-12-20T06:03:35.571852+0000","last_fullsized":"2023-12-20T06:03:35.571852+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:54:48.032421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.525225+0000","last_change":"2023-12-20T06:03:21.985481+0000","last_active":"2023-12-20T06:03:34.525225+0000","last_peered":"2023-12-20T06:03:34.525225+0000","last_clean":"2023-12-20T06:03:34.525225+0000","last_became_active":"2023-12-20T06:03:21.985202+0000","last_became_peered":"2023-12-20T06:03:21.985202+0000","last_unstale":"2023-12-20T06:03:34.525225+0000","last_undegraded":"2023-12-20T06:03:34.525225+0000","last_fullsized":"2023-12-20T06:03:34.525225+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:57:07.294252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572970+0000","last_change":"2023-12-20T06:03:34.563627+0000","last_active":"2023-12-20T06:03:35.572970+0000","last_peered":"2023-12-20T06:03:35.572970+0000","last_clean":"2023-12-20T06:03:35.572970+0000","last_became_active":"2023-12-20T06:03:34.563345+0000","last_became_peered":"2023-12-20T06:03:34.563345+0000","last_unstale":"2023-12-20T06:03:35.572970+0000","last_undegraded":"2023-12-20T06:03:35.572970+0000","last_fullsized":"2023-12-20T06:03:35.572970+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:16:53.854983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528253+0000","last_change":"2023-12-20T06:03:21.989526+0000","last_active":"2023-12-20T06:03:34.528253+0000","last_peered":"2023-12-20T06:03:34.528253+0000","last_clean":"2023-12-20T06:03:34.528253+0000","last_became_active":"2023-12-20T06:03:21.989062+0000","last_became_peered":"2023-12-20T06:03:21.989062+0000","last_unstale":"2023-12-20T06:03:34.528253+0000","last_undegraded":"2023-12-20T06:03:34.528253+0000","last_fullsized":"2023-12-20T06:03:34.528253+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:17.456340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572457+0000","last_change":"2023-12-20T06:03:34.553239+0000","last_active":"2023-12-20T06:03:35.572457+0000","last_peered":"2023-12-20T06:03:35.572457+0000","last_clean":"2023-12-20T06:03:35.572457+0000","last_became_active":"2023-12-20T06:03:34.552951+0000","last_became_peered":"2023-12-20T06:03:34.552951+0000","last_unstale":"2023-12-20T06:03:35.572457+0000","last_undegraded":"2023-12-20T06:03:35.572457+0000","last_fullsized":"2023-12-20T06:03:35.572457+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:58:00.185736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.538469+0000","last_change":"2023-12-20T06:03:21.978099+0000","last_active":"2023-12-20T06:03:34.538469+0000","last_peered":"2023-12-20T06:03:34.538469+0000","last_clean":"2023-12-20T06:03:34.538469+0000","last_became_active":"2023-12-20T06:03:21.977814+0000","last_became_peered":"2023-12-20T06:03:21.977814+0000","last_unstale":"2023-12-20T06:03:34.538469+0000","last_undegraded":"2023-12-20T06:03:34.538469+0000","last_fullsized":"2023-12-20T06:03:34.538469+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:17.140851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535420+0000","last_change":"2023-12-20T06:03:34.550163+0000","last_active":"2023-12-20T06:03:35.535420+0000","last_peered":"2023-12-20T06:03:35.535420+0000","last_clean":"2023-12-20T06:03:35.535420+0000","last_became_active":"2023-12-20T06:03:34.549905+0000","last_became_peered":"2023-12-20T06:03:34.549905+0000","last_unstale":"2023-12-20T06:03:35.535420+0000","last_undegraded":"2023-12-20T06:03:35.535420+0000","last_fullsized":"2023-12-20T06:03:35.535420+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:28.110798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.533764+0000","last_change":"2023-12-20T06:03:21.984020+0000","last_active":"2023-12-20T06:03:34.533764+0000","last_peered":"2023-12-20T06:03:34.533764+0000","last_clean":"2023-12-20T06:03:34.533764+0000","last_became_active":"2023-12-20T06:03:21.983429+0000","last_became_peered":"2023-12-20T06:03:21.983429+0000","last_unstale":"2023-12-20T06:03:34.533764+0000","last_undegraded":"2023-12-20T06:03:34.533764+0000","last_fullsized":"2023-12-20T06:03:34.533764+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535053+0000","last_change":"2023-12-20T06:03:34.546965+0000","last_active":"2023-12-20T06:03:35.535053+0000","last_peered":"2023-12-20T06:03:35.535053+0000","last_clean":"2023-12-20T06:03:35.535053+0000","last_became_active":"2023-12-20T06:03:34.546614+0000","last_became_peered":"2023-12-20T06:03:34.546614+0000","last_unstale":"2023-12-20T06:03:35.535053+0000","last_undegraded":"2023-12-20T06:03:35.535053+0000","last_fullsized":"2023-12-20T06:03:35.535053+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:23:20.670623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531376+0000","last_change":"2023-12-20T06:03:21.985095+0000","last_active":"2023-12-20T06:03:34.531376+0000","last_peered":"2023-12-20T06:03:34.531376+0000","last_clean":"2023-12-20T06:03:34.531376+0000","last_became_active":"2023-12-20T06:03:21.984659+0000","last_became_peered":"2023-12-20T06:03:21.984659+0000","last_unstale":"2023-12-20T06:03:34.531376+0000","last_undegraded":"2023-12-20T06:03:34.531376+0000","last_fullsized":"2023-12-20T06:03:34.531376+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:24:17.425367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535626+0000","last_change":"2023-12-20T06:03:34.550290+0000","last_active":"2023-12-20T06:03:35.535626+0000","last_peered":"2023-12-20T06:03:35.535626+0000","last_clean":"2023-12-20T06:03:35.535626+0000","last_became_active":"2023-12-20T06:03:34.549789+0000","last_became_peered":"2023-12-20T06:03:34.549789+0000","last_unstale":"2023-12-20T06:03:35.535626+0000","last_undegraded":"2023-12-20T06:03:35.535626+0000","last_fullsized":"2023-12-20T06:03:35.535626+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:53:04.744558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574330+0000","last_change":"2023-12-20T06:03:34.125243+0000","last_active":"2023-12-20T06:03:35.574330+0000","last_peered":"2023-12-20T06:03:35.574330+0000","last_clean":"2023-12-20T06:03:35.574330+0000","last_became_active":"2023-12-20T06:03:34.124962+0000","last_became_peered":"2023-12-20T06:03:34.124962+0000","last_unstale":"2023-12-20T06:03:35.574330+0000","last_undegraded":"2023-12-20T06:03:35.574330+0000","last_fullsized":"2023-12-20T06:03:35.574330+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:46:09.465814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.532133+0000","last_change":"2023-12-20T06:03:21.985587+0000","last_active":"2023-12-20T06:03:34.532133+0000","last_peered":"2023-12-20T06:03:34.532133+0000","last_clean":"2023-12-20T06:03:34.532133+0000","last_became_active":"2023-12-20T06:03:21.985231+0000","last_became_peered":"2023-12-20T06:03:21.985231+0000","last_unstale":"2023-12-20T06:03:34.532133+0000","last_undegraded":"2023-12-20T06:03:34.532133+0000","last_fullsized":"2023-12-20T06:03:34.532133+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:44:13.389812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"229'4","reported_seq":19,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531211+0000","last_change":"2023-12-20T06:03:21.558246+0000","last_active":"2023-12-20T06:03:34.531211+0000","last_peered":"2023-12-20T06:03:34.531211+0000","last_clean":"2023-12-20T06:03:34.531211+0000","last_became_active":"2023-12-20T06:03:21.557732+0000","last_became_peered":"2023-12-20T06:03:21.557732+0000","last_unstale":"2023-12-20T06:03:34.531211+0000","last_undegraded":"2023-12-20T06:03:34.531211+0000","last_fullsized":"2023-12-20T06:03:34.531211+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:01:07.326834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.537304+0000","last_change":"2023-12-20T06:03:34.551321+0000","last_active":"2023-12-20T06:03:35.537304+0000","last_peered":"2023-12-20T06:03:35.537304+0000","last_clean":"2023-12-20T06:03:35.537304+0000","last_became_active":"2023-12-20T06:03:34.550888+0000","last_became_peered":"2023-12-20T06:03:34.550888+0000","last_unstale":"2023-12-20T06:03:35.537304+0000","last_undegraded":"2023-12-20T06:03:35.537304+0000","last_fullsized":"2023-12-20T06:03:35.537304+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:34:13.655323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574578+0000","last_change":"2023-12-20T06:03:34.138421+0000","last_active":"2023-12-20T06:03:35.574578+0000","last_peered":"2023-12-20T06:03:35.574578+0000","last_clean":"2023-12-20T06:03:35.574578+0000","last_became_active":"2023-12-20T06:03:34.138122+0000","last_became_peered":"2023-12-20T06:03:34.138122+0000","last_unstale":"2023-12-20T06:03:35.574578+0000","last_undegraded":"2023-12-20T06:03:35.574578+0000","last_fullsized":"2023-12-20T06:03:35.574578+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:13:20.882471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531753+0000","last_change":"2023-12-20T06:03:21.975027+0000","last_active":"2023-12-20T06:03:34.531753+0000","last_peered":"2023-12-20T06:03:34.531753+0000","last_clean":"2023-12-20T06:03:34.531753+0000","last_became_active":"2023-12-20T06:03:21.974227+0000","last_became_peered":"2023-12-20T06:03:21.974227+0000","last_unstale":"2023-12-20T06:03:34.531753+0000","last_undegraded":"2023-12-20T06:03:34.531753+0000","last_fullsized":"2023-12-20T06:03:34.531753+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:38:46.179532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"229'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.543031+0000","last_change":"2023-12-20T06:03:21.559869+0000","last_active":"2023-12-20T06:03:34.543031+0000","last_peered":"2023-12-20T06:03:34.543031+0000","last_clean":"2023-12-20T06:03:34.543031+0000","last_became_active":"2023-12-20T06:03:21.559555+0000","last_became_peered":"2023-12-20T06:03:21.559555+0000","last_unstale":"2023-12-20T06:03:34.543031+0000","last_undegraded":"2023-12-20T06:03:34.543031+0000","last_fullsized":"2023-12-20T06:03:34.543031+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:15.926622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574334+0000","last_change":"2023-12-20T06:03:34.541169+0000","last_active":"2023-12-20T06:03:35.574334+0000","last_peered":"2023-12-20T06:03:35.574334+0000","last_clean":"2023-12-20T06:03:35.574334+0000","last_became_active":"2023-12-20T06:03:34.540560+0000","last_became_peered":"2023-12-20T06:03:34.540560+0000","last_unstale":"2023-12-20T06:03:35.574334+0000","last_undegraded":"2023-12-20T06:03:35.574334+0000","last_fullsized":"2023-12-20T06:03:35.574334+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T18:01:49.083465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"233'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571857+0000","last_change":"2023-12-20T06:03:34.133947+0000","last_active":"2023-12-20T06:03:35.571857+0000","last_peered":"2023-12-20T06:03:35.571857+0000","last_clean":"2023-12-20T06:03:35.571857+0000","last_became_active":"2023-12-20T06:03:34.133679+0000","last_became_peered":"2023-12-20T06:03:34.133679+0000","last_unstale":"2023-12-20T06:03:35.571857+0000","last_undegraded":"2023-12-20T06:03:35.571857+0000","last_fullsized":"2023-12-20T06:03:35.571857+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:51:19.699199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528779+0000","last_change":"2023-12-20T06:03:21.988797+0000","last_active":"2023-12-20T06:03:34.528779+0000","last_peered":"2023-12-20T06:03:34.528779+0000","last_clean":"2023-12-20T06:03:34.528779+0000","last_became_active":"2023-12-20T06:03:21.988418+0000","last_became_peered":"2023-12-20T06:03:21.988418+0000","last_unstale":"2023-12-20T06:03:34.528779+0000","last_undegraded":"2023-12-20T06:03:34.528779+0000","last_fullsized":"2023-12-20T06:03:34.528779+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:21:31.283475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540729+0000","last_change":"2023-12-20T06:03:21.566883+0000","last_active":"2023-12-20T06:03:34.540729+0000","last_peered":"2023-12-20T06:03:34.540729+0000","last_clean":"2023-12-20T06:03:34.540729+0000","last_became_active":"2023-12-20T06:03:21.566612+0000","last_became_peered":"2023-12-20T06:03:21.566612+0000","last_unstale":"2023-12-20T06:03:34.540729+0000","last_undegraded":"2023-12-20T06:03:34.540729+0000","last_fullsized":"2023-12-20T06:03:34.540729+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:03:33.646493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572648+0000","last_change":"2023-12-20T06:03:34.542361+0000","last_active":"2023-12-20T06:03:35.572648+0000","last_peered":"2023-12-20T06:03:35.572648+0000","last_clean":"2023-12-20T06:03:35.572648+0000","last_became_active":"2023-12-20T06:03:34.541901+0000","last_became_peered":"2023-12-20T06:03:34.541901+0000","last_unstale":"2023-12-20T06:03:35.572648+0000","last_undegraded":"2023-12-20T06:03:35.572648+0000","last_fullsized":"2023-12-20T06:03:35.572648+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:18:24.755969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570779+0000","last_change":"2023-12-20T06:03:34.125419+0000","last_active":"2023-12-20T06:03:35.570779+0000","last_peered":"2023-12-20T06:03:35.570779+0000","last_clean":"2023-12-20T06:03:35.570779+0000","last_became_active":"2023-12-20T06:03:34.125066+0000","last_became_peered":"2023-12-20T06:03:34.125066+0000","last_unstale":"2023-12-20T06:03:35.570779+0000","last_undegraded":"2023-12-20T06:03:35.570779+0000","last_fullsized":"2023-12-20T06:03:35.570779+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:15:59.325800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.538555+0000","last_change":"2023-12-20T06:03:21.976016+0000","last_active":"2023-12-20T06:03:34.538555+0000","last_peered":"2023-12-20T06:03:34.538555+0000","last_clean":"2023-12-20T06:03:34.538555+0000","last_became_active":"2023-12-20T06:03:21.975686+0000","last_became_peered":"2023-12-20T06:03:21.975686+0000","last_unstale":"2023-12-20T06:03:34.538555+0000","last_undegraded":"2023-12-20T06:03:34.538555+0000","last_fullsized":"2023-12-20T06:03:34.538555+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:40:22.788736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"229'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530275+0000","last_change":"2023-12-20T06:03:21.567258+0000","last_active":"2023-12-20T06:03:34.530275+0000","last_peered":"2023-12-20T06:03:34.530275+0000","last_clean":"2023-12-20T06:03:34.530275+0000","last_became_active":"2023-12-20T06:03:21.565535+0000","last_became_peered":"2023-12-20T06:03:21.565535+0000","last_unstale":"2023-12-20T06:03:34.530275+0000","last_undegraded":"2023-12-20T06:03:34.530275+0000","last_fullsized":"2023-12-20T06:03:34.530275+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:27:30.578745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572367+0000","last_change":"2023-12-20T06:03:34.548571+0000","last_active":"2023-12-20T06:03:35.572367+0000","last_peered":"2023-12-20T06:03:35.572367+0000","last_clean":"2023-12-20T06:03:35.572367+0000","last_became_active":"2023-12-20T06:03:34.546639+0000","last_became_peered":"2023-12-20T06:03:34.546639+0000","last_unstale":"2023-12-20T06:03:35.572367+0000","last_undegraded":"2023-12-20T06:03:35.572367+0000","last_fullsized":"2023-12-20T06:03:35.572367+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:27:22.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572277+0000","last_change":"2023-12-20T06:03:34.141640+0000","last_active":"2023-12-20T06:03:35.572277+0000","last_peered":"2023-12-20T06:03:35.572277+0000","last_clean":"2023-12-20T06:03:35.572277+0000","last_became_active":"2023-12-20T06:03:34.141373+0000","last_became_peered":"2023-12-20T06:03:34.141373+0000","last_unstale":"2023-12-20T06:03:35.572277+0000","last_undegraded":"2023-12-20T06:03:35.572277+0000","last_fullsized":"2023-12-20T06:03:35.572277+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:05.537840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537903+0000","last_change":"2023-12-20T06:03:21.988028+0000","last_active":"2023-12-20T06:03:34.537903+0000","last_peered":"2023-12-20T06:03:34.537903+0000","last_clean":"2023-12-20T06:03:34.537903+0000","last_became_active":"2023-12-20T06:03:21.986936+0000","last_became_peered":"2023-12-20T06:03:21.986936+0000","last_unstale":"2023-12-20T06:03:34.537903+0000","last_undegraded":"2023-12-20T06:03:34.537903+0000","last_fullsized":"2023-12-20T06:03:34.537903+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"229'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529334+0000","last_change":"2023-12-20T06:03:21.564120+0000","last_active":"2023-12-20T06:03:34.529334+0000","last_peered":"2023-12-20T06:03:34.529334+0000","last_clean":"2023-12-20T06:03:34.529334+0000","last_became_active":"2023-12-20T06:03:21.563826+0000","last_became_peered":"2023-12-20T06:03:21.563826+0000","last_unstale":"2023-12-20T06:03:34.529334+0000","last_undegraded":"2023-12-20T06:03:34.529334+0000","last_fullsized":"2023-12-20T06:03:34.529334+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535166+0000","last_change":"2023-12-20T06:03:34.550742+0000","last_active":"2023-12-20T06:03:35.535166+0000","last_peered":"2023-12-20T06:03:35.535166+0000","last_clean":"2023-12-20T06:03:35.535166+0000","last_became_active":"2023-12-20T06:03:34.550491+0000","last_became_peered":"2023-12-20T06:03:34.550491+0000","last_unstale":"2023-12-20T06:03:35.535166+0000","last_undegraded":"2023-12-20T06:03:35.535166+0000","last_fullsized":"2023-12-20T06:03:35.535166+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:03:18.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"233'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573236+0000","last_change":"2023-12-20T06:03:34.142117+0000","last_active":"2023-12-20T06:03:35.573236+0000","last_peered":"2023-12-20T06:03:35.573236+0000","last_clean":"2023-12-20T06:03:35.573236+0000","last_became_active":"2023-12-20T06:03:34.141765+0000","last_became_peered":"2023-12-20T06:03:34.141765+0000","last_unstale":"2023-12-20T06:03:35.573236+0000","last_undegraded":"2023-12-20T06:03:35.573236+0000","last_fullsized":"2023-12-20T06:03:35.573236+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:57.752322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.523985+0000","last_change":"2023-12-20T06:03:21.975377+0000","last_active":"2023-12-20T06:03:34.523985+0000","last_peered":"2023-12-20T06:03:34.523985+0000","last_clean":"2023-12-20T06:03:34.523985+0000","last_became_active":"2023-12-20T06:03:21.974995+0000","last_became_peered":"2023-12-20T06:03:21.974995+0000","last_unstale":"2023-12-20T06:03:34.523985+0000","last_undegraded":"2023-12-20T06:03:34.523985+0000","last_fullsized":"2023-12-20T06:03:34.523985+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:46.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528457+0000","last_change":"2023-12-20T06:03:21.565400+0000","last_active":"2023-12-20T06:03:34.528457+0000","last_peered":"2023-12-20T06:03:34.528457+0000","last_clean":"2023-12-20T06:03:34.528457+0000","last_became_active":"2023-12-20T06:03:21.565079+0000","last_became_peered":"2023-12-20T06:03:21.565079+0000","last_unstale":"2023-12-20T06:03:34.528457+0000","last_undegraded":"2023-12-20T06:03:34.528457+0000","last_fullsized":"2023-12-20T06:03:34.528457+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:22.788125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571276+0000","last_change":"2023-12-20T06:03:34.552533+0000","last_active":"2023-12-20T06:03:35.571276+0000","last_peered":"2023-12-20T06:03:35.571276+0000","last_clean":"2023-12-20T06:03:35.571276+0000","last_became_active":"2023-12-20T06:03:34.552056+0000","last_became_peered":"2023-12-20T06:03:34.552056+0000","last_unstale":"2023-12-20T06:03:35.571276+0000","last_undegraded":"2023-12-20T06:03:35.571276+0000","last_fullsized":"2023-12-20T06:03:35.571276+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:19.382530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"233'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573045+0000","last_change":"2023-12-20T06:03:34.139657+0000","last_active":"2023-12-20T06:03:35.573045+0000","last_peered":"2023-12-20T06:03:35.573045+0000","last_clean":"2023-12-20T06:03:35.573045+0000","last_became_active":"2023-12-20T06:03:34.139304+0000","last_became_peered":"2023-12-20T06:03:34.139304+0000","last_unstale":"2023-12-20T06:03:35.573045+0000","last_undegraded":"2023-12-20T06:03:35.573045+0000","last_fullsized":"2023-12-20T06:03:35.573045+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:02:31.037953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.536840+0000","last_change":"2023-12-20T06:03:21.977819+0000","last_active":"2023-12-20T06:03:34.536840+0000","last_peered":"2023-12-20T06:03:34.536840+0000","last_clean":"2023-12-20T06:03:34.536840+0000","last_became_active":"2023-12-20T06:03:21.977540+0000","last_became_peered":"2023-12-20T06:03:21.977540+0000","last_unstale":"2023-12-20T06:03:34.536840+0000","last_undegraded":"2023-12-20T06:03:34.536840+0000","last_fullsized":"2023-12-20T06:03:34.536840+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:44:13.389812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.524159+0000","last_change":"2023-12-20T06:03:21.564947+0000","last_active":"2023-12-20T06:03:34.524159+0000","last_peered":"2023-12-20T06:03:34.524159+0000","last_clean":"2023-12-20T06:03:34.524159+0000","last_became_active":"2023-12-20T06:03:21.564634+0000","last_became_peered":"2023-12-20T06:03:21.564634+0000","last_unstale":"2023-12-20T06:03:34.524159+0000","last_undegraded":"2023-12-20T06:03:34.524159+0000","last_fullsized":"2023-12-20T06:03:34.524159+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:05.795673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574239+0000","last_change":"2023-12-20T06:03:34.549447+0000","last_active":"2023-12-20T06:03:35.574239+0000","last_peered":"2023-12-20T06:03:35.574239+0000","last_clean":"2023-12-20T06:03:35.574239+0000","last_became_active":"2023-12-20T06:03:34.547913+0000","last_became_peered":"2023-12-20T06:03:34.547913+0000","last_unstale":"2023-12-20T06:03:35.574239+0000","last_undegraded":"2023-12-20T06:03:35.574239+0000","last_fullsized":"2023-12-20T06:03:35.574239+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:25.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"233'3","reported_seq":14,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575114+0000","last_change":"2023-12-20T06:03:34.121653+0000","last_active":"2023-12-20T06:03:35.575114+0000","last_peered":"2023-12-20T06:03:35.575114+0000","last_clean":"2023-12-20T06:03:35.575114+0000","last_became_active":"2023-12-20T06:03:34.121375+0000","last_became_peered":"2023-12-20T06:03:34.121375+0000","last_unstale":"2023-12-20T06:03:35.575114+0000","last_undegraded":"2023-12-20T06:03:35.575114+0000","last_fullsized":"2023-12-20T06:03:35.575114+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.275988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.542866+0000","last_change":"2023-12-20T06:03:21.989153+0000","last_active":"2023-12-20T06:03:34.542866+0000","last_peered":"2023-12-20T06:03:34.542866+0000","last_clean":"2023-12-20T06:03:34.542866+0000","last_became_active":"2023-12-20T06:03:21.988699+0000","last_became_peered":"2023-12-20T06:03:21.988699+0000","last_unstale":"2023-12-20T06:03:34.542866+0000","last_undegraded":"2023-12-20T06:03:34.542866+0000","last_fullsized":"2023-12-20T06:03:34.542866+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:11.133160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.541414+0000","last_change":"2023-12-20T06:03:21.574001+0000","last_active":"2023-12-20T06:03:34.541414+0000","last_peered":"2023-12-20T06:03:34.541414+0000","last_clean":"2023-12-20T06:03:34.541414+0000","last_became_active":"2023-12-20T06:03:21.573735+0000","last_became_peered":"2023-12-20T06:03:21.573735+0000","last_unstale":"2023-12-20T06:03:34.541414+0000","last_undegraded":"2023-12-20T06:03:34.541414+0000","last_fullsized":"2023-12-20T06:03:34.541414+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:13:55.127307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571492+0000","last_change":"2023-12-20T06:03:34.543697+0000","last_active":"2023-12-20T06:03:35.571492+0000","last_peered":"2023-12-20T06:03:35.571492+0000","last_clean":"2023-12-20T06:03:35.571492+0000","last_became_active":"2023-12-20T06:03:34.543314+0000","last_became_peered":"2023-12-20T06:03:34.543314+0000","last_unstale":"2023-12-20T06:03:35.571492+0000","last_undegraded":"2023-12-20T06:03:35.571492+0000","last_fullsized":"2023-12-20T06:03:35.571492+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:07:08.736697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"233'6","reported_seq":17,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573618+0000","last_change":"2023-12-20T06:03:34.131414+0000","last_active":"2023-12-20T06:03:35.573618+0000","last_peered":"2023-12-20T06:03:35.573618+0000","last_clean":"2023-12-20T06:03:35.573618+0000","last_became_active":"2023-12-20T06:03:34.130989+0000","last_became_peered":"2023-12-20T06:03:34.130989+0000","last_unstale":"2023-12-20T06:03:35.573618+0000","last_undegraded":"2023-12-20T06:03:35.573618+0000","last_fullsized":"2023-12-20T06:03:35.573618+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:07:36.615031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.541326+0000","last_change":"2023-12-20T06:03:21.967818+0000","last_active":"2023-12-20T06:03:34.541326+0000","last_peered":"2023-12-20T06:03:34.541326+0000","last_clean":"2023-12-20T06:03:34.541326+0000","last_became_active":"2023-12-20T06:03:21.967516+0000","last_became_peered":"2023-12-20T06:03:21.967516+0000","last_unstale":"2023-12-20T06:03:34.541326+0000","last_undegraded":"2023-12-20T06:03:34.541326+0000","last_fullsized":"2023-12-20T06:03:34.541326+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:07:37.126937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531510+0000","last_change":"2023-12-20T06:03:21.563619+0000","last_active":"2023-12-20T06:03:34.531510+0000","last_peered":"2023-12-20T06:03:34.531510+0000","last_clean":"2023-12-20T06:03:34.531510+0000","last_became_active":"2023-12-20T06:03:21.563273+0000","last_became_peered":"2023-12-20T06:03:21.563273+0000","last_unstale":"2023-12-20T06:03:34.531510+0000","last_undegraded":"2023-12-20T06:03:34.531510+0000","last_fullsized":"2023-12-20T06:03:34.531510+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:39:30.597039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575260+0000","last_change":"2023-12-20T06:03:34.550958+0000","last_active":"2023-12-20T06:03:35.575260+0000","last_peered":"2023-12-20T06:03:35.575260+0000","last_clean":"2023-12-20T06:03:35.575260+0000","last_became_active":"2023-12-20T06:03:34.549699+0000","last_became_peered":"2023-12-20T06:03:34.549699+0000","last_unstale":"2023-12-20T06:03:35.575260+0000","last_undegraded":"2023-12-20T06:03:35.575260+0000","last_fullsized":"2023-12-20T06:03:35.575260+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:02.206635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.537482+0000","last_change":"2023-12-20T06:03:34.128302+0000","last_active":"2023-12-20T06:03:35.537482+0000","last_peered":"2023-12-20T06:03:35.537482+0000","last_clean":"2023-12-20T06:03:35.537482+0000","last_became_active":"2023-12-20T06:03:34.127679+0000","last_became_peered":"2023-12-20T06:03:34.127679+0000","last_unstale":"2023-12-20T06:03:35.537482+0000","last_undegraded":"2023-12-20T06:03:35.537482+0000","last_fullsized":"2023-12-20T06:03:35.537482+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:30:25.302163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529676+0000","last_change":"2023-12-20T06:03:21.990214+0000","last_active":"2023-12-20T06:03:34.529676+0000","last_peered":"2023-12-20T06:03:34.529676+0000","last_clean":"2023-12-20T06:03:34.529676+0000","last_became_active":"2023-12-20T06:03:21.989788+0000","last_became_peered":"2023-12-20T06:03:21.989788+0000","last_unstale":"2023-12-20T06:03:34.529676+0000","last_undegraded":"2023-12-20T06:03:34.529676+0000","last_fullsized":"2023-12-20T06:03:34.529676+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"229'3","reported_seq":18,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534698+0000","last_change":"2023-12-20T06:03:21.564537+0000","last_active":"2023-12-20T06:03:34.534698+0000","last_peered":"2023-12-20T06:03:34.534698+0000","last_clean":"2023-12-20T06:03:34.534698+0000","last_became_active":"2023-12-20T06:03:21.564167+0000","last_became_peered":"2023-12-20T06:03:21.564167+0000","last_unstale":"2023-12-20T06:03:34.534698+0000","last_undegraded":"2023-12-20T06:03:34.534698+0000","last_fullsized":"2023-12-20T06:03:34.534698+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:41:11.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536057+0000","last_change":"2023-12-20T06:03:34.556503+0000","last_active":"2023-12-20T06:03:35.536057+0000","last_peered":"2023-12-20T06:03:35.536057+0000","last_clean":"2023-12-20T06:03:35.536057+0000","last_became_active":"2023-12-20T06:03:34.556125+0000","last_became_peered":"2023-12-20T06:03:34.556125+0000","last_unstale":"2023-12-20T06:03:35.536057+0000","last_undegraded":"2023-12-20T06:03:35.536057+0000","last_fullsized":"2023-12-20T06:03:35.536057+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:38:02.729064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574390+0000","last_change":"2023-12-20T06:03:34.121184+0000","last_active":"2023-12-20T06:03:35.574390+0000","last_peered":"2023-12-20T06:03:35.574390+0000","last_clean":"2023-12-20T06:03:35.574390+0000","last_became_active":"2023-12-20T06:03:34.120940+0000","last_became_peered":"2023-12-20T06:03:34.120940+0000","last_unstale":"2023-12-20T06:03:35.574390+0000","last_undegraded":"2023-12-20T06:03:35.574390+0000","last_fullsized":"2023-12-20T06:03:35.574390+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:38:43.241427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540715+0000","last_change":"2023-12-20T06:03:21.963443+0000","last_active":"2023-12-20T06:03:34.540715+0000","last_peered":"2023-12-20T06:03:34.540715+0000","last_clean":"2023-12-20T06:03:34.540715+0000","last_became_active":"2023-12-20T06:03:21.962645+0000","last_became_peered":"2023-12-20T06:03:21.962645+0000","last_unstale":"2023-12-20T06:03:34.540715+0000","last_undegraded":"2023-12-20T06:03:34.540715+0000","last_fullsized":"2023-12-20T06:03:34.540715+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:52:50.983047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"230'8","reported_seq":23,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.525303+0000","last_change":"2023-12-20T06:03:21.565407+0000","last_active":"2023-12-20T06:03:34.525303+0000","last_peered":"2023-12-20T06:03:34.525303+0000","last_clean":"2023-12-20T06:03:34.525303+0000","last_became_active":"2023-12-20T06:03:21.564376+0000","last_became_peered":"2023-12-20T06:03:21.564376+0000","last_unstale":"2023-12-20T06:03:34.525303+0000","last_undegraded":"2023-12-20T06:03:34.525303+0000","last_fullsized":"2023-12-20T06:03:34.525303+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:23:07.083767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572452+0000","last_change":"2023-12-20T06:03:34.560985+0000","last_active":"2023-12-20T06:03:35.572452+0000","last_peered":"2023-12-20T06:03:35.572452+0000","last_clean":"2023-12-20T06:03:35.572452+0000","last_became_active":"2023-12-20T06:03:34.560683+0000","last_became_peered":"2023-12-20T06:03:34.560683+0000","last_unstale":"2023-12-20T06:03:35.572452+0000","last_undegraded":"2023-12-20T06:03:35.572452+0000","last_fullsized":"2023-12-20T06:03:35.572452+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:03:47.233349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570814+0000","last_change":"2023-12-20T06:03:34.138804+0000","last_active":"2023-12-20T06:03:35.570814+0000","last_peered":"2023-12-20T06:03:35.570814+0000","last_clean":"2023-12-20T06:03:35.570814+0000","last_became_active":"2023-12-20T06:03:34.138379+0000","last_became_peered":"2023-12-20T06:03:34.138379+0000","last_unstale":"2023-12-20T06:03:35.570814+0000","last_undegraded":"2023-12-20T06:03:35.570814+0000","last_fullsized":"2023-12-20T06:03:35.570814+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:58.375033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537642+0000","last_change":"2023-12-20T06:03:21.971942+0000","last_active":"2023-12-20T06:03:34.537642+0000","last_peered":"2023-12-20T06:03:34.537642+0000","last_clean":"2023-12-20T06:03:34.537642+0000","last_became_active":"2023-12-20T06:03:21.971674+0000","last_became_peered":"2023-12-20T06:03:21.971674+0000","last_unstale":"2023-12-20T06:03:34.537642+0000","last_undegraded":"2023-12-20T06:03:34.537642+0000","last_fullsized":"2023-12-20T06:03:34.537642+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:56.167044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530363+0000","last_change":"2023-12-20T06:03:21.567100+0000","last_active":"2023-12-20T06:03:34.530363+0000","last_peered":"2023-12-20T06:03:34.530363+0000","last_clean":"2023-12-20T06:03:34.530363+0000","last_became_active":"2023-12-20T06:03:21.566805+0000","last_became_peered":"2023-12-20T06:03:21.566805+0000","last_unstale":"2023-12-20T06:03:34.530363+0000","last_undegraded":"2023-12-20T06:03:34.530363+0000","last_fullsized":"2023-12-20T06:03:34.530363+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:45:38.064958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570696+0000","last_change":"2023-12-20T06:03:34.539312+0000","last_active":"2023-12-20T06:03:35.570696+0000","last_peered":"2023-12-20T06:03:35.570696+0000","last_clean":"2023-12-20T06:03:35.570696+0000","last_became_active":"2023-12-20T06:03:34.538981+0000","last_became_peered":"2023-12-20T06:03:34.538981+0000","last_unstale":"2023-12-20T06:03:35.570696+0000","last_undegraded":"2023-12-20T06:03:35.570696+0000","last_fullsized":"2023-12-20T06:03:35.570696+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:46:47.498001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"233'8","reported_seq":19,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535056+0000","last_change":"2023-12-20T06:03:34.139382+0000","last_active":"2023-12-20T06:03:35.535056+0000","last_peered":"2023-12-20T06:03:35.535056+0000","last_clean":"2023-12-20T06:03:35.535056+0000","last_became_active":"2023-12-20T06:03:34.139032+0000","last_became_peered":"2023-12-20T06:03:34.139032+0000","last_unstale":"2023-12-20T06:03:35.535056+0000","last_undegraded":"2023-12-20T06:03:35.535056+0000","last_fullsized":"2023-12-20T06:03:35.535056+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:54:35.263517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530349+0000","last_change":"2023-12-20T06:03:21.984619+0000","last_active":"2023-12-20T06:03:34.530349+0000","last_peered":"2023-12-20T06:03:34.530349+0000","last_clean":"2023-12-20T06:03:34.530349+0000","last_became_active":"2023-12-20T06:03:21.983940+0000","last_became_peered":"2023-12-20T06:03:21.983940+0000","last_unstale":"2023-12-20T06:03:34.530349+0000","last_undegraded":"2023-12-20T06:03:34.530349+0000","last_fullsized":"2023-12-20T06:03:34.530349+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:13.633088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537728+0000","last_change":"2023-12-20T06:03:21.561473+0000","last_active":"2023-12-20T06:03:34.537728+0000","last_peered":"2023-12-20T06:03:34.537728+0000","last_clean":"2023-12-20T06:03:34.537728+0000","last_became_active":"2023-12-20T06:03:21.561038+0000","last_became_peered":"2023-12-20T06:03:21.561038+0000","last_unstale":"2023-12-20T06:03:34.537728+0000","last_undegraded":"2023-12-20T06:03:34.537728+0000","last_fullsized":"2023-12-20T06:03:34.537728+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:57:46.598880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570948+0000","last_change":"2023-12-20T06:03:34.537750+0000","last_active":"2023-12-20T06:03:35.570948+0000","last_peered":"2023-12-20T06:03:35.570948+0000","last_clean":"2023-12-20T06:03:35.570948+0000","last_became_active":"2023-12-20T06:03:34.537451+0000","last_became_peered":"2023-12-20T06:03:34.537451+0000","last_unstale":"2023-12-20T06:03:35.570948+0000","last_undegraded":"2023-12-20T06:03:35.570948+0000","last_fullsized":"2023-12-20T06:03:35.570948+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:01:20.913691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573972+0000","last_change":"2023-12-20T06:03:34.122211+0000","last_active":"2023-12-20T06:03:35.573972+0000","last_peered":"2023-12-20T06:03:35.573972+0000","last_clean":"2023-12-20T06:03:35.573972+0000","last_became_active":"2023-12-20T06:03:34.121938+0000","last_became_peered":"2023-12-20T06:03:34.121938+0000","last_unstale":"2023-12-20T06:03:35.573972+0000","last_undegraded":"2023-12-20T06:03:35.573972+0000","last_fullsized":"2023-12-20T06:03:35.573972+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:30:25.302163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529207+0000","last_change":"2023-12-20T06:03:21.987462+0000","last_active":"2023-12-20T06:03:34.529207+0000","last_peered":"2023-12-20T06:03:34.529207+0000","last_clean":"2023-12-20T06:03:34.529207+0000","last_became_active":"2023-12-20T06:03:21.986488+0000","last_became_peered":"2023-12-20T06:03:21.986488+0000","last_unstale":"2023-12-20T06:03:34.529207+0000","last_undegraded":"2023-12-20T06:03:34.529207+0000","last_fullsized":"2023-12-20T06:03:34.529207+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:00.841811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534367+0000","last_change":"2023-12-20T06:03:21.564490+0000","last_active":"2023-12-20T06:03:34.534367+0000","last_peered":"2023-12-20T06:03:34.534367+0000","last_clean":"2023-12-20T06:03:34.534367+0000","last_became_active":"2023-12-20T06:03:21.562571+0000","last_became_peered":"2023-12-20T06:03:21.562571+0000","last_unstale":"2023-12-20T06:03:34.534367+0000","last_undegraded":"2023-12-20T06:03:34.534367+0000","last_fullsized":"2023-12-20T06:03:34.534367+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571306+0000","last_change":"2023-12-20T06:03:34.545843+0000","last_active":"2023-12-20T06:03:35.571306+0000","last_peered":"2023-12-20T06:03:35.571306+0000","last_clean":"2023-12-20T06:03:35.571306+0000","last_became_active":"2023-12-20T06:03:34.545579+0000","last_became_peered":"2023-12-20T06:03:34.545579+0000","last_unstale":"2023-12-20T06:03:35.571306+0000","last_undegraded":"2023-12-20T06:03:35.571306+0000","last_fullsized":"2023-12-20T06:03:35.571306+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:00:21.937869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573193+0000","last_change":"2023-12-20T06:03:34.136207+0000","last_active":"2023-12-20T06:03:35.573193+0000","last_peered":"2023-12-20T06:03:35.573193+0000","last_clean":"2023-12-20T06:03:35.573193+0000","last_became_active":"2023-12-20T06:03:34.135946+0000","last_became_peered":"2023-12-20T06:03:34.135946+0000","last_unstale":"2023-12-20T06:03:35.573193+0000","last_undegraded":"2023-12-20T06:03:35.573193+0000","last_fullsized":"2023-12-20T06:03:35.573193+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:20:26.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528357+0000","last_change":"2023-12-20T06:03:21.978828+0000","last_active":"2023-12-20T06:03:34.528357+0000","last_peered":"2023-12-20T06:03:34.528357+0000","last_clean":"2023-12-20T06:03:34.528357+0000","last_became_active":"2023-12-20T06:03:21.977595+0000","last_became_peered":"2023-12-20T06:03:21.977595+0000","last_unstale":"2023-12-20T06:03:34.528357+0000","last_undegraded":"2023-12-20T06:03:34.528357+0000","last_fullsized":"2023-12-20T06:03:34.528357+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:53:41.601867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"229'8","reported_seq":23,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537981+0000","last_change":"2023-12-20T06:03:21.556640+0000","last_active":"2023-12-20T06:03:34.537981+0000","last_peered":"2023-12-20T06:03:34.537981+0000","last_clean":"2023-12-20T06:03:34.537981+0000","last_became_active":"2023-12-20T06:03:21.556362+0000","last_became_peered":"2023-12-20T06:03:21.556362+0000","last_unstale":"2023-12-20T06:03:34.537981+0000","last_undegraded":"2023-12-20T06:03:34.537981+0000","last_fullsized":"2023-12-20T06:03:34.537981+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:16:40.268127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572356+0000","last_change":"2023-12-20T06:03:34.545159+0000","last_active":"2023-12-20T06:03:35.572356+0000","last_peered":"2023-12-20T06:03:35.572356+0000","last_clean":"2023-12-20T06:03:35.572356+0000","last_became_active":"2023-12-20T06:03:34.544891+0000","last_became_peered":"2023-12-20T06:03:34.544891+0000","last_unstale":"2023-12-20T06:03:35.572356+0000","last_undegraded":"2023-12-20T06:03:35.572356+0000","last_fullsized":"2023-12-20T06:03:35.572356+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:25:10.820979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"233'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572406+0000","last_change":"2023-12-20T06:03:34.139831+0000","last_active":"2023-12-20T06:03:35.572406+0000","last_peered":"2023-12-20T06:03:35.572406+0000","last_clean":"2023-12-20T06:03:35.572406+0000","last_became_active":"2023-12-20T06:03:34.139367+0000","last_became_peered":"2023-12-20T06:03:34.139367+0000","last_unstale":"2023-12-20T06:03:35.572406+0000","last_undegraded":"2023-12-20T06:03:35.572406+0000","last_fullsized":"2023-12-20T06:03:35.572406+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:16.190203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531083+0000","last_change":"2023-12-20T06:03:21.972553+0000","last_active":"2023-12-20T06:03:34.531083+0000","last_peered":"2023-12-20T06:03:34.531083+0000","last_clean":"2023-12-20T06:03:34.531083+0000","last_became_active":"2023-12-20T06:03:21.972295+0000","last_became_peered":"2023-12-20T06:03:21.972295+0000","last_unstale":"2023-12-20T06:03:34.531083+0000","last_undegraded":"2023-12-20T06:03:34.531083+0000","last_fullsized":"2023-12-20T06:03:34.531083+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:44.890915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"229'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531092+0000","last_change":"2023-12-20T06:03:21.563512+0000","last_active":"2023-12-20T06:03:34.531092+0000","last_peered":"2023-12-20T06:03:34.531092+0000","last_clean":"2023-12-20T06:03:34.531092+0000","last_became_active":"2023-12-20T06:03:21.563130+0000","last_became_peered":"2023-12-20T06:03:21.563130+0000","last_unstale":"2023-12-20T06:03:34.531092+0000","last_undegraded":"2023-12-20T06:03:34.531092+0000","last_fullsized":"2023-12-20T06:03:34.531092+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:28.719459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.537429+0000","last_change":"2023-12-20T06:03:34.551376+0000","last_active":"2023-12-20T06:03:35.537429+0000","last_peered":"2023-12-20T06:03:35.537429+0000","last_clean":"2023-12-20T06:03:35.537429+0000","last_became_active":"2023-12-20T06:03:34.550896+0000","last_became_peered":"2023-12-20T06:03:34.550896+0000","last_unstale":"2023-12-20T06:03:35.537429+0000","last_undegraded":"2023-12-20T06:03:35.537429+0000","last_fullsized":"2023-12-20T06:03:35.537429+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:56:41.119728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535727+0000","last_change":"2023-12-20T06:03:34.131712+0000","last_active":"2023-12-20T06:03:35.535727+0000","last_peered":"2023-12-20T06:03:35.535727+0000","last_clean":"2023-12-20T06:03:35.535727+0000","last_became_active":"2023-12-20T06:03:34.131370+0000","last_became_peered":"2023-12-20T06:03:34.131370+0000","last_unstale":"2023-12-20T06:03:35.535727+0000","last_undegraded":"2023-12-20T06:03:35.535727+0000","last_fullsized":"2023-12-20T06:03:35.535727+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:02.003235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.526066+0000","last_change":"2023-12-20T06:03:21.991246+0000","last_active":"2023-12-20T06:03:34.526066+0000","last_peered":"2023-12-20T06:03:34.526066+0000","last_clean":"2023-12-20T06:03:34.526066+0000","last_became_active":"2023-12-20T06:03:21.990892+0000","last_became_peered":"2023-12-20T06:03:21.990892+0000","last_unstale":"2023-12-20T06:03:34.526066+0000","last_undegraded":"2023-12-20T06:03:34.526066+0000","last_fullsized":"2023-12-20T06:03:34.526066+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:52.328250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531601+0000","last_change":"2023-12-20T06:03:21.576334+0000","last_active":"2023-12-20T06:03:34.531601+0000","last_peered":"2023-12-20T06:03:34.531601+0000","last_clean":"2023-12-20T06:03:34.531601+0000","last_became_active":"2023-12-20T06:03:21.575847+0000","last_became_peered":"2023-12-20T06:03:21.575847+0000","last_unstale":"2023-12-20T06:03:34.531601+0000","last_undegraded":"2023-12-20T06:03:34.531601+0000","last_fullsized":"2023-12-20T06:03:34.531601+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:46:33.911145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574331+0000","last_change":"2023-12-20T06:03:34.548458+0000","last_active":"2023-12-20T06:03:35.574331+0000","last_peered":"2023-12-20T06:03:35.574331+0000","last_clean":"2023-12-20T06:03:35.574331+0000","last_became_active":"2023-12-20T06:03:34.548187+0000","last_became_peered":"2023-12-20T06:03:34.548187+0000","last_unstale":"2023-12-20T06:03:35.574331+0000","last_undegraded":"2023-12-20T06:03:35.574331+0000","last_fullsized":"2023-12-20T06:03:35.574331+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:21:43.853129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"233'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571675+0000","last_change":"2023-12-20T06:03:34.135718+0000","last_active":"2023-12-20T06:03:35.571675+0000","last_peered":"2023-12-20T06:03:35.571675+0000","last_clean":"2023-12-20T06:03:35.571675+0000","last_became_active":"2023-12-20T06:03:34.134508+0000","last_became_peered":"2023-12-20T06:03:34.134508+0000","last_unstale":"2023-12-20T06:03:35.571675+0000","last_undegraded":"2023-12-20T06:03:35.571675+0000","last_fullsized":"2023-12-20T06:03:35.571675+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:37:50.486590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530146+0000","last_change":"2023-12-20T06:03:21.988502+0000","last_active":"2023-12-20T06:03:34.530146+0000","last_peered":"2023-12-20T06:03:34.530146+0000","last_clean":"2023-12-20T06:03:34.530146+0000","last_became_active":"2023-12-20T06:03:21.988209+0000","last_became_peered":"2023-12-20T06:03:21.988209+0000","last_unstale":"2023-12-20T06:03:34.530146+0000","last_undegraded":"2023-12-20T06:03:34.530146+0000","last_fullsized":"2023-12-20T06:03:34.530146+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:35:37.344273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"229'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540816+0000","last_change":"2023-12-20T06:03:21.574438+0000","last_active":"2023-12-20T06:03:34.540816+0000","last_peered":"2023-12-20T06:03:34.540816+0000","last_clean":"2023-12-20T06:03:34.540816+0000","last_became_active":"2023-12-20T06:03:21.574161+0000","last_became_peered":"2023-12-20T06:03:21.574161+0000","last_unstale":"2023-12-20T06:03:34.540816+0000","last_undegraded":"2023-12-20T06:03:34.540816+0000","last_fullsized":"2023-12-20T06:03:34.540816+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:21:17.124107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574420+0000","last_change":"2023-12-20T06:03:34.541342+0000","last_active":"2023-12-20T06:03:35.574420+0000","last_peered":"2023-12-20T06:03:35.574420+0000","last_clean":"2023-12-20T06:03:35.574420+0000","last_became_active":"2023-12-20T06:03:34.540910+0000","last_became_peered":"2023-12-20T06:03:34.540910+0000","last_unstale":"2023-12-20T06:03:35.574420+0000","last_undegraded":"2023-12-20T06:03:35.574420+0000","last_fullsized":"2023-12-20T06:03:35.574420+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:03.024703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574476+0000","last_change":"2023-12-20T06:03:34.123898+0000","last_active":"2023-12-20T06:03:35.574476+0000","last_peered":"2023-12-20T06:03:35.574476+0000","last_clean":"2023-12-20T06:03:35.574476+0000","last_became_active":"2023-12-20T06:03:34.123599+0000","last_became_peered":"2023-12-20T06:03:34.123599+0000","last_unstale":"2023-12-20T06:03:35.574476+0000","last_undegraded":"2023-12-20T06:03:35.574476+0000","last_fullsized":"2023-12-20T06:03:35.574476+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:12:15.603076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530454+0000","last_change":"2023-12-20T06:03:21.985314+0000","last_active":"2023-12-20T06:03:34.530454+0000","last_peered":"2023-12-20T06:03:34.530454+0000","last_clean":"2023-12-20T06:03:34.530454+0000","last_became_active":"2023-12-20T06:03:21.984916+0000","last_became_peered":"2023-12-20T06:03:21.984916+0000","last_unstale":"2023-12-20T06:03:34.530454+0000","last_undegraded":"2023-12-20T06:03:34.530454+0000","last_fullsized":"2023-12-20T06:03:34.530454+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:55:10.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.525380+0000","last_change":"2023-12-20T06:03:21.576277+0000","last_active":"2023-12-20T06:03:34.525380+0000","last_peered":"2023-12-20T06:03:34.525380+0000","last_clean":"2023-12-20T06:03:34.525380+0000","last_became_active":"2023-12-20T06:03:21.575791+0000","last_became_peered":"2023-12-20T06:03:21.575791+0000","last_unstale":"2023-12-20T06:03:34.525380+0000","last_undegraded":"2023-12-20T06:03:34.525380+0000","last_fullsized":"2023-12-20T06:03:34.525380+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:24:57.234123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.534929+0000","last_change":"2023-12-20T06:03:34.555586+0000","last_active":"2023-12-20T06:03:35.534929+0000","last_peered":"2023-12-20T06:03:35.534929+0000","last_clean":"2023-12-20T06:03:35.534929+0000","last_became_active":"2023-12-20T06:03:34.555329+0000","last_became_peered":"2023-12-20T06:03:34.555329+0000","last_unstale":"2023-12-20T06:03:35.534929+0000","last_undegraded":"2023-12-20T06:03:35.534929+0000","last_fullsized":"2023-12-20T06:03:35.534929+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:14.916269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570432+0000","last_change":"2023-12-20T06:03:34.134679+0000","last_active":"2023-12-20T06:03:35.570432+0000","last_peered":"2023-12-20T06:03:35.570432+0000","last_clean":"2023-12-20T06:03:35.570432+0000","last_became_active":"2023-12-20T06:03:34.134384+0000","last_became_peered":"2023-12-20T06:03:34.134384+0000","last_unstale":"2023-12-20T06:03:35.570432+0000","last_undegraded":"2023-12-20T06:03:35.570432+0000","last_fullsized":"2023-12-20T06:03:35.570432+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:22.681346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.533321+0000","last_change":"2023-12-20T06:03:21.977046+0000","last_active":"2023-12-20T06:03:34.533321+0000","last_peered":"2023-12-20T06:03:34.533321+0000","last_clean":"2023-12-20T06:03:34.533321+0000","last_became_active":"2023-12-20T06:03:21.976756+0000","last_became_peered":"2023-12-20T06:03:21.976756+0000","last_unstale":"2023-12-20T06:03:34.533321+0000","last_undegraded":"2023-12-20T06:03:34.533321+0000","last_fullsized":"2023-12-20T06:03:34.533321+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:17.456340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530709+0000","last_change":"2023-12-20T06:03:21.573650+0000","last_active":"2023-12-20T06:03:34.530709+0000","last_peered":"2023-12-20T06:03:34.530709+0000","last_clean":"2023-12-20T06:03:34.530709+0000","last_became_active":"2023-12-20T06:03:21.573349+0000","last_became_peered":"2023-12-20T06:03:21.573349+0000","last_unstale":"2023-12-20T06:03:34.530709+0000","last_undegraded":"2023-12-20T06:03:34.530709+0000","last_fullsized":"2023-12-20T06:03:34.530709+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:22.788125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536535+0000","last_change":"2023-12-20T06:03:34.552429+0000","last_active":"2023-12-20T06:03:35.536535+0000","last_peered":"2023-12-20T06:03:35.536535+0000","last_clean":"2023-12-20T06:03:35.536535+0000","last_became_active":"2023-12-20T06:03:34.551942+0000","last_became_peered":"2023-12-20T06:03:34.551942+0000","last_unstale":"2023-12-20T06:03:35.536535+0000","last_undegraded":"2023-12-20T06:03:35.536535+0000","last_fullsized":"2023-12-20T06:03:35.536535+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:42.632244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570774+0000","last_change":"2023-12-20T06:03:34.133267+0000","last_active":"2023-12-20T06:03:35.570774+0000","last_peered":"2023-12-20T06:03:35.570774+0000","last_clean":"2023-12-20T06:03:35.570774+0000","last_became_active":"2023-12-20T06:03:34.132981+0000","last_became_peered":"2023-12-20T06:03:34.132981+0000","last_unstale":"2023-12-20T06:03:35.570774+0000","last_undegraded":"2023-12-20T06:03:35.570774+0000","last_fullsized":"2023-12-20T06:03:35.570774+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:14:30.574620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530446+0000","last_change":"2023-12-20T06:03:21.983774+0000","last_active":"2023-12-20T06:03:34.530446+0000","last_peered":"2023-12-20T06:03:34.530446+0000","last_clean":"2023-12-20T06:03:34.530446+0000","last_became_active":"2023-12-20T06:03:21.983388+0000","last_became_peered":"2023-12-20T06:03:21.983388+0000","last_unstale":"2023-12-20T06:03:34.530446+0000","last_undegraded":"2023-12-20T06:03:34.530446+0000","last_fullsized":"2023-12-20T06:03:34.530446+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:11.133160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528503+0000","last_change":"2023-12-20T06:03:21.567023+0000","last_active":"2023-12-20T06:03:34.528503+0000","last_peered":"2023-12-20T06:03:34.528503+0000","last_clean":"2023-12-20T06:03:34.528503+0000","last_became_active":"2023-12-20T06:03:21.566708+0000","last_became_peered":"2023-12-20T06:03:21.566708+0000","last_unstale":"2023-12-20T06:03:34.528503+0000","last_undegraded":"2023-12-20T06:03:34.528503+0000","last_fullsized":"2023-12-20T06:03:34.528503+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536633+0000","last_change":"2023-12-20T06:03:34.544736+0000","last_active":"2023-12-20T06:03:35.536633+0000","last_peered":"2023-12-20T06:03:35.536633+0000","last_clean":"2023-12-20T06:03:35.536633+0000","last_became_active":"2023-12-20T06:03:34.544230+0000","last_became_peered":"2023-12-20T06:03:34.544230+0000","last_unstale":"2023-12-20T06:03:35.536633+0000","last_undegraded":"2023-12-20T06:03:35.536633+0000","last_fullsized":"2023-12-20T06:03:35.536633+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:58:00.185736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573662+0000","last_change":"2023-12-20T06:03:34.134914+0000","last_active":"2023-12-20T06:03:35.573662+0000","last_peered":"2023-12-20T06:03:35.573662+0000","last_clean":"2023-12-20T06:03:35.573662+0000","last_became_active":"2023-12-20T06:03:34.134561+0000","last_became_peered":"2023-12-20T06:03:34.134561+0000","last_unstale":"2023-12-20T06:03:35.573662+0000","last_undegraded":"2023-12-20T06:03:35.573662+0000","last_fullsized":"2023-12-20T06:03:35.573662+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:24:28.973553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530241+0000","last_change":"2023-12-20T06:03:21.972573+0000","last_active":"2023-12-20T06:03:34.530241+0000","last_peered":"2023-12-20T06:03:34.530241+0000","last_clean":"2023-12-20T06:03:34.530241+0000","last_became_active":"2023-12-20T06:03:21.972302+0000","last_became_peered":"2023-12-20T06:03:21.972302+0000","last_unstale":"2023-12-20T06:03:34.530241+0000","last_undegraded":"2023-12-20T06:03:34.530241+0000","last_fullsized":"2023-12-20T06:03:34.530241+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:58:25.448451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528615+0000","last_change":"2023-12-20T06:03:21.570826+0000","last_active":"2023-12-20T06:03:34.528615+0000","last_peered":"2023-12-20T06:03:34.528615+0000","last_clean":"2023-12-20T06:03:34.528615+0000","last_became_active":"2023-12-20T06:03:21.570512+0000","last_became_peered":"2023-12-20T06:03:21.570512+0000","last_unstale":"2023-12-20T06:03:34.528615+0000","last_undegraded":"2023-12-20T06:03:34.528615+0000","last_fullsized":"2023-12-20T06:03:34.528615+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:52:49.965844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573072+0000","last_change":"2023-12-20T06:03:34.560892+0000","last_active":"2023-12-20T06:03:35.573072+0000","last_peered":"2023-12-20T06:03:35.573072+0000","last_clean":"2023-12-20T06:03:35.573072+0000","last_became_active":"2023-12-20T06:03:34.559695+0000","last_became_peered":"2023-12-20T06:03:34.559695+0000","last_unstale":"2023-12-20T06:03:35.573072+0000","last_undegraded":"2023-12-20T06:03:35.573072+0000","last_fullsized":"2023-12-20T06:03:35.573072+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:54:48.032421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535090+0000","last_change":"2023-12-20T06:03:34.132619+0000","last_active":"2023-12-20T06:03:35.535090+0000","last_peered":"2023-12-20T06:03:35.535090+0000","last_clean":"2023-12-20T06:03:35.535090+0000","last_became_active":"2023-12-20T06:03:34.132242+0000","last_became_peered":"2023-12-20T06:03:34.132242+0000","last_unstale":"2023-12-20T06:03:35.535090+0000","last_undegraded":"2023-12-20T06:03:35.535090+0000","last_fullsized":"2023-12-20T06:03:35.535090+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:10.999454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.533851+0000","last_change":"2023-12-20T06:03:21.976420+0000","last_active":"2023-12-20T06:03:34.533851+0000","last_peered":"2023-12-20T06:03:34.533851+0000","last_clean":"2023-12-20T06:03:34.533851+0000","last_became_active":"2023-12-20T06:03:21.976141+0000","last_became_peered":"2023-12-20T06:03:21.976141+0000","last_unstale":"2023-12-20T06:03:34.533851+0000","last_undegraded":"2023-12-20T06:03:34.533851+0000","last_fullsized":"2023-12-20T06:03:34.533851+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:27:59.998522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529393+0000","last_change":"2023-12-20T06:03:21.567122+0000","last_active":"2023-12-20T06:03:34.529393+0000","last_peered":"2023-12-20T06:03:34.529393+0000","last_clean":"2023-12-20T06:03:34.529393+0000","last_became_active":"2023-12-20T06:03:21.566816+0000","last_became_peered":"2023-12-20T06:03:21.566816+0000","last_unstale":"2023-12-20T06:03:34.529393+0000","last_undegraded":"2023-12-20T06:03:34.529393+0000","last_fullsized":"2023-12-20T06:03:34.529393+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:41:11.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535626+0000","last_change":"2023-12-20T06:03:34.554385+0000","last_active":"2023-12-20T06:03:35.535626+0000","last_peered":"2023-12-20T06:03:35.535626+0000","last_clean":"2023-12-20T06:03:35.535626+0000","last_became_active":"2023-12-20T06:03:34.552807+0000","last_became_peered":"2023-12-20T06:03:34.552807+0000","last_unstale":"2023-12-20T06:03:35.535626+0000","last_undegraded":"2023-12-20T06:03:35.535626+0000","last_fullsized":"2023-12-20T06:03:35.535626+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:49:08.149121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572516+0000","last_change":"2023-12-20T06:03:34.140357+0000","last_active":"2023-12-20T06:03:35.572516+0000","last_peered":"2023-12-20T06:03:35.572516+0000","last_clean":"2023-12-20T06:03:35.572516+0000","last_became_active":"2023-12-20T06:03:34.140032+0000","last_became_peered":"2023-12-20T06:03:34.140032+0000","last_unstale":"2023-12-20T06:03:35.572516+0000","last_undegraded":"2023-12-20T06:03:35.572516+0000","last_fullsized":"2023-12-20T06:03:35.572516+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:06:27.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534600+0000","last_change":"2023-12-20T06:03:21.975113+0000","last_active":"2023-12-20T06:03:34.534600+0000","last_peered":"2023-12-20T06:03:34.534600+0000","last_clean":"2023-12-20T06:03:34.534600+0000","last_became_active":"2023-12-20T06:03:21.974862+0000","last_became_peered":"2023-12-20T06:03:21.974862+0000","last_unstale":"2023-12-20T06:03:34.534600+0000","last_undegraded":"2023-12-20T06:03:34.534600+0000","last_fullsized":"2023-12-20T06:03:34.534600+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:07:37.126937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534155+0000","last_change":"2023-12-20T06:03:21.569876+0000","last_active":"2023-12-20T06:03:34.534155+0000","last_peered":"2023-12-20T06:03:34.534155+0000","last_clean":"2023-12-20T06:03:34.534155+0000","last_became_active":"2023-12-20T06:03:21.568286+0000","last_became_peered":"2023-12-20T06:03:21.568286+0000","last_unstale":"2023-12-20T06:03:34.534155+0000","last_undegraded":"2023-12-20T06:03:34.534155+0000","last_fullsized":"2023-12-20T06:03:34.534155+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535865+0000","last_change":"2023-12-20T06:03:34.557531+0000","last_active":"2023-12-20T06:03:35.535865+0000","last_peered":"2023-12-20T06:03:35.535865+0000","last_clean":"2023-12-20T06:03:35.535865+0000","last_became_active":"2023-12-20T06:03:34.557228+0000","last_became_peered":"2023-12-20T06:03:34.557228+0000","last_unstale":"2023-12-20T06:03:35.535865+0000","last_undegraded":"2023-12-20T06:03:35.535865+0000","last_fullsized":"2023-12-20T06:03:35.535865+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:36:23.565638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535726+0000","last_change":"2023-12-20T06:03:34.123821+0000","last_active":"2023-12-20T06:03:35.535726+0000","last_peered":"2023-12-20T06:03:35.535726+0000","last_clean":"2023-12-20T06:03:35.535726+0000","last_became_active":"2023-12-20T06:03:34.123356+0000","last_became_peered":"2023-12-20T06:03:34.123356+0000","last_unstale":"2023-12-20T06:03:35.535726+0000","last_undegraded":"2023-12-20T06:03:35.535726+0000","last_fullsized":"2023-12-20T06:03:35.535726+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T18:01:48.061998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530997+0000","last_change":"2023-12-20T06:03:21.991437+0000","last_active":"2023-12-20T06:03:34.530997+0000","last_peered":"2023-12-20T06:03:34.530997+0000","last_clean":"2023-12-20T06:03:34.530997+0000","last_became_active":"2023-12-20T06:03:21.991098+0000","last_became_peered":"2023-12-20T06:03:21.991098+0000","last_unstale":"2023-12-20T06:03:34.530997+0000","last_undegraded":"2023-12-20T06:03:34.530997+0000","last_fullsized":"2023-12-20T06:03:34.530997+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:48.782916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"230'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540595+0000","last_change":"2023-12-20T06:03:21.574738+0000","last_active":"2023-12-20T06:03:34.540595+0000","last_peered":"2023-12-20T06:03:34.540595+0000","last_clean":"2023-12-20T06:03:34.540595+0000","last_became_active":"2023-12-20T06:03:21.574282+0000","last_became_peered":"2023-12-20T06:03:21.574282+0000","last_unstale":"2023-12-20T06:03:34.540595+0000","last_undegraded":"2023-12-20T06:03:34.540595+0000","last_fullsized":"2023-12-20T06:03:34.540595+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:43:56.675318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571986+0000","last_change":"2023-12-20T06:03:34.544724+0000","last_active":"2023-12-20T06:03:35.571986+0000","last_peered":"2023-12-20T06:03:35.571986+0000","last_clean":"2023-12-20T06:03:35.571986+0000","last_became_active":"2023-12-20T06:03:34.544423+0000","last_became_peered":"2023-12-20T06:03:34.544423+0000","last_unstale":"2023-12-20T06:03:35.571986+0000","last_undegraded":"2023-12-20T06:03:35.571986+0000","last_fullsized":"2023-12-20T06:03:35.571986+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:08:59.188281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575410+0000","last_change":"2023-12-20T06:03:34.120721+0000","last_active":"2023-12-20T06:03:35.575410+0000","last_peered":"2023-12-20T06:03:35.575410+0000","last_clean":"2023-12-20T06:03:35.575410+0000","last_became_active":"2023-12-20T06:03:34.120453+0000","last_became_peered":"2023-12-20T06:03:34.120453+0000","last_unstale":"2023-12-20T06:03:35.575410+0000","last_undegraded":"2023-12-20T06:03:35.575410+0000","last_fullsized":"2023-12-20T06:03:35.575410+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:53:02.531233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530488+0000","last_change":"2023-12-20T06:03:21.985967+0000","last_active":"2023-12-20T06:03:34.530488+0000","last_peered":"2023-12-20T06:03:34.530488+0000","last_clean":"2023-12-20T06:03:34.530488+0000","last_became_active":"2023-12-20T06:03:21.985460+0000","last_became_peered":"2023-12-20T06:03:21.985460+0000","last_unstale":"2023-12-20T06:03:34.530488+0000","last_undegraded":"2023-12-20T06:03:34.530488+0000","last_fullsized":"2023-12-20T06:03:34.530488+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:06:16.446428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540868+0000","last_change":"2023-12-20T06:03:21.573978+0000","last_active":"2023-12-20T06:03:34.540868+0000","last_peered":"2023-12-20T06:03:34.540868+0000","last_clean":"2023-12-20T06:03:34.540868+0000","last_became_active":"2023-12-20T06:03:21.573682+0000","last_became_peered":"2023-12-20T06:03:21.573682+0000","last_unstale":"2023-12-20T06:03:34.540868+0000","last_undegraded":"2023-12-20T06:03:34.540868+0000","last_fullsized":"2023-12-20T06:03:34.540868+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:43.873712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575230+0000","last_change":"2023-12-20T06:03:34.557806+0000","last_active":"2023-12-20T06:03:35.575230+0000","last_peered":"2023-12-20T06:03:35.575230+0000","last_clean":"2023-12-20T06:03:35.575230+0000","last_became_active":"2023-12-20T06:03:34.557222+0000","last_became_peered":"2023-12-20T06:03:34.557222+0000","last_unstale":"2023-12-20T06:03:35.575230+0000","last_undegraded":"2023-12-20T06:03:35.575230+0000","last_fullsized":"2023-12-20T06:03:35.575230+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:01:20.913691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535766+0000","last_change":"2023-12-20T06:03:34.135175+0000","last_active":"2023-12-20T06:03:35.535766+0000","last_peered":"2023-12-20T06:03:35.535766+0000","last_clean":"2023-12-20T06:03:35.535766+0000","last_became_active":"2023-12-20T06:03:34.134886+0000","last_became_peered":"2023-12-20T06:03:34.134886+0000","last_unstale":"2023-12-20T06:03:35.535766+0000","last_undegraded":"2023-12-20T06:03:35.535766+0000","last_fullsized":"2023-12-20T06:03:35.535766+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:56:24.405375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534543+0000","last_change":"2023-12-20T06:03:21.980139+0000","last_active":"2023-12-20T06:03:34.534543+0000","last_peered":"2023-12-20T06:03:34.534543+0000","last_clean":"2023-12-20T06:03:34.534543+0000","last_became_active":"2023-12-20T06:03:21.979858+0000","last_became_peered":"2023-12-20T06:03:21.979858+0000","last_unstale":"2023-12-20T06:03:34.534543+0000","last_undegraded":"2023-12-20T06:03:34.534543+0000","last_fullsized":"2023-12-20T06:03:34.534543+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:21:31.283475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530356+0000","last_change":"2023-12-20T06:03:21.570642+0000","last_active":"2023-12-20T06:03:34.530356+0000","last_peered":"2023-12-20T06:03:34.530356+0000","last_clean":"2023-12-20T06:03:34.530356+0000","last_became_active":"2023-12-20T06:03:21.569631+0000","last_became_peered":"2023-12-20T06:03:21.569631+0000","last_unstale":"2023-12-20T06:03:34.530356+0000","last_undegraded":"2023-12-20T06:03:34.530356+0000","last_fullsized":"2023-12-20T06:03:34.530356+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536417+0000","last_change":"2023-12-20T06:03:34.549025+0000","last_active":"2023-12-20T06:03:35.536417+0000","last_peered":"2023-12-20T06:03:35.536417+0000","last_clean":"2023-12-20T06:03:35.536417+0000","last_became_active":"2023-12-20T06:03:34.548725+0000","last_became_peered":"2023-12-20T06:03:34.548725+0000","last_unstale":"2023-12-20T06:03:35.536417+0000","last_undegraded":"2023-12-20T06:03:35.536417+0000","last_fullsized":"2023-12-20T06:03:35.536417+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:01:20.913691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572723+0000","last_change":"2023-12-20T06:03:34.133700+0000","last_active":"2023-12-20T06:03:35.572723+0000","last_peered":"2023-12-20T06:03:35.572723+0000","last_clean":"2023-12-20T06:03:35.572723+0000","last_became_active":"2023-12-20T06:03:34.133371+0000","last_became_peered":"2023-12-20T06:03:34.133371+0000","last_unstale":"2023-12-20T06:03:35.572723+0000","last_undegraded":"2023-12-20T06:03:35.572723+0000","last_fullsized":"2023-12-20T06:03:35.572723+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:23:13.107999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.538278+0000","last_change":"2023-12-20T06:03:21.963940+0000","last_active":"2023-12-20T06:03:34.538278+0000","last_peered":"2023-12-20T06:03:34.538278+0000","last_clean":"2023-12-20T06:03:34.538278+0000","last_became_active":"2023-12-20T06:03:21.963679+0000","last_became_peered":"2023-12-20T06:03:21.963679+0000","last_unstale":"2023-12-20T06:03:34.538278+0000","last_undegraded":"2023-12-20T06:03:34.538278+0000","last_fullsized":"2023-12-20T06:03:34.538278+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528505+0000","last_change":"2023-12-20T06:03:21.573888+0000","last_active":"2023-12-20T06:03:34.528505+0000","last_peered":"2023-12-20T06:03:34.528505+0000","last_clean":"2023-12-20T06:03:34.528505+0000","last_became_active":"2023-12-20T06:03:21.572823+0000","last_became_peered":"2023-12-20T06:03:21.572823+0000","last_unstale":"2023-12-20T06:03:34.528505+0000","last_undegraded":"2023-12-20T06:03:34.528505+0000","last_fullsized":"2023-12-20T06:03:34.528505+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572461+0000","last_change":"2023-12-20T06:03:34.549076+0000","last_active":"2023-12-20T06:03:35.572461+0000","last_peered":"2023-12-20T06:03:35.572461+0000","last_clean":"2023-12-20T06:03:35.572461+0000","last_became_active":"2023-12-20T06:03:34.548834+0000","last_became_peered":"2023-12-20T06:03:34.548834+0000","last_unstale":"2023-12-20T06:03:35.572461+0000","last_undegraded":"2023-12-20T06:03:35.572461+0000","last_fullsized":"2023-12-20T06:03:35.572461+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:38:13.606277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573233+0000","last_change":"2023-12-20T06:03:34.129063+0000","last_active":"2023-12-20T06:03:35.573233+0000","last_peered":"2023-12-20T06:03:35.573233+0000","last_clean":"2023-12-20T06:03:35.573233+0000","last_became_active":"2023-12-20T06:03:34.128730+0000","last_became_peered":"2023-12-20T06:03:34.128730+0000","last_unstale":"2023-12-20T06:03:35.573233+0000","last_undegraded":"2023-12-20T06:03:35.573233+0000","last_fullsized":"2023-12-20T06:03:35.573233+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:30:44.160478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530539+0000","last_change":"2023-12-20T06:03:21.983107+0000","last_active":"2023-12-20T06:03:34.530539+0000","last_peered":"2023-12-20T06:03:34.530539+0000","last_clean":"2023-12-20T06:03:34.530539+0000","last_became_active":"2023-12-20T06:03:21.981255+0000","last_became_peered":"2023-12-20T06:03:21.981255+0000","last_unstale":"2023-12-20T06:03:34.530539+0000","last_undegraded":"2023-12-20T06:03:34.530539+0000","last_fullsized":"2023-12-20T06:03:34.530539+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537824+0000","last_change":"2023-12-20T06:03:21.558503+0000","last_active":"2023-12-20T06:03:34.537824+0000","last_peered":"2023-12-20T06:03:34.537824+0000","last_clean":"2023-12-20T06:03:34.537824+0000","last_became_active":"2023-12-20T06:03:21.557644+0000","last_became_peered":"2023-12-20T06:03:21.557644+0000","last_unstale":"2023-12-20T06:03:34.537824+0000","last_undegraded":"2023-12-20T06:03:34.537824+0000","last_fullsized":"2023-12-20T06:03:34.537824+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:48.619779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571077+0000","last_change":"2023-12-20T06:03:34.542787+0000","last_active":"2023-12-20T06:03:35.571077+0000","last_peered":"2023-12-20T06:03:35.571077+0000","last_clean":"2023-12-20T06:03:35.571077+0000","last_became_active":"2023-12-20T06:03:34.542503+0000","last_became_peered":"2023-12-20T06:03:34.542503+0000","last_unstale":"2023-12-20T06:03:35.571077+0000","last_undegraded":"2023-12-20T06:03:35.571077+0000","last_fullsized":"2023-12-20T06:03:35.571077+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:37:53.065801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571301+0000","last_change":"2023-12-20T06:03:34.137376+0000","last_active":"2023-12-20T06:03:35.571301+0000","last_peered":"2023-12-20T06:03:35.571301+0000","last_clean":"2023-12-20T06:03:35.571301+0000","last_became_active":"2023-12-20T06:03:34.137073+0000","last_became_peered":"2023-12-20T06:03:34.137073+0000","last_unstale":"2023-12-20T06:03:35.571301+0000","last_undegraded":"2023-12-20T06:03:35.571301+0000","last_fullsized":"2023-12-20T06:03:35.571301+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:57:18.842438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529301+0000","last_change":"2023-12-20T06:03:21.986446+0000","last_active":"2023-12-20T06:03:34.529301+0000","last_peered":"2023-12-20T06:03:34.529301+0000","last_clean":"2023-12-20T06:03:34.529301+0000","last_became_active":"2023-12-20T06:03:21.986120+0000","last_became_peered":"2023-12-20T06:03:21.986120+0000","last_unstale":"2023-12-20T06:03:34.529301+0000","last_undegraded":"2023-12-20T06:03:34.529301+0000","last_fullsized":"2023-12-20T06:03:34.529301+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:27:59.998522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534454+0000","last_change":"2023-12-20T06:03:21.570337+0000","last_active":"2023-12-20T06:03:34.534454+0000","last_peered":"2023-12-20T06:03:34.534454+0000","last_clean":"2023-12-20T06:03:34.534454+0000","last_became_active":"2023-12-20T06:03:21.569986+0000","last_became_peered":"2023-12-20T06:03:21.569986+0000","last_unstale":"2023-12-20T06:03:34.534454+0000","last_undegraded":"2023-12-20T06:03:34.534454+0000","last_fullsized":"2023-12-20T06:03:34.534454+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:28.719459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536980+0000","last_change":"2023-12-20T06:03:34.544566+0000","last_active":"2023-12-20T06:03:35.536980+0000","last_peered":"2023-12-20T06:03:35.536980+0000","last_clean":"2023-12-20T06:03:35.536980+0000","last_became_active":"2023-12-20T06:03:34.544303+0000","last_became_peered":"2023-12-20T06:03:34.544303+0000","last_unstale":"2023-12-20T06:03:35.536980+0000","last_undegraded":"2023-12-20T06:03:35.536980+0000","last_fullsized":"2023-12-20T06:03:35.536980+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:29:03.073420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573068+0000","last_change":"2023-12-20T06:03:34.135212+0000","last_active":"2023-12-20T06:03:35.573068+0000","last_peered":"2023-12-20T06:03:35.573068+0000","last_clean":"2023-12-20T06:03:35.573068+0000","last_became_active":"2023-12-20T06:03:34.134875+0000","last_became_peered":"2023-12-20T06:03:34.134875+0000","last_unstale":"2023-12-20T06:03:35.573068+0000","last_undegraded":"2023-12-20T06:03:35.573068+0000","last_fullsized":"2023-12-20T06:03:35.573068+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:24.762096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537453+0000","last_change":"2023-12-20T06:03:21.968100+0000","last_active":"2023-12-20T06:03:34.537453+0000","last_peered":"2023-12-20T06:03:34.537453+0000","last_clean":"2023-12-20T06:03:34.537453+0000","last_became_active":"2023-12-20T06:03:21.967593+0000","last_became_peered":"2023-12-20T06:03:21.967593+0000","last_unstale":"2023-12-20T06:03:34.537453+0000","last_undegraded":"2023-12-20T06:03:34.537453+0000","last_fullsized":"2023-12-20T06:03:34.537453+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:00:01.717868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529497+0000","last_change":"2023-12-20T06:03:21.568281+0000","last_active":"2023-12-20T06:03:34.529497+0000","last_peered":"2023-12-20T06:03:34.529497+0000","last_clean":"2023-12-20T06:03:34.529497+0000","last_became_active":"2023-12-20T06:03:21.567976+0000","last_became_peered":"2023-12-20T06:03:21.567976+0000","last_unstale":"2023-12-20T06:03:34.529497+0000","last_undegraded":"2023-12-20T06:03:34.529497+0000","last_fullsized":"2023-12-20T06:03:34.529497+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:03:18.004400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535774+0000","last_change":"2023-12-20T06:03:34.546321+0000","last_active":"2023-12-20T06:03:35.535774+0000","last_peered":"2023-12-20T06:03:35.535774+0000","last_clean":"2023-12-20T06:03:35.535774+0000","last_became_active":"2023-12-20T06:03:34.545968+0000","last_became_peered":"2023-12-20T06:03:34.545968+0000","last_unstale":"2023-12-20T06:03:35.535774+0000","last_undegraded":"2023-12-20T06:03:35.535774+0000","last_fullsized":"2023-12-20T06:03:35.535774+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:14:15.201659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"233'4","reported_seq":15,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574549+0000","last_change":"2023-12-20T06:03:34.132816+0000","last_active":"2023-12-20T06:03:35.574549+0000","last_peered":"2023-12-20T06:03:35.574549+0000","last_clean":"2023-12-20T06:03:35.574549+0000","last_became_active":"2023-12-20T06:03:34.132457+0000","last_became_peered":"2023-12-20T06:03:34.132457+0000","last_unstale":"2023-12-20T06:03:35.574549+0000","last_undegraded":"2023-12-20T06:03:35.574549+0000","last_fullsized":"2023-12-20T06:03:35.574549+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:57.468706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.539944+0000","last_change":"2023-12-20T06:03:21.973527+0000","last_active":"2023-12-20T06:03:34.539944+0000","last_peered":"2023-12-20T06:03:34.539944+0000","last_clean":"2023-12-20T06:03:34.539944+0000","last_became_active":"2023-12-20T06:03:21.973178+0000","last_became_peered":"2023-12-20T06:03:21.973178+0000","last_unstale":"2023-12-20T06:03:34.539944+0000","last_undegraded":"2023-12-20T06:03:34.539944+0000","last_fullsized":"2023-12-20T06:03:34.539944+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:55:50.794593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528875+0000","last_change":"2023-12-20T06:03:21.567717+0000","last_active":"2023-12-20T06:03:34.528875+0000","last_peered":"2023-12-20T06:03:34.528875+0000","last_clean":"2023-12-20T06:03:34.528875+0000","last_became_active":"2023-12-20T06:03:21.567425+0000","last_became_peered":"2023-12-20T06:03:21.567425+0000","last_unstale":"2023-12-20T06:03:34.528875+0000","last_undegraded":"2023-12-20T06:03:34.528875+0000","last_fullsized":"2023-12-20T06:03:34.528875+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:45.186933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"215'3223","reported_seq":4093,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.535519+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:03:34.535519+0000","last_peered":"2023-12-20T06:03:34.535519+0000","last_clean":"2023-12-20T06:03:34.535519+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:03:34.535519+0000","last_undegraded":"2023-12-20T06:03:34.535519+0000","last_fullsized":"2023-12-20T06:03:34.535519+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3223,"log_dups_size":0,"ondisk_log_size":3223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":561,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":561,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3223,"ondisk_log_size":3223,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575982,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60736,"kb_used_data":3448,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311104,"statfs":{"total":96636764160,"available":96574570496,"internally_reserved":0,"allocated":3530752,"data_stored":2024165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.69599999999999995,"15min":0.72799999999999998},"min":{"1min":0.45400000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":1.028,"15min":1.042},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.99099999999999999,"15min":1.161},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71499999999999997,"15min":0.74199999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":0.996,"15min":1.1279999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.71599999999999997,"15min":0.745},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.85499999999999998,"5min":0.96699999999999997,"15min":1.032},"last":0.63}]},{"osd":2,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.70599999999999996,"15min":0.73899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84499999999999997,"5min":0.95399999999999996,"15min":1.014},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.49099999999999999},"max":{"1min":0.88700000000000001,"5min":1.0089999999999999,"15min":1.0189999999999999},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.47699999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.85299999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.89900000000000002,"5min":1.0660000000000001,"15min":1.0840000000000001},"last":0.61099999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.04},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.02,"15min":1.02},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68300000000000005,"15min":0.66800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.052},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.875,"5min":1.0229999999999999,"15min":1.103},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76300000000000001,"15min":0.746},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72399999999999998,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93500000000000005,"5min":1.0740000000000001,"15min":1.081},"last":0.68300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575982,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51972,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319868,"statfs":{"total":96636764160,"available":96583544832,"internally_reserved":0,"allocated":2945024,"data_stored":1434317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71199999999999997,"15min":0.73999999999999999},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92400000000000004,"5min":1.016,"15min":1.0980000000000001},"last":0.63},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72999999999999998,"15min":0.75},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.006,"5min":1.016,"15min":1.05},"last":0.76200000000000001}]},{"osd":1,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.47399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90100000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91200000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81200000000000006}]},{"osd":2,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76900000000000002,"15min":0.78100000000000003},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97699999999999998,"5min":1.069,"15min":1.069},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.54400000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.93100000000000005,"5min":0.97799999999999998,"15min":1.0760000000000001},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78900000000000003},"min":{"1min":0.61299999999999999,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.0109999999999999,"5min":1.077,"15min":1.077},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.997,"5min":1.052,"15min":1.052},"last":0.66600000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.36099999999999999},"max":{"1min":0.96299999999999997,"5min":1.05,"15min":1.0529999999999999},"last":0.751},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66400000000000003,"15min":0.65100000000000002},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64900000000000002,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.55700000000000005,"5min":0.48899999999999999,"15min":0.45300000000000001},"max":{"1min":1.0429999999999999,"5min":1.103,"15min":1.1240000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.73399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575983,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51956,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319884,"statfs":{"total":96636764160,"available":96583561216,"internally_reserved":0,"allocated":2928640,"data_stored":1433657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71399999999999997,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91600000000000004,"5min":0.94799999999999995,"15min":1.077},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71499999999999997,"15min":0.746},"min":{"1min":0.45600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.80300000000000005,"5min":0.96699999999999997,"15min":1.028},"last":0.45600000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":0.875,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88500000000000001,"5min":0.98799999999999999,"15min":1.0209999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.751},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89000000000000001,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.75700000000000001,"15min":0.77000000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.48299999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0680000000000001},"last":0.53400000000000003}]},{"osd":3,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"max":{"1min":0.91000000000000003,"5min":1.05,"15min":1.0669999999999999},"last":0.624},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.0629999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.56000000000000005,"5min":0.42199999999999999,"15min":0.38200000000000001},"max":{"1min":0.94399999999999995,"5min":1.0640000000000001,"15min":1.077},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.64800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"max":{"1min":1.0649999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.0449999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.59299999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73799999999999999,"15min":0.70999999999999996},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.153,"15min":1.153},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.48299999999999998},"max":{"1min":0.98799999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.68600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575984,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47884,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323956,"statfs":{"total":96636764160,"available":96587730944,"internally_reserved":0,"allocated":2953216,"data_stored":1435895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.69299999999999995,"15min":0.72899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96499999999999997,"5min":0.98999999999999999,"15min":1.0209999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.69299999999999995,"15min":0.72299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.80900000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73099999999999998,"15min":0.752},"min":{"1min":0.498,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":0.89600000000000002,"5min":0.95699999999999996,"15min":1.0900000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73899999999999999,"15min":0.76400000000000001},"min":{"1min":0.47899999999999998,"5min":0.47399999999999998,"15min":0.44500000000000001},"max":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.746,"15min":0.76900000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98199999999999998,"5min":1.0089999999999999,"15min":1.101},"last":0.68100000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.75},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49199999999999999},"max":{"1min":0.88,"5min":1.014,"15min":1.014},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.80100000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1,"5min":1.04,"15min":1.081},"last":0.59499999999999997}]},{"osd":5,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64700000000000002,"15min":0.65000000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98099999999999998,"5min":1.05,"15min":1.05},"last":0.754},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.47599999999999998,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.96999999999999997,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.1379999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.41199999999999998},"max":{"1min":1.056,"5min":1.107,"15min":1.111},"last":0.80000000000000004}]},{"osd":7,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72899999999999998,"15min":0.72999999999999998},"min":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":1.0329999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.626,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.1100000000000001,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.77000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47880,"kb_used_data":2880,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323960,"statfs":{"total":96636764160,"available":96587735040,"internally_reserved":0,"allocated":2949120,"data_stored":1437114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.46899999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.748,"15min":2.8130000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.49299999999999999,"15min":0.52600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":1.357},"last":0.318}]},{"osd":1,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45700000000000002,"15min":0.47599999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65600000000000003,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53000000000000003,"15min":0.56699999999999995},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.57599999999999996,"5min":0.81799999999999995,"15min":1.4219999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.56399999999999995,"15min":0.58999999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.57999999999999996,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.53800000000000003,"15min":0.56999999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69099999999999995,"5min":0.86099999999999999,"15min":2.944},"last":0.436}]},{"osd":4,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77300000000000002,"15min":0.81499999999999995},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.98999999999999999,"5min":1.2090000000000001,"15min":1.669},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75900000000000001,"15min":0.80800000000000005},"min":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.86599999999999999,"5min":1.1299999999999999,"15min":1.99},"last":0.69199999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79600000000000004,"15min":0.82899999999999996},"min":{"1min":0.65100000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.93600000000000005,"5min":1.22,"15min":1.901},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.76300000000000001,"15min":0.81999999999999995},"min":{"1min":0.58799999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.875,"5min":1.196,"15min":2.8639999999999999},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.77500000000000002,"15min":0.81399999999999995},"min":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95599999999999996,"5min":1.194,"15min":1.575},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80500000000000005,"15min":0.84499999999999997},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0580000000000001,"5min":1.155,"15min":1.948},"last":0.79300000000000004}]},{"osd":7,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91900000000000004,"5min":1.1579999999999999,"15min":2.9049999999999998},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80700000000000005,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.502,"15min":0.502},"max":{"1min":1.0680000000000001,"5min":1.1850000000000001,"15min":1.754},"last":0.89100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575982,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51980,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319860,"statfs":{"total":96636764160,"available":96583536640,"internally_reserved":0,"allocated":2953216,"data_stored":1436608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.45000000000000001,"15min":0.45500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52900000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.60699999999999998,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.79800000000000004,"15min":0.82199999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30499999999999999},"max":{"1min":0.61199999999999999,"5min":0.92600000000000005,"15min":1.004},"last":0.51200000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.65500000000000003,"15min":0.67400000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.59199999999999997,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":1.254},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77900000000000003,"15min":0.79600000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":1.605,"15min":1.605},"last":0.67200000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97099999999999997,"5min":1.329,"15min":1.329},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98199999999999998,"5min":1.0740000000000001,"15min":1.095},"last":0.76100000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92200000000000004,"5min":1.919,"15min":1.919},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.95099999999999996,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.93200000000000005}]},{"osd":7,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0049999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.75},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.88200000000000001,"15min":0.88500000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.996,"5min":1.292,"15min":1.292},"last":0.98199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575984,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60172,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311668,"statfs":{"total":96636764160,"available":96575148032,"internally_reserved":0,"allocated":2953216,"data_stored":1435435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47899999999999998,"15min":0.46200000000000002},"min":{"1min":0.26500000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59299999999999997,"5min":0.746,"15min":0.81000000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.29399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.28599999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.71999999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":0.88800000000000001,"15min":0.93700000000000006},"last":0.45500000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.42999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.90100000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.93500000000000005},"last":0.44}]},{"osd":4,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.78500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92000000000000004,"5min":1.107,"15min":1.321},"last":0.875},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.995,"5min":1.139,"15min":1.25},"last":0.79300000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78700000000000003,"15min":0.80200000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.97899999999999998,"5min":1.224,"15min":1.2849999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0740000000000001,"5min":1.1990000000000001,"15min":1.4330000000000001},"last":0.95299999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.82399999999999995},"min":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.2430000000000001,"15min":1.341},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.56399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1379999999999999,"5min":1.177,"15min":1.3839999999999999},"last":0.82699999999999996}]},{"osd":7,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0900000000000001,"5min":1.272,"15min":1.4470000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0509999999999999,"5min":1.1599999999999999,"15min":1.3839999999999999},"last":0.91700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575983,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56644,"kb_used_data":3452,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315196,"statfs":{"total":96636764160,"available":96578760704,"internally_reserved":0,"allocated":3534848,"data_stored":2024731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.318},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33000000000000002},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.40300000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.60299999999999998}]},{"osd":4,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82999999999999996,"15min":0.82599999999999996},"min":{"1min":0.56699999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.081}]},{"osd":5,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.86099999999999999,"15min":0.85099999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.873,"5min":0.83399999999999996,"15min":0.84799999999999998},"min":{"1min":0.54600000000000004,"5min":0.495,"15min":0.495},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":1.1220000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.032},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.89100000000000001,"15min":0.871},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57099999999999995},"max":{"1min":5.6310000000000002,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.98599999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":5.7160000000000002,"5min":5.7160000000000002,"15min":5.7160000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.012,"5min":0.88200000000000001,"15min":0.871},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":4.2119999999999997,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.94399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:03:38.674 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:03:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.850+0000 7f2ab6a8c640 1 Processor -- start 2023-12-20T06:03:38.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.850+0000 7f2ab6a8c640 1 -- start start 2023-12-20T06:03:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2ab6a8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2ab015ce60 0x7f2ab00c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2ab6a8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ab00713e0 0x7f2ab0071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2ab6a8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2ab0071d80 con 0x7f2ab0160b30 2023-12-20T06:03:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2ab6a8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab0161690 con 0x7f2ab00713e0 2023-12-20T06:03:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2ab6a8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2ab00c9a40 con 0x7f2ab015ce60 2023-12-20T06:03:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2aaffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ab00713e0 0x7f2ab0071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2aaffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f2ab00713e0 0x7f2ab0071840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35580/0 (socket says 172.21.15.32:35580) 2023-12-20T06:03:38.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2aaffff640 1 -- 172.21.15.32:0/756777029 learned_addr learned my addr 172.21.15.32:0/756777029 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2ab4828640 1 --2- 172.21.15.32:0/756777029 >> v2:172.21.15.39:3300/0 conn(0x7f2ab015ce60 0x7f2ab00c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2aaffff640 1 -- 172.21.15.32:0/756777029 >> v2:172.21.15.39:3300/0 conn(0x7f2ab015ce60 msgr2=0x7f2ab00c9ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2aaffff640 1 --2- 172.21.15.32:0/756777029 >> v2:172.21.15.39:3300/0 conn(0x7f2ab015ce60 0x7f2ab00c9ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2aaffff640 1 -- 172.21.15.32:0/756777029 >> v1:172.21.15.32:6789/0 conn(0x7f2ab0160b30 legacy=0x7f2ab0160f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2aaffff640 1 -- 172.21.15.32:0/756777029 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2ab00c9180 con 0x7f2ab00713e0 2023-12-20T06:03:38.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.854+0000 7f2aaf7fe640 1 -- 172.21.15.32:0/756777029 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3464401841 0 0) 0x7f2ab0071d80 con 0x7f2ab0160b30 2023-12-20T06:03:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2aaffff640 1 --2- 172.21.15.32:0/756777029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab00713e0 0x7f2ab0071840 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00cc230 tx=0x7f2a9c01c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=190ca44d35bb8240 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2aaf7fe640 1 -- 172.21.15.32:0/756777029 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0ab5f0 con 0x7f2ab00713e0 2023-12-20T06:03:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2aaf7fe640 1 -- 172.21.15.32:0/756777029 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c0ab750 con 0x7f2ab00713e0 2023-12-20T06:03:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2aaf7fe640 1 -- 172.21.15.32:0/756777029 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0abae0 con 0x7f2ab00713e0 2023-12-20T06:03:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/756777029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab00713e0 msgr2=0x7f2ab0071840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 --2- 172.21.15.32:0/756777029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab00713e0 0x7f2ab0071840 secure :-1 s=READY pgs=2303 cs=0 l=1 rev1=1 crypto rx=0x7f2ab00cc230 tx=0x7f2a9c01c500 comp rx=0 tx=0).stop 2023-12-20T06:03:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/756777029 shutdown_connections 2023-12-20T06:03:38.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 --2- 172.21.15.32:0/756777029 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab00713e0 0x7f2ab0071840 unknown :-1 s=CLOSED pgs=2303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 --2- 172.21.15.32:0/756777029 >> v2:172.21.15.39:3300/0 conn(0x7f2ab015ce60 0x7f2ab00c9ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/756777029 >> 172.21.15.32:0/756777029 conn(0x7f2ab006a320 msgr2=0x7f2ab00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/756777029 shutdown_connections 2023-12-20T06:03:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/756777029 wait complete. 2023-12-20T06:03:38.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 Processor -- start 2023-12-20T06:03:38.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 -- start start 2023-12-20T06:03:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ab00713e0 0x7f2ab01527e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab015ce60 0x7f2ab015edf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab0160b30 0x7f2ab00cd690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab015e400 con 0x7f2ab015ce60 2023-12-20T06:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab0160390 con 0x7f2ab0160b30 2023-12-20T06:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2ab6a8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2ab015d780 con 0x7f2ab00713e0 2023-12-20T06:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2aaffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab015ce60 0x7f2ab015edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.858+0000 7f2aaffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab015ce60 0x7f2ab015edf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35588/0 (socket says 172.21.15.32:35588) 2023-12-20T06:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aaffff640 1 -- 172.21.15.32:0/2730078328 learned_addr learned my addr 172.21.15.32:0/2730078328 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2ab4828640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ab00713e0 0x7f2ab01527e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2ab5029640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab0160b30 0x7f2ab00cd690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aaffff640 1 -- 172.21.15.32:0/2730078328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ab00713e0 msgr2=0x7f2ab01527e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aaffff640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ab00713e0 0x7f2ab01527e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aaffff640 1 -- 172.21.15.32:0/2730078328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab0160b30 msgr2=0x7f2ab00cd690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aaffff640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab0160b30 0x7f2ab00cd690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aaffff640 1 -- 172.21.15.32:0/2730078328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2a9c0094d0 con 0x7f2ab015ce60 2023-12-20T06:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2ab5029640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab0160b30 0x7f2ab00cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aaffff640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab015ce60 0x7f2ab015edf0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0ad040 tx=0x7f2a9c01ad40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aad7fa640 1 -- 172.21.15.32:0/2730078328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0be5f0 con 0x7f2ab015ce60 2023-12-20T06:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aad7fa640 1 -- 172.21.15.32:0/2730078328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2a9c0be750 con 0x7f2ab015ce60 2023-12-20T06:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aad7fa640 1 -- 172.21.15.32:0/2730078328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0beae0 con 0x7f2ab015ce60 2023-12-20T06:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/2730078328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2ab015da60 con 0x7f2ab015ce60 2023-12-20T06:03:38.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/2730078328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2ab015e020 con 0x7f2ab015ce60 2023-12-20T06:03:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/2730078328 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2ab015e540 con 0x7f2ab015ce60 2023-12-20T06:03:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aad7fa640 1 -- 172.21.15.32:0/2730078328 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f2a9c0b43f0 con 0x7f2ab015ce60 2023-12-20T06:03:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aad7fa640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2a94065ca0 0x7f2a94068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.862+0000 7f2aad7fa640 1 -- 172.21.15.32:0/2730078328 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f2a9c0a8050 con 0x7f2ab015ce60 2023-12-20T06:03:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.866+0000 7f2ab4828640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2a94065ca0 0x7f2a94068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:38.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.866+0000 7f2ab4828640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2a94065ca0 0x7f2a94068160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009ad0 tx=0x7f2aa4008040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:38.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:38.866+0000 7f2aad7fa640 1 -- 172.21.15.32:0/2730078328 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2ab015e540 con 0x7f2ab015ce60 2023-12-20T06:03:39.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.022+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/2730078328 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f2ab01618b0 con 0x7f2a94065ca0 2023-12-20T06:03:39.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.034+0000 7f2aad7fa640 1 -- 172.21.15.32:0/2730078328 <== mgr.10956 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+528501 (secure 0 0 0) 0x7f2aa40119a0 con 0x7f2a94065ca0 2023-12-20T06:03:39.042 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:39.050 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:03:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/2730078328 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2a94065ca0 msgr2=0x7f2a94068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2a94065ca0 0x7f2a94068160 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009ad0 tx=0x7f2aa4008040 comp rx=0 tx=0).stop 2023-12-20T06:03:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/2730078328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab015ce60 msgr2=0x7f2ab015edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab015ce60 0x7f2ab015edf0 secure :-1 s=READY pgs=2304 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c0ad040 tx=0x7f2a9c01ad40 comp rx=0 tx=0).stop 2023-12-20T06:03:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/2730078328 shutdown_connections 2023-12-20T06:03:39.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2a94065ca0 0x7f2a94068160 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2ab0160b30 0x7f2ab00cd690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2ab015ce60 0x7f2ab015edf0 unknown :-1 s=CLOSED pgs=2304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 --2- 172.21.15.32:0/2730078328 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2ab00713e0 0x7f2ab01527e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/2730078328 >> 172.21.15.32:0/2730078328 conn(0x7f2ab006a320 msgr2=0x7f2ab00cc660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/2730078328 shutdown_connections 2023-12-20T06:03:39.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.042+0000 7f2ab6a8c640 1 -- 172.21.15.32:0/2730078328 wait complete. 2023-12-20T06:03:39.062 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":41,"stamp":"2023-12-20T06:03:37.661112+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":596333,"num_objects":45,"num_object_clones":0,"num_object_copies":90,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":45,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":652,"num_write_kb":2446,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":27,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3302,"ondisk_log_size":3302,"up":386,"acting":386,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":291,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":429224,"kb_used_data":24168,"kb_used_omap":0,"kb_used_meta":404992,"kb_avail":754545496,"statfs":{"total":773094113280,"available":772654587904,"internally_reserved":0,"allocated":24748032,"data_stored":12661922,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":414711808},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":36,"apply_latency_ms":36,"commit_latency_ns":36000000,"apply_latency_ns":36000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":3349,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":45,"num_write_kb":15,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.570006"},"pg_stats":[{"pgid":"31.3d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574240+0000","last_change":"2023-12-20T06:03:34.545959+0000","last_active":"2023-12-20T06:03:35.574240+0000","last_peered":"2023-12-20T06:03:35.574240+0000","last_clean":"2023-12-20T06:03:35.574240+0000","last_became_active":"2023-12-20T06:03:34.545677+0000","last_became_peered":"2023-12-20T06:03:34.545677+0000","last_unstale":"2023-12-20T06:03:35.574240+0000","last_undegraded":"2023-12-20T06:03:35.574240+0000","last_fullsized":"2023-12-20T06:03:35.574240+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:54:36.284984+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3f","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.541147+0000","last_change":"2023-12-20T06:03:21.976746+0000","last_active":"2023-12-20T06:03:34.541147+0000","last_peered":"2023-12-20T06:03:34.541147+0000","last_clean":"2023-12-20T06:03:34.541147+0000","last_became_active":"2023-12-20T06:03:21.976467+0000","last_became_peered":"2023-12-20T06:03:21.976467+0000","last_unstale":"2023-12-20T06:03:34.541147+0000","last_undegraded":"2023-12-20T06:03:34.541147+0000","last_fullsized":"2023-12-20T06:03:34.541147+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:13.213910+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.3c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573792+0000","last_change":"2023-12-20T06:03:34.547546+0000","last_active":"2023-12-20T06:03:35.573792+0000","last_peered":"2023-12-20T06:03:35.573792+0000","last_clean":"2023-12-20T06:03:35.573792+0000","last_became_active":"2023-12-20T06:03:34.547300+0000","last_became_peered":"2023-12-20T06:03:34.547300+0000","last_unstale":"2023-12-20T06:03:35.573792+0000","last_undegraded":"2023-12-20T06:03:35.573792+0000","last_fullsized":"2023-12-20T06:03:35.573792+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:28.794520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.3e","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.542778+0000","last_change":"2023-12-20T06:03:21.988965+0000","last_active":"2023-12-20T06:03:34.542778+0000","last_peered":"2023-12-20T06:03:34.542778+0000","last_clean":"2023-12-20T06:03:34.542778+0000","last_became_active":"2023-12-20T06:03:21.988334+0000","last_became_peered":"2023-12-20T06:03:21.988334+0000","last_unstale":"2023-12-20T06:03:34.542778+0000","last_undegraded":"2023-12-20T06:03:34.542778+0000","last_fullsized":"2023-12-20T06:03:34.542778+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:04.642017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.3f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575378+0000","last_change":"2023-12-20T06:03:34.556254+0000","last_active":"2023-12-20T06:03:35.575378+0000","last_peered":"2023-12-20T06:03:35.575378+0000","last_clean":"2023-12-20T06:03:35.575378+0000","last_became_active":"2023-12-20T06:03:34.555988+0000","last_became_peered":"2023-12-20T06:03:34.555988+0000","last_unstale":"2023-12-20T06:03:35.575378+0000","last_undegraded":"2023-12-20T06:03:35.575378+0000","last_fullsized":"2023-12-20T06:03:35.575378+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:42.632244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3d","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530357+0000","last_change":"2023-12-20T06:03:21.987434+0000","last_active":"2023-12-20T06:03:34.530357+0000","last_peered":"2023-12-20T06:03:34.530357+0000","last_clean":"2023-12-20T06:03:34.530357+0000","last_became_active":"2023-12-20T06:03:21.987118+0000","last_became_peered":"2023-12-20T06:03:21.987118+0000","last_unstale":"2023-12-20T06:03:34.530357+0000","last_undegraded":"2023-12-20T06:03:34.530357+0000","last_fullsized":"2023-12-20T06:03:34.530357+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575165+0000","last_change":"2023-12-20T06:03:34.549559+0000","last_active":"2023-12-20T06:03:35.575165+0000","last_peered":"2023-12-20T06:03:35.575165+0000","last_clean":"2023-12-20T06:03:35.575165+0000","last_became_active":"2023-12-20T06:03:34.549296+0000","last_became_peered":"2023-12-20T06:03:34.549296+0000","last_unstale":"2023-12-20T06:03:35.575165+0000","last_undegraded":"2023-12-20T06:03:35.575165+0000","last_fullsized":"2023-12-20T06:03:35.575165+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:39:44.183895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.3c","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.535103+0000","last_change":"2023-12-20T06:03:21.981328+0000","last_active":"2023-12-20T06:03:34.535103+0000","last_peered":"2023-12-20T06:03:34.535103+0000","last_clean":"2023-12-20T06:03:34.535103+0000","last_became_active":"2023-12-20T06:03:21.981001+0000","last_became_peered":"2023-12-20T06:03:21.981001+0000","last_unstale":"2023-12-20T06:03:34.535103+0000","last_undegraded":"2023-12-20T06:03:34.535103+0000","last_fullsized":"2023-12-20T06:03:34.535103+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:48.782916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.39","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570635+0000","last_change":"2023-12-20T06:03:34.560291+0000","last_active":"2023-12-20T06:03:35.570635+0000","last_peered":"2023-12-20T06:03:35.570635+0000","last_clean":"2023-12-20T06:03:35.570635+0000","last_became_active":"2023-12-20T06:03:34.559999+0000","last_became_peered":"2023-12-20T06:03:34.559999+0000","last_unstale":"2023-12-20T06:03:35.570635+0000","last_undegraded":"2023-12-20T06:03:35.570635+0000","last_fullsized":"2023-12-20T06:03:35.570635+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:08:10.768420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.3b","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537739+0000","last_change":"2023-12-20T06:03:21.978878+0000","last_active":"2023-12-20T06:03:34.537739+0000","last_peered":"2023-12-20T06:03:34.537739+0000","last_clean":"2023-12-20T06:03:34.537739+0000","last_became_active":"2023-12-20T06:03:21.977017+0000","last_became_peered":"2023-12-20T06:03:21.977017+0000","last_unstale":"2023-12-20T06:03:34.537739+0000","last_undegraded":"2023-12-20T06:03:34.537739+0000","last_fullsized":"2023-12-20T06:03:34.537739+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:55:10.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.38","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536476+0000","last_change":"2023-12-20T06:03:34.543028+0000","last_active":"2023-12-20T06:03:35.536476+0000","last_peered":"2023-12-20T06:03:35.536476+0000","last_clean":"2023-12-20T06:03:35.536476+0000","last_became_active":"2023-12-20T06:03:34.542240+0000","last_became_peered":"2023-12-20T06:03:34.542240+0000","last_unstale":"2023-12-20T06:03:35.536476+0000","last_undegraded":"2023-12-20T06:03:35.536476+0000","last_fullsized":"2023-12-20T06:03:35.536476+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:46:47.498001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.3a","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.524859+0000","last_change":"2023-12-20T06:03:21.976982+0000","last_active":"2023-12-20T06:03:34.524859+0000","last_peered":"2023-12-20T06:03:34.524859+0000","last_clean":"2023-12-20T06:03:34.524859+0000","last_became_active":"2023-12-20T06:03:21.976721+0000","last_became_peered":"2023-12-20T06:03:21.976721+0000","last_unstale":"2023-12-20T06:03:34.524859+0000","last_undegraded":"2023-12-20T06:03:34.524859+0000","last_fullsized":"2023-12-20T06:03:34.524859+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:57.966779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.3b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.534878+0000","last_change":"2023-12-20T06:03:34.539532+0000","last_active":"2023-12-20T06:03:35.534878+0000","last_peered":"2023-12-20T06:03:35.534878+0000","last_clean":"2023-12-20T06:03:35.534878+0000","last_became_active":"2023-12-20T06:03:34.539195+0000","last_became_peered":"2023-12-20T06:03:34.539195+0000","last_unstale":"2023-12-20T06:03:35.534878+0000","last_undegraded":"2023-12-20T06:03:35.534878+0000","last_fullsized":"2023-12-20T06:03:35.534878+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:25:10.820979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.39","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529760+0000","last_change":"2023-12-20T06:03:21.979840+0000","last_active":"2023-12-20T06:03:34.529760+0000","last_peered":"2023-12-20T06:03:34.529760+0000","last_clean":"2023-12-20T06:03:34.529760+0000","last_became_active":"2023-12-20T06:03:21.979570+0000","last_became_peered":"2023-12-20T06:03:21.979570+0000","last_unstale":"2023-12-20T06:03:34.529760+0000","last_undegraded":"2023-12-20T06:03:34.529760+0000","last_fullsized":"2023-12-20T06:03:34.529760+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.3a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572992+0000","last_change":"2023-12-20T06:03:34.555425+0000","last_active":"2023-12-20T06:03:35.572992+0000","last_peered":"2023-12-20T06:03:35.572992+0000","last_clean":"2023-12-20T06:03:35.572992+0000","last_became_active":"2023-12-20T06:03:34.555195+0000","last_became_peered":"2023-12-20T06:03:34.555195+0000","last_unstale":"2023-12-20T06:03:35.572992+0000","last_undegraded":"2023-12-20T06:03:35.572992+0000","last_fullsized":"2023-12-20T06:03:35.572992+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:53:54.171520+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.38","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530147+0000","last_change":"2023-12-20T06:03:21.976244+0000","last_active":"2023-12-20T06:03:34.530147+0000","last_peered":"2023-12-20T06:03:34.530147+0000","last_clean":"2023-12-20T06:03:34.530147+0000","last_became_active":"2023-12-20T06:03:21.975968+0000","last_became_peered":"2023-12-20T06:03:21.975968+0000","last_unstale":"2023-12-20T06:03:34.530147+0000","last_undegraded":"2023-12-20T06:03:34.530147+0000","last_fullsized":"2023-12-20T06:03:34.530147+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:07:21.324505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.35","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535762+0000","last_change":"2023-12-20T06:03:34.553662+0000","last_active":"2023-12-20T06:03:35.535762+0000","last_peered":"2023-12-20T06:03:35.535762+0000","last_clean":"2023-12-20T06:03:35.535762+0000","last_became_active":"2023-12-20T06:03:34.553406+0000","last_became_peered":"2023-12-20T06:03:34.553406+0000","last_unstale":"2023-12-20T06:03:35.535762+0000","last_undegraded":"2023-12-20T06:03:35.535762+0000","last_fullsized":"2023-12-20T06:03:35.535762+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:22:40.617264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.37","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530821+0000","last_change":"2023-12-20T06:03:21.987598+0000","last_active":"2023-12-20T06:03:34.530821+0000","last_peered":"2023-12-20T06:03:34.530821+0000","last_clean":"2023-12-20T06:03:34.530821+0000","last_became_active":"2023-12-20T06:03:21.987291+0000","last_became_peered":"2023-12-20T06:03:21.987291+0000","last_unstale":"2023-12-20T06:03:34.530821+0000","last_undegraded":"2023-12-20T06:03:34.530821+0000","last_fullsized":"2023-12-20T06:03:34.530821+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:04.642017+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.34","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536872+0000","last_change":"2023-12-20T06:03:34.554758+0000","last_active":"2023-12-20T06:03:35.536872+0000","last_peered":"2023-12-20T06:03:35.536872+0000","last_clean":"2023-12-20T06:03:35.536872+0000","last_became_active":"2023-12-20T06:03:34.553805+0000","last_became_peered":"2023-12-20T06:03:34.553805+0000","last_unstale":"2023-12-20T06:03:35.536872+0000","last_undegraded":"2023-12-20T06:03:35.536872+0000","last_fullsized":"2023-12-20T06:03:35.536872+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:38:13.606277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.36","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.526390+0000","last_change":"2023-12-20T06:03:21.969809+0000","last_active":"2023-12-20T06:03:34.526390+0000","last_peered":"2023-12-20T06:03:34.526390+0000","last_clean":"2023-12-20T06:03:34.526390+0000","last_became_active":"2023-12-20T06:03:21.969515+0000","last_became_peered":"2023-12-20T06:03:21.969515+0000","last_unstale":"2023-12-20T06:03:34.526390+0000","last_undegraded":"2023-12-20T06:03:34.526390+0000","last_fullsized":"2023-12-20T06:03:34.526390+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:15:47.777614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.37","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572576+0000","last_change":"2023-12-20T06:03:34.559372+0000","last_active":"2023-12-20T06:03:35.572576+0000","last_peered":"2023-12-20T06:03:35.572576+0000","last_clean":"2023-12-20T06:03:35.572576+0000","last_became_active":"2023-12-20T06:03:34.559122+0000","last_became_peered":"2023-12-20T06:03:34.559122+0000","last_unstale":"2023-12-20T06:03:35.572576+0000","last_undegraded":"2023-12-20T06:03:35.572576+0000","last_fullsized":"2023-12-20T06:03:35.572576+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:23:20.670623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.35","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540624+0000","last_change":"2023-12-20T06:03:21.979757+0000","last_active":"2023-12-20T06:03:34.540624+0000","last_peered":"2023-12-20T06:03:34.540624+0000","last_clean":"2023-12-20T06:03:34.540624+0000","last_became_active":"2023-12-20T06:03:21.979456+0000","last_became_peered":"2023-12-20T06:03:21.979456+0000","last_unstale":"2023-12-20T06:03:34.540624+0000","last_undegraded":"2023-12-20T06:03:34.540624+0000","last_fullsized":"2023-12-20T06:03:34.540624+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:23.805328+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.36","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571721+0000","last_change":"2023-12-20T06:03:34.537769+0000","last_active":"2023-12-20T06:03:35.571721+0000","last_peered":"2023-12-20T06:03:35.571721+0000","last_clean":"2023-12-20T06:03:35.571721+0000","last_became_active":"2023-12-20T06:03:34.537481+0000","last_became_peered":"2023-12-20T06:03:34.537481+0000","last_unstale":"2023-12-20T06:03:35.571721+0000","last_undegraded":"2023-12-20T06:03:35.571721+0000","last_fullsized":"2023-12-20T06:03:35.571721+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:04:55.642946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.34","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540635+0000","last_change":"2023-12-20T06:03:21.982252+0000","last_active":"2023-12-20T06:03:34.540635+0000","last_peered":"2023-12-20T06:03:34.540635+0000","last_clean":"2023-12-20T06:03:34.540635+0000","last_became_active":"2023-12-20T06:03:21.981980+0000","last_became_peered":"2023-12-20T06:03:21.981980+0000","last_unstale":"2023-12-20T06:03:34.540635+0000","last_undegraded":"2023-12-20T06:03:34.540635+0000","last_fullsized":"2023-12-20T06:03:34.540635+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:13.633088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.31","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570380+0000","last_change":"2023-12-20T06:03:34.554260+0000","last_active":"2023-12-20T06:03:35.570380+0000","last_peered":"2023-12-20T06:03:35.570380+0000","last_clean":"2023-12-20T06:03:35.570380+0000","last_became_active":"2023-12-20T06:03:34.553373+0000","last_became_peered":"2023-12-20T06:03:34.553373+0000","last_unstale":"2023-12-20T06:03:35.570380+0000","last_undegraded":"2023-12-20T06:03:35.570380+0000","last_fullsized":"2023-12-20T06:03:35.570380+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:14:15.201659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.33","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.539684+0000","last_change":"2023-12-20T06:03:21.975662+0000","last_active":"2023-12-20T06:03:34.539684+0000","last_peered":"2023-12-20T06:03:34.539684+0000","last_clean":"2023-12-20T06:03:34.539684+0000","last_became_active":"2023-12-20T06:03:21.975017+0000","last_became_peered":"2023-12-20T06:03:21.975017+0000","last_unstale":"2023-12-20T06:03:34.539684+0000","last_undegraded":"2023-12-20T06:03:34.539684+0000","last_fullsized":"2023-12-20T06:03:34.539684+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:06:16.446428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.30","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572375+0000","last_change":"2023-12-20T06:03:34.532302+0000","last_active":"2023-12-20T06:03:35.572375+0000","last_peered":"2023-12-20T06:03:35.572375+0000","last_clean":"2023-12-20T06:03:35.572375+0000","last_became_active":"2023-12-20T06:03:34.532026+0000","last_became_peered":"2023-12-20T06:03:34.532026+0000","last_unstale":"2023-12-20T06:03:35.572375+0000","last_undegraded":"2023-12-20T06:03:35.572375+0000","last_fullsized":"2023-12-20T06:03:35.572375+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:12.020921+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.32","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528317+0000","last_change":"2023-12-20T06:03:21.973459+0000","last_active":"2023-12-20T06:03:34.528317+0000","last_peered":"2023-12-20T06:03:34.528317+0000","last_clean":"2023-12-20T06:03:34.528317+0000","last_became_active":"2023-12-20T06:03:21.973101+0000","last_became_peered":"2023-12-20T06:03:21.973101+0000","last_unstale":"2023-12-20T06:03:34.528317+0000","last_undegraded":"2023-12-20T06:03:34.528317+0000","last_fullsized":"2023-12-20T06:03:34.528317+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:03:34.663696+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.33","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571294+0000","last_change":"2023-12-20T06:03:34.553541+0000","last_active":"2023-12-20T06:03:35.571294+0000","last_peered":"2023-12-20T06:03:35.571294+0000","last_clean":"2023-12-20T06:03:35.571294+0000","last_became_active":"2023-12-20T06:03:34.553268+0000","last_became_peered":"2023-12-20T06:03:34.553268+0000","last_unstale":"2023-12-20T06:03:35.571294+0000","last_undegraded":"2023-12-20T06:03:35.571294+0000","last_fullsized":"2023-12-20T06:03:35.571294+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:27:22.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.31","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537295+0000","last_change":"2023-12-20T06:03:21.982673+0000","last_active":"2023-12-20T06:03:34.537295+0000","last_peered":"2023-12-20T06:03:34.537295+0000","last_clean":"2023-12-20T06:03:34.537295+0000","last_became_active":"2023-12-20T06:03:21.982010+0000","last_became_peered":"2023-12-20T06:03:21.982010+0000","last_unstale":"2023-12-20T06:03:34.537295+0000","last_undegraded":"2023-12-20T06:03:34.537295+0000","last_fullsized":"2023-12-20T06:03:34.537295+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:32:19.664417+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.32","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572826+0000","last_change":"2023-12-20T06:03:34.544343+0000","last_active":"2023-12-20T06:03:35.572826+0000","last_peered":"2023-12-20T06:03:35.572826+0000","last_clean":"2023-12-20T06:03:35.572826+0000","last_became_active":"2023-12-20T06:03:34.544066+0000","last_became_peered":"2023-12-20T06:03:34.544066+0000","last_unstale":"2023-12-20T06:03:35.572826+0000","last_undegraded":"2023-12-20T06:03:35.572826+0000","last_fullsized":"2023-12-20T06:03:35.572826+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:49:46.188372+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.30","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537888+0000","last_change":"2023-12-20T06:03:21.977347+0000","last_active":"2023-12-20T06:03:34.537888+0000","last_peered":"2023-12-20T06:03:34.537888+0000","last_clean":"2023-12-20T06:03:34.537888+0000","last_became_active":"2023-12-20T06:03:21.977040+0000","last_became_peered":"2023-12-20T06:03:21.977040+0000","last_unstale":"2023-12-20T06:03:34.537888+0000","last_undegraded":"2023-12-20T06:03:34.537888+0000","last_fullsized":"2023-12-20T06:03:34.537888+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.2d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.537333+0000","last_change":"2023-12-20T06:03:34.554193+0000","last_active":"2023-12-20T06:03:35.537333+0000","last_peered":"2023-12-20T06:03:35.537333+0000","last_clean":"2023-12-20T06:03:35.537333+0000","last_became_active":"2023-12-20T06:03:34.553937+0000","last_became_peered":"2023-12-20T06:03:34.553937+0000","last_unstale":"2023-12-20T06:03:35.537333+0000","last_undegraded":"2023-12-20T06:03:35.537333+0000","last_fullsized":"2023-12-20T06:03:35.537333+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:39:44.183895+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2f","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529300+0000","last_change":"2023-12-20T06:03:21.973052+0000","last_active":"2023-12-20T06:03:34.529300+0000","last_peered":"2023-12-20T06:03:34.529300+0000","last_clean":"2023-12-20T06:03:34.529300+0000","last_became_active":"2023-12-20T06:03:21.972760+0000","last_became_peered":"2023-12-20T06:03:21.972760+0000","last_unstale":"2023-12-20T06:03:34.529300+0000","last_undegraded":"2023-12-20T06:03:34.529300+0000","last_fullsized":"2023-12-20T06:03:34.529300+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:16:41.285330+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.2c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574145+0000","last_change":"2023-12-20T06:03:34.549623+0000","last_active":"2023-12-20T06:03:35.574145+0000","last_peered":"2023-12-20T06:03:35.574145+0000","last_clean":"2023-12-20T06:03:35.574145+0000","last_became_active":"2023-12-20T06:03:34.549164+0000","last_became_peered":"2023-12-20T06:03:34.549164+0000","last_unstale":"2023-12-20T06:03:35.574145+0000","last_undegraded":"2023-12-20T06:03:35.574145+0000","last_fullsized":"2023-12-20T06:03:35.574145+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:41:30.995579+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2e","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.526390+0000","last_change":"2023-12-20T06:03:21.991211+0000","last_active":"2023-12-20T06:03:34.526390+0000","last_peered":"2023-12-20T06:03:34.526390+0000","last_clean":"2023-12-20T06:03:34.526390+0000","last_became_active":"2023-12-20T06:03:21.990873+0000","last_became_peered":"2023-12-20T06:03:21.990873+0000","last_unstale":"2023-12-20T06:03:34.526390+0000","last_undegraded":"2023-12-20T06:03:34.526390+0000","last_fullsized":"2023-12-20T06:03:34.526390+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:31:08.457333+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.2f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.537415+0000","last_change":"2023-12-20T06:03:34.546022+0000","last_active":"2023-12-20T06:03:35.537415+0000","last_peered":"2023-12-20T06:03:35.537415+0000","last_clean":"2023-12-20T06:03:35.537415+0000","last_became_active":"2023-12-20T06:03:34.545718+0000","last_became_peered":"2023-12-20T06:03:34.545718+0000","last_unstale":"2023-12-20T06:03:35.537415+0000","last_undegraded":"2023-12-20T06:03:35.537415+0000","last_fullsized":"2023-12-20T06:03:35.537415+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:56:54.852713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2d","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534615+0000","last_change":"2023-12-20T06:03:21.982366+0000","last_active":"2023-12-20T06:03:34.534615+0000","last_peered":"2023-12-20T06:03:34.534615+0000","last_clean":"2023-12-20T06:03:34.534615+0000","last_became_active":"2023-12-20T06:03:21.978098+0000","last_became_peered":"2023-12-20T06:03:21.978098+0000","last_unstale":"2023-12-20T06:03:34.534615+0000","last_undegraded":"2023-12-20T06:03:34.534615+0000","last_fullsized":"2023-12-20T06:03:34.534615+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.2e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575201+0000","last_change":"2023-12-20T06:03:34.553869+0000","last_active":"2023-12-20T06:03:35.575201+0000","last_peered":"2023-12-20T06:03:35.575201+0000","last_clean":"2023-12-20T06:03:35.575201+0000","last_became_active":"2023-12-20T06:03:34.553639+0000","last_became_peered":"2023-12-20T06:03:34.553639+0000","last_unstale":"2023-12-20T06:03:35.575201+0000","last_undegraded":"2023-12-20T06:03:35.575201+0000","last_fullsized":"2023-12-20T06:03:35.575201+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:34:13.655323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2c","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.525145+0000","last_change":"2023-12-20T06:03:21.967196+0000","last_active":"2023-12-20T06:03:34.525145+0000","last_peered":"2023-12-20T06:03:34.525145+0000","last_clean":"2023-12-20T06:03:34.525145+0000","last_became_active":"2023-12-20T06:03:21.966332+0000","last_became_peered":"2023-12-20T06:03:21.966332+0000","last_unstale":"2023-12-20T06:03:34.525145+0000","last_undegraded":"2023-12-20T06:03:34.525145+0000","last_fullsized":"2023-12-20T06:03:34.525145+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:51:08.151013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.29","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536617+0000","last_change":"2023-12-20T06:03:34.543964+0000","last_active":"2023-12-20T06:03:35.536617+0000","last_peered":"2023-12-20T06:03:35.536617+0000","last_clean":"2023-12-20T06:03:35.536617+0000","last_became_active":"2023-12-20T06:03:34.543592+0000","last_became_peered":"2023-12-20T06:03:34.543592+0000","last_unstale":"2023-12-20T06:03:35.536617+0000","last_undegraded":"2023-12-20T06:03:35.536617+0000","last_fullsized":"2023-12-20T06:03:35.536617+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:02.206635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.2b","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537812+0000","last_change":"2023-12-20T06:03:21.977328+0000","last_active":"2023-12-20T06:03:34.537812+0000","last_peered":"2023-12-20T06:03:34.537812+0000","last_clean":"2023-12-20T06:03:34.537812+0000","last_became_active":"2023-12-20T06:03:21.976722+0000","last_became_peered":"2023-12-20T06:03:21.976722+0000","last_unstale":"2023-12-20T06:03:34.537812+0000","last_undegraded":"2023-12-20T06:03:34.537812+0000","last_fullsized":"2023-12-20T06:03:34.537812+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:35:37.344273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.28","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572450+0000","last_change":"2023-12-20T06:03:34.559075+0000","last_active":"2023-12-20T06:03:35.572450+0000","last_peered":"2023-12-20T06:03:35.572450+0000","last_clean":"2023-12-20T06:03:35.572450+0000","last_became_active":"2023-12-20T06:03:34.556828+0000","last_became_peered":"2023-12-20T06:03:34.556828+0000","last_unstale":"2023-12-20T06:03:35.572450+0000","last_undegraded":"2023-12-20T06:03:35.572450+0000","last_fullsized":"2023-12-20T06:03:35.572450+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:46:47.498001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.2a","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537560+0000","last_change":"2023-12-20T06:03:21.974137+0000","last_active":"2023-12-20T06:03:34.537560+0000","last_peered":"2023-12-20T06:03:34.537560+0000","last_clean":"2023-12-20T06:03:34.537560+0000","last_became_active":"2023-12-20T06:03:21.973889+0000","last_became_peered":"2023-12-20T06:03:21.973889+0000","last_unstale":"2023-12-20T06:03:34.537560+0000","last_undegraded":"2023-12-20T06:03:34.537560+0000","last_fullsized":"2023-12-20T06:03:34.537560+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:08:46.618627+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536842+0000","last_change":"2023-12-20T06:03:34.541415+0000","last_active":"2023-12-20T06:03:35.536842+0000","last_peered":"2023-12-20T06:03:35.536842+0000","last_clean":"2023-12-20T06:03:35.536842+0000","last_became_active":"2023-12-20T06:03:34.541154+0000","last_became_peered":"2023-12-20T06:03:34.541154+0000","last_unstale":"2023-12-20T06:03:35.536842+0000","last_undegraded":"2023-12-20T06:03:35.536842+0000","last_fullsized":"2023-12-20T06:03:35.536842+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:54:48.032421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7],"acting":[4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.29","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.538161+0000","last_change":"2023-12-20T06:03:21.967911+0000","last_active":"2023-12-20T06:03:34.538161+0000","last_peered":"2023-12-20T06:03:34.538161+0000","last_clean":"2023-12-20T06:03:34.538161+0000","last_became_active":"2023-12-20T06:03:21.967093+0000","last_became_peered":"2023-12-20T06:03:21.967093+0000","last_unstale":"2023-12-20T06:03:34.538161+0000","last_undegraded":"2023-12-20T06:03:34.538161+0000","last_fullsized":"2023-12-20T06:03:34.538161+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:43:05.571007+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.2a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570788+0000","last_change":"2023-12-20T06:03:34.539342+0000","last_active":"2023-12-20T06:03:35.570788+0000","last_peered":"2023-12-20T06:03:35.570788+0000","last_clean":"2023-12-20T06:03:35.570788+0000","last_became_active":"2023-12-20T06:03:34.539047+0000","last_became_peered":"2023-12-20T06:03:34.539047+0000","last_unstale":"2023-12-20T06:03:35.570788+0000","last_undegraded":"2023-12-20T06:03:35.570788+0000","last_fullsized":"2023-12-20T06:03:35.570788+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:04:45.298268+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.28","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534526+0000","last_change":"2023-12-20T06:03:21.974077+0000","last_active":"2023-12-20T06:03:34.534526+0000","last_peered":"2023-12-20T06:03:34.534526+0000","last_clean":"2023-12-20T06:03:34.534526+0000","last_became_active":"2023-12-20T06:03:21.973810+0000","last_became_peered":"2023-12-20T06:03:21.973810+0000","last_unstale":"2023-12-20T06:03:34.534526+0000","last_undegraded":"2023-12-20T06:03:34.534526+0000","last_fullsized":"2023-12-20T06:03:34.534526+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:00.841811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.25","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535821+0000","last_change":"2023-12-20T06:03:34.553105+0000","last_active":"2023-12-20T06:03:35.535821+0000","last_peered":"2023-12-20T06:03:35.535821+0000","last_clean":"2023-12-20T06:03:35.535821+0000","last_became_active":"2023-12-20T06:03:34.552811+0000","last_became_peered":"2023-12-20T06:03:34.552811+0000","last_unstale":"2023-12-20T06:03:35.535821+0000","last_undegraded":"2023-12-20T06:03:35.535821+0000","last_fullsized":"2023-12-20T06:03:35.535821+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:53:17.378650+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.27","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.541237+0000","last_change":"2023-12-20T06:03:21.976310+0000","last_active":"2023-12-20T06:03:34.541237+0000","last_peered":"2023-12-20T06:03:34.541237+0000","last_clean":"2023-12-20T06:03:34.541237+0000","last_became_active":"2023-12-20T06:03:21.976042+0000","last_became_peered":"2023-12-20T06:03:21.976042+0000","last_unstale":"2023-12-20T06:03:34.541237+0000","last_undegraded":"2023-12-20T06:03:34.541237+0000","last_fullsized":"2023-12-20T06:03:34.541237+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:46.204136+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.24","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573869+0000","last_change":"2023-12-20T06:03:34.559246+0000","last_active":"2023-12-20T06:03:35.573869+0000","last_peered":"2023-12-20T06:03:35.573869+0000","last_clean":"2023-12-20T06:03:35.573869+0000","last_became_active":"2023-12-20T06:03:34.558427+0000","last_became_peered":"2023-12-20T06:03:34.558427+0000","last_unstale":"2023-12-20T06:03:35.573869+0000","last_undegraded":"2023-12-20T06:03:35.573869+0000","last_fullsized":"2023-12-20T06:03:35.573869+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:29:03.073420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.26","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531003+0000","last_change":"2023-12-20T06:03:21.981857+0000","last_active":"2023-12-20T06:03:34.531003+0000","last_peered":"2023-12-20T06:03:34.531003+0000","last_clean":"2023-12-20T06:03:34.531003+0000","last_became_active":"2023-12-20T06:03:21.981595+0000","last_became_peered":"2023-12-20T06:03:21.981595+0000","last_unstale":"2023-12-20T06:03:34.531003+0000","last_undegraded":"2023-12-20T06:03:34.531003+0000","last_fullsized":"2023-12-20T06:03:34.531003+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:39:40.018104+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.27","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575357+0000","last_change":"2023-12-20T06:03:34.546245+0000","last_active":"2023-12-20T06:03:35.575357+0000","last_peered":"2023-12-20T06:03:35.575357+0000","last_clean":"2023-12-20T06:03:35.575357+0000","last_became_active":"2023-12-20T06:03:34.545741+0000","last_became_peered":"2023-12-20T06:03:34.545741+0000","last_unstale":"2023-12-20T06:03:35.575357+0000","last_undegraded":"2023-12-20T06:03:35.575357+0000","last_fullsized":"2023-12-20T06:03:35.575357+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:15:01.352569+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.25","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540773+0000","last_change":"2023-12-20T06:03:21.979813+0000","last_active":"2023-12-20T06:03:34.540773+0000","last_peered":"2023-12-20T06:03:34.540773+0000","last_clean":"2023-12-20T06:03:34.540773+0000","last_became_active":"2023-12-20T06:03:21.979541+0000","last_became_peered":"2023-12-20T06:03:21.979541+0000","last_unstale":"2023-12-20T06:03:34.540773+0000","last_undegraded":"2023-12-20T06:03:34.540773+0000","last_fullsized":"2023-12-20T06:03:34.540773+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:29.736662+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.26","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571852+0000","last_change":"2023-12-20T06:03:34.553048+0000","last_active":"2023-12-20T06:03:35.571852+0000","last_peered":"2023-12-20T06:03:35.571852+0000","last_clean":"2023-12-20T06:03:35.571852+0000","last_became_active":"2023-12-20T06:03:34.552750+0000","last_became_peered":"2023-12-20T06:03:34.552750+0000","last_unstale":"2023-12-20T06:03:35.571852+0000","last_undegraded":"2023-12-20T06:03:35.571852+0000","last_fullsized":"2023-12-20T06:03:35.571852+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:54:48.032421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.24","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.525225+0000","last_change":"2023-12-20T06:03:21.985481+0000","last_active":"2023-12-20T06:03:34.525225+0000","last_peered":"2023-12-20T06:03:34.525225+0000","last_clean":"2023-12-20T06:03:34.525225+0000","last_became_active":"2023-12-20T06:03:21.985202+0000","last_became_peered":"2023-12-20T06:03:21.985202+0000","last_unstale":"2023-12-20T06:03:34.525225+0000","last_undegraded":"2023-12-20T06:03:34.525225+0000","last_fullsized":"2023-12-20T06:03:34.525225+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:57:07.294252+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.21","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572970+0000","last_change":"2023-12-20T06:03:34.563627+0000","last_active":"2023-12-20T06:03:35.572970+0000","last_peered":"2023-12-20T06:03:35.572970+0000","last_clean":"2023-12-20T06:03:35.572970+0000","last_became_active":"2023-12-20T06:03:34.563345+0000","last_became_peered":"2023-12-20T06:03:34.563345+0000","last_unstale":"2023-12-20T06:03:35.572970+0000","last_undegraded":"2023-12-20T06:03:35.572970+0000","last_fullsized":"2023-12-20T06:03:35.572970+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:16:53.854983+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.23","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528253+0000","last_change":"2023-12-20T06:03:21.989526+0000","last_active":"2023-12-20T06:03:34.528253+0000","last_peered":"2023-12-20T06:03:34.528253+0000","last_clean":"2023-12-20T06:03:34.528253+0000","last_became_active":"2023-12-20T06:03:21.989062+0000","last_became_peered":"2023-12-20T06:03:21.989062+0000","last_unstale":"2023-12-20T06:03:34.528253+0000","last_undegraded":"2023-12-20T06:03:34.528253+0000","last_fullsized":"2023-12-20T06:03:34.528253+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:17.456340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.20","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572457+0000","last_change":"2023-12-20T06:03:34.553239+0000","last_active":"2023-12-20T06:03:35.572457+0000","last_peered":"2023-12-20T06:03:35.572457+0000","last_clean":"2023-12-20T06:03:35.572457+0000","last_became_active":"2023-12-20T06:03:34.552951+0000","last_became_peered":"2023-12-20T06:03:34.552951+0000","last_unstale":"2023-12-20T06:03:35.572457+0000","last_undegraded":"2023-12-20T06:03:35.572457+0000","last_fullsized":"2023-12-20T06:03:35.572457+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:58:00.185736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.22","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.538469+0000","last_change":"2023-12-20T06:03:21.978099+0000","last_active":"2023-12-20T06:03:34.538469+0000","last_peered":"2023-12-20T06:03:34.538469+0000","last_clean":"2023-12-20T06:03:34.538469+0000","last_became_active":"2023-12-20T06:03:21.977814+0000","last_became_peered":"2023-12-20T06:03:21.977814+0000","last_unstale":"2023-12-20T06:03:34.538469+0000","last_undegraded":"2023-12-20T06:03:34.538469+0000","last_fullsized":"2023-12-20T06:03:34.538469+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:14:17.140851+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.23","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535420+0000","last_change":"2023-12-20T06:03:34.550163+0000","last_active":"2023-12-20T06:03:35.535420+0000","last_peered":"2023-12-20T06:03:35.535420+0000","last_clean":"2023-12-20T06:03:35.535420+0000","last_became_active":"2023-12-20T06:03:34.549905+0000","last_became_peered":"2023-12-20T06:03:34.549905+0000","last_unstale":"2023-12-20T06:03:35.535420+0000","last_undegraded":"2023-12-20T06:03:35.535420+0000","last_fullsized":"2023-12-20T06:03:35.535420+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:29:28.110798+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.21","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.533764+0000","last_change":"2023-12-20T06:03:21.984020+0000","last_active":"2023-12-20T06:03:34.533764+0000","last_peered":"2023-12-20T06:03:34.533764+0000","last_clean":"2023-12-20T06:03:34.533764+0000","last_became_active":"2023-12-20T06:03:21.983429+0000","last_became_peered":"2023-12-20T06:03:21.983429+0000","last_unstale":"2023-12-20T06:03:34.533764+0000","last_undegraded":"2023-12-20T06:03:34.533764+0000","last_fullsized":"2023-12-20T06:03:34.533764+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.22","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535053+0000","last_change":"2023-12-20T06:03:34.546965+0000","last_active":"2023-12-20T06:03:35.535053+0000","last_peered":"2023-12-20T06:03:35.535053+0000","last_clean":"2023-12-20T06:03:35.535053+0000","last_became_active":"2023-12-20T06:03:34.546614+0000","last_became_peered":"2023-12-20T06:03:34.546614+0000","last_unstale":"2023-12-20T06:03:35.535053+0000","last_undegraded":"2023-12-20T06:03:35.535053+0000","last_fullsized":"2023-12-20T06:03:35.535053+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:23:20.670623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.20","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531376+0000","last_change":"2023-12-20T06:03:21.985095+0000","last_active":"2023-12-20T06:03:34.531376+0000","last_peered":"2023-12-20T06:03:34.531376+0000","last_clean":"2023-12-20T06:03:34.531376+0000","last_became_active":"2023-12-20T06:03:21.984659+0000","last_became_peered":"2023-12-20T06:03:21.984659+0000","last_unstale":"2023-12-20T06:03:34.531376+0000","last_undegraded":"2023-12-20T06:03:34.531376+0000","last_fullsized":"2023-12-20T06:03:34.531376+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:24:17.425367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535626+0000","last_change":"2023-12-20T06:03:34.550290+0000","last_active":"2023-12-20T06:03:35.535626+0000","last_peered":"2023-12-20T06:03:35.535626+0000","last_clean":"2023-12-20T06:03:35.535626+0000","last_became_active":"2023-12-20T06:03:34.549789+0000","last_became_peered":"2023-12-20T06:03:34.549789+0000","last_unstale":"2023-12-20T06:03:35.535626+0000","last_undegraded":"2023-12-20T06:03:35.535626+0000","last_fullsized":"2023-12-20T06:03:35.535626+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:53:04.744558+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1d","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574330+0000","last_change":"2023-12-20T06:03:34.125243+0000","last_active":"2023-12-20T06:03:35.574330+0000","last_peered":"2023-12-20T06:03:35.574330+0000","last_clean":"2023-12-20T06:03:35.574330+0000","last_became_active":"2023-12-20T06:03:34.124962+0000","last_became_peered":"2023-12-20T06:03:34.124962+0000","last_unstale":"2023-12-20T06:03:35.574330+0000","last_undegraded":"2023-12-20T06:03:35.574330+0000","last_fullsized":"2023-12-20T06:03:35.574330+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:46:09.465814+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1e","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.532133+0000","last_change":"2023-12-20T06:03:21.985587+0000","last_active":"2023-12-20T06:03:34.532133+0000","last_peered":"2023-12-20T06:03:34.532133+0000","last_clean":"2023-12-20T06:03:34.532133+0000","last_became_active":"2023-12-20T06:03:21.985231+0000","last_became_peered":"2023-12-20T06:03:21.985231+0000","last_unstale":"2023-12-20T06:03:34.532133+0000","last_undegraded":"2023-12-20T06:03:34.532133+0000","last_fullsized":"2023-12-20T06:03:34.532133+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:44:13.389812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1f","version":"229'4","reported_seq":19,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531211+0000","last_change":"2023-12-20T06:03:21.558246+0000","last_active":"2023-12-20T06:03:34.531211+0000","last_peered":"2023-12-20T06:03:34.531211+0000","last_clean":"2023-12-20T06:03:34.531211+0000","last_became_active":"2023-12-20T06:03:21.557732+0000","last_became_peered":"2023-12-20T06:03:21.557732+0000","last_unstale":"2023-12-20T06:03:34.531211+0000","last_undegraded":"2023-12-20T06:03:34.531211+0000","last_fullsized":"2023-12-20T06:03:34.531211+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:01:07.326834+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.1d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.537304+0000","last_change":"2023-12-20T06:03:34.551321+0000","last_active":"2023-12-20T06:03:35.537304+0000","last_peered":"2023-12-20T06:03:35.537304+0000","last_clean":"2023-12-20T06:03:35.537304+0000","last_became_active":"2023-12-20T06:03:34.550888+0000","last_became_peered":"2023-12-20T06:03:34.550888+0000","last_unstale":"2023-12-20T06:03:35.537304+0000","last_undegraded":"2023-12-20T06:03:35.537304+0000","last_fullsized":"2023-12-20T06:03:35.537304+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:34:13.655323+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1c","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574578+0000","last_change":"2023-12-20T06:03:34.138421+0000","last_active":"2023-12-20T06:03:35.574578+0000","last_peered":"2023-12-20T06:03:35.574578+0000","last_clean":"2023-12-20T06:03:35.574578+0000","last_became_active":"2023-12-20T06:03:34.138122+0000","last_became_peered":"2023-12-20T06:03:34.138122+0000","last_unstale":"2023-12-20T06:03:35.574578+0000","last_undegraded":"2023-12-20T06:03:35.574578+0000","last_fullsized":"2023-12-20T06:03:35.574578+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:13:20.882471+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1f","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531753+0000","last_change":"2023-12-20T06:03:21.975027+0000","last_active":"2023-12-20T06:03:34.531753+0000","last_peered":"2023-12-20T06:03:34.531753+0000","last_clean":"2023-12-20T06:03:34.531753+0000","last_became_active":"2023-12-20T06:03:21.974227+0000","last_became_peered":"2023-12-20T06:03:21.974227+0000","last_unstale":"2023-12-20T06:03:34.531753+0000","last_undegraded":"2023-12-20T06:03:34.531753+0000","last_fullsized":"2023-12-20T06:03:34.531753+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:38:46.179532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.1e","version":"229'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.543031+0000","last_change":"2023-12-20T06:03:21.559869+0000","last_active":"2023-12-20T06:03:34.543031+0000","last_peered":"2023-12-20T06:03:34.543031+0000","last_clean":"2023-12-20T06:03:34.543031+0000","last_became_active":"2023-12-20T06:03:21.559555+0000","last_became_peered":"2023-12-20T06:03:21.559555+0000","last_unstale":"2023-12-20T06:03:34.543031+0000","last_undegraded":"2023-12-20T06:03:34.543031+0000","last_fullsized":"2023-12-20T06:03:34.543031+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:18:15.926622+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.1f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574334+0000","last_change":"2023-12-20T06:03:34.541169+0000","last_active":"2023-12-20T06:03:35.574334+0000","last_peered":"2023-12-20T06:03:35.574334+0000","last_clean":"2023-12-20T06:03:35.574334+0000","last_became_active":"2023-12-20T06:03:34.540560+0000","last_became_peered":"2023-12-20T06:03:34.540560+0000","last_unstale":"2023-12-20T06:03:35.574334+0000","last_undegraded":"2023-12-20T06:03:35.574334+0000","last_fullsized":"2023-12-20T06:03:35.574334+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T18:01:49.083465+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.1e","version":"233'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571857+0000","last_change":"2023-12-20T06:03:34.133947+0000","last_active":"2023-12-20T06:03:35.571857+0000","last_peered":"2023-12-20T06:03:35.571857+0000","last_clean":"2023-12-20T06:03:35.571857+0000","last_became_active":"2023-12-20T06:03:34.133679+0000","last_became_peered":"2023-12-20T06:03:34.133679+0000","last_unstale":"2023-12-20T06:03:35.571857+0000","last_undegraded":"2023-12-20T06:03:35.571857+0000","last_fullsized":"2023-12-20T06:03:35.571857+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:51:19.699199+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1d","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528779+0000","last_change":"2023-12-20T06:03:21.988797+0000","last_active":"2023-12-20T06:03:34.528779+0000","last_peered":"2023-12-20T06:03:34.528779+0000","last_clean":"2023-12-20T06:03:34.528779+0000","last_became_active":"2023-12-20T06:03:21.988418+0000","last_became_peered":"2023-12-20T06:03:21.988418+0000","last_unstale":"2023-12-20T06:03:34.528779+0000","last_undegraded":"2023-12-20T06:03:34.528779+0000","last_fullsized":"2023-12-20T06:03:34.528779+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:21:31.283475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.1c","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540729+0000","last_change":"2023-12-20T06:03:21.566883+0000","last_active":"2023-12-20T06:03:34.540729+0000","last_peered":"2023-12-20T06:03:34.540729+0000","last_clean":"2023-12-20T06:03:34.540729+0000","last_became_active":"2023-12-20T06:03:21.566612+0000","last_became_peered":"2023-12-20T06:03:21.566612+0000","last_unstale":"2023-12-20T06:03:34.540729+0000","last_undegraded":"2023-12-20T06:03:34.540729+0000","last_fullsized":"2023-12-20T06:03:34.540729+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:03:33.646493+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.18","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572648+0000","last_change":"2023-12-20T06:03:34.542361+0000","last_active":"2023-12-20T06:03:35.572648+0000","last_peered":"2023-12-20T06:03:35.572648+0000","last_clean":"2023-12-20T06:03:35.572648+0000","last_became_active":"2023-12-20T06:03:34.541901+0000","last_became_peered":"2023-12-20T06:03:34.541901+0000","last_unstale":"2023-12-20T06:03:35.572648+0000","last_undegraded":"2023-12-20T06:03:35.572648+0000","last_fullsized":"2023-12-20T06:03:35.572648+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:18:24.755969+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.19","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570779+0000","last_change":"2023-12-20T06:03:34.125419+0000","last_active":"2023-12-20T06:03:35.570779+0000","last_peered":"2023-12-20T06:03:35.570779+0000","last_clean":"2023-12-20T06:03:35.570779+0000","last_became_active":"2023-12-20T06:03:34.125066+0000","last_became_peered":"2023-12-20T06:03:34.125066+0000","last_unstale":"2023-12-20T06:03:35.570779+0000","last_undegraded":"2023-12-20T06:03:35.570779+0000","last_fullsized":"2023-12-20T06:03:35.570779+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:15:59.325800+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.1a","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.538555+0000","last_change":"2023-12-20T06:03:21.976016+0000","last_active":"2023-12-20T06:03:34.538555+0000","last_peered":"2023-12-20T06:03:34.538555+0000","last_clean":"2023-12-20T06:03:34.538555+0000","last_became_active":"2023-12-20T06:03:21.975686+0000","last_became_peered":"2023-12-20T06:03:21.975686+0000","last_unstale":"2023-12-20T06:03:34.538555+0000","last_undegraded":"2023-12-20T06:03:34.538555+0000","last_fullsized":"2023-12-20T06:03:34.538555+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:40:22.788736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1b","version":"229'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530275+0000","last_change":"2023-12-20T06:03:21.567258+0000","last_active":"2023-12-20T06:03:34.530275+0000","last_peered":"2023-12-20T06:03:34.530275+0000","last_clean":"2023-12-20T06:03:34.530275+0000","last_became_active":"2023-12-20T06:03:21.565535+0000","last_became_peered":"2023-12-20T06:03:21.565535+0000","last_unstale":"2023-12-20T06:03:34.530275+0000","last_undegraded":"2023-12-20T06:03:34.530275+0000","last_fullsized":"2023-12-20T06:03:34.530275+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:27:30.578745+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.19","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572367+0000","last_change":"2023-12-20T06:03:34.548571+0000","last_active":"2023-12-20T06:03:35.572367+0000","last_peered":"2023-12-20T06:03:35.572367+0000","last_clean":"2023-12-20T06:03:35.572367+0000","last_became_active":"2023-12-20T06:03:34.546639+0000","last_became_peered":"2023-12-20T06:03:34.546639+0000","last_unstale":"2023-12-20T06:03:35.572367+0000","last_undegraded":"2023-12-20T06:03:35.572367+0000","last_fullsized":"2023-12-20T06:03:35.572367+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:27:22.562148+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.18","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572277+0000","last_change":"2023-12-20T06:03:34.141640+0000","last_active":"2023-12-20T06:03:35.572277+0000","last_peered":"2023-12-20T06:03:35.572277+0000","last_clean":"2023-12-20T06:03:35.572277+0000","last_became_active":"2023-12-20T06:03:34.141373+0000","last_became_peered":"2023-12-20T06:03:34.141373+0000","last_unstale":"2023-12-20T06:03:35.572277+0000","last_undegraded":"2023-12-20T06:03:35.572277+0000","last_fullsized":"2023-12-20T06:03:35.572277+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:37:05.537840+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1b","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537903+0000","last_change":"2023-12-20T06:03:21.988028+0000","last_active":"2023-12-20T06:03:34.537903+0000","last_peered":"2023-12-20T06:03:34.537903+0000","last_clean":"2023-12-20T06:03:34.537903+0000","last_became_active":"2023-12-20T06:03:21.986936+0000","last_became_peered":"2023-12-20T06:03:21.986936+0000","last_unstale":"2023-12-20T06:03:34.537903+0000","last_undegraded":"2023-12-20T06:03:34.537903+0000","last_fullsized":"2023-12-20T06:03:34.537903+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:41.038169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.1a","version":"229'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529334+0000","last_change":"2023-12-20T06:03:21.564120+0000","last_active":"2023-12-20T06:03:34.529334+0000","last_peered":"2023-12-20T06:03:34.529334+0000","last_clean":"2023-12-20T06:03:34.529334+0000","last_became_active":"2023-12-20T06:03:21.563826+0000","last_became_peered":"2023-12-20T06:03:21.563826+0000","last_unstale":"2023-12-20T06:03:34.529334+0000","last_undegraded":"2023-12-20T06:03:34.529334+0000","last_fullsized":"2023-12-20T06:03:34.529334+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535166+0000","last_change":"2023-12-20T06:03:34.550742+0000","last_active":"2023-12-20T06:03:35.535166+0000","last_peered":"2023-12-20T06:03:35.535166+0000","last_clean":"2023-12-20T06:03:35.535166+0000","last_became_active":"2023-12-20T06:03:34.550491+0000","last_became_peered":"2023-12-20T06:03:34.550491+0000","last_unstale":"2023-12-20T06:03:35.535166+0000","last_undegraded":"2023-12-20T06:03:35.535166+0000","last_fullsized":"2023-12-20T06:03:35.535166+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:03:18.220186+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.1b","version":"233'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573236+0000","last_change":"2023-12-20T06:03:34.142117+0000","last_active":"2023-12-20T06:03:35.573236+0000","last_peered":"2023-12-20T06:03:35.573236+0000","last_clean":"2023-12-20T06:03:35.573236+0000","last_became_active":"2023-12-20T06:03:34.141765+0000","last_became_peered":"2023-12-20T06:03:34.141765+0000","last_unstale":"2023-12-20T06:03:35.573236+0000","last_undegraded":"2023-12-20T06:03:35.573236+0000","last_fullsized":"2023-12-20T06:03:35.573236+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:57.752322+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.18","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.523985+0000","last_change":"2023-12-20T06:03:21.975377+0000","last_active":"2023-12-20T06:03:34.523985+0000","last_peered":"2023-12-20T06:03:34.523985+0000","last_clean":"2023-12-20T06:03:34.523985+0000","last_became_active":"2023-12-20T06:03:21.974995+0000","last_became_peered":"2023-12-20T06:03:21.974995+0000","last_unstale":"2023-12-20T06:03:34.523985+0000","last_undegraded":"2023-12-20T06:03:34.523985+0000","last_fullsized":"2023-12-20T06:03:34.523985+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:46.826847+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.19","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528457+0000","last_change":"2023-12-20T06:03:21.565400+0000","last_active":"2023-12-20T06:03:34.528457+0000","last_peered":"2023-12-20T06:03:34.528457+0000","last_clean":"2023-12-20T06:03:34.528457+0000","last_became_active":"2023-12-20T06:03:21.565079+0000","last_became_peered":"2023-12-20T06:03:21.565079+0000","last_unstale":"2023-12-20T06:03:34.528457+0000","last_undegraded":"2023-12-20T06:03:34.528457+0000","last_fullsized":"2023-12-20T06:03:34.528457+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:22.788125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4],"acting":[5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571276+0000","last_change":"2023-12-20T06:03:34.552533+0000","last_active":"2023-12-20T06:03:35.571276+0000","last_peered":"2023-12-20T06:03:35.571276+0000","last_clean":"2023-12-20T06:03:35.571276+0000","last_became_active":"2023-12-20T06:03:34.552056+0000","last_became_peered":"2023-12-20T06:03:34.552056+0000","last_unstale":"2023-12-20T06:03:35.571276+0000","last_undegraded":"2023-12-20T06:03:35.571276+0000","last_fullsized":"2023-12-20T06:03:35.571276+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:19.382530+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.1a","version":"233'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573045+0000","last_change":"2023-12-20T06:03:34.139657+0000","last_active":"2023-12-20T06:03:35.573045+0000","last_peered":"2023-12-20T06:03:35.573045+0000","last_clean":"2023-12-20T06:03:35.573045+0000","last_became_active":"2023-12-20T06:03:34.139304+0000","last_became_peered":"2023-12-20T06:03:34.139304+0000","last_unstale":"2023-12-20T06:03:35.573045+0000","last_undegraded":"2023-12-20T06:03:35.573045+0000","last_fullsized":"2023-12-20T06:03:35.573045+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:02:31.037953+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.19","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.536840+0000","last_change":"2023-12-20T06:03:21.977819+0000","last_active":"2023-12-20T06:03:34.536840+0000","last_peered":"2023-12-20T06:03:34.536840+0000","last_clean":"2023-12-20T06:03:34.536840+0000","last_became_active":"2023-12-20T06:03:21.977540+0000","last_became_peered":"2023-12-20T06:03:21.977540+0000","last_unstale":"2023-12-20T06:03:34.536840+0000","last_undegraded":"2023-12-20T06:03:34.536840+0000","last_fullsized":"2023-12-20T06:03:34.536840+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:44:13.389812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.18","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.524159+0000","last_change":"2023-12-20T06:03:21.564947+0000","last_active":"2023-12-20T06:03:34.524159+0000","last_peered":"2023-12-20T06:03:34.524159+0000","last_clean":"2023-12-20T06:03:34.524159+0000","last_became_active":"2023-12-20T06:03:21.564634+0000","last_became_peered":"2023-12-20T06:03:21.564634+0000","last_unstale":"2023-12-20T06:03:34.524159+0000","last_undegraded":"2023-12-20T06:03:34.524159+0000","last_fullsized":"2023-12-20T06:03:34.524159+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:05.795673+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.14","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574239+0000","last_change":"2023-12-20T06:03:34.549447+0000","last_active":"2023-12-20T06:03:35.574239+0000","last_peered":"2023-12-20T06:03:35.574239+0000","last_clean":"2023-12-20T06:03:35.574239+0000","last_became_active":"2023-12-20T06:03:34.547913+0000","last_became_peered":"2023-12-20T06:03:34.547913+0000","last_unstale":"2023-12-20T06:03:35.574239+0000","last_undegraded":"2023-12-20T06:03:35.574239+0000","last_fullsized":"2023-12-20T06:03:35.574239+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:09:25.198414+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.15","version":"233'3","reported_seq":14,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575114+0000","last_change":"2023-12-20T06:03:34.121653+0000","last_active":"2023-12-20T06:03:35.575114+0000","last_peered":"2023-12-20T06:03:35.575114+0000","last_clean":"2023-12-20T06:03:35.575114+0000","last_became_active":"2023-12-20T06:03:34.121375+0000","last_became_peered":"2023-12-20T06:03:34.121375+0000","last_unstale":"2023-12-20T06:03:35.575114+0000","last_undegraded":"2023-12-20T06:03:35.575114+0000","last_fullsized":"2023-12-20T06:03:35.575114+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:44:21.275988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.16","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.542866+0000","last_change":"2023-12-20T06:03:21.989153+0000","last_active":"2023-12-20T06:03:34.542866+0000","last_peered":"2023-12-20T06:03:34.542866+0000","last_clean":"2023-12-20T06:03:34.542866+0000","last_became_active":"2023-12-20T06:03:21.988699+0000","last_became_peered":"2023-12-20T06:03:21.988699+0000","last_unstale":"2023-12-20T06:03:34.542866+0000","last_undegraded":"2023-12-20T06:03:34.542866+0000","last_fullsized":"2023-12-20T06:03:34.542866+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:11.133160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.17","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.541414+0000","last_change":"2023-12-20T06:03:21.574001+0000","last_active":"2023-12-20T06:03:34.541414+0000","last_peered":"2023-12-20T06:03:34.541414+0000","last_clean":"2023-12-20T06:03:34.541414+0000","last_became_active":"2023-12-20T06:03:21.573735+0000","last_became_peered":"2023-12-20T06:03:21.573735+0000","last_unstale":"2023-12-20T06:03:34.541414+0000","last_undegraded":"2023-12-20T06:03:34.541414+0000","last_fullsized":"2023-12-20T06:03:34.541414+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:13:55.127307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5],"acting":[7,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.15","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571492+0000","last_change":"2023-12-20T06:03:34.543697+0000","last_active":"2023-12-20T06:03:35.571492+0000","last_peered":"2023-12-20T06:03:35.571492+0000","last_clean":"2023-12-20T06:03:35.571492+0000","last_became_active":"2023-12-20T06:03:34.543314+0000","last_became_peered":"2023-12-20T06:03:34.543314+0000","last_unstale":"2023-12-20T06:03:35.571492+0000","last_undegraded":"2023-12-20T06:03:35.571492+0000","last_fullsized":"2023-12-20T06:03:35.571492+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:07:08.736697+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.14","version":"233'6","reported_seq":17,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573618+0000","last_change":"2023-12-20T06:03:34.131414+0000","last_active":"2023-12-20T06:03:35.573618+0000","last_peered":"2023-12-20T06:03:35.573618+0000","last_clean":"2023-12-20T06:03:35.573618+0000","last_became_active":"2023-12-20T06:03:34.130989+0000","last_became_peered":"2023-12-20T06:03:34.130989+0000","last_unstale":"2023-12-20T06:03:35.573618+0000","last_undegraded":"2023-12-20T06:03:35.573618+0000","last_fullsized":"2023-12-20T06:03:35.573618+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:07:36.615031+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.17","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.541326+0000","last_change":"2023-12-20T06:03:21.967818+0000","last_active":"2023-12-20T06:03:34.541326+0000","last_peered":"2023-12-20T06:03:34.541326+0000","last_clean":"2023-12-20T06:03:34.541326+0000","last_became_active":"2023-12-20T06:03:21.967516+0000","last_became_peered":"2023-12-20T06:03:21.967516+0000","last_unstale":"2023-12-20T06:03:34.541326+0000","last_undegraded":"2023-12-20T06:03:34.541326+0000","last_fullsized":"2023-12-20T06:03:34.541326+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:07:37.126937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.16","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531510+0000","last_change":"2023-12-20T06:03:21.563619+0000","last_active":"2023-12-20T06:03:34.531510+0000","last_peered":"2023-12-20T06:03:34.531510+0000","last_clean":"2023-12-20T06:03:34.531510+0000","last_became_active":"2023-12-20T06:03:21.563273+0000","last_became_peered":"2023-12-20T06:03:21.563273+0000","last_unstale":"2023-12-20T06:03:34.531510+0000","last_undegraded":"2023-12-20T06:03:34.531510+0000","last_fullsized":"2023-12-20T06:03:34.531510+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:39:30.597039+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.16","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575260+0000","last_change":"2023-12-20T06:03:34.550958+0000","last_active":"2023-12-20T06:03:35.575260+0000","last_peered":"2023-12-20T06:03:35.575260+0000","last_clean":"2023-12-20T06:03:35.575260+0000","last_became_active":"2023-12-20T06:03:34.549699+0000","last_became_peered":"2023-12-20T06:03:34.549699+0000","last_unstale":"2023-12-20T06:03:35.575260+0000","last_undegraded":"2023-12-20T06:03:35.575260+0000","last_fullsized":"2023-12-20T06:03:35.575260+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:22:02.206635+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.17","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.537482+0000","last_change":"2023-12-20T06:03:34.128302+0000","last_active":"2023-12-20T06:03:35.537482+0000","last_peered":"2023-12-20T06:03:35.537482+0000","last_clean":"2023-12-20T06:03:35.537482+0000","last_became_active":"2023-12-20T06:03:34.127679+0000","last_became_peered":"2023-12-20T06:03:34.127679+0000","last_unstale":"2023-12-20T06:03:35.537482+0000","last_undegraded":"2023-12-20T06:03:35.537482+0000","last_fullsized":"2023-12-20T06:03:35.537482+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:30:25.302163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"29.14","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529676+0000","last_change":"2023-12-20T06:03:21.990214+0000","last_active":"2023-12-20T06:03:34.529676+0000","last_peered":"2023-12-20T06:03:34.529676+0000","last_clean":"2023-12-20T06:03:34.529676+0000","last_became_active":"2023-12-20T06:03:21.989788+0000","last_became_peered":"2023-12-20T06:03:21.989788+0000","last_unstale":"2023-12-20T06:03:34.529676+0000","last_undegraded":"2023-12-20T06:03:34.529676+0000","last_fullsized":"2023-12-20T06:03:34.529676+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.15","version":"229'3","reported_seq":18,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534698+0000","last_change":"2023-12-20T06:03:21.564537+0000","last_active":"2023-12-20T06:03:34.534698+0000","last_peered":"2023-12-20T06:03:34.534698+0000","last_clean":"2023-12-20T06:03:34.534698+0000","last_became_active":"2023-12-20T06:03:21.564167+0000","last_became_peered":"2023-12-20T06:03:21.564167+0000","last_unstale":"2023-12-20T06:03:34.534698+0000","last_undegraded":"2023-12-20T06:03:34.534698+0000","last_fullsized":"2023-12-20T06:03:34.534698+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:41:11.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.17","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536057+0000","last_change":"2023-12-20T06:03:34.556503+0000","last_active":"2023-12-20T06:03:35.536057+0000","last_peered":"2023-12-20T06:03:35.536057+0000","last_clean":"2023-12-20T06:03:35.536057+0000","last_became_active":"2023-12-20T06:03:34.556125+0000","last_became_peered":"2023-12-20T06:03:34.556125+0000","last_unstale":"2023-12-20T06:03:35.536057+0000","last_undegraded":"2023-12-20T06:03:35.536057+0000","last_fullsized":"2023-12-20T06:03:35.536057+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:38:02.729064+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.16","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574390+0000","last_change":"2023-12-20T06:03:34.121184+0000","last_active":"2023-12-20T06:03:35.574390+0000","last_peered":"2023-12-20T06:03:35.574390+0000","last_clean":"2023-12-20T06:03:35.574390+0000","last_became_active":"2023-12-20T06:03:34.120940+0000","last_became_peered":"2023-12-20T06:03:34.120940+0000","last_unstale":"2023-12-20T06:03:35.574390+0000","last_undegraded":"2023-12-20T06:03:35.574390+0000","last_fullsized":"2023-12-20T06:03:35.574390+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:38:43.241427+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.15","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540715+0000","last_change":"2023-12-20T06:03:21.963443+0000","last_active":"2023-12-20T06:03:34.540715+0000","last_peered":"2023-12-20T06:03:34.540715+0000","last_clean":"2023-12-20T06:03:34.540715+0000","last_became_active":"2023-12-20T06:03:21.962645+0000","last_became_peered":"2023-12-20T06:03:21.962645+0000","last_unstale":"2023-12-20T06:03:34.540715+0000","last_undegraded":"2023-12-20T06:03:34.540715+0000","last_fullsized":"2023-12-20T06:03:34.540715+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:52:50.983047+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.14","version":"230'8","reported_seq":23,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.525303+0000","last_change":"2023-12-20T06:03:21.565407+0000","last_active":"2023-12-20T06:03:34.525303+0000","last_peered":"2023-12-20T06:03:34.525303+0000","last_clean":"2023-12-20T06:03:34.525303+0000","last_became_active":"2023-12-20T06:03:21.564376+0000","last_became_peered":"2023-12-20T06:03:21.564376+0000","last_unstale":"2023-12-20T06:03:34.525303+0000","last_undegraded":"2023-12-20T06:03:34.525303+0000","last_fullsized":"2023-12-20T06:03:34.525303+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:23:07.083767+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":2805,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.10","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572452+0000","last_change":"2023-12-20T06:03:34.560985+0000","last_active":"2023-12-20T06:03:35.572452+0000","last_peered":"2023-12-20T06:03:35.572452+0000","last_clean":"2023-12-20T06:03:35.572452+0000","last_became_active":"2023-12-20T06:03:34.560683+0000","last_became_peered":"2023-12-20T06:03:34.560683+0000","last_unstale":"2023-12-20T06:03:35.572452+0000","last_undegraded":"2023-12-20T06:03:35.572452+0000","last_fullsized":"2023-12-20T06:03:35.572452+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:03:47.233349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.11","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570814+0000","last_change":"2023-12-20T06:03:34.138804+0000","last_active":"2023-12-20T06:03:35.570814+0000","last_peered":"2023-12-20T06:03:35.570814+0000","last_clean":"2023-12-20T06:03:35.570814+0000","last_became_active":"2023-12-20T06:03:34.138379+0000","last_became_peered":"2023-12-20T06:03:34.138379+0000","last_unstale":"2023-12-20T06:03:35.570814+0000","last_undegraded":"2023-12-20T06:03:35.570814+0000","last_fullsized":"2023-12-20T06:03:35.570814+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:18:58.375033+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.12","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537642+0000","last_change":"2023-12-20T06:03:21.971942+0000","last_active":"2023-12-20T06:03:34.537642+0000","last_peered":"2023-12-20T06:03:34.537642+0000","last_clean":"2023-12-20T06:03:34.537642+0000","last_became_active":"2023-12-20T06:03:21.971674+0000","last_became_peered":"2023-12-20T06:03:21.971674+0000","last_unstale":"2023-12-20T06:03:34.537642+0000","last_undegraded":"2023-12-20T06:03:34.537642+0000","last_fullsized":"2023-12-20T06:03:34.537642+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:06:56.167044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"28.13","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530363+0000","last_change":"2023-12-20T06:03:21.567100+0000","last_active":"2023-12-20T06:03:34.530363+0000","last_peered":"2023-12-20T06:03:34.530363+0000","last_clean":"2023-12-20T06:03:34.530363+0000","last_became_active":"2023-12-20T06:03:21.566805+0000","last_became_peered":"2023-12-20T06:03:21.566805+0000","last_unstale":"2023-12-20T06:03:34.530363+0000","last_undegraded":"2023-12-20T06:03:34.530363+0000","last_fullsized":"2023-12-20T06:03:34.530363+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:45:38.064958+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"31.11","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570696+0000","last_change":"2023-12-20T06:03:34.539312+0000","last_active":"2023-12-20T06:03:35.570696+0000","last_peered":"2023-12-20T06:03:35.570696+0000","last_clean":"2023-12-20T06:03:35.570696+0000","last_became_active":"2023-12-20T06:03:34.538981+0000","last_became_peered":"2023-12-20T06:03:34.538981+0000","last_unstale":"2023-12-20T06:03:35.570696+0000","last_undegraded":"2023-12-20T06:03:35.570696+0000","last_fullsized":"2023-12-20T06:03:35.570696+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:46:47.498001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.10","version":"233'8","reported_seq":19,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535056+0000","last_change":"2023-12-20T06:03:34.139382+0000","last_active":"2023-12-20T06:03:35.535056+0000","last_peered":"2023-12-20T06:03:35.535056+0000","last_clean":"2023-12-20T06:03:35.535056+0000","last_became_active":"2023-12-20T06:03:34.139032+0000","last_became_peered":"2023-12-20T06:03:34.139032+0000","last_unstale":"2023-12-20T06:03:35.535056+0000","last_undegraded":"2023-12-20T06:03:35.535056+0000","last_fullsized":"2023-12-20T06:03:35.535056+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:54:35.263517+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.13","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530349+0000","last_change":"2023-12-20T06:03:21.984619+0000","last_active":"2023-12-20T06:03:34.530349+0000","last_peered":"2023-12-20T06:03:34.530349+0000","last_clean":"2023-12-20T06:03:34.530349+0000","last_became_active":"2023-12-20T06:03:21.983940+0000","last_became_peered":"2023-12-20T06:03:21.983940+0000","last_unstale":"2023-12-20T06:03:34.530349+0000","last_undegraded":"2023-12-20T06:03:34.530349+0000","last_fullsized":"2023-12-20T06:03:34.530349+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:13.633088+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.12","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537728+0000","last_change":"2023-12-20T06:03:21.561473+0000","last_active":"2023-12-20T06:03:34.537728+0000","last_peered":"2023-12-20T06:03:34.537728+0000","last_clean":"2023-12-20T06:03:34.537728+0000","last_became_active":"2023-12-20T06:03:21.561038+0000","last_became_peered":"2023-12-20T06:03:21.561038+0000","last_unstale":"2023-12-20T06:03:34.537728+0000","last_undegraded":"2023-12-20T06:03:34.537728+0000","last_fullsized":"2023-12-20T06:03:34.537728+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:57:46.598880+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,5],"acting":[0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.12","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570948+0000","last_change":"2023-12-20T06:03:34.537750+0000","last_active":"2023-12-20T06:03:35.570948+0000","last_peered":"2023-12-20T06:03:35.570948+0000","last_clean":"2023-12-20T06:03:35.570948+0000","last_became_active":"2023-12-20T06:03:34.537451+0000","last_became_peered":"2023-12-20T06:03:34.537451+0000","last_unstale":"2023-12-20T06:03:35.570948+0000","last_undegraded":"2023-12-20T06:03:35.570948+0000","last_fullsized":"2023-12-20T06:03:35.570948+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:01:20.913691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.13","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573972+0000","last_change":"2023-12-20T06:03:34.122211+0000","last_active":"2023-12-20T06:03:35.573972+0000","last_peered":"2023-12-20T06:03:35.573972+0000","last_clean":"2023-12-20T06:03:35.573972+0000","last_became_active":"2023-12-20T06:03:34.121938+0000","last_became_peered":"2023-12-20T06:03:34.121938+0000","last_unstale":"2023-12-20T06:03:35.573972+0000","last_undegraded":"2023-12-20T06:03:35.573972+0000","last_fullsized":"2023-12-20T06:03:35.573972+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:30:25.302163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"29.10","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529207+0000","last_change":"2023-12-20T06:03:21.987462+0000","last_active":"2023-12-20T06:03:34.529207+0000","last_peered":"2023-12-20T06:03:34.529207+0000","last_clean":"2023-12-20T06:03:34.529207+0000","last_became_active":"2023-12-20T06:03:21.986488+0000","last_became_peered":"2023-12-20T06:03:21.986488+0000","last_unstale":"2023-12-20T06:03:34.529207+0000","last_undegraded":"2023-12-20T06:03:34.529207+0000","last_fullsized":"2023-12-20T06:03:34.529207+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:48:00.841811+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.11","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534367+0000","last_change":"2023-12-20T06:03:21.564490+0000","last_active":"2023-12-20T06:03:34.534367+0000","last_peered":"2023-12-20T06:03:34.534367+0000","last_clean":"2023-12-20T06:03:34.534367+0000","last_became_active":"2023-12-20T06:03:21.562571+0000","last_became_peered":"2023-12-20T06:03:21.562571+0000","last_unstale":"2023-12-20T06:03:34.534367+0000","last_undegraded":"2023-12-20T06:03:34.534367+0000","last_fullsized":"2023-12-20T06:03:34.534367+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.13","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571306+0000","last_change":"2023-12-20T06:03:34.545843+0000","last_active":"2023-12-20T06:03:35.571306+0000","last_peered":"2023-12-20T06:03:35.571306+0000","last_clean":"2023-12-20T06:03:35.571306+0000","last_became_active":"2023-12-20T06:03:34.545579+0000","last_became_peered":"2023-12-20T06:03:34.545579+0000","last_unstale":"2023-12-20T06:03:35.571306+0000","last_undegraded":"2023-12-20T06:03:35.571306+0000","last_fullsized":"2023-12-20T06:03:35.571306+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:00:21.937869+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.12","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573193+0000","last_change":"2023-12-20T06:03:34.136207+0000","last_active":"2023-12-20T06:03:35.573193+0000","last_peered":"2023-12-20T06:03:35.573193+0000","last_clean":"2023-12-20T06:03:35.573193+0000","last_became_active":"2023-12-20T06:03:34.135946+0000","last_became_peered":"2023-12-20T06:03:34.135946+0000","last_unstale":"2023-12-20T06:03:35.573193+0000","last_undegraded":"2023-12-20T06:03:35.573193+0000","last_fullsized":"2023-12-20T06:03:35.573193+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:20:26.882709+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.11","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528357+0000","last_change":"2023-12-20T06:03:21.978828+0000","last_active":"2023-12-20T06:03:34.528357+0000","last_peered":"2023-12-20T06:03:34.528357+0000","last_clean":"2023-12-20T06:03:34.528357+0000","last_became_active":"2023-12-20T06:03:21.977595+0000","last_became_peered":"2023-12-20T06:03:21.977595+0000","last_unstale":"2023-12-20T06:03:34.528357+0000","last_undegraded":"2023-12-20T06:03:34.528357+0000","last_fullsized":"2023-12-20T06:03:34.528357+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:53:41.601867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"28.10","version":"229'8","reported_seq":23,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537981+0000","last_change":"2023-12-20T06:03:21.556640+0000","last_active":"2023-12-20T06:03:34.537981+0000","last_peered":"2023-12-20T06:03:34.537981+0000","last_clean":"2023-12-20T06:03:34.537981+0000","last_became_active":"2023-12-20T06:03:21.556362+0000","last_became_peered":"2023-12-20T06:03:21.556362+0000","last_unstale":"2023-12-20T06:03:34.537981+0000","last_undegraded":"2023-12-20T06:03:34.537981+0000","last_fullsized":"2023-12-20T06:03:34.537981+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:16:40.268127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"31.c","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572356+0000","last_change":"2023-12-20T06:03:34.545159+0000","last_active":"2023-12-20T06:03:35.572356+0000","last_peered":"2023-12-20T06:03:35.572356+0000","last_clean":"2023-12-20T06:03:35.572356+0000","last_became_active":"2023-12-20T06:03:34.544891+0000","last_became_peered":"2023-12-20T06:03:34.544891+0000","last_unstale":"2023-12-20T06:03:35.572356+0000","last_undegraded":"2023-12-20T06:03:35.572356+0000","last_fullsized":"2023-12-20T06:03:35.572356+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:25:10.820979+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"30.d","version":"233'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572406+0000","last_change":"2023-12-20T06:03:34.139831+0000","last_active":"2023-12-20T06:03:35.572406+0000","last_peered":"2023-12-20T06:03:35.572406+0000","last_clean":"2023-12-20T06:03:35.572406+0000","last_became_active":"2023-12-20T06:03:34.139367+0000","last_became_peered":"2023-12-20T06:03:34.139367+0000","last_unstale":"2023-12-20T06:03:35.572406+0000","last_undegraded":"2023-12-20T06:03:35.572406+0000","last_fullsized":"2023-12-20T06:03:35.572406+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:22:16.190203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.e","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531083+0000","last_change":"2023-12-20T06:03:21.972553+0000","last_active":"2023-12-20T06:03:34.531083+0000","last_peered":"2023-12-20T06:03:34.531083+0000","last_clean":"2023-12-20T06:03:34.531083+0000","last_became_active":"2023-12-20T06:03:21.972295+0000","last_became_peered":"2023-12-20T06:03:21.972295+0000","last_unstale":"2023-12-20T06:03:34.531083+0000","last_undegraded":"2023-12-20T06:03:34.531083+0000","last_fullsized":"2023-12-20T06:03:34.531083+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:44.890915+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.f","version":"229'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531092+0000","last_change":"2023-12-20T06:03:21.563512+0000","last_active":"2023-12-20T06:03:34.531092+0000","last_peered":"2023-12-20T06:03:34.531092+0000","last_clean":"2023-12-20T06:03:34.531092+0000","last_became_active":"2023-12-20T06:03:21.563130+0000","last_became_peered":"2023-12-20T06:03:21.563130+0000","last_unstale":"2023-12-20T06:03:34.531092+0000","last_undegraded":"2023-12-20T06:03:34.531092+0000","last_fullsized":"2023-12-20T06:03:34.531092+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:28.719459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.d","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.537429+0000","last_change":"2023-12-20T06:03:34.551376+0000","last_active":"2023-12-20T06:03:35.537429+0000","last_peered":"2023-12-20T06:03:35.537429+0000","last_clean":"2023-12-20T06:03:35.537429+0000","last_became_active":"2023-12-20T06:03:34.550896+0000","last_became_peered":"2023-12-20T06:03:34.550896+0000","last_unstale":"2023-12-20T06:03:35.537429+0000","last_undegraded":"2023-12-20T06:03:35.537429+0000","last_fullsized":"2023-12-20T06:03:35.537429+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:56:41.119728+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.c","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535727+0000","last_change":"2023-12-20T06:03:34.131712+0000","last_active":"2023-12-20T06:03:35.535727+0000","last_peered":"2023-12-20T06:03:35.535727+0000","last_clean":"2023-12-20T06:03:35.535727+0000","last_became_active":"2023-12-20T06:03:34.131370+0000","last_became_peered":"2023-12-20T06:03:34.131370+0000","last_unstale":"2023-12-20T06:03:35.535727+0000","last_undegraded":"2023-12-20T06:03:35.535727+0000","last_fullsized":"2023-12-20T06:03:35.535727+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:02.003235+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.f","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.526066+0000","last_change":"2023-12-20T06:03:21.991246+0000","last_active":"2023-12-20T06:03:34.526066+0000","last_peered":"2023-12-20T06:03:34.526066+0000","last_clean":"2023-12-20T06:03:34.526066+0000","last_became_active":"2023-12-20T06:03:21.990892+0000","last_became_peered":"2023-12-20T06:03:21.990892+0000","last_unstale":"2023-12-20T06:03:34.526066+0000","last_undegraded":"2023-12-20T06:03:34.526066+0000","last_fullsized":"2023-12-20T06:03:34.526066+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:18:52.328250+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"28.e","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.531601+0000","last_change":"2023-12-20T06:03:21.576334+0000","last_active":"2023-12-20T06:03:34.531601+0000","last_peered":"2023-12-20T06:03:34.531601+0000","last_clean":"2023-12-20T06:03:34.531601+0000","last_became_active":"2023-12-20T06:03:21.575847+0000","last_became_peered":"2023-12-20T06:03:21.575847+0000","last_unstale":"2023-12-20T06:03:34.531601+0000","last_undegraded":"2023-12-20T06:03:34.531601+0000","last_fullsized":"2023-12-20T06:03:34.531601+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:46:33.911145+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7],"acting":[0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574331+0000","last_change":"2023-12-20T06:03:34.548458+0000","last_active":"2023-12-20T06:03:35.574331+0000","last_peered":"2023-12-20T06:03:35.574331+0000","last_clean":"2023-12-20T06:03:35.574331+0000","last_became_active":"2023-12-20T06:03:34.548187+0000","last_became_peered":"2023-12-20T06:03:34.548187+0000","last_unstale":"2023-12-20T06:03:35.574331+0000","last_undegraded":"2023-12-20T06:03:35.574331+0000","last_fullsized":"2023-12-20T06:03:35.574331+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:21:43.853129+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5],"acting":[2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"30.f","version":"233'1","reported_seq":12,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571675+0000","last_change":"2023-12-20T06:03:34.135718+0000","last_active":"2023-12-20T06:03:35.571675+0000","last_peered":"2023-12-20T06:03:35.571675+0000","last_clean":"2023-12-20T06:03:35.571675+0000","last_became_active":"2023-12-20T06:03:34.134508+0000","last_became_peered":"2023-12-20T06:03:34.134508+0000","last_unstale":"2023-12-20T06:03:35.571675+0000","last_undegraded":"2023-12-20T06:03:35.571675+0000","last_fullsized":"2023-12-20T06:03:35.571675+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:37:50.486590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.c","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530146+0000","last_change":"2023-12-20T06:03:21.988502+0000","last_active":"2023-12-20T06:03:34.530146+0000","last_peered":"2023-12-20T06:03:34.530146+0000","last_clean":"2023-12-20T06:03:34.530146+0000","last_became_active":"2023-12-20T06:03:21.988209+0000","last_became_peered":"2023-12-20T06:03:21.988209+0000","last_unstale":"2023-12-20T06:03:34.530146+0000","last_undegraded":"2023-12-20T06:03:34.530146+0000","last_fullsized":"2023-12-20T06:03:34.530146+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:35:37.344273+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.d","version":"229'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540816+0000","last_change":"2023-12-20T06:03:21.574438+0000","last_active":"2023-12-20T06:03:34.540816+0000","last_peered":"2023-12-20T06:03:34.540816+0000","last_clean":"2023-12-20T06:03:34.540816+0000","last_became_active":"2023-12-20T06:03:21.574161+0000","last_became_peered":"2023-12-20T06:03:21.574161+0000","last_unstale":"2023-12-20T06:03:34.540816+0000","last_undegraded":"2023-12-20T06:03:34.540816+0000","last_fullsized":"2023-12-20T06:03:34.540816+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:21:17.124107+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.f","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574420+0000","last_change":"2023-12-20T06:03:34.541342+0000","last_active":"2023-12-20T06:03:35.574420+0000","last_peered":"2023-12-20T06:03:35.574420+0000","last_clean":"2023-12-20T06:03:35.574420+0000","last_became_active":"2023-12-20T06:03:34.540910+0000","last_became_peered":"2023-12-20T06:03:34.540910+0000","last_unstale":"2023-12-20T06:03:35.574420+0000","last_undegraded":"2023-12-20T06:03:35.574420+0000","last_fullsized":"2023-12-20T06:03:35.574420+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:03.024703+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"30.e","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574476+0000","last_change":"2023-12-20T06:03:34.123898+0000","last_active":"2023-12-20T06:03:35.574476+0000","last_peered":"2023-12-20T06:03:35.574476+0000","last_clean":"2023-12-20T06:03:35.574476+0000","last_became_active":"2023-12-20T06:03:34.123599+0000","last_became_peered":"2023-12-20T06:03:34.123599+0000","last_unstale":"2023-12-20T06:03:35.574476+0000","last_undegraded":"2023-12-20T06:03:35.574476+0000","last_fullsized":"2023-12-20T06:03:35.574476+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:12:15.603076+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.d","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530454+0000","last_change":"2023-12-20T06:03:21.985314+0000","last_active":"2023-12-20T06:03:34.530454+0000","last_peered":"2023-12-20T06:03:34.530454+0000","last_clean":"2023-12-20T06:03:34.530454+0000","last_became_active":"2023-12-20T06:03:21.984916+0000","last_became_peered":"2023-12-20T06:03:21.984916+0000","last_unstale":"2023-12-20T06:03:34.530454+0000","last_undegraded":"2023-12-20T06:03:34.530454+0000","last_fullsized":"2023-12-20T06:03:34.530454+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:55:10.381599+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.c","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.525380+0000","last_change":"2023-12-20T06:03:21.576277+0000","last_active":"2023-12-20T06:03:34.525380+0000","last_peered":"2023-12-20T06:03:34.525380+0000","last_clean":"2023-12-20T06:03:34.525380+0000","last_became_active":"2023-12-20T06:03:21.575791+0000","last_became_peered":"2023-12-20T06:03:21.575791+0000","last_unstale":"2023-12-20T06:03:34.525380+0000","last_undegraded":"2023-12-20T06:03:34.525380+0000","last_fullsized":"2023-12-20T06:03:34.525380+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:24:57.234123+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"31.8","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.534929+0000","last_change":"2023-12-20T06:03:34.555586+0000","last_active":"2023-12-20T06:03:35.534929+0000","last_peered":"2023-12-20T06:03:35.534929+0000","last_clean":"2023-12-20T06:03:35.534929+0000","last_became_active":"2023-12-20T06:03:34.555329+0000","last_became_peered":"2023-12-20T06:03:34.555329+0000","last_unstale":"2023-12-20T06:03:35.534929+0000","last_undegraded":"2023-12-20T06:03:35.534929+0000","last_fullsized":"2023-12-20T06:03:35.534929+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:14.916269+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.9","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570432+0000","last_change":"2023-12-20T06:03:34.134679+0000","last_active":"2023-12-20T06:03:35.570432+0000","last_peered":"2023-12-20T06:03:35.570432+0000","last_clean":"2023-12-20T06:03:35.570432+0000","last_became_active":"2023-12-20T06:03:34.134384+0000","last_became_peered":"2023-12-20T06:03:34.134384+0000","last_unstale":"2023-12-20T06:03:35.570432+0000","last_undegraded":"2023-12-20T06:03:35.570432+0000","last_fullsized":"2023-12-20T06:03:35.570432+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:22.681346+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.a","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.533321+0000","last_change":"2023-12-20T06:03:21.977046+0000","last_active":"2023-12-20T06:03:34.533321+0000","last_peered":"2023-12-20T06:03:34.533321+0000","last_clean":"2023-12-20T06:03:34.533321+0000","last_became_active":"2023-12-20T06:03:21.976756+0000","last_became_peered":"2023-12-20T06:03:21.976756+0000","last_unstale":"2023-12-20T06:03:34.533321+0000","last_undegraded":"2023-12-20T06:03:34.533321+0000","last_fullsized":"2023-12-20T06:03:34.533321+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:57:17.456340+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.b","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530709+0000","last_change":"2023-12-20T06:03:21.573650+0000","last_active":"2023-12-20T06:03:34.530709+0000","last_peered":"2023-12-20T06:03:34.530709+0000","last_clean":"2023-12-20T06:03:34.530709+0000","last_became_active":"2023-12-20T06:03:21.573349+0000","last_became_peered":"2023-12-20T06:03:21.573349+0000","last_unstale":"2023-12-20T06:03:34.530709+0000","last_undegraded":"2023-12-20T06:03:34.530709+0000","last_fullsized":"2023-12-20T06:03:34.530709+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:41:22.788125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.9","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536535+0000","last_change":"2023-12-20T06:03:34.552429+0000","last_active":"2023-12-20T06:03:35.536535+0000","last_peered":"2023-12-20T06:03:35.536535+0000","last_clean":"2023-12-20T06:03:35.536535+0000","last_became_active":"2023-12-20T06:03:34.551942+0000","last_became_peered":"2023-12-20T06:03:34.551942+0000","last_unstale":"2023-12-20T06:03:35.536535+0000","last_undegraded":"2023-12-20T06:03:35.536535+0000","last_fullsized":"2023-12-20T06:03:35.536535+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:22:42.632244+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.8","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.570774+0000","last_change":"2023-12-20T06:03:34.133267+0000","last_active":"2023-12-20T06:03:35.570774+0000","last_peered":"2023-12-20T06:03:35.570774+0000","last_clean":"2023-12-20T06:03:35.570774+0000","last_became_active":"2023-12-20T06:03:34.132981+0000","last_became_peered":"2023-12-20T06:03:34.132981+0000","last_unstale":"2023-12-20T06:03:35.570774+0000","last_undegraded":"2023-12-20T06:03:35.570774+0000","last_fullsized":"2023-12-20T06:03:35.570774+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:14:30.574620+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.b","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530446+0000","last_change":"2023-12-20T06:03:21.983774+0000","last_active":"2023-12-20T06:03:34.530446+0000","last_peered":"2023-12-20T06:03:34.530446+0000","last_clean":"2023-12-20T06:03:34.530446+0000","last_became_active":"2023-12-20T06:03:21.983388+0000","last_became_peered":"2023-12-20T06:03:21.983388+0000","last_unstale":"2023-12-20T06:03:34.530446+0000","last_undegraded":"2023-12-20T06:03:34.530446+0000","last_fullsized":"2023-12-20T06:03:34.530446+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:22:11.133160+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.a","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528503+0000","last_change":"2023-12-20T06:03:21.567023+0000","last_active":"2023-12-20T06:03:34.528503+0000","last_peered":"2023-12-20T06:03:34.528503+0000","last_clean":"2023-12-20T06:03:34.528503+0000","last_became_active":"2023-12-20T06:03:21.566708+0000","last_became_peered":"2023-12-20T06:03:21.566708+0000","last_unstale":"2023-12-20T06:03:34.528503+0000","last_undegraded":"2023-12-20T06:03:34.528503+0000","last_fullsized":"2023-12-20T06:03:34.528503+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3],"acting":[5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.a","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536633+0000","last_change":"2023-12-20T06:03:34.544736+0000","last_active":"2023-12-20T06:03:35.536633+0000","last_peered":"2023-12-20T06:03:35.536633+0000","last_clean":"2023-12-20T06:03:35.536633+0000","last_became_active":"2023-12-20T06:03:34.544230+0000","last_became_peered":"2023-12-20T06:03:34.544230+0000","last_unstale":"2023-12-20T06:03:35.536633+0000","last_undegraded":"2023-12-20T06:03:35.536633+0000","last_fullsized":"2023-12-20T06:03:35.536633+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:58:00.185736+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.b","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573662+0000","last_change":"2023-12-20T06:03:34.134914+0000","last_active":"2023-12-20T06:03:35.573662+0000","last_peered":"2023-12-20T06:03:35.573662+0000","last_clean":"2023-12-20T06:03:35.573662+0000","last_became_active":"2023-12-20T06:03:34.134561+0000","last_became_peered":"2023-12-20T06:03:34.134561+0000","last_unstale":"2023-12-20T06:03:35.573662+0000","last_undegraded":"2023-12-20T06:03:35.573662+0000","last_fullsized":"2023-12-20T06:03:35.573662+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:24:28.973553+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.8","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530241+0000","last_change":"2023-12-20T06:03:21.972573+0000","last_active":"2023-12-20T06:03:34.530241+0000","last_peered":"2023-12-20T06:03:34.530241+0000","last_clean":"2023-12-20T06:03:34.530241+0000","last_became_active":"2023-12-20T06:03:21.972302+0000","last_became_peered":"2023-12-20T06:03:21.972302+0000","last_unstale":"2023-12-20T06:03:34.530241+0000","last_undegraded":"2023-12-20T06:03:34.530241+0000","last_fullsized":"2023-12-20T06:03:34.530241+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:58:25.448451+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"28.9","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528615+0000","last_change":"2023-12-20T06:03:21.570826+0000","last_active":"2023-12-20T06:03:34.528615+0000","last_peered":"2023-12-20T06:03:34.528615+0000","last_clean":"2023-12-20T06:03:34.528615+0000","last_became_active":"2023-12-20T06:03:21.570512+0000","last_became_peered":"2023-12-20T06:03:21.570512+0000","last_unstale":"2023-12-20T06:03:34.528615+0000","last_undegraded":"2023-12-20T06:03:34.528615+0000","last_fullsized":"2023-12-20T06:03:34.528615+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:52:49.965844+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.b","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573072+0000","last_change":"2023-12-20T06:03:34.560892+0000","last_active":"2023-12-20T06:03:35.573072+0000","last_peered":"2023-12-20T06:03:35.573072+0000","last_clean":"2023-12-20T06:03:35.573072+0000","last_became_active":"2023-12-20T06:03:34.559695+0000","last_became_peered":"2023-12-20T06:03:34.559695+0000","last_unstale":"2023-12-20T06:03:35.573072+0000","last_undegraded":"2023-12-20T06:03:35.573072+0000","last_fullsized":"2023-12-20T06:03:35.573072+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:54:48.032421+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.a","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535090+0000","last_change":"2023-12-20T06:03:34.132619+0000","last_active":"2023-12-20T06:03:35.535090+0000","last_peered":"2023-12-20T06:03:35.535090+0000","last_clean":"2023-12-20T06:03:35.535090+0000","last_became_active":"2023-12-20T06:03:34.132242+0000","last_became_peered":"2023-12-20T06:03:34.132242+0000","last_unstale":"2023-12-20T06:03:35.535090+0000","last_undegraded":"2023-12-20T06:03:35.535090+0000","last_fullsized":"2023-12-20T06:03:35.535090+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:10.999454+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.9","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.533851+0000","last_change":"2023-12-20T06:03:21.976420+0000","last_active":"2023-12-20T06:03:34.533851+0000","last_peered":"2023-12-20T06:03:34.533851+0000","last_clean":"2023-12-20T06:03:34.533851+0000","last_became_active":"2023-12-20T06:03:21.976141+0000","last_became_peered":"2023-12-20T06:03:21.976141+0000","last_unstale":"2023-12-20T06:03:34.533851+0000","last_undegraded":"2023-12-20T06:03:34.533851+0000","last_fullsized":"2023-12-20T06:03:34.533851+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:27:59.998522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.8","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529393+0000","last_change":"2023-12-20T06:03:21.567122+0000","last_active":"2023-12-20T06:03:34.529393+0000","last_peered":"2023-12-20T06:03:34.529393+0000","last_clean":"2023-12-20T06:03:34.529393+0000","last_became_active":"2023-12-20T06:03:21.566816+0000","last_became_peered":"2023-12-20T06:03:21.566816+0000","last_unstale":"2023-12-20T06:03:34.529393+0000","last_undegraded":"2023-12-20T06:03:34.529393+0000","last_fullsized":"2023-12-20T06:03:34.529393+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:41:11.756512+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.4","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535626+0000","last_change":"2023-12-20T06:03:34.554385+0000","last_active":"2023-12-20T06:03:35.535626+0000","last_peered":"2023-12-20T06:03:35.535626+0000","last_clean":"2023-12-20T06:03:35.535626+0000","last_became_active":"2023-12-20T06:03:34.552807+0000","last_became_peered":"2023-12-20T06:03:34.552807+0000","last_unstale":"2023-12-20T06:03:35.535626+0000","last_undegraded":"2023-12-20T06:03:35.535626+0000","last_fullsized":"2023-12-20T06:03:35.535626+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:49:08.149121+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.5","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572516+0000","last_change":"2023-12-20T06:03:34.140357+0000","last_active":"2023-12-20T06:03:35.572516+0000","last_peered":"2023-12-20T06:03:35.572516+0000","last_clean":"2023-12-20T06:03:35.572516+0000","last_became_active":"2023-12-20T06:03:34.140032+0000","last_became_peered":"2023-12-20T06:03:34.140032+0000","last_unstale":"2023-12-20T06:03:35.572516+0000","last_undegraded":"2023-12-20T06:03:35.572516+0000","last_fullsized":"2023-12-20T06:03:35.572516+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:06:27.994614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"29.6","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534600+0000","last_change":"2023-12-20T06:03:21.975113+0000","last_active":"2023-12-20T06:03:34.534600+0000","last_peered":"2023-12-20T06:03:34.534600+0000","last_clean":"2023-12-20T06:03:34.534600+0000","last_became_active":"2023-12-20T06:03:21.974862+0000","last_became_peered":"2023-12-20T06:03:21.974862+0000","last_unstale":"2023-12-20T06:03:34.534600+0000","last_undegraded":"2023-12-20T06:03:34.534600+0000","last_fullsized":"2023-12-20T06:03:34.534600+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:07:37.126937+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.7","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534155+0000","last_change":"2023-12-20T06:03:21.569876+0000","last_active":"2023-12-20T06:03:34.534155+0000","last_peered":"2023-12-20T06:03:34.534155+0000","last_clean":"2023-12-20T06:03:34.534155+0000","last_became_active":"2023-12-20T06:03:21.568286+0000","last_became_peered":"2023-12-20T06:03:21.568286+0000","last_unstale":"2023-12-20T06:03:34.534155+0000","last_undegraded":"2023-12-20T06:03:34.534155+0000","last_fullsized":"2023-12-20T06:03:34.534155+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:10:32.405173+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.5","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535865+0000","last_change":"2023-12-20T06:03:34.557531+0000","last_active":"2023-12-20T06:03:35.535865+0000","last_peered":"2023-12-20T06:03:35.535865+0000","last_clean":"2023-12-20T06:03:35.535865+0000","last_became_active":"2023-12-20T06:03:34.557228+0000","last_became_peered":"2023-12-20T06:03:34.557228+0000","last_unstale":"2023-12-20T06:03:35.535865+0000","last_undegraded":"2023-12-20T06:03:35.535865+0000","last_fullsized":"2023-12-20T06:03:35.535865+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:36:23.565638+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"30.4","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535726+0000","last_change":"2023-12-20T06:03:34.123821+0000","last_active":"2023-12-20T06:03:35.535726+0000","last_peered":"2023-12-20T06:03:35.535726+0000","last_clean":"2023-12-20T06:03:35.535726+0000","last_became_active":"2023-12-20T06:03:34.123356+0000","last_became_peered":"2023-12-20T06:03:34.123356+0000","last_unstale":"2023-12-20T06:03:35.535726+0000","last_undegraded":"2023-12-20T06:03:35.535726+0000","last_fullsized":"2023-12-20T06:03:35.535726+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T18:01:48.061998+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"29.7","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530997+0000","last_change":"2023-12-20T06:03:21.991437+0000","last_active":"2023-12-20T06:03:34.530997+0000","last_peered":"2023-12-20T06:03:34.530997+0000","last_clean":"2023-12-20T06:03:34.530997+0000","last_became_active":"2023-12-20T06:03:21.991098+0000","last_became_peered":"2023-12-20T06:03:21.991098+0000","last_unstale":"2023-12-20T06:03:34.530997+0000","last_undegraded":"2023-12-20T06:03:34.530997+0000","last_fullsized":"2023-12-20T06:03:34.530997+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:14:48.782916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.6","version":"230'1","reported_seq":16,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540595+0000","last_change":"2023-12-20T06:03:21.574738+0000","last_active":"2023-12-20T06:03:34.540595+0000","last_peered":"2023-12-20T06:03:34.540595+0000","last_clean":"2023-12-20T06:03:34.540595+0000","last_became_active":"2023-12-20T06:03:21.574282+0000","last_became_peered":"2023-12-20T06:03:21.574282+0000","last_unstale":"2023-12-20T06:03:34.540595+0000","last_undegraded":"2023-12-20T06:03:34.540595+0000","last_fullsized":"2023-12-20T06:03:34.540595+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:43:56.675318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.6","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571986+0000","last_change":"2023-12-20T06:03:34.544724+0000","last_active":"2023-12-20T06:03:35.571986+0000","last_peered":"2023-12-20T06:03:35.571986+0000","last_clean":"2023-12-20T06:03:35.571986+0000","last_became_active":"2023-12-20T06:03:34.544423+0000","last_became_peered":"2023-12-20T06:03:34.544423+0000","last_unstale":"2023-12-20T06:03:35.571986+0000","last_undegraded":"2023-12-20T06:03:35.571986+0000","last_fullsized":"2023-12-20T06:03:35.571986+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:08:59.188281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.7","version":"233'2","reported_seq":13,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575410+0000","last_change":"2023-12-20T06:03:34.120721+0000","last_active":"2023-12-20T06:03:35.575410+0000","last_peered":"2023-12-20T06:03:35.575410+0000","last_clean":"2023-12-20T06:03:35.575410+0000","last_became_active":"2023-12-20T06:03:34.120453+0000","last_became_peered":"2023-12-20T06:03:34.120453+0000","last_unstale":"2023-12-20T06:03:35.575410+0000","last_undegraded":"2023-12-20T06:03:35.575410+0000","last_fullsized":"2023-12-20T06:03:35.575410+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:53:02.531233+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.4","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530488+0000","last_change":"2023-12-20T06:03:21.985967+0000","last_active":"2023-12-20T06:03:34.530488+0000","last_peered":"2023-12-20T06:03:34.530488+0000","last_clean":"2023-12-20T06:03:34.530488+0000","last_became_active":"2023-12-20T06:03:21.985460+0000","last_became_peered":"2023-12-20T06:03:21.985460+0000","last_unstale":"2023-12-20T06:03:34.530488+0000","last_undegraded":"2023-12-20T06:03:34.530488+0000","last_fullsized":"2023-12-20T06:03:34.530488+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:06:16.446428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.5","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.540868+0000","last_change":"2023-12-20T06:03:21.573978+0000","last_active":"2023-12-20T06:03:34.540868+0000","last_peered":"2023-12-20T06:03:34.540868+0000","last_clean":"2023-12-20T06:03:34.540868+0000","last_became_active":"2023-12-20T06:03:21.573682+0000","last_became_peered":"2023-12-20T06:03:21.573682+0000","last_unstale":"2023-12-20T06:03:34.540868+0000","last_undegraded":"2023-12-20T06:03:34.540868+0000","last_fullsized":"2023-12-20T06:03:34.540868+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:47:43.873712+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"31.7","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.575230+0000","last_change":"2023-12-20T06:03:34.557806+0000","last_active":"2023-12-20T06:03:35.575230+0000","last_peered":"2023-12-20T06:03:35.575230+0000","last_clean":"2023-12-20T06:03:35.575230+0000","last_became_active":"2023-12-20T06:03:34.557222+0000","last_became_peered":"2023-12-20T06:03:34.557222+0000","last_unstale":"2023-12-20T06:03:35.575230+0000","last_undegraded":"2023-12-20T06:03:35.575230+0000","last_fullsized":"2023-12-20T06:03:35.575230+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:01:20.913691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.6","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535766+0000","last_change":"2023-12-20T06:03:34.135175+0000","last_active":"2023-12-20T06:03:35.535766+0000","last_peered":"2023-12-20T06:03:35.535766+0000","last_clean":"2023-12-20T06:03:35.535766+0000","last_became_active":"2023-12-20T06:03:34.134886+0000","last_became_peered":"2023-12-20T06:03:34.134886+0000","last_unstale":"2023-12-20T06:03:35.535766+0000","last_undegraded":"2023-12-20T06:03:35.535766+0000","last_fullsized":"2023-12-20T06:03:35.535766+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:56:24.405375+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"29.5","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534543+0000","last_change":"2023-12-20T06:03:21.980139+0000","last_active":"2023-12-20T06:03:34.534543+0000","last_peered":"2023-12-20T06:03:34.534543+0000","last_clean":"2023-12-20T06:03:34.534543+0000","last_became_active":"2023-12-20T06:03:21.979858+0000","last_became_peered":"2023-12-20T06:03:21.979858+0000","last_unstale":"2023-12-20T06:03:34.534543+0000","last_undegraded":"2023-12-20T06:03:34.534543+0000","last_fullsized":"2023-12-20T06:03:34.534543+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:21:31.283475+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"28.4","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530356+0000","last_change":"2023-12-20T06:03:21.570642+0000","last_active":"2023-12-20T06:03:34.530356+0000","last_peered":"2023-12-20T06:03:34.530356+0000","last_clean":"2023-12-20T06:03:34.530356+0000","last_became_active":"2023-12-20T06:03:21.569631+0000","last_became_peered":"2023-12-20T06:03:21.569631+0000","last_unstale":"2023-12-20T06:03:34.530356+0000","last_undegraded":"2023-12-20T06:03:34.530356+0000","last_fullsized":"2023-12-20T06:03:34.530356+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.0","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536417+0000","last_change":"2023-12-20T06:03:34.549025+0000","last_active":"2023-12-20T06:03:35.536417+0000","last_peered":"2023-12-20T06:03:35.536417+0000","last_clean":"2023-12-20T06:03:35.536417+0000","last_became_active":"2023-12-20T06:03:34.548725+0000","last_became_peered":"2023-12-20T06:03:34.548725+0000","last_unstale":"2023-12-20T06:03:35.536417+0000","last_undegraded":"2023-12-20T06:03:35.536417+0000","last_fullsized":"2023-12-20T06:03:35.536417+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:01:20.913691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.1","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572723+0000","last_change":"2023-12-20T06:03:34.133700+0000","last_active":"2023-12-20T06:03:35.572723+0000","last_peered":"2023-12-20T06:03:35.572723+0000","last_clean":"2023-12-20T06:03:35.572723+0000","last_became_active":"2023-12-20T06:03:34.133371+0000","last_became_peered":"2023-12-20T06:03:34.133371+0000","last_unstale":"2023-12-20T06:03:35.572723+0000","last_undegraded":"2023-12-20T06:03:35.572723+0000","last_fullsized":"2023-12-20T06:03:35.572723+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:23:13.107999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.2","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.538278+0000","last_change":"2023-12-20T06:03:21.963940+0000","last_active":"2023-12-20T06:03:34.538278+0000","last_peered":"2023-12-20T06:03:34.538278+0000","last_clean":"2023-12-20T06:03:34.538278+0000","last_became_active":"2023-12-20T06:03:21.963679+0000","last_became_peered":"2023-12-20T06:03:21.963679+0000","last_unstale":"2023-12-20T06:03:34.538278+0000","last_undegraded":"2023-12-20T06:03:34.538278+0000","last_fullsized":"2023-12-20T06:03:34.538278+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:26:32.396951+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"28.3","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528505+0000","last_change":"2023-12-20T06:03:21.573888+0000","last_active":"2023-12-20T06:03:34.528505+0000","last_peered":"2023-12-20T06:03:34.528505+0000","last_clean":"2023-12-20T06:03:34.528505+0000","last_became_active":"2023-12-20T06:03:21.572823+0000","last_became_peered":"2023-12-20T06:03:21.572823+0000","last_unstale":"2023-12-20T06:03:34.528505+0000","last_undegraded":"2023-12-20T06:03:34.528505+0000","last_fullsized":"2023-12-20T06:03:34.528505+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:17:12.713668+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"31.1","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.572461+0000","last_change":"2023-12-20T06:03:34.549076+0000","last_active":"2023-12-20T06:03:35.572461+0000","last_peered":"2023-12-20T06:03:35.572461+0000","last_clean":"2023-12-20T06:03:35.572461+0000","last_became_active":"2023-12-20T06:03:34.548834+0000","last_became_peered":"2023-12-20T06:03:34.548834+0000","last_unstale":"2023-12-20T06:03:35.572461+0000","last_undegraded":"2023-12-20T06:03:35.572461+0000","last_fullsized":"2023-12-20T06:03:35.572461+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:38:13.606277+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"30.0","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573233+0000","last_change":"2023-12-20T06:03:34.129063+0000","last_active":"2023-12-20T06:03:35.573233+0000","last_peered":"2023-12-20T06:03:35.573233+0000","last_clean":"2023-12-20T06:03:35.573233+0000","last_became_active":"2023-12-20T06:03:34.128730+0000","last_became_peered":"2023-12-20T06:03:34.128730+0000","last_unstale":"2023-12-20T06:03:35.573233+0000","last_undegraded":"2023-12-20T06:03:35.573233+0000","last_fullsized":"2023-12-20T06:03:35.573233+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:30:44.160478+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.3","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.530539+0000","last_change":"2023-12-20T06:03:21.983107+0000","last_active":"2023-12-20T06:03:34.530539+0000","last_peered":"2023-12-20T06:03:34.530539+0000","last_clean":"2023-12-20T06:03:34.530539+0000","last_became_active":"2023-12-20T06:03:21.981255+0000","last_became_peered":"2023-12-20T06:03:21.981255+0000","last_unstale":"2023-12-20T06:03:34.530539+0000","last_undegraded":"2023-12-20T06:03:34.530539+0000","last_fullsized":"2023-12-20T06:03:34.530539+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:04:16.224867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.2","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537824+0000","last_change":"2023-12-20T06:03:21.558503+0000","last_active":"2023-12-20T06:03:34.537824+0000","last_peered":"2023-12-20T06:03:34.537824+0000","last_clean":"2023-12-20T06:03:34.537824+0000","last_became_active":"2023-12-20T06:03:21.557644+0000","last_became_peered":"2023-12-20T06:03:21.557644+0000","last_unstale":"2023-12-20T06:03:34.537824+0000","last_undegraded":"2023-12-20T06:03:34.537824+0000","last_fullsized":"2023-12-20T06:03:34.537824+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:48.619779+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"31.2","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571077+0000","last_change":"2023-12-20T06:03:34.542787+0000","last_active":"2023-12-20T06:03:35.571077+0000","last_peered":"2023-12-20T06:03:35.571077+0000","last_clean":"2023-12-20T06:03:35.571077+0000","last_became_active":"2023-12-20T06:03:34.542503+0000","last_became_peered":"2023-12-20T06:03:34.542503+0000","last_unstale":"2023-12-20T06:03:35.571077+0000","last_undegraded":"2023-12-20T06:03:35.571077+0000","last_fullsized":"2023-12-20T06:03:35.571077+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:37:53.065801+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"30.3","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.571301+0000","last_change":"2023-12-20T06:03:34.137376+0000","last_active":"2023-12-20T06:03:35.571301+0000","last_peered":"2023-12-20T06:03:35.571301+0000","last_clean":"2023-12-20T06:03:35.571301+0000","last_became_active":"2023-12-20T06:03:34.137073+0000","last_became_peered":"2023-12-20T06:03:34.137073+0000","last_unstale":"2023-12-20T06:03:35.571301+0000","last_undegraded":"2023-12-20T06:03:35.571301+0000","last_fullsized":"2023-12-20T06:03:35.571301+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:57:18.842438+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"29.0","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529301+0000","last_change":"2023-12-20T06:03:21.986446+0000","last_active":"2023-12-20T06:03:34.529301+0000","last_peered":"2023-12-20T06:03:34.529301+0000","last_clean":"2023-12-20T06:03:34.529301+0000","last_became_active":"2023-12-20T06:03:21.986120+0000","last_became_peered":"2023-12-20T06:03:21.986120+0000","last_unstale":"2023-12-20T06:03:34.529301+0000","last_undegraded":"2023-12-20T06:03:34.529301+0000","last_fullsized":"2023-12-20T06:03:34.529301+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:27:59.998522+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"28.1","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.534454+0000","last_change":"2023-12-20T06:03:21.570337+0000","last_active":"2023-12-20T06:03:34.534454+0000","last_peered":"2023-12-20T06:03:34.534454+0000","last_clean":"2023-12-20T06:03:34.534454+0000","last_became_active":"2023-12-20T06:03:21.569986+0000","last_became_peered":"2023-12-20T06:03:21.569986+0000","last_unstale":"2023-12-20T06:03:34.534454+0000","last_undegraded":"2023-12-20T06:03:34.534454+0000","last_fullsized":"2023-12-20T06:03:34.534454+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:37:28.719459+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"31.3","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.536980+0000","last_change":"2023-12-20T06:03:34.544566+0000","last_active":"2023-12-20T06:03:35.536980+0000","last_peered":"2023-12-20T06:03:35.536980+0000","last_clean":"2023-12-20T06:03:35.536980+0000","last_became_active":"2023-12-20T06:03:34.544303+0000","last_became_peered":"2023-12-20T06:03:34.544303+0000","last_unstale":"2023-12-20T06:03:35.536980+0000","last_undegraded":"2023-12-20T06:03:35.536980+0000","last_fullsized":"2023-12-20T06:03:35.536980+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:29:03.073420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"30.2","version":"0'0","reported_seq":11,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.573068+0000","last_change":"2023-12-20T06:03:34.135212+0000","last_active":"2023-12-20T06:03:35.573068+0000","last_peered":"2023-12-20T06:03:35.573068+0000","last_clean":"2023-12-20T06:03:35.573068+0000","last_became_active":"2023-12-20T06:03:34.134875+0000","last_became_peered":"2023-12-20T06:03:34.134875+0000","last_unstale":"2023-12-20T06:03:35.573068+0000","last_undegraded":"2023-12-20T06:03:35.573068+0000","last_fullsized":"2023-12-20T06:03:35.573068+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:19:24.762096+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"29.1","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.537453+0000","last_change":"2023-12-20T06:03:21.968100+0000","last_active":"2023-12-20T06:03:34.537453+0000","last_peered":"2023-12-20T06:03:34.537453+0000","last_clean":"2023-12-20T06:03:34.537453+0000","last_became_active":"2023-12-20T06:03:21.967593+0000","last_became_peered":"2023-12-20T06:03:21.967593+0000","last_unstale":"2023-12-20T06:03:34.537453+0000","last_undegraded":"2023-12-20T06:03:34.537453+0000","last_fullsized":"2023-12-20T06:03:34.537453+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:00:01.717868+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4],"acting":[7,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.0","version":"0'0","reported_seq":15,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.529497+0000","last_change":"2023-12-20T06:03:21.568281+0000","last_active":"2023-12-20T06:03:34.529497+0000","last_peered":"2023-12-20T06:03:34.529497+0000","last_clean":"2023-12-20T06:03:34.529497+0000","last_became_active":"2023-12-20T06:03:21.567976+0000","last_became_peered":"2023-12-20T06:03:21.567976+0000","last_unstale":"2023-12-20T06:03:34.529497+0000","last_undegraded":"2023-12-20T06:03:34.529497+0000","last_fullsized":"2023-12-20T06:03:34.529497+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:03:18.004400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"31.1e","version":"0'0","reported_seq":10,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.535774+0000","last_change":"2023-12-20T06:03:34.546321+0000","last_active":"2023-12-20T06:03:35.535774+0000","last_peered":"2023-12-20T06:03:35.535774+0000","last_clean":"2023-12-20T06:03:35.535774+0000","last_became_active":"2023-12-20T06:03:34.545968+0000","last_became_peered":"2023-12-20T06:03:34.545968+0000","last_unstale":"2023-12-20T06:03:35.535774+0000","last_undegraded":"2023-12-20T06:03:35.535774+0000","last_fullsized":"2023-12-20T06:03:35.535774+0000","mapping_epoch":232,"log_start":"0'0","ondisk_log_start":"0'0","created":232,"last_epoch_clean":233,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:34.091164+0000","last_clean_scrub_stamp":"2023-12-20T06:03:34.091164+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:14:15.201659+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"30.1f","version":"233'4","reported_seq":15,"reported_epoch":234,"state":"active+clean","last_fresh":"2023-12-20T06:03:35.574549+0000","last_change":"2023-12-20T06:03:34.132816+0000","last_active":"2023-12-20T06:03:35.574549+0000","last_peered":"2023-12-20T06:03:35.574549+0000","last_clean":"2023-12-20T06:03:35.574549+0000","last_became_active":"2023-12-20T06:03:34.132457+0000","last_became_peered":"2023-12-20T06:03:34.132457+0000","last_unstale":"2023-12-20T06:03:35.574549+0000","last_undegraded":"2023-12-20T06:03:35.574549+0000","last_fullsized":"2023-12-20T06:03:35.574549+0000","mapping_epoch":231,"log_start":"0'0","ondisk_log_start":"0'0","created":231,"last_epoch_clean":232,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:33.069696+0000","last_clean_scrub_stamp":"2023-12-20T06:03:33.069696+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:44:57.468706+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"29.1c","version":"0'0","reported_seq":14,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.539944+0000","last_change":"2023-12-20T06:03:21.973527+0000","last_active":"2023-12-20T06:03:34.539944+0000","last_peered":"2023-12-20T06:03:34.539944+0000","last_clean":"2023-12-20T06:03:34.539944+0000","last_became_active":"2023-12-20T06:03:21.973178+0000","last_became_peered":"2023-12-20T06:03:21.973178+0000","last_unstale":"2023-12-20T06:03:34.539944+0000","last_undegraded":"2023-12-20T06:03:34.539944+0000","last_fullsized":"2023-12-20T06:03:34.539944+0000","mapping_epoch":228,"log_start":"0'0","ondisk_log_start":"0'0","created":228,"last_epoch_clean":229,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:21.521510+0000","last_clean_scrub_stamp":"2023-12-20T06:03:21.521510+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:55:50.794593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3],"acting":[7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"28.1d","version":"229'2","reported_seq":17,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.528875+0000","last_change":"2023-12-20T06:03:21.567717+0000","last_active":"2023-12-20T06:03:34.528875+0000","last_peered":"2023-12-20T06:03:34.528875+0000","last_clean":"2023-12-20T06:03:34.528875+0000","last_became_active":"2023-12-20T06:03:21.567425+0000","last_became_peered":"2023-12-20T06:03:21.567425+0000","last_unstale":"2023-12-20T06:03:34.528875+0000","last_undegraded":"2023-12-20T06:03:34.528875+0000","last_fullsized":"2023-12-20T06:03:34.528875+0000","mapping_epoch":227,"log_start":"0'0","ondisk_log_start":"0'0","created":227,"last_epoch_clean":228,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:03:20.504307+0000","last_clean_scrub_stamp":"2023-12-20T06:03:20.504307+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:42:45.186933+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"1.0","version":"215'3223","reported_seq":4093,"reported_epoch":233,"state":"active+clean","last_fresh":"2023-12-20T06:03:34.535519+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:03:34.535519+0000","last_peered":"2023-12-20T06:03:34.535519+0000","last_clean":"2023-12-20T06:03:34.535519+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:03:34.535519+0000","last_undegraded":"2023-12-20T06:03:34.535519+0000","last_fullsized":"2023-12-20T06:03:34.535519+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3223,"log_dups_size":0,"ondisk_log_size":3223,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":561,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":31,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":30,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":29,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":28,"num_pg":32,"stat_sum":{"num_bytes":3603,"num_objects":22,"num_object_clones":0,"num_object_copies":44,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":22,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":47,"num_write_kb":16,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":7206,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":41,"ondisk_log_size":41,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2077,"num_read_kb":4261,"num_write":561,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3223,"ondisk_log_size":3223,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575982,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60736,"kb_used_data":3448,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311104,"statfs":{"total":96636764160,"available":96574570496,"internally_reserved":0,"allocated":3530752,"data_stored":2024165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.69599999999999995,"15min":0.72799999999999998},"min":{"1min":0.45400000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":1.028,"15min":1.042},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.99099999999999999,"15min":1.161},"last":0.51100000000000001}]},{"osd":1,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71499999999999997,"15min":0.74199999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":0.996,"15min":1.1279999999999999},"last":0.48299999999999998},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.71599999999999997,"15min":0.745},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.85499999999999998,"5min":0.96699999999999997,"15min":1.032},"last":0.63}]},{"osd":2,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.70599999999999996,"15min":0.73899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84499999999999997,"5min":0.95399999999999996,"15min":1.014},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.49099999999999999},"max":{"1min":0.88700000000000001,"5min":1.0089999999999999,"15min":1.0189999999999999},"last":0.56100000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.47699999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.85299999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.64800000000000002},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.89900000000000002,"5min":1.0660000000000001,"15min":1.0840000000000001},"last":0.61099999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.04},"last":0.40000000000000002},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.02,"15min":1.02},"last":0.58999999999999997}]},{"osd":5,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68300000000000005,"15min":0.66800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.052},"last":0.65800000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.875,"5min":1.0229999999999999,"15min":1.103},"last":0.66800000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76300000000000001,"15min":0.746},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.57499999999999996},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72399999999999998,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93500000000000005,"5min":1.0740000000000001,"15min":1.081},"last":0.68300000000000005}]}]},{"osd":6,"up_from":13,"seq":55834575982,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51972,"kb_used_data":2876,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319868,"statfs":{"total":96636764160,"available":96583544832,"internally_reserved":0,"allocated":2945024,"data_stored":1434317,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.68600000000000005,"5min":0.71199999999999997,"15min":0.73999999999999999},"min":{"1min":0.45100000000000001,"5min":0.35799999999999998,"15min":0.35799999999999998},"max":{"1min":0.92400000000000004,"5min":1.016,"15min":1.0980000000000001},"last":0.63},{"interface":"front","average":{"1min":0.73599999999999999,"5min":0.72999999999999998,"15min":0.75},"min":{"1min":0.48199999999999998,"5min":0.376,"15min":0.376},"max":{"1min":1.006,"5min":1.016,"15min":1.05},"last":0.76200000000000001}]},{"osd":1,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.72699999999999998,"5min":0.72899999999999998,"15min":0.73699999999999999},"min":{"1min":0.47399999999999998,"5min":0.40100000000000002,"15min":0.40100000000000002},"max":{"1min":0.90100000000000002,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.54200000000000004},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.75600000000000001,"15min":0.76800000000000002},"min":{"1min":0.50900000000000001,"5min":0.42099999999999999,"15min":0.42099999999999999},"max":{"1min":0.91200000000000003,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.81200000000000006}]},{"osd":2,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.746,"5min":0.76900000000000002,"15min":0.78100000000000003},"min":{"1min":0.53800000000000003,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.97699999999999998,"5min":1.069,"15min":1.069},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.73899999999999999,"15min":0.76300000000000001},"min":{"1min":0.54400000000000004,"5min":0.441,"15min":0.441},"max":{"1min":0.93100000000000005,"5min":0.97799999999999998,"15min":1.0760000000000001},"last":0.57599999999999996}]},{"osd":3,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.78200000000000003,"5min":0.78200000000000003,"15min":0.78900000000000003},"min":{"1min":0.61299999999999999,"5min":0.51700000000000002,"15min":0.46500000000000002},"max":{"1min":1.0109999999999999,"5min":1.077,"15min":1.077},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.72499999999999998,"5min":0.76200000000000001,"15min":0.76300000000000001},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.997,"5min":1.052,"15min":1.052},"last":0.66600000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.63400000000000001,"15min":0.63300000000000001},"min":{"1min":0.46000000000000002,"5min":0.40200000000000002,"15min":0.36099999999999999},"max":{"1min":0.96299999999999997,"5min":1.05,"15min":1.0529999999999999},"last":0.751},{"interface":"front","average":{"1min":0.70799999999999996,"5min":0.66400000000000003,"15min":0.65100000000000002},"min":{"1min":0.498,"5min":0.41999999999999998,"15min":0.40200000000000002},"max":{"1min":0.98399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.82099999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.67800000000000005,"5min":0.64900000000000002,"15min":0.64000000000000001},"min":{"1min":0.44500000000000001,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":0.98699999999999999,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.59999999999999998},{"interface":"front","average":{"1min":0.72299999999999998,"5min":0.70799999999999996,"15min":0.70299999999999996},"min":{"1min":0.47599999999999998,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":1.0209999999999999,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:02:35 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.73899999999999999,"15min":0.73299999999999998},"min":{"1min":0.55700000000000005,"5min":0.48899999999999999,"15min":0.45300000000000001},"max":{"1min":1.0429999999999999,"5min":1.103,"15min":1.1240000000000001},"last":0.79500000000000004},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.72499999999999998,"15min":0.72899999999999998},"min":{"1min":0.47199999999999998,"5min":0.47199999999999998,"15min":0.41999999999999998},"max":{"1min":1.0580000000000001,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.73399999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575983,"num_pgs":38,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51956,"kb_used_data":2860,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319884,"statfs":{"total":96636764160,"available":96583561216,"internally_reserved":0,"allocated":2928640,"data_stored":1433657,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71399999999999997,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91600000000000004,"5min":0.94799999999999995,"15min":1.077},"last":0.63700000000000001},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71499999999999997,"15min":0.746},"min":{"1min":0.45600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.80300000000000005,"5min":0.96699999999999997,"15min":1.028},"last":0.45600000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":0.875,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.47799999999999998},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88500000000000001,"5min":0.98799999999999999,"15min":1.0209999999999999},"last":0.66600000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.751},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89000000000000001,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.61099999999999999},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.75700000000000001,"15min":0.77000000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.48299999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0680000000000001},"last":0.53400000000000003}]},{"osd":3,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"max":{"1min":0.91000000000000003,"5min":1.05,"15min":1.0669999999999999},"last":0.624},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.0629999999999999},"last":0.51600000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.56000000000000005,"5min":0.42199999999999999,"15min":0.38200000000000001},"max":{"1min":0.94399999999999995,"5min":1.0640000000000001,"15min":1.077},"last":0.56799999999999995},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.64800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"max":{"1min":1.0649999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.50600000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.0449999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.59299999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73799999999999999,"15min":0.70999999999999996},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.153,"15min":1.153},"last":0.55300000000000005},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.48299999999999998},"max":{"1min":0.98799999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.68600000000000005}]}]},{"osd":4,"up_from":13,"seq":55834575984,"num_pgs":43,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47884,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323956,"statfs":{"total":96636764160,"available":96587730944,"internally_reserved":0,"allocated":2953216,"data_stored":1435895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.60499999999999998,"5min":0.69299999999999995,"15min":0.72899999999999998},"min":{"1min":0.36199999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.96499999999999997,"5min":0.98999999999999999,"15min":1.0209999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.61199999999999999,"5min":0.69299999999999995,"15min":0.72299999999999998},"min":{"1min":0.432,"5min":0.432,"15min":0.432},"max":{"1min":0.80900000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.70799999999999996}]},{"osd":1,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.73099999999999998,"15min":0.752},"min":{"1min":0.498,"5min":0.48999999999999999,"15min":0.45100000000000001},"max":{"1min":0.89600000000000002,"5min":0.95699999999999996,"15min":1.0900000000000001},"last":0.73399999999999999},{"interface":"front","average":{"1min":0.69499999999999995,"5min":0.73899999999999999,"15min":0.76400000000000001},"min":{"1min":0.47899999999999998,"5min":0.47399999999999998,"15min":0.44500000000000001},"max":{"1min":1.054,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.56799999999999995}]},{"osd":2,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69399999999999995,"5min":0.746,"15min":0.76900000000000002},"min":{"1min":0.46200000000000002,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":1.0029999999999999,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.754,"15min":0.77700000000000002},"min":{"1min":0.41099999999999998,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.98199999999999998,"5min":1.0089999999999999,"15min":1.101},"last":0.68100000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.67500000000000004,"5min":0.72299999999999998,"15min":0.75},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.49199999999999999},"max":{"1min":0.88,"5min":1.014,"15min":1.014},"last":0.46300000000000002},{"interface":"front","average":{"1min":0.71799999999999997,"5min":0.77200000000000002,"15min":0.80100000000000005},"min":{"1min":0.44700000000000001,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":1,"5min":1.04,"15min":1.081},"last":0.59499999999999997}]},{"osd":5,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.69999999999999996,"5min":0.64700000000000002,"15min":0.65000000000000002},"min":{"1min":0.38600000000000001,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":0.98099999999999998,"5min":1.05,"15min":1.05},"last":0.754},{"interface":"front","average":{"1min":0.68400000000000005,"5min":0.66700000000000004,"15min":0.66200000000000003},"min":{"1min":0.47599999999999998,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.96999999999999997,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.54400000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.67900000000000005,"15min":0.67700000000000005},"min":{"1min":0.44400000000000001,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0569999999999999,"5min":1.0569999999999999,"15min":1.1379999999999999},"last":0.66000000000000003},{"interface":"front","average":{"1min":0.78000000000000003,"5min":0.73399999999999999,"15min":0.73499999999999999},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.41199999999999998},"max":{"1min":1.056,"5min":1.107,"15min":1.111},"last":0.80000000000000004}]},{"osd":7,"last update":"Wed Dec 20 06:03:01 2023","interfaces":[{"interface":"back","average":{"1min":0.73199999999999998,"5min":0.72899999999999998,"15min":0.72999999999999998},"min":{"1min":0.54100000000000004,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":1.0329999999999999,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.63100000000000001},{"interface":"front","average":{"1min":0.81000000000000005,"5min":0.76800000000000002,"15min":0.78700000000000003},"min":{"1min":0.626,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.1100000000000001,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.77000000000000002}]}]},{"osd":3,"up_from":13,"seq":55834575983,"num_pgs":22,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":47880,"kb_used_data":2880,"kb_used_omap":0,"kb_used_meta":44992,"kb_avail":94323960,"statfs":{"total":96636764160,"available":96587735040,"internally_reserved":0,"allocated":2949120,"data_stored":1437114,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":46071808},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.46899999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.748,"15min":2.8130000000000002},"last":0.35799999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.49299999999999999,"15min":0.52600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":1.357},"last":0.318}]},{"osd":1,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45700000000000002,"15min":0.47599999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65600000000000003,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.41599999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53000000000000003,"15min":0.56699999999999995},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.57599999999999996,"5min":0.81799999999999995,"15min":1.4219999999999999},"last":0.39700000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.56399999999999995,"15min":0.58999999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.57999999999999996,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.46800000000000003},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.53800000000000003,"15min":0.56999999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69099999999999995,"5min":0.86099999999999999,"15min":2.944},"last":0.436}]},{"osd":4,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77300000000000002,"15min":0.81499999999999995},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.98999999999999999,"5min":1.2090000000000001,"15min":1.669},"last":0.82299999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75900000000000001,"15min":0.80800000000000005},"min":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.86599999999999999,"5min":1.1299999999999999,"15min":1.99},"last":0.69199999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79600000000000004,"15min":0.82899999999999996},"min":{"1min":0.65100000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.93600000000000005,"5min":1.22,"15min":1.901},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.753,"5min":0.76300000000000001,"15min":0.81999999999999995},"min":{"1min":0.58799999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.875,"5min":1.196,"15min":2.8639999999999999},"last":0.69999999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.77500000000000002,"15min":0.81399999999999995},"min":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95599999999999996,"5min":1.194,"15min":1.575},"last":0.86599999999999999},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80500000000000005,"15min":0.84499999999999997},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0580000000000001,"5min":1.155,"15min":1.948},"last":0.79300000000000004}]},{"osd":7,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91900000000000004,"5min":1.1579999999999999,"15min":2.9049999999999998},"last":0.80500000000000005},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80700000000000005,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.502,"15min":0.502},"max":{"1min":1.0680000000000001,"5min":1.1850000000000001,"15min":1.754},"last":0.89100000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575982,"num_pgs":47,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":51980,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319860,"statfs":{"total":96636764160,"available":96583536640,"internally_reserved":0,"allocated":2953216,"data_stored":1436608,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.45000000000000001,"15min":0.45500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52900000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.36399999999999999},{"interface":"front","average":{"1min":0.438,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.60699999999999998,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.49299999999999999}]},{"osd":1,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.79800000000000004,"15min":0.82199999999999995},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30499999999999999},"max":{"1min":0.61199999999999999,"5min":0.92600000000000005,"15min":1.004},"last":0.51200000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.65500000000000003,"15min":0.67400000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.59199999999999997,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.53900000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":1.254},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77900000000000003,"15min":0.79600000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":1.605,"15min":1.605},"last":0.67200000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97099999999999997,"5min":1.329,"15min":1.329},"last":0.80800000000000005},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98199999999999998,"5min":1.0740000000000001,"15min":1.095},"last":0.76100000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92200000000000004,"5min":1.919,"15min":1.919},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.95099999999999996,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.93200000000000005}]},{"osd":7,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0049999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":0.75},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.88200000000000001,"15min":0.88500000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.996,"5min":1.292,"15min":1.292},"last":0.98199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575984,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60172,"kb_used_data":2884,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311668,"statfs":{"total":96636764160,"available":96575148032,"internally_reserved":0,"allocated":2953216,"data_stored":1435435,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":7,"apply_latency_ms":7,"commit_latency_ns":7000000,"apply_latency_ns":7000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.44800000000000001,"5min":0.47899999999999998,"15min":0.46200000000000002},"min":{"1min":0.26500000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.59299999999999997,"5min":0.746,"15min":0.81000000000000005},"last":0.25800000000000001},{"interface":"front","average":{"1min":0.51600000000000001,"5min":0.50900000000000001,"15min":0.51400000000000001},"min":{"1min":0.29399999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.88200000000000001,"5min":0.88200000000000001,"15min":0.88200000000000001},"last":0.28599999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.52400000000000002,"5min":0.53000000000000003,"15min":0.53100000000000003},"min":{"1min":0.31,"5min":0.31,"15min":0.31},"max":{"1min":0.71999999999999997,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.36499999999999999},{"interface":"front","average":{"1min":0.55100000000000005,"5min":0.56899999999999995,"15min":0.57399999999999995},"min":{"1min":0.38100000000000001,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":0.79700000000000004,"5min":0.88800000000000001,"15min":0.93700000000000006},"last":0.45500000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.628,"5min":0.61399999999999999,"15min":0.61599999999999999},"min":{"1min":0.42999999999999999,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.90100000000000002},"last":0.38300000000000001},{"interface":"front","average":{"1min":0.53300000000000003,"5min":0.55400000000000005,"15min":0.55300000000000005},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.78300000000000003,"5min":0.88700000000000001,"15min":0.93500000000000005},"last":0.44}]},{"osd":4,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.76600000000000001,"15min":0.78500000000000003},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.92000000000000004,"5min":1.107,"15min":1.321},"last":0.875},{"interface":"front","average":{"1min":0.78500000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.57999999999999996,"5min":0.57399999999999995,"15min":0.57399999999999995},"max":{"1min":0.995,"5min":1.139,"15min":1.25},"last":0.79300000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.78700000000000003,"15min":0.80200000000000005},"min":{"1min":0.46000000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":0.97899999999999998,"5min":1.224,"15min":1.2849999999999999},"last":0.78400000000000003},{"interface":"front","average":{"1min":0.746,"5min":0.78500000000000003,"15min":0.80700000000000005},"min":{"1min":0.41499999999999998,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.0740000000000001,"5min":1.1990000000000001,"15min":1.4330000000000001},"last":0.95299999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.80000000000000004,"15min":0.82399999999999995},"min":{"1min":0.57299999999999995,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":1.0389999999999999,"5min":1.2430000000000001,"15min":1.341},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.77900000000000003,"5min":0.80500000000000005,"15min":0.81200000000000006},"min":{"1min":0.56399999999999995,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.1379999999999999,"5min":1.177,"15min":1.3839999999999999},"last":0.82699999999999996}]},{"osd":7,"last update":"Wed Dec 20 06:02:41 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.84799999999999998,"15min":0.86399999999999999},"min":{"1min":0.60399999999999998,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.0900000000000001,"5min":1.272,"15min":1.4470000000000001},"last":0.80700000000000005},{"interface":"front","average":{"1min":0.81299999999999994,"5min":0.83799999999999997,"15min":0.86199999999999999},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.0509999999999999,"5min":1.1599999999999999,"15min":1.3839999999999999},"last":0.91700000000000004}]}]},{"osd":0,"up_from":13,"seq":55834575983,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56644,"kb_used_data":3452,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94315196,"statfs":{"total":96636764160,"available":96578760704,"internally_reserved":0,"allocated":3534848,"data_stored":2024731,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46800000000000003,"15min":0.46600000000000003},"min":{"1min":0.26600000000000001,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.998,"5min":0.998,"15min":0.998},"last":0.318},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.56899999999999995,"15min":0.55900000000000005},"min":{"1min":0.34799999999999998,"5min":0.33200000000000002,"15min":0.33000000000000002},"max":{"1min":3.681,"5min":3.681,"15min":3.681},"last":0.40300000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.54700000000000004,"15min":0.53800000000000003},"min":{"1min":0.35199999999999998,"5min":0.35199999999999998,"15min":0.35199999999999998},"max":{"1min":1.5209999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.41899999999999998},{"interface":"front","average":{"1min":0.68700000000000006,"5min":0.59699999999999998,"15min":0.59499999999999997},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":3.7290000000000001,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.46999999999999997}]},{"osd":3,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.56799999999999995,"5min":0.55300000000000005,"15min":0.56100000000000005},"min":{"1min":0.32300000000000001,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":1.6899999999999999,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.49399999999999999},{"interface":"front","average":{"1min":0.64100000000000001,"5min":0.627,"15min":0.628},"min":{"1min":0.40999999999999998,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":2.3660000000000001,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.60299999999999998}]},{"osd":4,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.91000000000000003,"5min":0.82999999999999996,"15min":0.82599999999999996},"min":{"1min":0.56699999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":3.1480000000000001,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.82099999999999995,"5min":0.82499999999999996,"15min":0.83399999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"max":{"1min":1.5549999999999999,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":1.081}]},{"osd":5,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0429999999999999,"5min":0.86099999999999999,"15min":0.85099999999999998},"min":{"1min":0.54100000000000004,"5min":0.54100000000000004,"15min":0.54100000000000004},"max":{"1min":5.6769999999999996,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.85199999999999998},{"interface":"front","average":{"1min":0.873,"5min":0.83399999999999996,"15min":0.84799999999999998},"min":{"1min":0.54600000000000004,"5min":0.495,"15min":0.495},"max":{"1min":2.23,"5min":2.23,"15min":2.23},"last":1.1220000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":0.81899999999999995,"5min":0.82599999999999996,"15min":0.84199999999999997},"min":{"1min":0.55500000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.482,"5min":1.482,"15min":1.482},"last":1.032},{"interface":"front","average":{"1min":1.0740000000000001,"5min":0.89100000000000001,"15min":0.871},"min":{"1min":0.58499999999999996,"5min":0.58499999999999996,"15min":0.57099999999999995},"max":{"1min":5.6310000000000002,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.98599999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:03:12 2023","interfaces":[{"interface":"back","average":{"1min":1.0600000000000001,"5min":0.88900000000000001,"15min":0.879},"min":{"1min":0.53600000000000003,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":5.7160000000000002,"5min":5.7160000000000002,"15min":5.7160000000000002},"last":0.78700000000000003},{"interface":"front","average":{"1min":1.012,"5min":0.88200000000000001,"15min":0.871},"min":{"1min":0.52200000000000002,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":4.2119999999999997,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.94399999999999995}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":672,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2861,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2875,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":56,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":28,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":29,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":34,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1656,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":30,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":31,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:03:39.069 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T06:03:39.069 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.246+0000 7ff952e8c640 1 Processor -- start 2023-12-20T06:03:39.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.246+0000 7ff952e8c640 1 -- start start 2023-12-20T06:03:39.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff952e8c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff94c1599e0 0x7ff94c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:39.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff952e8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff94c0c90e0 0x7ff94c0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:39.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff952e8c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff94c0711d0 con 0x7ff94c0b0b30 2023-12-20T06:03:39.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff952e8c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94c0c98c0 con 0x7ff94c1599e0 2023-12-20T06:03:39.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff952e8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff94c0c9ba0 con 0x7ff94c0c90e0 2023-12-20T06:03:39.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff950c28640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff94c1599e0 0x7ff94c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff950c28640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff94c1599e0 0x7ff94c0aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35594/0 (socket says 172.21.15.32:35594) 2023-12-20T06:03:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff950c28640 1 -- 172.21.15.32:0/1030165335 learned_addr learned my addr 172.21.15.32:0/1030165335 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff951429640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ff94c0b0b30 0x7ff94c15d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47050/0 (socket says 172.21.15.32:47050) 2023-12-20T06:03:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff94bfff640 1 --2- 172.21.15.32:0/1030165335 >> v2:172.21.15.39:3300/0 conn(0x7ff94c0c90e0 0x7ff94c0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff950c28640 1 -- 172.21.15.32:0/1030165335 >> v2:172.21.15.39:3300/0 conn(0x7ff94c0c90e0 msgr2=0x7ff94c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff950c28640 1 --2- 172.21.15.32:0/1030165335 >> v2:172.21.15.39:3300/0 conn(0x7ff94c0c90e0 0x7ff94c0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff950c28640 1 -- 172.21.15.32:0/1030165335 >> v1:172.21.15.32:6789/0 conn(0x7ff94c0b0b30 legacy=0x7ff94c15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff950c28640 1 -- 172.21.15.32:0/1030165335 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff94c0c9e80 con 0x7ff94c1599e0 2023-12-20T06:03:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff94bfff640 1 --2- 172.21.15.32:0/1030165335 >> v2:172.21.15.39:3300/0 conn(0x7ff94c0c90e0 0x7ff94c0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:39.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.250+0000 7ff94b7fe640 1 -- 172.21.15.32:0/1030165335 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3346887240 0 0) 0x7ff94c0711d0 con 0x7ff94c0b0b30 2023-12-20T06:03:39.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff950c28640 1 --2- 172.21.15.32:0/1030165335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c1599e0 0x7ff94c0aece0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7ff94c06ab20 tx=0x7ff94001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d16bdf5ebc920c8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:39.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff94b7fe640 1 -- 172.21.15.32:0/1030165335 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400ab5f0 con 0x7ff94c1599e0 2023-12-20T06:03:39.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff94b7fe640 1 -- 172.21.15.32:0/1030165335 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff9400ab750 con 0x7ff94c1599e0 2023-12-20T06:03:39.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff94b7fe640 1 -- 172.21.15.32:0/1030165335 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff9400abae0 con 0x7ff94c1599e0 2023-12-20T06:03:39.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff952e8c640 1 -- 172.21.15.32:0/1030165335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c1599e0 msgr2=0x7ff94c0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:39.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff952e8c640 1 --2- 172.21.15.32:0/1030165335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c1599e0 0x7ff94c0aece0 secure :-1 s=READY pgs=2306 cs=0 l=1 rev1=1 crypto rx=0x7ff94c06ab20 tx=0x7ff94001c4d0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff952e8c640 1 -- 172.21.15.32:0/1030165335 shutdown_connections 2023-12-20T06:03:39.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff952e8c640 1 --2- 172.21.15.32:0/1030165335 >> v2:172.21.15.39:3300/0 conn(0x7ff94c0c90e0 0x7ff94c0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff952e8c640 1 --2- 172.21.15.32:0/1030165335 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c1599e0 0x7ff94c0aece0 unknown :-1 s=CLOSED pgs=2306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff952e8c640 1 -- 172.21.15.32:0/1030165335 >> 172.21.15.32:0/1030165335 conn(0x7ff94c06a320 msgr2=0x7ff94c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:39.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff952e8c640 1 -- 172.21.15.32:0/1030165335 shutdown_connections 2023-12-20T06:03:39.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff952e8c640 1 -- 172.21.15.32:0/1030165335 wait complete. 2023-12-20T06:03:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff952e8c640 1 Processor -- start 2023-12-20T06:03:39.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.254+0000 7ff952e8c640 1 -- start start 2023-12-20T06:03:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff952e8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff94c0b0b30 0x7ff94c15c900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff952e8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c0c90e0 0x7ff94c14cdc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff952e8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff94c1599e0 0x7ff94c14d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff952e8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c15b860 con 0x7ff94c0c90e0 2023-12-20T06:03:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff952e8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c15bb40 con 0x7ff94c1599e0 2023-12-20T06:03:39.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff952e8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff94c15be20 con 0x7ff94c0b0b30 2023-12-20T06:03:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff94bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c0c90e0 0x7ff94c14cdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff94bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c0c90e0 0x7ff94c14cdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35604/0 (socket says 172.21.15.32:35604) 2023-12-20T06:03:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff94bfff640 1 -- 172.21.15.32:0/3387853217 learned_addr learned my addr 172.21.15.32:0/3387853217 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff950c28640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff94c0b0b30 0x7ff94c15c900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff951429640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff94c1599e0 0x7ff94c14d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff94bfff640 1 -- 172.21.15.32:0/3387853217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff94c0b0b30 msgr2=0x7ff94c15c900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff94bfff640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff94c0b0b30 0x7ff94c15c900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff94bfff640 1 -- 172.21.15.32:0/3387853217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff94c1599e0 msgr2=0x7ff94c14d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff94bfff640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff94c1599e0 0x7ff94c14d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff94bfff640 1 -- 172.21.15.32:0/3387853217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff940009480 con 0x7ff94c0c90e0 2023-12-20T06:03:39.260 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff94bfff640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c0c90e0 0x7ff94c14cdc0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7ff93800e810 tx=0x7ff93800ecd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff9497fa640 1 -- 172.21.15.32:0/3387853217 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93800c8d0 con 0x7ff94c0c90e0 2023-12-20T06:03:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff9497fa640 1 -- 172.21.15.32:0/3387853217 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff93800ca30 con 0x7ff94c0c90e0 2023-12-20T06:03:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff9497fa640 1 -- 172.21.15.32:0/3387853217 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff93800cde0 con 0x7ff94c0c90e0 2023-12-20T06:03:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff952e8c640 1 -- 172.21.15.32:0/3387853217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff94c15c100 con 0x7ff94c0c90e0 2023-12-20T06:03:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff952e8c640 1 -- 172.21.15.32:0/3387853217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff94c15b480 con 0x7ff94c0c90e0 2023-12-20T06:03:39.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff952e8c640 1 -- 172.21.15.32:0/3387853217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff94c04e000 con 0x7ff94c0c90e0 2023-12-20T06:03:39.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.258+0000 7ff9497fa640 1 -- 172.21.15.32:0/3387853217 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff938013410 con 0x7ff94c0c90e0 2023-12-20T06:03:39.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.262+0000 7ff9497fa640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff930065ca0 0x7ff930068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:39.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.262+0000 7ff9497fa640 1 -- 172.21.15.32:0/3387853217 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7ff938083910 con 0x7ff94c0c90e0 2023-12-20T06:03:39.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.262+0000 7ff950c28640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff930065ca0 0x7ff930068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:39.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.262+0000 7ff950c28640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff930065ca0 0x7ff930068160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9400af000 tx=0x7ff9400a7000 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:39.264 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.266+0000 7ff9497fa640 1 -- 172.21.15.32:0/3387853217 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff938050a30 con 0x7ff94c0c90e0 2023-12-20T06:03:39.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.454+0000 7ff952e8c640 1 -- 172.21.15.32:0/3387853217 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7ff94c0b12a0 con 0x7ff94c0c90e0 2023-12-20T06:03:39.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.454+0000 7ff9497fa640 1 -- 172.21.15.32:0/3387853217 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5909 (secure 0 0 0) 0x7ff938054050 con 0x7ff94c0c90e0 2023-12-20T06:03:39.455 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:39.455 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:21.951240+0000","modified":"2023-12-20T06:03:24.980742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11301":{"gid":11301,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1623322556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1623322556},{"type":"v1","addr":"172.21.15.32:6837","nonce":1623322556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:34.513908+0000","modified":"2023-12-20T06:03:37.549965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11298":{"gid":11298,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6836/60134876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":60134876},{"type":"v1","addr":"172.21.15.32:6836","nonce":60134876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-20T06:03:39.456 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 118 2023-12-20T06:03:39.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.454+0000 7ff952e8c640 1 -- 172.21.15.32:0/3387853217 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff930065ca0 msgr2=0x7ff930068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:39.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.454+0000 7ff952e8c640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff930065ca0 0x7ff930068160 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7ff9400af000 tx=0x7ff9400a7000 comp rx=0 tx=0).stop 2023-12-20T06:03:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.458+0000 7ff952e8c640 1 -- 172.21.15.32:0/3387853217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c0c90e0 msgr2=0x7ff94c14cdc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.458+0000 7ff952e8c640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c0c90e0 0x7ff94c14cdc0 secure :-1 s=READY pgs=2307 cs=0 l=1 rev1=1 crypto rx=0x7ff93800e810 tx=0x7ff93800ecd0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.458+0000 7ff952e8c640 1 -- 172.21.15.32:0/3387853217 shutdown_connections 2023-12-20T06:03:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.458+0000 7ff952e8c640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff930065ca0 0x7ff930068160 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.458+0000 7ff952e8c640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff94c1599e0 0x7ff94c14d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.458+0000 7ff952e8c640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff94c0c90e0 0x7ff94c14cdc0 unknown :-1 s=CLOSED pgs=2307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.458+0000 7ff952e8c640 1 --2- 172.21.15.32:0/3387853217 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff94c0b0b30 0x7ff94c15c900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.458+0000 7ff952e8c640 1 -- 172.21.15.32:0/3387853217 >> 172.21.15.32:0/3387853217 conn(0x7ff94c06a320 msgr2=0x7ff94c0b2320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.458+0000 7ff952e8c640 1 -- 172.21.15.32:0/3387853217 shutdown_connections 2023-12-20T06:03:39.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:39.458+0000 7ff952e8c640 1 -- 172.21.15.32:0/3387853217 wait complete. 2023-12-20T06:03:39.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 118, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T06:03:34.513908+0000', 'modified': '2023-12-20T06:03:37.549965+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 11298}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_11298': {'gid': 11298, 'name': 'c', 'rank': 0, 'incarnation': 114, 'state': 'up:active', 'state_seq': 6, 'addr': '172.21.15.32:6836/60134876', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6834', 'nonce': 60134876}, {'type': 'v1', 'addr': '172.21.15.32:6836', 'nonce': 60134876}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [31], 'metadata_pool': 30, 'enabled': True, 'fs_name': 'backup_fs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T06:03:39.879 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T06:03:39.880 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.c status 2023-12-20T06:03:40.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.026+0000 7f12e497b640 1 Processor -- start 2023-12-20T06:03:40.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.026+0000 7f12e497b640 1 -- start start 2023-12-20T06:03:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.026+0000 7f12e497b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f12e00b0d00 0x7f12e0164a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.026+0000 7f12e497b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e00711d0 con 0x7f12e00aece0 2023-12-20T06:03:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.026+0000 7f12e497b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f12e00c98c0 con 0x7f12e01599e0 2023-12-20T06:03:40.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.026+0000 7f12e497b640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f12e00c9ba0 con 0x7f12e00b0d00 2023-12-20T06:03:40.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.026+0000 7f12ded9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f12e00b0d00 0x7f12e0164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.030+0000 7f12ded9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f12e00b0d00 0x7f12e0164a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35614/0 (socket says 172.21.15.32:35614) 2023-12-20T06:03:40.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.030+0000 7f12ded9d640 1 -- 172.21.15.32:0/866192692 learned_addr learned my addr 172.21.15.32:0/866192692 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:40.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.030+0000 7f12ded9d640 1 -- 172.21.15.32:0/866192692 >> v1:172.21.15.39:6789/0 conn(0x7f12e01599e0 legacy=0x7f12e00c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.030+0000 7f12ded9d640 1 -- 172.21.15.32:0/866192692 >> v1:172.21.15.32:6789/0 conn(0x7f12e00aece0 legacy=0x7f12e00b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.030+0000 7f12ded9d640 1 -- 172.21.15.32:0/866192692 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12e00c9e80 con 0x7f12e00b0d00 2023-12-20T06:03:40.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.030+0000 7f12ddd9b640 1 -- 172.21.15.32:0/866192692 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2206441592 0 0) 0x7f12e00711d0 con 0x7f12e00aece0 2023-12-20T06:03:40.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.030+0000 7f12ded9d640 1 --2- 172.21.15.32:0/866192692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e00b0d00 0x7f12e0164a70 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009e50 tx=0x7f12d801ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7920200773aedd28 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.030+0000 7f12ddd9b640 1 -- 172.21.15.32:0/866192692 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80a57d0 con 0x7f12e00b0d00 2023-12-20T06:03:40.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.030+0000 7f12ddd9b640 1 -- 172.21.15.32:0/866192692 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d80a5930 con 0x7f12e00b0d00 2023-12-20T06:03:40.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.030+0000 7f12ddd9b640 1 -- 172.21.15.32:0/866192692 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80a5cc0 con 0x7f12e00b0d00 2023-12-20T06:03:40.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 -- 172.21.15.32:0/866192692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e00b0d00 msgr2=0x7f12e0164a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 --2- 172.21.15.32:0/866192692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e00b0d00 0x7f12e0164a70 secure :-1 s=READY pgs=2309 cs=0 l=1 rev1=1 crypto rx=0x7f12d8009e50 tx=0x7f12d801ca10 comp rx=0 tx=0).stop 2023-12-20T06:03:40.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 -- 172.21.15.32:0/866192692 shutdown_connections 2023-12-20T06:03:40.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 --2- 172.21.15.32:0/866192692 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e00b0d00 0x7f12e0164a70 unknown :-1 s=CLOSED pgs=2309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 -- 172.21.15.32:0/866192692 >> 172.21.15.32:0/866192692 conn(0x7f12e006a320 msgr2=0x7f12e00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:40.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 -- 172.21.15.32:0/866192692 shutdown_connections 2023-12-20T06:03:40.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 -- 172.21.15.32:0/866192692 wait complete. 2023-12-20T06:03:40.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 Processor -- start 2023-12-20T06:03:40.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 -- start start 2023-12-20T06:03:40.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f12e00aece0 0x7f12e00cd070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e00b0d00 0x7f12e00c6e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e01599e0 0x7f12e00c73d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00ce260 con 0x7f12e01599e0 2023-12-20T06:03:40.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00c4520 con 0x7f12e00b0d00 2023-12-20T06:03:40.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12e497b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f12e00cd650 con 0x7f12e00aece0 2023-12-20T06:03:40.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12ded9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e01599e0 0x7f12e00c73d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12ded9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e01599e0 0x7f12e00c73d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35622/0 (socket says 172.21.15.32:35622) 2023-12-20T06:03:40.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12ded9d640 1 -- 172.21.15.32:0/2837245273 learned_addr learned my addr 172.21.15.32:0/2837245273 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:40.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12d7fff640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e00b0d00 0x7f12e00c6e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12de59c640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f12e00aece0 0x7f12e00cd070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12ded9d640 1 -- 172.21.15.32:0/2837245273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f12e00aece0 msgr2=0x7f12e00cd070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12ded9d640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f12e00aece0 0x7f12e00cd070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12ded9d640 1 -- 172.21.15.32:0/2837245273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e00b0d00 msgr2=0x7f12e00c6e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12ded9d640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e00b0d00 0x7f12e00c6e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12ded9d640 1 -- 172.21.15.32:0/2837245273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f12d8009b30 con 0x7f12e01599e0 2023-12-20T06:03:40.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12d7fff640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e00b0d00 0x7f12e00c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:40.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.034+0000 7f12de59c640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f12e00aece0 0x7f12e00cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:40.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.038+0000 7f12ded9d640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e01599e0 0x7f12e00c73d0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f12d80af040 tx=0x7f12d80197b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.038+0000 7f12d6ffd640 1 -- 172.21.15.32:0/2837245273 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80b8670 con 0x7f12e01599e0 2023-12-20T06:03:40.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.038+0000 7f12d6ffd640 1 -- 172.21.15.32:0/2837245273 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d80b87f0 con 0x7f12e01599e0 2023-12-20T06:03:40.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.038+0000 7f12d6ffd640 1 -- 172.21.15.32:0/2837245273 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d80b8ba0 con 0x7f12e01599e0 2023-12-20T06:03:40.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.038+0000 7f12e497b640 1 -- 172.21.15.32:0/2837245273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f12e00cd930 con 0x7f12e01599e0 2023-12-20T06:03:40.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.038+0000 7f12e497b640 1 -- 172.21.15.32:0/2837245273 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e00cdef0 con 0x7f12e01599e0 2023-12-20T06:03:40.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.038+0000 7f12d6ffd640 1 -- 172.21.15.32:0/2837245273 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f12d80b8d00 con 0x7f12e01599e0 2023-12-20T06:03:40.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.038+0000 7f12d6ffd640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f12c0065ca0 0x7f12c0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.038+0000 7f12d6ffd640 1 -- 172.21.15.32:0/2837245273 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f12d811f080 con 0x7f12e01599e0 2023-12-20T06:03:40.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.038+0000 7f12de59c640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f12c0065ca0 0x7f12c0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.038+0000 7f12e7cd6000 1 Processor -- start 2023-12-20T06:03:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 -- start start 2023-12-20T06:03:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12de59c640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f12c0065ca0 0x7f12c0068160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12c8000fc0 tx=0x7f12c8009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d8f070 0x559f47d8f450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f990 0x559f47d44c80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d451c0 0x559f47d455a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f47b68c90 con 0x559f47d8f990 2023-12-20T06:03:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f47b9a220 con 0x559f47d451c0 2023-12-20T06:03:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f47b977f0 con 0x559f47d8f070 2023-12-20T06:03:40.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f990 0x559f47d44c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f990 0x559f47d44c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35636/0 (socket says 172.21.15.32:35636) 2023-12-20T06:03:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12d7fff640 1 -- 172.21.15.32:0/4092695025 learned_addr learned my addr 172.21.15.32:0/4092695025 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12de59c640 1 --2- 172.21.15.32:0/4092695025 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d8f070 0x559f47d8f450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12ded9d640 1 --2- 172.21.15.32:0/4092695025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d451c0 0x559f47d455a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12d7fff640 1 -- 172.21.15.32:0/4092695025 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d8f070 msgr2=0x559f47d8f450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12d7fff640 1 --2- 172.21.15.32:0/4092695025 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d8f070 0x559f47d8f450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12d7fff640 1 -- 172.21.15.32:0/4092695025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d451c0 msgr2=0x559f47d455a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12d7fff640 1 --2- 172.21.15.32:0/4092695025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d451c0 0x559f47d455a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12d7fff640 1 -- 172.21.15.32:0/4092695025 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f4788f890 con 0x559f47d8f990 2023-12-20T06:03:40.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12ded9d640 1 --2- 172.21.15.32:0/4092695025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d451c0 0x559f47d455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12d7fff640 1 --2- 172.21.15.32:0/4092695025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f990 0x559f47d44c80 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f12d0001790 tx=0x7f12d0001c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79329a5eeda10106 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12bffff640 1 -- 172.21.15.32:0/4092695025 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d000b070 con 0x559f47d8f990 2023-12-20T06:03:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12bffff640 1 -- 172.21.15.32:0/4092695025 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12d000d730 con 0x559f47d8f990 2023-12-20T06:03:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12bffff640 1 -- 172.21.15.32:0/4092695025 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d00054e0 con 0x559f47d8f990 2023-12-20T06:03:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 -- 172.21.15.32:0/4092695025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f990 msgr2=0x559f47d44c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/4092695025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f990 0x559f47d44c80 secure :-1 s=READY pgs=2311 cs=0 l=1 rev1=1 crypto rx=0x7f12d0001790 tx=0x7f12d0001c50 comp rx=0 tx=0).stop 2023-12-20T06:03:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 -- 172.21.15.32:0/4092695025 shutdown_connections 2023-12-20T06:03:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/4092695025 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d451c0 0x559f47d455a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/4092695025 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f990 0x559f47d44c80 unknown :-1 s=CLOSED pgs=2311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/4092695025 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d8f070 0x559f47d8f450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 -- 172.21.15.32:0/4092695025 >> 172.21.15.32:0/4092695025 conn(0x559f47d4b950 msgr2=0x559f47d43560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 -- 172.21.15.32:0/4092695025 shutdown_connections 2023-12-20T06:03:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 -- 172.21.15.32:0/4092695025 wait complete. 2023-12-20T06:03:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 -- start start 2023-12-20T06:03:40.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.042+0000 7f12e7cd6000 1 Processor -- start 2023-12-20T06:03:40.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d451c0 0x559f47d4a750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47d51080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47d515c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f47b930e0 con 0x559f47d8f070 2023-12-20T06:03:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f47b32310 con 0x559f47d8f990 2023-12-20T06:03:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f47b50770 con 0x559f47d451c0 2023-12-20T06:03:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 10 client.0 _ll_register_callbacks cb 0x559f47d30b60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:03:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47d51080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47d51080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35648/0 (socket says 172.21.15.32:35648) 2023-12-20T06:03:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12d7fff640 1 -- 172.21.15.32:0/1598412037 learned_addr learned my addr 172.21.15.32:0/1598412037 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12de59c640 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d451c0 0x559f47d4a750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12ded9d640 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47d515c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12d7fff640 1 -- 172.21.15.32:0/1598412037 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d451c0 msgr2=0x559f47d4a750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:40.053 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12d7fff640 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d451c0 0x559f47d4a750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12d7fff640 1 -- 172.21.15.32:0/1598412037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 msgr2=0x559f47d515c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12d7fff640 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47d515c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12d7fff640 1 -- 172.21.15.32:0/1598412037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12d00013c0 con 0x559f47d8f070 2023-12-20T06:03:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12d7fff640 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47d51080 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f12d0001c90 tx=0x7f12d0007810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9d23271269629a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:03:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12bd7fa640 1 -- 172.21.15.32:0/1598412037 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d000dce0 con 0x559f47d8f070 2023-12-20T06:03:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1598412037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f47b62c10 con 0x559f47d8f070 2023-12-20T06:03:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12bd7fa640 1 -- 172.21.15.32:0/1598412037 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f12d000de40 con 0x559f47d8f070 2023-12-20T06:03:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12bd7fa640 1 -- 172.21.15.32:0/1598412037 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12d0003600 con 0x559f47d8f070 2023-12-20T06:03:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 10 client.11595 fetch_fsmap learned FSMap version 118 2023-12-20T06:03:40.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1598412037 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) v3 -- 0x559f47b58260 con 0x559f47d8f070 2023-12-20T06:03:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12bd7fa640 1 -- 172.21.15.32:0/1598412037 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f47b62c10 con 0x559f47d8f070 2023-12-20T06:03:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12bd7fa640 1 -- 172.21.15.32:0/1598412037 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 118) v1 ==== 3681+0+0 (secure 0 0 0) 0x7f12d0024020 con 0x559f47d8f070 2023-12-20T06:03:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 10 client.11595 fetch_fsmap finished waiting for FSMap version 118 2023-12-20T06:03:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 10 client.11595 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-20T06:03:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 20 client.11595 populate_metadata read hostname 'smithi032' 2023-12-20T06:03:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.32:6834/60134876,v1:172.21.15.32:6836/60134876] conn(0x559f47d968d0 0x559f47d96cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 4 client.11595 mds_command: new command op to 11298 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T06:03:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1598412037 --> [v2:172.21.15.32:6834/60134876,v1:172.21.15.32:6836/60134876] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x559f47b91a30 con 0x559f47d968d0 2023-12-20T06:03:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12de59c640 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.32:6834/60134876,v1:172.21.15.32:6836/60134876] conn(0x559f47d968d0 0x559f47d96cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12de59c640 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.32:6834/60134876,v1:172.21.15.32:6836/60134876] conn(0x559f47d968d0 0x559f47d96cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6789b4e8e56338cd server_cookie=f4b0b628a84f2e55 in_seq=0 out_seq=0 2023-12-20T06:03:40.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.046+0000 7f12bd7fa640 10 client.11595 ms_handle_connect on v2:172.21.15.32:6834/60134876 2023-12-20T06:03:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12bd7fa640 1 -- 172.21.15.32:0/1598412037 <== mds.0 v2:172.21.15.32:6834/60134876 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x559f47b91a30 con 0x559f47d968d0 2023-12-20T06:03:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12bd7fa640 10 client.11595 handle_command_reply: tid=0 2023-12-20T06:03:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 1 client.11595 shutdown 2023-12-20T06:03:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1598412037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 msgr2=0x559f47d51080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47d51080 secure :-1 s=READY pgs=2312 cs=0 l=1 rev1=1 crypto rx=0x7f12d0001c90 tx=0x7f12d0007810 comp rx=0 tx=0).stop 2023-12-20T06:03:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1598412037 shutdown_connections 2023-12-20T06:03:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.32:6834/60134876,v1:172.21.15.32:6836/60134876] conn(0x559f47d968d0 0x559f47d96cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47d515c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47d51080 unknown :-1 s=CLOSED pgs=2312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/1598412037 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d451c0 0x559f47d4a750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12bd7fa640 0 client.11595 ms_handle_reset on v2:172.21.15.32:6834/60134876 2023-12-20T06:03:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1598412037 >> 172.21.15.32:0/1598412037 conn(0x559f47d4b950 msgr2=0x559f47d43c60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1598412037 shutdown_connections 2023-12-20T06:03:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1598412037 wait complete. 2023-12-20T06:03:40.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.050+0000 7f12e7cd6000 20 client.11595 trim_cache size 0 max 16384 2023-12-20T06:03:40.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 Processor -- start 2023-12-20T06:03:40.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 -- start start 2023-12-20T06:03:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d451c0 0x559f47de6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d8f070 0x559f47de7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47de7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f47850940 con 0x559f47d451c0 2023-12-20T06:03:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f47b94670 con 0x559f47d8f990 2023-12-20T06:03:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f47b881f0 con 0x559f47d8f070 2023-12-20T06:03:40.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12de59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d451c0 0x559f47de6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12de59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d451c0 0x559f47de6ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35654/0 (socket says 172.21.15.32:35654) 2023-12-20T06:03:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12de59c640 1 -- 172.21.15.32:0/1945933362 learned_addr learned my addr 172.21.15.32:0/1945933362 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12ded9d640 1 --2- 172.21.15.32:0/1945933362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47de7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12d7fff640 1 --2- 172.21.15.32:0/1945933362 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d8f070 0x559f47de7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12de59c640 1 -- 172.21.15.32:0/1945933362 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d8f070 msgr2=0x559f47de7410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12de59c640 1 --2- 172.21.15.32:0/1945933362 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d8f070 0x559f47de7410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12de59c640 1 -- 172.21.15.32:0/1945933362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 msgr2=0x559f47de7950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12de59c640 1 --2- 172.21.15.32:0/1945933362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47de7950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12de59c640 1 -- 172.21.15.32:0/1945933362 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x559f47b3e3b0 con 0x559f47d451c0 2023-12-20T06:03:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12de59c640 1 --2- 172.21.15.32:0/1945933362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d451c0 0x559f47de6ed0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f12c800d520 tx=0x7f12c8013000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdd68df4aaad3e55 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12d4ff9640 1 -- 172.21.15.32:0/1945933362 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c80235a0 con 0x559f47d451c0 2023-12-20T06:03:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12d4ff9640 1 -- 172.21.15.32:0/1945933362 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f12c8023700 con 0x559f47d451c0 2023-12-20T06:03:40.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12d4ff9640 1 -- 172.21.15.32:0/1945933362 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12c8023ab0 con 0x559f47d451c0 2023-12-20T06:03:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1945933362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d451c0 msgr2=0x559f47de6ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/1945933362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d451c0 0x559f47de6ed0 secure :-1 s=READY pgs=2313 cs=0 l=1 rev1=1 crypto rx=0x7f12c800d520 tx=0x7f12c8013000 comp rx=0 tx=0).stop 2023-12-20T06:03:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1945933362 shutdown_connections 2023-12-20T06:03:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/1945933362 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47de7950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.074 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/1945933362 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d8f070 0x559f47de7410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/1945933362 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d451c0 0x559f47de6ed0 unknown :-1 s=CLOSED pgs=2313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1945933362 >> 172.21.15.32:0/1945933362 conn(0x559f47d4b950 msgr2=0x559f47d50f40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1945933362 shutdown_connections 2023-12-20T06:03:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 -- 172.21.15.32:0/1945933362 wait complete. 2023-12-20T06:03:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 -- start start 2023-12-20T06:03:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.066+0000 7f12e7cd6000 1 Processor -- start 2023-12-20T06:03:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d451c0 0x559f47dec590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47decad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47dcfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f47b3b860 con 0x559f47d8f070 2023-12-20T06:03:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x559f47b9b4b0 con 0x559f47d8f990 2023-12-20T06:03:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x559f47b908f0 con 0x559f47d451c0 2023-12-20T06:03:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 10 client.0 _ll_register_callbacks cb 0x559f47b41640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:03:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47decad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12d7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47decad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35658/0 (socket says 172.21.15.32:35658) 2023-12-20T06:03:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12d7fff640 1 -- 172.21.15.32:0/3500386317 learned_addr learned my addr 172.21.15.32:0/3500386317 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12de59c640 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d451c0 0x559f47dec590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12ded9d640 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47dcfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12d7fff640 1 -- 172.21.15.32:0/3500386317 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d451c0 msgr2=0x559f47dec590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12d7fff640 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d451c0 0x559f47dec590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12d7fff640 1 -- 172.21.15.32:0/3500386317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 msgr2=0x559f47dcfd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12d7fff640 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47dcfd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12d7fff640 1 -- 172.21.15.32:0/3500386317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f12e00c98c0 con 0x559f47d8f070 2023-12-20T06:03:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12ded9d640 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47dcfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12d7fff640 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47decad0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f12d0003ea0 tx=0x7f12d0009040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4105a436aee6e7f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12bd7fa640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:03:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12bd7fa640 1 -- 172.21.15.32:0/3500386317 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f12d0003800 con 0x559f47d8f070 2023-12-20T06:03:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12bd7fa640 1 -- 172.21.15.32:0/3500386317 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f12d0003960 con 0x559f47d8f070 2023-12-20T06:03:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 1 -- 172.21.15.32:0/3500386317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x559f47b58e00 con 0x559f47d8f070 2023-12-20T06:03:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12bd7fa640 1 -- 172.21.15.32:0/3500386317 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=118) v2 ==== 24+0+0 (secure 0 0 0) 0x7f12d0003b20 con 0x559f47d8f070 2023-12-20T06:03:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 10 client.11601 fetch_fsmap learned FSMap version 118 2023-12-20T06:03:40.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 1 -- 172.21.15.32:0/3500386317 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=118,monmap=2+}) v3 -- 0x559f4788fcc0 con 0x559f47d8f070 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12bd7fa640 1 -- 172.21.15.32:0/3500386317 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x559f47b58e00 con 0x559f47d8f070 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12bd7fa640 1 -- 172.21.15.32:0/3500386317 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 118) v1 ==== 3681+0+0 (secure 0 0 0) 0x7f12d0024030 con 0x559f47d8f070 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 10 client.11601 fetch_fsmap finished waiting for FSMap version 118 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 10 client.11601 resolve_mds: resolved name 'c' to daemon mds.c 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 20 client.11601 populate_metadata read hostname 'smithi032' 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.32:6834/60134876,v1:172.21.15.32:6836/60134876] conn(0x559f47dd02a0 0x559f47dd0680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 4 client.11601 mds_command: new command op to 11298 tid=0[{"prefix": "status"}] 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 1 -- 172.21.15.32:0/3500386317 --> [v2:172.21.15.32:6834/60134876,v1:172.21.15.32:6836/60134876] -- command(tid 0: {"prefix": "status"}) v1 -- 0x559f47b8c160 con 0x559f47dd02a0 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12de59c640 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.32:6834/60134876,v1:172.21.15.32:6836/60134876] conn(0x559f47dd02a0 0x559f47dd0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12de59c640 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.32:6834/60134876,v1:172.21.15.32:6836/60134876] conn(0x559f47dd02a0 0x559f47dd0680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35f9c3ae6955969b server_cookie=d7f6fcabd0b47bbc in_seq=0 out_seq=0 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12bd7fa640 10 client.11601 ms_handle_connect on v2:172.21.15.32:6834/60134876 2023-12-20T06:03:40.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12bd7fa640 1 -- 172.21.15.32:0/3500386317 <== mds.0 v2:172.21.15.32:6834/60134876 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+338 (crc 0 0 0) 0x559f47b8c160 con 0x559f47dd02a0 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12bd7fa640 10 client.11601 handle_command_reply: tid=0 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.070+0000 7f12e7cd6000 1 client.11601 shutdown 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e7cd6000 1 -- 172.21.15.32:0/3500386317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 msgr2=0x559f47decad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47decad0 secure :-1 s=READY pgs=2314 cs=0 l=1 rev1=1 crypto rx=0x7f12d0003ea0 tx=0x7f12d0009040 comp rx=0 tx=0).stop 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e7cd6000 1 -- 172.21.15.32:0/3500386317 shutdown_connections 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.32:6834/60134876,v1:172.21.15.32:6836/60134876] conn(0x559f47dd02a0 0x559f47dd0680 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x559f47d8f990 0x559f47dcfd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x559f47d8f070 0x559f47decad0 unknown :-1 s=CLOSED pgs=2314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12bd7fa640 0 client.11601 ms_handle_reset on v2:172.21.15.32:6834/60134876 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e7cd6000 1 --2- 172.21.15.32:0/3500386317 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x559f47d451c0 0x559f47dec590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e7cd6000 1 -- 172.21.15.32:0/3500386317 >> 172.21.15.32:0/3500386317 conn(0x559f47d4b950 msgr2=0x559f47d8c030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e7cd6000 1 -- 172.21.15.32:0/3500386317 shutdown_connections 2023-12-20T06:03:40.079 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e7cd6000 1 -- 172.21.15.32:0/3500386317 wait complete. 2023-12-20T06:03:40.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e7cd6000 20 client.11601 trim_cache size 0 max 16384 2023-12-20T06:03:40.080 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:03:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T06:03:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T06:03:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 11298, 2023-12-20T06:03:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T06:03:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T06:03:40.080 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "backup_fs", 2023-12-20T06:03:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.5545586800000004, 2023-12-20T06:03:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 118, 2023-12-20T06:03:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 234, 2023-12-20T06:03:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 234, 2023-12-20T06:03:40.081 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 22.573082155000002 2023-12-20T06:03:40.081 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:03:40.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 -- 172.21.15.32:0/2837245273 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f12c0065ca0 msgr2=0x7f12c0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f12c0065ca0 0x7f12c0068160 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f12c8000fc0 tx=0x7f12c8009040 comp rx=0 tx=0).stop 2023-12-20T06:03:40.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 -- 172.21.15.32:0/2837245273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e01599e0 msgr2=0x7f12e00c73d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e01599e0 0x7f12e00c73d0 secure :-1 s=READY pgs=2310 cs=0 l=1 rev1=1 crypto rx=0x7f12d80af040 tx=0x7f12d80197b0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 -- 172.21.15.32:0/2837245273 shutdown_connections 2023-12-20T06:03:40.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f12c0065ca0 0x7f12c0068160 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f12e01599e0 0x7f12e00c73d0 unknown :-1 s=CLOSED pgs=2310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f12e00b0d00 0x7f12e00c6e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 --2- 172.21.15.32:0/2837245273 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f12e00aece0 0x7f12e00cd070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 -- 172.21.15.32:0/2837245273 >> 172.21.15.32:0/2837245273 conn(0x7f12e006a320 msgr2=0x7f12e0158570 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:40.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 -- 172.21.15.32:0/2837245273 shutdown_connections 2023-12-20T06:03:40.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.074+0000 7f12e497b640 1 -- 172.21.15.32:0/2837245273 wait complete. 2023-12-20T06:03:40.093 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T06:03:40.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb55f2c640 1 Processor -- start 2023-12-20T06:03:40.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb55f2c640 1 -- start start 2023-12-20T06:03:40.239 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb55f2c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fbb5006ce60 0x7fbb5015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb55f2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb5015f110 0x7fbb5015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb55f2c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbb5015bbd0 con 0x7fbb500af5e0 2023-12-20T06:03:40.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb55f2c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbb5015beb0 con 0x7fbb5006ce60 2023-12-20T06:03:40.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb55f2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb5015e910 con 0x7fbb5015f110 2023-12-20T06:03:40.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb4f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fbb500af5e0 0x7fbb500af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47066/0 (socket says 172.21.15.32:47066) 2023-12-20T06:03:40.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb4f7fe640 1 -- 172.21.15.32:0/3343577159 learned_addr learned my addr 172.21.15.32:0/3343577159 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:40.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb4effd640 1 --2- 172.21.15.32:0/3343577159 >> v2:172.21.15.32:3300/0 conn(0x7fbb5006ce60 0x7fbb5015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb4ffff640 1 --2- 172.21.15.32:0/3343577159 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb5015f110 0x7fbb5015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb4e7fc640 1 -- 172.21.15.32:0/3343577159 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3699999924 0 0) 0x7fbb5015bbd0 con 0x7fbb500af5e0 2023-12-20T06:03:40.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb4e7fc640 1 -- 172.21.15.32:0/3343577159 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fbb38003530 con 0x7fbb500af5e0 2023-12-20T06:03:40.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb4effd640 1 -- 172.21.15.32:0/3343577159 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb5015f110 msgr2=0x7fbb5015c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:40.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb4effd640 1 --2- 172.21.15.32:0/3343577159 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb5015f110 0x7fbb5015c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb4effd640 1 -- 172.21.15.32:0/3343577159 >> v1:172.21.15.32:6789/0 conn(0x7fbb500af5e0 legacy=0x7fbb500af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.241 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.238+0000 7fbb4effd640 1 -- 172.21.15.32:0/3343577159 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb5015ebf0 con 0x7fbb5006ce60 2023-12-20T06:03:40.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb4effd640 1 --2- 172.21.15.32:0/3343577159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb5006ce60 0x7fbb5015b280 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fbb440097c0 tx=0x7fbb4401c3f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71029a797fce8fc9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb4e7fc640 1 -- 172.21.15.32:0/3343577159 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb5015b9d0 con 0x7fbb5006ce60 2023-12-20T06:03:40.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb4e7fc640 1 -- 172.21.15.32:0/3343577159 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb440ad640 con 0x7fbb5006ce60 2023-12-20T06:03:40.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb4e7fc640 1 -- 172.21.15.32:0/3343577159 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440ad9d0 con 0x7fbb5006ce60 2023-12-20T06:03:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb55f2c640 1 -- 172.21.15.32:0/3343577159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb5006ce60 msgr2=0x7fbb5015b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb55f2c640 1 --2- 172.21.15.32:0/3343577159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb5006ce60 0x7fbb5015b280 secure :-1 s=READY pgs=2316 cs=0 l=1 rev1=1 crypto rx=0x7fbb440097c0 tx=0x7fbb4401c3f0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb55f2c640 1 -- 172.21.15.32:0/3343577159 shutdown_connections 2023-12-20T06:03:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb55f2c640 1 --2- 172.21.15.32:0/3343577159 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb5006ce60 0x7fbb5015b280 unknown :-1 s=CLOSED pgs=2316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb55f2c640 1 --2- 172.21.15.32:0/3343577159 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb5015f110 0x7fbb5015c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb55f2c640 1 -- 172.21.15.32:0/3343577159 >> 172.21.15.32:0/3343577159 conn(0x7fbb5006a320 msgr2=0x7fbb500ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:40.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb55f2c640 1 -- 172.21.15.32:0/3343577159 shutdown_connections 2023-12-20T06:03:40.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.242+0000 7fbb55f2c640 1 -- 172.21.15.32:0/3343577159 wait complete. 2023-12-20T06:03:40.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb55f2c640 1 Processor -- start 2023-12-20T06:03:40.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb55f2c640 1 -- start start 2023-12-20T06:03:40.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb55f2c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb5006ce60 0x7fbb50153e90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb55f2c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb500af5e0 0x7fbb50071b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb55f2c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb5015f110 0x7fbb500cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb55f2c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50153b80 con 0x7fbb500af5e0 2023-12-20T06:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb55f2c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbb500716a0 con 0x7fbb5006ce60 2023-12-20T06:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb55f2c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbb50070d70 con 0x7fbb5015f110 2023-12-20T06:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb4effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb500af5e0 0x7fbb50071b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb4effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb500af5e0 0x7fbb50071b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35680/0 (socket says 172.21.15.32:35680) 2023-12-20T06:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb4effd640 1 -- 172.21.15.32:0/999237596 learned_addr learned my addr 172.21.15.32:0/999237596 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb4f7fe640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb5006ce60 0x7fbb50153e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb4ffff640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb5015f110 0x7fbb500cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.249 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb4effd640 1 -- 172.21.15.32:0/999237596 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb5015f110 msgr2=0x7fbb500cb170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb4effd640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb5015f110 0x7fbb500cb170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb4effd640 1 -- 172.21.15.32:0/999237596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb5006ce60 msgr2=0x7fbb50153e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb4effd640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb5006ce60 0x7fbb50153e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb4effd640 1 -- 172.21.15.32:0/999237596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbb44009420 con 0x7fbb500af5e0 2023-12-20T06:03:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb4effd640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb500af5e0 0x7fbb50071b30 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fbb440b0040 tx=0x7fbb44019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb37fff640 1 -- 172.21.15.32:0/999237596 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440b6620 con 0x7fbb500af5e0 2023-12-20T06:03:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb37fff640 1 -- 172.21.15.32:0/999237596 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbb440a6040 con 0x7fbb500af5e0 2023-12-20T06:03:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb37fff640 1 -- 172.21.15.32:0/999237596 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbb440b69e0 con 0x7fbb500af5e0 2023-12-20T06:03:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb55f2c640 1 -- 172.21.15.32:0/999237596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbb50071050 con 0x7fbb500af5e0 2023-12-20T06:03:40.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb55f2c640 1 -- 172.21.15.32:0/999237596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbb501573a0 con 0x7fbb500af5e0 2023-12-20T06:03:40.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.246+0000 7fbb55f2c640 1 -- 172.21.15.32:0/999237596 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbb50071850 con 0x7fbb500af5e0 2023-12-20T06:03:40.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.250+0000 7fbb37fff640 1 -- 172.21.15.32:0/999237596 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fbb440b4040 con 0x7fbb500af5e0 2023-12-20T06:03:40.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.250+0000 7fbb37fff640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb3c065cf0 0x7fbb3c0681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.250+0000 7fbb37fff640 1 -- 172.21.15.32:0/999237596 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7fbb4411dbf0 con 0x7fbb500af5e0 2023-12-20T06:03:40.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.250+0000 7fbb4f7fe640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb3c065cf0 0x7fbb3c0681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.250+0000 7fbb4f7fe640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb3c065cf0 0x7fbb3c0681b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbb400010f0 tx=0x7fbb40007bf0 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.254+0000 7fbb37fff640 1 -- 172.21.15.32:0/999237596 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbb50071850 con 0x7fbb500af5e0 2023-12-20T06:03:40.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.434+0000 7fbb55f2c640 1 -- 172.21.15.32:0/999237596 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fbb50158af0 con 0x7fbb3c065cf0 2023-12-20T06:03:40.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.434+0000 7fbb37fff640 1 -- 172.21.15.32:0/999237596 <== mgr.10956 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+82 (secure 0 0 0) 0x7fbb40007620 con 0x7fbb3c065cf0 2023-12-20T06:03:40.435 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T06:03:40.435 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:40.436 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T06:03:40.436 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:03:40.436 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:03:40.436 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "backup_fs" 2023-12-20T06:03:40.436 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:03:40.436 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T06:03:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 -- 172.21.15.32:0/999237596 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb3c065cf0 msgr2=0x7fbb3c0681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb3c065cf0 0x7fbb3c0681b0 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7fbb400010f0 tx=0x7fbb40007bf0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 -- 172.21.15.32:0/999237596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb500af5e0 msgr2=0x7fbb50071b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb500af5e0 0x7fbb50071b30 secure :-1 s=READY pgs=2317 cs=0 l=1 rev1=1 crypto rx=0x7fbb440b0040 tx=0x7fbb44019190 comp rx=0 tx=0).stop 2023-12-20T06:03:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 -- 172.21.15.32:0/999237596 shutdown_connections 2023-12-20T06:03:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbb3c065cf0 0x7fbb3c0681b0 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbb5015f110 0x7fbb500cb170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbb500af5e0 0x7fbb50071b30 unknown :-1 s=CLOSED pgs=2317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 --2- 172.21.15.32:0/999237596 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbb5006ce60 0x7fbb50153e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 -- 172.21.15.32:0/999237596 >> 172.21.15.32:0/999237596 conn(0x7fbb5006a320 msgr2=0x7fbb500af9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 -- 172.21.15.32:0/999237596 shutdown_connections 2023-12-20T06:03:40.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.438+0000 7fbb55f2c640 1 -- 172.21.15.32:0/999237596 wait complete. 2023-12-20T06:03:40.457 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T06:03:40.604 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.602+0000 7f24b64b6640 1 Processor -- start 2023-12-20T06:03:40.605 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.602+0000 7f24b64b6640 1 -- start start 2023-12-20T06:03:40.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24b64b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24b01609d0 0x7f24b0160e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24b64b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24b00cb870 0x7f24b00cbc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24b64b6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f24b01607c0 con 0x7f24b01603e0 2023-12-20T06:03:40.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24b64b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f24b00d0540 con 0x7f24b01609d0 2023-12-20T06:03:40.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24b64b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b00cecd0 con 0x7f24b00cb870 2023-12-20T06:03:40.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24af7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24b01609d0 0x7f24b0160e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24af7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f24b01609d0 0x7f24b0160e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35688/0 (socket says 172.21.15.32:35688) 2023-12-20T06:03:40.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24af7fe640 1 -- 172.21.15.32:0/3146007233 learned_addr learned my addr 172.21.15.32:0/3146007233 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:40.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24b4a53640 1 --2- 172.21.15.32:0/3146007233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24b00cb870 0x7f24b00cbc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24af7fe640 1 -- 172.21.15.32:0/3146007233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24b00cb870 msgr2=0x7f24b00cbc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:40.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24af7fe640 1 --2- 172.21.15.32:0/3146007233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24b00cb870 0x7f24b00cbc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24af7fe640 1 -- 172.21.15.32:0/3146007233 >> v1:172.21.15.39:6789/0 conn(0x7f24b01603e0 legacy=0x7f24b00c9ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.606+0000 7f24af7fe640 1 -- 172.21.15.32:0/3146007233 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24b00cff80 con 0x7f24b01609d0 2023-12-20T06:03:40.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24af7fe640 1 --2- 172.21.15.32:0/3146007233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24b01609d0 0x7f24b0160e40 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009870 tx=0x7f24a401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c1b3e05909e64d0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24aeffd640 1 -- 172.21.15.32:0/3146007233 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40ab680 con 0x7f24b01609d0 2023-12-20T06:03:40.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24aeffd640 1 -- 172.21.15.32:0/3146007233 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a40ab7e0 con 0x7f24b01609d0 2023-12-20T06:03:40.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24aeffd640 1 -- 172.21.15.32:0/3146007233 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40abb70 con 0x7f24b01609d0 2023-12-20T06:03:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24b64b6640 1 -- 172.21.15.32:0/3146007233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24b01609d0 msgr2=0x7f24b0160e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24b64b6640 1 --2- 172.21.15.32:0/3146007233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24b01609d0 0x7f24b0160e40 secure :-1 s=READY pgs=2318 cs=0 l=1 rev1=1 crypto rx=0x7f24a4009870 tx=0x7f24a401c480 comp rx=0 tx=0).stop 2023-12-20T06:03:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24b64b6640 1 -- 172.21.15.32:0/3146007233 shutdown_connections 2023-12-20T06:03:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24b64b6640 1 --2- 172.21.15.32:0/3146007233 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24b00cb870 0x7f24b00cbc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24b64b6640 1 --2- 172.21.15.32:0/3146007233 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24b01609d0 0x7f24b0160e40 unknown :-1 s=CLOSED pgs=2318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24b64b6640 1 -- 172.21.15.32:0/3146007233 >> 172.21.15.32:0/3146007233 conn(0x7f24b0098c20 msgr2=0x7f24b00cf870 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:40.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24b64b6640 1 -- 172.21.15.32:0/3146007233 shutdown_connections 2023-12-20T06:03:40.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24b64b6640 1 -- 172.21.15.32:0/3146007233 wait complete. 2023-12-20T06:03:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24b64b6640 1 Processor -- start 2023-12-20T06:03:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.610+0000 7f24b64b6640 1 -- start start 2023-12-20T06:03:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24b64b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24b00cb870 0x7f24b0152850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24b64b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24b01603e0 0x7f24b0153d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24b64b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b01609d0 0x7f24b006cd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24b64b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b0153060 con 0x7f24b01603e0 2023-12-20T06:03:40.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24b64b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f24b0157830 con 0x7f24b01609d0 2023-12-20T06:03:40.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24b64b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f24b01532b0 con 0x7f24b00cb870 2023-12-20T06:03:40.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24af7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24b01603e0 0x7f24b0153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24af7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24b01603e0 0x7f24b0153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35698/0 (socket says 172.21.15.32:35698) 2023-12-20T06:03:40.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24af7fe640 1 -- 172.21.15.32:0/2926626683 learned_addr learned my addr 172.21.15.32:0/2926626683 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:40.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24affff640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24b00cb870 0x7f24b0152850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24b4a53640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b01609d0 0x7f24b006cd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24af7fe640 1 -- 172.21.15.32:0/2926626683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24b00cb870 msgr2=0x7f24b0152850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24af7fe640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24b00cb870 0x7f24b0152850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24af7fe640 1 -- 172.21.15.32:0/2926626683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b01609d0 msgr2=0x7f24b006cd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:40.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24af7fe640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b01609d0 0x7f24b006cd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:40.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24af7fe640 1 -- 172.21.15.32:0/2926626683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f24a40094d0 con 0x7f24b01603e0 2023-12-20T06:03:40.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24affff640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24b00cb870 0x7f24b0152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:40.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24b4a53640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b01609d0 0x7f24b006cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:40.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24af7fe640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24b01603e0 0x7f24b0153d90 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f24a401a0b0 tx=0x7f24a401acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24acff9640 1 -- 172.21.15.32:0/2926626683 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40b4800 con 0x7f24b01603e0 2023-12-20T06:03:40.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24acff9640 1 -- 172.21.15.32:0/2926626683 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f24a40b4980 con 0x7f24b01603e0 2023-12-20T06:03:40.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24b64b6640 1 -- 172.21.15.32:0/2926626683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f24b0153590 con 0x7f24b01603e0 2023-12-20T06:03:40.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24acff9640 1 -- 172.21.15.32:0/2926626683 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f24a40b4d30 con 0x7f24b01603e0 2023-12-20T06:03:40.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24b64b6640 1 -- 172.21.15.32:0/2926626683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f24b0152d90 con 0x7f24b01603e0 2023-12-20T06:03:40.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24b64b6640 1 -- 172.21.15.32:0/2926626683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f24b004e000 con 0x7f24b01603e0 2023-12-20T06:03:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.614+0000 7f24acff9640 1 -- 172.21.15.32:0/2926626683 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 134) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f24a40c7440 con 0x7f24b01603e0 2023-12-20T06:03:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.618+0000 7f24acff9640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f249c065cd0 0x7f249c068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.618+0000 7f24acff9640 1 -- 172.21.15.32:0/2926626683 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f24a411e830 con 0x7f24b01603e0 2023-12-20T06:03:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.618+0000 7f24affff640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f249c065cd0 0x7f249c068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:40.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.618+0000 7f24affff640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f249c065cd0 0x7f249c068190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f24a0007fd0 tx=0x7f24a0009040 comp rx=0 tx=0).ready entity=mgr.10956 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:40.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.622+0000 7f24acff9640 1 -- 172.21.15.32:0/2926626683 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f24a40eb9d0 con 0x7f24b01603e0 2023-12-20T06:03:40.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:40.818+0000 7f24b64b6640 1 -- 172.21.15.32:0/2926626683 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7f24b0153a40 con 0x7f24b01603e0 2023-12-20T06:03:41.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.598+0000 7f24acff9640 1 -- 172.21.15.32:0/2926626683 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 135) v1 ==== 85449+0+0 (secure 0 0 0) 0x7f24a40ebda0 con 0x7f24b01603e0 2023-12-20T06:03:41.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.602+0000 7f24acff9640 1 -- 172.21.15.32:0/2926626683 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v135) v1 ==== 92+0+0 (secure 0 0 0) 0x7f24a40f2fe0 con 0x7f24b01603e0 2023-12-20T06:03:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 -- 172.21.15.32:0/2926626683 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f249c065cd0 msgr2=0x7f249c068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f249c065cd0 0x7f249c068190 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f24a0007fd0 tx=0x7f24a0009040 comp rx=0 tx=0).stop 2023-12-20T06:03:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 -- 172.21.15.32:0/2926626683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24b01603e0 msgr2=0x7f24b0153d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24b01603e0 0x7f24b0153d90 secure :-1 s=READY pgs=2319 cs=0 l=1 rev1=1 crypto rx=0x7f24a401a0b0 tx=0x7f24a401acc0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.606 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 -- 172.21.15.32:0/2926626683 shutdown_connections 2023-12-20T06:03:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f249c065cd0 0x7f249c068190 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f24b01609d0 0x7f24b006cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f24b01603e0 0x7f24b0153d90 unknown :-1 s=CLOSED pgs=2319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 --2- 172.21.15.32:0/2926626683 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f24b00cb870 0x7f24b0152850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 -- 172.21.15.32:0/2926626683 >> 172.21.15.32:0/2926626683 conn(0x7f24b0098c20 msgr2=0x7f24b015f5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 -- 172.21.15.32:0/2926626683 shutdown_connections 2023-12-20T06:03:41.607 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.606+0000 7f24b64b6640 1 -- 172.21.15.32:0/2926626683 wait complete. 2023-12-20T06:03:41.631 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T06:03:41.777 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:41.774+0000 7f2cc5666180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:03:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f350ca41640 1 Processor -- start 2023-12-20T06:03:41.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f350ca41640 1 -- start start 2023-12-20T06:03:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f350ca41640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f35080d14d0 0x7f35080b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f350ca41640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35080d2650 con 0x7f350815ce60 2023-12-20T06:03:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f350ca41640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f35080d1a40 con 0x7f35080d3300 2023-12-20T06:03:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f350ca41640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f35080d1d20 con 0x7f35080d14d0 2023-12-20T06:03:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f350659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f350815ce60 0x7f35080d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47074/0 (socket says 172.21.15.32:47074) 2023-12-20T06:03:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f350659c640 1 -- 172.21.15.32:0/953328644 learned_addr learned my addr 172.21.15.32:0/953328644 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:41.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f3506d9d640 1 --2- 172.21.15.32:0/953328644 >> v2:172.21.15.32:3300/0 conn(0x7f35080d14d0 0x7f35080b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f350559a640 1 -- 172.21.15.32:0/953328644 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2204408650 0 0) 0x7f35080d2650 con 0x7f350815ce60 2023-12-20T06:03:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f350559a640 1 -- 172.21.15.32:0/953328644 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f34f0003530 con 0x7f350815ce60 2023-12-20T06:03:41.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f3506d9d640 1 -- 172.21.15.32:0/953328644 >> v1:172.21.15.39:6789/0 conn(0x7f35080d3300 legacy=0x7f35080d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:41.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f350559a640 1 -- 172.21.15.32:0/953328644 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2143030831 0 0) 0x7f35080d1a40 con 0x7f35080d3300 2023-12-20T06:03:41.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f3506d9d640 1 -- 172.21.15.32:0/953328644 >> v1:172.21.15.32:6789/0 conn(0x7f350815ce60 legacy=0x7f35080d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:41.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.786+0000 7f3506d9d640 1 -- 172.21.15.32:0/953328644 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f35080d2000 con 0x7f35080d14d0 2023-12-20T06:03:41.790 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f3506d9d640 1 --2- 172.21.15.32:0/953328644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080d14d0 0x7f35080b6ea0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f3500009d90 tx=0x7f350001f130 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee7b621592df2179 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f350559a640 1 -- 172.21.15.32:0/953328644 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35000af600 con 0x7f35080d14d0 2023-12-20T06:03:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f350559a640 1 -- 172.21.15.32:0/953328644 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f35000ab050 con 0x7f35080d14d0 2023-12-20T06:03:41.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f350559a640 1 -- 172.21.15.32:0/953328644 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f35000af980 con 0x7f35080d14d0 2023-12-20T06:03:41.791 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f38bdb4f180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:03:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f350ca41640 1 -- 172.21.15.32:0/953328644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080d14d0 msgr2=0x7f35080b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f350ca41640 1 --2- 172.21.15.32:0/953328644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080d14d0 0x7f35080b6ea0 secure :-1 s=READY pgs=2322 cs=0 l=1 rev1=1 crypto rx=0x7f3500009d90 tx=0x7f350001f130 comp rx=0 tx=0).stop 2023-12-20T06:03:41.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f350ca41640 1 -- 172.21.15.32:0/953328644 shutdown_connections 2023-12-20T06:03:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f350ca41640 1 --2- 172.21.15.32:0/953328644 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080d14d0 0x7f35080b6ea0 unknown :-1 s=CLOSED pgs=2322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f350ca41640 1 -- 172.21.15.32:0/953328644 >> 172.21.15.32:0/953328644 conn(0x7f350806b5e0 msgr2=0x7f350806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f350ca41640 1 -- 172.21.15.32:0/953328644 shutdown_connections 2023-12-20T06:03:41.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.790+0000 7f350ca41640 1 -- 172.21.15.32:0/953328644 wait complete. 2023-12-20T06:03:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f350ca41640 1 Processor -- start 2023-12-20T06:03:41.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f350ca41640 1 -- start start 2023-12-20T06:03:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f350ca41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35080d14d0 0x7f350814f0b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f350ca41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080d3300 0x7f35080a95a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f350ca41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f350815ce60 0x7f35081f1ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:41.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f350ca41640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35080cbf30 con 0x7f35080d3300 2023-12-20T06:03:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f350ca41640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f35080cb320 con 0x7f350815ce60 2023-12-20T06:03:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f350ca41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f35080cb600 con 0x7f35080d14d0 2023-12-20T06:03:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f3505d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080d3300 0x7f35080a95a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f350659c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35080d14d0 0x7f350814f0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f3505d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080d3300 0x7f35080a95a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35740/0 (socket says 172.21.15.32:35740) 2023-12-20T06:03:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f3505d9b640 1 -- 172.21.15.32:0/3612275768 learned_addr learned my addr 172.21.15.32:0/3612275768 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f3506d9d640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f350815ce60 0x7f35081f1ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f3505d9b640 1 -- 172.21.15.32:0/3612275768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35080d14d0 msgr2=0x7f350814f0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f3505d9b640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35080d14d0 0x7f350814f0b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f3505d9b640 1 -- 172.21.15.32:0/3612275768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f350815ce60 msgr2=0x7f35081f1ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f3505d9b640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f350815ce60 0x7f35081f1ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f3505d9b640 1 -- 172.21.15.32:0/3612275768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3500009a70 con 0x7f35080d3300 2023-12-20T06:03:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f3505d9b640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080d3300 0x7f35080a95a0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00c520 tx=0x7f34fc00c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f34f6ffd640 1 -- 172.21.15.32:0/3612275768 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc003600 con 0x7f35080d3300 2023-12-20T06:03:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f34f6ffd640 1 -- 172.21.15.32:0/3612275768 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f34fc003780 con 0x7f35080d3300 2023-12-20T06:03:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f350ca41640 1 -- 172.21.15.32:0/3612275768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f35080cb8e0 con 0x7f35080d3300 2023-12-20T06:03:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f34f6ffd640 1 -- 172.21.15.32:0/3612275768 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f34fc003b30 con 0x7f35080d3300 2023-12-20T06:03:41.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.794+0000 7f350ca41640 1 -- 172.21.15.32:0/3612275768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f35080cc3c0 con 0x7f35080d3300 2023-12-20T06:03:41.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.798+0000 7f350ca41640 1 -- 172.21.15.32:0/3612275768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f35080cc0a0 con 0x7f35080d3300 2023-12-20T06:03:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.798+0000 7f34f6ffd640 1 -- 172.21.15.32:0/3612275768 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 135) v1 ==== 85449+0+0 (secure 0 0 0) 0x7f34fc003c90 con 0x7f35080d3300 2023-12-20T06:03:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.798+0000 7f34f6ffd640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34e8065cf0 0x7f34e80681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.798+0000 7f34f6ffd640 1 -- 172.21.15.32:0/3612275768 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f34fc082ea0 con 0x7f35080d3300 2023-12-20T06:03:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.798+0000 7f350659c640 1 -- 172.21.15.32:0/3612275768 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34e8065cf0 msgr2=0x7f34e80681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:41.800 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.798+0000 7f350659c640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34e8065cf0 0x7f34e80681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:41.803 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.802+0000 7f34f6ffd640 1 -- 172.21.15.32:0/3612275768 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f34fc0502a0 con 0x7f35080d3300 2023-12-20T06:03:41.962 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.962+0000 7f350ca41640 1 -- 172.21.15.32:0/3612275768 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7f350804e000 con 0x7f35080d3300 2023-12-20T06:03:41.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.966+0000 7f34f6ffd640 1 -- 172.21.15.32:0/3612275768 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v77)=0 v77) v1 ==== 138+0+0 (secure 0 0 0) 0x7f34fc017070 con 0x7f35080d3300 2023-12-20T06:03:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 -- 172.21.15.32:0/3612275768 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34e8065cf0 msgr2=0x7f34e80681b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34e8065cf0 0x7f34e80681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 -- 172.21.15.32:0/3612275768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080d3300 msgr2=0x7f35080a95a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:41.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080d3300 0x7f35080a95a0 secure :-1 s=READY pgs=2323 cs=0 l=1 rev1=1 crypto rx=0x7f34fc00c520 tx=0x7f34fc00c9e0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 -- 172.21.15.32:0/3612275768 shutdown_connections 2023-12-20T06:03:41.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f34e8065cf0 0x7f34e80681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f350815ce60 0x7f35081f1ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f35080d3300 0x7f35080a95a0 unknown :-1 s=CLOSED pgs=2323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 --2- 172.21.15.32:0/3612275768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f35080d14d0 0x7f350814f0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:41.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 -- 172.21.15.32:0/3612275768 >> 172.21.15.32:0/3612275768 conn(0x7f350806b5e0 msgr2=0x7f3508066030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:41.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 -- 172.21.15.32:0/3612275768 shutdown_connections 2023-12-20T06:03:41.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:41.970+0000 7f350ca41640 1 -- 172.21.15.32:0/3612275768 wait complete. 2023-12-20T06:03:41.991 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T06:03:42.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.138+0000 7f0818f6d640 1 Processor -- start 2023-12-20T06:03:42.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.138+0000 7f0818f6d640 1 -- start start 2023-12-20T06:03:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f0818f6d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0814070940 0x7f08140caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f0818f6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081415b6a0 con 0x7f081415abb0 2023-12-20T06:03:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f0818f6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f081406d8c0 con 0x7f08140b0cb0 2023-12-20T06:03:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f0818f6d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f081406e160 con 0x7f0814070940 2023-12-20T06:03:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f081259c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f081415abb0 0x7f081415bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47088/0 (socket says 172.21.15.32:47088) 2023-12-20T06:03:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f081259c640 1 -- 172.21.15.32:0/1135626318 learned_addr learned my addr 172.21.15.32:0/1135626318 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:42.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f0812d9d640 1 --2- 172.21.15.32:0/1135626318 >> v2:172.21.15.32:3300/0 conn(0x7f0814070940 0x7f08140caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f081159a640 1 -- 172.21.15.32:0/1135626318 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 480222612 0 0) 0x7f081415b6a0 con 0x7f081415abb0 2023-12-20T06:03:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f081159a640 1 -- 172.21.15.32:0/1135626318 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07fc003530 con 0x7f081415abb0 2023-12-20T06:03:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f081159a640 1 -- 172.21.15.32:0/1135626318 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 519060455 0 0) 0x7f081406d8c0 con 0x7f08140b0cb0 2023-12-20T06:03:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f081159a640 1 -- 172.21.15.32:0/1135626318 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f081415b6a0 con 0x7f08140b0cb0 2023-12-20T06:03:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f0812d9d640 1 -- 172.21.15.32:0/1135626318 >> v1:172.21.15.39:6789/0 conn(0x7f08140b0cb0 legacy=0x7f0814070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f0812d9d640 1 -- 172.21.15.32:0/1135626318 >> v1:172.21.15.32:6789/0 conn(0x7f081415abb0 legacy=0x7f081415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:42.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.142+0000 7f0812d9d640 1 -- 172.21.15.32:0/1135626318 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f081406dba0 con 0x7f0814070940 2023-12-20T06:03:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.146+0000 7f0812d9d640 1 --2- 172.21.15.32:0/1135626318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0814070940 0x7f08140caaf0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f080c009d90 tx=0x7f080c01c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fc3e2ce13af4328f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.146+0000 7f081159a640 1 -- 172.21.15.32:0/1135626318 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c0a5a10 con 0x7f0814070940 2023-12-20T06:03:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.146+0000 7f081159a640 1 -- 172.21.15.32:0/1135626318 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c0a5b70 con 0x7f0814070940 2023-12-20T06:03:42.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.146+0000 7f081159a640 1 -- 172.21.15.32:0/1135626318 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c0ad440 con 0x7f0814070940 2023-12-20T06:03:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.146+0000 7f0818f6d640 1 -- 172.21.15.32:0/1135626318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0814070940 msgr2=0x7f08140caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.146+0000 7f0818f6d640 1 --2- 172.21.15.32:0/1135626318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0814070940 0x7f08140caaf0 secure :-1 s=READY pgs=2325 cs=0 l=1 rev1=1 crypto rx=0x7f080c009d90 tx=0x7f080c01c950 comp rx=0 tx=0).stop 2023-12-20T06:03:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.146+0000 7f0818f6d640 1 -- 172.21.15.32:0/1135626318 shutdown_connections 2023-12-20T06:03:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.146+0000 7f0818f6d640 1 --2- 172.21.15.32:0/1135626318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0814070940 0x7f08140caaf0 unknown :-1 s=CLOSED pgs=2325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.146+0000 7f0818f6d640 1 -- 172.21.15.32:0/1135626318 >> 172.21.15.32:0/1135626318 conn(0x7f081406a510 msgr2=0x7f08140ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:42.148 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.146+0000 7f0818f6d640 1 -- 172.21.15.32:0/1135626318 shutdown_connections 2023-12-20T06:03:42.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.146+0000 7f0818f6d640 1 -- 172.21.15.32:0/1135626318 wait complete. 2023-12-20T06:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0818f6d640 1 Processor -- start 2023-12-20T06:03:42.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0818f6d640 1 -- start start 2023-12-20T06:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0818f6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0814070940 0x7f0814159340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0818f6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08140b0cb0 0x7f08140a8c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0818f6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f081415abb0 0x7f0814150710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0818f6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0814150c50 con 0x7f0814070940 2023-12-20T06:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0818f6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08141582a0 con 0x7f081415abb0 2023-12-20T06:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0818f6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0814158580 con 0x7f08140b0cb0 2023-12-20T06:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f081259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0814070940 0x7f0814159340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:42.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0812d9d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f081415abb0 0x7f0814150710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f081259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0814070940 0x7f0814159340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35766/0 (socket says 172.21.15.32:35766) 2023-12-20T06:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0811d9b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08140b0cb0 0x7f08140a8c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f081259c640 1 -- 172.21.15.32:0/177554119 learned_addr learned my addr 172.21.15.32:0/177554119 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0812d9d640 1 -- 172.21.15.32:0/177554119 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08140b0cb0 msgr2=0x7f08140a8c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0812d9d640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08140b0cb0 0x7f08140a8c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0812d9d640 1 -- 172.21.15.32:0/177554119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0814070940 msgr2=0x7f0814159340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0812d9d640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0814070940 0x7f0814159340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0812d9d640 1 -- 172.21.15.32:0/177554119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f080c009a70 con 0x7f081415abb0 2023-12-20T06:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0811d9b640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08140b0cb0 0x7f08140a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:42.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f081259c640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0814070940 0x7f0814159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0812d9d640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f081415abb0 0x7f0814150710 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f080c0ac000 tx=0x7f080c0196f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:42.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0802ffd640 1 -- 172.21.15.32:0/177554119 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c0add20 con 0x7f081415abb0 2023-12-20T06:03:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0802ffd640 1 -- 172.21.15.32:0/177554119 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f080c0ade80 con 0x7f081415abb0 2023-12-20T06:03:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0802ffd640 1 -- 172.21.15.32:0/177554119 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f080c0b9650 con 0x7f081415abb0 2023-12-20T06:03:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.150+0000 7f0818f6d640 1 -- 172.21.15.32:0/177554119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0814158860 con 0x7f081415abb0 2023-12-20T06:03:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.154+0000 7f0818f6d640 1 -- 172.21.15.32:0/177554119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0814158e20 con 0x7f081415abb0 2023-12-20T06:03:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.154+0000 7f0818f6d640 1 -- 172.21.15.32:0/177554119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0814159010 con 0x7f081415abb0 2023-12-20T06:03:42.155 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.154+0000 7f0802ffd640 1 -- 172.21.15.32:0/177554119 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 135) v1 ==== 85449+0+0 (secure 0 0 0) 0x7f080c0b97b0 con 0x7f081415abb0 2023-12-20T06:03:42.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.154+0000 7f0802ffd640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07f4065cf0 0x7f07f40681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.154+0000 7f0802ffd640 1 -- 172.21.15.32:0/177554119 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7f080c11ed30 con 0x7f081415abb0 2023-12-20T06:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.154+0000 7f081259c640 1 -- 172.21.15.32:0/177554119 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07f4065cf0 msgr2=0x7f07f40681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:42.157 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.154+0000 7f081259c640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07f4065cf0 0x7f07f40681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:42.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.166+0000 7f0802ffd640 1 -- 172.21.15.32:0/177554119 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f080c0ec130 con 0x7f081415abb0 2023-12-20T06:03:42.224 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:42.222+0000 7f2cc5666180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:03:42.247 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:42.246+0000 7f38bdb4f180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:03:42.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.326+0000 7f0818f6d640 1 -- 172.21.15.32:0/177554119 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f081404e000 con 0x7f081415abb0 2023-12-20T06:03:42.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.338+0000 7f0802ffd640 1 -- 172.21.15.32:0/177554119 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v78)=0 v78) v1 ==== 133+0+0 (secure 0 0 0) 0x7f080c009450 con 0x7f081415abb0 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 -- 172.21.15.32:0/177554119 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07f4065cf0 msgr2=0x7f07f40681b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07f4065cf0 0x7f07f40681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 -- 172.21.15.32:0/177554119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f081415abb0 msgr2=0x7f0814150710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f081415abb0 0x7f0814150710 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7f080c0ac000 tx=0x7f080c0196f0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 -- 172.21.15.32:0/177554119 shutdown_connections 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07f4065cf0 0x7f07f40681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f081415abb0 0x7f0814150710 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08140b0cb0 0x7f08140a8c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 --2- 172.21.15.32:0/177554119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0814070940 0x7f0814159340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 -- 172.21.15.32:0/177554119 >> 172.21.15.32:0/177554119 conn(0x7f081406a510 msgr2=0x7f0814159e00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 -- 172.21.15.32:0/177554119 shutdown_connections 2023-12-20T06:03:42.343 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.342+0000 7f0818f6d640 1 -- 172.21.15.32:0/177554119 wait complete. 2023-12-20T06:03:42.363 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:42.363 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p mandatory_fs_argument_test_dir) 2023-12-20T06:03:42.377 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add mandatory_fs_argument_test_dir 1M 2023-12-20T06:03:42.382 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:42.378+0000 7f2cc5666180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:03:42.409 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:42.410+0000 7f38bdb4f180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:03:42.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.558+0000 7fd138a04640 1 Processor -- start 2023-12-20T06:03:42.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.558+0000 7fd138a04640 1 -- start start 2023-12-20T06:03:42.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd138a04640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd13415ccf0 0x7fd1340cd270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:42.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd138a04640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd1340cd7b0 0x7fd1340cf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:42.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd138a04640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd1340d10f0 con 0x7fd1340cf920 2023-12-20T06:03:42.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd138a04640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1340cfdb0 con 0x7fd13415ccf0 2023-12-20T06:03:42.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd138a04640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd1340ce2f0 con 0x7fd1340cd7b0 2023-12-20T06:03:42.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd1337fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd13415ccf0 0x7fd1340cd270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:42.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd1337fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd13415ccf0 0x7fd1340cd270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35782/0 (socket says 172.21.15.32:35782) 2023-12-20T06:03:42.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd1337fe640 1 -- 172.21.15.32:0/4282738179 learned_addr learned my addr 172.21.15.32:0/4282738179 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:42.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd132ffd640 1 --2- 172.21.15.32:0/4282738179 >> v2:172.21.15.39:3300/0 conn(0x7fd1340cd7b0 0x7fd1340cf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:42.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd1337fe640 1 -- 172.21.15.32:0/4282738179 >> v2:172.21.15.39:3300/0 conn(0x7fd1340cd7b0 msgr2=0x7fd1340cf3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:42.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd1337fe640 1 --2- 172.21.15.32:0/4282738179 >> v2:172.21.15.39:3300/0 conn(0x7fd1340cd7b0 0x7fd1340cf3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd1337fe640 1 -- 172.21.15.32:0/4282738179 >> v1:172.21.15.39:6789/0 conn(0x7fd1340cf920 legacy=0x7fd1340b4490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:42.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd1337fe640 1 -- 172.21.15.32:0/4282738179 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1340cdd30 con 0x7fd13415ccf0 2023-12-20T06:03:42.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd1337fe640 1 --2- 172.21.15.32:0/4282738179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13415ccf0 0x7fd1340cd270 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fd1340ccfc0 tx=0x7fd12401c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b536fc4985288d1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:42.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.562+0000 7fd1327fc640 1 -- 172.21.15.32:0/4282738179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1240ab600 con 0x7fd13415ccf0 2023-12-20T06:03:42.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.566+0000 7fd1327fc640 1 -- 172.21.15.32:0/4282738179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd1240a7050 con 0x7fd13415ccf0 2023-12-20T06:03:42.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.566+0000 7fd1327fc640 1 -- 172.21.15.32:0/4282738179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd1240ab980 con 0x7fd13415ccf0 2023-12-20T06:03:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.566+0000 7fd138a04640 1 -- 172.21.15.32:0/4282738179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13415ccf0 msgr2=0x7fd1340cd270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.566+0000 7fd138a04640 1 --2- 172.21.15.32:0/4282738179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13415ccf0 0x7fd1340cd270 secure :-1 s=READY pgs=2326 cs=0 l=1 rev1=1 crypto rx=0x7fd1340ccfc0 tx=0x7fd12401c470 comp rx=0 tx=0).stop 2023-12-20T06:03:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.566+0000 7fd138a04640 1 -- 172.21.15.32:0/4282738179 shutdown_connections 2023-12-20T06:03:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.566+0000 7fd138a04640 1 --2- 172.21.15.32:0/4282738179 >> v2:172.21.15.39:3300/0 conn(0x7fd1340cd7b0 0x7fd1340cf3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.566+0000 7fd138a04640 1 --2- 172.21.15.32:0/4282738179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd13415ccf0 0x7fd1340cd270 unknown :-1 s=CLOSED pgs=2326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.566+0000 7fd138a04640 1 -- 172.21.15.32:0/4282738179 >> 172.21.15.32:0/4282738179 conn(0x7fd1340658e0 msgr2=0x7fd134065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:42.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.566+0000 7fd138a04640 1 -- 172.21.15.32:0/4282738179 shutdown_connections 2023-12-20T06:03:42.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.566+0000 7fd138a04640 1 -- 172.21.15.32:0/4282738179 wait complete. 2023-12-20T06:03:42.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.566+0000 7fd138a04640 1 Processor -- start 2023-12-20T06:03:42.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd138a04640 1 -- start start 2023-12-20T06:03:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd138a04640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1340cd7b0 0x7fd134151110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd138a04640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1340cf920 0x7fd134152d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd138a04640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd13415ccf0 0x7fd1340c86f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd138a04640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd1340c82d0 con 0x7fd1340cf920 2023-12-20T06:03:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd138a04640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd134153710 con 0x7fd13415ccf0 2023-12-20T06:03:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd138a04640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd1341539f0 con 0x7fd1340cd7b0 2023-12-20T06:03:42.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1340cf920 0x7fd134152d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd132ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1340cf920 0x7fd134152d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35798/0 (socket says 172.21.15.32:35798) 2023-12-20T06:03:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd132ffd640 1 -- 172.21.15.32:0/1283859355 learned_addr learned my addr 172.21.15.32:0/1283859355 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd1337fe640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1340cd7b0 0x7fd134151110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd133fff640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd13415ccf0 0x7fd1340c86f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd132ffd640 1 -- 172.21.15.32:0/1283859355 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1340cd7b0 msgr2=0x7fd134151110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd132ffd640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd1340cd7b0 0x7fd134151110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd132ffd640 1 -- 172.21.15.32:0/1283859355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd13415ccf0 msgr2=0x7fd1340c86f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd132ffd640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd13415ccf0 0x7fd1340c86f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd132ffd640 1 -- 172.21.15.32:0/1283859355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd1240094d0 con 0x7fd1340cf920 2023-12-20T06:03:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd132ffd640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd1340cf920 0x7fd134152d30 secure :-1 s=READY pgs=2327 cs=0 l=1 rev1=1 crypto rx=0x7fd12800e870 tx=0x7fd12800ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:42.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd128009510 con 0x7fd1340cf920 2023-12-20T06:03:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd12800eec0 con 0x7fd1340cf920 2023-12-20T06:03:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd138a04640 1 -- 172.21.15.32:0/1283859355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd1340c7ef0 con 0x7fd1340cf920 2023-12-20T06:03:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd12800cb40 con 0x7fd1340cf920 2023-12-20T06:03:42.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.570+0000 7fd138a04640 1 -- 172.21.15.32:0/1283859355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd134151650 con 0x7fd1340cf920 2023-12-20T06:03:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.574+0000 7fd138a04640 1 -- 172.21.15.32:0/1283859355 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd1340c8410 con 0x7fd1340cf920 2023-12-20T06:03:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.574+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 135) v1 ==== 85449+0+0 (secure 0 0 0) 0x7fd12800ccc0 con 0x7fd1340cf920 2023-12-20T06:03:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.574+0000 7fd11bfff640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 0x7fd1200681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.574+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(234..234 src has 1..234) v4 ==== 6629+0+0 (secure 0 0 0) 0x7fd128083af0 con 0x7fd1340cf920 2023-12-20T06:03:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.574+0000 7fd1337fe640 1 -- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 msgr2=0x7fd1200681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:42.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.574+0000 7fd1337fe640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 0x7fd1200681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:42.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.578+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd1340c8410 con 0x7fd1340cf920 2023-12-20T06:03:42.746 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:03:42.750 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:03:42.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.758+0000 7fd138a04640 1 -- 172.21.15.32:0/1283859355 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd13415d0d0 con 0x7fd120065cf0 2023-12-20T06:03:42.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.778+0000 7fd1337fe640 1 -- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 msgr2=0x7fd1200681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:42.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:42.778+0000 7fd1337fe640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 0x7fd1200681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T06:03:42.895 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:42.894+0000 7f2cc5666180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:03:42.934 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:42.934+0000 7f38bdb4f180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:03:42.956 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:42.954+0000 7f2cc5666180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:03:42.995 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:42.994+0000 7f38bdb4f180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:03:43.015 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:43.014+0000 7f2cc5666180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:03:43.055 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:43.054+0000 7f38bdb4f180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:03:43.106 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:43.106+0000 7f2cc5666180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:03:43.146 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:43.146+0000 7f38bdb4f180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:03:43.166 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:43.162+0000 7f2cc5666180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:03:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:43.182+0000 7fd1337fe640 1 -- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 msgr2=0x7fd1200681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:43.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:43.182+0000 7fd1337fe640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 0x7fd1200681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T06:03:43.208 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:43.206+0000 7f38bdb4f180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:03:43.255 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:43.254+0000 7f2cc5666180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:03:43.297 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:43.298+0000 7f38bdb4f180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:03:43.314 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:43.314+0000 7f2cc5666180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:03:43.356 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:43.354+0000 7f38bdb4f180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:03:43.466 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:43.462+0000 7f2cc5666180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:03:43.508 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:43.506+0000 7f38bdb4f180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:03:43.528 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:43.526+0000 7f2cc5666180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:03:43.570 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:43.570+0000 7f38bdb4f180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:03:43.777 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:03:43.778 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:03:43.778 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:03:43.786 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:43.786+0000 7f2cc5666180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:03:43.822 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:03:43.822 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:03:43.822 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:03:43.831 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:43.830+0000 7f38bdb4f180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:03:43.929 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:43.926+0000 7f2cc5666180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:03:43.975 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:43.974+0000 7f38bdb4f180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:03:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:43.986+0000 7fd1337fe640 1 -- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 msgr2=0x7fd1200681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:43.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:43.986+0000 7fd1337fe640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 0x7fd1200681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T06:03:44.060 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:44.058+0000 7f2cc5666180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:03:44.109 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:44.110+0000 7f38bdb4f180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:03:44.134 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:44.134+0000 7f2cc5666180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:03:44.183 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:44.182+0000 7f38bdb4f180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:03:44.277 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:44.274+0000 7f2cc5666180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:03:44.327 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:44.326+0000 7f38bdb4f180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:03:44.846 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:44.846+0000 7f2cc5666180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:03:44.900 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:44.902+0000 7f38bdb4f180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:03:44.935 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:44.934+0000 7f2cc5666180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:03:44.990 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:44.990+0000 7f38bdb4f180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:03:45.092 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:45.090+0000 7f2cc5666180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:03:45.147 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.146+0000 7f38bdb4f180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:03:45.179 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:45.178+0000 7f2cc5666180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:03:45.235 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.234+0000 7f38bdb4f180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:03:45.513 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:45.510+0000 7f2cc5666180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:03:45.573 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.574+0000 7f38bdb4f180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:03:45.577 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:45.574+0000 7f2cc5666180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:03:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:45.590+0000 7fd1337fe640 1 -- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 msgr2=0x7fd1200681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:45.589 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:45.590+0000 7fd1337fe640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 0x7fd1200681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-20T06:03:45.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:45.638+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 136) v1 ==== 85449+0+0 (secure 0 0 0) 0x7fd128051110 con 0x7fd1340cf920 2023-12-20T06:03:45.640 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.638+0000 7f38bdb4f180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:03:45.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:45.666+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 137) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fd1280520f0 con 0x7fd1340cf920 2023-12-20T06:03:45.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:45.666+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 msgr2=0x7fd1200681b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:45.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:45.666+0000 7fd11bfff640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 0x7fd1200681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:45.829 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.830+0000 7f38402b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.829 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.830+0000 7f38402b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.829 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.830+0000 7f38402b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.829 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.830+0000 7f38402b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.829 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.830+0000 7f38402b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.830 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.830+0000 7f3842ab5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.830 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.830+0000 7f3842ab5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.830 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.830+0000 7f3842ab5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.830 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.830+0000 7f3842ab5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.830 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.830+0000 7f3842ab5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.842 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.842+0000 7f3842ab5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.842 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.842+0000 7f3842ab5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.842 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.842+0000 7f3842ab5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.842 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.842+0000 7f3842ab5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.842 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.842+0000 7f3842ab5640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.843 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.842+0000 7f38402b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.843 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.842+0000 7f38402b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.843 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.842+0000 7f38402b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.843 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.842+0000 7f38402b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:45.843 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:45.842+0000 7f38402b0640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:46.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.674+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 <== mon.0 v2:172.21.15.32:3300/0 9 ==== mgrmap(e 138) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fd1280519e0 con 0x7fd1340cf920 2023-12-20T06:03:46.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.678+0000 7fd11bfff640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 0x7fd1200681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:46.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.678+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "mandatory_fs_argument_test_dir", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fd12001b8c0 con 0x7fd120065cf0 2023-12-20T06:03:46.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.678+0000 7fd1337fe640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 0x7fd1200681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:46.682 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.682+0000 7fd1337fe640 1 --2- 172.21.15.32:0/1283859355 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd120065cf0 0x7fd1200681b0 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fd1240ad000 tx=0x7fd1240a6000 comp rx=0 tx=0).ready entity=mgr.11194 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:46.684 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:46.682+0000 7f384a2c4640 -1 mgr.server reply reply (22) Invalid argument filesystem argument is required when there is more than one file system 2023-12-20T06:03:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.686+0000 7fd11bfff640 1 -- 172.21.15.32:0/1283859355 <== mgr.11194 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: -22 filesystem argument is required when there is more than one file system) v1 ==== 79+0+0 (secure 0 0 0) 0x7fd1240040c0 con 0x7fd120065cf0 2023-12-20T06:03:46.685 INFO:teuthology.orchestra.run.smithi032.stderr:Error EINVAL: filesystem argument is required when there is more than one file system 2023-12-20T06:03:46.696 DEBUG:teuthology.orchestra.run:got remote process result: 22 2023-12-20T06:03:46.696 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:46.696 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir mandatory_fs_argument_test_dir) 2023-12-20T06:03:46.708 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T06:03:46.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.898+0000 7fd3b3ad1640 1 Processor -- start 2023-12-20T06:03:46.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.898+0000 7fd3b3ad1640 1 -- start start 2023-12-20T06:03:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.898+0000 7fd3b3ad1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3ac070490 0x7fd3ac15fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.898+0000 7fd3b3ad1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd3ac06d240 0x7fd3ac160210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.898+0000 7fd3b3ad1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3ac070870 0x7fd3ac0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.898+0000 7fd3b3ad1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ac160750 con 0x7fd3ac070870 2023-12-20T06:03:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.898+0000 7fd3b3ad1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3ac06c480 con 0x7fd3ac06d240 2023-12-20T06:03:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.898+0000 7fd3b3ad1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac06c760 con 0x7fd3ac070490 2023-12-20T06:03:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b206e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3ac070870 0x7fd3ac0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b206e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3ac070870 0x7fd3ac0ca1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35954/0 (socket says 172.21.15.32:35954) 2023-12-20T06:03:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b206e640 1 -- 172.21.15.32:0/2966484585 learned_addr learned my addr 172.21.15.32:0/2966484585 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:46.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b106c640 1 --2- 172.21.15.32:0/2966484585 >> v2:172.21.15.39:3300/0 conn(0x7fd3ac06d240 0x7fd3ac160210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:46.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b186d640 1 --2- 172.21.15.32:0/2966484585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3ac070490 0x7fd3ac15fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:46.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b206e640 1 -- 172.21.15.32:0/2966484585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3ac070490 msgr2=0x7fd3ac15fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:46.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b206e640 1 --2- 172.21.15.32:0/2966484585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3ac070490 0x7fd3ac15fcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:46.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b206e640 1 -- 172.21.15.32:0/2966484585 >> v2:172.21.15.39:3300/0 conn(0x7fd3ac06d240 msgr2=0x7fd3ac160210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:46.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b206e640 1 --2- 172.21.15.32:0/2966484585 >> v2:172.21.15.39:3300/0 conn(0x7fd3ac06d240 0x7fd3ac160210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:46.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b206e640 1 -- 172.21.15.32:0/2966484585 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3ac06ca40 con 0x7fd3ac070870 2023-12-20T06:03:46.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b206e640 1 --2- 172.21.15.32:0/2966484585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3ac070870 0x7fd3ac0ca1a0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0716d0 tx=0x7fd3a401c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bb1d98ca3bc09729 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:46.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b086b640 1 -- 172.21.15.32:0/2966484585 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40a5780 con 0x7fd3ac070870 2023-12-20T06:03:46.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b086b640 1 -- 172.21.15.32:0/2966484585 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a40a58e0 con 0x7fd3ac070870 2023-12-20T06:03:46.904 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.902+0000 7fd3b086b640 1 -- 172.21.15.32:0/2966484585 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a40a5c70 con 0x7fd3ac070870 2023-12-20T06:03:46.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/2966484585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3ac070870 msgr2=0x7fd3ac0ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:46.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 --2- 172.21.15.32:0/2966484585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3ac070870 0x7fd3ac0ca1a0 secure :-1 s=READY pgs=2341 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0716d0 tx=0x7fd3a401c9a0 comp rx=0 tx=0).stop 2023-12-20T06:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/2966484585 shutdown_connections 2023-12-20T06:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 --2- 172.21.15.32:0/2966484585 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3ac070870 0x7fd3ac0ca1a0 unknown :-1 s=CLOSED pgs=2341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 --2- 172.21.15.32:0/2966484585 >> v2:172.21.15.39:3300/0 conn(0x7fd3ac06d240 0x7fd3ac160210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 --2- 172.21.15.32:0/2966484585 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3ac070490 0x7fd3ac15fcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/2966484585 >> 172.21.15.32:0/2966484585 conn(0x7fd3ac06a510 msgr2=0x7fd3ac0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/2966484585 shutdown_connections 2023-12-20T06:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/2966484585 wait complete. 2023-12-20T06:03:46.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 Processor -- start 2023-12-20T06:03:46.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 -- start start 2023-12-20T06:03:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3ac06d240 0x7fd3ac0c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3ac070490 0x7fd3ac0b2310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3ac070870 0x7fd3ac0b2850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac0c5e20 con 0x7fd3ac06d240 2023-12-20T06:03:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac15abb0 con 0x7fd3ac070870 2023-12-20T06:03:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b3ad1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3ac15ae90 con 0x7fd3ac070490 2023-12-20T06:03:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b186d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3ac06d240 0x7fd3ac0c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:46.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b186d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3ac06d240 0x7fd3ac0c42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35968/0 (socket says 172.21.15.32:35968) 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b186d640 1 -- 172.21.15.32:0/237728179 learned_addr learned my addr 172.21.15.32:0/237728179 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b186d640 1 -- 172.21.15.32:0/237728179 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3ac070490 msgr2=0x7fd3ac0b2310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b186d640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3ac070490 0x7fd3ac0b2310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b186d640 1 -- 172.21.15.32:0/237728179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3ac070870 msgr2=0x7fd3ac0b2850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b186d640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3ac070870 0x7fd3ac0b2850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.906+0000 7fd3b186d640 1 -- 172.21.15.32:0/237728179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a4009b30 con 0x7fd3ac06d240 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.910+0000 7fd3b186d640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3ac06d240 0x7fd3ac0c42d0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800e930 tx=0x7fd3a800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.910+0000 7fd3a27fc640 1 -- 172.21.15.32:0/237728179 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a8009690 con 0x7fd3ac06d240 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.910+0000 7fd3a27fc640 1 -- 172.21.15.32:0/237728179 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a800ca00 con 0x7fd3ac06d240 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.910+0000 7fd3a27fc640 1 -- 172.21.15.32:0/237728179 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a800cd70 con 0x7fd3ac06d240 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.910+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/237728179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ac0c62b0 con 0x7fd3ac06d240 2023-12-20T06:03:46.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.910+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/237728179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3ac0b0a30 con 0x7fd3ac06d240 2023-12-20T06:03:46.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.910+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/237728179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3ac15acf0 con 0x7fd3ac06d240 2023-12-20T06:03:46.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.910+0000 7fd3a27fc640 1 -- 172.21.15.32:0/237728179 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 138) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fd3a8013470 con 0x7fd3ac06d240 2023-12-20T06:03:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.910+0000 7fd3a27fc640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd390065900 0x7fd390067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.910+0000 7fd3a27fc640 1 -- 172.21.15.32:0/237728179 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7059+0+0 (secure 0 0 0) 0x7fd3a8083720 con 0x7fd3ac06d240 2023-12-20T06:03:46.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.910+0000 7fd3b106c640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd390065900 0x7fd390067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:46.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.914+0000 7fd3b106c640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd390065900 0x7fd390067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009a60 tx=0x7fd39c009520 comp rx=0 tx=0).ready entity=mgr.11194 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:46.916 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:46.914+0000 7fd3a27fc640 1 -- 172.21.15.32:0/237728179 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd3ac15acf0 con 0x7fd3ac06d240 2023-12-20T06:03:47.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.110+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/237728179 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fd3ac15acf0 con 0x7fd3ac06d240 2023-12-20T06:03:47.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.690+0000 7fd3a27fc640 1 -- 172.21.15.32:0/237728179 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 139) v1 ==== 85433+0+0 (secure 0 0 0) 0x7fd3a8051290 con 0x7fd3ac06d240 2023-12-20T06:03:47.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.690+0000 7fd3a27fc640 1 -- 172.21.15.32:0/237728179 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v139) v1 ==== 93+0+0 (secure 0 0 0) 0x7fd3a806dc40 con 0x7fd3ac06d240 2023-12-20T06:03:47.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/237728179 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd390065900 msgr2=0x7fd390067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:47.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd390065900 0x7fd390067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fd39c009a60 tx=0x7fd39c009520 comp rx=0 tx=0).stop 2023-12-20T06:03:47.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/237728179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3ac06d240 msgr2=0x7fd3ac0c42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:47.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3ac06d240 0x7fd3ac0c42d0 secure :-1 s=READY pgs=2342 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800e930 tx=0x7fd3a800edf0 comp rx=0 tx=0).stop 2023-12-20T06:03:47.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/237728179 shutdown_connections 2023-12-20T06:03:47.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd390065900 0x7fd390067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:47.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3ac070870 0x7fd3ac0b2850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:47.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3ac070490 0x7fd3ac0b2310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:47.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 --2- 172.21.15.32:0/237728179 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3ac06d240 0x7fd3ac0c42d0 unknown :-1 s=CLOSED pgs=2342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:47.695 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/237728179 >> 172.21.15.32:0/237728179 conn(0x7fd3ac06a510 msgr2=0x7fd3ac070fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:47.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/237728179 shutdown_connections 2023-12-20T06:03:47.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:47.694+0000 7fd3b3ad1640 1 -- 172.21.15.32:0/237728179 wait complete. 2023-12-20T06:03:47.719 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T06:03:47.719 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:03:47.847 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:03:47.869 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:47.866+0000 7ff84022d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:03:47.877 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:47.878+0000 7f0144f7e180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:03:47.924 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T06:03:47.925 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:03:47.925 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:03:47.935 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:03:47.935 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:03:47.986 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:48.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.170+0000 7f8169dbc640 1 Processor -- start 2023-12-20T06:03:48.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.170+0000 7f8169dbc640 1 -- start start 2023-12-20T06:03:48.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f8169dbc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f816406caf0 0x7f81640705a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f8169dbc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f81640ae140 con 0x7f816415d9a0 2023-12-20T06:03:48.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f8169dbc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8164072170 con 0x7f81641593c0 2023-12-20T06:03:48.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f8169dbc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f81640ae9e0 con 0x7f816406caf0 2023-12-20T06:03:48.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f8162ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f816415d9a0 0x7f816406c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47094/0 (socket says 172.21.15.32:47094) 2023-12-20T06:03:48.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f8162ffd640 1 -- 172.21.15.32:0/1427381289 learned_addr learned my addr 172.21.15.32:0/1427381289 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:48.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f8163fff640 1 --2- 172.21.15.32:0/1427381289 >> v2:172.21.15.39:3300/0 conn(0x7f816406caf0 0x7f81640705a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f81627fc640 1 -- 172.21.15.32:0/1427381289 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1728837765 0 0) 0x7f81640ae140 con 0x7f816415d9a0 2023-12-20T06:03:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f81627fc640 1 -- 172.21.15.32:0/1427381289 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f814c003450 con 0x7f816415d9a0 2023-12-20T06:03:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f81627fc640 1 -- 172.21.15.32:0/1427381289 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1142896197 0 0) 0x7f8164072170 con 0x7f81641593c0 2023-12-20T06:03:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f81627fc640 1 -- 172.21.15.32:0/1427381289 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f81640ae140 con 0x7f81641593c0 2023-12-20T06:03:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f8163fff640 1 -- 172.21.15.32:0/1427381289 >> v1:172.21.15.39:6789/0 conn(0x7f81641593c0 legacy=0x7f81640afdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f8163fff640 1 -- 172.21.15.32:0/1427381289 >> v1:172.21.15.32:6789/0 conn(0x7f816415d9a0 legacy=0x7f816406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.174+0000 7f8163fff640 1 -- 172.21.15.32:0/1427381289 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f81640ae390 con 0x7f816406caf0 2023-12-20T06:03:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f8163fff640 1 --2- 172.21.15.32:0/1427381289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f816406caf0 0x7f81640705a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f815c009d90 tx=0x7f815c01c800 comp rx=0 tx=0).ready entity=mon.1 client_cookie=86208e726e5019b9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f81627fc640 1 -- 172.21.15.32:0/1427381289 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c0a57a0 con 0x7f816406caf0 2023-12-20T06:03:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f81627fc640 1 -- 172.21.15.32:0/1427381289 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c0a5900 con 0x7f816406caf0 2023-12-20T06:03:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f81627fc640 1 -- 172.21.15.32:0/1427381289 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c0a5c90 con 0x7f816406caf0 2023-12-20T06:03:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f8169dbc640 1 -- 172.21.15.32:0/1427381289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f816406caf0 msgr2=0x7f81640705a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f8169dbc640 1 --2- 172.21.15.32:0/1427381289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f816406caf0 0x7f81640705a0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f815c009d90 tx=0x7f815c01c800 comp rx=0 tx=0).stop 2023-12-20T06:03:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f8169dbc640 1 -- 172.21.15.32:0/1427381289 shutdown_connections 2023-12-20T06:03:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f8169dbc640 1 --2- 172.21.15.32:0/1427381289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f816406caf0 0x7f81640705a0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.179 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f8169dbc640 1 -- 172.21.15.32:0/1427381289 >> 172.21.15.32:0/1427381289 conn(0x7f816406a320 msgr2=0x7f81640ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f8169dbc640 1 -- 172.21.15.32:0/1427381289 shutdown_connections 2023-12-20T06:03:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f8169dbc640 1 -- 172.21.15.32:0/1427381289 wait complete. 2023-12-20T06:03:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.178+0000 7f8169dbc640 1 Processor -- start 2023-12-20T06:03:48.180 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8169dbc640 1 -- start start 2023-12-20T06:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8169dbc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f816406caf0 0x7f816414fd60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8169dbc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81641593c0 0x7f8164147d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8169dbc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f816415d9a0 0x7f81640c6370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8169dbc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81640c58d0 con 0x7f816415d9a0 2023-12-20T06:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8169dbc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f81640c8150 con 0x7f816406caf0 2023-12-20T06:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8169dbc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f81640c5b70 con 0x7f81641593c0 2023-12-20T06:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8163fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f816415d9a0 0x7f81640c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8163fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f816415d9a0 0x7f81640c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35996/0 (socket says 172.21.15.32:35996) 2023-12-20T06:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8163fff640 1 -- 172.21.15.32:0/2178410668 learned_addr learned my addr 172.21.15.32:0/2178410668 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:48.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8162ffd640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81641593c0 0x7f8164147d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f81637fe640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f816406caf0 0x7f816414fd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8163fff640 1 -- 172.21.15.32:0/2178410668 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81641593c0 msgr2=0x7f8164147d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8163fff640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81641593c0 0x7f8164147d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8163fff640 1 -- 172.21.15.32:0/2178410668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f816406caf0 msgr2=0x7f816414fd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8163fff640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f816406caf0 0x7f816414fd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8163fff640 1 -- 172.21.15.32:0/2178410668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f815c009a70 con 0x7f816415d9a0 2023-12-20T06:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8163fff640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f816415d9a0 0x7f81640c6370 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f815c0b0040 tx=0x7f815c0195a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f814bfff640 1 -- 172.21.15.32:0/2178410668 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c0b9650 con 0x7f816415d9a0 2023-12-20T06:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f814bfff640 1 -- 172.21.15.32:0/2178410668 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f815c0b97b0 con 0x7f816415d9a0 2023-12-20T06:03:48.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8169dbc640 1 -- 172.21.15.32:0/2178410668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f81640c5e50 con 0x7f816415d9a0 2023-12-20T06:03:48.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f814bfff640 1 -- 172.21.15.32:0/2178410668 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f815c0b9b40 con 0x7f816415d9a0 2023-12-20T06:03:48.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8169dbc640 1 -- 172.21.15.32:0/2178410668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8164148250 con 0x7f816415d9a0 2023-12-20T06:03:48.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.182+0000 7f8169dbc640 1 -- 172.21.15.32:0/2178410668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f816404e000 con 0x7f816415d9a0 2023-12-20T06:03:48.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.186+0000 7f814bfff640 1 -- 172.21.15.32:0/2178410668 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 139) v1 ==== 85433+0+0 (secure 0 0 0) 0x7f815c0b9ca0 con 0x7f816415d9a0 2023-12-20T06:03:48.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.186+0000 7f814bfff640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8150065ca0 0x7f8150068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.186+0000 7f81637fe640 1 -- 172.21.15.32:0/2178410668 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8150065ca0 msgr2=0x7f8150068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:48.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.186+0000 7f814bfff640 1 -- 172.21.15.32:0/2178410668 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f815c11eb40 con 0x7f816415d9a0 2023-12-20T06:03:48.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.186+0000 7f81637fe640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8150065ca0 0x7f8150068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:48.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.190+0000 7f814bfff640 1 -- 172.21.15.32:0/2178410668 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f815c0ebc60 con 0x7f816415d9a0 2023-12-20T06:03:48.311 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:48.310+0000 7ff84022d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:03:48.321 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:48.322+0000 7f0144f7e180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:03:48.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.350+0000 7f8169dbc640 1 -- 172.21.15.32:0/2178410668 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f81640c6010 con 0x7f816415d9a0 2023-12-20T06:03:48.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f814bfff640 1 -- 172.21.15.32:0/2178410668 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v235) v1 ==== 74+0+20431 (secure 0 0 0) 0x7f815c0ef280 con 0x7f816415d9a0 2023-12-20T06:03:48.358 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:48.358 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":235,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:45.663060+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:03:20.399088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-20T06:03:20.926367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:03:32.174574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:03:33.509224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3054548475":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2026838481":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/3886671554":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1847933543":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/4013366117":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/805068630":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6833/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1823890749":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6832/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2852735264":"2023-12-21T06:03:45.663020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 -- 172.21.15.32:0/2178410668 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8150065ca0 msgr2=0x7f8150068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8150065ca0 0x7f8150068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 -- 172.21.15.32:0/2178410668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f816415d9a0 msgr2=0x7f81640c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f816415d9a0 0x7f81640c6370 secure :-1 s=READY pgs=2346 cs=0 l=1 rev1=1 crypto rx=0x7f815c0b0040 tx=0x7f815c0195a0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 -- 172.21.15.32:0/2178410668 shutdown_connections 2023-12-20T06:03:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f8150065ca0 0x7f8150068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f816415d9a0 0x7f81640c6370 unknown :-1 s=CLOSED pgs=2346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f81641593c0 0x7f8164147d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 --2- 172.21.15.32:0/2178410668 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f816406caf0 0x7f816414fd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 -- 172.21.15.32:0/2178410668 >> 172.21.15.32:0/2178410668 conn(0x7f816406a320 msgr2=0x7f81640ccef0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 -- 172.21.15.32:0/2178410668 shutdown_connections 2023-12-20T06:03:48.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.358+0000 7f8169dbc640 1 -- 172.21.15.32:0/2178410668 wait complete. 2023-12-20T06:03:48.381 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T06:03:20.399088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '229', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T06:03:20.926367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '230', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-20T06:03:32.174574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-20T06:03:33.509224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '234', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:03:48.381 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:03:48.468 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:48.466+0000 7ff84022d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:03:48.482 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:48.482+0000 7f0144f7e180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:03:48.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe160041640 1 Processor -- start 2023-12-20T06:03:48.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe160041640 1 -- start start 2023-12-20T06:03:48.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe160041640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe1580b1980 0x7fe1580cd370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe160041640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe15815c4e0 con 0x7fe1580cd8b0 2023-12-20T06:03:48.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe160041640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe1580cc120 con 0x7fe15815c100 2023-12-20T06:03:48.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe160041640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe1580ce810 con 0x7fe1580b1980 2023-12-20T06:03:48.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe15d5dc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe1580b1980 0x7fe1580cd370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe15e5de640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fe1580cd8b0 0x7fe1580cec30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:47098/0 (socket says 172.21.15.32:47098) 2023-12-20T06:03:48.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe15e5de640 1 -- 172.21.15.32:0/137401260 learned_addr learned my addr 172.21.15.32:0/137401260 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:48.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe15cddb640 1 -- 172.21.15.32:0/137401260 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1307836442 0 0) 0x7fe15815c4e0 con 0x7fe1580cd8b0 2023-12-20T06:03:48.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe15cddb640 1 -- 172.21.15.32:0/137401260 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fe144003530 con 0x7fe1580cd8b0 2023-12-20T06:03:48.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe15d5dc640 1 -- 172.21.15.32:0/137401260 >> v1:172.21.15.39:6789/0 conn(0x7fe15815c100 legacy=0x7fe1580cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe15cddb640 1 -- 172.21.15.32:0/137401260 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 547596566 0 0) 0x7fe1580cc120 con 0x7fe15815c100 2023-12-20T06:03:48.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe15d5dc640 1 -- 172.21.15.32:0/137401260 >> v1:172.21.15.32:6789/0 conn(0x7fe1580cd8b0 legacy=0x7fe1580cec30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.530+0000 7fe15d5dc640 1 -- 172.21.15.32:0/137401260 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe1580cc400 con 0x7fe1580b1980 2023-12-20T06:03:48.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe15d5dc640 1 --2- 172.21.15.32:0/137401260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1580b1980 0x7fe1580cd370 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe1480097c0 tx=0x7fe14801c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b51c5edb5ec084ea server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:48.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe15cddb640 1 -- 172.21.15.32:0/137401260 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480ab670 con 0x7fe1580b1980 2023-12-20T06:03:48.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe15cddb640 1 -- 172.21.15.32:0/137401260 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1480ab7d0 con 0x7fe1580b1980 2023-12-20T06:03:48.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe15cddb640 1 -- 172.21.15.32:0/137401260 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480abb60 con 0x7fe1580b1980 2023-12-20T06:03:48.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe160041640 1 -- 172.21.15.32:0/137401260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1580b1980 msgr2=0x7fe1580cd370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe160041640 1 --2- 172.21.15.32:0/137401260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1580b1980 0x7fe1580cd370 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7fe1480097c0 tx=0x7fe14801c450 comp rx=0 tx=0).stop 2023-12-20T06:03:48.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe160041640 1 -- 172.21.15.32:0/137401260 shutdown_connections 2023-12-20T06:03:48.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe160041640 1 --2- 172.21.15.32:0/137401260 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe1580b1980 0x7fe1580cd370 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe160041640 1 -- 172.21.15.32:0/137401260 >> 172.21.15.32:0/137401260 conn(0x7fe158099d70 msgr2=0x7fe1580d0aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:48.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe160041640 1 -- 172.21.15.32:0/137401260 shutdown_connections 2023-12-20T06:03:48.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe160041640 1 -- 172.21.15.32:0/137401260 wait complete. 2023-12-20T06:03:48.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe160041640 1 Processor -- start 2023-12-20T06:03:48.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.538+0000 7fe160041640 1 -- start start 2023-12-20T06:03:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe160041640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1580b1980 0x7fe1580b5ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe160041640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1580cd8b0 0x7fe1581f4520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe160041640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15815c100 0x7fe1581f4a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe160041640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580cdd60 con 0x7fe1580cd8b0 2023-12-20T06:03:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe160041640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580b3c70 con 0x7fe15815c100 2023-12-20T06:03:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe160041640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe1580b3990 con 0x7fe1580b1980 2023-12-20T06:03:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe15d5dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1580cd8b0 0x7fe1581f4520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe15d5dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1580cd8b0 0x7fe1581f4520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:35998/0 (socket says 172.21.15.32:35998) 2023-12-20T06:03:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe15d5dc640 1 -- 172.21.15.32:0/3289634924 learned_addr learned my addr 172.21.15.32:0/3289634924 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe15e5de640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15815c100 0x7fe1581f4a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe15dddd640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1580b1980 0x7fe1580b5ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe15d5dc640 1 -- 172.21.15.32:0/3289634924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1580b1980 msgr2=0x7fe1580b5ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe15d5dc640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1580b1980 0x7fe1580b5ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe15d5dc640 1 -- 172.21.15.32:0/3289634924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15815c100 msgr2=0x7fe1581f4a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe15d5dc640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15815c100 0x7fe1581f4a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe15d5dc640 1 -- 172.21.15.32:0/3289634924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe148009420 con 0x7fe1580cd8b0 2023-12-20T06:03:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe15d5dc640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1580cd8b0 0x7fe1581f4520 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fe1480097c0 tx=0x7fe14801ac90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:48.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe14e7fc640 1 -- 172.21.15.32:0/3289634924 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480bf610 con 0x7fe1580cd8b0 2023-12-20T06:03:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe14e7fc640 1 -- 172.21.15.32:0/3289634924 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe1480bf770 con 0x7fe1580cd8b0 2023-12-20T06:03:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe160041640 1 -- 172.21.15.32:0/3289634924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe1580b40c0 con 0x7fe1580cd8b0 2023-12-20T06:03:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe14e7fc640 1 -- 172.21.15.32:0/3289634924 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe1480bfb00 con 0x7fe1580cd8b0 2023-12-20T06:03:48.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe160041640 1 -- 172.21.15.32:0/3289634924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe1580b4680 con 0x7fe1580cd8b0 2023-12-20T06:03:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.542+0000 7fe160041640 1 -- 172.21.15.32:0/3289634924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe15804e000 con 0x7fe1580cd8b0 2023-12-20T06:03:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.546+0000 7fe14e7fc640 1 -- 172.21.15.32:0/3289634924 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 139) v1 ==== 85433+0+0 (secure 0 0 0) 0x7fe1480b5400 con 0x7fe1580cd8b0 2023-12-20T06:03:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.546+0000 7fe14e7fc640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe13c065ca0 0x7fe13c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.546+0000 7fe14e7fc640 1 -- 172.21.15.32:0/3289634924 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7059+0+0 (secure 0 0 0) 0x7fe14811ee50 con 0x7fe1580cd8b0 2023-12-20T06:03:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.546+0000 7fe15dddd640 1 -- 172.21.15.32:0/3289634924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe13c065ca0 msgr2=0x7fe13c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:48.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.546+0000 7fe15dddd640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe13c065ca0 0x7fe13c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:48.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.554+0000 7fe14e7fc640 1 -- 172.21.15.32:0/3289634924 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe1480ebf90 con 0x7fe1580cd8b0 2023-12-20T06:03:48.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.714+0000 7fe160041640 1 -- 172.21.15.32:0/3289634924 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fe1580ce120 con 0x7fe1580cd8b0 2023-12-20T06:03:48.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.714+0000 7fe14e7fc640 1 -- 172.21.15.32:0/3289634924 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v235) v1 ==== 93+0+10 (secure 0 0 0) 0x7fe1480c4020 con 0x7fe1580cd8b0 2023-12-20T06:03:48.715 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:03:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 -- 172.21.15.32:0/3289634924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe13c065ca0 msgr2=0x7fe13c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe13c065ca0 0x7fe13c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 -- 172.21.15.32:0/3289634924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1580cd8b0 msgr2=0x7fe1581f4520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1580cd8b0 0x7fe1581f4520 secure :-1 s=READY pgs=2348 cs=0 l=1 rev1=1 crypto rx=0x7fe1480097c0 tx=0x7fe14801ac90 comp rx=0 tx=0).stop 2023-12-20T06:03:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 -- 172.21.15.32:0/3289634924 shutdown_connections 2023-12-20T06:03:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe13c065ca0 0x7fe13c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe15815c100 0x7fe1581f4a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe1580cd8b0 0x7fe1581f4520 unknown :-1 s=CLOSED pgs=2348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 --2- 172.21.15.32:0/3289634924 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe1580b1980 0x7fe1580b5ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 -- 172.21.15.32:0/3289634924 >> 172.21.15.32:0/3289634924 conn(0x7fe158099d70 msgr2=0x7fe15815b0b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 -- 172.21.15.32:0/3289634924 shutdown_connections 2023-12-20T06:03:48.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.718+0000 7fe160041640 1 -- 172.21.15.32:0/3289634924 wait complete. 2023-12-20T06:03:48.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:03:48.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b84b6640 1 Processor -- start 2023-12-20T06:03:48.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b84b6640 1 -- start start 2023-12-20T06:03:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b84b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3b015ce60 0x7fd3b00d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b84b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3b00d3130 0x7fd3b00d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b84b6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3b00d2650 con 0x7fd3b00d1300 2023-12-20T06:03:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b84b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3b00d35a0 con 0x7fd3b015ce60 2023-12-20T06:03:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b84b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b00d1a40 con 0x7fd3b00d3130 2023-12-20T06:03:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b6252640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3b015ce60 0x7fd3b00d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b5a51640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3b00d3130 0x7fd3b00d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b6a53640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd3b00d1300 0x7fd3b00b46e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55784/0 (socket says 172.21.15.32:55784) 2023-12-20T06:03:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b6252640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3b015ce60 0x7fd3b00d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47062/0 (socket says 172.21.15.32:47062) 2023-12-20T06:03:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b6a53640 1 -- 172.21.15.32:0/891285786 learned_addr learned my addr 172.21.15.32:0/891285786 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:48.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b6252640 1 -- 172.21.15.32:0/891285786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3b00d3130 msgr2=0x7fd3b00d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:48.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b6252640 1 --2- 172.21.15.32:0/891285786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3b00d3130 0x7fd3b00d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b6252640 1 -- 172.21.15.32:0/891285786 >> v1:172.21.15.32:6789/0 conn(0x7fd3b00d1300 legacy=0x7fd3b00b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b6252640 1 -- 172.21.15.32:0/891285786 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3b00d1d20 con 0x7fd3b015ce60 2023-12-20T06:03:48.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b5250640 1 -- 172.21.15.32:0/891285786 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3166544736 0 0) 0x7fd3b00d2650 con 0x7fd3b00d1300 2023-12-20T06:03:48.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.886+0000 7fd3b5a51640 1 --2- 172.21.15.32:0/891285786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3b00d3130 0x7fd3b00d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b6252640 1 --2- 172.21.15.32:0/891285786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b015ce60 0x7fd3b00d2bf0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0097c0 tx=0x7fd3ac01ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ef2995af6b045 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b5250640 1 -- 172.21.15.32:0/891285786 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0a8970 con 0x7fd3b015ce60 2023-12-20T06:03:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b5250640 1 -- 172.21.15.32:0/891285786 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3ac0a8ad0 con 0x7fd3b015ce60 2023-12-20T06:03:48.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b5250640 1 -- 172.21.15.32:0/891285786 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3ac0a8e60 con 0x7fd3b015ce60 2023-12-20T06:03:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b84b6640 1 -- 172.21.15.32:0/891285786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b015ce60 msgr2=0x7fd3b00d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b84b6640 1 --2- 172.21.15.32:0/891285786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b015ce60 0x7fd3b00d2bf0 secure :-1 s=READY pgs=2350 cs=0 l=1 rev1=1 crypto rx=0x7fd3ac0097c0 tx=0x7fd3ac01ec70 comp rx=0 tx=0).stop 2023-12-20T06:03:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b84b6640 1 -- 172.21.15.32:0/891285786 shutdown_connections 2023-12-20T06:03:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b84b6640 1 --2- 172.21.15.32:0/891285786 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3b00d3130 0x7fd3b00d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b84b6640 1 --2- 172.21.15.32:0/891285786 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b015ce60 0x7fd3b00d2bf0 unknown :-1 s=CLOSED pgs=2350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b84b6640 1 -- 172.21.15.32:0/891285786 >> 172.21.15.32:0/891285786 conn(0x7fd3b006b5e0 msgr2=0x7fd3b006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:48.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b84b6640 1 -- 172.21.15.32:0/891285786 shutdown_connections 2023-12-20T06:03:48.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.890+0000 7fd3b84b6640 1 -- 172.21.15.32:0/891285786 wait complete. 2023-12-20T06:03:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.894+0000 7fd3b84b6640 1 Processor -- start 2023-12-20T06:03:48.895 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.894+0000 7fd3b84b6640 1 -- start start 2023-12-20T06:03:48.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.894+0000 7fd3b84b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b00d1300 0x7fd3b00cb1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.894+0000 7fd3b84b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3b00d3130 0x7fd3b0158a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.894+0000 7fd3b84b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b015ce60 0x7fd3b01600f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.894+0000 7fd3b84b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0154270 con 0x7fd3b015ce60 2023-12-20T06:03:48.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.894+0000 7fd3b84b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0156ab0 con 0x7fd3b00d1300 2023-12-20T06:03:48.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.894+0000 7fd3b84b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd3b0156d90 con 0x7fd3b00d3130 2023-12-20T06:03:48.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b6a53640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b015ce60 0x7fd3b01600f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b6a53640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b015ce60 0x7fd3b01600f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47076/0 (socket says 172.21.15.32:47076) 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b6252640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b00d1300 0x7fd3b00cb1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b6a53640 1 -- 172.21.15.32:0/1975489379 learned_addr learned my addr 172.21.15.32:0/1975489379 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b5a51640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3b00d3130 0x7fd3b0158a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b6a53640 1 -- 172.21.15.32:0/1975489379 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3b00d3130 msgr2=0x7fd3b0158a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b6a53640 1 --2- 172.21.15.32:0/1975489379 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3b00d3130 0x7fd3b0158a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b6a53640 1 -- 172.21.15.32:0/1975489379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b00d1300 msgr2=0x7fd3b00cb1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b6a53640 1 --2- 172.21.15.32:0/1975489379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b00d1300 0x7fd3b00cb1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b6a53640 1 -- 172.21.15.32:0/1975489379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3a0009420 con 0x7fd3b015ce60 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b6a53640 1 --2- 172.21.15.32:0/1975489379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b015ce60 0x7fd3b01600f0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800c780 tx=0x7fd3a800cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3a6ffd640 1 -- 172.21.15.32:0/1975489379 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a800ce30 con 0x7fd3b015ce60 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3a6ffd640 1 -- 172.21.15.32:0/1975489379 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd3a800a640 con 0x7fd3b015ce60 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3a6ffd640 1 -- 172.21.15.32:0/1975489379 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd3a800aa30 con 0x7fd3b015ce60 2023-12-20T06:03:48.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b84b6640 1 -- 172.21.15.32:0/1975489379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd3ac009420 con 0x7fd3b015ce60 2023-12-20T06:03:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b84b6640 1 -- 172.21.15.32:0/1975489379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd3b0153f00 con 0x7fd3b015ce60 2023-12-20T06:03:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3b84b6640 1 -- 172.21.15.32:0/1975489379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd3b01543e0 con 0x7fd3b015ce60 2023-12-20T06:03:48.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.898+0000 7fd3a6ffd640 1 -- 172.21.15.32:0/1975489379 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 139) v1 ==== 85433+0+0 (secure 0 0 0) 0x7fd3a800ab90 con 0x7fd3b015ce60 2023-12-20T06:03:48.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.902+0000 7fd3a6ffd640 1 --2- 172.21.15.32:0/1975489379 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd394065ca0 0x7fd394068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:48.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.902+0000 7fd3a6ffd640 1 -- 172.21.15.32:0/1975489379 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7059+0+0 (secure 0 0 0) 0x7fd3a8082fd0 con 0x7fd3b015ce60 2023-12-20T06:03:48.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.902+0000 7fd3b6252640 1 -- 172.21.15.32:0/1975489379 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd394065ca0 msgr2=0x7fd394068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:48.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.902+0000 7fd3b6252640 1 --2- 172.21.15.32:0/1975489379 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd394065ca0 0x7fd394068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:48.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:48.906+0000 7fd3a6ffd640 1 -- 172.21.15.32:0/1975489379 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd3a80500f0 con 0x7fd3b015ce60 2023-12-20T06:03:48.983 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:48.982+0000 7ff84022d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:03:49.011 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:49.010+0000 7f0144f7e180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:03:49.046 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:49.042+0000 7ff84022d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:03:49.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.066+0000 7fd3b84b6640 1 -- 172.21.15.32:0/1975489379 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd3b004e000 con 0x7fd3b015ce60 2023-12-20T06:03:49.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.066+0000 7fd3a6ffd640 1 -- 172.21.15.32:0/1975489379 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v235) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd3a80180a0 con 0x7fd3b015ce60 2023-12-20T06:03:49.067 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:03:49.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 -- 172.21.15.32:0/1975489379 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd394065ca0 msgr2=0x7fd394068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 --2- 172.21.15.32:0/1975489379 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd394065ca0 0x7fd394068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 -- 172.21.15.32:0/1975489379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b015ce60 msgr2=0x7fd3b01600f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 --2- 172.21.15.32:0/1975489379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b015ce60 0x7fd3b01600f0 secure :-1 s=READY pgs=2351 cs=0 l=1 rev1=1 crypto rx=0x7fd3a800c780 tx=0x7fd3a800cc40 comp rx=0 tx=0).stop 2023-12-20T06:03:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 -- 172.21.15.32:0/1975489379 shutdown_connections 2023-12-20T06:03:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 --2- 172.21.15.32:0/1975489379 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd394065ca0 0x7fd394068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 --2- 172.21.15.32:0/1975489379 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3b015ce60 0x7fd3b01600f0 unknown :-1 s=CLOSED pgs=2351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 --2- 172.21.15.32:0/1975489379 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3b00d3130 0x7fd3b0158a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 --2- 172.21.15.32:0/1975489379 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd3b00d1300 0x7fd3b00cb1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 -- 172.21.15.32:0/1975489379 >> 172.21.15.32:0/1975489379 conn(0x7fd3b006b5e0 msgr2=0x7fd3b00b4190 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 -- 172.21.15.32:0/1975489379 shutdown_connections 2023-12-20T06:03:49.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.070+0000 7fd3b84b6640 1 -- 172.21.15.32:0/1975489379 wait complete. 2023-12-20T06:03:49.073 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:49.074+0000 7f0144f7e180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:03:49.089 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:03:49.105 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:49.102+0000 7ff84022d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:03:49.133 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:49.134+0000 7f0144f7e180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:03:49.197 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:49.194+0000 7ff84022d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:03:49.226 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:49.226+0000 7f0144f7e180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:03:49.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.238+0000 7f5fd0d8f640 1 Processor -- start 2023-12-20T06:03:49.240 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.238+0000 7f5fd0d8f640 1 -- start start 2023-12-20T06:03:49.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fd0d8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc15abb0 0x7f5fcc0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fd0d8f640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fcc0b0cb0 0x7f5fcc15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fd0d8f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5fcc15b6a0 con 0x7f5fcc070230 2023-12-20T06:03:49.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fd0d8f640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5fcc06e160 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fd0d8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc15c070 con 0x7f5fcc15abb0 2023-12-20T06:03:49.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fc3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fcc0b0cb0 0x7f5fcc15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:49.242 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fca59c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc15abb0 0x7f5fcc0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:49.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fc3fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f5fcc0b0cb0 0x7f5fcc15bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47082/0 (socket says 172.21.15.32:47082) 2023-12-20T06:03:49.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fc3fff640 1 -- 172.21.15.32:0/2114069423 learned_addr learned my addr 172.21.15.32:0/2114069423 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:49.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fc3fff640 1 -- 172.21.15.32:0/2114069423 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc15abb0 msgr2=0x7f5fcc0caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:49.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fc3fff640 1 --2- 172.21.15.32:0/2114069423 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc15abb0 0x7f5fcc0caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fc3fff640 1 -- 172.21.15.32:0/2114069423 >> v1:172.21.15.39:6789/0 conn(0x7f5fcc070230 legacy=0x7f5fcc06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:49.243 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.242+0000 7f5fc3fff640 1 -- 172.21.15.32:0/2114069423 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fcc06d8c0 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fc3fff640 1 --2- 172.21.15.32:0/2114069423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc0b0cb0 0x7f5fcc15bb30 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009870 tx=0x7f5fb401c540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3e445c58fb3a56d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:49.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fc9d9b640 1 -- 172.21.15.32:0/2114069423 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40a9400 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fc9d9b640 1 -- 172.21.15.32:0/2114069423 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb40a9560 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fc9d9b640 1 -- 172.21.15.32:0/2114069423 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40a98f0 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/2114069423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc0b0cb0 msgr2=0x7f5fcc15bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:49.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fd0d8f640 1 --2- 172.21.15.32:0/2114069423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc0b0cb0 0x7f5fcc15bb30 secure :-1 s=READY pgs=2352 cs=0 l=1 rev1=1 crypto rx=0x7f5fb4009870 tx=0x7f5fb401c540 comp rx=0 tx=0).stop 2023-12-20T06:03:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/2114069423 shutdown_connections 2023-12-20T06:03:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fd0d8f640 1 --2- 172.21.15.32:0/2114069423 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc0b0cb0 0x7f5fcc15bb30 unknown :-1 s=CLOSED pgs=2352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fd0d8f640 1 --2- 172.21.15.32:0/2114069423 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc15abb0 0x7f5fcc0caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/2114069423 >> 172.21.15.32:0/2114069423 conn(0x7f5fcc06a510 msgr2=0x7f5fcc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/2114069423 shutdown_connections 2023-12-20T06:03:49.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/2114069423 wait complete. 2023-12-20T06:03:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fd0d8f640 1 Processor -- start 2023-12-20T06:03:49.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.246+0000 7f5fd0d8f640 1 -- start start 2023-12-20T06:03:49.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fd0d8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc070230 0x7f5fcc159670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fd0d8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc0b0cb0 0x7f5fcc149210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fd0d8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc15abb0 0x7f5fcc0cccb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fd0d8f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc14df20 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fd0d8f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc14d310 con 0x7f5fcc070230 2023-12-20T06:03:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fd0d8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5fcc14d5f0 con 0x7f5fcc15abb0 2023-12-20T06:03:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc0b0cb0 0x7f5fcc149210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fca59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc070230 0x7f5fcc159670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc0b0cb0 0x7f5fcc149210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47086/0 (socket says 172.21.15.32:47086) 2023-12-20T06:03:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fca59c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc070230 0x7f5fcc159670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:42170/0 (socket says 172.21.15.32:42170) 2023-12-20T06:03:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc3fff640 1 -- 172.21.15.32:0/1269048787 learned_addr learned my addr 172.21.15.32:0/1269048787 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:49.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fcad9d640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc15abb0 0x7f5fcc0cccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc3fff640 1 -- 172.21.15.32:0/1269048787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc15abb0 msgr2=0x7f5fcc0cccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc3fff640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc15abb0 0x7f5fcc0cccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc3fff640 1 -- 172.21.15.32:0/1269048787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc070230 msgr2=0x7f5fcc159670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc3fff640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc070230 0x7f5fcc159670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc3fff640 1 -- 172.21.15.32:0/1269048787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5fb40094d0 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc3fff640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc0b0cb0 0x7f5fcc149210 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f5fb401a170 tx=0x7f5fb401ad80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc2ffd640 1 -- 172.21.15.32:0/1269048787 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40a9400 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc2ffd640 1 -- 172.21.15.32:0/1269048787 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5fb40a9e90 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fc2ffd640 1 -- 172.21.15.32:0/1269048787 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5fb40b1a00 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/1269048787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5fcc14d8d0 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/1269048787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5fcc159bb0 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.250+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/1269048787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5fcc04e000 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.254+0000 7f5fc2ffd640 1 -- 172.21.15.32:0/1269048787 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 139) v1 ==== 85433+0+0 (secure 0 0 0) 0x7f5fb40ba3a0 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.254+0000 7f5fc2ffd640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fac065ca0 0x7f5fac068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.254+0000 7f5fc2ffd640 1 -- 172.21.15.32:0/1269048787 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f5fb411def0 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.254+0000 7f5fca59c640 1 -- 172.21.15.32:0/1269048787 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fac065ca0 msgr2=0x7f5fac068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:49.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.254+0000 7f5fca59c640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fac065ca0 0x7f5fac068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:49.258 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:49.254+0000 7ff84022d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:03:49.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.262+0000 7f5fc2ffd640 1 -- 172.21.15.32:0/1269048787 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5fb40eaf00 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.291 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:49.290+0000 7f0144f7e180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:03:49.346 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:49.346+0000 7ff84022d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:03:49.380 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:49.378+0000 7f0144f7e180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:03:49.406 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:49.406+0000 7ff84022d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:03:49.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/1269048787 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f5fcc0b36e0 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fc2ffd640 1 -- 172.21.15.32:0/1269048787 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v235) v1 ==== 100+0+11 (secure 0 0 0) 0x7f5fb40a6030 con 0x7f5fcc0b0cb0 2023-12-20T06:03:49.422 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:03:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/1269048787 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fac065ca0 msgr2=0x7f5fac068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:49.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fac065ca0 0x7f5fac068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/1269048787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc0b0cb0 msgr2=0x7f5fcc149210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc0b0cb0 0x7f5fcc149210 secure :-1 s=READY pgs=2353 cs=0 l=1 rev1=1 crypto rx=0x7f5fb401a170 tx=0x7f5fb401ad80 comp rx=0 tx=0).stop 2023-12-20T06:03:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/1269048787 shutdown_connections 2023-12-20T06:03:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5fac065ca0 0x7f5fac068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5fcc15abb0 0x7f5fcc0cccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5fcc0b0cb0 0x7f5fcc149210 unknown :-1 s=CLOSED pgs=2353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 --2- 172.21.15.32:0/1269048787 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5fcc070230 0x7f5fcc159670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/1269048787 >> 172.21.15.32:0/1269048787 conn(0x7f5fcc06a510 msgr2=0x7f5fcc0b3c00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/1269048787 shutdown_connections 2023-12-20T06:03:49.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.422+0000 7f5fd0d8f640 1 -- 172.21.15.32:0/1269048787 wait complete. 2023-12-20T06:03:49.441 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:49.442+0000 7f0144f7e180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:03:49.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-20T06:03:49.560 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:49.558+0000 7ff84022d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:03:49.595 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:49.594+0000 7f0144f7e180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:03:49.622 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:49.622+0000 7ff84022d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:03:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.634+0000 7f30325db640 1 Processor -- start 2023-12-20T06:03:49.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.634+0000 7f30325db640 1 -- start start 2023-12-20T06:03:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f30325db640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f302c0d3300 0x7f302c0d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f30325db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f302c0d1300 0x7f302c0b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.638 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f30325db640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f302c0d2650 con 0x7f302c15ce60 2023-12-20T06:03:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f30325db640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f302c0d1a40 con 0x7f302c0d3300 2023-12-20T06:03:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f30325db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c0d1d20 con 0x7f302c0d1300 2023-12-20T06:03:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f302bfff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f302c15ce60 0x7f302c0d2bf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55788/0 (socket says 172.21.15.32:55788) 2023-12-20T06:03:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f302bfff640 1 -- 172.21.15.32:0/2114745561 learned_addr learned my addr 172.21.15.32:0/2114745561 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f302b7fe640 1 --2- 172.21.15.32:0/2114745561 >> v2:172.21.15.39:3300/0 conn(0x7f302c0d3300 0x7f302c0d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f3030b78640 1 --2- 172.21.15.32:0/2114745561 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f302c0d1300 0x7f302c0b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f302affd640 1 -- 172.21.15.32:0/2114745561 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3819599183 0 0) 0x7f302c0d2650 con 0x7f302c15ce60 2023-12-20T06:03:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f302affd640 1 -- 172.21.15.32:0/2114745561 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f3014003530 con 0x7f302c15ce60 2023-12-20T06:03:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f3030b78640 1 -- 172.21.15.32:0/2114745561 >> v2:172.21.15.39:3300/0 conn(0x7f302c0d3300 msgr2=0x7f302c0d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:49.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f3030b78640 1 --2- 172.21.15.32:0/2114745561 >> v2:172.21.15.39:3300/0 conn(0x7f302c0d3300 0x7f302c0d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f3030b78640 1 -- 172.21.15.32:0/2114745561 >> v1:172.21.15.32:6789/0 conn(0x7f302c15ce60 legacy=0x7f302c0d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:49.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.638+0000 7f3030b78640 1 -- 172.21.15.32:0/2114745561 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f302c0d2000 con 0x7f302c0d1300 2023-12-20T06:03:49.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f3030b78640 1 --2- 172.21.15.32:0/2114745561 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f302c0d1300 0x7f302c0b5e90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3024009d90 tx=0x7f302401f130 comp rx=0 tx=0).ready entity=mon.2 client_cookie=437d113181f498a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f302affd640 1 -- 172.21.15.32:0/2114745561 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240af600 con 0x7f302c0d1300 2023-12-20T06:03:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f302affd640 1 -- 172.21.15.32:0/2114745561 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30240ab050 con 0x7f302c0d1300 2023-12-20T06:03:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f302affd640 1 -- 172.21.15.32:0/2114745561 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30240af980 con 0x7f302c0d1300 2023-12-20T06:03:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f30325db640 1 -- 172.21.15.32:0/2114745561 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f302c0d1300 msgr2=0x7f302c0b5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f30325db640 1 --2- 172.21.15.32:0/2114745561 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f302c0d1300 0x7f302c0b5e90 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f3024009d90 tx=0x7f302401f130 comp rx=0 tx=0).stop 2023-12-20T06:03:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f30325db640 1 -- 172.21.15.32:0/2114745561 shutdown_connections 2023-12-20T06:03:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f30325db640 1 --2- 172.21.15.32:0/2114745561 >> v2:172.21.15.39:3300/0 conn(0x7f302c0d3300 0x7f302c0d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f30325db640 1 --2- 172.21.15.32:0/2114745561 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f302c0d1300 0x7f302c0b5e90 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f30325db640 1 -- 172.21.15.32:0/2114745561 >> 172.21.15.32:0/2114745561 conn(0x7f302c06b5e0 msgr2=0x7f302c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f30325db640 1 -- 172.21.15.32:0/2114745561 shutdown_connections 2023-12-20T06:03:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f30325db640 1 -- 172.21.15.32:0/2114745561 wait complete. 2023-12-20T06:03:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f30325db640 1 Processor -- start 2023-12-20T06:03:49.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.642+0000 7f30325db640 1 -- start start 2023-12-20T06:03:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f30325db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f302c0d1300 0x7f302c0c8d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f30325db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f302c0d3300 0x7f302c15fc00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f30325db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f302c15ce60 0x7f302c160140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f30325db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c14b490 con 0x7f302c0d3300 2023-12-20T06:03:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f30325db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f302c1607a0 con 0x7f302c0d1300 2023-12-20T06:03:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f30325db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f302c14ab60 con 0x7f302c15ce60 2023-12-20T06:03:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f302b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f302c0d3300 0x7f302c15fc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:49.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f302b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f302c0d3300 0x7f302c15fc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47096/0 (socket says 172.21.15.32:47096) 2023-12-20T06:03:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f302b7fe640 1 -- 172.21.15.32:0/3534417634 learned_addr learned my addr 172.21.15.32:0/3534417634 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f302bfff640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f302c0d1300 0x7f302c0c8d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f3030b78640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f302c15ce60 0x7f302c160140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f302b7fe640 1 -- 172.21.15.32:0/3534417634 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f302c15ce60 msgr2=0x7f302c160140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f302b7fe640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f302c15ce60 0x7f302c160140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f302b7fe640 1 -- 172.21.15.32:0/3534417634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f302c0d1300 msgr2=0x7f302c0c8d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f302b7fe640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f302c0d1300 0x7f302c0c8d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f302b7fe640 1 -- 172.21.15.32:0/3534417634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3024009a70 con 0x7f302c0d3300 2023-12-20T06:03:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f302b7fe640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f302c0d3300 0x7f302c15fc00 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f302000e900 tx=0x7f302000edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f3028ff9640 1 -- 172.21.15.32:0/3534417634 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3020009690 con 0x7f302c0d3300 2023-12-20T06:03:49.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f3028ff9640 1 -- 172.21.15.32:0/3534417634 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f302000ca00 con 0x7f302c0d3300 2023-12-20T06:03:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f3028ff9640 1 -- 172.21.15.32:0/3534417634 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f302000cd90 con 0x7f302c0d3300 2023-12-20T06:03:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f30325db640 1 -- 172.21.15.32:0/3534417634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f302c14ae40 con 0x7f302c0d3300 2023-12-20T06:03:49.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f30325db640 1 -- 172.21.15.32:0/3534417634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f302c0c9270 con 0x7f302c0d3300 2023-12-20T06:03:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.646+0000 7f30325db640 1 -- 172.21.15.32:0/3534417634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f302c14b600 con 0x7f302c0d3300 2023-12-20T06:03:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.650+0000 7f3028ff9640 1 -- 172.21.15.32:0/3534417634 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 139) v1 ==== 85433+0+0 (secure 0 0 0) 0x7f3020013470 con 0x7f302c0d3300 2023-12-20T06:03:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.650+0000 7f3028ff9640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3018065ca0 0x7f3018068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.650+0000 7f3028ff9640 1 -- 172.21.15.32:0/3534417634 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f3020083d30 con 0x7f302c0d3300 2023-12-20T06:03:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.650+0000 7f302bfff640 1 -- 172.21.15.32:0/3534417634 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3018065ca0 msgr2=0x7f3018068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:49.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.650+0000 7f302bfff640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3018065ca0 0x7f3018068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:49.658 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.658+0000 7f3028ff9640 1 -- 172.21.15.32:0/3534417634 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3020050e50 con 0x7f302c0d3300 2023-12-20T06:03:49.663 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:49.662+0000 7f0144f7e180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:03:49.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.830+0000 7f30325db640 1 -- 172.21.15.32:0/3534417634 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f302c04e000 con 0x7f302c0d3300 2023-12-20T06:03:49.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.830+0000 7f3028ff9640 1 -- 172.21.15.32:0/3534417634 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v235) v1 ==== 107+0+11 (secure 0 0 0) 0x7f30200170b0 con 0x7f302c0d3300 2023-12-20T06:03:49.831 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:03:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 -- 172.21.15.32:0/3534417634 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3018065ca0 msgr2=0x7f3018068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3018065ca0 0x7f3018068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 -- 172.21.15.32:0/3534417634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f302c0d3300 msgr2=0x7f302c15fc00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:49.834 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f302c0d3300 0x7f302c15fc00 secure :-1 s=READY pgs=2355 cs=0 l=1 rev1=1 crypto rx=0x7f302000e900 tx=0x7f302000edc0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 -- 172.21.15.32:0/3534417634 shutdown_connections 2023-12-20T06:03:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3018065ca0 0x7f3018068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f302c15ce60 0x7f302c160140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f302c0d3300 0x7f302c15fc00 unknown :-1 s=CLOSED pgs=2355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 --2- 172.21.15.32:0/3534417634 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f302c0d1300 0x7f302c0c8d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 -- 172.21.15.32:0/3534417634 >> 172.21.15.32:0/3534417634 conn(0x7f302c06b5e0 msgr2=0x7f302c0b5e50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 -- 172.21.15.32:0/3534417634 shutdown_connections 2023-12-20T06:03:49.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:49.834+0000 7f30325db640 1 -- 172.21.15.32:0/3534417634 wait complete. 2023-12-20T06:03:49.870 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-20T06:03:49.875 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:03:49.875 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:03:49.875 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:03:49.883 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:49.882+0000 7ff84022d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:03:49.915 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:03:49.915 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:03:49.915 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:03:49.924 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:49.922+0000 7f0144f7e180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:03:50.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f8090b8f640 1 Processor -- start 2023-12-20T06:03:50.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f8090b8f640 1 -- start start 2023-12-20T06:03:50.026 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:50.026+0000 7ff84022d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:03:50.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f8090b8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f808c0cdac0 0x7f808c0ce1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f8090b8f640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808c0ce700 con 0x7f808c070960 2023-12-20T06:03:50.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f8090b8f640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f808c069d20 con 0x7f808c159a60 2023-12-20T06:03:50.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f8090b8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0b2590 con 0x7f808c0cdac0 2023-12-20T06:03:50.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f808ad9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f808c070960 0x7f808c06a770 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55798/0 (socket says 172.21.15.32:55798) 2023-12-20T06:03:50.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f808ad9d640 1 -- 172.21.15.32:0/959781885 learned_addr learned my addr 172.21.15.32:0/959781885 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:50.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f8089d9b640 1 --2- 172.21.15.32:0/959781885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f808c0cdac0 0x7f808c0ce1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f8089d9b640 1 -- 172.21.15.32:0/959781885 >> v1:172.21.15.39:6789/0 conn(0x7f808c159a60 legacy=0x7f808c0b1e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f808959a640 1 -- 172.21.15.32:0/959781885 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1443872063 0 0) 0x7f808c0ce700 con 0x7f808c070960 2023-12-20T06:03:50.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f8089d9b640 1 -- 172.21.15.32:0/959781885 >> v1:172.21.15.32:6789/0 conn(0x7f808c070960 legacy=0x7f808c06a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.026+0000 7f8089d9b640 1 -- 172.21.15.32:0/959781885 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f808c0ce930 con 0x7f808c0cdac0 2023-12-20T06:03:50.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f8089d9b640 1 --2- 172.21.15.32:0/959781885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f808c0cdac0 0x7f808c0ce1c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8080009770 tx=0x7f808001c260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=237cdd75276b8a85 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:50.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f808959a640 1 -- 172.21.15.32:0/959781885 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80800ac440 con 0x7f808c0cdac0 2023-12-20T06:03:50.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f808959a640 1 -- 172.21.15.32:0/959781885 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80800ac5a0 con 0x7f808c0cdac0 2023-12-20T06:03:50.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f808959a640 1 -- 172.21.15.32:0/959781885 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80800ac930 con 0x7f808c0cdac0 2023-12-20T06:03:50.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f8090b8f640 1 -- 172.21.15.32:0/959781885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f808c0cdac0 msgr2=0x7f808c0ce1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f8090b8f640 1 --2- 172.21.15.32:0/959781885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f808c0cdac0 0x7f808c0ce1c0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8080009770 tx=0x7f808001c260 comp rx=0 tx=0).stop 2023-12-20T06:03:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f8090b8f640 1 -- 172.21.15.32:0/959781885 shutdown_connections 2023-12-20T06:03:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f8090b8f640 1 --2- 172.21.15.32:0/959781885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f808c0cdac0 0x7f808c0ce1c0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f8090b8f640 1 -- 172.21.15.32:0/959781885 >> 172.21.15.32:0/959781885 conn(0x7f808c0cf8d0 msgr2=0x7f808c0cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f8090b8f640 1 -- 172.21.15.32:0/959781885 shutdown_connections 2023-12-20T06:03:50.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f8090b8f640 1 -- 172.21.15.32:0/959781885 wait complete. 2023-12-20T06:03:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f8090b8f640 1 Processor -- start 2023-12-20T06:03:50.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.034+0000 7f8090b8f640 1 -- start start 2023-12-20T06:03:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8090b8f640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f808c070960 0x7f808c0afc90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8090b8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808c0cdac0 0x7f808c0c6a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8090b8f640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f808c159a60 0x7f808c0c5370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8090b8f640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0b0bf0 con 0x7f808c0cdac0 2023-12-20T06:03:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8090b8f640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0c2e90 con 0x7f808c159a60 2023-12-20T06:03:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8090b8f640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f808c0b0350 con 0x7f808c070960 2023-12-20T06:03:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8089d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808c0cdac0 0x7f808c0c6a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.039 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8089d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808c0cdac0 0x7f808c0c6a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47110/0 (socket says 172.21.15.32:47110) 2023-12-20T06:03:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8089d9b640 1 -- 172.21.15.32:0/2280674714 learned_addr learned my addr 172.21.15.32:0/2280674714 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f808a59c640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f808c070960 0x7f808c0afc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f808ad9d640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f808c159a60 0x7f808c0c5370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8089d9b640 1 -- 172.21.15.32:0/2280674714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f808c070960 msgr2=0x7f808c0afc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8089d9b640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f808c070960 0x7f808c0afc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8089d9b640 1 -- 172.21.15.32:0/2280674714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f808c159a60 msgr2=0x7f808c0c5370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8089d9b640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f808c159a60 0x7f808c0c5370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8089d9b640 1 -- 172.21.15.32:0/2280674714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8080009420 con 0x7f808c0cdac0 2023-12-20T06:03:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8089d9b640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808c0cdac0 0x7f808c0c6a70 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f8080018e70 tx=0x7f8080019000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f807affd640 1 -- 172.21.15.32:0/2280674714 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80800b5810 con 0x7f808c0cdac0 2023-12-20T06:03:50.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f807affd640 1 -- 172.21.15.32:0/2280674714 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80800b5990 con 0x7f808c0cdac0 2023-12-20T06:03:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8090b8f640 1 -- 172.21.15.32:0/2280674714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f808c0b0630 con 0x7f808c0cdac0 2023-12-20T06:03:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f807affd640 1 -- 172.21.15.32:0/2280674714 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80800b5d40 con 0x7f808c0cdac0 2023-12-20T06:03:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8090b8f640 1 -- 172.21.15.32:0/2280674714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f808c0c3130 con 0x7f808c0cdac0 2023-12-20T06:03:50.041 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.038+0000 7f8090b8f640 1 -- 172.21.15.32:0/2280674714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f808c0c3320 con 0x7f808c0cdac0 2023-12-20T06:03:50.042 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.042+0000 7f807affd640 1 -- 172.21.15.32:0/2280674714 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 139) v1 ==== 85433+0+0 (secure 0 0 0) 0x7f80800b3040 con 0x7f808c0cdac0 2023-12-20T06:03:50.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.042+0000 7f807affd640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f806c065ca0 0x7f806c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.042+0000 7f807affd640 1 -- 172.21.15.32:0/2280674714 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f808011daa0 con 0x7f808c0cdac0 2023-12-20T06:03:50.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.042+0000 7f808a59c640 1 -- 172.21.15.32:0/2280674714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f806c065ca0 msgr2=0x7f806c068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:50.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.042+0000 7f808a59c640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f806c065ca0 0x7f806c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:50.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.046+0000 7f807affd640 1 -- 172.21.15.32:0/2280674714 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f80800eabc0 con 0x7f808c0cdac0 2023-12-20T06:03:50.074 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:50.074+0000 7f0144f7e180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:03:50.161 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:50.158+0000 7ff84022d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:03:50.209 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:50.210+0000 7f0144f7e180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:03:50.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.218+0000 7f8090b8f640 1 -- 172.21.15.32:0/2280674714 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f808c0c3320 con 0x7f808c0cdac0 2023-12-20T06:03:50.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.218+0000 7f807affd640 1 -- 172.21.15.32:0/2280674714 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v235) v1 ==== 103+0+11 (secure 0 0 0) 0x7f80800ee1e0 con 0x7f808c0cdac0 2023-12-20T06:03:50.221 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:03:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 -- 172.21.15.32:0/2280674714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f806c065ca0 msgr2=0x7f806c068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f806c065ca0 0x7f806c068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 -- 172.21.15.32:0/2280674714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808c0cdac0 msgr2=0x7f808c0c6a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808c0cdac0 0x7f808c0c6a70 secure :-1 s=READY pgs=2357 cs=0 l=1 rev1=1 crypto rx=0x7f8080018e70 tx=0x7f8080019000 comp rx=0 tx=0).stop 2023-12-20T06:03:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 -- 172.21.15.32:0/2280674714 shutdown_connections 2023-12-20T06:03:50.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f806c065ca0 0x7f806c068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f808c159a60 0x7f808c0c5370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f808c0cdac0 0x7f808c0c6a70 unknown :-1 s=CLOSED pgs=2357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 --2- 172.21.15.32:0/2280674714 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f808c070960 0x7f808c0afc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 -- 172.21.15.32:0/2280674714 >> 172.21.15.32:0/2280674714 conn(0x7f808c0cf8d0 msgr2=0x7f808c156020 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 -- 172.21.15.32:0/2280674714 shutdown_connections 2023-12-20T06:03:50.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.222+0000 7f8090b8f640 1 -- 172.21.15.32:0/2280674714 wait complete. 2023-12-20T06:03:50.236 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:50.234+0000 7ff84022d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:03:50.243 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T06:03:50.290 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:50.286+0000 7f0144f7e180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:03:50.381 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:50.378+0000 7ff84022d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:03:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.430+0000 7f83f556d640 1 Processor -- start 2023-12-20T06:03:50.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.430+0000 7f83f556d640 1 -- start start 2023-12-20T06:03:50.432 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:50.430+0000 7f0144f7e180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:03:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83f556d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f83f01599e0 0x7f83f00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83f556d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f00b0d00 0x7f83f00701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83f556d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f83f00c98c0 con 0x7f83f00c90e0 2023-12-20T06:03:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83f556d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f83f00c9ba0 con 0x7f83f01599e0 2023-12-20T06:03:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83f556d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00c9e80 con 0x7f83f00b0d00 2023-12-20T06:03:50.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83eeffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f83f01599e0 0x7f83f00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83ef7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f00b0d00 0x7f83f00701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83e7fff640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f83f00c90e0 0x7f83f00b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:53542/0 (socket says 172.21.15.32:53542) 2023-12-20T06:03:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83e7fff640 1 -- 172.21.15.32:0/451406219 learned_addr learned my addr 172.21.15.32:0/451406219 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83ef7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f00b0d00 0x7f83f00701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49906/0 (socket says 172.21.15.32:49906) 2023-12-20T06:03:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83ee7fc640 1 -- 172.21.15.32:0/451406219 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3808617167 0 0) 0x7f83f00c98c0 con 0x7f83f00c90e0 2023-12-20T06:03:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83ee7fc640 1 -- 172.21.15.32:0/451406219 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f83dc003530 con 0x7f83f00c90e0 2023-12-20T06:03:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83ef7fe640 1 -- 172.21.15.32:0/451406219 >> v2:172.21.15.39:3300/0 conn(0x7f83f01599e0 msgr2=0x7f83f00aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83ef7fe640 1 --2- 172.21.15.32:0/451406219 >> v2:172.21.15.39:3300/0 conn(0x7f83f01599e0 0x7f83f00aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83ef7fe640 1 -- 172.21.15.32:0/451406219 >> v1:172.21.15.39:6789/0 conn(0x7f83f00c90e0 legacy=0x7f83f00b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83ef7fe640 1 -- 172.21.15.32:0/451406219 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83f00ca160 con 0x7f83f00b0d00 2023-12-20T06:03:50.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.434+0000 7f83eeffd640 1 --2- 172.21.15.32:0/451406219 >> v2:172.21.15.39:3300/0 conn(0x7f83f01599e0 0x7f83f00aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83ef7fe640 1 --2- 172.21.15.32:0/451406219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f00b0d00 0x7f83f00701c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009d90 tx=0x7f83e801c890 comp rx=0 tx=0).ready entity=mon.2 client_cookie=93482d760222e8ba server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83ee7fc640 1 -- 172.21.15.32:0/451406219 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80a5970 con 0x7f83f00b0d00 2023-12-20T06:03:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83ee7fc640 1 -- 172.21.15.32:0/451406219 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83e80a5ad0 con 0x7f83f00b0d00 2023-12-20T06:03:50.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83ee7fc640 1 -- 172.21.15.32:0/451406219 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83e80a5e40 con 0x7f83f00b0d00 2023-12-20T06:03:50.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83f556d640 1 -- 172.21.15.32:0/451406219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f00b0d00 msgr2=0x7f83f00701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83f556d640 1 --2- 172.21.15.32:0/451406219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f00b0d00 0x7f83f00701c0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f83e8009d90 tx=0x7f83e801c890 comp rx=0 tx=0).stop 2023-12-20T06:03:50.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83f556d640 1 -- 172.21.15.32:0/451406219 shutdown_connections 2023-12-20T06:03:50.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83f556d640 1 --2- 172.21.15.32:0/451406219 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f00b0d00 0x7f83f00701c0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83f556d640 1 --2- 172.21.15.32:0/451406219 >> v2:172.21.15.39:3300/0 conn(0x7f83f01599e0 0x7f83f00aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83f556d640 1 -- 172.21.15.32:0/451406219 >> 172.21.15.32:0/451406219 conn(0x7f83f006a320 msgr2=0x7f83f00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:50.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83f556d640 1 -- 172.21.15.32:0/451406219 shutdown_connections 2023-12-20T06:03:50.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83f556d640 1 -- 172.21.15.32:0/451406219 wait complete. 2023-12-20T06:03:50.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83f556d640 1 Processor -- start 2023-12-20T06:03:50.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83f556d640 1 -- start start 2023-12-20T06:03:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.438+0000 7f83f556d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f00b0d00 0x7f83f014b370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83f556d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f00c90e0 0x7f83f014c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83f556d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f01599e0 0x7f83f00c6060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83f556d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00ccb10 con 0x7f83f00c90e0 2023-12-20T06:03:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83f556d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00ccdb0 con 0x7f83f00b0d00 2023-12-20T06:03:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83f556d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f83f00cd090 con 0x7f83f01599e0 2023-12-20T06:03:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f00c90e0 0x7f83f014c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f00c90e0 0x7f83f014c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47112/0 (socket says 172.21.15.32:47112) 2023-12-20T06:03:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e7fff640 1 -- 172.21.15.32:0/3681936769 learned_addr learned my addr 172.21.15.32:0/3681936769 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:50.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83eeffd640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f00b0d00 0x7f83f014b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83ef7fe640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f01599e0 0x7f83f00c6060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e7fff640 1 -- 172.21.15.32:0/3681936769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f01599e0 msgr2=0x7f83f00c6060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e7fff640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f01599e0 0x7f83f00c6060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e7fff640 1 -- 172.21.15.32:0/3681936769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f00b0d00 msgr2=0x7f83f014b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e7fff640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f00b0d00 0x7f83f014b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e7fff640 1 -- 172.21.15.32:0/3681936769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f83e00093b0 con 0x7f83f00c90e0 2023-12-20T06:03:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83eeffd640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f00b0d00 0x7f83f014b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83ef7fe640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f01599e0 0x7f83f00c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e7fff640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f00c90e0 0x7f83f014c170 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f83d800c4d0 tx=0x7f83d800c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e77fe640 1 -- 172.21.15.32:0/3681936769 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8005cf0 con 0x7f83f00c90e0 2023-12-20T06:03:50.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e77fe640 1 -- 172.21.15.32:0/3681936769 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f83d8005e50 con 0x7f83f00c90e0 2023-12-20T06:03:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e77fe640 1 -- 172.21.15.32:0/3681936769 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f83d8007270 con 0x7f83f00c90e0 2023-12-20T06:03:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83f556d640 1 -- 172.21.15.32:0/3681936769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f83e8009a70 con 0x7f83f00c90e0 2023-12-20T06:03:50.445 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83f556d640 1 -- 172.21.15.32:0/3681936769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f83f00cd650 con 0x7f83f00c90e0 2023-12-20T06:03:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83f556d640 1 -- 172.21.15.32:0/3681936769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f83f00cd840 con 0x7f83f00c90e0 2023-12-20T06:03:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.442+0000 7f83e77fe640 1 -- 172.21.15.32:0/3681936769 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 139) v1 ==== 85433+0+0 (secure 0 0 0) 0x7f83d80073d0 con 0x7f83f00c90e0 2023-12-20T06:03:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.446+0000 7f83e77fe640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d0065ca0 0x7f83d0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.446+0000 7f83e77fe640 1 -- 172.21.15.32:0/3681936769 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f83d8082ff0 con 0x7f83f00c90e0 2023-12-20T06:03:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.446+0000 7f83eeffd640 1 -- 172.21.15.32:0/3681936769 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d0065ca0 msgr2=0x7f83d0068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:50.446 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.446+0000 7f83eeffd640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d0065ca0 0x7f83d0068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:50.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.450+0000 7f83e77fe640 1 -- 172.21.15.32:0/3681936769 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f83d80501b0 con 0x7f83f00c90e0 2023-12-20T06:03:50.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.622+0000 7f83f556d640 1 -- 172.21.15.32:0/3681936769 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7f83f004e000 con 0x7f83f00c90e0 2023-12-20T06:03:50.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83e77fe640 1 -- 172.21.15.32:0/3681936769 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v235) v1 ==== 81+0+575 (secure 0 0 0) 0x7f83d8017080 con 0x7f83f00c90e0 2023-12-20T06:03:50.625 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3054548475 2023-12-21T06:03:45.663020+0000 2023-12-20T06:03:50.625 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2026838481 2023-12-21T06:03:45.663020+0000 2023-12-20T06:03:50.625 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3886671554 2023-12-21T06:03:45.663020+0000 2023-12-20T06:03:50.625 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1847933543 2023-12-21T06:03:45.663020+0000 2023-12-20T06:03:50.625 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/4013366117 2023-12-21T06:03:45.663020+0000 2023-12-20T06:03:50.626 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/805068630 2023-12-21T06:03:45.663020+0000 2023-12-20T06:03:50.626 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/64546 2023-12-21T06:03:45.663020+0000 2023-12-20T06:03:50.626 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1823890749 2023-12-21T06:03:45.663020+0000 2023-12-20T06:03:50.626 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/64546 2023-12-21T06:03:45.663020+0000 2023-12-20T06:03:50.626 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2852735264 2023-12-21T06:03:45.663020+0000 2023-12-20T06:03:50.627 INFO:teuthology.orchestra.run.smithi032.stderr:listed 10 entries 2023-12-20T06:03:50.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 -- 172.21.15.32:0/3681936769 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d0065ca0 msgr2=0x7f83d0068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:50.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d0065ca0 0x7f83d0068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 -- 172.21.15.32:0/3681936769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f00c90e0 msgr2=0x7f83f014c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f00c90e0 0x7f83f014c170 secure :-1 s=READY pgs=2358 cs=0 l=1 rev1=1 crypto rx=0x7f83d800c4d0 tx=0x7f83d800c990 comp rx=0 tx=0).stop 2023-12-20T06:03:50.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 -- 172.21.15.32:0/3681936769 shutdown_connections 2023-12-20T06:03:50.627 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f83d0065ca0 0x7f83d0068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f83f01599e0 0x7f83f00c6060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f83f00c90e0 0x7f83f014c170 unknown :-1 s=CLOSED pgs=2358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 --2- 172.21.15.32:0/3681936769 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f83f00b0d00 0x7f83f014b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 -- 172.21.15.32:0/3681936769 >> 172.21.15.32:0/3681936769 conn(0x7f83f006a320 msgr2=0x7f83f0153a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:50.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 -- 172.21.15.32:0/3681936769 shutdown_connections 2023-12-20T06:03:50.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.626+0000 7f83f556d640 1 -- 172.21.15.32:0/3681936769 wait complete. 2023-12-20T06:03:50.647 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T06:03:50.647 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:50.647 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T06:03:50.676 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T06:03:50.676 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:03:50.677 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T06:03:50.730 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:03:50.730 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:03:50.740 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:03:50.740 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:03:50.740 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:03:50.782 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:03:50.782 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:03:50.782 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:03:50.792 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:03:50.792 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:50.961 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:50.958+0000 7ff84022d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:03:50.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f214d804640 1 Processor -- start 2023-12-20T06:03:50.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f214d804640 1 -- start start 2023-12-20T06:03:50.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f214d804640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21480cb760 0x7f21480700e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f214d804640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2148070700 con 0x7f214806cf80 2023-12-20T06:03:50.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f214d804640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f21480709e0 con 0x7f214815e0e0 2023-12-20T06:03:50.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f214d804640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f21480b1650 con 0x7f21480cb760 2023-12-20T06:03:50.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f21477fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f21480cb760 0x7f21480700e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f213ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f214806cf80 0x7f21480cb050 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55810/0 (socket says 172.21.15.32:55810) 2023-12-20T06:03:50.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f213ffff640 1 -- 172.21.15.32:0/4107134013 learned_addr learned my addr 172.21.15.32:0/4107134013 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f21477fe640 1 -- 172.21.15.32:0/4107134013 >> v1:172.21.15.39:6789/0 conn(0x7f214815e0e0 legacy=0x7f21480c5890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f21467fc640 1 -- 172.21.15.32:0/4107134013 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3595572416 0 0) 0x7f2148070700 con 0x7f214806cf80 2023-12-20T06:03:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f21477fe640 1 -- 172.21.15.32:0/4107134013 >> v1:172.21.15.32:6789/0 conn(0x7f214806cf80 legacy=0x7f21480cb050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.973 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.970+0000 7f21477fe640 1 -- 172.21.15.32:0/4107134013 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2148070cc0 con 0x7f21480cb760 2023-12-20T06:03:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.974+0000 7f21477fe640 1 --2- 172.21.15.32:0/4107134013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480cb760 0x7f21480700e0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f2140009d90 tx=0x7f214001c8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=170bf8c9c9c5a1af server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.974+0000 7f21467fc640 1 -- 172.21.15.32:0/4107134013 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21400a5780 con 0x7f21480cb760 2023-12-20T06:03:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.974+0000 7f21467fc640 1 -- 172.21.15.32:0/4107134013 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f21400a58e0 con 0x7f21480cb760 2023-12-20T06:03:50.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.974+0000 7f21467fc640 1 -- 172.21.15.32:0/4107134013 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21400a5c70 con 0x7f21480cb760 2023-12-20T06:03:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.974+0000 7f214d804640 1 -- 172.21.15.32:0/4107134013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480cb760 msgr2=0x7f21480700e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.974+0000 7f214d804640 1 --2- 172.21.15.32:0/4107134013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480cb760 0x7f21480700e0 secure :-1 s=READY pgs=2360 cs=0 l=1 rev1=1 crypto rx=0x7f2140009d90 tx=0x7f214001c8c0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.974+0000 7f214d804640 1 -- 172.21.15.32:0/4107134013 shutdown_connections 2023-12-20T06:03:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.974+0000 7f214d804640 1 --2- 172.21.15.32:0/4107134013 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480cb760 0x7f21480700e0 unknown :-1 s=CLOSED pgs=2360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.974+0000 7f214d804640 1 -- 172.21.15.32:0/4107134013 >> 172.21.15.32:0/4107134013 conn(0x7f2148069d20 msgr2=0x7f214806a130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:50.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.974+0000 7f214d804640 1 -- 172.21.15.32:0/4107134013 shutdown_connections 2023-12-20T06:03:50.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.974+0000 7f214d804640 1 -- 172.21.15.32:0/4107134013 wait complete. 2023-12-20T06:03:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.978+0000 7f214d804640 1 Processor -- start 2023-12-20T06:03:50.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.978+0000 7f214d804640 1 -- start start 2023-12-20T06:03:50.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.978+0000 7f214d804640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f214806cf80 0x7f2148153df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.978+0000 7f214d804640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480cb760 0x7f2148155820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.978+0000 7f214d804640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f214815e0e0 0x7f2148155d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f214d804640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2148154330 con 0x7f21480cb760 2023-12-20T06:03:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f214d804640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f21481637d0 con 0x7f214806cf80 2023-12-20T06:03:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f214d804640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2148163a70 con 0x7f214815e0e0 2023-12-20T06:03:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f213ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480cb760 0x7f2148155820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f2146ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f214806cf80 0x7f2148153df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f213ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480cb760 0x7f2148155820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47136/0 (socket says 172.21.15.32:47136) 2023-12-20T06:03:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f213ffff640 1 -- 172.21.15.32:0/265817504 learned_addr learned my addr 172.21.15.32:0/265817504 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:50.983 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f21477fe640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f214815e0e0 0x7f2148155d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f2146ffd640 1 -- 172.21.15.32:0/265817504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f214815e0e0 msgr2=0x7f2148155d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f2146ffd640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f214815e0e0 0x7f2148155d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f2146ffd640 1 -- 172.21.15.32:0/265817504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480cb760 msgr2=0x7f2148155820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f2146ffd640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480cb760 0x7f2148155820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f2146ffd640 1 -- 172.21.15.32:0/265817504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2140009a70 con 0x7f214806cf80 2023-12-20T06:03:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f213ffff640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480cb760 0x7f2148155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f2146ffd640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f214806cf80 0x7f2148153df0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2138000f70 tx=0x7f213800c820 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f2144ff9640 1 -- 172.21.15.32:0/265817504 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f21380052d0 con 0x7f214806cf80 2023-12-20T06:03:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f2144ff9640 1 -- 172.21.15.32:0/265817504 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2138005430 con 0x7f214806cf80 2023-12-20T06:03:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f214d804640 1 -- 172.21.15.32:0/265817504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2148163d50 con 0x7f214806cf80 2023-12-20T06:03:50.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f2144ff9640 1 -- 172.21.15.32:0/265817504 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2138003840 con 0x7f214806cf80 2023-12-20T06:03:50.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f214d804640 1 -- 172.21.15.32:0/265817504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2148164310 con 0x7f214806cf80 2023-12-20T06:03:50.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.982+0000 7f214d804640 1 -- 172.21.15.32:0/265817504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2148164530 con 0x7f214806cf80 2023-12-20T06:03:50.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.986+0000 7f2144ff9640 1 -- 172.21.15.32:0/265817504 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 139) v1 ==== 85433+0+0 (secure 0 0 0) 0x7f21380039a0 con 0x7f214806cf80 2023-12-20T06:03:50.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.986+0000 7f2144ff9640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2128065ca0 0x7f2128068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:50.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.986+0000 7f213ffff640 1 -- 172.21.15.32:0/265817504 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2128065ca0 msgr2=0x7f2128068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:50.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.986+0000 7f2144ff9640 1 -- 172.21.15.32:0/265817504 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7059+0+0 (secure 0 0 0) 0x7f2138082fc0 con 0x7f214806cf80 2023-12-20T06:03:50.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.986+0000 7f213ffff640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2128065ca0 0x7f2128068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:50.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:50.990+0000 7f2144ff9640 1 -- 172.21.15.32:0/265817504 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f21380501b0 con 0x7f214806cf80 2023-12-20T06:03:51.023 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.022+0000 7f0144f7e180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:03:51.052 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:51.050+0000 7ff84022d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:03:51.113 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.114+0000 7f0144f7e180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:03:51.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f214d804640 1 -- 172.21.15.32:0/265817504 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f2148074100 con 0x7f214806cf80 2023-12-20T06:03:51.182 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f2144ff9640 1 -- 172.21.15.32:0/265817504 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5909 (secure 0 0 0) 0x7f2138017090 con 0x7f214806cf80 2023-12-20T06:03:51.183 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:51.183 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:21.951240+0000","modified":"2023-12-20T06:03:24.980742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11301":{"gid":11301,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1623322556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1623322556},{"type":"v1","addr":"172.21.15.32:6837","nonce":1623322556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:34.513908+0000","modified":"2023-12-20T06:03:37.549965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11298":{"gid":11298,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6836/60134876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":60134876},{"type":"v1","addr":"172.21.15.32:6836","nonce":60134876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-20T06:03:51.183 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 118 2023-12-20T06:03:51.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f214d804640 1 -- 172.21.15.32:0/265817504 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2128065ca0 msgr2=0x7f2128068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:51.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f214d804640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2128065ca0 0x7f2128068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f214d804640 1 -- 172.21.15.32:0/265817504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f214806cf80 msgr2=0x7f2148153df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:51.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f214d804640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f214806cf80 0x7f2148153df0 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7f2138000f70 tx=0x7f213800c820 comp rx=0 tx=0).stop 2023-12-20T06:03:51.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f214d804640 1 -- 172.21.15.32:0/265817504 shutdown_connections 2023-12-20T06:03:51.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f214d804640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2128065ca0 0x7f2128068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f214d804640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f214815e0e0 0x7f2148155d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f214d804640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f21480cb760 0x7f2148155820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f214d804640 1 --2- 172.21.15.32:0/265817504 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f214806cf80 0x7f2148153df0 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.182+0000 7f214d804640 1 -- 172.21.15.32:0/265817504 >> 172.21.15.32:0/265817504 conn(0x7f2148069d20 msgr2=0x7f21480c5890 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:51.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.186+0000 7f214d804640 1 -- 172.21.15.32:0/265817504 shutdown_connections 2023-12-20T06:03:51.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.186+0000 7f214d804640 1 -- 172.21.15.32:0/265817504 wait complete. 2023-12-20T06:03:51.206 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:51.210 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:51.206+0000 7ff84022d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:03:51.272 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.270+0000 7f0144f7e180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:03:51.298 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:51.298+0000 7ff84022d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:03:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.350+0000 7fb905d25640 1 Processor -- start 2023-12-20T06:03:51.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.350+0000 7fb905d25640 1 -- start start 2023-12-20T06:03:51.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.350+0000 7fb905d25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb90015cdb0 0x7fb9000d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.350+0000 7fb905d25640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb9000d2740 0x7fb9000cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb905d25640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9000d1120 0x7fb9000cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb905d25640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb90015d190 con 0x7fb9000d1120 2023-12-20T06:03:51.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb905d25640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb9000d16a0 con 0x7fb9000d2740 2023-12-20T06:03:51.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb905d25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9000d1980 con 0x7fb90015cdb0 2023-12-20T06:03:51.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb8fffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb9000d1120 0x7fb9000cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:51.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb8feffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb9000d2740 0x7fb9000cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:51.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb8ff7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb90015cdb0 0x7fb9000d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:51.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb8feffd640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb9000d2740 0x7fb9000cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:42228/0 (socket says 172.21.15.32:42228) 2023-12-20T06:03:51.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb8feffd640 1 -- 172.21.15.32:0/1516460486 learned_addr learned my addr 172.21.15.32:0/1516460486 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:51.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb8feffd640 1 -- 172.21.15.32:0/1516460486 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb90015cdb0 msgr2=0x7fb9000d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:51.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb8feffd640 1 --2- 172.21.15.32:0/1516460486 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb90015cdb0 0x7fb9000d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb8feffd640 1 -- 172.21.15.32:0/1516460486 >> v2:172.21.15.32:3300/0 conn(0x7fb9000d1120 msgr2=0x7fb9000cfab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:51.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb8feffd640 1 --2- 172.21.15.32:0/1516460486 >> v2:172.21.15.32:3300/0 conn(0x7fb9000d1120 0x7fb9000cfab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb8feffd640 1 -- 172.21.15.32:0/1516460486 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9000d1c60 con 0x7fb9000d2740 2023-12-20T06:03:51.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.354+0000 7fb8feffd640 1 --2- 172.21.15.32:0/1516460486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9000d2740 0x7fb9000cf570 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb9000d3c60 tx=0x7fb8f401ecf0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=76623ccc7d83fa4a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:51.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb8fe7fc640 1 -- 172.21.15.32:0/1516460486 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40a89b0 con 0x7fb9000d2740 2023-12-20T06:03:51.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb8fe7fc640 1 -- 172.21.15.32:0/1516460486 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f40a8b10 con 0x7fb9000d2740 2023-12-20T06:03:51.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb8fe7fc640 1 -- 172.21.15.32:0/1516460486 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40a8ea0 con 0x7fb9000d2740 2023-12-20T06:03:51.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 -- 172.21.15.32:0/1516460486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9000d2740 msgr2=0x7fb9000cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:51.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 --2- 172.21.15.32:0/1516460486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9000d2740 0x7fb9000cf570 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fb9000d3c60 tx=0x7fb8f401ecf0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 -- 172.21.15.32:0/1516460486 shutdown_connections 2023-12-20T06:03:51.358 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 --2- 172.21.15.32:0/1516460486 >> v2:172.21.15.32:3300/0 conn(0x7fb9000d1120 0x7fb9000cfab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 --2- 172.21.15.32:0/1516460486 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9000d2740 0x7fb9000cf570 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 --2- 172.21.15.32:0/1516460486 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb90015cdb0 0x7fb9000d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 -- 172.21.15.32:0/1516460486 >> 172.21.15.32:0/1516460486 conn(0x7fb900065ee0 msgr2=0x7fb90006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:51.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 -- 172.21.15.32:0/1516460486 shutdown_connections 2023-12-20T06:03:51.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 -- 172.21.15.32:0/1516460486 wait complete. 2023-12-20T06:03:51.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 Processor -- start 2023-12-20T06:03:51.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 -- start start 2023-12-20T06:03:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.358+0000 7fb905d25640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9000d1120 0x7fb9000cb640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb905d25640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9000d2740 0x7fb90015d520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb905d25640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb90015cdb0 0x7fb90015efd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb905d25640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9000c8340 con 0x7fb90015cdb0 2023-12-20T06:03:51.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb905d25640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb900150ca0 con 0x7fb9000d1120 2023-12-20T06:03:51.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb905d25640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9000cbc70 con 0x7fb9000d2740 2023-12-20T06:03:51.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8feffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9000d2740 0x7fb90015d520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:51.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8fffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb90015cdb0 0x7fb90015efd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:51.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8ff7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9000d1120 0x7fb9000cb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:51.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8feffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9000d2740 0x7fb90015d520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:49930/0 (socket says 172.21.15.32:49930) 2023-12-20T06:03:51.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8feffd640 1 -- 172.21.15.32:0/1163060324 learned_addr learned my addr 172.21.15.32:0/1163060324 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:51.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8ff7fe640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9000d1120 0x7fb9000cb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:42230/0 (socket says 172.21.15.32:42230) 2023-12-20T06:03:51.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8feffd640 1 -- 172.21.15.32:0/1163060324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9000d1120 msgr2=0x7fb9000cb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:51.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8feffd640 1 --2- 172.21.15.32:0/1163060324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9000d1120 0x7fb9000cb640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8feffd640 1 -- 172.21.15.32:0/1163060324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb90015cdb0 msgr2=0x7fb90015efd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:51.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8feffd640 1 --2- 172.21.15.32:0/1163060324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb90015cdb0 0x7fb90015efd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8feffd640 1 -- 172.21.15.32:0/1163060324 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb8f40094d0 con 0x7fb9000d2740 2023-12-20T06:03:51.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8feffd640 1 --2- 172.21.15.32:0/1163060324 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9000d2740 0x7fb90015d520 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb8f401d3a0 tx=0x7fb8f401ba90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:51.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8e7fff640 1 -- 172.21.15.32:0/1163060324 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b0d40 con 0x7fb9000d2740 2023-12-20T06:03:51.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8e7fff640 1 -- 172.21.15.32:0/1163060324 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb8f40bb460 con 0x7fb9000d2740 2023-12-20T06:03:51.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb8e7fff640 1 -- 172.21.15.32:0/1163060324 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb8f40b0ea0 con 0x7fb9000d2740 2023-12-20T06:03:51.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb905d25640 1 -- 172.21.15.32:0/1163060324 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9000cbf50 con 0x7fb9000d2740 2023-12-20T06:03:51.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb905d25640 1 -- 172.21.15.32:0/1163060324 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9000cc510 con 0x7fb9000d2740 2023-12-20T06:03:51.365 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.362+0000 7f0144f7e180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:03:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.362+0000 7fb905d25640 1 -- 172.21.15.32:0/1163060324 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb90004e000 con 0x7fb9000d2740 2023-12-20T06:03:51.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.366+0000 7fb8e7fff640 1 -- 172.21.15.32:0/1163060324 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 139) v1 ==== 85433+0+0 (secure 0 0 0) 0x7fb8f40b83f0 con 0x7fb9000d2740 2023-12-20T06:03:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.366+0000 7fb8e7fff640 1 --2- 172.21.15.32:0/1163060324 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8ec065ca0 0x7fb8ec068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.366+0000 7fb8e7fff640 1 -- 172.21.15.32:0/1163060324 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7059+0+0 (secure 0 0 0) 0x7fb8f4121380 con 0x7fb9000d2740 2023-12-20T06:03:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.366+0000 7fb8ff7fe640 1 -- 172.21.15.32:0/1163060324 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8ec065ca0 msgr2=0x7fb8ec068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:51.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.366+0000 7fb8ff7fe640 1 --2- 172.21.15.32:0/1163060324 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8ec065ca0 0x7fb8ec068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:51.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.370+0000 7fb8e7fff640 1 -- 172.21.15.32:0/1163060324 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb8f40ee4a0 con 0x7fb9000d2740 2023-12-20T06:03:51.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.534+0000 7fb905d25640 1 -- 172.21.15.32:0/1163060324 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb900150de0 con 0x7fb9000d2740 2023-12-20T06:03:51.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.534+0000 7fb8e7fff640 1 -- 172.21.15.32:0/1163060324 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v235) v1 ==== 74+0+20431 (secure 0 0 0) 0x7fb8f40f1ac0 con 0x7fb9000d2740 2023-12-20T06:03:51.535 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:51.535 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":235,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:45.663060+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:03:20.399088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-20T06:03:20.926367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:03:32.174574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:03:33.509224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/3054548475":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2026838481":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/3886671554":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1847933543":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/4013366117":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/805068630":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6833/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1823890749":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6832/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2852735264":"2023-12-21T06:03:45.663020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.534+0000 7fb905d25640 1 -- 172.21.15.32:0/1163060324 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8ec065ca0 msgr2=0x7fb8ec068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.534+0000 7fb905d25640 1 --2- 172.21.15.32:0/1163060324 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8ec065ca0 0x7fb8ec068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.534+0000 7fb905d25640 1 -- 172.21.15.32:0/1163060324 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9000d2740 msgr2=0x7fb90015d520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.534+0000 7fb905d25640 1 --2- 172.21.15.32:0/1163060324 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9000d2740 0x7fb90015d520 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fb8f401d3a0 tx=0x7fb8f401ba90 comp rx=0 tx=0).stop 2023-12-20T06:03:51.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.534+0000 7fb905d25640 1 -- 172.21.15.32:0/1163060324 shutdown_connections 2023-12-20T06:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.534+0000 7fb905d25640 1 --2- 172.21.15.32:0/1163060324 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb8ec065ca0 0x7fb8ec068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.538+0000 7fb905d25640 1 --2- 172.21.15.32:0/1163060324 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb90015cdb0 0x7fb90015efd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.538+0000 7fb905d25640 1 --2- 172.21.15.32:0/1163060324 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9000d2740 0x7fb90015d520 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.538+0000 7fb905d25640 1 --2- 172.21.15.32:0/1163060324 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9000d1120 0x7fb9000cb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.538+0000 7fb905d25640 1 -- 172.21.15.32:0/1163060324 >> 172.21.15.32:0/1163060324 conn(0x7fb900065ee0 msgr2=0x7fb90015b3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.538+0000 7fb905d25640 1 -- 172.21.15.32:0/1163060324 shutdown_connections 2023-12-20T06:03:51.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.538+0000 7fb905d25640 1 -- 172.21.15.32:0/1163060324 wait complete. 2023-12-20T06:03:51.557 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 28, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T06:03:20.399088+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '229', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 29, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T06:03:20.926367+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '230', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.6299999952316284, 'score_stable': 1.6299999952316284, 'optimal_score': 1, 'raw_score_acting': 1.6299999952316284, 'raw_score_stable': 1.6299999952316284, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-20T06:03:32.174574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-20T06:03:33.509224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '234', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:03:51.558 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:03:51.638 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:51.638+0000 7ff84022d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:03:51.703 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:03:51.702+0000 7ff84022d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:03:51.704 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.702+0000 7f0144f7e180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:03:51.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca49af2640 1 Processor -- start 2023-12-20T06:03:51.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca49af2640 1 -- start start 2023-12-20T06:03:51.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca49af2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fca440b0760 0x7fca440b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca49af2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fca440cc4b0 0x7fca440cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca49af2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fca440b1100 con 0x7fca44157480 2023-12-20T06:03:51.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca49af2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca440c9f50 con 0x7fca440cc4b0 2023-12-20T06:03:51.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca49af2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fca440cde20 con 0x7fca440b0760 2023-12-20T06:03:51.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca437fe640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fca44157480 0x7fca440c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:53554/0 (socket says 172.21.15.32:53554) 2023-12-20T06:03:51.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca437fe640 1 -- 172.21.15.32:0/26740918 learned_addr learned my addr 172.21.15.32:0/26740918 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:51.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca42ffd640 1 --2- 172.21.15.32:0/26740918 >> v2:172.21.15.39:3300/0 conn(0x7fca440b0760 0x7fca440b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:51.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca43fff640 1 --2- 172.21.15.32:0/26740918 >> v2:172.21.15.32:3300/0 conn(0x7fca440cc4b0 0x7fca440cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:51.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca427fc640 1 -- 172.21.15.32:0/26740918 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2350673448 0 0) 0x7fca440b1100 con 0x7fca44157480 2023-12-20T06:03:51.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca427fc640 1 -- 172.21.15.32:0/26740918 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fca2c003530 con 0x7fca44157480 2023-12-20T06:03:51.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca42ffd640 1 -- 172.21.15.32:0/26740918 >> v2:172.21.15.32:3300/0 conn(0x7fca440cc4b0 msgr2=0x7fca440cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:51.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca42ffd640 1 --2- 172.21.15.32:0/26740918 >> v2:172.21.15.32:3300/0 conn(0x7fca440cc4b0 0x7fca440cc8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca42ffd640 1 -- 172.21.15.32:0/26740918 >> v1:172.21.15.39:6789/0 conn(0x7fca44157480 legacy=0x7fca440c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:51.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca42ffd640 1 -- 172.21.15.32:0/26740918 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca440c8e30 con 0x7fca440b0760 2023-12-20T06:03:51.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.706+0000 7fca43fff640 1 --2- 172.21.15.32:0/26740918 >> v2:172.21.15.32:3300/0 conn(0x7fca440cc4b0 0x7fca440cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:51.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca42ffd640 1 --2- 172.21.15.32:0/26740918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440b0760 0x7fca440b0bc0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fca4406e0d0 tx=0x7fca3801c3a0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e0680b4c1bf9cd6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:51.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca427fc640 1 -- 172.21.15.32:0/26740918 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380ac600 con 0x7fca440b0760 2023-12-20T06:03:51.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca427fc640 1 -- 172.21.15.32:0/26740918 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca380a8050 con 0x7fca440b0760 2023-12-20T06:03:51.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca427fc640 1 -- 172.21.15.32:0/26740918 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380ac980 con 0x7fca440b0760 2023-12-20T06:03:51.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca49af2640 1 -- 172.21.15.32:0/26740918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440b0760 msgr2=0x7fca440b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:51.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca49af2640 1 --2- 172.21.15.32:0/26740918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440b0760 0x7fca440b0bc0 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fca4406e0d0 tx=0x7fca3801c3a0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca49af2640 1 -- 172.21.15.32:0/26740918 shutdown_connections 2023-12-20T06:03:51.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca49af2640 1 --2- 172.21.15.32:0/26740918 >> v2:172.21.15.32:3300/0 conn(0x7fca440cc4b0 0x7fca440cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca49af2640 1 --2- 172.21.15.32:0/26740918 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440b0760 0x7fca440b0bc0 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca49af2640 1 -- 172.21.15.32:0/26740918 >> 172.21.15.32:0/26740918 conn(0x7fca4406a510 msgr2=0x7fca440ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca49af2640 1 -- 172.21.15.32:0/26740918 shutdown_connections 2023-12-20T06:03:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.710+0000 7fca49af2640 1 -- 172.21.15.32:0/26740918 wait complete. 2023-12-20T06:03:51.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca49af2640 1 Processor -- start 2023-12-20T06:03:51.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca49af2640 1 -- start start 2023-12-20T06:03:51.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca49af2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440b0760 0x7fca44142bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca49af2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca440cc4b0 0x7fca44070130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca49af2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca44157480 0x7fca44070670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca49af2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca44149720 con 0x7fca440cc4b0 2023-12-20T06:03:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca49af2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fca4414bc40 con 0x7fca440b0760 2023-12-20T06:03:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca49af2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fca4414b030 con 0x7fca44157480 2023-12-20T06:03:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca42ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca440cc4b0 0x7fca44070130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca42ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca440cc4b0 0x7fca44070130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47164/0 (socket says 172.21.15.32:47164) 2023-12-20T06:03:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca42ffd640 1 -- 172.21.15.32:0/297022280 learned_addr learned my addr 172.21.15.32:0/297022280 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca437fe640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440b0760 0x7fca44142bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca43fff640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca44157480 0x7fca44070670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca42ffd640 1 -- 172.21.15.32:0/297022280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca44157480 msgr2=0x7fca44070670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca42ffd640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca44157480 0x7fca44070670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca42ffd640 1 -- 172.21.15.32:0/297022280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440b0760 msgr2=0x7fca44142bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca42ffd640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440b0760 0x7fca44142bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca42ffd640 1 -- 172.21.15.32:0/297022280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fca3c009a70 con 0x7fca440cc4b0 2023-12-20T06:03:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca42ffd640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca440cc4b0 0x7fca44070130 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fca38018fb0 tx=0x7fca3801abe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca2bfff640 1 -- 172.21.15.32:0/297022280 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca38006e20 con 0x7fca440cc4b0 2023-12-20T06:03:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca2bfff640 1 -- 172.21.15.32:0/297022280 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fca380b7450 con 0x7fca440cc4b0 2023-12-20T06:03:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca49af2640 1 -- 172.21.15.32:0/297022280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fca38009420 con 0x7fca440cc4b0 2023-12-20T06:03:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.714+0000 7fca2bfff640 1 -- 172.21.15.32:0/297022280 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fca380b77c0 con 0x7fca440cc4b0 2023-12-20T06:03:51.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.718+0000 7fca49af2640 1 -- 172.21.15.32:0/297022280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fca4414b5f0 con 0x7fca440cc4b0 2023-12-20T06:03:51.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.718+0000 7fca49af2640 1 -- 172.21.15.32:0/297022280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fca4414c0b0 con 0x7fca440cc4b0 2023-12-20T06:03:51.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.718+0000 7fca2bfff640 1 -- 172.21.15.32:0/297022280 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 139) v1 ==== 85433+0+0 (secure 0 0 0) 0x7fca380b3070 con 0x7fca440cc4b0 2023-12-20T06:03:51.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.718+0000 7fca2bfff640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca30065ca0 0x7fca30068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:51.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.718+0000 7fca2bfff640 1 -- 172.21.15.32:0/297022280 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(235..235 src has 1..235) v4 ==== 7059+0+0 (secure 0 0 0) 0x7fca3811dcd0 con 0x7fca440cc4b0 2023-12-20T06:03:51.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.718+0000 7fca437fe640 1 -- 172.21.15.32:0/297022280 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca30065ca0 msgr2=0x7fca30068160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:03:51.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.718+0000 7fca437fe640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca30065ca0 0x7fca30068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:03:51.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.722+0000 7fca2bfff640 1 -- 172.21.15.32:0/297022280 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fca380ead00 con 0x7fca440cc4b0 2023-12-20T06:03:51.771 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.770+0000 7f0144f7e180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:03:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.782+0000 7fca2bfff640 1 -- 172.21.15.32:0/297022280 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 140) v1 ==== 84939+0+0 (secure 0 0 0) 0x7fca380eb0d0 con 0x7fca440cc4b0 2023-12-20T06:03:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.782+0000 7fca2bfff640 1 -- 172.21.15.32:0/297022280 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca30065ca0 msgr2=0x7fca30068160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:03:51.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.782+0000 7fca2bfff640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca30065ca0 0x7fca30068160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.858 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.858+0000 7f00cd6f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.858 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.858+0000 7f00cd6f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.858 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.858+0000 7f00cd6f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.858 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.858+0000 7f00cd6f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.858 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.858+0000 7f00cd6f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.859 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.858+0000 7f00cf6fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.859 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.858+0000 7f00cf6fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.859 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.858+0000 7f00cf6fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.859 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.858+0000 7f00cf6fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.859 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.858+0000 7f00cf6fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.871 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.870+0000 7f00cd6f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.871 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.870+0000 7f00cd6f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.871 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.870+0000 7f00cd6f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.871 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.870+0000 7f00cd6f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.871 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.870+0000 7f00cd6f8640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.871 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.870+0000 7f00cf6fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.872 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.870+0000 7f00cf6fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.872 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.870+0000 7f00cf6fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.872 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.870+0000 7f00cf6fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.872 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:03:51.870+0000 7f00cf6fc640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:03:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca49af2640 1 -- 172.21.15.32:0/297022280 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fca4404e000 con 0x7fca440cc4b0 2023-12-20T06:03:51.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca2bfff640 1 -- 172.21.15.32:0/297022280 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v236) v1 ==== 93+0+10 (secure 0 0 0) 0x7fca38103b60 con 0x7fca440cc4b0 2023-12-20T06:03:51.909 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:03:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca49af2640 1 -- 172.21.15.32:0/297022280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca440cc4b0 msgr2=0x7fca44070130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca49af2640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca440cc4b0 0x7fca44070130 secure :-1 s=READY pgs=2361 cs=0 l=1 rev1=1 crypto rx=0x7fca38018fb0 tx=0x7fca3801abe0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca49af2640 1 -- 172.21.15.32:0/297022280 shutdown_connections 2023-12-20T06:03:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca49af2640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fca30065ca0 0x7fca30068160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca49af2640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fca44157480 0x7fca44070670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca49af2640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fca440cc4b0 0x7fca44070130 unknown :-1 s=CLOSED pgs=2361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca49af2640 1 --2- 172.21.15.32:0/297022280 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fca440b0760 0x7fca44142bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca49af2640 1 -- 172.21.15.32:0/297022280 >> 172.21.15.32:0/297022280 conn(0x7fca4406a510 msgr2=0x7fca4406d4f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca49af2640 1 -- 172.21.15.32:0/297022280 shutdown_connections 2023-12-20T06:03:51.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:51.910+0000 7fca49af2640 1 -- 172.21.15.32:0/297022280 wait complete. 2023-12-20T06:03:51.931 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:03:52.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3f28c640 1 Processor -- start 2023-12-20T06:03:52.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3f28c640 1 -- start start 2023-12-20T06:03:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3f28c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7feb380b0760 0x7feb380b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3f28c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380cc4b0 0x7feb380cc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3f28c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7feb380b1100 con 0x7feb38157480 2023-12-20T06:03:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3f28c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7feb380c9f50 con 0x7feb380b0760 2023-12-20T06:03:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3f28c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb380cde20 con 0x7feb380cc4b0 2023-12-20T06:03:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3d028640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7feb38157480 0x7feb380c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55822/0 (socket says 172.21.15.32:55822) 2023-12-20T06:03:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3d028640 1 -- 172.21.15.32:0/1712209364 learned_addr learned my addr 172.21.15.32:0/1712209364 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:52.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2816338143 0 0) 0x7feb380b1100 con 0x7feb38157480 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3c827640 1 --2- 172.21.15.32:0/1712209364 >> v2:172.21.15.39:3300/0 conn(0x7feb380b0760 0x7feb380b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3d829640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380cc4b0 0x7feb380cc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7feb24005740 con 0x7feb38157480 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 783944044 0 0) 0x7feb24005740 con 0x7feb38157480 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380cc4b0 msgr2=0x7feb380cc920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380cc4b0 0x7feb380cc920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 >> v2:172.21.15.39:3300/0 conn(0x7feb380b0760 msgr2=0x7feb380b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 --2- 172.21.15.32:0/1712209364 >> v2:172.21.15.39:3300/0 conn(0x7feb380b0760 0x7feb380b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb380c8e30 con 0x7feb38157480 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7feb340016a0 con 0x7feb38157480 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3c827640 1 --2- 172.21.15.32:0/1712209364 >> v2:172.21.15.39:3300/0 conn(0x7feb380b0760 0x7feb380b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 >> v1:172.21.15.32:6789/0 conn(0x7feb38157480 legacy=0x7feb380c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb24005b90 0x7feb24005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240069e0 0x7feb24006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380cc4b0 0x7feb240073a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb380c9f50 con 0x7feb24005b90 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb380cde20 con 0x7feb240069e0 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb2400dc50 con 0x7feb380cc4b0 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3c827640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240069e0 0x7feb24006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3d028640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb24005b90 0x7feb24005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3d829640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380cc4b0 0x7feb240073a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3c827640 1 -- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380cc4b0 msgr2=0x7feb240073a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3c827640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380cc4b0 0x7feb240073a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3c827640 1 -- 172.21.15.32:0/1712209364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb24005b90 msgr2=0x7feb24005fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3c827640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb24005b90 0x7feb24005fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3c827640 1 -- 172.21.15.32:0/1712209364 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb24005740 con 0x7feb240069e0 2023-12-20T06:03:52.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.078+0000 7feb3d028640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb24005b90 0x7feb24005fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.082+0000 7feb3c827640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240069e0 0x7feb24006e60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7feb280098a0 tx=0x7feb2801c4e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=b68024d3d3e5e9ac server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.082+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280aa670 con 0x7feb240069e0 2023-12-20T06:03:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.082+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb280aa7d0 con 0x7feb240069e0 2023-12-20T06:03:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.082+0000 7feb2ffff640 1 -- 172.21.15.32:0/1712209364 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280aab40 con 0x7feb240069e0 2023-12-20T06:03:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.082+0000 7feb3f28c640 1 -- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240069e0 msgr2=0x7feb24006e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.082+0000 7feb3f28c640 1 --2- 172.21.15.32:0/1712209364 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240069e0 0x7feb24006e60 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7feb280098a0 tx=0x7feb2801c4e0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.085 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.082+0000 7feb3d829640 1 -- 172.21.15.32:0/1712209364 reap_dead start 2023-12-20T06:03:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.082+0000 7feb3f28c640 1 -- 172.21.15.32:0/1712209364 shutdown_connections 2023-12-20T06:03:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.082+0000 7feb3f28c640 1 -- 172.21.15.32:0/1712209364 >> 172.21.15.32:0/1712209364 conn(0x7feb3806a510 msgr2=0x7feb380ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3f28c640 1 -- 172.21.15.32:0/1712209364 shutdown_connections 2023-12-20T06:03:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3f28c640 1 -- 172.21.15.32:0/1712209364 wait complete. 2023-12-20T06:03:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3f28c640 1 Processor -- start 2023-12-20T06:03:52.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3f28c640 1 -- start start 2023-12-20T06:03:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3f28c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240069e0 0x7feb381f9290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3f28c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb3814f6c0 0x7feb380c64c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3f28c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380c6a00 0x7feb38158c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3f28c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb380c6ea0 con 0x7feb3814f6c0 2023-12-20T06:03:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3f28c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7feb381f87d0 con 0x7feb240069e0 2023-12-20T06:03:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3f28c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7feb381f8ab0 con 0x7feb380c6a00 2023-12-20T06:03:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3c827640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb3814f6c0 0x7feb380c64c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3c827640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb3814f6c0 0x7feb380c64c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47294/0 (socket says 172.21.15.32:47294) 2023-12-20T06:03:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3c827640 1 -- 172.21.15.32:0/848186477 learned_addr learned my addr 172.21.15.32:0/848186477 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3d028640 1 --2- 172.21.15.32:0/848186477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240069e0 0x7feb381f9290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3d829640 1 --2- 172.21.15.32:0/848186477 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380c6a00 0x7feb38158c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3c827640 1 -- 172.21.15.32:0/848186477 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380c6a00 msgr2=0x7feb38158c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3c827640 1 --2- 172.21.15.32:0/848186477 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380c6a00 0x7feb38158c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3c827640 1 -- 172.21.15.32:0/848186477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240069e0 msgr2=0x7feb381f9290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3c827640 1 --2- 172.21.15.32:0/848186477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240069e0 0x7feb381f9290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3c827640 1 -- 172.21.15.32:0/848186477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7feb381f8d90 con 0x7feb3814f6c0 2023-12-20T06:03:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.086+0000 7feb3c827640 1 --2- 172.21.15.32:0/848186477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb3814f6c0 0x7feb380c64c0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7feb280098a0 tx=0x7feb24007a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.090+0000 7feb2dffb640 1 -- 172.21.15.32:0/848186477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb280aac00 con 0x7feb3814f6c0 2023-12-20T06:03:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.090+0000 7feb2dffb640 1 -- 172.21.15.32:0/848186477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7feb280aad60 con 0x7feb3814f6c0 2023-12-20T06:03:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.090+0000 7feb2dffb640 1 -- 172.21.15.32:0/848186477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7feb28004260 con 0x7feb3814f6c0 2023-12-20T06:03:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.090+0000 7feb3f28c640 1 -- 172.21.15.32:0/848186477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7feb380c7180 con 0x7feb3814f6c0 2023-12-20T06:03:52.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.090+0000 7feb3f28c640 1 -- 172.21.15.32:0/848186477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7feb381534e0 con 0x7feb3814f6c0 2023-12-20T06:03:52.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.090+0000 7feb3f28c640 1 -- 172.21.15.32:0/848186477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7feb381536d0 con 0x7feb3814f6c0 2023-12-20T06:03:52.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.090+0000 7feb2dffb640 1 -- 172.21.15.32:0/848186477 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 140) v1 ==== 84939+0+0 (secure 0 0 0) 0x7feb280043c0 con 0x7feb3814f6c0 2023-12-20T06:03:52.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.090+0000 7feb2dffb640 1 -- 172.21.15.32:0/848186477 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7403+0+0 (secure 0 0 0) 0x7feb2811d450 con 0x7feb3814f6c0 2023-12-20T06:03:52.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.094+0000 7feb2dffb640 1 -- 172.21.15.32:0/848186477 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7feb381536d0 con 0x7feb3814f6c0 2023-12-20T06:03:52.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.250+0000 7feb3f28c640 1 -- 172.21.15.32:0/848186477 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7feb381536d0 con 0x7feb3814f6c0 2023-12-20T06:03:52.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.254+0000 7feb2dffb640 1 -- 172.21.15.32:0/848186477 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v236) v1 ==== 104+0+11 (secure 0 0 0) 0x7feb381536d0 con 0x7feb3814f6c0 2023-12-20T06:03:52.253 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:03:52.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.254+0000 7feb3f28c640 1 -- 172.21.15.32:0/848186477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb3814f6c0 msgr2=0x7feb380c64c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.254+0000 7feb3f28c640 1 --2- 172.21.15.32:0/848186477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb3814f6c0 0x7feb380c64c0 secure :-1 s=READY pgs=2372 cs=0 l=1 rev1=1 crypto rx=0x7feb280098a0 tx=0x7feb24007a10 comp rx=0 tx=0).stop 2023-12-20T06:03:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.254+0000 7feb3f28c640 1 -- 172.21.15.32:0/848186477 shutdown_connections 2023-12-20T06:03:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.254+0000 7feb3f28c640 1 --2- 172.21.15.32:0/848186477 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7feb380c6a00 0x7feb38158c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.254+0000 7feb3f28c640 1 --2- 172.21.15.32:0/848186477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7feb3814f6c0 0x7feb380c64c0 unknown :-1 s=CLOSED pgs=2372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.254+0000 7feb3f28c640 1 --2- 172.21.15.32:0/848186477 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7feb240069e0 0x7feb381f9290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.254+0000 7feb3f28c640 1 -- 172.21.15.32:0/848186477 >> 172.21.15.32:0/848186477 conn(0x7feb3806a510 msgr2=0x7feb38154150 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.254+0000 7feb3f28c640 1 -- 172.21.15.32:0/848186477 shutdown_connections 2023-12-20T06:03:52.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.254+0000 7feb3f28c640 1 -- 172.21.15.32:0/848186477 wait complete. 2023-12-20T06:03:52.275 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:03:52.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41c09db640 1 Processor -- start 2023-12-20T06:03:52.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41c09db640 1 -- start start 2023-12-20T06:03:52.423 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41c09db640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f41bc06d240 0x7f41bc0ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41c09db640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc06c480 con 0x7f41bc070490 2023-12-20T06:03:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41c09db640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f41bc06c760 con 0x7f41bc1603e0 2023-12-20T06:03:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41c09db640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f41bc06ca40 con 0x7f41bc06d240 2023-12-20T06:03:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41ba59c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f41bc070490 0x7f41bc15fcd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55828/0 (socket says 172.21.15.32:55828) 2023-12-20T06:03:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41ba59c640 1 -- 172.21.15.32:0/3128887129 learned_addr learned my addr 172.21.15.32:0/3128887129 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:52.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41b3fff640 1 --2- 172.21.15.32:0/3128887129 >> v2:172.21.15.39:3300/0 conn(0x7f41bc06d240 0x7f41bc0ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41b9d9b640 1 -- 172.21.15.32:0/3128887129 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4033305510 0 0) 0x7f41bc06c480 con 0x7f41bc070490 2023-12-20T06:03:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41b9d9b640 1 -- 172.21.15.32:0/3128887129 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f41a8003530 con 0x7f41bc070490 2023-12-20T06:03:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41b3fff640 1 -- 172.21.15.32:0/3128887129 >> v1:172.21.15.39:6789/0 conn(0x7f41bc1603e0 legacy=0x7f41bc0ca6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41b9d9b640 1 -- 172.21.15.32:0/3128887129 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2224910908 0 0) 0x7f41bc06c760 con 0x7f41bc1603e0 2023-12-20T06:03:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41b3fff640 1 -- 172.21.15.32:0/3128887129 >> v1:172.21.15.32:6789/0 conn(0x7f41bc070490 legacy=0x7f41bc15fcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.422+0000 7f41b3fff640 1 -- 172.21.15.32:0/3128887129 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41bc06cd20 con 0x7f41bc06d240 2023-12-20T06:03:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.426+0000 7f41b3fff640 1 --2- 172.21.15.32:0/3128887129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41bc06d240 0x7f41bc0ca1a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f41a40097c0 tx=0x7f41a401c3c0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b4d5b315a0804a6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.426+0000 7f41b9d9b640 1 -- 172.21.15.32:0/3128887129 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a40ac640 con 0x7f41bc06d240 2023-12-20T06:03:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.426+0000 7f41b9d9b640 1 -- 172.21.15.32:0/3128887129 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a40ac7a0 con 0x7f41bc06d240 2023-12-20T06:03:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.426+0000 7f41b9d9b640 1 -- 172.21.15.32:0/3128887129 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a40acb30 con 0x7f41bc06d240 2023-12-20T06:03:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.426+0000 7f41c09db640 1 -- 172.21.15.32:0/3128887129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41bc06d240 msgr2=0x7f41bc0ca1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.426+0000 7f41c09db640 1 --2- 172.21.15.32:0/3128887129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41bc06d240 0x7f41bc0ca1a0 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f41a40097c0 tx=0x7f41a401c3c0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.426+0000 7f41c09db640 1 -- 172.21.15.32:0/3128887129 shutdown_connections 2023-12-20T06:03:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.426+0000 7f41c09db640 1 --2- 172.21.15.32:0/3128887129 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41bc06d240 0x7f41bc0ca1a0 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.426+0000 7f41c09db640 1 -- 172.21.15.32:0/3128887129 >> 172.21.15.32:0/3128887129 conn(0x7f41bc06a510 msgr2=0x7f41bc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.426+0000 7f41c09db640 1 -- 172.21.15.32:0/3128887129 shutdown_connections 2023-12-20T06:03:52.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.426+0000 7f41c09db640 1 -- 172.21.15.32:0/3128887129 wait complete. 2023-12-20T06:03:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41c09db640 1 Processor -- start 2023-12-20T06:03:52.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41c09db640 1 -- start start 2023-12-20T06:03:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41c09db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41bc06d240 0x7f41bc1f8860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41c09db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc070490 0x7f41bc1f8da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41c09db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41bc1603e0 0x7f41bc15a3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41c09db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0c5010 con 0x7f41bc070490 2023-12-20T06:03:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41c09db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0c4a50 con 0x7f41bc1603e0 2023-12-20T06:03:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41c09db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f41bc0c4d30 con 0x7f41bc06d240 2023-12-20T06:03:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc070490 0x7f41bc1f8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc070490 0x7f41bc1f8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47298/0 (socket says 172.21.15.32:47298) 2023-12-20T06:03:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b3fff640 1 -- 172.21.15.32:0/327864128 learned_addr learned my addr 172.21.15.32:0/327864128 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41ba59c640 1 --2- 172.21.15.32:0/327864128 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41bc06d240 0x7f41bc1f8860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41bad9d640 1 --2- 172.21.15.32:0/327864128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41bc1603e0 0x7f41bc15a3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b3fff640 1 -- 172.21.15.32:0/327864128 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41bc06d240 msgr2=0x7f41bc1f8860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b3fff640 1 --2- 172.21.15.32:0/327864128 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41bc06d240 0x7f41bc1f8860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b3fff640 1 -- 172.21.15.32:0/327864128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41bc1603e0 msgr2=0x7f41bc15a3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b3fff640 1 --2- 172.21.15.32:0/327864128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41bc1603e0 0x7f41bc15a3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b3fff640 1 -- 172.21.15.32:0/327864128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f41a4009420 con 0x7f41bc070490 2023-12-20T06:03:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b3fff640 1 --2- 172.21.15.32:0/327864128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc070490 0x7f41bc1f8da0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f41a40097c0 tx=0x7f41a4019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b2ffd640 1 -- 172.21.15.32:0/327864128 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4004220 con 0x7f41bc070490 2023-12-20T06:03:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b2ffd640 1 -- 172.21.15.32:0/327864128 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f41a4004380 con 0x7f41bc070490 2023-12-20T06:03:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41b2ffd640 1 -- 172.21.15.32:0/327864128 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f41a4003860 con 0x7f41bc070490 2023-12-20T06:03:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41c09db640 1 -- 172.21.15.32:0/327864128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f41bc158500 con 0x7f41bc070490 2023-12-20T06:03:52.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.430+0000 7f41c09db640 1 -- 172.21.15.32:0/327864128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f41bc0c58b0 con 0x7f41bc070490 2023-12-20T06:03:52.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.434+0000 7f41c09db640 1 -- 172.21.15.32:0/327864128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f41bc158360 con 0x7f41bc070490 2023-12-20T06:03:52.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.434+0000 7f41b2ffd640 1 -- 172.21.15.32:0/327864128 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 140) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f41a40b3040 con 0x7f41bc070490 2023-12-20T06:03:52.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.434+0000 7f41b2ffd640 1 -- 172.21.15.32:0/327864128 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7403+0+0 (secure 0 0 0) 0x7f41a411c8c0 con 0x7f41bc070490 2023-12-20T06:03:52.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.438+0000 7f41b2ffd640 1 -- 172.21.15.32:0/327864128 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f41bc158360 con 0x7f41bc070490 2023-12-20T06:03:52.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.594+0000 7f41c09db640 1 -- 172.21.15.32:0/327864128 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f41bc158360 con 0x7f41bc070490 2023-12-20T06:03:52.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.594+0000 7f41b2ffd640 1 -- 172.21.15.32:0/327864128 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v236) v1 ==== 100+0+11 (secure 0 0 0) 0x7f41bc158360 con 0x7f41bc070490 2023-12-20T06:03:52.596 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:03:52.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.598+0000 7f41c09db640 1 -- 172.21.15.32:0/327864128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc070490 msgr2=0x7f41bc1f8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.598+0000 7f41c09db640 1 --2- 172.21.15.32:0/327864128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc070490 0x7f41bc1f8da0 secure :-1 s=READY pgs=2374 cs=0 l=1 rev1=1 crypto rx=0x7f41a40097c0 tx=0x7f41a4019160 comp rx=0 tx=0).stop 2023-12-20T06:03:52.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.598+0000 7f41c09db640 1 -- 172.21.15.32:0/327864128 shutdown_connections 2023-12-20T06:03:52.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.598+0000 7f41c09db640 1 --2- 172.21.15.32:0/327864128 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f41bc1603e0 0x7f41bc15a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.598+0000 7f41c09db640 1 --2- 172.21.15.32:0/327864128 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f41bc070490 0x7f41bc1f8da0 unknown :-1 s=CLOSED pgs=2374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.598+0000 7f41c09db640 1 --2- 172.21.15.32:0/327864128 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f41bc06d240 0x7f41bc1f8860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.598+0000 7f41c09db640 1 -- 172.21.15.32:0/327864128 >> 172.21.15.32:0/327864128 conn(0x7f41bc06a510 msgr2=0x7f41bc0b3a50 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:52.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.598+0000 7f41c09db640 1 -- 172.21.15.32:0/327864128 shutdown_connections 2023-12-20T06:03:52.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.598+0000 7f41c09db640 1 -- 172.21.15.32:0/327864128 wait complete. 2023-12-20T06:03:52.618 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-20T06:03:52.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.762+0000 7f829c883640 1 Processor -- start 2023-12-20T06:03:52.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.762+0000 7f829c883640 1 -- start start 2023-12-20T06:03:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f829c883640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82980affe0 0x7f8298071220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f829c883640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82980cdce0 0x7f82980cc510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f829c883640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f8298071820 con 0x7f82980b13b0 2023-12-20T06:03:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f829c883640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f82980723a0 con 0x7f82980cdce0 2023-12-20T06:03:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f829c883640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82980b17f0 con 0x7f82980affe0 2023-12-20T06:03:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f8297fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82980cdce0 0x7f82980cc510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f8297fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f82980cdce0 0x7f82980cc510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47314/0 (socket says 172.21.15.32:47314) 2023-12-20T06:03:52.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f8297fff640 1 -- 172.21.15.32:0/1747501821 learned_addr learned my addr 172.21.15.32:0/1747501821 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f82977fe640 1 --2- 172.21.15.32:0/1747501821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82980affe0 0x7f8298071220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f8297fff640 1 -- 172.21.15.32:0/1747501821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82980affe0 msgr2=0x7f8298071220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f8297fff640 1 --2- 172.21.15.32:0/1747501821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82980affe0 0x7f8298071220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f8297fff640 1 -- 172.21.15.32:0/1747501821 >> v1:172.21.15.39:6789/0 conn(0x7f82980b13b0 legacy=0x7f82980b03c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f8297fff640 1 -- 172.21.15.32:0/1747501821 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8298071b00 con 0x7f82980cdce0 2023-12-20T06:03:52.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.766+0000 7f8297fff640 1 --2- 172.21.15.32:0/1747501821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82980cdce0 0x7f82980cc510 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f8290009e50 tx=0x7f829001ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11153205c8b9dae9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f82967fc640 1 -- 172.21.15.32:0/1747501821 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900a57d0 con 0x7f82980cdce0 2023-12-20T06:03:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f82967fc640 1 -- 172.21.15.32:0/1747501821 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f82900a5930 con 0x7f82980cdce0 2023-12-20T06:03:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f82967fc640 1 -- 172.21.15.32:0/1747501821 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f82900a5cc0 con 0x7f82980cdce0 2023-12-20T06:03:52.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 -- 172.21.15.32:0/1747501821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82980cdce0 msgr2=0x7f82980cc510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 --2- 172.21.15.32:0/1747501821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82980cdce0 0x7f82980cc510 secure :-1 s=READY pgs=2375 cs=0 l=1 rev1=1 crypto rx=0x7f8290009e50 tx=0x7f829001ca10 comp rx=0 tx=0).stop 2023-12-20T06:03:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 -- 172.21.15.32:0/1747501821 shutdown_connections 2023-12-20T06:03:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 --2- 172.21.15.32:0/1747501821 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82980cdce0 0x7f82980cc510 unknown :-1 s=CLOSED pgs=2375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 --2- 172.21.15.32:0/1747501821 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82980affe0 0x7f8298071220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 -- 172.21.15.32:0/1747501821 >> 172.21.15.32:0/1747501821 conn(0x7f829806a4e0 msgr2=0x7f829806bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 -- 172.21.15.32:0/1747501821 shutdown_connections 2023-12-20T06:03:52.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 -- 172.21.15.32:0/1747501821 wait complete. 2023-12-20T06:03:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 Processor -- start 2023-12-20T06:03:52.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 -- start start 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82980affe0 0x7f8298148fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.770+0000 7f829c883640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82980b13b0 0x7f82981494e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f829c883640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82980cdce0 0x7f82980abb40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f829c883640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f8298149a20 con 0x7f82980affe0 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f829c883640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f82980b1930 con 0x7f82980b13b0 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f829c883640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f82980accf0 con 0x7f82980cdce0 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f82977fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82980affe0 0x7f8298148fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f82977fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82980affe0 0x7f8298148fa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47328/0 (socket says 172.21.15.32:47328) 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f82977fe640 1 -- 172.21.15.32:0/2633451289 learned_addr learned my addr 172.21.15.32:0/2633451289 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f8296ffd640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82980b13b0 0x7f82981494e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f8297fff640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82980cdce0 0x7f82980abb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f82977fe640 1 -- 172.21.15.32:0/2633451289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82980cdce0 msgr2=0x7f82980abb40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f82977fe640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82980cdce0 0x7f82980abb40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f82977fe640 1 -- 172.21.15.32:0/2633451289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82980b13b0 msgr2=0x7f82981494e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f82977fe640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82980b13b0 0x7f82981494e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f82977fe640 1 -- 172.21.15.32:0/2633451289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f8290009b30 con 0x7f82980affe0 2023-12-20T06:03:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f82977fe640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82980affe0 0x7f8298148fa0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f828800e8a0 tx=0x7f828800eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f827ffff640 1 -- 172.21.15.32:0/2633451289 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8288009510 con 0x7f82980affe0 2023-12-20T06:03:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f827ffff640 1 -- 172.21.15.32:0/2633451289 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f828800c940 con 0x7f82980affe0 2023-12-20T06:03:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f827ffff640 1 -- 172.21.15.32:0/2633451289 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f828800ccb0 con 0x7f82980affe0 2023-12-20T06:03:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f829c883640 1 -- 172.21.15.32:0/2633451289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f82980acf60 con 0x7f82980affe0 2023-12-20T06:03:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f829c883640 1 -- 172.21.15.32:0/2633451289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f8298149d90 con 0x7f82980affe0 2023-12-20T06:03:52.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f829c883640 1 -- 172.21.15.32:0/2633451289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f8298074100 con 0x7f82980affe0 2023-12-20T06:03:52.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.774+0000 7f827ffff640 1 -- 172.21.15.32:0/2633451289 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 140) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f8288013470 con 0x7f82980affe0 2023-12-20T06:03:52.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.778+0000 7f827ffff640 1 -- 172.21.15.32:0/2633451289 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7403+0+0 (secure 0 0 0) 0x7f8288083380 con 0x7f82980affe0 2023-12-20T06:03:52.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.782+0000 7f827ffff640 1 -- 172.21.15.32:0/2633451289 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f8288050b40 con 0x7f82980affe0 2023-12-20T06:03:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.790+0000 7f827ffff640 1 -- 172.21.15.32:0/2633451289 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 141) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f82880505f0 con 0x7f82980affe0 2023-12-20T06:03:52.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.790+0000 7f827ffff640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f828406d990 0x7f828406fd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:52.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.794+0000 7f8296ffd640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f828406d990 0x7f828406fd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:52.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.794+0000 7f8296ffd640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f828406d990 0x7f828406fd80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f829806c870 tx=0x7f828c009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:52.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.946+0000 7f829c883640 1 -- 172.21.15.32:0/2633451289 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f8298157400 con 0x7f82980affe0 2023-12-20T06:03:52.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f827ffff640 1 -- 172.21.15.32:0/2633451289 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v236) v1 ==== 107+0+11 (secure 0 0 0) 0x7f828807e800 con 0x7f82980affe0 2023-12-20T06:03:52.949 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:03:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 -- 172.21.15.32:0/2633451289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f828406d990 msgr2=0x7f828406fd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f828406d990 0x7f828406fd80 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f829806c870 tx=0x7f828c009040 comp rx=0 tx=0).stop 2023-12-20T06:03:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 -- 172.21.15.32:0/2633451289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82980affe0 msgr2=0x7f8298148fa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82980affe0 0x7f8298148fa0 secure :-1 s=READY pgs=2376 cs=0 l=1 rev1=1 crypto rx=0x7f828800e8a0 tx=0x7f828800eca0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 -- 172.21.15.32:0/2633451289 shutdown_connections 2023-12-20T06:03:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f828406d990 0x7f828406fd80 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f82980cdce0 0x7f82980abb40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f82980b13b0 0x7f82981494e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 --2- 172.21.15.32:0/2633451289 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f82980affe0 0x7f8298148fa0 unknown :-1 s=CLOSED pgs=2376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 -- 172.21.15.32:0/2633451289 >> 172.21.15.32:0/2633451289 conn(0x7f829806a4e0 msgr2=0x7f82980ca090 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 -- 172.21.15.32:0/2633451289 shutdown_connections 2023-12-20T06:03:52.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:52.950+0000 7f829c883640 1 -- 172.21.15.32:0/2633451289 wait complete. 2023-12-20T06:03:52.970 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-20T06:03:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.114+0000 7f92ea37b640 1 Processor -- start 2023-12-20T06:03:53.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.114+0000 7f92ea37b640 1 -- start start 2023-12-20T06:03:53.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92ea37b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f92e415abb0 0x7f92e40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92ea37b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 0x7f92e415bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92ea37b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92e415b6a0 con 0x7f92e4070230 2023-12-20T06:03:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92ea37b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f92e406e160 con 0x7f92e415abb0 2023-12-20T06:03:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92ea37b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e415c070 con 0x7f92e40b0cb0 2023-12-20T06:03:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e8918640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92e4070230 0x7f92e406c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55830/0 (socket says 172.21.15.32:55830) 2023-12-20T06:03:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e8918640 1 -- 172.21.15.32:0/1804080576 learned_addr learned my addr 172.21.15.32:0/1804080576 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e37fe640 1 --2- 172.21.15.32:0/1804080576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 0x7f92e415bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e3fff640 1 --2- 172.21.15.32:0/1804080576 >> v2:172.21.15.39:3300/0 conn(0x7f92e415abb0 0x7f92e40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e2ffd640 1 -- 172.21.15.32:0/1804080576 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2330924339 0 0) 0x7f92e415b6a0 con 0x7f92e4070230 2023-12-20T06:03:53.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e2ffd640 1 -- 172.21.15.32:0/1804080576 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92cc003530 con 0x7f92e4070230 2023-12-20T06:03:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e2ffd640 1 -- 172.21.15.32:0/1804080576 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3588419673 0 0) 0x7f92cc003530 con 0x7f92e4070230 2023-12-20T06:03:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e37fe640 1 -- 172.21.15.32:0/1804080576 >> v2:172.21.15.39:3300/0 conn(0x7f92e415abb0 msgr2=0x7f92e40caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e37fe640 1 --2- 172.21.15.32:0/1804080576 >> v2:172.21.15.39:3300/0 conn(0x7f92e415abb0 0x7f92e40caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e37fe640 1 -- 172.21.15.32:0/1804080576 >> v1:172.21.15.32:6789/0 conn(0x7f92e4070230 legacy=0x7f92e406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e37fe640 1 -- 172.21.15.32:0/1804080576 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92e406d8c0 con 0x7f92e40b0cb0 2023-12-20T06:03:53.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.118+0000 7f92e3fff640 1 --2- 172.21.15.32:0/1804080576 >> v2:172.21.15.39:3300/0 conn(0x7f92e415abb0 0x7f92e40caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:53.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92e37fe640 1 --2- 172.21.15.32:0/1804080576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 0x7f92e415bb30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097c0 tx=0x7f92d801c3a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=38074033e7f571c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92e2ffd640 1 -- 172.21.15.32:0/1804080576 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc001a00 con 0x7f92e40b0cb0 2023-12-20T06:03:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92e2ffd640 1 -- 172.21.15.32:0/1804080576 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92d80a7050 con 0x7f92e40b0cb0 2023-12-20T06:03:53.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92e2ffd640 1 -- 172.21.15.32:0/1804080576 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92d80ab620 con 0x7f92e40b0cb0 2023-12-20T06:03:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92ea37b640 1 -- 172.21.15.32:0/1804080576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 msgr2=0x7f92e415bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92ea37b640 1 --2- 172.21.15.32:0/1804080576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 0x7f92e415bb30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7f92d80097c0 tx=0x7f92d801c3a0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92ea37b640 1 -- 172.21.15.32:0/1804080576 shutdown_connections 2023-12-20T06:03:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92ea37b640 1 --2- 172.21.15.32:0/1804080576 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 0x7f92e415bb30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92ea37b640 1 --2- 172.21.15.32:0/1804080576 >> v2:172.21.15.39:3300/0 conn(0x7f92e415abb0 0x7f92e40caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92ea37b640 1 -- 172.21.15.32:0/1804080576 >> 172.21.15.32:0/1804080576 conn(0x7f92e406a510 msgr2=0x7f92e40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92ea37b640 1 -- 172.21.15.32:0/1804080576 shutdown_connections 2023-12-20T06:03:53.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92ea37b640 1 -- 172.21.15.32:0/1804080576 wait complete. 2023-12-20T06:03:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92ea37b640 1 Processor -- start 2023-12-20T06:03:53.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.122+0000 7f92ea37b640 1 -- start start 2023-12-20T06:03:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92ea37b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92e4070230 0x7f92e40cb3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92ea37b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 0x7f92e40cd400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92ea37b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e415abb0 0x7f92e40cd940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92ea37b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40c6bf0 con 0x7f92e415abb0 2023-12-20T06:03:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92ea37b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92e413f8f0 con 0x7f92e4070230 2023-12-20T06:03:53.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92ea37b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92e40c6250 con 0x7f92e40b0cb0 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e8918640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e415abb0 0x7f92e40cd940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e37fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 0x7f92e40cd400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e8918640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e415abb0 0x7f92e40cd940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47332/0 (socket says 172.21.15.32:47332) 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e8918640 1 -- 172.21.15.32:0/678189550 learned_addr learned my addr 172.21.15.32:0/678189550 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92e4070230 0x7f92e40cb3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e8918640 1 -- 172.21.15.32:0/678189550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 msgr2=0x7f92e40cd400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e8918640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 0x7f92e40cd400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e8918640 1 -- 172.21.15.32:0/678189550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92e4070230 msgr2=0x7f92e40cb3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e8918640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92e4070230 0x7f92e40cb3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e8918640 1 -- 172.21.15.32:0/678189550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92d4009420 con 0x7f92e415abb0 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e37fe640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 0x7f92e40cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:53.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e8918640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e415abb0 0x7f92e40cd940 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00cca0 tx=0x7f92dc00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e0ff9640 1 -- 172.21.15.32:0/678189550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc012400 con 0x7f92e415abb0 2023-12-20T06:03:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e0ff9640 1 -- 172.21.15.32:0/678189550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92dc012560 con 0x7f92e415abb0 2023-12-20T06:03:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92e0ff9640 1 -- 172.21.15.32:0/678189550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92dc012910 con 0x7f92e415abb0 2023-12-20T06:03:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92ea37b640 1 -- 172.21.15.32:0/678189550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92d8009420 con 0x7f92e415abb0 2023-12-20T06:03:53.129 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92ea37b640 1 -- 172.21.15.32:0/678189550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92e40c6810 con 0x7f92e415abb0 2023-12-20T06:03:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.126+0000 7f92ea37b640 1 -- 172.21.15.32:0/678189550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92e40c6d30 con 0x7f92e415abb0 2023-12-20T06:03:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.130+0000 7f92e0ff9640 1 -- 172.21.15.32:0/678189550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 141) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f92dc007070 con 0x7f92e415abb0 2023-12-20T06:03:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.130+0000 7f92e0ff9640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f92d00658b0 0x7f92d0067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.130+0000 7f92e0ff9640 1 -- 172.21.15.32:0/678189550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7403+0+0 (secure 0 0 0) 0x7f92dc083590 con 0x7f92e415abb0 2023-12-20T06:03:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.130+0000 7f92e3fff640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f92d00658b0 0x7f92d0067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.130+0000 7f92e3fff640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f92d00658b0 0x7f92d0067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009cf0 tx=0x7f92d4009190 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:53.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.134+0000 7f92e0ff9640 1 -- 172.21.15.32:0/678189550 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f92e40c6d30 con 0x7f92e415abb0 2023-12-20T06:03:53.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 -- 172.21.15.32:0/678189550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f92e40c6d30 con 0x7f92e415abb0 2023-12-20T06:03:53.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92e0ff9640 1 -- 172.21.15.32:0/678189550 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v236) v1 ==== 103+0+11 (secure 0 0 0) 0x7f92e40c6d30 con 0x7f92e415abb0 2023-12-20T06:03:53.290 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:03:53.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 -- 172.21.15.32:0/678189550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f92d00658b0 msgr2=0x7f92d0067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f92d00658b0 0x7f92d0067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f92d4009cf0 tx=0x7f92d4009190 comp rx=0 tx=0).stop 2023-12-20T06:03:53.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 -- 172.21.15.32:0/678189550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e415abb0 msgr2=0x7f92e40cd940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e415abb0 0x7f92e40cd940 secure :-1 s=READY pgs=2378 cs=0 l=1 rev1=1 crypto rx=0x7f92dc00cca0 tx=0x7f92dc00a430 comp rx=0 tx=0).stop 2023-12-20T06:03:53.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 -- 172.21.15.32:0/678189550 shutdown_connections 2023-12-20T06:03:53.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f92d00658b0 0x7f92d0067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92e415abb0 0x7f92e40cd940 unknown :-1 s=CLOSED pgs=2378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92e40b0cb0 0x7f92e40cd400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 --2- 172.21.15.32:0/678189550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92e4070230 0x7f92e40cb3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 -- 172.21.15.32:0/678189550 >> 172.21.15.32:0/678189550 conn(0x7f92e406a510 msgr2=0x7f92e4154380 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:53.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 -- 172.21.15.32:0/678189550 shutdown_connections 2023-12-20T06:03:53.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.290+0000 7f92ea37b640 1 -- 172.21.15.32:0/678189550 wait complete. 2023-12-20T06:03:53.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:53.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.454+0000 7fb318139640 1 Processor -- start 2023-12-20T06:03:53.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.454+0000 7fb318139640 1 -- start start 2023-12-20T06:03:53.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.454+0000 7fb318139640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb3100aeda0 0x7fb31006f5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb318139640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31006f150 con 0x7fb310159c00 2023-12-20T06:03:53.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb318139640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb31006fb20 con 0x7fb3100ce9b0 2023-12-20T06:03:53.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb318139640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb3100aeb10 con 0x7fb3100aeda0 2023-12-20T06:03:53.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb315ed5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb310159c00 0x7fb3100cb300 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55840/0 (socket says 172.21.15.32:55840) 2023-12-20T06:03:53.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb315ed5640 1 -- 172.21.15.32:0/3598542879 learned_addr learned my addr 172.21.15.32:0/3598542879 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:53.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb3156d4640 1 --2- 172.21.15.32:0/3598542879 >> v2:172.21.15.39:3300/0 conn(0x7fb3100aeda0 0x7fb31006f5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2807146591 0 0) 0x7fb31006f150 con 0x7fb310159c00 2023-12-20T06:03:53.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb2fc005740 con 0x7fb310159c00 2023-12-20T06:03:53.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2586149549 0 0) 0x7fb2fc005740 con 0x7fb310159c00 2023-12-20T06:03:53.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 >> v2:172.21.15.39:3300/0 conn(0x7fb3100aeda0 msgr2=0x7fb31006f5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:53.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 --2- 172.21.15.32:0/3598542879 >> v2:172.21.15.39:3300/0 conn(0x7fb3100aeda0 0x7fb31006f5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 >> v1:172.21.15.39:6789/0 conn(0x7fb3100ce9b0 legacy=0x7fb3100712d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb3100cbae0 con 0x7fb310159c00 2023-12-20T06:03:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fb30c001680 con 0x7fb310159c00 2023-12-20T06:03:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb3156d4640 1 --2- 172.21.15.32:0/3598542879 >> v2:172.21.15.39:3300/0 conn(0x7fb3100aeda0 0x7fb31006f5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 >> v1:172.21.15.32:6789/0 conn(0x7fb310159c00 legacy=0x7fb3100cb300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc005b90 0x7fb2fc005fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc0069e0 0x7fb2fc006e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc0073a0 0x7fb2fc00dba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb3100aeb10 con 0x7fb2fc005b90 2023-12-20T06:03:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc00e240 con 0x7fb2fc0069e0 2023-12-20T06:03:53.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb2fc00e400 con 0x7fb2fc0073a0 2023-12-20T06:03:53.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb315ed5640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc005b90 0x7fb2fc005fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb3156d4640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc0069e0 0x7fb2fc006e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb3166d6640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc0073a0 0x7fb2fc00dba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb315ed5640 1 -- 172.21.15.32:0/3598542879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc0073a0 msgr2=0x7fb2fc00dba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:53.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb315ed5640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc0073a0 0x7fb2fc00dba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb315ed5640 1 -- 172.21.15.32:0/3598542879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc0069e0 msgr2=0x7fb2fc006e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:53.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb315ed5640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc0069e0 0x7fb2fc006e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb315ed5640 1 -- 172.21.15.32:0/3598542879 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb2fc005740 con 0x7fb2fc005b90 2023-12-20T06:03:53.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb3156d4640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb2fc0069e0 0x7fb2fc006e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:53.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb3166d6640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc0073a0 0x7fb2fc00dba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:53.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.458+0000 7fb3166d6640 1 -- 172.21.15.32:0/3598542879 reap_dead start 2023-12-20T06:03:53.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb315ed5640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc005b90 0x7fb2fc005fb0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00c240 tx=0x7fb30c01cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e27820d6efaee2c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:53.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c005ae0 con 0x7fb2fc005b90 2023-12-20T06:03:53.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c005c40 con 0x7fb2fc005b90 2023-12-20T06:03:53.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb314ed3640 1 -- 172.21.15.32:0/3598542879 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0072b0 con 0x7fb2fc005b90 2023-12-20T06:03:53.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb318139640 1 -- 172.21.15.32:0/3598542879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc005b90 msgr2=0x7fb2fc005fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb318139640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc005b90 0x7fb2fc005fb0 secure :-1 s=READY pgs=2380 cs=0 l=1 rev1=1 crypto rx=0x7fb30c00c240 tx=0x7fb30c01cc40 comp rx=0 tx=0).stop 2023-12-20T06:03:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb318139640 1 -- 172.21.15.32:0/3598542879 shutdown_connections 2023-12-20T06:03:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb318139640 1 --2- 172.21.15.32:0/3598542879 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb2fc005b90 0x7fb2fc005fb0 unknown :-1 s=CLOSED pgs=2380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb318139640 1 -- 172.21.15.32:0/3598542879 >> 172.21.15.32:0/3598542879 conn(0x7fb3100cc310 msgr2=0x7fb3100cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb318139640 1 -- 172.21.15.32:0/3598542879 shutdown_connections 2023-12-20T06:03:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb318139640 1 -- 172.21.15.32:0/3598542879 wait complete. 2023-12-20T06:03:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb318139640 1 Processor -- start 2023-12-20T06:03:53.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.462+0000 7fb318139640 1 -- start start 2023-12-20T06:03:53.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb318139640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc005b90 0x7fb310164ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb318139640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb31015ac40 0x7fb3100b4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb318139640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb31015b0c0 0x7fb3100b4bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb318139640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310152ab0 con 0x7fb31015ac40 2023-12-20T06:03:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb318139640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb310152d70 con 0x7fb31015b0c0 2023-12-20T06:03:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb318139640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb310153050 con 0x7fb2fc005b90 2023-12-20T06:03:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb315ed5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc005b90 0x7fb310164ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb315ed5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc005b90 0x7fb310164ec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:50166/0 (socket says 172.21.15.32:50166) 2023-12-20T06:03:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb315ed5640 1 -- 172.21.15.32:0/494351200 learned_addr learned my addr 172.21.15.32:0/494351200 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb3156d4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb31015ac40 0x7fb3100b4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb3166d6640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb31015b0c0 0x7fb3100b4bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb315ed5640 1 -- 172.21.15.32:0/494351200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb31015b0c0 msgr2=0x7fb3100b4bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb315ed5640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb31015b0c0 0x7fb3100b4bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb315ed5640 1 -- 172.21.15.32:0/494351200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb31015ac40 msgr2=0x7fb3100b4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb315ed5640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb31015ac40 0x7fb3100b4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb315ed5640 1 -- 172.21.15.32:0/494351200 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb310153330 con 0x7fb2fc005b90 2023-12-20T06:03:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb3156d4640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb31015ac40 0x7fb3100b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb315ed5640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc005b90 0x7fb310164ec0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb30c01c6b0 tx=0x7fb2fc007a10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb3067fc640 1 -- 172.21.15.32:0/494351200 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0b4070 con 0x7fb2fc005b90 2023-12-20T06:03:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb3067fc640 1 -- 172.21.15.32:0/494351200 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb30c001650 con 0x7fb2fc005b90 2023-12-20T06:03:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb3067fc640 1 -- 172.21.15.32:0/494351200 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb30c0072b0 con 0x7fb2fc005b90 2023-12-20T06:03:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb318139640 1 -- 172.21.15.32:0/494351200 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb310153610 con 0x7fb2fc005b90 2023-12-20T06:03:53.469 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb318139640 1 -- 172.21.15.32:0/494351200 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb310151ee0 con 0x7fb2fc005b90 2023-12-20T06:03:53.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.466+0000 7fb318139640 1 -- 172.21.15.32:0/494351200 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb31004e000 con 0x7fb2fc005b90 2023-12-20T06:03:53.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.470+0000 7fb3067fc640 1 -- 172.21.15.32:0/494351200 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 141) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fb30c007410 con 0x7fb2fc005b90 2023-12-20T06:03:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.470+0000 7fb3067fc640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2f40658b0 0x7fb2f4067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.470+0000 7fb3067fc640 1 -- 172.21.15.32:0/494351200 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7403+0+0 (secure 0 0 0) 0x7fb30c11ce00 con 0x7fb2fc005b90 2023-12-20T06:03:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.470+0000 7fb3156d4640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2f40658b0 0x7fb2f4067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.470+0000 7fb3156d4640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2f40658b0 0x7fb2f4067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb300009fd0 tx=0x7fb300009190 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:53.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.474+0000 7fb3067fc640 1 -- 172.21.15.32:0/494351200 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb30c0ea5e0 con 0x7fb2fc005b90 2023-12-20T06:03:53.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.666+0000 7fb318139640 1 -- 172.21.15.32:0/494351200 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb3100cc070 con 0x7fb2fc005b90 2023-12-20T06:03:53.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.666+0000 7fb3067fc640 1 -- 172.21.15.32:0/494351200 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5909 (secure 0 0 0) 0x7fb30c0bd090 con 0x7fb2fc005b90 2023-12-20T06:03:53.669 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:53.669 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:21.951240+0000","modified":"2023-12-20T06:03:24.980742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11301":{"gid":11301,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1623322556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1623322556},{"type":"v1","addr":"172.21.15.32:6837","nonce":1623322556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:34.513908+0000","modified":"2023-12-20T06:03:37.549965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11298":{"gid":11298,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6836/60134876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":60134876},{"type":"v1","addr":"172.21.15.32:6836","nonce":60134876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-20T06:03:53.669 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 118 2023-12-20T06:03:53.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 -- 172.21.15.32:0/494351200 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2f40658b0 msgr2=0x7fb2f4067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2f40658b0 0x7fb2f4067d70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7fb300009fd0 tx=0x7fb300009190 comp rx=0 tx=0).stop 2023-12-20T06:03:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 -- 172.21.15.32:0/494351200 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc005b90 msgr2=0x7fb310164ec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc005b90 0x7fb310164ec0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fb30c01c6b0 tx=0x7fb2fc007a10 comp rx=0 tx=0).stop 2023-12-20T06:03:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 -- 172.21.15.32:0/494351200 shutdown_connections 2023-12-20T06:03:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb2f40658b0 0x7fb2f4067d70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb31015b0c0 0x7fb3100b4bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb31015ac40 0x7fb3100b4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 --2- 172.21.15.32:0/494351200 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb2fc005b90 0x7fb310164ec0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 -- 172.21.15.32:0/494351200 >> 172.21.15.32:0/494351200 conn(0x7fb3100cc310 msgr2=0x7fb3100cd790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 -- 172.21.15.32:0/494351200 shutdown_connections 2023-12-20T06:03:53.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.670+0000 7fb318139640 1 -- 172.21.15.32:0/494351200 wait complete. 2023-12-20T06:03:53.690 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:53.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.838+0000 7fb1cc6e5640 1 Processor -- start 2023-12-20T06:03:53.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.838+0000 7fb1cc6e5640 1 -- start start 2023-12-20T06:03:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1cc6e5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1c415ce30 0x7fb1c4161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1cc6e5640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb1c406deb0 0x7fb1c40cd520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1cc6e5640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1c4071920 con 0x7fb1c4070950 2023-12-20T06:03:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1cc6e5640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c4071080 con 0x7fb1c415ce30 2023-12-20T06:03:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1cc6e5640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1c4071360 con 0x7fb1c406deb0 2023-12-20T06:03:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1ca481640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1c415ce30 0x7fb1c4161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1ca481640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1c415ce30 0x7fb1c4161560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47356/0 (socket says 172.21.15.32:47356) 2023-12-20T06:03:53.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1ca481640 1 -- 172.21.15.32:0/161788115 learned_addr learned my addr 172.21.15.32:0/161788115 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1cac82640 1 --2- 172.21.15.32:0/161788115 >> v2:172.21.15.39:3300/0 conn(0x7fb1c406deb0 0x7fb1c40cd520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1ca481640 1 -- 172.21.15.32:0/161788115 >> v2:172.21.15.39:3300/0 conn(0x7fb1c406deb0 msgr2=0x7fb1c40cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1ca481640 1 --2- 172.21.15.32:0/161788115 >> v2:172.21.15.39:3300/0 conn(0x7fb1c406deb0 0x7fb1c40cd520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1ca481640 1 -- 172.21.15.32:0/161788115 >> v1:172.21.15.39:6789/0 conn(0x7fb1c4070950 legacy=0x7fb1c406d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1ca481640 1 -- 172.21.15.32:0/161788115 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c4071640 con 0x7fb1c415ce30 2023-12-20T06:03:53.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.842+0000 7fb1cac82640 1 --2- 172.21.15.32:0/161788115 >> v2:172.21.15.39:3300/0 conn(0x7fb1c406deb0 0x7fb1c40cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:53.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1ca481640 1 --2- 172.21.15.32:0/161788115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c415ce30 0x7fb1c4161560 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40c95b0 tx=0x7fb1c001c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7007e58aecc03657 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:53.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1c947f640 1 -- 172.21.15.32:0/161788115 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00ad680 con 0x7fb1c415ce30 2023-12-20T06:03:53.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1c947f640 1 -- 172.21.15.32:0/161788115 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1c00ad7e0 con 0x7fb1c415ce30 2023-12-20T06:03:53.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1c947f640 1 -- 172.21.15.32:0/161788115 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1c00adb70 con 0x7fb1c415ce30 2023-12-20T06:03:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/161788115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c415ce30 msgr2=0x7fb1c4161560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1cc6e5640 1 --2- 172.21.15.32:0/161788115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c415ce30 0x7fb1c4161560 secure :-1 s=READY pgs=2381 cs=0 l=1 rev1=1 crypto rx=0x7fb1c40c95b0 tx=0x7fb1c001c470 comp rx=0 tx=0).stop 2023-12-20T06:03:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/161788115 shutdown_connections 2023-12-20T06:03:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1cc6e5640 1 --2- 172.21.15.32:0/161788115 >> v2:172.21.15.39:3300/0 conn(0x7fb1c406deb0 0x7fb1c40cd520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1cc6e5640 1 --2- 172.21.15.32:0/161788115 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c415ce30 0x7fb1c4161560 unknown :-1 s=CLOSED pgs=2381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/161788115 >> 172.21.15.32:0/161788115 conn(0x7fb1c40cba50 msgr2=0x7fb1c40cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/161788115 shutdown_connections 2023-12-20T06:03:53.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/161788115 wait complete. 2023-12-20T06:03:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1cc6e5640 1 Processor -- start 2023-12-20T06:03:53.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.846+0000 7fb1cc6e5640 1 -- start start 2023-12-20T06:03:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cc6e5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c406deb0 0x7fb1c40c55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cc6e5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4070950 0x7fb1c40c2ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cc6e5640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c415ce30 0x7fb1c409e290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cc6e5640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40af840 con 0x7fb1c4070950 2023-12-20T06:03:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cc6e5640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40af280 con 0x7fb1c406deb0 2023-12-20T06:03:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cc6e5640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1c40af560 con 0x7fb1c415ce30 2023-12-20T06:03:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1c9c80640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4070950 0x7fb1c40c2ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cac82640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c415ce30 0x7fb1c409e290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1ca481640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c406deb0 0x7fb1c40c55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cac82640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c415ce30 0x7fb1c409e290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:50182/0 (socket says 172.21.15.32:50182) 2023-12-20T06:03:53.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cac82640 1 -- 172.21.15.32:0/2685839439 learned_addr learned my addr 172.21.15.32:0/2685839439 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1c9c80640 1 -- 172.21.15.32:0/2685839439 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c415ce30 msgr2=0x7fb1c409e290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1c9c80640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c415ce30 0x7fb1c409e290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1c9c80640 1 -- 172.21.15.32:0/2685839439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c406deb0 msgr2=0x7fb1c40c55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1c9c80640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c406deb0 0x7fb1c40c55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1c9c80640 1 -- 172.21.15.32:0/2685839439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1c00094d0 con 0x7fb1c4070950 2023-12-20T06:03:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1c9c80640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4070950 0x7fb1c40c2ae0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c3c0 tx=0x7fb1b400c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1baffd640 1 -- 172.21.15.32:0/2685839439 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b40052d0 con 0x7fb1c4070950 2023-12-20T06:03:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1baffd640 1 -- 172.21.15.32:0/2685839439 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b4005430 con 0x7fb1c4070950 2023-12-20T06:03:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/2685839439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1c41564d0 con 0x7fb1c4070950 2023-12-20T06:03:53.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1baffd640 1 -- 172.21.15.32:0/2685839439 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b4007260 con 0x7fb1c4070950 2023-12-20T06:03:53.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/2685839439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1c40c5ae0 con 0x7fb1c4070950 2023-12-20T06:03:53.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/2685839439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1c404e000 con 0x7fb1c4070950 2023-12-20T06:03:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.850+0000 7fb1baffd640 1 -- 172.21.15.32:0/2685839439 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 142) v1 ==== 85376+0+0 (secure 0 0 0) 0x7fb1b40073c0 con 0x7fb1c4070950 2023-12-20T06:03:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.854+0000 7fb1baffd640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb1a8065c40 0x7fb1a8068100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.854+0000 7fb1baffd640 1 -- 172.21.15.32:0/2685839439 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7403+0+0 (secure 0 0 0) 0x7fb1b4082cc0 con 0x7fb1c4070950 2023-12-20T06:03:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.854+0000 7fb1ca481640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb1a8065c40 0x7fb1a8068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:53.854 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.854+0000 7fb1ca481640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb1a8065c40 0x7fb1a8068100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00af000 tx=0x7fb1c0008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:53.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:53.858+0000 7fb1baffd640 1 -- 172.21.15.32:0/2685839439 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb1b4089080 con 0x7fb1c4070950 2023-12-20T06:03:54.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.022+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/2685839439 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb1c40b4f50 con 0x7fb1c4070950 2023-12-20T06:03:54.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1baffd640 1 -- 172.21.15.32:0/2685839439 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v236) v1 ==== 74+0+20923 (secure 0 0 0) 0x7fb1b404feb0 con 0x7fb1c4070950 2023-12-20T06:03:54.026 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:54.026 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":236,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:51.776828+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:03:20.399088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-20T06:03:20.926367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:03:32.174574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:03:33.509224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1232037079":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/2587801540":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/789250710":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/1603186500":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/2783112768":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/3054548475":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2026838481":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/597963739":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/274812680":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/3886671554":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1847933543":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/4013366117":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/582500261":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/805068630":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6833/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1823890749":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6832/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2852735264":"2023-12-21T06:03:45.663020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/2685839439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb1a8065c40 msgr2=0x7fb1a8068100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb1a8065c40 0x7fb1a8068100 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7fb1c00af000 tx=0x7fb1c0008040 comp rx=0 tx=0).stop 2023-12-20T06:03:54.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/2685839439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4070950 msgr2=0x7fb1c40c2ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4070950 0x7fb1c40c2ae0 secure :-1 s=READY pgs=2382 cs=0 l=1 rev1=1 crypto rx=0x7fb1b400c3c0 tx=0x7fb1b400c880 comp rx=0 tx=0).stop 2023-12-20T06:03:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/2685839439 shutdown_connections 2023-12-20T06:03:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb1a8065c40 0x7fb1a8068100 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1c415ce30 0x7fb1c409e290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1c4070950 0x7fb1c40c2ae0 unknown :-1 s=CLOSED pgs=2382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 --2- 172.21.15.32:0/2685839439 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1c406deb0 0x7fb1c40c55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/2685839439 >> 172.21.15.32:0/2685839439 conn(0x7fb1c40cba50 msgr2=0x7fb1c415e900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/2685839439 shutdown_connections 2023-12-20T06:03:54.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.026+0000 7fb1cc6e5640 1 -- 172.21.15.32:0/2685839439 wait complete. 2023-12-20T06:03:54.048 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T06:03:54.048 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.190+0000 7fc5265bc640 1 Processor -- start 2023-12-20T06:03:54.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.190+0000 7fc5265bc640 1 -- start start 2023-12-20T06:03:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc5265bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc520071af0 0x7fc5200c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc5265bc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc5200c9a40 con 0x7fc520160b30 2023-12-20T06:03:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc5265bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc520161660 con 0x7fc52015ce60 2023-12-20T06:03:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc5265bc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc5200c9180 con 0x7fc520071af0 2023-12-20T06:03:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc524b59640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc520160b30 0x7fc520160f50 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55848/0 (socket says 172.21.15.32:55848) 2023-12-20T06:03:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc524b59640 1 -- 172.21.15.32:0/268275747 learned_addr learned my addr 172.21.15.32:0/268275747 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:54.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc51f7fe640 1 --2- 172.21.15.32:0/268275747 >> v2:172.21.15.39:3300/0 conn(0x7fc520071af0 0x7fc5200c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc51effd640 1 -- 172.21.15.32:0/268275747 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1044564402 0 0) 0x7fc5200c9a40 con 0x7fc520160b30 2023-12-20T06:03:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc51effd640 1 -- 172.21.15.32:0/268275747 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc508003530 con 0x7fc520160b30 2023-12-20T06:03:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc51effd640 1 -- 172.21.15.32:0/268275747 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3145054687 0 0) 0x7fc520161660 con 0x7fc52015ce60 2023-12-20T06:03:54.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc51effd640 1 -- 172.21.15.32:0/268275747 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc5200c9a40 con 0x7fc52015ce60 2023-12-20T06:03:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc51f7fe640 1 -- 172.21.15.32:0/268275747 >> v1:172.21.15.39:6789/0 conn(0x7fc52015ce60 legacy=0x7fc5200713e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc51f7fe640 1 -- 172.21.15.32:0/268275747 >> v1:172.21.15.32:6789/0 conn(0x7fc520160b30 legacy=0x7fc520160f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.194+0000 7fc51f7fe640 1 -- 172.21.15.32:0/268275747 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc5200c93f0 con 0x7fc520071af0 2023-12-20T06:03:54.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc51f7fe640 1 --2- 172.21.15.32:0/268275747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc520071af0 0x7fc5200c9ed0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc5140097c0 tx=0x7fc51401c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=df9f8d45496cdc76 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc51effd640 1 -- 172.21.15.32:0/268275747 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514004230 con 0x7fc520071af0 2023-12-20T06:03:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc51effd640 1 -- 172.21.15.32:0/268275747 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc5140a7050 con 0x7fc520071af0 2023-12-20T06:03:54.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc51effd640 1 -- 172.21.15.32:0/268275747 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514003620 con 0x7fc520071af0 2023-12-20T06:03:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc5265bc640 1 -- 172.21.15.32:0/268275747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc520071af0 msgr2=0x7fc5200c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc5265bc640 1 --2- 172.21.15.32:0/268275747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc520071af0 0x7fc5200c9ed0 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7fc5140097c0 tx=0x7fc51401c480 comp rx=0 tx=0).stop 2023-12-20T06:03:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc5265bc640 1 -- 172.21.15.32:0/268275747 shutdown_connections 2023-12-20T06:03:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc5265bc640 1 --2- 172.21.15.32:0/268275747 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc520071af0 0x7fc5200c9ed0 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc5265bc640 1 -- 172.21.15.32:0/268275747 >> 172.21.15.32:0/268275747 conn(0x7fc52006a320 msgr2=0x7fc5200ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc5265bc640 1 -- 172.21.15.32:0/268275747 shutdown_connections 2023-12-20T06:03:54.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc5265bc640 1 -- 172.21.15.32:0/268275747 wait complete. 2023-12-20T06:03:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc5265bc640 1 Processor -- start 2023-12-20T06:03:54.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.198+0000 7fc5265bc640 1 -- start start 2023-12-20T06:03:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc5265bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc520071af0 0x7fc520153810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc5265bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52015ce60 0x7fc520150c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc5265bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc520160b30 0x7fc5200c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc5265bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc5200c4ef0 con 0x7fc52015ce60 2023-12-20T06:03:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc5265bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc520151150 con 0x7fc520160b30 2023-12-20T06:03:54.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc5265bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc5200c2c00 con 0x7fc520071af0 2023-12-20T06:03:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52015ce60 0x7fc520150c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52015ce60 0x7fc520150c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47376/0 (socket says 172.21.15.32:47376) 2023-12-20T06:03:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51f7fe640 1 -- 172.21.15.32:0/2278713826 learned_addr learned my addr 172.21.15.32:0/2278713826 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc524b59640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc520160b30 0x7fc5200c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51ffff640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc520071af0 0x7fc520153810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51f7fe640 1 -- 172.21.15.32:0/2278713826 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc520071af0 msgr2=0x7fc520153810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51f7fe640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc520071af0 0x7fc520153810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51f7fe640 1 -- 172.21.15.32:0/2278713826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc520160b30 msgr2=0x7fc5200c39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51f7fe640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc520160b30 0x7fc5200c39c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51f7fe640 1 -- 172.21.15.32:0/2278713826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc514009420 con 0x7fc52015ce60 2023-12-20T06:03:54.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51f7fe640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52015ce60 0x7fc520150c10 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fc5140ae040 tx=0x7fc514019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51cff9640 1 -- 172.21.15.32:0/2278713826 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc514003af0 con 0x7fc52015ce60 2023-12-20T06:03:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51cff9640 1 -- 172.21.15.32:0/2278713826 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc514003c50 con 0x7fc52015ce60 2023-12-20T06:03:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc5265bc640 1 -- 172.21.15.32:0/2278713826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc5200c2ee0 con 0x7fc52015ce60 2023-12-20T06:03:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc51cff9640 1 -- 172.21.15.32:0/2278713826 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc5140ab440 con 0x7fc52015ce60 2023-12-20T06:03:54.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc5265bc640 1 -- 172.21.15.32:0/2278713826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc5200c34a0 con 0x7fc52015ce60 2023-12-20T06:03:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.202+0000 7fc5265bc640 1 -- 172.21.15.32:0/2278713826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc5200c3690 con 0x7fc52015ce60 2023-12-20T06:03:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.206+0000 7fc51cff9640 1 -- 172.21.15.32:0/2278713826 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 142) v1 ==== 85376+0+0 (secure 0 0 0) 0x7fc5140b2040 con 0x7fc52015ce60 2023-12-20T06:03:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.206+0000 7fc51cff9640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc50c065c10 0x7fc50c0680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.206+0000 7fc51cff9640 1 -- 172.21.15.32:0/2278713826 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7403+0+0 (secure 0 0 0) 0x7fc51411d9d0 con 0x7fc52015ce60 2023-12-20T06:03:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.206+0000 7fc51ffff640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc50c065c10 0x7fc50c0680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.206+0000 7fc51ffff640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc50c065c10 0x7fc50c0680d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc510001240 tx=0x7fc510009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:54.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.210+0000 7fc51cff9640 1 -- 172.21.15.32:0/2278713826 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc5140eaa10 con 0x7fc52015ce60 2023-12-20T06:03:54.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.398+0000 7fc5265bc640 1 -- 172.21.15.32:0/2278713826 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc52004e000 con 0x7fc52015ce60 2023-12-20T06:03:54.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.398+0000 7fc51cff9640 1 -- 172.21.15.32:0/2278713826 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5909 (secure 0 0 0) 0x7fc5140b0090 con 0x7fc52015ce60 2023-12-20T06:03:54.400 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:54.400 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:21.951240+0000","modified":"2023-12-20T06:03:24.980742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11301":{"gid":11301,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1623322556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1623322556},{"type":"v1","addr":"172.21.15.32:6837","nonce":1623322556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:34.513908+0000","modified":"2023-12-20T06:03:37.549965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11298":{"gid":11298,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6836/60134876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":60134876},{"type":"v1","addr":"172.21.15.32:6836","nonce":60134876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-20T06:03:54.400 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 118 2023-12-20T06:03:54.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 -- 172.21.15.32:0/2278713826 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc50c065c10 msgr2=0x7fc50c0680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc50c065c10 0x7fc50c0680d0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc510001240 tx=0x7fc510009040 comp rx=0 tx=0).stop 2023-12-20T06:03:54.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 -- 172.21.15.32:0/2278713826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52015ce60 msgr2=0x7fc520150c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52015ce60 0x7fc520150c10 secure :-1 s=READY pgs=2385 cs=0 l=1 rev1=1 crypto rx=0x7fc5140ae040 tx=0x7fc514019220 comp rx=0 tx=0).stop 2023-12-20T06:03:54.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 -- 172.21.15.32:0/2278713826 shutdown_connections 2023-12-20T06:03:54.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc50c065c10 0x7fc50c0680d0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc520160b30 0x7fc5200c39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc52015ce60 0x7fc520150c10 unknown :-1 s=CLOSED pgs=2385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 --2- 172.21.15.32:0/2278713826 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc520071af0 0x7fc520153810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 -- 172.21.15.32:0/2278713826 >> 172.21.15.32:0/2278713826 conn(0x7fc52006a320 msgr2=0x7fc52006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:54.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 -- 172.21.15.32:0/2278713826 shutdown_connections 2023-12-20T06:03:54.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.402+0000 7fc5265bc640 1 -- 172.21.15.32:0/2278713826 wait complete. 2023-12-20T06:03:54.421 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:03:54.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.566+0000 7fb1bc873640 1 Processor -- start 2023-12-20T06:03:54.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.566+0000 7fb1bc873640 1 -- start start 2023-12-20T06:03:54.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.566+0000 7fb1bc873640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1b80aece0 0x7fb1b80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.566+0000 7fb1bc873640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1b80b0b30 0x7fb1b80701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.566+0000 7fb1bc873640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb1b80c98c0 con 0x7fb1b81599e0 2023-12-20T06:03:54.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.566+0000 7fb1bc873640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb1b80c9ba0 con 0x7fb1b80b0b30 2023-12-20T06:03:54.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.566+0000 7fb1bc873640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b80c9e80 con 0x7fb1b80aece0 2023-12-20T06:03:54.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b6d9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb1b80b0b30 0x7fb1b80701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b659c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb1b81599e0 0x7fb1b80c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55862/0 (socket says 172.21.15.32:55862) 2023-12-20T06:03:54.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b659c640 1 -- 172.21.15.32:0/1108869107 learned_addr learned my addr 172.21.15.32:0/1108869107 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:54.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b5d9b640 1 --2- 172.21.15.32:0/1108869107 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1b80aece0 0x7fb1b80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b6d9d640 1 -- 172.21.15.32:0/1108869107 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1b80aece0 msgr2=0x7fb1b80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:54.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b6d9d640 1 --2- 172.21.15.32:0/1108869107 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1b80aece0 0x7fb1b80b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b6d9d640 1 -- 172.21.15.32:0/1108869107 >> v1:172.21.15.32:6789/0 conn(0x7fb1b81599e0 legacy=0x7fb1b80c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b6d9d640 1 -- 172.21.15.32:0/1108869107 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b80ca160 con 0x7fb1b80b0b30 2023-12-20T06:03:54.571 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b559a640 1 -- 172.21.15.32:0/1108869107 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4245777441 0 0) 0x7fb1b80c98c0 con 0x7fb1b81599e0 2023-12-20T06:03:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b6d9d640 1 --2- 172.21.15.32:0/1108869107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b80b0b30 0x7fb1b80701c0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009d90 tx=0x7fb1b001c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=83a4fde8fba1f2ee server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b559a640 1 -- 172.21.15.32:0/1108869107 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b00a5920 con 0x7fb1b80b0b30 2023-12-20T06:03:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b559a640 1 -- 172.21.15.32:0/1108869107 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1b00a5a80 con 0x7fb1b80b0b30 2023-12-20T06:03:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.570+0000 7fb1b559a640 1 -- 172.21.15.32:0/1108869107 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1b00a5e10 con 0x7fb1b80b0b30 2023-12-20T06:03:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 -- 172.21.15.32:0/1108869107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b80b0b30 msgr2=0x7fb1b80701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.573 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 --2- 172.21.15.32:0/1108869107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b80b0b30 0x7fb1b80701c0 secure :-1 s=READY pgs=2387 cs=0 l=1 rev1=1 crypto rx=0x7fb1b0009d90 tx=0x7fb1b001c950 comp rx=0 tx=0).stop 2023-12-20T06:03:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 -- 172.21.15.32:0/1108869107 shutdown_connections 2023-12-20T06:03:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 --2- 172.21.15.32:0/1108869107 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b80b0b30 0x7fb1b80701c0 unknown :-1 s=CLOSED pgs=2387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 --2- 172.21.15.32:0/1108869107 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1b80aece0 0x7fb1b80b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 -- 172.21.15.32:0/1108869107 >> 172.21.15.32:0/1108869107 conn(0x7fb1b806a320 msgr2=0x7fb1b80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 -- 172.21.15.32:0/1108869107 shutdown_connections 2023-12-20T06:03:54.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 -- 172.21.15.32:0/1108869107 wait complete. 2023-12-20T06:03:54.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 Processor -- start 2023-12-20T06:03:54.575 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 -- start start 2023-12-20T06:03:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1b80aece0 0x7fb1b80c8610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b80b0b30 0x7fb1b815aad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1b81599e0 0x7fb1b814fd00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b8162c70 con 0x7fb1b80b0b30 2023-12-20T06:03:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b81620a0 con 0x7fb1b80aece0 2023-12-20T06:03:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1bc873640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb1b8162340 con 0x7fb1b81599e0 2023-12-20T06:03:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1b5d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b80b0b30 0x7fb1b815aad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1b5d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b80b0b30 0x7fb1b815aad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47386/0 (socket says 172.21.15.32:47386) 2023-12-20T06:03:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.574+0000 7fb1b5d9b640 1 -- 172.21.15.32:0/2060156111 learned_addr learned my addr 172.21.15.32:0/2060156111 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1b6d9d640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1b81599e0 0x7fb1b814fd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1b659c640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1b80aece0 0x7fb1b80c8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1b5d9b640 1 -- 172.21.15.32:0/2060156111 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1b81599e0 msgr2=0x7fb1b814fd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1b5d9b640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1b81599e0 0x7fb1b814fd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1b5d9b640 1 -- 172.21.15.32:0/2060156111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1b80aece0 msgr2=0x7fb1b80c8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1b5d9b640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1b80aece0 0x7fb1b80c8610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1b5d9b640 1 -- 172.21.15.32:0/2060156111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb1b0009a70 con 0x7fb1b80b0b30 2023-12-20T06:03:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1b5d9b640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b80b0b30 0x7fb1b815aad0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00e930 tx=0x7fb1ac00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1a6ffd640 1 -- 172.21.15.32:0/2060156111 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac009690 con 0x7fb1b80b0b30 2023-12-20T06:03:54.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1a6ffd640 1 -- 172.21.15.32:0/2060156111 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb1ac00ca00 con 0x7fb1b80b0b30 2023-12-20T06:03:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1a6ffd640 1 -- 172.21.15.32:0/2060156111 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb1ac00cd70 con 0x7fb1b80b0b30 2023-12-20T06:03:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1bc873640 1 -- 172.21.15.32:0/2060156111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb1b8162620 con 0x7fb1b80b0b30 2023-12-20T06:03:54.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1bc873640 1 -- 172.21.15.32:0/2060156111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb1b8150240 con 0x7fb1b80b0b30 2023-12-20T06:03:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1bc873640 1 -- 172.21.15.32:0/2060156111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb1b806acf0 con 0x7fb1b80b0b30 2023-12-20T06:03:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1a6ffd640 1 -- 172.21.15.32:0/2060156111 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 142) v1 ==== 85376+0+0 (secure 0 0 0) 0x7fb1ac013470 con 0x7fb1b80b0b30 2023-12-20T06:03:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1a6ffd640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb198065c40 0x7fb198068100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.578+0000 7fb1a6ffd640 1 -- 172.21.15.32:0/2060156111 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7403+0+0 (secure 0 0 0) 0x7fb1ac084200 con 0x7fb1b80b0b30 2023-12-20T06:03:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.582+0000 7fb1b659c640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb198065c40 0x7fb198068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.582+0000 7fb1b659c640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb198065c40 0x7fb198068100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80010f0 tx=0x7fb1a8009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:54.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.586+0000 7fb1a6ffd640 1 -- 172.21.15.32:0/2060156111 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb1ac051230 con 0x7fb1b80b0b30 2023-12-20T06:03:54.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.774+0000 7fb1bc873640 1 -- 172.21.15.32:0/2060156111 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fb1b804e000 con 0x7fb1b80b0b30 2023-12-20T06:03:54.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.774+0000 7fb1a6ffd640 1 -- 172.21.15.32:0/2060156111 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 118 v118) v1 ==== 95+0+5909 (secure 0 0 0) 0x7fb1ac054850 con 0x7fb1b80b0b30 2023-12-20T06:03:54.777 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:54.777 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":118,"default_fscid":14,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":112,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:21.951240+0000","modified":"2023-12-20T06:03:24.980742+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11301},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11301":{"gid":11301,"name":"a","rank":0,"incarnation":108,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/1623322556","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":1623322556},{"type":"v1","addr":"172.21.15.32:6837","nonce":1623322556}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[29],"metadata_pool":28,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":14},{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:34.513908+0000","modified":"2023-12-20T06:03:37.549965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11298":{"gid":11298,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6836/60134876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":60134876},{"type":"v1","addr":"172.21.15.32:6836","nonce":60134876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-20T06:03:54.777 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 118 2023-12-20T06:03:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 -- 172.21.15.32:0/2060156111 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb198065c40 msgr2=0x7fb198068100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb198065c40 0x7fb198068100 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fb1a80010f0 tx=0x7fb1a8009040 comp rx=0 tx=0).stop 2023-12-20T06:03:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 -- 172.21.15.32:0/2060156111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b80b0b30 msgr2=0x7fb1b815aad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b80b0b30 0x7fb1b815aad0 secure :-1 s=READY pgs=2388 cs=0 l=1 rev1=1 crypto rx=0x7fb1ac00e930 tx=0x7fb1ac00edf0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 -- 172.21.15.32:0/2060156111 shutdown_connections 2023-12-20T06:03:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb198065c40 0x7fb198068100 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb1b81599e0 0x7fb1b814fd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb1b80b0b30 0x7fb1b815aad0 unknown :-1 s=CLOSED pgs=2388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 --2- 172.21.15.32:0/2060156111 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb1b80aece0 0x7fb1b80c8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 -- 172.21.15.32:0/2060156111 >> 172.21.15.32:0/2060156111 conn(0x7fb1b806a320 msgr2=0x7fb1b80c4900 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 -- 172.21.15.32:0/2060156111 shutdown_connections 2023-12-20T06:03:54.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.778+0000 7fb1bc873640 1 -- 172.21.15.32:0/2060156111 wait complete. 2023-12-20T06:03:54.798 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:54.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5aadc0640 1 Processor -- start 2023-12-20T06:03:54.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5aadc0640 1 -- start start 2023-12-20T06:03:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5aadc0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fa5a40aece0 0x7fa5a40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5aadc0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a40c98c0 con 0x7fa5a41599e0 2023-12-20T06:03:54.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5aadc0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa5a40c9ba0 con 0x7fa5a40b0b30 2023-12-20T06:03:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5aadc0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa5a40c9e80 con 0x7fa5a40aece0 2023-12-20T06:03:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a8b5c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa5a41599e0 0x7fa5a40c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55868/0 (socket says 172.21.15.32:55868) 2023-12-20T06:03:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a8b5c640 1 -- 172.21.15.32:0/2779132956 learned_addr learned my addr 172.21.15.32:0/2779132956 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a3fff640 1 --2- 172.21.15.32:0/2779132956 >> v2:172.21.15.39:3300/0 conn(0x7fa5a40aece0 0x7fa5a40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1438693379 0 0) 0x7fa5a40c98c0 con 0x7fa5a41599e0 2023-12-20T06:03:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa594003530 con 0x7fa5a41599e0 2023-12-20T06:03:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3217996938 0 0) 0x7fa5a40c9ba0 con 0x7fa5a40b0b30 2023-12-20T06:03:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa5a40c98c0 con 0x7fa5a40b0b30 2023-12-20T06:03:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 206082972 0 0) 0x7fa594003530 con 0x7fa5a41599e0 2023-12-20T06:03:54.946 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 >> v2:172.21.15.39:3300/0 conn(0x7fa5a40aece0 msgr2=0x7fa5a40b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 --2- 172.21.15.32:0/2779132956 >> v2:172.21.15.39:3300/0 conn(0x7fa5a40aece0 0x7fa5a40b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 >> v1:172.21.15.39:6789/0 conn(0x7fa5a40b0b30 legacy=0x7fa5a40701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a40ca160 con 0x7fa5a41599e0 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fa598001800 con 0x7fa5a41599e0 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 >> v1:172.21.15.32:6789/0 conn(0x7fa5a41599e0 legacy=0x7fa5a40c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 --2- 172.21.15.32:0/2779132956 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa594003b60 0x7fa594003f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 --2- 172.21.15.32:0/2779132956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5940047d0 0x7fa594004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 --2- 172.21.15.32:0/2779132956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa594005190 0x7fa59400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40c9e80 con 0x7fa5940047d0 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa59400c060 con 0x7fa594005190 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa59400c220 con 0x7fa594003b60 2023-12-20T06:03:54.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a3fff640 1 --2- 172.21.15.32:0/2779132956 >> v2:172.21.15.39:3300/0 conn(0x7fa5a40aece0 0x7fa5a40b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a8b5c640 1 --2- 172.21.15.32:0/2779132956 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa594003b60 0x7fa594003f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.942+0000 7fa5a3fff640 1 --2- 172.21.15.32:0/2779132956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5940047d0 0x7fa594004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5a935d640 1 --2- 172.21.15.32:0/2779132956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa594005190 0x7fa59400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5a3fff640 1 -- 172.21.15.32:0/2779132956 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa594003b60 msgr2=0x7fa594003f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5a3fff640 1 --2- 172.21.15.32:0/2779132956 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa594003b60 0x7fa594003f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5a3fff640 1 -- 172.21.15.32:0/2779132956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa594005190 msgr2=0x7fa59400b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5a3fff640 1 --2- 172.21.15.32:0/2779132956 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa594005190 0x7fa59400b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5a3fff640 1 -- 172.21.15.32:0/2779132956 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa594003530 con 0x7fa5940047d0 2023-12-20T06:03:54.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5a935d640 1 -- 172.21.15.32:0/2779132956 reap_dead start 2023-12-20T06:03:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5a3fff640 1 --2- 172.21.15.32:0/2779132956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5940047d0 0x7fa594004c50 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fa590009c90 tx=0x7fa59001cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad36db19a3fc20bc server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa590004020 con 0x7fa5940047d0 2023-12-20T06:03:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa590004180 con 0x7fa5940047d0 2023-12-20T06:03:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5a37fe640 1 -- 172.21.15.32:0/2779132956 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5900a5780 con 0x7fa5940047d0 2023-12-20T06:03:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2779132956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5940047d0 msgr2=0x7fa594004c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.946+0000 7fa5aadc0640 1 --2- 172.21.15.32:0/2779132956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5940047d0 0x7fa594004c50 secure :-1 s=READY pgs=2390 cs=0 l=1 rev1=1 crypto rx=0x7fa590009c90 tx=0x7fa59001cbd0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2779132956 shutdown_connections 2023-12-20T06:03:54.950 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 --2- 172.21.15.32:0/2779132956 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5940047d0 0x7fa594004c50 unknown :-1 s=CLOSED pgs=2390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2779132956 >> 172.21.15.32:0/2779132956 conn(0x7fa5a406a320 msgr2=0x7fa5a40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2779132956 shutdown_connections 2023-12-20T06:03:54.951 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2779132956 wait complete. 2023-12-20T06:03:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 Processor -- start 2023-12-20T06:03:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 -- start start 2023-12-20T06:03:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5940047d0 0x7fa5a40b3270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a414fd90 0x7fa5a40c6240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a41586e0 0x7fa5a4158b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a4150210 con 0x7fa5a414fd90 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a41590a0 con 0x7fa5a41586e0 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5aadc0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa5a40c6780 con 0x7fa5940047d0 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a414fd90 0x7fa5a40c6240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5a3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a414fd90 0x7fa5a40c6240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47402/0 (socket says 172.21.15.32:47402) 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5a3fff640 1 -- 172.21.15.32:0/2807051620 learned_addr learned my addr 172.21.15.32:0/2807051620 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5a8b5c640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5940047d0 0x7fa5a40b3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5a935d640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a41586e0 0x7fa5a4158b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5a3fff640 1 -- 172.21.15.32:0/2807051620 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5940047d0 msgr2=0x7fa5a40b3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5a3fff640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5940047d0 0x7fa5a40b3270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5a3fff640 1 -- 172.21.15.32:0/2807051620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a41586e0 msgr2=0x7fa5a4158b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:54.954 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5a3fff640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a41586e0 0x7fa5a4158b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5a3fff640 1 -- 172.21.15.32:0/2807051620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa5a40c7ac0 con 0x7fa5a414fd90 2023-12-20T06:03:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.950+0000 7fa5a8b5c640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5940047d0 0x7fa5a40b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5a3fff640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a414fd90 0x7fa5a40c6240 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fa59001ca00 tx=0x7fa594005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5a17fa640 1 -- 172.21.15.32:0/2807051620 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5900a5980 con 0x7fa5a414fd90 2023-12-20T06:03:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5a17fa640 1 -- 172.21.15.32:0/2807051620 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa5900a5ae0 con 0x7fa5a414fd90 2023-12-20T06:03:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5a17fa640 1 -- 172.21.15.32:0/2807051620 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa5900a5e90 con 0x7fa5a414fd90 2023-12-20T06:03:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2807051620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa5a40c6a20 con 0x7fa5a414fd90 2023-12-20T06:03:54.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2807051620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa5a40c6fe0 con 0x7fa5a414fd90 2023-12-20T06:03:54.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2807051620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa5a404e000 con 0x7fa5a414fd90 2023-12-20T06:03:54.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5a17fa640 1 -- 172.21.15.32:0/2807051620 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 142) v1 ==== 85376+0+0 (secure 0 0 0) 0x7fa590006040 con 0x7fa5a414fd90 2023-12-20T06:03:54.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5a17fa640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa588065c30 0x7fa5880680f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:54.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5a17fa640 1 -- 172.21.15.32:0/2807051620 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7403+0+0 (secure 0 0 0) 0x7fa59011e680 con 0x7fa5a414fd90 2023-12-20T06:03:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5a8b5c640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa588065c30 0x7fa5880680f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:54.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.954+0000 7fa5a8b5c640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa588065c30 0x7fa5880680f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa598000c10 tx=0x7fa598009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:54.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:54.958+0000 7fa5a17fa640 1 -- 172.21.15.32:0/2807051620 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa5900eb7a0 con 0x7fa5a414fd90 2023-12-20T06:03:55.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.118+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2807051620 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fa5a4072310 con 0x7fa5a414fd90 2023-12-20T06:03:55.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5a17fa640 1 -- 172.21.15.32:0/2807051620 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v236) v1 ==== 74+0+20923 (secure 0 0 0) 0x7fa5900eedc0 con 0x7fa5a414fd90 2023-12-20T06:03:55.122 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:55.122 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":236,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:51.776828+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":28,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:03:20.399088+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"229","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":29,"pool_name":"cephfs_data","create_time":"2023-12-20T06:03:20.926367+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"230","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.6299999952316284,"score_stable":1.6299999952316284,"optimal_score":1,"raw_score_acting":1.6299999952316284,"raw_score_stable":1.6299999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:03:32.174574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:03:33.509224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1232037079":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/2587801540":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/789250710":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/1603186500":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/2783112768":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/3054548475":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2026838481":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/597963739":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/274812680":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/3886671554":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1847933543":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/4013366117":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/582500261":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/805068630":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6833/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1823890749":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6832/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2852735264":"2023-12-21T06:03:45.663020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:55.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2807051620 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa588065c30 msgr2=0x7fa5880680f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa588065c30 0x7fa5880680f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7fa598000c10 tx=0x7fa598009040 comp rx=0 tx=0).stop 2023-12-20T06:03:55.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2807051620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a414fd90 msgr2=0x7fa5a40c6240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a414fd90 0x7fa5a40c6240 secure :-1 s=READY pgs=2391 cs=0 l=1 rev1=1 crypto rx=0x7fa59001ca00 tx=0x7fa594005800 comp rx=0 tx=0).stop 2023-12-20T06:03:55.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2807051620 shutdown_connections 2023-12-20T06:03:55.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa588065c30 0x7fa5880680f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa5a41586e0 0x7fa5a4158b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa5a414fd90 0x7fa5a40c6240 unknown :-1 s=CLOSED pgs=2391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 --2- 172.21.15.32:0/2807051620 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa5940047d0 0x7fa5a40b3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2807051620 >> 172.21.15.32:0/2807051620 conn(0x7fa5a406a320 msgr2=0x7fa5a41e4080 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:55.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2807051620 shutdown_connections 2023-12-20T06:03:55.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.122+0000 7fa5aadc0640 1 -- 172.21.15.32:0/2807051620 wait complete. 2023-12-20T06:03:55.144 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T06:03:55.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b8df0640 1 Processor -- start 2023-12-20T06:03:55.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b8df0640 1 -- start start 2023-12-20T06:03:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b8df0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f76b40aece0 0x7f76b40b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b8df0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b40c98c0 con 0x7f76b40b0b30 2023-12-20T06:03:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b8df0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76b40c9ba0 con 0x7f76b41599e0 2023-12-20T06:03:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b8df0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76b40c9e80 con 0x7f76b40aece0 2023-12-20T06:03:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b2d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f76b40b0b30 0x7f76b40701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55882/0 (socket says 172.21.15.32:55882) 2023-12-20T06:03:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b2d9d640 1 -- 172.21.15.32:0/1220505091 learned_addr learned my addr 172.21.15.32:0/1220505091 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:55.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b1d9b640 1 --2- 172.21.15.32:0/1220505091 >> v2:172.21.15.39:3300/0 conn(0x7f76b40aece0 0x7f76b40b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b159a640 1 -- 172.21.15.32:0/1220505091 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 835733281 0 0) 0x7f76b40c98c0 con 0x7f76b40b0b30 2023-12-20T06:03:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b159a640 1 -- 172.21.15.32:0/1220505091 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f769c003530 con 0x7f76b40b0b30 2023-12-20T06:03:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b1d9b640 1 -- 172.21.15.32:0/1220505091 >> v1:172.21.15.39:6789/0 conn(0x7f76b41599e0 legacy=0x7f76b40c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b1d9b640 1 -- 172.21.15.32:0/1220505091 >> v1:172.21.15.32:6789/0 conn(0x7f76b40b0b30 legacy=0x7f76b40701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.290+0000 7f76b1d9b640 1 -- 172.21.15.32:0/1220505091 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76b40ca160 con 0x7f76b40aece0 2023-12-20T06:03:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.294+0000 7f76b1d9b640 1 --2- 172.21.15.32:0/1220505091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b40aece0 0x7f76b40b05f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f76a80097c0 tx=0x7f76a801c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=1f6b0107d675c631 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.294+0000 7f76b159a640 1 -- 172.21.15.32:0/1220505091 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a80a8050 con 0x7f76b40aece0 2023-12-20T06:03:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.294+0000 7f76b159a640 1 -- 172.21.15.32:0/1220505091 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a80ac5f0 con 0x7f76b40aece0 2023-12-20T06:03:55.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.294+0000 7f76b159a640 1 -- 172.21.15.32:0/1220505091 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a80ac980 con 0x7f76b40aece0 2023-12-20T06:03:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.294+0000 7f76b8df0640 1 -- 172.21.15.32:0/1220505091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b40aece0 msgr2=0x7f76b40b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.294+0000 7f76b8df0640 1 --2- 172.21.15.32:0/1220505091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b40aece0 0x7f76b40b05f0 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f76a80097c0 tx=0x7f76a801c480 comp rx=0 tx=0).stop 2023-12-20T06:03:55.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.294+0000 7f76b8df0640 1 -- 172.21.15.32:0/1220505091 shutdown_connections 2023-12-20T06:03:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.294+0000 7f76b8df0640 1 --2- 172.21.15.32:0/1220505091 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b40aece0 0x7f76b40b05f0 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.294+0000 7f76b8df0640 1 -- 172.21.15.32:0/1220505091 >> 172.21.15.32:0/1220505091 conn(0x7f76b406a320 msgr2=0x7f76b40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.294+0000 7f76b8df0640 1 -- 172.21.15.32:0/1220505091 shutdown_connections 2023-12-20T06:03:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.294+0000 7f76b8df0640 1 -- 172.21.15.32:0/1220505091 wait complete. 2023-12-20T06:03:55.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b8df0640 1 Processor -- start 2023-12-20T06:03:55.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b8df0640 1 -- start start 2023-12-20T06:03:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b8df0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b40aece0 0x7f76b414fd70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b8df0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b40b0b30 0x7f76b415c510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b8df0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76b41599e0 0x7f76b4150cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b8df0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b4152250 con 0x7f76b40b0b30 2023-12-20T06:03:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b8df0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76b41502b0 con 0x7f76b40aece0 2023-12-20T06:03:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b8df0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76b415b470 con 0x7f76b41599e0 2023-12-20T06:03:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b1d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b40b0b30 0x7f76b415c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b1d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b40b0b30 0x7f76b415c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47410/0 (socket says 172.21.15.32:47410) 2023-12-20T06:03:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b1d9b640 1 -- 172.21.15.32:0/1509389305 learned_addr learned my addr 172.21.15.32:0/1509389305 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b259c640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b40aece0 0x7f76b414fd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b2d9d640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76b41599e0 0x7f76b4150cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b1d9b640 1 -- 172.21.15.32:0/1509389305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76b41599e0 msgr2=0x7f76b4150cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b1d9b640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76b41599e0 0x7f76b4150cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b1d9b640 1 -- 172.21.15.32:0/1509389305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b40aece0 msgr2=0x7f76b414fd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b1d9b640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b40aece0 0x7f76b414fd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b1d9b640 1 -- 172.21.15.32:0/1509389305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76a8009420 con 0x7f76b40b0b30 2023-12-20T06:03:55.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b1d9b640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b40b0b30 0x7f76b415c510 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f76a801a0b0 tx=0x7f76a8019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:55.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76a2ffd640 1 -- 172.21.15.32:0/1509389305 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a8006e20 con 0x7f76b40b0b30 2023-12-20T06:03:55.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76a2ffd640 1 -- 172.21.15.32:0/1509389305 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76a80b5620 con 0x7f76b40b0b30 2023-12-20T06:03:55.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76a2ffd640 1 -- 172.21.15.32:0/1509389305 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76a80b59b0 con 0x7f76b40b0b30 2023-12-20T06:03:55.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b8df0640 1 -- 172.21.15.32:0/1509389305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76b415b750 con 0x7f76b40b0b30 2023-12-20T06:03:55.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.298+0000 7f76b8df0640 1 -- 172.21.15.32:0/1509389305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76b415bd10 con 0x7f76b40b0b30 2023-12-20T06:03:55.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.302+0000 7f76b8df0640 1 -- 172.21.15.32:0/1509389305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76b404e000 con 0x7f76b40b0b30 2023-12-20T06:03:55.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.302+0000 7f76a2ffd640 1 -- 172.21.15.32:0/1509389305 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 142) v1 ==== 85376+0+0 (secure 0 0 0) 0x7f76a80b3040 con 0x7f76b40b0b30 2023-12-20T06:03:55.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.302+0000 7f76a2ffd640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7694065c40 0x7f7694068100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.302+0000 7f76a2ffd640 1 -- 172.21.15.32:0/1509389305 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(236..236 src has 1..236) v4 ==== 7403+0+0 (secure 0 0 0) 0x7f76a811d800 con 0x7f76b40b0b30 2023-12-20T06:03:55.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.302+0000 7f76b259c640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7694065c40 0x7f7694068100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.302+0000 7f76b259c640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7694065c40 0x7f7694068100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f76a4000fc0 tx=0x7f76a4009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:55.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.306+0000 7f76a2ffd640 1 -- 172.21.15.32:0/1509389305 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f76a80ea710 con 0x7f76b40b0b30 2023-12-20T06:03:55.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.502+0000 7f76b8df0640 1 -- 172.21.15.32:0/1509389305 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f76b4165860 con 0x7f76b40b0b30 2023-12-20T06:03:55.502 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T06:03:55.502+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T06:03:55.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76a2ffd640 1 -- 172.21.15.32:0/1509389305 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v119) v1 ==== 161+0+0 (secure 0 0 0) 0x7f76a80b1020 con 0x7f76b40b0b30 2023-12-20T06:03:55.527 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T06:03:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76b8df0640 1 -- 172.21.15.32:0/1509389305 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7694065c40 msgr2=0x7f7694068100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76b8df0640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7694065c40 0x7f7694068100 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f76a4000fc0 tx=0x7f76a4009040 comp rx=0 tx=0).stop 2023-12-20T06:03:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76b8df0640 1 -- 172.21.15.32:0/1509389305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b40b0b30 msgr2=0x7f76b415c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76b8df0640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b40b0b30 0x7f76b415c510 secure :-1 s=READY pgs=2393 cs=0 l=1 rev1=1 crypto rx=0x7f76a801a0b0 tx=0x7f76a8019220 comp rx=0 tx=0).stop 2023-12-20T06:03:55.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76b8df0640 1 -- 172.21.15.32:0/1509389305 shutdown_connections 2023-12-20T06:03:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76b8df0640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7694065c40 0x7f7694068100 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76b8df0640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76b41599e0 0x7f76b4150cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76b8df0640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76b40b0b30 0x7f76b415c510 unknown :-1 s=CLOSED pgs=2393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76b8df0640 1 --2- 172.21.15.32:0/1509389305 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76b40aece0 0x7f76b414fd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76b8df0640 1 -- 172.21.15.32:0/1509389305 >> 172.21.15.32:0/1509389305 conn(0x7f76b406a320 msgr2=0x7f76b4156fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.526+0000 7f76b8df0640 1 -- 172.21.15.32:0/1509389305 shutdown_connections 2023-12-20T06:03:55.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.530+0000 7f76b8df0640 1 -- 172.21.15.32:0/1509389305 wait complete. 2023-12-20T06:03:55.534 INFO:tasks.ceph.mds.a.smithi032.stderr: -4187> 2023-12-20T06:03:23.375+0000 7f0fc3cef640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T06:03:55.549 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T06:03:55.586 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T06:03:55.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4403020640 1 Processor -- start 2023-12-20T06:03:55.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4403020640 1 -- start start 2023-12-20T06:03:55.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4403020640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43fc15abb0 0x7f43fc0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4403020640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43fc0b0cb0 0x7f43fc15bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4403020640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f43fc070230 0x7f43fc0706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4403020640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43fc06e160 con 0x7f43fc15abb0 2023-12-20T06:03:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4403020640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43fc070be0 con 0x7f43fc070230 2023-12-20T06:03:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4403020640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc15b6a0 con 0x7f43fc0b0cb0 2023-12-20T06:03:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4400dbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43fc15abb0 0x7f43fc0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4400dbc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f43fc15abb0 0x7f43fc0caaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47416/0 (socket says 172.21.15.32:47416) 2023-12-20T06:03:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4400dbc640 1 -- 172.21.15.32:0/3281560113 learned_addr learned my addr 172.21.15.32:0/3281560113 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f44015bd640 1 --2- 172.21.15.32:0/3281560113 >> v2:172.21.15.39:3300/0 conn(0x7f43fc070230 0x7f43fc0706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f43fbfff640 1 --2- 172.21.15.32:0/3281560113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43fc0b0cb0 0x7f43fc15bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4400dbc640 1 -- 172.21.15.32:0/3281560113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43fc0b0cb0 msgr2=0x7f43fc15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4400dbc640 1 --2- 172.21.15.32:0/3281560113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43fc0b0cb0 0x7f43fc15bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4400dbc640 1 -- 172.21.15.32:0/3281560113 >> v2:172.21.15.39:3300/0 conn(0x7f43fc070230 msgr2=0x7f43fc0706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4400dbc640 1 --2- 172.21.15.32:0/3281560113 >> v2:172.21.15.39:3300/0 conn(0x7f43fc070230 0x7f43fc0706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.698+0000 7f4400dbc640 1 -- 172.21.15.32:0/3281560113 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43fc15c070 con 0x7f43fc15abb0 2023-12-20T06:03:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f4400dbc640 1 --2- 172.21.15.32:0/3281560113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43fc15abb0 0x7f43fc0caaf0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0b4470 tx=0x7f43f001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dfd1077d2944d8d9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f43fb7fe640 1 -- 172.21.15.32:0/3281560113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f00aa620 con 0x7f43fc15abb0 2023-12-20T06:03:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f43fb7fe640 1 -- 172.21.15.32:0/3281560113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f00aa780 con 0x7f43fc15abb0 2023-12-20T06:03:55.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f43fb7fe640 1 -- 172.21.15.32:0/3281560113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f00aab10 con 0x7f43fc15abb0 2023-12-20T06:03:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f4403020640 1 -- 172.21.15.32:0/3281560113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43fc15abb0 msgr2=0x7f43fc0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f4403020640 1 --2- 172.21.15.32:0/3281560113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43fc15abb0 0x7f43fc0caaf0 secure :-1 s=READY pgs=2394 cs=0 l=1 rev1=1 crypto rx=0x7f43fc0b4470 tx=0x7f43f001c500 comp rx=0 tx=0).stop 2023-12-20T06:03:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f4403020640 1 -- 172.21.15.32:0/3281560113 shutdown_connections 2023-12-20T06:03:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f4403020640 1 --2- 172.21.15.32:0/3281560113 >> v2:172.21.15.39:3300/0 conn(0x7f43fc070230 0x7f43fc0706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f4403020640 1 --2- 172.21.15.32:0/3281560113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43fc0b0cb0 0x7f43fc15bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f4403020640 1 --2- 172.21.15.32:0/3281560113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43fc15abb0 0x7f43fc0caaf0 unknown :-1 s=CLOSED pgs=2394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f4403020640 1 -- 172.21.15.32:0/3281560113 >> 172.21.15.32:0/3281560113 conn(0x7f43fc06a510 msgr2=0x7f43fc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f4403020640 1 -- 172.21.15.32:0/3281560113 shutdown_connections 2023-12-20T06:03:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.702+0000 7f4403020640 1 -- 172.21.15.32:0/3281560113 wait complete. 2023-12-20T06:03:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4403020640 1 Processor -- start 2023-12-20T06:03:55.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4403020640 1 -- start start 2023-12-20T06:03:55.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4403020640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43fc070230 0x7f43fc149760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4403020640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43fc0b0cb0 0x7f43fc1527c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4403020640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43fc15abb0 0x7f43fc1f54e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4403020640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc149ca0 con 0x7f43fc15abb0 2023-12-20T06:03:55.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4403020640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc152330 con 0x7f43fc0b0cb0 2023-12-20T06:03:55.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4403020640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43fc151a00 con 0x7f43fc070230 2023-12-20T06:03:55.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f44015bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43fc15abb0 0x7f43fc1f54e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f44015bd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43fc15abb0 0x7f43fc1f54e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47432/0 (socket says 172.21.15.32:47432) 2023-12-20T06:03:55.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f44015bd640 1 -- 172.21.15.32:0/946808506 learned_addr learned my addr 172.21.15.32:0/946808506 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:55.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f43fbfff640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43fc0b0cb0 0x7f43fc1527c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4400dbc640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43fc070230 0x7f43fc149760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f44015bd640 1 -- 172.21.15.32:0/946808506 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43fc070230 msgr2=0x7f43fc149760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f44015bd640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43fc070230 0x7f43fc149760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f44015bd640 1 -- 172.21.15.32:0/946808506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43fc0b0cb0 msgr2=0x7f43fc1527c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:55.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f44015bd640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43fc0b0cb0 0x7f43fc1527c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:55.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f44015bd640 1 -- 172.21.15.32:0/946808506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43f00094d0 con 0x7f43fc15abb0 2023-12-20T06:03:55.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4400dbc640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43fc070230 0x7f43fc149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:55.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f44015bd640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43fc15abb0 0x7f43fc1f54e0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f43f400eed0 tx=0x7f43f400b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:55.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f43f97fa640 1 -- 172.21.15.32:0/946808506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4010040 con 0x7f43fc15abb0 2023-12-20T06:03:55.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f43f97fa640 1 -- 172.21.15.32:0/946808506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43f400bd20 con 0x7f43fc15abb0 2023-12-20T06:03:55.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4403020640 1 -- 172.21.15.32:0/946808506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f43fc151ce0 con 0x7f43fc15abb0 2023-12-20T06:03:55.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f43f97fa640 1 -- 172.21.15.32:0/946808506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43f4014640 con 0x7f43fc15abb0 2023-12-20T06:03:55.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.706+0000 7f4403020640 1 -- 172.21.15.32:0/946808506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43fc14c1a0 con 0x7f43fc15abb0 2023-12-20T06:03:55.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.710+0000 7f4403020640 1 -- 172.21.15.32:0/946808506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f43fc04e000 con 0x7f43fc15abb0 2023-12-20T06:03:55.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.710+0000 7f43f97fa640 1 -- 172.21.15.32:0/946808506 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 142) v1 ==== 85376+0+0 (secure 0 0 0) 0x7f43f40147a0 con 0x7f43fc15abb0 2023-12-20T06:03:55.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.710+0000 7f43f97fa640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f43e0065c10 0x7f43e00680d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:55.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.710+0000 7f43f97fa640 1 -- 172.21.15.32:0/946808506 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7489+0+0 (secure 0 0 0) 0x7f43f4084350 con 0x7f43fc15abb0 2023-12-20T06:03:55.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.710+0000 7f4400dbc640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f43e0065c10 0x7f43e00680d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:55.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.710+0000 7f4400dbc640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f43e0065c10 0x7f43e00680d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f43f00ae000 tx=0x7f43f0008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:55.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.714+0000 7f43f97fa640 1 -- 172.21.15.32:0/946808506 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f43f4051280 con 0x7f43fc15abb0 2023-12-20T06:03:55.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:55.910+0000 7f4403020640 1 -- 172.21.15.32:0/946808506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f43fc152470 con 0x7f43fc15abb0 2023-12-20T06:03:56.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.538+0000 7f43f97fa640 1 -- 172.21.15.32:0/946808506 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f43f4051650 con 0x7f43fc15abb0 2023-12-20T06:03:56.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f43f97fa640 1 -- 172.21.15.32:0/946808506 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v120) v1 ==== 104+0+0 (secure 0 0 0) 0x7f43f407fe70 con 0x7f43fc15abb0 2023-12-20T06:03:56.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 -- 172.21.15.32:0/946808506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f43e0065c10 msgr2=0x7f43e00680d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:56.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f43e0065c10 0x7f43e00680d0 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7f43f00ae000 tx=0x7f43f0008040 comp rx=0 tx=0).stop 2023-12-20T06:03:56.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 -- 172.21.15.32:0/946808506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43fc15abb0 msgr2=0x7f43fc1f54e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:56.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43fc15abb0 0x7f43fc1f54e0 secure :-1 s=READY pgs=2395 cs=0 l=1 rev1=1 crypto rx=0x7f43f400eed0 tx=0x7f43f400b480 comp rx=0 tx=0).stop 2023-12-20T06:03:56.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 -- 172.21.15.32:0/946808506 shutdown_connections 2023-12-20T06:03:56.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f43e0065c10 0x7f43e00680d0 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:56.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43fc15abb0 0x7f43fc1f54e0 unknown :-1 s=CLOSED pgs=2395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:56.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43fc0b0cb0 0x7f43fc1527c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:56.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 --2- 172.21.15.32:0/946808506 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43fc070230 0x7f43fc149760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:56.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 -- 172.21.15.32:0/946808506 >> 172.21.15.32:0/946808506 conn(0x7f43fc06a510 msgr2=0x7f43fc1544d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:56.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 -- 172.21.15.32:0/946808506 shutdown_connections 2023-12-20T06:03:56.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.542+0000 7f4403020640 1 -- 172.21.15.32:0/946808506 wait complete. 2023-12-20T06:03:56.566 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T06:03:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f623ddcc640 1 Processor -- start 2023-12-20T06:03:56.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f623ddcc640 1 -- start start 2023-12-20T06:03:56.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f623ddcc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f62380ca1e0 0x7f623806d8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:56.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f623ddcc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f623806dde0 0x7f6238070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:56.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f623ddcc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f62380ca6a0 con 0x7f62380ce040 2023-12-20T06:03:56.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f623ddcc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f62380ca980 con 0x7f62380ca1e0 2023-12-20T06:03:56.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f623ddcc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62380cac60 con 0x7f623806dde0 2023-12-20T06:03:56.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f62377fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f62380ce040 0x7f62381608a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55898/0 (socket says 172.21.15.32:55898) 2023-12-20T06:03:56.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f62377fe640 1 -- 172.21.15.32:0/3667270483 learned_addr learned my addr 172.21.15.32:0/3667270483 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:56.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f6237fff640 1 --2- 172.21.15.32:0/3667270483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f623806dde0 0x7f6238070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:56.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f6236ffd640 1 --2- 172.21.15.32:0/3667270483 >> v2:172.21.15.39:3300/0 conn(0x7f62380ca1e0 0x7f623806d8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:56.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f62367fc640 1 -- 172.21.15.32:0/3667270483 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1003128184 0 0) 0x7f62380ca6a0 con 0x7f62380ce040 2023-12-20T06:03:56.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f62367fc640 1 -- 172.21.15.32:0/3667270483 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6220003530 con 0x7f62380ce040 2023-12-20T06:03:56.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f6237fff640 1 -- 172.21.15.32:0/3667270483 >> v2:172.21.15.39:3300/0 conn(0x7f62380ca1e0 msgr2=0x7f623806d8a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:56.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f6237fff640 1 --2- 172.21.15.32:0/3667270483 >> v2:172.21.15.39:3300/0 conn(0x7f62380ca1e0 0x7f623806d8a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:56.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f62367fc640 1 -- 172.21.15.32:0/3667270483 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1423188833 0 0) 0x7f6220003530 con 0x7f62380ce040 2023-12-20T06:03:56.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f6237fff640 1 -- 172.21.15.32:0/3667270483 >> v1:172.21.15.32:6789/0 conn(0x7f62380ce040 legacy=0x7f62381608a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:56.762 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.758+0000 7f6237fff640 1 -- 172.21.15.32:0/3667270483 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6238160fb0 con 0x7f623806dde0 2023-12-20T06:03:56.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f6237fff640 1 --2- 172.21.15.32:0/3667270483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f623806dde0 0x7f6238070230 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6230009d90 tx=0x7f623001c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=f87002c121cda168 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:56.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f62367fc640 1 -- 172.21.15.32:0/3667270483 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6228001680 con 0x7f623806dde0 2023-12-20T06:03:56.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f62367fc640 1 -- 172.21.15.32:0/3667270483 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62300a6500 con 0x7f623806dde0 2023-12-20T06:03:56.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f62367fc640 1 -- 172.21.15.32:0/3667270483 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62300a6890 con 0x7f623806dde0 2023-12-20T06:03:56.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f623ddcc640 1 -- 172.21.15.32:0/3667270483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f623806dde0 msgr2=0x7f6238070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:56.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f623ddcc640 1 --2- 172.21.15.32:0/3667270483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f623806dde0 0x7f6238070230 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f6230009d90 tx=0x7f623001c7e0 comp rx=0 tx=0).stop 2023-12-20T06:03:56.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f623ddcc640 1 -- 172.21.15.32:0/3667270483 shutdown_connections 2023-12-20T06:03:56.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f623ddcc640 1 --2- 172.21.15.32:0/3667270483 >> v2:172.21.15.39:3300/0 conn(0x7f62380ca1e0 0x7f623806d8a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:56.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f623ddcc640 1 --2- 172.21.15.32:0/3667270483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f623806dde0 0x7f6238070230 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:56.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f623ddcc640 1 -- 172.21.15.32:0/3667270483 >> 172.21.15.32:0/3667270483 conn(0x7f623806a510 msgr2=0x7f62380ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:56.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f623ddcc640 1 -- 172.21.15.32:0/3667270483 shutdown_connections 2023-12-20T06:03:56.765 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.762+0000 7f623ddcc640 1 -- 172.21.15.32:0/3667270483 wait complete. 2023-12-20T06:03:56.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f623ddcc640 1 Processor -- start 2023-12-20T06:03:56.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f623ddcc640 1 -- start start 2023-12-20T06:03:56.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f623ddcc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f623806dde0 0x7f62380c5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:56.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f623ddcc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62380ca1e0 0x7f623814ca00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:56.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f623ddcc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62380ce040 0x7f623815b400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:56.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f623ddcc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62380c8900 con 0x7f62380ce040 2023-12-20T06:03:56.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f623ddcc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f62380c63e0 con 0x7f623806dde0 2023-12-20T06:03:56.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f623ddcc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f62380c7e20 con 0x7f62380ca1e0 2023-12-20T06:03:56.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f6237fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62380ce040 0x7f623815b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:56.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f6237fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62380ce040 0x7f623815b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47440/0 (socket says 172.21.15.32:47440) 2023-12-20T06:03:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f6237fff640 1 -- 172.21.15.32:0/1324426526 learned_addr learned my addr 172.21.15.32:0/1324426526 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f62377fe640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f623806dde0 0x7f62380c5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f6236ffd640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62380ca1e0 0x7f623814ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f6237fff640 1 -- 172.21.15.32:0/1324426526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62380ca1e0 msgr2=0x7f623814ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f6237fff640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62380ca1e0 0x7f623814ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f6237fff640 1 -- 172.21.15.32:0/1324426526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f623806dde0 msgr2=0x7f62380c5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f6237fff640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f623806dde0 0x7f62380c5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f6237fff640 1 -- 172.21.15.32:0/1324426526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6230009a70 con 0x7f62380ce040 2023-12-20T06:03:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f6237fff640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62380ce040 0x7f623815b400 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f62300b0040 tx=0x7f62380cb150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f621ffff640 1 -- 172.21.15.32:0/1324426526 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62300b8630 con 0x7f62380ce040 2023-12-20T06:03:56.770 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f621ffff640 1 -- 172.21.15.32:0/1324426526 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f62300b87b0 con 0x7f62380ce040 2023-12-20T06:03:56.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f621ffff640 1 -- 172.21.15.32:0/1324426526 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f62300b8b60 con 0x7f62380ce040 2023-12-20T06:03:56.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.766+0000 7f623ddcc640 1 -- 172.21.15.32:0/1324426526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f62380c8100 con 0x7f62380ce040 2023-12-20T06:03:56.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.770+0000 7f623ddcc640 1 -- 172.21.15.32:0/1324426526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f62380c7ad0 con 0x7f62380ce040 2023-12-20T06:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.770+0000 7f623ddcc640 1 -- 172.21.15.32:0/1324426526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f623804e000 con 0x7f62380ce040 2023-12-20T06:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.770+0000 7f621ffff640 1 -- 172.21.15.32:0/1324426526 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f62300b8cc0 con 0x7f62380ce040 2023-12-20T06:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.770+0000 7f621ffff640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6224065ca0 0x7f6224068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.770+0000 7f621ffff640 1 -- 172.21.15.32:0/1324426526 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(237..237 src has 1..237) v4 ==== 7489+0+0 (secure 0 0 0) 0x7f62300aa070 con 0x7f62380ce040 2023-12-20T06:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.770+0000 7f62377fe640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6224065ca0 0x7f6224068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:56.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.770+0000 7f62377fe640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6224065ca0 0x7f6224068160 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6228000fc0 tx=0x7f62280019c0 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:56.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.774+0000 7f621ffff640 1 -- 172.21.15.32:0/1324426526 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f62300ec100 con 0x7f62380ce040 2023-12-20T06:03:56.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:56.938+0000 7f623ddcc640 1 -- 172.21.15.32:0/1324426526 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f623806d040 con 0x7f62380ce040 2023-12-20T06:03:57.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.834+0000 7f621ffff640 1 -- 172.21.15.32:0/1324426526 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v238) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6230009450 con 0x7f62380ce040 2023-12-20T06:03:57.836 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T06:03:57.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 -- 172.21.15.32:0/1324426526 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6224065ca0 msgr2=0x7f6224068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:57.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6224065ca0 0x7f6224068160 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f6228000fc0 tx=0x7f62280019c0 comp rx=0 tx=0).stop 2023-12-20T06:03:57.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 -- 172.21.15.32:0/1324426526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62380ce040 msgr2=0x7f623815b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:57.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62380ce040 0x7f623815b400 secure :-1 s=READY pgs=2397 cs=0 l=1 rev1=1 crypto rx=0x7f62300b0040 tx=0x7f62380cb150 comp rx=0 tx=0).stop 2023-12-20T06:03:57.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 -- 172.21.15.32:0/1324426526 shutdown_connections 2023-12-20T06:03:57.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6224065ca0 0x7f6224068160 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:57.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f62380ce040 0x7f623815b400 unknown :-1 s=CLOSED pgs=2397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:57.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f62380ca1e0 0x7f623814ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:57.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 --2- 172.21.15.32:0/1324426526 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f623806dde0 0x7f62380c5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:57.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 -- 172.21.15.32:0/1324426526 >> 172.21.15.32:0/1324426526 conn(0x7f623806a510 msgr2=0x7f623815ca40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:57.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 -- 172.21.15.32:0/1324426526 shutdown_connections 2023-12-20T06:03:57.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:57.838+0000 7f623ddcc640 1 -- 172.21.15.32:0/1324426526 wait complete. 2023-12-20T06:03:57.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T06:03:58.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.014+0000 7f6bd162d640 1 Processor -- start 2023-12-20T06:03:58.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bd162d640 1 -- start start 2023-12-20T06:03:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bd162d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6bcc06dcb0 0x7f6bcc0ca050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bd162d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc06e100 con 0x7f6bcc070230 2023-12-20T06:03:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bd162d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6bcc0ca590 con 0x7f6bcc158520 2023-12-20T06:03:58.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bd162d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc0ca7f0 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bcb7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6bcc070230 0x7f6bcc15b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55904/0 (socket says 172.21.15.32:55904) 2023-12-20T06:03:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bcb7fe640 1 -- 172.21.15.32:0/1668135403 learned_addr learned my addr 172.21.15.32:0/1668135403 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bc3fff640 1 --2- 172.21.15.32:0/1668135403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6bcc06dcb0 0x7f6bcc0ca050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bca7fc640 1 -- 172.21.15.32:0/1668135403 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1731284357 0 0) 0x7f6bc40014a0 con 0x7f6bcc070230 2023-12-20T06:03:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bca7fc640 1 -- 172.21.15.32:0/1668135403 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6bb8003530 con 0x7f6bcc070230 2023-12-20T06:03:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bca7fc640 1 -- 172.21.15.32:0/1668135403 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2682769427 0 0) 0x7f6bcc0ca590 con 0x7f6bcc158520 2023-12-20T06:03:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bc3fff640 1 -- 172.21.15.32:0/1668135403 >> v1:172.21.15.39:6789/0 conn(0x7f6bcc158520 legacy=0x7f6bcc15ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bc3fff640 1 -- 172.21.15.32:0/1668135403 >> v1:172.21.15.32:6789/0 conn(0x7f6bcc070230 legacy=0x7f6bcc15b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:58.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.018+0000 7f6bc3fff640 1 -- 172.21.15.32:0/1668135403 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bcc0caad0 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.022+0000 7f6bc3fff640 1 --2- 172.21.15.32:0/1668135403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6bcc06dcb0 0x7f6bcc0ca050 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40097f0 tx=0x7f6bb401c240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e32aea32a058cfc server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:58.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.022+0000 7f6bca7fc640 1 -- 172.21.15.32:0/1668135403 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bc4001b90 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.022+0000 7f6bca7fc640 1 -- 172.21.15.32:0/1668135403 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bb40a7050 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.022+0000 7f6bca7fc640 1 -- 172.21.15.32:0/1668135403 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bb40ab620 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.022+0000 7f6bd162d640 1 -- 172.21.15.32:0/1668135403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6bcc06dcb0 msgr2=0x7f6bcc0ca050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:58.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.022+0000 7f6bd162d640 1 --2- 172.21.15.32:0/1668135403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6bcc06dcb0 0x7f6bcc0ca050 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40097f0 tx=0x7f6bb401c240 comp rx=0 tx=0).stop 2023-12-20T06:03:58.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.022+0000 7f6bd162d640 1 -- 172.21.15.32:0/1668135403 shutdown_connections 2023-12-20T06:03:58.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.022+0000 7f6bd162d640 1 --2- 172.21.15.32:0/1668135403 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6bcc06dcb0 0x7f6bcc0ca050 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:58.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.022+0000 7f6bd162d640 1 -- 172.21.15.32:0/1668135403 >> 172.21.15.32:0/1668135403 conn(0x7f6bcc06a510 msgr2=0x7f6bcc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:58.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.022+0000 7f6bd162d640 1 -- 172.21.15.32:0/1668135403 shutdown_connections 2023-12-20T06:03:58.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.022+0000 7f6bd162d640 1 -- 172.21.15.32:0/1668135403 wait complete. 2023-12-20T06:03:58.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bd162d640 1 Processor -- start 2023-12-20T06:03:58.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bd162d640 1 -- start start 2023-12-20T06:03:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bd162d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc06dcb0 0x7f6bcc15a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bd162d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bcc070230 0x7f6bcc14f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bd162d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6bcc158520 0x7f6bcc14fe80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bd162d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc15aaa0 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bd162d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc166590 con 0x7f6bcc070230 2023-12-20T06:03:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bd162d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6bcc14fc60 con 0x7f6bcc158520 2023-12-20T06:03:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bcaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc06dcb0 0x7f6bcc15a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bcaffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc06dcb0 0x7f6bcc15a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:47450/0 (socket says 172.21.15.32:47450) 2023-12-20T06:03:58.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bcaffd640 1 -- 172.21.15.32:0/2823246859 learned_addr learned my addr 172.21.15.32:0/2823246859 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bc3fff640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bcc070230 0x7f6bcc14f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bcb7fe640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6bcc158520 0x7f6bcc14fe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bcaffd640 1 -- 172.21.15.32:0/2823246859 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6bcc158520 msgr2=0x7f6bcc14fe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bcaffd640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6bcc158520 0x7f6bcc14fe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bcaffd640 1 -- 172.21.15.32:0/2823246859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bcc070230 msgr2=0x7f6bcc14f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bcaffd640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bcc070230 0x7f6bcc14f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bcaffd640 1 -- 172.21.15.32:0/2823246859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6bb4009420 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bcaffd640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc06dcb0 0x7f6bcc15a560 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc001ec0 tx=0x7f6bbc00c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:58.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bc37fe640 1 -- 172.21.15.32:0/2823246859 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0054e0 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bc37fe640 1 -- 172.21.15.32:0/2823246859 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6bbc003600 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bd162d640 1 -- 172.21.15.32:0/2823246859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6bcc163ca0 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bc37fe640 1 -- 172.21.15.32:0/2823246859 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6bbc0039b0 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bd162d640 1 -- 172.21.15.32:0/2823246859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6bcc164260 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.026+0000 7f6bd162d640 1 -- 172.21.15.32:0/2823246859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6bcc04e000 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.030+0000 7f6bc37fe640 1 -- 172.21.15.32:0/2823246859 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f6bbc003b10 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.030+0000 7f6bc37fe640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6bac065ca0 0x7f6bac068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.030+0000 7f6bc37fe640 1 -- 172.21.15.32:0/2823246859 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(238..238 src has 1..238) v4 ==== 7051+0+0 (secure 0 0 0) 0x7f6bbc0831a0 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.030+0000 7f6bc3fff640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6bac065ca0 0x7f6bac068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:58.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.030+0000 7f6bc3fff640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6bac065ca0 0x7f6bac068160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40ad000 tx=0x7f6bb40a6000 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:58.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.034+0000 7f6bc37fe640 1 -- 172.21.15.32:0/2823246859 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6bbc050220 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.198+0000 7f6bd162d640 1 -- 172.21.15.32:0/2823246859 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6bcc15f250 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.838 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T06:03:58.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.838+0000 7f6bc37fe640 1 -- 172.21.15.32:0/2823246859 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v239) v1 ==== 169+0+0 (secure 0 0 0) 0x7f6bbc0170d0 con 0x7f6bcc06dcb0 2023-12-20T06:03:58.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.838+0000 7f6bd162d640 1 -- 172.21.15.32:0/2823246859 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6bac065ca0 msgr2=0x7f6bac068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:58.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.838+0000 7f6bd162d640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6bac065ca0 0x7f6bac068160 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f6bb40ad000 tx=0x7f6bb40a6000 comp rx=0 tx=0).stop 2023-12-20T06:03:58.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.838+0000 7f6bd162d640 1 -- 172.21.15.32:0/2823246859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc06dcb0 msgr2=0x7f6bcc15a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:58.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.838+0000 7f6bd162d640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc06dcb0 0x7f6bcc15a560 secure :-1 s=READY pgs=2399 cs=0 l=1 rev1=1 crypto rx=0x7f6bbc001ec0 tx=0x7f6bbc00c8e0 comp rx=0 tx=0).stop 2023-12-20T06:03:58.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.842+0000 7f6bd162d640 1 -- 172.21.15.32:0/2823246859 shutdown_connections 2023-12-20T06:03:58.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.842+0000 7f6bd162d640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6bac065ca0 0x7f6bac068160 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:58.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.842+0000 7f6bd162d640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6bcc158520 0x7f6bcc14fe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:58.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.842+0000 7f6bd162d640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6bcc070230 0x7f6bcc14f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:58.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.842+0000 7f6bd162d640 1 --2- 172.21.15.32:0/2823246859 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6bcc06dcb0 0x7f6bcc15a560 unknown :-1 s=CLOSED pgs=2399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:58.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.842+0000 7f6bd162d640 1 -- 172.21.15.32:0/2823246859 >> 172.21.15.32:0/2823246859 conn(0x7f6bcc06a510 msgr2=0x7f6bcc0c36f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:58.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.842+0000 7f6bd162d640 1 -- 172.21.15.32:0/2823246859 shutdown_connections 2023-12-20T06:03:58.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:58.842+0000 7f6bd162d640 1 -- 172.21.15.32:0/2823246859 wait complete. 2023-12-20T06:03:58.864 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:03:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.022+0000 7f178ea66640 1 Processor -- start 2023-12-20T06:03:59.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.022+0000 7f178ea66640 1 -- start start 2023-12-20T06:03:59.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.022+0000 7f178ea66640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17880aece0 0x7f17880b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.022+0000 7f178ea66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17880b0b30 0x7f17880701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.022+0000 7f178ea66640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f17880c98c0 con 0x7f17881599e0 2023-12-20T06:03:59.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.022+0000 7f178ea66640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f17880c9ba0 con 0x7f17880aece0 2023-12-20T06:03:59.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.022+0000 7f178ea66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17880c9e80 con 0x7f17880b0b30 2023-12-20T06:03:59.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.022+0000 7f1787fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f17880aece0 0x7f17880b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.022+0000 7f178c802640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f17881599e0 0x7f17880c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36154/0 (socket says 172.21.15.32:36154) 2023-12-20T06:03:59.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.022+0000 7f178c802640 1 -- 172.21.15.32:0/491946546 learned_addr learned my addr 172.21.15.32:0/491946546 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:59.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.026+0000 7f178d003640 1 --2- 172.21.15.32:0/491946546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17880b0b30 0x7f17880701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.026+0000 7f1787fff640 1 -- 172.21.15.32:0/491946546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17880b0b30 msgr2=0x7f17880701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:59.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.026+0000 7f1787fff640 1 --2- 172.21.15.32:0/491946546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17880b0b30 0x7f17880701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.026+0000 7f1787fff640 1 -- 172.21.15.32:0/491946546 >> v1:172.21.15.32:6789/0 conn(0x7f17881599e0 legacy=0x7f17880c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.026+0000 7f1787fff640 1 -- 172.21.15.32:0/491946546 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f17880ca160 con 0x7f17880aece0 2023-12-20T06:03:59.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.026+0000 7f17877fe640 1 -- 172.21.15.32:0/491946546 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3560524980 0 0) 0x7f17880c98c0 con 0x7f17881599e0 2023-12-20T06:03:59.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.026+0000 7f1787fff640 1 --2- 172.21.15.32:0/491946546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17880aece0 0x7f17880b05f0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f17740097c0 tx=0x7f177401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a915a5d4f42b6e7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:59.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.026+0000 7f17877fe640 1 -- 172.21.15.32:0/491946546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17740ad640 con 0x7f17880aece0 2023-12-20T06:03:59.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.026+0000 7f17877fe640 1 -- 172.21.15.32:0/491946546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17740ad7a0 con 0x7f17880aece0 2023-12-20T06:03:59.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.026+0000 7f17877fe640 1 -- 172.21.15.32:0/491946546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17740adb30 con 0x7f17880aece0 2023-12-20T06:03:59.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 -- 172.21.15.32:0/491946546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17880aece0 msgr2=0x7f17880b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 --2- 172.21.15.32:0/491946546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17880aece0 0x7f17880b05f0 secure :-1 s=READY pgs=2401 cs=0 l=1 rev1=1 crypto rx=0x7f17740097c0 tx=0x7f177401c480 comp rx=0 tx=0).stop 2023-12-20T06:03:59.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 -- 172.21.15.32:0/491946546 shutdown_connections 2023-12-20T06:03:59.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 --2- 172.21.15.32:0/491946546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17880aece0 0x7f17880b05f0 unknown :-1 s=CLOSED pgs=2401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 --2- 172.21.15.32:0/491946546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17880b0b30 0x7f17880701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 -- 172.21.15.32:0/491946546 >> 172.21.15.32:0/491946546 conn(0x7f178806a320 msgr2=0x7f17880ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:59.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 -- 172.21.15.32:0/491946546 shutdown_connections 2023-12-20T06:03:59.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 -- 172.21.15.32:0/491946546 wait complete. 2023-12-20T06:03:59.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 Processor -- start 2023-12-20T06:03:59.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 -- start start 2023-12-20T06:03:59.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17880aece0 0x7f178814d560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17880b0b30 0x7f178815aa80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17881599e0 0x7f178815c600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f178814dda0 con 0x7f17880b0b30 2023-12-20T06:03:59.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f17880ac1c0 con 0x7f17881599e0 2023-12-20T06:03:59.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178ea66640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f17880ac460 con 0x7f17880aece0 2023-12-20T06:03:59.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f1787fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17880b0b30 0x7f178815aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f1787fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17880b0b30 0x7f178815aa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58188/0 (socket says 172.21.15.32:58188) 2023-12-20T06:03:59.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f1787fff640 1 -- 172.21.15.32:0/1576492209 learned_addr learned my addr 172.21.15.32:0/1576492209 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:59.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178d003640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17881599e0 0x7f178815c600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178c802640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17880aece0 0x7f178814d560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f1787fff640 1 -- 172.21.15.32:0/1576492209 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17880aece0 msgr2=0x7f178814d560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f1787fff640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17880aece0 0x7f178814d560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f1787fff640 1 -- 172.21.15.32:0/1576492209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17881599e0 msgr2=0x7f178815c600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f1787fff640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17881599e0 0x7f178815c600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f1787fff640 1 -- 172.21.15.32:0/1576492209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1774009420 con 0x7f17880b0b30 2023-12-20T06:03:59.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178c802640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17880aece0 0x7f178814d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:59.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.030+0000 7f178d003640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17881599e0 0x7f178815c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:03:59.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.034+0000 7f1787fff640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17880b0b30 0x7f178815aa80 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f177401a0b0 tx=0x7f1774019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:59.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.034+0000 7f17857fa640 1 -- 172.21.15.32:0/1576492209 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17740b6610 con 0x7f17880b0b30 2023-12-20T06:03:59.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.034+0000 7f17857fa640 1 -- 172.21.15.32:0/1576492209 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f17740a6040 con 0x7f17880b0b30 2023-12-20T06:03:59.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.034+0000 7f178ea66640 1 -- 172.21.15.32:0/1576492209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f17880ac740 con 0x7f17880b0b30 2023-12-20T06:03:59.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.034+0000 7f17857fa640 1 -- 172.21.15.32:0/1576492209 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f17740b69b0 con 0x7f17880b0b30 2023-12-20T06:03:59.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.034+0000 7f178ea66640 1 -- 172.21.15.32:0/1576492209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f17880acd00 con 0x7f17880b0b30 2023-12-20T06:03:59.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.034+0000 7f178ea66640 1 -- 172.21.15.32:0/1576492209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f178806acf0 con 0x7f17880b0b30 2023-12-20T06:03:59.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.034+0000 7f17857fa640 1 -- 172.21.15.32:0/1576492209 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f17740b4040 con 0x7f17880b0b30 2023-12-20T06:03:59.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.034+0000 7f17857fa640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f176c065cd0 0x7f176c068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.038+0000 7f17857fa640 1 -- 172.21.15.32:0/1576492209 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f177411db80 con 0x7f17880b0b30 2023-12-20T06:03:59.037 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.038+0000 7f178c802640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f176c065cd0 0x7f176c068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.038+0000 7f178c802640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f176c065cd0 0x7f176c068190 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f177c004880 tx=0x7f177c008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:59.040 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.038+0000 7f17857fa640 1 -- 172.21.15.32:0/1576492209 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f17740eaf00 con 0x7f17880b0b30 2023-12-20T06:03:59.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.198+0000 7f178ea66640 1 -- 172.21.15.32:0/1576492209 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f178804e000 con 0x7f17880b0b30 2023-12-20T06:03:59.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f17857fa640 1 -- 172.21.15.32:0/1576492209 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+17341 (secure 0 0 0) 0x7f177411de60 con 0x7f17880b0b30 2023-12-20T06:03:59.202 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:03:59.202 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":239,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:58.834991+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:03:32.174574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:03:33.509224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/1623322556":"2023-12-21T06:03:55.505418+0000","172.21.15.32:0/1232037079":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/2587801540":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/789250710":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/1603186500":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/2783112768":"2023-12-21T06:03:51.776767+0000","172.21.15.32:6837/1623322556":"2023-12-21T06:03:55.505418+0000","172.21.15.32:0/3054548475":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2026838481":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/597963739":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/274812680":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/3886671554":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1847933543":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/4013366117":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/582500261":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/805068630":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6833/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1823890749":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6832/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2852735264":"2023-12-21T06:03:45.663020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:03:59.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 -- 172.21.15.32:0/1576492209 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f176c065cd0 msgr2=0x7f176c068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f176c065cd0 0x7f176c068190 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f177c004880 tx=0x7f177c008040 comp rx=0 tx=0).stop 2023-12-20T06:03:59.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 -- 172.21.15.32:0/1576492209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17880b0b30 msgr2=0x7f178815aa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17880b0b30 0x7f178815aa80 secure :-1 s=READY pgs=2402 cs=0 l=1 rev1=1 crypto rx=0x7f177401a0b0 tx=0x7f1774019220 comp rx=0 tx=0).stop 2023-12-20T06:03:59.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 -- 172.21.15.32:0/1576492209 shutdown_connections 2023-12-20T06:03:59.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f176c065cd0 0x7f176c068190 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f17881599e0 0x7f178815c600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f17880b0b30 0x7f178815aa80 unknown :-1 s=CLOSED pgs=2402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 --2- 172.21.15.32:0/1576492209 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f17880aece0 0x7f178814d560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 -- 172.21.15.32:0/1576492209 >> 172.21.15.32:0/1576492209 conn(0x7f178806a320 msgr2=0x7f1788071320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 -- 172.21.15.32:0/1576492209 shutdown_connections 2023-12-20T06:03:59.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.202+0000 7f178ea66640 1 -- 172.21.15.32:0/1576492209 wait complete. 2023-12-20T06:03:59.224 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 30, 'pool_name': 'backup_fs_metadata', 'create_time': '2023-12-20T06:03:32.174574+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '233', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'backup_fs'}}, 'read_balance': {'score_acting': 2.5, 'score_stable': 2.5, 'optimal_score': 1, 'raw_score_acting': 2.5, 'raw_score_stable': 2.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 31, 'pool_name': 'backup_fs_data', 'create_time': '2023-12-20T06:03:33.509224+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '234', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'backup_fs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:03:59.224 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:03:59.371 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.370+0000 7f76c968c640 1 Processor -- start 2023-12-20T06:03:59.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.370+0000 7f76c968c640 1 -- start start 2023-12-20T06:03:59.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c968c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f76c40b0cb0 0x7f76c40caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c968c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76c4070230 0x7f76c4070680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c968c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f76c4070bc0 con 0x7f76c415abb0 2023-12-20T06:03:59.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c968c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c406e160 con 0x7f76c4070230 2023-12-20T06:03:59.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c968c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f76c415b6a0 con 0x7f76c40b0cb0 2023-12-20T06:03:59.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f76c4070230 0x7f76c4070680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c2ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f76c415abb0 0x7f76c415bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36164/0 (socket says 172.21.15.32:36164) 2023-12-20T06:03:59.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c2ffd640 1 -- 172.21.15.32:0/2794837018 learned_addr learned my addr 172.21.15.32:0/2794837018 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:59.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76bbfff640 1 --2- 172.21.15.32:0/2794837018 >> v2:172.21.15.39:3300/0 conn(0x7f76c40b0cb0 0x7f76c40caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c37fe640 1 -- 172.21.15.32:0/2794837018 >> v2:172.21.15.39:3300/0 conn(0x7f76c40b0cb0 msgr2=0x7f76c40caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c37fe640 1 --2- 172.21.15.32:0/2794837018 >> v2:172.21.15.39:3300/0 conn(0x7f76c40b0cb0 0x7f76c40caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c37fe640 1 -- 172.21.15.32:0/2794837018 >> v1:172.21.15.32:6789/0 conn(0x7f76c415abb0 legacy=0x7f76c415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c37fe640 1 -- 172.21.15.32:0/2794837018 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76c406d8c0 con 0x7f76c4070230 2023-12-20T06:03:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c27fc640 1 -- 172.21.15.32:0/2794837018 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2621958900 0 0) 0x7f76b4001120 con 0x7f76c415abb0 2023-12-20T06:03:59.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76bbfff640 1 --2- 172.21.15.32:0/2794837018 >> v2:172.21.15.39:3300/0 conn(0x7f76c40b0cb0 0x7f76c40caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:03:59.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c37fe640 1 --2- 172.21.15.32:0/2794837018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c4070230 0x7f76c4070680 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f76c40b4470 tx=0x7f76bc01c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8eee4d9b42c7fe7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:59.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c27fc640 1 -- 172.21.15.32:0/2794837018 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0a59d0 con 0x7f76c4070230 2023-12-20T06:03:59.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c27fc640 1 -- 172.21.15.32:0/2794837018 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76bc0a5b30 con 0x7f76c4070230 2023-12-20T06:03:59.377 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.374+0000 7f76c27fc640 1 -- 172.21.15.32:0/2794837018 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76bc0ad440 con 0x7f76c4070230 2023-12-20T06:03:59.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 -- 172.21.15.32:0/2794837018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c4070230 msgr2=0x7f76c4070680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 --2- 172.21.15.32:0/2794837018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c4070230 0x7f76c4070680 secure :-1 s=READY pgs=2404 cs=0 l=1 rev1=1 crypto rx=0x7f76c40b4470 tx=0x7f76bc01c910 comp rx=0 tx=0).stop 2023-12-20T06:03:59.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 -- 172.21.15.32:0/2794837018 shutdown_connections 2023-12-20T06:03:59.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 --2- 172.21.15.32:0/2794837018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c4070230 0x7f76c4070680 unknown :-1 s=CLOSED pgs=2404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 --2- 172.21.15.32:0/2794837018 >> v2:172.21.15.39:3300/0 conn(0x7f76c40b0cb0 0x7f76c40caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 -- 172.21.15.32:0/2794837018 >> 172.21.15.32:0/2794837018 conn(0x7f76c406a510 msgr2=0x7f76c40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:59.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 -- 172.21.15.32:0/2794837018 shutdown_connections 2023-12-20T06:03:59.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 -- 172.21.15.32:0/2794837018 wait complete. 2023-12-20T06:03:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 Processor -- start 2023-12-20T06:03:59.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 -- start start 2023-12-20T06:03:59.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c4070230 0x7f76c4143ef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40b0cb0 0x7f76c4147100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.381 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76c415abb0 0x7f76c4147640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4152e90 con 0x7f76c40b0cb0 2023-12-20T06:03:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4142010 con 0x7f76c4070230 2023-12-20T06:03:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76c968c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f76c4141d30 con 0x7f76c415abb0 2023-12-20T06:03:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40b0cb0 0x7f76c4147100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76bbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40b0cb0 0x7f76c4147100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58210/0 (socket says 172.21.15.32:58210) 2023-12-20T06:03:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.378+0000 7f76bbfff640 1 -- 172.21.15.32:0/4106939336 learned_addr learned my addr 172.21.15.32:0/4106939336 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bbfff640 1 -- 172.21.15.32:0/4106939336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76c415abb0 msgr2=0x7f76c4147640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bbfff640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76c415abb0 0x7f76c4147640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bbfff640 1 -- 172.21.15.32:0/4106939336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c4070230 msgr2=0x7f76c4143ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bbfff640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c4070230 0x7f76c4143ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bbfff640 1 -- 172.21.15.32:0/4106939336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f76bc009a70 con 0x7f76c40b0cb0 2023-12-20T06:03:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bbfff640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40b0cb0 0x7f76c4147100 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00e7e0 tx=0x7f76ac00eca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bb7fe640 1 -- 172.21.15.32:0/4106939336 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00c6d0 con 0x7f76c40b0cb0 2023-12-20T06:03:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bb7fe640 1 -- 172.21.15.32:0/4106939336 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f76ac00c830 con 0x7f76c40b0cb0 2023-12-20T06:03:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bb7fe640 1 -- 172.21.15.32:0/4106939336 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f76ac00cbe0 con 0x7f76c40b0cb0 2023-12-20T06:03:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76c968c640 1 -- 172.21.15.32:0/4106939336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f76c4143410 con 0x7f76c40b0cb0 2023-12-20T06:03:59.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76c968c640 1 -- 172.21.15.32:0/4106939336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f76c41439d0 con 0x7f76c40b0cb0 2023-12-20T06:03:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76c968c640 1 -- 172.21.15.32:0/4106939336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f76c40b4000 con 0x7f76c40b0cb0 2023-12-20T06:03:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bb7fe640 1 -- 172.21.15.32:0/4106939336 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f76ac00cd40 con 0x7f76c40b0cb0 2023-12-20T06:03:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bb7fe640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f76a4065ca0 0x7f76a4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76bb7fe640 1 -- 172.21.15.32:0/4106939336 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f76ac083810 con 0x7f76c40b0cb0 2023-12-20T06:03:59.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.382+0000 7f76c2ffd640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f76a4065ca0 0x7f76a4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.386+0000 7f76c2ffd640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f76a4065ca0 0x7f76a4068160 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f76c41531d0 tx=0x7f76b4007970 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:59.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.386+0000 7f76bb7fe640 1 -- 172.21.15.32:0/4106939336 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f76ac050b20 con 0x7f76c40b0cb0 2023-12-20T06:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 -- 172.21.15.32:0/4106939336 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f76c404e000 con 0x7f76c40b0cb0 2023-12-20T06:03:59.545 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76bb7fe640 1 -- 172.21.15.32:0/4106939336 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v239) v1 ==== 93+0+10 (secure 0 0 0) 0x7f76ac054140 con 0x7f76c40b0cb0 2023-12-20T06:03:59.546 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:03:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 -- 172.21.15.32:0/4106939336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f76a4065ca0 msgr2=0x7f76a4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f76a4065ca0 0x7f76a4068160 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f76c41531d0 tx=0x7f76b4007970 comp rx=0 tx=0).stop 2023-12-20T06:03:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 -- 172.21.15.32:0/4106939336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40b0cb0 msgr2=0x7f76c4147100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40b0cb0 0x7f76c4147100 secure :-1 s=READY pgs=2405 cs=0 l=1 rev1=1 crypto rx=0x7f76ac00e7e0 tx=0x7f76ac00eca0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 -- 172.21.15.32:0/4106939336 shutdown_connections 2023-12-20T06:03:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f76a4065ca0 0x7f76a4068160 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f76c415abb0 0x7f76c4147640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f76c40b0cb0 0x7f76c4147100 unknown :-1 s=CLOSED pgs=2405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 --2- 172.21.15.32:0/4106939336 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f76c4070230 0x7f76c4143ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 -- 172.21.15.32:0/4106939336 >> 172.21.15.32:0/4106939336 conn(0x7f76c406a510 msgr2=0x7f76c4154b70 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:59.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 -- 172.21.15.32:0/4106939336 shutdown_connections 2023-12-20T06:03:59.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.546+0000 7f76c968c640 1 -- 172.21.15.32:0/4106939336 wait complete. 2023-12-20T06:03:59.567 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_metadata pg_num 2023-12-20T06:03:59.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2867b96640 1 Processor -- start 2023-12-20T06:03:59.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2867b96640 1 -- start start 2023-12-20T06:03:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2867b96640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28600b0760 0x7f28600b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2867b96640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f28600cc4b0 0x7f28600cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2867b96640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f28600b1100 con 0x7f2860157480 2023-12-20T06:03:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2867b96640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f28600c9f50 con 0x7f28600cc4b0 2023-12-20T06:03:59.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2867b96640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f28600cde20 con 0x7f28600b0760 2023-12-20T06:03:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2865131640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28600b0760 0x7f28600b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2865932640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f2860157480 0x7f28600c9840 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:42268/0 (socket says 172.21.15.32:42268) 2023-12-20T06:03:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2865932640 1 -- 172.21.15.32:0/373872913 learned_addr learned my addr 172.21.15.32:0/373872913 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:03:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2866133640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f28600cc4b0 0x7f28600cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2864930640 1 -- 172.21.15.32:0/373872913 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2186124936 0 0) 0x7f28600b1100 con 0x7f2860157480 2023-12-20T06:03:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2864930640 1 -- 172.21.15.32:0/373872913 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f284c003530 con 0x7f2860157480 2023-12-20T06:03:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2866133640 1 -- 172.21.15.32:0/373872913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28600b0760 msgr2=0x7f28600b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:03:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2866133640 1 --2- 172.21.15.32:0/373872913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28600b0760 0x7f28600b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2866133640 1 -- 172.21.15.32:0/373872913 >> v1:172.21.15.39:6789/0 conn(0x7f2860157480 legacy=0x7f28600c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2866133640 1 -- 172.21.15.32:0/373872913 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f28600c8e30 con 0x7f28600cc4b0 2023-12-20T06:03:59.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.714+0000 7f2865131640 1 --2- 172.21.15.32:0/373872913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28600b0760 0x7f28600b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2866133640 1 --2- 172.21.15.32:0/373872913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600cc4b0 0x7f28600cc940 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2858009d90 tx=0x7f285801c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=ed923a4a001c7d37 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2864930640 1 -- 172.21.15.32:0/373872913 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28580a5950 con 0x7f28600cc4b0 2023-12-20T06:03:59.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2864930640 1 -- 172.21.15.32:0/373872913 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f28580a5ab0 con 0x7f28600cc4b0 2023-12-20T06:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2864930640 1 -- 172.21.15.32:0/373872913 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f28580a5e20 con 0x7f28600cc4b0 2023-12-20T06:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2867b96640 1 -- 172.21.15.32:0/373872913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600cc4b0 msgr2=0x7f28600cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2867b96640 1 --2- 172.21.15.32:0/373872913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600cc4b0 0x7f28600cc940 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f2858009d90 tx=0x7f285801c890 comp rx=0 tx=0).stop 2023-12-20T06:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2867b96640 1 -- 172.21.15.32:0/373872913 shutdown_connections 2023-12-20T06:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2867b96640 1 --2- 172.21.15.32:0/373872913 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600cc4b0 0x7f28600cc940 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2867b96640 1 --2- 172.21.15.32:0/373872913 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f28600b0760 0x7f28600b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2867b96640 1 -- 172.21.15.32:0/373872913 >> 172.21.15.32:0/373872913 conn(0x7f286006a510 msgr2=0x7f28600ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2867b96640 1 -- 172.21.15.32:0/373872913 shutdown_connections 2023-12-20T06:03:59.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.718+0000 7f2867b96640 1 -- 172.21.15.32:0/373872913 wait complete. 2023-12-20T06:03:59.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2867b96640 1 Processor -- start 2023-12-20T06:03:59.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2867b96640 1 -- start start 2023-12-20T06:03:59.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2867b96640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b0760 0x7f286014f060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2867b96640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28600cc4b0 0x7f286015adf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2867b96640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2860157480 0x7f286015d570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2867b96640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f286014f5a0 con 0x7f28600cc4b0 2023-12-20T06:03:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2867b96640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2860154800 con 0x7f28600b0760 2023-12-20T06:03:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2867b96640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2860154ae0 con 0x7f2860157480 2023-12-20T06:03:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2865131640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28600cc4b0 0x7f286015adf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2865131640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28600cc4b0 0x7f286015adf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58222/0 (socket says 172.21.15.32:58222) 2023-12-20T06:03:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2865131640 1 -- 172.21.15.32:0/2858960887 learned_addr learned my addr 172.21.15.32:0/2858960887 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:03:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2865932640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b0760 0x7f286014f060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2866133640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2860157480 0x7f286015d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2865131640 1 -- 172.21.15.32:0/2858960887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2860157480 msgr2=0x7f286015d570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2865131640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2860157480 0x7f286015d570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2865131640 1 -- 172.21.15.32:0/2858960887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b0760 msgr2=0x7f286014f060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2865131640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b0760 0x7f286014f060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2865131640 1 -- 172.21.15.32:0/2858960887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2850009420 con 0x7f28600cc4b0 2023-12-20T06:03:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2865131640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28600cc4b0 0x7f286015adf0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f285000e980 tx=0x7f285000ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f28567fc640 1 -- 172.21.15.32:0/2858960887 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f285000ca00 con 0x7f28600cc4b0 2023-12-20T06:03:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f28567fc640 1 -- 172.21.15.32:0/2858960887 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f285000cb60 con 0x7f28600cc4b0 2023-12-20T06:03:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f28567fc640 1 -- 172.21.15.32:0/2858960887 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2850013470 con 0x7f28600cc4b0 2023-12-20T06:03:59.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2867b96640 1 -- 172.21.15.32:0/2858960887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2858009a70 con 0x7f28600cc4b0 2023-12-20T06:03:59.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.722+0000 7f2867b96640 1 -- 172.21.15.32:0/2858960887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f28601550a0 con 0x7f28600cc4b0 2023-12-20T06:03:59.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.726+0000 7f2867b96640 1 -- 172.21.15.32:0/2858960887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f286004e000 con 0x7f28600cc4b0 2023-12-20T06:03:59.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.726+0000 7f28567fc640 1 -- 172.21.15.32:0/2858960887 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f28500135d0 con 0x7f28600cc4b0 2023-12-20T06:03:59.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.726+0000 7f28567fc640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2844065ca0 0x7f2844068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:03:59.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.726+0000 7f28567fc640 1 -- 172.21.15.32:0/2858960887 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f2850083c10 con 0x7f28600cc4b0 2023-12-20T06:03:59.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.726+0000 7f2865932640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2844065ca0 0x7f2844068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:03:59.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.726+0000 7f2865932640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2844065ca0 0x7f2844068160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f285c007fd0 tx=0x7f285c009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:03:59.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.730+0000 7f28567fc640 1 -- 172.21.15.32:0/2858960887 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2850050ee0 con 0x7f28600cc4b0 2023-12-20T06:03:59.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.886+0000 7f2867b96640 1 -- 172.21.15.32:0/2858960887 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"} v 0) v1 -- 0x7f28601670a0 con 0x7f28600cc4b0 2023-12-20T06:03:59.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.886+0000 7f28567fc640 1 -- 172.21.15.32:0/2858960887 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_metadata", "var": "pg_num"}]=0 v239) v1 ==== 107+0+11 (secure 0 0 0) 0x7f28500170b0 con 0x7f28600cc4b0 2023-12-20T06:03:59.888 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:03:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 -- 172.21.15.32:0/2858960887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2844065ca0 msgr2=0x7f2844068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2844065ca0 0x7f2844068160 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f285c007fd0 tx=0x7f285c009040 comp rx=0 tx=0).stop 2023-12-20T06:03:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 -- 172.21.15.32:0/2858960887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28600cc4b0 msgr2=0x7f286015adf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:03:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28600cc4b0 0x7f286015adf0 secure :-1 s=READY pgs=2406 cs=0 l=1 rev1=1 crypto rx=0x7f285000e980 tx=0x7f285000ee40 comp rx=0 tx=0).stop 2023-12-20T06:03:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 -- 172.21.15.32:0/2858960887 shutdown_connections 2023-12-20T06:03:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2844065ca0 0x7f2844068160 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2860157480 0x7f286015d570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f28600cc4b0 0x7f286015adf0 unknown :-1 s=CLOSED pgs=2406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 --2- 172.21.15.32:0/2858960887 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f28600b0760 0x7f286014f060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:03:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 -- 172.21.15.32:0/2858960887 >> 172.21.15.32:0/2858960887 conn(0x7f286006a510 msgr2=0x7f28600c9840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:03:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 -- 172.21.15.32:0/2858960887 shutdown_connections 2023-12-20T06:03:59.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:03:59.890+0000 7f2867b96640 1 -- 172.21.15.32:0/2858960887 wait complete. 2023-12-20T06:03:59.909 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get backup_fs_data pg_num 2023-12-20T06:04:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.058+0000 7f0a4c6b6640 1 Processor -- start 2023-12-20T06:04:00.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.058+0000 7f0a4c6b6640 1 -- start start 2023-12-20T06:04:00.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.058+0000 7f0a4c6b6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f0a44157480 0x7f0a440c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.058+0000 7f0a4c6b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440cd4c0 0x7f0a440b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.058+0000 7f0a4c6b6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a440b1100 con 0x7f0a440b0760 2023-12-20T06:04:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.058+0000 7f0a4c6b6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a440cde20 con 0x7f0a44157480 2023-12-20T06:04:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.058+0000 7f0a4c6b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440c8e30 con 0x7f0a440cd4c0 2023-12-20T06:04:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a4ac53640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440cd4c0 0x7f0a440b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a49c51640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f0a440b0760 0x7f0a4415d660 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:42278/0 (socket says 172.21.15.32:42278) 2023-12-20T06:04:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a49c51640 1 -- 172.21.15.32:0/3842861660 learned_addr learned my addr 172.21.15.32:0/3842861660 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a4ac53640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440cd4c0 0x7f0a440b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:34350/0 (socket says 172.21.15.32:34350) 2023-12-20T06:04:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a4a452640 1 --2- 172.21.15.32:0/3842861660 >> v2:172.21.15.39:3300/0 conn(0x7f0a44157480 0x7f0a440c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a49450640 1 -- 172.21.15.32:0/3842861660 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1615345793 0 0) 0x7f0a440b1100 con 0x7f0a440b0760 2023-12-20T06:04:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a49450640 1 -- 172.21.15.32:0/3842861660 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a30003530 con 0x7f0a440b0760 2023-12-20T06:04:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a4a452640 1 -- 172.21.15.32:0/3842861660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440cd4c0 msgr2=0x7f0a440b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:00.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a4a452640 1 --2- 172.21.15.32:0/3842861660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440cd4c0 0x7f0a440b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a4a452640 1 -- 172.21.15.32:0/3842861660 >> v1:172.21.15.39:6789/0 conn(0x7f0a440b0760 legacy=0x7f0a4415d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a4a452640 1 -- 172.21.15.32:0/3842861660 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a440ce070 con 0x7f0a44157480 2023-12-20T06:04:00.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a4ac53640 1 --2- 172.21.15.32:0/3842861660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440cd4c0 0x7f0a440b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:00.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a4a452640 1 --2- 172.21.15.32:0/3842861660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a44157480 0x7f0a440c9840 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000bd90 tx=0x7f0a4001e890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=9376f1677dec2139 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a49450640 1 -- 172.21.15.32:0/3842861660 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a400a7880 con 0x7f0a44157480 2023-12-20T06:04:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a49450640 1 -- 172.21.15.32:0/3842861660 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a400a79e0 con 0x7f0a44157480 2023-12-20T06:04:00.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.062+0000 7f0a49450640 1 -- 172.21.15.32:0/3842861660 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a400a7d70 con 0x7f0a44157480 2023-12-20T06:04:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3842861660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a44157480 msgr2=0x7f0a440c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 --2- 172.21.15.32:0/3842861660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a44157480 0x7f0a440c9840 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000bd90 tx=0x7f0a4001e890 comp rx=0 tx=0).stop 2023-12-20T06:04:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3842861660 shutdown_connections 2023-12-20T06:04:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 --2- 172.21.15.32:0/3842861660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440cd4c0 0x7f0a440b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 --2- 172.21.15.32:0/3842861660 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a44157480 0x7f0a440c9840 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3842861660 >> 172.21.15.32:0/3842861660 conn(0x7f0a4406a510 msgr2=0x7f0a440ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3842861660 shutdown_connections 2023-12-20T06:04:00.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3842861660 wait complete. 2023-12-20T06:04:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 Processor -- start 2023-12-20T06:04:00.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 -- start start 2023-12-20T06:04:00.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440b0760 0x7f0a44163590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a440cd4c0 0x7f0a441f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a44157480 0x7f0a440c38c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440c5d40 con 0x7f0a44157480 2023-12-20T06:04:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440c6020 con 0x7f0a440cd4c0 2023-12-20T06:04:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4c6b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a440c6300 con 0x7f0a440b0760 2023-12-20T06:04:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4ac53640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a44157480 0x7f0a440c38c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4a452640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440b0760 0x7f0a44163590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4ac53640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a44157480 0x7f0a440c38c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58236/0 (socket says 172.21.15.32:58236) 2023-12-20T06:04:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a4ac53640 1 -- 172.21.15.32:0/3100068074 learned_addr learned my addr 172.21.15.32:0/3100068074 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.066+0000 7f0a49c51640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a440cd4c0 0x7f0a441f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a4ac53640 1 -- 172.21.15.32:0/3100068074 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440b0760 msgr2=0x7f0a44163590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a4ac53640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440b0760 0x7f0a44163590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a4ac53640 1 -- 172.21.15.32:0/3100068074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a440cd4c0 msgr2=0x7f0a441f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a4ac53640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a440cd4c0 0x7f0a441f58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a4ac53640 1 -- 172.21.15.32:0/3100068074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c009a70 con 0x7f0a44157480 2023-12-20T06:04:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a4ac53640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a44157480 0x7f0a440c38c0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00efb0 tx=0x7f0a3c00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a3affd640 1 -- 172.21.15.32:0/3100068074 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c014480 con 0x7f0a44157480 2023-12-20T06:04:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a3affd640 1 -- 172.21.15.32:0/3100068074 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c00f040 con 0x7f0a44157480 2023-12-20T06:04:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a3affd640 1 -- 172.21.15.32:0/3100068074 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c014800 con 0x7f0a44157480 2023-12-20T06:04:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3100068074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a4000ba70 con 0x7f0a44157480 2023-12-20T06:04:00.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3100068074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a441565b0 con 0x7f0a44157480 2023-12-20T06:04:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3100068074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a440c6b00 con 0x7f0a44157480 2023-12-20T06:04:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a3affd640 1 -- 172.21.15.32:0/3100068074 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f0a3c014960 con 0x7f0a44157480 2023-12-20T06:04:00.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a3affd640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a28065ca0 0x7f0a28068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a3affd640 1 -- 172.21.15.32:0/3100068074 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f0a3c084900 con 0x7f0a44157480 2023-12-20T06:04:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.070+0000 7f0a4a452640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a28065ca0 0x7f0a28068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.073 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.074+0000 7f0a4a452640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a28065ca0 0x7f0a28068160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a400b1000 tx=0x7f0a400a9070 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:00.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.074+0000 7f0a3affd640 1 -- 172.21.15.32:0/3100068074 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0a440c6b00 con 0x7f0a44157480 2023-12-20T06:04:00.234 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.234+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3100068074 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"} v 0) v1 -- 0x7f0a440c6b00 con 0x7f0a44157480 2023-12-20T06:04:00.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.234+0000 7f0a3affd640 1 -- 172.21.15.32:0/3100068074 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "backup_fs_data", "var": "pg_num"}]=0 v239) v1 ==== 103+0+11 (secure 0 0 0) 0x7f0a440c6b00 con 0x7f0a44157480 2023-12-20T06:04:00.235 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:04:00.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3100068074 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a28065ca0 msgr2=0x7f0a28068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a28065ca0 0x7f0a28068160 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f0a400b1000 tx=0x7f0a400a9070 comp rx=0 tx=0).stop 2023-12-20T06:04:00.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3100068074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a44157480 msgr2=0x7f0a440c38c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a44157480 0x7f0a440c38c0 secure :-1 s=READY pgs=2407 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c00efb0 tx=0x7f0a3c00c490 comp rx=0 tx=0).stop 2023-12-20T06:04:00.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3100068074 shutdown_connections 2023-12-20T06:04:00.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a28065ca0 0x7f0a28068160 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a44157480 0x7f0a440c38c0 unknown :-1 s=CLOSED pgs=2407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a440cd4c0 0x7f0a441f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 --2- 172.21.15.32:0/3100068074 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a440b0760 0x7f0a44163590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3100068074 >> 172.21.15.32:0/3100068074 conn(0x7f0a4406a510 msgr2=0x7f0a44070ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:00.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3100068074 shutdown_connections 2023-12-20T06:04:00.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.238+0000 7f0a4c6b6640 1 -- 172.21.15.32:0/3100068074 wait complete. 2023-12-20T06:04:00.256 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:04:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.398+0000 7f14be196640 1 Processor -- start 2023-12-20T06:04:00.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.398+0000 7f14be196640 1 -- start start 2023-12-20T06:04:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14be196640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f14b80aece0 0x7f14b80b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14be196640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14b80b0b30 0x7f14b80701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14be196640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f14b80c98c0 con 0x7f14b81599e0 2023-12-20T06:04:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14be196640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f14b80c9ba0 con 0x7f14b80aece0 2023-12-20T06:04:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14be196640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b80c9e80 con 0x7f14b80b0b30 2023-12-20T06:04:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14b77fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f14b81599e0 0x7f14b80c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36180/0 (socket says 172.21.15.32:36180) 2023-12-20T06:04:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14b77fe640 1 -- 172.21.15.32:0/3739677844 learned_addr learned my addr 172.21.15.32:0/3739677844 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14b6ffd640 1 --2- 172.21.15.32:0/3739677844 >> v2:172.21.15.32:3300/0 conn(0x7f14b80aece0 0x7f14b80b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14b7fff640 1 --2- 172.21.15.32:0/3739677844 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14b80b0b30 0x7f14b80701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14b67fc640 1 -- 172.21.15.32:0/3739677844 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1530859310 0 0) 0x7f14b80c98c0 con 0x7f14b81599e0 2023-12-20T06:04:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14b67fc640 1 -- 172.21.15.32:0/3739677844 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f14a0003530 con 0x7f14b81599e0 2023-12-20T06:04:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14b6ffd640 1 -- 172.21.15.32:0/3739677844 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14b80b0b30 msgr2=0x7f14b80701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:00.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14b6ffd640 1 --2- 172.21.15.32:0/3739677844 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14b80b0b30 0x7f14b80701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14b6ffd640 1 -- 172.21.15.32:0/3739677844 >> v1:172.21.15.32:6789/0 conn(0x7f14b81599e0 legacy=0x7f14b80c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.404 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.402+0000 7f14b6ffd640 1 -- 172.21.15.32:0/3739677844 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14b80ca160 con 0x7f14b80aece0 2023-12-20T06:04:00.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14b6ffd640 1 --2- 172.21.15.32:0/3739677844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80aece0 0x7f14b80b05f0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0097c0 tx=0x7f14ac01c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a15c0aa13e0e283e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:00.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14b67fc640 1 -- 172.21.15.32:0/3739677844 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0ad640 con 0x7f14b80aece0 2023-12-20T06:04:00.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14b67fc640 1 -- 172.21.15.32:0/3739677844 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14ac0ad7a0 con 0x7f14b80aece0 2023-12-20T06:04:00.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14b67fc640 1 -- 172.21.15.32:0/3739677844 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14ac0adb30 con 0x7f14b80aece0 2023-12-20T06:04:00.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14be196640 1 -- 172.21.15.32:0/3739677844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80aece0 msgr2=0x7f14b80b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14be196640 1 --2- 172.21.15.32:0/3739677844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80aece0 0x7f14b80b05f0 secure :-1 s=READY pgs=2409 cs=0 l=1 rev1=1 crypto rx=0x7f14ac0097c0 tx=0x7f14ac01c480 comp rx=0 tx=0).stop 2023-12-20T06:04:00.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14be196640 1 -- 172.21.15.32:0/3739677844 shutdown_connections 2023-12-20T06:04:00.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14be196640 1 --2- 172.21.15.32:0/3739677844 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b80aece0 0x7f14b80b05f0 unknown :-1 s=CLOSED pgs=2409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14be196640 1 --2- 172.21.15.32:0/3739677844 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14b80b0b30 0x7f14b80701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14be196640 1 -- 172.21.15.32:0/3739677844 >> 172.21.15.32:0/3739677844 conn(0x7f14b806a320 msgr2=0x7f14b80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:00.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14be196640 1 -- 172.21.15.32:0/3739677844 shutdown_connections 2023-12-20T06:04:00.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14be196640 1 -- 172.21.15.32:0/3739677844 wait complete. 2023-12-20T06:04:00.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14be196640 1 Processor -- start 2023-12-20T06:04:00.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.406+0000 7f14be196640 1 -- start start 2023-12-20T06:04:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14be196640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14b80aece0 0x7f14b80b17f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14be196640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14b80b0b30 0x7f14b81545b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14be196640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b81599e0 0x7f14b815a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14be196640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b8142c80 con 0x7f14b81599e0 2023-12-20T06:04:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14be196640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f14b815cf40 con 0x7f14b80b0b30 2023-12-20T06:04:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14be196640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f14b815c370 con 0x7f14b80aece0 2023-12-20T06:04:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b81599e0 0x7f14b815a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b81599e0 0x7f14b815a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58258/0 (socket says 172.21.15.32:58258) 2023-12-20T06:04:00.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b7fff640 1 -- 172.21.15.32:0/1905177438 learned_addr learned my addr 172.21.15.32:0/1905177438 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b6ffd640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14b80b0b30 0x7f14b81545b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b77fe640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14b80aece0 0x7f14b80b17f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b7fff640 1 -- 172.21.15.32:0/1905177438 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14b80aece0 msgr2=0x7f14b80b17f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b7fff640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14b80aece0 0x7f14b80b17f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b7fff640 1 -- 172.21.15.32:0/1905177438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14b80b0b30 msgr2=0x7f14b81545b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b7fff640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14b80b0b30 0x7f14b81545b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b7fff640 1 -- 172.21.15.32:0/1905177438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f14ac009420 con 0x7f14b81599e0 2023-12-20T06:04:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b6ffd640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14b80b0b30 0x7f14b81545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14b7fff640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b81599e0 0x7f14b815a330 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f14b000efc0 tx=0x7f14b000c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f149ffff640 1 -- 172.21.15.32:0/1905177438 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0014480 con 0x7f14b81599e0 2023-12-20T06:04:00.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f149ffff640 1 -- 172.21.15.32:0/1905177438 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f14b000f040 con 0x7f14b81599e0 2023-12-20T06:04:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f149ffff640 1 -- 172.21.15.32:0/1905177438 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f14b0014820 con 0x7f14b81599e0 2023-12-20T06:04:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14be196640 1 -- 172.21.15.32:0/1905177438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f14b815c610 con 0x7f14b81599e0 2023-12-20T06:04:00.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14be196640 1 -- 172.21.15.32:0/1905177438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f14b815cbd0 con 0x7f14b81599e0 2023-12-20T06:04:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f14be196640 1 -- 172.21.15.32:0/1905177438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f14b815d0d0 con 0x7f14b81599e0 2023-12-20T06:04:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.410+0000 7f149ffff640 1 -- 172.21.15.32:0/1905177438 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f14b0014980 con 0x7f14b81599e0 2023-12-20T06:04:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.414+0000 7f149ffff640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f14a4065ca0 0x7f14a4068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.414+0000 7f149ffff640 1 -- 172.21.15.32:0/1905177438 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f14b00845e0 con 0x7f14b81599e0 2023-12-20T06:04:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.414+0000 7f14b77fe640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f14a4065ca0 0x7f14a4068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.414+0000 7f14b77fe640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f14a4065ca0 0x7f14a4068160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f14a8007fd0 tx=0x7f14a8009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:00.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.418+0000 7f149ffff640 1 -- 172.21.15.32:0/1905177438 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f14b0051790 con 0x7f14b81599e0 2023-12-20T06:04:00.608 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.606+0000 7f14be196640 1 -- 172.21.15.32:0/1905177438 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f14b804e000 con 0x7f14b81599e0 2023-12-20T06:04:00.609 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f149ffff640 1 -- 172.21.15.32:0/1905177438 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4714 (secure 0 0 0) 0x7f14b0054db0 con 0x7f14b81599e0 2023-12-20T06:04:00.609 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:00.609 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":9731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/99271174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":99271174},{"type":"v1","addr":"172.21.15.32:6837","nonce":99271174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:34.513908+0000","modified":"2023-12-20T06:03:37.549965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11298":{"gid":11298,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6836/60134876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":60134876},{"type":"v1","addr":"172.21.15.32:6836","nonce":60134876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-20T06:04:00.610 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 120 2023-12-20T06:04:00.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 -- 172.21.15.32:0/1905177438 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f14a4065ca0 msgr2=0x7f14a4068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f14a4065ca0 0x7f14a4068160 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f14a8007fd0 tx=0x7f14a8009040 comp rx=0 tx=0).stop 2023-12-20T06:04:00.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 -- 172.21.15.32:0/1905177438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b81599e0 msgr2=0x7f14b815a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b81599e0 0x7f14b815a330 secure :-1 s=READY pgs=2410 cs=0 l=1 rev1=1 crypto rx=0x7f14b000efc0 tx=0x7f14b000c490 comp rx=0 tx=0).stop 2023-12-20T06:04:00.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 -- 172.21.15.32:0/1905177438 shutdown_connections 2023-12-20T06:04:00.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f14a4065ca0 0x7f14a4068160 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f14b81599e0 0x7f14b815a330 unknown :-1 s=CLOSED pgs=2410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f14b80b0b30 0x7f14b81545b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 --2- 172.21.15.32:0/1905177438 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f14b80aece0 0x7f14b80b17f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 -- 172.21.15.32:0/1905177438 >> 172.21.15.32:0/1905177438 conn(0x7f14b806a320 msgr2=0x7f14b80cdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:00.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 -- 172.21.15.32:0/1905177438 shutdown_connections 2023-12-20T06:04:00.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.610+0000 7f14be196640 1 -- 172.21.15.32:0/1905177438 wait complete. 2023-12-20T06:04:00.630 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:04:00.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.774+0000 7fb70f3db640 1 Processor -- start 2023-12-20T06:04:00.775 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.774+0000 7fb70f3db640 1 -- start start 2023-12-20T06:04:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.774+0000 7fb70f3db640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb70806cf10 0x7fb7080c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.774+0000 7fb70f3db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb708164ef0 0x7fb7080c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.774+0000 7fb70f3db640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb7080ae3f0 con 0x7fb7080ae010 2023-12-20T06:04:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.774+0000 7fb70f3db640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb708161800 con 0x7fb70806cf10 2023-12-20T06:04:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.774+0000 7fb70f3db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb708165380 con 0x7fb708164ef0 2023-12-20T06:04:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.774+0000 7fb70c976640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb70806cf10 0x7fb7080c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.774+0000 7fb70c976640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fb70806cf10 0x7fb7080c94a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58272/0 (socket says 172.21.15.32:58272) 2023-12-20T06:04:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.774+0000 7fb70c976640 1 -- 172.21.15.32:0/2262318666 learned_addr learned my addr 172.21.15.32:0/2262318666 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.774+0000 7fb70d978640 1 --2- 172.21.15.32:0/2262318666 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb708164ef0 0x7fb7080c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.778+0000 7fb70c976640 1 -- 172.21.15.32:0/2262318666 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb708164ef0 msgr2=0x7fb7080c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.778+0000 7fb70c976640 1 --2- 172.21.15.32:0/2262318666 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb708164ef0 0x7fb7080c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.778+0000 7fb70c976640 1 -- 172.21.15.32:0/2262318666 >> v1:172.21.15.39:6789/0 conn(0x7fb7080ae010 legacy=0x7fb708071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.778+0000 7fb70c976640 1 -- 172.21.15.32:0/2262318666 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb708165660 con 0x7fb70806cf10 2023-12-20T06:04:00.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.778+0000 7fb6fffff640 1 -- 172.21.15.32:0/2262318666 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1649466486 0 0) 0x7fb7080ae3f0 con 0x7fb7080ae010 2023-12-20T06:04:00.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.778+0000 7fb70c976640 1 --2- 172.21.15.32:0/2262318666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70806cf10 0x7fb7080c94a0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009870 tx=0x7fb6f801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a041218126fb63dc server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:00.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.778+0000 7fb6fffff640 1 -- 172.21.15.32:0/2262318666 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f80ab680 con 0x7fb70806cf10 2023-12-20T06:04:00.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.778+0000 7fb6fffff640 1 -- 172.21.15.32:0/2262318666 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f80ab7e0 con 0x7fb70806cf10 2023-12-20T06:04:00.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.778+0000 7fb6fffff640 1 -- 172.21.15.32:0/2262318666 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f80abb70 con 0x7fb70806cf10 2023-12-20T06:04:00.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.778+0000 7fb70f3db640 1 -- 172.21.15.32:0/2262318666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70806cf10 msgr2=0x7fb7080c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.778+0000 7fb70f3db640 1 --2- 172.21.15.32:0/2262318666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70806cf10 0x7fb7080c94a0 secure :-1 s=READY pgs=2411 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009870 tx=0x7fb6f801c480 comp rx=0 tx=0).stop 2023-12-20T06:04:00.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 -- 172.21.15.32:0/2262318666 shutdown_connections 2023-12-20T06:04:00.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 --2- 172.21.15.32:0/2262318666 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb708164ef0 0x7fb7080c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 --2- 172.21.15.32:0/2262318666 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70806cf10 0x7fb7080c94a0 unknown :-1 s=CLOSED pgs=2411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 -- 172.21.15.32:0/2262318666 >> 172.21.15.32:0/2262318666 conn(0x7fb70806a320 msgr2=0x7fb7080ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:00.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 -- 172.21.15.32:0/2262318666 shutdown_connections 2023-12-20T06:04:00.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 -- 172.21.15.32:0/2262318666 wait complete. 2023-12-20T06:04:00.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 Processor -- start 2023-12-20T06:04:00.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 -- start start 2023-12-20T06:04:00.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7080ae010 0x7fb70815bd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70814ca40 0x7fb708158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7080cd2e0 0x7fb7080cd790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb7080cdcd0 con 0x7fb70814ca40 2023-12-20T06:04:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb70815c370 con 0x7fb7080ae010 2023-12-20T06:04:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70f3db640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb708160160 con 0x7fb7080cd2e0 2023-12-20T06:04:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70c976640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70814ca40 0x7fb708158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70c976640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70814ca40 0x7fb708158e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58280/0 (socket says 172.21.15.32:58280) 2023-12-20T06:04:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70c976640 1 -- 172.21.15.32:0/2189482245 learned_addr learned my addr 172.21.15.32:0/2189482245 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70d177640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7080ae010 0x7fb70815bd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70d978640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7080cd2e0 0x7fb7080cd790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70c976640 1 -- 172.21.15.32:0/2189482245 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7080cd2e0 msgr2=0x7fb7080cd790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70c976640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7080cd2e0 0x7fb7080cd790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70c976640 1 -- 172.21.15.32:0/2189482245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7080ae010 msgr2=0x7fb70815bd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70c976640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7080ae010 0x7fb70815bd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70c976640 1 -- 172.21.15.32:0/2189482245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f80094d0 con 0x7fb70814ca40 2023-12-20T06:04:00.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.782+0000 7fb70c976640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70814ca40 0x7fb708158e00 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009870 tx=0x7fb6f801acc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:00.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.786+0000 7fb6fdffb640 1 -- 172.21.15.32:0/2189482245 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f80abd60 con 0x7fb70814ca40 2023-12-20T06:04:00.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.786+0000 7fb6fdffb640 1 -- 172.21.15.32:0/2189482245 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6f80b8070 con 0x7fb70814ca40 2023-12-20T06:04:00.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.786+0000 7fb6fdffb640 1 -- 172.21.15.32:0/2189482245 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6f8004230 con 0x7fb70814ca40 2023-12-20T06:04:00.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.786+0000 7fb70f3db640 1 -- 172.21.15.32:0/2189482245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb708160400 con 0x7fb70814ca40 2023-12-20T06:04:00.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.786+0000 7fb70f3db640 1 -- 172.21.15.32:0/2189482245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb7081609c0 con 0x7fb70814ca40 2023-12-20T06:04:00.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.786+0000 7fb70f3db640 1 -- 172.21.15.32:0/2189482245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb708160bb0 con 0x7fb70814ca40 2023-12-20T06:04:00.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.786+0000 7fb6fdffb640 1 -- 172.21.15.32:0/2189482245 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb6f8004390 con 0x7fb70814ca40 2023-12-20T06:04:00.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.786+0000 7fb6fdffb640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6ec065ca0 0x7fb6ec068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.786+0000 7fb6fdffb640 1 -- 172.21.15.32:0/2189482245 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fb6f811e160 con 0x7fb70814ca40 2023-12-20T06:04:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.786+0000 7fb70d177640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6ec065ca0 0x7fb6ec068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:00.789 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.790+0000 7fb70d177640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6ec065ca0 0x7fb6ec068160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb70814e0d0 tx=0x7fb704009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:00.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.790+0000 7fb6fdffb640 1 -- 172.21.15.32:0/2189482245 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb6f80eb380 con 0x7fb70814ca40 2023-12-20T06:04:00.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.950+0000 7fb70f3db640 1 -- 172.21.15.32:0/2189482245 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fb70804e000 con 0x7fb70814ca40 2023-12-20T06:04:00.953 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb6fdffb640 1 -- 172.21.15.32:0/2189482245 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+17341 (secure 0 0 0) 0x7fb6f80ee9a0 con 0x7fb70814ca40 2023-12-20T06:04:00.953 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:00.953 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":239,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:58.834991+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:03:32.174574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:03:33.509224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/1623322556":"2023-12-21T06:03:55.505418+0000","172.21.15.32:0/1232037079":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/2587801540":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/789250710":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/1603186500":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/2783112768":"2023-12-21T06:03:51.776767+0000","172.21.15.32:6837/1623322556":"2023-12-21T06:03:55.505418+0000","172.21.15.32:0/3054548475":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2026838481":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/597963739":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/274812680":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/3886671554":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1847933543":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/4013366117":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/582500261":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/805068630":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6833/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1823890749":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6832/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2852735264":"2023-12-21T06:03:45.663020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:04:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 -- 172.21.15.32:0/2189482245 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6ec065ca0 msgr2=0x7fb6ec068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6ec065ca0 0x7fb6ec068160 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fb70814e0d0 tx=0x7fb704009040 comp rx=0 tx=0).stop 2023-12-20T06:04:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 -- 172.21.15.32:0/2189482245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70814ca40 msgr2=0x7fb708158e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70814ca40 0x7fb708158e00 secure :-1 s=READY pgs=2412 cs=0 l=1 rev1=1 crypto rx=0x7fb6f8009870 tx=0x7fb6f801acc0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 -- 172.21.15.32:0/2189482245 shutdown_connections 2023-12-20T06:04:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6ec065ca0 0x7fb6ec068160 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb7080cd2e0 0x7fb7080cd790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb70814ca40 0x7fb708158e00 unknown :-1 s=CLOSED pgs=2412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 --2- 172.21.15.32:0/2189482245 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb7080ae010 0x7fb70815bd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 -- 172.21.15.32:0/2189482245 >> 172.21.15.32:0/2189482245 conn(0x7fb70806a320 msgr2=0x7fb7080cc6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:00.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 -- 172.21.15.32:0/2189482245 shutdown_connections 2023-12-20T06:04:00.957 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:00.954+0000 7fb70f3db640 1 -- 172.21.15.32:0/2189482245 wait complete. 2023-12-20T06:04:00.975 INFO:tasks.cephfs.filesystem:Destroying file system backup_fs and related pools 2023-12-20T06:04:00.975 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:04:01.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.118+0000 7fcb4ed19640 1 Processor -- start 2023-12-20T06:04:01.120 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.118+0000 7fcb4ed19640 1 -- start start 2023-12-20T06:04:01.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4ed19640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcb480cd650 0x7fcb4815f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4ed19640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb480cb0e0 con 0x7fcb48158d40 2023-12-20T06:04:01.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4ed19640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcb4815f940 con 0x7fcb48158960 2023-12-20T06:04:01.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4ed19640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcb480cba60 con 0x7fcb480cd650 2023-12-20T06:04:01.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4d516640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcb48158d40 0x7fcb4815ebd0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36196/0 (socket says 172.21.15.32:36196) 2023-12-20T06:04:01.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4d516640 1 -- 172.21.15.32:0/1509956599 learned_addr learned my addr 172.21.15.32:0/1509956599 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:01.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4e518640 1 --2- 172.21.15.32:0/1509956599 >> v2:172.21.15.39:3300/0 conn(0x7fcb480cd650 0x7fcb4815f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4cd15640 1 -- 172.21.15.32:0/1509956599 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 44257448 0 0) 0x7fcb480cb0e0 con 0x7fcb48158d40 2023-12-20T06:04:01.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4cd15640 1 -- 172.21.15.32:0/1509956599 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb34003530 con 0x7fcb48158d40 2023-12-20T06:04:01.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4cd15640 1 -- 172.21.15.32:0/1509956599 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1459580888 0 0) 0x7fcb4815f940 con 0x7fcb48158960 2023-12-20T06:04:01.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4cd15640 1 -- 172.21.15.32:0/1509956599 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fcb480cb0e0 con 0x7fcb48158960 2023-12-20T06:04:01.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4e518640 1 -- 172.21.15.32:0/1509956599 >> v1:172.21.15.39:6789/0 conn(0x7fcb48158960 legacy=0x7fcb480ccf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4e518640 1 -- 172.21.15.32:0/1509956599 >> v1:172.21.15.32:6789/0 conn(0x7fcb48158d40 legacy=0x7fcb4815ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.124 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.122+0000 7fcb4e518640 1 -- 172.21.15.32:0/1509956599 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb480cb3a0 con 0x7fcb480cd650 2023-12-20T06:04:01.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4e518640 1 --2- 172.21.15.32:0/1509956599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb480cd650 0x7fcb4815f2e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009d90 tx=0x7fcb4001c920 comp rx=0 tx=0).ready entity=mon.1 client_cookie=979404e00f4a6fc6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:01.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4cd15640 1 -- 172.21.15.32:0/1509956599 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400a58f0 con 0x7fcb480cd650 2023-12-20T06:04:01.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4cd15640 1 -- 172.21.15.32:0/1509956599 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb400a5a50 con 0x7fcb480cd650 2023-12-20T06:04:01.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4cd15640 1 -- 172.21.15.32:0/1509956599 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb400a5de0 con 0x7fcb480cd650 2023-12-20T06:04:01.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4ed19640 1 -- 172.21.15.32:0/1509956599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb480cd650 msgr2=0x7fcb4815f2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4ed19640 1 --2- 172.21.15.32:0/1509956599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb480cd650 0x7fcb4815f2e0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7fcb40009d90 tx=0x7fcb4001c920 comp rx=0 tx=0).stop 2023-12-20T06:04:01.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4ed19640 1 -- 172.21.15.32:0/1509956599 shutdown_connections 2023-12-20T06:04:01.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4ed19640 1 --2- 172.21.15.32:0/1509956599 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb480cd650 0x7fcb4815f2e0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4ed19640 1 -- 172.21.15.32:0/1509956599 >> 172.21.15.32:0/1509956599 conn(0x7fcb4806e280 msgr2=0x7fcb4806e690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:01.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4ed19640 1 -- 172.21.15.32:0/1509956599 shutdown_connections 2023-12-20T06:04:01.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4ed19640 1 -- 172.21.15.32:0/1509956599 wait complete. 2023-12-20T06:04:01.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4ed19640 1 Processor -- start 2023-12-20T06:04:01.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.126+0000 7fcb4ed19640 1 -- start start 2023-12-20T06:04:01.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4ed19640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb480cd650 0x7fcb48164bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4ed19640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb48158960 0x7fcb480b5560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4ed19640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcb48158d40 0x7fcb4806c400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4ed19640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4806bf70 con 0x7fcb48158960 2023-12-20T06:04:01.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4ed19640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4806b360 con 0x7fcb480cd650 2023-12-20T06:04:01.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4ed19640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcb4806b640 con 0x7fcb48158d40 2023-12-20T06:04:01.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4d516640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb48158960 0x7fcb480b5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4d516640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb48158960 0x7fcb480b5560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58294/0 (socket says 172.21.15.32:58294) 2023-12-20T06:04:01.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4d516640 1 -- 172.21.15.32:0/2001150613 learned_addr learned my addr 172.21.15.32:0/2001150613 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:01.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4dd17640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb480cd650 0x7fcb48164bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4e518640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcb48158d40 0x7fcb4806c400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4d516640 1 -- 172.21.15.32:0/2001150613 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcb48158d40 msgr2=0x7fcb4806c400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4d516640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcb48158d40 0x7fcb4806c400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4d516640 1 -- 172.21.15.32:0/2001150613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb480cd650 msgr2=0x7fcb48164bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4d516640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb480cd650 0x7fcb48164bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4d516640 1 -- 172.21.15.32:0/2001150613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcb40009a70 con 0x7fcb48158960 2023-12-20T06:04:01.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4d516640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb48158960 0x7fcb480b5560 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800c420 tx=0x7fcb3800c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:01.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb3e7fc640 1 -- 172.21.15.32:0/2001150613 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38005cf0 con 0x7fcb48158960 2023-12-20T06:04:01.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb3e7fc640 1 -- 172.21.15.32:0/2001150613 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcb38005e50 con 0x7fcb48158960 2023-12-20T06:04:01.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb3e7fc640 1 -- 172.21.15.32:0/2001150613 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcb38007310 con 0x7fcb48158960 2023-12-20T06:04:01.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4ed19640 1 -- 172.21.15.32:0/2001150613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcb4806b920 con 0x7fcb48158960 2023-12-20T06:04:01.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4ed19640 1 -- 172.21.15.32:0/2001150613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcb48165130 con 0x7fcb48158960 2023-12-20T06:04:01.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.130+0000 7fcb4ed19640 1 -- 172.21.15.32:0/2001150613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcb4806c0e0 con 0x7fcb48158960 2023-12-20T06:04:01.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.134+0000 7fcb3e7fc640 1 -- 172.21.15.32:0/2001150613 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fcb38007470 con 0x7fcb48158960 2023-12-20T06:04:01.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.134+0000 7fcb3e7fc640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcb2c065ca0 0x7fcb2c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.134+0000 7fcb3e7fc640 1 -- 172.21.15.32:0/2001150613 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fcb38082ff0 con 0x7fcb48158960 2023-12-20T06:04:01.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.134+0000 7fcb4dd17640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcb2c065ca0 0x7fcb2c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.134+0000 7fcb4dd17640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcb2c065ca0 0x7fcb2c068160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcb440011f0 tx=0x7fcb44009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:01.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.138+0000 7fcb3e7fc640 1 -- 172.21.15.32:0/2001150613 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcb38050230 con 0x7fcb48158960 2023-12-20T06:04:01.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.326+0000 7fcb4ed19640 1 -- 172.21.15.32:0/2001150613 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcb48074100 con 0x7fcb48158960 2023-12-20T06:04:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.326+0000 7fcb3e7fc640 1 -- 172.21.15.32:0/2001150613 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4714 (secure 0 0 0) 0x7fcb38017020 con 0x7fcb48158960 2023-12-20T06:04:01.328 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:01.328 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":9731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/99271174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":99271174},{"type":"v1","addr":"172.21.15.32:6837","nonce":99271174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:34.513908+0000","modified":"2023-12-20T06:03:37.549965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11298":{"gid":11298,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6836/60134876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":60134876},{"type":"v1","addr":"172.21.15.32:6836","nonce":60134876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-20T06:04:01.328 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 120 2023-12-20T06:04:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 -- 172.21.15.32:0/2001150613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcb2c065ca0 msgr2=0x7fcb2c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcb2c065ca0 0x7fcb2c068160 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcb440011f0 tx=0x7fcb44009040 comp rx=0 tx=0).stop 2023-12-20T06:04:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 -- 172.21.15.32:0/2001150613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb48158960 msgr2=0x7fcb480b5560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb48158960 0x7fcb480b5560 secure :-1 s=READY pgs=2414 cs=0 l=1 rev1=1 crypto rx=0x7fcb3800c420 tx=0x7fcb3800c8e0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 -- 172.21.15.32:0/2001150613 shutdown_connections 2023-12-20T06:04:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcb2c065ca0 0x7fcb2c068160 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcb48158d40 0x7fcb4806c400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcb48158960 0x7fcb480b5560 unknown :-1 s=CLOSED pgs=2414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 --2- 172.21.15.32:0/2001150613 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcb480cd650 0x7fcb48164bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 -- 172.21.15.32:0/2001150613 >> 172.21.15.32:0/2001150613 conn(0x7fcb4806e280 msgr2=0x7fcb48159120 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 -- 172.21.15.32:0/2001150613 shutdown_connections 2023-12-20T06:04:01.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.330+0000 7fcb4ed19640 1 -- 172.21.15.32:0/2001150613 wait complete. 2023-12-20T06:04:01.349 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:04:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf15604640 1 Processor -- start 2023-12-20T06:04:01.493 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf15604640 1 -- start start 2023-12-20T06:04:01.494 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf15604640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbf100b2310 0x7fbf100b27a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf15604640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 0x7fbf100cb7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf15604640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fbf100caf50 con 0x7fbf101600a0 2023-12-20T06:04:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf15604640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fbf100b2ce0 con 0x7fbf100b2310 2023-12-20T06:04:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf15604640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf100ce840 con 0x7fbf100cb3c0 2023-12-20T06:04:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf0f7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 0x7fbf100cb7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf0e7fc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fbf100b2310 0x7fbf100b27a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf0f7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 0x7fbf100cb7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:34424/0 (socket says 172.21.15.32:34424) 2023-12-20T06:04:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf0f7fe640 1 -- 172.21.15.32:0/2180893534 learned_addr learned my addr 172.21.15.32:0/2180893534 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:01.495 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf0effd640 1 --1- 172.21.15.32:0/2180893534 >> v1:172.21.15.39:6789/0 conn(0x7fbf101600a0 0x7fbf10161170 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:42310/0 (socket says 172.21.15.32:42310) 2023-12-20T06:04:01.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf0e7fc640 1 -- 172.21.15.32:0/2180893534 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 msgr2=0x7fbf100cb7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:01.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf0e7fc640 1 --2- 172.21.15.32:0/2180893534 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 0x7fbf100cb7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf0e7fc640 1 -- 172.21.15.32:0/2180893534 >> v1:172.21.15.39:6789/0 conn(0x7fbf101600a0 legacy=0x7fbf10161170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf0dffb640 1 -- 172.21.15.32:0/2180893534 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1228147352 0 0) 0x7fbf100caf50 con 0x7fbf101600a0 2023-12-20T06:04:01.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf0e7fc640 1 -- 172.21.15.32:0/2180893534 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf100ceb20 con 0x7fbf100b2310 2023-12-20T06:04:01.496 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.494+0000 7fbf0f7fe640 1 --2- 172.21.15.32:0/2180893534 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 0x7fbf100cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:01.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf0e7fc640 1 --2- 172.21.15.32:0/2180893534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf100b2310 0x7fbf100b27a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbf040097c0 tx=0x7fbf0401ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8bf2ab13a1b2c93b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:01.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf0dffb640 1 -- 172.21.15.32:0/2180893534 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf040a8970 con 0x7fbf100b2310 2023-12-20T06:04:01.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf0dffb640 1 -- 172.21.15.32:0/2180893534 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf040a8ad0 con 0x7fbf100b2310 2023-12-20T06:04:01.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf0dffb640 1 -- 172.21.15.32:0/2180893534 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf040a8e60 con 0x7fbf100b2310 2023-12-20T06:04:01.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf15604640 1 -- 172.21.15.32:0/2180893534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf100b2310 msgr2=0x7fbf100b27a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf15604640 1 --2- 172.21.15.32:0/2180893534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf100b2310 0x7fbf100b27a0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7fbf040097c0 tx=0x7fbf0401ec70 comp rx=0 tx=0).stop 2023-12-20T06:04:01.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf15604640 1 -- 172.21.15.32:0/2180893534 shutdown_connections 2023-12-20T06:04:01.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf15604640 1 --2- 172.21.15.32:0/2180893534 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 0x7fbf100cb7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf15604640 1 --2- 172.21.15.32:0/2180893534 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf100b2310 0x7fbf100b27a0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf15604640 1 -- 172.21.15.32:0/2180893534 >> 172.21.15.32:0/2180893534 conn(0x7fbf100cd310 msgr2=0x7fbf100cd720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:01.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf15604640 1 -- 172.21.15.32:0/2180893534 shutdown_connections 2023-12-20T06:04:01.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf15604640 1 -- 172.21.15.32:0/2180893534 wait complete. 2023-12-20T06:04:01.500 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.498+0000 7fbf15604640 1 Processor -- start 2023-12-20T06:04:01.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf15604640 1 -- start start 2023-12-20T06:04:01.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf15604640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100b2310 0x7fbf100c41c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf15604640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 0x7fbf1015df10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf15604640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf101600a0 0x7fbf1015e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf15604640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf10147770 con 0x7fbf100b2310 2023-12-20T06:04:01.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf15604640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fbf10147a20 con 0x7fbf101600a0 2023-12-20T06:04:01.503 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf15604640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fbf10147c70 con 0x7fbf100cb3c0 2023-12-20T06:04:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100b2310 0x7fbf100c41c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100b2310 0x7fbf100c41c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58308/0 (socket says 172.21.15.32:58308) 2023-12-20T06:04:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0effd640 1 -- 172.21.15.32:0/4026706716 learned_addr learned my addr 172.21.15.32:0/4026706716 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0e7fc640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 0x7fbf1015df10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0f7fe640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf101600a0 0x7fbf1015e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0effd640 1 -- 172.21.15.32:0/4026706716 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 msgr2=0x7fbf1015df10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0effd640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 0x7fbf1015df10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0effd640 1 -- 172.21.15.32:0/4026706716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf101600a0 msgr2=0x7fbf1015e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0effd640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf101600a0 0x7fbf1015e450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0effd640 1 -- 172.21.15.32:0/4026706716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fbf08009b30 con 0x7fbf100b2310 2023-12-20T06:04:01.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0f7fe640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf101600a0 0x7fbf1015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:01.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf0effd640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100b2310 0x7fbf100c41c0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000c4b0 tx=0x7fbf0000c970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:01.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbef77fe640 1 -- 172.21.15.32:0/4026706716 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf000074e0 con 0x7fbf100b2310 2023-12-20T06:04:01.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbef77fe640 1 -- 172.21.15.32:0/4026706716 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fbf00007660 con 0x7fbf100b2310 2023-12-20T06:04:01.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf15604640 1 -- 172.21.15.32:0/4026706716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fbf04009420 con 0x7fbf100b2310 2023-12-20T06:04:01.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbef77fe640 1 -- 172.21.15.32:0/4026706716 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fbf00004a30 con 0x7fbf100b2310 2023-12-20T06:04:01.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf15604640 1 -- 172.21.15.32:0/4026706716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fbf10148230 con 0x7fbf100b2310 2023-12-20T06:04:01.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.502+0000 7fbf15604640 1 -- 172.21.15.32:0/4026706716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fbf10074100 con 0x7fbf100b2310 2023-12-20T06:04:01.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.506+0000 7fbef77fe640 1 -- 172.21.15.32:0/4026706716 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fbf00004b90 con 0x7fbf100b2310 2023-12-20T06:04:01.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.506+0000 7fbef77fe640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbefc065ca0 0x7fbefc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.506+0000 7fbef77fe640 1 -- 172.21.15.32:0/4026706716 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6637+0+0 (secure 0 0 0) 0x7fbf00082eb0 con 0x7fbf100b2310 2023-12-20T06:04:01.506 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.506+0000 7fbf0e7fc640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbefc065ca0 0x7fbefc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.506+0000 7fbf0e7fc640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbefc065ca0 0x7fbefc068160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbf040b2000 tx=0x7fbf04008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:01.509 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.510+0000 7fbef77fe640 1 -- 172.21.15.32:0/4026706716 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fbf00053b60 con 0x7fbf100b2310 2023-12-20T06:04:01.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.698+0000 7fbf15604640 1 -- 172.21.15.32:0/4026706716 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fbf100b5480 con 0x7fbf100b2310 2023-12-20T06:04:01.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.698+0000 7fbef77fe640 1 -- 172.21.15.32:0/4026706716 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 120 v120) v1 ==== 95+0+4714 (secure 0 0 0) 0x7fbf00018020 con 0x7fbf100b2310 2023-12-20T06:04:01.700 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:01.700 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":120,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":9731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/99271174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":99271174},{"type":"v1","addr":"172.21.15.32:6837","nonce":99271174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107}],"filesystems":[{"mdsmap":{"epoch":118,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:03:34.513908+0000","modified":"2023-12-20T06:03:37.549965+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":11298},"failed":[],"damaged":[],"stopped":[],"info":{"gid_11298":{"gid":11298,"name":"c","rank":0,"incarnation":114,"state":"up:active","state_seq":6,"addr":"172.21.15.32:6836/60134876","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":60134876},{"type":"v1","addr":"172.21.15.32:6836","nonce":60134876}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[31],"metadata_pool":30,"enabled":true,"fs_name":"backup_fs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":15}]} 2023-12-20T06:04:01.700 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 120 2023-12-20T06:04:01.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 -- 172.21.15.32:0/4026706716 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbefc065ca0 msgr2=0x7fbefc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbefc065ca0 0x7fbefc068160 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7fbf040b2000 tx=0x7fbf04008040 comp rx=0 tx=0).stop 2023-12-20T06:04:01.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 -- 172.21.15.32:0/4026706716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100b2310 msgr2=0x7fbf100c41c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100b2310 0x7fbf100c41c0 secure :-1 s=READY pgs=2415 cs=0 l=1 rev1=1 crypto rx=0x7fbf0000c4b0 tx=0x7fbf0000c970 comp rx=0 tx=0).stop 2023-12-20T06:04:01.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 -- 172.21.15.32:0/4026706716 shutdown_connections 2023-12-20T06:04:01.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fbefc065ca0 0x7fbefc068160 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fbf101600a0 0x7fbf1015e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fbf100cb3c0 0x7fbf1015df10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 --2- 172.21.15.32:0/4026706716 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fbf100b2310 0x7fbf100c41c0 unknown :-1 s=CLOSED pgs=2415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 -- 172.21.15.32:0/4026706716 >> 172.21.15.32:0/4026706716 conn(0x7fbf100cd310 msgr2=0x7fbf100b34b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:01.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 -- 172.21.15.32:0/4026706716 shutdown_connections 2023-12-20T06:04:01.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.702+0000 7fbf15604640 1 -- 172.21.15.32:0/4026706716 wait complete. 2023-12-20T06:04:01.722 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:04:01.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.866+0000 7f63f267d640 1 Processor -- start 2023-12-20T06:04:01.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.866+0000 7f63f267d640 1 -- start start 2023-12-20T06:04:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f267d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63ec159ab0 0x7f63ec0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f267d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63ec0cd2e0 0x7f63ec071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f267d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f63ec071f40 con 0x7f63ec0b3e90 2023-12-20T06:04:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f267d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f63ec0ca0b0 con 0x7f63ec0cd2e0 2023-12-20T06:04:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f267d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec0ad7e0 con 0x7f63ec159ab0 2023-12-20T06:04:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f0e7a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f63ec0cd2e0 0x7f63ec071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f1e7c640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f63ec0b3e90 0x7f63ec0cc140 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36208/0 (socket says 172.21.15.32:36208) 2023-12-20T06:04:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f1e7c640 1 -- 172.21.15.32:0/3258216348 learned_addr learned my addr 172.21.15.32:0/3258216348 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f167b640 1 --2- 172.21.15.32:0/3258216348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63ec159ab0 0x7f63ec0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f0e7a640 1 -- 172.21.15.32:0/3258216348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63ec159ab0 msgr2=0x7f63ec0714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f0e7a640 1 --2- 172.21.15.32:0/3258216348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63ec159ab0 0x7f63ec0714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f0e7a640 1 -- 172.21.15.32:0/3258216348 >> v1:172.21.15.32:6789/0 conn(0x7f63ec0b3e90 legacy=0x7f63ec0cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63e3fff640 1 -- 172.21.15.32:0/3258216348 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 804095545 0 0) 0x7f63ec071f40 con 0x7f63ec0b3e90 2023-12-20T06:04:01.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f0e7a640 1 -- 172.21.15.32:0/3258216348 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63ec0ada80 con 0x7f63ec0cd2e0 2023-12-20T06:04:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.870+0000 7f63f0e7a640 1 --2- 172.21.15.32:0/3258216348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63ec0cd2e0 0x7f63ec071a00 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0097c0 tx=0x7f63dc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c7db8f9b20a178 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63e3fff640 1 -- 172.21.15.32:0/3258216348 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0ab600 con 0x7f63ec0cd2e0 2023-12-20T06:04:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63e3fff640 1 -- 172.21.15.32:0/3258216348 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63dc0a7050 con 0x7f63ec0cd2e0 2023-12-20T06:04:01.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63e3fff640 1 -- 172.21.15.32:0/3258216348 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63dc0ab980 con 0x7f63ec0cd2e0 2023-12-20T06:04:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 -- 172.21.15.32:0/3258216348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63ec0cd2e0 msgr2=0x7f63ec071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 --2- 172.21.15.32:0/3258216348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63ec0cd2e0 0x7f63ec071a00 secure :-1 s=READY pgs=2417 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0097c0 tx=0x7f63dc01c450 comp rx=0 tx=0).stop 2023-12-20T06:04:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 -- 172.21.15.32:0/3258216348 shutdown_connections 2023-12-20T06:04:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 --2- 172.21.15.32:0/3258216348 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63ec0cd2e0 0x7f63ec071a00 unknown :-1 s=CLOSED pgs=2417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 --2- 172.21.15.32:0/3258216348 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63ec159ab0 0x7f63ec0714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 -- 172.21.15.32:0/3258216348 >> 172.21.15.32:0/3258216348 conn(0x7f63ec06c060 msgr2=0x7f63ec06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:01.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 -- 172.21.15.32:0/3258216348 shutdown_connections 2023-12-20T06:04:01.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 -- 172.21.15.32:0/3258216348 wait complete. 2023-12-20T06:04:01.875 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 Processor -- start 2023-12-20T06:04:01.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 -- start start 2023-12-20T06:04:01.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63ec0b3e90 0x7f63ec151b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.874+0000 7f63f267d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63ec0cd2e0 0x7f63ec0c3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f267d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ec159ab0 0x7f63ec0c4220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f267d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec0ac680 con 0x7f63ec0b3e90 2023-12-20T06:04:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f267d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec155cf0 con 0x7f63ec159ab0 2023-12-20T06:04:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f267d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f63ec155fd0 con 0x7f63ec0cd2e0 2023-12-20T06:04:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f167b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63ec0b3e90 0x7f63ec151b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f167b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63ec0b3e90 0x7f63ec151b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58320/0 (socket says 172.21.15.32:58320) 2023-12-20T06:04:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f167b640 1 -- 172.21.15.32:0/2887078395 learned_addr learned my addr 172.21.15.32:0/2887078395 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f0e7a640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63ec0cd2e0 0x7f63ec0c3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f1e7c640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ec159ab0 0x7f63ec0c4220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f167b640 1 -- 172.21.15.32:0/2887078395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63ec0cd2e0 msgr2=0x7f63ec0c3ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f167b640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63ec0cd2e0 0x7f63ec0c3ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f167b640 1 -- 172.21.15.32:0/2887078395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ec159ab0 msgr2=0x7f63ec0c4220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f167b640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ec159ab0 0x7f63ec0c4220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f167b640 1 -- 172.21.15.32:0/2887078395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63dc009420 con 0x7f63ec0b3e90 2023-12-20T06:04:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f167b640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63ec0b3e90 0x7f63ec151b30 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f63e800d5f0 tx=0x7f63e800dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63e1ffb640 1 -- 172.21.15.32:0/2887078395 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e80125f0 con 0x7f63ec0b3e90 2023-12-20T06:04:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63e1ffb640 1 -- 172.21.15.32:0/2887078395 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63e8012750 con 0x7f63ec0b3e90 2023-12-20T06:04:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63e1ffb640 1 -- 172.21.15.32:0/2887078395 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63e8012ae0 con 0x7f63ec0b3e90 2023-12-20T06:04:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f267d640 1 -- 172.21.15.32:0/2887078395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f63ec1562b0 con 0x7f63ec0b3e90 2023-12-20T06:04:01.880 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f267d640 1 -- 172.21.15.32:0/2887078395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f63ec156ab0 con 0x7f63ec0b3e90 2023-12-20T06:04:01.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63f267d640 1 -- 172.21.15.32:0/2887078395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f63ec074100 con 0x7f63ec0b3e90 2023-12-20T06:04:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.878+0000 7f63e1ffb640 1 -- 172.21.15.32:0/2887078395 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f63e8012c40 con 0x7f63ec0b3e90 2023-12-20T06:04:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.882+0000 7f63e1ffb640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f63d0065cf0 0x7f63d00681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.882+0000 7f63e1ffb640 1 -- 172.21.15.32:0/2887078395 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f63e8083cf0 con 0x7f63ec0b3e90 2023-12-20T06:04:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.882+0000 7f63f0e7a640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f63d0065cf0 0x7f63d00681b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:01.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.882+0000 7f63f0e7a640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f63d0065cf0 0x7f63d00681b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0ad000 tx=0x7f63dc008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:01.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:01.886+0000 7f63e1ffb640 1 -- 172.21.15.32:0/2887078395 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f63e8051040 con 0x7f63ec0b3e90 2023-12-20T06:04:02.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63f267d640 1 -- 172.21.15.32:0/2887078395 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63ec0b38c0 con 0x7f63ec0b3e90 2023-12-20T06:04:02.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63e1ffb640 1 -- 172.21.15.32:0/2887078395 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v239) v1 ==== 74+0+17341 (secure 0 0 0) 0x7f63e8054660 con 0x7f63ec0b3e90 2023-12-20T06:04:02.046 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:02.047 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":239,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:03:58.834991+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":30,"pool_name":"backup_fs_metadata","create_time":"2023-12-20T06:03:32.174574+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"233","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"backup_fs"}},"read_balance":{"score_acting":2.5,"score_stable":2.5,"optimal_score":1,"raw_score_acting":2.5,"raw_score_stable":2.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":31,"pool_name":"backup_fs_data","create_time":"2023-12-20T06:03:33.509224+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"234","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"backup_fs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6835/1623322556":"2023-12-21T06:03:55.505418+0000","172.21.15.32:0/1232037079":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/2587801540":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/789250710":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/1603186500":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/2783112768":"2023-12-21T06:03:51.776767+0000","172.21.15.32:6837/1623322556":"2023-12-21T06:03:55.505418+0000","172.21.15.32:0/3054548475":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2026838481":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/597963739":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/274812680":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/3886671554":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1847933543":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/4013366117":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/582500261":"2023-12-21T06:03:51.776767+0000","172.21.15.32:0/805068630":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6833/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/1823890749":"2023-12-21T06:03:45.663020+0000","172.21.15.32:6832/64546":"2023-12-21T06:03:45.663020+0000","172.21.15.32:0/2852735264":"2023-12-21T06:03:45.663020+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:04:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63f267d640 1 -- 172.21.15.32:0/2887078395 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f63d0065cf0 msgr2=0x7f63d00681b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63f267d640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f63d0065cf0 0x7f63d00681b0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f63dc0ad000 tx=0x7f63dc008040 comp rx=0 tx=0).stop 2023-12-20T06:04:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63f267d640 1 -- 172.21.15.32:0/2887078395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63ec0b3e90 msgr2=0x7f63ec151b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63f267d640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63ec0b3e90 0x7f63ec151b30 secure :-1 s=READY pgs=2418 cs=0 l=1 rev1=1 crypto rx=0x7f63e800d5f0 tx=0x7f63e800dab0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63f267d640 1 -- 172.21.15.32:0/2887078395 shutdown_connections 2023-12-20T06:04:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63f267d640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f63d0065cf0 0x7f63d00681b0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63f267d640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f63ec159ab0 0x7f63ec0c4220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63f267d640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63ec0cd2e0 0x7f63ec0c3ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63f267d640 1 --2- 172.21.15.32:0/2887078395 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f63ec0b3e90 0x7f63ec151b30 unknown :-1 s=CLOSED pgs=2418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.046+0000 7f63f267d640 1 -- 172.21.15.32:0/2887078395 >> 172.21.15.32:0/2887078395 conn(0x7f63ec06c060 msgr2=0x7f63ec0b0980 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.050+0000 7f63f267d640 1 -- 172.21.15.32:0/2887078395 shutdown_connections 2023-12-20T06:04:02.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.050+0000 7f63f267d640 1 -- 172.21.15.32:0/2887078395 wait complete. 2023-12-20T06:04:02.068 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail backup_fs 2023-12-20T06:04:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f10593f0640 1 Processor -- start 2023-12-20T06:04:02.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f10593f0640 1 -- start start 2023-12-20T06:04:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f10593f0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f105406ce60 0x7f105415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f10593f0640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f105415f110 0x7f105415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f10593f0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f105415bbd0 con 0x7f10540af5e0 2023-12-20T06:04:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f10593f0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105415beb0 con 0x7f105406ce60 2023-12-20T06:04:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f10593f0640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f105415e910 con 0x7f105415f110 2023-12-20T06:04:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f104bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f105406ce60 0x7f105415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f104bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f105406ce60 0x7f105415b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58332/0 (socket says 172.21.15.32:58332) 2023-12-20T06:04:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f104bfff640 1 -- 172.21.15.32:0/2891514830 learned_addr learned my addr 172.21.15.32:0/2891514830 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f10537fe640 1 --2- 172.21.15.32:0/2891514830 >> v2:172.21.15.39:3300/0 conn(0x7f105415f110 0x7f105415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f104bfff640 1 -- 172.21.15.32:0/2891514830 >> v2:172.21.15.39:3300/0 conn(0x7f105415f110 msgr2=0x7f105415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f104bfff640 1 --2- 172.21.15.32:0/2891514830 >> v2:172.21.15.39:3300/0 conn(0x7f105415f110 0x7f105415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f104bfff640 1 -- 172.21.15.32:0/2891514830 >> v1:172.21.15.39:6789/0 conn(0x7f10540af5e0 legacy=0x7f10540af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:02.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.214+0000 7f104bfff640 1 -- 172.21.15.32:0/2891514830 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f105415ebf0 con 0x7f105406ce60 2023-12-20T06:04:02.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f104bfff640 1 --2- 172.21.15.32:0/2891514830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f105406ce60 0x7f105415b280 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f10540c8d40 tx=0x7f103c01c470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba5dcecf5ff4c2d6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:02.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10527fc640 1 -- 172.21.15.32:0/2891514830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f105415b9d0 con 0x7f105406ce60 2023-12-20T06:04:02.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10527fc640 1 -- 172.21.15.32:0/2891514830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f103c0ab610 con 0x7f105406ce60 2023-12-20T06:04:02.218 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10527fc640 1 -- 172.21.15.32:0/2891514830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f103c0ab9a0 con 0x7f105406ce60 2023-12-20T06:04:02.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10593f0640 1 -- 172.21.15.32:0/2891514830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f105406ce60 msgr2=0x7f105415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:02.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10593f0640 1 --2- 172.21.15.32:0/2891514830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f105406ce60 0x7f105415b280 secure :-1 s=READY pgs=2419 cs=0 l=1 rev1=1 crypto rx=0x7f10540c8d40 tx=0x7f103c01c470 comp rx=0 tx=0).stop 2023-12-20T06:04:02.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10593f0640 1 -- 172.21.15.32:0/2891514830 shutdown_connections 2023-12-20T06:04:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10593f0640 1 --2- 172.21.15.32:0/2891514830 >> v2:172.21.15.39:3300/0 conn(0x7f105415f110 0x7f105415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10593f0640 1 --2- 172.21.15.32:0/2891514830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f105406ce60 0x7f105415b280 unknown :-1 s=CLOSED pgs=2419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10593f0640 1 -- 172.21.15.32:0/2891514830 >> 172.21.15.32:0/2891514830 conn(0x7f105406a320 msgr2=0x7f10540ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10593f0640 1 -- 172.21.15.32:0/2891514830 shutdown_connections 2023-12-20T06:04:02.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10593f0640 1 -- 172.21.15.32:0/2891514830 wait complete. 2023-12-20T06:04:02.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.218+0000 7f10593f0640 1 Processor -- start 2023-12-20T06:04:02.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f10593f0640 1 -- start start 2023-12-20T06:04:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f10593f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f105406ce60 0x7f10541f7b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f10593f0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10540af5e0 0x7f10540b2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f10593f0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f105415f110 0x7f10541f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f10593f0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105414b7b0 con 0x7f105406ce60 2023-12-20T06:04:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f10593f0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f105414b940 con 0x7f105415f110 2023-12-20T06:04:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f10593f0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10540b2960 con 0x7f10540af5e0 2023-12-20T06:04:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f1052ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f105406ce60 0x7f10541f7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f1052ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f105406ce60 0x7f10541f7b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58336/0 (socket says 172.21.15.32:58336) 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f1052ffd640 1 -- 172.21.15.32:0/1543642822 learned_addr learned my addr 172.21.15.32:0/1543642822 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f104bfff640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10540af5e0 0x7f10540b2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f10537fe640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f105415f110 0x7f10541f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f1052ffd640 1 -- 172.21.15.32:0/1543642822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10540af5e0 msgr2=0x7f10540b2420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f1052ffd640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10540af5e0 0x7f10540b2420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f1052ffd640 1 -- 172.21.15.32:0/1543642822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f105415f110 msgr2=0x7f10541f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f1052ffd640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f105415f110 0x7f10541f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f1052ffd640 1 -- 172.21.15.32:0/1543642822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f103c0094d0 con 0x7f105406ce60 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f1052ffd640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f105406ce60 0x7f10541f7b60 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f104400c480 tx=0x7f104400c940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f104b7fe640 1 -- 172.21.15.32:0/1543642822 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10440052d0 con 0x7f105406ce60 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f104b7fe640 1 -- 172.21.15.32:0/1543642822 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1044005430 con 0x7f105406ce60 2023-12-20T06:04:02.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f104b7fe640 1 -- 172.21.15.32:0/1543642822 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1044003890 con 0x7f105406ce60 2023-12-20T06:04:02.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f10593f0640 1 -- 172.21.15.32:0/1543642822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10541503c0 con 0x7f105406ce60 2023-12-20T06:04:02.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f10593f0640 1 -- 172.21.15.32:0/1543642822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1054150980 con 0x7f105406ce60 2023-12-20T06:04:02.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.222+0000 7f10593f0640 1 -- 172.21.15.32:0/1543642822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f105404e000 con 0x7f105406ce60 2023-12-20T06:04:02.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.226+0000 7f104b7fe640 1 -- 172.21.15.32:0/1543642822 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f10440039f0 con 0x7f105406ce60 2023-12-20T06:04:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.226+0000 7f104b7fe640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065ca0 0x7f1034068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.226+0000 7f104b7fe640 1 -- 172.21.15.32:0/1543642822 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(239..239 src has 1..239) v4 ==== 6637+0+0 (secure 0 0 0) 0x7f1044083480 con 0x7f105406ce60 2023-12-20T06:04:02.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.226+0000 7f104bfff640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065ca0 0x7f1034068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.226+0000 7f104bfff640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065ca0 0x7f1034068160 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f103c0af000 tx=0x7f103c0b6040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:02.230 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.230+0000 7f104b7fe640 1 -- 172.21.15.32:0/1543642822 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f10440507e0 con 0x7f105406ce60 2023-12-20T06:04:02.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.418+0000 7f10593f0640 1 -- 172.21.15.32:0/1543642822 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "backup_fs"} v 0) v1 -- 0x7f10540cf450 con 0x7f105406ce60 2023-12-20T06:04:02.420 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T06:04:02.418+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T06:04:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f104b7fe640 1 -- 172.21.15.32:0/1543642822 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "backup_fs"}]=0 backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v121) v1 ==== 167+0+0 (secure 0 0 0) 0x7f1044053e00 con 0x7f105406ce60 2023-12-20T06:04:02.437 INFO:teuthology.orchestra.run.smithi032.stderr:backup_fs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T06:04:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 -- 172.21.15.32:0/1543642822 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065ca0 msgr2=0x7f1034068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:02.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065ca0 0x7f1034068160 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f103c0af000 tx=0x7f103c0b6040 comp rx=0 tx=0).stop 2023-12-20T06:04:02.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 -- 172.21.15.32:0/1543642822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f105406ce60 msgr2=0x7f10541f7b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:02.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f105406ce60 0x7f10541f7b60 secure :-1 s=READY pgs=2420 cs=0 l=1 rev1=1 crypto rx=0x7f104400c480 tx=0x7f104400c940 comp rx=0 tx=0).stop 2023-12-20T06:04:02.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 -- 172.21.15.32:0/1543642822 shutdown_connections 2023-12-20T06:04:02.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1034065ca0 0x7f1034068160 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f105415f110 0x7f10541f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10540af5e0 0x7f10540b2420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 --2- 172.21.15.32:0/1543642822 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f105406ce60 0x7f10541f7b60 unknown :-1 s=CLOSED pgs=2420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 -- 172.21.15.32:0/1543642822 >> 172.21.15.32:0/1543642822 conn(0x7f105406a320 msgr2=0x7f10540ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:02.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 -- 172.21.15.32:0/1543642822 shutdown_connections 2023-12-20T06:04:02.441 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.438+0000 7f10593f0640 1 -- 172.21.15.32:0/1543642822 wait complete. 2023-12-20T06:04:02.445 INFO:tasks.ceph.mds.c.smithi032.stderr: -1544> 2023-12-20T06:03:38.374+0000 7fd0fc8a5640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T06:04:02.464 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm backup_fs --yes-i-really-mean-it 2023-12-20T06:04:02.492 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T06:04:02.610 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c8308c640 1 Processor -- start 2023-12-20T06:04:02.611 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c8308c640 1 -- start start 2023-12-20T06:04:02.612 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c8308c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c7c157480 0x7f6c7c0c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c8308c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6c7c0cd4c0 0x7f6c7c0b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c8308c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c7c0b1100 con 0x7f6c7c0b0760 2023-12-20T06:04:02.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c8308c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7c0cde20 con 0x7f6c7c157480 2023-12-20T06:04:02.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c8308c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c7c0c8e30 con 0x7f6c7c0cd4c0 2023-12-20T06:04:02.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c80e28640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c7c157480 0x7f6c7c0c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c80e28640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6c7c157480 0x7f6c7c0c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58346/0 (socket says 172.21.15.32:58346) 2023-12-20T06:04:02.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c80e28640 1 -- 172.21.15.32:0/1613785477 learned_addr learned my addr 172.21.15.32:0/1613785477 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:02.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c81629640 1 --2- 172.21.15.32:0/1613785477 >> v2:172.21.15.39:3300/0 conn(0x7f6c7c0cd4c0 0x7f6c7c0b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.613 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c80e28640 1 -- 172.21.15.32:0/1613785477 >> v2:172.21.15.39:3300/0 conn(0x7f6c7c0cd4c0 msgr2=0x7f6c7c0b0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:02.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c80e28640 1 --2- 172.21.15.32:0/1613785477 >> v2:172.21.15.39:3300/0 conn(0x7f6c7c0cd4c0 0x7f6c7c0b0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c80e28640 1 -- 172.21.15.32:0/1613785477 >> v1:172.21.15.39:6789/0 conn(0x7f6c7c0b0760 legacy=0x7f6c7c15d660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:02.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c80e28640 1 -- 172.21.15.32:0/1613785477 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c7c0ce070 con 0x7f6c7c157480 2023-12-20T06:04:02.614 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.610+0000 7f6c81629640 1 --2- 172.21.15.32:0/1613785477 >> v2:172.21.15.39:3300/0 conn(0x7f6c7c0cd4c0 0x7f6c7c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:02.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c80e28640 1 --2- 172.21.15.32:0/1613785477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c157480 0x7f6c7c0c9840 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c071420 tx=0x7f6c7001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c9f1ddc3a46884a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:02.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c7b7fe640 1 -- 172.21.15.32:0/1613785477 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c700ab600 con 0x7f6c7c157480 2023-12-20T06:04:02.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c7b7fe640 1 -- 172.21.15.32:0/1613785477 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c700a7050 con 0x7f6c7c157480 2023-12-20T06:04:02.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c7b7fe640 1 -- 172.21.15.32:0/1613785477 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c700ab980 con 0x7f6c7c157480 2023-12-20T06:04:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c8308c640 1 -- 172.21.15.32:0/1613785477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c157480 msgr2=0x7f6c7c0c9840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c8308c640 1 --2- 172.21.15.32:0/1613785477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c157480 0x7f6c7c0c9840 secure :-1 s=READY pgs=2423 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c071420 tx=0x7f6c7001c400 comp rx=0 tx=0).stop 2023-12-20T06:04:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c8308c640 1 -- 172.21.15.32:0/1613785477 shutdown_connections 2023-12-20T06:04:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c8308c640 1 --2- 172.21.15.32:0/1613785477 >> v2:172.21.15.39:3300/0 conn(0x7f6c7c0cd4c0 0x7f6c7c0b0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c8308c640 1 --2- 172.21.15.32:0/1613785477 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c157480 0x7f6c7c0c9840 unknown :-1 s=CLOSED pgs=2423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c8308c640 1 -- 172.21.15.32:0/1613785477 >> 172.21.15.32:0/1613785477 conn(0x7f6c7c06a510 msgr2=0x7f6c7c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c8308c640 1 -- 172.21.15.32:0/1613785477 shutdown_connections 2023-12-20T06:04:02.617 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.614+0000 7f6c8308c640 1 -- 172.21.15.32:0/1613785477 wait complete. 2023-12-20T06:04:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c8308c640 1 Processor -- start 2023-12-20T06:04:02.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c8308c640 1 -- start start 2023-12-20T06:04:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c8308c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0b0760 0x7f6c7c147410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c8308c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c7c0cd4c0 0x7f6c7c070750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c8308c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c7c157480 0x7f6c7c070c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c8308c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c157f50 con 0x7f6c7c0b0760 2023-12-20T06:04:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c8308c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c147950 con 0x7f6c7c0cd4c0 2023-12-20T06:04:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c8308c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c7c15b1d0 con 0x7f6c7c157480 2023-12-20T06:04:02.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c80e28640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0b0760 0x7f6c7c147410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c80e28640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0b0760 0x7f6c7c147410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58348/0 (socket says 172.21.15.32:58348) 2023-12-20T06:04:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c80e28640 1 -- 172.21.15.32:0/4226614083 learned_addr learned my addr 172.21.15.32:0/4226614083 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c81629640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c7c157480 0x7f6c7c070c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c7bfff640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c7c0cd4c0 0x7f6c7c070750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c80e28640 1 -- 172.21.15.32:0/4226614083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c7c157480 msgr2=0x7f6c7c070c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c80e28640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c7c157480 0x7f6c7c070c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c80e28640 1 -- 172.21.15.32:0/4226614083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c7c0cd4c0 msgr2=0x7f6c7c070750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c80e28640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c7c0cd4c0 0x7f6c7c070750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c80e28640 1 -- 172.21.15.32:0/4226614083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c700094d0 con 0x7f6c7c0b0760 2023-12-20T06:04:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c7bfff640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c7c0cd4c0 0x7f6c7c070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:02.621 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c80e28640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0b0760 0x7f6c7c147410 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f6c7001aab0 tx=0x7f6c700191a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:02.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c797fa640 1 -- 172.21.15.32:0/4226614083 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c700b4640 con 0x7f6c7c0b0760 2023-12-20T06:04:02.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c797fa640 1 -- 172.21.15.32:0/4226614083 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c700b47c0 con 0x7f6c7c0b0760 2023-12-20T06:04:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c797fa640 1 -- 172.21.15.32:0/4226614083 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c700b4b70 con 0x7f6c7c0b0760 2023-12-20T06:04:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.618+0000 7f6c8308c640 1 -- 172.21.15.32:0/4226614083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c7c1f9550 con 0x7f6c7c0b0760 2023-12-20T06:04:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.622+0000 7f6c8308c640 1 -- 172.21.15.32:0/4226614083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c7c157900 con 0x7f6c7c0b0760 2023-12-20T06:04:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.622+0000 7f6c8308c640 1 -- 172.21.15.32:0/4226614083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c7c04e000 con 0x7f6c7c0b0760 2023-12-20T06:04:02.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.622+0000 7f6c797fa640 1 -- 172.21.15.32:0/4226614083 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f6c700b2070 con 0x7f6c7c0b0760 2023-12-20T06:04:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.622+0000 7f6c797fa640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c60065ca0 0x7f6c60068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.622+0000 7f6c797fa640 1 -- 172.21.15.32:0/4226614083 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f6c7011dae0 con 0x7f6c7c0b0760 2023-12-20T06:04:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.622+0000 7f6c7bfff640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c60065ca0 0x7f6c60068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:02.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.622+0000 7f6c7bfff640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c60065ca0 0x7f6c60068160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c163610 tx=0x7f6c68008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:02.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.626+0000 7f6c797fa640 1 -- 172.21.15.32:0/4226614083 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c700eac60 con 0x7f6c7c0b0760 2023-12-20T06:04:02.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:02.822+0000 7f6c8308c640 1 -- 172.21.15.32:0/4226614083 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f6c7c164580 con 0x7f6c7c0b0760 2023-12-20T06:04:03.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.438+0000 7f6c797fa640 1 -- 172.21.15.32:0/4226614083 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "backup_fs", "yes_i_really_mean_it": true}]=0 v122) v1 ==== 107+0+0 (secure 0 0 0) 0x7f6c700ee280 con 0x7f6c7c0b0760 2023-12-20T06:04:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 -- 172.21.15.32:0/4226614083 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c60065ca0 msgr2=0x7f6c60068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c60065ca0 0x7f6c60068160 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6c7c163610 tx=0x7f6c68008040 comp rx=0 tx=0).stop 2023-12-20T06:04:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 -- 172.21.15.32:0/4226614083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0b0760 msgr2=0x7f6c7c147410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0b0760 0x7f6c7c147410 secure :-1 s=READY pgs=2424 cs=0 l=1 rev1=1 crypto rx=0x7f6c7001aab0 tx=0x7f6c700191a0 comp rx=0 tx=0).stop 2023-12-20T06:04:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 -- 172.21.15.32:0/4226614083 shutdown_connections 2023-12-20T06:04:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c60065ca0 0x7f6c60068160 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c7c157480 0x7f6c7c070c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c7c0cd4c0 0x7f6c7c070750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:03.443 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 --2- 172.21.15.32:0/4226614083 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c7c0b0760 0x7f6c7c147410 unknown :-1 s=CLOSED pgs=2424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 -- 172.21.15.32:0/4226614083 >> 172.21.15.32:0/4226614083 conn(0x7f6c7c06a510 msgr2=0x7f6c7c156500 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 -- 172.21.15.32:0/4226614083 shutdown_connections 2023-12-20T06:04:03.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.442+0000 7f6c8308c640 1 -- 172.21.15.32:0/4226614083 wait complete. 2023-12-20T06:04:03.466 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_metadata backup_fs_metadata --yes-i-really-really-mean-it 2023-12-20T06:04:03.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.662+0000 7f69bd2d1640 1 Processor -- start 2023-12-20T06:04:03.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.662+0000 7f69bd2d1640 1 -- start start 2023-12-20T06:04:03.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69bd2d1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69b806d170 0x7f69b8167b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:03.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69bd2d1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b806ae40 con 0x7f69b806b2d0 2023-12-20T06:04:03.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69bd2d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f69b806a230 con 0x7f69b815cc00 2023-12-20T06:04:03.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69bd2d1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f69b806a510 con 0x7f69b806d170 2023-12-20T06:04:03.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69b77fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f69b806d170 0x7f69b8167b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:03.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69affff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f69b806b2d0 0x7f69b806ca60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36216/0 (socket says 172.21.15.32:36216) 2023-12-20T06:04:03.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69affff640 1 -- 172.21.15.32:0/973226883 learned_addr learned my addr 172.21.15.32:0/973226883 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:03.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69b77fe640 1 -- 172.21.15.32:0/973226883 >> v1:172.21.15.39:6789/0 conn(0x7f69b815cc00 legacy=0x7f69b80cd410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:03.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69b77fe640 1 -- 172.21.15.32:0/973226883 >> v1:172.21.15.32:6789/0 conn(0x7f69b806b2d0 legacy=0x7f69b806ca60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:03.666 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69b77fe640 1 -- 172.21.15.32:0/973226883 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b806a7f0 con 0x7f69b806d170 2023-12-20T06:04:03.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69b77fe640 1 --2- 172.21.15.32:0/973226883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b806d170 0x7f69b8167b40 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009d90 tx=0x7f69b001c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3f8f86ced1b62e6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:03.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.666+0000 7f69b67fc640 1 -- 172.21.15.32:0/973226883 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b00a5840 con 0x7f69b806d170 2023-12-20T06:04:03.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69b67fc640 1 -- 172.21.15.32:0/973226883 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69b00a59a0 con 0x7f69b806d170 2023-12-20T06:04:03.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69b67fc640 1 -- 172.21.15.32:0/973226883 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69b00a5d30 con 0x7f69b806d170 2023-12-20T06:04:03.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 -- 172.21.15.32:0/973226883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b806d170 msgr2=0x7f69b8167b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:03.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 --2- 172.21.15.32:0/973226883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b806d170 0x7f69b8167b40 secure :-1 s=READY pgs=2426 cs=0 l=1 rev1=1 crypto rx=0x7f69b0009d90 tx=0x7f69b001c7e0 comp rx=0 tx=0).stop 2023-12-20T06:04:03.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 -- 172.21.15.32:0/973226883 shutdown_connections 2023-12-20T06:04:03.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 --2- 172.21.15.32:0/973226883 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b806d170 0x7f69b8167b40 unknown :-1 s=CLOSED pgs=2426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:03.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 -- 172.21.15.32:0/973226883 >> 172.21.15.32:0/973226883 conn(0x7f69b80cf8d0 msgr2=0x7f69b80cbad0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:03.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 -- 172.21.15.32:0/973226883 shutdown_connections 2023-12-20T06:04:03.670 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 -- 172.21.15.32:0/973226883 wait complete. 2023-12-20T06:04:03.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 Processor -- start 2023-12-20T06:04:03.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 -- start start 2023-12-20T06:04:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69b806b2d0 0x7f69b81f32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b806d170 0x7f69b80c8480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f69b815cc00 0x7f69b80c7b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b81f5db0 con 0x7f69b806d170 2023-12-20T06:04:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f69b80c89c0 con 0x7f69b806b2d0 2023-12-20T06:04:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69bd2d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f69b80c80b0 con 0x7f69b815cc00 2023-12-20T06:04:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69affff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b806d170 0x7f69b80c8480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:03.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69affff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b806d170 0x7f69b80c8480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58376/0 (socket says 172.21.15.32:58376) 2023-12-20T06:04:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69affff640 1 -- 172.21.15.32:0/1546309132 learned_addr learned my addr 172.21.15.32:0/1546309132 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69b77fe640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f69b815cc00 0x7f69b80c7b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.670+0000 7f69b6ffd640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69b806b2d0 0x7f69b81f32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69affff640 1 -- 172.21.15.32:0/1546309132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f69b815cc00 msgr2=0x7f69b80c7b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69affff640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f69b815cc00 0x7f69b80c7b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69affff640 1 -- 172.21.15.32:0/1546309132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69b806b2d0 msgr2=0x7f69b81f32f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69affff640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69b806b2d0 0x7f69b81f32f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69affff640 1 -- 172.21.15.32:0/1546309132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f69b0009a70 con 0x7f69b806d170 2023-12-20T06:04:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69affff640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b806d170 0x7f69b80c8480 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009e80 tx=0x7f69a000c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69af7fe640 1 -- 172.21.15.32:0/1546309132 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a00078f0 con 0x7f69b806d170 2023-12-20T06:04:03.675 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69af7fe640 1 -- 172.21.15.32:0/1546309132 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69a0007a50 con 0x7f69b806d170 2023-12-20T06:04:03.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69af7fe640 1 -- 172.21.15.32:0/1546309132 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69a0007e00 con 0x7f69b806d170 2023-12-20T06:04:03.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69bd2d1640 1 -- 172.21.15.32:0/1546309132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f69b81f6000 con 0x7f69b806d170 2023-12-20T06:04:03.676 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69bd2d1640 1 -- 172.21.15.32:0/1546309132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f69b81f65c0 con 0x7f69b806d170 2023-12-20T06:04:03.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69bd2d1640 1 -- 172.21.15.32:0/1546309132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f69b8071870 con 0x7f69b806d170 2023-12-20T06:04:03.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.674+0000 7f69af7fe640 1 -- 172.21.15.32:0/1546309132 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f69a0009070 con 0x7f69b806d170 2023-12-20T06:04:03.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.678+0000 7f69af7fe640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6998065ca0 0x7f6998068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:03.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.678+0000 7f69af7fe640 1 -- 172.21.15.32:0/1546309132 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(240..240 src has 1..240) v4 ==== 6723+0+0 (secure 0 0 0) 0x7f69a0085810 con 0x7f69b806d170 2023-12-20T06:04:03.677 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.678+0000 7f69b6ffd640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6998065ca0 0x7f6998068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:03.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.678+0000 7f69b6ffd640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6998065ca0 0x7f6998068160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f69b80b0ae0 tx=0x7f69a8009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:03.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.682+0000 7f69af7fe640 1 -- 172.21.15.32:0/1546309132 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f69a0052b10 con 0x7f69b806d170 2023-12-20T06:04:03.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:03.842+0000 7f69bd2d1640 1 -- 172.21.15.32:0/1546309132 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f69b804e000 con 0x7f69b806d170 2023-12-20T06:04:04.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.454+0000 7f69af7fe640 1 -- 172.21.15.32:0/1546309132 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_metadata", "pool2": "backup_fs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_metadata' removed v241) v1 ==== 190+0+0 (secure 0 0 0) 0x7f69a0056130 con 0x7f69b806d170 2023-12-20T06:04:04.454 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'backup_fs_metadata' removed 2023-12-20T06:04:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.454+0000 7f69bd2d1640 1 -- 172.21.15.32:0/1546309132 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6998065ca0 msgr2=0x7f6998068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.454+0000 7f69bd2d1640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6998065ca0 0x7f6998068160 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f69b80b0ae0 tx=0x7f69a8009040 comp rx=0 tx=0).stop 2023-12-20T06:04:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.454+0000 7f69bd2d1640 1 -- 172.21.15.32:0/1546309132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b806d170 msgr2=0x7f69b80c8480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:04.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.454+0000 7f69bd2d1640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b806d170 0x7f69b80c8480 secure :-1 s=READY pgs=2427 cs=0 l=1 rev1=1 crypto rx=0x7f69a0009e80 tx=0x7f69a000c430 comp rx=0 tx=0).stop 2023-12-20T06:04:04.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.458+0000 7f69bd2d1640 1 -- 172.21.15.32:0/1546309132 shutdown_connections 2023-12-20T06:04:04.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.458+0000 7f69bd2d1640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6998065ca0 0x7f6998068160 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:04.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.458+0000 7f69bd2d1640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f69b815cc00 0x7f69b80c7b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:04.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.458+0000 7f69bd2d1640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f69b806d170 0x7f69b80c8480 unknown :-1 s=CLOSED pgs=2427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:04.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.458+0000 7f69bd2d1640 1 --2- 172.21.15.32:0/1546309132 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f69b806b2d0 0x7f69b81f32f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:04.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.458+0000 7f69bd2d1640 1 -- 172.21.15.32:0/1546309132 >> 172.21.15.32:0/1546309132 conn(0x7f69b80cf8d0 msgr2=0x7f69b80c9790 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:04.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.458+0000 7f69bd2d1640 1 -- 172.21.15.32:0/1546309132 shutdown_connections 2023-12-20T06:04:04.458 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.458+0000 7f69bd2d1640 1 -- 172.21.15.32:0/1546309132 wait complete. 2023-12-20T06:04:04.480 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm backup_fs_data backup_fs_data --yes-i-really-really-mean-it 2023-12-20T06:04:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.634+0000 7fd86b28c640 1 Processor -- start 2023-12-20T06:04:04.635 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.634+0000 7fd86b28c640 1 -- start start 2023-12-20T06:04:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.634+0000 7fd86b28c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd864158520 0x7fd86406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.634+0000 7fd86b28c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8640ca7f0 con 0x7fd864070230 2023-12-20T06:04:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.634+0000 7fd86b28c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8640caad0 con 0x7fd8640ca050 2023-12-20T06:04:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.634+0000 7fd86b28c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8640cadb0 con 0x7fd864158520 2023-12-20T06:04:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.634+0000 7fd869829640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd864070230 0x7fd86415b0c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36232/0 (socket says 172.21.15.32:36232) 2023-12-20T06:04:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.634+0000 7fd869829640 1 -- 172.21.15.32:0/4066350287 learned_addr learned my addr 172.21.15.32:0/4066350287 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.634+0000 7fd869028640 1 --2- 172.21.15.32:0/4066350287 >> v2:172.21.15.39:3300/0 conn(0x7fd864158520 0x7fd86406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4035386484 0 0) 0x7fd8640ca7f0 con 0x7fd864070230 2023-12-20T06:04:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd850003530 con 0x7fd864070230 2023-12-20T06:04:04.639 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1329658757 0 0) 0x7fd8640caad0 con 0x7fd8640ca050 2023-12-20T06:04:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8640ca7f0 con 0x7fd8640ca050 2023-12-20T06:04:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 488436841 0 0) 0x7fd850003530 con 0x7fd864070230 2023-12-20T06:04:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 >> v2:172.21.15.39:3300/0 conn(0x7fd864158520 msgr2=0x7fd86406dcb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 --2- 172.21.15.32:0/4066350287 >> v2:172.21.15.39:3300/0 conn(0x7fd864158520 0x7fd86406dcb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 >> v1:172.21.15.39:6789/0 conn(0x7fd8640ca050 legacy=0x7fd86415ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd86415e780 con 0x7fd864070230 2023-12-20T06:04:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fd85c001a00 con 0x7fd864070230 2023-12-20T06:04:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 >> v1:172.21.15.32:6789/0 conn(0x7fd864070230 legacy=0x7fd86415b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 --2- 172.21.15.32:0/4066350287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd850003b10 0x7fd850003f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:04.640 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 --2- 172.21.15.32:0/4066350287 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8500047d0 0x7fd850004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 --2- 172.21.15.32:0/4066350287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 0x7fd85000b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640cadb0 con 0x7fd850005190 2023-12-20T06:04:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd85000c0c0 con 0x7fd850003b10 2023-12-20T06:04:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd85000c280 con 0x7fd8500047d0 2023-12-20T06:04:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd869028640 1 --2- 172.21.15.32:0/4066350287 >> v2:172.21.15.39:3300/0 conn(0x7fd864158520 0x7fd86406dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd869829640 1 --2- 172.21.15.32:0/4066350287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 0x7fd85000b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd868827640 1 --2- 172.21.15.32:0/4066350287 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8500047d0 0x7fd850004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd869028640 1 --2- 172.21.15.32:0/4066350287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd850003b10 0x7fd850003f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:04.641 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd869829640 1 -- 172.21.15.32:0/4066350287 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8500047d0 msgr2=0x7fd850004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd869829640 1 --2- 172.21.15.32:0/4066350287 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8500047d0 0x7fd850004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd869829640 1 -- 172.21.15.32:0/4066350287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd850003b10 msgr2=0x7fd850003f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd869829640 1 --2- 172.21.15.32:0/4066350287 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd850003b10 0x7fd850003f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:04.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd869829640 1 -- 172.21.15.32:0/4066350287 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd850003530 con 0x7fd850005190 2023-12-20T06:04:04.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.638+0000 7fd869829640 1 -- 172.21.15.32:0/4066350287 reap_dead start 2023-12-20T06:04:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd869829640 1 --2- 172.21.15.32:0/4066350287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 0x7fd85000b990 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00a7a0 tx=0x7fd85c01d2a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52973d9ec4418847 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0ac620 con 0x7fd850005190 2023-12-20T06:04:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd85c0a8070 con 0x7fd850005190 2023-12-20T06:04:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd85bfff640 1 -- 172.21.15.32:0/4066350287 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd85c0ac9e0 con 0x7fd850005190 2023-12-20T06:04:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 -- 172.21.15.32:0/4066350287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 msgr2=0x7fd85000b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 --2- 172.21.15.32:0/4066350287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 0x7fd85000b990 secure :-1 s=READY pgs=2429 cs=0 l=1 rev1=1 crypto rx=0x7fd85c00a7a0 tx=0x7fd85c01d2a0 comp rx=0 tx=0).stop 2023-12-20T06:04:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 -- 172.21.15.32:0/4066350287 shutdown_connections 2023-12-20T06:04:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 --2- 172.21.15.32:0/4066350287 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 0x7fd85000b990 unknown :-1 s=CLOSED pgs=2429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 -- 172.21.15.32:0/4066350287 >> 172.21.15.32:0/4066350287 conn(0x7fd86406a510 msgr2=0x7fd8640ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 -- 172.21.15.32:0/4066350287 shutdown_connections 2023-12-20T06:04:04.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 -- 172.21.15.32:0/4066350287 wait complete. 2023-12-20T06:04:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 Processor -- start 2023-12-20T06:04:04.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 -- start start 2023-12-20T06:04:04.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 0x7fd8640c6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:04.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8641543c0 0x7fd8640cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:04.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640cb730 0x7fd8640b3410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:04.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640cbbb0 con 0x7fd850005190 2023-12-20T06:04:04.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640cd060 con 0x7fd8641543c0 2023-12-20T06:04:04.647 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.642+0000 7fd86b28c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8640cc450 con 0x7fd8640cb730 2023-12-20T06:04:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd869028640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 0x7fd8640c6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd869028640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 0x7fd8640c6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58388/0 (socket says 172.21.15.32:58388) 2023-12-20T06:04:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd869028640 1 -- 172.21.15.32:0/495844632 learned_addr learned my addr 172.21.15.32:0/495844632 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd868827640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8641543c0 0x7fd8640cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd869829640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640cb730 0x7fd8640b3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd869028640 1 -- 172.21.15.32:0/495844632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640cb730 msgr2=0x7fd8640b3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd869028640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640cb730 0x7fd8640b3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd869028640 1 -- 172.21.15.32:0/495844632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8641543c0 msgr2=0x7fd8640cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd869028640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8641543c0 0x7fd8640cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd869028640 1 -- 172.21.15.32:0/495844632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8640cc730 con 0x7fd850005190 2023-12-20T06:04:04.648 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd868827640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8641543c0 0x7fd8640cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:04.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd869028640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 0x7fd8640c6aa0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fd860011ca0 tx=0x7fd86000f610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:04.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd859ffb640 1 -- 172.21.15.32:0/495844632 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd86000fd20 con 0x7fd850005190 2023-12-20T06:04:04.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd859ffb640 1 -- 172.21.15.32:0/495844632 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd86000fe80 con 0x7fd850005190 2023-12-20T06:04:04.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd859ffb640 1 -- 172.21.15.32:0/495844632 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd860004240 con 0x7fd850005190 2023-12-20T06:04:04.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd86b28c640 1 -- 172.21.15.32:0/495844632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8640cca10 con 0x7fd850005190 2023-12-20T06:04:04.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd86b28c640 1 -- 172.21.15.32:0/495844632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd864154840 con 0x7fd850005190 2023-12-20T06:04:04.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd86b28c640 1 -- 172.21.15.32:0/495844632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd86404e000 con 0x7fd850005190 2023-12-20T06:04:04.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.646+0000 7fd859ffb640 1 -- 172.21.15.32:0/495844632 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fd8600043a0 con 0x7fd850005190 2023-12-20T06:04:04.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.650+0000 7fd859ffb640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd848065ca0 0x7fd848068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:04.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.650+0000 7fd859ffb640 1 -- 172.21.15.32:0/495844632 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(241..241 src has 1..241) v4 ==== 6279+0+0 (secure 0 0 0) 0x7fd860083ec0 con 0x7fd850005190 2023-12-20T06:04:04.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.650+0000 7fd868827640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd848065ca0 0x7fd848068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:04.651 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.650+0000 7fd868827640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd848065ca0 0x7fd848068160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd864156fa0 tx=0x7fd854009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:04.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.654+0000 7fd859ffb640 1 -- 172.21.15.32:0/495844632 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd860051370 con 0x7fd850005190 2023-12-20T06:04:04.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:04.810+0000 7fd86b28c640 1 -- 172.21.15.32:0/495844632 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7fd864157c30 con 0x7fd850005190 2023-12-20T06:04:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.518+0000 7fd859ffb640 1 -- 172.21.15.32:0/495844632 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "backup_fs_data", "pool2": "backup_fs_data", "yes_i_really_really_mean_it": true}]=0 pool 'backup_fs_data' removed v242) v1 ==== 178+0+0 (secure 0 0 0) 0x7fd860022020 con 0x7fd850005190 2023-12-20T06:04:05.517 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'backup_fs_data' removed 2023-12-20T06:04:05.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 -- 172.21.15.32:0/495844632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd848065ca0 msgr2=0x7fd848068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd848065ca0 0x7fd848068160 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd864156fa0 tx=0x7fd854009040 comp rx=0 tx=0).stop 2023-12-20T06:04:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 -- 172.21.15.32:0/495844632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 msgr2=0x7fd8640c6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 0x7fd8640c6aa0 secure :-1 s=READY pgs=2430 cs=0 l=1 rev1=1 crypto rx=0x7fd860011ca0 tx=0x7fd86000f610 comp rx=0 tx=0).stop 2023-12-20T06:04:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 -- 172.21.15.32:0/495844632 shutdown_connections 2023-12-20T06:04:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd848065ca0 0x7fd848068160 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8640cb730 0x7fd8640b3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8641543c0 0x7fd8640cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 --2- 172.21.15.32:0/495844632 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd850005190 0x7fd8640c6aa0 unknown :-1 s=CLOSED pgs=2430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 -- 172.21.15.32:0/495844632 >> 172.21.15.32:0/495844632 conn(0x7fd86406a510 msgr2=0x7fd8640b2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 -- 172.21.15.32:0/495844632 shutdown_connections 2023-12-20T06:04:05.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.522+0000 7fd86b28c640 1 -- 172.21.15.32:0/495844632 wait complete. 2023-12-20T06:04:05.545 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T06:04:05.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.698+0000 7f53888a7640 1 Processor -- start 2023-12-20T06:04:05.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.698+0000 7f53888a7640 1 -- start start 2023-12-20T06:04:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.698+0000 7f53888a7640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f53840c8be0 0x7f53840ccfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.698+0000 7f53888a7640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53840cc160 con 0x7f5384159100 2023-12-20T06:04:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.698+0000 7f53888a7640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f53840c9e50 con 0x7f53840b0720 2023-12-20T06:04:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.698+0000 7f53888a7640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f53840ca130 con 0x7f53840c8be0 2023-12-20T06:04:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.698+0000 7f53837fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5384159100 0x7f53840b0010 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36238/0 (socket says 172.21.15.32:36238) 2023-12-20T06:04:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.698+0000 7f53837fe640 1 -- 172.21.15.32:0/584423241 learned_addr learned my addr 172.21.15.32:0/584423241 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:05.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.698+0000 7f5383fff640 1 --2- 172.21.15.32:0/584423241 >> v2:172.21.15.39:3300/0 conn(0x7f53840c8be0 0x7f53840ccfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.698+0000 7f53827fc640 1 -- 172.21.15.32:0/584423241 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1651329883 0 0) 0x7f53840cc160 con 0x7f5384159100 2023-12-20T06:04:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f53827fc640 1 -- 172.21.15.32:0/584423241 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f536c003530 con 0x7f5384159100 2023-12-20T06:04:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f53827fc640 1 -- 172.21.15.32:0/584423241 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4072773515 0 0) 0x7f53840c9e50 con 0x7f53840b0720 2023-12-20T06:04:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f53827fc640 1 -- 172.21.15.32:0/584423241 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f53840cc160 con 0x7f53840b0720 2023-12-20T06:04:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f5383fff640 1 -- 172.21.15.32:0/584423241 >> v1:172.21.15.39:6789/0 conn(0x7f53840b0720 legacy=0x7f53840cc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f5383fff640 1 -- 172.21.15.32:0/584423241 >> v1:172.21.15.32:6789/0 conn(0x7f5384159100 legacy=0x7f53840b0010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:05.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f5383fff640 1 -- 172.21.15.32:0/584423241 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f53840ca410 con 0x7f53840c8be0 2023-12-20T06:04:05.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f5383fff640 1 --2- 172.21.15.32:0/584423241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53840c8be0 0x7f53840ccfc0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f537c009d90 tx=0x7f537c01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=da99d7a869d700ed server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:05.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f53827fc640 1 -- 172.21.15.32:0/584423241 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0a58a0 con 0x7f53840c8be0 2023-12-20T06:04:05.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f53827fc640 1 -- 172.21.15.32:0/584423241 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c0a5a00 con 0x7f53840c8be0 2023-12-20T06:04:05.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f53827fc640 1 -- 172.21.15.32:0/584423241 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0a5d90 con 0x7f53840c8be0 2023-12-20T06:04:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f53888a7640 1 -- 172.21.15.32:0/584423241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53840c8be0 msgr2=0x7f53840ccfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.702+0000 7f53888a7640 1 --2- 172.21.15.32:0/584423241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53840c8be0 0x7f53840ccfc0 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f537c009d90 tx=0x7f537c01c950 comp rx=0 tx=0).stop 2023-12-20T06:04:05.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 -- 172.21.15.32:0/584423241 shutdown_connections 2023-12-20T06:04:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 --2- 172.21.15.32:0/584423241 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53840c8be0 0x7f53840ccfc0 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 -- 172.21.15.32:0/584423241 >> 172.21.15.32:0/584423241 conn(0x7f538406c060 msgr2=0x7f538406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 -- 172.21.15.32:0/584423241 shutdown_connections 2023-12-20T06:04:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 -- 172.21.15.32:0/584423241 wait complete. 2023-12-20T06:04:05.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 Processor -- start 2023-12-20T06:04:05.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 -- start start 2023-12-20T06:04:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53840b0720 0x7f53840c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53840c8be0 0x7f53840aedf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:05.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384159100 0x7f53840af330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53840c3150 con 0x7f5384159100 2023-12-20T06:04:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f53841597e0 con 0x7f53840c8be0 2023-12-20T06:04:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53888a7640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f53840c2540 con 0x7f53840b0720 2023-12-20T06:04:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f5383fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384159100 0x7f53840af330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f5383fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384159100 0x7f53840af330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58398/0 (socket says 172.21.15.32:58398) 2023-12-20T06:04:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f5383fff640 1 -- 172.21.15.32:0/1579680474 learned_addr learned my addr 172.21.15.32:0/1579680474 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53837fe640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53840b0720 0x7f53840c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f5382ffd640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53840c8be0 0x7f53840aedf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f5383fff640 1 -- 172.21.15.32:0/1579680474 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53840b0720 msgr2=0x7f53840c1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:05.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f5383fff640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53840b0720 0x7f53840c1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f5383fff640 1 -- 172.21.15.32:0/1579680474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53840c8be0 msgr2=0x7f53840aedf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f5383fff640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53840c8be0 0x7f53840aedf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f5383fff640 1 -- 172.21.15.32:0/1579680474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f537c009a70 con 0x7f5384159100 2023-12-20T06:04:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.706+0000 7f53837fe640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53840b0720 0x7f53840c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.710+0000 7f5383fff640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384159100 0x7f53840af330 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f537c0b0040 tx=0x7f537c0196f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.710+0000 7f536bfff640 1 -- 172.21.15.32:0/1579680474 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0b9650 con 0x7f5384159100 2023-12-20T06:04:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.710+0000 7f536bfff640 1 -- 172.21.15.32:0/1579680474 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f537c0b97b0 con 0x7f5384159100 2023-12-20T06:04:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.710+0000 7f536bfff640 1 -- 172.21.15.32:0/1579680474 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f537c0b9b60 con 0x7f5384159100 2023-12-20T06:04:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.710+0000 7f53888a7640 1 -- 172.21.15.32:0/1579680474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f53840c2820 con 0x7f5384159100 2023-12-20T06:04:05.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.710+0000 7f53888a7640 1 -- 172.21.15.32:0/1579680474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f53840c2de0 con 0x7f5384159100 2023-12-20T06:04:05.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.710+0000 7f53888a7640 1 -- 172.21.15.32:0/1579680474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5384074100 con 0x7f5384159100 2023-12-20T06:04:05.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.710+0000 7f536bfff640 1 -- 172.21.15.32:0/1579680474 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f537c0b9cc0 con 0x7f5384159100 2023-12-20T06:04:05.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.710+0000 7f536bfff640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5370065ca0 0x7f5370068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:05.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.710+0000 7f536bfff640 1 -- 172.21.15.32:0/1579680474 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5859+0+0 (secure 0 0 0) 0x7f537c11e990 con 0x7f5384159100 2023-12-20T06:04:05.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.714+0000 7f53837fe640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5370065ca0 0x7f5370068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:05.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.714+0000 7f53837fe640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5370065ca0 0x7f5370068160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5374000f70 tx=0x7f5374009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:05.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.714+0000 7f536bfff640 1 -- 172.21.15.32:0/1579680474 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f537c0ebfd0 con 0x7f5384159100 2023-12-20T06:04:05.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.874+0000 7f53888a7640 1 -- 172.21.15.32:0/1579680474 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7f53840cdbd0 con 0x7f5384159100 2023-12-20T06:04:05.879 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.878+0000 7f536bfff640 1 -- 172.21.15.32:0/1579680474 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v79) v1 ==== 120+0+0 (secure 0 0 0) 0x7f537c0ef5f0 con 0x7f5384159100 2023-12-20T06:04:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 -- 172.21.15.32:0/1579680474 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5370065ca0 msgr2=0x7f5370068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5370065ca0 0x7f5370068160 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f5374000f70 tx=0x7f5374009040 comp rx=0 tx=0).stop 2023-12-20T06:04:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 -- 172.21.15.32:0/1579680474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384159100 msgr2=0x7f53840af330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384159100 0x7f53840af330 secure :-1 s=READY pgs=2432 cs=0 l=1 rev1=1 crypto rx=0x7f537c0b0040 tx=0x7f537c0196f0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 -- 172.21.15.32:0/1579680474 shutdown_connections 2023-12-20T06:04:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5370065ca0 0x7f5370068160 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5384159100 0x7f53840af330 unknown :-1 s=CLOSED pgs=2432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f53840c8be0 0x7f53840aedf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 --2- 172.21.15.32:0/1579680474 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f53840b0720 0x7f53840c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 -- 172.21.15.32:0/1579680474 >> 172.21.15.32:0/1579680474 conn(0x7f538406c060 msgr2=0x7f5384153cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 -- 172.21.15.32:0/1579680474 shutdown_connections 2023-12-20T06:04:05.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:05.882+0000 7f53888a7640 1 -- 172.21.15.32:0/1579680474 wait complete. 2023-12-20T06:04:05.902 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T06:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.050+0000 7f73b568c640 1 Processor -- start 2023-12-20T06:04:06.052 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.050+0000 7f73b568c640 1 -- start start 2023-12-20T06:04:06.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73b568c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f73b015abb0 0x7f73b00caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73b568c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f73b00b0cb0 0x7f73b015bb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73b568c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f73b0070230 0x7f73b00706c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73b568c640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b006e160 con 0x7f73b00b0cb0 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73b568c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f73b0070c00 con 0x7f73b015abb0 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73b568c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b015b6a0 con 0x7f73b0070230 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73a7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f73b00b0cb0 0x7f73b015bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73a7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f73b00b0cb0 0x7f73b015bb30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58414/0 (socket says 172.21.15.32:58414) 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73a7fff640 1 -- 172.21.15.32:0/3172367895 learned_addr learned my addr 172.21.15.32:0/3172367895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73af7fe640 1 --2- 172.21.15.32:0/3172367895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f73b0070230 0x7f73b00706c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73aeffd640 1 --2- 172.21.15.32:0/3172367895 >> v2:172.21.15.39:3300/0 conn(0x7f73b015abb0 0x7f73b00caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73a7fff640 1 -- 172.21.15.32:0/3172367895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f73b0070230 msgr2=0x7f73b00706c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73a7fff640 1 --2- 172.21.15.32:0/3172367895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f73b0070230 0x7f73b00706c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73a7fff640 1 -- 172.21.15.32:0/3172367895 >> v2:172.21.15.39:3300/0 conn(0x7f73b015abb0 msgr2=0x7f73b00caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73a7fff640 1 --2- 172.21.15.32:0/3172367895 >> v2:172.21.15.39:3300/0 conn(0x7f73b015abb0 0x7f73b00caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73a7fff640 1 -- 172.21.15.32:0/3172367895 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73b015c070 con 0x7f73b00b0cb0 2023-12-20T06:04:06.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.054+0000 7f73af7fe640 1 --2- 172.21.15.32:0/3172367895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f73b0070230 0x7f73b00706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73a7fff640 1 --2- 172.21.15.32:0/3172367895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73b00b0cb0 0x7f73b015bb30 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f73b00713f0 tx=0x7f739801c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3be289ae3c2672 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73ae7fc640 1 -- 172.21.15.32:0/3172367895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73980aa620 con 0x7f73b00b0cb0 2023-12-20T06:04:06.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73ae7fc640 1 -- 172.21.15.32:0/3172367895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73980aa780 con 0x7f73b00b0cb0 2023-12-20T06:04:06.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73ae7fc640 1 -- 172.21.15.32:0/3172367895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73980aab10 con 0x7f73b00b0cb0 2023-12-20T06:04:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73b568c640 1 -- 172.21.15.32:0/3172367895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73b00b0cb0 msgr2=0x7f73b015bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73b568c640 1 --2- 172.21.15.32:0/3172367895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73b00b0cb0 0x7f73b015bb30 secure :-1 s=READY pgs=2433 cs=0 l=1 rev1=1 crypto rx=0x7f73b00713f0 tx=0x7f739801c500 comp rx=0 tx=0).stop 2023-12-20T06:04:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73b568c640 1 -- 172.21.15.32:0/3172367895 shutdown_connections 2023-12-20T06:04:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73b568c640 1 --2- 172.21.15.32:0/3172367895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f73b0070230 0x7f73b00706c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73b568c640 1 --2- 172.21.15.32:0/3172367895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73b00b0cb0 0x7f73b015bb30 unknown :-1 s=CLOSED pgs=2433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73b568c640 1 --2- 172.21.15.32:0/3172367895 >> v2:172.21.15.39:3300/0 conn(0x7f73b015abb0 0x7f73b00caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73b568c640 1 -- 172.21.15.32:0/3172367895 >> 172.21.15.32:0/3172367895 conn(0x7f73b006a510 msgr2=0x7f73b00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73b568c640 1 -- 172.21.15.32:0/3172367895 shutdown_connections 2023-12-20T06:04:06.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73b568c640 1 -- 172.21.15.32:0/3172367895 wait complete. 2023-12-20T06:04:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73b568c640 1 Processor -- start 2023-12-20T06:04:06.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.058+0000 7f73b568c640 1 -- start start 2023-12-20T06:04:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73b568c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73b0070230 0x7f73b00c4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73b568c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f73b00b0cb0 0x7f73b0140920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73b568c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73b015abb0 0x7f73b0140e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73b568c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0145970 con 0x7f73b0070230 2023-12-20T06:04:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73b568c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0145c50 con 0x7f73b015abb0 2023-12-20T06:04:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73b568c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f73b0145f30 con 0x7f73b00b0cb0 2023-12-20T06:04:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73aeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73b0070230 0x7f73b00c4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73aeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73b0070230 0x7f73b00c4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58426/0 (socket says 172.21.15.32:58426) 2023-12-20T06:04:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73aeffd640 1 -- 172.21.15.32:0/2180699196 learned_addr learned my addr 172.21.15.32:0/2180699196 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:06.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73a7fff640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f73b00b0cb0 0x7f73b0140920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73af7fe640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73b015abb0 0x7f73b0140e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73aeffd640 1 -- 172.21.15.32:0/2180699196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f73b00b0cb0 msgr2=0x7f73b0140920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73aeffd640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f73b00b0cb0 0x7f73b0140920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73aeffd640 1 -- 172.21.15.32:0/2180699196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73b015abb0 msgr2=0x7f73b0140e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73aeffd640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73b015abb0 0x7f73b0140e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73aeffd640 1 -- 172.21.15.32:0/2180699196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f73980094d0 con 0x7f73b0070230 2023-12-20T06:04:06.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73aeffd640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73b0070230 0x7f73b00c4830 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f73a000d5f0 tx=0x7f73a000dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73a77fe640 1 -- 172.21.15.32:0/2180699196 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a00125f0 con 0x7f73b0070230 2023-12-20T06:04:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73a77fe640 1 -- 172.21.15.32:0/2180699196 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f73a0012750 con 0x7f73b0070230 2023-12-20T06:04:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73a77fe640 1 -- 172.21.15.32:0/2180699196 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f73a0012ae0 con 0x7f73b0070230 2023-12-20T06:04:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73b568c640 1 -- 172.21.15.32:0/2180699196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f73b0146210 con 0x7f73b0070230 2023-12-20T06:04:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73b568c640 1 -- 172.21.15.32:0/2180699196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f73b0146a10 con 0x7f73b0070230 2023-12-20T06:04:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73b568c640 1 -- 172.21.15.32:0/2180699196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f73b004e000 con 0x7f73b0070230 2023-12-20T06:04:06.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.062+0000 7f73a77fe640 1 -- 172.21.15.32:0/2180699196 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f73a0012c40 con 0x7f73b0070230 2023-12-20T06:04:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.066+0000 7f73a77fe640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7390065ca0 0x7f7390068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.066+0000 7f73a77fe640 1 -- 172.21.15.32:0/2180699196 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5859+0+0 (secure 0 0 0) 0x7f73a0083b10 con 0x7f73b0070230 2023-12-20T06:04:06.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.066+0000 7f73a7fff640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7390065ca0 0x7f7390068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.066+0000 7f73a7fff640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7390065ca0 0x7f7390068160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f73980ae000 tx=0x7f73980b5040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:06.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.070+0000 7f73a77fe640 1 -- 172.21.15.32:0/2180699196 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f73a0051050 con 0x7f73b0070230 2023-12-20T06:04:06.227 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.226+0000 7f73b568c640 1 -- 172.21.15.32:0/2180699196 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f73b00709c0 con 0x7f73b0070230 2023-12-20T06:04:06.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.230+0000 7f73a77fe640 1 -- 172.21.15.32:0/2180699196 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v80) v1 ==== 115+0+0 (secure 0 0 0) 0x7f73a0007380 con 0x7f73b0070230 2023-12-20T06:04:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 -- 172.21.15.32:0/2180699196 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7390065ca0 msgr2=0x7f7390068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7390065ca0 0x7f7390068160 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f73980ae000 tx=0x7f73980b5040 comp rx=0 tx=0).stop 2023-12-20T06:04:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 -- 172.21.15.32:0/2180699196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73b0070230 msgr2=0x7f73b00c4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73b0070230 0x7f73b00c4830 secure :-1 s=READY pgs=2434 cs=0 l=1 rev1=1 crypto rx=0x7f73a000d5f0 tx=0x7f73a000dab0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 -- 172.21.15.32:0/2180699196 shutdown_connections 2023-12-20T06:04:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7390065ca0 0x7f7390068160 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f73b015abb0 0x7f73b0140e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f73b00b0cb0 0x7f73b0140920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 --2- 172.21.15.32:0/2180699196 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f73b0070230 0x7f73b00c4830 unknown :-1 s=CLOSED pgs=2434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 -- 172.21.15.32:0/2180699196 >> 172.21.15.32:0/2180699196 conn(0x7f73b006a510 msgr2=0x7f73b01548b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:06.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 -- 172.21.15.32:0/2180699196 shutdown_connections 2023-12-20T06:04:06.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.234+0000 7f73b568c640 1 -- 172.21.15.32:0/2180699196 wait complete. 2023-12-20T06:04:06.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T06:04:06.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.398+0000 7fea6c255640 1 Processor -- start 2023-12-20T06:04:06.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.398+0000 7fea6c255640 1 -- start start 2023-12-20T06:04:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.402+0000 7fea6c255640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea6415abb0 0x7fea640caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.402+0000 7fea6c255640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6415b6a0 con 0x7fea64070230 2023-12-20T06:04:06.402 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.402+0000 7fea6c255640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fea6406d8c0 con 0x7fea640b0cb0 2023-12-20T06:04:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.402+0000 7fea6c255640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fea6406e160 con 0x7fea6415abb0 2023-12-20T06:04:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.402+0000 7fea69ff1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fea6415abb0 0x7fea640caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.402+0000 7fea6a7f2640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fea64070230 0x7fea6406c3e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36242/0 (socket says 172.21.15.32:36242) 2023-12-20T06:04:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.402+0000 7fea6a7f2640 1 -- 172.21.15.32:0/1138945643 learned_addr learned my addr 172.21.15.32:0/1138945643 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.402+0000 7fea69ff1640 1 -- 172.21.15.32:0/1138945643 >> v1:172.21.15.39:6789/0 conn(0x7fea640b0cb0 legacy=0x7fea6415bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.402+0000 7fea69ff1640 1 -- 172.21.15.32:0/1138945643 >> v1:172.21.15.32:6789/0 conn(0x7fea64070230 legacy=0x7fea6406c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.402+0000 7fea68fef640 1 -- 172.21.15.32:0/1138945643 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1344403466 0 0) 0x7fea6415b6a0 con 0x7fea64070230 2023-12-20T06:04:06.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.402+0000 7fea69ff1640 1 -- 172.21.15.32:0/1138945643 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea6406dba0 con 0x7fea6415abb0 2023-12-20T06:04:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea69ff1640 1 --2- 172.21.15.32:0/1138945643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6415abb0 0x7fea640caaf0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fea600097c0 tx=0x7fea6001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e4bc95209eed0ac server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:06.405 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea68fef640 1 -- 172.21.15.32:0/1138945643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea600ab680 con 0x7fea6415abb0 2023-12-20T06:04:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea68fef640 1 -- 172.21.15.32:0/1138945643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea600ab7e0 con 0x7fea6415abb0 2023-12-20T06:04:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea68fef640 1 -- 172.21.15.32:0/1138945643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea600abb70 con 0x7fea6415abb0 2023-12-20T06:04:06.406 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea6c255640 1 -- 172.21.15.32:0/1138945643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6415abb0 msgr2=0x7fea640caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea6c255640 1 --2- 172.21.15.32:0/1138945643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6415abb0 0x7fea640caaf0 secure :-1 s=READY pgs=2436 cs=0 l=1 rev1=1 crypto rx=0x7fea600097c0 tx=0x7fea6001c480 comp rx=0 tx=0).stop 2023-12-20T06:04:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea6c255640 1 -- 172.21.15.32:0/1138945643 shutdown_connections 2023-12-20T06:04:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea6c255640 1 --2- 172.21.15.32:0/1138945643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea6415abb0 0x7fea640caaf0 unknown :-1 s=CLOSED pgs=2436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea6c255640 1 -- 172.21.15.32:0/1138945643 >> 172.21.15.32:0/1138945643 conn(0x7fea6406a510 msgr2=0x7fea640ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea6c255640 1 -- 172.21.15.32:0/1138945643 shutdown_connections 2023-12-20T06:04:06.407 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea6c255640 1 -- 172.21.15.32:0/1138945643 wait complete. 2023-12-20T06:04:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea6c255640 1 Processor -- start 2023-12-20T06:04:06.408 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.406+0000 7fea6c255640 1 -- start start 2023-12-20T06:04:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea6c255640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea64070230 0x7fea640b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea6c255640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea640b0cb0 0x7fea640b2fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea6c255640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea6415abb0 0x7fea6413d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea6c255640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea64157120 con 0x7fea640b0cb0 2023-12-20T06:04:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea6c255640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fea64156e10 con 0x7fea6415abb0 2023-12-20T06:04:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea6c255640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fea64159ca0 con 0x7fea64070230 2023-12-20T06:04:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea697f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea640b0cb0 0x7fea640b2fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.411 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea697f0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea640b0cb0 0x7fea640b2fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58444/0 (socket says 172.21.15.32:58444) 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea697f0640 1 -- 172.21.15.32:0/2116540717 learned_addr learned my addr 172.21.15.32:0/2116540717 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea69ff1640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea64070230 0x7fea640b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea697f0640 1 -- 172.21.15.32:0/2116540717 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea64070230 msgr2=0x7fea640b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea697f0640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea64070230 0x7fea640b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea697f0640 1 -- 172.21.15.32:0/2116540717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea6415abb0 msgr2=0x7fea6413d2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea6a7f2640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea6415abb0 0x7fea6413d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea697f0640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea6415abb0 0x7fea6413d2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea697f0640 1 -- 172.21.15.32:0/2116540717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fea60009420 con 0x7fea640b0cb0 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea6a7f2640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea6415abb0 0x7fea6413d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea697f0640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea640b0cb0 0x7fea640b2fb0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fea5400c500 tx=0x7fea5400c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea5a7fc640 1 -- 172.21.15.32:0/2116540717 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea540052d0 con 0x7fea640b0cb0 2023-12-20T06:04:06.412 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea5a7fc640 1 -- 172.21.15.32:0/2116540717 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fea54005430 con 0x7fea640b0cb0 2023-12-20T06:04:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea5a7fc640 1 -- 172.21.15.32:0/2116540717 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fea54012830 con 0x7fea640b0cb0 2023-12-20T06:04:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea6c255640 1 -- 172.21.15.32:0/2116540717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fea64159f80 con 0x7fea640b0cb0 2023-12-20T06:04:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea6c255640 1 -- 172.21.15.32:0/2116540717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fea64150110 con 0x7fea640b0cb0 2023-12-20T06:04:06.413 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea6c255640 1 -- 172.21.15.32:0/2116540717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fea6404e000 con 0x7fea640b0cb0 2023-12-20T06:04:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.410+0000 7fea5a7fc640 1 -- 172.21.15.32:0/2116540717 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fea54012990 con 0x7fea640b0cb0 2023-12-20T06:04:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.414+0000 7fea5a7fc640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fea48065ca0 0x7fea48068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.414+0000 7fea5a7fc640 1 -- 172.21.15.32:0/2116540717 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5859+0+0 (secure 0 0 0) 0x7fea54083130 con 0x7fea640b0cb0 2023-12-20T06:04:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.414+0000 7fea69ff1640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fea48065ca0 0x7fea48068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.414 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.414+0000 7fea69ff1640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fea48065ca0 0x7fea48068160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fea600af000 tx=0x7fea600a7000 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:06.417 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.418+0000 7fea5a7fc640 1 -- 172.21.15.32:0/2116540717 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fea54050700 con 0x7fea640b0cb0 2023-12-20T06:04:06.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.574+0000 7fea6c255640 1 -- 172.21.15.32:0/2116540717 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7fea64157950 con 0x7fea640b0cb0 2023-12-20T06:04:06.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea5a7fc640 1 -- 172.21.15.32:0/2116540717 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v81) v1 ==== 104+0+0 (secure 0 0 0) 0x7fea54053d20 con 0x7fea640b0cb0 2023-12-20T06:04:06.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 -- 172.21.15.32:0/2116540717 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fea48065ca0 msgr2=0x7fea48068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fea48065ca0 0x7fea48068160 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fea600af000 tx=0x7fea600a7000 comp rx=0 tx=0).stop 2023-12-20T06:04:06.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 -- 172.21.15.32:0/2116540717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea640b0cb0 msgr2=0x7fea640b2fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea640b0cb0 0x7fea640b2fb0 secure :-1 s=READY pgs=2437 cs=0 l=1 rev1=1 crypto rx=0x7fea5400c500 tx=0x7fea5400c9c0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 -- 172.21.15.32:0/2116540717 shutdown_connections 2023-12-20T06:04:06.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fea48065ca0 0x7fea48068160 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fea6415abb0 0x7fea6413d2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fea640b0cb0 0x7fea640b2fb0 unknown :-1 s=CLOSED pgs=2437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 --2- 172.21.15.32:0/2116540717 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fea64070230 0x7fea640b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 -- 172.21.15.32:0/2116540717 >> 172.21.15.32:0/2116540717 conn(0x7fea6406a510 msgr2=0x7fea64155300 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:06.584 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 -- 172.21.15.32:0/2116540717 shutdown_connections 2023-12-20T06:04:06.585 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.582+0000 7fea6c255640 1 -- 172.21.15.32:0/2116540717 wait complete. 2023-12-20T06:04:06.603 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument' 2023-12-20T06:04:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.746+0000 7fb9ef066640 1 Processor -- start 2023-12-20T06:04:06.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.746+0000 7fb9ef066640 1 -- start start 2023-12-20T06:04:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.746+0000 7fb9ef066640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9e806ce60 0x7fb9e815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.746+0000 7fb9ef066640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e815bbd0 con 0x7fb9e815f110 2023-12-20T06:04:06.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.746+0000 7fb9ef066640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb9e815beb0 con 0x7fb9e80af5e0 2023-12-20T06:04:06.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.746+0000 7fb9ef066640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e815e910 con 0x7fb9e806ce60 2023-12-20T06:04:06.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.746+0000 7fb9ed603640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb9e815f110 0x7fb9e815c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36250/0 (socket says 172.21.15.32:36250) 2023-12-20T06:04:06.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.746+0000 7fb9ed603640 1 -- 172.21.15.32:0/3758463017 learned_addr learned my addr 172.21.15.32:0/3758463017 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:06.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.746+0000 7fb9e7fff640 1 --2- 172.21.15.32:0/3758463017 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9e806ce60 0x7fb9e815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.750+0000 7fb9e77fe640 1 -- 172.21.15.32:0/3758463017 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2761676167 0 0) 0x7fb9e815bbd0 con 0x7fb9e815f110 2023-12-20T06:04:06.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.750+0000 7fb9e77fe640 1 -- 172.21.15.32:0/3758463017 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9d8003530 con 0x7fb9e815f110 2023-12-20T06:04:06.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.750+0000 7fb9e77fe640 1 -- 172.21.15.32:0/3758463017 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2184409087 0 0) 0x7fb9e815beb0 con 0x7fb9e80af5e0 2023-12-20T06:04:06.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.750+0000 7fb9e77fe640 1 -- 172.21.15.32:0/3758463017 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb9e815bbd0 con 0x7fb9e80af5e0 2023-12-20T06:04:06.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.750+0000 7fb9e7fff640 1 -- 172.21.15.32:0/3758463017 >> v1:172.21.15.39:6789/0 conn(0x7fb9e80af5e0 legacy=0x7fb9e80af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.750+0000 7fb9e7fff640 1 -- 172.21.15.32:0/3758463017 >> v1:172.21.15.32:6789/0 conn(0x7fb9e815f110 legacy=0x7fb9e815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.750+0000 7fb9e7fff640 1 -- 172.21.15.32:0/3758463017 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9e815ebf0 con 0x7fb9e806ce60 2023-12-20T06:04:06.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.750+0000 7fb9e7fff640 1 --2- 172.21.15.32:0/3758463017 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9e806ce60 0x7fb9e815b280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40097f0 tx=0x7fb9d401c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5d5fa0b9ece50a24 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:06.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.750+0000 7fb9e77fe640 1 -- 172.21.15.32:0/3758463017 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e815b9d0 con 0x7fb9e806ce60 2023-12-20T06:04:06.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.750+0000 7fb9e77fe640 1 -- 172.21.15.32:0/3758463017 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9d40ac630 con 0x7fb9e806ce60 2023-12-20T06:04:06.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.750+0000 7fb9e77fe640 1 -- 172.21.15.32:0/3758463017 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9d40ac9c0 con 0x7fb9e806ce60 2023-12-20T06:04:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 -- 172.21.15.32:0/3758463017 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9e806ce60 msgr2=0x7fb9e815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 --2- 172.21.15.32:0/3758463017 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9e806ce60 0x7fb9e815b280 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fb9d40097f0 tx=0x7fb9d401c430 comp rx=0 tx=0).stop 2023-12-20T06:04:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 -- 172.21.15.32:0/3758463017 shutdown_connections 2023-12-20T06:04:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 --2- 172.21.15.32:0/3758463017 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9e806ce60 0x7fb9e815b280 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 -- 172.21.15.32:0/3758463017 >> 172.21.15.32:0/3758463017 conn(0x7fb9e806a320 msgr2=0x7fb9e80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 -- 172.21.15.32:0/3758463017 shutdown_connections 2023-12-20T06:04:06.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 -- 172.21.15.32:0/3758463017 wait complete. 2023-12-20T06:04:06.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 Processor -- start 2023-12-20T06:04:06.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 -- start start 2023-12-20T06:04:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9e806ce60 0x7fb9e80c8070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9e80af5e0 0x7fb9e81654b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9e815f110 0x7fb9e81659f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e80b1aa0 con 0x7fb9e815f110 2023-12-20T06:04:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e8166010 con 0x7fb9e80af5e0 2023-12-20T06:04:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ef066640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb9e81662f0 con 0x7fb9e806ce60 2023-12-20T06:04:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ed603640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9e815f110 0x7fb9e81659f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ed603640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9e815f110 0x7fb9e81659f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58460/0 (socket says 172.21.15.32:58460) 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ed603640 1 -- 172.21.15.32:0/3450454546 learned_addr learned my addr 172.21.15.32:0/3450454546 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9e7fff640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9e80af5e0 0x7fb9e81654b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.754+0000 7fb9ece02640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9e806ce60 0x7fb9e80c8070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9ed603640 1 -- 172.21.15.32:0/3450454546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9e806ce60 msgr2=0x7fb9e80c8070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9ed603640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9e806ce60 0x7fb9e80c8070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9ed603640 1 -- 172.21.15.32:0/3450454546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9e80af5e0 msgr2=0x7fb9e81654b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9ed603640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9e80af5e0 0x7fb9e81654b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9ed603640 1 -- 172.21.15.32:0/3450454546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb9d4009420 con 0x7fb9e815f110 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9e7fff640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9e80af5e0 0x7fb9e81654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9ed603640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9e815f110 0x7fb9e81659f0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000caf0 tx=0x7fb9e000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9e57fa640 1 -- 172.21.15.32:0/3450454546 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e000aca0 con 0x7fb9e815f110 2023-12-20T06:04:06.759 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9e57fa640 1 -- 172.21.15.32:0/3450454546 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb9e000ae00 con 0x7fb9e815f110 2023-12-20T06:04:06.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9e57fa640 1 -- 172.21.15.32:0/3450454546 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb9e0012620 con 0x7fb9e815f110 2023-12-20T06:04:06.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9ef066640 1 -- 172.21.15.32:0/3450454546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb9e81665d0 con 0x7fb9e815f110 2023-12-20T06:04:06.760 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9ef066640 1 -- 172.21.15.32:0/3450454546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb9e80c85b0 con 0x7fb9e815f110 2023-12-20T06:04:06.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9ef066640 1 -- 172.21.15.32:0/3450454546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb9e80b1c10 con 0x7fb9e815f110 2023-12-20T06:04:06.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9e57fa640 1 -- 172.21.15.32:0/3450454546 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb9e0007070 con 0x7fb9e815f110 2023-12-20T06:04:06.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9e57fa640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb9cc065ca0 0x7fb9cc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:06.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.758+0000 7fb9e57fa640 1 -- 172.21.15.32:0/3450454546 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5859+0+0 (secure 0 0 0) 0x7fb9e0083630 con 0x7fb9e815f110 2023-12-20T06:04:06.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.762+0000 7fb9ece02640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb9cc065ca0 0x7fb9cc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:06.761 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.762+0000 7fb9ece02640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb9cc065ca0 0x7fb9cc068160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc001240 tx=0x7fb9dc009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:06.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.762+0000 7fb9e57fa640 1 -- 172.21.15.32:0/3450454546 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb9e0050be0 con 0x7fb9e815f110 2023-12-20T06:04:06.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:06.974+0000 7fb9ef066640 1 -- 172.21.15.32:0/3450454546 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]} v 0) v1 -- 0x7fb9e804e000 con 0x7fb9e815f110 2023-12-20T06:04:07.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.526+0000 7fb9e57fa640 1 -- 172.21.15.32:0/3450454546 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesMandatoryFSArgument.test_snap_schedule_without_fs_argument"]}]=0 v3128) v1 ==== 187+0+0 (secure 0 0 0) 0x7fb9e0054200 con 0x7fb9e815f110 2023-12-20T06:04:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 -- 172.21.15.32:0/3450454546 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb9cc065ca0 msgr2=0x7fb9cc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb9cc065ca0 0x7fb9cc068160 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fb9dc001240 tx=0x7fb9dc009040 comp rx=0 tx=0).stop 2023-12-20T06:04:07.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 -- 172.21.15.32:0/3450454546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9e815f110 msgr2=0x7fb9e81659f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9e815f110 0x7fb9e81659f0 secure :-1 s=READY pgs=2439 cs=0 l=1 rev1=1 crypto rx=0x7fb9e000caf0 tx=0x7fb9e000a060 comp rx=0 tx=0).stop 2023-12-20T06:04:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 -- 172.21.15.32:0/3450454546 shutdown_connections 2023-12-20T06:04:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb9cc065ca0 0x7fb9cc068160 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb9e815f110 0x7fb9e81659f0 unknown :-1 s=CLOSED pgs=2439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb9e80af5e0 0x7fb9e81654b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 --2- 172.21.15.32:0/3450454546 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb9e806ce60 0x7fb9e80c8070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 -- 172.21.15.32:0/3450454546 >> 172.21.15.32:0/3450454546 conn(0x7fb9e806a320 msgr2=0x7fb9e8156ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 -- 172.21.15.32:0/3450454546 shutdown_connections 2023-12-20T06:04:07.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.530+0000 7fb9ef066640 1 -- 172.21.15.32:0/3450454546 wait complete. 2023-12-20T06:04:07.555 INFO:tasks.cephfs_test_runner:Test command fails without --fs argument in presence of multiple fs ... ok 2023-12-20T06:04:07.556 INFO:tasks.cephfs_test_runner:Starting test: test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) Test the correctness of snap directory name 2023-12-20T06:04:07.556 INFO:tasks.cephfs_test_runner:test_snap_dir_name (tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir) 2023-12-20T06:04:07.556 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-12-20T06:04:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.702+0000 7f7f2e360640 1 Processor -- start 2023-12-20T06:04:07.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.702+0000 7f7f2e360640 1 -- start start 2023-12-20T06:04:07.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f2e360640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f2806ce60 0x7f7f2815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:07.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f2e360640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2815bbd0 con 0x7f7f280af5e0 2023-12-20T06:04:07.706 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f2e360640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7f2815beb0 con 0x7f7f2815f110 2023-12-20T06:04:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f2e360640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f2815e910 con 0x7f7f2806ce60 2023-12-20T06:04:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f27fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7f280af5e0 0x7f7f280af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:36264/0 (socket says 172.21.15.32:36264) 2023-12-20T06:04:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f27fff640 1 -- 172.21.15.32:0/2718269619 learned_addr learned my addr 172.21.15.32:0/2718269619 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f277fe640 1 --2- 172.21.15.32:0/2718269619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f2806ce60 0x7f7f2815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f26ffd640 1 -- 172.21.15.32:0/2718269619 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1796760638 0 0) 0x7f7f2815bbd0 con 0x7f7f280af5e0 2023-12-20T06:04:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f26ffd640 1 -- 172.21.15.32:0/2718269619 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f10003530 con 0x7f7f280af5e0 2023-12-20T06:04:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f26ffd640 1 -- 172.21.15.32:0/2718269619 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4060949219 0 0) 0x7f7f2815beb0 con 0x7f7f2815f110 2023-12-20T06:04:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f26ffd640 1 -- 172.21.15.32:0/2718269619 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7f2815bbd0 con 0x7f7f2815f110 2023-12-20T06:04:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f277fe640 1 -- 172.21.15.32:0/2718269619 >> v1:172.21.15.39:6789/0 conn(0x7f7f2815f110 legacy=0x7f7f2815c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:07.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f26ffd640 1 -- 172.21.15.32:0/2718269619 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 935504674 0 0) 0x7f7f10003530 con 0x7f7f280af5e0 2023-12-20T06:04:07.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f277fe640 1 -- 172.21.15.32:0/2718269619 >> v1:172.21.15.32:6789/0 conn(0x7f7f280af5e0 legacy=0x7f7f280af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:07.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.706+0000 7f7f277fe640 1 -- 172.21.15.32:0/2718269619 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f2815ebf0 con 0x7f7f2806ce60 2023-12-20T06:04:07.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f277fe640 1 --2- 172.21.15.32:0/2718269619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f2806ce60 0x7f7f2815b280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0097f0 tx=0x7f7f1c01c350 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b50843240ec19b4a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:07.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f26ffd640 1 -- 172.21.15.32:0/2718269619 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f2815b9d0 con 0x7f7f2806ce60 2023-12-20T06:04:07.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f26ffd640 1 -- 172.21.15.32:0/2718269619 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c0a7050 con 0x7f7f2806ce60 2023-12-20T06:04:07.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f26ffd640 1 -- 172.21.15.32:0/2718269619 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f18001800 con 0x7f7f2806ce60 2023-12-20T06:04:07.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f2e360640 1 -- 172.21.15.32:0/2718269619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f2806ce60 msgr2=0x7f7f2815b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:07.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f2e360640 1 --2- 172.21.15.32:0/2718269619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f2806ce60 0x7f7f2815b280 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c0097f0 tx=0x7f7f1c01c350 comp rx=0 tx=0).stop 2023-12-20T06:04:07.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f2e360640 1 -- 172.21.15.32:0/2718269619 shutdown_connections 2023-12-20T06:04:07.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f2e360640 1 --2- 172.21.15.32:0/2718269619 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f2806ce60 0x7f7f2815b280 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:07.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f2e360640 1 -- 172.21.15.32:0/2718269619 >> 172.21.15.32:0/2718269619 conn(0x7f7f2806a320 msgr2=0x7f7f280ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:07.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f2e360640 1 -- 172.21.15.32:0/2718269619 shutdown_connections 2023-12-20T06:04:07.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f2e360640 1 -- 172.21.15.32:0/2718269619 wait complete. 2023-12-20T06:04:07.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f2e360640 1 Processor -- start 2023-12-20T06:04:07.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.710+0000 7f7f2e360640 1 -- start start 2023-12-20T06:04:07.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f2e360640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f2806ce60 0x7f7f28146ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:07.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f2e360640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f280af5e0 0x7f7f280cb170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:07.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f2e360640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f2815f110 0x7f7f280cb6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:07.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f2e360640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f28163f00 con 0x7f7f280af5e0 2023-12-20T06:04:07.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f2e360640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7f281641e0 con 0x7f7f2815f110 2023-12-20T06:04:07.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f2e360640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7f281644c0 con 0x7f7f2806ce60 2023-12-20T06:04:07.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f277fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f280af5e0 0x7f7f280cb170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:07.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f277fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f280af5e0 0x7f7f280cb170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58472/0 (socket says 172.21.15.32:58472) 2023-12-20T06:04:07.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f277fe640 1 -- 172.21.15.32:0/4269687036 learned_addr learned my addr 172.21.15.32:0/4269687036 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:07.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f27fff640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f2806ce60 0x7f7f28146ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:07.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f2c8fd640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f2815f110 0x7f7f280cb6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f277fe640 1 -- 172.21.15.32:0/4269687036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f2806ce60 msgr2=0x7f7f28146ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f277fe640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f2806ce60 0x7f7f28146ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f277fe640 1 -- 172.21.15.32:0/4269687036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f2815f110 msgr2=0x7f7f280cb6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f277fe640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f2815f110 0x7f7f280cb6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f277fe640 1 -- 172.21.15.32:0/4269687036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7f1c009420 con 0x7f7f280af5e0 2023-12-20T06:04:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f277fe640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f280af5e0 0x7f7f280cb170 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c018f60 tx=0x7f7f1c0190f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f24ff9640 1 -- 172.21.15.32:0/4269687036 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c004020 con 0x7f7f280af5e0 2023-12-20T06:04:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f24ff9640 1 -- 172.21.15.32:0/4269687036 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7f1c004180 con 0x7f7f280af5e0 2023-12-20T06:04:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f24ff9640 1 -- 172.21.15.32:0/4269687036 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7f1c0044f0 con 0x7f7f280af5e0 2023-12-20T06:04:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f2e360640 1 -- 172.21.15.32:0/4269687036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7f281647a0 con 0x7f7f280af5e0 2023-12-20T06:04:07.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f2e360640 1 -- 172.21.15.32:0/4269687036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7f28147020 con 0x7f7f280af5e0 2023-12-20T06:04:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.714+0000 7f7f2e360640 1 -- 172.21.15.32:0/4269687036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7f2804e000 con 0x7f7f280af5e0 2023-12-20T06:04:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.718+0000 7f7f24ff9640 1 -- 172.21.15.32:0/4269687036 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f7f1c0b2040 con 0x7f7f280af5e0 2023-12-20T06:04:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.718+0000 7f7f24ff9640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f14065ca0 0x7f7f14068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.718+0000 7f7f24ff9640 1 -- 172.21.15.32:0/4269687036 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5859+0+0 (secure 0 0 0) 0x7f7f1c11cad0 con 0x7f7f280af5e0 2023-12-20T06:04:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.718+0000 7f7f27fff640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f14065ca0 0x7f7f14068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:07.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.718+0000 7f7f27fff640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f14065ca0 0x7f7f14068160 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f18001160 tx=0x7f7f18001b80 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:07.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.722+0000 7f7f24ff9640 1 -- 172.21.15.32:0/4269687036 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7f1c0e9fd0 con 0x7f7f280af5e0 2023-12-20T06:04:07.938 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:07.938+0000 7f7f2e360640 1 -- 172.21.15.32:0/4269687036 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f7f28164c50 con 0x7f7f280af5e0 2023-12-20T06:04:08.535 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.534+0000 7f7f24ff9640 1 -- 172.21.15.32:0/4269687036 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Starting test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3129) v1 ==== 158+0+0 (secure 0 0 0) 0x7f7f1c0b0030 con 0x7f7f280af5e0 2023-12-20T06:04:08.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 -- 172.21.15.32:0/4269687036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f14065ca0 msgr2=0x7f7f14068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:08.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f14065ca0 0x7f7f14068160 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f7f18001160 tx=0x7f7f18001b80 comp rx=0 tx=0).stop 2023-12-20T06:04:08.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 -- 172.21.15.32:0/4269687036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f280af5e0 msgr2=0x7f7f280cb170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f280af5e0 0x7f7f280cb170 secure :-1 s=READY pgs=2441 cs=0 l=1 rev1=1 crypto rx=0x7f7f1c018f60 tx=0x7f7f1c0190f0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 -- 172.21.15.32:0/4269687036 shutdown_connections 2023-12-20T06:04:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7f14065ca0 0x7f7f14068160 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7f2815f110 0x7f7f280cb6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7f280af5e0 0x7f7f280cb170 unknown :-1 s=CLOSED pgs=2441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 --2- 172.21.15.32:0/4269687036 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7f2806ce60 0x7f7f28146ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 -- 172.21.15.32:0/4269687036 >> 172.21.15.32:0/4269687036 conn(0x7f7f2806a320 msgr2=0x7f7f2815a1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 -- 172.21.15.32:0/4269687036 shutdown_connections 2023-12-20T06:04:08.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.538+0000 7f7f2e360640 1 -- 172.21.15.32:0/4269687036 wait complete. 2023-12-20T06:04:08.562 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mon mon_allow_pool_delete True 2023-12-20T06:04:08.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efe05360640 1 Processor -- start 2023-12-20T06:04:08.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efe05360640 1 -- start start 2023-12-20T06:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efe05360640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe00070490 0x7efe0015fcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efe05360640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0006c480 con 0x7efe0006d240 2023-12-20T06:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efe05360640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7efe0006c760 con 0x7efe00070870 2023-12-20T06:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efe05360640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7efe0006ca40 con 0x7efe00070490 2023-12-20T06:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efdfeffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe00070490 0x7efe0015fcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efdfeffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7efe00070490 0x7efe0015fcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58488/0 (socket says 172.21.15.32:58488) 2023-12-20T06:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efdfeffd640 1 -- 172.21.15.32:0/3353400246 learned_addr learned my addr 172.21.15.32:0/3353400246 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:08.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efdfeffd640 1 -- 172.21.15.32:0/3353400246 >> v1:172.21.15.39:6789/0 conn(0x7efe00070870 legacy=0x7efe000cc3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efdfeffd640 1 -- 172.21.15.32:0/3353400246 >> v1:172.21.15.32:6789/0 conn(0x7efe0006d240 legacy=0x7efe000ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:08.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.714+0000 7efdfeffd640 1 -- 172.21.15.32:0/3353400246 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efe0006cd20 con 0x7efe00070490 2023-12-20T06:04:08.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.718+0000 7efdfeffd640 1 --2- 172.21.15.32:0/3353400246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe00070490 0x7efe0015fcd0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009870 tx=0x7efdf001c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bbb8335d058a3da0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.718+0000 7efdfe7fc640 1 -- 172.21.15.32:0/3353400246 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00ab680 con 0x7efe00070490 2023-12-20T06:04:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.718+0000 7efdfe7fc640 1 -- 172.21.15.32:0/3353400246 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf00ab7e0 con 0x7efe00070490 2023-12-20T06:04:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.718+0000 7efdfe7fc640 1 -- 172.21.15.32:0/3353400246 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf00abb70 con 0x7efe00070490 2023-12-20T06:04:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.718+0000 7efe05360640 1 -- 172.21.15.32:0/3353400246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe00070490 msgr2=0x7efe0015fcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.718+0000 7efe05360640 1 --2- 172.21.15.32:0/3353400246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe00070490 0x7efe0015fcd0 secure :-1 s=READY pgs=2443 cs=0 l=1 rev1=1 crypto rx=0x7efdf0009870 tx=0x7efdf001c480 comp rx=0 tx=0).stop 2023-12-20T06:04:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.718+0000 7efe05360640 1 -- 172.21.15.32:0/3353400246 shutdown_connections 2023-12-20T06:04:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.718+0000 7efe05360640 1 --2- 172.21.15.32:0/3353400246 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe00070490 0x7efe0015fcd0 unknown :-1 s=CLOSED pgs=2443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.718+0000 7efe05360640 1 -- 172.21.15.32:0/3353400246 >> 172.21.15.32:0/3353400246 conn(0x7efe0006a510 msgr2=0x7efe000ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.718+0000 7efe05360640 1 -- 172.21.15.32:0/3353400246 shutdown_connections 2023-12-20T06:04:08.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.718+0000 7efe05360640 1 -- 172.21.15.32:0/3353400246 wait complete. 2023-12-20T06:04:08.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efe05360640 1 Processor -- start 2023-12-20T06:04:08.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efe05360640 1 -- start start 2023-12-20T06:04:08.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efe05360640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe0006d240 0x7efe0014b6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:08.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efe05360640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe00070490 0x7efe000c8130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efe05360640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe00070870 0x7efe000c5a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efe05360640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe00156f70 con 0x7efe00070870 2023-12-20T06:04:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efe05360640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7efe00156360 con 0x7efe00070490 2023-12-20T06:04:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efe05360640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7efe00156640 con 0x7efe0006d240 2023-12-20T06:04:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdff7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe00070870 0x7efe000c5a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdff7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe00070870 0x7efe000c5a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:58496/0 (socket says 172.21.15.32:58496) 2023-12-20T06:04:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdff7fe640 1 -- 172.21.15.32:0/559354684 learned_addr learned my addr 172.21.15.32:0/559354684 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdf7fff640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe00070490 0x7efe000c8130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:08.725 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdfeffd640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe0006d240 0x7efe0014b6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdff7fe640 1 -- 172.21.15.32:0/559354684 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe0006d240 msgr2=0x7efe0014b6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdff7fe640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe0006d240 0x7efe0014b6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdff7fe640 1 -- 172.21.15.32:0/559354684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe00070490 msgr2=0x7efe000c8130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdff7fe640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe00070490 0x7efe000c8130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdff7fe640 1 -- 172.21.15.32:0/559354684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7efdf00094d0 con 0x7efe00070870 2023-12-20T06:04:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdff7fe640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe00070870 0x7efe000c5a60 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c730 tx=0x7efdf800cbf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdf77fe640 1 -- 172.21.15.32:0/559354684 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf8003b80 con 0x7efe00070870 2023-12-20T06:04:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdf77fe640 1 -- 172.21.15.32:0/559354684 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7efdf8003ce0 con 0x7efe00070870 2023-12-20T06:04:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efe05360640 1 -- 172.21.15.32:0/559354684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7efe00156920 con 0x7efe00070870 2023-12-20T06:04:08.726 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efdf77fe640 1 -- 172.21.15.32:0/559354684 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7efdf800aa30 con 0x7efe00070870 2023-12-20T06:04:08.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efe05360640 1 -- 172.21.15.32:0/559354684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7efe0014bc20 con 0x7efe00070870 2023-12-20T06:04:08.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.722+0000 7efe05360640 1 -- 172.21.15.32:0/559354684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7efe001570e0 con 0x7efe00070870 2023-12-20T06:04:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.726+0000 7efdf77fe640 1 -- 172.21.15.32:0/559354684 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7efdf800ab90 con 0x7efe00070870 2023-12-20T06:04:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.726+0000 7efdf77fe640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efde0065ca0 0x7efde0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.726+0000 7efdf77fe640 1 -- 172.21.15.32:0/559354684 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5859+0+0 (secure 0 0 0) 0x7efdf8083350 con 0x7efe00070870 2023-12-20T06:04:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.726+0000 7efdfeffd640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efde0065ca0 0x7efde0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:08.728 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.726+0000 7efdfeffd640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efde0065ca0 0x7efde0068160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efdf00af000 tx=0x7efdf00b6040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:08.731 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.730+0000 7efdf77fe640 1 -- 172.21.15.32:0/559354684 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7efdf8050920 con 0x7efe00070870 2023-12-20T06:04:08.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.890+0000 7efe05360640 1 -- 172.21.15.32:0/559354684 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_allow_pool_delete}] v 0) v1 -- 0x7efe0004e000 con 0x7efe00070870 2023-12-20T06:04:08.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efdf77fe640 1 -- 172.21.15.32:0/559354684 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mon_allow_pool_delete}]=0 v82)=0 v82) v1 ==== 122+0+0 (secure 0 0 0) 0x7efdf8053f40 con 0x7efe00070870 2023-12-20T06:04:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 -- 172.21.15.32:0/559354684 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efde0065ca0 msgr2=0x7efde0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efde0065ca0 0x7efde0068160 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7efdf00af000 tx=0x7efdf00b6040 comp rx=0 tx=0).stop 2023-12-20T06:04:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 -- 172.21.15.32:0/559354684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe00070870 msgr2=0x7efe000c5a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:08.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe00070870 0x7efe000c5a60 secure :-1 s=READY pgs=2444 cs=0 l=1 rev1=1 crypto rx=0x7efdf800c730 tx=0x7efdf800cbf0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 -- 172.21.15.32:0/559354684 shutdown_connections 2023-12-20T06:04:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7efde0065ca0 0x7efde0068160 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7efe00070870 0x7efe000c5a60 unknown :-1 s=CLOSED pgs=2444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7efe00070490 0x7efe000c8130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 --2- 172.21.15.32:0/559354684 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7efe0006d240 0x7efe0014b6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 -- 172.21.15.32:0/559354684 >> 172.21.15.32:0/559354684 conn(0x7efe0006a510 msgr2=0x7efe00070fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 -- 172.21.15.32:0/559354684 shutdown_connections 2023-12-20T06:04:08.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:08.898+0000 7efe05360640 1 -- 172.21.15.32:0/559354684 wait complete. 2023-12-20T06:04:08.919 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T06:04:08.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:04:09.015 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:04:09.121 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T06:04:09.121 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:04:09.121 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:09.132 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:04:09.132 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:04:09.132 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:09.143 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:04:09.143 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:04:09.143 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:09.185 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:04:09.185 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:04:09.185 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:09.199 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:04:09.199 INFO:tasks.cephfs.cephfs_test_case:[, , , ] 2023-12-20T06:04:09.199 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:04:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.370+0000 7f52a0afc640 1 Processor -- start 2023-12-20T06:04:09.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.370+0000 7f52a0afc640 1 -- start start 2023-12-20T06:04:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f52a0afc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f529c1599e0 0x7f529c0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f52a0afc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f529c0b0d00 0x7f529c0701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f52a0afc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f529c0c98c0 con 0x7f529c0c90e0 2023-12-20T06:04:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f52a0afc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c0c9ba0 con 0x7f529c1599e0 2023-12-20T06:04:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f52a0afc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f529c0c9e80 con 0x7f529c0b0d00 2023-12-20T06:04:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f529a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f529c1599e0 0x7f529c0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f529a59c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f529c1599e0 0x7f529c0aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49818/0 (socket says 172.21.15.32:49818) 2023-12-20T06:04:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f529a59c640 1 -- 172.21.15.32:0/2641825519 learned_addr learned my addr 172.21.15.32:0/2641825519 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f529ad9d640 1 --2- 172.21.15.32:0/2641825519 >> v2:172.21.15.39:3300/0 conn(0x7f529c0b0d00 0x7f529c0701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f529a59c640 1 -- 172.21.15.32:0/2641825519 >> v2:172.21.15.39:3300/0 conn(0x7f529c0b0d00 msgr2=0x7f529c0701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f529a59c640 1 --2- 172.21.15.32:0/2641825519 >> v2:172.21.15.39:3300/0 conn(0x7f529c0b0d00 0x7f529c0701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f529a59c640 1 -- 172.21.15.32:0/2641825519 >> v1:172.21.15.39:6789/0 conn(0x7f529c0c90e0 legacy=0x7f529c0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:09.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.374+0000 7f529a59c640 1 -- 172.21.15.32:0/2641825519 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f529c0ca160 con 0x7f529c1599e0 2023-12-20T06:04:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f529a59c640 1 --2- 172.21.15.32:0/2641825519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529c1599e0 0x7f529c0aece0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f529c0b0060 tx=0x7f528c01c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c05b99b67a2876f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f5299d9b640 1 -- 172.21.15.32:0/2641825519 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c0ac600 con 0x7f529c1599e0 2023-12-20T06:04:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f5299d9b640 1 -- 172.21.15.32:0/2641825519 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f528c0a8050 con 0x7f529c1599e0 2023-12-20T06:04:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f5299d9b640 1 -- 172.21.15.32:0/2641825519 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f528c0ac980 con 0x7f529c1599e0 2023-12-20T06:04:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 -- 172.21.15.32:0/2641825519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529c1599e0 msgr2=0x7f529c0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 --2- 172.21.15.32:0/2641825519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529c1599e0 0x7f529c0aece0 secure :-1 s=READY pgs=2445 cs=0 l=1 rev1=1 crypto rx=0x7f529c0b0060 tx=0x7f528c01c490 comp rx=0 tx=0).stop 2023-12-20T06:04:09.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 -- 172.21.15.32:0/2641825519 shutdown_connections 2023-12-20T06:04:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 --2- 172.21.15.32:0/2641825519 >> v2:172.21.15.39:3300/0 conn(0x7f529c0b0d00 0x7f529c0701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 --2- 172.21.15.32:0/2641825519 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529c1599e0 0x7f529c0aece0 unknown :-1 s=CLOSED pgs=2445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 -- 172.21.15.32:0/2641825519 >> 172.21.15.32:0/2641825519 conn(0x7f529c06a320 msgr2=0x7f529c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 -- 172.21.15.32:0/2641825519 shutdown_connections 2023-12-20T06:04:09.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 -- 172.21.15.32:0/2641825519 wait complete. 2023-12-20T06:04:09.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 Processor -- start 2023-12-20T06:04:09.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 -- start start 2023-12-20T06:04:09.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f529c0b0d00 0x7f529c158ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529c0c90e0 0x7f529c14f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f529c1599e0 0x7f529c14db50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c1592c0 con 0x7f529c0c90e0 2023-12-20T06:04:09.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f529c165a80 con 0x7f529c0b0d00 2023-12-20T06:04:09.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.378+0000 7f52a0afc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f529c14e090 con 0x7f529c1599e0 2023-12-20T06:04:09.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5293fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529c0c90e0 0x7f529c14f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5293fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529c0c90e0 0x7f529c14f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49824/0 (socket says 172.21.15.32:49824) 2023-12-20T06:04:09.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5293fff640 1 -- 172.21.15.32:0/1899016483 learned_addr learned my addr 172.21.15.32:0/1899016483 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:09.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f529a59c640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f529c0b0d00 0x7f529c158ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5293fff640 1 -- 172.21.15.32:0/1899016483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f529c1599e0 msgr2=0x7f529c14db50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f529ad9d640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f529c1599e0 0x7f529c14db50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5293fff640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f529c1599e0 0x7f529c14db50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5293fff640 1 -- 172.21.15.32:0/1899016483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f529c0b0d00 msgr2=0x7f529c158ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5293fff640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f529c0b0d00 0x7f529c158ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5293fff640 1 -- 172.21.15.32:0/1899016483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f528c009480 con 0x7f529c0c90e0 2023-12-20T06:04:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f529a59c640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f529c0b0d00 0x7f529c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f529ad9d640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f529c1599e0 0x7f529c14db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5293fff640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529c0c90e0 0x7f529c14f6d0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f528400c5b0 tx=0x7f528400ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5292ffd640 1 -- 172.21.15.32:0/1899016483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284012830 con 0x7f529c0c90e0 2023-12-20T06:04:09.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5292ffd640 1 -- 172.21.15.32:0/1899016483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5284012990 con 0x7f529c0c90e0 2023-12-20T06:04:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5292ffd640 1 -- 172.21.15.32:0/1899016483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5284012d40 con 0x7f529c0c90e0 2023-12-20T06:04:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f52a0afc640 1 -- 172.21.15.32:0/1899016483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f529c14ee30 con 0x7f529c0c90e0 2023-12-20T06:04:09.385 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f52a0afc640 1 -- 172.21.15.32:0/1899016483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f529c14f3f0 con 0x7f529c0c90e0 2023-12-20T06:04:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f52a0afc640 1 -- 172.21.15.32:0/1899016483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f529c165c10 con 0x7f529c0c90e0 2023-12-20T06:04:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.382+0000 7f5292ffd640 1 -- 172.21.15.32:0/1899016483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f528401b400 con 0x7f529c0c90e0 2023-12-20T06:04:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.386+0000 7f5292ffd640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f527c065ca0 0x7f527c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.386+0000 7f5292ffd640 1 -- 172.21.15.32:0/1899016483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(242..242 src has 1..242) v4 ==== 5859+0+0 (secure 0 0 0) 0x7f52840838f0 con 0x7f529c0c90e0 2023-12-20T06:04:09.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.386+0000 7f529a59c640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f527c065ca0 0x7f527c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.386+0000 7f529a59c640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f527c065ca0 0x7f527c068160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f528c0ae000 tx=0x7f528c0a7000 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:09.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.386+0000 7f5292ffd640 1 -- 172.21.15.32:0/1899016483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5284050e10 con 0x7f529c0c90e0 2023-12-20T06:04:09.578 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.578+0000 7f52a0afc640 1 -- 172.21.15.32:0/1899016483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f529c04e000 con 0x7f529c0c90e0 2023-12-20T06:04:09.579 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.578+0000 7f5292ffd640 1 -- 172.21.15.32:0/1899016483 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 122 v122) v1 ==== 95+0+3527 (secure 0 0 0) 0x7f5284016070 con 0x7f529c0c90e0 2023-12-20T06:04:09.580 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:09.580 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":122,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":9323,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/3025406072","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":3025406072},{"type":"v1","addr":"172.21.15.39:6835","nonce":3025406072}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":9731,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/99271174","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":99271174},{"type":"v1","addr":"172.21.15.32:6837","nonce":99271174}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":120},{"gid":10924,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/1410690353","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":1410690353},{"type":"v1","addr":"172.21.15.39:6834","nonce":1410690353}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":107},{"gid":11979,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/108734040","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":108734040},{"type":"v1","addr":"172.21.15.32:6836","nonce":108734040}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":122}],"filesystems":[]} 2023-12-20T06:04:09.580 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 122 2023-12-20T06:04:09.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 -- 172.21.15.32:0/1899016483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f527c065ca0 msgr2=0x7f527c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:09.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f527c065ca0 0x7f527c068160 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f528c0ae000 tx=0x7f528c0a7000 comp rx=0 tx=0).stop 2023-12-20T06:04:09.581 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 -- 172.21.15.32:0/1899016483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529c0c90e0 msgr2=0x7f529c14f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529c0c90e0 0x7f529c14f6d0 secure :-1 s=READY pgs=2446 cs=0 l=1 rev1=1 crypto rx=0x7f528400c5b0 tx=0x7f528400ca70 comp rx=0 tx=0).stop 2023-12-20T06:04:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 -- 172.21.15.32:0/1899016483 shutdown_connections 2023-12-20T06:04:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f527c065ca0 0x7f527c068160 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f529c1599e0 0x7f529c14db50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f529c0c90e0 0x7f529c14f6d0 unknown :-1 s=CLOSED pgs=2446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.582 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 --2- 172.21.15.32:0/1899016483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f529c0b0d00 0x7f529c158ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 -- 172.21.15.32:0/1899016483 >> 172.21.15.32:0/1899016483 conn(0x7f529c06a320 msgr2=0x7f529c0aebb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 -- 172.21.15.32:0/1899016483 shutdown_connections 2023-12-20T06:04:09.583 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.582+0000 7f52a0afc640 1 -- 172.21.15.32:0/1899016483 wait complete. 2023-12-20T06:04:09.602 INFO:tasks.ceph.mds.a:Restarting daemon 2023-12-20T06:04:09.602 INFO:tasks.ceph.mds.a:Stopping old one... 2023-12-20T06:04:09.603 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T06:04:09.603 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:04:09.603 INFO:tasks.ceph.mds.c:Restarting daemon 2023-12-20T06:04:09.603 INFO:tasks.ceph.mds.c:Stopping old one... 2023-12-20T06:04:09.603 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T06:04:09.604 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:04:09.604 INFO:tasks.ceph.mds.b:Restarting daemon 2023-12-20T06:04:09.604 INFO:tasks.ceph.mds.b:Stopping old one... 2023-12-20T06:04:09.604 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T06:04:09.604 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:04:09.604 INFO:tasks.ceph.mds.d:Restarting daemon 2023-12-20T06:04:09.605 INFO:tasks.ceph.mds.d:Stopping old one... 2023-12-20T06:04:09.605 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T06:04:09.605 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:04:09.615 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T06:04:09.615 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i a 2023-12-20T06:04:09.616 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T06:04:09.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i c 2023-12-20T06:04:09.622 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T06:04:09.622 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i b 2023-12-20T06:04:09.623 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T06:04:09.624 DEBUG:teuthology.orchestra.run.smithi039:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mds -f --cluster ceph -i d 2023-12-20T06:04:09.664 INFO:tasks.ceph.mds.a:Started 2023-12-20T06:04:09.665 DEBUG:teuthology.parallel:result is None 2023-12-20T06:04:09.666 INFO:tasks.ceph.mds.c:Started 2023-12-20T06:04:09.667 DEBUG:teuthology.parallel:result is None 2023-12-20T06:04:09.673 INFO:tasks.ceph.mds.b:Started 2023-12-20T06:04:09.674 DEBUG:teuthology.parallel:result is None 2023-12-20T06:04:09.675 INFO:tasks.ceph.mds.d:Started 2023-12-20T06:04:09.676 DEBUG:teuthology.parallel:result is None 2023-12-20T06:04:09.676 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist clear 2023-12-20T06:04:09.755 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T06:04:09.755 INFO:tasks.ceph.mds.c.smithi032.stdout:starting mds.c at 2023-12-20T06:04:09.778 INFO:tasks.ceph.mds.b.smithi039.stdout:starting mds.b at 2023-12-20T06:04:09.778 INFO:tasks.ceph.mds.d.smithi039.stdout:starting mds.d at 2023-12-20T06:04:09.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.830+0000 7f97b1866640 1 Processor -- start 2023-12-20T06:04:09.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.830+0000 7f97b1866640 1 -- start start 2023-12-20T06:04:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97b1866640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97ac1599e0 0x7f97ac0aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97b1866640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f97ac0c90e0 0x7f97ac0b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97b1866640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f97ac0711d0 con 0x7f97ac0b0b30 2023-12-20T06:04:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97b1866640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f97ac0c98c0 con 0x7f97ac0c90e0 2023-12-20T06:04:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97b1866640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0c9ba0 con 0x7f97ac1599e0 2023-12-20T06:04:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97ab7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f97ac0b0b30 0x7f97ac15d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55312/0 (socket says 172.21.15.32:55312) 2023-12-20T06:04:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97ab7fe640 1 -- 172.21.15.32:0/1278989945 learned_addr learned my addr 172.21.15.32:0/1278989945 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97aaffd640 1 --2- 172.21.15.32:0/1278989945 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97ac1599e0 0x7f97ac0aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97a3fff640 1 --2- 172.21.15.32:0/1278989945 >> v2:172.21.15.39:3300/0 conn(0x7f97ac0c90e0 0x7f97ac0b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97aa7fc640 1 -- 172.21.15.32:0/1278989945 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1212756 0 0) 0x7f97ac0711d0 con 0x7f97ac0b0b30 2023-12-20T06:04:09.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97aa7fc640 1 -- 172.21.15.32:0/1278989945 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9798003530 con 0x7f97ac0b0b30 2023-12-20T06:04:09.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97aaffd640 1 -- 172.21.15.32:0/1278989945 >> v2:172.21.15.39:3300/0 conn(0x7f97ac0c90e0 msgr2=0x7f97ac0b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:09.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97aaffd640 1 --2- 172.21.15.32:0/1278989945 >> v2:172.21.15.39:3300/0 conn(0x7f97ac0c90e0 0x7f97ac0b05f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97aaffd640 1 -- 172.21.15.32:0/1278989945 >> v1:172.21.15.32:6789/0 conn(0x7f97ac0b0b30 legacy=0x7f97ac15d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:09.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97aaffd640 1 -- 172.21.15.32:0/1278989945 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f97ac0c9e80 con 0x7f97ac1599e0 2023-12-20T06:04:09.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.834+0000 7f97a3fff640 1 --2- 172.21.15.32:0/1278989945 >> v2:172.21.15.39:3300/0 conn(0x7f97ac0c90e0 0x7f97ac0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:09.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97aaffd640 1 --2- 172.21.15.32:0/1278989945 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97ac1599e0 0x7f97ac0aece0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f979c0097a0 tx=0x7f979c01c430 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b11478a929ac1eae server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:09.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97aa7fc640 1 -- 172.21.15.32:0/1278989945 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c004230 con 0x7f97ac1599e0 2023-12-20T06:04:09.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97aa7fc640 1 -- 172.21.15.32:0/1278989945 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f979c0a7050 con 0x7f97ac1599e0 2023-12-20T06:04:09.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97aa7fc640 1 -- 172.21.15.32:0/1278989945 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f979c003620 con 0x7f97ac1599e0 2023-12-20T06:04:09.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97b1866640 1 -- 172.21.15.32:0/1278989945 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97ac1599e0 msgr2=0x7f97ac0aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:09.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97b1866640 1 --2- 172.21.15.32:0/1278989945 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97ac1599e0 0x7f97ac0aece0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f979c0097a0 tx=0x7f979c01c430 comp rx=0 tx=0).stop 2023-12-20T06:04:09.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97b1866640 1 -- 172.21.15.32:0/1278989945 shutdown_connections 2023-12-20T06:04:09.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97b1866640 1 --2- 172.21.15.32:0/1278989945 >> v2:172.21.15.39:3300/0 conn(0x7f97ac0c90e0 0x7f97ac0b05f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97b1866640 1 --2- 172.21.15.32:0/1278989945 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97ac1599e0 0x7f97ac0aece0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97b1866640 1 -- 172.21.15.32:0/1278989945 >> 172.21.15.32:0/1278989945 conn(0x7f97ac06a320 msgr2=0x7f97ac0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:09.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97b1866640 1 -- 172.21.15.32:0/1278989945 shutdown_connections 2023-12-20T06:04:09.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97b1866640 1 -- 172.21.15.32:0/1278989945 wait complete. 2023-12-20T06:04:09.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97b1866640 1 Processor -- start 2023-12-20T06:04:09.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.838+0000 7f97b1866640 1 -- start start 2023-12-20T06:04:09.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97b1866640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97ac0b0b30 0x7f97ac0c4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97b1866640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97ac0c90e0 0x7f97ac14e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97b1866640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1599e0 0x7f97ac14eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97b1866640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac1f6000 con 0x7f97ac1599e0 2023-12-20T06:04:09.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97b1866640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac0c4da0 con 0x7f97ac0c90e0 2023-12-20T06:04:09.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97b1866640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f97ac1f53f0 con 0x7f97ac0b0b30 2023-12-20T06:04:09.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1599e0 0x7f97ac14eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97aaffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97ac0b0b30 0x7f97ac0c4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.843 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97ab7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1599e0 0x7f97ac14eb50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49874/0 (socket says 172.21.15.32:49874) 2023-12-20T06:04:09.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97ab7fe640 1 -- 172.21.15.32:0/4122910066 learned_addr learned my addr 172.21.15.32:0/4122910066 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:09.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97a3fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97ac0c90e0 0x7f97ac14e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97ab7fe640 1 -- 172.21.15.32:0/4122910066 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97ac0b0b30 msgr2=0x7f97ac0c4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:09.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97ab7fe640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97ac0b0b30 0x7f97ac0c4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97ab7fe640 1 -- 172.21.15.32:0/4122910066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97ac0c90e0 msgr2=0x7f97ac14e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:09.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97ab7fe640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97ac0c90e0 0x7f97ac14e610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:09.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97ab7fe640 1 -- 172.21.15.32:0/4122910066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9794009420 con 0x7f97ac1599e0 2023-12-20T06:04:09.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97a3fff640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97ac0c90e0 0x7f97ac14e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:09.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97ab7fe640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1599e0 0x7f97ac14eb50 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f97a400caa0 tx=0x7f97a400cf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:09.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97a8ff9640 1 -- 172.21.15.32:0/4122910066 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a400ac00 con 0x7f97ac1599e0 2023-12-20T06:04:09.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97a8ff9640 1 -- 172.21.15.32:0/4122910066 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f97a400ad60 con 0x7f97ac1599e0 2023-12-20T06:04:09.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97b1866640 1 -- 172.21.15.32:0/4122910066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f979c0093b0 con 0x7f97ac1599e0 2023-12-20T06:04:09.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97a8ff9640 1 -- 172.21.15.32:0/4122910066 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f97a4012620 con 0x7f97ac1599e0 2023-12-20T06:04:09.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97b1866640 1 -- 172.21.15.32:0/4122910066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f97ac1f59b0 con 0x7f97ac1599e0 2023-12-20T06:04:09.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97b1866640 1 -- 172.21.15.32:0/4122910066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f97ac04e000 con 0x7f97ac1599e0 2023-12-20T06:04:09.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.842+0000 7f97a8ff9640 1 -- 172.21.15.32:0/4122910066 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f97a4007070 con 0x7f97ac1599e0 2023-12-20T06:04:09.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.846+0000 7f97a8ff9640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f978c065ca0 0x7f978c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:09.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.846+0000 7f97a8ff9640 1 -- 172.21.15.32:0/4122910066 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(246..246 src has 1..246) v4 ==== 5859+0+0 (secure 0 0 0) 0x7f97a40834a0 con 0x7f97ac1599e0 2023-12-20T06:04:09.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.846+0000 7f97aaffd640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f978c065ca0 0x7f978c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:09.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.846+0000 7f97aaffd640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f978c065ca0 0x7f978c068160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f979c0ad000 tx=0x7f979c0a6000 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:09.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:09.850+0000 7f97a8ff9640 1 -- 172.21.15.32:0/4122910066 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f97a4050b80 con 0x7f97ac1599e0 2023-12-20T06:04:10.019 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.018+0000 7f97b1866640 1 -- 172.21.15.32:0/4122910066 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist clear"} v 0) v1 -- 0x7f97ac0c7af0 con 0x7f97ac1599e0 2023-12-20T06:04:10.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.818+0000 7f97a8ff9640 1 -- 172.21.15.32:0/4122910066 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist clear"}]=0 removed all blocklist entries v247) v1 ==== 97+0+0 (secure 0 0 0) 0x7f97a40541a0 con 0x7f97ac1599e0 2023-12-20T06:04:10.821 INFO:teuthology.orchestra.run.smithi032.stderr: removed all blocklist entries 2023-12-20T06:04:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 -- 172.21.15.32:0/4122910066 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f978c065ca0 msgr2=0x7f978c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:10.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f978c065ca0 0x7f978c068160 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f979c0ad000 tx=0x7f979c0a6000 comp rx=0 tx=0).stop 2023-12-20T06:04:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 -- 172.21.15.32:0/4122910066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1599e0 msgr2=0x7f97ac14eb50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1599e0 0x7f97ac14eb50 secure :-1 s=READY pgs=2455 cs=0 l=1 rev1=1 crypto rx=0x7f97a400caa0 tx=0x7f97a400cf60 comp rx=0 tx=0).stop 2023-12-20T06:04:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 -- 172.21.15.32:0/4122910066 shutdown_connections 2023-12-20T06:04:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f978c065ca0 0x7f978c068160 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f97ac1599e0 0x7f97ac14eb50 unknown :-1 s=CLOSED pgs=2455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f97ac0c90e0 0x7f97ac14e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 --2- 172.21.15.32:0/4122910066 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f97ac0b0b30 0x7f97ac0c4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 -- 172.21.15.32:0/4122910066 >> 172.21.15.32:0/4122910066 conn(0x7f97ac06a320 msgr2=0x7f97ac0aeb20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 -- 172.21.15.32:0/4122910066 shutdown_connections 2023-12-20T06:04:10.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.822+0000 7f97b1866640 1 -- 172.21.15.32:0/4122910066 wait complete. 2023-12-20T06:04:10.846 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth ls --format=json-pretty 2023-12-20T06:04:10.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e3d8d1640 1 Processor -- start 2023-12-20T06:04:10.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e3d8d1640 1 -- start start 2023-12-20T06:04:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e3d8d1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e381599e0 0x7f1e380aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:10.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e3d8d1640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1e380b0d00 0x7f1e380701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e3d8d1640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1e380c98c0 con 0x7f1e380c90e0 2023-12-20T06:04:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e3d8d1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e380c9ba0 con 0x7f1e381599e0 2023-12-20T06:04:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e3d8d1640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1e380c9e80 con 0x7f1e380b0d00 2023-12-20T06:04:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e36ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e381599e0 0x7f1e380aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e36ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1e381599e0 0x7f1e380aece0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49886/0 (socket says 172.21.15.32:49886) 2023-12-20T06:04:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e36ffd640 1 -- 172.21.15.32:0/863913448 learned_addr learned my addr 172.21.15.32:0/863913448 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e377fe640 1 --2- 172.21.15.32:0/863913448 >> v2:172.21.15.39:3300/0 conn(0x7f1e380b0d00 0x7f1e380701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e36ffd640 1 -- 172.21.15.32:0/863913448 >> v2:172.21.15.39:3300/0 conn(0x7f1e380b0d00 msgr2=0x7f1e380701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e36ffd640 1 --2- 172.21.15.32:0/863913448 >> v2:172.21.15.39:3300/0 conn(0x7f1e380b0d00 0x7f1e380701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:10.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e36ffd640 1 -- 172.21.15.32:0/863913448 >> v1:172.21.15.39:6789/0 conn(0x7f1e380c90e0 legacy=0x7f1e380b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:10.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.994+0000 7f1e36ffd640 1 -- 172.21.15.32:0/863913448 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e380ca160 con 0x7f1e381599e0 2023-12-20T06:04:11.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e36ffd640 1 --2- 172.21.15.32:0/863913448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e381599e0 0x7f1e380aece0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f1e380b0060 tx=0x7f1e2801c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75cf5ffc2c11b594 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:11.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e367fc640 1 -- 172.21.15.32:0/863913448 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e280ac600 con 0x7f1e381599e0 2023-12-20T06:04:11.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e367fc640 1 -- 172.21.15.32:0/863913448 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e280a8050 con 0x7f1e381599e0 2023-12-20T06:04:11.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e367fc640 1 -- 172.21.15.32:0/863913448 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e280ac980 con 0x7f1e381599e0 2023-12-20T06:04:11.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/863913448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e381599e0 msgr2=0x7f1e380aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e3d8d1640 1 --2- 172.21.15.32:0/863913448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e381599e0 0x7f1e380aece0 secure :-1 s=READY pgs=2456 cs=0 l=1 rev1=1 crypto rx=0x7f1e380b0060 tx=0x7f1e2801c490 comp rx=0 tx=0).stop 2023-12-20T06:04:11.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/863913448 shutdown_connections 2023-12-20T06:04:11.001 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e3d8d1640 1 --2- 172.21.15.32:0/863913448 >> v2:172.21.15.39:3300/0 conn(0x7f1e380b0d00 0x7f1e380701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e3d8d1640 1 --2- 172.21.15.32:0/863913448 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e381599e0 0x7f1e380aece0 unknown :-1 s=CLOSED pgs=2456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/863913448 >> 172.21.15.32:0/863913448 conn(0x7f1e3806a320 msgr2=0x7f1e380ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:11.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/863913448 shutdown_connections 2023-12-20T06:04:11.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:10.998+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/863913448 wait complete. 2023-12-20T06:04:11.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e3d8d1640 1 Processor -- start 2023-12-20T06:04:11.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e3d8d1640 1 -- start start 2023-12-20T06:04:11.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e3d8d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e380b0d00 0x7f1e38154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e3d8d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e380c90e0 0x7f1e38148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e3d8d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e381599e0 0x7f1e38148bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e3d8d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3814bb30 con 0x7f1e380b0d00 2023-12-20T06:04:11.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e3d8d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3815a290 con 0x7f1e381599e0 2023-12-20T06:04:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e3d8d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1e3815a570 con 0x7f1e380c90e0 2023-12-20T06:04:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e36ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e380b0d00 0x7f1e38154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e36ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e380b0d00 0x7f1e38154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49894/0 (socket says 172.21.15.32:49894) 2023-12-20T06:04:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e36ffd640 1 -- 172.21.15.32:0/3535160901 learned_addr learned my addr 172.21.15.32:0/3535160901 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e2ffff640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e380c90e0 0x7f1e38148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e377fe640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e381599e0 0x7f1e38148bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e36ffd640 1 -- 172.21.15.32:0/3535160901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e380c90e0 msgr2=0x7f1e38148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e36ffd640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e380c90e0 0x7f1e38148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e36ffd640 1 -- 172.21.15.32:0/3535160901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e381599e0 msgr2=0x7f1e38148bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e36ffd640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e381599e0 0x7f1e38148bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e36ffd640 1 -- 172.21.15.32:0/3535160901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e28009480 con 0x7f1e380b0d00 2023-12-20T06:04:11.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e36ffd640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e380b0d00 0x7f1e38154b50 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f1e2801a0c0 tx=0x7f1e28019230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:11.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.002+0000 7f1e34ff9640 1 -- 172.21.15.32:0/3535160901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e280bb070 con 0x7f1e380b0d00 2023-12-20T06:04:11.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.006+0000 7f1e34ff9640 1 -- 172.21.15.32:0/3535160901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e28003970 con 0x7f1e380b0d00 2023-12-20T06:04:11.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.006+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/3535160901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1e3815a850 con 0x7f1e380b0d00 2023-12-20T06:04:11.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.006+0000 7f1e34ff9640 1 -- 172.21.15.32:0/3535160901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e28003d00 con 0x7f1e380b0d00 2023-12-20T06:04:11.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.006+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/3535160901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1e3815b050 con 0x7f1e380b0d00 2023-12-20T06:04:11.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.006+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/3535160901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1e3804e000 con 0x7f1e380b0d00 2023-12-20T06:04:11.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.006+0000 7f1e34ff9640 1 -- 172.21.15.32:0/3535160901 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f1e280b3070 con 0x7f1e380b0d00 2023-12-20T06:04:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.006+0000 7f1e34ff9640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e18065ca0 0x7f1e18068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.006+0000 7f1e34ff9640 1 -- 172.21.15.32:0/3535160901 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f1e2811cfe0 con 0x7f1e380b0d00 2023-12-20T06:04:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.006+0000 7f1e2ffff640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e18065ca0 0x7f1e18068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.006+0000 7f1e2ffff640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e18065ca0 0x7f1e18068160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1e380ce070 tx=0x7f1e20009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:11.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.010+0000 7f1e34ff9640 1 -- 172.21.15.32:0/3535160901 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1e280ea900 con 0x7f1e380b0d00 2023-12-20T06:04:11.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.218+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/3535160901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth ls", "format": "json-pretty"} v 0) v1 -- 0x7f1e3815ad00 con 0x7f1e380b0d00 2023-12-20T06:04:11.221 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e34ff9640 1 -- 172.21.15.32:0/3535160901 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth ls", "format": "json-pretty"}]=0 v53) v1 ==== 80+0+5304 (secure 0 0 0) 0x7f1e28003e60 con 0x7f1e380b0d00 2023-12-20T06:04:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:11.221 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:04:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "auth_dump": [ 2023-12-20T06:04:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.a", 2023-12-20T06:04:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlVWdqGRAAsQ4EQxkwNDHwoT5P2jAvHQ==", 2023-12-20T06:04:11.221 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:04:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:04:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:04:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.b", 2023-12-20T06:04:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlYEcFJhAAouCOK78puZypmIDUdoHXEg==", 2023-12-20T06:04:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.222 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.c", 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlLlzwIBAAL/YQVHMQ0iUmtValsTEQkA==", 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:04:11.223 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mds.d", 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlkPKSLRAA0n8M8bppGBLqf5f+kEosvA==", 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.224 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.0", 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA1bYJl9T8PHRAAh36LDLcIyEj563hOqtxoDQ==", 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.1", 2023-12-20T06:04:11.225 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA4bYJlE+NjIxAA5836xHY6R5gIFW/4ceaz8w==", 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.2", 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA7bYJleYzJKRAAkpHVi0v/WFd5l7vwK4jJcw==", 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:04:11.226 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.3", 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA+bYJlPJvOMhAAg+hMj1sWFk2mJDboPXAuOA==", 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.227 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.4", 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBIbYJlwfJVEBAAQ82TT5/zbvQs//rwtFgqsg==", 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.5", 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBLbYJlLuAtGBAAOFZxfFslqjBQQ50tQA9fEw==", 2023-12-20T06:04:11.228 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.6", 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBObYJlQ3l3IBAAjfgXl8YDTL3t547f6FRp+g==", 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:04:11.229 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "osd.7", 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQBRbYJlH1o4KBAA6xkuJuEQYcKJTq+/BtnZIA==", 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow profile osd", 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile osd", 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.230 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.0", 2023-12-20T06:04:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWlbtMRAAOb2KhEopJ9HOA9r2u86OnA==", 2023-12-20T06:04:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:04:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow r", 2023-12-20T06:04:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rw tag cephfs data=cephfs" 2023-12-20T06:04:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.1", 2023-12-20T06:04:11.231 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlCVmYABAA+jMZ746tvlpn+4Erdddkcg==", 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.2", 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJliVGBOBAAZkx1M0E0qJG7mRPq9zj+2A==", 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:04:11.232 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.3", 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQA0bYJlZ8rbBhAA06J6TvENU9Rp8Tzzyp5HGA==", 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow", 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow r", 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow rw", 2023-12-20T06:04:11.233 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow rwx" 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "client.admin", 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAybYJlGz5nOBAA/RjjFmglbpbU1BOnQM8kzg==", 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "mgr": "allow *", 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow *", 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.234 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.x", 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlETwPFRAA1M/3EGsNIYLTHDrYUDKhwQ==", 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: }, 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "entity": "mgr.y", 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "key": "AQAzbYJlWIMtERAAPyCPqy+8EJH+AinFJb8KWA==", 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "caps": { 2023-12-20T06:04:11.235 INFO:teuthology.orchestra.run.smithi032.stdout: "mds": "allow *", 2023-12-20T06:04:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: "mon": "allow profile mgr", 2023-12-20T06:04:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: "osd": "allow *" 2023-12-20T06:04:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:11.236 INFO:teuthology.orchestra.run.smithi032.stdout: ] 2023-12-20T06:04:11.236 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:04:11.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/3535160901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e18065ca0 msgr2=0x7f1e18068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e18065ca0 0x7f1e18068160 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f1e380ce070 tx=0x7f1e20009040 comp rx=0 tx=0).stop 2023-12-20T06:04:11.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/3535160901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e380b0d00 msgr2=0x7f1e38154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.237 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e380b0d00 0x7f1e38154b50 secure :-1 s=READY pgs=2457 cs=0 l=1 rev1=1 crypto rx=0x7f1e2801a0c0 tx=0x7f1e28019230 comp rx=0 tx=0).stop 2023-12-20T06:04:11.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/3535160901 shutdown_connections 2023-12-20T06:04:11.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e18065ca0 0x7f1e18068160 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1e381599e0 0x7f1e38148bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1e380c90e0 0x7f1e38148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 --2- 172.21.15.32:0/3535160901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1e380b0d00 0x7f1e38154b50 unknown :-1 s=CLOSED pgs=2457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/3535160901 >> 172.21.15.32:0/3535160901 conn(0x7f1e3806a320 msgr2=0x7f1e3815c170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:11.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/3535160901 shutdown_connections 2023-12-20T06:04:11.238 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.222+0000 7f1e3d8d1640 1 -- 172.21.15.32:0/3535160901 wait complete. 2023-12-20T06:04:11.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:04:11.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f633a9bc640 1 Processor -- start 2023-12-20T06:04:11.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f633a9bc640 1 -- start start 2023-12-20T06:04:11.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f633a9bc640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f63340af5e0 0x7f63340af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f633a9bc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f633406ce60 0x7f633415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f633a9bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f633415bbd0 con 0x7f633415f110 2023-12-20T06:04:11.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f633a9bc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633415beb0 con 0x7f633406ce60 2023-12-20T06:04:11.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f633a9bc640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f633415e910 con 0x7f63340af5e0 2023-12-20T06:04:11.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f63337fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f633406ce60 0x7f633415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f63337fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f633406ce60 0x7f633415b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49904/0 (socket says 172.21.15.32:49904) 2023-12-20T06:04:11.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f63337fe640 1 -- 172.21.15.32:0/4022145800 learned_addr learned my addr 172.21.15.32:0/4022145800 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:11.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f6333fff640 1 --2- 172.21.15.32:0/4022145800 >> v2:172.21.15.39:3300/0 conn(0x7f63340af5e0 0x7f63340af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f63337fe640 1 -- 172.21.15.32:0/4022145800 >> v2:172.21.15.39:3300/0 conn(0x7f63340af5e0 msgr2=0x7f63340af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:11.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f63337fe640 1 --2- 172.21.15.32:0/4022145800 >> v2:172.21.15.39:3300/0 conn(0x7f63340af5e0 0x7f63340af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f63337fe640 1 -- 172.21.15.32:0/4022145800 >> v1:172.21.15.39:6789/0 conn(0x7f633415f110 legacy=0x7f633415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:11.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.386+0000 7f63337fe640 1 -- 172.21.15.32:0/4022145800 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f633415ebf0 con 0x7f633406ce60 2023-12-20T06:04:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f63337fe640 1 --2- 172.21.15.32:0/4022145800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633406ce60 0x7f633415b280 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f633406ac10 tx=0x7f632801c4c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f71316403001d0d server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f6332ffd640 1 -- 172.21.15.32:0/4022145800 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280ab620 con 0x7f633406ce60 2023-12-20T06:04:11.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f6332ffd640 1 -- 172.21.15.32:0/4022145800 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63280ab780 con 0x7f633406ce60 2023-12-20T06:04:11.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f6332ffd640 1 -- 172.21.15.32:0/4022145800 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f63280abb10 con 0x7f633406ce60 2023-12-20T06:04:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f633a9bc640 1 -- 172.21.15.32:0/4022145800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633406ce60 msgr2=0x7f633415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f633a9bc640 1 --2- 172.21.15.32:0/4022145800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633406ce60 0x7f633415b280 secure :-1 s=READY pgs=2458 cs=0 l=1 rev1=1 crypto rx=0x7f633406ac10 tx=0x7f632801c4c0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f633a9bc640 1 -- 172.21.15.32:0/4022145800 shutdown_connections 2023-12-20T06:04:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f633a9bc640 1 --2- 172.21.15.32:0/4022145800 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633406ce60 0x7f633415b280 unknown :-1 s=CLOSED pgs=2458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f633a9bc640 1 --2- 172.21.15.32:0/4022145800 >> v2:172.21.15.39:3300/0 conn(0x7f63340af5e0 0x7f63340af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f633a9bc640 1 -- 172.21.15.32:0/4022145800 >> 172.21.15.32:0/4022145800 conn(0x7f633406a320 msgr2=0x7f63340ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f633a9bc640 1 -- 172.21.15.32:0/4022145800 shutdown_connections 2023-12-20T06:04:11.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.390+0000 7f633a9bc640 1 -- 172.21.15.32:0/4022145800 wait complete. 2023-12-20T06:04:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f633a9bc640 1 Processor -- start 2023-12-20T06:04:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f633a9bc640 1 -- start start 2023-12-20T06:04:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f633a9bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633406ce60 0x7f633415c420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.396 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f633a9bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63340af5e0 0x7f6334148690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f633a9bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633415f110 0x7f633414baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f633a9bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633414b610 con 0x7f633406ce60 2023-12-20T06:04:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f633a9bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f633414bfe0 con 0x7f633415f110 2023-12-20T06:04:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f633a9bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f633414aa00 con 0x7f63340af5e0 2023-12-20T06:04:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6333fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633406ce60 0x7f633415c420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6333fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633406ce60 0x7f633415c420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49912/0 (socket says 172.21.15.32:49912) 2023-12-20T06:04:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6333fff640 1 -- 172.21.15.32:0/74548878 learned_addr learned my addr 172.21.15.32:0/74548878 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6338f59640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633415f110 0x7f633414baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f63337fe640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63340af5e0 0x7f6334148690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6333fff640 1 -- 172.21.15.32:0/74548878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63340af5e0 msgr2=0x7f6334148690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6333fff640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63340af5e0 0x7f6334148690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6333fff640 1 -- 172.21.15.32:0/74548878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633415f110 msgr2=0x7f633414baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6333fff640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633415f110 0x7f633414baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6333fff640 1 -- 172.21.15.32:0/74548878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f63280094d0 con 0x7f633406ce60 2023-12-20T06:04:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6333fff640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633406ce60 0x7f633415c420 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f632400d800 tx=0x7f632400dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6330ff9640 1 -- 172.21.15.32:0/74548878 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324018070 con 0x7f633406ce60 2023-12-20T06:04:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6330ff9640 1 -- 172.21.15.32:0/74548878 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f63240059a0 con 0x7f633406ce60 2023-12-20T06:04:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f633a9bc640 1 -- 172.21.15.32:0/74548878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f633414ace0 con 0x7f633406ce60 2023-12-20T06:04:11.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f6330ff9640 1 -- 172.21.15.32:0/74548878 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6324005b50 con 0x7f633406ce60 2023-12-20T06:04:11.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.394+0000 7f633a9bc640 1 -- 172.21.15.32:0/74548878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f633414b2a0 con 0x7f633406ce60 2023-12-20T06:04:11.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.398+0000 7f633a9bc640 1 -- 172.21.15.32:0/74548878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f633404e000 con 0x7f633406ce60 2023-12-20T06:04:11.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.398+0000 7f6330ff9640 1 -- 172.21.15.32:0/74548878 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f6324005cb0 con 0x7f633406ce60 2023-12-20T06:04:11.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.398+0000 7f6330ff9640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6320065ca0 0x7f6320068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.398+0000 7f6330ff9640 1 -- 172.21.15.32:0/74548878 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f63240830e0 con 0x7f633406ce60 2023-12-20T06:04:11.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.398+0000 7f63337fe640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6320065ca0 0x7f6320068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.398+0000 7f63337fe640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6320065ca0 0x7f6320068160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63280af000 tx=0x7f63280b6040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:11.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.402+0000 7f6330ff9640 1 -- 172.21.15.32:0/74548878 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6324050af0 con 0x7f633406ce60 2023-12-20T06:04:11.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.562+0000 7f633a9bc640 1 -- 172.21.15.32:0/74548878 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f63340cf450 con 0x7f633406ce60 2023-12-20T06:04:11.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.562+0000 7f6330ff9640 1 -- 172.21.15.32:0/74548878 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v247) v1 ==== 74+0+12441 (secure 0 0 0) 0x7f6324054110 con 0x7f633406ce60 2023-12-20T06:04:11.565 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:11.565 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":247,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:04:10.804598+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":31,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":232,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:04:11.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 -- 172.21.15.32:0/74548878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6320065ca0 msgr2=0x7f6320068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6320065ca0 0x7f6320068160 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f63280af000 tx=0x7f63280b6040 comp rx=0 tx=0).stop 2023-12-20T06:04:11.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 -- 172.21.15.32:0/74548878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633406ce60 msgr2=0x7f633415c420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633406ce60 0x7f633415c420 secure :-1 s=READY pgs=2459 cs=0 l=1 rev1=1 crypto rx=0x7f632400d800 tx=0x7f632400dcc0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 -- 172.21.15.32:0/74548878 shutdown_connections 2023-12-20T06:04:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6320065ca0 0x7f6320068160 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f633415f110 0x7f633414baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f63340af5e0 0x7f6334148690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 --2- 172.21.15.32:0/74548878 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f633406ce60 0x7f633415c420 unknown :-1 s=CLOSED pgs=2459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 -- 172.21.15.32:0/74548878 >> 172.21.15.32:0/74548878 conn(0x7f633406a320 msgr2=0x7f63340cb7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 -- 172.21.15.32:0/74548878 shutdown_connections 2023-12-20T06:04:11.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.566+0000 7f633a9bc640 1 -- 172.21.15.32:0/74548878 wait complete. 2023-12-20T06:04:11.586 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:04:11.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:04:11.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6f62f9640 1 Processor -- start 2023-12-20T06:04:11.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6f62f9640 1 -- start start 2023-12-20T06:04:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6f62f9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb6f0070940 0x7fb6f00caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6f62f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f015b6a0 con 0x7fb6f015abb0 2023-12-20T06:04:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6f62f9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb6f006d8c0 con 0x7fb6f00b0cb0 2023-12-20T06:04:11.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6f62f9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb6f006e160 con 0x7fb6f0070940 2023-12-20T06:04:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6effff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb6f015abb0 0x7fb6f015bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55324/0 (socket says 172.21.15.32:55324) 2023-12-20T06:04:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6effff640 1 -- 172.21.15.32:0/1446278441 learned_addr learned my addr 172.21.15.32:0/1446278441 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6f4896640 1 --2- 172.21.15.32:0/1446278441 >> v2:172.21.15.39:3300/0 conn(0x7fb6f0070940 0x7fb6f00caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2080771535 0 0) 0x7fb6f015b6a0 con 0x7fb6f015abb0 2023-12-20T06:04:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb6d8003530 con 0x7fb6f015abb0 2023-12-20T06:04:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1036582639 0 0) 0x7fb6d8003530 con 0x7fb6f015abb0 2023-12-20T06:04:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 >> v2:172.21.15.39:3300/0 conn(0x7fb6f0070940 msgr2=0x7fb6f00caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 --2- 172.21.15.32:0/1446278441 >> v2:172.21.15.39:3300/0 conn(0x7fb6f0070940 0x7fb6f00caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 >> v1:172.21.15.39:6789/0 conn(0x7fb6f00b0cb0 legacy=0x7fb6f0070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f006dba0 con 0x7fb6f015abb0 2023-12-20T06:04:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fb6e0001680 con 0x7fb6f015abb0 2023-12-20T06:04:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 >> v1:172.21.15.32:6789/0 conn(0x7fb6f015abb0 legacy=0x7fb6f015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 --2- 172.21.15.32:0/1446278441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d8003980 0x7fb6d8003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 --2- 172.21.15.32:0/1446278441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d80047d0 0x7fb6d8004c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 --2- 172.21.15.32:0/1446278441 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6d8005190 0x7fb6d800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f006e160 con 0x7fb6d8003980 2023-12-20T06:04:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d800c030 con 0x7fb6d80047d0 2023-12-20T06:04:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6d800c1f0 con 0x7fb6d8005190 2023-12-20T06:04:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6effff640 1 --2- 172.21.15.32:0/1446278441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d8003980 0x7fb6d8003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6ef7fe640 1 --2- 172.21.15.32:0/1446278441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d80047d0 0x7fb6d8004c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6f4896640 1 --2- 172.21.15.32:0/1446278441 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6d8005190 0x7fb6d800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6effff640 1 -- 172.21.15.32:0/1446278441 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6d8005190 msgr2=0x7fb6d800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:11.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6effff640 1 --2- 172.21.15.32:0/1446278441 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6d8005190 0x7fb6d800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6effff640 1 -- 172.21.15.32:0/1446278441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d80047d0 msgr2=0x7fb6d8004c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:11.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6effff640 1 --2- 172.21.15.32:0/1446278441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d80047d0 0x7fb6d8004c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6effff640 1 -- 172.21.15.32:0/1446278441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6d8003530 con 0x7fb6d8003980 2023-12-20T06:04:11.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.734+0000 7fb6f4896640 1 -- 172.21.15.32:0/1446278441 reap_dead start 2023-12-20T06:04:11.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.738+0000 7fb6effff640 1 --2- 172.21.15.32:0/1446278441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d8003980 0x7fb6d8003da0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000c1c0 tx=0x7fb6e001cb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4ae96f3f1b934a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.738+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e0005ae0 con 0x7fb6d8003980 2023-12-20T06:04:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.738+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e0005c40 con 0x7fb6d8003980 2023-12-20T06:04:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.738+0000 7fb6eeffd640 1 -- 172.21.15.32:0/1446278441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e00a5a90 con 0x7fb6d8003980 2023-12-20T06:04:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.738+0000 7fb6f62f9640 1 -- 172.21.15.32:0/1446278441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d8003980 msgr2=0x7fb6d8003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.738+0000 7fb6f62f9640 1 --2- 172.21.15.32:0/1446278441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d8003980 0x7fb6d8003da0 secure :-1 s=READY pgs=2461 cs=0 l=1 rev1=1 crypto rx=0x7fb6e000c1c0 tx=0x7fb6e001cb70 comp rx=0 tx=0).stop 2023-12-20T06:04:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.738+0000 7fb6f62f9640 1 -- 172.21.15.32:0/1446278441 shutdown_connections 2023-12-20T06:04:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.738+0000 7fb6f62f9640 1 --2- 172.21.15.32:0/1446278441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6d8003980 0x7fb6d8003da0 unknown :-1 s=CLOSED pgs=2461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.738+0000 7fb6f62f9640 1 -- 172.21.15.32:0/1446278441 >> 172.21.15.32:0/1446278441 conn(0x7fb6f006a510 msgr2=0x7fb6f00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:11.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.738+0000 7fb6f62f9640 1 -- 172.21.15.32:0/1446278441 shutdown_connections 2023-12-20T06:04:11.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.738+0000 7fb6f62f9640 1 -- 172.21.15.32:0/1446278441 wait complete. 2023-12-20T06:04:11.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6f62f9640 1 Processor -- start 2023-12-20T06:04:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6f62f9640 1 -- start start 2023-12-20T06:04:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6f62f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d8003980 0x7fb6f014e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6f62f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f015d660 0x7fb6f01589c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6f62f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6f015dae0 0x7fb6f0158f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6f62f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f00cd0f0 con 0x7fb6f015d660 2023-12-20T06:04:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6f62f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f00cc4e0 con 0x7fb6d8003980 2023-12-20T06:04:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6f62f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb6f00cc7c0 con 0x7fb6f015dae0 2023-12-20T06:04:11.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f015d660 0x7fb6f01589c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ef7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f015d660 0x7fb6f01589c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49926/0 (socket says 172.21.15.32:49926) 2023-12-20T06:04:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ef7fe640 1 -- 172.21.15.32:0/16281658 learned_addr learned my addr 172.21.15.32:0/16281658 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6effff640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d8003980 0x7fb6f014e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6f4896640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6f015dae0 0x7fb6f0158f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ef7fe640 1 -- 172.21.15.32:0/16281658 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6f015dae0 msgr2=0x7fb6f0158f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ef7fe640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6f015dae0 0x7fb6f0158f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ef7fe640 1 -- 172.21.15.32:0/16281658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d8003980 msgr2=0x7fb6f014e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ef7fe640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d8003980 0x7fb6f014e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ef7fe640 1 -- 172.21.15.32:0/16281658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb6f00ccaa0 con 0x7fb6f015d660 2023-12-20T06:04:11.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6f4896640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6f015dae0 0x7fb6f0158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:11.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ef7fe640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f015d660 0x7fb6f01589c0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400f570 tx=0x7fb6e400fa30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:11.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ecff9640 1 -- 172.21.15.32:0/16281658 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e40052e0 con 0x7fb6f015d660 2023-12-20T06:04:11.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ecff9640 1 -- 172.21.15.32:0/16281658 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb6e4005440 con 0x7fb6f015d660 2023-12-20T06:04:11.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6ecff9640 1 -- 172.21.15.32:0/16281658 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb6e4003820 con 0x7fb6f015d660 2023-12-20T06:04:11.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.742+0000 7fb6f62f9640 1 -- 172.21.15.32:0/16281658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb6f00ccd80 con 0x7fb6f015d660 2023-12-20T06:04:11.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.746+0000 7fb6f62f9640 1 -- 172.21.15.32:0/16281658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb6f014e8f0 con 0x7fb6f015d660 2023-12-20T06:04:11.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.746+0000 7fb6f62f9640 1 -- 172.21.15.32:0/16281658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb6f00cd260 con 0x7fb6f015d660 2023-12-20T06:04:11.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.746+0000 7fb6ecff9640 1 -- 172.21.15.32:0/16281658 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fb6e4003980 con 0x7fb6f015d660 2023-12-20T06:04:11.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.746+0000 7fb6ecff9640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6dc065ca0 0x7fb6dc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:11.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.746+0000 7fb6ecff9640 1 -- 172.21.15.32:0/16281658 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fb6e4082fe0 con 0x7fb6f015d660 2023-12-20T06:04:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.746+0000 7fb6effff640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6dc065ca0 0x7fb6dc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:11.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.746+0000 7fb6effff640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6dc065ca0 0x7fb6dc068160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00ab000 tx=0x7fb6e00a7070 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:11.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.750+0000 7fb6ecff9640 1 -- 172.21.15.32:0/16281658 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb6e40509a0 con 0x7fb6f015d660 2023-12-20T06:04:11.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.906+0000 7fb6f62f9640 1 -- 172.21.15.32:0/16281658 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fb6f004e000 con 0x7fb6f015d660 2023-12-20T06:04:11.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.906+0000 7fb6ecff9640 1 -- 172.21.15.32:0/16281658 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v247) v1 ==== 93+0+10 (secure 0 0 0) 0x7fb6e4053fc0 con 0x7fb6f015d660 2023-12-20T06:04:11.909 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:04:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 -- 172.21.15.32:0/16281658 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6dc065ca0 msgr2=0x7fb6dc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6dc065ca0 0x7fb6dc068160 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fb6e00ab000 tx=0x7fb6e00a7070 comp rx=0 tx=0).stop 2023-12-20T06:04:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 -- 172.21.15.32:0/16281658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f015d660 msgr2=0x7fb6f01589c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:11.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f015d660 0x7fb6f01589c0 secure :-1 s=READY pgs=2462 cs=0 l=1 rev1=1 crypto rx=0x7fb6e400f570 tx=0x7fb6e400fa30 comp rx=0 tx=0).stop 2023-12-20T06:04:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 -- 172.21.15.32:0/16281658 shutdown_connections 2023-12-20T06:04:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb6dc065ca0 0x7fb6dc068160 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb6f015dae0 0x7fb6f0158f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb6f015d660 0x7fb6f01589c0 unknown :-1 s=CLOSED pgs=2462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 --2- 172.21.15.32:0/16281658 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb6d8003980 0x7fb6f014e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 -- 172.21.15.32:0/16281658 >> 172.21.15.32:0/16281658 conn(0x7fb6f006a510 msgr2=0x7fb6f00ca0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 -- 172.21.15.32:0/16281658 shutdown_connections 2023-12-20T06:04:11.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:11.910+0000 7fb6f62f9640 1 -- 172.21.15.32:0/16281658 wait complete. 2023-12-20T06:04:11.930 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2023-12-20T06:04:12.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee12f52640 1 Processor -- start 2023-12-20T06:04:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee12f52640 1 -- start start 2023-12-20T06:04:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee12f52640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fee0c0d2740 0x7fee0c0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee12f52640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee0c0cfab0 0x7fee0c0b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee12f52640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fee0c0cff00 con 0x7fee0c15cdb0 2023-12-20T06:04:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee12f52640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fee0c15d190 con 0x7fee0c0d2740 2023-12-20T06:04:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee12f52640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c0d16a0 con 0x7fee0c0cfab0 2023-12-20T06:04:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee10cee640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fee0c15cdb0 0x7fee0c0d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55334/0 (socket says 172.21.15.32:55334) 2023-12-20T06:04:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee10cee640 1 -- 172.21.15.32:0/3017180481 learned_addr learned my addr 172.21.15.32:0/3017180481 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:12.077 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee0bfff640 1 --2- 172.21.15.32:0/3017180481 >> v2:172.21.15.39:3300/0 conn(0x7fee0c0d2740 0x7fee0c0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee114ef640 1 --2- 172.21.15.32:0/3017180481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee0c0cfab0 0x7fee0c0b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee0b7fe640 1 -- 172.21.15.32:0/3017180481 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 578155380 0 0) 0x7fee00001120 con 0x7fee0c15cdb0 2023-12-20T06:04:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee0b7fe640 1 -- 172.21.15.32:0/3017180481 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fedfc003530 con 0x7fee0c15cdb0 2023-12-20T06:04:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee0b7fe640 1 -- 172.21.15.32:0/3017180481 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1721878320 0 0) 0x7fedfc003530 con 0x7fee0c15cdb0 2023-12-20T06:04:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee0bfff640 1 -- 172.21.15.32:0/3017180481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee0c0cfab0 msgr2=0x7fee0c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee0bfff640 1 --2- 172.21.15.32:0/3017180481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee0c0cfab0 0x7fee0c0b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee0bfff640 1 -- 172.21.15.32:0/3017180481 >> v1:172.21.15.32:6789/0 conn(0x7fee0c15cdb0 legacy=0x7fee0c0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee0bfff640 1 -- 172.21.15.32:0/3017180481 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee0c0d1980 con 0x7fee0c0d2740 2023-12-20T06:04:12.078 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.074+0000 7fee114ef640 1 --2- 172.21.15.32:0/3017180481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee0c0cfab0 0x7fee0c0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:12.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.078+0000 7fee0bfff640 1 --2- 172.21.15.32:0/3017180481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee0c0d2740 0x7fee0c0cf570 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fedf80097c0 tx=0x7fedf801eb90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8c149f070b58e841 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:12.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.078+0000 7fee0b7fe640 1 -- 172.21.15.32:0/3017180481 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00001810 con 0x7fee0c0d2740 2023-12-20T06:04:12.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.078+0000 7fee0b7fe640 1 -- 172.21.15.32:0/3017180481 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fedf80a85d0 con 0x7fee0c0d2740 2023-12-20T06:04:12.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.078+0000 7fee0b7fe640 1 -- 172.21.15.32:0/3017180481 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fedf80a8940 con 0x7fee0c0d2740 2023-12-20T06:04:12.081 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 -- 172.21.15.32:0/3017180481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee0c0d2740 msgr2=0x7fee0c0cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 --2- 172.21.15.32:0/3017180481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee0c0d2740 0x7fee0c0cf570 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fedf80097c0 tx=0x7fedf801eb90 comp rx=0 tx=0).stop 2023-12-20T06:04:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 -- 172.21.15.32:0/3017180481 shutdown_connections 2023-12-20T06:04:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 --2- 172.21.15.32:0/3017180481 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee0c0d2740 0x7fee0c0cf570 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 --2- 172.21.15.32:0/3017180481 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee0c0cfab0 0x7fee0c0b5c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 -- 172.21.15.32:0/3017180481 >> 172.21.15.32:0/3017180481 conn(0x7fee0c065ee0 msgr2=0x7fee0c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 -- 172.21.15.32:0/3017180481 shutdown_connections 2023-12-20T06:04:12.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 -- 172.21.15.32:0/3017180481 wait complete. 2023-12-20T06:04:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 Processor -- start 2023-12-20T06:04:12.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 -- start start 2023-12-20T06:04:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c0cfab0 0x7fee0c15a950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee0c0d2740 0x7fee0c1548c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee0c15cdb0 0x7fee0c0cb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c15ae90 con 0x7fee0c0cfab0 2023-12-20T06:04:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c1595d0 con 0x7fee0c0d2740 2023-12-20T06:04:12.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee12f52640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fee0c1598b0 con 0x7fee0c15cdb0 2023-12-20T06:04:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee10cee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c0cfab0 0x7fee0c15a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee10cee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c0cfab0 0x7fee0c15a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49940/0 (socket says 172.21.15.32:49940) 2023-12-20T06:04:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee10cee640 1 -- 172.21.15.32:0/332670537 learned_addr learned my addr 172.21.15.32:0/332670537 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee0bfff640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee0c0d2740 0x7fee0c1548c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee10cee640 1 -- 172.21.15.32:0/332670537 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee0c15cdb0 msgr2=0x7fee0c0cb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee10cee640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee0c15cdb0 0x7fee0c0cb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee10cee640 1 -- 172.21.15.32:0/332670537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee0c0d2740 msgr2=0x7fee0c1548c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee10cee640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee0c0d2740 0x7fee0c1548c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee10cee640 1 -- 172.21.15.32:0/332670537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fee04009a70 con 0x7fee0c0cfab0 2023-12-20T06:04:12.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee0bfff640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee0c0d2740 0x7fee0c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.082+0000 7fee10cee640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c0cfab0 0x7fee0c15a950 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c750 tx=0x7fee0000cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.086+0000 7fee097fa640 1 -- 172.21.15.32:0/332670537 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00005ae0 con 0x7fee0c0cfab0 2023-12-20T06:04:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.086+0000 7fee097fa640 1 -- 172.21.15.32:0/332670537 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fee00005c40 con 0x7fee0c0cfab0 2023-12-20T06:04:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.086+0000 7fee097fa640 1 -- 172.21.15.32:0/332670537 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fee00007070 con 0x7fee0c0cfab0 2023-12-20T06:04:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.086+0000 7fee12f52640 1 -- 172.21.15.32:0/332670537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fedf8009420 con 0x7fee0c0cfab0 2023-12-20T06:04:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.086+0000 7fee12f52640 1 -- 172.21.15.32:0/332670537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fee0c159e70 con 0x7fee0c0cfab0 2023-12-20T06:04:12.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.086+0000 7fee12f52640 1 -- 172.21.15.32:0/332670537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fee0c04e000 con 0x7fee0c0cfab0 2023-12-20T06:04:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.086+0000 7fee097fa640 1 -- 172.21.15.32:0/332670537 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fee000071d0 con 0x7fee0c0cfab0 2023-12-20T06:04:12.089 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.086+0000 7fee097fa640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fedf0065ca0 0x7fedf0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.086+0000 7fee097fa640 1 -- 172.21.15.32:0/332670537 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 4913+0+0 (secure 0 0 0) 0x7fee00082d80 con 0x7fee0c0cfab0 2023-12-20T06:04:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.086+0000 7fee0bfff640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fedf0065ca0 0x7fedf0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:12.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.090+0000 7fee0bfff640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fedf0065ca0 0x7fedf0068160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fedf80b2000 tx=0x7fedf80aa070 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:12.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.090+0000 7fee097fa640 1 -- 172.21.15.32:0/332670537 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fee000506f0 con 0x7fee0c0cfab0 2023-12-20T06:04:12.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.250+0000 7fee12f52640 1 -- 172.21.15.32:0/332670537 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) v1 -- 0x7fee0c0cff00 con 0x7fee0c0cfab0 2023-12-20T06:04:12.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.250+0000 7fee097fa640 1 -- 172.21.15.32:0/332670537 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v247) v1 ==== 84+0+65 (secure 0 0 0) 0x7fee0c0cff00 con 0x7fee0c0cfab0 2023-12-20T06:04:12.252 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:12.252 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T06:04:12.252 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:12.252 INFO:teuthology.orchestra.run.smithi032.stdout: "poolnum": 1, 2023-12-20T06:04:12.252 INFO:teuthology.orchestra.run.smithi032.stdout: "poolname": ".mgr" 2023-12-20T06:04:12.252 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:12.252 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T06:04:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 -- 172.21.15.32:0/332670537 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fedf0065ca0 msgr2=0x7fedf0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fedf0065ca0 0x7fedf0068160 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fedf80b2000 tx=0x7fedf80aa070 comp rx=0 tx=0).stop 2023-12-20T06:04:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 -- 172.21.15.32:0/332670537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c0cfab0 msgr2=0x7fee0c15a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:12.254 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c0cfab0 0x7fee0c15a950 secure :-1 s=READY pgs=2464 cs=0 l=1 rev1=1 crypto rx=0x7fee0000c750 tx=0x7fee0000cc10 comp rx=0 tx=0).stop 2023-12-20T06:04:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 -- 172.21.15.32:0/332670537 shutdown_connections 2023-12-20T06:04:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fedf0065ca0 0x7fedf0068160 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fee0c15cdb0 0x7fee0c0cb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fee0c0d2740 0x7fee0c1548c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 --2- 172.21.15.32:0/332670537 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fee0c0cfab0 0x7fee0c15a950 unknown :-1 s=CLOSED pgs=2464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 -- 172.21.15.32:0/332670537 >> 172.21.15.32:0/332670537 conn(0x7fee0c065ee0 msgr2=0x7fee0c0cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 -- 172.21.15.32:0/332670537 shutdown_connections 2023-12-20T06:04:12.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.254+0000 7fee12f52640 1 -- 172.21.15.32:0/332670537 wait complete. 2023-12-20T06:04:12.274 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2023-12-20T06:04:12.274 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2023-12-20T06:04:12.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f652e9ce640 1 Processor -- start 2023-12-20T06:04:12.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f652e9ce640 1 -- start start 2023-12-20T06:04:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f652e9ce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f65280b1930 0x7f65280b07f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f652e9ce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f652806dd50 con 0x7f6528159af0 2023-12-20T06:04:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f652e9ce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f65280b1140 con 0x7f65280b0d30 2023-12-20T06:04:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f652e9ce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f65280b1da0 con 0x7f65280b1930 2023-12-20T06:04:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f6527fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6528159af0 0x7f652806d2d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55342/0 (socket says 172.21.15.32:55342) 2023-12-20T06:04:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f6527fff640 1 -- 172.21.15.32:0/2679374899 learned_addr learned my addr 172.21.15.32:0/2679374899 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f65277fe640 1 --2- 172.21.15.32:0/2679374899 >> v2:172.21.15.39:3300/0 conn(0x7f65280b1930 0x7f65280b07f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:12.424 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f6526ffd640 1 -- 172.21.15.32:0/2679374899 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4011101371 0 0) 0x7f652806dd50 con 0x7f6528159af0 2023-12-20T06:04:12.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f6526ffd640 1 -- 172.21.15.32:0/2679374899 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6510003530 con 0x7f6528159af0 2023-12-20T06:04:12.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f65277fe640 1 -- 172.21.15.32:0/2679374899 >> v1:172.21.15.39:6789/0 conn(0x7f65280b0d30 legacy=0x7f65280cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:12.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f65277fe640 1 -- 172.21.15.32:0/2679374899 >> v1:172.21.15.32:6789/0 conn(0x7f6528159af0 legacy=0x7f652806d2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:12.425 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.421+0000 7f65277fe640 1 -- 172.21.15.32:0/2679374899 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f65280cc940 con 0x7f65280b1930 2023-12-20T06:04:12.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.425+0000 7f65277fe640 1 --2- 172.21.15.32:0/2679374899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65280b1930 0x7f65280b07f0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f651c0097c0 tx=0x7f651c01c450 comp rx=0 tx=0).ready entity=mon.1 client_cookie=29a21a8cfc7281c3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:12.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.425+0000 7f6526ffd640 1 -- 172.21.15.32:0/2679374899 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c0ab650 con 0x7f65280b1930 2023-12-20T06:04:12.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.425+0000 7f6526ffd640 1 -- 172.21.15.32:0/2679374899 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f651c0ab7b0 con 0x7f65280b1930 2023-12-20T06:04:12.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.425+0000 7f6526ffd640 1 -- 172.21.15.32:0/2679374899 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f651c0abb20 con 0x7f65280b1930 2023-12-20T06:04:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.425+0000 7f652e9ce640 1 -- 172.21.15.32:0/2679374899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65280b1930 msgr2=0x7f65280b07f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.425+0000 7f652e9ce640 1 --2- 172.21.15.32:0/2679374899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65280b1930 0x7f65280b07f0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7f651c0097c0 tx=0x7f651c01c450 comp rx=0 tx=0).stop 2023-12-20T06:04:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.425+0000 7f652e9ce640 1 -- 172.21.15.32:0/2679374899 shutdown_connections 2023-12-20T06:04:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.425+0000 7f652e9ce640 1 --2- 172.21.15.32:0/2679374899 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65280b1930 0x7f65280b07f0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.425+0000 7f652e9ce640 1 -- 172.21.15.32:0/2679374899 >> 172.21.15.32:0/2679374899 conn(0x7f652806a510 msgr2=0x7f65280ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:12.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.425+0000 7f652e9ce640 1 -- 172.21.15.32:0/2679374899 shutdown_connections 2023-12-20T06:04:12.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.425+0000 7f652e9ce640 1 -- 172.21.15.32:0/2679374899 wait complete. 2023-12-20T06:04:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652e9ce640 1 Processor -- start 2023-12-20T06:04:12.430 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652e9ce640 1 -- start start 2023-12-20T06:04:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652e9ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65280b0d30 0x7f65280c8160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652e9ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65280b1930 0x7f652813a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652e9ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528159af0 0x7f65280c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652e9ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f65281457a0 con 0x7f6528159af0 2023-12-20T06:04:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652e9ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6528145220 con 0x7f65280b0d30 2023-12-20T06:04:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652e9ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f65281454c0 con 0x7f65280b1930 2023-12-20T06:04:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652cf6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528159af0 0x7f65280c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652cf6b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528159af0 0x7f65280c4ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49956/0 (socket says 172.21.15.32:49956) 2023-12-20T06:04:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652cf6b640 1 -- 172.21.15.32:0/3303974628 learned_addr learned my addr 172.21.15.32:0/3303974628 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:12.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f6527fff640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65280b0d30 0x7f65280c8160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:12.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f65277fe640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65280b1930 0x7f652813a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:12.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652cf6b640 1 -- 172.21.15.32:0/3303974628 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65280b1930 msgr2=0x7f652813a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:12.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652cf6b640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65280b1930 0x7f652813a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652cf6b640 1 -- 172.21.15.32:0/3303974628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65280b0d30 msgr2=0x7f65280c8160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:12.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652cf6b640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65280b0d30 0x7f65280c8160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:12.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652cf6b640 1 -- 172.21.15.32:0/3303974628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f651c009420 con 0x7f6528159af0 2023-12-20T06:04:12.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652cf6b640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528159af0 0x7f65280c4ab0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f652000c710 tx=0x7f652000cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:12.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f6524ff9640 1 -- 172.21.15.32:0/3303974628 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520006d60 con 0x7f6528159af0 2023-12-20T06:04:12.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f6524ff9640 1 -- 172.21.15.32:0/3303974628 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f652000ce20 con 0x7f6528159af0 2023-12-20T06:04:12.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f652e9ce640 1 -- 172.21.15.32:0/3303974628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f65280c7610 con 0x7f6528159af0 2023-12-20T06:04:12.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.429+0000 7f6524ff9640 1 -- 172.21.15.32:0/3303974628 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6520003bc0 con 0x7f6528159af0 2023-12-20T06:04:12.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.433+0000 7f652e9ce640 1 -- 172.21.15.32:0/3303974628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f65280c86a0 con 0x7f6528159af0 2023-12-20T06:04:12.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.433+0000 7f652e9ce640 1 -- 172.21.15.32:0/3303974628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f65280c7470 con 0x7f6528159af0 2023-12-20T06:04:12.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.433+0000 7f6524ff9640 1 -- 172.21.15.32:0/3303974628 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f6520003d20 con 0x7f6528159af0 2023-12-20T06:04:12.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.433+0000 7f6524ff9640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6514065cd0 0x7f6514068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:12.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.433+0000 7f6524ff9640 1 -- 172.21.15.32:0/3303974628 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(247..247 src has 1..247) v4 ==== 4913+0+0 (secure 0 0 0) 0x7f6520082d30 con 0x7f6528159af0 2023-12-20T06:04:12.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.433+0000 7f6527fff640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6514065cd0 0x7f6514068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:12.436 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.433+0000 7f6527fff640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6514065cd0 0x7f6514068190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f65180011f0 tx=0x7f6518007d40 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:12.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.437+0000 7f6524ff9640 1 -- 172.21.15.32:0/3303974628 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f65280c7470 con 0x7f6528159af0 2023-12-20T06:04:12.597 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:12.597+0000 7f652e9ce640 1 -- 172.21.15.32:0/3303974628 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) v1 -- 0x7f65280c7470 con 0x7f6528159af0 2023-12-20T06:04:12.810 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:04:12.814 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:04:13.580 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.577+0000 7f6524ff9640 1 -- 172.21.15.32:0/3303974628 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v248) v1 ==== 138+0+0 (secure 0 0 0) 0x7f65280c7470 con 0x7f6528159af0 2023-12-20T06:04:13.580 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' created 2023-12-20T06:04:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 -- 172.21.15.32:0/3303974628 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6514065cd0 msgr2=0x7f6514068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6514065cd0 0x7f6514068190 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f65180011f0 tx=0x7f6518007d40 comp rx=0 tx=0).stop 2023-12-20T06:04:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 -- 172.21.15.32:0/3303974628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528159af0 msgr2=0x7f65280c4ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528159af0 0x7f65280c4ab0 secure :-1 s=READY pgs=2466 cs=0 l=1 rev1=1 crypto rx=0x7f652000c710 tx=0x7f652000cbd0 comp rx=0 tx=0).stop 2023-12-20T06:04:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 -- 172.21.15.32:0/3303974628 shutdown_connections 2023-12-20T06:04:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6514065cd0 0x7f6514068190 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:13.586 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6528159af0 0x7f65280c4ab0 unknown :-1 s=CLOSED pgs=2466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f65280b1930 0x7f652813a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 --2- 172.21.15.32:0/3303974628 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f65280b0d30 0x7f65280c8160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:13.587 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 -- 172.21.15.32:0/3303974628 >> 172.21.15.32:0/3303974628 conn(0x7f652806a510 msgr2=0x7f6528158100 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 -- 172.21.15.32:0/3303974628 shutdown_connections 2023-12-20T06:04:13.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.585+0000 7f652e9ce640 1 -- 172.21.15.32:0/3303974628 wait complete. 2023-12-20T06:04:13.612 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.9 2023-12-20T06:04:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.765+0000 7fa4f61ce640 1 Processor -- start 2023-12-20T06:04:13.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.765+0000 7fa4f61ce640 1 -- start start 2023-12-20T06:04:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.765+0000 7fa4f61ce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4f00adf00 0x7fa4f00cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.765+0000 7fa4f61ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4f00c4320 0x7fa4f00cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.765+0000 7fa4f61ce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa4f0159e60 con 0x7fa4f006da40 2023-12-20T06:04:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.765+0000 7fa4f61ce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa4f00c3630 con 0x7fa4f00adf00 2023-12-20T06:04:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.765+0000 7fa4f61ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00ce380 con 0x7fa4f00c4320 2023-12-20T06:04:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.765+0000 7fa4ef7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4f00adf00 0x7fa4f00cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:13.768 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.765+0000 7fa4ef7fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa4f00adf00 0x7fa4f00cd850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49970/0 (socket says 172.21.15.32:49970) 2023-12-20T06:04:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.765+0000 7fa4ef7fe640 1 -- 172.21.15.32:0/3249286125 learned_addr learned my addr 172.21.15.32:0/3249286125 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.769+0000 7fa4eeffd640 1 --2- 172.21.15.32:0/3249286125 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4f00c4320 0x7fa4f00cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.769+0000 7fa4ef7fe640 1 -- 172.21.15.32:0/3249286125 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4f00c4320 msgr2=0x7fa4f00cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.769+0000 7fa4ef7fe640 1 --2- 172.21.15.32:0/3249286125 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4f00c4320 0x7fa4f00cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.769+0000 7fa4ef7fe640 1 -- 172.21.15.32:0/3249286125 >> v1:172.21.15.39:6789/0 conn(0x7fa4f006da40 legacy=0x7fa4f015d550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:13.769 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.769+0000 7fa4ef7fe640 1 -- 172.21.15.32:0/3249286125 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4f00c3840 con 0x7fa4f00adf00 2023-12-20T06:04:13.771 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.769+0000 7fa4ef7fe640 1 --2- 172.21.15.32:0/3249286125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4f00adf00 0x7fa4f00cd850 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009870 tx=0x7fa4e001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af3e78ef9ff444ad server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.769+0000 7fa4ee7fc640 1 -- 172.21.15.32:0/3249286125 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00aa5f0 con 0x7fa4f00adf00 2023-12-20T06:04:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.769+0000 7fa4ee7fc640 1 -- 172.21.15.32:0/3249286125 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e00aa750 con 0x7fa4f00adf00 2023-12-20T06:04:13.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.769+0000 7fa4ee7fc640 1 -- 172.21.15.32:0/3249286125 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e00aaae0 con 0x7fa4f00adf00 2023-12-20T06:04:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.769+0000 7fa4f61ce640 1 -- 172.21.15.32:0/3249286125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4f00adf00 msgr2=0x7fa4f00cd850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.769+0000 7fa4f61ce640 1 --2- 172.21.15.32:0/3249286125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4f00adf00 0x7fa4f00cd850 secure :-1 s=READY pgs=2467 cs=0 l=1 rev1=1 crypto rx=0x7fa4e0009870 tx=0x7fa4e001c4d0 comp rx=0 tx=0).stop 2023-12-20T06:04:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 -- 172.21.15.32:0/3249286125 shutdown_connections 2023-12-20T06:04:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 --2- 172.21.15.32:0/3249286125 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4f00c4320 0x7fa4f00cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 --2- 172.21.15.32:0/3249286125 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4f00adf00 0x7fa4f00cd850 unknown :-1 s=CLOSED pgs=2467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 -- 172.21.15.32:0/3249286125 >> 172.21.15.32:0/3249286125 conn(0x7fa4f006a4b0 msgr2=0x7fa4f00b03c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 -- 172.21.15.32:0/3249286125 shutdown_connections 2023-12-20T06:04:13.773 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 -- 172.21.15.32:0/3249286125 wait complete. 2023-12-20T06:04:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 Processor -- start 2023-12-20T06:04:13.774 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 -- start start 2023-12-20T06:04:13.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4f006da40 0x7fa4f015aa40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:13.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4f00adf00 0x7fa4f0155e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:13.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4f00c4320 0x7fa4f00c7710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:13.776 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f00c7c50 con 0x7fa4f00adf00 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f015b0d0 con 0x7fa4f006da40 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4f61ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa4f015b3b0 con 0x7fa4f00c4320 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4eeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4f00adf00 0x7fa4f0155e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4eeffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4f00adf00 0x7fa4f0155e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49978/0 (socket says 172.21.15.32:49978) 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4eeffd640 1 -- 172.21.15.32:0/2780542189 learned_addr learned my addr 172.21.15.32:0/2780542189 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4ef7fe640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4f006da40 0x7fa4f015aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4effff640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4f00c4320 0x7fa4f00c7710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4eeffd640 1 -- 172.21.15.32:0/2780542189 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4f00c4320 msgr2=0x7fa4f00c7710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4eeffd640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4f00c4320 0x7fa4f00c7710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4eeffd640 1 -- 172.21.15.32:0/2780542189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4f006da40 msgr2=0x7fa4f015aa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4eeffd640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4f006da40 0x7fa4f015aa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.773+0000 7fa4eeffd640 1 -- 172.21.15.32:0/2780542189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa4e00094d0 con 0x7fa4f00adf00 2023-12-20T06:04:13.777 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.777+0000 7fa4eeffd640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4f00adf00 0x7fa4f0155e60 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400d5f0 tx=0x7fa4e400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:13.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.777+0000 7fa4d7fff640 1 -- 172.21.15.32:0/2780542189 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e40125f0 con 0x7fa4f00adf00 2023-12-20T06:04:13.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.777+0000 7fa4d7fff640 1 -- 172.21.15.32:0/2780542189 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa4e4012750 con 0x7fa4f00adf00 2023-12-20T06:04:13.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.777+0000 7fa4f61ce640 1 -- 172.21.15.32:0/2780542189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa4f015b690 con 0x7fa4f00adf00 2023-12-20T06:04:13.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.777+0000 7fa4d7fff640 1 -- 172.21.15.32:0/2780542189 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa4e4012ae0 con 0x7fa4f00adf00 2023-12-20T06:04:13.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.777+0000 7fa4f61ce640 1 -- 172.21.15.32:0/2780542189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa4f015bc50 con 0x7fa4f00adf00 2023-12-20T06:04:13.778 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.777+0000 7fa4f61ce640 1 -- 172.21.15.32:0/2780542189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa4f015be10 con 0x7fa4f00adf00 2023-12-20T06:04:13.779 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.777+0000 7fa4d7fff640 1 -- 172.21.15.32:0/2780542189 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fa4e4012c40 con 0x7fa4f00adf00 2023-12-20T06:04:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.777+0000 7fa4d7fff640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa4dc065ca0 0x7fa4dc068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.777+0000 7fa4d7fff640 1 -- 172.21.15.32:0/2780542189 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(248..248 src has 1..248) v4 ==== 5283+0+0 (secure 0 0 0) 0x7fa4e40844e0 con 0x7fa4f00adf00 2023-12-20T06:04:13.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.777+0000 7fa4ef7fe640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa4dc065ca0 0x7fa4dc068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:13.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.781+0000 7fa4ef7fe640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa4dc065ca0 0x7fa4dc068160 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00ae000 tx=0x7fa4e00a7000 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:13.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.781+0000 7fa4d7fff640 1 -- 172.21.15.32:0/2780542189 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa4e4051c40 con 0x7fa4f00adf00 2023-12-20T06:04:13.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:13.941+0000 7fa4f61ce640 1 -- 172.21.15.32:0/2780542189 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9} v 0) v1 -- 0x7fa4f004e000 con 0x7fa4f00adf00 2023-12-20T06:04:14.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.585+0000 7fa4d7fff640 1 -- 172.21.15.32:0/2780542189 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data' created v249) v1 ==== 170+0+0 (secure 0 0 0) 0x7fa4e4055260 con 0x7fa4f00adf00 2023-12-20T06:04:14.589 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' created 2023-12-20T06:04:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 -- 172.21.15.32:0/2780542189 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa4dc065ca0 msgr2=0x7fa4dc068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa4dc065ca0 0x7fa4dc068160 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7fa4e00ae000 tx=0x7fa4e00a7000 comp rx=0 tx=0).stop 2023-12-20T06:04:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 -- 172.21.15.32:0/2780542189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4f00adf00 msgr2=0x7fa4f0155e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4f00adf00 0x7fa4f0155e60 secure :-1 s=READY pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400d5f0 tx=0x7fa4e400dab0 comp rx=0 tx=0).stop 2023-12-20T06:04:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 -- 172.21.15.32:0/2780542189 shutdown_connections 2023-12-20T06:04:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa4dc065ca0 0x7fa4dc068160 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa4f00c4320 0x7fa4f00c7710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:14.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa4f00adf00 0x7fa4f0155e60 secure :-1 s=CLOSED pgs=2468 cs=0 l=1 rev1=1 crypto rx=0x7fa4e400d5f0 tx=0x7fa4e400dab0 comp rx=0 tx=0).stop 2023-12-20T06:04:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 --2- 172.21.15.32:0/2780542189 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa4f006da40 0x7fa4f015aa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 -- 172.21.15.32:0/2780542189 >> 172.21.15.32:0/2780542189 conn(0x7fa4f006a4b0 msgr2=0x7fa4f0070fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 -- 172.21.15.32:0/2780542189 shutdown_connections 2023-12-20T06:04:14.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.597+0000 7fa4f61ce640 1 -- 172.21.15.32:0/2780542189 wait complete. 2023-12-20T06:04:14.633 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2023-12-20T06:04:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.777+0000 7f55c2391640 1 Processor -- start 2023-12-20T06:04:14.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.777+0000 7f55c2391640 1 -- start start 2023-12-20T06:04:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55c2391640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f55bc071c70 0x7f55bc070180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:14.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55c2391640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0cc800 con 0x7f55bc072050 2023-12-20T06:04:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55c2391640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f55bc0c95c0 con 0x7f55bc0caf70 2023-12-20T06:04:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55c2391640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f55bc0c98a0 con 0x7f55bc071c70 2023-12-20T06:04:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55c0b8e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f55bc072050 0x7f55bc165560 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55344/0 (socket says 172.21.15.32:55344) 2023-12-20T06:04:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55c0b8e640 1 -- 172.21.15.32:0/1708982344 learned_addr learned my addr 172.21.15.32:0/1708982344 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55c138f640 1 --2- 172.21.15.32:0/1708982344 >> v2:172.21.15.39:3300/0 conn(0x7f55bc071c70 0x7f55bc070180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55b3fff640 1 -- 172.21.15.32:0/1708982344 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2268748182 0 0) 0x7f55bc0cc800 con 0x7f55bc072050 2023-12-20T06:04:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55b3fff640 1 -- 172.21.15.32:0/1708982344 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f55a8003530 con 0x7f55bc072050 2023-12-20T06:04:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55c138f640 1 -- 172.21.15.32:0/1708982344 >> v1:172.21.15.39:6789/0 conn(0x7f55bc0caf70 legacy=0x7f55bc0cd2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:14.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55c138f640 1 -- 172.21.15.32:0/1708982344 >> v1:172.21.15.32:6789/0 conn(0x7f55bc072050 legacy=0x7f55bc165560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:14.784 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.781+0000 7f55c138f640 1 -- 172.21.15.32:0/1708982344 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55bc0ccaa0 con 0x7f55bc071c70 2023-12-20T06:04:14.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55c138f640 1 --2- 172.21.15.32:0/1708982344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55bc071c70 0x7f55bc070180 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f55b80097c0 tx=0x7f55b801ec70 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5ad9eefbc84b6ca2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55b3fff640 1 -- 172.21.15.32:0/1708982344 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80a8970 con 0x7f55bc071c70 2023-12-20T06:04:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55b3fff640 1 -- 172.21.15.32:0/1708982344 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55b80a8ad0 con 0x7f55bc071c70 2023-12-20T06:04:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55b3fff640 1 -- 172.21.15.32:0/1708982344 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55b80a8e40 con 0x7f55bc071c70 2023-12-20T06:04:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55c2391640 1 -- 172.21.15.32:0/1708982344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55bc071c70 msgr2=0x7f55bc070180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55c2391640 1 --2- 172.21.15.32:0/1708982344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55bc071c70 0x7f55bc070180 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f55b80097c0 tx=0x7f55b801ec70 comp rx=0 tx=0).stop 2023-12-20T06:04:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55c2391640 1 -- 172.21.15.32:0/1708982344 shutdown_connections 2023-12-20T06:04:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55c2391640 1 --2- 172.21.15.32:0/1708982344 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55bc071c70 0x7f55bc070180 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55c2391640 1 -- 172.21.15.32:0/1708982344 >> 172.21.15.32:0/1708982344 conn(0x7f55bc06a4e0 msgr2=0x7f55bc06bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55c2391640 1 -- 172.21.15.32:0/1708982344 shutdown_connections 2023-12-20T06:04:14.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55c2391640 1 -- 172.21.15.32:0/1708982344 wait complete. 2023-12-20T06:04:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55c2391640 1 Processor -- start 2023-12-20T06:04:14.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.785+0000 7f55c2391640 1 -- start start 2023-12-20T06:04:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c2391640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f55bc071c70 0x7f55bc156ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c2391640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc072050 0x7f55bc1606b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c2391640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55bc0caf70 0x7f55bc0c84a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c2391640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc157000 con 0x7f55bc072050 2023-12-20T06:04:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c2391640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc164680 con 0x7f55bc0caf70 2023-12-20T06:04:14.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c2391640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f55bc164960 con 0x7f55bc071c70 2023-12-20T06:04:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c0b8e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc072050 0x7f55bc1606b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c0b8e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc072050 0x7f55bc1606b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49986/0 (socket says 172.21.15.32:49986) 2023-12-20T06:04:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c0b8e640 1 -- 172.21.15.32:0/2951728851 learned_addr learned my addr 172.21.15.32:0/2951728851 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c138f640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f55bc071c70 0x7f55bc156ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c1b90640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55bc0caf70 0x7f55bc0c84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c0b8e640 1 -- 172.21.15.32:0/2951728851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f55bc071c70 msgr2=0x7f55bc156ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c0b8e640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f55bc071c70 0x7f55bc156ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c0b8e640 1 -- 172.21.15.32:0/2951728851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55bc0caf70 msgr2=0x7f55bc0c84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c0b8e640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55bc0caf70 0x7f55bc0c84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:14.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c0b8e640 1 -- 172.21.15.32:0/2951728851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f55b8009420 con 0x7f55bc072050 2023-12-20T06:04:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c138f640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f55bc071c70 0x7f55bc156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c0b8e640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc072050 0x7f55bc1606b0 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00c4d0 tx=0x7f55ac00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55b1ffb640 1 -- 172.21.15.32:0/2951728851 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac005cf0 con 0x7f55bc072050 2023-12-20T06:04:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55b1ffb640 1 -- 172.21.15.32:0/2951728851 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f55ac005e50 con 0x7f55bc072050 2023-12-20T06:04:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c2391640 1 -- 172.21.15.32:0/2951728851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f55bc164c40 con 0x7f55bc072050 2023-12-20T06:04:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55b1ffb640 1 -- 172.21.15.32:0/2951728851 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f55ac007270 con 0x7f55bc072050 2023-12-20T06:04:14.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c2391640 1 -- 172.21.15.32:0/2951728851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f55bc160bf0 con 0x7f55bc072050 2023-12-20T06:04:14.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.789+0000 7f55c2391640 1 -- 172.21.15.32:0/2951728851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f55bc1650f0 con 0x7f55bc072050 2023-12-20T06:04:14.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.793+0000 7f55b1ffb640 1 -- 172.21.15.32:0/2951728851 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f55ac0073d0 con 0x7f55bc072050 2023-12-20T06:04:14.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.793+0000 7f55b1ffb640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55a0065ca0 0x7f55a0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.793+0000 7f55b1ffb640 1 -- 172.21.15.32:0/2951728851 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(249..249 src has 1..249) v4 ==== 5665+0+0 (secure 0 0 0) 0x7f55ac082b50 con 0x7f55bc072050 2023-12-20T06:04:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.793+0000 7f55c138f640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55a0065ca0 0x7f55a0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:14.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.793+0000 7f55c138f640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55a0065ca0 0x7f55a0068160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f55b80b2000 tx=0x7f55b80aa070 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:14.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.797+0000 7f55b1ffb640 1 -- 172.21.15.32:0/2951728851 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f55ac0501e0 con 0x7f55bc072050 2023-12-20T06:04:14.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.985+0000 7f55c2391640 1 -- 172.21.15.32:0/2951728851 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) v1 -- 0x7f55bc074100 con 0x7f55bc072050 2023-12-20T06:04:14.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:14.993+0000 7f55b1ffb640 1 -- 172.21.15.32:0/2951728851 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 32 and data pool 33 v124) v1 ==== 174+0+0 (secure 0 0 0) 0x7f55ac017020 con 0x7f55bc072050 2023-12-20T06:04:15.000 INFO:teuthology.orchestra.run.smithi032.stderr:new fs with metadata pool 32 and data pool 33 2023-12-20T06:04:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.001+0000 7f55c2391640 1 -- 172.21.15.32:0/2951728851 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55a0065ca0 msgr2=0x7f55a0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.001+0000 7f55c2391640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55a0065ca0 0x7f55a0068160 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f55b80b2000 tx=0x7f55b80aa070 comp rx=0 tx=0).stop 2023-12-20T06:04:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.001+0000 7f55c2391640 1 -- 172.21.15.32:0/2951728851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc072050 msgr2=0x7f55bc1606b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.001+0000 7f55c2391640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc072050 0x7f55bc1606b0 secure :-1 s=READY pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00c4d0 tx=0x7f55ac00c990 comp rx=0 tx=0).stop 2023-12-20T06:04:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.001+0000 7f55c2391640 1 -- 172.21.15.32:0/2951728851 shutdown_connections 2023-12-20T06:04:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.001+0000 7f55c2391640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f55a0065ca0 0x7f55a0068160 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.001+0000 7f55c2391640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f55bc0caf70 0x7f55bc0c84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.001+0000 7f55c2391640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f55bc072050 0x7f55bc1606b0 secure :-1 s=CLOSED pgs=2470 cs=0 l=1 rev1=1 crypto rx=0x7f55ac00c4d0 tx=0x7f55ac00c990 comp rx=0 tx=0).stop 2023-12-20T06:04:15.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.001+0000 7f55c2391640 1 --2- 172.21.15.32:0/2951728851 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f55bc071c70 0x7f55bc156ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.001+0000 7f55c2391640 1 -- 172.21.15.32:0/2951728851 >> 172.21.15.32:0/2951728851 conn(0x7f55bc06a4e0 msgr2=0x7f55bc0b3bb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:15.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.005+0000 7f55c2391640 1 -- 172.21.15.32:0/2951728851 shutdown_connections 2023-12-20T06:04:15.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.005+0000 7f55c2391640 1 -- 172.21.15.32:0/2951728851 wait complete. 2023-12-20T06:04:15.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:04:15.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.189+0000 7f08ca9bc640 1 Processor -- start 2023-12-20T06:04:15.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.189+0000 7f08ca9bc640 1 -- start start 2023-12-20T06:04:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.189+0000 7f08ca9bc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08c406ce60 0x7f08c415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.189+0000 7f08ca9bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08c415f110 0x7f08c415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.189+0000 7f08ca9bc640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f08c415bbd0 con 0x7f08c40af5e0 2023-12-20T06:04:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.189+0000 7f08ca9bc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f08c415beb0 con 0x7f08c406ce60 2023-12-20T06:04:15.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.189+0000 7f08ca9bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c415e910 con 0x7f08c415f110 2023-12-20T06:04:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08c406ce60 0x7f08c415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c37fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f08c406ce60 0x7f08c415b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:49990/0 (socket says 172.21.15.32:49990) 2023-12-20T06:04:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c37fe640 1 -- 172.21.15.32:0/462253947 learned_addr learned my addr 172.21.15.32:0/462253947 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c8f59640 1 --2- 172.21.15.32:0/462253947 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08c415f110 0x7f08c415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c37fe640 1 -- 172.21.15.32:0/462253947 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08c415f110 msgr2=0x7f08c415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c37fe640 1 --2- 172.21.15.32:0/462253947 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08c415f110 0x7f08c415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c37fe640 1 -- 172.21.15.32:0/462253947 >> v1:172.21.15.39:6789/0 conn(0x7f08c40af5e0 legacy=0x7f08c40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c37fe640 1 -- 172.21.15.32:0/462253947 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08c415ebf0 con 0x7f08c406ce60 2023-12-20T06:04:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c37fe640 1 --2- 172.21.15.32:0/462253947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08c406ce60 0x7f08c415b280 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009870 tx=0x7f08b801c4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fee2d3cbc00c40b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:15.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c2ffd640 1 -- 172.21.15.32:0/462253947 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08c415b9d0 con 0x7f08c406ce60 2023-12-20T06:04:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c2ffd640 1 -- 172.21.15.32:0/462253947 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08b80ab650 con 0x7f08c406ce60 2023-12-20T06:04:15.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.193+0000 7f08c2ffd640 1 -- 172.21.15.32:0/462253947 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08b80ab9e0 con 0x7f08c406ce60 2023-12-20T06:04:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 -- 172.21.15.32:0/462253947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08c406ce60 msgr2=0x7f08c415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 --2- 172.21.15.32:0/462253947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08c406ce60 0x7f08c415b280 secure :-1 s=READY pgs=2471 cs=0 l=1 rev1=1 crypto rx=0x7f08b8009870 tx=0x7f08b801c4b0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 -- 172.21.15.32:0/462253947 shutdown_connections 2023-12-20T06:04:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 --2- 172.21.15.32:0/462253947 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08c415f110 0x7f08c415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 --2- 172.21.15.32:0/462253947 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08c406ce60 0x7f08c415b280 unknown :-1 s=CLOSED pgs=2471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 -- 172.21.15.32:0/462253947 >> 172.21.15.32:0/462253947 conn(0x7f08c406a320 msgr2=0x7f08c40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:15.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 -- 172.21.15.32:0/462253947 shutdown_connections 2023-12-20T06:04:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 -- 172.21.15.32:0/462253947 wait complete. 2023-12-20T06:04:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 Processor -- start 2023-12-20T06:04:15.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 -- start start 2023-12-20T06:04:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08c406ce60 0x7f08c414fce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08c40af5e0 0x7f08c40b1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08c415f110 0x7f08c415a000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c4159be0 con 0x7f08c415f110 2023-12-20T06:04:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f08c4150220 con 0x7f08c406ce60 2023-12-20T06:04:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08ca9bc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f08c40c5960 con 0x7f08c40af5e0 2023-12-20T06:04:15.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08c8f59640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08c415f110 0x7f08c415a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08c8f59640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08c415f110 0x7f08c415a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50002/0 (socket says 172.21.15.32:50002) 2023-12-20T06:04:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08c8f59640 1 -- 172.21.15.32:0/3728013633 learned_addr learned my addr 172.21.15.32:0/3728013633 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08c37fe640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08c40af5e0 0x7f08c40b1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.197+0000 7f08c3fff640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08c406ce60 0x7f08c414fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c8f59640 1 -- 172.21.15.32:0/3728013633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08c40af5e0 msgr2=0x7f08c40b1aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c8f59640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08c40af5e0 0x7f08c40b1aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c8f59640 1 -- 172.21.15.32:0/3728013633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08c406ce60 msgr2=0x7f08c414fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c8f59640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08c406ce60 0x7f08c414fce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c8f59640 1 -- 172.21.15.32:0/3728013633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f08b80094d0 con 0x7f08c415f110 2023-12-20T06:04:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c37fe640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08c40af5e0 0x7f08c40b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:15.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c8f59640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08c415f110 0x7f08c415a000 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00efc0 tx=0x7f08bc00c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c0ff9640 1 -- 172.21.15.32:0/3728013633 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc009ba0 con 0x7f08c415f110 2023-12-20T06:04:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c0ff9640 1 -- 172.21.15.32:0/3728013633 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f08bc00f040 con 0x7f08c415f110 2023-12-20T06:04:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c0ff9640 1 -- 172.21.15.32:0/3728013633 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f08bc0146c0 con 0x7f08c415f110 2023-12-20T06:04:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08ca9bc640 1 -- 172.21.15.32:0/3728013633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f08c40c5c40 con 0x7f08c415f110 2023-12-20T06:04:15.204 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08ca9bc640 1 -- 172.21.15.32:0/3728013633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f08c4159800 con 0x7f08c415f110 2023-12-20T06:04:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08ca9bc640 1 -- 172.21.15.32:0/3728013633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f08c4159d20 con 0x7f08c415f110 2023-12-20T06:04:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c0ff9640 1 -- 172.21.15.32:0/3728013633 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f08bc014820 con 0x7f08c415f110 2023-12-20T06:04:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c0ff9640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f08b0065cd0 0x7f08b0068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.201+0000 7f08c0ff9640 1 -- 172.21.15.32:0/3728013633 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f08bc0840f0 con 0x7f08c415f110 2023-12-20T06:04:15.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.205+0000 7f08c3fff640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f08b0065cd0 0x7f08b0068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.205+0000 7f08c3fff640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f08b0065cd0 0x7f08b0068190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08b4000fc0 tx=0x7f08b4009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:15.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.205+0000 7f08c0ff9640 1 -- 172.21.15.32:0/3728013633 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f08c4159d20 con 0x7f08c415f110 2023-12-20T06:04:15.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 -- 172.21.15.32:0/3728013633 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f08c4159d20 con 0x7f08c415f110 2023-12-20T06:04:15.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08c0ff9640 1 -- 172.21.15.32:0/3728013633 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v250) v1 ==== 74+0+16110 (secure 0 0 0) 0x7f08c4159d20 con 0x7f08c415f110 2023-12-20T06:04:15.371 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:15.371 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":250,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:04:14.992347+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:04:12.651034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"250","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-20T06:04:13.997753+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"250","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:04:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 -- 172.21.15.32:0/3728013633 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f08b0065cd0 msgr2=0x7f08b0068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f08b0065cd0 0x7f08b0068190 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f08b4000fc0 tx=0x7f08b4009040 comp rx=0 tx=0).stop 2023-12-20T06:04:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 -- 172.21.15.32:0/3728013633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08c415f110 msgr2=0x7f08c415a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08c415f110 0x7f08c415a000 secure :-1 s=READY pgs=2472 cs=0 l=1 rev1=1 crypto rx=0x7f08bc00efc0 tx=0x7f08bc00c490 comp rx=0 tx=0).stop 2023-12-20T06:04:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 -- 172.21.15.32:0/3728013633 shutdown_connections 2023-12-20T06:04:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f08b0065cd0 0x7f08b0068190 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f08c415f110 0x7f08c415a000 unknown :-1 s=CLOSED pgs=2472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f08c40af5e0 0x7f08c40b1aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 --2- 172.21.15.32:0/3728013633 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f08c406ce60 0x7f08c414fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 -- 172.21.15.32:0/3728013633 >> 172.21.15.32:0/3728013633 conn(0x7f08c406a320 msgr2=0x7f08c40cb170 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 -- 172.21.15.32:0/3728013633 shutdown_connections 2023-12-20T06:04:15.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.369+0000 7f08ca9bc640 1 -- 172.21.15.32:0/3728013633 wait complete. 2023-12-20T06:04:15.393 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:04:15.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.533+0000 7fe9f4fa6640 1 Processor -- start 2023-12-20T06:04:15.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.533+0000 7fe9f4fa6640 1 -- start start 2023-12-20T06:04:15.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9f4fa6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fe9f01599e0 0x7fe9f00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9f4fa6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9f00b0d00 0x7fe9f00701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9f4fa6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fe9f00c98c0 con 0x7fe9f00c90e0 2023-12-20T06:04:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9f4fa6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f00c9ba0 con 0x7fe9f00b0d00 2023-12-20T06:04:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9f4fa6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fe9f00c9e80 con 0x7fe9f01599e0 2023-12-20T06:04:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9eed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9f00b0d00 0x7fe9f00701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9eed9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fe9f00b0d00 0x7fe9f00701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50014/0 (socket says 172.21.15.32:50014) 2023-12-20T06:04:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9eed9d640 1 -- 172.21.15.32:0/3833103504 learned_addr learned my addr 172.21.15.32:0/3833103504 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9ee59c640 1 --2- 172.21.15.32:0/3833103504 >> v2:172.21.15.39:3300/0 conn(0x7fe9f01599e0 0x7fe9f00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9eed9d640 1 -- 172.21.15.32:0/3833103504 >> v2:172.21.15.39:3300/0 conn(0x7fe9f01599e0 msgr2=0x7fe9f00aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:15.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9eed9d640 1 --2- 172.21.15.32:0/3833103504 >> v2:172.21.15.39:3300/0 conn(0x7fe9f01599e0 0x7fe9f00aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9eed9d640 1 -- 172.21.15.32:0/3833103504 >> v1:172.21.15.39:6789/0 conn(0x7fe9f00c90e0 legacy=0x7fe9f00b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.540 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.537+0000 7fe9eed9d640 1 -- 172.21.15.32:0/3833103504 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9f00ca160 con 0x7fe9f00b0d00 2023-12-20T06:04:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9eed9d640 1 --2- 172.21.15.32:0/3833103504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9f00b0d00 0x7fe9f00701c0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fe9f006e3c0 tx=0x7fe9e801ebb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e3b0d09f4c050bd server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:15.541 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9edd9b640 1 -- 172.21.15.32:0/3833103504 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80a7aa0 con 0x7fe9f00b0d00 2023-12-20T06:04:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9edd9b640 1 -- 172.21.15.32:0/3833103504 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e80a7c00 con 0x7fe9f00b0d00 2023-12-20T06:04:15.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9edd9b640 1 -- 172.21.15.32:0/3833103504 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e80af3f0 con 0x7fe9f00b0d00 2023-12-20T06:04:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/3833103504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9f00b0d00 msgr2=0x7fe9f00701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9f4fa6640 1 --2- 172.21.15.32:0/3833103504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9f00b0d00 0x7fe9f00701c0 secure :-1 s=READY pgs=2473 cs=0 l=1 rev1=1 crypto rx=0x7fe9f006e3c0 tx=0x7fe9e801ebb0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/3833103504 shutdown_connections 2023-12-20T06:04:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9f4fa6640 1 --2- 172.21.15.32:0/3833103504 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9f00b0d00 0x7fe9f00701c0 unknown :-1 s=CLOSED pgs=2473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9f4fa6640 1 --2- 172.21.15.32:0/3833103504 >> v2:172.21.15.39:3300/0 conn(0x7fe9f01599e0 0x7fe9f00aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/3833103504 >> 172.21.15.32:0/3833103504 conn(0x7fe9f006a320 msgr2=0x7fe9f00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/3833103504 shutdown_connections 2023-12-20T06:04:15.543 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/3833103504 wait complete. 2023-12-20T06:04:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9f4fa6640 1 Processor -- start 2023-12-20T06:04:15.544 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.541+0000 7fe9f4fa6640 1 -- start start 2023-12-20T06:04:15.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9f4fa6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9f00b0d00 0x7fe9f00c70f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9f4fa6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f00c90e0 0x7fe9f0154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9f4fa6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe9f01599e0 0x7fe9f0155450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9f4fa6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f014c800 con 0x7fe9f00b0d00 2023-12-20T06:04:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9f4fa6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00b12e0 con 0x7fe9f00c90e0 2023-12-20T06:04:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9f4fa6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fe9f00c7b20 con 0x7fe9f01599e0 2023-12-20T06:04:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9ee59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9f00b0d00 0x7fe9f00c70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9ee59c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9f00b0d00 0x7fe9f00c70f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50030/0 (socket says 172.21.15.32:50030) 2023-12-20T06:04:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9ee59c640 1 -- 172.21.15.32:0/1461461669 learned_addr learned my addr 172.21.15.32:0/1461461669 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:15.547 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9eed9d640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe9f01599e0 0x7fe9f0155450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9e7fff640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f00c90e0 0x7fe9f0154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9ee59c640 1 -- 172.21.15.32:0/1461461669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe9f01599e0 msgr2=0x7fe9f0155450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9ee59c640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe9f01599e0 0x7fe9f0155450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9ee59c640 1 -- 172.21.15.32:0/1461461669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f00c90e0 msgr2=0x7fe9f0154f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9ee59c640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f00c90e0 0x7fe9f0154f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9ee59c640 1 -- 172.21.15.32:0/1461461669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fe9e8009b30 con 0x7fe9f00b0d00 2023-12-20T06:04:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9ee59c640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9f00b0d00 0x7fe9f00c70f0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000e9b0 tx=0x7fe9e000ee70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9e6ffd640 1 -- 172.21.15.32:0/1461461669 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e0009670 con 0x7fe9f00b0d00 2023-12-20T06:04:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9e6ffd640 1 -- 172.21.15.32:0/1461461669 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fe9e000c9a0 con 0x7fe9f00b0d00 2023-12-20T06:04:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9e6ffd640 1 -- 172.21.15.32:0/1461461669 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fe9e000cd10 con 0x7fe9f00b0d00 2023-12-20T06:04:15.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/1461461669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fe9f00c7e00 con 0x7fe9f00b0d00 2023-12-20T06:04:15.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/1461461669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fe9f00c83c0 con 0x7fe9f00b0d00 2023-12-20T06:04:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/1461461669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fe9f004e000 con 0x7fe9f00b0d00 2023-12-20T06:04:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.545+0000 7fe9e6ffd640 1 -- 172.21.15.32:0/1461461669 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fe9e0013470 con 0x7fe9f00b0d00 2023-12-20T06:04:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.549+0000 7fe9e6ffd640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe9d0065ca0 0x7fe9d0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.549+0000 7fe9e6ffd640 1 -- 172.21.15.32:0/1461461669 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fe9e0083c80 con 0x7fe9f00b0d00 2023-12-20T06:04:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.549+0000 7fe9e7fff640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe9d0065ca0 0x7fe9d0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.550 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.549+0000 7fe9e7fff640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe9d0065ca0 0x7fe9d0068160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9f014b4e0 tx=0x7fe9d8009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:15.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.553+0000 7fe9e6ffd640 1 -- 172.21.15.32:0/1461461669 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fe9e00512c0 con 0x7fe9f00b0d00 2023-12-20T06:04:15.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.713+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/1461461669 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fe9f00c8640 con 0x7fe9f00b0d00 2023-12-20T06:04:15.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.713+0000 7fe9e6ffd640 1 -- 172.21.15.32:0/1461461669 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v250) v1 ==== 74+0+16110 (secure 0 0 0) 0x7fe9e0021020 con 0x7fe9f00b0d00 2023-12-20T06:04:15.716 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:15.717 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":250,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:04:14.992347+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:04:12.651034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"250","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-20T06:04:13.997753+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"250","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:04:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/1461461669 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe9d0065ca0 msgr2=0x7fe9d0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe9d0065ca0 0x7fe9d0068160 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7fe9f014b4e0 tx=0x7fe9d8009040 comp rx=0 tx=0).stop 2023-12-20T06:04:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/1461461669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9f00b0d00 msgr2=0x7fe9f00c70f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9f00b0d00 0x7fe9f00c70f0 secure :-1 s=READY pgs=2474 cs=0 l=1 rev1=1 crypto rx=0x7fe9e000e9b0 tx=0x7fe9e000ee70 comp rx=0 tx=0).stop 2023-12-20T06:04:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/1461461669 shutdown_connections 2023-12-20T06:04:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fe9d0065ca0 0x7fe9d0068160 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fe9f01599e0 0x7fe9f0155450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fe9f00c90e0 0x7fe9f0154f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.719 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 --2- 172.21.15.32:0/1461461669 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fe9f00b0d00 0x7fe9f00c70f0 unknown :-1 s=CLOSED pgs=2474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/1461461669 >> 172.21.15.32:0/1461461669 conn(0x7fe9f006a320 msgr2=0x7fe9f00714b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/1461461669 shutdown_connections 2023-12-20T06:04:15.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.717+0000 7fe9f4fa6640 1 -- 172.21.15.32:0/1461461669 wait complete. 2023-12-20T06:04:15.738 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2023-12-20T06:04:15.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d9e8c640 1 Processor -- start 2023-12-20T06:04:15.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d9e8c640 1 -- start start 2023-12-20T06:04:15.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d9e8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff6d4164670 0x7ff6d40ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d9e8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 0x7ff6d40cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d9e8c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff6d40cb0b0 con 0x7ff6d4070230 2023-12-20T06:04:15.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d9e8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff6d40ca810 con 0x7ff6d4164670 2023-12-20T06:04:15.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d9e8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d40caaf0 con 0x7ff6d406da20 2023-12-20T06:04:15.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d2ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 0x7ff6d40cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d37fe640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7ff6d4164670 0x7ff6d40ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d2ffd640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 0x7ff6d40cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:45570/0 (socket says 172.21.15.32:45570) 2023-12-20T06:04:15.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d2ffd640 1 -- 172.21.15.32:0/1275921884 learned_addr learned my addr 172.21.15.32:0/1275921884 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:15.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d2ffd640 1 -- 172.21.15.32:0/1275921884 >> v2:172.21.15.39:3300/0 conn(0x7ff6d4164670 msgr2=0x7ff6d40ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:15.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d2ffd640 1 --2- 172.21.15.32:0/1275921884 >> v2:172.21.15.39:3300/0 conn(0x7ff6d4164670 0x7ff6d40ca1a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d2ffd640 1 -- 172.21.15.32:0/1275921884 >> v1:172.21.15.39:6789/0 conn(0x7ff6d4070230 legacy=0x7ff6d406ca30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d2ffd640 1 -- 172.21.15.32:0/1275921884 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6d40cadd0 con 0x7ff6d406da20 2023-12-20T06:04:15.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.885+0000 7ff6d37fe640 1 --2- 172.21.15.32:0/1275921884 >> v2:172.21.15.39:3300/0 conn(0x7ff6d4164670 0x7ff6d40ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:15.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d2ffd640 1 --2- 172.21.15.32:0/1275921884 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 0x7ff6d40cc3b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009870 tx=0x7ff6c801c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2f3063f1674cff5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d27fc640 1 -- 172.21.15.32:0/1275921884 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80ab600 con 0x7ff6d406da20 2023-12-20T06:04:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d27fc640 1 -- 172.21.15.32:0/1275921884 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c80a7050 con 0x7ff6d406da20 2023-12-20T06:04:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d27fc640 1 -- 172.21.15.32:0/1275921884 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80ab980 con 0x7ff6d406da20 2023-12-20T06:04:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/1275921884 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 msgr2=0x7ff6d40cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d9e8c640 1 --2- 172.21.15.32:0/1275921884 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 0x7ff6d40cc3b0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7ff6c8009870 tx=0x7ff6c801c3f0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/1275921884 shutdown_connections 2023-12-20T06:04:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d9e8c640 1 --2- 172.21.15.32:0/1275921884 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 0x7ff6d40cc3b0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.892 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d9e8c640 1 --2- 172.21.15.32:0/1275921884 >> v2:172.21.15.39:3300/0 conn(0x7ff6d4164670 0x7ff6d40ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/1275921884 >> 172.21.15.32:0/1275921884 conn(0x7ff6d406a510 msgr2=0x7ff6d40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:15.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/1275921884 shutdown_connections 2023-12-20T06:04:15.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.889+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/1275921884 wait complete. 2023-12-20T06:04:15.893 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d9e8c640 1 Processor -- start 2023-12-20T06:04:15.894 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d9e8c640 1 -- start start 2023-12-20T06:04:15.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d9e8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 0x7ff6d4152b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d9e8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d4070230 0x7ff6d4159d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d9e8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6d4164670 0x7ff6d415d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d9e8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d4153070 con 0x7ff6d4070230 2023-12-20T06:04:15.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d9e8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d415a760 con 0x7ff6d4164670 2023-12-20T06:04:15.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d9e8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff6d415aa40 con 0x7ff6d406da20 2023-12-20T06:04:15.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d4070230 0x7ff6d4159d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d37fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 0x7ff6d4152b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d2ffd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d4070230 0x7ff6d4159d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50034/0 (socket says 172.21.15.32:50034) 2023-12-20T06:04:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d2ffd640 1 -- 172.21.15.32:0/3601414483 learned_addr learned my addr 172.21.15.32:0/3601414483 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d3fff640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6d4164670 0x7ff6d415d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d2ffd640 1 -- 172.21.15.32:0/3601414483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 msgr2=0x7ff6d4152b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d2ffd640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 0x7ff6d4152b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d2ffd640 1 -- 172.21.15.32:0/3601414483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6d4164670 msgr2=0x7ff6d415d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:15.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d2ffd640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6d4164670 0x7ff6d415d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d2ffd640 1 -- 172.21.15.32:0/3601414483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6c4009420 con 0x7ff6d4070230 2023-12-20T06:04:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d2ffd640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d4070230 0x7ff6d4159d80 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80ae040 tx=0x7ff6c8019190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6bbfff640 1 -- 172.21.15.32:0/3601414483 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80b4420 con 0x7ff6d4070230 2023-12-20T06:04:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6bbfff640 1 -- 172.21.15.32:0/3601414483 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6c80b45a0 con 0x7ff6d4070230 2023-12-20T06:04:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/3601414483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff6c80094d0 con 0x7ff6d4070230 2023-12-20T06:04:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6bbfff640 1 -- 172.21.15.32:0/3601414483 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6c80b4950 con 0x7ff6d4070230 2023-12-20T06:04:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/3601414483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff6d4070bd0 con 0x7ff6d4070230 2023-12-20T06:04:15.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.893+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/3601414483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff6d404e000 con 0x7ff6d4070230 2023-12-20T06:04:15.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.897+0000 7ff6bbfff640 1 -- 172.21.15.32:0/3601414483 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7ff6c80b2070 con 0x7ff6d4070230 2023-12-20T06:04:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.897+0000 7ff6bbfff640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6c0065ca0 0x7ff6c0068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.897+0000 7ff6bbfff640 1 -- 172.21.15.32:0/3601414483 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(250..250 src has 1..250) v4 ==== 5765+0+0 (secure 0 0 0) 0x7ff6c811d090 con 0x7ff6d4070230 2023-12-20T06:04:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.897+0000 7ff6d37fe640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6c0065ca0 0x7ff6c0068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:15.901 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.897+0000 7ff6d37fe640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6c0065ca0 0x7ff6c0068160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009cf0 tx=0x7ff6c4009190 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:15.902 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:15.901+0000 7ff6bbfff640 1 -- 172.21.15.32:0/3601414483 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff6c80ea5d0 con 0x7ff6d4070230 2023-12-20T06:04:16.095 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:16.093+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/3601414483 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) v1 -- 0x7ff6d40cf4d0 con 0x7ff6d4070230 2023-12-20T06:04:17.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.005+0000 7ff6bbfff640 1 -- 172.21.15.32:0/3601414483 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v127) v1 ==== 118+0+0 (secure 0 0 0) 0x7ff6c80edbf0 con 0x7ff6d4070230 2023-12-20T06:04:17.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/3601414483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6c0065ca0 msgr2=0x7ff6c0068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6c0065ca0 0x7ff6c0068160 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7ff6c4009cf0 tx=0x7ff6c4009190 comp rx=0 tx=0).stop 2023-12-20T06:04:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/3601414483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d4070230 msgr2=0x7ff6d4159d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d4070230 0x7ff6d4159d80 secure :-1 s=READY pgs=2475 cs=0 l=1 rev1=1 crypto rx=0x7ff6c80ae040 tx=0x7ff6c8019190 comp rx=0 tx=0).stop 2023-12-20T06:04:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/3601414483 shutdown_connections 2023-12-20T06:04:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff6c0065ca0 0x7ff6c0068160 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff6d4164670 0x7ff6d415d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff6d4070230 0x7ff6d4159d80 unknown :-1 s=CLOSED pgs=2475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 --2- 172.21.15.32:0/3601414483 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff6d406da20 0x7ff6d4152b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/3601414483 >> 172.21.15.32:0/3601414483 conn(0x7ff6d406a510 msgr2=0x7ff6d4163840 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/3601414483 shutdown_connections 2023-12-20T06:04:17.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.009+0000 7ff6d9e8c640 1 -- 172.21.15.32:0/3601414483 wait complete. 2023-12-20T06:04:17.033 DEBUG:tasks.cephfs.filesystem:fs_config: {} 2023-12-20T06:04:17.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay false 2023-12-20T06:04:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bd04c4640 1 Processor -- start 2023-12-20T06:04:17.185 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bd04c4640 1 -- start start 2023-12-20T06:04:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bd04c4640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f5bc8070940 0x7f5bc80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bd04c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc815b6a0 con 0x7f5bc80b0cb0 2023-12-20T06:04:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bd04c4640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f5bc806d8c0 con 0x7f5bc815abb0 2023-12-20T06:04:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bd04c4640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f5bc806e160 con 0x7f5bc8070940 2023-12-20T06:04:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcda5f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f5bc80b0cb0 0x7f5bc8070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55346/0 (socket says 172.21.15.32:55346) 2023-12-20T06:04:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcda5f640 1 -- 172.21.15.32:0/2506538207 learned_addr learned my addr 172.21.15.32:0/2506538207 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:17.188 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcea61640 1 --2- 172.21.15.32:0/2506538207 >> v2:172.21.15.39:3300/0 conn(0x7f5bc8070940 0x7f5bc80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2778681248 0 0) 0x7f5bc815b6a0 con 0x7f5bc80b0cb0 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f5bb4003530 con 0x7f5bc80b0cb0 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2689535657 0 0) 0x7f5bb4003530 con 0x7f5bc80b0cb0 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 >> v2:172.21.15.39:3300/0 conn(0x7f5bc8070940 msgr2=0x7f5bc80caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 --2- 172.21.15.32:0/2506538207 >> v2:172.21.15.39:3300/0 conn(0x7f5bc8070940 0x7f5bc80caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 >> v1:172.21.15.39:6789/0 conn(0x7f5bc815abb0 legacy=0x7f5bc815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc806dba0 con 0x7f5bc80b0cb0 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f5bb8001680 con 0x7f5bc80b0cb0 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 >> v1:172.21.15.32:6789/0 conn(0x7f5bc80b0cb0 legacy=0x7f5bc8070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 --2- 172.21.15.32:0/2506538207 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5bb4003980 0x7f5bb4003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 --2- 172.21.15.32:0/2506538207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bb4004380 0x7f5bb4004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 --2- 172.21.15.32:0/2506538207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 0x7f5bb400b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc806e160 con 0x7f5bb4005050 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb400bfb0 con 0x7f5bb4004380 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bb400c170 con 0x7f5bb4003980 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcea61640 1 --2- 172.21.15.32:0/2506538207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 0x7f5bb400b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:17.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bce260640 1 --2- 172.21.15.32:0/2506538207 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5bb4003980 0x7f5bb4003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.185+0000 7f5bcda5f640 1 --2- 172.21.15.32:0/2506538207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bb4004380 0x7f5bb4004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bcea61640 1 -- 172.21.15.32:0/2506538207 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5bb4003980 msgr2=0x7f5bb4003d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bcea61640 1 --2- 172.21.15.32:0/2506538207 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5bb4003980 0x7f5bb4003d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bcea61640 1 -- 172.21.15.32:0/2506538207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bb4004380 msgr2=0x7f5bb4004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bcea61640 1 --2- 172.21.15.32:0/2506538207 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bb4004380 0x7f5bb4004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:17.190 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bcea61640 1 -- 172.21.15.32:0/2506538207 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bb4003530 con 0x7f5bb4005050 2023-12-20T06:04:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bcea61640 1 -- 172.21.15.32:0/2506538207 reap_dead start 2023-12-20T06:04:17.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bcea61640 1 --2- 172.21.15.32:0/2506538207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 0x7f5bb400b990 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009f90 tx=0x7f5bc001cc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa8c1620fbe6feb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:17.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bb8007ea0 con 0x7f5bb4005050 2023-12-20T06:04:17.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc000f040 con 0x7f5bb4005050 2023-12-20T06:04:17.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bcd25e640 1 -- 172.21.15.32:0/2506538207 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc00039a0 con 0x7f5bb4005050 2023-12-20T06:04:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bd04c4640 1 -- 172.21.15.32:0/2506538207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 msgr2=0x7f5bb400b990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bd04c4640 1 --2- 172.21.15.32:0/2506538207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 0x7f5bb400b990 secure :-1 s=READY pgs=2477 cs=0 l=1 rev1=1 crypto rx=0x7f5bc0009f90 tx=0x7f5bc001cc60 comp rx=0 tx=0).stop 2023-12-20T06:04:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bd04c4640 1 -- 172.21.15.32:0/2506538207 shutdown_connections 2023-12-20T06:04:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bd04c4640 1 --2- 172.21.15.32:0/2506538207 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 0x7f5bb400b990 unknown :-1 s=CLOSED pgs=2477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.189+0000 7f5bd04c4640 1 -- 172.21.15.32:0/2506538207 >> 172.21.15.32:0/2506538207 conn(0x7f5bc806a510 msgr2=0x7f5bc80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 -- 172.21.15.32:0/2506538207 shutdown_connections 2023-12-20T06:04:17.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 -- 172.21.15.32:0/2506538207 wait complete. 2023-12-20T06:04:17.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 Processor -- start 2023-12-20T06:04:17.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 -- start start 2023-12-20T06:04:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 0x7f5bc8154f10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5bc815e000 0x7f5bc80b0100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc80ccd50 0x7f5bc80c3ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc8155450 con 0x7f5bb4005050 2023-12-20T06:04:17.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80cc8c0 con 0x7f5bc80ccd50 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f5bc80c5950 con 0x7f5bc815e000 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bce260640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 0x7f5bc8154f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bce260640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 0x7f5bc8154f10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50052/0 (socket says 172.21.15.32:50052) 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bce260640 1 -- 172.21.15.32:0/3130538104 learned_addr learned my addr 172.21.15.32:0/3130538104 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bcda5f640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5bc815e000 0x7f5bc80b0100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bcea61640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc80ccd50 0x7f5bc80c3ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bce260640 1 -- 172.21.15.32:0/3130538104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5bc815e000 msgr2=0x7f5bc80b0100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bce260640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5bc815e000 0x7f5bc80b0100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bce260640 1 -- 172.21.15.32:0/3130538104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc80ccd50 msgr2=0x7f5bc80c3ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bce260640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc80ccd50 0x7f5bc80c3ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bce260640 1 -- 172.21.15.32:0/3130538104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5bc80c5c30 con 0x7f5bb4005050 2023-12-20T06:04:17.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bce260640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 0x7f5bc8154f10 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400f840 tx=0x7f5bc400fd00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bbeffd640 1 -- 172.21.15.32:0/3130538104 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc40014a0 con 0x7f5bb4005050 2023-12-20T06:04:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bbeffd640 1 -- 172.21.15.32:0/3130538104 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f5bc4005320 con 0x7f5bb4005050 2023-12-20T06:04:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 -- 172.21.15.32:0/3130538104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f5bc80cc550 con 0x7f5bb4005050 2023-12-20T06:04:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bbeffd640 1 -- 172.21.15.32:0/3130538104 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f5bc4006a20 con 0x7f5bb4005050 2023-12-20T06:04:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.193+0000 7f5bd04c4640 1 -- 172.21.15.32:0/3130538104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f5bc80cd1d0 con 0x7f5bb4005050 2023-12-20T06:04:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.197+0000 7f5bd04c4640 1 -- 172.21.15.32:0/3130538104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f5bc80c5e70 con 0x7f5bb4005050 2023-12-20T06:04:17.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.197+0000 7f5bbeffd640 1 -- 172.21.15.32:0/3130538104 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f5bc4006b80 con 0x7f5bb4005050 2023-12-20T06:04:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.197+0000 7f5bbeffd640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5bac065ca0 0x7f5bac068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:17.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.197+0000 7f5bbeffd640 1 -- 172.21.15.32:0/3130538104 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f5bc4082ca0 con 0x7f5bb4005050 2023-12-20T06:04:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.197+0000 7f5bcda5f640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5bac065ca0 0x7f5bac068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:17.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.197+0000 7f5bcda5f640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5bac065ca0 0x7f5bac068160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80ce590 tx=0x7f5bb8009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:17.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.201+0000 7f5bbeffd640 1 -- 172.21.15.32:0/3130538104 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f5bc80c5e70 con 0x7f5bb4005050 2023-12-20T06:04:17.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:17.397+0000 7f5bd04c4640 1 -- 172.21.15.32:0/3130538104 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"} v 0) v1 -- 0x7f5bc80cc3b0 con 0x7f5bb4005050 2023-12-20T06:04:18.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bbeffd640 1 -- 172.21.15.32:0/3130538104 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "false"}]=0 v128) v1 ==== 122+0+0 (secure 0 0 0) 0x7f5bc80cc3b0 con 0x7f5bb4005050 2023-12-20T06:04:18.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 -- 172.21.15.32:0/3130538104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5bac065ca0 msgr2=0x7f5bac068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5bac065ca0 0x7f5bac068160 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f5bc80ce590 tx=0x7f5bb8009040 comp rx=0 tx=0).stop 2023-12-20T06:04:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 -- 172.21.15.32:0/3130538104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 msgr2=0x7f5bc8154f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 0x7f5bc8154f10 secure :-1 s=READY pgs=2478 cs=0 l=1 rev1=1 crypto rx=0x7f5bc400f840 tx=0x7f5bc400fd00 comp rx=0 tx=0).stop 2023-12-20T06:04:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 -- 172.21.15.32:0/3130538104 shutdown_connections 2023-12-20T06:04:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5bac065ca0 0x7f5bac068160 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f5bc80ccd50 0x7f5bc80c3ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f5bc815e000 0x7f5bc80b0100 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 --2- 172.21.15.32:0/3130538104 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f5bb4005050 0x7f5bc8154f10 unknown :-1 s=CLOSED pgs=2478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 -- 172.21.15.32:0/3130538104 >> 172.21.15.32:0/3130538104 conn(0x7f5bc806a510 msgr2=0x7f5bc8156510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 -- 172.21.15.32:0/3130538104 shutdown_connections 2023-12-20T06:04:18.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.009+0000 7f5bd04c4640 1 -- 172.21.15.32:0/3130538104 wait complete. 2023-12-20T06:04:18.033 DEBUG:tasks.cephfs.filesystem:Not Creating any subvols for filesystem 'cephfs' 2023-12-20T06:04:18.033 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:04:18.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3f7e9640 1 Processor -- start 2023-12-20T06:04:18.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3f7e9640 1 -- start start 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3f7e9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcc3806ce60 0x7fcc3815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3f7e9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc3815f110 0x7fcc3815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3f7e9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcc3815bbd0 con 0x7fcc380af5e0 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3f7e9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3815beb0 con 0x7fcc3815f110 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3f7e9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcc3815e910 con 0x7fcc3806ce60 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3d585640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fcc380af5e0 0x7fcc380af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55354/0 (socket says 172.21.15.32:55354) 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3dd86640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcc3815f110 0x7fcc3815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3d585640 1 -- 172.21.15.32:0/2835668723 learned_addr learned my addr 172.21.15.32:0/2835668723 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3cd84640 1 --2- 172.21.15.32:0/2835668723 >> v2:172.21.15.39:3300/0 conn(0x7fcc3806ce60 0x7fcc3815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3dd86640 1 -- 172.21.15.32:0/2835668723 >> v2:172.21.15.39:3300/0 conn(0x7fcc3806ce60 msgr2=0x7fcc3815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3dd86640 1 --2- 172.21.15.32:0/2835668723 >> v2:172.21.15.39:3300/0 conn(0x7fcc3806ce60 0x7fcc3815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3dd86640 1 -- 172.21.15.32:0/2835668723 >> v1:172.21.15.32:6789/0 conn(0x7fcc380af5e0 legacy=0x7fcc380af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3dd86640 1 -- 172.21.15.32:0/2835668723 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc3815ebf0 con 0x7fcc3815f110 2023-12-20T06:04:18.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc2ffff640 1 -- 172.21.15.32:0/2835668723 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2334730389 0 0) 0x7fcc3815bbd0 con 0x7fcc380af5e0 2023-12-20T06:04:18.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.181+0000 7fcc3cd84640 1 --2- 172.21.15.32:0/2835668723 >> v2:172.21.15.39:3300/0 conn(0x7fcc3806ce60 0x7fcc3815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:18.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc3dd86640 1 --2- 172.21.15.32:0/2835668723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3815f110 0x7fcc3815c440 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fcc3806abe0 tx=0x7fcc3001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a64fc114e2de0535 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:18.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc2ffff640 1 -- 172.21.15.32:0/2835668723 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc3815b9d0 con 0x7fcc3815f110 2023-12-20T06:04:18.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc2ffff640 1 -- 172.21.15.32:0/2835668723 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc300a58e0 con 0x7fcc3815f110 2023-12-20T06:04:18.186 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc2ffff640 1 -- 172.21.15.32:0/2835668723 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc300a5c70 con 0x7fcc3815f110 2023-12-20T06:04:18.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/2835668723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3815f110 msgr2=0x7fcc3815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc3f7e9640 1 --2- 172.21.15.32:0/2835668723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3815f110 0x7fcc3815c440 secure :-1 s=READY pgs=2480 cs=0 l=1 rev1=1 crypto rx=0x7fcc3806abe0 tx=0x7fcc3001c880 comp rx=0 tx=0).stop 2023-12-20T06:04:18.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/2835668723 shutdown_connections 2023-12-20T06:04:18.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc3f7e9640 1 --2- 172.21.15.32:0/2835668723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3815f110 0x7fcc3815c440 unknown :-1 s=CLOSED pgs=2480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc3f7e9640 1 --2- 172.21.15.32:0/2835668723 >> v2:172.21.15.39:3300/0 conn(0x7fcc3806ce60 0x7fcc3815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/2835668723 >> 172.21.15.32:0/2835668723 conn(0x7fcc3806a320 msgr2=0x7fcc380ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:18.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/2835668723 shutdown_connections 2023-12-20T06:04:18.187 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.185+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/2835668723 wait complete. 2023-12-20T06:04:18.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3f7e9640 1 Processor -- start 2023-12-20T06:04:18.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3f7e9640 1 -- start start 2023-12-20T06:04:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3f7e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806ce60 0x7fcc38159dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3f7e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc380af5e0 0x7fcc38150d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3f7e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc3815f110 0x7fcc38157660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3f7e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc380b2940 con 0x7fcc3806ce60 2023-12-20T06:04:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3f7e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38158df0 con 0x7fcc3815f110 2023-12-20T06:04:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3f7e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcc38159930 con 0x7fcc380af5e0 2023-12-20T06:04:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3d585640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806ce60 0x7fcc38159dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3d585640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806ce60 0x7fcc38159dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50078/0 (socket says 172.21.15.32:50078) 2023-12-20T06:04:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3d585640 1 -- 172.21.15.32:0/3751038894 learned_addr learned my addr 172.21.15.32:0/3751038894 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3dd86640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc3815f110 0x7fcc38157660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.191 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3cd84640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc380af5e0 0x7fcc38150d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3d585640 1 -- 172.21.15.32:0/3751038894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc380af5e0 msgr2=0x7fcc38150d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3d585640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc380af5e0 0x7fcc38150d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3d585640 1 -- 172.21.15.32:0/3751038894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc3815f110 msgr2=0x7fcc38157660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3d585640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc3815f110 0x7fcc38157660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3d585640 1 -- 172.21.15.32:0/3751038894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcc30009a70 con 0x7fcc3806ce60 2023-12-20T06:04:18.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3d585640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806ce60 0x7fcc38159dc0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fcc340010f0 tx=0x7fcc3400cbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:18.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc2dffb640 1 -- 172.21.15.32:0/3751038894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc34007270 con 0x7fcc3806ce60 2023-12-20T06:04:18.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc2dffb640 1 -- 172.21.15.32:0/3751038894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcc340073d0 con 0x7fcc3806ce60 2023-12-20T06:04:18.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/3751038894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcc38159000 con 0x7fcc3806ce60 2023-12-20T06:04:18.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc2dffb640 1 -- 172.21.15.32:0/3751038894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcc340057b0 con 0x7fcc3806ce60 2023-12-20T06:04:18.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/3751038894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcc381595c0 con 0x7fcc3806ce60 2023-12-20T06:04:18.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.189+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/3751038894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcc3804e000 con 0x7fcc3806ce60 2023-12-20T06:04:18.193 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.193+0000 7fcc2dffb640 1 -- 172.21.15.32:0/3751038894 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fcc34005910 con 0x7fcc3806ce60 2023-12-20T06:04:18.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.193+0000 7fcc2dffb640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcc1c065cd0 0x7fcc1c068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.193+0000 7fcc2dffb640 1 -- 172.21.15.32:0/3751038894 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fcc34082bc0 con 0x7fcc3806ce60 2023-12-20T06:04:18.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.193+0000 7fcc3cd84640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcc1c065cd0 0x7fcc1c068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.193+0000 7fcc3cd84640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcc1c065cd0 0x7fcc1c068190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc380b11d0 tx=0x7fcc28008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:18.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.197+0000 7fcc2dffb640 1 -- 172.21.15.32:0/3751038894 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcc340501e0 con 0x7fcc3806ce60 2023-12-20T06:04:18.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.389+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/3751038894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fcc380cf450 con 0x7fcc3806ce60 2023-12-20T06:04:18.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.389+0000 7fcc2dffb640 1 -- 172.21.15.32:0/3751038894 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4722 (secure 0 0 0) 0x7fcc34017070 con 0x7fcc3806ce60 2023-12-20T06:04:18.392 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:18.392 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3779641739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3779641739},{"type":"v1","addr":"172.21.15.39:6834","nonce":3779641739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/473427978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":473427978},{"type":"v1","addr":"172.21.15.39:6835","nonce":473427978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12060,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3927543497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3927543497},{"type":"v1","addr":"172.21.15.32:6836","nonce":3927543497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:04:14.992681+0000","modified":"2023-12-20T06:04:18.007320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12063":{"gid":12063,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3133738681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3133738681},{"type":"v1","addr":"172.21.15.32:6837","nonce":3133738681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-20T06:04:18.392 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 128 2023-12-20T06:04:18.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/3751038894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcc1c065cd0 msgr2=0x7fcc1c068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcc1c065cd0 0x7fcc1c068190 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fcc380b11d0 tx=0x7fcc28008040 comp rx=0 tx=0).stop 2023-12-20T06:04:18.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/3751038894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806ce60 msgr2=0x7fcc38159dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806ce60 0x7fcc38159dc0 secure :-1 s=READY pgs=2481 cs=0 l=1 rev1=1 crypto rx=0x7fcc340010f0 tx=0x7fcc3400cbe0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/3751038894 shutdown_connections 2023-12-20T06:04:18.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcc1c065cd0 0x7fcc1c068190 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcc3815f110 0x7fcc38157660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcc380af5e0 0x7fcc38150d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 --2- 172.21.15.32:0/3751038894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcc3806ce60 0x7fcc38159dc0 unknown :-1 s=CLOSED pgs=2481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/3751038894 >> 172.21.15.32:0/3751038894 conn(0x7fcc3806a320 msgr2=0x7fcc38165420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:18.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/3751038894 shutdown_connections 2023-12-20T06:04:18.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.393+0000 7fcc3f7e9640 1 -- 172.21.15.32:0/3751038894 wait complete. 2023-12-20T06:04:18.413 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:04:18.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.553+0000 7f67dc2fc640 1 Processor -- start 2023-12-20T06:04:18.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67dc2fc640 1 -- start start 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67dc2fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d40af5e0 0x7f67d40af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67dc2fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67d406ce60 0x7f67d415b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67dc2fc640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f67d415bbd0 con 0x7f67d415f110 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67dc2fc640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f67d415beb0 con 0x7f67d406ce60 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67dc2fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d415e910 con 0x7f67d40af5e0 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67d9897640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f67d406ce60 0x7f67d415b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67da899640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f67d415f110 0x7f67d415c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:55364/0 (socket says 172.21.15.32:55364) 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67da899640 1 -- 172.21.15.32:0/2766826091 learned_addr learned my addr 172.21.15.32:0/2766826091 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67da098640 1 --2- 172.21.15.32:0/2766826091 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d40af5e0 0x7f67d40af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67d9897640 1 -- 172.21.15.32:0/2766826091 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d40af5e0 msgr2=0x7f67d40af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67d9897640 1 --2- 172.21.15.32:0/2766826091 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d40af5e0 0x7f67d40af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67d9897640 1 -- 172.21.15.32:0/2766826091 >> v1:172.21.15.32:6789/0 conn(0x7f67d415f110 legacy=0x7f67d415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67d9897640 1 -- 172.21.15.32:0/2766826091 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67d415ebf0 con 0x7f67d406ce60 2023-12-20T06:04:18.559 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67da098640 1 --2- 172.21.15.32:0/2766826091 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d40af5e0 0x7f67d40af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:18.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.557+0000 7f67d9096640 1 -- 172.21.15.32:0/2766826091 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1392439565 0 0) 0x7f67d415bbd0 con 0x7f67d415f110 2023-12-20T06:04:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67d9897640 1 --2- 172.21.15.32:0/2766826091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d406ce60 0x7f67d415b280 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f67c40097c0 tx=0x7f67c401c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc873e6bb40c1dcb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67d9096640 1 -- 172.21.15.32:0/2766826091 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40ac640 con 0x7f67d406ce60 2023-12-20T06:04:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67d9096640 1 -- 172.21.15.32:0/2766826091 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c40ac7a0 con 0x7f67d406ce60 2023-12-20T06:04:18.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67d9096640 1 -- 172.21.15.32:0/2766826091 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40acb30 con 0x7f67d406ce60 2023-12-20T06:04:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2766826091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d406ce60 msgr2=0x7f67d415b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67dc2fc640 1 --2- 172.21.15.32:0/2766826091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d406ce60 0x7f67d415b280 secure :-1 s=READY pgs=2483 cs=0 l=1 rev1=1 crypto rx=0x7f67c40097c0 tx=0x7f67c401c480 comp rx=0 tx=0).stop 2023-12-20T06:04:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2766826091 shutdown_connections 2023-12-20T06:04:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67dc2fc640 1 --2- 172.21.15.32:0/2766826091 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d406ce60 0x7f67d415b280 unknown :-1 s=CLOSED pgs=2483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67dc2fc640 1 --2- 172.21.15.32:0/2766826091 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d40af5e0 0x7f67d40af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2766826091 >> 172.21.15.32:0/2766826091 conn(0x7f67d406a320 msgr2=0x7f67d40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2766826091 shutdown_connections 2023-12-20T06:04:18.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2766826091 wait complete. 2023-12-20T06:04:18.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67dc2fc640 1 Processor -- start 2023-12-20T06:04:18.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.561+0000 7f67dc2fc640 1 -- start start 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67dc2fc640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d406ce60 0x7f67d40c3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67dc2fc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d40af5e0 0x7f67d40b2af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67dc2fc640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d415f110 0x7f67d40b3030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67dc2fc640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d40c43e0 con 0x7f67d40af5e0 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67dc2fc640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f67d40ce640 con 0x7f67d406ce60 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67dc2fc640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f67d4153850 con 0x7f67d415f110 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67d9897640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d40af5e0 0x7f67d40b2af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67d9897640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d40af5e0 0x7f67d40b2af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:50096/0 (socket says 172.21.15.32:50096) 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67d9897640 1 -- 172.21.15.32:0/2940344955 learned_addr learned my addr 172.21.15.32:0/2940344955 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67da899640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d415f110 0x7f67d40b3030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67da098640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d406ce60 0x7f67d40c3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67d9897640 1 -- 172.21.15.32:0/2940344955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d415f110 msgr2=0x7f67d40b3030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67d9897640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d415f110 0x7f67d40b3030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67d9897640 1 -- 172.21.15.32:0/2940344955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d406ce60 msgr2=0x7f67d40c3ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67d9897640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d406ce60 0x7f67d40c3ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67d9897640 1 -- 172.21.15.32:0/2940344955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f67c4009420 con 0x7f67d40af5e0 2023-12-20T06:04:18.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67da899640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d415f110 0x7f67d40b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:18.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67d9897640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d40af5e0 0x7f67d40b2af0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f67c401a0b0 tx=0x7f67c4019220 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:18.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67caffd640 1 -- 172.21.15.32:0/2940344955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40b5810 con 0x7f67d40af5e0 2023-12-20T06:04:18.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67caffd640 1 -- 172.21.15.32:0/2940344955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f67c40b5970 con 0x7f67d40af5e0 2023-12-20T06:04:18.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67caffd640 1 -- 172.21.15.32:0/2940344955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f67c40b5d00 con 0x7f67d40af5e0 2023-12-20T06:04:18.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2940344955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f67d4153b30 con 0x7f67d40af5e0 2023-12-20T06:04:18.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2940344955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f67d40ce920 con 0x7f67d40af5e0 2023-12-20T06:04:18.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.565+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2940344955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f67d404e000 con 0x7f67d40af5e0 2023-12-20T06:04:18.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.569+0000 7f67caffd640 1 -- 172.21.15.32:0/2940344955 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f67c40b3040 con 0x7f67d40af5e0 2023-12-20T06:04:18.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.569+0000 7f67caffd640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f67b8065ca0 0x7f67b8068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.569+0000 7f67caffd640 1 -- 172.21.15.32:0/2940344955 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f67c411d860 con 0x7f67d40af5e0 2023-12-20T06:04:18.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.569+0000 7f67da098640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f67b8065ca0 0x7f67b8068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.569+0000 7f67da098640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f67b8065ca0 0x7f67b8068160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67d0006fd0 tx=0x7f67d0008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:18.574 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.573+0000 7f67caffd640 1 -- 172.21.15.32:0/2940344955 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f67c40eade0 con 0x7f67d40af5e0 2023-12-20T06:04:18.734 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.733+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2940344955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f67d4153fe0 con 0x7f67d40af5e0 2023-12-20T06:04:18.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.733+0000 7f67caffd640 1 -- 172.21.15.32:0/2940344955 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v251) v1 ==== 74+0+16097 (secure 0 0 0) 0x7f67c40b5e60 con 0x7f67d40af5e0 2023-12-20T06:04:18.735 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:18.735 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":251,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:04:15.995582+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:04:12.651034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"250","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-20T06:04:13.997753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:04:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2940344955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f67b8065ca0 msgr2=0x7f67b8068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f67b8065ca0 0x7f67b8068160 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f67d0006fd0 tx=0x7f67d0008040 comp rx=0 tx=0).stop 2023-12-20T06:04:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2940344955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d40af5e0 msgr2=0x7f67d40b2af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d40af5e0 0x7f67d40b2af0 secure :-1 s=READY pgs=2484 cs=0 l=1 rev1=1 crypto rx=0x7f67c401a0b0 tx=0x7f67c4019220 comp rx=0 tx=0).stop 2023-12-20T06:04:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2940344955 shutdown_connections 2023-12-20T06:04:18.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f67b8065ca0 0x7f67b8068160 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f67d415f110 0x7f67d40b3030 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f67d40af5e0 0x7f67d40b2af0 unknown :-1 s=CLOSED pgs=2484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 --2- 172.21.15.32:0/2940344955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f67d406ce60 0x7f67d40c3ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2940344955 >> 172.21.15.32:0/2940344955 conn(0x7f67d406a320 msgr2=0x7f67d415c420 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2940344955 shutdown_connections 2023-12-20T06:04:18.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.737+0000 7f67dc2fc640 1 -- 172.21.15.32:0/2940344955 wait complete. 2023-12-20T06:04:18.756 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2023-12-20T06:04:18.757 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:04:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f3100d18640 1 Processor -- start 2023-12-20T06:04:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f3100d18640 1 -- start start 2023-12-20T06:04:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f3100d18640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f30fc15cdb0 0x7f30fc0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f3100d18640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30fc0d2740 0x7f30fc0cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f3100d18640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30fc0d1120 0x7f30fc0cfab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f3100d18640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc15d190 con 0x7f30fc0d1120 2023-12-20T06:04:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f3100d18640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f30fc0d16a0 con 0x7f30fc15cdb0 2023-12-20T06:04:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f3100d18640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc0d1980 con 0x7f30fc0d2740 2023-12-20T06:04:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30fad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30fc0d1120 0x7f30fc0cfab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30fad9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f30fc0d1120 0x7f30fc0cfab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36894/0 (socket says 172.21.15.32:36894) 2023-12-20T06:04:18.905 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30fad9d640 1 -- 172.21.15.32:0/607924510 learned_addr learned my addr 172.21.15.32:0/607924510 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30fa59c640 1 --2- 172.21.15.32:0/607924510 >> v2:172.21.15.39:3300/0 conn(0x7f30fc15cdb0 0x7f30fc0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30f3fff640 1 --2- 172.21.15.32:0/607924510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30fc0d2740 0x7f30fc0cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30fad9d640 1 -- 172.21.15.32:0/607924510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30fc0d2740 msgr2=0x7f30fc0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30fad9d640 1 --2- 172.21.15.32:0/607924510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30fc0d2740 0x7f30fc0cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30fad9d640 1 -- 172.21.15.32:0/607924510 >> v2:172.21.15.39:3300/0 conn(0x7f30fc15cdb0 msgr2=0x7f30fc0d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30fad9d640 1 --2- 172.21.15.32:0/607924510 >> v2:172.21.15.39:3300/0 conn(0x7f30fc15cdb0 0x7f30fc0d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30fad9d640 1 -- 172.21.15.32:0/607924510 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30fc0d1c60 con 0x7f30fc0d1120 2023-12-20T06:04:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30fa59c640 1 --2- 172.21.15.32:0/607924510 >> v2:172.21.15.39:3300/0 conn(0x7f30fc15cdb0 0x7f30fc0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:18.906 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.901+0000 7f30f3fff640 1 --2- 172.21.15.32:0/607924510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30fc0d2740 0x7f30fc0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:18.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f30fad9d640 1 --2- 172.21.15.32:0/607924510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc0d1120 0x7f30fc0cfab0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0667b0 tx=0x7f30f401f1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=838cbf298c83d0fa server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:18.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f30f9d9b640 1 -- 172.21.15.32:0/607924510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f40af400 con 0x7f30fc0d1120 2023-12-20T06:04:18.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f30f9d9b640 1 -- 172.21.15.32:0/607924510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f40ab050 con 0x7f30fc0d1120 2023-12-20T06:04:18.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f30f9d9b640 1 -- 172.21.15.32:0/607924510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f40af780 con 0x7f30fc0d1120 2023-12-20T06:04:18.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f3100d18640 1 -- 172.21.15.32:0/607924510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc0d1120 msgr2=0x7f30fc0cfab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.907 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f3100d18640 1 --2- 172.21.15.32:0/607924510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc0d1120 0x7f30fc0cfab0 secure :-1 s=READY pgs=2485 cs=0 l=1 rev1=1 crypto rx=0x7f30fc0667b0 tx=0x7f30f401f1c0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f3100d18640 1 -- 172.21.15.32:0/607924510 shutdown_connections 2023-12-20T06:04:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f3100d18640 1 --2- 172.21.15.32:0/607924510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc0d1120 0x7f30fc0cfab0 unknown :-1 s=CLOSED pgs=2485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f3100d18640 1 --2- 172.21.15.32:0/607924510 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30fc0d2740 0x7f30fc0cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f3100d18640 1 --2- 172.21.15.32:0/607924510 >> v2:172.21.15.39:3300/0 conn(0x7f30fc15cdb0 0x7f30fc0d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f3100d18640 1 -- 172.21.15.32:0/607924510 >> 172.21.15.32:0/607924510 conn(0x7f30fc065ee0 msgr2=0x7f30fc06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f3100d18640 1 -- 172.21.15.32:0/607924510 shutdown_connections 2023-12-20T06:04:18.908 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.905+0000 7f3100d18640 1 -- 172.21.15.32:0/607924510 wait complete. 2023-12-20T06:04:18.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f3100d18640 1 Processor -- start 2023-12-20T06:04:18.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f3100d18640 1 -- start start 2023-12-20T06:04:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f3100d18640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30fc0d1120 0x7f30fc148d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f3100d18640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30fc0d2740 0x7f30fc1f60a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f3100d18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc15cdb0 0x7f30fc0acdb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f3100d18640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc149580 con 0x7f30fc15cdb0 2023-12-20T06:04:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f3100d18640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc149260 con 0x7f30fc0d2740 2023-12-20T06:04:18.910 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f3100d18640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f30fc1f6b60 con 0x7f30fc0d1120 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30fad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc15cdb0 0x7f30fc0acdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30fad9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc15cdb0 0x7f30fc0acdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36906/0 (socket says 172.21.15.32:36906) 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30fad9d640 1 -- 172.21.15.32:0/4159996135 learned_addr learned my addr 172.21.15.32:0/4159996135 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30fa59c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30fc0d1120 0x7f30fc148d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30f3fff640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30fc0d2740 0x7f30fc1f60a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30fad9d640 1 -- 172.21.15.32:0/4159996135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30fc0d1120 msgr2=0x7f30fc148d20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30fad9d640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30fc0d1120 0x7f30fc148d20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30fad9d640 1 -- 172.21.15.32:0/4159996135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30fc0d2740 msgr2=0x7f30fc1f60a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30fad9d640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30fc0d2740 0x7f30fc1f60a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30fad9d640 1 -- 172.21.15.32:0/4159996135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f30f4009b30 con 0x7f30fc15cdb0 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30fad9d640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc15cdb0 0x7f30fc0acdb0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f30f40b2040 tx=0x7f30f401bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30f2ffd640 1 -- 172.21.15.32:0/4159996135 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f40afde0 con 0x7f30fc15cdb0 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30f2ffd640 1 -- 172.21.15.32:0/4159996135 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f30f4009510 con 0x7f30fc15cdb0 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f30f2ffd640 1 -- 172.21.15.32:0/4159996135 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f30f40ba660 con 0x7f30fc15cdb0 2023-12-20T06:04:18.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f3100d18640 1 -- 172.21.15.32:0/4159996135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f30fc1f6e40 con 0x7f30fc15cdb0 2023-12-20T06:04:18.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.909+0000 7f3100d18640 1 -- 172.21.15.32:0/4159996135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f30fc149a70 con 0x7f30fc15cdb0 2023-12-20T06:04:18.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.913+0000 7f3100d18640 1 -- 172.21.15.32:0/4159996135 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f30fc1f6fd0 con 0x7f30fc15cdb0 2023-12-20T06:04:18.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.913+0000 7f30f2ffd640 1 -- 172.21.15.32:0/4159996135 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f30f40b6040 con 0x7f30fc15cdb0 2023-12-20T06:04:18.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.913+0000 7f30f2ffd640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f30dc065cd0 0x7f30dc068190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:18.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.913+0000 7f30f2ffd640 1 -- 172.21.15.32:0/4159996135 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f30f4121770 con 0x7f30fc15cdb0 2023-12-20T06:04:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.913+0000 7f30fa59c640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f30dc065cd0 0x7f30dc068190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:18.915 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.913+0000 7f30fa59c640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f30dc065cd0 0x7f30dc068190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f30ec006e90 tx=0x7f30ec008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:18.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:18.917+0000 7f30f2ffd640 1 -- 172.21.15.32:0/4159996135 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f30f40eeec0 con 0x7f30fc15cdb0 2023-12-20T06:04:19.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.077+0000 7f3100d18640 1 -- 172.21.15.32:0/4159996135 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f30fc0d2220 con 0x7f30dc065cd0 2023-12-20T06:04:19.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.085+0000 7f30f2ffd640 1 -- 172.21.15.32:0/4159996135 <== mgr.11715 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289435 (secure 0 0 0) 0x7f30ec0108d0 con 0x7f30dc065cd0 2023-12-20T06:04:19.090 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:19.091 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:04:19.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 -- 172.21.15.32:0/4159996135 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f30dc065cd0 msgr2=0x7f30dc068190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f30dc065cd0 0x7f30dc068190 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f30ec006e90 tx=0x7f30ec008040 comp rx=0 tx=0).stop 2023-12-20T06:04:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 -- 172.21.15.32:0/4159996135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc15cdb0 msgr2=0x7f30fc0acdb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc15cdb0 0x7f30fc0acdb0 secure :-1 s=READY pgs=2486 cs=0 l=1 rev1=1 crypto rx=0x7f30f40b2040 tx=0x7f30f401bf60 comp rx=0 tx=0).stop 2023-12-20T06:04:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 -- 172.21.15.32:0/4159996135 shutdown_connections 2023-12-20T06:04:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f30dc065cd0 0x7f30dc068190 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f30fc15cdb0 0x7f30fc0acdb0 unknown :-1 s=CLOSED pgs=2486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f30fc0d2740 0x7f30fc1f60a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 --2- 172.21.15.32:0/4159996135 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f30fc0d1120 0x7f30fc148d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 -- 172.21.15.32:0/4159996135 >> 172.21.15.32:0/4159996135 conn(0x7f30fc065ee0 msgr2=0x7f30fc0c9c30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 -- 172.21.15.32:0/4159996135 shutdown_connections 2023-12-20T06:04:19.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.089+0000 7f3100d18640 1 -- 172.21.15.32:0/4159996135 wait complete. 2023-12-20T06:04:19.110 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-12-20T06:04:17.806793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2173,"num_read_kb":4467,"num_write":625,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3272,"ondisk_log_size":3272,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439208,"kb_used_data":25960,"kb_used_omap":0,"kb_used_meta":413184,"kb_avail":754535512,"statfs":{"total":773094113280,"available":772644364288,"internally_reserved":0,"allocated":26583040,"data_stored":13627620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":423100416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.225334"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.011118+0000","last_change":"2023-12-20T06:04:15.026954+0000","last_active":"2023-12-20T06:04:16.011118+0000","last_peered":"2023-12-20T06:04:16.011118+0000","last_clean":"2023-12-20T06:04:16.011118+0000","last_became_active":"2023-12-20T06:04:15.026638+0000","last_became_peered":"2023-12-20T06:04:15.026638+0000","last_unstale":"2023-12-20T06:04:16.011118+0000","last_undegraded":"2023-12-20T06:04:16.011118+0000","last_fullsized":"2023-12-20T06:04:16.011118+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:27:23.439943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.009633+0000","last_change":"2023-12-20T06:04:15.034718+0000","last_active":"2023-12-20T06:04:16.009633+0000","last_peered":"2023-12-20T06:04:16.009633+0000","last_clean":"2023-12-20T06:04:16.009633+0000","last_became_active":"2023-12-20T06:04:15.034456+0000","last_became_peered":"2023-12-20T06:04:15.034456+0000","last_unstale":"2023-12-20T06:04:16.009633+0000","last_undegraded":"2023-12-20T06:04:16.009633+0000","last_fullsized":"2023-12-20T06:04:16.009633+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:19:05.245786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820981+0000","last_change":"2023-12-20T06:04:15.023349+0000","last_active":"2023-12-20T06:04:16.820981+0000","last_peered":"2023-12-20T06:04:16.820981+0000","last_clean":"2023-12-20T06:04:16.820981+0000","last_became_active":"2023-12-20T06:04:15.020032+0000","last_became_peered":"2023-12-20T06:04:15.020032+0000","last_unstale":"2023-12-20T06:04:16.820981+0000","last_undegraded":"2023-12-20T06:04:16.820981+0000","last_fullsized":"2023-12-20T06:04:16.820981+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:37:04.055455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820500+0000","last_change":"2023-12-20T06:04:15.022832+0000","last_active":"2023-12-20T06:04:16.820500+0000","last_peered":"2023-12-20T06:04:16.820500+0000","last_clean":"2023-12-20T06:04:16.820500+0000","last_became_active":"2023-12-20T06:04:15.022565+0000","last_became_peered":"2023-12-20T06:04:15.022565+0000","last_unstale":"2023-12-20T06:04:16.820500+0000","last_undegraded":"2023-12-20T06:04:16.820500+0000","last_fullsized":"2023-12-20T06:04:16.820500+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:54:27.659337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820186+0000","last_change":"2023-12-20T06:04:15.028145+0000","last_active":"2023-12-20T06:04:16.820186+0000","last_peered":"2023-12-20T06:04:16.820186+0000","last_clean":"2023-12-20T06:04:16.820186+0000","last_became_active":"2023-12-20T06:04:15.027770+0000","last_became_peered":"2023-12-20T06:04:15.027770+0000","last_unstale":"2023-12-20T06:04:16.820186+0000","last_undegraded":"2023-12-20T06:04:16.820186+0000","last_fullsized":"2023-12-20T06:04:16.820186+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:12:20.835387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812567+0000","last_change":"2023-12-20T06:04:15.017083+0000","last_active":"2023-12-20T06:04:16.812567+0000","last_peered":"2023-12-20T06:04:16.812567+0000","last_clean":"2023-12-20T06:04:16.812567+0000","last_became_active":"2023-12-20T06:04:15.016828+0000","last_became_peered":"2023-12-20T06:04:15.016828+0000","last_unstale":"2023-12-20T06:04:16.812567+0000","last_undegraded":"2023-12-20T06:04:16.812567+0000","last_fullsized":"2023-12-20T06:04:16.812567+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:26.547776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820196+0000","last_change":"2023-12-20T06:04:15.033099+0000","last_active":"2023-12-20T06:04:16.820196+0000","last_peered":"2023-12-20T06:04:16.820196+0000","last_clean":"2023-12-20T06:04:16.820196+0000","last_became_active":"2023-12-20T06:04:15.032609+0000","last_became_peered":"2023-12-20T06:04:15.032609+0000","last_unstale":"2023-12-20T06:04:16.820196+0000","last_undegraded":"2023-12-20T06:04:16.820196+0000","last_fullsized":"2023-12-20T06:04:16.820196+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:08:46.920960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010662+0000","last_change":"2023-12-20T06:04:15.023347+0000","last_active":"2023-12-20T06:04:16.010662+0000","last_peered":"2023-12-20T06:04:16.010662+0000","last_clean":"2023-12-20T06:04:16.010662+0000","last_became_active":"2023-12-20T06:04:15.023043+0000","last_became_peered":"2023-12-20T06:04:15.023043+0000","last_unstale":"2023-12-20T06:04:16.010662+0000","last_undegraded":"2023-12-20T06:04:16.010662+0000","last_fullsized":"2023-12-20T06:04:16.010662+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:31:54.461212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822446+0000","last_change":"2023-12-20T06:04:15.016762+0000","last_active":"2023-12-20T06:04:16.822446+0000","last_peered":"2023-12-20T06:04:16.822446+0000","last_clean":"2023-12-20T06:04:16.822446+0000","last_became_active":"2023-12-20T06:04:15.016504+0000","last_became_peered":"2023-12-20T06:04:15.016504+0000","last_unstale":"2023-12-20T06:04:16.822446+0000","last_undegraded":"2023-12-20T06:04:16.822446+0000","last_fullsized":"2023-12-20T06:04:16.822446+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:19:15.528169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820950+0000","last_change":"2023-12-20T06:04:15.030301+0000","last_active":"2023-12-20T06:04:16.820950+0000","last_peered":"2023-12-20T06:04:16.820950+0000","last_clean":"2023-12-20T06:04:16.820950+0000","last_became_active":"2023-12-20T06:04:15.029992+0000","last_became_peered":"2023-12-20T06:04:15.029992+0000","last_unstale":"2023-12-20T06:04:16.820950+0000","last_undegraded":"2023-12-20T06:04:16.820950+0000","last_fullsized":"2023-12-20T06:04:16.820950+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:59:33.073877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820664+0000","last_change":"2023-12-20T06:04:15.024722+0000","last_active":"2023-12-20T06:04:16.820664+0000","last_peered":"2023-12-20T06:04:16.820664+0000","last_clean":"2023-12-20T06:04:16.820664+0000","last_became_active":"2023-12-20T06:04:15.024083+0000","last_became_peered":"2023-12-20T06:04:15.024083+0000","last_unstale":"2023-12-20T06:04:16.820664+0000","last_undegraded":"2023-12-20T06:04:16.820664+0000","last_fullsized":"2023-12-20T06:04:16.820664+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:59:56.344194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010886+0000","last_change":"2023-12-20T06:04:15.016325+0000","last_active":"2023-12-20T06:04:16.010886+0000","last_peered":"2023-12-20T06:04:16.010886+0000","last_clean":"2023-12-20T06:04:16.010886+0000","last_became_active":"2023-12-20T06:04:15.013781+0000","last_became_peered":"2023-12-20T06:04:15.013781+0000","last_unstale":"2023-12-20T06:04:16.010886+0000","last_undegraded":"2023-12-20T06:04:16.010886+0000","last_fullsized":"2023-12-20T06:04:16.010886+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:17:36.518319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107603+0000","last_change":"2023-12-20T06:04:15.025021+0000","last_active":"2023-12-20T06:04:16.107603+0000","last_peered":"2023-12-20T06:04:16.107603+0000","last_clean":"2023-12-20T06:04:16.107603+0000","last_became_active":"2023-12-20T06:04:15.024762+0000","last_became_peered":"2023-12-20T06:04:15.024762+0000","last_unstale":"2023-12-20T06:04:16.107603+0000","last_undegraded":"2023-12-20T06:04:16.107603+0000","last_fullsized":"2023-12-20T06:04:16.107603+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:33:23.963598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107604+0000","last_change":"2023-12-20T06:04:15.018183+0000","last_active":"2023-12-20T06:04:16.107604+0000","last_peered":"2023-12-20T06:04:16.107604+0000","last_clean":"2023-12-20T06:04:16.107604+0000","last_became_active":"2023-12-20T06:04:15.017874+0000","last_became_peered":"2023-12-20T06:04:15.017874+0000","last_unstale":"2023-12-20T06:04:16.107604+0000","last_undegraded":"2023-12-20T06:04:16.107604+0000","last_fullsized":"2023-12-20T06:04:16.107604+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:19:05.245786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107802+0000","last_change":"2023-12-20T06:04:15.028339+0000","last_active":"2023-12-20T06:04:16.107802+0000","last_peered":"2023-12-20T06:04:16.107802+0000","last_clean":"2023-12-20T06:04:16.107802+0000","last_became_active":"2023-12-20T06:04:15.028087+0000","last_became_peered":"2023-12-20T06:04:15.028087+0000","last_unstale":"2023-12-20T06:04:16.107802+0000","last_undegraded":"2023-12-20T06:04:16.107802+0000","last_fullsized":"2023-12-20T06:04:16.107802+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:55:05.066303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812552+0000","last_change":"2023-12-20T06:04:15.024698+0000","last_active":"2023-12-20T06:04:16.812552+0000","last_peered":"2023-12-20T06:04:16.812552+0000","last_clean":"2023-12-20T06:04:16.812552+0000","last_became_active":"2023-12-20T06:04:15.024045+0000","last_became_peered":"2023-12-20T06:04:15.024045+0000","last_unstale":"2023-12-20T06:04:16.812552+0000","last_undegraded":"2023-12-20T06:04:16.812552+0000","last_fullsized":"2023-12-20T06:04:16.812552+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:39:53.409301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007078+0000","last_change":"2023-12-20T06:04:15.034428+0000","last_active":"2023-12-20T06:04:16.007078+0000","last_peered":"2023-12-20T06:04:16.007078+0000","last_clean":"2023-12-20T06:04:16.007078+0000","last_became_active":"2023-12-20T06:04:15.034154+0000","last_became_peered":"2023-12-20T06:04:15.034154+0000","last_unstale":"2023-12-20T06:04:16.007078+0000","last_undegraded":"2023-12-20T06:04:16.007078+0000","last_fullsized":"2023-12-20T06:04:16.007078+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:57:11.599590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821165+0000","last_change":"2023-12-20T06:04:15.032943+0000","last_active":"2023-12-20T06:04:16.821165+0000","last_peered":"2023-12-20T06:04:16.821165+0000","last_clean":"2023-12-20T06:04:16.821165+0000","last_became_active":"2023-12-20T06:04:15.031957+0000","last_became_peered":"2023-12-20T06:04:15.031957+0000","last_unstale":"2023-12-20T06:04:16.821165+0000","last_undegraded":"2023-12-20T06:04:16.821165+0000","last_fullsized":"2023-12-20T06:04:16.821165+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:39:53.409301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820691+0000","last_change":"2023-12-20T06:04:15.030149+0000","last_active":"2023-12-20T06:04:16.820691+0000","last_peered":"2023-12-20T06:04:16.820691+0000","last_clean":"2023-12-20T06:04:16.820691+0000","last_became_active":"2023-12-20T06:04:15.029866+0000","last_became_peered":"2023-12-20T06:04:15.029866+0000","last_unstale":"2023-12-20T06:04:16.820691+0000","last_undegraded":"2023-12-20T06:04:16.820691+0000","last_fullsized":"2023-12-20T06:04:16.820691+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:20:34.577593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812754+0000","last_change":"2023-12-20T06:04:15.012374+0000","last_active":"2023-12-20T06:04:16.812754+0000","last_peered":"2023-12-20T06:04:16.812754+0000","last_clean":"2023-12-20T06:04:16.812754+0000","last_became_active":"2023-12-20T06:04:15.012052+0000","last_became_peered":"2023-12-20T06:04:15.012052+0000","last_unstale":"2023-12-20T06:04:16.812754+0000","last_undegraded":"2023-12-20T06:04:16.812754+0000","last_fullsized":"2023-12-20T06:04:16.812754+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:08:46.920960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822101+0000","last_change":"2023-12-20T06:04:15.026651+0000","last_active":"2023-12-20T06:04:16.822101+0000","last_peered":"2023-12-20T06:04:16.822101+0000","last_clean":"2023-12-20T06:04:16.822101+0000","last_became_active":"2023-12-20T06:04:15.026357+0000","last_became_peered":"2023-12-20T06:04:15.026357+0000","last_unstale":"2023-12-20T06:04:16.822101+0000","last_undegraded":"2023-12-20T06:04:16.822101+0000","last_fullsized":"2023-12-20T06:04:16.822101+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:22:34.434439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822199+0000","last_change":"2023-12-20T06:04:15.017311+0000","last_active":"2023-12-20T06:04:16.822199+0000","last_peered":"2023-12-20T06:04:16.822199+0000","last_clean":"2023-12-20T06:04:16.822199+0000","last_became_active":"2023-12-20T06:04:15.016074+0000","last_became_peered":"2023-12-20T06:04:15.016074+0000","last_unstale":"2023-12-20T06:04:16.822199+0000","last_undegraded":"2023-12-20T06:04:16.822199+0000","last_fullsized":"2023-12-20T06:04:16.822199+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:32:16.096870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820324+0000","last_change":"2023-12-20T06:04:15.028424+0000","last_active":"2023-12-20T06:04:16.820324+0000","last_peered":"2023-12-20T06:04:16.820324+0000","last_clean":"2023-12-20T06:04:16.820324+0000","last_became_active":"2023-12-20T06:04:15.028118+0000","last_became_peered":"2023-12-20T06:04:15.028118+0000","last_unstale":"2023-12-20T06:04:16.820324+0000","last_undegraded":"2023-12-20T06:04:16.820324+0000","last_fullsized":"2023-12-20T06:04:16.820324+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:40:09.280259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007195+0000","last_change":"2023-12-20T06:04:15.020161+0000","last_active":"2023-12-20T06:04:16.007195+0000","last_peered":"2023-12-20T06:04:16.007195+0000","last_clean":"2023-12-20T06:04:16.007195+0000","last_became_active":"2023-12-20T06:04:15.019350+0000","last_became_peered":"2023-12-20T06:04:15.019350+0000","last_unstale":"2023-12-20T06:04:16.007195+0000","last_undegraded":"2023-12-20T06:04:16.007195+0000","last_fullsized":"2023-12-20T06:04:16.007195+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:05:08.516815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107839+0000","last_change":"2023-12-20T06:04:15.029858+0000","last_active":"2023-12-20T06:04:16.107839+0000","last_peered":"2023-12-20T06:04:16.107839+0000","last_clean":"2023-12-20T06:04:16.107839+0000","last_became_active":"2023-12-20T06:04:15.029628+0000","last_became_peered":"2023-12-20T06:04:15.029628+0000","last_unstale":"2023-12-20T06:04:16.107839+0000","last_undegraded":"2023-12-20T06:04:16.107839+0000","last_fullsized":"2023-12-20T06:04:16.107839+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:33:55.595521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821163+0000","last_change":"2023-12-20T06:04:15.030617+0000","last_active":"2023-12-20T06:04:16.821163+0000","last_peered":"2023-12-20T06:04:16.821163+0000","last_clean":"2023-12-20T06:04:16.821163+0000","last_became_active":"2023-12-20T06:04:15.030356+0000","last_became_peered":"2023-12-20T06:04:15.030356+0000","last_unstale":"2023-12-20T06:04:16.821163+0000","last_undegraded":"2023-12-20T06:04:16.821163+0000","last_fullsized":"2023-12-20T06:04:16.821163+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:53:45.234376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820816+0000","last_change":"2023-12-20T06:04:15.028795+0000","last_active":"2023-12-20T06:04:16.820816+0000","last_peered":"2023-12-20T06:04:16.820816+0000","last_clean":"2023-12-20T06:04:16.820816+0000","last_became_active":"2023-12-20T06:04:15.028501+0000","last_became_peered":"2023-12-20T06:04:15.028501+0000","last_unstale":"2023-12-20T06:04:16.820816+0000","last_undegraded":"2023-12-20T06:04:16.820816+0000","last_fullsized":"2023-12-20T06:04:16.820816+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:41:29.330308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107795+0000","last_change":"2023-12-20T06:04:15.031751+0000","last_active":"2023-12-20T06:04:16.107795+0000","last_peered":"2023-12-20T06:04:16.107795+0000","last_clean":"2023-12-20T06:04:16.107795+0000","last_became_active":"2023-12-20T06:04:15.030983+0000","last_became_peered":"2023-12-20T06:04:15.030983+0000","last_unstale":"2023-12-20T06:04:16.107795+0000","last_undegraded":"2023-12-20T06:04:16.107795+0000","last_fullsized":"2023-12-20T06:04:16.107795+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:08:03.478691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007128+0000","last_change":"2023-12-20T06:04:15.035283+0000","last_active":"2023-12-20T06:04:16.007128+0000","last_peered":"2023-12-20T06:04:16.007128+0000","last_clean":"2023-12-20T06:04:16.007128+0000","last_became_active":"2023-12-20T06:04:15.034958+0000","last_became_peered":"2023-12-20T06:04:15.034958+0000","last_unstale":"2023-12-20T06:04:16.007128+0000","last_undegraded":"2023-12-20T06:04:16.007128+0000","last_fullsized":"2023-12-20T06:04:16.007128+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:03.705924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107625+0000","last_change":"2023-12-20T06:04:15.018149+0000","last_active":"2023-12-20T06:04:16.107625+0000","last_peered":"2023-12-20T06:04:16.107625+0000","last_clean":"2023-12-20T06:04:16.107625+0000","last_became_active":"2023-12-20T06:04:15.017679+0000","last_became_peered":"2023-12-20T06:04:15.017679+0000","last_unstale":"2023-12-20T06:04:16.107625+0000","last_undegraded":"2023-12-20T06:04:16.107625+0000","last_fullsized":"2023-12-20T06:04:16.107625+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:52.510739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010661+0000","last_change":"2023-12-20T06:04:15.024986+0000","last_active":"2023-12-20T06:04:16.010661+0000","last_peered":"2023-12-20T06:04:16.010661+0000","last_clean":"2023-12-20T06:04:16.010661+0000","last_became_active":"2023-12-20T06:04:15.024700+0000","last_became_peered":"2023-12-20T06:04:15.024700+0000","last_unstale":"2023-12-20T06:04:16.010661+0000","last_undegraded":"2023-12-20T06:04:16.010661+0000","last_fullsized":"2023-12-20T06:04:16.010661+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:04:21.452369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820966+0000","last_change":"2023-12-20T06:04:15.029574+0000","last_active":"2023-12-20T06:04:16.820966+0000","last_peered":"2023-12-20T06:04:16.820966+0000","last_clean":"2023-12-20T06:04:16.820966+0000","last_became_active":"2023-12-20T06:04:15.029161+0000","last_became_peered":"2023-12-20T06:04:15.029161+0000","last_unstale":"2023-12-20T06:04:16.820966+0000","last_undegraded":"2023-12-20T06:04:16.820966+0000","last_fullsized":"2023-12-20T06:04:16.820966+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:51.205184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"250'4","reported_seq":15,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007268+0000","last_change":"2023-12-20T06:04:14.616151+0000","last_active":"2023-12-20T06:04:16.007268+0000","last_peered":"2023-12-20T06:04:16.007268+0000","last_clean":"2023-12-20T06:04:16.007268+0000","last_became_active":"2023-12-20T06:04:14.615873+0000","last_became_peered":"2023-12-20T06:04:14.615873+0000","last_unstale":"2023-12-20T06:04:16.007268+0000","last_undegraded":"2023-12-20T06:04:16.007268+0000","last_fullsized":"2023-12-20T06:04:16.007268+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:06:23.720731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107845+0000","last_change":"2023-12-20T06:04:15.027605+0000","last_active":"2023-12-20T06:04:16.107845+0000","last_peered":"2023-12-20T06:04:16.107845+0000","last_clean":"2023-12-20T06:04:16.107845+0000","last_became_active":"2023-12-20T06:04:15.027355+0000","last_became_peered":"2023-12-20T06:04:15.027355+0000","last_unstale":"2023-12-20T06:04:16.107845+0000","last_undegraded":"2023-12-20T06:04:16.107845+0000","last_fullsized":"2023-12-20T06:04:16.107845+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T18:02:29.573283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"250'1","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107979+0000","last_change":"2023-12-20T06:04:14.606296+0000","last_active":"2023-12-20T06:04:16.107979+0000","last_peered":"2023-12-20T06:04:16.107979+0000","last_clean":"2023-12-20T06:04:16.107979+0000","last_became_active":"2023-12-20T06:04:14.606034+0000","last_became_peered":"2023-12-20T06:04:14.606034+0000","last_unstale":"2023-12-20T06:04:16.107979+0000","last_undegraded":"2023-12-20T06:04:16.107979+0000","last_fullsized":"2023-12-20T06:04:16.107979+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:08:50.240928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821098+0000","last_change":"2023-12-20T06:04:15.020844+0000","last_active":"2023-12-20T06:04:16.821098+0000","last_peered":"2023-12-20T06:04:16.821098+0000","last_clean":"2023-12-20T06:04:16.821098+0000","last_became_active":"2023-12-20T06:04:15.020327+0000","last_became_peered":"2023-12-20T06:04:15.020327+0000","last_unstale":"2023-12-20T06:04:16.821098+0000","last_undegraded":"2023-12-20T06:04:16.821098+0000","last_fullsized":"2023-12-20T06:04:16.821098+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:25:09.673494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.813052+0000","last_change":"2023-12-20T06:04:14.609615+0000","last_active":"2023-12-20T06:04:16.813052+0000","last_peered":"2023-12-20T06:04:16.813052+0000","last_clean":"2023-12-20T06:04:16.813052+0000","last_became_active":"2023-12-20T06:04:14.606481+0000","last_became_peered":"2023-12-20T06:04:14.606481+0000","last_unstale":"2023-12-20T06:04:16.813052+0000","last_undegraded":"2023-12-20T06:04:16.813052+0000","last_fullsized":"2023-12-20T06:04:16.813052+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:04:20.435058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010988+0000","last_change":"2023-12-20T06:04:15.026414+0000","last_active":"2023-12-20T06:04:16.010988+0000","last_peered":"2023-12-20T06:04:16.010988+0000","last_clean":"2023-12-20T06:04:16.010988+0000","last_became_active":"2023-12-20T06:04:15.025955+0000","last_became_peered":"2023-12-20T06:04:15.025955+0000","last_unstale":"2023-12-20T06:04:16.010988+0000","last_undegraded":"2023-12-20T06:04:16.010988+0000","last_fullsized":"2023-12-20T06:04:16.010988+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:28:58.470204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812809+0000","last_change":"2023-12-20T06:04:14.612409+0000","last_active":"2023-12-20T06:04:16.812809+0000","last_peered":"2023-12-20T06:04:16.812809+0000","last_clean":"2023-12-20T06:04:16.812809+0000","last_became_active":"2023-12-20T06:04:14.612148+0000","last_became_peered":"2023-12-20T06:04:14.612148+0000","last_unstale":"2023-12-20T06:04:16.812809+0000","last_undegraded":"2023-12-20T06:04:16.812809+0000","last_fullsized":"2023-12-20T06:04:16.812809+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:24:39.898860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822035+0000","last_change":"2023-12-20T06:04:15.019053+0000","last_active":"2023-12-20T06:04:16.822035+0000","last_peered":"2023-12-20T06:04:16.822035+0000","last_clean":"2023-12-20T06:04:16.822035+0000","last_became_active":"2023-12-20T06:04:15.018794+0000","last_became_peered":"2023-12-20T06:04:15.018794+0000","last_unstale":"2023-12-20T06:04:16.822035+0000","last_undegraded":"2023-12-20T06:04:16.822035+0000","last_fullsized":"2023-12-20T06:04:16.822035+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:30:46.585415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"250'1","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820606+0000","last_change":"2023-12-20T06:04:14.606279+0000","last_active":"2023-12-20T06:04:16.820606+0000","last_peered":"2023-12-20T06:04:16.820606+0000","last_clean":"2023-12-20T06:04:16.820606+0000","last_became_active":"2023-12-20T06:04:14.605930+0000","last_became_peered":"2023-12-20T06:04:14.605930+0000","last_unstale":"2023-12-20T06:04:16.820606+0000","last_undegraded":"2023-12-20T06:04:16.820606+0000","last_fullsized":"2023-12-20T06:04:16.820606+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:24:39.898860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812896+0000","last_change":"2023-12-20T06:04:15.029990+0000","last_active":"2023-12-20T06:04:16.812896+0000","last_peered":"2023-12-20T06:04:16.812896+0000","last_clean":"2023-12-20T06:04:16.812896+0000","last_became_active":"2023-12-20T06:04:15.029734+0000","last_became_peered":"2023-12-20T06:04:15.029734+0000","last_unstale":"2023-12-20T06:04:16.812896+0000","last_undegraded":"2023-12-20T06:04:16.812896+0000","last_fullsized":"2023-12-20T06:04:16.812896+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:22:51.632261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"250'1","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006969+0000","last_change":"2023-12-20T06:04:14.607464+0000","last_active":"2023-12-20T06:04:16.006969+0000","last_peered":"2023-12-20T06:04:16.006969+0000","last_clean":"2023-12-20T06:04:16.006969+0000","last_became_active":"2023-12-20T06:04:14.607200+0000","last_became_peered":"2023-12-20T06:04:14.607200+0000","last_unstale":"2023-12-20T06:04:16.006969+0000","last_undegraded":"2023-12-20T06:04:16.006969+0000","last_fullsized":"2023-12-20T06:04:16.006969+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:12:25.714647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812552+0000","last_change":"2023-12-20T06:04:15.029634+0000","last_active":"2023-12-20T06:04:16.812552+0000","last_peered":"2023-12-20T06:04:16.812552+0000","last_clean":"2023-12-20T06:04:16.812552+0000","last_became_active":"2023-12-20T06:04:15.028447+0000","last_became_peered":"2023-12-20T06:04:15.028447+0000","last_unstale":"2023-12-20T06:04:16.812552+0000","last_undegraded":"2023-12-20T06:04:16.812552+0000","last_fullsized":"2023-12-20T06:04:16.812552+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:20:34.577593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820155+0000","last_change":"2023-12-20T06:04:14.610708+0000","last_active":"2023-12-20T06:04:16.820155+0000","last_peered":"2023-12-20T06:04:16.820155+0000","last_clean":"2023-12-20T06:04:16.820155+0000","last_became_active":"2023-12-20T06:04:14.610395+0000","last_became_peered":"2023-12-20T06:04:14.610395+0000","last_unstale":"2023-12-20T06:04:16.820155+0000","last_undegraded":"2023-12-20T06:04:16.820155+0000","last_fullsized":"2023-12-20T06:04:16.820155+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:10:04.670922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820436+0000","last_change":"2023-12-20T06:04:15.031728+0000","last_active":"2023-12-20T06:04:16.820436+0000","last_peered":"2023-12-20T06:04:16.820436+0000","last_clean":"2023-12-20T06:04:16.820436+0000","last_became_active":"2023-12-20T06:04:15.031437+0000","last_became_peered":"2023-12-20T06:04:15.031437+0000","last_unstale":"2023-12-20T06:04:16.820436+0000","last_undegraded":"2023-12-20T06:04:16.820436+0000","last_fullsized":"2023-12-20T06:04:16.820436+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:22:51.632261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820322+0000","last_change":"2023-12-20T06:04:14.595955+0000","last_active":"2023-12-20T06:04:16.820322+0000","last_peered":"2023-12-20T06:04:16.820322+0000","last_clean":"2023-12-20T06:04:16.820322+0000","last_became_active":"2023-12-20T06:04:14.595565+0000","last_became_peered":"2023-12-20T06:04:14.595565+0000","last_unstale":"2023-12-20T06:04:16.820322+0000","last_undegraded":"2023-12-20T06:04:16.820322+0000","last_fullsized":"2023-12-20T06:04:16.820322+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:44:53.092997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821093+0000","last_change":"2023-12-20T06:04:15.031975+0000","last_active":"2023-12-20T06:04:16.821093+0000","last_peered":"2023-12-20T06:04:16.821093+0000","last_clean":"2023-12-20T06:04:16.821093+0000","last_became_active":"2023-12-20T06:04:15.031705+0000","last_became_peered":"2023-12-20T06:04:15.031705+0000","last_unstale":"2023-12-20T06:04:16.821093+0000","last_undegraded":"2023-12-20T06:04:16.821093+0000","last_fullsized":"2023-12-20T06:04:16.821093+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:58:11.789777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007323+0000","last_change":"2023-12-20T06:04:14.610927+0000","last_active":"2023-12-20T06:04:16.007323+0000","last_peered":"2023-12-20T06:04:16.007323+0000","last_clean":"2023-12-20T06:04:16.007323+0000","last_became_active":"2023-12-20T06:04:14.610622+0000","last_became_peered":"2023-12-20T06:04:14.610622+0000","last_unstale":"2023-12-20T06:04:16.007323+0000","last_undegraded":"2023-12-20T06:04:16.007323+0000","last_fullsized":"2023-12-20T06:04:16.007323+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:17:09.477999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007196+0000","last_change":"2023-12-20T06:04:15.022033+0000","last_active":"2023-12-20T06:04:16.007196+0000","last_peered":"2023-12-20T06:04:16.007196+0000","last_clean":"2023-12-20T06:04:16.007196+0000","last_became_active":"2023-12-20T06:04:15.021775+0000","last_became_peered":"2023-12-20T06:04:15.021775+0000","last_unstale":"2023-12-20T06:04:16.007196+0000","last_undegraded":"2023-12-20T06:04:16.007196+0000","last_fullsized":"2023-12-20T06:04:16.007196+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:08:48.223773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812744+0000","last_change":"2023-12-20T06:04:14.606809+0000","last_active":"2023-12-20T06:04:16.812744+0000","last_peered":"2023-12-20T06:04:16.812744+0000","last_clean":"2023-12-20T06:04:16.812744+0000","last_became_active":"2023-12-20T06:04:14.606456+0000","last_became_peered":"2023-12-20T06:04:14.606456+0000","last_unstale":"2023-12-20T06:04:16.812744+0000","last_undegraded":"2023-12-20T06:04:16.812744+0000","last_fullsized":"2023-12-20T06:04:16.812744+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:15:30.384161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107668+0000","last_change":"2023-12-20T06:04:15.022070+0000","last_active":"2023-12-20T06:04:16.107668+0000","last_peered":"2023-12-20T06:04:16.107668+0000","last_clean":"2023-12-20T06:04:16.107668+0000","last_became_active":"2023-12-20T06:04:15.020840+0000","last_became_peered":"2023-12-20T06:04:15.020840+0000","last_unstale":"2023-12-20T06:04:16.107668+0000","last_undegraded":"2023-12-20T06:04:16.107668+0000","last_fullsized":"2023-12-20T06:04:16.107668+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:43:22.692480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"250'3","reported_seq":14,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.813080+0000","last_change":"2023-12-20T06:04:14.608212+0000","last_active":"2023-12-20T06:04:16.813080+0000","last_peered":"2023-12-20T06:04:16.813080+0000","last_clean":"2023-12-20T06:04:16.813080+0000","last_became_active":"2023-12-20T06:04:14.606396+0000","last_became_peered":"2023-12-20T06:04:14.606396+0000","last_unstale":"2023-12-20T06:04:16.813080+0000","last_undegraded":"2023-12-20T06:04:16.813080+0000","last_fullsized":"2023-12-20T06:04:16.813080+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:44:53.092997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007322+0000","last_change":"2023-12-20T06:04:15.025640+0000","last_active":"2023-12-20T06:04:16.007322+0000","last_peered":"2023-12-20T06:04:16.007322+0000","last_clean":"2023-12-20T06:04:16.007322+0000","last_became_active":"2023-12-20T06:04:15.025393+0000","last_became_peered":"2023-12-20T06:04:15.025393+0000","last_unstale":"2023-12-20T06:04:16.007322+0000","last_undegraded":"2023-12-20T06:04:16.007322+0000","last_fullsized":"2023-12-20T06:04:16.007322+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:45:19.685093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"250'6","reported_seq":17,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006988+0000","last_change":"2023-12-20T06:04:14.612263+0000","last_active":"2023-12-20T06:04:16.006988+0000","last_peered":"2023-12-20T06:04:16.006988+0000","last_clean":"2023-12-20T06:04:16.006988+0000","last_became_active":"2023-12-20T06:04:14.611940+0000","last_became_peered":"2023-12-20T06:04:14.611940+0000","last_unstale":"2023-12-20T06:04:16.006988+0000","last_undegraded":"2023-12-20T06:04:16.006988+0000","last_fullsized":"2023-12-20T06:04:16.006988+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:14:40.868048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006874+0000","last_change":"2023-12-20T06:04:15.023822+0000","last_active":"2023-12-20T06:04:16.006874+0000","last_peered":"2023-12-20T06:04:16.006874+0000","last_clean":"2023-12-20T06:04:16.006874+0000","last_became_active":"2023-12-20T06:04:15.023577+0000","last_became_peered":"2023-12-20T06:04:15.023577+0000","last_unstale":"2023-12-20T06:04:16.006874+0000","last_undegraded":"2023-12-20T06:04:16.006874+0000","last_fullsized":"2023-12-20T06:04:16.006874+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:00:11.404637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010847+0000","last_change":"2023-12-20T06:04:14.606823+0000","last_active":"2023-12-20T06:04:16.010847+0000","last_peered":"2023-12-20T06:04:16.010847+0000","last_clean":"2023-12-20T06:04:16.010847+0000","last_became_active":"2023-12-20T06:04:14.604714+0000","last_became_peered":"2023-12-20T06:04:14.604714+0000","last_unstale":"2023-12-20T06:04:16.010847+0000","last_undegraded":"2023-12-20T06:04:16.010847+0000","last_fullsized":"2023-12-20T06:04:16.010847+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:06:42.173623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812916+0000","last_change":"2023-12-20T06:04:15.028720+0000","last_active":"2023-12-20T06:04:16.812916+0000","last_peered":"2023-12-20T06:04:16.812916+0000","last_clean":"2023-12-20T06:04:16.812916+0000","last_became_active":"2023-12-20T06:04:15.028413+0000","last_became_peered":"2023-12-20T06:04:15.028413+0000","last_unstale":"2023-12-20T06:04:16.812916+0000","last_undegraded":"2023-12-20T06:04:16.812916+0000","last_fullsized":"2023-12-20T06:04:16.812916+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:40:09.280259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822132+0000","last_change":"2023-12-20T06:04:14.607764+0000","last_active":"2023-12-20T06:04:16.822132+0000","last_peered":"2023-12-20T06:04:16.822132+0000","last_clean":"2023-12-20T06:04:16.822132+0000","last_became_active":"2023-12-20T06:04:14.607456+0000","last_became_peered":"2023-12-20T06:04:14.607456+0000","last_unstale":"2023-12-20T06:04:16.822132+0000","last_undegraded":"2023-12-20T06:04:16.822132+0000","last_fullsized":"2023-12-20T06:04:16.822132+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:05:07.499505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006817+0000","last_change":"2023-12-20T06:04:15.019988+0000","last_active":"2023-12-20T06:04:16.006817+0000","last_peered":"2023-12-20T06:04:16.006817+0000","last_clean":"2023-12-20T06:04:16.006817+0000","last_became_active":"2023-12-20T06:04:15.019623+0000","last_became_peered":"2023-12-20T06:04:15.019623+0000","last_unstale":"2023-12-20T06:04:16.006817+0000","last_undegraded":"2023-12-20T06:04:16.006817+0000","last_fullsized":"2023-12-20T06:04:16.006817+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:38:34.548311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.009509+0000","last_change":"2023-12-20T06:04:14.612181+0000","last_active":"2023-12-20T06:04:16.009509+0000","last_peered":"2023-12-20T06:04:16.009509+0000","last_clean":"2023-12-20T06:04:16.009509+0000","last_became_active":"2023-12-20T06:04:14.611534+0000","last_became_peered":"2023-12-20T06:04:14.611534+0000","last_unstale":"2023-12-20T06:04:16.009509+0000","last_undegraded":"2023-12-20T06:04:16.009509+0000","last_fullsized":"2023-12-20T06:04:16.009509+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T18:02:28.555973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006821+0000","last_change":"2023-12-20T06:04:15.020621+0000","last_active":"2023-12-20T06:04:16.006821+0000","last_peered":"2023-12-20T06:04:16.006821+0000","last_clean":"2023-12-20T06:04:16.006821+0000","last_became_active":"2023-12-20T06:04:15.020358+0000","last_became_peered":"2023-12-20T06:04:15.020358+0000","last_unstale":"2023-12-20T06:04:16.006821+0000","last_undegraded":"2023-12-20T06:04:16.006821+0000","last_fullsized":"2023-12-20T06:04:16.006821+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:43:16.055416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"250'8","reported_seq":19,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812688+0000","last_change":"2023-12-20T06:04:14.611694+0000","last_active":"2023-12-20T06:04:16.812688+0000","last_peered":"2023-12-20T06:04:16.812688+0000","last_clean":"2023-12-20T06:04:16.812688+0000","last_became_active":"2023-12-20T06:04:14.611419+0000","last_became_peered":"2023-12-20T06:04:14.611419+0000","last_unstale":"2023-12-20T06:04:16.812688+0000","last_undegraded":"2023-12-20T06:04:16.812688+0000","last_fullsized":"2023-12-20T06:04:16.812688+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:23:47.189516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107799+0000","last_change":"2023-12-20T06:04:15.030753+0000","last_active":"2023-12-20T06:04:16.107799+0000","last_peered":"2023-12-20T06:04:16.107799+0000","last_clean":"2023-12-20T06:04:16.107799+0000","last_became_active":"2023-12-20T06:04:15.029827+0000","last_became_peered":"2023-12-20T06:04:15.029827+0000","last_unstale":"2023-12-20T06:04:16.107799+0000","last_undegraded":"2023-12-20T06:04:16.107799+0000","last_fullsized":"2023-12-20T06:04:16.107799+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:43.514521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"250'1","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.011118+0000","last_change":"2023-12-20T06:04:14.607628+0000","last_active":"2023-12-20T06:04:16.011118+0000","last_peered":"2023-12-20T06:04:16.011118+0000","last_clean":"2023-12-20T06:04:16.011118+0000","last_became_active":"2023-12-20T06:04:14.607093+0000","last_became_peered":"2023-12-20T06:04:14.607093+0000","last_unstale":"2023-12-20T06:04:16.011118+0000","last_undegraded":"2023-12-20T06:04:16.011118+0000","last_fullsized":"2023-12-20T06:04:16.011118+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:30:02.428212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.009584+0000","last_change":"2023-12-20T06:04:15.034348+0000","last_active":"2023-12-20T06:04:16.009584+0000","last_peered":"2023-12-20T06:04:16.009584+0000","last_clean":"2023-12-20T06:04:16.009584+0000","last_became_active":"2023-12-20T06:04:15.034065+0000","last_became_peered":"2023-12-20T06:04:15.034065+0000","last_unstale":"2023-12-20T06:04:16.009584+0000","last_undegraded":"2023-12-20T06:04:16.009584+0000","last_fullsized":"2023-12-20T06:04:16.009584+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:42:11.485396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812767+0000","last_change":"2023-12-20T06:04:14.607358+0000","last_active":"2023-12-20T06:04:16.812767+0000","last_peered":"2023-12-20T06:04:16.812767+0000","last_clean":"2023-12-20T06:04:16.812767+0000","last_became_active":"2023-12-20T06:04:14.607100+0000","last_became_peered":"2023-12-20T06:04:14.607100+0000","last_unstale":"2023-12-20T06:04:16.812767+0000","last_undegraded":"2023-12-20T06:04:16.812767+0000","last_fullsized":"2023-12-20T06:04:16.812767+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:44:39.078018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107747+0000","last_change":"2023-12-20T06:04:15.028413+0000","last_active":"2023-12-20T06:04:16.107747+0000","last_peered":"2023-12-20T06:04:16.107747+0000","last_clean":"2023-12-20T06:04:16.107747+0000","last_became_active":"2023-12-20T06:04:15.028141+0000","last_became_peered":"2023-12-20T06:04:15.028141+0000","last_unstale":"2023-12-20T06:04:16.107747+0000","last_undegraded":"2023-12-20T06:04:16.107747+0000","last_fullsized":"2023-12-20T06:04:16.107747+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:23:41.923063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"250'1","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.009230+0000","last_change":"2023-12-20T06:04:14.614344+0000","last_active":"2023-12-20T06:04:16.009230+0000","last_peered":"2023-12-20T06:04:16.009230+0000","last_clean":"2023-12-20T06:04:16.009230+0000","last_became_active":"2023-12-20T06:04:14.614107+0000","last_became_peered":"2023-12-20T06:04:14.614107+0000","last_unstale":"2023-12-20T06:04:16.009230+0000","last_undegraded":"2023-12-20T06:04:16.009230+0000","last_fullsized":"2023-12-20T06:04:16.009230+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:10:04.670922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821029+0000","last_change":"2023-12-20T06:04:15.023242+0000","last_active":"2023-12-20T06:04:16.821029+0000","last_peered":"2023-12-20T06:04:16.821029+0000","last_clean":"2023-12-20T06:04:16.821029+0000","last_became_active":"2023-12-20T06:04:15.022920+0000","last_became_peered":"2023-12-20T06:04:15.022920+0000","last_unstale":"2023-12-20T06:04:16.821029+0000","last_undegraded":"2023-12-20T06:04:16.821029+0000","last_fullsized":"2023-12-20T06:04:16.821029+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:26.067257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107837+0000","last_change":"2023-12-20T06:04:14.609097+0000","last_active":"2023-12-20T06:04:16.107837+0000","last_peered":"2023-12-20T06:04:16.107837+0000","last_clean":"2023-12-20T06:04:16.107837+0000","last_became_active":"2023-12-20T06:04:14.608790+0000","last_became_peered":"2023-12-20T06:04:14.608790+0000","last_unstale":"2023-12-20T06:04:16.107837+0000","last_undegraded":"2023-12-20T06:04:16.107837+0000","last_fullsized":"2023-12-20T06:04:16.107837+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:05:24.770775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812714+0000","last_change":"2023-12-20T06:04:15.015038+0000","last_active":"2023-12-20T06:04:16.812714+0000","last_peered":"2023-12-20T06:04:16.812714+0000","last_clean":"2023-12-20T06:04:16.812714+0000","last_became_active":"2023-12-20T06:04:15.014734+0000","last_became_peered":"2023-12-20T06:04:15.014734+0000","last_unstale":"2023-12-20T06:04:16.812714+0000","last_undegraded":"2023-12-20T06:04:16.812714+0000","last_fullsized":"2023-12-20T06:04:16.812714+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:59:33.073877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820352+0000","last_change":"2023-12-20T06:04:14.601943+0000","last_active":"2023-12-20T06:04:16.820352+0000","last_peered":"2023-12-20T06:04:16.820352+0000","last_clean":"2023-12-20T06:04:16.820352+0000","last_became_active":"2023-12-20T06:04:14.601666+0000","last_became_peered":"2023-12-20T06:04:14.601666+0000","last_unstale":"2023-12-20T06:04:16.820352+0000","last_undegraded":"2023-12-20T06:04:16.820352+0000","last_fullsized":"2023-12-20T06:04:16.820352+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T18:02:28.555973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.813024+0000","last_change":"2023-12-20T06:04:15.036938+0000","last_active":"2023-12-20T06:04:16.813024+0000","last_peered":"2023-12-20T06:04:16.813024+0000","last_clean":"2023-12-20T06:04:16.813024+0000","last_became_active":"2023-12-20T06:04:15.036597+0000","last_became_peered":"2023-12-20T06:04:15.036597+0000","last_unstale":"2023-12-20T06:04:16.813024+0000","last_undegraded":"2023-12-20T06:04:16.813024+0000","last_fullsized":"2023-12-20T06:04:16.813024+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:58:11.789777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010748+0000","last_change":"2023-12-20T06:04:14.610114+0000","last_active":"2023-12-20T06:04:16.010748+0000","last_peered":"2023-12-20T06:04:16.010748+0000","last_clean":"2023-12-20T06:04:16.010748+0000","last_became_active":"2023-12-20T06:04:14.609666+0000","last_became_peered":"2023-12-20T06:04:14.609666+0000","last_unstale":"2023-12-20T06:04:16.010748+0000","last_undegraded":"2023-12-20T06:04:16.010748+0000","last_fullsized":"2023-12-20T06:04:16.010748+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:03:04.243865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006917+0000","last_change":"2023-12-20T06:04:15.020228+0000","last_active":"2023-12-20T06:04:16.006917+0000","last_peered":"2023-12-20T06:04:16.006917+0000","last_clean":"2023-12-20T06:04:16.006917+0000","last_became_active":"2023-12-20T06:04:15.019979+0000","last_became_peered":"2023-12-20T06:04:15.019979+0000","last_unstale":"2023-12-20T06:04:16.006917+0000","last_undegraded":"2023-12-20T06:04:16.006917+0000","last_fullsized":"2023-12-20T06:04:16.006917+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:54:08.613432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.009567+0000","last_change":"2023-12-20T06:04:14.613233+0000","last_active":"2023-12-20T06:04:16.009567+0000","last_peered":"2023-12-20T06:04:16.009567+0000","last_clean":"2023-12-20T06:04:16.009567+0000","last_became_active":"2023-12-20T06:04:14.612954+0000","last_became_peered":"2023-12-20T06:04:14.612954+0000","last_unstale":"2023-12-20T06:04:16.009567+0000","last_undegraded":"2023-12-20T06:04:16.009567+0000","last_fullsized":"2023-12-20T06:04:16.009567+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:12:02.350675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822304+0000","last_change":"2023-12-20T06:04:15.021729+0000","last_active":"2023-12-20T06:04:16.822304+0000","last_peered":"2023-12-20T06:04:16.822304+0000","last_clean":"2023-12-20T06:04:16.822304+0000","last_became_active":"2023-12-20T06:04:15.020957+0000","last_became_peered":"2023-12-20T06:04:15.020957+0000","last_unstale":"2023-12-20T06:04:16.822304+0000","last_undegraded":"2023-12-20T06:04:16.822304+0000","last_fullsized":"2023-12-20T06:04:16.822304+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:45.469137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010796+0000","last_change":"2023-12-20T06:04:14.606002+0000","last_active":"2023-12-20T06:04:16.010796+0000","last_peered":"2023-12-20T06:04:16.010796+0000","last_clean":"2023-12-20T06:04:16.010796+0000","last_became_active":"2023-12-20T06:04:14.605694+0000","last_became_peered":"2023-12-20T06:04:14.605694+0000","last_unstale":"2023-12-20T06:04:16.010796+0000","last_undegraded":"2023-12-20T06:04:16.010796+0000","last_fullsized":"2023-12-20T06:04:16.010796+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:35:10.965321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.108023+0000","last_change":"2023-12-20T06:04:15.029526+0000","last_active":"2023-12-20T06:04:16.108023+0000","last_peered":"2023-12-20T06:04:16.108023+0000","last_clean":"2023-12-20T06:04:16.108023+0000","last_became_active":"2023-12-20T06:04:15.029266+0000","last_became_peered":"2023-12-20T06:04:15.029266+0000","last_unstale":"2023-12-20T06:04:16.108023+0000","last_undegraded":"2023-12-20T06:04:16.108023+0000","last_fullsized":"2023-12-20T06:04:16.108023+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:55:16.774802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107941+0000","last_change":"2023-12-20T06:04:14.605601+0000","last_active":"2023-12-20T06:04:16.107941+0000","last_peered":"2023-12-20T06:04:16.107941+0000","last_clean":"2023-12-20T06:04:16.107941+0000","last_became_active":"2023-12-20T06:04:14.605317+0000","last_became_peered":"2023-12-20T06:04:14.605317+0000","last_unstale":"2023-12-20T06:04:16.107941+0000","last_undegraded":"2023-12-20T06:04:16.107941+0000","last_fullsized":"2023-12-20T06:04:16.107941+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:07:48.209205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007238+0000","last_change":"2023-12-20T06:04:15.021464+0000","last_active":"2023-12-20T06:04:16.007238+0000","last_peered":"2023-12-20T06:04:16.007238+0000","last_clean":"2023-12-20T06:04:16.007238+0000","last_became_active":"2023-12-20T06:04:15.019928+0000","last_became_peered":"2023-12-20T06:04:15.019928+0000","last_unstale":"2023-12-20T06:04:16.007238+0000","last_undegraded":"2023-12-20T06:04:16.007238+0000","last_fullsized":"2023-12-20T06:04:16.007238+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:40:47.345135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821201+0000","last_change":"2023-12-20T06:04:14.607988+0000","last_active":"2023-12-20T06:04:16.821201+0000","last_peered":"2023-12-20T06:04:16.821201+0000","last_clean":"2023-12-20T06:04:16.821201+0000","last_became_active":"2023-12-20T06:04:14.605885+0000","last_became_peered":"2023-12-20T06:04:14.605885+0000","last_unstale":"2023-12-20T06:04:16.821201+0000","last_undegraded":"2023-12-20T06:04:16.821201+0000","last_fullsized":"2023-12-20T06:04:16.821201+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:12:02.350675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820603+0000","last_change":"2023-12-20T06:04:15.023993+0000","last_active":"2023-12-20T06:04:16.820603+0000","last_peered":"2023-12-20T06:04:16.820603+0000","last_clean":"2023-12-20T06:04:16.820603+0000","last_became_active":"2023-12-20T06:04:15.023708+0000","last_became_peered":"2023-12-20T06:04:15.023708+0000","last_unstale":"2023-12-20T06:04:16.820603+0000","last_undegraded":"2023-12-20T06:04:16.820603+0000","last_fullsized":"2023-12-20T06:04:16.820603+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:27:16.568576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820791+0000","last_change":"2023-12-20T06:04:14.605328+0000","last_active":"2023-12-20T06:04:16.820791+0000","last_peered":"2023-12-20T06:04:16.820791+0000","last_clean":"2023-12-20T06:04:16.820791+0000","last_became_active":"2023-12-20T06:04:14.604947+0000","last_became_peered":"2023-12-20T06:04:14.604947+0000","last_unstale":"2023-12-20T06:04:16.820791+0000","last_undegraded":"2023-12-20T06:04:16.820791+0000","last_fullsized":"2023-12-20T06:04:16.820791+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:06:42.173623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821089+0000","last_change":"2023-12-20T06:04:15.022561+0000","last_active":"2023-12-20T06:04:16.821089+0000","last_peered":"2023-12-20T06:04:16.821089+0000","last_clean":"2023-12-20T06:04:16.821089+0000","last_became_active":"2023-12-20T06:04:15.020667+0000","last_became_peered":"2023-12-20T06:04:15.020667+0000","last_unstale":"2023-12-20T06:04:16.821089+0000","last_undegraded":"2023-12-20T06:04:16.821089+0000","last_fullsized":"2023-12-20T06:04:16.821089+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:08.600616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007075+0000","last_change":"2023-12-20T06:04:14.610008+0000","last_active":"2023-12-20T06:04:16.007075+0000","last_peered":"2023-12-20T06:04:16.007075+0000","last_clean":"2023-12-20T06:04:16.007075+0000","last_became_active":"2023-12-20T06:04:14.609653+0000","last_became_peered":"2023-12-20T06:04:14.609653+0000","last_unstale":"2023-12-20T06:04:16.007075+0000","last_undegraded":"2023-12-20T06:04:16.007075+0000","last_fullsized":"2023-12-20T06:04:16.007075+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:25:44.191425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107751+0000","last_change":"2023-12-20T06:04:15.033904+0000","last_active":"2023-12-20T06:04:16.107751+0000","last_peered":"2023-12-20T06:04:16.107751+0000","last_clean":"2023-12-20T06:04:16.107751+0000","last_became_active":"2023-12-20T06:04:15.033497+0000","last_became_peered":"2023-12-20T06:04:15.033497+0000","last_unstale":"2023-12-20T06:04:16.107751+0000","last_undegraded":"2023-12-20T06:04:16.107751+0000","last_fullsized":"2023-12-20T06:04:16.107751+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:12:03.367985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822097+0000","last_change":"2023-12-20T06:04:14.611805+0000","last_active":"2023-12-20T06:04:16.822097+0000","last_peered":"2023-12-20T06:04:16.822097+0000","last_clean":"2023-12-20T06:04:16.822097+0000","last_became_active":"2023-12-20T06:04:14.611271+0000","last_became_peered":"2023-12-20T06:04:14.611271+0000","last_unstale":"2023-12-20T06:04:16.822097+0000","last_undegraded":"2023-12-20T06:04:16.822097+0000","last_fullsized":"2023-12-20T06:04:16.822097+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:00:10.387327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107939+0000","last_change":"2023-12-20T06:04:15.027636+0000","last_active":"2023-12-20T06:04:16.107939+0000","last_peered":"2023-12-20T06:04:16.107939+0000","last_clean":"2023-12-20T06:04:16.107939+0000","last_became_active":"2023-12-20T06:04:15.027377+0000","last_became_peered":"2023-12-20T06:04:15.027377+0000","last_unstale":"2023-12-20T06:04:16.107939+0000","last_undegraded":"2023-12-20T06:04:16.107939+0000","last_fullsized":"2023-12-20T06:04:16.107939+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:10:05.688232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822208+0000","last_change":"2023-12-20T06:04:14.606148+0000","last_active":"2023-12-20T06:04:16.822208+0000","last_peered":"2023-12-20T06:04:16.822208+0000","last_clean":"2023-12-20T06:04:16.822208+0000","last_became_active":"2023-12-20T06:04:14.605822+0000","last_became_peered":"2023-12-20T06:04:14.605822+0000","last_unstale":"2023-12-20T06:04:16.822208+0000","last_undegraded":"2023-12-20T06:04:16.822208+0000","last_fullsized":"2023-12-20T06:04:16.822208+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:38:33.531001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107922+0000","last_change":"2023-12-20T06:04:15.027128+0000","last_active":"2023-12-20T06:04:16.107922+0000","last_peered":"2023-12-20T06:04:16.107922+0000","last_clean":"2023-12-20T06:04:16.107922+0000","last_became_active":"2023-12-20T06:04:15.026859+0000","last_became_peered":"2023-12-20T06:04:15.026859+0000","last_unstale":"2023-12-20T06:04:16.107922+0000","last_undegraded":"2023-12-20T06:04:16.107922+0000","last_fullsized":"2023-12-20T06:04:16.107922+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:42:11.485396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107972+0000","last_change":"2023-12-20T06:04:14.607389+0000","last_active":"2023-12-20T06:04:16.107972+0000","last_peered":"2023-12-20T06:04:16.107972+0000","last_clean":"2023-12-20T06:04:16.107972+0000","last_became_active":"2023-12-20T06:04:14.606834+0000","last_became_peered":"2023-12-20T06:04:14.606834+0000","last_unstale":"2023-12-20T06:04:16.107972+0000","last_undegraded":"2023-12-20T06:04:16.107972+0000","last_fullsized":"2023-12-20T06:04:16.107972+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:09:38.660788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820072+0000","last_change":"2023-12-20T06:04:15.013346+0000","last_active":"2023-12-20T06:04:16.820072+0000","last_peered":"2023-12-20T06:04:16.820072+0000","last_clean":"2023-12-20T06:04:16.820072+0000","last_became_active":"2023-12-20T06:04:15.013036+0000","last_became_peered":"2023-12-20T06:04:15.013036+0000","last_unstale":"2023-12-20T06:04:16.820072+0000","last_undegraded":"2023-12-20T06:04:16.820072+0000","last_fullsized":"2023-12-20T06:04:16.820072+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:55.406087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007120+0000","last_change":"2023-12-20T06:04:14.609116+0000","last_active":"2023-12-20T06:04:16.007120+0000","last_peered":"2023-12-20T06:04:16.007120+0000","last_clean":"2023-12-20T06:04:16.007120+0000","last_became_active":"2023-12-20T06:04:14.608854+0000","last_became_peered":"2023-12-20T06:04:14.608854+0000","last_unstale":"2023-12-20T06:04:16.007120+0000","last_undegraded":"2023-12-20T06:04:16.007120+0000","last_fullsized":"2023-12-20T06:04:16.007120+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:32:52.049835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820272+0000","last_change":"2023-12-20T06:04:15.021469+0000","last_active":"2023-12-20T06:04:16.820272+0000","last_peered":"2023-12-20T06:04:16.820272+0000","last_clean":"2023-12-20T06:04:16.820272+0000","last_became_active":"2023-12-20T06:04:15.021184+0000","last_became_peered":"2023-12-20T06:04:15.021184+0000","last_unstale":"2023-12-20T06:04:16.820272+0000","last_undegraded":"2023-12-20T06:04:16.820272+0000","last_fullsized":"2023-12-20T06:04:16.820272+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:14:41.885358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"236'3234","reported_seq":4147,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-12-20T06:04:15.016278+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:04:15.016278+0000","last_peered":"2023-12-20T06:04:15.016278+0000","last_clean":"2023-12-20T06:04:15.016278+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:04:15.016278+0000","last_undegraded":"2023-12-20T06:04:15.016278+0000","last_fullsized":"2023-12-20T06:04:15.016278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3234,"log_dups_size":0,"ondisk_log_size":3234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2173,"num_read_kb":4467,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2173,"num_read_kb":4467,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3234,"ondisk_log_size":3234,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575990,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60964,"kb_used_data":3676,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94310876,"statfs":{"total":96636764160,"available":96574337024,"internally_reserved":0,"allocated":3764224,"data_stored":2145708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.69599999999999995,"15min":0.72799999999999998},"min":{"1min":0.45400000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":1.028,"15min":1.042},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.99099999999999999,"15min":1.161},"last":0.61499999999999999}]},{"osd":1,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71499999999999997,"15min":0.74199999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":0.996,"15min":1.1279999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.71599999999999997,"15min":0.745},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.85499999999999998,"5min":0.96699999999999997,"15min":1.032},"last":1.0389999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.70599999999999996,"15min":0.73899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84499999999999997,"5min":0.95399999999999996,"15min":1.014},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.49099999999999999},"max":{"1min":0.88700000000000001,"5min":1.0089999999999999,"15min":1.0189999999999999},"last":0.85799999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.47699999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.85299999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.89900000000000002,"5min":1.0660000000000001,"15min":1.0840000000000001},"last":0.98899999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.04},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.02,"15min":1.02},"last":0.83099999999999996}]},{"osd":5,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68300000000000005,"15min":0.66800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.052},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.875,"5min":1.0229999999999999,"15min":1.103},"last":0.80900000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76300000000000001,"15min":0.746},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72399999999999998,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93500000000000005,"5min":1.0740000000000001,"15min":1.081},"last":0.96099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575990,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52196,"kb_used_data":3100,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319644,"statfs":{"total":96636764160,"available":96583315456,"internally_reserved":0,"allocated":3174400,"data_stored":1555382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.68300000000000005,"15min":0.73299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.89100000000000001,"5min":1.016,"15min":1.0980000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.70499999999999996,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.94499999999999995,"5min":1.006,"15min":1.05},"last":0.82299999999999995}]},{"osd":1,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.72499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.72199999999999998,"15min":0.75800000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84999999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.66500000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.747,"15min":0.77100000000000002},"min":{"1min":0.52400000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96199999999999997,"5min":1.069,"15min":1.069},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71799999999999997,"15min":0.75800000000000001},"min":{"1min":0.53100000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.85999999999999999,"5min":0.97799999999999998,"15min":1.0760000000000001},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.78100000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46500000000000002},"max":{"1min":0.95199999999999996,"5min":1.077,"15min":1.077},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.72999999999999998,"15min":0.75600000000000001},"min":{"1min":0.49099999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92500000000000004,"5min":1.052,"15min":1.052},"last":0.65100000000000002}]},{"osd":4,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84399999999999997,"5min":1.05,"15min":1.0529999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67200000000000004,"15min":0.65500000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.86399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":0.876,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.46800000000000003,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":0.91700000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.502}]},{"osd":7,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.74399999999999999,"15min":0.73699999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98199999999999998,"5min":1.103,"15min":1.1240000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72199999999999998,"15min":0.72899999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.41999999999999998},"max":{"1min":0.92600000000000005,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.59799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52188,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319652,"statfs":{"total":96636764160,"available":96583323648,"internally_reserved":0,"allocated":3166208,"data_stored":1555270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71399999999999997,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91600000000000004,"5min":0.94799999999999995,"15min":1.077},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71499999999999997,"15min":0.746},"min":{"1min":0.45600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.80300000000000005,"5min":0.96699999999999997,"15min":1.028},"last":0.746}]},{"osd":1,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":0.875,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88500000000000001,"5min":0.98799999999999999,"15min":1.0209999999999999},"last":0.81000000000000005}]},{"osd":2,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.751},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89000000000000001,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.75700000000000001,"15min":0.77000000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.48299999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0680000000000001},"last":0.79000000000000004}]},{"osd":3,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"max":{"1min":0.91000000000000003,"5min":1.05,"15min":1.0669999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.0629999999999999},"last":0.89800000000000002}]},{"osd":4,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.56000000000000005,"5min":0.42199999999999999,"15min":0.38200000000000001},"max":{"1min":0.94399999999999995,"5min":1.0640000000000001,"15min":1.077},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.64800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"max":{"1min":1.0649999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.0449999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.55500000000000005}]},{"osd":7,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73799999999999999,"15min":0.70999999999999996},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.153,"15min":1.153},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.48299999999999998},"max":{"1min":0.98799999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.76000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52208,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319632,"statfs":{"total":96636764160,"available":96583303168,"internally_reserved":0,"allocated":3186688,"data_stored":1556890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67500000000000004,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.98999999999999999,"15min":1.0209999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.71799999999999997},"min":{"1min":0.46200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.92600000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.96699999999999997}]},{"osd":1,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83999999999999997,"5min":0.95699999999999996,"15min":1.0900000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47399999999999998,"15min":0.44500000000000001},"max":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67100000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72299999999999998,"15min":0.76200000000000001},"min":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.101},"last":0.77400000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70299999999999996,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":1.014,"15min":1.014},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97799999999999998,"5min":1.04,"15min":1.081},"last":0.755}]},{"osd":5,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":1.0589999999999999}]},{"osd":6,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.1379999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.749,"15min":0.745},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.41199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.82099999999999995}]},{"osd":7,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.0740000000000001,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":1.0349999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575991,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52204,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319636,"statfs":{"total":96636764160,"available":96583307264,"internally_reserved":0,"allocated":3182592,"data_stored":1557416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.46899999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.748,"15min":2.8130000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.49299999999999999,"15min":0.52600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":1.357},"last":0.29899999999999999}]},{"osd":1,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45700000000000002,"15min":0.47599999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65600000000000003,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.503},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53000000000000003,"15min":0.56699999999999995},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.57599999999999996,"5min":0.81799999999999995,"15min":1.4219999999999999},"last":0.45600000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.56399999999999995,"15min":0.58999999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.57999999999999996,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.438},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.53800000000000003,"15min":0.56999999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69099999999999995,"5min":0.86099999999999999,"15min":2.944},"last":0.39100000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77300000000000002,"15min":0.81499999999999995},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.98999999999999999,"5min":1.2090000000000001,"15min":1.669},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75900000000000001,"15min":0.80800000000000005},"min":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.86599999999999999,"5min":1.1299999999999999,"15min":1.99},"last":0.86599999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79600000000000004,"15min":0.82899999999999996},"min":{"1min":0.65100000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.93600000000000005,"5min":1.22,"15min":1.901},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.76300000000000001,"15min":0.81999999999999995},"min":{"1min":0.58799999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.875,"5min":1.196,"15min":2.8639999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.77500000000000002,"15min":0.81399999999999995},"min":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95599999999999996,"5min":1.194,"15min":1.575},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80500000000000005,"15min":0.84499999999999997},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0580000000000001,"5min":1.155,"15min":1.948},"last":0.72299999999999998}]},{"osd":7,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91900000000000004,"5min":1.1579999999999999,"15min":2.9049999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80700000000000005,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.502,"15min":0.502},"max":{"1min":1.0680000000000001,"5min":1.1850000000000001,"15min":1.754},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575990,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52188,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319652,"statfs":{"total":96636764160,"available":96583323648,"internally_reserved":0,"allocated":3166208,"data_stored":1555270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.45000000000000001,"15min":0.45500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52900000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.501},{"interface":"front","average":{"1min":0.438,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.60699999999999998,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.79800000000000004,"15min":0.82199999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30499999999999999},"max":{"1min":0.61199999999999999,"5min":0.92600000000000005,"15min":1.004},"last":0.57699999999999996}]},{"osd":3,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.65500000000000003,"15min":0.67400000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.59199999999999997,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.73999999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":1.254},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77900000000000003,"15min":0.79600000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":1.605,"15min":1.605},"last":0.80600000000000005}]},{"osd":5,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97099999999999997,"5min":1.329,"15min":1.329},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98199999999999998,"5min":1.0740000000000001,"15min":1.095},"last":0.88}]},{"osd":6,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92200000000000004,"5min":1.919,"15min":1.919},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.95099999999999996,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0049999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.02},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.88200000000000001,"15min":0.88500000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.996,"5min":1.292,"15min":1.292},"last":1.0009999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60396,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311444,"statfs":{"total":96636764160,"available":96574918656,"internally_reserved":0,"allocated":3182592,"data_stored":1556046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.504,"15min":0.51600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.42599999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.51700000000000002,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.57399999999999995},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.64500000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53300000000000003,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71399999999999997,"5min":0.78300000000000003,"15min":0.93500000000000005},"last":0.49199999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93100000000000005,"5min":1.107,"15min":1.321},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.81100000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.75700000000000001}]},{"osd":5,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.0649999999999999,"5min":1.224,"15min":1.2849999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77800000000000002,"15min":0.80400000000000005},"min":{"1min":0.55700000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.78400000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.80400000000000005,"15min":0.82699999999999996},"min":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.488,"5min":2.488,"15min":2.488},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.53200000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.84199999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.85799999999999998},"min":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.82499999999999996,"15min":0.85999999999999999},"min":{"1min":0.621,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.3839999999999999},"last":0.999}]}]},{"osd":0,"up_from":13,"seq":55834575991,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56864,"kb_used_data":3672,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94314976,"statfs":{"total":96636764160,"available":96578535424,"internally_reserved":0,"allocated":3760128,"data_stored":2145638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.29299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":0.998,"15min":0.998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.54300000000000004,"15min":0.55200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.63900000000000001,"5min":3.681,"15min":3.681},"last":0.52200000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60999999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.502},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.48099999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.67300000000000004,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.42699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66000000000000003,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80500000000000005,"15min":0.82799999999999996},"min":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"max":{"1min":1.081,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.61199999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.84499999999999997,"15min":0.84399999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.042,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81599999999999995,"15min":0.84299999999999997},"min":{"1min":0.53700000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.1220000000000001,"5min":2.23,"15min":2.23},"last":0.55900000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.83599999999999997},"min":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0529999999999999,"5min":1.482,"15min":1.482},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.72299999999999998}]},{"osd":7,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96099999999999997,"5min":5.7160000000000002,"15min":5.7160000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.86399999999999999,"15min":0.86599999999999999},"min":{"1min":0.56899999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94399999999999995,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.63}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:04:19.115 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:04:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.293+0000 7fc86c6f2640 1 Processor -- start 2023-12-20T06:04:19.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.293+0000 7fc86c6f2640 1 -- start start 2023-12-20T06:04:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.293+0000 7fc86c6f2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc8641599e0 0x7fc8640aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.293+0000 7fc86c6f2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8640c98c0 con 0x7fc8640b0d00 2023-12-20T06:04:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.293+0000 7fc86c6f2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc8640c9ba0 con 0x7fc8640c90e0 2023-12-20T06:04:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.293+0000 7fc86c6f2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc8640c9e80 con 0x7fc8641599e0 2023-12-20T06:04:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.293+0000 7fc86ac8f640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc8640b0d00 0x7fc8640701c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48942/0 (socket says 172.21.15.32:48942) 2023-12-20T06:04:19.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.293+0000 7fc86ac8f640 1 -- 172.21.15.32:0/2179132568 learned_addr learned my addr 172.21.15.32:0/2179132568 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:19.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.297+0000 7fc86a48e640 1 --2- 172.21.15.32:0/2179132568 >> v2:172.21.15.39:3300/0 conn(0x7fc8641599e0 0x7fc8640aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:19.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.297+0000 7fc86948c640 1 -- 172.21.15.32:0/2179132568 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3359488702 0 0) 0x7fc8640c98c0 con 0x7fc8640b0d00 2023-12-20T06:04:19.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.297+0000 7fc86948c640 1 -- 172.21.15.32:0/2179132568 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc850005740 con 0x7fc8640b0d00 2023-12-20T06:04:19.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.297+0000 7fc86a48e640 1 -- 172.21.15.32:0/2179132568 >> v1:172.21.15.39:6789/0 conn(0x7fc8640c90e0 legacy=0x7fc8640b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.297+0000 7fc86a48e640 1 -- 172.21.15.32:0/2179132568 >> v1:172.21.15.32:6789/0 conn(0x7fc8640b0d00 legacy=0x7fc8640701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.297+0000 7fc86948c640 1 -- 172.21.15.32:0/2179132568 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3339048983 0 0) 0x7fc850005740 con 0x7fc8640b0d00 2023-12-20T06:04:19.299 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.297+0000 7fc86a48e640 1 -- 172.21.15.32:0/2179132568 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8640ca160 con 0x7fc8641599e0 2023-12-20T06:04:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.297+0000 7fc86a48e640 1 --2- 172.21.15.32:0/2179132568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8641599e0 0x7fc8640aece0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc8600097a0 tx=0x7fc86001c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=7e9e52591de9647f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.297+0000 7fc86948c640 1 -- 172.21.15.32:0/2179132568 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c001a00 con 0x7fc8641599e0 2023-12-20T06:04:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86948c640 1 -- 172.21.15.32:0/2179132568 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc8600a7050 con 0x7fc8641599e0 2023-12-20T06:04:19.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86948c640 1 -- 172.21.15.32:0/2179132568 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc8600ab600 con 0x7fc8641599e0 2023-12-20T06:04:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 -- 172.21.15.32:0/2179132568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8641599e0 msgr2=0x7fc8640aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 --2- 172.21.15.32:0/2179132568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8641599e0 0x7fc8640aece0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7fc8600097a0 tx=0x7fc86001c350 comp rx=0 tx=0).stop 2023-12-20T06:04:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 -- 172.21.15.32:0/2179132568 shutdown_connections 2023-12-20T06:04:19.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 --2- 172.21.15.32:0/2179132568 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8641599e0 0x7fc8640aece0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 -- 172.21.15.32:0/2179132568 >> 172.21.15.32:0/2179132568 conn(0x7fc86406a320 msgr2=0x7fc8640ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 -- 172.21.15.32:0/2179132568 shutdown_connections 2023-12-20T06:04:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 -- 172.21.15.32:0/2179132568 wait complete. 2023-12-20T06:04:19.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 Processor -- start 2023-12-20T06:04:19.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 -- start start 2023-12-20T06:04:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8640b0d00 0x7fc864146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:19.306 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8640c90e0 0x7fc8640c8090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8641599e0 0x7fc86415c270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640c7600 con 0x7fc8641599e0 2023-12-20T06:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc864147140 con 0x7fc8640b0d00 2023-12-20T06:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86c6f2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc8640ce440 con 0x7fc8640c90e0 2023-12-20T06:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86ac8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8641599e0 0x7fc86415c270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86ac8f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8641599e0 0x7fc86415c270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36910/0 (socket says 172.21.15.32:36910) 2023-12-20T06:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86ac8f640 1 -- 172.21.15.32:0/1370351216 learned_addr learned my addr 172.21.15.32:0/1370351216 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc86a48e640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8640b0d00 0x7fc864146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.301+0000 7fc869c8d640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8640c90e0 0x7fc8640c8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:19.307 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc86ac8f640 1 -- 172.21.15.32:0/1370351216 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8640c90e0 msgr2=0x7fc8640c8090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc86ac8f640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8640c90e0 0x7fc8640c8090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc86ac8f640 1 -- 172.21.15.32:0/1370351216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8640b0d00 msgr2=0x7fc864146c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc86ac8f640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8640b0d00 0x7fc864146c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc86ac8f640 1 -- 172.21.15.32:0/1370351216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc8600093b0 con 0x7fc8641599e0 2023-12-20T06:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc86ac8f640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8641599e0 0x7fc86415c270 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00cc30 tx=0x7fc85c00a430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc85affd640 1 -- 172.21.15.32:0/1370351216 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c00ce40 con 0x7fc8641599e0 2023-12-20T06:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc85affd640 1 -- 172.21.15.32:0/1370351216 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc85c012400 con 0x7fc8641599e0 2023-12-20T06:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc85affd640 1 -- 172.21.15.32:0/1370351216 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc85c0127b0 con 0x7fc8641599e0 2023-12-20T06:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc86c6f2640 1 -- 172.21.15.32:0/1370351216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc8640ce6e0 con 0x7fc8641599e0 2023-12-20T06:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc86c6f2640 1 -- 172.21.15.32:0/1370351216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc8640c7290 con 0x7fc8641599e0 2023-12-20T06:04:19.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc86c6f2640 1 -- 172.21.15.32:0/1370351216 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc8640c70f0 con 0x7fc8641599e0 2023-12-20T06:04:19.309 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc85affd640 1 -- 172.21.15.32:0/1370351216 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fc85c012910 con 0x7fc8641599e0 2023-12-20T06:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc85affd640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc848065ca0 0x7fc848068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc85affd640 1 -- 172.21.15.32:0/1370351216 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fc85c083c70 con 0x7fc8641599e0 2023-12-20T06:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.305+0000 7fc86a48e640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc848065ca0 0x7fc848068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:19.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.309+0000 7fc86a48e640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc848065ca0 0x7fc848068160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8600ad000 tx=0x7fc8600a6000 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:19.312 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.309+0000 7fc85affd640 1 -- 172.21.15.32:0/1370351216 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc8640c70f0 con 0x7fc8641599e0 2023-12-20T06:04:19.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.465+0000 7fc86c6f2640 1 -- 172.21.15.32:0/1370351216 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7fc864159dc0 con 0x7fc848065ca0 2023-12-20T06:04:19.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.473+0000 7fc85affd640 1 -- 172.21.15.32:0/1370351216 <== mgr.11715 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+289435 (secure 0 0 0) 0x7fc8600ba070 con 0x7fc848065ca0 2023-12-20T06:04:19.477 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:19.479 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:04:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 -- 172.21.15.32:0/1370351216 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc848065ca0 msgr2=0x7fc848068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc848065ca0 0x7fc848068160 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fc8600ad000 tx=0x7fc8600a6000 comp rx=0 tx=0).stop 2023-12-20T06:04:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 -- 172.21.15.32:0/1370351216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8641599e0 msgr2=0x7fc86415c270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8641599e0 0x7fc86415c270 secure :-1 s=READY pgs=2488 cs=0 l=1 rev1=1 crypto rx=0x7fc85c00cc30 tx=0x7fc85c00a430 comp rx=0 tx=0).stop 2023-12-20T06:04:19.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 -- 172.21.15.32:0/1370351216 shutdown_connections 2023-12-20T06:04:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc848065ca0 0x7fc848068160 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc8641599e0 0x7fc86415c270 unknown :-1 s=CLOSED pgs=2488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc8640c90e0 0x7fc8640c8090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 --2- 172.21.15.32:0/1370351216 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc8640b0d00 0x7fc864146c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 -- 172.21.15.32:0/1370351216 >> 172.21.15.32:0/1370351216 conn(0x7fc86406a320 msgr2=0x7fc864154da0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 -- 172.21.15.32:0/1370351216 shutdown_connections 2023-12-20T06:04:19.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.477+0000 7fc86c6f2640 1 -- 172.21.15.32:0/1370351216 wait complete. 2023-12-20T06:04:19.496 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":31,"stamp":"2023-12-20T06:04:17.806793+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":592730,"num_objects":23,"num_object_clones":0,"num_object_copies":46,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":23,"num_whiteouts":0,"num_read":2173,"num_read_kb":4467,"num_write":625,"num_write_kb":2430,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3272,"ondisk_log_size":3272,"up":194,"acting":194,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":146,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":8,"kb":754974720,"kb_used":439208,"kb_used_data":25960,"kb_used_omap":0,"kb_used_meta":413184,"kb_avail":754535512,"statfs":{"total":773094113280,"available":772644364288,"internally_reserved":0,"allocated":26583040,"data_stored":13627620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":423100416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":31,"apply_latency_ms":31,"commit_latency_ns":31000000,"apply_latency_ns":31000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"4.225334"},"pg_stats":[{"pgid":"33.3f","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.011118+0000","last_change":"2023-12-20T06:04:15.026954+0000","last_active":"2023-12-20T06:04:16.011118+0000","last_peered":"2023-12-20T06:04:16.011118+0000","last_clean":"2023-12-20T06:04:16.011118+0000","last_became_active":"2023-12-20T06:04:15.026638+0000","last_became_peered":"2023-12-20T06:04:15.026638+0000","last_unstale":"2023-12-20T06:04:16.011118+0000","last_undegraded":"2023-12-20T06:04:16.011118+0000","last_fullsized":"2023-12-20T06:04:16.011118+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:27:23.439943+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.3e","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.009633+0000","last_change":"2023-12-20T06:04:15.034718+0000","last_active":"2023-12-20T06:04:16.009633+0000","last_peered":"2023-12-20T06:04:16.009633+0000","last_clean":"2023-12-20T06:04:16.009633+0000","last_became_active":"2023-12-20T06:04:15.034456+0000","last_became_peered":"2023-12-20T06:04:15.034456+0000","last_unstale":"2023-12-20T06:04:16.009633+0000","last_undegraded":"2023-12-20T06:04:16.009633+0000","last_fullsized":"2023-12-20T06:04:16.009633+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:19:05.245786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6],"acting":[7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.3d","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820981+0000","last_change":"2023-12-20T06:04:15.023349+0000","last_active":"2023-12-20T06:04:16.820981+0000","last_peered":"2023-12-20T06:04:16.820981+0000","last_clean":"2023-12-20T06:04:16.820981+0000","last_became_active":"2023-12-20T06:04:15.020032+0000","last_became_peered":"2023-12-20T06:04:15.020032+0000","last_unstale":"2023-12-20T06:04:16.820981+0000","last_undegraded":"2023-12-20T06:04:16.820981+0000","last_fullsized":"2023-12-20T06:04:16.820981+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:37:04.055455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3c","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820500+0000","last_change":"2023-12-20T06:04:15.022832+0000","last_active":"2023-12-20T06:04:16.820500+0000","last_peered":"2023-12-20T06:04:16.820500+0000","last_clean":"2023-12-20T06:04:16.820500+0000","last_became_active":"2023-12-20T06:04:15.022565+0000","last_became_peered":"2023-12-20T06:04:15.022565+0000","last_unstale":"2023-12-20T06:04:16.820500+0000","last_undegraded":"2023-12-20T06:04:16.820500+0000","last_fullsized":"2023-12-20T06:04:16.820500+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:54:27.659337+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3b","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820186+0000","last_change":"2023-12-20T06:04:15.028145+0000","last_active":"2023-12-20T06:04:16.820186+0000","last_peered":"2023-12-20T06:04:16.820186+0000","last_clean":"2023-12-20T06:04:16.820186+0000","last_became_active":"2023-12-20T06:04:15.027770+0000","last_became_peered":"2023-12-20T06:04:15.027770+0000","last_unstale":"2023-12-20T06:04:16.820186+0000","last_undegraded":"2023-12-20T06:04:16.820186+0000","last_fullsized":"2023-12-20T06:04:16.820186+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:12:20.835387+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.3a","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812567+0000","last_change":"2023-12-20T06:04:15.017083+0000","last_active":"2023-12-20T06:04:16.812567+0000","last_peered":"2023-12-20T06:04:16.812567+0000","last_clean":"2023-12-20T06:04:16.812567+0000","last_became_active":"2023-12-20T06:04:15.016828+0000","last_became_peered":"2023-12-20T06:04:15.016828+0000","last_unstale":"2023-12-20T06:04:16.812567+0000","last_undegraded":"2023-12-20T06:04:16.812567+0000","last_fullsized":"2023-12-20T06:04:16.812567+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:40:26.547776+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.39","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820196+0000","last_change":"2023-12-20T06:04:15.033099+0000","last_active":"2023-12-20T06:04:16.820196+0000","last_peered":"2023-12-20T06:04:16.820196+0000","last_clean":"2023-12-20T06:04:16.820196+0000","last_became_active":"2023-12-20T06:04:15.032609+0000","last_became_peered":"2023-12-20T06:04:15.032609+0000","last_unstale":"2023-12-20T06:04:16.820196+0000","last_undegraded":"2023-12-20T06:04:16.820196+0000","last_fullsized":"2023-12-20T06:04:16.820196+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:08:46.920960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.38","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010662+0000","last_change":"2023-12-20T06:04:15.023347+0000","last_active":"2023-12-20T06:04:16.010662+0000","last_peered":"2023-12-20T06:04:16.010662+0000","last_clean":"2023-12-20T06:04:16.010662+0000","last_became_active":"2023-12-20T06:04:15.023043+0000","last_became_peered":"2023-12-20T06:04:15.023043+0000","last_unstale":"2023-12-20T06:04:16.010662+0000","last_undegraded":"2023-12-20T06:04:16.010662+0000","last_fullsized":"2023-12-20T06:04:16.010662+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:31:54.461212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.37","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822446+0000","last_change":"2023-12-20T06:04:15.016762+0000","last_active":"2023-12-20T06:04:16.822446+0000","last_peered":"2023-12-20T06:04:16.822446+0000","last_clean":"2023-12-20T06:04:16.822446+0000","last_became_active":"2023-12-20T06:04:15.016504+0000","last_became_peered":"2023-12-20T06:04:15.016504+0000","last_unstale":"2023-12-20T06:04:16.822446+0000","last_undegraded":"2023-12-20T06:04:16.822446+0000","last_fullsized":"2023-12-20T06:04:16.822446+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:19:15.528169+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.36","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820950+0000","last_change":"2023-12-20T06:04:15.030301+0000","last_active":"2023-12-20T06:04:16.820950+0000","last_peered":"2023-12-20T06:04:16.820950+0000","last_clean":"2023-12-20T06:04:16.820950+0000","last_became_active":"2023-12-20T06:04:15.029992+0000","last_became_peered":"2023-12-20T06:04:15.029992+0000","last_unstale":"2023-12-20T06:04:16.820950+0000","last_undegraded":"2023-12-20T06:04:16.820950+0000","last_fullsized":"2023-12-20T06:04:16.820950+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:59:33.073877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.35","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820664+0000","last_change":"2023-12-20T06:04:15.024722+0000","last_active":"2023-12-20T06:04:16.820664+0000","last_peered":"2023-12-20T06:04:16.820664+0000","last_clean":"2023-12-20T06:04:16.820664+0000","last_became_active":"2023-12-20T06:04:15.024083+0000","last_became_peered":"2023-12-20T06:04:15.024083+0000","last_unstale":"2023-12-20T06:04:16.820664+0000","last_undegraded":"2023-12-20T06:04:16.820664+0000","last_fullsized":"2023-12-20T06:04:16.820664+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:59:56.344194+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.34","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010886+0000","last_change":"2023-12-20T06:04:15.016325+0000","last_active":"2023-12-20T06:04:16.010886+0000","last_peered":"2023-12-20T06:04:16.010886+0000","last_clean":"2023-12-20T06:04:16.010886+0000","last_became_active":"2023-12-20T06:04:15.013781+0000","last_became_peered":"2023-12-20T06:04:15.013781+0000","last_unstale":"2023-12-20T06:04:16.010886+0000","last_undegraded":"2023-12-20T06:04:16.010886+0000","last_fullsized":"2023-12-20T06:04:16.010886+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:17:36.518319+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2],"acting":[4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.33","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107603+0000","last_change":"2023-12-20T06:04:15.025021+0000","last_active":"2023-12-20T06:04:16.107603+0000","last_peered":"2023-12-20T06:04:16.107603+0000","last_clean":"2023-12-20T06:04:16.107603+0000","last_became_active":"2023-12-20T06:04:15.024762+0000","last_became_peered":"2023-12-20T06:04:15.024762+0000","last_unstale":"2023-12-20T06:04:16.107603+0000","last_undegraded":"2023-12-20T06:04:16.107603+0000","last_fullsized":"2023-12-20T06:04:16.107603+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:33:23.963598+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.32","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107604+0000","last_change":"2023-12-20T06:04:15.018183+0000","last_active":"2023-12-20T06:04:16.107604+0000","last_peered":"2023-12-20T06:04:16.107604+0000","last_clean":"2023-12-20T06:04:16.107604+0000","last_became_active":"2023-12-20T06:04:15.017874+0000","last_became_peered":"2023-12-20T06:04:15.017874+0000","last_unstale":"2023-12-20T06:04:16.107604+0000","last_undegraded":"2023-12-20T06:04:16.107604+0000","last_fullsized":"2023-12-20T06:04:16.107604+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:19:05.245786+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.31","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107802+0000","last_change":"2023-12-20T06:04:15.028339+0000","last_active":"2023-12-20T06:04:16.107802+0000","last_peered":"2023-12-20T06:04:16.107802+0000","last_clean":"2023-12-20T06:04:16.107802+0000","last_became_active":"2023-12-20T06:04:15.028087+0000","last_became_peered":"2023-12-20T06:04:15.028087+0000","last_unstale":"2023-12-20T06:04:16.107802+0000","last_undegraded":"2023-12-20T06:04:16.107802+0000","last_fullsized":"2023-12-20T06:04:16.107802+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:55:05.066303+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.30","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812552+0000","last_change":"2023-12-20T06:04:15.024698+0000","last_active":"2023-12-20T06:04:16.812552+0000","last_peered":"2023-12-20T06:04:16.812552+0000","last_clean":"2023-12-20T06:04:16.812552+0000","last_became_active":"2023-12-20T06:04:15.024045+0000","last_became_peered":"2023-12-20T06:04:15.024045+0000","last_unstale":"2023-12-20T06:04:16.812552+0000","last_undegraded":"2023-12-20T06:04:16.812552+0000","last_fullsized":"2023-12-20T06:04:16.812552+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:39:53.409301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2f","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007078+0000","last_change":"2023-12-20T06:04:15.034428+0000","last_active":"2023-12-20T06:04:16.007078+0000","last_peered":"2023-12-20T06:04:16.007078+0000","last_clean":"2023-12-20T06:04:16.007078+0000","last_became_active":"2023-12-20T06:04:15.034154+0000","last_became_peered":"2023-12-20T06:04:15.034154+0000","last_unstale":"2023-12-20T06:04:16.007078+0000","last_undegraded":"2023-12-20T06:04:16.007078+0000","last_fullsized":"2023-12-20T06:04:16.007078+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:57:11.599590+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2e","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821165+0000","last_change":"2023-12-20T06:04:15.032943+0000","last_active":"2023-12-20T06:04:16.821165+0000","last_peered":"2023-12-20T06:04:16.821165+0000","last_clean":"2023-12-20T06:04:16.821165+0000","last_became_active":"2023-12-20T06:04:15.031957+0000","last_became_peered":"2023-12-20T06:04:15.031957+0000","last_unstale":"2023-12-20T06:04:16.821165+0000","last_undegraded":"2023-12-20T06:04:16.821165+0000","last_fullsized":"2023-12-20T06:04:16.821165+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:39:53.409301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2d","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820691+0000","last_change":"2023-12-20T06:04:15.030149+0000","last_active":"2023-12-20T06:04:16.820691+0000","last_peered":"2023-12-20T06:04:16.820691+0000","last_clean":"2023-12-20T06:04:16.820691+0000","last_became_active":"2023-12-20T06:04:15.029866+0000","last_became_peered":"2023-12-20T06:04:15.029866+0000","last_unstale":"2023-12-20T06:04:16.820691+0000","last_undegraded":"2023-12-20T06:04:16.820691+0000","last_fullsized":"2023-12-20T06:04:16.820691+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:20:34.577593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.2c","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812754+0000","last_change":"2023-12-20T06:04:15.012374+0000","last_active":"2023-12-20T06:04:16.812754+0000","last_peered":"2023-12-20T06:04:16.812754+0000","last_clean":"2023-12-20T06:04:16.812754+0000","last_became_active":"2023-12-20T06:04:15.012052+0000","last_became_peered":"2023-12-20T06:04:15.012052+0000","last_unstale":"2023-12-20T06:04:16.812754+0000","last_undegraded":"2023-12-20T06:04:16.812754+0000","last_fullsized":"2023-12-20T06:04:16.812754+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:08:46.920960+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.2b","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822101+0000","last_change":"2023-12-20T06:04:15.026651+0000","last_active":"2023-12-20T06:04:16.822101+0000","last_peered":"2023-12-20T06:04:16.822101+0000","last_clean":"2023-12-20T06:04:16.822101+0000","last_became_active":"2023-12-20T06:04:15.026357+0000","last_became_peered":"2023-12-20T06:04:15.026357+0000","last_unstale":"2023-12-20T06:04:16.822101+0000","last_undegraded":"2023-12-20T06:04:16.822101+0000","last_fullsized":"2023-12-20T06:04:16.822101+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:22:34.434439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.2a","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822199+0000","last_change":"2023-12-20T06:04:15.017311+0000","last_active":"2023-12-20T06:04:16.822199+0000","last_peered":"2023-12-20T06:04:16.822199+0000","last_clean":"2023-12-20T06:04:16.822199+0000","last_became_active":"2023-12-20T06:04:15.016074+0000","last_became_peered":"2023-12-20T06:04:15.016074+0000","last_unstale":"2023-12-20T06:04:16.822199+0000","last_undegraded":"2023-12-20T06:04:16.822199+0000","last_fullsized":"2023-12-20T06:04:16.822199+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:32:16.096870+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.29","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820324+0000","last_change":"2023-12-20T06:04:15.028424+0000","last_active":"2023-12-20T06:04:16.820324+0000","last_peered":"2023-12-20T06:04:16.820324+0000","last_clean":"2023-12-20T06:04:16.820324+0000","last_became_active":"2023-12-20T06:04:15.028118+0000","last_became_peered":"2023-12-20T06:04:15.028118+0000","last_unstale":"2023-12-20T06:04:16.820324+0000","last_undegraded":"2023-12-20T06:04:16.820324+0000","last_fullsized":"2023-12-20T06:04:16.820324+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:40:09.280259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0],"acting":[6,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.28","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007195+0000","last_change":"2023-12-20T06:04:15.020161+0000","last_active":"2023-12-20T06:04:16.007195+0000","last_peered":"2023-12-20T06:04:16.007195+0000","last_clean":"2023-12-20T06:04:16.007195+0000","last_became_active":"2023-12-20T06:04:15.019350+0000","last_became_peered":"2023-12-20T06:04:15.019350+0000","last_unstale":"2023-12-20T06:04:16.007195+0000","last_undegraded":"2023-12-20T06:04:16.007195+0000","last_fullsized":"2023-12-20T06:04:16.007195+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:05:08.516815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.27","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107839+0000","last_change":"2023-12-20T06:04:15.029858+0000","last_active":"2023-12-20T06:04:16.107839+0000","last_peered":"2023-12-20T06:04:16.107839+0000","last_clean":"2023-12-20T06:04:16.107839+0000","last_became_active":"2023-12-20T06:04:15.029628+0000","last_became_peered":"2023-12-20T06:04:15.029628+0000","last_unstale":"2023-12-20T06:04:16.107839+0000","last_undegraded":"2023-12-20T06:04:16.107839+0000","last_fullsized":"2023-12-20T06:04:16.107839+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:33:55.595521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.26","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821163+0000","last_change":"2023-12-20T06:04:15.030617+0000","last_active":"2023-12-20T06:04:16.821163+0000","last_peered":"2023-12-20T06:04:16.821163+0000","last_clean":"2023-12-20T06:04:16.821163+0000","last_became_active":"2023-12-20T06:04:15.030356+0000","last_became_peered":"2023-12-20T06:04:15.030356+0000","last_unstale":"2023-12-20T06:04:16.821163+0000","last_undegraded":"2023-12-20T06:04:16.821163+0000","last_fullsized":"2023-12-20T06:04:16.821163+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:53:45.234376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,6],"acting":[5,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.25","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820816+0000","last_change":"2023-12-20T06:04:15.028795+0000","last_active":"2023-12-20T06:04:16.820816+0000","last_peered":"2023-12-20T06:04:16.820816+0000","last_clean":"2023-12-20T06:04:16.820816+0000","last_became_active":"2023-12-20T06:04:15.028501+0000","last_became_peered":"2023-12-20T06:04:15.028501+0000","last_unstale":"2023-12-20T06:04:16.820816+0000","last_undegraded":"2023-12-20T06:04:16.820816+0000","last_fullsized":"2023-12-20T06:04:16.820816+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:41:29.330308+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.24","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107795+0000","last_change":"2023-12-20T06:04:15.031751+0000","last_active":"2023-12-20T06:04:16.107795+0000","last_peered":"2023-12-20T06:04:16.107795+0000","last_clean":"2023-12-20T06:04:16.107795+0000","last_became_active":"2023-12-20T06:04:15.030983+0000","last_became_peered":"2023-12-20T06:04:15.030983+0000","last_unstale":"2023-12-20T06:04:16.107795+0000","last_undegraded":"2023-12-20T06:04:16.107795+0000","last_fullsized":"2023-12-20T06:04:16.107795+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:08:03.478691+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.23","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007128+0000","last_change":"2023-12-20T06:04:15.035283+0000","last_active":"2023-12-20T06:04:16.007128+0000","last_peered":"2023-12-20T06:04:16.007128+0000","last_clean":"2023-12-20T06:04:16.007128+0000","last_became_active":"2023-12-20T06:04:15.034958+0000","last_became_peered":"2023-12-20T06:04:15.034958+0000","last_unstale":"2023-12-20T06:04:16.007128+0000","last_undegraded":"2023-12-20T06:04:16.007128+0000","last_fullsized":"2023-12-20T06:04:16.007128+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:03.705924+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.22","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107625+0000","last_change":"2023-12-20T06:04:15.018149+0000","last_active":"2023-12-20T06:04:16.107625+0000","last_peered":"2023-12-20T06:04:16.107625+0000","last_clean":"2023-12-20T06:04:16.107625+0000","last_became_active":"2023-12-20T06:04:15.017679+0000","last_became_peered":"2023-12-20T06:04:15.017679+0000","last_unstale":"2023-12-20T06:04:16.107625+0000","last_undegraded":"2023-12-20T06:04:16.107625+0000","last_fullsized":"2023-12-20T06:04:16.107625+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:07:52.510739+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.21","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010661+0000","last_change":"2023-12-20T06:04:15.024986+0000","last_active":"2023-12-20T06:04:16.010661+0000","last_peered":"2023-12-20T06:04:16.010661+0000","last_clean":"2023-12-20T06:04:16.010661+0000","last_became_active":"2023-12-20T06:04:15.024700+0000","last_became_peered":"2023-12-20T06:04:15.024700+0000","last_unstale":"2023-12-20T06:04:16.010661+0000","last_undegraded":"2023-12-20T06:04:16.010661+0000","last_fullsized":"2023-12-20T06:04:16.010661+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:04:21.452369+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.1e","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820966+0000","last_change":"2023-12-20T06:04:15.029574+0000","last_active":"2023-12-20T06:04:16.820966+0000","last_peered":"2023-12-20T06:04:16.820966+0000","last_clean":"2023-12-20T06:04:16.820966+0000","last_became_active":"2023-12-20T06:04:15.029161+0000","last_became_peered":"2023-12-20T06:04:15.029161+0000","last_unstale":"2023-12-20T06:04:16.820966+0000","last_undegraded":"2023-12-20T06:04:16.820966+0000","last_fullsized":"2023-12-20T06:04:16.820966+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:15:51.205184+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1f","version":"250'4","reported_seq":15,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007268+0000","last_change":"2023-12-20T06:04:14.616151+0000","last_active":"2023-12-20T06:04:16.007268+0000","last_peered":"2023-12-20T06:04:16.007268+0000","last_clean":"2023-12-20T06:04:16.007268+0000","last_became_active":"2023-12-20T06:04:14.615873+0000","last_became_peered":"2023-12-20T06:04:14.615873+0000","last_unstale":"2023-12-20T06:04:16.007268+0000","last_undegraded":"2023-12-20T06:04:16.007268+0000","last_fullsized":"2023-12-20T06:04:16.007268+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:06:23.720731+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.1f","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107845+0000","last_change":"2023-12-20T06:04:15.027605+0000","last_active":"2023-12-20T06:04:16.107845+0000","last_peered":"2023-12-20T06:04:16.107845+0000","last_clean":"2023-12-20T06:04:16.107845+0000","last_became_active":"2023-12-20T06:04:15.027355+0000","last_became_peered":"2023-12-20T06:04:15.027355+0000","last_unstale":"2023-12-20T06:04:16.107845+0000","last_undegraded":"2023-12-20T06:04:16.107845+0000","last_fullsized":"2023-12-20T06:04:16.107845+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T18:02:29.573283+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1e","version":"250'1","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107979+0000","last_change":"2023-12-20T06:04:14.606296+0000","last_active":"2023-12-20T06:04:16.107979+0000","last_peered":"2023-12-20T06:04:16.107979+0000","last_clean":"2023-12-20T06:04:16.107979+0000","last_became_active":"2023-12-20T06:04:14.606034+0000","last_became_peered":"2023-12-20T06:04:14.606034+0000","last_unstale":"2023-12-20T06:04:16.107979+0000","last_undegraded":"2023-12-20T06:04:16.107979+0000","last_fullsized":"2023-12-20T06:04:16.107979+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:08:50.240928+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1c","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821098+0000","last_change":"2023-12-20T06:04:15.020844+0000","last_active":"2023-12-20T06:04:16.821098+0000","last_peered":"2023-12-20T06:04:16.821098+0000","last_clean":"2023-12-20T06:04:16.821098+0000","last_became_active":"2023-12-20T06:04:15.020327+0000","last_became_peered":"2023-12-20T06:04:15.020327+0000","last_unstale":"2023-12-20T06:04:16.821098+0000","last_undegraded":"2023-12-20T06:04:16.821098+0000","last_fullsized":"2023-12-20T06:04:16.821098+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:25:09.673494+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,1],"acting":[6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.1d","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.813052+0000","last_change":"2023-12-20T06:04:14.609615+0000","last_active":"2023-12-20T06:04:16.813052+0000","last_peered":"2023-12-20T06:04:16.813052+0000","last_clean":"2023-12-20T06:04:16.813052+0000","last_became_active":"2023-12-20T06:04:14.606481+0000","last_became_peered":"2023-12-20T06:04:14.606481+0000","last_unstale":"2023-12-20T06:04:16.813052+0000","last_undegraded":"2023-12-20T06:04:16.813052+0000","last_fullsized":"2023-12-20T06:04:16.813052+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:04:20.435058+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1d","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010988+0000","last_change":"2023-12-20T06:04:15.026414+0000","last_active":"2023-12-20T06:04:16.010988+0000","last_peered":"2023-12-20T06:04:16.010988+0000","last_clean":"2023-12-20T06:04:16.010988+0000","last_became_active":"2023-12-20T06:04:15.025955+0000","last_became_peered":"2023-12-20T06:04:15.025955+0000","last_unstale":"2023-12-20T06:04:16.010988+0000","last_undegraded":"2023-12-20T06:04:16.010988+0000","last_fullsized":"2023-12-20T06:04:16.010988+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:28:58.470204+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,0],"acting":[4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"32.1c","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812809+0000","last_change":"2023-12-20T06:04:14.612409+0000","last_active":"2023-12-20T06:04:16.812809+0000","last_peered":"2023-12-20T06:04:16.812809+0000","last_clean":"2023-12-20T06:04:16.812809+0000","last_became_active":"2023-12-20T06:04:14.612148+0000","last_became_peered":"2023-12-20T06:04:14.612148+0000","last_unstale":"2023-12-20T06:04:16.812809+0000","last_undegraded":"2023-12-20T06:04:16.812809+0000","last_fullsized":"2023-12-20T06:04:16.812809+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:24:39.898860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,4],"acting":[1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.1a","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822035+0000","last_change":"2023-12-20T06:04:15.019053+0000","last_active":"2023-12-20T06:04:16.822035+0000","last_peered":"2023-12-20T06:04:16.822035+0000","last_clean":"2023-12-20T06:04:16.822035+0000","last_became_active":"2023-12-20T06:04:15.018794+0000","last_became_peered":"2023-12-20T06:04:15.018794+0000","last_unstale":"2023-12-20T06:04:16.822035+0000","last_undegraded":"2023-12-20T06:04:16.822035+0000","last_fullsized":"2023-12-20T06:04:16.822035+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:30:46.585415+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.1b","version":"250'1","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820606+0000","last_change":"2023-12-20T06:04:14.606279+0000","last_active":"2023-12-20T06:04:16.820606+0000","last_peered":"2023-12-20T06:04:16.820606+0000","last_clean":"2023-12-20T06:04:16.820606+0000","last_became_active":"2023-12-20T06:04:14.605930+0000","last_became_peered":"2023-12-20T06:04:14.605930+0000","last_unstale":"2023-12-20T06:04:16.820606+0000","last_undegraded":"2023-12-20T06:04:16.820606+0000","last_fullsized":"2023-12-20T06:04:16.820606+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:24:39.898860+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4],"acting":[6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.1b","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812896+0000","last_change":"2023-12-20T06:04:15.029990+0000","last_active":"2023-12-20T06:04:16.812896+0000","last_peered":"2023-12-20T06:04:16.812896+0000","last_clean":"2023-12-20T06:04:16.812896+0000","last_became_active":"2023-12-20T06:04:15.029734+0000","last_became_peered":"2023-12-20T06:04:15.029734+0000","last_unstale":"2023-12-20T06:04:16.812896+0000","last_undegraded":"2023-12-20T06:04:16.812896+0000","last_fullsized":"2023-12-20T06:04:16.812896+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:22:51.632261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7],"acting":[1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.1a","version":"250'1","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006969+0000","last_change":"2023-12-20T06:04:14.607464+0000","last_active":"2023-12-20T06:04:16.006969+0000","last_peered":"2023-12-20T06:04:16.006969+0000","last_clean":"2023-12-20T06:04:16.006969+0000","last_became_active":"2023-12-20T06:04:14.607200+0000","last_became_peered":"2023-12-20T06:04:14.607200+0000","last_unstale":"2023-12-20T06:04:16.006969+0000","last_undegraded":"2023-12-20T06:04:16.006969+0000","last_fullsized":"2023-12-20T06:04:16.006969+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:12:25.714647+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7],"acting":[3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.18","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812552+0000","last_change":"2023-12-20T06:04:15.029634+0000","last_active":"2023-12-20T06:04:16.812552+0000","last_peered":"2023-12-20T06:04:16.812552+0000","last_clean":"2023-12-20T06:04:16.812552+0000","last_became_active":"2023-12-20T06:04:15.028447+0000","last_became_peered":"2023-12-20T06:04:15.028447+0000","last_unstale":"2023-12-20T06:04:16.812552+0000","last_undegraded":"2023-12-20T06:04:16.812552+0000","last_fullsized":"2023-12-20T06:04:16.812552+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:20:34.577593+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.19","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820155+0000","last_change":"2023-12-20T06:04:14.610708+0000","last_active":"2023-12-20T06:04:16.820155+0000","last_peered":"2023-12-20T06:04:16.820155+0000","last_clean":"2023-12-20T06:04:16.820155+0000","last_became_active":"2023-12-20T06:04:14.610395+0000","last_became_peered":"2023-12-20T06:04:14.610395+0000","last_unstale":"2023-12-20T06:04:16.820155+0000","last_undegraded":"2023-12-20T06:04:16.820155+0000","last_fullsized":"2023-12-20T06:04:16.820155+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:10:04.670922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.19","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820436+0000","last_change":"2023-12-20T06:04:15.031728+0000","last_active":"2023-12-20T06:04:16.820436+0000","last_peered":"2023-12-20T06:04:16.820436+0000","last_clean":"2023-12-20T06:04:16.820436+0000","last_became_active":"2023-12-20T06:04:15.031437+0000","last_became_peered":"2023-12-20T06:04:15.031437+0000","last_unstale":"2023-12-20T06:04:16.820436+0000","last_undegraded":"2023-12-20T06:04:16.820436+0000","last_fullsized":"2023-12-20T06:04:16.820436+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:22:51.632261+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.18","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820322+0000","last_change":"2023-12-20T06:04:14.595955+0000","last_active":"2023-12-20T06:04:16.820322+0000","last_peered":"2023-12-20T06:04:16.820322+0000","last_clean":"2023-12-20T06:04:16.820322+0000","last_became_active":"2023-12-20T06:04:14.595565+0000","last_became_peered":"2023-12-20T06:04:14.595565+0000","last_unstale":"2023-12-20T06:04:16.820322+0000","last_undegraded":"2023-12-20T06:04:16.820322+0000","last_fullsized":"2023-12-20T06:04:16.820322+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:44:53.092997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3],"acting":[6,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.16","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821093+0000","last_change":"2023-12-20T06:04:15.031975+0000","last_active":"2023-12-20T06:04:16.821093+0000","last_peered":"2023-12-20T06:04:16.821093+0000","last_clean":"2023-12-20T06:04:16.821093+0000","last_became_active":"2023-12-20T06:04:15.031705+0000","last_became_peered":"2023-12-20T06:04:15.031705+0000","last_unstale":"2023-12-20T06:04:16.821093+0000","last_undegraded":"2023-12-20T06:04:16.821093+0000","last_fullsized":"2023-12-20T06:04:16.821093+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:58:11.789777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7],"acting":[6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"32.17","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007323+0000","last_change":"2023-12-20T06:04:14.610927+0000","last_active":"2023-12-20T06:04:16.007323+0000","last_peered":"2023-12-20T06:04:16.007323+0000","last_clean":"2023-12-20T06:04:16.007323+0000","last_became_active":"2023-12-20T06:04:14.610622+0000","last_became_peered":"2023-12-20T06:04:14.610622+0000","last_unstale":"2023-12-20T06:04:16.007323+0000","last_undegraded":"2023-12-20T06:04:16.007323+0000","last_fullsized":"2023-12-20T06:04:16.007323+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:17:09.477999+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.17","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007196+0000","last_change":"2023-12-20T06:04:15.022033+0000","last_active":"2023-12-20T06:04:16.007196+0000","last_peered":"2023-12-20T06:04:16.007196+0000","last_clean":"2023-12-20T06:04:16.007196+0000","last_became_active":"2023-12-20T06:04:15.021775+0000","last_became_peered":"2023-12-20T06:04:15.021775+0000","last_unstale":"2023-12-20T06:04:16.007196+0000","last_undegraded":"2023-12-20T06:04:16.007196+0000","last_fullsized":"2023-12-20T06:04:16.007196+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:08:48.223773+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.16","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812744+0000","last_change":"2023-12-20T06:04:14.606809+0000","last_active":"2023-12-20T06:04:16.812744+0000","last_peered":"2023-12-20T06:04:16.812744+0000","last_clean":"2023-12-20T06:04:16.812744+0000","last_became_active":"2023-12-20T06:04:14.606456+0000","last_became_peered":"2023-12-20T06:04:14.606456+0000","last_unstale":"2023-12-20T06:04:16.812744+0000","last_undegraded":"2023-12-20T06:04:16.812744+0000","last_fullsized":"2023-12-20T06:04:16.812744+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:15:30.384161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.14","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107668+0000","last_change":"2023-12-20T06:04:15.022070+0000","last_active":"2023-12-20T06:04:16.107668+0000","last_peered":"2023-12-20T06:04:16.107668+0000","last_clean":"2023-12-20T06:04:16.107668+0000","last_became_active":"2023-12-20T06:04:15.020840+0000","last_became_peered":"2023-12-20T06:04:15.020840+0000","last_unstale":"2023-12-20T06:04:16.107668+0000","last_undegraded":"2023-12-20T06:04:16.107668+0000","last_fullsized":"2023-12-20T06:04:16.107668+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:43:22.692480+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,4],"acting":[2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.15","version":"250'3","reported_seq":14,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.813080+0000","last_change":"2023-12-20T06:04:14.608212+0000","last_active":"2023-12-20T06:04:16.813080+0000","last_peered":"2023-12-20T06:04:16.813080+0000","last_clean":"2023-12-20T06:04:16.813080+0000","last_became_active":"2023-12-20T06:04:14.606396+0000","last_became_peered":"2023-12-20T06:04:14.606396+0000","last_unstale":"2023-12-20T06:04:16.813080+0000","last_undegraded":"2023-12-20T06:04:16.813080+0000","last_fullsized":"2023-12-20T06:04:16.813080+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:44:53.092997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.15","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007322+0000","last_change":"2023-12-20T06:04:15.025640+0000","last_active":"2023-12-20T06:04:16.007322+0000","last_peered":"2023-12-20T06:04:16.007322+0000","last_clean":"2023-12-20T06:04:16.007322+0000","last_became_active":"2023-12-20T06:04:15.025393+0000","last_became_peered":"2023-12-20T06:04:15.025393+0000","last_unstale":"2023-12-20T06:04:16.007322+0000","last_undegraded":"2023-12-20T06:04:16.007322+0000","last_fullsized":"2023-12-20T06:04:16.007322+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:45:19.685093+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.14","version":"250'6","reported_seq":17,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006988+0000","last_change":"2023-12-20T06:04:14.612263+0000","last_active":"2023-12-20T06:04:16.006988+0000","last_peered":"2023-12-20T06:04:16.006988+0000","last_clean":"2023-12-20T06:04:16.006988+0000","last_became_active":"2023-12-20T06:04:14.611940+0000","last_became_peered":"2023-12-20T06:04:14.611940+0000","last_unstale":"2023-12-20T06:04:16.006988+0000","last_undegraded":"2023-12-20T06:04:16.006988+0000","last_fullsized":"2023-12-20T06:04:16.006988+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:14:40.868048+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1564,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.12","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006874+0000","last_change":"2023-12-20T06:04:15.023822+0000","last_active":"2023-12-20T06:04:16.006874+0000","last_peered":"2023-12-20T06:04:16.006874+0000","last_clean":"2023-12-20T06:04:16.006874+0000","last_became_active":"2023-12-20T06:04:15.023577+0000","last_became_peered":"2023-12-20T06:04:15.023577+0000","last_unstale":"2023-12-20T06:04:16.006874+0000","last_undegraded":"2023-12-20T06:04:16.006874+0000","last_fullsized":"2023-12-20T06:04:16.006874+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:00:11.404637+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.13","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010847+0000","last_change":"2023-12-20T06:04:14.606823+0000","last_active":"2023-12-20T06:04:16.010847+0000","last_peered":"2023-12-20T06:04:16.010847+0000","last_clean":"2023-12-20T06:04:16.010847+0000","last_became_active":"2023-12-20T06:04:14.604714+0000","last_became_peered":"2023-12-20T06:04:14.604714+0000","last_unstale":"2023-12-20T06:04:16.010847+0000","last_undegraded":"2023-12-20T06:04:16.010847+0000","last_fullsized":"2023-12-20T06:04:16.010847+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:06:42.173623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.13","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812916+0000","last_change":"2023-12-20T06:04:15.028720+0000","last_active":"2023-12-20T06:04:16.812916+0000","last_peered":"2023-12-20T06:04:16.812916+0000","last_clean":"2023-12-20T06:04:16.812916+0000","last_became_active":"2023-12-20T06:04:15.028413+0000","last_became_peered":"2023-12-20T06:04:15.028413+0000","last_unstale":"2023-12-20T06:04:16.812916+0000","last_undegraded":"2023-12-20T06:04:16.812916+0000","last_fullsized":"2023-12-20T06:04:16.812916+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:40:09.280259+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.12","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822132+0000","last_change":"2023-12-20T06:04:14.607764+0000","last_active":"2023-12-20T06:04:16.822132+0000","last_peered":"2023-12-20T06:04:16.822132+0000","last_clean":"2023-12-20T06:04:16.822132+0000","last_became_active":"2023-12-20T06:04:14.607456+0000","last_became_peered":"2023-12-20T06:04:14.607456+0000","last_unstale":"2023-12-20T06:04:16.822132+0000","last_undegraded":"2023-12-20T06:04:16.822132+0000","last_fullsized":"2023-12-20T06:04:16.822132+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:05:07.499505+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.10","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006817+0000","last_change":"2023-12-20T06:04:15.019988+0000","last_active":"2023-12-20T06:04:16.006817+0000","last_peered":"2023-12-20T06:04:16.006817+0000","last_clean":"2023-12-20T06:04:16.006817+0000","last_became_active":"2023-12-20T06:04:15.019623+0000","last_became_peered":"2023-12-20T06:04:15.019623+0000","last_unstale":"2023-12-20T06:04:16.006817+0000","last_undegraded":"2023-12-20T06:04:16.006817+0000","last_fullsized":"2023-12-20T06:04:16.006817+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:38:34.548311+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2],"acting":[3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.11","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.009509+0000","last_change":"2023-12-20T06:04:14.612181+0000","last_active":"2023-12-20T06:04:16.009509+0000","last_peered":"2023-12-20T06:04:16.009509+0000","last_clean":"2023-12-20T06:04:16.009509+0000","last_became_active":"2023-12-20T06:04:14.611534+0000","last_became_peered":"2023-12-20T06:04:14.611534+0000","last_unstale":"2023-12-20T06:04:16.009509+0000","last_undegraded":"2023-12-20T06:04:16.009509+0000","last_fullsized":"2023-12-20T06:04:16.009509+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T18:02:28.555973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.11","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006821+0000","last_change":"2023-12-20T06:04:15.020621+0000","last_active":"2023-12-20T06:04:16.006821+0000","last_peered":"2023-12-20T06:04:16.006821+0000","last_clean":"2023-12-20T06:04:16.006821+0000","last_became_active":"2023-12-20T06:04:15.020358+0000","last_became_peered":"2023-12-20T06:04:15.020358+0000","last_unstale":"2023-12-20T06:04:16.006821+0000","last_undegraded":"2023-12-20T06:04:16.006821+0000","last_fullsized":"2023-12-20T06:04:16.006821+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:43:16.055416+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5],"acting":[3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.10","version":"250'8","reported_seq":19,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812688+0000","last_change":"2023-12-20T06:04:14.611694+0000","last_active":"2023-12-20T06:04:16.812688+0000","last_peered":"2023-12-20T06:04:16.812688+0000","last_clean":"2023-12-20T06:04:16.812688+0000","last_became_active":"2023-12-20T06:04:14.611419+0000","last_became_peered":"2023-12-20T06:04:14.611419+0000","last_unstale":"2023-12-20T06:04:16.812688+0000","last_undegraded":"2023-12-20T06:04:16.812688+0000","last_fullsized":"2023-12-20T06:04:16.812688+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T16:23:47.189516+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.e","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107799+0000","last_change":"2023-12-20T06:04:15.030753+0000","last_active":"2023-12-20T06:04:16.107799+0000","last_peered":"2023-12-20T06:04:16.107799+0000","last_clean":"2023-12-20T06:04:16.107799+0000","last_became_active":"2023-12-20T06:04:15.029827+0000","last_became_peered":"2023-12-20T06:04:15.029827+0000","last_unstale":"2023-12-20T06:04:16.107799+0000","last_undegraded":"2023-12-20T06:04:16.107799+0000","last_fullsized":"2023-12-20T06:04:16.107799+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:53:43.514521+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.f","version":"250'1","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.011118+0000","last_change":"2023-12-20T06:04:14.607628+0000","last_active":"2023-12-20T06:04:16.011118+0000","last_peered":"2023-12-20T06:04:16.011118+0000","last_clean":"2023-12-20T06:04:16.011118+0000","last_became_active":"2023-12-20T06:04:14.607093+0000","last_became_peered":"2023-12-20T06:04:14.607093+0000","last_unstale":"2023-12-20T06:04:16.011118+0000","last_undegraded":"2023-12-20T06:04:16.011118+0000","last_fullsized":"2023-12-20T06:04:16.011118+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:30:02.428212+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6],"acting":[4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.f","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.009584+0000","last_change":"2023-12-20T06:04:15.034348+0000","last_active":"2023-12-20T06:04:16.009584+0000","last_peered":"2023-12-20T06:04:16.009584+0000","last_clean":"2023-12-20T06:04:16.009584+0000","last_became_active":"2023-12-20T06:04:15.034065+0000","last_became_peered":"2023-12-20T06:04:15.034065+0000","last_unstale":"2023-12-20T06:04:16.009584+0000","last_undegraded":"2023-12-20T06:04:16.009584+0000","last_fullsized":"2023-12-20T06:04:16.009584+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:42:11.485396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,2],"acting":[7,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"32.e","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812767+0000","last_change":"2023-12-20T06:04:14.607358+0000","last_active":"2023-12-20T06:04:16.812767+0000","last_peered":"2023-12-20T06:04:16.812767+0000","last_clean":"2023-12-20T06:04:16.812767+0000","last_became_active":"2023-12-20T06:04:14.607100+0000","last_became_peered":"2023-12-20T06:04:14.607100+0000","last_unstale":"2023-12-20T06:04:16.812767+0000","last_undegraded":"2023-12-20T06:04:16.812767+0000","last_fullsized":"2023-12-20T06:04:16.812767+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:44:39.078018+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"33.c","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107747+0000","last_change":"2023-12-20T06:04:15.028413+0000","last_active":"2023-12-20T06:04:16.107747+0000","last_peered":"2023-12-20T06:04:16.107747+0000","last_clean":"2023-12-20T06:04:16.107747+0000","last_became_active":"2023-12-20T06:04:15.028141+0000","last_became_peered":"2023-12-20T06:04:15.028141+0000","last_unstale":"2023-12-20T06:04:16.107747+0000","last_undegraded":"2023-12-20T06:04:16.107747+0000","last_fullsized":"2023-12-20T06:04:16.107747+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:23:41.923063+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.d","version":"250'1","reported_seq":12,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.009230+0000","last_change":"2023-12-20T06:04:14.614344+0000","last_active":"2023-12-20T06:04:16.009230+0000","last_peered":"2023-12-20T06:04:16.009230+0000","last_clean":"2023-12-20T06:04:16.009230+0000","last_became_active":"2023-12-20T06:04:14.614107+0000","last_became_peered":"2023-12-20T06:04:14.614107+0000","last_unstale":"2023-12-20T06:04:16.009230+0000","last_undegraded":"2023-12-20T06:04:16.009230+0000","last_fullsized":"2023-12-20T06:04:16.009230+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:10:04.670922+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.d","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821029+0000","last_change":"2023-12-20T06:04:15.023242+0000","last_active":"2023-12-20T06:04:16.821029+0000","last_peered":"2023-12-20T06:04:16.821029+0000","last_clean":"2023-12-20T06:04:16.821029+0000","last_became_active":"2023-12-20T06:04:15.022920+0000","last_became_peered":"2023-12-20T06:04:15.022920+0000","last_unstale":"2023-12-20T06:04:16.821029+0000","last_undegraded":"2023-12-20T06:04:16.821029+0000","last_fullsized":"2023-12-20T06:04:16.821029+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T08:21:26.067257+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.c","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107837+0000","last_change":"2023-12-20T06:04:14.609097+0000","last_active":"2023-12-20T06:04:16.107837+0000","last_peered":"2023-12-20T06:04:16.107837+0000","last_clean":"2023-12-20T06:04:16.107837+0000","last_became_active":"2023-12-20T06:04:14.608790+0000","last_became_peered":"2023-12-20T06:04:14.608790+0000","last_unstale":"2023-12-20T06:04:16.107837+0000","last_undegraded":"2023-12-20T06:04:16.107837+0000","last_fullsized":"2023-12-20T06:04:16.107837+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:05:24.770775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.a","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.812714+0000","last_change":"2023-12-20T06:04:15.015038+0000","last_active":"2023-12-20T06:04:16.812714+0000","last_peered":"2023-12-20T06:04:16.812714+0000","last_clean":"2023-12-20T06:04:16.812714+0000","last_became_active":"2023-12-20T06:04:15.014734+0000","last_became_peered":"2023-12-20T06:04:15.014734+0000","last_unstale":"2023-12-20T06:04:16.812714+0000","last_undegraded":"2023-12-20T06:04:16.812714+0000","last_fullsized":"2023-12-20T06:04:16.812714+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:59:33.073877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.b","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820352+0000","last_change":"2023-12-20T06:04:14.601943+0000","last_active":"2023-12-20T06:04:16.820352+0000","last_peered":"2023-12-20T06:04:16.820352+0000","last_clean":"2023-12-20T06:04:16.820352+0000","last_became_active":"2023-12-20T06:04:14.601666+0000","last_became_peered":"2023-12-20T06:04:14.601666+0000","last_unstale":"2023-12-20T06:04:16.820352+0000","last_undegraded":"2023-12-20T06:04:16.820352+0000","last_fullsized":"2023-12-20T06:04:16.820352+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T18:02:28.555973+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,7],"acting":[5,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.b","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.813024+0000","last_change":"2023-12-20T06:04:15.036938+0000","last_active":"2023-12-20T06:04:16.813024+0000","last_peered":"2023-12-20T06:04:16.813024+0000","last_clean":"2023-12-20T06:04:16.813024+0000","last_became_active":"2023-12-20T06:04:15.036597+0000","last_became_peered":"2023-12-20T06:04:15.036597+0000","last_unstale":"2023-12-20T06:04:16.813024+0000","last_undegraded":"2023-12-20T06:04:16.813024+0000","last_fullsized":"2023-12-20T06:04:16.813024+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:58:11.789777+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"32.a","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010748+0000","last_change":"2023-12-20T06:04:14.610114+0000","last_active":"2023-12-20T06:04:16.010748+0000","last_peered":"2023-12-20T06:04:16.010748+0000","last_clean":"2023-12-20T06:04:16.010748+0000","last_became_active":"2023-12-20T06:04:14.609666+0000","last_became_peered":"2023-12-20T06:04:14.609666+0000","last_unstale":"2023-12-20T06:04:16.010748+0000","last_undegraded":"2023-12-20T06:04:16.010748+0000","last_fullsized":"2023-12-20T06:04:16.010748+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:03:04.243865+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3],"acting":[4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.8","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.006917+0000","last_change":"2023-12-20T06:04:15.020228+0000","last_active":"2023-12-20T06:04:16.006917+0000","last_peered":"2023-12-20T06:04:16.006917+0000","last_clean":"2023-12-20T06:04:16.006917+0000","last_became_active":"2023-12-20T06:04:15.019979+0000","last_became_peered":"2023-12-20T06:04:15.019979+0000","last_unstale":"2023-12-20T06:04:16.006917+0000","last_undegraded":"2023-12-20T06:04:16.006917+0000","last_fullsized":"2023-12-20T06:04:16.006917+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:54:08.613432+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.9","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.009567+0000","last_change":"2023-12-20T06:04:14.613233+0000","last_active":"2023-12-20T06:04:16.009567+0000","last_peered":"2023-12-20T06:04:16.009567+0000","last_clean":"2023-12-20T06:04:16.009567+0000","last_became_active":"2023-12-20T06:04:14.612954+0000","last_became_peered":"2023-12-20T06:04:14.612954+0000","last_unstale":"2023-12-20T06:04:16.009567+0000","last_undegraded":"2023-12-20T06:04:16.009567+0000","last_fullsized":"2023-12-20T06:04:16.009567+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:12:02.350675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1],"acting":[7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"33.9","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822304+0000","last_change":"2023-12-20T06:04:15.021729+0000","last_active":"2023-12-20T06:04:16.822304+0000","last_peered":"2023-12-20T06:04:16.822304+0000","last_clean":"2023-12-20T06:04:16.822304+0000","last_became_active":"2023-12-20T06:04:15.020957+0000","last_became_peered":"2023-12-20T06:04:15.020957+0000","last_unstale":"2023-12-20T06:04:16.822304+0000","last_undegraded":"2023-12-20T06:04:16.822304+0000","last_fullsized":"2023-12-20T06:04:16.822304+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:10:45.469137+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2],"acting":[0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"32.8","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.010796+0000","last_change":"2023-12-20T06:04:14.606002+0000","last_active":"2023-12-20T06:04:16.010796+0000","last_peered":"2023-12-20T06:04:16.010796+0000","last_clean":"2023-12-20T06:04:16.010796+0000","last_became_active":"2023-12-20T06:04:14.605694+0000","last_became_peered":"2023-12-20T06:04:14.605694+0000","last_unstale":"2023-12-20T06:04:16.010796+0000","last_undegraded":"2023-12-20T06:04:16.010796+0000","last_fullsized":"2023-12-20T06:04:16.010796+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:35:10.965321+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5],"acting":[4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"33.6","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.108023+0000","last_change":"2023-12-20T06:04:15.029526+0000","last_active":"2023-12-20T06:04:16.108023+0000","last_peered":"2023-12-20T06:04:16.108023+0000","last_clean":"2023-12-20T06:04:16.108023+0000","last_became_active":"2023-12-20T06:04:15.029266+0000","last_became_peered":"2023-12-20T06:04:15.029266+0000","last_unstale":"2023-12-20T06:04:16.108023+0000","last_undegraded":"2023-12-20T06:04:16.108023+0000","last_fullsized":"2023-12-20T06:04:16.108023+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T07:55:16.774802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.7","version":"250'2","reported_seq":13,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107941+0000","last_change":"2023-12-20T06:04:14.605601+0000","last_active":"2023-12-20T06:04:16.107941+0000","last_peered":"2023-12-20T06:04:16.107941+0000","last_clean":"2023-12-20T06:04:16.107941+0000","last_became_active":"2023-12-20T06:04:14.605317+0000","last_became_peered":"2023-12-20T06:04:14.605317+0000","last_unstale":"2023-12-20T06:04:16.107941+0000","last_undegraded":"2023-12-20T06:04:16.107941+0000","last_fullsized":"2023-12-20T06:04:16.107941+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:07:48.209205+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,0],"acting":[2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.7","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007238+0000","last_change":"2023-12-20T06:04:15.021464+0000","last_active":"2023-12-20T06:04:16.007238+0000","last_peered":"2023-12-20T06:04:16.007238+0000","last_clean":"2023-12-20T06:04:16.007238+0000","last_became_active":"2023-12-20T06:04:15.019928+0000","last_became_peered":"2023-12-20T06:04:15.019928+0000","last_unstale":"2023-12-20T06:04:16.007238+0000","last_undegraded":"2023-12-20T06:04:16.007238+0000","last_fullsized":"2023-12-20T06:04:16.007238+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:40:47.345135+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4],"acting":[3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"32.6","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821201+0000","last_change":"2023-12-20T06:04:14.607988+0000","last_active":"2023-12-20T06:04:16.821201+0000","last_peered":"2023-12-20T06:04:16.821201+0000","last_clean":"2023-12-20T06:04:16.821201+0000","last_became_active":"2023-12-20T06:04:14.605885+0000","last_became_peered":"2023-12-20T06:04:14.605885+0000","last_unstale":"2023-12-20T06:04:16.821201+0000","last_undegraded":"2023-12-20T06:04:16.821201+0000","last_fullsized":"2023-12-20T06:04:16.821201+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:12:02.350675+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"33.4","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820603+0000","last_change":"2023-12-20T06:04:15.023993+0000","last_active":"2023-12-20T06:04:16.820603+0000","last_peered":"2023-12-20T06:04:16.820603+0000","last_clean":"2023-12-20T06:04:16.820603+0000","last_became_active":"2023-12-20T06:04:15.023708+0000","last_became_peered":"2023-12-20T06:04:15.023708+0000","last_unstale":"2023-12-20T06:04:16.820603+0000","last_undegraded":"2023-12-20T06:04:16.820603+0000","last_fullsized":"2023-12-20T06:04:16.820603+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T13:27:16.568576+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0],"acting":[5,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.5","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820791+0000","last_change":"2023-12-20T06:04:14.605328+0000","last_active":"2023-12-20T06:04:16.820791+0000","last_peered":"2023-12-20T06:04:16.820791+0000","last_clean":"2023-12-20T06:04:16.820791+0000","last_became_active":"2023-12-20T06:04:14.604947+0000","last_became_peered":"2023-12-20T06:04:14.604947+0000","last_unstale":"2023-12-20T06:04:16.820791+0000","last_undegraded":"2023-12-20T06:04:16.820791+0000","last_fullsized":"2023-12-20T06:04:16.820791+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T15:06:42.173623+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5],"acting":[6,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"33.5","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.821089+0000","last_change":"2023-12-20T06:04:15.022561+0000","last_active":"2023-12-20T06:04:16.821089+0000","last_peered":"2023-12-20T06:04:16.821089+0000","last_clean":"2023-12-20T06:04:16.821089+0000","last_became_active":"2023-12-20T06:04:15.020667+0000","last_became_peered":"2023-12-20T06:04:15.020667+0000","last_unstale":"2023-12-20T06:04:16.821089+0000","last_undegraded":"2023-12-20T06:04:16.821089+0000","last_fullsized":"2023-12-20T06:04:16.821089+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T06:30:08.600616+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1],"acting":[5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.4","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007075+0000","last_change":"2023-12-20T06:04:14.610008+0000","last_active":"2023-12-20T06:04:16.007075+0000","last_peered":"2023-12-20T06:04:16.007075+0000","last_clean":"2023-12-20T06:04:16.007075+0000","last_became_active":"2023-12-20T06:04:14.609653+0000","last_became_peered":"2023-12-20T06:04:14.609653+0000","last_unstale":"2023-12-20T06:04:16.007075+0000","last_undegraded":"2023-12-20T06:04:16.007075+0000","last_fullsized":"2023-12-20T06:04:16.007075+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T17:25:44.191425+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.2","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107751+0000","last_change":"2023-12-20T06:04:15.033904+0000","last_active":"2023-12-20T06:04:16.107751+0000","last_peered":"2023-12-20T06:04:16.107751+0000","last_clean":"2023-12-20T06:04:16.107751+0000","last_became_active":"2023-12-20T06:04:15.033497+0000","last_became_peered":"2023-12-20T06:04:15.033497+0000","last_unstale":"2023-12-20T06:04:16.107751+0000","last_undegraded":"2023-12-20T06:04:16.107751+0000","last_fullsized":"2023-12-20T06:04:16.107751+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:12:03.367985+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3],"acting":[2,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.3","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822097+0000","last_change":"2023-12-20T06:04:14.611805+0000","last_active":"2023-12-20T06:04:16.822097+0000","last_peered":"2023-12-20T06:04:16.822097+0000","last_clean":"2023-12-20T06:04:16.822097+0000","last_became_active":"2023-12-20T06:04:14.611271+0000","last_became_peered":"2023-12-20T06:04:14.611271+0000","last_unstale":"2023-12-20T06:04:16.822097+0000","last_undegraded":"2023-12-20T06:04:16.822097+0000","last_fullsized":"2023-12-20T06:04:16.822097+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:00:10.387327+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.3","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107939+0000","last_change":"2023-12-20T06:04:15.027636+0000","last_active":"2023-12-20T06:04:16.107939+0000","last_peered":"2023-12-20T06:04:16.107939+0000","last_clean":"2023-12-20T06:04:16.107939+0000","last_became_active":"2023-12-20T06:04:15.027377+0000","last_became_peered":"2023-12-20T06:04:15.027377+0000","last_unstale":"2023-12-20T06:04:16.107939+0000","last_undegraded":"2023-12-20T06:04:16.107939+0000","last_fullsized":"2023-12-20T06:04:16.107939+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T11:10:05.688232+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.2","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.822208+0000","last_change":"2023-12-20T06:04:14.606148+0000","last_active":"2023-12-20T06:04:16.822208+0000","last_peered":"2023-12-20T06:04:16.822208+0000","last_clean":"2023-12-20T06:04:16.822208+0000","last_became_active":"2023-12-20T06:04:14.605822+0000","last_became_peered":"2023-12-20T06:04:14.605822+0000","last_unstale":"2023-12-20T06:04:16.822208+0000","last_undegraded":"2023-12-20T06:04:16.822208+0000","last_fullsized":"2023-12-20T06:04:16.822208+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:38:33.531001+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6],"acting":[0,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"33.0","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107922+0000","last_change":"2023-12-20T06:04:15.027128+0000","last_active":"2023-12-20T06:04:16.107922+0000","last_peered":"2023-12-20T06:04:16.107922+0000","last_clean":"2023-12-20T06:04:16.107922+0000","last_became_active":"2023-12-20T06:04:15.026859+0000","last_became_peered":"2023-12-20T06:04:15.026859+0000","last_unstale":"2023-12-20T06:04:16.107922+0000","last_undegraded":"2023-12-20T06:04:16.107922+0000","last_fullsized":"2023-12-20T06:04:16.107922+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:42:11.485396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"32.1","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.107972+0000","last_change":"2023-12-20T06:04:14.607389+0000","last_active":"2023-12-20T06:04:16.107972+0000","last_peered":"2023-12-20T06:04:16.107972+0000","last_clean":"2023-12-20T06:04:16.107972+0000","last_became_active":"2023-12-20T06:04:14.606834+0000","last_became_peered":"2023-12-20T06:04:14.606834+0000","last_unstale":"2023-12-20T06:04:16.107972+0000","last_undegraded":"2023-12-20T06:04:16.107972+0000","last_fullsized":"2023-12-20T06:04:16.107972+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T14:09:38.660788+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,7],"acting":[2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"33.1","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820072+0000","last_change":"2023-12-20T06:04:15.013346+0000","last_active":"2023-12-20T06:04:16.820072+0000","last_peered":"2023-12-20T06:04:16.820072+0000","last_clean":"2023-12-20T06:04:16.820072+0000","last_became_active":"2023-12-20T06:04:15.013036+0000","last_became_peered":"2023-12-20T06:04:15.013036+0000","last_unstale":"2023-12-20T06:04:16.820072+0000","last_undegraded":"2023-12-20T06:04:16.820072+0000","last_fullsized":"2023-12-20T06:04:16.820072+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:28:55.406087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2],"acting":[5,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"32.0","version":"0'0","reported_seq":11,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.007120+0000","last_change":"2023-12-20T06:04:14.609116+0000","last_active":"2023-12-20T06:04:16.007120+0000","last_peered":"2023-12-20T06:04:16.007120+0000","last_clean":"2023-12-20T06:04:16.007120+0000","last_became_active":"2023-12-20T06:04:14.608854+0000","last_became_peered":"2023-12-20T06:04:14.608854+0000","last_unstale":"2023-12-20T06:04:16.007120+0000","last_undegraded":"2023-12-20T06:04:16.007120+0000","last_fullsized":"2023-12-20T06:04:16.007120+0000","mapping_epoch":248,"log_start":"0'0","ondisk_log_start":"0'0","created":248,"last_epoch_clean":249,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:13.563671+0000","last_clean_scrub_stamp":"2023-12-20T06:04:13.563671+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:32:52.049835+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"33.20","version":"0'0","reported_seq":10,"reported_epoch":251,"state":"active+clean","last_fresh":"2023-12-20T06:04:16.820272+0000","last_change":"2023-12-20T06:04:15.021469+0000","last_active":"2023-12-20T06:04:16.820272+0000","last_peered":"2023-12-20T06:04:16.820272+0000","last_clean":"2023-12-20T06:04:16.820272+0000","last_became_active":"2023-12-20T06:04:15.021184+0000","last_became_peered":"2023-12-20T06:04:15.021184+0000","last_unstale":"2023-12-20T06:04:16.820272+0000","last_undegraded":"2023-12-20T06:04:16.820272+0000","last_fullsized":"2023-12-20T06:04:16.820272+0000","mapping_epoch":249,"log_start":"0'0","ondisk_log_start":"0'0","created":249,"last_epoch_clean":250,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T06:04:14.580982+0000","last_clean_scrub_stamp":"2023-12-20T06:04:14.580982+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T10:14:41.885358+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2],"acting":[6,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"1.0","version":"236'3234","reported_seq":4147,"reported_epoch":250,"state":"active+clean","last_fresh":"2023-12-20T06:04:15.016278+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:04:15.016278+0000","last_peered":"2023-12-20T06:04:15.016278+0000","last_clean":"2023-12-20T06:04:15.016278+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:04:15.016278+0000","last_undegraded":"2023-12-20T06:04:15.016278+0000","last_fullsized":"2023-12-20T06:04:15.016278+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3234,"log_dups_size":0,"ondisk_log_size":3234,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2173,"num_read_kb":4467,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":33,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":8},{"poolid":32,"num_pg":32,"stat_sum":{"num_bytes":2362,"num_objects":21,"num_object_clones":0,"num_object_copies":42,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":21,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":44,"num_write_kb":14,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":13,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":65536,"data_stored":4724,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":38,"ondisk_log_size":38,"up":64,"acting":64,"num_store_stats":8},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2173,"num_read_kb":4467,"num_write":581,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3234,"ondisk_log_size":3234,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575990,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60964,"kb_used_data":3676,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94310876,"statfs":{"total":96636764160,"available":96574337024,"internally_reserved":0,"allocated":3764224,"data_stored":2145708,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63700000000000001,"5min":0.69599999999999995,"15min":0.72799999999999998},"min":{"1min":0.45400000000000001,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.89100000000000001,"5min":1.028,"15min":1.042},"last":0.58499999999999996},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.70399999999999996,"15min":0.73199999999999998},"min":{"1min":0.39700000000000002,"5min":0.39700000000000002,"15min":0.34499999999999997},"max":{"1min":0.871,"5min":0.99099999999999999,"15min":1.161},"last":0.61499999999999999}]},{"osd":1,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67200000000000004,"5min":0.71499999999999997,"15min":0.74199999999999999},"min":{"1min":0.433,"5min":0.433,"15min":0.432},"max":{"1min":0.89800000000000002,"5min":0.996,"15min":1.1279999999999999},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.65900000000000003,"5min":0.71599999999999997,"15min":0.745},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.85499999999999998,"5min":0.96699999999999997,"15min":1.032},"last":1.0389999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.64100000000000001,"5min":0.70599999999999996,"15min":0.73899999999999999},"min":{"1min":0.46999999999999997,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.84499999999999997,"5min":0.95399999999999996,"15min":1.014},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.74199999999999999,"15min":0.75900000000000001},"min":{"1min":0.499,"5min":0.499,"15min":0.49099999999999999},"max":{"1min":0.88700000000000001,"5min":1.0089999999999999,"15min":1.0189999999999999},"last":0.85799999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.67000000000000004,"5min":0.71599999999999997,"15min":0.73999999999999999},"min":{"1min":0.47699999999999998,"5min":0.442,"15min":0.442},"max":{"1min":0.85299999999999998,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":1.0189999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77000000000000002,"15min":0.78400000000000003},"min":{"1min":0.41199999999999998,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.89900000000000002,"5min":1.0660000000000001,"15min":1.0840000000000001},"last":0.98899999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.61199999999999999,"15min":0.59599999999999997},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.97199999999999998,"5min":0.97199999999999998,"15min":1.04},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.65600000000000003,"5min":0.66300000000000003,"15min":0.65000000000000002},"min":{"1min":0.41799999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.88300000000000001,"5min":1.02,"15min":1.02},"last":0.83099999999999996}]},{"osd":5,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.68300000000000005,"15min":0.66800000000000004},"min":{"1min":0.37,"5min":0.37,"15min":0.37},"max":{"1min":0.99199999999999999,"5min":0.99199999999999999,"15min":1.052},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.69799999999999995,"15min":0.68899999999999995},"min":{"1min":0.45300000000000001,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":0.875,"5min":1.0229999999999999,"15min":1.103},"last":0.80900000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:03:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73599999999999999,"5min":0.76300000000000001,"15min":0.746},"min":{"1min":0.442,"5min":0.442,"15min":0.442},"max":{"1min":0.92200000000000004,"5min":1.0449999999999999,"15min":1.2110000000000001},"last":0.89400000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.72399999999999998,"15min":0.72099999999999997},"min":{"1min":0.52100000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.93500000000000005,"5min":1.0740000000000001,"15min":1.081},"last":0.96099999999999997}]}]},{"osd":6,"up_from":13,"seq":55834575990,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52196,"kb_used_data":3100,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319644,"statfs":{"total":96636764160,"available":96583315456,"internally_reserved":0,"allocated":3174400,"data_stored":1555382,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.64500000000000002,"5min":0.68300000000000005,"15min":0.73299999999999998},"min":{"1min":0.312,"5min":0.312,"15min":0.312},"max":{"1min":0.89100000000000001,"5min":1.016,"15min":1.0980000000000001},"last":0.74099999999999999},{"interface":"front","average":{"1min":0.66200000000000003,"5min":0.70499999999999996,"15min":0.74299999999999999},"min":{"1min":0.433,"5min":0.376,"15min":0.376},"max":{"1min":0.94499999999999995,"5min":1.006,"15min":1.05},"last":0.82299999999999995}]},{"osd":1,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.625,"5min":0.69899999999999995,"15min":0.72499999999999998},"min":{"1min":0.39100000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.96699999999999997,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.71799999999999997},{"interface":"front","average":{"1min":0.64300000000000002,"5min":0.72199999999999998,"15min":0.75800000000000001},"min":{"1min":0.371,"5min":0.371,"15min":0.371},"max":{"1min":0.84999999999999998,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.66500000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.68799999999999994,"5min":0.747,"15min":0.77100000000000002},"min":{"1min":0.52400000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.96199999999999997,"5min":1.069,"15min":1.069},"last":0.80100000000000005},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71799999999999997,"15min":0.75800000000000001},"min":{"1min":0.53100000000000003,"5min":0.441,"15min":0.441},"max":{"1min":0.85999999999999999,"5min":0.97799999999999998,"15min":1.0760000000000001},"last":0.76500000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.76300000000000001,"15min":0.78100000000000003},"min":{"1min":0.51300000000000001,"5min":0.51300000000000001,"15min":0.46500000000000002},"max":{"1min":0.95199999999999996,"5min":1.077,"15min":1.077},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.65300000000000002,"5min":0.72999999999999998,"15min":0.75600000000000001},"min":{"1min":0.49099999999999999,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.92500000000000004,"5min":1.052,"15min":1.052},"last":0.65100000000000002}]},{"osd":4,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60199999999999998,"5min":0.63,"15min":0.63500000000000001},"min":{"1min":0.34899999999999998,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.84399999999999997,"5min":1.05,"15min":1.0529999999999999},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.67200000000000004,"15min":0.65500000000000003},"min":{"1min":0.40600000000000003,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.86399999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.68899999999999995}]},{"osd":5,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.60899999999999999,"5min":0.64500000000000002,"15min":0.64100000000000001},"min":{"1min":0.41999999999999998,"5min":0.39700000000000002,"15min":0.377},"max":{"1min":0.876,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.53300000000000003},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.71499999999999997,"15min":0.70699999999999996},"min":{"1min":0.46800000000000003,"5min":0.45700000000000002,"15min":0.44900000000000001},"max":{"1min":0.91700000000000004,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.502}]},{"osd":7,"last update":"Wed Dec 20 06:03:36 2023","interfaces":[{"interface":"back","average":{"1min":0.73999999999999999,"5min":0.74399999999999999,"15min":0.73699999999999999},"min":{"1min":0.45100000000000001,"5min":0.45100000000000001,"15min":0.45100000000000001},"max":{"1min":0.98199999999999998,"5min":1.103,"15min":1.1240000000000001},"last":0.55700000000000005},{"interface":"front","average":{"1min":0.69799999999999995,"5min":0.72199999999999998,"15min":0.72899999999999998},"min":{"1min":0.46700000000000003,"5min":0.46700000000000003,"15min":0.41999999999999998},"max":{"1min":0.92600000000000005,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.59799999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575991,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52188,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319652,"statfs":{"total":96636764160,"available":96583323648,"internally_reserved":0,"allocated":3166208,"data_stored":1555270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.67300000000000004,"5min":0.71399999999999997,"15min":0.73499999999999999},"min":{"1min":0.503,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.91600000000000004,"5min":0.94799999999999995,"15min":1.077},"last":0.68600000000000005},{"interface":"front","average":{"1min":0.64400000000000002,"5min":0.71499999999999997,"15min":0.746},"min":{"1min":0.45600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.80300000000000005,"5min":0.96699999999999997,"15min":1.028},"last":0.746}]},{"osd":1,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.73599999999999999,"15min":0.751},"min":{"1min":0.47299999999999998,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":0.875,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.73499999999999999},{"interface":"front","average":{"1min":0.64800000000000002,"5min":0.72699999999999998,"15min":0.75800000000000001},"min":{"1min":0.443,"5min":0.443,"15min":0.443},"max":{"1min":0.88500000000000001,"5min":0.98799999999999999,"15min":1.0209999999999999},"last":0.81000000000000005}]},{"osd":2,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.63400000000000001,"5min":0.71399999999999997,"15min":0.751},"min":{"1min":0.44900000000000001,"5min":0.44900000000000001,"15min":0.44900000000000001},"max":{"1min":0.89000000000000001,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.83599999999999997},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.75700000000000001,"15min":0.77000000000000002},"min":{"1min":0.55200000000000005,"5min":0.55200000000000005,"15min":0.48299999999999998},"max":{"1min":1.0669999999999999,"5min":1.0669999999999999,"15min":1.0680000000000001},"last":0.79000000000000004}]},{"osd":3,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.75900000000000001,"5min":0.77900000000000003,"15min":0.78400000000000003},"min":{"1min":0.54200000000000004,"5min":0.53800000000000003,"15min":0.52000000000000002},"max":{"1min":0.91000000000000003,"5min":1.05,"15min":1.0669999999999999},"last":0.86499999999999999},{"interface":"front","average":{"1min":0.72199999999999998,"5min":0.76000000000000001,"15min":0.77700000000000002},"min":{"1min":0.60699999999999998,"5min":0.52100000000000002,"15min":0.52000000000000002},"max":{"1min":0.95499999999999996,"5min":0.98099999999999998,"15min":1.0629999999999999},"last":0.89800000000000002}]},{"osd":4,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.749,"5min":0.65900000000000003,"15min":0.64600000000000002},"min":{"1min":0.56000000000000005,"5min":0.42199999999999999,"15min":0.38200000000000001},"max":{"1min":0.94399999999999995,"5min":1.0640000000000001,"15min":1.077},"last":0.53400000000000003},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.67500000000000004,"15min":0.64800000000000002},"min":{"1min":0.42499999999999999,"5min":0.42499999999999999,"15min":0.41999999999999998},"max":{"1min":1.0649999999999999,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.58599999999999997}]},{"osd":6,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.72899999999999998,"5min":0.72099999999999997,"15min":0.70399999999999996},"min":{"1min":0.51100000000000001,"5min":0.49299999999999999,"15min":0.439},"max":{"1min":1.0449999999999999,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.61499999999999999},{"interface":"front","average":{"1min":0.69599999999999995,"5min":0.67300000000000004,"15min":0.65300000000000002},"min":{"1min":0.46800000000000003,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":1.097,"5min":1.097,"15min":1.097},"last":0.55500000000000005}]},{"osd":7,"last update":"Wed Dec 20 06:03:23 2023","interfaces":[{"interface":"back","average":{"1min":0.74199999999999999,"5min":0.73799999999999999,"15min":0.70999999999999996},"min":{"1min":0.53700000000000003,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1,"5min":1.153,"15min":1.153},"last":0.69999999999999996},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.78100000000000003,"15min":0.76700000000000002},"min":{"1min":0.58399999999999996,"5min":0.56299999999999994,"15min":0.48299999999999998},"max":{"1min":0.98799999999999999,"5min":1.0409999999999999,"15min":1.0409999999999999},"last":0.76000000000000001}]}]},{"osd":4,"up_from":13,"seq":55834575992,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52208,"kb_used_data":3112,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319632,"statfs":{"total":96636764160,"available":96583303168,"internally_reserved":0,"allocated":3186688,"data_stored":1556890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67500000000000004,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.98999999999999999,"15min":1.0209999999999999},"last":0.60199999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.71799999999999997},"min":{"1min":0.46200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.92600000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.96699999999999997}]},{"osd":1,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83999999999999997,"5min":0.95699999999999996,"15min":1.0900000000000001},"last":0.65400000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47399999999999998,"15min":0.44500000000000001},"max":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67100000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72299999999999998,"15min":0.76200000000000001},"min":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.70999999999999996},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.101},"last":0.77400000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70299999999999996,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":1.014,"15min":1.014},"last":0.90700000000000003},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97799999999999998,"5min":1.04,"15min":1.081},"last":0.755}]},{"osd":5,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":1.0589999999999999}]},{"osd":6,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.1379999999999999},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.749,"15min":0.745},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.41199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.82099999999999995}]},{"osd":7,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":1.0229999999999999},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.0740000000000001,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":1.0349999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575991,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52204,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319636,"statfs":{"total":96636764160,"available":96583307264,"internally_reserved":0,"allocated":3182592,"data_stored":1557416,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42899999999999999,"5min":0.45400000000000001,"15min":0.46899999999999997},"min":{"1min":0.255,"5min":0.255,"15min":0.255},"max":{"1min":0.58799999999999997,"5min":0.748,"15min":2.8130000000000002},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.45300000000000001,"5min":0.49299999999999999,"15min":0.52600000000000002},"min":{"1min":0.28000000000000003,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.70499999999999996,"5min":0.70599999999999996,"15min":1.357},"last":0.29899999999999999}]},{"osd":1,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.45700000000000002,"15min":0.47599999999999998},"min":{"1min":0.29499999999999998,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.65600000000000003,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.503},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.53000000000000003,"15min":0.56699999999999995},"min":{"1min":0.375,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.57599999999999996,"5min":0.81799999999999995,"15min":1.4219999999999999},"last":0.45600000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.46999999999999997,"5min":0.56399999999999995,"15min":0.58999999999999997},"min":{"1min":0.36099999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.57999999999999996,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.438},{"interface":"front","average":{"1min":0.47299999999999998,"5min":0.53800000000000003,"15min":0.56999999999999995},"min":{"1min":0.29899999999999999,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.69099999999999995,"5min":0.86099999999999999,"15min":2.944},"last":0.39100000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.76800000000000002,"5min":0.77300000000000002,"15min":0.81499999999999995},"min":{"1min":0.57299999999999995,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.98999999999999999,"5min":1.2090000000000001,"15min":1.669},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.73199999999999998,"5min":0.75900000000000001,"15min":0.80800000000000005},"min":{"1min":0.55000000000000004,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":0.86599999999999999,"5min":1.1299999999999999,"15min":1.99},"last":0.86599999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.77400000000000002,"5min":0.79600000000000004,"15min":0.82899999999999996},"min":{"1min":0.65100000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.93600000000000005,"5min":1.22,"15min":1.901},"last":0.81999999999999995},{"interface":"front","average":{"1min":0.753,"5min":0.76300000000000001,"15min":0.81999999999999995},"min":{"1min":0.58799999999999997,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.875,"5min":1.196,"15min":2.8639999999999999},"last":0.78200000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.74399999999999999,"5min":0.77500000000000002,"15min":0.81399999999999995},"min":{"1min":0.58599999999999997,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.95599999999999996,"5min":1.194,"15min":1.575},"last":0.85299999999999998},{"interface":"front","average":{"1min":0.78800000000000003,"5min":0.80500000000000005,"15min":0.84499999999999997},"min":{"1min":0.60299999999999998,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":1.0580000000000001,"5min":1.155,"15min":1.948},"last":0.72299999999999998}]},{"osd":7,"last update":"Wed Dec 20 06:03:28 2023","interfaces":[{"interface":"back","average":{"1min":0.78700000000000003,"5min":0.79000000000000004,"15min":0.83599999999999997},"min":{"1min":0.57099999999999995,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91900000000000004,"5min":1.1579999999999999,"15min":2.9049999999999998},"last":0.88500000000000001},{"interface":"front","average":{"1min":0.82199999999999995,"5min":0.80700000000000005,"15min":0.85299999999999998},"min":{"1min":0.623,"5min":0.502,"15min":0.502},"max":{"1min":1.0680000000000001,"5min":1.1850000000000001,"15min":1.754},"last":0.75800000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575990,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52188,"kb_used_data":3092,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319652,"statfs":{"total":96636764160,"available":96583323648,"internally_reserved":0,"allocated":3166208,"data_stored":1555270,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.372,"5min":0.45000000000000001,"15min":0.45500000000000002},"min":{"1min":0.26000000000000001,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.52900000000000003,"5min":1.0049999999999999,"15min":1.0049999999999999},"last":0.501},{"interface":"front","average":{"1min":0.438,"5min":0.51800000000000002,"15min":0.51900000000000002},"min":{"1min":0.33000000000000002,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.60699999999999998,"5min":1.4319999999999999,"15min":1.4319999999999999},"last":0.55500000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.44700000000000001,"5min":0.48499999999999999,"15min":0.48999999999999999},"min":{"1min":0.27900000000000003,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.68000000000000005,"5min":0.79800000000000004,"15min":0.82199999999999995},"last":0.47999999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.52700000000000002,"15min":0.54500000000000004},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.30499999999999999},"max":{"1min":0.61199999999999999,"5min":0.92600000000000005,"15min":1.004},"last":0.57699999999999996}]},{"osd":3,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.53300000000000003,"5min":0.629,"15min":0.64000000000000001},"min":{"1min":0.38,"5min":0.38,"15min":0.38},"max":{"1min":0.76000000000000001,"5min":1.3660000000000001,"15min":1.3660000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.53700000000000003,"5min":0.65500000000000003,"15min":0.67400000000000004},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.59199999999999997,"5min":1.6919999999999999,"15min":1.6919999999999999},"last":0.73999999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.70699999999999996,"5min":0.77200000000000002,"15min":0.79500000000000004},"min":{"1min":0.56599999999999995,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.90800000000000003,"5min":1.119,"15min":1.254},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.69399999999999995,"5min":0.77900000000000003,"15min":0.79600000000000004},"min":{"1min":0.40000000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.90300000000000002,"5min":1.605,"15min":1.605},"last":0.80600000000000005}]},{"osd":5,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.79800000000000004,"15min":0.80100000000000005},"min":{"1min":0.498,"5min":0.498,"15min":0.498},"max":{"1min":0.97099999999999997,"5min":1.329,"15min":1.329},"last":0.82799999999999996},{"interface":"front","average":{"1min":0.71599999999999997,"5min":0.81000000000000005,"15min":0.82499999999999996},"min":{"1min":0.53000000000000003,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.98199999999999998,"5min":1.0740000000000001,"15min":1.095},"last":0.88}]},{"osd":6,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.72199999999999998,"5min":0.80900000000000005,"15min":0.81599999999999995},"min":{"1min":0.50800000000000001,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":0.92200000000000004,"5min":1.919,"15min":1.919},"last":0.76000000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.81799999999999995,"15min":0.82599999999999996},"min":{"1min":0.55100000000000005,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.95099999999999996,"5min":1.2450000000000001,"15min":1.2450000000000001},"last":0.78500000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:03:22 2023","interfaces":[{"interface":"back","average":{"1min":0.76900000000000002,"5min":0.85499999999999998,"15min":0.85599999999999998},"min":{"1min":0.54600000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.0049999999999999,"5min":1.7869999999999999,"15min":1.7869999999999999},"last":1.02},{"interface":"front","average":{"1min":0.81499999999999995,"5min":0.88200000000000001,"15min":0.88500000000000001},"min":{"1min":0.57199999999999995,"5min":0.57099999999999995,"15min":0.57099999999999995},"max":{"1min":0.996,"5min":1.292,"15min":1.292},"last":1.0009999999999999}]}]},{"osd":1,"up_from":13,"seq":55834575992,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60396,"kb_used_data":3108,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311444,"statfs":{"total":96636764160,"available":96574918656,"internally_reserved":0,"allocated":3182592,"data_stored":1556046,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.504,"15min":0.51600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.42599999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.51700000000000002,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.54800000000000004},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.57399999999999995},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.64500000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.56999999999999995},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53300000000000003,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71399999999999997,"5min":0.78300000000000003,"15min":0.93500000000000005},"last":0.49199999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93100000000000005,"5min":1.107,"15min":1.321},"last":0.88900000000000001},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.81100000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.75700000000000001}]},{"osd":5,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.0649999999999999,"5min":1.224,"15min":1.2849999999999999},"last":0.72399999999999998},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77800000000000002,"15min":0.80400000000000005},"min":{"1min":0.55700000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.78400000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.80400000000000005,"15min":0.82699999999999996},"min":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.488,"5min":2.488,"15min":2.488},"last":0.90400000000000003},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.53200000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.84199999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.85799999999999998},"min":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.98499999999999999},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.82499999999999996,"15min":0.85999999999999999},"min":{"1min":0.621,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.3839999999999999},"last":0.999}]}]},{"osd":0,"up_from":13,"seq":55834575991,"num_pgs":12,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56864,"kb_used_data":3672,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94314976,"statfs":{"total":96636764160,"available":96578535424,"internally_reserved":0,"allocated":3760128,"data_stored":2145638,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.29299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":0.998,"15min":0.998},"last":0.33900000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.54300000000000004,"15min":0.55200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.63900000000000001,"5min":3.681,"15min":3.681},"last":0.52200000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60999999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.502},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.48099999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.67300000000000004,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.42699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66000000000000003,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.54300000000000004}]},{"osd":4,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.59199999999999997},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80500000000000005,"15min":0.82799999999999996},"min":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"max":{"1min":1.081,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.61199999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.84499999999999997,"15min":0.84399999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.042,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81599999999999995,"15min":0.84299999999999997},"min":{"1min":0.53700000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.1220000000000001,"5min":2.23,"15min":2.23},"last":0.55900000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.83599999999999997},"min":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0529999999999999,"5min":1.482,"15min":1.482},"last":0.58099999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.72299999999999998}]},{"osd":7,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96099999999999997,"5min":5.7160000000000002,"15min":5.7160000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.86399999999999999,"15min":0.86599999999999999},"min":{"1min":0.56899999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94399999999999995,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.63}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":776,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":2146,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1620,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":112,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":32,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":70,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":33,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:04:19.501 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2023-12-20T06:04:19.501 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph auth caps client.0 mon 'allow r' osd 'allow rw tag cephfs data=cephfs' mds allow 2023-12-20T06:04:19.691 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.689+0000 7f58432f9640 1 Processor -- start 2023-12-20T06:04:19.692 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.689+0000 7f58432f9640 1 -- start start 2023-12-20T06:04:19.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.689+0000 7f58432f9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f583c06ca40 0x7f583c06ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:19.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.689+0000 7f58432f9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583c0cb710 con 0x7f583c0cd3f0 2023-12-20T06:04:19.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.689+0000 7f58432f9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f583c06a7f0 con 0x7f583c1615d0 2023-12-20T06:04:19.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.689+0000 7f58432f9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f583c0cddd0 con 0x7f583c06ca40 2023-12-20T06:04:19.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.689+0000 7f5841095640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f583c06ca40 0x7f583c06ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:19.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.689+0000 7f5841896640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f583c0cd3f0 0x7f583c0c9800 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48944/0 (socket says 172.21.15.32:48944) 2023-12-20T06:04:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.689+0000 7f5841896640 1 -- 172.21.15.32:0/2587308060 learned_addr learned my addr 172.21.15.32:0/2587308060 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f5841095640 1 -- 172.21.15.32:0/2587308060 >> v1:172.21.15.39:6789/0 conn(0x7f583c1615d0 legacy=0x7f583c06ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f5841095640 1 -- 172.21.15.32:0/2587308060 >> v1:172.21.15.32:6789/0 conn(0x7f583c0cd3f0 legacy=0x7f583c0c9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.694 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f5841095640 1 -- 172.21.15.32:0/2587308060 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f583c069ec0 con 0x7f583c06ca40 2023-12-20T06:04:19.696 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f5841095640 1 --2- 172.21.15.32:0/2587308060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c06ca40 0x7f583c06ac80 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f58380097c0 tx=0x7f583801ec70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2c1b811ad33e44a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f5833fff640 1 -- 172.21.15.32:0/2587308060 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58380a8970 con 0x7f583c06ca40 2023-12-20T06:04:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f5833fff640 1 -- 172.21.15.32:0/2587308060 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58380a8ad0 con 0x7f583c06ca40 2023-12-20T06:04:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f5833fff640 1 -- 172.21.15.32:0/2587308060 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58380a8e60 con 0x7f583c06ca40 2023-12-20T06:04:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f58432f9640 1 -- 172.21.15.32:0/2587308060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c06ca40 msgr2=0x7f583c06ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f58432f9640 1 --2- 172.21.15.32:0/2587308060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c06ca40 0x7f583c06ac80 secure :-1 s=READY pgs=2490 cs=0 l=1 rev1=1 crypto rx=0x7f58380097c0 tx=0x7f583801ec70 comp rx=0 tx=0).stop 2023-12-20T06:04:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f58432f9640 1 -- 172.21.15.32:0/2587308060 shutdown_connections 2023-12-20T06:04:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f58432f9640 1 --2- 172.21.15.32:0/2587308060 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c06ca40 0x7f583c06ac80 unknown :-1 s=CLOSED pgs=2490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.693+0000 7f58432f9640 1 -- 172.21.15.32:0/2587308060 >> 172.21.15.32:0/2587308060 conn(0x7f583c0cf8e0 msgr2=0x7f583c0cbab0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f58432f9640 1 -- 172.21.15.32:0/2587308060 shutdown_connections 2023-12-20T06:04:19.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f58432f9640 1 -- 172.21.15.32:0/2587308060 wait complete. 2023-12-20T06:04:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f58432f9640 1 Processor -- start 2023-12-20T06:04:19.698 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f58432f9640 1 -- start start 2023-12-20T06:04:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f58432f9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f583c06ca40 0x7f583c1e7d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f58432f9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0cd3f0 0x7f583c0c8240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f58432f9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f583c0b4490 0x7f583c0c8780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f58432f9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c1e82d0 con 0x7f583c0cd3f0 2023-12-20T06:04:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f58432f9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f583c1e6a20 con 0x7f583c0b4490 2023-12-20T06:04:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f58432f9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f583c1e6d00 con 0x7f583c06ca40 2023-12-20T06:04:19.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5840894640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0cd3f0 0x7f583c0c8240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5840894640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0cd3f0 0x7f583c0c8240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36932/0 (socket says 172.21.15.32:36932) 2023-12-20T06:04:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5840894640 1 -- 172.21.15.32:0/3717440891 learned_addr learned my addr 172.21.15.32:0/3717440891 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5841095640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f583c06ca40 0x7f583c1e7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5841896640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f583c0b4490 0x7f583c0c8780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5840894640 1 -- 172.21.15.32:0/3717440891 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f583c06ca40 msgr2=0x7f583c1e7d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5840894640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f583c06ca40 0x7f583c1e7d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5840894640 1 -- 172.21.15.32:0/3717440891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f583c0b4490 msgr2=0x7f583c0c8780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5840894640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f583c0b4490 0x7f583c0c8780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5840894640 1 -- 172.21.15.32:0/3717440891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f5838009420 con 0x7f583c0cd3f0 2023-12-20T06:04:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5841896640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f583c0b4490 0x7f583c0c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:19.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5840894640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0cd3f0 0x7f583c0c8240 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f582c00c500 tx=0x7f582c00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5831ffb640 1 -- 172.21.15.32:0/3717440891 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c005220 con 0x7f583c0cd3f0 2023-12-20T06:04:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5831ffb640 1 -- 172.21.15.32:0/3717440891 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f582c005380 con 0x7f583c0cd3f0 2023-12-20T06:04:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f5831ffb640 1 -- 172.21.15.32:0/3717440891 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f582c007270 con 0x7f583c0cd3f0 2023-12-20T06:04:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.697+0000 7f58432f9640 1 -- 172.21.15.32:0/3717440891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f583c1e6fe0 con 0x7f583c0cd3f0 2023-12-20T06:04:19.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.701+0000 7f58432f9640 1 -- 172.21.15.32:0/3717440891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f583c1e75a0 con 0x7f583c0cd3f0 2023-12-20T06:04:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.701+0000 7f58432f9640 1 -- 172.21.15.32:0/3717440891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f583c1e7760 con 0x7f583c0cd3f0 2023-12-20T06:04:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.701+0000 7f5831ffb640 1 -- 172.21.15.32:0/3717440891 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f582c0073d0 con 0x7f583c0cd3f0 2023-12-20T06:04:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.701+0000 7f5831ffb640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5820065ca0 0x7f5820068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.701+0000 7f5831ffb640 1 -- 172.21.15.32:0/3717440891 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f582c082e40 con 0x7f583c0cd3f0 2023-12-20T06:04:19.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.701+0000 7f5841095640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5820065ca0 0x7f5820068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:19.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.705+0000 7f5841095640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5820065ca0 0x7f5820068160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f58380b2000 tx=0x7f5838008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:19.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.705+0000 7f5831ffb640 1 -- 172.21.15.32:0/3717440891 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f582c050470 con 0x7f583c0cd3f0 2023-12-20T06:04:19.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.917+0000 7f58432f9640 1 -- 172.21.15.32:0/3717440891 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]} v 0) v1 -- 0x7f583c06b010 con 0x7f583c0cd3f0 2023-12-20T06:04:19.924 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.921+0000 7f5831ffb640 1 -- 172.21.15.32:0/3717440891 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "auth caps", "entity": "client.0", "caps": ["mon", "allow r", "osd", "allow rw tag cephfs data=cephfs", "mds", "allow"]}]=0 updated caps for client.0 v55) v1 ==== 190+0+0 (secure 0 0 0) 0x7f582c053a90 con 0x7f583c0cd3f0 2023-12-20T06:04:19.924 INFO:teuthology.orchestra.run.smithi032.stderr:updated caps for client.0 2023-12-20T06:04:19.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 -- 172.21.15.32:0/3717440891 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5820065ca0 msgr2=0x7f5820068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5820065ca0 0x7f5820068160 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f58380b2000 tx=0x7f5838008040 comp rx=0 tx=0).stop 2023-12-20T06:04:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 -- 172.21.15.32:0/3717440891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0cd3f0 msgr2=0x7f583c0c8240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0cd3f0 0x7f583c0c8240 secure :-1 s=READY pgs=2491 cs=0 l=1 rev1=1 crypto rx=0x7f582c00c500 tx=0x7f582c00c9c0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 -- 172.21.15.32:0/3717440891 shutdown_connections 2023-12-20T06:04:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f5820065ca0 0x7f5820068160 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f583c0b4490 0x7f583c0c8780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f583c0cd3f0 0x7f583c0c8240 unknown :-1 s=CLOSED pgs=2491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 --2- 172.21.15.32:0/3717440891 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f583c06ca40 0x7f583c1e7d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 -- 172.21.15.32:0/3717440891 >> 172.21.15.32:0/3717440891 conn(0x7f583c0cf8e0 msgr2=0x7f583c06ce20 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:19.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 -- 172.21.15.32:0/3717440891 shutdown_connections 2023-12-20T06:04:19.928 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:19.925+0000 7f58432f9640 1 -- 172.21.15.32:0/3717440891 wait complete. 2023-12-20T06:04:19.946 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:04:20.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a4d398640 1 Processor -- start 2023-12-20T06:04:20.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a4d398640 1 -- start start 2023-12-20T06:04:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a4d398640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a4815d080 0x7f0a480cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a4d398640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a4815d970 con 0x7f0a4815de00 2023-12-20T06:04:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a4d398640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0a480ce180 con 0x7f0a48069d20 2023-12-20T06:04:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a4d398640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0a480ce740 con 0x7f0a4815d080 2023-12-20T06:04:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a467fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0a4815d080 0x7f0a480cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a477fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f0a4815de00 0x7f0a4806a100 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48946/0 (socket says 172.21.15.32:48946) 2023-12-20T06:04:20.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a477fe640 1 -- 172.21.15.32:0/4138429418 learned_addr learned my addr 172.21.15.32:0/4138429418 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a45ffb640 1 -- 172.21.15.32:0/4138429418 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3542327667 0 0) 0x7f0a4815d970 con 0x7f0a4815de00 2023-12-20T06:04:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a45ffb640 1 -- 172.21.15.32:0/4138429418 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f0a30003530 con 0x7f0a4815de00 2023-12-20T06:04:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a467fc640 1 -- 172.21.15.32:0/4138429418 >> v1:172.21.15.39:6789/0 conn(0x7f0a48069d20 legacy=0x7f0a480cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a467fc640 1 -- 172.21.15.32:0/4138429418 >> v1:172.21.15.32:6789/0 conn(0x7f0a4815de00 legacy=0x7f0a4806a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.089+0000 7f0a467fc640 1 -- 172.21.15.32:0/4138429418 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a480ce460 con 0x7f0a4815d080 2023-12-20T06:04:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.093+0000 7f0a467fc640 1 --2- 172.21.15.32:0/4138429418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4815d080 0x7f0a480cbbb0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0097c0 tx=0x7f0a3c01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2df6de589d4f982 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.093+0000 7f0a45ffb640 1 -- 172.21.15.32:0/4138429418 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c0ab670 con 0x7f0a4815d080 2023-12-20T06:04:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.093+0000 7f0a45ffb640 1 -- 172.21.15.32:0/4138429418 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a3c0ab7d0 con 0x7f0a4815d080 2023-12-20T06:04:20.096 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.093+0000 7f0a45ffb640 1 -- 172.21.15.32:0/4138429418 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a3c0abb60 con 0x7f0a4815d080 2023-12-20T06:04:20.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.093+0000 7f0a4d398640 1 -- 172.21.15.32:0/4138429418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4815d080 msgr2=0x7f0a480cbbb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.093+0000 7f0a4d398640 1 --2- 172.21.15.32:0/4138429418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4815d080 0x7f0a480cbbb0 secure :-1 s=READY pgs=2493 cs=0 l=1 rev1=1 crypto rx=0x7f0a3c0097c0 tx=0x7f0a3c01c450 comp rx=0 tx=0).stop 2023-12-20T06:04:20.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.093+0000 7f0a4d398640 1 -- 172.21.15.32:0/4138429418 shutdown_connections 2023-12-20T06:04:20.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.093+0000 7f0a4d398640 1 --2- 172.21.15.32:0/4138429418 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4815d080 0x7f0a480cbbb0 unknown :-1 s=CLOSED pgs=2493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.093+0000 7f0a4d398640 1 -- 172.21.15.32:0/4138429418 >> 172.21.15.32:0/4138429418 conn(0x7f0a480cc310 msgr2=0x7f0a480cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:20.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.093+0000 7f0a4d398640 1 -- 172.21.15.32:0/4138429418 shutdown_connections 2023-12-20T06:04:20.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.093+0000 7f0a4d398640 1 -- 172.21.15.32:0/4138429418 wait complete. 2023-12-20T06:04:20.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a4d398640 1 Processor -- start 2023-12-20T06:04:20.098 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a4d398640 1 -- start start 2023-12-20T06:04:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a4d398640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a48069d20 0x7f0a480c5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a4d398640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4815d080 0x7f0a480b3470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a4d398640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4815de00 0x7f0a480b39b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a4d398640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a480ac130 con 0x7f0a4815de00 2023-12-20T06:04:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a4d398640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0a480abe50 con 0x7f0a4815d080 2023-12-20T06:04:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a4d398640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0a48151800 con 0x7f0a48069d20 2023-12-20T06:04:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4815de00 0x7f0a480b39b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.101 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4815de00 0x7f0a480b39b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36944/0 (socket says 172.21.15.32:36944) 2023-12-20T06:04:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a477fe640 1 -- 172.21.15.32:0/3703230895 learned_addr learned my addr 172.21.15.32:0/3703230895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a46ffd640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a48069d20 0x7f0a480c5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a477fe640 1 -- 172.21.15.32:0/3703230895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a48069d20 msgr2=0x7f0a480c5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a467fc640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4815d080 0x7f0a480b3470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a477fe640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a48069d20 0x7f0a480c5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a477fe640 1 -- 172.21.15.32:0/3703230895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4815d080 msgr2=0x7f0a480b3470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a477fe640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4815d080 0x7f0a480b3470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a477fe640 1 -- 172.21.15.32:0/3703230895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0a3c009420 con 0x7f0a4815de00 2023-12-20T06:04:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a46ffd640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a48069d20 0x7f0a480c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:20.102 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a477fe640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4815de00 0x7f0a480b39b0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000cb40 tx=0x7f0a4000a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a2f7fe640 1 -- 172.21.15.32:0/3703230895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a4000aca0 con 0x7f0a4815de00 2023-12-20T06:04:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a2f7fe640 1 -- 172.21.15.32:0/3703230895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0a4000ae00 con 0x7f0a4815de00 2023-12-20T06:04:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a2f7fe640 1 -- 172.21.15.32:0/3703230895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0a40012620 con 0x7f0a4815de00 2023-12-20T06:04:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a4d398640 1 -- 172.21.15.32:0/3703230895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0a48151ae0 con 0x7f0a4815de00 2023-12-20T06:04:20.103 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.097+0000 7f0a4d398640 1 -- 172.21.15.32:0/3703230895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0a480c6220 con 0x7f0a4815de00 2023-12-20T06:04:20.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.101+0000 7f0a4d398640 1 -- 172.21.15.32:0/3703230895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0a480c9d30 con 0x7f0a4815de00 2023-12-20T06:04:20.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.101+0000 7f0a2f7fe640 1 -- 172.21.15.32:0/3703230895 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f0a40007070 con 0x7f0a4815de00 2023-12-20T06:04:20.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.101+0000 7f0a2f7fe640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a34065ca0 0x7f0a34068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.101+0000 7f0a2f7fe640 1 -- 172.21.15.32:0/3703230895 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f0a400836f0 con 0x7f0a4815de00 2023-12-20T06:04:20.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.101+0000 7f0a46ffd640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a34065ca0 0x7f0a34068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.101+0000 7f0a46ffd640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a34065ca0 0x7f0a34068160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a480b5420 tx=0x7f0a38009040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:20.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.105+0000 7f0a2f7fe640 1 -- 172.21.15.32:0/3703230895 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0a40050c70 con 0x7f0a4815de00 2023-12-20T06:04:20.297 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 -- 172.21.15.32:0/3703230895 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0a48151660 con 0x7f0a4815de00 2023-12-20T06:04:20.298 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a2f7fe640 1 -- 172.21.15.32:0/3703230895 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4722 (secure 0 0 0) 0x7f0a48151660 con 0x7f0a4815de00 2023-12-20T06:04:20.298 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:20.299 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3779641739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3779641739},{"type":"v1","addr":"172.21.15.39:6834","nonce":3779641739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/473427978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":473427978},{"type":"v1","addr":"172.21.15.39:6835","nonce":473427978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12060,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3927543497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3927543497},{"type":"v1","addr":"172.21.15.32:6836","nonce":3927543497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:04:14.992681+0000","modified":"2023-12-20T06:04:18.007320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12063":{"gid":12063,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3133738681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3133738681},{"type":"v1","addr":"172.21.15.32:6837","nonce":3133738681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-20T06:04:20.299 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 128 2023-12-20T06:04:20.300 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 -- 172.21.15.32:0/3703230895 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a34065ca0 msgr2=0x7f0a34068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a34065ca0 0x7f0a34068160 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f0a480b5420 tx=0x7f0a38009040 comp rx=0 tx=0).stop 2023-12-20T06:04:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 -- 172.21.15.32:0/3703230895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4815de00 msgr2=0x7f0a480b39b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4815de00 0x7f0a480b39b0 secure :-1 s=READY pgs=2494 cs=0 l=1 rev1=1 crypto rx=0x7f0a4000cb40 tx=0x7f0a4000a060 comp rx=0 tx=0).stop 2023-12-20T06:04:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 -- 172.21.15.32:0/3703230895 shutdown_connections 2023-12-20T06:04:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f0a34065ca0 0x7f0a34068160 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0a4815de00 0x7f0a480b39b0 unknown :-1 s=CLOSED pgs=2494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0a4815d080 0x7f0a480b3470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 --2- 172.21.15.32:0/3703230895 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0a48069d20 0x7f0a480c5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 -- 172.21.15.32:0/3703230895 >> 172.21.15.32:0/3703230895 conn(0x7f0a480cc310 msgr2=0x7f0a4815f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 -- 172.21.15.32:0/3703230895 shutdown_connections 2023-12-20T06:04:20.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.297+0000 7f0a4d398640 1 -- 172.21.15.32:0/3703230895 wait complete. 2023-12-20T06:04:20.556 INFO:tasks.ceph.mds.a.smithi032.stderr:2023-12-20T06:04:20.557+0000 7f007ed58640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T06:04:20.723 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 128, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2023-12-20T06:04:14.992681+0000', 'modified': '2023-12-20T06:04:18.007320+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 12063}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_12063': {'gid': 12063, 'name': 'a', 'rank': 0, 'incarnation': 124, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.32:6837/3133738681', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.32:6835', 'nonce': 3133738681}, {'type': 'v1', 'addr': '172.21.15.32:6837', 'nonce': 3133738681}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540138322906710015, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2'}}}}, 'data_pools': [33], 'metadata_pool': 32, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0} 2023-12-20T06:04:20.724 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2023-12-20T06:04:20.724 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.a status 2023-12-20T06:04:20.870 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f4696d640 1 Processor -- start 2023-12-20T06:04:20.871 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f4696d640 1 -- start start 2023-12-20T06:04:20.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f4696d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f401599e0 0x7f9f400aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f4696d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9f400b0d00 0x7f9f400701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f4696d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9f400c98c0 con 0x7f9f400c90e0 2023-12-20T06:04:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f4696d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f400c9ba0 con 0x7f9f401599e0 2023-12-20T06:04:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f4696d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9f400c9e80 con 0x7f9f400b0d00 2023-12-20T06:04:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f3ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9f401599e0 0x7f9f400aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f3f7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9f400c90e0 0x7f9f400b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48950/0 (socket says 172.21.15.32:48950) 2023-12-20T06:04:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f3f7fe640 1 -- 172.21.15.32:0/2616145437 learned_addr learned my addr 172.21.15.32:0/2616145437 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:20.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f44f0a640 1 --2- 172.21.15.32:0/2616145437 >> v2:172.21.15.39:3300/0 conn(0x7f9f400b0d00 0x7f9f400701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f3ffff640 1 -- 172.21.15.32:0/2616145437 >> v2:172.21.15.39:3300/0 conn(0x7f9f400b0d00 msgr2=0x7f9f400701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/2616145437 >> v2:172.21.15.39:3300/0 conn(0x7f9f400b0d00 0x7f9f400701c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f3ffff640 1 -- 172.21.15.32:0/2616145437 >> v1:172.21.15.32:6789/0 conn(0x7f9f400c90e0 legacy=0x7f9f400b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f3ffff640 1 -- 172.21.15.32:0/2616145437 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f400ca160 con 0x7f9f401599e0 2023-12-20T06:04:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f3effd640 1 -- 172.21.15.32:0/2616145437 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3658315260 0 0) 0x7f9f400c98c0 con 0x7f9f400c90e0 2023-12-20T06:04:20.874 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.869+0000 7f9f44f0a640 1 --2- 172.21.15.32:0/2616145437 >> v2:172.21.15.39:3300/0 conn(0x7f9f400b0d00 0x7f9f400701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/2616145437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f401599e0 0x7f9f400aece0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f9f400b0060 tx=0x7f9f3001c410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffdeb5786d40f6ed server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:20.876 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f3effd640 1 -- 172.21.15.32:0/2616145437 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f300ab600 con 0x7f9f401599e0 2023-12-20T06:04:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f3effd640 1 -- 172.21.15.32:0/2616145437 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f300a7050 con 0x7f9f401599e0 2023-12-20T06:04:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f3effd640 1 -- 172.21.15.32:0/2616145437 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f300ab980 con 0x7f9f401599e0 2023-12-20T06:04:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f4696d640 1 -- 172.21.15.32:0/2616145437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f401599e0 msgr2=0x7f9f400aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f4696d640 1 --2- 172.21.15.32:0/2616145437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f401599e0 0x7f9f400aece0 secure :-1 s=READY pgs=2496 cs=0 l=1 rev1=1 crypto rx=0x7f9f400b0060 tx=0x7f9f3001c410 comp rx=0 tx=0).stop 2023-12-20T06:04:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f4696d640 1 -- 172.21.15.32:0/2616145437 shutdown_connections 2023-12-20T06:04:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f4696d640 1 --2- 172.21.15.32:0/2616145437 >> v2:172.21.15.39:3300/0 conn(0x7f9f400b0d00 0x7f9f400701c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f4696d640 1 --2- 172.21.15.32:0/2616145437 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f401599e0 0x7f9f400aece0 unknown :-1 s=CLOSED pgs=2496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.877 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f4696d640 1 -- 172.21.15.32:0/2616145437 >> 172.21.15.32:0/2616145437 conn(0x7f9f4006a320 msgr2=0x7f9f400ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f4696d640 1 -- 172.21.15.32:0/2616145437 shutdown_connections 2023-12-20T06:04:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.873+0000 7f9f4696d640 1 -- 172.21.15.32:0/2616145437 wait complete. 2023-12-20T06:04:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f4696d640 1 Processor -- start 2023-12-20T06:04:20.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f4696d640 1 -- start start 2023-12-20T06:04:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f4696d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9f400b0d00 0x7f9f400ce090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f4696d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f400c90e0 0x7f9f40154b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f4696d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f401599e0 0x7f9f4014b940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f4696d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400c7450 con 0x7f9f400c90e0 2023-12-20T06:04:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f4696d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400c7730 con 0x7f9f401599e0 2023-12-20T06:04:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f4696d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9f400c7a10 con 0x7f9f400b0d00 2023-12-20T06:04:20.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f400c90e0 0x7f9f40154b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f400c90e0 0x7f9f40154b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36956/0 (socket says 172.21.15.32:36956) 2023-12-20T06:04:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3f7fe640 1 -- 172.21.15.32:0/2745563404 learned_addr learned my addr 172.21.15.32:0/2745563404 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9f400b0d00 0x7f9f400ce090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f44f0a640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f401599e0 0x7f9f4014b940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3f7fe640 1 -- 172.21.15.32:0/2745563404 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9f400b0d00 msgr2=0x7f9f400ce090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3f7fe640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9f400b0d00 0x7f9f400ce090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3f7fe640 1 -- 172.21.15.32:0/2745563404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f401599e0 msgr2=0x7f9f4014b940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3f7fe640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f401599e0 0x7f9f4014b940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3f7fe640 1 -- 172.21.15.32:0/2745563404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9f300093b0 con 0x7f9f400c90e0 2023-12-20T06:04:20.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f44f0a640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f401599e0 0x7f9f4014b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9f400b0d00 0x7f9f400ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3f7fe640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f400c90e0 0x7f9f40154b50 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f9f340011f0 tx=0x7f9f3400c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3cff9640 1 -- 172.21.15.32:0/2745563404 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f340074e0 con 0x7f9f400c90e0 2023-12-20T06:04:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3cff9640 1 -- 172.21.15.32:0/2745563404 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f34007640 con 0x7f9f400c90e0 2023-12-20T06:04:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f3cff9640 1 -- 172.21.15.32:0/2745563404 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f34004a10 con 0x7f9f400c90e0 2023-12-20T06:04:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f4696d640 1 -- 172.21.15.32:0/2745563404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9f400c7cf0 con 0x7f9f400c90e0 2023-12-20T06:04:20.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.877+0000 7f9f4696d640 1 -- 172.21.15.32:0/2745563404 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f400ce5d0 con 0x7f9f400c90e0 2023-12-20T06:04:20.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.881+0000 7f9f3cff9640 1 -- 172.21.15.32:0/2745563404 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f9f34004b70 con 0x7f9f400c90e0 2023-12-20T06:04:20.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.881+0000 7f9f3cff9640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9f2c065ca0 0x7f9f2c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.881+0000 7f9f3cff9640 1 -- 172.21.15.32:0/2745563404 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9f34082970 con 0x7f9f400c90e0 2023-12-20T06:04:20.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.881+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9f2c065ca0 0x7f9f2c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.881+0000 7f9f49cbe000 1 Processor -- start 2023-12-20T06:04:20.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 -- start start 2023-12-20T06:04:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9f2c065ca0 0x7f9f2c068160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f300ad000 tx=0x7f9f300a6000 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904af9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904af9990 0x563904aaec80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904aaf1c0 0x563904aaf5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639048d2c90 con 0x563904af9070 2023-12-20T06:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563904904220 con 0x563904aaf1c0 2023-12-20T06:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639049017f0 con 0x563904af9990 2023-12-20T06:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f3ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904af9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f3ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904af9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36968/0 (socket says 172.21.15.32:36968) 2023-12-20T06:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f3ffff640 1 -- 172.21.15.32:0/1739531795 learned_addr learned my addr 172.21.15.32:0/1739531795 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f3f7fe640 1 --2- 172.21.15.32:0/1739531795 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904af9990 0x563904aaec80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f44f0a640 1 --2- 172.21.15.32:0/1739531795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904aaf1c0 0x563904aaf5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.889 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f3ffff640 1 -- 172.21.15.32:0/1739531795 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904af9990 msgr2=0x563904aaec80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/1739531795 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904af9990 0x563904aaec80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f3ffff640 1 -- 172.21.15.32:0/1739531795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904aaf1c0 msgr2=0x563904aaf5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/1739531795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904aaf1c0 0x563904aaf5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f3ffff640 1 -- 172.21.15.32:0/1739531795 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639045f9890 con 0x563904af9070 2023-12-20T06:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/1739531795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904af9450 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f9f3001c450 tx=0x7f9f30004740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d8581e73a131e7b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f25ffb640 1 -- 172.21.15.32:0/1739531795 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f300ab400 con 0x563904af9070 2023-12-20T06:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f25ffb640 1 -- 172.21.15.32:0/1739531795 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f300c3650 con 0x563904af9070 2023-12-20T06:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f25ffb640 1 -- 172.21.15.32:0/1739531795 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f300c39c0 con 0x563904af9070 2023-12-20T06:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 -- 172.21.15.32:0/1739531795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 msgr2=0x563904af9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.890 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/1739531795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904af9450 secure :-1 s=READY pgs=2498 cs=0 l=1 rev1=1 crypto rx=0x7f9f3001c450 tx=0x7f9f30004740 comp rx=0 tx=0).stop 2023-12-20T06:04:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 -- 172.21.15.32:0/1739531795 shutdown_connections 2023-12-20T06:04:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/1739531795 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904aaf1c0 0x563904aaf5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/1739531795 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904af9990 0x563904aaec80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/1739531795 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904af9450 unknown :-1 s=CLOSED pgs=2498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 -- 172.21.15.32:0/1739531795 >> 172.21.15.32:0/1739531795 conn(0x563904ab5950 msgr2=0x563904aad560 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 -- 172.21.15.32:0/1739531795 shutdown_connections 2023-12-20T06:04:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 -- 172.21.15.32:0/1739531795 wait complete. 2023-12-20T06:04:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 -- start start 2023-12-20T06:04:20.891 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.885+0000 7f9f49cbe000 1 Processor -- start 2023-12-20T06:04:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 0x563904ab4750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9070 0x563904abb080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9990 0x563904abb5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639048fd0e0 con 0x563904af9990 2023-12-20T06:04:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x56390489c310 con 0x563904af9070 2023-12-20T06:04:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639048ba770 con 0x563904aaf1c0 2023-12-20T06:04:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 10 client.0 _ll_register_callbacks cb 0x563904a9ab60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:04:20.896 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f44f0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9990 0x563904abb5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f44f0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9990 0x563904abb5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36980/0 (socket says 172.21.15.32:36980) 2023-12-20T06:04:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f44f0a640 1 -- 172.21.15.32:0/4023152285 learned_addr learned my addr 172.21.15.32:0/4023152285 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f3f7fe640 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9070 0x563904abb080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 0x563904ab4750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f44f0a640 1 -- 172.21.15.32:0/4023152285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 msgr2=0x563904ab4750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f44f0a640 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 0x563904ab4750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f44f0a640 1 -- 172.21.15.32:0/4023152285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9070 msgr2=0x563904abb080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f44f0a640 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9070 0x563904abb080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f44f0a640 1 -- 172.21.15.32:0/4023152285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f400ca160 con 0x563904af9990 2023-12-20T06:04:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f44f0a640 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9990 0x563904abb5c0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f9f3800fbf0 tx=0x7f9f3800d370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9de848238211f5a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:20.897 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f1effd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f1effd640 1 -- 172.21.15.32:0/4023152285 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f380193f0 con 0x563904af9990 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 1 -- 172.21.15.32:0/4023152285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639048ccc10 con 0x563904af9990 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f1effd640 1 -- 172.21.15.32:0/4023152285 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9f38019550 con 0x563904af9990 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f1effd640 1 -- 172.21.15.32:0/4023152285 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f38007b20 con 0x563904af9990 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 10 client.12204 fetch_fsmap learned FSMap version 128 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 1 -- 172.21.15.32:0/4023152285 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x5639048c2260 con 0x563904af9990 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f1effd640 1 -- 172.21.15.32:0/4023152285 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639048ccc10 con 0x563904af9990 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f1effd640 1 -- 172.21.15.32:0/4023152285 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 128) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9f38019710 con 0x563904af9990 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 10 client.12204 fetch_fsmap finished waiting for FSMap version 128 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 10 client.12204 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 20 client.12204 populate_metadata read hostname 'smithi032' 2023-12-20T06:04:20.898 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.32:6835/3133738681,v1:172.21.15.32:6837/3133738681] conn(0x563904b008d0 0x563904b00cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 4 client.12204 mds_command: new command op to 12063 tid=0[{"prefix": "get_command_descriptions"}] 2023-12-20T06:04:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f49cbe000 1 -- 172.21.15.32:0/4023152285 --> [v2:172.21.15.32:6835/3133738681,v1:172.21.15.32:6837/3133738681] -- command(tid 0: {"prefix": "get_command_descriptions"}) v1 -- 0x5639048fba30 con 0x563904b008d0 2023-12-20T06:04:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.32:6835/3133738681,v1:172.21.15.32:6837/3133738681] conn(0x563904b008d0 0x563904b00cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.32:6835/3133738681,v1:172.21.15.32:6837/3133738681] conn(0x563904b008d0 0x563904b00cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=116af8c546394d1c server_cookie=f571bc1da62746ed in_seq=0 out_seq=0 2023-12-20T06:04:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.889+0000 7f9f1effd640 10 client.12204 ms_handle_connect on v2:172.21.15.32:6835/3133738681 2023-12-20T06:04:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f1effd640 1 -- 172.21.15.32:0/4023152285 <== mds.0 v2:172.21.15.32:6835/3133738681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+17819 (crc 0 0 0) 0x5639048fba30 con 0x563904b008d0 2023-12-20T06:04:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f1effd640 10 client.12204 handle_command_reply: tid=0 2023-12-20T06:04:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 1 client.12204 shutdown 2023-12-20T06:04:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 1 -- 172.21.15.32:0/4023152285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9990 msgr2=0x563904abb5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9990 0x563904abb5c0 secure :-1 s=READY pgs=2499 cs=0 l=1 rev1=1 crypto rx=0x7f9f3800fbf0 tx=0x7f9f3800d370 comp rx=0 tx=0).stop 2023-12-20T06:04:20.899 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 1 -- 172.21.15.32:0/4023152285 shutdown_connections 2023-12-20T06:04:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.32:6835/3133738681,v1:172.21.15.32:6837/3133738681] conn(0x563904b008d0 0x563904b00cb0 crc :-1 s=READY pgs=12 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9990 0x563904abb5c0 unknown :-1 s=CLOSED pgs=2499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9070 0x563904abb080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/4023152285 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 0x563904ab4750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f1effd640 0 client.12204 ms_handle_reset on v2:172.21.15.32:6835/3133738681 2023-12-20T06:04:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 1 -- 172.21.15.32:0/4023152285 >> 172.21.15.32:0/4023152285 conn(0x563904ab5950 msgr2=0x563904aadc60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 1 -- 172.21.15.32:0/4023152285 shutdown_connections 2023-12-20T06:04:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 1 -- 172.21.15.32:0/4023152285 wait complete. 2023-12-20T06:04:20.900 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.893+0000 7f9f49cbe000 20 client.12204 trim_cache size 0 max 16384 2023-12-20T06:04:20.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.905+0000 7f9f49cbe000 1 Processor -- start 2023-12-20T06:04:20.909 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 -- start start 2023-12-20T06:04:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 0x563904b50ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904b51410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.911 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9990 0x563904b51950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639045ba940 con 0x563904af9070 2023-12-20T06:04:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639048ffc40 con 0x563904af9990 2023-12-20T06:04:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639048fa4e0 con 0x563904aaf1c0 2023-12-20T06:04:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f3f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904b51410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f3f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904b51410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36988/0 (socket says 172.21.15.32:36988) 2023-12-20T06:04:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f3f7fe640 1 -- 172.21.15.32:0/505626315 learned_addr learned my addr 172.21.15.32:0/505626315 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f3f7fe640 1 -- 172.21.15.32:0/505626315 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 msgr2=0x563904b50ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f3f7fe640 1 --2- 172.21.15.32:0/505626315 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 0x563904b50ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f3f7fe640 1 -- 172.21.15.32:0/505626315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9990 msgr2=0x563904b51950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f3f7fe640 1 --2- 172.21.15.32:0/505626315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9990 0x563904b51950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.912 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f3f7fe640 1 -- 172.21.15.32:0/505626315 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x5639048a83b0 con 0x563904af9070 2023-12-20T06:04:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f3f7fe640 1 --2- 172.21.15.32:0/505626315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904b51410 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f9f340538b0 tx=0x7f9f340543c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f16afa1986eb21a4 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f267fc640 1 -- 172.21.15.32:0/505626315 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3406b1c0 con 0x563904af9070 2023-12-20T06:04:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f267fc640 1 -- 172.21.15.32:0/505626315 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9f340502c0 con 0x563904af9070 2023-12-20T06:04:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f267fc640 1 -- 172.21.15.32:0/505626315 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f3405b920 con 0x563904af9070 2023-12-20T06:04:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 -- 172.21.15.32:0/505626315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 msgr2=0x563904b51410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/505626315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904b51410 secure :-1 s=READY pgs=2500 cs=0 l=1 rev1=1 crypto rx=0x7f9f340538b0 tx=0x7f9f340543c0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 -- 172.21.15.32:0/505626315 shutdown_connections 2023-12-20T06:04:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/505626315 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9990 0x563904b51950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/505626315 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904af9070 0x563904b51410 unknown :-1 s=CLOSED pgs=2500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.913 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/505626315 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 0x563904b50ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 -- 172.21.15.32:0/505626315 >> 172.21.15.32:0/505626315 conn(0x563904ab5950 msgr2=0x563904abaf40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 -- 172.21.15.32:0/505626315 shutdown_connections 2023-12-20T06:04:20.914 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 -- 172.21.15.32:0/505626315 wait complete. 2023-12-20T06:04:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 -- start start 2023-12-20T06:04:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.909+0000 7f9f49cbe000 1 Processor -- start 2023-12-20T06:04:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 0x563904b3a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9990 0x563904b3a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.918 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904b3abd0 0x563904b3afb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x5639048fbf70 con 0x563904b3abd0 2023-12-20T06:04:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x563904ab1d70 con 0x563904af9990 2023-12-20T06:04:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x5639048a5860 con 0x563904aaf1c0 2023-12-20T06:04:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 10 client.0 _ll_register_callbacks cb 0x5639048ab640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2023-12-20T06:04:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f44f0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904b3abd0 0x563904b3afb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f44f0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904b3abd0 0x563904b3afb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:36996/0 (socket says 172.21.15.32:36996) 2023-12-20T06:04:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f44f0a640 1 -- 172.21.15.32:0/835470777 learned_addr learned my addr 172.21.15.32:0/835470777 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f3f7fe640 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9990 0x563904b3a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 0x563904b3a150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.919 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f44f0a640 1 -- 172.21.15.32:0/835470777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 msgr2=0x563904b3a150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f44f0a640 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 0x563904b3a150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f44f0a640 1 -- 172.21.15.32:0/835470777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9990 msgr2=0x563904b3a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f44f0a640 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9990 0x563904b3a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f44f0a640 1 -- 172.21.15.32:0/835470777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9f400ce5d0 con 0x563904b3abd0 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f44f0a640 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904b3abd0 0x563904b3afb0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f9f38002240 tx=0x7f9f38004ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95e6db3876ebad23 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f1effd640 10 client.0 ms_handle_connect on v2:172.21.15.32:3300/0 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f1effd640 1 -- 172.21.15.32:0/835470777 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9f380195f0 con 0x563904b3abd0 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f1effd640 1 -- 172.21.15.32:0/835470777 <== mon.0 v2:172.21.15.32:3300/0 2 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9f38007570 con 0x563904b3abd0 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 -- 172.21.15.32:0/835470777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=fsmap handle=1) v1 -- 0x5639048c2e00 con 0x563904b3abd0 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f1effd640 1 -- 172.21.15.32:0/835470777 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_get_version_reply(handle=1 version=128) v2 ==== 24+0+0 (secure 0 0 0) 0x7f9f3800d870 con 0x563904b3abd0 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 10 client.12210 fetch_fsmap learned FSMap version 128 2023-12-20T06:04:20.920 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 -- 172.21.15.32:0/835470777 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,fsmap=128,monmap=2+}) v3 -- 0x5639045f9cc0 con 0x563904b3abd0 2023-12-20T06:04:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f1effd640 1 -- 172.21.15.32:0/835470777 <== mon.0 v2:172.21.15.32:3300/0 4 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x5639048c2e00 con 0x563904b3abd0 2023-12-20T06:04:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f1effd640 1 -- 172.21.15.32:0/835470777 <== mon.0 v2:172.21.15.32:3300/0 5 ==== fsmap(e 128) v1 ==== 3114+0+0 (secure 0 0 0) 0x7f9f38005d20 con 0x563904b3abd0 2023-12-20T06:04:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 10 client.12210 fetch_fsmap finished waiting for FSMap version 128 2023-12-20T06:04:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 10 client.12210 resolve_mds: resolved name 'a' to daemon mds.a 2023-12-20T06:04:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 20 client.12210 populate_metadata read hostname 'smithi032' 2023-12-20T06:04:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.32:6835/3133738681,v1:172.21.15.32:6837/3133738681] conn(0x563904b5c160 0x563904b5e550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 4 client.12210 mds_command: new command op to 12063 tid=0[{"prefix": "status"}] 2023-12-20T06:04:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 -- 172.21.15.32:0/835470777 --> [v2:172.21.15.32:6835/3133738681,v1:172.21.15.32:6837/3133738681] -- command(tid 0: {"prefix": "status"}) v1 -- 0x5639048f6160 con 0x563904b5c160 2023-12-20T06:04:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.32:6835/3133738681,v1:172.21.15.32:6837/3133738681] conn(0x563904b5c160 0x563904b5e550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:20.921 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f3ffff640 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.32:6835/3133738681,v1:172.21.15.32:6837/3133738681] conn(0x563904b5c160 0x563904b5e550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23d2d4553d99879d server_cookie=ce4a7ecc406e6f0f in_seq=0 out_seq=0 2023-12-20T06:04:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f1effd640 10 client.12210 ms_handle_connect on v2:172.21.15.32:6835/3133738681 2023-12-20T06:04:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f1effd640 1 -- 172.21.15.32:0/835470777 <== mds.0 v2:172.21.15.32:6835/3133738681 1 ==== command_reply(tid 0: 0 ) v1 ==== 8+0+329 (crc 0 0 0) 0x5639048f6160 con 0x563904b5c160 2023-12-20T06:04:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f1effd640 10 client.12210 handle_command_reply: tid=0 2023-12-20T06:04:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 client.12210 shutdown 2023-12-20T06:04:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 -- 172.21.15.32:0/835470777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904b3abd0 msgr2=0x563904b3afb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904b3abd0 0x563904b3afb0 secure :-1 s=READY pgs=2501 cs=0 l=1 rev1=1 crypto rx=0x7f9f38002240 tx=0x7f9f38004ac0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 -- 172.21.15.32:0/835470777 shutdown_connections 2023-12-20T06:04:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.32:6835/3133738681,v1:172.21.15.32:6837/3133738681] conn(0x563904b5c160 0x563904b5e550 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x563904b3abd0 0x563904b3afb0 unknown :-1 s=CLOSED pgs=2501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x563904af9990 0x563904b3a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.922 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 --2- 172.21.15.32:0/835470777 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x563904aaf1c0 0x563904b3a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f1effd640 0 client.12210 ms_handle_reset on v2:172.21.15.32:6835/3133738681 2023-12-20T06:04:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 -- 172.21.15.32:0/835470777 >> 172.21.15.32:0/835470777 conn(0x563904ab5950 msgr2=0x563904af6030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 -- 172.21.15.32:0/835470777 shutdown_connections 2023-12-20T06:04:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.913+0000 7f9f49cbe000 1 -- 172.21.15.32:0/835470777 wait complete. 2023-12-20T06:04:20.923 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f49cbe000 20 client.12210 trim_cache size 0 max 16384 2023-12-20T06:04:20.923 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:04:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "cluster_fsid": "9b496156-2b6f-4897-9af4-fbc686ad55e7", 2023-12-20T06:04:20.923 INFO:teuthology.orchestra.run.smithi032.stdout: "whoami": 0, 2023-12-20T06:04:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "id": 12063, 2023-12-20T06:04:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "want_state": "up:active", 2023-12-20T06:04:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "state": "up:active", 2023-12-20T06:04:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "fs_name": "cephfs", 2023-12-20T06:04:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "rank_uptime": 5.9235677219999996, 2023-12-20T06:04:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "mdsmap_epoch": 128, 2023-12-20T06:04:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch": 251, 2023-12-20T06:04:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "osdmap_epoch_barrier": 251, 2023-12-20T06:04:20.924 INFO:teuthology.orchestra.run.smithi032.stdout: "uptime": 11.156798508 2023-12-20T06:04:20.924 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:04:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 -- 172.21.15.32:0/2745563404 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9f2c065ca0 msgr2=0x7f9f2c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9f2c065ca0 0x7f9f2c068160 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f9f300ad000 tx=0x7f9f300a6000 comp rx=0 tx=0).stop 2023-12-20T06:04:20.925 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 -- 172.21.15.32:0/2745563404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f400c90e0 msgr2=0x7f9f40154b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f400c90e0 0x7f9f40154b50 secure :-1 s=READY pgs=2497 cs=0 l=1 rev1=1 crypto rx=0x7f9f340011f0 tx=0x7f9f3400c9b0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 -- 172.21.15.32:0/2745563404 shutdown_connections 2023-12-20T06:04:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9f2c065ca0 0x7f9f2c068160 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9f401599e0 0x7f9f4014b940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9f400c90e0 0x7f9f40154b50 unknown :-1 s=CLOSED pgs=2497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 --2- 172.21.15.32:0/2745563404 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9f400b0d00 0x7f9f400ce090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 -- 172.21.15.32:0/2745563404 >> 172.21.15.32:0/2745563404 conn(0x7f9f4006a320 msgr2=0x7f9f4014fb90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 -- 172.21.15.32:0/2745563404 shutdown_connections 2023-12-20T06:04:20.926 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:20.917+0000 7f9f4696d640 1 -- 172.21.15.32:0/2745563404 wait complete. 2023-12-20T06:04:20.936 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs ls 2023-12-20T06:04:21.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.077+0000 7f196d55a640 1 Processor -- start 2023-12-20T06:04:21.080 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.077+0000 7f196d55a640 1 -- start start 2023-12-20T06:04:21.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196d55a640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f19680ca5f0 0x7f19680c8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:21.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196d55a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19680b0700 0x7f19680cc1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:21.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196d55a640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1968071ea0 con 0x7f19680ca210 2023-12-20T06:04:21.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196d55a640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1968071290 con 0x7f19680b0700 2023-12-20T06:04:21.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196d55a640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1968071570 con 0x7f19680ca5f0 2023-12-20T06:04:21.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196cd59640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19680b0700 0x7f19680cc1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:21.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196cd59640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f19680b0700 0x7f19680cc1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37008/0 (socket says 172.21.15.32:37008) 2023-12-20T06:04:21.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196cd59640 1 -- 172.21.15.32:0/1903129027 learned_addr learned my addr 172.21.15.32:0/1903129027 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:21.083 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f19677fe640 1 --2- 172.21.15.32:0/1903129027 >> v2:172.21.15.39:3300/0 conn(0x7f19680ca5f0 0x7f19680c8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:21.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196cd59640 1 -- 172.21.15.32:0/1903129027 >> v2:172.21.15.39:3300/0 conn(0x7f19680ca5f0 msgr2=0x7f19680c8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:21.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196cd59640 1 --2- 172.21.15.32:0/1903129027 >> v2:172.21.15.39:3300/0 conn(0x7f19680ca5f0 0x7f19680c8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:21.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196cd59640 1 -- 172.21.15.32:0/1903129027 >> v1:172.21.15.39:6789/0 conn(0x7f19680ca210 legacy=0x7f19680afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:21.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f196cd59640 1 -- 172.21.15.32:0/1903129027 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1968071850 con 0x7f19680b0700 2023-12-20T06:04:21.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.081+0000 7f19677fe640 1 --2- 172.21.15.32:0/1903129027 >> v2:172.21.15.39:3300/0 conn(0x7f19680ca5f0 0x7f19680c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f196cd59640 1 --2- 172.21.15.32:0/1903129027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19680b0700 0x7f19680cc1d0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f196806e190 tx=0x7f196001eba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fce67e991025fb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f1966ffd640 1 -- 172.21.15.32:0/1903129027 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19600a7c10 con 0x7f19680b0700 2023-12-20T06:04:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f1966ffd640 1 -- 172.21.15.32:0/1903129027 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f19600ab070 con 0x7f19680b0700 2023-12-20T06:04:21.086 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f1966ffd640 1 -- 172.21.15.32:0/1903129027 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f19600b1400 con 0x7f19680b0700 2023-12-20T06:04:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f196d55a640 1 -- 172.21.15.32:0/1903129027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19680b0700 msgr2=0x7f19680cc1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f196d55a640 1 --2- 172.21.15.32:0/1903129027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19680b0700 0x7f19680cc1d0 secure :-1 s=READY pgs=2502 cs=0 l=1 rev1=1 crypto rx=0x7f196806e190 tx=0x7f196001eba0 comp rx=0 tx=0).stop 2023-12-20T06:04:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f196d55a640 1 -- 172.21.15.32:0/1903129027 shutdown_connections 2023-12-20T06:04:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f196d55a640 1 --2- 172.21.15.32:0/1903129027 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19680b0700 0x7f19680cc1d0 unknown :-1 s=CLOSED pgs=2502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f196d55a640 1 --2- 172.21.15.32:0/1903129027 >> v2:172.21.15.39:3300/0 conn(0x7f19680ca5f0 0x7f19680c8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f196d55a640 1 -- 172.21.15.32:0/1903129027 >> 172.21.15.32:0/1903129027 conn(0x7f196806c060 msgr2=0x7f196806c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f196d55a640 1 -- 172.21.15.32:0/1903129027 shutdown_connections 2023-12-20T06:04:21.087 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f196d55a640 1 -- 172.21.15.32:0/1903129027 wait complete. 2023-12-20T06:04:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f196d55a640 1 Processor -- start 2023-12-20T06:04:21.088 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.085+0000 7f196d55a640 1 -- start start 2023-12-20T06:04:21.090 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f196d55a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19680b0700 0x7f196815bf30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f196d55a640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f19680ca210 0x7f19680ccdd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f196d55a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19680ca5f0 0x7f19680cd310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f196d55a640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f19680b43d0 con 0x7f19680b0700 2023-12-20T06:04:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f196d55a640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f196815b490 con 0x7f19680ca5f0 2023-12-20T06:04:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f196d55a640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f19680b40f0 con 0x7f19680ca210 2023-12-20T06:04:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1967fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19680b0700 0x7f196815bf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1967fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19680b0700 0x7f196815bf30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37022/0 (socket says 172.21.15.32:37022) 2023-12-20T06:04:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1967fff640 1 -- 172.21.15.32:0/189333188 learned_addr learned my addr 172.21.15.32:0/189333188 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:21.091 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f19677fe640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f19680ca210 0x7f19680ccdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f196cd59640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19680ca5f0 0x7f19680cd310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1967fff640 1 -- 172.21.15.32:0/189333188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f19680ca210 msgr2=0x7f19680ccdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1967fff640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f19680ca210 0x7f19680ccdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1967fff640 1 -- 172.21.15.32:0/189333188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19680ca5f0 msgr2=0x7f19680cd310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1967fff640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19680ca5f0 0x7f19680cd310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1967fff640 1 -- 172.21.15.32:0/189333188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f196000bd40 con 0x7f19680b0700 2023-12-20T06:04:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f196cd59640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19680ca5f0 0x7f19680cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1967fff640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19680b0700 0x7f196815bf30 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f195800c500 tx=0x7f195800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1964ff9640 1 -- 172.21.15.32:0/189333188 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958005220 con 0x7f19680b0700 2023-12-20T06:04:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1964ff9640 1 -- 172.21.15.32:0/189333188 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1958005380 con 0x7f19680b0700 2023-12-20T06:04:21.092 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1964ff9640 1 -- 172.21.15.32:0/189333188 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1958007270 con 0x7f19680b0700 2023-12-20T06:04:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f196d55a640 1 -- 172.21.15.32:0/189333188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f196815b730 con 0x7f19680b0700 2023-12-20T06:04:21.093 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f196d55a640 1 -- 172.21.15.32:0/189333188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f19681490d0 con 0x7f19680b0700 2023-12-20T06:04:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f196d55a640 1 -- 172.21.15.32:0/189333188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f196815b2f0 con 0x7f19680b0700 2023-12-20T06:04:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.089+0000 7f1964ff9640 1 -- 172.21.15.32:0/189333188 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7f19580073d0 con 0x7f19680b0700 2023-12-20T06:04:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.093+0000 7f1964ff9640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1954065ca0 0x7f1954068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.093+0000 7f1964ff9640 1 -- 172.21.15.32:0/189333188 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f1958082dc0 con 0x7f19680b0700 2023-12-20T06:04:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.093+0000 7f19677fe640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1954065ca0 0x7f1954068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:21.094 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.093+0000 7f19677fe640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1954065ca0 0x7f1954068160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f19680c86d0 tx=0x7f195c008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:21.097 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.097+0000 7f1964ff9640 1 -- 172.21.15.32:0/189333188 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f196815b2f0 con 0x7f19680b0700 2023-12-20T06:04:21.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.289+0000 7f196d55a640 1 -- 172.21.15.32:0/189333188 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs ls"} v 0) v1 -- 0x7f196815b2f0 con 0x7f19680b0700 2023-12-20T06:04:21.292 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.289+0000 7f1964ff9640 1 -- 172.21.15.32:0/189333188 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v128) v1 ==== 53+0+73 (secure 0 0 0) 0x7f196815b2f0 con 0x7f19680b0700 2023-12-20T06:04:21.293 INFO:teuthology.orchestra.run.smithi032.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data ] 2023-12-20T06:04:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 -- 172.21.15.32:0/189333188 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1954065ca0 msgr2=0x7f1954068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1954065ca0 0x7f1954068160 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f19680c86d0 tx=0x7f195c008040 comp rx=0 tx=0).stop 2023-12-20T06:04:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 -- 172.21.15.32:0/189333188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19680b0700 msgr2=0x7f196815bf30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19680b0700 0x7f196815bf30 secure :-1 s=READY pgs=2503 cs=0 l=1 rev1=1 crypto rx=0x7f195800c500 tx=0x7f195800c9c0 comp rx=0 tx=0).stop 2023-12-20T06:04:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 -- 172.21.15.32:0/189333188 shutdown_connections 2023-12-20T06:04:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1954065ca0 0x7f1954068160 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f19680ca5f0 0x7f19680cd310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f19680ca210 0x7f19680ccdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 --2- 172.21.15.32:0/189333188 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f19680b0700 0x7f196815bf30 unknown :-1 s=CLOSED pgs=2503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:21.295 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 -- 172.21.15.32:0/189333188 >> 172.21.15.32:0/189333188 conn(0x7f196806c060 msgr2=0x7f1968159850 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:21.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 -- 172.21.15.32:0/189333188 shutdown_connections 2023-12-20T06:04:21.296 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:21.293+0000 7f196d55a640 1 -- 172.21.15.32:0/189333188 wait complete. 2023-12-20T06:04:21.314 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2023-12-20T06:04:21.314 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2023-12-20T06:04:21.314 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi032.front.sepia.ceph.com 2023-12-20T06:04:21.314 INFO:tasks.cephfs.mount:self.client.name = client.0 2023-12-20T06:04:21.314 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2023-12-20T06:04:21.314 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2023-12-20T06:04:21.314 INFO:tasks.cephfs.mount:self.cephfs_mntpt = / 2023-12-20T06:04:21.314 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2023-12-20T06:04:21.314 INFO:tasks.cephfs.mount:Setting the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:04:21.314 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:21.315 DEBUG:teuthology.orchestra.run.smithi032:> ip addr 2023-12-20T06:04:21.325 INFO:teuthology.orchestra.run.smithi032.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2023-12-20T06:04:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2023-12-20T06:04:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: inet 127.0.0.1/8 scope host lo 2023-12-20T06:04:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:04:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 ::1/128 scope host 2023-12-20T06:04:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:04:21.326 INFO:teuthology.orchestra.run.smithi032.stdout:2: eno1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:04:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0c brd ff:ff:ff:ff:ff:ff 2023-12-20T06:04:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f0 2023-12-20T06:04:21.326 INFO:teuthology.orchestra.run.smithi032.stdout:3: ens1f0: mtu 1500 qdisc mq state UP group default qlen 1000 2023-12-20T06:04:21.326 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:02 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f0 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.21.15.32/20 brd 172.21.15.255 scope global dynamic ens1f0 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft 36598sec preferred_lft 36598sec 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::ae1f:6bff:fef8:2602/64 scope link 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout:4: eno2: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0c:c4:7a:6b:fd:0d brd ff:ff:ff:ff:ff:ff 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp7s0f1 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout:5: ens1f1: mtu 1500 qdisc noop state DOWN group default qlen 1000 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether ac:1f:6b:f8:26:03 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout: altname enp1s0f1 2023-12-20T06:04:21.327 INFO:teuthology.orchestra.run.smithi032.stdout:6: docker0: mtu 1500 qdisc noqueue state DOWN group default 2023-12-20T06:04:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 02:42:7d:07:a8:f9 brd ff:ff:ff:ff:ff:ff 2023-12-20T06:04:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: inet 172.17.0.1/16 brd 172.17.255.255 scope global docker0 2023-12-20T06:04:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:04:21.328 INFO:teuthology.orchestra.run.smithi032.stdout:7: ceph-brx: mtu 1500 qdisc noqueue state UP group default qlen 1000 2023-12-20T06:04:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 5e:1f:84:6d:79:ae brd ff:ff:ff:ff:ff:ff 2023-12-20T06:04:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: inet 192.168.255.254/16 brd 192.168.255.255 scope global ceph-brx 2023-12-20T06:04:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:04:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::5c1f:84ff:fe6d:79ae/64 scope link 2023-12-20T06:04:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:04:21.328 INFO:teuthology.orchestra.run.smithi032.stdout:8: brx.0@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T06:04:21.328 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether 0e:5c:f6:1f:82:2f brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T06:04:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c5c:f6ff:fe1f:822f/64 scope link 2023-12-20T06:04:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:04:21.329 INFO:teuthology.orchestra.run.smithi032.stdout:9: brx.1@if2: mtu 1500 qdisc noqueue master ceph-brx state UP group default qlen 1000 2023-12-20T06:04:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: link/ether c6:a6:3d:69:d0:db brd ff:ff:ff:ff:ff:ff link-netns ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T06:04:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: inet6 fe80::c4a6:3dff:fe69:d0db/64 scope link 2023-12-20T06:04:21.329 INFO:teuthology.orchestra.run.smithi032.stdout: valid_lft forever preferred_lft forever 2023-12-20T06:04:21.329 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:21.329 DEBUG:teuthology.orchestra.run.smithi032:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2023-12-20T06:04:21.388 INFO:teuthology.orchestra.run.smithi032.stdout:1 2023-12-20T06:04:21.390 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:21.390 DEBUG:teuthology.orchestra.run.smithi032:> route 2023-12-20T06:04:21.443 INFO:teuthology.orchestra.run.smithi032.stdout:Kernel IP routing table 2023-12-20T06:04:21.443 INFO:teuthology.orchestra.run.smithi032.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2023-12-20T06:04:21.443 INFO:teuthology.orchestra.run.smithi032.stdout:default _gateway 0.0.0.0 UG 0 0 0 ens1f0 2023-12-20T06:04:21.443 INFO:teuthology.orchestra.run.smithi032.stdout:172.17.0.0 0.0.0.0 255.255.0.0 U 0 0 0 docker0 2023-12-20T06:04:21.443 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 0 0 0 ens1f0 2023-12-20T06:04:21.443 INFO:teuthology.orchestra.run.smithi032.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2023-12-20T06:04:21.444 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:21.444 DEBUG:teuthology.orchestra.run.smithi032:> (cd / && exec bash -c ' 2023-12-20T06:04:21.444 DEBUG:teuthology.orchestra.run.smithi032:> set -e 2023-12-20T06:04:21.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -o ens1f0 -i ceph-brx -j ACCEPT 2023-12-20T06:04:21.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -A FORWARD -i ens1f0 -o ceph-brx -j ACCEPT 2023-12-20T06:04:21.444 DEBUG:teuthology.orchestra.run.smithi032:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o ens1f0 -j MASQUERADE 2023-12-20T06:04:21.444 DEBUG:teuthology.orchestra.run.smithi032:> ') 2023-12-20T06:04:21.538 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:21.538 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T06:04:21.548 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T06:04:21.548 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T06:04:21.548 INFO:tasks.cephfs.mount:Resuming the 'ceph-ns--home-ubuntu-cephtest-mnt.0' netns for '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:04:21.548 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:21.548 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link set brx.0 up 2023-12-20T06:04:21.609 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T06:04:21.610 DEBUG:teuthology.orchestra.run.smithi032:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2023-12-20T06:04:21.662 INFO:teuthology.orchestra.run.smithi032.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2023-12-20T06:04:21.662 INFO:teuthology.orchestra.run:Running command with timeout 60 2023-12-20T06:04:21.662 DEBUG:teuthology.orchestra.run.smithi032:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2023-12-20T06:04:21.712 INFO:tasks.cephfs.kernel_mount:mounting using device: 0@.cephfs=/ 2023-12-20T06:04:21.713 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:21.713 DEBUG:teuthology.orchestra.run.smithi032:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph 0@.cephfs=/ /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:04:21.809 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:04:21.809 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T06:04:21.822 INFO:teuthology.orchestra.run.smithi032.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,nofallback 2023-12-20T06:04:21.822 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes". 2023-12-20T06:04:21.822 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: resolved to: "172.21.15.32:3300,172.21.15.39:3300,172.21.15.39:3301" 2023-12-20T06:04:21.822 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: trying mount with new device syntax: 0@9b496156-2b6f-4897-9af4-fbc686ad55e7.cephfs=/ 2023-12-20T06:04:21.822 INFO:teuthology.orchestra.run.smithi032.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,ms_mode=prefer-crc,key=0,mon_addr=172.21.15.32:3300/172.21.15.39:3300/172.21.15.39:3301" will pass to kernel 2023-12-20T06:04:21.826 INFO:tasks.cephfs.kernel_mount:mount command passed 2023-12-20T06:04:21.826 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:21.826 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2023-12-20T06:04:21.848 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:21.848 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec bash -c 'getfattr --only-values -n ceph.client_id .') 2023-12-20T06:04:21.914 INFO:teuthology.orchestra.run.smithi032.stdout:client9938 2023-12-20T06:04:21.914 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:04:21.914 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:21.966 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs volume ls 2023-12-20T06:04:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.153+0000 7fa38d918640 1 Processor -- start 2023-12-20T06:04:22.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.153+0000 7fa38d918640 1 -- start start 2023-12-20T06:04:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa38d918640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa38815ce30 0x7fa388161560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa38d918640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa388071920 con 0x7fa38806deb0 2023-12-20T06:04:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa38d918640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa388071080 con 0x7fa388070950 2023-12-20T06:04:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa38d918640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa388071360 con 0x7fa38815ce30 2023-12-20T06:04:22.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3877fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fa38806deb0 0x7fa3880cd520 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48962/0 (socket says 172.21.15.32:48962) 2023-12-20T06:04:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3877fe640 1 -- 172.21.15.32:0/3479567901 learned_addr learned my addr 172.21.15.32:0/3479567901 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa386ffd640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa38815ce30 0x7fa388161560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3011362601 0 0) 0x7fa388071920 con 0x7fa38806deb0 2023-12-20T06:04:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fa374003530 con 0x7fa38806deb0 2023-12-20T06:04:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 444730165 0 0) 0x7fa374003530 con 0x7fa38806deb0 2023-12-20T06:04:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa38815ce30 msgr2=0x7fa388161560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa38815ce30 0x7fa388161560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 >> v1:172.21.15.39:6789/0 conn(0x7fa388070950 legacy=0x7fa38806d7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa388071640 con 0x7fa38806deb0 2023-12-20T06:04:22.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fa380001a00 con 0x7fa38806deb0 2023-12-20T06:04:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 >> v1:172.21.15.32:6789/0 conn(0x7fa38806deb0 legacy=0x7fa3880cd520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3740039b0 0x7fa374003d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374004360 0x7fa374004830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa38815ce30 0x7fa374005130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa388071360 con 0x7fa374004360 2023-12-20T06:04:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa37400ba40 con 0x7fa3740039b0 2023-12-20T06:04:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa37400bc00 con 0x7fa38815ce30 2023-12-20T06:04:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa37ffff640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374004360 0x7fa374004830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa386ffd640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3740039b0 0x7fa374003d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa3877fe640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa38815ce30 0x7fa374005130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa37ffff640 1 -- 172.21.15.32:0/3479567901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa38815ce30 msgr2=0x7fa374005130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa37ffff640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa38815ce30 0x7fa374005130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa37ffff640 1 -- 172.21.15.32:0/3479567901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3740039b0 msgr2=0x7fa374003d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa37ffff640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3740039b0 0x7fa374003d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.157+0000 7fa37ffff640 1 -- 172.21.15.32:0/3479567901 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa374003530 con 0x7fa374004360 2023-12-20T06:04:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa37ffff640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374004360 0x7fa374004830 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fa370001910 tx=0x7fa37001ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8a84dde60831e9a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:22.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3700a88d0 con 0x7fa374004360 2023-12-20T06:04:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa3700a8a50 con 0x7fa374004360 2023-12-20T06:04:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa3867fc640 1 -- 172.21.15.32:0/3479567901 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3700a8e00 con 0x7fa374004360 2023-12-20T06:04:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa38d918640 1 -- 172.21.15.32:0/3479567901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374004360 msgr2=0x7fa374004830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa38d918640 1 --2- 172.21.15.32:0/3479567901 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa374004360 0x7fa374004830 secure :-1 s=READY pgs=2505 cs=0 l=1 rev1=1 crypto rx=0x7fa370001910 tx=0x7fa37001ecb0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa3877fe640 1 -- 172.21.15.32:0/3479567901 reap_dead start 2023-12-20T06:04:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa38d918640 1 -- 172.21.15.32:0/3479567901 shutdown_connections 2023-12-20T06:04:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa38d918640 1 -- 172.21.15.32:0/3479567901 >> 172.21.15.32:0/3479567901 conn(0x7fa3880cba50 msgr2=0x7fa3880cbe60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa38d918640 1 -- 172.21.15.32:0/3479567901 shutdown_connections 2023-12-20T06:04:22.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa38d918640 1 -- 172.21.15.32:0/3479567901 wait complete. 2023-12-20T06:04:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.161+0000 7fa38d918640 1 Processor -- start 2023-12-20T06:04:22.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa38d918640 1 -- start start 2023-12-20T06:04:22.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa38d918640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa374004360 0x7fa3880b2600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa38d918640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3880c53d0 0x7fa3881f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa38d918640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3880afa80 0x7fa38809ebd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa38d918640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa38809e740 con 0x7fa3880c53d0 2023-12-20T06:04:22.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa38d918640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa3880c5bb0 con 0x7fa3880afa80 2023-12-20T06:04:22.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa38d918640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa38809de10 con 0x7fa374004360 2023-12-20T06:04:22.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa37ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3880c53d0 0x7fa3881f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa37ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3880c53d0 0x7fa3881f1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37036/0 (socket says 172.21.15.32:37036) 2023-12-20T06:04:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa37ffff640 1 -- 172.21.15.32:0/3961925496 learned_addr learned my addr 172.21.15.32:0/3961925496 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa386ffd640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa374004360 0x7fa3880b2600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa3877fe640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3880afa80 0x7fa38809ebd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa37ffff640 1 -- 172.21.15.32:0/3961925496 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa374004360 msgr2=0x7fa3880b2600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa37ffff640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa374004360 0x7fa3880b2600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa37ffff640 1 -- 172.21.15.32:0/3961925496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3880afa80 msgr2=0x7fa38809ebd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa37ffff640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3880afa80 0x7fa38809ebd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa37ffff640 1 -- 172.21.15.32:0/3961925496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa38809e0f0 con 0x7fa3880c53d0 2023-12-20T06:04:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa37ffff640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3880c53d0 0x7fa3881f1be0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fa3700b3040 tx=0x7fa374005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:22.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa384ff9640 1 -- 172.21.15.32:0/3961925496 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3700050c0 con 0x7fa3880c53d0 2023-12-20T06:04:22.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa384ff9640 1 -- 172.21.15.32:0/3961925496 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa370005220 con 0x7fa3880c53d0 2023-12-20T06:04:22.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa384ff9640 1 -- 172.21.15.32:0/3961925496 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa3700067e0 con 0x7fa3880c53d0 2023-12-20T06:04:22.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa38d918640 1 -- 172.21.15.32:0/3961925496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa38809e3d0 con 0x7fa3880c53d0 2023-12-20T06:04:22.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa38d918640 1 -- 172.21.15.32:0/3961925496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa3880c5850 con 0x7fa3880c53d0 2023-12-20T06:04:22.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.165+0000 7fa38d918640 1 -- 172.21.15.32:0/3961925496 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa38809e560 con 0x7fa3880c53d0 2023-12-20T06:04:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.169+0000 7fa384ff9640 1 -- 172.21.15.32:0/3961925496 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fa3700b7040 con 0x7fa3880c53d0 2023-12-20T06:04:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.169+0000 7fa384ff9640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa368065ca0 0x7fa368068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.169+0000 7fa384ff9640 1 -- 172.21.15.32:0/3961925496 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa3701203d0 con 0x7fa3880c53d0 2023-12-20T06:04:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.169+0000 7fa386ffd640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa368065ca0 0x7fa368068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.169+0000 7fa386ffd640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa368065ca0 0x7fa368068160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa378009b60 tx=0x7fa378009190 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:22.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.173+0000 7fa384ff9640 1 -- 172.21.15.32:0/3961925496 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa3700edbb0 con 0x7fa3880c53d0 2023-12-20T06:04:22.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.349+0000 7fa38d918640 1 -- 172.21.15.32:0/3961925496 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs volume ls", "target": ["mon-mgr", ""]}) v1 -- 0x7fa388070d90 con 0x7fa368065ca0 2023-12-20T06:04:22.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.349+0000 7fa384ff9640 1 -- 172.21.15.32:0/3961925496 <== mgr.11715 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7fa378019070 con 0x7fa368065ca0 2023-12-20T06:04:22.352 INFO:teuthology.orchestra.run.smithi032.stdout:[ 2023-12-20T06:04:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: { 2023-12-20T06:04:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: "name": "cephfs" 2023-12-20T06:04:22.353 INFO:teuthology.orchestra.run.smithi032.stdout: } 2023-12-20T06:04:22.353 INFO:teuthology.orchestra.run.smithi032.stdout:] 2023-12-20T06:04:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 -- 172.21.15.32:0/3961925496 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa368065ca0 msgr2=0x7fa368068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa368065ca0 0x7fa368068160 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7fa378009b60 tx=0x7fa378009190 comp rx=0 tx=0).stop 2023-12-20T06:04:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 -- 172.21.15.32:0/3961925496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3880c53d0 msgr2=0x7fa3881f1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3880c53d0 0x7fa3881f1be0 secure :-1 s=READY pgs=2506 cs=0 l=1 rev1=1 crypto rx=0x7fa3700b3040 tx=0x7fa374005800 comp rx=0 tx=0).stop 2023-12-20T06:04:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 -- 172.21.15.32:0/3961925496 shutdown_connections 2023-12-20T06:04:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa368065ca0 0x7fa368068160 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa3880afa80 0x7fa38809ebd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa3880c53d0 0x7fa3881f1be0 unknown :-1 s=CLOSED pgs=2506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 --2- 172.21.15.32:0/3961925496 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa374004360 0x7fa3880b2600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 -- 172.21.15.32:0/3961925496 >> 172.21.15.32:0/3961925496 conn(0x7fa3880cba50 msgr2=0x7fa38815d520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:22.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 -- 172.21.15.32:0/3961925496 shutdown_connections 2023-12-20T06:04:22.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.353+0000 7fa38d918640 1 -- 172.21.15.32:0/3961925496 wait complete. 2023-12-20T06:04:22.374 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module enable snap_schedule 2023-12-20T06:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.517+0000 7fa9aef6d640 1 Processor -- start 2023-12-20T06:04:22.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.517+0000 7fa9aef6d640 1 -- start start 2023-12-20T06:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.517+0000 7fa9aef6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9a815abb0 0x7fa9a80caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.517+0000 7fa9aef6d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9a8070230 0x7fa9a80706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.517+0000 7fa9aef6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fa9a815b6a0 con 0x7fa9a80b0cb0 2023-12-20T06:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.517+0000 7fa9aef6d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fa9a806e160 con 0x7fa9a8070230 2023-12-20T06:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.517+0000 7fa9aef6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a8070be0 con 0x7fa9a815abb0 2023-12-20T06:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9ad50a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9a8070230 0x7fa9a80706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9ad50a640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fa9a8070230 0x7fa9a80706a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37048/0 (socket says 172.21.15.32:37048) 2023-12-20T06:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9ad50a640 1 -- 172.21.15.32:0/4041275113 learned_addr learned my addr 172.21.15.32:0/4041275113 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9acd09640 1 --2- 172.21.15.32:0/4041275113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9a815abb0 0x7fa9a80caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9ad50a640 1 -- 172.21.15.32:0/4041275113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9a815abb0 msgr2=0x7fa9a80caaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9ad50a640 1 --2- 172.21.15.32:0/4041275113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9a815abb0 0x7fa9a80caaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9ad50a640 1 -- 172.21.15.32:0/4041275113 >> v1:172.21.15.39:6789/0 conn(0x7fa9a80b0cb0 legacy=0x7fa9a815bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9ad50a640 1 -- 172.21.15.32:0/4041275113 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a806d8c0 con 0x7fa9a8070230 2023-12-20T06:04:22.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9acd09640 1 --2- 172.21.15.32:0/4041275113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9a815abb0 0x7fa9a80caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:22.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9ad50a640 1 --2- 172.21.15.32:0/4041275113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a8070230 0x7fa9a80706a0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009e50 tx=0x7fa9a001c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f6dbbf53ce85df3 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:22.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9a77fe640 1 -- 172.21.15.32:0/4041275113 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00a5850 con 0x7fa9a8070230 2023-12-20T06:04:22.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9a77fe640 1 -- 172.21.15.32:0/4041275113 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a00a59b0 con 0x7fa9a8070230 2023-12-20T06:04:22.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.521+0000 7fa9a77fe640 1 -- 172.21.15.32:0/4041275113 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00a5d40 con 0x7fa9a8070230 2023-12-20T06:04:22.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 -- 172.21.15.32:0/4041275113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a8070230 msgr2=0x7fa9a80706a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 --2- 172.21.15.32:0/4041275113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a8070230 0x7fa9a80706a0 secure :-1 s=READY pgs=2507 cs=0 l=1 rev1=1 crypto rx=0x7fa9a0009e50 tx=0x7fa9a001c9d0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 -- 172.21.15.32:0/4041275113 shutdown_connections 2023-12-20T06:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 --2- 172.21.15.32:0/4041275113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a8070230 0x7fa9a80706a0 unknown :-1 s=CLOSED pgs=2507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 --2- 172.21.15.32:0/4041275113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9a815abb0 0x7fa9a80caaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 -- 172.21.15.32:0/4041275113 >> 172.21.15.32:0/4041275113 conn(0x7fa9a806a510 msgr2=0x7fa9a80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 -- 172.21.15.32:0/4041275113 shutdown_connections 2023-12-20T06:04:22.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 -- 172.21.15.32:0/4041275113 wait complete. 2023-12-20T06:04:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 Processor -- start 2023-12-20T06:04:22.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 -- start start 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9a8070230 0x7fa9a813f2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9a80b0cb0 0x7fa9a81466c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815abb0 0x7fa9a8146c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a8143140 con 0x7fa9a815abb0 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a81439e0 con 0x7fa9a80b0cb0 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9aef6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fa9a8141ca0 con 0x7fa9a8070230 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9ad50a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815abb0 0x7fa9a8146c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9ad50a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815abb0 0x7fa9a8146c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37054/0 (socket says 172.21.15.32:37054) 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9ad50a640 1 -- 172.21.15.32:0/3275934276 learned_addr learned my addr 172.21.15.32:0/3275934276 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9acd09640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9a8070230 0x7fa9a813f2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.525+0000 7fa9a7fff640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9a80b0cb0 0x7fa9a81466c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9ad50a640 1 -- 172.21.15.32:0/3275934276 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9a8070230 msgr2=0x7fa9a813f2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:22.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9ad50a640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9a8070230 0x7fa9a813f2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9ad50a640 1 -- 172.21.15.32:0/3275934276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9a80b0cb0 msgr2=0x7fa9a81466c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9ad50a640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9a80b0cb0 0x7fa9a81466c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9ad50a640 1 -- 172.21.15.32:0/3275934276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fa9a0009b30 con 0x7fa9a815abb0 2023-12-20T06:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9ad50a640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815abb0 0x7fa9a8146c00 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00af040 tx=0x7fa9a0019770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:22.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9a57fa640 1 -- 172.21.15.32:0/3275934276 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00b8670 con 0x7fa9a815abb0 2023-12-20T06:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9a57fa640 1 -- 172.21.15.32:0/3275934276 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fa9a00b87d0 con 0x7fa9a815abb0 2023-12-20T06:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9a57fa640 1 -- 172.21.15.32:0/3275934276 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fa9a00b8b60 con 0x7fa9a815abb0 2023-12-20T06:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9aef6d640 1 -- 172.21.15.32:0/3275934276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fa9a8143390 con 0x7fa9a815abb0 2023-12-20T06:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9aef6d640 1 -- 172.21.15.32:0/3275934276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fa9a813f820 con 0x7fa9a815abb0 2023-12-20T06:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9aef6d640 1 -- 172.21.15.32:0/3275934276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fa9a8143b70 con 0x7fa9a815abb0 2023-12-20T06:04:22.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9a57fa640 1 -- 172.21.15.32:0/3275934276 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 143) v1 ==== 85432+0+0 (secure 0 0 0) 0x7fa9a00b8cc0 con 0x7fa9a815abb0 2023-12-20T06:04:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9a57fa640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa98c065ca0 0x7fa98c068160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9a57fa640 1 -- 172.21.15.32:0/3275934276 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fa9a011ec50 con 0x7fa9a815abb0 2023-12-20T06:04:22.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.529+0000 7fa9acd09640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa98c065ca0 0x7fa98c068160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:22.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.533+0000 7fa9acd09640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa98c065ca0 0x7fa98c068160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006fd0 tx=0x7fa99c008040 comp rx=0 tx=0).ready entity=mgr.11715 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:22.536 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.533+0000 7fa9a57fa640 1 -- 172.21.15.32:0/3275934276 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fa9a00ec3d0 con 0x7fa9a815abb0 2023-12-20T06:04:22.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:22.733+0000 7fa9aef6d640 1 -- 172.21.15.32:0/3275934276 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) v1 -- 0x7fa9a815b300 con 0x7fa9a815abb0 2023-12-20T06:04:23.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.557+0000 7fa9a57fa640 1 -- 172.21.15.32:0/3275934276 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 144) v1 ==== 85449+0+0 (secure 0 0 0) 0x7fa9a00ec7a0 con 0x7fa9a815abb0 2023-12-20T06:04:23.560 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.557+0000 7fa9a57fa640 1 -- 172.21.15.32:0/3275934276 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v144) v1 ==== 92+0+0 (secure 0 0 0) 0x7fa9a00f2500 con 0x7fa9a815abb0 2023-12-20T06:04:23.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 -- 172.21.15.32:0/3275934276 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa98c065ca0 msgr2=0x7fa98c068160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:23.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa98c065ca0 0x7fa98c068160 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fa99c006fd0 tx=0x7fa99c008040 comp rx=0 tx=0).stop 2023-12-20T06:04:23.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 -- 172.21.15.32:0/3275934276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815abb0 msgr2=0x7fa9a8146c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:23.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815abb0 0x7fa9a8146c00 secure :-1 s=READY pgs=2508 cs=0 l=1 rev1=1 crypto rx=0x7fa9a00af040 tx=0x7fa9a0019770 comp rx=0 tx=0).stop 2023-12-20T06:04:23.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 -- 172.21.15.32:0/3275934276 shutdown_connections 2023-12-20T06:04:23.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fa98c065ca0 0x7fa98c068160 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.564 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fa9a815abb0 0x7fa9a8146c00 unknown :-1 s=CLOSED pgs=2508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fa9a80b0cb0 0x7fa9a81466c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 --2- 172.21.15.32:0/3275934276 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fa9a8070230 0x7fa9a813f2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 -- 172.21.15.32:0/3275934276 >> 172.21.15.32:0/3275934276 conn(0x7fa9a806a510 msgr2=0x7fa9a80c4be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:23.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 -- 172.21.15.32:0/3275934276 shutdown_connections 2023-12-20T06:04:23.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.561+0000 7fa9aef6d640 1 -- 172.21.15.32:0/3275934276 wait complete. 2023-12-20T06:04:23.588 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/allow_m_granularity True 2023-12-20T06:04:23.738 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:23.733+0000 7fae9c248180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:04:23.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.741+0000 7fb0e817b640 1 Processor -- start 2023-12-20T06:04:23.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.741+0000 7fb0e817b640 1 -- start start 2023-12-20T06:04:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e817b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fb0e00d14d0 0x7fb0e00b6ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e817b640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e00d2650 con 0x7fb0e00d3300 2023-12-20T06:04:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e817b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fb0e00d1a40 con 0x7fb0e015ce60 2023-12-20T06:04:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e817b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fb0e00d1d20 con 0x7fb0e00d14d0 2023-12-20T06:04:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e5716640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fb0e00d3300 0x7fb0e00d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48978/0 (socket says 172.21.15.32:48978) 2023-12-20T06:04:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e5716640 1 -- 172.21.15.32:0/4273483233 learned_addr learned my addr 172.21.15.32:0/4273483233 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e6718640 1 --2- 172.21.15.32:0/4273483233 >> v2:172.21.15.39:3300/0 conn(0x7fb0e00d14d0 0x7fb0e00b6ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:23.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e4f15640 1 -- 172.21.15.32:0/4273483233 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2850713074 0 0) 0x7fb0e00d2650 con 0x7fb0e00d3300 2023-12-20T06:04:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e4f15640 1 -- 172.21.15.32:0/4273483233 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fb0cc003530 con 0x7fb0e00d3300 2023-12-20T06:04:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e4f15640 1 -- 172.21.15.32:0/4273483233 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1441062528 0 0) 0x7fb0e00d1a40 con 0x7fb0e015ce60 2023-12-20T06:04:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e6718640 1 -- 172.21.15.32:0/4273483233 >> v1:172.21.15.39:6789/0 conn(0x7fb0e015ce60 legacy=0x7fb0e00d2bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e6718640 1 -- 172.21.15.32:0/4273483233 >> v1:172.21.15.32:6789/0 conn(0x7fb0e00d3300 legacy=0x7fb0e00d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:23.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e6718640 1 -- 172.21.15.32:0/4273483233 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0e00d2000 con 0x7fb0e00d14d0 2023-12-20T06:04:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.745+0000 7fb0e6718640 1 --2- 172.21.15.32:0/4273483233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0e00d14d0 0x7fb0e00b6ea0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009d90 tx=0x7fb0d801f130 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3b30d41618a59205 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e4f15640 1 -- 172.21.15.32:0/4273483233 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80af600 con 0x7fb0e00d14d0 2023-12-20T06:04:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e4f15640 1 -- 172.21.15.32:0/4273483233 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0d80ab050 con 0x7fb0e00d14d0 2023-12-20T06:04:23.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e4f15640 1 -- 172.21.15.32:0/4273483233 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0d80af980 con 0x7fb0e00d14d0 2023-12-20T06:04:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 -- 172.21.15.32:0/4273483233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0e00d14d0 msgr2=0x7fb0e00b6ea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 --2- 172.21.15.32:0/4273483233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0e00d14d0 0x7fb0e00b6ea0 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7fb0d8009d90 tx=0x7fb0d801f130 comp rx=0 tx=0).stop 2023-12-20T06:04:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 -- 172.21.15.32:0/4273483233 shutdown_connections 2023-12-20T06:04:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 --2- 172.21.15.32:0/4273483233 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0e00d14d0 0x7fb0e00b6ea0 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 -- 172.21.15.32:0/4273483233 >> 172.21.15.32:0/4273483233 conn(0x7fb0e006b5e0 msgr2=0x7fb0e006b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 -- 172.21.15.32:0/4273483233 shutdown_connections 2023-12-20T06:04:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 -- 172.21.15.32:0/4273483233 wait complete. 2023-12-20T06:04:23.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 Processor -- start 2023-12-20T06:04:23.752 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:23.749+0000 7f4252049180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:04:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 -- start start 2023-12-20T06:04:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e00d14d0 0x7fb0e00ca330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0e00d3300 0x7fb0e0149230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0e015ce60 0x7fb0e0148770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:23.754 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015f210 con 0x7fb0e00d14d0 2023-12-20T06:04:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015e620 con 0x7fb0e00d3300 2023-12-20T06:04:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.749+0000 7fb0e817b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fb0e015e870 con 0x7fb0e015ce60 2023-12-20T06:04:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e5f17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e00d14d0 0x7fb0e00ca330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e5716640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0e00d3300 0x7fb0e0149230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e5f17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e00d14d0 0x7fb0e00ca330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37074/0 (socket says 172.21.15.32:37074) 2023-12-20T06:04:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e5f17640 1 -- 172.21.15.32:0/3982247955 learned_addr learned my addr 172.21.15.32:0/3982247955 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e6718640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0e015ce60 0x7fb0e0148770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e5f17640 1 -- 172.21.15.32:0/3982247955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0e015ce60 msgr2=0x7fb0e0148770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e5f17640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0e015ce60 0x7fb0e0148770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e5f17640 1 -- 172.21.15.32:0/3982247955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0e00d3300 msgr2=0x7fb0e0149230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:23.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e5f17640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0e00d3300 0x7fb0e0149230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e5f17640 1 -- 172.21.15.32:0/3982247955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fb0d8009a70 con 0x7fb0e00d14d0 2023-12-20T06:04:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e5716640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0e00d3300 0x7fb0e0149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e5f17640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e00d14d0 0x7fb0e00ca330 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00c4d0 tx=0x7fb0dc00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0d67fc640 1 -- 172.21.15.32:0/3982247955 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc005480 con 0x7fb0e00d14d0 2023-12-20T06:04:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0d67fc640 1 -- 172.21.15.32:0/3982247955 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fb0dc003600 con 0x7fb0e00d14d0 2023-12-20T06:04:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0d67fc640 1 -- 172.21.15.32:0/3982247955 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fb0dc0039b0 con 0x7fb0e00d14d0 2023-12-20T06:04:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e817b640 1 -- 172.21.15.32:0/3982247955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fb0e015eb50 con 0x7fb0e00d14d0 2023-12-20T06:04:23.756 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e817b640 1 -- 172.21.15.32:0/3982247955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fb0e00ca870 con 0x7fb0e00d14d0 2023-12-20T06:04:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0e817b640 1 -- 172.21.15.32:0/3982247955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fb0e004e000 con 0x7fb0e00d14d0 2023-12-20T06:04:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.753+0000 7fb0d67fc640 1 -- 172.21.15.32:0/3982247955 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 144) v1 ==== 85449+0+0 (secure 0 0 0) 0x7fb0dc003b10 con 0x7fb0e00d14d0 2023-12-20T06:04:23.757 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.757+0000 7fb0d67fc640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb0c4065cf0 0x7fb0c40681b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:23.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.757+0000 7fb0d67fc640 1 -- 172.21.15.32:0/3982247955 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fb0dc082c70 con 0x7fb0e00d14d0 2023-12-20T06:04:23.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.757+0000 7fb0e5716640 1 -- 172.21.15.32:0/3982247955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb0c4065cf0 msgr2=0x7fb0c40681b0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:23.758 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.757+0000 7fb0e5716640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb0c4065cf0 0x7fb0c40681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:23.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.761+0000 7fb0d67fc640 1 -- 172.21.15.32:0/3982247955 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fb0dc050360 con 0x7fb0e00d14d0 2023-12-20T06:04:23.927 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.925+0000 7fb0e817b640 1 -- 172.21.15.32:0/3982247955 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) v1 -- 0x7fb0e01e7860 con 0x7fb0e00d14d0 2023-12-20T06:04:23.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.929+0000 7fb0d67fc640 1 -- 172.21.15.32:0/3982247955 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v83)=0 v83) v1 ==== 138+0+0 (secure 0 0 0) 0x7fb0dc017070 con 0x7fb0e00d14d0 2023-12-20T06:04:23.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 -- 172.21.15.32:0/3982247955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb0c4065cf0 msgr2=0x7fb0c40681b0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:23.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb0c4065cf0 0x7fb0c40681b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 -- 172.21.15.32:0/3982247955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e00d14d0 msgr2=0x7fb0e00ca330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e00d14d0 0x7fb0e00ca330 secure :-1 s=READY pgs=2513 cs=0 l=1 rev1=1 crypto rx=0x7fb0dc00c4d0 tx=0x7fb0dc00c990 comp rx=0 tx=0).stop 2023-12-20T06:04:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 -- 172.21.15.32:0/3982247955 shutdown_connections 2023-12-20T06:04:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fb0c4065cf0 0x7fb0c40681b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fb0e015ce60 0x7fb0e0148770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fb0e00d3300 0x7fb0e0149230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 --2- 172.21.15.32:0/3982247955 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fb0e00d14d0 0x7fb0e00ca330 unknown :-1 s=CLOSED pgs=2513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 -- 172.21.15.32:0/3982247955 >> 172.21.15.32:0/3982247955 conn(0x7fb0e006b5e0 msgr2=0x7fb0e0066030 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 -- 172.21.15.32:0/3982247955 shutdown_connections 2023-12-20T06:04:23.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:23.933+0000 7fb0e817b640 1 -- 172.21.15.32:0/3982247955 wait complete. 2023-12-20T06:04:23.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config set mgr mgr/snap_schedule/dump_on_update True 2023-12-20T06:04:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.101+0000 7f918dec4640 1 Processor -- start 2023-12-20T06:04:24.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.101+0000 7f918dec4640 1 -- start start 2023-12-20T06:04:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f918dec4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918815ce60 0x7f91880d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f918dec4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f91880d14d0 0x7f91880b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f918dec4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f91880d2650 con 0x7f91880d3130 2023-12-20T06:04:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f918dec4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f91880d3570 con 0x7f91880d14d0 2023-12-20T06:04:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f918dec4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91880d1a40 con 0x7f918815ce60 2023-12-20T06:04:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f9186ffd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f91880d3130 0x7f91880d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:48988/0 (socket says 172.21.15.32:48988) 2023-12-20T06:04:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f9186ffd640 1 -- 172.21.15.32:0/165075321 learned_addr learned my addr 172.21.15.32:0/165075321 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f9187fff640 1 --2- 172.21.15.32:0/165075321 >> v2:172.21.15.32:3300/0 conn(0x7f91880d14d0 0x7f91880b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:24.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f91877fe640 1 --2- 172.21.15.32:0/165075321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918815ce60 0x7f91880d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f91867fc640 1 -- 172.21.15.32:0/165075321 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3849167063 0 0) 0x7f91880d2650 con 0x7f91880d3130 2023-12-20T06:04:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f91867fc640 1 -- 172.21.15.32:0/165075321 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9170003530 con 0x7f91880d3130 2023-12-20T06:04:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f91877fe640 1 -- 172.21.15.32:0/165075321 >> v2:172.21.15.32:3300/0 conn(0x7f91880d14d0 msgr2=0x7f91880b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f91877fe640 1 --2- 172.21.15.32:0/165075321 >> v2:172.21.15.32:3300/0 conn(0x7f91880d14d0 0x7f91880b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f91877fe640 1 -- 172.21.15.32:0/165075321 >> v1:172.21.15.32:6789/0 conn(0x7f91880d3130 legacy=0x7f91880d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f91877fe640 1 -- 172.21.15.32:0/165075321 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f91880d1d20 con 0x7f918815ce60 2023-12-20T06:04:24.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.105+0000 7f9187fff640 1 --2- 172.21.15.32:0/165075321 >> v2:172.21.15.32:3300/0 conn(0x7f91880d14d0 0x7f91880b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f91877fe640 1 --2- 172.21.15.32:0/165075321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918815ce60 0x7f91880d2bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f91780097c0 tx=0x7f917801ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=36dfe1e16d4e6b45 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f91867fc640 1 -- 172.21.15.32:0/165075321 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780a8970 con 0x7f918815ce60 2023-12-20T06:04:24.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f91867fc640 1 -- 172.21.15.32:0/165075321 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91780a8ad0 con 0x7f918815ce60 2023-12-20T06:04:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f91867fc640 1 -- 172.21.15.32:0/165075321 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780a8e60 con 0x7f918815ce60 2023-12-20T06:04:24.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f918dec4640 1 -- 172.21.15.32:0/165075321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918815ce60 msgr2=0x7f91880d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f918dec4640 1 --2- 172.21.15.32:0/165075321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918815ce60 0x7f91880d2bf0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f91780097c0 tx=0x7f917801ec70 comp rx=0 tx=0).stop 2023-12-20T06:04:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f918dec4640 1 -- 172.21.15.32:0/165075321 shutdown_connections 2023-12-20T06:04:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f918dec4640 1 --2- 172.21.15.32:0/165075321 >> v2:172.21.15.32:3300/0 conn(0x7f91880d14d0 0x7f91880b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f918dec4640 1 --2- 172.21.15.32:0/165075321 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f918815ce60 0x7f91880d2bf0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f918dec4640 1 -- 172.21.15.32:0/165075321 >> 172.21.15.32:0/165075321 conn(0x7f918806b5e0 msgr2=0x7f918806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f918dec4640 1 -- 172.21.15.32:0/165075321 shutdown_connections 2023-12-20T06:04:24.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f918dec4640 1 -- 172.21.15.32:0/165075321 wait complete. 2023-12-20T06:04:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f918dec4640 1 Processor -- start 2023-12-20T06:04:24.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.109+0000 7f918dec4640 1 -- start start 2023-12-20T06:04:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f918dec4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880d14d0 0x7f918815ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f918dec4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880d3130 0x7f918815d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f918dec4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f918815ce60 0x7f91880cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f918dec4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880cccb0 con 0x7f918815ce60 2023-12-20T06:04:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f918dec4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f91880cb920 con 0x7f91880d3130 2023-12-20T06:04:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f918dec4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f91880cbc00 con 0x7f91880d14d0 2023-12-20T06:04:24.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f91877fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880d14d0 0x7f918815ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f9187fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f918815ce60 0x7f91880cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f91877fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880d14d0 0x7f918815ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:47956/0 (socket says 172.21.15.32:47956) 2023-12-20T06:04:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f9186ffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880d3130 0x7f918815d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f91877fe640 1 -- 172.21.15.32:0/714310888 learned_addr learned my addr 172.21.15.32:0/714310888 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f91877fe640 1 -- 172.21.15.32:0/714310888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880d3130 msgr2=0x7f918815d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f91877fe640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880d3130 0x7f918815d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f91877fe640 1 -- 172.21.15.32:0/714310888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f918815ce60 msgr2=0x7f91880cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f91877fe640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f918815ce60 0x7f91880cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f91877fe640 1 -- 172.21.15.32:0/714310888 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9180009a70 con 0x7f91880d14d0 2023-12-20T06:04:24.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f9186ffd640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880d3130 0x7f918815d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f9187fff640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f918815ce60 0x7f91880cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f91877fe640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880d14d0 0x7f918815ace0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f91780b3040 tx=0x7f917801ba10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f916ffff640 1 -- 172.21.15.32:0/714310888 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780b0d60 con 0x7f91880d14d0 2023-12-20T06:04:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f916ffff640 1 -- 172.21.15.32:0/714310888 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f91780b9430 con 0x7f91880d14d0 2023-12-20T06:04:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f916ffff640 1 -- 172.21.15.32:0/714310888 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f91780b97e0 con 0x7f91880d14d0 2023-12-20T06:04:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f918dec4640 1 -- 172.21.15.32:0/714310888 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9178009420 con 0x7f91880d14d0 2023-12-20T06:04:24.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f918dec4640 1 -- 172.21.15.32:0/714310888 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f918815f810 con 0x7f91880d14d0 2023-12-20T06:04:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f918dec4640 1 -- 172.21.15.32:0/714310888 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f918815f390 con 0x7f91880d14d0 2023-12-20T06:04:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.113+0000 7f916ffff640 1 -- 172.21.15.32:0/714310888 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 144) v1 ==== 85449+0+0 (secure 0 0 0) 0x7f91780b7040 con 0x7f91880d14d0 2023-12-20T06:04:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.117+0000 7f916ffff640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9174065d20 0x7f91740681e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.117+0000 7f916ffff640 1 -- 172.21.15.32:0/714310888 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7f9178120ef0 con 0x7f91880d14d0 2023-12-20T06:04:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.117+0000 7f9186ffd640 1 -- 172.21.15.32:0/714310888 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9174065d20 msgr2=0x7f91740681e0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:24.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.117+0000 7f9186ffd640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9174065d20 0x7f91740681e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:24.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.121+0000 7f916ffff640 1 -- 172.21.15.32:0/714310888 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f918815f390 con 0x7f91880d14d0 2023-12-20T06:04:24.180 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:24.177+0000 7fae9c248180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:04:24.203 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:24.201+0000 7f4252049180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:04:24.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.281+0000 7f918dec4640 1 -- 172.21.15.32:0/714310888 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) v1 -- 0x7f918815f390 con 0x7f91880d14d0 2023-12-20T06:04:24.290 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.289+0000 7f916ffff640 1 -- 172.21.15.32:0/714310888 <== mon.2 v2:172.21.15.39:3301/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v84)=0 v84) v1 ==== 133+0+0 (secure 0 0 0) 0x7f918815f390 con 0x7f91880d14d0 2023-12-20T06:04:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.289+0000 7f918dec4640 1 -- 172.21.15.32:0/714310888 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9174065d20 msgr2=0x7f91740681e0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.289+0000 7f918dec4640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9174065d20 0x7f91740681e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.289+0000 7f918dec4640 1 -- 172.21.15.32:0/714310888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880d14d0 msgr2=0x7f918815ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.289+0000 7f918dec4640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880d14d0 0x7f918815ace0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f91780b3040 tx=0x7f917801ba10 comp rx=0 tx=0).stop 2023-12-20T06:04:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.289+0000 7f918dec4640 1 -- 172.21.15.32:0/714310888 shutdown_connections 2023-12-20T06:04:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.289+0000 7f918dec4640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9174065d20 0x7f91740681e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.289+0000 7f918dec4640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f918815ce60 0x7f91880cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.293 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.289+0000 7f918dec4640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f91880d3130 0x7f918815d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.289+0000 7f918dec4640 1 --2- 172.21.15.32:0/714310888 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f91880d14d0 0x7f918815ace0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.289+0000 7f918dec4640 1 -- 172.21.15.32:0/714310888 >> 172.21.15.32:0/714310888 conn(0x7f918806b5e0 msgr2=0x7f91880b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.293+0000 7f918dec4640 1 -- 172.21.15.32:0/714310888 shutdown_connections 2023-12-20T06:04:24.294 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.293+0000 7f918dec4640 1 -- 172.21.15.32:0/714310888 wait complete. 2023-12-20T06:04:24.313 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:24.313 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec mkdir -p snap_test_dir1) 2023-12-20T06:04:24.329 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule add snap_test_dir1 1M 2023-12-20T06:04:24.339 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:24.337+0000 7fae9c248180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:04:24.363 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:24.361+0000 7f4252049180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:04:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.509+0000 7fddbb5e9640 1 Processor -- start 2023-12-20T06:04:24.511 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.509+0000 7fddbb5e9640 1 -- start start 2023-12-20T06:04:24.513 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.509+0000 7fddbb5e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415ce60 0x7fddb40d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.509+0000 7fddbb5e9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fddb40d14d0 0x7fddb40b5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.509+0000 7fddbb5e9640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fddb40d2650 con 0x7fddb40d3130 2023-12-20T06:04:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddbb5e9640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fddb40d3570 con 0x7fddb40d14d0 2023-12-20T06:04:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddbb5e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb40d1a40 con 0x7fddb415ce60 2023-12-20T06:04:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddb9385640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415ce60 0x7fddb40d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddb8b84640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fddb40d3130 0x7fddb40d0dc0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:46178/0 (socket says 172.21.15.32:46178) 2023-12-20T06:04:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddb8b84640 1 -- 172.21.15.32:0/1869778817 learned_addr learned my addr 172.21.15.32:0/1869778817 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddb9b86640 1 --2- 172.21.15.32:0/1869778817 >> v2:172.21.15.32:3300/0 conn(0x7fddb40d14d0 0x7fddb40b5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddabfff640 1 -- 172.21.15.32:0/1869778817 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3462161359 0 0) 0x7fddb40d2650 con 0x7fddb40d3130 2023-12-20T06:04:24.514 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddabfff640 1 -- 172.21.15.32:0/1869778817 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdda0003530 con 0x7fddb40d3130 2023-12-20T06:04:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddb9385640 1 -- 172.21.15.32:0/1869778817 >> v2:172.21.15.32:3300/0 conn(0x7fddb40d14d0 msgr2=0x7fddb40b5e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddb9385640 1 --2- 172.21.15.32:0/1869778817 >> v2:172.21.15.32:3300/0 conn(0x7fddb40d14d0 0x7fddb40b5e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddb9385640 1 -- 172.21.15.32:0/1869778817 >> v1:172.21.15.39:6789/0 conn(0x7fddb40d3130 legacy=0x7fddb40d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddb9385640 1 -- 172.21.15.32:0/1869778817 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddb40d1d20 con 0x7fddb415ce60 2023-12-20T06:04:24.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.513+0000 7fddb9b86640 1 --2- 172.21.15.32:0/1869778817 >> v2:172.21.15.32:3300/0 conn(0x7fddb40d14d0 0x7fddb40b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:24.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.517+0000 7fddb9385640 1 --2- 172.21.15.32:0/1869778817 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415ce60 0x7fddb40d2bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fddb00097c0 tx=0x7fddb001ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d99229a12f911cd9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:24.524 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.521+0000 7fddabfff640 1 -- 172.21.15.32:0/1869778817 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00a8970 con 0x7fddb415ce60 2023-12-20T06:04:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.521+0000 7fddabfff640 1 -- 172.21.15.32:0/1869778817 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb00a8ad0 con 0x7fddb415ce60 2023-12-20T06:04:24.525 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.521+0000 7fddabfff640 1 -- 172.21.15.32:0/1869778817 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00a8e60 con 0x7fddb415ce60 2023-12-20T06:04:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 -- 172.21.15.32:0/1869778817 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415ce60 msgr2=0x7fddb40d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 --2- 172.21.15.32:0/1869778817 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415ce60 0x7fddb40d2bf0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fddb00097c0 tx=0x7fddb001ec70 comp rx=0 tx=0).stop 2023-12-20T06:04:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 -- 172.21.15.32:0/1869778817 shutdown_connections 2023-12-20T06:04:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 --2- 172.21.15.32:0/1869778817 >> v2:172.21.15.32:3300/0 conn(0x7fddb40d14d0 0x7fddb40b5e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 --2- 172.21.15.32:0/1869778817 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415ce60 0x7fddb40d2bf0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 -- 172.21.15.32:0/1869778817 >> 172.21.15.32:0/1869778817 conn(0x7fddb406b5e0 msgr2=0x7fddb406b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:24.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 -- 172.21.15.32:0/1869778817 shutdown_connections 2023-12-20T06:04:24.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 -- 172.21.15.32:0/1869778817 wait complete. 2023-12-20T06:04:24.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 Processor -- start 2023-12-20T06:04:24.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 -- start start 2023-12-20T06:04:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40d14d0 0x7fddb415ace0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb40d3130 0x7fddb415d5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415ce60 0x7fddb40cad80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb40cccb0 con 0x7fddb40d14d0 2023-12-20T06:04:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fddb40cb920 con 0x7fddb40d3130 2023-12-20T06:04:24.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddbb5e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fddb40cbc00 con 0x7fddb415ce60 2023-12-20T06:04:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddb9385640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40d14d0 0x7fddb415ace0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddb9385640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40d14d0 0x7fddb415ace0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:37112/0 (socket says 172.21.15.32:37112) 2023-12-20T06:04:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddb9385640 1 -- 172.21.15.32:0/2461170127 learned_addr learned my addr 172.21.15.32:0/2461170127 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddb8b84640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb40d3130 0x7fddb415d5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.525+0000 7fddb9b86640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415ce60 0x7fddb40cad80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fddb9385640 1 -- 172.21.15.32:0/2461170127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415ce60 msgr2=0x7fddb40cad80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fddb9385640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415ce60 0x7fddb40cad80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fddb9385640 1 -- 172.21.15.32:0/2461170127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb40d3130 msgr2=0x7fddb415d5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fddb9385640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb40d3130 0x7fddb415d5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fddb9385640 1 -- 172.21.15.32:0/2461170127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fddac009a70 con 0x7fddb40d14d0 2023-12-20T06:04:24.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fddb9385640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40d14d0 0x7fddb415ace0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fddb001b880 tx=0x7fddb001ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:24.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fdda9ffb640 1 -- 172.21.15.32:0/2461170127 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00b1d10 con 0x7fddb40d14d0 2023-12-20T06:04:24.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fdda9ffb640 1 -- 172.21.15.32:0/2461170127 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fddb00b1e70 con 0x7fddb40d14d0 2023-12-20T06:04:24.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fdda9ffb640 1 -- 172.21.15.32:0/2461170127 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fddb00b9630 con 0x7fddb40d14d0 2023-12-20T06:04:24.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fddbb5e9640 1 -- 172.21.15.32:0/2461170127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fddb0009420 con 0x7fddb40d14d0 2023-12-20T06:04:24.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fddbb5e9640 1 -- 172.21.15.32:0/2461170127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fddb415f810 con 0x7fddb40d14d0 2023-12-20T06:04:24.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fddbb5e9640 1 -- 172.21.15.32:0/2461170127 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fddb415f390 con 0x7fddb40d14d0 2023-12-20T06:04:24.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fdda9ffb640 1 -- 172.21.15.32:0/2461170127 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 144) v1 ==== 85449+0+0 (secure 0 0 0) 0x7fddb00b7040 con 0x7fddb40d14d0 2023-12-20T06:04:24.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.529+0000 7fdda9ffb640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:24.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.533+0000 7fdda9ffb640 1 -- 172.21.15.32:0/2461170127 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(251..251 src has 1..251) v4 ==== 5765+0+0 (secure 0 0 0) 0x7fddb0120c20 con 0x7fddb40d14d0 2023-12-20T06:04:24.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.533+0000 7fddb8b84640 1 -- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 msgr2=0x7fdd98068230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:24.534 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.533+0000 7fddb8b84640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:24.537 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.537+0000 7fdda9ffb640 1 -- 172.21.15.32:0/2461170127 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fddb415f390 con 0x7fddb40d14d0 2023-12-20T06:04:24.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.713+0000 7fddbb5e9640 1 -- 172.21.15.32:0/2461170127 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fddb40d22e0 con 0x7fdd98065d70 2023-12-20T06:04:24.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.737+0000 7fddb8b84640 1 -- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 msgr2=0x7fdd98068230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:24.737 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:24.737+0000 7fddb8b84640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T06:04:24.850 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:24.849+0000 7fae9c248180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:04:24.878 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:24.877+0000 7f4252049180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:04:24.912 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:24.909+0000 7fae9c248180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:04:24.939 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:24.937+0000 7f4252049180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:04:24.970 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:24.969+0000 7fae9c248180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:04:24.997 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:24.997+0000 7f4252049180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:04:25.061 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:25.057+0000 7fae9c248180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:04:25.088 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:25.085+0000 7f4252049180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:04:25.121 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:25.117+0000 7fae9c248180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:04:25.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:25.141+0000 7fddb8b84640 1 -- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 msgr2=0x7fdd98068230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:25.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:25.141+0000 7fddb8b84640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2023-12-20T06:04:25.147 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:25.145+0000 7f4252049180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:04:25.210 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:25.209+0000 7fae9c248180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:04:25.235 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:25.233+0000 7f4252049180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:04:25.270 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:25.265+0000 7fae9c248180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:04:25.294 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:25.293+0000 7f4252049180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:04:25.421 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:25.417+0000 7fae9c248180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:04:25.445 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:25.445+0000 7f4252049180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:04:25.482 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:25.481+0000 7fae9c248180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:04:25.506 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:25.505+0000 7f4252049180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:04:25.733 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:04:25.733 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:04:25.733 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:04:25.742 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:25.737+0000 7fae9c248180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:04:25.756 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:04:25.756 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:04:25.756 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:04:25.765 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:25.765+0000 7f4252049180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:04:25.885 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:25.881+0000 7fae9c248180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:04:25.908 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:25.905+0000 7f4252049180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:04:25.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:25.945+0000 7fddb8b84640 1 -- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 msgr2=0x7fdd98068230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:25.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:25.945+0000 7fddb8b84640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2023-12-20T06:04:26.016 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:26.013+0000 7fae9c248180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:04:26.039 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:26.037+0000 7f4252049180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:04:26.091 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:26.089+0000 7fae9c248180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:04:26.113 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:26.113+0000 7f4252049180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:04:26.234 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:26.229+0000 7fae9c248180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:04:26.255 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:26.253+0000 7f4252049180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:04:26.804 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:26.801+0000 7fae9c248180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:04:26.821 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:26.821+0000 7f4252049180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:04:26.893 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:26.889+0000 7fae9c248180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:04:26.910 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:26.909+0000 7f4252049180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:04:27.051 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:27.049+0000 7fae9c248180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:04:27.067 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.065+0000 7f4252049180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:04:27.139 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:27.137+0000 7fae9c248180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:04:27.155 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.153+0000 7f4252049180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:04:27.474 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:27.473+0000 7fae9c248180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:04:27.488 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.485+0000 7f4252049180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:04:27.538 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:27.537+0000 7fae9c248180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:04:27.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:27.549+0000 7fddb8b84640 1 -- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 msgr2=0x7fdd98068230 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:27.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:27.549+0000 7fddb8b84640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2023-12-20T06:04:27.552 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.549+0000 7f4252049180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:04:27.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:27.561+0000 7fdda9ffb640 1 -- 172.21.15.32:0/2461170127 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 145) v1 ==== 84956+0+0 (secure 0 0 0) 0x7fddb00ee730 con 0x7fddb40d14d0 2023-12-20T06:04:27.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:27.561+0000 7fdda9ffb640 1 -- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 msgr2=0x7fdd98068230 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:27.563 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:27.561+0000 7fdda9ffb640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:27.680 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.677+0000 7f41d7331640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:27.680 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.677+0000 7f41d7331640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:27.680 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.677+0000 7f41d7331640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:27.680 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.677+0000 7f41d7331640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:27.680 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.677+0000 7f41d7331640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:27.681 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.677+0000 7f41d3b2a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:27.681 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.677+0000 7f41d3b2a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:27.681 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.677+0000 7f41d3b2a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:27.681 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.677+0000 7f41d3b2a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:27.681 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:27.677+0000 7f41d3b2a640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.569+0000 7fdda9ffb640 1 -- 172.21.15.32:0/2461170127 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mgrmap(e 146) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fddb0109770 con 0x7fddb40d14d0 2023-12-20T06:04:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.569+0000 7fdda9ffb640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:28.570 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.569+0000 7fdda9ffb640 1 -- 172.21.15.32:0/2461170127 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "path": "snap_test_dir1", "snap_schedule": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7fdd98033790 con 0x7fdd98065d70 2023-12-20T06:04:28.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.569+0000 7fddb8b84640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:28.576 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.573+0000 7fddb8b84640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdda4007fd0 tx=0x7fdda4009040 comp rx=0 tx=0).ready entity=mgr.12243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:28.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fdda9ffb640 1 -- 172.21.15.32:0/2461170127 <== mgr.12243 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+37 (secure 0 0 0) 0x7fdd98033790 con 0x7fdd98065d70 2023-12-20T06:04:28.650 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule set for path snap_test_dir1 2023-12-20T06:04:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fddbb5e9640 1 -- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 msgr2=0x7fdd98068230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fddbb5e9640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdda4007fd0 tx=0x7fdda4009040 comp rx=0 tx=0).stop 2023-12-20T06:04:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fddbb5e9640 1 -- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40d14d0 msgr2=0x7fddb415ace0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fddbb5e9640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40d14d0 0x7fddb415ace0 secure :-1 s=READY pgs=2515 cs=0 l=1 rev1=1 crypto rx=0x7fddb001b880 tx=0x7fddb001ba10 comp rx=0 tx=0).stop 2023-12-20T06:04:28.653 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fddbb5e9640 1 -- 172.21.15.32:0/2461170127 shutdown_connections 2023-12-20T06:04:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fddbb5e9640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdd98065d70 0x7fdd98068230 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fddbb5e9640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fddb415ce60 0x7fddb40cad80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fddbb5e9640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fddb40d3130 0x7fddb415d5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fddbb5e9640 1 --2- 172.21.15.32:0/2461170127 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fddb40d14d0 0x7fddb415ace0 unknown :-1 s=CLOSED pgs=2515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fddbb5e9640 1 -- 172.21.15.32:0/2461170127 >> 172.21.15.32:0/2461170127 conn(0x7fddb406b5e0 msgr2=0x7fddb40b1940 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.649+0000 7fddbb5e9640 1 -- 172.21.15.32:0/2461170127 shutdown_connections 2023-12-20T06:04:28.654 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.653+0000 7fddbb5e9640 1 -- 172.21.15.32:0/2461170127 wait complete. 2023-12-20T06:04:28.674 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule set for path snap_test_dir1 2023-12-20T06:04:28.675 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule retention add snap_test_dir1 1M 2023-12-20T06:04:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.821+0000 7f74e2a2d640 1 Processor -- start 2023-12-20T06:04:28.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.821+0000 7f74e2a2d640 1 -- start start 2023-12-20T06:04:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74e2a2d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74dc159250 0x7f74dc0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:28.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74e2a2d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74dc159630 con 0x7f74dc0ce290 2023-12-20T06:04:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74e2a2d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f74dc0c9480 con 0x7f74dc070940 2023-12-20T06:04:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74e2a2d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f74dc06cf90 con 0x7f74dc159250 2023-12-20T06:04:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74dbfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f74dc159250 0x7f74dc0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74db7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f74dc0ce290 0x7f74dc070230 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41016/0 (socket says 172.21.15.32:41016) 2023-12-20T06:04:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74db7fe640 1 -- 172.21.15.32:0/216602450 learned_addr learned my addr 172.21.15.32:0/216602450 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74daffd640 1 -- 172.21.15.32:0/216602450 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4160876822 0 0) 0x7f74dc159630 con 0x7f74dc0ce290 2023-12-20T06:04:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74daffd640 1 -- 172.21.15.32:0/216602450 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f74c4003530 con 0x7f74dc0ce290 2023-12-20T06:04:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74dbfff640 1 -- 172.21.15.32:0/216602450 >> v1:172.21.15.39:6789/0 conn(0x7f74dc070940 legacy=0x7f74dc0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:28.828 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74daffd640 1 -- 172.21.15.32:0/216602450 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2431309212 0 0) 0x7f74dc0c9480 con 0x7f74dc070940 2023-12-20T06:04:28.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74dbfff640 1 -- 172.21.15.32:0/216602450 >> v1:172.21.15.32:6789/0 conn(0x7f74dc0ce290 legacy=0x7f74dc070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:28.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.825+0000 7f74dbfff640 1 -- 172.21.15.32:0/216602450 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74dc06d270 con 0x7f74dc159250 2023-12-20T06:04:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74dbfff640 1 --2- 172.21.15.32:0/216602450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74dc159250 0x7f74dc0caa30 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0097c0 tx=0x7f74cc01c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b912dc682648f47 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74daffd640 1 -- 172.21.15.32:0/216602450 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0ac630 con 0x7f74dc159250 2023-12-20T06:04:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74daffd640 1 -- 172.21.15.32:0/216602450 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74cc0ac790 con 0x7f74dc159250 2023-12-20T06:04:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74daffd640 1 -- 172.21.15.32:0/216602450 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74cc0acb20 con 0x7f74dc159250 2023-12-20T06:04:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74e2a2d640 1 -- 172.21.15.32:0/216602450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74dc159250 msgr2=0x7f74dc0caa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74e2a2d640 1 --2- 172.21.15.32:0/216602450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74dc159250 0x7f74dc0caa30 secure :-1 s=READY pgs=2527 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0097c0 tx=0x7f74cc01c450 comp rx=0 tx=0).stop 2023-12-20T06:04:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74e2a2d640 1 -- 172.21.15.32:0/216602450 shutdown_connections 2023-12-20T06:04:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74e2a2d640 1 --2- 172.21.15.32:0/216602450 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74dc159250 0x7f74dc0caa30 unknown :-1 s=CLOSED pgs=2527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:28.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74e2a2d640 1 -- 172.21.15.32:0/216602450 >> 172.21.15.32:0/216602450 conn(0x7f74dc06a510 msgr2=0x7f74dc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74e2a2d640 1 -- 172.21.15.32:0/216602450 shutdown_connections 2023-12-20T06:04:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74e2a2d640 1 -- 172.21.15.32:0/216602450 wait complete. 2023-12-20T06:04:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74e2a2d640 1 Processor -- start 2023-12-20T06:04:28.832 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.829+0000 7f74e2a2d640 1 -- start start 2023-12-20T06:04:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e2a2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74dc070940 0x7f74dc14ca70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e2a2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74dc0ce290 0x7f74dc14d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e2a2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74dc159250 0x7f74dc14d950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:28.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e2a2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc14de90 con 0x7f74dc159250 2023-12-20T06:04:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e2a2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc14d050 con 0x7f74dc0ce290 2023-12-20T06:04:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e2a2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f74dc14f5b0 con 0x7f74dc070940 2023-12-20T06:04:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e0fca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74dc159250 0x7f74dc14d950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e0fca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74dc159250 0x7f74dc14d950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54816/0 (socket says 172.21.15.32:54816) 2023-12-20T06:04:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e0fca640 1 -- 172.21.15.32:0/2571857318 learned_addr learned my addr 172.21.15.32:0/2571857318 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74db7fe640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74dc0ce290 0x7f74dc14d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74dbfff640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74dc070940 0x7f74dc14ca70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e0fca640 1 -- 172.21.15.32:0/2571857318 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74dc070940 msgr2=0x7f74dc14ca70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e0fca640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74dc070940 0x7f74dc14ca70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:28.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e0fca640 1 -- 172.21.15.32:0/2571857318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74dc0ce290 msgr2=0x7f74dc14d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e0fca640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74dc0ce290 0x7f74dc14d410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e0fca640 1 -- 172.21.15.32:0/2571857318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f74cc009420 con 0x7f74dc159250 2023-12-20T06:04:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e0fca640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74dc159250 0x7f74dc14d950 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f74d400c750 tx=0x7f74d400cc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74d8ff9640 1 -- 172.21.15.32:0/2571857318 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d4003db0 con 0x7f74dc159250 2023-12-20T06:04:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74d8ff9640 1 -- 172.21.15.32:0/2571857318 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f74d400ce60 con 0x7f74dc159250 2023-12-20T06:04:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74d8ff9640 1 -- 172.21.15.32:0/2571857318 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f74d40052b0 con 0x7f74dc159250 2023-12-20T06:04:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e2a2d640 1 -- 172.21.15.32:0/2571857318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f74dc14f890 con 0x7f74dc159250 2023-12-20T06:04:28.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e2a2d640 1 -- 172.21.15.32:0/2571857318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f74dc14f260 con 0x7f74dc159250 2023-12-20T06:04:28.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.833+0000 7f74e2a2d640 1 -- 172.21.15.32:0/2571857318 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f74dc06cd20 con 0x7f74dc159250 2023-12-20T06:04:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.837+0000 7f74d8ff9640 1 -- 172.21.15.32:0/2571857318 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 146) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f74d4005410 con 0x7f74dc159250 2023-12-20T06:04:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.837+0000 7f74d8ff9640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f74c8065900 0x7f74c8067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.837+0000 7f74d8ff9640 1 -- 172.21.15.32:0/2571857318 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7f74d4082f90 con 0x7f74dc159250 2023-12-20T06:04:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.837+0000 7f74dbfff640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f74c8065900 0x7f74c8067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:28.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.837+0000 7f74dbfff640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f74c8065900 0x7f74c8067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0ae000 tx=0x7f74cc0a7000 comp rx=0 tx=0).ready entity=mgr.12243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:28.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:28.841+0000 7f74d8ff9640 1 -- 172.21.15.32:0/2571857318 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f74d40509d0 con 0x7f74dc159250 2023-12-20T06:04:29.020 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.017+0000 7f74e2a2d640 1 -- 172.21.15.32:0/2571857318 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "path": "snap_test_dir1", "retention_spec_or_period": "1M", "target": ["mon-mgr", ""]}) v1 -- 0x7f74dc0caf70 con 0x7f74c8065900 2023-12-20T06:04:29.072 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.069+0000 7f74d8ff9640 1 -- 172.21.15.32:0/2571857318 <== mgr.12243 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+39 (secure 0 0 0) 0x7f74cc0bc0d0 con 0x7f74c8065900 2023-12-20T06:04:29.073 INFO:teuthology.orchestra.run.smithi032.stdout:Retention added to path snap_test_dir1 2023-12-20T06:04:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 -- 172.21.15.32:0/2571857318 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f74c8065900 msgr2=0x7f74c8067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f74c8065900 0x7f74c8067dc0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f74cc0ae000 tx=0x7f74cc0a7000 comp rx=0 tx=0).stop 2023-12-20T06:04:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 -- 172.21.15.32:0/2571857318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74dc159250 msgr2=0x7f74dc14d950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74dc159250 0x7f74dc14d950 secure :-1 s=READY pgs=2528 cs=0 l=1 rev1=1 crypto rx=0x7f74d400c750 tx=0x7f74d400cc10 comp rx=0 tx=0).stop 2023-12-20T06:04:29.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 -- 172.21.15.32:0/2571857318 shutdown_connections 2023-12-20T06:04:29.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f74c8065900 0x7f74c8067dc0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f74dc159250 0x7f74dc14d950 unknown :-1 s=CLOSED pgs=2528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f74dc0ce290 0x7f74dc14d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 --2- 172.21.15.32:0/2571857318 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f74dc070940 0x7f74dc14ca70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 -- 172.21.15.32:0/2571857318 >> 172.21.15.32:0/2571857318 conn(0x7f74dc06a510 msgr2=0x7f74dc06c610 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:29.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 -- 172.21.15.32:0/2571857318 shutdown_connections 2023-12-20T06:04:29.076 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.073+0000 7f74e2a2d640 1 -- 172.21.15.32:0/2571857318 wait complete. 2023-12-20T06:04:29.094 DEBUG:tasks.cephfs.test_snap_schedules:res=Retention added to path snap_test_dir1 2023-12-20T06:04:29.099 INFO:tasks.cephfs.test_snap_schedules:expecting snap snap_test_dir1/.snap/scheduled-2023-12-20-06_05_00 in ~31s... 2023-12-20T06:04:29.099 DEBUG:tasks.cephfs.test_snap_schedules:expected_schedule: ['1M'], expected_retention: [{'M': 1}] 2023-12-20T06:04:29.100 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule list --format json snap_test_dir1 2023-12-20T06:04:29.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.241+0000 7f725bbce640 1 Processor -- start 2023-12-20T06:04:29.244 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.241+0000 7f725bbce640 1 -- start start 2023-12-20T06:04:29.246 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f725bbce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7254164670 0x7f72540ca1a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f725bbce640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f725406da20 0x7f72540cc3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f725bbce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7254070230 0x7f7254070660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f725bbce640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f72540cb0b0 con 0x7f725406da20 2023-12-20T06:04:29.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f725bbce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7254070ba0 con 0x7f7254164670 2023-12-20T06:04:29.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f725bbce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72540ca810 con 0x7f7254070230 2023-12-20T06:04:29.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f7259169640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f725406da20 0x7f72540cc3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:29.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f7259169640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f725406da20 0x7f72540cc3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54818/0 (socket says 172.21.15.32:54818) 2023-12-20T06:04:29.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f7259169640 1 -- 172.21.15.32:0/2348267629 learned_addr learned my addr 172.21.15.32:0/2348267629 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:29.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f7259169640 1 -- 172.21.15.32:0/2348267629 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7254070230 msgr2=0x7f7254070660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:29.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f7259169640 1 --2- 172.21.15.32:0/2348267629 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7254070230 0x7f7254070660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f7259169640 1 -- 172.21.15.32:0/2348267629 >> v2:172.21.15.39:3300/0 conn(0x7f7254164670 msgr2=0x7f72540ca1a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:29.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f7259169640 1 --2- 172.21.15.32:0/2348267629 >> v2:172.21.15.39:3300/0 conn(0x7f7254164670 0x7f72540ca1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.245+0000 7f7259169640 1 -- 172.21.15.32:0/2348267629 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72540caaf0 con 0x7f725406da20 2023-12-20T06:04:29.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f7259169640 1 --2- 172.21.15.32:0/2348267629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f725406da20 0x7f72540cc3b0 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f725406e430 tx=0x7f724401c3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e23108537972dcca server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f7258968640 1 -- 172.21.15.32:0/2348267629 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440ab600 con 0x7f725406da20 2023-12-20T06:04:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f7258968640 1 -- 172.21.15.32:0/2348267629 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72440a7050 con 0x7f725406da20 2023-12-20T06:04:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f7258968640 1 -- 172.21.15.32:0/2348267629 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440ab980 con 0x7f725406da20 2023-12-20T06:04:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f725bbce640 1 -- 172.21.15.32:0/2348267629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f725406da20 msgr2=0x7f72540cc3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f725bbce640 1 --2- 172.21.15.32:0/2348267629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f725406da20 0x7f72540cc3b0 secure :-1 s=READY pgs=2529 cs=0 l=1 rev1=1 crypto rx=0x7f725406e430 tx=0x7f724401c3b0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f725bbce640 1 -- 172.21.15.32:0/2348267629 shutdown_connections 2023-12-20T06:04:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f725bbce640 1 --2- 172.21.15.32:0/2348267629 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7254070230 0x7f7254070660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f725bbce640 1 --2- 172.21.15.32:0/2348267629 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f725406da20 0x7f72540cc3b0 unknown :-1 s=CLOSED pgs=2529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f725bbce640 1 --2- 172.21.15.32:0/2348267629 >> v2:172.21.15.39:3300/0 conn(0x7f7254164670 0x7f72540ca1a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f725bbce640 1 -- 172.21.15.32:0/2348267629 >> 172.21.15.32:0/2348267629 conn(0x7f725406a510 msgr2=0x7f72540ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:29.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f725bbce640 1 -- 172.21.15.32:0/2348267629 shutdown_connections 2023-12-20T06:04:29.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f725bbce640 1 -- 172.21.15.32:0/2348267629 wait complete. 2023-12-20T06:04:29.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f725bbce640 1 Processor -- start 2023-12-20T06:04:29.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.249+0000 7f725bbce640 1 -- start start 2023-12-20T06:04:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725bbce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f725406da20 0x7f7254153bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725bbce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7254070230 0x7f7254154390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725bbce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7254164670 0x7f72541548d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725bbce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540c6b30 con 0x7f7254164670 2023-12-20T06:04:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725bbce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f72540c6cc0 con 0x7f725406da20 2023-12-20T06:04:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725bbce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f72540cde80 con 0x7f7254070230 2023-12-20T06:04:29.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725a16b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7254164670 0x7f72541548d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f7259169640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7254070230 0x7f7254154390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725996a640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f725406da20 0x7f7254153bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f7259169640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7254070230 0x7f7254154390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:56354/0 (socket says 172.21.15.32:56354) 2023-12-20T06:04:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f7259169640 1 -- 172.21.15.32:0/2598780113 learned_addr learned my addr 172.21.15.32:0/2598780113 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f7259169640 1 -- 172.21.15.32:0/2598780113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f725406da20 msgr2=0x7f7254153bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f7259169640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f725406da20 0x7f7254153bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f7259169640 1 -- 172.21.15.32:0/2598780113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7254164670 msgr2=0x7f72541548d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f7259169640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7254164670 0x7f72541548d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f7259169640 1 -- 172.21.15.32:0/2598780113 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f72440094d0 con 0x7f7254070230 2023-12-20T06:04:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725a16b640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7254164670 0x7f72541548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:29.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f7259169640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7254070230 0x7f7254154390 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f72440ae040 tx=0x7f724401abf0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f724a7fc640 1 -- 172.21.15.32:0/2598780113 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440b6620 con 0x7f7254070230 2023-12-20T06:04:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f724a7fc640 1 -- 172.21.15.32:0/2598780113 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f72440b67a0 con 0x7f7254070230 2023-12-20T06:04:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725bbce640 1 -- 172.21.15.32:0/2598780113 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f72540ce160 con 0x7f7254070230 2023-12-20T06:04:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f724a7fc640 1 -- 172.21.15.32:0/2598780113 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f72440b6b50 con 0x7f7254070230 2023-12-20T06:04:29.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725bbce640 1 -- 172.21.15.32:0/2598780113 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f72541540f0 con 0x7f7254070230 2023-12-20T06:04:29.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.253+0000 7f725bbce640 1 -- 172.21.15.32:0/2598780113 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f72540c6e30 con 0x7f7254070230 2023-12-20T06:04:29.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.257+0000 7f724a7fc640 1 -- 172.21.15.32:0/2598780113 <== mon.2 v2:172.21.15.39:3301/0 4 ==== mgrmap(e 146) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f72440b2040 con 0x7f7254070230 2023-12-20T06:04:29.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.257+0000 7f724a7fc640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7238065900 0x7f7238067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.257+0000 7f724a7fc640 1 -- 172.21.15.32:0/2598780113 <== mon.2 v2:172.21.15.39:3301/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7f724411d740 con 0x7f7254070230 2023-12-20T06:04:29.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.257+0000 7f725996a640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7238065900 0x7f7238067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:29.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.257+0000 7f725996a640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7238065900 0x7f7238067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7250009990 tx=0x7f7250009110 comp rx=0 tx=0).ready entity=mgr.12243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:29.261 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.261+0000 7f724a7fc640 1 -- 172.21.15.32:0/2598780113 <== mon.2 v2:172.21.15.39:3301/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f72440eb3d0 con 0x7f7254070230 2023-12-20T06:04:29.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.437+0000 7f725bbce640 1 -- 172.21.15.32:0/2598780113 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "path": "snap_test_dir1", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7254164a50 con 0x7f7238065900 2023-12-20T06:04:29.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.441+0000 7f724a7fc640 1 -- 172.21.15.32:0/2598780113 <== mgr.12243 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+72 (secure 0 0 0) 0x7f72500095e0 con 0x7f7238065900 2023-12-20T06:04:29.445 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:29.445 INFO:teuthology.orchestra.run.smithi032.stdout:{"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-20T06:04:29.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 -- 172.21.15.32:0/2598780113 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7238065900 msgr2=0x7f7238067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7238065900 0x7f7238067dc0 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f7250009990 tx=0x7f7250009110 comp rx=0 tx=0).stop 2023-12-20T06:04:29.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 -- 172.21.15.32:0/2598780113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7254070230 msgr2=0x7f7254154390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7254070230 0x7f7254154390 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f72440ae040 tx=0x7f724401abf0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 -- 172.21.15.32:0/2598780113 shutdown_connections 2023-12-20T06:04:29.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7238065900 0x7f7238067dc0 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7254164670 0x7f72541548d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.447 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7254070230 0x7f7254154390 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 --2- 172.21.15.32:0/2598780113 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f725406da20 0x7f7254153bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 -- 172.21.15.32:0/2598780113 >> 172.21.15.32:0/2598780113 conn(0x7f725406a510 msgr2=0x7f72540c3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:29.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 -- 172.21.15.32:0/2598780113 shutdown_connections 2023-12-20T06:04:29.448 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.445+0000 7f725bbce640 1 -- 172.21.15.32:0/2598780113 wait complete. 2023-12-20T06:04:29.466 DEBUG:tasks.cephfs.test_snap_schedules:res= {"path": "snap_test_dir1", "schedule": ["1M"], "retention": [{"M": 1}]} 2023-12-20T06:04:29.467 DEBUG:tasks.cephfs.test_snap_schedules:json_res: {'path': 'snap_test_dir1', 'schedule': ['1M'], 'retention': [{'M': 1}]} 2023-12-20T06:04:29.467 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs snap-schedule remove snap_test_dir1 2023-12-20T06:04:29.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.613+0000 7f10e22a8640 1 Processor -- start 2023-12-20T06:04:29.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.613+0000 7f10e22a8640 1 -- start start 2023-12-20T06:04:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10e22a8640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f10dc165270 0x7f10dc0701a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10e22a8640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10dc0cd0e0 con 0x7f10dc0c9820 2023-12-20T06:04:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10e22a8640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f10dc165820 con 0x7f10dc0c1a10 2023-12-20T06:04:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10e22a8640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f10dc165b00 con 0x7f10dc165270 2023-12-20T06:04:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10e0aa5640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f10dc0c9820 0x7f10dc164b60 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41018/0 (socket says 172.21.15.32:41018) 2023-12-20T06:04:29.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10e0aa5640 1 -- 172.21.15.32:0/3712244051 learned_addr learned my addr 172.21.15.32:0/3712244051 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10e1aa7640 1 --2- 172.21.15.32:0/3712244051 >> v2:172.21.15.39:3300/0 conn(0x7f10dc165270 0x7f10dc0701a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10d3fff640 1 -- 172.21.15.32:0/3712244051 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3538440179 0 0) 0x7f10dc0cd0e0 con 0x7f10dc0c9820 2023-12-20T06:04:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10d3fff640 1 -- 172.21.15.32:0/3712244051 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f10c8003530 con 0x7f10dc0c9820 2023-12-20T06:04:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10e1aa7640 1 -- 172.21.15.32:0/3712244051 >> v1:172.21.15.39:6789/0 conn(0x7f10dc0c1a10 legacy=0x7f10dc0c9110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10d3fff640 1 -- 172.21.15.32:0/3712244051 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3388267857 0 0) 0x7f10dc165820 con 0x7f10dc0c1a10 2023-12-20T06:04:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10e1aa7640 1 -- 172.21.15.32:0/3712244051 >> v1:172.21.15.32:6789/0 conn(0x7f10dc0c9820 legacy=0x7f10dc164b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.620 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.617+0000 7f10e1aa7640 1 -- 172.21.15.32:0/3712244051 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10dc0cca90 con 0x7f10dc165270 2023-12-20T06:04:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.621+0000 7f10e1aa7640 1 --2- 172.21.15.32:0/3712244051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10dc165270 0x7f10dc0701a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009d90 tx=0x7f10d401c890 comp rx=0 tx=0).ready entity=mon.1 client_cookie=eb1610963ee150a2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.621+0000 7f10d3fff640 1 -- 172.21.15.32:0/3712244051 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40a58b0 con 0x7f10dc165270 2023-12-20T06:04:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.621+0000 7f10d3fff640 1 -- 172.21.15.32:0/3712244051 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d40a5a10 con 0x7f10dc165270 2023-12-20T06:04:29.622 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.621+0000 7f10d3fff640 1 -- 172.21.15.32:0/3712244051 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40a5da0 con 0x7f10dc165270 2023-12-20T06:04:29.623 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.621+0000 7f10e22a8640 1 -- 172.21.15.32:0/3712244051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10dc165270 msgr2=0x7f10dc0701a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.621+0000 7f10e22a8640 1 --2- 172.21.15.32:0/3712244051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10dc165270 0x7f10dc0701a0 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f10d4009d90 tx=0x7f10d401c890 comp rx=0 tx=0).stop 2023-12-20T06:04:29.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.621+0000 7f10e22a8640 1 -- 172.21.15.32:0/3712244051 shutdown_connections 2023-12-20T06:04:29.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.621+0000 7f10e22a8640 1 --2- 172.21.15.32:0/3712244051 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10dc165270 0x7f10dc0701a0 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.621+0000 7f10e22a8640 1 -- 172.21.15.32:0/3712244051 >> 172.21.15.32:0/3712244051 conn(0x7f10dc06c060 msgr2=0x7f10dc06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:29.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.621+0000 7f10e22a8640 1 -- 172.21.15.32:0/3712244051 shutdown_connections 2023-12-20T06:04:29.624 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.621+0000 7f10e22a8640 1 -- 172.21.15.32:0/3712244051 wait complete. 2023-12-20T06:04:29.625 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e22a8640 1 Processor -- start 2023-12-20T06:04:29.626 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e22a8640 1 -- start start 2023-12-20T06:04:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e22a8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10dc0c1a10 0x7f10dc15b5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.628 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e22a8640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10dc0c9820 0x7f10dc1475b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e22a8640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10dc165270 0x7f10dc147af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e22a8640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc158170 con 0x7f10dc165270 2023-12-20T06:04:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e22a8640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc159610 con 0x7f10dc0c9820 2023-12-20T06:04:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e22a8640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f10dc0c43c0 con 0x7f10dc0c1a10 2023-12-20T06:04:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e1aa7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10dc165270 0x7f10dc147af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e1aa7640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10dc165270 0x7f10dc147af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54832/0 (socket says 172.21.15.32:54832) 2023-12-20T06:04:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e0aa5640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10dc0c9820 0x7f10dc1475b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e1aa7640 1 -- 172.21.15.32:0/2004059894 learned_addr learned my addr 172.21.15.32:0/2004059894 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:29.629 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e12a6640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10dc0c1a10 0x7f10dc15b5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e1aa7640 1 -- 172.21.15.32:0/2004059894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10dc0c1a10 msgr2=0x7f10dc15b5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e1aa7640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10dc0c1a10 0x7f10dc15b5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e1aa7640 1 -- 172.21.15.32:0/2004059894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10dc0c9820 msgr2=0x7f10dc1475b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e1aa7640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10dc0c9820 0x7f10dc1475b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e1aa7640 1 -- 172.21.15.32:0/2004059894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f10d4009a70 con 0x7f10dc165270 2023-12-20T06:04:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e0aa5640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10dc0c9820 0x7f10dc1475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e1aa7640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10dc165270 0x7f10dc147af0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f10d40b0040 tx=0x7f10d4019630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10d1ffb640 1 -- 172.21.15.32:0/2004059894 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d4003ba0 con 0x7f10dc165270 2023-12-20T06:04:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10d1ffb640 1 -- 172.21.15.32:0/2004059894 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f10d4003d00 con 0x7f10dc165270 2023-12-20T06:04:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10d1ffb640 1 -- 172.21.15.32:0/2004059894 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f10d40ad890 con 0x7f10dc165270 2023-12-20T06:04:29.630 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e22a8640 1 -- 172.21.15.32:0/2004059894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f10dc0c46a0 con 0x7f10dc165270 2023-12-20T06:04:29.631 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.625+0000 7f10e22a8640 1 -- 172.21.15.32:0/2004059894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f10dc0b3690 con 0x7f10dc165270 2023-12-20T06:04:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.629+0000 7f10e22a8640 1 -- 172.21.15.32:0/2004059894 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f10dc1597d0 con 0x7f10dc165270 2023-12-20T06:04:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.629+0000 7f10d1ffb640 1 -- 172.21.15.32:0/2004059894 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 146) v1 ==== 85083+0+0 (secure 0 0 0) 0x7f10d40b4050 con 0x7f10dc165270 2023-12-20T06:04:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.629+0000 7f10d1ffb640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f10c0065900 0x7f10c0067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.629+0000 7f10d1ffb640 1 -- 172.21.15.32:0/2004059894 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7f10d411df90 con 0x7f10dc165270 2023-12-20T06:04:29.632 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.629+0000 7f10e12a6640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f10c0065900 0x7f10c0067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:29.633 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.633+0000 7f10e12a6640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f10c0065900 0x7f10c0067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f10d8007fd0 tx=0x7f10d8009040 comp rx=0 tx=0).ready entity=mgr.12243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:29.636 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.633+0000 7f10d1ffb640 1 -- 172.21.15.32:0/2004059894 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f10d40ebb50 con 0x7f10dc165270 2023-12-20T06:04:29.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.817+0000 7f10e22a8640 1 -- 172.21.15.32:0/2004059894 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "fs snap-schedule remove", "path": "snap_test_dir1", "target": ["mon-mgr", ""]}) v1 -- 0x7f10dc074100 con 0x7f10c0065900 2023-12-20T06:04:29.883 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.881+0000 7f10d1ffb640 1 -- 172.21.15.32:0/2004059894 <== mgr.12243 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 ) v1 ==== 8+0+41 (secure 0 0 0) 0x7f10dc074100 con 0x7f10c0065900 2023-12-20T06:04:29.883 INFO:teuthology.orchestra.run.smithi032.stdout:Schedule removed for path snap_test_dir1 2023-12-20T06:04:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 -- 172.21.15.32:0/2004059894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f10c0065900 msgr2=0x7f10c0067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f10c0065900 0x7f10c0067dc0 secure :-1 s=READY pgs=25 cs=0 l=1 rev1=1 crypto rx=0x7f10d8007fd0 tx=0x7f10d8009040 comp rx=0 tx=0).stop 2023-12-20T06:04:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 -- 172.21.15.32:0/2004059894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10dc165270 msgr2=0x7f10dc147af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10dc165270 0x7f10dc147af0 secure :-1 s=READY pgs=2531 cs=0 l=1 rev1=1 crypto rx=0x7f10d40b0040 tx=0x7f10d4019630 comp rx=0 tx=0).stop 2023-12-20T06:04:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 -- 172.21.15.32:0/2004059894 shutdown_connections 2023-12-20T06:04:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f10c0065900 0x7f10c0067dc0 unknown :-1 s=CLOSED pgs=25 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f10dc165270 0x7f10dc147af0 unknown :-1 s=CLOSED pgs=2531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f10dc0c9820 0x7f10dc1475b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 --2- 172.21.15.32:0/2004059894 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f10dc0c1a10 0x7f10dc15b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 -- 172.21.15.32:0/2004059894 >> 172.21.15.32:0/2004059894 conn(0x7f10dc06c060 msgr2=0x7f10dc0c8fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 -- 172.21.15.32:0/2004059894 shutdown_connections 2023-12-20T06:04:29.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:29.885+0000 7f10e22a8640 1 -- 172.21.15.32:0/2004059894 wait complete. 2023-12-20T06:04:29.905 DEBUG:tasks.cephfs.test_snap_schedules:res=Schedule removed for path snap_test_dir1 2023-12-20T06:04:29.905 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:29.905 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec ls snap_test_dir1/.snap) 2023-12-20T06:04:29.918 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:29.918 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest/mnt.0 && exec rmdir snap_test_dir1) 2023-12-20T06:04:29.971 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr module disable snap_schedule 2023-12-20T06:04:30.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdeca5cb640 1 Processor -- start 2023-12-20T06:04:30.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdeca5cb640 1 -- start start 2023-12-20T06:04:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdeca5cb640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdec415a920 0x7fdec40ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdeca5cb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fdec40b0010 0x7fdec40b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdeca5cb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdec40c9ff0 con 0x7fdec40cafa0 2023-12-20T06:04:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdeca5cb640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec40b4510 con 0x7fdec415a920 2023-12-20T06:04:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdeca5cb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fdec40b09e0 con 0x7fdec40b0010 2023-12-20T06:04:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdec95c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdec415a920 0x7fdec40ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdec95c9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fdec415a920 0x7fdec40ca690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54860/0 (socket says 172.21.15.32:54860) 2023-12-20T06:04:30.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdec95c9640 1 -- 172.21.15.32:0/1053734510 learned_addr learned my addr 172.21.15.32:0/1053734510 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdec8dc8640 1 --2- 172.21.15.32:0/1053734510 >> v2:172.21.15.39:3300/0 conn(0x7fdec40b0010 0x7fdec40b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdec95c9640 1 -- 172.21.15.32:0/1053734510 >> v2:172.21.15.39:3300/0 conn(0x7fdec40b0010 msgr2=0x7fdec40b04a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdec95c9640 1 --2- 172.21.15.32:0/1053734510 >> v2:172.21.15.39:3300/0 conn(0x7fdec40b0010 0x7fdec40b04a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdec95c9640 1 -- 172.21.15.32:0/1053734510 >> v1:172.21.15.39:6789/0 conn(0x7fdec40cafa0 legacy=0x7fdec40cb410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:30.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.157+0000 7fdec95c9640 1 -- 172.21.15.32:0/1053734510 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec40c93e0 con 0x7fdec415a920 2023-12-20T06:04:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdec95c9640 1 --2- 172.21.15.32:0/1053734510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec415a920 0x7fdec40ca690 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fdec40b1160 tx=0x7fdec001c490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f45b1b194137aea8 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdebbfff640 1 -- 172.21.15.32:0/1053734510 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00aa5f0 con 0x7fdec415a920 2023-12-20T06:04:30.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdebbfff640 1 -- 172.21.15.32:0/1053734510 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdec00aa750 con 0x7fdec415a920 2023-12-20T06:04:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdebbfff640 1 -- 172.21.15.32:0/1053734510 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdec00aaae0 con 0x7fdec415a920 2023-12-20T06:04:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1053734510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec415a920 msgr2=0x7fdec40ca690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdeca5cb640 1 --2- 172.21.15.32:0/1053734510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec415a920 0x7fdec40ca690 secure :-1 s=READY pgs=2533 cs=0 l=1 rev1=1 crypto rx=0x7fdec40b1160 tx=0x7fdec001c490 comp rx=0 tx=0).stop 2023-12-20T06:04:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1053734510 shutdown_connections 2023-12-20T06:04:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdeca5cb640 1 --2- 172.21.15.32:0/1053734510 >> v2:172.21.15.39:3300/0 conn(0x7fdec40b0010 0x7fdec40b04a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdeca5cb640 1 --2- 172.21.15.32:0/1053734510 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec415a920 0x7fdec40ca690 unknown :-1 s=CLOSED pgs=2533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1053734510 >> 172.21.15.32:0/1053734510 conn(0x7fdec406c060 msgr2=0x7fdec406c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1053734510 shutdown_connections 2023-12-20T06:04:30.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.161+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1053734510 wait complete. 2023-12-20T06:04:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeca5cb640 1 Processor -- start 2023-12-20T06:04:30.165 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeca5cb640 1 -- start start 2023-12-20T06:04:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeca5cb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec40b0010 0x7fdec40c20c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeca5cb640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdec40cafa0 0x7fdec4158cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeca5cb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec415a920 0x7fdec4159210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeca5cb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec41455c0 con 0x7fdec415a920 2023-12-20T06:04:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeca5cb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4144800 con 0x7fdec40b0010 2023-12-20T06:04:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeca5cb640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdec4144ae0 con 0x7fdec40cafa0 2023-12-20T06:04:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec9dca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec415a920 0x7fdec4159210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec8dc8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdec40cafa0 0x7fdec4158cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:30.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec95c9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec40b0010 0x7fdec40c20c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec8dc8640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdec40cafa0 0x7fdec4158cd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:56404/0 (socket says 172.21.15.32:56404) 2023-12-20T06:04:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec8dc8640 1 -- 172.21.15.32:0/1861991259 learned_addr learned my addr 172.21.15.32:0/1861991259 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec9dca640 1 -- 172.21.15.32:0/1861991259 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdec40cafa0 msgr2=0x7fdec4158cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec9dca640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdec40cafa0 0x7fdec4158cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec9dca640 1 -- 172.21.15.32:0/1861991259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec40b0010 msgr2=0x7fdec40c20c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec9dca640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec40b0010 0x7fdec40c20c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec9dca640 1 -- 172.21.15.32:0/1861991259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdec00094d0 con 0x7fdec415a920 2023-12-20T06:04:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec8dc8640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdec40cafa0 0x7fdec4158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec95c9640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec40b0010 0x7fdec40c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdec9dca640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec415a920 0x7fdec4159210 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00c780 tx=0x7fdebc00cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:30.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeb9ffb640 1 -- 172.21.15.32:0/1861991259 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc00aa50 con 0x7fdec415a920 2023-12-20T06:04:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeb9ffb640 1 -- 172.21.15.32:0/1861991259 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fdebc00abb0 con 0x7fdec415a920 2023-12-20T06:04:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeb9ffb640 1 -- 172.21.15.32:0/1861991259 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fdebc00ce30 con 0x7fdec415a920 2023-12-20T06:04:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1861991259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdec4144dc0 con 0x7fdec415a920 2023-12-20T06:04:30.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.165+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1861991259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdec40c2600 con 0x7fdec415a920 2023-12-20T06:04:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.169+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1861991259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdec41452e0 con 0x7fdec415a920 2023-12-20T06:04:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.169+0000 7fdeb9ffb640 1 -- 172.21.15.32:0/1861991259 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 146) v1 ==== 85083+0+0 (secure 0 0 0) 0x7fdebc012420 con 0x7fdec415a920 2023-12-20T06:04:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.169+0000 7fdeb9ffb640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdea8065900 0x7fdea8067dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.169+0000 7fdeb9ffb640 1 -- 172.21.15.32:0/1861991259 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7fdebc083140 con 0x7fdec415a920 2023-12-20T06:04:30.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.169+0000 7fdec95c9640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdea8065900 0x7fdea8067dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:30.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.169+0000 7fdec95c9640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdea8065900 0x7fdea8067dc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdec00ae000 tx=0x7fdec00b5040 comp rx=0 tx=0).ready entity=mgr.12243 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:30.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.173+0000 7fdeb9ffb640 1 -- 172.21.15.32:0/1861991259 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fdec41452e0 con 0x7fdec415a920 2023-12-20T06:04:30.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.369+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1861991259 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "mgr module disable", "module": "snap_schedule"} v 0) v1 -- 0x7fdec41452e0 con 0x7fdec415a920 2023-12-20T06:04:30.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.525+0000 7fdeb9ffb640 1 -- 172.21.15.32:0/1861991259 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 147) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fdebc051000 con 0x7fdec415a920 2023-12-20T06:04:30.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeb9ffb640 1 -- 172.21.15.32:0/1861991259 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "mgr module disable", "module": "snap_schedule"}]=0 v147) v1 ==== 93+0+0 (secure 0 0 0) 0x7fdebc080880 con 0x7fdec415a920 2023-12-20T06:04:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1861991259 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdea8065900 msgr2=0x7fdea8067dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdea8065900 0x7fdea8067dc0 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fdec00ae000 tx=0x7fdec00b5040 comp rx=0 tx=0).stop 2023-12-20T06:04:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1861991259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec415a920 msgr2=0x7fdec4159210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec415a920 0x7fdec4159210 secure :-1 s=READY pgs=2534 cs=0 l=1 rev1=1 crypto rx=0x7fdebc00c780 tx=0x7fdebc00cc40 comp rx=0 tx=0).stop 2023-12-20T06:04:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1861991259 shutdown_connections 2023-12-20T06:04:30.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fdea8065900 0x7fdea8067dc0 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdec415a920 0x7fdec4159210 unknown :-1 s=CLOSED pgs=2534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdec40cafa0 0x7fdec4158cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 --2- 172.21.15.32:0/1861991259 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdec40b0010 0x7fdec40c20c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1861991259 >> 172.21.15.32:0/1861991259 conn(0x7fdec406c060 msgr2=0x7fdec40aca80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1861991259 shutdown_connections 2023-12-20T06:04:30.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:30.529+0000 7fdeca5cb640 1 -- 172.21.15.32:0/1861991259 wait complete. 2023-12-20T06:04:30.553 INFO:teuthology.nuke.actions:Clearing teuthology firewall rules... 2023-12-20T06:04:30.553 DEBUG:teuthology.orchestra.run.smithi032:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:04:30.667 DEBUG:teuthology.orchestra.run.smithi039:> sudo sh -c 'iptables-save | grep -v teuthology | iptables-restore' 2023-12-20T06:04:30.707 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:30.705+0000 7f560c28d180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:04:30.714 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:30.713+0000 7fe7e0848180 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2023-12-20T06:04:30.756 INFO:teuthology.nuke.actions:Cleared teuthology firewall rules. 2023-12-20T06:04:30.757 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:04:30.757 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:30.767 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:04:30.767 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:30.818 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:04:31.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.001+0000 7f07f0441640 1 Processor -- start 2023-12-20T06:04:31.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.001+0000 7f07f0441640 1 -- start start 2023-12-20T06:04:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07f0441640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f07e815cdb0 0x7f07e80d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07f0441640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f07e80d2740 0x7f07e80cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07f0441640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f07e80b6ca0 con 0x7f07e80d1120 2023-12-20T06:04:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07f0441640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e815d190 con 0x7f07e80d2740 2023-12-20T06:04:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07f0441640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f07e80d16a0 con 0x7f07e815cdb0 2023-12-20T06:04:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07ee9de640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f07e80d1120 0x7f07e80b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41038/0 (socket says 172.21.15.32:41038) 2023-12-20T06:04:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07ee9de640 1 -- 172.21.15.32:0/2434791581 learned_addr learned my addr 172.21.15.32:0/2434791581 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07ed9dc640 1 --2- 172.21.15.32:0/2434791581 >> v2:172.21.15.32:3300/0 conn(0x7f07e80d2740 0x7f07e80cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07ee1dd640 1 --2- 172.21.15.32:0/2434791581 >> v2:172.21.15.39:3300/0 conn(0x7f07e815cdb0 0x7f07e80d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07ed1db640 1 -- 172.21.15.32:0/2434791581 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1786298300 0 0) 0x7f07e00014a0 con 0x7f07e80d1120 2023-12-20T06:04:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07ed1db640 1 -- 172.21.15.32:0/2434791581 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f07d4003530 con 0x7f07e80d1120 2023-12-20T06:04:31.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07ed9dc640 1 -- 172.21.15.32:0/2434791581 >> v2:172.21.15.39:3300/0 conn(0x7f07e815cdb0 msgr2=0x7f07e80d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07ed9dc640 1 --2- 172.21.15.32:0/2434791581 >> v2:172.21.15.39:3300/0 conn(0x7f07e815cdb0 0x7f07e80d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07ed9dc640 1 -- 172.21.15.32:0/2434791581 >> v1:172.21.15.32:6789/0 conn(0x7f07e80d1120 legacy=0x7f07e80b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.005+0000 7f07ed9dc640 1 -- 172.21.15.32:0/2434791581 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07e80d1980 con 0x7f07e80d2740 2023-12-20T06:04:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07ed9dc640 1 --2- 172.21.15.32:0/2434791581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80d2740 0x7f07e80cf570 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f07e80667b0 tx=0x7f07d801ec50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=908d21df01234747 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07ed1db640 1 -- 172.21.15.32:0/2434791581 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80a8950 con 0x7f07e80d2740 2023-12-20T06:04:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07ed1db640 1 -- 172.21.15.32:0/2434791581 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d80a8ab0 con 0x7f07e80d2740 2023-12-20T06:04:31.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07ed1db640 1 -- 172.21.15.32:0/2434791581 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80a8e20 con 0x7f07e80d2740 2023-12-20T06:04:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07f0441640 1 -- 172.21.15.32:0/2434791581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80d2740 msgr2=0x7f07e80cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07f0441640 1 --2- 172.21.15.32:0/2434791581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80d2740 0x7f07e80cf570 secure :-1 s=READY pgs=2539 cs=0 l=1 rev1=1 crypto rx=0x7f07e80667b0 tx=0x7f07d801ec50 comp rx=0 tx=0).stop 2023-12-20T06:04:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07f0441640 1 -- 172.21.15.32:0/2434791581 shutdown_connections 2023-12-20T06:04:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07f0441640 1 --2- 172.21.15.32:0/2434791581 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80d2740 0x7f07e80cf570 unknown :-1 s=CLOSED pgs=2539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07f0441640 1 --2- 172.21.15.32:0/2434791581 >> v2:172.21.15.39:3300/0 conn(0x7f07e815cdb0 0x7f07e80d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07f0441640 1 -- 172.21.15.32:0/2434791581 >> 172.21.15.32:0/2434791581 conn(0x7f07e8065ee0 msgr2=0x7f07e806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07f0441640 1 -- 172.21.15.32:0/2434791581 shutdown_connections 2023-12-20T06:04:31.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07f0441640 1 -- 172.21.15.32:0/2434791581 wait complete. 2023-12-20T06:04:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07f0441640 1 Processor -- start 2023-12-20T06:04:31.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.009+0000 7f07f0441640 1 -- start start 2023-12-20T06:04:31.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07f0441640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07e80d1120 0x7f07e814dd80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07f0441640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80d2740 0x7f07e814cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07f0441640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07e815cdb0 0x7f07e81501c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07f0441640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e814fac0 con 0x7f07e80d2740 2023-12-20T06:04:31.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07f0441640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f07e814f630 con 0x7f07e80d1120 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07f0441640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f07e814ea20 con 0x7f07e815cdb0 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ee1dd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07e80d1120 0x7f07e814dd80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ed9dc640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80d2740 0x7f07e814cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ee1dd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07e80d1120 0x7f07e814dd80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:44666/0 (socket says 172.21.15.32:44666) 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ee1dd640 1 -- 172.21.15.32:0/1085826601 learned_addr learned my addr 172.21.15.32:0/1085826601 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ee9de640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07e815cdb0 0x7f07e81501c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ed9dc640 1 -- 172.21.15.32:0/1085826601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07e815cdb0 msgr2=0x7f07e81501c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ed9dc640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07e815cdb0 0x7f07e81501c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ed9dc640 1 -- 172.21.15.32:0/1085826601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07e80d1120 msgr2=0x7f07e814dd80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ed9dc640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07e80d1120 0x7f07e814dd80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ed9dc640 1 -- 172.21.15.32:0/1085826601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f07d8009420 con 0x7f07e80d2740 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ee1dd640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07e80d1120 0x7f07e814dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:31.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07ed9dc640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80d2740 0x7f07e814cf20 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f07d80af000 tx=0x7f07d801b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07deffd640 1 -- 172.21.15.32:0/1085826601 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80b1db0 con 0x7f07e80d2740 2023-12-20T06:04:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07deffd640 1 -- 172.21.15.32:0/1085826601 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f07d80b9410 con 0x7f07e80d2740 2023-12-20T06:04:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07deffd640 1 -- 172.21.15.32:0/1085826601 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f07d80b97a0 con 0x7f07e80d2740 2023-12-20T06:04:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07f0441640 1 -- 172.21.15.32:0/1085826601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f07e814ed00 con 0x7f07e80d2740 2023-12-20T06:04:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07f0441640 1 -- 172.21.15.32:0/1085826601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f07e814f2c0 con 0x7f07e80d2740 2023-12-20T06:04:31.016 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.013+0000 7f07f0441640 1 -- 172.21.15.32:0/1085826601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f07e804e000 con 0x7f07e80d2740 2023-12-20T06:04:31.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.017+0000 7f07deffd640 1 -- 172.21.15.32:0/1085826601 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 147) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f07d80b7040 con 0x7f07e80d2740 2023-12-20T06:04:31.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.017+0000 7f07deffd640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07cc065bb0 0x7f07cc068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.017+0000 7f07deffd640 1 -- 172.21.15.32:0/1085826601 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7f07d8120ee0 con 0x7f07e80d2740 2023-12-20T06:04:31.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.017+0000 7f07ee1dd640 1 -- 172.21.15.32:0/1085826601 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07cc065bb0 msgr2=0x7f07cc068070 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:31.018 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.017+0000 7f07ee1dd640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07cc065bb0 0x7f07cc068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:31.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.025+0000 7f07deffd640 1 -- 172.21.15.32:0/1085826601 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f07d80ee5b0 con 0x7f07e80d2740 2023-12-20T06:04:31.146 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:31.141+0000 7f560c28d180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:04:31.164 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:31.161+0000 7fe7e0848180 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2023-12-20T06:04:31.189 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.189+0000 7f07f0441640 1 -- 172.21.15.32:0/1085826601 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f07e806b8d0 con 0x7f07e80d2740 2023-12-20T06:04:31.192 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.189+0000 7f07deffd640 1 -- 172.21.15.32:0/1085826601 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v252) v1 ==== 74+0+16711 (secure 0 0 0) 0x7f07d80f1bd0 con 0x7f07e80d2740 2023-12-20T06:04:31.192 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:31.192 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":252,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:04:27.557133+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:04:12.651034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"250","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-20T06:04:13.997753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1269951230":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2916844765":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2625213384":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2535253896":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/766391941":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/3950316648":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2606010957":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/3612789559":"2023-12-21T06:04:27.557099+0000","172.21.15.32:6833/64546":"2023-12-21T06:04:27.557099+0000","172.21.15.32:6832/64546":"2023-12-21T06:04:27.557099+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:04:31.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 -- 172.21.15.32:0/1085826601 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07cc065bb0 msgr2=0x7f07cc068070 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:31.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07cc065bb0 0x7f07cc068070 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 -- 172.21.15.32:0/1085826601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80d2740 msgr2=0x7f07e814cf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80d2740 0x7f07e814cf20 secure :-1 s=READY pgs=2540 cs=0 l=1 rev1=1 crypto rx=0x7f07d80af000 tx=0x7f07d801b9f0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 -- 172.21.15.32:0/1085826601 shutdown_connections 2023-12-20T06:04:31.194 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f07cc065bb0 0x7f07cc068070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f07e815cdb0 0x7f07e81501c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f07e80d2740 0x7f07e814cf20 unknown :-1 s=CLOSED pgs=2540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 --2- 172.21.15.32:0/1085826601 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f07e80d1120 0x7f07e814dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 -- 172.21.15.32:0/1085826601 >> 172.21.15.32:0/1085826601 conn(0x7f07e8065ee0 msgr2=0x7f07e815be40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:31.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 -- 172.21.15.32:0/1085826601 shutdown_connections 2023-12-20T06:04:31.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.193+0000 7f07f0441640 1 -- 172.21.15.32:0/1085826601 wait complete. 2023-12-20T06:04:31.214 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T06:04:12.651034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '250', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T06:04:13.997753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '251', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:04:31.215 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:04:31.303 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:31.301+0000 7f560c28d180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:04:31.326 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:31.325+0000 7fe7e0848180 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2023-12-20T06:04:31.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40b64d1640 1 Processor -- start 2023-12-20T06:04:31.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40b64d1640 1 -- start start 2023-12-20T06:04:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40b64d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40b0157480 0x7f40b00c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40b64d1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40b00b0760 0x7f40b00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40b64d1640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40b00b1100 con 0x7f40b015d660 2023-12-20T06:04:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40b64d1640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40b00cde20 con 0x7f40b00b0760 2023-12-20T06:04:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40b64d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b00c8e30 con 0x7f40b0157480 2023-12-20T06:04:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40a7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40b00b0760 0x7f40b00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40b4a6e640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f40b015d660 0x7f40b015daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41040/0 (socket says 172.21.15.32:41040) 2023-12-20T06:04:31.364 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40b4a6e640 1 -- 172.21.15.32:0/2909876126 learned_addr learned my addr 172.21.15.32:0/2909876126 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:31.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40affff640 1 --2- 172.21.15.32:0/2909876126 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40b0157480 0x7f40b00c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.361+0000 7f40af7fe640 1 -- 172.21.15.32:0/2909876126 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2960027255 0 0) 0x7f40b00b1100 con 0x7f40b015d660 2023-12-20T06:04:31.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40af7fe640 1 -- 172.21.15.32:0/2909876126 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f409c003530 con 0x7f40b015d660 2023-12-20T06:04:31.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40a7fff640 1 -- 172.21.15.32:0/2909876126 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40b0157480 msgr2=0x7f40b00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:31.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40a7fff640 1 --2- 172.21.15.32:0/2909876126 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40b0157480 0x7f40b00c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40a7fff640 1 -- 172.21.15.32:0/2909876126 >> v1:172.21.15.32:6789/0 conn(0x7f40b015d660 legacy=0x7f40b015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.365 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40a7fff640 1 -- 172.21.15.32:0/2909876126 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40b00ce070 con 0x7f40b00b0760 2023-12-20T06:04:31.367 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40a7fff640 1 --2- 172.21.15.32:0/2909876126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b00b0760 0x7f40b00b0bc0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f40980097c0 tx=0x7f409801c3c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516a141cd0f6017e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:31.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40af7fe640 1 -- 172.21.15.32:0/2909876126 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40980ac620 con 0x7f40b00b0760 2023-12-20T06:04:31.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40af7fe640 1 -- 172.21.15.32:0/2909876126 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40980ac780 con 0x7f40b00b0760 2023-12-20T06:04:31.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40af7fe640 1 -- 172.21.15.32:0/2909876126 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40980acb10 con 0x7f40b00b0760 2023-12-20T06:04:31.368 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40b64d1640 1 -- 172.21.15.32:0/2909876126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b00b0760 msgr2=0x7f40b00b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40b64d1640 1 --2- 172.21.15.32:0/2909876126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b00b0760 0x7f40b00b0bc0 secure :-1 s=READY pgs=2542 cs=0 l=1 rev1=1 crypto rx=0x7f40980097c0 tx=0x7f409801c3c0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40b64d1640 1 -- 172.21.15.32:0/2909876126 shutdown_connections 2023-12-20T06:04:31.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.365+0000 7f40b64d1640 1 --2- 172.21.15.32:0/2909876126 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b00b0760 0x7f40b00b0bc0 unknown :-1 s=CLOSED pgs=2542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 --2- 172.21.15.32:0/2909876126 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40b0157480 0x7f40b00c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 -- 172.21.15.32:0/2909876126 >> 172.21.15.32:0/2909876126 conn(0x7f40b006a510 msgr2=0x7f40b00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:31.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 -- 172.21.15.32:0/2909876126 shutdown_connections 2023-12-20T06:04:31.369 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 -- 172.21.15.32:0/2909876126 wait complete. 2023-12-20T06:04:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 Processor -- start 2023-12-20T06:04:31.370 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 -- start start 2023-12-20T06:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40b00b0760 0x7f40b00c88b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40b0157480 0x7f40b015c000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b015d660 0x7f40b01548a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b0166cf0 con 0x7f40b015d660 2023-12-20T06:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40b015c670 con 0x7f40b0157480 2023-12-20T06:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b64d1640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40b015c950 con 0x7f40b00b0760 2023-12-20T06:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b4a6e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b015d660 0x7f40b01548a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b4a6e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b015d660 0x7f40b01548a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54918/0 (socket says 172.21.15.32:54918) 2023-12-20T06:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40affff640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40b00b0760 0x7f40b00c88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b4a6e640 1 -- 172.21.15.32:0/1431515262 learned_addr learned my addr 172.21.15.32:0/1431515262 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:31.372 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40a7fff640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40b0157480 0x7f40b015c000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b4a6e640 1 -- 172.21.15.32:0/1431515262 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40b00b0760 msgr2=0x7f40b00c88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b4a6e640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40b00b0760 0x7f40b00c88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b4a6e640 1 -- 172.21.15.32:0/1431515262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40b0157480 msgr2=0x7f40b015c000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b4a6e640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40b0157480 0x7f40b015c000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.373 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b4a6e640 1 -- 172.21.15.32:0/1431515262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4098009420 con 0x7f40b015d660 2023-12-20T06:04:31.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40affff640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40b00b0760 0x7f40b00c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:31.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40b4a6e640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b015d660 0x7f40b01548a0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f40a800cb40 tx=0x7f40a800a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:31.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.369+0000 7f40ad7fa640 1 -- 172.21.15.32:0/1431515262 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a800aca0 con 0x7f40b015d660 2023-12-20T06:04:31.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.373+0000 7f40ad7fa640 1 -- 172.21.15.32:0/1431515262 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40a800ae00 con 0x7f40b015d660 2023-12-20T06:04:31.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.373+0000 7f40b64d1640 1 -- 172.21.15.32:0/1431515262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40b0166910 con 0x7f40b015d660 2023-12-20T06:04:31.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.373+0000 7f40ad7fa640 1 -- 172.21.15.32:0/1431515262 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40a8012620 con 0x7f40b015d660 2023-12-20T06:04:31.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.373+0000 7f40b64d1640 1 -- 172.21.15.32:0/1431515262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40b01553b0 con 0x7f40b015d660 2023-12-20T06:04:31.374 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.373+0000 7f40b64d1640 1 -- 172.21.15.32:0/1431515262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f40b0166e30 con 0x7f40b015d660 2023-12-20T06:04:31.375 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.373+0000 7f40ad7fa640 1 -- 172.21.15.32:0/1431515262 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 147) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f40a8007070 con 0x7f40b015d660 2023-12-20T06:04:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.373+0000 7f40ad7fa640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4090065b80 0x7f4090068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.373+0000 7f40ad7fa640 1 -- 172.21.15.32:0/1431515262 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7f40a80834b0 con 0x7f40b015d660 2023-12-20T06:04:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.373+0000 7f40affff640 1 -- 172.21.15.32:0/1431515262 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4090065b80 msgr2=0x7f4090068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:31.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.373+0000 7f40affff640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4090065b80 0x7f4090068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:31.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.377+0000 7f40ad7fa640 1 -- 172.21.15.32:0/1431515262 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f40b0166e30 con 0x7f40b015d660 2023-12-20T06:04:31.538 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.537+0000 7f40b64d1640 1 -- 172.21.15.32:0/1431515262 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7f40b0166e30 con 0x7f40b015d660 2023-12-20T06:04:31.539 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.537+0000 7f40ad7fa640 1 -- 172.21.15.32:0/1431515262 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v252) v1 ==== 93+0+10 (secure 0 0 0) 0x7f40a8017020 con 0x7f40b015d660 2023-12-20T06:04:31.539 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 -- 172.21.15.32:0/1431515262 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4090065b80 msgr2=0x7f4090068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4090065b80 0x7f4090068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 -- 172.21.15.32:0/1431515262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b015d660 msgr2=0x7f40b01548a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b015d660 0x7f40b01548a0 secure :-1 s=READY pgs=2543 cs=0 l=1 rev1=1 crypto rx=0x7f40a800cb40 tx=0x7f40a800a060 comp rx=0 tx=0).stop 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 -- 172.21.15.32:0/1431515262 shutdown_connections 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4090065b80 0x7f4090068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40b015d660 0x7f40b01548a0 unknown :-1 s=CLOSED pgs=2543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40b0157480 0x7f40b015c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 --2- 172.21.15.32:0/1431515262 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40b00b0760 0x7f40b00c88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 -- 172.21.15.32:0/1431515262 >> 172.21.15.32:0/1431515262 conn(0x7f40b006a510 msgr2=0x7f40b00ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 -- 172.21.15.32:0/1431515262 shutdown_connections 2023-12-20T06:04:31.542 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.541+0000 7f40b64d1640 1 -- 172.21.15.32:0/1431515262 wait complete. 2023-12-20T06:04:31.562 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:04:31.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8bf16d640 1 Processor -- start 2023-12-20T06:04:31.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8bf16d640 1 -- start start 2023-12-20T06:04:31.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8bf16d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8b806cdf0 0x7fd8b8070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8bf16d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd8b8070770 0x7fd8b80cc840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8bf16d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd8b80cc3b0 con 0x7fd8b81591e0 2023-12-20T06:04:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8bf16d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd8b8070be0 con 0x7fd8b8070770 2023-12-20T06:04:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8bf16d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b80c8e30 con 0x7fd8b806cdf0 2023-12-20T06:04:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8bcf09640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fd8b81591e0 0x7fd8b80caa30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41056/0 (socket says 172.21.15.32:41056) 2023-12-20T06:04:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8bcf09640 1 -- 172.21.15.32:0/1517946534 learned_addr learned my addr 172.21.15.32:0/1517946534 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8bd70a640 1 --2- 172.21.15.32:0/1517946534 >> v2:172.21.15.39:3300/0 conn(0x7fd8b8070770 0x7fd8b80cc840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8b7fff640 1 --2- 172.21.15.32:0/1517946534 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8b806cdf0 0x7fd8b8070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8b77fe640 1 -- 172.21.15.32:0/1517946534 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 345876933 0 0) 0x7fd8b80cc3b0 con 0x7fd8b81591e0 2023-12-20T06:04:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8b77fe640 1 -- 172.21.15.32:0/1517946534 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd8a8003530 con 0x7fd8b81591e0 2023-12-20T06:04:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8b7fff640 1 -- 172.21.15.32:0/1517946534 >> v2:172.21.15.39:3300/0 conn(0x7fd8b8070770 msgr2=0x7fd8b80cc840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8b7fff640 1 --2- 172.21.15.32:0/1517946534 >> v2:172.21.15.39:3300/0 conn(0x7fd8b8070770 0x7fd8b80cc840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8b7fff640 1 -- 172.21.15.32:0/1517946534 >> v1:172.21.15.32:6789/0 conn(0x7fd8b81591e0 legacy=0x7fd8b80caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8b7fff640 1 -- 172.21.15.32:0/1517946534 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b806d280 con 0x7fd8b806cdf0 2023-12-20T06:04:31.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.709+0000 7fd8bd70a640 1 --2- 172.21.15.32:0/1517946534 >> v2:172.21.15.39:3300/0 conn(0x7fd8b8070770 0x7fd8b80cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:31.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8b7fff640 1 --2- 172.21.15.32:0/1517946534 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8b806cdf0 0x7fd8b8070230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40097c0 tx=0x7fd8a401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=ebb7ede741ebbdb2 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:31.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8b77fe640 1 -- 172.21.15.32:0/1517946534 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40ab600 con 0x7fd8b806cdf0 2023-12-20T06:04:31.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8b77fe640 1 -- 172.21.15.32:0/1517946534 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8a40a7050 con 0x7fd8b806cdf0 2023-12-20T06:04:31.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8b77fe640 1 -- 172.21.15.32:0/1517946534 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8a40ab960 con 0x7fd8b806cdf0 2023-12-20T06:04:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8bf16d640 1 -- 172.21.15.32:0/1517946534 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8b806cdf0 msgr2=0x7fd8b8070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8bf16d640 1 --2- 172.21.15.32:0/1517946534 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8b806cdf0 0x7fd8b8070230 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fd8a40097c0 tx=0x7fd8a401c3f0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8bf16d640 1 -- 172.21.15.32:0/1517946534 shutdown_connections 2023-12-20T06:04:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8bf16d640 1 --2- 172.21.15.32:0/1517946534 >> v2:172.21.15.39:3300/0 conn(0x7fd8b8070770 0x7fd8b80cc840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8bf16d640 1 --2- 172.21.15.32:0/1517946534 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8b806cdf0 0x7fd8b8070230 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8bf16d640 1 -- 172.21.15.32:0/1517946534 >> 172.21.15.32:0/1517946534 conn(0x7fd8b806a510 msgr2=0x7fd8b80ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8bf16d640 1 -- 172.21.15.32:0/1517946534 shutdown_connections 2023-12-20T06:04:31.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.713+0000 7fd8bf16d640 1 -- 172.21.15.32:0/1517946534 wait complete. 2023-12-20T06:04:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bf16d640 1 Processor -- start 2023-12-20T06:04:31.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bf16d640 1 -- start start 2023-12-20T06:04:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bf16d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8070770 0x7fd8b8145360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bf16d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8b81591e0 0x7fd8b814e820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bf16d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8b814ed60 0x7fd8b8146880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bf16d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b81458a0 con 0x7fd8b8070770 2023-12-20T06:04:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bf16d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b814f210 con 0x7fd8b81591e0 2023-12-20T06:04:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bf16d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd8b8154ca0 con 0x7fd8b814ed60 2023-12-20T06:04:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bcf09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8070770 0x7fd8b8145360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bcf09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8070770 0x7fd8b8145360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54928/0 (socket says 172.21.15.32:54928) 2023-12-20T06:04:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bcf09640 1 -- 172.21.15.32:0/830417584 learned_addr learned my addr 172.21.15.32:0/830417584 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8b7fff640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8b81591e0 0x7fd8b814e820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bd70a640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8b814ed60 0x7fd8b8146880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bcf09640 1 -- 172.21.15.32:0/830417584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8b814ed60 msgr2=0x7fd8b8146880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bcf09640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8b814ed60 0x7fd8b8146880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bcf09640 1 -- 172.21.15.32:0/830417584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8b81591e0 msgr2=0x7fd8b814e820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bcf09640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8b81591e0 0x7fd8b814e820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bcf09640 1 -- 172.21.15.32:0/830417584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd8b0009a70 con 0x7fd8b8070770 2023-12-20T06:04:31.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bcf09640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8070770 0x7fd8b8145360 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00c3c0 tx=0x7fd8ac00c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8b57fa640 1 -- 172.21.15.32:0/830417584 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac005db0 con 0x7fd8b8070770 2023-12-20T06:04:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8b57fa640 1 -- 172.21.15.32:0/830417584 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd8ac0070d0 con 0x7fd8b8070770 2023-12-20T06:04:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8b57fa640 1 -- 172.21.15.32:0/830417584 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd8ac0074a0 con 0x7fd8b8070770 2023-12-20T06:04:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.717+0000 7fd8bf16d640 1 -- 172.21.15.32:0/830417584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd8a4009420 con 0x7fd8b8070770 2023-12-20T06:04:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.721+0000 7fd8bf16d640 1 -- 172.21.15.32:0/830417584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd8b8145da0 con 0x7fd8b8070770 2023-12-20T06:04:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.721+0000 7fd8bf16d640 1 -- 172.21.15.32:0/830417584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd8b804e000 con 0x7fd8b8070770 2023-12-20T06:04:31.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.721+0000 7fd8b57fa640 1 -- 172.21.15.32:0/830417584 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 147) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fd8ac007600 con 0x7fd8b8070770 2023-12-20T06:04:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.721+0000 7fd8b57fa640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd89c065b80 0x7fd89c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.721+0000 7fd8b57fa640 1 -- 172.21.15.32:0/830417584 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7fd8ac082ad0 con 0x7fd8b8070770 2023-12-20T06:04:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.721+0000 7fd8b7fff640 1 -- 172.21.15.32:0/830417584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd89c065b80 msgr2=0x7fd89c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:31.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.721+0000 7fd8b7fff640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd89c065b80 0x7fd89c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:31.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.725+0000 7fd8b57fa640 1 -- 172.21.15.32:0/830417584 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd8ac050240 con 0x7fd8b8070770 2023-12-20T06:04:31.811 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:31.809+0000 7f560c28d180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:04:31.848 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:31.845+0000 7fe7e0848180 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2023-12-20T06:04:31.873 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:31.869+0000 7f560c28d180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:04:31.884 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.881+0000 7fd8bf16d640 1 -- 172.21.15.32:0/830417584 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fd8b806d840 con 0x7fd8b8070770 2023-12-20T06:04:31.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8b57fa640 1 -- 172.21.15.32:0/830417584 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v252) v1 ==== 104+0+11 (secure 0 0 0) 0x7fd8ac0170a0 con 0x7fd8b8070770 2023-12-20T06:04:31.885 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:04:31.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 -- 172.21.15.32:0/830417584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd89c065b80 msgr2=0x7fd89c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:31.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd89c065b80 0x7fd89c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 -- 172.21.15.32:0/830417584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8070770 msgr2=0x7fd8b8145360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:31.887 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8070770 0x7fd8b8145360 secure :-1 s=READY pgs=2545 cs=0 l=1 rev1=1 crypto rx=0x7fd8ac00c3c0 tx=0x7fd8ac00c7c0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 -- 172.21.15.32:0/830417584 shutdown_connections 2023-12-20T06:04:31.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd89c065b80 0x7fd89c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd8b814ed60 0x7fd8b8146880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd8b81591e0 0x7fd8b814e820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 --2- 172.21.15.32:0/830417584 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd8b8070770 0x7fd8b8145360 unknown :-1 s=CLOSED pgs=2545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:31.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 -- 172.21.15.32:0/830417584 >> 172.21.15.32:0/830417584 conn(0x7fd8b806a510 msgr2=0x7fd8b81480e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:31.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 -- 172.21.15.32:0/830417584 shutdown_connections 2023-12-20T06:04:31.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:31.885+0000 7fd8bf16d640 1 -- 172.21.15.32:0/830417584 wait complete. 2023-12-20T06:04:31.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:04:31.912 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:31.909+0000 7fe7e0848180 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2023-12-20T06:04:31.932 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:31.929+0000 7f560c28d180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:04:31.973 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:31.973+0000 7fe7e0848180 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2023-12-20T06:04:32.023 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:32.021+0000 7f560c28d180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:04:32.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb1a918640 1 Processor -- start 2023-12-20T06:04:32.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb1a918640 1 -- start start 2023-12-20T06:04:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb1a918640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb1415cdf0 0x7ffb140cde70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb1a918640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ffb140ccec0 0x7ffb140ca9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb1a918640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ffb14071bb0 con 0x7ffb140716f0 2023-12-20T06:04:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb1a918640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ffb14071e90 con 0x7ffb140ccec0 2023-12-20T06:04:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb1a918640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb14072170 con 0x7ffb1415cdf0 2023-12-20T06:04:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb137fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7ffb140716f0 0x7ffb140cc7b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41072/0 (socket says 172.21.15.32:41072) 2023-12-20T06:04:32.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb137fe640 1 -- 172.21.15.32:0/177106228 learned_addr learned my addr 172.21.15.32:0/177106228 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb18eb5640 1 --2- 172.21.15.32:0/177106228 >> v2:172.21.15.32:3300/0 conn(0x7ffb140ccec0 0x7ffb140ca9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb13fff640 1 --2- 172.21.15.32:0/177106228 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb1415cdf0 0x7ffb140cde70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb13fff640 1 -- 172.21.15.32:0/177106228 >> v2:172.21.15.32:3300/0 conn(0x7ffb140ccec0 msgr2=0x7ffb140ca9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb12ffd640 1 -- 172.21.15.32:0/177106228 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3911285570 0 0) 0x7ffb14071bb0 con 0x7ffb140716f0 2023-12-20T06:04:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb13fff640 1 --2- 172.21.15.32:0/177106228 >> v2:172.21.15.32:3300/0 conn(0x7ffb140ccec0 0x7ffb140ca9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb13fff640 1 -- 172.21.15.32:0/177106228 >> v1:172.21.15.32:6789/0 conn(0x7ffb140716f0 legacy=0x7ffb140cc7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.057+0000 7ffb13fff640 1 -- 172.21.15.32:0/177106228 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb140ca480 con 0x7ffb1415cdf0 2023-12-20T06:04:32.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb13fff640 1 --2- 172.21.15.32:0/177106228 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb1415cdf0 0x7ffb140cde70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffb040097c0 tx=0x7ffb0401c3f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d27e6acd8935704e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:32.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb12ffd640 1 -- 172.21.15.32:0/177106228 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb040ab600 con 0x7ffb1415cdf0 2023-12-20T06:04:32.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb12ffd640 1 -- 172.21.15.32:0/177106228 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb040a7050 con 0x7ffb1415cdf0 2023-12-20T06:04:32.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb12ffd640 1 -- 172.21.15.32:0/177106228 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb040ab980 con 0x7ffb1415cdf0 2023-12-20T06:04:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb1a918640 1 -- 172.21.15.32:0/177106228 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb1415cdf0 msgr2=0x7ffb140cde70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb1a918640 1 --2- 172.21.15.32:0/177106228 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb1415cdf0 0x7ffb140cde70 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7ffb040097c0 tx=0x7ffb0401c3f0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb1a918640 1 -- 172.21.15.32:0/177106228 shutdown_connections 2023-12-20T06:04:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb1a918640 1 --2- 172.21.15.32:0/177106228 >> v2:172.21.15.32:3300/0 conn(0x7ffb140ccec0 0x7ffb140ca9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb1a918640 1 --2- 172.21.15.32:0/177106228 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb1415cdf0 0x7ffb140cde70 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb1a918640 1 -- 172.21.15.32:0/177106228 >> 172.21.15.32:0/177106228 conn(0x7ffb1406b230 msgr2=0x7ffb1406b640 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb1a918640 1 -- 172.21.15.32:0/177106228 shutdown_connections 2023-12-20T06:04:32.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.061+0000 7ffb1a918640 1 -- 172.21.15.32:0/177106228 wait complete. 2023-12-20T06:04:32.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb1a918640 1 Processor -- start 2023-12-20T06:04:32.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb1a918640 1 -- start start 2023-12-20T06:04:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb1a918640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb140716f0 0x7ffb141513a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb1a918640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140ccec0 0x7ffb140c5dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb1a918640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb1415cdf0 0x7ffb1415abc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb1a918640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb1415a730 con 0x7ffb140ccec0 2023-12-20T06:04:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb1a918640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ffb14159b20 con 0x7ffb140716f0 2023-12-20T06:04:32.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb1a918640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ffb14159e00 con 0x7ffb1415cdf0 2023-12-20T06:04:32.067 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:32.065+0000 7fe7e0848180 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb137fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140ccec0 0x7ffb140c5dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb137fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140ccec0 0x7ffb140c5dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54944/0 (socket says 172.21.15.32:54944) 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb13fff640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb140716f0 0x7ffb141513a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb137fe640 1 -- 172.21.15.32:0/3634960550 learned_addr learned my addr 172.21.15.32:0/3634960550 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb18eb5640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb1415cdf0 0x7ffb1415abc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb137fe640 1 -- 172.21.15.32:0/3634960550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb1415cdf0 msgr2=0x7ffb1415abc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb137fe640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb1415cdf0 0x7ffb1415abc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb137fe640 1 -- 172.21.15.32:0/3634960550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb140716f0 msgr2=0x7ffb141513a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb137fe640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb140716f0 0x7ffb141513a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb137fe640 1 -- 172.21.15.32:0/3634960550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ffb04009420 con 0x7ffb140ccec0 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb137fe640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140ccec0 0x7ffb140c5dd0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7ffb080011f0 tx=0x7ffb0800c9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb10ff9640 1 -- 172.21.15.32:0/3634960550 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb080074e0 con 0x7ffb140ccec0 2023-12-20T06:04:32.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb10ff9640 1 -- 172.21.15.32:0/3634960550 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ffb08007640 con 0x7ffb140ccec0 2023-12-20T06:04:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb10ff9640 1 -- 172.21.15.32:0/3634960550 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ffb08004a10 con 0x7ffb140ccec0 2023-12-20T06:04:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.065+0000 7ffb1a918640 1 -- 172.21.15.32:0/3634960550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ffb1415a0e0 con 0x7ffb140ccec0 2023-12-20T06:04:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.069+0000 7ffb1a918640 1 -- 172.21.15.32:0/3634960550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ffb140c6310 con 0x7ffb140ccec0 2023-12-20T06:04:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.069+0000 7ffb1a918640 1 -- 172.21.15.32:0/3634960550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ffb1404e000 con 0x7ffb140ccec0 2023-12-20T06:04:32.070 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.069+0000 7ffb10ff9640 1 -- 172.21.15.32:0/3634960550 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 147) v1 ==== 85331+0+0 (secure 0 0 0) 0x7ffb08004b70 con 0x7ffb140ccec0 2023-12-20T06:04:32.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.069+0000 7ffb10ff9640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffb00065b80 0x7ffb00068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.069+0000 7ffb10ff9640 1 -- 172.21.15.32:0/3634960550 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7ffb080827b0 con 0x7ffb140ccec0 2023-12-20T06:04:32.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.069+0000 7ffb13fff640 1 -- 172.21.15.32:0/3634960550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffb00065b80 msgr2=0x7ffb00068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:32.071 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.069+0000 7ffb13fff640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffb00065b80 0x7ffb00068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:32.075 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.073+0000 7ffb10ff9640 1 -- 172.21.15.32:0/3634960550 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ffb0804fe80 con 0x7ffb140ccec0 2023-12-20T06:04:32.083 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:32.081+0000 7f560c28d180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:04:32.129 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:32.129+0000 7fe7e0848180 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2023-12-20T06:04:32.171 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:32.169+0000 7f560c28d180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:04:32.218 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:32.217+0000 7fe7e0848180 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2023-12-20T06:04:32.230 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:32.225+0000 7f560c28d180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:04:32.232 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.229+0000 7ffb1a918640 1 -- 172.21.15.32:0/3634960550 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7ffb140d02a0 con 0x7ffb140ccec0 2023-12-20T06:04:32.233 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb10ff9640 1 -- 172.21.15.32:0/3634960550 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v252) v1 ==== 100+0+11 (secure 0 0 0) 0x7ffb080180a0 con 0x7ffb140ccec0 2023-12-20T06:04:32.233 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:04:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 -- 172.21.15.32:0/3634960550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffb00065b80 msgr2=0x7ffb00068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffb00065b80 0x7ffb00068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 -- 172.21.15.32:0/3634960550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140ccec0 msgr2=0x7ffb140c5dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.235 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140ccec0 0x7ffb140c5dd0 secure :-1 s=READY pgs=2547 cs=0 l=1 rev1=1 crypto rx=0x7ffb080011f0 tx=0x7ffb0800c9b0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 -- 172.21.15.32:0/3634960550 shutdown_connections 2023-12-20T06:04:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ffb00065b80 0x7ffb00068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ffb1415cdf0 0x7ffb1415abc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ffb140ccec0 0x7ffb140c5dd0 unknown :-1 s=CLOSED pgs=2547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 --2- 172.21.15.32:0/3634960550 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ffb140716f0 0x7ffb141513a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 -- 172.21.15.32:0/3634960550 >> 172.21.15.32:0/3634960550 conn(0x7ffb1406b230 msgr2=0x7ffb1415d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 -- 172.21.15.32:0/3634960550 shutdown_connections 2023-12-20T06:04:32.236 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.233+0000 7ffb1a918640 1 -- 172.21.15.32:0/3634960550 wait complete. 2023-12-20T06:04:32.255 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2023-12-20T06:04:32.280 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:32.277+0000 7fe7e0848180 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2023-12-20T06:04:32.380 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:32.377+0000 7f560c28d180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:04:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4888b640 1 Processor -- start 2023-12-20T06:04:32.426 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4888b640 1 -- start start 2023-12-20T06:04:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4888b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 0x7faa44160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.427 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4888b640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faa44161330 0x7faa44153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4888b640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7faa440cd4d0 con 0x7faa440b8480 2023-12-20T06:04:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4888b640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7faa440ce4e0 con 0x7faa44161330 2023-12-20T06:04:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4888b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa440cd720 con 0x7faa4406b630 2023-12-20T06:04:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4359e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faa44161330 0x7faa44153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 0x7faa44160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4259c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 0x7faa44160df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:56490/0 (socket says 172.21.15.32:56490) 2023-12-20T06:04:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4259c640 1 -- 172.21.15.32:0/3425179768 learned_addr learned my addr 172.21.15.32:0/3425179768 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4359e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7faa44161330 0x7faa44153610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:44710/0 (socket says 172.21.15.32:44710) 2023-12-20T06:04:32.428 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa42d9d640 1 --1- 172.21.15.32:0/3425179768 >> v1:172.21.15.39:6789/0 conn(0x7faa440b8480 0x7faa4406af20 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:60184/0 (socket says 172.21.15.32:60184) 2023-12-20T06:04:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4359e640 1 -- 172.21.15.32:0/3425179768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 msgr2=0x7faa44160df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4359e640 1 --2- 172.21.15.32:0/3425179768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 0x7faa44160df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4359e640 1 -- 172.21.15.32:0/3425179768 >> v1:172.21.15.39:6789/0 conn(0x7faa440b8480 legacy=0x7faa4406af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4359e640 1 -- 172.21.15.32:0/3425179768 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa440cda00 con 0x7faa44161330 2023-12-20T06:04:32.429 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.425+0000 7faa4259c640 1 --2- 172.21.15.32:0/3425179768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 0x7faa44160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa4359e640 1 --2- 172.21.15.32:0/3425179768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa44161330 0x7faa44153610 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009d90 tx=0x7faa3c01c950 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f483bd3b97edf2a5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa41d9b640 1 -- 172.21.15.32:0/3425179768 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0a5900 con 0x7faa44161330 2023-12-20T06:04:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa41d9b640 1 -- 172.21.15.32:0/3425179768 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3c0a5a60 con 0x7faa44161330 2023-12-20T06:04:32.431 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa41d9b640 1 -- 172.21.15.32:0/3425179768 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3c0a5df0 con 0x7faa44161330 2023-12-20T06:04:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa4888b640 1 -- 172.21.15.32:0/3425179768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa44161330 msgr2=0x7faa44153610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa4888b640 1 --2- 172.21.15.32:0/3425179768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa44161330 0x7faa44153610 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7faa3c009d90 tx=0x7faa3c01c950 comp rx=0 tx=0).stop 2023-12-20T06:04:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa4888b640 1 -- 172.21.15.32:0/3425179768 shutdown_connections 2023-12-20T06:04:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa4888b640 1 --2- 172.21.15.32:0/3425179768 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa44161330 0x7faa44153610 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa4888b640 1 --2- 172.21.15.32:0/3425179768 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 0x7faa44160df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.432 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa4888b640 1 -- 172.21.15.32:0/3425179768 >> 172.21.15.32:0/3425179768 conn(0x7faa440d2a50 msgr2=0x7faa440d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa4888b640 1 -- 172.21.15.32:0/3425179768 shutdown_connections 2023-12-20T06:04:32.433 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.429+0000 7faa4888b640 1 -- 172.21.15.32:0/3425179768 wait complete. 2023-12-20T06:04:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4888b640 1 Processor -- start 2023-12-20T06:04:32.434 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4888b640 1 -- start start 2023-12-20T06:04:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4888b640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 0x7faa441570a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4888b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa440b8480 0x7faa4414ac80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4888b640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa44161330 0x7faa4414b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4888b640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa441532d0 con 0x7faa440b8480 2023-12-20T06:04:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4888b640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7faa44152e40 con 0x7faa44161330 2023-12-20T06:04:32.435 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4888b640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7faa441575e0 con 0x7faa4406b630 2023-12-20T06:04:32.436 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:32.433+0000 7fe7e0848180 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2023-12-20T06:04:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa42d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 0x7faa441570a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa440b8480 0x7faa4414ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4359e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa44161330 0x7faa4414b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa42d9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 0x7faa441570a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:56498/0 (socket says 172.21.15.32:56498) 2023-12-20T06:04:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa440b8480 0x7faa4414ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54952/0 (socket says 172.21.15.32:54952) 2023-12-20T06:04:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa42d9d640 1 -- 172.21.15.32:0/280690251 learned_addr learned my addr 172.21.15.32:0/280690251 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:32.437 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4259c640 1 -- 172.21.15.32:0/280690251 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 msgr2=0x7faa441570a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4259c640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 0x7faa441570a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4259c640 1 -- 172.21.15.32:0/280690251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa44161330 msgr2=0x7faa4414b1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4259c640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa44161330 0x7faa4414b1c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4259c640 1 -- 172.21.15.32:0/280690251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7faa380094d0 con 0x7faa440b8480 2023-12-20T06:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4359e640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa44161330 0x7faa4414b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4259c640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa440b8480 0x7faa4414ac80 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7faa3800e940 tx=0x7faa3800ee00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa2b7fe640 1 -- 172.21.15.32:0/280690251 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa3800ca00 con 0x7faa440b8480 2023-12-20T06:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa2b7fe640 1 -- 172.21.15.32:0/280690251 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7faa3800cb60 con 0x7faa440b8480 2023-12-20T06:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa4888b640 1 -- 172.21.15.32:0/280690251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7faa3c009a70 con 0x7faa440b8480 2023-12-20T06:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.433+0000 7faa2b7fe640 1 -- 172.21.15.32:0/280690251 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7faa38004030 con 0x7faa440b8480 2023-12-20T06:04:32.438 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.437+0000 7faa4888b640 1 -- 172.21.15.32:0/280690251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7faa44152510 con 0x7faa440b8480 2023-12-20T06:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.437+0000 7faa4888b640 1 -- 172.21.15.32:0/280690251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7faa44152ff0 con 0x7faa440b8480 2023-12-20T06:04:32.439 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.437+0000 7faa2b7fe640 1 -- 172.21.15.32:0/280690251 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 147) v1 ==== 85331+0+0 (secure 0 0 0) 0x7faa38004190 con 0x7faa440b8480 2023-12-20T06:04:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.437+0000 7faa2b7fe640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faa30065b80 0x7faa30068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.437+0000 7faa42d9d640 1 -- 172.21.15.32:0/280690251 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faa30065b80 msgr2=0x7faa30068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.437+0000 7faa2b7fe640 1 -- 172.21.15.32:0/280690251 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7faa38083500 con 0x7faa440b8480 2023-12-20T06:04:32.440 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.437+0000 7faa42d9d640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faa30065b80 0x7faa30068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:32.442 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:32.441+0000 7f560c28d180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:04:32.444 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.441+0000 7faa2b7fe640 1 -- 172.21.15.32:0/280690251 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7faa44152ff0 con 0x7faa440b8480 2023-12-20T06:04:32.498 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:32.497+0000 7fe7e0848180 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2023-12-20T06:04:32.615 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.613+0000 7faa4888b640 1 -- 172.21.15.32:0/280690251 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) v1 -- 0x7faa44152ff0 con 0x7faa440b8480 2023-12-20T06:04:32.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.613+0000 7faa2b7fe640 1 -- 172.21.15.32:0/280690251 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v252) v1 ==== 81+0+575 (secure 0 0 0) 0x7faa44152ff0 con 0x7faa440b8480 2023-12-20T06:04:32.616 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/1269951230 2023-12-21T06:04:27.557099+0000 2023-12-20T06:04:32.616 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2916844765 2023-12-21T06:04:27.557099+0000 2023-12-20T06:04:32.616 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2625213384 2023-12-21T06:04:27.557099+0000 2023-12-20T06:04:32.616 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2535253896 2023-12-21T06:04:27.557099+0000 2023-12-20T06:04:32.616 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/766391941 2023-12-21T06:04:27.557099+0000 2023-12-20T06:04:32.616 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3950316648 2023-12-21T06:04:27.557099+0000 2023-12-20T06:04:32.616 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/2606010957 2023-12-21T06:04:27.557099+0000 2023-12-20T06:04:32.616 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:0/3612789559 2023-12-21T06:04:27.557099+0000 2023-12-20T06:04:32.617 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6833/64546 2023-12-21T06:04:27.557099+0000 2023-12-20T06:04:32.617 INFO:teuthology.orchestra.run.smithi032.stdout:172.21.15.32:6832/64546 2023-12-21T06:04:27.557099+0000 2023-12-20T06:04:32.617 INFO:teuthology.orchestra.run.smithi032.stderr:listed 10 entries 2023-12-20T06:04:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 -- 172.21.15.32:0/280690251 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faa30065b80 msgr2=0x7faa30068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faa30065b80 0x7faa30068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 -- 172.21.15.32:0/280690251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa440b8480 msgr2=0x7faa4414ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa440b8480 0x7faa4414ac80 secure :-1 s=READY pgs=2548 cs=0 l=1 rev1=1 crypto rx=0x7faa3800e940 tx=0x7faa3800ee00 comp rx=0 tx=0).stop 2023-12-20T06:04:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 -- 172.21.15.32:0/280690251 shutdown_connections 2023-12-20T06:04:32.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faa30065b80 0x7faa30068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7faa44161330 0x7faa4414b1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7faa440b8480 0x7faa4414ac80 unknown :-1 s=CLOSED pgs=2548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 --2- 172.21.15.32:0/280690251 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7faa4406b630 0x7faa441570a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 -- 172.21.15.32:0/280690251 >> 172.21.15.32:0/280690251 conn(0x7faa440d2a50 msgr2=0x7faa4415fea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 -- 172.21.15.32:0/280690251 shutdown_connections 2023-12-20T06:04:32.619 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.617+0000 7faa4888b640 1 -- 172.21.15.32:0/280690251 wait complete. 2023-12-20T06:04:32.638 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2023-12-20T06:04:32.638 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:32.638 DEBUG:teuthology.orchestra.run.smithi032:> sudo umount /home/ubuntu/cephtest/mnt.0 2023-12-20T06:04:32.692 INFO:tasks.ceph.mgr.x.smithi039.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:04:32.693 INFO:tasks.ceph.mgr.x.smithi039.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:04:32.693 INFO:tasks.ceph.mgr.x.smithi039.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:04:32.701 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T06:04:32.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:32.701 DEBUG:teuthology.orchestra.run.smithi032:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2023-12-20T06:04:32.702 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:32.697+0000 7f560c28d180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:04:32.725 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:04:32.725 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:32.734 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:04:32.734 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:04:32.734 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:32.755 INFO:tasks.ceph.mgr.y.smithi032.stderr:/usr/lib/python3/dist-packages/scipy/__init__.py:67: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2023-12-20T06:04:32.755 INFO:tasks.ceph.mgr.y.smithi032.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2023-12-20T06:04:32.755 INFO:tasks.ceph.mgr.y.smithi032.stderr: from numpy import show_config as show_numpy_config 2023-12-20T06:04:32.766 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:04:32.766 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:04:32.766 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:32.766 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:32.765+0000 7fe7e0848180 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2023-12-20T06:04:32.786 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:04:32.786 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:04:32.844 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:32.841+0000 7f560c28d180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:04:32.913 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:32.913+0000 7fe7e0848180 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:04:32.933 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fdaa628c640 1 Processor -- start 2023-12-20T06:04:32.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fdaa628c640 1 -- start start 2023-12-20T06:04:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fdaa628c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdaa00d2740 0x7fdaa00cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fdaa628c640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa015d190 con 0x7fdaa00cfab0 2023-12-20T06:04:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fdaa628c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fdaa00d16a0 con 0x7fdaa015cdb0 2023-12-20T06:04:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fdaa628c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa00cff40 con 0x7fdaa00d2740 2023-12-20T06:04:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fdaa4829640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fdaa00cfab0 0x7fdaa00b5c90 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41082/0 (socket says 172.21.15.32:41082) 2023-12-20T06:04:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fdaa4829640 1 -- 172.21.15.32:0/649019136 learned_addr learned my addr 172.21.15.32:0/649019136 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:32.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fda9f7fe640 1 --2- 172.21.15.32:0/649019136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdaa00d2740 0x7fdaa00cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fda9effd640 1 -- 172.21.15.32:0/649019136 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2745595395 0 0) 0x7fdaa015d190 con 0x7fdaa00cfab0 2023-12-20T06:04:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fda9effd640 1 -- 172.21.15.32:0/649019136 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fda88003530 con 0x7fdaa00cfab0 2023-12-20T06:04:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fda9effd640 1 -- 172.21.15.32:0/649019136 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2698801683 0 0) 0x7fdaa00d16a0 con 0x7fdaa015cdb0 2023-12-20T06:04:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fda9effd640 1 -- 172.21.15.32:0/649019136 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fdaa015d190 con 0x7fdaa015cdb0 2023-12-20T06:04:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fda9f7fe640 1 -- 172.21.15.32:0/649019136 >> v1:172.21.15.39:6789/0 conn(0x7fdaa015cdb0 legacy=0x7fdaa00d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fda9f7fe640 1 -- 172.21.15.32:0/649019136 >> v1:172.21.15.32:6789/0 conn(0x7fdaa00cfab0 legacy=0x7fdaa00b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.933+0000 7fda9f7fe640 1 -- 172.21.15.32:0/649019136 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fdaa00d1980 con 0x7fdaa00d2740 2023-12-20T06:04:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.937+0000 7fda9f7fe640 1 --2- 172.21.15.32:0/649019136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdaa00d2740 0x7fdaa00cf570 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fda940097f0 tx=0x7fda9401ecb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4f9b43c3f6903ae9 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.937+0000 7fda9effd640 1 -- 172.21.15.32:0/649019136 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda940a89b0 con 0x7fdaa00d2740 2023-12-20T06:04:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.937+0000 7fda9effd640 1 -- 172.21.15.32:0/649019136 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda940a8b10 con 0x7fdaa00d2740 2023-12-20T06:04:32.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.937+0000 7fda9effd640 1 -- 172.21.15.32:0/649019136 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda940a8ea0 con 0x7fdaa00d2740 2023-12-20T06:04:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.937+0000 7fdaa628c640 1 -- 172.21.15.32:0/649019136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdaa00d2740 msgr2=0x7fdaa00cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.937+0000 7fdaa628c640 1 --2- 172.21.15.32:0/649019136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdaa00d2740 0x7fdaa00cf570 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7fda940097f0 tx=0x7fda9401ecb0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.937+0000 7fdaa628c640 1 -- 172.21.15.32:0/649019136 shutdown_connections 2023-12-20T06:04:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.937+0000 7fdaa628c640 1 --2- 172.21.15.32:0/649019136 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdaa00d2740 0x7fdaa00cf570 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.937+0000 7fdaa628c640 1 -- 172.21.15.32:0/649019136 >> 172.21.15.32:0/649019136 conn(0x7fdaa0065ee0 msgr2=0x7fdaa006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.937+0000 7fdaa628c640 1 -- 172.21.15.32:0/649019136 shutdown_connections 2023-12-20T06:04:32.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.937+0000 7fdaa628c640 1 -- 172.21.15.32:0/649019136 wait complete. 2023-12-20T06:04:32.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fdaa628c640 1 Processor -- start 2023-12-20T06:04:32.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fdaa628c640 1 -- start start 2023-12-20T06:04:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fdaa628c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaa00cfab0 0x7fdaa0145f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fdaa628c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00d2740 0x7fdaa00c8b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fdaa628c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdaa015cdb0 0x7fdaa00c90a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fdaa628c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa015e220 con 0x7fdaa00d2740 2023-12-20T06:04:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fdaa628c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa015e500 con 0x7fdaa00cfab0 2023-12-20T06:04:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fdaa628c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fdaa015e7e0 con 0x7fdaa015cdb0 2023-12-20T06:04:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00d2740 0x7fdaa00c8b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00d2740 0x7fdaa00c8b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54958/0 (socket says 172.21.15.32:54958) 2023-12-20T06:04:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9f7fe640 1 -- 172.21.15.32:0/3462638642 learned_addr learned my addr 172.21.15.32:0/3462638642 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9ffff640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaa00cfab0 0x7fdaa0145f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fdaa4829640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdaa015cdb0 0x7fdaa00c90a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:32.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9f7fe640 1 -- 172.21.15.32:0/3462638642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdaa015cdb0 msgr2=0x7fdaa00c90a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9f7fe640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdaa015cdb0 0x7fdaa00c90a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.944 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9f7fe640 1 -- 172.21.15.32:0/3462638642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaa00cfab0 msgr2=0x7fdaa0145f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9f7fe640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaa00cfab0 0x7fdaa0145f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9f7fe640 1 -- 172.21.15.32:0/3462638642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fda94009420 con 0x7fdaa00d2740 2023-12-20T06:04:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9f7fe640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00d2740 0x7fdaa00c8b60 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fda9401b8c0 tx=0x7fda9401ba50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9cff9640 1 -- 172.21.15.32:0/3462638642 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda940b0d80 con 0x7fdaa00d2740 2023-12-20T06:04:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9cff9640 1 -- 172.21.15.32:0/3462638642 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fda940b9430 con 0x7fdaa00d2740 2023-12-20T06:04:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fda9cff9640 1 -- 172.21.15.32:0/3462638642 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fda940b97e0 con 0x7fdaa00d2740 2023-12-20T06:04:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fdaa628c640 1 -- 172.21.15.32:0/3462638642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fdaa015eac0 con 0x7fdaa00d2740 2023-12-20T06:04:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.941+0000 7fdaa628c640 1 -- 172.21.15.32:0/3462638642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fdaa01466a0 con 0x7fdaa00d2740 2023-12-20T06:04:32.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.945+0000 7fdaa628c640 1 -- 172.21.15.32:0/3462638642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fdaa004e000 con 0x7fdaa00d2740 2023-12-20T06:04:32.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.945+0000 7fda9cff9640 1 -- 172.21.15.32:0/3462638642 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 147) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fda940b7040 con 0x7fdaa00d2740 2023-12-20T06:04:32.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.945+0000 7fda9cff9640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fda8c065b80 0x7fda8c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:32.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.945+0000 7fda9ffff640 1 -- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fda8c065b80 msgr2=0x7fda8c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:32.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.945+0000 7fda9cff9640 1 -- 172.21.15.32:0/3462638642 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7fda94120e30 con 0x7fdaa00d2740 2023-12-20T06:04:32.949 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.945+0000 7fda9ffff640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fda8c065b80 0x7fda8c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:32.952 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:32.953+0000 7fda9cff9640 1 -- 172.21.15.32:0/3462638642 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fda940ee5a0 con 0x7fdaa00d2740 2023-12-20T06:04:32.976 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:32.973+0000 7f560c28d180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:04:33.052 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:33.049+0000 7f560c28d180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:04:33.053 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:33.053+0000 7fe7e0848180 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2023-12-20T06:04:33.128 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:33.125+0000 7fe7e0848180 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2023-12-20T06:04:33.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.145+0000 7fdaa628c640 1 -- 172.21.15.32:0/3462638642 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fdaa006b8d0 con 0x7fdaa00d2740 2023-12-20T06:04:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.145+0000 7fda9cff9640 1 -- 172.21.15.32:0/3462638642 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4722 (secure 0 0 0) 0x7fda940b5090 con 0x7fdaa00d2740 2023-12-20T06:04:33.147 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:33.147 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3779641739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3779641739},{"type":"v1","addr":"172.21.15.39:6834","nonce":3779641739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/473427978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":473427978},{"type":"v1","addr":"172.21.15.39:6835","nonce":473427978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12060,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3927543497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3927543497},{"type":"v1","addr":"172.21.15.32:6836","nonce":3927543497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:04:14.992681+0000","modified":"2023-12-20T06:04:18.007320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12063":{"gid":12063,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3133738681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3133738681},{"type":"v1","addr":"172.21.15.32:6837","nonce":3133738681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-20T06:04:33.147 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 128 2023-12-20T06:04:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fda9ffff640 1 -- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fda8c065b80 msgr2=0x7fda8c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fda9ffff640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fda8c065b80 0x7fda8c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2023-12-20T06:04:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 -- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fda8c065b80 msgr2=0x7fda8c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:33.149 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fda8c065b80 0x7fda8c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 -- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00d2740 msgr2=0x7fdaa00c8b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00d2740 0x7fdaa00c8b60 secure :-1 s=READY pgs=2550 cs=0 l=1 rev1=1 crypto rx=0x7fda9401b8c0 tx=0x7fda9401ba50 comp rx=0 tx=0).stop 2023-12-20T06:04:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 -- 172.21.15.32:0/3462638642 shutdown_connections 2023-12-20T06:04:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fda8c065b80 0x7fda8c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fdaa015cdb0 0x7fdaa00c90a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fdaa00d2740 0x7fdaa00c8b60 unknown :-1 s=CLOSED pgs=2550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 --2- 172.21.15.32:0/3462638642 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fdaa00cfab0 0x7fdaa0145f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 -- 172.21.15.32:0/3462638642 >> 172.21.15.32:0/3462638642 conn(0x7fdaa0065ee0 msgr2=0x7fdaa00d0be0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 -- 172.21.15.32:0/3462638642 shutdown_connections 2023-12-20T06:04:33.150 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.149+0000 7fdaa628c640 1 -- 172.21.15.32:0/3462638642 wait complete. 2023-12-20T06:04:33.170 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:04:33.193 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:33.189+0000 7f560c28d180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:04:33.276 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:33.273+0000 7fe7e0848180 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2023-12-20T06:04:33.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.313+0000 7fd34e695640 1 Processor -- start 2023-12-20T06:04:33.316 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.313+0000 7fd34e695640 1 -- start start 2023-12-20T06:04:33.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd34e695640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd34815cdb0 0x7fd3480d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd34e695640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3480d2740 0x7fd3480cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd34e695640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd3480b6ca0 con 0x7fd3480d1120 2023-12-20T06:04:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd34e695640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd34815d190 con 0x7fd3480d2740 2023-12-20T06:04:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd34e695640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd3480d16a0 con 0x7fd34815cdb0 2023-12-20T06:04:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd3477fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3480d2740 0x7fd3480cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd3477fe640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd3480d2740 0x7fd3480cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54974/0 (socket says 172.21.15.32:54974) 2023-12-20T06:04:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd3477fe640 1 -- 172.21.15.32:0/3729269739 learned_addr learned my addr 172.21.15.32:0/3729269739 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd347fff640 1 --2- 172.21.15.32:0/3729269739 >> v2:172.21.15.39:3300/0 conn(0x7fd34815cdb0 0x7fd3480d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:33.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd3477fe640 1 -- 172.21.15.32:0/3729269739 >> v2:172.21.15.39:3300/0 conn(0x7fd34815cdb0 msgr2=0x7fd3480d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:33.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd3477fe640 1 --2- 172.21.15.32:0/3729269739 >> v2:172.21.15.39:3300/0 conn(0x7fd34815cdb0 0x7fd3480d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd3477fe640 1 -- 172.21.15.32:0/3729269739 >> v1:172.21.15.39:6789/0 conn(0x7fd3480d1120 legacy=0x7fd3480b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:33.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.317+0000 7fd3477fe640 1 -- 172.21.15.32:0/3729269739 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd3480d1980 con 0x7fd3480d2740 2023-12-20T06:04:33.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd3477fe640 1 --2- 172.21.15.32:0/3729269739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480d2740 0x7fd3480cf570 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fd3480667b0 tx=0x7fd33c01ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65e52412b5b3959c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:33.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd346ffd640 1 -- 172.21.15.32:0/3729269739 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c0a89f0 con 0x7fd3480d2740 2023-12-20T06:04:33.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd346ffd640 1 -- 172.21.15.32:0/3729269739 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c0a8b50 con 0x7fd3480d2740 2023-12-20T06:04:33.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd346ffd640 1 -- 172.21.15.32:0/3729269739 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c0b0440 con 0x7fd3480d2740 2023-12-20T06:04:33.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd34e695640 1 -- 172.21.15.32:0/3729269739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480d2740 msgr2=0x7fd3480cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:33.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd34e695640 1 --2- 172.21.15.32:0/3729269739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480d2740 0x7fd3480cf570 secure :-1 s=READY pgs=2551 cs=0 l=1 rev1=1 crypto rx=0x7fd3480667b0 tx=0x7fd33c01ecf0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd34e695640 1 -- 172.21.15.32:0/3729269739 shutdown_connections 2023-12-20T06:04:33.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd34e695640 1 --2- 172.21.15.32:0/3729269739 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480d2740 0x7fd3480cf570 unknown :-1 s=CLOSED pgs=2551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd34e695640 1 --2- 172.21.15.32:0/3729269739 >> v2:172.21.15.39:3300/0 conn(0x7fd34815cdb0 0x7fd3480d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd34e695640 1 -- 172.21.15.32:0/3729269739 >> 172.21.15.32:0/3729269739 conn(0x7fd348065ee0 msgr2=0x7fd34806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:33.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd34e695640 1 -- 172.21.15.32:0/3729269739 shutdown_connections 2023-12-20T06:04:33.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd34e695640 1 -- 172.21.15.32:0/3729269739 wait complete. 2023-12-20T06:04:33.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd34e695640 1 Processor -- start 2023-12-20T06:04:33.324 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.321+0000 7fd34e695640 1 -- start start 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd34e695640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3480d1120 0x7fd3481f5d60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd34e695640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480d2740 0x7fd3480c9c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd34e695640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd34815cdb0 0x7fd348150950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd34e695640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd3481f62a0 con 0x7fd3480d2740 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd34e695640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd34815f1c0 con 0x7fd34815cdb0 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd34e695640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd348151650 con 0x7fd3480d1120 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd3477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480d2740 0x7fd3480c9c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd3477fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480d2740 0x7fd3480c9c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:54990/0 (socket says 172.21.15.32:54990) 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd3477fe640 1 -- 172.21.15.32:0/2958424856 learned_addr learned my addr 172.21.15.32:0/2958424856 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd347fff640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3480d1120 0x7fd3481f5d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd34cc32640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd34815cdb0 0x7fd348150950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd3477fe640 1 -- 172.21.15.32:0/2958424856 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3480d1120 msgr2=0x7fd3481f5d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd3477fe640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3480d1120 0x7fd3481f5d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd3477fe640 1 -- 172.21.15.32:0/2958424856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd34815cdb0 msgr2=0x7fd348150950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:33.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd3477fe640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd34815cdb0 0x7fd348150950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd3477fe640 1 -- 172.21.15.32:0/2958424856 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd33c0094d0 con 0x7fd3480d2740 2023-12-20T06:04:33.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd3477fe640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480d2740 0x7fd3480c9c30 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0b3040 tx=0x7fd33c01ba90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:33.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd344ff9640 1 -- 172.21.15.32:0/2958424856 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c0ba600 con 0x7fd3480d2740 2023-12-20T06:04:33.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd344ff9640 1 -- 172.21.15.32:0/2958424856 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd33c0ba760 con 0x7fd3480d2740 2023-12-20T06:04:33.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd344ff9640 1 -- 172.21.15.32:0/2958424856 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd33c0bab10 con 0x7fd3480d2740 2023-12-20T06:04:33.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd34e695640 1 -- 172.21.15.32:0/2958424856 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd34814fc80 con 0x7fd3480d2740 2023-12-20T06:04:33.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd34e695640 1 -- 172.21.15.32:0/2958424856 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd34815eb70 con 0x7fd3480d2740 2023-12-20T06:04:33.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.325+0000 7fd34e695640 1 -- 172.21.15.32:0/2958424856 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd34804e000 con 0x7fd3480d2740 2023-12-20T06:04:33.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.329+0000 7fd344ff9640 1 -- 172.21.15.32:0/2958424856 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 147) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fd33c0bac70 con 0x7fd3480d2740 2023-12-20T06:04:33.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.329+0000 7fd344ff9640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd334065b80 0x7fd334068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.329+0000 7fd344ff9640 1 -- 172.21.15.32:0/2958424856 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7fd33c1218b0 con 0x7fd3480d2740 2023-12-20T06:04:33.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.329+0000 7fd347fff640 1 -- 172.21.15.32:0/2958424856 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd334065b80 msgr2=0x7fd334068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:33.330 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.329+0000 7fd347fff640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd334065b80 0x7fd334068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:33.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.337+0000 7fd344ff9640 1 -- 172.21.15.32:0/2958424856 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd33c0ef0b0 con 0x7fd3480d2740 2023-12-20T06:04:33.497 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.497+0000 7fd34e695640 1 -- 172.21.15.32:0/2958424856 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7fd34806b8d0 con 0x7fd3480d2740 2023-12-20T06:04:33.499 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.497+0000 7fd344ff9640 1 -- 172.21.15.32:0/2958424856 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v252) v1 ==== 74+0+16711 (secure 0 0 0) 0x7fd33c0f26d0 con 0x7fd3480d2740 2023-12-20T06:04:33.499 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:33.499 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":252,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:04:27.557133+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:04:12.651034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"250","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-20T06:04:13.997753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1269951230":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2916844765":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2625213384":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2535253896":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/766391941":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/3950316648":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2606010957":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/3612789559":"2023-12-21T06:04:27.557099+0000","172.21.15.32:6833/64546":"2023-12-21T06:04:27.557099+0000","172.21.15.32:6832/64546":"2023-12-21T06:04:27.557099+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:04:33.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 -- 172.21.15.32:0/2958424856 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd334065b80 msgr2=0x7fd334068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:33.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd334065b80 0x7fd334068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 -- 172.21.15.32:0/2958424856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480d2740 msgr2=0x7fd3480c9c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:33.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480d2740 0x7fd3480c9c30 secure :-1 s=READY pgs=2552 cs=0 l=1 rev1=1 crypto rx=0x7fd33c0b3040 tx=0x7fd33c01ba90 comp rx=0 tx=0).stop 2023-12-20T06:04:33.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 -- 172.21.15.32:0/2958424856 shutdown_connections 2023-12-20T06:04:33.501 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd334065b80 0x7fd334068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd34815cdb0 0x7fd348150950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd3480d2740 0x7fd3480c9c30 unknown :-1 s=CLOSED pgs=2552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 --2- 172.21.15.32:0/2958424856 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd3480d1120 0x7fd3481f5d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 -- 172.21.15.32:0/2958424856 >> 172.21.15.32:0/2958424856 conn(0x7fd348065ee0 msgr2=0x7fd34815bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:33.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 -- 172.21.15.32:0/2958424856 shutdown_connections 2023-12-20T06:04:33.502 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.501+0000 7fd34e695640 1 -- 172.21.15.32:0/2958424856 wait complete. 2023-12-20T06:04:33.521 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2023-12-20T04:28:26.187001+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '17', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_acting': 8, 'score_stable': 8, 'optimal_score': 0.25, 'raw_score_acting': 2, 'raw_score_stable': 2, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 32, 'pool_name': 'cephfs_metadata', 'create_time': '2023-12-20T06:04:12.651034+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '250', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 33, 'pool_name': 'cephfs_data', 'create_time': '2023-12-20T06:04:13.997753+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 2, 'min_size': 1, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '251', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_acting': 1.75, 'score_stable': 1.75, 'optimal_score': 1, 'raw_score_acting': 1.75, 'raw_score_stable': 1.75, 'primary_affinity_weighted': 1, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2023-12-20T06:04:33.522 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2023-12-20T06:04:33.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.669+0000 7fc6bdfa2640 1 Processor -- start 2023-12-20T06:04:33.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.669+0000 7fc6bdfa2640 1 -- start start 2023-12-20T06:04:33.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.669+0000 7fc6bdfa2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fc6b80cf3e0 0x7fc6b80cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.669+0000 7fc6bdfa2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fc6b80d1580 0x7fc6b80d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.669+0000 7fc6bdfa2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc6b80cfd80 con 0x7fc6b815ccf0 2023-12-20T06:04:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.669+0000 7fc6bdfa2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b80ce2f0 con 0x7fc6b80d1580 2023-12-20T06:04:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.669+0000 7fc6bdfa2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fc6b80d0b70 con 0x7fc6b80cf3e0 2023-12-20T06:04:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.669+0000 7fc6bcfa0640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc6b815ccf0 0x7fc6b80cd270 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41088/0 (socket says 172.21.15.32:41088) 2023-12-20T06:04:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.669+0000 7fc6bcfa0640 1 -- 172.21.15.32:0/4210986142 learned_addr learned my addr 172.21.15.32:0/4210986142 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.673+0000 7fc6bd7a1640 1 --2- 172.21.15.32:0/4210986142 >> v2:172.21.15.32:3300/0 conn(0x7fc6b80d1580 0x7fc6b80d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:33.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.673+0000 7fc6b7fff640 1 --2- 172.21.15.32:0/4210986142 >> v2:172.21.15.39:3300/0 conn(0x7fc6b80cf3e0 0x7fc6b80cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:33.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.673+0000 7fc6bd7a1640 1 -- 172.21.15.32:0/4210986142 >> v2:172.21.15.39:3300/0 conn(0x7fc6b80cf3e0 msgr2=0x7fc6b80cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:33.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.673+0000 7fc6bd7a1640 1 --2- 172.21.15.32:0/4210986142 >> v2:172.21.15.39:3300/0 conn(0x7fc6b80cf3e0 0x7fc6b80cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.673+0000 7fc6bd7a1640 1 -- 172.21.15.32:0/4210986142 >> v1:172.21.15.32:6789/0 conn(0x7fc6b815ccf0 legacy=0x7fc6b80cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:33.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.673+0000 7fc6bd7a1640 1 -- 172.21.15.32:0/4210986142 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6b80d10f0 con 0x7fc6b80d1580 2023-12-20T06:04:33.674 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.673+0000 7fc6b7fff640 1 --2- 172.21.15.32:0/4210986142 >> v2:172.21.15.39:3300/0 conn(0x7fc6b80cf3e0 0x7fc6b80cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:33.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6bd7a1640 1 --2- 172.21.15.32:0/4210986142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b80d1580 0x7fc6b80d1a10 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80ccfc0 tx=0x7fc6b001c850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6472a15c31b667df server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:33.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6b77fe640 1 -- 172.21.15.32:0/4210986142 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00a5910 con 0x7fc6b80d1580 2023-12-20T06:04:33.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6b77fe640 1 -- 172.21.15.32:0/4210986142 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6b00a5a70 con 0x7fc6b80d1580 2023-12-20T06:04:33.678 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6b77fe640 1 -- 172.21.15.32:0/4210986142 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6b00a5e00 con 0x7fc6b80d1580 2023-12-20T06:04:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/4210986142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b80d1580 msgr2=0x7fc6b80d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6bdfa2640 1 --2- 172.21.15.32:0/4210986142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b80d1580 0x7fc6b80d1a10 secure :-1 s=READY pgs=2554 cs=0 l=1 rev1=1 crypto rx=0x7fc6b80ccfc0 tx=0x7fc6b001c850 comp rx=0 tx=0).stop 2023-12-20T06:04:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/4210986142 shutdown_connections 2023-12-20T06:04:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6bdfa2640 1 --2- 172.21.15.32:0/4210986142 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b80d1580 0x7fc6b80d1a10 unknown :-1 s=CLOSED pgs=2554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6bdfa2640 1 --2- 172.21.15.32:0/4210986142 >> v2:172.21.15.39:3300/0 conn(0x7fc6b80cf3e0 0x7fc6b80cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.679 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/4210986142 >> 172.21.15.32:0/4210986142 conn(0x7fc6b80658e0 msgr2=0x7fc6b8065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:33.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/4210986142 shutdown_connections 2023-12-20T06:04:33.680 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.677+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/4210986142 wait complete. 2023-12-20T06:04:33.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bdfa2640 1 Processor -- start 2023-12-20T06:04:33.681 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bdfa2640 1 -- start start 2023-12-20T06:04:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bdfa2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc6b80cf3e0 0x7fc6b8148560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bdfa2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b80d1580 0x7fc6b8150980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bdfa2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6b815ccf0 0x7fc6b8150ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bdfa2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80ca200 con 0x7fc6b80d1580 2023-12-20T06:04:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bdfa2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b80c9b80 con 0x7fc6b815ccf0 2023-12-20T06:04:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bdfa2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc6b8151700 con 0x7fc6b80cf3e0 2023-12-20T06:04:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b80d1580 0x7fc6b8150980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b80d1580 0x7fc6b8150980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55012/0 (socket says 172.21.15.32:55012) 2023-12-20T06:04:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b7fff640 1 -- 172.21.15.32:0/2782049367 learned_addr learned my addr 172.21.15.32:0/2782049367 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:33.683 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bcfa0640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc6b80cf3e0 0x7fc6b8148560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:33.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bd7a1640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6b815ccf0 0x7fc6b8150ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:33.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b7fff640 1 -- 172.21.15.32:0/2782049367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc6b80cf3e0 msgr2=0x7fc6b8148560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:33.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b7fff640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc6b80cf3e0 0x7fc6b8148560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b7fff640 1 -- 172.21.15.32:0/2782049367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6b815ccf0 msgr2=0x7fc6b8150ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:33.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b7fff640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6b815ccf0 0x7fc6b8150ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b7fff640 1 -- 172.21.15.32:0/2782049367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc6a4009420 con 0x7fc6b80d1580 2023-12-20T06:04:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b7fff640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b80d1580 0x7fc6b8150980 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400e980 tx=0x7fc6a400ee40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b57fa640 1 -- 172.21.15.32:0/2782049367 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a400ca00 con 0x7fc6b80d1580 2023-12-20T06:04:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b57fa640 1 -- 172.21.15.32:0/2782049367 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc6a400cb60 con 0x7fc6b80d1580 2023-12-20T06:04:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/2782049367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc6b0009a70 con 0x7fc6b80d1580 2023-12-20T06:04:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6b57fa640 1 -- 172.21.15.32:0/2782049367 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc6a4013470 con 0x7fc6b80d1580 2023-12-20T06:04:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.681+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/2782049367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc6b8151400 con 0x7fc6b80d1580 2023-12-20T06:04:33.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.685+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/2782049367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc6b8074100 con 0x7fc6b80d1580 2023-12-20T06:04:33.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.685+0000 7fc6b57fa640 1 -- 172.21.15.32:0/2782049367 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 147) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fc6a40135d0 con 0x7fc6b80d1580 2023-12-20T06:04:33.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.685+0000 7fc6b57fa640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc69c065b80 0x7fc69c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:33.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.685+0000 7fc6b57fa640 1 -- 172.21.15.32:0/2782049367 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7fc6a4083680 con 0x7fc6b80d1580 2023-12-20T06:04:33.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.685+0000 7fc6bcfa0640 1 -- 172.21.15.32:0/2782049367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc69c065b80 msgr2=0x7fc69c068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:33.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.685+0000 7fc6bcfa0640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc69c065b80 0x7fc69c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:33.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.689+0000 7fc6b57fa640 1 -- 172.21.15.32:0/2782049367 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc6a4050e60 con 0x7fc6b80d1580 2023-12-20T06:04:33.760 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:33.757+0000 7f560c28d180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:04:33.848 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:33.845+0000 7f560c28d180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:04:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/2782049367 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) v1 -- 0x7fc6b80d0720 con 0x7fc6b80d1580 2023-12-20T06:04:33.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6b57fa640 1 -- 172.21.15.32:0/2782049367 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v252) v1 ==== 93+0+10 (secure 0 0 0) 0x7fc6a4054480 con 0x7fc6b80d1580 2023-12-20T06:04:33.850 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 1 2023-12-20T06:04:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/2782049367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc69c065b80 msgr2=0x7fc69c068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc69c065b80 0x7fc69c068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/2782049367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b80d1580 msgr2=0x7fc6b8150980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b80d1580 0x7fc6b8150980 secure :-1 s=READY pgs=2555 cs=0 l=1 rev1=1 crypto rx=0x7fc6a400e980 tx=0x7fc6a400ee40 comp rx=0 tx=0).stop 2023-12-20T06:04:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/2782049367 shutdown_connections 2023-12-20T06:04:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc69c065b80 0x7fc69c068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc6b815ccf0 0x7fc6b8150ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc6b80d1580 0x7fc6b8150980 unknown :-1 s=CLOSED pgs=2555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 --2- 172.21.15.32:0/2782049367 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc6b80cf3e0 0x7fc6b8148560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/2782049367 >> 172.21.15.32:0/2782049367 conn(0x7fc6b80658e0 msgr2=0x7fc6b80b7a60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:33.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/2782049367 shutdown_connections 2023-12-20T06:04:33.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:33.849+0000 7fc6bdfa2640 1 -- 172.21.15.32:0/2782049367 wait complete. 2023-12-20T06:04:33.870 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:33.869+0000 7fe7e0848180 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2023-12-20T06:04:33.872 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2023-12-20T06:04:33.961 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:33.961+0000 7fe7e0848180 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2023-12-20T06:04:34.004 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:34.001+0000 7f560c28d180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:04:34.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabc1b98640 1 Processor -- start 2023-12-20T06:04:34.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabc1b98640 1 -- start start 2023-12-20T06:04:34.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabc1b98640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc0b0cb0 0x7fabbc0caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.022 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabc1b98640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabbc15b6a0 con 0x7fabbc15abb0 2023-12-20T06:04:34.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabc1b98640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fabbc06d8c0 con 0x7fabbc070230 2023-12-20T06:04:34.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabc1b98640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc06e160 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabbb7fe640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fabbc15abb0 0x7fabbc15bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41102/0 (socket says 172.21.15.32:41102) 2023-12-20T06:04:34.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabbb7fe640 1 -- 172.21.15.32:0/1456250971 learned_addr learned my addr 172.21.15.32:0/1456250971 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:34.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabbaffd640 1 --2- 172.21.15.32:0/1456250971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc0b0cb0 0x7fabbc0caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabba7fc640 1 -- 172.21.15.32:0/1456250971 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3159577422 0 0) 0x7fabbc15b6a0 con 0x7fabbc15abb0 2023-12-20T06:04:34.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabba7fc640 1 -- 172.21.15.32:0/1456250971 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7faba4003530 con 0x7fabbc15abb0 2023-12-20T06:04:34.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabba7fc640 1 -- 172.21.15.32:0/1456250971 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3901806130 0 0) 0x7fabbc06d8c0 con 0x7fabbc070230 2023-12-20T06:04:34.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabba7fc640 1 -- 172.21.15.32:0/1456250971 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fabbc15b6a0 con 0x7fabbc070230 2023-12-20T06:04:34.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabbaffd640 1 -- 172.21.15.32:0/1456250971 >> v1:172.21.15.39:6789/0 conn(0x7fabbc070230 legacy=0x7fabbc06c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabbaffd640 1 -- 172.21.15.32:0/1456250971 >> v1:172.21.15.32:6789/0 conn(0x7fabbc15abb0 legacy=0x7fabbc15bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.021+0000 7fabbaffd640 1 -- 172.21.15.32:0/1456250971 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabbc06dba0 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.025+0000 7fabbaffd640 1 --2- 172.21.15.32:0/1456250971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc0b0cb0 0x7fabbc0caaf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fabb00097f0 tx=0x7fabb001c4c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=dafe50447099b26c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:34.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.025+0000 7fabba7fc640 1 -- 172.21.15.32:0/1456250971 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb00aa620 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.025+0000 7fabba7fc640 1 -- 172.21.15.32:0/1456250971 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb00aa780 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.026 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.025+0000 7fabba7fc640 1 -- 172.21.15.32:0/1456250971 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb00aab10 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.025+0000 7fabc1b98640 1 -- 172.21.15.32:0/1456250971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc0b0cb0 msgr2=0x7fabbc0caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.025+0000 7fabc1b98640 1 --2- 172.21.15.32:0/1456250971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc0b0cb0 0x7fabbc0caaf0 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7fabb00097f0 tx=0x7fabb001c4c0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.025+0000 7fabc1b98640 1 -- 172.21.15.32:0/1456250971 shutdown_connections 2023-12-20T06:04:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.025+0000 7fabc1b98640 1 --2- 172.21.15.32:0/1456250971 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc0b0cb0 0x7fabbc0caaf0 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.025+0000 7fabc1b98640 1 -- 172.21.15.32:0/1456250971 >> 172.21.15.32:0/1456250971 conn(0x7fabbc06a510 msgr2=0x7fabbc0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.025+0000 7fabc1b98640 1 -- 172.21.15.32:0/1456250971 shutdown_connections 2023-12-20T06:04:34.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.025+0000 7fabc1b98640 1 -- 172.21.15.32:0/1456250971 wait complete. 2023-12-20T06:04:34.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabc1b98640 1 Processor -- start 2023-12-20T06:04:34.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabc1b98640 1 -- start start 2023-12-20T06:04:34.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabc1b98640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc070230 0x7fabbc158de0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabc1b98640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc0b0cb0 0x7fabbc14eee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabc1b98640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc15abb0 0x7fabbc14f420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabc1b98640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc14f960 con 0x7fabbc15abb0 2023-12-20T06:04:34.030 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabc1b98640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc1f60e0 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabc1b98640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fabbc1514a0 con 0x7fabbc070230 2023-12-20T06:04:34.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbb7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc070230 0x7fabbc158de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbbfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc15abb0 0x7fabbc14f420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbaffd640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc0b0cb0 0x7fabbc14eee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbb7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc070230 0x7fabbc158de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:56562/0 (socket says 172.21.15.32:56562) 2023-12-20T06:04:34.031 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbb7fe640 1 -- 172.21.15.32:0/3439958559 learned_addr learned my addr 172.21.15.32:0/3439958559 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:34.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbaffd640 1 -- 172.21.15.32:0/3439958559 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc070230 msgr2=0x7fabbc158de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbaffd640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc070230 0x7fabbc158de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbaffd640 1 -- 172.21.15.32:0/3439958559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc15abb0 msgr2=0x7fabbc14f420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbaffd640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc15abb0 0x7fabbc14f420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbaffd640 1 -- 172.21.15.32:0/3439958559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fabb0009420 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbbfff640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc15abb0 0x7fabbc14f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:34.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabbaffd640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc0b0cb0 0x7fabbc14eee0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fabb00ab000 tx=0x7fabb001ad00 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:34.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7faba3fff640 1 -- 172.21.15.32:0/3439958559 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb00aadc0 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7faba3fff640 1 -- 172.21.15.32:0/3439958559 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fabb00be410 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabc1b98640 1 -- 172.21.15.32:0/3439958559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fabbc150c00 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7faba3fff640 1 -- 172.21.15.32:0/3439958559 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fabb00be7c0 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabc1b98640 1 -- 172.21.15.32:0/3439958559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fabbc1511c0 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.029+0000 7fabc1b98640 1 -- 172.21.15.32:0/3439958559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fabbc1f6230 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.033+0000 7faba3fff640 1 -- 172.21.15.32:0/3439958559 <== mon.1 v2:172.21.15.39:3300/0 4 ==== mgrmap(e 147) v1 ==== 85331+0+0 (secure 0 0 0) 0x7fabb00be920 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.033+0000 7faba3fff640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 0x7faba8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.033+0000 7fabbb7fe640 1 -- 172.21.15.32:0/3439958559 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 msgr2=0x7faba8068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:34.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.033+0000 7faba3fff640 1 -- 172.21.15.32:0/3439958559 <== mon.1 v2:172.21.15.39:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7fabb011dd00 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.033+0000 7fabbb7fe640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 0x7faba8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:34.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.037+0000 7faba3fff640 1 -- 172.21.15.32:0/3439958559 <== mon.1 v2:172.21.15.39:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fabbc1f6230 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.093 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:34.089+0000 7f560c28d180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:04:34.125 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.125+0000 7fe7e0848180 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2023-12-20T06:04:34.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.197+0000 7fabc1b98640 1 -- 172.21.15.32:0/3439958559 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) v1 -- 0x7fabbc1f6230 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.197+0000 7faba3fff640 1 -- 172.21.15.32:0/3439958559 <== mon.1 v2:172.21.15.39:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v252) v1 ==== 104+0+11 (secure 0 0 0) 0x7fabbc1f6230 con 0x7fabbc0b0cb0 2023-12-20T06:04:34.198 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 32 2023-12-20T06:04:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 -- 172.21.15.32:0/3439958559 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 msgr2=0x7faba8068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 0x7faba8068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 -- 172.21.15.32:0/3439958559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc0b0cb0 msgr2=0x7fabbc14eee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc0b0cb0 0x7fabbc14eee0 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fabb00ab000 tx=0x7fabb001ad00 comp rx=0 tx=0).stop 2023-12-20T06:04:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 -- 172.21.15.32:0/3439958559 shutdown_connections 2023-12-20T06:04:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7faba8065b80 0x7faba8068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fabbc15abb0 0x7fabbc14f420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fabbc0b0cb0 0x7fabbc14eee0 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 --2- 172.21.15.32:0/3439958559 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fabbc070230 0x7fabbc158de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 -- 172.21.15.32:0/3439958559 >> 172.21.15.32:0/3439958559 conn(0x7fabbc06a510 msgr2=0x7fabbc0c3660 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 -- 172.21.15.32:0/3439958559 shutdown_connections 2023-12-20T06:04:34.203 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.201+0000 7fabc1b98640 1 -- 172.21.15.32:0/3439958559 wait complete. 2023-12-20T06:04:34.223 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.221+0000 7fe7e0848180 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2023-12-20T06:04:34.229 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2023-12-20T06:04:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.373+0000 7f50df59e640 1 Processor -- start 2023-12-20T06:04:34.376 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.373+0000 7f50df59e640 1 -- start start 2023-12-20T06:04:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50df59e640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f50e00cf3e0 0x7f50e00cf840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50df59e640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50e00d1580 0x7f50e00d1a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50df59e640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f50e00cfd80 con 0x7f50e015ccf0 2023-12-20T06:04:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50df59e640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e00ce2f0 con 0x7f50e00d1580 2023-12-20T06:04:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50df59e640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f50e00d0b70 con 0x7f50e00cf3e0 2023-12-20T06:04:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50ded9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50e00d1580 0x7f50e00d1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50ded9d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f50e00d1580 0x7f50e00d1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55034/0 (socket says 172.21.15.32:55034) 2023-12-20T06:04:34.378 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50ded9d640 1 -- 172.21.15.32:0/172920872 learned_addr learned my addr 172.21.15.32:0/172920872 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50d7fff640 1 --2- 172.21.15.32:0/172920872 >> v2:172.21.15.39:3300/0 conn(0x7f50e00cf3e0 0x7f50e00cf840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50ded9d640 1 -- 172.21.15.32:0/172920872 >> v2:172.21.15.39:3300/0 conn(0x7f50e00cf3e0 msgr2=0x7f50e00cf840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50ded9d640 1 --2- 172.21.15.32:0/172920872 >> v2:172.21.15.39:3300/0 conn(0x7f50e00cf3e0 0x7f50e00cf840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50ded9d640 1 -- 172.21.15.32:0/172920872 >> v1:172.21.15.39:6789/0 conn(0x7f50e015ccf0 legacy=0x7f50e00cd270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.379 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.377+0000 7f50ded9d640 1 -- 172.21.15.32:0/172920872 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50e00d10f0 con 0x7f50e00d1580 2023-12-20T06:04:34.380 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50ded9d640 1 --2- 172.21.15.32:0/172920872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e00d1580 0x7f50e00d1a10 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f50e00ccfc0 tx=0x7f50d801c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a88780840dfcb4c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50ddd9b640 1 -- 172.21.15.32:0/172920872 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80a57d0 con 0x7f50e00d1580 2023-12-20T06:04:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50ddd9b640 1 -- 172.21.15.32:0/172920872 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d80a5930 con 0x7f50e00d1580 2023-12-20T06:04:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50ddd9b640 1 -- 172.21.15.32:0/172920872 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80a5cc0 con 0x7f50e00d1580 2023-12-20T06:04:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 -- 172.21.15.32:0/172920872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e00d1580 msgr2=0x7f50e00d1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 --2- 172.21.15.32:0/172920872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e00d1580 0x7f50e00d1a10 secure :-1 s=READY pgs=2557 cs=0 l=1 rev1=1 crypto rx=0x7f50e00ccfc0 tx=0x7f50d801c910 comp rx=0 tx=0).stop 2023-12-20T06:04:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 -- 172.21.15.32:0/172920872 shutdown_connections 2023-12-20T06:04:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 --2- 172.21.15.32:0/172920872 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e00d1580 0x7f50e00d1a10 unknown :-1 s=CLOSED pgs=2557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 --2- 172.21.15.32:0/172920872 >> v2:172.21.15.39:3300/0 conn(0x7f50e00cf3e0 0x7f50e00cf840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 -- 172.21.15.32:0/172920872 >> 172.21.15.32:0/172920872 conn(0x7f50e00658e0 msgr2=0x7f50e0065cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 -- 172.21.15.32:0/172920872 shutdown_connections 2023-12-20T06:04:34.382 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 -- 172.21.15.32:0/172920872 wait complete. 2023-12-20T06:04:34.383 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 Processor -- start 2023-12-20T06:04:34.384 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 -- start start 2023-12-20T06:04:34.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50e00cf3e0 0x7f50e0153dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f50e00d1580 0x7f50e0154310 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.381+0000 7f50df59e640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e015ccf0 0x7f50e0147780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50df59e640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e015e2f0 con 0x7f50e015ccf0 2023-12-20T06:04:34.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50df59e640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f50e00c6990 con 0x7f50e00cf3e0 2023-12-20T06:04:34.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50df59e640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f50e015dc30 con 0x7f50e00d1580 2023-12-20T06:04:34.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50ded9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e015ccf0 0x7f50e0147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50ded9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e015ccf0 0x7f50e0147780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55044/0 (socket says 172.21.15.32:55044) 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50ded9d640 1 -- 172.21.15.32:0/3630526101 learned_addr learned my addr 172.21.15.32:0/3630526101 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50de59c640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50e00cf3e0 0x7f50e0153dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50d7fff640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f50e00d1580 0x7f50e0154310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50ded9d640 1 -- 172.21.15.32:0/3630526101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f50e00d1580 msgr2=0x7f50e0154310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50ded9d640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f50e00d1580 0x7f50e0154310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50ded9d640 1 -- 172.21.15.32:0/3630526101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50e00cf3e0 msgr2=0x7f50e0153dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50ded9d640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50e00cf3e0 0x7f50e0153dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50ded9d640 1 -- 172.21.15.32:0/3630526101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f50d8009b30 con 0x7f50e015ccf0 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50d7fff640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f50e00d1580 0x7f50e0154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50ded9d640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e015ccf0 0x7f50e0147780 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f50d8019520 tx=0x7f50d80196b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50d6ffd640 1 -- 172.21.15.32:0/3630526101 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80b8670 con 0x7f50e015ccf0 2023-12-20T06:04:34.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50d6ffd640 1 -- 172.21.15.32:0/3630526101 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f50d80b87d0 con 0x7f50e015ccf0 2023-12-20T06:04:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50d6ffd640 1 -- 172.21.15.32:0/3630526101 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f50d80b8b80 con 0x7f50e015ccf0 2023-12-20T06:04:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50df59e640 1 -- 172.21.15.32:0/3630526101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f50e015df10 con 0x7f50e015ccf0 2023-12-20T06:04:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50df59e640 1 -- 172.21.15.32:0/3630526101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f50e00c6540 con 0x7f50e015ccf0 2023-12-20T06:04:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50df59e640 1 -- 172.21.15.32:0/3630526101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f50e015e0a0 con 0x7f50e015ccf0 2023-12-20T06:04:34.388 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.385+0000 7f50d6ffd640 1 -- 172.21.15.32:0/3630526101 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 147) v1 ==== 85331+0+0 (secure 0 0 0) 0x7f50d80b8ce0 con 0x7f50e015ccf0 2023-12-20T06:04:34.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.389+0000 7f50d6ffd640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f50c0065b80 0x7f50c0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.389+0000 7f50d6ffd640 1 -- 172.21.15.32:0/3630526101 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(252..252 src has 1..252) v4 ==== 6195+0+0 (secure 0 0 0) 0x7f50e00d2760 con 0x7f50e015ccf0 2023-12-20T06:04:34.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.389+0000 7f50de59c640 1 -- 172.21.15.32:0/3630526101 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f50c0065b80 msgr2=0x7f50c0068040 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.32:6832/64546 2023-12-20T06:04:34.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.389+0000 7f50de59c640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f50c0065b80 0x7f50c0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2023-12-20T06:04:34.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.393+0000 7f50d6ffd640 1 -- 172.21.15.32:0/3630526101 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f50d80ec2c0 con 0x7f50e015ccf0 2023-12-20T06:04:34.428 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:34.425+0000 7f560c28d180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:04:34.491 INFO:tasks.ceph.mgr.x.smithi039.stderr:2023-12-20T06:04:34.489+0000 7f560c28d180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:04:34.552 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.549+0000 7f50df59e640 1 -- 172.21.15.32:0/3630526101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) v1 -- 0x7f50e0074100 con 0x7f50e015ccf0 2023-12-20T06:04:34.553 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50d6ffd640 1 -- 172.21.15.32:0/3630526101 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v252) v1 ==== 100+0+11 (secure 0 0 0) 0x7f50d80bc0a0 con 0x7f50e015ccf0 2023-12-20T06:04:34.553 INFO:teuthology.orchestra.run.smithi032.stdout:pg_num: 64 2023-12-20T06:04:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 -- 172.21.15.32:0/3630526101 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f50c0065b80 msgr2=0x7f50c0068040 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2023-12-20T06:04:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f50c0065b80 0x7f50c0068040 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 -- 172.21.15.32:0/3630526101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e015ccf0 msgr2=0x7f50e0147780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e015ccf0 0x7f50e0147780 secure :-1 s=READY pgs=2558 cs=0 l=1 rev1=1 crypto rx=0x7f50d8019520 tx=0x7f50d80196b0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 -- 172.21.15.32:0/3630526101 shutdown_connections 2023-12-20T06:04:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f50c0065b80 0x7f50c0068040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f50e015ccf0 0x7f50e0147780 unknown :-1 s=CLOSED pgs=2558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f50e00d1580 0x7f50e0154310 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 --2- 172.21.15.32:0/3630526101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f50e00cf3e0 0x7f50e0153dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 -- 172.21.15.32:0/3630526101 >> 172.21.15.32:0/3630526101 conn(0x7f50e00658e0 msgr2=0x7f50e00b5dc0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:34.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 -- 172.21.15.32:0/3630526101 shutdown_connections 2023-12-20T06:04:34.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.553+0000 7f50df59e640 1 -- 172.21.15.32:0/3630526101 wait complete. 2023-12-20T06:04:34.568 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.565+0000 7fe7e0848180 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2023-12-20T06:04:34.575 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:04:34.637 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.637+0000 7fe7e0848180 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2023-12-20T06:04:34.721 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.721+0000 7fe769a60640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:34.722 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.721+0000 7fe769a60640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:34.722 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.721+0000 7fe769a60640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:34.722 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.721+0000 7fe769a60640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:34.722 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.721+0000 7fe769a60640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:34.722 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.721+0000 7fe767a5c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:34.722 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.721+0000 7fe767a5c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:34.722 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.721+0000 7fe767a5c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:34.722 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.721+0000 7fe767a5c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:34.722 INFO:tasks.ceph.mgr.y.smithi032.stderr:2023-12-20T06:04:34.721+0000 7fe767a5c640 -1 client.0 error registering admin socket command: (17) File exists 2023-12-20T06:04:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c19066640 1 Processor -- start 2023-12-20T06:04:34.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c19066640 1 -- start start 2023-12-20T06:04:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c19066640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c140d2740 0x7f0c140cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c19066640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c140cfec0 con 0x7f0c140cfab0 2023-12-20T06:04:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c19066640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f0c1415d190 con 0x7f0c1415cdb0 2023-12-20T06:04:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c19066640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f0c140d16a0 con 0x7f0c140d2740 2023-12-20T06:04:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c0bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c140d2740 0x7f0c140cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c0bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f0c140d2740 0x7f0c140cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55156/0 (socket says 172.21.15.32:55156) 2023-12-20T06:04:34.740 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c0bfff640 1 -- 172.21.15.32:0/1336223472 learned_addr learned my addr 172.21.15.32:0/1336223472 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c0bfff640 1 -- 172.21.15.32:0/1336223472 >> v1:172.21.15.39:6789/0 conn(0x7f0c1415cdb0 legacy=0x7f0c140d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c0bfff640 1 -- 172.21.15.32:0/1336223472 >> v1:172.21.15.32:6789/0 conn(0x7f0c140cfab0 legacy=0x7f0c140b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c0bfff640 1 -- 172.21.15.32:0/1336223472 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0c140d1980 con 0x7f0c140d2740 2023-12-20T06:04:34.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.737+0000 7f0c1259c640 1 -- 172.21.15.32:0/1336223472 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1462780781 0 0) 0x7f0c1415d190 con 0x7f0c1415cdb0 2023-12-20T06:04:34.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.741+0000 7f0c0bfff640 1 --2- 172.21.15.32:0/1336223472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c140d2740 0x7f0c140cf570 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009870 tx=0x7f0bfc01c460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2e1c731be4bc1e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:34.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.741+0000 7f0c1259c640 1 -- 172.21.15.32:0/1336223472 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0ab600 con 0x7f0c140d2740 2023-12-20T06:04:34.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.741+0000 7f0c1259c640 1 -- 172.21.15.32:0/1336223472 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0bfc0ab760 con 0x7f0c140d2740 2023-12-20T06:04:34.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.741+0000 7f0c1259c640 1 -- 172.21.15.32:0/1336223472 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0bfc0abaf0 con 0x7f0c140d2740 2023-12-20T06:04:34.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.741+0000 7f0c19066640 1 -- 172.21.15.32:0/1336223472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c140d2740 msgr2=0x7f0c140cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.741+0000 7f0c19066640 1 --2- 172.21.15.32:0/1336223472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c140d2740 0x7f0c140cf570 secure :-1 s=READY pgs=2568 cs=0 l=1 rev1=1 crypto rx=0x7f0bfc009870 tx=0x7f0bfc01c460 comp rx=0 tx=0).stop 2023-12-20T06:04:34.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.741+0000 7f0c19066640 1 -- 172.21.15.32:0/1336223472 shutdown_connections 2023-12-20T06:04:34.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.741+0000 7f0c19066640 1 --2- 172.21.15.32:0/1336223472 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c140d2740 0x7f0c140cf570 unknown :-1 s=CLOSED pgs=2568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.741+0000 7f0c19066640 1 -- 172.21.15.32:0/1336223472 >> 172.21.15.32:0/1336223472 conn(0x7f0c14065ee0 msgr2=0x7f0c1406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:34.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.741+0000 7f0c19066640 1 -- 172.21.15.32:0/1336223472 shutdown_connections 2023-12-20T06:04:34.745 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.741+0000 7f0c19066640 1 -- 172.21.15.32:0/1336223472 wait complete. 2023-12-20T06:04:34.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c19066640 1 Processor -- start 2023-12-20T06:04:34.746 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c19066640 1 -- start start 2023-12-20T06:04:34.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c19066640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c140cfab0 0x7f0c140cc130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c19066640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c140d2740 0x7f0c141570d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c19066640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c1415cdb0 0x7f0c140c8650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:34.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c19066640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c14155cc0 con 0x7f0c140cfab0 2023-12-20T06:04:34.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c19066640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140cbca0 con 0x7f0c140d2740 2023-12-20T06:04:34.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c19066640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f0c140cb090 con 0x7f0c1415cdb0 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c12d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c140cfab0 0x7f0c140cc130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c12d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c140cfab0 0x7f0c140cc130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55162/0 (socket says 172.21.15.32:55162) 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c12d9d640 1 -- 172.21.15.32:0/1613250911 learned_addr learned my addr 172.21.15.32:0/1613250911 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c0bfff640 1 --2- 172.21.15.32:0/1613250911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c140d2740 0x7f0c141570d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c12d9d640 1 -- 172.21.15.32:0/1613250911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c1415cdb0 msgr2=0x7f0c140c8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c1359e640 1 --2- 172.21.15.32:0/1613250911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c1415cdb0 0x7f0c140c8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c12d9d640 1 --2- 172.21.15.32:0/1613250911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c1415cdb0 0x7f0c140c8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c12d9d640 1 -- 172.21.15.32:0/1613250911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c140d2740 msgr2=0x7f0c141570d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c12d9d640 1 --2- 172.21.15.32:0/1613250911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c140d2740 0x7f0c141570d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c12d9d640 1 -- 172.21.15.32:0/1613250911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f0bfc0094d0 con 0x7f0c140cfab0 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c0bfff640 1 --2- 172.21.15.32:0/1613250911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c140d2740 0x7f0c141570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c1359e640 1 --2- 172.21.15.32:0/1613250911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c1415cdb0 0x7f0c140c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c12d9d640 1 --2- 172.21.15.32:0/1613250911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c140cfab0 0x7f0c140cc130 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c4a0 tx=0x7f0c0400c960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c0b7fe640 1 -- 172.21.15.32:0/1613250911 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c04005480 con 0x7f0c140cfab0 2023-12-20T06:04:34.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c0b7fe640 1 -- 172.21.15.32:0/1613250911 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f0c04003600 con 0x7f0c140cfab0 2023-12-20T06:04:34.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c0b7fe640 1 -- 172.21.15.32:0/1613250911 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f0c040039b0 con 0x7f0c140cfab0 2023-12-20T06:04:34.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c19066640 1 -- 172.21.15.32:0/1613250911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f0c140cb370 con 0x7f0c140cfab0 2023-12-20T06:04:34.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.745+0000 7f0c19066640 1 -- 172.21.15.32:0/1613250911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f0c140cb930 con 0x7f0c140cfab0 2023-12-20T06:04:34.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.749+0000 7f0c19066640 1 -- 172.21.15.32:0/1613250911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f0c1404e000 con 0x7f0c140cfab0 2023-12-20T06:04:34.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.749+0000 7f0c0b7fe640 1 -- 172.21.15.32:0/1613250911 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 149) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f0c04003b10 con 0x7f0c140cfab0 2023-12-20T06:04:34.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.749+0000 7f0c0b7fe640 1 -- 172.21.15.32:0/1613250911 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f0c040825a0 con 0x7f0c140cfab0 2023-12-20T06:04:34.755 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.753+0000 7f0c0b7fe640 1 -- 172.21.15.32:0/1613250911 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f0c0404ffd0 con 0x7f0c140cfab0 2023-12-20T06:04:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.945+0000 7f0c19066640 1 -- 172.21.15.32:0/1613250911 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f0c1406b8d0 con 0x7f0c140cfab0 2023-12-20T06:04:34.945 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.945+0000 7f0c0b7fe640 1 -- 172.21.15.32:0/1613250911 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4722 (secure 0 0 0) 0x7f0c04017020 con 0x7f0c140cfab0 2023-12-20T06:04:34.946 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:34.946 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3779641739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3779641739},{"type":"v1","addr":"172.21.15.39:6834","nonce":3779641739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/473427978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":473427978},{"type":"v1","addr":"172.21.15.39:6835","nonce":473427978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12060,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3927543497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3927543497},{"type":"v1","addr":"172.21.15.32:6836","nonce":3927543497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:04:14.992681+0000","modified":"2023-12-20T06:04:18.007320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12063":{"gid":12063,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3133738681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3133738681},{"type":"v1","addr":"172.21.15.32:6837","nonce":3133738681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-20T06:04:34.946 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 128 2023-12-20T06:04:34.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.945+0000 7f0c19066640 1 -- 172.21.15.32:0/1613250911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c140cfab0 msgr2=0x7f0c140cc130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:34.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.945+0000 7f0c19066640 1 --2- 172.21.15.32:0/1613250911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c140cfab0 0x7f0c140cc130 secure :-1 s=READY pgs=2569 cs=0 l=1 rev1=1 crypto rx=0x7f0c0400c4a0 tx=0x7f0c0400c960 comp rx=0 tx=0).stop 2023-12-20T06:04:34.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.945+0000 7f0c19066640 1 -- 172.21.15.32:0/1613250911 shutdown_connections 2023-12-20T06:04:34.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.945+0000 7f0c19066640 1 --2- 172.21.15.32:0/1613250911 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f0c1415cdb0 0x7f0c140c8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.945+0000 7f0c19066640 1 --2- 172.21.15.32:0/1613250911 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f0c140d2740 0x7f0c141570d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.945+0000 7f0c19066640 1 --2- 172.21.15.32:0/1613250911 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f0c140cfab0 0x7f0c140cc130 unknown :-1 s=CLOSED pgs=2569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:34.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.945+0000 7f0c19066640 1 -- 172.21.15.32:0/1613250911 >> 172.21.15.32:0/1613250911 conn(0x7f0c14065ee0 msgr2=0x7f0c14153820 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:34.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.945+0000 7f0c19066640 1 -- 172.21.15.32:0/1613250911 shutdown_connections 2023-12-20T06:04:34.948 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:34.945+0000 7f0c19066640 1 -- 172.21.15.32:0/1613250911 wait complete. 2023-12-20T06:04:34.967 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:04:35.109 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f228f5e9640 1 Processor -- start 2023-12-20T06:04:35.110 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f228f5e9640 1 -- start start 2023-12-20T06:04:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f228f5e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22880d2740 0x7f22880cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f228f5e9640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f22880cfab0 0x7f22880b5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f228f5e9640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f228815d190 con 0x7f228815cdb0 2023-12-20T06:04:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f228f5e9640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f22880cff40 con 0x7f22880cfab0 2023-12-20T06:04:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f228f5e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f22880d16a0 con 0x7f22880d2740 2023-12-20T06:04:35.111 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f228d385640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f228815cdb0 0x7f22880d0be0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41124/0 (socket says 172.21.15.32:41124) 2023-12-20T06:04:35.112 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f228d385640 1 -- 172.21.15.32:0/1909272271 learned_addr learned my addr 172.21.15.32:0/1909272271 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2051463290 0 0) 0x7f228815d190 con 0x7f228815cdb0 2023-12-20T06:04:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f228db86640 1 --2- 172.21.15.32:0/1909272271 >> v2:172.21.15.39:3300/0 conn(0x7f22880cfab0 0x7f22880b5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f228cb84640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22880d2740 0x7f22880cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.113 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2274003530 con 0x7f228815cdb0 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 3867713196 0 0) 0x7f2274003530 con 0x7f228815cdb0 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22880d2740 msgr2=0x7f22880cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22880d2740 0x7f22880cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 >> v2:172.21.15.39:3300/0 conn(0x7f22880cfab0 msgr2=0x7f22880b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 --2- 172.21.15.32:0/1909272271 >> v2:172.21.15.39:3300/0 conn(0x7f22880cfab0 0x7f22880b5c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f22880d1980 con 0x7f228815cdb0 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f2284001680 con 0x7f228815cdb0 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 >> v1:172.21.15.32:6789/0 conn(0x7f228815cdb0 legacy=0x7f22880d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2274003980 0x7f2274003da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22880d2740 0x7f22740047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274004d10 0x7f2274005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880cff40 con 0x7f2274003980 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880d16a0 con 0x7f2274004d10 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.109+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f227400b990 con 0x7f22880d2740 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f228d385640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2274003980 0x7f2274003da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.114 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f228db86640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274004d10 0x7f2274005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f228cb84640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22880d2740 0x7f22740047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f228d385640 1 -- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22880d2740 msgr2=0x7f22740047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f228d385640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22880d2740 0x7f22740047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f228d385640 1 -- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274004d10 msgr2=0x7f2274005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f228d385640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274004d10 0x7f2274005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f228d385640 1 -- 172.21.15.32:0/1909272271 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2274003530 con 0x7f2274003980 2023-12-20T06:04:35.115 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f228cb84640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f22880d2740 0x7f22740047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f228d385640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2274003980 0x7f2274003da0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f228400c240 tx=0x7f228401f3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dde54bc07a164b1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284005ae0 con 0x7f2274003980 2023-12-20T06:04:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f22840ab070 con 0x7f2274003980 2023-12-20T06:04:35.116 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.113+0000 7f227ffff640 1 -- 172.21.15.32:0/1909272271 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2284005ea0 con 0x7f2274003980 2023-12-20T06:04:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 -- 172.21.15.32:0/1909272271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2274003980 msgr2=0x7f2274003da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 --2- 172.21.15.32:0/1909272271 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2274003980 0x7f2274003da0 secure :-1 s=READY pgs=2571 cs=0 l=1 rev1=1 crypto rx=0x7f228400c240 tx=0x7f228401f3d0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228db86640 1 -- 172.21.15.32:0/1909272271 reap_dead start 2023-12-20T06:04:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 -- 172.21.15.32:0/1909272271 shutdown_connections 2023-12-20T06:04:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 -- 172.21.15.32:0/1909272271 >> 172.21.15.32:0/1909272271 conn(0x7f2288065ee0 msgr2=0x7f228806b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 -- 172.21.15.32:0/1909272271 shutdown_connections 2023-12-20T06:04:35.117 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 -- 172.21.15.32:0/1909272271 wait complete. 2023-12-20T06:04:35.118 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 Processor -- start 2023-12-20T06:04:35.119 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 -- start start 2023-12-20T06:04:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274003980 0x7f22880c8dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f228813a590 0x7f228813aa10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2288151540 0x7f22881519c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f228814f990 con 0x7f228813a590 2023-12-20T06:04:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f22880c9300 con 0x7f2274003980 2023-12-20T06:04:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228f5e9640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f228813af50 con 0x7f2288151540 2023-12-20T06:04:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228cb84640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f228813a590 0x7f228813aa10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228cb84640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f228813a590 0x7f228813aa10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55178/0 (socket says 172.21.15.32:55178) 2023-12-20T06:04:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228cb84640 1 -- 172.21.15.32:0/2062993316 learned_addr learned my addr 172.21.15.32:0/2062993316 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:35.121 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228d385640 1 --2- 172.21.15.32:0/2062993316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274003980 0x7f22880c8dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228db86640 1 --2- 172.21.15.32:0/2062993316 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2288151540 0x7f22881519c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228cb84640 1 -- 172.21.15.32:0/2062993316 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2288151540 msgr2=0x7f22881519c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228cb84640 1 --2- 172.21.15.32:0/2062993316 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2288151540 0x7f22881519c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228cb84640 1 -- 172.21.15.32:0/2062993316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274003980 msgr2=0x7f22880c8dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228cb84640 1 --2- 172.21.15.32:0/2062993316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274003980 0x7f22880c8dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.117+0000 7f228cb84640 1 -- 172.21.15.32:0/2062993316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2288151f00 con 0x7f228813a590 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.121+0000 7f228cb84640 1 --2- 172.21.15.32:0/2062993316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f228813a590 0x7f228813aa10 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f22780115f0 tx=0x7f2278011ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.121+0000 7f227dffb640 1 -- 172.21.15.32:0/2062993316 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2278004220 con 0x7f228813a590 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.121+0000 7f227dffb640 1 -- 172.21.15.32:0/2062993316 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2278004380 con 0x7f228813a590 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.121+0000 7f228f5e9640 1 -- 172.21.15.32:0/2062993316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f22881f6340 con 0x7f228813a590 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.121+0000 7f227dffb640 1 -- 172.21.15.32:0/2062993316 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f227800f8f0 con 0x7f228813a590 2023-12-20T06:04:35.122 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.121+0000 7f228f5e9640 1 -- 172.21.15.32:0/2062993316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f228814eff0 con 0x7f228813a590 2023-12-20T06:04:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.121+0000 7f228f5e9640 1 -- 172.21.15.32:0/2062993316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f228804e000 con 0x7f228813a590 2023-12-20T06:04:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.121+0000 7f227dffb640 1 -- 172.21.15.32:0/2062993316 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 149) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f227800fa50 con 0x7f228813a590 2023-12-20T06:04:35.123 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.121+0000 7f227dffb640 1 -- 172.21.15.32:0/2062993316 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f2278082fd0 con 0x7f228813a590 2023-12-20T06:04:35.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.125+0000 7f227dffb640 1 -- 172.21.15.32:0/2062993316 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2278050a00 con 0x7f228813a590 2023-12-20T06:04:35.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.285+0000 7f228f5e9640 1 -- 172.21.15.32:0/2062993316 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f22881f6340 con 0x7f228813a590 2023-12-20T06:04:35.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.285+0000 7f227dffb640 1 -- 172.21.15.32:0/2062993316 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v253) v1 ==== 74+0+17082 (secure 0 0 0) 0x7f22881f6340 con 0x7f228813a590 2023-12-20T06:04:35.287 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:35.287 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":253,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:04:34.642501+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:04:12.651034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"250","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-20T06:04:13.997753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1848378616":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/3742412743":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/1548603036":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/2131291246":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/2118335854":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/1269951230":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2916844765":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2625213384":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/375930125":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/2535253896":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/766391941":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/3950316648":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2606010957":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/3612789559":"2023-12-21T06:04:27.557099+0000","172.21.15.32:6833/64546":"2023-12-21T06:04:27.557099+0000","172.21.15.32:6832/64546":"2023-12-21T06:04:27.557099+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:04:35.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.285+0000 7f228f5e9640 1 -- 172.21.15.32:0/2062993316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f228813a590 msgr2=0x7f228813aa10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.285+0000 7f228f5e9640 1 --2- 172.21.15.32:0/2062993316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f228813a590 0x7f228813aa10 secure :-1 s=READY pgs=2572 cs=0 l=1 rev1=1 crypto rx=0x7f22780115f0 tx=0x7f2278011ab0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.285+0000 7f228f5e9640 1 -- 172.21.15.32:0/2062993316 shutdown_connections 2023-12-20T06:04:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.285+0000 7f228f5e9640 1 --2- 172.21.15.32:0/2062993316 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2288151540 0x7f22881519c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.285+0000 7f228f5e9640 1 --2- 172.21.15.32:0/2062993316 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f228813a590 0x7f228813aa10 unknown :-1 s=CLOSED pgs=2572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.285+0000 7f228f5e9640 1 --2- 172.21.15.32:0/2062993316 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2274003980 0x7f22880c8dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.285+0000 7f228f5e9640 1 -- 172.21.15.32:0/2062993316 >> 172.21.15.32:0/2062993316 conn(0x7f2288065ee0 msgr2=0x7f2288154130 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.289+0000 7f228f5e9640 1 -- 172.21.15.32:0/2062993316 shutdown_connections 2023-12-20T06:04:35.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.289+0000 7f228f5e9640 1 -- 172.21.15.32:0/2062993316 wait complete. 2023-12-20T06:04:35.309 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T06:04:35.309 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:04:35.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c467b3640 1 Processor -- start 2023-12-20T06:04:35.454 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c467b3640 1 -- start start 2023-12-20T06:04:35.455 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c467b3640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6c4015cdb0 0x7f6c400d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c467b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c400d2740 0x7f6c400cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c467b3640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6c400b6ca0 con 0x7f6c400d1120 2023-12-20T06:04:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c467b3640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6c4015d190 con 0x7f6c4015cdb0 2023-12-20T06:04:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c467b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400d16a0 con 0x7f6c400d2740 2023-12-20T06:04:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c44d50640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f6c400d1120 0x7f6c400b45e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41126/0 (socket says 172.21.15.32:41126) 2023-12-20T06:04:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c44d50640 1 -- 172.21.15.32:0/1594717093 learned_addr learned my addr 172.21.15.32:0/1594717093 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c3f7fe640 1 --2- 172.21.15.32:0/1594717093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c400d2740 0x7f6c400cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.456 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c3ffff640 1 --2- 172.21.15.32:0/1594717093 >> v2:172.21.15.39:3300/0 conn(0x7f6c4015cdb0 0x7f6c400d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c3effd640 1 -- 172.21.15.32:0/1594717093 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4243974619 0 0) 0x7f6c380014a0 con 0x7f6c400d1120 2023-12-20T06:04:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c3effd640 1 -- 172.21.15.32:0/1594717093 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f6c28003530 con 0x7f6c400d1120 2023-12-20T06:04:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c3f7fe640 1 -- 172.21.15.32:0/1594717093 >> v2:172.21.15.39:3300/0 conn(0x7f6c4015cdb0 msgr2=0x7f6c400d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c3f7fe640 1 --2- 172.21.15.32:0/1594717093 >> v2:172.21.15.39:3300/0 conn(0x7f6c4015cdb0 0x7f6c400d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c3f7fe640 1 -- 172.21.15.32:0/1594717093 >> v1:172.21.15.32:6789/0 conn(0x7f6c400d1120 legacy=0x7f6c400b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.457 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.453+0000 7f6c3f7fe640 1 -- 172.21.15.32:0/1594717093 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c400d1980 con 0x7f6c400d2740 2023-12-20T06:04:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c3f7fe640 1 --2- 172.21.15.32:0/1594717093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c400d2740 0x7f6c400cf570 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6c340097c0 tx=0x7f6c3401ec70 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2de167381afa308c server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c3effd640 1 -- 172.21.15.32:0/1594717093 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c340a8970 con 0x7f6c400d2740 2023-12-20T06:04:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c3effd640 1 -- 172.21.15.32:0/1594717093 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c340a8ad0 con 0x7f6c400d2740 2023-12-20T06:04:35.459 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c3effd640 1 -- 172.21.15.32:0/1594717093 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c340a8e40 con 0x7f6c400d2740 2023-12-20T06:04:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c467b3640 1 -- 172.21.15.32:0/1594717093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c400d2740 msgr2=0x7f6c400cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c467b3640 1 --2- 172.21.15.32:0/1594717093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c400d2740 0x7f6c400cf570 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7f6c340097c0 tx=0x7f6c3401ec70 comp rx=0 tx=0).stop 2023-12-20T06:04:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c467b3640 1 -- 172.21.15.32:0/1594717093 shutdown_connections 2023-12-20T06:04:35.460 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c467b3640 1 --2- 172.21.15.32:0/1594717093 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c400d2740 0x7f6c400cf570 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c467b3640 1 --2- 172.21.15.32:0/1594717093 >> v2:172.21.15.39:3300/0 conn(0x7f6c4015cdb0 0x7f6c400d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c467b3640 1 -- 172.21.15.32:0/1594717093 >> 172.21.15.32:0/1594717093 conn(0x7f6c40065ee0 msgr2=0x7f6c4006b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:35.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c467b3640 1 -- 172.21.15.32:0/1594717093 shutdown_connections 2023-12-20T06:04:35.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.457+0000 7f6c467b3640 1 -- 172.21.15.32:0/1594717093 wait complete. 2023-12-20T06:04:35.461 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c467b3640 1 Processor -- start 2023-12-20T06:04:35.462 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c467b3640 1 -- start start 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c467b3640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c400d1120 0x7f6c4015d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c467b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400d2740 0x7f6c4015ad90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c467b3640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4015cdb0 0x7f6c4015b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c467b3640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400cc320 con 0x7f6c400d2740 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c467b3640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400cb710 con 0x7f6c4015cdb0 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c467b3640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6c400cb9f0 con 0x7f6c400d1120 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400d2740 0x7f6c4015ad90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400d2740 0x7f6c4015ad90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55190/0 (socket says 172.21.15.32:55190) 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3f7fe640 1 -- 172.21.15.32:0/3434633388 learned_addr learned my addr 172.21.15.32:0/3434633388 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c44d50640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4015cdb0 0x7f6c4015b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3ffff640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c400d1120 0x7f6c4015d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3f7fe640 1 -- 172.21.15.32:0/3434633388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c400d1120 msgr2=0x7f6c4015d360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.464 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3f7fe640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c400d1120 0x7f6c4015d360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3f7fe640 1 -- 172.21.15.32:0/3434633388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4015cdb0 msgr2=0x7f6c4015b2d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3f7fe640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4015cdb0 0x7f6c4015b2d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3f7fe640 1 -- 172.21.15.32:0/3434633388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6c34009420 con 0x7f6c400d2740 2023-12-20T06:04:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3f7fe640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400d2740 0x7f6c4015ad90 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f6c3401c8a0 tx=0x7f6c3401ba10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3cff9640 1 -- 172.21.15.32:0/3434633388 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c340bd070 con 0x7f6c400d2740 2023-12-20T06:04:35.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3cff9640 1 -- 172.21.15.32:0/3434633388 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6c34004250 con 0x7f6c400d2740 2023-12-20T06:04:35.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c3cff9640 1 -- 172.21.15.32:0/3434633388 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6c34003620 con 0x7f6c400d2740 2023-12-20T06:04:35.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.461+0000 7f6c467b3640 1 -- 172.21.15.32:0/3434633388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6c400cbcd0 con 0x7f6c400d2740 2023-12-20T06:04:35.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.465+0000 7f6c467b3640 1 -- 172.21.15.32:0/3434633388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6c4015d8a0 con 0x7f6c400d2740 2023-12-20T06:04:35.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.465+0000 7f6c467b3640 1 -- 172.21.15.32:0/3434633388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6c4004e000 con 0x7f6c400d2740 2023-12-20T06:04:35.466 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.465+0000 7f6c3cff9640 1 -- 172.21.15.32:0/3434633388 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 149) v1 ==== 84939+0+0 (secure 0 0 0) 0x7f6c340b7040 con 0x7f6c400d2740 2023-12-20T06:04:35.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.465+0000 7f6c3cff9640 1 -- 172.21.15.32:0/3434633388 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f6c3411fdf0 con 0x7f6c400d2740 2023-12-20T06:04:35.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.469+0000 7f6c3cff9640 1 -- 172.21.15.32:0/3434633388 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6c340ed9c0 con 0x7f6c400d2740 2023-12-20T06:04:35.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.661+0000 7f6c3cff9640 1 -- 172.21.15.32:0/3434633388 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 150) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f6c340ed470 con 0x7f6c400d2740 2023-12-20T06:04:35.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.661+0000 7f6c3cff9640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c2c01c9c0 0x7f6c2c06d250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.665+0000 7f6c3ffff640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c2c01c9c0 0x7f6c2c06d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.665+0000 7f6c467b3640 1 -- 172.21.15.32:0/3434633388 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f6c4006b8d0 con 0x7f6c400d2740 2023-12-20T06:04:35.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c3ffff640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c2c01c9c0 0x7f6c2c06d250 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009b60 tx=0x7f6c30009190 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:35.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c3cff9640 1 -- 172.21.15.32:0/3434633388 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4722 (secure 0 0 0) 0x7f6c340f4ed0 con 0x7f6c400d2740 2023-12-20T06:04:35.669 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:35.670 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3779641739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3779641739},{"type":"v1","addr":"172.21.15.39:6834","nonce":3779641739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/473427978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":473427978},{"type":"v1","addr":"172.21.15.39:6835","nonce":473427978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12060,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3927543497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3927543497},{"type":"v1","addr":"172.21.15.32:6836","nonce":3927543497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:04:14.992681+0000","modified":"2023-12-20T06:04:18.007320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12063":{"gid":12063,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3133738681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3133738681},{"type":"v1","addr":"172.21.15.32:6837","nonce":3133738681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-20T06:04:35.670 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 128 2023-12-20T06:04:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 -- 172.21.15.32:0/3434633388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c2c01c9c0 msgr2=0x7f6c2c06d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c2c01c9c0 0x7f6c2c06d250 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f6c30009b60 tx=0x7f6c30009190 comp rx=0 tx=0).stop 2023-12-20T06:04:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 -- 172.21.15.32:0/3434633388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400d2740 msgr2=0x7f6c4015ad90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400d2740 0x7f6c4015ad90 secure :-1 s=READY pgs=2574 cs=0 l=1 rev1=1 crypto rx=0x7f6c3401c8a0 tx=0x7f6c3401ba10 comp rx=0 tx=0).stop 2023-12-20T06:04:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 -- 172.21.15.32:0/3434633388 shutdown_connections 2023-12-20T06:04:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6c2c01c9c0 0x7f6c2c06d250 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6c4015cdb0 0x7f6c4015b2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6c400d2740 0x7f6c4015ad90 unknown :-1 s=CLOSED pgs=2574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 --2- 172.21.15.32:0/3434633388 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6c400d1120 0x7f6c4015d360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 -- 172.21.15.32:0/3434633388 >> 172.21.15.32:0/3434633388 conn(0x7f6c40065ee0 msgr2=0x7f6c40157e40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:35.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 -- 172.21.15.32:0/3434633388 shutdown_connections 2023-12-20T06:04:35.673 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.669+0000 7f6c467b3640 1 -- 172.21.15.32:0/3434633388 wait complete. 2023-12-20T06:04:35.691 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:04:35.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.833+0000 7fc251596640 1 Processor -- start 2023-12-20T06:04:35.835 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.833+0000 7fc251596640 1 -- start start 2023-12-20T06:04:35.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.833+0000 7fc251596640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c160ff0 0x7fc24c0cf160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.833+0000 7fc251596640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c0ce6c0 con 0x7fc24c15cd70 2023-12-20T06:04:35.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.833+0000 7fc251596640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fc24c1615d0 con 0x7fc24c0b2ae0 2023-12-20T06:04:35.836 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.833+0000 7fc251596640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c1618b0 con 0x7fc24c160ff0 2023-12-20T06:04:35.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.833+0000 7fc24affd640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fc24c15cd70 0x7fc24c0b23d0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41132/0 (socket says 172.21.15.32:41132) 2023-12-20T06:04:35.837 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.833+0000 7fc24affd640 1 -- 172.21.15.32:0/474185885 learned_addr learned my addr 172.21.15.32:0/474185885 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:35.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24b7fe640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c160ff0 0x7fc24c0cf160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1488973592 0 0) 0x7fc24c0ce6c0 con 0x7fc24c15cd70 2023-12-20T06:04:35.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fc238003530 con 0x7fc24c15cd70 2023-12-20T06:04:35.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 1344283943 0 0) 0x7fc238003530 con 0x7fc24c15cd70 2023-12-20T06:04:35.838 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c160ff0 msgr2=0x7fc24c0cf160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c160ff0 0x7fc24c0cf160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 >> v1:172.21.15.39:6789/0 conn(0x7fc24c0b2ae0 legacy=0x7fc24c1608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c0ce960 con 0x7fc24c15cd70 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7fc23c001680 con 0x7fc24c15cd70 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 >> v1:172.21.15.32:6789/0 conn(0x7fc24c15cd70 legacy=0x7fc24c0b23d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c160ff0 0x7fc2380039b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 0x7fc2380047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc238004d10 0x7fc238005190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c1618b0 con 0x7fc238003ef0 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc23800b9f0 con 0x7fc238004d10 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc23800bbb0 con 0x7fc24c160ff0 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc243fff640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 0x7fc2380047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24affd640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c160ff0 0x7fc2380039b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc24b7fe640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc238004d10 0x7fc238005190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc243fff640 1 -- 172.21.15.32:0/474185885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c160ff0 msgr2=0x7fc2380039b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc243fff640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c160ff0 0x7fc2380039b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.839 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc243fff640 1 -- 172.21.15.32:0/474185885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc238004d10 msgr2=0x7fc238005190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:35.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc243fff640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc238004d10 0x7fc238005190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.840 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.837+0000 7fc243fff640 1 -- 172.21.15.32:0/474185885 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc238003530 con 0x7fc238003ef0 2023-12-20T06:04:35.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc243fff640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 0x7fc2380047d0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fc234001860 tx=0x7fc23401eb90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c27da6cc04aff2f5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:35.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2340a87b0 con 0x7fc238003ef0 2023-12-20T06:04:35.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc2340a8910 con 0x7fc238003ef0 2023-12-20T06:04:35.841 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc24a7fc640 1 -- 172.21.15.32:0/474185885 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc2340a8d00 con 0x7fc238003ef0 2023-12-20T06:04:35.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc251596640 1 -- 172.21.15.32:0/474185885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 msgr2=0x7fc2380047d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc251596640 1 --2- 172.21.15.32:0/474185885 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 0x7fc2380047d0 secure :-1 s=READY pgs=2576 cs=0 l=1 rev1=1 crypto rx=0x7fc234001860 tx=0x7fc23401eb90 comp rx=0 tx=0).stop 2023-12-20T06:04:35.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc24b7fe640 1 -- 172.21.15.32:0/474185885 reap_dead start 2023-12-20T06:04:35.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc251596640 1 -- 172.21.15.32:0/474185885 shutdown_connections 2023-12-20T06:04:35.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc251596640 1 -- 172.21.15.32:0/474185885 >> 172.21.15.32:0/474185885 conn(0x7fc24c0cd3d0 msgr2=0x7fc24c0cd7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:35.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc251596640 1 -- 172.21.15.32:0/474185885 shutdown_connections 2023-12-20T06:04:35.842 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc251596640 1 -- 172.21.15.32:0/474185885 wait complete. 2023-12-20T06:04:35.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc251596640 1 Processor -- start 2023-12-20T06:04:35.844 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.841+0000 7fc251596640 1 -- start start 2023-12-20T06:04:35.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc251596640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 0x7fc24c159970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.845 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc251596640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc24c0b3ed0 0x7fc24c15ac20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc251596640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c0b4350 0x7fc24c15b160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc251596640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0b5150 con 0x7fc238003ef0 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc251596640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0c9050 con 0x7fc24c0b3ed0 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc251596640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fc24c0c92f0 con 0x7fc24c0b4350 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc24affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 0x7fc24c159970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc24affd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 0x7fc24c159970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55210/0 (socket says 172.21.15.32:55210) 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc24affd640 1 -- 172.21.15.32:0/4041843487 learned_addr learned my addr 172.21.15.32:0/4041843487 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc243fff640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc24c0b3ed0 0x7fc24c15ac20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc24b7fe640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c0b4350 0x7fc24c15b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc24affd640 1 -- 172.21.15.32:0/4041843487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c0b4350 msgr2=0x7fc24c15b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc24affd640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c0b4350 0x7fc24c15b160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc24affd640 1 -- 172.21.15.32:0/4041843487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc24c0b3ed0 msgr2=0x7fc24c15ac20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc24affd640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc24c0b3ed0 0x7fc24c15ac20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:35.846 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc24affd640 1 -- 172.21.15.32:0/4041843487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fc24c159eb0 con 0x7fc238003ef0 2023-12-20T06:04:35.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc243fff640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc24c0b3ed0 0x7fc24c15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:35.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc24affd640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 0x7fc24c159970 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fc23c007290 tx=0x7fc23c007750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:35.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc2437fe640 1 -- 172.21.15.32:0/4041843487 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c001660 con 0x7fc238003ef0 2023-12-20T06:04:35.847 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc2437fe640 1 -- 172.21.15.32:0/4041843487 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fc23c00f940 con 0x7fc238003ef0 2023-12-20T06:04:35.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc251596640 1 -- 172.21.15.32:0/4041843487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fc24c0b48b0 con 0x7fc238003ef0 2023-12-20T06:04:35.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc2437fe640 1 -- 172.21.15.32:0/4041843487 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fc23c00fcd0 con 0x7fc238003ef0 2023-12-20T06:04:35.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc251596640 1 -- 172.21.15.32:0/4041843487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fc24c0b4e70 con 0x7fc238003ef0 2023-12-20T06:04:35.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc251596640 1 -- 172.21.15.32:0/4041843487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fc24c04e000 con 0x7fc238003ef0 2023-12-20T06:04:35.848 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.845+0000 7fc2437fe640 1 -- 172.21.15.32:0/4041843487 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 150) v1 ==== 85066+0+0 (secure 0 0 0) 0x7fc23c01d460 con 0x7fc238003ef0 2023-12-20T06:04:35.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.849+0000 7fc2437fe640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc22c0658b0 0x7fc22c067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:35.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.849+0000 7fc2437fe640 1 -- 172.21.15.32:0/4041843487 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6453+0+0 (secure 0 0 0) 0x7fc23c083320 con 0x7fc238003ef0 2023-12-20T06:04:35.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.849+0000 7fc243fff640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc22c0658b0 0x7fc22c067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:35.849 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.849+0000 7fc243fff640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc22c0658b0 0x7fc22c067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc2340b2000 tx=0x7fc2340aa070 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:35.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:35.849+0000 7fc2437fe640 1 -- 172.21.15.32:0/4041843487 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fc23c050c60 con 0x7fc238003ef0 2023-12-20T06:04:36.044 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.041+0000 7fc251596640 1 -- 172.21.15.32:0/4041843487 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7fc24c0b68d0 con 0x7fc238003ef0 2023-12-20T06:04:36.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc2437fe640 1 -- 172.21.15.32:0/4041843487 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 128 v128) v1 ==== 95+0+4722 (secure 0 0 0) 0x7fc23c054280 con 0x7fc238003ef0 2023-12-20T06:04:36.045 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:36.045 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":128,"default_fscid":16,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3779641739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3779641739},{"type":"v1","addr":"172.21.15.39:6834","nonce":3779641739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/473427978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":473427978},{"type":"v1","addr":"172.21.15.39:6835","nonce":473427978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12060,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3927543497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3927543497},{"type":"v1","addr":"172.21.15.32:6836","nonce":3927543497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123}],"filesystems":[{"mdsmap":{"epoch":128,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2023-12-20T06:04:14.992681+0000","modified":"2023-12-20T06:04:18.007320+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"max_mds":1,"in":[0],"up":{"mds_0":12063},"failed":[],"damaged":[],"stopped":[],"info":{"gid_12063":{"gid":12063,"name":"a","rank":0,"incarnation":124,"state":"up:active","state_seq":3,"addr":"172.21.15.32:6837/3133738681","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3133738681},{"type":"v1","addr":"172.21.15.32:6837","nonce":3133738681}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}}}},"data_pools":[33],"metadata_pool":32,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0},"id":16}]} 2023-12-20T06:04:36.046 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 128 2023-12-20T06:04:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 -- 172.21.15.32:0/4041843487 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc22c0658b0 msgr2=0x7fc22c067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc22c0658b0 0x7fc22c067d70 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7fc2340b2000 tx=0x7fc2340aa070 comp rx=0 tx=0).stop 2023-12-20T06:04:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 -- 172.21.15.32:0/4041843487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 msgr2=0x7fc24c159970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 0x7fc24c159970 secure :-1 s=READY pgs=2577 cs=0 l=1 rev1=1 crypto rx=0x7fc23c007290 tx=0x7fc23c007750 comp rx=0 tx=0).stop 2023-12-20T06:04:36.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 -- 172.21.15.32:0/4041843487 shutdown_connections 2023-12-20T06:04:36.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fc22c0658b0 0x7fc22c067d70 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fc24c0b4350 0x7fc24c15b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fc24c0b3ed0 0x7fc24c15ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 --2- 172.21.15.32:0/4041843487 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fc238003ef0 0x7fc24c159970 unknown :-1 s=CLOSED pgs=2577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 -- 172.21.15.32:0/4041843487 >> 172.21.15.32:0/4041843487 conn(0x7fc24c0cd3d0 msgr2=0x7fc24c0b02b0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:36.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 -- 172.21.15.32:0/4041843487 shutdown_connections 2023-12-20T06:04:36.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.045+0000 7fc251596640 1 -- 172.21.15.32:0/4041843487 wait complete. 2023-12-20T06:04:36.066 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:04:36.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.209+0000 7f2b93d6d640 1 Processor -- start 2023-12-20T06:04:36.211 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.209+0000 7f2b93d6d640 1 -- start start 2023-12-20T06:04:36.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.209+0000 7f2b93d6d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f2b8c0af5e0 0x7f2b8c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.212 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.209+0000 7f2b93d6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8c15b990 con 0x7f2b8c15f110 2023-12-20T06:04:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.209+0000 7f2b93d6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f2b8c15bbd0 con 0x7f2b8c06ce60 2023-12-20T06:04:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.209+0000 7f2b93d6d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2b8c15beb0 con 0x7f2b8c0af5e0 2023-12-20T06:04:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.209+0000 7f2b9230a640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f2b8c15f110 0x7f2b8c15c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41142/0 (socket says 172.21.15.32:41142) 2023-12-20T06:04:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.209+0000 7f2b9230a640 1 -- 172.21.15.32:0/2062420458 learned_addr learned my addr 172.21.15.32:0/2062420458 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.209+0000 7f2b91b09640 1 --2- 172.21.15.32:0/2062420458 >> v2:172.21.15.39:3300/0 conn(0x7f2b8c0af5e0 0x7f2b8c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.209+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 170647101 0 0) 0x7f2b840014a0 con 0x7f2b8c15f110 2023-12-20T06:04:36.213 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.209+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2b78003530 con 0x7f2b8c15f110 2023-12-20T06:04:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 2443652932 0 0) 0x7f2b78003530 con 0x7f2b8c15f110 2023-12-20T06:04:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 >> v2:172.21.15.39:3300/0 conn(0x7f2b8c0af5e0 msgr2=0x7f2b8c0af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 --2- 172.21.15.32:0/2062420458 >> v2:172.21.15.39:3300/0 conn(0x7f2b8c0af5e0 0x7f2b8c0af9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.215 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 >> v1:172.21.15.39:6789/0 conn(0x7f2b8c06ce60 legacy=0x7f2b8c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 --> v1:172.21.15.32:6789/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c15e910 con 0x7f2b8c15f110 2023-12-20T06:04:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 <== mon.0 v1:172.21.15.32:6789/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (unknown 3787085578 0 0) 0x7f2b84001b90 con 0x7f2b8c15f110 2023-12-20T06:04:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b91b09640 1 --2- 172.21.15.32:0/2062420458 >> v2:172.21.15.39:3300/0 conn(0x7f2b8c0af5e0 0x7f2b8c0af9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 >> v1:172.21.15.32:6789/0 conn(0x7f2b8c15f110 legacy=0x7f2b8c15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 --2- 172.21.15.32:0/2062420458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 0x7f2b78003d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 --2- 172.21.15.32:0/2062420458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b78004380 0x7f2b78004850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 --2- 172.21.15.32:0/2062420458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b78005050 0x7f2b7800b990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c15beb0 con 0x7f2b78003980 2023-12-20T06:04:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7800bfb0 con 0x7f2b78005050 2023-12-20T06:04:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b7800c170 con 0x7f2b78004380 2023-12-20T06:04:36.216 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b91b09640 1 --2- 172.21.15.32:0/2062420458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 0x7f2b78003d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b91308640 1 --2- 172.21.15.32:0/2062420458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b78004380 0x7f2b78004850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b9230a640 1 --2- 172.21.15.32:0/2062420458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b78005050 0x7f2b7800b990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b91b09640 1 -- 172.21.15.32:0/2062420458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b78004380 msgr2=0x7f2b78004850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b91b09640 1 --2- 172.21.15.32:0/2062420458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b78004380 0x7f2b78004850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b91b09640 1 -- 172.21.15.32:0/2062420458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b78005050 msgr2=0x7f2b7800b990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b91b09640 1 --2- 172.21.15.32:0/2062420458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b78005050 0x7f2b7800b990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b91b09640 1 -- 172.21.15.32:0/2062420458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b78003530 con 0x7f2b78003980 2023-12-20T06:04:36.217 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b9230a640 1 -- 172.21.15.32:0/2062420458 reap_dead start 2023-12-20T06:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.213+0000 7f2b91b09640 1 --2- 172.21.15.32:0/2062420458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 0x7f2b78003d60 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009c90 tx=0x7f2b8801cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ddb6724e7352691 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b880a5890 con 0x7f2b78003980 2023-12-20T06:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b880a59f0 con 0x7f2b78003980 2023-12-20T06:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b90b07640 1 -- 172.21.15.32:0/2062420458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b880a5e20 con 0x7f2b78003980 2023-12-20T06:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 -- 172.21.15.32:0/2062420458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 msgr2=0x7f2b78003d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 --2- 172.21.15.32:0/2062420458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 0x7f2b78003d60 secure :-1 s=READY pgs=2579 cs=0 l=1 rev1=1 crypto rx=0x7f2b88009c90 tx=0x7f2b8801cbd0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 -- 172.21.15.32:0/2062420458 shutdown_connections 2023-12-20T06:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 --2- 172.21.15.32:0/2062420458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 0x7f2b78003d60 unknown :-1 s=CLOSED pgs=2579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.219 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 -- 172.21.15.32:0/2062420458 >> 172.21.15.32:0/2062420458 conn(0x7f2b8c06a320 msgr2=0x7f2b8c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 -- 172.21.15.32:0/2062420458 shutdown_connections 2023-12-20T06:04:36.220 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 -- 172.21.15.32:0/2062420458 wait complete. 2023-12-20T06:04:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 Processor -- start 2023-12-20T06:04:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 -- start start 2023-12-20T06:04:36.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 0x7f2b8c150f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b8c0c1360 0x7f2b8c0cc660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b8c0c5830 0x7f2b8c0ccba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c15c310 con 0x7f2b78003980 2023-12-20T06:04:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c0b39c0 con 0x7f2b8c0c5830 2023-12-20T06:04:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b93d6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f2b8c15c5d0 con 0x7f2b8c0c1360 2023-12-20T06:04:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.217+0000 7f2b91b09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 0x7f2b8c150f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b91b09640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 0x7f2b8c150f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55228/0 (socket says 172.21.15.32:55228) 2023-12-20T06:04:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b91b09640 1 -- 172.21.15.32:0/338244458 learned_addr learned my addr 172.21.15.32:0/338244458 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b91308640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b8c0c1360 0x7f2b8c0cc660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b9230a640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b8c0c5830 0x7f2b8c0ccba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b91b09640 1 -- 172.21.15.32:0/338244458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b8c0c1360 msgr2=0x7f2b8c0cc660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b91b09640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b8c0c1360 0x7f2b8c0cc660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b91b09640 1 -- 172.21.15.32:0/338244458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b8c0c5830 msgr2=0x7f2b8c0ccba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b91b09640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b8c0c5830 0x7f2b8c0ccba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b91b09640 1 -- 172.21.15.32:0/338244458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2b8c15c8b0 con 0x7f2b78003980 2023-12-20T06:04:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b91b09640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 0x7f2b8c150f30 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f2b8801ca00 tx=0x7f2b78005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b827fc640 1 -- 172.21.15.32:0/338244458 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b880ad850 con 0x7f2b78003980 2023-12-20T06:04:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b827fc640 1 -- 172.21.15.32:0/338244458 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2b880ad9b0 con 0x7f2b78003980 2023-12-20T06:04:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b827fc640 1 -- 172.21.15.32:0/338244458 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2b880add60 con 0x7f2b78003980 2023-12-20T06:04:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b93d6d640 1 -- 172.21.15.32:0/338244458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f2b8c15cb90 con 0x7f2b78003980 2023-12-20T06:04:36.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b93d6d640 1 -- 172.21.15.32:0/338244458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f2b8c15d0b0 con 0x7f2b78003980 2023-12-20T06:04:36.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b93d6d640 1 -- 172.21.15.32:0/338244458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f2b8c15d2a0 con 0x7f2b78003980 2023-12-20T06:04:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b827fc640 1 -- 172.21.15.32:0/338244458 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 150) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f2b880b4040 con 0x7f2b78003980 2023-12-20T06:04:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.221+0000 7f2b827fc640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b700658e0 0x7f2b70067da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.225+0000 7f2b827fc640 1 -- 172.21.15.32:0/338244458 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f2b8811d240 con 0x7f2b78003980 2023-12-20T06:04:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.225+0000 7f2b91308640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b700658e0 0x7f2b70067da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.226 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.225+0000 7f2b91308640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b700658e0 0x7f2b70067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0b3220 tx=0x7f2b7c009040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:36.228 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.225+0000 7f2b827fc640 1 -- 172.21.15.32:0/338244458 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f2b880eaaf0 con 0x7f2b78003980 2023-12-20T06:04:36.386 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.385+0000 7f2b93d6d640 1 -- 172.21.15.32:0/338244458 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f2b8c04e000 con 0x7f2b78003980 2023-12-20T06:04:36.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.385+0000 7f2b827fc640 1 -- 172.21.15.32:0/338244458 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v253) v1 ==== 74+0+17082 (secure 0 0 0) 0x7f2b880b2090 con 0x7f2b78003980 2023-12-20T06:04:36.388 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:36.388 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":253,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:04:34.642501+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":32,"pool_name":"cephfs_metadata","create_time":"2023-12-20T06:04:12.651034+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"250","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":33,"pool_name":"cephfs_data","create_time":"2023-12-20T06:04:13.997753+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"251","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_acting":1.75,"score_stable":1.75,"optimal_score":1,"raw_score_acting":1.75,"raw_score_stable":1.75,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:0/1848378616":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/3742412743":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/1548603036":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/2131291246":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/2118335854":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/1269951230":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2916844765":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2625213384":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/375930125":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/2535253896":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/766391941":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/3950316648":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2606010957":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/3612789559":"2023-12-21T06:04:27.557099+0000","172.21.15.32:6833/64546":"2023-12-21T06:04:27.557099+0000","172.21.15.32:6832/64546":"2023-12-21T06:04:27.557099+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:04:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 -- 172.21.15.32:0/338244458 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b700658e0 msgr2=0x7f2b70067da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b700658e0 0x7f2b70067da0 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f2b8c0b3220 tx=0x7f2b7c009040 comp rx=0 tx=0).stop 2023-12-20T06:04:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 -- 172.21.15.32:0/338244458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 msgr2=0x7f2b8c150f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 0x7f2b8c150f30 secure :-1 s=READY pgs=2580 cs=0 l=1 rev1=1 crypto rx=0x7f2b8801ca00 tx=0x7f2b78005800 comp rx=0 tx=0).stop 2023-12-20T06:04:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 -- 172.21.15.32:0/338244458 shutdown_connections 2023-12-20T06:04:36.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2b700658e0 0x7f2b70067da0 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f2b8c0c5830 0x7f2b8c0ccba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f2b8c0c1360 0x7f2b8c0cc660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 --2- 172.21.15.32:0/338244458 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f2b78003980 0x7f2b8c150f30 unknown :-1 s=CLOSED pgs=2580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 -- 172.21.15.32:0/338244458 >> 172.21.15.32:0/338244458 conn(0x7f2b8c06a320 msgr2=0x7f2b8c0ce950 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 -- 172.21.15.32:0/338244458 shutdown_connections 2023-12-20T06:04:36.391 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.389+0000 7f2b93d6d640 1 -- 172.21.15.32:0/338244458 wait complete. 2023-12-20T06:04:36.409 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2023-12-20T06:04:36.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.553+0000 7f80441ce640 1 Processor -- start 2023-12-20T06:04:36.555 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.553+0000 7f80441ce640 1 -- start start 2023-12-20T06:04:36.556 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.553+0000 7f80441ce640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f803c0af5e0 0x7f803c0af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.553+0000 7f80441ce640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c15b990 con 0x7f803c15f110 2023-12-20T06:04:36.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.553+0000 7f80441ce640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f803c15bbd0 con 0x7f803c06ce60 2023-12-20T06:04:36.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.553+0000 7f80441ce640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f803c15beb0 con 0x7f803c0af5e0 2023-12-20T06:04:36.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.553+0000 7f804276b640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f803c15f110 0x7f803c15c440 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41156/0 (socket says 172.21.15.32:41156) 2023-12-20T06:04:36.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.553+0000 7f804276b640 1 -- 172.21.15.32:0/1388928155 learned_addr learned my addr 172.21.15.32:0/1388928155 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:36.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.553+0000 7f8041f6a640 1 --2- 172.21.15.32:0/1388928155 >> v2:172.21.15.39:3300/0 conn(0x7f803c0af5e0 0x7f803c0af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.553+0000 7f8040f68640 1 -- 172.21.15.32:0/1388928155 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 3518704074 0 0) 0x7f80340014a0 con 0x7f803c15f110 2023-12-20T06:04:36.557 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.553+0000 7f8040f68640 1 -- 172.21.15.32:0/1388928155 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f8028003530 con 0x7f803c15f110 2023-12-20T06:04:36.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.557+0000 7f8040f68640 1 -- 172.21.15.32:0/1388928155 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 626492570 0 0) 0x7f8028003530 con 0x7f803c15f110 2023-12-20T06:04:36.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.557+0000 7f8041f6a640 1 -- 172.21.15.32:0/1388928155 >> v1:172.21.15.39:6789/0 conn(0x7f803c06ce60 legacy=0x7f803c15b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.557+0000 7f8041f6a640 1 -- 172.21.15.32:0/1388928155 >> v1:172.21.15.32:6789/0 conn(0x7f803c15f110 legacy=0x7f803c15c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.558 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.557+0000 7f8041f6a640 1 -- 172.21.15.32:0/1388928155 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f803c15e910 con 0x7f803c0af5e0 2023-12-20T06:04:36.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.557+0000 7f8041f6a640 1 --2- 172.21.15.32:0/1388928155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c0af5e0 0x7f803c0af9c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f80380097a0 tx=0x7f803801c350 comp rx=0 tx=0).ready entity=mon.1 client_cookie=3eb9879112016833 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:36.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.557+0000 7f8040f68640 1 -- 172.21.15.32:0/1388928155 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8034001b90 con 0x7f803c0af5e0 2023-12-20T06:04:36.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.557+0000 7f8040f68640 1 -- 172.21.15.32:0/1388928155 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f80380a7050 con 0x7f803c0af5e0 2023-12-20T06:04:36.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.557+0000 7f8040f68640 1 -- 172.21.15.32:0/1388928155 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380ab620 con 0x7f803c0af5e0 2023-12-20T06:04:36.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.557+0000 7f80441ce640 1 -- 172.21.15.32:0/1388928155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c0af5e0 msgr2=0x7f803c0af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.557+0000 7f80441ce640 1 --2- 172.21.15.32:0/1388928155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c0af5e0 0x7f803c0af9c0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7f80380097a0 tx=0x7f803801c350 comp rx=0 tx=0).stop 2023-12-20T06:04:36.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 -- 172.21.15.32:0/1388928155 shutdown_connections 2023-12-20T06:04:36.561 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 --2- 172.21.15.32:0/1388928155 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c0af5e0 0x7f803c0af9c0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 -- 172.21.15.32:0/1388928155 >> 172.21.15.32:0/1388928155 conn(0x7f803c06a320 msgr2=0x7f803c0ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:36.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 -- 172.21.15.32:0/1388928155 shutdown_connections 2023-12-20T06:04:36.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 -- 172.21.15.32:0/1388928155 wait complete. 2023-12-20T06:04:36.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 Processor -- start 2023-12-20T06:04:36.562 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 -- start start 2023-12-20T06:04:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c06ce60 0x7f803c145830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c0af5e0 0x7f803c0b2580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f803c15f110 0x7f803c0b2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c1473a0 con 0x7f803c06ce60 2023-12-20T06:04:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f803c156490 con 0x7f803c0af5e0 2023-12-20T06:04:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f80441ce640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f803c0b30c0 con 0x7f803c15f110 2023-12-20T06:04:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f8041f6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c06ce60 0x7f803c145830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f8041f6a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c06ce60 0x7f803c145830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55232/0 (socket says 172.21.15.32:55232) 2023-12-20T06:04:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f8041f6a640 1 -- 172.21.15.32:0/3812407962 learned_addr learned my addr 172.21.15.32:0/3812407962 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:36.566 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f804276b640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f803c15f110 0x7f803c0b2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f8041769640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c0af5e0 0x7f803c0b2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f8041f6a640 1 -- 172.21.15.32:0/3812407962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f803c15f110 msgr2=0x7f803c0b2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f8041f6a640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f803c15f110 0x7f803c0b2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f8041f6a640 1 -- 172.21.15.32:0/3812407962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c0af5e0 msgr2=0x7f803c0b2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f8041f6a640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c0af5e0 0x7f803c0b2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f8041f6a640 1 -- 172.21.15.32:0/3812407962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f80380093b0 con 0x7f803c06ce60 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.561+0000 7f8041f6a640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c06ce60 0x7f803c145830 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f80380ae040 tx=0x7f803801ab90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.565+0000 7f80327fc640 1 -- 172.21.15.32:0/3812407962 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f80380059a0 con 0x7f803c06ce60 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.565+0000 7f80327fc640 1 -- 172.21.15.32:0/3812407962 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f8038005b00 con 0x7f803c06ce60 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.565+0000 7f80327fc640 1 -- 172.21.15.32:0/3812407962 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f8038005e70 con 0x7f803c06ce60 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.565+0000 7f80441ce640 1 -- 172.21.15.32:0/3812407962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f803c156750 con 0x7f803c06ce60 2023-12-20T06:04:36.567 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.565+0000 7f80441ce640 1 -- 172.21.15.32:0/3812407962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f803c156d10 con 0x7f803c06ce60 2023-12-20T06:04:36.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.565+0000 7f80441ce640 1 -- 172.21.15.32:0/3812407962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f803c04e000 con 0x7f803c06ce60 2023-12-20T06:04:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.565+0000 7f80327fc640 1 -- 172.21.15.32:0/3812407962 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 150) v1 ==== 85066+0+0 (secure 0 0 0) 0x7f8038006020 con 0x7f803c06ce60 2023-12-20T06:04:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.565+0000 7f80327fc640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f80200658b0 0x7f8020067d70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.565+0000 7f80327fc640 1 -- 172.21.15.32:0/3812407962 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(253..253 src has 1..253) v4 ==== 6453+0+0 (secure 0 0 0) 0x7f803811c9f0 con 0x7f803c06ce60 2023-12-20T06:04:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.565+0000 7f8041769640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f80200658b0 0x7f8020067d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.569+0000 7f8041769640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f80200658b0 0x7f8020067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f803c15e7a0 tx=0x7f802c009040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:36.572 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.569+0000 7f80327fc640 1 -- 172.21.15.32:0/3812407962 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f80380ea330 con 0x7f803c06ce60 2023-12-20T06:04:36.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.669+0000 7f80327fc640 1 -- 172.21.15.32:0/3812407962 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 151) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f803811ed50 con 0x7f803c06ce60 2023-12-20T06:04:36.772 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.769+0000 7f80441ce640 1 -- 172.21.15.32:0/3812407962 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) v1 -- 0x7f803c0cf450 con 0x7f803c06ce60 2023-12-20T06:04:36.773 INFO:tasks.ceph.mon.a.smithi032.stderr:2023-12-20T06:04:36.773+0000 7f68ef39b640 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2023-12-20T06:04:36.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.789+0000 7f80327fc640 1 -- 172.21.15.32:0/3812407962 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v129) v1 ==== 161+0+0 (secure 0 0 0) 0x7f8038103540 con 0x7f803c06ce60 2023-12-20T06:04:36.793 INFO:teuthology.orchestra.run.smithi032.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2023-12-20T06:04:36.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 -- 172.21.15.32:0/3812407962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f80200658b0 msgr2=0x7f8020067d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f80200658b0 0x7f8020067d70 secure :-1 s=READY pgs=24 cs=0 l=1 rev1=1 crypto rx=0x7f803c15e7a0 tx=0x7f802c009040 comp rx=0 tx=0).stop 2023-12-20T06:04:36.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 -- 172.21.15.32:0/3812407962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c06ce60 msgr2=0x7f803c145830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.795 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c06ce60 0x7f803c145830 secure :-1 s=READY pgs=2582 cs=0 l=1 rev1=1 crypto rx=0x7f80380ae040 tx=0x7f803801ab90 comp rx=0 tx=0).stop 2023-12-20T06:04:36.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 -- 172.21.15.32:0/3812407962 shutdown_connections 2023-12-20T06:04:36.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f80200658b0 0x7f8020067d70 unknown :-1 s=CLOSED pgs=24 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f803c15f110 0x7f803c0b2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f803c0af5e0 0x7f803c0b2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 --2- 172.21.15.32:0/3812407962 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f803c06ce60 0x7f803c145830 unknown :-1 s=CLOSED pgs=2582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 -- 172.21.15.32:0/3812407962 >> 172.21.15.32:0/3812407962 conn(0x7f803c06a320 msgr2=0x7f803c159540 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:36.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 -- 172.21.15.32:0/3812407962 shutdown_connections 2023-12-20T06:04:36.796 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.793+0000 7f80441ce640 1 -- 172.21.15.32:0/3812407962 wait complete. 2023-12-20T06:04:36.801 INFO:tasks.ceph.mds.a.smithi032.stderr: -3879> 2023-12-20T06:04:20.557+0000 7f007ed58640 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2023-12-20T06:04:36.816 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2023-12-20T06:04:36.849 INFO:tasks.ceph.mds.a.smithi032.stdout:starting mds.a at 2023-12-20T06:04:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.961+0000 7f1ea7004640 1 Processor -- start 2023-12-20T06:04:36.964 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.961+0000 7f1ea7004640 1 -- start start 2023-12-20T06:04:36.966 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea7004640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1ea015a920 0x7f1ea00ca690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea7004640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f1ea00b26d0 0x7f1ea00b04a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea7004640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1ea00c9ff0 con 0x7f1ea00b0010 2023-12-20T06:04:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea7004640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea00b4510 con 0x7f1ea00b26d0 2023-12-20T06:04:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea7004640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1ea00b09e0 con 0x7f1ea015a920 2023-12-20T06:04:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea5801640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f1ea00b0010 0x7f1ea00cafa0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:41160/0 (socket says 172.21.15.32:41160) 2023-12-20T06:04:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea5801640 1 -- 172.21.15.32:0/3632843593 learned_addr learned my addr 172.21.15.32:0/3632843593 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea6803640 1 --2- 172.21.15.32:0/3632843593 >> v2:172.21.15.32:3300/0 conn(0x7f1ea00b26d0 0x7f1ea00b04a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea6002640 1 --2- 172.21.15.32:0/3632843593 >> v2:172.21.15.39:3300/0 conn(0x7f1ea015a920 0x7f1ea00ca690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.967 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea6803640 1 -- 172.21.15.32:0/3632843593 >> v2:172.21.15.39:3300/0 conn(0x7f1ea015a920 msgr2=0x7f1ea00ca690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:36.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea6803640 1 --2- 172.21.15.32:0/3632843593 >> v2:172.21.15.39:3300/0 conn(0x7f1ea015a920 0x7f1ea00ca690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea6803640 1 -- 172.21.15.32:0/3632843593 >> v1:172.21.15.32:6789/0 conn(0x7f1ea00b0010 legacy=0x7f1ea00cafa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.968 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.965+0000 7f1ea6803640 1 -- 172.21.15.32:0/3632843593 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1ea00c93e0 con 0x7f1ea00b26d0 2023-12-20T06:04:36.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea6803640 1 --2- 172.21.15.32:0/3632843593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea00b26d0 0x7f1ea00b04a0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f1ea006fcb0 tx=0x7f1e9801c8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebb2cba01fe58027 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:36.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea5000640 1 -- 172.21.15.32:0/3632843593 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980a58c0 con 0x7f1ea00b26d0 2023-12-20T06:04:36.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea5000640 1 -- 172.21.15.32:0/3632843593 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e980a5a20 con 0x7f1ea00b26d0 2023-12-20T06:04:36.970 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea5000640 1 -- 172.21.15.32:0/3632843593 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980a5db0 con 0x7f1ea00b26d0 2023-12-20T06:04:36.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea7004640 1 -- 172.21.15.32:0/3632843593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea00b26d0 msgr2=0x7f1ea00b04a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea7004640 1 --2- 172.21.15.32:0/3632843593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea00b26d0 0x7f1ea00b04a0 secure :-1 s=READY pgs=2588 cs=0 l=1 rev1=1 crypto rx=0x7f1ea006fcb0 tx=0x7f1e9801c8e0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea7004640 1 -- 172.21.15.32:0/3632843593 shutdown_connections 2023-12-20T06:04:36.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea7004640 1 --2- 172.21.15.32:0/3632843593 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea00b26d0 0x7f1ea00b04a0 unknown :-1 s=CLOSED pgs=2588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.971 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea7004640 1 --2- 172.21.15.32:0/3632843593 >> v2:172.21.15.39:3300/0 conn(0x7f1ea015a920 0x7f1ea00ca690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea7004640 1 -- 172.21.15.32:0/3632843593 >> 172.21.15.32:0/3632843593 conn(0x7f1ea006c060 msgr2=0x7f1ea006c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:36.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea7004640 1 -- 172.21.15.32:0/3632843593 shutdown_connections 2023-12-20T06:04:36.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea7004640 1 -- 172.21.15.32:0/3632843593 wait complete. 2023-12-20T06:04:36.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea7004640 1 Processor -- start 2023-12-20T06:04:36.972 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.969+0000 7f1ea7004640 1 -- start start 2023-12-20T06:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea7004640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1ea00b0010 0x7f1ea0158e00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea7004640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ea00b26d0 0x7f1ea00cd220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea7004640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea015a920 0x7f1ea00cd760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea7004640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00cdca0 con 0x7f1ea015a920 2023-12-20T06:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea7004640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00cdf50 con 0x7f1ea00b26d0 2023-12-20T06:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea7004640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1ea00c3ad0 con 0x7f1ea00b0010 2023-12-20T06:04:36.975 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea6803640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea015a920 0x7f1ea00cd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea6803640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea015a920 0x7f1ea00cd760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55270/0 (socket says 172.21.15.32:55270) 2023-12-20T06:04:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea6803640 1 -- 172.21.15.32:0/2804852431 learned_addr learned my addr 172.21.15.32:0/2804852431 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea6002640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1ea00b0010 0x7f1ea0158e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea5801640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ea00b26d0 0x7f1ea00cd220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea6803640 1 -- 172.21.15.32:0/2804852431 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1ea00b0010 msgr2=0x7f1ea0158e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea6803640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1ea00b0010 0x7f1ea0158e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea6803640 1 -- 172.21.15.32:0/2804852431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ea00b26d0 msgr2=0x7f1ea00cd220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea6803640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ea00b26d0 0x7f1ea00cd220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea6803640 1 -- 172.21.15.32:0/2804852431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1e98009a70 con 0x7f1ea015a920 2023-12-20T06:04:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea5801640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ea00b26d0 0x7f1ea00cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:36.976 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea6803640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea015a920 0x7f1ea00cd760 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f1e980194f0 tx=0x7f1e98019680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1e967fc640 1 -- 172.21.15.32:0/2804852431 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980b9860 con 0x7f1ea015a920 2023-12-20T06:04:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1e967fc640 1 -- 172.21.15.32:0/2804852431 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1e980b99c0 con 0x7f1ea015a920 2023-12-20T06:04:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1e967fc640 1 -- 172.21.15.32:0/2804852431 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1e980b9d70 con 0x7f1ea015a920 2023-12-20T06:04:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea7004640 1 -- 172.21.15.32:0/2804852431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1ea00c3db0 con 0x7f1ea015a920 2023-12-20T06:04:36.977 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea7004640 1 -- 172.21.15.32:0/2804852431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1ea00c4370 con 0x7f1ea015a920 2023-12-20T06:04:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.973+0000 7f1ea7004640 1 -- 172.21.15.32:0/2804852431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1ea0074100 con 0x7f1ea015a920 2023-12-20T06:04:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.977+0000 7f1e967fc640 1 -- 172.21.15.32:0/2804852431 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 151) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f1e980ad440 con 0x7f1ea015a920 2023-12-20T06:04:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.977+0000 7f1e967fc640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e84065af0 0x7f1e84067fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.977+0000 7f1e967fc640 1 -- 172.21.15.32:0/2804852431 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6539+0+0 (secure 0 0 0) 0x7f1e9811e890 con 0x7f1ea015a920 2023-12-20T06:04:36.978 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.977+0000 7f1ea6002640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e84065af0 0x7f1e84067fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:36.979 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.977+0000 7f1ea6002640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e84065af0 0x7f1e84067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009ac0 tx=0x7f1e9c009190 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:36.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:36.981+0000 7f1e967fc640 1 -- 172.21.15.32:0/2804852431 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1e980ebfd0 con 0x7f1ea015a920 2023-12-20T06:04:37.173 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.173+0000 7f1ea7004640 1 -- 172.21.15.32:0/2804852431 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) v1 -- 0x7f1ea00abd60 con 0x7f1ea015a920 2023-12-20T06:04:37.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.793+0000 7f1e967fc640 1 -- 172.21.15.32:0/2804852431 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v130) v1 ==== 104+0+0 (secure 0 0 0) 0x7f1e980ef5f0 con 0x7f1ea015a920 2023-12-20T06:04:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 -- 172.21.15.32:0/2804852431 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e84065af0 msgr2=0x7f1e84067fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e84065af0 0x7f1e84067fb0 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f1e9c009ac0 tx=0x7f1e9c009190 comp rx=0 tx=0).stop 2023-12-20T06:04:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 -- 172.21.15.32:0/2804852431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea015a920 msgr2=0x7f1ea00cd760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea015a920 0x7f1ea00cd760 secure :-1 s=READY pgs=2589 cs=0 l=1 rev1=1 crypto rx=0x7f1e980194f0 tx=0x7f1e98019680 comp rx=0 tx=0).stop 2023-12-20T06:04:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 -- 172.21.15.32:0/2804852431 shutdown_connections 2023-12-20T06:04:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1e84065af0 0x7f1e84067fb0 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1ea015a920 0x7f1ea00cd760 unknown :-1 s=CLOSED pgs=2589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1ea00b26d0 0x7f1ea00cd220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:37.798 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 --2- 172.21.15.32:0/2804852431 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1ea00b0010 0x7f1ea0158e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:37.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 -- 172.21.15.32:0/2804852431 >> 172.21.15.32:0/2804852431 conn(0x7f1ea006c060 msgr2=0x7f1ea0155320 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:37.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 -- 172.21.15.32:0/2804852431 shutdown_connections 2023-12-20T06:04:37.799 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:37.797+0000 7f1ea7004640 1 -- 172.21.15.32:0/2804852431 wait complete. 2023-12-20T06:04:37.822 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2023-12-20T06:04:38.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b18e04640 1 Processor -- start 2023-12-20T06:04:38.021 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b18e04640 1 -- start start 2023-12-20T06:04:38.023 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b18e04640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6b14159770 0x7f6b140ca790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b18e04640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b140cc130 0x7f6b14159b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b18e04640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6b140ce070 con 0x7f6b140affe0 2023-12-20T06:04:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b18e04640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b140cacd0 con 0x7f6b140cc130 2023-12-20T06:04:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b18e04640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6b1415a480 con 0x7f6b14159770 2023-12-20T06:04:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b12ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b140cc130 0x7f6b14159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b12ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6b140cc130 0x7f6b14159b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55280/0 (socket says 172.21.15.32:55280) 2023-12-20T06:04:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b12ffd640 1 -- 172.21.15.32:0/3319616895 learned_addr learned my addr 172.21.15.32:0/3319616895 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b137fe640 1 --2- 172.21.15.32:0/3319616895 >> v2:172.21.15.39:3300/0 conn(0x7f6b14159770 0x7f6b140ca790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b12ffd640 1 -- 172.21.15.32:0/3319616895 >> v2:172.21.15.39:3300/0 conn(0x7f6b14159770 msgr2=0x7f6b140ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:38.024 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b12ffd640 1 --2- 172.21.15.32:0/3319616895 >> v2:172.21.15.39:3300/0 conn(0x7f6b14159770 0x7f6b140ca790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b12ffd640 1 -- 172.21.15.32:0/3319616895 >> v1:172.21.15.39:6789/0 conn(0x7f6b140affe0 legacy=0x7f6b140b03f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:38.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.021+0000 7f6b12ffd640 1 -- 172.21.15.32:0/3319616895 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b1415a760 con 0x7f6b140cc130 2023-12-20T06:04:38.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b12ffd640 1 --2- 172.21.15.32:0/3319616895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b140cc130 0x7f6b14159b50 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f6b1406e460 tx=0x7f6b0801c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef5edfe8be047a60 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:38.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b127fc640 1 -- 172.21.15.32:0/3319616895 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080aa430 con 0x7f6b140cc130 2023-12-20T06:04:38.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b127fc640 1 -- 172.21.15.32:0/3319616895 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b080aa590 con 0x7f6b140cc130 2023-12-20T06:04:38.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b127fc640 1 -- 172.21.15.32:0/3319616895 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b080aa920 con 0x7f6b140cc130 2023-12-20T06:04:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b18e04640 1 -- 172.21.15.32:0/3319616895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b140cc130 msgr2=0x7f6b14159b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b18e04640 1 --2- 172.21.15.32:0/3319616895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b140cc130 0x7f6b14159b50 secure :-1 s=READY pgs=2590 cs=0 l=1 rev1=1 crypto rx=0x7f6b1406e460 tx=0x7f6b0801c4d0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b18e04640 1 -- 172.21.15.32:0/3319616895 shutdown_connections 2023-12-20T06:04:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b18e04640 1 --2- 172.21.15.32:0/3319616895 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b140cc130 0x7f6b14159b50 unknown :-1 s=CLOSED pgs=2590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b18e04640 1 --2- 172.21.15.32:0/3319616895 >> v2:172.21.15.39:3300/0 conn(0x7f6b14159770 0x7f6b140ca790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b18e04640 1 -- 172.21.15.32:0/3319616895 >> 172.21.15.32:0/3319616895 conn(0x7f6b1406a4e0 msgr2=0x7f6b1406bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:38.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b18e04640 1 -- 172.21.15.32:0/3319616895 shutdown_connections 2023-12-20T06:04:38.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.025+0000 7f6b18e04640 1 -- 172.21.15.32:0/3319616895 wait complete. 2023-12-20T06:04:38.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b18e04640 1 Processor -- start 2023-12-20T06:04:38.029 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b18e04640 1 -- start start 2023-12-20T06:04:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b18e04640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b140affe0 0x7f6b140c2050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:38.032 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b18e04640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b140cc130 0x7f6b140b21f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b18e04640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b14159770 0x7f6b141f8b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b18e04640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b14158aa0 con 0x7f6b140affe0 2023-12-20T06:04:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b18e04640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6b140b2730 con 0x7f6b14159770 2023-12-20T06:04:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b18e04640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6b14167470 con 0x7f6b140cc130 2023-12-20T06:04:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b137fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b140affe0 0x7f6b140c2050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b137fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b140affe0 0x7f6b140c2050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:55284/0 (socket says 172.21.15.32:55284) 2023-12-20T06:04:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b137fe640 1 -- 172.21.15.32:0/3051285718 learned_addr learned my addr 172.21.15.32:0/3051285718 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b12ffd640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b140cc130 0x7f6b140b21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:38.033 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b13fff640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b14159770 0x7f6b141f8b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b137fe640 1 -- 172.21.15.32:0/3051285718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b140cc130 msgr2=0x7f6b140b21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b137fe640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b140cc130 0x7f6b140b21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b137fe640 1 -- 172.21.15.32:0/3051285718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b14159770 msgr2=0x7f6b141f8b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b137fe640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b14159770 0x7f6b141f8b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b137fe640 1 -- 172.21.15.32:0/3051285718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6b080094d0 con 0x7f6b140affe0 2023-12-20T06:04:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b12ffd640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b140cc130 0x7f6b140b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b137fe640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b140affe0 0x7f6b140c2050 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400d5f0 tx=0x7f6b0400dab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6afbfff640 1 -- 172.21.15.32:0/3051285718 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b040125f0 con 0x7f6b140affe0 2023-12-20T06:04:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6afbfff640 1 -- 172.21.15.32:0/3051285718 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6b04012750 con 0x7f6b140affe0 2023-12-20T06:04:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b18e04640 1 -- 172.21.15.32:0/3051285718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6b140b2940 con 0x7f6b140affe0 2023-12-20T06:04:38.034 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6afbfff640 1 -- 172.21.15.32:0/3051285718 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6b04012ae0 con 0x7f6b140affe0 2023-12-20T06:04:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b18e04640 1 -- 172.21.15.32:0/3051285718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6b14166b40 con 0x7f6b140affe0 2023-12-20T06:04:38.035 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.029+0000 7f6b18e04640 1 -- 172.21.15.32:0/3051285718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6b14074100 con 0x7f6b140affe0 2023-12-20T06:04:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.033+0000 7f6afbfff640 1 -- 172.21.15.32:0/3051285718 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 151) v1 ==== 85274+0+0 (secure 0 0 0) 0x7f6b04012c40 con 0x7f6b140affe0 2023-12-20T06:04:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.033+0000 7f6afbfff640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b00065b20 0x7f6b00067fe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.033+0000 7f6afbfff640 1 -- 172.21.15.32:0/3051285718 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(254..254 src has 1..254) v4 ==== 6539+0+0 (secure 0 0 0) 0x7f6b040839d0 con 0x7f6b140affe0 2023-12-20T06:04:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.033+0000 7f6b12ffd640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b00065b20 0x7f6b00067fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:38.036 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.033+0000 7f6b12ffd640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b00065b20 0x7f6b00067fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6b080ae000 tx=0x7f6b080a7000 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:38.038 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.037+0000 7f6afbfff640 1 -- 172.21.15.32:0/3051285718 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6b04050f60 con 0x7f6b140affe0 2023-12-20T06:04:38.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.197+0000 7f6b18e04640 1 -- 172.21.15.32:0/3051285718 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7f6b140734f0 con 0x7f6b140affe0 2023-12-20T06:04:38.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.809+0000 7f6afbfff640 1 -- 172.21.15.32:0/3051285718 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6b04051330 con 0x7f6b140affe0 2023-12-20T06:04:38.812 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.809+0000 7f6afbfff640 1 -- 172.21.15.32:0/3051285718 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v255) v1 ==== 181+0+0 (secure 0 0 0) 0x7f6b0407f750 con 0x7f6b140affe0 2023-12-20T06:04:38.812 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_metadata' removed 2023-12-20T06:04:38.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 -- 172.21.15.32:0/3051285718 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b00065b20 msgr2=0x7f6b00067fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:38.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b00065b20 0x7f6b00067fe0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f6b080ae000 tx=0x7f6b080a7000 comp rx=0 tx=0).stop 2023-12-20T06:04:38.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 -- 172.21.15.32:0/3051285718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b140affe0 msgr2=0x7f6b140c2050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:38.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b140affe0 0x7f6b140c2050 secure :-1 s=READY pgs=2591 cs=0 l=1 rev1=1 crypto rx=0x7f6b0400d5f0 tx=0x7f6b0400dab0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 -- 172.21.15.32:0/3051285718 shutdown_connections 2023-12-20T06:04:38.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6b00065b20 0x7f6b00067fe0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6b14159770 0x7f6b141f8b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6b140cc130 0x7f6b140b21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 --2- 172.21.15.32:0/3051285718 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6b140affe0 0x7f6b140c2050 unknown :-1 s=CLOSED pgs=2591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 -- 172.21.15.32:0/3051285718 >> 172.21.15.32:0/3051285718 conn(0x7f6b1406a4e0 msgr2=0x7f6b1415b910 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:38.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 -- 172.21.15.32:0/3051285718 shutdown_connections 2023-12-20T06:04:38.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.813+0000 7f6b18e04640 1 -- 172.21.15.32:0/3051285718 wait complete. 2023-12-20T06:04:38.838 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2023-12-20T06:04:38.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.993+0000 7ff692bf0640 1 Processor -- start 2023-12-20T06:04:38.994 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.993+0000 7ff692bf0640 1 -- start start 2023-12-20T06:04:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.993+0000 7ff692bf0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff68c0b1d90 0x7ff68c0b5d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.993+0000 7ff692bf0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff68c0b62d0 0x7ff68c0d2040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:38.996 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.993+0000 7ff692bf0640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7ff68c0d1030 con 0x7ff68c1603e0 2023-12-20T06:04:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.993+0000 7ff692bf0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7ff68c0d1280 con 0x7ff68c0b1d90 2023-12-20T06:04:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.993+0000 7ff692bf0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0d1560 con 0x7ff68c0b62d0 2023-12-20T06:04:38.997 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.997+0000 7ff68bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff68c0b1d90 0x7ff68c0b5d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:38.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.997+0000 7ff68bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7ff68c0b1d90 0x7ff68c0b5d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51210/0 (socket says 172.21.15.32:51210) 2023-12-20T06:04:38.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.997+0000 7ff68bfff640 1 -- 172.21.15.32:0/3150573333 learned_addr learned my addr 172.21.15.32:0/3150573333 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:38.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.997+0000 7ff69118d640 1 --2- 172.21.15.32:0/3150573333 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff68c0b62d0 0x7ff68c0d2040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:38.998 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.997+0000 7ff69118d640 1 -- 172.21.15.32:0/3150573333 >> v2:172.21.15.32:3300/0 conn(0x7ff68c0b1d90 msgr2=0x7ff68c0b5d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:38.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.997+0000 7ff69118d640 1 --2- 172.21.15.32:0/3150573333 >> v2:172.21.15.32:3300/0 conn(0x7ff68c0b1d90 0x7ff68c0b5d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:38.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.997+0000 7ff69118d640 1 -- 172.21.15.32:0/3150573333 >> v1:172.21.15.32:6789/0 conn(0x7ff68c1603e0 legacy=0x7ff68c0b8580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:38.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.997+0000 7ff69118d640 1 -- 172.21.15.32:0/3150573333 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff68c0d1840 con 0x7ff68c0b62d0 2023-12-20T06:04:38.999 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:38.997+0000 7ff68bfff640 1 --2- 172.21.15.32:0/3150573333 >> v2:172.21.15.32:3300/0 conn(0x7ff68c0b1d90 0x7ff68c0b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:39.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff69118d640 1 --2- 172.21.15.32:0/3150573333 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff68c0b62d0 0x7ff68c0d2040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff684009d90 tx=0x7ff68401c850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=28fc7eb48dd3ab04 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:39.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff68b7fe640 1 -- 172.21.15.32:0/3150573333 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6840a57e0 con 0x7ff68c0b62d0 2023-12-20T06:04:39.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff68b7fe640 1 -- 172.21.15.32:0/3150573333 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff6840a5940 con 0x7ff68c0b62d0 2023-12-20T06:04:39.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff68b7fe640 1 -- 172.21.15.32:0/3150573333 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff6840a5cd0 con 0x7ff68c0b62d0 2023-12-20T06:04:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff692bf0640 1 -- 172.21.15.32:0/3150573333 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff68c0b62d0 msgr2=0x7ff68c0d2040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff692bf0640 1 --2- 172.21.15.32:0/3150573333 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff68c0b62d0 0x7ff68c0d2040 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7ff684009d90 tx=0x7ff68401c850 comp rx=0 tx=0).stop 2023-12-20T06:04:39.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff692bf0640 1 -- 172.21.15.32:0/3150573333 shutdown_connections 2023-12-20T06:04:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff692bf0640 1 --2- 172.21.15.32:0/3150573333 >> v2:172.21.15.32:3300/0 conn(0x7ff68c0b1d90 0x7ff68c0b5d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff692bf0640 1 --2- 172.21.15.32:0/3150573333 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff68c0b62d0 0x7ff68c0d2040 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff692bf0640 1 -- 172.21.15.32:0/3150573333 >> 172.21.15.32:0/3150573333 conn(0x7ff68c06b5e0 msgr2=0x7ff68c06b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff692bf0640 1 -- 172.21.15.32:0/3150573333 shutdown_connections 2023-12-20T06:04:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff692bf0640 1 -- 172.21.15.32:0/3150573333 wait complete. 2023-12-20T06:04:39.004 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff692bf0640 1 Processor -- start 2023-12-20T06:04:39.005 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.001+0000 7ff692bf0640 1 -- start start 2023-12-20T06:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff692bf0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff68c0b1d90 0x7ff68c0ca020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff692bf0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c0b62d0 0x7ff68c14ff50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff692bf0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c1603e0 0x7ff68c15cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:39.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff692bf0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0c6b80 con 0x7ff68c0b62d0 2023-12-20T06:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff692bf0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c1515c0 con 0x7ff68c1603e0 2023-12-20T06:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff692bf0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7ff68c0c68a0 con 0x7ff68c0b1d90 2023-12-20T06:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff68bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c0b62d0 0x7ff68c14ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff68bfff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c0b62d0 0x7ff68c14ff50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51214/0 (socket says 172.21.15.32:51214) 2023-12-20T06:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff68bfff640 1 -- 172.21.15.32:0/2343564925 learned_addr learned my addr 172.21.15.32:0/2343564925 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff69118d640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c1603e0 0x7ff68c15cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff69098c640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff68c0b1d90 0x7ff68c0ca020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff68bfff640 1 -- 172.21.15.32:0/2343564925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff68c0b1d90 msgr2=0x7ff68c0ca020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff68bfff640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff68c0b1d90 0x7ff68c0ca020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:39.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff68bfff640 1 -- 172.21.15.32:0/2343564925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c1603e0 msgr2=0x7ff68c15cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff68bfff640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c1603e0 0x7ff68c15cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff68bfff640 1 -- 172.21.15.32:0/2343564925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7ff6780094d0 con 0x7ff68c0b62d0 2023-12-20T06:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff68bfff640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c0b62d0 0x7ff68c14ff50 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7ff67800e930 tx=0x7ff67800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff6897fa640 1 -- 172.21.15.32:0/2343564925 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff67800ca00 con 0x7ff68c0b62d0 2023-12-20T06:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff6897fa640 1 -- 172.21.15.32:0/2343564925 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7ff67800cb60 con 0x7ff68c0b62d0 2023-12-20T06:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff6897fa640 1 -- 172.21.15.32:0/2343564925 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7ff678013470 con 0x7ff68c0b62d0 2023-12-20T06:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff692bf0640 1 -- 172.21.15.32:0/2343564925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7ff684009a70 con 0x7ff68c0b62d0 2023-12-20T06:04:39.009 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff692bf0640 1 -- 172.21.15.32:0/2343564925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7ff68c15c590 con 0x7ff68c0b62d0 2023-12-20T06:04:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.005+0000 7ff692bf0640 1 -- 172.21.15.32:0/2343564925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7ff68c15c110 con 0x7ff68c0b62d0 2023-12-20T06:04:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.009+0000 7ff6897fa640 1 -- 172.21.15.32:0/2343564925 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7ff6780135d0 con 0x7ff68c0b62d0 2023-12-20T06:04:39.010 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.009+0000 7ff6897fa640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff670065b80 0x7ff670068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.009+0000 7ff6897fa640 1 -- 172.21.15.32:0/2343564925 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(255..255 src has 1..255) v4 ==== 6101+0+0 (secure 0 0 0) 0x7ff678083670 con 0x7ff68c0b62d0 2023-12-20T06:04:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.009+0000 7ff69098c640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff670065b80 0x7ff670068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:39.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.009+0000 7ff69098c640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff670065b80 0x7ff670068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff680007fd0 tx=0x7ff680007e40 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:39.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.013+0000 7ff6897fa640 1 -- 172.21.15.32:0/2343564925 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7ff68c15c110 con 0x7ff68c0b62d0 2023-12-20T06:04:39.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.173+0000 7ff692bf0640 1 -- 172.21.15.32:0/2343564925 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) v1 -- 0x7ff68c15c110 con 0x7ff68c0b62d0 2023-12-20T06:04:39.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.809+0000 7ff6897fa640 1 -- 172.21.15.32:0/2343564925 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v256) v1 ==== 169+0+0 (secure 0 0 0) 0x7ff68c15c110 con 0x7ff68c0b62d0 2023-12-20T06:04:39.811 INFO:teuthology.orchestra.run.smithi032.stderr:pool 'cephfs_data' removed 2023-12-20T06:04:39.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 -- 172.21.15.32:0/2343564925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff670065b80 msgr2=0x7ff670068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:39.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff670065b80 0x7ff670068040 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7ff680007fd0 tx=0x7ff680007e40 comp rx=0 tx=0).stop 2023-12-20T06:04:39.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 -- 172.21.15.32:0/2343564925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c0b62d0 msgr2=0x7ff68c14ff50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:39.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c0b62d0 0x7ff68c14ff50 secure :-1 s=READY pgs=2593 cs=0 l=1 rev1=1 crypto rx=0x7ff67800e930 tx=0x7ff67800edf0 comp rx=0 tx=0).stop 2023-12-20T06:04:39.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 -- 172.21.15.32:0/2343564925 shutdown_connections 2023-12-20T06:04:39.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7ff670065b80 0x7ff670068040 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:39.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7ff68c1603e0 0x7ff68c15cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:39.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7ff68c0b62d0 0x7ff68c14ff50 unknown :-1 s=CLOSED pgs=2593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:39.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 --2- 172.21.15.32:0/2343564925 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7ff68c0b1d90 0x7ff68c0ca020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:39.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 -- 172.21.15.32:0/2343564925 >> 172.21.15.32:0/2343564925 conn(0x7ff68c06b5e0 msgr2=0x7ff68c0b70a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:39.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 -- 172.21.15.32:0/2343564925 shutdown_connections 2023-12-20T06:04:39.817 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.813+0000 7ff692bf0640 1 -- 172.21.15.32:0/2343564925 wait complete. 2023-12-20T06:04:39.849 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/allow_m_granularity 2023-12-20T06:04:40.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.997+0000 7fce17160640 1 Processor -- start 2023-12-20T06:04:40.000 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:39.997+0000 7fce17160640 1 -- start start 2023-12-20T06:04:40.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce17160640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fce10070940 0x7fce100caaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce17160640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1015b6a0 con 0x7fce1015abb0 2023-12-20T06:04:40.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce17160640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fce1006d8c0 con 0x7fce100b0cb0 2023-12-20T06:04:40.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce17160640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fce1006e160 con 0x7fce10070940 2023-12-20T06:04:40.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce14efc640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7fce1015abb0 0x7fce1015bb30 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40746/0 (socket says 172.21.15.32:40746) 2023-12-20T06:04:40.002 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce14efc640 1 -- 172.21.15.32:0/1255769328 learned_addr learned my addr 172.21.15.32:0/1255769328 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:40.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce156fd640 1 --2- 172.21.15.32:0/1255769328 >> v2:172.21.15.32:3300/0 conn(0x7fce10070940 0x7fce100caaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce0f7fe640 1 -- 172.21.15.32:0/1255769328 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2671595275 0 0) 0x7fce1015b6a0 con 0x7fce1015abb0 2023-12-20T06:04:40.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce0f7fe640 1 -- 172.21.15.32:0/1255769328 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce00003530 con 0x7fce1015abb0 2023-12-20T06:04:40.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce0f7fe640 1 -- 172.21.15.32:0/1255769328 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2465491228 0 0) 0x7fce1006d8c0 con 0x7fce100b0cb0 2023-12-20T06:04:40.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce0f7fe640 1 -- 172.21.15.32:0/1255769328 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fce1015b6a0 con 0x7fce100b0cb0 2023-12-20T06:04:40.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce156fd640 1 -- 172.21.15.32:0/1255769328 >> v1:172.21.15.39:6789/0 conn(0x7fce100b0cb0 legacy=0x7fce10070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce156fd640 1 -- 172.21.15.32:0/1255769328 >> v1:172.21.15.32:6789/0 conn(0x7fce1015abb0 legacy=0x7fce1015bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.003 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.001+0000 7fce156fd640 1 -- 172.21.15.32:0/1255769328 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce1006dba0 con 0x7fce10070940 2023-12-20T06:04:40.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce156fd640 1 --2- 172.21.15.32:0/1255769328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce10070940 0x7fce100caaf0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7fce08009d90 tx=0x7fce0801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f6c08bbbdb4d75b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:40.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce0f7fe640 1 -- 172.21.15.32:0/1255769328 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce080a5a10 con 0x7fce10070940 2023-12-20T06:04:40.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce0f7fe640 1 -- 172.21.15.32:0/1255769328 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fce080a5b70 con 0x7fce10070940 2023-12-20T06:04:40.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce0f7fe640 1 -- 172.21.15.32:0/1255769328 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fce080ad440 con 0x7fce10070940 2023-12-20T06:04:40.006 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce17160640 1 -- 172.21.15.32:0/1255769328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce10070940 msgr2=0x7fce100caaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce17160640 1 --2- 172.21.15.32:0/1255769328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce10070940 0x7fce100caaf0 secure :-1 s=READY pgs=2595 cs=0 l=1 rev1=1 crypto rx=0x7fce08009d90 tx=0x7fce0801c950 comp rx=0 tx=0).stop 2023-12-20T06:04:40.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce17160640 1 -- 172.21.15.32:0/1255769328 shutdown_connections 2023-12-20T06:04:40.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce17160640 1 --2- 172.21.15.32:0/1255769328 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce10070940 0x7fce100caaf0 unknown :-1 s=CLOSED pgs=2595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce17160640 1 -- 172.21.15.32:0/1255769328 >> 172.21.15.32:0/1255769328 conn(0x7fce1006a510 msgr2=0x7fce100ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:40.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce17160640 1 -- 172.21.15.32:0/1255769328 shutdown_connections 2023-12-20T06:04:40.007 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce17160640 1 -- 172.21.15.32:0/1255769328 wait complete. 2023-12-20T06:04:40.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce17160640 1 Processor -- start 2023-12-20T06:04:40.008 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.005+0000 7fce17160640 1 -- start start 2023-12-20T06:04:40.011 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce17160640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce10070940 0x7fce101445e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce17160640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce100b0cb0 0x7fce100cb150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce17160640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce1015abb0 0x7fce100cb690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce17160640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1014dd50 con 0x7fce100b0cb0 2023-12-20T06:04:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce17160640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fce1014d180 con 0x7fce1015abb0 2023-12-20T06:04:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce17160640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fce1014d420 con 0x7fce10070940 2023-12-20T06:04:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce100b0cb0 0x7fce100cb150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce100b0cb0 0x7fce100cb150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51234/0 (socket says 172.21.15.32:51234) 2023-12-20T06:04:40.012 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0ffff640 1 -- 172.21.15.32:0/327380506 learned_addr learned my addr 172.21.15.32:0/327380506 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce156fd640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce1015abb0 0x7fce100cb690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce14efc640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce10070940 0x7fce101445e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0ffff640 1 -- 172.21.15.32:0/327380506 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce10070940 msgr2=0x7fce101445e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0ffff640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce10070940 0x7fce101445e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0ffff640 1 -- 172.21.15.32:0/327380506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce1015abb0 msgr2=0x7fce100cb690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0ffff640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce1015abb0 0x7fce100cb690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0ffff640 1 -- 172.21.15.32:0/327380506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fce08009a70 con 0x7fce100b0cb0 2023-12-20T06:04:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce14efc640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce10070940 0x7fce101445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0ffff640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce100b0cb0 0x7fce100cb150 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c500 tx=0x7fcdfc00c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:40.013 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0d7fa640 1 -- 172.21.15.32:0/327380506 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc0052d0 con 0x7fce100b0cb0 2023-12-20T06:04:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0d7fa640 1 -- 172.21.15.32:0/327380506 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcdfc005430 con 0x7fce100b0cb0 2023-12-20T06:04:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce17160640 1 -- 172.21.15.32:0/327380506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fce1014d700 con 0x7fce100b0cb0 2023-12-20T06:04:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0d7fa640 1 -- 172.21.15.32:0/327380506 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcdfc012870 con 0x7fce100b0cb0 2023-12-20T06:04:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce17160640 1 -- 172.21.15.32:0/327380506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fce10144b20 con 0x7fce100b0cb0 2023-12-20T06:04:40.014 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce17160640 1 -- 172.21.15.32:0/327380506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fce1014dec0 con 0x7fce100b0cb0 2023-12-20T06:04:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.009+0000 7fce0d7fa640 1 -- 172.21.15.32:0/327380506 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcdfc0129d0 con 0x7fce100b0cb0 2023-12-20T06:04:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.013+0000 7fce0d7fa640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcdf4065b80 0x7fcdf4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.013+0000 7fce0d7fa640 1 -- 172.21.15.32:0/327380506 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7fcdfc082f90 con 0x7fce100b0cb0 2023-12-20T06:04:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.013+0000 7fce14efc640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcdf4065b80 0x7fcdf4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.015 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.013+0000 7fce14efc640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcdf4065b80 0x7fcdf4068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fce040011f0 tx=0x7fce04009040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:40.017 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.017+0000 7fce0d7fa640 1 -- 172.21.15.32:0/327380506 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcdfc0507c0 con 0x7fce100b0cb0 2023-12-20T06:04:40.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.173+0000 7fce17160640 1 -- 172.21.15.32:0/327380506 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"} v 0) v1 -- 0x7fce1004e000 con 0x7fce100b0cb0 2023-12-20T06:04:40.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.177+0000 7fce0d7fa640 1 -- 172.21.15.32:0/327380506 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/allow_m_granularity"}]=0 v85) v1 ==== 120+0+0 (secure 0 0 0) 0x7fcdfc053de0 con 0x7fce100b0cb0 2023-12-20T06:04:40.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 -- 172.21.15.32:0/327380506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcdf4065b80 msgr2=0x7fcdf4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcdf4065b80 0x7fcdf4068040 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fce040011f0 tx=0x7fce04009040 comp rx=0 tx=0).stop 2023-12-20T06:04:40.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 -- 172.21.15.32:0/327380506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce100b0cb0 msgr2=0x7fce100cb150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce100b0cb0 0x7fce100cb150 secure :-1 s=READY pgs=2596 cs=0 l=1 rev1=1 crypto rx=0x7fcdfc00c500 tx=0x7fcdfc00c9c0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.183 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 -- 172.21.15.32:0/327380506 shutdown_connections 2023-12-20T06:04:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcdf4065b80 0x7fcdf4068040 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fce1015abb0 0x7fce100cb690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fce100b0cb0 0x7fce100cb150 unknown :-1 s=CLOSED pgs=2596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 --2- 172.21.15.32:0/327380506 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fce10070940 0x7fce101445e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 -- 172.21.15.32:0/327380506 >> 172.21.15.32:0/327380506 conn(0x7fce1006a510 msgr2=0x7fce10150520 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 -- 172.21.15.32:0/327380506 shutdown_connections 2023-12-20T06:04:40.184 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.181+0000 7fce17160640 1 -- 172.21.15.32:0/327380506 wait complete. 2023-12-20T06:04:40.203 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mgr mgr/snap_schedule/dump_on_update 2023-12-20T06:04:40.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.345+0000 7f3fd4da6640 1 Processor -- start 2023-12-20T06:04:40.349 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fd4da6640 1 -- start start 2023-12-20T06:04:40.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fd4da6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fd0157480 0x7f3fd00c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fd4da6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fd00b0760 0x7f3fd00b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.351 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fd4da6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f3fd015d660 0x7f3fd015daf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fd4da6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd00b1100 con 0x7f3fd00b0760 2023-12-20T06:04:40.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fd4da6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f3fd00cde20 con 0x7f3fd015d660 2023-12-20T06:04:40.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fd4da6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd00c8e30 con 0x7f3fd0157480 2023-12-20T06:04:40.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fc7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fd00b0760 0x7f3fd00b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fc7fff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f3fd00b0760 0x7f3fd00b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51248/0 (socket says 172.21.15.32:51248) 2023-12-20T06:04:40.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fc7fff640 1 -- 172.21.15.32:0/407333744 learned_addr learned my addr 172.21.15.32:0/407333744 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:40.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fced9d640 1 --2- 172.21.15.32:0/407333744 >> v2:172.21.15.39:3300/0 conn(0x7f3fd015d660 0x7f3fd015daf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fce59c640 1 --2- 172.21.15.32:0/407333744 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fd0157480 0x7f3fd00c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fc7fff640 1 -- 172.21.15.32:0/407333744 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fd0157480 msgr2=0x7f3fd00c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:40.352 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fc7fff640 1 --2- 172.21.15.32:0/407333744 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fd0157480 0x7f3fd00c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fc7fff640 1 -- 172.21.15.32:0/407333744 >> v2:172.21.15.39:3300/0 conn(0x7f3fd015d660 msgr2=0x7f3fd015daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:40.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fc7fff640 1 --2- 172.21.15.32:0/407333744 >> v2:172.21.15.39:3300/0 conn(0x7f3fd015d660 0x7f3fd015daf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.353 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.349+0000 7f3fc7fff640 1 -- 172.21.15.32:0/407333744 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fd00ce070 con 0x7f3fd00b0760 2023-12-20T06:04:40.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fc7fff640 1 --2- 172.21.15.32:0/407333744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fd00b0760 0x7f3fd00b0bc0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00b49b0 tx=0x7f3fb801c440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40d8a7fa9c970706 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:40.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fcdd9b640 1 -- 172.21.15.32:0/407333744 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80ab600 con 0x7f3fd00b0760 2023-12-20T06:04:40.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fcdd9b640 1 -- 172.21.15.32:0/407333744 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb80ab760 con 0x7f3fd00b0760 2023-12-20T06:04:40.354 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fcdd9b640 1 -- 172.21.15.32:0/407333744 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80abaf0 con 0x7f3fd00b0760 2023-12-20T06:04:40.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fd4da6640 1 -- 172.21.15.32:0/407333744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fd00b0760 msgr2=0x7f3fd00b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fd4da6640 1 --2- 172.21.15.32:0/407333744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fd00b0760 0x7f3fd00b0bc0 secure :-1 s=READY pgs=2597 cs=0 l=1 rev1=1 crypto rx=0x7f3fd00b49b0 tx=0x7f3fb801c440 comp rx=0 tx=0).stop 2023-12-20T06:04:40.355 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fd4da6640 1 -- 172.21.15.32:0/407333744 shutdown_connections 2023-12-20T06:04:40.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fd4da6640 1 --2- 172.21.15.32:0/407333744 >> v2:172.21.15.39:3300/0 conn(0x7f3fd015d660 0x7f3fd015daf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fd4da6640 1 --2- 172.21.15.32:0/407333744 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fd00b0760 0x7f3fd00b0bc0 unknown :-1 s=CLOSED pgs=2597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fd4da6640 1 --2- 172.21.15.32:0/407333744 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fd0157480 0x7f3fd00c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fd4da6640 1 -- 172.21.15.32:0/407333744 >> 172.21.15.32:0/407333744 conn(0x7f3fd006a510 msgr2=0x7f3fd00ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:40.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fd4da6640 1 -- 172.21.15.32:0/407333744 shutdown_connections 2023-12-20T06:04:40.356 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fd4da6640 1 -- 172.21.15.32:0/407333744 wait complete. 2023-12-20T06:04:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fd4da6640 1 Processor -- start 2023-12-20T06:04:40.357 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.353+0000 7f3fd4da6640 1 -- start start 2023-12-20T06:04:40.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fd4da6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fd00b0760 0x7f3fd0156490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fd4da6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fd0157480 0x7f3fd00c5cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fd4da6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd015d660 0x7f3fd00c46b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fd4da6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd01569d0 con 0x7f3fd0157480 2023-12-20T06:04:40.359 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fd4da6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd01f8f60 con 0x7f3fd015d660 2023-12-20T06:04:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fd4da6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f3fd01f9240 con 0x7f3fd00b0760 2023-12-20T06:04:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fd0157480 0x7f3fd00c5cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fd0157480 0x7f3fd00c5cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51254/0 (socket says 172.21.15.32:51254) 2023-12-20T06:04:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc7fff640 1 -- 172.21.15.32:0/2882776148 learned_addr learned my addr 172.21.15.32:0/2882776148 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fced9d640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd015d660 0x7f3fd00c46b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fce59c640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fd00b0760 0x7f3fd0156490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc7fff640 1 -- 172.21.15.32:0/2882776148 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fd00b0760 msgr2=0x7f3fd0156490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc7fff640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fd00b0760 0x7f3fd0156490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc7fff640 1 -- 172.21.15.32:0/2882776148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd015d660 msgr2=0x7f3fd00c46b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.360 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc7fff640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd015d660 0x7f3fd00c46b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc7fff640 1 -- 172.21.15.32:0/2882776148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3fb80094d0 con 0x7f3fd0157480 2023-12-20T06:04:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc7fff640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fd0157480 0x7f3fd00c5cf0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f3fb801aaf0 tx=0x7f3fb801ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc6ffd640 1 -- 172.21.15.32:0/2882776148 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80be5f0 con 0x7f3fd0157480 2023-12-20T06:04:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc6ffd640 1 -- 172.21.15.32:0/2882776148 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3fb80be770 con 0x7f3fd0157480 2023-12-20T06:04:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fd4da6640 1 -- 172.21.15.32:0/2882776148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f3fd01f9520 con 0x7f3fd0157480 2023-12-20T06:04:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fc6ffd640 1 -- 172.21.15.32:0/2882776148 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3fb80beb20 con 0x7f3fd0157480 2023-12-20T06:04:40.361 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fd4da6640 1 -- 172.21.15.32:0/2882776148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f3fd00c6230 con 0x7f3fd0157480 2023-12-20T06:04:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.357+0000 7f3fd4da6640 1 -- 172.21.15.32:0/2882776148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f3fd006dce0 con 0x7f3fd0157480 2023-12-20T06:04:40.362 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.361+0000 7f3fc6ffd640 1 -- 172.21.15.32:0/2882776148 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3fb80b43f0 con 0x7f3fd0157480 2023-12-20T06:04:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.361+0000 7f3fc6ffd640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3fb0065bb0 0x7f3fb0068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.361+0000 7f3fc6ffd640 1 -- 172.21.15.32:0/2882776148 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f3fb811d850 con 0x7f3fd0157480 2023-12-20T06:04:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.361+0000 7f3fce59c640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3fb0065bb0 0x7f3fb0068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.363 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.361+0000 7f3fce59c640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3fb0065bb0 0x7f3fb0068070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009a40 tx=0x7f3fc0009190 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:40.366 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.365+0000 7f3fc6ffd640 1 -- 172.21.15.32:0/2882776148 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3fb80eb010 con 0x7f3fd0157480 2023-12-20T06:04:40.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.521+0000 7f3fd4da6640 1 -- 172.21.15.32:0/2882776148 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"} v 0) v1 -- 0x7f3fd004e000 con 0x7f3fd0157480 2023-12-20T06:04:40.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.525+0000 7f3fc6ffd640 1 -- 172.21.15.32:0/2882776148 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mgr", "name": "mgr/snap_schedule/dump_on_update"}]=0 v86) v1 ==== 115+0+0 (secure 0 0 0) 0x7f3fb80ee630 con 0x7f3fd0157480 2023-12-20T06:04:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 -- 172.21.15.32:0/2882776148 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3fb0065bb0 msgr2=0x7f3fb0068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3fb0065bb0 0x7f3fb0068070 secure :-1 s=READY pgs=31 cs=0 l=1 rev1=1 crypto rx=0x7f3fc0009a40 tx=0x7f3fc0009190 comp rx=0 tx=0).stop 2023-12-20T06:04:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 -- 172.21.15.32:0/2882776148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fd0157480 msgr2=0x7f3fd00c5cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fd0157480 0x7f3fd00c5cf0 secure :-1 s=READY pgs=2598 cs=0 l=1 rev1=1 crypto rx=0x7f3fb801aaf0 tx=0x7f3fb801ac80 comp rx=0 tx=0).stop 2023-12-20T06:04:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 -- 172.21.15.32:0/2882776148 shutdown_connections 2023-12-20T06:04:40.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f3fb0065bb0 0x7f3fb0068070 unknown :-1 s=CLOSED pgs=31 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f3fd015d660 0x7f3fd00c46b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f3fd0157480 0x7f3fd00c5cf0 unknown :-1 s=CLOSED pgs=2598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 --2- 172.21.15.32:0/2882776148 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f3fd00b0760 0x7f3fd0156490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 -- 172.21.15.32:0/2882776148 >> 172.21.15.32:0/2882776148 conn(0x7f3fd006a510 msgr2=0x7f3fd00ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 -- 172.21.15.32:0/2882776148 shutdown_connections 2023-12-20T06:04:40.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.529+0000 7f3fd4da6640 1 -- 172.21.15.32:0/2882776148 wait complete. 2023-12-20T06:04:40.550 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config rm mon mon_allow_pool_delete 2023-12-20T06:04:40.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccfb18640 1 Processor -- start 2023-12-20T06:04:40.697 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccfb18640 1 -- start start 2023-12-20T06:04:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccfb18640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cc815ce60 0x7f9cc80d2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccfb18640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f9cc80d3130 0x7f9cc80d0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.699 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccfb18640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cc80d1300 0x7f9cc80b46e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccfb18640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc80d35a0 con 0x7f9cc815ce60 2023-12-20T06:04:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccfb18640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9cc80d2650 con 0x7f9cc80d3130 2023-12-20T06:04:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccfb18640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80d1a40 con 0x7f9cc80d1300 2023-12-20T06:04:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccd8b4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cc815ce60 0x7f9cc80d2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccd8b4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9cc815ce60 0x7f9cc80d2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51258/0 (socket says 172.21.15.32:51258) 2023-12-20T06:04:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccd8b4640 1 -- 172.21.15.32:0/1700960250 learned_addr learned my addr 172.21.15.32:0/1700960250 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9cce0b5640 1 --2- 172.21.15.32:0/1700960250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cc80d1300 0x7f9cc80b46e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccd0b3640 1 --2- 172.21.15.32:0/1700960250 >> v2:172.21.15.39:3300/0 conn(0x7f9cc80d3130 0x7f9cc80d0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccd8b4640 1 -- 172.21.15.32:0/1700960250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cc80d1300 msgr2=0x7f9cc80b46e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccd8b4640 1 --2- 172.21.15.32:0/1700960250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cc80d1300 0x7f9cc80b46e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.700 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccd8b4640 1 -- 172.21.15.32:0/1700960250 >> v2:172.21.15.39:3300/0 conn(0x7f9cc80d3130 msgr2=0x7f9cc80d0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:40.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccd8b4640 1 --2- 172.21.15.32:0/1700960250 >> v2:172.21.15.39:3300/0 conn(0x7f9cc80d3130 0x7f9cc80d0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccd8b4640 1 -- 172.21.15.32:0/1700960250 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cc80d1d20 con 0x7f9cc815ce60 2023-12-20T06:04:40.701 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.697+0000 7f9ccd0b3640 1 --2- 172.21.15.32:0/1700960250 >> v2:172.21.15.39:3300/0 conn(0x7f9cc80d3130 0x7f9cc80d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:40.702 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccd8b4640 1 --2- 172.21.15.32:0/1700960250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cc815ce60 0x7f9cc80d2bf0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8160a10 tx=0x7f9cc401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3709262017282100 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccc8b2640 1 -- 172.21.15.32:0/1700960250 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40a89b0 con 0x7f9cc815ce60 2023-12-20T06:04:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccc8b2640 1 -- 172.21.15.32:0/1700960250 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc40a8b10 con 0x7f9cc815ce60 2023-12-20T06:04:40.703 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccc8b2640 1 -- 172.21.15.32:0/1700960250 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc40a8ea0 con 0x7f9cc815ce60 2023-12-20T06:04:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1700960250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cc815ce60 msgr2=0x7f9cc80d2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccfb18640 1 --2- 172.21.15.32:0/1700960250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cc815ce60 0x7f9cc80d2bf0 secure :-1 s=READY pgs=2599 cs=0 l=1 rev1=1 crypto rx=0x7f9cc8160a10 tx=0x7f9cc401ecf0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1700960250 shutdown_connections 2023-12-20T06:04:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccfb18640 1 --2- 172.21.15.32:0/1700960250 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cc80d1300 0x7f9cc80b46e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccfb18640 1 --2- 172.21.15.32:0/1700960250 >> v2:172.21.15.39:3300/0 conn(0x7f9cc80d3130 0x7f9cc80d0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccfb18640 1 --2- 172.21.15.32:0/1700960250 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cc815ce60 0x7f9cc80d2bf0 unknown :-1 s=CLOSED pgs=2599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1700960250 >> 172.21.15.32:0/1700960250 conn(0x7f9cc806b5e0 msgr2=0x7f9cc806b9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1700960250 shutdown_connections 2023-12-20T06:04:40.704 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.701+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1700960250 wait complete. 2023-12-20T06:04:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccfb18640 1 Processor -- start 2023-12-20T06:04:40.705 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccfb18640 1 -- start start 2023-12-20T06:04:40.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccfb18640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80d1300 0x7f9cc8158890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.707 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccfb18640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cc80d3130 0x7f9cc81e2480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccfb18640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cc815ce60 0x7f9cc80c8220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccfb18640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc81f1fe0 con 0x7f9cc815ce60 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccfb18640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80ca380 con 0x7f9cc80d1300 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccfb18640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cc80ca0a0 con 0x7f9cc80d3130 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cce0b5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cc815ce60 0x7f9cc80c8220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cce0b5640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cc815ce60 0x7f9cc80c8220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51268/0 (socket says 172.21.15.32:51268) 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cce0b5640 1 -- 172.21.15.32:0/1312202810 learned_addr learned my addr 172.21.15.32:0/1312202810 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccd0b3640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cc80d3130 0x7f9cc81e2480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccd8b4640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80d1300 0x7f9cc8158890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cce0b5640 1 -- 172.21.15.32:0/1312202810 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cc80d3130 msgr2=0x7f9cc81e2480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cce0b5640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cc80d3130 0x7f9cc81e2480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.708 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cce0b5640 1 -- 172.21.15.32:0/1312202810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80d1300 msgr2=0x7f9cc8158890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cce0b5640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80d1300 0x7f9cc8158890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cce0b5640 1 -- 172.21.15.32:0/1312202810 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cb8009420 con 0x7f9cc815ce60 2023-12-20T06:04:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cce0b5640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cc815ce60 0x7f9cc80c8220 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000eed0 tx=0x7f9cc000b480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cbe7fc640 1 -- 172.21.15.32:0/1312202810 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0010040 con 0x7f9cc815ce60 2023-12-20T06:04:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cbe7fc640 1 -- 172.21.15.32:0/1312202810 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc000bd10 con 0x7f9cc815ce60 2023-12-20T06:04:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9cbe7fc640 1 -- 172.21.15.32:0/1312202810 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc0014640 con 0x7f9cc815ce60 2023-12-20T06:04:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1312202810 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cc40094d0 con 0x7f9cc815ce60 2023-12-20T06:04:40.709 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1312202810 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cc81f2790 con 0x7f9cc815ce60 2023-12-20T06:04:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.705+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1312202810 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cc804e000 con 0x7f9cc815ce60 2023-12-20T06:04:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.709+0000 7f9cbe7fc640 1 -- 172.21.15.32:0/1312202810 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9cc00147a0 con 0x7f9cc815ce60 2023-12-20T06:04:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.709+0000 7f9cbe7fc640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cac065b80 0x7f9cac068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.709+0000 7f9cbe7fc640 1 -- 172.21.15.32:0/1312202810 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f9cc0083ba0 con 0x7f9cc815ce60 2023-12-20T06:04:40.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.709+0000 7f9ccd8b4640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cac065b80 0x7f9cac068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:40.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.709+0000 7f9ccd8b4640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cac065b80 0x7f9cac068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40b2000 tx=0x7f9cc40aa070 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:40.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.713+0000 7f9cbe7fc640 1 -- 172.21.15.32:0/1312202810 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9cc0051490 con 0x7f9cc815ce60 2023-12-20T06:04:40.873 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.873+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1312202810 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"} v 0) v1 -- 0x7f9cc81f2980 con 0x7f9cc815ce60 2023-12-20T06:04:40.878 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9cbe7fc640 1 -- 172.21.15.32:0/1312202810 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "config rm", "who": "mon", "name": "mon_allow_pool_delete"}]=0 v87) v1 ==== 104+0+0 (secure 0 0 0) 0x7f9cc0054ab0 con 0x7f9cc815ce60 2023-12-20T06:04:40.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1312202810 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cac065b80 msgr2=0x7f9cac068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cac065b80 0x7f9cac068040 secure :-1 s=READY pgs=32 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40b2000 tx=0x7f9cc40aa070 comp rx=0 tx=0).stop 2023-12-20T06:04:40.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1312202810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cc815ce60 msgr2=0x7f9cc80c8220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:40.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cc815ce60 0x7f9cc80c8220 secure :-1 s=READY pgs=2600 cs=0 l=1 rev1=1 crypto rx=0x7f9cc000eed0 tx=0x7f9cc000b480 comp rx=0 tx=0).stop 2023-12-20T06:04:40.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1312202810 shutdown_connections 2023-12-20T06:04:40.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cac065b80 0x7f9cac068040 unknown :-1 s=CLOSED pgs=32 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.881 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cc815ce60 0x7f9cc80c8220 unknown :-1 s=CLOSED pgs=2600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cc80d3130 0x7f9cc81e2480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 --2- 172.21.15.32:0/1312202810 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cc80d1300 0x7f9cc8158890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:40.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1312202810 >> 172.21.15.32:0/1312202810 conn(0x7f9cc806b5e0 msgr2=0x7f9cc80b7d90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:40.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1312202810 shutdown_connections 2023-12-20T06:04:40.882 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:40.877+0000 7f9ccfb18640 1 -- 172.21.15.32:0/1312202810 wait complete. 2023-12-20T06:04:40.900 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph log 'Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name' 2023-12-20T06:04:41.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.041+0000 7f58f7752640 1 Processor -- start 2023-12-20T06:04:41.043 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.041+0000 7f58f7752640 1 -- start start 2023-12-20T06:04:41.045 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.041+0000 7f58f7752640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f58f006cf10 0x7f58f00c94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.041+0000 7f58f7752640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f58f0164ef0 0x7f58f00c99e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.041+0000 7f58f7752640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f58f00ae3f0 con 0x7f58f00ae010 2023-12-20T06:04:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.041+0000 7f58f7752640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f0161800 con 0x7f58f006cf10 2023-12-20T06:04:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.041+0000 7f58f7752640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f58f0165380 con 0x7f58f0164ef0 2023-12-20T06:04:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58f54ee640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f58f00ae010 0x7f58f0071470 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40760/0 (socket says 172.21.15.32:40760) 2023-12-20T06:04:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58f54ee640 1 -- 172.21.15.32:0/1457349698 learned_addr learned my addr 172.21.15.32:0/1457349698 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58f4ced640 1 --2- 172.21.15.32:0/1457349698 >> v2:172.21.15.32:3300/0 conn(0x7f58f006cf10 0x7f58f00c94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58f5cef640 1 --2- 172.21.15.32:0/1457349698 >> v2:172.21.15.39:3300/0 conn(0x7f58f0164ef0 0x7f58f00c99e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58f4ced640 1 -- 172.21.15.32:0/1457349698 >> v2:172.21.15.39:3300/0 conn(0x7f58f0164ef0 msgr2=0x7f58f00c99e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:41.046 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58f4ced640 1 --2- 172.21.15.32:0/1457349698 >> v2:172.21.15.39:3300/0 conn(0x7f58f0164ef0 0x7f58f00c99e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:41.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58f4ced640 1 -- 172.21.15.32:0/1457349698 >> v1:172.21.15.32:6789/0 conn(0x7f58f00ae010 legacy=0x7f58f0071470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:41.047 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58f4ced640 1 -- 172.21.15.32:0/1457349698 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58f0165660 con 0x7f58f006cf10 2023-12-20T06:04:41.048 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58f4ced640 1 --2- 172.21.15.32:0/1457349698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f006cf10 0x7f58f00c94a0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f58f00cf420 tx=0x7f58e001c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa41c0985c92716a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:41.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58e7fff640 1 -- 172.21.15.32:0/1457349698 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00ac600 con 0x7f58f006cf10 2023-12-20T06:04:41.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58e7fff640 1 -- 172.21.15.32:0/1457349698 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58e00a8050 con 0x7f58f006cf10 2023-12-20T06:04:41.049 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.045+0000 7f58e7fff640 1 -- 172.21.15.32:0/1457349698 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58e00ac980 con 0x7f58f006cf10 2023-12-20T06:04:41.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 -- 172.21.15.32:0/1457349698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f006cf10 msgr2=0x7f58f00c94a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:41.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 --2- 172.21.15.32:0/1457349698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f006cf10 0x7f58f00c94a0 secure :-1 s=READY pgs=2602 cs=0 l=1 rev1=1 crypto rx=0x7f58f00cf420 tx=0x7f58e001c3a0 comp rx=0 tx=0).stop 2023-12-20T06:04:41.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 -- 172.21.15.32:0/1457349698 shutdown_connections 2023-12-20T06:04:41.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 --2- 172.21.15.32:0/1457349698 >> v2:172.21.15.39:3300/0 conn(0x7f58f0164ef0 0x7f58f00c99e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:41.050 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 --2- 172.21.15.32:0/1457349698 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f006cf10 0x7f58f00c94a0 unknown :-1 s=CLOSED pgs=2602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:41.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 -- 172.21.15.32:0/1457349698 >> 172.21.15.32:0/1457349698 conn(0x7f58f006a320 msgr2=0x7f58f00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:41.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 -- 172.21.15.32:0/1457349698 shutdown_connections 2023-12-20T06:04:41.051 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 -- 172.21.15.32:0/1457349698 wait complete. 2023-12-20T06:04:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 Processor -- start 2023-12-20T06:04:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 -- start start 2023-12-20T06:04:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00ae010 0x7f58f0161970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f0161eb0 0x7f58f014d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58f014d840 0x7f58f01638f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f0162330 con 0x7f58f00ae010 2023-12-20T06:04:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f58f00b1440 con 0x7f58f014d840 2023-12-20T06:04:41.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f7752640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f58f0158b80 con 0x7f58f0161eb0 2023-12-20T06:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f54ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00ae010 0x7f58f0161970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f54ee640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00ae010 0x7f58f0161970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51282/0 (socket says 172.21.15.32:51282) 2023-12-20T06:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f54ee640 1 -- 172.21.15.32:0/1662247998 learned_addr learned my addr 172.21.15.32:0/1662247998 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f5cef640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58f014d840 0x7f58f01638f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f4ced640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f0161eb0 0x7f58f014d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f54ee640 1 -- 172.21.15.32:0/1662247998 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f0161eb0 msgr2=0x7f58f014d300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f54ee640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f0161eb0 0x7f58f014d300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f54ee640 1 -- 172.21.15.32:0/1662247998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58f014d840 msgr2=0x7f58f01638f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.049+0000 7f58f54ee640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58f014d840 0x7f58f01638f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.053+0000 7f58f54ee640 1 -- 172.21.15.32:0/1662247998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f58e0009420 con 0x7f58f00ae010 2023-12-20T06:04:41.055 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.053+0000 7f58f54ee640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00ae010 0x7f58f0161970 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00c4d0 tx=0x7f58ec00c990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.053+0000 7f58e5ffb640 1 -- 172.21.15.32:0/1662247998 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec007480 con 0x7f58f00ae010 2023-12-20T06:04:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.053+0000 7f58e5ffb640 1 -- 172.21.15.32:0/1662247998 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f58ec0075e0 con 0x7f58f00ae010 2023-12-20T06:04:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.053+0000 7f58e5ffb640 1 -- 172.21.15.32:0/1662247998 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f58ec0059b0 con 0x7f58f00ae010 2023-12-20T06:04:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.053+0000 7f58f7752640 1 -- 172.21.15.32:0/1662247998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f58f00b16e0 con 0x7f58f00ae010 2023-12-20T06:04:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.053+0000 7f58f7752640 1 -- 172.21.15.32:0/1662247998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f58f00b1ca0 con 0x7f58f00ae010 2023-12-20T06:04:41.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.053+0000 7f58f7752640 1 -- 172.21.15.32:0/1662247998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f58f004e000 con 0x7f58f00ae010 2023-12-20T06:04:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.053+0000 7f58e5ffb640 1 -- 172.21.15.32:0/1662247998 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f58ec005b10 con 0x7f58f00ae010 2023-12-20T06:04:41.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.057+0000 7f58e5ffb640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f58d4065bb0 0x7f58d4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.057+0000 7f58e5ffb640 1 -- 172.21.15.32:0/1662247998 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f58ec0826d0 con 0x7f58f00ae010 2023-12-20T06:04:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.057+0000 7f58f4ced640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f58d4065bb0 0x7f58d4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:41.058 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.057+0000 7f58f4ced640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f58d4065bb0 0x7f58d4068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58e00ae000 tx=0x7f58e00a7000 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:41.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.061+0000 7f58e5ffb640 1 -- 172.21.15.32:0/1662247998 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f58ec0500d0 con 0x7f58f00ae010 2023-12-20T06:04:41.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.273+0000 7f58f7752640 1 -- 172.21.15.32:0/1662247998 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]} v 0) v1 -- 0x7f58f0070df0 con 0x7f58f00ae010 2023-12-20T06:04:41.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58e5ffb640 1 -- 172.21.15.32:0/1662247998 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "log", "logtext": ["Ended test tasks.cephfs.test_snap_schedules.TestSnapSchedulesSnapdir.test_snap_dir_name"]}]=0 v3158) v1 ==== 155+0+0 (secure 0 0 0) 0x7f58ec017020 con 0x7f58f00ae010 2023-12-20T06:04:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 -- 172.21.15.32:0/1662247998 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f58d4065bb0 msgr2=0x7f58d4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f58d4065bb0 0x7f58d4068070 secure :-1 s=READY pgs=33 cs=0 l=1 rev1=1 crypto rx=0x7f58e00ae000 tx=0x7f58e00a7000 comp rx=0 tx=0).stop 2023-12-20T06:04:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 -- 172.21.15.32:0/1662247998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00ae010 msgr2=0x7f58f0161970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00ae010 0x7f58f0161970 secure :-1 s=READY pgs=2603 cs=0 l=1 rev1=1 crypto rx=0x7f58ec00c4d0 tx=0x7f58ec00c990 comp rx=0 tx=0).stop 2023-12-20T06:04:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 -- 172.21.15.32:0/1662247998 shutdown_connections 2023-12-20T06:04:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f58d4065bb0 0x7f58d4068070 unknown :-1 s=CLOSED pgs=33 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f58f014d840 0x7f58f01638f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f58f0161eb0 0x7f58f014d300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:41.829 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 --2- 172.21.15.32:0/1662247998 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f58f00ae010 0x7f58f0161970 unknown :-1 s=CLOSED pgs=2603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 -- 172.21.15.32:0/1662247998 >> 172.21.15.32:0/1662247998 conn(0x7f58f006a320 msgr2=0x7f58f00700c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 -- 172.21.15.32:0/1662247998 shutdown_connections 2023-12-20T06:04:41.830 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:41.825+0000 7f58f7752640 1 -- 172.21.15.32:0/1662247998 wait complete. 2023-12-20T06:04:41.850 INFO:tasks.cephfs_test_runner:Test the correctness of snap directory name ... ok 2023-12-20T06:04:41.851 INFO:tasks.cephfs_test_runner: 2023-12-20T06:04:41.851 INFO:tasks.cephfs_test_runner:---------------------------------------------------------------------- 2023-12-20T06:04:41.851 INFO:tasks.cephfs_test_runner:Ran 13 tests in 5735.000s 2023-12-20T06:04:41.851 INFO:tasks.cephfs_test_runner: 2023-12-20T06:04:41.851 INFO:tasks.cephfs_test_runner:OK 2023-12-20T06:04:41.852 DEBUG:teuthology.run_tasks:Unwinding manager cephfs_test_runner 2023-12-20T06:04:41.861 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2023-12-20T06:04:41.887 INFO:tasks.kclient:Unmounting kernel clients... 2023-12-20T06:04:41.887 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:04:41.888 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:41.898 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:04:41.898 DEBUG:teuthology.orchestra.run.smithi032:> set -ex 2023-12-20T06:04:41.898 DEBUG:teuthology.orchestra.run.smithi032:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:41.954 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16357764k,nr_inodes=4089441,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=7526 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3276000k,nr_inodes=819000,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-0 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-1 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-2 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-3 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3276004k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.2 nsfs rw 0 0 2023-12-20T06:04:41.954 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:04:41.954 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:41.965 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:04:41.965 DEBUG:teuthology.orchestra.run.smithi039:> set -ex 2023-12-20T06:04:41.965 DEBUG:teuthology.orchestra.run.smithi039:> dd if=/proc/self/mounts of=/dev/stdout 2023-12-20T06:04:42.015 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: sysfs /sys sysfs rw,nosuid,nodev,noexec,relatime 0 0 proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 udev /dev devtmpfs rw,nosuid,relatime,size=16354688k,nr_inodes=4088672,mode=755 0 0 devpts /dev/pts devpts rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 /dev/sda1 / ext4 rw,relatime,errors=remount-ro 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,nosuid,nodev 0 0 tmpfs /run/lock tmpfs rw,nosuid,nodev,noexec,relatime,size=5120k 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=30,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=8564 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,nosuid,nodev,noexec,relatime 0 0 nfsd /proc/fs/nfsd nfsd rw,relatime 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 ramfs /run/credentials/systemd-sysusers.service ramfs ro,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /run/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,nosuid,nodev,relatime,size=3275384k,nr_inodes=818846,mode=700,uid=1000,gid=1271 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_1 /var/lib/ceph/osd/ceph-4 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_2 /var/lib/ceph/osd/ceph-5 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_3 /var/lib/ceph/osd/ceph-6 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/mapper/vg_nvme-lv_4 /var/lib/ceph/osd/ceph-7 xfs rw,noatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 tmpfs /run/netns tmpfs rw,nosuid,nodev,noexec,relatime,size=3275388k,mode=755 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.1 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 nsfs /run/netns/ceph-ns--home-ubuntu-cephtest-mnt.3 nsfs rw 0 0 2023-12-20T06:04:42.016 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.016 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T06:04:42.026 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2023-12-20T06:04:42.027 INFO:teuthology.orchestra.run.smithi032.stdout:ceph-ns--home-ubuntu-cephtest-mnt.2 (id: 1) 2023-12-20T06:04:42.027 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.027 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2023-12-20T06:04:42.091 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.092 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.2 2023-12-20T06:04:42.180 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.180 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-12-20T06:04:42.379 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.379 DEBUG:teuthology.orchestra.run.smithi032:> ip netns list 2023-12-20T06:04:42.389 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.390 DEBUG:teuthology.orchestra.run.smithi032:> sudo ip link delete ceph-brx 2023-12-20T06:04:42.457 INFO:teuthology.orchestra.run.smithi032.stderr:Cannot find device "ceph-brx" 2023-12-20T06:04:42.459 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T06:04:42.459 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.459 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2023-12-20T06:04:42.470 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.1 (id: 0) 2023-12-20T06:04:42.471 INFO:teuthology.orchestra.run.smithi039.stdout:ceph-ns--home-ubuntu-cephtest-mnt.3 (id: 1) 2023-12-20T06:04:42.471 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.471 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.1 2023-12-20T06:04:42.537 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.537 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.3 2023-12-20T06:04:42.611 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.611 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2023-12-20T06:04:42.845 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.845 DEBUG:teuthology.orchestra.run.smithi039:> ip netns list 2023-12-20T06:04:42.856 INFO:teuthology.orchestra.run:Running command with timeout 300 2023-12-20T06:04:42.856 DEBUG:teuthology.orchestra.run.smithi039:> sudo ip link delete ceph-brx 2023-12-20T06:04:42.867 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:04:42.871 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:04:42.923 INFO:teuthology.orchestra.run.smithi039.stderr:Cannot find device "ceph-brx" 2023-12-20T06:04:42.925 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T06:04:42.926 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2023-12-20T06:04:42.937 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2023-12-20T06:04:42.937 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:04:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33b5bf0640 1 Processor -- start 2023-12-20T06:04:43.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33b5bf0640 1 -- start start 2023-12-20T06:04:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33b5bf0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33b006ce60 0x7f33b015b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33b5bf0640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33b015f110 0x7f33b015c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33b5bf0640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f33b015bbd0 con 0x7f33b00af5e0 2023-12-20T06:04:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33b5bf0640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f33b015beb0 con 0x7f33b015f110 2023-12-20T06:04:43.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33b5bf0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b015e910 con 0x7f33b006ce60 2023-12-20T06:04:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33affff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33b015f110 0x7f33b015c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33affff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f33b015f110 0x7f33b015c440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51296/0 (socket says 172.21.15.32:51296) 2023-12-20T06:04:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33affff640 1 -- 172.21.15.32:0/44538773 learned_addr learned my addr 172.21.15.32:0/44538773 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33aeffd640 1 --2- 172.21.15.32:0/44538773 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33b006ce60 0x7f33b015b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33affff640 1 -- 172.21.15.32:0/44538773 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33b006ce60 msgr2=0x7f33b015b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33affff640 1 --2- 172.21.15.32:0/44538773 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33b006ce60 0x7f33b015b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33affff640 1 -- 172.21.15.32:0/44538773 >> v1:172.21.15.39:6789/0 conn(0x7f33b00af5e0 legacy=0x7f33b00af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.133 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.129+0000 7f33affff640 1 -- 172.21.15.32:0/44538773 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33b015ebf0 con 0x7f33b015f110 2023-12-20T06:04:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33affff640 1 --2- 172.21.15.32:0/44538773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b015f110 0x7f33b015c440 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009e50 tx=0x7f33a801c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b67ba8a2070c6d33 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33ae7fc640 1 -- 172.21.15.32:0/44538773 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a80a57e0 con 0x7f33b015f110 2023-12-20T06:04:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33ae7fc640 1 -- 172.21.15.32:0/44538773 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a80a5940 con 0x7f33b015f110 2023-12-20T06:04:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33ae7fc640 1 -- 172.21.15.32:0/44538773 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a80a5cd0 con 0x7f33b015f110 2023-12-20T06:04:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33b5bf0640 1 -- 172.21.15.32:0/44538773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b015f110 msgr2=0x7f33b015c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33b5bf0640 1 --2- 172.21.15.32:0/44538773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b015f110 0x7f33b015c440 secure :-1 s=READY pgs=2604 cs=0 l=1 rev1=1 crypto rx=0x7f33a8009e50 tx=0x7f33a801c980 comp rx=0 tx=0).stop 2023-12-20T06:04:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33b5bf0640 1 -- 172.21.15.32:0/44538773 shutdown_connections 2023-12-20T06:04:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33b5bf0640 1 --2- 172.21.15.32:0/44538773 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b015f110 0x7f33b015c440 unknown :-1 s=CLOSED pgs=2604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33b5bf0640 1 --2- 172.21.15.32:0/44538773 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33b006ce60 0x7f33b015b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33b5bf0640 1 -- 172.21.15.32:0/44538773 >> 172.21.15.32:0/44538773 conn(0x7f33b006a320 msgr2=0x7f33b00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33b5bf0640 1 -- 172.21.15.32:0/44538773 shutdown_connections 2023-12-20T06:04:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.133+0000 7f33b5bf0640 1 -- 172.21.15.32:0/44538773 wait complete. 2023-12-20T06:04:43.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33b5bf0640 1 Processor -- start 2023-12-20T06:04:43.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33b5bf0640 1 -- start start 2023-12-20T06:04:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33b5bf0640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b006ce60 0x7f33b014ca60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33b5bf0640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33b00af5e0 0x7f33b0071cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33b5bf0640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33b015f110 0x7f33b00c4050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33b5bf0640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b00c5070 con 0x7f33b006ce60 2023-12-20T06:04:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33b5bf0640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f33b0072210 con 0x7f33b00af5e0 2023-12-20T06:04:43.140 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33b5bf0640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f33b00705d0 con 0x7f33b015f110 2023-12-20T06:04:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33af7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b006ce60 0x7f33b014ca60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33af7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b006ce60 0x7f33b014ca60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51306/0 (socket says 172.21.15.32:51306) 2023-12-20T06:04:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33af7fe640 1 -- 172.21.15.32:0/3977857595 learned_addr learned my addr 172.21.15.32:0/3977857595 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33aeffd640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33b00af5e0 0x7f33b0071cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33affff640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33b015f110 0x7f33b00c4050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33af7fe640 1 -- 172.21.15.32:0/3977857595 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33b015f110 msgr2=0x7f33b00c4050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33af7fe640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33b015f110 0x7f33b00c4050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33af7fe640 1 -- 172.21.15.32:0/3977857595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33b00af5e0 msgr2=0x7f33b0071cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.141 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33af7fe640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33b00af5e0 0x7f33b0071cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33af7fe640 1 -- 172.21.15.32:0/3977857595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f33a8009b30 con 0x7f33b006ce60 2023-12-20T06:04:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33affff640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33b015f110 0x7f33b00c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33af7fe640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b006ce60 0x7f33b014ca60 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c450 tx=0x7f33a000c910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f3397fff640 1 -- 172.21.15.32:0/3977857595 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a00052d0 con 0x7f33b006ce60 2023-12-20T06:04:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f3397fff640 1 -- 172.21.15.32:0/3977857595 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f33a0005430 con 0x7f33b006ce60 2023-12-20T06:04:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f3397fff640 1 -- 172.21.15.32:0/3977857595 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f33a0003890 con 0x7f33b006ce60 2023-12-20T06:04:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33b5bf0640 1 -- 172.21.15.32:0/3977857595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f33b00c4590 con 0x7f33b006ce60 2023-12-20T06:04:43.142 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33b5bf0640 1 -- 172.21.15.32:0/3977857595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f33b014d040 con 0x7f33b006ce60 2023-12-20T06:04:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.137+0000 7f33b5bf0640 1 -- 172.21.15.32:0/3977857595 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f33b004e000 con 0x7f33b006ce60 2023-12-20T06:04:43.143 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.141+0000 7f3397fff640 1 -- 172.21.15.32:0/3977857595 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f33a00039f0 con 0x7f33b006ce60 2023-12-20T06:04:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.141+0000 7f3397fff640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f339c065bb0 0x7f339c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.141+0000 7f3397fff640 1 -- 172.21.15.32:0/3977857595 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f33a0082d90 con 0x7f33b006ce60 2023-12-20T06:04:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.141+0000 7f33aeffd640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f339c065bb0 0x7f339c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.144 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.141+0000 7f33aeffd640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f339c065bb0 0x7f339c068070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f33b015e6f0 tx=0x7f33a4008040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:43.146 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.145+0000 7f3397fff640 1 -- 172.21.15.32:0/3977857595 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f33a0050640 con 0x7f33b006ce60 2023-12-20T06:04:43.305 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.301+0000 7f33b5bf0640 1 -- 172.21.15.32:0/3977857595 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f33b0158af0 con 0x7f339c065bb0 2023-12-20T06:04:43.308 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.305+0000 7f3397fff640 1 -- 172.21.15.32:0/3977857595 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50366 (secure 0 0 0) 0x7f33a40108a0 con 0x7f339c065bb0 2023-12-20T06:04:43.308 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:43.310 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:04:43.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 -- 172.21.15.32:0/3977857595 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f339c065bb0 msgr2=0x7f339c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f339c065bb0 0x7f339c068070 secure :-1 s=READY pgs=34 cs=0 l=1 rev1=1 crypto rx=0x7f33b015e6f0 tx=0x7f33a4008040 comp rx=0 tx=0).stop 2023-12-20T06:04:43.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 -- 172.21.15.32:0/3977857595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b006ce60 msgr2=0x7f33b014ca60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b006ce60 0x7f33b014ca60 secure :-1 s=READY pgs=2605 cs=0 l=1 rev1=1 crypto rx=0x7f33a000c450 tx=0x7f33a000c910 comp rx=0 tx=0).stop 2023-12-20T06:04:43.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 -- 172.21.15.32:0/3977857595 shutdown_connections 2023-12-20T06:04:43.310 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f339c065bb0 0x7f339c068070 unknown :-1 s=CLOSED pgs=34 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f33b015f110 0x7f33b00c4050 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f33b00af5e0 0x7f33b0071cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 --2- 172.21.15.32:0/3977857595 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f33b006ce60 0x7f33b014ca60 unknown :-1 s=CLOSED pgs=2605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 -- 172.21.15.32:0/3977857595 >> 172.21.15.32:0/3977857595 conn(0x7f33b006a320 msgr2=0x7f33b0156ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:43.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 -- 172.21.15.32:0/3977857595 shutdown_connections 2023-12-20T06:04:43.311 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.309+0000 7f33b5bf0640 1 -- 172.21.15.32:0/3977857595 wait complete. 2023-12-20T06:04:43.329 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-12-20T06:04:42.673707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3256,"ondisk_log_size":3256,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":439640,"kb_used_data":26392,"kb_used_omap":0,"kb_used_meta":413184,"kb_avail":754535080,"statfs":{"total":773094113280,"available":772643921920,"internally_reserved":0,"allocated":27025408,"data_stored":13873593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":423100416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668220,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-262,"num_write_kb":-2054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001086"},"pg_stats":[{"pgid":"1.0","version":"253'3256","reported_seq":4230,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-20T06:04:38.984634+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:04:38.984634+0000","last_peered":"2023-12-20T06:04:38.984634+0000","last_clean":"2023-12-20T06:04:38.984634+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:04:38.984634+0000","last_undegraded":"2023-12-20T06:04:38.984634+0000","last_fullsized":"2023-12-20T06:04:38.984634+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3256,"log_dups_size":0,"ondisk_log_size":3256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3256,"ondisk_log_size":3256,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575994,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61028,"kb_used_data":3740,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94310812,"statfs":{"total":96636764160,"available":96574271488,"internally_reserved":0,"allocated":3829760,"data_stored":2179341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95499999999999996,"5min":1.028,"15min":1.042},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.67300000000000004,"15min":0.72199999999999998},"min":{"1min":0.42899999999999999,"5min":0.39700000000000002,"15min":0.34499999999999997},"max":{"1min":0.90100000000000002,"5min":0.94799999999999995,"15min":0.99099999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69399999999999995,"15min":0.73199999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91800000000000004,"5min":0.94799999999999995,"15min":1.036},"last":0.626},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70499999999999996,"15min":0.73499999999999999},"min":{"1min":0.47599999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.69199999999999995,"15min":0.73099999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91000000000000003,"5min":0.95399999999999996,"15min":1.014},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49099999999999999},"max":{"1min":0.93300000000000005,"5min":1.0089999999999999,"15min":1.0189999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.73499999999999999},"min":{"1min":0.47599999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.77800000000000002},"min":{"1min":0.58299999999999996,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.98899999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.88200000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.628,"15min":0.59699999999999998},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.04},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.997,"5min":1.02,"15min":1.02},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.54300000000000004,"5min":0.37,"15min":0.37},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0449999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.84899999999999998}]},{"osd":6,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.57499999999999996,"5min":0.442,"15min":0.442},"max":{"1min":0.97299999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.79700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575994,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52240,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319600,"statfs":{"total":96636764160,"available":96583270400,"internally_reserved":0,"allocated":3219456,"data_stored":1582664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.72699999999999998},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.85099999999999998,"5min":1.016,"15min":1.0980000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68899999999999995,"15min":0.73499999999999999},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90600000000000003,"5min":1.006,"15min":1.05},"last":0.52000000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67800000000000005,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.88},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.47999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.97299999999999998,"15min":1.0960000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72999999999999998,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90100000000000002,"5min":0.97699999999999998,"15min":1.069},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.69199999999999995,"15min":0.746},"min":{"1min":0.47399999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.875,"5min":0.96799999999999997,"15min":1.0760000000000001},"last":0.64500000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93400000000000005,"5min":1.042,"15min":1.077},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71299999999999997,"15min":0.748},"min":{"1min":0.504,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.879,"5min":1.0309999999999999,"15min":1.052},"last":0.80800000000000005}]},{"osd":4,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75800000000000001,"5min":1.05,"15min":1.05},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.86699999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.377},"max":{"1min":0.79300000000000004,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71199999999999997,"15min":0.70499999999999996},"min":{"1min":0.502,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.90300000000000002}]},{"osd":7,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82899999999999996,"5min":1.103,"15min":1.1240000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97999999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.97999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575995,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.077},"last":0.628},{"interface":"front","average":{"1min":0.62,"5min":0.69099999999999995,"15min":0.73499999999999999},"min":{"1min":0.45600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.84199999999999997,"5min":0.96599999999999997,"15min":1.028},"last":0.65200000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.74399999999999999},"min":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82099999999999995,"5min":0.98799999999999999,"15min":1.0209999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68999999999999995,"15min":0.73899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.874},{"interface":"front","average":{"1min":0.628,"5min":0.72599999999999998,"15min":0.76100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48299999999999998},"max":{"1min":0.79000000000000004,"5min":1.0669999999999999,"15min":1.0680000000000001},"last":0.57999999999999996}]},{"osd":3,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75600000000000001,"15min":0.77800000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99199999999999999,"5min":1.05,"15min":1.0669999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.73399999999999999,"15min":0.76600000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89800000000000002,"5min":0.98099999999999998,"15min":1.0629999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65800000000000003,"15min":0.64900000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.077},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.437,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94599999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.51400000000000001,"5min":0.49299999999999999,"15min":0.46899999999999997},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.86899999999999999,"5min":1.097,"15min":1.097},"last":0.52900000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73299999999999998,"15min":0.70999999999999996},"min":{"1min":0.51200000000000001,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.0409999999999999,"5min":1.153,"15min":1.153},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77300000000000002,"15min":0.76600000000000001},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.55500000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.92000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575996,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67500000000000004,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.98999999999999999,"15min":1.0209999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.71799999999999997},"min":{"1min":0.46200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.92600000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.55800000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83999999999999997,"5min":0.95699999999999996,"15min":1.0900000000000001},"last":0.747},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47399999999999998,"15min":0.44500000000000001},"max":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72299999999999998,"15min":0.76200000000000001},"min":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.101},"last":0.76200000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70299999999999996,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":1.014,"15min":1.014},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97799999999999998,"5min":1.04,"15min":1.081},"last":0.65800000000000003}]},{"osd":5,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.1379999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.749,"15min":0.745},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.41199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.78600000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.0740000000000001,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.85999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575995,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45100000000000001,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.74399999999999999,"5min":0.748,"15min":2.8130000000000002},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.47899999999999998,"15min":0.52000000000000002},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56999999999999995,"5min":0.70599999999999996,"15min":1.357},"last":0.626}]},{"osd":1,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.44500000000000001,"15min":0.47299999999999998},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.628,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50900000000000001,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61199999999999999,"5min":0.747,"15min":1.4219999999999999},"last":1.292}]},{"osd":2,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54100000000000004,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.56699999999999995,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":1.042},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52800000000000002,"15min":0.56399999999999995},"min":{"1min":0.39100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.86099999999999999,"15min":2.944},"last":1.2749999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.80300000000000005},"min":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.92900000000000005,"5min":1.159,"15min":1.669},"last":1.4330000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.74099999999999999,"15min":0.79800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":1.1299999999999999,"15min":1.99},"last":1.093}]},{"osd":5,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.77100000000000002,"15min":0.81899999999999995},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.89300000000000002,"5min":1.1100000000000001,"15min":1.901},"last":1.343},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.749,"15min":0.80800000000000005},"min":{"1min":0.55900000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85499999999999998,"5min":1.196,"15min":2.8639999999999999},"last":1.407}]},{"osd":6,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.752,"15min":0.80400000000000005},"min":{"1min":0.57299999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89200000000000002,"5min":1.119,"15min":1.575},"last":1.4870000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77700000000000002,"15min":0.83399999999999996},"min":{"1min":0.57899999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.80100000000000005,"5min":1.1479999999999999,"15min":1.948},"last":1.1359999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.77300000000000002,"15min":0.82899999999999996},"min":{"1min":0.59499999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1579999999999999,"15min":2.9049999999999998},"last":1.587},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.84499999999999997},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":0.91000000000000003,"5min":1.1850000000000001,"15min":1.754},"last":1.383}]}]},{"osd":2,"up_from":13,"seq":55834575994,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42899999999999999,"15min":0.45100000000000001},"min":{"1min":0.29499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54000000000000004,"5min":0.75800000000000001,"15min":1.0049999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.497,"15min":0.51300000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.59899999999999998,"5min":0.93799999999999994,"15min":1.4319999999999999},"last":0.36699999999999999}]},{"osd":1,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46999999999999997,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65600000000000003,"5min":0.79800000000000004,"15min":0.82199999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.33300000000000002,"15min":0.30499999999999999},"max":{"1min":0.59999999999999998,"5min":0.92600000000000005,"15min":1.004},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60299999999999998,"15min":0.63400000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":1.089,"15min":1.3660000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.622,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89000000000000001,"5min":1.069,"15min":1.6919999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.75600000000000001,"15min":0.78900000000000003},"min":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93799999999999994,"5min":1.119,"15min":1.254},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.74399999999999999,"15min":0.78700000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.085,"15min":1.605},"last":0.85699999999999998}]},{"osd":5,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.5,"5min":0.498,"15min":0.498},"max":{"1min":0.85799999999999998,"5min":1.1639999999999999,"15min":1.329},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78400000000000003,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88,"5min":1.0609999999999999,"15min":1.095},"last":0.90500000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.76800000000000002,"15min":0.80500000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83199999999999996,"5min":1.157,"15min":1.919},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":0.81999999999999995},"min":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96399999999999997,"5min":1.117,"15min":1.2450000000000001},"last":0.96899999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.02,"5min":1.3360000000000001,"15min":1.7869999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.84599999999999997,"15min":0.876},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0009999999999999,"5min":1.262,"15min":1.292},"last":0.84099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575996,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60432,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311408,"statfs":{"total":96636764160,"available":96574881792,"internally_reserved":0,"allocated":3219456,"data_stored":1582664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.504,"15min":0.51600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.51700000000000002,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.57399999999999995},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.55800000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53300000000000003,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71399999999999997,"5min":0.78300000000000003,"15min":0.93500000000000005},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93100000000000005,"5min":1.107,"15min":1.321},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.81100000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.80900000000000005}]},{"osd":5,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.0649999999999999,"5min":1.224,"15min":1.2849999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77800000000000002,"15min":0.80400000000000005},"min":{"1min":0.55700000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.89500000000000002}]},{"osd":6,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.80400000000000005,"15min":0.82699999999999996},"min":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.488,"5min":2.488,"15min":2.488},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.53200000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.76800000000000002}]},{"osd":7,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.85799999999999998},"min":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.82499999999999996,"15min":0.85999999999999999},"min":{"1min":0.621,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.3839999999999999},"last":0.95999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575995,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56916,"kb_used_data":3724,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94314924,"statfs":{"total":96636764160,"available":96578482176,"internally_reserved":0,"allocated":3813376,"data_stored":2173032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.29299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":0.998,"15min":0.998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.54300000000000004,"15min":0.55200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.63900000000000001,"5min":3.681,"15min":3.681},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60999999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.502}]},{"osd":3,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.67300000000000004,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.42699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66000000000000003,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80500000000000005,"15min":0.82799999999999996},"min":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"max":{"1min":1.081,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.84499999999999997,"15min":0.84399999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.042,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81599999999999995,"15min":0.84299999999999997},"min":{"1min":0.53700000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.1220000000000001,"5min":2.23,"15min":2.23},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.83599999999999997},"min":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0529999999999999,"5min":1.482,"15min":1.482},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96099999999999997,"5min":5.7160000000000002,"15min":5.7160000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.86399999999999999,"15min":0.86599999999999999},"min":{"1min":0.56899999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94399999999999995,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:04:43.331 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:04:43.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f4315eb6640 1 Processor -- start 2023-12-20T06:04:43.474 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f4315eb6640 1 -- start start 2023-12-20T06:04:43.475 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f4315eb6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f43101593c0 0x7f431015d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f4315eb6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43100afdb0 0x7f43100cee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f4315eb6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f43100ae140 con 0x7f431006c3e0 2023-12-20T06:04:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f4315eb6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f43100ae9e0 con 0x7f43101593c0 2023-12-20T06:04:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f4315eb6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f431015dee0 con 0x7f43100afdb0 2023-12-20T06:04:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f430ffff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f431006c3e0 0x7f43100705a0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40766/0 (socket says 172.21.15.32:40766) 2023-12-20T06:04:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f430ffff640 1 -- 172.21.15.32:0/140357767 learned_addr learned my addr 172.21.15.32:0/140357767 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f430effd640 1 --2- 172.21.15.32:0/140357767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43100afdb0 0x7f43100cee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f430f7fe640 1 --2- 172.21.15.32:0/140357767 >> v2:172.21.15.39:3300/0 conn(0x7f43101593c0 0x7f431015d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f430e7fc640 1 -- 172.21.15.32:0/140357767 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 377799366 0 0) 0x7f43100ae140 con 0x7f431006c3e0 2023-12-20T06:04:43.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f430e7fc640 1 -- 172.21.15.32:0/140357767 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f42f8005740 con 0x7f431006c3e0 2023-12-20T06:04:43.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f430effd640 1 -- 172.21.15.32:0/140357767 >> v2:172.21.15.39:3300/0 conn(0x7f43101593c0 msgr2=0x7f431015d9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:43.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f430effd640 1 --2- 172.21.15.32:0/140357767 >> v2:172.21.15.39:3300/0 conn(0x7f43101593c0 0x7f431015d9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f430effd640 1 -- 172.21.15.32:0/140357767 >> v1:172.21.15.32:6789/0 conn(0x7f431006c3e0 legacy=0x7f43100705a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f430effd640 1 -- 172.21.15.32:0/140357767 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4310072170 con 0x7f43100afdb0 2023-12-20T06:04:43.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.473+0000 7f430f7fe640 1 --2- 172.21.15.32:0/140357767 >> v2:172.21.15.39:3300/0 conn(0x7f43101593c0 0x7f431015d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:43.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f430effd640 1 --2- 172.21.15.32:0/140357767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43100afdb0 0x7f43100cee70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f43040097c0 tx=0x7f430401c3c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=3379cea9b5f7733e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:43.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f430e7fc640 1 -- 172.21.15.32:0/140357767 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43040ab600 con 0x7f43100afdb0 2023-12-20T06:04:43.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f430e7fc640 1 -- 172.21.15.32:0/140357767 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43040a7050 con 0x7f43100afdb0 2023-12-20T06:04:43.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f430e7fc640 1 -- 172.21.15.32:0/140357767 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43040ab980 con 0x7f43100afdb0 2023-12-20T06:04:43.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f4315eb6640 1 -- 172.21.15.32:0/140357767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43100afdb0 msgr2=0x7f43100cee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f4315eb6640 1 --2- 172.21.15.32:0/140357767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43100afdb0 0x7f43100cee70 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f43040097c0 tx=0x7f430401c3c0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f4315eb6640 1 -- 172.21.15.32:0/140357767 shutdown_connections 2023-12-20T06:04:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f4315eb6640 1 --2- 172.21.15.32:0/140357767 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f43100afdb0 0x7f43100cee70 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f4315eb6640 1 --2- 172.21.15.32:0/140357767 >> v2:172.21.15.39:3300/0 conn(0x7f43101593c0 0x7f431015d9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f4315eb6640 1 -- 172.21.15.32:0/140357767 >> 172.21.15.32:0/140357767 conn(0x7f431006a320 msgr2=0x7f43100ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f4315eb6640 1 -- 172.21.15.32:0/140357767 shutdown_connections 2023-12-20T06:04:43.481 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.477+0000 7f4315eb6640 1 -- 172.21.15.32:0/140357767 wait complete. 2023-12-20T06:04:43.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f4315eb6640 1 Processor -- start 2023-12-20T06:04:43.482 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f4315eb6640 1 -- start start 2023-12-20T06:04:43.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f4315eb6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f431006c3e0 0x7f431014fd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.484 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f4315eb6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43100afdb0 0x7f4310157e20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f4315eb6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43101593c0 0x7f4310158360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f4315eb6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43100b4110 con 0x7f43100afdb0 2023-12-20T06:04:43.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f4315eb6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f43100b3820 con 0x7f43101593c0 2023-12-20T06:04:43.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f4315eb6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f43100b3ac0 con 0x7f431006c3e0 2023-12-20T06:04:43.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43100afdb0 0x7f4310157e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430effd640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43100afdb0 0x7f4310157e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51310/0 (socket says 172.21.15.32:51310) 2023-12-20T06:04:43.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430effd640 1 -- 172.21.15.32:0/2578176914 learned_addr learned my addr 172.21.15.32:0/2578176914 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:43.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430f7fe640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f431006c3e0 0x7f431014fd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.485 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430ffff640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43101593c0 0x7f4310158360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430effd640 1 -- 172.21.15.32:0/2578176914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f431006c3e0 msgr2=0x7f431014fd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430effd640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f431006c3e0 0x7f431014fd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430effd640 1 -- 172.21.15.32:0/2578176914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43101593c0 msgr2=0x7f4310158360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430effd640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43101593c0 0x7f4310158360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430effd640 1 -- 172.21.15.32:0/2578176914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f43000093b0 con 0x7f43100afdb0 2023-12-20T06:04:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430f7fe640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f431006c3e0 0x7f431014fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f430effd640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43100afdb0 0x7f4310157e20 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f43040ae040 tx=0x7f4304019160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f42f7fff640 1 -- 172.21.15.32:0/2578176914 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43040b4430 con 0x7f43100afdb0 2023-12-20T06:04:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f42f7fff640 1 -- 172.21.15.32:0/2578176914 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f43040b45b0 con 0x7f43100afdb0 2023-12-20T06:04:43.486 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f42f7fff640 1 -- 172.21.15.32:0/2578176914 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f43040b4960 con 0x7f43100afdb0 2023-12-20T06:04:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f4315eb6640 1 -- 172.21.15.32:0/2578176914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4304009420 con 0x7f43100afdb0 2023-12-20T06:04:43.487 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.481+0000 7f4315eb6640 1 -- 172.21.15.32:0/2578176914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f43101502d0 con 0x7f43100afdb0 2023-12-20T06:04:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.485+0000 7f4315eb6640 1 -- 172.21.15.32:0/2578176914 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f431004e000 con 0x7f43100afdb0 2023-12-20T06:04:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.485+0000 7f42f7fff640 1 -- 172.21.15.32:0/2578176914 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f43040b2040 con 0x7f43100afdb0 2023-12-20T06:04:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.485+0000 7f42f7fff640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42fc065b80 0x7f42fc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.485+0000 7f42f7fff640 1 -- 172.21.15.32:0/2578176914 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f430411d2b0 con 0x7f43100afdb0 2023-12-20T06:04:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.485+0000 7f430f7fe640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42fc065b80 0x7f42fc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.485+0000 7f430f7fe640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42fc065b80 0x7f42fc068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4300009d20 tx=0x7f4300008040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:43.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.489+0000 7f42f7fff640 1 -- 172.21.15.32:0/2578176914 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f43040eaa50 con 0x7f43100afdb0 2023-12-20T06:04:43.644 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.641+0000 7f4315eb6640 1 -- 172.21.15.32:0/2578176914 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f43101597a0 con 0x7f42fc065b80 2023-12-20T06:04:43.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f42f7fff640 1 -- 172.21.15.32:0/2578176914 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50366 (secure 0 0 0) 0x7f4300009970 con 0x7f42fc065b80 2023-12-20T06:04:43.647 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:43.647 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:04:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 -- 172.21.15.32:0/2578176914 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42fc065b80 msgr2=0x7f42fc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42fc065b80 0x7f42fc068040 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f4300009d20 tx=0x7f4300008040 comp rx=0 tx=0).stop 2023-12-20T06:04:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 -- 172.21.15.32:0/2578176914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43100afdb0 msgr2=0x7f4310157e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43100afdb0 0x7f4310157e20 secure :-1 s=READY pgs=2607 cs=0 l=1 rev1=1 crypto rx=0x7f43040ae040 tx=0x7f4304019160 comp rx=0 tx=0).stop 2023-12-20T06:04:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 -- 172.21.15.32:0/2578176914 shutdown_connections 2023-12-20T06:04:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f42fc065b80 0x7f42fc068040 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.649 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f43101593c0 0x7f4310158360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f43100afdb0 0x7f4310157e20 unknown :-1 s=CLOSED pgs=2607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 --2- 172.21.15.32:0/2578176914 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f431006c3e0 0x7f431014fd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 -- 172.21.15.32:0/2578176914 >> 172.21.15.32:0/2578176914 conn(0x7f431006a320 msgr2=0x7f4310164b80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 -- 172.21.15.32:0/2578176914 shutdown_connections 2023-12-20T06:04:43.650 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.645+0000 7f4315eb6640 1 -- 172.21.15.32:0/2578176914 wait complete. 2023-12-20T06:04:43.668 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-12-20T06:04:42.673707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3256,"ondisk_log_size":3256,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":439640,"kb_used_data":26392,"kb_used_omap":0,"kb_used_meta":413184,"kb_avail":754535080,"statfs":{"total":773094113280,"available":772643921920,"internally_reserved":0,"allocated":27025408,"data_stored":13873593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":423100416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668220,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-262,"num_write_kb":-2054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001086"},"pg_stats":[{"pgid":"1.0","version":"253'3256","reported_seq":4230,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-20T06:04:38.984634+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:04:38.984634+0000","last_peered":"2023-12-20T06:04:38.984634+0000","last_clean":"2023-12-20T06:04:38.984634+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:04:38.984634+0000","last_undegraded":"2023-12-20T06:04:38.984634+0000","last_fullsized":"2023-12-20T06:04:38.984634+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3256,"log_dups_size":0,"ondisk_log_size":3256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3256,"ondisk_log_size":3256,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575994,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61028,"kb_used_data":3740,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94310812,"statfs":{"total":96636764160,"available":96574271488,"internally_reserved":0,"allocated":3829760,"data_stored":2179341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95499999999999996,"5min":1.028,"15min":1.042},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.67300000000000004,"15min":0.72199999999999998},"min":{"1min":0.42899999999999999,"5min":0.39700000000000002,"15min":0.34499999999999997},"max":{"1min":0.90100000000000002,"5min":0.94799999999999995,"15min":0.99099999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69399999999999995,"15min":0.73199999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91800000000000004,"5min":0.94799999999999995,"15min":1.036},"last":0.626},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70499999999999996,"15min":0.73499999999999999},"min":{"1min":0.47599999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.69199999999999995,"15min":0.73099999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91000000000000003,"5min":0.95399999999999996,"15min":1.014},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49099999999999999},"max":{"1min":0.93300000000000005,"5min":1.0089999999999999,"15min":1.0189999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.73499999999999999},"min":{"1min":0.47599999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.77800000000000002},"min":{"1min":0.58299999999999996,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.98899999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.88200000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.628,"15min":0.59699999999999998},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.04},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.997,"5min":1.02,"15min":1.02},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.54300000000000004,"5min":0.37,"15min":0.37},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0449999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.84899999999999998}]},{"osd":6,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.57499999999999996,"5min":0.442,"15min":0.442},"max":{"1min":0.97299999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.79700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575994,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52240,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319600,"statfs":{"total":96636764160,"available":96583270400,"internally_reserved":0,"allocated":3219456,"data_stored":1582664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.72699999999999998},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.85099999999999998,"5min":1.016,"15min":1.0980000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68899999999999995,"15min":0.73499999999999999},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90600000000000003,"5min":1.006,"15min":1.05},"last":0.52000000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67800000000000005,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.88},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.47999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.97299999999999998,"15min":1.0960000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72999999999999998,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90100000000000002,"5min":0.97699999999999998,"15min":1.069},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.69199999999999995,"15min":0.746},"min":{"1min":0.47399999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.875,"5min":0.96799999999999997,"15min":1.0760000000000001},"last":0.64500000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93400000000000005,"5min":1.042,"15min":1.077},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71299999999999997,"15min":0.748},"min":{"1min":0.504,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.879,"5min":1.0309999999999999,"15min":1.052},"last":0.80800000000000005}]},{"osd":4,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75800000000000001,"5min":1.05,"15min":1.05},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.86699999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.377},"max":{"1min":0.79300000000000004,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71199999999999997,"15min":0.70499999999999996},"min":{"1min":0.502,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.90300000000000002}]},{"osd":7,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82899999999999996,"5min":1.103,"15min":1.1240000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97999999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.97999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575995,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.077},"last":0.628},{"interface":"front","average":{"1min":0.62,"5min":0.69099999999999995,"15min":0.73499999999999999},"min":{"1min":0.45600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.84199999999999997,"5min":0.96599999999999997,"15min":1.028},"last":0.65200000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.74399999999999999},"min":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82099999999999995,"5min":0.98799999999999999,"15min":1.0209999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68999999999999995,"15min":0.73899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.874},{"interface":"front","average":{"1min":0.628,"5min":0.72599999999999998,"15min":0.76100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48299999999999998},"max":{"1min":0.79000000000000004,"5min":1.0669999999999999,"15min":1.0680000000000001},"last":0.57999999999999996}]},{"osd":3,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75600000000000001,"15min":0.77800000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99199999999999999,"5min":1.05,"15min":1.0669999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.73399999999999999,"15min":0.76600000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89800000000000002,"5min":0.98099999999999998,"15min":1.0629999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65800000000000003,"15min":0.64900000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.077},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.437,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94599999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.51400000000000001,"5min":0.49299999999999999,"15min":0.46899999999999997},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.86899999999999999,"5min":1.097,"15min":1.097},"last":0.52900000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73299999999999998,"15min":0.70999999999999996},"min":{"1min":0.51200000000000001,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.0409999999999999,"5min":1.153,"15min":1.153},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77300000000000002,"15min":0.76600000000000001},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.55500000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.92000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575996,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67500000000000004,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.98999999999999999,"15min":1.0209999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.71799999999999997},"min":{"1min":0.46200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.92600000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.55800000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83999999999999997,"5min":0.95699999999999996,"15min":1.0900000000000001},"last":0.747},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47399999999999998,"15min":0.44500000000000001},"max":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72299999999999998,"15min":0.76200000000000001},"min":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.101},"last":0.76200000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70299999999999996,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":1.014,"15min":1.014},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97799999999999998,"5min":1.04,"15min":1.081},"last":0.65800000000000003}]},{"osd":5,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.1379999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.749,"15min":0.745},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.41199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.78600000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.0740000000000001,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.85999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575995,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45100000000000001,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.74399999999999999,"5min":0.748,"15min":2.8130000000000002},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.47899999999999998,"15min":0.52000000000000002},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56999999999999995,"5min":0.70599999999999996,"15min":1.357},"last":0.626}]},{"osd":1,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.44500000000000001,"15min":0.47299999999999998},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.628,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50900000000000001,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61199999999999999,"5min":0.747,"15min":1.4219999999999999},"last":1.292}]},{"osd":2,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54100000000000004,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.56699999999999995,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":1.042},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52800000000000002,"15min":0.56399999999999995},"min":{"1min":0.39100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.86099999999999999,"15min":2.944},"last":1.2749999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.80300000000000005},"min":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.92900000000000005,"5min":1.159,"15min":1.669},"last":1.4330000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.74099999999999999,"15min":0.79800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":1.1299999999999999,"15min":1.99},"last":1.093}]},{"osd":5,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.77100000000000002,"15min":0.81899999999999995},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.89300000000000002,"5min":1.1100000000000001,"15min":1.901},"last":1.343},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.749,"15min":0.80800000000000005},"min":{"1min":0.55900000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85499999999999998,"5min":1.196,"15min":2.8639999999999999},"last":1.407}]},{"osd":6,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.752,"15min":0.80400000000000005},"min":{"1min":0.57299999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89200000000000002,"5min":1.119,"15min":1.575},"last":1.4870000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77700000000000002,"15min":0.83399999999999996},"min":{"1min":0.57899999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.80100000000000005,"5min":1.1479999999999999,"15min":1.948},"last":1.1359999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.77300000000000002,"15min":0.82899999999999996},"min":{"1min":0.59499999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1579999999999999,"15min":2.9049999999999998},"last":1.587},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.84499999999999997},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":0.91000000000000003,"5min":1.1850000000000001,"15min":1.754},"last":1.383}]}]},{"osd":2,"up_from":13,"seq":55834575994,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42899999999999999,"15min":0.45100000000000001},"min":{"1min":0.29499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54000000000000004,"5min":0.75800000000000001,"15min":1.0049999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.497,"15min":0.51300000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.59899999999999998,"5min":0.93799999999999994,"15min":1.4319999999999999},"last":0.36699999999999999}]},{"osd":1,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46999999999999997,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65600000000000003,"5min":0.79800000000000004,"15min":0.82199999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.33300000000000002,"15min":0.30499999999999999},"max":{"1min":0.59999999999999998,"5min":0.92600000000000005,"15min":1.004},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60299999999999998,"15min":0.63400000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":1.089,"15min":1.3660000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.622,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89000000000000001,"5min":1.069,"15min":1.6919999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.75600000000000001,"15min":0.78900000000000003},"min":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93799999999999994,"5min":1.119,"15min":1.254},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.74399999999999999,"15min":0.78700000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.085,"15min":1.605},"last":0.85699999999999998}]},{"osd":5,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.5,"5min":0.498,"15min":0.498},"max":{"1min":0.85799999999999998,"5min":1.1639999999999999,"15min":1.329},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78400000000000003,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88,"5min":1.0609999999999999,"15min":1.095},"last":0.90500000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.76800000000000002,"15min":0.80500000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83199999999999996,"5min":1.157,"15min":1.919},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":0.81999999999999995},"min":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96399999999999997,"5min":1.117,"15min":1.2450000000000001},"last":0.96899999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.02,"5min":1.3360000000000001,"15min":1.7869999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.84599999999999997,"15min":0.876},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0009999999999999,"5min":1.262,"15min":1.292},"last":0.84099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575996,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60432,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311408,"statfs":{"total":96636764160,"available":96574881792,"internally_reserved":0,"allocated":3219456,"data_stored":1582664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.504,"15min":0.51600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.51700000000000002,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.57399999999999995},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.55800000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53300000000000003,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71399999999999997,"5min":0.78300000000000003,"15min":0.93500000000000005},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93100000000000005,"5min":1.107,"15min":1.321},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.81100000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.80900000000000005}]},{"osd":5,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.0649999999999999,"5min":1.224,"15min":1.2849999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77800000000000002,"15min":0.80400000000000005},"min":{"1min":0.55700000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.89500000000000002}]},{"osd":6,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.80400000000000005,"15min":0.82699999999999996},"min":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.488,"5min":2.488,"15min":2.488},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.53200000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.76800000000000002}]},{"osd":7,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.85799999999999998},"min":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.82499999999999996,"15min":0.85999999999999999},"min":{"1min":0.621,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.3839999999999999},"last":0.95999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575995,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56916,"kb_used_data":3724,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94314924,"statfs":{"total":96636764160,"available":96578482176,"internally_reserved":0,"allocated":3813376,"data_stored":2173032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.29299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":0.998,"15min":0.998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.54300000000000004,"15min":0.55200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.63900000000000001,"5min":3.681,"15min":3.681},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60999999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.502}]},{"osd":3,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.67300000000000004,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.42699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66000000000000003,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80500000000000005,"15min":0.82799999999999996},"min":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"max":{"1min":1.081,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.84499999999999997,"15min":0.84399999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.042,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81599999999999995,"15min":0.84299999999999997},"min":{"1min":0.53700000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.1220000000000001,"5min":2.23,"15min":2.23},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.83599999999999997},"min":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0529999999999999,"5min":1.482,"15min":1.482},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96099999999999997,"5min":5.7160000000000002,"15min":5.7160000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.86399999999999999,"15min":0.86599999999999999},"min":{"1min":0.56899999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94399999999999995,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:04:43.669 INFO:tasks.ceph.ceph_manager.ceph:clean! 2023-12-20T06:04:43.669 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:04:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa3b6d640 1 Processor -- start 2023-12-20T06:04:43.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa3b6d640 1 -- start start 2023-12-20T06:04:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa3b6d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a9c15cdb0 0x7f7a9c0d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa3b6d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9c15d190 con 0x7f7a9c0cfc80 2023-12-20T06:04:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa3b6d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7a9c0d16a0 con 0x7f7a9c0d2740 2023-12-20T06:04:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa3b6d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7a9c0d1980 con 0x7f7a9c15cdb0 2023-12-20T06:04:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa1909640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a9c15cdb0 0x7f7a9c0d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa1909640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7a9c15cdb0 0x7f7a9c0d0be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51320/0 (socket says 172.21.15.32:51320) 2023-12-20T06:04:43.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa1909640 1 -- 172.21.15.32:0/2548559017 learned_addr learned my addr 172.21.15.32:0/2548559017 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa1909640 1 -- 172.21.15.32:0/2548559017 >> v1:172.21.15.39:6789/0 conn(0x7f7a9c0d2740 legacy=0x7f7a9c0cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa1909640 1 -- 172.21.15.32:0/2548559017 >> v1:172.21.15.32:6789/0 conn(0x7f7a9c0cfc80 legacy=0x7f7a9c0b5c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.813+0000 7f7aa1909640 1 -- 172.21.15.32:0/2548559017 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a9c0d1c60 con 0x7f7a9c15cdb0 2023-12-20T06:04:43.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.817+0000 7f7aa1909640 1 --2- 172.21.15.32:0/2548559017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c15cdb0 0x7f7a9c0d0be0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009870 tx=0x7f7a9801ed30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abc6ea667174f285 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:43.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.817+0000 7f7aa0907640 1 -- 172.21.15.32:0/2548559017 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a980a8a30 con 0x7f7a9c15cdb0 2023-12-20T06:04:43.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.817+0000 7f7aa0907640 1 -- 172.21.15.32:0/2548559017 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a980a8b90 con 0x7f7a9c15cdb0 2023-12-20T06:04:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.817+0000 7f7aa0907640 1 -- 172.21.15.32:0/2548559017 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a980b0440 con 0x7f7a9c15cdb0 2023-12-20T06:04:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.817+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/2548559017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c15cdb0 msgr2=0x7f7a9c0d0be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.817+0000 7f7aa3b6d640 1 --2- 172.21.15.32:0/2548559017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c15cdb0 0x7f7a9c0d0be0 secure :-1 s=READY pgs=2609 cs=0 l=1 rev1=1 crypto rx=0x7f7a98009870 tx=0x7f7a9801ed30 comp rx=0 tx=0).stop 2023-12-20T06:04:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.817+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/2548559017 shutdown_connections 2023-12-20T06:04:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.817+0000 7f7aa3b6d640 1 --2- 172.21.15.32:0/2548559017 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c15cdb0 0x7f7a9c0d0be0 unknown :-1 s=CLOSED pgs=2609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.817+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/2548559017 >> 172.21.15.32:0/2548559017 conn(0x7f7a9c065ee0 msgr2=0x7f7a9c06b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.817+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/2548559017 shutdown_connections 2023-12-20T06:04:43.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.817+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/2548559017 wait complete. 2023-12-20T06:04:43.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa3b6d640 1 Processor -- start 2023-12-20T06:04:43.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa3b6d640 1 -- start start 2023-12-20T06:04:43.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa3b6d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a9c0cfc80 0x7f7a9c0c86d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa3b6d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0d2740 0x7f7a9c150870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa3b6d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a9c15cdb0 0x7f7a9c15e9c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.823 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa3b6d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c0c8c10 con 0x7f7a9c0d2740 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa3b6d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c14f870 con 0x7f7a9c15cdb0 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa3b6d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7a9c150db0 con 0x7f7a9c0cfc80 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa1108640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0d2740 0x7f7a9c150870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa1108640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0d2740 0x7f7a9c150870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51330/0 (socket says 172.21.15.32:51330) 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa1108640 1 -- 172.21.15.32:0/1373767264 learned_addr learned my addr 172.21.15.32:0/1373767264 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa1909640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a9c0cfc80 0x7f7a9c0c86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa210a640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a9c15cdb0 0x7f7a9c15e9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa1108640 1 -- 172.21.15.32:0/1373767264 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a9c0cfc80 msgr2=0x7f7a9c0c86d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa1108640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a9c0cfc80 0x7f7a9c0c86d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa1108640 1 -- 172.21.15.32:0/1373767264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a9c15cdb0 msgr2=0x7f7a9c15e9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa1108640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a9c15cdb0 0x7f7a9c15e9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.824 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa1108640 1 -- 172.21.15.32:0/1373767264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7a980094d0 con 0x7f7a9c0d2740 2023-12-20T06:04:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa210a640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a9c15cdb0 0x7f7a9c15e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa1108640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0d2740 0x7f7a9c150870 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00c530 tx=0x7f7a8c00c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7a927fc640 1 -- 172.21.15.32:0/1373767264 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c005220 con 0x7f7a9c0d2740 2023-12-20T06:04:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7a927fc640 1 -- 172.21.15.32:0/1373767264 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7a8c005380 con 0x7f7a9c0d2740 2023-12-20T06:04:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7a927fc640 1 -- 172.21.15.32:0/1373767264 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7a8c007270 con 0x7f7a9c0d2740 2023-12-20T06:04:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/1373767264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7a9c150580 con 0x7f7a9c0d2740 2023-12-20T06:04:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/1373767264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7a9c14ef40 con 0x7f7a9c0d2740 2023-12-20T06:04:43.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.821+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/1373767264 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7a9c1f2d10 con 0x7f7a9c0d2740 2023-12-20T06:04:43.826 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.825+0000 7f7a927fc640 1 -- 172.21.15.32:0/1373767264 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7a8c0073d0 con 0x7f7a9c0d2740 2023-12-20T06:04:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.825+0000 7f7a927fc640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a80065b80 0x7f7a80068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.825+0000 7f7a927fc640 1 -- 172.21.15.32:0/1373767264 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f7a8c082b70 con 0x7f7a9c0d2740 2023-12-20T06:04:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.825+0000 7f7aa1909640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a80065b80 0x7f7a80068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:43.827 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.825+0000 7f7aa1909640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a80065b80 0x7f7a80068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7a980b2000 tx=0x7f7a98009040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:43.831 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.829+0000 7f7a927fc640 1 -- 172.21.15.32:0/1373767264 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7a8c050420 con 0x7f7a9c0d2740 2023-12-20T06:04:43.986 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.985+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/1373767264 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f7a9c0d2220 con 0x7f7a80065b80 2023-12-20T06:04:43.988 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.985+0000 7f7a927fc640 1 -- 172.21.15.32:0/1373767264 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50366 (secure 0 0 0) 0x7f7a980b05c0 con 0x7f7a80065b80 2023-12-20T06:04:43.989 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:43.989 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:04:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/1373767264 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a80065b80 msgr2=0x7f7a80068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a80065b80 0x7f7a80068040 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f7a980b2000 tx=0x7f7a98009040 comp rx=0 tx=0).stop 2023-12-20T06:04:43.990 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/1373767264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0d2740 msgr2=0x7f7a9c150870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0d2740 0x7f7a9c150870 secure :-1 s=READY pgs=2610 cs=0 l=1 rev1=1 crypto rx=0x7f7a8c00c530 tx=0x7f7a8c00c9f0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/1373767264 shutdown_connections 2023-12-20T06:04:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7a80065b80 0x7f7a80068040 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7a9c15cdb0 0x7f7a9c15e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7a9c0d2740 0x7f7a9c150870 unknown :-1 s=CLOSED pgs=2610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 --2- 172.21.15.32:0/1373767264 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7a9c0cfc80 0x7f7a9c0c86d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/1373767264 >> 172.21.15.32:0/1373767264 conn(0x7f7a9c065ee0 msgr2=0x7f7a9c0cd5c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/1373767264 shutdown_connections 2023-12-20T06:04:43.991 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:43.989+0000 7f7aa3b6d640 1 -- 172.21.15.32:0/1373767264 wait complete. 2023-12-20T06:04:44.009 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":10,"stamp":"2023-12-20T06:04:42.673707+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3256,"ondisk_log_size":3256,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":152,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":4,"kb":754974720,"kb_used":439640,"kb_used_data":26392,"kb_used_omap":0,"kb_used_meta":413184,"kb_avail":754535080,"statfs":{"total":773094113280,"available":772643921920,"internally_reserved":0,"allocated":27025408,"data_stored":13873593,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":423100416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":16,"apply_latency_ms":16,"commit_latency_ns":16000000,"apply_latency_ns":16000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-668220,"num_objects":-25,"num_object_clones":0,"num_object_copies":-50,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":-25,"num_whiteouts":0,"num_read":-221,"num_read_kb":-75,"num_write":-262,"num_write_kb":-2054,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-14,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001086"},"pg_stats":[{"pgid":"1.0","version":"253'3256","reported_seq":4230,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-20T06:04:38.984634+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:04:38.984634+0000","last_peered":"2023-12-20T06:04:38.984634+0000","last_clean":"2023-12-20T06:04:38.984634+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:04:38.984634+0000","last_undegraded":"2023-12-20T06:04:38.984634+0000","last_fullsized":"2023-12-20T06:04:38.984634+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3256,"log_dups_size":0,"ondisk_log_size":3256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3256,"ondisk_log_size":3256,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575994,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61028,"kb_used_data":3740,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94310812,"statfs":{"total":96636764160,"available":96574271488,"internally_reserved":0,"allocated":3829760,"data_stored":2179341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95499999999999996,"5min":1.028,"15min":1.042},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.67300000000000004,"15min":0.72199999999999998},"min":{"1min":0.42899999999999999,"5min":0.39700000000000002,"15min":0.34499999999999997},"max":{"1min":0.90100000000000002,"5min":0.94799999999999995,"15min":0.99099999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69399999999999995,"15min":0.73199999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91800000000000004,"5min":0.94799999999999995,"15min":1.036},"last":0.626},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70499999999999996,"15min":0.73499999999999999},"min":{"1min":0.47599999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.69199999999999995,"15min":0.73099999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91000000000000003,"5min":0.95399999999999996,"15min":1.014},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49099999999999999},"max":{"1min":0.93300000000000005,"5min":1.0089999999999999,"15min":1.0189999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.73499999999999999},"min":{"1min":0.47599999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.77800000000000002},"min":{"1min":0.58299999999999996,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.98899999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.88200000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.628,"15min":0.59699999999999998},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.04},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.997,"5min":1.02,"15min":1.02},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.54300000000000004,"5min":0.37,"15min":0.37},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0449999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.84899999999999998}]},{"osd":6,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.57499999999999996,"5min":0.442,"15min":0.442},"max":{"1min":0.97299999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.79700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575994,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":52240,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319600,"statfs":{"total":96636764160,"available":96583270400,"internally_reserved":0,"allocated":3219456,"data_stored":1582664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.72699999999999998},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.85099999999999998,"5min":1.016,"15min":1.0980000000000001},"last":0.84699999999999998},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68899999999999995,"15min":0.73499999999999999},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90600000000000003,"5min":1.006,"15min":1.05},"last":0.52000000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67800000000000005,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.88},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.47999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.97299999999999998,"15min":1.0960000000000001},"last":0.60799999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72999999999999998,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90100000000000002,"5min":0.97699999999999998,"15min":1.069},"last":0.78100000000000003},{"interface":"front","average":{"1min":0.623,"5min":0.69199999999999995,"15min":0.746},"min":{"1min":0.47399999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.875,"5min":0.96799999999999997,"15min":1.0760000000000001},"last":0.64500000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93400000000000005,"5min":1.042,"15min":1.077},"last":0.73899999999999999},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71299999999999997,"15min":0.748},"min":{"1min":0.504,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.879,"5min":1.0309999999999999,"15min":1.052},"last":0.80800000000000005}]},{"osd":4,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75800000000000001,"5min":1.05,"15min":1.05},"last":0.71199999999999997},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.86699999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.86299999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.377},"max":{"1min":0.79300000000000004,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71199999999999997,"15min":0.70499999999999996},"min":{"1min":0.502,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.90300000000000002}]},{"osd":7,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82899999999999996,"5min":1.103,"15min":1.1240000000000001},"last":0.82199999999999995},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97999999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.97999999999999998}]}]},{"osd":5,"up_from":13,"seq":55834575995,"num_pgs":13,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.077},"last":0.628},{"interface":"front","average":{"1min":0.62,"5min":0.69099999999999995,"15min":0.73499999999999999},"min":{"1min":0.45600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.84199999999999997,"5min":0.96599999999999997,"15min":1.028},"last":0.65200000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.74399999999999999},"min":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.67700000000000005},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82099999999999995,"5min":0.98799999999999999,"15min":1.0209999999999999},"last":0.59799999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68999999999999995,"15min":0.73899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.874},{"interface":"front","average":{"1min":0.628,"5min":0.72599999999999998,"15min":0.76100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48299999999999998},"max":{"1min":0.79000000000000004,"5min":1.0669999999999999,"15min":1.0680000000000001},"last":0.57999999999999996}]},{"osd":3,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75600000000000001,"15min":0.77800000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99199999999999999,"5min":1.05,"15min":1.0669999999999999},"last":0.79000000000000004},{"interface":"front","average":{"1min":0.624,"5min":0.73399999999999999,"15min":0.76600000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89800000000000002,"5min":0.98099999999999998,"15min":1.0629999999999999},"last":0.82899999999999996}]},{"osd":4,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65800000000000003,"15min":0.64900000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.077},"last":0.77000000000000002},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.437,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94599999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.55500000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.51400000000000001,"5min":0.49299999999999999,"15min":0.46899999999999997},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.69499999999999995},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.86899999999999999,"5min":1.097,"15min":1.097},"last":0.52900000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73299999999999998,"15min":0.70999999999999996},"min":{"1min":0.51200000000000001,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.0409999999999999,"5min":1.153,"15min":1.153},"last":0.72699999999999998},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77300000000000002,"15min":0.76600000000000001},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.55500000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.92000000000000004}]}]},{"osd":4,"up_from":13,"seq":55834575996,"num_pgs":20,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67500000000000004,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.98999999999999999,"15min":1.0209999999999999},"last":0.73299999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.71799999999999997},"min":{"1min":0.46200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.92600000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.55800000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83999999999999997,"5min":0.95699999999999996,"15min":1.0900000000000001},"last":0.747},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47399999999999998,"15min":0.44500000000000001},"max":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72299999999999998,"15min":0.76200000000000001},"min":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.52700000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.101},"last":0.76200000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70299999999999996,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":1.014,"15min":1.014},"last":0.47299999999999998},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97799999999999998,"5min":1.04,"15min":1.081},"last":0.65800000000000003}]},{"osd":5,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.58999999999999997},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.63700000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.1379999999999999},"last":0.61799999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.749,"15min":0.745},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.41199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.78600000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.0740000000000001,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.85999999999999999}]}]},{"osd":3,"up_from":13,"seq":55834575995,"num_pgs":19,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45100000000000001,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.74399999999999999,"5min":0.748,"15min":2.8130000000000002},"last":1.1839999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.47899999999999998,"15min":0.52000000000000002},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56999999999999995,"5min":0.70599999999999996,"15min":1.357},"last":0.626}]},{"osd":1,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.44500000000000001,"15min":0.47299999999999998},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.628,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.84999999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50900000000000001,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61199999999999999,"5min":0.747,"15min":1.4219999999999999},"last":1.292}]},{"osd":2,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54100000000000004,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.56699999999999995,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":1.042},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52800000000000002,"15min":0.56399999999999995},"min":{"1min":0.39100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.86099999999999999,"15min":2.944},"last":1.2749999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.80300000000000005},"min":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.92900000000000005,"5min":1.159,"15min":1.669},"last":1.4330000000000001},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.74099999999999999,"15min":0.79800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":1.1299999999999999,"15min":1.99},"last":1.093}]},{"osd":5,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.77100000000000002,"15min":0.81899999999999995},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.89300000000000002,"5min":1.1100000000000001,"15min":1.901},"last":1.343},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.749,"15min":0.80800000000000005},"min":{"1min":0.55900000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85499999999999998,"5min":1.196,"15min":2.8639999999999999},"last":1.407}]},{"osd":6,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.752,"15min":0.80400000000000005},"min":{"1min":0.57299999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89200000000000002,"5min":1.119,"15min":1.575},"last":1.4870000000000001},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77700000000000002,"15min":0.83399999999999996},"min":{"1min":0.57899999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.80100000000000005,"5min":1.1479999999999999,"15min":1.948},"last":1.1359999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.77300000000000002,"15min":0.82899999999999996},"min":{"1min":0.59499999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1579999999999999,"15min":2.9049999999999998},"last":1.587},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.84499999999999997},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":0.91000000000000003,"5min":1.1850000000000001,"15min":1.754},"last":1.383}]}]},{"osd":2,"up_from":13,"seq":55834575994,"num_pgs":27,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42899999999999999,"15min":0.45100000000000001},"min":{"1min":0.29499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54000000000000004,"5min":0.75800000000000001,"15min":1.0049999999999999},"last":0.34799999999999998},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.497,"15min":0.51300000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.59899999999999998,"5min":0.93799999999999994,"15min":1.4319999999999999},"last":0.36699999999999999}]},{"osd":1,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46999999999999997,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65600000000000003,"5min":0.79800000000000004,"15min":0.82199999999999995},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.33300000000000002,"15min":0.30499999999999999},"max":{"1min":0.59999999999999998,"5min":0.92600000000000005,"15min":1.004},"last":0.41199999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60299999999999998,"15min":0.63400000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":1.089,"15min":1.3660000000000001},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.622,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89000000000000001,"5min":1.069,"15min":1.6919999999999999},"last":0.55100000000000005}]},{"osd":4,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.75600000000000001,"15min":0.78900000000000003},"min":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93799999999999994,"5min":1.119,"15min":1.254},"last":0.70299999999999996},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.74399999999999999,"15min":0.78700000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.085,"15min":1.605},"last":0.85699999999999998}]},{"osd":5,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.5,"5min":0.498,"15min":0.498},"max":{"1min":0.85799999999999998,"5min":1.1639999999999999,"15min":1.329},"last":1.0329999999999999},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78400000000000003,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88,"5min":1.0609999999999999,"15min":1.095},"last":0.90500000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.76800000000000002,"15min":0.80500000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83199999999999996,"5min":1.157,"15min":1.919},"last":0.88800000000000001},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":0.81999999999999995},"min":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96399999999999997,"5min":1.117,"15min":1.2450000000000001},"last":0.96899999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.02,"5min":1.3360000000000001,"15min":1.7869999999999999},"last":0.96199999999999997},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.84599999999999997,"15min":0.876},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0009999999999999,"5min":1.262,"15min":1.292},"last":0.84099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834575996,"num_pgs":16,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":60432,"kb_used_data":3144,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311408,"statfs":{"total":96636764160,"available":96574881792,"internally_reserved":0,"allocated":3219456,"data_stored":1582664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.42299999999999999,"5min":0.46500000000000002,"15min":0.46200000000000002},"min":{"1min":0.25800000000000001,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.89900000000000002,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.34999999999999998},{"interface":"front","average":{"1min":0.48199999999999998,"5min":0.504,"15min":0.51600000000000001},"min":{"1min":0.28599999999999998,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":1.04,"5min":1.04,"15min":1.04},"last":0.51500000000000001}]},{"osd":2,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.47499999999999998,"5min":0.51700000000000002,"15min":0.52900000000000003},"min":{"1min":0.36499999999999999,"5min":0.31,"15min":0.31},"max":{"1min":0.79600000000000004,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.41999999999999998},{"interface":"front","average":{"1min":0.54500000000000004,"5min":0.56299999999999994,"15min":0.57399999999999995},"min":{"1min":0.39400000000000002,"5min":0.36599999999999999,"15min":0.36599999999999999},"max":{"1min":1.081,"5min":1.081,"15min":1.081},"last":0.55800000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.54100000000000004,"5min":0.59799999999999998,"15min":0.61199999999999999},"min":{"1min":0.38300000000000001,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":0.98899999999999999,"5min":0.98899999999999999,"15min":0.98899999999999999},"last":0.46000000000000002},{"interface":"front","average":{"1min":0.48299999999999998,"5min":0.53300000000000003,"15min":0.55000000000000004},"min":{"1min":0.35099999999999998,"5min":0.33000000000000002,"15min":0.33000000000000002},"max":{"1min":0.71399999999999997,"5min":0.78300000000000003,"15min":0.93500000000000005},"last":0.53300000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.745,"5min":0.76600000000000001,"15min":0.78400000000000003},"min":{"1min":0.52200000000000002,"5min":0.443,"15min":0.443},"max":{"1min":0.93100000000000005,"5min":1.107,"15min":1.321},"last":0.93500000000000005},{"interface":"front","average":{"1min":0.76700000000000002,"5min":0.79700000000000004,"15min":0.81100000000000005},"min":{"1min":0.53600000000000003,"5min":0.53600000000000003,"15min":0.53600000000000003},"max":{"1min":1.3680000000000001,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.80900000000000005}]},{"osd":5,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71599999999999997,"5min":0.77400000000000002,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.46000000000000002,"15min":0.443},"max":{"1min":1.0649999999999999,"5min":1.224,"15min":1.2849999999999999},"last":0.73999999999999999},{"interface":"front","average":{"1min":0.75800000000000001,"5min":0.77800000000000002,"15min":0.80400000000000005},"min":{"1min":0.55700000000000005,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":1.448,"5min":1.448,"15min":1.448},"last":0.89500000000000002}]},{"osd":6,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.83699999999999997,"5min":0.80400000000000005,"15min":0.82699999999999996},"min":{"1min":0.55300000000000005,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":2.488,"5min":2.488,"15min":2.488},"last":0.91500000000000004},{"interface":"front","average":{"1min":0.79600000000000004,"5min":0.80000000000000004,"15min":0.81100000000000005},"min":{"1min":0.53200000000000003,"5min":0.52500000000000002,"15min":0.52500000000000002},"max":{"1min":1.7250000000000001,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.76800000000000002}]},{"osd":7,"last update":"Wed Dec 20 06:03:42 2023","interfaces":[{"interface":"back","average":{"1min":0.76500000000000001,"5min":0.82699999999999996,"15min":0.85799999999999998},"min":{"1min":0.54000000000000004,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":1.6799999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.79700000000000004},{"interface":"front","average":{"1min":0.80700000000000005,"5min":0.82499999999999996,"15min":0.85999999999999999},"min":{"1min":0.621,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.208,"5min":1.208,"15min":1.3839999999999999},"last":0.95999999999999996}]}]},{"osd":0,"up_from":13,"seq":55834575995,"num_pgs":21,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56916,"kb_used_data":3724,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94314924,"statfs":{"total":96636764160,"available":96578482176,"internally_reserved":0,"allocated":3813376,"data_stored":2173032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.29299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":0.998,"15min":0.998},"last":0.34300000000000003},{"interface":"front","average":{"1min":0.434,"5min":0.54300000000000004,"15min":0.55200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.63900000000000001,"5min":3.681,"15min":3.681},"last":0.56499999999999995}]},{"osd":2,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60999999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.40500000000000003},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.502}]},{"osd":3,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.67300000000000004,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.42399999999999999},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.42699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66000000000000003,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.63700000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.61699999999999999},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80500000000000005,"15min":0.82799999999999996},"min":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"max":{"1min":1.081,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.54900000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.84499999999999997,"15min":0.84399999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.042,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.60299999999999998},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81599999999999995,"15min":0.84299999999999997},"min":{"1min":0.53700000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.1220000000000001,"5min":2.23,"15min":2.23},"last":0.53000000000000003}]},{"osd":6,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.83599999999999997},"min":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0529999999999999,"5min":1.482,"15min":1.482},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.65600000000000003}]},{"osd":7,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96099999999999997,"5min":5.7160000000000002,"15min":5.7160000000000002},"last":0.68899999999999995},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.86399999999999999,"15min":0.86599999999999999},"min":{"1min":0.56899999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94399999999999995,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.57799999999999996}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:04:44.011 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2023-12-20T06:04:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.153+0000 7f9e413a2640 1 Processor -- start 2023-12-20T06:04:44.156 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.153+0000 7f9e413a2640 1 -- start start 2023-12-20T06:04:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e413a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e3c159ab0 0x7f9e3c0714c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e413a2640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e3c0b3e90 0x7f9e3c071a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e413a2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9e3c071f40 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e413a2640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9e3c0ca0b0 con 0x7f9e3c0b3e90 2023-12-20T06:04:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e413a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0ad7e0 con 0x7f9e3c159ab0 2023-12-20T06:04:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e40ba1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e3c0b3e90 0x7f9e3c071a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e40ba1640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9e3c0b3e90 0x7f9e3c071a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51344/0 (socket says 172.21.15.32:51344) 2023-12-20T06:04:44.158 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e40ba1640 1 -- 172.21.15.32:0/847564665 learned_addr learned my addr 172.21.15.32:0/847564665 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e3bfff640 1 --2- 172.21.15.32:0/847564665 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e3c159ab0 0x7f9e3c0714c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e40ba1640 1 -- 172.21.15.32:0/847564665 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e3c159ab0 msgr2=0x7f9e3c0714c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e40ba1640 1 --2- 172.21.15.32:0/847564665 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e3c159ab0 0x7f9e3c0714c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e40ba1640 1 -- 172.21.15.32:0/847564665 >> v1:172.21.15.39:6789/0 conn(0x7f9e3c0cd2e0 legacy=0x7f9e3c0cc140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:44.159 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e40ba1640 1 -- 172.21.15.32:0/847564665 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e3c0ada80 con 0x7f9e3c0b3e90 2023-12-20T06:04:44.160 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e40ba1640 1 --2- 172.21.15.32:0/847564665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0b3e90 0x7f9e3c071a00 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009e50 tx=0x7f9e3401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3852136247b45e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e3affd640 1 -- 172.21.15.32:0/847564665 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340a57c0 con 0x7f9e3c0b3e90 2023-12-20T06:04:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e3affd640 1 -- 172.21.15.32:0/847564665 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e340a5920 con 0x7f9e3c0b3e90 2023-12-20T06:04:44.161 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.157+0000 7f9e3affd640 1 -- 172.21.15.32:0/847564665 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e340a5cb0 con 0x7f9e3c0b3e90 2023-12-20T06:04:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 -- 172.21.15.32:0/847564665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0b3e90 msgr2=0x7f9e3c071a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 --2- 172.21.15.32:0/847564665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0b3e90 0x7f9e3c071a00 secure :-1 s=READY pgs=2611 cs=0 l=1 rev1=1 crypto rx=0x7f9e34009e50 tx=0x7f9e3401c9e0 comp rx=0 tx=0).stop 2023-12-20T06:04:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 -- 172.21.15.32:0/847564665 shutdown_connections 2023-12-20T06:04:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 --2- 172.21.15.32:0/847564665 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0b3e90 0x7f9e3c071a00 unknown :-1 s=CLOSED pgs=2611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 --2- 172.21.15.32:0/847564665 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e3c159ab0 0x7f9e3c0714c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:44.162 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 -- 172.21.15.32:0/847564665 >> 172.21.15.32:0/847564665 conn(0x7f9e3c06c060 msgr2=0x7f9e3c06c470 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 -- 172.21.15.32:0/847564665 shutdown_connections 2023-12-20T06:04:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 -- 172.21.15.32:0/847564665 wait complete. 2023-12-20T06:04:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 Processor -- start 2023-12-20T06:04:44.163 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 -- start start 2023-12-20T06:04:44.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e3c0b3e90 0x7f9e3c158ec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:44.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0cd2e0 0x7f9e3c1661f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:44.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e3c159ab0 0x7f9e3c0c5640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:44.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0c81c0 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0c84a0 con 0x7f9e3c0b3e90 2023-12-20T06:04:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e413a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9e3c0c8780 con 0x7f9e3c159ab0 2023-12-20T06:04:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e3b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0cd2e0 0x7f9e3c1661f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e3b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0cd2e0 0x7f9e3c1661f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51348/0 (socket says 172.21.15.32:51348) 2023-12-20T06:04:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.161+0000 7f9e3b7fe640 1 -- 172.21.15.32:0/573893018 learned_addr learned my addr 172.21.15.32:0/573893018 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e3bfff640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e3c0b3e90 0x7f9e3c158ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e40ba1640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e3c159ab0 0x7f9e3c0c5640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e3b7fe640 1 -- 172.21.15.32:0/573893018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e3c159ab0 msgr2=0x7f9e3c0c5640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e3b7fe640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e3c159ab0 0x7f9e3c0c5640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:44.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e3b7fe640 1 -- 172.21.15.32:0/573893018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e3c0b3e90 msgr2=0x7f9e3c158ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:44.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e3b7fe640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e3c0b3e90 0x7f9e3c158ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:44.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e3b7fe640 1 -- 172.21.15.32:0/573893018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9e34009b30 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e40ba1640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e3c159ab0 0x7f9e3c0c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:44.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e3bfff640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e3c0b3e90 0x7f9e3c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:44.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e3b7fe640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0cd2e0 0x7f9e3c1661f0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c3c0 tx=0x7f9e3000c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:44.168 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e38ff9640 1 -- 172.21.15.32:0/573893018 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e300052d0 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e38ff9640 1 -- 172.21.15.32:0/573893018 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9e30005430 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e413a2640 1 -- 172.21.15.32:0/573893018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9e3c0c8a60 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e38ff9640 1 -- 172.21.15.32:0/573893018 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9e30003840 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e413a2640 1 -- 172.21.15.32:0/573893018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9e3c0c1c10 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e413a2640 1 -- 172.21.15.32:0/573893018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9e3c074100 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e38ff9640 1 -- 172.21.15.32:0/573893018 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9e300039a0 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e38ff9640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9e28065bb0 0x7f9e28068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e38ff9640 1 -- 172.21.15.32:0/573893018 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f9e30082810 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.165+0000 7f9e3bfff640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9e28065bb0 0x7f9e28068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:44.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.169+0000 7f9e3bfff640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9e28065bb0 0x7f9e28068070 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0b0f70 tx=0x7f9e2c008040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:44.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.169+0000 7f9e38ff9640 1 -- 172.21.15.32:0/573893018 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9e300500c0 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.331 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.329+0000 7f9e413a2640 1 -- 172.21.15.32:0/573893018 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) v1 -- 0x7f9e3c0c8300 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.329+0000 7f9e38ff9640 1 -- 172.21.15.32:0/573893018 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v256) v1 ==== 74+0+13556 (secure 0 0 0) 0x7f9e3c0c8300 con 0x7f9e3c0cd2e0 2023-12-20T06:04:44.332 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:44.332 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":256,"fsid":"9b496156-2b6f-4897-9af4-fbc686ad55e7","created":"2023-12-20T04:28:12.897530+0000","modified":"2023-12-20T06:04:39.804945+0000","last_up_change":"2023-12-20T04:28:24.520651+0000","last_in_change":"2023-12-20T04:28:15.484604+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":33,"max_osd":8,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"reef","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2023-12-20T04:28:26.187001+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_acting":8,"score_stable":8,"optimal_score":0.25,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"6c8ebcf5-82d4-4280-9a4d-91671cb1b199","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6808","nonce":38080},{"type":"v1","addr":"172.21.15.32:6809","nonce":38080}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6810","nonce":38080},{"type":"v1","addr":"172.21.15.32:6811","nonce":38080}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6814","nonce":38080},{"type":"v1","addr":"172.21.15.32:6815","nonce":38080}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6812","nonce":38080},{"type":"v1","addr":"172.21.15.32:6813","nonce":38080}]},"public_addr":"172.21.15.32:6809/38080","cluster_addr":"172.21.15.32:6811/38080","heartbeat_back_addr":"172.21.15.32:6815/38080","heartbeat_front_addr":"172.21.15.32:6813/38080","state":["exists","up"]},{"osd":1,"uuid":"14ba777b-bae8-4e35-b63d-eafde55014d1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6800","nonce":38077},{"type":"v1","addr":"172.21.15.32:6801","nonce":38077}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6802","nonce":38077},{"type":"v1","addr":"172.21.15.32:6803","nonce":38077}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6806","nonce":38077},{"type":"v1","addr":"172.21.15.32:6807","nonce":38077}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6804","nonce":38077},{"type":"v1","addr":"172.21.15.32:6805","nonce":38077}]},"public_addr":"172.21.15.32:6801/38077","cluster_addr":"172.21.15.32:6803/38077","heartbeat_back_addr":"172.21.15.32:6807/38077","heartbeat_front_addr":"172.21.15.32:6805/38077","state":["exists","up"]},{"osd":2,"uuid":"b5f84b62-e85d-43c1-ba26-54638415dfc3","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6818","nonce":38082},{"type":"v1","addr":"172.21.15.32:6820","nonce":38082}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6822","nonce":38082},{"type":"v1","addr":"172.21.15.32:6824","nonce":38082}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6830","nonce":38082},{"type":"v1","addr":"172.21.15.32:6831","nonce":38082}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6826","nonce":38082},{"type":"v1","addr":"172.21.15.32:6828","nonce":38082}]},"public_addr":"172.21.15.32:6820/38082","cluster_addr":"172.21.15.32:6824/38082","heartbeat_back_addr":"172.21.15.32:6831/38082","heartbeat_front_addr":"172.21.15.32:6828/38082","state":["exists","up"]},{"osd":3,"uuid":"b654e597-0ae4-42f3-a726-adcbd8c6d9a1","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6816","nonce":38083},{"type":"v1","addr":"172.21.15.32:6817","nonce":38083}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6819","nonce":38083},{"type":"v1","addr":"172.21.15.32:6821","nonce":38083}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6827","nonce":38083},{"type":"v1","addr":"172.21.15.32:6829","nonce":38083}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6823","nonce":38083},{"type":"v1","addr":"172.21.15.32:6825","nonce":38083}]},"public_addr":"172.21.15.32:6817/38083","cluster_addr":"172.21.15.32:6821/38083","heartbeat_back_addr":"172.21.15.32:6829/38083","heartbeat_front_addr":"172.21.15.32:6825/38083","state":["exists","up"]},{"osd":4,"uuid":"90de0eda-35f4-45e0-b4ac-7d603612c980","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6800","nonce":38278},{"type":"v1","addr":"172.21.15.39:6801","nonce":38278}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6802","nonce":38278},{"type":"v1","addr":"172.21.15.39:6803","nonce":38278}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6806","nonce":38278},{"type":"v1","addr":"172.21.15.39:6807","nonce":38278}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6804","nonce":38278},{"type":"v1","addr":"172.21.15.39:6805","nonce":38278}]},"public_addr":"172.21.15.39:6801/38278","cluster_addr":"172.21.15.39:6803/38278","heartbeat_back_addr":"172.21.15.39:6807/38278","heartbeat_front_addr":"172.21.15.39:6805/38278","state":["exists","up"]},{"osd":5,"uuid":"81996767-dac0-469c-b381-07c0d34da475","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6824","nonce":38280},{"type":"v1","addr":"172.21.15.39:6825","nonce":38280}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6826","nonce":38280},{"type":"v1","addr":"172.21.15.39:6827","nonce":38280}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6830","nonce":38280},{"type":"v1","addr":"172.21.15.39:6831","nonce":38280}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6828","nonce":38280},{"type":"v1","addr":"172.21.15.39:6829","nonce":38280}]},"public_addr":"172.21.15.39:6825/38280","cluster_addr":"172.21.15.39:6827/38280","heartbeat_back_addr":"172.21.15.39:6831/38280","heartbeat_front_addr":"172.21.15.39:6829/38280","state":["exists","up"]},{"osd":6,"uuid":"e93092a0-e6d4-4ba7-97bd-50eab2ed529e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6816","nonce":38281},{"type":"v1","addr":"172.21.15.39:6817","nonce":38281}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6818","nonce":38281},{"type":"v1","addr":"172.21.15.39:6819","nonce":38281}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6822","nonce":38281},{"type":"v1","addr":"172.21.15.39:6823","nonce":38281}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6820","nonce":38281},{"type":"v1","addr":"172.21.15.39:6821","nonce":38281}]},"public_addr":"172.21.15.39:6817/38281","cluster_addr":"172.21.15.39:6819/38281","heartbeat_back_addr":"172.21.15.39:6823/38281","heartbeat_front_addr":"172.21.15.39:6821/38281","state":["exists","up"]},{"osd":7,"uuid":"215a2838-2ba5-49d3-a86c-5ed5dd346fd5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":249,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6808","nonce":38279},{"type":"v1","addr":"172.21.15.39:6809","nonce":38279}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6810","nonce":38279},{"type":"v1","addr":"172.21.15.39:6811","nonce":38279}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6814","nonce":38279},{"type":"v1","addr":"172.21.15.39:6815","nonce":38279}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6812","nonce":38279},{"type":"v1","addr":"172.21.15.39:6813","nonce":38279}]},"public_addr":"172.21.15.39:6809/38279","cluster_addr":"172.21.15.39:6811/38279","heartbeat_back_addr":"172.21.15.39:6815/38279","heartbeat_front_addr":"172.21.15.39:6813/38279","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.098547+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.129333+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.265240+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.220908+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.147380+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.241836+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:21.211804+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138322906710015,"old_weight":0,"last_purged_snaps_scrub":"2023-12-20T04:28:20.932974+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.32:6837/3133738681":"2023-12-21T06:04:36.776416+0000","172.21.15.32:6835/3133738681":"2023-12-21T06:04:36.776416+0000","172.21.15.32:0/1848378616":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/3742412743":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/1548603036":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/2131291246":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/2118335854":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/1269951230":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2916844765":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2625213384":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/375930125":"2023-12-21T06:04:34.642452+0000","172.21.15.32:0/2535253896":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/766391941":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/3950316648":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/2606010957":"2023-12-21T06:04:27.557099+0000","172.21.15.32:0/3612789559":"2023-12-21T06:04:27.557099+0000","172.21.15.32:6833/64546":"2023-12-21T06:04:27.557099+0000","172.21.15.32:6832/64546":"2023-12-21T06:04:27.557099+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":15,"snaps":[{"begin":2,"length":2}]},{"pool":21,"snaps":[{"begin":2,"length":32}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2023-12-20T06:04:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 -- 172.21.15.32:0/573893018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9e28065bb0 msgr2=0x7f9e28068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9e28065bb0 0x7f9e28068070 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0b0f70 tx=0x7f9e2c008040 comp rx=0 tx=0).stop 2023-12-20T06:04:44.334 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 -- 172.21.15.32:0/573893018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0cd2e0 msgr2=0x7f9e3c1661f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0cd2e0 0x7f9e3c1661f0 secure :-1 s=READY pgs=2612 cs=0 l=1 rev1=1 crypto rx=0x7f9e3000c3c0 tx=0x7f9e3000c880 comp rx=0 tx=0).stop 2023-12-20T06:04:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 -- 172.21.15.32:0/573893018 shutdown_connections 2023-12-20T06:04:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9e28065bb0 0x7f9e28068070 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9e3c159ab0 0x7f9e3c0c5640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9e3c0cd2e0 0x7f9e3c1661f0 unknown :-1 s=CLOSED pgs=2612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 --2- 172.21.15.32:0/573893018 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9e3c0b3e90 0x7f9e3c158ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 -- 172.21.15.32:0/573893018 >> 172.21.15.32:0/573893018 conn(0x7f9e3c06c060 msgr2=0x7f9e3c0b13c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 -- 172.21.15.32:0/573893018 shutdown_connections 2023-12-20T06:04:44.335 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:44.333+0000 7f9e413a2640 1 -- 172.21.15.32:0/573893018 wait complete. 2023-12-20T06:04:45.355 INFO:tasks.ceph:Scrubbing osd.0 2023-12-20T06:04:45.355 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2023-12-20T06:04:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.513+0000 7f797d6c4640 1 Processor -- start 2023-12-20T06:04:45.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.513+0000 7f797d6c4640 1 -- start start 2023-12-20T06:04:45.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f797d6c4640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7978158520 0x7f797806dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f797d6c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f79780ca050 0x7f797815ef80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f797d6c4640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f79780ca7f0 con 0x7f7978070230 2023-12-20T06:04:45.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f797d6c4640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f79780caad0 con 0x7f7978158520 2023-12-20T06:04:45.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f797d6c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79780cadb0 con 0x7f79780ca050 2023-12-20T06:04:45.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f7976ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7978158520 0x7f797806dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f7976ffd640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7978158520 0x7f797806dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51354/0 (socket says 172.21.15.32:51354) 2023-12-20T06:04:45.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f7976ffd640 1 -- 172.21.15.32:0/3280913530 learned_addr learned my addr 172.21.15.32:0/3280913530 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:45.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f796ffff640 1 --2- 172.21.15.32:0/3280913530 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f79780ca050 0x7f797815ef80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f7976ffd640 1 -- 172.21.15.32:0/3280913530 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f79780ca050 msgr2=0x7f797815ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:45.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f7976ffd640 1 --2- 172.21.15.32:0/3280913530 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f79780ca050 0x7f797815ef80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f7976ffd640 1 -- 172.21.15.32:0/3280913530 >> v1:172.21.15.32:6789/0 conn(0x7f7978070230 legacy=0x7f797815b0c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f7976ffd640 1 -- 172.21.15.32:0/3280913530 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f797815e780 con 0x7f7978158520 2023-12-20T06:04:45.521 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f7976ffd640 1 --2- 172.21.15.32:0/3280913530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7978158520 0x7f797806dcb0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f7968009870 tx=0x7f796801c510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8b3c93dc3f6651b server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:45.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.517+0000 7f79767fc640 1 -- 172.21.15.32:0/3280913530 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79680a95f0 con 0x7f7978158520 2023-12-20T06:04:45.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f79767fc640 1 -- 172.21.15.32:0/3280913530 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f79680a9750 con 0x7f7978158520 2023-12-20T06:04:45.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f79767fc640 1 -- 172.21.15.32:0/3280913530 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f79680a9ae0 con 0x7f7978158520 2023-12-20T06:04:45.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 -- 172.21.15.32:0/3280913530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7978158520 msgr2=0x7f797806dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 --2- 172.21.15.32:0/3280913530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7978158520 0x7f797806dcb0 secure :-1 s=READY pgs=2614 cs=0 l=1 rev1=1 crypto rx=0x7f7968009870 tx=0x7f796801c510 comp rx=0 tx=0).stop 2023-12-20T06:04:45.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 -- 172.21.15.32:0/3280913530 shutdown_connections 2023-12-20T06:04:45.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 --2- 172.21.15.32:0/3280913530 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f79780ca050 0x7f797815ef80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 --2- 172.21.15.32:0/3280913530 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7978158520 0x7f797806dcb0 unknown :-1 s=CLOSED pgs=2614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 -- 172.21.15.32:0/3280913530 >> 172.21.15.32:0/3280913530 conn(0x7f797806a510 msgr2=0x7f79780ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:45.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 -- 172.21.15.32:0/3280913530 shutdown_connections 2023-12-20T06:04:45.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 -- 172.21.15.32:0/3280913530 wait complete. 2023-12-20T06:04:45.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 Processor -- start 2023-12-20T06:04:45.523 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 -- start start 2023-12-20T06:04:45.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7978070230 0x7f79781f5800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79780ca050 0x7f7978153180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7978158520 0x7f79780b2a00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7978153a50 con 0x7f79780ca050 2023-12-20T06:04:45.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f79781f6030 con 0x7f7978158520 2023-12-20T06:04:45.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f797d6c4640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f79781f6310 con 0x7f7978070230 2023-12-20T06:04:45.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f796ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79780ca050 0x7f7978153180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.526 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f796ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79780ca050 0x7f7978153180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51360/0 (socket says 172.21.15.32:51360) 2023-12-20T06:04:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.521+0000 7f796ffff640 1 -- 172.21.15.32:0/371732286 learned_addr learned my addr 172.21.15.32:0/371732286 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f7976ffd640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7978070230 0x7f79781f5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f79777fe640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7978158520 0x7f79780b2a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f796ffff640 1 -- 172.21.15.32:0/371732286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7978070230 msgr2=0x7f79781f5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f796ffff640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7978070230 0x7f79781f5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f796ffff640 1 -- 172.21.15.32:0/371732286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7978158520 msgr2=0x7f79780b2a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f796ffff640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7978158520 0x7f79780b2a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f796ffff640 1 -- 172.21.15.32:0/371732286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f79680094d0 con 0x7f79780ca050 2023-12-20T06:04:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f7976ffd640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7978070230 0x7f79781f5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:45.527 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f79777fe640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7978158520 0x7f79780b2a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:45.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f796ffff640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79780ca050 0x7f7978153180 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f796000d800 tx=0x7f796000dcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:45.528 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f796f7fe640 1 -- 172.21.15.32:0/371732286 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796000b6c0 con 0x7f79780ca050 2023-12-20T06:04:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f796f7fe640 1 -- 172.21.15.32:0/371732286 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f796000b820 con 0x7f79780ca050 2023-12-20T06:04:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f796f7fe640 1 -- 172.21.15.32:0/371732286 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f796000bb90 con 0x7f79780ca050 2023-12-20T06:04:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f797d6c4640 1 -- 172.21.15.32:0/371732286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f79781f65f0 con 0x7f79780ca050 2023-12-20T06:04:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.525+0000 7f797d6c4640 1 -- 172.21.15.32:0/371732286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f79781f5d40 con 0x7f79780ca050 2023-12-20T06:04:45.529 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.529+0000 7f797d6c4640 1 -- 172.21.15.32:0/371732286 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f79780ca460 con 0x7f79780ca050 2023-12-20T06:04:45.530 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.529+0000 7f796f7fe640 1 -- 172.21.15.32:0/371732286 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f796000bcf0 con 0x7f79780ca050 2023-12-20T06:04:45.531 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.529+0000 7f796f7fe640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7958065b80 0x7f7958068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.529+0000 7f796f7fe640 1 -- 172.21.15.32:0/371732286 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f7960083800 con 0x7f79780ca050 2023-12-20T06:04:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.529+0000 7f7976ffd640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7958065b80 0x7f7958068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.529+0000 7f796f7fe640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] conn(0x7f795806bfd0 0x7f795806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.529+0000 7f796f7fe640 1 -- 172.21.15.32:0/371732286 --> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f795806eae0 con 0x7f795806bfd0 2023-12-20T06:04:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.529+0000 7f796f7fe640 1 -- 172.21.15.32:0/371732286 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=256) v2 ==== 24+0+0 (secure 0 0 0) 0x7f79600882c0 con 0x7f79780ca050 2023-12-20T06:04:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.529+0000 7f79777fe640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] conn(0x7f795806bfd0 0x7f795806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.532 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.529+0000 7f7976ffd640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7958065b80 0x7f7958068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f79680ae000 tx=0x7f79680b3040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.529+0000 7f79777fe640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] conn(0x7f795806bfd0 0x7f795806e410 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:45.533 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.533+0000 7f796f7fe640 1 -- 172.21.15.32:0/371732286 <== osd.0 v2:172.21.15.32:6808/38080 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f795806eae0 con 0x7f795806bfd0 2023-12-20T06:04:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f797d6c4640 1 -- 172.21.15.32:0/371732286 --> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f797815bbe0 con 0x7f795806bfd0 2023-12-20T06:04:45.546 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f796f7fe640 1 -- 172.21.15.32:0/371732286 <== osd.0 v2:172.21.15.32:6808/38080 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7970001f40 con 0x7f795806bfd0 2023-12-20T06:04:45.546 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:04:45.546 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-20T06:04:45.547 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:04:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f797d6c4640 1 -- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] conn(0x7f795806bfd0 msgr2=0x7f795806e410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f797d6c4640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:6808/38080,v1:172.21.15.32:6809/38080] conn(0x7f795806bfd0 0x7f795806e410 crc :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f797d6c4640 1 -- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7958065b80 msgr2=0x7f7958068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f797d6c4640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7958065b80 0x7f7958068040 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f79680ae000 tx=0x7f79680b3040 comp rx=0 tx=0).stop 2023-12-20T06:04:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f797d6c4640 1 -- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79780ca050 msgr2=0x7f7978153180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f797d6c4640 1 --2- 172.21.15.32:0/371732286 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f79780ca050 0x7f7978153180 secure :-1 s=READY pgs=2615 cs=0 l=1 rev1=1 crypto rx=0x7f796000d800 tx=0x7f796000dcc0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f79777fe640 1 -- 172.21.15.32:0/371732286 reap_dead start 2023-12-20T06:04:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f797d6c4640 1 -- 172.21.15.32:0/371732286 shutdown_connections 2023-12-20T06:04:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f797d6c4640 1 -- 172.21.15.32:0/371732286 >> 172.21.15.32:0/371732286 conn(0x7f797806a510 msgr2=0x7f79780b0a80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:45.548 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f797d6c4640 1 -- 172.21.15.32:0/371732286 shutdown_connections 2023-12-20T06:04:45.549 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.545+0000 7f797d6c4640 1 -- 172.21.15.32:0/371732286 wait complete. 2023-12-20T06:04:45.565 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2023-12-20T06:04:45.710 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf3b0b6640 1 Processor -- start 2023-12-20T06:04:45.711 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf3b0b6640 1 -- start start 2023-12-20T06:04:45.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf3b0b6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fcf3415cdb0 0x7fcf340d0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.712 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf3b0b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf340d2740 0x7fcf340cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf3b0b6640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fcf340b6ca0 con 0x7fcf340d1120 2023-12-20T06:04:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf3b0b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf3415d190 con 0x7fcf340d2740 2023-12-20T06:04:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf3b0b6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fcf340d16a0 con 0x7fcf3415cdb0 2023-12-20T06:04:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf2bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf340d2740 0x7fcf340cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf2bfff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fcf340d2740 0x7fcf340cf570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51362/0 (socket says 172.21.15.32:51362) 2023-12-20T06:04:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf2bfff640 1 -- 172.21.15.32:0/2237873087 learned_addr learned my addr 172.21.15.32:0/2237873087 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf38e52640 1 --2- 172.21.15.32:0/2237873087 >> v2:172.21.15.39:3300/0 conn(0x7fcf3415cdb0 0x7fcf340d0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.713 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf2bfff640 1 -- 172.21.15.32:0/2237873087 >> v2:172.21.15.39:3300/0 conn(0x7fcf3415cdb0 msgr2=0x7fcf340d0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf2bfff640 1 --2- 172.21.15.32:0/2237873087 >> v2:172.21.15.39:3300/0 conn(0x7fcf3415cdb0 0x7fcf340d0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf2bfff640 1 -- 172.21.15.32:0/2237873087 >> v1:172.21.15.39:6789/0 conn(0x7fcf340d1120 legacy=0x7fcf340b45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:45.714 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.709+0000 7fcf2bfff640 1 -- 172.21.15.32:0/2237873087 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf340d1980 con 0x7fcf340d2740 2023-12-20T06:04:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf2bfff640 1 --2- 172.21.15.32:0/2237873087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf340d2740 0x7fcf340cf570 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fcf340667b0 tx=0x7fcf2401ecf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=65f5ad1730987e7a server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:45.715 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf2b7fe640 1 -- 172.21.15.32:0/2237873087 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf240a89f0 con 0x7fcf340d2740 2023-12-20T06:04:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf2b7fe640 1 -- 172.21.15.32:0/2237873087 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf240a8b50 con 0x7fcf340d2740 2023-12-20T06:04:45.716 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf2b7fe640 1 -- 172.21.15.32:0/2237873087 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf240b0440 con 0x7fcf340d2740 2023-12-20T06:04:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/2237873087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf340d2740 msgr2=0x7fcf340cf570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf3b0b6640 1 --2- 172.21.15.32:0/2237873087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf340d2740 0x7fcf340cf570 secure :-1 s=READY pgs=2616 cs=0 l=1 rev1=1 crypto rx=0x7fcf340667b0 tx=0x7fcf2401ecf0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/2237873087 shutdown_connections 2023-12-20T06:04:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf3b0b6640 1 --2- 172.21.15.32:0/2237873087 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf340d2740 0x7fcf340cf570 unknown :-1 s=CLOSED pgs=2616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf3b0b6640 1 --2- 172.21.15.32:0/2237873087 >> v2:172.21.15.39:3300/0 conn(0x7fcf3415cdb0 0x7fcf340d0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/2237873087 >> 172.21.15.32:0/2237873087 conn(0x7fcf34065ee0 msgr2=0x7fcf3406b0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/2237873087 shutdown_connections 2023-12-20T06:04:45.717 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.713+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/2237873087 wait complete. 2023-12-20T06:04:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf3b0b6640 1 Processor -- start 2023-12-20T06:04:45.718 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf3b0b6640 1 -- start start 2023-12-20T06:04:45.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf3b0b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf340d1120 0x7fcf3415e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.720 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf3b0b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf340d2740 0x7fcf3414e740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf3b0b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf3415cdb0 0x7fcf3414d120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf3b0b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3414ec80 con 0x7fcf3415cdb0 2023-12-20T06:04:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf3b0b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3414d660 con 0x7fcf340d2740 2023-12-20T06:04:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf3b0b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fcf3415d8f0 con 0x7fcf340d1120 2023-12-20T06:04:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf39653640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf3415cdb0 0x7fcf3414d120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf39653640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf3415cdb0 0x7fcf3414d120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51366/0 (socket says 172.21.15.32:51366) 2023-12-20T06:04:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf38e52640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf340d1120 0x7fcf3415e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf39653640 1 -- 172.21.15.32:0/1861792650 learned_addr learned my addr 172.21.15.32:0/1861792650 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf2bfff640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf340d2740 0x7fcf3414e740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.721 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf39653640 1 -- 172.21.15.32:0/1861792650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf340d1120 msgr2=0x7fcf3415e990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf39653640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf340d1120 0x7fcf3415e990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf39653640 1 -- 172.21.15.32:0/1861792650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf340d2740 msgr2=0x7fcf3414e740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf39653640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf340d2740 0x7fcf3414e740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf39653640 1 -- 172.21.15.32:0/1861792650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fcf240094d0 con 0x7fcf3415cdb0 2023-12-20T06:04:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf39653640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf3415cdb0 0x7fcf3414d120 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c770 tx=0x7fcf2c00cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf297fa640 1 -- 172.21.15.32:0/1861792650 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c003b80 con 0x7fcf3415cdb0 2023-12-20T06:04:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf297fa640 1 -- 172.21.15.32:0/1861792650 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fcf2c003ce0 con 0x7fcf3415cdb0 2023-12-20T06:04:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf297fa640 1 -- 172.21.15.32:0/1861792650 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fcf2c00aa90 con 0x7fcf3415cdb0 2023-12-20T06:04:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/1861792650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fcf3415dbd0 con 0x7fcf3415cdb0 2023-12-20T06:04:45.722 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.717+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/1861792650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fcf3415e190 con 0x7fcf3415cdb0 2023-12-20T06:04:45.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.721+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/1861792650 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fcf3404e000 con 0x7fcf3415cdb0 2023-12-20T06:04:45.723 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.721+0000 7fcf297fa640 1 -- 172.21.15.32:0/1861792650 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fcf2c00abf0 con 0x7fcf3415cdb0 2023-12-20T06:04:45.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.721+0000 7fcf297fa640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf20065b80 0x7fcf20068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:45.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.721+0000 7fcf297fa640 1 -- 172.21.15.32:0/1861792650 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7fcf2c082ef0 con 0x7fcf3415cdb0 2023-12-20T06:04:45.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.721+0000 7fcf38e52640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf20065b80 0x7fcf20068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:45.724 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.721+0000 7fcf38e52640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf20065b80 0x7fcf20068040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcf3414e6f0 tx=0x7fcf30008040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:45.727 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.725+0000 7fcf297fa640 1 -- 172.21.15.32:0/1861792650 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fcf2c0507c0 con 0x7fcf3415cdb0 2023-12-20T06:04:45.885 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/1861792650 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "0", "target": ["mon-mgr", ""]}) v1 -- 0x7fcf340d1f40 con 0x7fcf20065b80 2023-12-20T06:04:45.886 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf297fa640 1 -- 172.21.15.32:0/1861792650 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 0 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fcf30004020 con 0x7fcf20065b80 2023-12-20T06:04:45.886 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 0 to deep-scrub 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/1861792650 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf20065b80 msgr2=0x7fcf20068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf20065b80 0x7fcf20068040 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fcf3414e6f0 tx=0x7fcf30008040 comp rx=0 tx=0).stop 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/1861792650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf3415cdb0 msgr2=0x7fcf3414d120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf3415cdb0 0x7fcf3414d120 secure :-1 s=READY pgs=2617 cs=0 l=1 rev1=1 crypto rx=0x7fcf2c00c770 tx=0x7fcf2c00cc30 comp rx=0 tx=0).stop 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/1861792650 shutdown_connections 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fcf20065b80 0x7fcf20068040 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fcf3415cdb0 0x7fcf3414d120 unknown :-1 s=CLOSED pgs=2617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fcf340d2740 0x7fcf3414e740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 --2- 172.21.15.32:0/1861792650 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fcf340d1120 0x7fcf3415e990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/1861792650 >> 172.21.15.32:0/1861792650 conn(0x7fcf34065ee0 msgr2=0x7fcf3415bfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/1861792650 shutdown_connections 2023-12-20T06:04:45.888 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:45.885+0000 7fcf3b0b6640 1 -- 172.21.15.32:0/1861792650 wait complete. 2023-12-20T06:04:45.907 INFO:tasks.ceph:Scrubbing osd.1 2023-12-20T06:04:45.907 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2023-12-20T06:04:46.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f71a2909640 1 Processor -- start 2023-12-20T06:04:46.054 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f71a2909640 1 -- start start 2023-12-20T06:04:46.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f71a2909640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f719c0ce290 0x7f719c070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f71a2909640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c159630 con 0x7f719c070770 2023-12-20T06:04:46.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f71a2909640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f719c0c9480 con 0x7f719c159250 2023-12-20T06:04:46.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f71a2909640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f719c070ba0 con 0x7f719c0ce290 2023-12-20T06:04:46.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f71a0ea6640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f719c070770 0x7f719c0cc5b0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40796/0 (socket says 172.21.15.32:40796) 2023-12-20T06:04:46.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f71a0ea6640 1 -- 172.21.15.32:0/4048972252 learned_addr learned my addr 172.21.15.32:0/4048972252 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:46.056 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f719b7fe640 1 --2- 172.21.15.32:0/4048972252 >> v2:172.21.15.39:3300/0 conn(0x7f719c0ce290 0x7f719c070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f719affd640 1 -- 172.21.15.32:0/4048972252 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2315920280 0 0) 0x7f719c159630 con 0x7f719c070770 2023-12-20T06:04:46.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f719affd640 1 -- 172.21.15.32:0/4048972252 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f7184003450 con 0x7f719c070770 2023-12-20T06:04:46.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f719affd640 1 -- 172.21.15.32:0/4048972252 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1670754736 0 0) 0x7f719c0c9480 con 0x7f719c159250 2023-12-20T06:04:46.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f719affd640 1 -- 172.21.15.32:0/4048972252 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f719c159630 con 0x7f719c159250 2023-12-20T06:04:46.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f719b7fe640 1 -- 172.21.15.32:0/4048972252 >> v1:172.21.15.39:6789/0 conn(0x7f719c159250 legacy=0x7f719c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f719b7fe640 1 -- 172.21.15.32:0/4048972252 >> v1:172.21.15.32:6789/0 conn(0x7f719c070770 legacy=0x7f719c0cc5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.057 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.053+0000 7f719b7fe640 1 -- 172.21.15.32:0/4048972252 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f719c06cf90 con 0x7f719c0ce290 2023-12-20T06:04:46.059 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.057+0000 7f719b7fe640 1 --2- 172.21.15.32:0/4048972252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f719c0ce290 0x7f719c070230 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f71900097c0 tx=0x7f719001c480 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c9faa9f7a77227b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.057+0000 7f719affd640 1 -- 172.21.15.32:0/4048972252 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71900ab600 con 0x7f719c0ce290 2023-12-20T06:04:46.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.057+0000 7f719affd640 1 -- 172.21.15.32:0/4048972252 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f71900ab760 con 0x7f719c0ce290 2023-12-20T06:04:46.060 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.057+0000 7f719affd640 1 -- 172.21.15.32:0/4048972252 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f71900abaf0 con 0x7f719c0ce290 2023-12-20T06:04:46.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.057+0000 7f71a2909640 1 -- 172.21.15.32:0/4048972252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f719c0ce290 msgr2=0x7f719c070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.057+0000 7f71a2909640 1 --2- 172.21.15.32:0/4048972252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f719c0ce290 0x7f719c070230 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7f71900097c0 tx=0x7f719001c480 comp rx=0 tx=0).stop 2023-12-20T06:04:46.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 -- 172.21.15.32:0/4048972252 shutdown_connections 2023-12-20T06:04:46.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 --2- 172.21.15.32:0/4048972252 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f719c0ce290 0x7f719c070230 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 -- 172.21.15.32:0/4048972252 >> 172.21.15.32:0/4048972252 conn(0x7f719c06a510 msgr2=0x7f719c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:46.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 -- 172.21.15.32:0/4048972252 shutdown_connections 2023-12-20T06:04:46.061 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 -- 172.21.15.32:0/4048972252 wait complete. 2023-12-20T06:04:46.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 Processor -- start 2023-12-20T06:04:46.062 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 -- start start 2023-12-20T06:04:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f719c070770 0x7f719c149180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f719c0ce290 0x7f719c0b3f90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f719c159250 0x7f719c145460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c14ac00 con 0x7f719c159250 2023-12-20T06:04:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f719c0b44d0 con 0x7f719c0ce290 2023-12-20T06:04:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a2909640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f719c14ae10 con 0x7f719c070770 2023-12-20T06:04:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a0ea6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f719c159250 0x7f719c145460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a0ea6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f719c159250 0x7f719c145460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51372/0 (socket says 172.21.15.32:51372) 2023-12-20T06:04:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a0ea6640 1 -- 172.21.15.32:0/3572044020 learned_addr learned my addr 172.21.15.32:0/3572044020 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f719b7fe640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f719c0ce290 0x7f719c0b3f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.064 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f719bfff640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f719c070770 0x7f719c149180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.065 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a0ea6640 1 -- 172.21.15.32:0/3572044020 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f719c070770 msgr2=0x7f719c149180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a0ea6640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f719c070770 0x7f719c149180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a0ea6640 1 -- 172.21.15.32:0/3572044020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f719c0ce290 msgr2=0x7f719c0b3f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a0ea6640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f719c0ce290 0x7f719c0b3f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.061+0000 7f71a0ea6640 1 -- 172.21.15.32:0/3572044020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7190009420 con 0x7f719c159250 2023-12-20T06:04:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f71a0ea6640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f719c159250 0x7f719c145460 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f719400cb40 tx=0x7f719400a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f7198ff9640 1 -- 172.21.15.32:0/3572044020 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f719400aca0 con 0x7f719c159250 2023-12-20T06:04:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f7198ff9640 1 -- 172.21.15.32:0/3572044020 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f719400ae00 con 0x7f719c159250 2023-12-20T06:04:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f7198ff9640 1 -- 172.21.15.32:0/3572044020 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7194012600 con 0x7f719c159250 2023-12-20T06:04:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f71a2909640 1 -- 172.21.15.32:0/3572044020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f719c14b0f0 con 0x7f719c159250 2023-12-20T06:04:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f71a2909640 1 -- 172.21.15.32:0/3572044020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f719c14b6b0 con 0x7f719c159250 2023-12-20T06:04:46.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f71a2909640 1 -- 172.21.15.32:0/3572044020 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f719c0c9110 con 0x7f719c159250 2023-12-20T06:04:46.067 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f7198ff9640 1 -- 172.21.15.32:0/3572044020 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7194007070 con 0x7f719c159250 2023-12-20T06:04:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f7198ff9640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7188065b80 0x7f7188068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f7198ff9640 1 -- 172.21.15.32:0/3572044020 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f71940832b0 con 0x7f719c159250 2023-12-20T06:04:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f7198ff9640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] conn(0x7f718806bfd0 0x7f718806e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f719bfff640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7188065b80 0x7f7188068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f7198ff9640 1 -- 172.21.15.32:0/3572044020 --> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f718806eae0 con 0x7f718806bfd0 2023-12-20T06:04:46.068 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f7198ff9640 1 -- 172.21.15.32:0/3572044020 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=256) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7194085890 con 0x7f719c159250 2023-12-20T06:04:46.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.065+0000 7f719b7fe640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] conn(0x7f718806bfd0 0x7f718806e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.069+0000 7f719bfff640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7188065b80 0x7f7188068040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f718c0011f0 tx=0x7f718c009040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.069+0000 7f719b7fe640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] conn(0x7f718806bfd0 0x7f718806e410 crc :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.069 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.069+0000 7f7198ff9640 1 -- 172.21.15.32:0/3572044020 <== osd.1 v2:172.21.15.32:6800/38077 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f718806eae0 con 0x7f718806bfd0 2023-12-20T06:04:46.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a2909640 1 -- 172.21.15.32:0/3572044020 --> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f719c06d820 con 0x7f718806bfd0 2023-12-20T06:04:46.082 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f7198ff9640 1 -- 172.21.15.32:0/3572044020 <== osd.1 v2:172.21.15.32:6800/38077 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f719c06d820 con 0x7f718806bfd0 2023-12-20T06:04:46.083 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:04:46.083 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-20T06:04:46.083 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:04:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a2909640 1 -- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] conn(0x7f718806bfd0 msgr2=0x7f718806e410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a2909640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:6800/38077,v1:172.21.15.32:6801/38077] conn(0x7f718806bfd0 0x7f718806e410 crc :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a2909640 1 -- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7188065b80 msgr2=0x7f7188068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a2909640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7188065b80 0x7f7188068040 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f718c0011f0 tx=0x7f718c009040 comp rx=0 tx=0).stop 2023-12-20T06:04:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a2909640 1 -- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f719c159250 msgr2=0x7f719c145460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a2909640 1 --2- 172.21.15.32:0/3572044020 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f719c159250 0x7f719c145460 secure :-1 s=READY pgs=2619 cs=0 l=1 rev1=1 crypto rx=0x7f719400cb40 tx=0x7f719400a060 comp rx=0 tx=0).stop 2023-12-20T06:04:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a0ea6640 1 -- 172.21.15.32:0/3572044020 reap_dead start 2023-12-20T06:04:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a2909640 1 -- 172.21.15.32:0/3572044020 shutdown_connections 2023-12-20T06:04:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a2909640 1 -- 172.21.15.32:0/3572044020 >> 172.21.15.32:0/3572044020 conn(0x7f719c06a510 msgr2=0x7f719c0c4a40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a2909640 1 -- 172.21.15.32:0/3572044020 shutdown_connections 2023-12-20T06:04:46.084 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.081+0000 7f71a2909640 1 -- 172.21.15.32:0/3572044020 wait complete. 2023-12-20T06:04:46.102 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2023-12-20T06:04:46.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1ce97640 1 Processor -- start 2023-12-20T06:04:46.245 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1ce97640 1 -- start start 2023-12-20T06:04:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1ce97640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c1815ce60 0x7f9c180c9ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1ce97640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9c18161240 0x7f9c18071840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1ce97640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9c18071d80 con 0x7f9c180713e0 2023-12-20T06:04:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1ce97640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f9c181616d0 con 0x7f9c1815ce60 2023-12-20T06:04:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1ce97640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c180c9a40 con 0x7f9c18161240 2023-12-20T06:04:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c1815ce60 0x7f9c180c9ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1659c640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f9c1815ce60 0x7f9c180c9ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51388/0 (socket says 172.21.15.32:51388) 2023-12-20T06:04:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1659c640 1 -- 172.21.15.32:0/4205247140 learned_addr learned my addr 172.21.15.32:0/4205247140 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:46.247 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c16d9d640 1 --2- 172.21.15.32:0/4205247140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9c18161240 0x7f9c18071840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1659c640 1 -- 172.21.15.32:0/4205247140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9c18161240 msgr2=0x7f9c18071840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1659c640 1 --2- 172.21.15.32:0/4205247140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9c18161240 0x7f9c18071840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1659c640 1 -- 172.21.15.32:0/4205247140 >> v1:172.21.15.39:6789/0 conn(0x7f9c180713e0 legacy=0x7f9c18160b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.248 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.245+0000 7f9c1659c640 1 -- 172.21.15.32:0/4205247140 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c180c9180 con 0x7f9c1815ce60 2023-12-20T06:04:46.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c1659c640 1 --2- 172.21.15.32:0/4205247140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1815ce60 0x7f9c180c9ed0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f9c08006fd0 tx=0x7f9c0801c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff42dedd81639c7f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.250 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c15d9b640 1 -- 172.21.15.32:0/4205247140 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c080ab610 con 0x7f9c1815ce60 2023-12-20T06:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c15d9b640 1 -- 172.21.15.32:0/4205247140 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c080ab770 con 0x7f9c1815ce60 2023-12-20T06:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c15d9b640 1 -- 172.21.15.32:0/4205247140 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c080abb00 con 0x7f9c1815ce60 2023-12-20T06:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c1ce97640 1 -- 172.21.15.32:0/4205247140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1815ce60 msgr2=0x7f9c180c9ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c1ce97640 1 --2- 172.21.15.32:0/4205247140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1815ce60 0x7f9c180c9ed0 secure :-1 s=READY pgs=2620 cs=0 l=1 rev1=1 crypto rx=0x7f9c08006fd0 tx=0x7f9c0801c520 comp rx=0 tx=0).stop 2023-12-20T06:04:46.251 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c1ce97640 1 -- 172.21.15.32:0/4205247140 shutdown_connections 2023-12-20T06:04:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c1ce97640 1 --2- 172.21.15.32:0/4205247140 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9c18161240 0x7f9c18071840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c1ce97640 1 --2- 172.21.15.32:0/4205247140 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1815ce60 0x7f9c180c9ed0 unknown :-1 s=CLOSED pgs=2620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c1ce97640 1 -- 172.21.15.32:0/4205247140 >> 172.21.15.32:0/4205247140 conn(0x7f9c1806a320 msgr2=0x7f9c180ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c1ce97640 1 -- 172.21.15.32:0/4205247140 shutdown_connections 2023-12-20T06:04:46.252 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.249+0000 7f9c1ce97640 1 -- 172.21.15.32:0/4205247140 wait complete. 2023-12-20T06:04:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1ce97640 1 Processor -- start 2023-12-20T06:04:46.253 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1ce97640 1 -- start start 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1ce97640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9c180713e0 0x7f9c18154870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1ce97640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1815ce60 0x7f9c18152790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1ce97640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c18161240 0x7f9c18152cd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1ce97640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1815ed30 con 0x7f9c1815ce60 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1ce97640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1815df70 con 0x7f9c18161240 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1ce97640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9c1815e250 con 0x7f9c180713e0 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1815ce60 0x7f9c18152790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0ffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1815ce60 0x7f9c18152790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51390/0 (socket says 172.21.15.32:51390) 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0ffff640 1 -- 172.21.15.32:0/1756176614 learned_addr learned my addr 172.21.15.32:0/1756176614 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1659c640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9c180713e0 0x7f9c18154870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c16d9d640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c18161240 0x7f9c18152cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0ffff640 1 -- 172.21.15.32:0/1756176614 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9c180713e0 msgr2=0x7f9c18154870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.255 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0ffff640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9c180713e0 0x7f9c18154870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0ffff640 1 -- 172.21.15.32:0/1756176614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c18161240 msgr2=0x7f9c18152cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0ffff640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c18161240 0x7f9c18152cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0ffff640 1 -- 172.21.15.32:0/1756176614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9c08009480 con 0x7f9c1815ce60 2023-12-20T06:04:46.256 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0ffff640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1815ce60 0x7f9c18152790 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000c500 tx=0x7f9c0000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0effd640 1 -- 172.21.15.32:0/1756176614 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00005220 con 0x7f9c1815ce60 2023-12-20T06:04:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0effd640 1 -- 172.21.15.32:0/1756176614 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9c00005380 con 0x7f9c1815ce60 2023-12-20T06:04:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c0effd640 1 -- 172.21.15.32:0/1756176614 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9c00007270 con 0x7f9c1815ce60 2023-12-20T06:04:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1ce97640 1 -- 172.21.15.32:0/1756176614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9c1815e530 con 0x7f9c1815ce60 2023-12-20T06:04:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1ce97640 1 -- 172.21.15.32:0/1756176614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9c18154490 con 0x7f9c1815ce60 2023-12-20T06:04:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.253+0000 7f9c1ce97640 1 -- 172.21.15.32:0/1756176614 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9c1804e000 con 0x7f9c1815ce60 2023-12-20T06:04:46.257 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.257+0000 7f9c0effd640 1 -- 172.21.15.32:0/1756176614 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9c000073d0 con 0x7f9c1815ce60 2023-12-20T06:04:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.257+0000 7f9c0effd640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bf8065b80 0x7f9bf8068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.257+0000 7f9c0effd640 1 -- 172.21.15.32:0/1756176614 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f9c00082bc0 con 0x7f9c1815ce60 2023-12-20T06:04:46.258 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.257+0000 7f9c1659c640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bf8065b80 0x7f9bf8068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.259 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.257+0000 7f9c1659c640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bf8065b80 0x7f9bf8068040 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9c080af000 tx=0x7f9c08009040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.262 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.261+0000 7f9c0effd640 1 -- 172.21.15.32:0/1756176614 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9c000504a0 con 0x7f9c1815ce60 2023-12-20T06:04:46.418 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.417+0000 7f9c1ce97640 1 -- 172.21.15.32:0/1756176614 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "1", "target": ["mon-mgr", ""]}) v1 -- 0x7f9c181618b0 con 0x7f9bf8065b80 2023-12-20T06:04:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.417+0000 7f9c0effd640 1 -- 172.21.15.32:0/1756176614 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 1 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9c080040c0 con 0x7f9bf8065b80 2023-12-20T06:04:46.419 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 1 to deep-scrub 2023-12-20T06:04:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 -- 172.21.15.32:0/1756176614 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bf8065b80 msgr2=0x7f9bf8068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bf8065b80 0x7f9bf8068040 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f9c080af000 tx=0x7f9c08009040 comp rx=0 tx=0).stop 2023-12-20T06:04:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 -- 172.21.15.32:0/1756176614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1815ce60 msgr2=0x7f9c18152790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1815ce60 0x7f9c18152790 secure :-1 s=READY pgs=2621 cs=0 l=1 rev1=1 crypto rx=0x7f9c0000c500 tx=0x7f9c0000c9c0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 -- 172.21.15.32:0/1756176614 shutdown_connections 2023-12-20T06:04:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9bf8065b80 0x7f9bf8068040 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.421 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9c18161240 0x7f9c18152cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9c1815ce60 0x7f9c18152790 unknown :-1 s=CLOSED pgs=2621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 --2- 172.21.15.32:0/1756176614 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9c180713e0 0x7f9c18154870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 -- 172.21.15.32:0/1756176614 >> 172.21.15.32:0/1756176614 conn(0x7f9c1806a320 msgr2=0x7f9c180ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 -- 172.21.15.32:0/1756176614 shutdown_connections 2023-12-20T06:04:46.422 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.421+0000 7f9c1ce97640 1 -- 172.21.15.32:0/1756176614 wait complete. 2023-12-20T06:04:46.440 INFO:tasks.ceph:Scrubbing osd.2 2023-12-20T06:04:46.440 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2023-12-20T06:04:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.585+0000 7f40dc02d640 1 Processor -- start 2023-12-20T06:04:46.588 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40dc02d640 1 -- start start 2023-12-20T06:04:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40dc02d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40d4158520 0x7f40d406dcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.590 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40dc02d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f40d4070230 0x7f40d40706a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40dc02d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f40d40ca7f0 con 0x7f40d40ca050 2023-12-20T06:04:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40dc02d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d4070be0 con 0x7f40d4158520 2023-12-20T06:04:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40dc02d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f40d40caad0 con 0x7f40d4070230 2023-12-20T06:04:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40d9dc9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40d4158520 0x7f40d406dcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40d9dc9640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f40d4158520 0x7f40d406dcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51406/0 (socket says 172.21.15.32:51406) 2023-12-20T06:04:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40d9dc9640 1 -- 172.21.15.32:0/1569281993 learned_addr learned my addr 172.21.15.32:0/1569281993 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40da5ca640 1 --2- 172.21.15.32:0/1569281993 >> v2:172.21.15.39:3300/0 conn(0x7f40d4070230 0x7f40d40706a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40d9dc9640 1 -- 172.21.15.32:0/1569281993 >> v2:172.21.15.39:3300/0 conn(0x7f40d4070230 msgr2=0x7f40d40706a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40d9dc9640 1 --2- 172.21.15.32:0/1569281993 >> v2:172.21.15.39:3300/0 conn(0x7f40d4070230 0x7f40d40706a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40d9dc9640 1 -- 172.21.15.32:0/1569281993 >> v1:172.21.15.39:6789/0 conn(0x7f40d40ca050 legacy=0x7f40d415ef80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:46.591 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.589+0000 7f40d9dc9640 1 -- 172.21.15.32:0/1569281993 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d40cadb0 con 0x7f40d4158520 2023-12-20T06:04:46.593 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40d9dc9640 1 --2- 172.21.15.32:0/1569281993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4158520 0x7f40d406dcb0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f40d40b24e0 tx=0x7f40d001c400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d4eb420beed5441 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40d8dc7640 1 -- 172.21.15.32:0/1569281993 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d00ab600 con 0x7f40d4158520 2023-12-20T06:04:46.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40d8dc7640 1 -- 172.21.15.32:0/1569281993 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40d00ab760 con 0x7f40d4158520 2023-12-20T06:04:46.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40d8dc7640 1 -- 172.21.15.32:0/1569281993 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40d00abaf0 con 0x7f40d4158520 2023-12-20T06:04:46.594 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40dc02d640 1 -- 172.21.15.32:0/1569281993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4158520 msgr2=0x7f40d406dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40dc02d640 1 --2- 172.21.15.32:0/1569281993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4158520 0x7f40d406dcb0 secure :-1 s=READY pgs=2622 cs=0 l=1 rev1=1 crypto rx=0x7f40d40b24e0 tx=0x7f40d001c400 comp rx=0 tx=0).stop 2023-12-20T06:04:46.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40dc02d640 1 -- 172.21.15.32:0/1569281993 shutdown_connections 2023-12-20T06:04:46.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40dc02d640 1 --2- 172.21.15.32:0/1569281993 >> v2:172.21.15.39:3300/0 conn(0x7f40d4070230 0x7f40d40706a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40dc02d640 1 --2- 172.21.15.32:0/1569281993 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4158520 0x7f40d406dcb0 unknown :-1 s=CLOSED pgs=2622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40dc02d640 1 -- 172.21.15.32:0/1569281993 >> 172.21.15.32:0/1569281993 conn(0x7f40d406a510 msgr2=0x7f40d40ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:46.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40dc02d640 1 -- 172.21.15.32:0/1569281993 shutdown_connections 2023-12-20T06:04:46.595 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40dc02d640 1 -- 172.21.15.32:0/1569281993 wait complete. 2023-12-20T06:04:46.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40dc02d640 1 Processor -- start 2023-12-20T06:04:46.596 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.593+0000 7f40dc02d640 1 -- start start 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40dc02d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40d4070230 0x7f40d415c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40dc02d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40d40ca050 0x7f40d415cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40dc02d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4158520 0x7f40d41f2dc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40dc02d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d415d5c0 con 0x7f40d4158520 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40dc02d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f40d41532e0 con 0x7f40d4070230 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40dc02d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f40d41535c0 con 0x7f40d40ca050 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40da5ca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4158520 0x7f40d41f2dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40d9dc9640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40d4070230 0x7f40d415c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40da5ca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4158520 0x7f40d41f2dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51410/0 (socket says 172.21.15.32:51410) 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40da5ca640 1 -- 172.21.15.32:0/1339966101 learned_addr learned my addr 172.21.15.32:0/1339966101 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40d95c8640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40d40ca050 0x7f40d415cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.598 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40da5ca640 1 -- 172.21.15.32:0/1339966101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40d40ca050 msgr2=0x7f40d415cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40da5ca640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f40d40ca050 0x7f40d415cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40da5ca640 1 -- 172.21.15.32:0/1339966101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40d4070230 msgr2=0x7f40d415c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40da5ca640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f40d4070230 0x7f40d415c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.599 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40da5ca640 1 -- 172.21.15.32:0/1339966101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f40d00094d0 con 0x7f40d4158520 2023-12-20T06:04:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40da5ca640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4158520 0x7f40d41f2dc0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f40cc007e40 tx=0x7f40cc00e4b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40ca7fc640 1 -- 172.21.15.32:0/1339966101 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc014620 con 0x7f40d4158520 2023-12-20T06:04:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40ca7fc640 1 -- 172.21.15.32:0/1339966101 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f40cc00ee90 con 0x7f40d4158520 2023-12-20T06:04:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40ca7fc640 1 -- 172.21.15.32:0/1339966101 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f40cc0149e0 con 0x7f40d4158520 2023-12-20T06:04:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40dc02d640 1 -- 172.21.15.32:0/1339966101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f40d41538a0 con 0x7f40d4158520 2023-12-20T06:04:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40dc02d640 1 -- 172.21.15.32:0/1339966101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f40d415d2d0 con 0x7f40d4158520 2023-12-20T06:04:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.597+0000 7f40dc02d640 1 -- 172.21.15.32:0/1339966101 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f40d40ca4a0 con 0x7f40d4158520 2023-12-20T06:04:46.600 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.601+0000 7f40ca7fc640 1 -- 172.21.15.32:0/1339966101 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f40cc014b40 con 0x7f40d4158520 2023-12-20T06:04:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.601+0000 7f40ca7fc640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f40b8065bd0 0x7f40b8068090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.601+0000 7f40ca7fc640 1 -- 172.21.15.32:0/1339966101 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f40cc083b00 con 0x7f40d4158520 2023-12-20T06:04:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.601+0000 7f40d9dc9640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f40b8065bd0 0x7f40b8068090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.601+0000 7f40ca7fc640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] conn(0x7f40b806c020 0x7f40b806e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.601+0000 7f40ca7fc640 1 -- 172.21.15.32:0/1339966101 --> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f40b806eb30 con 0x7f40b806c020 2023-12-20T06:04:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.601+0000 7f40ca7fc640 1 -- 172.21.15.32:0/1339966101 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=256) v2 ==== 24+0+0 (secure 0 0 0) 0x7f40cc0882a0 con 0x7f40d4158520 2023-12-20T06:04:46.602 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.601+0000 7f40d95c8640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] conn(0x7f40b806c020 0x7f40b806e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.601+0000 7f40d9dc9640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f40b8065bd0 0x7f40b8068090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f40d00af000 tx=0x7f40d00b6040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.601+0000 7f40d95c8640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] conn(0x7f40b806c020 0x7f40b806e460 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.603 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.601+0000 7f40ca7fc640 1 -- 172.21.15.32:0/1339966101 <== osd.2 v2:172.21.15.32:6818/38082 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f40b806eb30 con 0x7f40b806c020 2023-12-20T06:04:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.613+0000 7f40dc02d640 1 -- 172.21.15.32:0/1339966101 --> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f40d404e000 con 0x7f40b806c020 2023-12-20T06:04:46.616 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.613+0000 7f40ca7fc640 1 -- 172.21.15.32:0/1339966101 <== osd.2 v2:172.21.15.32:6818/38082 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f40d404e000 con 0x7f40b806c020 2023-12-20T06:04:46.617 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:04:46.617 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-20T06:04:46.617 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:04:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.617+0000 7f40dc02d640 1 -- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] conn(0x7f40b806c020 msgr2=0x7f40b806e460 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.617+0000 7f40dc02d640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:6818/38082,v1:172.21.15.32:6820/38082] conn(0x7f40b806c020 0x7f40b806e460 crc :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.617+0000 7f40dc02d640 1 -- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f40b8065bd0 msgr2=0x7f40b8068090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.617+0000 7f40dc02d640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f40b8065bd0 0x7f40b8068090 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f40d00af000 tx=0x7f40d00b6040 comp rx=0 tx=0).stop 2023-12-20T06:04:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.617+0000 7f40dc02d640 1 -- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4158520 msgr2=0x7f40d41f2dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.617+0000 7f40dc02d640 1 --2- 172.21.15.32:0/1339966101 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f40d4158520 0x7f40d41f2dc0 secure :-1 s=READY pgs=2623 cs=0 l=1 rev1=1 crypto rx=0x7f40cc007e40 tx=0x7f40cc00e4b0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.617+0000 7f40da5ca640 1 -- 172.21.15.32:0/1339966101 reap_dead start 2023-12-20T06:04:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.617+0000 7f40dc02d640 1 -- 172.21.15.32:0/1339966101 shutdown_connections 2023-12-20T06:04:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.617+0000 7f40dc02d640 1 -- 172.21.15.32:0/1339966101 >> 172.21.15.32:0/1339966101 conn(0x7f40d406a510 msgr2=0x7f40d40c4580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.617+0000 7f40dc02d640 1 -- 172.21.15.32:0/1339966101 shutdown_connections 2023-12-20T06:04:46.618 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.617+0000 7f40dc02d640 1 -- 172.21.15.32:0/1339966101 wait complete. 2023-12-20T06:04:46.635 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2023-12-20T06:04:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.777+0000 7f92bf02d640 1 Processor -- start 2023-12-20T06:04:46.780 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.777+0000 7f92bf02d640 1 -- start start 2023-12-20T06:04:46.781 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92bf02d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b81599e0 0x7f92b80aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92bf02d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b80c98c0 con 0x7f92b80c90e0 2023-12-20T06:04:46.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92bf02d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f92b80c9ba0 con 0x7f92b80b0d00 2023-12-20T06:04:46.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92bf02d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b80c9e80 con 0x7f92b81599e0 2023-12-20T06:04:46.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92b7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f92b80c90e0 0x7f92b80b05f0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40798/0 (socket says 172.21.15.32:40798) 2023-12-20T06:04:46.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92b7fff640 1 -- 172.21.15.32:0/1096032949 learned_addr learned my addr 172.21.15.32:0/1096032949 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:46.782 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92bcdc9640 1 --2- 172.21.15.32:0/1096032949 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b81599e0 0x7f92b80aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92b77fe640 1 -- 172.21.15.32:0/1096032949 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2941319728 0 0) 0x7f92b80c98c0 con 0x7f92b80c90e0 2023-12-20T06:04:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92b77fe640 1 -- 172.21.15.32:0/1096032949 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f92a8003530 con 0x7f92b80c90e0 2023-12-20T06:04:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92b77fe640 1 -- 172.21.15.32:0/1096032949 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 192096175 0 0) 0x7f92b80c9ba0 con 0x7f92b80b0d00 2023-12-20T06:04:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92bcdc9640 1 -- 172.21.15.32:0/1096032949 >> v1:172.21.15.39:6789/0 conn(0x7f92b80b0d00 legacy=0x7f92b80701c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92bcdc9640 1 -- 172.21.15.32:0/1096032949 >> v1:172.21.15.32:6789/0 conn(0x7f92b80c90e0 legacy=0x7f92b80b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.783 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.781+0000 7f92bcdc9640 1 -- 172.21.15.32:0/1096032949 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92b80ca160 con 0x7f92b81599e0 2023-12-20T06:04:46.785 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92bcdc9640 1 --2- 172.21.15.32:0/1096032949 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b81599e0 0x7f92b80aece0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0097d0 tx=0x7f92ac01c3e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1c90a3d645d83e5 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92b77fe640 1 -- 172.21.15.32:0/1096032949 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0ab600 con 0x7f92b81599e0 2023-12-20T06:04:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92b77fe640 1 -- 172.21.15.32:0/1096032949 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92ac0ab760 con 0x7f92b81599e0 2023-12-20T06:04:46.786 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92b77fe640 1 -- 172.21.15.32:0/1096032949 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92ac0abad0 con 0x7f92b81599e0 2023-12-20T06:04:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92bf02d640 1 -- 172.21.15.32:0/1096032949 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b81599e0 msgr2=0x7f92b80aece0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92bf02d640 1 --2- 172.21.15.32:0/1096032949 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b81599e0 0x7f92b80aece0 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0097d0 tx=0x7f92ac01c3e0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92bf02d640 1 -- 172.21.15.32:0/1096032949 shutdown_connections 2023-12-20T06:04:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92bf02d640 1 --2- 172.21.15.32:0/1096032949 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b81599e0 0x7f92b80aece0 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92bf02d640 1 -- 172.21.15.32:0/1096032949 >> 172.21.15.32:0/1096032949 conn(0x7f92b806a320 msgr2=0x7f92b80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92bf02d640 1 -- 172.21.15.32:0/1096032949 shutdown_connections 2023-12-20T06:04:46.787 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92bf02d640 1 -- 172.21.15.32:0/1096032949 wait complete. 2023-12-20T06:04:46.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92bf02d640 1 Processor -- start 2023-12-20T06:04:46.788 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.785+0000 7f92bf02d640 1 -- start start 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bf02d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b80b0d00 0x7f92b80cd040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bf02d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b80c90e0 0x7f92b814bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bf02d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b81599e0 0x7f92b814d700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bf02d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8156a30 con 0x7f92b81599e0 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bf02d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8156c80 con 0x7f92b80b0d00 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bf02d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f92b8156f60 con 0x7f92b80c90e0 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bd5ca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b81599e0 0x7f92b814d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bd5ca640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b81599e0 0x7f92b814d700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51420/0 (socket says 172.21.15.32:51420) 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bd5ca640 1 -- 172.21.15.32:0/2167807863 learned_addr learned my addr 172.21.15.32:0/2167807863 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92b7fff640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b80c90e0 0x7f92b814bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bcdc9640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b80b0d00 0x7f92b80cd040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.791 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bd5ca640 1 -- 172.21.15.32:0/2167807863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b80c90e0 msgr2=0x7f92b814bb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bd5ca640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b80c90e0 0x7f92b814bb30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bd5ca640 1 -- 172.21.15.32:0/2167807863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b80b0d00 msgr2=0x7f92b80cd040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bd5ca640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b80b0d00 0x7f92b80cd040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bd5ca640 1 -- 172.21.15.32:0/2167807863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f92ac0093b0 con 0x7f92b81599e0 2023-12-20T06:04:46.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92b7fff640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b80c90e0 0x7f92b814bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:46.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bd5ca640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b81599e0 0x7f92b814d700 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c780 tx=0x7f92b000cc40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92b57fa640 1 -- 172.21.15.32:0/2167807863 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b0003db0 con 0x7f92b81599e0 2023-12-20T06:04:46.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92b57fa640 1 -- 172.21.15.32:0/2167807863 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f92b000ce90 con 0x7f92b81599e0 2023-12-20T06:04:46.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bf02d640 1 -- 172.21.15.32:0/2167807863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f92b8157240 con 0x7f92b81599e0 2023-12-20T06:04:46.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92b57fa640 1 -- 172.21.15.32:0/2167807863 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f92b00052b0 con 0x7f92b81599e0 2023-12-20T06:04:46.792 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bf02d640 1 -- 172.21.15.32:0/2167807863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f92b8157800 con 0x7f92b81599e0 2023-12-20T06:04:46.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.789+0000 7f92bf02d640 1 -- 172.21.15.32:0/2167807863 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f92b8157a20 con 0x7f92b81599e0 2023-12-20T06:04:46.793 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.793+0000 7f92b57fa640 1 -- 172.21.15.32:0/2167807863 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f92b0005410 con 0x7f92b81599e0 2023-12-20T06:04:46.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.793+0000 7f92b57fa640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f929c065b80 0x7f929c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:46.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.793+0000 7f92b57fa640 1 -- 172.21.15.32:0/2167807863 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f92b00831d0 con 0x7f92b81599e0 2023-12-20T06:04:46.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.793+0000 7f92bcdc9640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f929c065b80 0x7f929c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:46.794 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.793+0000 7f92bcdc9640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f929c065b80 0x7f929c068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0af000 tx=0x7f92ac0a7000 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:46.797 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.797+0000 7f92b57fa640 1 -- 172.21.15.32:0/2167807863 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f92b0050a90 con 0x7f92b81599e0 2023-12-20T06:04:46.955 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.953+0000 7f92bf02d640 1 -- 172.21.15.32:0/2167807863 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "2", "target": ["mon-mgr", ""]}) v1 -- 0x7f92b80acc90 con 0x7f929c065b80 2023-12-20T06:04:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.953+0000 7f92b57fa640 1 -- 172.21.15.32:0/2167807863 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 2 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f92b80acc90 con 0x7f929c065b80 2023-12-20T06:04:46.956 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 2 to deep-scrub 2023-12-20T06:04:46.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 -- 172.21.15.32:0/2167807863 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f929c065b80 msgr2=0x7f929c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f929c065b80 0x7f929c068040 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f92ac0af000 tx=0x7f92ac0a7000 comp rx=0 tx=0).stop 2023-12-20T06:04:46.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 -- 172.21.15.32:0/2167807863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b81599e0 msgr2=0x7f92b814d700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:46.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b81599e0 0x7f92b814d700 secure :-1 s=READY pgs=2625 cs=0 l=1 rev1=1 crypto rx=0x7f92b000c780 tx=0x7f92b000cc40 comp rx=0 tx=0).stop 2023-12-20T06:04:46.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 -- 172.21.15.32:0/2167807863 shutdown_connections 2023-12-20T06:04:46.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f929c065b80 0x7f929c068040 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f92b81599e0 0x7f92b814d700 unknown :-1 s=CLOSED pgs=2625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.958 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f92b80c90e0 0x7f92b814bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 --2- 172.21.15.32:0/2167807863 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f92b80b0d00 0x7f92b80cd040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 -- 172.21.15.32:0/2167807863 >> 172.21.15.32:0/2167807863 conn(0x7f92b806a320 msgr2=0x7f92b814ee30 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 -- 172.21.15.32:0/2167807863 shutdown_connections 2023-12-20T06:04:46.959 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:46.957+0000 7f92bf02d640 1 -- 172.21.15.32:0/2167807863 wait complete. 2023-12-20T06:04:46.977 INFO:tasks.ceph:Scrubbing osd.3 2023-12-20T06:04:46.977 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2023-12-20T06:04:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f6134118640 1 Processor -- start 2023-12-20T06:04:47.125 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f6134118640 1 -- start start 2023-12-20T06:04:47.126 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f6134118640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f612c159af0 0x7f612c0b1930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f6134118640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f612c06d2d0 0x7f612c06d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f6134118640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f612c06dd50 con 0x7f612c0b07f0 2023-12-20T06:04:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f6134118640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612c0cc940 con 0x7f612c06d2d0 2023-12-20T06:04:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f6134118640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f612c0ccc20 con 0x7f612c159af0 2023-12-20T06:04:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f61316b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f612c06d2d0 0x7f612c06d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f61316b3640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f612c06d2d0 0x7f612c06d740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51424/0 (socket says 172.21.15.32:51424) 2023-12-20T06:04:47.127 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f61316b3640 1 -- 172.21.15.32:0/145215370 learned_addr learned my addr 172.21.15.32:0/145215370 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f6131eb4640 1 --2- 172.21.15.32:0/145215370 >> v2:172.21.15.39:3300/0 conn(0x7f612c159af0 0x7f612c0b1930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f61316b3640 1 -- 172.21.15.32:0/145215370 >> v2:172.21.15.39:3300/0 conn(0x7f612c159af0 msgr2=0x7f612c0b1930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f61316b3640 1 --2- 172.21.15.32:0/145215370 >> v2:172.21.15.39:3300/0 conn(0x7f612c159af0 0x7f612c0b1930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f61316b3640 1 -- 172.21.15.32:0/145215370 >> v1:172.21.15.32:6789/0 conn(0x7f612c0b07f0 legacy=0x7f612c0cd700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f61316b3640 1 -- 172.21.15.32:0/145215370 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f612c0ccf00 con 0x7f612c06d2d0 2023-12-20T06:04:47.128 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.125+0000 7f6131eb4640 1 --2- 172.21.15.32:0/145215370 >> v2:172.21.15.39:3300/0 conn(0x7f612c159af0 0x7f612c0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2023-12-20T06:04:47.130 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f61316b3640 1 --2- 172.21.15.32:0/145215370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f612c06d2d0 0x7f612c06d740 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f612c0c8de0 tx=0x7f611c01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6584e2ddd8829ce7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6130eb2640 1 -- 172.21.15.32:0/145215370 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c0aa650 con 0x7f612c06d2d0 2023-12-20T06:04:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6130eb2640 1 -- 172.21.15.32:0/145215370 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c0aa7b0 con 0x7f612c06d2d0 2023-12-20T06:04:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6130eb2640 1 -- 172.21.15.32:0/145215370 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c0aab40 con 0x7f612c06d2d0 2023-12-20T06:04:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6134118640 1 -- 172.21.15.32:0/145215370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f612c06d2d0 msgr2=0x7f612c06d740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6134118640 1 --2- 172.21.15.32:0/145215370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f612c06d2d0 0x7f612c06d740 secure :-1 s=READY pgs=2627 cs=0 l=1 rev1=1 crypto rx=0x7f612c0c8de0 tx=0x7f611c01c4d0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6134118640 1 -- 172.21.15.32:0/145215370 shutdown_connections 2023-12-20T06:04:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6134118640 1 --2- 172.21.15.32:0/145215370 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f612c06d2d0 0x7f612c06d740 unknown :-1 s=CLOSED pgs=2627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6134118640 1 --2- 172.21.15.32:0/145215370 >> v2:172.21.15.39:3300/0 conn(0x7f612c159af0 0x7f612c0b1930 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6134118640 1 -- 172.21.15.32:0/145215370 >> 172.21.15.32:0/145215370 conn(0x7f612c06a510 msgr2=0x7f612c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6134118640 1 -- 172.21.15.32:0/145215370 shutdown_connections 2023-12-20T06:04:47.131 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6134118640 1 -- 172.21.15.32:0/145215370 wait complete. 2023-12-20T06:04:47.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.129+0000 7f6134118640 1 Processor -- start 2023-12-20T06:04:47.132 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f6134118640 1 -- start start 2023-12-20T06:04:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f6134118640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f612c06d2d0 0x7f612c1f4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f6134118640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f612c0b07f0 0x7f612c142000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f6134118640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f612c159af0 0x7f612c142c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f6134118640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c142540 con 0x7f612c0b07f0 2023-12-20T06:04:47.134 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f6134118640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f612c143ce0 con 0x7f612c06d2d0 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f6134118640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f612c0c5d50 con 0x7f612c159af0 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61316b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f612c0b07f0 0x7f612c142000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61316b3640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f612c0b07f0 0x7f612c142000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51436/0 (socket says 172.21.15.32:51436) 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61316b3640 1 -- 172.21.15.32:0/98665441 learned_addr learned my addr 172.21.15.32:0/98665441 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61326b5640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f612c159af0 0x7f612c142c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f6131eb4640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f612c06d2d0 0x7f612c1f4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61316b3640 1 -- 172.21.15.32:0/98665441 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f612c159af0 msgr2=0x7f612c142c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61316b3640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f612c159af0 0x7f612c142c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61316b3640 1 -- 172.21.15.32:0/98665441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f612c06d2d0 msgr2=0x7f612c1f4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61316b3640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f612c06d2d0 0x7f612c1f4ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61316b3640 1 -- 172.21.15.32:0/98665441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f611c0094d0 con 0x7f612c0b07f0 2023-12-20T06:04:47.135 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61316b3640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f612c0b07f0 0x7f612c142000 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f611c0ac040 tx=0x7f611c01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61227fc640 1 -- 172.21.15.32:0/98665441 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c0b3800 con 0x7f612c0b07f0 2023-12-20T06:04:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61227fc640 1 -- 172.21.15.32:0/98665441 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f611c0b3980 con 0x7f612c0b07f0 2023-12-20T06:04:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f6134118640 1 -- 172.21.15.32:0/98665441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f612c0b1190 con 0x7f612c0b07f0 2023-12-20T06:04:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f61227fc640 1 -- 172.21.15.32:0/98665441 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f611c0b3d30 con 0x7f612c0b07f0 2023-12-20T06:04:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f6134118640 1 -- 172.21.15.32:0/98665441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f612c143a00 con 0x7f612c0b07f0 2023-12-20T06:04:47.136 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.133+0000 7f6134118640 1 -- 172.21.15.32:0/98665441 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f612c0b1010 con 0x7f612c0b07f0 2023-12-20T06:04:47.137 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.137+0000 7f61227fc640 1 -- 172.21.15.32:0/98665441 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f611c0b2380 con 0x7f612c0b07f0 2023-12-20T06:04:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.137+0000 7f61227fc640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6110065bb0 0x7f6110068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.137+0000 7f61227fc640 1 -- 172.21.15.32:0/98665441 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f611c11dd10 con 0x7f612c0b07f0 2023-12-20T06:04:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.137+0000 7f61227fc640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] conn(0x7f611006c000 0x7f611006e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.137+0000 7f61227fc640 1 -- 172.21.15.32:0/98665441 --> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f611006eb10 con 0x7f611006c000 2023-12-20T06:04:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.137+0000 7f61227fc640 1 -- 172.21.15.32:0/98665441 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=256) v2 ==== 24+0+0 (secure 0 0 0) 0x7f611c1202f0 con 0x7f612c0b07f0 2023-12-20T06:04:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.137+0000 7f6131eb4640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6110065bb0 0x7f6110068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.138 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.137+0000 7f61326b5640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] conn(0x7f611006c000 0x7f611006e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.137+0000 7f6131eb4640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6110065bb0 0x7f6110068070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6128006fd0 tx=0x7f6128008040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.137+0000 7f61326b5640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] conn(0x7f611006c000 0x7f611006e440 crc :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.139 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.137+0000 7f61227fc640 1 -- 172.21.15.32:0/98665441 <== osd.3 v2:172.21.15.32:6816/38083 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f611006eb10 con 0x7f611006c000 2023-12-20T06:04:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.149+0000 7f6134118640 1 -- 172.21.15.32:0/98665441 --> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f612c06c5a0 con 0x7f611006c000 2023-12-20T06:04:47.152 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.149+0000 7f61227fc640 1 -- 172.21.15.32:0/98665441 <== osd.3 v2:172.21.15.32:6816/38083 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6124001e80 con 0x7f611006c000 2023-12-20T06:04:47.152 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:04:47.152 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-20T06:04:47.153 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:04:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.149+0000 7f6134118640 1 -- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] conn(0x7f611006c000 msgr2=0x7f611006e440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.153 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.149+0000 7f6134118640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:6816/38083,v1:172.21.15.32:6817/38083] conn(0x7f611006c000 0x7f611006e440 crc :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.153+0000 7f6134118640 1 -- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6110065bb0 msgr2=0x7f6110068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.153+0000 7f6134118640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6110065bb0 0x7f6110068070 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f6128006fd0 tx=0x7f6128008040 comp rx=0 tx=0).stop 2023-12-20T06:04:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.153+0000 7f6134118640 1 -- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f612c0b07f0 msgr2=0x7f612c142000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.153+0000 7f6134118640 1 --2- 172.21.15.32:0/98665441 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f612c0b07f0 0x7f612c142000 secure :-1 s=READY pgs=2628 cs=0 l=1 rev1=1 crypto rx=0x7f611c0ac040 tx=0x7f611c01ad10 comp rx=0 tx=0).stop 2023-12-20T06:04:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.153+0000 7f61326b5640 1 -- 172.21.15.32:0/98665441 reap_dead start 2023-12-20T06:04:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.153+0000 7f6134118640 1 -- 172.21.15.32:0/98665441 shutdown_connections 2023-12-20T06:04:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.153+0000 7f6134118640 1 -- 172.21.15.32:0/98665441 >> 172.21.15.32:0/98665441 conn(0x7f612c06a510 msgr2=0x7f612c0731c0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.153+0000 7f6134118640 1 -- 172.21.15.32:0/98665441 shutdown_connections 2023-12-20T06:04:47.154 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.153+0000 7f6134118640 1 -- 172.21.15.32:0/98665441 wait complete. 2023-12-20T06:04:47.171 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2023-12-20T06:04:47.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f189c803640 1 Processor -- start 2023-12-20T06:04:47.315 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f189c803640 1 -- start start 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f189c803640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18980b8480 0x7f189806af20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f189c803640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f189806b460 0x7f1898160df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f189c803640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1898161330 0x7f1898153610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f189c803640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f189806b8c0 con 0x7f189806b460 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f189c803640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f18980cd4d0 con 0x7f1898161330 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f189c803640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18980ce4e0 con 0x7f18980b8480 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f188ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f189806b460 0x7f1898160df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f188ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f189806b460 0x7f1898160df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51446/0 (socket says 172.21.15.32:51446) 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f188ffff640 1 -- 172.21.15.32:0/4174851660 learned_addr learned my addr 172.21.15.32:0/4174851660 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f1896d9d640 1 --2- 172.21.15.32:0/4174851660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18980b8480 0x7f189806af20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.313+0000 7f189759e640 1 --2- 172.21.15.32:0/4174851660 >> v2:172.21.15.39:3300/0 conn(0x7f1898161330 0x7f1898153610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.317+0000 7f188ffff640 1 -- 172.21.15.32:0/4174851660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18980b8480 msgr2=0x7f189806af20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:47.317 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.317+0000 7f188ffff640 1 --2- 172.21.15.32:0/4174851660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18980b8480 0x7f189806af20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.317+0000 7f188ffff640 1 -- 172.21.15.32:0/4174851660 >> v2:172.21.15.39:3300/0 conn(0x7f1898161330 msgr2=0x7f1898153610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.317+0000 7f188ffff640 1 --2- 172.21.15.32:0/4174851660 >> v2:172.21.15.39:3300/0 conn(0x7f1898161330 0x7f1898153610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.318 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.317+0000 7f188ffff640 1 -- 172.21.15.32:0/4174851660 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18980cd720 con 0x7f189806b460 2023-12-20T06:04:47.319 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.317+0000 7f188ffff640 1 --2- 172.21.15.32:0/4174851660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189806b460 0x7f1898160df0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f189806bae0 tx=0x7f188001c500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b5423d3f4ba6a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.317+0000 7f189659c640 1 -- 172.21.15.32:0/4174851660 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18800ac410 con 0x7f189806b460 2023-12-20T06:04:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.317+0000 7f189659c640 1 -- 172.21.15.32:0/4174851660 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f18800ac570 con 0x7f189806b460 2023-12-20T06:04:47.320 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.317+0000 7f189659c640 1 -- 172.21.15.32:0/4174851660 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f18800ac900 con 0x7f189806b460 2023-12-20T06:04:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.317+0000 7f189c803640 1 -- 172.21.15.32:0/4174851660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189806b460 msgr2=0x7f1898160df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.317+0000 7f189c803640 1 --2- 172.21.15.32:0/4174851660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189806b460 0x7f1898160df0 secure :-1 s=READY pgs=2629 cs=0 l=1 rev1=1 crypto rx=0x7f189806bae0 tx=0x7f188001c500 comp rx=0 tx=0).stop 2023-12-20T06:04:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 -- 172.21.15.32:0/4174851660 shutdown_connections 2023-12-20T06:04:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 --2- 172.21.15.32:0/4174851660 >> v2:172.21.15.39:3300/0 conn(0x7f1898161330 0x7f1898153610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 --2- 172.21.15.32:0/4174851660 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189806b460 0x7f1898160df0 unknown :-1 s=CLOSED pgs=2629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 --2- 172.21.15.32:0/4174851660 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18980b8480 0x7f189806af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 -- 172.21.15.32:0/4174851660 >> 172.21.15.32:0/4174851660 conn(0x7f18980d2a50 msgr2=0x7f18980d2e60 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 -- 172.21.15.32:0/4174851660 shutdown_connections 2023-12-20T06:04:47.321 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 -- 172.21.15.32:0/4174851660 wait complete. 2023-12-20T06:04:47.322 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 Processor -- start 2023-12-20T06:04:47.323 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 -- start start 2023-12-20T06:04:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189806b460 0x7f18980c6d20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18980b8480 0x7f18980c3600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1898161330 0x7f18980c52b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18980c3e80 con 0x7f189806b460 2023-12-20T06:04:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f18980c4160 con 0x7f1898161330 2023-12-20T06:04:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189c803640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f18980c4440 con 0x7f18980b8480 2023-12-20T06:04:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f1896d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189806b460 0x7f18980c6d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.325 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f1896d9d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189806b460 0x7f18980c6d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51462/0 (socket says 172.21.15.32:51462) 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f1896d9d640 1 -- 172.21.15.32:0/1247868736 learned_addr learned my addr 172.21.15.32:0/1247868736 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f188ffff640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18980b8480 0x7f18980c3600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f189759e640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1898161330 0x7f18980c52b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f1896d9d640 1 -- 172.21.15.32:0/1247868736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18980b8480 msgr2=0x7f18980c3600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f1896d9d640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18980b8480 0x7f18980c3600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f1896d9d640 1 -- 172.21.15.32:0/1247868736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1898161330 msgr2=0x7f18980c52b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f1896d9d640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1898161330 0x7f18980c52b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.321+0000 7f1896d9d640 1 -- 172.21.15.32:0/1247868736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f18800094d0 con 0x7f189806b460 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.325+0000 7f1896d9d640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189806b460 0x7f18980c6d20 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f188800e8b0 tx=0x7f188800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.325+0000 7f188f7fe640 1 -- 172.21.15.32:0/1247868736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188800c940 con 0x7f189806b460 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.325+0000 7f188f7fe640 1 -- 172.21.15.32:0/1247868736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f188800caa0 con 0x7f189806b460 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.325+0000 7f188f7fe640 1 -- 172.21.15.32:0/1247868736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f188800ce30 con 0x7f189806b460 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.325+0000 7f189c803640 1 -- 172.21.15.32:0/1247868736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f18980c4ab0 con 0x7f189806b460 2023-12-20T06:04:47.326 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.325+0000 7f189c803640 1 -- 172.21.15.32:0/1247868736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f18980c3b40 con 0x7f189806b460 2023-12-20T06:04:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.325+0000 7f189c803640 1 -- 172.21.15.32:0/1247868736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1898074100 con 0x7f189806b460 2023-12-20T06:04:47.327 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.325+0000 7f188f7fe640 1 -- 172.21.15.32:0/1247868736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1888013470 con 0x7f189806b460 2023-12-20T06:04:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.325+0000 7f188f7fe640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1878065b80 0x7f1878068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.325+0000 7f188f7fe640 1 -- 172.21.15.32:0/1247868736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f1888083450 con 0x7f189806b460 2023-12-20T06:04:47.328 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.325+0000 7f188ffff640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1878065b80 0x7f1878068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.329 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.329+0000 7f188ffff640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1878065b80 0x7f1878068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18800ae000 tx=0x7f1880009040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.332 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.329+0000 7f188f7fe640 1 -- 172.21.15.32:0/1247868736 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1888050db0 con 0x7f189806b460 2023-12-20T06:04:47.488 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.485+0000 7f189c803640 1 -- 172.21.15.32:0/1247868736 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "3", "target": ["mon-mgr", ""]}) v1 -- 0x7f18980cdfc0 con 0x7f1878065b80 2023-12-20T06:04:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f188f7fe640 1 -- 172.21.15.32:0/1247868736 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 3 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f18800ac900 con 0x7f1878065b80 2023-12-20T06:04:47.489 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 3 to deep-scrub 2023-12-20T06:04:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 -- 172.21.15.32:0/1247868736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1878065b80 msgr2=0x7f1878068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1878065b80 0x7f1878068040 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f18800ae000 tx=0x7f1880009040 comp rx=0 tx=0).stop 2023-12-20T06:04:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 -- 172.21.15.32:0/1247868736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189806b460 msgr2=0x7f18980c6d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189806b460 0x7f18980c6d20 secure :-1 s=READY pgs=2630 cs=0 l=1 rev1=1 crypto rx=0x7f188800e8b0 tx=0x7f188800ed70 comp rx=0 tx=0).stop 2023-12-20T06:04:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 -- 172.21.15.32:0/1247868736 shutdown_connections 2023-12-20T06:04:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1878065b80 0x7f1878068040 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1898161330 0x7f18980c52b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f18980b8480 0x7f18980c3600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 --2- 172.21.15.32:0/1247868736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f189806b460 0x7f18980c6d20 unknown :-1 s=CLOSED pgs=2630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.491 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 -- 172.21.15.32:0/1247868736 >> 172.21.15.32:0/1247868736 conn(0x7f18980d2a50 msgr2=0x7f1898154580 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 -- 172.21.15.32:0/1247868736 shutdown_connections 2023-12-20T06:04:47.492 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.489+0000 7f189c803640 1 -- 172.21.15.32:0/1247868736 wait complete. 2023-12-20T06:04:47.510 INFO:tasks.ceph:Scrubbing osd.4 2023-12-20T06:04:47.510 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 config set osd_debug_deep_scrub_sleep 0 2023-12-20T06:04:47.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69330b6640 1 Processor -- start 2023-12-20T06:04:47.657 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69330b6640 1 -- start start 2023-12-20T06:04:47.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69330b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f692c0ca790 0x7f692c159b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69330b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f692c0affe0 0x7f692c0b0450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69330b6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f692c0ce070 con 0x7f692c159770 2023-12-20T06:04:47.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69330b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f692c0b0990 con 0x7f692c0affe0 2023-12-20T06:04:47.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69330b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c15a480 con 0x7f692c0ca790 2023-12-20T06:04:47.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69328b5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f692c0affe0 0x7f692c0b0450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.659 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69328b5640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f692c0affe0 0x7f692c0b0450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51464/0 (socket says 172.21.15.32:51464) 2023-12-20T06:04:47.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69328b5640 1 -- 172.21.15.32:0/3304768119 learned_addr learned my addr 172.21.15.32:0/3304768119 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:47.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69328b5640 1 -- 172.21.15.32:0/3304768119 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f692c0ca790 msgr2=0x7f692c159b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:47.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69328b5640 1 --2- 172.21.15.32:0/3304768119 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f692c0ca790 0x7f692c159b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69328b5640 1 -- 172.21.15.32:0/3304768119 >> v1:172.21.15.32:6789/0 conn(0x7f692c159770 legacy=0x7f692c0cc130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.660 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.657+0000 7f69328b5640 1 -- 172.21.15.32:0/3304768119 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f692c15a760 con 0x7f692c0affe0 2023-12-20T06:04:47.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69328b5640 1 --2- 172.21.15.32:0/3304768119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f692c0affe0 0x7f692c0b0450 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f6924009e50 tx=0x7f692401c9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ace28d15e7bb044 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69310b2640 1 -- 172.21.15.32:0/3304768119 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69240a58a0 con 0x7f692c0affe0 2023-12-20T06:04:47.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69310b2640 1 -- 172.21.15.32:0/3304768119 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f69240a5a00 con 0x7f692c0affe0 2023-12-20T06:04:47.662 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69310b2640 1 -- 172.21.15.32:0/3304768119 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69240a5d90 con 0x7f692c0affe0 2023-12-20T06:04:47.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69330b6640 1 -- 172.21.15.32:0/3304768119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f692c0affe0 msgr2=0x7f692c0b0450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69330b6640 1 --2- 172.21.15.32:0/3304768119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f692c0affe0 0x7f692c0b0450 secure :-1 s=READY pgs=2632 cs=0 l=1 rev1=1 crypto rx=0x7f6924009e50 tx=0x7f692401c9e0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.663 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69330b6640 1 -- 172.21.15.32:0/3304768119 shutdown_connections 2023-12-20T06:04:47.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69330b6640 1 --2- 172.21.15.32:0/3304768119 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f692c0affe0 0x7f692c0b0450 unknown :-1 s=CLOSED pgs=2632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69330b6640 1 --2- 172.21.15.32:0/3304768119 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f692c0ca790 0x7f692c159b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69330b6640 1 -- 172.21.15.32:0/3304768119 >> 172.21.15.32:0/3304768119 conn(0x7f692c06a4e0 msgr2=0x7f692c06bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:47.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69330b6640 1 -- 172.21.15.32:0/3304768119 shutdown_connections 2023-12-20T06:04:47.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69330b6640 1 -- 172.21.15.32:0/3304768119 wait complete. 2023-12-20T06:04:47.664 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.661+0000 7f69330b6640 1 Processor -- start 2023-12-20T06:04:47.665 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69330b6640 1 -- start start 2023-12-20T06:04:47.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69330b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f692c0affe0 0x7f692c150530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69330b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c0ca790 0x7f692c0c8df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69330b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f692c159770 0x7f692c153620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69330b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0c9cb0 con 0x7f692c0affe0 2023-12-20T06:04:47.667 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69330b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0c9f90 con 0x7f692c0ca790 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69330b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f692c0ca270 con 0x7f692c159770 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69320b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f692c0affe0 0x7f692c150530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69320b4640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f692c0affe0 0x7f692c150530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51472/0 (socket says 172.21.15.32:51472) 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69320b4640 1 -- 172.21.15.32:0/150437554 learned_addr learned my addr 172.21.15.32:0/150437554 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69328b5640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f692c159770 0x7f692c153620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69318b3640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c0ca790 0x7f692c0c8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69320b4640 1 -- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f692c159770 msgr2=0x7f692c153620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69320b4640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f692c159770 0x7f692c153620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69320b4640 1 -- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c0ca790 msgr2=0x7f692c0c8df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69320b4640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c0ca790 0x7f692c0c8df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69320b4640 1 -- 172.21.15.32:0/150437554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6924009b30 con 0x7f692c0affe0 2023-12-20T06:04:47.668 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69318b3640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f692c0ca790 0x7f692c0c8df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:47.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69320b4640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f692c0affe0 0x7f692c150530 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f692800c3c0 tx=0x7f692800c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f6922ffd640 1 -- 172.21.15.32:0/150437554 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f69280052d0 con 0x7f692c0affe0 2023-12-20T06:04:47.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f6922ffd640 1 -- 172.21.15.32:0/150437554 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6928005430 con 0x7f692c0affe0 2023-12-20T06:04:47.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f6922ffd640 1 -- 172.21.15.32:0/150437554 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6928003840 con 0x7f692c0affe0 2023-12-20T06:04:47.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69330b6640 1 -- 172.21.15.32:0/150437554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f692c14bce0 con 0x7f692c0affe0 2023-12-20T06:04:47.669 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69330b6640 1 -- 172.21.15.32:0/150437554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f692c14b9d0 con 0x7f692c0affe0 2023-12-20T06:04:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.665+0000 7f69330b6640 1 -- 172.21.15.32:0/150437554 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f692c0ca490 con 0x7f692c0affe0 2023-12-20T06:04:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.669+0000 7f6922ffd640 1 -- 172.21.15.32:0/150437554 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f69280039a0 con 0x7f692c0affe0 2023-12-20T06:04:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.669+0000 7f6922ffd640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6910065c00 0x7f69100680c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.669+0000 7f6922ffd640 1 -- 172.21.15.32:0/150437554 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f6928082e00 con 0x7f692c0affe0 2023-12-20T06:04:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.669+0000 7f69318b3640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6910065c00 0x7f69100680c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.669+0000 7f6922ffd640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] conn(0x7f691006c050 0x7f691006e490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.669+0000 7f6922ffd640 1 -- 172.21.15.32:0/150437554 --> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f691006eb60 con 0x7f691006c050 2023-12-20T06:04:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.669+0000 7f6922ffd640 1 -- 172.21.15.32:0/150437554 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=256) v2 ==== 24+0+0 (secure 0 0 0) 0x7f6928083170 con 0x7f692c0affe0 2023-12-20T06:04:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.669+0000 7f69318b3640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6910065c00 0x7f69100680c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f691c006fd0 tx=0x7f691c008040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.671 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.669+0000 7f69328b5640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] conn(0x7f691006c050 0x7f691006e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.669+0000 7f69328b5640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] conn(0x7f691006c050 0x7f691006e490 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.672 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.669+0000 7f6922ffd640 1 -- 172.21.15.32:0/150437554 <== osd.4 v2:172.21.15.39:6800/38278 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f691006eb60 con 0x7f691006c050 2023-12-20T06:04:47.685 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69330b6640 1 -- 172.21.15.32:0/150437554 --> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f692c071220 con 0x7f691006c050 2023-12-20T06:04:47.686 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f6922ffd640 1 -- 172.21.15.32:0/150437554 <== osd.4 v2:172.21.15.39:6800/38278 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f6924009210 con 0x7f691006c050 2023-12-20T06:04:47.686 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:04:47.686 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-20T06:04:47.686 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:04:47.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69330b6640 1 -- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] conn(0x7f691006c050 msgr2=0x7f691006e490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69330b6640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.39:6800/38278,v1:172.21.15.39:6801/38278] conn(0x7f691006c050 0x7f691006e490 crc :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.687 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69330b6640 1 -- 172.21.15.32:0/150437554 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6910065c00 msgr2=0x7f69100680c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69330b6640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6910065c00 0x7f69100680c0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7f691c006fd0 tx=0x7f691c008040 comp rx=0 tx=0).stop 2023-12-20T06:04:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69330b6640 1 -- 172.21.15.32:0/150437554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f692c0affe0 msgr2=0x7f692c150530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69330b6640 1 --2- 172.21.15.32:0/150437554 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f692c0affe0 0x7f692c150530 secure :-1 s=READY pgs=2633 cs=0 l=1 rev1=1 crypto rx=0x7f692800c3c0 tx=0x7f692800c880 comp rx=0 tx=0).stop 2023-12-20T06:04:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69328b5640 1 -- 172.21.15.32:0/150437554 reap_dead start 2023-12-20T06:04:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69330b6640 1 -- 172.21.15.32:0/150437554 shutdown_connections 2023-12-20T06:04:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69330b6640 1 -- 172.21.15.32:0/150437554 >> 172.21.15.32:0/150437554 conn(0x7f692c06a4e0 msgr2=0x7f692c06bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69330b6640 1 -- 172.21.15.32:0/150437554 shutdown_connections 2023-12-20T06:04:47.688 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.685+0000 7f69330b6640 1 -- 172.21.15.32:0/150437554 wait complete. 2023-12-20T06:04:47.705 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 4 2023-12-20T06:04:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9bcf17640 1 Processor -- start 2023-12-20T06:04:47.850 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9bcf17640 1 -- start start 2023-12-20T06:04:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9bcf17640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd9b8069d20 0x7fd9b80cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9bcf17640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9b815d080 0x7fd9b80cd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9bcf17640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd9b815d970 con 0x7fd9b815de00 2023-12-20T06:04:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9bcf17640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b80ce740 con 0x7fd9b815d080 2023-12-20T06:04:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9bcf17640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd9b80cdf00 con 0x7fd9b8069d20 2023-12-20T06:04:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9b5d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9b815d080 0x7fd9b80cd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9b5d9b640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7fd9b815d080 0x7fd9b80cd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51476/0 (socket says 172.21.15.32:51476) 2023-12-20T06:04:47.852 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9b5d9b640 1 -- 172.21.15.32:0/490320467 learned_addr learned my addr 172.21.15.32:0/490320467 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9b659c640 1 --2- 172.21.15.32:0/490320467 >> v2:172.21.15.39:3300/0 conn(0x7fd9b8069d20 0x7fd9b80cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9b5d9b640 1 -- 172.21.15.32:0/490320467 >> v2:172.21.15.39:3300/0 conn(0x7fd9b8069d20 msgr2=0x7fd9b80cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9b5d9b640 1 --2- 172.21.15.32:0/490320467 >> v2:172.21.15.39:3300/0 conn(0x7fd9b8069d20 0x7fd9b80cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9b5d9b640 1 -- 172.21.15.32:0/490320467 >> v1:172.21.15.32:6789/0 conn(0x7fd9b815de00 legacy=0x7fd9b806a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.853 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.849+0000 7fd9b5d9b640 1 -- 172.21.15.32:0/490320467 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9b80ce180 con 0x7fd9b815d080 2023-12-20T06:04:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9b5d9b640 1 --2- 172.21.15.32:0/490320467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b815d080 0x7fd9b80cd9c0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80cd990 tx=0x7fd9ac01c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b78a106ab562a1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9b559a640 1 -- 172.21.15.32:0/490320467 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0aa430 con 0x7fd9b815d080 2023-12-20T06:04:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9b559a640 1 -- 172.21.15.32:0/490320467 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac0aa590 con 0x7fd9b815d080 2023-12-20T06:04:47.855 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9b559a640 1 -- 172.21.15.32:0/490320467 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0aa920 con 0x7fd9b815d080 2023-12-20T06:04:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9bcf17640 1 -- 172.21.15.32:0/490320467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b815d080 msgr2=0x7fd9b80cd9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9bcf17640 1 --2- 172.21.15.32:0/490320467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b815d080 0x7fd9b80cd9c0 secure :-1 s=READY pgs=2635 cs=0 l=1 rev1=1 crypto rx=0x7fd9b80cd990 tx=0x7fd9ac01c4d0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9bcf17640 1 -- 172.21.15.32:0/490320467 shutdown_connections 2023-12-20T06:04:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9bcf17640 1 --2- 172.21.15.32:0/490320467 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b815d080 0x7fd9b80cd9c0 unknown :-1 s=CLOSED pgs=2635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9bcf17640 1 --2- 172.21.15.32:0/490320467 >> v2:172.21.15.39:3300/0 conn(0x7fd9b8069d20 0x7fd9b80cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9bcf17640 1 -- 172.21.15.32:0/490320467 >> 172.21.15.32:0/490320467 conn(0x7fd9b80cc310 msgr2=0x7fd9b80cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9bcf17640 1 -- 172.21.15.32:0/490320467 shutdown_connections 2023-12-20T06:04:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.853+0000 7fd9bcf17640 1 -- 172.21.15.32:0/490320467 wait complete. 2023-12-20T06:04:47.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9bcf17640 1 Processor -- start 2023-12-20T06:04:47.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9bcf17640 1 -- start start 2023-12-20T06:04:47.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9bcf17640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b8069d20 0x7fd9b80b5200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9bcf17640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b815d080 0x7fd9b80ad8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9bcf17640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b815de00 0x7fd9b80b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9bcf17640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b8154ce0 con 0x7fd9b815d080 2023-12-20T06:04:47.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9bcf17640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b8154060 con 0x7fd9b815de00 2023-12-20T06:04:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9bcf17640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd9b8154340 con 0x7fd9b8069d20 2023-12-20T06:04:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9b5d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b815d080 0x7fd9b80ad8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9b5d9b640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b815d080 0x7fd9b80ad8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51490/0 (socket says 172.21.15.32:51490) 2023-12-20T06:04:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9b5d9b640 1 -- 172.21.15.32:0/1483371830 learned_addr learned my addr 172.21.15.32:0/1483371830 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9b659c640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b8069d20 0x7fd9b80b5200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9b6d9d640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b815de00 0x7fd9b80b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9b5d9b640 1 -- 172.21.15.32:0/1483371830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b8069d20 msgr2=0x7fd9b80b5200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9b5d9b640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b8069d20 0x7fd9b80b5200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9b5d9b640 1 -- 172.21.15.32:0/1483371830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b815de00 msgr2=0x7fd9b80b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9b5d9b640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b815de00 0x7fd9b80b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9b5d9b640 1 -- 172.21.15.32:0/1483371830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd9ac0094d0 con 0x7fd9b815d080 2023-12-20T06:04:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9b5d9b640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b815d080 0x7fd9b80ad8f0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0ac040 tx=0x7fd9ac01ad10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9a6ffd640 1 -- 172.21.15.32:0/1483371830 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0b3800 con 0x7fd9b815d080 2023-12-20T06:04:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9a6ffd640 1 -- 172.21.15.32:0/1483371830 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd9ac0b3960 con 0x7fd9b815d080 2023-12-20T06:04:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9bcf17640 1 -- 172.21.15.32:0/1483371830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd9b815e390 con 0x7fd9b815d080 2023-12-20T06:04:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9a6ffd640 1 -- 172.21.15.32:0/1483371830 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0b3d10 con 0x7fd9b815d080 2023-12-20T06:04:47.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.857+0000 7fd9bcf17640 1 -- 172.21.15.32:0/1483371830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd9b8154900 con 0x7fd9b815d080 2023-12-20T06:04:47.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.861+0000 7fd9bcf17640 1 -- 172.21.15.32:0/1483371830 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd9b804e000 con 0x7fd9b815d080 2023-12-20T06:04:47.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.861+0000 7fd9a6ffd640 1 -- 172.21.15.32:0/1483371830 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd9ac0b2380 con 0x7fd9b815d080 2023-12-20T06:04:47.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.861+0000 7fd9a6ffd640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd998065b80 0x7fd998068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:47.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.861+0000 7fd9a6ffd640 1 -- 172.21.15.32:0/1483371830 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7fd9ac11da00 con 0x7fd9b815d080 2023-12-20T06:04:47.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.861+0000 7fd9b659c640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd998065b80 0x7fd998068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:47.864 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.861+0000 7fd9b659c640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd998065b80 0x7fd998068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8007a60 tx=0x7fd9a8008040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:47.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:47.865+0000 7fd9a6ffd640 1 -- 172.21.15.32:0/1483371830 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd9ac0eb250 con 0x7fd9b815d080 2023-12-20T06:04:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.021+0000 7fd9bcf17640 1 -- 172.21.15.32:0/1483371830 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "4", "target": ["mon-mgr", ""]}) v1 -- 0x7fd9b815e210 con 0x7fd998065b80 2023-12-20T06:04:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9a6ffd640 1 -- 172.21.15.32:0/1483371830 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 4 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd9a8010990 con 0x7fd998065b80 2023-12-20T06:04:48.025 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 4 to deep-scrub 2023-12-20T06:04:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 -- 172.21.15.32:0/1483371830 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd998065b80 msgr2=0x7fd998068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd998065b80 0x7fd998068040 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fd9a8007a60 tx=0x7fd9a8008040 comp rx=0 tx=0).stop 2023-12-20T06:04:48.027 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 -- 172.21.15.32:0/1483371830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b815d080 msgr2=0x7fd9b80ad8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b815d080 0x7fd9b80ad8f0 secure :-1 s=READY pgs=2636 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0ac040 tx=0x7fd9ac01ad10 comp rx=0 tx=0).stop 2023-12-20T06:04:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 -- 172.21.15.32:0/1483371830 shutdown_connections 2023-12-20T06:04:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd998065b80 0x7fd998068040 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd9b815de00 0x7fd9b80b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd9b815d080 0x7fd9b80ad8f0 unknown :-1 s=CLOSED pgs=2636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 --2- 172.21.15.32:0/1483371830 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd9b8069d20 0x7fd9b80b5200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 -- 172.21.15.32:0/1483371830 >> 172.21.15.32:0/1483371830 conn(0x7fd9b80cc310 msgr2=0x7fd9b80cac40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 -- 172.21.15.32:0/1483371830 shutdown_connections 2023-12-20T06:04:48.028 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.025+0000 7fd9bcf17640 1 -- 172.21.15.32:0/1483371830 wait complete. 2023-12-20T06:04:48.047 INFO:tasks.ceph:Scrubbing osd.5 2023-12-20T06:04:48.047 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 config set osd_debug_deep_scrub_sleep 0 2023-12-20T06:04:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.193+0000 7f7e44c8c640 1 Processor -- start 2023-12-20T06:04:48.195 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.193+0000 7f7e44c8c640 1 -- start start 2023-12-20T06:04:48.196 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.193+0000 7f7e44c8c640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7e400ce290 0x7f7e40070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.193+0000 7f7e44c8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e40070770 0x7f7e400cc5b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.193+0000 7f7e44c8c640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7e40159630 con 0x7f7e40159250 2023-12-20T06:04:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.193+0000 7f7e44c8c640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7e40070be0 con 0x7f7e400ce290 2023-12-20T06:04:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.193+0000 7f7e44c8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400c9480 con 0x7f7e40070770 2023-12-20T06:04:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.193+0000 7f7e3ed9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e40070770 0x7f7e400cc5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.193+0000 7f7e3ed9d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e40070770 0x7f7e400cc5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:50054/0 (socket says 172.21.15.32:50054) 2023-12-20T06:04:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.193+0000 7f7e3ed9d640 1 -- 172.21.15.32:0/171361875 learned_addr learned my addr 172.21.15.32:0/171361875 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.193+0000 7f7e37fff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7e400ce290 0x7f7e40070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e3ed9d640 1 -- 172.21.15.32:0/171361875 >> v2:172.21.15.39:3300/0 conn(0x7f7e400ce290 msgr2=0x7f7e40070230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:48.197 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e3ed9d640 1 --2- 172.21.15.32:0/171361875 >> v2:172.21.15.39:3300/0 conn(0x7f7e400ce290 0x7f7e40070230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e3ed9d640 1 -- 172.21.15.32:0/171361875 >> v1:172.21.15.39:6789/0 conn(0x7f7e40159250 legacy=0x7f7e400caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.198 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e3ed9d640 1 -- 172.21.15.32:0/171361875 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e4006cf90 con 0x7f7e40070770 2023-12-20T06:04:48.199 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e3ed9d640 1 --2- 172.21.15.32:0/171361875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e40070770 0x7f7e400cc5b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009e50 tx=0x7f7e3801ca10 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d14040c7ddbc78f1 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e3dd9b640 1 -- 172.21.15.32:0/171361875 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e380a5860 con 0x7f7e40070770 2023-12-20T06:04:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e3dd9b640 1 -- 172.21.15.32:0/171361875 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e380a59c0 con 0x7f7e40070770 2023-12-20T06:04:48.200 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e3dd9b640 1 -- 172.21.15.32:0/171361875 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e380a5d50 con 0x7f7e40070770 2023-12-20T06:04:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e44c8c640 1 -- 172.21.15.32:0/171361875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e40070770 msgr2=0x7f7e400cc5b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e44c8c640 1 --2- 172.21.15.32:0/171361875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e40070770 0x7f7e400cc5b0 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f7e38009e50 tx=0x7f7e3801ca10 comp rx=0 tx=0).stop 2023-12-20T06:04:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e44c8c640 1 -- 172.21.15.32:0/171361875 shutdown_connections 2023-12-20T06:04:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e44c8c640 1 --2- 172.21.15.32:0/171361875 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e40070770 0x7f7e400cc5b0 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e44c8c640 1 --2- 172.21.15.32:0/171361875 >> v2:172.21.15.39:3300/0 conn(0x7f7e400ce290 0x7f7e40070230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.197+0000 7f7e44c8c640 1 -- 172.21.15.32:0/171361875 >> 172.21.15.32:0/171361875 conn(0x7f7e4006a510 msgr2=0x7f7e400ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:48.201 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e44c8c640 1 -- 172.21.15.32:0/171361875 shutdown_connections 2023-12-20T06:04:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e44c8c640 1 -- 172.21.15.32:0/171361875 wait complete. 2023-12-20T06:04:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e44c8c640 1 Processor -- start 2023-12-20T06:04:48.202 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e44c8c640 1 -- start start 2023-12-20T06:04:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e44c8c640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e40070770 0x7f7e400b2680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e44c8c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e400ce290 0x7f7e40147b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e44c8c640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e40159250 0x7f7e4014abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e44c8c640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400b2c60 con 0x7f7e400ce290 2023-12-20T06:04:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e44c8c640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400b2f40 con 0x7f7e40070770 2023-12-20T06:04:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e44c8c640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7e400b3220 con 0x7f7e40159250 2023-12-20T06:04:48.205 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e37fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e400ce290 0x7f7e40147b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e37fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e400ce290 0x7f7e40147b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51498/0 (socket says 172.21.15.32:51498) 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e37fff640 1 -- 172.21.15.32:0/2036782643 learned_addr learned my addr 172.21.15.32:0/2036782643 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e3e59c640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e40070770 0x7f7e400b2680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e3ed9d640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e40159250 0x7f7e4014abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e37fff640 1 -- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e40159250 msgr2=0x7f7e4014abe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e37fff640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7e40159250 0x7f7e4014abe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e37fff640 1 -- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e40070770 msgr2=0x7f7e400b2680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e37fff640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e40070770 0x7f7e400b2680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e37fff640 1 -- 172.21.15.32:0/2036782643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7e38009b30 con 0x7f7e400ce290 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e3e59c640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7e40070770 0x7f7e400b2680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.201+0000 7f7e37fff640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e400ce290 0x7f7e40147b50 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800e900 tx=0x7f7e2800edc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.206 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e36ffd640 1 -- 172.21.15.32:0/2036782643 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e28009690 con 0x7f7e400ce290 2023-12-20T06:04:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e36ffd640 1 -- 172.21.15.32:0/2036782643 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f7e2800ca00 con 0x7f7e400ce290 2023-12-20T06:04:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e36ffd640 1 -- 172.21.15.32:0/2036782643 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7e2800cdb0 con 0x7f7e400ce290 2023-12-20T06:04:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e44c8c640 1 -- 172.21.15.32:0/2036782643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7e400b3500 con 0x7f7e400ce290 2023-12-20T06:04:48.207 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e44c8c640 1 -- 172.21.15.32:0/2036782643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7e40148090 con 0x7f7e400ce290 2023-12-20T06:04:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e44c8c640 1 -- 172.21.15.32:0/2036782643 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f7e400c9110 con 0x7f7e400ce290 2023-12-20T06:04:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e36ffd640 1 -- 172.21.15.32:0/2036782643 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7e28013470 con 0x7f7e400ce290 2023-12-20T06:04:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e36ffd640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7e20065b80 0x7f7e20068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e36ffd640 1 -- 172.21.15.32:0/2036782643 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f7e28083530 con 0x7f7e400ce290 2023-12-20T06:04:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e36ffd640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] conn(0x7f7e2006bfd0 0x7f7e2006e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e36ffd640 1 -- 172.21.15.32:0/2036782643 --> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f7e2006eae0 con 0x7f7e2006bfd0 2023-12-20T06:04:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e36ffd640 1 -- 172.21.15.32:0/2036782643 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=256) v2 ==== 24+0+0 (secure 0 0 0) 0x7f7e28085b10 con 0x7f7e400ce290 2023-12-20T06:04:48.208 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e3e59c640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7e20065b80 0x7f7e20068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e3ed9d640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] conn(0x7f7e2006bfd0 0x7f7e2006e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.205+0000 7f7e3e59c640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7e20065b80 0x7f7e20068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7e30007fd0 tx=0x7f7e30009040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.209 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.209+0000 7f7e3ed9d640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] conn(0x7f7e2006bfd0 0x7f7e2006e410 crc :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.210 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.209+0000 7f7e36ffd640 1 -- 172.21.15.32:0/2036782643 <== osd.5 v2:172.21.15.39:6824/38280 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f7e2006eae0 con 0x7f7e2006bfd0 2023-12-20T06:04:48.222 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e44c8c640 1 -- 172.21.15.32:0/2036782643 --> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f7e400b39b0 con 0x7f7e2006bfd0 2023-12-20T06:04:48.223 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e36ffd640 1 -- 172.21.15.32:0/2036782643 <== osd.5 v2:172.21.15.39:6824/38280 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f7e380090d0 con 0x7f7e2006bfd0 2023-12-20T06:04:48.223 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:04:48.223 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-20T06:04:48.223 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:04:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e44c8c640 1 -- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] conn(0x7f7e2006bfd0 msgr2=0x7f7e2006e410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.224 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e44c8c640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.39:6824/38280,v1:172.21.15.39:6825/38280] conn(0x7f7e2006bfd0 0x7f7e2006e410 crc :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e44c8c640 1 -- 172.21.15.32:0/2036782643 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7e20065b80 msgr2=0x7f7e20068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e44c8c640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7e20065b80 0x7f7e20068040 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f7e30007fd0 tx=0x7f7e30009040 comp rx=0 tx=0).stop 2023-12-20T06:04:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e44c8c640 1 -- 172.21.15.32:0/2036782643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e400ce290 msgr2=0x7f7e40147b50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e44c8c640 1 --2- 172.21.15.32:0/2036782643 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7e400ce290 0x7f7e40147b50 secure :-1 s=READY pgs=2637 cs=0 l=1 rev1=1 crypto rx=0x7f7e2800e900 tx=0x7f7e2800edc0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e3ed9d640 1 -- 172.21.15.32:0/2036782643 reap_dead start 2023-12-20T06:04:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e44c8c640 1 -- 172.21.15.32:0/2036782643 shutdown_connections 2023-12-20T06:04:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e44c8c640 1 -- 172.21.15.32:0/2036782643 >> 172.21.15.32:0/2036782643 conn(0x7f7e4006a510 msgr2=0x7f7e4006c3e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e44c8c640 1 -- 172.21.15.32:0/2036782643 shutdown_connections 2023-12-20T06:04:48.225 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.221+0000 7f7e44c8c640 1 -- 172.21.15.32:0/2036782643 wait complete. 2023-12-20T06:04:48.242 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 5 2023-12-20T06:04:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.385+0000 7f1fbafdb640 1 Processor -- start 2023-12-20T06:04:48.387 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.385+0000 7f1fbafdb640 1 -- start start 2023-12-20T06:04:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.385+0000 7f1fbafdb640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1fb40af5e0 0x7f1fb40af9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.385+0000 7f1fbafdb640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb415f110 0x7f1fb415c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.385+0000 7f1fbafdb640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f1fb415b990 con 0x7f1fb406ce60 2023-12-20T06:04:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.385+0000 7f1fbafdb640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f1fb415bbd0 con 0x7f1fb40af5e0 2023-12-20T06:04:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.385+0000 7f1fbafdb640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb415beb0 con 0x7f1fb415f110 2023-12-20T06:04:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.385+0000 7f1fb8d77640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1fb40af5e0 0x7f1fb40af9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.389 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.385+0000 7f1fb8d77640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f1fb40af5e0 0x7f1fb40af9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3300/0 says I am v2:172.21.15.32:34694/0 (socket says 172.21.15.32:34694) 2023-12-20T06:04:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.385+0000 7f1fb8d77640 1 -- 172.21.15.32:0/1498136343 learned_addr learned my addr 172.21.15.32:0/1498136343 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.385+0000 7f1fb9578640 1 --2- 172.21.15.32:0/1498136343 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb415f110 0x7f1fb415c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fb8d77640 1 -- 172.21.15.32:0/1498136343 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb415f110 msgr2=0x7f1fb415c440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fb8d77640 1 --2- 172.21.15.32:0/1498136343 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb415f110 0x7f1fb415c440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fb8d77640 1 -- 172.21.15.32:0/1498136343 >> v1:172.21.15.39:6789/0 conn(0x7f1fb406ce60 legacy=0x7f1fb415b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.390 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fb8d77640 1 -- 172.21.15.32:0/1498136343 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fb415e910 con 0x7f1fb40af5e0 2023-12-20T06:04:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fb8d77640 1 --2- 172.21.15.32:0/1498136343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb40af5e0 0x7f1fb40af9c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8006fd0 tx=0x7f1fa801c4f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c91af0406abdd2eb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fb37fe640 1 -- 172.21.15.32:0/1498136343 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80aa670 con 0x7f1fb40af5e0 2023-12-20T06:04:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fb37fe640 1 -- 172.21.15.32:0/1498136343 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa80aa7d0 con 0x7f1fb40af5e0 2023-12-20T06:04:48.392 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fb37fe640 1 -- 172.21.15.32:0/1498136343 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa80aab60 con 0x7f1fb40af5e0 2023-12-20T06:04:48.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fbafdb640 1 -- 172.21.15.32:0/1498136343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb40af5e0 msgr2=0x7f1fb40af9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fbafdb640 1 --2- 172.21.15.32:0/1498136343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb40af5e0 0x7f1fb40af9c0 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7f1fa8006fd0 tx=0x7f1fa801c4f0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fbafdb640 1 -- 172.21.15.32:0/1498136343 shutdown_connections 2023-12-20T06:04:48.393 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fbafdb640 1 --2- 172.21.15.32:0/1498136343 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb415f110 0x7f1fb415c440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fbafdb640 1 --2- 172.21.15.32:0/1498136343 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb40af5e0 0x7f1fb40af9c0 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fbafdb640 1 -- 172.21.15.32:0/1498136343 >> 172.21.15.32:0/1498136343 conn(0x7f1fb406a320 msgr2=0x7f1fb40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fbafdb640 1 -- 172.21.15.32:0/1498136343 shutdown_connections 2023-12-20T06:04:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.389+0000 7f1fbafdb640 1 -- 172.21.15.32:0/1498136343 wait complete. 2023-12-20T06:04:48.394 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fbafdb640 1 Processor -- start 2023-12-20T06:04:48.395 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fbafdb640 1 -- start start 2023-12-20T06:04:48.397 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fbafdb640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb406ce60 0x7f1fb4151f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fbafdb640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb40af5e0 0x7f1fb40c71c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fbafdb640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb415f110 0x7f1fb40c7700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fbafdb640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40c7c40 con 0x7f1fb40af5e0 2023-12-20T06:04:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fbafdb640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40b1c40 con 0x7f1fb415f110 2023-12-20T06:04:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fbafdb640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f1fb40b1ee0 con 0x7f1fb406ce60 2023-12-20T06:04:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fb3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb40af5e0 0x7f1fb40c71c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fb3fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb40af5e0 0x7f1fb40c71c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51510/0 (socket says 172.21.15.32:51510) 2023-12-20T06:04:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fb3fff640 1 -- 172.21.15.32:0/3984369350 learned_addr learned my addr 172.21.15.32:0/3984369350 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fb8d77640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb406ce60 0x7f1fb4151f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.398 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fb9578640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb415f110 0x7f1fb40c7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fb3fff640 1 -- 172.21.15.32:0/3984369350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb406ce60 msgr2=0x7f1fb4151f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fb3fff640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb406ce60 0x7f1fb4151f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fb3fff640 1 -- 172.21.15.32:0/3984369350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb415f110 msgr2=0x7f1fb40c7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fb3fff640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb415f110 0x7f1fb40c7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.393+0000 7f1fb3fff640 1 -- 172.21.15.32:0/3984369350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f1fa8009480 con 0x7f1fb40af5e0 2023-12-20T06:04:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fb3fff640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb40af5e0 0x7f1fb40c71c0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c500 tx=0x7f1fa000c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fb17fa640 1 -- 172.21.15.32:0/3984369350 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0005220 con 0x7f1fb40af5e0 2023-12-20T06:04:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fb17fa640 1 -- 172.21.15.32:0/3984369350 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f1fa0005380 con 0x7f1fb40af5e0 2023-12-20T06:04:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fbafdb640 1 -- 172.21.15.32:0/3984369350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f1fb40b21c0 con 0x7f1fb40af5e0 2023-12-20T06:04:48.399 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fb17fa640 1 -- 172.21.15.32:0/3984369350 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f1fa0007270 con 0x7f1fb40af5e0 2023-12-20T06:04:48.400 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fbafdb640 1 -- 172.21.15.32:0/3984369350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f1fb40b2780 con 0x7f1fb40af5e0 2023-12-20T06:04:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fbafdb640 1 -- 172.21.15.32:0/3984369350 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f1fb40b2970 con 0x7f1fb40af5e0 2023-12-20T06:04:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fb17fa640 1 -- 172.21.15.32:0/3984369350 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f1fa00073d0 con 0x7f1fb40af5e0 2023-12-20T06:04:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fb17fa640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f98065b80 0x7f1f98068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fb17fa640 1 -- 172.21.15.32:0/3984369350 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f1fa00828d0 con 0x7f1fb40af5e0 2023-12-20T06:04:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fb8d77640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f98065b80 0x7f1f98068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.401 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.397+0000 7f1fb8d77640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f98065b80 0x7f1f98068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80ae000 tx=0x7f1fa80a7000 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.403 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.401+0000 7f1fb17fa640 1 -- 172.21.15.32:0/3984369350 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f1fa0050250 con 0x7f1fb40af5e0 2023-12-20T06:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 -- 172.21.15.32:0/3984369350 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "5", "target": ["mon-mgr", ""]}) v1 -- 0x7f1fb415c190 con 0x7f1f98065b80 2023-12-20T06:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fb17fa640 1 -- 172.21.15.32:0/3984369350 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 5 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f1fa80040c0 con 0x7f1f98065b80 2023-12-20T06:04:48.565 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 5 to deep-scrub 2023-12-20T06:04:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 -- 172.21.15.32:0/3984369350 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f98065b80 msgr2=0x7f1f98068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f98065b80 0x7f1f98068040 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f1fa80ae000 tx=0x7f1fa80a7000 comp rx=0 tx=0).stop 2023-12-20T06:04:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 -- 172.21.15.32:0/3984369350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb40af5e0 msgr2=0x7f1fb40c71c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb40af5e0 0x7f1fb40c71c0 secure :-1 s=READY pgs=2638 cs=0 l=1 rev1=1 crypto rx=0x7f1fa000c500 tx=0x7f1fa000c9c0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 -- 172.21.15.32:0/3984369350 shutdown_connections 2023-12-20T06:04:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f1f98065b80 0x7f1f98068040 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.568 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f1fb415f110 0x7f1fb40c7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f1fb40af5e0 0x7f1fb40c71c0 unknown :-1 s=CLOSED pgs=2638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 --2- 172.21.15.32:0/3984369350 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f1fb406ce60 0x7f1fb4151f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 -- 172.21.15.32:0/3984369350 >> 172.21.15.32:0/3984369350 conn(0x7f1fb406a320 msgr2=0x7f1fb40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:48.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 -- 172.21.15.32:0/3984369350 shutdown_connections 2023-12-20T06:04:48.569 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.565+0000 7f1fbafdb640 1 -- 172.21.15.32:0/3984369350 wait complete. 2023-12-20T06:04:48.587 INFO:tasks.ceph:Scrubbing osd.6 2023-12-20T06:04:48.587 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 config set osd_debug_deep_scrub_sleep 0 2023-12-20T06:04:48.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.733+0000 7f254a1a2640 1 Processor -- start 2023-12-20T06:04:48.735 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.733+0000 7f254a1a2640 1 -- start start 2023-12-20T06:04:48.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.733+0000 7f254a1a2640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f25441613a0 0x7f25440cc940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.733+0000 7f254a1a2640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25440cd400 con 0x7f254406c900 2023-12-20T06:04:48.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.733+0000 7f254a1a2640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f25441608c0 con 0x7f25440cce80 2023-12-20T06:04:48.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.733+0000 7f254a1a2640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f2544160f80 con 0x7f25441613a0 2023-12-20T06:04:48.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.737+0000 7f25491a0640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f254406c900 0x7f254406cce0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:40812/0 (socket says 172.21.15.32:40812) 2023-12-20T06:04:48.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.737+0000 7f25491a0640 1 -- 172.21.15.32:0/3371146540 learned_addr learned my addr 172.21.15.32:0/3371146540 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:48.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.737+0000 7f254899f640 1 --2- 172.21.15.32:0/3371146540 >> v2:172.21.15.39:3300/0 conn(0x7f25441613a0 0x7f25440cc940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.738 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.737+0000 7f253bfff640 1 -- 172.21.15.32:0/3371146540 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1194000436 0 0) 0x7f25440cd400 con 0x7f254406c900 2023-12-20T06:04:48.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.737+0000 7f253bfff640 1 -- 172.21.15.32:0/3371146540 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f2530003530 con 0x7f254406c900 2023-12-20T06:04:48.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.737+0000 7f254899f640 1 -- 172.21.15.32:0/3371146540 >> v1:172.21.15.39:6789/0 conn(0x7f25440cce80 legacy=0x7f254415d0a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.737+0000 7f253bfff640 1 -- 172.21.15.32:0/3371146540 <== mon.0 v1:172.21.15.32:6789/0 2 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 214+0+0 (unknown 172586949 0 0) 0x7f2530003530 con 0x7f254406c900 2023-12-20T06:04:48.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.737+0000 7f254899f640 1 -- 172.21.15.32:0/3371146540 >> v1:172.21.15.32:6789/0 conn(0x7f254406c900 legacy=0x7f254406cce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.739 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.737+0000 7f254899f640 1 -- 172.21.15.32:0/3371146540 --> v2:172.21.15.39:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2544160ba0 con 0x7f25441613a0 2023-12-20T06:04:48.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.737+0000 7f254899f640 1 --2- 172.21.15.32:0/3371146540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25441613a0 0x7f25440cc940 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f25340097c0 tx=0x7f253401c310 comp rx=0 tx=0).ready entity=mon.1 client_cookie=a65fe0eca7d2c1b6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f253bfff640 1 -- 172.21.15.32:0/3371146540 <== mon.1 v2:172.21.15.39:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2540001680 con 0x7f25441613a0 2023-12-20T06:04:48.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f253bfff640 1 -- 172.21.15.32:0/3371146540 <== mon.1 v2:172.21.15.39:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f25340a7050 con 0x7f25441613a0 2023-12-20T06:04:48.741 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f253bfff640 1 -- 172.21.15.32:0/3371146540 <== mon.1 v2:172.21.15.39:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f25340ab620 con 0x7f25441613a0 2023-12-20T06:04:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 -- 172.21.15.32:0/3371146540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25441613a0 msgr2=0x7f25440cc940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 --2- 172.21.15.32:0/3371146540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25441613a0 0x7f25440cc940 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7f25340097c0 tx=0x7f253401c310 comp rx=0 tx=0).stop 2023-12-20T06:04:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 -- 172.21.15.32:0/3371146540 shutdown_connections 2023-12-20T06:04:48.742 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 --2- 172.21.15.32:0/3371146540 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25441613a0 0x7f25440cc940 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 -- 172.21.15.32:0/3371146540 >> 172.21.15.32:0/3371146540 conn(0x7f25440cb880 msgr2=0x7f25440cbc90 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 -- 172.21.15.32:0/3371146540 shutdown_connections 2023-12-20T06:04:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 -- 172.21.15.32:0/3371146540 wait complete. 2023-12-20T06:04:48.743 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 Processor -- start 2023-12-20T06:04:48.744 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 -- start start 2023-12-20T06:04:48.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254406c900 0x7f25441542d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25440cce80 0x7f25440c85e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25441613a0 0x7f25440accd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f25440ac840 con 0x7f25440cce80 2023-12-20T06:04:48.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f2544156940 con 0x7f25441613a0 2023-12-20T06:04:48.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254a1a2640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f25440c8b20 con 0x7f254406c900 2023-12-20T06:04:48.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254899f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25440cce80 0x7f25440c85e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254899f640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25440cce80 0x7f25440c85e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:51526/0 (socket says 172.21.15.32:51526) 2023-12-20T06:04:48.747 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.741+0000 7f254899f640 1 -- 172.21.15.32:0/979483631 learned_addr learned my addr 172.21.15.32:0/979483631 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:48.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f25491a0640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254406c900 0x7f25441542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f25499a1640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25441613a0 0x7f25440accd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f254899f640 1 -- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254406c900 msgr2=0x7f25441542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f254899f640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254406c900 0x7f25441542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f254899f640 1 -- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25441613a0 msgr2=0x7f25440accd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f254899f640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f25441613a0 0x7f25440accd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f254899f640 1 -- 172.21.15.32:0/979483631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f2534009420 con 0x7f25440cce80 2023-12-20T06:04:48.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f25491a0640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f254406c900 0x7f25441542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:48.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f254899f640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25440cce80 0x7f25440c85e0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f2534018f20 tx=0x7f25340190b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.748 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f2539ffb640 1 -- 172.21.15.32:0/979483631 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534004020 con 0x7f25440cce80 2023-12-20T06:04:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f2539ffb640 1 -- 172.21.15.32:0/979483631 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f2534004180 con 0x7f25440cce80 2023-12-20T06:04:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f2539ffb640 1 -- 172.21.15.32:0/979483631 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f2534004510 con 0x7f25440cce80 2023-12-20T06:04:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f254a1a2640 1 -- 172.21.15.32:0/979483631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f25440abc30 con 0x7f25440cce80 2023-12-20T06:04:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f254a1a2640 1 -- 172.21.15.32:0/979483631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f25440ac1f0 con 0x7f25440cce80 2023-12-20T06:04:48.749 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f254a1a2640 1 -- 172.21.15.32:0/979483631 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f2544074100 con 0x7f25440cce80 2023-12-20T06:04:48.750 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f2539ffb640 1 -- 172.21.15.32:0/979483631 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f25340b2040 con 0x7f25440cce80 2023-12-20T06:04:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.745+0000 7f2539ffb640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2528065bb0 0x7f2528068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.749+0000 7f2539ffb640 1 -- 172.21.15.32:0/979483631 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f253411d480 con 0x7f25440cce80 2023-12-20T06:04:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.749+0000 7f25491a0640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2528065bb0 0x7f2528068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.749+0000 7f2539ffb640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] conn(0x7f252806c000 0x7f252806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.749+0000 7f2539ffb640 1 -- 172.21.15.32:0/979483631 --> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f252806eb10 con 0x7f252806c000 2023-12-20T06:04:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.749+0000 7f2539ffb640 1 -- 172.21.15.32:0/979483631 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=256) v2 ==== 24+0+0 (secure 0 0 0) 0x7f253411fa60 con 0x7f25440cce80 2023-12-20T06:04:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.749+0000 7f25499a1640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] conn(0x7f252806c000 0x7f252806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.749+0000 7f25491a0640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2528065bb0 0x7f2528068070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2540000f70 tx=0x7f2540009040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.751 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.749+0000 7f25499a1640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] conn(0x7f252806c000 0x7f252806e440 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.752 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.749+0000 7f2539ffb640 1 -- 172.21.15.32:0/979483631 <== osd.6 v2:172.21.15.39:6816/38281 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f252806eb10 con 0x7f252806c000 2023-12-20T06:04:48.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.761+0000 7f254a1a2640 1 -- 172.21.15.32:0/979483631 --> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f25440ac980 con 0x7f252806c000 2023-12-20T06:04:48.764 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.761+0000 7f2539ffb640 1 -- 172.21.15.32:0/979483631 <== osd.6 v2:172.21.15.39:6816/38281 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f25440ac980 con 0x7f252806c000 2023-12-20T06:04:48.765 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:04:48.765 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-20T06:04:48.765 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:04:48.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.761+0000 7f254a1a2640 1 -- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] conn(0x7f252806c000 msgr2=0x7f252806e440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.761+0000 7f254a1a2640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.39:6816/38281,v1:172.21.15.39:6817/38281] conn(0x7f252806c000 0x7f252806e440 crc :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.765+0000 7f254a1a2640 1 -- 172.21.15.32:0/979483631 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2528065bb0 msgr2=0x7f2528068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.766 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.765+0000 7f254a1a2640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f2528065bb0 0x7f2528068070 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7f2540000f70 tx=0x7f2540009040 comp rx=0 tx=0).stop 2023-12-20T06:04:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.765+0000 7f254a1a2640 1 -- 172.21.15.32:0/979483631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25440cce80 msgr2=0x7f25440c85e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.765+0000 7f254a1a2640 1 --2- 172.21.15.32:0/979483631 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f25440cce80 0x7f25440c85e0 secure :-1 s=READY pgs=2640 cs=0 l=1 rev1=1 crypto rx=0x7f2534018f20 tx=0x7f25340190b0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.765+0000 7f25499a1640 1 -- 172.21.15.32:0/979483631 reap_dead start 2023-12-20T06:04:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.765+0000 7f254a1a2640 1 -- 172.21.15.32:0/979483631 shutdown_connections 2023-12-20T06:04:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.765+0000 7f254a1a2640 1 -- 172.21.15.32:0/979483631 >> 172.21.15.32:0/979483631 conn(0x7f25440cb880 msgr2=0x7f254415d0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.765+0000 7f254a1a2640 1 -- 172.21.15.32:0/979483631 shutdown_connections 2023-12-20T06:04:48.767 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.765+0000 7f254a1a2640 1 -- 172.21.15.32:0/979483631 wait complete. 2023-12-20T06:04:48.783 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 6 2023-12-20T06:04:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7d6a41640 1 Processor -- start 2023-12-20T06:04:48.929 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7d6a41640 1 -- start start 2023-12-20T06:04:48.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7d6a41640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7d01599e0 0x7fd7d00aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7d6a41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d00c90e0 0x7fd7d00b05f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7d6a41640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7fd7d00711d0 con 0x7fd7d00b0b30 2023-12-20T06:04:48.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7d6a41640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7fd7d00c98c0 con 0x7fd7d01599e0 2023-12-20T06:04:48.930 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7d6a41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d00c9ba0 con 0x7fd7d00c90e0 2023-12-20T06:04:48.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7cf7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d00c90e0 0x7fd7d00b05f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7cffff640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7fd7d01599e0 0x7fd7d00aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7cf7fe640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d00c90e0 0x7fd7d00b05f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:40376/0 (socket says 172.21.15.32:40376) 2023-12-20T06:04:48.931 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7d4fde640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7fd7d00b0b30 0x7fd7d015d6e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:48010/0 (socket says 172.21.15.32:48010) 2023-12-20T06:04:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7cf7fe640 1 -- 172.21.15.32:0/4165216734 learned_addr learned my addr 172.21.15.32:0/4165216734 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7ceffd640 1 -- 172.21.15.32:0/4165216734 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 4086110380 0 0) 0x7fd7d00711d0 con 0x7fd7d00b0b30 2023-12-20T06:04:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7ceffd640 1 -- 172.21.15.32:0/4165216734 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7fd7b8003530 con 0x7fd7d00b0b30 2023-12-20T06:04:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7cf7fe640 1 -- 172.21.15.32:0/4165216734 >> v2:172.21.15.39:3300/0 conn(0x7fd7d01599e0 msgr2=0x7fd7d00aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7cf7fe640 1 --2- 172.21.15.32:0/4165216734 >> v2:172.21.15.39:3300/0 conn(0x7fd7d01599e0 0x7fd7d00aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7cf7fe640 1 -- 172.21.15.32:0/4165216734 >> v1:172.21.15.39:6789/0 conn(0x7fd7d00b0b30 legacy=0x7fd7d015d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7cf7fe640 1 -- 172.21.15.32:0/4165216734 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7d00c9e80 con 0x7fd7d00c90e0 2023-12-20T06:04:48.932 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.929+0000 7fd7cffff640 1 --2- 172.21.15.32:0/4165216734 >> v2:172.21.15.39:3300/0 conn(0x7fd7d01599e0 0x7fd7d00aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2023-12-20T06:04:48.934 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7cf7fe640 1 --2- 172.21.15.32:0/4165216734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d00c90e0 0x7fd7d00b05f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009870 tx=0x7fd7c401c510 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b2331316a58acbfb server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7ceffd640 1 -- 172.21.15.32:0/4165216734 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40a7070 con 0x7fd7d00c90e0 2023-12-20T06:04:48.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7ceffd640 1 -- 172.21.15.32:0/4165216734 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c40ab430 con 0x7fd7d00c90e0 2023-12-20T06:04:48.935 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7ceffd640 1 -- 172.21.15.32:0/4165216734 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40ab7c0 con 0x7fd7d00c90e0 2023-12-20T06:04:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7d6a41640 1 -- 172.21.15.32:0/4165216734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d00c90e0 msgr2=0x7fd7d00b05f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7d6a41640 1 --2- 172.21.15.32:0/4165216734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d00c90e0 0x7fd7d00b05f0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7fd7c4009870 tx=0x7fd7c401c510 comp rx=0 tx=0).stop 2023-12-20T06:04:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7d6a41640 1 -- 172.21.15.32:0/4165216734 shutdown_connections 2023-12-20T06:04:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7d6a41640 1 --2- 172.21.15.32:0/4165216734 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d00c90e0 0x7fd7d00b05f0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7d6a41640 1 --2- 172.21.15.32:0/4165216734 >> v2:172.21.15.39:3300/0 conn(0x7fd7d01599e0 0x7fd7d00aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7d6a41640 1 -- 172.21.15.32:0/4165216734 >> 172.21.15.32:0/4165216734 conn(0x7fd7d006a320 msgr2=0x7fd7d00ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7d6a41640 1 -- 172.21.15.32:0/4165216734 shutdown_connections 2023-12-20T06:04:48.936 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.933+0000 7fd7d6a41640 1 -- 172.21.15.32:0/4165216734 wait complete. 2023-12-20T06:04:48.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7d6a41640 1 Processor -- start 2023-12-20T06:04:48.937 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7d6a41640 1 -- start start 2023-12-20T06:04:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7d6a41640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d00b0b30 0x7fd7d014b4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7d6a41640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d00c90e0 0x7fd7d00b3940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7d6a41640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d01599e0 0x7fd7d0071350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7d6a41640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d015acd0 con 0x7fd7d00c90e0 2023-12-20T06:04:48.939 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7d6a41640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d015afb0 con 0x7fd7d00b0b30 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7d6a41640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7fd7d015b290 con 0x7fd7d01599e0 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7cf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d00c90e0 0x7fd7d00b3940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7cf7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d00c90e0 0x7fd7d00b3940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34076/0 (socket says 172.21.15.32:34076) 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7cf7fe640 1 -- 172.21.15.32:0/2183940964 learned_addr learned my addr 172.21.15.32:0/2183940964 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7cffff640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d00b0b30 0x7fd7d014b4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7d4fde640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d01599e0 0x7fd7d0071350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7cf7fe640 1 -- 172.21.15.32:0/2183940964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d01599e0 msgr2=0x7fd7d0071350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7cf7fe640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d01599e0 0x7fd7d0071350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7cf7fe640 1 -- 172.21.15.32:0/2183940964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d00b0b30 msgr2=0x7fd7d014b4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7cf7fe640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d00b0b30 0x7fd7d014b4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7cf7fe640 1 -- 172.21.15.32:0/2183940964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7fd7c00093b0 con 0x7fd7d00c90e0 2023-12-20T06:04:48.940 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7cf7fe640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d00c90e0 0x7fd7d00b3940 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40ad040 tx=0x7fd7c40192b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7ccff9640 1 -- 172.21.15.32:0/2183940964 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40bc070 con 0x7fd7d00c90e0 2023-12-20T06:04:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7ccff9640 1 -- 172.21.15.32:0/2183940964 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7fd7c40abe20 con 0x7fd7d00c90e0 2023-12-20T06:04:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7d6a41640 1 -- 172.21.15.32:0/2183940964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7fd7c40094d0 con 0x7fd7d00c90e0 2023-12-20T06:04:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7ccff9640 1 -- 172.21.15.32:0/2183940964 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7fd7c40b4660 con 0x7fd7d00c90e0 2023-12-20T06:04:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.937+0000 7fd7d6a41640 1 -- 172.21.15.32:0/2183940964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7fd7d015b850 con 0x7fd7d00c90e0 2023-12-20T06:04:48.941 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.941+0000 7fd7d6a41640 1 -- 172.21.15.32:0/2183940964 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7fd7d015ba40 con 0x7fd7d00c90e0 2023-12-20T06:04:48.942 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.941+0000 7fd7ccff9640 1 -- 172.21.15.32:0/2183940964 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7fd7c40b47c0 con 0x7fd7d00c90e0 2023-12-20T06:04:48.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.941+0000 7fd7ccff9640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd7bc065b80 0x7fd7bc068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:48.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.941+0000 7fd7ccff9640 1 -- 172.21.15.32:0/2183940964 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7fd7c411cc00 con 0x7fd7d00c90e0 2023-12-20T06:04:48.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.941+0000 7fd7cffff640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd7bc065b80 0x7fd7bc068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:48.943 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.941+0000 7fd7cffff640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd7bc065b80 0x7fd7bc068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009c30 tx=0x7fd7c0006f90 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:48.947 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:48.945+0000 7fd7ccff9640 1 -- 172.21.15.32:0/2183940964 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7fd7c40ea3a0 con 0x7fd7d00c90e0 2023-12-20T06:04:49.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.101+0000 7fd7d6a41640 1 -- 172.21.15.32:0/2183940964 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "6", "target": ["mon-mgr", ""]}) v1 -- 0x7fd7d00c9540 con 0x7fd7bc065b80 2023-12-20T06:04:49.104 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.101+0000 7fd7ccff9640 1 -- 172.21.15.32:0/2183940964 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 6 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7fd7c0009880 con 0x7fd7bc065b80 2023-12-20T06:04:49.104 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 6 to deep-scrub 2023-12-20T06:04:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 -- 172.21.15.32:0/2183940964 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd7bc065b80 msgr2=0x7fd7bc068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd7bc065b80 0x7fd7bc068040 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fd7c0009c30 tx=0x7fd7c0006f90 comp rx=0 tx=0).stop 2023-12-20T06:04:49.106 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 -- 172.21.15.32:0/2183940964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d00c90e0 msgr2=0x7fd7d00b3940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d00c90e0 0x7fd7d00b3940 secure :-1 s=READY pgs=2641 cs=0 l=1 rev1=1 crypto rx=0x7fd7c40ad040 tx=0x7fd7c40192b0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 -- 172.21.15.32:0/2183940964 shutdown_connections 2023-12-20T06:04:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7fd7bc065b80 0x7fd7bc068040 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7fd7d01599e0 0x7fd7d0071350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7fd7d00c90e0 0x7fd7d00b3940 unknown :-1 s=CLOSED pgs=2641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 --2- 172.21.15.32:0/2183940964 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7fd7d00b0b30 0x7fd7d014b4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 -- 172.21.15.32:0/2183940964 >> 172.21.15.32:0/2183940964 conn(0x7fd7d006a320 msgr2=0x7fd7d0154dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 -- 172.21.15.32:0/2183940964 shutdown_connections 2023-12-20T06:04:49.107 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.105+0000 7fd7d6a41640 1 -- 172.21.15.32:0/2183940964 wait complete. 2023-12-20T06:04:49.126 INFO:tasks.ceph:Scrubbing osd.7 2023-12-20T06:04:49.126 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 config set osd_debug_deep_scrub_sleep 0 2023-12-20T06:04:49.273 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c21820640 1 Processor -- start 2023-12-20T06:04:49.274 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c21820640 1 -- start start 2023-12-20T06:04:49.275 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c21820640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f4c1c159250 0x7f4c1c0caa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c21820640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c1c0ce290 0x7f4c1c070230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c21820640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c1c070770 0x7f4c1c15cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c21820640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c070be0 con 0x7f4c1c0ce290 2023-12-20T06:04:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c21820640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f4c1c159630 con 0x7f4c1c159250 2023-12-20T06:04:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c21820640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0c9480 con 0x7f4c1c070770 2023-12-20T06:04:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1a7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c1c0ce290 0x7f4c1c070230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1a7fc640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f4c1c0ce290 0x7f4c1c070230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34082/0 (socket says 172.21.15.32:34082) 2023-12-20T06:04:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1a7fc640 1 -- 172.21.15.32:0/1452930248 learned_addr learned my addr 172.21.15.32:0/1452930248 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1b7fe640 1 --2- 172.21.15.32:0/1452930248 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c1c070770 0x7f4c1c15cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.276 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1affd640 1 --2- 172.21.15.32:0/1452930248 >> v2:172.21.15.39:3300/0 conn(0x7f4c1c159250 0x7f4c1c0caa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1a7fc640 1 -- 172.21.15.32:0/1452930248 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c1c070770 msgr2=0x7f4c1c15cd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:49.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1a7fc640 1 --2- 172.21.15.32:0/1452930248 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c1c070770 0x7f4c1c15cd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1a7fc640 1 -- 172.21.15.32:0/1452930248 >> v2:172.21.15.39:3300/0 conn(0x7f4c1c159250 msgr2=0x7f4c1c0caa30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:49.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1a7fc640 1 --2- 172.21.15.32:0/1452930248 >> v2:172.21.15.39:3300/0 conn(0x7f4c1c159250 0x7f4c1c0caa30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1a7fc640 1 -- 172.21.15.32:0/1452930248 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c1c06cf90 con 0x7f4c1c0ce290 2023-12-20T06:04:49.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1b7fe640 1 --2- 172.21.15.32:0/1452930248 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c1c070770 0x7f4c1c15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:49.277 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.273+0000 7f4c1affd640 1 --2- 172.21.15.32:0/1452930248 >> v2:172.21.15.39:3300/0 conn(0x7f4c1c159250 0x7f4c1c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:49.278 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c1a7fc640 1 --2- 172.21.15.32:0/1452930248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1c0ce290 0x7f4c1c070230 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c06e390 tx=0x7f4c1001c4d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41f77beb35f45d31 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c19ffb640 1 -- 172.21.15.32:0/1452930248 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c100ab610 con 0x7f4c1c0ce290 2023-12-20T06:04:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c19ffb640 1 -- 172.21.15.32:0/1452930248 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c100ab770 con 0x7f4c1c0ce290 2023-12-20T06:04:49.279 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c19ffb640 1 -- 172.21.15.32:0/1452930248 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c100abb00 con 0x7f4c1c0ce290 2023-12-20T06:04:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c21820640 1 -- 172.21.15.32:0/1452930248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1c0ce290 msgr2=0x7f4c1c070230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c21820640 1 --2- 172.21.15.32:0/1452930248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1c0ce290 0x7f4c1c070230 secure :-1 s=READY pgs=2642 cs=0 l=1 rev1=1 crypto rx=0x7f4c1c06e390 tx=0x7f4c1001c4d0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c21820640 1 -- 172.21.15.32:0/1452930248 shutdown_connections 2023-12-20T06:04:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c21820640 1 --2- 172.21.15.32:0/1452930248 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c1c070770 0x7f4c1c15cd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c21820640 1 --2- 172.21.15.32:0/1452930248 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1c0ce290 0x7f4c1c070230 unknown :-1 s=CLOSED pgs=2642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c21820640 1 --2- 172.21.15.32:0/1452930248 >> v2:172.21.15.39:3300/0 conn(0x7f4c1c159250 0x7f4c1c0caa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c21820640 1 -- 172.21.15.32:0/1452930248 >> 172.21.15.32:0/1452930248 conn(0x7f4c1c06a510 msgr2=0x7f4c1c0ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c21820640 1 -- 172.21.15.32:0/1452930248 shutdown_connections 2023-12-20T06:04:49.280 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.277+0000 7f4c21820640 1 -- 172.21.15.32:0/1452930248 wait complete. 2023-12-20T06:04:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c21820640 1 Processor -- start 2023-12-20T06:04:49.281 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c21820640 1 -- start start 2023-12-20T06:04:49.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c21820640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c1c070770 0x7f4c1c1410f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.283 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c21820640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c1c0ce290 0x7f4c1c0b2340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c21820640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1c159250 0x7f4c1c0b2cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c21820640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0b1eb0 con 0x7f4c1c159250 2023-12-20T06:04:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c21820640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c15bff0 con 0x7f4c1c070770 2023-12-20T06:04:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c21820640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f4c1c0b1580 con 0x7f4c1c0ce290 2023-12-20T06:04:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c1b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1c159250 0x7f4c1c0b2cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c1b7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1c159250 0x7f4c1c0b2cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34092/0 (socket says 172.21.15.32:34092) 2023-12-20T06:04:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c1b7fe640 1 -- 172.21.15.32:0/1536957736 learned_addr learned my addr 172.21.15.32:0/1536957736 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c1a7fc640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c1c0ce290 0x7f4c1c0b2340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c1affd640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c1c070770 0x7f4c1c1410f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.284 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c1b7fe640 1 -- 172.21.15.32:0/1536957736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c1c0ce290 msgr2=0x7f4c1c0b2340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c1b7fe640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4c1c0ce290 0x7f4c1c0b2340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c1b7fe640 1 -- 172.21.15.32:0/1536957736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c1c070770 msgr2=0x7f4c1c1410f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c1b7fe640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f4c1c070770 0x7f4c1c1410f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c1b7fe640 1 -- 172.21.15.32:0/1536957736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f4c100094d0 con 0x7f4c1c159250 2023-12-20T06:04:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c1b7fe640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1c159250 0x7f4c1c0b2cb0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f4c14007d20 tx=0x7f4c1400d490 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c03fff640 1 -- 172.21.15.32:0/1536957736 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c1400f040 con 0x7f4c1c159250 2023-12-20T06:04:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c03fff640 1 -- 172.21.15.32:0/1536957736 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f4c1400dda0 con 0x7f4c1c159250 2023-12-20T06:04:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c03fff640 1 -- 172.21.15.32:0/1536957736 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f4c14003bc0 con 0x7f4c1c159250 2023-12-20T06:04:49.285 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c21820640 1 -- 172.21.15.32:0/1536957736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4c1c0b1860 con 0x7f4c1c159250 2023-12-20T06:04:49.286 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c21820640 1 -- 172.21.15.32:0/1536957736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f4c1c141630 con 0x7f4c1c159250 2023-12-20T06:04:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.281+0000 7f4c21820640 1 -- 172.21.15.32:0/1536957736 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_get_version(what=osdmap handle=1) v1 -- 0x7f4c1c0caf70 con 0x7f4c1c159250 2023-12-20T06:04:49.287 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.285+0000 7f4c03fff640 1 -- 172.21.15.32:0/1536957736 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f4c14003d20 con 0x7f4c1c159250 2023-12-20T06:04:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.285+0000 7f4c03fff640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4c08065bb0 0x7f4c08068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.285+0000 7f4c03fff640 1 -- 172.21.15.32:0/1536957736 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f4c14083ff0 con 0x7f4c1c159250 2023-12-20T06:04:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.285+0000 7f4c1affd640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4c08065bb0 0x7f4c08068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.285+0000 7f4c03fff640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] conn(0x7f4c0806c000 0x7f4c0806e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.285+0000 7f4c03fff640 1 -- 172.21.15.32:0/1536957736 --> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] -- command(tid 1: {"prefix": "get_command_descriptions"}) v1 -- 0x7f4c0806eb10 con 0x7f4c0806c000 2023-12-20T06:04:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.285+0000 7f4c03fff640 1 -- 172.21.15.32:0/1536957736 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_get_version_reply(handle=1 version=256) v2 ==== 24+0+0 (secure 0 0 0) 0x7f4c14084360 con 0x7f4c1c159250 2023-12-20T06:04:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.285+0000 7f4c1a7fc640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] conn(0x7f4c0806c000 0x7f4c0806e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.285+0000 7f4c1affd640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4c08065bb0 0x7f4c08068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0045f0 tx=0x7f4c0c008040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:49.288 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.285+0000 7f4c1a7fc640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] conn(0x7f4c0806c000 0x7f4c0806e440 crc :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:49.289 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.285+0000 7f4c03fff640 1 -- 172.21.15.32:0/1536957736 <== osd.7 v2:172.21.15.39:6808/38279 1 ==== command_reply(tid 1: 0 ) v1 ==== 8+0+25106 (crc 0 0 0) 0x7f4c0806eb10 con 0x7f4c0806c000 2023-12-20T06:04:49.301 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c21820640 1 -- 172.21.15.32:0/1536957736 --> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] -- command(tid 2: {"prefix": "config set", "var": "osd_debug_deep_scrub_sleep", "val": ["0"]}) v1 -- 0x7f4c1c0c9110 con 0x7f4c0806c000 2023-12-20T06:04:49.302 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c03fff640 1 -- 172.21.15.32:0/1536957736 <== osd.7 v2:172.21.15.39:6808/38279 2 ==== command_reply(tid 2: 0 ) v1 ==== 8+0+105 (crc 0 0 0) 0x7f4c10004040 con 0x7f4c0806c000 2023-12-20T06:04:49.302 INFO:teuthology.orchestra.run.smithi032.stdout:{ 2023-12-20T06:04:49.302 INFO:teuthology.orchestra.run.smithi032.stdout: "success": "osd_debug_deep_scrub_sleep = '0.000000' (not observed, change may require restart) " 2023-12-20T06:04:49.302 INFO:teuthology.orchestra.run.smithi032.stdout:} 2023-12-20T06:04:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c21820640 1 -- 172.21.15.32:0/1536957736 >> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] conn(0x7f4c0806c000 msgr2=0x7f4c0806e440 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c21820640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.39:6808/38279,v1:172.21.15.39:6809/38279] conn(0x7f4c0806c000 0x7f4c0806e440 crc :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c21820640 1 -- 172.21.15.32:0/1536957736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4c08065bb0 msgr2=0x7f4c08068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c21820640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f4c08065bb0 0x7f4c08068070 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f4c0c0045f0 tx=0x7f4c0c008040 comp rx=0 tx=0).stop 2023-12-20T06:04:49.303 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c21820640 1 -- 172.21.15.32:0/1536957736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1c159250 msgr2=0x7f4c1c0b2cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c21820640 1 --2- 172.21.15.32:0/1536957736 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f4c1c159250 0x7f4c1c0b2cb0 secure :-1 s=READY pgs=2643 cs=0 l=1 rev1=1 crypto rx=0x7f4c14007d20 tx=0x7f4c1400d490 comp rx=0 tx=0).stop 2023-12-20T06:04:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c1b7fe640 1 -- 172.21.15.32:0/1536957736 reap_dead start 2023-12-20T06:04:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c21820640 1 -- 172.21.15.32:0/1536957736 shutdown_connections 2023-12-20T06:04:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c21820640 1 -- 172.21.15.32:0/1536957736 >> 172.21.15.32:0/1536957736 conn(0x7f4c1c06a510 msgr2=0x7f4c1c0ce690 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c21820640 1 -- 172.21.15.32:0/1536957736 shutdown_connections 2023-12-20T06:04:49.304 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.301+0000 7f4c21820640 1 -- 172.21.15.32:0/1536957736 wait complete. 2023-12-20T06:04:49.321 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 7 2023-12-20T06:04:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd9066640 1 Processor -- start 2023-12-20T06:04:49.465 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd9066640 1 -- start start 2023-12-20T06:04:49.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd9066640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cd40b0d00 0x7f9cd4164a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd9066640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd40711d0 con 0x7f9cd41599e0 2023-12-20T06:04:49.467 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd9066640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f9cd40c98c0 con 0x7f9cd40aece0 2023-12-20T06:04:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd9066640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd40c9ba0 con 0x7f9cd40b0d00 2023-12-20T06:04:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd2d9d640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f9cd41599e0 0x7f9cd40c90e0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:39894/0 (socket says 172.21.15.32:39894) 2023-12-20T06:04:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd2d9d640 1 -- 172.21.15.32:0/3764773626 learned_addr learned my addr 172.21.15.32:0/3764773626 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:04:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd359e640 1 --2- 172.21.15.32:0/3764773626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cd40b0d00 0x7f9cd4164a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd1d9b640 1 -- 172.21.15.32:0/3764773626 <== mon.0 v1:172.21.15.32:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2915760332 0 0) 0x7f9cd40711d0 con 0x7f9cd41599e0 2023-12-20T06:04:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd1d9b640 1 -- 172.21.15.32:0/3764773626 --> v1:172.21.15.32:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cbc003530 con 0x7f9cd41599e0 2023-12-20T06:04:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd1d9b640 1 -- 172.21.15.32:0/3764773626 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 2338294674 0 0) 0x7f9cd40c98c0 con 0x7f9cd40aece0 2023-12-20T06:04:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd1d9b640 1 -- 172.21.15.32:0/3764773626 --> v1:172.21.15.39:6789/0 -- auth(proto 2 36 bytes epoch 0) v1 -- 0x7f9cd40711d0 con 0x7f9cd40aece0 2023-12-20T06:04:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd359e640 1 -- 172.21.15.32:0/3764773626 >> v1:172.21.15.39:6789/0 conn(0x7f9cd40aece0 legacy=0x7f9cd40b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd359e640 1 -- 172.21.15.32:0/3764773626 >> v1:172.21.15.32:6789/0 conn(0x7f9cd41599e0 legacy=0x7f9cd40c90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.468 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.465+0000 7f9cd359e640 1 -- 172.21.15.32:0/3764773626 --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9cd40c9e80 con 0x7f9cd40b0d00 2023-12-20T06:04:49.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.469+0000 7f9cd359e640 1 --2- 172.21.15.32:0/3764773626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cd40b0d00 0x7f9cd4164a70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009dc0 tx=0x7f9ccc01c8a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=46a6badcbdd2284e server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:49.470 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.469+0000 7f9cd1d9b640 1 -- 172.21.15.32:0/3764773626 <== mon.2 v2:172.21.15.39:3301/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc0a5850 con 0x7f9cd40b0d00 2023-12-20T06:04:49.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.469+0000 7f9cd1d9b640 1 -- 172.21.15.32:0/3764773626 <== mon.2 v2:172.21.15.39:3301/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9ccc0a59b0 con 0x7f9cd40b0d00 2023-12-20T06:04:49.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.469+0000 7f9cd1d9b640 1 -- 172.21.15.32:0/3764773626 <== mon.2 v2:172.21.15.39:3301/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9ccc0a5d20 con 0x7f9cd40b0d00 2023-12-20T06:04:49.471 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.469+0000 7f9cd9066640 1 -- 172.21.15.32:0/3764773626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cd40b0d00 msgr2=0x7f9cd4164a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.469+0000 7f9cd9066640 1 --2- 172.21.15.32:0/3764773626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cd40b0d00 0x7f9cd4164a70 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f9ccc009dc0 tx=0x7f9ccc01c8a0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.469+0000 7f9cd9066640 1 -- 172.21.15.32:0/3764773626 shutdown_connections 2023-12-20T06:04:49.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.469+0000 7f9cd9066640 1 --2- 172.21.15.32:0/3764773626 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cd40b0d00 0x7f9cd4164a70 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.469+0000 7f9cd9066640 1 -- 172.21.15.32:0/3764773626 >> 172.21.15.32:0/3764773626 conn(0x7f9cd406a320 msgr2=0x7f9cd40ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:49.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.469+0000 7f9cd9066640 1 -- 172.21.15.32:0/3764773626 shutdown_connections 2023-12-20T06:04:49.472 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.469+0000 7f9cd9066640 1 -- 172.21.15.32:0/3764773626 wait complete. 2023-12-20T06:04:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd9066640 1 Processor -- start 2023-12-20T06:04:49.473 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd9066640 1 -- start start 2023-12-20T06:04:49.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd9066640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cd40aece0 0x7f9cd415bbd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd9066640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40b0d00 0x7f9cd414f170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd9066640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cd41599e0 0x7f9cd414f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd9066640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd41500d0 con 0x7f9cd40b0d00 2023-12-20T06:04:49.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd9066640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd415d0e0 con 0x7f9cd41599e0 2023-12-20T06:04:49.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd9066640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f9cd414fbf0 con 0x7f9cd40aece0 2023-12-20T06:04:49.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40b0d00 0x7f9cd414f170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.476 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd259c640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40b0d00 0x7f9cd414f170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34106/0 (socket says 172.21.15.32:34106) 2023-12-20T06:04:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd259c640 1 -- 172.21.15.32:0/1235680068 learned_addr learned my addr 172.21.15.32:0/1235680068 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd2d9d640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cd40aece0 0x7f9cd415bbd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd359e640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cd41599e0 0x7f9cd414f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd259c640 1 -- 172.21.15.32:0/1235680068 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cd40aece0 msgr2=0x7f9cd415bbd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd259c640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cd40aece0 0x7f9cd415bbd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd259c640 1 -- 172.21.15.32:0/1235680068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cd41599e0 msgr2=0x7f9cd414f6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd259c640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cd41599e0 0x7f9cd414f6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd259c640 1 -- 172.21.15.32:0/1235680068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f9ccc009a70 con 0x7f9cd40b0d00 2023-12-20T06:04:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd259c640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40b0d00 0x7f9cd414f170 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800c470 tx=0x7f9cc800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:49.477 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cbb7fe640 1 -- 172.21.15.32:0/1235680068 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc8005480 con 0x7f9cd40b0d00 2023-12-20T06:04:49.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cbb7fe640 1 -- 172.21.15.32:0/1235680068 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f9cc8003600 con 0x7f9cd40b0d00 2023-12-20T06:04:49.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cbb7fe640 1 -- 172.21.15.32:0/1235680068 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f9cc80039b0 con 0x7f9cd40b0d00 2023-12-20T06:04:49.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd9066640 1 -- 172.21.15.32:0/1235680068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f9cd415c540 con 0x7f9cd40b0d00 2023-12-20T06:04:49.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.473+0000 7f9cd9066640 1 -- 172.21.15.32:0/1235680068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f9cd415ca90 con 0x7f9cd40b0d00 2023-12-20T06:04:49.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.477+0000 7f9cd9066640 1 -- 172.21.15.32:0/1235680068 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f9cd40c6880 con 0x7f9cd40b0d00 2023-12-20T06:04:49.478 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.477+0000 7f9cbb7fe640 1 -- 172.21.15.32:0/1235680068 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f9cc8003b10 con 0x7f9cd40b0d00 2023-12-20T06:04:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.477+0000 7f9cbb7fe640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cc0065b80 0x7f9cc0068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.477+0000 7f9cbb7fe640 1 -- 172.21.15.32:0/1235680068 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f9cc8082850 con 0x7f9cd40b0d00 2023-12-20T06:04:49.479 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.477+0000 7f9cd2d9d640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cc0065b80 0x7f9cc0068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.480 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.477+0000 7f9cd2d9d640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cc0065b80 0x7f9cc0068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40010f0 tx=0x7f9cc4007bf0 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:49.483 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.481+0000 7f9cbb7fe640 1 -- 172.21.15.32:0/1235680068 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f9cd40c6880 con 0x7f9cd40b0d00 2023-12-20T06:04:49.642 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 -- 172.21.15.32:0/1235680068 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "osd deep-scrub", "who": "7", "target": ["mon-mgr", ""]}) v1 -- 0x7f9cd4159dc0 con 0x7f9cc0065b80 2023-12-20T06:04:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cbb7fe640 1 -- 172.21.15.32:0/1235680068 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 instructed osd(s) 7 to deep-scrub) v1 ==== 41+0+0 (secure 0 0 0) 0x7f9cc4007620 con 0x7f9cc0065b80 2023-12-20T06:04:49.643 INFO:teuthology.orchestra.run.smithi032.stderr:instructed osd(s) 7 to deep-scrub 2023-12-20T06:04:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 -- 172.21.15.32:0/1235680068 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cc0065b80 msgr2=0x7f9cc0068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cc0065b80 0x7f9cc0068040 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f9cc40010f0 tx=0x7f9cc4007bf0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 -- 172.21.15.32:0/1235680068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40b0d00 msgr2=0x7f9cd414f170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40b0d00 0x7f9cd414f170 secure :-1 s=READY pgs=2645 cs=0 l=1 rev1=1 crypto rx=0x7f9cc800c470 tx=0x7f9cc800c930 comp rx=0 tx=0).stop 2023-12-20T06:04:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 -- 172.21.15.32:0/1235680068 shutdown_connections 2023-12-20T06:04:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f9cc0065b80 0x7f9cc0068040 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f9cd41599e0 0x7f9cd414f6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f9cd40b0d00 0x7f9cd414f170 unknown :-1 s=CLOSED pgs=2645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.645 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 --2- 172.21.15.32:0/1235680068 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f9cd40aece0 0x7f9cd415bbd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 -- 172.21.15.32:0/1235680068 >> 172.21.15.32:0/1235680068 conn(0x7f9cd406a320 msgr2=0x7f9cd40cd510 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 -- 172.21.15.32:0/1235680068 shutdown_connections 2023-12-20T06:04:49.646 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.641+0000 7f9cd9066640 1 -- 172.21.15.32:0/1235680068 wait complete. 2023-12-20T06:04:49.664 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:04:49.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.805+0000 7f6de702d640 1 Processor -- start 2023-12-20T06:04:49.808 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.805+0000 7f6de702d640 1 -- start start 2023-12-20T06:04:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de702d640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f6de015d080 0x7f6de00cbbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de702d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6de015de00 0x7f6de006a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de702d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f6de015d970 con 0x7f6de0069d20 2023-12-20T06:04:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de702d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de00ce740 con 0x7f6de015de00 2023-12-20T06:04:49.810 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de702d640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f6de006a640 con 0x7f6de015d080 2023-12-20T06:04:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de55ca640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6de015de00 0x7f6de006a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de55ca640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f6de015de00 0x7f6de006a100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34118/0 (socket says 172.21.15.32:34118) 2023-12-20T06:04:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de55ca640 1 -- 172.21.15.32:0/1776780760 learned_addr learned my addr 172.21.15.32:0/1776780760 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6ddffff640 1 --2- 172.21.15.32:0/1776780760 >> v2:172.21.15.39:3300/0 conn(0x7f6de015d080 0x7f6de00cbbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de55ca640 1 -- 172.21.15.32:0/1776780760 >> v2:172.21.15.39:3300/0 conn(0x7f6de015d080 msgr2=0x7f6de00cbbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:04:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de55ca640 1 --2- 172.21.15.32:0/1776780760 >> v2:172.21.15.39:3300/0 conn(0x7f6de015d080 0x7f6de00cbbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de55ca640 1 -- 172.21.15.32:0/1776780760 >> v1:172.21.15.39:6789/0 conn(0x7f6de0069d20 legacy=0x7f6de00cd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.811 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.809+0000 7f6de55ca640 1 -- 172.21.15.32:0/1776780760 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6de00ce180 con 0x7f6de015de00 2023-12-20T06:04:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de55ca640 1 --2- 172.21.15.32:0/1776780760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6de015de00 0x7f6de006a100 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f6de00c9e10 tx=0x7f6dd801c9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12e9d2151e17ce1f server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6ddf7fe640 1 -- 172.21.15.32:0/1776780760 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80a5860 con 0x7f6de015de00 2023-12-20T06:04:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6ddf7fe640 1 -- 172.21.15.32:0/1776780760 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dd80a59c0 con 0x7f6de015de00 2023-12-20T06:04:49.813 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6ddf7fe640 1 -- 172.21.15.32:0/1776780760 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dd80a5d50 con 0x7f6de015de00 2023-12-20T06:04:49.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 -- 172.21.15.32:0/1776780760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6de015de00 msgr2=0x7f6de006a100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.814 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 --2- 172.21.15.32:0/1776780760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6de015de00 0x7f6de006a100 secure :-1 s=READY pgs=2646 cs=0 l=1 rev1=1 crypto rx=0x7f6de00c9e10 tx=0x7f6dd801c9a0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 -- 172.21.15.32:0/1776780760 shutdown_connections 2023-12-20T06:04:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 --2- 172.21.15.32:0/1776780760 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6de015de00 0x7f6de006a100 unknown :-1 s=CLOSED pgs=2646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 --2- 172.21.15.32:0/1776780760 >> v2:172.21.15.39:3300/0 conn(0x7f6de015d080 0x7f6de00cbbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 -- 172.21.15.32:0/1776780760 >> 172.21.15.32:0/1776780760 conn(0x7f6de00cc310 msgr2=0x7f6de00cc720 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 -- 172.21.15.32:0/1776780760 shutdown_connections 2023-12-20T06:04:49.815 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 -- 172.21.15.32:0/1776780760 wait complete. 2023-12-20T06:04:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 Processor -- start 2023-12-20T06:04:49.816 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 -- start start 2023-12-20T06:04:49.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6de0069d20 0x7f6de00ae360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6de015d080 0x7f6de0149d30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.818 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de015de00 0x7f6de015ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00b2150 con 0x7f6de015d080 2023-12-20T06:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00b15d0 con 0x7f6de015de00 2023-12-20T06:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.813+0000 7f6de702d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f6de00b18b0 con 0x7f6de0069d20 2023-12-20T06:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6de015d080 0x7f6de0149d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddffff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6de015d080 0x7f6de0149d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:34120/0 (socket says 172.21.15.32:34120) 2023-12-20T06:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddffff640 1 -- 172.21.15.32:0/158232583 learned_addr learned my addr 172.21.15.32:0/158232583 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6de4dc9640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6de0069d20 0x7f6de00ae360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6de55ca640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de015de00 0x7f6de015ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddffff640 1 -- 172.21.15.32:0/158232583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6de0069d20 msgr2=0x7f6de00ae360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddffff640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6de0069d20 0x7f6de00ae360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.819 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddffff640 1 -- 172.21.15.32:0/158232583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de015de00 msgr2=0x7f6de015ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddffff640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de015de00 0x7f6de015ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddffff640 1 -- 172.21.15.32:0/158232583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f6dd8009b30 con 0x7f6de015d080 2023-12-20T06:04:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6de55ca640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de015de00 0x7f6de015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:04:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddffff640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6de015d080 0x7f6de0149d30 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00e930 tx=0x7f6dcc00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddd7fa640 1 -- 172.21.15.32:0/158232583 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc009690 con 0x7f6de015d080 2023-12-20T06:04:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddd7fa640 1 -- 172.21.15.32:0/158232583 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f6dcc00ca00 con 0x7f6de015d080 2023-12-20T06:04:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddd7fa640 1 -- 172.21.15.32:0/158232583 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f6dcc00cd70 con 0x7f6de015d080 2023-12-20T06:04:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6de702d640 1 -- 172.21.15.32:0/158232583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f6de00b1b90 con 0x7f6de015d080 2023-12-20T06:04:49.820 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6de702d640 1 -- 172.21.15.32:0/158232583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f6de00aff40 con 0x7f6de015d080 2023-12-20T06:04:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6de702d640 1 -- 172.21.15.32:0/158232583 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f6de015bc10 con 0x7f6de015d080 2023-12-20T06:04:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddd7fa640 1 -- 172.21.15.32:0/158232583 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f6dcc013470 con 0x7f6de015d080 2023-12-20T06:04:49.821 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.817+0000 7f6ddd7fa640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6dc4065bb0 0x7f6dc4068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:04:49.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.821+0000 7f6ddd7fa640 1 -- 172.21.15.32:0/158232583 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f6dcc083450 con 0x7f6de015d080 2023-12-20T06:04:49.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.821+0000 7f6de4dc9640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6dc4065bb0 0x7f6dc4068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:04:49.822 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.821+0000 7f6de4dc9640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6dc4065bb0 0x7f6dc4068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4007fd0 tx=0x7f6dd4009040 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:04:49.825 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.825+0000 7f6ddd7fa640 1 -- 172.21.15.32:0/158232583 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f6dcc050e70 con 0x7f6de015d080 2023-12-20T06:04:49.980 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.977+0000 7f6de702d640 1 -- 172.21.15.32:0/158232583 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f6de015e2b0 con 0x7f6dc4065bb0 2023-12-20T06:04:49.982 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6ddd7fa640 1 -- 172.21.15.32:0/158232583 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50410 (secure 0 0 0) 0x7f6dd40014f0 con 0x7f6dc4065bb0 2023-12-20T06:04:49.983 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:04:49.983 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:04:49.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 -- 172.21.15.32:0/158232583 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6dc4065bb0 msgr2=0x7f6dc4068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.984 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6dc4065bb0 0x7f6dc4068070 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f6dd4007fd0 tx=0x7f6dd4009040 comp rx=0 tx=0).stop 2023-12-20T06:04:49.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 -- 172.21.15.32:0/158232583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6de015d080 msgr2=0x7f6de0149d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:04:49.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6de015d080 0x7f6de0149d30 secure :-1 s=READY pgs=2647 cs=0 l=1 rev1=1 crypto rx=0x7f6dcc00e930 tx=0x7f6dcc00edf0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 -- 172.21.15.32:0/158232583 shutdown_connections 2023-12-20T06:04:49.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f6dc4065bb0 0x7f6dc4068070 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f6de015de00 0x7f6de015ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f6de015d080 0x7f6de0149d30 unknown :-1 s=CLOSED pgs=2647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 --2- 172.21.15.32:0/158232583 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f6de0069d20 0x7f6de00ae360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:04:49.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 -- 172.21.15.32:0/158232583 >> 172.21.15.32:0/158232583 conn(0x7f6de00cc310 msgr2=0x7f6de01529f0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:04:49.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 -- 172.21.15.32:0/158232583 shutdown_connections 2023-12-20T06:04:49.985 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:04:49.981+0000 7f6de702d640 1 -- 172.21.15.32:0/158232583 wait complete. 2023-12-20T06:04:50.004 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":13,"stamp":"2023-12-20T06:04:48.674746+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3256,"ondisk_log_size":3256,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":439688,"kb_used_data":26440,"kb_used_omap":0,"kb_used_meta":413184,"kb_avail":754535032,"statfs":{"total":773094113280,"available":772643872768,"internally_reserved":0,"allocated":27074560,"data_stored":13892520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":423100416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":5,"apply_latency_ms":5,"commit_latency_ns":5000000,"apply_latency_ns":5000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":96,"num_read_kb":206,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"9.869072"},"pg_stats":[{"pgid":"1.0","version":"253'3256","reported_seq":4230,"reported_epoch":255,"state":"active+clean","last_fresh":"2023-12-20T06:04:38.984634+0000","last_change":"2023-12-20T04:28:28.107310+0000","last_active":"2023-12-20T06:04:38.984634+0000","last_peered":"2023-12-20T06:04:38.984634+0000","last_clean":"2023-12-20T06:04:38.984634+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:04:38.984634+0000","last_undegraded":"2023-12-20T06:04:38.984634+0000","last_fullsized":"2023-12-20T06:04:38.984634+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2023-12-20T04:28:26.830051+0000","last_clean_scrub_stamp":"2023-12-20T04:28:26.830051+0000","objects_scrubbed":0,"log_size":3256,"log_dups_size":0,"ondisk_log_size":3256,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T09:21:14.062253+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3256,"ondisk_log_size":3256,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575995,"num_pgs":15,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61028,"kb_used_data":3740,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94310812,"statfs":{"total":96636764160,"available":96574271488,"internally_reserved":0,"allocated":3829760,"data_stored":2179341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95499999999999996,"5min":1.028,"15min":1.042},"last":0.81699999999999995},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.67300000000000004,"15min":0.72199999999999998},"min":{"1min":0.42899999999999999,"5min":0.39700000000000002,"15min":0.34499999999999997},"max":{"1min":0.90100000000000002,"5min":0.94799999999999995,"15min":0.99099999999999999},"last":0.56100000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69399999999999995,"15min":0.73199999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91800000000000004,"5min":0.94799999999999995,"15min":1.036},"last":0.626},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70499999999999996,"15min":0.73499999999999999},"min":{"1min":0.47599999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.71399999999999997}]},{"osd":2,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.69199999999999995,"15min":0.73099999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91000000000000003,"5min":0.95399999999999996,"15min":1.014},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49099999999999999},"max":{"1min":0.93300000000000005,"5min":1.0089999999999999,"15min":1.0189999999999999},"last":0.77200000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.73499999999999999},"min":{"1min":0.47599999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.92300000000000004},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.77800000000000002},"min":{"1min":0.58299999999999996,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.98899999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.88200000000000001}]},{"osd":4,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.628,"15min":0.59699999999999998},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.04},"last":0.60999999999999999},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.997,"5min":1.02,"15min":1.02},"last":0.66900000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.54300000000000004,"5min":0.37,"15min":0.37},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0449999999999999},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.84899999999999998}]},{"osd":6,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.57499999999999996,"5min":0.442,"15min":0.442},"max":{"1min":0.97299999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.75900000000000001},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.79700000000000004}]}]},{"osd":6,"up_from":13,"seq":55834575995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.72699999999999998},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.85099999999999998,"5min":1.016,"15min":1.0980000000000001},"last":0.61299999999999999},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68899999999999995,"15min":0.73499999999999999},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90600000000000003,"5min":1.006,"15min":1.05},"last":0.71399999999999997}]},{"osd":1,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67800000000000005,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.67300000000000004},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.47999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.97299999999999998,"15min":1.0960000000000001},"last":0.68999999999999995}]},{"osd":2,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72999999999999998,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90100000000000002,"5min":0.97699999999999998,"15min":1.069},"last":0.755},{"interface":"front","average":{"1min":0.623,"5min":0.69199999999999995,"15min":0.746},"min":{"1min":0.47399999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.875,"5min":0.96799999999999997,"15min":1.0760000000000001},"last":0.64800000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93400000000000005,"5min":1.042,"15min":1.077},"last":0.58199999999999996},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71299999999999997,"15min":0.748},"min":{"1min":0.504,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.879,"5min":1.0309999999999999,"15min":1.052},"last":0.53800000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75800000000000001,"5min":1.05,"15min":1.05},"last":0.65900000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.86699999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.77300000000000002}]},{"osd":5,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.377},"max":{"1min":0.79300000000000004,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71199999999999997,"15min":0.70499999999999996},"min":{"1min":0.502,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.83999999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82899999999999996,"5min":1.103,"15min":1.1240000000000001},"last":0.86299999999999999},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97999999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.73699999999999999}]}]},{"osd":5,"up_from":13,"seq":55834575996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.077},"last":0.83199999999999996},{"interface":"front","average":{"1min":0.62,"5min":0.69099999999999995,"15min":0.73499999999999999},"min":{"1min":0.45600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.84199999999999997,"5min":0.96599999999999997,"15min":1.028},"last":0.81599999999999995}]},{"osd":1,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.74399999999999999},"min":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82099999999999995,"5min":0.98799999999999999,"15min":1.0209999999999999},"last":0.67400000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68999999999999995,"15min":0.73899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.628,"5min":0.72599999999999998,"15min":0.76100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48299999999999998},"max":{"1min":0.79000000000000004,"5min":1.0669999999999999,"15min":1.0680000000000001},"last":0.86799999999999999}]},{"osd":3,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75600000000000001,"15min":0.77800000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99199999999999999,"5min":1.05,"15min":1.0669999999999999},"last":0.74399999999999999},{"interface":"front","average":{"1min":0.624,"5min":0.73399999999999999,"15min":0.76600000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89800000000000002,"5min":0.98099999999999998,"15min":1.0629999999999999},"last":0.72299999999999998}]},{"osd":4,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65800000000000003,"15min":0.64900000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.077},"last":0.90500000000000003},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.437,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94599999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.57399999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.51400000000000001,"5min":0.49299999999999999,"15min":0.46899999999999997},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.78900000000000003},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.86899999999999999,"5min":1.097,"15min":1.097},"last":0.69799999999999995}]},{"osd":7,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73299999999999998,"15min":0.70999999999999996},"min":{"1min":0.51200000000000001,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.0409999999999999,"5min":1.153,"15min":1.153},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77300000000000002,"15min":0.76600000000000001},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.55500000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.85199999999999998}]}]},{"osd":4,"up_from":13,"seq":55834575997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67500000000000004,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.98999999999999999,"15min":1.0209999999999999},"last":0.69599999999999995},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.71799999999999997},"min":{"1min":0.46200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.92600000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.61399999999999999}]},{"osd":1,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83999999999999997,"5min":0.95699999999999996,"15min":1.0900000000000001},"last":0.78500000000000003},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47399999999999998,"15min":0.44500000000000001},"max":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72299999999999998,"15min":0.76200000000000001},"min":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.101},"last":0.57199999999999995}]},{"osd":3,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70299999999999996,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":1.014,"15min":1.014},"last":0.67600000000000005},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97799999999999998,"5min":1.04,"15min":1.081},"last":0.84899999999999998}]},{"osd":5,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.63800000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.71499999999999997}]},{"osd":6,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.1379999999999999},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.749,"15min":0.745},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.41199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.52100000000000002}]},{"osd":7,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.75600000000000001},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.0740000000000001,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.81599999999999995}]}]},{"osd":3,"up_from":13,"seq":55834575996,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45100000000000001,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.74399999999999999,"5min":0.748,"15min":2.8130000000000002},"last":0.36599999999999999},{"interface":"front","average":{"1min":0.441,"5min":0.47899999999999998,"15min":0.52000000000000002},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56999999999999995,"5min":0.70599999999999996,"15min":1.357},"last":0.46700000000000003}]},{"osd":1,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.44500000000000001,"15min":0.47299999999999998},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.628,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.41099999999999998},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50900000000000001,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61199999999999999,"5min":0.747,"15min":1.4219999999999999},"last":0.53100000000000003}]},{"osd":2,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54100000000000004,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.56699999999999995,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.55200000000000005},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52800000000000002,"15min":0.56399999999999995},"min":{"1min":0.39100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.86099999999999999,"15min":2.944},"last":0.77200000000000002}]},{"osd":4,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.80300000000000005},"min":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.92900000000000005,"5min":1.159,"15min":1.669},"last":0.60499999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.74099999999999999,"15min":0.79800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":1.1299999999999999,"15min":1.99},"last":0.80400000000000005}]},{"osd":5,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.77100000000000002,"15min":0.81899999999999995},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.89300000000000002,"5min":1.1100000000000001,"15min":1.901},"last":0.73699999999999999},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.749,"15min":0.80800000000000005},"min":{"1min":0.55900000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85499999999999998,"5min":1.196,"15min":2.8639999999999999},"last":0.70999999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.752,"15min":0.80400000000000005},"min":{"1min":0.57299999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89200000000000002,"5min":1.119,"15min":1.575},"last":0.65200000000000002},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77700000000000002,"15min":0.83399999999999996},"min":{"1min":0.57899999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.80100000000000005,"5min":1.1479999999999999,"15min":1.948},"last":0.67100000000000004}]},{"osd":7,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.77300000000000002,"15min":0.82899999999999996},"min":{"1min":0.59499999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1579999999999999,"15min":2.9049999999999998},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.84499999999999997},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":0.91000000000000003,"5min":1.1850000000000001,"15min":1.754},"last":0.69799999999999995}]}]},{"osd":2,"up_from":13,"seq":55834575995,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42899999999999999,"15min":0.45100000000000001},"min":{"1min":0.29499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54000000000000004,"5min":0.75800000000000001,"15min":1.0049999999999999},"last":0.34000000000000002},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.497,"15min":0.51300000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.59899999999999998,"5min":0.93799999999999994,"15min":1.4319999999999999},"last":0.55900000000000005}]},{"osd":1,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46999999999999997,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65600000000000003,"5min":0.79800000000000004,"15min":0.82199999999999995},"last":0.35599999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.33300000000000002,"15min":0.30499999999999999},"max":{"1min":0.59999999999999998,"5min":0.92600000000000005,"15min":1.004},"last":0.64100000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60299999999999998,"15min":0.63400000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":1.089,"15min":1.3660000000000001},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.622,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89000000000000001,"5min":1.069,"15min":1.6919999999999999},"last":0.69799999999999995}]},{"osd":4,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.75600000000000001,"15min":0.78900000000000003},"min":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93799999999999994,"5min":1.119,"15min":1.254},"last":0.59499999999999997},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.74399999999999999,"15min":0.78700000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.085,"15min":1.605},"last":0.61799999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.5,"5min":0.498,"15min":0.498},"max":{"1min":0.85799999999999998,"5min":1.1639999999999999,"15min":1.329},"last":0.58399999999999996},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78400000000000003,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88,"5min":1.0609999999999999,"15min":1.095},"last":0.73899999999999999}]},{"osd":6,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.76800000000000002,"15min":0.80500000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83199999999999996,"5min":1.157,"15min":1.919},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":0.81999999999999995},"min":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96399999999999997,"5min":1.117,"15min":1.2450000000000001},"last":0.67600000000000005}]},{"osd":7,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.02,"5min":1.3360000000000001,"15min":1.7869999999999999},"last":0.76800000000000002},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.84599999999999997,"15min":0.876},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0009999999999999,"5min":1.262,"15min":1.292},"last":0.73199999999999998}]}]},{"osd":1,"up_from":13,"seq":55834575997,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60448,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311392,"statfs":{"total":96636764160,"available":96574865408,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.44500000000000001,"15min":0.45800000000000002},"min":{"1min":0.23899999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.72599999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.498},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.51200000000000001},"min":{"1min":0.32100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70899999999999996,"5min":1.04,"15min":1.04},"last":0.33600000000000002}]},{"osd":2,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.32800000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.45000000000000001},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.57199999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76000000000000001,"5min":1.081,"15min":1.081},"last":0.41699999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.499,"5min":0.51800000000000002,"15min":0.54500000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67700000000000005,"5min":0.78300000000000003,"15min":0.93500000000000005},"last":0.53700000000000003}]},{"osd":4,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.749,"15min":0.77600000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95099999999999996,"5min":1.107,"15min":1.321},"last":0.70699999999999996},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77100000000000002,"15min":0.80200000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91600000000000004,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.64000000000000001}]},{"osd":5,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.748,"15min":0.78700000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93200000000000005,"5min":1.157,"15min":1.2849999999999999},"last":0.65600000000000003},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76100000000000001,"15min":0.79500000000000004},"min":{"1min":0.46999999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98499999999999999,"5min":1.448,"15min":1.448},"last":0.79000000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78800000000000003,"15min":0.81899999999999995},"min":{"1min":0.59399999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97199999999999998,"5min":2.488,"15min":2.488},"last":0.72799999999999998},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.77300000000000002,"15min":0.80100000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90000000000000002,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.80500000000000005}]},{"osd":7,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.80100000000000005,"15min":0.84599999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.745},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.54100000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.101,"5min":1.208,"15min":1.331},"last":0.755}]}]},{"osd":0,"up_from":13,"seq":55834575996,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56932,"kb_used_data":3740,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94314908,"statfs":{"total":96636764160,"available":96578465792,"internally_reserved":0,"allocated":3829760,"data_stored":2179341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.29299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":0.998,"15min":0.998},"last":0.40300000000000002},{"interface":"front","average":{"1min":0.434,"5min":0.54300000000000004,"15min":0.55200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.63900000000000001,"5min":3.681,"15min":3.681},"last":0.29299999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60999999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.45100000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.46500000000000002}]},{"osd":3,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.67300000000000004,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.374},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.42699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66000000000000003,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.47899999999999998}]},{"osd":4,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.53800000000000003},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80500000000000005,"15min":0.82799999999999996},"min":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"max":{"1min":1.081,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.65200000000000002}]},{"osd":5,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.84499999999999997,"15min":0.84399999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.042,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.67100000000000004},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81599999999999995,"15min":0.84299999999999997},"min":{"1min":0.53700000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.1220000000000001,"5min":2.23,"15min":2.23},"last":0.56100000000000005}]},{"osd":6,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.83599999999999997},"min":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0529999999999999,"5min":1.482,"15min":1.482},"last":0.68100000000000005},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.54700000000000004}]},{"osd":7,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96099999999999997,"5min":5.7160000000000002,"15min":5.7160000000000002},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.86399999999999999,"15min":0.86599999999999999},"min":{"1min":0.56899999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94399999999999995,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.58699999999999997}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:04:50.008 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2023-12-20T04:28:26.830051+0000 time.struct_time(tm_year=2023, tm_mon=12, tm_mday=20, tm_hour=4, tm_min=28, tm_sec=26, tm_wday=2, tm_yday=354, tm_isdst=-1) <= time.struct_time(tm_year=2023, tm_mon=12, tm_mday=20, tm_hour=6, tm_min=4, tm_sec=44, tm_wday=2, tm_yday=354, tm_isdst=0) 2023-12-20T06:04:50.008 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2023-12-20T06:05:10.009 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2023-12-20T06:05:10.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.160+0000 7f3795e2d640 1 Processor -- start 2023-12-20T06:05:10.164 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.160+0000 7f3795e2d640 1 -- start start 2023-12-20T06:05:10.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f3795e2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37901599e0 0x7f37900aece0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f3795e2d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37900b0d00 0x7f37900701c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.166 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f3795e2d640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f37900c98c0 con 0x7f37900c90e0 2023-12-20T06:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f3795e2d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f37900c9ba0 con 0x7f37900b0d00 2023-12-20T06:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f3795e2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37900c9e80 con 0x7f37901599e0 2023-12-20T06:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f378ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37900b0d00 0x7f37900701c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f378ffff640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f37900b0d00 0x7f37900701c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41826/0 (socket says 172.21.15.32:41826) 2023-12-20T06:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f378ffff640 1 -- 172.21.15.32:0/2512521703 learned_addr learned my addr 172.21.15.32:0/2512521703 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f378f7fe640 1 --2- 172.21.15.32:0/2512521703 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37901599e0 0x7f37900aece0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f378ffff640 1 -- 172.21.15.32:0/2512521703 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37901599e0 msgr2=0x7f37900aece0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f378ffff640 1 --2- 172.21.15.32:0/2512521703 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37901599e0 0x7f37900aece0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f378ffff640 1 -- 172.21.15.32:0/2512521703 >> v1:172.21.15.32:6789/0 conn(0x7f37900c90e0 legacy=0x7f37900b05f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f378ffff640 1 -- 172.21.15.32:0/2512521703 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f37900ca160 con 0x7f37900b0d00 2023-12-20T06:05:10.167 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.164+0000 7f378f7fe640 1 --2- 172.21.15.32:0/2512521703 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37901599e0 0x7f37900aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:05:10.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f378ffff640 1 --2- 172.21.15.32:0/2512521703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37900b0d00 0x7f37900701c0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f3788009e50 tx=0x7f378801c950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=256b76b2cece50c7 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:05:10.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f378e7fc640 1 -- 172.21.15.32:0/2512521703 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37880a5830 con 0x7f37900b0d00 2023-12-20T06:05:10.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f378e7fc640 1 -- 172.21.15.32:0/2512521703 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f37880a5990 con 0x7f37900b0d00 2023-12-20T06:05:10.169 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f378e7fc640 1 -- 172.21.15.32:0/2512521703 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37880a5d20 con 0x7f37900b0d00 2023-12-20T06:05:10.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f3795e2d640 1 -- 172.21.15.32:0/2512521703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37900b0d00 msgr2=0x7f37900701c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f3795e2d640 1 --2- 172.21.15.32:0/2512521703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37900b0d00 0x7f37900701c0 secure :-1 s=READY pgs=2649 cs=0 l=1 rev1=1 crypto rx=0x7f3788009e50 tx=0x7f378801c950 comp rx=0 tx=0).stop 2023-12-20T06:05:10.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f3795e2d640 1 -- 172.21.15.32:0/2512521703 shutdown_connections 2023-12-20T06:05:10.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f3795e2d640 1 --2- 172.21.15.32:0/2512521703 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37900b0d00 0x7f37900701c0 unknown :-1 s=CLOSED pgs=2649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f3795e2d640 1 --2- 172.21.15.32:0/2512521703 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37901599e0 0x7f37900aece0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.170 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f3795e2d640 1 -- 172.21.15.32:0/2512521703 >> 172.21.15.32:0/2512521703 conn(0x7f379006a320 msgr2=0x7f37900ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:05:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f3795e2d640 1 -- 172.21.15.32:0/2512521703 shutdown_connections 2023-12-20T06:05:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f3795e2d640 1 -- 172.21.15.32:0/2512521703 wait complete. 2023-12-20T06:05:10.171 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f3795e2d640 1 Processor -- start 2023-12-20T06:05:10.172 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.168+0000 7f3795e2d640 1 -- start start 2023-12-20T06:05:10.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3795e2d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37900b0d00 0x7f379014c8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.174 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3795e2d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37900c90e0 0x7f3790157e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3795e2d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37901599e0 0x7f37900cc8d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3795e2d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37901f60b0 con 0x7f37900b0d00 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3795e2d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f37901f6390 con 0x7f37900c90e0 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3795e2d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f37901f6670 con 0x7f37901599e0 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f378f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37900b0d00 0x7f379014c8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f378f7fe640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37900b0d00 0x7f379014c8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41838/0 (socket says 172.21.15.32:41838) 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f378f7fe640 1 -- 172.21.15.32:0/3448150151 learned_addr learned my addr 172.21.15.32:0/3448150151 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f378f7fe640 1 -- 172.21.15.32:0/3448150151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37901599e0 msgr2=0x7f37900cc8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f378f7fe640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37901599e0 0x7f37900cc8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f378f7fe640 1 -- 172.21.15.32:0/3448150151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37900c90e0 msgr2=0x7f3790157e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f378f7fe640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37900c90e0 0x7f3790157e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f378f7fe640 1 -- 172.21.15.32:0/3448150151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f3788009b30 con 0x7f37900b0d00 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f378f7fe640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37900b0d00 0x7f379014c8a0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f378000d7d0 tx=0x7f378000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3777fff640 1 -- 172.21.15.32:0/3448150151 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f37800059a0 con 0x7f37900b0d00 2023-12-20T06:05:10.175 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3777fff640 1 -- 172.21.15.32:0/3448150151 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f3780005b00 con 0x7f37900b0d00 2023-12-20T06:05:10.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3795e2d640 1 -- 172.21.15.32:0/3448150151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f37901f6950 con 0x7f37900b0d00 2023-12-20T06:05:10.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3777fff640 1 -- 172.21.15.32:0/3448150151 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f3780005e90 con 0x7f37900b0d00 2023-12-20T06:05:10.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3795e2d640 1 -- 172.21.15.32:0/3448150151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f37901f7020 con 0x7f37900b0d00 2023-12-20T06:05:10.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3795e2d640 1 -- 172.21.15.32:0/3448150151 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f379004e000 con 0x7f37900b0d00 2023-12-20T06:05:10.176 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.172+0000 7f3777fff640 1 -- 172.21.15.32:0/3448150151 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f3780004030 con 0x7f37900b0d00 2023-12-20T06:05:10.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.176+0000 7f3777fff640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f377c065bb0 0x7f377c068070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.176+0000 7f3777fff640 1 -- 172.21.15.32:0/3448150151 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f37800832e0 con 0x7f37900b0d00 2023-12-20T06:05:10.177 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.176+0000 7f378effd640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f377c065bb0 0x7f377c068070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.178 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.176+0000 7f378effd640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f377c065bb0 0x7f377c068070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3784000f70 tx=0x7f37840079c0 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:05:10.181 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.180+0000 7f3777fff640 1 -- 172.21.15.32:0/3448150151 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f3780050b90 con 0x7f37900b0d00 2023-12-20T06:05:10.337 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.336+0000 7f3795e2d640 1 -- 172.21.15.32:0/3448150151 --> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) v1 -- 0x7f3790159dc0 con 0x7f377c065bb0 2023-12-20T06:05:10.340 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.336+0000 7f3777fff640 1 -- 172.21.15.32:0/3448150151 <== mgr.12348 v2:172.21.15.32:6832/64546 1 ==== mgr_command_reply(tid 0: 0 dumped all) v1 ==== 18+0+50328 (secure 0 0 0) 0x7f3784005080 con 0x7f377c065bb0 2023-12-20T06:05:10.341 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:05:10.341 INFO:teuthology.orchestra.run.smithi032.stderr:dumped all 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 -- 172.21.15.32:0/3448150151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f377c065bb0 msgr2=0x7f377c068070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f377c065bb0 0x7f377c068070 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f3784000f70 tx=0x7f37840079c0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 -- 172.21.15.32:0/3448150151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37900b0d00 msgr2=0x7f379014c8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37900b0d00 0x7f379014c8a0 secure :-1 s=READY pgs=2650 cs=0 l=1 rev1=1 crypto rx=0x7f378000d7d0 tx=0x7f378000dc90 comp rx=0 tx=0).stop 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 -- 172.21.15.32:0/3448150151 shutdown_connections 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f377c065bb0 0x7f377c068070 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f37901599e0 0x7f37900cc8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f37900c90e0 0x7f3790157e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 --2- 172.21.15.32:0/3448150151 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f37900b0d00 0x7f379014c8a0 unknown :-1 s=CLOSED pgs=2650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 -- 172.21.15.32:0/3448150151 >> 172.21.15.32:0/3448150151 conn(0x7f379006a320 msgr2=0x7f37900add40 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 -- 172.21.15.32:0/3448150151 shutdown_connections 2023-12-20T06:05:10.342 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.340+0000 7f3795e2d640 1 -- 172.21.15.32:0/3448150151 wait complete. 2023-12-20T06:05:10.361 INFO:teuthology.orchestra.run.smithi032.stdout:{"pg_ready":true,"pg_map":{"version":23,"stamp":"2023-12-20T06:05:08.679370+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3257,"ondisk_log_size":3257,"up":2,"acting":2,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":2,"num_osds":8,"num_per_pool_osds":8,"num_per_pool_omap_osds":2,"kb":754974720,"kb_used":439688,"kb_used_data":26440,"kb_used_omap":0,"kb_used_meta":413184,"kb_avail":754535032,"statfs":{"total":773094113280,"available":772643872768,"internally_reserved":0,"allocated":27074560,"data_stored":13892520,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":423100416},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002775"},"pg_stats":[{"pgid":"1.0","version":"256'3257","reported_seq":4243,"reported_epoch":256,"state":"active+clean","last_fresh":"2023-12-20T06:04:54.491863+0000","last_change":"2023-12-20T06:04:54.491992+0000","last_active":"2023-12-20T06:04:54.491863+0000","last_peered":"2023-12-20T06:04:54.491863+0000","last_clean":"2023-12-20T06:04:54.491863+0000","last_became_active":"2023-12-20T04:28:28.106009+0000","last_became_peered":"2023-12-20T04:28:28.106009+0000","last_unstale":"2023-12-20T06:04:54.491863+0000","last_undegraded":"2023-12-20T06:04:54.491863+0000","last_fullsized":"2023-12-20T06:04:54.491863+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"256'3257","last_scrub_stamp":"2023-12-20T06:04:54.491839+0000","last_deep_scrub":"256'3257","last_deep_scrub_stamp":"2023-12-20T06:04:54.491839+0000","last_clean_scrub_stamp":"2023-12-20T06:04:54.491839+0000","objects_scrubbed":2,"log_size":3257,"log_dups_size":0,"ondisk_log_size":3257,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":1,"scrub_schedule":"periodic scrub scheduled @ 2023-12-21T12:42:09.241165+0000","scrub_duration":0.0096257619999999995,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2365,"num_read_kb":4879,"num_write":621,"num_write_kb":2416,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1187840,"data_stored":1180736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3257,"ondisk_log_size":3257,"up":2,"acting":2,"num_store_stats":2}],"osd_stats":[{"osd":7,"up_from":13,"seq":55834575999,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":61028,"kb_used_data":3740,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94310812,"statfs":{"total":96636764160,"available":96574271488,"internally_reserved":0,"allocated":3829760,"data_stored":2179341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61299999999999999,"5min":0.67600000000000005,"15min":0.71599999999999997},"min":{"1min":0.43099999999999999,"5min":0.40200000000000002,"15min":0.40200000000000002},"max":{"1min":0.95499999999999996,"5min":1.028,"15min":1.042},"last":0.85999999999999999},{"interface":"front","average":{"1min":0.59999999999999998,"5min":0.67300000000000004,"15min":0.72199999999999998},"min":{"1min":0.42899999999999999,"5min":0.39700000000000002,"15min":0.34499999999999997},"max":{"1min":0.90100000000000002,"5min":0.94799999999999995,"15min":0.99099999999999999},"last":0.69499999999999995}]},{"osd":1,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69399999999999995,"15min":0.73199999999999998},"min":{"1min":0.33400000000000002,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.91800000000000004,"5min":0.94799999999999995,"15min":1.036},"last":0.89200000000000002},{"interface":"front","average":{"1min":0.65100000000000002,"5min":0.70499999999999996,"15min":0.73499999999999999},"min":{"1min":0.47599999999999998,"5min":0.441,"15min":0.441},"max":{"1min":1.0389999999999999,"5min":1.0389999999999999,"15min":1.0389999999999999},"last":0.84699999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.69199999999999995,"15min":0.73099999999999998},"min":{"1min":0.42999999999999999,"5min":0.42999999999999999,"15min":0.42999999999999999},"max":{"1min":0.91000000000000003,"5min":0.95399999999999996,"15min":1.014},"last":0.76200000000000001},{"interface":"front","average":{"1min":0.69199999999999995,"5min":0.73499999999999999,"15min":0.75},"min":{"1min":0.49299999999999999,"5min":0.49299999999999999,"15min":0.49099999999999999},"max":{"1min":0.93300000000000005,"5min":1.0089999999999999,"15min":1.0189999999999999},"last":0.80100000000000005}]},{"osd":3,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68000000000000005,"5min":0.70899999999999996,"15min":0.73499999999999999},"min":{"1min":0.47599999999999998,"5min":0.442,"15min":0.442},"max":{"1min":1.0660000000000001,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.82899999999999996},{"interface":"front","average":{"1min":0.74099999999999999,"5min":0.76700000000000002,"15min":0.77800000000000002},"min":{"1min":0.58299999999999996,"5min":0.41199999999999998,"15min":0.40699999999999997},"max":{"1min":0.98899999999999999,"5min":1.0660000000000001,"15min":1.0660000000000001},"last":0.93999999999999995}]},{"osd":4,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.628,"15min":0.59699999999999998},"min":{"1min":0.40000000000000002,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.99099999999999999,"5min":0.99099999999999999,"15min":1.04},"last":0.51500000000000001},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.66900000000000004,"15min":0.65100000000000002},"min":{"1min":0.41999999999999998,"5min":0.41399999999999998,"15min":0.39600000000000002},"max":{"1min":0.997,"5min":1.02,"15min":1.02},"last":0.53900000000000003}]},{"osd":5,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.69799999999999995,"15min":0.67000000000000004},"min":{"1min":0.54300000000000004,"5min":0.37,"15min":0.37},"max":{"1min":1.0329999999999999,"5min":1.0329999999999999,"15min":1.0449999999999999},"last":0.73799999999999999},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.70799999999999996,"15min":0.68799999999999994},"min":{"1min":0.55700000000000005,"5min":0.45300000000000001,"15min":0.45300000000000001},"max":{"1min":1.103,"5min":1.103,"15min":1.103},"last":0.60399999999999998}]},{"osd":6,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.78100000000000003,"5min":0.76600000000000001,"15min":0.746},"min":{"1min":0.57499999999999996,"5min":0.442,"15min":0.442},"max":{"1min":0.97299999999999998,"5min":1.0449999999999999,"15min":1.0449999999999999},"last":0.72599999999999998},{"interface":"front","average":{"1min":0.77000000000000002,"5min":0.73999999999999999,"15min":0.72199999999999998},"min":{"1min":0.52600000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":1.1910000000000001,"5min":1.1910000000000001,"15min":1.1910000000000001},"last":0.70999999999999996}]}]},{"osd":6,"up_from":13,"seq":55834575999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65800000000000003,"5min":0.67100000000000004,"15min":0.72699999999999998},"min":{"1min":0.40799999999999997,"5min":0.312,"15min":0.312},"max":{"1min":0.85099999999999998,"5min":1.016,"15min":1.0980000000000001},"last":0.82699999999999996},{"interface":"front","average":{"1min":0.65800000000000003,"5min":0.68899999999999995,"15min":0.73499999999999999},"min":{"1min":0.42199999999999999,"5min":0.376,"15min":0.376},"max":{"1min":0.90600000000000003,"5min":1.006,"15min":1.05},"last":0.63400000000000001}]},{"osd":1,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.622,"5min":0.67800000000000005,"15min":0.71499999999999997},"min":{"1min":0.45000000000000001,"5min":0.39100000000000001,"15min":0.39100000000000001},"max":{"1min":0.88,"5min":1.0089999999999999,"15min":1.0089999999999999},"last":0.65000000000000002},{"interface":"front","average":{"1min":0.65500000000000003,"5min":0.70099999999999996,"15min":0.75},"min":{"1min":0.47999999999999998,"5min":0.371,"15min":0.371},"max":{"1min":0.81299999999999994,"5min":0.97299999999999998,"15min":1.0960000000000001},"last":0.61299999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.72999999999999998,"15min":0.76500000000000001},"min":{"1min":0.46300000000000002,"5min":0.45200000000000001,"15min":0.45200000000000001},"max":{"1min":0.90100000000000002,"5min":0.97699999999999998,"15min":1.069},"last":0.66900000000000004},{"interface":"front","average":{"1min":0.623,"5min":0.69199999999999995,"15min":0.746},"min":{"1min":0.47399999999999998,"5min":0.441,"15min":0.441},"max":{"1min":0.875,"5min":0.96799999999999997,"15min":1.0760000000000001},"last":0.68799999999999994}]},{"osd":3,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.68500000000000005,"5min":0.746,"15min":0.77300000000000002},"min":{"1min":0.436,"5min":0.436,"15min":0.436},"max":{"1min":0.93400000000000005,"5min":1.042,"15min":1.077},"last":0.58699999999999997},{"interface":"front","average":{"1min":0.67900000000000005,"5min":0.71299999999999997,"15min":0.748},"min":{"1min":0.504,"5min":0.46999999999999997,"15min":0.46999999999999997},"max":{"1min":0.879,"5min":1.0309999999999999,"15min":1.052},"last":0.73099999999999998}]},{"osd":4,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.59999999999999998,"5min":0.63300000000000001,"15min":0.63300000000000001},"min":{"1min":0.439,"5min":0.34899999999999998,"15min":0.34899999999999998},"max":{"1min":0.75800000000000001,"5min":1.05,"15min":1.05},"last":0.77900000000000003},{"interface":"front","average":{"1min":0.63400000000000001,"5min":0.66300000000000003,"15min":0.65100000000000002},"min":{"1min":0.41599999999999998,"5min":0.40600000000000003,"15min":0.40200000000000002},"max":{"1min":0.86699999999999999,"5min":1.1339999999999999,"15min":1.1339999999999999},"last":0.52600000000000002}]},{"osd":5,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.57499999999999996,"5min":0.63500000000000001,"15min":0.63500000000000001},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.377},"max":{"1min":0.79300000000000004,"5min":0.98699999999999999,"15min":1.0009999999999999},"last":0.40100000000000002},{"interface":"front","average":{"1min":0.68100000000000005,"5min":0.71199999999999997,"15min":0.70499999999999996},"min":{"1min":0.502,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.90300000000000002,"5min":1.0209999999999999,"15min":1.0209999999999999},"last":0.71399999999999997}]},{"osd":7,"last update":"Wed Dec 20 06:04:38 2023","interfaces":[{"interface":"back","average":{"1min":0.65300000000000002,"5min":0.72499999999999998,"15min":0.73099999999999998},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.82899999999999996,"5min":1.103,"15min":1.1240000000000001},"last":0.75700000000000001},{"interface":"front","average":{"1min":0.66400000000000003,"5min":0.71999999999999997,"15min":0.72199999999999998},"min":{"1min":0.39000000000000001,"5min":0.39000000000000001,"15min":0.39000000000000001},"max":{"1min":0.97999999999999998,"5min":1.0580000000000001,"15min":1.0700000000000001},"last":0.54400000000000004}]}]},{"osd":5,"up_from":13,"seq":55834576000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.626,"5min":0.69299999999999995,"15min":0.72699999999999998},"min":{"1min":0.42399999999999999,"5min":0.39200000000000002,"15min":0.39200000000000002},"max":{"1min":0.95799999999999996,"5min":0.95799999999999996,"15min":1.077},"last":0.47499999999999998},{"interface":"front","average":{"1min":0.62,"5min":0.69099999999999995,"15min":0.73499999999999999},"min":{"1min":0.45600000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.84199999999999997,"5min":0.96599999999999997,"15min":1.028},"last":0.65100000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.63900000000000001,"5min":0.71099999999999997,"15min":0.74399999999999999},"min":{"1min":0.47799999999999998,"5min":0.47299999999999998,"15min":0.442},"max":{"1min":0.88500000000000001,"5min":1.0409999999999999,"15min":1.0669999999999999},"last":0.61899999999999999},{"interface":"front","average":{"1min":0.65200000000000002,"5min":0.71399999999999997,"15min":0.75},"min":{"1min":0.42699999999999999,"5min":0.42699999999999999,"15min":0.42699999999999999},"max":{"1min":0.82099999999999995,"5min":0.98799999999999999,"15min":1.0209999999999999},"last":0.752}]},{"osd":2,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.61799999999999999,"5min":0.68999999999999995,"15min":0.73899999999999999},"min":{"1min":0.38900000000000001,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.83599999999999997,"5min":0.96299999999999997,"15min":1.0249999999999999},"last":0.69899999999999995},{"interface":"front","average":{"1min":0.628,"5min":0.72599999999999998,"15min":0.76100000000000001},"min":{"1min":0.49399999999999999,"5min":0.49399999999999999,"15min":0.48299999999999998},"max":{"1min":0.79000000000000004,"5min":1.0669999999999999,"15min":1.0680000000000001},"last":0.71899999999999997}]},{"osd":3,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69099999999999995,"5min":0.75600000000000001,"15min":0.77800000000000002},"min":{"1min":0.50900000000000001,"5min":0.50900000000000001,"15min":0.50900000000000001},"max":{"1min":0.99199999999999999,"5min":1.05,"15min":1.0669999999999999},"last":0.58799999999999997},{"interface":"front","average":{"1min":0.624,"5min":0.73399999999999999,"15min":0.76600000000000001},"min":{"1min":0.47699999999999998,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.89800000000000002,"5min":0.98099999999999998,"15min":1.0629999999999999},"last":0.56299999999999994}]},{"osd":4,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.627,"5min":0.65800000000000003,"15min":0.64900000000000002},"min":{"1min":0.36399999999999999,"5min":0.36399999999999999,"15min":0.36399999999999999},"max":{"1min":1.0169999999999999,"5min":1.0169999999999999,"15min":1.077},"last":0.50700000000000001},{"interface":"front","average":{"1min":0.63600000000000001,"5min":0.67100000000000004,"15min":0.65300000000000002},"min":{"1min":0.437,"5min":0.42499999999999999,"15min":0.42499999999999999},"max":{"1min":0.94599999999999995,"5min":1.1100000000000001,"15min":1.1100000000000001},"last":0.79200000000000004}]},{"osd":6,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.66700000000000004,"5min":0.71599999999999997,"15min":0.70499999999999996},"min":{"1min":0.51400000000000001,"5min":0.49299999999999999,"15min":0.46899999999999997},"max":{"1min":0.92800000000000005,"5min":1.0820000000000001,"15min":1.0820000000000001},"last":0.64000000000000001},{"interface":"front","average":{"1min":0.60499999999999998,"5min":0.66300000000000003,"15min":0.65300000000000002},"min":{"1min":0.436,"5min":0.41499999999999998,"15min":0.40699999999999997},"max":{"1min":0.86899999999999999,"5min":1.097,"15min":1.097},"last":0.66900000000000004}]},{"osd":7,"last update":"Wed Dec 20 06:04:25 2023","interfaces":[{"interface":"back","average":{"1min":0.69699999999999995,"5min":0.73299999999999998,"15min":0.70999999999999996},"min":{"1min":0.51200000000000001,"5min":0.496,"15min":0.48499999999999999},"max":{"1min":1.0409999999999999,"5min":1.153,"15min":1.153},"last":0.68500000000000005},{"interface":"front","average":{"1min":0.73399999999999999,"5min":0.77300000000000002,"15min":0.76600000000000001},"min":{"1min":0.58099999999999996,"5min":0.56299999999999994,"15min":0.55500000000000005},"max":{"1min":1.0760000000000001,"5min":1.0760000000000001,"15min":1.0760000000000001},"last":0.73599999999999999}]}]},{"osd":4,"up_from":13,"seq":55834576001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64200000000000002,"5min":0.67500000000000004,"15min":0.72099999999999997},"min":{"1min":0.42099999999999999,"5min":0.36199999999999999,"15min":0.36199999999999999},"max":{"1min":0.94999999999999996,"5min":0.98999999999999999,"15min":1.0209999999999999},"last":0.72899999999999998},{"interface":"front","average":{"1min":0.66600000000000004,"5min":0.68300000000000005,"15min":0.71799999999999997},"min":{"1min":0.46200000000000002,"5min":0.432,"15min":0.432},"max":{"1min":0.92600000000000005,"5min":1.0669999999999999,"15min":1.0669999999999999},"last":0.77200000000000002}]},{"osd":1,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.64800000000000002,"5min":0.70999999999999996,"15min":0.746},"min":{"1min":0.441,"5min":0.441,"15min":0.441},"max":{"1min":0.83999999999999997,"5min":0.95699999999999996,"15min":1.0900000000000001},"last":0.68999999999999995},{"interface":"front","average":{"1min":0.70499999999999996,"5min":0.72899999999999998,"15min":0.76100000000000001},"min":{"1min":0.48099999999999998,"5min":0.47399999999999998,"15min":0.44500000000000001},"max":{"1min":0.91800000000000004,"5min":1.0580000000000001,"15min":1.0580000000000001},"last":0.66500000000000004}]},{"osd":2,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65500000000000003,"5min":0.72299999999999998,"15min":0.76200000000000001},"min":{"1min":0.47099999999999997,"5min":0.46200000000000002,"15min":0.46200000000000002},"max":{"1min":0.90800000000000003,"5min":1.0940000000000001,"15min":1.0980000000000001},"last":0.80400000000000005},{"interface":"front","average":{"1min":0.70999999999999996,"5min":0.747,"15min":0.77200000000000002},"min":{"1min":0.51400000000000001,"5min":0.41099999999999998,"15min":0.41099999999999998},"max":{"1min":1.042,"5min":1.042,"15min":1.101},"last":0.78900000000000003}]},{"osd":3,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.65000000000000002,"5min":0.70299999999999996,"15min":0.74399999999999999},"min":{"1min":0.46300000000000002,"5min":0.46300000000000002,"15min":0.46300000000000002},"max":{"1min":0.89300000000000002,"5min":1.014,"15min":1.014},"last":0.65300000000000002},{"interface":"front","average":{"1min":0.72599999999999998,"5min":0.75800000000000001,"15min":0.79700000000000004},"min":{"1min":0.52500000000000002,"5min":0.44700000000000001,"15min":0.44700000000000001},"max":{"1min":0.97799999999999998,"5min":1.04,"15min":1.081},"last":0.84799999999999998}]},{"osd":5,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.72299999999999998,"5min":0.67300000000000004,"15min":0.66000000000000003},"min":{"1min":0.505,"5min":0.38600000000000001,"15min":0.38600000000000001},"max":{"1min":1.0960000000000001,"5min":1.0960000000000001,"15min":1.0960000000000001},"last":0.54400000000000004},{"interface":"front","average":{"1min":0.68500000000000005,"5min":0.67700000000000005,"15min":0.66800000000000004},"min":{"1min":0.47399999999999998,"5min":0.42499999999999999,"15min":0.35799999999999998},"max":{"1min":0.88,"5min":1.0169999999999999,"15min":1.0800000000000001},"last":0.58899999999999997}]},{"osd":6,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.70499999999999996,"5min":0.68999999999999995,"15min":0.68300000000000005},"min":{"1min":0.52300000000000002,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":1.0389999999999999,"5min":1.0569999999999999,"15min":1.1379999999999999},"last":0.623},{"interface":"front","average":{"1min":0.77200000000000002,"5min":0.749,"15min":0.745},"min":{"1min":0.56100000000000005,"5min":0.46800000000000003,"15min":0.41199999999999998},"max":{"1min":1.163,"5min":1.163,"15min":1.163},"last":0.60599999999999998}]},{"osd":7,"last update":"Wed Dec 20 06:04:03 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.73499999999999999,"15min":0.73599999999999999},"min":{"1min":0.52600000000000002,"5min":0.47899999999999998,"15min":0.38500000000000001},"max":{"1min":0.92700000000000005,"5min":1.0600000000000001,"15min":1.0600000000000001},"last":0.71099999999999997},{"interface":"front","average":{"1min":0.75900000000000001,"5min":0.77100000000000002,"15min":0.79100000000000004},"min":{"1min":0.57799999999999996,"5min":0.52600000000000002,"15min":0.434},"max":{"1min":1.0740000000000001,"5min":1.1319999999999999,"15min":1.1679999999999999},"last":0.752}]}]},{"osd":3,"up_from":13,"seq":55834576000,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.442,"5min":0.45100000000000001,"15min":0.46800000000000003},"min":{"1min":0.33900000000000002,"5min":0.255,"15min":0.255},"max":{"1min":0.74399999999999999,"5min":0.748,"15min":2.8130000000000002},"last":0.33800000000000002},{"interface":"front","average":{"1min":0.441,"5min":0.47899999999999998,"15min":0.52000000000000002},"min":{"1min":0.29899999999999999,"5min":0.28000000000000003,"15min":0.28000000000000003},"max":{"1min":0.56999999999999995,"5min":0.70599999999999996,"15min":1.357},"last":0.41999999999999998}]},{"osd":1,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.433,"5min":0.44500000000000001,"15min":0.47299999999999998},"min":{"1min":0.32100000000000001,"5min":0.29499999999999998,"15min":0.29499999999999998},"max":{"1min":0.628,"5min":0.81299999999999994,"15min":2.0230000000000001},"last":0.53200000000000003},{"interface":"front","average":{"1min":0.47999999999999998,"5min":0.50900000000000001,"15min":0.55900000000000005},"min":{"1min":0.39700000000000002,"5min":0.30599999999999999,"15min":0.30599999999999999},"max":{"1min":0.61199999999999999,"5min":0.747,"15min":1.4219999999999999},"last":0.60999999999999999}]},{"osd":2,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.47599999999999998,"5min":0.54100000000000004,"15min":0.58199999999999996},"min":{"1min":0.36799999999999999,"5min":0.36099999999999999,"15min":0.36099999999999999},"max":{"1min":0.56699999999999995,"5min":0.77500000000000002,"15min":2.0550000000000002},"last":0.59799999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.52800000000000002,"15min":0.56399999999999995},"min":{"1min":0.39100000000000001,"5min":0.29899999999999999,"15min":0.29899999999999999},"max":{"1min":0.70199999999999996,"5min":0.86099999999999999,"15min":2.944},"last":0.442}]},{"osd":4,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.73099999999999998,"5min":0.753,"15min":0.80300000000000005},"min":{"1min":0.55400000000000005,"5min":0.52400000000000002,"15min":0.51700000000000002},"max":{"1min":0.92900000000000005,"5min":1.159,"15min":1.669},"last":0.92400000000000004},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.74099999999999999,"15min":0.79800000000000004},"min":{"1min":0.46800000000000003,"5min":0.46800000000000003,"15min":0.46800000000000003},"max":{"1min":0.877,"5min":1.1299999999999999,"15min":1.99},"last":1.056}]},{"osd":5,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.72399999999999998,"5min":0.77100000000000002,"15min":0.81899999999999995},"min":{"1min":0.52000000000000002,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":0.89300000000000002,"5min":1.1100000000000001,"15min":1.901},"last":0.93799999999999994},{"interface":"front","average":{"1min":0.70299999999999996,"5min":0.749,"15min":0.80800000000000005},"min":{"1min":0.55900000000000005,"5min":0.51300000000000001,"15min":0.51300000000000001},"max":{"1min":0.85499999999999998,"5min":1.196,"15min":2.8639999999999999},"last":0.81799999999999995}]},{"osd":6,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.71699999999999997,"5min":0.752,"15min":0.80400000000000005},"min":{"1min":0.57299999999999995,"5min":0.53300000000000003,"15min":0.53300000000000003},"max":{"1min":0.89200000000000002,"5min":1.119,"15min":1.575},"last":0.97099999999999997},{"interface":"front","average":{"1min":0.71199999999999997,"5min":0.77700000000000002,"15min":0.83399999999999996},"min":{"1min":0.57899999999999996,"5min":0.52400000000000002,"15min":0.52400000000000002},"max":{"1min":0.80100000000000005,"5min":1.1479999999999999,"15min":1.948},"last":0.98399999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:04:29 2023","interfaces":[{"interface":"back","average":{"1min":0.754,"5min":0.77300000000000002,"15min":0.82899999999999996},"min":{"1min":0.59499999999999997,"5min":0.47699999999999998,"15min":0.47699999999999998},"max":{"1min":0.91100000000000003,"5min":1.1579999999999999,"15min":2.9049999999999998},"last":0.95499999999999996},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.78900000000000003,"15min":0.84499999999999997},"min":{"1min":0.60699999999999998,"5min":0.502,"15min":0.502},"max":{"1min":0.91000000000000003,"5min":1.1850000000000001,"15min":1.754},"last":1.0680000000000001}]}]},{"osd":2,"up_from":13,"seq":55834575999,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":52256,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":49088,"kb_avail":94319584,"statfs":{"total":96636764160,"available":96583254016,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":50266112},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.40500000000000003,"5min":0.42899999999999999,"15min":0.45100000000000001},"min":{"1min":0.29499999999999998,"5min":0.26000000000000001,"15min":0.26000000000000001},"max":{"1min":0.54000000000000004,"5min":0.75800000000000001,"15min":1.0049999999999999},"last":0.42299999999999999},{"interface":"front","average":{"1min":0.44500000000000001,"5min":0.497,"15min":0.51300000000000001},"min":{"1min":0.32200000000000001,"5min":0.32200000000000001,"15min":0.32200000000000001},"max":{"1min":0.59899999999999998,"5min":0.93799999999999994,"15min":1.4319999999999999},"last":0.59499999999999997}]},{"osd":1,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.436,"5min":0.46999999999999997,"15min":0.48699999999999999},"min":{"1min":0.33000000000000002,"5min":0.27900000000000003,"15min":0.27900000000000003},"max":{"1min":0.65600000000000003,"5min":0.79800000000000004,"15min":0.82199999999999995},"last":0.47399999999999998},{"interface":"front","average":{"1min":0.49299999999999999,"5min":0.51400000000000001,"15min":0.54200000000000004},"min":{"1min":0.36599999999999999,"5min":0.33300000000000002,"15min":0.30499999999999999},"max":{"1min":0.59999999999999998,"5min":0.92600000000000005,"15min":1.004},"last":0.50600000000000001}]},{"osd":3,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.54200000000000004,"5min":0.60299999999999998,"15min":0.63400000000000001},"min":{"1min":0.375,"5min":0.375,"15min":0.375},"max":{"1min":0.84999999999999998,"5min":1.089,"15min":1.3660000000000001},"last":0.622},{"interface":"front","average":{"1min":0.58099999999999996,"5min":0.622,"15min":0.66700000000000004},"min":{"1min":0.47499999999999998,"5min":0.38900000000000001,"15min":0.38900000000000001},"max":{"1min":0.89000000000000001,"5min":1.069,"15min":1.6919999999999999},"last":0.73499999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.75600000000000001,"15min":0.78900000000000003},"min":{"1min":0.53600000000000003,"5min":0.51700000000000002,"15min":0.51700000000000002},"max":{"1min":0.93799999999999994,"5min":1.119,"15min":1.254},"last":0.69399999999999995},{"interface":"front","average":{"1min":0.67600000000000005,"5min":0.74399999999999999,"15min":0.78700000000000003},"min":{"1min":0.48099999999999998,"5min":0.40000000000000002,"15min":0.40000000000000002},"max":{"1min":0.92100000000000004,"5min":1.085,"15min":1.605},"last":0.77900000000000003}]},{"osd":5,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.70599999999999996,"5min":0.77000000000000002,"15min":0.79200000000000004},"min":{"1min":0.5,"5min":0.498,"15min":0.498},"max":{"1min":0.85799999999999998,"5min":1.1639999999999999,"15min":1.329},"last":0.67400000000000004},{"interface":"front","average":{"1min":0.74299999999999999,"5min":0.78400000000000003,"15min":0.81899999999999995},"min":{"1min":0.59899999999999998,"5min":0.53000000000000003,"15min":0.53000000000000003},"max":{"1min":0.88,"5min":1.0609999999999999,"15min":1.095},"last":0.64000000000000001}]},{"osd":6,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.66500000000000004,"5min":0.76800000000000002,"15min":0.80500000000000005},"min":{"1min":0.41299999999999998,"5min":0.41299999999999998,"15min":0.41299999999999998},"max":{"1min":0.83199999999999996,"5min":1.157,"15min":1.919},"last":0.66200000000000003},{"interface":"front","average":{"1min":0.74399999999999999,"5min":0.79600000000000004,"15min":0.81999999999999995},"min":{"1min":0.56599999999999995,"5min":0.55100000000000005,"15min":0.55100000000000005},"max":{"1min":0.96399999999999997,"5min":1.117,"15min":1.2450000000000001},"last":0.754}]},{"osd":7,"last update":"Wed Dec 20 06:04:27 2023","interfaces":[{"interface":"back","average":{"1min":0.77800000000000002,"5min":0.83299999999999996,"15min":0.84899999999999998},"min":{"1min":0.54800000000000004,"5min":0.54600000000000004,"15min":0.54600000000000004},"max":{"1min":1.02,"5min":1.3360000000000001,"15min":1.7869999999999999},"last":0.72299999999999998},{"interface":"front","average":{"1min":0.75600000000000001,"5min":0.84599999999999997,"15min":0.876},"min":{"1min":0.54700000000000004,"5min":0.54700000000000004,"15min":0.54700000000000004},"max":{"1min":1.0009999999999999,"5min":1.262,"15min":1.292},"last":0.71099999999999997}]}]},{"osd":1,"up_from":13,"seq":55834576001,"num_pgs":0,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":0,"kb":94371840,"kb_used":60448,"kb_used_data":3160,"kb_used_omap":0,"kb_used_meta":57280,"kb_avail":94311392,"statfs":{"total":96636764160,"available":96574865408,"internally_reserved":0,"allocated":3235840,"data_stored":1588973,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":58654720},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.41199999999999998,"5min":0.44500000000000001,"15min":0.45800000000000002},"min":{"1min":0.23899999999999999,"5min":0.22700000000000001,"15min":0.22700000000000001},"max":{"1min":0.72599999999999998,"5min":0.89900000000000002,"15min":0.89900000000000002},"last":0.46400000000000002},{"interface":"front","average":{"1min":0.45700000000000002,"5min":0.49299999999999999,"15min":0.51200000000000001},"min":{"1min":0.32100000000000001,"5min":0.28399999999999997,"15min":0.28399999999999997},"max":{"1min":0.70899999999999996,"5min":1.04,"15min":1.04},"last":0.496}]},{"osd":2,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.48899999999999999,"5min":0.503,"15min":0.52500000000000002},"min":{"1min":0.32800000000000001,"5min":0.31,"15min":0.31},"max":{"1min":0.78600000000000003,"5min":0.82599999999999996,"15min":0.82599999999999996},"last":0.55600000000000005},{"interface":"front","average":{"1min":0.54100000000000004,"5min":0.55600000000000005,"15min":0.57199999999999995},"min":{"1min":0.35899999999999999,"5min":0.35899999999999999,"15min":0.35899999999999999},"max":{"1min":0.76000000000000001,"5min":1.081,"15min":1.081},"last":0.56899999999999995}]},{"osd":3,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.57899999999999996,"5min":0.58699999999999997,"15min":0.60999999999999999},"min":{"1min":0.40100000000000002,"5min":0.38300000000000001,"15min":0.38300000000000001},"max":{"1min":1.014,"5min":1.014,"15min":1.014},"last":0.63400000000000001},{"interface":"front","average":{"1min":0.499,"5min":0.51800000000000002,"15min":0.54500000000000004},"min":{"1min":0.30399999999999999,"5min":0.30399999999999999,"15min":0.30399999999999999},"max":{"1min":0.67700000000000005,"5min":0.78300000000000003,"15min":0.93500000000000005},"last":0.52300000000000002}]},{"osd":4,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.71799999999999997,"5min":0.749,"15min":0.77600000000000002},"min":{"1min":0.41599999999999998,"5min":0.41599999999999998,"15min":0.41599999999999998},"max":{"1min":0.95099999999999996,"5min":1.107,"15min":1.321},"last":1.0109999999999999},{"interface":"front","average":{"1min":0.72099999999999997,"5min":0.77100000000000002,"15min":0.80200000000000005},"min":{"1min":0.51500000000000001,"5min":0.51500000000000001,"15min":0.51500000000000001},"max":{"1min":0.91600000000000004,"5min":1.3680000000000001,"15min":1.3680000000000001},"last":0.86199999999999999}]},{"osd":5,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.69199999999999995,"5min":0.748,"15min":0.78700000000000003},"min":{"1min":0.43099999999999999,"5min":0.43099999999999999,"15min":0.43099999999999999},"max":{"1min":0.93200000000000005,"5min":1.157,"15min":1.2849999999999999},"last":0.89500000000000002},{"interface":"front","average":{"1min":0.73099999999999998,"5min":0.76100000000000001,"15min":0.79500000000000004},"min":{"1min":0.46999999999999997,"5min":0.41499999999999998,"15min":0.41499999999999998},"max":{"1min":0.98499999999999999,"5min":1.448,"15min":1.448},"last":0.875}]},{"osd":6,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.75,"5min":0.78800000000000003,"15min":0.81899999999999995},"min":{"1min":0.59399999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.97199999999999998,"5min":2.488,"15min":2.488},"last":0.99199999999999999},{"interface":"front","average":{"1min":0.69099999999999995,"5min":0.77300000000000002,"15min":0.80100000000000005},"min":{"1min":0.48499999999999999,"5min":0.48499999999999999,"15min":0.48499999999999999},"max":{"1min":0.90000000000000002,"5min":1.7250000000000001,"15min":1.7250000000000001},"last":0.94199999999999995}]},{"osd":7,"last update":"Wed Dec 20 06:04:42 2023","interfaces":[{"interface":"back","average":{"1min":0.73899999999999999,"5min":0.80100000000000005,"15min":0.84599999999999997},"min":{"1min":0.53400000000000003,"5min":0.53400000000000003,"15min":0.50900000000000001},"max":{"1min":0.98599999999999999,"5min":1.6799999999999999,"15min":1.6799999999999999},"last":0.97599999999999998},{"interface":"front","average":{"1min":0.80400000000000005,"5min":0.81000000000000005,"15min":0.85199999999999998},"min":{"1min":0.54100000000000004,"5min":0.50800000000000001,"15min":0.50800000000000001},"max":{"1min":1.101,"5min":1.208,"15min":1.331},"last":1.0700000000000001}]}]},{"osd":0,"up_from":13,"seq":55834576000,"num_pgs":1,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":94371840,"kb_used":56932,"kb_used_data":3740,"kb_used_omap":0,"kb_used_meta":53184,"kb_avail":94314908,"statfs":{"total":96636764160,"available":96578465792,"internally_reserved":0,"allocated":3829760,"data_stored":2179341,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":54460416},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.434,"5min":0.45800000000000002,"15min":0.46400000000000002},"min":{"1min":0.29299999999999998,"5min":0.26600000000000001,"15min":0.26600000000000001},"max":{"1min":0.59099999999999997,"5min":0.998,"15min":0.998},"last":0.44900000000000001},{"interface":"front","average":{"1min":0.434,"5min":0.54300000000000004,"15min":0.55200000000000005},"min":{"1min":0.309,"5min":0.309,"15min":0.309},"max":{"1min":0.63900000000000001,"5min":3.681,"15min":3.681},"last":0.60599999999999998}]},{"osd":2,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.45900000000000002,"5min":0.53100000000000003,"15min":0.53300000000000003},"min":{"1min":0.33300000000000002,"5min":0.33300000000000002,"15min":0.33300000000000002},"max":{"1min":0.60999999999999999,"5min":1.5209999999999999,"15min":1.5209999999999999},"last":0.51000000000000001},{"interface":"front","average":{"1min":0.48999999999999999,"5min":0.58199999999999996,"15min":0.58899999999999997},"min":{"1min":0.35799999999999998,"5min":0.33400000000000002,"15min":0.33400000000000002},"max":{"1min":0.69399999999999995,"5min":3.7290000000000001,"15min":3.7290000000000001},"last":0.72499999999999998}]},{"osd":3,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.49299999999999999,"5min":0.53900000000000003,"15min":0.55800000000000005},"min":{"1min":0.35199999999999998,"5min":0.32300000000000001,"15min":0.31900000000000001},"max":{"1min":0.67300000000000004,"5min":1.6899999999999999,"15min":1.6899999999999999},"last":0.56599999999999995},{"interface":"front","average":{"1min":0.53100000000000003,"5min":0.60499999999999998,"15min":0.622},"min":{"1min":0.42699999999999999,"5min":0.40999999999999998,"15min":0.40999999999999998},"max":{"1min":0.66000000000000003,"5min":2.3660000000000001,"15min":2.3660000000000001},"last":0.74299999999999999}]},{"osd":4,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77600000000000002,"5min":0.81699999999999995,"15min":0.82199999999999995},"min":{"1min":0.56999999999999995,"5min":0.47899999999999998,"15min":0.47899999999999998},"max":{"1min":0.91300000000000003,"5min":3.1480000000000001,"15min":3.1480000000000001},"last":0.68799999999999994},{"interface":"front","average":{"1min":0.73999999999999999,"5min":0.80500000000000005,"15min":0.82799999999999996},"min":{"1min":0.53400000000000003,"5min":0.53000000000000003,"15min":0.51400000000000001},"max":{"1min":1.081,"5min":1.5549999999999999,"15min":1.5549999999999999},"last":0.79800000000000004}]},{"osd":5,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.73499999999999999,"5min":0.84499999999999997,"15min":0.84399999999999997},"min":{"1min":0.53800000000000003,"5min":0.53800000000000003,"15min":0.53800000000000003},"max":{"1min":1.042,"5min":5.6769999999999996,"15min":5.6769999999999996},"last":0.81799999999999995},{"interface":"front","average":{"1min":0.78200000000000003,"5min":0.81599999999999995,"15min":0.84299999999999997},"min":{"1min":0.53700000000000003,"5min":0.495,"15min":0.495},"max":{"1min":1.1220000000000001,"5min":2.23,"15min":2.23},"last":0.82799999999999996}]},{"osd":6,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.75600000000000001,"5min":0.80800000000000005,"15min":0.83599999999999997},"min":{"1min":0.55800000000000005,"5min":0.55500000000000005,"15min":0.55500000000000005},"max":{"1min":1.0529999999999999,"5min":1.482,"15min":1.482},"last":0.66100000000000003},{"interface":"front","average":{"1min":0.76100000000000001,"5min":0.86899999999999999,"15min":0.86299999999999999},"min":{"1min":0.50700000000000001,"5min":0.50700000000000001,"15min":0.50700000000000001},"max":{"1min":1.004,"5min":5.6310000000000002,"15min":5.6310000000000002},"last":0.85899999999999999}]},{"osd":7,"last update":"Wed Dec 20 06:04:13 2023","interfaces":[{"interface":"back","average":{"1min":0.77700000000000002,"5min":0.86399999999999999,"15min":0.871},"min":{"1min":0.58699999999999997,"5min":0.52700000000000002,"15min":0.52700000000000002},"max":{"1min":0.96099999999999997,"5min":5.7160000000000002,"15min":5.7160000000000002},"last":0.89600000000000002},{"interface":"front","average":{"1min":0.77500000000000002,"5min":0.86399999999999999,"15min":0.86599999999999999},"min":{"1min":0.56899999999999995,"5min":0.45700000000000002,"15min":0.45700000000000002},"max":{"1min":0.94399999999999995,"5min":4.2119999999999997,"15min":4.2119999999999997},"last":0.877}]}]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2023-12-20T06:05:10.363 DEBUG:teuthology.orchestra.run.smithi032:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2023-12-20T06:05:10.504 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.500+0000 7f459f96d640 1 Processor -- start 2023-12-20T06:05:10.505 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459f96d640 1 -- start start 2023-12-20T06:05:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459f96d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4598157480 0x7f45980c9840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459f96d640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45980b0760 0x7f45980b0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459f96d640 1 -- --> v1:172.21.15.39:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f45980b1100 con 0x7f459815d660 2023-12-20T06:05:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459f96d640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f45980cde20 con 0x7f45980b0760 2023-12-20T06:05:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459f96d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45980c8e30 con 0x7f4598157480 2023-12-20T06:05:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459cf08640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45980b0760 0x7f45980b0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459d709640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4598157480 0x7f45980c9840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.507 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459cf08640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f45980b0760 0x7f45980b0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41854/0 (socket says 172.21.15.32:41854) 2023-12-20T06:05:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459df0a640 1 --1- >> v1:172.21.15.39:6789/0 conn(0x7f459815d660 0x7f459815daf0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.39:6789/0 says I am v1:172.21.15.32:37576/0 (socket says 172.21.15.32:37576) 2023-12-20T06:05:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459cf08640 1 -- 172.21.15.32:0/240080723 learned_addr learned my addr 172.21.15.32:0/240080723 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:05:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459d709640 1 --2- 172.21.15.32:0/240080723 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4598157480 0x7f45980c9840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.39:3301/0 says I am v2:172.21.15.32:48168/0 (socket says 172.21.15.32:48168) 2023-12-20T06:05:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459cf08640 1 -- 172.21.15.32:0/240080723 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4598157480 msgr2=0x7f45980c9840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:05:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459cf08640 1 --2- 172.21.15.32:0/240080723 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4598157480 0x7f45980c9840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459cf08640 1 -- 172.21.15.32:0/240080723 >> v1:172.21.15.39:6789/0 conn(0x7f459815d660 legacy=0x7f459815daf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f458ffff640 1 -- 172.21.15.32:0/240080723 <== mon.1 v1:172.21.15.39:6789/0 1 ==== auth_reply(proto 2 0 (0) Success) v1 ==== 33+0+0 (unknown 1366675250 0 0) 0x7f45980b1100 con 0x7f459815d660 2023-12-20T06:05:10.508 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.504+0000 7f459cf08640 1 -- 172.21.15.32:0/240080723 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45980ce070 con 0x7f45980b0760 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f459cf08640 1 --2- 172.21.15.32:0/240080723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45980b0760 0x7f45980b0bc0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f4588009870 tx=0x7f458801c480 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd75cd52ac21f496 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f458ffff640 1 -- 172.21.15.32:0/240080723 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45880ab680 con 0x7f45980b0760 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f458ffff640 1 -- 172.21.15.32:0/240080723 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f45880ab7e0 con 0x7f45980b0760 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f458ffff640 1 -- 172.21.15.32:0/240080723 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f45880abb70 con 0x7f45980b0760 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f459f96d640 1 -- 172.21.15.32:0/240080723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45980b0760 msgr2=0x7f45980b0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f459f96d640 1 --2- 172.21.15.32:0/240080723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45980b0760 0x7f45980b0bc0 secure :-1 s=READY pgs=2651 cs=0 l=1 rev1=1 crypto rx=0x7f4588009870 tx=0x7f458801c480 comp rx=0 tx=0).stop 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f459f96d640 1 -- 172.21.15.32:0/240080723 shutdown_connections 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f459f96d640 1 --2- 172.21.15.32:0/240080723 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f45980b0760 0x7f45980b0bc0 unknown :-1 s=CLOSED pgs=2651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f459f96d640 1 --2- 172.21.15.32:0/240080723 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4598157480 0x7f45980c9840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f459f96d640 1 -- 172.21.15.32:0/240080723 >> 172.21.15.32:0/240080723 conn(0x7f459806a510 msgr2=0x7f45980ce7e0 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f459f96d640 1 -- 172.21.15.32:0/240080723 shutdown_connections 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.508+0000 7f459f96d640 1 -- 172.21.15.32:0/240080723 wait complete. 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459f96d640 1 Processor -- start 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459f96d640 1 -- start start 2023-12-20T06:05:10.515 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459f96d640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45980b0760 0x7f45981f9630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459f96d640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4598157480 0x7f459806d510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459f96d640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459815d660 0x7f4598141e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459f96d640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45981f9210 con 0x7f459815d660 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459f96d640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f45981f8c20 con 0x7f45980b0760 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459f96d640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f45981f9b70 con 0x7f4598157480 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459df0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459815d660 0x7f4598141e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459df0a640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459815d660 0x7f4598141e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41858/0 (socket says 172.21.15.32:41858) 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459df0a640 1 -- 172.21.15.32:0/2646477979 learned_addr learned my addr 172.21.15.32:0/2646477979 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459cf08640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4598157480 0x7f459806d510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459d709640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45980b0760 0x7f45981f9630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459df0a640 1 -- 172.21.15.32:0/2646477979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4598157480 msgr2=0x7f459806d510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459df0a640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4598157480 0x7f459806d510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459df0a640 1 -- 172.21.15.32:0/2646477979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45980b0760 msgr2=0x7f45981f9630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459df0a640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45980b0760 0x7f45981f9630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.516 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459df0a640 1 -- 172.21.15.32:0/2646477979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f45880094d0 con 0x7f459815d660 2023-12-20T06:05:10.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459df0a640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459815d660 0x7f4598141e30 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f459000c810 tx=0x7f459000ccd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:05:10.517 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f458dffb640 1 -- 172.21.15.32:0/2646477979 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459000a640 con 0x7f459815d660 2023-12-20T06:05:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f458dffb640 1 -- 172.21.15.32:0/2646477979 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(0 keys) v1 ==== 4+0+0 (secure 0 0 0) 0x7f459000a7c0 con 0x7f459815d660 2023-12-20T06:05:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459f96d640 1 -- 172.21.15.32:0/2646477979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f4598142940 con 0x7f459815d660 2023-12-20T06:05:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f458dffb640 1 -- 172.21.15.32:0/2646477979 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f459000ab70 con 0x7f459815d660 2023-12-20T06:05:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459f96d640 1 -- 172.21.15.32:0/2646477979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f45981f8e30 con 0x7f459815d660 2023-12-20T06:05:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.512+0000 7f459f96d640 1 -- 172.21.15.32:0/2646477979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f45981f9350 con 0x7f459815d660 2023-12-20T06:05:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.516+0000 7f458dffb640 1 -- 172.21.15.32:0/2646477979 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f459000acd0 con 0x7f459815d660 2023-12-20T06:05:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.516+0000 7f458dffb640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f457c065b80 0x7f457c068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.518 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.516+0000 7f458dffb640 1 -- 172.21.15.32:0/2646477979 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f45900829d0 con 0x7f459815d660 2023-12-20T06:05:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.516+0000 7f459d709640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f457c065b80 0x7f457c068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.519 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.516+0000 7f459d709640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f457c065b80 0x7f457c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4594009ac0 tx=0x7f4594009190 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:05:10.522 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.520+0000 7f458dffb640 1 -- 172.21.15.32:0/2646477979 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f45981f9350 con 0x7f459815d660 2023-12-20T06:05:10.684 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.680+0000 7f459f96d640 1 -- 172.21.15.32:0/2646477979 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command([{prefix=config set, name=mon_health_to_clog}] v 0) v1 -- 0x7f45981f9350 con 0x7f459815d660 2023-12-20T06:05:10.689 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.688+0000 7f458dffb640 1 -- 172.21.15.32:0/2646477979 <== mon.0 v2:172.21.15.32:3300/0 7 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f45900018e0 con 0x7f459815d660 2023-12-20T06:05:10.690 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.688+0000 7f458dffb640 1 -- 172.21.15.32:0/2646477979 <== mon.0 v2:172.21.15.32:3300/0 8 ==== mon_command_ack([{prefix=config set, name=mon_health_to_clog}]=0 v88)=0 v88) v1 ==== 123+0+0 (secure 0 0 0) 0x7f45981f9350 con 0x7f459815d660 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.688+0000 7f459f96d640 1 -- 172.21.15.32:0/2646477979 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f457c065b80 msgr2=0x7f457c068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.688+0000 7f459f96d640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f457c065b80 0x7f457c068040 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4594009ac0 tx=0x7f4594009190 comp rx=0 tx=0).stop 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.688+0000 7f459f96d640 1 -- 172.21.15.32:0/2646477979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459815d660 msgr2=0x7f4598141e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.688+0000 7f459f96d640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459815d660 0x7f4598141e30 secure :-1 s=READY pgs=2652 cs=0 l=1 rev1=1 crypto rx=0x7f459000c810 tx=0x7f459000ccd0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.688+0000 7f459f96d640 1 -- 172.21.15.32:0/2646477979 shutdown_connections 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.688+0000 7f459f96d640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f457c065b80 0x7f457c068040 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.688+0000 7f459f96d640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f459815d660 0x7f4598141e30 unknown :-1 s=CLOSED pgs=2652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.688+0000 7f459f96d640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f4598157480 0x7f459806d510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.692+0000 7f459f96d640 1 --2- 172.21.15.32:0/2646477979 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f45980b0760 0x7f45981f9630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.692+0000 7f459f96d640 1 -- 172.21.15.32:0/2646477979 >> 172.21.15.32:0/2646477979 conn(0x7f459806a510 msgr2=0x7f45980ce770 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.692+0000 7f459f96d640 1 -- 172.21.15.32:0/2646477979 shutdown_connections 2023-12-20T06:05:10.693 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.692+0000 7f459f96d640 1 -- 172.21.15.32:0/2646477979 wait complete. 2023-12-20T06:05:10.712 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2023-12-20T06:05:10.712 DEBUG:teuthology.orchestra.run.smithi032:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2023-12-20T06:05:10.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.852+0000 7f7cfe8b6640 1 Processor -- start 2023-12-20T06:05:10.856 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.852+0000 7f7cfe8b6640 1 -- start start 2023-12-20T06:05:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cfe8b6640 1 --2- >> v2:172.21.15.39:3300/0 conn(0x7f7cf806ce60 0x7f7cf815b280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.857 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cfe8b6640 1 --2- >> v2:172.21.15.32:3300/0 conn(0x7f7cf815f110 0x7f7cf815c440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cfe8b6640 1 -- --> v1:172.21.15.32:6789/0 -- auth(proto 0 30 bytes epoch 0) v1 -- 0x7f7cf815bbd0 con 0x7f7cf80af5e0 2023-12-20T06:05:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cfe8b6640 1 -- --> v2:172.21.15.32:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf815beb0 con 0x7f7cf815f110 2023-12-20T06:05:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cfe8b6640 1 -- --> v2:172.21.15.39:3300/0 -- mon_getmap magic: 0 v1 -- 0x7f7cf815e910 con 0x7f7cf806ce60 2023-12-20T06:05:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cf7fff640 1 --1- >> v1:172.21.15.32:6789/0 conn(0x7f7cf80af5e0 0x7f7cf80af9c0 :-1 s=CONNECTING_WAIT_BANNER_AND_IDENTIFY pgs=0 cs=0 l=0).handle_server_banner_and_identify peer v1:172.21.15.32:6789/0 says I am v1:172.21.15.32:52972/0 (socket says 172.21.15.32:52972) 2023-12-20T06:05:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cf7fff640 1 -- 172.21.15.32:0/2926731742 learned_addr learned my addr 172.21.15.32:0/2926731742 (peer_addr_for_me v1:172.21.15.32:0/0) 2023-12-20T06:05:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cfce53640 1 --2- 172.21.15.32:0/2926731742 >> v2:172.21.15.32:3300/0 conn(0x7f7cf815f110 0x7f7cf815c440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cf77fe640 1 --2- 172.21.15.32:0/2926731742 >> v2:172.21.15.39:3300/0 conn(0x7f7cf806ce60 0x7f7cf815b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cfce53640 1 -- 172.21.15.32:0/2926731742 >> v2:172.21.15.39:3300/0 conn(0x7f7cf806ce60 msgr2=0x7f7cf815b280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2023-12-20T06:05:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cfce53640 1 --2- 172.21.15.32:0/2926731742 >> v2:172.21.15.39:3300/0 conn(0x7f7cf806ce60 0x7f7cf815b280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.858 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cfce53640 1 -- 172.21.15.32:0/2926731742 >> v1:172.21.15.32:6789/0 conn(0x7f7cf80af5e0 legacy=0x7f7cf80af9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.859 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.856+0000 7f7cfce53640 1 -- 172.21.15.32:0/2926731742 --> v2:172.21.15.32:3300/0 -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf815ebf0 con 0x7f7cf815f110 2023-12-20T06:05:10.860 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfce53640 1 --2- 172.21.15.32:0/2926731742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf815f110 0x7f7cf815c440 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f7cf806abe0 tx=0x7f7cf001c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a4ef758a391ce6 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:05:10.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cf6ffd640 1 -- 172.21.15.32:0/2926731742 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf815b9d0 con 0x7f7cf815f110 2023-12-20T06:05:10.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cf6ffd640 1 -- 172.21.15.32:0/2926731742 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7cf00a58e0 con 0x7f7cf815f110 2023-12-20T06:05:10.861 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cf6ffd640 1 -- 172.21.15.32:0/2926731742 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7cf00a5c90 con 0x7f7cf815f110 2023-12-20T06:05:10.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/2926731742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf815f110 msgr2=0x7f7cf815c440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 --2- 172.21.15.32:0/2926731742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf815f110 0x7f7cf815c440 secure :-1 s=READY pgs=2654 cs=0 l=1 rev1=1 crypto rx=0x7f7cf806abe0 tx=0x7f7cf001c880 comp rx=0 tx=0).stop 2023-12-20T06:05:10.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/2926731742 shutdown_connections 2023-12-20T06:05:10.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 --2- 172.21.15.32:0/2926731742 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf815f110 0x7f7cf815c440 unknown :-1 s=CLOSED pgs=2654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 --2- 172.21.15.32:0/2926731742 >> v2:172.21.15.39:3300/0 conn(0x7f7cf806ce60 0x7f7cf815b280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/2926731742 >> 172.21.15.32:0/2926731742 conn(0x7f7cf806a320 msgr2=0x7f7cf80ce760 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:05:10.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/2926731742 shutdown_connections 2023-12-20T06:05:10.862 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/2926731742 wait complete. 2023-12-20T06:05:10.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 Processor -- start 2023-12-20T06:05:10.863 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 -- start start 2023-12-20T06:05:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf806ce60 0x7f7cf80cbcf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 --2- >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7cf80af5e0 0x7f7cf8141770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 --2- >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf815f110 0x7f7cf80b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 -- --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf81428d0 con 0x7f7cf806ce60 2023-12-20T06:05:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 -- --> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf8142d60 con 0x7f7cf815f110 2023-12-20T06:05:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.860+0000 7f7cfe8b6640 1 -- --> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] -- mon_getmap magic: 0 v1 -- 0x7f7cf8141fa0 con 0x7f7cf80af5e0 2023-12-20T06:05:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf806ce60 0x7f7cf80cbcf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf7fff640 1 --2- >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf806ce60 0x7f7cf80cbcf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.32:3300/0 says I am v2:172.21.15.32:41864/0 (socket says 172.21.15.32:41864) 2023-12-20T06:05:10.866 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf7fff640 1 -- 172.21.15.32:0/3490788139 learned_addr learned my addr 172.21.15.32:0/3490788139 (peer_addr_for_me v2:172.21.15.32:0/0) 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf77fe640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7cf80af5e0 0x7f7cf8141770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cfce53640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf815f110 0x7f7cf80b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf7fff640 1 -- 172.21.15.32:0/3490788139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7cf80af5e0 msgr2=0x7f7cf8141770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf7fff640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7cf80af5e0 0x7f7cf8141770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf7fff640 1 -- 172.21.15.32:0/3490788139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf815f110 msgr2=0x7f7cf80b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf7fff640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf815f110 0x7f7cf80b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf7fff640 1 -- 172.21.15.32:0/3490788139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({config=0+,monmap=0+}) v3 -- 0x7f7cf0009a70 con 0x7f7cf806ce60 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cfce53640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf815f110 0x7f7cf80b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf7fff640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf806ce60 0x7f7cf80cbcf0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800c500 tx=0x7f7ce800c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf4ff9640 1 -- 172.21.15.32:0/3490788139 <== mon.0 v2:172.21.15.32:3300/0 1 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce8007500 con 0x7f7cf806ce60 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf4ff9640 1 -- 172.21.15.32:0/3490788139 <== mon.0 v2:172.21.15.32:3300/0 2 ==== config(1 keys) v1 ==== 35+0+0 (secure 0 0 0) 0x7f7ce8007660 con 0x7f7cf806ce60 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/3490788139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({mgrmap=0+}) v3 -- 0x7f7cf8142280 con 0x7f7cf806ce60 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf4ff9640 1 -- 172.21.15.32:0/3490788139 <== mon.0 v2:172.21.15.32:3300/0 3 ==== mon_map magic: 0 v1 ==== 413+0+0 (secure 0 0 0) 0x7f7ce80059b0 con 0x7f7cf806ce60 2023-12-20T06:05:10.867 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/3490788139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_subscribe({osdmap=0}) v3 -- 0x7f7cf8141cb0 con 0x7f7cf806ce60 2023-12-20T06:05:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/3490788139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) v1 -- 0x7f7cf804e000 con 0x7f7cf806ce60 2023-12-20T06:05:10.868 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf4ff9640 1 -- 172.21.15.32:0/3490788139 <== mon.0 v2:172.21.15.32:3300/0 4 ==== mgrmap(e 152) v1 ==== 85330+0+0 (secure 0 0 0) 0x7f7ce8005b10 con 0x7f7cf806ce60 2023-12-20T06:05:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.864+0000 7f7cf4ff9640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7ce4065b80 0x7f7ce4068040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2023-12-20T06:05:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.868+0000 7f7cf4ff9640 1 -- 172.21.15.32:0/3490788139 <== mon.0 v2:172.21.15.32:3300/0 5 ==== osd_map(256..256 src has 1..256) v4 ==== 5687+0+0 (secure 0 0 0) 0x7f7ce80827e0 con 0x7f7cf806ce60 2023-12-20T06:05:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.868+0000 7f7cf77fe640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7ce4065b80 0x7f7ce4068040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2023-12-20T06:05:10.869 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.868+0000 7f7cf77fe640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7ce4065b80 0x7f7ce4068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009b60 tx=0x7f7cec009190 comp rx=0 tx=0).ready entity=mgr.12348 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2023-12-20T06:05:10.872 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:10.868+0000 7f7cf4ff9640 1 -- 172.21.15.32:0/3490788139 <== mon.0 v2:172.21.15.32:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) v1 ==== 72+0+172167 (secure 0 0 0) 0x7f7ce8050140 con 0x7f7cf806ce60 2023-12-20T06:05:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.060+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/3490788139 --> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) v1 -- 0x7f7cf80cf450 con 0x7f7cf806ce60 2023-12-20T06:05:11.063 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.060+0000 7f7cf4ff9640 1 -- 172.21.15.32:0/3490788139 <== mon.0 v2:172.21.15.32:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 130 v130) v1 ==== 95+0+3535 (secure 0 0 0) 0x7f7ce8017020 con 0x7f7cf806ce60 2023-12-20T06:05:11.064 INFO:teuthology.orchestra.run.smithi032.stdout: 2023-12-20T06:05:11.064 INFO:teuthology.orchestra.run.smithi032.stdout:{"epoch":130,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":11632,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6834/3779641739","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6832","nonce":3779641739},{"type":"v1","addr":"172.21.15.39:6834","nonce":3779641739}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":11635,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.39:6835/473427978","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.39:6833","nonce":473427978},{"type":"v1","addr":"172.21.15.39:6835","nonce":473427978}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12060,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6836/3927543497","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6834","nonce":3927543497},{"type":"v1","addr":"172.21.15.32:6836","nonce":3927543497}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":123},{"gid":12498,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.32:6837/3273893989","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.32:6835","nonce":3273893989},{"type":"v1","addr":"172.21.15.32:6837","nonce":3273893989}]},"join_fscid":-1,"export_targets":[],"features":4540138322906710015,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2"}},"epoch":130}],"filesystems":[]} 2023-12-20T06:05:11.064 INFO:teuthology.orchestra.run.smithi032.stderr:dumped fsmap epoch 130 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/3490788139 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7ce4065b80 msgr2=0x7f7ce4068040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7ce4065b80 0x7f7ce4068040 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7f7cec009b60 tx=0x7f7cec009190 comp rx=0 tx=0).stop 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/3490788139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf806ce60 msgr2=0x7f7cf80cbcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf806ce60 0x7f7cf80cbcf0 secure :-1 s=READY pgs=2655 cs=0 l=1 rev1=1 crypto rx=0x7f7ce800c500 tx=0x7f7ce800c9c0 comp rx=0 tx=0).stop 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/3490788139 shutdown_connections 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.32:6832/64546,v1:172.21.15.32:6833/64546] conn(0x7f7ce4065b80 0x7f7ce4068040 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.39:3300/0,v1:172.21.15.39:6789/0] conn(0x7f7cf815f110 0x7f7cf80b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.39:3301/0,v1:172.21.15.39:6790/0] conn(0x7f7cf80af5e0 0x7f7cf8141770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 --2- 172.21.15.32:0/3490788139 >> [v2:172.21.15.32:3300/0,v1:172.21.15.32:6789/0] conn(0x7f7cf806ce60 0x7f7cf80cbcf0 unknown :-1 s=CLOSED pgs=2655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/3490788139 >> 172.21.15.32:0/3490788139 conn(0x7f7cf806a320 msgr2=0x7f7cf8158e80 unknown :-1 s=STATE_NONE l=0).mark_down 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/3490788139 shutdown_connections 2023-12-20T06:05:11.066 INFO:teuthology.orchestra.run.smithi032.stderr:2023-12-20T06:05:11.064+0000 7f7cfe8b6640 1 -- 172.21.15.32:0/3490788139 wait complete. 2023-12-20T06:05:11.085 DEBUG:tasks.cephfs.filesystem:already dead... 2023-12-20T06:05:11.085 INFO:teuthology.misc:Shutting down mds daemons... 2023-12-20T06:05:11.085 DEBUG:tasks.ceph.mds.a:waiting for process to exit 2023-12-20T06:05:11.085 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:11.097 INFO:tasks.ceph.mds.a:Stopped 2023-12-20T06:05:11.097 DEBUG:tasks.ceph.mds.c:waiting for process to exit 2023-12-20T06:05:11.098 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:11.157 INFO:tasks.ceph.mds.c:Stopped 2023-12-20T06:05:11.157 DEBUG:tasks.ceph.mds.b:waiting for process to exit 2023-12-20T06:05:11.157 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:11.172 INFO:tasks.ceph.mds.b:Stopped 2023-12-20T06:05:11.172 DEBUG:tasks.ceph.mds.d:waiting for process to exit 2023-12-20T06:05:11.172 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:11.232 INFO:tasks.ceph.mds.d:Stopped 2023-12-20T06:05:11.232 INFO:teuthology.misc:Shutting down osd daemons... 2023-12-20T06:05:11.233 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2023-12-20T06:05:11.233 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:11.325 INFO:tasks.ceph.osd.0:Stopped 2023-12-20T06:05:11.325 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2023-12-20T06:05:11.325 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:12.955 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:05:12.959 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:05:16.913 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2023-12-20T06:05:17.426 INFO:tasks.ceph.osd.1:Stopped 2023-12-20T06:05:17.427 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2023-12-20T06:05:17.427 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:17.485 INFO:tasks.ceph.osd.2:Stopped 2023-12-20T06:05:17.485 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2023-12-20T06:05:17.485 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:17.579 INFO:tasks.ceph.osd.3:Stopped 2023-12-20T06:05:17.579 DEBUG:tasks.ceph.osd.4:waiting for process to exit 2023-12-20T06:05:17.579 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:17.666 INFO:tasks.ceph.osd.4:Stopped 2023-12-20T06:05:17.667 DEBUG:tasks.ceph.osd.5:waiting for process to exit 2023-12-20T06:05:17.667 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:22.617 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 is failed for ~0s 2023-12-20T06:05:22.617 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2023-12-20T06:05:23.768 INFO:tasks.ceph.osd.5:Stopped 2023-12-20T06:05:23.768 DEBUG:tasks.ceph.osd.6:waiting for process to exit 2023-12-20T06:05:23.768 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:23.861 INFO:tasks.ceph.osd.6:Stopped 2023-12-20T06:05:23.861 DEBUG:tasks.ceph.osd.7:waiting for process to exit 2023-12-20T06:05:23.861 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:23.952 INFO:tasks.ceph.osd.7:Stopped 2023-12-20T06:05:23.952 INFO:teuthology.misc:Shutting down mgr daemons... 2023-12-20T06:05:23.952 DEBUG:tasks.ceph.mgr.y:waiting for process to exit 2023-12-20T06:05:23.952 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:24.017 INFO:tasks.ceph.mgr.y:Stopped 2023-12-20T06:05:24.017 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2023-12-20T06:05:24.017 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:24.074 INFO:tasks.ceph.mgr.x:Stopped 2023-12-20T06:05:24.074 INFO:teuthology.misc:Shutting down mon daemons... 2023-12-20T06:05:24.074 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2023-12-20T06:05:24.074 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:24.105 INFO:tasks.ceph.mon.a:Stopped 2023-12-20T06:05:24.105 DEBUG:tasks.ceph.mon.b:waiting for process to exit 2023-12-20T06:05:24.105 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:24.130 INFO:tasks.ceph.mon.b:Stopped 2023-12-20T06:05:24.130 DEBUG:tasks.ceph.mon.c:waiting for process to exit 2023-12-20T06:05:24.130 INFO:teuthology.orchestra.run:waiting for 300 2023-12-20T06:05:24.202 INFO:tasks.ceph.mon.c:Stopped 2023-12-20T06:05:24.202 INFO:tasks.ceph:Checking cluster log for badness... 2023-12-20T06:05:24.202 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-20T06:05:24.232 INFO:teuthology.orchestra.run.smithi032.stdout:2023-12-20T04:42:14.088181+0000 mon.a (mon.0) 640 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-12-20T06:05:24.233 WARNING:tasks.ceph:Found errors (ERR|WRN|SEC) in cluster log 2023-12-20T06:05:24.233 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[SEC\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-20T06:05:24.311 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[ERR\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-20T06:05:24.386 DEBUG:teuthology.orchestra.run.smithi032:> sudo egrep '\[WRN\]' /var/log/ceph/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v 'overall HEALTH_' | egrep -v '\(FS_DEGRADED\)' | egrep -v '\(MDS_FAILED\)' | egrep -v '\(MDS_DEGRADED\)' | egrep -v '\(FS_WITH_FAILED_MDS\)' | egrep -v '\(MDS_DAMAGE\)' | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v '\(FS_INLINE_DATA_DEPRECATED\)' | egrep -v '\(POOL_APP_NOT_ENABLED\)' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | head -n 1 2023-12-20T06:05:24.461 INFO:teuthology.orchestra.run.smithi032.stdout:2023-12-20T04:42:14.088181+0000 mon.a (mon.0) 640 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY) 2023-12-20T06:05:24.462 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T06:05:24.462 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2023-12-20T06:05:24.754 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T06:05:24.754 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2023-12-20T06:05:24.924 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T06:05:24.924 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2023-12-20T06:05:25.091 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T06:05:25.092 DEBUG:teuthology.orchestra.run.smithi032:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2023-12-20T06:05:25.369 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-4 on ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T06:05:25.369 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-4 2023-12-20T06:05:25.531 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-5 on ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T06:05:25.531 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-5 2023-12-20T06:05:25.683 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-6 on ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T06:05:25.683 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-6 2023-12-20T06:05:25.890 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-7 on ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T06:05:25.890 DEBUG:teuthology.orchestra.run.smithi039:> sync && sudo umount -f /var/lib/ceph/osd/ceph-7 2023-12-20T06:05:26.067 INFO:tasks.ceph:Archiving mon data... 2023-12-20T06:05:26.068 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/mon/ceph-a to /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395/data/mon.a.tgz 2023-12-20T06:05:26.068 DEBUG:teuthology.orchestra.run.smithi032:> mktemp 2023-12-20T06:05:26.078 INFO:teuthology.orchestra.run.smithi032.stdout:/tmp/tmp.vcUiPoZgpn 2023-12-20T06:05:26.078 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.vcUiPoZgpn 2023-12-20T06:05:26.709 DEBUG:teuthology.orchestra.run.smithi032:> sudo chmod 0666 /tmp/tmp.vcUiPoZgpn 2023-12-20T06:05:26.786 DEBUG:teuthology.orchestra.remote:smithi032:/tmp/tmp.vcUiPoZgpn is 4MB 2023-12-20T06:05:26.893 DEBUG:teuthology.orchestra.run.smithi032:> rm -fr /tmp/tmp.vcUiPoZgpn 2023-12-20T06:05:26.904 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/mon/ceph-b to /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395/data/mon.b.tgz 2023-12-20T06:05:26.904 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2023-12-20T06:05:26.913 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.h56R8qDV74 2023-12-20T06:05:26.913 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-b -- . > /tmp/tmp.h56R8qDV74 2023-12-20T06:05:27.535 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.h56R8qDV74 2023-12-20T06:05:27.614 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.h56R8qDV74 is 4MB 2023-12-20T06:05:27.618 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.5 has been restored 2023-12-20T06:05:27.734 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.h56R8qDV74 2023-12-20T06:05:27.744 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/mon/ceph-c to /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395/data/mon.c.tgz 2023-12-20T06:05:27.744 DEBUG:teuthology.orchestra.run.smithi039:> mktemp 2023-12-20T06:05:27.797 INFO:teuthology.orchestra.run.smithi039.stdout:/tmp/tmp.DmERCXRpmv 2023-12-20T06:05:27.798 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-c -- . > /tmp/tmp.DmERCXRpmv 2023-12-20T06:05:28.426 DEBUG:teuthology.orchestra.run.smithi039:> sudo chmod 0666 /tmp/tmp.DmERCXRpmv 2023-12-20T06:05:28.502 DEBUG:teuthology.orchestra.remote:smithi039:/tmp/tmp.DmERCXRpmv is 4MB 2023-12-20T06:05:28.606 DEBUG:teuthology.orchestra.run.smithi039:> rm -fr /tmp/tmp.DmERCXRpmv 2023-12-20T06:05:28.616 INFO:tasks.ceph:Cleaning ceph cluster... 2023-12-20T06:05:28.616 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-20T06:05:28.620 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2023-12-20T06:05:28.678 INFO:tasks.ceph:Checking for errors in any valgrind logs... 2023-12-20T06:05:28.678 DEBUG:teuthology.orchestra.run.smithi032:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-20T06:05:28.684 DEBUG:teuthology.orchestra.run.smithi039:> sudo zgrep '' /var/log/ceph/valgrind/* /dev/null | sort | uniq 2023-12-20T06:05:28.721 INFO:teuthology.orchestra.run.smithi032.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-20T06:05:28.749 INFO:teuthology.orchestra.run.smithi039.stderr:gzip: /var/log/ceph/valgrind/*.gz: No such file or directory 2023-12-20T06:05:28.755 INFO:tasks.ceph:Archiving crash dumps... 2023-12-20T06:05:28.757 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395/remote/smithi032/crash 2023-12-20T06:05:28.758 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-20T06:05:28.799 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/lib/ceph/crash to /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395/remote/smithi039/crash 2023-12-20T06:05:28.800 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2023-12-20T06:05:28.821 INFO:tasks.ceph:Shutting down logrotate 2023-12-20T06:05:28.821 DEBUG:teuthology.orchestra.run.smithi032:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:05:28.848 DEBUG:teuthology.orchestra.run.smithi039:> sudo logrotate /etc/logrotate.d/ceph-test.conf 2023-12-20T06:05:28.920 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-20T06:05:28.942 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm /etc/logrotate.d/ceph-test.conf 2023-12-20T06:05:28.961 INFO:tasks.ceph:Compressing logs... 2023-12-20T06:05:28.962 DEBUG:teuthology.orchestra.run.smithi032:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-20T06:05:28.988 DEBUG:teuthology.orchestra.run.smithi039:> sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-20T06:05:50.911 INFO:tasks.ceph:Archiving logs... 2023-12-20T06:05:50.911 DEBUG:teuthology.misc:Transferring archived files from smithi032:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395/remote/smithi032/log 2023-12-20T06:05:50.913 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-20T06:05:53.534 DEBUG:teuthology.misc:Transferring archived files from smithi039:/var/log/ceph to /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395/remote/smithi039/log 2023-12-20T06:05:53.534 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /var/log/ceph -- . 2023-12-20T06:05:54.913 DEBUG:teuthology.run_tasks:Unwinding manager install 2023-12-20T06:05:54.924 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2023-12-20T06:05:54.924 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-20T06:05:54.928 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2023-12-20T06:05:55.019 DEBUG:teuthology.run_tasks:Unwinding manager clock 2023-12-20T06:05:55.028 INFO:teuthology.task.clock:Checking final clock skew... 2023-12-20T06:05:55.029 DEBUG:teuthology.orchestra.run.smithi032:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-20T06:05:55.033 DEBUG:teuthology.orchestra.run.smithi039:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2023-12-20T06:05:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: remote refid st t when poll reach delay offset jitter 2023-12-20T06:05:55.050 INFO:teuthology.orchestra.run.smithi032.stdout:============================================================================== 2023-12-20T06:05:55.050 INFO:teuthology.orchestra.run.smithi032.stdout:*hv01.front.sepi 44.4.53.2 2 u 138 256 377 0.131 -3.834 0.259 2023-12-20T06:05:55.050 INFO:teuthology.orchestra.run.smithi032.stdout:+hv02.front.sepi 96.245.170.99 3 u 167 256 377 0.105 -3.018 0.176 2023-12-20T06:05:55.050 INFO:teuthology.orchestra.run.smithi032.stdout:+hv03.front.sepi 74.6.168.72 3 u 213 256 377 0.093 -2.291 0.457 2023-12-20T06:05:55.050 INFO:teuthology.orchestra.run.smithi032.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2023-12-20T06:05:55.053 INFO:teuthology.orchestra.run.smithi039.stdout: remote refid st t when poll reach delay offset jitter 2023-12-20T06:05:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:============================================================================== 2023-12-20T06:05:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:*hv01.front.sepi 44.4.53.2 2 u 52 128 377 0.097 -1.073 0.130 2023-12-20T06:05:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:+hv02.front.sepi 96.245.170.99 3 u 95 128 377 0.107 -0.525 0.170 2023-12-20T06:05:55.053 INFO:teuthology.orchestra.run.smithi039.stdout:+hv03.front.sepi 74.6.168.72 3 u 111 128 377 0.096 -0.068 0.069 2023-12-20T06:05:55.053 INFO:teuthology.orchestra.run.smithi039.stdout: hv04.front.sepi .INIT. 16 u - 1024 0 0.000 +0.000 0.000 2023-12-20T06:05:55.054 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2023-12-20T06:05:55.063 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2023-12-20T06:05:55.063 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2023-12-20T06:05:55.074 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2023-12-20T06:05:55.084 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2023-12-20T06:05:55.095 INFO:teuthology.task.internal:Duration was 6617.248938 seconds 2023-12-20T06:05:55.095 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2023-12-20T06:05:55.106 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2023-12-20T06:05:55.106 DEBUG:teuthology.orchestra.run.smithi032:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-20T06:05:55.109 DEBUG:teuthology.orchestra.run.smithi039:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2023-12-20T06:05:55.220 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2023-12-20T06:05:55.220 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi032.front.sepia.ceph.com 2023-12-20T06:05:55.220 DEBUG:teuthology.orchestra.run.smithi032:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-20T06:05:55.243 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi039.front.sepia.ceph.com 2023-12-20T06:05:55.243 DEBUG:teuthology.orchestra.run.smithi039:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2023-12-20T06:05:55.266 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2023-12-20T06:05:55.266 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-20T06:05:55.291 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2023-12-20T06:05:55.350 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2023-12-20T06:05:55.350 DEBUG:teuthology.orchestra.run.smithi032:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-20T06:05:55.376 DEBUG:teuthology.orchestra.run.smithi039:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2023-12-20T06:05:55.471 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2023-12-20T06:05:55.480 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2023-12-20T06:05:55.481 DEBUG:teuthology.orchestra.run.smithi032:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-20T06:05:55.502 DEBUG:teuthology.orchestra.run.smithi039:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2023-12-20T06:05:55.523 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2023-12-20T06:05:55.534 DEBUG:teuthology.orchestra.run.smithi032:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-20T06:05:55.551 DEBUG:teuthology.orchestra.run.smithi039:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2023-12-20T06:05:55.567 INFO:teuthology.orchestra.run.smithi032.stdout:kernel.core_pattern = core 2023-12-20T06:05:55.584 INFO:teuthology.orchestra.run.smithi039.stdout:kernel.core_pattern = core 2023-12-20T06:05:55.604 DEBUG:teuthology.orchestra.run.smithi032:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-20T06:05:55.635 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T06:05:55.636 DEBUG:teuthology.orchestra.run.smithi039:> test -e /home/ubuntu/cephtest/archive/coredump 2023-12-20T06:05:55.657 DEBUG:teuthology.orchestra.run:got remote process result: 1 2023-12-20T06:05:55.657 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2023-12-20T06:05:55.667 INFO:teuthology.task.internal:Transferring archived files... 2023-12-20T06:05:55.667 DEBUG:teuthology.misc:Transferring archived files from smithi032:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395/remote/smithi032 2023-12-20T06:05:55.667 DEBUG:teuthology.orchestra.run.smithi032:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-20T06:05:55.703 DEBUG:teuthology.misc:Transferring archived files from smithi039:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395/remote/smithi039 2023-12-20T06:05:55.703 DEBUG:teuthology.orchestra.run.smithi039:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2023-12-20T06:05:55.727 INFO:teuthology.task.internal:Removing archive directory... 2023-12-20T06:05:55.727 DEBUG:teuthology.orchestra.run.smithi032:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-20T06:05:55.748 DEBUG:teuthology.orchestra.run.smithi039:> rm -rf -- /home/ubuntu/cephtest/archive 2023-12-20T06:05:55.782 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2023-12-20T06:05:55.791 INFO:teuthology.task.internal:Not uploading archives. 2023-12-20T06:05:55.792 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2023-12-20T06:05:55.803 INFO:teuthology.task.internal:Tidying up after the test... 2023-12-20T06:05:55.804 DEBUG:teuthology.orchestra.run.smithi032:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-20T06:05:55.807 DEBUG:teuthology.orchestra.run.smithi039:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2023-12-20T06:05:55.812 INFO:teuthology.orchestra.run.smithi032.stdout: 658055 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 20 06:05 /home/ubuntu/cephtest 2023-12-20T06:05:55.838 INFO:teuthology.orchestra.run.smithi039.stdout: 658054 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Dec 20 06:05 /home/ubuntu/cephtest 2023-12-20T06:05:55.840 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2023-12-20T06:05:55.849 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2023-12-20T06:05:55.878 INFO:teuthology.nuke:Checking targets against current locks 2023-12-20T06:05:55.902 DEBUG:teuthology.nuke:shortname: smithi032 2023-12-20T06:05:55.902 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-20T06:05:55.922 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi032.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-20 03:59:44.362898', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBMVliqZJMAR+8i1UIgvLMOW2HwRq2iziIYQuIUOiaOB3RW74bqK8M0Px+7m7U1vi9gSVfien5K+r+Pq5a8Qk7kI='} 2023-12-20T06:05:55.927 DEBUG:teuthology.nuke:shortname: smithi039 2023-12-20T06:05:55.927 INFO:teuthology.task.internal.check_lock:Checking locks... 2023-12-20T06:05:55.946 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi039.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/vshankar-2023-12-20_03:59:06-fs-wip-vshankar-testing1-2023-12-18-1207-reef-2-distro-default-smithi/7497395', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'ubuntu', 'os_version': '22.04', 'arch': 'x86_64', 'locked': True, 'locked_since': '2023-12-20 03:59:44.361480', 'locked_by': 'scheduled_vshankar@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBLNmc+OQbgX+2td9QV+1fnzDuUIn09Fw7zDPWqNzdyN1TBzSxdTCN0yAAQXLDFQO9E0yC2E05BGvJuuIhqwybGY='} 2023-12-20T06:05:55.965 INFO:teuthology.orchestra.console.smithi032:Power off 2023-12-20T06:05:55.965 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-20T06:05:55.988 DEBUG:teuthology.orchestra.console.smithi032:power off output: Chassis Power Control: Down/Off 2023-12-20T06:05:55.988 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T06:05:56.001 INFO:teuthology.orchestra.console.smithi039:Power off 2023-12-20T06:05:56.001 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2023-12-20T06:05:56.010 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-12-20T06:05:56.021 DEBUG:teuthology.orchestra.console.smithi039:power off output: Chassis Power Control: Down/Off 2023-12-20T06:05:56.021 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T06:05:56.041 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2023-12-20T06:06:00.011 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T06:06:00.042 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T06:06:00.120 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-12-20T06:06:00.149 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2023-12-20T06:06:04.123 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T06:06:04.150 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T06:06:04.232 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is on 2023-12-20T06:06:04.257 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is on 2023-12-20T06:06:08.234 DEBUG:teuthology.orchestra.console.smithi032:pexpect command: ipmitool -H smithi032.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T06:06:08.258 DEBUG:teuthology.orchestra.console.smithi039:pexpect command: ipmitool -H smithi039.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2023-12-20T06:06:08.345 DEBUG:teuthology.orchestra.console.smithi032:check power output: Chassis Power is off 2023-12-20T06:06:08.368 DEBUG:teuthology.orchestra.console.smithi039:check power output: Chassis Power is off 2023-12-20T06:06:08.446 INFO:teuthology.orchestra.console.smithi032:Power off completed 2023-12-20T06:06:08.469 INFO:teuthology.orchestra.console.smithi039:Power off completed 2023-12-20T06:06:08.574 INFO:teuthology.run:Summary data: client.0-kernel-sha1: 93572f31db899adbd1540d7954ab247e32731c84 client.1-kernel-sha1: 93572f31db899adbd1540d7954ab247e32731c84 description: fs/functional/{begin/{0-install 1-ceph 2-logrotate} clusters/1a3s-mds-4c-client conf/{client mds mon osd} distro/{ubuntu_latest} mount/kclient/{mount-syntax/{v2} mount overrides/{distro/testing/k-testing ms-die-on-skipped}} objectstore/bluestore-bitmap overrides/{ignorelist_health ignorelist_wrongly_marked_down no_client_pidfile} tasks/snap-schedule} duration: 6617.2489376068115 failure_reason: '"2023-12-20T04:42:14.088181+0000 mon.a (mon.0) 640 : cluster [WRN] Health check failed: Reduced data availability: 2 pgs peering (PG_AVAILABILITY)" in cluster log' flavor: default owner: scheduled_vshankar@teuthology success: false 2023-12-20T06:06:08.576 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2023-12-20T06:06:08.723 INFO:teuthology.run:FAIL